[cut: only the first 33554432 bytes of this file were read]
<?xml version="1.0" encoding="utf-8"?>
<posts>
  <row Id="1" PostTypeId="1" CreationDate="2011-11-29T20:50:01.480" Score="7" ViewCount="416" Body="&lt;p&gt;Can you explain the difference between these two computational methods ? &lt;/p&gt;&#xA;" OwnerUserId="23" LastActivityDate="2022-01-17T21:25:28.513" Title="What is the difference between Coupled Cluster SD and SD(T)?" Tags="|computational-chemistry|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2" PostTypeId="1" CreationDate="2011-11-29T20:53:30.957" Score="14" ViewCount="700" Body="&lt;p&gt;Are there cases where Hartree-Fock is not a good approximation to compute equilibrium geometry when the molecule is in a non bond-breaking condition? &lt;/p&gt;&#xA;" OwnerUserId="23" LastActivityDate="2013-04-13T20:32:39.427" Title="Is Hartree-Fock always a good approximation for molecular geometries and no bond breaking ?" Tags="|computational-chemistry|hartree-fock|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3" PostTypeId="1" AcceptedAnswerId="85" CreationDate="2011-11-29T20:54:36.293" Score="16" ViewCount="11425" Body="&lt;p&gt;What is counterpoise correction exactly ? Can you explain when it is needed and why ? &lt;/p&gt;&#xA;" OwnerUserId="23" LastActivityDate="2018-07-03T16:35:18.100" Title="What is counterpoise correction?" Tags="|computational-chemistry|basis-set|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4" PostTypeId="1" CreationDate="2011-11-29T20:56:22.840" Score="9" ViewCount="1546" Body="&lt;p&gt;How do I pick a basis set for an ab-initio Hartree-Fock evaluation ? In other words, what are the important characteristics of a basis set so that a proper choice can be made ?&lt;/p&gt;&#xA;" OwnerUserId="23" LastActivityDate="2013-10-04T15:33:46.770" Title="How to choose a basis set for ab-initio evaluations ?" Tags="|computational-chemistry|hartree-fock|basis-set|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5" PostTypeId="1" CreationDate="2011-11-29T20:58:09.953" Score="10" ViewCount="1976" Body="&lt;p&gt;Some basis sets are said to be &quot;correlation consistent&quot;. What does it mean in practice ? &lt;/p&gt;&#xA;" OwnerUserId="23" LastActivityDate="2020-04-12T14:10:40.057" Title="What does it mean for a basis set to be &quot;correlation consistent&quot; ?" Tags="|computational-chemistry|basis-set|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6" PostTypeId="1" CreationDate="2011-11-29T21:00:04.240" Score="6" ViewCount="6250" Body="&lt;p&gt;Given two computational programs, one using a Gaussian basis, and the other using Slater basis, what are the practical differences, advantages and disadvantages for each choice ? &lt;/p&gt;&#xA;" OwnerUserId="23" LastActivityDate="2011-12-07T21:19:50.543" Title="Differences between Gaussian and Slater functions on the quality of the results?" Tags="|computational-chemistry|basis-set|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7" PostTypeId="1" AcceptedAnswerId="13" CreationDate="2011-11-29T21:04:10.743" Score="17" ViewCount="353" Body="&lt;p&gt;Our workgroup produces a desktop application that simulates building energy performance. It is a .NET application and when the user is running a lot of simulations, they can be quite time consuming. The simulations are totally parallelizable, and we have some very significant &lt;a href=&quot;http://www.nrel.gov/csc/capabilities.html&quot;&gt;HPC&lt;/a&gt; computing resources at &quot;the office&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One idea we've had is to allow the users to offload simulations that we know will be very long running (while each individual simulation runs about 30-120 seconds, running large numbers of simulations could take several days). Has anyone done this before? If so, did you use any libraries to make the job easier? Was it worth the effort? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;edited to add: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Individual tasks for offloading the simulation would be  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;packaging a file (about 5Mb),   &lt;/li&gt;&#xA;&lt;li&gt;uploading it to our servers,  &lt;/li&gt;&#xA;&lt;li&gt;decomposing the package into individual simulations (each takes about 30-120 seconds and is totally parallelizable), the number of simulations is a function of the number of options selected by the user (insulation, building orientation, etc) and the worst case of selecting every possible option would result in about 1E50 simulations. Running 100 to ~1E5 simulations isn't unknown, but the majority of users will run less than 10.   &lt;/li&gt;&#xA;&lt;li&gt;reassembling the completed simulations and downloading the now much larger file. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We aren't sure what interface to use, as our group is new to this, and with budget cuts, it might get completed in time, but needs to be easy for the next folks (if any) to maintain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This app already uses .NET 4 and can expand to use all the cores that the user has (our dev machines have 8 cores).&lt;/p&gt;&#xA;" OwnerUserId="16" LastEditorUserId="16" LastEditDate="2011-11-30T22:37:49.630" LastActivityDate="2013-01-20T11:15:24.260" Title="Desktop software with HPC resources for back end number crunching" Tags="|hpc|partitioning|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8" PostTypeId="1" CreationDate="2011-11-29T21:07:38.860" Score="17" ViewCount="2492" Body="&lt;p&gt;My background is in genomics, but I have recently been working with problems related to protein structure. I wrote a few relevant programs in C, building my own PDB file parser from scratch in the process. I didn't worry about making a really robust parser, I just knew that building one myself would be the best way to force myself to really understand the PDB format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that I've gone through this process, I'm looking for something a bit more robust and mature. Are there are any open-source protein structure libraries implemented in C? I was able to find a few on Google, but I had never heard of any of them before and they don't seem to be very mature or stable. A slightly related question: is everyone really doing all of these types of computations using Python? or homebrew code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. I'm essentially looking for a library that includes a PDB file parser, functions for calculating bond angles, bond lengths, torsion angles, surface accessible surface area, etc.&lt;/p&gt;&#xA;" OwnerUserId="47" LastActivityDate="2011-12-15T03:46:20.117" Title="Parsing protein structure data in C" Tags="|computational-biology|pdb|c|" AnswerCount="8" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="9" PostTypeId="1" AcceptedAnswerId="10" CreationDate="2011-11-29T21:17:51.367" Score="16" ViewCount="1330" Body="&lt;p&gt;I'm new to molecular dynamics (MD) simulations. What is the complexity of a molecular dynamics simulation in terms of simulation time? In other words, if I want increase the simulated time from 10 nanoseconds to 20 nanoseconds, what can I expect in terms of the increase in runtime?&lt;/p&gt;&#xA;" OwnerUserId="47" LastActivityDate="2011-11-30T20:42:21.173" Title="Complexity of MD simulations" Tags="|molecular-dynamics|complexity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10" PostTypeId="2" ParentId="9" CreationDate="2011-11-29T21:33:16.477" Score="18" Body="&lt;p&gt;Molecular dynamics simulations are linear (&lt;code&gt;O(n)&lt;/code&gt;) in the length of time simulated (assuming that the single timesteps ($\Delta t$) are unchanged).  Since each timestep is dependent only on the previous configuration (and not any of those prior to that), increasing the number of timesteps results in a linear increase in time.&lt;/p&gt;&#xA;" OwnerUserId="63" LastEditorUserId="63" LastEditDate="2011-11-30T20:42:21.173" LastActivityDate="2011-11-30T20:42:21.173" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11" PostTypeId="2" ParentId="8" CreationDate="2011-11-29T21:34:18.357" Score="8" Body="&lt;p&gt;There is a &lt;a href=&quot;http://salilab.org/~drussel/pdb/&quot;&gt;C++ library&lt;/a&gt; which is quite mature. This is probably as close as you will get to C. I myself haven't found any usable C library yet. You could use the C++ library and still write most of your code in C using &lt;code&gt;extern C { }&lt;/code&gt; in the C++ code.&lt;/p&gt;&#xA;" OwnerUserId="14" LastActivityDate="2011-11-29T21:34:18.357" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12" PostTypeId="2" ParentId="7" CreationDate="2011-11-29T21:41:09.323" Score="5" Body="&lt;p&gt;If you are in the .NET world, have you looked at the multi-threading support in .NET 4; and the asynchronous features that are being added to .NET 4.5?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multithreading does not help you offload processing to other machines, but it does let you better utilize the multicore PCs we're all using these days. 4.0 has some nice features in this regard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4.5 is currently in technology preview - so it is worth a look to see if it will help. There was an article in MSDN introducing the new asynchronous features in the last month or so (Oct or Nov, and definitely no earlier than Sept). It looked like these features were being pushed as a way of parallelizing traditionally less parallel code, but IIRC they also included some multi-server support.&lt;/p&gt;&#xA;" OwnerUserId="74" LastEditorUserId="74" LastEditDate="2011-11-29T22:22:24.460" LastActivityDate="2011-11-29T22:22:24.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13" PostTypeId="2" ParentId="7" CreationDate="2011-11-29T21:47:15.860" Score="10" Body="&lt;p&gt;You might consider using Condor: &lt;a href=&quot;http://research.cs.wisc.edu/condor/&quot;&gt;http://research.cs.wisc.edu/condor/&lt;/a&gt;. It could be overkill, depending on what kind of HPC resources you are talking about and how much they will change. However, its fairly easy to get starting, and it takes care of launching independent jobs, fault tolerance, load balance, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be more helpful if you were more specific about the individual tasks (how long, how asynchronous) and what interface you would use for interaction (launch process, library, etc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matt&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-11-29T21:47:15.860" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14" PostTypeId="2" ParentId="8" CreationDate="2011-11-29T21:49:18.060" Score="6" Body="&lt;p&gt;&lt;a href=&quot;http://openbabel.org/wiki/Main_Page&quot;&gt;OpenBabel&lt;/a&gt; is C++, but it's commonly used for general structural things - its main focus is conversion between and the ability to read a wide range of formats. As far as I know it doesn't have the ability to calculate SAS areas, though. &lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2011-11-29T21:49:18.060" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15" PostTypeId="2" ParentId="6" CreationDate="2011-11-29T21:53:36.743" Score="8" Body="&lt;p&gt;As a general rule, Slater basis elements are closer to the actual solution, and therefore fewer of them are needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of a Gaussian basis is the Gaussian Product Theorem, which allows products of gaussians at different centers to be easily integrated.  This allows for a significant speed up when attempting to converge around multiple nearby centers.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2011-11-29T21:53:36.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16" PostTypeId="1" AcceptedAnswerId="20" CreationDate="2011-11-29T22:07:40.703" Score="8" ViewCount="566" Body="&lt;p&gt;When VASP 5 was released, the performance was mostly slower than our make of VASP 4.6. I wrote it off as an optimization issue, and went on in my life. Then, in VASP 5.2, with the release notes, I realized that the it was all due to slower FFT, and is now fixed. Is there any way for me, as a user, to tell if the VASP make that I am using is built with the faster FFTW?&lt;/p&gt;&#xA;" OwnerUserId="80" LastActivityDate="2011-11-29T22:52:03.707" Title="How can I tell VASP 5.2 is compiled with FFTW3?" Tags="|fftw|compiling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17" PostTypeId="2" ParentId="3" CreationDate="2011-11-29T22:21:21.553" Score="10" Body="&lt;p&gt;A counterpoise correction is an &lt;em&gt;a posteriori&lt;/em&gt; correction that may be applied to correct for the &lt;a href=&quot;http://en.wikipedia.org/wiki/Basis_set_superposition_error&quot; rel=&quot;noreferrer&quot;&gt;basis set superposition error (BSSE)&lt;/a&gt;. More specifically, it uses mixed basis sets with &quot;Ghost orbitals&quot;. For more information see,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Boys, S.F. and Bernardi, F., &lt;a href=&quot;http://dx.doi.org/10.1080/00268977000101561&quot; rel=&quot;noreferrer&quot;&gt;&quot;The calculation of small molecular interactions by the differences of separate total energies. Some procedures with reduced errors&quot;&lt;/a&gt;, &lt;em&gt;Mol. Phys.&lt;/em&gt;, 19 (1970), 553.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="80" LastEditorUserId="127" LastEditDate="2011-11-30T15:46:51.277" LastActivityDate="2011-11-30T15:46:51.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18" PostTypeId="2" ParentId="4" CreationDate="2011-11-29T22:40:34.730" Score="11" Body="&lt;p&gt;This is a pretty big topic, but the basic important qualities of an atom-centered basis set are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&lt;li&gt;The number of separate, unlinked functions it uses at the valence level (the so-called *zeta* count), which allow the modeling of that many different electronic environments, practically speaking.&lt;/li&gt;&#xA;&lt;li&gt;Additional higher angular momentum so-called *polarisation* functions to allow for more angular complexity.&lt;/li&gt;&#xA;&lt;li&gt;Whether it has diffuse functions to model more spacially extensive orbitals - so called *augmented* functions.&lt;/li&gt;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Generally, though, for Hartree-Fock:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&lt;li&gt;Double-zeta for very quick, very rough structural parameters.&lt;/li&gt;&#xA;&lt;li&gt;Double-zeta + polarisation for slightly less rough structural information, large systems (100+ atoms), or simple bonding environments like alkanes.&lt;/li&gt;&#xA;&lt;li&gt;Triple-zeta for better accuracy and useful energies.&lt;/li&gt;&#xA;&lt;li&gt;Augmented functions if you're working with anions.&lt;/li&gt;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Going to a better basis set than this for Hartree-Fock probably won't gain you much. In any case, unless you're moving on to a more complex method, you should probably be using density-functional theory (DFT) instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information on basis sets including how different basis sets are labelled, I recommend chapter 5.2 in Jensen's &quot;Introduction to Computational Chemistry&quot;.&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2011-11-29T22:40:34.730" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19" PostTypeId="2" ParentId="1" CreationDate="2011-11-29T22:43:37.113" Score="7" Body="&lt;p&gt;In short, (T) means a perturbative (non-iterative) correction to a CCSD calculation. From a &lt;a href=&quot;http://www.ipc.kit.edu/theochem/download/Kapitel3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;talk demonstrating the derivation of the CCSD(T)&lt;/a&gt; method:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The CCSD(T) method consists of a perturbative&#xA;(noniterative) correction to the CCSD energy.&lt;/p&gt;&#xA;&lt;p&gt;• Utilizing the expressions from perturbation theory,&#xA;approximate second order triples amplitudes *&lt;span class=&quot;math-container&quot;&gt;$\hat{T}^{(2)}_3$&lt;/span&gt; are&#xA;generated from the CCSD &lt;span class=&quot;math-container&quot;&gt;$\hat{T}_2$&lt;/span&gt; amplitudes (rather than from&#xA;first order amplitudes).&lt;/p&gt;&#xA;&lt;p&gt;• The second order corrected wave function is then used to&#xA;compute the fourth and fifth order energy corrections, which&#xA;are added to the CCSD energy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Coupled_cluster&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; has a decent page about CC in general. A very good &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/10.1002/9780470125915.ch2&quot; rel=&quot;nofollow noreferrer&quot;&gt;review&lt;/a&gt;&lt;sup&gt;[1]&lt;/sup&gt; by T. Daniel Crawford and Henry F. Schaefer III is available online. There is a section discussing the (T) correction specifically.&#xA;&lt;a href=&quot;https://doi.org/10.1002/9780470125915.ch2&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Rev. Comput. Chem.&lt;/em&gt; &lt;strong&gt;2007,&lt;/strong&gt; 33–136&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;References:&lt;/h3&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Crawford, T. D.; Schaefer, H. F. An Introduction to Coupled Cluster Theory for Computational Chemists. &lt;em&gt;Rev. Comput. Chem.&lt;/em&gt; &lt;strong&gt;2007,&lt;/strong&gt; 33–136. &lt;a href=&quot;https://doi.org/10.1002/9780470125915.ch2&quot; rel=&quot;nofollow noreferrer&quot;&gt;DOI: 10.1002/9780470125915.ch2&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="79" LastEditorUserId="23791" LastEditDate="2022-01-17T21:25:28.513" LastActivityDate="2022-01-17T21:25:28.513" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20" PostTypeId="2" ParentId="16" CreationDate="2011-11-29T22:52:03.707" Score="6" Body="&lt;p&gt;There are a few possibilities: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ask whoever built it. :) &lt;/li&gt;&#xA;&lt;li&gt;Run strings vasp | grep -i fft and see if there's evidence of the version buried in the executable. &lt;/li&gt;&#xA;&lt;li&gt;Run ldd vasp and look for a dynamically linked fft library.  This last option is unlikely to produce results because you're more likely to have a static FFTW library (if I recall the build process for FFTW correctly).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27" LastActivityDate="2011-11-29T22:52:03.707" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21" PostTypeId="1" CreationDate="2011-11-29T23:01:58.557" Score="15" ViewCount="5627" Body="&lt;p&gt;Specifically I want to extend work involving B3LYP started with Gaussian 03 but continued with GAMESS-US. The energies provided by the default B3LYP methods are not the same. There is a discussion about this in the GAMESS-US manual (Further Information section):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note that B3LYP in GAMESS is based in part on the VWN5&#xA;  electron gas correlation functional. Since there are five&#xA;  formulae with two possible parameterizations mentioned in&#xA;  the VWN paper about local correlation, other programs may&#xA;  use other choices, and therefore generate different B3LYP&#xA;  energies. For example, NWChem's manual says it uses the&#xA;  &quot;VWN 1 functional with RPA parameters as opposed to the&#xA;  prescribed Monte Carlo parameters&quot; as its default. Should&#xA;  you wish to use this VWN1 formula in a B3LYP hybrid, simply&#xA;  choose &quot;DFTTYP=B3LYP1&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It says the defaults are different between GAMESS and NWCHEM and that there is an option to get GAMESS to do the same calculation as NWCHEM is doing by default.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I get a G03 and a GAMESS B3LYP calculation to agree?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are the differences between various software packages' default implementation of B3LYP and their capabilities, i.e. can their B3LYP definitions/implementations be adjusted?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="79" LastEditorUserId="852" LastEditDate="2012-02-02T03:22:03.777" LastActivityDate="2013-08-02T21:05:24.253" Title="How is B3LYP implemented in Gaussin 0*, GAMESS-US, Molpro, ... etc?" Tags="|computational-chemistry|density-functional-theory|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22" PostTypeId="1" CreationDate="2011-11-29T23:26:12.437" Score="7" ViewCount="1129" Body="&lt;p&gt;I often use inverse interval graphs to represent biologically relevant features along a genomic sequence. For example, given a (relatively) small genomic region, the graph would contain a node for each gene in the region, and there would be an edge between two nodes if the corresponding genes do &lt;em&gt;not&lt;/em&gt; overlap (the complement of the interval graph).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently implemented a program that uses the Bron-Kerbosch algorithm to list all maximal cliques for an inverse interval graph corresponding to a small genomic region. This algorithm has an exponential complexity, but I didn't worry about it at first since I didn't expect more than a handful of genes to be present in a given region. However, I have since encountered a few cases in my data that the Bron-Kerbosch algorithm cannot handle in a reasonable amount of time or space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Clique_problem#Listing_all_maximal_cliques&quot;&gt;This Wikipedia article&lt;/a&gt; mentions several alternatives to the Bron-Kerbosch algorithm that have a polynomial complexity as long as the number of cliques is polynomially bounded. I have begun looking into &lt;a href=&quot;http://epubs.siam.org/sicomp/resource/1/smjcat/v14/i1/p210_s1&quot;&gt;Chiba &amp;amp; Nishizeki's algorithm&lt;/a&gt; as a possible alternative, but the formulation is very dense. Before I spend too much time looking into this, I wanted to ask two questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First, since my graph is an &lt;em&gt;inverse interval graph&lt;/em&gt; and not an interval graph, am I actually guaranteed polynomial runtime with these alternative algorithms?&lt;/li&gt;&#xA;&lt;li&gt;Second, assuming I do want to implement one of these alternative algorithms, do any of them have available reference implementations? I don't expect a reference implementation to work out of the box with my problem, but it would be extremely helpful when trying to adapt the algorithm for my program.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="47" LastActivityDate="2012-02-03T01:38:46.457" Title="Alternative to Bron-Kerbosch algorithm for enumerating maximal cliques in inverse interval graphs" Tags="|computational-biology|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23" PostTypeId="2" ParentId="8" CreationDate="2011-11-29T23:34:46.197" Score="3" Body="&lt;p&gt;If you're willing to go with C++, then I'd recommend &lt;a href=&quot;http://esbtl.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;ESBTL&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of dealing with files in PDB format, you might consider downloading them from the Protein Data Bank in &lt;a href=&quot;http://pdbml.pdb.org/&quot; rel=&quot;nofollow&quot;&gt;PDBML format&lt;/a&gt;, which is actually XML. You can then parse PDBML files using your favorite XML library for C (for example, with &lt;a href=&quot;http://xmlsoft.org/&quot; rel=&quot;nofollow&quot;&gt;Libxml2&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2011-12-01T05:58:54.380" LastActivityDate="2011-12-01T05:58:54.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24" PostTypeId="1" CreationDate="2011-11-29T23:39:45.603" Score="17" ViewCount="296" Body="&lt;p&gt;I'm from the field of accelerator physics, specifically related to circular &lt;a href=&quot;http://en.wikipedia.org/wiki/Storage_ring&quot;&gt;storage rings&lt;/a&gt; for synchrotron light sources.  High energy electrons circulate around the ring, guided by magnetic fields.  The electrons circulate billions of times and one wants to predict the stability.  You can describe the motion of the electrons at one point in the ring in terms of phase space (position, momentum space).  Each turn around the ring, the particle returns to a new position and momentum, and this defines a map in phase space called the &quot;one-turn map&quot;.  We may assume that there is a fixed point at the origin, and so it can be expanded in a power series.  Thus, one wants to know about the stability of iterated power series maps.  There are many hard questions about this, and the topic has an old history.   Numerous libraries have been implemented- to implement so called Truncated Power Series Algebra.  (See e.g. &lt;a href=&quot;http://cern.ch/AccelConf/pac97/papers/pdf/9P113.PDF&quot;&gt;this paper&lt;/a&gt; about zlib by Y. Yan.  More background on the physics and one approach to analysis is the normal form approach, e.g. Bazzani et. al. &lt;a href=&quot;http://cdsweb.cern.ch/record/262179?ln=en&quot;&gt;here&lt;/a&gt;.)  The question is how to use such a library, and how to solve the stability problem.  The main approach used in beam dynamics has been normal form analysis, which I don't believe has been successful.  I wonder if some kind of spectral methods have been developed in other fields (perhaps along the lines of something like &lt;a href=&quot;http://prl.aps.org/abstract/PRL/v85/i17/p3620_1&quot;&gt;this&lt;/a&gt;?).  Can someone think of another domain where the long term stability of iterated power series maps with a fixed point at the origin are analyzed, so we might share knowledge or get some fresh ideas? One example I know of is the work of Fishman and &quot;Accelerator Modes&quot; in atomic physics.  Are there others?  What other systems can be modeled as a kicked rotor, or a Henon map?&lt;/p&gt;&#xA;" OwnerUserId="84" LastEditorUserId="84" LastEditDate="2011-11-30T20:27:15.823" LastActivityDate="2011-11-30T20:27:15.823" Title="Uses of power series maps" Tags="|algorithms|polynomials|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25" PostTypeId="2" ParentId="8" CreationDate="2011-11-29T23:46:41.263" Score="3" Body="&lt;p&gt;I know this is about C, but there is a great way to do this using the GLGRAPHICS library, which implements OPENGL in Processing (a java based framework with c++ like syntax). OpenGL is basically the same regardless of what language you use, so Java shouldn't make too big of a performance difference. Anyway, the GLGRAPHICS library comes with a pdb viewer that works out of the box. Its super easy to tweak and the Processing forum has a really responsive following. It could be a good starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GLGRAPHICS Library &#xA;&lt;a href=&quot;http://sourceforge.net/projects/glgraphics/files/glgraphics/1.0/&quot; rel=&quot;nofollow&quot;&gt;http://sourceforge.net/projects/glgraphics/files/glgraphics/1.0/&lt;/a&gt;&#xA;see GLGRAPHICS/examples/PDBview&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Processing&#xA;&lt;a href=&quot;http://processing.org/&quot; rel=&quot;nofollow&quot;&gt;http://processing.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, you might want to check out this (less 3d but still cool example) &lt;a href=&quot;http://www.mydisk.se/tgn380/webpage/RECEPTOR/&quot; rel=&quot;nofollow&quot;&gt;http://www.mydisk.se/tgn380/webpage/RECEPTOR/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2011-11-29T23:46:41.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26" PostTypeId="1" CreationDate="2011-11-29T23:51:40.310" Score="5" ViewCount="110" Body="&lt;p&gt;&lt;a href=&quot;http://sccn.ucsd.edu/eeglab/&quot; rel=&quot;nofollow&quot;&gt;EEGLab&lt;/a&gt; doesn't work well due to the data being from a Neuromag MEG. Any thoughts for alternatives?&lt;/p&gt;&#xA;" OwnerUserId="101" LastEditorUserId="123" LastEditDate="2012-08-11T20:31:38.867" LastActivityDate="2012-08-11T20:31:38.867" Title="What are some good ICA packages for physiological data, like MEG?" Tags="|neuroscience|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27" PostTypeId="1" AcceptedAnswerId="1467" CreationDate="2011-11-30T00:02:08.387" Score="18" ViewCount="326" Body="&lt;p&gt;Suppose I have a matrix equation recursively defined as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A[n] = inverse([1 - b[n]A[n+1]]) * a[n]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then the equation for A[1] looks similar to a continued fraction, for which there are some highly efficient methods that avoid tedious recalculation (See &quot;Numerical Recipes&quot; for some examples). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I wonder if there are analogous methods that allow the coefficients b[n] and a[n] to be matrices, with the only constraint that b[n] A[n+1] be a square matrix so that the matrix &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 - b[n]A[n+1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is actually invertible. &lt;/p&gt;&#xA;" OwnerUserId="105" LastActivityDate="2012-07-05T04:31:33.290" Title="Is there an efficient algorithm for matrix-valued continued fractions?" Tags="|algorithms|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28" PostTypeId="2" ParentId="24" CreationDate="2011-11-30T01:03:06.833" Score="5" Body="&lt;p&gt;You probably know this already, but it sounds like something from the world of chaos theory and fractals? (hence it is computationally &quot;difficult&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your question, have you looked at the world of planetary mechanics and N-body problems? These are also forced to use iterative solutions, and the fundamental underlying physics is N^2, although the force sources are typically allowed to move around as well - just to complicate things further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a long time since I've looked at them, but your mention of phase maps of stability sound very much like Henon Maps. I'm sure these must have wider applications, but they are usually described in terms of planetary stabilities (eg. the stability of a second moon in a planet-moon system).&lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2011-11-30T01:03:06.833" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29" PostTypeId="2" ParentId="24" CreationDate="2011-11-30T01:05:58.963" Score="4" Body="&lt;p&gt;You could look into asymptotic behavior of discrete &lt;a href=&quot;http://en.wikipedia.org/wiki/Dynamical_systems&quot; rel=&quot;nofollow&quot;&gt;Dynamical Systems&lt;/a&gt;. There is both a rich theoretical literature on this topic in mathematics and more applied literature in physics and computer science. &lt;/p&gt;&#xA;" OwnerUserId="19" LastActivityDate="2011-11-30T01:05:58.963" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="30" PostTypeId="1" AcceptedAnswerId="31" CreationDate="2011-11-30T05:16:40.617" Score="23" ViewCount="14465" Body="&lt;p&gt;I am using &lt;a href=&quot;http://mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt;'s nonlinear solver package &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/SNES/index.html&quot;&gt;SNES&lt;/a&gt; to solve a system of nonlinear equations obtained by discretizing a partial differential equation. How can I determine why the solver is not converging and what can I do to successfully solve my equations?&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2011-12-02T06:57:49.743" LastActivityDate="2019-03-29T19:12:51.567" Title="Why is Newton's method not converging?" Tags="|petsc|pde|implicit-methods|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="31" PostTypeId="2" ParentId="30" CreationDate="2011-11-30T05:23:46.830" Score="44" Body="&lt;p&gt;Newton's method may not converge for many reasons, here are some of the most common.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The Jacobian is wrong (or correct in sequential but not in parallel).&lt;/li&gt;&#xA;&lt;li&gt;The linear system is not solved or is not solved accurately enough.&lt;/li&gt;&#xA;&lt;li&gt;The Jacobian system has a singularity that the linear solver is not handling.&lt;/li&gt;&#xA;&lt;li&gt;There is a bug in the function evaluation routine.&lt;/li&gt;&#xA;&lt;li&gt;The function is not continuous or does not have continuous first derivatives (e.g. phase change or TVD limiters).&lt;/li&gt;&#xA;&lt;li&gt;The equations may not have a solution (e.g. limit cycle instead of a steady state) or there may be a &quot;hill&quot; between the initial guess and the steady state (e.g. reactants must ignite and burn before reaching a steady state, but the steady-state residual will be larger during combustion).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here are some of the ways to help debug lack of convergence of Newton.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Run with the options &lt;code&gt;-snes_monitor -ksp_monitor_true_residual -snes_converged_reason -ksp_converged_reason&lt;/code&gt;.&#xA;If the linear solve does not converge, check if the Jacobian is correct, then see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/513/why-is-my-iterative-linear-solver-not-converging&quot;&gt;this question&lt;/a&gt;.&#xA;If the preconditioned residual converges, but the true residual does not, the preconditioner may be singular.&#xA;If the linear solve converges well, but the line search fails, the Jacobian may be incorrect.&lt;/li&gt;&#xA;&lt;li&gt;Run with &lt;code&gt;-pc_type lu or -pc_type svd&lt;/code&gt; to see if the problem is a poor linear solver&lt;/li&gt;&#xA;&lt;li&gt;Run with &lt;code&gt;-mat_view&lt;/code&gt; or &lt;code&gt;-mat_view_draw&lt;/code&gt; to see if the Jacobian looks reasonable&lt;/li&gt;&#xA;&lt;li&gt;Run with &lt;code&gt;-snes_type test -snes_test_display&lt;/code&gt; to see if the Jacobian you are using is wrong. Compare the output when you add &lt;code&gt;-mat_fd_type ds&lt;/code&gt; to see if the result is sensitive to the choice of differencing parameter.&lt;/li&gt;&#xA;&lt;li&gt;Run with &lt;code&gt;-snes_mf_operator -pc_type lu&lt;/code&gt; to see if the Jacobian you are using is wrong. If the problem is too large for a direct solve, try &lt;code&gt;-snes_mf_operator -pc_type ksp -ksp_ksp_rtol 1e-12&lt;/code&gt;. Compare the output when you add &lt;code&gt;-mat_mffd_type ds&lt;/code&gt; to see if the result is sensitive to choice of differencing parameter.&lt;/li&gt;&#xA;&lt;li&gt;Run on one processor to see if the problem is only in parallel.&lt;/li&gt;&#xA;&lt;li&gt;Run with &lt;code&gt;-snes_ls_monitor&lt;/code&gt; to see if the line search is failing (this is usually a sign of a bad Jacobian).&lt;/li&gt;&#xA;&lt;li&gt;Run with &lt;code&gt;-info&lt;/code&gt; to get more detailed information on the solution process.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here are some ways to help the Newton process if everything above checks out&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Run with grid sequencing (&lt;code&gt;-snes_grid_sequence&lt;/code&gt; is all you need if working with a &lt;code&gt;DM&lt;/code&gt;) to generate better initial guess on your finer mesh&lt;/li&gt;&#xA;&lt;li&gt;Run with quad precision (&lt;code&gt;./configure --with-precision=__float128 --download-f2cblaslapack&lt;/code&gt; with PETSc 3.2 and later, needs version 4.6 or later GNU compilers)&lt;/li&gt;&#xA;&lt;li&gt;Change the units (nondimensionalization), boundary condition scaling, or formulation so that the Jacobian is better conditioned.&lt;/li&gt;&#xA;&lt;li&gt;Mollify features in the function that do not have continuous first derivatives (often occurs when there are &lt;code&gt;if&lt;/code&gt; statements in the residual evaluation, e.g. phase change or TVD limiters). Use a variational inequality solver (&lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/SNES/SNESVINEWTONRSLS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNESVINEWTONRSLS&lt;/a&gt;) if the discontinuities are of fundamental importance.&lt;/li&gt;&#xA;&lt;li&gt;Try a trust region method (&lt;code&gt;-ts_type tr&lt;/code&gt;, may have to adjust parameters).&lt;/li&gt;&#xA;&lt;li&gt;Run with some continuation parameter from a point where you know the solution, see &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/TS/TSPSEUDO.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TSPSEUDO&lt;/a&gt; for solving steady-state problems.&#xA;There are homotopy solver packages like PHCpack that can get you all possible solutions (and tell you that it has found them all) but those are not scalable and cannot solve anything but small problems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="24963" LastEditDate="2019-03-29T19:12:51.567" LastActivityDate="2019-03-29T19:12:51.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32" PostTypeId="1" AcceptedAnswerId="147" CreationDate="2011-11-30T05:31:13.777" Score="10" ViewCount="209" Body="&lt;p&gt;Various important sets of polynomials (Legendre, Chebyshev, etc.) are orthogonal over some real interval with some weighting.  Are there known families of polynomials that are orthogonal over other curves in the complex plane?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, I would like a basis for the polynomials of degree n that is orthogonal over, say, the circle &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-1 + \exp(it)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for  $0\le t&amp;lt; 2\pi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason that I'm posting this here is that I have a numerical problem involving a matrix of polynomial values over points in the complex plane.  Using the monomial basis, it becomes very ill-conditioned for most sets of points.  I'd like to use another basis to improve the conditioning, but it's not clear that using, say, Legendre or Chebyshev polynomials will improve the conditioning for general curves in the complex plane.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="127" LastEditDate="2011-12-01T17:04:22.730" LastActivityDate="2011-12-01T17:05:54.787" Title="Polynomials that are orthogonal over curves in the complex plane" Tags="|basis-set|special-functions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="34" PostTypeId="1" AcceptedAnswerId="35" CreationDate="2011-11-30T05:49:09.673" Score="16" ViewCount="4967" Body="&lt;p&gt;I have a PETSc &lt;code&gt;Mat&lt;/code&gt; and would like to estimate its condition number.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-09-17T14:16:04.630" Title="How can I estimate the condition number of a large sparse matrix using PETSc?" Tags="|petsc|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="35" PostTypeId="2" ParentId="34" CreationDate="2011-11-30T05:51:28.267" Score="12" Body="&lt;p&gt;For small matrices, the condition number can be reliably computed using the singular value decomposition. Do a &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/KSP/KSPSolve.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;KSPSolve()&lt;/code&gt;&lt;/a&gt; with the matrix and run with &lt;code&gt;-pc_type svd -pc_svd_monitor&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For larger matrices, you can estimate the condition number using a Krylov method. For example, the Arnoldi iteration performed by GMRES incrementally computes a Hessenberg decomposition. The extremal singular values and eigenvalues of the Hessenberg matrix are good approximations to those of the original matrix. To have PETSc estimate eigenvalues this way, run with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-ksp_monitor_singular_value -ksp_type gmres -ksp_gmres_restart 1000 -pc_type none&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These options say to estimate the extremal singular values on each Krylov iteration. GMRES is used to compute the Krylov space (you could also use CG) with a huge restart. At restarts, GMRES discards the current Krylov space, so all the progress on singular value estimates is lost in a restart. The final option &lt;code&gt;-pc_type none&lt;/code&gt; says to perform this iteration on the unpreconditioned matrix. By default, the preconditioned operator ($P^{-1} A$ or $A P^{-1}$) would be used, so you would end up with an estimate for the preconditioned operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will generally be accurate for the largest singular values, but may overestimate the smallest singular value unless the method has converged. If you have a solver for the matrix (e.g. using &lt;code&gt;KSPSolve()&lt;/code&gt;), then you can estimate the smallest singular value of $A$ using the same procedure applied to $A^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use &lt;a href=&quot;http://www.grycap.upv.es/slepc/&quot; rel=&quot;noreferrer&quot;&gt;SLEPc&lt;/a&gt; if you need more accurate estimates of the smallest singular value (and for all other eigenvalue and singular value problems).&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-09-17T14:16:04.630" LastActivityDate="2012-09-17T14:16:04.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="36" PostTypeId="1" CreationDate="2011-11-30T05:55:29.450" Score="17" ViewCount="956" Body="&lt;p&gt;Many numerical methods for hyperbolic PDEs are based on the use of Riemann solvers.  Such solvers are essential for accurately capturing shock waves.  There are a range of such solvers available for the most well-studied systems (e.g., exact solvers, Roe solvers, HLL solvers).  How should I decide which to use?&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-01-06T20:17:53.570" LastActivityDate="2012-01-06T20:17:53.570" Title="How can I choose a good Riemann solver when numerically solving a system of hyperbolic PDEs?" Tags="|pde|hyperbolic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="37" PostTypeId="1" CreationDate="2011-11-30T05:59:14.643" Score="25" ViewCount="2014" Body="&lt;p&gt;Many numerical approaches to CFD can be extended to arbitrarily high order (for instance, discontinuous Galerkin methods, WENO methods, spectral differencing, etc.).  How should I choose an appropriate order of accuracy for a given problem?&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-05-24T17:07:36.323" Title="When is a high order method useful for computational fluid dynamics simulations?" Tags="|fluid-dynamics|accuracy|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="38" PostTypeId="1" AcceptedAnswerId="98" CreationDate="2011-11-30T06:04:20.440" Score="17" ViewCount="1370" Body="&lt;p&gt;Numerical methods for solving PDEs (or ODEs) fall into two broad categories: explicit and implicit methods.  Implicit methods allow larger stable timesteps but require more work per step.  For hyperbolic PDEs, the common wisdom is that implicit methods usually do not pay off because the use of timesteps larger than those allowed by the CFL condition leads to very inaccurate results.  However, implicit methods are used in some cases.  For a given application, how should one choose whether to use an explicit or implicit method?&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-11-30T12:43:13.780" LastActivityDate="2011-11-30T16:16:15.363" Title="When should implicit methods be used in the integration of hyperbolic PDEs?" Tags="|pde|hyperbolic-pde|stability|implicit-methods|explicit-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="39" PostTypeId="1" CreationDate="2011-11-30T06:08:22.897" Score="7" ViewCount="577" Body="&lt;p&gt;I have a relatively simple convex optimization problem that involves less than 100 variables but contains a terribly ill-conditioned matrix.  I have tried CVX and CPLEX; even though both can typically solve the problem in about 1 second, both fail when the condition number of the matrix becomes very large.  An arbitrary-precision solver would be able to solve this problem quickly and accurately.  Does any such implementation exist?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; The conditioning of the problem has been considered in detail and is not part of this question.  I'm just asking about software.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-12-02T15:24:41.313" LastActivityDate="2012-07-17T12:28:24.423" Title="Does there exist an arbitrary-precision convex optimization solver?" Tags="|precision|floating-point|convex-optimization|conditioning|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="40" PostTypeId="1" AcceptedAnswerId="479" CreationDate="2011-11-30T06:09:01.203" Score="15" ViewCount="941" Body="&lt;p&gt;Standard multigrid and domain decomposition methods do not work, but I have large 3D problems and direct solvers are not an option. What methods should I try?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How are my choices affected by the following considerations?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;coefficients vary over several orders of magnitude, or&lt;/li&gt;&#xA;&lt;li&gt;finite element versus finite different methods are used&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-07-24T18:09:54.400" Title="What is a scalable preconditioner for high-frequency Helmholtz?" Tags="|pde|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="41" PostTypeId="1" AcceptedAnswerId="48" CreationDate="2011-11-30T06:16:31.517" Score="33" ViewCount="1922" Body="&lt;p&gt;Engineers often insist on using locally conservative methods such as finite volume, conservative finite difference, or discontinuous Galerkin methods for solving PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can go wrong when using a method that is not locally conservative?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, so local conservation is important for hyperbolic PDEs, what about elliptic PDEs?&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2011-11-30T06:51:48.787" LastActivityDate="2019-04-30T05:42:29.040" Title="Why is local conservation important when solving PDEs?" Tags="|pde|hyperbolic-pde|fluid-dynamics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="42" PostTypeId="2" ParentId="5" CreationDate="2011-11-30T06:19:24.107" Score="3" Body="&lt;p&gt;Wikipedia has an answer here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Basis_set_(chemistry)#Correlation-consistent_basis_sets&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Basis_set_(chemistry)#Correlation-consistent_basis_sets&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: adding introductory text from Wikipedia:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Some of the most widely used basis sets are those developed by Dunning&#xA;  and coworkers, since they are designed to converge systematically to&#xA;  the complete-basis-set (CBS) limit using empirical extrapolation&#xA;  techniques. For first- and second-row atoms, the basis sets are&#xA;  cc-pVNZ where N=D,T,Q,5,6,... (D=double, T=triples, etc.). The 'cc-p',&#xA;  stands for 'correlation-consistent polarized' and the 'V' indicates&#xA;  they are valence-only basis sets. They include successively larger&#xA;  shells of polarization (correlating) functions (d, f, g, etc.). More&#xA;  recently these 'correlation-consistent polarized' basis sets have&#xA;  become widely used and are the current state of the art for correlated&#xA;  or post-Hartree-Fock calculations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-11-30T12:03:47.120" LastActivityDate="2011-11-30T12:03:47.120" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="43" PostTypeId="1" AcceptedAnswerId="44" CreationDate="2011-11-30T06:22:34.080" Score="14" ViewCount="460" Body="&lt;p&gt;High Reynolds number flows produce very thin boundary layers. If wall resolution is used in Large Eddy Simulation, the aspect ratio may be on the order of $10^6$. Many methods become unstable in this regime because the inf-sup constant degrades as the square root of the aspect ratio or worse. The inf-sup constant is important because it affects the condition number of the linear system and the approximation properties of the discrete solution. In particular, the following a priori bounds on the discrete error hold (Brezzi and Fortin 1991)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{split}&#xA;\mu \lVert {\mathbf u} - \mathbf u_h \rVert_{H^1} \le C \left[ \frac{\mu}{\beta} \inf_{\mathbf v \in \mathcal V} \lVert{\mathbf u - \mathbf v}\rVert_{H^1} + \inf_{q \in \mathcal Q} \lVert p-q \rVert_{L^2} \right] \\&#xA;\lVert{p - p_h}\rVert_{L^2} \le \frac{C}{\beta} \left[ \frac{\mu}{\beta} \inf_{\mathbf v \in \mathcal V} \lVert{\mathbf u - \mathbf v}\rVert_{H^1} + \inf_{q\in \mathcal Q} \lVert{p-q}\rVert_{L^2} \right]&#xA;\end{split}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mu$ is the dynamic viscosity and $\beta$ is the inf-sup constant. From this we see that as $\beta \to 0$, the velocity and (especially) pressure approximations becomes worse than the best available in the finite element space (i.e. the constant of Galerkin optimality grows as $\beta^{-1}$ and $\beta^{-2}$ respectively).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What methods have uniform inf-sup stability independent of the aspect ratio?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which of these can be used with unstructured meshes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do the estimates generalize to high order approximations?&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-07-22T12:26:30.023" LastActivityDate="2013-07-22T12:26:30.023" Title="What spatial discretizations work for incompressible flow with anisotropic boundary meshes?" Tags="|pde|finite-element|fluid-dynamics|stability|incompressible|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="44" PostTypeId="2" ParentId="43" CreationDate="2011-11-30T06:44:32.197" Score="14" Body="&lt;p&gt;MAC finite difference schemes (Harlow and Welch 1965) are uniformly stable, but require smooth structured grids and are only second order accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite element methods are preferred for unstructured and high order methods. For continuous Galerkin finite element methods, there are no known spaces that have optimal approximation properties and are uniformly stable.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$Q_k-P_{k-1}^{\mathrm{disc}}$ has optimal approximation properties and is locally conservative, but the inf-sup constant degrades as the square root of the aspect ratio. See Bernardi &amp;amp; Maday 1999 for details.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$Q_k-Q_{k-2}^{\mathrm{disc}}$ has inf-sup constant independent of the aspect ratio and is locally conservative, but the inf-sup constant scales as $\mathcal{O}\left(k^{\frac{1-d}{2}}\right)$ as the polynomial order is increased (Maday et al. 1992) on shape-regular meshes. On meshes with hanging nodes or reentrant corners, this bound is sharp in 2D (Schoetzau et al 1998), but further degrades to $k^{-3/2}$ in 3D (Toselli &amp;amp; Schwab 2003).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The rotated $Q_1-P_0$ nonconforming element from Rannacher &amp;amp; Turek 1994 is uniformly stable, has optimal approximation properties, and is locally conservative, but it does not satisfy a discrete Korn's inequality, so it needs boundary corrections for some boundary conditions and cannot be used for variable viscosity flows. Subsequent work by the authors has saught to stabilize these methods using edge fluxes, but the resulting discretizations lose many of the attractive efficiency properties.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ainsworth and Coggins 2000 construct highly technical spaces that do somewhat better, but seem to be of limited utility.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For discontinuous Galerkin, the picture is somewhat better:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The discontinuous space $Q_k-Q_{k-1}$ is uniformly stable and has optimal approximation properties (Schoetzau, Schwab, and Toselli 2004). This combination is not available with continuous velocity spaces. The inf-sup constant is still dependent on polynomial degree, however, scaling as $k^{-3/2}$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="21" LastEditDate="2011-12-01T21:59:08.853" LastActivityDate="2011-12-01T21:59:08.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="45" PostTypeId="1" AcceptedAnswerId="56" CreationDate="2011-11-30T06:59:31.680" Score="18" ViewCount="643" Body="&lt;p&gt;Physical quantities like pressure, density, energy, temperature, and concentration should always be positive, but numerical methods sometimes compute negative values during the solution process. This is not okay because the equations will compute complex or infinite values (typically crashing the code). Which numerical methods can be used to guarantee that these quantities remain positive? Which of these methods is most efficient?&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-07-14T10:44:03.567" Title="Which methods can ensure that physical quantities remain positive throughout a PDE simulation?" Tags="|pde|fluid-dynamics|hyperbolic-pde|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="46" PostTypeId="1" CreationDate="2011-11-30T07:18:55.320" Score="43" ViewCount="2925" Body="&lt;p&gt;In evaluating the quality of a piece of software you are about to use (whether it's something you wrote or a canned package) in computational work, it is often a good idea to see how well it works on standard data sets or problems. Where might one obtain these tests for verifying computational routines?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(One website/book per answer, please.)&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2017-11-10T02:58:44.463" Title="Where can one obtain good data sets/test problems for testing algorithms/routines?" Tags="|testing|data-sets|" AnswerCount="18" CommentCount="4" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="47" PostTypeId="1" AcceptedAnswerId="55" CreationDate="2011-11-30T07:19:27.980" Score="12" ViewCount="2348" Body="&lt;p&gt;I've been using &lt;a href=&quot;http://www.gnu.org/s/gsl/&quot;&gt;GSL&lt;/a&gt; as the foundation of many of my simulations, but it's a little bit overkill for my purposes and it defines its own complex type for legacy reasons. Rather than code my own Runge-Kutta ODE solver, which would probably not be very efficient, are there any open source ODE solvers that use the native C99 complex type?&lt;/p&gt;&#xA;" OwnerUserId="116" LastActivityDate="2011-12-18T00:44:21.790" Title="Is there an open source set of ODE solvers for C that use the native C99 complex type?" Tags="|c|ode|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="48" PostTypeId="2" ParentId="41" CreationDate="2011-11-30T07:29:41.640" Score="32" Body="&lt;p&gt;In the solution of nonlinear hyperbolic PDEs, discontinuities (&quot;shocks&quot;) appear even when the initial condition is smooth.  In the presence of discontinuities, the notion of solution can only be defined in the weak sense.  The numerical velocity of a shock depends on the correct Rankine-Hugoniot conditions being imposed, which in turn depends on numerically satisfying the integral conservation law locally.  The &lt;a href=&quot;http://en.wikipedia.org/wiki/Lax%E2%80%93Wendroff_theorem&quot;&gt;Lax-Wendroff theorem&lt;/a&gt; guarantees that a convergent numerical method will converge to a weak solution of the hyperbolic conservation law &lt;em&gt;only if the method is conservative.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not only do you need to use a conservative method, in fact you need to use a method that conserves &lt;em&gt;the right quantities.&lt;/em&gt;  There's a nice example that explains this in LeVeque's &quot;Finite Volume Methods for Hyperbolic Problems&quot;, Section 11.12 and Section 12.9.  If you discretize Burgers' equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t + 1/2 (u^2)_x = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;via the consistent discretization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U^{n+1}_i = U^n_i  - \frac{\Delta t}{\Delta x} U^n_i (U^n_i-U^n_{i-1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you will observe that shocks move at the wrong speed, no matter how much you refine the grid.  That is, the numerical solution &lt;em&gt;will not converge to the true solution&lt;/em&gt;.  If you instead use the conservative discretization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U^{n+1}_i = U^n_i  - \frac{\Delta t}{2\Delta x} ( (U^n_i)^2-(U^n_{i-1})^2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;based on flux-differencing, shocks will move at the correct speed (which is the average of the states to the left and the right of the shock, for this equation).  This example is illustrated in &lt;a href=&quot;http://nbviewer.ipython.org/github/numerical-mathematics/nonlinear-waves-course/blob/master/importance_of_conservation.ipynb&quot;&gt;this IPython notebook I wrote&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For linear hyperbolic PDEs, and for other types of PDEs which typically have smooth solutions, local conservation is not a necessary ingredient for convergence.  However, it may be important for other reasons (e.g., if the total mass is a quantity of interest).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-04-16T12:49:56.963" LastActivityDate="2015-04-16T12:49:56.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="49" PostTypeId="5" CreationDate="2011-11-30T07:29:59.610" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-11-30T07:29:59.610" LastActivityDate="2011-11-30T07:29:59.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="50" PostTypeId="4" CreationDate="2011-11-30T07:29:59.610" Score="0" Body="The branch of chemistry dealing with the use of mathematical models to solve problems in chemistry." OwnerUserId="127" LastEditorUserId="1193" LastEditDate="2012-05-14T23:06:48.177" LastActivityDate="2012-05-14T23:06:48.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="51" PostTypeId="5" CreationDate="2011-11-30T07:32:07.607" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-11-30T07:32:07.607" LastActivityDate="2011-11-30T07:32:07.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="52" PostTypeId="4" CreationDate="2011-11-30T07:32:07.607" Score="0" Body="Partial differential equations (PDEs) are equations that relate the partial derivatives of a function of more than one variable. This tag is intended for questions on modeling phenomena with PDEs, solving PDEs, and other related aspects." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-11-30T08:41:01.007" LastActivityDate="2011-11-30T08:41:01.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="53" PostTypeId="2" ParentId="45" CreationDate="2011-11-30T07:34:27.837" Score="6" Body="&lt;p&gt;Assuming we are solving hyperbolic equations without any source terms and assuming we provide physical initial conditions, making sure the numerical scheme we use is &lt;strong&gt;Total Variation Diminishing&lt;/strong&gt; is a good way of ensuring the &quot;physicality&quot; of the computed solution. Since a TVD scheme preserves monotonicity, no new minima or maxima will be created and the solution will remain bounded by the initial values that we hopefully set correctly.&#xA;Of course the issue is that TVD schemes are not the most obvious ones. Among linear schemes, only first order schemes are TVD (Godunov 1954). So since the 50's, a variety of non-linear TVD schemes have been developed to combine high-accuracy and monotonicity for the solution of hyperbolic equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my applications, solving Navier-Stokes equations with large pressure/density gradients, we use a hybrid &lt;strong&gt;MUSCL&lt;/strong&gt;-central scheme to capture the large gradients/discontinuities and retain good accuracy away from them. The first MUSCL scheme (MUSCL stands for Monotone Upstream-centered Schemes for Conservation Laws) was devised by Van Leer in 1979.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know more about this subject, please consult the works of Harten, Van Leer, Lax, Sod and Toro.&lt;/p&gt;&#xA;" OwnerUserId="95" LastEditorUserId="95" LastEditDate="2011-11-30T08:00:24.863" LastActivityDate="2011-11-30T08:00:24.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="54" PostTypeId="1" AcceptedAnswerId="57" CreationDate="2011-11-30T07:51:57.293" Score="9" ViewCount="1488" Body="&lt;p&gt;I solve multi-species, compressible Navier-Stokes equations on a 3D structured grid. I have obtained a solution on a given grid (let's say a relatively coarse one). I want now to refine my grid and interpolate my previous solution on my new grid before restarting my simulation. Currently, we have an interpolation tool that builds a k-d tree of the 2 grids and then can use 2 different methods to compute the values on the new grid:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;simple averaging&lt;/li&gt;&#xA;&lt;li&gt;inverse-distance-weighted (IDW)&lt;/li&gt;&#xA;&lt;li&gt;moving least squares (MLS)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want to focus on accuracy because since I deal with large gradients, not capturing them correctly will generate waves when I restart my computation. I first tried simple averaging but the accuracy was not good enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought MLS method with polynomials of order 2 would give me reasonable results since it is supposed to be non-oscillatory. However, when I look at my interpolated field, I see local minima/maxima that overshoot values of my initial field. Does this mean the implementation of MLS in this program is not correct? Should I be careful with the size of my stencil and the order of the polynomials? Which other method would you recommend?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance !&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2011-11-30T08:47:55.320" Title="What is the most accurate interpolation method for a 3D-flowfield on a structured grid?" Tags="|fluid-dynamics|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="55" PostTypeId="2" ParentId="47" CreationDate="2011-11-30T08:38:19.247" Score="10" Body="&lt;p&gt;You might consider it &quot;overkill&quot;, but &lt;a href=&quot;http://mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt;'s time integration package can be used with C99 complex (configure &lt;code&gt;--with-scalar-type=complex&lt;/code&gt;). Supported methods include&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/TS/TSRK.html&quot;&gt;explicit Runge-Kutta&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/TS/TSSSP.html&quot;&gt;low-memory strong stability-preserving Runge-Kutta&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/TS/TSROSW.html&quot;&gt;Rosenbrock-W&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/TS/TSARKIMEX.html&quot;&gt;additive Runge-Kutta IMEX&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These implementations are most appropriate for high-dimensional problems such as semi-discretized partial differential equations (method of lines).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-11-30T08:38:19.247" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="56" PostTypeId="2" ParentId="45" CreationDate="2011-11-30T08:39:28.083" Score="14" Body="&lt;p&gt;The most common method is to reset negative values to some small, positive number.  Of course, this is not a mathematically sound solution.  A better general approach that may work and is easy, is to reduce the size of your time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Negative values often arise in the solution of hyperbolic PDEs, because the appearance of shocks can lead to oscillations, which will tend to create negative values if there are near-vacuum states near the shock.  Using a &lt;strong&gt;total variation diminishing (TVD)&lt;/strong&gt; or other non-oscillatory (&lt;strong&gt;ENO, WENO&lt;/strong&gt;) method can reduce this tendency.  Those methods are based on using nonlinear limiters to compute derivatives of the solution.  However, you may &lt;em&gt;still&lt;/em&gt; get negative values for several reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you use the method of lines and apply a high-order time integrator.  Most TVD schemes are provably TVD only in the semi-discrete sense or with Euler's method.  For higher order time integration, you should use a &lt;strong&gt;strong stability preserving (SSP)&lt;/strong&gt; time discretization; these schemes are also known as &quot;contractive&quot; or &quot;monotonicity preserving&quot;.  There is a recent book on the subject by Sigal Gottlieb, Chi-Wang Shu, and myself.&lt;/li&gt;&#xA;&lt;li&gt;If you don't use &lt;strong&gt;local characteristic decomposition&lt;/strong&gt; for systems of equations, your solution will not be TVD (TVD schemes only possess that property for scalar problems).  So it's best to reconstruct/interpolate in characteristic variables.&lt;/li&gt;&#xA;&lt;li&gt;If you have a nonlinear system, negative values can arise even if you use local characteristic decomposition.  For instance, any linearized Riemann solver (such as a Roe solver) for the shallow water equations or the Euler equations can be shown to generate negative values in sufficiently challenging conditions.  A solution is to use an &lt;strong&gt;HLL solver&lt;/strong&gt; (or a variant of HLL); some of those are provably positive.&lt;/li&gt;&#xA;&lt;li&gt;TVD schemes are only second order; higher order non-oscillatory schemes like WENO do not strictly satisfy TVD or maximum principles.  But a new modification of those high-order schemes does; it is developed in several recent papers by Xiangxiong Zhang (a student of Chi-Wang Shu).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are, of course, many other specialized approaches for particular equations, such as in David George's GeoClaw code, which uses a Riemann solver with extra non-physical waves to enforce positivity.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-11-30T09:47:35.590" LastActivityDate="2011-11-30T09:47:35.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="57" PostTypeId="2" ParentId="54" CreationDate="2011-11-30T08:47:55.320" Score="4" Body="&lt;p&gt;You could use monotone cubic splines:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Monotone_cubic_interpolation&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Monotone_cubic_interpolation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An explanation of how to do it in multi-D is here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1285766&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=1285766&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option would be weighted essentially non-oscillatory interpolation; there is a recent review paper on the topic by Chi-Wang Shu.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2011-11-30T08:47:55.320" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="58" PostTypeId="1" AcceptedAnswerId="64" CreationDate="2011-11-30T08:51:01.270" Score="28" ViewCount="4297" Body="&lt;p&gt;What's the state-of-the-art in the approximation of highly oscillatory integrals in both one dimension and higher dimensions to arbitrary precision?&lt;/p&gt;&#xA;" OwnerUserId="131" LastEditorUserId="4244" LastEditDate="2013-06-17T07:37:45.813" LastActivityDate="2013-06-17T07:37:45.813" Title="What's the state-of-the-art in highly oscillatory integral computation?" Tags="|quadrature|precision|numerical-analysis|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="59" PostTypeId="1" AcceptedAnswerId="70" CreationDate="2011-11-30T09:03:00.457" Score="13" ViewCount="402" Body="&lt;p&gt;Source terms, such as those due to bathymetry in the shallow water equations, need to be integrated in a special way in order to preserve physical steady states. Is there a general way to construct well-balanced methods, or does it require special techniques for each equation?&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-11-30T12:02:21.410" Title="How to construct well-balanced finite volume and discontinuous Galerkin methods for hyperbolic PDEs with source terms?" Tags="|pde|hyperbolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="60" PostTypeId="1" CreationDate="2011-11-30T09:10:44.560" Score="14" ViewCount="657" Body="&lt;p&gt;Multigrid methods usually solve Dirichlet problems on levels (e.g. point Jacobi or Gauss-Seidel). When using continuous finite element methods, it is much less expensive to assemble small Neumann problems than to assemble small Dirichlet problems. Non-overlapping domain decomposition methods such as BDDC (like FETI-DP) can be interpreted as multigrid methods that solve &quot;pinned&quot; Neumann problems on levels. Unfortunately, the condition number for multilevel BDDC scales as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C \left(1 + \log \left(\frac{H}{h}\right)\right)^{2L}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L$ is the number of levels and $H/h$ is the coarsening ratio. In contrast, the condition number for multigrid methods with smoothers based on Dirichlet problems have a condition number independent of the number of levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to solve &quot;pinned&quot; Neumann problems without losing level-independence?&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2011-12-01T02:20:06.057" LastActivityDate="2011-12-02T07:30:29.347" Title="Is there a multigrid algorithm that solves Neumann problems and has a convergence rate independent of the number of levels?" Tags="|pde|multigrid|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="61" PostTypeId="5" CreationDate="2011-11-30T09:12:41.187" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-11-30T09:12:41.187" LastActivityDate="2011-11-30T09:12:41.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="62" PostTypeId="4" CreationDate="2011-11-30T09:12:41.187" Score="0" Body="Questions about Computational Fluid Dynamics, focusing on the numerical tools that can be used to solve the Navier-Stokes equations." OwnerUserId="95" LastEditorUserId="95" LastEditDate="2011-11-30T15:25:33.423" LastActivityDate="2011-11-30T15:25:33.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="63" PostTypeId="1" CreationDate="2011-11-30T09:26:20.090" Score="5" ViewCount="164" Body="&lt;p&gt;The idea behind Large Eddy Simulation is to solve explicitely the scales of the flow that are resolved by the computational grid and to model the sub-grid scales. Ideally, the cut-off between these scales is located in the inertial range of the local turbulent field. This is because Kolmogorov's hypotheses predict a universal behavior of the turbulence in that inertial range and thus accurate modeling should be possible for the sub-grid scales. In practice, unless we consider a simple periodic box with isotropic turbulence, the cut-off between the resolved scales and the sub-grid does not occur in the inertial range for every point in the domain. Which begs the question: how can we know our current simulation is consistent with the LES framework?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which method do you use to evaluate your LES? Does it matter whether you are using an implicit or explicit filtering of the small scales?&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2011-12-04T04:14:34.850" Title="Which tool/metric do you recommend to evaluate the quality and validity of a Large Eddy Simulation?" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="64" PostTypeId="2" ParentId="58" CreationDate="2011-11-30T09:39:53.280" Score="21" Body="&lt;p&gt;I'm not entirely familiar with what's now done for cubatures (multidimensional integration), so I'll restrict myself to quadrature formulae.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of effective methods for the quadrature of oscillatory integrals. There are methods suited for finite oscillatory integrals, and there are methods for infinite oscillatory integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For infinite oscillatory integrals, two of the more effective methods used are Longman's method and the modified double exponential quadrature due to Ooura and Mori. (But see also these &lt;a href=&quot;http://dx.doi.org/10.1093/imanum/24.3.365&quot;&gt;two&lt;/a&gt; &lt;a href=&quot;http://dx.doi.org/10.1093/imanum/drh022&quot;&gt;papers&lt;/a&gt; by Arieh Iserles.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Longman's method relies on converting the oscillatory integral into an alternating series by splitting the integration interval, and then summing the alternating series with a sequence transformation method. For instance, when integrating an oscillatory integral of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_0^\infty f(t)\sin\,t\mathrm dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one converts this into the alternating sum&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{k=0}^\infty \int_{k\pi}^{(k+1)\pi} f(t)\sin\,t\mathrm dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms of this alternating sum are computed with some quadrature method like Romberg's scheme or Gaussian quadrature. Longman's &lt;a href=&quot;http://dx.doi.org/10.1017/S030500410003187X&quot;&gt;original method&lt;/a&gt; used the &lt;a href=&quot;http://mathworld.wolfram.com/EulersSeriesTransformation.html&quot;&gt;Euler transformation&lt;/a&gt;, but modern implementations replace Euler with more powerful convergence acceleration methods like the &lt;a href=&quot;http://dx.doi.org/10.1090/S0025-5718-1956-0084056-6&quot;&gt;Shanks transformation&lt;/a&gt; or the &lt;a href=&quot;http://dx.doi.org/10.1080/00207167308803075&quot;&gt;Levin transformation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://dx.doi.org/10.1016/S0377-0427%2899%2900223-X&quot;&gt;double exponential quadrature&lt;/a&gt; method, on the other hand, makes a clever change of variables, and then uses the trapezoidal rule to numerically evaluate the transformed integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For finite oscillatory integrals, Piessens (one of the contributors of QUADPACK) and Branders, in &lt;a href=&quot;http://dx.doi.org/10.1016/0771-050X%2875%2990033-9&quot;&gt;two&lt;/a&gt; &lt;a href=&quot;http://dx.doi.org/10.1007/BF02253692&quot;&gt;papers&lt;/a&gt;, detail a modification of Clenshaw-Curtis quadrature (that is, constructing an Chebyshev polynomial expansion of the nonoscillatory part of the integrand). &lt;a href=&quot;http://dx.doi.org/10.1016/0377-0427%2894%2900118-9&quot;&gt;Levin's method&lt;/a&gt;, on the other hand, uses a collocation method for the quadrature. (I am told there is now a more practical version of the old standby, Filon's method, but I've no experience with it.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;These are the methods I remember offhand; I'm sure I've forgotten other good methods for oscillatory integrals. I will edit this answer later if I remember them.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2011-11-30T09:39:53.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="65" PostTypeId="1" AcceptedAnswerId="76" CreationDate="2011-11-30T09:57:13.420" Score="17" ViewCount="635" Body="&lt;p&gt;Given an evolution PDE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t = Au + Bu$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A,B$ are (possibly nonlinear) differential operators that don't commute, a common numerical approach is to alternate between solving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t = Au$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t = Bu.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest implementation of this is known as Godunov splitting and is 1st-order accurate.  Another well-known approach, known as Strang splitting, is 2nd-order accurate.  Do higher order operator splitting methods (or alternative multiphysics discretization approaches) exist?&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-07-28T08:31:55.367" Title="Are there operator splitting approaches for multiphysics PDEs that achieve high order convergence?" Tags="|pde|multiphysics|operator-splitting|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="66" PostTypeId="2" ParentId="46" CreationDate="2011-11-30T11:12:42.827" Score="9" Body="&lt;p&gt;For testing graph partitioning algorithms, there is &lt;a href=&quot;http://staffweb.cms.gre.ac.uk/~wc06/partition/&quot;&gt;Walshaw's Graph Partitioning Archive&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2011-11-30T11:12:42.827" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="67" PostTypeId="2" ParentId="46" CreationDate="2011-11-30T11:14:05.210" Score="15" Body="&lt;p&gt;If you are interested in conducting an analysis on sparse matrices, I would also consider &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/matrices/&quot;&gt;Davis's University of Florida Sparse Matrix Collection&lt;/a&gt; and the &lt;a href=&quot;http://math.nist.gov/MatrixMarket&quot;&gt;Matrix Market&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="739" LastEditDate="2012-03-30T02:09:02.320" LastActivityDate="2012-03-30T02:09:02.320" CommentCount="1" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="68" PostTypeId="5" CreationDate="2011-11-30T11:35:50.780" Score="0" Body="&lt;p&gt;Attached is an excerpt from &lt;a href=&quot;http://cms.mpi.univie.ac.at/marsweb/index.php?option=com_content&amp;amp;view=article&amp;amp;id=59&amp;amp;Itemid=68&quot; rel=&quot;nofollow&quot;&gt;the VASP site&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VASP computes an approximate solution to the many-body Schrödinger equation, either within density functional theory (DFT), solving the Kohn-Sham equations, or within the Hartree-Fock (HF) approximation, solving the Roothaan equations. Hybrid functionals that mix the Hartree-Fock approach with density functional theory are implemented as well. Furthermore, Green's functions methods (GW quasiparticles, and ACFDT-RPA) and many-body perturbation theory (2nd-order Møller-Plesset) are available in VASP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In VASP, central quantities, like the one-electron orbitals, the electronic charge density, and the local potential are expressed in plane wave basis sets. The interactions between the electrons and ions are described using norm-conserving or ultrasoft pseudopotentials, or the projector-augmented-wave method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To determine the electronic groundstate, VASP makes use of efficient iterative matrix diagonalisation techniques, like the residual minimisation method with direct inversion of the iterative subspace (RMM-DIIS) or blocked Davidson algorithms. These are coupled to highly efficient Broyden and Pulay density mixing schemes to speed up the self-consistency cycle.&lt;/p&gt;&#xA;" OwnerUserId="80" LastEditorUserId="80" LastEditDate="2011-11-30T15:25:41.503" LastActivityDate="2011-11-30T15:25:41.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="69" PostTypeId="4" CreationDate="2011-11-30T11:35:50.780" Score="0" Body="The Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles." OwnerUserId="80" LastEditorUserId="80" LastEditDate="2011-11-30T15:25:43.860" LastActivityDate="2011-11-30T15:25:43.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="70" PostTypeId="2" ParentId="59" CreationDate="2011-11-30T12:02:21.410" Score="7" Body="&lt;p&gt;The short answer is: it requires specific work for different equations, but there are some general techniques that suggest how to do it.  Essentially, given a first order evolution PDE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t = Au + Bu$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A,B$ are some (possibly differential) operators, the steady states are those for which &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Au + Bu=0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is common to use a splitting approach in which $A$ and $B$ are discretized in different ways.  Then there will be truncation errors associated with each of these discretizations, and the truncation errors will generally not cancel even in the case of a steady state.  A classical example (as mentioned in the question) is the shallow water equations with bathymetry, in which $A$ represents convective terms and $B$ represents the momentum forcing due to the variable bottom height.  There are many papers published in the last few years that give different ways to exactly maintain the steady state solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach that I like is the use of f-wave Riemann solvers &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.115.7930&amp;amp;rep=rep1&amp;amp;type=pdf&quot;&gt;as proposed by Bale et. al.&lt;/a&gt;.  The idea is to discretize the convective terms with a Godunov-type method, but to subtract off the contribution from the other terms &lt;em&gt;inside&lt;/em&gt; the Riemann solver.  Then in the case of the steady state, no waves are generated.  However, this requires that the convective and source terms be calculated exactly (so as to cancel exactly).  That's possible for the shallow water equations, but more difficult for many other systems.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2011-11-30T12:02:21.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="71" PostTypeId="5" CreationDate="2011-11-30T12:42:29.177" Score="0" Body="&lt;p&gt;Hyperbolic partial differential equations describe time-dependent wave behavior.  Examples of applications modeled by hyperbolic PDEs include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fluid dynamics&lt;/li&gt;&#xA;&lt;li&gt;Water waves&lt;/li&gt;&#xA;&lt;li&gt;Acoustic waves&lt;/li&gt;&#xA;&lt;li&gt;Electromagnetic waves&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Nonlinear hyperbolic PDEs pose a unique computational challenge because of the tendency of discontinuities (shocks) to appear even in solutions with smooth initial data.  Numerical methods deal with these discontinuities by using limiters (to avoid oscillation in the computation of derivatives) and Riemann solvers (local solutions based on piecewise constant discontinuous data).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-11-30T15:25:35.280" LastActivityDate="2011-11-30T15:25:35.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="72" PostTypeId="4" CreationDate="2011-11-30T12:42:29.177" Score="0" Body="Hyperbolic partial differential equations describe wave behavior." OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-11-30T15:25:37.387" LastActivityDate="2011-11-30T15:25:37.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="73" PostTypeId="2" ParentId="45" CreationDate="2011-11-30T13:46:56.137" Score="4" Body="&lt;p&gt;The above answers apply to time-dependent problems, but you could also demand positivity in a simple elliptic equation. In this case, you could formulate it as a &lt;a href=&quot;http://en.wikipedia.org/wiki/Variational_inequality&quot; rel=&quot;nofollow&quot;&gt;variational inequality&lt;/a&gt;, giving bounds for the variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In PETSc, there are two VI solvers. One uses a reduced-space method, where variables in active constraints are removed from the system to be solved. The other uses a &lt;a href=&quot;http://epubs.siam.org/ebooks/siam/mos-siam_series_on_optimization/mo11&quot; rel=&quot;nofollow&quot;&gt;semi-smooth Newton method&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-11-30T13:46:56.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="74" PostTypeId="1" AcceptedAnswerId="1705" CreationDate="2011-11-30T14:05:53.540" Score="38" ViewCount="2367" Body="&lt;p&gt;We know that $\mathbf A$ is symmetric and positive-definite. We know that $\mathbf B$ is orthogonal:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: is $\mathbf B \cdot\mathbf A \cdot\mathbf B^\top$ symmetric and positive-definite? &#xA;Answer: Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: Could a computer have told us this?&#xA;Answer: Probably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any symbolic algebra systems (like Mathematica) that handle and propagate known facts about matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: To be clear I'm asking this question about abstractly defined matrices. I.e. I don't have explicit entries for $A$ and $B$, I just know that they are both matrices and have particular attribues like symetric, positive definite, etc.... &lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-03-17T15:44:42.997" LastActivityDate="2012-10-26T00:50:47.277" Title="Symbolic software packages for Matrix expressions?" Tags="|matrix|symbolic-computation|" AnswerCount="6" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="75" PostTypeId="2" ParentId="40" CreationDate="2011-11-30T14:25:50.317" Score="4" Body="&lt;p&gt;The H-matrix stuff from Jack Poulson and Lexing Ying is the most efficient method I know. This should be released in the spring but they have given presentations on it.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-11-30T14:25:50.317" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="76" PostTypeId="2" ParentId="65" CreationDate="2011-11-30T14:30:18.113" Score="7" Body="&lt;p&gt;It was my understanding that the &lt;a href=&quot;http://en.wikipedia.org/wiki/Baker%E2%80%93Campbell%E2%80%93Hausdorff_formula&quot;&gt;BCH&lt;/a&gt; formula was a systematic way to approximate the matrix exponential of two non-commutative matrices.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-11-30T14:30:18.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="77" PostTypeId="2" ParentId="74" CreationDate="2011-11-30T14:35:35.193" Score="2" Body="&lt;p&gt;In Mathematica you can at least check these properties for specific matrices.  For example, the matrix &lt;code&gt;A&lt;/code&gt; as you described:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[1]:= A = {{2.0,-1.0,0.0},{-1.0,2.0,-1.0},{0.0,-1.0,2.0}};&#xA;        {SymmetricMatrixQ[A],PositiveDefiniteMatrixQ[A]}&#xA;Out[2]= {True,True}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For matrix &lt;code&gt;B&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[3]:= B = {{0, -0.80, -0.60}, {0.80, -0.36, 0.48}, {0.60, 0.48, -0.64}};&#xA;        Transpose[B] == Inverse[B]&#xA;Out[4]= True&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[5]:= c = B.A.Transpose[B];&#xA;        {SymmetricMatrixQ[c],PositiveDefiniteMatrixQ[c]}&#xA;Out[6]= {True,True}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://reference.wolfram.com/mathematica/guide/MatricesAndLinearAlgebra.html&quot; rel=&quot;nofollow&quot;&gt;Mathematica Matrices and Linear Algebra Documentation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="62" LastEditorUserId="123" LastEditDate="2011-12-03T18:13:50.690" LastActivityDate="2011-12-03T18:13:50.690" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="78" PostTypeId="1" AcceptedAnswerId="644" CreationDate="2011-11-30T14:44:51.663" Score="16" ViewCount="1091" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Automatic_differentiation&quot;&gt;Automatic differentiation&lt;/a&gt; allows us to numerically evaluate the derivative of a program on a particular input. There is a theorem that this computation can done at a cost less than five times the cost to run the original program. This factor of five is an upper bound. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In what situations can this cost be further reduced? Many in-the-field derivative codes run at near the speed of the original program. What is done to obtain this speed-up? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are traits of the original program that can be exploited to speed up computation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What software engineering tricks can be employed to speed up computation?&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2011-11-30T14:51:30.677" LastActivityDate="2021-11-01T15:41:39.233" Title="When is automatic differentiation cheap?" Tags="|automatic-differentiation|efficiency|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="79" PostTypeId="5" CreationDate="2011-11-30T14:55:31.877" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-11-30T14:55:31.877" LastActivityDate="2011-11-30T14:55:31.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="80" PostTypeId="4" CreationDate="2011-11-30T14:55:31.877" Score="0" Body="Ordinary Differential Equations (ODEs) contain functions of only one independent variable, and one or more of their derivatives with respect to that variable. This tag is intended for questions on modeling phenomena with ODEs, solving ODEs, and other related aspects." OwnerUserId="116" LastEditorUserId="127" LastEditDate="2011-11-30T19:34:26.137" LastActivityDate="2011-11-30T19:34:26.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="81" PostTypeId="1" AcceptedAnswerId="84" CreationDate="2011-11-30T15:02:52.630" Score="58" ViewCount="9256" Body="&lt;p&gt;Sparse linear systems turn up with increasing frequency in applications. One has a lot of routines to choose from for solving these systems. At the highest level, there is a watershed between direct (e.g. sparse Gaussian elimination or Cholesky decomposition, with special ordering algorithms, and multifrontal methods) and iterative (e.g. GMRES, (bi-)conjugate gradient) methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does one determine whether to use a direct or an iterative method? Having made that choice, how does one pick a particular algorithm? I already know about the exploitation of symmetry (e.g. use conjugate gradient for a sparse symmetric positive definite system), but are there any other considerations like this to be considered in picking a method?&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="4244" LastEditDate="2013-07-24T06:55:16.823" LastActivityDate="2022-10-02T22:45:10.770" Title="What guidelines should I follow when choosing a sparse linear system solver?" Tags="|linear-algebra|linear-solver|sparse-matrix|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="82" PostTypeId="2" ParentId="78" CreationDate="2011-11-30T15:08:58.797" Score="3" Body="&lt;p&gt;Any AD still needs these intrinsics to be provided, so I can't see what that has to do with generic complexity of an expression. I am guessing you can classify complexity by the number of paths through the expression graph since you phrase AD this way. Andrew Lyons has good work on series-parallel graphs here.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-11-30T15:08:58.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="83" PostTypeId="1" AcceptedAnswerId="101" CreationDate="2011-11-30T15:10:04.833" Score="97" ViewCount="132801" Body="&lt;p&gt;I have several challenging non-convex global optimization problems to solve.  Currently I use &lt;strong&gt;MATLAB's Optimization Toolbox (specifically, &lt;code&gt;fmincon()&lt;/code&gt; with algorithm=&lt;code&gt;'sqp'&lt;/code&gt;), which is quite effective&lt;/strong&gt;.  However, most of my code is in Python, and I'd love to do the optimization in Python as well.  Is there a NLP solver with Python bindings that can compete with &lt;code&gt;fmincon()&lt;/code&gt;?  It must &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;be able to handle nonlinear equality and inequality constraints &lt;/li&gt;&#xA;&lt;li&gt;not require the user to provide a Jacobian.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's okay if it doesn't guarantee a global optimum (&lt;code&gt;fmincon()&lt;/code&gt; does not). I'm looking for something that robustly converges to a local optimum even for challenging problems, and even if it's slightly slower than &lt;code&gt;fmincon()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried several of the solvers available through OpenOpt and found them to be inferior to MATLAB's &lt;code&gt;fmincon/sqp&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Just for emphasis&lt;/strong&gt; I already have a tractable formulation and a good solver.  My goal is merely to change languages in order to have a more streamlined workflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Geoff points out that some characteristics of the problem may be relevant.  They are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;10-400 decision variables&lt;/li&gt;&#xA;&lt;li&gt;4-100 polynomial equality constraints (polynomial degree ranges from 1 to about 8)&lt;/li&gt;&#xA;&lt;li&gt;A number of rational inequality constraints equal to about twice the number of decision variables&lt;/li&gt;&#xA;&lt;li&gt;The objective function is one of the decision variables&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The Jacobian of the equality constraints is dense, as is the Jacobian of the inequality constraints.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-12-13T05:49:12.210" LastActivityDate="2022-08-16T09:35:08.667" Title="Is there a high quality nonlinear programming solver for Python?" Tags="|python|optimization|nonlinear-programming|" AnswerCount="19" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="84" PostTypeId="2" ParentId="81" CreationDate="2011-11-30T15:17:03.490" Score="42" Body="&lt;p&gt;The important thing when choosing iterative solvers is the spectrum of the operator, see &lt;a href=&quot;http://ecommons.cornell.edu/handle/1813/5577&quot;&gt;this paper&lt;/a&gt;. However, there are so many negative results, see &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/publication/PDF/1992_52.pdf&quot;&gt;this paper&lt;/a&gt; where no iterative solver wins for all problems and &lt;a href=&quot;http://dx.doi.org/10.1137/S0895479894275030&quot;&gt;this paper&lt;/a&gt; in which they prove they can get any convergence curve for GMRES for any spectrum. Thus, it seems impossible to predict the behavior of iterative solvers except in a few isolated cases, Therefore, your best option is to try them all, using a system like &lt;a href=&quot;http://www.mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt;, which also has direct solvers.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="127" LastEditDate="2011-11-30T15:25:31.583" LastActivityDate="2011-11-30T15:25:31.583" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="85" PostTypeId="2" ParentId="3" CreationDate="2011-11-30T15:26:46.807" Score="16" Body="&lt;p&gt;&lt;strong&gt;Counterpoise correction&lt;/strong&gt; is a method to limit an error that results when studying an &lt;em&gt;intermolecular&lt;/em&gt; reaction using an incomplete basis set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically a basis set is not converged, and a calculation could always be improved with more basis functions. This is especially true for long range interactions, ie one often needs to add diffuse functions to the set to account for the small about of electron density well away from the atomic centers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When studying bond breaking/forming between two separate reactants, a &lt;em&gt;basis set superposition error&lt;/em&gt; (BSSE) can occur. The basis set localized on one reactant can act as diffuse functions for the electrons from the other reactant, and vise versa. The error is largest at some problem-specific intermediate range. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to correct this is to use a larger and larger basis set. If one uses a sufficiently accurate description of the atomic orbitals far away from the atomic centers (more and more diffuse functions in a traditional calculation), then it wont matter if additional basis functions (from another reactants atomic orbital description) occupy that same long range region. The added basis functions from the other reactant are unnecessary and wont improve the quality of the calculation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not always possible to use a larger basis set, because it's often too computationally expensive to increase the basis set. Alternatively, one can calculate a &lt;strong&gt;counterpoise correction&lt;/strong&gt;, which approximates the bias to the quality of the calculation that results in the intermediate range. To get the corrected energy involves three steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Calculate the energy with both reactants, including all electrons and nuclei. This results in the energy of the complex of reactant 1 and reactant 2: $W_{12}$&lt;/li&gt;&#xA;&lt;li&gt;Repeat the calculation for each reactant, on their own, using the same geometry they are in, in the complex. This results in values $W_1$ and $W_2$&lt;/li&gt;&#xA;&lt;li&gt;Repeat the calculation for each reactant, but with a modified basis set: in addition to the individual reactant's basis set, the other reactant's basis is also used. For example, for reactant 1, these added basis functions are localized where reactant 2 is localized in the complex. These calculations do not include the nuclei or the electrons of the other reactant. This results in energies $W_1^{*}$ and $W_2^{*}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;counterpoise correction&lt;/strong&gt; is calculated from the energies of the individual reactants: $\Delta W_{c} = (W_1^*-W_1)+(W_2^*-W_2)$. This represents the lowering of the energy due to the addition of other reactant's basis set. Since the energy is lower or the same (with variational methods) with added basis functions, this value should be negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corrected &lt;em&gt;interaction&lt;/em&gt; energy is then $$\Delta W_{int} = W_{12}-W_1-W_2-\Delta W_c = W_{12} - W_1^*-W_2^*$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why does this matter?&lt;/strong&gt; This correction will depend on the geometries of the reactants. When they are very far from one another, it will be very small: they don't influence one another. When they are very close, this effect will be small, for the same reasoning. It's the intermediate distances that have the largest BSSE. These are the distances at or approaching the transition state, which serves as the bottleneck for the reaction. If you are not accounting for the artificial improvement near the transition state, then you will get an incorrect approximation of the activation energy, the energy difference between this transition state and the separated-reactant limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some questions to think about are: &lt;strong&gt;How can this be done for &lt;em&gt;intramolecular&lt;/em&gt; reactions? Is this even important in those cases?&lt;/strong&gt; Professor David Sherrill addresses these questions and other complicated cases in a freely available, &lt;a href=&quot;http://vergil.chemistry.gatech.edu/notes/cp.pdf&quot; rel=&quot;noreferrer&quot;&gt;self-published document&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="79" LastEditorUserId="28121" LastEditDate="2018-07-03T16:35:18.100" LastActivityDate="2018-07-03T16:35:18.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="86" PostTypeId="5" CreationDate="2011-11-30T15:27:15.633" Score="0" Body="&lt;p&gt;By definition numerics refers to the investigation of numeric patterns, e.g. &lt;a href=&quot;http://en.wikipedia.org/wiki/Bible_Numerics&quot; rel=&quot;nofollow&quot;&gt;in the bible&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-09-04T11:31:05.257" LastActivityDate="2013-09-04T11:31:05.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="87" PostTypeId="4" CreationDate="2011-11-30T15:27:15.633" Score="0" Body="Widely used as a synonym for numerical-analysis, in particular in the German speaking community. " OwnerUserId="127" LastEditorUserId="4244" LastEditDate="2013-09-04T11:30:19.847" LastActivityDate="2013-09-04T11:30:19.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="88" PostTypeId="5" CreationDate="2011-11-30T15:29:24.007" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-11-30T15:29:24.007" LastActivityDate="2011-11-30T15:29:24.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="89" PostTypeId="4" CreationDate="2011-11-30T15:29:24.007" Score="0" Body="The Protein Data Bank (*.pdb) format is a file format usually used for the computer representation of peptides and proteins." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-11-30T15:36:06.210" LastActivityDate="2011-11-30T15:36:06.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="90" PostTypeId="5" CreationDate="2011-11-30T15:32:31.427" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-11-30T15:32:31.427" LastActivityDate="2011-11-30T15:32:31.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="91" PostTypeId="4" CreationDate="2011-11-30T15:32:31.427" Score="0" Body="Interpolation is the process of estimating the values of a function, when the function's values are known only at a particular set of points. Questions on interpolation in one or more dimensions, as well as algorithms for doing so, should have this tag." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-11-30T15:36:03.073" LastActivityDate="2011-11-30T15:36:03.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="92" PostTypeId="5" CreationDate="2011-11-30T15:40:20.593" Score="0" Body="&lt;p&gt;C is a general purpose programming language. Originally intended for implementing system software, C and its close relatives are the principle languages used in most modern operating systems. Due to its thin layer of abstraction and low overhead, C allows efficient coding of algorithms and data structures useful for scientific computation. Use this tag for questions involving the C programming language.&lt;/p&gt;&#xA;" OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-11T22:39:13.737" LastActivityDate="2021-11-11T22:39:13.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="93" PostTypeId="4" CreationDate="2011-11-30T15:40:20.593" Score="0" Body="For questions related to computational software and libraries written in the C programming language" OwnerUserId="116" LastEditorUserId="23791" LastEditDate="2021-11-11T22:39:26.797" LastActivityDate="2021-11-11T22:39:26.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="95" PostTypeId="5" CreationDate="2011-11-30T16:03:26.967" Score="0" Body="&lt;p&gt;Questions on subjects like linear and nonlinear programming can also have this tag.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-11-30T19:34:32.063" LastActivityDate="2011-11-30T19:34:32.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="96" PostTypeId="4" CreationDate="2011-11-30T16:03:26.967" Score="0" Body="This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions." OwnerUserId="127" LastEditorUserId="4244" LastEditDate="2013-05-20T22:08:07.997" LastActivityDate="2013-05-20T22:08:07.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="97" PostTypeId="2" ParentId="8" CreationDate="2011-11-30T16:15:10.623" Score="4" Body="&lt;p&gt;To start with your &quot;related question&quot;: I do PDB parsing in Python, even when the subsequent processing needs to be done in some compiled language for speed. C is simply not a good language for parsing, in particular not for messy formats like PDB. And that's probably why there aren't any stable and mature PDB parsers in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something else you can consider, assuming your goal is to work with structures from the PDB, is to use the mmCIF format rather than the PDB format. mmCIF is much more regular and thus easier to parse. You also get more parseable information from the mmCIF version of the PDB entries than from the PDB versions. You can find a C parser for mmCIF on the PDB Web site (it's called CIFPARSE), although you will note that its development has been dropped in favor of a C++ rewrite called CIFPARSE-OBJ.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-11-30T16:15:10.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="98" PostTypeId="2" ParentId="38" CreationDate="2011-11-30T16:16:15.363" Score="18" Body="&lt;p&gt;The central question is which physical processes (waves or source terms) have time scales that you are interested in resolving and which you would prefer to step over. If you are not interested in the fastest time scale in the system, then the equations are called &quot;stiff&quot;. Hyperbolic conservation laws are typically written as first-order systems&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_t + \nabla\cdot F(u) = G(u,\nabla u,...) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ contains conserved variables, $F$ is the flux, and $G$ is called the &quot;source term&quot;. Note that with this terminology, the flux $F$ does not contain derivatives, therefore diffusive and dispersive terms must go in $G$. It is quite common to use implicit or semi-implicit integration when source terms are stiff, as with many chemical reaction problems and when diffusion or dispersion is present. Chemical reaction can usually be implicitly solved locally in each element since it is not coupled to neighboring cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute wave speeds, we examine the eigenvalues of the flux Jacobian $A = [\partial F/\partial u]$. If we decide that the &lt;em&gt;phase&lt;/em&gt; of certain waves is not of physical interest, then we may want to step over them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you are simulating the long-time evolution of an ocean, you may not be interested in surface gravity waves (e.g. tsunamis). Unfortunately, changing the wave speed (either slowing it down to use explicit methods or speeding it up to a &quot;rigid lid&quot; model that can use a projection) changes the physics by changing the way vortices propagate. Vortices in the ocean are an effect where the gravity wave is almost balanced with convection, but not quite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example is compressible Euler, e.g. airflow through a data center. The acoustic wave speed is much faster than convection and only the latter is important for heat transfer. If you are not interested in acoustics, you may want to use an implicit method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relative efficiency of an implicit method is dependent on the cost to solve the algebraic systems at each step/stage compared to the step size that can be used with explicit methods. Solving such algebraic systems efficiently is an active topic of research. (Make another question and I will answer it and reference from here.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also want to use implicit methods if:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;your equations have meaningful steady states that you want to explore directly, perhaps with to characterize stability&lt;/li&gt;&#xA;&lt;li&gt;you are solving inverse/data assimilation problems involving long time history&lt;/li&gt;&#xA;&lt;li&gt;you want to circumvent order barriers to use very high order time integration methods with certain stability properties&lt;/li&gt;&#xA;&lt;li&gt;you are using space-time adaptive methods&lt;/li&gt;&#xA;&lt;li&gt;you are using a spatial discretization that already requires solving an algebraic system (e.g. continuous finite element methods with consistent mass matrix)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-11-30T16:16:15.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="99" PostTypeId="2" ParentId="46" CreationDate="2011-11-30T16:42:42.623" Score="13" Body="&lt;p&gt;The method of manufactured solutions is a standard for testing PDEs and other solvers.  Most symbolic algebra systems have facilities for generating code, this is useful for creating manufactured solutions.  SymPy and Maple have the function ccode, among others for this purpose.  &lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2011-11-30T16:42:42.623" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="100" PostTypeId="2" ParentId="37" CreationDate="2011-11-30T16:52:00.917" Score="13" Body="&lt;p&gt;In practice, most people stick to relatively low orders, usually first or second order.  This view is often challenged by more theoretical researchers that believe in more accurate answers .  The rate of convergence for simple smooth problems is well documented, for example see Bill Mitchell's &lt;a href=&quot;http://math.nist.gov/~WMitchell/papers/nistir7824.pdf&quot;&gt;comparison of hp adaptivity&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While for theoretical works it is nice to see what the convergence rate are, for more application oriented among us this concern is balanced with constitutive laws, necessary precision, and code complexity.  It doesn't make much since in many porous media problems that solve over a highly discontinuous media to have high order methods,  the numerical error will dominate the discretization errors.  The same concern applies for problems that include a large number of degrees of freedom.  Since low-order implicit methods have a smaller bandwidth and often a better conditioning, the high order method becomes too costly to solve.  Finally the code complexity of switching orders and types of polynomials is usually too much for the graduate students running the application codes.&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2011-11-30T16:52:00.917" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="101" PostTypeId="2" ParentId="83" CreationDate="2011-11-30T16:58:35.333" Score="38" Body="&lt;p&gt;&lt;code&gt;fmincon()&lt;/code&gt;, as you mentioned, employs several strategies that are well-known in nonlinear optimization that attempt to find a local minimum without much regard for whether the global optimum has been found.  If you're okay with this, then I think you have phrased the question correctly (nonlinear optimization).&lt;/p&gt;&#xA;&lt;p&gt;The best package I'm aware of for general nonlinear optimization is &lt;a href=&quot;https://projects.coin-or.org/Ipopt&quot; rel=&quot;nofollow noreferrer&quot;&gt;IPOPT&lt;/a&gt;[1].  Apparently &lt;a href=&quot;https://github.com/xuy/pyipopt&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matthew Xu maintains a set of Python bindings to IPOPT&lt;/a&gt;, so this might be somewhere to start.&lt;/p&gt;&#xA;&lt;p&gt;UPDATE: the current maintained Python bindings for IPOPT seems to be &lt;a href=&quot;https://pypi.org/project/ipyopt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ipyopt&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[1]: Andreas Wachter is a personal friend, so I may be a bit biased.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="25092" LastEditDate="2022-08-16T09:35:08.667" LastActivityDate="2022-08-16T09:35:08.667" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="102" PostTypeId="2" ParentId="24" CreationDate="2011-11-30T17:01:50.677" Score="1" Body="&lt;p&gt;It might be useful to look into Taylor model methods; &lt;a href=&quot;http://metronu.ulb.ac.be/imacs/papers/T2-I-102-0658.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; seems to be a nice overview article. Try if &lt;a href=&quot;http://bt.pa.msu.edu/index_cosy.htm&quot; rel=&quot;nofollow&quot;&gt;COSY infinity&lt;/a&gt; can do what you want.&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2011-11-30T17:01:50.677" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="103" PostTypeId="1" AcceptedAnswerId="108" CreationDate="2011-11-30T17:24:35.207" Score="9" ViewCount="784" Body="&lt;p&gt;Are there any good papers and or codes that couple discontinuous galerkin finite element solvers with Riemann solvers?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to explore coupling elliptic and hyperbolic problems but most splitting methods are ad hoc at best.  Since I have a large amount of FEniCS code, I would like to just couple the Riemann solver with it.  While a simple Roe solver would be a beginning, I'm looking for guidance on using more complicated methods.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="276" LastEditDate="2012-01-06T20:05:14.890" LastActivityDate="2016-07-06T02:14:30.653" Title="Coupling FEM DG methods to Riemann solvers" Tags="|pde|finite-element|hyperbolic-pde|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="104" PostTypeId="1" AcceptedAnswerId="125" CreationDate="2011-11-30T17:32:43.020" Score="25" ViewCount="11298" Body="&lt;p&gt;There are a number of different libraries out there that solve a sparse linear system of equations, however I'm finding it difficult to figure out what the differences are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell there are three major packages:  &lt;a href=&quot;http://trilinos.sandia.gov/&quot; rel=&quot;noreferrer&quot;&gt;Trilinos&lt;/a&gt;, &lt;a href=&quot;http://www.mcs.anl.gov/petsc&quot; rel=&quot;noreferrer&quot;&gt;PETSc&lt;/a&gt;, and &lt;a href=&quot;https://software.intel.com/en-us/intel-mkl&quot; rel=&quot;noreferrer&quot;&gt;Intel MKL&lt;/a&gt;.  They can all do sparse matrix solves, they are all fast (as far as I can tell, I haven't been able to find solid benchmarks on any of them), and they are all parallelizable.  What I can't find is the differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what are the differences between the different sparse linear system solvers out there?&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="4780" LastEditDate="2015-10-15T00:08:23.537" LastActivityDate="2015-10-15T00:08:23.537" Title="Libraries for solving sparse linear systems" Tags="|linear-algebra|linear-solver|libraries|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="105" PostTypeId="2" ParentId="83" CreationDate="2011-11-30T17:34:05.673" Score="5" Body="&lt;p&gt;What about scipy.fmin_slsqp ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html&quot; rel=&quot;noreferrer&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="51" LastActivityDate="2011-11-30T17:34:05.673" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="106" PostTypeId="2" ParentId="46" CreationDate="2011-11-30T17:35:14.497" Score="6" Body="&lt;p&gt;We use weather data sets in our building energy simulation software. For the US, the data sets consist of weather observations taken (usually at airports) every hour for the preceeding 20 years. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://apps1.eere.energy.gov/buildings/energyplus/cfm/weather_data.cfm&quot;&gt;Data sets available for download&lt;/a&gt;.&lt;br&gt;&#xA;&lt;a href=&quot;http://www.nrel.gov/docs/fy08osti/43156.pdf&quot;&gt;Manual to describe the file format&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="16" LastActivityDate="2011-11-30T17:35:14.497" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="107" PostTypeId="2" ParentId="104" CreationDate="2011-11-30T17:47:22.747" Score="9" Body="&lt;p&gt;MKL does not do distributed parallelism (e.g. MPI), and the support for sparse solvers is rudimentary, definitely not at the level of the other two. Currently, there is only one meaningful benchmark: scalable performance of Sparse Matrix-Vector product (SpMV). Since this is memory bandwidth limited, you can only screw it up. Both PETSc and Trilinos do fine on this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real difference is which programming environment makes you more productive.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-11-30T17:47:22.747" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="108" PostTypeId="2" ParentId="103" CreationDate="2011-11-30T17:56:15.000" Score="5" Body="&lt;p&gt;I suggest looking at the literature on &lt;strong&gt;DG methods for incompressible flow&lt;/strong&gt;, which has the mixed hyperbolic-elliptic character you mention.  There are a lot of approaches.  &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045793007000400#appd001&quot; rel=&quot;nofollow&quot;&gt;This paper&lt;/a&gt;, for instance, even uses an exact Riemann solver.  &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999100964751&quot; rel=&quot;nofollow&quot;&gt;This one&lt;/a&gt; suggests using a discontinuous space for the hyperbolic part and a continuous one for the elliptic part.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-11-30T18:32:50.730" LastActivityDate="2011-11-30T18:32:50.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="109" PostTypeId="5" CreationDate="2011-11-30T18:23:31.060" Score="0" Body="&lt;p&gt;HPC systems usually come in 3 varieties:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Supercomputers.  &lt;/li&gt;&#xA;&lt;li&gt;Cluster computers, sometimes called &lt;a href=&quot;http://en.wikipedia.org/wiki/Beowulf_%28computing%29&quot; rel=&quot;nofollow&quot;&gt;Beowulf clusters&lt;/a&gt; are generally built out of a number of regular computers in order to get the throughput of supercomputers. &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Grid_computing&quot; rel=&quot;nofollow&quot;&gt;Grid computers&lt;/a&gt;, are also generally built out of a number of regular computers.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The main difference between Grids and Clusters have to do with the communication between computing nodes (a node will be a CPU core). If there will be a lot of communication, one generally calls it a cluster. In contrast, Grids tend to have little communication between nodes.&lt;/p&gt;&#xA;" OwnerUserId="16" LastEditorUserId="16" LastEditDate="2011-11-30T19:34:14.700" LastActivityDate="2011-11-30T19:34:14.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="110" PostTypeId="4" CreationDate="2011-11-30T18:23:31.060" Score="0" Body="High Performance Computing generally uses large arrays of computers connected in clusters or grids." OwnerUserId="16" LastEditorUserId="16" LastEditDate="2011-11-30T19:34:17.540" LastActivityDate="2011-11-30T19:34:17.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="111" PostTypeId="2" ParentId="103" CreationDate="2011-11-30T19:34:23.527" Score="5" Body="&lt;p&gt;As with many high order methods, the accuracy of the scheme is often less sensitive to the Riemann solver. None of the DG papers for hyperbolic problems will actually be using averages, however. The most common choice is a Rusanov (aka. Local Lax-Friedrichs) flux, which is very simple if you have an upper bound for the fastest wave speed.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-11-30T19:34:23.527" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="112" PostTypeId="1" AcceptedAnswerId="113" CreationDate="2011-11-30T19:58:35.353" Score="8" ViewCount="956" Body="&lt;p&gt;It is quite painful to discover that a few-day long job is going to be prematurely killed due to an error in setting walltime limit for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to change it for a running &lt;a href=&quot;http://en.wikipedia.org/wiki/Portable_Batch_System&quot; rel=&quot;nofollow&quot;&gt;PBS&lt;/a&gt; job?&lt;/p&gt;&#xA;" OwnerUserId="102" LastEditorUserId="123" LastEditDate="2012-08-11T08:58:45.913" LastActivityDate="2015-07-08T16:45:08.643" Title="Prolonging PBS job" Tags="|hpc|pbs|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="113" PostTypeId="2" ParentId="112" CreationDate="2011-11-30T20:03:45.463" Score="9" Body="&lt;p&gt;This is system-specific, but you are looking for the &lt;code&gt;qalter&lt;/code&gt; command.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2011-11-30T20:03:45.463" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="114" PostTypeId="1" AcceptedAnswerId="119" CreationDate="2011-11-30T20:10:35.450" Score="48" ViewCount="6343" Body="&lt;p&gt;In my numerical analysis courses, I learned to analyze the efficiency of algorithms by counting the number of floating-point operations (flops) they require, relative to the size of the problem.  For instance, in Trefethen &amp;amp; Bau's text on Numerical Linear Algebra, there are even 3D-looking pictures of the flop counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now it's fashionable to say that &quot;flops are free&quot; because the memory latency to fetch anything not in cache is so much greater than the cost of a flop.  But we're still teaching students to count flops, at least in numerical analysis courses.  Should we be teaching them to count memory accesses instead?  Do we need to write new textbooks?  Or is memory access too machine-specific to spend time on?  What is the long-term trend going to be in terms of whether flops or memory access is the bottleneck?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: some of the answers below seem to be answering a different question like &quot;Should I obsessively rewrite my implementation to save a few flops or improve cache performance?&quot;  But what I'm asking is more along the lines of &quot;&lt;strong&gt;Is it more useful to estimate algorithmic complexity in terms of arithmetic operations or memory accesses&lt;/strong&gt;?&quot;&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-01-09T05:37:38.500" LastActivityDate="2014-04-07T22:13:45.710" Title="Is algorithmic analysis by flop-counting obsolete?" Tags="|algorithms|performance|complexity|education|architecture|" AnswerCount="7" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="115" PostTypeId="5" CreationDate="2011-11-30T20:36:08.157" Score="0" Body="&lt;p&gt;From the Gaussian 03 Product information Page:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Gaussian 03 is used by chemists, chemical engineers, biochemists, physicists and others for research in established and emerging areas of chemical interest.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Starting from the basic laws of quantum mechanics, Gaussian predicts the energies, molecular structures, and vibrational frequencies of molecular systems, along with numerous molecular properties derived from these basic computation types. It can be used to study molecules and reactions under a wide range of conditions, including both stable species and compounds which are difficult or impossible to observe experimentally such as short-lived intermediates and transition structures. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Questions using this tag should address how to accomplish specific calculations using this program and version. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information about this program, visit &lt;a href=&quot;http://www.gaussian.com/&quot; rel=&quot;nofollow&quot;&gt;http://www.gaussian.com/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="79" LastEditorUserId="79" LastEditDate="2011-11-30T21:08:24.260" LastActivityDate="2011-11-30T21:08:24.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="116" PostTypeId="4" CreationDate="2011-11-30T20:36:08.157" Score="0" Body="Gaussian is a computational chemistry, electronic structure software package produced by Gaussian, Inc. Gaussian 03 refers to the 2003 release, including all subsequent updates. " OwnerUserId="79" LastEditorUserId="79" LastEditDate="2011-11-30T21:08:31.000" LastActivityDate="2011-11-30T21:08:31.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="117" PostTypeId="5" CreationDate="2011-11-30T20:59:00.343" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-11-30T20:59:00.343" LastActivityDate="2011-11-30T20:59:00.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="118" PostTypeId="4" CreationDate="2011-11-30T20:59:00.343" Score="0" Body="Issues related to the representation of numerical quantities in a finite representation in a given base differing from their exact mathematical value." OwnerUserId="63" LastEditorUserId="63" LastEditDate="2011-11-30T21:08:34.980" LastActivityDate="2011-11-30T21:08:34.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="119" PostTypeId="2" ParentId="114" CreationDate="2011-11-30T21:36:46.630" Score="32" Body="&lt;p&gt;I think the (first order) right thing to do is look at the ratio of flops to bytes needed in the algorithm, which I call $\beta$. Let $F_{\mathrm{max}}$ be the maximum flop rate of the processor, and $B_{\mathrm{max}}$ the maximum bandwidth. If $\frac{F_{\mathrm{max}}}{\beta} &amp;gt; B_{\mathrm{max}}$, then the algorithm will be bandwidth limited. If $B_{\mathrm{max}}\beta &amp;gt; F_{\mathrm{max}}$, the algorithm is flop limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think counting memory accesses is mandatory, but we should also be thinking about:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How much local memory is required&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How much possible concurrency we have&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then you can start to analyze algorithms for modern hardware.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-11-30T21:36:46.630" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="120" PostTypeId="2" ParentId="112" CreationDate="2011-11-30T21:40:38.497" Score="2" Body="&lt;p&gt;Contact the IT administrators over your cluster. I know of people that have had other problems, such as jobs that won't start. Once our IT admin's found out, they took care of it. &lt;/p&gt;&#xA;" OwnerUserId="56" LastActivityDate="2011-11-30T21:40:38.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="121" PostTypeId="2" ParentId="2" CreationDate="2011-11-30T21:42:46.997" Score="8" Body="&lt;p&gt;No, there are several cases where the approximation becomes unphysical and inaccurate. To name a few I'm aware of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;excited molecule states, basis functions are typically optimized to describe ground states. &lt;em&gt;Configuration Interaction (CI)&lt;/em&gt; methods are rather used here. HF covers ground states best.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;electron correlation, especially if correlation changes with internuclear separation. HF assumes independent electrons. There are Post-HF-Methods for taking this correlation into account based on ordinary HF, e.g. &lt;em&gt;Møller-Plesset many-body perturbation theory&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For not small to medium-sized molecules (&gt;20 Atoms) less time-consuming &lt;em&gt;semi-empirical&lt;/em&gt;  or hybrid (HF+DFT) instead of pure HF methods are used. &lt;/p&gt;&#xA;" OwnerUserId="138" LastActivityDate="2011-11-30T21:42:46.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="122" PostTypeId="2" ParentId="8" CreationDate="2011-11-30T21:45:32.200" Score="0" Body="&lt;p&gt;You might also consider another &quot;language&quot; like &lt;a href=&quot;http://cns-online.org/v1.3/&quot; rel=&quot;nofollow&quot;&gt;CNS&lt;/a&gt;, which is specifically designed for working with molecular structures and has the functions you mentioned.&lt;/p&gt;&#xA;" OwnerUserId="56" LastActivityDate="2011-11-30T21:45:32.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="123" PostTypeId="2" ParentId="83" CreationDate="2011-11-30T22:17:18.253" Score="4" Body="&lt;p&gt;There's &lt;a href=&quot;http://cvxmod.net/&quot; rel=&quot;nofollow&quot;&gt;cvxmod&lt;/a&gt;, a Python wrapper around Stephen Boyd's convex optimization software. It's part of the &lt;a href=&quot;http://sagemath.org/&quot; rel=&quot;nofollow&quot;&gt;Sage&lt;/a&gt; package.&lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2011-11-30T22:17:18.253" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="124" PostTypeId="1" CreationDate="2011-11-30T22:30:29.197" Score="6" ViewCount="186" Body="&lt;p&gt;I am modelling a heat distribution in optical element irradiated by laser. System is radially symmetric, and element is thin, i.e. heat value depends only on distance from center. Heat is received via laser radiation and is removed from upper and lower sides of the element. A significant characteristic of the problem is that heat absorbtion coefficient depends on the current temperature of an element in a complex periodic way. I'm researching the stationary distributions by running simulation until equilibrium appears. An interesting feature of a model is that there are several stationary states possible, depending on initial distribution (which is a consequence of heat absorbtion depending on current temperature). So, the system is described by parabolic partial differential equation with some coefficients depending on current temperature. I solve it with implicit numerical scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the question is: if parameters of a system do not change in time, is it possible to get not a stationary distribution, but distribution changing in time (possibly, oscillating around some state). All initial distributions which I've tried after some time developed into one or several possible for given parameters stationary distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;This is the PDE of the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xD;&#xA;\begin{aligned}&#xD;&#xA;C\frac{\partial u}{\partial t} = k\left(\frac{\partial^2 u}{\partial r^2} + \frac{1}{r}\frac{\partial u}{\partial r}\right) - \frac{\alpha}{l}u + \frac{A(u)}{l}I(r),\\r \in (0,R), \ t \in (0,T \ ];&#xD;&#xA;\\&#xD;&#xA;C\frac{\partial u}{\partial t} = 2u\frac{\partial^2 u}{\partial r^2} - \frac{\alpha}{l}u + \frac{A(u)}{l}I(r),\\ r=0;&#xD;&#xA;\\&#xD;&#xA;\left. \frac{\partial u}{\partial r} \right|_{r=R}=0;&#xD;&#xA;\\&#xD;&#xA;\left. \frac{\partial u}{\partial r} \right|_{r=0}=0;&#xD;&#xA;&#xD;&#xA;\end{aligned}&#xD;&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\xi=kl$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n=n_0+\alpha_n u$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$l=l_0(1+\alpha_l u)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k=\frac{2\pi n}{\alpha}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A(n,l)=4\chi k l \frac{1+n^2}{4n^2+(1-n^2)^2\sin^2\xi}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$I(r)=\frac{P}{\pi a^2}\exp(-\frac{r^2}{a^2})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\left. u\right|_{t=0}=0$ is the initial condition, then the system evolves into some &quot;minimal&quot; stationary distribution. Then it is possible to increase the laser power P, wait and decrease it back. Then, depending on coefficients, the system can evolve into new stationary state. The reason is the nonconstant and periodic radiation absorbtion function A which depends on current temperature of the element. This leads to equilibrium between heat absorbtion and removal for several distributions, not one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, while nonuniquiness of stationary distributions is the proven fact, I failed to achieve oscillating distribution by the simulation. But I cannot be sure that it doesn't exist - that's why I asked the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should mention that I don't know anything about solving this PDE (with nonlinear A) in exact way, so numerical methods are the only way I can use.&lt;/p&gt;&#xA;" OwnerUserId="90" LastEditorUserId="90" LastEditDate="2011-12-05T02:44:58.243" LastActivityDate="2011-12-07T08:22:53.307" Title="Can heat distribution in an optical element irradiated by laser be oscillating?" Tags="|diffusion|pde|implicit-methods|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="125" PostTypeId="2" ParentId="104" CreationDate="2011-11-30T22:38:00.580" Score="31" Body="&lt;p&gt;There are many more out there, all with different goals and views of the problems.  It really depends on what you are trying to solve.  Here is an incomplete list of packages out there.  Feel free to add more details.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Large Distributed Iterative Solver Packages&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt; — packages focused around Krylov subspace methods and easy switching between linear solvers.  Much lighter weight than others in this category.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://trilinos.sandia.gov/&quot;&gt;Trilinos&lt;/a&gt; — a large set of packages aimed at FEM applications&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://acts.nersc.gov/hypre/&quot;&gt;Hypre&lt;/a&gt; — similar to the two above. Notable because of its very good multigrid solvers (which can be downloaded by PETSc).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Parallel Direct Solver Packages&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://graal.ens-lyon.fr/MUMPS/&quot;&gt;MUMPS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot;&gt;SuperLU&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Serial Direct Solver Packages&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/SuiteSparse/&quot;&gt;SuiteSparse&lt;/a&gt; — &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/umfpack/&quot;&gt;UMFPACK&lt;/a&gt; is a really good solver, but many other special purpose solvers exist here.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.intel.com/MathKernelLibrary&quot;&gt;Intel Math Kernel Library&lt;/a&gt; — High-quality library from Intel; also has a Parallel Iterative Solver (but nothing massively parallel).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.simunova.com/en/node/24&quot;&gt;Matrix Template Library&lt;/a&gt; — Generics can sometimes make the code much faster.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Interactive Environments (more for very small systems)&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mathworks.com/products/matlab/index.html&quot;&gt;MATLAB&lt;/a&gt; — industry standard&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/sparse.html&quot;&gt;SciPy.Sparse&lt;/a&gt; — if you like Python&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;&lt;a href=&quot;http://www.wolfram.com/mathematica/&quot;&gt;Mathematica&lt;/a&gt;&lt;/em&gt; — supports the manipulation of &lt;a href=&quot;http://reference.wolfram.com/mathematica/guide/SparseArrays.html&quot;&gt;&lt;code&gt;SparseArray[]&lt;/code&gt;&lt;/a&gt; objects.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Other Lists&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot;&gt;Jack Dongarra's list&lt;/a&gt; of Freely Available Software for Linear Algebra.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30" LastEditorUserId="6923" LastEditDate="2013-12-24T18:46:27.787" LastActivityDate="2013-12-24T18:46:27.787" CommentCount="1" CommunityOwnedDate="2011-11-30T22:38:00.580" ContentLicense="CC BY-SA 3.0" />
  <row Id="126" PostTypeId="1" AcceptedAnswerId="127" CreationDate="2011-11-30T22:55:54.300" Score="12" ViewCount="1426" Body="&lt;p&gt;I am solving a physical problem using implicit numerical scheme. This leads me to solving a linear equation with tridiagonal matrix. I've coded &lt;a href=&quot;http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot;&gt;this algorithm&lt;/a&gt; from Wikipedia. I wonder if there is an efficient library which allows to solve this type of equation in optimised way. An important note is that matrix itself changes only when system parameters are changing, so I had an opportunity to precalculate some algorithm steps for a nice peformance bonus. I am using C++.&lt;/p&gt;&#xA;" OwnerUserId="90" LastEditorUserId="276" LastEditDate="2012-01-11T08:44:45.603" LastActivityDate="2012-03-06T08:32:14.153" Title="Efficient tridiagonal matrix algorithm implementation" Tags="|linear-algebra|libraries|c++|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="127" PostTypeId="2" ParentId="126" CreationDate="2011-11-30T23:30:51.397" Score="15" Body="&lt;p&gt;You should probably start with the LAPACK implementation, ?gtsv, e.g., &lt;a href=&quot;http://www.netlib.org/lapack/double/dgtsv.f&quot;&gt;dgtsv&lt;/a&gt;. If you want a distributed-memory version, then you might want to start with ScaLAPACK's p?gtsv.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Since your matrix does not change very often, you can avoid redundantly factoring the tridiagonal matrix by breaking up the LAPACK routine ?gtsv into the factorization step, ?gttrf, and the solve stage, ?gttrs. Similarly named routines exist in ScaLAPACK that serve the same purpose.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2011-11-30T23:30:51.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="128" PostTypeId="1" AcceptedAnswerId="134" CreationDate="2011-11-30T23:44:48.473" Score="6" ViewCount="644" Body="&lt;p&gt;Informally in our lab, we have developed 2 metrics to compare CFD solvers over&#xA;the range of machines we have access to.  One is called &lt;strong&gt;COMP&lt;/strong&gt;, which stands&#xA;for COde Machine Performance. This single number is supposed to represent the&#xA;absolute performance of a given code on a given machine. It is computed, for a&#xA;given run, by multiplying the number of cells per computing/processing core by&#xA;the number of iterations performed and then dividing by the runtime. In an&#xA;ideal situation, this number should be constant no matter the number of cores&#xA;being used, the size of the grid or the duration of the run. It directly&#xA;indicates on how many cells one iteration can be performed by one core in one&#xA;second. By extension, the acronym COMP will be used as unit for measuring the&#xA;performance of the codes. For examples, if a given run yields &quot;3.2 k COMPs&quot;, it&#xA;means the code is able to process one iteration on 3200 cells per core per&#xA;second, or 3200 iterations on one cell per core per second, or any similar&#xA;combination.  Derived from COMP, we have obvious metrics like speed, speed-up&#xA;and efficiency, which are just expressing in different ways raw performance and&#xA;scaling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other metric, which is designed to compare the efficiency of different&#xA;schemes/codes on the same machines, simply looks at the amount of CPU time&#xA;required per unit of physical time simulated. Of course, this leaves many parameters out&#xA;of the analysis, like the grid or the accuracy of the obtained solution. But we&#xA;strive to compare runs on equivalent grid/accuracy (for example, if you compare a 4th order&#xA;scheme with a 2nd order scheme, you should probably use half as many points for&#xA;a similar accuracy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think of these metrics? Do they appear valid to you? Do you know or&#xA;use other similar metrics to benchmark your CFD solver?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should also add that we usually deal with explicit schemes on structured &#xA;grids, although we are now starting to do some comparisons with a DG code. The&#xA;reasoning might be different for unstructured grids and/or implicit schemes.&lt;/p&gt;&#xA;" OwnerUserId="95" LastEditorUserId="95" LastEditDate="2012-08-09T01:44:19.730" LastActivityDate="2012-08-09T01:44:19.730" Title="Benchmarking performance in CFD: how to compare machines and codes?" Tags="|performance|fluid-dynamics|hpc|benchmarking|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="129" PostTypeId="1" AcceptedAnswerId="131" CreationDate="2011-12-01T00:19:38.763" Score="8" ViewCount="280" Body="&lt;p&gt;I've been considering using a multigrid method to calculate the eigenvalues of a particular PDE.  I know that multigrid is extremely good at finding the least eigenvalues and their associated eigenvectors, in part because these eigenvectors are very smooth.  Is multigrid also a good method to calculate &lt;strong&gt;all&lt;/strong&gt; the eigenvalues and eigenvectors of a PDE? &lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="127" LastEditDate="2011-12-03T05:19:37.403" LastActivityDate="2011-12-03T05:19:37.403" Title="Is multigrid useful for finding all eigenvalues and eigenvectors of a differential equation, or only the lowest eigenvalues?" Tags="|multigrid|eigensystem|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="130" PostTypeId="2" ParentId="124" CreationDate="2011-12-01T00:48:19.707" Score="3" Body="&lt;p&gt;It depends on the differential equation. Some parabolic PDEs should, given constant boundary conditions, converge to a steady state.  Some will oscillate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of an oscillating PDE is the Time-Dependent Schroedinger Equation (TDSE).  If one selects an initial state which is a superposition of energy eigenstates, then the system will oscillate forever. &lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2011-12-01T00:48:19.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="131" PostTypeId="2" ParentId="129" CreationDate="2011-12-01T01:13:10.687" Score="5" Body="&lt;p&gt;For large problems, you are almost always interested in eigenvalues with specific properties instead of &lt;em&gt;all&lt;/em&gt; the eigenvalues at once. Eigenvalue methods for such problems are generally based on solving with a shifted operator. Multigrid can often be used to solve these shifted problems. The &lt;a href=&quot;http://www.grycap.upv.es/slepc/documentation/manual.htm&quot; rel=&quot;noreferrer&quot;&gt;SLEPc&lt;/a&gt; documentation contains a practical description of many of these algorithms. The library solves the linear systems in the shift-and-invert methods using &lt;a href=&quot;http://mcs.anl.gov/petsc&quot; rel=&quot;noreferrer&quot;&gt;PETSc&lt;/a&gt;, so you can experiment with multigrid (and other) preconditioners when solving eigenvalue problems.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-12-01T01:13:10.687" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="132" PostTypeId="5" CreationDate="2011-12-01T01:52:11.730" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-01T01:52:11.730" LastActivityDate="2011-12-01T01:52:11.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="133" PostTypeId="4" CreationDate="2011-12-01T01:52:11.730" Score="0" Body="A matrix with entries only on the main diagonal and the diagonals immediately above and below it. Can be solved efficiently with the Thomas Algorithm.  Appear in discretizations of the second derivative." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2011-12-01T02:50:39.960" LastActivityDate="2011-12-01T02:50:39.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="134" PostTypeId="2" ParentId="128" CreationDate="2011-12-01T02:08:32.643" Score="3" Body="&lt;p&gt;At the end of the day, the only things that matter are &lt;strong&gt;wall-clock time to a chosen accuracy&lt;/strong&gt; and &lt;strong&gt;Watts (or dollars) to a chosen accuracy&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For implementation and hardware performance, I like to measure in terms of memory bandwidth and flop/s per Watt or per dollar. If the performance of the code is very far below the hardware peak according to these metrics, then there is likely some implementation inefficiency. Alternatively, if even very simple benchmarks like &lt;a href=&quot;http://www.cs.virginia.edu/stream/&quot; rel=&quot;nofollow&quot;&gt;STREAM&lt;/a&gt; perform well below machine peak, then the machine may have bottlenecks that reduce its realizable performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The efficiency of an algorithm really can't be measured by &quot;cells per second&quot; except within a restricted class. If you measure by that metric, I argue that you have missed the point.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-12-01T02:08:32.643" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="135" PostTypeId="1" AcceptedAnswerId="137" CreationDate="2011-12-01T04:30:28.930" Score="20" ViewCount="6995" Body="&lt;p&gt;You've managed to finally find out how the atoms are spatially arranged on your newly-discovered molecular entity. Through, say, spectroscopic means, you are now in possession of a bunch of atom coordinates, atom types, bond lengths, bond types, and whatnot for your molecule. You are now interested in determining the point group (symmetry group) of your molecule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simple molecules like methane ($T_d$) or benzene ($D_{6h}$), it is a simple matter of visual inspection to determine the point group in which a molecule belongs. However, this isn't so feasible when the molecule is a bit on the large side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a molecule stored in some convenient data format (*.pdb, *.mol, etc.), how do you algorithmically determine the molecule's symmetry group?&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="276" LastEditDate="2012-03-29T18:00:16.210" LastActivityDate="2021-09-09T14:47:52.370" Title="How does one determine the point group of a molecule?" Tags="|computational-chemistry|" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="136" PostTypeId="1" AcceptedAnswerId="139" CreationDate="2011-12-01T05:33:05.913" Score="28" ViewCount="5661" Body="&lt;p&gt;How can the gravitational n-body problem be solved numerically in parallel?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is precision-complexity tradeoff possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does precision influence the quality of the model? &lt;/p&gt;&#xA;" OwnerUserId="148" LastEditorUserId="20904" LastEditDate="2019-08-28T00:59:21.007" LastActivityDate="2019-08-28T00:59:21.007" Title="How can the gravitational n-body problem be solved in parallel?" Tags="|algorithms|ode|parallel-computing|complexity|precision|" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="137" PostTypeId="2" ParentId="135" CreationDate="2011-12-01T05:40:16.150" Score="13" Body="&lt;p&gt;My primary experience is with crystal structures, and there is only a finite number of &lt;a href=&quot;http://books.google.com/books?id=r4GIU2wJCAEC&amp;amp;pg=PA54&quot; rel=&quot;noreferrer&quot;&gt;point symmetries&lt;/a&gt; that show up in a crystal. So, the algorithm I would use is slightly different than what you'd use in a molecule. But, it is unlikely with a large molecule that the continuous symmetries will show up, like the axial symmetry in H$_2$ or CO$_2$, so the methods should overlap quite well.  When determining the symmetry in a system, there are two different, but related, symmetries to consider: local and global.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Local Symmetry&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Local symmetry is the symmetry of the local environment around a specific point. In particular, the symmetry at each atomic location determines the local atomic splitting and to some extent the chemical environment, and is a subgroup of the global symmetry.  For example, in benzene the local symmetry consists of two reflection planes and a $C_2$ axis ($180^\circ$ rotation symmetry). (Obviously, only two of the operations are necessary to generate the entire local point group.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/WqnT9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an algorithmic perspective, what we've done is to first find the nearest neighbors of the target atom, and then enumerate all the ways we can rotate that environment about the central atom and have it remain the same. More mathematically, it is solving for all orthogonal matrices, $\mathbf{A}$, such that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A}(\vec{x}_i - \vec{x}_c) = \vec{x}_j - \vec{x}_c$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\vec{x}_i$ and $\vec{x}_j$ are the positions of atoms of the same species and $\vec{x}_c$ is the position of the central, or target, atom. But, I'd look at simpler forms first, like whether or not a reflection plane exists, prior to trying to solve for $\mathbf{A}$ in general.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thought is to use the angular momentum matrices as generators of rotation, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A} = \exp(i \phi \hat{n}\cdot\vec{\mathbf{L}})$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\hat{n} \in \mathbb{R}^3$ is a unit vector about which a rotation of with angle $\phi$ is performed, and $\vec{\mathbf{L}} = (\mathbf{L}_x,\ \mathbf{L}_y,\ \mathbf{L}_z)$ is the vector of three dimensional angular momentum matrices. $\mathbf{A}$ would then have only 3 unknowns. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Global Symmetry&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Where the local symmetry determines the environment around a single atom, the global symmetry dictates how the atoms interchange with each other. The first step in determining the global symmetry is to determine the equivalent atoms.  First, determine the types of and the relative directions to the nearest neighbor (and second nearest, or higher, if desired) atoms. Two atoms are then equivalent, if their neighbors have the same spatial arrangements.  This is straightforward to calculate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second step is roughly the same as that found in the local symmetry case, except that the center of mass of the molecule is likely the symmetry center.  At this point, if the local symmetries have been determined, only a few unique operations may need to be found to generate the entire group.  For example, in the &lt;a href=&quot;http://cst-www.nrl.navy.mil/lattice/struk/b20.html&quot; rel=&quot;noreferrer&quot;&gt;B20 crystal structure&lt;/a&gt;, each atom has a local &lt;a href=&quot;http://www.webqc.org/symmetrypointgroup-c3.html&quot; rel=&quot;noreferrer&quot;&gt;$C_3$ symmetry&lt;/a&gt;, and the full point group is generated by including a 2-fold ($180^\circ$ rotation) &lt;a href=&quot;http://en.wikipedia.org/wiki/Screw_axis&quot; rel=&quot;noreferrer&quot;&gt;screw axis&lt;/a&gt; which transforms one atom into another. In benzene, two operations are required: a 6-fold ($60^\circ$) rotation through the central axis and a reflection plane bisecting a bond.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: For the B20 structure, you can use two of the $C_3$ axes, instead, to generate the full group.  This should allow you to avoid having to figure out a way to automatically determine the screw axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Caution&lt;/strong&gt;: A caution on using the ideas in the local symmetry section in the global section, to be a symmetry operation, the environment must also be transformed. So, that if you find $\mathbf{A}$, from above, it will only give a candidate symmetry as the transformation may not similarly change the environment appropriately, and further checks are needed.  For instance, if the benzene ring had hydrogen atoms sticking out of the plane of the ring along one side, then the reflection plane bisecting the carbon-carbon bond would be fine, but a $180^\circ$ rotation similarly bisecting the bond would not because it would not reproduce the local environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit - Translations&lt;/strong&gt;: There is one other complication that the above discussion on local symmetry ignores: translations. Formally, the correct symmetry operation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A}(\vec{x}_i - \vec{x}_c) + \vec{t} = \vec{x}_j - \vec{x}_c$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{A}$ and $\vec{x}_k$, as above, and $\vec{t}$ is an arbitrary translation. In a symmorphic crystal, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{t} = n_1 \vec{a}_1 + n_2 \vec{a}_2 + n_3 \vec{a}_3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\vec{a}_i$ is a primitive lattice translation and $n_i \in \mathbb{Z}$, so the point group and translations are completely separable. In a non-symmorphic crystal, $\vec{t}$ can consist of non-primitive translations.  The difference between the two is simply that for a symmorphic crystal, a single center of rotation can be found, but for non-symmorphic crystals, this is not true.  A molecular system is likely to be &quot;non-symmorphic&quot; in this latter sense, and require the addition of translations to fully realize the group.&lt;/p&gt;&#xA;" OwnerUserId="85" LastEditorUserId="127" LastEditDate="2011-12-07T19:17:12.807" LastActivityDate="2011-12-07T19:17:12.807" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="138" PostTypeId="2" ParentId="136" CreationDate="2011-12-01T05:54:09.850" Score="14" Body="&lt;p&gt;Look at the &lt;a href=&quot;http://en.wikipedia.org/wiki/Fast_multipole_method&quot;&gt;fast multipole method&lt;/a&gt;.  It is highly scalable and $O(n)$.  It allows trading off between precision and cost.  &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1654123&quot;&gt;Here's an example where it is run at 42 Tflops on a GPU cluster&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2011-12-01T05:54:09.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="139" PostTypeId="2" ParentId="136" CreationDate="2011-12-01T06:03:22.217" Score="29" Body="&lt;p&gt;There is a wide variety of algorithms; &lt;a href=&quot;http://en.wikipedia.org/wiki/Barnes%E2%80%93Hut_simulation&quot;&gt;Barnes Hut&lt;/a&gt; is a popular $\mathcal{O}(N \log N)$ method, and the &lt;a href=&quot;http://en.wikipedia.org/wiki/Fast_multipole_method&quot;&gt;Fast Multipole Method&lt;/a&gt; is a much more sophisticated $\mathcal{O}(N)$ alternative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both methods make use of a tree data structure where nodes essentially only interact with their nearest neighbors at each level of the tree; you can think of splitting the tree between the set of processes at a sufficient depth, and then having them cooperate only at the highest levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a recent paper discussing FMM on petascale machines &lt;a href=&quot;http://arxiv.org/abs/1106.5273&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2013-01-29T23:10:02.917" LastActivityDate="2013-01-29T23:10:02.917" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="141" PostTypeId="2" ParentId="8" CreationDate="2011-12-01T09:59:53.560" Score="1" Body="&lt;p&gt;While this is prehaps not the intended use of the software, I have found the parsers and general protein-structure processing methods found within the &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow&quot;&gt;VMD&lt;/a&gt; source code to be robust and comprehensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VMD is an open-source molecular visualisation tool. It is relatively staright-forward to extend when that is appropriate, otherwise code can be cannibalised from it.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2011-12-01T09:59:53.560" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="142" PostTypeId="2" ParentId="126" CreationDate="2011-12-01T13:50:26.320" Score="2" Body="&lt;p&gt;&lt;strong&gt;For distributed parallel systems&lt;/strong&gt;: I have not tried ScaLAPACK, which has a parallel tridiagonal solver, for which there are examples available &lt;a href=&quot;http://acts.nersc.gov/scalapack/hands-on/etc/pddttrdrv/pddttrdrv.c.html&quot; rel=&quot;nofollow&quot;&gt;online&lt;/a&gt;. I have tried with some success a method proposed by David Moulton in a &lt;a href=&quot;http://math.lanl.gov/Research/Highlights/par-line-solve.shtml&quot; rel=&quot;nofollow&quot;&gt;LANL publication&lt;/a&gt;. Coding this up might be more than you want to do, but by using LAPACK, it is strait forward. &lt;/p&gt;&#xA;" OwnerUserId="79" LastActivityDate="2011-12-01T13:50:26.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="143" PostTypeId="1" AcceptedAnswerId="419" CreationDate="2011-12-01T15:53:12.003" Score="9" ViewCount="858" Body="&lt;p&gt;Whenever one uses a quasi-Monte Carlo method for cubature or optimization, it seems that there's a wide variety of low-discrepancy sequences to choose from, associated with the names of van der Corput, Halton, Hammersley, Faure, Niederreiter, Sobol', and other names I don't quite remember. Are there any good rules of thumb on how to choose the most appropriate low-discrepancy sequence for your computations?&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2011-12-15T10:18:21.407" Title="How do I know which low-discrepancy sequence to use?" Tags="|monte-carlo|random-number-generation|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="144" PostTypeId="2" ParentId="143" CreationDate="2011-12-01T16:10:25.427" Score="2" Body="&lt;p&gt;My understanding is that the situation mirrors that found in iterative methods. We can prove general results, like $\frac{1}{N}$ decay of the quadrature error, but cannot determine the optimal algorithm for a given problem &lt;em&gt;a priori&lt;/em&gt;. &lt;a href=&quot;http://roth.cs.kuleuven.be/wiki/Public_software&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is a catalogue of open source software for sequence generation. I think the best approach here is more testing and comparison.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-01T16:10:25.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="145" PostTypeId="2" ParentId="135" CreationDate="2011-12-01T16:45:30.757" Score="12" Body="&lt;p&gt;There's old code for the purpose that's used in a few packages, called SYMMOL. The algorithm it uses is described in the following paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;T. Pilati and A. Forni, &lt;a href=&quot;http://dx.doi.org/10.1107/S0021889898002180&quot; rel=&quot;noreferrer&quot;&gt;&quot;SYMMOL: a program to find the maximum symmetry group in an atom cluster, given a prefixed tolerance&quot;&lt;/a&gt;, &lt;em&gt;J. Appl. Cryst.&lt;/em&gt; 1998. 31, 503-504.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Basically, it determines the centre of inertia, then applies possible symmetry operations and attempts to determine whether a transformation vector exists to map the operated geometry onto the original within a given tolerance. The code itself is no longer available from the authors' site, but it is available (with a set of example input files) &lt;a href=&quot;http://www.ccp14.ac.uk/ccp/web-mirrors/symmol/~pila/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="70" LastEditorUserId="127" LastEditDate="2013-06-04T16:57:27.300" LastActivityDate="2013-06-04T16:57:27.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="146" PostTypeId="2" ParentId="63" CreationDate="2011-12-01T16:55:59.280" Score="2" Body="&lt;p&gt;I'm not really an expert when it comes to LES, but I believe the following paper might be a good starting point:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Assessment Measures for Engineering LES Applications&quot;&#xA;I. Celik, M. Klein, and J. Janicka, J. Fluids Eng. 131, 031102 (2009), DOI:&lt;a href=&quot;http://dx.doi.org/10.1115/1.3059703&quot; rel=&quot;nofollow&quot;&gt;10.1115/1.3059703&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2011-12-01T16:55:59.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="147" PostTypeId="2" ParentId="32" CreationDate="2011-12-01T17:05:54.787" Score="5" Body="&lt;p&gt;I gave an &lt;a href=&quot;https://math.stackexchange.com/a/86984&quot;&gt;answer&lt;/a&gt; @ math.SE.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2011-12-01T17:05:54.787" CommentCount="0" CommunityOwnedDate="2011-12-01T17:05:54.787" ContentLicense="CC BY-SA 3.0" />
  <row Id="148" PostTypeId="5" CreationDate="2011-12-01T17:34:33.520" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-01T17:34:33.520" LastActivityDate="2011-12-01T17:34:33.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="149" PostTypeId="4" CreationDate="2011-12-01T17:34:33.520" Score="0" Body="Questions related to Large Eddy Simulation (LES), a modeling technique for Computational Fluid Dynamics  . This can include discussions of turbulence models (with or without combustion), validity of LES in a given problem, etc..." OwnerUserId="95" LastEditorUserId="95" LastEditDate="2011-12-01T21:59:17.917" LastActivityDate="2011-12-01T21:59:17.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="150" PostTypeId="2" ParentId="143" CreationDate="2011-12-01T18:21:35.880" Score="0" Body="&lt;p&gt;Where we've used monte carlo simulation, the recommendation (from folks who work in the other national labs) is to use &quot;good&quot; &lt;a href=&quot;http://en.wikipedia.org/wiki/Cryptographically_secure_pseudorandom_number_generator&quot; rel=&quot;nofollow&quot;&gt;pseudo-random number generators&lt;/a&gt;. The last one I worked on used &lt;a href=&quot;http://www.schneier.com/yarrow.html&quot; rel=&quot;nofollow&quot;&gt;Yarrow&lt;/a&gt;, but newer ones would use &lt;a href=&quot;http://en.wikipedia.org/wiki/Fortuna_%28PRNG%29&quot; rel=&quot;nofollow&quot;&gt;Fortuna&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Mersenne_twister&quot; rel=&quot;nofollow&quot;&gt;Mersenne-Twister&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16" LastActivityDate="2011-12-01T18:21:35.880" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="151" PostTypeId="2" ParentId="46" CreationDate="2011-12-01T18:36:26.863" Score="10" Body="&lt;p&gt;A &lt;a href=&quot;http://pitagora.dm.uniba.it/~testset/&quot;&gt;test set for IVPs&lt;/a&gt; (Initial Value Problems for ODE solvers) is currently maintained by people from the University of Bari, Italy, who took it over from CWI Amsterdam.&lt;/p&gt;&#xA;" OwnerUserId="153" LastActivityDate="2011-12-01T18:36:26.863" CommentCount="1" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="152" PostTypeId="1" AcceptedAnswerId="169" CreationDate="2011-12-01T18:39:00.447" Score="38" ViewCount="1738" Body="&lt;p&gt;It's pretty clear from a survey of the top500 that the industry is trending towards an &lt;a href=&quot;http://www.scidacreview.org/1001/images/hardware05.jpg&quot;&gt;exponential increase in processing cores&lt;/a&gt;.  The largest supercomputers all use MPI for communication between nodes, though there does not appear to be a clear trend for on-node parallelism, with the simplest (but not necessarily the most efficient) approach to map a single MPI process to each core, automatic parallelization from the compiler, OpenMP, pthreads, CUDA, Cilk, and OpenCL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am one of a group of scientists maintaining and developing a code that has potential to be used on some of the largest supercomputers in the world.  Assuming finite developer time, how do I future-proof myself so that I can take advantage of the performance of the world's most powerful machine?  What assumptions should I make about process interconnect architecture?  What paradigms are going to suffer as we enter the manycore era?  Will Partitioned Global Address Space languages be available &quot;in production&quot; on petascale machines?&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2013-05-03T16:41:52.890" Title="What programming paradigms should I be investing in if I want my code to run on petascale machines in the future?" Tags="|exascale|programming-paradigms|hpc|" AnswerCount="7" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="153" PostTypeId="2" ParentId="21" CreationDate="2011-12-01T18:40:41.253" Score="6" Body="&lt;p&gt;The Gaussian implementation of B3LYP uses the VWN3 functional, according to the &lt;a href=&quot;http://gaussian.com/g_tech/g_ur/k_dft.htm&quot; rel=&quot;noreferrer&quot;&gt;manual&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Making Gaussian use the VWN5 functional instead for it is a bit tricky, but can &lt;a href=&quot;http://wiki.ctcc.no/doku.php?id=public%3ahowto%3agaussian_tricks&quot; rel=&quot;noreferrer&quot;&gt;apparently&lt;/a&gt; be done by adding all the following to the route line:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;bv5lyp&lt;/code&gt; - to specify which functional components - Becke exchange, and VWN5 local, LYP non-local correlation.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;iop(3/76=1000002000)&lt;/code&gt; - 20% HF exchange, plus&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;iop(3/77=0720008000)&lt;/code&gt; - 72% Becke non-local exchange, plus 80% Slater local exchange, plus&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;iop(3/78=0810010000)&lt;/code&gt; - 81% LYP non-local correlation, plus 100% of the V5LYP VWN5 local correlation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(You can see why people try to avoid using the IOP keyword.) More information on using these is on the aforementioned &lt;a href=&quot;http://gaussian.com/g_tech/g_ur/k_dft.htm&quot; rel=&quot;noreferrer&quot;&gt;DFT keywords page of the Gaussian manual&lt;/a&gt;, under 'User-Defined Models'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not that familiar with GAMESS, but it doesn't seem to have the option of using the VWN3 version of B3LYP, so it doesn't seem as though you can go the other way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for these and adaptability in other packages, I know Turbomole has listed both B3LYP (using VWN5) and B3LYP_Gaussian (using VWN3), and the &lt;a href=&quot;http://www.scm.com/Doc/Doc2010/ADF/ADFUsersGuide/page90.html&quot; rel=&quot;noreferrer&quot;&gt;manual for ADF&lt;/a&gt; suggests you can only use VWN5 for B3LYP there, but you can tweak the amount of HF exchange if that's something you want to do.&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2011-12-01T18:40:41.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="154" PostTypeId="2" ParentId="152" CreationDate="2011-12-01T19:07:47.517" Score="8" Body="&lt;p&gt;Let's start by discussing a strategy for intranode code (computing which does not touch the interconnect), since I think MPI is a good choice for internode code. I think its senseless to talk about nodes with fewer than 100 cores, so at least a current GPU or MIC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its a fact that pthreads alone cannot get you maximum performance on any modern chip, because you must take advantage of the vector unit (true since the first Cray). On Intel and AMD you can use intrinsics, but these are not portable, and in my opinion clunky. CUDA and OpenCL have vectorization built into the library and make it easy to get maximum performance. All the new hardware of which I am aware has this vector requirement, so any solution should take this into account. For me, CUDA/OpenCL is the current way to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, all these machines will be NUMA, which is harder to program, but I think the kernel strategy works. You divide up work and data into small units. These will probably be automatically scheduled, as happens currently in CUDA and OpenCL, but you can specify dependencies. For problems which fit the streaming paradigm, this chunking can also be done automatically. Intel TBB does this, but I prefer the higher-level library approach exemplified by &lt;a href=&quot;http://code.google.com/p/thrust/&quot; rel=&quot;nofollow&quot;&gt;Thrust&lt;/a&gt; and &lt;a href=&quot;http://code.google.com/p/cusp-library/&quot; rel=&quot;nofollow&quot;&gt;Cusp&lt;/a&gt;, which can target CUDA or (soon) TBB.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="393" LastEditDate="2012-05-31T19:44:54.427" LastActivityDate="2012-05-31T19:44:54.427" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="155" PostTypeId="2" ParentId="152" CreationDate="2011-12-01T19:33:18.753" Score="2" Body="&lt;p&gt;I suspect that even the most well-thought out answers to this question will be obsolete in five to ten years. Given the uncertainty of future programming paradigms it may not be worthwhile to spend a great deal of time pre-optimizing your codebase. &lt;/p&gt;&#xA;" OwnerUserId="19" LastActivityDate="2011-12-01T19:33:18.753" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="158" PostTypeId="2" ParentId="152" CreationDate="2011-12-01T22:49:12.513" Score="3" Body="&lt;p&gt;Really massive amounts of cores also open trivial yet surprisingly useful perspective -- just to use it to run many iterations of the whole simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Significant part of computational research nowadays boils down to scanning some parameter space, screening large pool of initial conditions or calculating a distribution of some result in a resampling manner; all those tasks are embarrassingly parallel, thus Amdahl-proof.&lt;/p&gt;&#xA;" OwnerUserId="102" LastActivityDate="2011-12-01T22:49:12.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="159" PostTypeId="1" AcceptedAnswerId="170" CreationDate="2011-12-02T01:02:58.350" Score="58" ViewCount="3775" Body="&lt;p&gt;A lot of numerical algorithms (integration, differentiation, interpolation, special functions, etc.) are available in scientific computation libraries like &lt;a href=&quot;http://www.gnu.org/s/gsl/&quot;&gt;GSL&lt;/a&gt;. But I often see code with &quot;hand-rolled&quot; implementations of these functions. For small programs which are not necessarily intended for public distribution, is it common practice among computational scientists to just implement numerical algorithms yourself (by which I mean copying or transcribing from a website, Numerical Recipes, or similar) when you need them? If so, is there a particular reason to &lt;em&gt;avoid&lt;/em&gt; linking to something like GSL, or is it just more &quot;tradition&quot; than anything else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask because I'm a big fan of &lt;a href=&quot;http://en.wikipedia.org/wiki/Code_reuse&quot;&gt;code reuse&lt;/a&gt;, which would suggest that I should try to use existing implementations when possible. But I'm curious whether there are reasons that principle is less valuable in &lt;em&gt;scientific&lt;/em&gt; computation than in general programming.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Forgot to mention: I'm specifically asking about C and C++, as opposed to languages like Python where there is a clear benefit (speed of execution) to using a library.&lt;/p&gt;&#xA;" OwnerUserId="103" LastEditorUserId="127" LastEditDate="2011-12-02T02:53:49.213" LastActivityDate="2017-07-02T06:52:12.030" Title="Is it common not to use libraries for standard numerical algorithms, and why?" Tags="|libraries|c|" AnswerCount="17" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="160" PostTypeId="1" AcceptedAnswerId="176" CreationDate="2011-12-02T01:11:57.650" Score="19" ViewCount="4101" Body="&lt;p&gt;A recent answer mentioned the use of &lt;a href=&quot;http://en.wikipedia.org/wiki/Fortuna_%28PRNG%29&quot;&gt;Fortuna&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Mersenne_twister&quot;&gt;Mersenne Twister&lt;/a&gt; Random Number Generators (&lt;a href=&quot;http://en.wikipedia.org/wiki/Pseudorandom_number_generator&quot;&gt;RNGs&lt;/a&gt;) to seed a &lt;a href=&quot;http://en.wikipedia.org/wiki/Monte_Carlo_method&quot;&gt;Monte Carlo simulation&lt;/a&gt;. I hadn't heard of Fortuna before so I looked it up - looks like it is mainly intended for cryptographic use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently use a Mersenne Twister in production code to seed a K-Means algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which (Fortuna or Mersenne Twister) is considered the best for &quot;algorithmic seeding&quot; applications (eg. seeding Monte Carlo and K-Means)? Or is it a &quot;toss up&quot; - i.e. use the most convenient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From where I am sitting, &quot;best&quot; should provide the highest quality random numbers, operate quickly, and (possibly) have a low memory footprint. Of these, quality is probably the most important for most of us.&lt;/p&gt;&#xA;" OwnerUserId="74" LastEditorUserId="20688" LastEditDate="2019-06-22T22:22:02.513" LastActivityDate="2019-06-22T22:22:02.513" Title="Is Fortuna or Mersenne Twister preferable as an algorithmic RNG?" Tags="|algorithms|random-number-generation|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="161" PostTypeId="2" ParentId="159" CreationDate="2011-12-02T01:22:45.637" Score="33" Body="&lt;p&gt;There is substantial programmer overhead involved in linking to a library function, especially if that library is new to the programmer. It is often simpler to just rewrite simple algorithms rather than figure out the specifics of a particular library. As the algorithms become more complex this behavior switches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python has excelled at reducing this overhead with tools like pip/easy_install and a uniform data structure interface (i.e. every library seems to take and produce a numpy array). &lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2011-12-02T17:39:45.410" LastActivityDate="2011-12-02T17:39:45.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="162" PostTypeId="2" ParentId="159" CreationDate="2011-12-02T01:31:05.430" Score="10" Body="&lt;p&gt;I think that implementing an algorithm instead of using a library can sometimes give a better understanding and control of the model. When I am coding some program for scientific computations, it's important for me to understand what I am doing. Implementing the important algorithms helps me to get a better knowledge of the problem and achieve better control of it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, sometimes it's not a trivial task to select a library that is needed for getting a solution, so it's better to search for already-implemented algorithms when you are sure what are you trying to achieve and why do you want it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the algorithms are complex, then coding them by hand gives you the opportunity to improve performance/quality of solution using task-specific features. And sometimes it is necessary to change the algorithm a bit, which is easier if you know the code that you wrote and you can edit it in the way you want.&lt;/p&gt;&#xA;" OwnerUserId="90" LastEditorUserId="123" LastEditDate="2011-12-03T18:06:15.140" LastActivityDate="2011-12-03T18:06:15.140" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="163" PostTypeId="1" AcceptedAnswerId="598" CreationDate="2011-12-02T01:34:38.593" Score="5" ViewCount="153" Body="&lt;p&gt;Texts, articles, and papers on Maximum Entropy Classifiers tend to come in two varieties: the more popular &quot;upper level&quot;, and the more technical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The popular variety are good at explaining the Maximum Entropy concept and why such classifiers are considered to be better than Naive Bayes classifiers (and also why they are harder to calculate). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the discussions on how the coefficients are determined are much more heavy going. Or they have been when I've tried to read them - perhaps I need some sleep and no customer interruptions :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any intermediate descriptions of maximum entropy coefficient determination - especially for the gradient / quasi-Newtonian methods (rather than the iterative methods).&#xA;Ie. how these methods are used to determine the best coefficients that fit the training data.  I've looked at code and I guess I have a conceptual problem understanding the connection between the classifier code itself, and gradient code (LBFGSB, CG, etc).&lt;/p&gt;&#xA;" OwnerUserId="74" LastEditorUserId="74" LastEditDate="2011-12-08T16:36:24.057" LastActivityDate="2012-01-07T08:56:42.550" Title="Looking for an understandable discussion of creating Maximum Entropy classifiers" Tags="|machine-learning|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="164" PostTypeId="5" CreationDate="2011-12-02T03:44:54.793" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-02T03:44:54.793" LastActivityDate="2011-12-02T03:44:54.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="165" PostTypeId="4" CreationDate="2011-12-02T03:44:54.793" Score="0" Body="Questions on the algorithms for and uses of (pseudo)random number generators in scientific computing." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-12-02T19:06:40.507" LastActivityDate="2011-12-02T19:06:40.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="166" PostTypeId="5" CreationDate="2011-12-02T03:46:04.947" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-02T03:46:04.947" LastActivityDate="2011-12-02T03:46:04.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="167" PostTypeId="4" CreationDate="2011-12-02T03:46:04.947" Score="0" Body="Questions on Monte Carlo methods, methods that require the repeated generation of (pseudo-, quasi-)random numbers for computing their results." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-12-02T19:05:28.480" LastActivityDate="2011-12-02T19:05:28.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="168" PostTypeId="2" ParentId="159" CreationDate="2011-12-02T03:54:16.737" Score="12" Body="&lt;p&gt;I think it &lt;strong&gt;is&lt;/strong&gt; quite common, with some algorithms more likely to be re-implemented than others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a tricky trade-off between how annoying a library is to install, how hard it is to implement the algorithm yourself, how hard it is to optimize it, and how well the library fits your needs. Also, sometimes using a library is just overkill: I used the slow bisection algorithm in one of my programs because I called it only a few times and I didn't want to add a library just for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it easy for you to write a well-optimized version? If it is, you might be better off doing it. You're going to get exactly what you need and you won't depend on anyone's work. But of course you really need to know what you're doing: even simple algorithms can be tricky to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be curious to see a study on this, but from my biased perspective, scientists often use libraries for linear algebra and random number generators, with most of the remaining code being homemade.&lt;/p&gt;&#xA;" OwnerUserId="91" LastEditorUserId="123" LastEditDate="2011-12-03T18:02:30.263" LastActivityDate="2011-12-03T18:02:30.263" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="169" PostTypeId="2" ParentId="152" CreationDate="2011-12-02T05:15:55.637" Score="36" Body="&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Historical Perspective&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;It is really impossible to say what the new paradigms will be like in the future, for example a good historical perspective I suggest reading Ken Kennedy's &lt;a href=&quot;http://dx.doi.org/10.1145/1238844.1238851&quot;&gt;Rise and Fall of HPF&lt;/a&gt;.  Kennedy gives an account of two emerging patterns, MPI versus a smart compiler, and details how MPI had the right amount of early adopters and flexibility to dominate.  HPF eventually fixed its problems but it was too late.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many ways, several paradigms, such as PGAS and OpenMP, are following that same HPF trend.  The early codes have not been flexible enough to use well and left a lot of performance on the table.  But the promise of not having to write every iota of the parallel algorithm is a attractive goal.  So the pursuit of new models are always being pursued.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Clear Trends in Hardware&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Now the success of MPI has often been cited as to being closely tied to how it models the hardware it runs on.  Roughly each node has a few number of processes and passing the messages to local point-to-point or through coordinated collective operations is easily done in the cluster space.  Because of this, I don't trust anyone who gives a paradigm that doesn't follow closely to new hardware trends, I actually was convinced of this opinion by the work from &lt;a href=&quot;http://www.cs.rice.edu/~vs3/home/Vivek_Sarkar.html&quot;&gt;Vivak Sarakar&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In keeping with that here are three trends that are clearly making headway in new architectures. And let me be clear, there are now &lt;strong&gt;twelve&lt;/strong&gt; different architectures being marketed in HPC.  This up from less than 5 years ago only featuring x86, so the coming days will see lots of opportunities for using hardware in different and interesting ways&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Special Purpose Chips: Think large vector units like accelerators (view espoused by Bill Dally of Nvidia)&lt;/li&gt;&#xA;&lt;li&gt;Low Power Chips: ARM based clusters (to accomodate power budgets)&lt;/li&gt;&#xA;&lt;li&gt;Tiling of Chips: think tiling of chips with different specifications (work of &lt;a href=&quot;http://people.csail.mit.edu/agarwal/&quot;&gt;Avant Argwal&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Current Models&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The current model is actually 3 levels deep.  While there are many codes using two of these levels well, not many have emerged using all three.  I believe that to first get to exascale one needs to invest in determining if you code can run at all three levels.  This is probably the safest path for iterating well with the current trends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me iterate on the models and how they will need to change based on the predicted new hardware views.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Distributed&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The players at the distributed level largely fall into MPI and PGAS languages.  MPI is a clear winner right now, but PGAS languages such as UPC and Chapel are making headways into the space.  One good indication is the HPC Benchmark Challenge.  PGAS languages are giving very elegant implementations of the benchmarks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most interesting point here is that while this model currently only works at the node level, it will be an important model inside a node for Tiled architectures.  One indication is the Intel SCC chip, which fundamentally acted like a distributed system.  The SCC team created their own MPI implementation and many teams were successful at porting community libraries to this architecture.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to be honest PGAS really has a good story for stepping in to this space.  Do you really want to program MPI internode and then have to do the same trick intranode?  One big deal with these tiled architectures is that they will have different clock speeds on the chips and major differences in bandwidth to memory so performant codes must take this into account.  &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;On-node shared memory&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Here we see MPI often being &quot;good enough&quot;, but PThreads (and libraries deriving from PThreads such as Intel Parallel Building Blocks) and OpenMP are still used often.  The common view is that there will be a time when there are enough shared memory threads that MPI's socket model will break down for RPC or you need a lighter weight process running on the core. Already you can see the indications of IBM Bluegene systems having problems with shared memory MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Matt comments, the largest performance boost for compute intensive codes is the vectorization of the serial code.  While many people assume this is true in accelerators, it is also critical for on-node machines as well.  I believe Westmere has a 4 wide FPU, thus one can only get a quarter of the flops without vectorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I don't see the current OpenMP stepping into this space well, there is a place for low-powered or tiles chips to use more light threads. OpenMP has difficulty describing how the data flow works and as more threads are used I only see this trend becoming more exaggerated, just look at examples of what one has to do to get proper prefetching with OpenMP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both OpenMP and PThreads at a course enough level can take advantage of the vectorization necessary to get a good percentage of peak, but doing so requires breaking down your algorithms in a way that vectorization is natural.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Co-processor&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Finally the emergence of the co-processor (GPU, MIC, Cell acclerators) has taken hold.  It is becoming clear that no path to exascale will be complete without them.  At SC11, every Bell prize contestent used them very effectively to get to the low petaflops.  While CUDA and OpenCL have dominated the current market, I have hopes for OpenACC and PGAS compilers entering the space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to get to exascale, one proposal is to couple the low powered chips to lots of co-processors.  This will pretty well kill off the middle layer of the current stack and use codes that manage decision problems on the main chip and shuffle off work to the co-processors.  This means that for code to work quite effectively a person must rethink the algorithms in terms of kernels (or codelets), that is branchless instruction level parallel snippets.  As far as I know, a solution to this evolution is pretty wide open.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;How this affects the app developer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Now to get to your question.  If you want to protect yourself from the oncoming complexities of exascale machines, you should do a few things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Develop your algorithms to fit at least three levels of parallel hierarchy.&lt;/li&gt;&#xA;&lt;li&gt;Design your algorithms in terms of kernels that can be moved between the heirarchy.&lt;/li&gt;&#xA;&lt;li&gt;Relax your need for any sequential processes, all of these effects will happen asynchronously because synchronous execution is just not possible.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want to be performant today, MPI + CUDA/OpenCL is good enough but UPC is getting there so its not a bad idea to take a few days and learn it.  OpenMP gets you started but leads to problems once the code needs to be refactored. PThreads requires completely rewriting your code to its style.  Which makes MPI + CUDA/OpenCL the current best model.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;What is not discussed here&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;While all this talk of exascale is nice, something not really discussed here is getting data onto and off of the machines.  While there have been many advances in memory systems, we don't see them in commodity cluster (just too darned expensive).  Now that data intensive computing is becoming a large focus of all the super computing conferences, there is bound to be a bigger movement into the high memory bandwidth space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This brings to the other trend that might happen (if the right funding agencies get involved). Machines are going to become more and more specialized for the type of computing required.  We already see &quot;data-intensive&quot; machines being funded by the NSF, but these machines are on a different track than the 2019 Exascale Grand Challenge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This became longer than expected ask for references where you need them in the comments&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="30" LastEditDate="2011-12-02T20:10:24.673" LastActivityDate="2011-12-02T20:10:24.673" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="170" PostTypeId="2" ParentId="159" CreationDate="2011-12-02T05:19:40.730" Score="51" Body="&lt;p&gt;I used to implement everything myself, but lately have begun using libraries much more.  I think there are several very important advantages of using a library, beyond just the issue of whether you have to write a routine yourself or not.  If you use a library, you get&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Code that has been &lt;strong&gt;tested&lt;/strong&gt; by hundreds/thousands/more users&lt;/li&gt;&#xA;&lt;li&gt;Code that will continue to be &lt;strong&gt;updated&lt;/strong&gt; and &lt;strong&gt;improved&lt;/strong&gt; in the future, without any work on your part&lt;/li&gt;&#xA;&lt;li&gt;Optimized code that is more &lt;strong&gt;efficient&lt;/strong&gt; and perhaps more &lt;strong&gt;scalable&lt;/strong&gt; than what you would write in a first attempt&lt;/li&gt;&#xA;&lt;li&gt;Depending on the library, by establishing an interface to it in your code you may get access to many algorithms that you currently don't use but may want to in the future&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the last bullet point above, I'm thinking of large libraries like &lt;a href=&quot;http://trilinos.sandia.gov/&quot;&gt;Trilinos&lt;/a&gt; or &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot;&gt;PETSc&lt;/a&gt;.  I can reinforce this with a couple of concrete personal examples in development of &lt;a href=&quot;http://numerics.kaust.edu.sa/pyclaw&quot;&gt;PyClaw&lt;/a&gt;.  Although it would have been straightforward to parallelize &lt;a href=&quot;http://www.clawpack.org&quot;&gt;Clawpack&lt;/a&gt; with MPI calls, we chose to use PETSc.  This allowed us to limit the paralle code in the package to less than 300 lines of Python, but even better, by putting our data in PETSc's format we gained immediate access to PETSc's implicit solvers, enabling current work on an implicit solver in PyClaw.  As a second example, PyClaw initially included hand-code fifth-order WENO reconstruction, but we eventually decided to rely on the &lt;a href=&quot;http://memmett.github.com/PyWENO/&quot;&gt;PyWENO&lt;/a&gt; package for this.  This was a huge gain, since PyWENO can automatically generate WENO routines of any order in several languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you use libraries, you can &lt;strong&gt;contribute&lt;/strong&gt; back by developing improvements or finding bugs, which will benefit many other people, whereas debugging or improving your own code only benefits you.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-12-02T05:27:35.550" LastActivityDate="2011-12-02T05:27:35.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="171" PostTypeId="2" ParentId="159" CreationDate="2011-12-02T05:25:20.157" Score="8" Body="&lt;p&gt;One answer is that there are so many slight variations to numerical code that it is really hard to encapsulate that in a library. Take this in comparison to web software, which is often easy to install and has a clear set of inputs and outputs.  I think more common is people grabbing a framework, or big library that acts like a framework (Trilinos/PETSc), and using that ecosystem to get the benefits of using community codes.  &lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2011-12-02T05:25:20.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="172" PostTypeId="1" CreationDate="2011-12-02T06:01:30.133" Score="16" ViewCount="340" Body="&lt;p&gt;Multi-physics simulation involves coupling multiple &quot;physics&quot;, often with different space and/or time scales. Additionally, the single-physics codes are often written by different teams. The most commonly used coupling technique is first-order operator splitting, but this has poor accuracy and stability properties. How do I determine which algorithms will be effective for a problem of interest, and how should I structure my software to make these algorithms available?&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-12-03T21:26:15.753" Title="What are the best practices for algorithms and implementation of multi-physics simulations?" Tags="|pde|multiphysics|implicit-methods|software|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="173" PostTypeId="1" AcceptedAnswerId="189" CreationDate="2011-12-02T06:23:00.930" Score="19" ViewCount="2771" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Lax_equivalence_theorem&quot;&gt;Lax equivalence theorem&lt;/a&gt; states that consistency and stability of a numerical scheme for a linear initial value problem is a necessary and sufficient condition for convergence. But for nonlinear problems, numerical methods can converge very plausibly to incorrect results, despite being consistent and stable. For example, &lt;a href=&quot;http://arxiv.org/abs/1109.1446&quot;&gt;this paper&lt;/a&gt; shows how a first order Godunov method applied to the 1D linearized shallow water equations converges to an incorrect solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evidently self-convergence under mesh and time step refinement is not sufficient, but exact solutions are generally not available for nonlinear PDEs, so how can one determine if a numerical method is converging to a genuine solution?&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-01-09T16:09:47.760" Title="How to determine if a numerical solution to a PDE is converging to a continuum solution?" Tags="|pde|stability|convergence|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="174" PostTypeId="2" ParentId="159" CreationDate="2011-12-02T06:53:12.290" Score="2" Body="&lt;p&gt;The group I work with uses libraries as much as possible. I'm one of the few programmers, and the rest of the folks picked up programming on the job. They know enough of their own limitations to know where they shouldn't be dabbling. IMSL is the preferred library. Stuff like GSL would be forbidden due to licensing restrictions, even though this is a federal agency and we give our software away anyway. &lt;/p&gt;&#xA;" OwnerUserId="16" LastActivityDate="2011-12-02T06:53:12.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="175" PostTypeId="2" ParentId="60" CreationDate="2011-12-02T07:30:29.347" Score="2" Body="&lt;p&gt;I'm not sure how different this is from BDDC, and it's not very thoroughly analyzed, but this seemed interesting when I read it before:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.ucla.edu/~jteran/papers/MST10.pdf&quot; rel=&quot;nofollow&quot;&gt;A parallel multigrid Poisson solver for fluids simulation on large grids&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12" LastActivityDate="2011-12-02T07:30:29.347" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="176" PostTypeId="2" ParentId="160" CreationDate="2011-12-02T07:41:35.760" Score="14" Body="&lt;p&gt;Well, everything is a trade-off of some sort or another. For random number generators, I group them into 3 basic categories:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Good enough for homework.  &lt;/li&gt;&#xA;&lt;li&gt;Good enough to bet your company on.   &lt;/li&gt;&#xA;&lt;li&gt;Good enough to bet your country on.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Linear congruential PRNGs (the method generally implemented in most libraries) are solidly in category 1. Both Fortuna and Mersenne Twister are solidly in category 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an interesting article on how messing up a shuffling algorithm can cost you your company/casino, I recommend &lt;a href=&quot;http://www.developer.com/java/other/article.php/616221&quot;&gt;this one from 1999&lt;/a&gt;. Due to link rot, the images are gone, but figure 4, the one where you plot next number out of the PRNG against the previous number generated, is a set of parallel lines. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As J.M. points out, Fortuna is slow. As you've pointed out, Mersenne Twister is reasonably fast. &lt;/p&gt;&#xA;" OwnerUserId="16" LastEditorUserId="127" LastEditDate="2011-12-02T19:05:07.750" LastActivityDate="2011-12-02T19:05:07.750" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="177" PostTypeId="2" ParentId="46" CreationDate="2011-12-02T13:20:59.440" Score="7" Body="&lt;p&gt;If you are interested in benchmarking algorithms related to molecular structures, the &lt;a href=&quot;http://pubchem.ncbi.nlm.nih.gov/&quot;&gt;pubchem database&lt;/a&gt; has a large collection of mostly organic molecules. This may be useful to compare predictions of molecular properties obtained with different models/programs. The site has several options for downloading large batches of molecules that satisfy some predefined criteria (e.g. chemical composition).&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2011-12-02T13:20:59.440" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="178" PostTypeId="2" ParentId="39" CreationDate="2011-12-02T13:40:26.670" Score="3" Body="&lt;p&gt;As far as I know, there is no complete library that does what you are looking for, but parts of it are available. &lt;a href=&quot;http://code.google.com/p/mpmath/&quot; rel=&quot;nofollow&quot;&gt;MPMath&lt;/a&gt; is an arbitrary precision library for Python that contains a matrix module with linear algebra functionality. It should not be too difficult to implement a convex solver with this library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note.&lt;/strong&gt; I agree with the comments above. If a numerical problem is terribly ill-conditioned, I'd recommend that you understand why, and try to improve the description of the problem to lower the condition number.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2011-12-02T13:40:26.670" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="179" PostTypeId="2" ParentId="172" CreationDate="2011-12-02T13:49:20.367" Score="5" Body="&lt;p&gt;I strongly advocate a fully coupled assembly since this can easily reproduce the operator split versions. Specifically, the routines which calculate the residual and Jacobian for different physics can be separate, but the framework should be able to combine them to form a unified residual for the entire system. This is how PETSc works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the operator split solutions can be used as a preconditioner for the fully coupled system, or as a solver in its own right, all from the command line. Moreover, some couplings can be preserved, while others can be disgarded. PETSc handles this through the PC FieldSplit interface. This allows replication of hybrid schemes, such as semi-implicit ICE for fluid dynamics.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-02T13:49:20.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="180" PostTypeId="1" AcceptedAnswerId="1255" CreationDate="2011-12-02T13:59:15.417" Score="5" ViewCount="394" Body="&lt;p&gt;I am looking for a method to estimate the contribution from Pauli repulsion to the interaction energy of a molecular dimer in an electronic structure computation (e.g. with Density Functional Theory). I'm aware of &lt;a href=&quot;http://www.physics.udel.edu/~szalewic/SAPT/SAPT.html&quot; rel=&quot;nofollow&quot;&gt;SAPT&lt;/a&gt; and SAPT(DFT) decomposition schemes, but as far as I understand, these estimate the contribution from the exchange interaction (amongst other contributions), which contains both Pauli repulsion and covalent bonding. Are there other energy decomposition schemes (or other methods in general) that can estimate the purely repulsive part of the exchange interaction?&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="127" LastEditDate="2011-12-07T04:49:10.967" LastActivityDate="2012-02-09T15:42:38.287" Title="Pauli repulsion contribution in electronic structure computations" Tags="|computational-chemistry|density-functional-theory|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="181" PostTypeId="1" CreationDate="2011-12-02T14:29:55.947" Score="5" ViewCount="234" Body="&lt;p&gt;I formulate questions about ion densities in biological and materials problems using Classical Density Functional Theory, as in &lt;a href=&quot;http://jcp.aip.org/resource/1/jcpsa6/v132/i12/p124101_s1&quot;&gt;this paper&lt;/a&gt; which is also on the &lt;a href=&quot;http://arxiv.org/abs/0910.1531&quot;&gt;arXiv&lt;/a&gt;. The discretization in that paper for the Rosenfeld functional for hard spheres is just pointwise collocation on a regular grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know of no convergence or stability results for this, or any other discretizations of these equations. Do any such results exist? Moreover, has there been a finite element or finite volume formulation of these equations?&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-30T23:35:20.903" Title="Discretization of Classical Density Functional Theory (CDFT) problems" Tags="|stability|convergence|density-functional-theory|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="182" PostTypeId="1" AcceptedAnswerId="186" CreationDate="2011-12-02T17:28:01.600" Score="14" ViewCount="1997" Body="&lt;p&gt;I was playing around with &lt;a href=&quot;http://mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt; and noticed that when I run my program with more than one process via &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/mpi&quot;&gt;MPI&lt;/a&gt; it seems to run even &lt;em&gt;slower&lt;/em&gt;! How can I check to see what is going on?&lt;/p&gt;&#xA;" OwnerUserId="142" LastActivityDate="2021-04-01T09:19:45.977" Title="Why is my parallel solver slower than my sequential solver?" Tags="|linear-algebra|petsc|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="183" PostTypeId="2" ParentId="182" CreationDate="2011-12-02T17:59:02.927" Score="8" Body="&lt;p&gt;Whenever trying to parallelise a program you have to balance out a number of costs, but primarily there is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The cost of running each computation&lt;/li&gt;&#xA;&lt;li&gt;The cost of any communications between those computations&lt;/li&gt;&#xA;&lt;li&gt;The cost of managing those computations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If your computations are &lt;a href=&quot;http://en.wikipedia.org/wiki/Embarrassingly_parallel&quot;&gt;embarrassingly parallel&lt;/a&gt; then the communications cost will be very low (input and output only) and the management cost should be very low.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have interdependencies between computations, the communications cost can go up significantly. If you have a complex algorithm that takes different time to complete for any given computation, then the management complexity goes up, as you try to efficiently make use of the resources you have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with any form of optimisation, the key is to benchmark. Look at how it performs without MPI, how it performs with MPI and one process, then look at how it scales.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are playing with CUDA, try giving it &lt;em&gt;much&lt;/em&gt; more data. One test here resulted in a negative speedup. We gave it 1000 times more data and the GP-GPU version finished in almost the same time, while the version running on the main CPU took 1000 times as long.&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2011-12-02T17:59:02.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="184" PostTypeId="5" CreationDate="2011-12-02T18:48:39.943" Score="0" Body="&lt;p&gt;It supports MPI, shared memory pthreads, and NVIDIA GPUs, as well as hybrid MPI-shared memory pthreads or MPI-GPU parallelism.  The home page is &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="63" LastEditorUserId="63" LastEditDate="2011-12-02T19:05:48.900" LastActivityDate="2011-12-02T19:05:48.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="185" PostTypeId="4" CreationDate="2011-12-02T18:48:39.943" Score="0" Body="PETSc is an open source suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations." OwnerUserId="63" LastEditorUserId="63" LastEditDate="2011-12-07T17:40:07.383" LastActivityDate="2011-12-07T17:40:07.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="186" PostTypeId="2" ParentId="182" CreationDate="2011-12-02T19:17:45.903" Score="13" Body="&lt;p&gt;This can arise from &lt;strong&gt;architectural&lt;/strong&gt; factors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the same memory bandwidth is available for both one or more processes, then you will see almost no speedup since SpMV and related linear algebra operations are memory bandwidth limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might also be the case that communication overhead overwhelms you local computation. For example, in linear iterative methods we recommend having at least 10,000 unknowns per process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;strong&gt;numerical&lt;/strong&gt; factors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parallel preconditioners are often weaker than their serial counterparts. For example, block Jacobi gets weaker the more blocks you use. Thus, you need to account for extra time spent on extra linear iterations. Nonlinear conditions in general does not work this way, so Newton iterations are often constant. &lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-02T19:17:45.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="187" PostTypeId="1" AcceptedAnswerId="188" CreationDate="2011-12-02T19:59:07.543" Score="57" ViewCount="25934" Body="&lt;p&gt;I recently encountered a case where I needed an integer division operation on a chip that lacked one (ARM Cortex-A8). While trying to research why that must be, I found out that in general division takes many more cycles than addition, subtraction or multiplication on pretty much any integer (or fixed-point) architecture. Why is this the case? Is it not representable with a two-layer AND-OR logic like everything else?&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2012-05-04T19:38:24.170" LastActivityDate="2023-08-13T11:45:06.363" Title="Why is division so much more complex than other arithmetic operations?" Tags="|computer-arithmetic|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="188" PostTypeId="2" ParentId="187" CreationDate="2011-12-02T20:13:13.907" Score="41" Body="&lt;p&gt;Division is an &lt;a href=&quot;http://en.wikipedia.org/wiki/Division_%28digital%29&quot;&gt;iterative algorithm&lt;/a&gt; where the result from the quotient must be shifted to the remainder using a Euclidean measure, see &lt;a href=&quot;http://en.wikipedia.org/wiki/Division_algorithm#Statement_of_theorem&quot;&gt;2&lt;/a&gt;; whereas, multiplication can be reduced to a (fixed) series of bit manipulation tricks.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="30" LastEditDate="2011-12-03T21:52:49.043" LastActivityDate="2011-12-03T21:52:49.043" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="189" PostTypeId="2" ParentId="173" CreationDate="2011-12-02T20:17:47.820" Score="15" Body="&lt;p&gt;There are two main classes of solutions to be discussed in this regard.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&quot;Sufficiently&quot; Smooth Solutions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://dx.doi.org/10.1007/BF01386051&quot; rel=&quot;noreferrer&quot;&gt;Strang's classical paper&lt;/a&gt; it is shown that the Lax equivalence theorem (i.e., the idea that consistency plus stability implies convergence) extends to nonlinear PDE solutions &lt;em&gt;if they have a certain number of continuous derivatives&lt;/em&gt;.  Note that that paper is focused on hyperbolic problems, but the result carries over to parabolic problems.  The number of derivatives needed is a technical point, but this approach is usually applicable to solutions that satisfy the PDE in a strong sense.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Discontinuous solutions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;At the other extreme, we have PDE &quot;solutions&quot; with &lt;strong&gt;discontinuities&lt;/strong&gt;, which typically arise from &lt;strong&gt;nonlinear hyperbolic conservation laws&lt;/strong&gt;.  In this situation, of course, the solution cannot be said to satisfy the PDE in the strong sense, as it is non-differentiable at one or more points.  Instead, a notion of &lt;strong&gt;weak solution&lt;/strong&gt; must be introduced, which essentially amounts to requiring that the solution satisfy an integral conservation law.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proving convergence of a sequence of solutions is also more difficult in this case, as $L_p$-stability is not enough; usually the sequence must be shown to lie in a compact space, such as the set of $L_\infty$ functions with some finite maximum total variation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the sequence can be shown to converge to something, and if the method is conservative, then the Lax-Wendroff theorem guarantees that it will converge to a weak solution of the conservation law.  However, &lt;strong&gt;such solutions are not unique&lt;/strong&gt;.  Determining which weak solution is &quot;correct&quot; requires information that is not contained in the hyperbolic PDE.  Generally, hyperbolic PDEs are obtained by neglecting parabolic terms in a continuum model, and &lt;strong&gt;the correct weak solution can depend on exactly what parabolic terms were discarded&lt;/strong&gt; (this last point is the focus of &lt;a href=&quot;http://arxiv.org/abs/1109.1446&quot; rel=&quot;noreferrer&quot;&gt;the paper linked to in the question above&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a rich and involved topic, and the mathematical theory is far from complete.  Most convergence proofs are for 1D problems and rely on specialized techniques.  Thus nearly all  of the actual computational solutions of hyperbolic conservation laws in practice &lt;strong&gt;cannot be proved convergent&lt;/strong&gt; with existing tools.  For a practical discussion from a computational standpoint, see &lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot; rel=&quot;noreferrer&quot;&gt;LeVeque's book&lt;/a&gt; (chapters 8, 12, and 15); for a more rigorous and detailed treatment I'd suggest &lt;a href=&quot;http://books.google.com/books?id=49bXK26O_b4C&amp;amp;lpg=PA75&amp;amp;ots=8RYfgz3WOJ&amp;amp;dq=dafermos%20hyperbolic%20continuum&amp;amp;pg=PA74#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;Dafermos&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="119" LastEditDate="2013-01-09T16:09:47.760" LastActivityDate="2013-01-09T16:09:47.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="190" PostTypeId="1" CreationDate="2011-12-03T03:33:13.740" Score="18" ViewCount="393" Body="&lt;p&gt;Developing scientific algorithms is a highly iterative process often involving changing lots of parameters that I will want to vary either as part of my experimental design or as part of tweaking algorithm performance. What strategies can I take for structuring these parameters so that I can easily change them between iterations and so that I can easily add new ones?&lt;/p&gt;&#xA;" OwnerUserId="165" LastEditorUserId="123" LastEditDate="2011-12-08T08:23:56.920" LastActivityDate="2011-12-08T08:23:56.920" Title="What programming strategies can I take for easily modifying algorithm parameters?" Tags="|algorithms|testing|programming-paradigms|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="191" PostTypeId="2" ParentId="160" CreationDate="2011-12-03T03:59:57.667" Score="4" Body="&lt;p&gt;The default choice in the &quot;cryptographic&quot; category is &lt;a href=&quot;http://en.wikipedia.org/wiki/Blum-Blum-Shub&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blum-Blum-Shub&lt;/a&gt;, I think. As the wikipedia page says already, this is not suitable for simulations because it's just too darn slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are running on a unix-like system, then you could also consider getting your random numbers directly from &lt;a href=&quot;http://en.wikipedia.org/wiki//dev/urandom&quot; rel=&quot;nofollow noreferrer&quot;&gt;/dev/urandom&lt;/a&gt;, the operating system service that provides good (though not necessarily crypto) quality random numbers. Depending on the particular OS you are using, this may use the Yarrow algorithm - of which Fortuna is a variant. But the most interesting aspect is that the operating system has access to some true random numbers: thermal noise from internal temperature sensors, for example. Typically, this data is mixed into the random pool whenever it becomes available to keep the data unpredictable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This concept of mixing in randomness suggests that it might be possible to get the best of both worlds as follows. Use a faster, reasonably good quality random number generator such as Mersenne as your basic RNG. Maintain a second, better quality random number generator as well - e.g. Fortuna. Every so many numbers, say 25, run one iteration of the better RNG and add the result into the state of your basic RNG. This way you would get fairly high performance and fairly high quality results. (I would guess it would be useless for crypto,  because the strength of this composite generator might well be the strength of the weakest link. But for simulations, where you typically do not have a malicious adversary, it might work.)&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2011-12-03T03:59:57.667" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="192" PostTypeId="2" ParentId="190" CreationDate="2011-12-03T04:56:01.950" Score="15" Body="&lt;p&gt;It is cumbersome for the user to specify every aspect of an algorithm. If the algorithm allows nested components, then no finite number of options would be sufficient. Therefore, it is critical that options do not necessarily &quot;bubble up&quot; to the top level, as in the case of explicit arguments or template parameters. This is sometimes called the &quot;configuration problem&quot; in software engineering. I believe &lt;a href=&quot;http://mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt; has a uniquely powerful system for configuration management. It is similar to the Service Locator pattern in &lt;a href=&quot;http://martinfowler.com/articles/injection.html&quot;&gt;Martin Fowler's essay on inversion of control&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc's configuration system works through a combination of user-specified configuration managed by the solver objects (with get and set queries) and the Options Database. Any component of the simulation can declare a configuration option, a default value, and a place to put the result. Nested objects have prefixes which can be composed, such that every object that needs configuration can be addressed independently. The options themselves can be read from the command line, environment, configuration files, or from code. When an option is declared, a help string and man page are specified, so that the &lt;code&gt;-help&lt;/code&gt; option is understandable and so that a properly linked GUI can be written.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The user calls a &lt;code&gt;SetFromOptions&lt;/code&gt; method to make an object configure itself based on command line options. Calling this function is optional, and may not be called if the user (person writing code that calls PETSc) is exposing the options through some other interface. We highly recommend that the user expose the options database because it gives the end user (person running the application) a great deal of power, but it is not required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical configuration, called via &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PetscObjectOptionsBegin(object); /* object has prefix and descriptive string */&#xA;PetscOptionsReal(&quot;-ts_atol&quot;,                                      /* options database key */&#xA;                 &quot;Absolute tolerance for local truncation error&quot;, /* long description */&#xA;                 &quot;TSSetTolerances&quot;,                               /* function and man page on topic */&#xA;                  ts-&amp;gt;atol,                                       /* current/default value *?&#xA;                  &amp;amp;ts-&amp;gt;atol,                                      /* place to store value */&#xA;                  &amp;amp;option_set);                                   /* TRUE if the option was set */&#xA;PetscOptionsList(&quot;-ts_type&quot;,&quot;Time stepping method&quot;,&quot;TSSetType&quot;,TSList,&#xA;                 defaultType,typeName,sizeof typeName,&amp;amp;option_set);&#xA;TSAdaptSetFromOptions(ts-&amp;gt;adapt);                                 /* configures adaptive controller method */&#xA;/* ... many others */&#xA;/* ... the following is only called from implicit implementations */&#xA;SNESSetFromOptions(ts-&amp;gt;snes);                                     /* configure nonlinear solver. */&#xA;PetscOptionsEnd();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;PetscOptionsList()&lt;/code&gt; presents the user with a choice from a dynamic list. There is a plugin architecture which new implementations can use to expose themselves as first-class to callers. (These implementations can be placed in shared libraries and used as first-class without recompiling programs.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;SNESSetFromOptions()&lt;/code&gt; recursively configures the linear solvers, preconditioners, and any other components that need configuration.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2011-12-03T05:13:36.617" LastActivityDate="2011-12-03T05:13:36.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="193" PostTypeId="2" ParentId="190" CreationDate="2011-12-03T06:25:41.447" Score="11" Body="&lt;p&gt;I've faced this problem several times when developing my own simulation codes from scratch: which parameters should go in an input file, which should be taken from the command line, etc. After some experimenting, the following turned out to be efficient. (It is not as advanced as PETSc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of writing an experimental simulation 'program', I'm more inclined to write a Python package that contains all the functions &amp;amp; classes needed to run the simulation. The traditional input file is then replaced by small Python script with 5 to 10 lines of code. Some lines are typically related to loading data files and specifying output. Others are instructions for the actual computation. Good default values for optional arguments in the Python package make it doable for beginners to use the library for simple simulations, while the advanced user still has access to all the bells and whistles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Atomistic Simulation Environment (also check out GPAW): &lt;a href=&quot;https://wiki.fysik.dtu.dk/ase/&quot;&gt;https://wiki.fysik.dtu.dk/ase/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;TAMkin: &lt;a href=&quot;http://molmod.ugent.be/code/wiki/TAMkin&quot;&gt;http://molmod.ugent.be/code/wiki/TAMkin&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="156" LastActivityDate="2011-12-03T06:25:41.447" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="194" PostTypeId="1" CreationDate="2011-12-03T09:44:54.887" Score="21" ViewCount="2168" Body="&lt;p&gt;I have an application that can be trivially parallelized but its performance is to a large extent I/O bound. The application reads a single input array stored in a file that is typically 2-5 GB in size (but I expect this number to grow in the future). A typical computation applies the same operation to each row or column of that array. For CPU-heavy operations, I get very good scaling up to about 100 processors, but for slower operations I/O and the related communication (NFS access) dominate and I can't use more than a few processors efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are efficient and portable (ideally portably efficient) options for such a situation? Parallel HDF5 seems promising. Does anyone have real-life experience with it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would MPI-I/O be something worth looking into? Can it work efficiently with a given file layout, or do I have to adapt everything?&lt;/p&gt;&#xA;" OwnerUserId="140" LastEditorUserId="127" LastEditDate="2011-12-12T17:03:52.560" LastActivityDate="2011-12-17T19:54:19.083" Title="Parallel I/O options, in particular parallel HDF5" Tags="|io|parallel-computing|mpi|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="195" PostTypeId="2" ParentId="182" CreationDate="2011-12-03T11:02:46.723" Score="3" Body="&lt;p&gt;I would recommend you do the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Make a profile of the time execution of your code, with and without parallelization. If you have doubts about how to do this, we can help you if you describe your code better.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can now focus on the parts that run slower in parallel. You should be aware that communication between processes may be slow. Like Mark and Sean pointed out, just because a problem can be divided into threads doesn't mean that doing so will be efficient. You have to look into it more deeply. But if you profile your code, it may help you to find any existing bugs. My two cents.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you explain what you are doing in more detail, e.g. with a workflow, someone may be able to give you a better explanation.&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="123" LastEditDate="2011-12-03T18:09:18.237" LastActivityDate="2011-12-03T18:09:18.237" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="196" PostTypeId="5" CreationDate="2011-12-03T11:40:29.633" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-03T11:40:29.633" LastActivityDate="2011-12-03T11:40:29.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="197" PostTypeId="4" CreationDate="2011-12-03T11:40:29.633" Score="0" Body="Questions on the testing and verification of computational methods." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-12-04T06:06:07.103" LastActivityDate="2011-12-04T06:06:07.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="198" PostTypeId="2" ParentId="46" CreationDate="2011-12-03T11:56:55.007" Score="6" Body="&lt;p&gt;For testing statistical algorithms, there is &lt;a href=&quot;http://www.crcpress.com/product/isbn/9780412399206&quot;&gt;&lt;em&gt;A Handbook of Small Data Sets&lt;/em&gt;&lt;/a&gt; by D.J. Hand, F. Daly, K. McConway, D. Lunn, and E. Ostrowski. Some of those data sets can be downloaded from &lt;a href=&quot;http://www.stat.ncsu.edu/working_groups/sas/sicl/data/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2011-12-03T11:56:55.007" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="199" PostTypeId="2" ParentId="190" CreationDate="2011-12-03T12:08:07.190" Score="2" Body="&lt;p&gt;As a first point, I would do the algorithm AND software as general as possible. I've learned this the hard way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you start with a simple test case. You can do this faster. But then, if you made the software too specific (too few parameters) for this initial case, you'll loose more and more time adapting it every time you add a new degree of freedom. What I do now it's spend more time at the beginning  making the thing pretty general, and increasing the variation of the parameters as I move forward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This involves more testing from the beginning since you'll have more parameters from the starting point, but will mean that you can latter play a lot with the algorithm at zero or a very low cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: the algorithm involves calculating the surface integral the dot product of two vector functions. Don't assume from the beginning the size, geometry and discretization of the surface if in the future you may want to change that. Make a dot-product function, make the surface as general as possibly, calculate the integral in a nice formal way. You can test each function you make separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the beginning, you can and start integrating over simple geometries and declaring may parameters at the start as constants. As time goes by, if you want to change the geometry, you can do it easily. Had you made assumptions at the beginning, you would have to change the whole code every time.&lt;/p&gt;&#xA;" OwnerUserId="157" LastActivityDate="2011-12-03T12:08:07.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="200" PostTypeId="2" ParentId="46" CreationDate="2011-12-03T16:25:33.443" Score="7" Body="&lt;p&gt;Arnold Neumaier maintains &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/test.html&quot;&gt;a stable of test problems&lt;/a&gt; for unconstrained and constrained optimization (nonlinear programming). Included in this collection are the now &lt;a href=&quot;http://dx.doi.org/10.1145/355934.355936&quot;&gt;standard test problems for unconstrained optimization&lt;/a&gt; due to Moré, Garbow, and Hillstrom.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2011-12-03T16:25:33.443" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="201" PostTypeId="5" CreationDate="2011-12-03T16:37:35.020" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-03T16:37:35.020" LastActivityDate="2011-12-03T16:37:35.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="202" PostTypeId="4" CreationDate="2011-12-03T16:37:35.020" Score="0" Body="Questions on the algorithmic/computational aspects of linear algebra, including the solution of linear systems, least squares problems, eigenproblems, and other such matters." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-12-04T06:06:05.167" LastActivityDate="2011-12-04T06:06:05.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="203" PostTypeId="1" AcceptedAnswerId="216" CreationDate="2011-12-03T16:57:11.553" Score="51" ViewCount="31593" Body="&lt;p&gt;Fortran has a special place in numerical programming. You can certainly make good and fast software in other languages, but Fortran keeps performing very well despite its age. Moreover, it's easier to make fast programs in Fortran. I've made fast programs in C++, but you have to be more careful about things like pointer aliasing. So, there has to be a reason for this, and a very technical one. Is it because the compiler can optimize more? I would really like to know technical details, so if I use another language I can take these things into consideration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I know -or so I think- that one thing is that the standard specifies that pointers are contiguous in memory always which means faster memory access. I believe you can do this in C++ by giving a flag to the compiler. In this way it helps to know what Fortran does good, so that if using another language we can imitate this.&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="41" LastEditDate="2011-12-06T14:21:21.673" LastActivityDate="2015-12-19T13:06:21.870" Title="What makes Fortran fast?" Tags="|performance|fortran|languages|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="205" PostTypeId="2" ParentId="194" CreationDate="2011-12-03T17:47:33.213" Score="5" Body="&lt;p&gt;We get good scaling up to the entire XT6 at ORNL using MPI/IO to output vectors. Here is the &lt;a href=&quot;http://petsc.cs.iit.edu/petsc/petsc-dev/annotate/c57880272773/src/vec/vec/impls/mpi/pdvec.c#l402&quot; rel=&quot;nofollow&quot;&gt;code&lt;/a&gt;. The I/O subsystems for many machines are not designed for massive parallelism, so I think @Dan is correct that I would try to minimize IO by only writing every few steps, or some other agglomeration strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as flexibly writing output in a scalable way, I have experience with &lt;a href=&quot;http://www.xdmf.org/index.php/Main_Page&quot; rel=&quot;nofollow&quot;&gt;XDMF&lt;/a&gt;, which can be effected by large parallel binary writes using HDF5 (like PETSc &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecView.html&quot; rel=&quot;nofollow&quot;&gt;VecView&lt;/a&gt;) coupled with a small amount of XML code written in serial to describe the layout. This can be read by visualization packages like &lt;a href=&quot;http://www.paraview.org/&quot; rel=&quot;nofollow&quot;&gt;Paraview&lt;/a&gt; or &lt;a href=&quot;http://code.enthought.com/projects/mayavi/&quot; rel=&quot;nofollow&quot;&gt;MayaVi2&lt;/a&gt;. Another way to do this is use the VTK format with appended binary data, however this requires that you know everything you want to write up front.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="21" LastEditDate="2011-12-03T20:55:02.703" LastActivityDate="2011-12-03T20:55:02.703" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="206" PostTypeId="1" AcceptedAnswerId="237" CreationDate="2011-12-03T17:58:13.287" Score="105" ViewCount="11413" Body="&lt;p&gt;I am strongly convinced of the value of using tests that verify a complete program (e.g. convergence tests), including an automated set of &lt;strong&gt;regression tests&lt;/strong&gt;.  After reading some programming books, I've gotten the nagging feeling that I &quot;ought to&quot; write &lt;strong&gt;unit tests&lt;/strong&gt; (i.e., tests that verify the correctness of a single function and do not amount to running the whole code to solve a problem) as well.  However, unit tests don't always seem to &lt;em&gt;fit&lt;/em&gt; with scientific codes, and end up feeling artificial or like a waste of time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should we write unit tests for research codes?&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="41" LastEditDate="2012-01-09T01:57:10.417" LastActivityDate="2022-06-01T23:01:21.997" Title="Is it worthwhile to write unit tests for scientific research codes?" Tags="|programming-paradigms|testing|" AnswerCount="14" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="207" PostTypeId="2" ParentId="46" CreationDate="2011-12-03T18:30:51.587" Score="4" Body="&lt;p&gt;For testing multivariate statistical analyses and machine learning algorithms, there is the UCI dataset repository at &lt;a href=&quot;http://www.ics.uci.edu/~mlearn/&quot; rel=&quot;nofollow&quot;&gt;http://www.ics.uci.edu/~mlearn/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2011-12-03T18:30:51.587" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="208" PostTypeId="2" ParentId="203" CreationDate="2011-12-03T18:54:41.550" Score="12" Body="&lt;p&gt;I don't think Fortran is that close to the metal (see other answer) but it tends to optimize very easily. Loops are simple, and the language readily supports vectorization extensions (okay when I used it in my first job we were targeting a wide range of vector big iron).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also the large factor of inertia. A lot of numeric code is in Fortran, so builders of high end servers and supercomputers make sure they write good optimizing Fortran compilers. The compilers are good (even on machines with a relative lack of high quality compilers) so the users continue to use the Fortran and even write new code in it. So the builders make sure their next generation have good compilers, etc...&lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2011-12-03T18:54:41.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="209" PostTypeId="2" ParentId="206" CreationDate="2011-12-03T19:06:49.820" Score="40" Body="&lt;p&gt;Scientific codes tend to have constellations of interlocking functions more often than the business codes I have worked on, usually due to the mathematical structure of the problem. So, I do not think unit tests for individual functions are very effective. However, I do think there is a class of unit tests which are effective, and are still quite different from whole program tests in that they target specific functionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just briefly define what I mean by these kinds of tests. &lt;a href=&quot;http://en.wikipedia.org/wiki/Regression_testing&quot;&gt;Regression testing&lt;/a&gt; looks for changes in existing behavior (validated somehow) when changes are made to the code. &lt;a href=&quot;http://en.wikipedia.org/wiki/Unit_test&quot;&gt;Unit testing&lt;/a&gt; runs a piece of code and checks that it gives a desired output based upon a specification. They are not that different, as the original regression test &lt;em&gt;was&lt;/em&gt; a unit test since I had to determine that the output was valid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My favorite example of a numerical unit test is testing the convergence rate of a finite element implementation. It is definitely not simple, but it takes a known solution to a PDE, runs several problems at decreasing mesh size $h$, and then fits the error norm to the curve $C h^r$ where $r$ is the convergence rate. I do this for the Poisson problem in PETSc using Python. I am not looking for a difference, as in regression, but a particularly rate $r$ specified for the given element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two more examples of unit testing, coming from &lt;a href=&quot;http://geodynamics.org/cig/software/pylith&quot;&gt;PyLith&lt;/a&gt;, are point location, which is a single function which is easy to produce synthetic results for, and creation of zero volume cohesive cells in a mesh, which involves several functions but addresses a circumscribed piece of functionality in the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many tests of this kind, including conservation and consistency tests. The operation is not that different from regression (you run a test and check the output against a standard), but the standard output comes from a specification rather than a previous run.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="21" LastEditDate="2011-12-03T20:22:38.023" LastActivityDate="2011-12-03T20:22:38.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="212" PostTypeId="1" AcceptedAnswerId="218" CreationDate="2011-12-03T20:16:28.993" Score="4" ViewCount="141" Body="&lt;p&gt;What efforts have been made to quantify the costs of the common numeric workhorse operations such as those found in LAPACK, BLAS, sparse matrix operations, etc....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have several libraries which implement identical interfaces found in BLAS and LAPACK. I suspect these all have different strengths and weaknesses on various architectures. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has there been any effort to compare them as a group for a particular sequential architecture? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has there been any effort to do this on more complex architectures, i.e. has anyone compared the various GPU or multi-core library implementations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general each library has it's own set of papers comparing a few operations with another (often less sophisticated) library. I'm looking for work which does a more substantial and broad review. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a perfect world we would like to be able to as questions like &quot;for an i7 chip with 4GB memory and roughly this memory hierarchy the LU decomposition is 20% faster using library X over library Y&quot;. Profiling every algorithm on every architecture is obviously infeasible. What work is feasible? Do we know generally when library X is preferable to library Y without understanding each library in depth?&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2011-12-03T20:27:03.153" LastActivityDate="2011-12-03T21:44:13.230" Title="Profiling numeric libraries" Tags="|libraries|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="213" PostTypeId="1" AcceptedAnswerId="214" CreationDate="2011-12-03T21:12:00.400" Score="8" ViewCount="1452" Body="&lt;p&gt;How can I compute the Schur complement: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ S = K_{bb} - K_{ba} K_{aa}^{-1} K_{ab} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ K=\begin{pmatrix} K_{aa} &amp;amp; K_{ab} \\ K_{ba} &amp;amp; K_{bb} \end{pmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(in some ordering) is a PETSc matrix (&lt;code&gt;Mat&lt;/code&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="7310" LastEditDate="2018-09-16T23:43:50.033" LastActivityDate="2018-09-17T15:12:52.557" Title="How can I compute the Schur complement in PETSc?" Tags="|pde|linear-algebra|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="214" PostTypeId="2" ParentId="213" CreationDate="2011-12-03T21:17:19.317" Score="13" Body="&lt;p&gt;It is very expensive to compute the Schur complement of a matrix and very rarely needed in practice. PETSc highly recommends avoiding algorithms that need it. The Schur complement of a matrix (dense or sparse) is essentially always dense, so begin by:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;forming a dense matrix $K_{ba}$, &lt;/li&gt;&#xA;&lt;li&gt;also create another dense matrix $T$ of the same size. &lt;/li&gt;&#xA;&lt;li&gt;Then either factor the matrix $K_{aa}$  directly with &lt;code&gt;MatLUFactor()&lt;/code&gt; or &lt;code&gt;MatCholeskyFactor()&lt;/code&gt;, or use &lt;code&gt;MatGetFactor()&lt;/code&gt; followed by &lt;code&gt;MatLUFactorSymbolic()&lt;/code&gt; followed by &lt;code&gt;MatLUFactorNumeric()&lt;/code&gt; if you wish to use and external solver package like SuperLU_Dist. Call the result &lt;code&gt;A&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Then call &lt;code&gt;MatMatSolve(A,Kba,T)&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Then call &lt;code&gt;MatMatMult(Kab,T,MAT_INITIAL_MATRIX,1.0,&amp;amp;S)&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Now call &lt;code&gt;MatAXPY(S,-1.0,Kbb,MAT_SUBSET_NONZERO)&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Followed by &lt;code&gt;MatScale(S,-1.0)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For computing Schur complements like this it does not make sense to use the &lt;code&gt;KSP&lt;/code&gt; iterative solvers since for solving many moderate size problems using a direct factorization is much faster than iterative solvers. As you can see, this requires a great deal of work space and computation so is best avoided. However, it is not necessary to assemble the Schur complement $S$ in order to solve systems with it. Use &lt;code&gt;MatCreateSchurComplement(Kaa,Kaa_pre,Kab,Kba,Kbb,&amp;amp;S)&lt;/code&gt; to create a matrix that applies the action of &lt;code&gt;S&lt;/code&gt; (using &lt;code&gt;Kaa_pre&lt;/code&gt; to solve with &lt;code&gt;Kaa&lt;/code&gt;), but does not assemble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you already have a block matrix $$ K = \begin{pmatrix} K_{aa} &amp;amp; K_{ab} \\ K_{ba} &amp;amp; K_{bb} \end{pmatrix} $$ (in some ordering), then you can create index sets (&lt;code&gt;IS&lt;/code&gt;) isa and isb to address each block, then use &lt;code&gt;MatGetSchurComplement()&lt;/code&gt; to create the Schur complement and/or an approximation suitable for preconditioning. Since $S$ is generally dense, standard preconditioning methods cannot typically be applied directly to Schur complements. There are many approaches to preconditioning Schur complements including using the SIMPLE approximation $ K_{bb} - K_{ba} \operatorname{diag}(K_{aa})^{-1} K_{ab} $ to create a sparse matrix that approximates the Schur complement (this is returned by default for the optional &quot;preconditioning&quot; matrix in &lt;code&gt;MatGetSchurComplement()&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative is to interpret the matrices as differential operators and apply approximate commutator arguments to find a spectrally equivalent operation that can be applied efficiently (see the &quot;PCD&quot; preconditioners from Elman, Silvester, and Wathen). A variant of this is the least squares commutator, which is closely related to the Moore-Penrose pseudoinverse, and is available in &lt;code&gt;PCLSC&lt;/code&gt; which operates on matrices of type &lt;code&gt;MATSCHURCOMPLEMENT&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="142" LastEditorUserId="20688" LastEditDate="2018-09-17T15:12:52.557" LastActivityDate="2018-09-17T15:12:52.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="215" PostTypeId="2" ParentId="172" CreationDate="2011-12-03T21:26:15.753" Score="5" Body="&lt;p&gt;Since this is a very active area of research, I hesitate to attempt an answer to this but I have some experience on what &lt;em&gt;not&lt;/em&gt; to try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do Not&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Take old application code A and old application B, then try to couple them together&lt;/li&gt;&#xA;&lt;li&gt;Use archaic (implying unusable in hindsight) code, instead of building a new application&lt;/li&gt;&#xA;&lt;li&gt;Require a huge framework (&gt; 10 required dependencies) on new users beginning to contribute&lt;/li&gt;&#xA;&lt;li&gt;Assume the data layout (meshes, matrices, vectors, etc.) is easy to write yourself&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use standard programming practices and, hopefully, good design patterns&lt;/li&gt;&#xA;&lt;li&gt;Read the literature on operator splitting to understand the limitations of accuracy and stability&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="142" LastActivityDate="2011-12-03T21:26:15.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="216" PostTypeId="2" ParentId="203" CreationDate="2011-12-03T21:37:22.530" Score="56" Body="&lt;p&gt;Language designers face many choices.  &lt;a href=&quot;https://en.wikipedia.org/wiki/Ken_Kennedy_%28computer_scientist%29&quot;&gt;Ken Kennedy&lt;/a&gt; emphasized two: (1) better abstractions and (2) higher- or lower-level (less or more machine-like) code.  While functional languages like Haskell and Scheme focus on the former, traditional scientific-computing languages like Fortran and C/C++ focused on the latter.  Saying that one language is faster than another is usually quite misleading: each language has a problem domain for which it excels. Fortran fares better in the domain of array-based numerical codes than other languages for two basic reasons: its array model and its explicitness.  &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Array Model&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Fortran programmers largely do array manipulations. For that, Fortran facilitates several compiler optimizations that are not available in other languages. The best example is vectorization: knowing the data layout enables the compiler to invoke assembly-level intrinsics over the array.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Language Explicitness&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;While it seems that a simpler language should compile &quot;better&quot; than a more complex one, that really isn't the case.  When one writes in an &lt;a href=&quot;https://en.wikipedia.org/wiki/Assembly_language&quot;&gt;assembly language&lt;/a&gt;, there isn't much a compiler can do: all it sees are very-fine-grained instructions. Fortran requires explicitness (thus, more work by the programmer) only in cases that yield real rewards for array-based computing. Fortran uses simple data types, basic control flow, and limited namespaces; by contrast, it does &lt;em&gt;not&lt;/em&gt; tell the computer how to load registers (which might be necessary for &lt;a href=&quot;https://en.wikipedia.org/wiki/Real-time_computing&quot;&gt;real-time&lt;/a&gt;).  Where Fortran &lt;em&gt;is&lt;/em&gt; explicit, it enables things like complete type inference, which helps novices to get started.  It also avoids one thing that often makes C slow: &lt;a href=&quot;https://en.wikipedia.org/wiki/Opaque_pointer&quot;&gt;opaque pointers&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Fortran Can Be Slow&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Fortran is &lt;em&gt;not&lt;/em&gt; fast for every task: that's why not many people use it for building GUIs or even for highly unstructured scientific computing.  Once you leave the world of arrays for graphs, decision trees, and other realms, this speed advantage quickly goes away.  See the &lt;a href=&quot;http://shootout.alioth.debian.org/&quot;&gt;computer language benchmarks&lt;/a&gt; for some examples and numbers.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="30" LastEditDate="2015-12-19T13:06:21.870" LastActivityDate="2015-12-19T13:06:21.870" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="217" PostTypeId="1" AcceptedAnswerId="234" CreationDate="2011-12-03T21:41:43.117" Score="5" ViewCount="240" Body="&lt;p&gt;I have a Cartesian mesh in $d$ dimensions, and I would like to enumerate all the subcells of a given hexahedral cell. If I am just enumerating the vertices of a cell (or cells that contain a vertex) I can use &lt;a href=&quot;http://en.wikipedia.org/wiki/Gray_code&quot; rel=&quot;nofollow&quot;&gt;Gray code&lt;/a&gt;, here is my &lt;a href=&quot;http://petsc.cs.iit.edu/petsc/petsc-dev/annotate/f99e0009e573/include/sieve/CartesianSieve.hh#l219&quot; rel=&quot;nofollow&quot;&gt;code&lt;/a&gt; doing that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that this works because the Gray code sequence traverses the vertices of a &lt;a href=&quot;http://en.wikipedia.org/wiki/Hypercube&quot; rel=&quot;nofollow&quot;&gt;hypercube&lt;/a&gt;, which is exactly what a $d$-dimensional hexahedralcell is. If the mesh is a tensor product (Cartesian), then the dual mesh is also made up of hypercubes and we can traverse cells around a vertex using the same algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this strategy be extended to handle edges and faces? Or, what strategy could handle them in a dimension independent way?&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="21" LastEditDate="2011-12-04T22:39:22.440" LastActivityDate="2011-12-16T03:38:46.360" Title="Enumerating hexahedral cell vertices and faces in arbitrary dimension" Tags="|high-dimensional|mesh-traversal|combinatorics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="218" PostTypeId="2" ParentId="212" CreationDate="2011-12-03T21:44:13.230" Score="4" Body="&lt;p&gt;Not only has there been effort, there is millions of dollars a year poured into the benchmarking by industry every year to prove their machine is better.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://icl.cs.utk.edu/hpcc/&quot; rel=&quot;nofollow&quot;&gt;HPC Challenge Benchmarks&lt;/a&gt; are a set of programs that can be reimplemented by and submitted to the HPCC board.  They give out reports on each architecture each year.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now comparing various library implementations is often a challenge, and probably something that library developers don't &lt;em&gt;really&lt;/em&gt; want.  If you design a library to do X and another library beats you at Y, then why do you care.  As a user I'm quite skeptical of such comparisons as well.  They usually miss key features of a library.&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2011-12-03T21:44:13.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="219" PostTypeId="2" ParentId="206" CreationDate="2011-12-03T22:13:50.663" Score="5" Body="&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea that any code is written without unit tests is anathema. Unless you prove your code correct and then prove the proof correct =P.&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2011-12-03T22:13:50.663" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="220" PostTypeId="2" ParentId="206" CreationDate="2011-12-03T22:52:41.850" Score="21" Body="&lt;p&gt;In my experience, as the complexity of scientific research codes increases, there is a need to have a very modular approach in programming. This can be painful for codes with a large and ancient based (&lt;code&gt;f77&lt;/code&gt; anyone?) but it is necessary moving forward. As a module gets built around a specific aspect of the code (for CFD applications, think Boundary Conditions or Thermodynamics), unit testing is very valuable to validate the new implementation and isolate issues and further software developments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These unit tests should be one level below code &lt;strong&gt;verification&lt;/strong&gt; (can I recover the analytical solution of my wave equation?) and 2 levels below code &lt;strong&gt;validation&lt;/strong&gt; (can I predict the correct peak RMS values in my turbulent pipe flow), simply insuring that the programming (are the arguments correctly passed, are the pointers pointing to the right thing?) and the &quot;math&quot; (this subroutine computes the friction coefficient. If I input a set of numbers and compute the solution by hand, does the routine yield the same result?) are correct. Basically going one level above what the compilers can spot, i.e. basic syntax errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would definitely recommend it for at least some crucial modules in your application. However, one has to realize that it is extremely tedious and time-consuming so unless you have unlimited man-power, I wouldn't recommend it for 100% of a complex code.&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2011-12-03T22:52:41.850" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="221" PostTypeId="2" ParentId="203" CreationDate="2011-12-03T23:45:57.110" Score="27" Body="&lt;p&gt;The design of Fortran allows the compiler to perform stronger optimizations in some cases, optimizations that are not generally available to C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One famous example is the handling of &lt;em&gt;aliasing&lt;/em&gt;. In Fortran, you can access a specific memory area only though the specific symbol associated with that memory area. This knowledge allows the compiler to employ smart tricks when it's time to cache: it knows if a value has potentially changed or not. Until F90, this was verified easily. When Fortran 90 introduced &lt;code&gt;pointers&lt;/code&gt;, the assumption was no longer true: you could access the same memory area through two (or more) symbols. This is the reason why you have to specify &lt;code&gt;target&lt;/code&gt; to arrays you want to address via pointers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting fact is that many constructs allow the compiler to perform parallelization without user intervention. Such luxury is possible due to the relative &quot;platform agnosticism&quot; of Fortran as a language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many other subtle tricks like this. In addition, remember that nobody uses Fortran today, except for numerical calculations, meaning that the core feature and selling point of Fortran compilers is the resulting code speed. As a result, vendors focused on this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, you can produce performant code also with other languages. It may, however, require special care or human intervention. The general point, however, is that performance is not a problem until there's a problem, and man time is way more expensive than computer time. So coding practices should focus on saving human time, rather than computer time. &lt;/p&gt;&#xA;" OwnerUserId="23" LastEditorUserId="88" LastEditDate="2011-12-07T17:39:27.663" LastActivityDate="2011-12-07T17:39:27.663" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="222" PostTypeId="2" ParentId="206" CreationDate="2011-12-03T23:53:16.860" Score="8" Body="&lt;p&gt;Yes and no. Certainly unittest for fundamental routines of the basic toolset you use to make your life easier, such as conversion routines, string mappings, basic physics and math, etc. When it comes to calculations classes or functions, they may generally require long run times, and you may actually prefer to test them as functional tests, instead of as units. Also, unittest and stress a lot those classes and entities whose level and use are going to change a lot (e.g. for optimization purposes) or whose internal details are going to be changed for whatever reason. The most typical example is a class wrapping a huge matrix, mapped from disk.&lt;/p&gt;&#xA;" OwnerUserId="23" LastActivityDate="2011-12-03T23:53:16.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="223" PostTypeId="1" CreationDate="2011-12-03T23:57:38.183" Score="33" ViewCount="15622" Body="&lt;p&gt;ATLAS is a free BLAS/LAPACK replacement that tunes itself to the machine when compiled. MKL is the commercial library shipped by Intel. Are these two libraries comparable when it comes to performance, or does MKL have the upper hand for some tasks? If so, which ones? &lt;/p&gt;&#xA;" OwnerUserId="23" LastEditorUserId="127" LastEditDate="2011-12-04T06:06:02.983" LastActivityDate="2018-09-17T01:07:39.613" Title="Performance differences between ATLAS and MKL?" Tags="|linear-algebra|libraries|performance|intel-mkl|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="224" PostTypeId="2" ParentId="223" CreationDate="2011-12-04T00:21:19.607" Score="14" Body="&lt;p&gt;BLAS is not monlithic. BLAS1 and BLAS2 are memory bandwidth limited, and there is not much you can do to speed them up beyond the obvious (loop unrolling, cache blocking for level 2). BLAS3 is more interesting and the prototypical benchmark here is matrix-matrix multiplication. To my knowledge GOTOBlas has always been the clear winner here, see for example &lt;a href=&quot;http://www.ats.ucla.edu/clusters/common/software/libraries/blas_benchmark.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;this comparison&lt;/a&gt; or &lt;a href=&quot;http://www.cs.fsu.edu/~engelen/courses/HPC/Tools.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; and this &lt;a href=&quot;http://dx.doi.org/10.1145/1377603.1377607&quot; rel=&quot;nofollow noreferrer&quot;&gt;justification&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is now years later, and we have the &lt;a href=&quot;https://github.com/flame/blis&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLIS Project&lt;/a&gt;. It is the best free alternative to MKL.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="21" LastEditDate="2018-09-17T01:07:39.613" LastActivityDate="2018-09-17T01:07:39.613" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="225" PostTypeId="1" CreationDate="2011-12-04T01:03:59.430" Score="3" ViewCount="963" Body="&lt;p&gt;I am using a basic singular value decomposition (via LAPACK) routine in FORTRAN to solve an overdetermined system in the form of $A\cdot X = B$ where $\mathrm{size}(A) = [m,n]$ with $m &amp;gt; n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My sample data points come from a noisy sine function and I am trying to use linear regression with $x^i$ as my basis functions. I find that (with the noisy sine function) I get very good approximations when I keep my polynomial low. That is, when I fit a function of the form &#xA;$$&#xD;&#xA;a_0 + a_1 x + a_2 x^2 + \ldots + a_Nx^N&#xD;&#xA;$$&#xA;with $N \lesssim 10$, I get great results. When I allow $N$ to get higher (I have 1000 data points), say, up to 250, my fit goes to hell and my sum of squares is $\approx 5.6\times 10^{41}$ (as opposed to about 200 with $N \lesssim 10$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does this happen? If a degree 6 polynomial, for example, provides the best fit, then shouldn't a degree $N$ polynomial produce that same fit with $0$ for $a_7 \ldots a_N$?&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="127" LastEditDate="2011-12-05T19:42:13.890" LastActivityDate="2011-12-05T19:42:13.890" Title="Linear regression via SVD not producing best fit with escalating polynomial degree" Tags="|linear-algebra|regression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="226" PostTypeId="2" ParentId="225" CreationDate="2011-12-04T01:12:13.060" Score="6" Body="&lt;p&gt;The problem here is that the condition number for your monomial basis becomes very large, meaning that the value of the monomial is very sensitive to the value of the coefficient. Thus, when you try to compute in this basis for high degree polynomials, your answer is very inaccurate. There are polynomial bases that are better behaved, like any of the families of orthogonal polynomials.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-04T01:12:13.060" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="227" PostTypeId="2" ParentId="206" CreationDate="2011-12-04T02:24:52.730" Score="5" Body="&lt;p&gt;I've used unit testing to good effect on several small scale (i.e. single programmer) codes, including the &lt;em&gt;third&lt;/em&gt; version of my dissertation analysis code in particle physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first two versions had collapsed under their own weight and the multiplication of interconnections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other have written that the interaction between modules are often the place where scientific coding breaks, and they're right about that. But it is a lot easier to diagnose &lt;em&gt;those&lt;/em&gt; problems when you can show conclusively that each module &lt;em&gt;is&lt;/em&gt; doing what it is meant to do.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="183" LastEditDate="2016-08-28T17:19:11.243" LastActivityDate="2016-08-28T17:19:11.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="228" PostTypeId="2" ParentId="159" CreationDate="2011-12-04T02:34:27.577" Score="19" Body="&lt;p&gt;One of the projects I'm involved in right now is writing a flexible simulation and analysis package for a class of particle physics detectors. One of the goals of this project is to provide &lt;em&gt;the&lt;/em&gt; code base to be used in these things for decades to come.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point point we already have &lt;em&gt;two dozen&lt;/em&gt; dependencies, making the build process such a nightmare that it has spun off a separate project managed out of the Fermilab computing center just to provide a reliable tool-chain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine that you encounter a need for some tool that &lt;em&gt;isn't&lt;/em&gt; in  that tool-chain (happened to me just last month). You have three choices&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Roll you own. With all the risks and hassles that involves.&lt;/li&gt;&#xA;&lt;li&gt;Scrape some code out of a library somewhere and include it directing in The Project. Meaning that you have to take on maintenance going forward, and that you'll have to understand someone else's code when that happens.&lt;/li&gt;&#xA;&lt;li&gt;Go to the people who maintain the tool-chain, beg them for what you need and then &lt;em&gt;wait&lt;/em&gt; for a release cycle to get it. These guys are pretty responsive, but you have to make the case for it without working code &lt;em&gt;or&lt;/em&gt; after you've done (1) or (2).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It's very easy to choose (1). Maybe too easy.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="183" LastEditDate="2011-12-04T13:58:21.110" LastActivityDate="2011-12-04T13:58:21.110" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="229" PostTypeId="1" AcceptedAnswerId="1263" CreationDate="2011-12-04T03:03:44.677" Score="12" ViewCount="663" Body="&lt;p&gt;In a particular class of detectors, our data comes out as pairs of points in two dimensions, and we want to string these points into lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data is noisy, and is binned in one direction but not in the other. We can't guarantee a hit in every bin even when each detector element is working, so there may be skips.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our current analysis chain looks like&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Adjust hits for the calibration of individual detector elements&lt;/li&gt;&#xA;&lt;li&gt;Find clusters&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Rough fit lines to the clusters&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Connect up clusters into longer line-like structures&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This question concerns step (3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We've been using a Hough transform for that step and it works well, but as we try to scale up from the test-bed to simulation of a full-scale project it becomes unacceptably slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a faster way.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For those who might care the actual use case here is a Liquid Argon Time-Projection Chamber&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="20688" LastEditDate="2019-06-22T19:21:18.070" LastActivityDate="2019-06-22T19:21:18.070" Title="Quickly finding rough lines in sets of points" Tags="|regression|computer-vision|pattern-recognition|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="230" PostTypeId="2" ParentId="225" CreationDate="2011-12-04T03:10:38.213" Score="3" Body="&lt;p&gt;Sections of &lt;a href=&quot;https://en.wikipedia.org/wiki/Vandermonde_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vandermonde matrices&lt;/a&gt; do tend to be ill conditioned. If indeed you were using the singular value decomposition for your fitting, you should have already seen a rash of tiny singular values. Did you remember to zero out those tiny singular values before computing the least squares solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ill-condition is easily visualized: if you plot successive members of the family $x^k$ for increasing $k$, you'll find that the members look almost indistinguishable for high enough $k$. This manifests itself in the columns becoming nearly linearly dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, like Matt says, you can use orthogonal polynomials. (I talked about them &lt;a href=&quot;https://mathoverflow.net/questions/38864/38900#38900&quot;&gt;here&lt;/a&gt;, but I suppose I should say something here for convenience.) If your data points are equispaced in the abscissas, you can use the Gram polynomials as a basis. If they are irregularly spaced, a fair bit of work is needed; see &lt;a href=&quot;http://epubs.siam.org/siap/resource/1/smjmap/v5/i2/p74_s1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Forsythe's paper&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2011-12-04T04:44:47.170" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="231" PostTypeId="2" ParentId="63" CreationDate="2011-12-04T04:14:34.850" Score="1" Body="&lt;p&gt;In a recent &lt;a href=&quot;http://10.2514/1.J050232&quot; rel=&quot;nofollow&quot; title=&quot;Georgiadis et al.&quot;&gt;paper&lt;/a&gt;, Georgiadis et al. suggested a useful albeit limited approach. He first made the astute remark that usually, LES researchers run the finest mesh they can afford when dealing with somewhat complex geometries. Which leaves little leeway for exhaustive grid influence study. Note that I'm not using grid independence here since for an implicit LES this is mathematically absurd: changing the grid also affect your turbulence modeling, not just the numerics. Anyway, he suggested the following test:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If a solution has been obtained on a baseline computational mesh, then a coarser grid constructed by removing every other mesh point in each coordinate direction, will result in a solution requiring only one-eighth of the resources (assuming the time step is held constant) as that of the baseline. We believe this is a result worth obtaining.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;He argues that if the results are qualitatively comparable, this is a good indication that the baseline LES is at least valid. On the other hand, if the coarse mesh simply crashes or yields obvious unphysical results, this might mean the original LES is not valid and requires more resolution.&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2011-12-04T04:14:34.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="232" PostTypeId="1" AcceptedAnswerId="233" CreationDate="2011-12-04T08:51:50.510" Score="3" ViewCount="167" Body="&lt;p&gt;I have a 50-base, artificially created RNA sequence and I would like to obtain its 3D structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What software should I use for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="102" LastEditorUserId="127" LastEditDate="2011-12-07T04:48:45.413" LastActivityDate="2011-12-07T04:48:45.413" Title="Artificial RNA 3D folding" Tags="|computational-biology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="233" PostTypeId="2" ParentId="232" CreationDate="2011-12-04T15:19:16.360" Score="4" Body="&lt;p&gt;I suggest either Shapiro's RNA2D3D or MC-FOLD:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RNA2D3D -&gt; &lt;a href=&quot;http://www-lmmb.ncifcrf.gov/~bshapiro/rna2d3d/rna2d3d.html&quot; rel=&quot;nofollow&quot;&gt;http://www-lmmb.ncifcrf.gov/~bshapiro/rna2d3d/rna2d3d.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MC-FOLD -&gt; &lt;a href=&quot;http://www.major.iric.ca/MC-Pipeline/&quot; rel=&quot;nofollow&quot;&gt;http://www.major.iric.ca/MC-Pipeline/&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These programs require the 2D structure to generate the 3D structure, but you can easily find a program for that. Actually there's one on the MC-Pipeline website so you can get from the RNA sequence to the 3D structure in one place. RNA2D3D is more interactive than MC-Fold, but as far as I know there is no webserver and you have to use the Linux program. MC-Fold can be used online. It's the basis for Parisien &amp;amp; Major's excellent Nature paper (&lt;a href=&quot;http://www.nature.com/nature/journal/v452/n7183/abs/nature06684.html&quot; rel=&quot;nofollow&quot;&gt;The MC-Fold and MC-Sym pipeline infers RNA structure from sequence data, 2008&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="91" LastActivityDate="2011-12-04T15:19:16.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="234" PostTypeId="2" ParentId="217" CreationDate="2011-12-05T05:33:42.317" Score="5" Body="&lt;p&gt;As you suggest, the binary representation works because the &lt;strong&gt;binary numbers with $d$ digits&lt;/strong&gt; can be thought of as vectors representing the &lt;strong&gt;coordinates of the vertices of the $d$-dimensional hypercube&lt;/strong&gt;.  Now notice that the coordinates of an edge midpoint can be obtained by averaging the coordinates of two (adjacent) vertices and the coordinates of a face midpoint can be obtained by averaging the coordinates of two edge midpoints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This suggests using a ternary (base 3) system, with each digit equal to 0, 1, or 2.  Consider a hypercube whose volume is the tensor product of the intervals $[0,2]$.  Then the coordinates of the vertices are all the $d$-digit ternary numbers with each digit equal to 0 or 2.  The edge midpoint coordinates are all the $d$-digit ternary numbers with &lt;em&gt;one digit equal to 1&lt;/em&gt; and the remaining digits equal to 0 or 2.  The face midpoint coordinates are all the $d$-digit ternary numbers with &lt;em&gt;two digits equal to to 1&lt;/em&gt; and the remaining digits equal to 0 or 2.  This extends in the obvious way to higher-dimensional analogues of faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for instance, the &lt;strong&gt;vertices&lt;/strong&gt; of a cube ($d=3$) are&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(0,0,0)&#xA;(0,0,2)&#xA;(0,2,0)&#xA;(2,0,0)&#xA;(0,2,2)&#xA;(2,0,2)&#xA;(2,2,0)&#xA;(2,2,2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;edges&lt;/strong&gt; are&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(0,0,1)&#xA;(2,0,1)&#xA;(0,2,1)&#xA;(2,2,1)&#xA;(0,1,0)&#xA;(2,1,0)&#xA;(0,1,2)&#xA;(2,1,2)&#xA;(1,0,0)&#xA;(1,0,2)&#xA;(1,2,2)&#xA;(1,2,0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the &lt;strong&gt;faces&lt;/strong&gt; are&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(1,1,0)&#xA;(1,1,2)&#xA;(1,0,1)&#xA;(1,2,1)&#xA;(0,1,1)&#xA;(2,1,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thinking about these coordinates even gives a simple way even to determine the number of $n$-faces of a $d$-dimensional hypercube.  These correspond to all $d$-digit ternary numbers with $n$ digits equal to 1 and the remaining digits equal to 0 or 2, so there are $2^{d-n}\cdot {d\choose{n}}$ of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two things I haven't addressed are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Ordering; the Gray code's ingenuity is that adjacent vertices in the list are adjacent in space.  I think you could achieve this with the representation I've described using an approach similar to Gray's, but the question doesn't specifically mention ordering so I didn't worry about it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How to code this up.  This is more of a programming question, and shouldn't be too difficult.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-12-05T17:58:58.973" LastActivityDate="2011-12-05T17:58:58.973" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="235" PostTypeId="2" ParentId="206" CreationDate="2011-12-05T09:25:50.440" Score="6" Body="&lt;p&gt;I'd approach this question pragmatically rather than dogmatically. Ask yourself the question: &quot;What could go wrong in function X?&quot; Imagine what happens to the output when you introduce some typical bugs into the code: a wrong prefactor, a wrong index, ... And then write unit tests that are likely to detect that kind of bug. If for a given function there is no way to write such tests without repeating the code of the function itself, then don't - but think about tests on the next higher level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A much more important problem with unit tests (or in fact &lt;i&gt;any&lt;/i&gt; tests) in scientific code is how to deal with the uncertainties of floating-point arithmetic. As far as I know there are no good general solutions yet.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-12-05T09:25:50.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="236" PostTypeId="2" ParentId="187" CreationDate="2011-12-05T13:21:41.317" Score="29" Body="&lt;p&gt;While all current CPU's appear to use an iterative approach as &lt;a href=&quot;https://scicomp.stackexchange.com/a/188/149&quot;&gt;aterrel suggests&lt;/a&gt;, there has been some work done on non-iterative approaches. &lt;a href=&quot;https://pdfs.semanticscholar.org/68ad/eb9e6235eca080469efa0f1ea2972837b4e0.pdf&quot; rel=&quot;noreferrer&quot;&gt;Variable Precision Floating Point Division and Square Root&lt;/a&gt; talks about a non-iterative implementation of floating point division and square root in an &lt;a href=&quot;http://en.wikipedia.org/wiki/Field-programmable_gate_array&quot; rel=&quot;noreferrer&quot;&gt;FPGA&lt;/a&gt;, using lookup tables and taylor series expansion.&lt;/p&gt;&#xA;&lt;p&gt;I suspect that the same techniques may make it possible to get these operations down to a single cycle (throughput, if not latency), but you are likely to need &lt;em&gt;huge&lt;/em&gt; lookup tables, and thus infeasibly large areas of silicon real-estate to do it.&lt;/p&gt;&#xA;&lt;h3&gt;Why would it not be feasible?&lt;/h3&gt;&#xA;&lt;p&gt;In designing CPU's there are many trade-offs to make. Functionality, complexity (number of transistors), speed and power consumption are all interrelated and the decisions made during design can make a huge impact on performance.&lt;/p&gt;&#xA;&lt;p&gt;A modern processor probably &lt;em&gt;could&lt;/em&gt; have a main floating point unit which dedicates enough transistors on the silicon to perform a floating point division in a &lt;em&gt;single cycle&lt;/em&gt;, but it would be unlikely to be an efficient use of those transistors.&lt;/p&gt;&#xA;&lt;p&gt;The floating point multiply made this transition from iterative to non-iterative a decade ago. These days, single cycle &lt;a href=&quot;http://en.wikipedia.org/wiki/Binary_multiplier&quot; rel=&quot;noreferrer&quot;&gt;multiply&lt;/a&gt; and even multiply-accumulate are commonplace, even in mobile processors.&lt;/p&gt;&#xA;&lt;p&gt;Before it became an efficient use of transistor budget, multiply, like division, was often performed by an iterative method. Back then, dedicated DSP processors might dedicate most of their silicon to a single fast &lt;a href=&quot;http://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate&quot; rel=&quot;noreferrer&quot;&gt;multiply accumulate (MAC)&lt;/a&gt; unit. A Core2duo CPU has a floating point &lt;a href=&quot;http://web.archive.org/web/20120102211744/http://www.behardware.com/articles/623-5/intel-core-2-duo-test.html&quot; rel=&quot;noreferrer&quot;&gt;multiply latency of 3&lt;/a&gt; (the value comes out of the pipeline 3 cycle after it went in), but can have 3 multiplies in flight at once, resulting in a single-cycle throughput, meanwhile it's SSE2 unit can pump out multiple FP multiplies in a single cycle.&lt;/p&gt;&#xA;&lt;p&gt;Instead of dedicating huge areas of silicon to a single-cycle divide unit, modern CPU's have multiple units, each of which can perform operations in parallel, but are optimised for their own specific situations. In fact, once you take into account &lt;a href=&quot;http://en.wikipedia.org/wiki/SIMD&quot; rel=&quot;noreferrer&quot;&gt;SIMD&lt;/a&gt; instructions such as &lt;a href=&quot;http://en.wikipedia.org/wiki/Streaming_SIMD_Extensions&quot; rel=&quot;noreferrer&quot;&gt;SSE&lt;/a&gt; or the CPU &lt;a href=&quot;http://en.wikipedia.org/wiki/Intel_HD_Graphics&quot; rel=&quot;noreferrer&quot;&gt;integrated graphics&lt;/a&gt; of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Sandy_Bridge&quot; rel=&quot;noreferrer&quot;&gt;Sandy Bridge&lt;/a&gt; or later CPU's, there may be &lt;em&gt;many&lt;/em&gt; such floating-point divide units on your CPU.&lt;/p&gt;&#xA;&lt;p&gt;If generic floating point division were more important to modern CPU's then it might make sense to dedicate enough silicon area to make it single cycle, however most chip makers have obviously decided that they can make better use of that silicon by using those gates for other things. Thus one operation is slower, but overall (for typical usage scenarios) the CPU is faster and/or consumes less power.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="-1" LastEditDate="2020-06-25T18:05:25.420" LastActivityDate="2020-06-25T18:05:25.420" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="237" PostTypeId="2" ParentId="206" CreationDate="2011-12-05T14:48:49.107" Score="92" Body="&lt;p&gt;For many years I was under the misapprehension that I didn't have enough time to write unit tests for my code. When I did write tests, they were bloated, heavy things which only encouraged me to think that I should only ever write unit tests when I &lt;em&gt;knew&lt;/em&gt; they were needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I started to use &lt;a href=&quot;http://en.wikipedia.org/wiki/Test-driven_development&quot; rel=&quot;noreferrer&quot;&gt;Test Driven Development&lt;/a&gt; and I found it to be a complete revelation. I'm now firmly convinced that I don't have the time &lt;strong&gt;&lt;em&gt;not&lt;/strong&gt; to write unit-tests&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, by developing with testing in mind you end up with cleaner interfaces, more focussed classes &amp;amp; modules and generally more &lt;a href=&quot;http://en.wikipedia.org/wiki/SOLID_%28object-oriented_design%29&quot; rel=&quot;noreferrer&quot;&gt;SOLID&lt;/a&gt;, testable code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every time I work with legacy code which doesn't have unit tests and have to manually test something, I keep thinking &quot;this would be so much quicker if this code already had unit tests&quot;. Every time I have to try and add unit test functionality to code with high coupling, I keep thinking &quot;this would be so much easier if it had been written in a de-coupled way&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing and contrasting the two experimental stations that I support. One has been around for a while and has a great deal of legacy code, while the other is relatively new.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When adding functionality to the old lab, it is often a case of getting down to the lab and spending many hours working through the implications of the functionality they need and how I can add that functionality without affecting any of the other functionality. The code is simply not set up to allow off-line testing, so pretty much everything has to be developed on-line. If I did try to develop off-line then I would end up with more &lt;a href=&quot;http://en.wikipedia.org/wiki/Mock_object&quot; rel=&quot;noreferrer&quot;&gt;mock objects&lt;/a&gt; than would be reasonable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the newer lab, I can usually add functionality by developing it off-line at my desk, mocking out only those things which are immediately required, and then only spending a short time in the lab, ironing out any remaining problems not picked up off-line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For clarity, and since &lt;a href=&quot;https://scicomp.stackexchange.com/users/1232/naught101&quot;&gt;@naught101&lt;/a&gt; asked...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tend to work on experimental control and data acquisition software, with some ad hoc data analysis, so the combination of TDD with revision control helps to document both changes in the underlying experiment hardware and as well as changes in data collection requirements over time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even in the situation of developing exploratory code however, I could see a significant benefit from having assumptions codified, along with the ability to see how those assumptions evolve over time.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2018-07-09T10:29:40.060" LastActivityDate="2018-07-09T10:29:40.060" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="238" PostTypeId="2" ParentId="180" CreationDate="2011-12-05T19:55:48.787" Score="2" Body="&lt;p&gt;I'm honestly not overly familiar with these schemes, but I believe the ADF package has a bonding energy decomposition scheme by Ziegler and Rauk implemented in it, and there's also the Morokuma scheme which looks to be available in at least GAMESS and ADF. I think these both produce the terms you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2011-12-05T19:55:48.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="240" PostTypeId="1" AcceptedAnswerId="249" CreationDate="2011-12-05T21:39:02.737" Score="7" ViewCount="253" Body="&lt;p&gt;I am researching heat diffusion in an optical element irradiated by laser. This problem is described by the PDE which I wrote down in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/124/can-heat-distribution-in-an-optical-element-irradiated-by-laser-be-oscillating&quot;&gt;this question&lt;/a&gt;. I am using an implicit numerical scheme to model the heat distribution evolving in time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some time has passed the system comes to a stationary distribution (corresponding with thermal equilibrium). One of the most interesting features of the model is the nonuniqueness of stationary state: if we'll start from another initial distribution, then we may come to another stationary state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, one of the task I need to solve is to find the set of stationary states for a given set of parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving it with following algorithm: I run the simulation until the heat distribution stops changing (difference between current and past distribution is lesser than given epsilon). So, after one stationary distribution is found, I increase and decrease laser power to find the next stationary state. By varying initial distributions for given set of parameters I compute a set of stationary distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, there are two problems. First, to compute the needed set with given accuracy I need to use small steps in space and time, small value of epsilon used to find stationary states and many steps of laser power variating. This is very computationally intensive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second problem is that it's easy to miss a distribution or count one as two, and I am not sure if there are other distributions which my algorithm cannot found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what algorithms should I consider to use instead of just using implicit scheme and waiting until equilibrium is estabilished? There are two significant cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;there is one possible stationary state, and the task is just to find it with given accuracy and minimal amount of computation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;there is a set of possible stationary distributions and the task is to find them all.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="90" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2011-12-07T17:39:12.487" Title="Optimal way to find stationary solutions of the PDE" Tags="|algorithms|diffusion|pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="241" PostTypeId="2" ParentId="160" CreationDate="2011-12-05T22:12:09.400" Score="0" Body="&lt;p&gt;I think the answer very much depends on the application you're intending for the RNG to be used for. I'd suggest a forth category for Tangurena's rough classification: &quot;Good with no real gain&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For many applications, it may simply not matter, and a properly cryptographic-grade RNG may simply slow down your tasks without any commensurate gain in validity. For example, much of the research I do just requires many, many millions of numbers coming roughly from a distribution I specify. Almost any RNG will do, so all I need is one that's not so catastrophically poor as to be worthless &lt;em&gt;as&lt;/em&gt; an RNG. Anything else is simply slowing down the work unnecessarily. I tend to use Mersenne Twister, but that's simply because it works well enough, I have the code, and it's reasonably fast.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2011-12-05T22:12:09.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="242" PostTypeId="1" AcceptedAnswerId="245" CreationDate="2011-12-05T22:48:54.717" Score="14" ViewCount="265" Body="&lt;p&gt;I work fairly heavily in mathematical biology/epidemiology, where most of the modeling/computational science work is still dominated by sets of ODEs, admittedly sometimes fairly elaborate sets of them. One of the plusses of these models is that they're rather easy to describe and replicate. A table of parameter values, and the equations themselves and you've given someone everything they need to replicate your research in whatever way they feel like implementing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But somewhat more complex models have started to become more popular. Agent-based models, in particular, seem to be both harder to describe in a publication, and harder to replicate, because they &lt;em&gt;aren't&lt;/em&gt; necessarily perfectly described by a set of ODEs. Are there any guidelines - or just practical experience - behind describing these models in a way that readers understand what happened, and make them relatively straightforward to replicate?&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="41" LastEditDate="2012-01-09T01:57:20.870" LastActivityDate="2013-10-30T09:45:09.043" Title="Best practices for describing agent-based models" Tags="|ode|computational-biology|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="243" PostTypeId="5" CreationDate="2011-12-05T23:14:37.170" Score="0" Body="&lt;p&gt;Algorithms are descriptions of the computations steps needed to solve a problem. They have well defined inputs and specific outputs that meet the given characteristics.  Algorithms can be expressed in flowcharts or pseudo-code.  To be useful to a computer, these must be translated into a specific programming language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions with the algorithm tag should be programming language independent, though they may use a specific language as an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information on algorithms can be found on the wikipedia page: &lt;a href=&quot;http://en.wikipedia.org/wiki/Algorithm&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="63" LastEditorUserId="63" LastEditDate="2011-12-06T22:13:44.700" LastActivityDate="2011-12-06T22:13:44.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="244" PostTypeId="4" CreationDate="2011-12-05T23:14:37.170" Score="0" Body="A description of the specific steps needed to solve a particular problem in an unambiguous way, expressed in an abstract form." OwnerUserId="63" LastEditorUserId="63" LastEditDate="2011-12-06T22:13:51.657" LastActivityDate="2011-12-06T22:13:51.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="245" PostTypeId="2" ParentId="242" CreationDate="2011-12-05T23:21:18.117" Score="4" Body="&lt;p&gt;I don't work in that business but naively I think there are three parts to a complete description&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A description of the data landscape they live in. Describe this in terms of the data structure (graph (directed or undirected, weighted or unweighted); tree; array; ...) and the data associated with each node. Make note of special case handling such as periodic boundary conditions or assumed state for neighbors outside the test region. Presumably this has a fairly clear connection with your problem domain.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A description of the internal state of the agent and how it makes decisions. Again, hopefully this has a reasonably clear interpretation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A description of the relative timing and/or synchronization of action and updates between the agents and the landscape; and between pair or groups of agents.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Pseudo-code (or even real code if it's not too polluted with implementation details) will help.&lt;/p&gt;&#xA;" OwnerUserId="183" LastActivityDate="2011-12-05T23:21:18.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="246" PostTypeId="5" CreationDate="2011-12-05T23:36:33.600" Score="0" Body="&lt;p&gt;Questions with the &lt;a href=&quot;/questions/tagged/libraries&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged 'libraries'&quot; rel=&quot;tag&quot;&gt;libraries&lt;/a&gt; tag should not be just about a specific library; those questions should be tagged with a tag specific to that library (see examples below).  Rather, these questions should be more general: about the creation, testing, and usage of libraries in the abstract.  Also appropriate are inquiries about what libraries have a given functionality or comparisons between multiple libraries (though in the latter case, it should also be tagged with the relevant libraries, if possible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of libraries:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;BLAS &lt;a href=&quot;http://netlib.org/blas/&quot; rel=&quot;nofollow&quot;&gt;http://netlib.org/blas/&lt;/a&gt; Routines that provide standard building blocks for performing basic vector and matrix operations.  One implementation of this is ATLAS &lt;a href=&quot;/questions/tagged/atlas&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged 'atlas'&quot; rel=&quot;tag&quot;&gt;atlas&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;LAPACK &lt;a href=&quot;http://www.netlib.org/lapack/&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lapack/&lt;/a&gt; A Fortran 90 library that provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems.&lt;/li&gt;&#xA;&lt;li&gt;PETSc &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/&lt;/a&gt; &lt;a href=&quot;/questions/tagged/petsc&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged 'petsc'&quot; rel=&quot;tag&quot;&gt;petsc&lt;/a&gt; A suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="63" LastEditorUserId="63" LastEditDate="2011-12-07T17:39:58.693" LastActivityDate="2011-12-07T17:39:58.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="247" PostTypeId="4" CreationDate="2011-12-05T23:36:33.600" Score="0" Body="A collection of related programming functions implementing a set of abstract data types, mathematical functions, or other higher level analysis algorithms so that they may be used, and reused, by other programs." OwnerUserId="63" LastEditorUserId="63" LastEditDate="2011-12-07T17:40:03.623" LastActivityDate="2011-12-07T17:40:03.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="249" PostTypeId="2" ParentId="240" CreationDate="2011-12-06T06:45:57.420" Score="8" Body="&lt;p&gt;If there is one steady state, then a standard method is to use pseudotransient continuation, which is essentially linearly implicit Euler with residual-based step size adaptivity. This approach has relatively strong convergence guarantees for a quadratically convergent method. &lt;a href=&quot;http://dx.doi.org/10.1137/S106482750241044X&quot;&gt;Coffey, Kelley, and Keyes (2003)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Methods to provably compute &lt;em&gt;all&lt;/em&gt; solutions are rarely available, though &lt;a href=&quot;http://archimedes.cheme.cmu.edu/baron/baron.html&quot;&gt;Baron&lt;/a&gt; can sometimes be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the steady state has manifold structure parametrized by a low-dimensional space, then you can explore it using continuation techniques, see the book by Allgower and Georg or by Seydel. Some software is available for this task, e.g. &lt;a href=&quot;http://multifario.sourceforge.net/&quot;&gt;Multifario&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-12-06T06:45:57.420" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="250" PostTypeId="1" AcceptedAnswerId="251" CreationDate="2011-12-06T08:53:38.333" Score="25" ViewCount="1583" Body="&lt;p&gt;I have a series of data points $(x_i,y_i)$ which I expect to (approximately) follow a function $y(x)$ that asymptotes to a line at large $x$. Essentially, $f(x) \equiv y(x) - (ax + b)$ approaches zero as $x \to \infty$, and the same can probably be said of all the derivatives $f'(x)$, $f''(x)$, etc. But I don't know what the functional form for $f(x)$ is, if it even has one that can be described in terms of elementary functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to get the best possible estimate of the asymptotic slope $a$. The obvious crude method is to pick out the last few data points and do a linear regression, but of course this will be inaccurate if $f(x)$ does not become &quot;flat enough&quot; within the range of $x$ for which I have data. The obvious less-crude method is to assume that $f(x) \approx \exp(-x)$ (or some other particular functional form) and fit to that using all the data, but the simple functions I've tried like $\exp(-x)$ or $\dfrac1{x}$ don't quite match the data at lower $x$ where $f(x)$ is large. Is there a known algorithm for determining the asymptotic slope that would do better, or that could provide a value for the slope along with a confidence interval, given my lack of knowledge of exactly how the data approach the asymptote?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This sort of task tends to come up frequently in my work with various data sets, so I'm mostly interested in general solutions, but by request I'm linking to the particular &lt;a href=&quot;http://static.ellipsix.net/ext-tmp/qs020peak.csv&quot; rel=&quot;noreferrer&quot;&gt;data set&lt;/a&gt; that prompted this question. As described in comments, the Wynn $\epsilon$ algorithm gives a value that, as far as I can tell, is somewhat off. Here is a plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uo9iP.png&quot; alt=&quot;Asymptotically linear data&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(It does look like there's a slight downward curve at high x values, but the theoretical model for this data predicts that it should be asymptotically linear.)&lt;/p&gt;&#xA;" OwnerUserId="103" LastEditorUserId="103" LastEditDate="2011-12-08T06:06:55.310" LastActivityDate="2011-12-08T06:06:55.310" Title="Is there a numerical algorithm for finding an asymptotic slope?" Tags="|algorithms|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="251" PostTypeId="2" ParentId="250" CreationDate="2011-12-06T09:24:30.310" Score="14" Body="&lt;p&gt;It's a rather rough algorithm, but I'd use the following procedure for a crude estimate: if, as you say, the purported $f(x)$ that represents your $(x_i,y_i)$ is already almost linear as $x$ increases, what I'd do is to take differences $\dfrac{y_{i+1}-y_i}{x_{i+1}-x_i}$, and then use an extrapolation algorithm like the &lt;a href=&quot;http://dx.doi.org/10.1090/S0025-5718-1956-0084056-6&quot;&gt;Shanks transformation&lt;/a&gt; to estimate the limit of the differences. The result is hopefully a good estimate of this asymptotic slope.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What follows is a &lt;em&gt;Mathematica&lt;/em&gt; demonstration. The Wynn $\epsilon$ algorithm is a convenient implementation of the Shanks transformation, and it is built in as the (hidden) function &lt;code&gt;SequenceLimit[]&lt;/code&gt;. We try out the procedure on the function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac4{x^2+3}+2 x+e^{-4 x}+3$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xdata = RandomReal[{20, 40}, 25];&#xA;ydata = Table[(3 + 13*E^(4*x) + 6*E^(4*x)*x + x^2 + 3*E^(4*x)*x^2 + &#xA;      2*E^(4*x)*x^3)/(E^(4*x)*(3 + x^2)), {x, xdata}];&#xA;&#xA;SequenceLimit[Differences[ydata]/Differences[xdata],&#xA;              Method -&amp;gt; {&quot;WynnEpsilon&quot;, Degree -&amp;gt; 2}]&#xA;1.999998&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I might as well show off how simple the algorithm is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wynnEpsilon[seq_?VectorQ] := &#xA; Module[{n = Length[seq], ep, res, v, w}, res = {};&#xA;  Do[ep[k] = seq[[k]];&#xA;   w = 0;&#xA;   Do[v = w; w = ep[j];&#xA;    ep[j] = &#xA;     v + (If[Abs[ep[j + 1] - w] &amp;gt; 10^-(Precision[w]), ep[j + 1] - w, &#xA;         10^-(Precision[w])])^-1;, {j, k - 1, 1, -1}];&#xA;   res = {res, ep[If[OddQ[k], 1, 2]]};, {k, n}];&#xA;  Flatten[res]]&#xA;&#xA;Last[wynnEpsilon[Differences[ydata]/Differences[xdata]]]&#xA;1.99966&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This implementation is adapted from &lt;a href=&quot;http://arxiv.org/abs/math/0306302&quot;&gt;Weniger's paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-12-06T17:53:58.913" LastActivityDate="2011-12-06T17:53:58.913" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="252" PostTypeId="1" CreationDate="2011-12-06T10:29:32.367" Score="7" ViewCount="6819" Body="&lt;p&gt;As from Title. Where can I find a database of simple chemical structures in XYZ format, possibly with a simple CAS -&gt; XYZ or InChi -&gt; XYZ REST service ? &lt;/p&gt;&#xA;" OwnerUserId="23" LastEditorUserId="127" LastEditDate="2011-12-06T11:14:25.327" LastActivityDate="2020-05-04T07:51:08.050" Title="Where can I find a database of simple chemical structures in XYZ format?" Tags="|computational-chemistry|data-sets|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="253" PostTypeId="5" CreationDate="2011-12-06T14:27:15.457" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-06T14:27:15.457" LastActivityDate="2011-12-06T14:27:15.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="254" PostTypeId="4" CreationDate="2011-12-06T14:27:15.457" Score="0" Body="Questions related to data used in testing computational methods." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-12-06T22:13:56.163" LastActivityDate="2011-12-06T22:13:56.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="255" PostTypeId="2" ParentId="252" CreationDate="2011-12-06T17:17:41.553" Score="8" Body="&lt;p&gt;ChemSpider (http://www.chemspider.com/) provides structures in mol format, which is simple enough to convert to XYZ, e.g. using OpenBabel (http://openbabel.org/wiki/Main_Page).&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-12-06T17:17:41.553" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="256" PostTypeId="2" ParentId="206" CreationDate="2011-12-06T21:59:55.773" Score="7" Body="&lt;p&gt;Absolutely!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What, that's not enough for you?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In scientific programming more than any other kind, we're developing based on trying to match a physical system. How will you know if you've done that other than by testing? Before you even start coding, decide how you're going to use your code and work out a few example runs. Try to catch any possible edge cases. Do it in a modular way - for example, for a neural network you might make a set of tests for a single neuron and a set of tests for a complete neural network. That way when you start writing code you can make sure your neuron works before you start working on the network. Working in stages like this means that when you run into a problem you only have the most recent 'stage' of code to test, the earlier stages have already been tested. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plus, once you have the tests, if you need to rewrite the code in a different language (converting to CUDA, for example), or even if you're just updating it, you already have the testcases and you can use them to make sure that both versions of your program work the same way.&lt;/p&gt;&#xA;" OwnerUserId="199" LastActivityDate="2011-12-06T21:59:55.773" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="257" PostTypeId="2" ParentId="223" CreationDate="2011-12-06T22:41:57.307" Score="20" Body="&lt;p&gt;MKL (from Intel) is optimized for Intel processors, and probably has the &quot;upper hand&quot; there in many cases. But it is also &quot;famous&quot; for choosing the &quot;worst&quot; code-paths for AMD processors, as described &lt;a href=&quot;http://en.wikipedia.org/wiki/Intel_C++_Compiler#Criticism&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="206" LastEditorUserId="276" LastEditDate="2012-01-02T22:59:53.323" LastActivityDate="2012-01-02T22:59:53.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="258" PostTypeId="2" ParentId="26" CreationDate="2011-12-06T23:32:51.433" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://fieldtrip.fcdonders.nl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fieldtrip&lt;/a&gt; implements ICA from EEGLab and works well with Neuromag (so far).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;FieldTrip is the Matlab software toolbox for MEG and EEG analysis that is being developed at the Centre for Cognitive Neuroimaging of the Donders Institute for Brain, Cognition and Behaviour together with collaborating institutes. The development of FieldTrip is currently supported by funding from the BrainGain and the Human Connectome projects. The FieldTrip software is released as open source under the GNU general public license.&lt;/p&gt;&#xA;&lt;p&gt;The software includes algorithms for simple and advanced analysis of MEG, EEG, and invasive electrophysiological data, such as time-frequency analysis, source reconstruction using dipoles, distributed sources and beamformers and non-parametric statistical testing. It supports the data formats of all major MEG systems (CTF, Neuromag, BTi) and of the most popular EEG systems, and new formats can be added easily. FieldTrip contains high-level functions that you can use to construct your own analysis protocols in Matlab. Furthermore, it easily allows developers to incorporate low-level algorithms for new EEG/MEG analysis methods.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="101" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-08-11T20:30:40.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="259" PostTypeId="2" ParentId="206" CreationDate="2011-12-07T03:07:39.013" Score="3" Body="&lt;p&gt;A slightly different approach that I used whilst developing a chemical solver (for complex geological domains) was what you could call &lt;em&gt;Unit Testing by Copy and Paste Snippet&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Building a test harness for the original code embedded in a large chemical system modeller was not feasible in the timeframe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I was able to work up an increasingly complex set of snippets showing how the (Boost Spirit) parser for the chemical formulas worked, as unit tests for different expressions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final, most complex unit test was very close to the code needed in the system, without having to change that code to be mockable. I was thus able to copy across my unit tested code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What makes this more than just a learning exercise and a true regression suite are two factors&#xA;- the unit tests kept in the main source and run as part of other tests for that application (and yes, they did pick up a side-effect from Boost Spirit changing 2 years later)&#xA;- because the code copied and pasted across was minimally modified in the real application, it could have comments referring back to the unit tests to help someone keep them in synch.&lt;/p&gt;&#xA;" OwnerUserId="213" LastActivityDate="2011-12-07T03:07:39.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="260" PostTypeId="1" CreationDate="2011-12-07T04:03:19.933" Score="3" ViewCount="197" Body="&lt;p&gt;As far as global optimal sequence alignment goes, is the Needleman-Wunsch and Hirschberg's algorithm still state of the art? Or have there been any improvements to these algorithms since they were published, or any newer algorithms?&lt;/p&gt;&#xA;" OwnerUserId="117" LastEditorUserId="127" LastEditDate="2011-12-07T04:48:18.473" LastActivityDate="2011-12-07T17:39:47.710" Title="Global optimal sequence alignment algorithms" Tags="|algorithms|computational-biology|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="261" PostTypeId="2" ParentId="260" CreationDate="2011-12-07T04:56:49.073" Score="6" Body="&lt;p&gt;According to Wing-Kin Sung's excellent &quot;Algorithms in Bioinformatics&quot; (2010, &lt;a href=&quot;http://books.google.com/books?id=4j_GfM0tyPUC&amp;amp;pg=PA30&quot; rel=&quot;nofollow&quot;&gt;pp 30-39&lt;/a&gt;), the fastest algorithm was discovered in 1980 by Masek and Paterson and can solve the global alignment problem in $O(nm /\log(n))$ time, which is barely better than the Needleman-Wunsch algorithm:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;W.J Masek and M.S. Paterson. 1980. &lt;a href=&quot;http://dx.doi.org/10.1016/0022-0000%2880%2990002-1&quot; rel=&quot;nofollow&quot;&gt;&quot;A faster algorithm computing string edit distances.&quot;&lt;/a&gt; &lt;em&gt;Journal of Computer and System Sciences&lt;/em&gt; 20(1):18-31.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sung's book has a short but well-written section on global alignment.&lt;/p&gt;&#xA;" OwnerUserId="91" LastEditorUserId="127" LastEditDate="2011-12-07T17:39:47.710" LastActivityDate="2011-12-07T17:39:47.710" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="262" PostTypeId="5" CreationDate="2011-12-07T05:28:41.497" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-07T05:28:41.497" LastActivityDate="2011-12-07T05:28:41.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="263" PostTypeId="4" CreationDate="2011-12-07T05:28:41.497" Score="0" Body="This tag is intended for questions that don't admit a definitive answer." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2011-12-14T14:42:22.700" LastActivityDate="2011-12-14T14:42:22.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="264" PostTypeId="2" ParentId="252" CreationDate="2011-12-07T05:54:37.630" Score="4" Body="&lt;p&gt;This site has some nice enough data: &#xA;&lt;a href=&quot;http://cccbdb.nist.gov/&quot; rel=&quot;nofollow&quot;&gt;http://cccbdb.nist.gov/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="80" LastActivityDate="2011-12-07T05:54:37.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="265" PostTypeId="1" AcceptedAnswerId="467" CreationDate="2011-12-07T06:42:05.743" Score="24" ViewCount="21159" Body="&lt;p&gt;I will start with my personal experience in our lab. Back in the ifort 9 and 10 days, we used to be quite aggressive with the optimizations, compiling with -O3 and processor specific flags (-xW -xSSE4.2 for example). But starting with ifort 11, we started to notice:&lt;br&gt;&#xA; 1. some inconsistencies in the results (because semantics were not preserved)&lt;br&gt;&#xA; 2. smaller gains compared to -O2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So currently, we usually simply compile with -O2 and -xhost. Do you have better suggestions with ifort 11? Will this change once again as we transition to ifort 12? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="95" LastEditorUserId="127" LastEditDate="2011-12-07T12:57:21.650" LastActivityDate="2020-03-06T19:42:35.017" Title="Intel Fortran Compiler: tips on optimization at compilation" Tags="|performance|hpc|compiling|fortran|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="266" PostTypeId="1" CreationDate="2011-12-07T07:28:20.717" Score="4" ViewCount="791" Body="&lt;p&gt;I am running into a problem with COMSOL 4.2.0.187 on Ubuntu (both 10.04 LTS and 11.04). When using the option &lt;code&gt;-np x&lt;/code&gt; with &lt;code&gt;x&lt;/code&gt; &gt; 1, COMSOL crashes systematically after a short, yet random amount of time with the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; [thread 140538491361024 also had an error] terminate called without an active exception &#xA;&amp;gt; Aborted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When setting only one working process (&lt;code&gt;-np 1&lt;/code&gt;), the error disappear. The COMSOL support staff  told me that there must be a problem with the threading libraries. I suspect that they just have no idea what to tell me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you guys have any idea as how to solve this problem? Have you ever encountered such an error with COMSOL or another piece of software?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot,&lt;/p&gt;&#xA;" OwnerUserId="76" LastEditorUserId="127" LastEditDate="2011-12-12T17:00:23.627" LastActivityDate="2011-12-12T17:00:23.627" Title="COMSOL on Ubuntu (10.04 LTS and 11.04)" Tags="|comsol|parallel-computing|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="267" PostTypeId="1" AcceptedAnswerId="273" CreationDate="2011-12-07T08:03:11.133" Score="4" ViewCount="369" Body="&lt;p&gt;Say I have a simulation that produces a single floating point number as a result, and a different number is produced each time the simulation runs. These numbers are randomly distributed according to some unknown distribution. I would like to approximate the underlying density distribution function based on a finite sample of numbers. &lt;p&gt;&#xA;I know that I could use a histogram, but the size of the bins, which is essentially arbitrary, will significantly affect the results. I would like a way to do this which has no &quot;arbitrariness&quot;.&lt;p&gt;&#xA;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="219" LastEditorUserId="127" LastEditDate="2011-12-07T14:06:10.603" LastActivityDate="2012-01-02T22:56:22.550" Title="Approximate a distribution function from a finite sample" Tags="|statistics|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="268" PostTypeId="2" ParentId="266" CreationDate="2011-12-07T08:11:25.483" Score="3" Body="&lt;p&gt;You don't seem to be running the latest version of COMSOL. You might want to install the latest updates and try again. See &lt;a href=&quot;http://www.comsol.com/support/updates/comsol42p3/&quot; rel=&quot;nofollow&quot;&gt;http://www.comsol.com/support/updates/comsol42p3/&lt;/a&gt;  I'm able to run comsol -np x (x &gt; 1) on a Scientific Linux 6.0 without problems.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2011-12-07T09:43:25.597" LastActivityDate="2011-12-07T09:43:25.597" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="270" PostTypeId="1" AcceptedAnswerId="276" CreationDate="2011-12-07T08:52:39.067" Score="4" ViewCount="965" Body="&lt;p&gt;First post. I'm working on this problem using Bayesian methods. In desperation I'm considering using p-values (shock horror), specifically posterior predictive p-values. So I need to simulate from the posterior predictive. I don't see a lot of statistics stuff here - it's mostly applied math at the moment. I could take this to stats.sx I suppose, but this is definitely a computational problem, so I thought I'd give it a shot here. If there are any Bayesians lurking here, now would be a good time to delurk. :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In what follows, the hats denote predictive values. Consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $p(\mathbf{p})$ be a Dirichlet prior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;p(\mathbf{p}) = (k-1)! I(\sum p_i = 1)&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the likelihood be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;p(y| \mathbf{p}) =  \prod_{i=1}^k p_i^{f_i}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the likelihood for a sequence of categorical variables. Assume the prior predictive is (calculation omitted)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;  p(y) = (k-1)! \frac{\prod_i f_i!}{(N + k - 1)!}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;  p(\mathbf{p}|y) = p(\mathbf{p}) p(y| \mathbf{p}) / p(y)&#xA;                  = \frac{ (k-1)! I(\sum p_i = 1) \prod_{i=1}^k p_i^{f_i} }&#xA;                          { (k-1)! \frac{\prod_i f_i!}{(N + k - 1)!} }&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dividing we obtain the posterior&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;  p(\mathbf{p}|y) = \frac{ I(\sum p_i = 1) \prod_{i=1}^k p_i^{f_i} (N + k - 1)! }{ \prod_i f_i! }&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define $f^{\prime}_i = f_i + \hat{f}_i$, and $N^{\prime} =&#xD;&#xA;N+\hat{N}$. Then the posterior predictive is (calculation omitted but can be added if anyone cares)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;  p(\hat{y}|y) = \int p(\hat{y}|\mathbf{p}) p(\mathbf{p}|y) d\mathbf{p}&#xA;                 = \frac{ \prod_i f^{\prime}_i! }{ \prod_i f_i! } \frac{ (N + k - 1)! }{(N^{\prime}+k-1)!}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can anyone suggest methods, preferably as efficient and simple as possible, to simulate from $p(\tilde{y}|y)$? This is a finite distribution, but too big to enumerate. At the moment, I'm not even sure what distribution it is. Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. Can some kind person create &quot;bayesian-computation&quot; and &quot;statistics&quot; tags, and tag this message with them? &quot;random&quot; is not very useful. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="20688" LastEditDate="2019-06-22T22:43:27.043" LastActivityDate="2019-06-22T22:43:27.043" Title="Sampling from posterior predictive distribution" Tags="|statistics|random-sampling|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="271" PostTypeId="2" ParentId="267" CreationDate="2011-12-07T08:54:30.470" Score="4" Body="&lt;p&gt;What you are looking for is &lt;a href=&quot;http://en.wikipedia.org/wiki/Kernel_density_estimation&quot; rel=&quot;nofollow&quot;&gt;Kernel Density Estimation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms to carry this out are built into many scientific software packages and libraries. There is still a degree of arbitrariness in the choice of kernel and bandwidth, but there are heuristics for these.&lt;/p&gt;&#xA;" OwnerUserId="220" LastEditorUserId="276" LastEditDate="2012-01-02T22:56:22.550" LastActivityDate="2012-01-02T22:56:22.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="272" PostTypeId="2" ParentId="266" CreationDate="2011-12-07T11:17:30.530" Score="5" Body="&lt;p&gt;That thread ID looks bogus to me, which leads me to believe someone is writing over memory. I think the most helpful thing you could do for the COMSOL guys is to run this under &lt;a href=&quot;http://valgrind.org/&quot;&gt;valgrind&lt;/a&gt; and send them the output.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-07T11:17:30.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="273" PostTypeId="2" ParentId="267" CreationDate="2011-12-07T13:07:51.997" Score="6" Body="&lt;p&gt;Since a distribution function contains more information than the finite set of numbers you start with, you clearly have to add information in the process. This information comes in the form of a model that you assume, and whose parameters you adjust to make the model fit your finite sample. Unless the nature of your problem suggests some model, the choice will always be arbitrary to some extent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a histgram, your model is that of a piecewise constant function with the pieces (bins) having a fixed width. With Kernel Density Estimation, your model is the kernel. There is no way around choosing a model, the best you can do is to make an informed choice based on what you know (or can reasonably assume) about your data.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-12-07T13:07:51.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="274" PostTypeId="2" ParentId="267" CreationDate="2011-12-07T14:07:56.767" Score="0" Body="&lt;p&gt;I think you also need to decide what you are going to do with this distribution when you've got it. If you think you can sample from it instead of running your simulation to get random variates from your process then you then have to ask yourself some more questions about how important accuracy is, and how variations in your approximation to the true density will affect the outcome of sampling from an approximated density versus getting samples via the process...&lt;/p&gt;&#xA;" OwnerUserId="97" LastActivityDate="2011-12-07T14:07:56.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="275" PostTypeId="2" ParentId="267" CreationDate="2011-12-07T14:10:54.713" Score="4" Body="&lt;p&gt;Kernel density estimation is a good suggestion. Another option is to construct the empirical CDF and then seek the distribution that best fits. Depending on the form of your output this might be more appropriate than a KDE, which is typically used for multimodal distributions. More detail on the empirical CDF here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Empirical_distribution_function&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Empirical_distribution_function&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you form the empirical CDF you can use a goodness of fit metric (&lt;a href=&quot;http://en.wikipedia.org/wiki/Goodness_of_fit&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Goodness_of_fit&lt;/a&gt;) or maximum likelihood estimation (&lt;a href=&quot;http://en.wikipedia.org/wiki/Maximum_likelihood&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Maximum_likelihood&lt;/a&gt;) to see which distribution type and parameter values best fit your data. The Pearson distribution is sometimes used in these situations if you have no good reason to assume the data follows one of the usual suspects due to its flexibility (&lt;a href=&quot;http://en.wikipedia.org/wiki/Pearson_distribution&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Pearson_distribution&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, as you noted, you only have finite data. It can be important to understand the uncertainty in the best fit given the limited data, which can be done via Bayesian inference (&lt;a href=&quot;http://en.wikipedia.org/wiki/Bayesian_inference&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Bayesian_inference&lt;/a&gt;). Properly implemented, this can account for both your model selection and the best-fit parameters given that model.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="63" LastEditDate="2011-12-07T20:36:05.340" LastActivityDate="2011-12-07T20:36:05.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="276" PostTypeId="2" ParentId="270" CreationDate="2011-12-07T14:17:12.817" Score="2" Body="&lt;p&gt;I believe the best (only?) way to accomplish this will be through Markov Chain Monte Carlo sampling:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will allow you to sample from the unknown posterior without knowing it's functional form.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;These are really more like questions than a better answer, but this seemed the best place to put them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you can reformulate the problem in terms of only $k-1$ unknowns since the last one is functionally dependent (the $p$'s must sum to one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming $f_i$ represents the observed frequency of each case (please correct that assumption if it's wrong), why include it as part of the prior? I'd suggest that your prior should not depend on the data. You might use an exponent of 0 to represent a vague prior instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless $k$ is huge, this problem is easily solved by MCMC. You could set it up in Winbugs or PyMC if you don't want to write your own code. Also, with MCMC, the above derivations aren't needed. You only need to specify the prior (which may just be constant) and the likelihood (which is pretty simple).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd also suggest not trying to simulate the posterior predictive directly, but instead simulate the posterior first and then the predictive is easily obtained from that by simulation.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2011-12-07T16:31:10.880" LastActivityDate="2011-12-07T16:31:10.880" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="277" PostTypeId="1" AcceptedAnswerId="279" CreationDate="2011-12-07T14:32:22.933" Score="16" ViewCount="711" Body="&lt;p&gt;I'm in a field that doesn't necessarily do a great deal of HPC work, and when it does encounter it, it's often the result of researchers from other fields exploring new applications to their methods and the like. Primarily what this means is that it never really gets introduced in the course of study, or get brought up much at workshops, seminars and the like - it's possible to go one's entire career without needing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the same time however, much of the work I do &lt;em&gt;could&lt;/em&gt; benefit from making better use of the HPC resources available to me - mostly in the form of nicely parallel Monte Carlo simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is finding resources for &lt;em&gt;learning&lt;/em&gt; how to go about using clusters, MPI, etc. And separating the good from the bad, given I don't know much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions for books either on programming on these types of systems, or on setting up and running very modest HPC setups of one's own?&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="123" LastEditDate="2012-01-09T05:37:18.737" LastActivityDate="2012-03-09T06:42:22.127" Title="How should I study creating and programming HPC systems?" Tags="|hpc|education|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="278" PostTypeId="2" ParentId="229" CreationDate="2011-12-07T14:38:36.110" Score="5" Body="&lt;p&gt;I colleague has found the &lt;a href=&quot;http://gandalf-library.sourceforge.net/tutorial/report/node134.html&quot; rel=&quot;noreferrer&quot;&gt;Fast Hough Transform&lt;/a&gt; in the &lt;a href=&quot;http://gandalf-library.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;Gandalf library&lt;/a&gt;, which looks very promising but may be a lot of work to integrate, so I am looking for other approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Gandalf implementation is interesting: they evaluation the accumulator space in a recursive way as if traversing a quad- or oct-tree. Regions without much density are thrown out as they go.&lt;/p&gt;&#xA;" OwnerUserId="183" LastActivityDate="2011-12-07T14:38:36.110" CommentCount="0" CommunityOwnedDate="2011-12-07T14:38:36.110" ContentLicense="CC BY-SA 3.0" />
  <row Id="279" PostTypeId="2" ParentId="277" CreationDate="2011-12-07T16:19:06.283" Score="13" Body="&lt;p&gt;Check out &lt;a href=&quot;http://hpcuniversity.org/&quot;&gt;HPC University&lt;/a&gt;.  In particular, the &lt;a href=&quot;http://hpcuniversity.org/resources/list/&quot;&gt;resources section&lt;/a&gt;, which includes things like &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;an &lt;a href=&quot;http://www.citutor.org/bounce.php?course=24&quot;&gt;Introduction to OpenMP&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.citutor.org/bounce.php?course=23&quot;&gt;Debugging Serial and Parallel Codes&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://osg-docdb.opensciencegrid.org/0000/000036/001/OSG-OperationsWorkshop.pdf&quot;&gt;Introduction to the Open Science Grid&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and much more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many &lt;a href=&quot;http://hpcuniversity.org/students/universityPrograms/&quot;&gt;higher education programs that include courses in HPC&lt;/a&gt;.  As an example, &lt;a href=&quot;http://www.kaust.edu.sa/academics/programs/downloads/AMCS.pdf&quot;&gt;my own program&lt;/a&gt; includes courses in&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;High performance computing (2 semesters)&lt;/li&gt;&#xA;&lt;li&gt;Parallel programming paradigms&lt;/li&gt;&#xA;&lt;li&gt;Scientific software engineering&lt;/li&gt;&#xA;&lt;li&gt;GPU and GPGPU Programming&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;as well as a variety of courses in scientific computing and numerical analysis.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2011-12-07T16:19:06.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="280" PostTypeId="1" CreationDate="2011-12-07T16:54:30.437" Score="1" ViewCount="64" Body="&lt;p&gt;I am annotating protein-coding genes in a region of the human genome and am looking for a set of assembled ESTs I can use as evidence for constructing gene models. A quick search of NCBI's SRA shows 4,318 human RNA-seq data sets, so I understand there is no shortage of data. I am hoping, however, to save the time that would be required to search through this massive amount of data, select an appropriate subset, and assemble the ESTs myself. I'm sure this has been done many times before, and for this particular task it doesn't make sense to repeat this process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any sort of general (i.e. not tissue-specific) reference EST/transcriptome assembly available for &lt;em&gt;Homo sapiens&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="47" LastActivityDate="2011-12-30T09:37:43.520" Title="Assembled human ESTs for annotation" Tags="|computational-biology|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="281" PostTypeId="2" ParentId="6" CreationDate="2011-12-07T18:19:59.423" Score="5" Body="&lt;p&gt;Advantage of the Gaussian basis is that you can use the Gaussian Product Theorem to simplify the two electron integrals at different centers. However, Gaussians ($e^{-\alpha r^2}$) don't capture the cusp of the wavefunction nor the exponential decay ($e^{-r}$) naturally, so one needs more Gaussians for the given accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Advantage of the Slater type orbitals (STO) $r^n e^{-\zeta r}$ is that they capture both the cusp and the exponential decay, but the disadvantage is that it is harder to calculate the two electron integrals at different centers (for atomic calculations this isn't a problem and so many atomic codes use STO).&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2011-12-07T18:19:59.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="282" PostTypeId="1" AcceptedAnswerId="297" CreationDate="2011-12-07T18:59:03.493" Score="15" ViewCount="408" Body="&lt;p&gt;Our plasma dynamics simulations often produce too much information. During the simulations we record various physical properties on a grid (x,y,z,t) that is as large as (8192x1024x1024x1500), for at least 10 properties. This information is processed after the simulation is completed. With it we&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;make movies of properties,&lt;/li&gt;&#xA;&lt;li&gt;perform a Fourier analysis,&lt;/li&gt;&#xA;&lt;li&gt;calculate average properties.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This simple dumping of as much information as possible worked fine when we studied smaller systems. This gave us the flexibility to interact with the results and decide later what we wanted to do with it. It also allowed us to allot our computational resources (CPU time) to simply running the simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We've begun the process of doing the Fourier analysis on the fly, and filtering for only a select range of length scales. For numerical reasons, we sometimes need to resolve length scales that are smaller than we are actually interested in, so in those cases, this filter helps greatly. We are also exploring various parallel IO libraries, e.g. &lt;a href=&quot;https://scicomp.stackexchange.com/q/194/79&quot;&gt;Parallel I/O options, in particular parallel HDF5&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What strategies are available to maximize the efficiency of data processing?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any benefit to performing all analysis (not including post processing, e.g. movies and plots) on the fly?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can imagine this issue coming up in other areas of research. For example, you might have a molecular dynamics simulation that needs to evolve for a long time, but you are interested in the brief moment when something interesting is happening. Or in CFD, the early time development might be slow, but once turbulence sets in, you might need a higher time resolution to monitor dynamics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there freely available examples of sophisticated results collection from simulations?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="79" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2011-12-09T12:14:10.113" Title="How to deal with too much data?" Tags="|performance|io|data-management|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="283" PostTypeId="1" AcceptedAnswerId="305" CreationDate="2011-12-07T19:08:33.370" Score="13" ViewCount="1510" Body="&lt;p&gt;I would like to write my own solver for compressible Euler equations, and most importantly I want it to work robustly in all situations. I would like it to be FE based (DG is ok). What are the possible methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of doing 0th order DG (finite volumes) and that should work very robustly. I have implemented a basic FVM solver and it works great, but the convergence is quite slow. However, this is definitely one option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have implemented an FE solver (works for any mesh and any polynomial order on any element) for linearized Euler equations, but I am getting spurious oscillations (and eventually it blows out, so I can't use it so solve my problem) and I have read in the literature that one needs to stabilize it. If I implement some stabilization, would that work robustly for all problems (=boundary conditions and geometries)? What will be the convergence rate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than that, is there some other robust methodology for Euler equations (i.e. higher order DG with some stabilization)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that many people tried lots of different things in their research codes, but I am interested in a robust method that works for all geometries and boundary conditions (edit: in 2D and 3D).&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="236" LastEditDate="2011-12-07T21:45:51.337" LastActivityDate="2011-12-08T06:45:45.513" Title="What are possible methods to solve compressible Euler equations" Tags="|pde|hyperbolic-pde|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="284" PostTypeId="2" ParentId="114" CreationDate="2011-12-07T19:17:31.843" Score="8" Body="&lt;p&gt;How you determine the costs of an algorithm depends on which &quot;level&quot; of scientific computing you work, and which (narrow or broad) class of problems you consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think about cache-optimization, this is clearly more relevant to, e.g., the implementation of numerical linear algebra packages like BLAS and similar libraries. So this belongs to low-level optimization, and it is fine if you have a fixed algorithm for a specific problem and with sufficient constraints on the input. For example, Cache optimization might be relevant to have a fast implementation of the conjugate gradient iteration if the matrix is promised to be sufficiently sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the broader the class of problems, the less you can predict on the actual computing (like, say, you don't know how sparse the input matrices of your CG implementation will really be). The broader the class of machines your program is supposed to run on, the less you can predict on the Cache architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, on a higher level of scientific computing, it might be more relevant to change the problem structure. For example, if you spend time in finding a good preconditioner for a linear system of equations, this kind of optimization usually beats any low-level optimization, because the number of iterations is drastically reduced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusio, cache optimization is useful only if there is nothing left to optimize by parallelism and reduction of asymptotic number of FLOPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it is wise to adapt the stance of theoretical computer science: In the end, improving the asymptotic complexity of an algorithm has more return than micro-optimization of some existing lines of code. Therefore, FLOPs counting is still prefered.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2011-12-07T19:17:31.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="285" PostTypeId="1" AcceptedAnswerId="2481" CreationDate="2011-12-07T19:22:19.900" Score="13" ViewCount="8238" Body="&lt;p&gt;Here is a typical usage of function pointers in C. I would like to do something similar in Fortran. I have some ideas, but I would like to know if there is some canonical way to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function pointers and contexts passed in by the user are stored, then called later.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;typedef PetscErrorCode (*TSIFunction)(TS,PetscReal,Vec,Vec,Vec,void*);&#xA;PetscErrorCode TSSetIFunction(TS ts,Vec res,TSIFunction f,void *ctx);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The user's function is called back using their context at various later times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In PETSc, they also make heavy use of string -&gt; function pointer tables. Everything is a plugin, so the user can register their own implementations and they are first-class.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#define PCGAMG &quot;gamg&quot;&#xA;  PCRegisterDynamic(PCGAMG         ,path,&quot;PCCreate_GAMG&quot;,PCCreate_GAMG);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This registers the creation routine in an &quot;FList&quot;, then PCSetFromOptions() offers the ability to choose this method versus any of the other choices. If the system supports dynamic loading, you can skip the compile-time dependency on the PCCreate_GAMG symbol and just pass NULL, then the symbol will be looked up in the shared library at run time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this one step beyond a &quot;factory&quot;, it's an inversion of control device similar to what Martin Fowler calls &quot;service locator&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: this came up in my private correspondence with Jed Brown, where he asked this question to me. I decided to outsource it and see what answers people can come up with.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="245" LastEditDate="2011-12-10T01:41:47.043" LastActivityDate="2012-06-09T07:07:30.783" Title="How to work with function pointers in Fortran in scientific programs" Tags="|petsc|fortran|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="286" PostTypeId="2" ParentId="285" CreationDate="2011-12-07T19:59:05.753" Score="9" Body="&lt;p&gt;There is a lot of what I'm guessing is PETSc-specific language in your question (with which I am unfamiliar), so there might be a wrinkle here I don't quite understand, but maybe this will still be useful to get you started.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you have to define the interface for the procedure, and then you can pass a pointer to a function that follows this interface. The following code shows an example. First, there is a module that defines the interface and shows a quick example of a chunk of code that would execute the routine provided by the user that follows that interface. Next is a program that shows how the user would use this module and define the function to be executed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MODULE xmod&#xA;&#xA;  ABSTRACT INTERFACE&#xA;  FUNCTION function_template(n,x) RESULT(y)&#xA;      INTEGER, INTENT(in) :: n&#xA;      REAL, INTENT(in) :: x(n)&#xA;      REAL :: y&#xA;  END FUNCTION function_template&#xA;  END INTERFACE&#xA;&#xA;CONTAINS&#xA;&#xA;  SUBROUTINE execute_function(n,x,func,y)&#xA;    INTEGER, INTENT(in) :: n&#xA;    REAL, INTENT(in) :: x(n)&#xA;    PROCEDURE(function_template), POINTER :: func&#xA;    REAL, INTENT(out) :: y&#xA;    y = func(n,x)&#xA;  END SUBROUTINE execute_function&#xA;&#xA;END MODULE xmod&#xA;&#xA;&#xA;PROGRAM xprog&#xA;&#xA;  USE xmod&#xA;&#xA;  REAL :: x(4), y&#xA;  PROCEDURE(function_template), POINTER :: func&#xA;&#xA;  x = [1.0, 2.0, 3.0, 4.0]&#xA;  func =&amp;gt; summation&#xA;&#xA;  CALL execute_function(4,x,func,y)&#xA;&#xA;  PRINT*, y  ! should give 10.0&#xA;&#xA;CONTAINS&#xA;&#xA;  FUNCTION summation(n,x) RESULT(y)&#xA;    INTEGER, INTENT(in) :: n&#xA;    REAL, INTENT(in) :: x(n)&#xA;    REAL :: y&#xA;    y = SUM(x)&#xA;  END FUNCTION summation&#xA;&#xA;END PROGRAM xprog&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastActivityDate="2011-12-07T19:59:05.753" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="287" PostTypeId="1" AcceptedAnswerId="288" CreationDate="2011-12-07T20:00:16.850" Score="5" ViewCount="1234" Body="&lt;p&gt;do you know good introductions into or surveys $hp$-adaptive finite elements?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular I do not know how the heuristics for choosing spatial refinement or increased polynomial degree are constructed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="123" LastEditDate="2012-01-31T20:26:28.067" LastActivityDate="2012-01-31T20:26:28.067" Title="Introductions to hp-FEM" Tags="|finite-element|education|reference-request|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="288" PostTypeId="2" ParentId="287" CreationDate="2011-12-07T20:11:16.300" Score="6" Body="&lt;p&gt;I found that &quot;Fully Automatic hp-Adaptivity in Three Dimensions&quot; by Rachowicz, Pardo, and Demkowicz has a pretty fully-developed discussion of the issues of choosing between h and p refinement and discusses trade-offs and the integration of those trade-offs into a refinement scheme.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's available here: &lt;a href=&quot;http://dx.doi.org/10.1016/j.cma.2005.08.022&quot; rel=&quot;noreferrer&quot;&gt;http://dx.doi.org/10.1016/j.cma.2005.08.022&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="143" LastEditDate="2011-12-07T20:20:31.583" LastActivityDate="2011-12-07T20:20:31.583" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="289" PostTypeId="2" ParentId="287" CreationDate="2011-12-07T20:18:38.077" Score="8" Body="&lt;p&gt;There is an excellent survey by William Mitchel, &lt;em&gt;A Comparison of hp-Adaptive Strategies for Elliptic Partial Differential Equations&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://math.nist.gov/~WMitchell/papers/hpexp_toms.pdf&quot;&gt;preprint&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://math.nist.gov/~WMitchell/papers/nistir7824.pdf&quot;&gt;long version&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30" LastActivityDate="2011-12-07T20:18:38.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="290" PostTypeId="1" AcceptedAnswerId="303" CreationDate="2011-12-07T20:32:29.440" Score="53" ViewCount="10956" Body="&lt;p&gt;I am used to thinking of finite-differences as a special case of finite-elements, on a very constrained grid. So what are the conditions on how to choose between Finite Difference Method (FDM) and Finite Element Method (FEM) as a numerical method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the side of Finite Difference Method (FDM), one may count that they are conceptually simpler and easier to implement than Finite Element Method (FEM). FEM have the benefit of being very flexible, e.g., the grids may be very non-uniform and the domains may have arbitrary shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only example I know where FDM has turned out superior to FEM is in&#xA;&lt;a href=&quot;http://www.math.oregonstate.edu/~mpesz/download/porous/celia90.pdf&quot; rel=&quot;noreferrer&quot;&gt;Celia, Bouloutas, Zarba&lt;/a&gt;, where the benefit is due to the FD method using a different discretization of time derivative, which, however, could be fixed for the finite element method.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="20572" LastEditDate="2016-11-18T17:51:48.053" LastActivityDate="2019-01-03T16:11:39.837" Title="What are criteria to choose between finite-differences and finite-elements" Tags="|pde|finite-element|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="291" PostTypeId="2" ParentId="6" CreationDate="2011-12-07T21:19:50.543" Score="3" Body="&lt;p&gt;From a &lt;em&gt;results&lt;/em&gt; point of view, for most things, it shouldn't really matter which you use - basis sets are generally constructed and labeled in such a way that for example a TZVP basis set will give comparable results whether it's using GTOs or STOs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cases where they're more likely to give different results will be properties that depend heavily on non-valence regions, for example, NMR data, which depend heavily upon the electronic density at and around the nucleus (for more info on this, you'll want to look up Fermi contact terms), and properties which may depend on the longer-distance decay of the density. STOs model the density at the nucleus more accurately assuming a point nucleus (which some programs don't - Gaussian for example uses a Gaussian for the nucleus by default), and neither type gets long-distance behaviour exactly right, as far as I know - STOs decay too slowly, GTOs too quickly.&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2011-12-07T21:19:50.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="292" PostTypeId="2" ParentId="290" CreationDate="2011-12-07T22:14:56.187" Score="10" Body="&lt;p&gt;Advantages of finite elements (FE):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;variational method (e.g. energies always drop with increasing &quot;p&quot; for Schroedinger equation, which is not true for FD)&lt;/li&gt;&#xA;&lt;li&gt;accurate at high orders (p=50 more more)&lt;/li&gt;&#xA;&lt;li&gt;once implemented, it is easy to do systematic convergence both in &quot;p&quot; and in &quot;h&quot; (as opposed to having special FD schemes for each order)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Advantages of finite differences (FD):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;easier to implement for lower orders&lt;/li&gt;&#xA;&lt;li&gt;possibly faster than FE for lower accuracies&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Sometimes people say &quot;finite differences&quot; to mean an integrator for ODE like Runge-Kutta or the Adams method. In that case, there is another advantage of FD:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;possible to solve nonlinear ODEs directly&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;while FE need some nonlinear iteration like the Newton method.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2011-12-07T22:14:56.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="293" PostTypeId="2" ParentId="282" CreationDate="2011-12-07T22:22:06.907" Score="3" Body="&lt;p&gt;Peter LePage is pretty famous in lattice-QCD circles for suggesting a method whereby unfeasibly-large lattice grids could be reduced by finding and apply good short ranges analytic solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is roughly equivalent to noticing that a set of well chosen splines can allow accurate integration with fewer knots than the trapezoid method (except that as in your case you get to take advantage of it over four dimensions at once).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is that you trade raw size of the data-set for more computation per node--step, but come out ahead in the end &lt;em&gt;because&lt;/em&gt; of the high dimensionality of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I's not a subject I know well enough to give any decent hints on, but it has worked in some fields in the past.&lt;/p&gt;&#xA;" OwnerUserId="183" LastActivityDate="2011-12-07T22:22:06.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="294" PostTypeId="2" ParentId="252" CreationDate="2011-12-07T22:25:44.550" Score="2" Body="&lt;p&gt;Finding reliable sites for this kind of thing can be kind of tricky, actually.  I recently had to search for small molecule structures and found that half the top sites on the Google search were down.  Therefore this kind of answer may be quite helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My preferred source of small ligand structures is the &lt;a href=&quot;http://www.bindingdb.org/bind/index.jsp&quot; rel=&quot;nofollow&quot;&gt;Binding DB&lt;/a&gt;.  It's in .mol file format, but how to convert easily has been mentioned in previous answers.  For a finite number of small molecules converting this by-hand is almost simpler than automating it.&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2011-12-07T22:25:44.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="295" PostTypeId="2" ParentId="282" CreationDate="2011-12-07T23:23:41.713" Score="8" Body="&lt;p&gt;I think the current masters of this art are the large particle physics experiments (I am most familiar with &lt;a href=&quot;http://www-cdf.fnal.gov/&quot;&gt;CDF&lt;/a&gt; and &lt;a href=&quot;http://www-d0.fnal.gov/&quot;&gt;D0&lt;/a&gt; because I am old and work at the University of Chicago). They have hardware triggers that discard petabytes (or more) a year. However, this is the whole subject of quantization/discretization, or &quot;throwing away only what you don't need&quot;. I am not sure you can give a sensible answer in general. It would be better to narrow down the problem to something like, &quot;I have a PDE simulation discretized in the following way and would like to efficiently downsample&quot;.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-07T23:23:41.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="296" PostTypeId="1" AcceptedAnswerId="306" CreationDate="2011-12-08T00:17:45.293" Score="13" ViewCount="420" Body="&lt;p&gt;In both domain decomposition (DD) and multigrid (MG) methods, one may compose the application of the block updates or coarse corrections as either &lt;em&gt;additive&lt;/em&gt; or &lt;em&gt;multiplicative&lt;/em&gt;. For pointwise solvers, this is the difference between the Jacobi and Gauss-Seidel iterations. The multiplicative smoother for $Ax = b$ acting as $S(x^{old}, b) = x^{new}$ is applied as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xD;&#xA;x_{i+1} = S_n(S_{n-1}( ..., S_1(x_i, b) ..., b), b)&#xD;&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the additive smoother is applied as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xD;&#xA;x_{i+1} = x_{i} + \displaystyle\sum_{\ell = 0}^{n}\lambda_\ell(S_\ell(x_i, b) - x_i)&#xD;&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some damping $\lambda_i$.  The general consensus appears to be that multiplicative smoothers have much more rapid convergence properties, but I was wondering: under what situations is the performance of the additive variants of these algorithms better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, Does anyone have any use cases where the additive variant should and/or does perform significantly better than the multiplicative variant?  Are there theoretical reasons for this?  Most literature on multigrid is &lt;a href=&quot;http://conan.iwr.uni-heidelberg.de/people/peter/pdf/BastianHackbuschWittum_Computing_1998.pdf&quot; rel=&quot;noreferrer&quot;&gt;fairly pessimistic&lt;/a&gt; about the Additive method, but it's used so much in the DD context as additive Schwarz.  This also extends to the much more general issue of composing linear and nonlinear solvers, and which type of constructions will perform well and perform well in parallel.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="142" LastEditDate="2011-12-10T01:43:09.770" LastActivityDate="2011-12-12T06:09:03.743" Title="In what application cases are additive preconditioning schemes superior to multiplicative ones?" Tags="|linear-algebra|performance|multigrid|domain-decomposition|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="297" PostTypeId="2" ParentId="282" CreationDate="2011-12-08T00:48:01.877" Score="11" Body="&lt;p&gt;I think you might have to split your output to match your targets:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;for the movies of properties, you probably don't need the full spatial resolution and all variables. Carefully choose what you want to show and think about the final resolution of the movie you will be showing, it probably won't have 8 billion pixels.&lt;/li&gt;&#xA;&lt;li&gt;For the Fourier analyses (or things like POD), if they are temporal, you can probably just sample a few hundred points wisely chosen in your domain. If they are spatial, you probably only need a few snapshots and not 1500. And again, not of all properties.&lt;/li&gt;&#xA;&lt;li&gt;For time averaging, you can just keep adding to the same field and don't have to worry about the time dimension right? Spatial averaging is painful though, especially if you want to look at its evolution over time. But more online processing before dumping the data could reduce the size of it...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This means quite a bit of work to have dedicated outputs instead of a big generic one but this should help keeping the cost and size down. Hope this helps !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just one more thing I want to add, in general, the full resolution of the data is only needed for restart files, ie files to restart your simulation. You don't need that many of these for a given simulation (let's say 100, so that if something happens between 2 restarts you lose at most 1% of your computation), whereas you probably want to crank up the frequency of output for your movies. And you can do that at just 1/64th of the resolution for example (1 every 4 points in each direction).&lt;/p&gt;&#xA;" OwnerUserId="95" LastEditorUserId="95" LastEditDate="2011-12-08T03:52:43.797" LastActivityDate="2011-12-08T03:52:43.797" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="298" PostTypeId="2" ParentId="287" CreationDate="2011-12-08T02:11:01.793" Score="1" Body="&lt;p&gt;Barna Szabo at Washington University was The Man in this area when I was practicing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/Finite_element_analysis.html?id=JsCg-QWUT28C&quot; rel=&quot;nofollow&quot;&gt;http://books.google.com/books/about/Finite_element_analysis.html?id=JsCg-QWUT28C&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="247" LastActivityDate="2011-12-08T02:11:01.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="299" PostTypeId="2" ParentId="290" CreationDate="2011-12-08T02:29:04.570" Score="8" Body="&lt;p&gt;It's important to distinguish between spatial and temporal schemes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite elements often use finite differences to integrate temporal terms (e.g. explicit Euler, implicit, Crank-Nicholson, or Runga Kutta for transient diffusion) and finite elements for spatial discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite elements lend themselves nicely to irregular meshes.  They can be based on variational principles, but they are usually generalized using the method of weighted residuals.  It's easy to develop libraries of elements that use different polynomial orders and enforce constraints like incompressibility using Lagrange multipliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both formulations are the means to an end: expressing a differential equation in terms of systems of equations and linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Statements about speed of one method over another need to be qualified by describing the algorithm.  For example, casting mechanical problems as hyperbolic dynamics problems can give faster results in some cases, because they replace matrix decomposition with multiplication and addition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll admit that I know a great deal more about finite element methods than I do finite differences.  FEM is available in commercial packages and are widely used in industry and academia to solve problems in solid mechanics and heat transfer.  I believe finite differences or finite volume approaches are used in computational fluid dynamics.&lt;/p&gt;&#xA;" OwnerUserId="247" LastEditorUserId="29633" LastEditDate="2019-01-03T16:11:39.837" LastActivityDate="2019-01-03T16:11:39.837" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="300" PostTypeId="2" ParentId="290" CreationDate="2011-12-08T05:19:51.583" Score="25" Body="&lt;p&gt;This question may be too broad to have a meaningful answer.  Most people who answer will only be familiar with some subset of all the kinds of FD and FE discretizations that may be used.  Note that both FD and FE &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;can be implemented on &lt;strong&gt;structured&lt;/strong&gt; or &lt;strong&gt;unstructured&lt;/strong&gt; grids (see &lt;a href=&quot;http://dx.doi.org/10.1002/nme.664&quot;&gt;this paper&lt;/a&gt; for just one example of a FD method on an unstructured grid)&lt;/li&gt;&#xA;&lt;li&gt;can be extended to &lt;strong&gt;arbitrarily high order of accuracy&lt;/strong&gt; (in many ways!)&lt;/li&gt;&#xA;&lt;li&gt;can be used to discretize in &lt;strong&gt;space&lt;/strong&gt; and/or in &lt;strong&gt;time&lt;/strong&gt;, perhaps in combination&lt;/li&gt;&#xA;&lt;li&gt;use either &lt;strong&gt;local&lt;/strong&gt; or &lt;strong&gt;global&lt;/strong&gt; basis functions (the latter lead to spectral methods of both FD and FE type)&lt;/li&gt;&#xA;&lt;li&gt;can be based on a &lt;strong&gt;continuous&lt;/strong&gt; or &lt;strong&gt;discontinuous&lt;/strong&gt; function space&lt;/li&gt;&#xA;&lt;li&gt;can be spatially &lt;strong&gt;explicit&lt;/strong&gt; or &lt;strong&gt;implicit&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;can be temporally &lt;strong&gt;explicit&lt;/strong&gt; or &lt;strong&gt;implicit&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You get the idea.  Of course, in a particular discipline, the FD and FE methods that people commonly implement and use may have very different features.  But this is usually not due to any inherent limitations of the two discretization approaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding FD schemes of arbitrarily high order: the coefficients of high order FD schemes can be automatically generated for any order; see &lt;a href=&quot;http://www.amath.washington.edu/~rjl/fdmbook/&quot;&gt;LeVeque's book&lt;/a&gt;, for instance.  Spectral collocation methods, which are FD methods, will converge faster than any power of the mesh spacing; see &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/spectral.html&quot;&gt;Trefethen's book&lt;/a&gt;, for instance.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-12-08T06:42:02.260" LastActivityDate="2011-12-08T06:42:02.260" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="301" PostTypeId="5" CreationDate="2011-12-08T05:26:36.857" Score="0" Body="&lt;p&gt;Originally named FORTRAN during its early years, it is now referred to as Fortran. This is valid for Fortran90 and onwards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The file extensions associated with this language are: &lt;code&gt;.for&lt;/code&gt;, &lt;code&gt;.f&lt;/code&gt;, &lt;code&gt;.f90&lt;/code&gt;, &lt;code&gt;.F&lt;/code&gt;, &lt;code&gt;.F90&lt;/code&gt;, &lt;code&gt;.f95&lt;/code&gt;, &lt;code&gt;.F95&lt;/code&gt;. The capitalized extension means the source code include macro &lt;code&gt;ifdef&lt;/code&gt; statement for the C preprocessor. &lt;/p&gt;&#xA;" OwnerUserId="95" LastEditorUserId="95" LastEditDate="2011-12-09T00:27:44.690" LastActivityDate="2011-12-09T00:27:44.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="302" PostTypeId="4" CreationDate="2011-12-08T05:26:36.857" Score="0" Body="Fortran is a general purpose programming language widely used by the scientific computing community." OwnerUserId="95" LastEditorUserId="95" LastEditDate="2011-12-09T00:27:48.887" LastActivityDate="2011-12-09T00:27:48.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="303" PostTypeId="2" ParentId="290" CreationDate="2011-12-08T05:39:12.730" Score="52" Body="&lt;p&gt;It is possible to write most specific finite difference methods as Petrov-Galerkin finite element methods with some choice of local reconstruction and quadrature, and most finite element methods can also be shown to be algebraically equivalent to some finite difference method. Therefore, we should choose a method based on which analysis framework we want to use, which terminology we like, which system for extensibility we like, and how we would like to structure software. The following generalizations hold true in the vast majority of variations in practical use, but many points can be circumvented.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Finite Difference&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Pros&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;efficient quadrature-free implementation&lt;/li&gt;&#xA;&lt;li&gt;aspect ratio independence and local conservation for certain schemes (e.g. MAC for incompressible flow)&lt;/li&gt;&#xA;&lt;li&gt;robust nonlinear methods for transport (e.g. ENO/WENO)&lt;/li&gt;&#xA;&lt;li&gt;M-matrix for some problems&lt;/li&gt;&#xA;&lt;li&gt;discrete maximum principle for some problems (e.g. mimetic finite differences)&lt;/li&gt;&#xA;&lt;li&gt;diagonal (usually identity) mass matrix&lt;/li&gt;&#xA;&lt;li&gt;inexpensive &lt;em&gt;nodal&lt;/em&gt; residual permits efficient nonlinear multigrid (FAS)&lt;/li&gt;&#xA;&lt;li&gt;cell-wise Vanka smoothers give efficient matrix-free smoothers for incompressible flow&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Cons&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;more difficult to implement &quot;physics&quot;&lt;/li&gt;&#xA;&lt;li&gt;staggered grids are sometimes quite technical&lt;/li&gt;&#xA;&lt;li&gt;higher than second order on unstructured grids is difficult&lt;/li&gt;&#xA;&lt;li&gt;no Galerkin orthogonality, so convergence may be more difficult to prove&lt;/li&gt;&#xA;&lt;li&gt;not a Galerkin method, so discretization and adjoints do not commute (relevant to optimization and inverse problems)&lt;/li&gt;&#xA;&lt;li&gt;self-adjoint continuum problems often yield non-symmetric matrices&lt;/li&gt;&#xA;&lt;li&gt;solution is only defined pointwise, so reconstruction at arbitrary locations is not uniquely defined&lt;/li&gt;&#xA;&lt;li&gt;boundary conditions tend to be complicated to implement&lt;/li&gt;&#xA;&lt;li&gt;discontinuous coefficients usually make the methods first order&lt;/li&gt;&#xA;&lt;li&gt;stencil grows if physics includes &quot;cross terms&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Finite Element&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Pros&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Galerkin orthogonality (discrete solution to coercive problems is within a constant of the best solution in the space)&lt;/li&gt;&#xA;&lt;li&gt;simple geometric flexibility&lt;/li&gt;&#xA;&lt;li&gt;discontinuous Galerkin offers robust transport algorithm, arbitrary order on unstructured grids&lt;/li&gt;&#xA;&lt;li&gt;cellwise entropy inequality guaranteeing $L^2$ stability holds independent of mesh, dimension, order of accuracy, and presence of discontinuous solutions, without needing nonlinear limiters&lt;/li&gt;&#xA;&lt;li&gt;easy of implementing boundary conditions&lt;/li&gt;&#xA;&lt;li&gt;can choose conservation statement by choosing test space&lt;/li&gt;&#xA;&lt;li&gt;discretization and adjoints commute (for Galerkin methods)&lt;/li&gt;&#xA;&lt;li&gt;elegant foundation in functional analysis&lt;/li&gt;&#xA;&lt;li&gt;at high order, local kernels can exploit tensor product structure that is missing with FD&lt;/li&gt;&#xA;&lt;li&gt;Lobatto quadrature can make methods energy-conserving (assuming a symplectic time integrator)&lt;/li&gt;&#xA;&lt;li&gt;high order accuracy even with discontinuous coefficients, as long as you can align to boundaries&lt;/li&gt;&#xA;&lt;li&gt;discontinuous coefficients inside elements can be accommodated with XFEM&lt;/li&gt;&#xA;&lt;li&gt;easy to handle multiple inf-sup conditions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Cons&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;many elements have trouble at high aspect ratio&lt;/li&gt;&#xA;&lt;li&gt;continuous FEM has trouble with transport (SUPG is diffusive &lt;em&gt;and&lt;/em&gt; oscillatory)&lt;/li&gt;&#xA;&lt;li&gt;DG usually has more degrees of freedom for same accuracy (though HDG is much better)&lt;/li&gt;&#xA;&lt;li&gt;continuous FEM does not provide cheap nodal problems, so nonlinear smoothers have much poorer constants&lt;/li&gt;&#xA;&lt;li&gt;usually more nonzeros in assembled matrices&lt;/li&gt;&#xA;&lt;li&gt;have to choose between consistent mass matrix (some nice properties, but has full inverse, thus requiring an implicit solve per time step) and lumped mass matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2011-12-08T06:13:05.903" LastActivityDate="2011-12-08T06:13:05.903" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="304" PostTypeId="1" AcceptedAnswerId="309" CreationDate="2011-12-08T05:41:02.653" Score="67" ViewCount="49650" Body="&lt;p&gt;In my computational science PhD program, we are working almost exclusively in C++ and Fortran. It seems like some professors prefer one over the other. I am wondering which one is 'better' or if one is better than the other in a certain circumstance.&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="30" LastEditDate="2011-12-08T15:44:17.637" LastActivityDate="2021-04-30T22:39:51.137" Title="C++ vs Fortran for HPC" Tags="|hpc|fortran|c++|languages|" AnswerCount="13" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="305" PostTypeId="2" ParentId="283" CreationDate="2011-12-08T05:44:35.340" Score="11" Body="&lt;p&gt;The principal numerical difficulty in solving a nonlinear first-order system of hyperbolic PDEs like the Euler equations (for compressible, inviscid flow) is that discontinuities (shock waves) appear in the solution after finite time, even if the initial data are smooth.  In order to deal with this, most modern codes use both&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;slope&lt;/strong&gt; (or &lt;strong&gt;flux&lt;/strong&gt;) &lt;strong&gt;limiters&lt;/strong&gt;, which provide a way of computing derivatives accurately near discontinuities without introducing spurious oscillations; and&lt;/li&gt;&#xA;&lt;li&gt;Approximate &lt;strong&gt;Riemann solvers&lt;/strong&gt;, which locally (at each grid edge/face) solve an initial value problem with piecewise constant initial data and a single discontinuity.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There exist finite difference (FD), finite volume (FV), and finite element (FE) discretizations that incorporate limiters and Riemann solvers, and all can be made highly accurate, at least away from shocks.  So it doesn't make sense to say categorically that FE methods converge faster than FV methods -- they will be comparable if comparable order discretizations are used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among FE methods, &lt;a href=&quot;http://en.wikipedia.org/wiki/Discontinuous_Galerkin_method&quot;&gt;discontinuous Galerkin methods&lt;/a&gt; are the most suitable here, since the solution will in fact be discontinuous.  If you want to implement your own, I suggest that you read &lt;a href=&quot;http://www.ima.umn.edu/preprints/scanned-preprint/preprints5/1662.pdf&quot;&gt;this review paper&lt;/a&gt; and get a copy of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387720650&quot;&gt;Hesthaven &amp;amp; Warburton's text&lt;/a&gt; to understand the basics.  Then there are lots of &lt;a href=&quot;http://scholar.google.com/scholar?hl=en&amp;amp;q=discontinuous%20galerkin%20compressible%20euler&quot;&gt;papers on DG for compressible flow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're willing to use someone else's code, and since I know you use Python, you might take a look at &lt;a href=&quot;http://mathema.tician.de/software/hedge&quot;&gt;Andreas Kloeckner's Hedge code&lt;/a&gt;, which has a Python interface and can run on GPUs.  There are probably other good DG codes available, and many good FV codes (such as &lt;a href=&quot;http://www.clawpack.org&quot;&gt;Clawpack&lt;/a&gt;, which also has a &lt;a href=&quot;http://numerics.kaust.edu.sa/pyclaw/&quot;&gt;Python interface&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also newer high-order methods such as spectral difference. For a recent perspective, see &lt;a href=&quot;http://www.dam.brown.edu/scicomp/media/report_files/BrownSC-2009-03.pdf&quot;&gt;Cheng &amp;amp; Shu 2009, High Order Schemes for CFD: A Review&lt;/a&gt; or &lt;a href=&quot;http://dx.doi.org/10.1016/j.paerosci.2005.03.003&quot;&gt;Ekaterinaris 2005, High-order accurate, low numerical diffusion methods for aerodynamics&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-12-08T06:45:45.513" LastActivityDate="2011-12-08T06:45:45.513" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="306" PostTypeId="2" ParentId="296" CreationDate="2011-12-08T05:48:12.710" Score="6" Body="&lt;p&gt;Additive methods expose more concurrency. They are generally only faster than multiplicative methods if you can use that concurrency. For example, coarse levels of multigrid are typically latency-limited. If you move coarse levels to smaller subcommunicators, then they could be solved independently from the finer levels. With a multiplicative scheme, all the procs have to wait while the coarse levels are solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if the algorithm needs reductions on every level, the additive variant may be able to coalesce them where as the multiplicative method is forced to perform them sequentially.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-12-08T05:48:12.710" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="307" PostTypeId="2" ParentId="304" CreationDate="2011-12-08T06:32:35.200" Score="39" Body="&lt;p&gt;I think that both C++ and Fortran are good enough and work well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I think that Fortran is better for &lt;em&gt;numeric&lt;/em&gt; scientific computing, for algorithms that can be expressed using arrays and don't need other sophisticated data structures, so in fields like finite differences/elements, PDE solvers, electronic structure calculations. Fortran is a domain specific language. In particular I think that it is easier to write &lt;em&gt;fast&lt;/em&gt; programs in Fortran than in C++, by a scientist (not necessarily a computer science expert).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++ is a general purpose language, so one can express any algorithm in it, and it is most definitely better for algorithms that can't be expressed using arrays, from HPC field probably some graphs, mesh generators, symbolic manipulation and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also possible to write array algorithms in C++, but in my experience, it requires much more computer science knowledge and in general more work (i.e. one needs to create or reuse classes for array manipulation, and handle memory management by hand or using some library like Teuchos from Trilinos). Non-experts tend to write pretty good Fortran programs, but horrible C++ programs (talking from my own experience).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I personally like Fortran a lot and I prefer it over C++ for numeric computing. I have spent over 2 years of programming in C++ daily, and almost a year programming in modern Fortran daily (in finite elements area). I use Python and Cython a lot too.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2011-12-08T06:32:35.200" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="308" PostTypeId="2" ParentId="304" CreationDate="2011-12-08T11:46:30.827" Score="8" Body="&lt;p&gt;The problem with C++ is that you have numerous chances to ruin performance, for instance by blindly using STL, exceptions, classes (virtual overhead plus alignment problems), operator overloading (redundant new/deletes) or templates (never-ending compilation and cryptic errors seem benign, but you can waste hours this way). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, more you gain better access to general libraries and possibly grater visibility of your code (although this strongly depends on the field, and you still have pure C). And you can still compensate the Fortran's lack of flexibility by wrapping its code in a script language like R, Lush, Matlab/Scilab or even Python, Ruby or Lua. &lt;/p&gt;&#xA;" OwnerUserId="102" LastActivityDate="2011-12-08T11:46:30.827" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="309" PostTypeId="2" ParentId="304" CreationDate="2011-12-08T13:56:24.130" Score="70" Body="&lt;p&gt;As so often, the choice depends on (1) the problem you are trying to solve, (2) the skills you have, and (3) the people you work with (unless it's a solo project). I'll leave (3) aside for the moment because it depends on everyone's individual situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem dependence: Fortran excels at array processing. If your problem can be described in terms of simple data structures and in particular arrays, Fortran is well adapted. Fortran programmers end up using arrays even in non-obvious cases (e.g. for representing graphs). C++ is better suited for complex and highly dynamic data structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Skill dependence: it takes a lot more programming experience to write good C++ programs than to write good Fortran programs. If you start out with little programming experience and only have so much time to learn that aspect of your job, you probably get a better return on investment learning Fortran than learning C++. Assuming, of course, that your problem is suited to Fortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there's more to programming than just Fortran and C++. I'd recommend to anyone going into computational science to start with a dynamic high-level language such as Python. Always remember that your time is more valuable than CPU time!&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-12-08T13:56:24.130" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="310" PostTypeId="2" ParentId="296" CreationDate="2011-12-08T14:40:54.283" Score="0" Body="&lt;p&gt;I will restate what @Jed said: The Multiplicative method always converges at least as well as the Additive method (asymptotically), so you only win based on concurrency, but that is architecture-dependent.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-08T14:40:54.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="311" PostTypeId="1" AcceptedAnswerId="312" CreationDate="2011-12-08T15:39:54.587" Score="3" ViewCount="1098" Body="&lt;p&gt;I am using the Weka workbench to train a protein fold classifier. I imported my training data into Weka and performed PCA-based feature selection. This seems to have worked fine, but now I cannot evaluate my trained classifier on the test data because the test data contains all the original attributes. Of course, if I try to run the feature selection on the test data, I will come up with a different set of features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Weka, after you have applied feature selection to a training set, how do you pull those same features out of a test set?&lt;/p&gt;&#xA;" OwnerUserId="47" LastActivityDate="2011-12-12T14:49:54.247" Title="Applying same feature selection to multiple data sets with Weka" Tags="|machine-learning|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="312" PostTypeId="2" ParentId="311" CreationDate="2011-12-08T16:10:03.163" Score="3" Body="&lt;p&gt;With any such modelling thing, you are going to have to recalculate the model using the new training set (ie. the original minus its test set).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual approach is to randomly extract a subset for testing. Then train using all of the remaining data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course there will be some random variability according to which are extracted, so you can repeat the process to get some statistical significance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your final model will not be trained on all of your data. You either have to live with that (what I usually see in the Natural Language Processing field), or once you have determined your best parameters compute the final model using all of your data - with the understand you won't be able to test it.&lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2011-12-08T16:10:03.163" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="313" PostTypeId="2" ParentId="304" CreationDate="2011-12-08T16:36:47.240" Score="5" Body="&lt;p&gt;Fortran is optimized for array/matrix computations and is a thorough pain to work with for any type of text parsing. C and C++ may not match up with Fortran in numerical computing (it's close),but I find it much easier to process text and organize data (i.e. custom data structures) with C/C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others have mentioned, don't count out dynamic interpreted languages (Python et al). They may not offer the face-melting speed of Fortan up front, but they allow you to focus more on solving your computational problem than all the details of implementation. Often you can implement a solution in Python, and if the performance is unacceptable, do some profiling, identify the problem areas, and either optimize that code using Cython or re-implement the entire program in a compiled language. Once you have the problem-solving logic fleshed out, the rest is just implementation and, with a good understanding of computing fundamentals, should be straightforward to represent in any variety of programming languages.&lt;/p&gt;&#xA;" OwnerUserId="47" LastActivityDate="2011-12-08T16:36:47.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="314" PostTypeId="2" ParentId="304" CreationDate="2011-12-08T17:49:08.827" Score="4" Body="&lt;p&gt;I'm currently working at one of the national labs. Most of the folks around me are mechanical engineers. Chatting with some of the folks in the HPC groups, they're doing mostly Linux and mostly C++. The group I'm currently in does mostly desktop applications and we use Windows and in descending order: C#, FORTRAN, Python, VBA and VB (6, not .NET). Some of the &lt;a href=&quot;http://doe2.com/&quot; rel=&quot;nofollow&quot;&gt;simulation engines&lt;/a&gt; we use were written at other national labs in FORTRAN. &lt;/p&gt;&#xA;" OwnerUserId="16" LastActivityDate="2011-12-08T17:49:08.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="315" PostTypeId="5" CreationDate="2011-12-09T01:13:34.590" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-09T01:13:34.590" LastActivityDate="2011-12-09T01:13:34.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="316" PostTypeId="4" CreationDate="2011-12-09T01:13:34.590" Score="0" Body="Questions about the execution speed and memory usage of algorithms, data structures, languages and libraries." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-13T00:51:52.030" LastActivityDate="2012-01-13T00:51:52.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="317" PostTypeId="5" CreationDate="2011-12-09T01:14:48.567" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-09T01:14:48.567" LastActivityDate="2011-12-09T01:14:48.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="318" PostTypeId="4" CreationDate="2011-12-09T01:14:48.567" Score="0" Body="Relating to the level of difficulty of a calculation or the asymptotic running time of an algorithm." OwnerUserId="261" LastEditorUserId="549" LastEditDate="2012-02-07T15:57:25.693" LastActivityDate="2012-02-07T15:57:25.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="319" PostTypeId="5" CreationDate="2011-12-09T01:16:27.020" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2011-12-09T01:16:27.020" LastActivityDate="2011-12-09T01:16:27.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="320" PostTypeId="4" CreationDate="2011-12-09T01:16:27.020" Score="0" Body="Machine learning, a branch of artificial intelligence, is the science of getting computer systems to meaningfully act without being explicitly programmed by human." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2013-11-20T01:07:58.937" LastActivityDate="2013-11-20T01:07:58.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="321" PostTypeId="2" ParentId="304" CreationDate="2011-12-09T01:56:06.907" Score="13" Body="&lt;p&gt;My approach has been to use C++ for everything but computational kernels, which are usually best written in assembly; this buys you all of the performance of the traditional HPC approach but allows you to simplify the interface, e.g., by overloading computational kernels like SGEMM/DGEMM/CGEMM/ZGEMM into a single routine, say Gemm. Clearly the abstraction level can be raised much higher by avoiding raw pointers and switching to opaque classes, but it is a nice first step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find the largest downside of C++ to overwhelmingly be the increase in compilation time, but, in my experience, the savings in development time more than make up for it. Another downside is that vendor C++ compilers tend to have more bugs than vendor C and Fortran compilers. In the past year, I think I have run into nearly ten bugs in C++ compilers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With all of that said, I think that the undoing of scientific packages written in low-level languages (and Fortran) is the reluctance to expose convenient interfaces for sophisticated data structures: most people are satisfied with the Fortran BLAS interface, as it only requires pointers and leading dimensions to describe matrices, but few people would argue that the usual 40-integer Fortran sparse-direct solver interface is anything close to convenient (cf. UHM, SuperLU, PETSc, and Trilinos). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, I argue for using assembly for low-level computational kernels, but higher level languages for everything else, especially when operating on non-trivial data structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this post resulted in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1194/how-much-better-are-fortran-compilers-really&quot;&gt;this comparison of the performance of C and Fortran on the kernel $y := \alpha x + y$&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-04-01T02:53:50.690" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="322" PostTypeId="2" ParentId="277" CreationDate="2011-12-09T05:33:43.627" Score="2" Body="&lt;p&gt;This is my plan for teaching myself some HPC foundations this coming spring. I've been a developer for the past decade and half, and work exclusively with Microsoft technologies. I'm also a student, so I can get some software free through &lt;a href=&quot;http://msdn.microsoft.com/en-us/academic/dd759402&quot; rel=&quot;nofollow noreferrer&quot;&gt;MSDN Academic Alliance&lt;/a&gt; (your profile indicates you're a grad student, so you should check to see if your school participates), and making room for stuff at home is the limiting factor. My office/den/ManCave already has a number of computers in it so I can already test some stuff. Windows Server 2008 HPC is listed in some available packages, but it seemed to disappear off my school's downloadable software during the past semester (&quot;cluster compute pack&quot; is an add-on for server 2003, while server 2008 hpc comes with a stripped down version of server 2008, with ccp, you'll have to disable services you don't need). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My planned topology is the &quot;SOA Enabled Cluster&quot; which is the 4th image on &lt;a href=&quot;http://www.microsoft.com/hpc/en/us/product/cluster-computing.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; (click the tiny squares under the main picture). I already have Active Directory and some other servers already set up, so I'm thinking about getting 4 cheap Dell small form factor computers off eBay (refurb Dell 655 SFF appear to be running in the $100-200 ballpark). Since I'm only trying to learn some stuff, I believe I only need a &quot;head node&quot; a &quot;broker node&quot; and 2 &quot;compute nodes&quot; and a few spare ethernet cards to see what the private and application busses (from that image) do. The MSDNAA license permits up to 16 compute nodes which is far more than I'm looking for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been reading &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0262692759&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beowulf Cluster Computing with Windows&lt;/a&gt;, and I think I've got an idea what I'm up against for &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7/desktop-software-with-hpc-resources-for-back-end-number-crunching&quot;&gt;some ideas&lt;/a&gt; at the office (there is a linux version of the book, but I have 0 linux skills/platforms at home - do what is right for yourself). There are a lot of big clusters available &lt;a href=&quot;http://www.nrel.gov/csc/capabilities.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;at the office&lt;/a&gt;, but they're already saturated, and educational &quot;I want/need to learn X&quot; time slots are very hard to get. One is already expected to know the stuff before one can submit jobs. &lt;/p&gt;&#xA;" OwnerUserId="16" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2011-12-09T05:33:43.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="323" PostTypeId="2" ParentId="282" CreationDate="2011-12-09T12:14:10.113" Score="3" Body="&lt;p&gt;The question is a bit broad, so I will provide a correspondingly vague answer that suggests possible techniques in such cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) On-the-fly processing, which you are already working on. One way to do on-the-fly processing and yet decouple it from the data-generating step is to generate a cyclic output file that always contains the last N steps, and have the analysis run in a separate process. Obviously you must synchronize the two to prevent a race condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Choosing the stored data more carefully. This is highly situation-specific, unfortunately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Compress your data before storing it, or use a storage library with integrated compression options, such as HDF5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Store regular checkpoints instead of full output. If you store a full checkpoint every N steps, i.e. enough data to restart the simulation from there, you can reconstruct the missing data in a highly parallel fashion if and when necessary. Note that in the case of Monte-Carlo methods, the checkpoint must include the state of the random number generators. You can actually consider this a highly application-specific compression technique.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-12-09T12:14:10.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="324" PostTypeId="1" CreationDate="2011-12-09T22:11:55.210" Score="6" ViewCount="598" Body="&lt;p&gt;What efficient implementations of a 'drizzle' algorithm are available?  The problem is, given a timestream of data in which each element is associated with a pixel in a map, how do you create that map?  Each pixel may have many data points associated with it.  Each data point may need to be weighted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in python/numpy, given a data array &lt;code&gt;d&lt;/code&gt;, a weight array &lt;code&gt;w&lt;/code&gt;, a map &lt;code&gt;m&lt;/code&gt;, a weight map &lt;code&gt;wm&lt;/code&gt;, and a mapping from d to m &lt;code&gt;xinds,yinds&lt;/code&gt;, you could do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for jj,(xx,yy) in enumerate(xinds,yinds):&#xA;    m[xx,yy] += (d*w)[jj]&#xA;    wm[xx,yy] += w[jj]&#xA;final_image = m/wm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;xx&lt;/code&gt;,&lt;code&gt;yy&lt;/code&gt;,&lt;code&gt;d&lt;/code&gt;, and &lt;code&gt;w&lt;/code&gt; have the same length.  Also, &lt;code&gt;xx&lt;/code&gt; and &lt;code&gt;yy&lt;/code&gt; are matrix &lt;code&gt;x&lt;/code&gt;,&lt;code&gt;y&lt;/code&gt; locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this be made more efficient?  Are there tools in python or libraries in other languages to do this?  Am I even calling the algorithm by its right name?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An efficient implementation in &lt;code&gt;IDL&lt;/code&gt; using the &lt;code&gt;histogram&lt;/code&gt; function is shown at &lt;a href=&quot;http://www.idlcoyote.com/code_tips/drizzling.html&quot; rel=&quot;nofollow&quot;&gt;David Fanning's website&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;After asking this question, I realized I had the answer... &lt;code&gt;numpy.bincount&lt;/code&gt; does exactly what I want in numpy.  If the mapping &lt;code&gt;t = xinds + yinds*xsize&lt;/code&gt; where &lt;code&gt;xsize&lt;/code&gt; is the x-dimension of the map&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# shapes of x,y indices need to be flat&#xA;x,y = (a.ravel() for a in numpy.indices(m.shape))&#xA;dc = numpy.bincount(t,d*w)&#xA;wc = numpy.bincount(t,w)&#xA;m[x,y] = dc&#xA;wm[x,y] = wc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;http://code.google.com/p/agpy/source/browse/trunk/AG_image_tools/drizzle.py&quot; rel=&quot;nofollow&quot;&gt;a function implementing this&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would still be useful to know of other implementations of this algorithm.  Or perhaps ways to compute the &lt;code&gt;t&lt;/code&gt; mapping and different weighting schemes - I don't discuss that at all above, but I think in the context in which the term was coined (Hubble imaging) there are complexities involved in determining both variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2: Corollary - what if I want to median-drizzle?  i.e., instead of averaging for the final map, median?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(this is not valid code, but vaguely pseudo-code... you can't have 2-dimensional lists in python, though nested lists are OK)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for jj,(xx,yy) in enumerate(xinds,yinds):&#xA;    m[xx,yy].append((d*w)[jj])&#xA;for jj,(xx,yy) in enumerate(xinds,yinds):&#xA;    m[xx,yy] = median(m[xx,yy])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26" LastEditorUserId="26" LastEditDate="2011-12-10T17:06:26.097" LastActivityDate="2011-12-10T17:06:26.097" Title="An efficient 'drizzle' algorithm?" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="325" PostTypeId="2" ParentId="324" CreationDate="2011-12-09T23:39:10.867" Score="5" Body="&lt;p&gt;This algorithm is easy to write in a for-loop fashion and hard to express as a series of vector operations, which makes is a perfect candidate for writing it in a lower-level language (like C) and just linking it to Python (for instance with Cython). Unless there is a drizzle lib for Python, this will be both fastest and easiest solution.&lt;/p&gt;&#xA;" OwnerUserId="102" LastActivityDate="2011-12-09T23:39:10.867" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="326" PostTypeId="1" AcceptedAnswerId="487" CreationDate="2011-12-10T03:17:36.290" Score="8" ViewCount="390" Body="&lt;p&gt;I use an example from finite element theory, but anybody who maintains a large datastructure and successively extends it will find something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have an unstructured mesh of points and triangles, where the points are given by coordinates (say $x$ and $y$) and the triangles each consist of three point indices (say $i$,$j$ and $k$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As common in FEM, the mesh will be successively refined. If we resort to global regular refinement, the number of triangles will grow by factor $4$ with each iteration of the refinement. Depending on how this is done, the memory layout will develop differently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say the mesh occopies memory cells 1 to 300, anything beyond there being free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example 1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We allocate the space for the new mesh, cells 301 to 1501, fill it up with the data of the refined mesh, and forget the old one. The next refined mesh will be placed in cells 1501 to 6300, the next one in 6301 to 21500, and so on. The location of the current mesh will move in memory &quot;to the right&quot;, while a huge patch will not be used. We may run out of memory prematurely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might observe in the above example, that this will only hinder us for one refinestep, because even without that fragmentation we would run out of total memory one refinement later. As the vertex array is taken into account, too, the problem can become more severe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this be circumvented?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example 2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Realloc the triangle array to cells 1..1200. Create the new mesh in cells 1201 to 2400. Copy the content of that working copy to cells 1..1200, and forget the working copy. Repeat similarly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, still we run out of memory prematurely, because we need a working copy. How about this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example 3:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Realloc the triangle array to cells 1..1500. Copy old mesh to 1201 .. 1500. Create new mesh in cells 1..1200. Then forget the copy of the old mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case here is artificial, because one would not use global mesh refinement on these scales. If the growth is much smaller, memory realignment is possible to avoid fragmentation. However, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Does memory fragmentation ever become critical in practical scientific computing/high-performance computing?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If at all, how do you avoid it? Maybe my machine model is even wrong, and the OS by some heavy magic does realign the memory tacitly, or manages fragmented blocks on the heap.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;More specific, how does it impact grid management?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="238" LastEditorUserId="21" LastEditDate="2011-12-10T17:17:25.840" LastActivityDate="2012-10-05T01:34:27.830" Title="(How) do you take into account memory fragmentation?" Tags="|performance|hpc|memory-management|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="327" PostTypeId="1" AcceptedAnswerId="343" CreationDate="2011-12-10T05:13:08.473" Score="4" ViewCount="247" Body="&lt;p&gt;I want to calculate &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\binom{100}{i}a^i(1-a)^{(100-i)}$ for different $i$ with $a=0.001$ using &#xA;GMP-GNU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this be done? &lt;/p&gt;&#xA;" OwnerUserId="277" LastActivityDate="2011-12-22T23:36:10.043" Title="Evaluating $\binom{100}{i}a^i(1-a)^{(100-i)}$ in GMP-GNU" Tags="|software|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="328" PostTypeId="2" ParentId="326" CreationDate="2011-12-10T05:25:18.710" Score="4" Body="&lt;p&gt;1) No. Solver memory far far far outweighs mesh memory. Even if you run the lightest, explicit solver, the mesh is at most 25% of the memory of the simulation, and much more likely &amp;lt; 10%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Break down your allocation and use memory pooling. You do not need to allocate a contiguous chunk for the entire mesh since you usually only need to iterate over local pieces. Introducing one level of indirection does not impact the performance in a meaningful way.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-10T05:25:18.710" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="329" PostTypeId="2" ParentId="206" CreationDate="2011-12-10T09:27:24.070" Score="32" Body="&lt;p&gt;Ever since I read about Test-Driven Development in &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0735619670&quot;&gt;Code Complete, 2nd edition&lt;/a&gt;, I've used a &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks&quot;&gt;unit testing framework&lt;/a&gt; as part of my development strategy, and it's dramatically increased my productivity by reducing the amount of time I spent debugging because the various tests I write are diagnostic. As a side benefit, I'm much more confident in my scientific results, and have used my unit tests on a number of occasions to defend my results. If there is an error in a unit test, I can usually figure out why pretty quickly. If my application crashes and all of my unit tests pass, I do a code coverage analysis to see what parts of my code aren't exercised, as well as step through the code with a debugger to pinpoint the source of the error. Then I write a new test to make sure that the bug stays fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of the tests I write aren't pure unit tests. Strictly defined, &lt;b&gt;unit tests&lt;/b&gt; are supposed to exercise the functionality of one function. When I can test easily a single function using mock data, I do that. Other times, I can't easily mock the data I need to write a test that exercises the functionality of a given function, so I'll test that function along with others in an integration test. &lt;b&gt;Integration tests&lt;/b&gt; test the behavior of multiple functions at once. As Matt points out, scientific codes are often a constellation of interlocking functions, but often times, certain functions are called in sequence, and unit tests can be written to test the output at intermediate steps. For example, if my production code calls five functions in sequence, I'll write five tests. The first test will call the first function only (so it's a unit test). Then second test will call the first and second functions, the third test will call the first three functions, and so on. Even if I could write unit tests for every single function in my code, I would write integration tests anyway, because bugs can arise when various modular pieces of a program are combined. Finally, after writing all of the unit tests and integration tests I think I need, I'll wrap my case studies in unit tests and use them for regression testing, because I want my results to be repeatable. If they're not repeatable, and I get different results, I want to know why. The failure of a regression test may not be a real issue, but it will force me to figure out if the new results are at least as trustworthy as the old results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also worthwhile along with unit testing are static code analysis, memory debuggers, and compiling with compiler warning flags to catch simple errors and unused code.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="149" LastEditDate="2012-07-12T09:41:33.770" LastActivityDate="2012-07-12T09:41:33.770" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="330" PostTypeId="1" AcceptedAnswerId="333" CreationDate="2011-12-10T10:37:43.847" Score="48" ViewCount="2646" Body="&lt;p&gt;Many times, when I've inherited or encountered scientific code written by other people (or occasionally, even my own work), I've noticed that documentation is either sparse or nonexistent. If I'm lucky, I see informative comments. If I'm very lucky, there's even Doxygen comments and a Doxyfile so that I have function interfaces and some formatted HTML to consult. If I'm extremely lucky, there's a PDF manual and examples in addition to the Doxygen and source file comments, and I'm ecstatic, because it makes my life much, much easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What information and tools are useful in documenting source code? For that matter, what information and tools are useful to document the data and results that accompany that source code, in the case of scientific software?&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="127" LastEditDate="2011-12-10T12:20:54.060" LastActivityDate="2019-06-06T14:27:15.510" Title="What are good ways to document scientific software?" Tags="|software|documentation|" AnswerCount="8" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="331" PostTypeId="2" ParentId="330" CreationDate="2011-12-10T12:26:14.450" Score="5" Body="&lt;p&gt;To address your question on how to document data and results, I'd recommend something like &lt;a href=&quot;http://docs.python.org/library/doctest.html#simple-usage-checking-examples-in-a-text-file&quot; rel=&quot;noreferrer&quot;&gt;Python's doctest module&lt;/a&gt;. This allows you to write tutorials or tests in a way that can be automatically validated.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2011-12-10T12:26:14.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="332" PostTypeId="1" AcceptedAnswerId="3294" CreationDate="2011-12-10T13:47:19.753" Score="8" ViewCount="242" Body="&lt;p&gt;My question is probably going to be too general to answer it with a couple words. Could you please suggest a good reading in that case. Projection methods are used to reduce size of the solution space for the problems. And there are at least two very interesting applications (from my point of view). The first is the solving of continuum mechanics problems (Finite Element,Ritz methods) and the second is solving of systems of linear equations (Krylov subspace methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The question is the following:&lt;/strong&gt; Is there a theory or some part of analysis that studies projection methods in all their applications? If so, can other methods, like finite volume methods, be built from this starting point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I studied FEA at university but at the moment, all the discrete approximations are like a set of isolated &quot;tools&quot; that I can use in some particular case.&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="188" LastEditorUserId="127" LastEditDate="2011-12-12T11:40:19.627" LastActivityDate="2022-07-25T13:28:38.620" Title="Is there one general approach to build a projection methods for different problems?" Tags="|linear-algebra|finite-element|discretization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="333" PostTypeId="2" ParentId="330" CreationDate="2011-12-10T15:06:04.280" Score="46" Body="&lt;p&gt;I think that documentation for scientific software can be divided into three categories, all of which are necessary for full understanding. The easiest and most common is individual method documentation. There are many systems for this. You mention &lt;a href=&quot;http://www.doxygen.nl/&quot; rel=&quot;noreferrer&quot;&gt;doxygen&lt;/a&gt;, Python has &lt;a href=&quot;http://docs.python.org/library/pydoc.html&quot; rel=&quot;noreferrer&quot;&gt;pydoc&lt;/a&gt;, and in PETSc we have our own package &lt;em&gt;sowing&lt;/em&gt; which generates the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html&quot; rel=&quot;noreferrer&quot;&gt;following&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for any piece of software which goes beyond a simple utility, you need a manual. This provides a high-level view of the purpose of the package, and how its different functionalities integrate to achieve this purpose. It helps a new user structure their code, often through the use of examples. In PETSc, we just use LaTeX for the manual, but the &lt;a href=&quot;http://numerics.kaust.edu.sa/pyclaw/&quot; rel=&quot;noreferrer&quot;&gt;PyClaw&lt;/a&gt; package uses the &lt;a href=&quot;http://sphinx.pocoo.org/&quot; rel=&quot;noreferrer&quot;&gt;Sphinx&lt;/a&gt; framework which I am very impressed with. One thing that we have implemented in the sowing package that I find very useful is the link between example code and function documentation. For example, &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/src/snes/examples/tutorials/ex5.c.html&quot; rel=&quot;noreferrer&quot;&gt;this example&lt;/a&gt; solves the Bratu equation. Notice how you can follow the links for any custom type or function call and get to the low-level documentation, and how those pages link back to examples using them. This is how I learn about new functionality which other people in the project contribute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A frequently overlooked part of documentation, I think, is developer documentation. It is not uncommon to publish a coding-style document, and instructions for interacting with the repository. However, it is very rare to explain the design decisions made before implementation. These decisions always involve tradeoffs, and the situation with respect to hardware and algorithms will necessarily change over time. Without a discussion of the tradeoffs reviewed and rationale for particular design decisions, later programmers are left to recreate the entire process on their own. I think this is a major impediment to successful maintenance and improvement of old codes when the original developers are no longer in charge.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="29137" LastEditDate="2018-12-16T14:14:16.673" LastActivityDate="2018-12-16T14:14:16.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="334" PostTypeId="2" ParentId="330" CreationDate="2011-12-10T15:30:18.227" Score="9" Body="&lt;p&gt;This is a good question. To a first approximation, the code should attempt to be self documenting. So, for example, if the software is command line, you should be able to do &lt;code&gt;executable --help&lt;/code&gt; or &lt;code&gt;executable -h&lt;/code&gt; or even &lt;code&gt;executable&lt;/code&gt; (if the executable does nothing with no arguments), and have a brief usage message return. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, I think that it is unrealistic to expect scientific developers to spend a great deal of time documenting their software, so I suggest keeping it simple. A short text manual with the basic methods and options and annotated &lt;em&gt;working and tested&lt;/em&gt; examples of usage, graduated from simple to more complex (usage examples are very important and frequently neglected) is considerably better than nothing and much more than most scientific software offers. I'd also like to add a pet peeve about usage examples. Simple means simple. That means if you are trying to illustrate a method, you don't add in ten extraneous concepts or methods to confuse the reader. Keep it simple and annotate so the reader knows what the example is supposed to be doing. I'd also suggest tying the manual usage examples into a test suite somehow so they continue working when the code is changed. I don't actually know how to do this in a nice way, so please feel free to educate me. If the developers want to get more fancy, sure they can use nice markup languages and so forth, add man pages and so on. If the software is mathematical, plain text will of course not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, the authors should consider writing a paper for publication if appropriate. This would usually address design decisions and give a more high level perspective on the software than a manual does, or can be expected to do. This would address the documentation of design decisions that @Matt was talking about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the most important documentation of all is the code itself, which should be commented as necessary. This assumes the code is free software. If it isn't, then it is substantially less useful as scientific software (do you really want to use a black box where you can't see how the methods are implemented?) and I for one would not use it.&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="38" LastEditDate="2011-12-10T15:41:31.813" LastActivityDate="2011-12-10T15:41:31.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="335" PostTypeId="2" ParentId="330" CreationDate="2011-12-10T16:22:37.273" Score="20" Body="&lt;h3&gt;In-code documentation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The most important thing is to use the documentation facilities in your chosen development environment, so that means &lt;a href=&quot;http://docs.python.org/library/pydoc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pydoc&lt;/a&gt; for python, &lt;a href=&quot;http://en.wikipedia.org/wiki/Javadoc&quot; rel=&quot;nofollow noreferrer&quot;&gt;javadoc&lt;/a&gt; in java or &lt;a href=&quot;http://msdn.microsoft.com/en-us/library/b2s063f7%28v=VS.100%29.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;xml comments&lt;/a&gt; in C#. These make it easy to write the documentation at the same time as writing the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you rely on coming back and documenting things later, you may not get around to it, but if you do it as you are writing the code, then what needs to be documented will be fresh in your mind. C# even has the option to issue a compilation warning if the XML documentation is incomplete or inconsistent with the actual code.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Tests as documentation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Another important aspect is having good integration and unit tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often documentation concentrates on what classes and methods do in isolation, skipping over how they are used together to solve your problem. Tests often put these into context by showing how they interact with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, unit-tests often point out external dependencies explicitly through which things need to be &lt;a href=&quot;http://en.wikipedia.org/wiki/Mock_object&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mock&lt;/a&gt;ed out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also find that using &lt;a href=&quot;http://en.wikipedia.org/wiki/Test-driven_development&quot; rel=&quot;nofollow noreferrer&quot;&gt;Test-driven development&lt;/a&gt; I write software which is easier to use, because I'm using it right from the word go. With a good testing framework, making code easier to test and making it easy to use are often the same thing.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Higher level documentation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Finally there is what to do about system level and architectural documentation. Many would advocate writing such documentation in a wiki or using Word or other word processor, but for me the best place for such documentation is alongside the code, in a plain text format that is version control system friendly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just like with in-code documentation, if you store your higher level documentation in your code repository then you are more likely to keep it up to date. You also get the benefit that when you pull out version X.Y of the code, you also get version X.Y of the documentation. In addition, if you use a VCS friendly format, then it means that it is easy to branch, diff and merge your documentation, just like your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I quite like &lt;a href=&quot;http://docutils.sourceforge.net/rst.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;reStructuredText (rst)&lt;/a&gt;, as it is easy to produce both html pages and pdf documents from it using &lt;a href=&quot;http://www.sphinx-doc.org/en/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sphinx&lt;/a&gt;, and is much friendlier than &lt;a href=&quot;http://en.wikipedia.org/wiki/LaTeX&quot; rel=&quot;nofollow noreferrer&quot;&gt;LaTeX&lt;/a&gt;, yet can still include &lt;a href=&quot;http://docutils.sourceforge.net/docs/ref/rst/directives.html#math&quot; rel=&quot;nofollow noreferrer&quot;&gt;LaTeX math expressions&lt;/a&gt; when you need them.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2019-06-06T14:27:15.510" LastActivityDate="2019-06-06T14:27:15.510" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="336" PostTypeId="2" ParentId="159" CreationDate="2011-12-10T19:07:56.703" Score="5" Body="&lt;p&gt;I think one should use tested libraries as much as possible. Most people are not experts in numerical computing and probably will not be able to implement a solution as correctly and carefully as what's available in well-tested libraries. That said, however, it is sometimes the case that there are no available libraries that implement the combination of capabilities  needed in a given application. I've seen this happen in the technical area in which I work; existing codes didn't solve all the cases, and someone ended up implementing a solver from scratch that did.&lt;/p&gt;&#xA;" OwnerUserId="283" LastActivityDate="2011-12-10T19:07:56.703" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="337" PostTypeId="2" ParentId="206" CreationDate="2011-12-10T21:48:44.283" Score="6" Body="&lt;p&gt;The mantra is &quot;Untested code is broken code&quot; and that came from the boss.&#xA;Rather than repeat all the good reasons for doing unit testing, I want to just add a few specifics.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Memory usage should be tested. Every function which allocates memory should be tested making sure that the functions that store and retrieve data into that memory are doing the right thing. This is even more important in the GPU world.&lt;/li&gt;&#xA;&lt;li&gt;While briefly mentioned before, testing edge cases is extremely important. Think of these tests the same way you test the result of any calculation. Make sure the code behaves at the edges and fails gracefully (however you define that in your simulation) when input parameters or data falls outside acceptable boundaries. The thinking involved in writing this sort of test helps sharpen your work and may be one of the reasons that you rarely find someone who has written unit tests but does not find the process useful.&lt;/li&gt;&#xA;&lt;li&gt;Use a test framework (as mentioned by Geoff who provided a nice link). I have used the BOOST test framework in conjunction with the CTest system of CMake and can recommend it as an easy way to quickly write unit tests (as well as validation and regression tests).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="69" LastEditorUserId="20688" LastEditDate="2020-04-07T00:13:51.910" LastActivityDate="2020-04-07T00:13:51.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="338" PostTypeId="1" AcceptedAnswerId="339" CreationDate="2011-12-10T22:07:58.037" Score="7" ViewCount="8674" Body="&lt;p&gt;I'm looking for a decent implementation of Gillespie's Direct Method in Python, as if I code the algorithm myself I'm nigh positive I'll do it inefficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone have a favorite?&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="127" LastEditDate="2011-12-11T02:59:17.400" LastActivityDate="2019-01-10T21:23:21.670" Title="Python implementations of Gillespie's direct method" Tags="|algorithms|python|monte-carlo|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="339" PostTypeId="2" ParentId="338" CreationDate="2011-12-10T22:41:36.973" Score="7" Body="&lt;p&gt;(Disclaimer: I haven't used any of the packages or code below.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding any implementation of Gillespie's method in Python was a bit of a challenge; the most fruitful search terms seemed to be &quot;tau leap&quot; or &quot;kinetic Monte Carlo.&quot; &lt;a href=&quot;http://pyinsci.blogspot.com/2008/10/fast-gillespies-direct-algorithm-in.html&quot; rel=&quot;noreferrer&quot;&gt;This blog post&lt;/a&gt; implements Gillespie's algorithm, though it's not clear that it's efficient. One of the commenters mentions some other Gillespie/SSA algorithm implementations in Python that solve problems out of a textbook; the relevant problems are 6.3 through 6.6 on &lt;a href=&quot;http://wiki.deductivethinking.com/wiki/Python_Programs_for_Modelling_Infectious_Diseases_book&quot; rel=&quot;noreferrer&quot;&gt;this website&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most promising implementations in Python that I found were &lt;a href=&quot;http://stompy.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;StochPy&lt;/a&gt; and Python language bindings to &lt;a href=&quot;http://www.copasi.org/tiki-read_article.php?articleId=39&quot; rel=&quot;noreferrer&quot;&gt;COPASI&lt;/a&gt;. I also found what looked like promising C++ codes in &lt;a href=&quot;http://www.sandia.gov/~sjplimp/download.html#spparks&quot; rel=&quot;noreferrer&quot;&gt;SPPARKS&lt;/a&gt;, &lt;a href=&quot;http://www.sysbio.pl/stocks/&quot; rel=&quot;noreferrer&quot;&gt;STOCKS&lt;/a&gt;, and &lt;a href=&quot;http://sourceforge.net/projects/stochkit/&quot; rel=&quot;noreferrer&quot;&gt;StochKit&lt;/a&gt;; they're not Python, but maybe you can wrap around one of them in Python if it looks particularly promising to you.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2011-12-10T22:49:33.553" LastActivityDate="2011-12-10T22:49:33.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="342" PostTypeId="2" ParentId="83" CreationDate="2011-12-11T05:58:39.350" Score="43" Body="&lt;p&gt;I work in a lab that does global optimization of mixed-integer and non-convex problems. My experience with open source optimization solvers has been that the better ones are typically written in a compiled language, and they fare poorly compared to commercial optimization packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can formulate your problem as an explicit system of equations and need a free solver, your best bet is probably IPOPT, as Aron said. Other free solvers can be found on the &lt;a href=&quot;http://www.coin-or.org/&quot;&gt;COIN-OR&lt;/a&gt; web site. To my knowledge, the nonlinear solvers do not have Python bindings provided by the developers; any bindings you find would be third-party. In order to obtain good solutions, you would also have to wrap any nonlinear, convex solver you found in appropriate stochastic global optimization heuristics, or in a deterministic global optimization algorithm such as branch-and-bound. Alternatively, you could use Bonmin or Couenne, both of which are deterministic non-convex optimization solvers that perform serviceably well compared to the state-of-the-art solver, &lt;a href=&quot;http://archimedes.cheme.cmu.edu/baron/baron.html&quot;&gt;BARON&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can purchase a commercial optimization solver, you might consider looking at the &lt;a href=&quot;http://gams.com/&quot;&gt;GAMS&lt;/a&gt; modeling language, which includes several nonlinear optimization solvers. Of particular mention are the interfaces to the solvers CONOPT, SNOPT, and BARON. (CONOPT and SNOPT are convex solvers.) A kludgey solution that I've used in the past is to use the Fortran (or Matlab) language bindings to GAMS to write a GAMS file and call GAMS from Fortran (or Matlab) to calculate the solution of an optimization problem. GAMS has Python language bindings, and a very responsive support staff willing to help out if there's any trouble. (Disclaimer: I have no affiliation with GAMS, but my lab does own a GAMS license.) The commercial solvers should be no worse than &lt;code&gt;fmincon&lt;/code&gt;; in fact, I'd be surprised if they weren't a lot better. If your problems are sufficiently small in size, then you may not even need to purchase a GAMS license and licenses to solvers, because an evaluation copy of GAMS may be downloaded from their web site. Otherwise, you would probably want to decide which solvers to purchase in conjunction with a GAMS license. It's worth noting that BARON requires a mixed-integer linear programming solver, and that licenses for the two best mixed-integer linear programming solvers CPLEX and GUROBI are free for academics, so you might be able to get away with just purchasing the GAMS interfaces rather than the interfaces and the solver licenses, which can save you quite a bit of money.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This point bears repeating: for any of the deterministic non-convex optimization solvers I've mentioned above, you need to be able to formulate the model as an explicit set of equations. Otherwise, the non-convex optimization algorithms won't work, because all of them rely on symbolic analysis to construct convex relaxations for branch-and-bound-like algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: One thought that hadn't occurred to me at first was that you could also call the Toolkit for Advanced Optimization (&lt;a href=&quot;http://www.mcs.anl.gov/research/projects/tao/index.html&quot;&gt;TAO&lt;/a&gt;) and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot;&gt;PETSc&lt;/a&gt; using &lt;a href=&quot;http://code.google.com/p/tao4py/&quot;&gt;tao4py&lt;/a&gt; and &lt;a href=&quot;http://code.google.com/p/petsc4py/&quot;&gt;petsc4py&lt;/a&gt;, which would have the potential added benefit of easier parallelization, and leveraging familiarity with PETSc and the &lt;a href=&quot;http://acts.nersc.gov/tools.html&quot;&gt;ACTS&lt;/a&gt; tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE #2: Based on the additional information you mentioned, sequential quadratic programming (SQP) methods are going to be your best bet. SQP methods are generally considered more robust than interior point methods, but have the drawback of requiring dense linear solves. Since you care more about robustness than speed, SQP is going to be your best bet. I can't find a good SQP solver out there written in Python (and apparently, neither could Sven Leyffer at Argonne in this &lt;a href=&quot;http://wiki.mcs.anl.gov/leyffer/images/7/75/NLP-Solvers.pdf&quot;&gt;technical report&lt;/a&gt;). I'm guessing that the algorithms implemented in packages like SciPy and OpenOpt have the basic skeleton of some SQP algorithms implemented, but without the specialized heuristics that more advanced codes use to overcome convergence issues. You could try &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt&quot;&gt;NLopt&lt;/a&gt;, written by Steven Johnson at MIT. I don't have high hopes for it because it doesn't have any reputation that I know of, but Steven Johnson is a brilliant guy who writes good software (after all, he did co-write FFTW). It does implement a version of SQP; if it's good software, let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping that TAO would have something in the way of a constrained optimization solver, but it doesn't. You could certainly use what they have to build one up; they have a lot of the components there. As you pointed out, though, it'd be much more work for you to do that, and if you're going to that sort of trouble, you might as well be a TAO developer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that additional information, you are more likely to get better results calling GAMS from Python (if that's an option at all), or trying to patch up the IPOPT Python interface. Since IPOPT uses an interior point method, it won't be as robust, but maybe Andreas' implementation of an interior point method is considerably better than Matlab's implementation of SQP, in which case, you may not be sacrificing robustness at all. You'd have to run some case studies to know for sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're already aware of the trick to reformulate the rational inequality constraints as polynomial inequality constraints (it's in your book); the reason this would help BARON and some other nonconvex solvers is that it can use term analysis to generate additional valid inequalities that it can use as cuts to improve and speed up solver convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Excluding the GAMS Python bindings and the Python interface to IPOPT, the answer is no, there aren't any high quality nonlinear programming solvers for Python yet. Maybe @Dominique will change that with NLPy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE #3: More wild stabs at finding a Python-based solver yielded &lt;a href=&quot;http://sourceforge.net/apps/mediawiki/pagmo/index.php?title=Main_Page&quot;&gt;PyGMO&lt;/a&gt;, which is a set of Python bindings to PaGMO, a C++ based global multiobjective optimization solver. Although it was created for multiobjective optimization, it can also be used to single objective nonlinear programming, and has Python interfaces to IPOPT and SNOPT, among other solvers. It was developed within the &lt;a href=&quot;http://www.esa.int/esaCP/index.html&quot;&gt;European Space Agency&lt;/a&gt;, so hopefully there's a community behind it. It was also released relatively recently (November 24, 2011).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2011-12-15T23:42:05.557" LastActivityDate="2011-12-15T23:42:05.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="343" PostTypeId="2" ParentId="327" CreationDate="2011-12-11T07:10:48.380" Score="3" Body="&lt;p&gt;I hope this is what you were looking for.  Here is a program to print out '1':&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/* gcc -lgmp foo.c -o foo */&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;gmp.h&amp;gt;&#xA;&#xA;int main(int argc, char **argv) {&#xA;  int t, i, n=100;&#xA;  mpz_t fac_num, fac_den;&#xA;  mpq_t a, q, tq, sum;&#xA;&#xA;  mpz_init(fac_num); mpz_init(fac_den); &#xA;  mpq_init(a); mpq_init(q); mpq_init(tq); &#xA;  mpq_init(sum);&#xA;&#xA;  mpq_set_ui(sum, 0, 1);&#xA;&#xA;  for (i=0; i&amp;lt;=n; i++) {&#xA;&#xA;    mpq_set_ui(a, 1, 1000);&#xA;    mpq_set_ui(q, 999, 1000);&#xA;    mpq_set_ui(tq, 1, 1);&#xA;&#xA;    mpz_set_ui(fac_num, 1);&#xA;    mpz_set_ui(fac_den, 1);&#xA;    for (t=0; t&amp;lt;i; t++) {&#xA;      mpz_mul_ui(fac_num, fac_num, n-t);&#xA;      mpz_mul_ui(fac_den, fac_den, t+1);&#xA;      mpq_mul(tq, tq, a);&#xA;    }&#xA;&#xA;    for (; t&amp;lt;n; t++) &#xA;      mpq_mul(tq, tq, q);&#xA;&#xA;    mpq_set_ui(q, 1, 1);&#xA;    mpq_set_num(q, fac_num);&#xA;    mpq_set_den(q, fac_den);&#xA;    mpq_mul(q, q, tq);&#xA;&#xA;    mpq_add(sum, sum, q);&#xA;&#xA;  }&#xA;&#xA;  mpq_canonicalize(sum);&#xA;&#xA;  gmp_printf(&quot;%Qd\n&quot;, sum);&#xA;&#xA;  mpz_clear(fac_num); mpz_clear(fac_den); &#xA;  mpq_clear(a); mpq_clear(q); mpq_clear(tq);&#xA;  mpq_clear(sum);&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There might be more efficient ways to do the factorial and you can do exponentiation by squaring, but I leave those for the reader.&lt;/p&gt;&#xA;" OwnerUserId="290" LastActivityDate="2011-12-11T07:10:48.380" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="345" PostTypeId="2" ParentId="296" CreationDate="2011-12-12T00:39:58.527" Score="5" Body="&lt;p&gt;For SPD problems additive methods are better for MG smoothing for several reasons as mentioned already and a few more: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@Article{Adams-02, &#xA;author = {Adams, M.~F. and Brezina, M. and Hu, J. J. and Tuminaro, R. S.}, &#xA;title = {Parallel multigrid smoothing: polynomial versus {G}auss-{S}eidel}, &#xA;journal = {J. Comp. Phys.}, &#xA;year = {2003}, &#xA;volume = {188}, &#xA;number = {2}, &#xA;pages = {593-610} }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Multiplicative methods do however have the correct spectral properties straight-out-of-the-box for an MG smoother, that is, they do not need damping. This can be a big win for hyperbolic problems where polynomial smoothing is not very nice.&lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="123" LastEditDate="2011-12-12T06:09:03.743" LastActivityDate="2011-12-12T06:09:03.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="346" PostTypeId="1" AcceptedAnswerId="349" CreationDate="2011-12-12T01:24:00.923" Score="36" ViewCount="2943" Body="&lt;p&gt;I've heard people say that plots produced by ORIGIN tend to look polished and &quot;professional,&quot; whereas plots produced by Mathematica do not. However, most plot-creation programs are quite configurable and it stands to reason that with the right settings for things like tick location and labeling, font and color choices, label alignment, and so on, I should be able to make a figure with Mathematica/matplotlib/Gnuplot/etc. that looks as good as those that come from ORIGIN. But what does it mean for a figure to be &quot;professional&quot; in this context?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if my goal is to create the best looking figures possible for inclusion in a scientific paper, what design choices are generally recommended towards that goal? Obviously one has to choose the appropriate &lt;em&gt;kind&lt;/em&gt; of plot, e.g. bar graph vs. scatter plot, and linear vs. logarithmic scale, but those are choices that we always think about regardless of which plotting program we are using. I'm more interested in the things we don't normally think about, which are normally set according to some plotting program's defaults, but which could be changed to improve the look of the plot.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2014-07-15T15:06:01.350" Title="What attributes make a figure &quot;professional-quality&quot;?" Tags="|visualization|" AnswerCount="6" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="347" PostTypeId="2" ParentId="346" CreationDate="2011-12-12T01:35:09.383" Score="8" Body="&lt;p&gt;The best figures I have been able to make personally have been with the TeX package &lt;a href=&quot;http://en.wikipedia.org/wiki/PGF/TikZ&quot; rel=&quot;nofollow noreferrer&quot;&gt;PGF/TikZ&lt;/a&gt;. If you use LaTex, as many in the hard sciences do, you have probably already heard of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also appears to be the leader in LaTex graphics packages. A sizeable proportion on the questions on &lt;a href=&quot;https://tex.stackexchange.com/&quot;&gt;the TeX StackExchange site&lt;/a&gt; are about PGF/TikZ. I'm not sure why the results are so good, but certainly one advantage PGF/TikZ has over other packages when using LaTeX is that it simply integrates better with the text. For one thing, the fonts in the figure will be the same as in the text.&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="-1" LastEditDate="2017-04-13T12:36:11.990" LastActivityDate="2014-07-15T15:06:01.350" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="348" PostTypeId="2" ParentId="346" CreationDate="2011-12-12T02:28:12.943" Score="14" Body="&lt;p&gt;If we're talking about data figures I'd go to the sources: Edward Tufte's &lt;a href=&quot;http://www.edwardtufte.com/tufte/books_vdqi&quot;&gt;&lt;em&gt;The Visual Display of Qualitative Information&lt;/em&gt;&lt;/a&gt;&#xA;and &lt;a href=&quot;http://www.edwardtufte.com/tufte/books_be&quot;&gt;&lt;em&gt;Beautiful Evidence&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mr. Tufte of course goes into some details, but the principle that stands out for me is not spending ink on frames and decoration, but instead making as much of your ink as possible carry information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Amended per Mark's request:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some major points from &lt;em&gt;The Visual Display of Qualitative Information&lt;/em&gt; are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;show the data in a way that does not distort or obfuscate what it has to say&lt;/li&gt;&#xA;&lt;li&gt;arrange displays to allow for comparisons between different data at different levels&lt;/li&gt;&#xA;&lt;li&gt;integrate the graphic aspects with the statistical and verbal descriptions&lt;/li&gt;&#xA;&lt;li&gt;maximize data to ink ratio by removing elements that do not serve any purpose (or are made redundant by other elements) and use what elements there are to convey additional information (axes that are variants on a boxplot, for example)&lt;/li&gt;&#xA;&lt;li&gt;small multiples can be used to arrange higher dimensional data sets to allow comparison along these additional dimensions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Beautiful Evidence&lt;/em&gt; is a wider ranging book in its scope.  I'll just reproduce the chapter titles:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mapped Pictures: Images as Evidence and Explanation&lt;/li&gt;&#xA;&lt;li&gt;Sparklines: Intense, Simple, Word-Sized Graphics&lt;/li&gt;&#xA;&lt;li&gt;Links and Causal Arrows: Ambiguity in Action&lt;/li&gt;&#xA;&lt;li&gt;Words, Numbers, Images -- Together&lt;/li&gt;&#xA;&lt;li&gt;The Fundamental Principles of Analytical Design&lt;/li&gt;&#xA;&lt;li&gt;Corruption in Evidence Presentations: Effects Without Causes, Cherry Picking, Overreaching, Chartjunk, and the Rage to Conclude&lt;/li&gt;&#xA;&lt;li&gt;The Cognitive Style of PowerPoint: Pitching Out Corrupts Within&lt;/li&gt;&#xA;&lt;li&gt;Sculptural Pedestals: Meaning, Practice, Depedestalization&lt;/li&gt;&#xA;&lt;li&gt;Landscape Sculptures&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One of the interesting observations in &lt;em&gt;Beautiful Evidence&lt;/em&gt; is that we generally use &lt;em&gt;high density&lt;/em&gt; output devices (a 300 DPI printer is a low density device these days) for printed material, but often draw our figures for screen or line printer, which wastes an enormous potential for conveying information.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="38" LastEditDate="2011-12-14T14:42:13.140" LastActivityDate="2011-12-14T14:42:13.140" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="349" PostTypeId="2" ParentId="346" CreationDate="2011-12-12T02:52:31.040" Score="13" Body="&lt;p&gt;IMO, what makes a figure &quot;professional quality&quot; is defined by the journal/publisher rules. Which actually translates to &quot;publication quality&quot; which is relative depending on where you publish. Some universal rules seem to stand out - invariant of the plotting software being used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) A figure should contain as few elements possible required to convey the information/idea/argument. A figure should be easily read/understood within few seconds - if it takes you longer to understand what is going on on the figure, it might be too much information. This is sometimes hard to check as you are familiar with your own plot/data - no matter how over-encumbered it is - showing it to few colleagues to see if they can read it easily helps. (not to be mistaken with understanding the physical meaning behind the figure - this usually takes quite some time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If you have to use colors, it is best to limit yourself to few, ideally on the opposite side of the color wheel. E.g. blue and red is better than blue and green. A figure may have many shades - but it is best to have few main colors. I often like to use blue (low values) and red (high values) with a white transition in-between. Always have in mind color-blind readers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Tick marks, contour labels etc. all should be easily readable without a magnifying glass - so, similar font size as the journal body text. You can check if everything is readable by printing out a hardcopy with figure widths of 3 and 6 inches (these are common figure sizes in scientific journals).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Finally, make sure every single element of the figure has its purpose. If there is anything that is not conveying useful information - throw it out. It will help the readability of the figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the point where you are comfortable with customizing all the little elements that make a figure - tick marks, labels etc., it does not really matter which tool you use as long as you are able to produce a clean eps.&lt;/p&gt;&#xA;" OwnerUserId="264" LastEditorUserId="264" LastEditDate="2011-12-12T03:08:43.307" LastActivityDate="2011-12-12T03:08:43.307" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="350" PostTypeId="1" AcceptedAnswerId="356" CreationDate="2011-12-12T03:55:38.597" Score="8" ViewCount="889" Body="&lt;p&gt;I am solving a system using singular value decomposition. The singular values (before scaling) are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.82277e+29&#xA;1.95011e+27&#xA;1.15033e+23&#xA;1.45291e+21&#xA;4.79336e+17&#xA;7.48116e+15&#xA;8.31087e+12&#xA;1.71838e+11&#xA;5.63232e+08&#xA;2.17863e+08&#xA;9.02783e+07&#xA;1.72345e+07&#xA;1.73889e+05&#xA;8.09382e+02&#xA;2.16644e+00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have found that accepting all the singular values and their associated contribution to my solution vector yields poor results. I scale them all by the largest number, yielding singular values of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.0&#xA;1.06986e-02&#xA;6.31091e-07&#xA;7.97089e-09&#xA;2.62971e-12&#xA;4.10428e-14&#xA;4.55948e-17&#xA;9.42732e-19&#xA;3.08998e-21&#xA;1.19523e-21&#xA;4.95281e-22&#xA;9.45510e-23&#xA;9.53980e-25&#xA;4.44040e-27&#xA;1.18854e-29&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The best solution only starts to become bad if I include the last two, and only become good around the $10^{-19}$ term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a sharp drop in accuracy when I include the last 2 terms. Why is that? What are the criteria for including/not including singular values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My matrix equation comes from a linear least squares fitting where I am using a polynomial basis set to fit some noisy data I created. I am solving the standard overdetermined system ($m \times n$ matrix where $m \gg n$) by multiplying each side ($A\cdot X = B$) by the transpose of $A$ ($A^\top A X = A^\top X$) and performing SVD on that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am judging the answers to my solutions by how well it approximates my noisy data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also noticed that, even on the 'good' fits, I am not fitting very well near zero (my data ranges from $-10$ to $10$). Why is that?&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="127" LastEditDate="2013-05-27T18:07:18.757" LastActivityDate="2013-05-27T18:07:18.757" Title="What should be the criteria for accepting/rejecting singular values?" Tags="|linear-algebra|regression|svd|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="351" PostTypeId="1" AcceptedAnswerId="353" CreationDate="2011-12-12T04:37:23.220" Score="170" ViewCount="140074" Body="&lt;p&gt;Does anyone have recommendations on a usable, fast C++ matrix library?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by usable is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Matrix objects have an intuitive interface (ex.: I can use rows and columns while indexing)&lt;/li&gt;&#xA;&lt;li&gt;I can do anything with the matrix class that I can do with LAPACK and BLAS&lt;/li&gt;&#xA;&lt;li&gt;Easy to learn and use API&lt;/li&gt;&#xA;&lt;li&gt;Relatively painless to install in Linux (I use Ubuntu 11.04 right now)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To me, usability is more important than speed or memory usage right now, to avoid premature optimization. In writing the code, I could always use 1-D arrays (or STL vectors) and proper index or pointer arithmetic to emulate a matrix, but I'd prefer not to in order to avoid bugs. I'd also like to focus my mental effort on the actual problem I'm trying to solve and program into the problem domain, rather than use part of my finite attention to remember all of the little programming tricks I used to emulate matrices as arrays, and remember LAPACK commands, et cetera. Plus, the less code I have to write, and the more standardized it is, the better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dense versus sparse doesn't matter yet; some of the matrices I am dealing with will be sparse, but not all of them. However, if a particular package handles dense or sparse matrices well, it is worth mentioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Templating doesn't matter much to me either, since I'll be working with standard numeric types and don't need to store anything other than doubles, floats, or ints. It's nice, but not necessary for what I'd like to do.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2011-12-19T05:36:11.993" LastActivityDate="2019-01-06T13:29:39.570" Title="Recommendations for a usable, fast C++ matrix library?" Tags="|linear-algebra|software|implicit-methods|" AnswerCount="8" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="352" PostTypeId="2" ParentId="346" CreationDate="2011-12-12T04:58:29.977" Score="5" Body="&lt;p&gt;I've had reasonable success using the Mathematica package &lt;a href=&quot;http://scidraw.nd.edu/levelscheme/&quot;&gt;LevelScheme&lt;/a&gt;. It's execution model differs slightly from traditional Mathematica programming, so there is learning curve associated with its use.  But, it is capable of providing fine control to plot generation which is difficult in plain Mathematica.  Also, as a side package, there is a package for generating custom tick marks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Once the version with Mathematica v.8 support comes out it is going to be renamed as SciDraw.)&lt;/p&gt;&#xA;" OwnerUserId="85" LastActivityDate="2011-12-12T04:58:29.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="353" PostTypeId="2" ParentId="351" CreationDate="2011-12-12T05:21:13.363" Score="154" Body="&lt;p&gt;I've gathered the following from online research so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used &lt;a href=&quot;http://arma.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;Armadillo&lt;/a&gt; a little bit, and found the interface to be intuitive enough, and it was easy to locate binary packages for Ubuntu (and I'm assuming other Linux distros). I haven't compiled it from source, but my hope is that it wouldn't be too difficult. It meets most of my design criteria, and uses dense linear algebra. It can call LAPACK or MKL routines. There generally is no need to compile Armadillo, it is a purely template-based library: You just include the header and link to BLAS/LAPACK or MKL etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard good things about &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;noreferrer&quot;&gt;Eigen&lt;/a&gt;, but haven't used it. It &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Benchmark&quot; rel=&quot;noreferrer&quot;&gt;claims to be fast&lt;/a&gt;, uses templating, and supports dense linear algebra. It doesn't have LAPACK or BLAS as a dependency, but appears to be able to do everything that LAPACK can do (plus some things LAPACK can't). A lot of projects use Eigen, which is promising. It has a binary package for Ubuntu, but as a header-only library it's trivial to use elsewhere too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.simunova.com/de/mtl4/&quot; rel=&quot;noreferrer&quot;&gt;Matrix Template Library&lt;/a&gt; version 4 also looks promising, and uses templating. It supports both dense and sparse linear algebra, and can call &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/umfpack/&quot; rel=&quot;noreferrer&quot;&gt;UMFPACK&lt;/a&gt; as a sparse solver. The features are somewhat unclear from their website. It has a binary package for Ubuntu, downloadable from their web site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;noreferrer&quot;&gt;PETSc&lt;/a&gt;, written by a team at Argonne National Laboratory, has access to sparse and dense linear solvers, so I'm presuming that it can function as a matrix library. It's written in C, but has C++ bindings, I think (and even if it didn't, calling C from C++ is no problem). The documentation is incredibly thorough. The package is a bit overkill for what I want to do now (matrix multiplication and indexing to set up mixed-integer linear programs), but could be useful as a matrix format for me in the future, or for other people who have different needs than I do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://trilinos.sandia.gov/index.html&quot; rel=&quot;noreferrer&quot;&gt;Trilinos&lt;/a&gt;, written by a team at Sandia National Laboratory, provides object-oriented C++ interfaces for dense and sparse matrices through its Epetra component, and templated interfaces for dense and sparse matrices through its Tpetra component. It also has components that provide linear solver and eigensolver functionality. The documentation does not seem to be as polished or prominent as PETSc; Trilinos seems like the Sandia analog of PETSc. PETSc can call some of the Trilinos solvers. Binaries for Trilinos are available for Linux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://sf.net/projects/blitz/&quot; rel=&quot;noreferrer&quot;&gt;Blitz&lt;/a&gt; is a C++ object-oriented library that has Linux binaries. It doesn't seem to be actively maintained (2012-06-29: a new version has just appeared yesterday!), although the mailing list is active, so there is some community that uses it. It doesn't appear to do much in the way of numerical linear algebra beyond BLAS, and looks like a dense matrix library. It uses templates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.boost.org/doc/libs/1_48_0/libs/numeric/ublas/doc/index.htm&quot; rel=&quot;noreferrer&quot;&gt;Boost::uBLAS&lt;/a&gt; is a C++ object-oriented library and part of the Boost project. It supports templating and dense numerical linear algebra. I've heard it's not particularly fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://math.nist.gov/tnt/history.html&quot; rel=&quot;noreferrer&quot;&gt;Template Numerical Toolkit&lt;/a&gt; is a C++ object-oriented library developed by NIST. Its author, Roldan Pozo, seems to contribute patches occasionally, but it doesn't seem to be under active development any longer (last update was 2010). It focuses on dense linear algebra, and provides interfaces for some basic matrix decompositions and an eigenvalue solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://libelemental.org/&quot; rel=&quot;noreferrer&quot;&gt;Elemental&lt;/a&gt;, developed by Jack Poulson, is a distributed memory (parallel) dense linear algebra software package written in a style similar to &lt;a href=&quot;http://z.cs.utexas.edu/wiki/flame.wiki/FrontPage&quot; rel=&quot;noreferrer&quot;&gt;FLAME&lt;/a&gt;. For a list of features and background on the project, see his &lt;a href=&quot;http://libelemental.org/documentation/&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt;. FLAME itself has an associated library for sequential and shared-memory dense linear algebra, called &lt;a href=&quot;http://z.cs.utexas.edu/wiki/flame.wiki/libflame/&quot; rel=&quot;noreferrer&quot;&gt;libflame&lt;/a&gt;, which appears to be written in object-oriented C. Libflame looks a lot like LAPACK, but with better notation underlying the algorithms to make development of fast numerical linear algebra libraries more of a science and less of a black art.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other libraries that can be added to the list; if we're counting sparse linear algebra packages as &quot;matrix libraries&quot;, the best free one I know of in C is &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/SuiteSparse/&quot; rel=&quot;noreferrer&quot;&gt;SuiteSparse&lt;/a&gt;, which is programmed in object-oriented style. I've used SuiteSparse and found it fairly easy to pick up; it depends on BLAS and LAPACK for some of the algorithms that decompose sparse problems into lots of small, dense linear algebra subproblems. The lead author of the package, Tim Davis, is incredibly helpful and a great all-around guy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.hsl.rl.ac.uk/&quot; rel=&quot;noreferrer&quot;&gt;Harwell Subroutine Libraries&lt;/a&gt; are famous for their sparse linear algebra routines, and are free for academic users, though you have to go through this process of filling out a form and receiving an e-mail for each file that you want to download. Since the subroutines often have dependencies, using one solver might require downloading five or six files, and the process can get somewhat tedious, especially since the form approval is not instantaneous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also other sparse linear algebra solvers, but as far as I can tell, &lt;a href=&quot;http://graal.ens-lyon.fr/MUMPS/index.php?page=home&quot; rel=&quot;noreferrer&quot;&gt;MUMPS&lt;/a&gt; and other packages are focused mostly on the solution of linear systems, and solving linear systems is the least of my concerns right now. (Maybe later, I will need that functionality, and it could be useful for others.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="28966" LastEditDate="2019-01-03T19:16:24.583" LastActivityDate="2019-01-03T19:16:24.583" CommentCount="12" CommunityOwnedDate="2012-03-07T12:34:32.630" ContentLicense="CC BY-SA 4.0" />
  <row Id="354" PostTypeId="2" ParentId="304" CreationDate="2011-12-12T05:25:53.147" Score="7" Body="&lt;p&gt;Three facts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;F77-style n-dimensional arrays in C: No problem using &lt;a href=&quot;http://pypi.python.org/pypi/cnd&quot;&gt;CnD&lt;/a&gt; (a shameless plug, admittedly)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;F90's module system is poorly designed and hostile to build environments. (A module's name doesn't have to match its filename, e.g.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Fortran does not support refactoring well. Pulling some bit of functionality out of a function requires you to touch four places: Actual code, variable declarations, argument declarations, and argument list. C gets by with two places to touch. This compounds the effect of the failure to manage data well (described below): Since small-scale modularity is so painful, just about everybody writes gigantic subroutines.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One personal impression:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fortran does not work well for managing data. Try returning a pointer to a user-opaque data structure in F77 or F90. (&lt;code&gt;transfer()&lt;/code&gt;, here we come)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="299" LastActivityDate="2011-12-12T05:25:53.147" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="355" PostTypeId="2" ParentId="350" CreationDate="2011-12-12T05:45:26.373" Score="12" Body="&lt;p&gt;Augh!! No, &lt;em&gt;no&lt;/em&gt;, a thousand times, &lt;strong&gt;no&lt;/strong&gt;!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason people use SVD is precisely to &lt;strong&gt;avoid&lt;/strong&gt; having to form the cross-product matrix $\mathbf A^\top\mathbf A$, since the formation of this matrix is a nice recipe for forming ill-conditioned linear systems! The decomposition is meant to be applied &lt;strong&gt;directly&lt;/strong&gt; to $\mathbf A$. (See also some of my &lt;a href=&quot;https://stats.stackexchange.com/a/1882&quot;&gt;previous&lt;/a&gt; &lt;a href=&quot;https://math.stackexchange.com/a/3871&quot;&gt;answers&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have mentioned to you before that the usual criterion for zeroing out singular values is to compare them with the product of the largest singular value and machine epsilon. However, this is rendered moot by your formation of the cross-product matrix. Please do try running the decomposition again, but this time, on the design matrix itself instead of the cross-product matrix. Any other way is flagrant abuse of the decomposition.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="-1" LastEditDate="2017-04-13T12:44:17.923" LastActivityDate="2011-12-12T05:45:26.373" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="356" PostTypeId="2" ParentId="350" CreationDate="2011-12-12T07:12:02.883" Score="12" Body="&lt;p&gt;The standard tolerance for forming a pseudoinverse is to only invert singular values that are at least $\max(m,n) \epsilon \|A\|_2$, where $A$ is $m \times n$, $\epsilon$ is the machine precision, and $\|A\|_2$ coincides with the largest singular value of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, as J.M. mentioned, it is much more stable to avoid forming $A^H A$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we need to compute the singular value decomposition&#xA;$$&#xD;&#xA;[U,\Sigma,V] = \mathrm{svd}(A)&#xD;&#xA;$$&#xA;then, we can define the pseudoinverse through&#xA;$$&#xD;&#xA;A^\dagger = V\; f(\Sigma)\; U^H,&#xD;&#xA;$$&#xA;where &#xA;$$&#xD;&#xA;f(\sigma) = \begin{cases}1/\sigma,\quad\sigma \ge \max(m,n) \epsilon \|A\|_2,\\&#xD;&#xA;0,\quad\text{otherwise}\end{cases}&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution can then be computed as &#xA;$$&#xD;&#xA;X = A^\dagger B.&#xD;&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2011-12-12T16:42:45.610" LastActivityDate="2011-12-12T16:42:45.610" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="357" PostTypeId="2" ParentId="36" CreationDate="2011-12-12T08:18:02.273" Score="4" Body="&lt;p&gt;For the numerical solution of hyperbolic PDEs the use of Riemann solvers are essential components of conservative shock capturing methods for accurate simulation of wave problems which may have shocks (discontinuous jumps in conserved variables). To be able to obtain accurate solutions to such problems, we need to use proper upwinding techniques -- the Riemann solver is responsible for this. The Riemann solver seek an accurate solution to the interface problem between cells (fx. in Finite Volumes) or elements (fx. in Discontinuous Galerkin Finite Element Methods). The solution of this interface problem is based on solution of either side of the interface and seeks to use this as basis for accurate reconstruction of the (numerical) flux (in terms of conserved variables) across the interface. Goal is to mimic the underlying physics as accurately as possible/necessary to obtain accurate solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two standard approaches to the solution of such (local to the interface) Riemann problems, namely, exact and approximate Riemann solvers. For many PDEs there are no exact closed-form solution available in which case we have to resort to approximate Riemann solvers. In practice, it may also be (too) expensive to exactly solve the Riemann problems in which case it may be more practical to resort to approximate Riemann solvers. For the same reason Lax-Freidrichs type fluxes are widely used as a simple means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, the choice between Riemann solvers has to do with how accurately one seeks to take represent the wave speeds of the solution and the resulting efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is problem dependent. The Riemann problem is based on data from either side of cell interfaces. To reconstruct the flux at the interface based on this data we must know information about the full wave structure of the hyperbolic PDE in question. This makes the Riemann problem problem-dependent and therefore also the choice of Riemann solver. In short, exact solvers seek to take into account the full wave structure, the Roe solver is based on local approximation (by linearization and special averaging) of the local wave structure, the HLL solver is based on estimating two dominant wave speeds in the local wave structure and then impose conservation by satisfying Rankine-Hugoniot condition to hold across shocks or contact discontinuities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the choice between specific solvers, exact solvers or approximate Roe/HLL/etc solvers depends on striking a balance between accuracy (in mimicking the underlying physics of the model equations) and efficiency needs. In the end -- as I see it -- in practical application it is often efficiency requirements that dictates the use of approximate Riemann solvers (fx. of the Lax-Friedrichs type). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good exposition to the subject is given by E. F. Toro in his textbook &quot;Riemann solvers and numerical methods for fluid dynamics&quot;, Springer.&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="304" LastEditDate="2011-12-13T13:36:18.647" LastActivityDate="2011-12-13T13:36:18.647" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="358" PostTypeId="2" ParentId="332" CreationDate="2011-12-12T08:26:33.680" Score="3" Body="&lt;p&gt;For the solution of differential equations, it may be useful to think in terms of The Method of Weighted Residuals (MWR) as coined by Crandall (1956) and described in a first review by Finnlayson and Scriven (1966) as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The Method of Weighted Residuals unifies many approximate methods of&#xA;  solution of differential equations that are being used currently.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The Method of Weighted Residuals is an engineer’s tool for finding approximate solutions to the equations of change of distributed systems.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In short, the MWR method unifies in a systematic way several common discretization methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this what you were thinking of?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the solution of systems of linear equations, I see Krylov subspace methods as a general approach to build projection methods. The most problem-specific part of these methods is the choice of preconditioner for acceleration of convergence -- and this is usually problem specific how to make that choice.&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2011-12-12T08:26:33.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="359" PostTypeId="2" ParentId="83" CreationDate="2011-12-12T08:31:36.460" Score="1" Body="&lt;p&gt;Since MATLAB has a JIT compiler while CPython does not yet (at least, until pypy gets full numpy support). It seems like you want a free solver that outperforms commercially produced &lt;code&gt;fmincon&lt;/code&gt;. Isn't it too much? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;IIRC among commercial NLP solvers, only snopt has provided a Python API until now (although it's rather ugly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which OpenOpt solvers have you tried? How many variables and constraints do you have in your nonconvex task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try IPOPT through OpenOpt / Funcdesigner API without installation on &lt;a href=&quot;http://sage.openopt.org/welcome&quot; rel=&quot;nofollow&quot;&gt;OpenOpt Sage server&lt;/a&gt; (pay attention to the &quot;switch from sage to python&quot; picture).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algencan, also connected to OpenOpt, says it solve nonconvex probs rather good (unfortunately it's absent in OO sage server). On the other hand, there is no gradient-based solver capable of yielding a guaranteed solution for nonconvex problems. Even some convex NLPs with 2 variables can be constructed where any solver will fail due to machine roundoff errors, e.g. $10^{300}(x-0.1)^2 + 10^{-300}(y-0.2)^2$ with starting guess like $(x,y) = (1,1)$.&lt;/p&gt;&#xA;" OwnerUserId="308" LastEditorUserId="276" LastEditDate="2012-01-02T23:13:43.847" LastActivityDate="2012-01-02T23:13:43.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="361" PostTypeId="2" ParentId="304" CreationDate="2011-12-12T14:22:07.000" Score="1" Body="&lt;p&gt;What Jack P. I think is trying to say is that you should mix and match.  A good piece of software is carefully layered.  Different layers may map more naturally, or efficiently, to different languages.  You should choose the most appropriate language for each layer.  You should also understand how languages can interoperate, which may affect what language you choose for what layer.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better question is what examples of excellently designed software are out there that are worth studying to learn about how to design layered software.  &lt;/p&gt;&#xA;" OwnerUserId="320" LastActivityDate="2011-12-12T14:22:07.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="362" PostTypeId="2" ParentId="311" CreationDate="2011-12-12T14:49:54.247" Score="1" Body="&lt;p&gt;It appears that the PCA-based feature selection generates pseudo-features that are linear combinations of the original features. After performing feature selection on the training set, I have not found a way to pull out the same pseudo-features from the test set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you're not tied to a particular feature selection method (PCA in this case), you can alternatively use a feature selection method that is more straightforward. For example, I ended up using a method that sorts the features according to information gain. Using this method, it is easy to identify the top &lt;code&gt;n&lt;/code&gt; features that give the best information gain and extract only those features from the training and test data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As suggested in the comments, there may be additional alternatives implemented in different languages (which I would be willing to consider, given a more detailed response), but this is the best I could find in the Weka environment.&lt;/p&gt;&#xA;" OwnerUserId="47" LastActivityDate="2011-12-12T14:49:54.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="363" PostTypeId="2" ParentId="83" CreationDate="2011-12-12T15:07:50.830" Score="7" Body="&lt;p&gt;Though this does not entirely answer your question, I author a Python package for nonlinear programming named NLPy. The most recent version may be retrieved from &lt;a href=&quot;https://github.com/dpo/nlpy&quot;&gt;https://github.com/dpo/nlpy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I must stress that NLPy is research-grade and the solvers included are by no means as robust as more seasoned codes like IPOPT. Moreover, they currently require that Jacobians be provided. That being said, the point of NLPy is to provide the tools needed for researchers to assemble custom solvers if they need to. At any rate, I'll be interested to hear your comments offline if you do give it a try. You may also find the related packages &lt;a href=&quot;https://github.com/dpo/pykrylov&quot;&gt;https://github.com/dpo/pykrylov&lt;/a&gt; and &lt;a href=&quot;https://github.com/dpo/pyorder&quot;&gt;https://github.com/dpo/pyorder&lt;/a&gt; useful. Currently, the documentation of NLPy is definitely lacking. The other two should be reasonable.&lt;/p&gt;&#xA;" OwnerUserId="324" LastActivityDate="2011-12-12T15:07:50.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="364" PostTypeId="2" ParentId="36" CreationDate="2011-12-12T15:45:01.627" Score="2" Body="&lt;p&gt;I've been lead to believe that for low order numerics one needs high quality Riemann solvers and for high order numerics one can use low quality Riemann solvers.  Intuitively there's some number of FLOPs necessary to capture the physics below which one is hosed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, yes, there's also zero content in this answer from an evaluation metric perspective...&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2011-12-12T15:45:01.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="365" PostTypeId="1" AcceptedAnswerId="370" CreationDate="2011-12-12T16:36:05.907" Score="16" ViewCount="1860" Body="&lt;p&gt;Which approaches are used in practice for estimating the condition number of large sparse matrices?&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="127" LastEditDate="2011-12-14T14:42:08.600" LastActivityDate="2012-07-26T15:25:50.647" Title="Estimation of condition numbers for very large matrices" Tags="|linear-algebra|matrix|conditioning|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="366" PostTypeId="1" AcceptedAnswerId="371" CreationDate="2011-12-12T16:45:58.037" Score="23" ViewCount="4722" Body="&lt;p&gt;The OpenCL programming paradigm promises to be a royalty free opens standard for heterogenous computing. Should we invest our time in developing software based on OpenCL? Pros/cons? &lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-04-16T15:09:16.450" Title="Future of OpenCL?" Tags="|hpc|programming-paradigms|" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="367" PostTypeId="2" ParentId="365" CreationDate="2011-12-12T16:50:33.797" Score="12" Body="&lt;p&gt;My previous answer recommended Dixon's 1983 paper, &lt;a href=&quot;http://dx.doi.org/10.1137/0720053&quot; rel=&quot;noreferrer&quot;&gt;&quot;Estimating extremal eigenvalues and condition numbers of matrices&quot;&lt;/a&gt;. It essentially boils down to a modest number of matrix-vector multiplications and solves against Gaussian random vectors and is essentially the power algorithm coupled with a priori error bounds which are not dependent on the spectrum of the operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the same sense that Krylov algorithms are strictly better than the power algorithm, Kuczynski and Wozniakowski &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.4725&quot; rel=&quot;noreferrer&quot;&gt;analyzed an analogue to Dixon's algorithm based upon Lanczos decompositions&lt;/a&gt; which will converge significantly faster on average.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-07-26T15:25:50.647" LastActivityDate="2012-07-26T15:25:50.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="369" PostTypeId="2" ParentId="366" CreationDate="2011-12-12T17:45:17.260" Score="7" Body="&lt;p&gt;Whether you should invest your time in developing software based on &lt;a href=&quot;http://en.wikipedia.org/wiki/OpenCL&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCL&lt;/a&gt; is a question only you can answer. If it looks like it has the potential to solve the problems you are facing right now, and no other open solution does, your best course of action is probably to take a risk on implementing a small project with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that goes well, you can try it with larger projects and so on until you either build up enough confidence to standardise on it, or discard it in favour of some other solution (which might be your own proprietary solution, another open solution or even another proprietary solution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The wonderful thing about the open source movement is that because &lt;em&gt;you have the source&lt;/em&gt; you have everything you need to fork the project if necessary. Even if the community itself doesn't give you the facilities you need, there's nothing stopping you implementing those facilities yourselves. Also, if you wanted those facilities, there is a distinct possibility that other users might want them, so would appreciate it if you contributed those changes back to the core project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not only that, but if you make it better from your perspective, it might make it better for others, encourage them to submit their own enhancements and ultimately make the software better for everyone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Finally, yes this is a rather generic answer to rather a generic question. To answer more fully, we need to know what your concerns over OpenCL are. Is it maturity? Community support? Ease of use? Time needed to learn? Time to develop? Changing your procedures? When you ask about the Pros and Cons, which &lt;a href=&quot;https://stackoverflow.com/questions/2915350/directcompute-versus-opencl-for-gpu-programming&quot;&gt;other products&lt;/a&gt; are you attempting to &lt;a href=&quot;https://stackoverflow.com/questions/3172220/opencl-vs-directcompute&quot;&gt;compare OpenCL to&lt;/a&gt;? What research have you already done? What features do you need to support your heterogeneous computing environment?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2011-12-12T17:45:17.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="370" PostTypeId="2" ParentId="365" CreationDate="2011-12-12T17:45:51.287" Score="8" Body="&lt;p&gt;It is very common to project the matrix into the Krylov space (generated by repeated application on a vector) and then to get the condition number of the projected matrix. In PETSc, this can be done automatically using -ksp_monitor_singular_value.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-12T17:45:51.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="371" PostTypeId="2" ParentId="366" CreationDate="2011-12-12T17:53:38.537" Score="19" Body="&lt;p&gt;The question is too broad and vague to be really be answered. However, I do see one notable point against OpenCL, from the point of view of scientific computing, which is rarely emphasized. So far, there has been no effort to produce open source, infrastructure libraries for OpenCL, whereas CUDA has several excellent options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://developer.nvidia.com/cuBLAS&quot;&gt;CUBLAS&lt;/a&gt;, &lt;a href=&quot;http://developer.nvidia.com/cuFFT&quot;&gt;CUFFT&lt;/a&gt;, &lt;a href=&quot;http://developer.nvidia.com/cuSPARSE&quot;&gt;CUSPARSE&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://code.google.com/p/thrust/&quot;&gt;Thrust&lt;/a&gt;, &lt;a href=&quot;http://code.google.com/p/cusp-library/&quot;&gt;Cusp&lt;/a&gt;, &lt;a href=&quot;http://code.google.com/p/opencurrent/&quot;&gt;OpenCurrent&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/features/gpus.html&quot;&gt;PETSc support&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I believe this will really hurt OpenCL since a major facilitator of adoption is high quality, open libraries.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="21" LastEditDate="2011-12-12T18:43:12.680" LastActivityDate="2011-12-12T18:43:12.680" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="372" PostTypeId="2" ParentId="366" CreationDate="2011-12-12T19:03:33.933" Score="4" Body="&lt;p&gt;The most important factor is that CUDA will remain supported only by NVIDIA hardware. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, if you want to make robust and portable software, OpenCL is the only option. At most you can build around some currently CUDA-powered libraries and hope they will get extended over OpenCL in future pulling your code with it.&lt;/p&gt;&#xA;" OwnerUserId="102" LastActivityDate="2011-12-12T19:03:33.933" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="373" PostTypeId="1" AcceptedAnswerId="374" CreationDate="2011-12-12T19:03:49.283" Score="7" ViewCount="899" Body="&lt;p&gt;This may be a Petsc newbie question, but...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Petsc to solve a large sparse linear system. The initial creation of the matrix is fairly slow, which I'm given to understand is largely due to memory allocations; that's tolerable for the moment. My problem is that as the simulation progresses, the structure of my domain/mesh changes such that the size of the resulting matrix will need to be increased. Do I really need to create a brand new matrix for this each time, or is there a way to just resize an existing matrix? (i.e. in order to avoid the cost of re-allocating the entire matrix from scratch.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried calling MatSetSizes, but it gives me an error: &quot;Cannot change/reset row sizes to [...]&quot;  Am I doing something wrong, or is this a fundamental limitation?&lt;/p&gt;&#xA;" OwnerUserId="326" LastEditorUserId="326" LastEditDate="2011-12-12T19:41:19.677" LastActivityDate="2012-01-02T23:02:55.003" Title="Is it possible to dynamically resize a sparse matrix in the Petsc library?" Tags="|pde|linear-algebra|petsc|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="374" PostTypeId="2" ParentId="373" CreationDate="2011-12-12T19:35:46.257" Score="10" Body="&lt;p&gt;The general advice given on the mailing list is usually about &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/faq.html#efficient-assembly&quot;&gt;efficient assembly&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short answer is, no; this is a fundamental limitation. The longer answer is that, for one, there is no &quot;end-all-be-all&quot; sparse matrix implementation. In general, it will be faster to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;loop over your mesh / domain, counting how many entries you'll need in your assembled matrix&lt;/li&gt;&#xA;&lt;li&gt;create / allocate said matrix&lt;/li&gt;&#xA;&lt;li&gt;assemble matrix&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Most of the time will be spent in the linear / nonlinear solvers, so the matrix assembly is small (well, at least that's the goal) compared to computing the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to check out creating a VecScatter; or even better, trying to use PETSc's DMDA (for structured) or Sieve (experimental, for unstructured and mostly FEM domains) to manage your mesh. These PETSc mesh objects will maintain much of the nitty-gritty partitioning, matrix creation, and vec scattering for you, so you don't have to write the MPI code yourself.&lt;/p&gt;&#xA;" OwnerUserId="142" LastEditorUserId="142" LastEditDate="2011-12-13T00:44:06.520" LastActivityDate="2011-12-13T00:44:06.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="375" PostTypeId="2" ParentId="373" CreationDate="2011-12-12T19:38:21.483" Score="4" Body="&lt;p&gt;The short answer is no.  Once a matrix has been assembled, you can't resize it, MatSetSizes() is used for setting the size before assembly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would look into making the matrix creation faster - if you preallocate the matrix appropriately, see for example &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/faq.html#efficient-assembly&quot; rel=&quot;nofollow&quot;&gt;this question in the PETSc FAQ&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="336" LastEditorUserId="276" LastEditDate="2012-01-02T23:02:55.003" LastActivityDate="2012-01-02T23:02:55.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="376" PostTypeId="1" AcceptedAnswerId="1294" CreationDate="2011-12-12T20:11:28.633" Score="36" ViewCount="8183" Body="&lt;p&gt;I am looking for information from anyone that has tried to use OpenCL in their scientific code. Has anyone tried (recently) &lt;a href=&quot;http://viennacl.sourceforge.net/&quot;&gt;ViennaCL&lt;/a&gt;? If so, how does it compare to &lt;a href=&quot;http://code.google.com/p/cusp-library/&quot;&gt;cusp&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about &lt;a href=&quot;http://www.clusterchimps.org/ocltools.php&quot;&gt;OCLTools&lt;/a&gt;? Does it live up to the promise? If so, would it be a feasible way to start writing math kernels in OpenCL?&lt;/p&gt;&#xA;" OwnerUserId="142" LastEditorUserId="9" LastEditDate="2012-02-18T12:45:47.367" LastActivityDate="2012-02-18T12:45:47.367" Title="Mathematical Libraries for OpenCL?" Tags="|parallel-computing|libraries|programming-paradigms|viennacl|opencl|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="377" PostTypeId="2" ParentId="159" CreationDate="2011-12-12T22:16:56.240" Score="6" Body="&lt;p&gt;My 2 cents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it is easier to write generally about this, rather than just&#xA;about C/C++. First, libraries in languages like Python are not&#xA;necessarily used to get a speed benefit, even if that is a&#xA;consequence. I think&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/a/162/38&quot;&gt;@David&lt;/a&gt; covered the&#xA;reasons pretty well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking it from the top, the language implementation to some extent&#xA;dictates what libraries you have access to. Commonly used languages in&#xA;computational science include C, C++, Python, Perl, Java, Fortran, and&#xA;R. Less common examples might be Ocaml and Common Lisp. Now, since&#xA;most of these languages are written in C, they have a natural &lt;a href=&quot;http://en.wikipedia.org/wiki/Foreign_function_interface&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foreign&#xA;function&#xA;interface&lt;/a&gt; to&#xA;C. However, it is not so easy to call, say, a Perl library from Python&#xA;or vice versa. So in practice people tend to either&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a library written in their implementation language, usally&#xA;something that is part of the standard libraries, or otherwise widely&#xA;available, or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Call a C/C++ library through the languages FFI. This assumes that&#xA;a wrapper does not already exist, since if it does, it is not easily&#xA;distinguishable from (1).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(2) is usually harder, because you have to wrap the C/C++ function&#xA;yourself. Also, you have to either bundle the library, or add an extra&#xA;dependency. For that reason, people are more likely to use the builtin&#xA;language libraries rather than use GSL for example, which is in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For very generic routines, say generating random samples from&#xA;distributions, or basic numerical routines like quadrature of&#xA;integrals, it is easy and common to reuse some library. As the&#xA;functionality one is trying to implement becomes more complex, it&#xA;becomes exponentially more unlikely that one is going to find the&#xA;exact function one wants in another library, and even one does, one&#xA;could spend lots of time searching and finally adapting the function&#xA;as necessary (the code style/design could be a problem for&#xA;example). And as discussed above, one has access to only a subset of&#xA;the libraries out there. On the other hand, implementing an algorithm&#xA;oneself if it is complex and not the main focus can be daunting, and&#xA;of course one has to deal with those pesky speed issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, this becomes an optimization problem in cost/benefit analysis. My&#xA;experience is that even for comparatively standard techniques like&#xA;MCMC, I usually wind up writing my own code, because it fits better&#xA;with how I am designing the overall software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, even if you end up not using the code, it is possible to&#xA;learn from other people's code. I don't know how often scientists&#xA;actually bother to do this, though. My impression is that reading&#xA;other people's code to learn is more a software engineer thing.&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2011-12-12T22:16:56.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="378" PostTypeId="2" ParentId="81" CreationDate="2011-12-12T22:49:52.820" Score="20" Body="&lt;p&gt;The choice between direct and iterative methods is dependent on goals and problem at hand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Direct methods, we can note:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Coefficient_matrix&quot; rel=&quot;noreferrer&quot;&gt;coefficient matrix&lt;/a&gt; of the linear system changes over the course of computation and may for sparse systems exhaust memory requirements and increase work effort due to fill-in&lt;/li&gt;&#xA;&lt;li&gt;Must complete to give useful results&lt;/li&gt;&#xA;&lt;li&gt;Factorization can be reused in subsequent steps if multiple right-hand sides are present&lt;/li&gt;&#xA;&lt;li&gt;Can be used for solving linear systems only.&lt;/li&gt;&#xA;&lt;li&gt;Seldom fails.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For Iterative methods, we can note:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The goal is to give a partial result only after a small number of iterations.&lt;/li&gt;&#xA;&lt;li&gt;Solution effort should be less than direct methods for the same problem.&lt;/li&gt;&#xA;&lt;li&gt;Economical with respect to storage (no fill-in)&lt;/li&gt;&#xA;&lt;li&gt;Often easy to program.&lt;/li&gt;&#xA;&lt;li&gt;A known approximate solution can be exploited.&lt;/li&gt;&#xA;&lt;li&gt;Sometimes they are fast and sometimes they are not (sometimes even divergent).&lt;/li&gt;&#xA;&lt;li&gt;For complex problems, iterative methods are considerably less robust compared to direct methods.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Guidelines for when to use direct or iterative methods? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Iterative methods when the coefficient matrix is sparse and direct methods cannot exploit sparsity efficiently (avoid creating fill-in).&lt;/li&gt;&#xA;&lt;li&gt;Direct methods for multiple right-hand sides.&lt;/li&gt;&#xA;&lt;li&gt;Iterative methods can be more efficient if accuracy is of less concern&lt;/li&gt;&#xA;&lt;li&gt;Iterative methods for nonlinear systems of equations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="304" LastEditorUserId="127" LastEditDate="2016-12-11T09:44:06.993" LastActivityDate="2016-12-11T09:44:06.993" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="379" PostTypeId="2" ParentId="194" CreationDate="2011-12-12T23:27:59.187" Score="1" Body="&lt;p&gt;I assume your scalability issue is related to output, and not to input. Parallel input is rather simple - what I do is each CPU opens the input NetCDF file and reads the part of the array that belongs to its tile (there might be a limit to how many readers can open the same NetCDF file but I am not sure). Parallel output is more problematic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am doing currently is not quite optimal, but works for now. I gather the whole thing on one CPU and do the serial output. In the meantime, other players wait for the writer to finish. This worked well for me because I managed to keep the computation over output ratio quite high - so scalability would be good for well over 200 CPUs. But this is not the solution you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another solution is what Yann suggested - write serially to N files and have a drone CPU assemble the tiles to one piece - if it is RAM permitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from parallel I/O libraries suggested in previous answers, you also may want to look into Parallel NetCDF &lt;a href=&quot;http://trac.mcs.anl.gov/projects/parallel-netcdf&quot; rel=&quot;nofollow&quot;&gt;http://trac.mcs.anl.gov/projects/parallel-netcdf&lt;/a&gt;, since you are already comfortable with NetCDF and MPI. I didn't use it in practice, but do plan to go in that direction when I hit the wall with gathering + serial I/O.&lt;/p&gt;&#xA;" OwnerUserId="264" LastActivityDate="2011-12-12T23:27:59.187" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="380" PostTypeId="2" ParentId="159" CreationDate="2011-12-13T00:19:45.747" Score="5" Body="&lt;p&gt;The fundamental problem is often with the interface between the application and the library.  An application programmer has knowledge about the problem that is often lost when passing the problem (for example as a matrix) to a library.  This knowledge is such that exploiting it more than offsets the benefits of using the highly optimized library.  As a result, the application programmer &quot;rolls&quot; his/her own implementation that does exploit the knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, a really good library needs for such knowledge to be passed from the application into the library, so that the library, too, can take advantage of it.&lt;/p&gt;&#xA;" OwnerUserId="320" LastActivityDate="2011-12-13T00:19:45.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="381" PostTypeId="2" ParentId="346" CreationDate="2011-12-13T02:36:57.370" Score="18" Body="&lt;p&gt;There are a couple elements I look for when I consider something &quot;publication-quality&quot; in either my own work, or what I'm considering when looking at others. They are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;High resolution, and preferably vector-based. This one should be fairly obvious by now, but you'd be surprised.&lt;/li&gt;&#xA;&lt;li&gt;A lack of clutter. I should be able to see what's happening in your figure, and see it quickly. There's few things I hate more than someone trying to take the &quot;High Ink:Paper ratio&quot; guidance and using it to try to cram an entire manuscript in a single figure.&lt;/li&gt;&#xA;&lt;li&gt;Prints well. This is the one that's actually most important for me, and when I'm reviewing papers, one I always test. &quot;Do the figures print?&quot; More than once, I've hit figures whose points are completely obfuscated when printed in grayscale, which renders them worthless for my purposes (I don't read on screens).&lt;/li&gt;&#xA;&lt;li&gt;Evidence that the creator knows how to use graphics settings. No odd-ball axis choices, tick marks in the right place, etc.&lt;/li&gt;&#xA;&lt;li&gt;Combined with #2, a lack of &quot;flourish&quot; that's entirely graphical in nature. Shadows, needless 3-D, etc. that really do nothing but waste the readers time.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Most of those are honestly creator-specific, rather than program specific. I've seen terrible plots done in R, and excellent plots done in Excel.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2011-12-13T02:36:57.370" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="382" PostTypeId="1" CreationDate="2011-12-13T02:48:21.017" Score="7" ViewCount="7413" Body="&lt;p&gt;I'm looking for C/C++ implementations of functions that return random variates multinomial and Dirichlet distributions. This is in the context of a calculation for posterior predictive p-values, part of which includes a MCMC step. I've been using Python, but am rewriting parts of my code in C++ for speed reasons. I've been using the implementations from numpy.random thus far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current options include&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Pull the implementations from numpy.random, which are most likely&#xA;written in C.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the R versions of these functions, which are conveniently&#xA;available for Debian in the&#xA;r-&lt;a href=&quot;http://packages.debian.org/squeeze/r-mathlib&quot;&gt;mathlib&lt;/a&gt; library. This&#xA;library has been my default choice for such things for a long time,&#xA;because the R people know their stuff when it comes to probabilistic&#xA;computing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, I'd be open to other versions of these functions. Suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: r-mathlib doesn't appear to have a function for sampling from the Dirichet distribution, though according to &lt;a href=&quot;http://en.wikipedia.org/wiki/Dirichlet_distribution#Gamma_distribution&quot;&gt;Wikipedia's entry on the Dirichlet distribution&lt;/a&gt; I can sum Gamma variates. I wonder if that is a good way of doing this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2: If possible, please comment on why you think the implementation you suggest is a good choice.&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="38" LastEditDate="2011-12-13T08:46:21.693" LastActivityDate="2014-04-02T19:49:37.397" Title="Looking for C/C++ implementations of sampling from multinomial and Dirichlet distributions" Tags="|random-number-generation|statistics|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="383" PostTypeId="2" ParentId="382" CreationDate="2011-12-13T05:37:41.833" Score="10" Body="&lt;p&gt;I would recommend the &lt;a href=&quot;http://www.gnu.org/s/gsl/&quot;&gt;GNU Scientific Library (GSL)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GSL has both &lt;code&gt;gsl_ran_dirichlet&lt;/code&gt; and &lt;code&gt;gsl_ran_multinomial&lt;/code&gt; for the Dirichlet and multinomial distributions respectively.  See the manual for the Dirichlet distribution is &lt;a href=&quot;http://www.gnu.org/s/gsl/manual/html_node/The-Dirichlet-Distribution.html&quot;&gt;here&lt;/a&gt; and the manual page for the multinomial distribution is &lt;a href=&quot;http://www.gnu.org/s/gsl/manual/html_node/The-Multinomial-Distribution.html&quot;&gt;here&lt;/a&gt;.  Full documentation can be found &lt;a href=&quot;http://www.gnu.org/s/gsl/manual/gsl-ref.ps.gz&quot;&gt;here&lt;/a&gt; or online &lt;a href=&quot;http://www.gnu.org/s/gsl/manual/html_node/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="290" LastActivityDate="2011-12-13T05:37:41.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="384" PostTypeId="1" AcceptedAnswerId="385" CreationDate="2011-12-13T05:57:40.373" Score="9" ViewCount="343" Body="&lt;p&gt;When solving sparse linear systems using direct factorization methods, the ordering strategy used significantly impacts the fill-in factor of non-zero elements in the factors. One such ordering strategy is nested dissection. I am wondering if it is possible to come up with the nested dissection ordering ahead of time given only the grid parameters (assume an M x N square finite difference grid with first order differences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&#xA;I just found that there is code that does this:&#xA;&lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/meshnd/&quot;&gt;http://www.cise.ufl.edu/research/sparse/meshnd/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2011-12-13T12:46:40.433" LastActivityDate="2011-12-13T15:34:47.927" Title="Nested dissection on regular grid" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="385" PostTypeId="2" ParentId="384" CreationDate="2011-12-13T06:34:00.047" Score="8" Body="&lt;p&gt;Yes. I recently wrote code to do exactly this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you have an $n_x \times n_y$ grid, and that it is acceptable to have leaf nodes with 100 vertices. One can then define a recursive function where the arguments are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the dimensions and offsets of a rectangular subdomain&lt;/li&gt;&#xA;&lt;li&gt;a pointer into an array that will store the reordering&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The routine then simply has to compute the product of the local dimensions to decide whether or not the domain is an acceptably small to be a leaf, and then, if so, write the leaf node natural indices (say $\mathrm{natural}(x,y)=x+y n_x$ for an $n_x \times n_y$ grid), otherwise, cut the largest subdomain dimension, recurse on the left and right pieces, and then write the separator natural indices.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2011-12-13T15:34:47.927" LastActivityDate="2011-12-13T15:34:47.927" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="387" PostTypeId="2" ParentId="346" CreationDate="2011-12-13T17:06:17.033" Score="6" Body="&lt;p&gt;It's almost easier to characterize what constitutes a bad graph than what makes a graph good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some features of bad graphs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Excessively large or small fonts and symbols&lt;/li&gt;&#xA;&lt;li&gt;Excessively thin or thick lines for curves and other graph features&lt;/li&gt;&#xA;&lt;li&gt;Too many different variables being shown or varied at the same time&lt;/li&gt;&#xA;&lt;li&gt;Having inappropriate axis selections (log versus linear, range, etc.)&lt;/li&gt;&#xA;&lt;li&gt;Showing trends between data points with solid curves that indicate progressions or behavior that might not exist&lt;/li&gt;&#xA;&lt;li&gt;Giving no indication of the magnitude of uncertainties or errors&lt;/li&gt;&#xA;&lt;li&gt;Poorly captioned or labeled graphs (including units!)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In general, though, while most software packages are capable of creating good graphics, almost no program I've ever worked with &lt;em&gt;defaults&lt;/em&gt; to a state that produces good graphics. They always require tweaking: either font sizes, or display ranges, or axes or symbol choices, and so on. Currently, I prefer to use matplotlib; others in my group have migrated to SciDavis.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2011-12-13T23:43:11.037" LastActivityDate="2011-12-13T23:43:11.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="388" PostTypeId="1" AcceptedAnswerId="389" CreationDate="2011-12-13T17:12:55.567" Score="9" ViewCount="2114" Body="&lt;p&gt;I am trying to integrate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int^1_0 t^{2n+2}\exp\left({\frac{\alpha r_0}{t}}\right)dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is a simple transformation of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int^{\infty}_1 x^{2n}\exp(-\alpha r_0 x)dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using $t = \frac1{x}$ because it is difficult to numerically approximate improper integrals.&#xA;This does, however, lead to the problem of evaluating the new integrand near zero. It will be very easy to get the proper number of quadrature nodes seeing as the interval is only of length 1 (so the comparable $dt$ can be made very small), but what sort of considerations should I make when integrating near zero?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On some level, I think that simply taking $\int^1_\epsilon t^{2n+2}\exp({\frac{\alpha r_0} {t}})dt$ is a good idea where $\epsilon$ is some small number. However, what number should I choose? Should it be machine epsilon? Is division by machine epsilon a well quantified number? Furthermore, if division my machine epsilon (or close to it) gives an incredibly large number, then taking $\exp(\frac{1}{\epsilon})$ will become even larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I account for this? Is there a way to have a well defined numerical integral of this function? If not, what is the best way of integrating the function?&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="127" LastEditDate="2011-12-14T14:42:15.823" LastActivityDate="2012-01-02T23:06:03.657" Title="numerical integration with possible division by 'zero'" Tags="|quadrature|accuracy|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="389" PostTypeId="2" ParentId="388" CreationDate="2011-12-13T17:43:54.600" Score="10" Body="&lt;p&gt;This can be done by integration by parts:&#xA;$$&#xD;&#xA;  \int^\infty_1 x e^{-ax} = \frac{-1}{a} x e^{-ax}\mid^\infty_1 - \frac{-1}{a} \int^\infty_1 e^{-ax} = \frac{e^{-a}}{a} + \frac{e^{-a}}{a^2} = \frac{a+1}{a^2} e^{-a}&#xD;&#xA;$$&#xA;and continuing on by induction&#xA;$$&#xD;&#xA;  \int^\infty_1 x^k e^{-ax} = \frac{-1}{a} x^k e^{-ax}\mid^\infty_1 - \frac{-k}{a} \int^\infty_1 x^{k-1} e^{-ax} = \frac{e^{-a}}{a} + \frac{k}{a} \int^\infty_1 x^{k-1} e^{-ax}&#xD;&#xA;$$&#xA;so that&#xA;$$&#xD;&#xA;  I(k) = \frac{e^{-a}}{a} + \frac{k}{a} I(k-1)&#xD;&#xA;$$&#xA;and $I(0) = \frac{e^{-a}}{a}$.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-13T17:43:54.600" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="390" PostTypeId="2" ParentId="159" CreationDate="2011-12-13T18:45:54.367" Score="7" Body="&lt;p&gt;Before deciding whether or not to use libraries, I think you'd also want to figure out how much the use of a library will help your code. If you're going to be using a well-optimized library for a key computational kernel, then it's probably a lot more efficient than trying to write your own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you're writing a specialized routine that's only going to be called once during a program's execution, it may not be worth it to adapt your code to fit the framework required by a library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Another thing to think about: how much re-architecting will you need to do to take advantage of the library? Unless the man-hours you spend to fix the code are compensated for by corresponding gains in efficiency or numerical accuracy, it may not be worth it in the long run. Ideally, though, this is something you plan for when initially designing data structures and algorithms, so that the library's &quot;flow&quot; is taken into account from the ground up.)&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2011-12-13T18:45:54.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="391" PostTypeId="1" CreationDate="2011-12-13T18:48:12.483" Score="13" ViewCount="595" Body="&lt;p&gt;If we employ the Method of Lines for discretization (separate time and space discretization) of hyperbolic PDEs we obtain after spatial discretization by our favorite numerical method (fx. Finite Volume Method) does it matter in practice which ODE solver we employ for the temporal discretization (TVD/SSP/etc)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some additional information added:&#xA;The accuracy issue can be a problem for non-smooth problems. It is will-known that nonlinear hyperbolic PDEs can develop shocks in finite time despite the initial solution is smooth in which case accuracy can degrade to first order for high-order methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;ODE Stability analysis is typically done based on linearization to obtain a linear semi-discrete system of ODEs of the form q_t = J q (with q a perturbation vector), where the eigenvalues of J should be scaled inside the absolute stability region of chosen time-stepping method. Alternative strategies is to use pseudospectra or possibly an energy method for stability analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the motivation for TVD/SSP methods are to avoid spurious oscillations caused by the time-stepping methods which may result in unphysical behavior. Question is if experiences shows these types of time-stepping methods to be superior compared to, e.g., a classical work horse as explicit Runge-Kutta Method or others. Obviously, they should have better properties for classes of problem where solution may exhibit shocks. One could therefore argue that we should only employ these types of methods for time-integration. &lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="304" LastEditDate="2011-12-19T20:51:21.563" LastActivityDate="2013-03-18T13:21:01.773" Title="Which time-integration methods should we use for hyperbolic PDEs?" Tags="|hyperbolic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="392" PostTypeId="1" AcceptedAnswerId="398" CreationDate="2011-12-13T19:20:08.207" Score="27" ViewCount="823" Body="&lt;p&gt;Developers of software have the choice to choose an appropriate license in accordance with the goal(s) of the work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give some recommendations/experiences on which license to pick for software?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros/cons of &quot;giving away&quot; all the coded work as open source codes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to deal with industrial players which would like to benefit from the research code?&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="41" LastEditDate="2012-01-09T01:57:27.017" LastActivityDate="2023-12-08T11:54:02.780" Title="Recommendations and experiences on which license to choose for software?" Tags="|software|" AnswerCount="10" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="393" PostTypeId="2" ParentId="392" CreationDate="2011-12-13T19:41:54.857" Score="5" Body="&lt;p&gt;I use GPL, mainly because of the sentiment to the original open source movement (and thus hope that it will help its spread). Moreover, this is paradoxically a best way to protect your possible earnings from immoral capitalists -- as an author, you can always distribute the code on a different license in parallel and thus maintain proprietary version for a white-label business use.&lt;br&gt;&#xA;However, this may be also a con -- your funding source might made a disclaimer that all your work should become public domain, what goes beyond this license. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, the most important thing in that topic is that &lt;strong&gt;any&lt;/strong&gt; license is better than none, which is unfortunately quite often in the scientific development world -- and I just hate all those &lt;em&gt;/*Stolen from John Smith's program he has never publicized*/&lt;/em&gt; or &lt;em&gt;C  I think I saw this in Jane Smith's post on some group in 1995... or maybe 1993?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="102" LastActivityDate="2011-12-13T19:41:54.857" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="394" PostTypeId="2" ParentId="392" CreationDate="2011-12-13T19:59:24.823" Score="12" Body="&lt;p&gt;PETSc uses this &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/copyright.html&quot;&gt;license&lt;/a&gt;, which is a less restrictive form of &lt;a href=&quot;http://en.wikipedia.org/wiki/BSD_licenses&quot;&gt;BSD&lt;/a&gt;. The crucial difference from &lt;a href=&quot;http://www.gnu.org/copyleft/gpl.html&quot;&gt;GPL&lt;/a&gt;, is that the software can be used commercially. Many people have a principled objection to closed software, however in my experience no business will go near your code if it has a GPL license. Moreover, PETSc's industrial users have been incredibly valuable. They tend to run quite complex problems, which most academics would find more difficult than warranted for a publication. They have also contributed a lot of code back to PETSc, so that it would enter the normal support chain. I would advise against any license without commercial use potential, and in fact advocate the least restrictive license possible (you could definitely burn PETSc to a CD and try and sell it to the gullible).&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2011-12-13T19:59:24.823" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="395" PostTypeId="1" CreationDate="2011-12-13T20:00:32.077" Score="12" ViewCount="2415" Body="&lt;p&gt;I would like to visualize a molecular vibration that is &lt;strong&gt;not&lt;/strong&gt; a normal mode. I'd like to present a static, vectorial representation of the motion, and I would like some flexibility in the vector style (size, color etc). I'm also interested in producing a video of the vibration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are some good resources for displaying molecular vibrations?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My preference is for open source tools, but I'll entertain using commercial software if it really is better then the alternatives. &lt;/p&gt;&#xA;" OwnerUserId="79" LastActivityDate="2013-10-23T15:03:18.327" Title="What open source tools are available to visualize molecular vibrations?" Tags="|computational-chemistry|" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="396" PostTypeId="1" CreationDate="2011-12-13T21:46:28.780" Score="7" ViewCount="464" Body="&lt;p&gt;Is it possible to apply Richardson extrapolation with Euler-Maruyama scheme to improve strong rate of convergence of stochastic differential equations?&lt;/p&gt;&#xA;" OwnerUserId="353" LastEditorUserId="123" LastEditDate="2011-12-15T10:04:36.990" LastActivityDate="2016-01-28T18:32:04.617" Title="Richardson extrapolation for strong rate of convergence of SDE" Tags="|ode|convergence|stochastic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="397" PostTypeId="2" ParentId="395" CreationDate="2011-12-13T23:50:16.743" Score="5" Body="&lt;p&gt;Would &lt;a href=&quot;http://www.pymol.org&quot;&gt;Pymol&lt;/a&gt; or &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot;&gt;VMD&lt;/a&gt; be a suitable tool for your video needs? (VMD at least includes Tk/Tcl scripting features.) You would need a PDB-like description of your geometry in order to use VMD; this would probably be sufficient for Pymol as well (but I haven't used Pymol, so perhaps someone else can comment on that).&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2011-12-13T23:50:16.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="398" PostTypeId="2" ParentId="392" CreationDate="2011-12-14T00:37:50.977" Score="19" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Can anyone give some recommendations/experiences on which license to pick for software?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Which license you chose will depend on how &lt;em&gt;free&lt;/em&gt; you want your code to be, but free means different things to different people.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For proponents of &lt;a href=&quot;http://en.wikipedia.org/wiki/Permissive_free_software_licence&quot; rel=&quot;nofollow noreferrer&quot;&gt;permissive&lt;/a&gt; licenses, &lt;em&gt;free&lt;/em&gt; means allowing people now to use the software however they want to &lt;em&gt;right now&lt;/em&gt;, not worrying about how &lt;em&gt;free&lt;/em&gt;  future derivation are.&lt;/li&gt;&#xA;&lt;li&gt;For proponents of &lt;a href=&quot;http://en.wikipedia.org/wiki/Copyleft&quot; rel=&quot;nofollow noreferrer&quot;&gt;copyleft&lt;/a&gt; licenses, &lt;em&gt;free&lt;/em&gt; means ensuring that the software and any derivation of it &lt;em&gt;stays free&lt;/em&gt;, being prepared to sacrifice some &lt;em&gt;immediate freedoms&lt;/em&gt; to ensure that.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The more permissive a license is, the more people will be able to use it, but the less control you have over it. The more restrictive it is though, the more likely you are to put people off using your software in the first place.&lt;/p&gt;&#xA;&lt;p&gt;There are a number of free and open source licenses out there, including &lt;a href=&quot;http://en.wikipedia.org/wiki/GNU_General_Public_License&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPL &amp;lt;=2, GPL 3&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/GNU_Lesser_General_Public_License&quot; rel=&quot;nofollow noreferrer&quot;&gt;LGPL&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/BSD_licenses&quot; rel=&quot;nofollow noreferrer&quot;&gt;BSD&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Eclipse_Public_License&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eclipse&lt;/a&gt; and so on. There are pro's and cons to each, so read up on what restrictions they place on the code and decide who you want to be able to use it. &lt;em&gt;Warning&lt;/em&gt;, whichever you choose someone will complain - this &lt;strong&gt;is&lt;/strong&gt; &lt;em&gt;holy war&lt;/em&gt; territory.&lt;/p&gt;&#xA;&lt;p&gt;Overall it is a subtle balancing act, and it depends very much on the target audience for your software.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A great resource for determining which license is the right license for you is the very comprehensive, interactive &lt;a href=&quot;http://oss-watch.ac.uk/apps/licdiff/&quot; rel=&quot;nofollow noreferrer&quot;&gt;license differentiator&lt;/a&gt;, from Oxford Universities &lt;a href=&quot;http://www.oss-watch.ac.uk/resources/licdiff.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;OSS Watch&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In my opinion, both &lt;a href=&quot;http://en.wikipedia.org/wiki/Permissive_free_software_licence&quot; rel=&quot;nofollow noreferrer&quot;&gt;permissive&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Copyleft&quot; rel=&quot;nofollow noreferrer&quot;&gt;copyleft&lt;/a&gt; licenses are appropriate for scientific code - the important thing is that the code is open source in the first place. I believe that Science should be Open, and so should the code used to support that science.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What are the pros/cons of &amp;quot;giving away&amp;quot; all the coded work as open source codes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The idea of giving away your software is that if others find it useful then they will use it.&lt;/p&gt;&#xA;&lt;p&gt;If they use it they will find, report and often fix bugs, saving your effort of doing the same.&lt;/p&gt;&#xA;&lt;p&gt;If they like it and your software does &lt;em&gt;almost&lt;/em&gt; what they want, they might enhance your software and contribute those enhancements back.&lt;/p&gt;&#xA;&lt;p&gt;That's a lot of &lt;em&gt;ifs&lt;/em&gt; though.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How to deal with industrial players which would like to benefit from the research code?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Firstly, if you want to prohibit commercial use of your code, you can select a license with a no commercial re-use clause.&lt;/p&gt;&#xA;&lt;p&gt;Secondly, if you think someone might use your software to power a service, without ever actually distributing the code to anyone else, then you could consider the &lt;a href=&quot;http://en.wikipedia.org/wiki/Affero_General_Public_License&quot; rel=&quot;nofollow noreferrer&quot;&gt;Affero GPL&lt;/a&gt; which plugs that particular copyleft loophole.&lt;/p&gt;&#xA;&lt;p&gt;Thirdly, you can do the above &lt;em&gt;and&lt;/em&gt; offer a dual license option. Offering GPL or AGPL licenses for public download, and commercial licenses for a fee gives you the best of both worlds, and means that you might even be able to generate some revenue from commercial sales of your software which can help support your scientific activities.&lt;/p&gt;&#xA;&lt;p&gt;Note, if you are going to do this, offer it from the outset - that is likely to cause less friction from your open source contributors than starting to offer commercial licenses later on. If your community becomes popular, you don't want people accusing you of selling out if you weren't straight about the possibility of commercial exploitation later. Ideally you should set up a suitable &lt;a href=&quot;http://en.wikipedia.org/wiki/Contributor_License_Agreement#Rationale&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contributor License Agreement (CLA)&lt;/a&gt; before you start accepting third party contributions into your codebase.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/5225/does-open-source-licensing-my-code-limit-me-later/5228#5228&quot;&gt;This answer&lt;/a&gt; to &lt;a href=&quot;https://softwareengineering.stackexchange.com/q/5225/22493&quot;&gt;this question&lt;/a&gt; provides some good information on this option too.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2023-12-08T11:54:02.780" LastActivityDate="2023-12-08T11:54:02.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="399" PostTypeId="1" AcceptedAnswerId="401" CreationDate="2011-12-14T02:32:52.810" Score="27" ViewCount="5249" Body="&lt;p&gt;I'm not running any parallel code right now, but I anticipate running parallel code in the future using a hybrid of OpenMP and MPI. Debuggers have been invaluable tools for me when running serial projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone recommend a parallel debugger (or multiple debuggers) to use for debugging parallel software? Free software would be preferable, but don't hesitate to mention effective commercial software.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2019-08-24T15:20:47.770" Title="What software is good to use for parallel debugging?" Tags="|software|parallel-computing|" AnswerCount="8" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="400" PostTypeId="2" ParentId="399" CreationDate="2011-12-14T03:28:19.180" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.roguewave.com/products/totalview-family/totalview.aspx&quot; rel=&quot;nofollow&quot;&gt;Totalview&lt;/a&gt;. It's a commercial debugger. It's very easy to view the stack on each processor. You can see variable values (and change them) across processors/threads. You can plot vectors or matracies to visualize variable values. Apparently scripting is possible too (Tk/Tcl), for sofisticated watch point analysis, though I've never worked with this myself.&lt;/p&gt;&#xA;" OwnerUserId="79" LastEditorUserId="79" LastEditDate="2011-12-14T03:33:23.470" LastActivityDate="2011-12-14T03:33:23.470" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="401" PostTypeId="2" ParentId="399" CreationDate="2011-12-14T04:58:04.290" Score="18" Body="&lt;p&gt;There are basically two major, commercial choices out there: &lt;a href=&quot;http://www.allinea.com/products/ddt/&quot;&gt;DDT&lt;/a&gt; from Allinea (which is what we use at &lt;a href=&quot;http://www.tacc.utexas.edu&quot;&gt;TACC&lt;/a&gt;) and Totalview (as mentioned in the other comment). They have comparable features, are both actively developed, and are direct competitors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eclipse has their &lt;a href=&quot;http://www.eclipse.org/ptp/&quot;&gt;Parallel Tools Platform&lt;/a&gt;, which should include MPI and OpenMP programming support and a parallel debugger. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2011-12-14T04:58:04.290" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="402" PostTypeId="2" ParentId="399" CreationDate="2011-12-14T05:12:33.077" Score="4" Body="&lt;p&gt;There is &lt;a href=&quot;http://software.intel.com/en-us/articles/intel-parallel-studio-home/&quot; rel=&quot;nofollow&quot;&gt;Intel Parallel Studio&lt;/a&gt; which includes a parallel debugger. I've never worked with it but I've seen it used in a few demos. Here's a &lt;a href=&quot;http://software.intel.com/en-us/videos/channel/general/intel-parallel-composer-parallel-debugger-extension-tutorial/1127586736001&quot; rel=&quot;nofollow&quot;&gt;video tutorial&lt;/a&gt; that shows some of the features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also seen a few wrappers around gdb that worked reasonably well in certain cases.&lt;/p&gt;&#xA;" OwnerUserId="358" LastActivityDate="2011-12-14T05:12:33.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="403" PostTypeId="2" ParentId="392" CreationDate="2011-12-14T07:33:44.960" Score="4" Body="&lt;p&gt;Regardless of what license you choose to go with, remember to carefully check through all your funding agreements to make sure there aren't clauses in them that either dictate or restrict how you might go about licensing your software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know in my case many of my projects are built up from several layers of funding, a little bit here and a little bit there, and keeping track of how I'm &lt;em&gt;allowed&lt;/em&gt; to license my stuff by the people who keep the lights on and the machines running is fairly complex.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2011-12-14T07:33:44.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="405" PostTypeId="2" ParentId="395" CreationDate="2011-12-14T10:43:41.730" Score="3" Body="&lt;p&gt;I am not aware of any ready-made (&quot;point-and-click&quot;) tool for that task. But using a combination of&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A short Python script&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://dirac.cnrs-orleans.fr/MMTK/&quot; rel=&quot;nofollow&quot;&gt;Molecular Modelling Toolkit&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cgl.ucsf.edu/chimera/&quot; rel=&quot;nofollow&quot;&gt;Chimera&lt;/a&gt; or &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow&quot;&gt;VMD&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;you can get a nice result quickly, given some Python knowledge. In fact, a very similar script is provided as an example with the Molecular Modelling Toolkit. Look at Examples/Visualization/vector_field_chimera.py or Examples/Visualization/vector_field_vmd.py. You need to replace the normal mode calculation by whatever it takes to get your vibration data into the script.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-12-14T10:43:41.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="406" PostTypeId="2" ParentId="392" CreationDate="2011-12-14T10:54:28.217" Score="5" Body="&lt;p&gt;First, the pros/cons of going open source:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pro: more people will use your code, provide feedback, corrections, improvements, etc. You will end up having better code and more people that trust it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Con: if you ever want to base a business in your code, you have fewer options (but there are still a few, such as selling consultancy services)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for choosing a licence, I'd proceed in the following order:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does your employer/grant agency impose anything? Then you don't have a choice. Check this for all contributors to the code.&lt;/li&gt;&#xA;&lt;li&gt;Do you re-use code that has some particular licence that limits your choice? Then your choices are limited as well. In practice, integrating pieces of GPL-licenced code is the most frequent source of such limitations.&lt;/li&gt;&#xA;&lt;li&gt;Decide what you value more: the all-code-should-be-open philosophy behind the GPL and similar licences, or the encourage-the-widest-possible-use philosophy behind the BSD licence.&lt;/li&gt;&#xA;&lt;li&gt;Within each of the two big Open Source licence families, choose according to what is most common/accepted in your community.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-12-14T10:54:28.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="407" PostTypeId="2" ParentId="392" CreationDate="2011-12-14T12:27:33.133" Score="4" Body="&lt;p&gt;For large bodies of code I go for one of the licences described in the other answers, and usually LGPL. However, &lt;a href=&quot;http://wiki.creativecommons.org/Frequently_Asked_Questions#Can_I_apply_a_Creative_Commons_license_to_software.3F&quot; rel=&quot;nofollow&quot;&gt;although not usually recommended for software&lt;/a&gt;, for small self-contained scripts that I may be sending to a colleague in industry I often opt for a &lt;a href=&quot;http://creativecommons.org/choose/&quot; rel=&quot;nofollow&quot;&gt;Creative Commons&lt;/a&gt; licence. This is because they tend to be clearer for the individual that I send the code to, which stops any potential problems of misunderstanding. This has worked out well for me in the past.&lt;/p&gt;&#xA;" OwnerUserId="116" LastEditorUserId="116" LastEditDate="2011-12-16T13:01:58.007" LastActivityDate="2011-12-16T13:01:58.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="408" PostTypeId="2" ParentId="392" CreationDate="2011-12-14T12:58:56.293" Score="5" Body="&lt;p&gt;Most of my research is funded by public funds, and so I feel an obligation to use the least restrictive license possible. If other people in my country are helping to pay for this research, do I really have the right to tell them they can't integrate my code into a closed source and/or commercial software distribution? I expect that most of the people that use my code will be academic scientists, but I have no philosophical problems with businesses improving my software by integrating it with other (possibly commercial) software, putting a GUI on it, etc, to deliver a product that helps them make a profit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, I try to use licenses that require proper attribution. For instance, if a company &lt;em&gt;does&lt;/em&gt; fold my code into a larger commercial product, I want them to make it clear that my code can be obtained freely from me. But other than that, I want to place as little requirements on users of my code as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For software development that is not funded by taxpayer dollars, I understand that other licenses may be more appropriate.&lt;/p&gt;&#xA;" OwnerUserId="47" LastEditorUserId="47" LastEditDate="2011-12-14T13:37:39.193" LastActivityDate="2011-12-14T13:37:39.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="409" PostTypeId="2" ParentId="392" CreationDate="2011-12-14T14:27:43.777" Score="4" Body="&lt;p&gt;Unlike most of the people answering here (who are working in academic and/or public organizations), I am working in the commercial sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my products, the code is closed and there continue to be major business advantages to doing this. But there are of course other ways of doing it (e.g. as demonstrated by MySQL amongst others). I often see the LGPL + commercial license approach for libraries. I have yet to use such a library in a commercial system, but I would not rule it out (so far I've only used such libraries, eg. ALGLIB, at an R&amp;amp;D level). This contrasts with a GPL product - which I might use internally but would never use in a product, principally due to the viral nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I do release source code (how-to samples, free programs, etc), I typically use the Berkeley license. This seems to be much more in the spirit of &quot;free&quot; code, with attribution but without the GPL strings and politics. Perhaps this is why it (or similar licenses such as the MIT license) are so popular with universities and public organizations. Source code is given away in the true meaning of 'free' (here's some code, do what you want with it) but the author still gets credit/attribution.&lt;/p&gt;&#xA;" OwnerUserId="74" LastEditorUserId="74" LastEditDate="2011-12-17T18:26:34.143" LastActivityDate="2011-12-17T18:26:34.143" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="410" PostTypeId="2" ParentId="399" CreationDate="2011-12-14T15:10:19.147" Score="17" Body="&lt;p&gt;I must give the curmudgeon answer. My productivity has never been improved by any of the suggestions above. They are slow and expensive compared to my preferred option in parallel: one gdb session per process. Each gdb can connect to an MPI process and sit in an xterm (this happens automatically in PETSc using &lt;code&gt;-start_in_debugger&lt;/code&gt;). I have used this for 15 years, happily. Objections:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I can't look at global data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since MPI is a shared-none model, there is no global data, only local data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) This strategy does not scale to lots of processes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither do bugs. Bugs happen on individual processes, maybe with input from 1 or 2 neighbors. YOu can easily spawn gdb only on the participating processes (in PETSc you use &lt;code&gt;-debugger_nodes 0,5,17&lt;/code&gt; for example). Also, the above systems give up a lot when the run on every process, which makes them slow. The gdb method is, in fact, much more scalable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gdb is also very portable. It runs everywhere, understands C++ and Fortran, and allows you to execute arbitrary code inside the run. I have written special functions to easily display data when running in it.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="21" LastEditDate="2011-12-14T21:12:18.757" LastActivityDate="2011-12-14T21:12:18.757" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="411" PostTypeId="2" ParentId="399" CreationDate="2011-12-14T15:27:49.827" Score="-1" Body="&lt;p&gt;Here's a digest of some answers given to me previously :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenMP has timing functions: &lt;code&gt;omp_get_wtime()&lt;/code&gt; and &lt;code&gt;omp_get_wtick()&lt;/code&gt; -- &lt;a href=&quot;http://gcc.gnu.org/onlinedocs/libgomp/Runtime-Library-Routines.html#Runtime-Library-Routines&quot; rel=&quot;nofollow&quot;&gt;online docs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Google has a &lt;a href=&quot;http://code.google.com/p/google-perftools/&quot; rel=&quot;nofollow&quot;&gt;CPU profiler&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's &lt;a href=&quot;http://www.scalasca.org/&quot; rel=&quot;nofollow&quot;&gt;Scalasca&lt;/a&gt; that does OpenMP and MPI profile and analysis&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there's &lt;a href=&quot;http://www.cs.uoregon.edu/Research/tau/news.php&quot; rel=&quot;nofollow&quot;&gt;Tau&lt;/a&gt; and &lt;a href=&quot;http://software.intel.com/en-us/articles/intel-vtune-amplifier-xe/&quot; rel=&quot;nofollow&quot;&gt;vtune&lt;/a&gt; which I haven't used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="367" LastActivityDate="2011-12-14T15:27:49.827" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="412" PostTypeId="2" ParentId="265" CreationDate="2011-12-14T19:44:00.077" Score="6" Body="&lt;p&gt;I work for a company that produces large, highly technical applications in Fortran with IFort - the applications are very performance critical, since the speed (while maintaining accuracy) is the main selling point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We always compile with -O3, and to my knowledge, the main things that -O3 do are loop unrolling and prefetching. I suspect that most scientific computing applications benefit greatly from these, in particular the loop optimizations. We haven't seen compiler errors in quite some time (though we've seen a few linker errors when using Ifort on a Mac). We use 11.1.095.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="370" LastActivityDate="2011-12-14T19:44:00.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="413" PostTypeId="1" CreationDate="2011-12-14T19:48:49.383" Score="2" ViewCount="661" Body="&lt;p&gt;To do some calculation on QCD on Lattice, I needed a Monte Carlo multi integration. I wrote a a C++ program according to what I understood from the paper “ Lepage (1978) “  A New Algorith For Adaptive Multidimensional Integration” SLAC_Pub_1839 November 1976” The results I got, however, were not accurate, so I copied the written program from the books: a) Numeerical Recipies in C and  b) Numeerical Recipies in C  Examples. The program is running for 4 days on My PC with Intel Core 2 Due, @ 2.9 GHz. My question is from anyone who has experience on Vegas algorithm that this long running is normal or my program is in a non-ending loop. Thanks.&lt;/p&gt;&#xA;" OwnerDisplayName="Matrix" LastEditorUserId="16" LastEditDate="2011-12-14T22:56:08.943" LastActivityDate="2011-12-14T22:56:08.943" Title="Vegas, Monte Carlo multi integration, QCD" Tags="|monte-carlo|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="414" PostTypeId="2" ParentId="350" CreationDate="2011-12-14T20:07:14.263" Score="2" Body="&lt;p&gt;I think that several people here have provided valuable tips for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For future reference, however, your question of how to solve an ill-posed linear least squares problem could be answered by looking at the immense body of litterature on this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, you could use the TSVD (Truncated Singular Value Decomposition) as a simple method of obtaining the solution:&#xA;$$&#xD;&#xA;\mathbf{x}_k = \sum_{i=1}^{k} \frac{\mathbf{u}_i^H \mathbf{b}}{\sigma_i}\mathbf{v}_i  &#xD;&#xA;$$&#xA;where $\sigma_i$ is the i'th singular value, $\mathbf{u}_i$ and $\mathbf{v}_i$ are the i'th column in the matrices $U$ and $V$ from the factorization $USV^H=A$, $\mathbf{b}$ is the right-hand side of your problem, and the notation $\mathbf{u}^H$ means the complex conjugate of the entries and then turned to a row vector, such that $\mathbf{u}^H \mathbf{b}$ yields a scalar (dotproduct). Your solution is thus the vector $\mathbf{x}_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problem in this setting, aside from being forced to compute the SVD which is quite expensive, is how to choose the number of singular values to use, i.e. $k$. Again, there are a number of ways to do that, but the most popular ones would be the Discrepancy principle, the Generalized Cross-Validation method, and the L-curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this (and much more) is implemented, in Matlab, in the excellent toolbox &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/52&quot; rel=&quot;nofollow&quot;&gt;Regularization Tools&lt;/a&gt;, written by Prof. Per Christian Hansen, who has also published several papers and a few books on inverse problems. The toolbox is easy to use and should be quite easy to translate to other programming languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, while others have provided important insights on your application that suggest other approaches are more appropriate, the above is a quick summary on how you could solve the problem if you still need to.&lt;/p&gt;&#xA;" OwnerUserId="370" LastActivityDate="2011-12-14T20:07:14.263" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="417" PostTypeId="2" ParentId="413" CreationDate="2011-12-14T21:32:08.803" Score="2" Body="&lt;p&gt;I used VEGAS some time ago to calculate the hadronic cross section using partonic cross section and parton distributions and none of my calculations lasted that long. Around ~10-20min at most. I would go with the precompiled CUBA libraries. There are four different algorithms Cuhre, Divonne, Suave and Vegas, consult with the manual/paper for more information and then choose the one fits best at your needs.&lt;/p&gt;&#xA;" OwnerUserId="375" LastActivityDate="2011-12-14T21:32:08.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="418" PostTypeId="2" ParentId="46" CreationDate="2011-12-15T00:13:23.133" Score="3" Body="&lt;p&gt;Alan Genz proposed a test suite of functions in the paper &lt;em&gt;Testing multidimensional integration routines&lt;/em&gt;. I cannot find an online version of this paper, but references to it can be found in the &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0010465505000792&quot; rel=&quot;nofollow&quot;&gt;papers about the CUBA library&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="371" LastActivityDate="2011-12-15T00:13:23.133" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="419" PostTypeId="2" ParentId="143" CreationDate="2011-12-15T00:32:11.107" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Low-discrepancy_sequence#The_Koksma.E2.80.93Hlawka_inequality&quot; rel=&quot;nofollow&quot;&gt;Koksma-Hlawka Inequality&lt;/a&gt; states that if $f$ has bounded variation $V(f)$ in the sense of Hardy and Krause on the unit hypercube $I^s$, then the error bound is given by the product of $V(f)$ and the star-discrepancy of the sequence used. That the error depends on only the discrepancy yields two conclusions. First, choose the sequence with lowest discrepancy. Second, the regularity of the integrand plays no role in this bound and so in practice it's not very useful. To incorporate more information about the integrand into your error analysis, you have to either perform experimental comparisons between sequences or consider other approaches besides basic quasi-Monte Carlo.&lt;/p&gt;&#xA;" OwnerUserId="371" LastActivityDate="2011-12-15T00:32:11.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="420" PostTypeId="2" ParentId="46" CreationDate="2011-12-15T01:10:06.957" Score="7" Body="&lt;p&gt;The &lt;a href=&quot;http://www.cuter.rl.ac.uk/&quot;&gt;CUTEr&lt;/a&gt; web site updates the CUTE test set mentioned on Arnold Neumaier's web site with some additional problems for optimization and linear solvers. In addition, it provides software tools for the testing and updating of linear algebra and optimization solvers.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2011-12-15T01:10:06.957" CommentCount="0" CommunityOwnedDate="2012-01-13T09:12:04.030" ContentLicense="CC BY-SA 3.0" />
  <row Id="421" PostTypeId="1" AcceptedAnswerId="424" CreationDate="2011-12-15T01:21:29.393" Score="8" ViewCount="5349" Body="&lt;p&gt;I hope this is on topic, I found this through the proposal here:&#xA;&lt;a href=&quot;https://area51.meta.stackexchange.com/questions/320/shall-we-unite-computational-science-proposals&quot;&gt;https://area51.meta.stackexchange.com/questions/320/shall-we-unite-computational-science-proposals&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good visual description of BOW:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/gJOpr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't quite understand the point of some of the steps here however. I know feature extraction gets some form of unique features from an image. But that's it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Encoding seems like some form of kd-tree manipulation, but it seems like the most common method is histogram encoding. Just with the encoding stage I can get to object recognition, because with encoding I can directly search match feature columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the standard model goes on, and here is where I get completely lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no idea what pooling is. But the pooling step I'm interested in is max or sum pooling using pyramid match kernel. What is the point of pooling?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, with classification, some form of SVM is used to compare the input image classifier with the rest of the classifiers. I sort of understand this process, but it doesn't seem necessary unless I need to find what class an object belongs to, rather than recognizing an object itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So ignoring the classification part, how does encoding and pooling work? Why do I need to pool at all if I could achieve object recognition just by extracting features and comparing it with a database of feature converted to a kd-tree?&lt;/p&gt;&#xA;" OwnerUserId="376" LastEditorUserId="-1" LastEditDate="2017-05-03T08:41:07.100" LastActivityDate="2011-12-16T01:36:40.547" Title="How does each process in bag of visual words image classification work?" Tags="|computer-vision|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="422" PostTypeId="1" AcceptedAnswerId="425" CreationDate="2011-12-15T02:02:09.840" Score="17" ViewCount="3072" Body="&lt;p&gt;My thesis is on developing numerical methods for model reduction in combustion. I run my methods purely on the chemistry part of combustion simulations, and I have plenty of case studies for 0-D simulations (no flow). What I would like is to run simulations that do have flow in them, preferably 2-D or 3-D simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These simulations would need to be in parallel, due to the high computational requirements. I'd also need something that can interface with chemistry solvers like Chemkin or Cantera, for which I have the source code. (Chemkin is in Fortran 77, and Cantera is in C++.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the ideal case, I could specify a flow pattern using the basic knowledge of fluid mechanics I have from my grad program and some CFD package, add the chemistry, and run it. If I have to, I can set up the equations governing the fluid motion and chemistry for a simple case study based on an experimental setup used by a former collaborator, but I'd very much prefer not to roll my own CFD code unless there were a package or packages that made it extremely easy to do it. I'd be willing to spend 2-3 weeks on it; I don't know if this requirement rules out PETSc or Trilinos. If I have to spend any longer on it, I'd rather put it off until later, because I have a collaborator supplying a CFD code for case studies as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any experience using a CFD package or writing CFD code, and if so, can you recommend one? One thing I know I'd like to use is Strang splitting, but I'm not a CFD or PDE expert; I study the chemistry and the numerical methods for the model reduction. Also, please comment on how long it took you to get up to speed using the software you recommend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@FrenchKheldar makes a good point that I should mention the characteristics of problems I'd like to solve:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ideal (perfect) gas, single-phase&lt;/li&gt;&#xA;&lt;li&gt;Compressible&lt;/li&gt;&#xA;&lt;li&gt;Laminar flow is essential; turbulent flow is a plus. (I know a little bit about turbulence from previous work in numerical methods in CFD, but I haven't worked on CFD solvers; I just know a little about physics.)&lt;/li&gt;&#xA;&lt;li&gt;Zero-Mach-number formulation is okay (I don't care about shocks or supersonic flow)&lt;/li&gt;&#xA;&lt;li&gt;Combustion chemistry, ignoring Soret and Dufour fluxes, and treating diffusion as Fickian&lt;/li&gt;&#xA;&lt;li&gt;Geometry can be something simple&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I can write interfacing code, though the less I have to write, the better; @FrenchKheldar also points out that Cantera has Fortran and Python bindings. I use the Cantera Python bindings right now for rapid prototyping, so I'm comfortable with those also.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2011-12-15T19:37:07.907" LastActivityDate="2015-10-08T16:31:59.447" Title="Is there a good, easy-to-use, high quality open source CFD solver out there?" Tags="|pde|fluid-dynamics|software|parallel-computing|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="423" PostTypeId="2" ParentId="8" CreationDate="2011-12-15T03:46:20.117" Score="1" Body="&lt;p&gt;PDB files can be read and manipulated easily in C++ using the open-source (BSD licensed) &lt;a href=&quot;http://www.chemkit.org&quot; rel=&quot;nofollow&quot;&gt;chemkit&lt;/a&gt; library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example here is some code that will read in a PDB file for hemoglobin (PDB ID: &lt;a href=&quot;http://www.rcsb.org/pdb/explore/explore.do?structureId=2DHB&quot; rel=&quot;nofollow&quot;&gt;2DHB&lt;/a&gt;) and print out the total number of atoms, number of chains, and solvent accessible surface area:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;#include &amp;lt;chemkit/polymer.h&amp;gt;&#xA;#include &amp;lt;chemkit/polymerfile.h&amp;gt;&#xA;#include &amp;lt;chemkit/molecularsurface.h&amp;gt;&#xA;&#xA;using namespace chemkit;&#xA;&#xA;int main()&#xA;{&#xA;    PolymerFile file(&quot;2DHB.pdb&quot;);&#xA;    bool ok = file.read();&#xA;    if(!ok){&#xA;        std::cerr &amp;lt;&amp;lt; &quot;Failed to read file: &quot; &amp;lt;&amp;lt; file.errorString() &amp;lt;&amp;lt; std::endl;&#xA;        return -1;&#xA;    }&#xA;&#xA;    Polymer *protein = file.polymer();&#xA;    if(!protein){&#xA;        std::cerr &amp;lt;&amp;lt; &quot;File is empty.&quot; &amp;lt;&amp;lt; std::endl;&#xA;        return -1;&#xA;    }&#xA;&#xA;    std::cout &amp;lt;&amp;lt; &quot;Number of Atoms: &quot; &amp;lt;&amp;lt; protein-&amp;gt;atomCount() &amp;lt;&amp;lt; std::endl;&#xA;    std::cout &amp;lt;&amp;lt; &quot;Number of Chains: &quot; &amp;lt;&amp;lt; protein-&amp;gt;chainCount() &amp;lt;&amp;lt; std::endl;&#xA;&#xA;    MolecularSurface surface(protein);&#xA;    surface.setSurfaceType(MolecularSurface::SolventAccessible);&#xA;&#xA;    std::cout &amp;lt;&amp;lt; &quot;Surface Area: &quot; &amp;lt;&amp;lt; surface.surfaceArea() &amp;lt;&amp;lt; &quot; A^2&quot; &amp;lt;&amp;lt; std::endl;&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Number of Atoms: 2201&#xA;Number of Chains: 2&#xA;Surface Area: 14791 A^2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For more information see the documentation for the classes presented above:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://doc.chemkit.org/api/classchemkit_1_1_polymer.html&quot; rel=&quot;nofollow&quot;&gt;Polymer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://doc.chemkit.org/api/classchemkit_1_1io_1_1_polymer_file.html&quot; rel=&quot;nofollow&quot;&gt;PolymerFile&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://doc.chemkit.org/api/classchemkit_1_1_molecular_surface.html&quot; rel=&quot;nofollow&quot;&gt;MolecularSurface&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="377" LastActivityDate="2011-12-15T03:46:20.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="424" PostTypeId="2" ParentId="421" CreationDate="2011-12-15T04:38:52.340" Score="8" Body="&lt;p&gt;There are several ways to implmenet a BoW (Bag of Words) model, but generally speaking:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Encoding&lt;/strong&gt; is the quantization of the image patches that constitute the image or object to be classified. Basic encoding schemes work by first running K-means on the set of all (e.g. 3x3) pixel patches that you collect across multiple instances of your images/objects during a training/pre-processing step. This builds what is known a &lt;strong&gt;dictionary&lt;/strong&gt; represented by the centroids obtained from the clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of this process, you end up with K representative &quot;visual  words&quot; (the centroid of each cluster after K means ends) of 3x3 patches. These &quot;visual words&quot; represent what is usually understood as your visual dictionary. Once you have these visual words, encoding is the process of assigning each 3x3 patch within your image/object the closest 3x3 word (&lt;strong&gt;nearest neighbor&lt;/strong&gt;) in the dictionary. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Pooling&lt;/strong&gt; refers to the process of representing an image (or the object to be classified) as a &quot;bag of words&quot;. The word bag here is meant to convey that once you have encoded each patch with a word (a number between 1 and K), you build a new representation (a bag) that discards the spatial relationship between the words that constitute your image or object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This representation is often a histogram or a collection of spatially adjacent histograms of the 3x3 words (i.e. histograms of values 1 to K) that together form your object or image. &quot;Pooling&quot; is thus the process of building a histogram of words (i.e. pooling ~ &lt;strong&gt;&quot;sampling&quot; words&lt;/strong&gt; from the image to build a probability mass function of words)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify, the &lt;strong&gt;purpose&lt;/strong&gt; of pooling is two fold: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;By building a feature vector that is a histogram of words (as opposed to putting the full &quot;sentence of words&quot; in the feature vector), your descriptor will be invariant to changes in &quot;the ordering of words&quot;. In computer vision this translates into &lt;strong&gt;invariance&lt;/strong&gt; with respect to rotations and distortions of the image and object, which is a desirable thing to have.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the dictionary is small compared to the length of the sentence, a histogram of words has less dimensions than the original vector. &lt;strong&gt;Less dimensions&lt;/strong&gt; makes &lt;a href=&quot;http://en.wikipedia.org/wiki/Curse_of_dimensionality&quot; rel=&quot;noreferrer&quot;&gt;learning (training) much easier&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Classification&lt;/strong&gt; refers to the process of classifying your image/object now represented by a &quot;bag of words&quot;. Following the example above, the feature vector here would be the histogram (or histograms) of words (3x3 patches) for your object or image.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The above implies that your working word is an image patch (in the example I used, a 3x3 pixel patch). But nothing prevents you from defining words that correspond to other visual cues or descriptors such as SIFT feature vectors (which is the case in the diagram you posted).&lt;/p&gt;&#xA;" OwnerUserId="13" LastEditorUserId="13" LastEditDate="2011-12-16T01:36:40.547" LastActivityDate="2011-12-16T01:36:40.547" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="425" PostTypeId="2" ParentId="422" CreationDate="2011-12-15T08:13:09.383" Score="12" Body="&lt;p&gt;I'm a heavy user of &lt;a href=&quot;http://www.openfoam.com&quot; rel=&quot;noreferrer&quot;&gt;OpenFOAM&lt;/a&gt;, so naturally I'd recommend it. It has a large amount of &lt;a href=&quot;http://www.openfoam.com/features/&quot; rel=&quot;noreferrer&quot;&gt;features&lt;/a&gt; including combustion models (though not necessarily precisely what you need) and has been used together with &lt;a href=&quot;http://www.cfd-online.com/Forums/openfoam-solving/64841-cantera.html&quot; rel=&quot;noreferrer&quot;&gt;Canterra&lt;/a&gt; by other people. If you need a solver for a specific equation that hasn't been implemented yet, you can pretty much literally write your &lt;a href=&quot;http://www.openfoam.com/features/creating-solvers.php&quot; rel=&quot;noreferrer&quot;&gt;equations&lt;/a&gt;. I don't know what strang-splitting is (I don't work on combustion myself), but other people have used it in &lt;a href=&quot;http://www.combustion-institute.it/proceedings/XXXIV-ASICI/papers/34proci2011.I19.pdf&quot; rel=&quot;noreferrer&quot;&gt;OpenFOAM&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a limited amount of tutorials. The ones documented in the user guide do not include chemistry. There are some sample cases for the reaction solvers you can have a look at. Your best bet is to check out the OS-CFD &lt;a href=&quot;http://www.tfd.chalmers.se/~hani/kurser/OS_CFD/&quot; rel=&quot;noreferrer&quot;&gt;grad course website&lt;/a&gt; at Chalmers (link is for 2011, but it contains a link to the pages of previous years). The students there often document the solvers they are working on, e.g. &lt;a href=&quot;http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2010/ehsanYasari/ehsanYasariReport.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the effort to learn it: If you get by using one of the pre-defined models, it's fairly straightforward to use and you should be able to have results within weeks. If you have to descend beneath the top level (e.g. to add a new ODE solver), things can get more difficult quite fast and you'll appreciate to know C++.&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="59" LastEditDate="2013-11-10T14:27:43.980" LastActivityDate="2013-11-10T14:27:43.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="426" PostTypeId="2" ParentId="143" CreationDate="2011-12-15T10:18:21.407" Score="1" Body="&lt;p&gt;Each LDS sequence has specific advantages and caveats and discussing them would fill many books (check Lemieux's one for an introduction), however in general the most versatile one proved to be Sobol' sequence (make sure to use the latest parameters from Joe&amp;amp;Kuo). &#xA;All others always require in practice significant additional modifications to both integrand and LDS points (e.g. scrambling) w.r.t. to the basic formulations. &#xA;Sobol' also has many issues, but is flexible enough to be used in most sufficiently smooth integrations at low dimensions.&#xA;Anyway remember to check your QMC solution against a (slower) MC one: for low (reasonable) sample sizes QMC could return biased estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also more advanced LDS constructions for adapting to a given problems, but little SW available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some information on your specific task would be helpful to focus an answer.&lt;/p&gt;&#xA;" OwnerUserId="382" LastActivityDate="2011-12-15T10:18:21.407" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="427" PostTypeId="1" CreationDate="2011-12-15T12:07:23.783" Score="6" ViewCount="117" Body="&lt;p&gt;The cluster I work on has a filesystem labelled as &quot;fast&quot; with no further explanation. But the filesystem type is &quot;lustre&quot;, which strongly hints at a &lt;a href=&quot;http://www.lustre.org/&quot;&gt;Lustre&lt;/a&gt; filesystem. While general information about this file system type is easy to find, I have not been able to find any information about how to get the most performance from it, playing with the parameters that a programmer can work on: file layout and access patterns. Does anyone have experience with this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I expect replies of the kind &quot;just run benchmarks&quot;, that's of course what I did. I find that read access is sometimes dramatically faster then the NFS filesystem on the same machine, but sometimes clearly slower, although I don't see any significant difference between the two applications that gave these extreme results. That's why I'd like to understand the performance issues rather than just have numbers.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2011-12-15T16:29:46.230" Title="Performance of Lustre filesystems" Tags="|io|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="428" PostTypeId="2" ParentId="427" CreationDate="2011-12-15T12:37:15.847" Score="6" Body="&lt;p&gt;This is kind of an open question. Is the performance difference always reproducible? It's possible someone else was really beating up the filesystem during one of your tests. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lustre has some issues with locking that can cause performance problems if you're not careful. We'd need to know a lot more about the way the filesystem hardware and software are set up in order to begin to diagnose your problem. We'd also need to know a lot more about the two applications and their I/O patterns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presumably you've talked to the folks that run the computer you have access to, and they haven't been too helpful. If not, you'll really want to talk to them first.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2011-12-15T12:37:15.847" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="429" PostTypeId="1" CreationDate="2011-12-15T14:34:13.350" Score="5" ViewCount="281" Body="&lt;p&gt;What is the convergence rate for Monte-Carlo variance estimate for a random variable $X \in {L^q}(\Omega ,R),2 &amp;lt; q &amp;lt; 4$?&lt;/p&gt;&#xA;" OwnerUserId="353" LastActivityDate="2011-12-15T14:34:13.350" Title="Convergence rate of Monte-Carlo variance estimate" Tags="|monte-carlo|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="430" PostTypeId="1" CreationDate="2011-12-15T15:10:26.630" Score="4" ViewCount="436" Body="&lt;p&gt;I am trying to fit some raw data using a function of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(r) = \sum_{i=1}^{K} d_kS_k(n_k,\alpha_k,r)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S_k(n_k,\alpha_k,r) = \frac{\alpha_k ^{n_k+3}}{(n_k+2)!}r^{n_k}\exp(-\alpha_kr)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(note that the actual fit function is far more complex).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for each $k$, this has two non-linear parameters ($\alpha_k$ and $n_k$) and one linear parameters ($d_k$). I am considering various methods for non-linear optimization (genetic algorithms, conjugate gradient method, etc.) but I am wondering how I should work with the linear functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I include them in the non-linear optimization (use them as a variables when I take the gradient for CGM, etc)? Should I optimize the non-linear parameters &lt;em&gt;completely&lt;/em&gt; on $\alpha$ and $n$ and &lt;em&gt;then&lt;/em&gt; perform a linear fit with $d$? Should I mix and match - perform one sequence of conjugate gradient steps and then perform a linear fit, rinse and repeat?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: If it helps at all, $d_k$ is a real number, $n_k$ is a positive integer, $\alpha_k$ is a positive real.&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="127" LastEditDate="2011-12-23T11:32:52.033" LastActivityDate="2011-12-23T11:32:52.033" Title="best way to optimize a function with linear/non-linear parameters" Tags="|optimization|regression|nonlinear-programming|conjugate-gradient|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="431" PostTypeId="2" ParentId="430" CreationDate="2011-12-15T16:15:27.363" Score="3" Body="&lt;p&gt;For the least-squares case anyway, one standard method is Linda Kaufman's  &lt;a href=&quot;http://dx.doi.org/10.1007/BF01932995&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;variable projection&lt;/em&gt; algorithm&lt;/a&gt; (see &lt;a href=&quot;http://maths.anu.edu.au/~mike/GeneEtna.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; as well). This is based on &lt;a href=&quot;http://dx.doi.org/10.1137/0710036&quot; rel=&quot;nofollow&quot;&gt;earlier work by Gene Golub and Victor Pereyra&lt;/a&gt; for the separable nonlinear least squares problem, in which they show how the structure of the Moore-Penrose pseudoinverse can be exploited to yield algorithms that are more efficient when there are linear parameters present. &lt;a href=&quot;http://www.netlib.org/opt/varpro&quot; rel=&quot;nofollow&quot;&gt;This&lt;/a&gt; is one FORTRAN implementation of the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(But see also &lt;a href=&quot;http://dx.doi.org/10.1137/1022057&quot; rel=&quot;nofollow&quot;&gt;this paper by Ruhe and Wedin&lt;/a&gt; for a different method.)&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2011-12-15T16:15:27.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="432" PostTypeId="2" ParentId="427" CreationDate="2011-12-15T16:29:46.230" Score="4" Body="&lt;p&gt;The problem is, there are many definitions of &lt;em&gt;fast&lt;/em&gt;. Do you need high bandwidth, low latency or both? Do you need fast write only storage for off-line processing, or do you need immediate read-back capability on another machine on the network to process that data? Do you need to access a single file or many files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where I work, we use lustre extensively for some experimental stations. Our cluster can happily cope with streams of image data that saturate multiple Gigabit Ethernet links, but ask it for a directory listing in a directory with a thousand files and it crawls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since data might be distributed over many nodes, and those nodes cache aggressively, if you schedule things correctly then you can have multiple systems cooperate on processing data and each one have faster access to the data than if they had stored it locally. Certainly we have detectors here than can generate more data per second than a local disk can sustain. For these detectors, streaming to the lustre cluster is the &lt;em&gt;only&lt;/em&gt; realistic option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For our set-up, it seems that lustre is optimised for high bandwidth - fast access to large contiguous datasets, but as a single meta-data server is shared amongst all lustre nodes for any given filesystem, normal file system activities like &lt;code&gt;ls&lt;/code&gt; and &lt;code&gt;grep&lt;/code&gt; can really take a hit. This is apparently not an issue with the most recent version of lustre, but I haven't seen that in action.&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2011-12-15T16:29:46.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="433" PostTypeId="2" ParentId="395" CreationDate="2011-12-15T18:51:36.167" Score="5" Body="&lt;p&gt;The way I'd do it would probably be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Put the molecular geometry into &lt;a href=&quot;http://avogadro.openmolecules.net/wiki/Main_Page&quot; rel=&quot;noreferrer&quot;&gt;Avogadro&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Set up the view to be exactly the way I want it&lt;/li&gt;&#xA;&lt;li&gt;Export to &lt;a href=&quot;http://www.povray.org/&quot; rel=&quot;noreferrer&quot;&gt;POVRay&lt;/a&gt;, not rendering but keeping the input file&lt;/li&gt;&#xA;&lt;li&gt;Work out which atom is which in the POVRay file&lt;/li&gt;&#xA;&lt;li&gt;Add vectors using cylinders and cones (probably using a macro to define a vector to make it easier and visually consistent)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Avogadro can also render sequences of XYZ-formatted input to animations using POVRay and ffmpeg, but it's not something I've tried since I'm using Windows at the moment and Avogadro doesn't seem to have an option to specify where the povray executable is if it's not in your path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again depending on whether you are a Python aficionado or not, you may alternatively be able to set the forces on the atoms using the Avogadro Python console and then use the force vector display in Avogadro, but that's not something I've tried either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of any perfectly convenient tool that lets you put in vibrational parameters directly and visualize or animate them, though.&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2011-12-15T18:51:36.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="434" PostTypeId="2" ParentId="430" CreationDate="2011-12-15T19:30:54.827" Score="2" Body="&lt;p&gt;You should optimize all parameters simultaneously if you would like to obtain a (local epsilon-)optimal solution to the optimization problem corresponding to the fit you'd like to perform. J. M.'s suggestion is a good one for the least-squares case, but if the specifics of your fit aren't quite least-squares, the general strategy of fitting all parameters simultaneously will yield a better fit than optimizing groups of parameters separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $n_k$ is an integer, you'll want to use a mixed-integer nonlinear programming (MINLP) solver (or algorithm). Including an integral in your objective in an MINLP solver will be difficult unless you can evaluate it numerically in closed form, which will limit you to fixed quadrature. MINLP solvers require that problems be posed explicitly as equations, not as algorithms. (It is possible to optimize a problem posed as part of an algorithm using the algorithms in &lt;a href=&quot;http://link.aip.org/link/?SJE/20/573/1&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;, but the class of algorithms supported does not include algorithms that use adaptive quadrature.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2011-12-15T19:30:54.827" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="435" PostTypeId="1" AcceptedAnswerId="446" CreationDate="2011-12-15T20:40:46.883" Score="17" ViewCount="1457" Body="&lt;p&gt;I'm planning to use multigrid to calulate some eigenvalues and vectors, and I noticed PETSc has high-level support for multigrid.  The PETSc documentation says that this part of PETSc should not be used, as it is being replaced soon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which other libraries have high-level support for multigrid, and roughly how soon will PETSc release the new multigrid support?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2011-12-15T21:34:51.127" LastActivityDate="2022-12-04T04:36:20.707" Title="Which libraries have good high-level support for multigrid?" Tags="|petsc|libraries|multigrid|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="436" PostTypeId="2" ParentId="435" CreationDate="2011-12-15T20:55:31.097" Score="11" Body="&lt;p&gt;PETSc multigrid (as a preconditioner) is quite mature and may be used with any of the KSP (iterative Krylov method) solvers in PETSc by typing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-pc_type mg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this requires that you have some way of generating your coarse levels, such as having structured grids defined by PETSc DA objects, which will be coarsened automatically.&lt;/p&gt;&#xA;&lt;p&gt;Or, if you want to use algebraic multigrid from the HYPRE package, you can use&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-pc_type hypre&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or from the ML package with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-pc_type ml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are downloaded during the configuration process by appending&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--download-hypre=1 --download-ml=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to your &lt;code&gt;./configure&lt;/code&gt; command line.&lt;/p&gt;&#xA;&lt;p&gt;The part that is depreciated (for now) is the DMMG framework, which is being replaced by the SNES (nonlinear) FAS solver and better support for handling multilevel discretizations while using either MG or FAS as we speak.  The other replacement (for linear problems) is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-pc_type gamg -pc_gamg_type sa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is a newer code, native in PETSc, highly scalable smoothed-aggregation algebraic multigrid.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="20688" LastEditDate="2022-12-04T04:36:20.707" LastActivityDate="2022-12-04T04:36:20.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="437" PostTypeId="2" ParentId="422" CreationDate="2011-12-15T21:06:14.430" Score="3" Body="&lt;p&gt;You could use &lt;a href=&quot;http://numerics.kaust.edu.sa/pyclaw/&quot; rel=&quot;nofollow&quot;&gt;PyClaw&lt;/a&gt;, a parallel extension of &lt;a href=&quot;http://www.clawpack.org&quot; rel=&quot;nofollow&quot;&gt;Clawpack&lt;/a&gt; (note: I am one of PyClaw's main developers).  It includes 2D and 3D solvers for the inviscid Euler equations (compressible flow) of an ideal gas.  It also has Strang splitting built in, but you need to add the evaluation of viscous terms and chemistry yourself.  It should be straightforward to interface with Chemkin and Cantera, since PyClaw is written in Python and already includes both Fortran 77 and C code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PyClaw is relatively new (though the underlying Clawpack code is fairly old) and thus not as established as something like OpenFOAM.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2011-12-15T21:06:14.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="438" PostTypeId="1" AcceptedAnswerId="439" CreationDate="2011-12-15T16:54:15.223" Score="7" ViewCount="275" Body="&lt;p&gt;I'm preparing for a thesis in computer science on calculations based on concepts from &lt;a href=&quot;http://en.wikipedia.org/wiki/NeuroElectroDynamics&quot; rel=&quot;nofollow&quot;&gt;Neuroelectrodynamics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short this theory states, that information transfer is not done by spike time coding, but by movement of electric charges. Also this charges affect (and get affected by) proteins in the brain cells, so the cells can act as a kind of memory&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can electric charges be represented in terms of programming? Do you have to simulate each and every ion or are there overall principles so you can calculate with &quot;whole&quot; charges? I could imagine Circuit simulations already implement some calculations with electrodynamics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume there are already computation systems, but I didn't know what to look for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I'm quite new to the field of electrodynamic. Which book can you recommend on this topic?&lt;/p&gt;&#xA;" OwnerDisplayName="user390" LastEditorUserId="127" LastEditDate="2013-05-26T16:51:41.403" LastActivityDate="2013-05-26T16:51:41.403" Title="Representing charges in computer programming" Tags="|simulation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="439" PostTypeId="2" ParentId="438" CreationDate="2011-12-15T22:50:08.570" Score="7" Body="&lt;p&gt;As a general rule, when dealing with &quot;big&quot; things like cells, you shouldn't simulate the individual charged particles.  A useful abstraction is &lt;em&gt;charge density&lt;/em&gt;, the charge per unit volume, usually denoted $\rho$. The equations of electrodynamics can be formulated in terms of this charge density $\rho$ and the current density ${\bf J}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Electrodynamics is a big topic, and it can take a while to learn.  If you want to study electrodynamics at an undergraduate level, you could try &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/8120316010&quot; rel=&quot;nofollow&quot;&gt;Griffiths&lt;/a&gt;. At a graduate level, &lt;a href=&quot;http://www.powells.com/biblio/62-9780471309321-1&quot; rel=&quot;nofollow&quot;&gt;Jackson's&lt;/a&gt; book is the standard. However, if you're unfamiliar with charge density, you might want to look at a freshman-level calculus-based physics book like &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/143904838X&quot; rel=&quot;nofollow&quot;&gt;Serway&lt;/a&gt; or &lt;a href=&quot;http://www.abebooks.com/servlet/BookDetailsPL?bi=6488630865&amp;amp;searchurl=an%3Dhalliday%2Bresnick%2Bwalker%26sts%3Dt%26x%3D0%26y%3D0&quot; rel=&quot;nofollow&quot;&gt;Halliday, Resnick and Walker&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-03T23:43:39.507" LastActivityDate="2012-01-03T23:43:39.507" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="440" PostTypeId="2" ParentId="217" CreationDate="2011-12-16T03:38:46.360" Score="3" Body="&lt;p&gt;In addition to labeling all edges and faces there are also other questions I have found important in the same context:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Numbering them in some particular way. deal.II uses a numbering where we first consider&#xA;all edges that are parallel to the x-axis and sort them by their y,z-values (either 0 or 1) in lexicographic order, then all edges parallel to the y-axis and numbering them by their x,z-values, etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As for the total number: In 1d, there are ne_1=1 edge. In 2d, the square is formed by replicating the 1d line twice at y=0 and y=1, making for two lines, but then we also have to connect all nv_1=2 vertices of old and replicated line, so ne_2=2*ne_1+nv_1=2+2=4. Then construct the cube by replicating the two squares and connecting all vertices, so ne_3=2*ne_2+nv_2=2*4+4=12. You get the recursion.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-16T03:38:46.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="441" PostTypeId="2" ParentId="114" CreationDate="2011-12-16T03:56:49.113" Score="7" Body="&lt;p&gt;I have always refused to even think about counting flops, memory accesses, or whatever you have. That's a concept from the 1960s when what you did was pretty much given and only how you did it was up to algorithmic optimization. Think solving a finite element problem on a uniform x-y-z mesh using either Gaussian elimination of Jacobi iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you can optimize this to hell and save a few flops, gaining 10% of run time. Or you can think about implementing a multigrid method and an optimal block preconditioner, gaining a factor of 10 in run time. This is what we should train our students to do -- think about what complex, outer algorithms can gain you over trying to find a better inner algorithm. Your boss (Keyes) has these slides on progress in MHD computations that make this very point rather obvious.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-16T03:56:49.113" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="442" PostTypeId="2" ParentId="304" CreationDate="2011-12-16T04:06:10.090" Score="23" Body="&lt;p&gt;From my 15 years of thinking about scientific software: If your code runs 25% faster because you write it in Fortran, but it takes you 4 times as long to write it (no STL, difficulty implementing complex data structures, etc), then Fortran only wins if you spend a significant fraction of your day twiddling thumbs and waiting for your computations to finish. Given that for almost all of us the most valuable thing is our own time, the conclusion is this: use the language that allows you to develop, debug and test your code the fastest, within reason ignoring that it may be slower than maybe possible if you wrote it in Fortran.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-16T04:06:10.090" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="443" PostTypeId="2" ParentId="330" CreationDate="2011-12-16T04:21:32.453" Score="15" Body="&lt;p&gt;I will take objection with almost every point &lt;a href=&quot;https://scicomp.stackexchange.com/a/334/149&quot;&gt;Faheem makes&lt;/a&gt;. Specifically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1/ &quot;I think that it is unrealistic to expect scientific developers to spend a great deal of time documenting their software&quot;. This is a prescription for a failed project that soon nobody will be able to use who does not have access to the primary developers. It is for good reason that the big scientific computing libraries (e.g. PETSc, or deal.II) have extensive documentation that runs into the 1,000s of pages or more. You can't have a sizable user community if you don't have extensive documentation. I will agree, however, that example codes need to be simple and focused on a single concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2/ &quot;the authors should consider writing a paper for publication if appropriate&quot;. That's often not possible in practice. One can write papers on concepts and algorithms, but not on interface and other design decisions. Readers of such papers will get an idea what the implementation does it; users of the implementation will need to know what functions to call, what the arguments means, etc. As a user, one can most of the time live without the former and simply consider a library as a black box, but one can not do without interface information.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-03-02T16:29:06.927" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="444" PostTypeId="1" AcceptedAnswerId="448" CreationDate="2011-12-16T04:23:56.620" Score="9" ViewCount="713" Body="&lt;p&gt;For example, I would like to numerically compute the $L^2$-norm of $\displaystyle u = \frac{1}{(x^2+y^2+z^2)^{1/3}}$ in some domain that includes zero, I tried Gauss quadrature and it fails, it is kinda far from the real $L^2$-norm on the unit ball using spherical coordinates to integrate, is there any good way to do this? This problem is often seen in the finite element computing toy problems for domains with re-entrant corners. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2011-12-16T18:33:03.270" Title="What numerical quadrature to choose to integrate a function with singularities?" Tags="|finite-element|quadrature|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="445" PostTypeId="2" ParentId="159" CreationDate="2011-12-16T04:27:02.373" Score="3" Body="&lt;p&gt;In addition to all the things already said above, I will repeat my answer from the &quot;Fortran vs C++&quot; question: The most valuable asset a programmer has is her time. Yes, external dependencies are often awkward. But spending time to re-implement, debug and test algorithms that others have already implemented is almost always stupid, and the result will also rarely be as good as code that was written by experts on a particular topic. Re-use what others have done!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-16T04:27:02.373" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="446" PostTypeId="2" ParentId="435" CreationDate="2011-12-16T04:31:48.067" Score="7" Body="&lt;p&gt;Both PETSc and Trilinos have good algebraic multigrid methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;deal.II implements geometric multigrid methods for finite element discretizations, see for example &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_16.html&quot;&gt;the step-16 tutorial program&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="123" LastEditDate="2011-12-17T05:45:50.940" LastActivityDate="2011-12-17T05:45:50.940" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="447" PostTypeId="2" ParentId="399" CreationDate="2011-12-16T04:39:47.640" Score="1" Body="&lt;p&gt;For a couple simple ways to debug parallel codes, we've collected a few answers in the deal.II FAQs in the section on debugging: &lt;a href=&quot;https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#debugging-dealii-applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#debugging-dealii-applications&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="24110" LastEditDate="2017-05-28T07:11:09.203" LastActivityDate="2017-05-28T07:11:09.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="448" PostTypeId="2" ParentId="444" CreationDate="2011-12-16T08:39:23.197" Score="8" Body="&lt;p&gt;You should be able to get accurate results with &lt;a href=&quot;http://mpmath.googlecode.com/svn/trunk/doc/build/index.html&quot;&gt;mpmath&lt;/a&gt;, a Python module for arbitrary-precision floating-point computations.  There are examples of integration with singularities in the &lt;a href=&quot;http://mpmath.googlecode.com/svn/trunk/doc/build/calculus/integration.html&quot;&gt;documentation&lt;/a&gt;.  You'll want to explicitly tell it to break up the interval:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from mpmath import *&#xA;f = lambda x,y,z: 1./(x**2+y**2+z**2)**1./3&#xA;quad(f,[-1,0,1],[-1,0,1],[-1,0,1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You may need to increase the precision (e.g. &lt;code&gt;mp.dps=30&lt;/code&gt;) and it will likely be slow, but should be quite accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try nesting calls to MATLAB's &lt;code&gt;quadgk()&lt;/code&gt;, which uses adaptive Gauss-Kronrod quadrature in 1D.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="127" LastEditDate="2011-12-16T18:33:03.270" LastActivityDate="2011-12-16T18:33:03.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="449" PostTypeId="2" ParentId="438" CreationDate="2011-12-16T09:15:23.000" Score="3" Body="&lt;p&gt;The answer depends on what you're actually dealing with. If you're studying transport of ions in channels, then a &quot;charge-by-charge&quot; view might be appropriate. However, if you're operating at a systems level, then you'd want to express your model according to conservation principles. There's no easy way to answer this without having a better sense of what it is you want to do.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2011-12-16T09:15:23.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="450" PostTypeId="2" ParentId="114" CreationDate="2011-12-16T09:21:18.137" Score="24" Body="&lt;p&gt;I don't see why one has to be the &quot;winner&quot;; this isn't a zero-sum game, where flop counts and memory access have to drown the other out. You can teach both of them, and I think they both have their uses. After all, it's hard to say that your $O(N^4)$ algorithm with $O(N)$ memory accesses is necessarily going to be faster than your $O(N \log N)$ algorithm with $O(N^2)$ accesses. It all depends on the relative costs of the different parts (that pesky prefactor that we always ignore in these analyses!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a broader perspective, I think that the analysis of algorithmic performance should be &quot;all-inclusive.&quot; If we're teaching people to be actual HPC developers and users, then they need to understand what are the costs of programming in the real world. The abstract analysis models we have don't take into account the programmer's time. We should be thinking in terms of &quot;total time to solution,&quot; rather than just flop counts and algorithmic efficiency. It makes little sense to spend three or four programmer days to rewrite a routine that will save one second of computer time per job unless you're planning on running a few million calculations. Similarly, a few days' investment to save an hour or two of compute time quickly pays off. That novel algorithm may be amazing, but if it takes months to reprogram your code to take advantage of it, is it really worth it?&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-01-13T09:26:13.197" LastActivityDate="2012-01-13T09:26:13.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="451" PostTypeId="1" AcceptedAnswerId="452" CreationDate="2011-12-16T19:57:17.093" Score="15" ViewCount="809" Body="&lt;p&gt;I know that most methods of finding approximate solutions to PDEs scale poorly with the number of dimensions, and that Monte Carlo is used for situations that call for ~100 dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are good methods for efficiently numerically solving PDEs in ~4-10 dimensions? 10-100?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any methods besides Monte Carlo that scale well with the number of dimensions?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-04-16T13:36:06.323" Title="PDEs in Many Dimensions" Tags="|pde|monte-carlo|high-dimensional|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="452" PostTypeId="2" ParentId="451" CreationDate="2011-12-16T20:54:26.217" Score="14" Body="&lt;p&gt;A more structured way of providing a basis or quadrature (which may replace MC in many cases) in multiple dimensions is that of &lt;a href=&quot;http://en.wikipedia.org/wiki/Sparse_Grids&quot;&gt;sparse grids&lt;/a&gt;, which combines some family of one dimensional rules of varying order in such a way as to have merely exponential growth in dimension, $2^d$, rather than having it be that dimension is an exponent of the resolution $N^d$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is done through what is known as a Smolyak quadrature, which combines a series of one-dimensional rules $Q^1_l$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xD;&#xA;Q^d_n = \displaystyle\sum^{n}_{l}(Q^{1}_i - Q^{1}_{i-1})\otimes Q^{d-1}_{m-i+1}&#xD;&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is equivalent to the tensor product quadrature space with the high &lt;i&gt;mixed&lt;/i&gt; orders removed from the space.  If this is done in a severe enough fashion, the complexity may be improved greatly.  However, for one to be able to do this and maintain good approximation, the regularity of the solution has to have sufficiently vanishing mixed derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sparse grids have been beaten to death by the Griebel group for things like the &lt;a href=&quot;http://wissrech.ins.uni-bonn.de/research/pub/garcke/eigen_sparse_grid.pdf&quot;&gt;Schrödinger equation in configuration space&lt;/a&gt; and &lt;a href=&quot;http://www.prudsys.de/Service/Downloads/files/sparse_grids_SIAM.pdf&quot;&gt;other&lt;/a&gt; &lt;a href=&quot;http://wissrech.ins.uni-bonn.de/research/pub/holtz/diss_holtz.pdf&quot;&gt;high dimensional&lt;/a&gt; things with pretty good results.  In application, the basis functions used may be pretty general, as long as you can nest them.  For instance, plane-waves or hierarchical bases are common.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also pretty simple to code up yourself.  From my experience, actually getting it to work for these problems, however, is very hard.  A good &lt;a href=&quot;http://www.prudsys.de/Service/Downloads/files/sparseGridTutorial.pdf&quot;&gt;tutorial&lt;/a&gt; exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For problems whose solutions live in specialized Sobolev spaces featuring derivatives that rapidly die, the sparse grid approach may potentially yield even &lt;a href=&quot;http://page.math.tu-berlin.de/~yserenta/Publikationen/papers/skript.pdf&quot;&gt;greater results&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the Acta Numerica review paper, &lt;a href=&quot;http://dx.doi.org/10.1017/S0962492911000055&quot;&gt;Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="143" LastEditDate="2011-12-23T05:00:34.150" LastActivityDate="2011-12-23T05:00:34.150" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="453" PostTypeId="1" AcceptedAnswerId="455" CreationDate="2011-12-16T22:35:52.127" Score="7" ViewCount="305" Body="&lt;p&gt;I'm working on a machine with a version of PETSc compiled by someone else. Is there a straightforward way to find out which options were used at compile time from the installation itself?  For example, is there a file that has all of the compile time options that is generated by &lt;code&gt;make&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2018-10-30T23:26:08.263" Title="How can I tell which options PETSc was compiled with?" Tags="|petsc|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="454" PostTypeId="2" ParentId="391" CreationDate="2011-12-16T22:40:34.070" Score="2" Body="&lt;p&gt;Yes, it matters.  The usual two things to be concerned about:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Accuracy.  Some ODE schemes are more accurate than others, higher order, and so on.  The rule of thumb is to choose a method with order of accuracy similar to your spatial discretization.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stability.  For hyperbolic problems you expect the operator to have pure imaginary eigenvalues, so you want an ODE solver which includes some part of the imaginary access in its stability domain.  See for example Appendix G in Fornberg, A Practical Guide to Pseudospectral Methods.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With hyperbolic equations, some people want to insure that their solutions are always positive, so there are various kinds of filters and tricks to insure this.  But I know almost nothing about this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am far from an expert, but I thought I'd try to answer since the question has been here for a while.&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2011-12-16T22:40:34.070" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="455" PostTypeId="2" ParentId="453" CreationDate="2011-12-16T22:52:18.067" Score="9" Body="&lt;p&gt;There are a few ways to do this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Check &lt;code&gt;$PETSC_ARCH/conf/reconfigure-$PETSC_ARCH.py&lt;/code&gt; (where &lt;code&gt;$PETSC_ARCH&lt;/code&gt; is expanded, i.e. for me that is 'arch-c')&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Depending on which information you want, you could use &lt;code&gt;make getlinklibs&lt;/code&gt; or &lt;code&gt;make getincludedirs&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, you could check &lt;code&gt;$PETSC_ARCH/include/petscconf.h&lt;/code&gt; for all the standard &lt;code&gt;#ifdef&lt;/code&gt;'s that PETSc was configured with&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="142" LastActivityDate="2011-12-16T22:52:18.067" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="456" PostTypeId="2" ParentId="366" CreationDate="2011-12-16T23:28:11.690" Score="5" Body="&lt;p&gt;I think OpenCL is currently suffering from a lack of a &quot;champion&quot;. For example, if you visit the NVIDIA site right now (12/16/2011), you've got several &quot;Ken Burns Effect&quot; style shots on the splash page focusing on the scientific/industrial side of GPU computing, and ~1/4th of your navigation options point you toward things that will probably end up at CUDA. Manufacturers selling &quot;GPU computing&quot; servers and workstations are selling NVIDIA solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Competing offers from ATI are mixed in with the general AMD site, harder to find, and not as heavily featured in third-party solutions. Those solutions, and the ability to do OpenCL based programming certainly exist, but it's left a perception - at least in my mind, but in the minds of some other folks I've talked to - that the OpenCL platform's big corporate sponsors have already &quot;quit the field&quot;. People using OS X for example, are all probably too busy speculating about whether or not a Apple workstation will even exist in a year to have faith in them pushing OpenCL GPU computing.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2011-12-16T23:28:11.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="457" PostTypeId="2" ParentId="338" CreationDate="2011-12-16T23:29:41.043" Score="9" Body="&lt;p&gt;I would warmly recommend to use &lt;a href=&quot;http://sourceforge.net/projects/stochkit/&quot;&gt;StochKit&lt;/a&gt;, which is really extremely reliable and efficient. It implements the Gillespie's direct method as well as its most popular optimized variants. Furthermore, it has a built-in mechanism for selecting the appropriate simulation scheme depending on your network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is written in C++. If you really need to integrate it with a Python, that shouldn't be a problem. I use it extensively in MATLAB scripts (basically, MATLAB writes a descriptor of the simulation, calls StochKit, and finally performs the data analysis).  I'm sure you could do pretty much the same thing with NumPy or SciPy.&lt;/p&gt;&#xA;" OwnerUserId="76" LastActivityDate="2011-12-16T23:29:41.043" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="458" PostTypeId="2" ParentId="366" CreationDate="2011-12-17T03:54:53.780" Score="16" Body="&lt;p&gt;OpenCL vs what?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the question is OpenCL vs CUDA, I see a lot of hand-wringing over this question, and it seems crazy to me.  It doesn't matter.   Honest.   The kernels - where all the hard thinking has to go - are practically identical between the two languages; you could write macros for your favourite editor to do 99% of the work to bounce between OpenCL and CUDA.   It has to be that way; they're low-level control of ultimately pretty similar sorts of hardware.    Once you've figured out how to write your important kernels in {OpenCL,CUDA}, it's trivial to port them to {CUDA,OpenCL}.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boilerplate host code you have to write is similar, too, but CUDA keeps simple cases simple.   That's why we teach CUDA in our centre; you can leap straight into writing kernel code, whereas we'd have to spend 1-2 hours of our daylong course just explaining the kernel launching stuff for OpenCL.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even there the difference isn't that important; once you start doing more complicated thing (asynchronous kernels on multiple gpus), they're both equally complicated and again you can pretty much do a line-by-line translation from one to the other.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's OpenCL vs the directive-based approaches -- &lt;a href=&quot;http://www.openacc-standard.org/&quot;&gt;OpenACC&lt;/a&gt; or &lt;a href=&quot;http://www.caps-entreprise.com/hmpp.html&quot;&gt;HMPP&lt;/a&gt; or something -- those are probably (hopefully?) going to be good ways of programming these kinds of architectures in the future, where you can get 90% of the performance for 10% of the work.   But which choice will &quot;win&quot; remains to be seen and I wouldn't recommend spending a lot of time working with those just yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'd say, between CUDA or OpenCL pick a language that's convenient for you and use it, and don't worry too much about it.   The valuable part - figuring out how to decompose your problem into massively-parallel SIMD code for small cores with very little memory - is going to be pretty easily portable between programming models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're using NVIDIA hardware - and you probably are - then I typically recommend CUDA - Matt Knepley's point about the libraries is dead-on.   If you're not, then OpenCL.  &lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2011-12-17T03:54:53.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="459" PostTypeId="1" AcceptedAnswerId="460" CreationDate="2011-12-17T04:43:21.583" Score="9" ViewCount="9331" Body="&lt;p&gt;The &lt;code&gt;qsub&lt;/code&gt; command which submits PBS jobs has a &lt;code&gt;-p&lt;/code&gt; option that allows you to set the priority. From the man page:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Defines the priority of the job.  The priority argument must be a integer between -1024 and +1023 inclusive.  The default is no priority which is equivalent to a priority of zero.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And that's &lt;em&gt;all&lt;/em&gt; that is in the man page. Which leaves me wondering, what does the priority actually mean? For example: does the system just pick out the jobs with the highest priority to run, or does it use some combination of waiting time and priority? Is priority only measured relative to other jobs I've submitted, or does it cross user boundaries? Does a higher number mean higher priority, or is it like Linux's niceness? Basically I'm looking for a less sparse explanation of what this option is good for.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2011-12-17T15:47:44.083" Title="What does the priority of a PBS job really mean?" Tags="|hpc|pbs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="460" PostTypeId="2" ParentId="459" CreationDate="2011-12-17T05:16:57.643" Score="6" Body="&lt;p&gt;Here is what I know as a user (sysadmins might have a more accurate answer):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the queue system uses some algorithm to maximize the use of the resources while prioritizing some jobs over others. Let's say there are 100 cores available on the queue. A high priority job (more on this in a second) asks for 128 cores for 24 hours and was submitted first while a low priority job is now asking for 64 cores for 12 hours. If additional cores are not going to get available within 12 hours (thus allowing the big job to run), the queue will let the small job go through the queue even though it has lower priority and was submitted after the other one. It does so because there is no way the bog job will run in the next 12 hours and wants to maximize the use of resources.&lt;/li&gt;&#xA;&lt;li&gt;if using a queue where defining priority of a job is not explicitely allowed, different queues (standard, debug, long, special, ...) will give jobs a default priority. This allows more important project to go through the queue faster than less important ones for example. And prevents every computational scientist from setting his/her own job's priority to the max :)&lt;/li&gt;&#xA;&lt;li&gt;jobs will also gain priority over time. Not sure if the rate is dependent on the queue, but I believe higher number means higher priority.&lt;/li&gt;&#xA;&lt;li&gt;the priority is measured across all jobs in the queue system&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this helps !&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2011-12-17T05:16:57.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="461" PostTypeId="1" AcceptedAnswerId="462" CreationDate="2011-12-17T06:00:53.390" Score="28" ViewCount="8063" Body="&lt;p&gt;As far as I can tell, the two big generic US Department of Energy computational science software frameworks are  &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc&lt;/a&gt; and &lt;a href=&quot;http://trilinos.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trilinos&lt;/a&gt;. They seem similar at first glance, beyond differences in language (C versus C++). What are the main differences between the two frameworks, and what factors should influence choosing one over the other? (Ignore institutional bias and existing infrastructure.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="43535" LastEditDate="2022-07-11T14:26:56.680" LastActivityDate="2022-07-11T14:26:56.680" Title="What are the main differences between PETSc and Trilinos?" Tags="|petsc|software|trilinos|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="462" PostTypeId="2" ParentId="461" CreationDate="2011-12-17T12:49:57.597" Score="21" Body="&lt;p&gt;There are huge differences in culture, coding style, and capabilities.  Probably the fundamental difference is Trilinos tries to provide an environment for solving FEM problems and PETSc provides an environment for solving sparse linear algebra problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is that significant? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Trilinos will provide a large number of packages concerned with separate parts of the FEM solver. Sometimes these packages work together sometimes they don't.  Even the base components are in its own package and advanced C++ tools&lt;/li&gt;&#xA;&lt;li&gt;PETSC provides a small amount of core routines that can be built upon, but leaves the FEM solvers to third party packages.  Because of this, it is associated with a larger community than just FEM.  For example, even the eigen solvers are third party which is arguably a major part of linear algebra.&lt;/li&gt;&#xA;&lt;li&gt;Bottom line, Trilinos focuses working well within its own packages and PETSc has interfaces that call out to many middleware packages (I've often heard it called &quot;lighter-weight&quot; because of this but I wouldn't make that claim)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;IMHO, which you should use really depends on the problem.  Please share more details for us to answer that question.&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2011-12-17T12:49:57.597" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="463" PostTypeId="2" ParentId="388" CreationDate="2011-12-17T15:16:36.707" Score="1" Body="&lt;p&gt;Have a look at &lt;a href=&quot;http://www.netlib.org/quadpack/&quot; rel=&quot;nofollow&quot;&gt;QUADPACK&lt;/a&gt;. It has routines for integration over (semi-)infinite domains. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="276" LastEditDate="2012-01-02T23:06:03.657" LastActivityDate="2012-01-02T23:06:03.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="464" PostTypeId="2" ParentId="351" CreationDate="2011-12-17T15:26:24.830" Score="3" Body="&lt;p&gt;I have been using &lt;a href=&quot;http://getfem.org/gmm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMM++&lt;/a&gt; for some time and I'm happy with that. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2019-01-06T13:29:39.570" LastActivityDate="2019-01-06T13:29:39.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="465" PostTypeId="2" ParentId="459" CreationDate="2011-12-17T15:47:44.083" Score="4" Body="&lt;p&gt;FrenchKheldar's answer is very good.   The details will vary somewhat from system to system, but the basics are all there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On our system, we have an &lt;a href=&quot;https://support.scinet.utoronto.ca/wiki/index.php/Scheduler#Adjusting_Job_Priority&quot; rel=&quot;nofollow&quot;&gt;explanation&lt;/a&gt; of the -p flag in particular: it basically is as you suspect like nice, and like nice normal users can only make their jobs have lower (eg, use negative arguments to p) priority than normal.  This lets you shuffle around the order of your jobs with respect to each other, but probably doesn't greatly change the ordering of your jobs and someone else's.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2011-12-17T15:47:44.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="466" PostTypeId="2" ParentId="461" CreationDate="2011-12-17T15:58:33.860" Score="12" Body="&lt;p&gt;If you want to hear from the technical leads of various packages, taking questions from the point of view of a normal HPC user, Brock Palen and Jeff Squyres' &lt;a href=&quot;http://www.rce-cast.com/&quot;&gt;RCE podcast&lt;/a&gt; is an excellent resource.   They have episodes on &lt;a href=&quot;http://www.rce-cast.com/Podcast/rce-24-petsc.html&quot;&gt;PETSc&lt;/a&gt; and on &lt;a href=&quot;http://www.rce-cast.com/Podcast/rce-49-trilinos.html&quot;&gt;Trilinos&lt;/a&gt; which are very clear.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;aterrel is right in his descriptions -  PETSc is a (comparitively) small, integrated, well-thought out package of general-purpose linear and some nonlinear solvers, which one could then use in a solver framework; Trilinos is a collection of packages, with integration which is evolving, mostly aimed at being that solver framework, and includes things like ODE solvers, meshing packages, etc.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2011-12-17T15:58:33.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="467" PostTypeId="2" ParentId="265" CreationDate="2011-12-17T17:34:35.717" Score="19" Body="&lt;p&gt;We strongly recommend all our users start with &lt;code&gt;-O3 -xHost -ipo&lt;/code&gt; for both ifort 11 and ifort 12.  If there are particular floating point transformations enabled by O3 that affect the precision of some of your computations, you can turn those specifically off with &lt;code&gt;-fp-model precise -fp-model except&lt;/code&gt; (or, more drastically, &lt;code&gt;-fp-model strict&lt;/code&gt;) while retaining the other optimizations O3 enables, such as loop blocking for cache, loop fusion and unrolling, and memory access optimizations.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd advise trying the floating point model stuff on individual files and finding out where it makes a difference, rather than turning it off globally; it can be a ~15% speed bump, and you want to be able to keep that where it doesn't affect your calculations.  If you're not sure where the precision is being affected, you can play with turning on and off the floating point model flags for those files, or &lt;a href=&quot;http://www.scinethpc.ca/2011/11/testing-roundoff/&quot; rel=&quot;nofollow noreferrer&quot;&gt;playing with rounding modes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We recently gave a short talk to our users about optimization flags, focusing on the GNU and Intel compilers for x86; you can see the slides from that talk &lt;a href=&quot;https://support.scinet.utoronto.ca/wiki/images/7/77/Snug_techtalk_compiler.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Incidentally, while we're talking about choosing optimization flags for your code, every now and then it's also worth looking at the output of a &lt;a href=&quot;https://software.intel.com/en-us/fortran-compiler-auto-vectorization-tutorial-generating-a-vectorization-report&quot; rel=&quot;nofollow noreferrer&quot;&gt;vectorization report&lt;/a&gt; to see where the compiler attempted to vectorize a loop and couldn't. Sometimes there are small changes you can make to your loop which can result in vectorization being possible (which can be a 4x speedup).   Similarly, see &lt;a href=&quot;https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-optimization-report-options&quot; rel=&quot;nofollow noreferrer&quot;&gt;-qopt-report&lt;/a&gt;  for information on more general optimization reporting (&lt;a href=&quot;http://software.intel.com/file/6747&quot; rel=&quot;nofollow noreferrer&quot;&gt;-opt-report&lt;/a&gt;, for older versions).&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastEditorUserId="34277" LastEditorDisplayName="user389" LastEditDate="2020-03-06T19:42:35.017" LastActivityDate="2020-03-06T19:42:35.017" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="468" PostTypeId="2" ParentId="194" CreationDate="2011-12-17T17:51:26.053" Score="7" Body="&lt;p&gt;Parallel I/O can help you in this case, but if you're using (inherently pretty serial) NFS to serve your files, then it's not going to have the full effect you might want - there's going to be a serial bottleneck at the fileserver and having hundreds of processes making requests of the single server is not going to give you factors of hundreds speedup of doing it through a single process.   Still, it could help to a point, especially since it sounds like the bottleneck is &lt;em&gt;reading&lt;/em&gt; rather than writing, and it will be a big improvement if your system gets upgraded to a fully parallel filesystem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MPI-IO is very low-level; it's worth understanding something about it to know what's going on &quot;under the hood&quot; with parallel HDF5, &lt;a href=&quot;http://www.unidata.ucar.edu/software/netcdf/index.html&quot; rel=&quot;noreferrer&quot;&gt;NetCDF4&lt;/a&gt;, or &lt;a href=&quot;http://www.olcf.ornl.gov/center-projects/adios/&quot; rel=&quot;noreferrer&quot;&gt;ADIOS&lt;/a&gt;, but using it yourself is really only well suited for raw binary data where the structure is well known at compile time.  HDF5 and NetCDF4 are much more flexible.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if your data is relatively simple -- eg, the big data structures are mainly  n-dimensional arrays or vectors -- I recommend NetCDF4 (which is also parallel, and based on HDF5) rather than HDF5; it's &lt;em&gt;siginificantly&lt;/em&gt; simpler to use.   HDF5 is more complicated, and in exchange for that complexity allows very complicated data models.  But if that's a feature you don't need, it's faster to get started in NetCDF4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At our centre we have an afternoon-long and a day-long class on parallel I/O where we talk about the basic concepts, MPI-IO, HDF5, and NetCDF4; the slides can be found &lt;a href=&quot;https://support.scinet.utoronto.ca/wiki/index.php/Knowledge_Base%3a_Tutorials_and_Manuals#I.2FO&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastEditorDisplayName="user389" LastEditDate="2011-12-17T19:54:19.083" LastActivityDate="2011-12-17T19:54:19.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="469" PostTypeId="1" CreationDate="2011-12-17T17:58:45.240" Score="8" ViewCount="2144" Body="&lt;p&gt;I want to get started on scientific programming on the Mac using Objective-C.  I am very familiar with MATLAB which makes it easy to store complex-valued waveform data in vectors and generating plots.  I was wondering if there might be a good resource that might compare how you would implement common MATLAB operations (i.e. creating an array of complex values or generating a mesh plot) in an Objective-C/Cocoa environment.&lt;/p&gt;&#xA;" OwnerUserId="415" LastEditorUserId="81" LastEditDate="2011-12-23T17:41:48.640" LastActivityDate="2012-01-10T20:01:00.883" Title="Scientific Programming on Mac using Objective-C/Cocoa for MATLAB Users" Tags="|software|matlab|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="470" PostTypeId="1" AcceptedAnswerId="471" CreationDate="2011-12-17T18:44:26.553" Score="8" ViewCount="522" Body="&lt;p&gt;Sometimes it is useful in my work to modify someone else open-source code or find out how to develop specific things for your own application. However, not all software have good documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a good way to understand the overall structure of a code base?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, which routines calls which routines, etc. I might use a documentation tool such as Doxygen for this purpose myself, however, I was wondering if there is a better strategy?&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2011-12-21T20:05:32.517" Title="What is a good way to understand the overall structure of a code base?" Tags="|software|documentation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="471" PostTypeId="2" ParentId="470" CreationDate="2011-12-17T22:08:34.840" Score="9" Body="&lt;p&gt;The following threads are tangentially related:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/49098/im-tasked-with-leading-the-documentation-effort-for-an-existing-entirely-undoc&quot;&gt;This one on programmers.SE&lt;/a&gt; on leading the documentation effort for an undocumented project.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/90531/how-to-learn-an-api&quot;&gt;This one on programmers.SE&lt;/a&gt; on learning undocumented APIs.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/41539/methodology-for-documenting-existing-code-base&quot;&gt;This one on programmers.SE&lt;/a&gt; on documenting code.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/2747427/training-new-employees-on-undocumented-code&quot;&gt;This one on Stack Overflow&lt;/a&gt; on bringing employees up to speed on undocumented APIs.&lt;/li&gt;&#xA;&lt;li&gt;If you're inherited the code, &lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/82906/is-there-any-value-in-writing-unit-tests-for-code-that-already-works-when-inheri&quot;&gt;this one on programmers.SE&lt;/a&gt; might be helpful.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/67769/best-practices-and-tips-for-reading-source-code&quot;&gt;This one on programmers.SE&lt;/a&gt; mentions good practices for reading code, which is probably most directly related to your question at a low level.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the first part of my thesis, I spent 18 months modifying undocumented Fortran code, and one of the first tasks was to try and understand the overall structure of a code base. The most important thing I suggest you do is take notes in a text file any time you figure something out. You don't want to have to relearn or rediscover things during this time-consuming and frustrating process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, there was no &quot;API&quot; to speak of, in that the arguments of functions weren't self-documenting, because the previous programmer used Fortran 77-like style, and thus, short identifiers with little to no hint of what they meant. There were no tests, and since it's Fortran, no headers. Adding even more fun to the mix, there were a couple functions here and there written in C or C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things that worked for me (assuming you work in Linux):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;grep&lt;/code&gt;. Learn to love &lt;code&gt;grep&lt;/code&gt;; you'll use it in the shell to find where functions are declared and called, and the output will tell you what files to look in.&lt;/li&gt;&#xA;&lt;li&gt;The &quot;find&quot; command in your favorite code editor or IDE. Once you know what file to look in, the &quot;find&quot; command will save you time in hunting around for function calls.&lt;/li&gt;&#xA;&lt;li&gt;Aggressive refactoring, if you can change the code base. I made the variable names self-documenting so that I didn't have to spend mental effort relearning things I'd already figured out. I also streamlined the design of the code as I figured it out so it was less confusing. No more 1000 line long main program!&lt;/li&gt;&#xA;&lt;li&gt;Aggressive commenting. Again, if you can change the code base, comment things so that you know what you figured out. I didn't use Doxygen, but Doxygen is good for this.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;nm&lt;/code&gt;. It can be useful on libraries if you don't have source code for them, but want to know if a function you encountered is in that library. However, it only works if the library's symbols haven't been stripped.&lt;/li&gt;&#xA;&lt;li&gt;Step through the code with a debugger. It's way more efficient than using &lt;code&gt;print&lt;/code&gt; statements. &lt;code&gt;ddd&lt;/code&gt; and &lt;code&gt;gdb&lt;/code&gt; are great, and on virtually every Linux system out there. Feel free to use your favorite debugger.&lt;/li&gt;&#xA;&lt;li&gt;Bug the developers. This option is really best for very targeted questions. If you go to them (as I did), and say, &quot;I don't understand what's going on here,&quot; they might take pity on you and try to explain things in general detail, but that's going to be of limited use in the long run. You're going to have to do the leg work, because the developers didn't do it for you in writing documentation and explaining the structure of their code base in writing. The developers are really good for when you're really stuck on small things (if they remember what they did).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Things I'd wish I'd thought of earlier, or just weren't options for me:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Doxygen. If you tweak the Doxyfile options, Doxygen will automatically generate a lot of documentation for you, even without the special Doxygen commenting syntax, which could be a good place to start; I've used this on later projects I encountered and it's been incredibly helpful.&lt;/li&gt;&#xA;&lt;li&gt;Unit testing. If you can alter the code base, and you have a some idea of what it's supposed to do, write unit tests for various functions. (It's a useful skill to pick up regardless.)&lt;/li&gt;&#xA;&lt;li&gt;If you're working with C/C++, look at the headers.&lt;/li&gt;&#xA;&lt;li&gt;Write example programs. Not an option for me on that Fortran project, but it's been useful for me in picking up third-party APIs. Also, look at &lt;em&gt;their&lt;/em&gt; example programs, if they have any.&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;code&gt;gcov&lt;/code&gt; and &lt;code&gt;lcov&lt;/code&gt; to do a coverage analysis on typical runs of the code, if you have examples or executables to work with. If there are examples that are supposed to execute large parts of the code base, these two tools combined will tell you how many times each line of code is visited. It's most useful with debugging flags enabled. If a part of the code isn't visited at all, it's probably less important to understand it right away. If a part of the code is visited a lot, it's probably worth understanding what it is. Maybe the code is executed a lot because it's an unimportant loop, or it could be a key function that a lot of other functions rely on. You can't tell just on the coverage analysis, but the coverage analysis gives you some idea of where to focus your time.&lt;/li&gt;&#xA;&lt;li&gt;Static code analysis tools like &lt;code&gt;splint&lt;/code&gt; can tell you if something fishy is going on in the code, like some variables are never used.&lt;/li&gt;&#xA;&lt;li&gt;Profiling. Again, gives you data that doesn't immediately tell you what is or is not important, but suggests what might be important. If a lot of CPU time is spent calling one function, you may want to look at that and see what it does. You can also use the profiling output with &lt;code&gt;dot&lt;/code&gt; and &lt;code&gt;graphviz&lt;/code&gt; to generate call graphs and see how many times functions are called, like the coverage analysis. For complex codes, a graphical analysis could be a lot more helpful.&lt;/li&gt;&#xA;&lt;li&gt;If you're working in C, &lt;a href=&quot;http://frama-c.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Frama-C&lt;/a&gt; is supposed to be helpful in analyzing code, but I've never used it because it seemed too complicated to be worth the effort. I do some work in pure C, but it's mostly code that I write; I've never worked with undocumented C code.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2011-12-19T05:35:21.667" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="472" PostTypeId="1" AcceptedAnswerId="477" CreationDate="2011-12-17T23:03:42.123" Score="4" ViewCount="115" Body="&lt;p&gt;There are several correspondences between matrices and graphs, e.g., each matrix is the adjacancy matrix of a weighted graph. The terms support preconditioner or combinatorial preconditioner refer to preconditioners which have been designed using graph theoretical tools that rely on these correspondences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It certainly good to have another track of development besides, say, incomplete cholesky factorization or algebraic multigrid. But I wonder in which cases these preconditioners are superior to other usual preconditioners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In recent paper, these preconditioners have been applied to finite element discretizations of scalar elliptic PDEs: &lt;a href=&quot;http://www.cs.tau.ac.il/~stoledo/Pubs/ACST.pdf&quot; rel=&quot;nofollow&quot;&gt;Avron et al., Combinatorial preconditioners for scalar elliptic finite-elements problems. &lt;/a&gt; - As far as I understand, these preconditioners are outperformed by Incomplete Cholesky factorization, but on the other hand are very well-behaved in the presence of degenerated elements or very anisotropic coefficient tensors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the applications where these preconditioners are most promising?&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2011-12-18T14:31:08.567" Title="Applicability of combinatorial and support preconditioner" Tags="|linear-algebra|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="473" PostTypeId="2" ParentId="40" CreationDate="2011-12-18T00:39:17.207" Score="5" Body="&lt;p&gt;I think in general it's worth remembering that the most efficient methods we have (geometric and algebraic multigrid as well as, to a degree, domain decomposition) rely on the fact that solutions of PDEs are often smooth and that solving a coarser problem may yield a good approximation for the fine scale problem. The problem with the Helmholtz equation for high frequencies is that this assumption is not true: you do need a relatively fine mesh to represent the solution, and coarse mesh solvers will not be able to produce anything that's of much use. Consequently, the typical approaches to good preconditioners don't work in that case, and that is the underlying reason why there are no real good options in your case short of just throwing lots of processors at the problem; whether you use finite elements or finite difference then obviously doesn't make much of a difference since the problem is at the continuous level.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-18T00:39:17.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="474" PostTypeId="2" ParentId="47" CreationDate="2011-12-18T00:44:21.790" Score="1" Body="&lt;p&gt;Another option you have, unless the system is rather complicated, is to just convert from complex notation to a problem with two unknowns that represent the real and imaginary part. You can then use a standard real-valued ODE solver.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-18T00:44:21.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="475" PostTypeId="2" ParentId="152" CreationDate="2011-12-18T00:53:06.997" Score="8" Body="&lt;p&gt;I'll try a shorter answer than some of my esteemed colleagues on this thread ;-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My message to all my students is always that developer time is more valuable than CPU time. That means that if you have time to convert 100% of the code at 80% efficiency to run on big machines -- using a high level approach --, then you're better off than when you use a time-consuming low-level approach that gives you 100% efficiency on 20% of your code. As a consequence, I'm a big fan of high-level libraries. My favorite in this area are the threading building blocks (TBB) since it allows me to look at algorithms at the outermost loops and at a high level. It can also do all the things you may want to do with pthreads without the cruddiness of having to deal with OS functions, etc. I am not a fan of approaches that look at innermost loops because that's the wrong level to exploit intranode parallel resources -- so no OpenMP, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't speak with authority about OpenCL, CUDA, etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-18T00:53:06.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="476" PostTypeId="2" ParentId="114" CreationDate="2011-12-18T09:44:28.410" Score="9" Body="&lt;p&gt;As others have pointed out, the answer does of course depend on whether the bottleneck is the CPU or memory bandwidth.  For many algorithms that work on some arbitrarily-sized dataset, the bottleneck is usually the memory bandwidth as the dataset does not fit into the CPU cache.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, Knuth mentions that memory access analysis is more likely to stand the test of time, presumably because it is relatively simple (even when taking into account cache-friendliness) compared to the complexities of modern CPU pipelines and branch prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knuth uses the term &lt;em&gt;gigamems&lt;/em&gt; in Volume 4A of TAOCP, when analysing BDDs.  I'm not sure if he uses it in previous volumes.  He made the aforementioned comment about standing the test of time in his annual Christmas Tree Lecture in 2010.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, &lt;a href=&quot;http://queue.acm.org/detail.cfm?id=1814327&quot;&gt;You're Doing It Wrong&lt;/a&gt; shows that even analysing performance based on memory operations is not always straightforward as there are elements such as VM pressure that come into play if the data does not fit into physical RAM all at once.&lt;/p&gt;&#xA;" OwnerUserId="405" LastActivityDate="2011-12-18T09:44:28.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="477" PostTypeId="2" ParentId="472" CreationDate="2011-12-18T14:31:08.567" Score="5" Body="&lt;p&gt;I have not kept up with this literature for a while, but I see these methods as not quite a genuine &quot;another track of development&quot;.  My recollection is the theory was that it invoked classic linear algebra at the end of the day and end up looking like incomplete factorization methods.  That said if you have some ILU solver that you motivate with graph theory then more power to you, or if you find a useful implementation of a solver that invokes graph theory then by all means use it.  I just don't see the area as a generically interesting source of algorithms that are more useful than what has come out of classic linear algebra communities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And there is no comparison of these methods (and ILU) with AMG that explicitly exploit properties of individual discretizations of PDEs (eg, null spaces) and have solid theory, both abstract functional analysis type theory and linear algebra proofs of optimality.  Note, AMG needs smoothers to actually solve the problem and ILU or support graph methods can be perfectly useful as smoothers.&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2011-12-18T14:31:08.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="478" PostTypeId="2" ParentId="461" CreationDate="2011-12-18T14:44:42.773" Score="8" Body="&lt;p&gt;Let me add to aterrel's good comment that Trilinos is really a big bag of (Sandia) stuff and Petsc is a more focused library.  If you want to compare then you should compare PETSc's sparse solver support with Trilinos' ePetra/ML/etc sparse solver ecosystem, that do similar things.  Also, PETSc supports structured grids and Sandia has historically explicitly been an unstructured (FEM) house so Trilinos has little or no support for structured grids AFAIK.  And Trilinos has capabilities that PETSc does not touch like stochastic PDE support.&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2011-12-18T14:44:42.773" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="479" PostTypeId="2" ParentId="40" CreationDate="2011-12-18T22:20:43.323" Score="11" Body="&lt;p&gt;EDIT: The previous comment is now completely outdated. Please see &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/120871985&quot; rel=&quot;nofollow&quot;&gt;the related work section of the published paper&lt;/a&gt; for a more complete discussion, and &lt;a href=&quot;http://libelemental.org&quot; rel=&quot;nofollow&quot;&gt;Elemental&lt;/a&gt;, &lt;a href=&quot;http://github.com/poulson/Clique&quot; rel=&quot;nofollow&quot;&gt;Clique&lt;/a&gt;, and &lt;a href=&quot;http://github.org/poulson/PSP&quot; rel=&quot;nofollow&quot;&gt;PSP&lt;/a&gt; for the underlying software. &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/nla.1860/abstract&quot; rel=&quot;nofollow&quot;&gt;Two-grid preconditioners&lt;/a&gt; are also worth investigating.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2013-07-24T18:09:54.400" LastActivityDate="2013-07-24T18:09:54.400" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="480" PostTypeId="1" AcceptedAnswerId="481" CreationDate="2011-12-19T08:19:21.983" Score="30" ViewCount="22439" Body="&lt;p&gt;Standard &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference&quot;&gt;finite difference formulas&lt;/a&gt; are usable to numerically compute a derivative under the expectation that you have function values $f(x_k)$ at evenly spaced points, so that $h \equiv x_{k+1} - x_k$ is a constant. What if I have unevenly spaced points, so that $h$ now varies from one pair of adjacent points to the next? Obviously I can still compute a first derivative as $f&amp;#39;(x) \approx \frac{1}{h_k}[f(x_{k+1}) - f(x_k)]$, but are there numerical differentiation formulas at higher orders and accuracies that can adapt to variation in the grid size?&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2020-03-05T21:38:02.727" Title="How can I numerically differentiate an unevenly sampled function?" Tags="|finite-difference|discretization|" AnswerCount="6" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="481" PostTypeId="2" ParentId="480" CreationDate="2011-12-19T13:56:53.343" Score="31" Body="&lt;p&gt;J.M's comment is right: you can find an interpolating polynomial and differentiate it.  There are other ways of deriving such formulas; typically, they all lead to solving a &lt;a href=&quot;http://en.wikipedia.org/wiki/Vandermonde_matrix&quot;&gt;van der Monde system&lt;/a&gt; for the coefficients.  This approach is problematic when the finite difference stencil includes a large number of points, because the Vandermonde matrices become ill-conditioned.  A more numerically stable approach was devised by &lt;a href=&quot;http://dx.doi.org/10.2307/2008770&quot;&gt;Fornberg&lt;/a&gt;, and is explained more clearly and generally in &lt;a href=&quot;http://dx.doi.org/10.1137/S0036144596322507&quot;&gt;a second paper&lt;/a&gt; of his.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/matlab/fdcoeffF.m&quot;&gt;&lt;strong&gt;Here is a simple MATLAB script&lt;/strong&gt;&lt;/a&gt; that implements Fornberg's method to compute the coefficients of a finite difference approximation for any order derivative with any set of points.  For a nice explanation, see Chapter 1 of &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot;&gt;LeVeque's text on finite difference methods&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bit more on FD formulas:  Suppose you have a 1D grid.  If you use the whole set of grid points to determine a set of FD formulas, the resulting method is equivalent to finding an interpolating polynomial through the whole grid and differentiating that.  This approach is referred to as spectral collocation.  Alternatively, for each grid point you could determine a FD formula using just a few neighboring points.  This is what is done in traditional finite difference methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in the comments below, using finite differences of very high order can lead to oscillations (the Runge phenomenon) if the points are not chosen carefully.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2011-12-20T18:28:44.260" LastActivityDate="2011-12-20T18:28:44.260" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="482" PostTypeId="2" ParentId="469" CreationDate="2011-12-19T19:07:03.247" Score="10" Body="&lt;p&gt;I don't know of any Objective-C-specific libraries or frameworks, but there are a large number of libraries written in and available for plain C, and you should be able to use them with Objective-C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using C allows you to use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt;, &lt;a href=&quot;http://www.gnu.org/s/gsl/&quot; rel=&quot;nofollow&quot;&gt;gsl&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/LAPACK&quot; rel=&quot;nofollow&quot;&gt;LAPACK&lt;/a&gt;, and loads of other useful numerical libraries. I would advise against trying to code the matlab routines yourself. Internally, matlab likely uses well-known numerical libraries itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're looking for numerical libraries in an object-oriented language, you might want to look into using C++, which has &lt;a href=&quot;http://arma.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;Armadillo&lt;/a&gt; and &lt;a href=&quot;http://trilinos.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;Trilinos&lt;/a&gt; available.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-10T07:21:32.177" LastActivityDate="2012-01-10T07:21:32.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="483" PostTypeId="2" ParentId="37" CreationDate="2011-12-20T09:36:18.823" Score="1" Body="&lt;p&gt;Higher order methods can be used e.g. to solve the level set equation when using it to describe a two phase fluid flow within a Finite Volume Method framework. In this case, the WENO and ENO schemes are used to advect the level set function and a re-initialisation step is used to maintain it as a distance function from the fluid interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check this out:  &lt;a href=&quot;http://ftp.cc.ac.cn/lcfd/WENO_mem.html&quot; rel=&quot;nofollow&quot;&gt;http://ftp.cc.ac.cn/lcfd/WENO_mem.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, they are used in CFD simulations when dealing with discontinuities in the flow. &lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2011-12-20T09:36:18.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="484" PostTypeId="2" ParentId="37" CreationDate="2011-12-20T09:44:19.867" Score="4" Body="&lt;p&gt;&lt;strong&gt;Guidelines:&lt;/strong&gt; High-order methods for problem where solution is expected to be smooth and otherwise low-order methods and/or methods which can handle discontinuities in solutions. In cases where high-order methods can be exploited there can be significant saving in computational effort measured in terms CPU time as a result of high convergence rate. For elliptic problems which require solution of linear systems, high-order methods leads to less sparse operators and this have to be compensated by faster convergence rate. For time-dependent problems, if high-order methods can be exploited faster convergence rate and more accuracy can be achieved and for long integration times high-order methods are superior in terms of both accuracy and computational effort due to low numerical dispersion and dissipation errors.&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2011-12-20T09:44:19.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="485" PostTypeId="1" AcceptedAnswerId="489" CreationDate="2011-12-20T13:17:20.413" Score="13" ViewCount="2280" Body="&lt;p&gt;What would you say would be the advantages/disadvantages of two approaches to coding a general (finite volume, fem, dg) library for Computational Continuum Mechanics? This is how I see things right now, so please provide your own experiences and don't flame me for mine :) : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) C++:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;generic programming, virtual functions, overloading, speed... : all the genreic + OOP tools available to build whatever you want&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;low level libraries available mostly (no wide spread science&amp;amp;engineering library development such as the one for Python)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;2) Python + wrappers for parallel computing (pyOpenCL and others)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;huge amount of supporting libs of various kinds&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;code what you think: the implementation is done really fast&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;slower execution time &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you wanted to code a framework that would support various methods, work with complex geometries and problems, what would you choose and why? &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="41" LastEditDate="2012-01-09T01:57:33.787" LastActivityDate="2012-11-22T10:04:00.763" Title="C++ or Python for a development of CFD library" Tags="|libraries|programming-paradigms|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="486" PostTypeId="2" ParentId="485" CreationDate="2011-12-20T14:50:47.900" Score="2" Body="&lt;p&gt;I think there are more to this question. First and foremost a developer will typically prefer what he/her is familiar with unless significant advantages (e.g. in productivity, development time and tools). Personally, I give priority to being productive (time is usually the most scarce resource!) and this favors choices which are close to my experience base.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps also relevant to take into account are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Development time&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how much time is reserved for development&lt;/li&gt;&#xA;&lt;li&gt;when is results of the work to be delivered? and how?&lt;/li&gt;&#xA;&lt;li&gt;do a code already exist which can do the job? (uniqueness?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;4) Maintenance&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how many (people) resources are devoted to maintenance?&lt;/li&gt;&#xA;&lt;li&gt;how many people are to work on the code?&lt;/li&gt;&#xA;&lt;li&gt;Is the code to be released at some point? (criteria?)&lt;/li&gt;&#xA;&lt;li&gt;are the code going to rely on third-party libraries?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;5) Licensing issue&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is the code for research?&lt;/li&gt;&#xA;&lt;li&gt;is the code for commercial applications?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;6) Productivity and Fun factor (often overlooked!)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Where can one be most productive?&lt;/li&gt;&#xA;&lt;li&gt;Where can one have the most fun developing?&lt;/li&gt;&#xA;&lt;li&gt;Any opportunities to be a part of a (social) network?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="304" LastActivityDate="2011-12-20T14:50:47.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="487" PostTypeId="2" ParentId="326" CreationDate="2011-12-20T14:58:32.723" Score="11" Body="&lt;p&gt;I disagree with Matt about total memory use for the mesh and about extra indirection. For explicit methods, it is common for a very small number of vectors (e.g. 2) to represent all simulation state. For a scalar problem, just defining the coordinates of the mesh may be more than this, and if connectivity is explicit, it is substantially more. High-resolution simulations with explicit methods frequently need a huge number of time steps, so it's common to use reasonably small subdomains just to be able to complete a model run in a reasonable amount of wall time. Thus, it's not all that common to run out of memory because of the cost of storing the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A far more fundamental problem is the &lt;strong&gt;memory bandwidth requirement&lt;/strong&gt; for each step of an algorithm (e.g. evaluating a residual or taking a time step). For unstructured methods, this involves some mesh-based information, but it usually does not require accessing &lt;em&gt;all&lt;/em&gt; mesh storage. Note that even if solver memory dominates mesh memory (as is common for many implicit methods), the simulation is still frequently dependent on the bandwidth requirements of a mesh traversal. There are two factors:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What data is needed during routine mesh traversal?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In finite element methods, an element-to-global map is needed. For higher than first order, this is often implemented by associating degrees of freedom with intermediate entities like faces and edges, but once the element-to-global map is constructed, the intermediate entities are not needed. In particular, the connectivity of the intermediate entities is never used directly by a simulation. That information might not be touched for many iterations of an implicit solver or time integrator, but still be required during adaptivity or to set up a new function space. It is common to set up the element-to-global map in a simple array and not touch the &quot;mesh&quot; data structure during this period, in which case the storage format and data locality of the mesh itself is less important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite volume methods require the cell volumes, face-to-cell connectivity, face area, and face normals. Note that vertex coordinates or connectivity need not be available. In extreme examples (e.g. FUN3D), all other information is discarded during offline preprocessing (mesh generation) and only this reduced representation is available to the simulation. This is very efficient, but precludes moving meshes and adaptive refinement.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;How is that data laid out in memory?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For many simulations, with modest orders of accuracy and complexity of physics, performance is limited by memory bandwidth. Modern CPU architectures from IBM, Intel, AMD, and NVidia support an arithmetic intensity between 4 and 8 flops/byte. With such efficient floating point units, we should try to optimize our algorithms for memory bandwidth. This might involve somewhat deep algorithmic changes such as favoring unassembled high order methods (compare the &quot;assembled&quot; and (unassembled) &quot;tensor&quot; lines in the &lt;a href=&quot;https://github.com/jedbrown/dohp/wiki/Dohp&quot;&gt;second figure&lt;/a&gt;), but we can start by attempting to fully utilize the bandwidth provided by the hardware. This often involves computing orderings (of vertices, faces, cells, etc) such that cache is reused as well as possible. It also involves exploiting blocking and ordering unknowns so as to have minimal metadata and to activate the right number of data streams. (Usually 3D unstructured simulation ends up with &quot;too many&quot; streams, but some modern hardware like POWER7 needs many streams to saturate bandwidth, so it occasionally makes sense to intentionally organize data to activate more streams.) The &lt;a href=&quot;http://www.sc2000.org/techpapr/papers/pap.pap279.pdf&quot;&gt;PETSc-FUN3D work&lt;/a&gt; provides a classical, but still highly relevant discussion of these performance optimization for unstructured implicit CFD.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Suggestions for your problem&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do not be afraid of &lt;code&gt;malloc()&lt;/code&gt;. There is no need to pack the current refinement of the mesh into the same array as the coarser inactive part of the mesh. Whenever you no longer need an old part of the mesh, just &lt;code&gt;free()&lt;/code&gt; it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;After refinement, compute a good ordering for that refinement level (&lt;a href=&quot;http://en.wikipedia.org/wiki/Reverse_Cuthill-McKee&quot;&gt;Reverse Cuthill-McKee&lt;/a&gt; is popular, but more cache-specific orderings can also be used). If your load imbalance is reasonable (less than 10%, say), you can compute this new ordering locally (without parallel partitioning and redistribution). The cost will likely be similar to a single &quot;physics&quot; mesh traversal, but can speed up those traversals by a factor of 2 or more. This step will usually pay off any time mesh adaptivity does not occur on &lt;em&gt;each&lt;/em&gt; &quot;physics&quot; mesh traversal. If such frequent adaptivity is required for your problem, you are likely making small changes and should still reorder occasionally. I would still avoid fine-grained memory pools because it makes reordering more difficult, but you can use reasonably large chunks to balance peak memory usage, ordering cost, and cost of incremental updates.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Depending on the discretization, consider extracting a reduced representation with low memory bandwidth requirements for the &quot;physics&quot; traversals. Unnecessary indirection is bad because it increases the bandwidth requirements and, if the target of the indirection is irregular, causes poor cache reuse and inhibits prefetch.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-12-20T14:58:32.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="488" PostTypeId="2" ParentId="485" CreationDate="2011-12-20T15:46:17.353" Score="2" Body="&lt;p&gt;This depends whether your code can be written as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;some_library_specific_type grid;&#xA;&#xA;for t=0 to T do&#xA;    library_function_1(grid,...);&#xA;    library_function_2(grid,...);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or rather &lt;em&gt;must&lt;/em&gt; be written as something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;some_home_made_mixture_of_native_types grid;&#xA;&#xA;for t=0 to T do&#xA;    for all grid elements as g do&#xA;        some_function(g,...);&#xA;        library_function(g,...);&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the first case choose what you like most to code in; in the second case don't use any scripting language or prepare to suffer from execution time.&lt;/p&gt;&#xA;" OwnerUserId="102" LastEditorUserId="102" LastEditDate="2011-12-21T09:07:58.877" LastActivityDate="2011-12-21T09:07:58.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="489" PostTypeId="2" ParentId="485" CreationDate="2011-12-20T16:19:21.560" Score="15" Body="&lt;p&gt;I would aim to get the best of both worlds and code the &quot;user interface&quot; (that is, the framework of functions that the user of your library will call to describe the geometry and other properties of the problem) in Python to get the quick turnaround time, then write the simulation run time in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, I would probably mock even the simulation run time up in Python first, then replace it by C++ code piece by piece. Eventually you could consider having your Python code generate C++ source, to be compiled and linked to your runtime online, so that the actual simulation doesn't need to call into Python at all - only return the results at the end. The nice thing about this setup is that it's inherently agile: you start with the quickest and easiest working solution, you'll find out quickly what works and doesn't work, and once you have something you like you can start speeding it up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is how &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=dsolve/numeric&quot;&gt;Maple's ODE/DAE solver&lt;/a&gt; works, except using Maple instead of Python. Full disclosure: I work for them.)&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2011-12-20T16:19:21.560" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="490" PostTypeId="2" ParentId="480" CreationDate="2011-12-20T20:21:12.953" Score="-5" Body="&lt;p&gt;The simplest method is to use finite difference approximations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple two-point estimation is to compute the slope of a nearby secant line through the points (x,f(x)) and (x+h,f(x+h)).[1] Choosing a small number h, h represents a small change in x, and it can be either positive or negative. The slope of this line is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x+h)-f(x)\over h$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This expression is Newton's difference quotient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The slope of this secant line differs from the slope of the tangent line by an amount that is approximately proportional to h. As h approaches zero, the slope of the secant line approaches the slope of the tangent line. Therefore, the true derivative of f at x is the limit of the value of the difference quotient as the secant lines get closer and closer to being a tangent line&lt;/p&gt;&#xA;" OwnerUserId="433" LastEditorUserId="123" LastEditDate="2011-12-21T03:24:57.780" LastActivityDate="2011-12-21T03:24:57.780" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="491" PostTypeId="2" ParentId="290" CreationDate="2011-12-20T22:08:52.013" Score="11" Body="&lt;p&gt;Several nice replies already stated the Pros of finite element methods being flexible and powerful, here I will give another advantage of FEM, from Sobolev space and differential geometry point of view, is that the possibility of finite element space inheriting the physical continuity condition of the Sobolev spaces where the true solution lies in. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, Raviart-Thomas face element for plane elasticity, and mixed method for diffusion ; Nédélec edge element for computational electromagnetics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally the solution of a PDE, which is a differential $k$-form lying in the &quot;energy $L^2$-integrable&quot; space:&#xA;$$&#xD;&#xA;H\Lambda^k = \{\omega \in \Lambda^k: \omega \in L^2(\Lambda^k), \mathrm{d} \omega \in L^2(\Lambda^k)\}&#xD;&#xA;$$&#xA;where $\mathrm{d}$ is the exterior derivative, and we could build the de Rham cohomology around this space, which means we could construct an exact de Rham sequence like the following in 3D space: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\mathbb{R}^3 \xrightarrow{id}\, H(\mathbf{grad},\Omega) \xrightarrow{\nabla}\,&#xD;&#xA;H(\mathbf{curl},\Omega) \xrightarrow{\nabla \times}\,H(\mathrm{div},\Omega)&#xD;&#xA;\xrightarrow{\nabla \cdot}\, L^2(\Omega)&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the range of the operator is the null space of the next operator, and there are many nice properties about this, if we could build a finite element space to inherit this de Rham exact sequence, then the Galerkin method based on this finite element space will be stable and will converge to the real solution. And we could get the stability and approximation property of the interpolation operator simply by the commuting diagram from the de Rham sequence, plus we could build the a posteriori error estimation and adaptive mesh refining procedure based on this sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More about this please see Douglas Arnold's article in Acta Numerica: &quot;&#xA;&lt;a href=&quot;http://ima.umn.edu/~arnold/papers/acta.pdf&quot; rel=&quot;noreferrer&quot;&gt;Finite element exterior calculus,homological techniques, and applications&lt;/a&gt; &quot;&#xA;and &lt;a href=&quot;http://ima.umn.edu/~arnold/talks/feec-print.pdf&quot; rel=&quot;noreferrer&quot;&gt;a slide briefly introducing the idea&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-02-24T19:07:52.853" LastActivityDate="2012-02-24T19:07:52.853" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="492" PostTypeId="1" AcceptedAnswerId="493" CreationDate="2011-12-20T22:48:10.030" Score="4" ViewCount="615" Body="&lt;p&gt;I'm learning &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt; and trying to make the examples written in C.  However, when I use the provided makefile, I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/bin/ld: cannot find -lmkl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I check in the &lt;code&gt;mkl/lib&lt;/code&gt; directory, I find that none of the &lt;code&gt;.a&lt;/code&gt; or &lt;code&gt;.so&lt;/code&gt; files are called &lt;code&gt;libmkl&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do I need to recompile PETSc to have the built-in makefiles link to MKL?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Which library file should I link to if not &lt;code&gt;lmkl&lt;/code&gt;?&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2013-01-20T09:07:12.550" Title="PETSc's makefile system can't find MKL" Tags="|petsc|c|intel-mkl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="493" PostTypeId="2" ParentId="492" CreationDate="2011-12-20T23:10:48.120" Score="3" Body="&lt;p&gt;This seems like a linking problem with Intel's Math Kernel Library. You have many questions, so I will try to address each one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, when there is a linking problem, especially &quot;cannot find -lfoo&quot;, check the search directories. These are options that are usually specified with -L such as &quot;-L /path/to/libraries&quot;. In your case, the intel compilers should usually accept the &quot;-lmkl&quot; option which internally specifies the location to the libraries. &lt;i&gt;This does &lt;b&gt;not&lt;/b&gt; solve your problem, though.&lt;/i&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are usually PETSc, and you are getting this error, it means that you are trying to switch the compilers or BLAS/LAPACK library after building PETSc. &lt;b&gt;This is not supported&lt;/b&gt;. So, &lt;b&gt;yes&lt;/b&gt;, you will need to reconfigure and rebuild specifying the Intel compilers and MKL. See the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/installation.html#BLAS/LAPACK&quot; rel=&quot;nofollow&quot;&gt;installation help&lt;/a&gt; for more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many libraries that need to be linked so that you can use Intel's MKL. Intel even made a &lt;a href=&quot;http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/&quot; rel=&quot;nofollow&quot;&gt;helpful page&lt;/a&gt; to build the command-line link option.&lt;/p&gt;&#xA;" OwnerUserId="142" LastActivityDate="2011-12-20T23:10:48.120" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="494" PostTypeId="1" AcceptedAnswerId="495" CreationDate="2011-12-21T08:54:19.177" Score="13" ViewCount="2213" Body="&lt;p&gt;I'm working with the OpenFOAM C++ Computational Continuum Mechanics library (it can deal with fluid-solid interaction, MHD flows...) which uses arbitrary unstructured meshes. This was driven by the idea of using the advantage of fast generation (automatic usually) of unstructured meshes to simulate problems in complex geometries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, recently I've encountered another approach: octree adaptive carthesian meshes with cell &quot;cutting&quot;, where the agressive mesh refinement is used to describe a complex geometry. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the standpoint of the numerics, Carthesian meshes are much more accurate, so my question is: has anyone experience in using/implementing one or both of these approaches?  How do they compare agains each other? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm developing codes for two phase fluid flow and I noticed e.g. that the reconstruction of the field gradients can be easily made more accurate on Carthesian meshes, while unstructured mesh requires linear regression for abrupt changes in the field... &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="549" LastEditDate="2012-01-15T16:48:05.547" LastActivityDate="2012-01-15T19:15:21.707" Title="finite volume method: unstructured mesh vs octree adaptation + cell cutting" Tags="|mesh-generation|fluid-dynamics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="495" PostTypeId="2" ParentId="494" CreationDate="2011-12-21T19:33:19.683" Score="6" Body="&lt;p&gt;I think all the more modern FEM libraries (e.g. deal.II, libmesh, ...) use the octree-based scheme (or, to be more precise: oct-forests, with one tree starting from each cell of an unstructured coarse mesh). There are many advantages to this, primarily because you know the hierarchy of mesh cells. This implies that you can easily do coarsening, geometric multigrid, etc, all of which is incredibly difficult if you start with just a fine unstructured mesh. Furthermore, partitioning becomes an almost trivial problem. The downside of the approach is that if you have a complicated geometry, previously you only needed to describe it to the mesh generator whereas now you also have to describe it to the FEM code because you need the geometry when refining a cell that's located on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All else being equal, I think that the octree-based approach is far more flexible and useful than using one ginormous unstructured mesh.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-21T19:33:19.683" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="496" PostTypeId="2" ParentId="485" CreationDate="2011-12-21T19:39:01.390" Score="2" Body="&lt;p&gt;As a corollary to Allan's answer (that your own developer time is the most valuable resource): Use what others have already done. You say that you want to develop a library for computational continuum mechanics but there are already several of those that are so large that they will almost invariably already have everything that you need. Take a look at deal.II for example for everything that can be written as a finite element problem, OpenFOAM for fluid dynamics, or PyCLAW/CLAWPACK for hyperbolic problems. deal.II, for example, asks you to program in C++ but in reality the level of programming is often so high that one could say it's like a domain-specific language for FEM codes using C++ syntax.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-21T19:39:01.390" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="497" PostTypeId="2" ParentId="451" CreationDate="2011-12-21T19:48:51.717" Score="6" Body="&lt;p&gt;As a general rule, it's easy to understand why regular grids can't go much beyond 3 or 4-dimensional problems: in d dimensions, if you want to have a minimum of N points per coordinate direction, you'll get N^d points overall. Even for relatively nice functions in 1d, you need at least N=10 grid points to resolve them at all, so the overall number of points will be 10^d -- i.e. even on the biggest computers you're unlikely to go beyond d=9, and will probably not go much beyond &lt;em&gt;ever&lt;/em&gt;. Sparse grids can help in some circumstances if the solution function has certain properties, but in general, you'll have to live with the consequences of the curse of dimensionality and go with MCMC methods.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-21T19:48:51.717" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="498" PostTypeId="2" ParentId="326" CreationDate="2011-12-21T19:57:59.127" Score="5" Body="&lt;p&gt;In deal.II, we refine the mesh by throwing away old cells and replacing them with new ones.&#xA;But new ones are also placed into the memory holes left by deleted cells. All loops over all cells are then done in the order in which cells are encountered in memory to keep cache hits high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bigger question is how you store the data that defines cells. You could of course do&#xA;  struct Simplex {&#xA;    Vertex vertices[4];&#xA;    int    material_id;&#xA;    int    subdomain_id;&#xA;    bool   used;&#xA;    void * user_data;&#xA;  };&lt;/p&gt;&#xA;&#xA;&lt;p&gt;class Triangulation {&#xA;    Simplex *cells;&#xA;  };&#xA;but this is not cache efficient because most loops over all cells will only touch a subset of the data you store in your Simplex data structure and so only a fraction of the data that lands in the cache will actually be used. A better strategy is to do something like this:&#xA;  class Triangulation {&#xA;    Vertex *vertices;&#xA;    int    *material_ids;&#xA;    int    *subdomain_ids;&#xA;    bool   *used_flags;&#xA;    void*  *user_data;&#xA;  };&#xA;Because in loops over all cells subsequent iterations are likely to access the same subset of data that defines cells, read-ahead caches will only preload that data that you're actually going to use and will consequently lead to high cache hit rates.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-21T19:57:59.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="499" PostTypeId="2" ParentId="351" CreationDate="2011-12-21T20:02:23.990" Score="19" Body="&lt;p&gt;Of all the projects listed above, there are really only two heavy-weights that are extremely widely used (and for good reasons): PETSc and Trilinos. Both are professionally developed and have a large developer base. All the others are rather small projects compared to these two, and I would recommend going with them because (i) they will be supported for a long time to come and (ii) they are likely to already have all functionality you will ever need regarding linear algebra (and much more on top of that).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-21T20:02:23.990" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="500" PostTypeId="2" ParentId="470" CreationDate="2011-12-21T20:05:32.517" Score="3" Body="&lt;p&gt;I always tell my students to read a code from the bottom up: you start in main() and see what it calls. Typically this is only a small number of functions. Then you look into the functions called from main() which typically define the overall flow of the algorithm (time stepping loop, assembly, solver, output, etc). Go two or so levels deep to get an overview of the algorithm from 30,000 ft. The rest can often be gleaned from doxygen documentation etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as I said, the message is:&#xA;    Read code from the bottom up.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-21T20:05:32.517" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="501" PostTypeId="2" ParentId="461" CreationDate="2011-12-21T20:12:41.757" Score="6" Body="&lt;p&gt;As someone who has spent several years working with both, my perspective is that both packages aren't actually all that different. True, they use different languages, but they use it in very similar ways (both are object oriented, Trilinos makes little other use of C++ beyond using classes). Both support practically everything you will ever want to do with linear algebra (either through sub-packages or things they download on the fly, which from the user perspective doesn't make any difference). Finally, both have a good number of more obscure and probably not very widely used sub-packages (e.g. PETSc's mesh interface 'sieve', automatic differentiation in Trilinos, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, the appeal of Trilinos is two-fold:&#xA;- The number of obscure sub-packages in Trilinos is vastly larger; if I ever need something in direction X, I'm going to find it in Trilinos and it'll work with the rest of my code.&#xA;- Trilinos is far more conservative in their development strategy. PETSc renames things all the time and every release requires users to catch up with renamed functions, different libraries, etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-21T20:12:41.757" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="502" PostTypeId="2" ParentId="58" CreationDate="2011-12-21T22:20:47.870" Score="12" Body="&lt;p&gt;Besides &quot;multidimensional vs. single-dimensional&quot; and &quot;finite range vs. infinite range&quot;, an important categorization for methods is &quot;one specific type of oscillator (usually Fourier-type: $\sin(t)$, $\exp(it)$, etc, or Bessel-type: $J_0(t)$, etc.) vs. more general oscillator ($\exp(i g(t))$ or even more general oscillators $w(t)$)&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first, oscillatory integration methods focused on specific oscillators. As &lt;strong&gt;J. M.&lt;/strong&gt; said, prominent ones include Filon's method and the Clenshaw-Curtis method (these two are closely related) for finite range integrals, and series extrapolation based methods and the double-exponential method of Ooura and Mori for infinite range integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently, some general methods have been found. Two examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Levin's collocation-based method for any $\exp(i g(t))$ (&lt;a href=&quot;http://dx.doi.org/10.1090/S0025-5718-1982-0645668-7&quot;&gt;Levin 1982&lt;/a&gt;), or later for any oscillator $w(t)$ defined by a linear ODE (&lt;a href=&quot;http://dx.doi.org/10.1016/0377-0427%2894%2900118-9&quot;&gt;Levin 1996&lt;/a&gt; as linked by &lt;strong&gt;J. M.&lt;/strong&gt;). Mathematica uses &lt;a href=&quot;http://reference.wolfram.com/mathematica/tutorial/NIntegrateIntegrationRules.html#32844337&quot;&gt;Levin's method&lt;/a&gt; for integrals not covered by the more specialized rules.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Huybrechs and Vandewalle's method based on analytic continuation along a complex path where the integrand is non-oscillatory (&lt;a href=&quot;http://dx.doi.org/10.1137/050636814&quot;&gt;Huybrechs and Vandewalle 2006&lt;/a&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;No distinction is necessary between methods for finite and infinite range integrals for the more general methods, since a compactifying transformation can be applied to an infinite range integral, leading to a finite range oscillatory integral that can still be addressed with the general method, albeit with a different oscillator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Levin's method can be extended to multiple dimensions by iterating over the dimensions and other ways, but as far as I know all the methods described in literature so far have sample points that are an outer product of the one-dimensional sample points or some other thing that grows exponentially with dimension, so it rapidly gets out of hand. I'm not aware of more efficient methods for high dimensions; if any could be found that sample on a sparse grid in high dimensions it would be useful in applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Creating automatic routines for the more general methods may be difficult in most programming languages (C, Python, Fortran, etc) in which you would normally expect to program your integrand as a function/routine and pass it to the integrator routine, because the more general methods need to know the structure of the integrand (which parts look oscillatory, what type of oscillator, etc) and can't treat it as a &quot;black box&quot;.&lt;/p&gt;&#xA;" OwnerUserId="303" LastEditorUserId="303" LastEditDate="2011-12-23T03:59:43.330" LastActivityDate="2011-12-23T03:59:43.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="503" PostTypeId="1" AcceptedAnswerId="505" CreationDate="2011-12-22T02:17:09.660" Score="23" ViewCount="2643" Body="&lt;p&gt;Is there an $O(n^3+n^2 k)$ method to solve $k$ linear systems of the form $(D_i + A) x_i = b_i$ where $A$ is a fixed SPD matrix and $D_i$ are positive diagonal matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if each $D_i$ is scalar, it suffices to compute the SVD of $A$.  However, this breaks down for general $D$ due to lack of commutativity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Update&lt;/em&gt;: The answers so far are &quot;no&quot;.  Does anyone have any interesting intuition as to why?  A no answer means that there's no nontrivial way to compress the information between two noncommuting operators.  It isn't terribly surprisingly, but it'd be great to understand it better.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="276" LastEditDate="2011-12-24T19:40:44.443" LastActivityDate="2012-05-31T05:08:48.300" Title="Can diagonal plus fixed symmetric linear systems be solved in quadratic time after precomputation?" Tags="|linear-algebra|algorithms|performance|complexity|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="504" PostTypeId="2" ParentId="330" CreationDate="2011-12-22T02:57:29.430" Score="5" Body="&lt;p&gt;If you are interested in literate programming, have a look at &lt;a href=&quot;http://orgmode.org/worg/org-contrib/babel/index.html&quot; rel=&quot;noreferrer&quot;&gt;org-babel&lt;/a&gt;.  It is part of org-mode in Emacs and thus gives you a wide array of export options (LaTeX,PDF,HTML,ODT) for documentation.  Emacs can display images inside the buffer and let you write mathematical equations in LaTeX syntax so you don't have to limit yourself to plain text documentation.&lt;/p&gt;&#xA;" OwnerUserId="159" LastActivityDate="2011-12-22T02:57:29.430" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="505" PostTypeId="2" ParentId="503" CreationDate="2011-12-22T05:32:39.273" Score="21" Body="&lt;p&gt;The closest positive answers to your question that I could find is for sparse diagonal perturbations (see below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, I do not know of any algorithms for the general case, though there is a generalization of the technique you mentioned for scalar shifts from SPD matrices to all square matrices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given any square matrix $A$, there exists a Schur decomposition $A=U T U^H$, where $U$ is unitary and $T$ is upper triangular, and $A+\sigma I = U (T + \sigma I) U^H$ provides a Schur decomposition of $A + \sigma I$. Thus, your precomputation idea extends to all square matrices through the algorithm:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Compute $[U,T]=\mathrm{schur}(A)$ in at most $\mathcal{O}(n^3)$ work.&lt;/li&gt;&#xA;&lt;li&gt;Solve each $(A+\sigma I) x = b$ via $x := U (T +\sigma I)^{-1} U^H b$ in $\mathcal{O}(n^2)$ work (the middle inversion is simply back substitution).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This line of reasoning reduces to the approach you mentioned when $A$ is SPD since the Schur decomposition reduces to an EVD for normal matrices, and the EVD coincides with the SVD for Hermitian positive-definite matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Response to update:&lt;/em&gt; &#xA;Until I have a proof, which I do not, I refuse to claim that the answer is &quot;no&quot;. However, I can give some insights as to why it's hard, as well as another subcase where the answer is yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The essential difficulty is that, even though the update is diagonal, it is still in general full rank, so the primary tool for updating an inverse, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Woodbury_matrix_identity&quot;&gt;Sherman-Morrison-Woodbury formula&lt;/a&gt;, does not appear to help. Even though the scalar shift case is also full rank, it is an extremely special case since it commutes with every matrix, as you mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, if each $D$ was sparse, i.e., they each had $\mathcal{O}(1)$ nonzeros, then the Sherman-Morrison-Woodbury formula yields an $\mathcal{O}(n^2)$ solve with each pair $\{D,b\}$. For example, with a single nonzero at the $j$th diagonal entry, so that $D=\delta e_j e_j^H$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;[A^{-1}+\delta e_j e_j^H]^{-1} = A^{-1} - \frac{\delta A^{-1} e_j e_j^H A^{-1}}{1+\delta (e_j^H A^{-1} e_j)},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e_j$ is the $j$th &lt;a href=&quot;http://en.wikipedia.org/wiki/Standard_basis&quot;&gt;standard basis vector&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Another update:&lt;/em&gt; I should mention that I tried the $A^{-1}$ preconditioner that @GeoffOxberry suggested on a few random SPD $1000 \times 1000$ matrices using PCG and, perhaps not surprisingly, it seems to greatly reduce the number of iterations when $||D||_2/||A||_2$ is small, but not when it is $\mathcal{O}(1)$ or greater.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-05-31T05:08:48.300" LastActivityDate="2012-05-31T05:08:48.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="506" PostTypeId="2" ParentId="503" CreationDate="2011-12-22T07:37:40.563" Score="13" Body="&lt;p&gt;If $(D_{i} + A)$ is &lt;a href=&quot;http://en.wikipedia.org/wiki/Diagonally_dominant_matrix&quot;&gt;diagonally dominant&lt;/a&gt; for each $i$, then recent work by Koutis, Miller, and Peng (see &lt;a href=&quot;http://www.cs.cmu.edu/~jkoutis/research.html&quot;&gt;Koutis' website&lt;/a&gt; for work on symmetric diagonally dominant matrices) could be used to solve each system in $\mathcal{O}(n^2 \log(n))$ time (actually $\mathcal{O}(m\log(n))$ time, where $m$ is the maximum number of nonzero entries in $(D_{i} + A)$ over all $i$, so you could take advantage of sparsity as well). Then, the total running time would be $\mathcal{O}(n^2 \log(n) k)$, which is better than the $\mathcal{O}(n^3 k)$ approach of solving each system naïvely using dense linear algebra, but slightly worse than the quadratic run time you're asking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Significant sparsity in $(D_{i} + A)$ for all $i$ could be exploited by sparse solvers to yield an $\mathcal{O}(n^2 k)$ algorithm, but I'm guessing that if you had significant sparsity, then you would have mentioned it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also use $A^{-1}$ as a preconditioner to solve each system using iterative methods, and see how that works out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Response to update&lt;/em&gt;: @JackPaulson makes a great point from the standpoint of numerical linear algebra and algorithms. I will focus on computational complexity arguments instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computational complexity of the solution of linear systems and the computational complexity of matrix multiplication are essentially equal. (See &lt;a href=&quot;http://www.springer.com/new+&amp;amp;+forthcoming+titles+%28default%29/book/978-3-540-60582-9&quot;&gt;Algebraic Complexity Theory&lt;/a&gt;.) If you could find an algorithm that could compress the information between two non-commuting operators (ignoring the positive semidefinite part) and directly solve the collection of systems you're proposing in quadratic time in $n$, then it's likely that you could use such an algorithm to make inferences about faster matrix multiplication. It's difficult to see how positive semidefinite structure could be used in a dense, direct method for linear systems to decrease its computational complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like @JackPaulson, I'm unwilling to say that the answer is &quot;no&quot; without a proof, but given the connections above, the problem is very difficult and of current research interest. The best you could do from an asymptotic standpoint without leveraging special structure is an improvement on the Coppersmith and Winograd algorithm, yielding a $\mathcal{O}(n^{\alpha}k)$ algorithm, where $\alpha \approx 2.375$. That algorithm would be difficult to code, and would likely be slow for small matrices, because the constant factor preceding the asymptotic estimate is probably huge relative to Gaussian elimination.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2011-12-24T19:56:29.973" LastActivityDate="2011-12-24T19:56:29.973" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="507" PostTypeId="1" AcceptedAnswerId="509" CreationDate="2011-12-22T16:30:51.353" Score="28" ViewCount="25023" Body="&lt;p&gt;What considerations should I be making when choosing between BFGS and conjugate gradient for optimization? The function I am trying to fit with these variables are exponential functions; however, the actual objective function involves integration, among other things, and is very costly if that helps at all.&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="4244" LastEditDate="2013-11-15T10:01:40.860" LastActivityDate="2021-06-05T05:56:03.557" Title="BFGS vs. Conjugate Gradient Method" Tags="|optimization|conjugate-gradient|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="508" PostTypeId="2" ParentId="469" CreationDate="2011-12-22T17:22:52.023" Score="5" Body="&lt;p&gt;if you are completely unfamiliar with C, but are using matlab as your base starting point, i would actually recommend fortran90 over C. the languages are so incredibly similar it is scary (as far as syntax goes at least)&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2011-12-22T17:22:52.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="509" PostTypeId="2" ParentId="507" CreationDate="2011-12-22T18:36:13.560" Score="18" Body="&lt;p&gt;J.M. is right about storage. BFGS requires an approximate Hessian, but you can initialize it with the identity matrix and then just calculate the rank-two updates to the approximate Hessian as you go, as long as you have gradient information available, preferably analytically rather than through finite differences. BFGS is a quasi-Newton method, and will converge in fewer steps than CG, and has a little less of a tendency to get &quot;stuck&quot; and require slight algorithmic tweaks in order to achieve significant descent for each iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, CG requires matrix-vector products, which may be useful to you if you can calculate directional derivatives (again, analytically, or using finite differences). A finite difference calculation of a directional derivative will be much cheaper than a finite difference calculation of a Hessian, so if you choose to construct your algorithm using finite differences, just calculate the directional derivative directly. This observation, however, doesn't really apply to BFGS, which will calculate approximate Hessians using inner products of gradient information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of convergence rates, if $n$ is the number of decision variables in your problem, then $n$ CG iterations approximately equals one step of Newton's method. BFGS is a quasi-Newton method, but the same sort of observation should hold; you're likely to get convergence in fewer iterations with BFGS unless there are a couple CG directions in which there is a lot of descent, and then after a few CG iterations, you restart it. CG-like methods are cheaper if matrix-vector products are cheap and your problem is so large that storing the Hessian is difficult or impossible. BFGS involves some more vector-vector products to update its approximate Hessian, so each BFGS iteration will be more expensive, but you'll require fewer of them to reach a local minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would compare the two algorithms on a small test problem for your application if you know that storage won't be an issue. Without knowing the particular specifics of your problem, my guess is that BFGS will be faster, but you should really test the two algorithms to get a better idea of which will work better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, a word about automatic differentiation: having some experience with an in house automatic differentiation (AD) facility for Fortran (&lt;a href=&quot;http://yoric.mit.edu/DAEPACK&quot;&gt;DAEPACK&lt;/a&gt;), I can tell you that AD tools are often finicky. They may not necessarily be able to differentiate the code that they generate themselves. There are two types of AD tools: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;source-to-source AD tools&lt;/li&gt;&#xA;&lt;li&gt;operator overloading AD tools&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Source-to-source tools are essentially modified compilers that take source code you've written, parse it, and automatically generate new source code that computes the gradient of functions in your source code. Operator overloading AD tools require you to use the overloaded AD operators in your source code so that derivatives can be calculated, which would require additional effort on your part to calculate analytical derivatives with AD.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2011-12-22T18:36:13.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="510" PostTypeId="2" ParentId="327" CreationDate="2011-12-22T20:09:46.107" Score="2" Body="&lt;p&gt;I would also go for the evaluation of the logarithm of your expression log(binom(100,i)) + i*log(a)+(100-i)*log(1-a) and then take the exponential of the result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you tell us why you need multiple precision arithmetic? What is the accuracy that you need on the final result? I think you can get quite far with good series approximations, certainly when a is close to zero...&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2011-12-22T20:09:46.107" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="511" PostTypeId="2" ParentId="58" CreationDate="2011-12-22T21:06:02.310" Score="2" Body="&lt;p&gt;You could also check the work of Marnix Van Daele and co-authors. See for example &lt;a href=&quot;http://users.ugent.be/~mvdaele/voordrachten/toronto2011klein.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;http://users.ugent.be/~mvdaele/voordrachten/toronto2011klein.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="127" LastEditDate="2013-05-14T10:00:15.577" LastActivityDate="2013-05-14T10:00:15.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="512" PostTypeId="2" ParentId="327" CreationDate="2011-12-22T23:36:10.043" Score="1" Body="&lt;p&gt;Depending on the accuracy you need, think about just approximating the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least for the part {100 over i} you should be able to get a good approximation using techniques like Sterling's formula (http://en.wikipedia.org/wiki/Sterling's_approximation). The remainder can be computed relatively easily.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-22T23:36:10.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="513" PostTypeId="1" AcceptedAnswerId="514" CreationDate="2011-12-23T01:52:37.627" Score="30" ViewCount="15322" Body="&lt;p&gt;What can go wrong when using preconditoned Krylov methods from &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/index.html&quot;&gt;KSP&lt;/a&gt; (&lt;a href=&quot;http://mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt;'s linear solver package) to solve a sparse linear system such as those obtained by discretizing and linearizing partial differential equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What steps can I take to determine what is going wrong for my problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What changes can I make to successfully and efficiently solve my linear system?&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="81" LastEditDate="2012-01-07T01:29:00.020" LastActivityDate="2012-01-07T01:29:00.020" Title="Why is my iterative linear solver not converging?" Tags="|linear-algebra|petsc|preconditioning|krylov-method|iterative-method|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="514" PostTypeId="2" ParentId="513" CreationDate="2011-12-23T02:13:26.267" Score="30" Body="&lt;h2&gt;Initial advice&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Always run with &lt;code&gt;-ksp_converged_reason -ksp_monitor_true_residual&lt;/code&gt; when trying to learn why a method is not converging.&lt;/li&gt;&#xA;&lt;li&gt;Make the problem size and number of processes as small as possible to demonstrate the failure. You often gain insight by determining what small problems exhibit the behavior that is causing your method to break down and the turn-around time is reduced. Additionally, there are some investigation techniques that can only be used for small systems.&lt;/li&gt;&#xA;&lt;li&gt;If the issue only arises after a large number of time steps, continuation steps, or nonlinear solve steps, consider writing the model state out when failure occurs so that you can experiment quickly.&lt;/li&gt;&#xA;&lt;li&gt;Alternatively, especially if your software does not have checkpoint capability, use &lt;code&gt;-ksp_view_binary&lt;/code&gt; or &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatView.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;MatView()&lt;/code&gt;&lt;/a&gt; to save the linear system, then use the code at &lt;code&gt;$PETSC_DIR/src/ksp/ksp/examples/tutorials/ex10.c&lt;/code&gt; to read in the matrix and solve it (possibly with a different number of processes). This requires an assembled matrix, so it's usefulness can be somewhat limited.&lt;/li&gt;&#xA;&lt;li&gt;There are many possible solver choices (e.g. an infinite number available at the command line in PETSc due to an arbitrary number of levels of composition), see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/81/which-sparse-linear-system-solver-to-use/&quot;&gt;this question&lt;/a&gt; for general advice on choosing linear solvers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Common reasons for KSP not converging&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The equations are singular by accident (e.g. forgot to impose boundary conditions). Check this for a small problem using &lt;code&gt;-pc_type svd -pc_svd_monitor&lt;/code&gt;. Also try a direct solver with &lt;code&gt;-pc_type lu&lt;/code&gt; (via a third-party package in parallel, e.g. &lt;code&gt;-pc_type lu -pc_factor_mat_solver_package superlu_dist&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;The equations are intentionally singular (e.g. constant null space), but the Krylov method was not informed, see &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSetNullSpace.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;KSPSetNullSpace()&lt;/code&gt;&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The equations are intentionally singular and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSetNullSpace.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;KSPSetNullSpace()&lt;/code&gt;&lt;/a&gt; was used, but the right hand side is not consistent. You may have to call &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatNullSpaceRemove.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;MatNullSpaceRemove()&lt;/code&gt;&lt;/a&gt; on the right hand side before calling &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSolve.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;KSPSolve()&lt;/code&gt;&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The equations are indefinite so that standard preconditioners don't work. Usually you will know this from the physics, but you can check with -ksp_compute_eigenvalues &lt;code&gt;-ksp_gmres_restart 1000 -pc_type none&lt;/code&gt;. For simple saddle point problems, try &lt;code&gt;-pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_detect_saddle_point&lt;/code&gt;. See the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf&quot; rel=&quot;noreferrer&quot;&gt;User's Manual&lt;/a&gt; and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCFIELDSPLIT.html&quot; rel=&quot;noreferrer&quot;&gt;PCFIELDSPLIT man page&lt;/a&gt; for more details. For more difficult problems, read the literature to find robust methods and ask here (or &lt;code&gt;petsc-users@mcs.anl.gov&lt;/code&gt; or &lt;code&gt;petsc-maint@mcs.anl.gov&lt;/code&gt;) if you want advice about how to implement them. For example, see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/40/what-is-a-scalable-preconditioner-for-high-frequency-helmholtz&quot;&gt;this question&lt;/a&gt; for high frequency Helmholtz. For modest problem sizes, see if you can live with just using a direct solver.&lt;/li&gt;&#xA;&lt;li&gt;If the method converges in preconditioned residual, but not in true residual, the preconditioner is likely singular or nearly so. This is common for saddle point problems (e.g. incompressible flow) or strongly nonsymmetric operators (e.g. low-Mach hyperbolic problems with large time steps).&lt;/li&gt;&#xA;&lt;li&gt;The preconditioner is too weak or is unstable. See if &lt;code&gt;-pc_type asm -sub_pc_type lu&lt;/code&gt; improves the convergence rate. If GMRES is losing too much progress in the restart, see if longer restarts help &lt;code&gt;-ksp_gmres_restart 300&lt;/code&gt;. If a transpose is available, try &lt;code&gt;-ksp_type bcgs&lt;/code&gt; or other methods that do not require a restart. (Note that convergence with these methods is frequently erratic.)&lt;/li&gt;&#xA;&lt;li&gt;The preconditioning matrix may not be close to the (possibly unassembled) operator. Try solving with a direct solver, either in serial with &lt;code&gt;-pc_type lu&lt;/code&gt; or in parallel using a third-party package (e.g. &lt;code&gt;-pc_type lu -pc_factor_mat_solver_package superlu_dist&lt;/code&gt;, or &lt;code&gt;mumps&lt;/code&gt;). The method should converge in one iteration if the matrices are the same, and in a &quot;small&quot; number of iterations otherwise. Try &lt;code&gt;-snes_type test&lt;/code&gt; to check the matrices if solving a nonlinear problem.&lt;/li&gt;&#xA;&lt;li&gt;The preconditioner is nonlinear (e.g. a nested iterative solve), try &lt;code&gt;-ksp_type fgmres or -ksp_type gcr&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;You are using geometric multigrid, but some equations (often boundary conditions) are not scaled compatibly between levels. Try &lt;code&gt;-pc_mg_galerkin&lt;/code&gt; to algebraically construct a correctly scaled coarse operator or make sure that all the equations are scaled in the same way if you want to use rediscretized coarse levels.&lt;/li&gt;&#xA;&lt;li&gt;The matrix is very ill-conditioned. Check the condition number using the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/34/how-can-i-estimate-the-condition-number-of-a-large-sparse-matrix-using-petsc&quot;&gt;methods described here&lt;/a&gt;.&#xA;Try to improve it by choosing the relative scaling of components/boundary conditions.&#xA;Try &lt;code&gt;-ksp_diagonal_scale -ksp_diagonal_scale_fix&lt;/code&gt;. &#xA;Perhaps change the formulation of the problem to produce more friendly algebraic equations. If you cannot correct the scaling, you may need to use a direct solver.&lt;/li&gt;&#xA;&lt;li&gt;The matrix is nonlinear (e.g. evaluated using finite differencing of a nonlinear function). Try different differencing parameters (e.g. &lt;code&gt;-mat_mffd_type ds&lt;/code&gt;). Try using higher precision to make the differencing more accurate, &lt;code&gt;./configure --with-precision=__float128 --download-f2cblaslapack&lt;/code&gt;. Check if it converges in &quot;easier&quot; parameter regimes.&lt;/li&gt;&#xA;&lt;li&gt;A symmetric method is being used for a non-symmetric problem.&lt;/li&gt;&#xA;&lt;li&gt;Classical Gram-Schmidt is becoming unstable, try &lt;code&gt;-ksp_gmres_modifiedgramschmidt&lt;/code&gt; or use a method that orthogonalizes differently, e.g. &lt;code&gt;-ksp_type gcr&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2011-12-23T18:31:37.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="515" PostTypeId="2" ParentId="507" CreationDate="2011-12-23T04:12:05.353" Score="26" Body="&lt;p&gt;The associated cost of BFGS may be brought more in line with CG if you use the &lt;a href=&quot;http://dx.doi.org/10.1090/S0025-5718-1980-0572855-7&quot;&gt;limited memory&lt;/a&gt; &lt;a href=&quot;http://en.wikipedia.org/wiki/L-BFGS&quot;&gt;variants&lt;/a&gt; rather than the full-storage BFGS.  This computes the BFGS update for the last $m$ updates efficiently by a series of rank-one updates without needing to store more than the last $m$ solutions and gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, BFGS with a lot of updates stores information too far away from the current solution to be really useful in approximating the non-lagged Jacobian, and you can actually lose convergence if you store too much.  There are &quot;memoryless&quot; variants of BFGS that look a lot like &lt;a href=&quot;https://www.math.lsu.edu/~hozhang/papers/cgsurvey.pdf&quot;&gt;nonlinear conjugate gradients&lt;/a&gt; (see the final update described for one of these) for just these reasons.  Therefore, if you're willing to do L-BFGS rather than BFGS, the memory issues disappear and the &lt;a href=&quot;http://dx.doi.org/10.1137/0716059&quot;&gt;methods are related&lt;/a&gt;.  Anecdotal evidence points to restarting being a tricky issue, as it is sometimes &lt;a href=&quot;http://people.sc.fsu.edu/~inavon/5420a/qn.pdf&quot;&gt;unnecessary&lt;/a&gt; and sometimes very necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your choice between the two also depends heavily on the problems you are interested in.  If you have the resources, you can try both for your problems and decide which works better.  For example, I personally don't do optimization with these algorithms, but instead care about the solution of systems of nonlinear equations.  For these I have found that NCG works better and is easier to perform nonlinear preconditioning on. BFGS is more robust.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Frankly, my favorite method for these types of things is &lt;a href=&quot;http://arxiv.org/abs/1106.4426&quot;&gt;N-GMRES&lt;/a&gt;.  This is especially true if your gradient evaluation is very expensive, as in my experience it gives you the most bang for your buck by solving a small minimization problem on the last $m$ iterates to construct a new, lower-residual solution.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="127" LastEditDate="2011-12-23T05:03:57.303" LastActivityDate="2011-12-23T05:03:57.303" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="516" PostTypeId="1" AcceptedAnswerId="519" CreationDate="2011-12-23T10:18:45.153" Score="3" ViewCount="1317" Body="&lt;p&gt;We have a 3D (volume) unstructured, possibly hybrid, degenerative irregular mesh data structure that we are capable of generating (mostly composed of hexahedra and general polyhedra, using a mix of CSG and b-rep) and a set of triangulated surfaces that we know intersect the mesh. We would like to split the volume mesh by the set of intersecting triangulated surfaces and change the topology of the mesh accordingly. These meshes have several uses including computation and visualization. Also the geometries they both represent are highly irregular and complicated, so oct-trees or oct-forests are not being strongly considered. We already have an implementation to detect such intersections in the whole volume. The triangulated surfaces can intersect themselves inside the volume too and split the mesh in another number of topological regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we are looking for is an algorithm to split the mesh accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you know any library (whatever language, but if commercial ones, implemented in C++) or papers that would deal with this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are currently looking into using &lt;a href=&quot;http://en.wikipedia.org/wiki/Marching_cubes&quot; rel=&quot;nofollow&quot;&gt;Marching cubes&lt;/a&gt; and have started some development towards that effort, but are interested in existing implementations or research on the topic. We have also been looking into CGAL but it does not fit our purpose. We know OpenFOAM by name but we do not know if it can help us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help!&lt;/p&gt;&#xA;" OwnerUserId="455" LastEditorUserId="549" LastEditDate="2012-01-15T16:46:57.997" LastActivityDate="2012-01-15T16:46:57.997" Title="Looking for a library or algorithms to perfom clipping 3D unstructured meshes by a set of surfaces" Tags="|finite-element|libraries|geometry|mesh-generation|fluid-dynamics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="517" PostTypeId="1" CreationDate="2011-12-23T14:03:02.553" Score="4" ViewCount="212" Body="&lt;p&gt;I am looking for a reference - paper, book, discussion, anything that has a mathematical proof for stability of the conjugate gradient method in floating point arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something similar for modified Gram-Schmidt would be as good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="459" LastEditorUserId="127" LastEditDate="2012-02-16T23:28:13.910" LastActivityDate="2012-02-17T06:25:41.557" Title="Looking for a mathematical proof of stability in floating point arithmetic of CG - any reference?" Tags="|linear-algebra|floating-point|stability|conjugate-gradient|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="518" PostTypeId="2" ParentId="422" CreationDate="2011-12-23T14:11:20.857" Score="3" Body="&lt;p&gt;NIST's &lt;a href=&quot;http://fire.nist.gov/fds/&quot; rel=&quot;nofollow&quot;&gt;Fire Dynamics Simulator&lt;/a&gt; (FDS) sounds like what you want. FDS is a low Mach number flow solver. The density can change, but acoustic effects and shocks are neglected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FDS is fairly well documented, though, I admit I have not looked in the main flow solver routines. I also do not know much about how FDS handles combustion chemistry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest downloading the latest code from &lt;a href=&quot;https://code.google.com/p/fds-smv/&quot; rel=&quot;nofollow&quot;&gt;FDS's Google Code site&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="437" LastActivityDate="2011-12-23T14:11:20.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="519" PostTypeId="2" ParentId="516" CreationDate="2011-12-23T16:08:55.547" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://vtk.org&quot; rel=&quot;nofollow&quot;&gt;VTK&lt;/a&gt; has many utilities for cutting meshes with planes and implicitly-defined surfaces. It's possible that would work for you. To use it, you would probably feed your whole mesh into a &lt;a href=&quot;http://www.vtk.org/doc/nightly/html/classvtkUnstructuredGrid.html&quot; rel=&quot;nofollow&quot;&gt;vtkUnstructuredGrid&lt;/a&gt; (or similar) and have VTK do all the cutting. I think you will have a harder time if you want control over how topology changes when a cell is cut.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lower level and lightweight tool for geometric tasks relating to cutting cells is &lt;a href=&quot;http://math.lbl.gov/voro++/&quot; rel=&quot;nofollow&quot;&gt;Voro++&lt;/a&gt;. I have used it for some mesh preprocessing tasks and computing sizes of dual cells; it worked well and was unintrusive. Voro++ would be a good choice if you want to cut single cells or collections of cells, then use your own logic for changing mesh topology due to the cut.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned &lt;a href=&quot;http://cgal.org&quot; rel=&quot;nofollow&quot;&gt;CGAL&lt;/a&gt;, but did not state how it failed to meet your needs. Perhaps you could specify what you are looking for more precisely?&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2011-12-23T16:08:55.547" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="520" PostTypeId="2" ParentId="513" CreationDate="2011-12-23T18:11:52.260" Score="18" Body="&lt;p&gt;My advice to students is to try a direct solver in these cases. The reason is that there are two classes of reasons why a solver may not converge: (i) the matrix is wrong, or (ii) there is a problem with the solver/preconditioner. Direct solvers almost always yield something that you can compare with the solution you expect, so if the answer of the direct solver looks correct, then you know the problem is with the iterative solver/precondition. On the other hand, if the answer looks wrong, the problem is with assembling the matrix and right hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I typically just use UMFPACK as the direct solver. I'm sure it's simple to try something similar with PETSC.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2011-12-23T18:11:52.260" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="521" PostTypeId="2" ParentId="83" CreationDate="2011-12-23T19:36:33.350" Score="18" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/3ihSx.png&quot; alt=&quot;APM Python&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: see the new &lt;a href=&quot;http://gekko.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;GEKKO package&lt;/a&gt; that we just released.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://apmonitor.com/wiki/index.php/Main/PythonApp&quot; rel=&quot;noreferrer&quot;&gt;APM Python&lt;/a&gt; is a free optimization toolbox that has interfaces to APOPT, BPOPT, IPOPT, and other solvers.  It provides first (Jacobian) and second (Hessian) information to the solvers and provides an optional web-interface to view results. The APM Python client is installed with pip:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; pip install APMonitor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It can also be installed in a Python script with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;try:&#xA;    from APMonitor.apm import *&#xA;except:&#xA;    # Automatically install APMonitor&#xA;    import pip&#xA;    pip.main(['install','APMonitor'])&#xA;    from APMonitor.apm import *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We've done a couple benchmark tests and found that the combination of APOPT (active set method) and IPOPT (interior point method) can solve a large percentage of benchmark problems.  There are a number of example problems that are included with the download zip file.  The one that you'll probably want to start with is the Hock Schittkowski #71 problem.  It is the simplest example and demonstrates how to solve constrained optimization problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a browser interface and an API to Python / MATLAB.  The API to Python is a single script (apm.py) that is available for download from the apmonitor.com homepage.  Once the script is loaded into a Python code, it gives the ability to solve problems of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Nonlinear equations&lt;/li&gt;&#xA;&lt;li&gt;Mixed integer nonlinear programming&lt;/li&gt;&#xA;&lt;li&gt;Differential and algebraic equations&lt;/li&gt;&#xA;&lt;li&gt;Least squares model fitting&lt;/li&gt;&#xA;&lt;li&gt;Moving horizon estimation&lt;/li&gt;&#xA;&lt;li&gt;Nonlinear model predictive control&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the new user, the APM Python software has a Google Groups forum where a user can post questions.  There are webinars that showcase optimization problems in operations research and engineering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is an example of an optimization problem (hs71.apm).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Model&#xA;  Variables&#xA;    x[1] = 1, &amp;gt;=1, &amp;lt;=5&#xA;    x[2] = 5, &amp;gt;=1, &amp;lt;=5&#xA;    x[3] = 5, &amp;gt;=1, &amp;lt;=5&#xA;    x[4] = 1, &amp;gt;=1, &amp;lt;=5&#xA;  End Variables&#xA;&#xA;  Equations&#xA;    x[1] * x[2] * x[3] * x[4] &amp;gt; 25&#xA;    x[1]^2 + x[2]^2 + x[3]^2 + x[4]^2 = 40&#xA;&#xA;    minimize  x[1] * x[4] * (x[1]+x[2]+x[3]) + x[3]&#xA;  End Equations&#xA;End Model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The optimization problem is solved with the following Python script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from APMonitor.apm import *&#xA;server = 'http://byu.apmonitor.com'&#xA;&#xA;# Application name&#xA;app = 'eqn'&#xA;&#xA;# Clear previous application&#xA;apm(server,app,'clear all')&#xA;&#xA;# Load model file&#xA;apm_load(server,app,'hs71.apm')&#xA;&#xA;# Option to select solver (1=APOPT, 2=BPOPT, 3=IPOPT)&#xA;apm_option(server,app,'nlc.solver',3)&#xA;&#xA;# Solve on APM server&#xA;solver_output = apm(server,app,'solve')&#xA;&#xA;# Display solver output&#xA;print(solver_output)&#xA;&#xA;# Retrieve results&#xA;results = apm_sol(server,app)&#xA;&#xA;# Display results&#xA;print('--- Results of the Optimization Problem ---')&#xA;print(results)&#xA;&#xA;# Display Results in Web Viewer &#xA;url = apm_var(server,app)&#xA;print(&quot;Opened Web Viewer: &quot; + url)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;APM Python is a free web-service for optimization. The optimization problems are solved on remote servers and results are returned to the local Python script. An APMonitor local server is also available for download so that an Internet connection is not required (&lt;a href=&quot;http://apmonitor.com/wiki/index.php/Main/APMonitorServer&quot; rel=&quot;noreferrer&quot;&gt;Download server&lt;/a&gt;). We recently added parallel processing support for both MATLAB and Python. The Python module is compatible with Python 2.7 or Python 3+.&lt;/p&gt;&#xA;" OwnerUserId="461" LastEditorUserId="906" LastEditDate="2018-05-04T14:28:37.047" LastActivityDate="2018-05-04T14:28:37.047" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="522" PostTypeId="2" ParentId="517" CreationDate="2011-12-23T23:10:48.840" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/The_Lanczos_and_conjugate_gradient_algor.html?id=kvW1Q2CRJRQC&quot; rel=&quot;nofollow&quot;&gt;The Lanczos and Conjugate Gradient algorithms: From Theory to Finite Precision Computations&lt;/a&gt; presents a thorough analysis of CG in finite precision arithmetic.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="276" LastEditDate="2012-02-17T06:25:41.557" LastActivityDate="2012-02-17T06:25:41.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="525" PostTypeId="2" ParentId="461" CreationDate="2011-12-26T19:41:56.577" Score="2" Body="&lt;p&gt;PETSc can very easily be used with Fortran and the documentation/examples are fairly good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trilinos to me looked confusing (with all its sub-packages and naming scheme) and Fortran support was spotty (at least when I looked at it a few years back).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two ways to interact with Trilinos from Fortran: (1) pass raw data into a C++ wrapper that makes all the calls to Trilinos packages for your [such wrappers exist for several packages] or (2) use the new ForTrilinos interfaces that heavily exercise the object-oriented features of Fortran 2003.  Most of the features ForTrilinos requires are available in most compilers. the compiler implementations of these features is buggy but rapidly improving. ForTrilinos currently builds with the IBM and NAG compilers.  Current or near-future releases of the Cray, Intel, and Portland Group compilers all nominally support the requisite features modulo bug fixes.  The upcoming GCC 4.7.0 release will have all the requisite features but one, so wide compiler support is not too far in the future.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="688" LastEditDate="2012-01-27T23:13:43.623" LastActivityDate="2012-01-27T23:13:43.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="526" PostTypeId="2" ParentId="287" CreationDate="2011-12-26T20:37:21.557" Score="1" Body="&lt;p&gt;In short, the heuristics of whether to choose $h$-refinement or $p$-refinement is the question &quot;which benefits most in approximating the true solution&quot;: When the true solution is smooth, we could increase the order of the basis in the finite dimensional approximation space to get higher order convergence, however, when the true solution is not smooth, increasing the order of the polynomial order wouldn't benefit much, we simply use the lowest order element and do $h$-refinement, in other word, make the mesh finer and finer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, it really depends on the regularity of the true solution, and the question &quot;How do we extract the information of the regularity of the TRUE SOLUTION from the FINITE ELEMENT SOLUTION?&quot;. There are several ways to do it here I list two of them:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;a posteriori error estimation&quot;, we construct error estimator that uses only the quantities from the computation to estimate the local error and the local regularity, thus indicates where should we refine and the order of the element we should use.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;some a priori knowledge of the solution, from both the data and the domain. Data-wise, for example, consider the following non-smooth coefficient diffusion equation:&#xA;$$&#xD;&#xA;\left\{&#xD;&#xA;\begin{aligned}&#xD;&#xA;-\mathrm{div}(A\nabla u) &amp;amp;= f &amp;amp;\text{ in } \Omega&#xD;&#xA;\\&#xD;&#xA;u&amp;amp;= g &amp;amp;\text{ on } \partial \Omega&#xD;&#xA;\end{aligned} &#xD;&#xA;\right.&#xD;&#xA;$$&#xA;the regularity of the solution depends on $f,g$ and somewhat more importantly $A$, when $f\in L^2(\Omega)$ or $H^{-1}(\Omega)$, $g\in H^{1/2}(\partial \Omega)$, which are the usual assumptions for this equation, now if $A$ is non-smooth, we know that the solution has most an $H^{1+\epsilon}$ regularity, not the $H^2$ regularity for Poisson equation because of the harmonic lifting. Then we shouldn't use the quadratic element to approximate it since it doesn't benefit much from those extra DoFs. Domain-wise, we also know the regularity of the solution also depends on the geometry of the domain, or rather, what the boundary of the computational domain is like, does it have re-entrant corners? is it Lipschitz? does it satisfy interior ball condition? All these constraints would affect the local regularity of the solution, when the regularity of the true solution in some part of the computation domain is not good, use $h$-refinement. A famous toy problem would be the FEM on the Fichera corner.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;TL,DR: The heuristics for choosing $h$ or $p$ is the local regularity of the true solution: if the solution is smooth, use $p$-refinement; not smooth? use $h$-refinement.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2011-12-27T03:44:42.547" LastActivityDate="2011-12-27T03:44:42.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="527" PostTypeId="1" AcceptedAnswerId="528" CreationDate="2011-12-26T21:32:50.493" Score="25" ViewCount="789" Body="&lt;p&gt;Given a real function of real variables, is there software available that can automatically generate numerically-accurate code to calculate the function over all inputs on a machine equipped with IEEE 754 arithmetic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if the real function to be evaluated were:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sFd8F.gif&quot; alt=&quot;f(a, b, c) = \frac{-b - \sqrt{b^2 - 4ac}}{2a}&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The software would consider catastrophic cancellation and possibly output table lookups for certain sets of inputs to avoid a loss in computational accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is there software that can generate a pure table-based lookup routine to calculate a given function to high accuracy?&lt;/p&gt;&#xA;" OwnerUserId="472" LastActivityDate="2013-03-03T05:56:19.803" Title="Is there software that can autogenerate numerically-accurate floating point C routines from symbolic formulae?" Tags="|software|floating-point|accuracy|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="528" PostTypeId="2" ParentId="527" CreationDate="2011-12-26T23:03:39.213" Score="25" Body="&lt;p&gt;The best solution that I know of is to program the symbolic expressions in &lt;a href=&quot;http://reference.wolfram.com/mathematica/tutorial/GeneratingCAndFortranExpressions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematica&lt;/a&gt;, &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=CodeGeneration/C&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maple&lt;/a&gt;, or &lt;a href=&quot;http://docs.sympy.org/0.7.1/modules/utilities/codegen.html?highlight=codegen#sympy.utilities.codegen&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt;; all of the links go directly to the code generation documentation. All of the programs above can generate code in C or Fortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of the programs above mentions accuracy in IEEE 754 arithmetic; in general, it would be difficult to anticipate all sources of catastrophic cancellation, as @dmckee notes. It's hard to replace human expertise in numerical analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To provide a concrete example, consider calculating the trigonometric functions to high precision for arbitrary inputs in $[0, 2\pi]$. There are many strategies for doing so, some even hardware dependent, as see in the Wikipedia article &lt;a href=&quot;http://en.wikipedia.org/wiki/Generating_trigonometric_tables&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trigonometric Tables&lt;/a&gt;. All of the  algorithms require ingenuity and numerical analysis, even algorithms that depend on lookup tables and Taylor series or interpolation (see the Wikipedia article &lt;a href=&quot;http://en.wikipedia.org/wiki/Table-maker%27s_dilemma#The_table-maker.27s_dilemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Table-Maker's Dilemma&lt;/a&gt;). For more detail, see the related Stack Overflow question &lt;a href=&quot;https://stackoverflow.com/questions/345085/how-do-trigonometric-functions-work/345117#345117&quot;&gt;How do Trigonometric Functions work?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Software that generated code or routines to calculate arbitrary functions to high accuracy would not only need to be aware of cancellation errors, but also series approximants (Taylor, Padé, Chebyshev, rational, etc.) for calculating functions that are not defined in terms of a finite number of additions, subtractions, multiplications, divisions, and bit shifts. (See &lt;a href=&quot;http://en.wikipedia.org/wiki/Approximation_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximation Theory&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2011-12-27T21:09:37.410" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="529" PostTypeId="2" ParentId="527" CreationDate="2011-12-28T03:37:23.430" Score="4" Body="&lt;p&gt;Not in general, I can safely say the implementor of the code generator in SymPy didn't even try =P.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paolo Bientinesi developed a method for generating stability proofs of linear algebra algorithms, which are generated using Robert van de Geijn's FLAME notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://www.cs.utexas.edu/users/flame/pubs/Stability_SIMAX.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;, or a &lt;a href=&quot;http://www.aices.rwth-aachen.de:8080/aices/preprint/documents/AICES-2008-7.pdf&quot; rel=&quot;nofollow&quot;&gt;longer, working note version&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="276" LastEditDate="2011-12-28T03:59:42.433" LastActivityDate="2011-12-28T03:59:42.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="530" PostTypeId="1" AcceptedAnswerId="532" CreationDate="2011-12-28T09:15:17.197" Score="20" ViewCount="6177" Body="&lt;p&gt;What is the best (scalability and efficiency) algorithms for generating unstructured quad meshes in 2D? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find a good unstructured quad mesh-generator? (open-source preferred)&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="549" LastEditDate="2012-01-16T14:19:39.823" LastActivityDate="2022-12-23T01:34:48.340" Title="Unstructured quad mesh-generation?" Tags="|mesh-generation|computational-geometry|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="532" PostTypeId="2" ParentId="530" CreationDate="2011-12-28T19:45:39.397" Score="17" Body="&lt;p&gt;The are essentially two approaches to free quad meshing:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Direct&lt;/strong&gt; methods generate a quad mesh directly, usually by some advancing front method. The &lt;a href=&quot;http://dx.doi.org/10.1002/nme.1620320410&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paving paper&lt;/a&gt; is a standard reference and is the method used by &lt;a href=&quot;http://cubit.sandia.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUBIT&lt;/a&gt;, so you have seen these meshes in many publications.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Indirect&lt;/strong&gt; methods generate some intermediate decomposition of the domain (e.g. triangles) and then produce an all-quad mesh through recombination and/or further decomposition. &lt;a href=&quot;https://pages.cs.wisc.edu/%7Ecsverma/CS899_09/qmorph.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q-Morph&lt;/a&gt; is an example that is used by ANSYS.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note that smoothing is necessary for both approaches, sometimes with alternating topology fix-up and smoothing steps. Some open source tools have built-in smoothing facilities and the LGPL-licensed &lt;a href=&quot;http://www.cs.sandia.gov/optimization/knupp/Mesquite.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mesquite&lt;/a&gt; package is designed as a library specifically for mesh quality improvement.&lt;/p&gt;&#xA;&lt;p&gt;I know of two open source free-quad meshers:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Gmsh&lt;/strong&gt;&lt;/a&gt; (GPL with linking exception) can generate quad meshes using a recombination algorithm described in &lt;a href=&quot;http://dx.doi.org/10.1007/978-3-642-24734-7_25&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;Jaal&lt;/strong&gt; component of &lt;a href=&quot;http://trac.mcs.anl.gov/projects/fathom/wiki/MeshKit&quot; rel=&quot;nofollow noreferrer&quot;&gt;MeshKit&lt;/a&gt; (LGPL) is based on recombination similar to Q-Morph above, read &lt;a href=&quot;http://pages.cs.wisc.edu/%7Ecsverma/imr11.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the IMR-2011 paper&lt;/a&gt; for more details. You can download the source through the link above, but it is not ready for production use yet.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ccvweb.csres.utexas.edu/cvc/projects/project.php?pageIndex=1&amp;amp;proID=10&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;LBIE&lt;/strong&gt;&lt;/a&gt; generates quad and hex meshes from volumetric data. From what I can tell, it is an interactive environment rather than a library. The site says that the source is available under GPL upon request.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://cubit.sandia.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;CUBIT&lt;/strong&gt;&lt;/a&gt; is not open source (and although not expensive compared to commercial software, acquiring a license takes a long time), but produces high quality meshes and can be linked into other applications.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="14930" LastEditDate="2022-12-23T01:34:48.340" LastActivityDate="2022-12-23T01:34:48.340" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="533" PostTypeId="1" CreationDate="2011-12-29T15:43:01.393" Score="11" ViewCount="11048" Body="&lt;p&gt;This question arises from the need I have to prepare a lesson on the limitations of Density Functional Theory as a computational approach. I would like to know not only the limitations, but also reference texts I can use to prepare a more appealing and complete lesson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="23" LastEditorUserId="127" LastEditDate="2011-12-30T10:48:51.960" LastActivityDate="2017-11-28T15:08:44.240" Title="Limitations of Density Functional Theory as a computational method?" Tags="|density-functional-theory|" AnswerCount="7" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="534" PostTypeId="2" ParentId="527" CreationDate="2011-12-29T18:47:59.920" Score="14" Body="&lt;p&gt;If you want an idea of just how far we are away from such a software package, please look at the 2001 LAPACK working note on &lt;a href=&quot;http://www.netlib.org/lapack/lawns/lawn148.ps&quot;&gt;computing Givens rotations reliably and efficiently&lt;/a&gt;. I would expect most non-specialists (and many specialists!) in numerical analysis to be surprised at just how much analysis went into solving such an ostensibly simple problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $f,g \in \mathbb{C}$, find $c \in \mathbb{R}$ and $s \in \mathbb{C}$ such that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R(c,s) \left[\begin{array}{c} f \\ g \end{array}\right] = \left[\begin{array}{cc} c &amp;amp; s \\ -\bar s &amp;amp; c \end{array}\right] \left[\begin{array}{c} f \\ g \end{array}\right] = \left[\begin{array}{c} r \\ 0 \end{array}\right]$$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $R(c,s)$ is unitary. Balancing reliability with computational efficiency along with more subtle issues like continuity is highly nontrivial and unlikely to be automated in the forseeable future.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2011-12-29T18:47:59.920" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="535" PostTypeId="2" ParentId="527" CreationDate="2011-12-29T19:00:11.960" Score="11" Body="&lt;p&gt;Code generation and precompilation of mathematical expressions is becoming more popular. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While symbolic packages like SymPy, Mathematica, and Maple may include code generation I'm not confident that any of them also think hard about numerics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of other projects one could look into that are interested both in symbolics and in numerics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://deeplearning.net/software/theano/&quot; rel=&quot;nofollow&quot;&gt;Theano&lt;/a&gt; is such a project focused around array operations. They do identify and replace some operations known to be numerically ill-conditioned. I'm not certain that this includes your specific case but it's worth looking into. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://spiral.net/&quot; rel=&quot;nofollow&quot;&gt;Spiral&lt;/a&gt; might also be interesting to you. They also precompile an abstract syntax tree and also look out for numeric issues. They are more concerned with scalar operations (like your example). However they are also fairly specialized to a particular domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The growth in this field is encouraging however. One can be optimistic that your question will have a better answer in a few years. &lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-05-06T18:26:38.260" LastActivityDate="2012-05-06T18:26:38.260" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="536" PostTypeId="2" ParentId="533" CreationDate="2011-12-29T21:48:43.623" Score="12" Body="&lt;p&gt;One of the biggest limitations of density functional theory is that it doesn't correctly treat the &lt;a href=&quot;http://en.wikipedia.org/wiki/Exchange_interaction&quot;&gt;exchange interaction&lt;/a&gt;. While it has been shown that there exists a functional that will treat exchange correctly, this functional is unknown and semi-empirical methods are used to make approximations to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coulomb part of the functional that I see most often is also not exact, but I don't know whether the exact coulomb term is unknown or just has some unpleasant numerical property that keeps people from using it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only part of the Hamiltonian functional that is (as far as I know) exact is the term for the interaction with the external potential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either case, developing better approximations for the electron-electron coulomb and exchange terms is still an area of active research.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2011-12-29T21:48:43.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="537" PostTypeId="2" ParentId="533" CreationDate="2011-12-29T21:50:58.043" Score="4" Body="&lt;p&gt;If you're looking for a general reference, I would recommend the book by Parr and Yang, &lt;a href=&quot;http://books.google.com/books?id=mGOpScSIwU4C&amp;amp;printsec=frontcover&amp;amp;dq=Parr%20and%20Yang,%20Density%20Functional%20Theory%20of%20Atoms%20and%20Molecules&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=6tABT-7cE4Lu0gGZidWtAg&amp;amp;ved=0CDUQ6AEwAA#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Density Functional Theory of Atoms and Molecules&lt;/em&gt;&lt;/a&gt;. I used this book when I sat in on an advanced course in chemical kinetics which discussed how to use quantum chemistry to calculate energies and reaction rates. It's a reasonably complete overview of DFT, although it is somewhat outdated, as it was first published way back in the late 1980's.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="85" LastEditDate="2012-01-04T06:26:46.953" LastActivityDate="2012-01-04T06:26:46.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="539" PostTypeId="2" ParentId="27" CreationDate="2011-12-30T18:03:54.700" Score="6" Body="&lt;p&gt;I know that this answer makes a lot of assumptions, but it at least generalizes your algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that $\{A_n\}$, $\{B_n\}$, and the seeding matrix, $V_N$, all form a commuting family of normal matrices, where the eigenvalue decompositions of $\{A_n\}$ and $\{B_n\}$ are known a priori, say $U&amp;#39; V_N U = \Lambda_N$, $U&amp;#39; A_n U = \Omega_n$, and $U&amp;#39; B_n U = \Delta_n$, where $U$ is unitary and $\Lambda_N$, $\{\Omega_n\}$, and $\{\Delta_n\}$ are complex-valued diagonal matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once we have said decomposition, by induction,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$V_n = (I - B_n V_{n+1})^{-1} A_n = (I - U \Delta_n U&amp;#39; U\Lambda_{n+1} U&amp;#39;)^{-1} U \Omega_n U&amp;#39;,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be rearranged into the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$V_n = U (I - \Delta_n \Lambda_{n+1})^{-1} \Omega_n U&amp;#39; \equiv U \Lambda_n U&amp;#39;,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Lambda_n$ is of course still diagonal, so the entire family $\{V_n\}$ will necessarily commute with the other operators, and we have shown that the diagonal values of each $\Lambda_n$ are decoupled, so your fast scalar recursion formula can be applied independently on the eigenvalues of $V_N$ and the coefficient matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that a special case is when $A_n \equiv \alpha_n I$ and $B_n \equiv \beta_n I$, so that the only requirement is that $V_N$ be a normal matrix.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2011-12-30T18:10:24.053" LastActivityDate="2011-12-30T18:10:24.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="540" PostTypeId="1" CreationDate="2011-12-30T19:58:17.033" Score="19" ViewCount="3298" Body="&lt;p&gt;This is mostly aimed for elliptic PDEs over convex domains, so that I can get a good overview of the two methods.&lt;/p&gt;&#xA;" OwnerUserId="185" LastActivityDate="2012-01-10T09:31:59.440" Title="What is the advantage of multigrid over domain decomposition preconditioners, and vice versa?" Tags="|pde|multigrid|preconditioning|domain-decomposition|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="541" PostTypeId="1" AcceptedAnswerId="544" CreationDate="2011-12-30T20:18:53.793" Score="24" ViewCount="3839" Body="&lt;p&gt;I am looking to solve a constrained optimization problem where I know the bounds on some of the variables (specifically a boxed constraint).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \arg \min_u f(u,x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ c(u,x) = 0 $$&#xA;$$ a \le d(u,x) \le b $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is a vector of design variables, $x$ is a vector of state variables, and $c(u,x)$ is an equality constraint (usually a PDE). The lower and upper constraints $a$ and $b$ may be spatially variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which packages can handle systems of this form?&lt;/p&gt;&#xA;" OwnerUserId="142" LastEditorUserId="276" LastEditDate="2012-01-02T20:29:03.493" LastActivityDate="2015-03-16T03:59:14.127" Title="Software package for constrained optimization?" Tags="|pde|optimization|nonlinear-programming|" AnswerCount="8" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="542" PostTypeId="2" ParentId="541" CreationDate="2011-12-30T20:21:54.960" Score="2" Body="&lt;p&gt;If the problem is formulated as a complementarity problem, you can use &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/tao/&quot; rel=&quot;nofollow&quot;&gt;TAO&lt;/a&gt; (Toolkit for Advanced Optimization). Some of the methods in TAO, such as the reduced space method (a variant of the active set method), is currently available as part of SNES in PETSc (&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/SNES/SNESVI.html&quot; rel=&quot;nofollow&quot;&gt;SNESVI&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="185" LastActivityDate="2011-12-30T20:21:54.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="544" PostTypeId="2" ParentId="541" CreationDate="2011-12-30T20:53:06.990" Score="19" Body="&lt;p&gt;I decided to radically edit my answer based on some of the comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't used TAO. From perusing the documentation, it seems like the only way that TAO can handle constrained optimization problems (excluding the special case of only box constraints) is to convert the problem into a variational inequality using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Karush-Kuhn-Tucker&quot;&gt;Karush-Kuhn-Tucker (KKT) conditions&lt;/a&gt;, which are necessary under constraint qualification (the type that I usually see is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Slater_condition&quot;&gt;Slater point condition&lt;/a&gt;), and sufficient under convexity of the objective and constraints (more generally, Type 1 invexity). If $f$ is nonconvex, the variational inequality formulation using the KKT conditions is NOT equivalent to the original optimization problem, so strictly speaking, if you want a global optimum for the optimization problem, you should not express it as a variational inequality. It would be difficult to find a global optimum anyway for PDE-constrained optimization (see below), so maybe ignoring this detail is fine. Given what Wolfgang has said, I'd be skeptical of using TAO; I'm already skeptical because it doesn't implement methods for solving nonlinear programs (NLPs) as NLPs, rather than variational inequalities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not an expert on PDE-constrained optimization; co-workers and associates of mine work on ODE-constrained optimization problems. I know that for intrusive formulations, Larry Biegler (and others) will use collocation methods to discretize the PDE and make it a very large, sparse NLP, and then he will solve it using interior point methods. To really solve the problem to global optimality, you would also need to generate convex relaxations, but as far as I know, this approach is not taken because PDE-constrained optimization problems lead to such large NLPs that it would be difficult to solve them to global optimality. I mention these details only because problem formulation heavily influences choice of solver package. For nonintrusive formulations, repeated PDE solves yield gradient information for optimization algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some people who study ODE-constrained optimization problems use a similar approach of discretizing the problem using collocation and a numerical method, and then relaxing the resulting NLP to yield a convex formulation used in a global optimization algorithm. An alternate approach to ODE-constrained optimization is to relax the problem, and then discretize the ODE, which is the approach taken in my lab. It could be possible to relax certain classes of PDE-constrained optimization problems, but I don't know of any extant work done on that problem. (It was a potential project in my lab at one point.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, what matters is not the differentiability of the original PDE, but the differentiability of the discretization with respect to the decision variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the discretized problem is twice-differentiable with respect to the decision variables, the following packages will calculate a local solution:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.coin-or.org/projects/Ipopt.xml&quot;&gt;IPOPT&lt;/a&gt; is an open source interior point solver developed by Andreas Wächter at IBM. It's a very high quality code. As an interior-point solver, it is better for objective functions with large, sparse Jacobian matrices, and would be useful for PDE-constrained optimization&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sbsi-sol-optimize.com/asp/sol_product_snopt.htm&quot;&gt;SNOPT&lt;/a&gt; is a commercial sequential quadratic programming solver that is another high quality code. It is better for objective functions with small, dense Jacobian matrices, so I would not expect it to be useful for PDE-constrained optimization, but you could give it a try.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt&quot;&gt;NLopt&lt;/a&gt; is a small, open source code written by Steven Johnson at MIT that contains basic implementations of a number of nonlinear optimization algorithms. All of the algorithms should be adequate for solving bound-constrained problems. &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mathworks.com/help/toolbox/optim/ug/fmincon.html&quot;&gt;&lt;code&gt;fmincon&lt;/code&gt;&lt;/a&gt; in Matlab implements a number of algorithms (including interior point and sequential quadratic programming) for nonlinear optimization&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://gams.com/&quot;&gt;GAMS&lt;/a&gt; and &lt;a href=&quot;http://www.ampl.com/&quot;&gt;AMPL&lt;/a&gt; are both commercial modeling languages used to formulate optimization problems, and contain interfaces to a large number of nonlinear programming solvers. I know that GAMS has a trial version that can be used for smaller problems, and problem instances can also be submitted to the &lt;a href=&quot;http://neos1.chtc.wisc.edu/neos/solvers/index.html&quot;&gt;NEOS server&lt;/a&gt; for solution as well. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, it's possible that the discretization is only once differentiable with respect to the decision variables, in which case, you should use projected steepest descent or some other first-order optimization method when calculating a local solution. Since a lot of study focuses on problems where second-order methods can be used (and when you can use them, their superior convergence properties make them a better choice), I couldn't find many implementations of steepest descent that weren't solutions to homework problems. The &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Multimin-Algorithms-with-Derivatives.html&quot;&gt;GNU Scientific Library&lt;/a&gt; has an implementation, but it's only for demonstration purposes. You would probably need to code up your own implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the problem is only continuous with respect to the decision variables, then you could use direct methods to solve it locally. There is an excellent survey on direct methods by &lt;a href=&quot;http://www.cs.wm.edu/~va/research/sirev.pdf&quot;&gt;Kolda, Lewis, and Torczon&lt;/a&gt;. The most widely known of these methods is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Nelder-Mead_method&quot;&gt;Nelder-Mead simplex algorithm&lt;/a&gt;. It is not guaranteed to converge to a local minimum in multiple dimensions, but it has found considerable practical use anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choice of package really depends on the language you want to use to solve the problem, if solving the bound-constrained problem is only part of an algorithm you want to implement (or if it's the only step in your algorithm, in which case modeling languages become more viable for production code), the type and size of the problem, and if you need any parallelism.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-02T07:13:45.447" LastActivityDate="2012-01-02T07:13:45.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="545" PostTypeId="2" ParentId="181" CreationDate="2011-12-30T23:35:20.903" Score="2" Body="&lt;p&gt;Matt: Have you looked at the polymer self-consistent field theory literature? It's rather similar in spirit to classical DFT's, and discretizations of the system are routinely performed. There's a paper by Alexander-Katz et al. &lt;a href=&quot;http://dx.doi.org/10.1063/1.1827211&quot; rel=&quot;nofollow&quot;&gt;[JCP, 122, 014904 (2005)]&lt;/a&gt; that specifically discusses finite-size and discretization effects. That might be a place to start.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2011-12-30T23:35:20.903" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="546" PostTypeId="2" ParentId="540" CreationDate="2011-12-31T01:27:18.067" Score="19" Body="&lt;p&gt;Multigrid and multilevel domain decomposition methods have so much in common that each can usually be written as a special case of the other. The analysis frameworks are somewhat different, as a consequence of the different philosophies of each field. Generally speaking, multigrid methods use &lt;strong&gt;moderate coarsening rates&lt;/strong&gt; and &lt;strong&gt;simple smoothers&lt;/strong&gt; while domain decomposition methods use extremely &lt;strong&gt;rapid coarsening&lt;/strong&gt; and &lt;strong&gt;strong smoothers&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Multigrid (MG)&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Multigrid uses moderate coarsening rates and achieves robustness through modification of interpolation and smoothers. For elliptic problems, the interpolation operators should be &quot;low energy&quot;, such that they preserve the near-null space of the operator (e.g. rigid body modes). An example geometric approach to these low energy interpolants is &lt;a href=&quot;ftp://info.mcs.anl.gov/pub/tech_reports/reports/P710.pdf&quot; rel=&quot;noreferrer&quot;&gt;Wan, Chan, Smith (2000)&lt;/a&gt;, compare to the algebraic construction of smoothed aggregation &lt;a href=&quot;http://dx.doi.org/10.1007/BF02238511&quot; rel=&quot;noreferrer&quot;&gt;Vaněk, Mandel, Brezina (1996)&lt;/a&gt; (parallel implementations in &lt;a href=&quot;http://trilinos.sandia.gov/packages/ml/&quot; rel=&quot;noreferrer&quot;&gt;ML&lt;/a&gt; and &lt;a href=&quot;http://mcs.anl.gov/petsc&quot; rel=&quot;noreferrer&quot;&gt;PETSc&lt;/a&gt; via PCGAMG, the replacement for &lt;a href=&quot;http://www.columbia.edu/~ma2325/prometheus/&quot; rel=&quot;noreferrer&quot;&gt;Prometheus&lt;/a&gt;). &lt;a href=&quot;http://books.google.com/books?id=9ysyNPZoR24C&amp;amp;lpg=PP2&amp;amp;ots=rIyLSVyONR&amp;amp;dq=trottenberg&amp;amp;lr&amp;amp;pg=PP1#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;Trottenberg, Oosterlee, and Schüller's book&lt;/a&gt; is a good general reference on Multigrid methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most multigrid smoothers involve pointwise relaxation, either additively (Jacobi) or multiplicatively (Gauss Seidel). These correspond to tiny (single node or single element) Dirichlet problems. Some spectral adaptivity, robustness, and vectorizability can be achieved using Chebyshev smoothers, see &lt;a href=&quot;http://dx.doi.org/10.1016/S0021-9991%2803%2900194-3&quot; rel=&quot;noreferrer&quot;&gt;Adams, Brezina, Hu, Tuminaro (2003)&lt;/a&gt;. For non-symmetric (e.g. transport) problems, multiplicative smoothers like Gauss-Seidel are generally necessary, and upwinded interpolants may be used. Alternatively, smoothers for saddle point and stiff wave problems can be constructed by transforming via Schur-complement-inspired &quot;block preconditioners&quot; or by the related &quot;distributed relaxation&quot;, into systems in which simple smoothers are effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Textbook multigrid efficiency refers to solving to &lt;em&gt;discretization error&lt;/em&gt; in a small multiple of the cost of a few residual evaluations, as few as four, on the fine grid. This implies that the number of iterations to a fixed algebraic tolerance goes &lt;em&gt;down&lt;/em&gt; as the number of levels in increased. In parallel, the time estimate involves a logarithmic term arising due to synchronization implied by the multigrid hierarchy.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Domain Decomposition (DD)&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The first domain decomposition methods had only one level. With no coarse level, the condition number of the preconditioned operator cannot be less than $\mathcal O\big( \frac{L^2}{H^2}\big)$ where $L$ is the diameter of the domain and $H$ is the nominal subdomain size. In practice, condition numbers for one-level DD falls between this bound and $\mathcal O\big( \frac{L^2}{hH} \big)$ where $h$ is the element size. Note that the number of iterations needed by a Krylov method scales as the square root of the condition number. Optimized Schwarz methods &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/42540.pdf&quot; rel=&quot;noreferrer&quot;&gt;(Gander 2006)&lt;/a&gt; improve the constants and dependence on $H/h$ relative to Dirichlet and Neumann methods, but generally do not include coarse levels and thus degrade in the case of many subdomains. See the books by &lt;a href=&quot;http://books.google.com/books?id=dxwRLu1dBioC&amp;amp;lpg=PA207&amp;amp;ots=xQqGr5z8Fr&amp;amp;dq=smith%20gropp&amp;amp;pg=PP1#v=onepage&amp;amp;q=smith%20gropp&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;Smith, Bjørstad, and Gropp (1996)&lt;/a&gt; or &lt;a href=&quot;http://books.google.com/books?id=h7EVoI2g1nkC&amp;amp;lpg=PA1&amp;amp;ots=uXDIoT8_La&amp;amp;dq=toselli%20and%20widlund&amp;amp;lr&amp;amp;pg=PR1#v=onepage&amp;amp;q=toselli%20and%20widlund&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;Toselli and Widlund (2005)&lt;/a&gt; for a general reference to domain decomposition methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For optimal or quasi-optimal convergence rates, multiple levels are necessary. Most DD methods are posed as two-level methods and some are very difficult to extend to more levels. DD methods can be classified as overlapping or non-overlapping.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Overlapping&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;These Schwarz methods use overlap and are generally based on solving Dirichlet problems. The strength of the methods can be increased by increasing the overlap. This class of methods is usually robust, do not require local null-space identification or technical modifications for problems with local constraints (common in engineering solid mechanics), but involve extra work (especially in 3D) due the overlap. Additionally, for constrained problems like incompressible, the inf-sup constant of the overlapping strip usually appears, leading to suboptimal convergence rates. Modern overlapping methods using similar coarse spaces to BDDC/FETI-DP (discussed below) are developed by &lt;a href=&quot;http://epubs.siam.org/sinum/resource/1/sjnaam/v46/i4/p2153_s1&quot; rel=&quot;noreferrer&quot;&gt;Dorhmann, Klawonn, and Widlund (2008)&lt;/a&gt; and &lt;a href=&quot;http://dx.doi.org/10.1002/nme.2761&quot; rel=&quot;noreferrer&quot;&gt;Dohrmann and Widlund (2010)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Non-overlapping&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;These methods usually solve Neumann problems of some sort, which means that unlike Dirichlet methods, they cannot work with a globally assembled matrix, and instead require unassembled or partially assembled matrices. The most popular Neumann methods either enforce continuity between subdomains by balancing at every iteration or by Lagrange multipliers that will enforce continuity only once convergence is reached. The early methods of this sort (Balancing Neumann-Neumann and FETI) require precise characterization of the null space of each subdomain, both to construct the coarse level and to make the subdomain problems non-singular. Later methods (BDDC and FETI-DP) select subdomain corners and/or edge/face moments as coarse level degrees of freedom. See &lt;a href=&quot;http://dx.doi.org/10.1016/j.cma.2006.03.023&quot; rel=&quot;noreferrer&quot;&gt;Klawonn and Rheinbach (2007)&lt;/a&gt; for an in-depth discussion of coarse space selection for 3D elasticity. &lt;a href=&quot;http://dx.doi.org/10.1016/j.apnum.2004.09.022&quot; rel=&quot;noreferrer&quot;&gt;Mandel, Dohrmann, and Tazaur (2005)&lt;/a&gt; showed that BDDC and FETI-DP have all the same eigenvalues, except for possible 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;More than two levels&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Most DD methods are only posed as two-level methods, and some select coarse spaces that are inconvenient for use with more than two levels. Unfortunately, especially in 3D, the coarse level problems quickly become a bottleneck, limiting the problem sizes that can be solved. Additionally, the condition numbers of the preconditioned operators, especially for the DD methods based on Neumann problems, tend to scale as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \kappa( P_{\text{DD}}^{-1} A ) = C \Big(1 + log \frac H h \Big)^{2(L-1)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L$ is the number of levels. As long as aggressive coarsening is used, this is perhaps not so critical because $L \le 4$ should be capable of solving problems with more than $10^{12}$ degrees of freedom, but is certainly a concern. See &lt;a href=&quot;https://scicomp.stackexchange.com/questions/60/is-there-a-multigrid-algorithm-that-solves-neumann-problems-and-has-a-convergenc&quot;&gt;this question&lt;/a&gt; for further discussion of this limitation.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-08T01:00:44.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="547" PostTypeId="5" CreationDate="2011-12-31T04:30:51.383" Score="0" Body="&lt;p&gt;A means of solving ordinary and partial differential equations through discretization of the defining differential equations into  elements. Individual element mass and stiffness matrices are assembled and from them global stiffness and mass matrices are assembled and used to solve the coupled system. The Finite Element Method lends itself well to adaptive refinement, irregular geometry, and good error estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The methods discretize a complex physical system as a set of many elements having a very simple geometrical shape (e.g. a triangle-shaped plate), for which it it is simple to solve the differential equation. A global mass and stiffness matrix are then formed from the element matrices and used to solve the system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Wikipedia page on &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_element_method&quot; rel=&quot;nofollow&quot;&gt;FEMs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16952" LastEditorUserId="16952" LastEditDate="2015-07-20T04:01:02.647" LastActivityDate="2015-07-20T04:01:02.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="548" PostTypeId="4" CreationDate="2011-12-31T04:30:51.383" Score="0" Body="A means of solving ordinary and partial differential equations.  The domain of the problem is broken up into elements, and the solution in each element is expanded in a basis of functions. The Finite Element Method lends itself well to adaptive refinement, irregular geometry, and good error estimates." OwnerUserId="81" LastEditorUserId="549" LastEditDate="2012-01-14T16:13:33.370" LastActivityDate="2012-01-14T16:13:33.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="549" PostTypeId="1" AcceptedAnswerId="565" CreationDate="2011-12-31T04:39:47.420" Score="10" ViewCount="524" Body="&lt;p&gt;Right now I have a code that uses the Crank-Nicholson algorithm, but I think that I would like to move to a higher-order algorithm for timestepping. I know that the Crank-Nicholson algorithm is stable in the domain I want to work in, but I am concerned that some other algorithms may not be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to calculate the stability region of an algorithm, but it can be kind of a pain.  Does anybody know of any good reference for the stability properties of a large number of timestepping algorithms for parabolic PDEs?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="123" LastEditDate="2011-12-31T08:49:49.893" LastActivityDate="2017-07-08T10:52:44.647" Title="Where can I find a good reference for the stability properties of several methods of solving parabolic PDEs?" Tags="|pde|stability|parabolic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="550" PostTypeId="2" ParentId="549" CreationDate="2011-12-31T07:05:04.730" Score="10" Body="&lt;p&gt;Very short answer: for a comprehensive reference, you can't beat &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3540604529&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer and Wanner volume II&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short answer: Here are some MATLAB scripts to plot the stability region of a &lt;a href=&quot;https://bitbucket.org/rjleveque/fdmbook/src/d734c09130a3/matlab/plotBL.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear multistep&lt;/a&gt; or &lt;a href=&quot;https://bitbucket.org/rjleveque/fdmbook/src/d734c09130a3/matlab/plotS.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;Runge-Kutta method&lt;/a&gt;, given the coefficients.  You could also use the Python package &lt;a href=&quot;http://numerics.kaust.edu.sa/nodepy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;nodepy&lt;/a&gt; (disclaimer: it's my package and it's not the most polished piece of software, but plotting stability regions is one thing it does very well).  Instructions for plotting stability regions are &lt;a href=&quot;http://numerics.kaust.edu.sa/nodepy/stability.html#plotting-the-region-of-absolute-stability&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Longer answer: there are three classes of methods you could be interested in here.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;$A$-stable methods&lt;/strong&gt;, where &lt;em&gt;all of the left half of the complex plane&lt;/em&gt; lies in the stability region.  The most well-known examples are &lt;strong&gt;backward Euler&lt;/strong&gt; (1st order) and the &lt;strong&gt;implicit trapezoidal method&lt;/strong&gt; (which is what Crank-Nicholson uses).  For these methods, you don't need to know the details of the stability region; as long as the eigenvalues of your spatial discretization lie in the left half-plane, you'll have unconditional stability (no step size restriction).  Due to the &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_multistep_method#First_and_second_Dahlquist_barriers&quot; rel=&quot;nofollow noreferrer&quot;&gt;second Dahlquist barrier&lt;/a&gt;, you have to use Runge-Kutta methods if you want high order and $A$-stability.  Some examples of such methods are the Gauss-Legendre, Radau, and Lobatto methods.  All of those are fully implicit and thus rather expensive.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;$A(\alpha)$-stable methods&lt;/strong&gt;, which include a &lt;em&gt;sector in the left half-plane&lt;/em&gt;, including all of the negative real axis.  The most prominent of these are the &lt;strong&gt;backward differentiation&lt;/strong&gt; (BDF) methods and a variant known as &quot;numerical differentiation formulas&quot;, which are implemented in MATLAB's &lt;code&gt;ode15s()&lt;/code&gt;.  These are unconditionally stable as long as the eigenvalues of your spatial discretization lie in that sector, so the only thing you need to know about the stability region is the angle $\alpha$, which you can find in any reference on ODE solvers (for instance, p. 175 of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898716292&quot; rel=&quot;nofollow noreferrer&quot;&gt;LeVeque&lt;/a&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Explicit methods&lt;/strong&gt;, which will necessary include only a &lt;em&gt;finite interval&lt;/em&gt; on the negative real axis.  There are special &quot;stabilized&quot; explicit methods (in particular, the &lt;strong&gt;Runge-Kutta-Chebyshev methods&lt;/strong&gt;) that have large negative real axis stability regions and are suitable for mildly stiff problems, but usually not for parabolic problems.  A good entrance to that literature is &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2004.05.002&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which includes lots of information about the stability regions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I've assumed throughout that you're only interested in absolute stability.  For parabolic problems you would likely want an $L$-stable method as well, but it's simple to check $L$-stability of a method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: If you &lt;em&gt;really&lt;/em&gt; need to know everything about this topic, get a copy of &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/zamm.19870670128/abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dekker and Verwer's monograph&lt;/a&gt;.  It has one of the best existing introductions to concepts like one-sided Lipschitz constants, the logarithmic norm, and several deeper stability concepts.  It's out of print but you can usually find used copies on Amazon (for a price!)&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2017-07-08T10:52:44.647" LastActivityDate="2017-07-08T10:52:44.647" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="551" PostTypeId="1" CreationDate="2011-12-31T13:23:51.197" Score="5" ViewCount="2154" Body="&lt;p&gt;I am using backward Euler in a FEM scheme for a convection-diffusion problem. On a given mesh, I can take arbitrarily large time steps, as expected. But if I decrease time step, at some point it will generate oscillations in the solution (spikes). Is this a known behavior? It's not mentioned in text books, at least the ones I know. Would Crank–Nicolson scheme eliminate the issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above mentioned problem is only the simplest one to demonstrate the phenomenon. I attach an image of what is happening in my real problem: transient (here only 2D) incompressible flow. $\rho=1$, dynamic $\eta=1\times 10^{-3}$, flat inflow 1, cylinder diameter 0.1 ($R_e=100$). If timestep is 0.0001, such oscillations occur. If timestep is 0.005, solution becomes smooth and I reproduce von Karman vortices with proper frequency, so it is highly unlikely I have a bug in the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is standard Galerkin FEM with no stabilization solved with a direct solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other thoughts how this is possible and how to know how small timestep is &quot;too small&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7BCll.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any hints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dominik&lt;/p&gt;&#xA;" OwnerUserId="474" LastEditorUserId="81" LastEditDate="2012-01-09T01:53:42.053" LastActivityDate="2022-01-09T13:38:00.353" Title="Solution oscillations with a small timestep in backward Euler" Tags="|pde|stability|finite-element|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="552" PostTypeId="2" ParentId="551" CreationDate="2011-12-31T17:53:38.340" Score="3" Body="&lt;p&gt;Assuming that you keep the spatial grid (mesh) size constant and decrease the time step, one thing that could be happening is that as you decrease the time step, you move into the small, unstable region of the backward Euler method in the right half-plane of the stability diagram. (See &lt;a href=&quot;http://books.google.com/books?id=1Sg0zPVkjdMC&amp;amp;lpg=PP1&amp;amp;dq=Ascher%20Petzold&amp;amp;pg=PA52#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Ascher and Petzold&lt;/a&gt;, page 52.) The region of instability for backward Euler is $\{z \in \mathbb{C}: |1 - z| \leq 1\}$, where $z = h \lambda$, $h$ being your step size and $\lambda$ being the eigenvalue of the test equation $\dot{y}(t) = \lambda y(t)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the true solution operator for convection-diffusion only has eigenvalues with negative real part, if your semi-discretization yields eigenvalues with positive real part then it's a very bad semi-discretization!&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="85" LastEditDate="2012-01-04T06:26:35.720" LastActivityDate="2012-01-04T06:26:35.720" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="553" PostTypeId="2" ParentId="540" CreationDate="2011-12-31T19:44:05.257" Score="11" Body="&lt;p&gt;This is an excellent writeup but I think saying that (multilevel) DD and MG have a lot in common is not accurate, or at least not useful.  The methods are very different and I don't think that expertise in one is very useful in the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, the two communities use different definitions of complexity: DD optimizes the condition number of the preconditioned systems and MG optimizes work/memory complexity.  This is a big fundamental difference -- &quot;optimality&quot; has a totally different meaning in these two contexts.  Things don't change when you add in parallel complexity (although you get a log term added in MG).  The two communities are almost speaking different languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, MG has multilevel built into it and multilevel DD methods have all been developed with two level theory and implementations.  This limits the space of coarse grid spaces that you can use in MG -- they must be recursive.  For instance, You can not implement FETI in an MG framework.  People do some multilevel DD methods as Jed mentioned but at least some of the current popular DD methods do not seem to be implementable recursively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, I see the algorithms themselves, as practiced, as very different.  Qualitatively speaking I would say that DD methods project onto domain boundaries and solve this interface problem.  MG works directly with the native equations.  Avoiding this projection allows MG to be applied to nonlinear and unsymmetrical problems easily.  Although the theory all but goes away for nonlinear and unsymmetric problems they have worked for a lot of people.  MG also explicitly decouples the problem into two parts: the coarse grid space for scaling and an iterative solver (the smoother) to solve the physics.  This is critical in understanding and working with MG and is an attractive property to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although theoretically the smoothers and coarse grid spaces are tightly coupled, in practice you can often swap different smoother in and out as an optimization parameter.  As Jed mentioned point or vertex smoothers are popular and usually faster, but for challenging problems heavier smoothers can be useful. This plot is from my dissertation showing the solve time as a function of Poisson ratio for Jacobi, block Jacobi and &quot;additive Schwarz&quot; (overlapped).  Its a little hard to read but at the highest Poisson ratio (0.499) overlapping Schwarz is about 2x faster than (vertex) Jocobi whereas it is about 3x slower at pedestrian Poisson ratios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/L5IoI.png&quot; alt=&quot;Solve time vs. Poisson ratio for point, block and overlapped smoothers&quot;&gt; &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2011-12-31T19:50:36.663" LastActivityDate="2011-12-31T19:50:36.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="554" PostTypeId="2" ParentId="540" CreationDate="2012-01-02T01:55:49.900" Score="2" Body="&lt;p&gt;I want to make one small addition to Jed's excellent answer, namely that the motivations behind the two approaches are (or at least were) different.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Domain decomposition is motivated as a technique for parallel computing.  Especially for one-level methods DD is very natural to implement on a parallel machine - you divide the domain into pieces and give each piece to a different processor.  In some sense the motivation behind DD is to divide arithmetic operations between processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good parallel multigrid implementations exist, but it is often less natural to do in parallel.  Instead, the motivation behind multigrid is to do less arithmetic operations in the first place.&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2012-01-02T01:55:49.900" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="555" PostTypeId="2" ParentId="541" CreationDate="2012-01-02T05:06:46.243" Score="4" Body="&lt;p&gt;We've tried TAO but found it to be not very useful for inequality constrained problems. It's also essentially only in maintenance mode since at least 2003, with no real new features apart from updates to track changes in PETSc upon which it is built.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-01-02T05:06:46.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="556" PostTypeId="1" AcceptedAnswerId="558" CreationDate="2012-01-02T12:36:39.943" Score="5" ViewCount="1384" Body="&lt;p&gt;In my physics class, I had to calculate the trajectory of a projectile that was fired (very fast) with $v_0$ in an angle off a planet (radius $R$, mass $M$) from the surface. The projectile would reach a maximum distance and start to circle around the planet. The question was whether it would hit the planet. We solved that analytically, but I wonder how to simulate such a problem numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically I start off with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{r}_0 = \begin{pmatrix} 0 \\ 0 \\ R \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{v}_0 = v_0 \begin{pmatrix} \cos(30&amp;#176;) \\ 0 \\ \sin(30&amp;#176;) \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The acceleration on the projectile is just&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \vec{a}(\vec{r}) = \frac{\vec{F}(\vec{r})}{m} = G \frac{M}{r^3} \vec{r} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could implement a simple C++/Python program which basically does:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \vec{r}_{t+1} = \vec{r}_{t} + \vec{v}_{t} \cdot \mathrm dt $$&#xA;$$ \vec{v}_{t+1} = \vec{v}_{t} + \vec{a}_{t} \cdot \mathrm dt $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this seems a little unsuited as in either language I do not have &quot;natural&quot; vector classes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would Octave or Mathematica be better suited for this task? How would I implement this problem to obtain a list of x, y, z coordinates in order to plot the trajectory in gnuplot?&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="127" LastEditDate="2012-01-03T08:53:58.470" LastActivityDate="2012-01-04T04:00:23.977" Title="Implementing Euler's method for initial value ODEs" Tags="|software|ode|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="557" PostTypeId="2" ParentId="556" CreationDate="2012-01-02T15:39:42.580" Score="1" Body="&lt;p&gt;One option to look at for systems that involve evolution over time would be to consider using either MATLAB or Modelica. MATLAB is of course proprietary, but Modelica exists in many implementations, some of which, such as &lt;a href=&quot;http://openmodelica.org&quot; rel=&quot;nofollow&quot;&gt;OpenModelica&lt;/a&gt;, are definitely open-source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As someone who is relatively new to Modelica (I only learned of it about a year ago), I find that it is very good at implementing problems based on dynamical systems: sets of ordinary differential equations or differential-algebraic equations. In particular, the setup of problems is highly modularized, and you can write equations in &quot;natural&quot; form, rather than having to rewrite them in arbitrary forms to massage them into whatever format is required by a language or implementation such as what MATLAB requires. OpenModelica also offers some integrated graphing capabilities.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-02T15:39:42.580" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="558" PostTypeId="2" ParentId="556" CreationDate="2012-01-02T19:40:16.420" Score="5" Body="&lt;p&gt;Just use Python + NumPy. Here is the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from math import pi, sin, cos&#xA;from numpy import array, vstack&#xA;from numpy.linalg import norm&#xA;&#xA;from pylab import plot, savefig&#xA;&#xA;R = 10.&#xA;M = 1.&#xA;G = 1.&#xA;phi = 30 * 180. / pi&#xA;v0_magnitude = 0.1&#xA;r = array([0, 0, R])&#xA;v = v0_magnitude * array([cos(pi), 0, sin(pi)])&#xA;trajectory = []&#xA;&#xA;dt = 0.1&#xA;t = 0&#xA;while t &amp;lt; 100:&#xA;    trajectory.append(r.copy())&#xA;    a = -G * M * r / norm(r)**3&#xA;    v += a * dt&#xA;    r += v * dt&#xA;    t += dt&#xA;&#xA;trajectory = vstack(trajectory)&#xA;plot(trajectory[:, 0], trajectory[:, 2])&#xA;savefig(&quot;trajectory.png&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here is the graph:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/5oV2r.png&quot; alt=&quot;trajectory.png&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also uploaded the above code and the graph at this &lt;a href=&quot;https://gist.github.com/1551827&quot; rel=&quot;nofollow noreferrer&quot;&gt;gist&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2012-01-02T19:40:16.420" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="559" PostTypeId="1" AcceptedAnswerId="560" CreationDate="2012-01-02T23:03:47.553" Score="19" ViewCount="6637" Body="&lt;p&gt;Back towards the dawn of OS X, there seemed to be a great deal of hubbub, at least in the Mac world (I was nowhere near scientific computing at the time) about the Mac OS as a platform for scientific computing and HPC applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;XGrid came out of the box, Virginia Tech had their fancy Mac-based computing cluster, Stanford was doing cool things, etc. etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently however, things have been quiet. The Macresearch.org site is essentially a ghost town filled with spammer zombies, the XServe is dead, and an awful lot of the marketing literature and the like seems to even be from the pre-Intel processor area. But XGrid is still there, the whole *nix OS underpinning is there, and the platform seems to have decent support among Python, R, and some of the newer languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, from people who know more about this than I do...how fares OS X? Are they viable client-side computers for scientific computing? Is using them as a server/cluster/etc. through XGrid or something like it simply a novelty application?&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="41" LastEditDate="2012-01-09T01:56:43.410" LastActivityDate="2012-01-10T12:13:10.687" Title="State of the Mac OS in Scientific Computing and HPC" Tags="|software|hpc|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="560" PostTypeId="2" ParentId="559" CreationDate="2012-01-02T23:47:11.017" Score="13" Body="&lt;p&gt;I can't comment on the server side of things. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the client side, at the one computational science meeting I go to every year, the proportion of Mac users seems to have increased. I switched to a Mac because I got tired of dealing with my school-supplied Dell laptop failing at the drop of a hat. I switched to Macs for the hardware, primarily, since Consumer Reports rated them highly in terms of durability. I don't think that Macs are good for scientific computing unless you run Linux on them. The Linux support for the hardware tends to lag; usually, it's the wireless card that isn't supported (whenever they change it in a new model). If you're willing to accept the resource penalty that comes with running a virtual machine, it's an attractive option (and one that I personally use).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Macs require you to install a lot of libraries and software packages before you can do serious scientific computing. Anything that has a Mac installer is easy to manage, so if you do most of your development work with Matlab, Mathematica, Maple, Python, etc., it's easy to install and run that software on OS X natively. It's harder to track down hard core numerical software that has a Mac installer (think things like PETSc or CLAWPACK). Package managers like &lt;a href=&quot;https://trac.macports.org/news/&quot;&gt;MacPorts&lt;/a&gt; and &lt;a href=&quot;http://www.finkproject.org/&quot;&gt;Fink&lt;/a&gt; can help the situation if you want to use OS X only. You'll also have to compile a lot of packages from source. If you want to run your code anywhere else, you'll have to watch out for compatibility issues. Since Linux enjoys widespread usage in scientific computing, it's easier from a portability standpoint to develop code in Linux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard anecdotally from highly opinionated friends that setting up a development environment on a Mac is a huge pain in the ass, others have said it's not so bad. Your mileage may vary.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-02T23:47:11.017" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="561" PostTypeId="1" CreationDate="2012-01-03T00:51:00.833" Score="10" ViewCount="1616" Body="&lt;p&gt;Is it possible to get second order accuracy for hexahedral finite elements with fewer than 8 Gauss points without introducing unphysical modes?  A single central Gauss point introduces an unphysical shearing mode, and the standard symmetric arrangement of 8 Gauss points is expensive compared to tetrahedral discretizations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit&lt;/em&gt;: Someone asked for equations.  The equations I'm interested in are nonlinear elasticity, either dynamic or quasistatic.  The quasistatic equations are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla \cdot P\left(\nabla \phi \right) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\phi : \Omega \to \mathbf{R}^3$, $\Omega \subset \mathbf{R}^3$, and $P : \mathbf{R}^{3 \times 3} \to \mathbf{R}^{3 \times 3}$ is a hyperelastic first Piola-Kirchoff stress function.  An simple example is compressible neo-Hookean, where&#xA;$$ P(F) = \mu (F - F^{-T}) +\lambda F^{-T} \log \det F$$&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2012-01-04T20:54:09.983" LastActivityDate="2012-01-05T08:34:42.280" Title="Are 8 Gauss points required for second order hexahedral finite elements?" Tags="|finite-element|accuracy|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="562" PostTypeId="2" ParentId="559" CreationDate="2012-01-03T08:39:25.410" Score="15" Body="&lt;p&gt;@Geoff gives a good answer, but I think it's worth providing an alternative perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do everything on Macs -- in OS X, not a Linux VM -- including lots of scientific code development.  I mostly work in Fortran and Python.  For me, the convenience of &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;being able to do all my work in one OS and &lt;/li&gt;&#xA;&lt;li&gt;almost never deal with hardware failures or driver issues &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;is worth the cost of Mac-specific headaches.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The three main headaches are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Lack of an OS-standard package manager.  Once upon a time I used Fink, but eventually it led to more headaches and it's now obsolete.  I've heard good things about Macports and Homebrew, but my experience with Fink convinced me to just &quot;roll my own&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some of the built-in software is very outdated.  Particularly, Python and gcc.  This means that you need to install your own updated versions, which can be a hassle.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apple does not include a Fortran compiler!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It seems to me that Apple is paying less and less attention to their Unix-based power users.  Meanwhile, Linux keeps improving.  Eventually I will probably be pushed back into Linux.  But I will keep my Macbook until somebody else learns how to make decent batteries.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-01-10T12:06:55.487" LastActivityDate="2012-01-10T12:06:55.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="563" PostTypeId="2" ParentId="559" CreationDate="2012-01-03T09:43:46.030" Score="10" Body="&lt;p&gt;I would argue that the Mac is a better environment for computational &lt;em&gt;scientists&lt;/em&gt; than it is for computational &lt;em&gt;science&lt;/em&gt;. I would not want to use Macs in a commodity computing environment; the hardware is, relatively speaking, way too expensive for that. It can be a pain to get the software environment up to match the conditions needed for a particular package, but usually once you've figured it out the first time, it's a lot easier to maintain than a comparable Windows installation. (And, depending on the package managers, it can be as easy as Linux. MacPorts does do fairly well as a manager for OS X!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as a working platform for a busy computational scientist,  I believe that the Mac offers the additional bonus of being able to work and maintain in a single computing environment, rather than having to maintain a Linux box for &quot;serious&quot; work, and still needing to have another machine when colleagues working in other disciplines send me (for instance) Office files that need markup, or I have to fill out a University form that's only available as a RTF template that won't format right using LibreOffice and will be a pain in the neck to recreate in LaTeX. (At previous jobs, Linux owners pretty much had to have a non-Linux machine in addition, because to access the corporate environment more or less demanded it. Mac owners didn't really need a second machine.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, there are a number of Mac-only programs (like &lt;a href=&quot;http://www.macromates.org&quot;&gt;TextMate&lt;/a&gt;, &lt;a href=&quot;http://www.literatureandlatte.com/scrivener.php&quot;&gt;Scrivener&lt;/a&gt;, &lt;a href=&quot;http://www.mekentosj.org&quot;&gt;Papers&lt;/a&gt;, &lt;a href=&quot;http://www.culturedcode.com&quot;&gt;Things&lt;/a&gt;, &lt;a href=&quot;http://www.omnigroup.com/products/omnifocus/&quot;&gt;OmniFocus&lt;/a&gt;, or &lt;a href=&quot;http://bibdesk.sourceforge.net&quot;&gt;BibDesk&lt;/a&gt;) that make the Mac a much more pleasant environment in which to do work than on the PC or Linux. I find myself spending more time focusing on getting things done, than on what I need to do to get the software environment to do what I want (or need!) it to do.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-01-03T11:28:09.817" LastActivityDate="2012-01-03T11:28:09.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="564" PostTypeId="2" ParentId="559" CreationDate="2012-01-03T16:42:50.777" Score="6" Body="&lt;p&gt;Use of OS X in HPC and scientific computing is low and it has to do a lot with the pros and cons of OS X w.r.t. the alternative (Linux)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OS X Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Polished UI; still *nix&lt;/li&gt;&#xA;&lt;li&gt;Desktop/Design app(s) such as MS Office, Adobe programs well supported&lt;/li&gt;&#xA;&lt;li&gt;Multimedia very well supported&lt;/li&gt;&#xA;&lt;li&gt;Some people like the Apple ecosystem (iPhone, iTunes etc.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;OS X Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Runs on expensive hardware and not everyone likes Macbooks, specially people used to Thinkpads (keyboard+trackpoint)&lt;/li&gt;&#xA;&lt;li&gt;Cannot upgrade hardware (e.g. if you want to try the latest NVIDIA card with your CUDA app) on desktop/cluster&lt;/li&gt;&#xA;&lt;li&gt;Bloated GUI that cannot be customized (in Linux you can use a minimalist window manager)&lt;/li&gt;&#xA;&lt;li&gt;Package mgmt in Macports/Fink is sub par compared to Linux (Debian) distributions. Most packages are not even actively maintained or are orphaned&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some useful tools/programs traditionally did not run or still dont run on OS X. For example ...&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Sun Studio still doesnt work&lt;/li&gt;&#xA;&lt;li&gt;Valgrind only started working recently and not all features are supported&lt;/li&gt;&#xA;&lt;li&gt;Intel compilers have also been available in recent years&lt;/li&gt;&#xA;&lt;li&gt;Apple doesnt even package a Fortran compiler and you have to rely on 3rd party (mostly individuals) to build binaries that work only on certain OS X versions (which the individual has). Support is rare or non-existant in such cases&lt;/li&gt;&#xA;&lt;li&gt;Commericial scientific apps (ABAQUS, ANSYS, FLUENT and many more in industries such as oil/finance/engg etc.) do not run (natively) on OS X&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Linux (Debian) Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First class package management i.e., installation of compilers, most numerical/scientific libraries etc. is a command away&lt;/li&gt;&#xA;&lt;li&gt;Unlike Fink/Macports the packages are much better supported (in Debian you have the option of using bleeding edge/testing/stable versions)&lt;/li&gt;&#xA;&lt;li&gt;Most cluster(s) run some version of Debian/Red Hat so less hassle in porting code(s)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Linux (Debian) Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No standard UI&lt;/li&gt;&#xA;&lt;li&gt;Linux on desktop/laptop(s) may require some tweaking to get everything (suspend/resume, 3D video acceleration, sound etc.) working but this has improved a lot in recent years&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Baring some exceptions most users and cluster/system administrators find OS X easier for desktop productivity and NOT for scientific computing (compiling, using, developing stuff). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OTOH most Linux users find the latter easier than former and this is reflected int the entire HPC/scientific computing ecosystem.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-01-03T18:10:46.343" LastActivityDate="2012-01-03T18:10:46.343" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="565" PostTypeId="2" ParentId="549" CreationDate="2012-01-03T18:17:59.497" Score="5" Body="&lt;p&gt;My personal favourite is the book by John Strikwerda, &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898717938&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Finite Difference Schemes and Partial Differential Equations&quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He has a really nice treatment of the stability theory using Fourier analysis.  I only have the first edition, where he doesn't introduce the idea of a stability region.  According to the SIAM website, the second edition has added this material.&lt;/p&gt;&#xA;" OwnerUserId="319" LastEditorUserId="81" LastEditDate="2017-07-07T23:41:51.250" LastActivityDate="2017-07-07T23:41:51.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="566" PostTypeId="2" ParentId="338" CreationDate="2012-01-03T19:34:54.017" Score="5" Body="&lt;p&gt;As you may already know if you're asking this question, Gillespie's method is often used in computational systems biology.  There are a number of software system that provide implementations; you may want to browse the &lt;a href=&quot;http://sbml.org/SBML_Software_Guide&quot; rel=&quot;nofollow&quot;&gt;SBML Software Guide&lt;/a&gt; to find some of them (specifically, the ones that are known to support SBML, something that I realize may not be relevant to your needs). However, Geoff Oxberry is correct that Python-based implementations of Gillespie's algorithms seem to be uncommon compared to C++ and Java implementations.  In addition to StochPy, I'm aware of the following Python-based packages (in alphabetical order):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://cbbl.imim.es:8080/ByoDyn&quot; rel=&quot;nofollow&quot;&gt;ByoDyn&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sloppycell.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;SloppyCell&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://steps.sourceforge.net/STEPS/Home.html&quot; rel=&quot;nofollow&quot;&gt;STEPS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If, in your application, you can use software that's coded in something other than Python but can be &lt;em&gt;wrapped&lt;/em&gt; with a Python interface, then there is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://cain.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;Cain&lt;/a&gt; &amp;mdash; the core is C++, but it includes a Python-based interface&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://copasi.org/tiki-read_article.php?articleId=41&quot; rel=&quot;nofollow&quot;&gt;COPASI language bindings&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;several more options as mentioned by Geoff Oxberry in his reply on this page&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Finally, if you don't need your simulation to be run locally and can instead use web services, then there are the following options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.basis.ncl.ac.uk/WebServices.html&quot; rel=&quot;nofollow&quot;&gt;BASIS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(the simulation service is currently off-line, but will return) &lt;a href=&quot;http://sysbioapps.dyndns.org/&quot; rel=&quot;nofollow&quot;&gt;SBW&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;ByoDyn's online services (see link above)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These lists are still incomplete, but hopefully they will help.&lt;/p&gt;&#xA;" OwnerUserId="283" LastEditorUserId="283" LastEditDate="2012-01-04T20:12:36.537" LastActivityDate="2012-01-04T20:12:36.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="567" PostTypeId="2" ParentId="561" CreationDate="2012-01-04T03:50:24.910" Score="0" Body="&lt;p&gt;It's relatively obvious that you can't in general get away with fewer quadrature points per cell than there are degrees of freedom. In the case of trilinear elements on a 3d hexahedron, there are 8 degrees of freedom (one per vertex) so the minimum number of quadrature points would be eight as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Now, as for the argument why that is so, here's a sketch: We know that a finite element is only good if it satisfies the &quot;unisolvency&quot; condition, i.e., there needs to be a single solution. The problem with too few quadrature points is that the bilinear form you get using quadrature instead of the exact integral can't distinguish between different shape functions any more. To give an example, think of forming the mass matrix in 1d when using a discontinuous element with basis {1,x} on the reference element [-1,1]. Let's assume that we have a mesh with only one element, which is also the interval [-1,1]. The exact mass matrix is, of course, [[2,0],[0,2/3]], which is invertible as should be. On the other hand, if you approximate the integral using the midpoint rule the mass matrix would be [[2,0],[0,0]], which is not invertible and consequently completely useless. The reason is that a one-point quadrature formula can't distinguish between all linear functions (part of the trial space) that have the same value at the quadrature point; in other words, for the midpoint rule, the shape function 'x' is the same as the function '0' is the same as the function '-x'. In other words, while the trial space has dimension 2 with exact integrals, for the midpoint rule the space has dimension 1, even though there are two degrees of freedom -- that's the definition of a space that's not unisolvent.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-01-04T04:35:06.220" LastActivityDate="2012-01-04T04:35:06.220" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="568" PostTypeId="2" ParentId="556" CreationDate="2012-01-04T04:00:23.977" Score="1" Body="&lt;p&gt;Both Maple and Mathematica have easy interfaces to numerically solve differential equations and plot their results. Here&#xA;  &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/teaching/2010-fall-442/project-tools.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/teaching/2010-fall-442/project-tools.pdf&lt;/a&gt;&#xA;  &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/teaching/2010-fall-442/project-tools.mw&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/teaching/2010-fall-442/project-tools.mw&lt;/a&gt;&#xA;is an example with Maple, complete with cool graphs.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-01-04T04:00:23.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="570" PostTypeId="1" AcceptedAnswerId="573" CreationDate="2012-01-04T08:36:59.877" Score="2" ViewCount="528" Body="&lt;p&gt;Moore's law states that the number of transistors on an integrated circuit grow exponentially, roughly doubling at a period of 20 months. This affects the amount of memory available and the speed of computation, which roughly double at the same rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also affects the speed of floating-point computation, if the precision is fixed. But I wonder whether one can observe a similar growth in the precision available (for scientific computing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a follow-up, what would such growth imply for scientific computing? For example, if we regard a linear systems of equations $Ax = b$, then the condition number of the matrix $A$ describes how strong the solution $x$ is affected if we successively drop decimal places of $b$. The larger the condition number, the more change in $x$, the worse for computing. In the reverse direction, any increase in precision would naturally lead to an improved exactness in the solution, at rate a proportional to the condition number, which were a good thing. How can this be interpreted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS:&#xA;I have found engineering literature that, similarly, warns the reader that, e.g., a high convergence rate in numerical methods, i.e. $O(h^t)$ with large $t$, on the other hand means a strong detoriation if $h$ is increased, where $h$ denotes step-size or mesh-size in the usual contexts. This is similar to my point of view on the condition number (but may be equally wrong).&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-01-04T18:38:44.980" Title="Is there a Moore's law for floating-point precision, and what would it imply?" Tags="|floating-point|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="571" PostTypeId="2" ParentId="540" CreationDate="2012-01-04T12:52:16.883" Score="5" Body="&lt;p&gt;According to Jed's answer, MG uses moderate coarsening while DD uses rapid coarsening.&#xA;I think this makes a difference when they are parallelized.  There will be multiples of&#xA;communications and synchronizations for MG to go through many levels of coarsening that&#xA;are equivalent to a single coarsening of DD. Another point from Jed's answer is MG uses &#xA;cheap smoother and DD uses strong smoother. Considering the two points, it has been reported&#xA;that MG at coarse levels will have bad communication/computation ratios. So according to&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Amdahl%27s_law&quot; rel=&quot;nofollow&quot;&gt;Amdahl's law&lt;/a&gt;, the parallel speedup is not good.&#xA;A remedy of this is parallel coarse grids correction such as &lt;a href=&quot;http://www.ams.org/journals/mcom/1990-55-191/S0025-5718-1990-1023042-6/S0025-5718-1990-1023042-6.pdf&quot; rel=&quot;nofollow&quot;&gt;BPX preconditioner&lt;/a&gt;.  Besides, MG can use DD as smoother as Adams pointed out, and MG can also&#xA;be used within subdomains of DD.  Based on the considerations Barker have pointed out,&#xA;I guess using MG within DD is better, which exploits both parallelsim of DD and optimal &#xA;complexity of MG. &lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2012-01-10T09:31:59.440" LastActivityDate="2012-01-10T09:31:59.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="572" PostTypeId="1" CreationDate="2012-01-04T15:34:21.567" Score="3" ViewCount="466" Body="&lt;p&gt;I'm looking for the algorithm of Preparata and Supowit for testing a simple polygon for monotonicity in linear time.&#xA;I've found it referenced in many textbooks but I can't find the algorithm itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the information on the original article: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://md1.csa.com/partners/viewrecord.php?requester=gs&amp;amp;collection=TRD&amp;amp;recid=0164490CI&quot; rel=&quot;nofollow&quot;&gt;http://md1.csa.com/partners/viewrecord.php?requester=gs&amp;amp;collection=TRD&amp;amp;recid=0164490CI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="507" LastActivityDate="2012-01-04T16:55:43.493" Title="Testing a simple polygon for monotonicity in linear time question" Tags="|algorithms|geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="573" PostTypeId="2" ParentId="570" CreationDate="2012-01-04T16:04:08.617" Score="10" Body="&lt;p&gt;I honestly do not think there is a very strong analogy here: double precision has been the standard since 1985 (&lt;a href=&quot;http://en.wikipedia.org/wiki/IEEE_754-1985&quot;&gt;IEEE 754-1985&lt;/a&gt;), and there are libraries that can compute in arbitrary precision, e.g., &lt;a href=&quot;http://gmplib.org/&quot;&gt;GMP&lt;/a&gt;, although there is a significant drop in the number of floating point operations that can be computed per second (flops). If one was to try to draw any analogy to Moore's law, it would have to be limited to hardware-based floating point precision, and the timeframe for the doubling would at least be 25 years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am very much in favor of efficient high precision computation becoming more standard, but higher precision only reduces one of the sources of error; for example, small perturbations in the formation of the right-hand side could still be catastrophic.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-01-04T16:04:08.617" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="574" PostTypeId="2" ParentId="570" CreationDate="2012-01-04T16:47:32.600" Score="5" Body="&lt;p&gt;As far as perspective goes for scientific computing, it's not clear to me that additional precision in floating point computation would matter beyond a certain point. The most precise calculations of physical constants from measurements tend to be accurate to 10 or 11 significant figures, and at most 14. Typically, that level of precision isn't necessary even for scientific work, unless you're working at a place NIST or CERN, or in certain subdisciplines of physics. It is possible to represent that level of precision in inputs using double precision arithmetic; after that, accuracy of calculations depends on a variety of factors. I think it's important to keep the application in mind, however; there's no point in obsessing about accuracy to 10 decimal places if you can't measure what you're modeling to more than a part in 1,000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @JackPoulson points out, all increasing the level of precision does is decrease unit roundoff, which is one factor in error bounding results like those found in &lt;a href=&quot;http://books.google.com/books?id=epilvM5MMxwC&amp;amp;lpg=PP1&amp;amp;dq=accuracy%20and%20stability%20of%20numerical%20algorithms&amp;amp;pg=PA17#v=onepage&amp;amp;q=accuracy%20and%20stability%20of%20numerical%20algorithms&amp;amp;f=false&quot;&gt;&lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;&lt;/a&gt; (second edition) by Nick Higham. It is not a replacement for good practices in numerical analysis like proper scaling of models, good preconditioners, good choice of algorithms (such as using a QR factorization for least squares problems instead of solving the normal equations). The link above to page 17 of Higham's book provides a more thorough discussion of what increasing the precision gives you in terms of accuracy; he shows an example where increasing the precision does not yield any additional accuracy at all.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-04T18:38:44.980" LastActivityDate="2012-01-04T18:38:44.980" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="575" PostTypeId="2" ParentId="572" CreationDate="2012-01-04T16:55:43.493" Score="1" Body="&lt;p&gt;I assume that you're just looking for a link to the actual article? If so, it can be found &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0020019081900910&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. It is unfortunately behind a pay wall, so I hope that your university has access.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-01-04T16:55:43.493" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="576" PostTypeId="2" ParentId="541" CreationDate="2012-01-04T18:48:25.020" Score="3" Body="&lt;p&gt;Another alternative is &lt;a href=&quot;https://software.sandia.gov/opt++/index.html&quot; rel=&quot;nofollow&quot;&gt;OPT++&lt;/a&gt;. It supports linear and nonlinear constraints with an efficient nonlinear interior point solver, provides control for function accuracy (if numerical differentiation is required), control for step sizes, etc. I'm typically optimizing large implicit functions (e.g. FEM) where these types of controls can be useful.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="276" LastEditDate="2012-01-12T01:52:17.513" LastActivityDate="2012-01-12T01:52:17.513" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="577" PostTypeId="1" CreationDate="2012-01-05T02:34:00.207" Score="7" ViewCount="1862" Body="&lt;p&gt;For the final project in my computational physics class, &lt;a href=&quot;https://github.com/Amichai/ComputationalPhysics/blob/master/Computational1/ThreeDIsing.cs&quot; rel=&quot;noreferrer&quot;&gt;I've built&lt;/a&gt; and will be presenting results for monte carlo simulations of phase transition in the three dimensional ising model. Using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm&quot; rel=&quot;noreferrer&quot;&gt;metropolis algorithm&lt;/a&gt;, I've been able to graph the magnetism of the system after a long period of evolution against different starting temperatures and have been able to estimate the &lt;a href=&quot;http://en.wikipedia.org/wiki/Critical_point_%28thermodynamics%29&quot; rel=&quot;noreferrer&quot;&gt;critical point&lt;/a&gt; of the three dimensional ising model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eg: Below are simulation results showing the relationship between the system parameter beta (=Temperature * Boltzmann's constant / interaction energy) and the magnetism of the system per each spin after equilibration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/i5s3v.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;sub&gt;(For each trial represented in the graph above, the system was iterated/perturbed using the metropolis algorithm 10,000,000 times from a fully magnetized state to get an equilibrium state and then perturbed 1,000 more times while statistics for the average magnetization per spin were taken. System size: 50^3 spins.)&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also want to calculate &lt;a href=&quot;http://en.wikipedia.org/wiki/Critical_exponent&quot; rel=&quot;noreferrer&quot;&gt;critical exponents&lt;/a&gt;, but I don't know much about them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What critical exponents are “important” or relevant to the ising model simulation and what is the basic algorithm for determining them? Any introductory reading material on this issue would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="94" LastEditorUserId="94" LastEditDate="2012-01-05T06:53:14.170" LastActivityDate="2014-10-02T10:34:09.027" Title="3d Ising model simulation - what critical exponents should I be looking for and how do I find them?" Tags="|monte-carlo|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="578" PostTypeId="2" ParentId="533" CreationDate="2012-01-05T03:44:27.840" Score="5" Body="&lt;p&gt;An important limitation to density functional theory is DFT's characteristically poor treatment of long-range noncovalent interactions. Many functionals give very incorrect results for $\pi\cdots\pi$ stacking, hydrogen bonding and noble gas VdW dimerisation, however quite recent functionals such as the 'Minnesota' family of Trular and coworkers are specifically designed with ameliorating these drawbacks in mind, and they have been successfully used in modelling pathological cases like &lt;a href=&quot;http://pubs.rsc.org/en/Content/ArticleLanding/2008/CP/b717744e&quot;&gt;supramolecular complexation of buckyballs to corranulene-bearing 'buckycatchers'&lt;/a&gt;. In my (limited) experience, however, implementations of the meta-hybrid M06 functional are rather computationally expensive with respect to GGAs. Meanwhile, the pairwise-parameterised &lt;a href=&quot;http://toc.uni-muenster.de/DFTD3/&quot;&gt;DFT-D methodologies of Grimme and coworkers&lt;/a&gt; represent another promising avenue.&lt;/p&gt;&#xA;" OwnerUserId="512" LastActivityDate="2012-01-05T03:44:27.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="579" PostTypeId="2" ParentId="561" CreationDate="2012-01-05T08:34:42.280" Score="4" Body="&lt;p&gt;As far as finite element solid mechanics simulations are concerned, you can't use less than 8 quadrature points without using stabilization forces. In case of incompressible material (your case), the best solution for accuracy purpose is to use mixed formulation. You can refer to the book by Simo and Hughes : &lt;a href=&quot;http://books.google.fr/books/about/Computational_inelasticity.html?hl=fr&amp;amp;id=ftL2AJL8OPYC&quot; rel=&quot;nofollow&quot;&gt;http://books.google.fr/books/about/Computational_inelasticity.html?hl=fr&amp;amp;id=ftL2AJL8OPYC&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="517" LastActivityDate="2012-01-05T08:34:42.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="580" PostTypeId="1" CreationDate="2012-01-05T08:53:13.893" Score="6" ViewCount="584" Body="&lt;p&gt;Suppose we have an element in the polar coordinate as $0\leq r\leq 1$, $0\leq\theta\leq \alpha$. What are the correct basis functions in this element?  For this kind of mesh, there&#xA;are a lot of references on finite difference discretization. But I can not find a fem solution.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="276" LastEditDate="2012-01-06T20:03:54.690" LastActivityDate="2012-05-01T04:19:46.720" Title="How to do FEM in sector elements?" Tags="|finite-element|basis-set|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="581" PostTypeId="2" ParentId="577" CreationDate="2012-01-05T11:32:19.603" Score="6" Body="&lt;p&gt;For the Ising model, the relevant quantities are properties such as the correlation length, the magnetic susceptibility, and the heat capacity. What makes finding these exponents challenging is that the calculations have to be performed in the neighborhood of the critical point: you need to be close enough to the critical temperature so that you can see the changes in the measured properties, but not so close to it that the numerical values start getting unreliable as a result of the intrinsic divergence. (It's basically a log-log fitting problem, where you've done a number of simulations at temperatures in the vicinity of the critical point, and then fit the results.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for references, both of the books cited in the linked Wikipedia article, by Yeomans or by Stanley, offers a good introduction. There's also a good book by &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0198513933&quot; rel=&quot;nofollow&quot;&gt;Binney et al.&lt;/a&gt; that offers a lot of explanation on the topic (and has the additional bonus of having lots of examples with solutions!).&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-01-05T22:21:40.200" LastActivityDate="2012-01-05T22:21:40.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="582" PostTypeId="1" AcceptedAnswerId="585" CreationDate="2012-01-06T10:36:09.587" Score="17" ViewCount="4707" Body="&lt;p&gt;Consider $Ax=b$ with $A$ nearly singular which means there is an eigenvalue $\lambda_0$ of $A$ that is very small. The usual stop criterion of an iterative method is based on the residual $r_n:=b-Ax_n$ and regards the iterations can stop when $\|r_n\|/\|r_0\|&amp;lt;tol$ with $n$ the iteration number. But in the case we are considering, there could be large error $v$ living in the eigenspace associated with the small eigenvalue $\lambda_0$ which gives small residual $Av=\lambda_0v$.  Suppose the initial residual $r_0$ is large, then it might happen we stop at $\|r_n\|/\|r_0\|&amp;lt;tol$ but the error $x_n-x$ is still big.  What is a better error indicator in this case?  Is $\|x_{n}-x_{n-1}\|$ a good candidate? &lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="123" LastEditDate="2012-01-11T14:31:31.350" LastActivityDate="2012-07-25T20:32:49.090" Title="Stopping criteria for iterative linear solvers applied to nearly singular systems" Tags="|linear-algebra|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="583" PostTypeId="1" CreationDate="2012-01-06T12:06:02.433" Score="6" ViewCount="735" Body="&lt;p&gt;I'm thinking a bit about the Front Tracking method used for simulation of Two phase flow with sharp interfaces. The literature tells me that the main issue is the surface mesh representation (operations + container) of the fluidic interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone here know about a good C++ library for dynamically (truly dynamical: free storage, removal and deletion of elements from memory, not fictitius logical removal) adaptive surface meshes? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen some results used in the field of Computational Graphics where such surface meshes are reconstructed from point information that is recorded by cameras, and are evolved with very intensive topological changes... also, there are curvature based self-intersection algorithms out there used for cloth motion simulations... The knowledge seems to be out there, but there is not much happening in the OpenSource world. &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="549" LastEditDate="2012-01-15T16:45:39.497" LastActivityDate="2012-01-16T13:58:13.583" Title="Surface Mesh Library" Tags="|software|geometry|mesh-generation|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="585" PostTypeId="2" ParentId="582" CreationDate="2012-01-06T15:11:11.093" Score="14" Body="&lt;p&gt;Please &lt;strong&gt;never&lt;/strong&gt; use the difference between successive iterates to define a stopping criteria. This misdiagnoses stagnation for convergence. Most nonsymmetric matrix iterations are not monotone, and even GMRES in exact arithmetic with no restarts may stagnate for an arbitrary number of iterations (up to the dimension of the matrix) before converging suddenly. See examples in &lt;a href=&quot;http://dx.doi.org/10.1137/0613049&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nachtigal, Reddy, and Trefethen (1993)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A better way to define convergence&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;We are usually interested in the accuracy of our solution more than the size of the residual. Specifically, we might like to guarantee that the difference between an approximate solution $x_n$ and the exact solution $x$ satisfies $$|x_n - x| &amp;lt; c$$ for some user-specified $c$. It turns out that can achieve this by finding an $x_n$ such that $$|A x_n - b| &amp;lt; c\epsilon$$ where $\epsilon$ is the smallest singular value of $A$, due to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;|x_n - x| &amp;amp;= |A^{-1} A (x_n - x)| \\&#xA;&amp;amp; \le \frac 1 \epsilon |A x_n - A x| \\&#xA;&amp;amp; = \frac 1 \epsilon |A x_n - b| \\&#xA;&amp;amp; &amp;lt; \frac 1 \epsilon \cdot c \epsilon = c&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we have used that $1/\epsilon$ is the largest singular value of $A^{-1}$ (second line) and that $x$ exactly solves $A x = b$ (third line).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Estimating the smallest singular value $\epsilon$&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;An accurate estimate of the smallest singular value is usually not directly available from the problem, but it can be estimated as a byproduct of a conjugate gradient or GMRES iteration. Note that although estimates of the largest eigenvalues and singular values is usually quite good after only a few iterations, an accurate estimate of the smallest eigen/singular value $\epsilon$ is usually only obtained once convergence is reached. Before convergence, the estimate will generally be significantly larger than the true value. This suggests that you must actually solve the equations before you can define the correct tolerance $c\epsilon$. An automatic convergence tolerance that takes a user-provided accuracy $c$ for the solution and estimates the smallest singular value $\epsilon$ with the current state of the Krylov method might converge too early because the estimate of $\epsilon$ was much larger than the true value.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Notes&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The above discussion also works with $A$ replaced by the left-preconditioned operator $P^{-1}A$ and the preconditioned residual $P^{-1} (A x^n - b)$ or with the right-preconditioned operator $A P^{-1}$ and the error $P (x_n - x)$. If $P^{-1}$ is a good preconditioner, the preconditioned operator will be well-conditioned. For left-preconditioning, this means the preconditioned residual can be made small, but the true residual may not be. For right preconditioning, $|P(x_n - x)|$ is easily made small, but the true error $|x_n-x|$ may not be. This explains why left-preconditioning is better for making error small while right-preconditioning is better for making the residual small (and for debugging unstable preconditioners).&lt;/li&gt;&#xA;&lt;li&gt;See &lt;a href=&quot;https://scicomp.stackexchange.com/a/1057/119&quot;&gt;this answer&lt;/a&gt; for more discussion of norms minimized by GMRES and CG.&lt;/li&gt;&#xA;&lt;li&gt;The estimates of extremal singular values can be monitored using &lt;code&gt;-ksp_monitor_singular_value&lt;/code&gt; with any PETSc program. See &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPComputeExtremeSingularValues.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;KSPComputeExtremeSingularValues()&lt;/a&gt; to compute singular values from code.&lt;/li&gt;&#xA;&lt;li&gt;When using GMRES to estimate singular values, it is crucial that restarts not be used (e.g. &lt;code&gt;-ksp_gmres_restart 1000&lt;/code&gt; in PETSc).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-07-25T20:32:49.090" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="586" PostTypeId="1" CreationDate="2012-01-06T16:26:19.503" Score="10" ViewCount="254" Body="&lt;p&gt;There is at least one quite comprehensive &lt;a href=&quot;http://nines.cs.kuleuven.be/ecf/&quot;&gt;encyclopaedia&lt;/a&gt; of quadrature rules that doesn't seem to have been updated in quite a while and has restricted access.  This source refers to several classical and modern sources, and generally is well put together.  However, it approaches the construction of quadrature rules from the purely theoretical approach and therefore misses out on more practical methods for, say, finite element computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does a more multidisciplinary compendium for quadrature rules exist, or does anyone know of an open-source library that implements a wide swath of such methods for simple domains (such as those used for finite elements)?&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="143" LastEditDate="2012-01-06T20:39:21.523" LastActivityDate="2012-01-06T21:24:04.483" Title="Quadrature rules, methodologies, and references" Tags="|libraries|finite-element|quadrature|" AnswerCount="1" CommentCount="3" CommunityOwnedDate="2012-01-12T21:05:43.290" ContentLicense="CC BY-SA 3.0" />
  <row Id="588" PostTypeId="2" ParentId="583" CreationDate="2012-01-06T19:24:17.833" Score="1" Body="&lt;p&gt;Try VTK with netgen. I've been using them for years, they are very useful. but they do not come without issues.&lt;/p&gt;&#xA;" OwnerUserId="474" LastActivityDate="2012-01-06T19:24:17.833" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="589" PostTypeId="1" AcceptedAnswerId="591" CreationDate="2012-01-06T21:15:32.547" Score="2" ViewCount="953" Body="&lt;p&gt;I am trying to put together a complete list of all of the tools that computational scientists have found useful when trying to munge data, i.e. take data in one format, extract the useful bits, and put it into a different format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what are the libraries that you've used in the past to obtain data from fixed-width, CSV, XML, etc. files, to sort the data, and to output it into useful formats?&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="34" LastEditDate="2012-01-06T23:15:25.223" LastActivityDate="2012-02-16T20:04:07.953" Title="List of data munging libraries" Tags="|libraries|data-sets|pdb|data-management|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="590" PostTypeId="2" ParentId="586" CreationDate="2012-01-06T21:24:04.483" Score="2" Body="&lt;p&gt;The libmesh finite element library (libmesh.sf.net) has several families of rules (up to relatively high order) for different finite element types.  You might have a look there...&lt;/p&gt;&#xA;" OwnerUserId="520" LastActivityDate="2012-01-06T21:24:04.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="591" PostTypeId="2" ParentId="589" CreationDate="2012-01-06T22:21:37.690" Score="4" Body="&lt;p&gt;My favorite language for these tasks is Python; Perl also works for those who are so inclined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python modules:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://docs.python.org/library/csv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;csv&lt;/a&gt; module from the Python standard library handles parses CSV files.&lt;/li&gt;&#xA;&lt;li&gt;The Python standard library also has modules for &lt;a href=&quot;http://docs.python.org/library/markup.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;standard markup languages&lt;/a&gt; like XML, HTML, SGML, etc.&lt;/li&gt;&#xA;&lt;li&gt;For fixed-width strings, see &lt;a href=&quot;https://stackoverflow.com/questions/848537/writing-parsing-a-fixed-width-file-using-python&quot;&gt;this post on Stack Overflow&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I haven't used this module, but supposedly &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2744707/&quot; rel=&quot;nofollow noreferrer&quot;&gt;p3d&lt;/a&gt; is a Python module that will help with parsing protein data in pdb format.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://docs.python.org/library/re.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;regular expressions&lt;/a&gt; module in Python can be quite useful.&lt;/li&gt;&#xA;&lt;li&gt;You can also Google or check the &lt;a href=&quot;http://pypi.python.org/pypi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python Package Index&lt;/a&gt; for other Python packages that suit your needs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Perl modules:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://search.cpan.org/~makamaka/Text-CSV-1.21/lib/Text/CSV.pm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Text::CSV&lt;/a&gt; module works for CSV processing. I've used it, and it's relatively intuitive.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://search.cpan.org/~grantm/XML-Simple-2.18/lib/XML/Simple.pm&quot; rel=&quot;nofollow noreferrer&quot;&gt;XML::Simple&lt;/a&gt; and &lt;a href=&quot;http://search.cpan.org/~msergeant/XML-Parser-2.36/Parser.pm&quot; rel=&quot;nofollow noreferrer&quot;&gt;XML::Parser&lt;/a&gt; modules seem like they would work for XML processing, though there are a lot of XML modules available on the &lt;a href=&quot;http://search.cpan.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comprehensive Perl Archive Network&lt;/a&gt; (CPAN), so feel free to pick one that suits your preference.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://search.cpan.org/~jfearn/HTML-Tree-4.2/lib/HTML/Tree/Scanning.pod&quot; rel=&quot;nofollow noreferrer&quot;&gt;HTML::Tree::Scanning&lt;/a&gt; modules seem to serve the same sort of purpose for HTML.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/1494611/how-can-i-extract-columns-from-a-fixed-width-format-in-perl&quot;&gt;This post&lt;/a&gt; from Stack Overflow talks about extracting fixed-width data from files using Perl.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://perldoc.perl.org/perlre.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Regular expressions&lt;/a&gt; are baked into Perl.&lt;/li&gt;&#xA;&lt;li&gt;CPAN is also a good source to look for other libraries.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;C++ libraries:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are a bunch of XML parser libraries (see &lt;a href=&quot;https://stackoverflow.com/questions/170686/best-open-xml-parser-for-c&quot;&gt;this post&lt;/a&gt; on Stack Overflow). Particularly reputable are &lt;a href=&quot;http://expat.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Expat&lt;/a&gt;, &lt;a href=&quot;http://xerces.apache.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Xerces&lt;/a&gt;, and the C++ bindings to &lt;a href=&quot;http://gdome2.cs.unibo.it/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gdome2&lt;/a&gt; (my personal favorite).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/1120140/csv-parser-in-c&quot;&gt;This post&lt;/a&gt; from Stack Overflow talks about extracting data from CSV files.&lt;/li&gt;&#xA;&lt;li&gt;For other standard formats, there's a good chance there's a parser out there (HTML, etc.). A lot of times, the Perl and Python versions of parser libraries are language bindings for C or C++ parser libraries.&lt;/li&gt;&#xA;&lt;li&gt;For fixed-width files, you can use &lt;a href=&quot;http://www.cplusplus.com/reference/clibrary/cstdio/fprintf/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fprintf&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=2&amp;amp;ved=0CDcQFjAB&amp;amp;url=https://stackoverflow.com/questions/181624/c-what-regex-library-should-i-use&amp;amp;ei=eHIHT7a-FqHb0QGu_42EAg&amp;amp;usg=AFQjCNF0B87qu8nZzPI9p6SHGhwVs5WL4w&quot; rel=&quot;nofollow noreferrer&quot;&gt;This post&lt;/a&gt; from Stack Overflow talks about C++ regular expression libraries; also see &lt;a href=&quot;http://www.boost.org/doc/libs/1_48_0/libs/regex/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boost::Regex&lt;/a&gt; and &lt;a href=&quot;http://www.pcre.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCRE&lt;/a&gt; (from C).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;C libraries:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Again, there are many XML parser libraries. Worth checking out are &lt;a href=&quot;http://expat.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Expat&lt;/a&gt;, &lt;a href=&quot;http://gdome2.cs.unibo.it/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gdome2&lt;/a&gt;, and &lt;a href=&quot;http://xmlsoft.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;libxml2&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;A lot of people seem to roll their own C libraries for CSV parsing. I found &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=6&amp;amp;ved=0CFwQFjAF&amp;amp;url=http://www.chilkatsoft.com/refdoc/c_CkCsvRef.html&amp;amp;ei=SnMHT5rZBOj10gHs8ICJAg&amp;amp;usg=AFQjCNFEuDmblEIF8lk567mMO5sNUVmMFw&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation for a library&lt;/a&gt;, but it seems a little sketchy.&lt;/li&gt;&#xA;&lt;li&gt;For fixed-width files, you can again use &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=4&amp;amp;ved=0CEAQFjAD&amp;amp;url=http://pubs.opengroup.org/onlinepubs/009695399/functions/printf.html&amp;amp;ei=u3MHT_DBBYX30gG58cXFAg&amp;amp;usg=AFQjCNG2QKGjWOpmVUl2yHXdGpDvVA_RZQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;fprintf&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The best C regular expression library I know of is &lt;a href=&quot;http://www.pcre.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCRE&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-01-06T22:21:37.690" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="592" PostTypeId="1" AcceptedAnswerId="614" CreationDate="2012-01-06T22:28:46.350" Score="10" ViewCount="3886" Body="&lt;p&gt;I read &lt;a href=&quot;https://scicomp.stackexchange.com/a/585/81&quot;&gt;this&lt;/a&gt; answer, and realized I have been using the difference between sucessive iterates to define a stop criterion for an iterative method of finding eigenvalues/vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are good stop criteria for iterative methods that converge to eigenvalues and eigenvectors?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-09T22:18:45.390" Title="What is a good stop criterion when using an iterative method to find eigenvalues?" Tags="|linear-algebra|eigensystem|iterative-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="593" PostTypeId="1" AcceptedAnswerId="596" CreationDate="2012-01-06T23:20:42.833" Score="71" ViewCount="45298" Body="&lt;p&gt;The Fast Fourier Transform algorithm computes a Fourier decomposition under the assumption that its input points are equally spaced in the time domain, $t_k = kT$. What if they're not? Is there another algorithm I could use, or some way I could modify the FFT, to account for what is effectively a variable sampling rate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the solution depends on how the samples are distributed, there are two particular situations I'm most interested in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Constant sampling rate with jitter: $t_k = kT + \delta t_k$ where $\delta t_k$ is a randomly distributed variable. Suppose it's safe to say $|\delta t_k| &amp;lt; T/2$.&lt;/li&gt;&#xA;&lt;li&gt;Dropped samples from an otherwise constant sampling rate: $t_k = n_k T$ where $n_k \in\mathbb{Z}\ge k$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Motivation: first of all, this was one of the higher voted questions on the &lt;a href=&quot;http://area51.stackexchange.com/proposals/28815?phase=definition&quot;&gt;proposal&lt;/a&gt; for this site. But in addition, a while ago I got involved in a discussion about FFT usage (prompted by a &lt;a href=&quot;https://stackoverflow.com/questions/8582559/determining-frequency-of-an-array-in-python&quot;&gt;question on Stack Overflow&lt;/a&gt;) in which some input data with unevenly sampled points came up. It turned out that the timestamps on the data were wrong, but it got me thinking about how one could tackle this problem.&lt;/p&gt;&#xA;" OwnerUserId="103" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2019-01-16T08:56:14.003" Title="How do I take the FFT of unevenly spaced data?" Tags="|fourier-analysis|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="594" PostTypeId="5" CreationDate="2012-01-06T23:38:31.027" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-06T23:38:31.027" LastActivityDate="2012-01-06T23:38:31.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="595" PostTypeId="4" CreationDate="2012-01-06T23:38:31.027" Score="0" Body="A linearly independent set of elements of a vector space such that any element in that vector space can be expressed as a linear combination of the elements of the basis." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-07T06:15:09.770" LastActivityDate="2012-01-07T06:15:09.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="596" PostTypeId="2" ParentId="593" CreationDate="2012-01-06T23:54:30.017" Score="51" Body="&lt;p&gt;There is a wide variety of techniques for non-uniform FFT, and the most efficient ones are all meant for exactly your case: quasi-uniform samples. The basic idea is to smear the unevenly sampled sources onto a slightly finer (&quot;oversampled&quot;) uniform grid though local convolutions against Gaussians. A standard FFT can then be run on the oversampled uniform grid, and then the convolution against the Gaussians can be undone. Good implementations are something like $C^d$ times more expensive than a standard FFT in $d$ dimensions, where $C$ is something close to 4 or 5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend reading &lt;a href=&quot;http://math.nyu.edu/faculty/greengar/glee_nufft_sirev.pdf&quot;&gt;Accelerating the Nonuniform Fast Fourier Transform&lt;/a&gt; by Greengard and Lee.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There also exist fast, i.e., $O(N^d \log N)$ or faster, techniques when the sources and/or evaluation points are sparse, and there are also generalizations to more general integral operators, e.g., Fourier Integral Operators. If you are interested in these techniques, I recommend &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1654725&quot;&gt;Sparse Fourier transform via butterfly algorithm&lt;/a&gt; and &lt;a href=&quot;http://authors.library.caltech.edu/16481/&quot;&gt;A fast butterfly algorithm for the computation of Fourier Integral Operators&lt;/a&gt;. The price paid in these techniques versus standard FFT's is a much higher coefficient. &lt;em&gt;Disclaimer:&lt;/em&gt; My advisor wrote/cowrote those two papers, and I have spent a decent amount of time parallelizing those techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important point is that &lt;em&gt;all&lt;/em&gt; of the above techniques are approximations that can be made arbitrarily accurate at the expense of longer runtimes, whereas the standard FFT algorithm is exact.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2013-04-16T23:09:12.863" LastActivityDate="2013-04-16T23:09:12.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="597" PostTypeId="1" AcceptedAnswerId="599" CreationDate="2012-01-07T05:10:37.983" Score="11" ViewCount="338" Body="&lt;p&gt;For some simple convex domain $\Omega$ in 2D, we have some $u(x)$ satisfying the following equation:&#xA;$$&#xD;&#xA;-\mathrm{div}(A\nabla u)+cu^n = f&#xD;&#xA;$$&#xA;with certain Dirichlet and/or Neumann boundary conditions. To my knowledge, applying Newton's method in a finite element space would be a relative straightforward way to numerically solve this equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&#xA;(1) Is there a Sobolev theory for the well-posedness of the corresponding variational formulation of this equation assuming zero Dirichlet boundary condition? If so, what Banach space should we consider? &#xA;(2) What are the possible numerical approaches for this type of equation?&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2012-01-07T10:55:03.993" Title="What are the possible numerical schemes for a diffusion equation with a nonlinear reaction term?" Tags="|pde|finite-element|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="598" PostTypeId="2" ParentId="163" CreationDate="2012-01-07T08:56:42.550" Score="2" Body="&lt;p&gt;There's quite an extensive discussion in &lt;a href=&quot;http://books.google.co.uk/books/about/Probability_theory.html?id=tTN4HuUNXjgC&quot; rel=&quot;nofollow&quot;&gt;Jaynes, E. T., 2003, Probability Theory: The Logic of Science, Chapter 11&lt;/a&gt; which is probably worth slogging through.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, there is often a separation in stats/ML between the model being proposed, and the optimisation methods that are then used to solve them. In the case of simple models, the optimisation principles are often also simple. However it is sometimes the case that a seemingly simple model requires complex optimisation procedures in order to be efficient, and I believe that is the case here. Of course many different approaches can be taken to solve a single problem, and the most efficient on may be dependent on other factors (sample size, dimension, rank deficiency of the data etc). A recent book, &lt;a href=&quot;http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&amp;amp;tid=12674&quot; rel=&quot;nofollow&quot;&gt;Optimization for Machine Learning&lt;/a&gt;, collects together a series of papers on the subject, which provide a pretty comprehensive coverage of the area.&lt;/p&gt;&#xA;" OwnerUserId="136" LastActivityDate="2012-01-07T08:56:42.550" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="599" PostTypeId="2" ParentId="597" CreationDate="2012-01-07T10:55:03.993" Score="6" Body="&lt;p&gt;I see two approaches:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Arbitrary f(u). Simply put f ~ f(u0) on the right hand side of the equation, proceed with any non-linear solver, fixed point scheme is a good choice, because you do not have Jacobian anyway. Easiest to implement and use, most general, but possibly inferior performance, because Jacobian can not be exploited (is generally unknown).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) f(u) decomposed into series (polynomial, Fourier). More difficult to implement and use, may be difficult/impossible for some special f. But in return you can calculate and exploit the Jacobian in a Newton-like method, which will generally result in superior performance.&lt;/p&gt;&#xA;" OwnerUserId="474" LastActivityDate="2012-01-07T10:55:03.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="600" PostTypeId="2" ParentId="516" CreationDate="2012-01-07T13:40:29.567" Score="2" Body="&lt;p&gt;I'm using OpenFOAM for a while now (for my PhD as well), and I can only recommend it: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has a huge amount of physical models, and it's written in C++, which means: modularity, encapsulation, automatic parallelism of top level codes, boundary conditions bound to the fields but implemented as a model library, multi physics [6DoF, two phase flow, combustion, all kind of turbulence models, naval hydrodynamics solvers, molecular dynamics... there is a lot in it.. :)], equation mimicking (really easy way of working with new sets of governing equations), etc... btw, its under GPL so it's free to download and modify. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your specific purpose, there is a mesh generator available that does exactly what you want, called &quot;snappyHexMesh&quot;. It takes an STL and intersects it with the underlying unstructured mesh and creates a new mesh ready to be used with OpenFOAM solvers and libraries, so check this out: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.openfoam.org/docs/user/snappyHexMesh.php&quot; rel=&quot;nofollow&quot;&gt;snappyHexMesh official info&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hydroniumion.de/general/snappyhexmesh-tutorial/&quot; rel=&quot;nofollow&quot;&gt;a good tutorial on snappyHexMesh&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-01-07T13:40:29.567" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="601" PostTypeId="2" ParentId="540" CreationDate="2012-01-07T18:04:43.143" Score="1" Body="&lt;p&gt;&lt;strong&gt;@Jed Brown&lt;/strong&gt; About the condition number of one-level DD, I think the size of the subdomain $H$ should be in the denominator to multiply with the mesh size $h$. It is common for the one-level method that using more subdomains needs more iterations. One can also refer to the book of &lt;a href=&quot;http://books.google.ch/books?id=tpSPx68R3KwC&amp;amp;printsec=frontcover&amp;amp;dq=domain%20decomposition&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=EoYIT-mSEYmg4gSNgOWCCg&amp;amp;ved=0CEEQ6AEwAQ#v=onepage&amp;amp;q=domain%20decomposition&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Toselli and Widlund&lt;/a&gt;. For example, the condition number of the Schur complement is $\mathcal{O}(\frac{1}{Hh})$ from page 98 of the book. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another point is about the condition number of optimized Schwarz methods.  Actually, it improves the exponent of $h$ for one-level method (&lt;a href=&quot;http://www.ma.hw.ac.uk/~loisel/papers/osm-condition2.pdf&quot; rel=&quot;nofollow&quot;&gt;non-overlapping&lt;/a&gt; and &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/42540.pdf&quot; rel=&quot;nofollow&quot;&gt;overlapping&lt;/a&gt;) and in addition the exponent of $H$ for &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/coarse.pdf&quot; rel=&quot;nofollow&quot;&gt;two-level method&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2012-01-07T18:04:43.143" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="602" PostTypeId="5" CreationDate="2012-01-07T19:16:19.190" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-07T19:16:19.190" LastActivityDate="2012-01-07T19:16:19.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="603" PostTypeId="4" CreationDate="2012-01-07T19:16:19.190" Score="0" Body="The finite element method is a means of solving ordinary and partial differential equations. The domain of the problem is broken up into elements, and the solution in each element is expanded in a basis of functions. Finite element lends itself well to adaptive refinement and the method allows for good error estimates." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-09T01:53:44.720" LastActivityDate="2012-01-09T01:53:44.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="604" PostTypeId="5" CreationDate="2012-01-07T19:19:18.997" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-07T19:19:18.997" LastActivityDate="2012-01-07T19:19:18.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="605" PostTypeId="4" CreationDate="2012-01-07T19:19:18.997" Score="0" Body="Also called numerical integration, quadrature refers to the approximation of an integral made by evaluating the integrand at a finite number of points." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-09T01:53:47.170" LastActivityDate="2012-01-09T01:53:47.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="606" PostTypeId="2" ParentId="551" CreationDate="2012-01-08T07:30:25.423" Score="0" Body="&lt;p&gt;Did you do convergence tests of your code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an absolute stability point of view (think necessary but not sufficient conditions of stability for nonlinear PDEs) it should be safe to reduce the time step while maintaining the spatial discretization fixed in a method of lines type of discretization. Thus, it could be seen as surprising that the stability appear to worsen when time step is refined. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the backward euler method implemented and solved? Except for linear PDEs where direct solvers can be used immediately, it is typical to use either a fix-point iteration scheme or a Newton-Raphson method for solving the implicit scheme to advance the solution one time step. In both cases it should be favorable to decrease the time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How did you impose boundary conditions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more detailed description of your numerical scheme would be helpful to understand exactly what you have done.&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-01-08T07:30:25.423" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="607" PostTypeId="1" AcceptedAnswerId="610" CreationDate="2012-01-08T10:25:26.997" Score="7" ViewCount="1354" Body="&lt;p&gt;I am currently researching on the viability of using KS methods for solving large dense systems. What I wish to prove (or disprove) is that methods like CG, BiCG and QMR are as good (if not better) than the generic LU or QR decomposition methods in place today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Till now I have tried non-preconditioned versions of CG on symmetric positive definite matrix (duh!) and compared it against LU(DGESV). (I have written so in Fortran and MATLAB as well) The results are not pleasing. CG loses by orders of magnitude. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written (or found and optimized) codes for QMR, BiCG, CGNE, CGNR and others as well but their results are worse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I have two choices&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Try improve BLAS Level 2 routines to yield a better convergence. ( I use Intel MKL in Fortran). But, it turns out, there is nothing I can do to make this better. Or can I? Profiling shows bad use of threads and matrix-vector performance. Which is not surprising for BLAS 1/2 operations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Work on preconditioners and hope that I come across a good one which solves a linear system faster than LU/QR etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(Bonus Option) Leave this idea and try something else for my thesis. (I have time left to do this). Maybe something else in the domain of Krylov Subspace Methods.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any help is much appreciated.&#xA;P.S. I had no clue this group existed. This is awesome!&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="9" LastEditDate="2012-03-23T16:07:26.333" LastActivityDate="2012-03-23T16:07:26.333" Title="Krylov Subspace Methods for Dense Systems" Tags="|linear-algebra|parallel-computing|iterative-method|krylov-method|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="608" PostTypeId="1" AcceptedAnswerId="612" CreationDate="2012-01-08T10:40:56.970" Score="11" ViewCount="773" Body="&lt;p&gt;Concurrent to my research on Krylov Subspace Methods, I have the option of exploring mathematics behind HPC a step ahead or the theory of computation (hardware, OS, compilers etc.). &#xA;Currently, I know both enough to &lt;em&gt;just&lt;/em&gt; get by. For instance, I know how to derive the equations for CG and the basics of iterative methods but I am clueless about the details and more complicated stuff like Preconditioners and Convergence. Similarly, I know the basics of Finite Element Method (Weak Form, Non weak form, stuff like Codomain and Galerkin and stuff) but won't know the depths of it. On the computational front, I know how to code serially in all possible languages and can use OpenMP and MPI sufficiently well. I don't understand hardware and caching all that well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: What should one concentrate on: Math or Computation? Are they inseparable in HPC? Is it recommended for one to learn about one and not the other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT : I am currently majoring in Mechanical Engineering (which I regret) and have tons of courses in engineering and computation (fluids, heat transfer and so on). I will be joining graduate school for HPC this year and I want to strengthen some aspect (Math/Comp/Hybrid) before I begin my grad studies. I like math and comp equally (so the &quot;Do what you enjoy more&quot; is redundant).&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="123" LastEditDate="2012-01-09T05:35:49.643" LastActivityDate="2012-01-11T08:58:10.927" Title="Is it preferable to concentrate on studying math or computation?" Tags="|parallel-computing|education|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="609" PostTypeId="2" ParentId="608" CreationDate="2012-01-08T11:20:20.090" Score="12" Body="&lt;p&gt;HPC is a blend of mathematics, computation, computer science, and applications. You need to be able to understand all of them to be truly successful in the long run. However, you do not necessarily need to achieve the same level of proficiency in all of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the computation versus mathematics argument, for an engineer, I'd argue the numerical implementation issues are more important at first. If you wait until you've learned the mathematical theory and then begin implementation, you may spend a long time working on things that, while no doubt useful, may not directly impact your thesis research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I'd lean towards understanding the computational aspects at first, and then going back and filling out the holes in the mathematical theory. Hardware issues can also be learned--but a lot of how that affects software will also be platform-dependent, so again, it may not be the first item on your agenda.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Others may of course disagree with me; as you stated, this is more of an opinion piece than a factual question.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-08T11:20:20.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="610" PostTypeId="2" ParentId="607" CreationDate="2012-01-08T13:53:54.177" Score="13" Body="&lt;p&gt;I am extremely surprised that there is no mention of conditioning or the shape of the spectrum in your discussion, as it will be the decisive property in whether or not iterative methods can beat dense methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an extreme example, suppose that your dense matrix is some small perturbation of the identity matrix. Then most iterative methods will converge in $O(1)$ iterations, and dense matrix-vector multiplication is $O(n^2)$, so the solve cost is $O(n^2)$ rather than $O(n^3)$. There is clearly a large class of well-conditioned dense matrices that similar arguments could be made for. Whether or not they show up in practice is another question...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, when I generate random dense matrices, the performance of GMRES (and CG for HPD matrices) is rather terrible relative to an LU or QR solver. However, iterative methods would certainly be faster for very large matrices when only a handful of iterations are required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Since I was asked about experimental results, I thought I would show some MATLAB code and some timings which make the point a little more clear. The following generates a random $2000 \times 2000$ real dense SPD matrix with eigenvalues uniformly sampled from [1,2] and compares the timings of unpreconditioned CG (to 6 digits of accuracy) and a Cholesky solver. On my two year old iMac, CG is already four times faster.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    %&#xA;    % Compares CG and a dense Cholesky solve for a well-conditioned SPD matrix&#xA;    %&#xA;    n=2000;&#xA;    S=1;&#xA;    T=2;&#xA;    tol=1e-6;&#xA;&#xA;    % Build an n x n dense SPD matrix with eigenvalues uniformly sampled from &#xA;    % [S,T]&#xA;    D=diag(random('uniform',S,T,n,1));&#xA;    U=randn(n,n);&#xA;    [U,R]=qr(U);&#xA;    clear R;&#xA;    A=U*D*U';&#xA;&#xA;    % Set up a random right-hand side&#xA;    b=randn(n,1);&#xA;&#xA;    % Solve with CG to the specified tolerance, ensuring that the maximum&#xA;    % number of iterations is not reached&#xA;    fprintf(1,'Running CG...\n');&#xA;    tic; [x,flag,relres,iter]=pcg(A,b,tol,10*n); toc;&#xA;    iter&#xA;&#xA;    % Solve using a Cholesky decomposition&#xA;    fprintf(1,'Running a Cholesky solver...\n');&#xA;    opts.LT=false; opts.UT=false; opts.UHESS=false;&#xA;    opts.SYM=true; opts.POSDEF=true;&#xA;    opts.RECT=false; opts.TRANSA=false;&#xA;    tic; x=linsolve(A,b,opts); toc;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I saw the following results:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Running CG...&#xA;  Elapsed time is 0.074933 seconds.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;iter =&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;Running a Cholesky solver...&#xA;  Elapsed time is 0.279619 seconds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now, if $A$ is instead generated as the symmetric product a matrix with normally distributed entries, e.g., &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    A=randn(n,n);&#xA;    A=A'*A;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then the situation is not so good for CG. I saw the following output:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Running CG...&#xA;  Elapsed time is 28.568865 seconds.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;iter =&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    3892&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and the condition number was 1.0587e+09.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-01-08T17:32:34.277" LastActivityDate="2012-01-08T17:32:34.277" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="611" PostTypeId="2" ParentId="607" CreationDate="2012-01-08T15:54:03.410" Score="6" Body="&lt;p&gt;CG was originally abandoned because of this loss of accuracy that you observed.  It was revived when its use as an approximate solver was appreciated.  So I don't think you want to revisit this problem.  And preconditioning is very useful, diagonal preconditioning at the very least.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Iterative methods are very problem dependent so I would recommend finding an application that needs to solve dense systems, or even better move to sparse systems, to focus your research.  N.B., Krylov methods have been extensively investigated in the past 30-40 years, it might be hard to find a good topic here unless you have a lead of some sort or an expert adviser.  There are often properties of your application that can be exploited with preconditioning, which can provide the kernel of a dissertation.&lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2012-01-08T23:43:59.823" LastActivityDate="2012-01-08T23:43:59.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="612" PostTypeId="2" ParentId="608" CreationDate="2012-01-08T20:11:02.600" Score="10" Body="&lt;p&gt;I like aeismail's answer, but I'm going to provide an alternative perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In optimization, it's impossible to really learn the field without understanding real analysis. Even before you tackle numerical issues, you need to understand notions of convergence of sequences, because you are going to prove in classes that algorithms converge. You're going to have to understand concepts like continuity and differentiability on more than a superficial level. Consequently, real analysis is a prerequisite for courses in nonlinear programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thesis relates to methods for solving ordinary differential equations. Convergence issues, specifically things like &quot;if I reduce my local error tolerance, then my calculated numerical solution approaches the true solution of the equations I'm solving&quot; are again issues that require real analysis. To develop the theory for convergence issues required me (against my advisers' wishes) to take two semesters of real analysis. (It paid off with a couple manuscripts.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I know there are people out there who survive quite nicely in numerical methods and HPC without taking pure mathematics classes. It really depends on the niche that you want to occupy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to develop new methods, then theory classes are helpful. Theory classes are also helpful for general mathematical literacy; reading math papers becomes much, much easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to apply specific numerical methods to problems, numerical methods classes are more helpful. I believe this perspective is where aeismail is coming from, and it is a situation more common for engineers. (Disclaimer: We know each other, and graduated from the same department.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for HPC, the impression I've gotten is that experience is the best teacher. I took a parallel programming course, and it was slightly useful, but the main message of the class was to try things and see if they worked. If it's important for your thesis research, you'll get experience in HPC. If it's not, you won't, and it probably won't matter until you want to switch gears and tackle HPC problems. My thesis hasn't been especially HPC-heavy, at least in terms of what I program, so I haven't needed to pick up that set of skills.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To wrap up, you should probably concentrate on getting background in issues that relate to your thesis problem, keep in mind what you think you want to do in the future, and decide what broad, general background you need to communicate with other researchers in the community you'd like to join. Your PhD is going to be one of the last opportunities for you to take classes, and if you think you do want to learn math theory (or any subject, really), learning it on your own is considerably harder without establishing some sort of basic proficiency first.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-08T20:11:02.600" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="613" PostTypeId="2" ParentId="608" CreationDate="2012-01-08T21:23:31.617" Score="6" Body="&lt;p&gt;I agree with both aeismail and Oxberry. I decided to write an answer because you seem to be facing the same questions I was trying to find an answer to last year. I also majored in mechanical engineering (and hated it, specially solid mechanics), I spent a lot of time working with numerical methods in CFD or optimization. Now I am doing my masters in Applied Mathematics and Computational Sciences. From my point of view you first need to decide on what you want to do in the future. If you want to go into modeling or development of numerical methods then you should definitely go towards math. I spent two years working with Finite Volume and Finite Element methods without knowing the deep basis and now that I am taking classes in applied math it is all making a lot more sense to me. I realize how the methods work exactly and I am no longer walking blindly just experimenting with everything. It saves a lot of time and effort. But if you decide you want to go into developing software and related topics then you might want to focus on the HPC part. In my experience, there are a lot of packages out there that are optimized and ready to use for a lot of numerical applications. So it won't be the best idea for me to spend a lot of time developing my own software so I decided to work more on the math part. &lt;/p&gt;&#xA;" OwnerUserId="497" LastEditorUserId="276" LastEditDate="2012-01-11T08:58:10.927" LastActivityDate="2012-01-11T08:58:10.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="614" PostTypeId="2" ParentId="592" CreationDate="2012-01-08T22:30:53.010" Score="8" Body="&lt;p&gt;Youssef Saad's book &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/eig_book_2ndEd.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Numerical Methods for Large Eigenvalue Problems, 2nd edition&lt;/em&gt;&lt;/a&gt; uses the norm of the residual vector to define convergence criteria. He defines the residual vector as follows on page 59:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a matrix $\mathbf{A} \in \mathbb{C}^{n \times n}$, a putative eigenvalue $\widetilde{\lambda} \in \mathbb{C}$ and a putative eigenvector $\widetilde{\mathbf{u}} \in \mathbb{C}^{n}$ associated with $\widetilde{\lambda}$, the residual vector $\mathbf{r}$ associated with the pair $(\widetilde{\lambda}, \widetilde{\mathbf{u}})$ is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{r} = \mathbf{A}\widetilde{\mathbf{u}} - \widetilde{\lambda}\widetilde{\mathbf{u}}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of the error results in Saad's book are stated in terms of the norm of the residual vector (generally the 2-norm), and he uses the norm of the residual vector as a metric for convergence whenever he presents numerical results. Based on that evidence, the stopping criterion would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\|\mathbf{r}\| &amp;lt; \varepsilon.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.grycap.upv.es/slepc/&quot; rel=&quot;noreferrer&quot;&gt;SLEPc&lt;/a&gt; (based on &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;noreferrer&quot;&gt;PETSc&lt;/a&gt;), appears to use a similar convergence criterion in their &lt;a href=&quot;http://www.grycap.upv.es/slepc/handson/&quot; rel=&quot;noreferrer&quot;&gt;hands-on exercises&lt;/a&gt; (they use $\|\mathbf{r}\|/\|\widetilde{\lambda}\widetilde{\mathbf{u}}\|$ in Exercises 1 and 2 instead).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;a href=&quot;http://www.netlib.org/lapack/&quot; rel=&quot;noreferrer&quot;&gt;LAPACK&lt;/a&gt; does not necessarily use that metric for convergence (see for instance, in &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn15.pdf&quot; rel=&quot;noreferrer&quot;&gt;LAPACK working note (LAWN) #15&lt;/a&gt;, using Jacobi's method for calculating eigenvectors and eigenvalues of symmetric positive definite matrices). The &lt;a href=&quot;http://www.netlib.org/lapack/lawns/&quot; rel=&quot;noreferrer&quot;&gt;LAWNs&lt;/a&gt; are rather dense (pardon the pun) and technical, but if you're interested in seeing what high-quality implementations use for convergence, perhaps they're worth a detailed read.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-09T22:18:45.390" LastActivityDate="2012-01-09T22:18:45.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="615" PostTypeId="2" ParentId="593" CreationDate="2012-01-09T00:02:12.120" Score="10" Body="&lt;p&gt;In signal processing, &lt;a href=&quot;http://en.wikipedia.org/wiki/Aliasing&quot;&gt;aliasing&lt;/a&gt; is avoided by sending a signal through a low pass filter before sampling. Jack Poulson already explained one technique for non-uniform FFT using truncated Gaussians as low pass filters. One inconvenient feature of truncated Gaussians is that even after you have decided on the grid spacing for the FFT (=the sampling rate in signal processing), you still have two free parameters: The width of the Gaussian and the truncation radius. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I therefore prefer the &quot;hat&quot; function with a width of two grid cells as low pass filter. This has the effect that the zeroth Fourier order is exact, and that the lower Fourier orders will converge quadratically. The Fourier transform of the &quot;hat&quot; function is easy to compute (it is the square of the sinc function), which simplifies undoing the convolution after the FFT. Note that the &quot;hat&quot; function is the convolution of the characteristic function of the (centered) unit cell with itself. Any desired convergence rate can be achieved, by convoluting the unit cell more than once with itself, and using the resulting function instead of the &quot;hat&quot; function.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="605" LastEditDate="2012-01-17T13:06:09.477" LastActivityDate="2012-01-17T13:06:09.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="616" PostTypeId="1" CreationDate="2012-01-09T05:16:09.427" Score="8" ViewCount="1092" Body="&lt;p&gt;I have a set of files consisting of randomly selected points from a dataset, each file belonging to a particular class.  Each row in these files contains the coordinates in n-space of the point.  I'd like to compare the distributions in n-space of each of these files - and am inspired by the K-S test for comparing histograms.  From what I've read this method doesn't extend well to multivariate data.  I had previously used PCA - but all of my variance collapsed into a single noisy dimension and clustering methods were useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question - is there a reason I shouldn't just use an average of the K-S values across the histogram for each of the n-dimensions as a metric for the goodness of fit?  Is there a better method for comparing these distributions?  &lt;/p&gt;&#xA;" OwnerUserId="536" LastActivityDate="2012-04-19T15:09:21.240" Title="Kolmogorov–Smirnov test for multivariate data" Tags="|statistics|high-dimensional|pattern-recognition|regression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="617" PostTypeId="2" ParentId="608" CreationDate="2012-01-09T05:34:58.630" Score="9" Body="&lt;p&gt;Take as many courses as you can in both.  I did, and I don't regret it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you're interested in a research career, you can be successful with any mix of the two.  Find collaborators whose knowledge complements yours.  I know a significant amount of mathematics as it relates to accuracy and stability of numerical methods, but much less about HPC.  I have collaborators who know HPC very well, so working together we can get innovative numerical methods running on big machines.  I do the math and they do the computation, for the most part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I think that &lt;strong&gt;math&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is more fundamental&lt;/li&gt;&#xA;&lt;li&gt;is more challenging to learn&lt;/li&gt;&#xA;&lt;li&gt;remains relevant for a longer time period&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;whereas &lt;strong&gt;HPC&lt;/strong&gt; topics&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;change more rapidly&lt;/li&gt;&#xA;&lt;li&gt;can be picked up more easily on your own&lt;/li&gt;&#xA;&lt;li&gt;are less generally useful and more problem/application/machine specific&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is an overgeneralization and will surely attract disagreeing comments.  But I think there is truth to it.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-01-09T05:34:58.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="618" PostTypeId="1" AcceptedAnswerId="629" CreationDate="2012-01-09T09:47:51.987" Score="-2" ViewCount="312" Body="&lt;p&gt;I am looking for some clues for an optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem consists of arriving to a image by optimizing multiple layers with the pixel position probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an overview of the problem:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yHQn2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yHQn2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first image on the left is a zoom of the greyscale image with only two grey values, followed by the probability of each cell taking one value of grey, and the same probability for the value black. Finally, I am looking for a way to arrive to the start image by using only the two layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, I will be working with a complete image &lt;a href=&quot;http://upload.wikimedia.org/wikipedia/commons/f/fa/Grayscale_8bits_palette_sample_image.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;like this one&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Lqb1u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lqb1u.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that this will not give me a great result and that I need to add constraints to my problem; however, I am completely lost on the subject. I have found a lot of documentation for optimization, but not a good tutorial or examples for optimization problems defined in a Euclidean space. Is there documentation that I could read, a basic tutorial or code examples?&lt;/p&gt;&#xA;" OwnerUserId="538" LastEditorUserId="20688" LastEditDate="2020-06-15T05:40:00.183" LastActivityDate="2020-06-15T05:40:00.183" Title="Multi-objective optimization problem - Euclidean space" Tags="|algorithms|optimization|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="619" PostTypeId="1" AcceptedAnswerId="623" CreationDate="2012-01-09T11:44:14.550" Score="9" ViewCount="1026" Body="&lt;p&gt;Indefinite systems of matrices appear for example in the discretization of saddle point problems by mixed finite elements. The system matrix can then be put in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{pmatrix} A &amp;amp; B^t \\ B &amp;amp; C\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is negative (semi)-definite, $C$ is positive (semi-) definite and $B$ is arbitrary. Of course, depending on convention you may use definiteness conditions, but this is pretty much the structure of those matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these methods, Uzawa's method can be employed, which is actually just a &quot;trick&quot; to transform the system into an equivalent semi-definite system that can be solved by Conjugate Gradient, Gradient Descent and the like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I face an indefinite system which does not have such a block structure. Uzawa-type methods do not apply in that case. I am aware of the Minimal Residual method (MINRES) that has been introduced by Paige &amp;amp; Saunders, which is just a three-term recursion and seems to be easy to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Is MINRES generally a good choice, say, for prototyping? Is it of any practical relevance? Preconditioning is no central issue at the moment.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="549" LastEditDate="2012-03-04T22:45:07.170" LastActivityDate="2012-03-04T22:45:07.170" Title="Iterative methods for indefinite systems without block structure" Tags="|linear-algebra|iterative-method|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="621" PostTypeId="2" ParentId="619" CreationDate="2012-01-09T12:56:35.663" Score="1" Body="&lt;p&gt;MINRES is the best choice for this type of problem.&lt;/p&gt;&#xA;" OwnerUserId="540" LastEditorUserId="276" LastEditDate="2012-01-12T21:34:53.413" LastActivityDate="2012-01-12T21:34:53.413" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="622" PostTypeId="2" ParentId="619" CreationDate="2012-01-09T14:47:10.723" Score="4" Body="&lt;p&gt;A related question that might be of interest is &lt;a href=&quot;https://scicomp.stackexchange.com/questions/81/which-sparse-linear-system-solver-to-use&quot;&gt;What guidelines should I follow when choosing a sparse linear system solver?&lt;/a&gt;, although in this case, you would be interested in only the iterative methods. My understanding of iterative methods is that convergence for any given method is heavily dependent on the spectrum of your matrix. Even though you can't use Uzawa's method, you could still try GMRES, Biconjugate stabilized gradient, MINRES, the quasi-minimal residual method, and other iterative methods out there that apply to indefinite matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If coding the various methods is a concern, you could call solvers in your algorithm using a library like &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc&lt;/a&gt;, which implements a variety of iterative linear solvers.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-01-09T14:47:10.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="623" PostTypeId="2" ParentId="619" CreationDate="2012-01-09T15:56:17.893" Score="7" Body="&lt;p&gt;If you are not concerned about preconditioning, then MINRES is the standard choice. However, be aware that MINRES requires a symmetric positive definite preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are concerned with preconditioning, then it is important to consider the structural differences between most saddle point problems and general indefinite problems. Most saddle point problems arise when solving elliptic problems with constraints enforced by Lagrange multipliers. Incompressibility and contact constraints are common examples. For such problems, the operator is coercive on the subspace in which the constraint is satisfied, with Green's functions that decay rapidly. Such problems can be solved efficiently using block preconditioners (preconditioned Uzawa is a member of this family), multigrid with compatible smoothers (e.g. Vanka or based on block decomposition), or multilevel domain decomposition with appropriate local and coarse problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The prototypical example of an indefinite problem that is not a saddle point problem is the Helmholtz equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ -\nabla\cdot \big( a \nabla u \big) - k^2 u = f $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a(x)$ is uniformly bounded above and below by positive constants. For $k$ large, the Green's functions are highly oscillatory which makes preconditioning (and discretization) difficult. Two reasonable approaches are sweeping preconditioners based on perfectly matched layers and &quot;wave-ray multigrid&quot;, as described in answers to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/40/what-is-a-scalable-preconditioner-for-high-frequency-helmholtz&quot;&gt;this question&lt;/a&gt;. Unfortunately, these methods are rather custom for a specific equation and technical to implement.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-01-09T15:56:17.893" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="624" PostTypeId="2" ParentId="608" CreationDate="2012-01-09T16:06:12.823" Score="1" Body="&lt;p&gt;I don't believe in a theory/application dichotomy, but it's also important to approach fields in a way that is not completely out of context. Understanding theory I think provides you with a general intuition on the problem that is very valuable as it keeps you from having to focusing on concrete after concrete (i.e. one particular implementation versus another), and lets you look at the big picture. This understanding however doesn't arise out of a vacuum, and you can't BEGIN at this level.. that's not how the brain works. You can't arrive at the concept of a forest without having ever seen a tree!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is not to say that theory takes a subservient role in this question, either. It is to say that theoretical understanding provides one with significant mental economy when considering a class of problems, but it can't exist without the concretes that drive it (at least in the computational theories). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer your question: If all you are interested in is the implementation, that is the result, but not in improving/altering this implementation, theory won't be that important. If however you wish to produce your own, then you are at a competitive disadvantage with those who understand the theory better. Unless of course you produce your own over the years that happens to be better :)&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2012-01-09T16:06:12.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="625" PostTypeId="2" ParentId="395" CreationDate="2012-01-09T16:12:57.080" Score="2" Body="&lt;p&gt;Have you looked into &lt;a href=&quot;http://molekel.cscs.ch/wiki/pmwiki.php&quot; rel=&quot;nofollow&quot;&gt;Molekel&lt;/a&gt;?&#xA;If you only need (non-fractional) superpositions of vibrations, Molekel should fulfill your needs.&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2012-01-09T16:12:57.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="626" PostTypeId="1" CreationDate="2012-01-09T22:17:02.170" Score="2" ViewCount="1669" Body="&lt;p&gt;I have a question. I need to calculate the computational complexity of image segmentation algorithms. Can anyone please help me? &#xA;For example, I have a screen-size picture with white background containing k randomly-positioned black objects with random sizes (between 90*90 to 110*110) in it. I am going to calculate how long it takes for a rapid current computer to segment all black items in my image.&#xA;For example, use connected component labeling to segment the components: &lt;a href=&quot;http://en.wikipedia.org/wiki/Connected-component_labeling&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Connected-component_labeling&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;One-Pass(Image)&#xA;            [M, N]=size(Image);&#xA;            Connected = zeros(M,N);&#xA;            Mark = Value;&#xA;            Difference = Increment;&#xA;            Offsets = [-1; M; 1; -M];&#xA;            Index = [];&#xA;            No_of_Objects = 0; &#xA;&#xA;   for i: 1:M :&#xA;       for j: 1:N:&#xA;            if(Image(i,j)==1)            &#xA;                 No_of_Objects = No_of_Objects +1;            &#xA;                 Index = [((j-1)*M + i)];           &#xA;                 Connected(Index)=Mark;            &#xA;                 while ~isempty(Index)                &#xA;                      Image(Index)=0;                &#xA;                      Neighbors = bsxfun(@plus, Index, Offsets');&#xA;                      Neighbors = unique(Neighbors(:));                &#xA;                      Index = Neighbors(find(Image(Neighbors)));                                &#xA;                      Connected(Index)=Mark;&#xA;                 end            &#xA;                 Mark = Mark + Difference;&#xA;            end&#xA;      end&#xA;  end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="560" LastEditorUserId="127" LastEditDate="2012-01-15T23:05:47.273" LastActivityDate="2012-01-16T11:17:37.993" Title="Computational Complexity of Image Segmentation algorithms" Tags="|complexity|image-processing|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="627" PostTypeId="1" CreationDate="2012-01-09T23:10:52.487" Score="7" ViewCount="299" Body="&lt;h3&gt;Background and question&lt;/h3&gt;&#xA;&lt;p&gt;I often work with simulations of dynamical systems and I usually track a single parameter &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, such as the number of agents (for agents based models) or the error rate (for neural networks). This parameter usually has some interesting transient behavior. The type and length of this behavior depends on the random seed and simulation parameters. However, after the initial transient period, the system stabilizes and has small (in comparison to the size of the changes in the transient period) thermal fluctuations &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; around some mean value &lt;span class=&quot;math-container&quot;&gt;$x^*$&lt;/span&gt;. This mean value, and the size of thermal fluctuations depends on the simulation parameters, but not on the random seed, and are not known ahead of time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The goal is to have a good way to automatically stop the simulation once it has transitioned from the initial transient period to the long-term behavior.&lt;/strong&gt; We know that this transition will only happen once, and the transient period will have much more volatile behavior than the stable state.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Naive approaches&lt;/h3&gt;&#xA;&lt;p&gt;The naive approach that first pops into my mind (which I have seen used as win conditions for some neural networks, for instance) is to pick to parameters &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;, then if for the last &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; timesteps there are not two points &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$x' - x &amp;gt; E$&lt;/span&gt; then we conclude we have stabilized. This approach is easy, but not very rigorous. It also forces me to guess at what good values of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; should be.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively we can use a time &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; and confidence parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; as in &lt;a href=&quot;https://stats.stackexchange.com/questions/11544/testing-for-stability-in-a-time-series/11750#11750&quot;&gt;this answer&lt;/a&gt; and assume a normal distribution on the error. This will save us the effort of knowing the size of thermal fluctuations.&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$y_t = x_{t + 1} - x_{t}$&lt;/span&gt; be the change in the time series between timestep &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t + 1$&lt;/span&gt;. When the series is stable around &lt;span class=&quot;math-container&quot;&gt;$x^*$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; will fluctuate around zero with some standard error. Take the last &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y_t$&lt;/span&gt;'s and fit a Gaussian with confidence &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; using a function like Matlab's &lt;a href=&quot;http://www.mathworks.com/help/toolbox/stats/normfit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;normfit&lt;/a&gt;. The fit will give us a mean &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; confidence error on the mean &lt;span class=&quot;math-container&quot;&gt;$E_\mu$&lt;/span&gt; and a standard deviation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; with corresponding error &lt;span class=&quot;math-container&quot;&gt;$E_\sigma$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$0 \in (\mu - E_\mu, \mu + E_\mu)$&lt;/span&gt;, then you can accept. If you want to be extra sure, then you can also renormalize the &lt;span class=&quot;math-container&quot;&gt;$y_t$&lt;/span&gt;s by the &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; you found (so that you now have standard deviation &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;) and test with the &lt;a href=&quot;http://www.mathworks.com/help/toolbox/stats/kstest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kolmogorov-Smirnov&lt;/a&gt; test at the &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; confidence level, or one of the other strategies in &lt;a href=&quot;https://stats.stackexchange.com/q/14504/4872&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&lt;p&gt;This question was originally posted to &lt;a href=&quot;https://stats.stackexchange.com/q/11544/4872&quot;&gt;Cross Validated&lt;/a&gt; in a more time-series centered language. It generated some good discussion, but not an answer I was happy with. I tried &lt;a href=&quot;http://metaoptimize.com/qa/questions/6749/testing-for-stability-in-a-time-series&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaOptimize&lt;/a&gt; but the question generated zero interest there. I am not familiar with the tagging on Computational Science, so feel free to edit the tags.&lt;/p&gt;&#xA;" OwnerUserId="535" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2019-06-04T12:18:37.317" Title="Testing for stability of a simulated dynamical system" Tags="|algorithms|statistics|machine-learning|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="628" PostTypeId="2" ParentId="242" CreationDate="2012-01-09T23:27:41.373" Score="1" Body="&lt;p&gt;The best way by far is to include all of your code as supplementary material. If possible, also include files with the relevant random seeds needed to recreate your results. Not only does this allow people to recreate your results (which you might not care about), it also allows them to more easily continue where you left off. This allows for new collaborations and citations to your work. Unfortunately, this comes with the difficulty of forcing you to clean up your code, and make sure its bug free. Hence, it is more an ideal than what is usual in practice. But at the very least, you should archive a version of your code used to produce your results, that way if another researcher asks for code, you can produce it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of the description in your paper, then I would concentrate on a high-level, implementation independent description of the key novel features of the model (this is the practical part most good paper achieve). Concentrate on the features that will change the result qualitatively if they are tweaked. Most models I work with produce quantitative results, but the specific quantities are usually not of interest, only the qualitative behavior (since the parameters are usually far from ones observable in nature). Thus, I focus on describing the parts of the model, that if changed will change the qualitative behavior of the system. If this mindset forces me to describe every last detail of my model down to the implementation, then I know that my model is not very robust, and thus should be scrapped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good way to test if your in-paper description is sufficient, is to ask a friend (or student) who did not work on this project with you to describe how they might implement your model is pseudo-code. If they don't get stuck while trying this (as in they arrive at a sketch of a model which should produce the same qualitative results), then you know you have done a good job of description.&lt;/p&gt;&#xA;" OwnerUserId="535" LastActivityDate="2012-01-09T23:27:41.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="629" PostTypeId="2" ParentId="618" CreationDate="2012-01-10T01:52:37.200" Score="2" Body="&lt;p&gt;I believe you should clarify your question a bit more. But the way most optimization problems work is that you need to define a function (cost function) that you need to optimize. Now this function might have multiple parameters or objectives that depend on the problem you are solving. Once you have identified the function, you should use an optimization algorithm to find desired values of the parameters based on the objectives. Some of the methods you can use are Evolutionary Algorithms such as &lt;a href=&quot;http://www.icsi.berkeley.edu/~storn/code.html&quot; rel=&quot;nofollow&quot;&gt;Differential Evolution&lt;/a&gt;, &lt;a href=&quot;http://www.obitko.com/tutorials/genetic-algorithms/&quot; rel=&quot;nofollow&quot;&gt;Genetic Algorithms&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Particle_swarm_optimization&quot; rel=&quot;nofollow&quot;&gt;Particle Swarm&lt;/a&gt;. You could also use other gradient based methods about which you can probably find extensive literature via Google. I used the following references during my research on shape optimization in aerodynamics:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://books.google.ca/books/about/Differential_evolution.html?id=S67vX-KqVqUC&amp;amp;redir_esc=y&quot; rel=&quot;nofollow&quot;&gt;Differential evolution: a practical approach to global optimization&lt;/a&gt; by Kenneth V. Price, Rainer M. Storn, Jouni A. Lampinen,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://books.google.ca/books/about/Differential_evolution.html?id=kG7aP_v-SU4C&quot; rel=&quot;nofollow&quot;&gt;Differential evolution: in search of solutions&lt;/a&gt; by Vitaliy Feoktistov,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.it-weise.de&quot; rel=&quot;nofollow&quot;&gt;Global Optimization Algorithms Theory and Application&lt;/a&gt; by Thomas Weise.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="497" LastActivityDate="2012-01-10T01:52:37.200" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="630" PostTypeId="2" ParentId="616" CreationDate="2012-01-10T04:11:58.633" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://root.cern.ch/&quot; rel=&quot;nofollow&quot;&gt;ROOT&lt;/a&gt; supports Kolmogorov tests on higher dimensional histograms, and the &lt;a href=&quot;http://root.cern.ch/root/html532/TH2.html#TH2%3aKolmogorovTest&quot; rel=&quot;nofollow&quot;&gt;notes (for the 2D version)&lt;/a&gt; suggest that there is a ambiguity--which they deal with by punting: calculate it &lt;em&gt;both&lt;/em&gt; ways. I don't know if the code contains anymore details, but the comments sometimes have references to papers and the like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some additional interesting comments in the notes to &lt;a href=&quot;http://root.cern.ch/root/html532/TH1.html#TH1%3aKolmogorovTest&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;TH1::KolmogorovTest&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="183" LastEditDate="2012-01-10T23:02:35.617" LastActivityDate="2012-01-10T23:02:35.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="631" PostTypeId="2" ParentId="607" CreationDate="2012-01-10T04:47:36.177" Score="9" Body="&lt;p&gt;In most cases I know of where Krylov methods are used for dense problems, the operator is a low-rank perturbation of the identity (obtained by discretizing a continuum operator which is a compact perturbation of the identity). Such operators appear frequently in boundary integral equations as discretized Fredholm integral operators of the second kind. These operators have a few extreme eigenvalues, but the spectrum decays rapidly to the identity, so Krylov methods can be very effective even though the system is ill-conditioned due to the extreme eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these problems, the operator might be applied using some fast method such as FMM, but are sometimes assembled using dense summation (especially for 2D problems). If the boundary and/or coefficient structure is very complicated, the decay of the spectrum might not be particularly fast, in which case you might consider using &quot;multigrid of the second kind&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other examples, the &quot;dense&quot; operator might be obtained as a Schur complement, in which case there is likely a fast way to apply it without assembling, and approximations that are useful for preconditioning might be obtained through approximate commutator arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a general rule, Krylov methods are not good general-purpose methods for dense systems, but if your problem has exploitable structure such as good conditioning, decay of the spectrum, fast application, or effective preconditioners, then Krylov methods can certainly be useful.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-10T04:47:36.177" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="632" PostTypeId="2" ParentId="627" CreationDate="2012-01-10T08:04:20.360" Score="2" Body="&lt;p&gt;It seems to me that your problem is simply that you don't have a precise definition of &quot;transient&quot; and &quot;stable&quot; behavior.  If you did, you could just check when the defiinition is satisfied.  Until you use a precise definition, you don't even have a way of determining if one approach for distinguishing the two behaviors is better than another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible source of a better definition: in the theory of continuous dynamical systems, there is a concept of &lt;a href=&quot;http://en.wikipedia.org/wiki/Center_manifold&quot; rel=&quot;nofollow&quot;&gt;center manifold&lt;/a&gt;; all solutions decay exponentially toward the center manifold but travel slowly along the center manifold.  It might be possible to use this to come up with a quantitative definition of what you're calling transient behavior.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-01-12T21:09:35.373" LastActivityDate="2012-01-12T21:09:35.373" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="633" PostTypeId="2" ParentId="626" CreationDate="2012-01-10T08:24:48.830" Score="1" Body="&lt;p&gt;If you have a detailed description of the algorithm (preferably pseudocode), then calculating the complexity is simply a matter of going through it and counting operations.  If you'd like to provide some pseudocode, we could help you more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most introductory books on scientific computing include some examples of counting the number of operations in different algorithms.  &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/text.html&quot; rel=&quot;nofollow&quot;&gt;Trefethen &amp;amp; Bau&lt;/a&gt; has some detailed examples with pictures (the application is different, but the process is the same).&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-01-10T08:24:48.830" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="634" PostTypeId="5" CreationDate="2012-01-10T09:00:28.380" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-10T09:00:28.380" LastActivityDate="2012-01-10T09:00:28.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="635" PostTypeId="4" CreationDate="2012-01-10T09:00:28.380" Score="0" Body="Implicit methods are timestepping methods that use an inversion at every timestep. This allows for much better stability properties than explicit methods, though it comes with a serious speed penalty in some cases.  Examples of implicit methods include Backward Euler and Crank-Nicholson." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-11T08:47:59.870" LastActivityDate="2012-01-11T08:47:59.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="636" PostTypeId="1" CreationDate="2012-01-10T09:09:25.340" Score="5" ViewCount="139" Body="&lt;p&gt;Considering regular matrix approximation inequality&lt;/p&gt;&#xA;&#xA;&lt;p&gt;|| $A - QQ^TA $|| &amp;lt; e&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we try to approximate matrix $A$ by a lower rank orthonormal matrix $Q$. I've read an article on probabilistic algorithms for such an purpose. You simply pick random (Gaussian etc.) vectors $\omega$, then generate vectors from the range of $A$ by $y =A\omega$. Finally, construct $Q$ by these vectors, $y$, after Gram-Schmidt etc. Details can be found in &quot;Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume we have several matrices $A_i$ with &quot;same structure&quot; and each have the same rank $k$. I may give details on the structure if you interest. And we find $Q_i$ for each corresponding matrix $A_i$. Again assume we may define a distribution over $Q_i$ depending on the distribution we use during approximating matrices $Q_i$ (the distribution of $\omega$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, we get a new matrix $A$ with again same structure and rank $k$. But this time we only have a submatrix $A(:,J)$ i.e. only several columns. I want to approximate $Q$ for this new matrix using only given columns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize we have two phase:&lt;br&gt;&#xA;1. We somehow learn a model for $Q$ using given full matrices $A_i$&lt;br&gt;&#xA;2. Given a subsample $A(:,J)$ of a novel matrix $A$ (that is not present in the first phase), we derive its corresponding $Q$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope, you may simply recommend me some useful references. I do not even know where to look. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;" OwnerUserId="571" LastActivityDate="2012-01-10T09:09:25.340" Title="Probabilistic algorithms for matrix approximation" Tags="|statistics|matrix|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="637" PostTypeId="2" ParentId="559" CreationDate="2012-01-10T12:13:10.687" Score="4" Body="&lt;p&gt;I have been using nothing but Macs on the desktop (and laptop) for many years, doing scientific computing and scientific software development among other things. As others have pointed out, the quality of the hardware, the high quality of much Mac-specific software, and the ability to handle Word and Excel when necessary, make the Mac a very nice platform for daily use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also been running a Mac-based compute cluster as an experiment for a while. It's an experiment I am not tempted to do again. Compared to a Linux cluster, I don't see any significant advantages, other than ease of software installation if you have Macs on the desktop anyway (just install the same stuff). The disadvantages stand out clearly, most of all the lack of proper multiuser GUI support. On a Mac, one machine equals one screen and at most one logged-in user. That makes GUI-based tools a pain to use. Compared to that, even plain X-windows under Linux is a joy to use, and then there is VNC and NoMachine NX to do even better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, I know the Mac supports X windows, but most GUI programs for the Mac use the native interface.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2012-01-10T12:13:10.687" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="638" PostTypeId="2" ParentId="627" CreationDate="2012-01-10T16:28:01.480" Score="1" Body="&lt;p&gt;By your use of &quot;thermal fluctuation&quot;, I assume you have some statistical mechanics framework. In that case you might use quantities that are known to converge monotonically, e.g., the entropy of the Boltzman-weighted probability distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a well-defined state space, you could look for an equivalent of the separatrix to determine whether you have crossed into an attractive region, e.g., compute the net flux from one region of state space to another (the 50:50 region could be the separatrix, which would be &quot;transient&quot;). This approach is sometimes taken to identify transition states of chemical reactions.&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2012-01-10T16:28:01.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="639" PostTypeId="2" ParentId="469" CreationDate="2012-01-10T20:01:00.883" Score="3" Body="&lt;p&gt;You may want to look at this series of articles on scientific computing with Cocoa from &lt;a href=&quot;http://www.macresearch.org/cocoa_for_scientists&quot; rel=&quot;nofollow&quot;&gt;MacResearch&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="553" LastActivityDate="2012-01-10T20:01:00.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="640" PostTypeId="1" AcceptedAnswerId="641" CreationDate="2012-01-10T20:43:02.063" Score="13" ViewCount="4438" Body="&lt;p&gt;Many important problems can be expressed as a &lt;a href=&quot;http://en.wikipedia.org/wiki/Mixed_integer_linear_programming#Integer_unknowns&quot;&gt;mixed integer linear program&lt;/a&gt;. Unfortunately computing the optimal solution to this class of problems is NP-Complete. Luckily there are approximation algorithms that can sometimes provide quality solutions with only moderate amounts of computation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I analyze a particular mixed integer linear program to see if it lends itself to one of these approximation algorithms? What are the relevant traits or qualities such a program might possess? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the relevant algorithms in use today and how do these qualities map onto these algorithms? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What software packages should I look to for experimentation?&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-01-11T01:53:51.203" LastActivityDate="2012-01-11T01:53:51.203" Title="Efficient solution of mixed integer linear programs" Tags="|optimization|linear-programming|approximation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="641" PostTypeId="2" ParentId="640" CreationDate="2012-01-10T21:52:31.840" Score="16" Body="&lt;p&gt;Although mixed-integer linear programming (MILP) is indeed NP-complete, there are solvable (nontrivial) instances of mixed-integer linear programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NP-complete means that mixed integer linear programming is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) solvable in polynomial time with a nondeterministic Turing machine (the NP part)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) polynomial time reducible to 3-SAT (the complete part; for the rest of the discussion, this part really doesn't matter)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practical terms, since we don't have nondeterministic Turing machines and can't write algorithms that do things like, &quot;set our binary decision variables to all possible combinations of zero and one, and solve the resulting linear programs (LPs)&quot; in polynomial time, that means that in an asymptotic sense, MILP is $\mathcal{O}(2^{n})$, where $n$ is the number of binary variables. That statement means that as problem instances get very large, they will require large amounts of computing power to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That statement doesn't mean that &quot;small&quot; instances are intractable. Unfortunately, I can't make a precise statement of what small means for an MILP instance. I solve problems that have 3,000 or more binary decision variables on a routine basis. Depending on the problem formulation, the problems could take less than .01 seconds (which is the case for relatively under-constrained problems) or more than an hour (which is the case for problems where many constraints are active), because the problems seem to have favorable structure. I can say that state of the art LP solvers can solve LPs with several million continuous decision variables, and that without special structure, it's highly unlikely that a problem instance with somewhere around 1,000 to 10,000 binary variables will be solvable in reasonable amounts of time with state-of-the-art MILP algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think you have a solvable instance of MILP, you will want to use a branch-and-bound or branch-and-cut algorithm. The best implementations are &lt;a href=&quot;http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/&quot;&gt;CPLEX&lt;/a&gt; and &lt;a href=&quot;http://www.gurobi.com/&quot;&gt;Gurobi&lt;/a&gt;. Both of them are commercial products that have free academic licensing if you dig around enough. If you really need an open source solver, projects in the &lt;a href=&quot;http://www.coin-or.org/&quot;&gt;COIN-OR&lt;/a&gt; community are more appropriate, although the source packages can be finicky sometimes. The most relevant projects would be &lt;a href=&quot;http://www.coin-or.org/projects/Cbc.xml&quot;&gt;the CBC branch-and-cut solver&lt;/a&gt;, &lt;a href=&quot;http://www.coin-or.org/projects/SYMPHONY.xml&quot;&gt;the SYMPHONY solver&lt;/a&gt;, &lt;a href=&quot;http://www.coin-or.org/projects/Bcp.xml&quot;&gt;the BCP branch-cut-price solver&lt;/a&gt;, and &lt;a href=&quot;http://www.coin-or.org/projects/ABACUS.xml&quot;&gt;the ABACUS branch-and-cut solver&lt;/a&gt;. All of these projects will require multiple packages from &lt;a href=&quot;http://www.coin-or.org/&quot;&gt;COIN-OR&lt;/a&gt;, due to its modular structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like the option to try multiple solvers, your best bet is to use the &lt;a href=&quot;http://www.coin-or.org/projects/Osi.xml&quot;&gt;Open Solver Interface&lt;/a&gt; from &lt;a href=&quot;http://www.coin-or.org/&quot;&gt;COIN-OR&lt;/a&gt;. Be advised that portions of this interface will only let you set basic solver options, and that to set advanced options for solvers, you should consult the mailing lists of &lt;a href=&quot;http://www.coin-or.org/&quot;&gt;COIN-OR&lt;/a&gt; for further details. The commercial MILP solvers are MUCH (sometimes an order of magnitude or more) faster than the open source solvers. Another option for prototyping is use of an algebraic modeling language like &lt;a href=&quot;http://gams.com/&quot;&gt;GAMS&lt;/a&gt; or &lt;a href=&quot;http://www.ampl.com/&quot;&gt;AMPL&lt;/a&gt;. Both software packages are commercial, but have trial versions that can be used on small problem instances. For larger problem instances, you could submit GAMS or AMPL files to the &lt;a href=&quot;http://www.neos-server.org/neos/&quot;&gt;NEOS server&lt;/a&gt; to be solved; this server is available to the public.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a sufficiently large instance of MILP, then none of these solvers will work well. You could relax the integer variables to continuous variables, solve the problem, and then round to the nearest collection of integer variables that is a feasible solution of your problem instance. An optimal solution of the LP relaxation of your MILP will give you a lower bound on the optimal objective function value of your MILP (assuming minimization, of course), and a feasible solution of your MILP will give you an upper bound on the optimal objective function value of your MILP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're really lucky and your constraint matrix is &lt;a href=&quot;http://en.wikipedia.org/wiki/Totally_unimodular&quot;&gt;totally unimodular&lt;/a&gt;, then you can use an LP solver to generate integer solutions to your MILP, and you can solve your problem efficiently despite its large size. Other classes of problems have fast approximation algorithms, such as &lt;a href=&quot;http://en.wikipedia.org/wiki/Knapsack_problem&quot;&gt;knapsack problems&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Cutting_stock_problem&quot;&gt;cutting stock problems&lt;/a&gt;. Specialized MILP decomposition algorithms also exist for problems that have special structure, though I am not familiar with the particulars, since those topics are somewhat specialized and outside the scope of my thesis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of a fully &lt;a href=&quot;http://en.wikipedia.org/wiki/Polynomial-time_approximation_scheme&quot;&gt;polynomial time approximation scheme&lt;/a&gt; (FPTAS) specifically for MILP, though an FPTAS of a problem class that includes MILP exists (see &lt;a href=&quot;http://www.math.ucdavis.edu/~deloera/RECENT_WORK/mixed-approximation.pdf&quot;&gt;this paper&lt;/a&gt;). My recommendation would be to use one of the mixed-integer linear programming solvers above in conjunction with a time limit and appropriate tolerances on optimality gaps. Doing so would give you the best possible feasible solution to your MILP within the time limit, and if the solver terminates successfully prior to the time limit, the feasible solution would be optimal to within the optimality gap tolerances you set. This course of action would still give you bounds on the quality of the solution, because your feasible solution would be an upper bound, and the solver could give you an appropriate lower bound. The bound wouldn't be guaranteed to be within a certain factor optimal solution, but then again, any FPTAS will become more expensive as the approximation becomes better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most important thing you can do before you settle on an MILP formulation is to pick the strongest formulation that you can find; you can find advice on how to pick strong formulations in &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1886529191&quot;&gt;Introduction to Linear Optimization&lt;/a&gt; by Bertsimas and Tsitsiklis. The main idea is to pick a formulation whose constraints define a polytope that is as close to the convex hull of the formulation as possible (also see &lt;a href=&quot;http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-251j-introduction-to-mathematical-programming-fall-2009/lecture-notes/MIT6_251JF09_lec24.pdf&quot;&gt;these course notes&lt;/a&gt;). Picking a strong formulation can make a huge difference in the time it takes to solve a problem.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-10T21:57:54.093" LastActivityDate="2012-01-10T21:57:54.093" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="642" PostTypeId="1" AcceptedAnswerId="645" CreationDate="2012-01-10T23:33:51.693" Score="3" ViewCount="605" Body="&lt;p&gt;Suppose I have a scan from an STM image (very much like the things you see &lt;a href=&quot;https://www.google.com/search?q=stm%20lattice&amp;amp;tbm=isch&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;). Suppose I have a simple square lattice with lattice parameter a. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'd like to do is to numerically find the lattice parameter, measured in units of pixels, assuming that calibration is done elsewhere. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A first idea of mine was to have a function that creates a grid of points with lattice constant a, some offsets where the grid starts and also some angle for the rotation of the lattice. I'd then sum the values of the STM image at each grid point and return that. I'd then use some optimization toolbox (MATLAB, Python, ...) to find the parameters that maximize this sum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I run into problems doing this. For example, the grid points are calculated from the angle, lattice constant and offset and are then rounded to an integer value so that I can actually address my 2D image in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ImgData[x][y] &lt;/p&gt;&#xA;&#xA;&lt;p&gt;but most optimization routines will then go on to vary the parameters only very slightly, so that the rounded coordinates don't change. The program then assumes that it has found a local minimum/maximum since the function value doesn't change if the parameters are changed only slightly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other problems regarding the stability of this method, so I wondered if there is a more sophisticated way of doing this in an automated fashion. I could always take the FT of the image data and read off the lattice constant manually, but I'd then still like to put a best-fid grid overlay over my image data so I'd have to optimize for angle and offset.&lt;/p&gt;&#xA;" OwnerUserId="105" LastEditorUserId="183" LastEditDate="2012-01-10T23:48:52.703" LastActivityDate="2012-01-11T06:20:22.903" Title="Fitting a grid to an STM image" Tags="|regression|computer-vision|pattern-recognition|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="644" PostTypeId="2" ParentId="78" CreationDate="2012-01-11T00:55:33.060" Score="9" Body="&lt;p&gt;My limited understanding of AD parallels what Matt has said. To speed up the computation of derivatives, the structure of the expression graph must exploit sparsity and scarcity in the set of Jacobian matrices. (See &lt;a href=&quot;https://link.springer.com/article/10.1007/s10107-002-0329-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;&lt;span class=&quot;math-container&quot;&gt;$^1$&lt;/span&gt; by Griewank for more insight.) The software engineering tricks would likely be in the AD code itself to restructure the expression graph to take advantage of these properties in the set of Jacobian matrices. Knowing how the AD code generates an expression graph from the code you are writing would in turn help you better understand how to write code that requires fewer computations. Any good AD code should already take advantage of intrinsics with common subexpressions, but good AD codes are hard to write.&lt;/p&gt;&#xA;&lt;p&gt;The standard reference in the field is &lt;a href=&quot;https://epubs.siam.org/doi/book/10.1137/1.9780898717761?mobileUi=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Second Edition&lt;/a&gt; by Andreas Griewank and Andrea Walther, and should provide more detailed information on how to reduce the number of computations needed to evaluate the derivative of a program.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Griewank, A., Naumann, U. Accumulating Jacobians as chained sparse matrix products. Math. Program., Ser. A 95, 555–571 (2003). &lt;a href=&quot;https://doi.org/10.1007/s10107-002-0329-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1007/s10107-002-0329-7&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="276" LastEditorUserId="23791" LastEditDate="2021-11-01T15:41:39.233" LastActivityDate="2021-11-01T15:41:39.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="645" PostTypeId="2" ParentId="642" CreationDate="2012-01-11T02:03:34.697" Score="2" Body="&lt;p&gt;What you're doing is something like a fourier transform: you're projecting the image onto a function ($e^{ikx}$ in the case of the FT, and a sort of periodic &quot;block&quot; function in your case).  It might be a little slow, but you could calculate the inner product of the lattice function and your image for every possible lattice function, and then find the maximum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that doesn't work, there's a trick that might make the FT work in your case. You could select some value, and then zero out all parts of your image that are below this value. If the defects in your image are mostly low amplitude, this could make sure the largest FT peak is the correct one.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-11T06:20:22.903" LastActivityDate="2012-01-11T06:20:22.903" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="646" PostTypeId="5" CreationDate="2012-01-11T09:03:35.260" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-11T09:03:35.260" LastActivityDate="2012-01-11T09:03:35.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="647" PostTypeId="4" CreationDate="2012-01-11T09:03:35.260" Score="0" Body="Questions about libraries and standalone software designed or well-suited for scientific or numerical calculations." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-12T20:55:55.627" LastActivityDate="2012-01-12T20:55:55.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="648" PostTypeId="5" CreationDate="2012-01-11T09:06:09.660" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-11T09:06:09.660" LastActivityDate="2012-01-11T09:06:09.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="649" PostTypeId="4" CreationDate="2012-01-11T09:06:09.660" Score="0" Body="A method which produces a sequence of numerical approximations which converges (provided technical conditions are satisfied) to the solution of a problem, generally through repeated applications of some procedure. Examples include Newton's method for root finding, and Jacobi iteration for matrix-vector solves." OwnerUserId="81" LastEditorUserId="276" LastEditDate="2012-08-17T01:45:29.613" LastActivityDate="2012-08-17T01:45:29.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="650" PostTypeId="2" ParentId="582" CreationDate="2012-01-11T13:17:13.497" Score="7" Body="&lt;p&gt;Another way of looking at this problem is to consider the tools from discrete inverse problems, that is, problems which involve solving $Ax=b$ or $\min ||Ax-b||_2$ where $A$ is very ill-conditioned (i.e. the ratio between the first and last singular value $\sigma_1/\sigma_n$ is large).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, we have several methods for choosing the stopping criterion, and for an iterative method, I would recommend the L-curve criterion since it only involves quantities that are available already (DISCLAIMER: My advisor pioneered this method, so I am definitely biased towards it). I have used this with success in an iterative method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to monitor the residual norm $\rho_k=||Ax_k-b||_2$ and the solution norm $\eta_k=||x_k||_2$, where $x_k$ is the $k$'th iterate. As you iterate, this begins to draw the shape of an L in a loglog(rho,eta) plot, and the point at the corner of that L is the optimal choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This allows you to implement a criterion where you keep an eye on when you have passed the corner (i.e. looking at the gradient of $(\rho_k,\eta_k)$), and then choose the iterate that was located at the corner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I did it involved storing the last 20 iterates, and if the gradient $abs(\frac{\log(\eta_k)-\log(\eta_{k-1})}{\log(\rho_k)-\log(\rho_{k-1})})$ was larger than some threshold for 20 successive iterations, I knew that I was on the vertical part of the curve and that I had passed the corner. I then took the first iterate in my array (i.e. the one 20 iterations ago) as my solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also more detailed methods for finding the corner, and these work better but require storing a significant number of iterates. Play around with it a bit. If you are in matlab, you can use the toolbox Regularization Tools, which implements some of this (specifically the &quot;corner&quot; function is applicable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this approach is particularly suitable for large-scale problems, since the extra computing time involved is minuscule. &lt;/p&gt;&#xA;" OwnerUserId="370" LastEditorUserId="370" LastEditDate="2012-01-12T12:17:35.693" LastActivityDate="2012-01-12T12:17:35.693" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="651" PostTypeId="1" AcceptedAnswerId="659" CreationDate="2012-01-11T17:22:15.660" Score="6" ViewCount="455" Body="&lt;p&gt;There are many different linear solvers, some which work best for diagonally dominant matrices, some for symmetric, some for positive definite ones, some for banded matrices, etc... There are direct methods, iterative methods, Krylov space methods, etc... Is there a good heuristic to choose a linear solver that scales well in parallel as the problem size increases?  In particular, is there some sort of block diagram that can specify an efficient (or optimal) solver for specific matrix system types?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-02-01T23:12:19.440" LastActivityDate="2012-02-01T23:12:19.440" Title="What guidelines should I use when choosing a scalable linear solver?" Tags="|linear-algebra|performance|parallel-computing|matrix|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="652" PostTypeId="2" ParentId="583" CreationDate="2012-01-11T17:27:36.617" Score="1" Body="&lt;p&gt;Actually, I'm working on a very similar problem: two phase flow simulation through porous media from X-ray scanned rocks.  If you don't mind using the Octave freeware, there is a program called 'ISO2MESH', which can obtain surface meshes from black &amp;amp; white stacked images.  You can also produce tetrahedral meshes with this program, but its limited in the different file types that it can directly save to.  If you don't mind coding a parser to reconstruct the geometry/meshes in another program, then this program might be of great use to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I almost forgot to mention: ISO2MESH is also a freeware! :) You can obtain it &lt;a href=&quot;http://iso2mesh.sf.net&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="81" LastEditDate="2012-01-12T01:50:30.883" LastActivityDate="2012-01-12T01:50:30.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="654" PostTypeId="2" ParentId="81" CreationDate="2012-01-11T18:26:26.070" Score="8" Body="&lt;p&gt;I completely concur with the answers already given.  I wanted to add that all iterative methods require some sort of initial guess.  The quality of this initial guess can often affect the convergence rate of the method you choose.  Methods like Jacobi, Gauss Seidel, and Successive Over Relaxation all work to iteratively &amp;quot;smoothen out&amp;quot; as much error as possible at each step (&lt;a href=&quot;https://web.archive.org/web/20061130104431/http://www.kfunigraz.ac.at:80/imawww/borzi/mgintro.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;see this paper for details&lt;/a&gt;) .  The first few steps reduce the high frequency error rather quickly, but the low frequency error takes many more itrations to smoothen out.  This is what makes convergence slow for these methods.  In cases such as this, we can accelerate convergence by solving low frequency error (e.g. solving the same problem on a coarser mesh) first, then solving the higher frequency error (e.g. on a finer mesh).  If we apply this concept recursively by divide and conquer, we get what is called a Multi-grid method.  Even if the linear system is not symmetric, there are alternative implementations of multi-grid method for any nonsingular sparse matrix system (e.g. algebraic multi-grid method) which can accelerate the convergence of the solver.  Their scalability on parallel systems, however, is the subject of lots of &lt;a href=&quot;https://doi.org/10.1007/978-3-642-11842-5_5&quot; rel=&quot;nofollow noreferrer&quot;&gt; research&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="22284" LastEditDate="2022-05-09T14:42:02.783" LastActivityDate="2022-05-09T14:42:02.783" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="655" PostTypeId="2" ParentId="651" CreationDate="2012-01-11T21:31:04.663" Score="3" Body="&lt;p&gt;It's probably not even close to optimal, but here's the basic hueristic I use:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't use a direct solver unless the matrix involved is diagonal, tridiagonal, can be easily put into tridiagonal form (as with the split operator method), or has some other property that makes it easy to directly solve.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use whatever iterative solver your favorite library uses otherwise. Iterative solvers tend to be nicer to parallelize.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that there are lots of specific cases where you really need some specific algorithm, as indicated by &lt;a href=&quot;https://scicomp.stackexchange.com/a/378/81&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-11T21:31:04.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="656" PostTypeId="2" ParentId="74" CreationDate="2012-01-11T21:41:49.290" Score="2" Body="&lt;p&gt;It's been a while since I last used either of these packages, but I thought that you could do this in languages such as Mathematica through the use of assertions. Something like Assert[A, Symmetric] tells Mathematica that A is a symmetric matrix, and so on. I don't have access to either handy at the moment, so this is something that would have to be checked.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-03-15T21:12:40.817" LastActivityDate="2012-03-15T21:12:40.817" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="657" PostTypeId="2" ParentId="541" CreationDate="2012-01-12T02:45:01.433" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;http://wwwasdoc.web.cern.ch/wwwasdoc/minuit/minmain.html&quot; rel=&quot;nofollow&quot;&gt;MINUTE&lt;/a&gt; module of &lt;a href=&quot;http://cernlib.web.cern.ch/cernlib/&quot; rel=&quot;nofollow&quot;&gt;CERNLIB&lt;/a&gt; (long since ported to &lt;a href=&quot;http://root.cern.ch/drupal/&quot; rel=&quot;nofollow&quot;&gt;ROOT&lt;/a&gt;) uses a &lt;a href=&quot;http://wwwasdoc.web.cern.ch/wwwasdoc/minuit/node5.html#SECTION00321000000000000000&quot; rel=&quot;nofollow&quot;&gt;transformation on the input space&lt;/a&gt; to render box constrains into a space where they run $[-\infty,+\infty]$ and can thus be handled without special cases (at the cost of some speed, of course).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think that MINUTE will work well for your needs, but the transform may if you are forced to write some or all of the code yourself.&lt;/p&gt;&#xA;" OwnerUserId="183" LastActivityDate="2012-01-12T02:45:01.433" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="658" PostTypeId="2" ParentId="159" CreationDate="2012-01-12T03:20:37.790" Score="6" Body="&lt;p&gt;Thinking back to my second-year mechanics course, it occurs to me that part of reason I have implemented my own versions of well-known algorithms is that I was taught to do it that way. I can't think of a single example where I was taught how to interface to and link in a library in my undergrad physics education. I do have a fond memory of seeing for the first time a list of coordinates of a spinning golf ball, having computed the solution of the coupled Newton equations in FORTRAN myself. There is a certain thrill and satisfaction (even pride) that comes from calculating things from scratch.&lt;/p&gt;&#xA;" OwnerUserId="585" LastActivityDate="2012-01-12T03:20:37.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="659" PostTypeId="2" ParentId="651" CreationDate="2012-01-12T09:13:29.383" Score="8" Body="&lt;p&gt;I presume that by &quot;optimal&quot; you mean &quot;fastest solve&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the PDE solver guys have the best advice when it comes to advice about the most appropriate algorithm for certain discretizations and problems, such as Helmholtz problems (see, for instance, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/40/what-is-a-scalable-preconditioner-for-high-frequency-helmholtz&quot;&gt;this question&lt;/a&gt;) and other cases where problem structure can be exploited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a basic numerical linear algebra perspective, here are some general guidelines:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If your problem is dense, not low-rank, and small enough to fit into memory, direct solvers are a decent choice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your problem is ill-conditioned, preconditioning or regularizing is a good idea.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you can exploit structure (bandedness, symmetry, diagonal dominance), doing so will save you operations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your problem is sparse and small enough, sparse direct solvers can work well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your problem is sparse and large, use an iterative solver, because fill-in will render sparse direct solvers useless. The efficacy of an iterative solver is problem-dependent (see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/81/which-sparse-linear-system-solver-to-use&quot;&gt;this question&lt;/a&gt;) and depends on spectrum. You will probably want to try all of them, unless your problem has a specific structure, or you know from experience that a certain algorithm works well for your problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your problem is dense, and low-rank, you may be able to exploit that low-rank structure, either through use of iterative solvers, or specialized algorithms (which require special problem structure, as alluded to above with PDEs).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When it comes to parallelism, the best algorithms minimize communication, which is accomplished by structuring both the algorithm and the data accordingly. I am not an expert in these matters. I know that block cyclic matrix decompositions are popular for dense numerical linear algebra, and...that's it. I work almost entirely on serial algorithms, so I leave the discussion of parallelism to others.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-01-12T09:13:29.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="660" PostTypeId="1" CreationDate="2012-01-12T14:11:18.340" Score="40" ViewCount="2817" Body="&lt;p&gt;Software is a fundamental part of computational science, and is increasingly recognized as an essential part of the scientific record.  Given the value of using existing and well-tested code, it seems worthwhile to communicate the existence of useful codes as widely as possible and credit their creators.  In an academic setting, this means publishing some papers that are primarily focused on software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can one publish scholarly works whose primary focus is computational software?  To be completely clear, I'm referring to works that may not include any new mathematics, algorithms, etc. -- they are really focused on software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also be interested in hearing from those who have submitted such papers to these journals, what the experience was like and which venues they recommend.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Summary of answers given:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://toms.acm.org/&quot; rel=&quot;noreferrer&quot;&gt;Transactions on Mathematical Software&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.iospress.nl/journal/scientific-programming/&quot; rel=&quot;noreferrer&quot;&gt;Scientific Programming&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.siam.org/journals/sisc/policy.php&quot; rel=&quot;noreferrer&quot;&gt;SIAM Journal on Scientific Computing (SISC) Software section&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://journals.tdl.org/ans&quot; rel=&quot;noreferrer&quot;&gt;The Archive of Numerical Software&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.openresearchcomputation.com/&quot; rel=&quot;noreferrer&quot;&gt;Open Research Computation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/journal/00104655&quot; rel=&quot;noreferrer&quot;&gt;Computer Physics Communications&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/journal/09659978&quot; rel=&quot;noreferrer&quot;&gt;Advances in Engineering Software&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.jstatsoft.org/&quot; rel=&quot;noreferrer&quot;&gt;Journal of Statistical Software&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pubs.acs.org/journal/jctcce&quot; rel=&quot;noreferrer&quot;&gt;Journal of Chemical Theory and Computation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.scfbm.org/&quot; rel=&quot;noreferrer&quot;&gt;Source Code for Biology and Medicine&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.plosone.org/&quot; rel=&quot;noreferrer&quot;&gt;PLoS ONE&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291097-461X&quot; rel=&quot;noreferrer&quot;&gt;International Journal of Quantum Chemistry&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://journals.lww.com/epidem/pages/default.aspx&quot; rel=&quot;noreferrer&quot;&gt;Epidemiology&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.computer.org/portal/web/cise&quot; rel=&quot;noreferrer&quot;&gt;Computing in Science &amp;amp; Engineering&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291096-987X&quot; rel=&quot;noreferrer&quot;&gt;Journal of Computational Chemistry&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.geoscientific-model-development.net/index.html&quot; rel=&quot;noreferrer&quot;&gt;Geoscientific Model Development&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://jmlr.csail.mit.edu/mloss/mloss-info.html&quot; rel=&quot;noreferrer&quot;&gt;Journal of Machine Learning Research&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/mathematics/journal/12532&quot; rel=&quot;noreferrer&quot;&gt;Mathematical Programming Computation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://joss.theoj.org/&quot; rel=&quot;noreferrer&quot;&gt;Journal of Open Source Software&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.journals.elsevier.com/softwarex&quot; rel=&quot;noreferrer&quot;&gt;SoftwareX&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="123" LastEditorUserId="30021" LastEditDate="2019-12-01T13:22:21.573" LastActivityDate="2019-12-01T13:22:21.573" Title="Venues for publishing papers that emphasize software" Tags="|software|publications|" AnswerCount="11" CommentCount="4" CommunityOwnedDate="2012-01-13T09:09:53.130" ContentLicense="CC BY-SA 4.0" />
  <row Id="661" PostTypeId="1" AcceptedAnswerId="666" CreationDate="2012-01-12T14:14:59.723" Score="42" ViewCount="1160" Body="&lt;p&gt;Scholarly papers in scientific computing (and many other fields, nowadays) typically involve some amount of code or even whole software packages that were written specifically for that paper or were used to obtain results in the paper.  What is the best way to help readers of the paper access the code?  My current approach is to put a link to a Github repository (along with a particular version tag) in the paper or in a citation.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-01-30T08:51:11.237" Title="How to link code to publications" Tags="|publications|" AnswerCount="8" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="662" PostTypeId="2" ParentId="660" CreationDate="2012-01-12T15:12:49.547" Score="1" Body="&lt;p&gt;There are some application specific journals&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computers &amp;amp; Geoscience,&#xA;Computers &amp;amp; Fluids,&#xA;Computers &amp;amp; Structures,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2012-01-12T15:12:49.547" CommentCount="1" CommunityOwnedDate="2012-01-13T09:09:53.130" ContentLicense="CC BY-SA 3.0" />
  <row Id="663" PostTypeId="2" ParentId="660" CreationDate="2012-01-12T15:41:28.140" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/journal/09659978&quot; rel=&quot;noreferrer&quot;&gt;Advances in Engineering Software&lt;/a&gt; is a good one and the applications vary pretty widely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.jstatsoft.org/&quot; rel=&quot;noreferrer&quot;&gt;Journal of Statistical Software&lt;/a&gt; is another one that focuses on software, but is application-specific. A lot of R in this one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should add that I've never published in either of these, so I cannot speak to that experience, but I do follow them and find the articles to be good quality.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2012-01-12T15:41:28.140" CommentCount="0" CommunityOwnedDate="2012-01-13T09:09:53.130" ContentLicense="CC BY-SA 3.0" />
  <row Id="664" PostTypeId="1" AcceptedAnswerId="672" CreationDate="2012-01-12T15:51:07.273" Score="4" ViewCount="2650" Body="&lt;p&gt;I have a fortran 90 code that distributes blocks of computations (from a matrix) to multiple nodes in a cluster using MPI, but in each node, the for loops are executed in parallel using openmp.  I tried using the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mpif90 mycode.f&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which produces the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mycode.f(49): (col. 3) remark: LOOP WAS VECTORIZED.&#xA;mycode.f(68): (col. 4) remark: PARTIAL LOOP WAS VECTORIZED.&#xA;mycode.f(83): (col. 2) remark: LOOP WAS VECTORIZED.&#xA;mycode.f(92): (col. 2) remark: LOOP WAS VECTORIZED.&#xA;mycode.f(105): (col. 2) remark: LOOP WAS VECTORIZED.&#xA;mycode.f(114): (col. 2) remark: LOOP WAS VECTORIZED.&#xA;/tmp/ifortAPx6mR.o: In function `MAIN__':&#xA;mycode.f:(.text+0x1f2): undefined reference to `omp_set_num_threads_'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assume that the problem is that I need to set a parameter called &lt;code&gt;omp_set_num_threads_&lt;/code&gt; to a value somewhere, but I'm not sure where.  Any help would be greatly appreciated.&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-16T03:09:39.587" LastActivityDate="2012-01-16T03:09:39.587" Title="How do I compile a program that contains both MPI and OPENMP" Tags="|parallel-computing|fortran|compiling|mpi|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="665" PostTypeId="2" ParentId="660" CreationDate="2012-01-12T15:57:54.593" Score="6" Body="&lt;p&gt;There are some other application-specific journals to list: such as &lt;a href=&quot;http://www.sciencedirect.com/science/journal/00219991&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Journal of Computational Physics&lt;/em&gt;&lt;/a&gt; or &lt;a href=&quot;http://www.sciencedirect.com/science/journal/00104655&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Computer Physics Communications&lt;/em&gt;&lt;/a&gt;, that accept articles both about algorithms as well as the software used to implement them. If you're in the chemistry field, &lt;a href=&quot;http://pubs.acs.org/journal/jctcce&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Journal of Chemical Theory and Computation&lt;/em&gt;&lt;/a&gt; might be another journal to consider. All of these do allow packages to be published—I've seen codes I've used discussed in them. &lt;a href=&quot;http://www.sciencedirect.com/science/journal/00981354/35&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Computers and Chemical Engineering&lt;/em&gt;&lt;/a&gt; does allow software implementation papers, but they need to do something original—it can't be an &quot;incremental advance&quot; paper.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-12T15:57:54.593" CommentCount="4" CommunityOwnedDate="2012-01-13T09:09:53.130" ContentLicense="CC BY-SA 3.0" />
  <row Id="666" PostTypeId="2" ParentId="661" CreationDate="2012-01-12T16:05:19.257" Score="18" Body="&lt;p&gt;Well, I think you have a few options.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you have a stable page—such as one sponsored by a university or other non-profit institution that's unlikely to vanish anytime soon—you could publish there.&lt;/li&gt;&#xA;&lt;li&gt;You could use a service like Github or Bitbucket or SourceForge to distribute the code.&lt;/li&gt;&#xA;&lt;li&gt;If the code is of marginal general value (it's an analysis code for a specific set of conditions, etc.), you could make the code available as a &quot;supplemental information&quot; download with the paper in which you use it.&lt;/li&gt;&#xA;&lt;li&gt;You could use some combination of the above.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In any or all of these cases, however, you should indicate the sourcing clearly in the article, and indicate what kind of licensing it is (GPL, Creative Commons, etc.), so that there's no IP-related issues down the line.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-12T16:05:19.257" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="667" PostTypeId="1" AcceptedAnswerId="670" CreationDate="2012-01-12T16:12:00.237" Score="19" ViewCount="6020" Body="&lt;p&gt;I have taken a basic introduction to Finite Element Method, which did not emphasize a sophisticated understanding of a 'weak formulation'.  I understand that with the galerkin method, we multiply both sides of the (elliptical) PDE by a test function and then integrate (by parts or by Divergence theorem).  Sometimes, I have needed to integrate by parts twice before arriving at the appropriate weak formulation (based upon the answer in the back of the book).  But when I try to apply the same concept to other PDE's (lets say, they are still time-independent), I can't seem to recognize when the formulation is appropriate for discretization.  Is there any 'red flag' that can tell me that THIS FORM can be discretized into a linear system of equations?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, how do I choose an appropriate set of basis functions?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-01-12T17:10:12.207" Title="How to derive the Weak Formulation of a Partial Differential Equation for Finite Element Method?" Tags="|pde|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="668" PostTypeId="1" CreationDate="2012-01-12T16:49:03.533" Score="12" ViewCount="7766" Body="&lt;p&gt;Can anybody help me to find the books on numerical solutions(finite difference and Crank–Nicolson methods) of Poisson and diffusion equations including examples on irregular geometry, such as a domain consisting of the area between a rectangle and a circle (especially books or links on MATLAB code examples in this case)?&lt;/p&gt;&#xA;" OwnerUserId="587" LastEditorUserId="123" LastEditDate="2012-01-15T19:30:09.730" LastActivityDate="2017-07-14T13:30:48.287" Title="Finite differences on domains with irregular boundaries" Tags="|pde|finite-difference|" AnswerCount="6" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="669" PostTypeId="1" AcceptedAnswerId="853" CreationDate="2012-01-12T17:03:00.903" Score="5" ViewCount="331" Body="&lt;p&gt;Suppose I have a black and white image (composed of binary pixel values in a 2D cartesian array) that contains an irregular, nonconvex shape.  Let's further suppose that the shape is one connected region.  Instead of storing each individual pixel location (which may be too costly for very large images), I want to represent the exact same image as a set of 'space-filling' rectangles.  In doing so, each rectangle can be represented by its two antipodal corner points. Thus, there is no need to store information about each point inside the rectangle: we only need to store the matrix coordinates $(i,j)$ of the two opposite corner points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways that one can fill the space with rectangles.  So, my question is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can I fill the space with the FEWEST number of rectangles? (smallest data compression)&lt;/li&gt;&#xA;&lt;li&gt;How can I find this optimal set of space-filling rectangles in polynomial time? (or is this problem NP-hard?)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastEditorUserId="123" LastEditDate="2012-01-13T07:23:54.567" LastActivityDate="2012-01-17T20:12:26.743" Title="How to fill a 2D set over a cartesian lattice with as few rectangles as possible?" Tags="|optimization|geometry|data-management|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="670" PostTypeId="2" ParentId="667" CreationDate="2012-01-12T17:10:12.207" Score="21" Body="&lt;p&gt;Ask yourself the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, how does the integration by parts affect the solvability of the problem, and the space of solutions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, for which space of functions can you build a series of subspaces (the ansatz functions) that you can implement?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us regard the Poisson problem $u&amp;#39;&amp;#39; = f$ for $f \in L^2$, say, on $[0,1]$, with homogenous Dirichlet boundary conditions. By integration, the left and the right side of the equation can be regarded as bounded functionals on $L^2$, say for $\phi \in L^2$ we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi \mapsto \int u&amp;#39;&amp;#39; \phi dx$ and $\phi \mapsto \int f \phi dx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since any function in $L^2$ can be $L^2$-approximated by smooth functions with compact support, both integral functionals are completely known if you only know the values for all test functions. But with the test functions, you can perform integration by parts, and transform the left-hand side to the functional&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi \mapsto -\int u&amp;#39; \phi&amp;#39; dx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read this as: &quot;I take a test function $\phi$, compute its differential, and integrate it with -u' over [0,1], and return you the result.&quot; But that functional is not defined and bounded on $L^2$, since you cannot take the differential of an arbitrary $L^2$ function. They may look extremely strange in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still we observe, that this functional can be extended to the Sobolev space $H^1$, and it is even a bounded functional on $H_0^1$. That means, given $\phi \in H_0^1$, you can roughly estimate the value of $\int -u&amp;#39; \phi&amp;#39; dx$ by a multiple of the $H_0^1$-norm of $\phi&amp;#39;$. And, furthermore, the functional $\phi \mapsto \int f \phi dx$ is, of course, not only defined and bounded on $L^2$, but also defined and bounded on $H_0^1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can, e.g., apply the Lax-Milgram lemma, as it is presented in any PDE-book. A finite element book which describes it as well, only with functional analysis, is e.g. the classic by Ciarlet, or the rather new book by Braess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Lax-Milgram lemma gives PDE-people a nice tool for pure analysis, but they employ much stranger tools as well for their purpose. Still, these tools are also relevant for numerical analysits, because you can in fact build a discretization for these spaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in order to have discrete subspace of $H_0^1$, just take the hat functions. They do not have jumps and are piecewise differentiable. Their differential is a piecewise constant vector field. This construction works in $d=1,2,3,...$, which is fine, but can you come up with an ansatz space whose functions not only have a gradient (that is nice,i.e., square-integrable), but also whose gradients have in turn a divergence? (again, square-integrable). That is pretty hard in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the reason in general how you build weak formulations is that you want to apply the Lax-Milgram lemma, and to have a formulation such that the functions can in fact be implemented. ( For the record, neither is Lax-Milgram the last word in that context, nor are $H_0^1$ ansatz spaces the last word in discretization, see, e.g., Discontinuous Galerkin methods. )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case of mixed boundary conditions, the natural test space may differ from your your search space (in the analytic setting), but I have no idea how to describe that without referring to distribution theory, so I stop here. I hope this is helpful.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-01-12T17:10:12.207" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="671" PostTypeId="2" ParentId="627" CreationDate="2012-01-12T17:48:08.960" Score="1" Body="&lt;p&gt;As @aeismail comments, there doesn't seem to be a general method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I often do is take the &lt;a href=&quot;http://en.wikipedia.org/wiki/Autocorrelation&quot; rel=&quot;nofollow&quot;&gt;autocorrelation&lt;/a&gt; and end the simulation when it is no longer changing. Advantages:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Often part of a library (R, Fortran, Python, etc)&lt;/li&gt;&#xA;&lt;li&gt;Efficient&lt;/li&gt;&#xA;&lt;li&gt;Minimal parameter tweaking required&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="108" LastActivityDate="2012-01-12T17:48:08.960" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="672" PostTypeId="2" ParentId="664" CreationDate="2012-01-12T20:01:50.923" Score="7" Body="&lt;p&gt;mpif90 is just a wrapper for your underlying fortran compiler, which adds the necessary options for finding the necessary libraries and/or include files/modules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You just need to add the options necessary for compiling/linking with OpenMP; that would be -fopenmp (for the gnu compiler suite) or -openmp (for intel); not sure about others.  &quot;remark: LOOP WAS VECTORIZED&quot; looks like the intel compilers to me, so I'd suggest just trying&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mpif90 mycode.f -openmp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2012-01-12T20:01:50.923" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="674" PostTypeId="1" AcceptedAnswerId="683" CreationDate="2012-01-12T20:42:10.470" Score="12" ViewCount="1222" Body="&lt;p&gt;To use the Fast Fourier Transform (FFT) on uniformly sampled data, e.g. in connection with PDE solvers, it is well known that the FFT is an $\mathcal{O}(n\log(n)$) algorithm. How well do the FFT scale when processed in parallel for $n\to\infty$ (i.e. very large)?&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="81" LastEditDate="2012-01-12T22:25:04.253" LastActivityDate="2012-01-14T16:05:18.467" Title="Scalability of Fast Fourier Transform (FFT)" Tags="|pde|fftw|fourier-analysis|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="675" PostTypeId="2" ParentId="660" CreationDate="2012-01-12T20:48:06.833" Score="5" Body="&lt;p&gt;I've published a couple of software papers. One was in &lt;a href=&quot;http://www.scfbm.org/content/3/1/6&quot; rel=&quot;nofollow&quot;&gt;BMC Source Code in Biology and Medicine - &lt;em&gt;The Multiscale Systems Immunology project: software for cell-based immunological simulation&lt;/em&gt;&lt;/a&gt; and &lt;a href=&quot;http://www.plosone.org/article/info%3adoi/10.1371/journal.pone.0024982&quot; rel=&quot;nofollow&quot;&gt;PLoS ONE - &lt;em&gt;SNPpy - Database Management for SNP Data from Genome Wide Association Studies&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are two very different projects. The former is a simulation agent-based system written to model immunology systems, so belongs within computational biology. The latter is a database-based system to manage SNP data in a GWAS context, so belongs within bioinformatics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the publishing experience went - the wait time was low compared to mathematical papers, which was good. The reviewers did not give me a hard time, which was also good. On the cons side, publishing a software paper is insanely time consuming. You have a paper plus a codebase to deal with, not to mention user documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I did find disturbing was that I saw little evidence the reviewers had actually made an effort to run the software and zero evidence they had understood the code design or even looked at the code. Both the journals above had two reviewer reports each. In the former (BMC) case, one reviewer indicated he had tried to install/run the software and failed (an issue we addressed), but of the four reviews, that was the only one that specifically mentioned anything about the actual software. Most of the comments were more general, about scientific issues and addressing points I had made in the paper. Overall, I got the feelings the reviewers had mostly just read the paper. In an ideal world, a reviewer would have some interest in the software design and code details as well, including issues like performance, dependencies, portability, extensibility, and testing. The reviewers for SNPpy did ask about portability and extensibility. as I had claimed in the paper that the software was both portable and extensible, but I don't think they actually looked at the code. In fairness, getting an idea of what a non-trivial codebase is about is hard work, and perhaps more than can be reasonably expected of an unpaid reviewer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used LaTeX for both these papers. In the PLoS case, this involved quite a lot of jumping through hoops to make the manuscript look the way they wanted, since they don't actually use LaTeX and were running some conversion program. PLoS made a mess of converting my figures, which were done using TikZ, and were very nice, but you wouldn't know it looking at their version. It's also worth noting that PLoS doesn't provide manuscript proofs.&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="38" LastEditDate="2012-01-12T20:54:15.083" LastActivityDate="2012-01-12T20:54:15.083" CommentCount="0" CommunityOwnedDate="2012-01-13T09:09:53.130" ContentLicense="CC BY-SA 3.0" />
  <row Id="676" PostTypeId="2" ParentId="674" CreationDate="2012-01-12T22:16:24.437" Score="1" Body="&lt;p&gt;If you have an infinite number of processors, the DFT can be determined in $O(n)$ time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the naive algorithm, you could put each point of output on a seperate node, and calculate that fourier tranformed point in $O(\log n)$ time. Any fast algorithm should be able to at least match this scaling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you also need to collect all of the fourier transformed points in one array, which takes $O(n)$ time.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-12T22:27:38.383" LastActivityDate="2012-01-12T22:27:38.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="677" PostTypeId="2" ParentId="660" CreationDate="2012-01-13T00:29:49.690" Score="4" Body="&lt;p&gt;Two that I haven't seen on this list yet, which may be of interest to someone besides me at some point, is the &lt;em&gt;American Journal of Epidemiology&lt;/em&gt; and &lt;em&gt;Epidemiology&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you've written clever software to accomplish something Epidemiologists care about, they may very well publish it. I've seen recent at least short articles on SAS Macros to accomplish somewhat more sophisticated statistical analysis, a program that performs join-point regression, and one that does some field-specific things with directed acyclic graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turns out one of the mentioned publications was in &lt;em&gt;Epidemiology&lt;/em&gt;: &lt;a href=&quot;http://journals.lww.com/epidem/Fulltext/2011/09000/DAGitty__A_Graphical_Tool_for_Analyzing_Causal.22.aspx&quot; rel=&quot;nofollow&quot;&gt;http://journals.lww.com/epidem/Fulltext/2011/09000/DAGitty__A_Graphical_Tool_for_Analyzing_Causal.22.aspx&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither journal I suspect out and out &lt;em&gt;says&lt;/em&gt; they're looking for articles on software - what with being field-specific journals in a discipline that isn't software, but analysis including software isn't uncommon, and the link is an example of an article focusing entirely on a software product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AJE:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Papers dealing with the practical application of epidemiologic or statistical methods (such as the implementation of statistical techniques or the evaluation of interview strategies) are published in a section entitled Practice of Epidemiology.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Epidemiology: The article above was published as a letter. They also accept software reviews, if you've got a friendly colleague or someone in the field who might be interested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know for a fact they encourage electronic supplements with code, and I've had something at least reviewed there with a link to a Github repository.&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="110" LastEditDate="2012-01-13T07:44:47.913" LastActivityDate="2012-01-13T07:44:47.913" CommentCount="2" CommunityOwnedDate="2012-01-13T09:09:53.130" ContentLicense="CC BY-SA 3.0" />
  <row Id="678" PostTypeId="2" ParentId="660" CreationDate="2012-01-13T02:22:32.717" Score="6" Body="&lt;p&gt;The &lt;a href=&quot;http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291096-987X/homepage/ForAuthors.html&quot; rel=&quot;nofollow&quot;&gt;Journal of Computational Chemistry&lt;/a&gt; (Special Articles Section) and &lt;a href=&quot;http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291097-461X/homepage/ProductInformation.html&quot; rel=&quot;nofollow&quot;&gt;International Journal of Quantum Chemistry&lt;/a&gt; report on software and algorithms.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2012-01-15T19:45:44.150" LastActivityDate="2012-01-15T19:45:44.150" CommentCount="3" CommunityOwnedDate="2012-01-13T09:09:53.130" ContentLicense="CC BY-SA 3.0" />
  <row Id="679" PostTypeId="1" AcceptedAnswerId="680" CreationDate="2012-01-13T04:00:21.617" Score="11" ViewCount="1149" Body="&lt;p&gt;Which is computed faster, $a^b$ or $\log_a c$ or $\sqrt[b]{c}$? $a$, $b$ and $c$ are positive reals with $b&amp;gt;1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kinds of algorithms will you use in the comparison? What are their complexities?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when $c \equiv a^b$ or $c \approx a^b$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question was inspired by the comments on the &lt;a href=&quot;http://math.stackexchange.com&quot;&gt;Mathematics stack exchange&lt;/a&gt; question &lt;a href=&quot;https://math.stackexchange.com/questions/98171/what-is-the-purpose-of-stirlings-approximation-to-a-factorial&quot;&gt;What is the purpose of Stirling's approximation to a factorial?&lt;/a&gt;. Especially, those comments left by &lt;a href=&quot;https://math.stackexchange.com/users/5546/mjqxxxx&quot;&gt;mjqxxxx&lt;/a&gt;, &lt;a href=&quot;https://math.stackexchange.com/users/7933/thomas-andrews&quot;&gt;Thomas Andrews&lt;/a&gt; and me.&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-01-13T14:38:04.347" Title="Which is computed faster, $a^b$, $\log_a c$ or $\sqrt[b]{c}$?" Tags="|computer-arithmetic|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="680" PostTypeId="2" ParentId="679" CreationDate="2012-01-13T05:35:16.627" Score="8" Body="&lt;p&gt;See my answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/527/is-there-software-that-can-autogenerate-numerically-accurate-floating-point-c-ro&quot;&gt;this question&lt;/a&gt; for some related issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, computers can only add, subtract, multiply, divide, and bit shift. For the sake of argument, let's assume that you're &lt;em&gt;not&lt;/em&gt; calculating $a^b$ in the special case where $a$ is a power of 2 and $b$ is a natural number, because that case reduces to a bit shift, and is therefore easy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $b$ is a natural number, and you want to calculate $a^b$, you can use &lt;a href=&quot;http://en.wikipedia.org/wiki/Addition-chain_exponentiation&quot; rel=&quot;nofollow noreferrer&quot;&gt;addition-chain exponentiation&lt;/a&gt;. Every other case in your question is hard (in general).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some fast algorithms used to approximate these functions to high accuracy require black magic. To see what I mean by &quot;black magic,&quot; take a look at &lt;a href=&quot;http://martin.ankerl.com/2007/10/04/optimized-pow-approximation-for-java-and-c-c/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post by Martin Ankerl&lt;/a&gt; and an associated &lt;a href=&quot;http://nic.schraudolph.org/pubs/Schraudolph99.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; he links to in &lt;em&gt;Neural Computation&lt;/em&gt;. Also see the &lt;a href=&quot;http://en.wikipedia.org/wiki/CORDIC&quot; rel=&quot;nofollow noreferrer&quot;&gt;CORDIC&lt;/a&gt; algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar sorts of bit-flipping tricks are explained in &lt;a href=&quot;http://www.hackersdelight.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Hacker's Delight&lt;/em&gt;&lt;/a&gt; (the link is to the companion website for the book).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other ways to calculate good approximations use numerical analysis (see the Wikipedia article on &lt;a href=&quot;http://en.wikipedia.org/wiki/Approximation_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximation Theory&lt;/a&gt;). One bad way to do it is to rig up an appropriate differential equation and integrate it using a numerical method like Euler's method (like I said, a bad approximation, but you can do it). A better way to do it is to use series approximations. Taylor series converges far too slowly, so something like a Padé approximant or some other type of fast-converging series approximation could be used instead (other rational approximants, Chebyshev series, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm you use to approximate the functions above will depend on your architecture, speed requirements, and accuracy requirements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with talking about complexities is that any algorithm is only going to calculate a floating point approximation of the functions you mention, so the run time is certainly going to depend on the accuracy you demand of your approximation. Even taking that into account, I don't think that computational complexity is a good first approximation of performance; the size of your inputs is going to be measured in bits (i.e., the number of bits it takes to represent $a$, $b$, and $c$), which are going to be precision-dependent, rather than depending on the magnitudes of the numerical inputs themselves. For practical purposes, the precision of the numerical representation of numbers isn't going to vary much (single precision, double precision, quad precision), and you typically don't decide to use that precision based on any computational complexity estimates of scalar functions. The most relevant metric is wall-clock time, and unless you're using a special architecture (embedded systems) or your application really demands a fast exponential (see the blog post link and the &lt;em&gt;Neural Computation&lt;/em&gt; link above), the intrinsic libraries in your language of choice are probably just fine.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-13T06:33:07.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="681" PostTypeId="2" ParentId="679" CreationDate="2012-01-13T05:46:03.740" Score="0" Body="&lt;p&gt;Let me see if I can paraphrase the question:&lt;/p&gt;&#xA;&lt;h2&gt;Case 1:  How does &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt; compare to &lt;span class=&quot;math-container&quot;&gt;$\log_a(c)$&lt;/span&gt;, as &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; approaches infinity (the asymptotic sense)?&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;Answer&lt;/strong&gt;:  it really depends on whether or not c has any dependence on a, and how a compares to b (greater than, less than, or equal).&lt;/p&gt;&#xA;&lt;p&gt;I will run through some of the cases that you specified about the relationship between &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Assumption 1&lt;/strong&gt;:  Suppose &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is a constant, then &lt;span class=&quot;math-container&quot;&gt;$\log_a(c) = \ln(c)/\ln(a)$&lt;/span&gt;, which does not even approach infinity at all (it approaches zero).  In this case, &lt;span class=&quot;math-container&quot;&gt;$\log_a(c)$&lt;/span&gt; is an asymptotic lower bound for &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; approaches infinity, but it is not a &amp;quot;tight&amp;quot; asymptotic bound.  Using the nomenclature and notation popularized by Cormen et al (Introduction to Algorithms, 3rd Edition), &lt;span class=&quot;math-container&quot;&gt;$a^b = \omega(\log_a(c))$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Assumption 2&lt;/strong&gt;:  Suppose &lt;span class=&quot;math-container&quot;&gt;$c=a^b$&lt;/span&gt;.  Then, &lt;span class=&quot;math-container&quot;&gt;$\log_a(a^b)=b$&lt;/span&gt;.  Then, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is constant, while &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt; grows without bound as a approaches infinity.  Thus, &lt;span class=&quot;math-container&quot;&gt;$log_a(c)$&lt;/span&gt; again is an asymptotic lower bound, but not a tight one.  So, &lt;span class=&quot;math-container&quot;&gt;$a^b = \omega(\log_a(c))$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Assumption 3&lt;/strong&gt;:  Suppose &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is approximately &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt;.  It is unclear in what sense 'approximately' means here, but if we assume that this means &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt; is a &lt;em&gt;tight&lt;/em&gt; asymptotic bound (that is &lt;span class=&quot;math-container&quot;&gt;$c=\Theta(a^b)$&lt;/span&gt;, then this produces the same result as in assumption 2.&lt;/p&gt;&#xA;&lt;h2&gt;Case 2: Comparing &lt;span class=&quot;math-container&quot;&gt;$\log_a(c)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$c^{1/b}$&lt;/span&gt;.&lt;/h2&gt;&#xA;&lt;p&gt;Again, it depends on the relation between &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;.  Though there are more possibilities than I will enumerate here, we can consider the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Assumption 1&lt;/strong&gt;: Suppose &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is a constant.  Then &lt;span class=&quot;math-container&quot;&gt;$c^{1/b}$&lt;/span&gt; is a constant for any given &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;.  In this case, &lt;span class=&quot;math-container&quot;&gt;$c^{1/b} = o(\log_a(c))$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Assumption 2&lt;/strong&gt;: Suppose &lt;span class=&quot;math-container&quot;&gt;$c = a^b$&lt;/span&gt;.  Then, &lt;span class=&quot;math-container&quot;&gt;$\log_a(c)= a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c^{1/b} = a$&lt;/span&gt;.  Thus, &lt;span class=&quot;math-container&quot;&gt;$\log_a(c) = \Theta(c^{1/b})$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Assumption 3&lt;/strong&gt;: Suppose &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is approximately &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt;.  Again, if we assume this means &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt; is an asymptotically tight bound for &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, then we get the same result as assumption 2.&lt;/p&gt;&#xA;&lt;h2&gt;Case 3: Comparing &lt;span class=&quot;math-container&quot;&gt;$c^{1/b}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt;&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;Assumption 1&lt;/strong&gt;: Suppose &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is a constant.  Then &lt;span class=&quot;math-container&quot;&gt;$c^{1/b}$&lt;/span&gt; is a constant, while &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt; grows without bound as a approaches infinity.  Thus, &lt;span class=&quot;math-container&quot;&gt;$c^{1/b}=o(a^b)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Assumption 2&lt;/strong&gt;: Suppose &lt;span class=&quot;math-container&quot;&gt;$c=a^b$&lt;/span&gt;.  Then &lt;span class=&quot;math-container&quot;&gt;$c^{1/b}=a$&lt;/span&gt;. Since &lt;span class=&quot;math-container&quot;&gt;$b&amp;gt;1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$a^b$&lt;/span&gt; must always grow faster than &lt;span class=&quot;math-container&quot;&gt;$c^{1/b}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There may be a few more cases which I have omitted here, but in the end, it really depends on the assumptions that you make about the relation between the numbers &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; (and which variable is allowed to approach infinity!)&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-01-13T10:54:47.707" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="682" PostTypeId="2" ParentId="674" CreationDate="2012-01-13T07:41:16.240" Score="5" Body="&lt;p&gt;In the context of PDEs, it's important to recognize that the value of $n$ for a required 1D FFT will typically grow like the $d$th root of the total number of grid points, where the dimensionality $d$ is most often 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching for &quot;parallel FFT&quot; or &quot;pseudospectral scalability&quot; on Google Scholar yields a wealth of information that I'm unqualified to assess.  But this seems like a nice recent example of what can be accomplished in practice: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/1003.4322&quot; rel=&quot;nofollow&quot;&gt;A hybrid MPI-OpenMP scheme for scalable parallel pseudospectral computations for fluid turbulence&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Abstract:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A hybrid scheme that utilizes MPI for distributed memory parallelism and OpenMP for shared memory parallelism is presented. The work is motivated by the desire to achieve exceptionally high Reynolds numbers in pseudospectral computations of fluid turbulence on emerging petascale, high core-count, massively parallel processing systems. The hybrid implementation derives from and augments a well-tested scalable MPI-parallelized pseudospectral code. The hybrid paradigm leads to a new picture for the domain decomposition of the pseudospectral grids, which is helpful in understanding, among other things, the 3D transpose of the global data that is necessary for the parallel fast Fourier transforms that are the central component of the numerical discretizations. Details of the hybrid implementation are provided, and performance tests illustrate the utility of the method. It is shown that the hybrid scheme achieves near ideal scalability up to ~20000 compute cores with a maximum mean efficiency of 83%. Data are presented that demonstrate how to choose the optimal number of MPI processes and OpenMP threads in order to optimize code performance on two different platforms. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="123" LastEditorUserId="149" LastEditDate="2012-01-13T12:44:59.960" LastActivityDate="2012-01-13T12:44:59.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="683" PostTypeId="2" ParentId="674" CreationDate="2012-01-13T07:50:02.933" Score="10" Body="&lt;p&gt;This is more anecdotal evidence than demonstrated proof, but it appears that existing implementations for FFT's, such as &lt;a href=&quot;http://www.fftw.org&quot;&gt;FFTW&lt;/a&gt;, have a limit to their scaling ability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we started using LAMMPS's ${\bf k}$-space solvers in very large systems ($O(10^7)$ atoms), we found that the scaling continued, so long as we were able to keep the number of processors small enough that they could fit onto one rack. As soon as we tried to expand further (above about 4K processors, depending on the machine), the scaling broke down—apparently because the communication costs for pushing data between the processors became too large to maintain the scaling. [Recently, to circumvent this problem, they've introduced the ability to dedicate a certain partition of the processor allocation to FFT calculation.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the take-home message here is that FFT should scale up; however, there are sometimes unexpected limitations and interactions that come into play when one moves from the theoretical consideration of an algorithm's performance to its practical implementation on an actual HPC platform.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-13T07:50:02.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="684" PostTypeId="2" ParentId="664" CreationDate="2012-01-13T09:28:06.970" Score="2" Body="&lt;p&gt;Totally normally, with OpenMP enabled in the compiler and you including/linking MPI headers/libraries. Then when you start your job with mpiexec (or equivalent) the parts of your code written using MPI will run as MPI, parts marked with OpenMP pragmas (automatically or not) will run as OpenMP. There is nothing special in compilation, the problem is to write efficent code.&lt;/p&gt;&#xA;" OwnerUserId="474" LastActivityDate="2012-01-13T09:28:06.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="685" PostTypeId="2" ParentId="661" CreationDate="2012-01-13T10:04:26.160" Score="4" Body="&lt;p&gt;You may use some fancy pdf-techniques to simply attach the code to the pdf (that is, the code-files are embedded into the pdf and can be &quot;downloaded&quot; with a click on some button in the pdf). This can be accomplished with the &lt;a href=&quot;http://ctan.org/tex-archive/macros/latex/contrib/attachfile&quot; rel=&quot;nofollow&quot;&gt;attachfile&lt;/a&gt; package, for example.  Of course, this work with preprints (although I don't know if it already works with the arxiv) but you probably get problems with journal-files...&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2012-01-13T10:04:26.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="686" PostTypeId="2" ParentId="668" CreationDate="2012-01-13T10:05:06.057" Score="3" Body="&lt;p&gt;I think a good book at the very beginning is the book by Hackbusch:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/Elliptic_differential_equations.html?id=-ZPc_JYJFHgC&amp;amp;redir_esc=y&quot; rel=&quot;nofollow&quot;&gt;http://books.google.com/books/about/Elliptic_differential_equations.html?id=-ZPc_JYJFHgC&amp;amp;redir_esc=y&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular ch. 4.8, &quot;Discretization in an Arbitrary Domain&quot; might be interesting for you. The German version of that book can be downloaded for free (legally). I don't know whether this holds for the English version as well.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="238" LastEditDate="2012-01-13T10:10:08.030" LastActivityDate="2012-01-13T10:10:08.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="687" PostTypeId="1" CreationDate="2012-01-13T10:08:40.883" Score="3" ViewCount="128" Body="&lt;p&gt;In an application I have to solve a series of positive definite linear systems of the form $A^TA x = A^Tb$ (i.e. normal equations). The next system is obtained from the previous one by adding and/or removing a small number of columns to $A$ (usually one or two). The vector $b$ stays the same throughout. It seems clear that some kind of up-/downdating strategy should be applied, however, my questions are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What kind of up-/downdating shall I choose (QR, Cholesky, Matrix-Inversion-Lemma,others?,...)?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;On what conditions and requirements shall I make my choice (high precision needed,&#xA;  ill-conditioned matrices, well-conditioned ones, speed,...)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;P.S.: You may assume that $A$ and $A^T$ can be applied fast.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2012-01-13T10:08:40.883" Title="Up-/downdating methods for a series of normal equations" Tags="|linear-algebra|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="688" PostTypeId="2" ParentId="679" CreationDate="2012-01-13T10:54:07.003" Score="4" Body="&lt;p&gt;This is a &lt;strong&gt;good&lt;/strong&gt; question because understanding numerical algorithms and performance is an important prerequisite to being an effective computational scientist.  At the same time, it is a &lt;strong&gt;poor&lt;/strong&gt; question because the constraints as posed do not sufficiently qualify it to give a meaningful answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The performance of the three computations will strongly depend on the accuracy needed in the final result as well as the minimum precision required to represent the operands.  You qualify $a$, $b$, and $c$ as positive real numbers, but we also need to know how many binary digits $d_n$ are required to represent them accurately.  To understand the performance considerations for general real numbers, we first need to understand how computers represent integers as well as how it approximates real numbers using floating-point numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When computers operate on an integer $M$, then the number of binary digits needed is obviously equal to the log$_2$ of the magnitude of the integer, plus an extra bit for handling the sign:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$d_n=$log$_2|M|+1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the number -8 can be represented with 4 binary digits.  For performance and space-efficiency, arithmetic logic units (ALUs), responsible for numerical computations of integers on modern processing units, are designed to handle math on integers up to some fixed size, the most common these days being d=32 and d=64.  Not just x86 processors like in your computer have ALUs, they are a fundamental building block of computer architecture ubiquitous in today's electronic society.  If you are familiar with video game consoles, you might remember the Nintendo 64, a video game system named after the size (in bits), the arithmetic logic units on the console's processor were designed to handle.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integer additions, subtractions, and multiplications on arithmetic logic units are very efficient, and usually require no more than several cycles to compute.  Divisions are less performant, and on modern processors can require as many as several dozen cycles.  Performance depends on both the architecture of the processing unit (and corresponding implementation of the arithmetic logic unit) and its frequency.  Note that a 64-bit processor can usually perform arithmetic on $x$-bit operands at the same speed for $x$ anywhere between 1 and 64.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general computing, and especially in scientific computing, integer math is unwieldy for many computations, and another representation of numbers is needed, the so-called 'floating-point' representation.  Floating-point numbers represent a compromise between the way modern microprocessors work (carting data around in $n$-bit hunks) and the needs of computation by representing numbers on the processor in truncated scientific notation, using a fixed base $b$ (usually $b=2$ or $b=10$) and representing the number using two integers, a mantissa (significand in some circles) $s$, and an exponent $e$. A given number $x$ is then approximately represented as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x = s*b^e$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I say approximately because it should be obvious that even simple rationals such as $\frac{1}{3}$ cannot be represented exactly as a floating-point number for the standard bases.  The number of digits committed to the significand determine the accuracy of the number, which is relative to its own magnitude.  The &lt;a href=&quot;http://dx.doi.org/10.1109/IEEESTD.2008.4610935&quot; rel=&quot;nofollow&quot;&gt;IEEE 754 standard&lt;/a&gt; specifies a number of rules for how floating-point numbers are expected to behave, including the ranges of the significand and mantissa (and corresponding range and precision) for several important values of $d_n$, so that numerical computations are repeatable within some tolerance.&#xA;There is quite a bit of subtlety to how floating-point numbers work which I cannot hope to capture in this answer, for a good introduction I recommend &lt;a href=&quot;http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html&quot; rel=&quot;nofollow&quot;&gt;&quot;What Every Computer Scientist Should Know About Floating-Point Arithmetic&quot;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A significant amount of intellectual effort over the last 50 years has been invested in improving processor capability to compute arithmetic floating-point operations efficiently.  On modern processors, these computations are handled by one or more Floating-point units (FPUs), a more sophisticated version of the arithmetic logic unit designed to perform arithmetic operations on floating-point numbers and usually designed to handle both IEEE 754-specified 32-bit floating-point numbers (often referred to as 'floats') and 64-bit floating-point numbers (often referred to as 'doubles') efficiently.  Similar to arithmetic logic units, floating-point units can often compute addition, subtraction, and multiplication in just a few cycles, while division usually requires slightly more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases, IEEE 754 64-bit floating-point 'doubles' are sufficient for numerical computations, so let us assume that $a$, $b$, and $c$ are each represented as 64-bit doubles, and you are interested in the performance of the three computations as scalar operations on an Intel Nehalem architecture using the x87 floating-point instruction subset, i.e. you are not interested in calculating these operations in a for loop or over a range of data, and you don't want to use the vector extensions.  Instruction latency information is collected from &lt;a href=&quot;http://www.agner.org/optimize/instruction_tables.pdf&quot; rel=&quot;nofollow&quot;&gt;Agner Fog's excellent set of instruction reference tables&lt;/a&gt; for Intel/AMD architectures. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$a^b$&lt;/li&gt;&#xA;&lt;li&gt;log$_a c$&lt;/li&gt;&#xA;&lt;li&gt;$c^\frac{1}{b}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt; General exponentiation is often implemented with the following identity: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a^b = \beta^{a\cdot\text{log}_\beta b}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\beta$ is either $2$ or $e$ (in this case, I use $\beta=2$).  Assuming you are willing to throw away some accuracy in the result (the x87 unit does its computations in 80 bits of precision, but this is insufficient for certain ranges of values for $a$ and $b$), this computation can be done with the FYL2X hardware instruction to compute $t=a\cdot\text{log}_2 b$ and the F2XM1 hardware instruction (with some scaling help) to calculate $2^t$.  Assuming ~20 cycles for handling the scaling:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYL2X + F2XM1 + ~20 = 80 + 51 + ~20 = ~151 cycles &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.agner.org/optimize/instruction_tables.pdf&quot; rel=&quot;nofollow&quot;&gt;2&lt;/a&gt; This can be transformed to two logarithms and a division by the change of basis identity and does not need rescaling for an accurate result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2*FYL2X + FDIV = 2*80 + (7 to 27) = 167 to 187 cycles&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] This is equivalent to a division followed by an exponentiation, so [1] plus FDIV, ~175 cycles.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-01-13T14:38:04.347" LastActivityDate="2012-01-13T14:38:04.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="689" PostTypeId="2" ParentId="661" CreationDate="2012-01-13T14:18:28.063" Score="-2" Body="&lt;p&gt;As a reader, a statement in the paper to the effect that code can be obtained by contacting the author directly would be effective. As an author, this could help foster collaboration, and give me an opportunity to remind people to cite my article if they use the code in their work. &lt;/p&gt;&#xA;" OwnerUserId="585" LastActivityDate="2012-01-13T14:18:28.063" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="690" PostTypeId="2" ParentId="661" CreationDate="2012-01-13T15:49:14.030" Score="4" Body="&lt;p&gt;For small scripts that are specific to a specific research project, the best place for publication is the journal's Web site, as &quot;supplementary information&quot; to the paper. That's where it is easiest to find for someone who reads the article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More substantial packages that are of interest for other projects as well should better be published separately. Unfortunately there is no really good solution at this time. Ideally, a code publication would be permanently accessible through a DOI, just like a paper, but I am not aware of any hosting site that hands out DOIs and guarantees their permanence. Public repositories like Github or Bitbucket are perhaps the best bet for now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best solution would be to publish the paper packaged with the code and the data that go with it, but that is not yet technically feasible. I am working on a research prototype exploring this idea, see &lt;a href=&quot;http://dirac.cnrs-orleans.fr/plone/software/activepapers&quot; rel=&quot;nofollow&quot;&gt;this site&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2012-01-13T15:49:14.030" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="691" PostTypeId="2" ParentId="668" CreationDate="2012-01-13T15:51:51.223" Score="4" Body="&lt;p&gt;The key to making a finite difference scheme work on an irregular geometry is to have a 'shape' matrix with values that denote points outside, inside, and on the boundary of the domain.  Say we had a shape like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\begin{matrix}&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 2 &amp;amp; 2 &amp;amp; 2 &amp;amp; 2 &amp;amp; 2 &amp;amp; 2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 2 &amp;amp; 2 &amp;amp; 2 &amp;amp; 2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 2 &amp;amp; 2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;\end{matrix}&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The true domain (where all the non-zero entries of the matrix are) form a triangle pointed downward.  The 1's represent points on the boundary, while 2's represent interior points (unkowns, usually) We can assign node numbers as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\begin{matrix}&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; -1 &amp;amp; -1 &amp;amp; -1 &amp;amp; -1 &amp;amp; -1 &amp;amp; -1 &amp;amp; -1 &amp;amp; -1 &amp;amp; -1 &amp;amp; -1 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 1 &amp;amp; 2 &amp;amp; 3 &amp;amp; 4 &amp;amp; 5 &amp;amp; 6 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 7 &amp;amp; 8 &amp;amp; 9 &amp;amp; 10 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 11 &amp;amp; 12 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xD;&#xA;\end{matrix}&#xD;&#xA;$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, -1's represent the boundary locations.  Then, you can run a finite difference scheme over all entries in the matrix, but use an if statement to execute your scheme only on the interior nodes (from 1 to 12).  This approach is not the most efficient way to do it, but it will get the job done... if you can afford the memory, it might be good to store the (i,j) entries of all the interior nodes, and run a for loop only on those nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To create the geometry directly, you can do one of two things:&lt;br&gt;&#xA;1.  Create a black &amp;amp; white image manually, and import it to your program (easiest to implement, but impossible to refine your spatial resolution dx or dy).&lt;br&gt;&#xA;2.  Write code that will create discrete representations of the basic shapes that you want for any spatial resolution that you choose (harder to implement, but more robust for general finite difference schemes of any spatial resolution dx or dy).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to learn more about how to do this, you may want to consider watching these videos:&lt;br&gt;&#xA;&lt;a href=&quot;http://freevideolectures.com/Course/2664/Introduction-to-Computer-Graphics/2&quot; rel=&quot;nofollow&quot;&gt; NPTEL Computer Graphics Course, Video 2 (Raster Graphics)&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;http://freevideolectures.com/Course/2664/Introduction-to-Computer-Graphics/3&quot; rel=&quot;nofollow&quot;&gt; NPTEL Computer Graphics Course, Video 3 (Raster Graphics, continued)&lt;/a&gt;&lt;br&gt;&#xA;Check them out, and let me know if this addresses your question.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T14:23:51.890" LastActivityDate="2012-01-14T14:23:51.890" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="692" PostTypeId="2" ParentId="660" CreationDate="2012-01-13T16:12:46.410" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://www.computer.org/portal/web/cise&quot; rel=&quot;nofollow&quot;&gt;Computing in Science and Engineering&lt;/a&gt; is another option. It's a hybrid of a scientific journal (with refereed feature articles) and a magazine (with unrefereed regular columns).  It publishes articles that its readers are likely to find interesting. An article about software should thus contain some interesting material, e.g. about specific techniques used, that are useful to a wider audience than the software itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should perhaps add that I am a member of the editorial board of this magazine.&lt;/p&gt;&#xA;" OwnerUserId="140" LastEditorUserId="-1" LastEditDate="2012-01-17T13:05:09.493" LastActivityDate="2012-01-17T13:05:09.493" CommentCount="0" CommunityOwnedDate="2012-01-13T16:12:46.410" ContentLicense="CC BY-SA 3.0" />
  <row Id="693" PostTypeId="2" ParentId="661" CreationDate="2012-01-13T18:37:39.530" Score="3" Body="&lt;p&gt;I've taken two tactics, born of the fact that I anticipate changing institutions soon, so my university URL isn't stable in the slightest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the code is relatively short, I've tried including it as a supplemental appendix in the journal itself, under the assumption that they'll probably do a decent job keeping the paper and the code in roughly the same place. This is especially useful for code where there isn't a wide amount of general interest - code that's somewhat useless without the paper in question to provide context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for source-code, actual software, and more complicated projects or ones of general interest, I've been following your tactic of linking to a GitHub repository, which should at least be stable for the average productive lifespan of my papers.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2012-01-13T18:37:39.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="694" PostTypeId="2" ParentId="136" CreationDate="2012-01-13T19:05:32.320" Score="8" Body="&lt;p&gt;As an alternative source, you could also look at mesh-based Ewald-like methods. The genesis of the &quot;particle mesh&quot; methods (such as PPPM and smoothed particle mesh Ewald) lies in simulations of galaxies for astrophysics; the connection to charges was an unintentional side effect (that just happened to eventually overtake the original usage).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently, there has also been some literature on &lt;a href=&quot;http://www.gpucomputing.net/?q=node/6858&quot;&gt;multilevel summation methods&lt;/a&gt; which are akin in spirit to fast multipole methods and the Barnes-Hut, but may offer advantages in different circumstances (more general and flexible geometries, some efficiency gains, etc.).&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-13T19:05:32.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="696" PostTypeId="1" CreationDate="2012-01-13T19:56:03.180" Score="6" ViewCount="1591" Body="&lt;p&gt;I'm looking to use the finite element method with &lt;a href=&quot;http://en.wikipedia.org/wiki/B-splines&quot; rel=&quot;nofollow&quot;&gt;B-splines&lt;/a&gt; as my function basis. Which C/C++ libraries have good B-spline support?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I'm looking for an implementation of a stable algorithm, even if it's slow. I plan to precompute a lot of the inner products I need and store them in a file somewhere if the b-spline calculation gets slow enough to be bothersome.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-13T21:04:21.023" LastActivityDate="2013-09-10T14:44:47.180" Title="Which libraries have good implementations of Basis splines?" Tags="|finite-element|libraries|basis-set|c|c++|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="698" PostTypeId="1" AcceptedAnswerId="719" CreationDate="2012-01-13T23:28:54.673" Score="7" ViewCount="891" Body="&lt;p&gt;My question is related in spirit to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/114/is-algorithmic-analysis-by-flop-counting-obsolete&quot;&gt;&quot;Is algorithmic analysis by flop counting obsolete?&quot;&lt;/a&gt;.  Counting the number of computational operations in an algorithm is commonly used as a first-order model to aid in understanding the computational costs of an algorithm. However, the unit cost (in, say, time per operation) of data movement is greater than that of a processor cycle (see Peter Norvig's table of approximate timings for operations on a PC at &lt;a href=&quot;http://norvig.com/21-days.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;). Is there any type of analysis for communication similar to the analysis done for computational operations? If not, is coding up an algorithm and running it the only (or most effective way) to determine the costs of communication?&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-02-10T02:22:54.957" Title="When analyzing a parallel algorithm, how do you take communication costs into account?" Tags="|algorithms|performance|parallel-computing|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="699" PostTypeId="1" AcceptedAnswerId="701" CreationDate="2012-01-13T23:50:52.723" Score="19" ViewCount="1158" Body="&lt;p&gt;Reproducible research in computation aims to make the code needed to generate the results in a computational paper available to other researchers so that they can run this code to reproduce  the results in that paper. I'd like to make all of my research reproducible, but I'm running into a bit of a snag: a couple papers I am working on use an in-house automatic differentiation package (called &lt;a href=&quot;http://yoric.mit.edu/DAEPACK&quot;&gt;DAEPACK&lt;/a&gt;) on a proprietary library (&lt;a href=&quot;http://www.sandia.gov/chemkin/&quot;&gt;CHEMKIN-II&lt;/a&gt;; license terms unclear). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be overly time-consuming to replace these software components with open-source versions. An open-source replacement for CHEMKIN-II exists called &lt;a href=&quot;http://code.google.com/p/cantera/&quot;&gt;Cantera&lt;/a&gt;, but Cantera is in C++, whereas CHEMKIN-II is in Fortran 77. It would require a lot of effort to modify enough of the Cantera code so that it could be processed by automatic differentiation tools for C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that I need these proprietary packages, what is the best way to make my research as reproducible as possible, assuming that researchers may not have access to CHEMKIN-II? Since DAEPACK is a source-to-source translator, I don't necessarily need to distribute DAEPACK; I might be able to include its output, which would be Fortran source files that calculate derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, if you require proprietary software in your work, and that proprietary software isn't widely available (i.e, isn't MATLAB, Mathematica, etc.), how do you make your work reproducible?&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-04-06T17:53:15.120" LastActivityDate="2012-07-30T02:56:05.303" Title="What is the best way to do reproducible research if you need proprietary libraries?" Tags="|software|publications|documentation|reproducibility|" AnswerCount="6" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="700" PostTypeId="2" ParentId="698" CreationDate="2012-01-14T00:05:44.593" Score="4" Body="&lt;p&gt;This is a hard thing to do and there is nothing as general, well defined and powerful as complexity analysis that includes memory movement.  Memory complexity is at least a start.  Kung and others have done relevant work in this area (H. T. Kung. Memory requirements for balanced computer architectures. In Proceedings of the ACM Int’l. Symp. Computer Architecture (ISCA), Tokyo, Japan, 1986) and Jim Demmel's group at Berkeley has recently worked explicitly with complexity theory that include communication, but &quot;memory movement &quot; theory has far to go until it is as versatile as traditional complexity theory and analysis.&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2012-01-14T00:05:44.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="701" PostTypeId="2" ParentId="699" CreationDate="2012-01-14T00:12:12.790" Score="13" Body="&lt;p&gt;I don't think that the definition of &quot;reproducible research&quot; requires that the author provide for free all of the tools that are needed to reproduce the results obtained. If some of it is proprietary, then it is incumbent on the future user, not the author, to make the arrangements to acquire the needed software. (You wouldn't expect to have to build the software for somebody else to reproduce your results, right?) You would have the responsibility, of course, of specifying exactly which release of the proprietary software you used, but it is unreasonable to expect you to make a research code compatible with open-source software &lt;em&gt;just&lt;/em&gt; because it is open-source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[As an aside, in addition to talking about the reproducibility of software when some of it is proprietary, I think a more fundamental question needs to be addressed first: does the software give reproducible results across different installations (for instance, for different architectures and compiler versions? For instance, roundoff errors and differences in allocations as a result of differing numbers of processors means that many molecular simulation calculations will give different results just by changing the number of processors included in the allocation. (The statistical averages should reach the same place, but the trajectories may and probably will end up very different.)]&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-01-18T18:49:05.043" LastActivityDate="2012-01-18T18:49:05.043" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="702" PostTypeId="2" ParentId="669" CreationDate="2012-01-14T01:32:24.683" Score="3" Body="&lt;p&gt;The boundary of the domain has exactly two types of corners: convex 90 degree corners and concave 90 degree corners. For filling the domain with non-overlapping rectangles, only the concave corners are important. (I don't know how to upload pictures, sorry.) At each concave corner, we must have a &quot;cut&quot; in x- or y-direction. Each &quot;cut&quot; adds one rectangle, unless the &quot;cut&quot; happens to end in another concave corner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT Fixed the proposed solution based on the references given in the other answer.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, we define a bipartite graph with the axis-parallel &quot;cuts&quot; between concave corners as nodes. Two nodes are connected by an edge, if the corresponding &quot;cuts&quot; intersect each other, or have a common corner. A maximum independent set in this graph (a set of nodes with no edges between them) would translate into a solution of the original problem. Such a set can be derived from a maximum matching of the bipartite graph. (This is probably related to the Dulmage-Mendelsohn decomposition.)&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2012-01-15T23:49:32.647" LastActivityDate="2012-01-15T23:49:32.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="703" PostTypeId="2" ParentId="699" CreationDate="2012-01-14T02:46:56.043" Score="2" Body="&lt;p&gt;In your case, where you may not even have rights to distribute any of the code you used at all, a very in-depth description of the algorithm might be the best you can do.  Depending on how complex your code is, someone might be able to reproduce whatever is of interest to them from whatever tools they have available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, ask around and see how much of your work you're allowed to release, and then release that.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-01-14T02:46:56.043" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="704" PostTypeId="2" ParentId="698" CreationDate="2012-01-14T02:50:30.377" Score="2" Body="&lt;p&gt;According to the book &quot;Parallel Programming in C with MPI and OPENMP&quot; (2004) by M.J. Quinn, you can think of parallel programs in the &quot;task-channel&quot; model.  In it, the critical thing to understand is that &quot;a task may only send one message at a time, but it can receive a message at the same time it is sending a message&quot; (Quinn, 2004, pg 76).  So, you only need to consider how many messages are sent simultaneously at each iteration.  Also, communication is never instantaneous: there is always a latency period before even 1 bit is sent through a channel (think of it as though the processor needs to &quot;warm up&quot; before it can send a message).  We can usually assume that this is a constant, independent of the message length.  In general, we do not consider too many other &quot;hardware-dependent&quot; factors into the algorithm analysis such as bandwidth, network topology, or cache hit rate.  This is part of the reason why a parallel implementation does not perform as predicted by the algorithm analysis. It is also not uncommon for programmers to not know these parameters about the machine they are testing their code on, which makes experimental testing much more convenient.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-29T15:32:23.130" LastActivityDate="2012-01-29T15:32:23.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="705" PostTypeId="1" AcceptedAnswerId="10163" CreationDate="2012-01-14T03:07:57.867" Score="12" ViewCount="872" Body="&lt;p&gt;I have a Mathematica program which performs some integrals in 3 or 4 dimensions using the &lt;code&gt;QuasiMonteCarlo&lt;/code&gt; method. The problem is, it takes an annoyingly long time to run, to the point where some of these calculations can't complete in the maximum job time available on our HPC cluster. So I'm considering rewriting the program in C++, which I suspect will speed it up by a large factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked at the GSL docs and while there are sections on &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Quasi_002dRandom-Sequences.html&quot;&gt;quasirandom sequences&lt;/a&gt; and &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Monte-Carlo-Integration.html&quot;&gt;regular MC integration&lt;/a&gt;, I don't see anything that brings them together. Also a Google search or two didn't turn up anything that looked like a widely trusted implementation. What are my options for a well-tested implementation of QMC integration in C++?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the interest of consistency, I'd prefer to use something close to the Halton-Hammersley-Wozniakowski method &lt;a href=&quot;http://mathworld.wolfram.com/Quasi-MonteCarloIntegration.html&quot;&gt;that Mathematica implements&lt;/a&gt;, if that is an option.&lt;/p&gt;&#xA;" OwnerUserId="103" LastEditorUserId="127" LastEditDate="2012-01-14T04:46:42.717" LastActivityDate="2014-04-26T10:37:00.303" Title="Replacing Mathematica's QuasiMonteCarlo integration in C++" Tags="|libraries|monte-carlo|c++|mathematica|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="706" PostTypeId="2" ParentId="136" CreationDate="2012-01-14T03:22:05.737" Score="4" Body="&lt;p&gt;If you need a simple implementation approach which is not optimal in the asymptotic sense, you may want to consider using all-gather communication operations.  Since each of the N-bodies needs to know the gravitational effect of the other bodies, it is important for every processor to know the entire dataset.  This is what all-gather operations do.  There is a good book: Parallel Programming in C with MPI and OPENMP by Michael J. Quinn (2004) which discusses exactly this topic on page 82.  It might be worth looking at to give you a start.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-01-15T00:51:35.877" LastActivityDate="2012-01-15T00:51:35.877" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="707" PostTypeId="2" ParentId="696" CreationDate="2012-01-14T06:08:12.283" Score="5" Body="&lt;p&gt;The Gnu Scientific Library (GSL) has an implementation of B-splines, and the documentation can be found &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Basis-Splines.html&quot;&gt;here&lt;/a&gt;. According to the documentation, GSL uses De Boor's algorithm, which is numerically stable.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-14T06:38:26.463" LastActivityDate="2012-01-14T06:38:26.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="709" PostTypeId="2" ParentId="699" CreationDate="2012-01-14T13:27:50.100" Score="1" Body="&lt;p&gt;The best way to do reproducible research is not to rely on anything short lived like specific codes on specific platform. Yesterday we were computing on punched cards, today on electrons in silicon, tomorrow it may be on lasers with proteins or even living cells, the day after tomorrow on quantum dots, and who knows what will be next week.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reproducible research should make its concepts and ideas clear and not promote the currently available tools like hardware or software to a prerequisite to reproduce the research results. Algorithms used should be detailed in a clear universal form; current version of a specific hardware, compiler or software package can only be used as a tool to demonstrate the research results.&lt;/p&gt;&#xA;" OwnerUserId="474" LastActivityDate="2012-01-14T13:27:50.100" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="710" PostTypeId="5" CreationDate="2012-01-14T14:54:01.000" Score="0" Body="&lt;p&gt;Numerical algorithms often make a sequence of approximations that should converge to the &quot;correct solution&quot; in the limit sense.  However, due to issues such as finite precision or algorithmic implementation, small initial errors can be magnified quickly to produce an answer that does not satisfy our expected tolerance.  This is of particular interest to numerical solutions of differential and partial differential equations, but is not limited to these fields.  Subtraction of nearly equal quantities tends to be a common cause of numerical instability, but may not be the only source of instability.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-18T15:10:46.487" LastActivityDate="2012-01-18T15:10:46.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="711" PostTypeId="4" CreationDate="2012-01-14T14:54:01.000" Score="0" Body="The study of the propagation of errors in a numerical algorithm." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T16:14:30.887" LastActivityDate="2012-01-14T16:14:30.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="712" PostTypeId="5" CreationDate="2012-01-14T14:58:53.597" Score="0" Body="&lt;p&gt;C++ is a programming language created as an extension of C (&amp;quot;C with Classes&amp;quot;). C++ is widely used across many programming domains and is extensively to develop computational software&lt;/p&gt;&#xA;" OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-11T22:39:16.863" LastActivityDate="2021-11-11T22:39:16.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="713" PostTypeId="4" CreationDate="2012-01-14T14:58:53.597" Score="0" Body="For questions about writing C++ computational code or utilizing libraries written with C++." OwnerUserId="549" LastEditorUserId="23791" LastEditDate="2021-11-11T22:39:22.193" LastActivityDate="2021-11-11T22:39:22.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="714" PostTypeId="5" CreationDate="2012-01-14T15:02:28.537" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T15:02:28.537" LastActivityDate="2012-01-14T15:02:28.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="715" PostTypeId="4" CreationDate="2012-01-14T15:02:28.537" Score="0" Body="The generation of a set of interconnected geometric objects that approximate a desired domain or region in euclidean space." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T16:13:59.577" LastActivityDate="2012-01-14T16:13:59.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="716" PostTypeId="5" CreationDate="2012-01-14T15:03:36.617" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T15:03:36.617" LastActivityDate="2012-01-14T15:03:36.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="717" PostTypeId="4" CreationDate="2012-01-14T15:03:36.617" Score="0" Body="The study of the properties of fluids and gases in motion" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T16:13:46.303" LastActivityDate="2012-01-14T16:13:46.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="718" PostTypeId="1" AcceptedAnswerId="767" CreationDate="2012-01-14T15:06:22.800" Score="7" ViewCount="2073" Body="&lt;p&gt;Does anybody know of an implementation of &lt;a href=&quot;http://www.springerlink.com/content/vv750g6576n9p820/&quot;&gt;Nédélec elements&lt;/a&gt; that does not come along with a huge bulk of additional software?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a small library written in a language like Python, Matlab, or Octave? The only implementations I know of are part of larger FEM libraries which are too difficult to learn for the sole purpose of comparing with my own implementation. In the literature, there is virtually &lt;em&gt;nothing&lt;/em&gt; on how these elements look when implemented, not to speak of example code.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="348" LastEditDate="2012-01-14T22:31:19.567" LastActivityDate="2012-01-15T01:33:03.220" Title="Reference implementation of Nédélec-Elements" Tags="|finite-element|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="719" PostTypeId="2" ParentId="698" CreationDate="2012-01-14T15:07:51.657" Score="7" Body="&lt;p&gt;It's fairly common to construct performance models of parallel codes that include FLOPS, memory bandwidth, and communications.   A real world example of modelling performance of such a code is &lt;a href=&quot;http://www.ccs3.lanl.gov/pal/publications/papers/kerbyson05:POP.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, for the POP (v2) ocean code; it's just the first that came up in a google search.   I've given a simple example of how it works in my answer to this question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/815/how-can-i-reduce-the-communication-bottleneck-of-a-parallel-explicit-finite-diff&quot;&gt;How can I reduce the communication bottleneck of a parallel explicit finite difference scheme?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The communication part of these performance models generally count the number and size of messages, and use latency and bandwidth to estimate the time taken by point-to-point messages, with some logarithmic factor for collectives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with any modeling exercise, you can go nuts adding more terms to your model -- making communications on-node less expensive, and off-switch more expensive, including load imbalance, etc -- but even a fairly simple model gives you a fairly strong understanding of what is going on in your code, and allows you to quantitatively answer questions like &quot;is infiniband vs. ethernet likely to make a difference for this simulation&quot;, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to your last question, &quot;If not, is coding up an algorithm and running it the only (or most effective way) to determine the costs of communication?&quot; -- experiment always trumps theory, of course, but theory can be a useful guide.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastEditorUserId="-1" LastEditorDisplayName="user389" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-16T15:22:46.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="720" PostTypeId="2" ParentId="660" CreationDate="2012-01-14T15:54:44.863" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://www.geoscientific-model-development.net/submission/manuscript_types.html&quot; rel=&quot;nofollow&quot;&gt;Geoscientific Model Development&lt;/a&gt; is an open access journal for this purpose. Like other journals of the European Geophysical Union, it has an &lt;a href=&quot;http://www.geosci-model-dev-discuss.net/papers_in_open_discussion.html&quot; rel=&quot;nofollow&quot;&gt;open review process&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-14T15:54:44.863" CommentCount="0" CommunityOwnedDate="2012-01-14T15:54:44.863" ContentLicense="CC BY-SA 3.0" />
  <row Id="721" PostTypeId="5" CreationDate="2012-01-14T15:57:12.240" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T15:57:12.240" LastActivityDate="2012-01-14T15:57:12.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="722" PostTypeId="4" CreationDate="2012-01-14T15:57:12.240" Score="0" Body="Trilinos is a set of open source, parallel numerical libraries developed by Sandia National Laboratories." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T16:14:35.957" LastActivityDate="2012-01-14T16:14:35.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="723" PostTypeId="2" ParentId="674" CreationDate="2012-01-14T16:05:18.467" Score="7" Body="&lt;p&gt;The fast multipole method (FMM) is $O(n)$ and has much lower communication requirements, so it provides a highly scalable discrete Fourier transform. &lt;a href=&quot;http://dx.doi.org/10.1137/S1064827597316266&quot;&gt;Edelman, McCorquodale, and Toledo (1999) &lt;em&gt;The Future Fast Fourier Transform?&lt;/em&gt;&lt;/a&gt; analyze this approach and conclude that FMM would be preferable to conventional FFT at large scale. Note that the FMM is only approximate, so the constants are worse if you need very high accuracy. Thanks to Jack Poulson for pointing this out in a discussion last week.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-14T16:05:18.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="724" PostTypeId="5" CreationDate="2012-01-14T17:37:27.410" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T17:37:27.410" LastActivityDate="2012-01-14T17:37:27.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="725" PostTypeId="4" CreationDate="2012-01-14T17:37:27.410" Score="0" Body="An approach to solving systems of equations by projecting the problem from a fine scale representation onto a coarser one.  A coarse representation generally has fewer unknowns, making it faster to solve than the original problem.  The coarse solution can then be projected back onto the finer problem as an initial guess of the solution to the finer problem." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T18:52:24.783" LastActivityDate="2012-01-14T18:52:24.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="726" PostTypeId="5" CreationDate="2012-01-14T18:00:51.210" Score="0" Body="&lt;p&gt;Domain decomposition methods are employed in the solution of partial differential equations, frequently on parallel computers, by subdividing the problem domain and solving each piece individually.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="9" LastEditDate="2012-01-18T07:15:11.907" LastActivityDate="2012-01-18T07:15:11.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="727" PostTypeId="4" CreationDate="2012-01-14T18:00:51.210" Score="0" Body="A parallel algorithm design approach in which the data is divided into pieces and then computations are associated with the data.  This contrasts to 'functional decomposition', in which tasks or computations are divided first, then data is associated to them.  " OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-30T15:24:41.943" LastActivityDate="2012-01-30T15:24:41.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="728" PostTypeId="5" CreationDate="2012-01-14T18:09:07.513" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T18:09:07.513" LastActivityDate="2012-01-14T18:09:07.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="729" PostTypeId="4" CreationDate="2012-01-14T18:09:07.513" Score="0" Body="The study of collection, organization, analysis, and interpretation of data." OwnerUserId="549" LastEditorUserId="9" LastEditDate="2012-01-18T07:20:33.583" LastActivityDate="2012-01-18T07:20:33.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="730" PostTypeId="5" CreationDate="2012-01-14T18:12:59.037" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T18:12:59.037" LastActivityDate="2012-01-14T18:12:59.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="731" PostTypeId="4" CreationDate="2012-01-14T18:12:59.037" Score="0" Body="A method of representing numbers by a fixed number of significant digits, and the exponent of some base number.  They are characterized in the form  &#xD;&#xA;${(significant digits)}*base^{exponent}$.  &#xD;&#xA;Typically, numbers are represented with respect to base = 2 (binary)." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T18:51:27.267" LastActivityDate="2012-01-14T18:51:27.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="732" PostTypeId="5" CreationDate="2012-01-14T18:17:16.753" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T18:17:16.753" LastActivityDate="2012-01-14T18:17:16.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="733" PostTypeId="4" CreationDate="2012-01-14T18:17:16.753" Score="0" Body="A powerful mathematical programming software with high level symbolic manipulation capacity." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T18:52:01.057" LastActivityDate="2012-01-14T18:52:01.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="734" PostTypeId="5" CreationDate="2012-01-14T18:21:03.393" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T18:21:03.393" LastActivityDate="2012-01-14T18:21:03.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="735" PostTypeId="4" CreationDate="2012-01-14T18:21:03.393" Score="0" Body="A method of finding nearly-optimal solutions to a problem.  Generally, this terminology is applied to algorithms and heuristics for solving NP-Hard problems in computer science." OwnerUserId="549" LastEditorUserId="9" LastEditDate="2012-01-15T09:29:21.330" LastActivityDate="2012-01-15T09:29:21.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="736" PostTypeId="5" CreationDate="2012-01-14T18:22:48.647" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T18:22:48.647" LastActivityDate="2012-01-14T18:22:48.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="737" PostTypeId="4" CreationDate="2012-01-14T18:22:48.647" Score="0" Body="The study of solving computational problems by exploiting the use of multiple processors simultaneously." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-14T18:52:35.157" LastActivityDate="2012-01-14T18:52:35.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="738" PostTypeId="5" CreationDate="2012-01-14T18:25:06.880" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T18:25:06.880" LastActivityDate="2012-01-14T18:25:06.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="739" PostTypeId="4" CreationDate="2012-01-14T18:25:06.880" Score="0" Body="The process of representing a continuum space with a finite set of points/elements" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T09:32:49.593" LastActivityDate="2012-01-15T09:32:49.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="740" PostTypeId="5" CreationDate="2012-01-14T18:33:35.383" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://mathworks.com&quot; rel=&quot;nofollow&quot;&gt;MATLAB&lt;/a&gt; (MATrix LABoratory) is a commercial software (MathWorks; originally developed at the University of New Mexico) that offers a high-level programming language with particular emphasis on vector and matrix based operations and plotting capabilities in an interactive development environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions tagged &lt;a href=&quot;/questions/tagged/matlab&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;matlab&amp;#39;&quot; rel=&quot;tag&quot;&gt;matlab&lt;/a&gt; should be concerned with the mathematical capabilities of Matlab (e.g., solving linear systems, using &lt;code&gt;eigs&lt;/code&gt; with anonymous functions, choosing functions or toolboxes for a certain scientific problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions about the programming language of MATLAB (e.g., vectorizing loops, manipulating arrays) belong to &lt;a href=&quot;http://stackoverflow.com&quot;&gt;Stack Overflow&lt;/a&gt;; questions about the development environment (including manipulating plots) should be posted at &lt;a href=&quot;http://www.mathworks.com/matlabcentral/answers/&quot; rel=&quot;nofollow&quot;&gt;MATLAB Central&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="625" LastEditDate="2015-05-25T15:23:54.840" LastActivityDate="2015-05-25T15:23:54.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="741" PostTypeId="4" CreationDate="2012-01-14T18:33:35.383" Score="0" Body="Questions about the numerical programming language MATLAB." OwnerUserId="549" LastEditorUserId="1804" LastEditDate="2013-06-26T14:14:06.080" LastActivityDate="2013-06-26T14:14:06.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="742" PostTypeId="5" CreationDate="2012-01-14T21:28:18.940" Score="0" Body="&lt;p&gt;Assume a system of equations of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=b$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is a given positive definite matrix, $b$ is the given right-hand side, and $x$ is unknown. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Conjugate Gradient method exploits the idea that we can write the solution x as a linear combination of mutually $A$-conjugate basis vectors.  This allows us to exploit a scheme similar to gradient descent, but with $A$-conjugate vectors instead of residual vectors.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-01-15T09:34:55.010" LastActivityDate="2012-01-15T09:34:55.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="743" PostTypeId="4" CreationDate="2012-01-14T21:28:18.940" Score="0" Body="A popular krylov subspace method for solving linear systems of equations, particularly those that exhibit symmetric positive definiteness." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T05:06:01.497" LastActivityDate="2012-01-15T05:06:01.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="744" PostTypeId="5" CreationDate="2012-01-14T21:31:28.653" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T21:31:28.653" LastActivityDate="2012-01-14T21:31:28.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="745" PostTypeId="4" CreationDate="2012-01-14T21:31:28.653" Score="0" Body="In linear algebra, an $m$x$n$ matrix maps vectors from space $R^m$ to $R^n$.  &#xD;&#xA;&#xD;&#xA;Matrices are distinguished from multidimensional arrays, which is a general term for any rectangular array of information." OwnerUserId="549" LastEditorUserId="9" LastEditDate="2012-01-15T09:36:08.103" LastActivityDate="2012-01-15T09:36:08.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="746" PostTypeId="5" CreationDate="2012-01-14T21:35:35.027" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T21:35:35.027" LastActivityDate="2012-01-14T21:35:35.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="747" PostTypeId="4" CreationDate="2012-01-14T21:35:35.027" Score="0" Body="Compiling is the process of transforming source code written by a programmer into an object code, generally into a binary form which is understandable to a machine." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T05:05:09.720" LastActivityDate="2012-01-15T05:05:09.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="748" PostTypeId="5" CreationDate="2012-01-14T21:37:49.113" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T21:37:49.113" LastActivityDate="2012-01-14T21:37:49.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="749" PostTypeId="4" CreationDate="2012-01-14T21:37:49.113" Score="0" Body="The process of Inputing and Outputting information, particularly emphasis on user generated input/output at runtime." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T09:33:27.220" LastActivityDate="2012-01-15T09:33:27.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="750" PostTypeId="5" CreationDate="2012-01-14T21:42:33.817" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T21:42:33.817" LastActivityDate="2012-01-14T21:42:33.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="751" PostTypeId="4" CreationDate="2012-01-14T21:42:33.817" Score="0" Body="Order of Accuracy, in the context of numerical solutions to partial differential equations, refers to the rate at which the numerical solution would converge to the true solution as the spatial resolution (dx, dy, dz, etc..) decreases.  The error E(h) as a function of a stepsize h is generally given in the form:  &#xD;&#xA;&#xD;&#xA;$E(h)=C{h}^{p}$,  &#xD;&#xA;&#xD;&#xA;where C is a constant, and p is the order of accuracy." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T09:36:39.600" LastActivityDate="2012-01-15T09:36:39.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="752" PostTypeId="5" CreationDate="2012-01-14T21:48:15.240" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T21:48:15.240" LastActivityDate="2012-01-14T21:48:15.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="753" PostTypeId="4" CreationDate="2012-01-14T21:48:15.240" Score="0" Body="Exascale refers to the ability to realize ${10}^{18}$ floating point operations per second. While this threshold has not yet been reached, many researchers are currently working to develop ways to create exaflop machines, and potential programming paradigms to use them efficiently. " OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T09:31:59.910" LastActivityDate="2012-01-15T09:31:59.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="754" PostTypeId="5" CreationDate="2012-01-14T21:51:29.293" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T21:51:29.293" LastActivityDate="2012-01-14T21:51:29.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="755" PostTypeId="4" CreationDate="2012-01-14T21:51:29.293" Score="0" Body="Referring to Krylov Subspaces and the methods of solutions to linear systems of equations which exploit these spaces." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T09:33:30.230" LastActivityDate="2012-01-15T09:33:30.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="756" PostTypeId="5" CreationDate="2012-01-14T21:55:28.287" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T21:55:28.287" LastActivityDate="2012-01-14T21:55:28.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="757" PostTypeId="4" CreationDate="2012-01-14T21:55:28.287" Score="0" Body="ATLAS is the acronym for Automatically Tuned Linear Algebra Software, a library of linear algebra subroutines with the capacity to automatically optimize serial performance specifically to the architecture on which the software is installed." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T05:05:03.667" LastActivityDate="2012-01-15T05:05:03.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="758" PostTypeId="5" CreationDate="2012-01-14T21:57:17.567" Score="0" Body="&lt;p&gt;The finite difference method is a numerical method for solving differential equations. The derivatives are discretized in space and in time and approximated by &quot;finite differences&quot; (derived from Taylor's Theorem) e.g.:&#xA;$$&#xA;f^{'}(x) = \frac{f(x+h) - f(x)}{h}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where h is the distance between discrete grid locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite difference method can be solved using either explicit or implicit methods. For a more complete description see the &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_method&quot; rel=&quot;nofollow&quot;&gt;Wikipedia page on the Finite difference method&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16952" LastEditorUserId="16952" LastEditDate="2015-07-19T18:24:05.130" LastActivityDate="2015-07-19T18:24:05.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="759" PostTypeId="4" CreationDate="2012-01-14T21:57:17.567" Score="0" Body="Referring to the discretization of derivatives by Finite differences, and its applications to numerical solutions of partial differential equations." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T09:33:21.480" LastActivityDate="2012-01-15T09:33:21.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="760" PostTypeId="5" CreationDate="2012-01-14T21:58:36.223" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T21:58:36.223" LastActivityDate="2012-01-14T21:58:36.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="761" PostTypeId="4" CreationDate="2012-01-14T21:58:36.223" Score="0" Body="Referring to methods of solving ordinary and partial differential equations by solving a system of linear equations at each time step." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T09:32:13.420" LastActivityDate="2012-01-15T09:32:13.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="762" PostTypeId="5" CreationDate="2012-01-14T22:04:32.593" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T22:04:32.593" LastActivityDate="2012-01-14T22:04:32.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="763" PostTypeId="4" CreationDate="2012-01-14T22:04:32.593" Score="0" Body="Courant-Fredrich-Lewy Condition. For explicit finite difference methods, it is a condition that must be satisfied for the method to be numerically stable.  It places a constraint on the time step size ($\Delta{t}$) which will depend upon the spatial stepsize(s) ($\Delta{x}$,$\Delta{y}$,$\Delta{z}$,etc)." OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-01-15T00:59:54.180" LastActivityDate="2012-01-15T00:59:54.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="764" PostTypeId="5" CreationDate="2012-01-14T22:06:17.130" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-14T22:06:17.130" LastActivityDate="2012-01-14T22:06:17.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="765" PostTypeId="4" CreationDate="2012-01-14T22:06:17.130" Score="0" Body="Referring to optimization problems that consist only of linear constraints and a linear objective function." OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-02-07T02:34:12.193" LastActivityDate="2012-02-07T02:34:12.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="766" PostTypeId="2" ParentId="718" CreationDate="2012-01-14T23:08:13.440" Score="3" Body="&lt;p&gt;It's not a minimal complete finite element library, but &lt;a href=&quot;https://launchpad.net/fiat&quot; rel=&quot;nofollow&quot;&gt;FIAT&lt;/a&gt; tabulates Nedelec spaces (and many other spaces) and &lt;a href=&quot;https://launchpad.net/dolfin&quot; rel=&quot;nofollow&quot;&gt;DOLFIN&lt;/a&gt; has simple examples for testing. The new &lt;a href=&quot;http://fenicsproject.org/book/index.html&quot; rel=&quot;nofollow&quot;&gt;FEniCS Book&lt;/a&gt; offers a good explanation of design and implementation. Unfortunately, there is a somewhat opaque &quot;form compiler&quot; so it would take more effort to fully understand every step of the process. On the other hand, the output of the form compiler performs a task that is not difficult to understand.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-14T23:08:13.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="767" PostTypeId="2" ParentId="718" CreationDate="2012-01-15T01:33:03.220" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://math.uci.edu/~chenlong&quot; rel=&quot;nofollow&quot;&gt;Long Chen&lt;/a&gt; has built a very compact library called &lt;a href=&quot;http://ifem.wordpress.com&quot; rel=&quot;nofollow&quot;&gt;$i$FEM&lt;/a&gt;, it has a complete implementation of the first and second type of Nédélec elements up to quadratic order, including a very readable data structure for geometry and DoFs, the stiffness matrix assembling(the implementation is similar to the idea shown in &lt;a href=&quot;http://www.mathematik.hu-berlin.de/~cc/download/public/publications/paper/CMAMv5p333-361.pdf&quot; rel=&quot;nofollow&quot;&gt;Carstensen's paper&lt;/a&gt;), an adaptive mesh refining and coarsening procedure, also the multigrid solver based on &lt;a href=&quot;http://www.jstor.org/stable/10.2307/2587241&quot; rel=&quot;nofollow&quot;&gt;Hiptmair's multigrid for Maxwell paper&lt;/a&gt;, together with HX-preconditioning technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For even higher order Nédélec elements which are used in $hp$-FEM, I suggest you refer to the book written by Pavel Solin: &lt;a href=&quot;http://hpfem.org/~pavel/public/books.html&quot; rel=&quot;nofollow&quot;&gt;Higher-Order Finite Element methods&lt;/a&gt;, in chapter 2 it has an explicit representation of the higher order vector elements using the edge vector and the face normal vector, not just for $H(\mathbf{curl})$, also for $H(\mathrm{div})$, and the recursively defined shape functions fall into Long Chen's data structure and assembling subroutine quite neatly. &lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2012-01-15T01:33:03.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="768" PostTypeId="1" CreationDate="2012-01-15T02:01:34.937" Score="11" ViewCount="887" Body="&lt;p&gt;What numerical analysis situations become more/less stable, have faster/slower convergence, or are otherwise quite different when dealing with functions of complex variable instead of functions of a real variable? &lt;/p&gt;&#xA;" OwnerUserId="608" LastEditorUserId="276" LastEditDate="2012-01-15T12:48:37.853" LastActivityDate="2012-01-15T12:48:37.853" Title="Complex numerical analysis" Tags="|algorithms|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="769" PostTypeId="1" AcceptedAnswerId="784" CreationDate="2012-01-15T02:44:38.757" Score="3" ViewCount="500" Body="&lt;p&gt;I have heard that one of the current goals in scientific computing is to surpass the exascale level by 2018 or so... After having passed the petascale a few years ago, one would think that the exascale wouldn't be so much of a challenge... If it's theoretically possible, what challenges are there to computing at this speed?  Have we already reached the limitations of moore's law?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-04-17T03:31:48.323" Title="What are the current obstacles to reaching exascale computing?" Tags="|hpc|parallel-computing|exascale|architecture|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="770" PostTypeId="1" AcceptedAnswerId="856" CreationDate="2012-01-15T03:07:28.027" Score="6" ViewCount="396" Body="&lt;p&gt;I know that delaunay triangulation can be extended into arbitrary dimensions by solving the convex hull problem in $(p+1)$ dimensions and projecting the lower hull into dimension $p$ to obtain a mesh in dimension $p$.  However, this approach is asymptotically $\Omega({n}^{{p}/{2}})$.  I was wondering if there are any other algorithms to produce arbitrary dimensional meshes that can be implemented faster. If so, what are they, how do they work and what is their asymptotic runtime (in terms of the number of mesh points)?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T16:34:49.463" LastActivityDate="2012-01-17T21:48:46.497" Title="What efficient algorithms are there to generate arbitrary dimensional meshes of simplices?" Tags="|algorithms|mesh-generation|computational-geometry|delaunay-triangulation|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="771" PostTypeId="1" AcceptedAnswerId="857" CreationDate="2012-01-15T03:13:33.883" Score="10" ViewCount="7297" Body="&lt;p&gt;I have always been told that the Voronoi diagram is the dual of the Delaunay triangulation problem.  In what sense can they be duals of each other?  I thought that dual problems (i.e. in linear programming) are supposed to produce the same answer. Clearly, the two problems do not have the same solution. How can we consider them duals?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-03-30T03:49:11.897" LastActivityDate="2012-03-30T03:49:11.897" Title="How are the Voronoi Tesselation and Delaunay triangulation problems duals of each other?" Tags="|computational-geometry|delaunay-triangulation|voronoi-diagrams|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="772" PostTypeId="1" AcceptedAnswerId="779" CreationDate="2012-01-15T03:26:37.207" Score="16" ViewCount="32615" Body="&lt;p&gt;As I understand it, since a solution to a linear program always occurs at a vertex of its polyhedral feasible set (if a solution exists and the optimal objective function value is bounded from below, assuming a minimization problem), how can a search through the interior of the feasible region be better?  Does it converge faster?  Under what circumstances would it be advantageous to use simplex method over interior point methods?  Is one easier to implement in a code than the other?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="127" LastEditDate="2012-01-15T05:15:57.670" LastActivityDate="2013-04-10T00:56:53.773" Title="What are the advantages/disadvantages of interior point methods over simplex method for linear optimization?" Tags="|convex-optimization|linear-programming|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="773" PostTypeId="1" AcceptedAnswerId="783" CreationDate="2012-01-15T03:31:10.260" Score="9" ViewCount="1244" Body="&lt;p&gt;I have heard that some journals are rated more highly than others.  Is this true? And if so, what are the criteria for judging the value of one peer reviewed journal over another?  How do I find out its rating?  Will my publication be of less &quot;worth&quot; if it is accepted in a less reputable journal than, say, the SIAM Review?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-16T14:14:19.333" LastActivityDate="2012-01-16T14:14:19.333" Title="Is there a standard rating system for scientific journal publications?" Tags="|publications|" AnswerCount="7" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="775" PostTypeId="5" CreationDate="2012-01-15T03:53:50.777" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T03:53:50.777" LastActivityDate="2012-01-15T03:53:50.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="776" PostTypeId="4" CreationDate="2012-01-15T03:53:50.777" Score="0" Body="The study of efficient algorithms and data structures to solve various problems involving point sets, line segments, polygons, polyhedra, simplices, etc." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T05:05:27.473" LastActivityDate="2012-01-15T05:05:27.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="777" PostTypeId="5" CreationDate="2012-01-15T04:02:54.913" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T04:02:54.913" LastActivityDate="2012-01-15T04:02:54.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="778" PostTypeId="4" CreationDate="2012-01-15T04:02:54.913" Score="0" Body="The process of generating a subdivision of ${R}^{2}$ consisting of conforming triangles from a given point set.  The delaunay triangulation has the special property that no 4 points lie in the circumcircle of any given triangle.  The concept extends to ${R}^{3}$ (sometimes referred to as a tetrahedralization), and ${R}^{d}$ (as mesh of simplices)." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T09:32:45.100" LastActivityDate="2012-01-15T09:32:45.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="779" PostTypeId="2" ParentId="772" CreationDate="2012-01-15T04:56:32.600" Score="22" Body="&lt;p&gt;Based on personal experience, I'd say that simplex methods are marginally easier to understand how to implement than interior point methods, based on personal experience from implementing both primal simplex and a basic interior point method in MATLAB as part of taking a linear programming class. The main obstacles in primal simplex are making sure that you implement Phase I and Phase II correctly, and also that you implement an anticycling rule correctly. The main obstacles in implementing an interior point method for linear programming tend to be more about implementing the iterative method correctly, and scaling the barrier parameter accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a more complete discussion of the pros and cons of each algorithm in a textbook on linear programming, such as &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1886529191&quot;&gt;Introduction to Linear Optimization&lt;/a&gt; by Bertsimas and Tsitsiklis. (&lt;em&gt;Disclaimer:&lt;/em&gt; I learned linear programming from this textbook, and took linear programming at MIT from Bertsimas' wife.) Here are some of the basics:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pros of simplex:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given $n$ decision variables, usually converges in $O(n)$ operations with $O(n)$ pivots.&lt;/li&gt;&#xA;&lt;li&gt;Takes advantage of geometry of problem: visits vertices of feasible set and checks each visited vertex for optimality. (In primal simplex, the reduced cost can be used for this check.)&lt;/li&gt;&#xA;&lt;li&gt;Good for small problems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons of simplex:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given $n$ decision variables, you can always find a problem instance where the algorithm requires $O(2^{n})$ operations and pivots to arrive at a solution.&lt;/li&gt;&#xA;&lt;li&gt;Not so great for large problems, because pivoting operations become expensive. Cutting-plane algorithms or delayed column generation algorithms like Dantzig-Wolfe can sometimes compensate for this shortcoming.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Pros of interior point methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Have polynomial time asymptotic complexity of $O(n^{3.5}L^2\log{L}\log\log{L})$, where $L$ is the number of bits of input to the algorithm.&lt;/li&gt;&#xA;&lt;li&gt;Better for large, sparse problems because the linear algebra required for the algorithm is faster.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons of interior point methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It's not as intuitively satisfying because you're right, these methods don't visit vertices. They wander through the interior region, converging on a solution when successful.&lt;/li&gt;&#xA;&lt;li&gt;For small problems, simplex will probably be faster. (You can construct pathological cases, like the Klee-Minty cube.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="9" LastEditDate="2012-01-15T09:51:56.550" LastActivityDate="2012-01-15T09:51:56.550" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="780" PostTypeId="2" ParentId="772" CreationDate="2012-01-15T05:15:35.317" Score="3" Body="&lt;p&gt;The answer is easy. They both (simplex and interior point methods) are a mature field from an algorithmic point of view. They both work very well in practice. The good reputation of I.P.M. (interior point methods) is due to its polynomial complexity in the worst case. That is not the case for simplex which has combinatorial complexity. Nevertheless, combinatorial linear programs almost never happen in practice. For very large scale problems, I.P. seems to be a bit faster, but is not necessary the rule. In my opinion I.P. can be easy to understand and implement, but for sure, someone else can disagree, and that is fine. Now, in L.P, if the solution is unique, it is definitely be in a vertex, (both I.P. and Simplex do well here as well). The solution also can be on a face of the polyhedron or on an edge in which case, the adjacent vertex is (or vertices are) also a solution (again, both I.P. and simplex do well). So they are pretty much the same.&lt;/p&gt;&#xA;" OwnerUserId="611" LastEditorUserId="276" LastEditDate="2012-01-15T05:40:35.530" LastActivityDate="2012-01-15T05:40:35.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="781" PostTypeId="2" ParentId="773" CreationDate="2012-01-15T05:31:47.273" Score="5" Body="&lt;p&gt;It really depends on how much you believe in journal ratings. The most popular metric I am aware of is the impact factor. &lt;a href=&quot;http://en.wikipedia.org/wiki/Impact_factor&quot;&gt;This&lt;/a&gt; Wikipedia article describes the impact factor, how it is calculated, and how it is susceptible to manipulation, as well as alternative journal rating metrics. The most important thing worth noting is that impact factors for journals in different fields aren't necessarily comparable, even though the impact factor rating system tries to collapse the &quot;quality&quot; of each journal onto a single metric. (For an interesting discussion on why such an idea can lead to misleading or easily manipulable results, see Malcolm Gladwell's New Yorker essay, &lt;a href=&quot;http://www.newyorker.com/reporting/2011/02/14/110214fa_fact_gladwell&quot;&gt;The Order of Things&lt;/a&gt;, which discusses college rankings and car rankings.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The impact factor is the average number of citations received per paper published in that journal during the two preceding years, so the criteria for judging the value of one peer reviewed journal over another is how many times its recent articles get cited on an average, per article basis. This metric does mean that journals like &lt;em&gt;Science&lt;/em&gt; and &lt;em&gt;Nature&lt;/em&gt; are ranked highly, as one would expect, but it also means that &lt;em&gt;International Journal of Nonlinear Sciences and Simulation&lt;/em&gt; (3.100) gets ranked higher than &lt;em&gt;SIAM Journal of Scientific Computing&lt;/em&gt; (3.016), even though the latter journal is a lot more recognizable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your publication being of less &quot;worth,&quot; it depends on the person. I've been told by my more senior colleagues that having articles in more prestigious journals is helpful for things like tenure and hiring decisions; not having experience with such matters firsthand, I defer to the comments of people who are have more background with those sorts of situations. It depends on the community of people you are trying to reach or impress. If you publish something in a math journal, don't expect that people outside mathematics will necessarily read it. (The same advice applies to other subjects.) Also, you may decide to publish in a less prestigious journal because you want to get a result out quickly rather than wait a long time for it to be reviewed by more prestigious journal. I've been told that the SIAM journals can take a year or more to review a paper and publish it, which is backed up by looking at the time between the submitted date and the published date on the paper. Submission to a particular journal is also dependent on aims and scope, politics (who is on the editorial board, who is likely to review your paper), what community you want to reach out to (sort of goes with the advice about subject area above). I still consult other people on these matters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I look at articles, the &quot;prestige&quot; of a journal, to me, is really my internal feeling that reflects the valued opinions of other people I know in science (my advisers, colleagues, etc.). I'm more inclined to trust an article in a journal that is more prestigious because I feel it is more likely to be reliable, but there are certainly bad articles in prestigious journals, so prestige is not a replacement for doing your own due diligence. For example, there are some pretty serious typos in an otherwise good journal article I've cited from &lt;em&gt;SIAM Review&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, that's my rambling opinion, and as a (nearly) freshly-minted PhD, I'm sure that others with more experience will have much more perspective and insight to add.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-15T05:31:47.273" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="782" PostTypeId="2" ParentId="773" CreationDate="2012-01-15T07:23:27.043" Score="8" Body="&lt;p&gt;In an ideal world, you would only consider one thing when choosing where to submit a manuscript: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;which journal's readership includes the broadest audience that would&#xA;  be interested in this work?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In practice, you also consider other factors, perhaps most significantly&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;How much will publishing in this journal help my career?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Anyone who can understand your work well will &lt;strong&gt;judge it by its actual significance&lt;/strong&gt;, but those who are too far removed may &lt;strong&gt;use the quality of the journal as a proxy&lt;/strong&gt; for the quality of your paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is journal quality determined?  Experienced professionals have their own opinions based on experiences with each journal and the articles they read.  In some fields, like mathematics, this expert opinion can only be synthesized by talking to a lot of experts.  In other fields, like computer science, this has been explicitly codified into a &lt;a href=&quot;http://www.cs.wm.edu/~srgian/tier-jnl-final2008.html&quot;&gt;tiered ranking system&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most widely available metric for journal quality is the so-called &lt;a href=&quot;http://en.wikipedia.org/wiki/Impact_factor&quot;&gt;impact factor&lt;/a&gt;.  The danger of having a single, widely used metric for anything is that it may be &lt;a href=&quot;http://en.wikipedia.org/wiki/Impact_factor#Manipulations_of_the_impact_factor&quot;&gt;manipulated&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Impact_factor#Incorrect_application_of_impact_factor&quot;&gt;misapplied&lt;/a&gt;, or may simply not be useful for some of the things it is applied to.  For instance, since it only measures citations over two years, it doesn't make much sense for mathematics papers, which typically get most of their citations after two years -- often much later.  &lt;a href=&quot;http://arxiv.org/abs/1010.0278&quot;&gt;It also seems to be poorly correlated with expert opinion.&lt;/a&gt;  There are alternatives, for instance &lt;a href=&quot;http://www.eigenfactor.org/&quot;&gt;Eigenfactor&lt;/a&gt;.  And many people are now proposing &lt;a href=&quot;http://altmetrics.org/manifesto/&quot;&gt;the use of social web metrics to rank journals&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not use the impact factor or other such metrics in my own decisions, as I feel that my direct experience gives me a more accurate measure of a journal's significance.  An interesting exercise is to go through your personal library of references (e.g., in a bibtex file) and see the number of articles it contains from different journals.  That will probably give you the best indication of where you should publish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, there is a proposal to create an expert-based ranking of math journals; see &lt;a href=&quot;http://blog.mathunion.org/fileadmin/IMU/Report/WG_JRP_Report_01.pdf&quot;&gt;this report&lt;/a&gt; and &lt;a href=&quot;http://blog.mathunion.org/journals/?no_cache=1&quot;&gt;this official IMU blog&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also be aware that there has recently been an explosion of new low-quality journals based on an author-pay model.  Their publishers frequently spam large communities of scientists (often including many who are unqualified) in an attempt to get reasonably respectable names on their editorial boards and solicit paper submissions.  As far as I can tell, they are profit-oriented &lt;a href=&quot;http://en.wikipedia.org/wiki/Vanity_press&quot;&gt;vanity press&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-01-15T08:56:30.407" LastActivityDate="2012-01-15T08:56:30.407" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="783" PostTypeId="2" ParentId="773" CreationDate="2012-01-15T07:51:49.943" Score="5" Body="&lt;p&gt;What factors determine where I will publish a paper? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Will the people I want to read this paper see it?&lt;/strong&gt; If I'm following up on the work of another group (perhaps to show a different viewpoint, sometimes to show algorithmic improvements or to fix problems with a previous paper), I will want to submit the paper to the same journal, even if there's an impact factor issue. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As a young computational scientist still in the career development phase of my career, I would submit that there is an additional, and perhaps even more critical aspect to this question. . . .&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Will people who may be in a position to evaluate me see this paper?&lt;/strong&gt; I've often spoken with peers in the computational science field about the need to have a &quot;home turf&quot;: computational science is a highly interdisciplinary field. Unfortunately, we are not really able to be considered as computational scientists when it comes time to be considered for a permanent position. In the absence of working in a computational sciences department, We will have to apply for tenure in an existing department, which usually means that our &quot;peers&quot; will be other engineers, scientists, and mathematicians—many of whom do not really have a strong background in computational science. This means that even if you want to go in the cross-fertilizing direction, you still need to focus some of your publications in the go-to journals for your discipline. This is a challenge that many of our peers will not necessarily have to face, and it's an additional complication in our lives. But it's something we have to be aware of before we start working!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;How much competition do I have right now?&lt;/strong&gt; The more crowded a field, the more important it is to get the results in early. While it's great to try to go to &lt;em&gt;Nature&lt;/em&gt; or &lt;em&gt;Science&lt;/em&gt; with every paper out of your group (presuming you're not in pure math, or something similar), being first out of the gate in a &quot;hot&quot; field matters much more than publishing in the &quot;best&quot; journal in a field.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;How important is this paper?&lt;/strong&gt; A paper that represents a body of work that provides a lot of new data, but not really much in the way of ground-breaking insight, probably doesn't merit going to a top-level journal. It's probably better to look for a reputable journal. However, if you've really found something big, shoot high, so long as you're not worried about the time crunch that affects the big journals. (It can take longer, for instance, to publish in &lt;em&gt;Physical Review Letters&lt;/em&gt; than in one of the other &lt;em&gt;Physical Review&lt;/em&gt; series, which are of comparable quantity.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;After all of these are taken into account, then I'll start worrying about issues like &lt;a href=&quot;http://admin-apps.webofknowledge.com/JCR/JCR?&quot;&gt;impact factor&lt;/a&gt;, but then only as a loose quality control measure. Differences of 10-20% are essentially meaningless, but a 1.0 versus a 2.0, or a 2.0 versus a 3.0, does represent a measurable level of difference between journals. &lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-15T07:51:49.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="784" PostTypeId="2" ParentId="769" CreationDate="2012-01-15T09:50:26.880" Score="10" Body="&lt;p&gt;We reached the petascale through an extreme effort in concurrency.  While the computational capability of modern processing cores has increased by roughly an order of magnitude in the last ten years (see Herb Sutter's excellent article &lt;a href=&quot;http://www.gotw.ca/publications/concurrency-ddj.htm&quot;&gt;&quot;The Free Lunch is Over&quot;&lt;/a&gt;), we are now capable of much more sophisticated shared-memory and distributed memory architectures, as evidenced by the success of GPUs for scientific computing and massively parallel systems such as the IBM Blue Gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The largest challenge to exascale systems is fundamentally financial.  The &lt;a href=&quot;http://en.wikipedia.org/wiki/K_computer&quot;&gt;K computer&lt;/a&gt; cost well over 1 billion dollars to develop. An exaflop machine would require similar investment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second largest challenge is arguably power.  A system one hundred times the size of the 10 petaflop/s K machine would require 1 GW.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think another major challenge to the utility of an exascale machine is the need for algorithms and software that are not only extremely concurrent, but largely asynchronous as well, as the costs of synchronizing millions of processing cores will eventually overwhelm any benefits of a system so powerful.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-01-16T08:10:03.880" LastActivityDate="2012-01-16T08:10:03.880" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="785" PostTypeId="2" ParentId="768" CreationDate="2012-01-15T10:06:10.053" Score="9" Body="&lt;p&gt;Complex numerical differentiation is stable, unlike real numerical differentiation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See pages 32-33 of &quot;Applied and Computational Complex Analysis&quot; vol 3, Peter Henrici,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The Complex-Step Derivative Approximation&quot;, &#xA;JOAQUIM R. R. A. MARTINS, PETER STURDZA and JUAN J. ALONSO,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_differentiation#Complex_variable_methods&quot;&gt;this Wikipedia article&lt;/a&gt; on complex variable methods for numerical differentiation.&lt;/p&gt;&#xA;" OwnerUserId="608" LastEditorUserId="276" LastEditDate="2012-01-15T12:46:15.597" LastActivityDate="2012-01-15T12:46:15.597" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="786" PostTypeId="2" ParentId="768" CreationDate="2012-01-15T10:10:19.093" Score="3" Body="&lt;p&gt;Complex interval arithmetic uses different types of interval, e.g. rectangular or circular, so there's more to consider than when using real intervals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Complex interval arithmetic and its applications&quot;, Miodrag Petković, Ljiljana Petković&lt;/p&gt;&#xA;" OwnerUserId="608" LastActivityDate="2012-01-15T10:10:19.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="787" PostTypeId="2" ParentId="768" CreationDate="2012-01-15T10:12:40.680" Score="1" Body="&lt;p&gt;An article:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Numerical algorithms based on the theory of complex variable&quot;,&#xA;JN Lyness - Proceedings of the 1967 22nd national conference, 1967&lt;/p&gt;&#xA;" OwnerUserId="608" LastActivityDate="2012-01-15T10:12:40.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="788" PostTypeId="2" ParentId="773" CreationDate="2012-01-15T10:22:44.417" Score="3" Body="&lt;p&gt;Rating is necessarily a subjective activity. If you care about how others will rate your paper, then you need to figure out what &lt;em&gt;their&lt;/em&gt; criteria are. As a rule of thumb, the less competence they have in your field of work, the more they will rely on journal rating rather than rating your work, and the more they will rely on bibliometry (impact factors, h factors, ...) rather than personal experience.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2012-01-15T10:22:44.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="789" PostTypeId="2" ParentId="699" CreationDate="2012-01-15T10:33:22.517" Score="2" Body="&lt;p&gt;Publishing the generated Fortran code sounds like a very good solution in your case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a general rule, I think that the use of unpublished software is incompatible with the goals of reproducible research. That's one of the reasons why I see reproducible research as a long-term goal rather than something one could require to be applied immediately.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2012-01-15T10:33:22.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="790" PostTypeId="2" ParentId="494" CreationDate="2012-01-15T10:50:06.703" Score="2" Body="&lt;p&gt;Structured grids allow for a lot of assumptions that can be exploited for performance but are generally more difficult to implement and less efficient to perform than unstructured grids in presence of complex boundaries. Unstructured grids will efficiently approximate complex boundaries at no additional programming but very few assumptions can be made about the matrix structure. As always, there is no better approach other than the one better suited for your needs. The former is often employed in ocean, climate, cosmo/geo modeling, the latter in engineering problems.&lt;/p&gt;&#xA;" OwnerUserId="474" LastActivityDate="2012-01-15T10:50:06.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="791" PostTypeId="2" ParentId="773" CreationDate="2012-01-15T12:56:22.867" Score="2" Body="&lt;p&gt;My short answer is: Unfortunately, there is (almost).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is the &lt;a href=&quot;http://www.austms.org.au/Rankings/AustMS_final_ranked.html&quot; rel=&quot;nofollow&quot;&gt;rating by the Aust MS&lt;/a&gt; for mathematical journals and I think there are other ratings as well. However, in my opinion this is not helpful. I quote from my own blog: &quot;My opinion is that rating is possible but both useless and dangerous. It is possible, since you can ask experienced mathematicians and you will get a reliable answer. It is useless, since you either know which journals are good or you can ask a colleague (which is basically the same reason as the previous one). It is dangerous, since it offers the possibility to form decisions on tenure or grants on these numbers and shifts the focus from &lt;em&gt;what&lt;/em&gt; you publish to &lt;em&gt;where&lt;/em&gt; you publish.&quot;&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="549" LastEditDate="2012-01-16T01:37:14.693" LastActivityDate="2012-01-16T01:37:14.693" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="792" PostTypeId="2" ParentId="771" CreationDate="2012-01-15T13:03:35.400" Score="3" Body="&lt;p&gt;To elaborate on the comment by Geoff: Delaunay triangulation and Voronoi diagrams are &quot;objects&quot; rather than &quot;problems&quot;. Hence, speaking of &quot;solutions&quot; is a bit off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The duality is between tessalations and triangulations: To move from the triangulation to the tesselation, you form the Voronoi set of the vertices of the triangulation. To move from the Voronoi tesselation to the Delaunay triangulation, you connect the &quot;midpoints&quot; of two cells if they touch each other.&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="549" LastEditDate="2012-01-16T01:37:02.807" LastActivityDate="2012-01-16T01:37:02.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="793" PostTypeId="2" ParentId="771" CreationDate="2012-01-15T13:14:39.693" Score="5" Body="&lt;p&gt;The two problems ask to come up with tilings that are effectively the reverse of each other: the Voronoi tessellation of a set of points ${\bf P}$ is the set of geometric objects ${\bf G}$ such that every point in the interior of object $G_i$ is closer to point $P_i$ than it is to any other point $P_j \in {\bf P}, j \neq i$. The Delaunay triangulation is effectively the reverse of this: it is the set of triangles that join together the set of points ${\bf P}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a sense, this is similar to the duality existing between triangular and hexagonal lattices in statistical physics. The midpoints of the cells in an equilateral triangular lattice, when connected form a hexagonal lattice, and &lt;em&gt;vice versa&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it should be pointed out that not all Voronoi tessellations are duals of Delaunay triangulations; this relationship is probably valid only for &lt;em&gt;unweighted&lt;/em&gt; Voronoi tessellations. For weighted tessellation methods, in which something other than the Euclidean distance is used to determine the edges, the corresponence breaks down.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-15T13:14:39.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="794" PostTypeId="5" CreationDate="2012-01-15T17:11:16.517" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T17:11:16.517" LastActivityDate="2012-01-15T17:11:16.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="795" PostTypeId="4" CreationDate="2012-01-15T17:11:16.517" Score="0" Body="The generation of a set of interconnected geometric objects that approximate a desired domain or region in euclidean space." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-15T19:51:39.100" LastActivityDate="2012-01-15T19:51:39.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="796" PostTypeId="2" ParentId="660" CreationDate="2012-01-15T18:33:02.820" Score="4" Body="&lt;p&gt;Journal of Machine Learning Research (JMLR) has a 'Machine Learning Open Source Software' track. It is fairly selective (which I think is a good thing). The journal is entirely free (for authors and readers) and ran by volunteers.&lt;/p&gt;&#xA;" OwnerUserId="616" LastActivityDate="2012-01-15T18:33:02.820" CommentCount="0" CommunityOwnedDate="2012-01-15T18:33:02.820" ContentLicense="CC BY-SA 3.0" />
  <row Id="797" PostTypeId="2" ParentId="669" CreationDate="2012-01-15T18:36:32.050" Score="3" Body="&lt;p&gt;Actually, I found the answer &lt;a href=&quot;https://mathoverflow.net/questions/28303/split-polygon-into-minimum-amount-of-rectangles-and-triangles&quot;&gt; here &lt;/a&gt;.  Thanks for the help though...&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2012-01-15T18:36:32.050" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="798" PostTypeId="2" ParentId="494" CreationDate="2012-01-15T19:15:21.707" Score="4" Body="&lt;p&gt;Dynamic $h$-adaptive mesh refinement is very effective for picking up &lt;strong&gt;isotropic&lt;/strong&gt; features of the solution. Many AMR implementation enforce a refinement criterion like $2:1$ for octree refinement. This sometimes leads to a larger refined region than necessary, but usually some geometric grading is necessary anyway and different AMR implementations can keep the refinement localized within a modest constant factor of the best unstructured refined mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $h$-AMR has the most trouble is capturing &lt;strong&gt;anisotropic&lt;/strong&gt; features, because $h$-AMR does not have a way to rotate the coordinates to conform to these anisotropic features. For example, high Reynolds number CFD has extremely thin boundary layers and meshes with aspect ratio of $10^6$ or higher are used in many industrial simulations. Capturing such thin boundaries with an unaligned structured grid is quite inefficient. There are ways to nest locally rotated grids, but these methods are quite technical and outside the scope of octree refinement or cut-cell methods. Note that moving mesh methods (often called $r$-refinement, &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1441979158&quot; rel=&quot;nofollow&quot;&gt;Huang and Russell's recent book&lt;/a&gt; is a good resource) can be used to align to moving anisotropic features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that implicit time discretization and the method of lines are simpler and have nicer properties for methods in which the number of dofs and connectivity of the mesh is not changing. Additionally, provided the physics and spatial discretization is continuously differentiable, there will be a continuous adjoint (useful for sensitivity analysis, optimization, uncertainty quantification, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best choice is highly problem dependent, but for CFD problems with thin boundary layers, especially when using wall resolution instead of wall modeling, unstructured or block-structured conforming meshes are good choices.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-15T19:15:21.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="799" PostTypeId="2" ParentId="773" CreationDate="2012-01-15T20:04:39.863" Score="3" Body="&lt;p&gt;This is a conversation you should be having with the people above you that are going to evaluate you. If you're a PhD student, then you should be talking to your advisor and other professors about how their department and other departments at other institutions evaluate the work of tenure-track faculty when making their tenure decisions. If you're tenure-track faculty now, you should have already had this conversation with your department chair and other members of the department's budget council or similar bodies about how they evaluate the work of tenure candidates. If you're not a professor of some fashion, or if you're an academic in a place without tenure, you should ask the same questions of your supervisor about promotion and retention at your institution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My experience has been that most researchers don't really read or try to keep up with publications in journals relevant to their areas directly as they come out. Not many people have personal subscriptions to IJNMF, say, and read each month's issue cover to cover. There are just too many venues to keep up with. More often, I find, people are searching for related work, using email or google alerts to seek out related items, and following the work of colleagues and competitors directly. So, the important thing is not trying to get your work seen via journals, but to publish in the best possible venue as determined by those who will be judging your career.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way to get your work seen is to aggressively market it through conferences, seminars, free software, your website, etc. This probably won't impact your promotions committee's decision (or the like), but if people are seeing and using your work through other venues, and you give them preprints and links to where it is published in the journals, then that will increase your citation counts, which is something important to those above you. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-01-15T20:04:39.863" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="800" PostTypeId="2" ParentId="699" CreationDate="2012-01-15T20:52:36.693" Score="4" Body="&lt;p&gt;I have never been of the opinion that &quot;reproducible research&quot; means that your fellow readers and scientists should only have to click &quot;Run&quot; (or compile or what have you) to entirely reproduce your results. If nothing else, that would kill the research of anyone who uses a proprietary statistical analysis package, or make things &quot;unreproducable&quot; if they were made for a platform or language that fell out of fashion. Or for that matter, is someone wants to reproduce your code, but doesn't understand/use the language you use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CrossValidated had a similar discussion on the topic - does &quot;reproducible&quot; mean you can replicate my analysis on my data, or that the experiment itself can be rerun from scratch and provide confirmatory results. The link is here: &lt;a href=&quot;https://stats.stackexchange.com/questions/14999/how-are-we-defining-reproducible-research&quot;&gt;https://stats.stackexchange.com/questions/14999/how-are-we-defining-reproducible-research&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the more important element is to describe &lt;em&gt;what you did&lt;/em&gt; on the level where it &lt;em&gt;could be recoded&lt;/em&gt; by someone who was sufficiently interested, and if commercial tools were used, those tools are mentioned. Because that serves both the interests of replicating the experiment and double-checking for code errors.&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="-1" LastEditDate="2017-04-13T12:44:17.923" LastActivityDate="2012-01-15T20:52:36.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="801" PostTypeId="2" ParentId="626" CreationDate="2012-01-15T20:53:24.283" Score="0" Body="&lt;p&gt;If you have a pseudocode for the algorithm that you are implementing, one key to determining the asymptotic computational complexity is to observe the nested loops (for loops, while loops, etc...).  If you have nested for loops which contain only operations that are computed in constant time (or at least, bounded by a constant), then the asymptotic complexity rests in the total number of iterations invoked by the loops.  For example:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1 to n  &#xA;    for j=1 to n  &#xA;        DO STUFF... (all constant time operations)  &#xA;    endfor  &#xA;endfor  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This algorithm would run in $\Theta({n}^{2})$ time because the instructions &quot;DO STUFF...&quot; occur $n^{2}$ times. If, on the other hand, you had while loops or if-then statements, then&#xA;it becomes more important to ask yourself whether you are interested in the best case, worst case, or average case scenario, since each may be different.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-01-15T20:53:24.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="802" PostTypeId="2" ParentId="773" CreationDate="2012-01-15T21:00:44.483" Score="1" Body="&lt;p&gt;There are some ranking systems for citations, which one might argue are decent measures of prestige - important papers get cited more and the like - the most common of these is &quot;impact factor&quot;. But these all have their own problems - they're not constant by field, they're susceptible to manipulation by clever researchers, etc. Others that are &lt;em&gt;slightly&lt;/em&gt; better are the h-index and g-index, but...meh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, I've found there is a &quot;feel&quot; in the field for what's good and what's not - and more importantly, what's appropriate and what isn't for your paper. For example, I have one paper that managed to get into a very good journal, but its utterly inappropriate, so it languishes a bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your evaluators or colleagues would probably be the best sources to talk to. Experience is often the best judge of what counts as &quot;good&quot; for a particular paper that would lend itself well to a particular journal. General field (Biology, Medicine, etc.) rankings, or even worse, Academia-wide rankings may not be of any use.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2012-01-15T21:00:44.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="803" PostTypeId="5" CreationDate="2012-01-15T21:58:24.843" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T21:58:24.843" LastActivityDate="2012-01-15T21:58:24.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="804" PostTypeId="4" CreationDate="2012-01-15T21:58:24.843" Score="0" Body="A theoretical estimate of the rate at which the number of transistors on an integrated circuit (and consequently, computing speed) increases.  Coined by Caltech professor Carver Mead (1970), it was postulated that this rate was about once every 18 months.  Though, there are physical limitations which inhibit the applicability of this law indefinitely." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-16T06:10:57.413" LastActivityDate="2012-01-16T06:10:57.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="805" PostTypeId="2" ParentId="698" CreationDate="2012-01-15T22:01:48.127" Score="6" Body="&lt;p&gt;The &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0167819106800219&quot; rel=&quot;nofollow&quot;&gt;Hockney model&lt;/a&gt; is an excellent simple model for communication: the time to send a single message of $m$ words from one process to another is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;T(m) = \alpha + \beta m&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha$ is the so-called &lt;em&gt;latency&lt;/em&gt; cost, and $\beta$ is the inverse bandwidth. Typically, the larger the machine, the larger $\alpha$ is. Clearly the Hockney model does not take into account more subtle issues like network congestion, but it's hard to stress exactly how much mileage one can get out of this model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to see an analysis of various algorithms for MPI collective communication using the Hockney model, I highly recommend &lt;a href=&quot;http://www.cs.utexas.edu/users/flame/pubs/InterCol_TR.pdf&quot; rel=&quot;nofollow&quot;&gt;Collective Communication: Theory, Practice, and Experience&lt;/a&gt;. It discusses various algorithms for broadcast, reduce, scatter, gather, allgather, reduce-scatter, and allreduce on various network topologies.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-02-10T02:22:54.957" LastActivityDate="2012-02-10T02:22:54.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="806" PostTypeId="5" CreationDate="2012-01-15T22:12:59.863" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T22:12:59.863" LastActivityDate="2012-01-15T22:12:59.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="807" PostTypeId="4" CreationDate="2012-01-15T22:12:59.863" Score="0" Body="Generally referring to newton's method for solving systems of equations (both linear and non-linear) based upon a root-finding iteration." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-16T06:11:02.840" LastActivityDate="2012-01-16T06:11:02.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="808" PostTypeId="5" CreationDate="2012-01-15T22:15:07.957" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T22:15:07.957" LastActivityDate="2012-01-15T22:15:07.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="809" PostTypeId="4" CreationDate="2012-01-15T22:15:07.957" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T22:15:07.957" LastActivityDate="2012-01-15T22:15:07.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="810" PostTypeId="5" CreationDate="2012-01-15T22:16:38.003" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T22:16:38.003" LastActivityDate="2012-01-15T22:16:38.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="811" PostTypeId="4" CreationDate="2012-01-15T22:16:38.003" Score="0" Body="A method for solving optimization problems where solution iterates are in the interior of the feasible region." OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-01-16T03:03:42.713" LastActivityDate="2012-01-16T03:03:42.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="813" PostTypeId="5" CreationDate="2012-01-15T22:30:09.687" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-15T22:30:09.687" LastActivityDate="2012-01-15T22:30:09.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="814" PostTypeId="4" CreationDate="2012-01-15T22:30:09.687" Score="0" Body="For a given finite point set S, a voronoi diagram is a tessellation of a euclidean space.  In the 2D case, it consists of conforming convex polygons surrounding each point such that for given a point p in S, any point in the enclosing polygon is closer to p than any other point in S. " OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-16T06:12:39.530" LastActivityDate="2012-01-16T06:12:39.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="815" PostTypeId="1" AcceptedAnswerId="827" CreationDate="2012-01-15T23:44:48.110" Score="6" ViewCount="446" Body="&lt;p&gt;Suppose i was trying to solve a parabolic PDE (heat equation) on a rectangular domain using an explicit finite difference scheme.  I am storing my solution vector in a matrix form (because it closely resembles the shape of the domain).  I know that I can break the matrix into rectangular blocks and send information across the boundaries of each block (as needed) using ghost points.  However, when I use a fine grid, the processors have to send a lot of information between them, and this seems to slow down the overall performance.  Is there anything that I can do to increase the scalability of the algorithm?  Is there a theoretical limit to how much improvement I can expect through the use of ghost points?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="123" LastEditDate="2012-01-16T06:27:28.733" LastActivityDate="2012-01-16T18:02:21.863" Title="How can I reduce the communication bottleneck of a parallel explicit finite difference scheme?" Tags="|pde|parallel-computing|finite-difference|explicit-methods|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="816" PostTypeId="1" CreationDate="2012-01-16T01:03:57.587" Score="16" ViewCount="16158" Body="&lt;p&gt;I have a data set $x_{1}, x_{2}, \ldots, x_{k}$ and want to find the parameter $m$ such that it minimizes the sum $$\sum_{i=1}^{k}\big|m-x_i\big|.$$ &#xA;that is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{m}\sum_{i=1}^{k}\big|m-x_i\big|.$$&lt;/p&gt;&#xA;" OwnerUserId="627" LastEditorUserId="7951" LastEditDate="2018-08-15T14:52:23.973" LastActivityDate="2018-08-15T14:52:23.973" Title="Minimizing the Sum of Absolute Deviation ($ {L}_{1} $ Distance)" Tags="|optimization|convex-optimization|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="817" PostTypeId="2" ParentId="769" CreationDate="2012-01-16T02:21:06.073" Score="6" Body="&lt;p&gt;I agree with &lt;a href=&quot;https://scicomp.stackexchange.com/a/784/149&quot;&gt;Aron's answer&lt;/a&gt;, but I would go further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, as I mentioned in my first comment, there are multiple reports and presentations out there. If you look at the US Department of Energy (DOE) Advanced Scientific Computing Research (ASCR) Programs and Reports &lt;a href=&quot;http://science.energy.gov/ascr/news-and-resources/program-documents/&quot; rel=&quot;nofollow noreferrer&quot;&gt;web site&lt;/a&gt;, you'll see that the first 4 reports mention exascale in the title, and several reports since 2008 have mentioned computing &quot;at the extreme scale&quot; in the title, and include references to exascale computing. (As a side note, these reports also happen to be good for applying to government programs, because you get to see what these programs care about. Skimming some of these reports was part of the motivation for the comment.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think that the largest challenge to exascale systems is financial, given the considerable emphasis that computational scientists, and people in government, are placing on &quot;big iron&quot;. The effort to reach petascale included a plethora of DOE reports that trumpeted petascale as the next big important thing in computational science, ostensibly for national security reasons as well as making new science possible. As noted in the Fall 2010  DOE ASCR report &lt;a href=&quot;http://science.energy.gov/~/media/ascr/ascac/pdf/reports/Exascale_subcommittee_report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The Opportunities and Challenges of Exascale Computing&lt;/em&gt;&lt;/a&gt; on page 3, &quot;the benefits outweigh the costs.&quot; There was a major funding push to reach petascale, and I don't see why the funding would dry up anytime soon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The biggest challenge, in my opinion, is power, because that's going to be an engineering design limit that severely limits where you can place exascale computing facilities. To my knowledge, the only facilities that can supply 1 GW of power (from Aron's estimate) are large-scale hydroelectric facilities (like aeismail pointed out) or large-scale nuclear power plants (which tend to supply 1 - 1.3 GW of electricity, see &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_nuclear_reactors&quot; rel=&quot;nofollow noreferrer&quot;&gt;this list of nuclear reactors on Wikipedia&lt;/a&gt;). Both sources of power have drawbacks that will make facility placement tricky.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both power and cost are going to drive the development of entirely new technology. Estimates place exascale computing facilities constructed out of today's technology at a capital cost of \$100B &lt;em&gt;just for the computer&lt;/em&gt;, plus operating costs of \$1B/yr to supply the power, plus its own power plant, as discussed above. If you look at the 2012 US Budget (see the graphic &lt;a href=&quot;http://www.deathandtaxesposter.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for a relatively easy to read version), you'll see that the discretionary budget of the US is \$1.2T, and the DOE got around \$29.5B of that, and funding constraints are one of the reasons that we won't see an exascale system now. However, just because it's too expensive now does not mean that funding will prohibit progress, because the investment is going to focus on developing advanced computing technologies in the hopes of making them both cost-effective and energy-efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The design specs that the DOE is proposing call for an exascale system to be developed by 2018 with a power requirement of 20 MW. You can find these statistics on page 48 of &lt;a href=&quot;http://science.energy.gov/~/media/ascr/ascac/pdf/reports/Exascale_subcommittee_report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The Opportunities and Challenges of Exascale Computing&lt;/em&gt;&lt;/a&gt;. What's interesting about the statistics is how the factor of change the DOE would like to see in various components and metrics varies. For instance, the DOE would like a 2018 cluster to be 500 times faster than&#xA;its 2012 counterpart, but require 3 times the power, and have 33 times the memory. The foreseeable increase in computing power is going to come from concurrency; the DOE plans to scale up from 12 CPU/node in a 2012 cluster to 1000+ CPU/node in a 2018 cluster. For another perspective on how to realize an exascale system using different technology, see the 2008 DARPA report &lt;a href=&quot;http://www.cse.nd.edu/Reports/2008/TR-2008-13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Exascale Computing Study: Technology Challenges in Achieving Exascale Systems&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This scaling in the number of CPUs (and generally, in the number of components) is going to cause a number of problems. For starters, the sheer number of components is going to make it more likely that a component will fail during a supercomputing job. (For a rough metric of how many components of a particular type are expected to fail, take the time you expect your job to run, multiply it by the number of components of a given type, and divide by that component type's &lt;a href=&quot;http://en.wikipedia.org/wiki/Mean_time_between_failures&quot; rel=&quot;nofollow noreferrer&quot;&gt;mean time between failures (MTBF)&lt;/a&gt;.) Since component failure is more likely to occur due to quantity, software is going to have to be fault-tolerant. Also, as Aron noted, concurrency is going to be a big issue; if you want to take full advantage of exascale systems, you're going to have to use as many processors as you can, all the time. The increase in concurrency is going to require programmers to radically rethink how they design algorithms so that they can hide latency and avoid blocking conditions slowing down their code. For this reason, people propose designing asynchronous algorithms. Programming for concurrency is already a challenge in petascale systems, since many applications do not use more than 10-20% of the peak computing capacity of today's DOE supercomputing clusters; I know that there are some efforts focused on making it easier for developers to program more effectively in parallel (both in reducing developer time investments, and in increasing parallel performance). Memory locality is going to be an issue, since multicore systems are going to add another level in the memory hierarchy. One way to move closer to achieving the FLOPs per power benchmark is to use GPUs as part of a heterogeneous supercomputing cluster. Memory movement is going to become another potential bottleneck in parallel computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last thing I can think of is data. Even on today's supercomputing systems, we already have a problem in that simulations can generate more data than we know what to do with. In combustion applications, which is where most of my experience comes from, the largest supercomputing codes just throw away large quantities of data because they are limited in how much they can store. In addition to some physical intuition, they rely on algorithms such as feature extraction to help them decide what data to keep and what data to discard. As we move into the exascale, the issue of generating more data than you can store, and how you analyze it are going to become more important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And even after that long answer, I'm sure that I'm leaving stuff out.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-16T20:10:19.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="818" PostTypeId="5" CreationDate="2012-01-16T02:46:48.070" Score="0" Body="&lt;p&gt;A code 'scales well' if there is a linear relationship between the speedup and the number of processors used.  Often times, the linear relationship exists at first, but does not last indefinitely since communication costs dominate in the limit.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-16T06:11:20.873" LastActivityDate="2012-01-16T06:11:20.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="819" PostTypeId="4" CreationDate="2012-01-16T02:46:48.070" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-16T02:46:48.070" LastActivityDate="2012-01-16T02:46:48.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="820" PostTypeId="5" CreationDate="2012-01-16T03:16:52.647" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-16T03:16:52.647" LastActivityDate="2012-01-16T03:16:52.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="821" PostTypeId="4" CreationDate="2012-01-16T03:16:52.647" Score="0" Body="An application programming interface (API) for shared memory systems, with implementations in c, c++, and fortran." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-16T06:11:11.170" LastActivityDate="2012-01-16T06:11:11.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="822" PostTypeId="2" ParentId="815" CreationDate="2012-01-16T03:18:32.190" Score="7" Body="&lt;p&gt;Tying this question into a previous one ( &lt;a href=&quot;https://scicomp.stackexchange.com/questions/698/when-analyzing-a-parallel-algorithm-how-do-you-take-communication-costs-into-ac&quot;&gt;When analyzing a parallel algorithm, how do you take communication costs into account?&lt;/a&gt; ) on modeling parallel efficiency, let's create a simple model where we take a look at how the computation behaves as we change the number of processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we have a grid of size $N^d$, and $P$ processors -- we'll assume that $P$ is a perfect square/cube/whatever and that $N$ is evenly divided by $P^{1/d}$  (if it isn't, it just introduces some modest load imbalance).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the actual explicit computation involves $n_\mathrm{ops}$ operations (memory, or floating point) operations per zone, each of which has (time) cost $c_\mathrm{ops}$.   Further, let's model the time spent to perform a communication with a latency and a bandwidth; it takes a message of even zero size some time $l$ to go fron node to node, and there is in addition a finite bandwidth $b$ to the network, so that a message of size $s$ takes time $l + s/b$.   (In practice, things are usually a bit more complicated than this; local neighbours might be on-node, for instance, and splitting bandwidth between cores; and distant neighbours might not even be on the same switch.   So you might need to use something other than the nominal network parameters.   Still, this is a sensible start.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The total computation time taken by the decomposed problem per timestep is&#xA;$$&#xA;t_\mathrm{comp} = \frac{N^d}{P} n_\mathrm{ops} c_\mathrm{ops}&#xA;$$&#xA;and the total communication time is&#xA;$$&#xA;t_\mathrm{comm} = 2^d \left ( l + \frac{n_g N^{d-1}}{b P^{(d-1)/d}} \right )&#xA;$$&#xA;where at each timestep, one has to send to each of one's $2^d$ neighbours slice of $n_g \times (N/P^{1/d})^{(d-1)}$ guard cells.   (eg, in 2d, $n_g \times N/\sqrt{P}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Further, explicit parabolic methods generally for stability require timesteps that scale as $\Delta x^{-2}$, so that we would need to take a number of timesteps that scales as $N^2$ to reach a given physical elapsed time.  That's relevant for how the computational cost scales wtih $N$, but not parallel scalability, so we won't need this here.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parallel speedup here would be &#xA;$$&#xA;\begin{eqnarray*}&#xA;S &amp;amp; = &amp;amp; \frac{t(N,P=1)}{t(N,P)} \\&#xA; &amp;amp; = &amp;amp; \frac{N^d n_\mathrm{ops} c_\mathrm{ops}}{\frac{N^d}{P} n_\mathrm{ops} c_\mathrm{ops} + 2^d \left ( l + \frac{n_g N^{d-1}}{b P^{(d-1)/d}} \right )} \\&#xA; &amp;amp; = &amp;amp; \frac{P}{1 + \frac{P}{N^d n_\mathrm{ops} c_\mathrm{ops}} 2^d \left ( l + \frac{n_g N^{d-1}}{b P^{(d-1)/d}} \right )} \\&#xA; &amp;amp; = &amp;amp; \frac{P}{1 + \frac{t_\mathrm{comm}}{t_\mathrm{comp}}} \\&#xA;\end{eqnarray*}&#xA;$$&#xA;-- that is, the speedup would be exactly $P$ if the communications time were zero, which isn't all that surprising.   Looking a bit more closely at the communcations to computation ratio:&#xA;$$&#xA;\begin{eqnarray*}&#xA;\frac{t_\mathrm{comm}}{t_\mathrm{comp}} &amp;amp; = &amp;amp; \frac{P}{N^d n_\mathrm{ops} c_\mathrm{ops}} 2^d \left ( l + \frac{n_g N^{d-1}}{b P^{(d-1)/d}} \right ) \\&#xA;\end{eqnarray*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's look at the latency-dominated scenario; that is, you're sending small messages, so that the time for communications is dominated just by the &lt;em&gt;number&lt;/em&gt; of messages you're sending.  (You would always get to this point eventually, as the message size, as we'll see, goes down with number of processors.)   In this case, &#xA;$$&#xA;\begin{eqnarray*}&#xA;\frac{t_\mathrm{comm}}{t_\mathrm{comp}} &amp;amp; = &amp;amp; \frac{P}{N^d n_\mathrm{ops} c_\mathrm{ops}} 2^d  l  \\&#xA; &amp;amp; \propto &amp;amp; P / N^d&#xA;\end{eqnarray*}&#xA;$$&#xA;Here the relative parallel overhead goes linearly as you add more processors -- this is not because communications is getting more expensive, but because the computational cost is going down linearly in P as we're adding more processors and each only has to do $N^d/P$ work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the bandwidth dominated case -- for very large messages or very slow networks -- you'd instead have:&#xA;$$&#xA;\begin{eqnarray*}&#xA;\frac{t_\mathrm{comm}}{t_\mathrm{comp}} &amp;amp; = &amp;amp; \frac{P}{N^d n_\mathrm{ops} c_\mathrm{ops}} 2^d  \frac{n_g N^{d-1}}{b P^{(d-1)/d}}  \\&#xA; &amp;amp; \propto &amp;amp; P^{1/d} / N&#xA;\end{eqnarray*}&#xA;$$&#xA;In this regime -- which you are likely in at the beginning (especially in 3d) with comparitively large messages, especially on ethernet -- the overhead is much more modest, scaling as the cube root of P (in 3d), because as you increase the number of processes, you are also reducing the size of the messages you have to send because the grid is more finely divided and you have less overlap with your neighbouring processor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's fun to put some numbers in; let's say we do a weak scaling study in 3D, with $N^3/P$ fixed at $256^3$, and with 1 guardcell of double precision numbers, with gigabit ethernet so say latency is 100 microseconds and 1/bandwidth is about 0.06 microseconds/double.   Let's say you have to do 100 operations per zone, with each operation taking 1 nanoseconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you have&#xA;$$&#xA;\begin{eqnarray*}&#xA;\frac{t_\mathrm{comm}}{t_\mathrm{comp}} &amp;amp; = &amp;amp; \frac{P}{N^d n_\mathrm{ops} c_\mathrm{ops}} 2^d  \left( l + \frac{n_g N^{d-1}}{b P^{(d-1)/d}} \right) \\&#xA; &amp;amp; = &amp;amp; \frac{8}{256^3 (100 \times 10^{-9} \mathrm{sec})} \left ( 10^{-4} \mathrm{sec} +  256^2 \cdot 0.06 \cdot 10^{-6} \mathrm{sec} \right ) \\&#xA; &amp;amp; = &amp;amp; 0.02 \\&#xA;\end{eqnarray*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is constant in the weak scaling regime, since the work per processor -- even the communication work! -- is constant.   Since this is constant, you would have your speedup be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{eqnarray*}&#xA;S &amp;amp; = &amp;amp; \frac{P}{1 + 0.02} \\&#xA;   &amp;amp; = &amp;amp; 0.98 P&#xA;\end{eqnarray*}&#xA;$$&#xA;or an efficiency of 98%, which is quite good.     Note too that in this regime you are definitely bandwidth, rather than latency, dominated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can play with other scaling regimes, or play with other network transports. &#xA;Let's play with strong scaling, say a constant $N^3 = 1024^3$:  here you'd have&#xA;$$&#xA;\begin{eqnarray*}&#xA;\frac{t_\mathrm{comm}}{t_\mathrm{comp}} &amp;amp; =  &amp;amp; \frac{P}{N^d n_\mathrm{ops} c_\mathrm{ops}} 2^d  \left( l + \frac{n_g N^{d-1}}{b P^{(d-1)/d}} \right) \\&#xA; &amp;amp; = &amp;amp; \frac{8 P}{1024^3 (10 \times 10^{-9} \mathrm{sec})} \left ( 10^{-4} \mathrm{sec} +  \frac{1024^2}{P^{2/3}} \cdot 0.06 \cdot 10^{-6} \mathrm{sec} \right ) \\&#xA; &amp;amp; = &amp;amp; 0.0745 P \, \mathrm{sec}^{-1} \, \left ( 10^{-4} \,\mathrm{sec} + 6.29 \times 10^{-2} \,\mathrm{sec}\, P^{-2/3} \right ) \\&#xA;\end{eqnarray*}&#xA;$$&#xA;&lt;img src=&quot;https://i.stack.imgur.com/uIIJT.png&quot; alt=&quot;Scaling Plot&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is plotted above; obviously, treating things continuously when we've assumed that P is a perfect cube and divides N is problematic, but it still gives you a picture of what's going on.   For the first while, bandwidth dominates your communication overhead, but eventualy you switch over to latency.    Here, with the relative overhead increasing with $P$ (again, due to the fact that computation is decreasing), you definitely do not get a constant efficiency, but one which falls with $P$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, can you do better than this -- improve things with different approaches to coding?  Yes, there's a couple classic things you can do.  Probably most important is to overlap communications with computation.   Rather than communicating the guard cells, &lt;em&gt;then&lt;/em&gt; doing all the processor's computation, so that the total time taken is $t_\mathrm{comm} + t_\mathrm{comp}$, you can start the process of communicating the guardcells, then do the inner zones computation while communications is taking place (with infiniband, or with communications threads), and then once the receive is done, do the outer zones calculation.   This allows you to hide some of the communications time by doing real computation while it is occuring.  Then the time is, instead, $\max{t_\mathrm{comm}, t_\mathrm{comp}}$ -- or more precisely, $\max{t_\mathrm{comm}, t_\mathrm{comp,inner}} + t_\mathrm{comp,outer}$.  This can be a significant improvement!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note too here that in such a code there is often some kind of reduction operation at the end of each step; that isn't being modelled here.   Neither is I/O of any sort.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastEditorUserId="-1" LastEditorDisplayName="user389" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-01-16T18:02:21.863" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="823" PostTypeId="2" ParentId="816" CreationDate="2012-01-16T06:49:01.343" Score="16" Body="&lt;p&gt;A generalization of this problem to multiple dimensions is called the &lt;a href=&quot;http://en.wikipedia.org/wiki/Geometric_median&quot;&gt;geometric median problem&lt;/a&gt;. As David points out, the median is the solution for the 1-D case; there, you could use median-finding &lt;a href=&quot;http://en.wikipedia.org/wiki/Median_algorithm&quot;&gt;selection algorithms&lt;/a&gt;, which are more efficient than sorting. Sorts are $O(n\log n)$ whereas selection algorithms are $O(n)$; sorts are only more efficient if multiple selections are needed, in which case you could sort (expensively) once, and then repeatedly select from the sorted list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The link to the geometric median problem mentions solutions for multidimensional cases.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-16T06:49:01.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="824" PostTypeId="2" ParentId="816" CreationDate="2012-01-16T07:39:53.760" Score="22" Body="&lt;p&gt;Probably you ask for a proof that the median solves the problem? Well, this can be done like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is piecewise linear and hence differentiable except for the points $m=x_i$. What is the slope of the objective is some point $m\neq x_i$? Well, the slope is the sum of the slopes of the mappings $m\mapsto |m-x_j|$ and this is either $+1$ (for $m&gt;x_j$) or $-1$ (for $m&amp;lt;x_j$ ). Hence, the slope indicates how many $x_i$'s are smaller than $m$. You see that the slope is zero if there are equally many $x_i$'s smaller and larger than $m$ (for and even number of $x_i$'s). If there is an odd number of $x_i$'s then the slope is $-1$ left of the &quot;middlest&quot; one and $+1$ right of it, hence the middlest one is the minimum.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2012-01-16T07:39:53.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="825" PostTypeId="2" ParentId="626" CreationDate="2012-01-16T11:17:37.993" Score="2" Body="&lt;p&gt;It's kind of straight forward to calculate the complexity of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Connected-component_labeling#Two-pass&quot; rel=&quot;nofollow&quot;&gt;two-pass algorithm&lt;/a&gt; from the Wikipedia page you linked: First, one iterates from upper left to lower right over all pixels and assigns a preliminary label to each pixel while maintaining a map of equivalence relations for the labels. This needs four checks for every pixel. In the second pass one goes over the labeled components and replaces the preliminary labels according to the equivalence relation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Complexity an image with $N$ pixels, $M$ of which are &quot;foreground&quot;: &#xA;$4N$ for the first pass and at most $M$ for the second pass.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2012-01-16T11:17:37.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="826" PostTypeId="2" ParentId="583" CreationDate="2012-01-16T13:58:13.583" Score="1" Body="&lt;p&gt;Try the &lt;a href=&quot;http://lvelho.impa.br/a48/pages.html&quot; rel=&quot;nofollow&quot;&gt;A48 dynamic adaptive mesh library&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2012-01-16T13:58:13.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="827" PostTypeId="2" ParentId="815" CreationDate="2012-01-16T14:18:55.397" Score="4" Body="&lt;p&gt;Jonathan's answer gives a useful performance model for a simple explicit method. In practice, memory bandwidth and network latency are more common performance limiters than network bandwidth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To improve the efficiency, it is worth considering:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;high order methods reach a given accuracy with fewer grid cells while also doing more computational work per cell, choose variants with small vertex or edge separators (like spectral elements or DG) to keep communication volume low&lt;/li&gt;&#xA;&lt;li&gt;implicit methods solve the problem without the painful stability criteria for explicit methods, but tend to require more communication per step and are more involved to program&lt;/li&gt;&#xA;&lt;li&gt;Chebyshev Runge-Kutta methods with many stages greatly extend the stability region relative to low-stage methods for pure diffusion, but these are fragile if the eigenvalues drift from the negative real axis&lt;/li&gt;&#xA;&lt;li&gt;$s$-step methods (computing multiple time steps in one sweep) further hide latency and reduce memory bandwidth limitations, but need to communicate multiple ghost layers, so the constants are poor for higher order and higher dimension problems, especially with smallish subdomains&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-16T14:18:55.397" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="828" PostTypeId="1" AcceptedAnswerId="832" CreationDate="2012-01-16T14:46:23.257" Score="3" ViewCount="1748" Body="&lt;p&gt;Suppose I had the following problem:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U_{tt}=U_{xx}+U_{yy}$ in $\Omega=[0,1]\times[0,1]$&lt;br&gt;&#xA;$U(x,y,0)=f(x,y)$&lt;br&gt;&#xA;$U_{t}(x,y,0)=g(x,y)$&lt;br&gt;&#xA;$U=0$ on $\partial \Omega$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there is an explicit finite difference scheme to solve this problem of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{U(i,j,k+1) - 2U(i,j,k)+U(i,j,k-1)}{\Delta t^2} = \frac{U(i+1,j,k) - 2U(i,j,k)+U(i-1,j,k)}{\Delta x^2} + \frac{U(i,j+1,k) - 2U(i,j,k)+U(i,j-1,)}{\Delta y^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by using a centered finite difference in time.  My mind is thinking that it's possible to simply change the k's to k+1's to obtain an implicit scheme, but I haven't worked out the ensuing truncation error or stability analysis.  My guess is that it's likely to be wrong... How do I derive an implicit scheme for this PDE?  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2015-06-01T05:54:40.110" LastActivityDate="2015-06-01T05:54:40.110" Title="How to obtain an implicit finite difference scheme for the wave equation?" Tags="|pde|hyperbolic-pde|implicit-methods|finite-difference|discretization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="829" PostTypeId="1" AcceptedAnswerId="838" CreationDate="2012-01-16T14:54:40.483" Score="11" ViewCount="1702" Body="&lt;p&gt;As I understand it, the multigrid method solves a linear system by solving a coarser version of the same problem (there by eliminating low frequency error) then projecting back to the fine grid to smoothen out the high frequency errors.  For large systems, i can see how an iterative method can be implemented in parallel at each gridlevel.  Does this approach scale well in parallel?  Is there any other source of concurrency in the algorithm that one can exploit in parallel?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-01-16T16:18:31.473" Title="How can one parallelize a multigrid method for solving a linear system of equations?" Tags="|linear-algebra|parallel-computing|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="830" PostTypeId="1" AcceptedAnswerId="834" CreationDate="2012-01-16T15:02:38.193" Score="10" ViewCount="3224" Body="&lt;p&gt;I am trying to solve a linear system of equations that is sparse, but lacks any kind of banded structure.  I have heard that there is a way to extend the principles of a multigrid solver for implicit finite difference schemes to a general linear problem (if I'm not mistaken, it's called algebraic multigrid solver).  After reading some literature on it, I am still very confused on how to interpolate (i.e. prolong and restrict) between coarse and fine grids without exploiting the nice structure of the banded matrices like those from a finite difference scheme.  Is there some heuristic to it?  Can anyone give an example?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-02-13T14:34:21.360" Title="How to construct a prolongation and restriction operator for an algebraic multigrid solver?" Tags="|linear-algebra|multigrid|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="831" PostTypeId="1" AcceptedAnswerId="833" CreationDate="2012-01-16T15:13:57.040" Score="5" ViewCount="2627" Body="&lt;p&gt;I'm trying to create a figure of particle distribution from a reference surface in python. I plan to get a distribution in python and then prettify it with tikz. I tried this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xs = np.linspace(-1, 1, 10)&#xA;ys = np.linspace(-1, 1, 10)&#xA;X, Y = np.meshgrid(xs, ys)&#xA;&#xA;Z = X&#xA;for i in range (0, 10):&#xA;  for j in range (0, 10):&#xA;    Z[i][j] = 0&#xA;&#xA;xs = randomrange(10, -1, 1)&#xA;ys = randomrange(10, -1, 1)&#xA;zs = randomrange(10, 0, 10)&#xA;ax.plot_surface(X, Y, Z)&#xA;ax.scatter(xs, ys, zs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I get a single line with an error message:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;/usr/lib/python3.2/site-packages/mpl_toolkits/mplot3d/axes3d.py:1478:&#xA;  RuntimeWarning: nvalid value encountered in true_divide for n in&#xA;  normals])&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I assume that this is due to division by 0 (trying to color the surface). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I generate a reference surface then?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried MRocklin's solution and replaced&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Z = X &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Z = np.zeros_like(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I don't get any surface at all...&lt;/p&gt;&#xA;" OwnerUserId="648" LastEditorUserId="53" LastEditDate="2014-02-12T17:54:46.507" LastActivityDate="2014-02-12T17:54:46.507" Title="creating a flat surface in python" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="832" PostTypeId="2" ParentId="828" CreationDate="2012-01-16T15:17:25.553" Score="6" Body="&lt;p&gt;There is not much point using an implicit method for pure wave propagation because you have to resolve phase to have an accurate method. If you have a hyperbolic system in which some waves are very stiff (not interesting except for their influence on evolution of a slow manifold), you might want an implicit method. It is fairly problem-dependent whether you want an A- or L-stable method, a strong stability preserving method, or a symplectic method. Unfortunately, the order conditions are mutually exclusive such that it is not possible to have all attractive properties in one integration scheme. You can also consider IMEX methods.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-16T15:17:25.553" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="833" PostTypeId="2" ParentId="831" CreationDate="2012-01-16T15:38:43.207" Score="6" Body="&lt;p&gt;In the line &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Z = X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You're just renaming X to Z, not making a copy. Your code then proceeds to zero out all elements of X. You might want to try replacing this with the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Z = X.copy()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually, the first bit of your code could be replaced with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X, Y = np.mgrid[-1:1:10j, -1:1:10j]&#xA;Z = np.zeros_like(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the full Python code to produce a flat surface using the mpl toolkit mplot3d running python 2.7.2 from the &lt;a href=&quot;http://enthought.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EPD&lt;/a&gt; academic distribution 7.1-2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ipython --pylab&#xA;In [1]: from mpl_toolkits import mplot3d&#xA;&#xA;In [2]: f = figure()&#xA;&#xA;In [3]: ax = mplot3d.Axes3D(f)&#xA;&#xA;In [4]: X, Y = mgrid[-1:1:10j, -1:1:10j]&#xA;&#xA;In [5]: Z = zeros_like(X)&#xA;&#xA;In [6]: ax.plot_surface(X,Y,Z)&#xA;/home/mrocklin/Software/epd/lib/python2.7/site-packages/mpl_toolkits/mplot3d/axes3d.py:777: RuntimeWarning: invalid value encountered in divide&#xA;  n = n / proj3d.mod(n)&#xA;Out[6]: &amp;lt;mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0xaa13d2c&amp;gt;&#xA;&#xA;In [7]: show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xVUfK.png&quot; alt=&quot;A simple flat surface&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-01-16T22:04:54.737" LastActivityDate="2012-01-16T22:04:54.737" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="834" PostTypeId="2" ParentId="830" CreationDate="2012-01-16T15:44:01.843" Score="14" Body="&lt;p&gt;First, if you have a structured grid, you might want to use geometric instead of algebraic multigrid due to some theoretical and efficiency advantages (e.g. ability to rediscretize instead of using Galerkin coarse grid operators). Algebraic multigrid methods generally fall in two categories.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Classical Algebraic Multigrid&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This method was introduced by Brandt, McCormick, and Ruge in 1982 and has strong theory for &lt;a href=&quot;http://en.wikipedia.org/wiki/M-matrix&quot;&gt;$M$-matrices&lt;/a&gt;. &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~achi/LOP43.pdf&quot;&gt;Brandt (1986)&lt;/a&gt; has the standard convergence theory. See &lt;a href=&quot;http://acts.nersc.gov/hypre/&quot;&gt;&lt;strong&gt;BoomerAMG&lt;/strong&gt; from the Hypre package&lt;/a&gt; for a popular and highly scalable parallel implementation. A more robust and general variant is called Bootstrap AMG, see this &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~achi/LOP163.pdf&quot;&gt;recent paper from Brandt, Brannick, Kahl, and Livshits&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Smoothed Aggregation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Smoothed aggregation from &lt;a href=&quot;http://dx.doi.org/10.1007/BF02238511&quot;&gt;Vaněk, Mandel, and Brezina (1996)&lt;/a&gt; is a more recent method that has proven to be useful for vector problems like elasticity. The general approach is to start with vectors characterizing the near-null space of the operator (e.g. rigid body modes in elasticity), construct aggregates using connectivity of the matrix (usually by finding a maximal independent set of $A^T A$), and &quot;smooth&quot; the aggregates (using the operator) to create lower energy coarse basis functions. Popular parallel implementations are &lt;a href=&quot;http://trilinos.sandia.gov/packages/ml/&quot;&gt;&lt;strong&gt;ML&lt;/strong&gt; from Trilinos&lt;/a&gt;, &lt;a href=&quot;http://www.columbia.edu/~ma2325/prometheus/&quot;&gt;&lt;strong&gt;Prometheus&lt;/strong&gt; from Mark Adams&lt;/a&gt; (2004 Gordon Bell prize), &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/PC/PCGAMG.html&quot;&gt;&lt;strong&gt;PCGAMG&lt;/strong&gt; in PETSc&lt;/a&gt; (also from Mark Adams, mostly a complete replacement for Prometheus), and the smoothed aggregation component of the CUDA-based GPU code &lt;a href=&quot;http://code.google.com/p/cusp-library/&quot;&gt;&lt;strong&gt;CUSP&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that all software mentioned above can be accessed through a common interface using &lt;a href=&quot;http://mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-16T15:44:01.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="835" PostTypeId="2" ParentId="830" CreationDate="2012-01-16T15:51:26.683" Score="4" Body="&lt;p&gt;&quot;Multigrid&quot; by Trottenberg, et al, is an excellent book and it looks like most of it is available on Google books.  It has an appendix on AMG and you will probably need to get some background in MG from the rest of the book.   &quot;A Multigrid tutorial&quot; is also a good book.&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2012-01-16T15:51:26.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="836" PostTypeId="2" ParentId="668" CreationDate="2012-01-16T15:52:39.207" Score="3" Body="&lt;p&gt;I would suggest the following papers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite difference method at arbitrary irregular grids and its application in applied mechanics - Liszka Orkisz&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0045794980901492&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/0045794980901492&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite difference techniques for variable grids - Jensen &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mendeley.com/research/finite-difference-techniques-variable-grids-7/&quot; rel=&quot;nofollow&quot;&gt;http://www.mendeley.com/research/finite-difference-techniques-variable-grids-7/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving parabolic and hyperbolic equations by the generalized finite difference method - Benito Urena Gavete&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S037704270600687X&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/S037704270600687X&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically they describe how to generate finite difference stencial for nonstructured/irregular meshes. I don't know any book that treat this specific topic in depth, but Randall LeVeque's book might have something about it. Here is the link for the author's webpage, which contain some Matlab m-files for finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://faculty.washington.edu/rjl/booksnotes.html&quot; rel=&quot;nofollow&quot;&gt;http://faculty.washington.edu/rjl/booksnotes.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="484" LastActivityDate="2012-01-16T15:52:39.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="837" PostTypeId="2" ParentId="816" CreationDate="2012-01-16T16:09:10.767" Score="6" Body="&lt;p&gt;The explicit solution in terms of the median is correct, but in response to a comment by mayenew, here's another approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is well-known that $\ell^1$ minimization problems generally, and the posted problem in particular, can be solved by linear programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following LP formulation will do for the given exercise with unknowns $z_i,m$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ min \sum z_i $$&#xA;such that:&#xA;$$ z_i \ge m - x_i $$&#xA;$$ z_i \ge x_i - m $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly $z_i$ must equal $|x_i - m|$ at the minimum, so this asks the sum of absolute values of errors to be minimized.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2012-01-16T16:09:10.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="838" PostTypeId="2" ParentId="829" CreationDate="2012-01-16T16:18:31.473" Score="14" Body="&lt;p&gt;Parallel geometric multigrid is straightforward to implement on structured grids. Algebraic and unstructured multigrid are more technical, see &lt;a href=&quot;https://scicomp.stackexchange.com/a/834/119&quot;&gt;this answer&lt;/a&gt; for links to implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a multiplicative method (e.g. $V$-cycles), only one level can be computed on at a time. Since the number of levels is $\log_{c} N$ where $N$ is the number of degrees of freedom and $c$ is the coarsening factor (usually about $2^d$ or $3^d$ in $d$ dimensions), this logarithmic term is not removable. Additive methods sacrifice some constant factors, but can compute all levels concurrently, thus reducing the logarithmic factor to $\log_2 \log_c N$. I have yet to see an demonstration on real hardware in which the increased concurrency justifies the poorer constants and reduced robustness of additive methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gauss-Seidel is a very popular smoother which is multiplicative and would seem not to parallelize efficiently. For simple discretizations on structured grids, and when memory bandwidth is not a concern, the classical solution of red-black Gauss-Seidel is reasonable. For more complex problems and on modern hardware, &lt;a href=&quot;http://www.columbia.edu/~ma2325/gs.ps&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adams (2001)&lt;/a&gt; shows a much more efficient algorithm. For many problems, the simple approach of using independent Gauss-Seidel on each subdomain is entirely satisfactory. An alternative to Gauss-Seidel is to use damped Jacobi or polynomial smoothers, see &lt;a href=&quot;http://dx.doi.org/10.1016/S0021-9991%2803%2900194-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adams, Brezina, Hu, and Tuminaro (2003)&lt;/a&gt; for a comparison. The performance model for these smoothers is similar to any other stencil computation, and thus have optimal weak scalability, $\mathcal O (N/P)$ for subdomains large enough to cover latency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, coarse grids quickly reach the strong scalability limit (beyond which adding more processes increases run-time), so they should reside on ever smaller MPI communicators. This adds some mild complexity to the implementation. For problems in which the coarse levels have too much structure to continue coarsening, the coarse level solve can become a bottleneck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For testing various parallel multigrid methods, I recommend using a library like &lt;a href=&quot;http://mcs.anl.gov/petsc&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc&lt;/a&gt; which allow you to run many different algorithms with very little user code.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-01-16T16:18:31.473" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="839" PostTypeId="1" CreationDate="2012-01-16T21:11:16.030" Score="-2" ViewCount="2951" Body="&lt;p&gt;I need instructions on how to install a binary package in R.&lt;/p&gt;&#xA;" OwnerUserId="671" LastEditorUserId="3575" LastEditDate="2018-09-10T15:40:39.470" LastActivityDate="2018-09-10T15:40:39.470" Title="How do you install a binary R package?" Tags="|libraries|r|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="840" PostTypeId="2" ParentId="771" CreationDate="2012-01-16T21:31:14.077" Score="12" Body="&lt;p&gt;Just to illustrate what others are saying: the blue below is the Voronoi diagram, the red the dual Delaunay triangulation.  They are dual to one another as geometric plane graphs.  From the Voronoi diagram it is trivial to derive the Delaunay triangulation.  The reverse direction is not so obvious, but it remains true that from the Delaunay triangulation and some calculation you can compute the Voronoi diagram.&#xA;&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;img src=&quot;https://i.stack.imgur.com/OiHSV.jpg&quot; alt=&quot;Vor Diag Del Tri&quot;&gt;&#xA;&lt;br /&gt;&#xA;I computed these diagrams for 50 random points in Mathematica using the &lt;em&gt;ComputationalGeometry&lt;/em&gt; package.  See &lt;a href=&quot;http://cs.smith.edu/~orourke/DCG/#Voronoi&quot; rel=&quot;noreferrer&quot;&gt;this link&lt;/a&gt; for my code.&lt;/p&gt;&#xA;" OwnerUserId="673" LastEditorUserId="673" LastEditDate="2012-01-24T00:28:39.323" LastActivityDate="2012-01-24T00:28:39.323" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="842" PostTypeId="7" CreationDate="2012-01-17T00:15:31.173" Score="0" Body="&lt;p&gt;Computational Science - Stack Exchange is for questions and answers about &lt;strong&gt;computational methods&lt;/strong&gt; used in &lt;strong&gt;technical disciplines&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First-time posters are strongly advised to read &lt;a href=&quot;http://www.catb.org/esr/faqs/smart-questions.html&quot; rel=&quot;nofollow&quot;&gt;Eric Raymond's How to Ask Questions the Smart Way&lt;/a&gt; for advice on how to ask good questions (not for technical answers).&lt;/strong&gt; Although written for an audience asking questions about open source software, the same advice applies when asking questions about any subject area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This site is run by volunteers; the best way to get an answer to your question is to ask it in a way that makes it easy for someone with a broad computational science background to answer it with 15 minutes of effort or less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Topics that are usually a good fit for this site:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Questions about &lt;strong&gt;software packages or languages used in computational science&lt;/strong&gt; (e.g., PETSc, FEniCS, MATLAB, VMD, LAMMPS, Trilinos, LAPACK, SLEPc, R, NumPy, SciPy, Julia, Maple, Octave, GROMACS, CLAWPACK, PyClaw, CVX, YALMIP) &lt;em&gt;except&lt;/em&gt; Mathematica (which has its own site now). &#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Package-specific questions should focus more on features, such as &quot;How do I do (task) in (package)?&quot;, or specific problems in computational science.&lt;/strong&gt; Try to come up with a good minimal example. Pseudocode could be helpful here.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Don't ask us to debug code or look at long blocks of code.&lt;/strong&gt; These questions are better directed to package-specific message boards. It may be possible to rephrase them as questions asking about package features or a specific computational science problem.&lt;/li&gt;&#xA;&lt;li&gt;For guidance on installation questions, see &lt;a href=&quot;http://meta.scicomp.stackexchange.com/a/123/276&quot;&gt;this meta post&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Package developers interested in using this site as a resource should look at &lt;a href=&quot;http://meta.scicomp.stackexchange.com/questions/338/can-scicomp-stackexchange-be-used-for-project-specific-questions&quot;&gt;this meta question&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Package developers answering question should look at &lt;a href=&quot;http://meta.scicomp.stackexchange.com/questions/341/add-software-authorship-disclosures-to-our-badges&quot;&gt;this meta question&lt;/a&gt; for guidelines on disclosing project affiliations.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Questions about &lt;strong&gt;algorithms or methods used to solve problems in applied mathematics&lt;/strong&gt; (e.g., finding roots of a polynomial, finding the eigenvalues/eigenvectors of a matrix, solving an elliptic/parabolic/hyperbolic PDE)&lt;/li&gt;&#xA;&lt;li&gt;Questions that ask about appropriate &lt;strong&gt;methods for a given application area&lt;/strong&gt; (e.g., what numerical methods would I use when modeling shock waves, what numerical methods would I use when modeling combustion, what graph algorithms would I use if I wanted to understand social networks)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;There are certain subjects that will probably get better responses on our sister sites.&lt;/em&gt; If your question is about&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Programming&lt;/strong&gt;, ask on &lt;a href=&quot;http://stackoverflow.com/&quot;&gt;Stack Overflow&lt;/a&gt;, unless it is related to a computational science language (e.g. MATLAB) or requires computational science expertise to understand or answer.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Mathematics&lt;/strong&gt;, ask on &lt;a href=&quot;http://math.stackexchange.com&quot;&gt;math.stackexchange.com&lt;/a&gt; if it is oriented towards proofs and analytical calculation not of a numerical or computational character.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Physics&lt;/strong&gt;, ask on &lt;a href=&quot;http://physics.stackexchange.com/&quot;&gt;http://physics.stackexchange.com/&lt;/a&gt; if it is about physical theories or closed form solutions rather than computational methods.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Statistics&lt;/strong&gt;, ask on &lt;a href=&quot;http://stats.stackexchange.com/&quot;&gt;CrossValidated&lt;/a&gt; if it is oriented more towards data collection, data mining, probability theory, and statistics, and the question is not primarily of numerical or computational character.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Image Processing&lt;/strong&gt;, ask on &lt;a href=&quot;http://dsp.stackexchange.com/&quot;&gt;DSP Stack Exchange&lt;/a&gt; unless it relates to visualizing and processing scientific data, or unless it relates to implementing a specific algorithm for image processing, &lt;em&gt;as long as it is described in terms familiar to general computational scientists&lt;/em&gt; (because most of us are unfamiliar with image processing jargon).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Mathematica&lt;/strong&gt;, ask on &lt;a href=&quot;http://mathematica.stackexchange.com/&quot;&gt;Mathematica Stack Exchange&lt;/a&gt; if it is about using Mathematica to solve problems.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Questions about software bugs&lt;/strong&gt; are generally best asked on project e-mail lists rather than on this site.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;There are certain types of questions that will probably be closed.&lt;/em&gt; More complete guidelines can be found in the next section.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="276" LastEditDate="2013-06-03T07:12:19.737" LastActivityDate="2013-06-03T07:12:19.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="843" PostTypeId="2" ParentId="660" CreationDate="2012-01-17T04:34:11.577" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://www.iospress.nl/journal/scientific-programming/&quot; rel=&quot;nofollow&quot;&gt;Scientific Programming&lt;/a&gt; is an option that allows for a focus on the programming techniques employed in writing scientific software.  (Disclosure: I'm on the Editorial Advisory Board.)&lt;/p&gt;&#xA;" OwnerUserId="688" LastEditorUserId="123" LastEditDate="2012-07-15T20:08:06.880" LastActivityDate="2012-07-15T20:08:06.880" CommentCount="2" CommunityOwnedDate="2012-01-17T04:34:11.577" ContentLicense="CC BY-SA 3.0" />
  <row Id="844" PostTypeId="1" AcceptedAnswerId="850" CreationDate="2012-01-17T06:47:48.443" Score="17" ViewCount="2525" Body="&lt;p&gt;As far as I am aware, multigrid solvers use iterative smoothers such as Jacobi, Gauss-Seidel, and SOR to dampen the error at various frequencies.  Could a Krylov subspace method (like conjugate gradient, GMRES, etc.) be utilized instead?  I don't think they are classified as &quot;smootheners&quot;, but they can be used to approximate the coarse grid solution.  Can we expect to see analogous convergence to the solution as we would in a standard multigrid method? Or is it problem dependent?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-17T14:09:12.763" LastActivityDate="2012-01-17T15:13:18.533" Title="Can a Krylov subspace method be used as a smoother for multigrid?" Tags="|linear-algebra|multigrid|iterative-method|krylov-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="845" PostTypeId="2" ParentId="660" CreationDate="2012-01-17T09:19:30.457" Score="4" Body="&lt;p&gt;The mathematical Optimization Society recently launched the Journal&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mathematical Programming Computation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From their &quot;Aims and Scope&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Mathematical Programming Computation (MPC) publishes original research articles covering computational issues in mathematical optimization. Articles report on innovative software, comparative tests, modeling environments, libraries of data, and/or applications. A main feature of the journal is the inclusion of accompanying software and data with submitted manuscripts.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Concerning the topics, they write:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Topics covered in MPC include linear programming, convex optimization, nonlinear optimization, stochastic optimization, robust optimization, integer programming, combinatorial optimization, global optimization, network algorithms, and modeling languages.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="302" LastActivityDate="2012-01-17T09:19:30.457" CommentCount="0" CommunityOwnedDate="2012-01-17T09:19:30.457" ContentLicense="CC BY-SA 3.0" />
  <row Id="846" PostTypeId="1" AcceptedAnswerId="848" CreationDate="2012-01-17T10:33:39.027" Score="11" ViewCount="1586" Body="&lt;p&gt;I am looking for reasonably fast implementations of the discrete Fourier transform (DFT) on a 2D triangular or hexagonal lattice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate pointers to such implementations (especially ones easily usable from Python or Mathematica), and also to descriptions of how to reduce this problem to the 1D DFT, which is already built into many systems.&lt;/p&gt;&#xA;" OwnerUserId="605" LastActivityDate="2012-01-17T11:59:44.693" Title="Library for Fourier transform on triangle lattice" Tags="|libraries|fourier-analysis|" AnswerCount="1" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="847" PostTypeId="2" ParentId="593" CreationDate="2012-01-17T11:31:09.960" Score="8" Body="&lt;p&gt;If you are interested in software I can recommend the NFFT library (in C with an interface to MATLAB) which can be found &lt;a href=&quot;http://www-user.tu-chemnitz.de/~potts/nfft/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Note that there is also a PFFT library for parallel FFT computation and even a PNFFT library for parallel non-equispaces FFTs &lt;a href=&quot;https://github.com/mpip/pnfft&quot; rel=&quot;nofollow noreferrer&quot;&gt;by the same developers&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="302" LastEditDate="2018-12-17T13:07:46.937" LastActivityDate="2018-12-17T13:07:46.937" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="848" PostTypeId="2" ParentId="846" CreationDate="2012-01-17T11:37:53.347" Score="6" Body="&lt;p&gt;There are several papers by Markus Püschel on his web site &lt;a href=&quot;http://www.ece.cmu.edu/~smart/papers/triangle.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; that discuss Cooley-Tukey-like (so I'm guessing &quot;fast&quot;) algorithms for lattice transforms, such as DFTs on triangular and hexagonal 2-D lattices. In the triangular case, he calls the DFT the discrete triangle transform (DTT). Markus has a code called &lt;a href=&quot;http://www.spiral.net/&quot; rel=&quot;noreferrer&quot;&gt;SPIRAL&lt;/a&gt; that automatically generates code for transforms, but it appears that this DTT work is not part of SPIRAL, and there is no implementation on his web site that I can find. I'm beginning to think that @J.M. is right and that you might need to roll your own implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that the abstracts note is that for 2-D triangular and hexagonal lattices, the transform is not separable into 1-D components, so you won't be able to reduce the problem to two 1-D transforms.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-17T11:59:44.693" LastActivityDate="2012-01-17T11:59:44.693" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="850" PostTypeId="2" ParentId="844" CreationDate="2012-01-17T13:56:00.750" Score="19" Body="&lt;p&gt;Yes, you can, but Krylov methods generally do not have great smoothing properties. This is because they target the whole spectrum in an adaptive way that minimizes the residual or a suitable norm of the error. This will generally include some low frequency (long wavelength) modes that the coarse grids would have handled fine. Krylov smoothers also make the multigrid cycle nonlinear, so if multigrid is being used as a preconditioner for an outer Krylov method, the outer method should be &quot;flexible&quot; (e.g. GCR or FGMRES).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Krylov smoothers also greatly increases the number of dot products that must be computed, which becomes a significant bottleneck in parallel. However, even with these unattractive properties, Krylov smoothers are sometimes useful, especially for difficult problems in which good interpolation operators are not available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more popular alternative is to use polynomial smoothers (usually Chebyshev). These methods target a specified portion of the spectrum. For symmetric elliptic PDEs (where the discrete operator is symmetric positive definite or nearly so), it is common to estimate the largest eigenvalue $\lambda_\max$ of $D^{-1}A$ where $D^{-1}$ is the point-block Jacobi preconditioner for $A$ and target a range like $(0.1 \lambda_{\max}, 1.1 \lambda_{\max})$. Polynomial smoothers have no reductions and are linear operations (for any chose polynomial degree, usually chosen between $1$ and perhaps $5$). Usually a few iterations (say $5$ to $10$) of GMRES or CG are used to estimate $\lambda_\max$, so the user does not need to compute these themselves. The estimate of $\lambda_\max$ is also used by some algebraic multigrid methods to choose coarsening strategies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1016/S0021-9991%2803%2900194-3&quot;&gt;Adams, Brezina, Hu, and Tuminaro (2003)&lt;/a&gt; is a nice paper on parallel and algorithmic performance of polynomial smoothers. Note that polynomial smoothers tend to be less effective (and/or difficult to formulate) for non-symmetric problems, in which case you will likely want to use Gauss-Seidel or more sophisticated (block/distributed) relaxation schemes.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-01-17T15:13:18.533" LastActivityDate="2012-01-17T15:13:18.533" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="851" PostTypeId="1" AcceptedAnswerId="852" CreationDate="2012-01-17T14:41:57.860" Score="11" ViewCount="7817" Body="&lt;p&gt;As I understand it, successive over relaxation works by choosing a parameter $0\leq\omega\leq2$ and using a linear combination of a (quasi) Gauss-Seidel iteration and the value at the previous timestep... that is  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;${u}^{k+1} = (\omega){u_{gs}}^{k+1} + (1-\omega)u^{k}$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I state 'quasi' because ${u_{gs}}^{k+1}$ includes the latest information updated according to this rule, at any timestep.  (note that at $\omega=1$, this is exactly gauss-seidel).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, I have read that on optimal choice for $\omega$ (such that the iteration converges faster than any other) approaches 2 for the poisson problem as the spatial resolution approaches zero.  Does a similar trend exist for other symmetric, diagonally dominant problems?  That is, is there a way to choose omega optimally without embedding it into an adaptive optimization scheme? Are there other heuristics for other types of problems?  What kinds of problems would under-relaxation ($\omega&amp;lt;1$) be optimal?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="119" LastEditDate="2012-01-24T23:31:01.807" LastActivityDate="2012-01-24T23:31:01.807" Title="Are there any heuristics for optimizing the successive over-relaxation (SOR) method?" Tags="|linear-algebra|optimization|iterative-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="852" PostTypeId="2" ParentId="851" CreationDate="2012-01-17T17:06:38.207" Score="13" Body="&lt;h3&gt;Damped Jacobi&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Suppose the matrix $A$ has diagonal $D$. If the spectrum of $D^{-1}A$ lies in the interval $[a,b]$ of the positive real axis, then the iteration matrix of Jacobi with damping factor $\omega$&#xA;$$B_\text{Jacobi} = I - \omega D^{-1} A$$&#xA;has spectrum in the range $[1 - \omega b,1 - \omega a]$, so minimizing the spectral radius with $$\omega_{\text{opt}} = \frac 2 {a + b}$$ gives a convergence factor of&#xA;$$\rho_\text{opt} = 1 - \frac{2a}{a+b} = \frac{b-a}{a+b}.$$&#xA;If $a \ll b$, then this convergence factor is very poor, as expected. Note that it is relatively easy to estimate $b$ using a Krylov method, but quite expensive to estimate $a$.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Successive over-relaxation (SOR)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ams.org/journals/tran/1954-076-01/S0002-9947-1954-0059635-7/S0002-9947-1954-0059635-7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Young (1950)&lt;/a&gt; proved an optimal result for SOR applied to matrices with so-called &lt;em&gt;Property A&lt;/em&gt;, &lt;em&gt;consistent ordering&lt;/em&gt;, and positive real eigenvalues of $D^{-1}A$. Given a maximal eigenvalue $\mu_\max$ of the undamped Jacobi iteration matrix $I - D^{-1} A$ ($\mu_\max &amp;lt; 1$ is guaranteed by the assumptions in this case), the optimal damping factor for SOR is&#xA;$$ \omega_\text{opt} = 1 + \left( \frac{\mu_\max}{1 + \sqrt{1 - \mu_\max^2}} \right)^2$$&#xA;which results in a convergence rate of&#xA;$$\rho_\text{opt} = \omega_\text{opt} - 1.$$&#xA;Note that $\omega_\text{opt}$ approaches 2 when $\mu_\max \to 1$.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Comments&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It's not 1950 any more and it really doesn't make sense to use stationary iterative methods as solvers. Instead, we use them as smoothers for multigrid. In this context, we only care to target the upper end of the spectrum. Optimizing the relaxation factor in SOR causes SOR to produce very little damping of high frequencies (in exchange for better convergence on lower frequencies), so it is usually better to use standard Gauss-Seidel, corresponding to $\omega = 1$ in SOR. For nonsymmetric problems and problems with highly variable coefficients, under-relaxed SOR ($\omega &amp;lt;1$) may have better damping properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Estimating both eigenvalues of $D^{-1}A$ is expensive, but the largest eigenvalue can be estimated quickly using a few Krylov iterations. Polynomial smoothers (preconditioned with Jacobi) are more effective than multiple iterations of damped Jacobi and are easier to configure, so they should be preferred. See &lt;a href=&quot;https://scicomp.stackexchange.com/a/850/119&quot;&gt;this answer&lt;/a&gt; for more on polynomial smoothers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is sometimes claimed that SOR should not be used as a preconditioner for Krylov methods such as GMRES. This comes from the observation that the optimal relaxation parameter should place all eigenvalues of the iteration matrix $$B_\text{SOR} = 1 - \left(\frac 1 \omega D + L\right)^{-1} A$$ on a circle centered at the origin. The spectrum of the preconditioned operator $(\frac 1 \omega D + L)^{-1} A$ has eigenvalues on a circle of the same radius, but centered on 1. For poorly conditioned operators, the radius of the circle is quite close to 1, so GMRES sees eigenvalues close to the origin at a range of angles, which is usually not good for convergence. In practice, GMRES may converge reasonably when preconditioned with SOR, especially for problems that are already fairly well conditioned, but other preconditioners are often more effective.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-17T17:57:32.743" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="853" PostTypeId="2" ParentId="669" CreationDate="2012-01-17T20:12:26.743" Score="2" Body="&lt;p&gt;I hadn't thought about it too closely, but I think the basic algorithm would work like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Find the number of unique edges in either the $x$ or $y$ directions. &lt;/li&gt;&#xA;&lt;li&gt;Take the direction that has the smaller number of edges. Without loss of generality, we assume $x$ here.&lt;/li&gt;&#xA;&lt;li&gt;For each such edge, connecting $(x_{min},y_1)$ to $(x_{min},y_2)$, draw the largest possible rectangle, extending to terminate at the line $x = x_{max}$.&lt;/li&gt;&#xA;&lt;li&gt;Repeat this process for all edges on the &quot;other&quot; side (e.g., move rightwards) that have not yet been covered with a rectangle.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This should cover the whole domain, but with the possibility of overlaps as one starts from either the left or the right. However, the number of unique edges in either the $x$ or $y$ direction acts as an upper bound on the number of rectangles. &lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-01-17T20:12:26.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="854" PostTypeId="5" CreationDate="2012-01-17T20:37:05.740" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-17T20:37:05.740" LastActivityDate="2012-01-17T20:37:05.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="855" PostTypeId="4" CreationDate="2012-01-17T20:37:05.740" Score="0" Body="Successive over-relaxation, an iterative method for solving systems of linear equations closely related to the Gauss-Seidel method." OwnerUserId="549" LastEditorUserId="9" LastEditDate="2012-01-18T07:18:00.530" LastActivityDate="2012-01-18T07:18:00.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="856" PostTypeId="2" ParentId="770" CreationDate="2012-01-17T21:39:50.237" Score="5" Body="&lt;p&gt;How about;&#xA;Watson, D.F. Computing the n-dimensional Delaunay tessellation with application to Voronoi polytopes. Comput J 24, 167–172 (1981).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://comjnl.oxfordjournals.org/content/24/2/167.abstract&quot; rel=&quot;nofollow&quot;&gt;http://comjnl.oxfordjournals.org/content/24/2/167.abstract&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understood it, it should be $\mathcal{O}\left(n^{\frac{2p-1}{n}}\right)$, isn't that better then $Ω\left(n^{p/2}\right)$?&lt;/p&gt;&#xA;" OwnerUserId="700" LastEditorUserId="700" LastEditDate="2012-01-17T21:48:46.497" LastActivityDate="2012-01-17T21:48:46.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="857" PostTypeId="2" ParentId="771" CreationDate="2012-01-18T02:26:13.150" Score="13" Body="&lt;p&gt;The simple answer is that they are dual because for every delaunay triangulation there exists one and only one corresponding voronoi tessellation and vise versa. Thats true for most cases, but there are cases were the correspondence is not one to one. For example in the case when the voronoi tessellation is a regular square grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both the voronoi tessellation and the delaunay triangulation are non-trivial to calculate for a given set of points. But once you have found one the other one is easy to find.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the delaunay triangulation of a set of points, $P$, all triangles are &quot;delaunay&quot;, meaning that there are no points inside the circumcircle corresponding to any of the triangles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The voronoi tessellation for a set of points, $P$, consists of the set of voronoi cells $R$, such that for every point in $R_i$ are closer to $P_i$ then to any other point in $P$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the delaunay triangulation simply connect the neighboring triangles circumcircle centers.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the set of points $P$ and the voronoi tessellation simply connect neighboring cells points. This is of course given that you know the set of point $P$ used when constructing the voronoi tessellation.&lt;/p&gt;&#xA;" OwnerUserId="700" LastActivityDate="2012-01-18T02:26:13.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="858" PostTypeId="1" AcceptedAnswerId="895" CreationDate="2012-01-18T06:33:36.627" Score="9" ViewCount="3551" Body="&lt;p&gt;I have a list of vertical cut planes, and I have a polygonal mesh ( it's a 2D+0.5D mesh, something like a 2D mesh with an extra dimension, $z$ attached to each point). One can assume that the mesh contains vertices $V$ , edges $E$ and face $F$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mesh is a real life 3D terrain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, given a list of vertical cut planes, I want to find all the intersection points ( to form a polyline) between the cut planes and the polygonal mesh, in order to build a vertical view of the terrain for each of the cut plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, my approach is that I use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Bounding_volume&quot; rel=&quot;noreferrer&quot;&gt;Aabb tree&lt;/a&gt; to test each of the mesh triangles against the cut section, and then compute the intersection points subsequently for the cut section and the triangle that are within each other's Aabb tree range. But this approach doesn't make use of the mesh structure of the terrain, and I think we can significantly speed up the computation if we can make use of the mesh structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more efficient algorithm I can use for this purpose? &lt;/p&gt;&#xA;" OwnerUserId="690" LastEditorUserId="123" LastEditDate="2012-01-18T11:59:38.580" LastActivityDate="2012-01-22T10:56:12.720" Title="How to efficiently determine the intersection of a vertical cutting plane with a mesh" Tags="|computational-geometry|mesh-traversal|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="860" PostTypeId="2" ParentId="839" CreationDate="2012-01-18T11:44:09.897" Score="4" Body="&lt;p&gt;Copied from a comment I made above during discussions about topicality:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Detailed instructions for installing packages can be found on the &lt;a href=&quot;http://cran.r-project.org/&quot; rel=&quot;nofollow&quot;&gt;Comprehensive R Archive Network (CRAN) web site&lt;/a&gt; on &lt;a href=&quot;http://cran.r-project.org/doc/manuals/R-admin.html#Installing-packages&quot; rel=&quot;nofollow&quot;&gt;this page&lt;/a&gt;. So far as I can tell, there is no such thing as an &quot;executable R package&quot;; there are binary R packages.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-18T11:44:09.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="861" PostTypeId="1" CreationDate="2012-01-18T12:49:07.500" Score="19" ViewCount="4807" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Adaptive_mesh_refinement&quot;&gt;Adaptive mesh refinement&lt;/a&gt; (AMR) is a common technique for dealing with the problem of widely varying spatial scales in the numerical solution of PDEs.  What general-purpose libraries exist for AMR on structured grids?  Ideally I'd like something in the spirit of PETSc, where the library handles just the adaptive meshes and I provide the physics and discretization (finite difference/volume/element).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ideal library would be&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Modular&lt;/strong&gt;: doesn't dictate how I write my code or too much of my data structures&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;General&lt;/strong&gt;: doesn't care what kind of discretization I'm using&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Efficient&lt;/strong&gt;: doesn't incur too much overhead&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Parallel&lt;/strong&gt; and highly scalable&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Libraries that fit only a subset of these criteria would still be of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum&lt;/strong&gt;: I am aware of &lt;a href=&quot;http://math.boisestate.edu/~calhoun/www_personal/research/amr_software/index.html&quot;&gt;Donna Calhoun's extensive list of AMR packages&lt;/a&gt;, but I don't know which of them (if any) fit the criteria above.  So I'm mainly interested in hearing from people who have actual experience with one or (better yet) more packages, as to how they measure up in those terms.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-01-18T18:53:59.967" LastActivityDate="2012-03-02T09:17:16.810" Title="Is there a general-purpose library for structured grid adaptive mesh refinement?" Tags="|pde|libraries|parallel-computing|adaptive-mesh-refinement|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="862" PostTypeId="2" ParentId="861" CreationDate="2012-01-18T14:00:42.860" Score="14" Body="&lt;p&gt;One library to consider is &lt;a href=&quot;https://ccse.lbl.gov/BoxLib/index.html&quot;&gt;BoxLib&lt;/a&gt;.  Its key features (from the website) are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&#xA;&lt;li&gt;Support for block-structured AMR with optional subcycling in time&#xA;&lt;li&gt;Support for cell-centered, face-centered and node-centered data&#xA;&lt;li&gt;Support for hyperbolic, parabolic and elliptic solves on hierarchical grid structure&#xA;&lt;li&gt;C++ and Fortran90 versions&#xA;&lt;li&gt;Supports hybrid programming model with MPI and OpenMP&#xA;&lt;li&gt;Basis of mature applications in combustion, astrophysics, cosmology, and porous media&#xA;&lt;li&gt;Demonstrated scaling to over 200,000 processors&#xA;&lt;li&gt;Freely available to interested user&#xA;&lt;/li&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a &lt;a href=&quot;https://github.com/memmett/pyboxlib&quot;&gt;Python wrapper&lt;/a&gt; (written by me) to the Fortran version included (although it is fairly young).&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="123" LastEditDate="2012-01-26T12:13:54.360" LastActivityDate="2012-01-26T12:13:54.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="863" PostTypeId="2" ParentId="861" CreationDate="2012-01-18T14:12:45.283" Score="9" Body="&lt;p&gt;You should also look at &lt;a href=&quot;http://libmesh.sourceforge.net/&quot;&gt;libMesh&lt;/a&gt;. It's targeted at finite element methods, but other than that, I think it checks most of your boxes. Unlike BoxLib, it's a fully unstructured, mixed element type library, which is to stay that it supports tets, pyramids, prisms, and hexahedra in the same mesh. It also has one of the largest sets of integration rules for high-order polynomial basis functions around. It's set up to let you call PETSc (and some other libraries as well) directly, so you have the same solver scalability that PETSc does. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's certainly a libMesh way of doing things, but there's a PETSc way of doing things, too. So hopefully that won't scare you off.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-01-18T14:12:45.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="864" PostTypeId="2" ParentId="861" CreationDate="2012-01-18T16:19:40.843" Score="2" Body="&lt;p&gt;You didnt specify structured or unstructured.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at Paramesh, Pyramid, p4est, Dendro, Samrai and Chombo.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw Pyramid doesnt do coarsening.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2012-01-18T16:19:40.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="865" PostTypeId="1" AcceptedAnswerId="869" CreationDate="2012-01-18T17:10:37.290" Score="5" ViewCount="603" Body="&lt;p&gt;I have a linear system whose matrix that is diagonally dominant, non-symmetric, but banded.   Since the band-radius is 2 (producing only 5 variables per equation), a banded direct solver (gaussian elimination without pivoting) should be fairly fast to produce a solution.  I was just wondering if there is a way to parallelize a banded gaussian elimination?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-01-18T20:43:05.013" Title="How to parallelize a banded direct solver?" Tags="|linear-algebra|algorithms|parallel-computing|banded-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="866" PostTypeId="1" AcceptedAnswerId="868" CreationDate="2012-01-18T17:27:13.630" Score="18" ViewCount="2276" Body="&lt;p&gt;I'm trying to solve a 2D Poisson equation by finite differences.  In the process, I obtain a sparse matrix with only $5$ variables in each equation. For example, if the variables were $U$, then the discretization would yield:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U_{i-1,j} + U_{i+1,j} -4U_{i,j} + U_{i,j-1} + U_{i,j+1} = f_{i,j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that I can solve this system by an iterative method, but the thought occurred to me that if I ordered the variables appropriately, I might be able to obtain a banded matrix which could be solved by a direct method (i.e., Gaussian elimination w/o pivoting).  Is this possible?  Are there any strategies for doing this for other, perhaps less structured sparse systems?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="20417" LastEditDate="2018-03-15T16:48:35.903" LastActivityDate="2020-04-21T19:43:50.573" Title="How to reorder variables to produce a banded matrix of minimum bandwidth?" Tags="|linear-algebra|pde|finite-difference|sparse-matrix|banded-matrix|" AnswerCount="6" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="867" PostTypeId="2" ParentId="865" CreationDate="2012-01-18T17:28:52.380" Score="1" Body="&lt;p&gt;Cyclic reduction algorithm should be feasible. See for example &lt;a href=&quot;http://www.cse.illinois.edu/courses/cs554/notes/09_tridiagonal_8up.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cse.illinois.edu/courses/cs554/notes/09_tridiagonal_8up.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-01-18T17:28:52.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="868" PostTypeId="2" ParentId="866" CreationDate="2012-01-18T18:26:31.120" Score="13" Body="&lt;p&gt;This is a well-studied problem in the field of sparse-direct solvers. I highly recommend reading Joseph Liu's &lt;a href=&quot;http://www.jstor.org/pss/2132786&quot; rel=&quot;nofollow&quot;&gt;overview of the multifrontal method&lt;/a&gt; in order to get a better idea of how reorderings and supernodes effect fill-in and solution time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nested dissection is an extremely common way to generate the reordering, and essentially consists of recursive graph partitioning. &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/views/metis&quot; rel=&quot;nofollow&quot;&gt;MeTiS&lt;/a&gt; is the de facto standard for graph partitioning, and you can read about some of the ideas behind it &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/node/107&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. Another commonly used package is &lt;a href=&quot;http://www.labri.fr/perso/pelegrin/scotch/&quot; rel=&quot;nofollow&quot;&gt;SCOTCH&lt;/a&gt;, and &lt;a href=&quot;http://www.sandia.gov/~bahendr/chaco.html&quot; rel=&quot;nofollow&quot;&gt;Chaco&lt;/a&gt; is also important, as its authors introduced &lt;a href=&quot;http://www.sandia.gov/~bahendr/abstracts/multilevel.html&quot; rel=&quot;nofollow&quot;&gt;multi-level graph partitioning&lt;/a&gt;, which is also &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/node/78&quot; rel=&quot;nofollow&quot;&gt;the fundamental idea behind MeTiS&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;George and Liu showed &lt;a href=&quot;http://www.cse.illinois.edu/courses/cs598mh/george_liu.pdf&quot; rel=&quot;nofollow&quot;&gt;in their classic book&lt;/a&gt;  that 2d sparse-direct solutions only require $O(n^{3/2})$ work and $O(n \log n)$ memory, while 3d sparse-direct requires $O(n^2)$ work and $O(n^{4/3})$ memory.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-08-27T16:18:07.573" LastActivityDate="2012-08-27T16:18:07.573" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="869" PostTypeId="2" ParentId="865" CreationDate="2012-01-18T19:16:57.767" Score="6" Body="&lt;p&gt;From what I have seen from &lt;a href=&quot;https://scicomp.stackexchange.com/questions/866/how-to-reorder-variables-to-produce-a-banded-matrix-of-minimum-bandwidth&quot;&gt;your other question&lt;/a&gt;, your half-bandwidth is not actually two, as one of your diagonals is roughly $\sqrt{n}$ entries away from the main diagonal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, even if you really did have an $O(1)$ bandwidth, I would have pointed you to the general literature on parallel sparse-direct solvers, as $O(1)$ bandwidth can be thought of as a special case where the separators for nested-dissection can be trivially found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, I recommend chapter 4 of Anshul Gupta's dissertation, &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.46.2932&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analysis and Design of Scalable Parallel Algorithms for Scientific Computing&lt;/a&gt;. I found that chapter, along with &lt;a href=&quot;http://www.jstor.org/pss/2132786&quot; rel=&quot;nofollow noreferrer&quot;&gt;Liu's review paper&lt;/a&gt;, sufficient to implement a scalable sparse-direct $LDL^T$ factorization without pivoting. I would start with those papers before looking into adding pivoting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the machinery for the general sparse-direct case essentially reduces to the cyclic reduction algorithm that Heath discusses in @Allan P. Engsig-Karup's answer.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-18T20:43:05.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="870" PostTypeId="1" AcceptedAnswerId="944" CreationDate="2012-01-18T19:24:53.597" Score="4" ViewCount="1236" Body="&lt;p&gt;For a synchronized ODE system,I wanted to know if there is a program code in MATLAB available for plotting the &lt;a href=&quot;http://en.wikipedia.org/wiki/Lyapunov_exponent#Conditional_Lyapunov_exponent&quot; rel=&quot;nofollow&quot;&gt;conditional Lyapunov exponent&lt;/a&gt;.  For example, synchronization of identical Rossler system with a coupling term $F$ present in both the response and the driver.The $F$ is not a coefficient of any variable. This is where I have a problem.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do we have to include the coupling in the jacobian matrix and if yes how? &lt;/li&gt;&#xA;&lt;li&gt;How to plot the Lyapunov exponent as a function of different values of $F$? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help.&lt;/p&gt;&#xA;" OwnerUserId="711" LastEditorUserId="123" LastEditDate="2012-01-23T05:33:39.760" LastActivityDate="2016-07-05T05:49:17.773" Title="How to find conditional Lyapunov exponents" Tags="|ode|matlab|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="871" PostTypeId="2" ParentId="866" CreationDate="2012-01-18T20:57:20.810" Score="3" Body="&lt;p&gt;There's an algorithm called ADI (Alternating Direction Implicit) in applied math circles and Split-operator in physics circles that does basically what you describe. It's an iterative method, and it follows this basic procedure:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For every value of $y$ , relax in the $x$-direction. This matrix should be tridiagonal, so it can be solved directly in relatively little time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For every value of $x$ , relax in the $y$-direction. Again, this should be pretty quick.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat 1 and 2 until the error is as small as you want it to be.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I don't know the formal complexity of this algorithm, but I've found it to converge in fewer iterations than things like Jacobi and Gauss-Seidel every time I've used it.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-19T00:50:23.270" LastActivityDate="2012-01-19T00:50:23.270" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="872" PostTypeId="1" CreationDate="2012-01-18T21:45:32.603" Score="8" ViewCount="315" Body="&lt;p&gt;What numerical methods are available for finding the fixed point of an operator $A$ that is acting on functions $f : [a,b] \rightarrow [a,b]$?  I am looking for the function $f$ for which $Af = f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Essential details:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My function $f$ is actually a probability density function of a distribution on the circle (i.e. on the periodic internal $[0,2\pi)$).  This also means that $0 \le f(x) \le 1$ for any $x \in [0,2\pi)$, which makes the problem easier.  Let's say we have this function sampled with some density and we can compute the operator $A$ numerically.  The distribution is not necessarily continuous, i.e. it might look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QffdO.png&quot; alt=&quot;Discontinuous distribution&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for methods and suggestions on how to do this.  I am going to try to implement this in Mathematica first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;All messy details of my practical problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how I actually compute the operator $A$:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I take distribution which I have as a large number of samples drawn from it&lt;/li&gt;&#xA;&lt;li&gt;I estimate the &lt;a href=&quot;http://en.wikipedia.org/wiki/Probability_density_function&quot; rel=&quot;noreferrer&quot;&gt;PDF&lt;/a&gt; and convolve it with a &quot;square&quot; kernel (&quot;square&quot; means it's similar to the figure above).  This gives me a rather smooth function.&lt;/li&gt;&#xA;&lt;li&gt;I threshold the function to get another binary-valued function $g:[a,b] \rightarrow {0,1}$.&lt;/li&gt;&#xA;&lt;li&gt;From this binary-valued function I compute a large number of samples drawn from the distribution again&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I mentioned this because it means that instead of working with PFDs, we could also work with the smooth function (after the convolution) or the binary-valued function.  Actually I was working with the binary valued function in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that fixed points of plain real functions (not operators) can be &quot;attracting&quot; or &quot;repulsing&quot;, i.e. when applying the function repeatedly to a number, it either converges to a fixed point, or is repulsed by it.  I do not know what type the fixed point of my operator is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that my operator &lt;em&gt;always&lt;/em&gt; has the uniform distribution as a fixed point, but sometimes (depending on my parameters) it might have another one.  It is this other one I need to find.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did try applying the operator repeatedly to see what happens, and sometimes it converges to the fixed point I want.  But even if it doesn't converge, a fixed point may still exists (or it may only converge to that fixed point if I use a different initial condition).  So I need a more robust method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Even more messy details:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually my distributions are not on $[0,2\pi)$, but on a periodic interval $[0,a)$.  The fixed point will exist only for a certain value of $a$.  When I apply the operator repeatedly, and $a$ is a little off from the right value, the distribution appears to &quot;rotate&quot; on the periodic interval.  By measuring the amount of rotation, I can correct for the value of $a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why did I mention this complex (seemingly too localized) detail, that's not essential to the question?  Because when you read that the distribution is on a periodic interval, you will notice that if one distribution is a fixed point, any rotation of that distribution on the interval will also be a fixed point.  This may get you thinking if numerical errors in computing the application of the operator to the fixed point will cause the fixed point to appear to &quot;rotate&quot; slightly.  I can compensate for this, so this is not a problem.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="123" LastEditDate="2012-01-19T06:15:39.657" LastActivityDate="2012-01-19T09:48:36.673" Title="Finding the fixed point of an operator" Tags="|inverse-problem|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="873" PostTypeId="1" AcceptedAnswerId="880" CreationDate="2012-01-18T22:02:39.720" Score="7" ViewCount="555" Body="&lt;p&gt;I can see that there are multiple heuristics to achieve a matrix with minimum bandwidth.  As heuristics, they can't guarantee an optimal solution in polynomial time (after all, the problem is NP-complete). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder, then, for the finite difference &lt;a href=&quot;https://scicomp.stackexchange.com/questions/866/how-to-reorder-variables-to-produce-a-banded-matrix-of-minimum-bandwidth&quot;&gt;discretization of the Poisson problem&lt;/a&gt;, how does the bandwidth expand as the problem size increases? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the bandwidth approach the number of gridpoints as the spatial resolution decreases?  Or does it remain constant?  Which approaches produce the smallest bandwidth (asymptotically)?  If the cost is too great to produce a minimum bandwidth, then which algorithms balance the computational cost with the width cost?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-12-28T19:45:15.107" Title="What heuristics can be used to minimize the asymptotic matrix bandwidth of a 5-point Laplacian discretization?" Tags="|pde|finite-difference|complexity|banded-matrix|heuristics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="874" PostTypeId="1" CreationDate="2012-01-18T23:36:36.340" Score="8" ViewCount="3060" Body="&lt;p&gt;I faintly recall from my early &quot;numerics&quot; lectures that iterative linear solvers for $Ax=b$ often require that when $A$ is decomposed as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A=D + M$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where D is a diagonal matrix and $M$ has zero diagonal, &#xA;the elements of $D$ should be dominant over the entries in $M$ for iterative solvers to perform well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if that's not the case and the entries of $D$ become really small?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use a direct solver then? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, the linear system I want to solve involves a matrix &#xA;$$A(\omega) = D(\omega) + M$$&#xA;where the non-diagonal part is constant but the diagonal part depends on a parameter $\omega$ in some non-trivial way. So far, I don't see a way around solving $A(\omega) x = b$ for each $\omega$ anew.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The diagonal entries are of the form $A_{jj} = \omega + z_j + i\eta$ where $z_j$ is some real number that depends on the row we are in whereas $\eta$ is a very small convergence factor and $i$ is the imaginary unit. Could this lead to numerical instabilities when $\omega + z \approx 0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Well, maybe one more thing about the nature of $A(\omega)$: If one sets $\eta$ to $0$ exactly, then $A(\omega)$ is guaranteed to have poles. This is because ultimately I use this matrix to compute (many-body) Green's functions in frequency domain, and those need a convergence factor $\eta$ to move their poles off the real axis. The sum of absolute values of off-diagonal matrix elements in each row is $10$ at most, but the diagonal will always have some entries whose real part is very close or equal to zero. &lt;/p&gt;&#xA;" OwnerUserId="105" LastEditorUserId="105" LastEditDate="2012-01-20T07:04:36.197" LastActivityDate="2012-01-20T07:04:36.197" Title="Solving a non-symmetric non-diagonally dominant sparse system the best way" Tags="|linear-algebra|sparse-matrix|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="875" PostTypeId="2" ParentId="872" CreationDate="2012-01-19T00:49:14.837" Score="3" Body="&lt;p&gt;This is a classic example of what is known as an &lt;em&gt;inverse problem&lt;/em&gt;. You want to find a function $f$ that reproduces known behavior, rather than asking what behavior is produced by $f$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach for doing this is to use an approach like &lt;a href=&quot;http://pre.aps.org/abstract/PRE/v52/i4/p3730_1&quot; rel=&quot;nofollow&quot;&gt;reverse Monte Carlo&lt;/a&gt;, developed by Lyubartsev and Laaksonen. What they did was to try to find a potential $u(r)$ that reproduced a known radial distribution function $g(r)$. What you are looking to do is to find a function $f$ that reproduces ${\mathcal A}f = f$, so while the entirety of the approach may not be suitable, you will want to pay special attention to the iteration procedure they use—that is, how do they go from guess $u^{(k)}(r)$ to guess $u^{(k+1)}(r)$.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-01-19T09:48:36.673" LastActivityDate="2012-01-19T09:48:36.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="876" PostTypeId="2" ParentId="874" CreationDate="2012-01-19T01:06:52.073" Score="3" Body="&lt;p&gt;A diagonally dominant matrix is guaranteed to have either all positive (if the entries of the diagonal are all positive) or all negative (if the entries are all negative) eigenvalues, by Gershgorin's theorem.  Most iterative methods only work if the eigenvalues of the iteration matrix are in a particular region of the complex plane, so diagonal dominance ensures that all of the eigenvalues have either a stricly positive or strictly negative real part (or that all the eigenvalues lie within a particular radius of some number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your iteration matrix has eigenvalues which lie outside the prescribed region for a particular method, it usually won't work correctly. There are piles of methods out there, but I can't pick a specific method out without knowing more about the spectrum of $A(\omega)$.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="123" LastEditDate="2012-01-19T17:52:52.013" LastActivityDate="2012-01-19T17:52:52.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="877" PostTypeId="2" ParentId="866" CreationDate="2012-01-19T04:45:39.057" Score="4" Body="&lt;p&gt;Speaking of multifrontal methods, &lt;a href=&quot;http://www.cise.ufl.edu/~davis/&quot; rel=&quot;nofollow&quot;&gt;Tim Davis&lt;/a&gt;, who works on multifrontal methods for LU factorization (&lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/umfpack&quot; rel=&quot;nofollow&quot;&gt;UMFPACK&lt;/a&gt;) has a number of routines that will reorder matrices to minimize fill-in. You can find them as &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; as part of &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/SuiteSparse/&quot; rel=&quot;nofollow&quot;&gt;SuiteSparse&lt;/a&gt;. SuiteSparse uses MeTiS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One other thing to note: In some problems, you can be clever about ordering variables so that you get banded, or close to banded, patterns, which can save you the trouble (and the CPU time) of calling these algorithms. However, this clever reordering requires insight on your part and is nowhere near as general as the graph-theory-based reordering algorithms people have mentioned in their answers here.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-19T05:34:13.093" LastActivityDate="2012-01-19T05:34:13.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="878" PostTypeId="5" CreationDate="2012-01-19T11:18:20.777" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-19T11:18:20.777" LastActivityDate="2012-01-19T11:18:20.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="879" PostTypeId="4" CreationDate="2012-01-19T11:18:20.777" Score="0" Body="The goal of computer vision is an automatic extraction of information out of images. It includes methods for acquiring, processing, analyzing, and understanding images and high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner." OwnerUserId="302" LastEditorUserId="9701" LastEditDate="2016-11-10T15:55:42.110" LastActivityDate="2016-11-10T15:55:42.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="880" PostTypeId="2" ParentId="873" CreationDate="2012-01-19T13:16:30.647" Score="13" Body="&lt;p&gt;It depends on the shape of your domain and presence of local refinement, but for isotropic domains in $d$ dimensions, the minimum achievable bandwidth scales as $N^{\frac{d-1}{d}}$ where $N$ is the total number of elements (or vertices). You can use a method like &lt;a href=&quot;http://en.wikipedia.org/wiki/Reverse_Cuthill-McKee&quot; rel=&quot;noreferrer&quot;&gt;Reverse Cuthill-McKee&lt;/a&gt; to produce reasonable low-bandwidth orderings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;em&gt;minimizing bandwidth is not the objective of ordering techniques for sparse direct solvers&lt;/em&gt;. Indeed, this will lead to $N^{\frac{2d-1}{d}}$ nonzeros in the factors, but optimal orderings (usually found by orderings such as nested dissection which is used by most packages) have $N \log N$ nonzeros in 2D and $N^{4/3}$ nonzeros in 3D. See &lt;a href=&quot;http://www.cse.illinois.edu/courses/cs598mh/george_liu.pdf&quot; rel=&quot;noreferrer&quot;&gt;George, Liu, and Ng's book&lt;/a&gt; for details on this result.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Visual comparison of orderings&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The following graphically compares orderings for the 5-point Laplacian on a $12\times 12$ regular grid. For each ordering, I report the number of nonzeros in the $L$ and $U$ factors for a $100\times 100$ grid. Note that the difference is much more pronounced in 3D. You can plot the fill matrices and diagnostics with PETSc:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cd $PETSC_DIR/src/ksp/ksp/examples/tutorials &amp;amp;&amp;amp; make ex2  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$ ./ex2 -m 12 -n 12 -ksp_view -pc_type lu -pc_factor_mat_ordering_type nd -mat_view_draw -draw_pause 2&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Matrix $A$ and factor $L$ in natural ordering (1990198 nonzeros)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/w0YKO.png&quot; alt=&quot;$A$ in Natural ordering&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/UvIIP.png&quot; alt=&quot;$L$ factor in Natural ordering&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Matrix $A$ and factor $L$ in Reverse Cuthill-McKee ordering (1353100 nonzeros)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ndS4P.png&quot; alt=&quot;$A$ in Reverse Cuthill-McKee ordering&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/TWgt2.png&quot; alt=&quot;$L$ factor in Reverse Cuthill-McKee ordering&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Matrix $A$ and factor $L$ in Nested Dissection ordering (382934 nonzeros)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/LbBIl.png&quot; alt=&quot;$A$ in Nested Dissection ordering&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/HuiD9.png&quot; alt=&quot;$L$ factor in Nested Dissection Ordering&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, low-bandwidth orderings are often reasonable orderings (algorithmically) for incomplete factorization and relaxation like SOR and tend to reuse cache well (so are good for throughput). For maximum performance on structured grids, you can skip the column indices (because they have regular structure) and only store the stencil coefficients, or even recompute the coefficients on the fly. It is well worth careful profiling and analysis using a performance model to determine whether optimizations like this are worth the implementation time and reduced flexibility.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-19T13:16:30.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="881" PostTypeId="2" ParentId="874" CreationDate="2012-01-19T13:33:01.447" Score="4" Body="&lt;p&gt;Although it places some restrictions on which methods will work, lack of diagonal dominance or symmetry is not inherently catastrophic. However, these properties are commonly associated with the more difficult problem of non-local influence and difficulty of coarsening, and many &quot;black-box&quot; solvers will not work. To answer your question in a more substantive way, we would need to know the details of this particular system (physics, discretization, parameter regime).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My practical suggestion is to start with direct solvers and delve into iterative solvers only if necessary. You can spend a career developing robust iterative solvers for a particular difficult problem.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-19T13:33:01.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="882" PostTypeId="1" AcceptedAnswerId="2916" CreationDate="2012-01-19T14:26:19.983" Score="9" ViewCount="1404" Body="&lt;p&gt;I have heard that a fast fourier transform can be used to solve the poisson problem when the boundary conditions are all one type... Sine series for dirichlet, cosine for neumann, and both for periodic.  Considering a 2D rectangular domain, suppose two opposite sides have periodic boundary conditions, and the other two have dirichlet conditions.  Can a fast fourier transform be applied to solve this problem efficiently?  If so, wouldn't the exponential form be sufficient?  If not, what solver would you recommend for this situation?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="123" LastEditDate="2012-01-19T17:54:41.047" LastActivityDate="2015-05-12T16:15:32.800" Title="Which fourier series is needed to solve a 2D poisson problem with mixed boundary conditions using Fast Fourier Transform?" Tags="|pde|fourier-analysis|boundary-conditions|poisson|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="883" PostTypeId="2" ParentId="874" CreationDate="2012-01-19T15:06:14.270" Score="2" Body="&lt;p&gt;How large/sparse is your system? Do you really need to solve this iteratively?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest trying to solve it in Matlab or Octave using the sparse solver (just initialize the matrix using &quot;sparse&quot; and then use backslash). If that works for you, then use UMFPACK directly, which is what Matlab and Octave use internally for the sparse solves.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-01-19T15:06:14.270" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="884" PostTypeId="1" AcceptedAnswerId="886" CreationDate="2012-01-19T17:54:04.760" Score="4" ViewCount="350" Body="&lt;p&gt;Recently, I've been asking about methods to solve a finite difference discretization of the 2D Poisson equation (see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/866/how-to-reorder-variables-to-produce-a-banded-matrix-of-minimum-bandwidth&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/882/which-fourier-series-is-needed-to-solve-a-2d-poisson-problem-with-mix-boundary-c&quot;&gt;here&lt;/a&gt;) of the form:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U_{i-1,j} + U_{i+1,j} -4U_{i,j} + U_{i,j-1} + U_{i,j+1} = f_{i,j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The left hand side of this system is often referred to as the 5-point discrete Laplacian.  Suppose that left hand side of the linear system is not quite the traditional five-point discrete Laplacian, but instead has the form:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{k,l} a^{ij}_{kl} U_{kl} = a^{ij}_{i-1,j}U_{i-1,j} + a^{ij}_{i+1,j}U_{i+1,j} -a^{ij}_{i,j}U_{i,j} + a^{ij}_{i,j-1}U_{i,j-1} + a^{ij}_{i,j+1}U_{i,j+1} = f_{i,j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with non-negative off-diagonal ($a^{ij}_{k,l} \ge 0$ for all off-diagonal entries $(i,j) \ne (k,l)$) and zero row sums,&#xA;$$ \sum_{kl} a^{ij}_{kl} = 0 $$&#xA;for all $i,j$. From these conditions, the matrix $A = (a^{ij}_{kl})$ is the negative of an &lt;a href=&quot;http://en.wikipedia.org/wiki/M-matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;$M$-matrix&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is sparse and diagonally dominant (I would characterize it as &quot;weakly diagonally dominant&quot; because the diagonal term is exactly equal to the sum of the off diagonal elements of the same row).  Since it exhibits a similar structure to the five-point Laplacian (except perhaps the possibility of being symmetric?), I figured that there may be some similarities between methods to solve this system of equations and those of the five-point discrete Laplacian.  Specifically, I was thinking that one of the following may be true:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We can compact the sparsity of the matrix into a thinly banded system (since there are only five variables per equation) and use a direct solver.&lt;/li&gt;&#xA;&lt;li&gt;We can use some sort of Fast Fourier Transform to solve it. (I think this one is highly unlikely though.)&lt;/li&gt;&#xA;&lt;li&gt;We can use some sort of multigrid approach to solve the system iteratively.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the end, the systems will be very large and a parallel implementation will be necessary.&#xA;Any feedback on fast methods to approach this problem in parallel would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-01-30T14:08:41.563" Title="How to solve a problem with structure similar to a finite difference discretization of the 2D Poisson equation, but with non-symetric coefficients?" Tags="|linear-algebra|pde|parallel-computing|poisson|linear-solver|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="885" PostTypeId="1" AcceptedAnswerId="888" CreationDate="2012-01-19T19:28:32.363" Score="3" ViewCount="102" Body="&lt;p&gt;I see that there are many files in the ScaLAPACK library without any immediately obvious naming convention... I'm sure that if the first letter is &quot;p&quot;, it means parallel.  But I'm not sure how to make sense of the rest of the names as seen &lt;a href=&quot;http://www.netlib.org/scalapack/explore-html/dir_78dd70a4654792622dd59c7dbd6a0c9f.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="67" LastEditDate="2012-01-21T22:27:40.213" LastActivityDate="2012-01-21T22:27:40.213" Title="What is the naming convention used in ScaLAPACK?" Tags="|libraries|parallel-computing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="886" PostTypeId="2" ParentId="884" CreationDate="2012-01-19T19:39:39.683" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Please take the advice given by the answers to your &lt;a href=&quot;https://scicomp.stackexchange.com/q/866/119&quot;&gt;several&lt;/a&gt; &lt;a href=&quot;https://scicomp.stackexchange.com/q/873/119&quot;&gt;previous&lt;/a&gt; &lt;a href=&quot;https://scicomp.stackexchange.com/q/865/119&quot;&gt;questions&lt;/a&gt; and stop suggesting that banded direct solvers would be appropriate or this problem. They are not, and sparse direct solvers do not attempt to find low-bandwidth orderings, they choose orderings that reduce fill (like nested dissection).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;FFT is not appropriate for the case of general coefficients.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Geometric or (depending on problem structure) algebraic multigrid should work well for this problem, provided you handle a few possibilities correctly. Note that if the coefficients represent extreme anisotropy, then the continuum equations become similar to a bunch of decoupled 1D problems, so isotropic coarsening cannot be expected to work (at least without line smoothers). If the equations represent transport, then you may have to use upwinded interpolants and/or relaxation. Note that second order linear discretizations of transport produce oscillations, so you would likely be using the upwinded form already. If the coefficients are highly variable, you may need to construct low-energy interpolants.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In general, it is much better to say which equations you are solving in which regime with which discretization than it is to make superficial observations about the matrix structure.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-30T13:55:06.983" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="887" PostTypeId="1" CreationDate="2012-01-19T20:17:26.763" Score="9" ViewCount="1893" Body="&lt;p&gt;I have a couple of questions regarding the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve the Schrodinger equation in 1D using the crank nicolson discretization followed by inverting the resulting tridiagonal matrix. My problem has now evolved into a problem with periodic boundary conditions and so I have modified my code to use the Sherman Morrison algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose &lt;code&gt;v&lt;/code&gt; is my RHS in at each time step when I wish to invert the tridiagonal matrix. The size of &lt;code&gt;v&lt;/code&gt; is the number of grid points I have over space. When I set &lt;code&gt;v[0]&lt;/code&gt; and &lt;code&gt;v[-1]&lt;/code&gt; in terms of each other as is required in my periodic situation, my equation blows up. I cannot tell why this is happening. I am using python2.7 and scipy's inbuilt solve_banded to solve the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads me to my second question:&#xA;I used python because it is the language I know best, but I find it rather slow (even with the optimizations offered by numpy and scipy). I have tried using C++ as I am reasonably familiar with it. I thought I'd use the GSL which would be BLAS optimized, but found no documentation to create complex vectors or solve the tridiagonal matrix with such complex valued vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like objects in my program as I feel it would be the easiest way for me to generalize later to include coupling between wavefunctions thus I am sticking to an object oriented language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could try writing the tridiagonal matrix solver by hand, but I ran into problems when I did so in python. As I evolved over large times with finer and finer time steps, the error accumulated and gave me nonsense. Keeping this in mind, I decided to use the in-built methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice is much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Here is the relevant code snippet. The notation is borrowed from Wikipedia's page on the tridiagonal matrix (TDM) equation. v is the RHS of the crank nicolson algorithm at each time step. The vectors a, b and c are the diagonals of the TDM. The corrected algorithm for the periodic case is from &lt;a href=&quot;http://www.cfd-online.com/Wiki/Tridiagonal_matrix_algorithm_-_TDMA_%28Thomas_algorithm%29&quot;&gt;the CFD Wiki&lt;/a&gt;. I have done a little renaming. What they have called u, v I have called U, V (capitalized). I have called q the complement, y the temporary solution and the actual solution self.currentState. The assignment of v[0] and v[-1] is what is causing the problem here and thus has been commented out. You may ignore the factors of gamma. They are non-linear factors used to model Bose Einstein Condensates.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for T in np.arange(self.timeArraySize):&#xA;        for i in np.arange(0,self.spaceArraySize-1):&#xA;            v[i] = Y*self.currentState[i+1] + (1-2*Y)*self.currentState[i] + Y*self.currentState[i-1] - 1j*0.5*self.timeStep*potential[i]*self.currentState[i] - self.gamma*1j*0.5*self.timeStep*(abs(self.currentState[i])**2)*self.currentState[i]&#xA;            b[i] = 1+2*Y + 1j*0.5*self.timeStep*potential[i] + self.gamma*self.timeStep*1j*0.5*(abs(self.currentState[i])**2)&#xA;&#xA;        #v[0] = Y*self.currentState[1] + (1-2*Y)*self.currentState[0] + Y*self.currentState[-1] - 1j*0.5*self.timeStep*potential[0]*self.currentState[0]# - self.gamma*1j*0.5*self.timeStep*(abs(self.currentState[0])**2)*self.currentState[0]&#xA;        #v[-1] = Y*self.currentState[0] + (1-2*Y)*self.currentState[-1] + Y*self.currentState[-2] - 1j*0.5*self.timeStep*potential[-1]*self.currentState[-1]# - self.gamma*1j*0.5*self.timeStep*(abs(self.currentState[-1])**2)*self.currentState[-1]&#xA;        b[0] = 1+2*Y + 1j*0.5*self.timeStep*potential[0] + self.gamma*self.timeStep*1j*0.5*(abs(self.currentState[0])**2)&#xA;        b[-1] = 1+2*Y + 1j*0.5*self.timeStep*potential[-1] + self.gamma*self.timeStep*1j*0.5*(abs(self.currentState[-1])**2)&#xA;&#xA;        diagCorrection[0], diagCorrection[-1] = - b[0], - c[-1]*a[0]/b[0]&#xA;&#xA;        tridiag = np.matrix([&#xA;            c,&#xA;            b - diagCorrection,&#xA;            a,&#xA;        ])&#xA;&#xA;        temp = solve_banded((1,1), tridiag, v)&#xA;&#xA;        U = np.zeros(self.spaceArraySize, dtype=np.complex64)&#xA;        U[0], U[-1] = -b[0], c[-1]&#xA;&#xA;        V = np.zeros(self.spaceArraySize, dtype=np.complex64)&#xA;        V[0], V[-1] = 1, -a[0]/b[0]&#xA;&#xA;        complement = solve_banded((1,1), tridiag, U)&#xA;&#xA;        num = np.dot(V, temp)&#xA;        den = 1 + np.dot(V, complement)&#xA;&#xA;        self.currentState = temp  - (num/den)*complement&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="722" LastEditorUserId="722" LastEditDate="2012-01-20T05:25:55.970" LastActivityDate="2012-01-20T19:49:51.860" Title="Schrodinger equation with periodic boundary conditions" Tags="|pde|linear-algebra|boundary-conditions|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="888" PostTypeId="2" ParentId="885" CreationDate="2012-01-19T21:26:53.983" Score="5" Body="&lt;p&gt;For a number of the files in PBLAS, with some exceptions like &lt;code&gt;pilaenv()&lt;/code&gt;, the stuff after the initial &lt;code&gt;p&lt;/code&gt; follow a convention similar to BLAS. The second letter in particular determines if the routine handles &lt;code&gt;s&lt;/code&gt;ingle, &lt;code&gt;d&lt;/code&gt;ouble, &lt;code&gt;c&lt;/code&gt;omplex, or double complex (&lt;code&gt;z&lt;/code&gt;) types. The last portions are an effective shorthand for what the routine does, e.g. &lt;code&gt;axpy&lt;/code&gt; for the operation $\alpha\mathbf x+\mathbf y$, or &lt;code&gt;ger&lt;/code&gt; for the operation $\mathbf A+\alpha\mathbf x\mathbf y^\top$. &lt;a href=&quot;http://www.netlib.org/scalapack/explore-html/dir_35e25287b57dec2e0f537f7538ece03f.html&quot;&gt;ScaLAPACK proper&lt;/a&gt; follows a similar convention; thus, &lt;code&gt;pdgetrf()&lt;/code&gt; for example computes the LU decomposition (&lt;code&gt;getrf&lt;/code&gt;) of a matrix with &lt;code&gt;d&lt;/code&gt;ouble entries, and analogously for &lt;code&gt;psgetrf()&lt;/code&gt;, &lt;code&gt;pcgetrf()&lt;/code&gt;, and &lt;code&gt;pzgetrf()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2012-01-19T21:26:53.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="889" PostTypeId="1" CreationDate="2012-01-19T23:22:36.067" Score="1" ViewCount="85" Body="&lt;p&gt;I have a set of constraints over modular domains e.g. $\exists a \in A_i : x \equiv a \pmod{n_i}$ for all $i=0,\ldots,k$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is, does such an $x$ exist? I've been pointed to &lt;a href=&quot;https://en.wikipedia.org/wiki/Method_of_successive_substitution&quot; rel=&quot;nofollow&quot;&gt;method of successive substitution&lt;/a&gt; which ended up with me implementing the following python code.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import fractions&#xA;def exist(ruleList, acc = [0,1]): # currentCandidate , currentDomain&#xA;  if len(ruleList) == 0:&#xA;    return True&#xA;  else:&#xA;    rule = ruleList[0]&#xA;    n = rule[1]/fractions.gcd(rule[1], acc[1])&#xA;    newDomain = acc[1] * n  &#xA;    for candidate in (acc[0] + acc[1] * i for i in xrange(n)): &#xA;      if candidate % rule[1] in rule[0]:&#xA;        if exist(ruleList[1:], [candidate, newDomain]):&#xA;          return True&#xA;    return False&#xA;&#xA;rules = [([0,1,2],5),   # A_1,n_1&#xA;         ([4,5,6],9),   # A_2,n_2&#xA;         ([0,3,6],10)]  # A_3,n_3&#xA;&#xA;print (exists(rules),)  # True&#xA;&#xA;rules = [([0,2,3],5),   # A_1,n_1&#xA;         ([1,2,8],9),   # A_2,n_2&#xA;         ([4,6,9],10)]  # A_3,n_3&#xA;&#xA;print (exist(rules),)   # False&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is their a better way of implementing this or better algorithmic approach?&lt;/p&gt;&#xA;" OwnerUserId="656" LastEditorUserId="656" LastEditDate="2012-01-21T17:51:31.513" LastActivityDate="2012-01-21T17:51:31.513" Title="Constraint solving over modular domains" Tags="|algorithms|performance|python|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="890" PostTypeId="2" ParentId="887" CreationDate="2012-01-19T23:58:11.050" Score="2" Body="&lt;h3&gt;Second question&lt;/h3&gt;&#xA;&lt;p&gt;Code that calls Scipy/Numpy is usually only fast if it can be vectorized; you shouldn't have anything &amp;quot;slow&amp;quot; inside of a python loop. Even then, it's pretty much unavoidable that it will be at least a little slower than something using a similar library in a compiled language.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in np.arange(0,self.spaceArraySize-1):&#xA;            v[i] = Y*self.currentState[i+1] + (1-2*Y)*self.currentState[i]   ...&#xA;            b[i] = 1+2*Y + 1j*0.5*self.timeStep*potential[i] + ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is what I mean by &amp;quot;slow in a python loop&amp;quot;. Python's &lt;code&gt;for&lt;/code&gt; is unacceptably slow for most numerical applications, and Scipy/Numpy do not affect this at all. If you're going to use python, this inner loop should be expressed as one or two Numpy/Scipy functions, which those libraries may or may not provide. If they don't provide something that allows you to iterate over arrays like this and access adjacent elements, python is the wrong tool for what you want to do.&lt;/p&gt;&#xA;&lt;p&gt;Also, you're doing an inversion rather than a matrix-vector solve. A matrix inversion, followed by a matrix-vector multiply, is &lt;strong&gt;much&lt;/strong&gt; slower than a matrix-vector solve. This is almost certainly the thing slowing down your code more than anything else.&lt;/p&gt;&#xA;&lt;p&gt;If you want to use C/C++, GSL is kind of lacking when it comes to complex linear algebra. I'd recommend either using BLAS or LAPACK directly, or using a library like PETSc or Trilinos. If you have MKL installed, you can use that too. You might also want to check out Fortran 2008, which is object-oriented.&lt;/p&gt;&#xA;&lt;p&gt;Your matrices are sparse, so you'll want to make sure you use sparse libraries.&lt;/p&gt;&#xA;&lt;p&gt;I would also say that what you are doing here seems low-level enough that object-orientation should probably not be your primary concern. A Fortran 90+ array is probably a pretty good match to what you need, and F90 compilers can auto-parallelize some loops.&lt;/p&gt;&#xA;&lt;p&gt;Also, you may want to check out Octave or Matlab, which have the &lt;code&gt;sparse()&lt;/code&gt; function. If used properly, these should be able to run pretty quickly.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-01-20T19:49:51.860" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="891" PostTypeId="1" AcceptedAnswerId="899" CreationDate="2012-01-20T08:15:56.037" Score="22" ViewCount="6744" Body="&lt;p&gt;Consider an IVP for ODE system $y&amp;#39;=f(x,y)$, $y(x_0)=y_0$. Most commonly this problem is considered stiff when Jacobi matrix $\frac{\partial f}{\partial y}(x_0,y_0)$ has &lt;strong&gt;both&lt;/strong&gt; eigenvalues with very large negative real part and eigenvalues with very small negative real part (I consider only the stable case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, in the case of just one equation, for example Prothero-Robinson equation $y&amp;#39;=\lambda y + g&amp;#39;+\lambda g$, it is called stiff when $\lambda\ll -1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there are two questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why small eigenvalues are included in the definition of stiffness for ODE systems? I believe that the presence of only very large negative real parts is quite enough for system to be stiff, because this makes us use small timesteps for explicit methods.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, I know that the most common stiff problems (e. g. arising from parabolic PDEs) do have both large and small eigenvalues. So the second question: is there a good  natural example of large stiff system without very small eigenvalues (or alternatively with mild ratio $\lambda_{\max}/\lambda_{\min}$)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;OK, let's modify the question. Consider two two-dimensional linear ODE systems: first with eigenvalues {-1000000,-0.00000001} and second with {-1000000,-999999}. As for me, both of them are stiff. But if we consider stiffness ratio definition, the second system is not. The main question: why stiffness ratio is considered at all?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the second part of question is still important, lets paraphrase it: I'm looking for a &quot;natural&quot; large ODE system with big negative eigenvalues and mild stiffness ratio (not greater than, say, 100).&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="720" LastEditDate="2012-02-02T13:24:30.323" LastActivityDate="2012-02-02T13:24:30.323" Title="The definition of stiff ODE system" Tags="|ode|stiffness|" AnswerCount="4" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="892" PostTypeId="2" ParentId="891" CreationDate="2012-01-20T09:18:14.500" Score="12" Body="&lt;h2&gt;Part 1&lt;/h2&gt;&#xA;&lt;p&gt;Small eigenvalues are &lt;em&gt;not&lt;/em&gt; included in the definition of stiffness for ODE (initial value problem) systems. There is no satisfying definition of stiffness that I know of, but the best definitions I've come across are:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If a numerical method with a finite region of absolute stability, applied to a system with any initial conditions, is forced to use in a certain interval of integration a step length which is excessively small in relation to the smoothness of the exact solution in that interval, then the system is said to be stiff in that interval. (Lambert, J. D. (1992), &lt;em&gt;Numerical Methods for Ordinary Differential Systems&lt;/em&gt;, New York: Wiley.)&lt;/p&gt;&#xA;&lt;p&gt;An IVP [initial value problem] is &lt;em&gt;stiff&lt;/em&gt; in some interval &lt;span class=&quot;math-container&quot;&gt;$[0,b]$&lt;/span&gt; if the step size needed to maintain stability of the forward Euler method is much smaller than the step size required to represent the solution accurately. (Ascher, U. M. and Petzold, L. P. (1998), &lt;em&gt;Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations&lt;/em&gt;,  Philadelphia:SIAM.)&lt;/p&gt;&#xA;&lt;p&gt;Stiff equations are equations where certain implicit methods, in particular BDF, perform better, usually tremendously better, than explicit ones. (C. F. Curtiss &amp;amp; J. O. Hirschfelder (1952): &lt;em&gt;Integration of stiff equations.&lt;/em&gt; PNAS, vol. 38, p. 235-243)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The Wikipedia article on &lt;a href=&quot;http://en.wikipedia.org/wiki/Stiff_equation&quot; rel=&quot;noreferrer&quot;&gt;stiff equations&lt;/a&gt; goes on to attribute the following &amp;quot;statements&amp;quot; to Lambert:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A linear constant coefficient system is stiff if all of its eigenvalues have negative real part and the stiffness ratio is large.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stiffness occurs when stability requirements, rather than those of accuracy, constrain the step length. [Note that this &amp;quot;observation&amp;quot; is essentially the definition from Ascher and Petzold.]&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stiffness occurs when some components of the solution decay much more rapidly than others.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Each of these observations has counterexamples (though admittedly I couldn't produce one off the top of my head).&lt;/p&gt;&#xA;&lt;h2&gt;Part 2&lt;/h2&gt;&#xA;&lt;p&gt;Probably the best example I could come up with would be integrating any sort of large combustion reaction system in chemical kinetics under conditions that result in ignition. The system of equations will be stiff until ignition, and then it will no longer be stiff because the system has passed an initial transient. The ratio of largest to smallest eigenvalue should not be large except around the ignition event, though such systems tend to confound stiff integrators unless you set exceedingly strict integration tolerances.&lt;/p&gt;&#xA;&lt;p&gt;The book by Hairer and Wanner also gives several other examples in its first section (Part IV, section 1) that illustrate many other examples of stiff equations. (Wanner, G., Hairer, E., &lt;em&gt;Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems&lt;/em&gt; (2002), Springer.)&lt;/p&gt;&#xA;&lt;p&gt;Finally, it's worth pointing out the observation of C. W. Gear:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Although it is common to talk about &amp;quot;stiff differential equations,&amp;quot; an equation &lt;em&gt;per se&lt;/em&gt; is not stiff, a particular initial value problem for that equation may be stiff, in some regions, but the sizes of these regions depend on the initial values &lt;em&gt;and&lt;/em&gt; the error tolerance. (C. W. Gear (1982): &lt;em&gt;Automatic detection and treatment of oscillatory and/or stiff ordinary differential equations.&lt;/em&gt; In: Numerical integration of differential equations, Lecture notes in Math., Vol. 968, p. 190-206.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-01-20T09:29:05.527" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="893" PostTypeId="1" CreationDate="2012-01-20T12:05:09.337" Score="2" ViewCount="1100" Body="&lt;p&gt;I have transferred my MATLAB Lanczos solver for symmetric eigenvalue solvers to C++ with the help of Intel MKL and MTL4 libraries. I have some wrapper templates for MKL routines. However during the iterations, the results of my C++ implementation starts to deviate from the values that MATLAB finds. Up to some point in the iterations, the differences seem to be small but after about 9-10 iterations the results start to deviate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that this is due to the full re-orthogonalization that I used in the Lanczos solver. The strange thing is that I am using a simple Gram-Schmidt orthogonalization in MATLAB, however use of the same operations in C++ results in the difference in the \alpha and \beta coefficients of my problem. MATLAB uses double precision as far as I understand and in C++ I also use double precision by default. Intel MKL also uses double precision. However, it seems that the round-off errors introduced in the intermediate computations severely affect the performance of the C++ code, at least this is my guess apart from orthogonalizations. Any ideas to recover this strange problem?&lt;/p&gt;&#xA;" OwnerUserId="728" LastEditorUserId="123" LastEditDate="2012-01-21T12:01:37.670" LastActivityDate="2012-01-21T19:25:32.820" Title="Lanczos solver implementations in MATLAB/C++ give different results" Tags="|linear-algebra|software|eigensystem|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="894" PostTypeId="2" ParentId="893" CreationDate="2012-01-20T13:49:36.210" Score="3" Body="&lt;p&gt;I believe that this is the same question that you recently posed to the MUMPS list &lt;a href=&quot;https://listes.ens-lyon.fr/sympa/arc/mumps-users/2011-12/msg00051.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, so I am assuming that the C++ code you refer to is a wrapper for MUMPS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be honest, what language you're computing in is irrelevant as long as you are using the same &lt;a href=&quot;http://en.wikipedia.org/wiki/Double-precision_floating-point_format&quot; rel=&quot;nofollow&quot;&gt;floating point standard&lt;/a&gt; and the same algorithms. It is also not clear to me that your final answer is changing significantly; if you are worried about variations in the tridiagonal matrix, then make sure that &lt;em&gt;every&lt;/em&gt; algorithmic detail is the same between your two implementations.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-01-20T13:54:46.983" LastActivityDate="2012-01-20T13:54:46.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="895" PostTypeId="2" ParentId="858" CreationDate="2012-01-20T15:29:33.837" Score="5" Body="&lt;p&gt;You have an 2D mesh of height values. I'm going to assume you create the faces of the surface for the $i$,$j$th square using two triangles of $\left\langle\left(i,j\right),\left(i+1,j\right),\left(i+1,j+1\right)\right\rangle$ and $\left\langle\left(i,j\right),\left(i,j+1\right),\left(i+1,j+1\right)\right\rangle$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your vertical cut plain can be defined as a 2D line called $l$. $l$ can be represented by using 2 known points on the edge of the mesh or by using an 2D line equation $i = mj+c$, $l=\left\langle{m,c}\right\rangle$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intersection points are the intersection with any of the edges of triangles along that 2D line $l$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/duJkI.png&quot; alt=&quot;2D mesh with 2D line as a cut plane&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the points heights are worked out by using &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear interpolation&lt;/a&gt; between the heights given by the two array coordinated connected by the triangle edge being intersected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Distance along the cut plane is given by &lt;a href=&quot;https://en.wikipedia.org/wiki/Pythagorean_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pythagorean theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import math&#xA;from random import choice, uniform&#xA;&#xA;mesh = [[10,5,8],[12,7,5],[4,7,3],[4,6,4],[8,3,1]]&#xA;print mesh&#xA;&#xA;iMax = len(mesh)-1&#xA;jMax = len(mesh[0])-1&#xA;&#xA;# pick random vertical cut plane&#xA;if choice([True,False]):&#xA;  p1 = (uniform(0,iMax), choice([0,jMax]))&#xA;else:&#xA;  p1 = (choice([0,iMax]), uniform(0,jMax))&#xA;while True:&#xA;  if choice([True,False]):&#xA;    p2 = (uniform(0,iMax), choice([0,jMax]))&#xA;  else:&#xA;    p2 = (choice([0,iMax]), uniform(0,jMax))&#xA;  if p1[0] != p2[0] and  p1[1] != p2[1]:&#xA;    break&#xA;&#xA;print &quot;P1&quot;,p1&#xA;print &quot;P2&quot;,p2&#xA;&#xA;if p1[1] == p2[1]:&#xA;  m = float('Inf') # Not handling this, needs fix&#xA;  c = None&#xA;else:&#xA;  m = float(p2[0]-p1[0])/(p2[1]-p1[1])&#xA;  c = p1[0]-m*p1[1]&#xA;&#xA;# i=mj+c formular&#xA;print &quot;i={0}j+{1}&quot;.format(m,c)&#xA;&#xA;def floor(a):&#xA;  return int(math.floor(a))&#xA;&#xA;def ceil(a):&#xA;  return int(math.ceil(a))&#xA;&#xA;iL = floor(min(p1[0], p2[0]))&#xA;iU = ceil(max(p1[0], p2[0]))&#xA;jL = floor(min(p1[1], p2[1]))&#xA;jU = ceil(max(p1[1], p2[1]))&#xA;&#xA;print &quot;points to plot unordered, d is distance from p1 (pythagoras), h is height as given by linear interpolation&quot;&#xA;&#xA;def linearInterpolation(h0,h1,r):&#xA;  return (h1-h0)*r+h0&#xA;&#xA;def pythag(a,b):&#xA;  return math.sqrt(a**2 + b**2)&#xA;&#xA;for i in xrange(iL, iU+1):&#xA;  j = (i-c)/m&#xA;  if 0 &amp;lt;= j &amp;lt;= jMax:&#xA;    j1, j2 = floor(j), ceil(j)&#xA;    h = linearInterpolation(mesh[i][j1], mesh[i][j2], j - j1)&#xA;    d = pythag(p1[0]-i, p1[1]-j)&#xA;    print (d,h)&#xA;&#xA;for j in xrange(jL, jU+1):&#xA;  i = m*j+c&#xA;  if 0 &amp;lt;= i &amp;lt;= iMax:&#xA;    i1, i2 = floor(i), ceil(i)&#xA;    h = linearInterpolation(mesh[i1][j], mesh[i2][j], i - i1)&#xA;    d = pythag(p1[0]-i, p1[1]-j)&#xA;    print (d,h)&#xA;&#xA;for v in xrange(-iU-1, jU):&#xA;  i = (c - v)/(1.0-m)&#xA;  j = m*i+c&#xA;  if 0 &amp;lt;= i &amp;lt;= iMax and 0 &amp;lt;= j &amp;lt;= jMax:&#xA;    i1, i2 = floor(i), ceil(i)&#xA;    j1, j2 = floor(j), ceil(j)&#xA;    r = pythag(i - i1, j - j1) / math.sqrt(2)&#xA;    h = linearInterpolation(mesh[i1][j1], mesh[i2][j2], r)&#xA;    d = pythag(p1[0]-i, p1[1]-j)&#xA;    print (d,h)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="656" LastEditorUserId="656" LastEditDate="2012-01-22T10:56:12.720" LastActivityDate="2012-01-22T10:56:12.720" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="896" PostTypeId="1" AcceptedAnswerId="897" CreationDate="2012-01-20T16:49:41.750" Score="4" ViewCount="1954" Body="&lt;p&gt;I just installed the PETSc library.  This is what I did from the home directory ~&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c petsc-3.2-p6.tar.gz|tar -xof -&#xA;cd ~/petsc-3.2-p6&#xA;PETC_DIR=$PWD: export PETSC_DIR&#xA;./configure --download-f-blas-lapack=1&#xA;make all&#xA;make PETSC_DIR=/export/home/myusername/petsc-3.2-p6 PETSC_ARCH=arch-linux2-c-debug test&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(note: I did not install mpich since it was already installed on the system)&lt;br&gt;&#xA;Then, I tried to run an example from the following &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/tutorials/UTAustinTutorial2011.pdf&quot; rel=&quot;nofollow&quot;&gt;this tutorial, slide 46 of 197&lt;/a&gt;, which asks me to do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd ~/petsc-3.2-p6/src/snes/examples/tutorials&#xA;make ex5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the process, I get the following errors:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;makefile:24: /conf/variables: No such file or directory&#xA;makefile:24: /conf/rules: No such file or directory&#xA;makefile:24: /conf/test: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is the problem in the way I setup PETSc? Or is there something I need to modify in the makefile?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="67" LastEditDate="2012-01-20T17:07:00.023" LastActivityDate="2012-01-20T17:54:26.927" Title="How to run a PETSc example?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="897" PostTypeId="2" ParentId="896" CreationDate="2012-01-20T17:54:26.927" Score="6" Body="&lt;p&gt;You most likely haven't set the PETSC_DIR and PETSC_ARCH environment variables correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, you misspelled PETSC_DIR as PETC_DIR in the third line of your build example.&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2012-01-20T17:54:26.927" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="898" PostTypeId="1" AcceptedAnswerId="914" CreationDate="2012-01-20T18:24:09.957" Score="5" ViewCount="1592" Body="&lt;p&gt;I found the following thread in the physics stackexchange where I saw the video called output attached to the &lt;a href=&quot;https://physics.stackexchange.com/questions/12199/solving-schrodingers-equation-with-crank-nicolson-method&quot;&gt;main post&lt;/a&gt;. The video can be found &lt;a href=&quot;http://dl.dropbox.com/u/27657710/output.avi&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I go about creating such an animation?&lt;/p&gt;&#xA;" OwnerUserId="722" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2012-02-17T19:10:55.047" Title="How do I create an animation from a 2D model or dataset?" Tags="|visualization|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="899" PostTypeId="2" ParentId="891" CreationDate="2012-01-20T18:32:49.150" Score="14" Body="&lt;p&gt;Stiffness involves some separation of scales. In general, if you are interested in phase of the fastest mode in the system, then you have to resolve it and the system is not stiff. But frequently, you are interested in the long-term dynamics of a &quot;slow manifold&quot; rather than the precise rate at which a solution off the slow manifold approaches it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chemical reactions and reacting flows are common examples of stiff systems. The &lt;a href=&quot;http://www.scholarpedia.org/article/Van_der_Pol_oscillator&quot; rel=&quot;nofollow noreferrer&quot;&gt;van der Pol oscillator&lt;/a&gt; is a common benchmark problem for ODE integrators that has a tunable stiffness paramater.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An ocean is another example that is perhaps helpful to visualize. Tsunamis (surface gravity waves) travel at the speed of an airplane and produce complex wave structure, but dissipate over long time scales and are mostly inconsequential to the long-term dynamics of the ocean. Eddies, or the other hand, travel about 100 times slower at quite pedestrian speeds, but cause mixing and transport temperature, salinity, and biogeochemical tracers that are relevant. But the same physics that propagates a surface gravity wave also supports an eddy (a quasi-equilibrium structure), so eddy velocity, path under Coriolis, and rate of dissipation are dependent on the gravity wave speed. This presents an opportunity for a time integration scheme designed for stiff systems to step over the time scale of the gravity wave and only resolve the relevant dynamical time scales. See &lt;a href=&quot;http://dx.doi.org/10.1175/1520-0493%282002%29130%3C2611:AINCMF%3E2.0.CO;2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mousseau, Knoll, and Reisner (2002)&lt;/a&gt; for discussion of this problem with a comparison of splitting and fully implicit time integration schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related: &lt;a href=&quot;https://scicomp.stackexchange.com/q/38/119&quot;&gt;When should implicit methods be used in the integration of hyperbolic PDEs?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that diffusive processes are usually considered to be stiff because the fastest time scale in the discrete system is mesh-dependent, scaling with $(\Delta x)^2$, but the time scale of the relevant physics is mesh independent. In fact, the fastest time scales for a given mesh represent spatially local relaxation to the slower manifold on which longer spatial scales evolve, so implicit methods can be very accurate even in strong norms despite not resolving the fastest scales.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-01-22T02:00:16.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="900" PostTypeId="1" CreationDate="2012-01-20T19:59:52.093" Score="2" ViewCount="136" Body="&lt;p&gt;I'm having an issue with my project. When I run the code of Monte-Carlo simulation, on the local server (the machine in my office) it runs on a rate of roughly 100000 steps per 24 hours. When I run it on the cluster the rate drops to about half of that or even less. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to figure this but I can't find any reason. I don't want you to read my code and analyze it mainly because this is not your work. I only ask for you experience for any suggestion. What I got so far is this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I don't to any parallel computing. &lt;/li&gt;&#xA;&lt;li&gt;When I run the code on the local machine, I don't usually run anything heavy aside from it. &lt;/li&gt;&#xA;&lt;li&gt;I do run, several copies of the code on the local machine but not more than the number of processor - 1 on that machine. The IT guys aren't sure, but they think that no dual processing are enabled on the local server. (There was an unsolved issue with that)&lt;/li&gt;&#xA;&lt;li&gt;When I run a single copy of the code on the cluster it runs okay. &lt;/li&gt;&#xA;&lt;li&gt;A significant drop of performance occur only when the number of copies is 8.&lt;/li&gt;&#xA;&lt;li&gt;There are two kinds of machines on the cluster. 16 nodes (8 dual processor) and 24 nodes (I think that those are 6 qudriple processors). &lt;/li&gt;&#xA;&lt;li&gt;The code is written in C++. Even though it is considered bad programming, all my variables are global and I access them directly (not via function).&lt;/li&gt;&#xA;&lt;li&gt;I compile the code using &lt;code&gt;icc -O3 -fast -ip&lt;/code&gt;. Removing any of those flag either reduced the performance or didn't have any noticeable affect.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I think, from all of this that this is an issue with memory management but I don't know how to test this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have several question about this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Did any of encountered something similar?&lt;/li&gt;&#xA;&lt;li&gt;How can I test if this is a memory issue?&lt;/li&gt;&#xA;&lt;li&gt;Is there another flag I could add to my compilation? &lt;/li&gt;&#xA;&lt;li&gt;Are there any tips for efficient memory usage in c++?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help&lt;/p&gt;&#xA;" OwnerUserId="648" LastActivityDate="2012-01-20T20:23:44.750" Title="Low performance on sge cluster" Tags="|performance|hpc|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="901" PostTypeId="2" ParentId="900" CreationDate="2012-01-20T20:23:44.750" Score="2" Body="&lt;p&gt;Are you saying that when you run &gt; 8 copies of the job on a single node, performance drops significantly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This happens commonly for codes which require high memory bandwidth - that do a lot of reading and writing to memory for fairly modest amounts of computation.  At some point you saturate the capability of the memory subsystem, and adding more tasks simply makes things worse.   Doug Eadline has a good introductory-level discussion of this on his site ( &lt;a href=&quot;http://www.clustermonkey.net//content/view/306/1/&quot; rel=&quot;nofollow&quot;&gt;http://www.clustermonkey.net//content/view/306/1/&lt;/a&gt; ) -- he talks about things in terms of &quot;effective number of cores&quot;, which I'm not sure is all that helpful a way to think about the underlying problem, but it gives you an idea of how many cores you could profitably use on a node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may be able to restructure your code to use less memory bandwith (using something like &lt;a href=&quot;http://wiki.scinethpc.ca/wiki/index.php/Introduction_To_Performance#cachegrind_.28Memory_use_analysis%3a_GPC.29&quot; rel=&quot;nofollow&quot;&gt;cachegrind&lt;/a&gt; might help if you're inadvertantly using more than you need) in which case you should be able to use a larger count of tasks per node without starving the other processes of memory bandwidth; or you might not, in which case you're stuck with using fewer cores per node.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2012-01-20T20:23:44.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="902" PostTypeId="2" ParentId="898" CreationDate="2012-01-20T20:40:30.093" Score="7" Body="&lt;p&gt;I will assume that you know how to output all the frames you want in the movie.  Then a simple free tool to turn them into a movie as &lt;a href=&quot;http://www.imagemagick.org/script/convert.php&quot; rel=&quot;nofollow&quot;&gt;Imagemagick's convert&lt;/a&gt;.  If you are on a Linux system, you probably already have it or can easily get it through your package manager.  Depending on the format of movie you want to make, you will also need an appropriate encoder (e.g., &lt;a href=&quot;http://ffmpeg.org/&quot; rel=&quot;nofollow&quot;&gt;ffmpeg&lt;/a&gt;).  If you have a bunch of files&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;frame001.png&#xA;frame002.png&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you simply&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;convert *.png movie.mpg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are lots of options you can use to change the framerate and other things; see the documentation for convert linked above.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-01-23T05:25:57.200" LastActivityDate="2012-01-23T05:25:57.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="903" PostTypeId="2" ParentId="898" CreationDate="2012-01-20T22:08:36.993" Score="0" Body="&lt;p&gt;There is also a freeware program &lt;a href=&quot;http://ffmpeg.org/&quot; rel=&quot;nofollow&quot;&gt;ffmpeg&lt;/a&gt; which is freely available.  If you prefer to have a GUI to do it, you may also want to consider using &lt;a href=&quot;http://rsbweb.nih.gov/ij/&quot; rel=&quot;nofollow&quot;&gt;ImageJ&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-01-20T22:08:36.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="904" PostTypeId="1" CreationDate="2012-01-21T01:21:12.170" Score="2" ViewCount="180" Body="&lt;p&gt;&lt;a href=&quot;http://peersim.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;Peersim&lt;/a&gt; is a peer-to-peer network simulator.  I'm simply trying to get the program to print out the network (an edge list is fine).  Presumably this is possible, since there is an in-built &lt;a href=&quot;http://peersim.sourceforge.net/doc/peersim/graph/GraphIO.html#writeEdgeList%28peersim.graph.Graph,%20java.io.PrintStream%29&quot; rel=&quot;nofollow&quot;&gt;writeEdgeList&lt;/a&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect there is an easy way to do this by modifying this line in the example: &lt;code&gt;include.control io degree&lt;/code&gt;  (Section 5 in &lt;a href=&quot;http://peersim.sourceforge.net/tutorial2/tutorial2.html&quot; rel=&quot;nofollow&quot;&gt;tutorial 2&lt;/a&gt;) which instead prints out a table that lists the number of nodes with certain degrees.  But I don't know what to change &quot;degree&quot; to.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: How do I print out an edge list of a network in Peersim?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="732" LastActivityDate="2012-01-21T01:21:12.170" Title="How to print out a network in peersim?" Tags="|software|" AnswerCount="0" CommentCount="1" ClosedDate="2012-09-28T08:55:55.987" ContentLicense="CC BY-SA 3.0" />
  <row Id="905" PostTypeId="1" AcceptedAnswerId="909" CreationDate="2012-01-21T01:52:00.357" Score="2" ViewCount="7746" Body="&lt;p&gt;I'm trying to compile a hello world program using PETSc, based off of &lt;a href=&quot;http://acts.nersc.gov/events/Workshop2002/slides/Smith.pdf&quot; rel=&quot;nofollow&quot;&gt;this tutorial, slide 33 &lt;/a&gt;.  How would I compile this?  I know that I can't use a simple 'mpicc' command.  When running the program, would I use the standard mpirun or mpiexec?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2016-07-25T09:17:05.740" Title="Compiling and running a &quot;hello world&quot; program in PETSc" Tags="|petsc|compiling|mpi|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="906" PostTypeId="5" CreationDate="2012-01-21T02:07:56.197" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-21T02:07:56.197" LastActivityDate="2012-01-21T02:07:56.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="907" PostTypeId="4" CreationDate="2012-01-21T02:07:56.197" Score="0" Body="Message Passing Interface - a standard and highly portable parallel programming interface for distributed memory systems." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-21T04:49:11.583" LastActivityDate="2012-01-21T04:49:11.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="908" PostTypeId="1" AcceptedAnswerId="912" CreationDate="2012-01-21T02:15:54.080" Score="7" ViewCount="2403" Body="&lt;p&gt;I have an application in which I am computing a quantity which is approximated by an average over $M$ points. In theory, the average converges to the correct quantity when $M$ is infinite. In practice, the computation involves a $M\times M$ symmetric positive definite matrix, whose condition number gets huge when $M$ increases. I compute the condition number as the ratio of the biggest and smallest eigenvalues given by a &lt;a href=&quot;http://eigen.tuxfamily.org/dox/classEigen_1_1SelfAdjointEigenSolver.html&quot;&gt;selfadjoint eigensolver&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the average I'm computing does not seem to reach stationarity when I increase $M$, but instead starts deviating quite a bit from what I feel is the correct value when $M$ is really big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I choose the best $M$? Is there a way to put some error bars on the result I get, based on the condition number? What is a bad condition number for the $LDL^T$ cholesky decomposition and for a SPD matrix, e.g. when should I start worrying?&lt;/p&gt;&#xA;" OwnerUserId="733" LastEditorUserId="733" LastEditDate="2012-01-21T19:33:02.517" LastActivityDate="2012-01-26T01:40:15.320" Title="precision vs matrix condition number" Tags="|linear-algebra|algorithms|condition-number|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="909" PostTypeId="2" ParentId="905" CreationDate="2012-01-21T02:21:31.383" Score="3" Body="&lt;p&gt;Slides from 2002 are ancient. Have a look at newer tutorials at &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/tutorials/index.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/documentation/tutorials/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question, though, you will want to look at some of the examples in, say, &lt;code&gt;$PETSC_DIR/src/ksp/ksp/examples/tutorials&lt;/code&gt;. In particular, the makefile is how to compile an example program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A short example to compile &lt;code&gt;hello.c&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;include ${PETSC_DIR}/conf/variables&#xA;include ${PETSC_DIR}/conf/rules&#xA;&#xA;hello: hello.o  chkopts&#xA;    -${CLINKER} -o hello hello.o  ${PETSC_LIB}&#xA;    ${RM} hello.o&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You would then run:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;export PETSC_DIR=/path/to/petsc&lt;/code&gt; and &lt;code&gt;export PETSC_ARCH=petsc-arch&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please review the installation instructions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/installation.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/documentation/installation.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a successful installation of PETSc, run your own test:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&lt;code&gt;make hello&lt;/code&gt;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good tip for debugging (where you want to leave the .o files around) is to run:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&lt;code&gt;make hello RM=echo&lt;/code&gt;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="142" LastEditorUserId="348" LastEditDate="2012-01-22T23:01:51.690" LastActivityDate="2012-01-22T23:01:51.690" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="910" PostTypeId="1" AcceptedAnswerId="973" CreationDate="2012-01-21T02:32:21.930" Score="11" ViewCount="23757" Body="&lt;p&gt;Recently, I've encountered a bizarre problem with FORTRAN95.  I initialized variables X and Y as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X=1.0&#xA;Y=0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Later I add them together and print the result:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.10000000149012&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After examining the variables, it seems as though 0.1 is not represented in double precision with full accuracy.  Is there any way to avoid this?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="123" LastEditDate="2012-01-21T11:51:28.327" LastActivityDate="2012-03-15T17:12:36.240" Title="How to set double precision values in Fortran" Tags="|fortran|floating-point|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="911" PostTypeId="2" ParentId="910" CreationDate="2012-01-21T03:20:50.913" Score="13" Body="&lt;p&gt;You declared the variables as double precision, but you initialized them with single precision values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could have written:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X=1.0d0&#xA;Y=1.0d-1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/973/81&quot;&gt;Barron's answer&lt;/a&gt; below is another way of making a literal double precision, with the advantage that it allows you to change the precision of your variables at a later time.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-03-15T17:12:36.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="912" PostTypeId="2" ParentId="908" CreationDate="2012-01-21T05:28:39.380" Score="7" Body="&lt;p&gt;I'm confused as to why you're saying the $LDL^T$ decomposition gives you your eigenvalues; one can certainly compute the determinant (and inertia) of a symmetric matrix cheaply from it, but it does not directly yield the eigenvalues themselves, as $L \cdot L^T$ is not a similarity transformation unless $L=I$. Moreover, I would suggest using the Cholesky (i.e., $LL^T$) decomposition for SPD matrices instead of $LDL^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it does turn out that the condition number really is terrible, then you are still in decent shape, as preconditioning SPD matrices is usually not very challenging, especially for diagonally-dominant matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Since you updated your question, I will update my response.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you condition number is really getting as high as, say, $10^{10}$ (so that you will lose roughly 10 decimal digits, as @aeismail mentioned), for matrices which you can compute the dense eigensolution on a regular machine, then I would start by making sure that no numerical atrocities are being committed. In particular, I would make sure that your self-adjoint matrix is not being formed as the symmetric product of a matrix with itself, e.g., $A := B B&amp;#39;$, as $A$'s condition number will be the square of $B$'s condition number. It would be much better to work directly with $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, there are often ways to cheaply improve the quality of your solution. For instance, if you were solving a linear system with your $LDL^T$ decomposition, you could &quot;iteratively refine&quot; your solution by running a handful of iterations of GMRES (or your favorite iterative method) with your solution from the direct solve as the initial guess. You haven't stated what you want to do with your system, so I can't say much more. &lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-01-26T01:40:15.320" LastActivityDate="2012-01-26T01:40:15.320" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="913" PostTypeId="2" ParentId="908" CreationDate="2012-01-21T07:24:43.087" Score="2" Body="&lt;p&gt;There is also a brute-force way to cope with ill-conditioning for not very large matrices: you can try to increase the precision of the machine arithmetics. If you use C or other supported language  the obvious choice is &lt;a href=&quot;http://en.wikipedia.org/wiki/GNU_Multiple_Precision_Arithmetic_Library&quot; rel=&quot;nofollow&quot;&gt;GNU GMP library&lt;/a&gt;. But it can be made much simpler (and more computationally expensive) in Mathematica using SetPrecision[...] function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the more the condition number the more digits of precision you'll need to get the satisfactory result.&lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2012-01-21T07:24:43.087" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="914" PostTypeId="2" ParentId="898" CreationDate="2012-01-21T12:05:37.567" Score="6" Body="&lt;p&gt;In case you are interested in generating the individual frames, here is a little example in python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib&#xA;matplotlib.use('Agg')&#xA;import matplotlib.pyplot as plt   # For plotting graphs.&#xA;import numpy as np&#xA;import subprocess                 # For issuing commands to the OS.&#xA;import os&#xA;import sys                        # For determining the Python version.&#xA;import glob&#xA;&#xA;fileList = glob.glob(os.path.join(&quot;LEMtrace*.dat&quot;))&#xA;&#xA;style = {0: '--',&#xA;         1: '-',&#xA;         2: '-.',&#xA;         3: ':'}&#xA;color = {0: 'k',&#xA;         1: 'r',&#xA;         2: 'g',&#xA;         3: 'b'}&#xA;n = 0&#xA;for f in fileList:&#xA;    n = n + 1&#xA;    plt.figure()&#xA;    data = np.loadtxt(f)&#xA;    ax = plt.subplot(111)&#xA;    ax.plot(data[:,0],data[:,1],label='YO2',c=color[0],ls=style[0])&#xA;    ax.plot(data[:,0],data[:,2],label='YH2',c=color[1],ls=style[1])&#xA;    ax.plot(data[:,0],data[:,3],label='YOH',c=color[2],ls=style[2])&#xA;    ax.plot(data[:,0],data[:,8]/2000.,label='T/2000',c=color[3],ls=style[3])&#xA;    ax.set_xlabel('LEM cell')&#xA;    ax.set_ylabel('Mass fraction or T / 2000 K')&#xA;    ax.set_ylim((0.0,2.0))&#xA;    str = 'Time {0:.5f} ms'.format(n * 5.8e-5)&#xA;    plt.title(str)&#xA;    leg = ax.legend(loc='best', shadow=True, fancybox=True)&#xA;    fig = &quot;frame_%04d&quot;%(n)&#xA;    plt.savefig(fig+'.png',format='png')&#xA;    plt.clf()&#xA;&#xA;command = ('mencoder',&#xA;           'mf://*png',&#xA;           '-mf',&#xA;           'type=png:w=800:h=600:fps=5',&#xA;           '-ovc',&#xA;           'lavc',&#xA;           '-lavcopts',&#xA;           'vcodec=mpeg4',&#xA;           '-oac',&#xA;           'copy',&#xA;           '-o',&#xA;           'profile_LEM.avi')&#xA;print &quot;\n\nabout to execute:\n%s\n\n&quot; % ' '.join(command)&#xA;subprocess.check_call(command)&#xA;command = ('convert',&#xA;           'frame*png',&#xA;           '+dither',&#xA;           '-layers',&#xA;           'Optimize',&#xA;           '-colors',&#xA;           '256',&#xA;           '-depth',&#xA;           '8',&#xA;           'profile_LEM.gif')&#xA;&#xA;subprocess.check_call(command)&#xA;print &quot;\n\n The movie was written to 'output.avi'&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Assuming you have some 1D data stored in the &lt;code&gt;LEMtrace*.dat&lt;/code&gt; files, this program loads the data for each file using &lt;code&gt;numpy.loadtxt&lt;/code&gt; (great for text files), plots a simple graph using &lt;code&gt;matplotlib&lt;/code&gt; and output 1 frame for each file with nice little time counter. Then it uses either &lt;code&gt;mencoder&lt;/code&gt; or &lt;code&gt;convert&lt;/code&gt; to generate an &lt;code&gt;avi&lt;/code&gt; or &lt;code&gt;gif&lt;/code&gt; animation. Hope this helps !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: A not so pretty (because of the data) example of the output: &lt;img src=&quot;https://i.stack.imgur.com/EW1kV.gif&quot; alt=&quot;GIFsample&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2012-01-21T12:05:37.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="915" PostTypeId="1" AcceptedAnswerId="918" CreationDate="2012-01-21T19:18:57.627" Score="9" ViewCount="944" Body="&lt;p&gt;The title says most of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a lightweight and easy-to-use library that I can use for Android (NDK) projects.&lt;br&gt;&#xA;For dense stuff I like using Eigen but I haven't found many comprehensive (and documented!) libraries for sparse stuff that &quot;just work&quot; in a project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc seems a little heavy with all these MPI things but that's only my first impression.&#xA;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="740" LastEditorUserId="740" LastEditDate="2012-01-21T19:24:50.360" LastActivityDate="2012-01-21T22:57:38.950" Title="Which Sparse Matrix Solver Libraries can I run on Android?" Tags="|pde|linear-algebra|libraries|performance|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="916" PostTypeId="2" ParentId="893" CreationDate="2012-01-21T19:25:32.820" Score="1" Body="&lt;p&gt;I ran into a similar sort of issue when comparing results between SciPy/NumPy and MATLAB. (I was using the results from SciPy/NumPy as reference results in unit tests.) MATLAB uses its own version of LAPACK and BLAS. In order to get bit-for-bit agreement, you'd have to use the same libraries as MATLAB, especially since algorithms can change slightly (bug fixes, changes in ordering conventions) from version to version of LAPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I doubt this is the &lt;em&gt;exact&lt;/em&gt; problem you're facing (my use case was a QR factorization), but it should explain the minor discrepancies you're seeing, and probably not the magnification of those discrepancies unless you're otherwise using the exact same algorithm in both implementations, and that algorithm is numerically stable.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-21T19:25:32.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="917" PostTypeId="2" ParentId="915" CreationDate="2012-01-21T22:38:41.433" Score="3" Body="&lt;p&gt;Yousef Saad's Sparskit has no dependencies just a bunch of simple Fortran routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though Fortran on Android can be hard. For start see &lt;a href=&quot;http://specificimpulses.blogspot.com/2011/10/android-fortran-step-by-step-part-1.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other option is to use the simple routines that come with the book &quot;Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods&quot; available &lt;a href=&quot;http://www.netlib.org/templates/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2012-01-21T22:38:41.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="918" PostTypeId="2" ParentId="915" CreationDate="2012-01-21T22:57:38.950" Score="4" Body="&lt;p&gt;You could try building &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/CSparse/&quot; rel=&quot;nofollow&quot;&gt;CSparse&lt;/a&gt;, which seems to have no dependencies other than the C standard library, judging from the source. (You can strip out the MATLAB interface, or just not build it.) Other routines on &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/&quot; rel=&quot;nofollow&quot;&gt;Tim Davis' sparse linear algebra algorithms&lt;/a&gt; web site may also have few or no dependencies (for instance, you can build UMFPACK using its internal BLAS library instead of an external high-performance BLAS implementation).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-21T22:57:38.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="920" PostTypeId="2" ParentId="891" CreationDate="2012-01-22T01:07:28.110" Score="1" Body="&lt;p&gt;&lt;strong&gt;The absolute magnitude of the eigenvalues (in a linear, autonomous problem) alone has no meaning at all; it's an artifact of the units you choose to express the problem in.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The chain of comments is getting out of control, so I'm making this an answer.  I'm not going to answer the full question; as I said, &lt;a href=&quot;http://en.m.wikipedia.org/wiki/Stiff_equation&quot; rel=&quot;nofollow&quot;&gt;see wikipedia&lt;/a&gt; or the other answers here.  I'm just answering the bit that says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider two two-dimensional linear ODE systems: first with eigenvalues {-1000000,-0.00000001} and second with {-1000000,-999999}. As for me, both of them are stiff.  But if we consider stiffness ratio definition, the second system is not. The main question: why stiffness ratio is considered at all?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Okay, let's consider an example of the second case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_1&amp;#39;(t) = -1000000 y_1(t)$$&#xA;$$y_2&amp;#39;(t) = -999999 y_2(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's consider a time variable with different units: $t^* = 1000000\cdot t$.  Then calculus reveals that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_1&amp;#39;(t^*) = - y_1(t^*)$$&#xA;$$y_2&amp;#39;(t^*) = -0.999999 y_2(t^*)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 1: I chose a diagonal system to make it totally obvious, but if you try it with another system with those eigenvalues, you'll see the same effect, since multiplying a matrix by a constant multiplies its eigenvalues by the same constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 2: I'm not even discussing here whether the system is stiff.  I'm just pointing out that your proposed definition of stiffness (i.e., any problem with $|\lambda|\gg1$) makes no sense, since it would mean that stiffness depends on the units in which I choose to express the problem.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-01-22T01:47:41.960" LastActivityDate="2012-01-22T01:47:41.960" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="922" PostTypeId="2" ParentId="908" CreationDate="2012-01-22T07:22:37.087" Score="4" Body="&lt;p&gt;To comment on the question about the error bounds for the matrix: it's a little tricky to specify strict bounds, but there is a &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/text.html&quot; rel=&quot;nofollow&quot;&gt;heuristic rule&lt;/a&gt; in numerical linear algebra that a condition number of $M$ will lead to a loss of approximately $\log_b M$ base-$b$ digits of precision in the results of your calculation involving the matrix $M$. So, for instance, a condition number $M = 100$ means you would lose $\log_{10}(100) = 2$ digits of precision.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-01-24T13:26:39.283" LastActivityDate="2012-01-24T13:26:39.283" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="923" PostTypeId="1" CreationDate="2012-01-22T09:28:52.963" Score="22" ViewCount="2608" Body="&lt;p&gt;I have a two-dimensional function $f(x,y)$ whose values I would like to sample.  The function is very expensive to compute and it has a complex shape, so I need to find a way to get the most information about its shape using the least number of sample points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What good methods are there to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I have so far&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I start from an existing set of points where I have already computed the function value (this could be a square lattice of points or something else).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then I compute a Delaunay triangulation of these points.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If two neighbouring points in the Delaunay triangulation are far enough ($ &amp;gt; \Delta X$) and the function value differs sufficiently in them ($&amp;gt; \Delta f$), then I insert a new point midway inbetween them.  I do this for each neighbouring point-pair.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What's wrong with this method?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, it works relatively well, but on functions similar to this one it's not ideal because sample points tend to &quot;jump over&quot; the ridge and not even notice it's there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/9yosI.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It produces results like this (if the resolution of the initial point grid is sufficiently rough):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fFYKJ.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This plot above shows the points where the function value is calculated (actually Voronoi cells around them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/RE9Zu.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This plot above shows the linear interpolation generated from the same points, and compares it to Mathematica's built-in sampling method (for about the same starting resolution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How to improve it?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the main issue here is that my method decides whether to add a refinement point or not based on the gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be better to take into account the curvature or at least the second derivative when adding refinement points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a very simple to implement way to take into account the second derivative or curvature when the locations of my points are not constrained at all?  (I don't necessarily have a &lt;em&gt;square&lt;/em&gt; lattice of starting points, this should ideally be general.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or what other simple ways are there to calculate the position of refinement points in an optimal way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am going to implement this in Mathematica, but this question is mainly about the method.  For the &quot;easy to implement&quot; bit it does count that I'm using Mathematica though (i.e. this was easy to do so far because it has a package for doing Delaunay triangulation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What practical problem I am applying this to&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am calculating a phase diagram.  It has a complex shape.  In one region its value is 0, in another region it's between 0 and 1.  There's a sharp jump between the two regions (it's discontinuous).  In the region where the function is greater than zero there is both some smooth variation and a couple of discontinuities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function value is calculated based on a Monte Carlo simulation, so occasionally an incorrect function value or noise is to be expected (this very is rare, but for a large number of points it happens, e.g. when the steady state is not reached to due some random factor)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;I have &lt;a href=&quot;https://mathematica.stackexchange.com/questions/216/adaptive-sampling-for-slow-to-compute-functions-in-2d/&quot;&gt;asked this already on Mathematica.SE&lt;/a&gt; but I can't link to it because it's still in private beta.  This question here is about the method, not the implementation.&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Reply to @suki&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Is this the type of division you suggest, i.e. putting a new point in the middle of the triangles?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/3uGtd.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Z7leE.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/J63os.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/N6Nw2.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My concern here is that it seems to require special handling at the edges of the region, otherwise it will give very long and very thin triangles, as shown above.  Did you correct for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A problem which appear both with the method I describe and with @suki's suggestion to put subdivide based on triangles and put the subdivision points inside the triangle is that when there are discontinuities (as in my problem), recomputing the Delaunay triangulation after a step may cause triangles to change and perhaps some big triangles to appear which have different function values in the three vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are two examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nRPwi.png&quot; alt=&quot;ex1&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/208bP.png&quot; alt=&quot;ex2&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first shows the end result when sampling around a straight discontinuity.  The second shows the sampling point distribution for a similar case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What simple ways are there to avoid this?  Currently I am simply subdividing those egdes that disappear after a retriangulation, but this feels like a hack and needs to be done with care as in the case of symmetrical meshes (like a square grid) there are several valid Delaunay triangulations, hence edges might change randomly after retriangulation.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2012-02-28T13:16:17.703" Title="What simple methods are there for adaptively sampling a 2D function?" Tags="|adaptive-mesh-refinement|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="924" PostTypeId="1" CreationDate="2012-01-22T11:38:34.873" Score="8" ViewCount="2168" Body="&lt;p&gt;In Finite Element theory, and other methods in scientific computing for PDEs, one uses meshes which fulfill several regularity criteria, many of them being equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is of interest to have notions of shape-regularity in arbitrary dimensions. While some authors use this as if there were full analogies of the two-dimensional theory to higher dimensions, it seems no one has ever before established this rigorously. Do you know a reference on this?&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="1804" LastEditDate="2012-10-30T14:24:23.580" LastActivityDate="2012-12-29T15:32:09.857" Title="Shape regularity in higher dimensions" Tags="|finite-element|hpc|computational-geometry|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="926" PostTypeId="1" AcceptedAnswerId="937" CreationDate="2012-01-22T13:36:37.620" Score="25" ViewCount="20263" Body="&lt;p&gt;I want to learn MATLAB programming so that I can conduct some researh/analysis on my own and also, so that I can study/modify some MATLAB scripts that I have found online etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem is that I can't afford MATLAB. GNU Octave, from what I have heard, is quite MATLAB compatable. The challenge with Octave though is that the documentation is very sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So MATLAB on one hand, is prohibitively expensive (-ve), but has lots of good documentation, tutorials and scripts online (+), while Octave is free (as in beer), even though (understandably) it lacks comprehensive documentation, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking of downloading and installing Octave, but using MATLAB documentation (and scripts) to help me get going. I am aware that there are some differences between Octave and MATLAB - what I'm trying to work out is whether the differences are large enough to thwart my efforts to (effectively) &quot;learn MATLAB by using Octave&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any constructive feedback welcome.&lt;/p&gt;&#xA;" OwnerUserId="749" OwnerDisplayName="Homunculus Reticulli" LastEditorUserId="123" LastEditDate="2012-01-25T06:37:15.053" LastActivityDate="2022-12-02T23:02:07.677" Title="Is it possible to use Octave to learn MATLAB programming?" Tags="|software|matlab|languages|octave|" AnswerCount="12" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="927" PostTypeId="2" ParentId="926" CreationDate="2012-01-22T14:54:38.163" Score="9" Body="&lt;p&gt;For generally learning M-language programming and how MATLAB works, yes, Octave is just fine. If you need a particular toolbox as part of your work, though, and no one has implemented a free version of it, then you're out of luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A student edition of MATLAB isn't that expensive. If you're at a university, it's even possible that they have a site license. You should look into both possibilities. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-01-22T14:54:38.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="928" PostTypeId="2" ParentId="926" CreationDate="2012-01-22T15:27:06.707" Score="5" Body="&lt;p&gt;Practical example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use MATLAB at work, and I took Stanford's &lt;a href=&quot;http://www.ml-class.org/course/auth/welcome&quot; rel=&quot;nofollow&quot;&gt;free online machine learning class&lt;/a&gt; last semester.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did all the homework in octave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only noticed 2 differences (I was only using core functionality):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Octave is a little more flexible about what sorts of expressions can be indexed. So if you take advantage of that, your code won't be portable.&lt;/li&gt;&#xA;&lt;li&gt;In my installation the &lt;a href=&quot;http://www.mathworks.com/help/techdoc/ref/pcolor.html&quot; rel=&quot;nofollow&quot;&gt;pcolor()&lt;/a&gt; and image() plots don't work for reasonably sized arrays, like 1000x1000, small ones, like 50x50, work fine. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="747" LastEditorUserId="276" LastEditDate="2012-01-25T12:46:30.283" LastActivityDate="2012-01-25T12:46:30.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="929" PostTypeId="2" ParentId="926" CreationDate="2012-01-22T15:31:19.990" Score="18" Body="&lt;p&gt;GNU Octave is &quot;mostly compatible with Matlab&quot;, certain subtleties means not all scripts are portable from MATLAB to Octave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth reading the &lt;a href=&quot;https://www.gnu.org/software/octave/doc/interpreter/&quot;&gt;documentation for the language&lt;/a&gt; and/or compatibility notes in the &lt;a href=&quot;https://www.gnu.org/software/octave/FAQ.html#MATLAB-compatibility&quot;&gt;FAQs&lt;/a&gt; or on &lt;a href=&quot;https://en.wikibooks.org/wiki/MATLAB_Programming/Differences_between_Octave_and_MATLAB&quot;&gt;wikibooks&lt;/a&gt;. There are also &lt;a href=&quot;http://octave.org/wiki/index.php?title=FAQ#Porting_programs_from_Matlab_to_Octave&quot;&gt;porting notes&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://octave.sourceforge.net/packages.php&quot;&gt;Packages&lt;/a&gt; similar to MATLAB toolboxes exist, but you will need to check them out to work out how similar they are. Also there are attempts at conversion scripts that take care of the differences between the two languages, but to my knowledge none are perfect.&lt;/p&gt;&#xA;" OwnerUserId="656" LastEditorUserId="27" LastEditDate="2012-01-22T15:58:48.757" LastActivityDate="2012-01-22T15:58:48.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="930" PostTypeId="1" CreationDate="2012-01-22T16:15:58.250" Score="13" ViewCount="919" Body="&lt;p&gt;Usually one would consult a paper or book to find integration points and weights for unit triangle and tetrahedra. I am  looking for a method to automatically compute such points and weights. The following &lt;em&gt;Mathematica&lt;/em&gt; code example computes integration weights and points for unit line (quad/hexahedron) elements:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;unitGaussianQuadraturePoints[points_] := &#xA;  Sort[x /. &#xA;    Solve[Evaluate[LegendreP[points, x] == 0], {x}], ! &#xA;     OrderedQ[N[{#1, #2}]] &amp;amp;];&#xA;&#xA;unitGaussianQuadratureWeights[points_] := &#xA;  Module[{gps, f, int, integr, vars, eqns}, &#xA;   gps = unitGaussianQuadraturePoints[points];&#xA;   f[0, 0] := 1;&#xA;   f[0., 0] := 1.;&#xA;   f[x_, n_] := x^n;&#xA;   int = Integrate[f[x, #], x] &amp;amp; /@ Range[0, points - 1];&#xA;   integr = Subtract @@@ (int /. x :&amp;gt; {1, -1});&#xA;   vars = Table[Unique[c], {Length[gps]}];&#xA;   eqns = &#xA;    Table[Plus @@ Thread[Times[vars, f[#, i - 1] &amp;amp; /@ gps]] == &#xA;      integr[[i]], {i, points}];&#xA;   Return[(vars /. Solve[eqns, vars])];];&#xA;&#xA;&#xA;unitGaussianQuadratureWeights[2]&#xA;&#xA;{{1, 1}}&#xA;&#xA;unitGaussianQuadraturePoints[2]&#xA;&#xA;{1/Sqrt[3], -(1/Sqrt[3])}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am looking for a paper/book that describes algorithmically how this is done for triangles and/or tetrahedra. Can someone point me to some information about this. Thanks.&lt;/p&gt;&#xA;" OwnerDisplayName="user530" LastEditorUserId="123" LastEditDate="2012-01-22T19:50:01.290" LastActivityDate="2019-01-23T12:24:42.930" Title="Automatic generation of integration points and weights for triangles and tetrahedra" Tags="|symbolic-computation|reference-request|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="931" PostTypeId="2" ParentId="930" CreationDate="2012-01-22T17:23:04.990" Score="8" Body="&lt;p&gt;The &lt;a href=&quot;http://nines.cs.kuleuven.be/ecf/&quot;&gt;Encyclopaedia of Cubature Formulas&lt;/a&gt; has an extensive list of techniques for this purpose and associated references.&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2012-01-22T17:23:04.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="932" PostTypeId="2" ParentId="926" CreationDate="2012-01-22T17:28:35.027" Score="5" Body="&lt;p&gt;There are some subtle differences in Octave's interpretation of the MATLAB's programming language. Octave uses &quot;&lt;code&gt;endif&lt;/code&gt;&quot; and &quot;&lt;code&gt;endwhile&lt;/code&gt;&quot; to close &lt;code&gt;if&lt;/code&gt; and &lt;code&gt;while&lt;/code&gt; statements respectively. Octave also allows you to declare functions on the command line. It should, however, parse anything that MATLAB parses too, so if you stick to MATLAB's programming language, you should be fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be aware that the biggest differences between the two programs is in the built-in functions that do most of the numerical heavy-lifting, e.g. &quot;&lt;code&gt;quad&lt;/code&gt;&quot;, &quot;&lt;code&gt;ode15s&lt;/code&gt;&quot; and such. All of these functions should, however, be well documented when using the &quot;&lt;code&gt;help&lt;/code&gt;&quot; command. There are also mailing lists for Octave should you have more specific questions that aren't covered by the online help.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="276" LastEditDate="2012-01-25T12:47:46.007" LastActivityDate="2012-01-25T12:47:46.007" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="933" PostTypeId="2" ParentId="930" CreationDate="2012-01-22T17:31:32.893" Score="3" Body="&lt;p&gt;Here is a paper &lt;a href=&quot;http://journal.library.iisc.ernet.in/vol200405/paper6/rathod.pdf&quot; rel=&quot;nofollow&quot;&gt;http://journal.library.iisc.ernet.in/vol200405/paper6/rathod.pdf&lt;/a&gt; that describes how to map the unit triangle to the standard 2-square in order to calculate the weights and sampling points for the triangle in terms of Gauss-Legendre points for the standard 2-square.&lt;/p&gt;&#xA;" OwnerUserId="739" LastActivityDate="2012-01-22T17:31:32.893" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="934" PostTypeId="2" ParentId="923" CreationDate="2012-01-22T18:30:23.897" Score="10" Body="&lt;p&gt;I worked on a problem similar to this a while back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the main difference between between our implementations is that I was choosing where to add points based on the triangles, not the edges. I also choose new points inside the triangles instead of on the edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the feeling that adding points inside the triangles would make it more efficient by giving a small increase in the average distance from old points to the new one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway another nice thing about using triangles instead of edges is that it gives an estimate of the gradient vector, instead of the slope along this particular edge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my matlab code I used a base class to take care of most of the machinery, with a few abstract methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;weight(self)&lt;/code&gt; to decided the priority for which triangles to subdivide next.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;choosePoints(self,npoints = &quot;auto&quot;)&lt;/code&gt; to decide new points to evaluate based on the weight of each triangle.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I found this setup very flexible:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;setting a sub-class's &lt;code&gt;weight()&lt;/code&gt; function to the area of the triangle produces a constant mesh density.&lt;/li&gt;&#xA;&lt;li&gt;setting &lt;code&gt;weight()&lt;/code&gt; to calculate the average function value times the area of the triangle gives a sort of quasi-random probability sampling.&lt;/li&gt;&#xA;&lt;li&gt;using &lt;code&gt;var(triangle.zs)&lt;/code&gt; could do, for functions that have binary output, what I feel is a generalization of a bisection search to more than 1 dimension. &lt;/li&gt;&#xA;&lt;li&gt;using &lt;code&gt;area + var(triangle.zs)&lt;/code&gt; was pretty effective at putting a constant density everywhere, and an increased density along any slope (almost what you have now). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I used the variance of the z values, to approximate the importance of first order effects (slope) because the variance will never go to infinity like the slope can. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the last example the background density was good because I was searching for discontinuous blobs of high value in a space of low value. So it would slowly fill in the entire mesh and when it would find a blob it would concentrate on following the edge of the blob all the way around because of the high weight I put on the gradient (and that it only filled in the top &lt;code&gt;n&lt;/code&gt; triangles on each iteration). At the end I could know there were no (reasonably shaped) blobs (or holes in my blobs) of a size greater than the resulting background mesh density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like you I did get some bad points in my results, they weren't a problem for me because the error was such that if you re-ran nearby points they would probably give the correct answer. I'd just end up with blips of increased mesh density around my bad points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever you do I always recommend making the weights related to the triangle size so that, all else being equal, large triangles are broken up first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe a solution for you is to take my approach one step further and instead of evaluating triangles based on the contents of that triangular cell, evaluate based on that one and all three adjacent triangles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That will contain enough information to get an estimate of the complete Hessian matrix. you can get it by doing a least squares fit of &lt;code&gt;z = c1*x + C2*y c11*x^2+c12*x*y+c22*y^2&lt;/code&gt; over all the vertexes in the triangles of interest (center the coordinate system on the triangle first).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't use the gradient or Hessian (those constants) directly because they'll go to infinity at a discontinuity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the sum-square-error of the z values relative to a planar approximation of those points would be a useful measure of how interesting second order effects are. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Updated:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That looks reasonable to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I never actually got around to special casing the edges. It bothered me a little but for what I was doing, it was enough to just start with a lot of points around the edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;more elegant would be to combine our two approaches, weighting edges and triangles. Then if an edge is too long, cut it in half... I like the way that concept generalizes to higher dimensions (but the numbers get big fast)... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But since you don't expect the main body of the mesh to have high aspect ratio triangles, so you could use a function like Matlab's &lt;a href=&quot;http://www.mathworks.com/help/techdoc/ref/trirep.freeboundary.html&quot; rel=&quot;nofollow&quot;&gt;freeboundary&lt;/a&gt; function to find the boundary, then run the the same algorithm in one less dimension on the boundary. If done right, on a cube for example, you could get the same mesh density on the edges, on the faces, and inside the cube. Interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that I never found a good solution for was the fact that my version would never explore outside the convex hull of the initial point set.&lt;/p&gt;&#xA;" OwnerUserId="747" LastEditorUserId="747" LastEditDate="2012-01-24T03:14:33.643" LastActivityDate="2012-01-24T03:14:33.643" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="935" PostTypeId="2" ParentId="926" CreationDate="2012-01-22T18:35:57.647" Score="12" Body="&lt;p&gt;Yes you can learn MATLAB via Octave.  But Octave syntax is less restrictive and more in line with modern scripting languages. MATLAB seems behind in this respect.  See this wiki link &lt;a href=&quot;http://en.wikibooks.org/wiki/MATLAB_Programming/Differences_between_Octave_and_MATLAB&quot;&gt;MATLAB Programming/Differences between Octave and MATLAB&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another major difference to me were the availability of certain libraries for MATLAB but not for Octave.&lt;/p&gt;&#xA;" OwnerUserId="743" LastEditorUserId="276" LastEditDate="2012-01-25T12:47:12.043" LastActivityDate="2012-01-25T12:47:12.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="937" PostTypeId="2" ParentId="926" CreationDate="2012-01-22T19:45:37.553" Score="24" Body="&lt;p&gt;&lt;em&gt;Disclaimer: I sometimes get annoyed when somebody tries to tell me what they think I ought to do rather than answering the question I asked.  But I'm going to take a risk and suggest an alternative to you.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would suggest looking at &lt;strong&gt;Python's scientific computing packages: &lt;a href=&quot;https://numpy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;numpy&lt;/a&gt;, &lt;a href=&quot;https://matplotlib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;matplotlib&lt;/a&gt;, and &lt;a href=&quot;https://scipy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy&lt;/a&gt;&lt;/strong&gt;.  Together, they provide you most of the core functionality of MATLAB (in some cases they give you more than MATLAB).  They are free and open source, and nowadays may have a wider user base than Octave.  Much of the scientific computing infrastructure in Python is supported by &lt;a href=&quot;https://www.enthought.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enthought Inc.&lt;/a&gt;, and I would recommend that you install their free &lt;a href=&quot;http://enthought.com/products/epd_free.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enthought Python Distribution&lt;/a&gt;, which includes all three packages I mentioned and more.&lt;/p&gt;&#xA;&lt;p&gt;Of course, there are features of MATLAB (particularly in the toolboxes, but Octave doesn't have those) that Python lacks.  But I use Python in combination with lower-level languages for most of my work, and it allows me to program in a language almost as convenient as MATLAB, interface with a host of useful packages that MATLAB doesn't have, and to run the same easy-to-read code on a supercomputer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: you may also want to try the &lt;a href=&quot;https://www.anaconda.com/products/distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anaconda Python distribution&lt;/a&gt; from Continuum Analytics.  There is a free version that includes all the above packages and much more.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="22284" LastEditDate="2022-12-02T23:02:07.677" LastActivityDate="2022-12-02T23:02:07.677" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="938" PostTypeId="1" AcceptedAnswerId="950" CreationDate="2012-01-22T20:03:22.593" Score="17" ViewCount="404" Body="&lt;p&gt;In the numerical methods literature, many research papers consist of a description of a new algorithmic variation, followed by a few test problems comparing the new method with one or two existing methods.  This makes it difficult to determine&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How the new method would perform on &lt;em&gt;other problems&lt;/em&gt; of interest&lt;/li&gt;&#xA;&lt;li&gt;How the new method would compare to yet &lt;em&gt;other existing methods&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, both of these questions are usually critical for someone deciding whether to adopt the new method.  In order to improve this situation, it seems desirable to, if possible, get people to run a large set of test problems (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/46/where-can-one-obtain-good-data-sets-test-problems-for-testing-algorithms-routine&quot;&gt;see this question&lt;/a&gt;) with their method and catalog the results in a database for comparison with other methods.  A scientist or engineer could then query the database, perhaps indicating what solution/problem characteristics are most important to them (accuracy, efficiency, qualitative properties, etc.), and get a quantitative comparison of the performance of all methods in the database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize that there are many difficulties in implementing this idea (chiefly: getting people to actually run all those problems, and measuring computational cost).  But my question is: &lt;strong&gt;Does there exist such a database of results?&lt;/strong&gt; (say in some particular subfield) or has an approach like this been implemented anywhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Both of the answers posted so far are about databases of problems.  I'm asking about databases of results.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-02-03T06:43:54.673" Title="Databases of results for numerical codes" Tags="|pde|testing|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="939" PostTypeId="2" ParentId="926" CreationDate="2012-01-22T21:07:34.597" Score="7" Body="&lt;p&gt;I've used Octave and MATLAB interchangeably on the same pieces of code in the past; I've had no problems with compatibility.  The main differences as far as I can tell are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;MATLAB can be byte-compiled, and is slightly faster.&lt;/li&gt;&#xA;&lt;li&gt;Octave can use either &lt;code&gt;#&lt;/code&gt; or &lt;code&gt;%&lt;/code&gt; for comments, Matlab only uses &lt;code&gt;%&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Octave produces &lt;strong&gt;much&lt;/strong&gt; better plots.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2013-01-12T20:04:35.070" LastActivityDate="2013-01-12T20:04:35.070" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="940" PostTypeId="1" AcceptedAnswerId="1164" CreationDate="2012-01-22T21:40:36.737" Score="8" ViewCount="487" Body="&lt;p&gt;I'm looking into using artificial neural networks (ANN) to predict the reaction rates in my fluid instead of solving the full system of stiff ODEs. Some people from my lab have already done some work on that so I don't start from scratch but I am having problems with my applications. One of them I think relates to the quality of my dataset for training. We usually extract training data from CFD simulations that are either 1D/2D/3D. No matter what, we end up with a multidimensional array of data to feed to the neural network. To give you an idea of the size of the problem, I am looking into training 8 nets with 10 inputs and 1 output for each. I feel like a training set of about 100,000 points would be reasonable but the problem is that these 100,000 points need to cover a specific region of my multi-dimensional space. Simply extracting the whole flowfield from my simulations is not very good for 2 reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For each snapshot, only a small portion of the points lie in the region where I need a high sampling to make sure my training is accurate&lt;/li&gt;&#xA;&lt;li&gt;As I compile snapshots together, I end up with many near-duplicate points which (I believe) have a negative effect on my ANN training by a) biasing the training by putting more weight on these regions b) adding unnecessary points.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So I've been trying to filter the points I record before including them in my training set. As I see it, that involves checking whether a new point is within a certain n-dimensional radius of every point of my dataset. This brute force approach, which barring a few tricks scales like n^2, works so-so for extracting 10,000 points out of 100,000 (takes 30 min say) but breaks down as I increase the sizes and numbers of the snapshots... Clearly, there must be a more clever way of doing this, but I am not sure in which direction to start looking. I first tried with python and could move to FORTRAN to speed things up but I feel like I should look for a better strategy first. Is my only hope some kind of k-d tree? I have little to no experience with them and the problem that I see is that my tree will grow as I build my dataset and this can only increase the complexity. Would a python k-d tree library suit my need? Should I move to FORTRAN given the size of my problem? Any advice is appreciated, thank you !&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2019-05-22T18:19:55.727" Title="Filtering a dataset to get a more uniform distribution for neural network training" Tags="|performance|fortran|python|data-management|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="941" PostTypeId="5" CreationDate="2012-01-22T21:42:37.637" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-22T21:42:37.637" LastActivityDate="2012-01-22T21:42:37.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="942" PostTypeId="4" CreationDate="2012-01-22T21:42:37.637" Score="0" Body="Questions related to Artificial Neural Networks (ANN)" OwnerUserId="95" LastEditorUserId="95" LastEditDate="2012-01-23T05:26:27.463" LastActivityDate="2012-01-23T05:26:27.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="943" PostTypeId="1" AcceptedAnswerId="947" CreationDate="2012-01-23T03:34:55.007" Score="93" ViewCount="25403" Body="&lt;p&gt;So I've got a decent head for what problems I work with are best one in serial, and which can be managed in parallel. But right now, I don't have much of an idea of what's best handled by CPU-based computation, and what should be offloaded to a GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know its a basic question, but much of my searching gets caught in people clearly advocating for one or the other without really justifying &lt;em&gt;why&lt;/em&gt;, or somewhat vague rules of thumb. Looking for a more useful response here.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2021-01-30T01:05:37.720" Title="What kinds of problems lend themselves well to GPU computing?" Tags="|gpu|" AnswerCount="10" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="944" PostTypeId="2" ParentId="870" CreationDate="2012-01-23T05:25:19.180" Score="1" Body="&lt;p&gt;I don't know of any MATLAB code, but I believe the &lt;a href=&quot;http://biocircuits.ucsd.edu/pbryant/LyapOde4.zip&quot; rel=&quot;nofollow&quot;&gt;LyapOde software&lt;/a&gt; linked at the bottom of &lt;a href=&quot;http://biocircuits.ucsd.edu/pbryant/&quot; rel=&quot;nofollow&quot;&gt;Paul Bryant's page&lt;/a&gt; may implement what you want.  It is written in C.  From its documentation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The software also supports the calculation of Conditional Lyapunov Exponents or CLEs (see Pecora et al, Chaos Vol. 7, No. 4, (1997), 520). The CLEs are the Lyapunov exponents of a response system that receives unidirectional synchronizing stimulus from an identical drive system. Three types of stimulus are considered: continuous coupling, periodic impulse coupling, and substitution (full or fractional). More details are given in the discussion section below. The synchronizing stimulus can be applied to one or more of the variables. When any of the CLEs are positive or zero, they can be calculated in both synchronized and unsynchronized mode which typically give differing results. In the latter case the drive and response systems are allowed to freely deviate away from synchronization.&#xA;  Currently the software includes the equations for the standard Lorenz model, the Rossler system, the Rossler hyperchaos system, the 5-variable version of the Lorenz cyclic 1996 model, the Colpitts oscillator, and the driven Van der Pol oscillator.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-05-27T05:42:47.537" LastActivityDate="2015-05-27T05:42:47.537" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="946" PostTypeId="2" ParentId="943" CreationDate="2012-01-23T06:03:47.450" Score="23" Body="&lt;p&gt;Problems which have a high arithmetic intensity and regular memory access patterns are typically easy(ier) to implement on GPUs, and perform well on them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic difficulty in having high performance GPU code is that you have a ton of cores, and you want them to all be utilized to their full potency as much as possible. Problems which have irregular memory access patterns or do not have high arithmetic intensity make this difficult: either you spend a long time communicating results or you spend a long time fetching stuff from memory (which is slow!), and not enough time crunching numbers. Of course the potential for concurrency in your code is critical to its ability to be implemented well on GPU as well.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2012-01-23T06:03:47.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="947" PostTypeId="2" ParentId="943" CreationDate="2012-01-23T06:04:24.750" Score="68" Body="&lt;p&gt;GPU hardware has two particular strengths: raw compute (FLOPs) and memory bandwidth.  Most difficult computational problems fall into one of these two categories.  For example, dense linear algebra (A * B = C or Solve[Ax = y] or Diagonalize[A], etc) falls somewhere on the compute/memory bandwidth spectrum depending on system size.  Fast Fourier transforms (FFT) also fit this mold with high aggregate bandwidth needs.  As do other transformations, grid/mesh-based algorithms, Monte Carlo, etc.  If you look at the NVIDIA SDK &lt;a href=&quot;http://developer.nvidia.com/cuda-cc-sdk-code-samples&quot; rel=&quot;noreferrer&quot; title=&quot;Code Examples&quot;&gt;code examples&lt;/a&gt;, you can get a feel for the sorts of problems that are most commonly addressed.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the more instructive answer is to the question `What kinds of problems are GPUs really bad at?'  Most problems that don't fall into this category can be made to run on the GPU, though some take more effort than others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problems that don't map well are generally too small or too unpredictable.  Very small problems lack the parallelism needed to use all the threads on the GPU and/or could fit into a low-level cache on the CPU, substantially boosting CPU performance.  Unpredictable problems have too many meaningful branches, which can prevent data from efficiently streaming from GPU memory to the cores or reduce parallelism by breaking the &lt;a href=&quot;http://en.wikipedia.org/wiki/SIMD&quot; rel=&quot;noreferrer&quot;&gt;SIMD&lt;/a&gt; paradigm (see '&lt;a href=&quot;https://stackoverflow.com/questions/5897454/conditionals-in-gpu-programming&quot;&gt;divergent warps&lt;/a&gt;').  Examples of these kinds of problems include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Most graph algorithms (too unpredictable, especially in memory-space)&lt;/li&gt;&#xA;&lt;li&gt;Sparse linear algebra (but this is bad on the CPU too)&lt;/li&gt;&#xA;&lt;li&gt;Small signal processing problems (FFTs smaller than 1000 points, for example)&lt;/li&gt;&#xA;&lt;li&gt;Search&lt;/li&gt;&#xA;&lt;li&gt;Sort &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="521" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-05-27T15:42:50.860" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="948" PostTypeId="2" ParentId="943" CreationDate="2012-01-23T08:00:59.677" Score="15" Body="&lt;p&gt;&lt;em&gt;This is not intended as an answer on its own but rather an addition to the other answers by &lt;a href=&quot;https://scicomp.stackexchange.com/a/947/149&quot;&gt;maxhutch&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/a/946/149&quot;&gt;Reid.Atcheson&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the best out of GPUs your problem does not only need to be highly (or massively) parallel, but also the core algorithm that will be executed on the GPU, should be as small as possible. In &lt;em&gt;OpenCL&lt;/em&gt; terms this is mostly referred as the &lt;em&gt;kernel&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more precise, the kernel should fit into the register of each multiprocessing unit (or &lt;em&gt;compute unit&lt;/em&gt;) of the GPU. The exact size of the register is dependent on the GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the kernel is small enough, the raw data of the problem needs to fit into the GPU's local memory (read: &lt;em&gt;local memory&lt;/em&gt; (OpenCL) or &lt;em&gt;shared memory&lt;/em&gt; (CUDA) of a compute unit). Otherwise even the high memory bandwidth of the GPU is not fast enough to keep the &lt;em&gt;processing elements&lt;/em&gt; busy all the time.&lt;br&gt;&#xA;Usually this memory is about 16 to 32 KiByte &lt;em&gt;big&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="746" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-01-24T14:16:58.567" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="949" PostTypeId="2" ParentId="891" CreationDate="2012-01-23T11:15:19.790" Score="2" Body="&lt;p&gt;In fact Jed Brown has cleared the question for me. What I'm doing now is just putting his words in the context. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Both 2d linear ODE systems from above are stiff (i. e. hard to solve with explicit methods) on relative big time intervals (e. g. [0,1]).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The linear systems with large stiffness ratio can be considered &quot;more stiff&quot; because &lt;em&gt;most likely&lt;/em&gt; one needs to integrate them on large time interval. This is due to slow components corresponding to the smallest eigenvalues: the solution slowly tends to the steady state, and this steady state is usually important to reach.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On the other hand, integration of systems with small stiffness ratio on large intervals is not interestng: in this case the steady state is reached very fast and we can just extrapolate it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks to all for this discussion!&lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2012-01-23T11:15:19.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="950" PostTypeId="2" ParentId="938" CreationDate="2012-01-23T11:47:02.380" Score="9" Body="&lt;p&gt;Of the criteria you've specified, I think the closest project that I know of would be the &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/matrices/&quot;&gt;University of Florida sparse matrix collection&lt;/a&gt;. People routinely use this data set to compare sparse linear algebra solvers, and you can filter by application, number of nonzeros, dimensions of matrix, and so on with a really nice web interface, MATLAB interface, or Java GUI. I've seen tables of these problems listed in papers along with solver run time comparisons to 4 to 8 linear algebra solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I agree that it would be useful to compile such databases, and furthermore, I think the UF sparse matrix collection approach for compiling the data is an excellent one, and would make a great start for anyone thinking about realizing this idea. Running all of the problems, in practice, doesn't seem like a major difficulty as long as you can get access to all of the solvers; if you have access to the solvers, and a reliable standard reference machine with all of the necessary software installed, then it should be a matter of running a script and collecting the data. The difficulty, in my mind, would be getting people to give you their software, if it isn't open source. If it's commercial, you could buy it, or even possibly get people to donate the software, which is something I know that happens in the optimization community when people are building portable software interfaces for commercial codes, such as in the &lt;a href=&quot;http://www.coin-or.org/&quot;&gt;COIN-OR&lt;/a&gt; project. But if it's research software that is neither commercial nor open source, then you need to convince people to buy into the endeavor, and they may not trust a third party to assess their software fairly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know that in optimization, there are downloadable databases of problems (&lt;a href=&quot;http://www.cuter.rl.ac.uk/index.html&quot;&gt;CUTEr&lt;/a&gt;&#xA;comes to mind) and books of test problems for optimization. I have seen people (for instance, I'm specifically thinking of a talk by Ruth Misener at AIChE 2011) compare their optimization solver versus other solvers on databases of problems in presentations; I'm not sure what gets released publicly. I know that there's a tradition in optimization for comparison on a large scale (many solvers, many problems); I just don't think there's an online database available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that I think is important is that we distinguish here between &lt;em&gt;methods&lt;/em&gt; and &lt;em&gt;software implementations&lt;/em&gt;. In scientific computing, we all talk about which methods are faster or slower based on things like computational complexity metrics, or our experiences with various problems. When it comes to measuring computational time quantitatively, however, unless one counts the number of FLOPs in a particular algorithm, one has to implement the algorithm in software and then measure the performance in some way (memory usage, wall clock time of execution, etc.). It makes sense to assess the performance of a method when looking at computational complexity or FLOP counts, because we don't need an implementation to measure such things, but the moment we're interested in actual wall clock run times, talking about methods is only useful as an abstract, colloquial device. (For instance, sparse direct solvers are fast on sufficiently small sparse problems with good fill-reducing heuristics, such as nested dissection.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I bring up this distinction between methods and software because in such a database, I could also see the possibility of tracking the improvement in software over time. So, for instance, with something like, say, PETSc, or PyCLAW, or whatever software is being tested, it would be interesting to see what problems are affected positively (or negatively!) by upgrades in the software. This could be useful for researchers trying to decide if it's worth any potential costs in money and manpower to upgrade their codes. Another reason such a distinction is important is because a good method can be implemented badly; I think this possibility contributes to the reticence people sometimes have in sharing their research codes. A database of implementations would be able to distinguish between good and bad implementations of the same method (or types of methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think whatever comes of this idea (and I hope something comes of it, and would be willing to contribute after my PhD), it's important to emphasize that distinction between software and methods, because if we're running test problems, we're going to be posting results for software.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-23T11:47:02.380" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="951" PostTypeId="2" ParentId="943" CreationDate="2012-01-23T15:15:37.857" Score="11" Body="&lt;p&gt;Probably a more technical addition to the previous replies: CUDA (i.e. Nvidia) GPUs can be described as a set of processors that work autonomously on 32 threads each. The threads in each processor work in lock-step (think SIMD with vectors of length 32). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the most tempting way to work with GPUs is to pretend that absolutely everything runs in lock-step, this is not always the most efficient way of doing things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your code does &lt;em&gt;not&lt;/em&gt; parallelize nicely/automatically to hundreds/thousands of threads, you may be able to break it down into individual asynchronous tasks that &lt;em&gt;do&lt;/em&gt; parallelize well, and execute those with only 32 threads running in lock-step. CUDA provides a set of atomic instructions which make it possible to implement &lt;a href=&quot;http://en.wikipedia.org/wiki/Mutual_exclusion&quot;&gt;mutexes&lt;/a&gt; which in turn allows the processors to synchronize among themselves and process a list of tasks in a &lt;a href=&quot;http://en.wikipedia.org/wiki/Thread_pool_pattern&quot;&gt;thread pool&lt;/a&gt; paradigm. Your code would then work much in the same way as it does on a multi-core system, just keep in mind that each core then has 32 threads of its own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a small example, using CUDA, of how this works&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/* Global index of the next available task, assume this has been set to&#xA;   zero before spawning the kernel. */&#xA;__device__ int next_task;&#xA;&#xA;/* We will use this value as our mutex variable. Assume it has been set to&#xA;   zero before spawning the kernel. */&#xA;__device__ int tasks_mutex;&#xA;&#xA;/* Mutex routines using atomic compare-and-set. */&#xA;__device__ inline void cuda_mutex_lock ( int *m ) {&#xA;    while ( atomicCAS( m , 0 , 1 ) != 0 );&#xA;    }&#xA;__device__ inline void cuda_mutex_unlock ( int *m ) {&#xA;    atomicExch( m , 0 );&#xA;    }&#xA;&#xA;__device__ void task_do ( struct task *t ) {&#xA;&#xA;    /* Do whatever needs to be done for the task t using the 32 threads of&#xA;       a single warp. */&#xA;    }&#xA;&#xA;__global__ void main ( struct task *tasks , int nr_tasks ) {&#xA;&#xA;    __shared__ task_id;&#xA;&#xA;    /* Main task loop... */&#xA;    while ( next_task &amp;lt; nr_tasks ) {&#xA;&#xA;        /* The first thread in this block is responsible for picking-up a task. */&#xA;        if ( threadIdx.x == 0 ) {&#xA;&#xA;            /* Get a hold of the task mutex. */&#xA;            cuda_mutex_lock( &amp;amp;tasks_mutex );&#xA;&#xA;            /* Store the next task in the shared task_id variable so that all&#xA;               threads in this warp can see it. */&#xA;            task_id = next_task;&#xA;&#xA;            /* Increase the task counter. */&#xA;            next_tast += 1;&#xA;&#xA;            /* Make sure those last two writes to local and global memory can&#xA;               be seen by everybody. */&#xA;            __threadfence();&#xA;&#xA;            /* Unlock the task mutex. */&#xA;            cuda_mutex_unlock( &amp;amp;tasks_mutex );&#xA;&#xA;            }&#xA;&#xA;        /* As of here, all threads in this warp are back in sync, so if we&#xA;           got a valid task, perform it. */&#xA;        if ( task_id &amp;lt; nr_tasks )&#xA;            task_do( &amp;amp;tasks[ task_id ] );&#xA;&#xA;        } /* main loop. */&#xA;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You then have to call the kernel with &lt;code&gt;main&amp;lt;&amp;lt;&amp;lt;N,32&amp;gt;&amp;gt;&amp;gt;(tasks,nr_tasks)&lt;/code&gt; to make sure that each block contains only 32 threads and thus fits in a single warp. In this example I also assumed, for simplicity, that the tasks do not have any dependencies (e.g. one task depends on the results of another) or conflicts (e.g. work on the same global memory). If this is the case, then the task selection becomes a bit more complicated, but the structure is essentially the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is, of course, more complicated than just doing everything on one large batch of cells, but significantly broadens the type of problems for which GPUs can be used.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-01-24T15:34:19.780" LastActivityDate="2012-01-24T15:34:19.780" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="952" PostTypeId="1" CreationDate="2012-01-23T15:55:59.880" Score="8" ViewCount="1171" Body="&lt;p&gt;The computation of eigenmodes of a semi-circular membrane reduces to the following eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2u=k^2u\;,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the region of interest is a semi-circle defined by $r\in[0,1]$ and $\varphi\in[0,\pi]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is appropriate to work in cylindrical coordinates, where the Laplacian is written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2u=\frac{\partial^2u}{\partial r^2}+\frac{1}{r}\frac{\partial u}{\partial r}+\frac{1}{r^2}\frac{\partial^2 u}{\partial\varphi^2}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Boundary conditions fix the value of $u$ at the boundary of the semi-circle, where $u=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we make a discretization of $u$ with $u_{ij}=u(r_i,\varphi_j)$, where $r_i=(i+\frac{1}{2})h_r$ and $\varphi_j=(j+\frac{1}{2})h_\varphi$&#xA;$i,j=0\dots N-1$ and $h_r=1/N$, $h_r=\pi/N$. This is a &lt;em&gt;centered&lt;/em&gt; mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then use a finite difference approximation for the Laplacian and obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{eqnarray} \nabla^2u&amp;amp;\approx&amp;amp; &#xA;\frac{u_{i+1,j}-2u_{i,j}+u_{i-1,j}}{h_r^2}+\frac{1}{ih_r}\frac{u_{i+1,j}-u_{i-1,j}}{2h_r}+ \\ &#xA;&amp;amp;&amp;amp;+\frac{1}{(ih_r)^2}\frac{u_{i,j+1}-2u_{i,j}+u_{i,j-1}}{h_{\varphi}^2}\\&#xA;&amp;amp;=&amp;amp;k^2u_{ij} \end{eqnarray}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{eqnarray}&#xA;&amp;amp;&amp;amp;u_{i+1,j}\left(1+\frac{1}{2i}\right)+u_{i-1,j}\left(1-\frac{1}{2i}\right)\\ &#xA;&amp;amp;&amp;amp;+\frac{1}{i^2h_{\varphi}^2}\left(u_{i,j+1}+u_{i,j-1}\right)+u_{i,j}\left(-2-\frac{2}{i^2h_{\varphi}^2}-k^2h_r^2\right)=0\;.&#xA;\end{eqnarray}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because our mesh is centered we have to make the following replacement in the above equation: $i\to i+\frac{1}{2}$. This replacement also helps us get rid of the coordinate singularity for $i=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Boundary conditions at $\varphi=0,\pi$ and $r=0,1$ can be all handled with the same &lt;em&gt;trick&lt;/em&gt;, where we set at the boundary&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{i,j-1}=-u_{i,j}$$&#xA;$$u_{i,j+1}=-u_{i,j}$$&#xA;$$u_{i-1,j}=-u_{i,j}$$&#xA;$$u_{i+1,j}=-u_{i,j}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From $u_{ij}$ we then form a vector $\vec v$ and get a classical eigenvalue problem for a matrix $\rm A$, which is carefully formed from above equations&#xA;$${\rm A}{\vec v}=k^2h_r^2{\vec v}\;.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix is an unsymmetric real matrix and eigenvalues and eigenvectors can be obtained with a routine &lt;code&gt;dgeev&lt;/code&gt; from LAPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analytical solutions can easily be obtained by the method of separation of variables&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(r,\varphi)=R(r)\Phi(\varphi)\;.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(r,\varphi)_{nm}=\sin(n\varphi)J_n\left(\frac{\xi_n^{(m)}}{R}r\right)\;,$$&#xA;where $J_n$ is a (cylindrical) Bessel function of the first kind of order $n$ and $\xi_n^{(m)}$ is the $m$-th zero of $J_n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eigenvalues and frequencies are&#xA;\begin{equation}&#xA; \omega_{nm}=\sqrt{-k^2}=\frac{\xi_n^{(m)}}{R}\;.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that the numerical solution obtained by the described procedure does not match the analytical one. The difference is around $r=0$, so this boundary probably isn't correctly accounted for. My results can be seen in the following plots of both analytical and numerical solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the plot of the analytical solution for the first eigenfunction:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wn8Bq.png&quot; alt=&quot;Analytical solution for the first eigenfunction.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following plot shows the comparison of numerical results for three different discretizations, as far as my computational resources allow me to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4dCm4.png&quot; alt=&quot;Comparison of numerical solutions at different discretizations $N$.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next plot shows the $N^2$ dependence of differences between numerical and analytical solutions, normalized as $L^2(\vec{u}-\vec{u}_{analytical})/N^2$ in logarithmic scale. The slope of the linear regression is $0.5$, which means that absolute error is linearly decreasing with $N$.&#xA;This linear accuracy is not surprising, since the boundary conditions were satisfied to first order only, though other difference approximations were second order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rxTLH.png&quot; alt=&quot;Normalized absolute error as a function of number of grid points $N^2&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="764" LastEditorUserId="764" LastEditDate="2012-02-06T09:11:03.573" LastActivityDate="2012-07-02T09:25:29.093" Title="Laplacian eigenmodes on a semi-circular region with finite-difference method" Tags="|pde|finite-difference|eigensystem|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="953" PostTypeId="1" CreationDate="2012-01-18T21:59:42.487" Score="4" ViewCount="568" Body="&lt;p&gt;I'm talking about the vanilla sudoku game, with 9x9 grids equally split into 9 regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried a few approaches to estimate the probability that a specific number is in a specific location, but I can't seem to find the right pattern about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have this (partial) grid:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ORisC.png&quot; alt=&quot;Partial sudoku grid&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of calculation or method would help me find the probabilities of having a 8 in any of the free locations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I'd think there is a 50% chance of there being an 8 in any of the free locations of the two leftmost squares, but I'm not too sure what to think anymore with that rightmost almost-empty square.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I do realize that this specific example has multiple solutions; I couldn't come up with one that had just one solution but wasn't trivially easy to solve.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; I also realize that since 'good' sudokus only have one solution, as Thomas Andrews noted, the probability that a certain location contains an 8 is either 0 or 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, let's assume that I can determine the value of a square with 100% certainty &lt;em&gt;if and only if&lt;/em&gt; it can be found through the &lt;a href=&quot;http://www.su-doku.net/tech.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;naked single or hidden single techniques&lt;/a&gt; (that is, the number is either the only possible option for a square, or there is no other place where the number can be). Those two techniques are only enough to solve the most basic sudokus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I chose to observe the grid with those two techniques only, even though I'd get stuck at some state, it would still be possible to enumerate several successor states that would seem legal at first, and very obviously, many of them would have overlapping results. For instance, taking the partial grid above, several successor states would have an 8 in the second square. And by counting every single occurrence of the 8 value in the second square, divided by the number of seemingly legal successor states, I would get what I call the probability of an 8 being there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that, even for a computer, counting those is sloooow. So I am wondering what kind of maths I could use the get the right answer, without resorting to the manual count, just knowing the constraints.&lt;/p&gt;&#xA;" OwnerDisplayName="zneak" LastEditorUserId="16" LastEditDate="2012-01-25T02:13:45.997" LastActivityDate="2012-01-25T02:13:45.997" Title="What is the probabilistic model behind sudoku grids?" Tags="|combinatorics|constraints|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="954" PostTypeId="2" ParentId="953" CreationDate="2012-01-18T22:39:37.550" Score="3" Body="&lt;p&gt;I'm guessing that the probability you're looking for is &quot;given all possible sudoku puzzles that are compatible with this row, what percentage of them have an 8 in this single location.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the puzzle you've given, I would tend to 0.5 to the four locations in the left and center square (these are obviously correlated), and 1/3 to the three middle row locations in the right most 3x3 square (which are again correlated).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to figure out these numbers exactly would require a lot of work.&lt;/p&gt;&#xA;" OwnerDisplayName="Carl Brannen" LastActivityDate="2012-01-18T22:39:37.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="956" PostTypeId="2" ParentId="392" CreationDate="2012-01-23T18:39:44.763" Score="5" Body="&lt;p&gt;No one has spelt this out very clearly, so I think it's worth saying:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the open source licenses (notably: the GPL) are &lt;a href=&quot;http://en.wikipedia.org/wiki/Viral_license&quot;&gt;&quot;viral&quot;&lt;/a&gt; in the sense that whenever the code is used in a new project, the new project must be licensed in the same way.  Also, the code cannot be linked to (in certain ways) differently licensed code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One practical consequence is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I you implement a new numerical method in C, the license will not allow calling it from such common software like MATLAB or Mathematica&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you implement a new image processing algorithm, the license will not allow making a Photoshop plugin out of it&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;and so on ...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This will not only prevent commercial re-use, but also convenient re-use by other academics (if they use closed software), and if someone builds on top of your code, it will prevent them from giving their work away in a &quot;do-what-you-will-with-it&quot; way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is something you must consider before you finish formulating your license.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I put it this way because I have met people (not very familiar with open source licenses) who were not aware of this, or have not considered it from this angle.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;(This is just personal opinion, but I believe that applying such restrictions to work that comes from (publicly funded) academia is not appropriate.  So I either keep the code, or I give it away.)&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2012-01-23T19:31:06.513" LastActivityDate="2012-01-23T19:31:06.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="957" PostTypeId="2" ParentId="938" CreationDate="2012-01-23T19:53:03.920" Score="4" Body="&lt;p&gt;There has been an attempt to create a list of &quot;difficult&quot; integrals to compute numerically (see papers by Walter Gautschi, Robert Piessens and others) but no formal database. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the field of ODEs, there is the &lt;a href=&quot;http://www.dm.uniba.it/~testset/testsetivpsolvers/&quot; rel=&quot;nofollow&quot;&gt;IVP test set&lt;/a&gt;, a test set for Initial Value Problems. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="276" LastEditDate="2012-01-23T20:00:57.460" LastActivityDate="2012-01-23T20:00:57.460" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="959" PostTypeId="2" ParentId="898" CreationDate="2012-01-23T21:42:03.297" Score="5" Body="&lt;p&gt;I use Mathematica a lot for this kind of things...&#xA;Animation can be made very easily:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Manipulate[&#xA;  Plot[Sin[x + t] + Cos[0.3 x + t], {x, 0, 20},PlotRange -&amp;gt; {All, {-2, 2}}],&#xA;  {t, 1, 10}&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives you an interactive plot with a slider to drag for t.&lt;/p&gt;&#xA;&lt;p&gt;To save the animation as a movie just export it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Export[&amp;quot;movie.mov&amp;quot;,&#xA;  Table[Plot[Sin[x + t] + Cos[0.3 x + t], {x, 0, 20}, PlotRange -&amp;gt; {All, {-2, 2}}], &#xA;    {t, 1, 10, 0.1}&#xA;  ]&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/1fhfZ.gif&quot; alt=&quot;moving wave&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="700" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-01-23T21:42:03.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="960" PostTypeId="2" ParentId="898" CreationDate="2012-01-23T22:38:20.540" Score="0" Body="&lt;p&gt;For quick viz nothing beats gnuplot ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in `seq 1 100`; do echo &quot;set pm3d; set zrange [1:5000]; splot $i*x**2+$i*y**2&quot;; done | gnuplot&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="471" LastActivityDate="2012-01-23T22:38:20.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="961" PostTypeId="2" ParentId="938" CreationDate="2012-01-23T23:12:04.330" Score="3" Body="&lt;p&gt;I'm not familiar with such a database, but I think it would be very difficult to define in a useful way. A better system, in my opinion, is to release the code in a runnable form so that the user can extend to test as comprehensively as they like. In terms of community repositories, the &lt;a href=&quot;http://www.dm.uniba.it/~testset/testsetivpsolvers/&quot; rel=&quot;nofollow&quot;&gt;IVP test suite&lt;/a&gt; that GertVdE mentions also contains driver code to run several published ODE/DAE solvers on the test problems. In my publications, I try to release the code and all scripts used to create the analyze the output data and create the plots and tables appearing in the paper.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-23T23:12:04.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="962" PostTypeId="1" CreationDate="2012-01-23T21:11:48.610" Score="15" ViewCount="6287" Body="&lt;p&gt;There is a system of linear constraints ${\bf Ax} \leq {\bf b}$ . I wish to find a strictly&#xA;positive vector ${\bf x} &amp;gt; 0$ that satisfies these constraints. That means, $x_i &amp;gt; 0$ is&#xA;required for every component $x_i$ of ${\bf x}$.&#xA;How can I use an LP solver to find such a strictly positive vector ${\bf x}$ (or confirm that no&#xA;${\bf x}$ exists)? I cannot simply introduce another&#xA;system of constraints $x_i &amp;gt; 0$, because equality must always be permitted in&#xA;an LP—but I can use the LP solver several times, with changing&#xA;objective functions.&#xA;I think I should use the slack variable method, but I don't know how.&lt;/p&gt;&#xA;" OwnerUserId="773" OwnerDisplayName="sara" LastEditorUserId="123" LastEditDate="2012-01-24T11:54:07.757" LastActivityDate="2019-03-09T20:50:48.933" Title="Linear programming feasibility problem with strict positivity constraints" Tags="|optimization|linear-programming|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="964" PostTypeId="2" ParentId="923" CreationDate="2012-01-24T08:03:59.703" Score="0" Body="&lt;p&gt;I think the main problem in your heuristic is that you're considering the gradient in one dimension only and thus, in regions where dfdx is small but dfdy is large (as happens in the middle of your example), you will miss points when looking in the &quot;wrong&quot; dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One quick fix would be to consider sets of four points, taking their center of gravity, and approximating |dfdx|+|dfdy| using those four points. Another alternative is to take three points (i.e. a triangle) and take the maximum gradient of the surface over them.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-01-24T08:03:59.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="965" PostTypeId="2" ParentId="962" CreationDate="2012-01-24T09:22:02.713" Score="4" Body="&lt;p&gt;Feasibility problems are a slightly trickier game than general linear problems, which you have noted.  If you are solving approximately (by using a floating-point representation of the system of equations and constraints), it is legitimate to require $x_i &amp;gt;= \epsilon$, where $\epsilon$ is some very small numerical value, big enough to assure that $x_i$ actually lives in $\Re_+$, but small enough that a solution on the boundary is not considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might have to adjust $\epsilon$, and your solution will be qualified to &quot;within a factor of $\epsilon$&quot;, but this is sufficient for many situations.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-01-24T09:22:02.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="966" PostTypeId="2" ParentId="962" CreationDate="2012-01-24T09:32:43.600" Score="2" Body="&lt;p&gt;The answer given by aeismail is to be read carefully, regard the lp&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max( x_1 + x_2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_1 + x_2 \le 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_1,x_2 \ge 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has solutions $(1,0)$ and $(0,1)$ as well as others (degenerated). The generality of the question implys that these cases need to be treated as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are able to choose you objective function, you could try to modify it iteratively. E.g. Start with all coefficients for all variables equal to one, check wether you gain an approprate solution. If one variable is zero, rise it's coefficient and start again...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though I can not give a mathematical prove that this works (or a well defined procedure how to modify the objective function). I hope this helps :)&lt;/p&gt;&#xA;" OwnerDisplayName="user770" LastEditorUserId="81" LastEditDate="2012-01-24T18:46:01.270" LastActivityDate="2012-01-24T18:46:01.270" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="967" PostTypeId="2" ParentId="962" CreationDate="2012-01-24T10:37:08.693" Score="5" Body="&lt;p&gt;For an LP feasibility problem, I wouldn't use standard simplex. Standard primal (or dual) simplex algorithms will only visit the vertices of the feasible set of the primal (or dual) problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the feasible set of the problem you actually want to solve be $F = \{\mathbf{x}: \mathbf{Ax} \leq \mathbf{b}, \mathbf{x} &amp;gt; \mathbf{0}\}$, and suppose instead you were to solve the problem ($F_{\varepsilon}$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{alignat}{1}&#xD;&#xA;&amp;amp; \min_{\mathbf{x}} \quad 0 \\&#xD;&#xA;\textrm{s.t.} &amp;amp; \quad \mathbf{Ax} \leq \mathbf{b} \\&#xD;&#xA;&amp;amp; \quad \mathbf{x} \geq \varepsilon \cdot \mathbf{1}. &#xD;&#xA;\end{alignat}&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest approximant of the problem you want to solve is $F_{0}$, which admits slightly too many points. The problem is that the boundary of the positive orthant (i.e., the set $B = \{\mathbf{x}: \mathbf{x} \geq 0, \exists{i}: x_{i} = 0\}$ could make up part of the boundary of the feasible set of $F_{0}$. We'd like to exclude those points. One way of doing that is to do what Aron suggested, which is to set $\varepsilon$ to some small positive value, and then use any standard LP algorithm. This strategy is a good one, and will probably work in a wide variety of situations. However, it will fail if $F_{\varepsilon}$ is infeasible.  We know that $F_{0} \subset F \subset F_{\varepsilon}$ for all $\varepsilon &amp;gt; 0$ (to abuse notation and refer to a feasible set by its corresponding problem), and it's possible that even if you pick small positive values of $\varepsilon$, the LP solver will indicate that your LP is infeasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an LP solver, I'd use any interior point algorithm for LPs that starts with a feasible point and stays feasible, which is another way to exclude points in $B$. You needn't have to supply a feasible point to these algorithms; standard solvers will do it for you. Methods like affine scaling, potential reduction, and barrier methods set up auxiliary LPs that will find feasible solutions, and the iterates for these algorithms traverse the interior of the feasible region. You only need to locate one point in your feasible region, so as long as the auxiliary problems used by the LP solvers locate a feasible point for your problem, and that feasible point is strictly positive, you should be all right. If solving $F_{\varepsilon}$ fails for small positive values of $\varepsilon$, you might still be able to use these methods to locate a strictly positive feasible point within $F_{0}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't use simplex, though, because it will only explore the vertices of $F_{\varepsilon}$, which is exactly what you want to avoid doing.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-24T10:37:08.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="968" PostTypeId="2" ParentId="962" CreationDate="2012-01-24T11:49:23.977" Score="17" Body="&lt;p&gt;You can circumvent the problem of choosing a small &lt;span class=&quot;math-container&quot;&gt;$\epsilon&amp;gt;0$&lt;/span&gt; by being a bit more ambitious: Try to find &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Ax}\leq \mathbf{b}$&lt;/span&gt; and that the smallest entry in &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; is largest possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To that end, introduce a new variable&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{y} = \begin{bmatrix} \mathbf{x}\\ \epsilon\end{bmatrix}\in\mathbb{R}^{n+1}$$&lt;/span&gt;&#xA;(if &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; was in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;) and solve the following problem by an LP-solver&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\max_y [0 \dots 0\ 1]\cdot \mathbf{y}\quad \text{s.t.}\quad [A\ \mathbf{0}]\mathbf{y}\leq\mathbf{b}\quad \text{and}\quad  \mathbf{0}\leq \begin{bmatrix}1  &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1 \\ 0 &amp;amp; 1 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1  \\ &amp;amp; \ddots &amp;amp; &amp;amp;\vdots \\ 0 &amp;amp; \cdots &amp;amp; &amp;amp; 1 &amp;amp; -1 \end{bmatrix}\mathbf{y}.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a reformulation of the following problem:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\max \epsilon \quad\text{s.t}\quad \mathbf{Ax\leq b}\quad\text{and}\quad \mathbf{x}\geq\epsilon\mathbf{1}.$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="30190" LastEditDate="2019-03-09T20:50:48.933" LastActivityDate="2019-03-09T20:50:48.933" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="969" PostTypeId="1" CreationDate="2012-01-24T13:36:10.100" Score="6" ViewCount="309" Body="&lt;p&gt;A linear system of equations has the form $Ax = b$, where a matrix $A$ and a&#xA;vector $b$ are given, and I wish to find a solution vector $x$.&#xA;Suppose that the system $Ax = b$ has no feasible solution. Then I&#xA;wish to find a solution vector $x$ such that $Ax$ is “as close as possible” to $b$.&#xA;Of course, this notion is not clearly defined. Is there an LP formulation&#xA;of this problem?&lt;/p&gt;&#xA;" OwnerUserId="773" LastEditorUserId="123" LastEditDate="2012-01-25T06:38:47.390" LastActivityDate="2012-01-26T18:14:02.857" Title="Approximately &quot;solving&quot; a linear system of equations without a feasible solution" Tags="|linear-algebra|optimization|linear-programming|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="970" PostTypeId="2" ParentId="969" CreationDate="2012-01-24T13:52:28.573" Score="6" Body="&lt;p&gt;I assume what you're looking for is some $x$ such that $$\big\|Ax-b\big\|_2$$ is minimal. If this is the case, then what you need is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Moore-Penrose_pseudoinverse&quot; rel=&quot;nofollow&quot;&gt;pseudoinverse&lt;/a&gt; of $A$ or the &lt;a href=&quot;http://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow&quot;&gt;singular value decomposition&lt;/a&gt; of $A$ such that $U \Sigma V^\mathsf{T} = A$. The least-squares solution is then given by $$x = V\Sigma^+U^\mathsf{T}b$$ where $\Sigma^+$ is the diagonal matrix $\Sigma$ with all its non-zero entries inverted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab or Octave, you could do this as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[U,S,V] = svd(A);&#xA;ind = abs(S) &amp;gt; eps*S(1,1);&#xA;S( ind ) = 1./S( ind );&#xA;x = V*S*U'*b;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="541" LastEditorUserId="276" LastEditDate="2012-01-24T19:09:30.517" LastActivityDate="2012-01-24T19:09:30.517" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="971" PostTypeId="2" ParentId="969" CreationDate="2012-01-24T13:54:29.337" Score="2" Body="&lt;p&gt;You would need a more formal describtion of &quot;as close as possible&quot;. You could, for example, count the &quot;amount of violation&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min z_1 + z_2 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax \le b + z_1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax \ge b - z_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z_1, z_2 \ge 0$&lt;/p&gt;&#xA;" OwnerUserId="772" LastEditorUserId="81" LastEditDate="2012-01-24T18:46:10.287" LastActivityDate="2012-01-24T18:46:10.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="972" PostTypeId="2" ParentId="969" CreationDate="2012-01-24T13:56:54.603" Score="2" Body="&lt;p&gt;I should mention that you can approach this problem by using an iterative method to minimize $\|Ax-b\|^2$ such as the Landweber method or (faster and better) the method of conjugate gradients. This is probably faster than using the pseudoinverse (depending on implementation and many other details).&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2012-01-24T13:56:54.603" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="973" PostTypeId="2" ParentId="910" CreationDate="2012-01-24T13:58:43.197" Score="23" Body="&lt;p&gt;Another way to do this is to first explicitly specify the precision you desire in the variable using the &lt;a href=&quot;http://gcc.gnu.org/onlinedocs/gfortran/SELECTED_005fREAL_005fKIND.html&quot;&gt;SELECTED_REAL_KIND&lt;/a&gt; intrinsic and then use this to define and initialize the variables. Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;INTEGER, PARAMETER :: dp = SELECTED_REAL_KIND(15)&#xA;REAL(dp) :: x&#xA;x = 1.0_dp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A nice advantage to doing it this way is that you can store the definition of &lt;code&gt;dp&lt;/code&gt; in a module, then &lt;code&gt;USE&lt;/code&gt; that module where needed. Now if you ever want to change the precision of your program, you only have to change the definition of &lt;code&gt;dp&lt;/code&gt; in that one place instead of searching and replacing all the &lt;code&gt;D0&lt;/code&gt;s at the end of your variable initializations. (This is also why I'd recommend not using the &lt;code&gt;1.0D-1&lt;/code&gt; syntax to define Y as suggested. It works, but makes it harder to find and change all instances in the future.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fortranwiki.org/fortran/show/Real+precision&quot;&gt;This page&lt;/a&gt; on the Fortran Wiki gives some good additional information on &lt;code&gt;SELECTED_REAL_KIND&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2012-03-14T14:04:02.660" LastActivityDate="2012-03-14T14:04:02.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="974" PostTypeId="2" ParentId="938" CreationDate="2012-01-24T14:19:43.943" Score="1" Body="&lt;p&gt;The Computational Aeroacoustics Workshop on Benchmark Problems has a similar goal in mind for aeroacoustic problems. They first present a set of benchmark problems ahead of time, and have people solve them using their own methods and compare the results. For example, here's the fourth meeting's proceedings: &lt;a href=&quot;http://www.archive.org/details/nasa_techdoc_20040182258&quot; rel=&quot;nofollow&quot;&gt;http://www.archive.org/details/nasa_techdoc_20040182258&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although, from what I can tell they do not compare how efficient their codes are against each other, they mainly look at accuracy.&lt;/p&gt;&#xA;" OwnerUserId="739" LastActivityDate="2012-01-24T14:19:43.943" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="975" PostTypeId="1" CreationDate="2012-01-24T17:54:19.770" Score="2" ViewCount="188" Body="&lt;p&gt;I want to study the methods of analyzing textured images. So i searched google scholor but only found very old papers&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;statistical and structural approaches to texture 1979 haralick&lt;/li&gt;&#xA;&lt;li&gt;Image Texture Analysis Techniques - A Survey (1980)&lt;/li&gt;&#xA;&lt;li&gt;Texture analysis — a survey  July 1980&lt;/li&gt;&#xA;&lt;li&gt;Texture Analysis Methods – A Review 1998&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of a more recent survey paper ?&lt;/p&gt;&#xA;" OwnerUserId="774" LastEditorUserId="123" LastEditDate="2012-01-26T12:37:29.437" LastActivityDate="2012-02-09T08:34:40.237" Title="Texture analysis methods modern survey paper" Tags="|reference-request|image-processing|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="976" PostTypeId="1" AcceptedAnswerId="983" CreationDate="2012-01-24T18:56:26.793" Score="13" ViewCount="7727" Body="&lt;p&gt;When evaluating the number of FLOPs in a simple function, one can often just go down the expression tallying basic arithmetic operators.  However, in the case of mathematical statements involving even division, one cannot do this and expect to be able to compare with FLOP counts from functions with only additions and multiplications.  The situation is even worse when the operation is implemented in a library.  Therefore, it is imperative to have some reasonable notion of the performance of the special functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By special functions, we mean things like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt; exp()&#xA;&lt;li&gt; sqrt()&#xA;&lt;li&gt; sin/cos/tan()&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which are usually provided by system libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining the complexity of these is confounded even further by the fact that many of them are adaptive and have input-dependent complexity.  For example, numerically stable &lt;a href=&quot;http://www.convict.lu/Jeunes/ultimate_stuff/exp_ln_2.htm&quot;&gt;implementations&lt;/a&gt; of exp()  often adaptively rescale and use look-ups.  My initial impression here is that the best one may do in this case is ascertain the average behavior of the functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This entire discussion is, of course, highly dependent on the architecture.  For this discussion we can restrict ourselves to traditional general purpose architectures and exclude those with special function units (GPUs, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can find fairly simple attempts to &lt;a href=&quot;http://folding.stanford.edu/English/FAQ-flops&quot;&gt;standardize&lt;/a&gt; these for particular architectures for the sake of system vs. system comparison, but this is not acceptable if one cares about method vs. method performance. Which methodologies for determining the FLOP complexity of these functions are considered to be acceptable?  Are there any major pitfalls?&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="143" LastEditDate="2012-01-24T19:15:05.187" LastActivityDate="2012-07-23T02:13:59.903" Title="FLOP counting for library functions" Tags="|performance|complexity|floating-point|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="977" PostTypeId="2" ParentId="926" CreationDate="2012-01-24T18:57:44.000" Score="5" Body="&lt;p&gt;Regarding your question:&lt;br&gt;&#xA;I would say yes. Simply because the paradigms and principles are the same. The differences in the syntax are neglectable and can be summarized in a cheat sheet (in case you are working mainly with MATLAB one day).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Another suggestion for an alternative:&lt;/em&gt;&lt;br&gt;&#xA;As a MATLAB alternative I'm using &lt;a href=&quot;http://www.scilab.org/&quot;&gt;SciLab&lt;/a&gt;. It's free as Octave, has a decent community providing help and support and it's documentation is pretty good. The build-in documentation and help for functions is similar to MATLAB's one. (What I like most is the ability to execute the examples with one click). The syntax is similar to MATLAB as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the similarities and differences between SciLab and MATLAB, this document might be of interest for you: &lt;a href=&quot;http://wiki.scilab.org/Tutorials?action=AttachFile&amp;amp;do=get&amp;amp;target=Scilab4Matlab.pdf&quot;&gt;An Introduction to Scilab from a Matlab User's Point of View, Eike Rietsch, May 2010 (PDF)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another plus for SciLab over MATLAB: Start-up and GUI is a lot faster. I haven't tried Octave yet.&lt;/p&gt;&#xA;" OwnerUserId="746" LastActivityDate="2012-01-24T18:57:44.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="978" PostTypeId="2" ParentId="976" CreationDate="2012-01-24T19:15:48.850" Score="7" Body="&lt;p&gt;You could count them on real systems using &lt;a href=&quot;http://icl.cs.utk.edu/papi/&quot; title=&quot;PAPI&quot;&gt;PAPI&lt;/a&gt;, which grants access to hardware counters, and simple test programs.  My favorite PAPI interface/wrapper is &lt;a href=&quot;http://ipm-hpc.sourceforge.net/&quot; title=&quot;IPM&quot;&gt;IPM&lt;/a&gt; (Integrated Performance Monitor) but other solutions exist (&lt;a href=&quot;http://www.cs.uoregon.edu/Research/tau/home.php&quot; title=&quot;TAU&quot;&gt;TAU&lt;/a&gt;, for example).  This should give a fairly stable method-to-method comparison.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2012-01-24T19:39:38.903" LastActivityDate="2012-01-24T19:39:38.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="979" PostTypeId="2" ParentId="976" CreationDate="2012-01-24T19:36:59.127" Score="4" Body="&lt;p&gt;I am going to answer this question as if you asked:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;How do I analytically compare or predict the performance of algorithms that heavily rely on special functions, instead of the traditional multiply-add-carry FLOP counts that come from numerical linear algebra&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I agree with your first premise, that the performance of many special functions is architecture-dependent, and that although you can usually treat each of these functions as having constant cost, the size of the constant will vary, even between two processors from the same company but with different architectures (see &lt;a href=&quot;http://www.agner.org/optimize/instruction_tables.pdf&quot; rel=&quot;nofollow&quot;&gt;Agner Fog's instruction timing table&lt;/a&gt; for reference).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I disagree, though, that the focus of the comparison should be on the costs of the individual floating point operations.  I think that counting FLOPs is to some extent still useful, but that there are several much more important considerations that may make the cost of special functions less relevant when comparing two potential algorithms, and these should be explicitly examined first before going to a comparison of floating-point operations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Scalability - Algorithms featuring tasks that can be implemented efficiently on parallel architectures will dominate the scientific computing arena for the foreseeable future.  An algorithm with a better &quot;scalability&quot;, be it through lower communication, less need for synchronization, or better natural load balance, may employ more slower special functions and therefore be slower for small numbers of processes, but will eventually catch up as the number of processors is increased.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Temporal Locality of Reference - Does the algorithm reuse data between tasks, allowing the processor to avoid unnecessary memory traffic?  Each level of the memory hierarchy that an algorithm traverses adds another order of magnitude cost (roughly) to each memory access.  As a result, an algorithm with high density of special operations will likely be significantly faster than an algorithm with the equivalent number of simple function operations over a larger region of memory.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Memory Footprint - This is strongly related to the previous points, but as computers grow larger and larger, the amount of memory per core is actually trending downward.  There are two benefits to a small memory footprint.  The first is that a small amount of program data will likely be able to fit completely within the processor cache.  The second is that, for very large problems, an algorithm with a smaller memory footprint may be able to fit into processor memory, allowing problems to be solved that would otherwise exceed the capability of the computer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-01-24T19:36:59.127" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="980" PostTypeId="2" ParentId="926" CreationDate="2012-01-24T19:41:10.367" Score="6" Body="&lt;p&gt;It is indeed possible, especially if you use a GUI, such as &lt;a href=&quot;https://stackoverflow.com/questions/2267346/is-there-any-good-octave-gui-ide-for-windows&quot;&gt;GUIOctave&lt;/a&gt;.&#xA;Another language that you might find useful because many packages have been written for it is R. There is also a GUI for the R language, called RStudio, and a &lt;a href=&quot;http://www.math.umaine.edu/~hiebeler/comp/matlabR.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;translation document between R and MatLab&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="667" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-01-24T19:41:10.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="981" PostTypeId="2" ParentId="969" CreationDate="2012-01-24T20:02:22.630" Score="2" Body="&lt;p&gt;Pedro and Dirk's solutions are best if you want to minimize the $\ell_{2}$-norm of $Ax - b$. To solve it as an LP, you would minimize the $\ell_{1}$-norm of $Ax-b$, using a standard reformulation; I think this approach is what Martin was trying to express.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{alignat}{1}&#xD;&#xA;\min_{\mathbf{x}} &amp;amp; \quad\big\|Ax - b\big\|_{1},&#xD;&#xA;\end{alignat}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and express it using sum notation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{alignat}{1}&#xD;&#xA;\min_{\mathbf{x}} &amp;amp; \quad\sum_{i}\Big|\sum_{j}a_{ij}x_{j} - b_{i}\Big|,&#xD;&#xA;\end{alignat}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then reformulate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{alignat}{1}&#xD;&#xA;\min_{\mathbf{z}\geq\mathbf{0}, \mathbf{x}} &amp;amp; \quad\mathbf{1}^{T}\mathbf{z}, \\&#xD;&#xA;\textrm{s.t.} &amp;amp; \quad z_{i} \geq \sum_{j}a_{ij}x_{j}-b_{i}, \quad \forall{i}, \\&#xD;&#xA;&amp;amp; \quad z_{i} \geq b_{i} - \sum_{j}a_{ij}x_{j}, \quad \forall{i}.&#xD;&#xA;\end{alignat}$$&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-26T18:14:02.857" LastActivityDate="2012-01-26T18:14:02.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="982" PostTypeId="1" AcceptedAnswerId="1318" CreationDate="2012-01-24T20:48:22.223" Score="12" ViewCount="375" Body="&lt;p&gt;Is there an algorithm that enumerates the graphs that correspond to some Delaunay tessellation of points in 3D? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, is there an efficient parameterization of geometries that correspond to any &quot;Delaunay graph&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to enumerate systematically all stable geometries of molecules of a specified composition without any a priori knowledge of bonding etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: &#xA;Let $G_N$ be the set of graphs with $N$ vertices.&#xA;Let $D: \mathbb{R}^{3N} \to G_N$ be a map of $N$ points in $\mathbb{R}^3$ to a graph corresponding to a Delaunay tessellation of said points in 3D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I enumerate $D(\mathbb{R}^{3N})$ (efficiently)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, given a graph $g\in G_n$, how can I parameterize $D^{-1}(g)$ (efficiently)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Example in 2D: For 4 points there are 2 Delaunay graphs.&#xA;$$ \begin{matrix}&#xA;1 &amp;amp; - &amp;amp; 2 &amp;amp; - &amp;amp; 3 \\&#xA;&amp;amp;\diagdown &amp;amp;| &amp;amp; \diagup\\&#xA;&amp;amp;&amp;amp;4&#xA;\end{matrix}\mbox{ and }&#xA;\begin{matrix}&#xA;1 &amp;amp; - &amp;amp; 2\\&#xA;|&amp;amp; \times &amp;amp; |\\&#xA;3 &amp;amp; - &amp;amp; 4&#xA;\end{matrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or shown in an explicitly planar way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/RxG57.png&quot; alt=&quot;2D delaunay graphs for 4 points&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first of these graphs may be parameterized by any position of points 1, 2 and 4, i.e., $\mathbb{R}^{3\times 3}$, while point 3 would be any point $x_3(r,\theta)=c(x_1,x_2,x_4) + r\left(\begin{array}{c} \cos(\theta) \\ \sin(\theta)\end{array}\right)$ where $r$ is larger than the radius of the circle circumscribing points 1, 2, and 4 centered at $c(x_1,x_2,x_4)$ and $x_i$ is the position of point $i$.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="348" LastEditDate="2012-02-23T12:22:47.167" LastActivityDate="2012-02-23T12:22:47.167" Title="Enumeration of graphs deriving from Delaunay tessellations in 3D" Tags="|algorithms|computational-chemistry|delaunay-triangulation|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="983" PostTypeId="2" ParentId="976" CreationDate="2012-01-24T22:23:50.747" Score="10" Body="&lt;p&gt;It sounds like you want a way to evaluate how FPU-bound your code is, or how effectively you are using the FPU, rather than to count the number of flops according to same anachronistic definition of a &quot;flop&quot;. In other words, you want a metric that reaches the same peak if every floating point unit is running at full capacity every cycle. Let's look at an Intel Sandy Bridge to see how this might shake out.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Hardware-supported floating point operations&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This chip supports &lt;a href=&quot;http://en.wikipedia.org/wiki/Advanced_Vector_Extensions&quot;&gt;AVX&lt;/a&gt; instructions, so registers are 32 bytes long (holding 4 doubles). The superscalar architecture allows instructions to overlap, with most arithmetic instructions taking a few cycles to complete, even though a new instruction might be able to start on the next cycle. These semantics are usually abbreviated by writing latency/inverse throughput, a value of 5/2 would mean that the instruction takes 5 cycles to complete, but you can start a new instruction every other cycle (assuming that the operands are available, so no data dependence and not waiting for memory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three floating point arithmetic units per core, but the third is not relevant to our discussion, we'll call the relevant two the A and M units because their primary functions are addition and multiplication. Example instructions (see &lt;a href=&quot;http://www.agner.org/optimize/instruction_tables.pdf&quot;&gt;Agner Fog's tables&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;vaddpd&lt;/code&gt;: packed addition, occupying unit A for 1 cycle, latency/inverse throughput is 3/1&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;vmulpd&lt;/code&gt;: packed multiplication, unit M, 5/1&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;vmaxpd&lt;/code&gt;: packed select pairwise maximum, unit A, 3/1&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;vdivpd&lt;/code&gt;: packed divide, unit M (and some A), 21/20 to 45/44 depending on input&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;vsqrtpd&lt;/code&gt;: packed square root, some A and M, 21/21 to 43/43 depending on input&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;vrsqrtps&lt;/code&gt;: packed low-accuracy reciprocal square root for single precision input (8 &lt;code&gt;floats&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The precise semantics for what can overlap with &lt;code&gt;vdivpd&lt;/code&gt; and &lt;code&gt;vsqrtpd&lt;/code&gt; are apparently subtle and AFAIK, not documented anywhere. In most uses, I think there is little possibility for overlap, though the wording in the manual suggests that multiple threads may offer more possibility for overlap in this instruction. We can hit peak flops if we start a &lt;code&gt;vaddpd&lt;/code&gt; and &lt;code&gt;vmulpd&lt;/code&gt; on every cycle, for a total of 8 flops per cycle. Dense matrix-matrix multiply (&lt;code&gt;dgemm&lt;/code&gt;) can get reasonably close to this peak.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When counting flops for special instructions, I would look at how much of the FPU is occupied. Suppose for argument that in your range of input, &lt;code&gt;vdivpd&lt;/code&gt; took an average of 24 cycles to complete, fully occupying unit M, but addition could (if it was available) be executed concurrently for half the cycles. The FPU is capable of performing 24 packed multiplies and 24 packed additions during those cycles (perfectly interleaved &lt;code&gt;vaddpd&lt;/code&gt; and &lt;code&gt;vmulpd&lt;/code&gt;), but with a &lt;code&gt;vdivpd&lt;/code&gt;, the best we can do is 12 additional packed adds. If we suppose that the best possible way to do division is to use the hardware (reasonable), we might count the &lt;code&gt;vdivpd&lt;/code&gt; as 36 packed &quot;flops&quot;, indicating that we should count each scalar divide as 36 &quot;flops&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With reciprocal square root, it is sometimes possible to beat the hardware, especially if full accuracy is not needed, or if the range of input is narrow. As mentioned above, the &lt;code&gt;vrsqrtps&lt;/code&gt; instruction is very inexpensive, so (if in single precision) you could do one &lt;code&gt;vrsqrtps&lt;/code&gt; followed by one or two Newton iterations to clean up. These Newton iterations are just&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y *= (3 - x*y*y)*0.5;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If many of these operations need to be performed, this can be significantly faster than naive evaluation of &lt;code&gt;y = 1/sqrt(x)&lt;/code&gt;. Prior to the availability of hardware approximate reciprocal square root, some performance-sensitive code used &lt;a href=&quot;http://en.wikipedia.org/wiki/Fast_inverse_square_root&quot;&gt;infamous integer operations&lt;/a&gt; to find an initial guess for the Newton iteration.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Library-provided math functions&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;We can apply a similar heuristic to library-provided math functions. You can profile to determine the number of SSE instructions, but as we have discussed, that is not the whole story and a program that spends all its time evaluating special functions might not appear to get close to peak, which might be true, but isn't useful for telling you that all the time is spent out of your control on the FPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest using a good vector math library as the baseline (e.g. Intel's VML, part of MKL). Measure the number of cycles for each call and multiply by peak achievable flops over that number of cycles. So if a packed exponential takes 50 cycles to evaluate, count it as 100 flops times the register width. Unfortunately, vector math libraries are sometimes hard to call and don't have all the special functions, so you might end up doing scalar math, in which case you would count our hypothetical scalar exponential as 100 flops (even though it probably still takes 50 cycles, so you would only be getting 25% of &quot;peak&quot; if all the time is spent evaluating these exponentials).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others have mentioned, you can count cycles and hardware event counters using PAPI or various interfaces. For simple cycle counting, you can read the cycle counter directly using the &lt;code&gt;rdtsc&lt;/code&gt; instruction with a snippet of inline assembly.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-24T22:23:50.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="984" PostTypeId="5" CreationDate="2012-01-24T22:35:05.087" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-24T22:35:05.087" LastActivityDate="2012-01-24T22:35:05.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="985" PostTypeId="4" CreationDate="2012-01-24T22:35:05.087" Score="0" Body="Octave is a scientific programming &amp; scripting language with syntax similar to MATLAB and SCILAB." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-25T05:48:15.800" LastActivityDate="2012-01-25T05:48:15.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="986" PostTypeId="5" CreationDate="2012-01-24T22:36:08.690" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-24T22:36:08.690" LastActivityDate="2012-01-24T22:36:08.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="987" PostTypeId="4" CreationDate="2012-01-24T22:36:08.690" Score="0" Body="The study of the diffusion of diseases, and the mathematical models that describe its propagation in time and/or space." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-25T05:48:02.010" LastActivityDate="2012-01-25T05:48:02.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="988" PostTypeId="5" CreationDate="2012-01-24T22:37:19.500" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-24T22:37:19.500" LastActivityDate="2012-01-24T22:37:19.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="989" PostTypeId="4" CreationDate="2012-01-24T22:37:19.500" Score="0" Body="Referring to cubes in higher dimensional space ($R^{d}$ with $d&gt;3$)." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-25T05:48:12.260" LastActivityDate="2012-01-25T05:48:12.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="990" PostTypeId="5" CreationDate="2012-01-24T22:38:39.887" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-24T22:38:39.887" LastActivityDate="2012-01-24T22:38:39.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="991" PostTypeId="4" CreationDate="2012-01-24T22:38:39.887" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-24T22:38:39.887" LastActivityDate="2012-01-24T22:38:39.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="992" PostTypeId="5" CreationDate="2012-01-24T22:48:59.510" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-24T22:48:59.510" LastActivityDate="2012-01-24T22:48:59.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="993" PostTypeId="4" CreationDate="2012-01-24T22:48:59.510" Score="0" Body="Graphical Processing Unit - a specialized, relatively inexpensive hardware unit built for fast graphical computations and highly data-parallel scientific computations." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-25T05:48:04.180" LastActivityDate="2012-01-25T05:48:04.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="994" PostTypeId="5" CreationDate="2012-01-25T03:43:57.610" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-25T03:43:57.610" LastActivityDate="2012-01-25T03:43:57.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="995" PostTypeId="4" CreationDate="2012-01-25T03:43:57.610" Score="0" Body="An eigenvector of an operator is a vector such that the action of the operator is the same as multiplication by a constant, called the eigenvalue.  The eigensystem of an operator is the set of all such eigenvectors and their associated eigenvalues." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-25T05:47:46.130" LastActivityDate="2012-01-25T05:47:46.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="996" PostTypeId="1" AcceptedAnswerId="1003" CreationDate="2012-01-25T05:26:23.180" Score="3" ViewCount="1936" Body="&lt;p&gt;After configuring and building PETSc, I have successfully been able to run several examples.  In particular, I am working with &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex2.c.html&quot; rel=&quot;nofollow&quot;&gt;this example&lt;/a&gt;.  I have been able to run the program using the following commands:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;make ex2&#xA;mpiexec -n 4 ./ex2 -m 40 -n 40  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which produces the following output:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Norm of error 0.000642883 iterations 26&#xA;Norm of error 0.000642883 iterations 26&#xA;Norm of error 0.000642883 iterations 26&#xA;Norm of error 0.000642883 iterations 26 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This seems to tell me that the same problem was solved 4 times, rather than once by four processors in parallel.  Suspicious, I ran the program again using  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;mpiexec -n 4 ./ex2 -m 40 -n 40 -log_summary  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which produced the following output (note that it says that ./ex2 was run with only 1 processor:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Norm of error 0.000642883 iterations 26&#xA;************************************************************************************************************************&#xA;***             WIDEN YOUR WINDOW TO 120 CHARACTERS.  Use 'enscript -r -fCourier9' to print this document            ***&#xA;************************************************************************************************************************&#xA;&#xA;---------------------------------------------- PETSc Performance Summary: ----------------------------------------------&#xA;&#xA;./ex2 on a petsc-arc named utepgeon01.utep.edu with 1 processor, by pmdelgado2 Tue Jan 24 22:16:23 2012&#xA;Using Petsc Release Version 3.2.0, Patch 6, Wed Jan 11 09:28:45 CST 2012 &#xA;&#xA;                         Max       Max/Min        Avg      Total &#xA;Time (sec):           6.203e-02      1.00000   6.203e-02&#xA;Objects:              4.500e+01      1.00000   4.500e+01&#xA;Flops:                3.230e+06      1.00000   3.230e+06  3.230e+06&#xA;Flops/sec:            5.207e+07      1.00000   5.207e+07  5.207e+07&#xA;Memory:               7.996e+05      1.00000              7.996e+05&#xA;MPI Messages:         0.000e+00      0.00000   0.000e+00  0.000e+00&#xA;MPI Message Lengths:  0.000e+00      0.00000   0.000e+00  0.000e+00&#xA;MPI Reductions:       4.730e+02      1.00000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that the computer I'm using has 4 processors (after typing &lt;code&gt;grep processor /proc/cpuinfo&lt;/code&gt;).  After checking my reconfigure-petsc-arch.py file, I see the following:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python&#xA;if __name__ == '__main__':&#xA;  import sys&#xA;  import os&#xA;  sys.path.insert(0, os.path.abspath('config'))&#xA;  import configure&#xA;  configure_options = [&#xA;    '--download-f-blas-lapack=1',&#xA;    '--download-mpich=1',&#xA;    '--with-cc=gcc',&#xA;    '--with-fc=gfortran',&#xA;    'PETSC_ARCH=petsc-arch',&#xA;  ]&#xA;  configure.petsc_configure(configure_options)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there something else I need to do to distribute the work of solving the linear system in parallel? &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="9" LastEditDate="2012-01-26T09:36:17.143" LastActivityDate="2012-01-26T09:36:17.143" Title="Problems running a PETSc  example in parallel" Tags="|linear-algebra|parallel-computing|petsc|mpi|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="997" PostTypeId="2" ParentId="908" CreationDate="2012-01-25T14:09:44.127" Score="2" Body="&lt;p&gt;For rigorous, but probably weak error bounds, you can consult &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898715210&quot; rel=&quot;nofollow&quot;&gt;Nick Higham's &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;, 2nd edition&lt;/a&gt;. &lt;em&gt;A posteriori&lt;/em&gt; error estimates are probably only good to first order, but might be useful. Don Estep does work on these mostly with regards to solutions of partial differential equations, but he does give a linear algebra example buried in slide 72 of &lt;a href=&quot;http://www.stat.colostate.edu/~estep/research/talks/montrealcourse.pdf&quot; rel=&quot;nofollow&quot;&gt;this presentation&lt;/a&gt;, and a much more fleshed out example is given in slides 53-56 of &lt;a href=&quot;http://www.stat.colostate.edu/~estep/research/talks/adjointmulti_colloq.pdf&quot; rel=&quot;nofollow&quot;&gt;this presentation&lt;/a&gt;. These estimates can do a pretty good job if the error is small, but tend not to be so accurate for larger errors.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-25T14:09:44.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1001" PostTypeId="1" AcceptedAnswerId="1004" CreationDate="2012-01-25T16:20:38.820" Score="41" ViewCount="34988" Body="&lt;p&gt;I was comparing a few of my codes to &quot;stock&quot; MATLAB codes. I am surprised at the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran a sample code (Sparse Matrix)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 5000;&#xA;a = diag(rand(n,1));&#xA;b = rand(n,1);&#xA;disp('For a\b');&#xA;tic;a\b;toc;&#xA;disp('For LU');&#xA;tic;LULU;toc;&#xA;disp('For Conj Grad');&#xA;tic;conjgrad(a,b,1e-8);toc;&#xA;disp('Inv(A)*B');&#xA;tic;inv(a)*b;toc;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    For a\b&#xA;    Elapsed time is 0.052838 seconds.&#xA;&#xA;    For LU&#xA;    Elapsed time is 7.441331 seconds.&#xA;&#xA;    For Conj Grad&#xA;    Elapsed time is 3.819182 seconds.&#xA;&#xA;    Inv(A)*B&#xA;    Elapsed time is 38.511110 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For Dense Matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 2000;&#xA;a = rand(n,n);&#xA;b = rand(n,1);&#xA;disp('For a\b');&#xA;tic;a\b;toc;&#xA;disp('For LU');&#xA;tic;LULU;toc;&#xA;disp('For Conj Grad');&#xA;tic;conjgrad(a,b,1e-8);toc;&#xA;disp('For INV(A)*B');&#xA;tic;inv(a)*b;toc;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;For a\b&#xA;Elapsed time is 0.575926 seconds.&#xA;&#xA;For LU&#xA;Elapsed time is 0.654287 seconds.&#xA;&#xA;For Conj Grad&#xA;Elapsed time is 9.875896 seconds.&#xA;&#xA;Inv(A)*B&#xA;Elapsed time is 1.648074 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How the heck is a\b so awesome?&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="123" LastEditDate="2012-01-25T20:30:36.047" LastActivityDate="2024-04-01T09:49:05.907" Title="How does the MATLAB backslash operator solve $Ax=b$ for square matrices?" Tags="|linear-algebra|performance|matlab|" AnswerCount="5" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1002" PostTypeId="2" ParentId="1001" CreationDate="2012-01-25T17:34:49.033" Score="18" Body="&lt;p&gt;For sparse matrices, Matlab uses UMFPACK for the &quot;&lt;code&gt;\&lt;/code&gt;&quot; operation, which, in your example, basically runs through the values of &lt;code&gt;a&lt;/code&gt;, inverts them, and multiplies them with the values of &lt;code&gt;b&lt;/code&gt;. For this example, though, you should use &lt;code&gt;b./diag(a)&lt;/code&gt;, which is a lot faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For dense systems, the backslash-operator is a bit more complicated. A brief description of what is done when is given &lt;a href=&quot;http://www.mathworks.co.uk/support/solutions/en/data/1-172BD/index.html?product=ML&amp;amp;solution=1-172BD&quot;&gt;here&lt;/a&gt;. According to that description, in your example, Matlab would solve &lt;code&gt;a\b&lt;/code&gt; using backward substitution. For general square matrices, an LU-decomposition is used.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-01-25T17:34:49.033" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1003" PostTypeId="2" ParentId="996" CreationDate="2012-01-25T20:06:22.060" Score="2" Body="&lt;p&gt;After consulting the resources suggested, I found out that the computer that I was working with already had a different version of MPI installed on it, which PETSc was trying to use everytime I called the 'mpiexec' commmand.  When I built PETSc, I asked it to install MPI as well.  So, whenever I want to run a PETSc program in parallel, I needed to use the version that was installed with PETSc, not the one that was installed previously.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, I looked in my &lt;code&gt;$PETSC_DIR/$PETSC_ARCH/bin&lt;/code&gt; and found mpiexec in this directory.  So, I could run PETSc in parallel if I called it from the command prompt as:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$PETSC_DIR/$PETSC_ARCH/bin/mpiexec -n 4 ./ex2 -m 40 -n 40 -log_summary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;$PETSC_DIR&lt;/code&gt; is the main directory where all the PETSc files were installed, and &lt;code&gt;$PETSC_ARCH&lt;/code&gt; is the directory where all the PETSc architecture configuration files are installed.  However, to avoid having to type so much at the command prompt, I edited my .bashrc file (in my home directory in linux) with the following alias:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alias petsc=&quot;~/petsc-3.2-p6/petsch-arch/bin/mpiexec&quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(note: the ~ is there because I have it installed in my home directory)  This way, anytime I want to run a petsc file in parallel I could type something like this:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;petsc -n 4 ./ex2 -m 40 -n 40 -log_summary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-01-25T20:06:22.060" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1004" PostTypeId="2" ParentId="1001" CreationDate="2012-01-25T20:06:55.367" Score="43" Body="&lt;p&gt;In Matlab, the ‘\’ command invokes an algorithm which depends upon the structure of the matrix A and includes checks (small overhead) on properties of A.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If A is sparse and banded, employ a banded solver. &lt;/li&gt;&#xA;&lt;li&gt;If A is an upper or lower triangular matrix, employ a backward&#xA;substitution algorithm. &lt;/li&gt;&#xA;&lt;li&gt;If A is symmetric and has real positive diagonal elements, attempt a&#xA;Cholesky factorization. If A is sparse, employ reordering first to minimize&#xA;fill-in. &lt;/li&gt;&#xA;&lt;li&gt;If none of criteria above is fulfilled, do a general triangular factorization&#xA;using Gaussian elimination with partial pivoting. &lt;/li&gt;&#xA;&lt;li&gt;If A is sparse, then employ the UMFPACK library. &lt;/li&gt;&#xA;&lt;li&gt;If A is not square, employ algorithms based on QR factorization for&#xA;undetermined systems.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To reduce overhead it is possible to use the linsolve command in Matlab and select a suitable solver among these options yourself.&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-01-25T20:06:55.367" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1006" PostTypeId="2" ParentId="1001" CreationDate="2012-01-26T00:22:35.263" Score="6" Body="&lt;p&gt;As a rule of thumb, if you have a sparse matrix of reasonable complexity (i.e., it doesn't have to be the 5-point stencil but can in fact be a discretization of the Stokes equations for which the number of nonzeros per row is much larger than 5), then a sparse direct solver such as UMFPACK typically beats an iterative Krylov solver if the problem is no larger than around maybe 100,000 unknowns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, for most sparse matrices resulting from 2d discretizations, a direct solver is the fastest alternative. Only for 3d problems where you quickly get above 100,000 unknowns does it become imperative to use an iterative solver.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-01-26T00:22:35.263" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1007" PostTypeId="1" AcceptedAnswerId="1012" CreationDate="2012-01-26T01:33:05.623" Score="22" ViewCount="4427" Body="&lt;p&gt;Suppose that I have a function that takes as input several floating-point values (single or double), does some computation, and produces output floating-point values (also single or double). I am working primarily with MSVC 2008, but also plan to work with MinGW/GCC. I'm programming in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is the typical way of programmatically measuring how much error is in the results? Assuming that I need to use an arbitrary precision library: what is the best such library if I don't care about speed?&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="769" LastActivityDate="2022-11-27T23:23:52.077" Title="Analyzing Numerical Error in C++ Function" Tags="|floating-point|c++|" AnswerCount="7" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1009" PostTypeId="2" ParentId="1007" CreationDate="2012-01-26T02:45:02.530" Score="9" Body="&lt;p&gt;A nice portable and open source library for arbitrary precision floating point arithmetic (and much else) is &lt;a href=&quot;http://www.shoup.net/ntl/doc/tour-intro.html&quot; rel=&quot;nofollow&quot;&gt;Victor Shoup's NTL&lt;/a&gt;, which is available in C++ source form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a lower level is the &lt;a href=&quot;http://gmplib.org/&quot; rel=&quot;nofollow&quot;&gt;GNU Multiple Precision (GMP) Bignum Library&lt;/a&gt;, also an open source package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NTL can be used with GMP is faster performance is required, but NTL provides its own base routines that are certainly usable if you &quot;don't care about speed&quot;.  GMP claims to be the &quot;fastest bignum library&quot;.  GMP is largely written in C, but has a C++ interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Added:&lt;/strong&gt; While interval arithmetic can give upper and lower bounds on the exact answer in an automated way, this doesn't accurately measure the error in a &quot;standard&quot; precision computation because the interval size typically grows with each operation (either in a relative or absolute error sense).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical way of finding error size, either for rounding errors or for errors of discretization, etc., is to compute an extra precision value and compare that to the &quot;standard&quot; precision value.  Only modest extra precision is needed to determine the error size itself to reasonable accuracy, since the rounding errors alone are substantially larger in &quot;standard&quot; precision than they are in the extra precision computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point can be illustrated by comparing single and double precision computations.  Note that in C++ intermediate expressions are always computed in (at least) double precision, so if we want to illustrate what a computation in &quot;pure&quot; single precision would be like, we need to store out the intermediate values in single precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;C code snippet&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    float fa,fb;&#xA;    double da,db,err;&#xA;    fa = 4.0;&#xA;    fb = 3.0;&#xA;    fa = fa/fb;&#xA;    fa -= 1.0;&#xA;&#xA;    da = 4.0;&#xA;    db = 3.0;&#xA;    da = da/db;&#xA;    da -= 1.0;&#xA;&#xA;    err = fa - da;&#xA;    printf(&quot;Single precision error wrt double precision value\n&quot;);&#xA;    printf(&quot;Error in getting 1/3rd is %e\n&quot;,err);&#xA;    return 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output from above (Cygwin/MinGW32 GCC tool chain):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Single precision error wrt double precision value&#xA;Error in getting 1/3rd is 3.973643e-08&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thus the error is about what one expects in rounding 1/3rd to single precision.  One would not (I'd suspect) care about getting more than a couple of decimal places &lt;em&gt;in the error&lt;/em&gt; correct, since measurement of the error is for magnitude and not for exactitude.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2012-01-27T18:21:34.167" LastActivityDate="2012-01-27T18:21:34.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1010" PostTypeId="2" ParentId="1007" CreationDate="2012-01-26T02:49:05.000" Score="10" Body="&lt;p&gt;&lt;a href=&quot;http://gmplib.org/&quot;&gt;GMP&lt;/a&gt; (i.e., the GNU Multiple Precision library) is the best arbitrary precision library that I know of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any programmatic way to measure the error in the results of an arbitrary floating point function. One thing you could try is to calculate an interval extension of a function using &lt;a href=&quot;http://en.wikipedia.org/wiki/Interval_arithmetic&quot;&gt;interval arithmetic&lt;/a&gt;. In C++, you would have to use some sort of library to calculate the interval extensions; one such library is the &lt;a href=&quot;http://www.boost.org/doc/libs/1_37_0/libs/numeric/interval/doc/interval.htm&quot;&gt;Boost Interval Arithmetic Library&lt;/a&gt;. Basically, to measure the error, you would supply as arguments to your function intervals that have a width of 2 times unit roundoff (roughly), centered at the values of interest, and then your output would be a collection of intervals, the width of which would give you some conservative estimate of the error. A difficulty with this approach is that interval arithmetic used in this fashion can overestimate error by significant amounts, but this approach is the most &quot;programmatic&quot; one I can think of.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-26T02:49:05.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1011" PostTypeId="2" ParentId="1007" CreationDate="2012-01-26T04:42:38.030" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;http://www.mpfr.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU MPFR library&lt;/a&gt; is an arbitrary-precision float library that has high accuracy (in particular, correct rounding for all operations, which is not as easy as it sounds) as one of their main focus points. It uses GNU MP under the hood. It has an extension called &lt;a href=&quot;https://gforge.inria.fr/projects/mpfi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MPFI&lt;/a&gt; that does interval arithmetic, which - as &lt;a href=&quot;https://scicomp.stackexchange.com/a/1010/49&quot;&gt;Geoff's answer&lt;/a&gt; suggests - might come in handy for verification purposes: keep increasing the working precision until the resulting interval falls within small bounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This won't always work, though; in particular it's not necessarily effective if you're doing something like numerical integration, where every step carries an &quot;error&quot; independent of rounding problems. In that case, try a specialized package such as &lt;a href=&quot;http://bt.pa.msu.edu/index_cosy.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;COSY infinity&lt;/a&gt; which does this very well using specific algorithms to bound the integration error (and using so-called Taylor models instead of intervals).&lt;/p&gt;&#xA;" OwnerUserId="49" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-01-26T04:42:38.030" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1012" PostTypeId="2" ParentId="1007" CreationDate="2012-01-26T10:01:29.770" Score="21" Body="&lt;p&gt;If you're looking for a good bound on your rounding error, you don't necessarily need an aribtrary-precision library. You can use running error analysis instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wasn't able to find a good online reference, but it's all described in Section 3.3 of Nick Higham's book &quot;Accuracy and Stability of Numerical Algorithms&quot;. The idea is quite simple:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Re-factor your code so that you have a single assignment of a single arithmetic operation on each line.&lt;/li&gt;&#xA;&lt;li&gt;For each variable, e.g. &lt;code&gt;x&lt;/code&gt;, create a variable &lt;code&gt;x_err&lt;/code&gt; which is initialized to zero when &lt;code&gt;x&lt;/code&gt; is assigned a constant.&lt;/li&gt;&#xA;&lt;li&gt;For each operation, e.g. &lt;code&gt;z = x * y&lt;/code&gt;, update the variable &lt;code&gt;z_err&lt;/code&gt; using the standard model of floating-point arithmetic and the resulting &lt;code&gt;z&lt;/code&gt; and the running errors &lt;code&gt;x_err&lt;/code&gt; and &lt;code&gt;y_err&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The return value of your function should then also have a respective &lt;code&gt;_err&lt;/code&gt; value attached to it. This is a data-dependent bound on your total roundoff error.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The tricky part is step 3. For the most simple arithmetic operations, you can use the following rules:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;z = x + y&lt;/code&gt; -&gt; &lt;code&gt;z_err = u*abs(z) + x_err + y_err&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;z = x - y&lt;/code&gt; -&gt; &lt;code&gt;z_err = u*abs(z) + x_err + y_err&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;z = x * y&lt;/code&gt; -&gt; &lt;code&gt;z_err = u*abs(z) + x_err*abs(y) + y_err*abs(x)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;z = x / y&lt;/code&gt; -&gt; &lt;code&gt;z_err = u*abs(z) + (x_err*abs(y) + y_err*abs(x))/y^2&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;z = sqrt(x)&lt;/code&gt; -&gt; &lt;code&gt;z_err = u*abs(z) + x_err/(2*abs(z))&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;u = eps/2&lt;/code&gt; is the unit roundoff. Yes, the rules for &lt;code&gt;+&lt;/code&gt; and &lt;code&gt;-&lt;/code&gt; are the same. Rules for any other operation &lt;code&gt;op(x)&lt;/code&gt; can be easily extracted using the Taylor series expansion of the result applied to &lt;code&gt;op(x + x_err)&lt;/code&gt;. Or you can try googling. Or using Nick Higham's book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, consider the following Matlab/Octave code which evaluates a polynomials in the coefficients &lt;code&gt;a&lt;/code&gt; at a point &lt;code&gt;x&lt;/code&gt; using the Horner scheme:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function s = horner ( a , x )&#xA;    s = a(end);&#xA;    for k=length(a)-1:-1:1&#xA;        s = a(k) + x*s;&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the first step, we split-up the two operations in &lt;code&gt;s = a(k) + x*s&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function s = horner ( a , x )&#xA;    s = a(end);&#xA;    for k=length(a)-1:-1:1&#xA;        z = x*s;&#xA;        s = a(k) + z;&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We then introduce the &lt;code&gt;_err&lt;/code&gt; variables. Note that the inputs &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;x&lt;/code&gt; are assumed to be exact, but we could just as well also require the user to pass corresponding values for &lt;code&gt;a_err&lt;/code&gt; and &lt;code&gt;x_err&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [ s , s_err ] = horner ( a , x )&#xA;    s = a(end);&#xA;    s_err = 0;&#xA;    for k=length(a)-1:-1:1&#xA;        z = x*s;&#xA;        z_err = ...;&#xA;        s = a(k) + z;&#xA;        s_err = ...;&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, we apply the rules described above to get the error terms:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [ s , s_err ] = horner ( a , x )&#xA;    u = eps/2;&#xA;    s = a(end);&#xA;    s_err = 0;&#xA;    for k=length(a)-1:-1:1&#xA;        z = x*s;&#xA;        z_err = u*abs(z) + s_err*abs(x);&#xA;        s = a(k) + z;&#xA;        s_err = u*abs(s) + z_err;&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that since we have no &lt;code&gt;a_err&lt;/code&gt; or &lt;code&gt;x_err&lt;/code&gt;, e.g. they are assumed to be zero, the respective terms are simply ignored in the error expressions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Et voilà! We now have a Horner scheme which returns a data-dependent error estimate (note: this is an &lt;em&gt;upper bound&lt;/em&gt; on the error) alongside the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side note, since you're using C++, you might consider making your own class for floating-point values which carries around the &lt;code&gt;_err&lt;/code&gt; term and overloading all the arithmetic operations to update these values as described above. For large codes, this may be the easier, albeit computationally less efficient, route. Having said that, you may be able to find such a class online. A quick Google search gave me &lt;a href=&quot;http://ctn.cvut.cz/ap/download.php?id=497&quot;&gt;this link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Note that this all works only on machines adhering strictly to IEEE-754, i.e. all arithmetic operations are precise to $\pm u$. This analysis also gives a tighter, more realistic bound than using interval arithmetic since, by definition, you can not represent a number $x(1 \pm u)$ in floating point, i.e. your interval would just round to the number itself.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-01-26T13:59:36.700" LastActivityDate="2012-01-26T13:59:36.700" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1013" PostTypeId="2" ParentId="1007" CreationDate="2012-01-26T10:25:47.160" Score="8" Body="&lt;p&gt;&lt;strong&gt;Rigorous and automatic error estimation&lt;/strong&gt; can be achieved by &lt;a href=&quot;http://en.wikipedia.org/wiki/Interval_arithmetic&quot;&gt;interval analysis&lt;/a&gt;. You work with intervals instead of numbers. For example addition:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[a,b] + [c,d] = [min(a+c, a+d, b+c, b+d), max (a+c, a+d, b+c, b+d)] = [a+c, b+d]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Round-off can also be handled rigorously, see &lt;a href=&quot;http://en.wikipedia.org/wiki/Interval_arithmetic#Rounded_interval_arithmetic&quot;&gt;Rounded interval arithmetic&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As long as your input consists of narrow intervals, the estimates are OK and are faily cheap to compute. Unfortunately, the error is often overestimated, see the &lt;a href=&quot;http://en.wikipedia.org/wiki/Interval_arithmetic#Dependency_problem&quot;&gt;dependence problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know of any arbitrary precision interval arithmetic library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It depends on your problem at hand whether interval arithmetic can serve your needs or not.&lt;/p&gt;&#xA;" OwnerUserId="679" LastActivityDate="2012-01-26T10:25:47.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1014" PostTypeId="1" AcceptedAnswerId="1021" CreationDate="2012-01-26T10:31:06.187" Score="17" ViewCount="758" Body="&lt;p&gt;I have a very basic notion about interval arithmetic (IA), but it seems to be a very  interesting branch of computational science both theoretically and practically. It is clear that the obvious applications are verified computing and ill-posed problems,  but this is too abstract. Since there are a lot of people involved in applied computations here, &lt;strong&gt;I’m curious about real world problems which are hard or impossible to solve without IA&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="67" LastEditDate="2012-01-27T18:01:47.630" LastActivityDate="2020-10-16T17:55:24.877" Title="What are some applications which require interval arithmetic?" Tags="|reference-request|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1015" PostTypeId="1" AcceptedAnswerId="1017" CreationDate="2012-01-26T10:34:13.403" Score="16" ViewCount="538" Body="&lt;p&gt;I was reading &lt;a href=&quot;https://math.stackexchange.com/questions/42152/can-i-use-ravi-vakils-way-of-learning-for-elementary-subjects&quot;&gt;this&lt;/a&gt; on Math SE. The basic question is :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that someone wishes to study something advanced; one way to do this would be to start off from basics and build up. But the &quot;bigger picture&quot; can get lost in this process. One more method (which I prefer to call Recursive Wiki) is to pick up a paper and Google/Wiki the terms which one does not understand; Read them. Within them will be terms which are new; Google/Wiki them till you reach the &quot;base case&quot; of knowing the material thoroughly. Work Backwards till you understand the paper thoroughly. Repeat for other papers. This will allow knowledge gain while maintaining motivation. But, it may cause problems in fundamentals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is based on an &lt;a href=&quot;http://math.stanford.edu/~vakil/potentialstudents.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; by Prof. Vakil from Stanford. Here is an excerpt:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;.....mathematics is so rich and infinite that it is&#xA;  impossible to learn it systematically, and if you wait to master one&#xA;  topic before moving on to the next, you'll never get anywhere.&#xA;  Instead, you'll have tendrils of knowledge extending far from your&#xA;  comfort zone. Then you can later backfill from these tendrils, and&#xA;  extend your comfort zone; this is much easier to do than learning&#xA;  &quot;forwards&quot;. (Caution: this backfilling is necessary.....&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The general consensus among the opponents of such a method was that it was OK for fields like Algebraic Geometry where 100s of papers are published per quarter or String Theory research where if you tried to build up mathematical basics before you touched String Theory, you would be 80 with Alzheimer's. My question is: Is this a good strategy for studying CompSci?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Comp Sci is so multidisciplinary (and it is usually a necessity for engineers to know both Math and Computing), is such a recursive mode of study good enough for academic research? Or is the traditional mode too good to be replaced?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, I needed to know about Translation Lookaside Buffer (TLB) with 0 knowledge of Operating Systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recursive path (as per wikipedia) was:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLB &gt; Cache &gt; (Back) TLB &gt; Page Table &gt; (Back) TLB &gt; Virtual Address &gt; (Back) TLB &gt; Reread. Done&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel I know what TLB is and if I encounter it again, I'll know whats going on. Am I deceiving myself?&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-01-26T15:22:22.197" Title="How effective is the 'tendrils of knowledge' approach to Comp. Sci?" Tags="|education|reference-request|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1016" PostTypeId="2" ParentId="1014" CreationDate="2012-01-26T11:22:56.803" Score="7" Body="&lt;p&gt;Interval arithmetic gives you a proof with mathematical rigor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good examples of actual applications is the work of &lt;a href=&quot;http://www.nd.edu/~markst/publications.html&quot;&gt;Mark Stadtherr&lt;/a&gt; and his research group. In particular, phase equilibrium and stability calculations are successfully solved with intervals methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice collection of benchmarks, with reference to their physical background, is at the &lt;a href=&quot;http://www-sop.inria.fr/coprin/logiciels/ALIAS/Benches/benches.html&quot;&gt;ALIAS website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="679" LastEditorUserId="679" LastEditDate="2012-01-26T15:30:44.053" LastActivityDate="2012-01-26T15:30:44.053" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1017" PostTypeId="2" ParentId="1015" CreationDate="2012-01-26T15:22:22.197" Score="9" Body="&lt;p&gt;I don't think that you're deceiving yourself necessarily, but you should be careful using what you've learned. In this case it's relatively easy. If you need to design your code with TLB-awareness in mind, you've probably got what you need now, and you can prove this to yourself by writing programs. However, if someone asked you to design and implement a TLB, you might need to go do some more research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the important part of learning anything is to also try to understand the limits of your knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tendrils approach is clearly a good method for certain kinds of learning. You should use it where appropriate. The nice thing about the other method is that you learn lots of other things along the way which can be helpful in building your ideas and plans for attacking problems. E.g., you now know something about TLBs, but if you didn't encounter and teach yourself about non-temporal accesses (say, to pick an example at random) while you were learning about TLBs, you may have missed a hugely important piece of knowledge about optimizing memory systems. If you'd started at the very bottom in learning about memory systems on chips, you probably would have encountered (however shallowly) non-temporal accesses, so you'd know that you also need to keep these in mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest that you judiciously sprinkle learning in both categories as you work to solve your problems. It's rare that you start with a fully-formed problem statement anyway, so there will be some mix of working forwards and backwards and iteration about the nature of the problem as you work through it.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-01-26T15:22:22.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1018" PostTypeId="1" AcceptedAnswerId="1019" CreationDate="2012-01-26T16:06:05.977" Score="2" ViewCount="1389" Body="&lt;p&gt;When I run a PETSc example in parallel with the flag &quot;-log_summary&quot;, the first two tables of information look something like this:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                         Max       Max/Min        Avg      Total &#xA;Time (sec):           2.512e-01      1.00547   2.505e-01&#xA;Objects:              5.800e+01      1.00000   5.800e+01&#xA;Flops:                3.271e+05      1.00590   3.262e+05  1.305e+06&#xA;Flops/sec:            1.309e+06      1.01141   1.302e+06  5.207e+06&#xA;Memory:               2.471e+05      1.00599              9.856e+05&#xA;MPI Messages:         5.200e+01      2.00000   3.900e+01  1.560e+02&#xA;MPI Message Lengths:  1.569e+04      2.00000   3.017e+02  4.706e+04&#xA;MPI Reductions:       5.060e+02      1.00000&#xA;&#xA;Summary of Stages:   ----- Time ------  ----- Flops -----  --- Messages ---  -- Message Lengths --  -- Reductions --&#xA;                        Avg     %Total     Avg     %Total   counts   %Total     Avg         %Total   counts   %Total &#xA; 0:      Main Stage: 2.3873e-01  95.3%  1.3047e+06 100.0%  1.440e+02  92.3%  2.954e+02       97.9%  4.870e+02  96.2% &#xA; 1:        Assembly: 1.1753e-02   4.7%  0.0000e+00   0.0%  1.200e+01   7.7%  6.308e+00        2.1%  1.800e+01   3.6% &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I am profiling the parallel performance, I usually think that I should use some sort of &quot;total execution time&quot; metric.  However, the log_summary seems to give several measures of time.   Which among them can I use as the &quot;overall execution time&quot;?  Are there other metrics which are important to pay attention to when profiling the performance?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-01-26T18:52:11.367" Title="How to measure the overall performance of a PETSc program using the -log_summary flag?" Tags="|performance|parallel-computing|petsc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1019" PostTypeId="2" ParentId="1018" CreationDate="2012-01-26T18:52:11.367" Score="3" Body="&lt;p&gt;PETSc keeps profile statistics on each MPI process, then reduces them to obtain an average, maximum, and minimum for each.  If you are interested in the total execution time of your job, that will be equivalent to the first entry in the first column, though I should warn you that 0.2 seconds is likely not an interesting example to study for profiling.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are concerned about load balance, you would look at the max/min ratio and the average (your job seems fairly balanced).  The other metrics can give you a better understanding of where time is being spent in your code, as the PETSc library provides lightweight profiling, flop counting, and MPI message tracking.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-01-26T18:52:11.367" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1021" PostTypeId="2" ParentId="1014" CreationDate="2012-01-26T20:00:10.877" Score="14" Body="&lt;p&gt;This answer partly responds to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1014/what-are-some-applications-which-require-interval-arithmetic#comment1437_1016&quot;&gt;JackPoulson's comment&lt;/a&gt; (because it is long), and partly answers the question.&lt;/p&gt;&#xA;&lt;p&gt;Interval arithmetic is a computational procedure to give rigorous bounds on calculated quantities, only in the sense that the interval extension of a real-valued function over an interval encloses the image of that function over the same interval. Without calculating anything, interval arithmetic cannot give you any insight about what factors influence the numerical error in a calculation, whereas the theorems in Higham's book and others do give you insight into the factors influencing numerical error, at the cost of potentially weak bounds. Granted, the bounds obtained using interval arithmetic may also be weak, due to the so-called &lt;a href=&quot;http://en.wikipedia.org/wiki/Interval_arithmetic#Dependency_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;dependency problem&lt;/a&gt;, but sometimes they are much stronger. For instance, the interval bounds obtained using the integration package &lt;a href=&quot;http://bt.pa.msu.edu/index_cosy.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;COSY Infinity&lt;/a&gt; are much tighter than the types of error bounds you would get on numerical integration from the results of Dahlquist (see &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/3540566708&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer, Wanner, and Nørsett&lt;/a&gt; for details); these results (I am particularly referring to Theorems 10.2 and 10.6 in Part I) give more insight into sources of error, but the bounds are weak, whereas the bounds using COSY can be  tight. (They use several tricks to mitigate dependency issues.)&lt;/p&gt;&#xA;&lt;p&gt;I hesitate to use the word &amp;quot;proof&amp;quot; when describing what interval arithmetic does. There are proofs involving interval arithmetic, but calculating results using interval arithmetic with outward rounding is really just a means of bookkeeping to conservatively bound the range of a function. Interval arithmetic calculations are not proofs; they are a way to propagate uncertainty.&lt;/p&gt;&#xA;&lt;p&gt;As far as applications go, in addition to Stadtherr's work in chemical engineering, interval arithmetic has also been used to calculate bounds for particle beam experiments (see the work of Makino and Berz, linked to the COSY Infinity web site), they've been used in global optimization and chemical engineering design applications (among others) by &lt;a href=&quot;http://yoric.mit.edu/biblio&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barton&lt;/a&gt; (the link is to a list of publications), the design of spacecraft and global optimization (among others) by &lt;a href=&quot;http://arnold-neumaier.at/publist.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neumaier&lt;/a&gt; (again, the link is to a list of publications), global optimization and nonlinear equation solvers by &lt;a href=&quot;http://interval.louisiana.edu/preprints.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kearfott&lt;/a&gt; (another list of publications), and for uncertainty quantification (various sources; Barton is one of them).&lt;/p&gt;&#xA;&lt;p&gt;Finally, a disclaimer: Barton is one of my thesis advisers.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2022-09-27T22:20:32.817" LastActivityDate="2020-10-16T17:55:24.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1022" PostTypeId="5" CreationDate="2012-01-26T21:30:42.210" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-26T21:30:42.210" LastActivityDate="2012-01-26T21:30:42.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1023" PostTypeId="4" CreationDate="2012-01-26T21:30:42.210" Score="0" Body="Intel's Math Kernel Library, a linear algebra library optimized for Intel processors." OwnerUserId="81" LastEditorUserId="276" LastEditDate="2012-01-26T21:31:21.010" LastActivityDate="2012-01-26T21:31:21.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1024" PostTypeId="1" AcceptedAnswerId="1025" CreationDate="2012-01-27T00:44:12.833" Score="18" ViewCount="2670" Body="&lt;p&gt;I would like to learn how wavelet methods can be applied to PDE, but unfortunately I do not know a good resource to learn about this topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that many introductions to wavelets focus on interpolation theory, e.g., assembling a signal by a superposition of preferably few wavelets. Applications to PDEs are sometimes mentioned, without going deeper into that subject. I am interested in good summary articles for people who have seen a WFT but do not have any more knowledge on that topic. A good summary would be interesing as well, of course, if you think that can be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am particularly interested in getting an impression which kind of questions typically appear. For example, I know that finite elements are typically applied to a PDE on a bounded domain with Lipschitz boundary, which are the typical questions in choosing the ansatz space (conforming, non-conforming, geometry and combinatorics), how convergence theory is established (actually the Galerkin theory should not be so different for Wavelets), and I have some intuition which mathematical things are feasible in implementations. Such a bird's eye perspective on Wavelets for PDE would be very helpful for me.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="276" LastEditDate="2012-01-27T01:51:38.963" LastActivityDate="2012-01-28T21:00:39.113" Title="How can wavelets be applied to PDE?" Tags="|pde|wavelet|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1025" PostTypeId="2" ParentId="1024" CreationDate="2012-01-27T03:03:05.317" Score="23" Body="&lt;p&gt;Wavelets have nice multi-resolution approximation properties, but are not especially popular for solving PDEs. The most commonly cited reasons are difficulty imposing boundary conditions, treatment of unaligned anisotropy, evaluation of nonlinear terms, and efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wavelets were first to obtain strong convergence results for fully adaptive methods (see &lt;a href=&quot;http://www.ams.org/journals/mcom/2001-70-233/S0025-5718-00-01252-7/S0025-5718-00-01252-7.pdf&quot; rel=&quot;noreferrer&quot;&gt;Cohen, Dahmen, and DeVore 2001&lt;/a&gt; and &lt;a href=&quot;http://dx.doi.org/10.1007/s102080010027&quot; rel=&quot;noreferrer&quot;&gt;2002&lt;/a&gt;). However, this crucial theory was quickly followed by &lt;a href=&quot;http://10.1007/s00211-003-0492-7&quot; rel=&quot;noreferrer&quot;&gt;Binev, Dahmen, and DeVore (2004)&lt;/a&gt; who proved a similar result for adaptive finite element methods which are more popular for traditional PDE problems in moderate dimensions. Wavelet bases are popular for higher dimensional problems such as sparse tensor methods for stochastic PDEs &lt;a href=&quot;http://dx.doi.org/10.1017/S0962492911000055&quot; rel=&quot;noreferrer&quot;&gt;Schwab and Gittelson (2011)&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/a/1044/119&quot;&gt;this discussion&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Differential operators have bounded condition number when expressed in wavelet bases and preconditioned with Jacobi (thus Krylov methods converge in a constant number of iterations independent of resolution). This is related to the hierarchical multigrid methods of Yserentant (1984), &lt;a href=&quot;http://dx.doi.org/10.1007/BF01462238&quot; rel=&quot;noreferrer&quot;&gt;Bank, Dupont, and Yserentant (1988)&lt;/a&gt;, and others. Note that multiplicative multigrid methods have superior convergence properties to additive methods. A standard multigrid V-cycle is essentially equivalent to standard symmetric Gauss-Seidel in the wavelet basis with the usual ordering. Note that this is rarely the best way to implement, especially in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calederon-Zygmund operators and pseudo-differential operators are sparse in wavelet bases. Thus, many problems for which $\mathcal H$-matrices are useful with compact bases can be treated elegantly using wavelet bases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Differential operators are relatively more expensive to evaluate in wavelet bases and it can be difficult to establish desired conservation properties. Some authors (e.g. &lt;a href=&quot;http://dx.doi.org/10.1006/jcph.1995.1147&quot; rel=&quot;noreferrer&quot;&gt;Vasilyev, Paolucci, and Sen 1995)&lt;/a&gt; resort to collocation methods and use finite difference stencils to evaluate derivatives and nonlinear terms. If the wavelet expansion is blocked (usually good for computational efficiency), these methods become very similar to block-structured AMR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest &lt;a href=&quot;http://dx.doi.org/10.1006/jcph.1996.5562&quot; rel=&quot;noreferrer&quot;&gt;Beylkin and Keiser (1997)&lt;/a&gt; as a practical introduction to solving PDEs with wavelets. The &lt;a href=&quot;http://code.google.com/p/m-a-d-n-e-s-s/&quot; rel=&quot;noreferrer&quot;&gt;MADNESS&lt;/a&gt; code is based on these methods. It has support for immersed boundaries (see &lt;a href=&quot;http://dx.doi.org/10.1016/j.cpc.2011.07.001&quot; rel=&quot;noreferrer&quot;&gt;Reuter, Hill, and Harrison 2011&lt;/a&gt;), but has no efficient way to represent boundary layers in complicated geometry. The software is often used for chemistry problems in which geometry is not a concern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For general numerical analysis of wavelets, I suggest &lt;a href=&quot;http://books.google.com/books?id=Dz9RnDItrAYC&amp;amp;printsec=frontcover&amp;amp;dq=cohen%20wavelet&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=chIiT_CGB8rCgAei9sWSCQ&amp;amp;ved=0CD0Q6AEwAg#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;Cohen's 2003 book&lt;/a&gt;. It presents an analysis framework in which the continuum solution is manipulated up until you want to evaluate it to a given accuracy, at which point the wavelet basis is evaluated as necessary.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-01-28T21:00:39.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1026" PostTypeId="1" AcceptedAnswerId="1030" CreationDate="2012-01-27T12:16:43.427" Score="24" ViewCount="4997" Body="&lt;p&gt;As we know, orthogonal transformations methods (Givens rotations and Housholder reflections) for systems of linear equations are more expensive than Gaussian elimination, but theoretically have nicer stability properties in the sense that they do not change the condition number of the system. Although I know just &lt;a href=&quot;http://math.nist.gov/MatrixMarket/data/MMDELI/wilkinson/wilkinson.html&quot;&gt;one academic example&lt;/a&gt; of a matrix which is spoiled by Gaussian elimination with partial pivoting. And there's common opinion that it is very unlikely to meet this kind of behavior in practice (see &lt;a href=&quot;http://www.cs.usask.ca/~spiteri/M313/notes/Lecture22.pdf&quot;&gt;this lecture notes [pdf]&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, where shall we look for the answer on the topic? Parallel implementations? Updating?..&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="720" LastEditDate="2012-01-27T16:04:17.363" LastActivityDate="2012-01-29T14:34:03.717" Title="When do orthogonal transformations outperform Gaussian elimination?" Tags="|linear-algebra|reference-request|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1027" PostTypeId="2" ParentId="1026" CreationDate="2012-01-27T12:52:34.697" Score="3" Body="&lt;p&gt;How do you measure performance? Speed? Accuracy? Stability? A quick test in Matlab gives the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; N = 100;&#xA;&amp;gt;&amp;gt; A = randn(N); b = randn(N,1);&#xA;&amp;gt;&amp;gt; tic, for k=1:10000, [L,U,p] = lu(A,'vector'); x = U\(L\b(p)); end; norm(A*x-b), toc&#xA;ans =&#xA;   1.4303e-13&#xA;Elapsed time is 2.232487 seconds.&#xA;&amp;gt;&amp;gt; tic, for k=1:10000, [Q,R] = qr(A); x = R\(Q'*b); end; norm(A*x-b), toc             &#xA;ans =&#xA;   5.0311e-14&#xA;Elapsed time is 7.563242 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So solving a single system with an LU-decomposition is about three times as fast as solving it with a QR-decomposition, at the cost of half a decimal digit of accuracy (this example!).&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-01-27T12:52:34.697" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1028" PostTypeId="2" ParentId="1014" CreationDate="2012-01-27T13:56:33.203" Score="5" Body="&lt;p&gt;Another feature of interval arithmetic and its generalizations is that it allows &lt;em&gt;adaptive&lt;/em&gt; exploration of the domain of a function. It can thus be used for adaptive geometric modeling, processing, and rendering, just to take examples from computer graphics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interval methods have featured in some recent proofs of hard mathematical theorems such as the existence of chaos in the Lorenz attractor and the Kepler Conjecture. See &lt;a href=&quot;http://www.cs.utep.edu/interval-comp/kearfottPopular.pdf&quot;&gt;http://www.cs.utep.edu/interval-comp/kearfottPopular.pdf&lt;/a&gt; for these and other applications.&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2012-01-27T13:56:33.203" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1029" PostTypeId="2" ParentId="1026" CreationDate="2012-01-27T14:06:49.677" Score="3" Body="&lt;p&gt;The article you cite defends Gaussian Elimination by saying that even though it is numerically unstable it tends to do well on random matrices and since most matrices one can think of are like random matrices, we should be ok. This same statement can be said of many numerically unstable methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the space of all matrices. These methods work fine almost everywhere. That is 99.999...% of all matrices one could create will have no problems with unstable methods. There is only a very small fraction of matrices for which GE and others will have difficulty. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problems that researchers care about tend to be in that small fraction.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't construct matrices randomly. We construct matrices with very special properties that correspond to very special, non-random systems. These matrices are often ill-conditioned. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Geometrically you can consider the linear space of all matrices. There is a zero volume/measure subspace of singular matrices cuts through this space. Many problems that we construct are clustered around this subspace. They are not distributed randomly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example consider the heat equation or dispersion. These systems tend to remove information from the system (all initial states gravitate to a single final state) and as a result matrices that describe these equations are enormously singular. This process is very unlikely in a random situation yet ubiquitous in physical systems. &lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-01-27T14:13:50.397" LastActivityDate="2012-01-27T14:13:50.397" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1030" PostTypeId="2" ParentId="1026" CreationDate="2012-01-27T16:10:43.933" Score="25" Body="&lt;h2&gt;Accuracy&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Trefethen and Schreiber wrote an excellent paper, &lt;a href=&quot;http://www.cse.illinois.edu/courses/cs591mh/trefethen/trefethen_schreiber.pdf&quot;&gt;Average-case Stability of Gaussian Elimination&lt;/a&gt;, which discusses the accuracy side of your question. Here are a few of its conclusions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;For QR factorization with or without column pivoting, the average maximal element of the residual matrix is $O(n^{1/2})$, whereas for Gaussian elimination it is $O(n)$. &lt;strong&gt;This comparison reveals that Gaussian elimination is mildly unstable, but the instability would only be detectable for very large matrix problems solved in low precision. For most practical problems, Gaussian elimination is highly stable on average.&lt;/strong&gt;&quot; (Emphasis mine)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;After the first few steps of Gaussian elimination, the remaining matrix elements are approximately normally distributed, regardless of whether they started out that way.&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There is much more to the paper that I can't capture here, including the discussion of the worst-case matrix you mentioned, so I strongly recommend that you read it.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Performance&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For square real matrices, LU with partial pivoting requires roughly $2/3 n^3$ flops, whereas Householder-based QR requires roughly $4/3 n^3$ flops. Thus, for reasonably large square matrices, QR factorization will only be about twice as expensive as LU factorization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $m \times n$ matrices, where $m \ge n$, LU with partial pivoting requires $mn^2 - n^3/3$ flops, versus QR's $2mn^2 - 2n^3/3$ (which is still twice that of LU factorization). &lt;strong&gt;However&lt;/strong&gt;, it is surprisingly common for applications to produce very tall skinny matrices ($m \gg n$), and Demmel et al. have a nice paper, &lt;a href=&quot;http://techreports.lib.berkeley.edu/accessPages/EECS-2008-74.html&quot;&gt;Communication-avoiding parallel and sequential QR factorization&lt;/a&gt;, which (in section 4) discusses a clever algorithm which only requires $\log p$ messages to be sent when $p$ processors are used, versus the $n \log p$ messages of traditional approaches. The expense is that $O(n^3 \log p)$ extra flops are performed, but for very small $n$ this is often preferred to the latency cost of sending more messages (at least when only a single QR factorization needs to be performed).&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-01-27T16:10:43.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1031" PostTypeId="1" CreationDate="2012-01-27T16:12:05.423" Score="3" ViewCount="216" Body="&lt;p&gt;I have multiple objective functions which are related to several parameters. I want to minimise more than one objective functions using several parameters. Is it even possible using BFGS?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I used L-BFGS to minimise one objective function using several parameters, it worked fine. However, I am unable to minimise multiple objective functions. At the moment, as my functions are just numbers, I've added all of the objective functions and their associated gradients in order to minimise my objective funtion. I'm unsure if that's the best way to approach this situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kind regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mo&lt;/p&gt;&#xA;" OwnerUserId="794" LastEditorUserId="276" LastEditDate="2012-01-27T16:56:38.230" LastActivityDate="2012-01-27T16:56:38.230" Title="Can BFGS be used to minimise several functions at once?" Tags="|nonlinear-programming|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1032" PostTypeId="2" ParentId="1031" CreationDate="2012-01-27T16:52:44.607" Score="6" Body="&lt;p&gt;Not by itself, no. The optimal solutions of multiobjective optimization problems are &lt;a href=&quot;http://en.wikipedia.org/wiki/Pareto_set#Pareto_frontier&quot;&gt;Pareto optimal solutions&lt;/a&gt;. Multiobjective optimization problems have their own specialized algorithms, as can be seen in the corresponding &lt;a href=&quot;http://en.wikipedia.org/wiki/Multiobjective_optimization&quot;&gt;Wikipedia article&lt;/a&gt;. Some of these methods construct surrogate single objective optimization problems, which could be solved with L-BFGS; other methods do not construct surrogate problems (such as particle swarm optimization). If you're interested in solving multiobjective optimization problems, I suggest you search that literature for methods that might be appropriate for your problem.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-27T16:52:44.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1033" PostTypeId="1" AcceptedAnswerId="1035" CreationDate="2012-01-27T18:01:02.543" Score="10" ViewCount="1673" Body="&lt;p&gt;After some Bayesian update steps, I am left with a posterior distribution of the form of a mixture of normal distributions,$$\Pr(\theta| \text{data} ) = \sum_{i=1}^k w_i N(\mu_i, \sigma^2).$$ That is, the parameter $\theta$ is drawn from a distribution whose PDF is given as a weighted mixture of normal PDFs, and is not a sum of normal RVs. I would like to draw samples $\theta\sim\Pr(\theta|\text{data})$ to use in an importance sampling approximation of this posterior.&#xA;In practice, the sum over $i$ can have a large number of terms, so that it can be impractical to choose a term $i$ according to the weights $\{w_i\}$ and then draw $\theta\sim N(\mu_i, \sigma^2)$. Is there an efficient way of drawing samples from a posterior of this form?&lt;/p&gt;&#xA;" OwnerUserId="788" LastEditorUserId="123" LastEditDate="2012-01-31T20:29:18.860" LastActivityDate="2012-01-31T20:29:18.860" Title="Drawing samples from a finite mixture of normal distributions?" Tags="|monte-carlo|probability|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1034" PostTypeId="2" ParentId="1033" CreationDate="2012-01-27T18:27:04.077" Score="2" Body="&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: This answer is incorrect, stemming from confusion in terminology (see the comment chain below for details); I'm only leaving it up as a guidepost so that people do not repost this answer (besides Barron). Please do not vote it up or down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd just use properties of random variables to reduce it to a single normally distributed random variable. The &lt;a href=&quot;http://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables&quot; rel=&quot;nofollow&quot;&gt;sum of two independent, normally distributed random variables is itself a random variable&lt;/a&gt;, so if $X_{1} \sim N(\mu_{1}, \sigma_{1}^{2})$ and $X_{2} \sim N(\mu_{2}, \sigma_{2}^{2})$, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X_{1} + X_{2} \sim N(\mu_{1} + \mu_{2}, \sigma_{1}^{2} + \sigma_{2}^{2}).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if $w_{1} \in \mathbb{R}$, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w_{1}X_{1} \sim N(w_{1}\mu_{1}, w_{1}^{2}\sigma_{1}^{2}).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using these two results combined, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Pr(\theta | \rm{data}) \sim N\big(\sum_{i=1}^{k}w_{i}\mu_{i}, \sum_{i=1}^{k}w_{i}^{2}\sigma_{i}^{2}\big).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in this case, you'll only need to pull samples from a single distribution, which should be much more tractable.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-28T15:36:29.527" LastActivityDate="2012-01-28T15:36:29.527" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1035" PostTypeId="2" ParentId="1033" CreationDate="2012-01-27T19:17:02.807" Score="6" Body="&lt;p&gt;In principle one could preselect the number of samples to be drawn from each sub-distribution, then visit each sub distribution only once and draw than number of points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Find the random set $&amp;lt;n_1, n_2, \dots, n_k&amp;gt;$ such that $n = \sum_{i=1}^k n_i$ and respecting the weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that you do this by &lt;s&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Poisson_distribution#Generating_Poisson-distributed_random_variables&quot; rel=&quot;nofollow&quot;&gt;drawing a Poisson distribution&lt;/a&gt;&lt;/s&gt; a multinomial distribution (see the comments) of mean $w_i * n$ for each sub-distribution and then normalizing the sum to $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The work here is $\mathcal{O}(k) * \mathcal{O}(n)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i=1; i&amp;lt;=k; ++i)&#xA;   for (j=1; j&amp;lt;=n[i]; ++j)&#xA;      theta ~ N(mu[i],sigma[i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The work here is $\mathcal{O}(n)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Though this means that you don't get the in random order. If random order is required you must then shuffle the draws (also big $\mathcal{O}(n)$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like the first step is dominate in run time and of the same order as the naive algorithm, but if you are sure that all $w_i * n \gg 1$ you could approximate the Poisson distributions with Normal distributions and speed up the first step.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="183" LastEditDate="2012-01-27T22:50:14.573" LastActivityDate="2012-01-27T22:50:14.573" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1036" PostTypeId="2" ParentId="1033" CreationDate="2012-01-27T20:26:14.287" Score="3" Body="&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; The original version of this question asked about a &quot;weighted sum of normal distributions&quot; to which the following answer might be useful. However, after a good bit of discussion on this answer, the answer by @Geoff, and on the question itself, it became clear the question was really on sampling a &quot;mixture of normal distributions&quot; to which this answer is not applicable.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The sum of normal distributions is a normal distribution, so you could calculate the parameters of this single distribution and then simply draw samples from that. If we call that distribution $N(\mu_{sum},\sigma_{sum}^2)$ then, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\mu_{sum} = \sum_{i=1}^k w_i\mu_i&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\sigma_{sum}^2=\sum_{i=1}^k w_i^2 \sigma_i^2&#xD;&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2012-01-30T13:34:32.190" LastActivityDate="2012-01-30T13:34:32.190" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1038" PostTypeId="1" CreationDate="2012-01-28T00:46:49.217" Score="11" ViewCount="3395" Body="&lt;p&gt;I have a dense real symmetric square matrix. The dimension is about 1000x1000.&#xA;I need to compute the first principal component and wonder what the best algorithm to do this might be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that MATLAB uses the &lt;a href=&quot;http://en.wikipedia.org/wiki/Arnoldi%27s_algorithm&quot;&gt;Arnoldi&lt;/a&gt;/&lt;a href=&quot;http://en.wikipedia.org/wiki/Lanczos_algorithm&quot;&gt;Lanczos&lt;/a&gt; algorithms (for &lt;a href=&quot;http://www.mathworks.de/help/techdoc/ref/eigs.html&quot;&gt;&lt;code&gt;eigs&lt;/code&gt;&lt;/a&gt;). But from reading about them I'm not sure whether they have any advantages over simple &lt;a href=&quot;http://en.wikipedia.org/wiki/Power_iteration&quot;&gt;power iteration&lt;/a&gt;, since my matrix is not sparse and I'm only interested in the first eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any recommendations what's the fastest algorithm in this case?&lt;/p&gt;&#xA;" OwnerUserId="804" OwnerDisplayName="Mika Fischer" LastEditorUserId="276" LastEditDate="2012-01-28T19:25:38.800" LastActivityDate="2012-01-28T20:33:41.727" Title="What's the most efficient way to compute the eigenvector of a dense matrix corresponding to the eigenvalue of largest magnitude?" Tags="|linear-algebra|algorithms|eigensystem|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1039" PostTypeId="2" ParentId="1038" CreationDate="2012-01-28T16:01:33.023" Score="14" Body="&lt;p&gt;The fastest method will likely depend upon the spectrum and normality of your matrix, but in all cases Krylov algorithms should be strictly better than power iteration. G.W. Stewart has a nice discussion of this issue in Chapter 4, Section 3 of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898715032&quot;&gt;Matrix Algorithms, Volume II: Eigensystems&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The power method is based on the observation that if $A$ has a dominant eigenpair then &#xA;  under mild restrictions on $u$ the vectors $A^k u$ produce increasingly accurate &#xA;  approximations to the dominant eigenvector. However, at each step the power method &#xA;  considers only the single vector $A^k u$, which amounts to throwing away information &#xA;  contained in the previously generated vectors. It turns out that this information is &#xA;  valuable...&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and he goes on to show that, for $100 \times 100$ diagonal matrix with the $i$'th diagonal value set to $.95^i$ (counting from $i=0$), after 25 iterations the Krylov subspace captures the dominant eigenvector eight orders of magnitude better than power iteration.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-01-28T16:01:33.023" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1040" PostTypeId="1" AcceptedAnswerId="1041" CreationDate="2012-01-28T18:01:37.470" Score="11" ViewCount="1353" Body="&lt;p&gt;Assuming one wishes to study &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_linear_algebra&quot; rel=&quot;nofollow&quot;&gt;numerical linear algebra&lt;/a&gt; in depth (and follow journals on numerical linear algebra and matrix theory), which would be a better course/better book to take up at first:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With Hoffman and Kunze with proofs and rigor (I don't have problems with rigorous math).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With Prof. Strang's book with un-rigorous proofs or &quot;stated without proof&quot; approach but heavy on applications and &quot;real world&quot; problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any other you would recommend? (How about Gene Golub's book?)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I know some bits and parts of Strang's book (supplemented by his online lectures) and some portions of numerical linear algebra from Trefethen and Bau. But, I wish to have a more thorough understanding of the subject. I will mostly self-study the books.&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="348" LastEditDate="2012-02-13T12:05:46.757" LastActivityDate="2012-02-13T12:05:46.757" Title="Which linear algebra texts should I read before learning numerical linear algebra?" Tags="|linear-algebra|reference-request|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1041" PostTypeId="2" ParentId="1040" CreationDate="2012-01-28T18:17:01.730" Score="10" Body="&lt;p&gt;I'd probably start with Gil Strang's &lt;em&gt;Introduction to Linear Algebra&lt;/em&gt;. It's best to get a solid foundation of the subject without proofs before moving on to a rigorous introduction, like learning calculus before studying real analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you study Strang's book, if you're still interested in learning more about the rigor behind linear algebra, you could try Sheldon Axler's &lt;em&gt;Linear Algebra Done Right&lt;/em&gt;, Halmos' &lt;em&gt;Finite Dimensional Vector Spaces&lt;/em&gt; (sort of reads like Rudin), or Mike Artin's &lt;em&gt;Algebra&lt;/em&gt; (for more of an abstract algebra take on things; I took his first semester abstract algebra class and loved it). Meyer's book on Matrix Analysis is also supposed to be good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're more interested in numerical linear algebra after that, you could take a look at Trefethen and Bau, Demmel's &lt;em&gt;Applied Numerical Linear Algebra&lt;/em&gt;, and Stewart's books on Matrix Algorithms.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-28T18:17:01.730" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1042" PostTypeId="1" CreationDate="2012-01-28T19:44:16.997" Score="5" ViewCount="374" Body="&lt;p&gt;How would you represent functions of type $[-1, 1]^n \to \mathbb R \;$ for moderate $n$? How would you integrate them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For small $n$ (1-2) such functions can be represented as histograms, vectors in some base, etc. For really large $n$ one employs other techniques, like neural networks or for integration monte-carlo methods. But what if I have to deal with $n= \overline{6 \ldots 12} \;$ ? I need to be able to compute such functions (for later integration to $\mathbb R$), but how to store, how to represent them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having 100 points for each dimension will lead to $100^6 = 10^{12}$, whereas GiB is around $10^9$ bytes.&lt;/p&gt;&#xA;" OwnerUserId="805" OwnerDisplayName="Yrogirg" LastEditorUserId="805" LastEditDate="2012-02-03T18:17:46.173" LastActivityDate="2012-02-03T18:17:46.173" Title="Working with multi-dimensional functions" Tags="|high-dimensional|data-storage|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1043" PostTypeId="2" ParentId="1038" CreationDate="2012-01-28T20:33:41.727" Score="6" Body="&lt;p&gt;Power iteration is the simplest, but as mentioned above it would likely converge &lt;em&gt;very&lt;/em&gt; slowly if the matrix is very non-normal. You get a &quot;hump&quot; phenomenon where the sequence appears to diverge for many iterations before asymptotic behavior kicks in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your matrix is symmetric you could consider RQI iterations, which in the symmetric case yields cubic convergence: &lt;a href=&quot;http://en.wikipedia.org/wiki/Rayleigh_quotient_iteration&quot;&gt;http://en.wikipedia.org/wiki/Rayleigh_quotient_iteration&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What makes Arnoldi or Lanczos iterations very nice (at least in my opinion, but I do not research numerical linear algebra) is that they are very versatile. It is usually possible to control which eigenvalues they give you, and how many you get. This is especially true in the symmetric case (and even better if your matrix is definite). For symmetric problems they are very robust. As a black box they work well, but they are also very receptive to new problem information, such as the ability to solve systems involving the matrix. &lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2012-01-28T20:33:41.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1044" PostTypeId="2" ParentId="1042" CreationDate="2012-01-28T20:56:08.857" Score="8" Body="&lt;p&gt;First, recognize than the functions you are working with must have some decay for any &quot;efficient&quot; method to be possible. If your functions have rich structure in all modes of the full tensor product, then you can't do better. But many functions of interest have decay in their mixed derivatives, in which case efficient methods are available. The term &quot;separation rank&quot; is sometimes used to refer to how many mixed derivatives are needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sparse tensor grids are a common and rigorously analyzed approach. For appropriate classes of functions, sparse grids reduce storage requirements from $\mathcal O(n^d)$ for a full tensor-product grid to $\mathcal O(C^d n \log^{d-1} n)$ with (usually) small value of $C$. This is a very active field, especially as it pertains to stochastic partial differential equations (SPDEs). These review papers are good sources.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1017/S0962492904000182&quot;&gt;Bungartz and Griebel (2004) Sparse grids&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1017/S0962492911000055&quot;&gt;Schwab and Gittelson (2011) Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://amath.colorado.edu/pub/wavelets/papers/BEY-MOH-2005.pdf&quot;&gt;Beylkin and Mohlenkamp (2005) Algorithms for numerical analysis in high dimensions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For some problems in dimensions around 6, some physics applications use particle, particle-in-cell, or even naive discretizations of the space (e.g. spherical harmonics to discretize a distribution in angle space at every grid cell in physical space). These methods don't have the quasi-optimality with respect to dimension that the sparse grid techniques have, but they don't require any particular decay (low separation rank).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't need to perform general manipulations of the functions, but instead want to determine some specific properties, there may be other ways to formulate the question to avoid needing to represent it in these ways.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-28T20:56:08.857" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1045" PostTypeId="2" ParentId="1026" CreationDate="2012-01-29T14:34:03.717" Score="11" Body="&lt;p&gt;I'm surprised no one mentioned &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_least_squares_%28mathematics%29&quot;&gt;linear least squares problems&lt;/a&gt;, which occur frequently in scientific computing. If you want to use Gaussian elimination, you have to form and solve the normal equations, which look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A^{T}Ax = A^{T}b,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is a matrix of data points corresponding to observations of independent variables, $x$ is a vector of parameters to be found, and $b$ is a vector of data points corresponding to observations of a dependent variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Jack Poulson frequently points out, the condition number of $A^{T}A$ is the square of the condition number of $A$, so the normal equations can be disastrously ill-conditioned. In such cases, although QR- and SVD-based approaches are slower, they yield much more accurate results.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-01-29T14:34:03.717" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1046" PostTypeId="1" AcceptedAnswerId="1056" CreationDate="2012-01-29T15:38:09.877" Score="9" ViewCount="2585" Body="&lt;p&gt;I'm profiling the performance of PETSc's linear solvers.  As I understand it,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\text{speedup}=\frac{\text{Sequential Time}}{\text{Parallel Time}}.$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that running the parallel code on one processor can be used as a proxy for the sequential performance.  However, I don't think it is a good measure of an equivalent sequential code due to the parallel overhead incurred.  Often, the performance of a sequential code is faster than the parallel performance on a single processor.  I suppose that I could search for numerical libraries that implement the same solver, but there is no guarantee that the algorithm is really equivalent.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{Parallel performance on one processor} = \text{Sequential Time} + \text{Parallel Overhead}$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, if there is a way to quantify the parallel overhead, we could subtract it from the parallel time on one processor to get a better measure of the sequential time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my questions then would be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a way to compute the parallel overhead of a parallel code run on a single processor when no sequential code is available?  &lt;/li&gt;&#xA;&lt;li&gt;Is it really necessary?  Is the parallel performance on one processor good enough to approximate the sequential performance, in general?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastEditorUserId="81" LastEditDate="2012-01-29T23:39:41.097" LastActivityDate="2012-01-29T23:39:41.097" Title="How do I compute the parallel overhead of a parallel code run on a single processor when no sequential code is available?" Tags="|performance|petsc|parallel-computing|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1047" PostTypeId="2" ParentId="1046" CreationDate="2012-01-29T16:27:35.973" Score="0" Body="&lt;p&gt;I am not familiar with PETSc internals (unlike other PETSc experts here) but imho there should be no parallel overhead with PETSc as long as you run your job as a single process (i.e., no partitioning etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember PETSc can also be installed without MPI meaning whatever small MPI overhead that might be there (assuming if at all any actual MPI calls are made when running on 1 core which I highly doubt) can also be discounted for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This obviously is true when parallel overhead is mostly communication and not algorithmic.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-01-29T16:40:47.967" LastActivityDate="2012-01-29T16:40:47.967" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1048" PostTypeId="1" AcceptedAnswerId="1055" CreationDate="2012-01-29T16:37:07.897" Score="30" ViewCount="4456" Body="&lt;p&gt;As I understand it, there are two major categories of iterative methods for solving linear systems of equations:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Stationary Methods (Jacobi, Gauss-Seidel, SOR, Multigrid)  &lt;/li&gt;&#xA;&lt;li&gt;Krylov Subspace methods (Conjugate Gradient, GMRES, etc.)  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I understand that most stationary methods work by iteratively relaxing (smoothing) the Fourier modes of the error.  As I understand it, the Conjugate Gradient method (Krylov subspace method) works by &quot;stepping&quot; through an optimal set of search directions from powers of the matrix applied to the $n$th residual.  Is this principle common to all Krylov subspace methods?  If not, how do we characterize the principle behind the convergence of Krylov subspace methods, in general?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-01-29T18:45:35.460" LastActivityDate="2012-09-14T11:01:48.587" Title="What is the principle behind the convergence of Krylov subspace methods for solving linear systems of equations?" Tags="|linear-algebra|convergence|krylov-method|conjugate-gradient|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1049" PostTypeId="5" CreationDate="2012-01-29T16:51:44.400" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-29T16:51:44.400" LastActivityDate="2012-01-29T16:51:44.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1050" PostTypeId="4" CreationDate="2012-01-29T16:51:44.400" Score="0" Body="Broyden-Fletcher-Goldfarb-Shanno method - a method for non-linear optimization" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-29T17:07:19.687" LastActivityDate="2012-01-29T17:07:19.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1051" PostTypeId="5" CreationDate="2012-01-29T16:53:39.280" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-29T16:53:39.280" LastActivityDate="2012-01-29T16:53:39.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1052" PostTypeId="4" CreationDate="2012-01-29T16:53:39.280" Score="0" Body="Referring to sparse matrices with concentrations of non-zero elements along a combination of diagonals, subdiagonals, and/or superdiagonals." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-29T17:07:15.223" LastActivityDate="2012-01-29T17:07:15.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1053" PostTypeId="5" CreationDate="2012-01-29T16:57:55.220" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-29T16:57:55.220" LastActivityDate="2012-01-29T16:57:55.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1054" PostTypeId="4" CreationDate="2012-01-29T16:57:55.220" Score="0" Body="Referring to the ratio between the performance of a sequential algorithm to its parallel implementation" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-01T04:09:15.473" LastActivityDate="2012-02-01T04:09:15.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1055" PostTypeId="2" ParentId="1048" CreationDate="2012-01-29T18:22:51.437" Score="22" Body="&lt;p&gt;In general, all Krylov methods essentially seek a polynomial that is small when evaluated on the spectrum of the matrix. In particular, the $n$th residual of a Krylov method (with zero initial guess) can be written in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ r_n = P_n (A) b $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $P_n$ is some monic polynomial of degree $n$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is diagonalizable, with $A=V\Lambda V^{-1}$, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray*}&#xA;\|r_n\| &amp;amp;\leq&amp;amp; \|V\|\cdot  \|P_n(\Lambda)\|\cdot  \|V^{-1}\|\cdot  \|b\|\\&#xA;        &amp;amp;=&amp;amp;  \kappa(V) \cdot \|P_n(\Lambda)\| \cdot \|b\|.   &#xA;\end{eqnarray*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the event that $A$ is normal (e.g., symmetric or unitary) we know that $\kappa(V) = 1.$ GMRES constructs such a polynomial through Arnoldi iteration, while CG constructs the polynomial using a different inner product (see &lt;a href=&quot;https://scicomp.stackexchange.com/a/1057/119&quot;&gt;this answer&lt;/a&gt; for details). Similarly, BiCG constructs its polynomial through the nonsymmetric Lanczos process, while Chebyshev iteration uses prior information on the spectrum (usually estimates of the largest and smallest eigenvalues for symmetric definite matrices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a cool example (motivated by Trefethen + Bau), consider a matrix whose spectrum is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/c86nC.jpg&quot; alt=&quot;Spectrum of matrix&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB, I constructed this with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = rand(200,200);&#xA;[Q R] = qr(A);&#xA;A = (1/2)*Q + eye(200,200);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we consider GMRES, which constructs polynomials which actually &lt;em&gt;minimize&lt;/em&gt; the residual over all monic polynomials of degree $n$, we can easily predict the residual history by looking at the candidate polynomial&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P_n (z) = (1-z)^n $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which in our case gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ |P_n(z)| = \frac{1}{2^n} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $z$ in the spectrum of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if we run GMRES on a random RHS and compare the residual history with this polynomial, they ought to be quite similar (the candidate polynomial values are smaller than the GMRES residual because $\|b\|_2 &amp;gt; 1$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mERCM.jpg&quot; alt=&quot;Residual history&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-09-14T01:20:01.767" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1056" PostTypeId="2" ParentId="1046" CreationDate="2012-01-29T19:24:03.880" Score="5" Body="&lt;p&gt;I think that as long as you say what you measure speedup against, no one will fault you for using the time the parallel version of the code takes to run on one processor. If you also give the total time for one of your cases (say the uni-processor time), then people will be able to compare your implementation to others in the literature or to their own. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some problems, no uni-processor result will even be possible to compute given the constraints of memory or time. Given that, most people understand when looking at speedup results, that the things are computed relative to the smallest number of processors available and that the uni-processor datum is the parallel code run on one processor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are no hard and fast rules, but you should be explicit about what you are doing and give your reader enough information to compute other quantities that might interest them.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-01-29T19:24:03.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1057" PostTypeId="2" ParentId="1048" CreationDate="2012-01-29T21:54:20.367" Score="19" Body="&lt;h2&gt;On norms&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As an addendum to Reid.Atcheson's answer, I would like to clarify some issues regarding norms. At the $n^{\mathrm{th}}$ iteration, GMRES finds the polynomial $P_n$ that minimizes the $2$-norm of the residual&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r_n = A x_n - b = \big(P_n(A) - 1 \big)b - b = P_n(A) b .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose $A$ is SPD, so $A$ induces a norm and so does $A^{-1}$. Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align*}&#xD;&#xA;\lVert r_n \rVert_{A^{-1}} &amp;amp;= r_n^T A^{-1} r_n \\&#xD;&#xA; &amp;amp;= (A e_n)^T A^{-1} A e_n \\&#xD;&#xA; &amp;amp;= e_n^T A e_n \\&#xD;&#xA; &amp;amp;= \lVert e_n \rVert_{A}&#xD;&#xA;\end{align*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we have used the error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ e_n = x_n - x_* = x_n - A^{-1} b = A^{-1} r_n $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the $A$-norm of the error is equivalent to the $A^{-1}$ norm of the residual. Conjugate gradients minimizes the $A$-norm of the error which makes it relatively more accurate at resolving low energy modes.  The $2$-norm of the residual, which GMRES minimizes, is like the $A^T A$-norm of the error, and thus is weaker in the sense that low-energy modes are less well-resolved. Note that the $A$-norm of the residual is essentially worthless because it is even weaker on low-energy modes.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Sharpness of convergence bounds&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Finally, there is interesting literature regarding different Krylov methods and subtleties of GMRES convergence, especially for non-normal operators.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/0613049&quot;&gt;Nachtigal, Reddy, and Trefethen (1992) How fast are nonsymmetric matrix iterations?&lt;/a&gt; &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/publication/PDF/1992_52.pdf&quot;&gt;(author's pdf)&lt;/a&gt; gives examples of matrices for which one method beats all others by a large factor (at least the square root of the matrix size).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://eprints.maths.ox.ac.uk/1290&quot;&gt;Embree (1999) How descriptive are GMRES convergence bounds?&lt;/a&gt; gives an insightful discussion in terms of pseudospectra which give sharper bounds and also applies to non-diagonalizable matrices.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/S003614450139961&quot;&gt;Embree (2003) The tortoise and the hare restart GMRES&lt;/a&gt; &lt;a href=&quot;http://eprints.maths.ox.ac.uk/1229/1/NA-01-22.pdf&quot;&gt;(author pdf)&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/S0895479894275030&quot;&gt;Greenbaum, Pták, and Strakoš (1996) Any nonincreasing convergence curve is possible for GMRES&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-29T21:54:20.367" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1058" PostTypeId="1" AcceptedAnswerId="1066" CreationDate="2012-01-30T03:53:51.527" Score="5" ViewCount="1359" Body="&lt;p&gt;I know that a conjugate gradient method is guaranteed to converge to the solution of a linear system if the matrix is positive definite.   I'm working with a family of matrices that have the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;sparse&lt;/li&gt;&#xA;&lt;li&gt;symmetric&lt;/li&gt;&#xA;&lt;li&gt;weakly diagonally dominant&lt;/li&gt;&#xA;&lt;li&gt;invertible&lt;/li&gt;&#xA;&lt;li&gt;all nonzero entries are strictly positive. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't think these conditions are sufficient to show that the matrix is positive definite, but I still suspect that it will converge.  Are there other known conditions under which a system such as this is known to converge?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="123" LastEditDate="2012-01-31T20:24:46.040" LastActivityDate="2012-01-31T20:24:46.040" Title="Sufficient conditions to ensure convergence of the conjugate gradient method" Tags="|linear-algebra|convergence|conjugate-gradient|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1059" PostTypeId="2" ParentId="1058" CreationDate="2012-01-30T04:22:44.823" Score="5" Body="&lt;p&gt;I think diagonal dominance + symmetry gives positive semidefinite (definite if the dominance is strict). Since you know it's invertible that gives positive definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The diagonal entries must be nonnegative for this to be true. (Thanks google/wikipedia!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Diagonally_dominant_matrix&quot; rel=&quot;noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Diagonally_dominant_matrix&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, you're looking good.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2012-01-30T04:28:55.313" LastActivityDate="2012-01-30T04:28:55.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1060" PostTypeId="5" CreationDate="2012-01-30T04:35:20.477" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T04:35:20.477" LastActivityDate="2012-01-30T04:35:20.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1061" PostTypeId="4" CreationDate="2012-01-30T04:35:20.477" Score="0" Body="Referring to systems of equations such that the absolute value of the diagonal entry of each row is greater than or equal to the sum of the absolute values of the off-diagonal entries of the same row." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-30T12:04:17.607" LastActivityDate="2012-01-30T12:04:17.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1062" PostTypeId="5" CreationDate="2012-01-30T04:39:38.443" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T04:39:38.443" LastActivityDate="2012-01-30T04:39:38.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1063" PostTypeId="4" CreationDate="2012-01-30T04:39:38.443" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T04:39:38.443" LastActivityDate="2012-01-30T04:39:38.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1064" PostTypeId="5" CreationDate="2012-01-30T04:42:34.773" Score="0" Body="" OwnerUserId="119" LastEditorUserId="20688" LastEditDate="2019-07-16T20:54:22.450" LastActivityDate="2019-07-16T20:54:22.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1065" PostTypeId="4" CreationDate="2012-01-30T04:42:34.773" Score="0" Body="Questions related to storage, assembly, operations, and other aspects of dealing with sparse matrices, for which only non-zero elements are stored. Questions that do not with sparse matrices directly, but other means of using sparsity should be tagged with [sparse-operator]." OwnerUserId="549" LastEditorUserId="20688" LastEditDate="2019-07-16T20:54:22.450" LastActivityDate="2019-07-16T20:54:22.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1066" PostTypeId="2" ParentId="1058" CreationDate="2012-01-30T05:04:39.617" Score="15" Body="&lt;p&gt;The matrix you're referring to is positive definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigenvalues of the matrix must be real, because symetric matrices are equal to their own conjugate transpose, and are thus &lt;a href=&quot;http://en.wikipedia.org/wiki/Hermitian_matrix&quot;&gt;Hermitian&lt;/a&gt;. All eigenvalues of Hermitian matrices are real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all entries of the matrix are positive or zero and the matrix is weakly diagonally dominant, then all eigenvalues of the matrix must be positive or zero by &lt;a href=&quot;http://en.wikipedia.org/wiki/Gershgorin_circle_theorem&quot;&gt;Gershgorin's Circle Theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the matrix is invertible, zero cannot be an eigenvalue of the matrix by the invertible matrix theorem. This means that all eigenvalues of the matrix must be  real and  strictly greater than zero, which means the matrix is positive definite.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-01-30T05:04:39.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1067" PostTypeId="2" ParentId="395" CreationDate="2012-01-30T05:22:13.867" Score="4" Body="&lt;p&gt;There is a new VMD plugin &lt;a href=&quot;http://www.csb.pitt.edu/ProDy/plugins/index.html&quot; rel=&quot;nofollow&quot;&gt;NMWiz&lt;/a&gt; that might be helpful. NMWiz stands for Normal Mode Wizard, but it will help visualizing any vector describing a vibration. NMWiz is available in the latest version of VMD, 1.9.1, which is beta testing now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input file format for NMWiz is a simple one, called &lt;a href=&quot;http://www.csb.pitt.edu/ProDy/reference/usage/nmd-format.html&quot; rel=&quot;nofollow&quot;&gt;NMD&lt;/a&gt;. One line for coordinates and another line for your vector is sufficient. You can show arrows, scale, resize, color them the way you want. It can also generate animations (vibrations) by generating a trajectory on the fly and you can make a high quality movie out of that using VMD.&lt;/p&gt;&#xA;" OwnerUserId="812" LastActivityDate="2012-01-30T05:22:13.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1068" PostTypeId="1" AcceptedAnswerId="1070" CreationDate="2012-01-30T08:00:23.720" Score="2" ViewCount="234" Body="&lt;p&gt;I want to plot a few sets of data points on the same x-axis that have different units. How can I set different axes for each incompatible quantity?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-01-30T08:13:24.687" Title="How can I set different axes for different plots in gnuplot?" Tags="|visualization|gnuplot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1069" PostTypeId="1" CreationDate="2012-01-30T08:03:25.753" Score="5" ViewCount="2242" Body="&lt;p&gt;I have implemented a comparability graph recognition algorithm from  &lt;a href=&quot;http://www.elsevier.com/locate/inca/699916&quot;&gt;M.C. Golumbic's &quot;Algorithmic graph theory and perfect graphs&quot;&lt;/a&gt; book. It is hinted in &lt;a href=&quot;http://or.journal.informs.org/content/55/3/569&quot;&gt;Fekete, Schepers, and van der Veen's &quot;Exact Algorithm for Higher-Dimensional Orthogonal Packing&quot;&lt;/a&gt; paper that the algorithm can be modified such that given the negative recognition result you can obtain a two-chordless cycle. But it is not explained how it is done and my knowledge on graph theory isn't anything special.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current intuition is that if the recognition algorithm fails at level $k$ then a cycle must have been created where a transitive orientation is not possible. And this would mean the set of edges at the point of failure at level $k$ would induce a subgraph that contains the cycle I'm looking for, and then I just need to prune away the edges that have no involvment with the cycle created.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions&lt;/strong&gt;: Is my intuition correct and if not, what does Fekete, Schepers, and van der Veen's &quot;simple to modify&quot; hint mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing this for efficiency, I want to obtain two results with one algorithmic search.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import networkx as nx&#xA;&#xA;def comparability(G):&#xA;  classification = dict()&#xA;  k = 0 &#xA;  def explore(i, j): &#xA;    # k and classification from outer function&#xA;    adjacent_I = set(G.neighbors(i))&#xA;    adjacent_J = set(G.neighbors(j))&#xA;    classtest = lambda e: e in classification and abs(classification[e]) != k&#xA;    for m in adjacent_I:&#xA;      if m not in adjacent_J or classtest((j, m)):&#xA;        if (i, m) not in classification:&#xA;          classification[(i, m)] = k &#xA;          classification[(m, i)] = -k&#xA;          if not explore(i, m): &#xA;            return False&#xA;        elif (i, m) in classification and classification[(i, m)] == -k: &#xA;          return False&#xA;    for m in adjacent_J:&#xA;      if m not in adjacent_I or classtest((i, m)):&#xA;        if (m, j) not in classification:&#xA;          classification[(m, j)] = k &#xA;          classification[(j, m)] = -k&#xA;          if not explore(m, j): &#xA;            return False&#xA;        elif (m, j) in classification and classification[(m, j)] == -k: &#xA;          return False&#xA;    return True&#xA;  for i, j in G.edges():&#xA;    if (i, j) not in classification:&#xA;      k += 1&#xA;      classification[(i, j)] = k &#xA;      classification[(j, i)] = -k&#xA;      if not explore(i, j): &#xA;        return False, classification&#xA;  return True, classification&#xA;&#xA;G = nx.Graph()&#xA;G.add_nodes_from([1,2,3,4])&#xA;G.add_edges_from([(1,2),(1,3),(1,4),(3,4)])&#xA;print comparability(G) # True&#xA;&#xA;G = nx.Graph()&#xA;G.add_nodes_from([1,2,3,4,5,6])&#xA;G.add_edges_from([(1,2),(2,3),(2,4),(3,4),(3,5),(4,6)])&#xA;print comparability(G) # False&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="656" LastEditorUserId="656" LastEditDate="2012-01-30T11:53:28.473" LastActivityDate="2012-01-30T11:53:28.473" Title="Two-chordless cycle extraction from a failed comparability graph recognition" Tags="|algorithms|complexity|graph-theory|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1070" PostTypeId="2" ParentId="1068" CreationDate="2012-01-30T08:13:24.687" Score="5" Body="&lt;p&gt;GNUplot has the ability to have 2 x- and 2 y-axes per plot. The additional axes are called x2 and y2. according to &lt;a href=&quot;http://www.gnuplot.info/faq/faq.html#SECTION00068000000000000000&quot;&gt;GNUplot.info FAQ&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;http://gnuplot.sourceforge.net/demo/multiaxis.html&quot;&gt;demo&lt;/a&gt; of it being used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also have a read of &lt;code&gt;help plot&lt;/code&gt; from within the GNUplot program.&lt;/p&gt;&#xA;" OwnerUserId="656" LastActivityDate="2012-01-30T08:13:24.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1071" PostTypeId="2" ParentId="1040" CreationDate="2012-01-30T12:26:09.390" Score="3" Body="&lt;p&gt;I &quot;grew up&quot; with Golub &amp;amp; Van Loan. In my opinion, best book for both theory and implementation.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-01-30T12:26:09.390" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1072" PostTypeId="5" CreationDate="2012-01-30T14:32:02.833" Score="0" Body="&lt;p&gt;For a partial differential equation of the form $au_{xx} + bu_{xy}+cu_{yy}+du_x+eu_y+fu+g=0$, a parabolic PDE has the property that $b^2-4ac=0$.  The heat equation is a classic example of a parabolic PDE. &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-30T15:25:54.577" LastActivityDate="2012-01-30T15:25:54.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1073" PostTypeId="4" CreationDate="2012-01-30T14:32:02.833" Score="0" Body="A partial differential equation that describes diffusion phenomena." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-01T04:09:32.900" LastActivityDate="2012-02-01T04:09:32.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1074" PostTypeId="5" CreationDate="2012-01-30T16:21:41.310" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T16:21:41.310" LastActivityDate="2012-01-30T16:21:41.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1075" PostTypeId="4" CreationDate="2012-01-30T16:21:41.310" Score="0" Body="Referring to matters of learning/teaching computational science" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-01T04:09:48.937" LastActivityDate="2012-02-01T04:09:48.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1076" PostTypeId="5" CreationDate="2012-01-30T16:25:09.760" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T16:25:09.760" LastActivityDate="2012-01-30T16:25:09.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1077" PostTypeId="4" CreationDate="2012-01-30T16:25:09.760" Score="0" Body="Referring to the extra time required to coordinate parallel tasks, rather than performing useful (numerical) computations." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-01T04:09:26.253" LastActivityDate="2012-02-01T04:09:26.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1078" PostTypeId="5" CreationDate="2012-01-30T16:28:00.617" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T16:28:00.617" LastActivityDate="2012-01-30T16:28:00.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1079" PostTypeId="4" CreationDate="2012-01-30T16:28:00.617" Score="0" Body="Referring to a set of partial differential equations governing electromagnetic phenomena." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-01T04:09:38.873" LastActivityDate="2012-02-01T04:09:38.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1080" PostTypeId="5" CreationDate="2012-01-30T16:30:15.743" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T16:30:15.743" LastActivityDate="2012-01-30T16:30:15.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1081" PostTypeId="4" CreationDate="2012-01-30T16:30:15.743" Score="0" Body="A general purpose high-level programming language that emphasizes ease of code syntax and readability." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-30T17:36:02.657" LastActivityDate="2012-01-30T17:36:02.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1082" PostTypeId="5" CreationDate="2012-01-30T16:31:20.953" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T16:31:20.953" LastActivityDate="2012-01-30T16:31:20.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1083" PostTypeId="4" CreationDate="2012-01-30T16:31:20.953" Score="0" Body="This tag is for requests for books, papers, and citations." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-01-30T17:36:05.247" LastActivityDate="2012-01-30T17:36:05.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1084" PostTypeId="1" AcceptedAnswerId="1085" CreationDate="2012-01-30T18:10:19.960" Score="6" ViewCount="1654" Body="&lt;p&gt;I'm not very familiar with PBS (we have torque installed here), and I have only used it to run one process per job so far, so bear with me.&lt;/p&gt;&#xA;&lt;h3&gt;The actual problem&lt;/h3&gt;&#xA;&lt;p&gt;I am trying to use Mathematica on a cluster.  Mathematica's parallel processing is implemented by launching multiple worker processes (called subkernels), and sending commands to them to be run.&lt;/p&gt;&#xA;&lt;p&gt;Mathematica has some support for PBS, and is able to launch the subkernels as PBS jobs.  It does this using the &lt;code&gt;qsub&lt;/code&gt; command.&lt;/p&gt;&#xA;&lt;p&gt;While it is the subkernels that do most of the work (actually I have some C code linked into them), it is the master Mathematica kernel that collects the results.  Therefore the master kernel also uses a small but nonzero amount of CPU time.  This means that I cannot launch the master kernel on the headnode because after it has used up a certain amount of CPU-time, it will be auto-terminated (manual launching on the headnode is not allowed, only through &lt;code&gt;qsub&lt;/code&gt;).  So I would need to launch the master kernel using &lt;code&gt;qsub&lt;/code&gt; as well.  This will result in a &lt;em&gt;job&lt;/em&gt; launching more jobs (&lt;code&gt;qsub&lt;/code&gt; being called from within a job submitted with &lt;code&gt;qsub&lt;/code&gt;), which apparently does not work.&lt;/p&gt;&#xA;&lt;p&gt;It is possible to easily test this like this:&lt;/p&gt;&#xA;&lt;p&gt;Launch an &lt;a href=&quot;http://beige.ucs.indiana.edu/I590/node37.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;interactive job&amp;quot;&lt;/a&gt; with &lt;code&gt;qsub -I&lt;/code&gt; (this is actually how I'd prefer to launch the master kernel so I can manually monitor the progress of the calculation), then in the shell that you get try to submit another job with &lt;code&gt;qsub&lt;/code&gt;:  these nested &lt;code&gt;qsub&lt;/code&gt;s result in a &lt;code&gt;qsub: Job rejected by all possible destinations&lt;/code&gt; error.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1:&lt;/strong&gt;  Is there a way to get nested &lt;code&gt;qsub&lt;/code&gt;s to work?  Can a job queue up more jobs?  This would be the ideal solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2:&lt;/strong&gt; If this is not allowed, then what is the correct way to launch more than one process from a job, &lt;em&gt;and have them distributed on free cluster nodes&lt;/em&gt;?  I assume I need to go through PBS in some way, ask it which machines are free, start the processes on those machines (how?  with &lt;code&gt;ssh executablename&lt;/code&gt;?), then tell PBS how many processes I've actually started on which machine.  Is this correct?  If yes, how can the details be implemented?&lt;/p&gt;&#xA;&lt;p&gt;I'd prefer not to have to go this second route because it would involve re-implementing Mathematica's subkernel launching with a different mechanism then &lt;code&gt;qsub&lt;/code&gt;.  It is a lot of extra work.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-01-31T16:00:37.113" Title="Starting multiple processes from a single PBS job and distributing them on the free cluster nodes" Tags="|parallel-computing|pbs|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1085" PostTypeId="2" ParentId="1084" CreationDate="2012-01-30T20:51:03.807" Score="3" Body="&lt;p&gt;The issue with question 1 is that as separate jobs, they will be unable to communicate with each other since they may run at different times. This method is better for a cluster which you have full control of, and can run semi-interactively on a head node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since that is out, the alternative is to launch the kernels yourself from within a script running on the master kernel. To launch the master kernel from a PBS script, either write a &lt;a href=&quot;http://reference.wolfram.com/mathematica/ref/program/MathematicaScript.html&quot; rel=&quot;nofollow&quot;&gt;mathematica script&lt;/a&gt; with the header line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/path/to/mathkernel -script&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if v.8 is installed on the cluster, or &lt;a href=&quot;http://reference.wolfram.com/mathematica/ref/program/math.html&quot; rel=&quot;nofollow&quot;&gt;launch the kernel&lt;/a&gt; via&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/path/to/mathkernel -initfile &quot;file-containing-script&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if v.8 is not installed.  In either case, your script needs to open with the &lt;a href=&quot;http://reference.wolfram.com/mathematica/ParallelTools/tutorial/ConnectionMethods.html#118891009&quot; rel=&quot;nofollow&quot;&gt;following code&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Needs[&quot;SubKernels`RemoteKernels`&quot;];&#xA;kernels = LaunchKernels[ RemoteMachine[#] ]&amp;amp; /@ Environment[&quot;PBS_NODEFILE&quot;];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and should close with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CloseKernels[ kernels ];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you have you can use the kernels as you wish.  The master kernel is already working on one of the nodes, and will be available for processing as the slave kernels finish.&lt;/p&gt;&#xA;" OwnerUserId="85" LastActivityDate="2012-01-30T20:51:03.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1086" PostTypeId="5" CreationDate="2012-01-30T21:32:38.323" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-01-30T21:32:38.323" LastActivityDate="2012-01-30T21:32:38.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1087" PostTypeId="4" CreationDate="2012-01-30T21:32:38.323" Score="0" Body="Portable Batch System (PBS) is software that performs job scheduling, especially allocation of batch jobs among available resources. Derivatives include TORQUE, OpenPBS, and PBS Pro." OwnerUserId="63" LastEditorUserId="81" LastEditDate="2013-01-03T23:34:03.663" LastActivityDate="2013-01-03T23:34:03.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1088" PostTypeId="1" AcceptedAnswerId="1089" CreationDate="2012-01-31T01:49:22.160" Score="17" ViewCount="1736" Body="&lt;p&gt;I always calculate things by hand, but now my comrades are getting nasty and making a lot of repetitive exercises involving just plugging things in like the expression above. I am particularly interested in open-source software such as Python or R to simplify these kinds of equations. I &lt;a href=&quot;http://www.wolframalpha.com/input/?i=Simplify%28x%5E2%2B2x%2B3%2C+x%3D%5Csqrt%282%29%2at-1%29&quot;&gt;tried using Wolfram Alpha&lt;/a&gt;, but I was unsuccessful. What open-source software packages are able to substitute the expression $x=\sqrt{2}t-1$ into the equation $x^{2}+2x+3$ and simplify the result? Specifically, I am looking for a software package that has something like a &lt;code&gt;simplify&lt;/code&gt; command.&lt;/p&gt;&#xA;" OwnerDisplayName="user664" LastEditorUserId="67" LastEditDate="2012-01-31T20:06:25.900" LastActivityDate="2015-11-01T15:32:00.423" Title="Is there any open-source or easy-to-access software that can simplify algebraic expressions like $x^{2}+2x+3, x=\sqrt{2}t-1$?" Tags="|education|symbolic-computation|" AnswerCount="8" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1089" PostTypeId="2" ParentId="1088" CreationDate="2012-01-31T02:03:29.797" Score="29" Body="&lt;p&gt;You may want to look into &lt;a href=&quot;http://sympy.org/en/index.html&quot;&gt;SymPy&lt;/a&gt;, which is a Python library with your desired &lt;a href=&quot;http://docs.sympy.org/0.7.1/modules/simplify/simplify.html&quot;&gt;simplify command&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from sympy.abc import t&#xA;&amp;gt;&amp;gt;&amp;gt; import sympy&#xA;&amp;gt;&amp;gt;&amp;gt; x = t*2**(1/2) - 1&#xA;&amp;gt;&amp;gt;&amp;gt; x**2 + 2*x + 3&#xA;2*t + (t - 1)**2 + 1&#xA;&amp;gt;&amp;gt;&amp;gt; sympy.simplify(x**2 + 2*x + 3)&#xA;t**2 + 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="67" LastEditorUserId="276" LastEditDate="2012-02-05T20:06:39.703" LastActivityDate="2012-02-05T20:06:39.703" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1090" PostTypeId="2" ParentId="1088" CreationDate="2012-01-31T02:38:18.553" Score="9" Body="&lt;p&gt;&lt;a href=&quot;http://www.sagemath.org/doc/reference/sage/symbolic/expression.html&quot;&gt;Sage can do that&lt;/a&gt; (you're going to have to scroll pretty far down the page to get to the simplification part).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, make sure you read the general introduction to symbolic math in Sage. It's semantics and syntax are quite different from &lt;em&gt;Mathematica&lt;/em&gt;, which is what most people are familiar with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example from the documentation I linked you to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sage: var('x,y,z,a,b,c,d,e,f')&#xA;(x, y, z, a, b, c, d, e, f)&#xA;sage: t = a^2 + b^2 + (x+y)^3&#xA;# substitute with keyword arguments (works only with symbols)&#xA;sage: t.subs(a=c)&#xA;(x + y)^3 + b^2 + c^2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For your case, this should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var(f,x,t)&#xA;f=x^2+2*x+3&#xA;f.subs(x=(sqrt(2)*t-1))&#xA;f.simplify()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-01-31T06:08:12.597" LastActivityDate="2012-01-31T06:08:12.597" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1092" PostTypeId="2" ParentId="1088" CreationDate="2012-01-31T07:20:22.403" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://andrejv.github.com/wxmaxima/&quot;&gt;wxMaxima&lt;/a&gt; can simplify &lt;a href=&quot;http://math-blog.com/2007/06/04/a-10-minute-tutorial-for-solving-math-problems-with-maxima/&quot;&gt;expressions&lt;/a&gt;. I believe it's supposed to be a substitute for &lt;a href=&quot;http://www.maplesoft.com/products/Maple/index.aspx&quot;&gt;Maple&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2012-01-31T07:20:22.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1093" PostTypeId="2" ParentId="1088" CreationDate="2012-01-31T09:29:33.610" Score="8" Body="&lt;p&gt;You got several good answers already with high quality advanced open source packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to point to &lt;a href=&quot;http://www.mathics.net/&quot;&gt;http://www.mathics.net/&lt;/a&gt; (http://mathics.org/ if you want to download it), which is an open source CAS using the Mathematica syntax (which you may be familiar with a little bit if you're used WolramAlpha).  It is not nearly as complete as any of the other suggestions you got.  But it can do the (very simple) operations you talked about in your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are talking about in your question is not really simplification, but &lt;em&gt;substitution&lt;/em&gt; and &lt;em&gt;expansion&lt;/em&gt; (which, unlike more complex simplification, are very easy to implement operations available even in the most basic CAS):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Mathics it would look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eq = x^2 + 2x + 3&#xA;&#xA;eq /. x -&amp;gt; Sqrt[2] t - 1&#xA;&#xA;Expand[%]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In case you need a simplification function, it's called &lt;code&gt;Simplify[]&lt;/code&gt;, and would also work in place of &lt;code&gt;Expand[]&lt;/code&gt; in the example above.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2012-01-31T09:36:24.663" LastActivityDate="2012-01-31T09:36:24.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1094" PostTypeId="2" ParentId="1088" CreationDate="2012-01-31T12:45:27.473" Score="8" Body="&lt;p&gt;As &lt;a href=&quot;https://scicomp.stackexchange.com/a/1092/149&quot;&gt;akid&lt;/a&gt; suggested, &lt;a href=&quot;http://andrejv.github.com/wxmaxima/&quot; rel=&quot;nofollow noreferrer&quot;&gt;wxMaxima&lt;/a&gt; is a great graphical front end to the venerable &lt;a href=&quot;http://en.wikipedia.org/wiki/Lisp_%28programming_language%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;lisp&lt;/a&gt; based &lt;a href=&quot;http://en.wikipedia.org/wiki/Computer_algebra_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer Algebra System&lt;/a&gt; called &lt;a href=&quot;http://maxima.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maxima&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using your example, you would get something like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(%i1) &lt;code&gt;eq1: x=t*2**(1/2)-1;&lt;/code&gt;&lt;br&gt;&#xA;  (%o1) $x=\sqrt{2}t-1$&lt;br&gt;&#xA;  (%i2) &lt;code&gt;eq2: x**2+2*x+3;&lt;/code&gt;&lt;br&gt;&#xA;  (%o2) $x^{2}+2x+3$&lt;br&gt;&#xA;  (%i3) &lt;code&gt;eq3: subst(eq1, eq2);&lt;/code&gt;&lt;br&gt;&#xA;  (%o3) $(\sqrt{2}t-1)^2+2(\sqrt{2}t-1)+3$&lt;br&gt;&#xA;  (%i4) &lt;code&gt;ratsimp(eq3);&lt;/code&gt;&lt;br&gt;&#xA;  (%o4) $2t^2+2$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or you could just do it directly:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(%i5) &lt;code&gt;ratsimp(subst(x=t*2**(1/2)-1, x**2+2*x+3));&lt;/code&gt;&lt;br&gt;&#xA;  (%o5) $2t^2+2$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Maxima has a number of different ways of simplifying, but &lt;code&gt;ratsimp&lt;/code&gt; is a good first step.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-02-02T14:50:45.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1095" PostTypeId="2" ParentId="287" CreationDate="2012-01-31T13:38:33.960" Score="1" Body="&lt;p&gt;Once I was studying/comparing several FE code libraries and I remember one of them (Hermes) was really good and it had support for hp-fem. It might be a good idea to combine theory and practice as well...so here is the link for the library:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hpfem.org/hermes&quot; rel=&quot;nofollow&quot;&gt;http://hpfem.org/hermes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have good documentation and several examples and also a Python interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also suggest this book: Ch. Schwab, &quot;&lt;em&gt;p- and hp- Finite Element Methods: Theory and Applications to Solid and Fluid Mechanics&lt;/em&gt;&quot;&lt;/p&gt;&#xA;" OwnerUserId="484" LastActivityDate="2012-01-31T13:38:33.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1096" PostTypeId="2" ParentId="1084" CreationDate="2012-01-31T16:00:37.113" Score="2" Body="&lt;p&gt;I managed to figure out that nested &lt;code&gt;qsub&lt;/code&gt;s do work, but &lt;code&gt;qsub&lt;/code&gt; can only be executed on the headnode.  Thus one needs to use &lt;code&gt;ssh headnode qsub jobname&lt;/code&gt;, where &quot;headnode&quot; is the name of the headnode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is still a hack, the better (but lengthier) solution being @rcollyer's suggestion.&lt;/p&gt;&#xA;" OwnerUserId="605" LastActivityDate="2012-01-31T16:00:37.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1097" PostTypeId="2" ParentId="1088" CreationDate="2012-01-31T18:00:02.097" Score="5" Body="&lt;p&gt;I think I was able to get &lt;a href=&quot;http://www.wolframalpha.com/input/?i=x2%2B2x%2B3+where+x%3D%E2%88%9A2%2at%E2%88%921&quot;&gt;Wolfram Alpha to work&lt;/a&gt;. Maybe I'm wrong about what you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="836" LastEditorUserId="276" LastEditDate="2012-01-31T19:50:13.707" LastActivityDate="2012-01-31T19:50:13.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1098" PostTypeId="1" AcceptedAnswerId="1099" CreationDate="2012-01-31T18:34:17.513" Score="4" ViewCount="1010" Body="&lt;p&gt;I'm working with someone else's code that uses PETSc 3.0.0-p9.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have a vector at the end of the computation that is distributed among a number of different processors, and I want to output that vector to a file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the way they do it doesn't appear to work.  Is there an easy way to write (the real part) of a vector to a file in PETSc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current way it is done is to write the PETScVector to a C++ STL vector (on all processors), and then write the vector out to a file (on one processor).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach doesn't work, because only the values on that processor are written to file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a &quot;standard PETSc&quot; way of outputting a vector to a file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="276" LastEditDate="2012-01-31T20:39:40.260" LastActivityDate="2018-10-30T23:32:05.650" Title="Outputting a distributed vector in PETSc" Tags="|parallel-computing|libraries|petsc|io|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1099" PostTypeId="2" ParentId="1098" CreationDate="2012-01-31T19:27:19.647" Score="5" Body="&lt;p&gt;Normally you can use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecView.html&quot;&gt;&lt;code&gt;VecView()&lt;/code&gt;&lt;/a&gt; for everything. You probably want to open the &lt;code&gt;PetscViewer&lt;/code&gt; using &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Viewer/PetscViewerBinaryOpen.html&quot;&gt;PetscViewerBinaryOpen&lt;/a&gt;, in which case &lt;code&gt;VecView()&lt;/code&gt; will write efficiently in parallel to the binary file.. You can read it from a PETSc program using &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecLoad.html&quot;&gt;&lt;code&gt;VecLoad()&lt;/code&gt;&lt;/a&gt;, from MATLAB/Octave with &lt;code&gt;PetscBinaryRead()&lt;/code&gt; (add &lt;code&gt;$PETSC_DIR/bin/matlab/&lt;/code&gt; to your path), or from Python (scripts in &lt;code&gt;$PETSC_DIR/bin/pythonscripts/&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are concerned about ordering or something else, please clarify the question.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-01-31T19:27:19.647" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1100" PostTypeId="2" ParentId="1088" CreationDate="2012-01-31T19:32:30.683" Score="3" Body="&lt;p&gt;open source: &lt;a href=&quot;http://maxima.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;maxima&lt;/a&gt;. And check &lt;a href=&quot;http://en.wikipedia.org/wiki/Comparison_of_computer_algebra_systems&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt; for an overview.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-01-31T19:32:30.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1101" PostTypeId="1" AcceptedAnswerId="1104" CreationDate="2012-01-31T22:19:51.763" Score="5" ViewCount="1104" Body="&lt;p&gt;I am trying to implement &lt;a href=&quot;http://www.cse.ogi.edu/~rudmerwe/pubs/pdf/vanderMerwe_GNC2004.pdf&quot; rel=&quot;nofollow&quot;&gt;an algorithm from a paper&lt;/a&gt; which makes use a QR factorization of a real matrix $A$ as a means of one of forming the Cholesky factor of $A^T A$ without explicitly forming $A^T A$ (which is generally ill-conditioned).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular:&lt;br/&gt;&#xA;We have a covariance matrix $\mathbf{P}_x = \sum_{i=0}^{2L} w_i \mathbf{x}_i \mathbf{x}_i^T$ and we want to find the upper Cholesky factorization $\mathbf{S}_x^T \mathbf{S}_x = \mathbf{P}_x$.  The first weight, $w_0$, may be negative but the rest are positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The technique advocated in the paper is a QR decomposition followed by a Cholesky update (or downdate) for the negative weight. That is, let $\mathbf{P}_x = \mathbf{P}_0 + \mathbf{P}_i$ where $\mathbf{P}_0 = w_0 \mathbf{x}_0\mathbf{x}_0^T$ and $\mathbf{P}_i = \sum_{i=1}^{2L} w_i \mathbf{x}_i \mathbf{x}_i^T$. So if $\mathbf{S_0}^T \mathbf{S_0} = \mathbf{P}_0$, then (using MATLAB's syntax) $\mathbf{S}_x = \operatorname{cholupdate}(\mathbf{S}_0, \sqrt{\|w_0\|}\mathbf{x}_0, \operatorname{sign}(w_0))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trick comes from forming $\mathbf{S}_0$ without the Cholesky decomposition. Define $\mathbf{A} = \sum_{i=1}^{2L} \sqrt{w_i} \mathbf{x}_i$, and let $\mathbf{A} = \mathbf{Q}\mathbf{R}$.  It can be seen that $\mathbf{A}^T \mathbf{A} = \mathbf{S}_0^T \mathbf{S}_0 = (\mathbf{Q} \mathbf{R})^T \mathbf{Q} \mathbf{R} = \mathbf{R}^T \mathbf{Q}^T \mathbf{Q} \mathbf{R} = \mathbf{R}^T \mathbf{R}$, and so I would expect $\mathbf{R} = \mathbf{S}_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my MATLAB implementation shows that $\mathbf{R} \neq \mathbf{S}_0$. Am I incorrect in assuming that the Cholesky factorization is unique, or is there a subtlety that I am missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example MATLAB script which demonstrates the issue:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Weights&#xA;w = [-3, 1, 1, 1, 1, 1, 1];&#xA;&#xA;% Generate a whole bunch of random vectors&#xA;A = rand(3, 7)&#xA;&#xA;% Calculate the full P Matrix&#xA;Px = zeros(3,3);&#xA;for i = 1:7&#xA;    Px = Px + w(i) * A(:,i) * A(:,i).';&#xA;end&#xA;&#xA;% Calculate part of QR&#xA;A_dash = A(:, 2:7);&#xA;B = triu(qr(sqrt(w(2)) * A_dash.', 0));&#xA;&#xA;% Cholupdate it&#xA;S = cholupdate( B(1:3, 1:3), sqrt(abs(w(1))) * A(:,1), '-')&#xA;P = S.' * S&#xA;&#xA;% DisplayCholesky of full covariance&#xA;Px&#xA;Sx = chol(Px)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output from a particular run was:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A =&#xA;&#xA;    0.1788    0.5985    0.6999    0.0688    0.6544    0.7184    0.3251&#xA;    0.4229    0.4709    0.6385    0.3196    0.4076    0.9686    0.1056&#xA;    0.0942    0.6959    0.0336    0.5309    0.8200    0.5313    0.6110&#xA;&#xA;&#xA;S =&#xA;&#xA;   -1.3442   -1.1314   -1.1479&#xA;         0   -0.1750   -0.0802&#xA;         0         0   -0.8629&#xA;&#xA;&#xA;P =&#xA;&#xA;    1.8070    1.5209    1.5430&#xA;    1.5209    1.3107    1.3128&#xA;    1.5430    1.3128    2.0686&#xA;&#xA;&#xA;Px =&#xA;&#xA;    1.8070    1.5209    1.5430&#xA;    1.5209    1.3107    1.3128&#xA;    1.5430    1.3128    2.0686&#xA;&#xA;&#xA;Sx =&#xA;&#xA;    1.3442    1.1314    1.1479&#xA;         0    0.1750    0.0802&#xA;         0         0    0.8629&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So indeed, the covariance matrices are the same, but the Cholesky factors differ by a sign.  I have a couple of questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this the result of a &lt;a href=&quot;http://en.wikipedia.org/wiki/QR_decomposition&quot; rel=&quot;nofollow&quot;&gt;QR decomposition&lt;/a&gt; not being uniquely defined? I thought the &lt;a href=&quot;http://en.wikipedia.org/wiki/Cholesky_decomposition&quot; rel=&quot;nofollow&quot;&gt;Cholesky Decomposition&lt;/a&gt; &lt;strong&gt;was&lt;/strong&gt; uniquely defined! &lt;/li&gt;&#xA;&lt;li&gt;If so, what routine or method (MATLAB, C/C++, LAPACK) will obtain the same Cholesky factor?  &lt;/li&gt;&#xA;&lt;li&gt;Or, is there a better algorithm altogether?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="761" LastEditorUserId="67" LastEditDate="2012-02-01T04:35:55.800" LastActivityDate="2012-02-01T14:43:59.013" Title="Seemingly non-unique Cholesky factor via QR rectangularisation" Tags="|linear-algebra|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1102" PostTypeId="2" ParentId="1101" CreationDate="2012-01-31T22:50:13.040" Score="4" Body="&lt;p&gt;From &lt;a href=&quot;http://en.wikipedia.org/wiki/Cholesky_decomposition&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Cholesky decomposition is unique: given a Hermitian,&#xA;  positive-definite matrix $A$, there is only one lower triangular matrix&#xA;  $L$ with strictly positive diagonal entries such that $A = LL^*$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One of your &quot;Cholesky factors&quot; has a negative diagonal.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-02-01T14:43:59.013" LastActivityDate="2012-02-01T14:43:59.013" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1103" PostTypeId="2" ParentId="1101" CreationDate="2012-01-31T22:51:29.130" Score="4" Body="&lt;p&gt;Cholesky factorizations are only unique if the matrix $\mathbf{A}$ is Hermitian positive definite. Cholesky factorizations exist in the Hermitian positive semidefinite case, but then these factorizations are nonunique. Cholesky decompositions do not exist in all other cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The QR factorization of an full rank matrix is unique, if we restrict the diagonal of $\mathbf{R}$ to be positive.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-01-31T22:58:36.670" LastActivityDate="2012-01-31T22:58:36.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1104" PostTypeId="2" ParentId="1101" CreationDate="2012-01-31T22:57:28.840" Score="9" Body="&lt;p&gt;The issue is that standard implementations of the QR decomposition do not require that $R$ has a positive diagonal. I recommend reading &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn203.pdf&quot;&gt;LAPACK Working Note 203&lt;/a&gt;, which introduces an alternate algorithm which guarantees a positive diagonal, namely xLARFP (e.g., &lt;a href=&quot;http://icl.cs.utk.edu/projectsfiles/plasma/html/htmlbrowsing/zlarfp.f.html&quot;&gt;zlarfp&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use a QR decomposition which results in an $R$ with a positive diagonal, your Cholesky factors will agree (up to floating point error). Traditionally, Cholesky factors are defined to have positive diagonals (as a result of taking square roots of positive entries), but obviously if $A=L L^H$, then $A=(-L)(-L)^H$ also holds. More generally, $A=(LD)(LD)^H$ for arbitrary diagonal $D$ whose diagonal entries lie on the complex unit circle.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-01-31T23:17:47.850" LastActivityDate="2012-01-31T23:17:47.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1106" PostTypeId="1" CreationDate="2012-02-01T07:47:12.897" Score="6" ViewCount="3020" Body="&lt;p&gt;I can find the&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;random('normal', 0, 1, 10000,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;command in MATLAB but it generates half-normal variates. I would like to generate random half-normal variates. The half-normal distribution is defined by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{|Z|}(x)=\frac{2}{\sqrt{2\pi}}e^\frac{-x^{2}}{2}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I generate a random half-normal variates from random normal variates? &lt;/p&gt;&#xA;" OwnerUserId="838" LastEditorUserId="28098" LastEditDate="2018-09-26T10:44:08.083" LastActivityDate="2019-06-20T23:52:37.130" Title="How can I generate half-normal variates in MATLAB?" Tags="|matlab|probability|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1107" PostTypeId="2" ParentId="1106" CreationDate="2012-02-01T08:00:18.653" Score="10" Body="&lt;p&gt;As &lt;a href=&quot;http://en.wikipedia.org/wiki/Half-normal_distribution&quot;&gt;Wikipedia&lt;/a&gt; says, you can just compute a normally distributed random variable with a mean of zero and take the absolute value. The half-normal probability distribution function is identical to the normal PDF with $\mu = 0$ for $X &amp;gt; 0$, except for a factor of two for the normalization.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2012-02-01T08:00:18.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1108" PostTypeId="1" AcceptedAnswerId="1109" CreationDate="2012-02-01T15:04:42.430" Score="7" ViewCount="458" Body="&lt;p&gt;I have primarily dealt with Dense Matrices arising from Electrodynamics. However, I am interested in knowing where else Dense Matrices occur. I am especially interested in knowing where they occur in:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Discretization of PDEs&lt;/li&gt;&#xA;&lt;li&gt;Engineering&lt;/li&gt;&#xA;&lt;li&gt;Science&lt;/li&gt;&#xA;&lt;li&gt;Statistics&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Sparse matrices are present in FVM, FDM, FEM and so many others. Most of them have some or the other structure. Do &quot;unstructured&quot; dense matrices ever occur in problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I apologize for such a basic question but no one seems to know the answer to this (I tried Googling as well).&#xA;I came across &lt;a href=&quot;https://stackoverflow.com/questions/5276487/applications-of-dense-linear-algebra&quot;&gt;this&lt;/a&gt; but IMHO it isn't a satisfactory answer for my question. &lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-02-01T17:29:12.007" Title="Where do dense matrices occur?" Tags="|linear-algebra|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1109" PostTypeId="2" ParentId="1108" CreationDate="2012-02-01T15:30:04.477" Score="7" Body="&lt;p&gt;It is much more common for large sparse problems to be reduced to working with one or more smaller dense matrices. For example, the &lt;a href=&quot;http://epubs.siam.org/sirev/resource/1/siread/v34/i1/p82_s1&quot;&gt;multifrontal method&lt;/a&gt; is designed to reorganize a factorization of a sparse matrix into partial factorizations of many smaller dense matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Density_functional_theory&quot;&gt;Density functional theory&lt;/a&gt; is a case where the full matrices are often treated as unstructured and dense, as a nonlinear generalized eigenvalue problem needs to be solved and this is usually done through a sequence of linear dense generalized eigensolves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another instance is in &lt;a href=&quot;http://www.mcs.anl.gov/~anitescu/PUBLICATIONS/2010/lubin%2010%20dense.pdf&quot;&gt;saddle point problems arising in stochastic programming&lt;/a&gt;. And yet another is &lt;a href=&quot;http://en.wikipedia.org/wiki/Coupled_cluster&quot;&gt;coupled cluster&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-02-01T15:30:04.477" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1110" PostTypeId="2" ParentId="1108" CreationDate="2012-02-01T15:53:29.827" Score="4" Body="&lt;p&gt;The classical choice of state variables used to represent the thermodynamic state in combustion simulations makes the Jacobian of the right hand side of the ODEs (or PDEs, if not using operator splitting) dense. Modern codes can get around this limitation by representing the thermodynamic state using an alternate choice of state variables (see &lt;a href=&quot;http://web.mit.edu/greengp/publications/pdfs/On%20Upgrading%20the%20Numerics%20in%20Combustion%20Chemistry%20Codes.pdf&quot;&gt;On Upgrading the Numerics in Combustion Chemistry Codes&lt;/a&gt; for details).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-02-01T15:53:29.827" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1111" PostTypeId="2" ParentId="1108" CreationDate="2012-02-01T16:06:13.697" Score="5" Body="&lt;p&gt;In ODEs, dense matrices occur in Spectral Collocation Methods, e.g. as described in &quot;Spectral Methods in Matlab&quot; by L. N. Trefethen (SIAM, 2000) or as implemented in the &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot;&gt;Chebfun&lt;/a&gt; system.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-02-01T16:06:13.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1112" PostTypeId="2" ParentId="1108" CreationDate="2012-02-01T17:14:35.920" Score="9" Body="&lt;p&gt;To add to other responses,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Boundary integral methods for PDEs produce dense operators. These usually have structure allowing them to be applied using FMM or $\mathcal H$-matrix techniques, but direct summation is still commonly used. The boundary integral operators are usually compact perturbations of the identity, so the discrete operators are well approximated by low rank perturbations of the identity, consequently, Krylov methods tend to converge quickly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some diffuse scattering problems produce dense operators with little structure.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Inverse and optimization methods often produce dense operators. They are frequently Schur complements of sparse PDE operators and often have exploitable structure (e.g. well approximated by low rank perturbations of the identity).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Nonlinear time-periodic PDE problems (e.g. &lt;a href=&quot;http://www.princeton.edu/~tmerlis/ocean_spinup.pdf&quot;&gt;oceans&lt;/a&gt;; see also this for &lt;a href=&quot;http://elasticity.uwyo.edu/mechanical/faculty/Dimitri_Mavriplis/HOME/NEW_PAPERS/AIAA-2012-423-514.pdf&quot;&gt;helicopters&lt;/a&gt; which is denser than other methods, but not completely dense).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-01T17:14:35.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1113" PostTypeId="2" ParentId="1108" CreationDate="2012-02-01T17:14:59.073" Score="4" Body="&lt;p&gt;The essential difference between dense and sparse linear systems is that dense systems express relationships in which each unknown depends on all (or many) of the other unknowns, while sparse systems express relationships in which each unknown depends on a small fixed number of other unknowns (often these are its &quot;neighbors&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the numerical discretization of PDEs by the &lt;a href=&quot;http://en.wikipedia.org/wiki/Method_of_lines&quot; rel=&quot;nofollow&quot;&gt;method of lines&lt;/a&gt;, dense matrices occur &lt;strong&gt;whenever the basis functions of the spatial discretization have support over the whole problem domain&lt;/strong&gt; (the &lt;em&gt;support&lt;/em&gt; of a function is the region over which it takes non-zero values).  The most common example is that of pseudospectral methods, as mentioned by Pedro.  In that case, the basis functions are typically Fourier modes or Chebyshev polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in the question, typical finite difference and finite element discretizations lead to sparse matrices, because they use basis functions with support equal to a few times the mesh spacing.  However, there are finite difference and finite element discretizations that lead to dense matrices:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;spectral collocation methods&lt;/strong&gt; can naturally be viewed as finite difference methods where the finite difference stencil extends over the whole grid&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Spectral Galerkin methods&lt;/strong&gt; can naturally be viewed as finite element methods where the basis functions extend over the whole grid.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-02-01T17:29:12.007" LastActivityDate="2012-02-01T17:29:12.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1114" PostTypeId="1" CreationDate="2012-02-01T17:34:20.137" Score="11" ViewCount="1308" Body="&lt;p&gt;Given an arbitrary set of (numerical) square complex matrices $\mathcal{A}=\{A_1,A_2,\cdots,A_m\}$, I am interested in computing the real matrix Lie algebra generated by $\mathcal{A}$, call it $\mathcal{L_\mathcal{A}}$. That is, I would like a basis for&#xA;$$&#xD;&#xA;\mathcal{L_\mathcal{A}} = \mathbb{span_R}\{B:B\in\cup_{k=1}^{\infty}\mathcal{C}_k\}&#xD;&#xA;$$&#xA;where $\mathcal{C}_k$ is defined recursively as $\mathcal{C_1}=\mathcal{A}$, and $\mathcal{C_{k+1}}=\{[X,Y]:X,Y\in\cup_{j=1}^k\mathcal{C_j}\}$ for $k\geq 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This calculation comes up in (quantum) control theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am using a method found &lt;a href=&quot;http://books.google.ca/books?id=e5M0id5enzQC&amp;amp;lpg=PP1&amp;amp;ots=Eph9MPaLSi&amp;amp;dq=Domenico%20D%27Alessandro%20Introduction%20to%20Quantum%20Control%20and%20Dynamics&amp;amp;pg=PA82#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;here&lt;/a&gt; which searches only through repeated Lie brackets (i.e. ones of the form $[A_{j_1},[A_{j_2},[A_{j_3},\cdots[A_{j_{n-1}},A_{j_n}]\cdots]]]$), and is guaranteed to terminate. However I'm interested to know if there are any other (faster) methods. Perhaps using P. Hall bases? Perhaps a recursive algorithm? My default language at the moment is Matlab.&lt;/p&gt;&#xA;" OwnerUserId="848" LastEditorUserId="549" LastEditDate="2012-02-02T15:24:16.903" LastActivityDate="2012-02-02T15:24:16.903" Title="How can I compute a basis for a matrix Lie algebra given a finite set of generators?" Tags="|linear-algebra|matlab|basis-set|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1115" PostTypeId="1" AcceptedAnswerId="1121" CreationDate="2012-02-01T17:50:16.283" Score="3" ViewCount="1036" Body="&lt;p&gt;According to &lt;a href=&quot;http://polaris.cs.uiuc.edu/~padua/cs420/Chapter8.ppt&quot; rel=&quot;nofollow&quot;&gt;these lecture notes&lt;/a&gt;, a checkerboard block decomposition should exhibit better scalability when applied to parallel matrix-vector multiplication (presumably because of greater cache hit rate).  Since the conjugate gradient method also uses a matrix-vector multiplication, I would think that this form of block decomposition would be common place, if not standard.  However, when I look at software like PETSc, it seems that row or column striped block decomposition is more common.  Is there any advantage to row or column striped block decomposition of the matrix over the checkerboard counterpart for a parallel conjugate gradient method (other than, perhaps, ease of coding)?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-02-01T19:15:39.143" LastActivityDate="2012-02-01T19:38:48.870" Title="Is a checkerboard block decomposition of a matrix useful when solving a linear system with a parallel conjugate gradient method?" Tags="|linear-algebra|parallel-computing|conjugate-gradient|matrix|block-decomposition|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1116" PostTypeId="1" CreationDate="2012-02-01T18:55:52.613" Score="10" ViewCount="965" Body="&lt;p&gt;I solve a lot of PDEs numerically, but applied math isn't my field. I haven't picked up on which applied math journals I should read to keep up with recent developments in the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are good journals to read to keep up with recent developments in solving PDEs numerically?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-02-11T21:36:40.640" Title="Which journals should I read to keep up on advances in solving PDEs numerically?" Tags="|pde|publications|" AnswerCount="2" CommentCount="5" CommunityOwnedDate="2012-02-03T00:56:39.063" ContentLicense="CC BY-SA 3.0" />
  <row Id="1118" PostTypeId="2" ParentId="1116" CreationDate="2012-02-01T19:23:56.027" Score="9" Body="&lt;p&gt;A listing of all journals that publish papers on numerical solution of PDEs would be quite long, and it would be impossible for any one person to keep up with the entirety of the subject.  But if you want to stick to journals that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are highly regarded&lt;/li&gt;&#xA;&lt;li&gt;Include fairly general classes of PDEs (as opposed to only PDEs in some scientific field)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then I would suggest starting with&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.siam.org/journals/sisc.php&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;SIAM Journal on Scientific Computing (SISC)&lt;/strong&gt;&lt;/a&gt;: This one traditionally was focused entirely on new algorithms, but has &lt;a href=&quot;http://www.siam.org/journals/sisc/redefined.php&quot; rel=&quot;nofollow&quot;&gt;recently branched out to include software or application-oriented papers&lt;/a&gt;.  It targets more of an applied math community.  Usually somewhat slower turnaround time.  Lower acceptance rate (~50%).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.journals.elsevier.com/journal-of-computational-physics/&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Journal of Computational Physics&lt;/strong&gt;&lt;/a&gt;: This includes a lot of applications papers and a lot of algorithms papers; it's more applied than SISC.  Publishes a huge volume of articles (relatively speaking).  While it is still very highly regarded, and many consider it &quot;the place&quot; to publish numerical PDEs papers, some (myself included) feel that the quality has slipped in recent years.  This may accelerate if the Elsevier boycott takes off.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/mathematics/computational+science+&amp;amp;+engineering/journal/10915&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Journal of Scientific Computing&lt;/strong&gt;&lt;/a&gt;: Relatively similar to SISC in most respects.  I think the turnaround time for reviewing may be a bit faster and the &quot;prestigiousness&quot; may be a bit lower.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An easy way to keep up with new articles is to subscribe to the journal's RSS feed.  You should also definitely subscribe to the &lt;a href=&quot;http://arxiv.org/list/math.NA/recent&quot; rel=&quot;nofollow&quot;&gt;math.NA&lt;/a&gt; feed on arXiv.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I spend most of my time figuring out how to numerically solve PDEs; you can find a longer list of journals I follow &lt;a href=&quot;http://scienceinthesands.blogspot.com/2011/10/you-are-what-you-read-journals-i-follow.html&quot; rel=&quot;nofollow&quot;&gt;on my blog&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-02-11T21:36:40.640" LastActivityDate="2012-02-11T21:36:40.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1119" PostTypeId="2" ParentId="1116" CreationDate="2012-02-01T19:26:08.507" Score="11" Body="&lt;p&gt;In no particular order, I'll add more as I think of them.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SIAM Journal on Scientific Computing&lt;/li&gt;&#xA;&lt;li&gt;Journal of Scientific Computing&lt;/li&gt;&#xA;&lt;li&gt;Journal of Computational Physics&lt;/li&gt;&#xA;&lt;li&gt;Computer Methods in Applied Mechanics and Engineering&lt;/li&gt;&#xA;&lt;li&gt;International Journal of Numerical Methods in Engineering&lt;/li&gt;&#xA;&lt;li&gt;AIAA conference proceedings&lt;/li&gt;&#xA;&lt;li&gt;Transactions on Mathematical Software&lt;/li&gt;&#xA;&lt;li&gt;Computers and Structures&lt;/li&gt;&#xA;&lt;li&gt;Acta Numerica&lt;/li&gt;&#xA;&lt;li&gt;Numerische Mathematik&lt;/li&gt;&#xA;&lt;li&gt;SIAM Journal on Numerical Analysis&lt;/li&gt;&#xA;&lt;li&gt;Mathematics of Computation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, many application-specific journals have papers on computational methods. It is definitely worth watching appropriate sections of the arXiv.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before Google Reader killed sharing, we had a group of about 30 people that would monitor different journals and share relevant papers to discuss. We are now, begrudgingly, using Google+ and Reddit for that purpose.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-01T19:26:08.507" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1121" PostTypeId="2" ParentId="1115" CreationDate="2012-02-01T19:38:48.870" Score="4" Body="&lt;p&gt;These lecture notes apply to dense matrices. Krylov methods like CG are most commonly used with sparse matrices. PETSc's most commonly used matrix formats are also intended for sparse matrices. Assuming a good ordering, sparse matrices tend to have good locality in the sense that there are many more entries in the &quot;diagonal block&quot; (relating owned parts of the vector to owned parts of the vector) than in the &quot;off-diagonal block&quot;. In these cases, a block row (or column) partition is practical and close to optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For dense matrices, there are communication advantages to using block- or element-cyclic distributions for the matrix. ScaLAPACK uses a block-cyclic distribution. Jack Poulson's &lt;a href=&quot;http://code.google.com/p/elemental/&quot; rel=&quot;nofollow&quot;&gt;Elemental library&lt;/a&gt; (see &lt;a href=&quot;http://elemental.googlecode.com/files/Elemental-rev2.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; and references therein) demonstrates advantages of elemental cyclic distributions.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-01T19:38:48.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1122" PostTypeId="1" AcceptedAnswerId="1125" CreationDate="2012-02-01T19:40:25.313" Score="25" ViewCount="8782" Body="&lt;p&gt;A very common problem in Markov Chain Monte Carlo involves computing probabilities that are sum of large exponential terms,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ e^{a_1} + e^{a_2} + ... $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the components of $a$ can range from very small to very large.  My approach has been to factor out the largest exponential term $K := \max_{i}(a_{i})$ so that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a&amp;#39; =K + log\left( e^{a_1 - K} + e^{a_2 - K } + ... \right)$$ &#xA;$$e^{a&amp;#39;} \equiv e^{a_1} + e^{a_2} + ...$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is reasonable if all elements of $a$ are large, but not such a good idea if they aren't.   Of course, the smaller elements aren't contributing to the floating-point sum anyway, but I'm not sure how to reliably deal with them.  In R code, my approach looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if ( max(abs(a)) &amp;gt; max(a) )&#xA;  K &amp;lt;-  min(a)&#xA;else&#xA;  K &amp;lt;- max(a)&#xA;ans &amp;lt;- log(sum(exp(a-K))) + K&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It seems a common enough problem that there should be a standard solution, but I'm not sure what it is.  Thanks for any suggestions.&lt;/p&gt;&#xA;" OwnerUserId="551" LastEditorUserId="9" LastEditDate="2012-02-16T21:46:34.713" LastActivityDate="2015-03-23T18:22:13.837" Title="How to add large exponential terms reliably without overflow errors?" Tags="|monte-carlo|floating-point|statistics|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1123" PostTypeId="2" ParentId="351" CreationDate="2012-02-01T19:47:02.727" Score="24" Body="&lt;p&gt;&lt;a href=&quot;http://verdandi.gforge.inria.fr/doc/linear_algebra_libraries.pdf&quot;&gt;This document&lt;/a&gt; was written in March 2009 to help in the choice of a linear algebra library for a scientific library. It evaluates portability, high-level interface and licensing for several libraries, among them Eigen, GSL, Lapack++ MTL, PETSc, Trilinos and uBlas. It seems to be particularly fond of &lt;a href=&quot;https://github.com/michael-lehn/FLENS&quot;&gt;Flens&lt;/a&gt; and &lt;a href=&quot;https://gitorious.org/seldon/&quot;&gt;Seldon&lt;/a&gt;.&#xA;(One of the requirements was that C++ templates and sparse matrices had to be supported.)&lt;/p&gt;&#xA;" OwnerUserId="849" LastEditorUserId="1804" LastEditDate="2014-06-24T15:43:14.370" LastActivityDate="2014-06-24T15:43:14.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1125" PostTypeId="2" ParentId="1122" CreationDate="2012-02-01T20:47:55.637" Score="15" Body="&lt;p&gt;There is a straightforward solution with only two passes through the data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First compute&#xA;$$K := \max_i\; a_i,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which tells you that, if there are $n$ terms, then&#xA;$$\sum_i e^{a_i} \le n e^K.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you presumably don't have $n$ anywhere near as large as even $10^{20}$, you should have no worry about overflowing in the computation of&#xA;$$\tau := \sum_i e^{a_i-K} \le n$$&#xA;in double precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, compute $\tau$ and then your solution is $e^K \tau$.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-02-02T00:29:56.957" LastActivityDate="2012-02-02T00:29:56.957" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1129" PostTypeId="1" AcceptedAnswerId="1141" CreationDate="2012-02-01T21:35:59.310" Score="21" ViewCount="3843" Body="&lt;p&gt;For the solution of large linear systems $Ax=b$ using iterative methods, it is often of interest to introduce preconditioning, e.g. solve instead $M^{-1}(Ax=b)$, where $M$ is here used for left-preconditioning of the system. Typically, we should have that $M^{-1}\approx A^{-1}$ and provide the basis for (much more) efficient solution or reduction in computational resources (e.g. memory storage) in comparison with solution of the original system (i.e. when $M=A$). However, what guidelines should we use to choose the preconditioner? How do practioneers do this for their specific problem?&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="304" LastEditDate="2012-04-05T18:18:30.980" LastActivityDate="2012-04-10T06:55:44.487" Title="What guidelines should I use when searching for good preconditioning methods for a specific problem?" Tags="|linear-algebra|iterative-method|preconditioning|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1131" PostTypeId="1" AcceptedAnswerId="1133" CreationDate="2012-02-01T22:15:41.477" Score="10" ViewCount="1023" Body="&lt;p&gt;I have an ODE:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u&amp;#39;=-1000u+sin(t)$&lt;br&gt;&#xA;$u(0)=-\frac{1}{1000001}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that this particular ODE is stiff, analytically.  I also know that if we use an explicit (forward) time stepping method (Euler, Runge-Kutta, Adams, etc.), the method should return very large errors if the time step is too large.  So, I have two questions:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this how stiff ODEs are determined, in general, when an analytical expression for the error term is not available or derivable?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In general, when the ODE is stiff, how do I determine a &quot;small enough&quot; timestep?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastEditorUserId="276" LastEditDate="2012-02-02T05:58:43.083" LastActivityDate="2012-02-04T10:42:44.513" Title="Can I use an explicit time stepping scheme to determine numerically whether an ODE is stiff?" Tags="|ode|stiffness|time-integration|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1132" PostTypeId="2" ParentId="1122" CreationDate="2012-02-01T22:51:08.513" Score="10" Body="&lt;p&gt;To keep precision while you add doubles together you need to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Kahan Summation&lt;/a&gt;, this is the software equivalent to having a carry register.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is fine for most values, but if you are getting overflow then you are hitting the limits of &lt;a href=&quot;https://en.wikipedia.org/wiki/Double_precision_floating-point_format&quot; rel=&quot;noreferrer&quot;&gt;IEEE 754 double-precision&lt;/a&gt; which would be about $e^{709.783}$. At this point you need a new representation. You can detect an overflow at addition time by &lt;code&gt;doubleMax - sumSoFar &amp;lt; valueToAdd&lt;/code&gt; and also detect exponents to large to evaluate by &lt;code&gt;exponent &amp;gt; 709.783&lt;/code&gt;. At this point you can modify the interpretation of a double by shifting the exponent and keeping track of this shift. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This for the most part is the similar to your exponent offseting approach, but this version is kept it in base 2 and does not require an initial search to find the largest exponent. Hence $\mathit{value}\times2^\mathit{shift}$.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from math import exp, log, ceil&#xA;&#xA;doubleMAX = (1.0 + (1.0 - (2 ** -52))) * (2 ** (2 ** 10 - 1))&#xA;&#xA;def KahanSumExp(expvalues):&#xA;  expvalues.sort() # gives precision improvement in certain cases &#xA;  shift = 0 &#xA;  esum = 0.0 &#xA;  carry = 0.0 &#xA;  for exponent in expvalues:&#xA;    if exponent - shift * log(2) &amp;gt; 709.783:&#xA;      n = ceil((exponent - shift * log(2) - 709.783)/log(2))&#xA;      shift += n&#xA;      carry /= 2*n&#xA;      esum /= 2*n&#xA;    elif exponent - shift * log(2) &amp;lt; -708.396:&#xA;      n = floor((exponent - shift * log(2) - -708.396)/log(2))&#xA;      shift += n&#xA;      carry *= 2*n&#xA;      esum *= 2*n&#xA;    exponent -= shift * log(2)&#xA;    value = exp(exponent) - carry &#xA;    if doubleMAX - esum &amp;lt; value:&#xA;      shift += 1&#xA;      esum /= 2&#xA;      value /= 2&#xA;    tmp = esum + value &#xA;    carry = (tmp - esum) - value &#xA;    esum = tmp&#xA;  return esum, shift&#xA;&#xA;values = [10, 37, 34, 0.1, 0.0004, 34, 37.1, 37.2, 36.9, 709, 710, 711]&#xA;value, shift = KahanSumExp(values)&#xA;print &quot;{0} x 2^{1}&quot;.format(value, shift)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="656" LastEditorUserId="656" LastEditDate="2012-02-15T11:56:12.367" LastActivityDate="2012-02-15T11:56:12.367" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1133" PostTypeId="2" ParentId="1131" CreationDate="2012-02-01T22:55:51.253" Score="6" Body="&lt;p&gt;To answer your questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;As far as I know, in practice, if explicit methods require extraordinarily small time steps &lt;em&gt;relative to your time scale of interest&lt;/em&gt; (see answers to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/891/the-definition-of-stiff-ode-system&quot;&gt;this question on what it means for an ODE to be stiff&lt;/a&gt;) in order to yield accurate results, then for all intents and purposes, your problem is stiff. To determine requirements on step size, rely on one of the many libraries out there written by experts (the MATLAB suite is one example, also SUNDIALS, VODE, DASPK, DASSL, LSODE, etc.), which have adaptive time stepping heuristics. The SUNDIALS manual explains the decision rules they use to determine the size of the time steps that package takes, to give you an example of rules that are used in practice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Again, I'd use a library with adaptive time stepping in practice, because it's more efficient to do so. However, if you were coding up a method by yourself, using fixed step sizes, if you noticed large oscillations, or your solution &quot;blowing up&quot;, then you'd suspect that your time step was too large, and reduce it. Repeat until you get a reasonably well-behaved numerical solution. Textbooks like Ascher and Petzold and Hairer and Wanner have good examples of this phenomenon.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-02-01T23:34:41.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1134" PostTypeId="2" ParentId="1122" CreationDate="2012-02-01T22:59:49.480" Score="1" Body="&lt;p&gt;Your approach is solid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't need to know $K$ exactly, just well enough to avoid overflow. So you may be able to estimate $K$ analytically before you do any MCMC sampling.&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="276" LastEditDate="2012-02-01T23:09:43.657" LastActivityDate="2012-02-01T23:09:43.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1136" PostTypeId="1" AcceptedAnswerId="5361" CreationDate="2012-02-01T23:19:35.267" Score="3" ViewCount="3042" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently working with two quantum software packages: &lt;em&gt;&lt;a href=&quot;http://www.gaussian.com/g_prod/g09.htm&quot; rel=&quot;nofollow&quot;&gt;Gaussian 09&lt;/a&gt;&lt;/em&gt; and &lt;em&gt;&lt;a href=&quot;http://www.cfour.de/&quot; rel=&quot;nofollow&quot;&gt;CFour&lt;/a&gt;&lt;/em&gt;.  &lt;em&gt;CFour&lt;/em&gt; does a very good job &lt;a href=&quot;http://slater.chemie.uni-mainz.de/cfour/index.php?n=Main.Frozen-coreCalculations&quot; rel=&quot;nofollow&quot;&gt;giving explicit control&lt;/a&gt; over how you want to invoke frozen core approximations and explains how the default handling is done via the &lt;code&gt;Frozen_Core=On&lt;/code&gt; option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Gaussian 09&lt;/em&gt; has a documentation page regarding &lt;a href=&quot;http://www.gaussian.com/g_tech/g_ur/k_fc.htm&quot; rel=&quot;nofollow&quot;&gt;frozen core&lt;/a&gt;.  All my &lt;em&gt;Gaussian 09&lt;/em&gt; jobs with frozen core use the &lt;code&gt;FC&lt;/code&gt; flag in the route section.  It is important to note that frozen core is basis-set dependent, meaning, the criteria for choosing which core(s) to freeze is directly correlated with the implemented basis set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've implemented the following basis sets in &lt;em&gt;Gaussian&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;aug-cc-pVDZ &lt;/li&gt;&#xA;&lt;li&gt;aug-cc-pVTZ&lt;/li&gt;&#xA;&lt;li&gt;aug-cc-pVQZ&lt;/li&gt;&#xA;&lt;li&gt;aug-cc-pV5Z&lt;/li&gt;&#xA;&lt;li&gt;aug-cc-pV6Z&lt;/li&gt;&#xA;&lt;li&gt;aug-cc-pV(T+d)Z&lt;/li&gt;&#xA;&lt;li&gt;aug-cc-pV(Q+d)Z&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I Need&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to know if &lt;em&gt;CFour&lt;/em&gt; and &lt;em&gt;Gaussian 09&lt;/em&gt; handles the frozen core approximation the same (particularly related to the listed basis sets above).  More specifically, if I use the &lt;code&gt;FC&lt;/code&gt; option in &lt;em&gt;Gaussian&lt;/em&gt;, will invoking the &lt;code&gt;FROZEN_CORE=ON&lt;/code&gt; option be sufficient for comparable data or will I need to use the &lt;code&gt;DROPMO&lt;/code&gt; command and explicitly tell &lt;em&gt;CFour&lt;/em&gt; which core(s) to freeze?  If the latter, how do I determine which orbitals to freeze as to match that of &lt;em&gt;Gaussian 09&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer may be right in front of me on the documentation pages but it seems rather ambiguous.  Of course some benchmark computations could be done to analyze this but I'd rather ask before embarking on such an endeavor that eats precious CPU clock cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question is strictly about program handling and job control.&lt;/p&gt;&#xA;" OwnerUserId="852" LastEditorUserId="852" LastEditDate="2012-02-03T02:50:35.250" LastActivityDate="2013-05-03T16:10:30.130" Title="How does Gaussian 09 handle 'Frozen Core Approximations' and how does it compare to CFour handling?" Tags="|computational-chemistry|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1137" PostTypeId="5" CreationDate="2012-02-01T23:35:56.337" Score="0" Body="&lt;p&gt;Systems of Linear Equations are collections of linear equations involving the same set of variables and can be solved using many different methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions relating to solution methods for systems of linear equations should be asked here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples of relevant solution types and question topics are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Elimination of Variables&lt;/li&gt;&#xA;&lt;li&gt;Row Reduction&lt;/li&gt;&#xA;&lt;li&gt;Cramers Rule&lt;/li&gt;&#xA;&lt;li&gt;Matrix methods:matrix decomposition, linear least squares, iterative solutions&lt;/li&gt;&#xA;&lt;li&gt;Using LAPACK or other linear algebra solver libraries&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16952" LastEditorUserId="16952" LastEditDate="2015-07-20T04:02:03.897" LastActivityDate="2015-07-20T04:02:03.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1138" PostTypeId="4" CreationDate="2012-02-01T23:35:56.337" Score="0" Body="Referring to methods for solving linear systems of equations." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-01T23:40:03.523" LastActivityDate="2012-02-01T23:40:03.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1139" PostTypeId="5" CreationDate="2012-02-01T23:49:49.893" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-01T23:49:49.893" LastActivityDate="2012-02-01T23:49:49.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1140" PostTypeId="4" CreationDate="2012-02-01T23:49:49.893" Score="0" Body="Referring to ordinary differential equations that require an extremely small timestep to guarantee numerical stability." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-02T00:00:52.423" LastActivityDate="2012-02-02T00:00:52.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1141" PostTypeId="2" ParentId="1129" CreationDate="2012-02-01T23:51:38.410" Score="18" Body="&lt;p&gt;I originally didn't want to give an answer because this deserves a very long treatment, and hopefully someone else will still give it. However, I can certainly give a very brief overview of the recommended approach:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Perform a &lt;em&gt;thorough&lt;/em&gt; literature search.&lt;/li&gt;&#xA;&lt;li&gt;If that fails, try every preconditioner that makes sense that you can get your hands on. MATLAB, PETSc, and Trilinos are nice environments for this.&lt;/li&gt;&#xA;&lt;li&gt;If that fails, you should think carefully about the physics of your problem and see if it is possible to come up with a cheap approximate solution, perhaps even to a slightly changed version of your problem.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Examples of 3 are shifted Laplacian versions of Helmholtz, and &lt;a href=&quot;http://www.math.psu.edu/xu/&quot;&gt;Jinchao Xu's&lt;/a&gt; recent work on preconditioning the biharmonic operator with Laplacians. &lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-02-01T23:51:38.410" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1142" PostTypeId="5" CreationDate="2012-02-01T23:53:03.070" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-01T23:53:03.070" LastActivityDate="2012-02-01T23:53:03.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1143" PostTypeId="4" CreationDate="2012-02-01T23:53:03.070" Score="0" Body="Referring to a time independent partial differential equation of the form $\nabla^2u=f$" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-02T00:00:44.327" LastActivityDate="2012-02-02T00:00:44.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1144" PostTypeId="1" AcceptedAnswerId="1160" CreationDate="2012-02-02T09:05:55.933" Score="5" ViewCount="27502" Body="&lt;p&gt;I want to plot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{n}(x) = \begin{cases}   x-n &amp;amp; \text{for } n \leq x \leq n+1 \\&#xD;&#xA;                           2-x+n &amp;amp; \text{for } n+1\leq x \leq n+2 \\&#xD;&#xA;                               0 &amp;amp; \text{otherwise.}&#xD;&#xA;\end{cases} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I plot a piecewise function like this one using open-source software?&lt;/p&gt;&#xA;" OwnerDisplayName="user664" LastEditorUserId="276" LastEditDate="2012-02-05T16:44:08.647" LastActivityDate="2020-11-15T14:43:28.290" Title="How can I plot piece-wise defined function in some easily-accessed open-source tool?" Tags="|software|python|visualization|gnuplot|" AnswerCount="6" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1145" PostTypeId="2" ParentId="1144" CreationDate="2012-02-02T09:32:34.873" Score="3" Body="&lt;p&gt;If you just want to visualize your data, you may try exporting it in a text file to visualize it with &lt;a href=&quot;http://www.gnuplot.info/&quot; rel=&quot;nofollow&quot;&gt;gnuplot&lt;/a&gt;. For your simple example, you may try to plot it in gnuplot directly as in &lt;a href=&quot;http://en.wikibooks.org/wiki/Gnuplot#Types_of_functions&quot; rel=&quot;nofollow&quot;&gt;this example&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab/Octave, if you have your function as pairs of data x1/y1 and x2/y2, you can &lt;a href=&quot;http://www.mathworks.co.uk/help/techdoc/ref/plot.html&quot; rel=&quot;nofollow&quot;&gt;plot&lt;/a&gt; them using &lt;code&gt;plot( x1 , y1 , x2 , y2 )&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-02-02T09:32:34.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1146" PostTypeId="2" ParentId="1144" CreationDate="2012-02-02T09:54:57.730" Score="3" Body="&lt;p&gt;Another option would be to use the &lt;a href=&quot;http://matplotlib.sourceforge.net&quot; rel=&quot;nofollow&quot;&gt;matplotlib&lt;/a&gt; package in Python. You can create a function f(n, x) that evaluates the function rather easily, and then evaluate it for a set of points. The resulting plot commands are very MATLAB-like, so if you know MATLAB, the work in matplotlib will be quite easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're a student, you can get a free, easy-to-install academic version of Python with NumPy, matplotlib, and a whole bunch of other packages pre-installed via the &lt;a href=&quot;http://enthought.com/products/epd.php&quot; rel=&quot;nofollow&quot;&gt;Enthought Python Distribution&lt;/a&gt;. It's quite useful, and takes out a lot of the guesswork in terms of installing the core Python packages.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-02-02T09:54:57.730" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1147" PostTypeId="2" ParentId="1114" CreationDate="2012-02-02T13:02:10.063" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://planning.cs.uiuc.edu/node834.html&quot; rel=&quot;nofollow&quot;&gt;This link&lt;/a&gt; describes how to do this using P. Hall bases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On an only somewhat related note, if I were implementing this I would worry about the numerical instability of testing linear dependency. Make sure to use a method for testing independence of new matrices that allows for numerical inaccuracy - maybe comparing the norm of $A - p(A)$ to the norm of $A$, where $p$ is the projection onto the space of matrices you've found before.&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2012-02-02T13:02:10.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1148" PostTypeId="1" CreationDate="2012-02-02T14:20:04.393" Score="56" ViewCount="7532" Body="&lt;p&gt;Every scientist needs to know a bit about statistics: what correlation means, what a confidence interval is, and so on. Similarly, every scientist ought to know a bit about computing: the question is, what? What it is reasonable to expect every working scientist to know about building and using software? &lt;a href=&quot;http://software-carpentry.org/blog/2012/01/revising-the-curriculum.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Our list&lt;/a&gt; of core skills---the things people ought to know &lt;em&gt;before&lt;/em&gt; they tackle anything with &quot;cloud&quot; or &quot;peta&quot; in its name---is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;basic programming (loops, conditionals, lists, functions, and file I/O)&lt;/li&gt;&#xA;&lt;li&gt;the shell/basic shell scripting&lt;/li&gt;&#xA;&lt;li&gt;version control&lt;/li&gt;&#xA;&lt;li&gt;how much to test programs&lt;/li&gt;&#xA;&lt;li&gt;basic SQL &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There's a lot that &lt;em&gt;isn't&lt;/em&gt; in this list: matrix programming (MATLAB, NumPy, and the like), spreadsheets when used well, they're as powerful as most programming languages), task automation tools like Make, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So: what's on &lt;em&gt;your&lt;/em&gt; list? What do you think it's fair to expect every scientist to know these days? And what would you take out of the list above to make room for it? Nobody has enough time to learn everything.&lt;/p&gt;&#xA;" OwnerUserId="858" LastEditorUserId="-1" LastEditDate="2018-08-29T15:09:08.813" LastActivityDate="2018-08-29T15:09:08.813" Title="What core skills should every computational scientist have?" Tags="|education|" AnswerCount="7" CommentCount="11" ClosedDate="2012-02-03T19:49:48.450" CommunityOwnedDate="2012-02-02T22:59:53.497" ContentLicense="CC BY-SA 4.0" />
  <row Id="1149" PostTypeId="2" ParentId="1148" CreationDate="2012-02-02T15:05:59.967" Score="12" Body="&lt;p&gt;A computational scientist must have sufficient familiarity in computer science, mathematics, and an application field in science/engineering.  I would include skills in each of the following areas:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematics:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Numerical Analysis  &lt;/li&gt;&#xA;&lt;li&gt;Linear Algebra&lt;/li&gt;&#xA;&lt;li&gt;Ordinary, Partial, and/or Stochastic Differential Equations  &lt;/li&gt;&#xA;&lt;li&gt;Optimization  &lt;/li&gt;&#xA;&lt;li&gt;Statistics and/or Probability  &lt;/li&gt;&#xA;&lt;li&gt;Inverse Theory&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Computer Science:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Algorithms  &lt;/li&gt;&#xA;&lt;li&gt;Data Structures&lt;/li&gt;&#xA;&lt;li&gt;Parallel Programming (MPI,OpenMP,CUDA, etc.)  &lt;/li&gt;&#xA;&lt;li&gt;Scientific Visualization  &lt;/li&gt;&#xA;&lt;li&gt;Computer Architecture&lt;/li&gt;&#xA;&lt;li&gt;Using a Linux Environment&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Science / Engineering - depends on the application that you want to specialize in.  In my particular case (engineering), I would add things like continuum mechanics, heat transfer, fluid dynamics, finite element method, etc.  I would say that the more familiarity you have with multiple fields of science, the more versatile you are as a computational scientist.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-02T18:53:27.763" LastActivityDate="2012-02-02T18:53:27.763" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1150" PostTypeId="2" ParentId="1148" CreationDate="2012-02-02T15:21:58.200" Score="26" Body="&lt;p&gt;My own background is in Computer Science proper, so my opinions may be a bit biased. Having said that, I'd add &quot;basic algorithms and data structures&quot; to the list. What I mean with basics are essentially linear searching and sorting, and data structures such as balanced trees, heaps and or hash tables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why? Well, in most computational algorithms, you end up spending an extraordinary amount of time and effort shifting data around and not actually computing anything. Ever implement a Finite-Element code? That's about 90% data organization. The difference between getting this done and getting it done well can be an order of magnitude in computational efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One minor Computer Science-related point I would also add is a short introduction on how a processor actually works and what it's good at and what it is not. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Addition and multiplication are fast, division or transcendental functions are not. I've seen grown men replace a square root operation with something that required three divisions and think they've done something great (division and square root are just as expensive).&lt;/li&gt;&#xA;&lt;li&gt;Level 3 caches are getting bigger every year, yes, but the Level 0 cache, i.e. the really fast one, is still only a few kilobytes.&lt;/li&gt;&#xA;&lt;li&gt;Compilers are not magic. They may unroll small loops or vectorize extremely straight-forward operations, but they won't turn that bubblesort into a quicksort.&lt;/li&gt;&#xA;&lt;li&gt;Calling methods on polymorphic objects with multiple inheritance in your innermost loop may be conceptually sweet, but it will make your CPU want to kill itself.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is nitty-gritty boring stuff, but it takes just a few minutes to explain, and keeping it in mind will let you write good code from the get-go, or design algorithms that don't rely on nonexistent hardware features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to what to remove from the list, I think SQL is a bit much for Computational Scientists. Also, Software testing is important, but it is a science in itself. Unit testing and correct abstract data types are something that should be taught with basic programming, and does not require a two year master's programme.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-02-02T17:54:37.057" LastActivityDate="2012-02-02T17:54:37.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1151" PostTypeId="1" CreationDate="2012-02-02T16:03:09.210" Score="8" ViewCount="411" Body="&lt;p&gt;What are the &quot;Pros and Cons&quot; for these two methods of mesh-refinement? Both seem to be the prevalent methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can naturally imagine that global red refinement is comparatively easy to implement and attractive for research in multigrid-methods, while NVB is used in AFEM-theory because some attractive formal properties that ease error analysis, and furthermore local adaption with NVB seems easier to implement than with Red-Green-Refinement.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2013-07-28T15:27:19.457" Title="Red(-Green)-Refinement vs. Newest-Vertex-Bisection" Tags="|hpc|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1152" PostTypeId="2" ParentId="1148" CreationDate="2012-02-02T16:04:13.390" Score="48" Body="&lt;p&gt;&quot;Computational Scientist&quot; is somewhat broad because it includes people who doing numerical analysis with paper/LaTeX and proof-of-concept implementations, people writing general purpose libraries, and people developing applications that solve certain classes of problems, and end users that utilize those applications. The skills needed for these groups are different, but there is a great advantage to having some familiarity with the &quot;full stack&quot;. I'll describe what I think are the critical parts of this stack, people who work at that level should of course have deeper knowledge.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Domain knowledge (e.g. physics and engineering background)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Everyone should know the basics of the class of problems they are solving. If you work on PDEs, this would mean some general familiarity with a few classes of PDE (e.g. Poisson, elasticity, and incompressible and compressible Navier-Stokes), especially what properties are important to capture &quot;exactly&quot; and what can be up to discretization error (this informs method selection regarding local conservation and symplectic integrators). You should know about some functionals and analysis types of interest to applications (optimization of lift and drag, prediction of failure, parameter inversion, etc).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Mathematics&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Everyone should have some general familiarity with classes of methods relevant to their problem domain. This includes basic characteristics of sparse versus dense linear algebra, availability of &quot;fast methods&quot;, properties of spatial and temporal discretization techniques and how to evaluate what properties of a physical problem are needed for a discretization technique to be suitable. If you are mostly an end user, this knowledge can be very high level.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Software engineering and libraries&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Some familiarity with abstraction techniques and library design is useful for almost everyone in computational science. If you work on proof-of-concept methods, this will improve the organization of your code (making it easier for someone else to &quot;translate&quot; it into a robust implementation). If you work on scientific applications, this will make your software more extensible and make it easier to interface with libraries. Be defensive when developing code, such that errors are detected as early as possible and the error messages are as informative as possible.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Tools&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Working with software is an important part of computational science. Proficiency with your chosen language, editor support (e.g. tags, static analysis), and debugging tools (debugger, valgrind) greatly improves your development efficiency. If you work in batch environments, you should know how to submit jobs and get interactive sessions. If you work with compiled code, a working knowledge of compilers, linkers, and build tools like Make will save a lot of time. Version control is essential for everyone, even if you work alone. Learn Git or Mercurial and use it for every project. If you develop libraries, you should know the language standards reasonably completely so that you almost always write portable code the first time, otherwise you will be buried in user support requests when your code doesn't build in their funky environment.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;LaTeX&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;LaTeX is the de-facto standard for scientific publication and collaboration. Proficiency with LaTeX is important to be able to communicate your results, collaborate on proposals, etc. Scripting the creation of figures is also important for reproducibility and data provenance.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-02-03T12:46:44.607" LastActivityDate="2012-02-03T12:46:44.607" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1153" PostTypeId="2" ParentId="1131" CreationDate="2012-02-02T16:36:07.180" Score="9" Body="&lt;h2&gt;1. Can we numerically detect stiffness just by applying explicit methods?&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Suppose you have an initial value problem for some ODE on $[0,10]$.&#xA;You take considerably large stepsize $\tau=1$ and an explicit Euler&#xA;method, make your calculations &lt;em&gt;with constant step size&lt;/em&gt; $\tau$ and&#xA;get these points:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7o3q3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You estimate the error and it appears to be big. Okay, then you take&#xA;$\tau=0.1$ and obtain &lt;img src=&quot;https://i.stack.imgur.com/7E5qO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error estimate is acceptable now. Stepsize $\tau=0.1$ is small&#xA;relatively to $[0,10]^\star$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is the problem stiff? The answer is &lt;strong&gt;NO&lt;/strong&gt;! &lt;em&gt;Small stepsize here is required to correctly reproduce the oscillations of solution&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem we've solved is $$ y'(t)=-2\cos \pi t,\quad y(0)=1. $$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Now you take another ODE on the same interval, $\tau=1$ and explicit Euler gives you almost the same numerical solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7fUT0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You take $\tau=0.1$ and now the numerical solution is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OM7sk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error estimate now is small. Stepsize $\tau=0.1$ is small relatively to $[0,10]^\star$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this problem stiff? &lt;strong&gt;YES&lt;/strong&gt;! We've made very much small steps to reproduce the solution    which is changing very slowly. This is irrational! &lt;em&gt;The magnitude of time step here is limited by the stability properties of explicit Euler&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  y'(t)=-2 y(t)+\sin t/2,\quad y(0)=1.&#xA;  $$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;$^\star$ Note that the number of step sizes can be much greater if we take longer interval of integration.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Conclusion: &lt;strong&gt;the information about timesteps and corresponding errors is not sufficient to detect stiffness. You should also look at the obtained solution. If it varies slowly and stepsize is very small, the problem is most likely to be stiff. If the solution oscillates rapidly and you trust your error estimation technique then this problem is not stiff.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;2. How to determine the maximum stepsize which allows to integrate stiff problem with explicit method?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you use some black-box explicit solver with automatic step control then you need nothing to do: the software will take the required stepsize adaptively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But suppose you want to get the solution manually with constant stepsize, or just want to estimate how many hours you should wait until your explicit method crunches the problem.&#xA;Then you should know the spectrum of your Jacobi matrix. Suppose it is real and lies in $[\Lambda,0]$ (in your example $\Lambda=-1000$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you should calculate the real &lt;em&gt;stability interval&lt;/em&gt; (&lt;em&gt;stability domain&lt;/em&gt; in complex case) of your explicit method. It is not too difficult, you need just consult any textbook on this topic. In the case of explicit Euler this interval is $[-2,0]$. Now, if you want your solution not to blow-up, you should take $\tau$ such that $\Lambda\tau$ lies in the stability interval, i. e. in our case&#xA;$$&#xA;\tau\leq \frac{2}{|\Lambda|}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want more consistency, you should take &#xA;$$&#xA;\tau\leq \frac{1}{|\Lambda|},&#xA;$$&#xA;since for $1/|\Lambda|&amp;lt;\tau \leq 2/|\Lambda|$ your solution will most likely produce unnatural (but fading) oscillations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Of course such an analysis is mostly applicable for linear problems with known spectrum. For more practical problems we should rely on numerical methods of stiffness detection (see references and comments in other answers).&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="720" LastEditDate="2012-02-04T10:42:44.513" LastActivityDate="2012-02-04T10:42:44.513" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1154" PostTypeId="2" ParentId="1148" CreationDate="2012-02-02T17:14:34.940" Score="4" Body="&lt;p&gt;Great question followed by fascinating answers! I would like to butt in with just one small addition. As far as I have experienced (myself and vicariously), an All-in-One tool is usually really good to know. Such a tool could be MATLAB, Octave or even Python (with libraries). Whenever you have a problem across your &quot;comfort zone&quot;, a good idea (as far as I know and think) would be to try your hand at an All-in-One tool. You can try writing your own codes later.&#xA;The beauty of such packages is that the programming doesn't interfere with the understanding of what you are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take an example of Computer Graphics. Writing a code for translation, rotation or scaling of a figure is 10 lines of code in MATLAB (tops) but it can run for pages in C. I'm not saying C is  not good. All I am saying is that if you don't have a good reason to write codes in C, MATLAB would be a simpler, better and a more intuitive way out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some may disagree and state that C-like programming is a great way of building intuition. Maybe it is. But when you don't have to deal with a problem for more than a few times, it is hardly warranted to sit and write your own codes in a language like C.&lt;/p&gt;&#xA;" OwnerUserId="529" LastActivityDate="2012-02-02T17:14:34.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1156" PostTypeId="2" ParentId="1148" CreationDate="2012-02-02T18:20:56.933" Score="13" Body="&lt;p&gt;&lt;strong&gt;Floating point math.&lt;/strong&gt;&#xA;Most science deals with real world values, and real world values are often represented as floating point in the computer world.  There are many potential gotchas with floats that can wreak havoc on the meaningfulness of results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The favorite reference for this topic seems to be &quot;What Every Computer Scientist Should Know About Floating Point Arithmetic (1991)&quot; by David Goldberg &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.22.6768&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.22.6768&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="868" LastActivityDate="2012-02-02T18:20:56.933" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1157" PostTypeId="2" ParentId="1148" CreationDate="2012-02-02T18:48:30.440" Score="18" Body="&lt;p&gt;I might add to this later, but for starters, I'd take out &quot;shell scripting&quot; and replace it specifically with &quot;Python scripting&quot;. Python is much more portable than shell scripting, and more readable than comparable shell and scripting languages. Its large standard library and popularity in the sciences (with the possible exception of biology, which also uses Perl) makes it a great computational &lt;em&gt;lingua franca&lt;/em&gt;, not to mention a good first language for learning programming. It is now the first language taught to EECS majors at MIT, and it is popular in the job market, particularly in scientific computing. Its online documentation is extensive, and there are also a number of programming texts based on Python available online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Python, you could teach basic programming constructs, as well as scripting. In addition, Python has excellent support for unit testing, so Python could be used to teach unit testing as well. Python also has an extensive database API (which could replace or augment having to learn SQL), and a couple build utilities that offer Make-like functionality. I personally prefer SCons over Make, because I find Python easier to document and test than shell scripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, the motivating principle behind my blatant shilling for Python is efficiency. It's a lot easier to streamline your workflow if you can do most of your work in one language or one tool, especially when that tool is an expressive scripting language. Sure, I could do everything in C, but my program would be 5 times as long, and chances are, I don't need the speed. Instead, I can use Python to import data from a text file, plot it, call optimization routines, generate random variates, plot my results, write results to a text file, and unit test my code. If Python is too slow, it's possible to wrap Python around C, C++, or Fortran code that takes care of computationally intensive tasks. Python is, for me, a one-stop shop for most of my scientific computing needs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python isn't exactly MATLAB yet; SciPy and NumPy still have a ways to go in terms of functionality, but in terms of general utility, I use Python for a wider variety of tasks than MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-02-02T18:48:30.440" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1158" PostTypeId="2" ParentId="1148" CreationDate="2012-02-02T19:51:07.443" Score="-2" Body="&lt;p&gt;Common sense and gut feeling... The latter comes only with time and after having &quot;survived&quot; a couple of shameful experiences in the big bad world.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-02-02T19:51:07.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1159" PostTypeId="1" AcceptedAnswerId="1163" CreationDate="2012-02-02T19:53:11.303" Score="8" ViewCount="346" Body="&lt;p&gt;I am trying to solve a one-sided non-linear least-squares problem with linear constraints, i.e the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_{\mathbf{x}} \quad  \sum^m_{i=1} \mathbf{r}_i(\mathbf{x}) \qquad \text{ s.t } \quad A\mathbf{x} \leq \mathbf{b}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_i(\mathbf{x})=f_i(\mathbf{x})^2$ if $f_i(\mathbf{x})&amp;gt;0$, and $r_i(\mathbf{x})=0$ else. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In words, this can be considered a least-squares problem where only the positive residuals (the $f$'s) are included. &lt;strong&gt;I cannot stress enough that this is not a datafitting case. I am aware of what would happen if used for most datafitting cases, where the result would merely be a function that is &quot;above&quot; all the observations.&lt;/strong&gt; The application is for solving a very specific optimization problem that is normally solved in the minmax norm ($\min_{\mathbf{x}} ||\mathbf{f}(\mathbf{x})||_\infty$). In all practical cases, the solution does not reach zero, i.e. $||\mathbf{f}(\mathbf{x})||_\infty \neq 0$ due to the behaviour of the $f$ functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $f$ functions are non-linear, and we have access to their derivatives, such that we can analytically calculate the Jacobian without much extra trouble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have, with some success, applied a Levenberg-Marquardt algorithm where the objective function is formulated as above, i.e. the $f$'s below 0 are removed from the sum, and with the corresponding rows of the Jacobian $J$ set to zero (i.e. $J_{i,:}=0$ if $f_i(\mathbf{x})&amp;lt;=0$. This is rather crude but works OK, unfortunately we have not been able to incorporate the linear constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are aware of a number of methods that solve the NLLSQ problem with only bound constraints, but those methods obviously does not solve our problem. We have found only one NLLSQ with linear constraints, called DQED, and have used that with limited succes (we are unhappy with the number of iterations/function evaluations) by modifying our objective function as we did with Levenberg Marquardt.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What I am looking for&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Any suggestions for methods that solve the non-linear least squares problem with linear constraints. Also, suggestions on how to modify algorithms to incorporate the fact that we only include the positive residuals are more than welcome. Finally, any tips or thoughts are most welcome, though I must stress again that the formulation of the problem is not wrong, though I realize that it is not the most suitable one for optimization due to the lack of differentiability of $r_i(\mathbf{x})$ when $r_i(\mathbf{x})=0$.&lt;/p&gt;&#xA;" OwnerUserId="370" LastEditorUserId="27" LastEditDate="2012-02-02T22:27:44.317" LastActivityDate="2016-12-02T11:29:53.207" Title="One-sided non-linear least squares with linear constraints" Tags="|algorithms|optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1160" PostTypeId="2" ParentId="1144" CreationDate="2012-02-02T20:04:15.887" Score="10" Body="&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt &#xA;&#xA;def fun (n, x): &#xA;    if n &amp;lt;= x &amp;lt;= n + 1:&#xA;        return float(x) - n &#xA;    elif n + 1 &amp;lt;= x &amp;lt;= n + 2:&#xA;        return 2.0 - x + n &#xA;    return 0.0 &#xA;&#xA;vfun = np.vectorize(fun)&#xA;&#xA;x = np.linspace(0, 10, 1000)    &#xA;y = vfun(3, x)&#xA;&#xA;plt.plot(x, y, '-')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastEditorUserId="656" LastEditDate="2012-02-03T16:01:15.643" LastActivityDate="2012-02-03T16:01:15.643" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1162" PostTypeId="2" ParentId="1131" CreationDate="2012-02-02T21:06:48.803" Score="9" Body="&lt;p&gt;A better way to look at it is that for a stiff problem, &lt;strong&gt;any stable explicit calculation leads to an error that is much &lt;em&gt;smaller&lt;/em&gt; than the required error tolerance&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many good methods for automatically detecting stiffness using explicit schemes, especially embedded Runge-Kutta pairs.  See for example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1016/0898-1221%2877%2990112-2&quot;&gt;Detecting stiffness with the Fehlberg (4, 5) formulas&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1145/293686.287641&quot;&gt;Stiffness detection and estimation of dominant spectrum with explicit Runge-Kutta methods&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://link.aip.org/link/doi/10.1063/1.3498435&quot;&gt;Stiffness Detection Strategy for Explicit Runge Kutta Methods&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In faleichik's second example, as the step size is reduced, one would see a sudden dramatic decrease in the error to levels far below a typical desired tolerance as the stable timestep threshold is crossed.  So a good error estimator would indeed reveal the stiffness of the problem.  In the first problem, the error obtained with a stable step size would be in the range of the typical desired tolerance, indicating non-stiffness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note as a consequence that any problem becomes non-stiff if a sufficiently strict error tolerance is required.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-02-02T21:13:57.500" LastActivityDate="2012-02-02T21:13:57.500" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1163" PostTypeId="2" ParentId="1159" CreationDate="2012-02-02T22:44:35.873" Score="2" Body="&lt;p&gt;A lot of my answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/541/software-package-for-constrained-optimization&quot;&gt;this question&lt;/a&gt; also applies here; just ignore the PDE part and pretend I'm talking about a run-of-the-mill nonlinear optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, since you have continuous functions, you can use direct search methods, though the convergence of those methods can be slower than corresponding methods that use higher-order derivative information (which, in this case, you don't have). You could also try looking at nonsmooth optimization solvers, such as those that use bundle methods. The major name in the field is Frank Clarke, who developed much of the theory behind nonsmooth optimization; I don't know much about that body of literature otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that due to the nondifferentiabilities in your objective, using methods that assume first- or second-order derivative information is causing problems with convergence.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-02-02T22:44:35.873" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1164" PostTypeId="2" ParentId="940" CreationDate="2012-02-03T00:00:28.560" Score="5" Body="&lt;p&gt;In molecular dynamics simulations we have the same problem: given a cutoff radius $r_c$, find all particles within at most $r_c$ of each other. The simplest $\mathcal O(n)$ approach is to divide the space into cells of edge length of at least $r_c$ and to compare every particle in every cell to all particles in the 26 neighboring cells (in three dimensions, for $d$ dimensions, this is $3^d-1$). By construction, particles in other cells will be farther away than $r_c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case where you're only testing a single position for a fixed set of points, divide the space into cells of edge length $\ge r_c$, sort the points into the cells (can be done in $\mathcal O(n)$), and then for each trial point, find the cell in which that point lies (can be done in $\mathcal O(1)$) and check the distance to the points in that cell and the surrounding cells (can be done in $\mathcal O(1)$, which is dependent on the point density and $r_c$, but not on the total number of points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole procedure is described &lt;a href=&quot;http://en.wikipedia.org/wiki/Cell_lists&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. If you want more details, try googling for &quot;cell linked list&quot; or &quot;cell list&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/K-d_tree&quot; rel=&quot;noreferrer&quot;&gt;k-d tree&lt;/a&gt; is also a good approach, but may be more difficult to implement on your own (there seems to be a Python implementation &lt;a href=&quot;http://code.google.com/p/python-kdtree/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, but it doesn't allow adding points). Don't worry about the complexity of the tree, however, since, for a more or less uniform point distribution, the search depth will behave as $\mathcal O(\log_2 n)$ for $n$ points. That is, doubling the amount of points will make the search effort grow by a constant factor. You also only have to construct the tree once and can update it quickly when adding new points.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-02-03T00:08:12.403" LastActivityDate="2012-02-03T00:08:12.403" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1165" PostTypeId="2" ParentId="22" CreationDate="2012-02-03T01:16:30.317" Score="1" Body="&lt;p&gt;This question is two month old but I'll add information anyway. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Chiba &amp;amp; Nishizeki's algorithm is for any arbitrary graph so the complexity is guaranteed for any class of graph. It is hinted in &lt;a href=&quot;http://www.cs.ryerson.ca/~eharley/research/clique.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; that Chiba &amp;amp; Nishizeki's algorithm actually performs quite poor compared to the modified and optimized versions of Bron-Kerbosch algorithm. Of cause this could be because their implementation is poor or the data set they used.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The paper is usually the reference implementation, off the top of my head I don't know of any great libraries, I'm usually a hand coder type of guy. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you take into account that a maximal independent set in some graph, it is a maximal clique in the complementary graph. You can implement maximal independent set enumeration on the interval graph instead. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0020019088900658&quot; rel=&quot;nofollow&quot;&gt;D.S. Johnson's &quot;On generating all maximal independent sets&quot;&lt;/a&gt; has a nice and clear explanation of his polynomial delay algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also $\text{co-interval} \equiv \text{co-chordal} \cap \text{comparability}$, you may be able to take advantage that you are using $\text{comparability}$ graph. &lt;em&gt;If&lt;/em&gt; you don't need to enumerate maximal cliques and you only need a single instance (I known nothing about genomic sequences) then an &lt;a href=&quot;http://www.elsevier.com/wps/find/bookdescription.cws_home/699916/description#description&quot; rel=&quot;nofollow&quot;&gt;efficient max weight clique algorithm&lt;/a&gt; (chapter 5 section 7) exist for $\text{comparability}$ graphs. &lt;/p&gt;&#xA;" OwnerUserId="656" LastEditorUserId="656" LastEditDate="2012-02-03T01:38:46.457" LastActivityDate="2012-02-03T01:38:46.457" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1166" PostTypeId="2" ParentId="1144" CreationDate="2012-02-03T01:39:04.257" Score="0" Body="&lt;p&gt;If you don't really require it to be open-source, nor a program, then you can try &lt;a href=&quot;http://www.wolframalpha.com/&quot; rel=&quot;nofollow&quot;&gt;Wolfram Alpha&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="879" LastEditorUserId="276" LastEditDate="2012-02-03T16:22:36.447" LastActivityDate="2012-02-03T16:22:36.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1167" PostTypeId="1" AcceptedAnswerId="1189" CreationDate="2012-02-03T03:13:56.973" Score="4" ViewCount="304" Body="&lt;p&gt;As the title suggests, what are the differences between the frozen core approximation as implemented by Amsterdam Density Functional and effective core potentials (ECPs)?&lt;/p&gt;&#xA;" OwnerUserId="512" LastActivityDate="2019-01-23T14:37:12.030" Title="What is the difference between ECPs and ADF frozen cores?" Tags="|computational-chemistry|density-functional-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1168" PostTypeId="2" ParentId="940" CreationDate="2012-02-03T04:21:36.617" Score="3" Body="&lt;p&gt;The problem you have stated is a classic problem in Computational Geometry: Range Queries.  That is:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input: a subset S of n-dimensional euclidean space, and a set of points in that space P.&lt;br&gt;&#xA;Output:  the subset of P that intersects S.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/059651624X&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithms in a Nutshell&lt;/a&gt; describes how to solve a similar problem on page 292.  It describes an algorithm for a rectangular region S (not an n-dimensional sphere as in your case).  You can obtain a solution which is &lt;span class=&quot;math-container&quot;&gt;$O(n^{1-1/d}+r)$&lt;/span&gt;, where n is the number of points, d is the dimension of the space, and r is the number of points reported by the query.  If d is very large, the asymptotic performance is practically &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;, thus is what's normally referred to as the &quot;curse of dimensionality&quot;. That is, if the dimension is high, then performance suffers greatly!  Still, &lt;span class=&quot;math-container&quot;&gt;$O(n^{1-\frac{1}{d}}+r)$&lt;/span&gt; is still asymptotically faster than &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;, just not as much as we wish it could be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm involves a divide and conquer strategy (recursion) and a special data structure (kd-tree).  Here is an outline of the algorithm:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Begin MainProgram&#xA;&#xA;    results = a new SET   (a kd-tree)&#xA;    search(space,root,results)&#xA;    return results&#xA;endMainProgram&#xA;&#xA;Subroutine search(space,node,results)&#xA;    if (space contains node.region) then&#xA;        add node.point to the results&#xA;        for each descendant of node&#xA;            add d.point to results&#xA;        return&#xA;    if (space contains node.point) then &#xA;        add node.point to results&#xA;    if (space extends below node.coordinate) then&#xA;        search(space,node.below,results)&#xA;    if (space extends above node.coordinate) then&#xA;        search(space,node.above,results)&#xA;endsubroutine&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;-source: Algorithms in a Nutshell, pg 298&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key to the rectangular case is that we can easily define the data structure such that &#xA;we can include an entire subset all at once... hence the KD-tree.  The KD tree simply splits your n-dimensional space by cutting it successively by hyperplanes, much the same way a 3D space can be split by planes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your particular problem involving range queries in the radial direction (not rectangular boxes), you may be able to find a similar n-sphere based recursive space splitting... There is a similar data-structure called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Vantage-point_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt; vp-tree&lt;/a&gt; which is designed for implementation of space partitioning in hyper-spherical coordinates.  You may want to see &#xA;&#xA;&lt;p&gt;&quot;&gt;this publication for more details on the theory of vp-trees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, coding can be a hassle if your goal is really to use the algorithm as a tool to conduct science.  In which case, I would suggest looking into libraries that already implement these data structures that you can use.  A computational geometry library would be very useful in this situation.  The &lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/contents.html#part_XIII&quot; rel=&quot;nofollow noreferrer&quot;&gt;CGAL&lt;/a&gt; library has subroutines for d-dimensional range searches which may be of interest to you.  &lt;a href=&quot;http://www.cs.sunysb.edu/~algorith/files/range-search.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is another list &lt;/a&gt; of libraries with range query subroutines.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you are ok with obtaining most of the points in the spherical range that you are searching (but not exactly all), you may want to consider using an approximation algorithm such the one in &lt;a href=&quot;http://www.cs.umd.edu/~mount/Papers/soda05-range.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2019-05-22T18:19:55.727" LastActivityDate="2019-05-22T18:19:55.727" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1169" PostTypeId="1" AcceptedAnswerId="1193" CreationDate="2012-02-03T06:27:21.010" Score="6" ViewCount="273" Body="&lt;p&gt;My matrix sizes have grown beyond what can fit on the RAM but I have a function which defines each element cheaply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible use BLAS (in Fortran or even in MATLAB) in such cases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I had a matrix in memory, I could use &lt;code&gt;dgemv&lt;/code&gt; or &lt;code&gt;dgemm&lt;/code&gt; or whatever and pass my matrix as one of the arguments. Now, I have no matrix. How do I tackle that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could write my own codes, but no matter what I do, I doubt my programming will be as thorough as the guys up at Intel MKL. There is simply too much to keep track of in such kind of programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: My matrix is dense. Almost full dense. I have a function in terms of indices $(i,j)$ and a $10^{6} \times 1$ vector. Like : $$f(i,j) = \frac{|a(i) + a(j)|}{(i-j)^2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I'm not saying I &lt;strong&gt;must&lt;/strong&gt; use BLAS. I can use anything similar as long as it gets my job done. But as far as I know, there isn't any other library.&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="529" LastEditDate="2012-02-04T15:16:50.763" LastActivityDate="2017-09-11T07:35:21.590" Title="Is it possible to use BLAS if I have a function rather than a matrix?" Tags="|linear-algebra|blas|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1170" PostTypeId="2" ParentId="938" CreationDate="2012-02-03T06:43:54.673" Score="1" Body="&lt;p&gt;Within earthquake modeling there has been several of these sorts of efforts. The ones that come to mind are the &lt;a href=&quot;http://scecdata.usc.edu/cvws/&quot; rel=&quot;nofollow&quot;&gt;dynamic rupture simulation project&lt;/a&gt; and the source inversion project (http://eqsource.webfactional.com/wiki/). Both of these provides physical problems to solve and are agnostic (at some level) concerning the methods employed. Really these projects are about getting the domain specific modelers to compare with one another, not really a general set of test problems for every numerical pdes method. But you can compare with others results, and this has revealed a lot of problems with many codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possibility to look at would be the &lt;a href=&quot;http://www.reproducibility.org/wiki/Main_Page&quot; rel=&quot;nofollow&quot;&gt;madagascar project&lt;/a&gt;, which has been developed within the seismic imaging world but claims to be quite general purpose.&lt;/p&gt;&#xA;" OwnerUserId="880" LastActivityDate="2012-02-03T06:43:54.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1171" PostTypeId="1" CreationDate="2012-02-03T15:43:06.977" Score="4" ViewCount="1213" Body="&lt;p&gt;Diffusion kernels are &lt;a href=&quot;http://en.wikipedia.org/wiki/Kernel_trick&quot; rel=&quot;nofollow&quot;&gt;kernels&lt;/a&gt; which &quot;project&quot; information about graphs into $R^n$ so that certain machine learning techniques can be performed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read through &lt;a href=&quot;http://www.its.caltech.edu/~risi/papers/diffusion-kernels.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; and feel fairly comfortable with the theory behind these kernels, but I'm looking for guidance about how they can best be applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example problem I'm facing is that it involves calculating $e^{tA}$ for a large matrix $A$, which is not a trivial task. I know that sometimes finding $e^{tA}\vec{x}$ is a much simpler problem - is this one of those cases? And is the resulting distance I get the best to use, or are there further &quot;fine tuning&quot; methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it's relevant: my problem is essentially &quot;nearest neighbor&quot;. Each example is a &quot;bag&quot; of attributes from a set $S$, where $S$ is structured as an ontology (graph). So I want to find some distance metric that I can apply to my examples, based on the information I get from the graph.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: Suppose I have two points in $R^n$: $x$ and $y$. I can find the distance between these points by numerous methods; most obviously by using the Euclidean $||x-y||$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose my points are not in $R^n$, but rather in an arbitrary graph. How do I find the distance between them? (What does it even mean to say the &quot;distance&quot; between them?) This is the problem that diffusion kernels solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand at a high level what this solution is, but actually calculating it is harder. So I'm asking for help in figuring out how to calculate this. I think seeing someone step through an example would be ideal, but any guidance about calculation would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT 2: We have an ontology like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         Computer Science&#xA;         /               \&#xA;        AI             Theory of CS&#xA;      /   \               /        \&#xA;    NLP    Vision     Decidability   Computational Complexity&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How similar are &quot;decidability&quot; and &quot;computational complexity&quot;? What a diffusion kernel does is it provides a kernel to project this data into a more easily measurable space. &lt;/p&gt;&#xA;" OwnerUserId="847" LastEditorUserId="847" LastEditDate="2012-02-06T15:45:38.463" LastActivityDate="2012-02-06T15:45:38.463" Title="Diffusion kernel &quot;guide&quot;" Tags="|linear-algebra|machine-learning|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1172" PostTypeId="1" CreationDate="2012-02-03T17:15:20.457" Score="5" ViewCount="495" Body="&lt;p&gt;We have Maui and Torque on our lab's UNIX cluster. Right now, all jobs are served by FIFO. We'd like to implement a more fair policy, but I have not successfully implemented it. The online documentation was not quite clear, and nothing that I've tried to implement has had any effect (I've altered the config file, then restarted Maui).  Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="889" LastEditorUserId="103" LastEditDate="2012-02-03T20:20:08.647" LastActivityDate="2012-02-05T18:53:17.193" Title="Implementing a fair scheduling policy on Maui/Torque" Tags="|pbs|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1173" PostTypeId="1" AcceptedAnswerId="1174" CreationDate="2012-02-03T18:43:29.780" Score="14" ViewCount="2012" Body="&lt;p&gt;Wherever I have seen, PETSc tutorial/documents etc. say that it is useful for linear algebra and usually specifies that sparse systems will benefit. What about dense matrices? I am concerned about solving $Ax=b$ for dense $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written my own code for CG and QMR in Fortran. The basic construct is to rip apart the pseudo code and add BLAS routines wherever possible (&lt;code&gt;ddot&lt;/code&gt;, &lt;code&gt;dnrm&lt;/code&gt; and &lt;code&gt;dgemv&lt;/code&gt;) with a little self tuning. How will this compare to PETSc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the best answer would be for me to try it myself but because of reasons of time and others, that is not possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="276" LastEditDate="2012-02-03T20:36:56.080" LastActivityDate="2012-02-03T20:58:02.547" Title="How useful is PETSc for Dense Matrices?" Tags="|linear-algebra|petsc|blas|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1174" PostTypeId="2" ParentId="1173" CreationDate="2012-02-03T19:03:52.623" Score="7" Body="&lt;p&gt;If you have dense matrices with structure (e.g. fast transforms, Schur complements, etc), PETSc could be useful. In these cases, you won't be assembling the full matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For assembled dense systems, PETSc currently uses PLAPACK, but the matrix distribution in PETSc native format is not the best to minimize communication (for most operations). Jack Poulson, Matt Knepley, and I have discussed porting PETSc's dense linear algebra to use &lt;a href=&quot;http://code.google.com/p/elemental/&quot;&gt;Elemental&lt;/a&gt;, Jack's more modern dense linear algebra library. It hasn't happened yet, but we will do it when we have time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If want a full-featured dense linear algebra library, use Elemental. It is likely to always have a more complete API for those operations than PETSc. If you need to interact with sparse or unassembled systems, it likely makes sense to stick with PETSc and extract what you need to use Elemental for the dense parts (or help us write the PETSc interface to Elemental).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-03T19:03:52.623" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1175" PostTypeId="2" ParentId="1173" CreationDate="2012-02-03T20:58:02.547" Score="5" Body="&lt;p&gt;It's important to realize that parallel dense linear algebra libraries usually focus on level 3 BLAS routines (routines that perform $O(n^3)$ work with $O(n^2)$ data) and higher-level functionality like factorizations and eigensolvers. They usually don't tune the level 1 and level 2 operations that you're referring to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you mentioned that you are on a shared-memory system, I would have recommended &lt;a href=&quot;http://z.cs.utexas.edu/wiki/flame.wiki/libflame/&quot;&gt;libFLAME&lt;/a&gt; and/or &lt;a href=&quot;http://icl.cs.utk.edu/plasma/&quot;&gt;PLASMA&lt;/a&gt;, but I do not think that either will be significantly faster than vendor threaded BLAS for your level 1 and level 2 operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jed recommended &lt;a href=&quot;http://code.google.com/p/elemental&quot;&gt;Elemental&lt;/a&gt;, which I happen to develop, but I will again stress that level 1 and level 2 operations are not the main focus of parallel dense linear algebra libraries. I have honestly never benchmarked any of those routines.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-02-03T20:58:02.547" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1176" PostTypeId="1" CreationDate="2012-02-04T01:56:28.790" Score="8" ViewCount="282" Body="&lt;p&gt;We have an adaptive mesh refinement (AMR) code for solving the elastic wave equation with frictional fault interfaces (based on &lt;a href=&quot;https://commons.lbl.gov/display/chombo/Chombo+-+Software+for+Adaptive+Solutions+of+Partial+Differential+Equations&quot; rel=&quot;nofollow&quot;&gt;Chombo&lt;/a&gt; for those that are interested). One of the things that we have realized is that our results are being strongly affected by the presence of the outer absorbing boundary (which we implement as a simple characteristic boundary condition). For reference we currently use the multidimensional Godunov (Finite Volume) scheme of &lt;a href=&quot;http://dx.doi.org/10.1016/0021-9991%2890%2990233-Q&quot; rel=&quot;nofollow&quot;&gt;Colella&lt;/a&gt; and collaborators. Though we are not wed to these methods (just easy to use since they are already in Chombo) we do need adaptivity in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am wondering is if anyone one has any experience with more efficient absorbing boundary conditions with AMR using adaptive time stepping, such as perfectly matched layers or high-order boundary conditions. Any reason not to go down this road? My limited searching hasn't really turned up any useful references or mentions of this in the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: clarified that this is a finite volume method.&lt;/p&gt;&#xA;" OwnerUserId="880" LastEditorUserId="880" LastEditDate="2012-02-04T17:00:22.280" LastActivityDate="2012-02-04T17:00:22.280" Title="Adaptive mesh refinement with perfectly matched layers?" Tags="|hyperbolic-pde|adaptive-mesh-refinement|boundary-conditions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1177" PostTypeId="2" ParentId="304" CreationDate="2012-02-04T04:36:34.723" Score="8" Body="&lt;p&gt;Since I am new here, I was looking through old questions and found this one. Hopefully it's not taboo to answer old ones!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since no one else has mentioned this, figured I would. Fortran 2003 is almost fully &lt;a href=&quot;http://fortranwiki.org/fortran/show/Fortran+2003+status&quot;&gt;supported&lt;/a&gt; by most of the major compilers (intel, ibm, cray, NAG, PCG) even gcc with the (soon-to-be) newest release 4.7. Fortran 2003 (and 2008) is an object oriented language, albeit a bit more verbose than C++. One of things that I think is nice about Fortran is the fact that the standard committee see's scientific computing as it's primary audience (I thank Damian Rouson for pointing this out to me the other day).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I bring this all up not so that C++ programmers become Fortran programmers, but so that Fortran people know that they have more options now besides switching to C++ or emulating object oriented concepts in Fortran 90/95.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One caveat I will add is that there is a cost to being on the bleeding edge of what's implemented in the compilers. If you undertake a major project in Fortran 2003 right now you will stumble across bugs and continually need to update your compiler (especially if you use gcc), though this has gotten significantly better in the past few months!&lt;/p&gt;&#xA;" OwnerUserId="880" LastActivityDate="2012-02-04T04:36:34.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1178" PostTypeId="1" CreationDate="2012-02-04T06:15:31.200" Score="7" ViewCount="154" Body="&lt;p&gt;In the problem of &lt;a href=&quot;http://en.wikipedia.org/wiki/Protein%E2%80%93ligand_docking&quot; rel=&quot;nofollow&quot;&gt;protein-ligand docking&lt;/a&gt;, most of the time people are happy if they can just predict the final conformation the ligand adopts into the protein's binding pocket. Most of the time one can just use a physical based scoring function and an energy minimization method to try to find the minima. But in some other cases, scoring functions are very inaccurate to represent some more complex interactions; metal interactions, &lt;a href=&quot;http://en.wikipedia.org/wiki/Cation%E2%80%93pi_interaction&quot; rel=&quot;nofollow&quot;&gt;cation-pi&lt;/a&gt;, etc., and even &lt;a href=&quot;http://en.wikipedia.org/wiki/Covalent_bond&quot; rel=&quot;nofollow&quot;&gt;covalent bonding&lt;/a&gt;. So it seems that only &lt;a href=&quot;http://en.wikipedia.org/wiki/Ab_initio_quantum_chemistry_methods&quot; rel=&quot;nofollow&quot;&gt;ab-initio methods&lt;/a&gt; can shed light here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can quantum mechanical methods be used in docking for this purpose or not and which are the best methods, even if you require access to &lt;a href=&quot;http://en.wikipedia.org/wiki/High-performance_computing&quot; rel=&quot;nofollow&quot;&gt;HPC&lt;/a&gt; resources? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am mostly interested in covalent bond formation. It would be great also if you could provide some references, I did not find many in the literature.&lt;/p&gt;&#xA;" OwnerUserId="891" LastEditorUserId="625" LastEditDate="2012-02-05T07:34:12.587" LastActivityDate="2012-02-05T07:34:12.587" Title="Can quantum methods be applied to the protein-ligand docking problem?" Tags="|parallel-computing|computational-chemistry|computational-biology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1179" PostTypeId="1" AcceptedAnswerId="1184" CreationDate="2012-02-04T06:26:06.927" Score="8" ViewCount="2512" Body="&lt;p&gt;I would like to test the influence of solvation parameters in implicit solvation models and wonder which codes are freely available as standalone programs for protein folding of small proteins, and which use energy minimization approaches instead of dynamics, so I am not looking for molecular dynamics codes. &lt;/p&gt;&#xA;" OwnerUserId="891" LastEditorUserId="891" LastEditDate="2012-02-04T13:26:48.850" LastActivityDate="2012-05-22T21:34:29.013" Title="Are open-source codes available to study protein folding?" Tags="|computational-chemistry|computational-biology|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1181" PostTypeId="1" AcceptedAnswerId="1182" CreationDate="2012-02-04T07:19:58.280" Score="2" ViewCount="206" Body="&lt;p&gt;My comrades repeatedly encourages monotonous problems where the issue is the same: chain-rule and some basic arithmetic. Is there some computational way to derive power series approximations? Suppose I plug in some function such as $e^{x^{2}+x+ln(x)+x^{777}}$, $e^{x^2}$ or $x^{777}-ln(x)+e^{2x+1}$, some command such as &lt;code&gt;Series(e^{x^{2}})&lt;/code&gt; to get you the series? It would be useful to see graphically how things change, which would help me to learn things graphically.&lt;/p&gt;&#xA;" OwnerDisplayName="user664" LastEditorUserId="276" LastEditDate="2012-02-04T07:37:42.610" LastActivityDate="2012-02-04T19:59:01.560" Title="Power series approximation for any function such as $-e^{x^{2}}$ in some easily-accessed open-source software?" Tags="|software|education|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1182" PostTypeId="2" ParentId="1181" CreationDate="2012-02-04T07:35:22.457" Score="6" Body="&lt;p&gt;You have a few options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Sage: Documentation for power series is &lt;a href=&quot;http://www.sagemath.org/doc/constructions/calculus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. (Scroll down until you reach &quot;Power series.&quot;) Documentation for plotting functions is &lt;a href=&quot;http://www.sagemath.org/doc/reference/sage/plot/plot3d/plot3d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sage: var('x')&#xA;x&#xA;sage: f = exp(x^2 + x + log(x) + x^777)&#xA;sage: f.taylor(x, 0, 3)                &#xA;3/2*x^3 + x^2 + x&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://docs.sympy.org/dev/modules/mpmath/calculus/approximation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt; (Right on that page, first heading) + &lt;a href=&quot;http://matplotlib.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matplotlib&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ma.utexas.edu/maxima/maxima_29.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maxima&lt;/a&gt; (Search for &quot;Taylor series&quot; on that page.) + &lt;a href=&quot;http://www.gnuplot.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gnuplot&lt;/a&gt; (or Matplotlib, or another plotting package you prefer)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For any of the software packages listed in your earlier question on &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1088/is-there-any-open-source-or-easy-to-access-software-that-can-simplify-algebraic&quot;&gt;open-source math software packages&lt;/a&gt;, you can probably search around and find the appropriate command.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-02-04T19:59:01.560" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1183" PostTypeId="2" ParentId="1178" CreationDate="2012-02-04T08:36:46.830" Score="4" Body="&lt;p&gt;It's been known for quite some time that quantum methods have a useful role to play in studying protein-ligand docking. However, the challenge that you have—which has not really changed too much in recent years—is that ab initio methods do not cope well with very large systems. It's probably possible to study the interaction of a ligand with the relevant binding sites of the protein, but when you're stuck at modeling a few hundred to a few thousand atoms on very large machines, it's difficult to study the entire interacting system (which would include not only the protein and the ligand, but quite possibly also the aqueous solvent as well!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are some references available: &lt;a href=&quot;http://dx.doi.org/10.1021/ct700134r&quot; rel=&quot;nofollow&quot;&gt;Gresh et al.&lt;/a&gt; have proposed a &quot;bottom-up&quot; molecular mechanics strategy that incorporates ab initio data, while &lt;a href=&quot;http://dx.doi.org/10.1021/bi992306s&quot; rel=&quot;nofollow&quot;&gt;Morelli et al.&lt;/a&gt; combine ab initio docking calculations with experimental analysis to move towards studying protein-protein complexes.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-02-04T08:36:46.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1184" PostTypeId="2" ParentId="1179" CreationDate="2012-02-04T11:05:20.050" Score="8" Body="&lt;p&gt;The most popular molecular dynamics codes are &lt;a href=&quot;http://www.ks.uiuc.edu/Research/namd/&quot;&gt;namd&lt;/a&gt; and &lt;a href=&quot;http://www.gromacs.org/&quot;&gt;gromacs&lt;/a&gt;, and maybe also &lt;a href=&quot;http://www.deshawresearch.com/resources_desmond.html&quot;&gt;desmond&lt;/a&gt;. These packages are freely available and &quot;open source&quot; in the sense that the source code is made available for free. Wikipedia hosts a &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_software_for_molecular_mechanics_modeling&quot;&gt;list of software for molecular modeling&lt;/a&gt; which may contain other good links.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These codes, however, are also quite complex and therefore if you want to make changes of your own, e.g. to test new solvation models, you might spend quite a bit of time just trying to understand them. If ease of use trumps speed (probably not completely if you're looking at time-scales for folding), you might want to look at &lt;a href=&quot;http://dirac.cnrs-orleans.fr/MMTK/&quot;&gt;mmtk&lt;/a&gt;, which lets your program custom molecular dynamics simulations in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in speed, but not all the bells and whistles of a full-fledged simulation package, you may also be interested in molecular dynamics &lt;em&gt;libraries&lt;/em&gt; such as &lt;a href=&quot;http://simtk.org/home/openmm_suite&quot;&gt;OpenMM&lt;/a&gt; or &lt;a href=&quot;http://mdcore.sourceforge.net/&quot;&gt;mdcore&lt;/a&gt; (disclaimer: mdcore is my own software project). These will, however, require quite a bit more programming on your own side.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-02-06T11:57:29.263" LastActivityDate="2012-02-06T11:57:29.263" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1185" PostTypeId="2" ParentId="1178" CreationDate="2012-02-04T13:09:37.887" Score="2" Body="&lt;p&gt;There exist hybrid methods such as &lt;a href=&quot;http://www.gaussian.com/g_whitepap/oniom_technote.htm&quot; rel=&quot;nofollow&quot;&gt;ONIOM&lt;/a&gt; and &lt;a href=&quot;http://iqc.udg.es/~marcel/eng/index.html&quot; rel=&quot;nofollow&quot;&gt;QUILD&lt;/a&gt; (a rough cognate of ONIOM) that address the interface between QM and MM calculations and are well-suited to investigation of protein cofactors and probably refining ligand docking models. The ONIOM methodology is very simple but clever - you break the calculation into the bulk protein which is handled with MM, and the region of interest, which is dealt with using an electronic structure method. Forces on capping atoms shared by both regions are reconciled between the QM and MM calculations, and relative energies are obtained algebraically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach would not be suited to high-throughput docking studies and would instead be more useful for checking the viability of a few good leads. I know a guy who has built his PhD around using QUILD to (AFAIK) assess models of water binding modes to a protein active site. The space of possible solutions is reduced because the water has to datively bind to cofactor metals.&lt;/p&gt;&#xA;" OwnerUserId="512" LastActivityDate="2012-02-04T13:09:37.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1187" PostTypeId="2" ParentId="304" CreationDate="2012-02-04T13:40:28.300" Score="36" Body="&lt;p&gt;I'm also throwing my two cents in kind of late, but I've only just seen this thread and I feel that, for posterity, there are a few points that desperately need to be made.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note in the following that I will talk about C and not C++. Why? Well, otherwise it's apples and oranges to compare a full-fledged dynamically typed object-oriented language with something as static as Fortran. Yes, some modern implementations of the latest Fortran standards can do more than just that, but very few people actually use them, and so when we speak of Fortran, we think simple, static, and imperative language. That's where C is too, so I'll replace C with C++ for the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, any discussion of Fortran/C having better compilers is moot. Dedicated C/Fortran compilers are a thing of the past. Both gcc/gfortran and icc/ifc are just different front-ends to the same back-end, i.e. your program will be transformed into an abstract description by the front-end and then optimized and assembled by the back-end. If you write, semantically, the same code in Fortran or in C, the compiler will, in both cases, produce the same assembly which will run just as fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This now leads to my second point: why do we still see differences? The problem is that most comparisons are made by Fortran programmers trying something in C or vice-versa. Ever notice how most authors or poets prefer to write in their native languages? Would you want to write poetry in a language in which you don't feel completely confident or at home? Of course not... I myself consider C to be my &quot;native&quot; programming language. I did, however, also spend three years working in a group that used only Fortran, in which I have achieved a certain level of fluency. I would, however, never write anything on my own in Fortran since I'm more comfortable with C and, as a consequence, the resulting code will be &lt;em&gt;better&lt;/em&gt;, whatever you define that as.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the main difference is in the programmer, not the language. So there are no differences? Well, not quite. Here are a few examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;SIMD: Whether it's SSE, SSE3 or AltiVec, if you want to use them in Fortran, you better hope and pray that the compiler guesses &lt;em&gt;exactly&lt;/em&gt; what you want and does it so. Good luck. In C you generally have intrinsic functions for each architecture, or, more recently, general &lt;a href=&quot;http://gcc.gnu.org/onlinedocs/gcc-4.6.0/gcc/Vector-Extensions.html&quot;&gt;SIMD vector types in gcc&lt;/a&gt;. Most Fortran compilers will only use SIMD instructions to unroll loops, but if you have a kernel which works on short vectors of data in a non-obvious way, the compiler will very probably not see it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Different hardware architectures: The whole CUDA architecture is built around kernels in C. Yes, the Portland Group now has a &lt;a href=&quot;http://www.pgroup.com/resources/cudafortran.htm&quot;&gt;CUDA-capable fortran compiler&lt;/a&gt; too, but it's commercial, and most importantly, it's not from NVIDIA. Same goes for OpenCL, for which the best I could find is a &lt;a href=&quot;http://code.google.com/p/fortrancl/&quot;&gt;recent project&lt;/a&gt; which only supports a few basic calls.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Parallel programming: Yes, both MPI and OpenMP work just fine with both C and Fortran. However, if you want real control of your threads, i.e. if you have a fully dynamic shared-memory computation, you'll be out in the cold with Fortran. In C you have the standard pthreads which, while not warm and fuzzy, will still get you through the storm. In general, most computations that rely on access to the operating system, e.g. threads, processes, file system, etc... are better served with C. Oh, and don't try to do your own networking with Fortran.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ease of use: Fortran is closer to Matlab than C is. Once you've gotten over all the different keywords and how to declare variables, the rest of the code looks like Matlab, making it more accessible to users with limited programming experience.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Interoperability: When you create a struct in C, the layout of the actual data is straight-forward and deterministic. In Fortran, if you use pointer arrays or structured data, the actual layout of the data is strongly compiler-dependent, not straight-forward, and usually completely undocumented. You can call C from Fortran and vice-versa, but don't start thinking it may be as easy to pass anything more than a static array from one to the other and back.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is all somewhat geeky, low-level stuff, but this is High-Performance Computing we're talking about, right? If you're not interested in how to best exploit the underlying hardware paradigms, i.e. implementing and/or developing algorithms which are best for shared/distributed memory, threads, SIMD vectorisation, GPUs using SIMT, and so-on, then you're just doing math on a computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has gotten much longer that anything I entended, so here's a summary -- a set of take home messages of sorts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You will write the best code &lt;em&gt;you&lt;/em&gt; can in the language which &lt;em&gt;you&lt;/em&gt; know best.&lt;/li&gt;&#xA;&lt;li&gt;There is no difference in the quality of code produced by two compilers which use the same back-end -- it is &lt;em&gt;us&lt;/em&gt; who write bad code in one language or another.&lt;/li&gt;&#xA;&lt;li&gt;Despite feeling more low-level, Fortran is quite a high-level abstraction and won't let you access certain hardware/OS features directly, e.g. SIMD, threads, networking, etc...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-02-04T13:51:43.773" LastActivityDate="2012-02-04T13:51:43.773" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1188" PostTypeId="2" ParentId="1040" CreationDate="2012-02-04T14:35:43.763" Score="0" Body="&lt;p&gt;G. H. Golub and C. F. Van Loan, Matrix Computations, third edition, The Johns Hopkins University Press,Baltimore, 1996.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;N.J.Higham, Accuracy and Stability of Numerical Algorithms, SIAM, 1996.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Y.Saad, Iterative Methods for Sparse Linear Systems, SIAM, 2000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L.N.Trefethen and D.Bau,III, Numerical Linear Algebra, SIAM, 1997.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;H. A. Van der Vorst, Iterative Krylov Methods for Large Linear Systems, Cambridge University Press, 2003.&lt;/p&gt;&#xA;" OwnerUserId="897" LastActivityDate="2012-02-04T14:35:43.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1189" PostTypeId="2" ParentId="1167" CreationDate="2012-02-04T15:35:28.017" Score="5" Body="&lt;p&gt;Frozen core approximations use explicit orbitals for each core electron, which leads to explicit Coulombic potential terms:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \langle v|V_{val,core}|w\rangle = \sum_{i\in core} \int \frac{v^\ast(r)w(r)|\phi_i(r_1)|^2}{\|r-r_1\|}-\frac{v^\ast(r)\phi_i(r)\phi_i^\ast(r_1)w(r_1)}{\|r-r_1\|}d^3r_1d^3r$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Effective core potentials model the potential directly instead. See &lt;a href=&quot;http://www.ccc.uga.edu/lec_top/rltvt/node22.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for a discussion of effective core potentials.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2019-01-23T14:37:12.030" LastActivityDate="2019-01-23T14:37:12.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1190" PostTypeId="2" ParentId="1176" CreationDate="2012-02-04T15:46:34.227" Score="1" Body="&lt;p&gt;Is this using finite elements? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dont know much about PMLs but as long as the implementation is local to the element then it should not be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Afaik PML implementation in the frequency domain is local i.e., elements end up having a modified mass matrix, material coefficient matrix and the strain-displacement matrix. Not sure about the time domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can always use viscous dampers as the implementation is very simple only requiring changes to the element damping matrix.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2012-02-04T15:46:34.227" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1191" PostTypeId="1" CreationDate="2012-02-04T16:30:39.190" Score="7" ViewCount="184" Body="&lt;p&gt;I'm trying to implement a Conjugate Gradient solver using Hierarchical Basis Functions, following &lt;a href=&quot;http://bit.ly/yscHyN&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In section 3 the paper says that the hierarchical basis matrix $S$ can be decomposed into a &quot;series of very sparse matrices $S = S_1S_2...S_{L-1}$ &quot; but it doesn't tell the exact structure of these matrices $S_i$, nor in which way they are sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore: What is the structure of $S_i$ (I assume they are all similar in a way)? Why is $S_i$ sparse? What is the interpretation of applying $S_i$ to a vector?&lt;/p&gt;&#xA;" OwnerUserId="740" LastEditorUserId="276" LastEditDate="2012-02-05T19:55:47.470" LastActivityDate="2012-02-05T21:56:19.150" Title="Conjugate Gradient with Hierarchical Basis Functions: How can the hierarchical base be decomposed?" Tags="|linear-algebra|pde|finite-element|basis-set|computer-vision|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1192" PostTypeId="2" ParentId="1169" CreationDate="2012-02-04T17:52:20.290" Score="4" Body="&lt;p&gt;In a word: No. The reason MKL and other optimized libraries are so good is they are optimized to balance calculation versus memory use (i.e. your matrix is fully stored in RAM explicitly). If your matrix is indeed so large my only real suggestion would be to block it out yourself so that each individual block fits, and use BLAS on the blocks sequentially (accumulating results as necessary in a block matrix-multiply).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course you could write your own routines like you mention. It might be faster programming wise.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2012-02-04T17:52:20.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1193" PostTypeId="2" ParentId="1169" CreationDate="2012-02-04T19:37:06.250" Score="4" Body="&lt;p&gt;What do you want to do with this matrix? If you just want to multiply with a vector or another matrix, you should write that code yourself. If you want to factor the matrix, you will have to find the memory to store it. If you want to solve systems or eigenvalue problems, you could use PETSc with &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateShell.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;MatShell&lt;/code&gt;&lt;/a&gt;. Depending on the problem, you might need a preconditioner (which you might also be able to implement without assembling, if you like).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-04T19:37:06.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1194" PostTypeId="1" CreationDate="2012-02-04T21:13:52.490" Score="79" ViewCount="20913" Body="&lt;p&gt;This question is an extension of two discussions that came up recently in the replies to &quot;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/304/c-vs-fortran-for-hpc&quot;&gt;C++ vs Fortran for HPC&lt;/a&gt;&quot;. And it is a bit more of a challenge than a question...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the most often-heard arguments in favor of Fortran is that the compilers are just better. Since most C/Fortran compilers share the same back end, the code generated for semantically equivalent programs in both languages should be identical. One could argue, however, that C/Fortran is more/less easier for the compiler to optimize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I decided to try a simple test: I got a copy of &lt;a href=&quot;http://www.netlib.org/blas/daxpy.f&quot; rel=&quot;noreferrer&quot;&gt;daxpy.f&lt;/a&gt; and &lt;a href=&quot;http://www.netlib.org/clapack/cblas/daxpy.c&quot; rel=&quot;noreferrer&quot;&gt;daxpy.c&lt;/a&gt; and compiled them with gfortran/gcc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now daxpy.c is just an f2c translation of daxpy.f (automatically generated code, ugly as heck), so I took that code and cleaned it up a bit (meet daxpy_c), which basically meant re-writing the innermost loop as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for ( i = 0 ; i &amp;lt; n ; i++ )&#xA;    dy[i] += da * dx[i];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, I re-wrote it (enter daxpy_cvec) using gcc's vector syntax:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#define vector(elcount, type)  __attribute__((vector_size((elcount)*sizeof(type)))) type&#xA;vector(2,double) va = { da , da }, *vx, *vy;&#xA;&#xA;vx = (void *)dx; vy = (void *)dy;&#xA;for ( i = 0 ; i &amp;lt; (n/2 &amp;amp; ~1) ; i += 2 ) {&#xA;    vy[i] += va * vx[i];&#xA;    vy[i+1] += va * vx[i+1];&#xA;    }&#xA;for ( i = n &amp;amp; ~3 ; i &amp;lt; n ; i++ )&#xA;    dy[i] += da * dx[i];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that I use vectors of length 2 (that's all SSE2 allows) and that I process two vectors at a time. This is because on many architectures, we may have more multiplication units than we have vector elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All codes were compiled using gfortran/gcc version 4.5 with the flags &quot;-O3 -Wall -msse2 -march=native -ffast-math -fomit-frame-pointer -malign-double -fstrict-aliasing&quot;. On my laptop (Intel Core i5 CPU, M560, 2.67GHz) I got the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pedro@laika:~/work/fvsc$ ./test 1000000 10000&#xA;timing 1000000 runs with a vector of length 10000.&#xA;daxpy_f took 8156.7 ms.&#xA;daxpy_f2c took 10568.1 ms.&#xA;daxpy_c took 7912.8 ms.&#xA;daxpy_cvec took 5670.8 ms.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the original Fortran code takes a bit more than 8.1 seconds, the automatic translation thereof takes 10.5 seconds, the naive C implementation does it in 7.9 and the explicitly vectorized code does it in 5.6, marginally less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's Fortran being slightly slower than the naive C implementation and 50% slower than the vectorized C implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here's the question: I'm a native C programmer and so I'm quite confident that I did a good job on that code, but the Fortran code was last touched in 1993 and might therefore be a bit out of date. Since I don't feel as comfortable coding in Fortran as others here may, can anyone do a better job, i.e. more competitive compared to any of the two C versions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, can anybody try this test with icc/ifort? The vector syntax probably won't work, but I would be curious to see how the naive C version behaves there. Same goes for anybody with xlc/xlf lying around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've uploaded the sources and a Makefile &lt;a href=&quot;http://people.maths.ox.ac.uk/gonnet/fvsc/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. To get accurate timings, set CPU_TPS in test.c to the number of Hz on your CPU. If you find any improvements to any of the versions, please do post them here!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've added stali's test code to the files online and supplemented it with a C version. I modified the programs to do 1'000'000 loops on vectors of length 10'000 to be consistent with the previous test (and because my machine couldn't allocate vectors of length 1'000'000'000, as in stali's original code). Since the numbers are now a bit smaller, I used the option &lt;code&gt;-par-threshold:50&lt;/code&gt; to make the compiler more likely to parallelize. The icc/ifort version used is 12.1.2 20111128 and the results are as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=1 time ./icctest_c&#xA;3.27user 0.00system 0:03.27elapsed 99%CPU&#xA;&#xA;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=1 time ./icctest_f&#xA;3.29user 0.00system 0:03.29elapsed 99%CPU&#xA;&#xA;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=2 time ./icctest_c&#xA;4.89user 0.00system 0:02.60elapsed 188%CPU&#xA;&#xA;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=2 time ./icctest_f&#xA;4.91user 0.00system 0:02.60elapsed 188%CPU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In summary, the results are, for all practical purposes, identical for both the C and Fortran versions, and both codes parallelize automagically. Note that the fast times compared to the previous test are due to the use of single-precision floating point arithmetic!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I don't really like where the burden of proof is going here, I've re-coded stali's &lt;a href=&quot;http://bpaste.net/show/23035/&quot; rel=&quot;noreferrer&quot;&gt;matrix multiplication example&lt;/a&gt; in C and added it to the files on the &lt;a href=&quot;http://people.maths.ox.ac.uk/gonnet/fvsc/&quot; rel=&quot;noreferrer&quot;&gt;web&lt;/a&gt;. Here are the results of the tripple loop for one and two CPUs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=1 time ./mm_test_f 2500&#xA; triple do time   3.46421700000000     &#xA;3.63user 0.06system 0:03.70elapsed 99%CPU&#xA;&#xA;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=1 time ./mm_test_c 2500&#xA;triple do time 3.431997791385768&#xA;3.58user 0.10system 0:03.69elapsed 99%CPU&#xA;&#xA;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=2 time ./mm_test_f 2500&#xA; triple do time   5.09631900000000     &#xA;5.26user 0.06system 0:02.81elapsed 189%CPU&#xA;&#xA;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=2 time ./mm_test_c 2500&#xA;triple do time 2.298916975280899&#xA;4.78user 0.08system 0:02.62elapsed 184%CPU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;cpu_time&lt;/code&gt; in Fortran measuers the CPU time and not the wall-clock time, so I wrapped the calls in &lt;code&gt;time&lt;/code&gt; to compare them for 2 CPUs. There is no real difference between the results, except that the C version does a bit better on two cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for the &lt;code&gt;matmul&lt;/code&gt; command, of course only in Fortran as this intrinsic is not available in C:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=1 time ./mm_test_f 2500&#xA; matmul    time   23.6494780000000     &#xA;23.80user 0.08system 0:23.91elapsed 99%CPU&#xA;&#xA;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=2 time ./mm_test_f 2500&#xA; matmul    time   26.6176640000000     &#xA;26.75user 0.10system 0:13.62elapsed 197%CPU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Wow. That's absolutely terrible. Can anyone either find out what I'm doing wrong, or explain why this intrinsic is still somehow a good thing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't add the &lt;code&gt;dgemm&lt;/code&gt; calls to the benchmark as they are library calls to the same function in the Intel MKL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For future tests, can anyone suggest an example &lt;em&gt;known&lt;/em&gt; to be slower in C than in Fortran?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To verify stali's claim that the &lt;code&gt;matmul&lt;/code&gt; intrinsic is &quot;an order of magnitue&quot; faster than the explicit matrix product on smaller matrices, I modified his own code to multiply matrices of size 100x100 using both methods, 10'000 times each. The results, on one and two CPUs, are as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=1 time ./mm_test_f 10000 100&#xA; matmul    time   3.61222500000000     &#xA; triple do time   3.54022200000000     &#xA;7.15user 0.00system 0:07.16elapsed 99%CPU&#xA;&#xA;pedro@laika:~/work/fvsc$ OMP_NUM_THREADS=2 time ./mm_test_f 10000 100&#xA; matmul    time   4.54428400000000     &#xA; triple do time   4.31626900000000     &#xA;8.86user 0.00system 0:04.60elapsed 192%CPU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grisu is correct in pointing out that, without optimizations, gcc converts operations on complex numbers to library function calls while gfortran inlines them in a few instructions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The C compiler will generate the same, compact code if the option &lt;code&gt;-fcx-limited-range&lt;/code&gt; is set, i.e. the compiler is instructed to ignore potential over/under-flows in the intermediate values. This option is somehow set by default in gfortran and may lead to incorrect results. Forcing &lt;code&gt;-fno-cx-limited-range&lt;/code&gt; in gfortran didn't change anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this is actually an argument &lt;em&gt;against&lt;/em&gt; using gfortran for numerical calculations: Operations on complex values may over/under-flow even if the correct results are within the floating-point range. This is actually a Fortran standard. In gcc, or in C99 in general, the default is to do things strictly (read IEEE-754 compliant) unless otherwise specified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reminder:&lt;/strong&gt; Please keep in mind that the main question was whether Fortran compilers produce better code than C compilers. This is not the place for discussions as to the general merits of one language over another. What I would be really interested in is if anybody can find a way of coaxing gfortran to produce a daxpy as efficient as the one in C using explicit vectorization as this exemplifies the problems of having to rely on the compiler exclusively for SIMD optimization, or a case in which a Fortran compiler out-does its C counterpart.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2020-12-29T11:00:48.323" Title="How much better are Fortran compilers really?" Tags="|fortran|c|blas|benchmarking|" AnswerCount="5" CommentCount="24" ContentLicense="CC BY-SA 3.0" />
  <row Id="1195" PostTypeId="2" ParentId="1194" CreationDate="2012-02-04T22:46:23.170" Score="9" Body="&lt;p&gt;The way I would write AXPY in Fortran is slightly different. It is the exact translation of the math. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;m_blas.f90&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; module blas&#xA;&#xA;   interface axpy&#xA;     module procedure saxpy,daxpy&#xA;   end interface&#xA;&#xA; contains&#xA;&#xA;   subroutine daxpy(x,y,a)&#xA;     implicit none&#xA;     real(8) :: x(:),y(:),a&#xA;     y=a*x+y&#xA;   end subroutine daxpy&#xA;&#xA;   subroutine saxpy(x,y,a)&#xA;     implicit none&#xA;     real(4) :: x(:),y(:),a&#xA;     y=a*x+y&#xA;   end subroutine saxpy&#xA;&#xA; end module blas&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now let's call the above routine in a program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;test.f90&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; program main&#xA;&#xA;   use blas&#xA;   implicit none&#xA;&#xA;   real(4), allocatable :: x(:),y(:)&#xA;   real(4) :: a&#xA;   integer :: n&#xA;&#xA;   n=1000000000&#xA;   allocate(x(n),y(n))&#xA;   x=1.0&#xA;   y=2.0&#xA;   a=5.0&#xA;   call axpy(x,y,a)&#xA;   deallocate(x,y)&#xA;&#xA; end program main&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now let's compile and run it ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;login1$ ifort -fast -parallel m_blas.f90 test.f90&#xA;ipo: remark #11000: performing multi-file optimizations&#xA;ipo: remark #11005: generating object file /tmp/ipo_iforttdqZSA.o&#xA;&#xA;login1$ export OMP_NUM_THREADS=1&#xA;login1$ time ./a.out &#xA;real    0 m 4.697 s&#xA;user    0 m 1.972 s&#xA;sys     0 m 2.548 s&#xA;&#xA;login1$ export OMP_NUM_THREADS=2&#xA;login1$ time ./a.out &#xA;real    0 m 2.657 s&#xA;user    0 m 2.060 s&#xA;sys     0 m 2.744 s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that I am not using any loops or any explicit &lt;a href=&quot;http://en.wikipedia.org/wiki/OpenMP&quot;&gt;OpenMP&lt;/a&gt; directives. Would this be possible in C (that is, no use of loops and auto-parallelization)? I don't use C so I don't know.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-02-10T04:52:17.020" LastActivityDate="2012-02-10T04:52:17.020" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1196" PostTypeId="2" ParentId="1194" CreationDate="2012-02-04T23:19:51.293" Score="38" Body="&lt;p&gt;The difference in your timings seems to be due to the manual unrolling of the unit-stride Fortran &lt;strong&gt;daxpy&lt;/strong&gt;. The following timings are on a 2.67 GHz Xeon X5650, using the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./test 1000000 10000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Intel 11.1 compilers&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Fortran with manual unrolling: 8.7 sec&lt;br/&gt;&#xA;Fortran w/o manual unrolling: 5.8 sec&lt;br/&gt;&#xA;C w/o manual unrolling: 5.8 sec&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;GNU 4.1.2 compilers&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Fortran with manual unrolling: 8.3 sec&lt;br/&gt;&#xA;Fortran w/o manual unrolling: 13.5 sec&lt;br/&gt;&#xA;C w/o manual unrolling: 13.6 sec&lt;br/&gt;&#xA;C with vector attributes: 5.8 sec&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;GNU 4.4.5 compilers&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Fortran with manual unrolling: 8.1 sec&lt;br/&gt;&#xA;Fortran w/o manual unrolling: 7.4 sec&lt;br/&gt;&#xA;C w/o manual unrolling: 8.5 sec&lt;br/&gt;&#xA;C with vector atrributes: 5.8 sec&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Conclusions&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Manual unrolling helped the GNU 4.1.2 Fortran compilers on this architecture, but hurts the newer version (4.4.5) and the Intel Fortran compiler. &lt;/li&gt;&#xA;&lt;li&gt;The GNU 4.4.5 C compiler is much more competitive with Fortran than for version 4.2.1.&lt;/li&gt;&#xA;&lt;li&gt;Vector intrinsics allow the GCC performance to match the Intel compilers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Time to test more complicated routines like dgemv and dgemm?&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-02-05T01:17:21.427" LastActivityDate="2012-02-05T01:17:21.427" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1197" PostTypeId="2" ParentId="1172" CreationDate="2012-02-05T07:34:43.320" Score="1" Body="&lt;p&gt;What are the features of rational queuing policies? I'd argue that they include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;They restrict the total length of individual jobs.&lt;/strong&gt; On a shared machine, jobs should have finite run times. The allotment can perhaps be more generous on a small group cluster (4-5 days) than on a large shared resource operated by a university, national lab or other such organization (where anywhere from 1-4 days may be the norm).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;They place limits on the amount of the machine a single user can access when jobs are in the queue.&lt;/strong&gt; There's nothing worse than a machine hog that thinks &quot;all your core are belong to me&quot; [sic]. Thus, there's typically a requirement that no single user can have access to more than, say, one-fourth or one-third of a machine, so long as there are other jobs in the queue. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Perhaps most importantly, though:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Queues take into account the amount of resources &lt;em&gt;recently&lt;/em&gt; accessed.&lt;/strong&gt; This is the hardest part, though, because it's probably the most difficult to keep track of. However, the basic idea here is that somebody who has been running on a fourth of the machine for a month straight should have a substantially lower priority than somebody who is returning to start new jobs for the first time in a few months. It is equally important to notice the &lt;em&gt;recently&lt;/em&gt;: what you did six months or a year ago shouldn't matter much with respect to your current usage. Therefore, there should be some &quot;half-life&quot; on this: maybe a few weeks, maybe a few months. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It will take some time and tweaking to find the right balance, but these rules lie at the heart of most policies I've seen that haven't led to grumbling and teeth-gnashing.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-02-05T07:34:43.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1198" PostTypeId="2" ParentId="1144" CreationDate="2012-02-05T16:09:41.560" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://sympy.org/en/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt; is able to plot your piecewise function:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from sympy import *&#xA;x = Symbol('x')&#xA;n = 2 # you have to choose some explicit number for n&#xA;f = Piecewise((0, n&amp;lt;=x), (x-n, x&amp;lt;=n+1), (2-x+n, x&amp;lt;=n+1), (0, True))&#xA;plot(f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19" LastEditorUserId="28872" LastEditDate="2020-04-01T14:04:11.647" LastActivityDate="2020-04-01T14:04:11.647" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1199" PostTypeId="2" ParentId="1172" CreationDate="2012-02-05T18:53:17.193" Score="2" Body="&lt;p&gt;So as aeismail has suggested, &quot;fair&quot; is in the eyes of the beholder, and in general, any policy that doesn't let a users job run when they think it should will strike them as manifestly &quot;unfair&quot; and will generate nasty emails to the sysadmin.   So don't get too caught up in tweaking parameters; come up with a clear, simple policy, and adjust it only slowly over time as you get data, not reactively in response to complaints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, in terms of implementing things in maui, I'd suggest starting with &lt;a href=&quot;http://www.adaptivecomputing.com/resources/docs/maui/6.3fairshare.php&quot; rel=&quot;nofollow&quot;&gt;fairshare&lt;/a&gt;.  It lets you set target amounts of resources for groups over some window of time (we use 2 weeks here - pick a length of time over which, for your usage patterns, peakiness tends to average out), and as they get close to that usage, their priority starts to drop.  Note that they can still compute if there are idle cycles, but people who haven't used so much over that period will have their jobs take priority.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that part of the art here is in defining groups of users correctly, and in deciding on the original target amounts of resources which are &quot;fair&quot;.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2012-02-05T18:53:17.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1200" PostTypeId="2" ParentId="1191" CreationDate="2012-02-05T19:45:32.507" Score="4" Body="&lt;p&gt;Some research into the problem led me to two publications that do a fair job explaining the structure:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.zib.de/Publications/Reports/SC-87-02.pdf&quot; rel=&quot;nofollow&quot;&gt;Bank, Dupont, and Yserentant (1987)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;A book chapter by &lt;a href=&quot;http://books.google.com/books?id=rGMrAAAAYAAJ&amp;amp;lpg=PP1&amp;amp;pg=PA183#v=onepage&amp;amp;q&amp;amp;f=false&amp;amp;hl=en&quot; rel=&quot;nofollow&quot;&gt;Pinksy, Malhotra, and Thompson (1994)&lt;/a&gt; (Page 183).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Whereas [1] gives a fairly concise matrix structure of $\mathbf{S}_1$ (Eq. 5.3, two hierarchy levels only), it lacks a more detailed explanation.&lt;br&gt;&#xA;The authors of [2] give a more detailed derivation of the structure (Eq. 9.29 - 9.32), however  they use a rather unpractical inverse definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Combining both [1] and [2] I found the following to describe the decomposition structure pretty well:&#xA;Following [1] and initially taking a 2-Level approach, we can express $\mathbf{S}=\mathbf{S}_1$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{S} = \begin{pmatrix} I &amp;amp; 0 \\ \mathbf{R} &amp;amp; I \end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that if we compute $\mathbf{x} = \mathbf{S}\mathbf{y}$, we get a representation of the function in terms of nodal basis functions, coming from the hierarchical representation $\mathbf{y} = \begin{pmatrix} \mathbf{y}_2 &amp;amp; \mathbf{y}_1 \end{pmatrix}^T$ where $\mathbf{y_2}$ are the coarse-level coefficients and $\mathbf{y}_1$ are the fine-level coefficients .&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the hierarchical basis functions have larger influence regions on the coarse level, we can see that we need to incorporate this influence at the nodal level, which is what $\begin{pmatrix} \mathbf{R} &amp;amp; I \end{pmatrix}$ does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{R}$ is itself a sparse matrix and its rows contain the values of the coarse-level basis functions at the nodal-variable-location - which is mostly 0, except for the locations where the influence of a coarse-level function overlaps with the nodal-variable-location.&lt;br&gt;&#xA;A typical row looks like $\mathbf{R}_i = \begin{pmatrix} \mathbf{0} &amp;amp; a &amp;amp; b &amp;amp; \mathbf{0}\end{pmatrix}$ for a 1-dimensional problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to arrive at the final decomposition $\mathbf{S} = \mathbf{S_1S_2...S_{L-1}}$ we need to apply the above recursively:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A complete set of hierarchical coefficients looks like $\mathbf{y} = \begin{pmatrix} \mathbf{y_L} &amp;amp; \mathbf{y_{L-1}} &amp;amp; ... &amp;amp; \mathbf{y_1}\end{pmatrix}^T$.&lt;br&gt;&#xA;We first get rid of the level-L-coefficients by expressing this part of the signal in terms of level L-1 -coefficients:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{y}&amp;#39; = \begin{pmatrix} I &amp;amp; 0 &amp;amp; 0\\ \mathbf{R} &amp;amp; I &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; I\end{pmatrix}\cdot\mathbf{y} = \mathbf{S}_{L-1}\cdot\mathbf{y}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the coarsest basis is level L-1.&lt;br&gt;&#xA;We can now, again, express this in a finer basis (L-2) using $\mathbf{y}&amp;#39;&amp;#39; = \mathbf{S}_{L-2}\cdot\mathbf{y}&amp;#39;$ and so on.&lt;br&gt;&#xA;$\mathbf{S}_{L-2}$ looks exactly like $\mathbf{S}_{L-1}$ with the difference that $\mathbf{R}$ is a bit larger since there are more L-2 coefficients than L-1 coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end we will arrive at $\mathbf{y}^{L-1} = \mathbf{x} = \mathbf{S}_1\mathbf{y}^{L-2}$.&lt;br&gt;&#xA;$\mathbf{S_1}$ will then look exactly like described above.&lt;/p&gt;&#xA;" OwnerUserId="740" LastEditorUserId="119" LastEditDate="2012-02-05T21:56:19.150" LastActivityDate="2012-02-05T21:56:19.150" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1201" PostTypeId="2" ParentId="351" CreationDate="2012-02-05T20:11:17.920" Score="0" Body="&lt;p&gt;Surprised nobody has mentioned &lt;a href=&quot;http://www.edwardrosten.com/cvd/toon.html&quot; rel=&quot;nofollow&quot;&gt;TooN&lt;/a&gt; so far. Been happily using it for almost 3 years now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's very similar to &lt;a href=&quot;http://eigen.tuxfamily.org&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt; although not as comprehensive. However I think it has the nicer syntax in some ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also comes with classes that help modeling common transformations that are frequently encountered in Graphics and Vision, based on Lie Groups (Special Euclidean/Orthogonal in 2 and 3 dimensions, etc) and associated Lie Algebras.&lt;/p&gt;&#xA;" OwnerUserId="740" LastEditorUserId="740" LastEditDate="2012-02-06T14:29:01.367" LastActivityDate="2012-02-06T14:29:01.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1202" PostTypeId="1" CreationDate="2012-02-05T23:17:38.147" Score="15" ViewCount="770" Body="&lt;p&gt;I am very impressed with the serial performance of &lt;a href=&quot;http://www.cs.umn.edu/~saad/PDF/umsi-2004-75.pdf&quot;&gt;multilevel inverse-based ILU preconditioners&lt;/a&gt;, particularly for &lt;a href=&quot;http://epubs.siam.org/sisc/resource/1/sjoce3/v31/i5/p3781_s1&quot;&gt;heterogeneous Helmholtz&lt;/a&gt;, but I am surprised to not be able to find any open source implementations. In particular, &lt;a href=&quot;http://www.icm.tu-bs.de/~bolle/ilupack/&quot;&gt;ILUPACK&lt;/a&gt; makes binaries freely available to academics, but it does not appear that they release their source code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it really the case that no one has open sourced their implementation?&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="4244" LastEditDate="2013-05-30T22:08:28.937" LastActivityDate="2014-11-26T18:46:46.367" Title="Are there any open source inverse-based multilevel ILU implementations?" Tags="|pde|algorithms|linear-solver|libraries|preconditioning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1203" PostTypeId="1" AcceptedAnswerId="1204" CreationDate="2012-02-06T07:43:33.243" Score="4" ViewCount="210" Body="&lt;p&gt;I am working on a problem that involves solving many (thousands) of distinct linear systems of equations, each with thousands of variables.  Let's assume that the size of each matrix is exactly the same, and has the same structure (i.e. same symmetry, sparsity, diagonal dominance, etc).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming each linear system is solved by the same method, and each linear system is independent, I foresee three ways to approach solving all linear systems:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I can solve each linear system with all the processors at my disposal, one at a time.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I could allocate a certain number of linear systems to each processor, then solve each one sequentially.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A hybrid of the first two options... That is, if we have P processors, we can designate G groups of processors (assume P is a multiple of G for simplicity).  Then, for N linear systems of equations, we allocate N/G systems to each group (assume N is a multiple of G for simplicity).  Then each group can solve its N/G systems one by one in parallel using its P/G processors simultaneously.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I believe the mapping strategy that yields the solutions of all systems of equations in the fastest time depends both upon the size of each system, and the number of systems.  Which strategy works best:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If the number of systems remains constant, but the size each matrix system increases?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the size of each matrix system remains constant, but the number of systems increases?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-02-07T10:08:49.107" Title="What mapping strategy should I use when solving many large linear systems of equations?" Tags="|linear-algebra|parallel-computing|mapping-strategy|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1204" PostTypeId="2" ParentId="1203" CreationDate="2012-02-06T08:53:10.600" Score="3" Body="&lt;p&gt;Communication between nodes is (usually) much slower than accessing RAM local to that node. Strategy 2 is probably your best bet, unless you're systems are so big they can't fit into memory on one node any more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because communication between nodes is usually faster than communication with a hard disk, you should split your problem up between multiple nodes (strategy 3) to keep the whole thing in RAM if it won't fit on a single processor. If each problem is so large you need all your available computational power to fit it into memory, then use strategy 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you're dealing with matrices big enough to fill up all of the memory on one node (you'll need more than &quot;thousands of variables&quot; to do this), go with either strategy 1 or 3, depending on how big your systems are. If they're not big enough to fill up memory, go with strategy 2.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the matrices can all fit into main memory, go with strategy 2 regardless of how many systems you have.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-02-06T23:36:20.030" LastActivityDate="2012-02-06T23:36:20.030" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1205" PostTypeId="2" ParentId="1179" CreationDate="2012-02-06T13:33:31.033" Score="5" Body="&lt;p&gt;Besides &lt;a href=&quot;https://scicomp.stackexchange.com/a/1184/149&quot;&gt;Pedro&lt;/a&gt;'s suggestions, you could also look at &lt;a href=&quot;http://www.quantum-espresso.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Espresso&lt;/a&gt;, which among other things allows for &lt;a href=&quot;http://en.wikipedia.org/wiki/Car-Parrinello_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Car-Parrinello computations&lt;/a&gt;. It is (or at least, it was) written in Fortran 90.&lt;/p&gt;&#xA;" OwnerUserId="910" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-02-13T14:33:51.920" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1206" PostTypeId="2" ParentId="1171" CreationDate="2012-02-06T13:35:37.300" Score="1" Body="&lt;p&gt;Your samples are subsets of $S$, which is structured according to a graph $G$. The dimensionality of your problem is the cardinality of $S$. If $H$ is the graph Laplacian of $G$, then $\|x-y\|:=\sqrt{\langle x-y|H|x-y\rangle}$ is a metric. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your kernel could then be $e^{-t\|d\|^2}$ and you do your cluster analysis as everybody else does.&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2012-02-06T13:35:37.300" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1207" PostTypeId="1" AcceptedAnswerId="1261" CreationDate="2012-02-06T13:44:24.433" Score="13" ViewCount="1148" Body="&lt;p&gt;This question has been asked a &lt;a href=&quot;https://stackoverflow.com/questions/88740/best-way-to-get-into-an-open-source-project&quot;&gt;billion times&lt;/a&gt; on Stackoverflow however, the focus has always been on Non-Numerical Coding. I am looking for a project to contribute to within the confines of Numerical Computing and High Performance Computing. I would ideally prefer a small project but that's not necessary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best way to get involved in an open source project pertinent to SciComp?&#xA;Where does one start looking for projects? What would be the best way to enter such a project? Feature Requests/Bugs/Documentation or other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at something like ATLAS or &lt;a href=&quot;http://code.google.com/p/nix-os/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nix&lt;/a&gt;, how do you go from N00B to active?&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-05-20T18:04:47.810" Title="On Finding Open Source Projects To Contribute To" Tags="|software|" AnswerCount="5" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1208" PostTypeId="2" ParentId="1207" CreationDate="2012-02-06T14:17:26.937" Score="1" Body="&lt;p&gt;I have the same problem. &lt;strong&gt;I ended up solving my own problems with existing libraries&lt;/strong&gt; (for example COIN-OR), creating standalone executables for specific tasks. I plan to contribute these codes back once they are fairly polished and stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr/&gt;&#xA;These answers reflect well my opinion:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/a/203448/341970&quot;&gt;Finding an Open Source Project to work on&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/a/193495/341970&quot;&gt;Is there an easy way to find small open source projects?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Good luck anyhow!&lt;/p&gt;&#xA;" OwnerUserId="679" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-02-06T21:06:58.420" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1209" PostTypeId="5" CreationDate="2012-02-06T16:21:13.333" Score="0" Body="&lt;p&gt;Traditionally, we set the parameters of a model first (e.g. a partial differential equation), then observe the behavior of the system. An &quot;inverse problem&quot; is doing the reverse. We start with the observations of the output of the system, and try to determine the parameters of the model that produce these observations. Inverse theory describes different methods for accomplishing this task.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-06T18:01:39.107" LastActivityDate="2012-02-06T18:01:39.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1210" PostTypeId="4" CreationDate="2012-02-06T16:21:13.333" Score="0" Body="For questions pertaining to methods to estimate input parameters based upon output data." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-06T16:42:53.193" LastActivityDate="2012-02-06T16:42:53.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1211" PostTypeId="2" ParentId="1194" CreationDate="2012-02-06T16:54:04.533" Score="6" Body="&lt;p&gt;I think, it is not only interesting how a compiler optimizes the code for modern hardware. Especially between GNU C and GNU Fortran the code generation can be very different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's consider another example to show the differences between them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using complex numbers, the GNU C compiler produces a large overhead for nearly very basic arithmetic operation on a complex number. The Fortran compiler gives much better code. Let's have a look at the following small example in Fortran: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;COMPLEX*16 A,B,C&#xA;C=A*B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives (gfortran -g -o complex.f.o -c complex.f95; objdump -d -S complex.f.o): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C=A*B&#xA;  52:   dd 45 e0                fldl   -0x20(%ebp)&#xA;  55:   dd 45 e8                fldl   -0x18(%ebp)&#xA;  58:   dd 45 d0                fldl   -0x30(%ebp)&#xA;  5b:   dd 45 d8                fldl   -0x28(%ebp)&#xA;  5e:   d9 c3                   fld    %st(3)&#xA;  60:   d8 ca                   fmul   %st(2),%st&#xA;  62:   d9 c3                   fld    %st(3)&#xA;  64:   d8 ca                   fmul   %st(2),%st&#xA;  66:   d9 ca                   fxch   %st(2)&#xA;  68:   de cd                   fmulp  %st,%st(5)&#xA;  6a:   d9 ca                   fxch   %st(2)&#xA;  6c:   de cb                   fmulp  %st,%st(3)&#xA;  6e:   de e9                   fsubrp %st,%st(1)&#xA;  70:   d9 c9                   fxch   %st(1)&#xA;  72:   de c2                   faddp  %st,%st(2)&#xA;  74:   dd 5d c0                fstpl  -0x40(%ebp)&#xA;  77:   dd 5d c8                fstpl  -0x38(%ebp)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which are 39 bytes machine code. When we consider the same in C&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; double complex a,b,c; &#xA; c=a*b; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and take a look at the output ( done in the same way like above), we get: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  41:   8d 45 b8                lea    -0x48(%ebp),%eax&#xA;  44:   dd 5c 24 1c             fstpl  0x1c(%esp)&#xA;  48:   dd 5c 24 14             fstpl  0x14(%esp)&#xA;  4c:   dd 5c 24 0c             fstpl  0xc(%esp)&#xA;  50:   dd 5c 24 04             fstpl  0x4(%esp)&#xA;  54:   89 04 24                mov    %eax,(%esp)&#xA;  57:   e8 fc ff ff ff          call   58 &amp;lt;main+0x58&amp;gt;&#xA;  5c:   83 ec 04                sub    $0x4,%esp&#xA;  5f:   dd 45 b8                fldl   -0x48(%ebp)&#xA;  62:   dd 5d c8                fstpl  -0x38(%ebp)&#xA;  65:   dd 45 c0                fldl   -0x40(%ebp)&#xA;  68:   dd 5d d0                fstpl  -0x30(%ebp)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which are 39 bytes machine code too, but the function step 57 refer to, does the proper part of the work and perform the desired operation. So we have 27 byte machine code to run the multi operation. The function behind  is muldc3 provided by &lt;code&gt;libgcc_s.so&lt;/code&gt; and has a footprint of 1375 byte in machine code. This slows down the code dramatically and gives an interesting output when using a profiler. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we implement the BLAS examples above for &lt;code&gt;zaxpy&lt;/code&gt; and perform the same test, the Fortran compiler should gives better results than the C compiler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I used GCC 4.4.3 for this experiment, but I noticed this behavior an other GCC releases to.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in my opinion we do not only think about parallelization and vectorization when we think about which is the better compiler we also have to look how basic things are translated to the assembler code. If this translation gives bad code the optimization can only use this things as input. &lt;/p&gt;&#xA;" OwnerUserId="569" LastEditorUserId="1064" LastEditDate="2012-02-28T13:22:54.957" LastActivityDate="2012-02-28T13:22:54.957" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1212" PostTypeId="2" ParentId="1179" CreationDate="2012-02-06T17:24:24.673" Score="3" Body="&lt;p&gt;The term &quot;energy minimization&quot; usually refers to the minimization of the potential energy. For protein folding simulations, you really need to minimize free energy. If you run plain energy minimization on an unfolded protein, you will pretty soon get stuck in a local minimum. So you probably do want molecular dynamics, and use some protocol such as simulated annealing in order to get to progressively lower energies. See the &lt;a href=&quot;https://scicomp.stackexchange.com/a/1184/149&quot;&gt;answer by pedro&lt;/a&gt; for software suggestions.&lt;/p&gt;&#xA;" OwnerUserId="140" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-02-09T15:56:19.310" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1213" PostTypeId="1" CreationDate="2012-02-06T17:37:44.790" Score="4" ViewCount="356" Body="&lt;p&gt;I use the discontinuous galerkin method to solve the steady flow 1D shallow water equations with a bump at the bottom. This flow is frictionless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the runge-kutta method to approximate the time derivative of these equations and gauss-lobatto quadrature to approximate the integral equation in the weak form of the source term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution that I get always blows up and doesn't correspond with the exact solution, I think that I may be approximating the source term incorrectly but I don't know how to fix this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="912" LastEditorUserId="549" LastEditDate="2012-02-07T02:18:34.593" LastActivityDate="2012-02-07T02:18:34.593" Title="How does one handle the source term in the Shallow Water Equations when using the discontinuous galerkin method?" Tags="|finite-element|fluid-dynamics|" AnswerCount="0" CommentCount="6" ClosedDate="2013-05-24T05:11:33.537" ContentLicense="CC BY-SA 3.0" />
  <row Id="1214" PostTypeId="2" ParentId="952" CreationDate="2012-02-06T17:47:40.120" Score="2" Body="&lt;p&gt;In the interest of having at least some answer, it seems like the analysis you have above is correct. First-order boundary conditions do affect the accuracy of a second-order discretization (if I'm remembering LeVeque's book on finite difference approximations correctly; DavidKetcheson can help me out with that), so the solutions of the discrete problem appear to be converging to the solution of the analytical problem at the appropriate rate.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-02-06T17:47:40.120" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1215" PostTypeId="2" ParentId="1207" CreationDate="2012-02-06T21:38:37.370" Score="5" Body="&lt;p&gt;Warning: shameless self-promotion ahead! :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a lead developer of a small open-source scientific computing project:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://numerics.kaust.edu.sa/pyclaw/&quot;&gt;&lt;strong&gt;PyClaw&lt;/strong&gt;&lt;/a&gt;  (source &lt;a href=&quot;https://github.com/clawpack/pyclaw&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's part of the larger family of &lt;strong&gt;Clawpack&lt;/strong&gt; codes for solving hyperbolic PDEs.  PyClaw is designed to solve arbitrary hyperbolic systems of PDEs on logically quadrilateral or hexahedral grids.  It includes two different kinds of finite volume solvers and is scalable to large supercomputers (see our &lt;a href=&quot;http://arxiv.org/abs/1111.6583&quot;&gt;submitted journal paper&lt;/a&gt; including runs on up to 65K cores).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at the documentation and the &lt;a href=&quot;https://github.com/clawpack/pyclaw/issues&quot;&gt;issue tracker&lt;/a&gt;.  There's plenty that needs doing!  Where you can best contribute depends on your background and training.  Perhaps the best way to get started is to tackle a small issue and submit a pull request on Github.  Alternatively, it might make sense to send a message to the &lt;a href=&quot;http://groups.google.com/group/claw-dev&quot;&gt;developers mailing list&lt;/a&gt; introducing yourself and explaining your areas of expertise.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-02-06T21:38:37.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1216" PostTypeId="1" AcceptedAnswerId="1272" CreationDate="2012-02-06T23:32:03.483" Score="16" ViewCount="1495" Body="&lt;p&gt;I have a linear system of equations of size mxm, where m is large.  However, the variables that I'm interested in are just the first n variables (n is small compared to m).  Is there a way I can approximate the solution for the first m values without having to solve the entire system?  If so, would this approximation be faster than solving the full linear system?  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2017-02-12T16:31:05.933" LastActivityDate="2017-02-12T16:31:05.933" Title="Can the solution of a linear system of equations be approximated for only the first few variables?" Tags="|linear-algebra|approximation|" AnswerCount="4" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="1217" PostTypeId="2" ParentId="1203" CreationDate="2012-02-07T00:39:08.540" Score="3" Body="&lt;p&gt;I'd go for option two with a hybrid shared/distributed memory twist... I'm assuming that the 13 machines in your cluster all access the same shared file system and that each computer has enough memory to load 8 problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your code could then look something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int fd, next_task = 0, my_task, nr_tasks;&#xA;&#xA;/* Open the file you want to use to synchronize the nodes. */&#xA;if ( ( fd = open( &quot;sharedlockfile&quot; , O_CREAT ) ) &amp;lt; 0 ) {&#xA;    printf( &quot;failed to create lock file.\n&quot; );&#xA;    abort();&#xA;    }&#xA;&#xA;/* Initialize the curr_task in the file. */&#xA;if ( &amp;lt;I am the first node in this cluster!&amp;gt; )&#xA;    if ( write( fd , &amp;amp;next_task , sizeof(int) ) &amp;lt; 0 ) {&#xA;        printf( &quot;failed to write to lock file.&quot; );&#xA;        abort();&#xA;        }&#xA;&#xA;/* Main parallel loop. */&#xA;#pragma omp parallel shared(fd,next_task), private(mytask)&#xA;while ( next_task &amp;lt; nr_tasks ) {&#xA;&#xA;    /* Try to get a hold of the lock file, grab a task, and write the new counter. */&#xA;    #pragma omp critical&#xA;    { &#xA;&#xA;        /* Get a hold of the file lock (this will block until the file is free). */&#xA;        if ( flock( fd , LOCK_EX ) &amp;lt; 0 ) {&#xA;            printf( &quot;file locking failed.\n&quot; );&#xA;            abort(); // not sure I'm allowed to abort in a parallel block...&#xA;            }&#xA;&#xA;        /* Read the index of the next task. */&#xA;        if ( lseek( fd , 0 , SEEK_SET ) &amp;lt; 0 || read( fd , &amp;amp;next_task , sizeof(int) ) &amp;lt; 0 ) {&#xA;            printf( &quot;error reading next task id.\n&quot; );&#xA;            abort();&#xA;            }&#xA;&#xA;        /* Remember my task and update the counter. */&#xA;        my_task = next_task;&#xA;        next_task += 1;&#xA;&#xA;        /* Write the index of the next task back to the file. */&#xA;        if ( lseek( fd , 0 , SEEK_SET ) &amp;lt; 0 || write( fd , &amp;amp;next_task , sizeof(int) ) &amp;lt; 0 ) {&#xA;            printf( &quot;error writing next task id.\n&quot; );&#xA;            abort();&#xA;            }&#xA;&#xA;        /* Let go of the file. */&#xA;        if ( flock( fd , LOCK_UN ) &amp;lt; 0 ) {&#xA;            printf( &quot;file un-locking failed.\n&quot; );&#xA;            abort();&#xA;            }&#xA;&#xA;        } /* end of critical section */&#xA;&#xA;    /* Did we get a valid task ID? */&#xA;    if ( my_task &amp;lt; nr_tasks ) {&#xA;&#xA;        /*&#xA;         * Do whatever this task implies.&#xA;         */&#xA;&#xA;        } /* check if valid task ID. */&#xA;&#xA;    } /* Main loop. */&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please note that I haven't tested the above code! I've done similar things before and most of this is off the top of my head... Check out the manpage for &lt;code&gt;flock&lt;/code&gt; to be sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all the nodes in your cluster execute this, they should each spawn as many threads as they have cores (or as many as you specified in OMP_NUM_THREADS) and each core will try to acquire a task id. Access to &lt;code&gt;next_task&lt;/code&gt; is controlled on two levels:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By the &lt;code&gt;#pragma omp critical&lt;/code&gt; so that no two threads on the same node try to mess with the &lt;code&gt;next_task&lt;/code&gt; variable at the same time.&lt;/li&gt;&#xA;&lt;li&gt;By the exclusive file lock so that no two nodes read/update the &lt;code&gt;next_task&lt;/code&gt; variable stored in the file at the same time.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This should give you a good, adaptive, dynamic scheduling of your computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Uhm, may not be as easy as I thought with the locking as it seems to be fd-dependent and thus may not work accross nodes. There seems to be a good solution &lt;a href=&quot;https://stackoverflow.com/a/7325147&quot;&gt;here&lt;/a&gt; though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If solving each system on a separate core is too much of a strain for your memory bus, e.g. you notice that it's not scaling well at all, you might want to try &lt;a href=&quot;http://icl.cs.utk.edu/magma/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAGMA&lt;/a&gt;, which is like LAPACK, but scales very nicely on multicore architectures.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-02-07T10:08:49.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1218" PostTypeId="5" CreationDate="2012-02-07T02:28:15.393" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-07T02:28:15.393" LastActivityDate="2012-02-07T02:28:15.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1219" PostTypeId="4" CreationDate="2012-02-07T02:28:15.393" Score="0" Body="Referring to a heuristic that mimics the process of natural evolution to obtain the solutions of search and optimization problems." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-07T02:34:24.640" LastActivityDate="2012-02-07T02:34:24.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1220" PostTypeId="5" CreationDate="2012-02-07T02:47:19.003" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-07T02:47:19.003" LastActivityDate="2012-02-07T02:47:19.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1221" PostTypeId="4" CreationDate="2012-02-07T02:47:19.003" Score="0" Body="Referring to the study of brief oscillations whose amplitude grows and decays in a finite time. " OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-07T15:59:06.673" LastActivityDate="2012-02-07T15:59:06.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1222" PostTypeId="1" CreationDate="2012-02-07T11:23:59.590" Score="5" ViewCount="133" Body="&lt;p&gt;Problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an incremental online clustering algorithm which need 4 parameters that should be specified by the user before execution. The algorithm will gives &quot;good results&quot; if &quot;a good parameter values&quot; are chosen, but there is no way to know beforehand what are the best matching parameter values. We can for example perform many executions and evaluate the results after each execution until we find the optimal parameter values. However, if we change the dataset that the algorithm process, we need to determine again what are the optimal parameter values for that dataset. Moreover, the online algorithm is supposed to process a continuously arriving and evolving data stream, i.e. we don't have all data beforehand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is not the online clustering algorithm itself; I just wonder if there is any solutions/methods to automatically adapt the parameter values during the execution ? How is it possible to use an approximation algorithm to adapt the parameter values in my case ? Which approximate algorithm can I use and how to adapt it (if necessary) to this problem ? Any idea is welcome.&lt;/p&gt;&#xA;" OwnerUserId="888" LastActivityDate="2012-02-07T14:56:25.513" Title="Using an approximation algorithm to adapt parameter values of a given algorithm" Tags="|optimization|machine-learning|linear-programming|approximation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1223" PostTypeId="1" CreationDate="2012-02-07T14:11:04.763" Score="0" ViewCount="186" Body="&lt;p&gt;Consider the following original LP: $\mathit{min}$ c'$x$ s.t: $Ax=0 \wedge 0\le x\le 1$ . This is my original LP which has to be solved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, using some reductions, I reduced the original LP to the following converted LP: $\mathit{min}$ d'$y$ s.t: $By=0 \wedge 0\le y\le 1$. Using some relations between $x$ and $y$, I proved that these two LP are equivalent. This means the optimal solution for one of them gives the optimal solution for the other and visa verse. For optimal solutions the objective values are the same as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the converted LP cannot be solved exactly. Now, I have got an approximate optimal solution vector for the converted LP, in which values for each pairs of variables are within some $\epsilon$ of each other. This solution is not feasible for the original LP but still is optimal based on the variable transformation rules that I have used to show the equivalence of these two LP's. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is how to push this solution to be feasible by not ruining the objective value too much. How can I find the best lower bound?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="276" LastEditDate="2012-02-07T19:17:42.250" LastActivityDate="2012-03-08T11:57:54.113" Title="LP infeasibility" Tags="|linear-programming|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1224" PostTypeId="2" ParentId="1222" CreationDate="2012-02-07T14:56:25.513" Score="4" Body="&lt;p&gt;There are many different types of approximations (or &quot;surrogate models&quot;) you could try. Some that come to mind are &lt;a href=&quot;http://en.wikipedia.org/wiki/Kriging&quot; rel=&quot;nofollow&quot;&gt;Kriging&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Multivariate_adaptive_regression_splines&quot; rel=&quot;nofollow&quot;&gt;MARS&lt;/a&gt;, and &lt;a href=&quot;http://en.wikipedia.org/wiki/Radial_basis_function&quot; rel=&quot;nofollow&quot;&gt;Radial Basis Functions&lt;/a&gt;. These types of surrogate models (as opposed to polynomial regression) can accommodate a wide range of functional relationships, but you might need to experiment a bit to find which works best for your application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, since finding the optimal parameters is your ultimate goal, you might want to check out the &lt;a href=&quot;http://www.mendeley.com/research/efficient-global-optimization-of-expensive-blackbox-functions/&quot; rel=&quot;nofollow&quot;&gt;Efficient Global Optimization&lt;/a&gt; method. This method adaptively constructs a kriging model as it searches for the global optimum. I've had a lot of success with this method in the past.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2012-02-07T14:56:25.513" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1225" PostTypeId="5" CreationDate="2012-02-07T15:27:22.190" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-07T15:27:22.190" LastActivityDate="2012-02-07T15:27:22.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1226" PostTypeId="4" CreationDate="2012-02-07T15:27:22.190" Score="0" Body="Referring to methods that produce good &quot;approximate&quot; solutions, particularly for np-hard, np-complete, or combinatorial optimization problems." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-07T15:57:36.893" LastActivityDate="2012-02-07T15:57:36.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1227" PostTypeId="1" CreationDate="2012-02-07T15:32:02.583" Score="-5" ViewCount="1253" Body="&lt;p&gt;What is the form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Successive_over-relaxation&quot; rel=&quot;nofollow&quot;&gt;SOR iterative equation&lt;/a&gt; for the heat equation&#xA;$u_{xx}=u_{t}-1$ using centered differences both in time and spatial&#xA;derivatives and using &lt;a href=&quot;https://en.wikipedia.org/wiki/Crank%E2%80%93Nicolson_method&quot; rel=&quot;nofollow&quot;&gt;Crank-Nicolson method&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(u(x,0)=u(L,t)=u(0,t)=A;0&amp;lt;x&amp;lt;L,0&amp;lt;t&amp;lt;\infty)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me to obtain the SOR iterative equation?&lt;/p&gt;&#xA;" OwnerUserId="918" LastEditorUserId="9" LastEditDate="2012-03-13T15:06:41.757" LastActivityDate="2012-03-13T15:06:41.757" Title="Successive over-relaxation formation of heat equation?" Tags="|pde|finite-difference|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1228" PostTypeId="2" ParentId="1227" CreationDate="2012-02-07T20:49:27.707" Score="1" Body="&lt;p&gt;Assuming the time index is $j$, you can obtain the SOR iteration by solving your iteration for $u_{i,j+1}$.  Then you can iterate as:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{i,j+1}^{N+1}=\omega (U_{i,j}^{gs})^{N+1} +(1-\omega)(U_{i,j}^{prev})^{N}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $(U_{i,j}^{gs})^{N+1}$ is the gauss-seidel value obtained on the current iteration, and $(U_{i,j}^{prev})^{N}$ is the value obtained from the previous iteration.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-02-07T20:49:27.707" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1233" PostTypeId="2" ParentId="1216" CreationDate="2012-02-08T03:19:45.663" Score="4" Body="&lt;p&gt;The long answer is...sort of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can re-arrange your system of equations such that the farthest right $k$ columns are the variables which you wish to solve for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 1: Perform Gaussian Elimination so that the matrix is upper triangular.&#xA;Step 2: solve by back substitution for only the first (last) $k$ variables which you are interested in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will save you the computational complexity of having to solve for the last $n-k$ variables via back-substitution, which could be worth it if $n$ is as large as you say. Keep in mind that a fair amount of work will still have to be done for step 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, keep in mind that restricting the order in which you are going to perform back-substituion may restrict the form of the matrix (it takes away the ability to exchange columns) which could &lt;em&gt;possibly&lt;/em&gt; lead to an ill conditioned system, but I am not sure about that - just something to keep in mind.&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="276" LastEditDate="2012-02-11T17:17:37.577" LastActivityDate="2012-02-11T17:17:37.577" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1234" PostTypeId="1" AcceptedAnswerId="1237" CreationDate="2012-02-08T12:01:17.203" Score="11" ViewCount="2100" Body="&lt;p&gt;I'm just getting tucked into fortran 95 for some quantum mechanics simulations. Honestly, I've been spoiled by Octave so I've taken matrix exponentiation for granted. Given a (small, $n\leq 36$) &lt;em&gt;skew&lt;/em&gt;-Hermitian matrix of size $n\times n$, what is the most efficient way of using LAPACK to solve this problem? I'm not using the LAPACK95 wrapper, just direct calls to LAPACK.&lt;/p&gt;&#xA;" OwnerUserId="116" LastEditorUserId="123" LastEditDate="2012-02-09T08:19:46.040" LastActivityDate="2015-03-08T05:00:34.443" Title="Matrix exponential of a skew-Hermitian matrix with fortran 95 and LAPACK" Tags="|fortran|lapack|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1235" PostTypeId="2" ParentId="1234" CreationDate="2012-02-08T15:15:17.573" Score="0" Body="&lt;p&gt;If all you need is the matrix exponential multiplied by a vector, then &lt;a href=&quot;http://www.maths.uq.edu.au/expokit/fortran/zhexpv.f&quot; rel=&quot;nofollow&quot;&gt;this fortran subroutine&lt;/a&gt; may be of some use to you.  It computes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(e^A)v$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where v is a vector, and A is a regular hermitian matrix.  It is a subroutine from the &lt;a href=&quot;http://www.maths.uq.edu.au/expokit&quot; rel=&quot;nofollow&quot;&gt;EXPOKIT&lt;/a&gt; library&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, you may want to consider &lt;a href=&quot;http://www.maths.uq.edu.au/expokit/fortran/zgpadm.f&quot; rel=&quot;nofollow&quot;&gt; this&lt;/a&gt; subroutine, which works for any general complex matrix A.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-08T15:55:39.973" LastActivityDate="2012-02-08T15:55:39.973" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1236" PostTypeId="2" ParentId="1234" CreationDate="2012-02-08T15:25:53.590" Score="5" Body="&lt;p&gt;Since I'm on my phone, I can't link things easily, and will add links later. You'll probably want to look at the paper &quot;19 Dubious Ways to Calculate the Matrix Exponential&quot;, the Fortran library EXPOKIT, Jitse Niesen's paper on Krylov methods for calculating the Matrix exponential, and some of Nick Higham's recent papers on matrix exponentials. It's more common to need the product of a matrix exponential and a vector than the matrix exponential alone, and here, Krylov methods can be quite helpful. For smaller, dense matrices like the ones you describe, Padé methods might be better, but I've had a lot of success with Krylov methods when used inside exponential methods for numerical integration of ODEs. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-02-08T15:25:53.590" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1237" PostTypeId="2" ParentId="1234" CreationDate="2012-02-08T15:49:22.477" Score="16" Body="&lt;p&gt;Matrix exponentials of skew-Hermitian matrices are cheap to compute: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose $A$ is your skew-Hermitian matrix, then $iA$ is Hermitian, and via &lt;a href=&quot;http://www.netlib.org/lapack/complex16/zheevd.f&quot;&gt;zheevd&lt;/a&gt; and friends you can get the decomposition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$iA = U \Lambda U^H,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is the unitary eigenvector matrix and $\Lambda$ is real and diagonal. Then, trivially,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = U (-i \Lambda) U^H.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have $U$ and $\Lambda$, it is easy to compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\exp(A)=\exp(U (-i\Lambda) U^H)=U \exp(-i\Lambda) U^H$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by first exponentiating the eigenvalues, setting $B := U$ via &lt;a href=&quot;http://www.netlib.org/blas/zcopy.f&quot;&gt;zcopy&lt;/a&gt;, performing $B := B \exp(-i \Lambda)$ by running &lt;a href=&quot;http://www.netlib.org/blas/zscal.f&quot;&gt;zscal&lt;/a&gt; on each column with an exponentiated eigenvalue, and finally setting your result to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \exp(A) := B U^H$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;via &lt;a href=&quot;http://www.netlib.org/blas/zgemm.f&quot;&gt;zgemm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-02-09T03:26:35.207" LastActivityDate="2012-02-09T03:26:35.207" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1238" PostTypeId="5" CreationDate="2012-02-08T16:17:08.360" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-08T16:17:08.360" LastActivityDate="2012-02-08T16:17:08.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1239" PostTypeId="4" CreationDate="2012-02-08T16:17:08.360" Score="0" Body="LAPACK (Linear Algebra PACKage) is a commonly used library of subroutines for numerical linear algebra tasks, including solutions of linear sets of equations, linear least squares, eigenvalue problems, and singular value decomposition. LAPACK routines may be used with fortran, C and relatives and a variety of other languages." OwnerUserId="116" LastEditorUserId="116" LastEditDate="2012-02-08T16:39:20.433" LastActivityDate="2012-02-08T16:39:20.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1240" PostTypeId="1" CreationDate="2012-02-08T17:47:48.273" Score="7" ViewCount="376" Body="&lt;p&gt;In light of the recent &lt;a href=&quot;http://thecostofknowledge.com/&quot;&gt;petition to boycott Elsevier&lt;/a&gt;, I was wondering what options we have in Computational Science for&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Journals which are completely open-access,&lt;/li&gt;&#xA;&lt;li&gt;Journals which allow/support open-access articles.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am aware that there are quite a few sub-areas in Computational Science, each with their own set of specialized publications, but I'm looking for alternatives to more general venues such as the Journal of Computational Physics or the Journal of Computational and Applied Mathematics.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-02-11T16:57:17.493" Title="Open-access journals in Computational Science" Tags="|publications|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1241" PostTypeId="1" CreationDate="2012-02-08T18:26:46.197" Score="5" ViewCount="155" Body="&lt;p&gt;I would like to know whether the constant &#xA;in the time complexity of computing the median &#xA;is different from that of computing general quantiles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fx01&amp;lt;-function(ll,a) quantile(a[,ll],0.75)&#xA;fx02&amp;lt;-function(ll,a) median(a[,ll])&#xA;&#xA;n&amp;lt;-1000&#xA;d&amp;lt;-1000&#xA;a&amp;lt;-matrix(rnorm(n*d),n,d)&#xA;&#xA;system.time(lapply(1:d,fx01,a=a))&#xA;system.time(lapply(1:d,fx02,a=a))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Typically, I observe that computing a general quantile takes &#xA;2-4 times more time than a median. I would like to know&#xA;if this is an implementation issue or whether it is set in stone.&lt;/p&gt;&#xA;" OwnerUserId="479" LastEditorUserId="549" LastEditDate="2012-09-02T13:03:19.987" LastActivityDate="2013-03-25T16:08:06.100" Title="complexity constants in median computations same as that of general quantiles?" Tags="|complexity|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1242" PostTypeId="1" AcceptedAnswerId="1243" CreationDate="2012-02-08T23:27:41.387" Score="13" ViewCount="600" Body="&lt;p&gt;For applications requiring significant computational resources, high performance can be a critical factor when it comes to delivering scientific results or achieving &quot;break-throughs&quot; in reasonable time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much should time and effort should software developers invest in optimizing an application? What are the key criteria used? &lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="9" LastEditDate="2012-02-10T07:51:31.287" LastActivityDate="2012-02-15T10:53:47.373" Title="How much should scientific software be optimized?" Tags="|software|hpc|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1243" PostTypeId="2" ParentId="1242" CreationDate="2012-02-08T23:43:45.430" Score="15" Body="&lt;p&gt;In the vast majority of cases, improvements in &lt;em&gt;algorithms&lt;/em&gt; make a bigger difference than improvement in optimization. Algorithms are also more portable than low-level optimizations. My advice is to follow general best practices with respect to memory layout for cache reuse, avoiding excessive copies or communication, treating the file system in a sane way, and making floating point kernels have sufficient granularity for vectorization. Sometimes this is enough to reach an acceptably high fraction of &quot;peak&quot; (for this operation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Always sketch a performance model for operations that you think are important (or that you discover are important by profiling). Then you can use the performance model to estimate what a highly tuned implementation could deliver. If you decide that speedup is worth it (relative to the other things you could be doing), then make the optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the hardest challenge is designing high-level, important (in the sense that a lot of code will depend on these choices) interfaces and data structures so that you can optimize later without needing to change the API. In contrast to specific optimizations and general guidelines, I don't know how to teach this except through experience. Working with performance-sensitive open source software helps. As with any API decision, it's important to understand the problem space.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-08T23:43:45.430" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1244" PostTypeId="2" ParentId="1242" CreationDate="2012-02-08T23:48:18.247" Score="8" Body="&lt;p&gt;How would you define &quot;optimize&quot;? There's an entire spectrum from the development of better algorithms or computational models down to using hand-tuned assembler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion and experience, the low-hanging fruit is somewhere in the middle, e.g. choosing an algorithm that is best suited for the underlying computer architecture. The algorithm doesn't necessarily have to be novel and your understanding of the underlying architecture doesn't necessarily have to be very specific, e.g.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you know that your architecture supports SIMD, re-structure the calculation such that your operations can be written in terms of short vectors,&lt;/li&gt;&#xA;&lt;li&gt;If you know you're architecture is a multi-core computer, try to break-down your computational task into individual sub-tasks that don't interfere with each other and run them in parallel (think a DAG of your sub-tasks),&lt;/li&gt;&#xA;&lt;li&gt;If your underlying architecture has a GPU, think of ways you can re-formulate your computation as a group of threads marching through the data in lock-step,&lt;/li&gt;&#xA;&lt;li&gt;etc...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of the above features, e.g. SIMD, parallelism and GPUs, can be accessed without to much low-level knowledge, but only really offer an advantage in algorithms that can readily exploit them. &lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-02-08T23:48:18.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1245" PostTypeId="5" CreationDate="2012-02-09T03:04:47.950" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-09T03:04:47.950" LastActivityDate="2012-02-09T03:04:47.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1246" PostTypeId="4" CreationDate="2012-02-09T03:04:47.950" Score="0" Body="The study of the waves emitted by radiated energy and its interaction with matter. " OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-09T03:13:22.103" LastActivityDate="2012-02-09T03:13:22.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1247" PostTypeId="2" ParentId="1242" CreationDate="2012-02-09T05:36:02.580" Score="4" Body="&lt;p&gt;I agree with all the answers already put forth thus far... I just want to address one more overlooked aspect of code optimization: quality expectation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue of code optimization typically arises when the user tries to solve larger and larger problems and the code is insufficient to meet the user's needs/expectations.  The amount of time that one should invest in code optimization depends on the demand to meet this expectation.  It is certainly worth investing significant time if there is a critical need for a competitive advantage (e.g. finishing &amp;amp; publishing your research on a hot topic before others).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, how much time should be invested depends upon how fast you need it to be, and how portable you want the code to be.  Often, these two needs are in conflict with each other, and you have to decide which is more important before you begin optimization.  The more portable you want it, the more you have to rely on high level design changes to the code (algorithm/data structure).  The faster you want the code to perform, it must be tuned  with low level optimizations specific to a particular machine (e.g. code/compiler/runtime optimizations).&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-02-09T05:36:02.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1248" PostTypeId="2" ParentId="1240" CreationDate="2012-02-09T08:32:14.947" Score="11" Body="&lt;p&gt;&lt;strong&gt;The boycott of Elsevier should not be mistaken as a push for open access journals&lt;/strong&gt;.  The phrase &quot;open access&quot; does not appear in any of the reasons given for the boycott at &lt;a href=&quot;http://thecostofknowledge.com&quot; rel=&quot;nofollow&quot;&gt;http://thecostofknowledge.com&lt;/a&gt;.  Rather, the reasons are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;High prices &lt;/li&gt;&#xA;&lt;li&gt;Bundling&lt;/li&gt;&#xA;&lt;li&gt;Elsevier's support for SOPA and similar efforts&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;strong&gt;many open access journals run by commercial publishers also charge high prices&lt;/strong&gt;; they just charge them to authors instead of readers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very many journals allow one to put pre- and post-prints of articles on &lt;a href=&quot;http://arxiv.org/&quot; rel=&quot;nofollow&quot;&gt;arXiv&lt;/a&gt; or institutional servers, thus providing open access without the need to charge high fees to authors.  &lt;a href=&quot;http://www.sherpa.ac.uk/romeo/index.php&quot; rel=&quot;nofollow&quot;&gt;Sherpa/RoMEO&lt;/a&gt; has a searchable database of journals' policies on archiving pre-/post-prints.  For instance, all &lt;a href=&quot;http://epubs.siam.org/&quot; rel=&quot;nofollow&quot;&gt;SIAM journals&lt;/a&gt; are &quot;green&quot;, meaning that they allow archiving of both pre- and post-print versions.  They also have &lt;a href=&quot;http://www.siam.org/journals/prices.php&quot; rel=&quot;nofollow&quot;&gt;low subscription prices&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-02-11T16:57:17.493" LastActivityDate="2012-02-11T16:57:17.493" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1249" PostTypeId="2" ParentId="1242" CreationDate="2012-02-09T08:33:52.203" Score="4" Body="&lt;p&gt;You'll have to make the (cost) analysis of spending so many man-months (and those are always mythical :-) ) on gaining execution speed. You'll have to figure out how many times this piece of software will be used and by how many people so you can estimate the gain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rule of thumb, as always, is the famous 80/20 rule. At some moment it just doesn't add up any more to spend more and more time in gaining few percentages (or less) of running time. But you'll have to analyze.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I sincerely agree with the above posters: make sure your API is well thought out so it doesn't need many changes and make sure the code is portable and maintainable (think about having to re-analyze an algorithm you wrote and nitty-gritty optimized ten years ago). And make sure that you use good programming practice and standard libraries. Chances are reasonable somebody already thought about the most efficient algorithm for your application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To cite Donald Knuth: &quot;premature optimization is the root of all evil&quot;. So profile your code, but not too soon.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-02-09T08:33:52.203" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1251" PostTypeId="5" CreationDate="2012-02-09T14:52:45.463" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-09T14:52:45.463" LastActivityDate="2012-02-09T14:52:45.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1252" PostTypeId="4" CreationDate="2012-02-09T14:52:45.463" Score="0" Body="Refering to an np-hard combinatorial optimization problem: given a list of cities and their distances from eachother, find the shortest path that visits each city only once." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-09T15:56:42.710" LastActivityDate="2012-02-09T15:56:42.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1253" PostTypeId="5" CreationDate="2012-02-09T14:57:37.517" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-09T14:57:37.517" LastActivityDate="2012-02-09T14:57:37.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1254" PostTypeId="4" CreationDate="2012-02-09T14:57:37.517" Score="0" Body="Basic Linear Algebra Subprograms - A standard API library with vector-vector, matrix-vector, and matrix-matrix operations." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-09T15:56:33.990" LastActivityDate="2012-02-09T15:56:33.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1255" PostTypeId="2" ParentId="180" CreationDate="2012-02-09T15:42:38.287" Score="1" Body="&lt;p&gt;I found an energy decomposition scheme that does exactly what I was looking for. Here is a link to a paper describing the scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1063/1.3253797&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1063/1.3253797&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sum of the repulsive and electrostatic contribution to the interaction energy of a dimer is defined as the difference in energy between the dimer and the two separate monomers, where the electron density of the dimer is constrained to be the sum of the monomer densities. This paper shows that this energy can be computed variationally in the context of Kohn-Sham DFT. The electrostatic interaction is easily identified, and the residual can be attributed to Pauli repulsion.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2012-02-09T15:42:38.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1256" PostTypeId="2" ParentId="252" CreationDate="2012-02-09T16:02:17.447" Score="1" Body="&lt;p&gt;I like the &lt;a href=&quot;http://pubchem.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow&quot;&gt;Pubchem&lt;/a&gt; database for this purpose. Although it contains large molecules too, one can easily specify filters to get a list of &lt;em&gt;small&lt;/em&gt; molecules. The website also provides a batch download service, which is convenient for large datasets. One can use InChI identifiers to search for a specific compound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CAS to XYZ conversion does not really exist. The CAS (Chemical Abstract Service) is just a number for each molecule in the CAS database. It is not possible to derive somehow the structure from the CAS number. You can use the CAS number to look up the molecular structure, but is not free as far as I know. See www.cas.org for more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;InChI to XYZ conversion is possible with the &lt;a href=&quot;http://openbabel.org/&quot; rel=&quot;nofollow&quot;&gt;OpenBabel&lt;/a&gt; software. I'm not aware of a web service for this type of conversion.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2012-02-09T16:02:17.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1257" PostTypeId="2" ParentId="1240" CreationDate="2012-02-09T17:35:39.763" Score="-1" Body="&lt;p&gt;Hindawi Publishing has a couple: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hindawi.com/journals/ana/&quot; rel=&quot;nofollow&quot;&gt;http://www.hindawi.com/journals/ana/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hindawi.com/journals/cmmm/&quot; rel=&quot;nofollow&quot;&gt;http://www.hindawi.com/journals/cmmm/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hindawi.com/journals/mpe/&quot; rel=&quot;nofollow&quot;&gt;http://www.hindawi.com/journals/mpe/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for all these &quot;open access&quot; journals, the authors pay a substantial &quot;article processing charge&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't checked their &quot;rating&quot;, though.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-02-09T17:35:39.763" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1258" PostTypeId="1" CreationDate="2012-02-09T17:52:54.773" Score="8" ViewCount="4111" Body="&lt;p&gt;I have created a spline to fit my data in python using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;spline=scipy.interpolate.UnivariateSpline(energy, fpp, k=4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The equation I want to use involves a summation between n=2 and n=infinity, where n is the order of the differential at a point Eo. However, using;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;UnivariateSpline.__call__(spline, e0, nu=n)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to call in the value, I am unable to get a value for anything past the 4th order differential. Is there any other function that people know of for evaluating this function? Above about the 8th order there is a pre-multiplier which should set the value to zero but I still need to go higher than a 4th order. &lt;/p&gt;&#xA;" OwnerDisplayName="Karim Sutton" LastEditorUserId="123" LastEditDate="2012-02-11T14:35:32.943" LastActivityDate="2012-02-12T08:31:52.680" Title="High Order derivatives of splines using SciPy" Tags="|python|interpolation|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1259" PostTypeId="2" ParentId="1258" CreationDate="2012-02-10T07:38:09.620" Score="3" Body="&lt;p&gt;By default you use cubic splines, which are 3rd order piecewise polynomials. If you take the fourth derivative of a 3rd order polynomial, you will end up with 0. If you really need these high order differentials, using cubic splines will do you no good.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-02-10T07:38:09.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1260" PostTypeId="2" ParentId="1258" CreationDate="2012-02-10T08:13:50.853" Score="10" Body="&lt;p&gt;Examining the &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.UnivariateSpline.html#scipy.interpolate.UnivariateSpline&quot;&gt;documentation&lt;/a&gt; for the &lt;code&gt;UnivariateSpline&lt;/code&gt; object, it seems as though you're constructing a 4th order spline, which is why you can't get values for derivatives of greater than 4th order. (All of those derivatives, as you're aware, would be zero.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SciPy limits the polynomial degree of splines to be 5th order or less (i.e., &lt;code&gt;k &amp;lt;= 5&lt;/code&gt;). If you need an 8th order polynomial spline interpolant, you'd have to find an alternate library, or possibly code it up yourself.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-02-10T08:13:50.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1261" PostTypeId="2" ParentId="1207" CreationDate="2012-02-10T08:58:29.987" Score="12" Body="&lt;p&gt;I think you'll get most out of it if you contribute to projects you actively use for other work, since that motivates you to develop functionality you need yourself. Ultimately, this is how most open source software is written: by people who needed the functionality for one reason or another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of our own project, &lt;a href=&quot;http://www.dealii.org&quot;&gt;deal.II&lt;/a&gt;, I had written a description of how to contribute a while back on &lt;a href=&quot;http://dealii.sourceforge.net/index.php/Deal.II_Questions_and_Answers#I_want_to_contribute_to_the_development_of_deal.II.21&quot;&gt;our FAQs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="67" LastEditDate="2012-02-10T16:06:07.660" LastActivityDate="2012-02-10T16:06:07.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1262" PostTypeId="2" ParentId="1258" CreationDate="2012-02-10T10:27:20.483" Score="11" Body="&lt;p&gt;If you need higher-order derivatives, you won't get good results using equidistant data points. If you can sample your function at arbitrary nodes, I would recommend using Chebyshev points, i.e. $$x_k = \cos\left( \pi \frac{k}{n} \right), \quad k=0\dots n$$ for a polynomial of degree $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can evaluate the polynomial stably using &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.BarycentricInterpolator.html&quot;&gt;Barycentric Interpolation&lt;/a&gt;. Note that since you're using a polynomial of high degree over the entire interval, you will probably need less data points. Note also that this assumes that your data can be represented by a polynomial, i.e. it is continuous and smooth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting higher-order derivatives from the interpolant is a bit tricky and is ill-conditioned for high derivatives. It can be done, however, using &lt;a href=&quot;http://en.wikipedia.org/wiki/Chebyshev_polynomials&quot;&gt;Chebyshev polynomials&lt;/a&gt;. In Matlab/Octave (sorry, my Python is not good at all), though, you could do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% We will use the sine function as a test case&#xA;f = @(x) sin( 4*pi*x );&#xA;&#xA;% Set the number of points and the interval&#xA;N = 40;&#xA;a = 0; b = 1;&#xA;&#xA;% Create a Vandermonde-like matrix for the interpolation using the&#xA;% three-term recurrence relation for the Chebyshev polynomials.&#xA;x = cos( pi*[0:N-1]/(N-1) )';&#xA;V = ones( N ); V(:,2) = x;&#xA;for k=3:N, V(:,k) = 2*x.*V(:,k-1) - V(:,k-2); end;&#xA;&#xA;% Compute the Chebyshev coefficients of the interpolation. Note that we&#xA;% map the points x to the interval [a,b]. Note also that the matrix inverse&#xA;% can be either computed explicitly or evaluated using a discrete cosine transform.&#xA;c = V \ f( (a+b)/2 + (b-a)/2*x );&#xA;&#xA;% Compute the derivative: this is a bit trickier and relies on the relationship&#xA;% between Chebyshev polynomials of the first and second kind.&#xA;temp = [ 0 ; 0 ; 2*(N-1:-1:1)'.*c(end:-1:2) ];&#xA;cdiff = zeros( N+1 , 1 );&#xA;cdiff(1:2:end) = cumsum( temp(1:2:end) );&#xA;cdiff(2:2:end) = cumsum( temp(2:2:end) );&#xA;cdiff(end) = 0.5*cdiff(end);&#xA;cdiff = cdiff(end:-1:3);&#xA;&#xA;% Evaluate the derivative fp at the nodes x. This is useful if you want&#xA;% to use Barycentric Interpolation to evaluate it anywhere in the interval.&#xA;fp = V(:,1:n-1) * cdiff;&#xA;&#xA;% Evaluate the polynomial and its derivative at a set of points and plot them.&#xA;xx = linspace(-1,1,200)';&#xA;Vxx = ones( length(xx) , N ); Vxx(:,2) = xx;&#xA;for k=3:N, Vxx(:,k) = 2*xx.*Vxx(:,k-1) - Vxx(:,k-2); end;&#xA;plot( (a+b)/2 + (b-a)/2*xx , [ Vxx*c , Vxx(:,1:N-1)*cdiff ] );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code to compute the derivative can be re-applied several times to compute higher derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use Matlab, you may be interested in the &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot;&gt;Chebfun&lt;/a&gt; project, which does most of this automatically and from which parts of the code example above were taken. Chebfun can create an interpolant from literally any function, e.g. continuous, discontinuous, with singularities, etc... and then compute its integral, derivatives, use it to solve ODEs, etc...&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-02-10T13:57:49.927" LastActivityDate="2012-02-10T13:57:49.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1263" PostTypeId="2" ParentId="229" CreationDate="2012-02-10T15:09:22.333" Score="2" Body="&lt;p&gt;There is a probabilistic version of Hough transform (PHT) that is faster. As described by Bradski &amp;amp; Kaehler in their OpenCV book:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The idea is that the peak is going to be high enough anyhow, then&#xA;  hitting it only a fraction of time will be enough to find it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The OpenCV library presents &lt;a href=&quot;http://goo.gl/Cpj1A&quot; rel=&quot;nofollow&quot;&gt;an implementation&lt;/a&gt; for the PHT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other alternatives. It's not difficult to create a &lt;em&gt;distributed version&lt;/em&gt; of the Hough transform. Just break your point set into smaller chunks and use the MapReduce framework to sum up all the accumulators. Other idea is to perform a &lt;em&gt;coarse&lt;/em&gt; version of Hough transform using a parameter space with low resolution. Pick your best candidates and run a finer iteration using a parameter space presenting a higher resolution. &lt;em&gt;Maybe&lt;/em&gt; this is the idea behind the Gandalf's FHT.&lt;/p&gt;&#xA;" OwnerUserId="599" LastActivityDate="2012-02-10T15:09:22.333" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1264" PostTypeId="1" AcceptedAnswerId="1360" CreationDate="2012-02-10T20:09:21.473" Score="8" ViewCount="831" Body="&lt;p&gt;I'm playing with simple implementations of algorithms to find minimum vertex cover/maximum cardinality matching in bipartite graphs. However, I seem to have trouble googling for some test data sets that I could use to verify, if I got that one right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easy to devise simple examples that one could check on paper, but I'd hope for some larger/tricky benchmarks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking because I do remember from my research that you can find such benchmark data for machine learning problems, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers?&lt;/p&gt;&#xA;" OwnerUserId="953" LastActivityDate="2012-02-18T22:04:41.930" Title="Where to find data sets for testing minimum vertex cover algorithm for bipartite graphs?" Tags="|data-sets|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1265" PostTypeId="1" CreationDate="2012-02-11T08:50:16.043" Score="2" ViewCount="5258" Body="&lt;p&gt;I'm trying to figure out how to draw a wave equation progress in a 2D graph with Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this piece of code which effectively draw a 2D wave placing a droplet in the middle of the graph (I almost fully commented it to simplify things) and then letting it expanding till the border, then bouncing back (how can this code do that? I don't see any code taking the graph dimensions into account) and forth&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code should be pretty simple to someone who knows how a wave equation work, but to me.. it's just dark.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the C1 constant? And how does the &quot;un&quot; line succeed into evolving the wave with the convolution? I can't find theory about that nor anything that explains this to me&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c=1; % speed of wave;&#xA;dx=1; % space step;&#xA;dt=0.05; % time step;&#xA;&#xA;&#xA;szx=200;&#xA;szy=200; % size of the drawing area&#xA;&#xA;tm=3000; % time&#xA;k=0.002; % decay factor&#xA;dsz=3; % droplet size&#xA;da=0.07; % droplet amplitude&#xA;&#xA;&#xA;x=0:dx:(szx-dx); % Generate a x vector from 0 to 200 with step 1&#xA;y=0:dx:(szy-dx); % Generate a y vector from 0 to 200 with step 1&#xA;t=0:dt:tm; % time, generate a time vector from 0 to 3000 with step 0.05&#xA;&#xA;[X,Y] = meshgrid(x,y); % Create base planes for the graph&#xA;&#xA;Lx=length(x);&#xA;Ly=length(y);&#xA;&#xA;u=zeros(Ly,Lx); % initial value (a null matrix for u)&#xA;uo=u; % previous = curent =&amp;gt; velocties =0&#xA;&#xA;% Initialize the drawing and the axis&#xA;close all;&#xA;hf=figure;&#xA;ha=axes;&#xA;hi=imagesc(x,y,u);&#xA;set(ha,'clim',[-1 1]); % set colors&#xA;&#xA;&#xA;D=[0 1 0; 1 -4 1; 0 1 0]; % 2d laplace operator, this will be used in the convolution (probably a laplace 2d operator substitute?)&#xA;&#xA; % Kdt = decay factor * dt (timestep), that's how much the wave decays per time step&#xA; kdt=k*dt;&#xA; % !!!! this c1 is constant, but I can't figure out what it is&#xA; c1=dt^2*c^2/dx^2;&#xA;&#xA; % droplet as gaussian, the initial droplet is nothing else but a gaussian in a 2D matrix&#xA; xd=-2*dsz:dx:2*dsz;&#xA; yd=-2*dsz:dx:2*dsz;&#xA; [Xd,Yd] = meshgrid(xd,yd);&#xA; Zd=-da*exp(-(Xd/dsz).^2-(Yd/dsz).^2);&#xA;&#xA;&#xA; % Used to count droplets&#xA; One_single_droplet = 0;&#xA;&#xA;&#xA; for tt=t&#xA;     % !!!!!! Calculate wave equation evolution -&amp;gt; this is exactly where I can't figure out what does this mean&#xA;    un=(2-kdt)*u+(kdt-1)*uo+c1*conv2(u,D,'same');&#xA;&#xA;    uo=u; % current become old&#xA;    u=un; % new become current&#xA;&#xA;       % Draw the wave updating the graph matrix values (u)&#xA;       set(hi,'Cdata',u);&#xA;       drawnow;&#xA;&#xA;    % droplets, just one&#xA;    if One_single_droplet == 0&#xA;        x0d= 100;&#xA;        y0d= 100; % droplet center&#xA;&#xA;        % Place the droplet centered on x0d and y0d: adds to the u matrix (the wave function) the gaussian 2d droplet&#xA;        u(y0d-2*dsz:y0d+2*dsz,x0d-2*dsz:x0d+2*dsz)=...&#xA;            u(y0d-2*dsz:y0d+2*dsz,x0d-2*dsz:x0d+2*dsz)+Zd;&#xA;&#xA;        One_single_droplet = 1;&#xA;    end&#xA;&#xA; end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="851" LastActivityDate="2016-04-27T12:34:43.573" Title="Can't understand a simple wave equation matlab code" Tags="|pde|hyperbolic-pde|matlab|finite-difference|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1266" PostTypeId="2" ParentId="376" CreationDate="2012-02-11T10:23:39.533" Score="2" Body="&lt;p&gt;OpenCL can be used, however, there is a lack of infrastracture, e.g. important mature standard math libraries with tuned de facto standard linear algebra components and to some extent good profiling tools, albeit the latter problem has improved significantly for GPUs. This is available in CUDA as of today and can be contributed to a part of Nvidia's success with this programming model. However, OpenCL seems to be catching up (slowly). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today, as a starting point for GPU programming CUDA is fine, and if it is needed, there is nothing that prevents using OpenCL as an alternative, e.g. to make the code more portable. Essentially, the same kernel code can be used both in CUDA and OpenCL so it should not be a major problem to go from CUDA to OpenCL. This is confirmed by own experiences testing this out. From a performance perspective, the same optimization techniques can be used and for trivial concurrent code compilers should do the job fine (e.g. loop unrolling, etc.).&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-02-11T10:23:39.533" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1267" PostTypeId="2" ParentId="1265" CreationDate="2012-02-11T11:41:17.533" Score="4" Body="&lt;p&gt;It is appears to be a standard finite difference discretization via method of lines of the wave equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{tt} + c^2 \nabla^2u = 0, \quad (x,y)\in[0,L_x]\times[0,L_y], \quad t\geq0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Standard finite difference approximations can be used to formulate a discrete problem by introducing the formally second-order accurate approximations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{tt}\approx \frac{u^{n+1}_{ij}-2 u_{ij}^n + u^{n-1}_{ij}}{\Delta t^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^2u = u_{xx} + u_{yy} \approx \frac{u_{i+1,j}^n - 2u_{ij}^n + u_{i-1,j}}{\Delta x^2} + \frac{u_{i,j+1}^n - 2u_{ij}^n + u_{i,j-1}}{\Delta y^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{ij}^n = u(x_i,y_j,t_n)$ expressed the small-amplitude surface elevation at node $(x_i,y_j)$ at time $t_n$. If a uniform grid $x_i=hi$, $i=0,1,...,N_x$ and $y_j=jh$, $j=0,1,...,N_y$ with $h=\tfrac{L_x}{N_x}=\tfrac{L_y}{N_y}$ is introduced and with time defined as $t_n = n\Delta t$, $n=0,1,...$, then these approximations can be substituted for the continuous derivatives and then rewritten for the discrete equation to be expressed in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{ij}^{n+1} = 2u_{ij}^n - u_{ij}^{n-1} + c^2\frac{\Delta t^2}{h^2}(u_{i-1,j}^n + u_{i+1,j}^n - 4 u_{ij}^n + u_{i,j-1}^n + u_{i,j+1}^n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which provides the basis for locally (explicitly) updating the solution for each grid point in the spatial domain. Remark, that the update requires storage of solutions at two previous steps in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a discrete scheme for the wave equation without the damping term of the form $−ku_t$ included on the right hand side. The discretization of $u_t$ can be done explicitly using a standard backward difference operations.&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="304" LastEditDate="2012-02-11T22:38:30.200" LastActivityDate="2012-02-11T22:38:30.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1268" PostTypeId="2" ParentId="1265" CreationDate="2012-02-11T12:22:11.750" Score="3" Body="&lt;p&gt;We could start with the equations and explain the code, but I'll go the other way because it's a useful methodology for figuring out what code is doing. First let's consider&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conv2(u,D,'same');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which applies the difference stencil &lt;code&gt;D&lt;/code&gt; to &lt;code&gt;u&lt;/code&gt;. If you are not familiar with &lt;code&gt;conv2()&lt;/code&gt;, the operation (expressed using $0$-based indexing for &lt;code&gt;D&lt;/code&gt;) is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u\star D = \sum_{k=-1}^1 \ \sum_{l=-1}^1 u_{i+k,j+l} D_{k,l} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For constant stencils, this representation is much more efficient than storing a sparse matrix or writing your own code to apply the difference stencil. You can experiment easily in 1D to understand &lt;code&gt;conv&lt;/code&gt; and &lt;code&gt;conv2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; conv([0,0,1,0,0], [1,2,3], 'same')&#xA;ans =&#xA;&#xA;   0   1   2   3   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This difference stencil is multiplied by &lt;code&gt;c1 = c^2*dt^2/dx^2&lt;/code&gt; which happens to be the CFL number squared. For the moment, we can observe that &lt;code&gt;conv2(u,D,'same')/dx^2&lt;/code&gt; is the centered difference approximation to the Laplacian $\nabla^2 u$. Let's call this sub-expression &lt;code&gt;Lap_u&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for the rest of the expression&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;un = (2-kdt)*u + (kdt-1)*uo + c^2*dt^2*Lap_u;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Let's abuse syntax by moving the part not depending on &lt;code&gt;kdt = k*dt&lt;/code&gt; to the left side&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(un - 2*u + uo)/dt^2 = -k*(u-uo)/dt + c^2 * Lap_u;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The left side is a discrete centered difference approximation to $u_{tt}$ and &lt;code&gt;(u-uo)/dt&lt;/code&gt; is a first-order approximation to $u_t$, so the overall expression is approximating&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_{tt} = -k u_t + c^2 \nabla^2 u . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Leapfrog_method&quot; rel=&quot;nofollow&quot;&gt;Leapfrog method&lt;/a&gt; for the wave propagation term and &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler_integration&quot; rel=&quot;nofollow&quot;&gt;forward Euler&lt;/a&gt; for diffusion. This becomes more clear by writing the equation as a first order system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align}&#xD;&#xA;u_t &amp;amp;= v \\&#xD;&#xA;v_t &amp;amp;= -k v + c^2 \nabla^2 u&#xD;&#xA;\end{align}&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the discrete scheme at time index $n$ is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align}&#xD;&#xA;\frac{u^n - u^{n-1}}{\Delta t} &amp;amp;= v^{n-1/2} \\&#xD;&#xA;\frac{v^{n+1/2} - v^{n - 1/2}}{\Delta t} &amp;amp;= -k v^{n - 1/2} + c^2 \nabla^2 u^{n} .&#xD;&#xA;\end{align}&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Leapfrog method is &lt;em&gt;symplectic&lt;/em&gt; (energy conserving in this case), but the even and odd time levels $n$ are decoupled, so some decay (or other filtering) is necessary to prevent the solution from oscillating in time. You should be able to observe this in the code by taking $k \to 0$.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-11T12:22:11.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1270" PostTypeId="2" ParentId="1216" CreationDate="2012-02-11T16:55:39.867" Score="17" Body="&lt;h2&gt;Forming the Schur complement&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Suppose that you have permuted and partitioned your matrix into the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A=\left(\begin{array}{cc}A_{11} &amp;amp; A_{12} \\ A_{21} &amp;amp; A_{22}\end{array}\right),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;such that $A_{22}$ contains your degrees of freedom of interest and is much smaller than $A_{11}$, then one can form the Schur complement&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ S_{22} := A_{22} - A_{21} A_{11}^{-1} A_{12},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;either through a partial right-looking LU factorization or the explicit formula, and then $S_{22}$ can be understood in the following sense:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S_{22} x = y \;\;\rightarrow\;\; \left(\begin{array}{cc}A_{11} &amp;amp; A_{12}\\ A_{21} &amp;amp; A_{22}\end{array}\right) \left(\begin{array}{c}\star\\ x\end{array}\right)=\left(\begin{array}{c}0\\ y\end{array}\right),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\star$ represents the 'uninteresting' portion of the solution. Thus, provided a right-hand side which is only nonzero in the degrees of freedom of the Schur complement $S_{22}$, we need only solve against $S_{22}$ in order to get the portion of the solution corresponding to those degrees of freedom.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Computational complexity in unstructured dense case&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Setting $N$ to the height of $A$ and $n$ to the height of $A_{22}$, then the standard method for computing $S_{22}$ is to first factor $L_{11} U_{11} := A_{11}$ (let's ignore pivoting for now) in roughly $2/3 (N-n)^3$ work, then to form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ S_{22} := A_{22} - (A_{21} U_{11}^{-1})(L_{11}^{-1} A_{12}) = A_{22} - A_{21} A_{11}^{-1} A_{12}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using two triangle solves requiring $n(N-n)^2$ work each, and then performing the update to $A_{22}$ in $2n^2 (N-n)$ work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the total work is roughly $2/3 (N-n)^3 + 2n(N-n)^2 + 2n^2 (N-n)$. When $n$ is very small, $N-n \approx N$, so the cost can be seen to be roughly $2/3 N^3$, which is the cost of a full factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The benefit is that, if there is a very large number of right-hand sides to be solved with the same system of equations, then $S_{22}$ could potentially be reused a large number of times, where each solve would only require $2n^2$ work (rather than $2N^2$ work) if $S_{22}$ is factored.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Computational complexity in the (typical) sparse case&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If your sparse system arose from some type of finite-difference or finite-element approximation, then sparse-direct solvers will almost certainly be able to exploit some of the structure; 2d systems can be solved with $O(N^{3/2})$ work and $O(N \log N)$ storage, while 3d systems can be solved with $O(N^2)$ work and $O(N^{4/3})$ storage. The factored systems can then be solved with the same amount of work as the storage requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of bringing up the computational complexities is that, if $n \approx \sqrt{N}$ and you have a 2d system, then since the Schur complement will likely be dense, the solve complexity given the factored Schur complement will be $O(n^2) = O(N)$, which is only missing a logarithmic factor versus solving the full system! In 3d, it requires $O(N)$ work instead of $O(N^{4/3})$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;It is thus important to keep in mind that, in your case where $n=\sqrt{N}$, there will only be significant savings if you're working in several dimensions and have many right-hand sides to solve.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-02-11T16:55:39.867" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1271" PostTypeId="2" ParentId="1216" CreationDate="2012-02-11T21:12:27.847" Score="6" Body="&lt;h2&gt;The model reduction approach&lt;/h2&gt;&#xA;&lt;p&gt;Since Paul asked, I'll talk about what happens if you use projection-based model reduction methods on this problem. Suppose that you could come up with a projector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt; such that the range of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt;, denoted &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}(\mathbf{P})$&lt;/span&gt;, contains the solution to your linear system &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Ax} = \mathbf{b}$&lt;/span&gt;, and has dimension &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of unknowns for which you wish to solve in a linear system.&lt;/p&gt;&#xA;&lt;p&gt;A singular value decomposition of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt; will yield the following partitioned matrix:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{P} = \left[ \begin{array}{cc}\mathbf{V} &amp;amp; * \end{array} \right]\left[\begin{array}{cc}\mathrm{diag}(\mathbf{1}_{k}) &amp;amp; \mathbf{0} \\ \mathbf{0} &amp;amp;  \mathbf{0}\end{array}\right]\left[\begin{array}{c} \mathbf{W}^{T} \\ *\end{array}\right].$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The matrices obscured by stars matter for other things (like estimating error, etc.), but for now, we'll avoid dealing with extraneous details. It follows that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{P} = \mathbf{VW}^{T}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;is a full rank decomposition of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Essentially, you'll solve the system&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{PAx} = \mathbf{Pb}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;in a clever way, because &lt;span class=&quot;math-container&quot;&gt;$\mathbf{V}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt; also have the property that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}^{T}\mathbf{V} = \mathbf{I}$&lt;/span&gt;. Multiplying both sides of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{PAx} = \mathbf{Pb}$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}^{T}$&lt;/span&gt; and letting &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y} = \mathbf{V}\widehat{\mathbf{x}}$&lt;/span&gt; be an approximation for &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; yields&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{W}^{T}\mathbf{A}\widehat{\mathbf{x}} = \mathbf{W}^{T}\mathbf{b}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Solve for &lt;span class=&quot;math-container&quot;&gt;$\widehat{\mathbf{x}}$&lt;/span&gt;, premultiply it by &lt;span class=&quot;math-container&quot;&gt;$\mathbf{V}$&lt;/span&gt;, and you have &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt;, your approximation for &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Why the Schur complement approach is probably better&lt;/h2&gt;&#xA;&lt;p&gt;For starters, you have to pick &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt; somehow. If the solution to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Ax} = \mathbf{b}$&lt;/span&gt; is in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{R}(\mathbf{P})$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y} = \mathbf{x}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; isn't an approximation. Otherwise, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y} \neq \mathbf{x}$&lt;/span&gt;, and you introduce some approximation error. This approach doesn't really leverage at all the structure you mentioned wanting to exploit. If we pick &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt; such that its range is the standard unit basis in the coordinates of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; you want to calculate, the corresponding coordinates of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; will have errors in them. It's not clear how you'd want to pick &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt;. You could use an SVD of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt;, for instance, and select &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt; to be the product of the first &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; left singular vectors of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt; and the adjoint of the first &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; right singular vectors of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt;, assuming that singular vectors are arranged in decreasing order of singular value. This choice of projector would be equivalent to performing proper orthogonal decomposition on &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt;, and it would minimize the L&lt;span class=&quot;math-container&quot;&gt;$_{2}$&lt;/span&gt;-error in the approximate solution.&lt;/p&gt;&#xA;&lt;p&gt;In addition to introducing approximation errors, this approach also introduces three extra matrix multiplies on top of the linear solve of the smaller system and the work needed to calculate &lt;span class=&quot;math-container&quot;&gt;$\mathbf{V}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}$&lt;/span&gt;. &lt;strong&gt;Unless you're solving the same linear system a lot, only changing the right hand side, and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{P}$&lt;/span&gt; is still a &amp;quot;good&amp;quot; projection matrix for all of those systems, those extra costs will probably make solving the reduced system more expensive than solving your original system.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The drawbacks are much like JackPoulson's approach, except that you're not quite leveraging the structure that you mentioned.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-02-11T21:12:27.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1272" PostTypeId="2" ParentId="1216" CreationDate="2012-02-11T21:30:22.500" Score="13" Body="&lt;p&gt;As others have pointed out, this is difficult to do with a direct solver. That said, it isn't that hard to do with iterative solvers. To this end, note that most iterative solvers in one way or another minimize the error with respect to some norm. Oftentimes, this norm is either induced by the matrix itself, but sometimes it is also just the l2 vector norm. But that doesn't have to be the case: you can choose which norm you want to minimize the error (or residual) in, and you could, for example, choose a norm in which you weigh the components you care about with 1 and all others with 1e-12, i.e. for example something like&#xA;  $|| x ||^2  =  \sum_{i=1}^5 x_i^2 + $(1e-24)$\sum_{i=6}^N x_i^2$&#xA;and corresponding scalar product. Then write all steps of the iterative solver with respect&#xA;to this norm and scalar product, and you get an iterative solver that pays significantly&#xA;more attention to the vector elements you care about than to the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question of course is whether you need fewer iterations than with the norm/scalar product that weighs all components equally. But that should indeed be the case: let's&#xA;say you only care about the five first vector elements. Then you should need at most&#xA;five iterations to reduce the error by a factor of 1e12 since five iterations is what&#xA;is needed for the 5x5 system that describes them. That's not a proof but I'm pretty certain that you should indeed get away with a far smaller number of iterations if the weight in the norm (1e-12 above) is smaller than the tolerance with which you want to solve the linear system iteratively.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="549" LastEditDate="2012-03-04T15:07:00.103" LastActivityDate="2012-03-04T15:07:00.103" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1273" PostTypeId="1" CreationDate="2012-02-12T06:13:42.393" Score="9" ViewCount="259" Body="&lt;p&gt;This problem arises from a Bayesian statistical modeling project. In order to compute with my model, I need to perform an integration in which part of the integrand is the &quot;Pólya&quot; or &quot;Dirichlet-Multinomial&quot; Distribution,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(n\mid \alpha) = \frac{(N!)&#xA;  \Gamma(K\alpha)}{\Gamma(\alpha)^K \Gamma ( N + K\alpha)} &#xA;\prod_{i=1}^K \frac{\Gamma(n_i + \alpha)}{ n_i!}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $n_i$ and $N = \sum_{i=1}^K n_i$ are integers, $n = \left(n_1, n_2, \dots, n_K\right)$, and $\alpha &amp;gt; 0$. The integral I wish to compute, $\int_0^\infty (\text{other terms})p(n|\alpha) d\alpha$, works well for small $N$, but the quadrature methods I've attempted (in MATLAB) break down as $N$ becomes large. I haven't tried Monte Carlo; an accurate, fast quadrature method would be very nice for my project. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, the &quot;best&quot; method when $N$ is large is to compute $\log[p(n|\alpha)]$ over a grid in alpha, normalize, and exponentiate. This is inaccurate (I lose essentially all detail about the distribution except its peaks), but at least produces a number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any advice on improving this computation, or pointers to different algorithms/methods or existing software.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I should maybe add that that my evaluation of $p(n|\alpha)$, performed by computing $\log p(n|\alpha)$ using some carefully-written code to compute $\log \Gamma(x)$ for large $x$, does not appear to be causing any problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT 2: Additionally, &quot;large&quot; values would be on the order of $N\sim 10^8$, with the largest $n_i\sim 10^5$, along with many small values of $n_i$. The other terms are numerically well-behaved. As a simplification with roughly the appropriate tail behavior, you could take &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(\text{other terms}) = \exp(-\alpha)$&lt;/p&gt;&#xA;" OwnerUserId="960" LastEditorUserId="127" LastEditDate="2013-05-14T04:31:35.957" LastActivityDate="2023-03-23T03:02:20.290" Title="Suggestions for numerical integral over Pólya Distribution" Tags="|quadrature|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1274" PostTypeId="1" AcceptedAnswerId="1281" CreationDate="2012-02-12T06:54:51.790" Score="8" ViewCount="2466" Body="&lt;p&gt;Normally when I seed a sequential random number generator in C, I use the call  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;srand(time(NULL))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then use  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rand() mod N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to obtain a random integer between 0 and N-1.  However, when I do this in parallel, the calls to time(NULL) are so close to each other that they end up being exactly the same number.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried using a linear congruential random number generator:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{n+1} := a x_n + c\;\;\; (\bmod m) $ for some integers $a,c,$ and $m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that choosing $m=2^k$ for some large integer $k$ produces fast results because the modulus operator can be calculated by truncating digits.  However, I find it difficult to establish seeds that produce random sequences with a large period in parallel.  I know that &#xA;a period length is maximal if   &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;c and m are primes with respect to eachother&lt;/li&gt;&#xA;&lt;li&gt;a-1 is divisible by all prime factors of m&lt;/li&gt;&#xA;&lt;li&gt;if m is a multiple of 4, a-1 must also be a multiple of 4.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(source: &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_congruential_generator#Period_length&quot;&gt;wikipedia&lt;/a&gt;)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do I ensure that all random number streams have this maximal property?  In terms of MPI, how do I incorporate the &lt;code&gt;rank&lt;/code&gt; and &lt;code&gt;size&lt;/code&gt; to produce maximal periods using the linear congruential method?  Would it be easier to use a Lagged Fibonacci or Mersenne Twister to produce longer parallel random streams?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="67" LastEditDate="2012-02-12T16:47:37.687" LastActivityDate="2013-09-03T18:25:46.847" Title="How can I seed a parallel linear congruential pseudo-random number generator for maximal period?" Tags="|parallel-computing|random-number-generation|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1275" PostTypeId="5" CreationDate="2012-02-12T07:31:24.620" Score="0" Body="" OwnerUserId="549" LastEditorUserId="20688" LastEditDate="2019-06-22T22:20:54.400" LastActivityDate="2019-06-22T22:20:54.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1276" PostTypeId="4" CreationDate="2012-02-12T07:31:24.620" Score="0" Body="Questions on the algorithms for and uses of (pseudo)random number generators in scientific computing." OwnerUserId="549" LastEditorUserId="20688" LastEditDate="2019-06-22T22:20:54.400" LastActivityDate="2019-06-22T22:20:54.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1277" PostTypeId="2" ParentId="1258" CreationDate="2012-02-12T08:31:52.680" Score="2" Body="&lt;p&gt;In Scipy, if you try to compute the n-th order derivative of a k-th order spline, where n &gt; k, then you get a &lt;code&gt;ValueError&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ValueError: 0&amp;lt;=der=5&amp;lt;=k=4 must hold&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You could write something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def spline_der(spline, x, nu=0):&#xA;    sd = 0&#xA;    try:&#xA;        sd = spline(x, nu=nu)&#xA;    except ValueError:&#xA;        pass&#xA;    return sd&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;P.S.: As you can see, instead of using &lt;code&gt;__call__&lt;/code&gt;, you can simply write&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;spline(e0, nu=n)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="782" LastActivityDate="2012-02-12T08:31:52.680" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1278" PostTypeId="1" CreationDate="2012-02-12T09:44:05.833" Score="7" ViewCount="1023" Body="&lt;p&gt;As a part of other work I need to solve relatively large (~1E5x1E5) and sparse (~100 non-zero elements in each raw in few blocks) hermitian eigensystems. Usually only few eigenvalues+vectors are needed, but with high precision. Currently I am using Arpack (Arnoldi with shift-inverse when precision is preferable or spectrum folding when size is important). As an option plan to use TRLan (thick restart Lanczos) and try Chebyshev filtering instead of spectrum folding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably, newer methods exist for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="949" LastEditorUserId="123" LastEditDate="2012-02-12T11:09:14.763" LastActivityDate="2012-02-12T14:03:28.197" Title="Sparse hermitian eigensystems: are there better techniques than Arpack or TRLan?" Tags="|linear-algebra|eigensystem|matrix|sparse-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1279" PostTypeId="2" ParentId="1278" CreationDate="2012-02-12T13:54:39.643" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CCwQFjAA&amp;amp;url=http://www.grycap.upv.es/slepc/&amp;amp;ei=WcM3T6PzMMfB0QXyl_3AAg&amp;amp;usg=AFQjCNGQZXV4Mvmdtx11OYluVH2UVDFaWQ&amp;amp;sig2=OtLz6x6OZrhlkSON5aUvgg&quot; rel=&quot;nofollow&quot; title=&quot;SLEPc&quot;&gt;SLEPc&lt;/a&gt; is the package I am most familiar with for the scalable solution of sparse eigensystems.  The developers also maintain a &lt;a href=&quot;http://www.grycap.upv.es/slepc/documentation/reports/str6.pdf&quot; rel=&quot;nofollow&quot;&gt;survey of software for solving sparse eigensystems&lt;/a&gt; (last updated in 2009).&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-02-12T14:03:28.197" LastActivityDate="2012-02-12T14:03:28.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1280" PostTypeId="1" AcceptedAnswerId="1359" CreationDate="2012-02-12T15:37:37.127" Score="4" ViewCount="2828" Body="&lt;p&gt;I created a binary matrix using the following loop in c&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;char filename[6];&#xA;filename=sprintf(filename,&quot;file01&quot;);&#xA;moutput=fopen(filename,&quot;wb+&quot;);&#xA;for (i=0; i&amp;lt;2*m+1; i++){&#xA;      for (j=0; j&amp;lt;2*n+1; j++) {&#xA;            r=rmin+(rmax-rmin)*GetRand(N);&#xA;            fwrite(&amp;amp;r, sizeof(double), 1, moutput);&#xA;      }&#xA;}&#xA;fclose(moutput);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where r is a uniformly distributed pseudorandom double precision number between $rmin$ and $rmax$.  I verified that it wrote the values correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I am using &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/mat/examples/tutorials/ex1.c.html&quot; rel=&quot;nofollow&quot;&gt;this tutorial&lt;/a&gt; as a guideline to load &quot;file01&quot; as a binary matrix.  I ran the following script:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;make ex1&#xA;./ex1 -f0 file01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it does not seem to work.  I get an error saying:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0]PETSC ERROR: Unexpected data in file!&#xA;[0]PETSC ERROR: not matrix object in file!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is the problem that I wrote the binary file initially or that I'm using the wrong example to load this particular binary file?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-02-19T08:03:32.090" Title="How do I read a binary file into PETSc as a matrix?" Tags="|petsc|io|" AnswerCount="3" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1281" PostTypeId="2" ParentId="1274" CreationDate="2012-02-12T16:40:01.017" Score="9" Body="&lt;p&gt;The trick is to interleave each process's LCG stream: for $p$ processes, we modify the LCG&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_{n+1} := a x_n + c\;\;\; (\bmod m),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_{n+p} := A_p x_n + C_p\;\;\; (\bmod m),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_p$ and $C_p$ effectively step forward $p$ steps. We can quickly derive them by expanding the original LCG step:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;x_{n+2} = a (a x_n + c) + c\;\;\; (\bmod m) = a^2 x_n + (a+1)c\;\;\; (\bmod m),&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the pattern is that $A_p \equiv a^p \mod m,$ and $C_p$ is the result of, starting with the number $0$, successively multiplying by $a$ and adding $1$ $p$ times, then multiplying by $c$, all $\bmod m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last step is to ensure that each process's $p$-stride LCG does not overlap: simply initialize the process with rank $r$ with $x_r$ and a parallel LCG with individual periods of $N/p$ is ready, where $N$ is the original period and $p$ is the number of processes. If each process's modified LCG is used equally, the full period is recovered in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented this about six months ago (perhaps naively), and you can find the code &lt;a href=&quot;https://bitbucket.org/poulson/parallel-lcg/src&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-02-12T16:40:01.017" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1282" PostTypeId="2" ParentId="1280" CreationDate="2012-02-12T16:40:21.877" Score="1" Body="&lt;p&gt;I've never used PETSc before, but it seems to me that you're just writing the binary file in your own format, rather than what PETSc expects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example of generating a matrix, writing it to a binary file using PETSc, and reading the matrix back in: &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/mat/examples/tests/ex31.c.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-current/src/mat/examples/tests/ex31.c.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="739" LastActivityDate="2012-02-12T16:40:21.877" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1283" PostTypeId="1" AcceptedAnswerId="1507" CreationDate="2012-02-12T16:43:51.700" Score="10" ViewCount="710" Body="&lt;p&gt;I am solving a multiscale problem using the &lt;a href=&quot;http://www.math.princeton.edu/multiscale/review.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heterogeneous Multiscale Method (HMM)&lt;/a&gt;.  Essentially, my particular procedure uses the following iterative process:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve many local matrix systems.&lt;/li&gt;&#xA;&lt;li&gt;Compute a value of interest from the solutions of the local systems.&lt;/li&gt;&#xA;&lt;li&gt;Assemble a global matrix system from the local &quot;values of interest&quot;&lt;/li&gt;&#xA;&lt;li&gt;Solve the global matrix system&lt;/li&gt;&#xA;&lt;li&gt;Use the solution of the global matrix system to form new local matrix systems.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Repeat until some convergence criteria is met.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there are many local (independent) linear systems of equations and multiple systems can fit into local RAM memory, I figure it is best to load multiple &quot;local&quot; systems into each processor and solve each system sequentially (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/1203/what-mapping-strategy-should-i-use-when-solving-many-large-linear-systems-of-equ&quot;&gt;see this posted question&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question regards the best strategy to assemble and solve the global matrix system.  In my particular case, the global matrix system is small enough that it could fit entirely on any processor's RAM memory.  Furthermore, the local &amp;amp; global matrices do not change size between iterations.  So, I foresee one of three possible strategies:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Gather the &quot;values of interest&quot; onto a single processor, and assemble/solve the global matrix system sequentially on one processor.  &lt;/li&gt;&#xA;&lt;li&gt;Copy the values of interest onto every processor, and assemble/solve the same global matrix system sequentially on every processor.&lt;/li&gt;&#xA;&lt;li&gt;Assuming that each processor possesses the &quot;values of interest&quot; necessary to produce contiguous blocks of the global matrix, then we can assemble partitions of the global matrix locally, then solve them together in parallel.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I can see some advantages/disadvantages to each method.  In Method 1, no communication is necessary in the solving phase, but communication to and from the root processor may become a bottleneck (especially at scale).  Method 2 may require more interprocessor communications to assemble the global matrix than the first method, but no communication is needed in the solving phase or in the local matrix assembly stage that follows. Method 3 requires no interprocessor communication for assembly of the local or global matrices, but requires it in the solving phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that each local system is on the order of $10^3$x$10^3$ and there are $10^3$x$10^3$ local matrix systems. Let's further suppose that the global matrix system has size $10^3$x$10^3$.  Under these assumptions, which one the three aforementioned strategies will likely lead to a faster solution of the global system?  Are there other mapping strategies for the global matrix that might work faster per iteration?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-03-04T14:48:59.970" Title="How to assemble and solve a matrix system in parallel from values generated in different processors?" Tags="|parallel-computing|mapping-strategy|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1284" PostTypeId="5" CreationDate="2012-02-12T19:14:04.857" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-12T19:14:04.857" LastActivityDate="2012-02-12T19:14:04.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1285" PostTypeId="4" CreationDate="2012-02-12T19:14:04.857" Score="0" Body="Formerly FEMLAB, COMSOL is a finite element analysis software package for various engineering and physics applications, with special emphasis on coupled phenomena (multiphysics)." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-12T19:21:53.780" LastActivityDate="2012-02-12T19:21:53.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1286" PostTypeId="5" CreationDate="2012-02-12T19:18:37.657" Score="0" Body="&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Condition_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;condition number&lt;/a&gt; relates the error/deviation in some function/transformation to the error/deviation in its inputs.&lt;/p&gt;&#xA;&lt;p&gt;A problem with low condition number means low sensitivity to small changes in input values, making the problem &amp;quot;well-conditioned&amp;quot;. A problem with a large condition number has high sensitivity to small changes in input values, making the problem &amp;quot;ill-conditioned&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="23791" LastEditDate="2021-11-11T17:20:24.670" LastActivityDate="2021-11-11T17:20:24.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1287" PostTypeId="4" CreationDate="2012-02-12T19:18:37.657" Score="0" Body="For questions about calculating or interpreting condition numbers. Most commonly refers to the condition number of a matrix, but can be applied generally to analyzing the error in a function based on error in the input." OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-11T17:20:13.793" LastActivityDate="2021-11-11T17:20:13.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1288" PostTypeId="5" CreationDate="2012-02-12T19:20:10.387" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-12T19:20:10.387" LastActivityDate="2012-02-12T19:20:10.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1289" PostTypeId="4" CreationDate="2012-02-12T19:20:10.387" Score="0" Body="A non-intersecting path through a point set with two components: one path moves monotonically left to right and another path moving monotonically right to left." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-13T06:53:20.007" LastActivityDate="2012-02-13T06:53:20.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1290" PostTypeId="5" CreationDate="2012-02-12T19:22:05.890" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-12T19:22:05.890" LastActivityDate="2012-02-12T19:22:05.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1291" PostTypeId="4" CreationDate="2012-02-12T19:22:05.890" Score="0" Body="An algorithm design which saves solutions to subproblems and uses them to solve other (possibly larger) subproblems." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-12T19:43:00.713" LastActivityDate="2012-02-12T19:43:00.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1292" PostTypeId="1" CreationDate="2012-02-12T23:14:42.313" Score="3" ViewCount="68" Body="&lt;p&gt;Are there any existing software toolkits, libraries, frameworks or whatever for studying the quality of pseudorandom number generators that allow one to add a known amount of imperfection - e.g deliberate higher-order correlations - then obtain statistics of various RNGs whose outputs are so modified?    &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2012-08-11T20:26:24.287" Title="Adding deliberate imperfection to RNG output - toolkits?" Tags="|statistics|random-number-generation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1293" PostTypeId="2" ParentId="830" CreationDate="2012-02-12T23:27:04.647" Score="3" Body="&lt;p&gt;I would suggest chapter 8 of &quot;A Multigrid Tutorial&quot; (2Ed) by W. L. Briggs, V. E. Henson and S. F. McCormick. It gives a general idea on some important concepts like algebraic smoothness and strong dependence. It also explain how to define the interpolation operator (coarse-grid operator too) and how to select coarse grid. &lt;/p&gt;&#xA;" OwnerUserId="484" LastEditorUserId="-1" LastEditDate="2012-02-13T14:34:21.360" LastActivityDate="2012-02-13T14:34:21.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1294" PostTypeId="2" ParentId="376" CreationDate="2012-02-12T23:49:33.383" Score="26" Body="&lt;p&gt;First of all I wish to thanks Aron Ahmadia for pointing me to this thread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for OpenCL in scientific code: OpenCL is meant to be a low-level API, thus it is crucial to wrap this functionality in some way in order to reach a reasonable productivity. Moreover, as soon as several compute kernels are involved, code can get VERY dirty if OpenCL kernel and memory handles need to be heavily passed around within an application. I don't know OCLTools, thus I can't say whether they are useful in this regard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for ViennaCL: I'm the head of ViennaCL, so I've worked recently with the library. :-)&#xA;In the following I'll treat the request for a comparison with cusp in a slightly larger scope, namely ViennaCL versus the CUDA-based math libraries cusp and &lt;a href=&quot;http://icl.cs.utk.edu/magma/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAGMA&lt;/a&gt;. Only the present state is considered, even though there is a lot of ongoing development (at least on our side). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Functionality&lt;/strong&gt;. MAGMA provides BLAS-functionality for dense matrices via the usual function interfaces. Most of this functionality is also provided with ViennaCL 1.2.0 using operator overloads and other syntactic sugar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same three iterative solvers (CG, BiCGStab, GMRES) are provided with cusp and ViennaCL. The set of preconditioners differs notably: Cusp provides diagonal, SA-AMG and various Bridson preconditioners. ViennaCL offers incomplete LU factorizations, diagonal preconditioners, and recently various AMG flavors and Sparse Approximate Inverse preconditioners. To my knowledge, all cusp preconditioners run entirely on the GPU, while ViennaCL relies particularly during the setup phase on CPU-based computations. Currently, the number of sparse matrix formats is larger in cusp: COO, CSR, DIA, ELL, HYB, while ViennaCL 1.2.0 provides COO and CSR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of additional features provided with ViennaCL, which are not part of either MAGMA or cusp: Structured matrix types (Circulant, Hankel, etc.), fast Fourier transform, reordering algorithms (e.g. Cuthill-McKee) and wrappers for linear algebra types from other libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Performance.&lt;/strong&gt; The larger set of features and hardware support in ViennaCL typically comes at the cost of lower performance when compared to CUDA-based implementations. This is also partly due to the fact that CUDA is tailored to the architecture of NVIDIA products, while OpenCL represents in some sense a reasonable compromise between different many-core architectures. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, ViennaCL is at present slower than MAGMA, particularly at BLAS level 3. The reasons is the different focus of ViennaCL (sparse instead of dense linear algebra) and thus the higher degree of optimization in MAGMA. Particularly BLAS level 3 operations are currently considerably faster in MAGMA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, cusp provides slightly better overall performance in general. However, since sparse matrix operations are usually memory bandwidth limited, differences are considerably smaller and often negligible compared to data setup and the like. The choice of the preconditioner and its parameters usually has a higher impact on the overall execution time than any performance differences in sparse matrix-vector multiplications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Portability&lt;/strong&gt;. As for hardware portability, ViennaCL can use CPUs and GPUs from all major vendors thanks to OpenCL. In contrast, cusp and MAGMA rely on a suitable NVIDIA GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ViennaCL is header-only, can be compiled on a wide range of C++ compilers and only needs to be linked with the OpenCL library if GPU-support is required. In principle, the generic algorithms in ViennaCL can also be used without any OpenCL linkage, while cusp and MAGMA require the NVIDIA compiler for compilation and the CUDA library on the target system for execution. MAGMA also requires a BLAS library, which can sometimes be a bit of a hassle to find or install on a new system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;API&lt;/strong&gt;. MAGMA provides BLAS-style function interfaces for BLAS functionality. The C++ interface of cusp also uses some functions from BLAS, but no operator overloads.&#xA;Since most interfaces in ViennaCL are intentionally similar to Boost.uBLAS and feature syntactic sugar such as operator overloads, ViennaCL is also intended to be used like Boost.uBLAS. Thus, in addition to just calling a predefined set of operations and algorithms, our intention is to make a transition from purely CPU-based execution to GPU code as simple as possible, even if non-standard algorithms are to be used. In the case that a dedicated OpenCL kernel is required, there is also a framework for integrating your own OpenCL kernels in ViennaCL. Thus, ViennaCL aims a lot more towards &lt;strong&gt;high productivity in the sense that the time required for implementing new algorithms on the GPU is minimized&lt;/strong&gt;. These savings can significantly outweigh any performance penalty (if any) compared to cusp and MAGMA. (It has also be mentioned in the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/206/is-it-worthwhile-to-write-unit-tests-for-scientific-research-codes&quot;&gt;thread on unit testing&lt;/a&gt; that code development time is a precious resource in science.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are certainly a number of ideological issues (e.g. CUDA vs. OpenCL, BLAS-interface vs. operator overloads) throughout my comparison, but their discussion is beyond the scope of the initial question.&lt;/p&gt;&#xA;" OwnerUserId="966" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-02-12T23:49:33.383" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1295" PostTypeId="2" ParentId="1274" CreationDate="2012-02-13T00:12:41.100" Score="6" Body="&lt;p&gt;There's a very nice tutorial overview paper by Katzgrabber, &lt;a href=&quot;http://arxiv.org/abs/1005.4117&quot;&gt;Random Numbers In Scientific Computing: An Introduction&lt;/a&gt;, which I point people to who want to be a user of PRNGs for scientific computation.   Linear congruential generators are fast, but that's about all they have going for them; they have short periods, and they can very easily go wrong; perfectly reasonable looking combinations of a, c, and m can end up with horrifically correlated outputs, even if you satisfy the usual requirements between a, c, and m.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Worse, in one common case where m is a power of two (so the mod operation is fast), the lower order bits have a much shorter period than the sequence as a whole, so if you're doing rand() % N, you have an even shorter period than you would expect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a general rule of thumb, lagged-fibbonacci, MT, and WELL generators have much better properties and they're still pretty fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of seeding in parallel, Jack Poulson's method is nice because it gives a well-defined sequence of numbers divided evenly between the processors.   If that doesn't matter, you can do anything reasonable to seed the different PRNGs; the same paper suggests something lots of people have come up with independantly, hashing the PID or MPI task number with the time.   The particular formula suggested there is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;long seedgen(void)  {&#xA;    long s, seed, pid;&#xA;&#xA;    pid = getpid();&#xA;    s = time ( &amp;amp;seconds );&#xA;&#xA;    seed = abs(((s*181)*((pid-83)*359))%104729); &#xA;    return seed;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have no particular opinions about that specific implementation, but the general approach is certainly reasonable.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2012-02-13T00:12:41.100" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1296" PostTypeId="1" AcceptedAnswerId="5172" CreationDate="2012-02-13T03:33:06.667" Score="5" ViewCount="539" Body="&lt;p&gt;I'm using the Crank-Nicholson method to solve the time-dependent Schrödinger equation with the split-operator method. I'm getting some weird results that are probably the result of a bug somewhere in my code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just in case, I thought I should probably check to see if the method I'm using is unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does the split-operator method change the stability properties of the Crank-Nicolson method?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If so, how?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2013-02-01T21:52:46.997" LastActivityDate="2013-02-01T21:52:46.997" Title="Does ADI/Split-operator change the stability properties of the Crank-Nicholson method?" Tags="|pde|implicit-methods|parabolic-pde|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1297" PostTypeId="1" AcceptedAnswerId="1299" CreationDate="2012-02-13T11:18:38.747" Score="11" ViewCount="1482" Body="&lt;p&gt;In the Hartree-Fock self-consistent field method of solving the time-independent electronic Schroedinger equation, we seek to minimize the ground state energy, $E_{0}$, of a system of electrons in an external field with respect to the choice of spin orbitals, $\{\chi_{i}\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We do this by iteratively solving the 1-electron Hartree-Fock equations,&#xA;$$\hat{f}_{i}\chi(\mathbf{x}_{i})=\varepsilon\chi(\mathbf{x}_{i})$$&#xA;where $\mathbf{x}_{i}$ is the spin/spatial coordinate of electron $i$, $\varepsilon$ is the orbital eigenvalue and $\hat{f}_{i}$ is the Fock operator (a 1-electron operator), with the form&#xA;$$\hat{f}_{i} = -\frac{1}{2}\nabla^{2}_{i}-\sum_{A=1}^{M}\frac{Z_{A}}{r_{iA}}+V^{\mathrm{HF}}_{i}$$&#xA;(the summation runs over nuclei, here, with $Z_{A}$ being the nuclear charge on nucleus A and $r_{iA}$ being the distance between electron $i$ and nucleus $A$). $V^{\mathrm{HF}}_{i}$ is the average potential felt by electron $i$ due to all the other electrons in the system. Since $V_{i}^{\mathrm{HF}}$ is dependent on the spin orbitals, $\chi_{j}$, of the other electrons, we can say that the Fock operator is dependent on it's eigenfunctions. In &quot;Modern Quantum Chemistry&quot; by A. Szabo and N. Ostlund, pp. 54 (the first edition) they write that &lt;em&gt;&quot;the Hartree-Fock equation (2.52) is nonlinear and must be solved iteratively&quot;&lt;/em&gt;. I have studied the details of this iterative solution as part of my research, but for this question I think they are unimportant, except to state the basic structure of the method, which is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Make an initial guess of the spin-orbitals, $\{\chi_{i}\}$ and calculate $V_{i}^{\mathrm{HF}}$.&lt;/li&gt;&#xA;&lt;li&gt;Solve the eigenvalue equation above for these spin orbitals and obtain new spin-orbitals.&lt;/li&gt;&#xA;&lt;li&gt;Repeat the process with your new spin orbitals until self-consistency is reached.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In this case, self-consistency is achieved when the spin-orbitals which are used to make $V_{i}^{\mathrm{HF}}$ are the same as those obtained on solving the eigenvalue equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this: how can we know that this convergence will occur? Why do the eigenfunctions of the successive iterative solutions in some sense &quot;improve&quot; towards converged case? Is it not possible that the solution could diverge? I don't see how this is prevented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a further question, I would be interested to know why the the converged eigenfunctions (spin orbitals) give the best (i.e. lowest) ground state energy. It seems to me that the iterative solution of the equation somehow has convergence and energy minimization &quot;built-in&quot;. Perhaps there is some constraint built into the equations which ensures this convergence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cross-posted from the Physics Stack Exchange: &lt;a href=&quot;https://physics.stackexchange.com/q/20703/why-does-iteratively-solving-the-hartree-fock-equations-result-in-convergence&quot;&gt;https://physics.stackexchange.com/q/20703/why-does-iteratively-solving-the-hartree-fock-equations-result-in-convergence&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="932" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2013-12-28T22:05:43.657" Title="Why does iteratively solving the Hartree-Fock equations result in convergence?" Tags="|computational-chemistry|iterative-method|convergence|hartree-fock|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1298" PostTypeId="2" ParentId="1292" CreationDate="2012-02-13T12:46:35.393" Score="2" Body="&lt;p&gt;Did you have a look at &lt;a href=&quot;http://www.phy.duke.edu/~rgb/General/dieharder.php&quot; rel=&quot;nofollow&quot;&gt;dieharder&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dieharder is a random number generator (rng) testing suite. It is intended to test generators, not files of possibly random numbers as the latter is a fallacious view of what it means to be random. Is the number 7 random? If it is generated by a random process, it might be. If it is made up to serve the purpose of some argument (like this one) it is not. Perfect random number generators produce &quot;unlikely&quot; sequences of random numbers -- at exactly the right average rate. Testing a rng is therefore quite subtle.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="414" LastEditorUserId="123" LastEditDate="2012-08-11T20:26:24.287" LastActivityDate="2012-08-11T20:26:24.287" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1299" PostTypeId="2" ParentId="1297" CreationDate="2012-02-13T12:54:45.213" Score="8" Body="&lt;p&gt;The Hartree-Fock equations are the result of performing constrained &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;noreferrer&quot;&gt;Newton-Raphson minimization&lt;/a&gt; of the energy with respect to the parameter space of Slater determinants (I don't have my copy of Szabo-Ostlund at hand, but I believe this is pointed out in the derivation). Hence, HF-SCF will converge if your starting guess is in a convex region around a minimum. Elsewhere, it may or may not converge. SCF convergence fails all the time.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="932" LastEditDate="2012-02-14T13:22:23.177" LastActivityDate="2012-02-14T13:22:23.177" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1300" PostTypeId="1" CreationDate="2012-02-13T13:51:54.553" Score="11" ViewCount="4129" Body="&lt;p&gt;I would like to understand more details about the implementation of Algebraic Multigrid Methods (AMG). I have been reading &quot;A Multigrid Tutorial&quot;, which is quite good and explain all the details of the interpolation, coarse-grid operator and coarse grid selection for AMG. However, I think, there is nothing like playing around and reading a code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I would like to ask if anyone knows any classical AMG &quot;example code&quot;, like the geometric multigrid FORTRAN code available at the end of &quot;Multigrid Methods&quot; (SIAM) by S. F. McCormick. It is quite hard to get a high quality and production code like BoomerAMG to learn more about the method.&lt;/p&gt;&#xA;" OwnerUserId="484" LastEditorUserId="484" LastEditDate="2012-02-14T02:00:30.000" LastActivityDate="2013-08-24T15:58:48.387" Title="Algebraic Multigrid Code" Tags="|software|multigrid|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1301" PostTypeId="2" ParentId="1300" CreationDate="2012-02-13T14:01:49.833" Score="10" Body="&lt;p&gt;BoomerAMG is a part of the &lt;a href=&quot;https://computation.llnl.gov/casc/hypre/software.html&quot;&gt;Hypre&lt;/a&gt; package, which is dead simple to acquire.  A much less complex code if you're starting out looking at these methods might be &lt;a href=&quot;http://code.google.com/p/pyamg/&quot;&gt;PyAMG&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2012-02-13T14:01:49.833" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1302" PostTypeId="2" ParentId="1300" CreationDate="2012-02-13T14:32:19.630" Score="3" Body="&lt;p&gt;I highly recommend &lt;a href=&quot;http://www.kfunigraz.ac.at/imawww/borzi/mgintro.pdf&quot; rel=&quot;nofollow&quot;&gt;Alfio Borizi's&lt;/a&gt; introduction to algebraic multigrid method.  There is a sample fortran 77 code in appendix A.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-02-13T14:32:19.630" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1303" PostTypeId="2" ParentId="1297" CreationDate="2012-02-13T16:55:04.850" Score="5" Body="&lt;p&gt;Density functional theory (DFT) also uses a one-particle approach similar to Hartree-Fock, although the effective potential is a little more involved. To achieve a global minimum, the problem is approached as a non-linear fixed point problem which, as &lt;a href=&quot;https://scicomp.stackexchange.com/a/1299/85&quot;&gt;Deathbreath said&lt;/a&gt;, can be solved via a constrained &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Newton-Raphson minimization&lt;/a&gt;. A common approach in the DFT community is to use &lt;a href=&quot;http://en.wikipedia.org/wiki/Broyden%27s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Broyden's Method&lt;/a&gt; which if organized correctly (&lt;a href=&quot;http://iopscience.iop.org/0305-4470/17/6/002&quot; rel=&quot;nofollow noreferrer&quot;&gt;J Phys A &lt;strong&gt;17&lt;/strong&gt; (1984) L317&lt;/a&gt;) requires only two vectors: the current input and output. (See &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1441939547&quot; rel=&quot;nofollow noreferrer&quot;&gt;Singh and Nordstrom&lt;/a&gt;, p. 91-92, for a quick overview of this method, or &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521534402&quot; rel=&quot;nofollow noreferrer&quot;&gt;Martin&lt;/a&gt;, Appendix L, for a more complete overview of related techniques.) A more recent technique used in &lt;a href=&quot;http://www.wien2k.at/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wien2k&lt;/a&gt; attempts to overcome convergence difficulties with the Broyden method by employing a multi-secant method.(&lt;a href=&quot;http://prb.aps.org/abstract/PRB/v78/i7/e075114&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRB &lt;strong&gt;78&lt;/strong&gt; (2008) 075114&lt;/a&gt;, &lt;a href=&quot;http://arxiv.org/abs/0801.3098&quot; rel=&quot;nofollow noreferrer&quot;&gt;arXiv:0801.3098&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="85" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-02-14T13:22:27.810" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1304" PostTypeId="1" AcceptedAnswerId="1315" CreationDate="2012-02-13T19:48:02.910" Score="2" ViewCount="256" Body="&lt;p&gt;I need a program or an algorithm that computes the intersection of a mesh and a boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mesh is structured orthogonal in nature and the boundary is a circle (for example). This will be used for solving Poisson's equation using finite difference technique with 5 point unequal spacing stencil.&lt;/p&gt;&#xA;" OwnerUserId="581" LastEditorUserId="123" LastEditDate="2012-02-14T13:21:56.050" LastActivityDate="2012-02-15T16:04:25.783" Title="Algorithm to compute the intersection of meshlines with a boundary" Tags="|linear-algebra|finite-difference|geometry|mesh-generation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1306" PostTypeId="2" ParentId="1300" CreationDate="2012-02-13T22:13:17.760" Score="1" Body="&lt;p&gt;There is also the ML package that's part of Trilinos. Its reputation is equally good as that of BoomerAMG/hypre.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A newer Trilinos package for AMG is called MueLu, I believe, and should also be available in the more recent releases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these are open source.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-02-13T22:13:17.760" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1310" PostTypeId="1" AcceptedAnswerId="1311" CreationDate="2012-02-14T03:35:29.677" Score="10" ViewCount="2722" Body="&lt;p&gt;I recently &lt;a href=&quot;https://scicomp.stackexchange.com/q/1234/116&quot;&gt;asked a question&lt;/a&gt; along the same lines for skew-Hermitian matrices. Inspired by the success of that question, and after banging my head against a wall for a couple of hours, I'm looking at the matrix exponential of real asymmetric matrices. The route to finding the eigenvalues and eigenvectors seems rather convoluted, and I'm afraid I've gotten lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Background: Some time ago I asked &lt;a href=&quot;http://theoreticalphysics.stackexchange.com/q/505/569&quot; rel=&quot;nofollow noreferrer&quot;&gt;this question&lt;/a&gt; on the theoretical physics SE. The result allows me to phrase master equations as real asymmetric matrices. In the time-independent case, the master equation is solved by exponentiating this matrix. In the time-dependent case it will require integration. I'm only concerned with time-independence at the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upon looking at the various subroutines I think I should be calling (&lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/cpp/win/mkl/refman/lse/functn_gehrd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;?gehrd&lt;/a&gt;, &lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/cpp/win/mkl/refman/lse/functn_orghr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;?orghr&lt;/a&gt;, &lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/cpp/win/mkl/refman/lse/functn_hseqr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;?hseqr&lt;/a&gt;...) it is unclear if it would be simpler to cast the matrix from &lt;code&gt;real*8&lt;/code&gt; to &lt;code&gt;complex*16&lt;/code&gt; and proceed with the complex double versions of these routines, or stick with &lt;code&gt;real*8&lt;/code&gt; and take the hit of doubling the number of my arrays and making a complex matrix of them later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, which routines should I be calling (and in what order), and should I use the real double versions or the complex double versions? Below is an attempt at doing this with real double versions. I've become stuck finding the eigenvalues and eigenvectors of &lt;code&gt;L*t&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;function time_indep_master(s,L,t)&#xA;  ! s is the length of a side of L, which is square.&#xA;  ! L is a real*8, asymmetric square matrix.&#xA;  ! t is a real*8 value corresponding to time.&#xA;  ! This function (will) compute expm(L*t).&#xA;&#xA;  integer, intent(in)    :: s&#xA;  real*8,  intent(in)    :: L(s,s), t&#xA;  real*8                 :: tau(s-1), work(s), wr(s), wi(s), vl&#xA;  real*8, dimension(s,s) :: time_indep_master, A, H, vr&#xA;  integer                :: info, m, ifaill(2*s), ifailr(2*s)&#xA;  logical                :: sel(s)&#xA;&#xA;  A = L*t&#xA;  sel = .true.&#xA;&#xA;  call dgehrd(s,1,s,A,s,tau,work,s,info)&#xA;  H = A&#xA;  call dorghr(s,1,s,A,s,tau,work,s,info)&#xA;  call dhseqr('e','v',s,1,s,H,s,wr,wi,A,s,work,s,info)&#xA;  call dhsein('r','q','n',sel,H,s,wr,wi,vl,1,vr,s,2*s,m,work,ifaill,ifailr,info)&#xA;&#xA;  ! Confused now...&#xA;&#xA;end function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="116" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-10-09T21:51:41.827" Title="Matrix exponential of a real asymmetric matrix with Fortran 95 and LAPACK" Tags="|linear-algebra|fortran|lapack|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1311" PostTypeId="2" ParentId="1310" CreationDate="2012-02-14T03:55:11.963" Score="8" Body="&lt;p&gt;I would first think really hard about whether or not the matrix is really completely arbitrary: Is there any transformation that would make it Hermitian? Does the physics guarantee that the matrix should be diagonalizable (with a reasonably conditioned eigenvector matrix)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it turns out that there really isn't any symmetry to exploit, then you should start by reading &lt;a href=&quot;http://www.cs.cornell.edu/cv/researchpdf/19ways+.pdf&quot;&gt;Nineteen Dubious Ways to Compute the Matrix Exponential&lt;/a&gt;, which is the standard reference (and is written by the author of MATLAB and the coauthor of G&amp;amp;vL).&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-02-14T03:55:11.963" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1312" PostTypeId="2" ParentId="1310" CreationDate="2012-02-14T04:25:43.720" Score="7" Body="&lt;p&gt;To build on what Jack has said, the standard approach that seems to be used in software (like EXPOKIT, mentioned in your earlier question) is scaling-and-squaring followed by Padé approximation (Methods 2 and 3) or Krylov subspace methods (Method 20). In particular, if you're looking at exponential integrators, you'll want to consider the Krylov subspace methods, and look at papers on exponential integrators (some references are mentioned along with Method 20 in the Moler &amp;amp; van Loan paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're hell bent on using eigenvectors, consider using triangular systems of eigenvectors (Method 15); since your matrix may be nondiagonalizable, this approach might not be best, but it's better than trying to calculate the eigenvectors and eigenvalues directly (i.e., Method 14).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reduction to Hessenberg form is not a good idea (Method 13).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not apparent to me whether you would be better served with real or complex arithmetic, since Fortran complex arithmetic is fast, but might overflow/underflow (see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1194/how-much-better-are-fortran-compilers-really&quot;&gt;&quot;How much better are Fortran compilers really?&quot;&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can safely ignore Methods 5-7 (ODE solver-based methods are inefficient), Methods 8-13 (expensive), Method 14 (calculating eigenvectors of large matrices is hard without special structure and prone to numerical error in ill-conditioned cases), and Method 16 (calculating the Jordan decomposition of a matrix is numerically unstable). Methods 17-19 are trickier to implement; in particular, Methods 17 and 18 would require more reading. Method 1 is a fall-back option for scaling-and-squaring if Padé approximations don't work well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit #1&lt;/strong&gt;: Based on the comments in response to Jack's answer, block diagonalizing seems like an option, in which case, something like Method 18 (block-triangular diagonalization) is a very good method to use. I hesitated to recommend it at the beginning because your question didn't specify this structure, but if you have a transformation that block diagonalizes your matrix, it takes most of the complexity out of the approach. You'll just want to make sure that you use G. W. Stewart's trick of decomposing each block diagonal matrix $B_{j}$ into&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B_j = \gamma_j I + E_j,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\gamma_{j}$ is the average of the eigenvalues of the $j$th block diagonal matrix. This decomposition will make $E_{j}$ nearly nilpotent, which will enhance the accuracy of your matrix exponential calculations. This trick is discussed on page 26 of the version of the Moler &amp;amp; van Loan paper that Jack linked to.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-02-15T03:55:53.510" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1314" PostTypeId="1" AcceptedAnswerId="1319" CreationDate="2012-02-14T14:45:27.690" Score="3" ViewCount="618" Body="&lt;p&gt;I want to calculate a finite difference (something like this &lt;a href=&quot;https://stackoverflow.com/a/632538/984260&quot;&gt;SO Post&lt;/a&gt;). My data is as follows: I have x-values that are powers of two (4, 8, 16, 32 and 64). Corresponding to them are y-values, such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Y=f(X)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$ is a monotonic function. My interest is in calculating finite difference such as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\text{Diff}(x) = (y_2-y_1) /(x_2-x_1)$$&#xA;at different values of X, where X is in the domain (i.e. $4\le x \le 64$). Now my problem is that when I take piecewise linear approximation, Diff(x) changes a lot between 4-to-8 and 8-to-16. This is undesirable. If I try to fit quadratic equation, by taking three points at a time, and then calculate finite difference, it becomes negative at some points, which is not physically acceptable, since $f$ is monotonic function. In other words, I fitted $ax^2+bx+c$ and thus $\text{Diff}(X)=2ax+b$. So, the problem is that $2ax+b$ is not positive at all points in the domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point-out a method to solve this problem. I need to use C/C++ for solve this, I cannot use matlab. I just have five X values, so solution of any complexity is acceptable. Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="976" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-02-16T19:48:27.863" Title="How to obtain finite difference, which is continuous" Tags="|finite-difference|c|c++|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1315" PostTypeId="2" ParentId="1304" CreationDate="2012-02-14T14:47:08.193" Score="1" Body="&lt;p&gt;I would suggest (in case you are coding in C++), the CGAL library, which allows you to compute the intersection lists between axis aligned bounding boxes (AABBs, in your case, the mesh cells) and geometric primitives, assembled in queries (or even surface meshes, polyhedra, etc):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Box_intersection_d/Chapter_main.html&quot; rel=&quot;nofollow&quot;&gt;Intersecting sequences in CGAL&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would give you labeling information on the cells that are cut... there are algorithms for computing the intersections if you need them, as well, which return actual geometrical constructs as intersection results.&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-02-14T14:47:08.193" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1316" PostTypeId="2" ParentId="1314" CreationDate="2012-02-14T15:15:35.380" Score="1" Body="&lt;p&gt;Fitting by a parametrized family of functions (like degree $n$-polynomials, where there are $n+1$ free parameters corresponds to minimizing an error term on a parameter space. If the error term is a least squares approach, then this leads to a quadratic program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Quadratic_programming&quot; rel=&quot;nofollow&quot; title=&quot;Quadratic programming&quot;&gt;Quadratic Programming&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want your function to be non-negative, or to be a monotonic function (it is not clear what you mean), then you arrive a contrained minimization, which is another topic on its own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Constraint_programming&quot; rel=&quot;nofollow&quot; title=&quot;Constraint Programming&quot;&gt;Constraint programming&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, a low order polynomial should suffice. The first wikipedia article should lead to plenty of ways to solve your problem.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-02-14T15:15:35.380" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1317" PostTypeId="2" ParentId="1314" CreationDate="2012-02-14T15:21:48.460" Score="7" Body="&lt;p&gt;A simple and accurate approach in this case is to use standard differences in $\log$ space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align*}&#xD;&#xA;s(x) &amp;amp;= \log x \\&#xD;&#xA;\partial_x f(x) &amp;amp;= \partial_s f(e^{s(x)}) \partial_x s(x) \\&#xD;&#xA;&amp;amp;= \frac 1 x \partial_s f(e^{s(x)}) \\&#xD;&#xA;\end{align*}&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $s(x)$ is equally spaced, it is easy to approximate the derivative on the bottom line using any standard differencing.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-02-14T16:05:05.907" LastActivityDate="2012-02-14T16:05:05.907" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1318" PostTypeId="2" ParentId="982" CreationDate="2012-02-14T17:09:03.210" Score="4" Body="&lt;p&gt;In &lt;a href=&quot;http://joc.journal.informs.org/content/4/4/369&quot; rel=&quot;nofollow&quot;&gt;Hartvigsen, D.: &lt;em&gt;Recognizing Voronoi Diagrams with Linear Programming&lt;/em&gt;&lt;/a&gt; several algorithms based on linear programming for recognizing Voronoi tesellations are presented, and states that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] for each $R_i$ of a Voronoi diagram, the set of points in $R_i$ contained in some generating set $P$ is either a singleton or the interior of a polyhedron.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems that the topic of the existence and uniqueness of solution to the inverse Voronoi problem is also developed in &lt;a href=&quot;http://gradworks.umi.com/14/45/1445204.html&quot; rel=&quot;nofollow&quot;&gt;Winter, L. G.: &lt;em&gt;The inverse problem to the Voronoi diagram&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="782" LastEditorUserId="782" LastEditDate="2012-02-14T22:00:27.183" LastActivityDate="2012-02-14T22:00:27.183" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1319" PostTypeId="2" ParentId="1314" CreationDate="2012-02-14T18:24:13.367" Score="2" Body="&lt;p&gt;Instead of fitting a general polynomial, you way want to consider using piece-wise cubic splines to interpolate the function in the interval between each gridpoint.  one good property of cubic splines is that they minimize the bending energy of the fitted curve, which may lead to fewer radical oscillations (which may be the cause of the negative values you experience with the quadratic approach you previously tried).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can obtain a piecewise cubic spline by proposing a separate cubic polynomial of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_i(x)=a_ix^3+b_ix^2+c_ix+d_i$ in each interval $i$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;whose endpoints are the gridpoints you selected.  Then, you can impose the following conditions:&#xA;1.  All functions must agree with the data at each of the nodes &#xA;2.  The functions must be continuous at each of the node points&#xA;3.  The first and second derivatives of each function must continuous at each of the node points  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get two extra equations by imposing the so-called &quot;natural spline&quot; condition, that is the second derivative must be zero at the first and last node.  This should give you enough information to solve for each of the coefficients of each cubic polynomial in each interval.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-15T00:56:36.437" LastActivityDate="2012-02-15T00:56:36.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1320" PostTypeId="1" AcceptedAnswerId="1336" CreationDate="2012-02-14T18:26:13.863" Score="6" ViewCount="586" Body="&lt;p&gt;Suppose one is performing Gibbs sampling with a Boltzmann distribution (or if you prefer, simulated annealing) at finite temperature. In general we would want to anneal: as the sampler converges to equilibrium, the temperature should be decreased. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the sampler's rate of convergence (local in time) does not vary, there is no real reason to vary the temperature. On the other hand, if the convergence rate decreases, then the temperature should be decreased as well. This naturally leads to a heuristic that sets the temperature as (proportional to) the convergence rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this heuristic (or one like it) been explored in the literature? Specific references and/or buzzwords are most welcome.&lt;/p&gt;&#xA;" OwnerUserId="628" LastEditorUserId="628" LastEditDate="2012-02-15T02:17:50.730" LastActivityDate="2014-04-20T00:44:23.483" Title="Heuristic for Gibbs sampler annealing schedule" Tags="|algorithms|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1321" PostTypeId="1" AcceptedAnswerId="25097" CreationDate="2012-02-14T18:46:23.870" Score="5" ViewCount="339" Body="&lt;p&gt;Suppose we have a $n\times n$ Cauchy matrix of which the $ij$-th entry is given by:&#xA;$$&#xD;&#xA;A_{ij} = \frac{1}{a_i - b_j}&#xD;&#xA;$$&#xA;the assumption is that the distance between $\{a_i\}$ and $\{b_j\}$ is greater some positive constant, for example:&#xA;$$&#xD;&#xA;a_k = e^{i\frac{(2k+1)\pi}{n}}, \quad b_k = e^{i\frac{2k\pi}{n}}&#xD;&#xA;$$&#xA;then we have $\displaystyle \min_{i,j} |a_i - b_j|\geq O(\sin(\pi/n))$, and we want to prove something like the off diagonal entries can be represented by:&#xA;$$&#xD;&#xA;\frac{1}{a_i - b_j} = \sum^{O(\log(n))}_{k=1} C_k + O(\varepsilon)&#xD;&#xA;$$&#xA;Is this possible? or there is another type of rank bound for Cauchy matrix?&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="4405" LastEditDate="2016-10-01T14:00:18.887" LastActivityDate="2016-10-01T14:06:03.333" Title="Is it possible to prove that the off-diagonal blocks of the Cauchy matrix have numerical rank $O(\log n)$?" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1323" PostTypeId="2" ParentId="1321" CreationDate="2012-02-14T20:35:33.467" Score="2" Body="&lt;p&gt;This is perhaps dodging your question, but why do you want to insist on low-rank off-diagonal blocks? Displacement structure is also data-sparse and allows for &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.16.4488&quot; rel=&quot;nofollow&quot;&gt;stable quadratic-complexity factorization algorithms&lt;/a&gt;. There is also &lt;a href=&quot;http://math.mit.edu/~plamen/files/mds.pdf&quot; rel=&quot;nofollow&quot;&gt;a nice survey paper on the subject&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ming Gu gave &lt;a href=&quot;http://math.berkeley.edu/~mgu/Seminar/Fall2009/ToeplitzSeminarTalk.pdf&quot; rel=&quot;nofollow&quot;&gt;a talk&lt;/a&gt; which uses Hierarchically Semi-Separable (HSS) matrices to compress the Cauchy-like matrices which arise from a Fourier transform of Toeplitz matrices, resulting in an $O(n \log n)$ approximate solution, but I'm not certain if the compression technique will work for general Cauchy matrices.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-02-14T20:35:33.467" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1324" PostTypeId="2" ParentId="1280" CreationDate="2012-02-15T07:42:52.810" Score="4" Body="&lt;p&gt;I had the same problem before. My solution is quite simple. Just use the I/O functions from your language, e.g. fopen, fseek, fread, fclose in C, for each processor. I assume that you enclose your codes by PetscInitialize()/PetscFinalize() and run the executable by mpirun. So before I/O in the codes you would first decide the locations of the data that each processor would read (for example, from file size and the rank of this processor). You might assemble Vec's to save your data into petsc format. But I found VecLoad does not allow one to decide local (processor) sizes. I also tried MPI I/O functions but failed to get correct values (maybe due to data types). &lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2012-02-15T07:42:52.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1325" PostTypeId="2" ParentId="1242" CreationDate="2012-02-15T10:53:47.373" Score="3" Body="&lt;p&gt;Some additional advice:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Before doing any optimization of a working program, make sure you have a good set of test cases that help maintain the integrity of the code.  There's no point in getting wrong results faster.&lt;/li&gt;&#xA;&lt;li&gt;If your optimization makes the code less readable, keep the original version around, at least in the form of a comment, but better as an alternative version to be selected at compile time and run time. Your optimization needs may change as your problems and your machines evolve, and the original code may be a better starting point for the optimization you will do five years from now.&lt;/li&gt;&#xA;&lt;li&gt;If your optimized version turns out to be of minimal impact, but makes the code less readable, less universal, or less stable, go back to the original version. You lose more than you gain.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="140" LastActivityDate="2012-02-15T10:53:47.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1326" PostTypeId="2" ParentId="589" CreationDate="2012-02-15T12:56:32.010" Score="1" Body="&lt;p&gt;When the main task is changing the &lt;em&gt;shape or structure&lt;/em&gt; of data rather than reading a specific file format (like .mat or hfd5 for a matrix of the same shape), then I find &lt;a href=&quot;http://www.wolfram.com/&quot; rel=&quot;nofollow&quot;&gt;Mathematica&lt;/a&gt; excellent for the task.  Its native data structure is essentially a tree (list of lists of lists ...), and it can do flexible transformations on them using pattern matching or other means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has two weaknesses in my opinion:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If it doesn't already support the input/output file format, it's not the best tool to implement your own importer in (unless the format is trivial).  &lt;a href=&quot;http://reference.wolfram.com/mathematica/guide/ImportingAndExporting.html&quot; rel=&quot;nofollow&quot;&gt;Here's the listing of supported formats.&lt;/a&gt;  (&lt;a href=&quot;http://reference.wolfram.com/mathematica/ref/format/PDB.html&quot; rel=&quot;nofollow&quot;&gt;PDB&lt;/a&gt; is supported.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It works well if all your data fits comfortably into memory.  I find that it is not a very good tool for sequential data processing (i.e. reading a small bit of the input and outputting it immediately).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For kneading complex or structured data into the desired shape (think high dimensional ragged arrays or XML-like structured data), it is a very good tool though.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2012-02-15T13:23:27.950" LastActivityDate="2012-02-15T13:23:27.950" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1327" PostTypeId="2" ParentId="1304" CreationDate="2012-02-15T16:04:25.783" Score="0" Body="&lt;p&gt;If you describe the boundary via the zero-line of a signed distance function, then you can determine whether a boundary intersects a cell by testing whether all vertices of the cell have the same sign of the distance function (no intersection) or different signs (intersection). I believe that that's the usual algorithm.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-02-15T16:04:25.783" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1328" PostTypeId="1" CreationDate="2012-02-16T00:05:33.557" Score="5" ViewCount="123" Body="&lt;p&gt;I need a wavelet frame for $L^2[0,\infty)$. Moreover, the wavelet should be twice differentiable and with continuous second order derivatives. Hopefully, the wavelet should have compact support (either in the time-domain or in the frequency-domain). Is there such a wavelet?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previously I had asked similar questions &lt;a href=&quot;https://math.stackexchange.com/questions/109503/what-other-wavelets-besides-the-haar-system-form-a-basis-of-l20-1&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://math.stackexchange.com/questions/109447/is-there-a-wavelet-basis-for-l20-infty&quot;&gt;here&lt;/a&gt;. However I think this question fits well here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-02-16T00:05:33.557" Title="Wavelets frame for $L^2[0,\infty)$" Tags="|wavelet|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1329" PostTypeId="1" AcceptedAnswerId="1330" CreationDate="2012-02-16T09:46:25.027" Score="29" ViewCount="15218" Body="&lt;p&gt;If I have a square invertible matrix and I take its determinant, and I find that $\det(A) \approx 0$, does this imply that the matrix is poorly conditioned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the converse also true? Does an ill-conditioned matrix have a nearly zero determinant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is something I tried in Octave:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = rand(4,4);&#xA;det(a) %0.008&#xA;cond(a)%125&#xA;a(:,4) = 1*a(:,1) + 2*a(:,2) = 0.000000001*ones(4,1);&#xA;det(a)%1.8E-11&#xA;cond(a)%3.46E10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="529" LastEditorUserId="1748" LastEditDate="2013-05-25T19:58:16.527" LastActivityDate="2013-05-25T19:58:16.527" Title="Does a tiny determinant imply ill-conditioning of a matrix?" Tags="|linear-algebra|condition-number|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1330" PostTypeId="2" ParentId="1329" CreationDate="2012-02-16T10:20:18.513" Score="38" Body="&lt;p&gt;It's the largeness of the &lt;a href=&quot;http://mathworld.wolfram.com/ConditionNumber.html&quot;&gt;condition number&lt;/a&gt; $\kappa(\mathbf A)$ that measures the nearness to singularity, not the tininess of the determinant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the diagonal matrix $10^{-50} \mathbf I$ has tiny determinant, but is well-conditioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the flip side, consider the following family of square upper triangular matrices, due to &lt;a href=&quot;http://gdz.sub.uni-goettingen.de/index.php?id=resolveppn&amp;amp;PPN=PPN243919689_0193&amp;amp;DMDID=DMDLOG_0016&quot;&gt;Alexander Ostrowski&lt;/a&gt; (and also studied by Jim Wilkinson):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf U=\begin{pmatrix}1&amp;amp;2&amp;amp;\cdots&amp;amp;2\\&amp;amp;1&amp;amp;\ddots&amp;amp;\vdots\\&amp;amp;&amp;amp;\ddots&amp;amp;2\\&amp;amp;&amp;amp;&amp;amp;1\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The determinant of the $n\times n$ matrix $\mathbf U$ is always $1$, but the ratio of the largest to the smallest singular value (i.e. the 2-norm condition number $\kappa_2(\mathbf U)=\dfrac{\sigma_1}{\sigma_n}$) was shown by Ostrowski to be equal to $\cot^2\dfrac{\pi}{4n}$, which can be seen to increase for increasing $n$.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2012-02-16T10:20:18.513" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1332" PostTypeId="2" ParentId="589" CreationDate="2012-02-16T20:04:07.953" Score="1" Body="&lt;p&gt;I aggree that Python is really good on this task. I would like to add some tools to the list:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you want to extract data from the web, or maybe fill in a form easilly, then &lt;a href=&quot;http://wwwsearch.sourceforge.net/mechanize/&quot; rel=&quot;nofollow&quot;&gt;mechanize&lt;/a&gt; is the best option. &lt;/li&gt;&#xA;&lt;li&gt;I heard that &lt;a href=&quot;http://www.crummy.com/software/BeautifulSoup/&quot; rel=&quot;nofollow&quot;&gt;BeautifulSoup&lt;/a&gt; is &lt;em&gt;really really&lt;/em&gt; good on parsing XML/HTML files. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="484" LastActivityDate="2012-02-16T20:04:07.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1333" PostTypeId="2" ParentId="517" CreationDate="2012-02-16T20:19:08.713" Score="4" Body="&lt;p&gt;Some more references:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.karlin.mff.cuni.cz/~strakos/download/2006_MeSt.pdf&quot; rel=&quot;nofollow&quot;&gt;The Lanczos and conjugate gradient algorithms in ﬁnite precision arithmetic&lt;/a&gt; , G. Meurant&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://books.google.com.br/books?id=1P3z8KgFToQC&amp;amp;pg=PA35&amp;amp;hl=pt-BR&amp;amp;source=gbs_toc_r&amp;amp;cad=4&quot; rel=&quot;nofollow&quot;&gt;Iterative methods for solving linear systems&lt;/a&gt;, A. Greenbaum&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="484" LastActivityDate="2012-02-16T20:19:08.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1336" PostTypeId="2" ParentId="1320" CreationDate="2012-02-17T03:51:08.443" Score="2" Body="&lt;p&gt;In my work creating a &lt;a href=&quot;http://people.seas.harvard.edu/~ely/software.html&quot; rel=&quot;nofollow&quot;&gt;simulated annealing course scheduler&lt;/a&gt; (bottom of that page), I came across a good paper: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Huang, M., F. Romeo, and A. Sangiovanni-Vincentelli, &quot;An efficient general cooling schedule for simulated annealing,&quot; Proc. of the IEEE International Conference on Computer Aided Design (ICCAD), pp. 381-384, 1986.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This work outlines some adaptive cooling schedules for simulated annealing, where the recent variance of the &quot;scores&quot; is taken into account to dampen or accelerate the temperature change as needed. (&quot;Score&quot; means the Boltzmann distribution evaluated at the accepted proposal locations.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They investigate a 'phase transition' property of simulated annealing problems: cooling tends to proceed  steadily, goes through a phase change with large temperature drop, then continues steadily. This phase change helps quantify the complexity of the optimization problem.&lt;/p&gt;&#xA;" OwnerDisplayName="user563" LastActivityDate="2012-02-17T03:51:08.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1337" PostTypeId="1" AcceptedAnswerId="1385" CreationDate="2012-02-17T05:19:10.080" Score="6" ViewCount="1025" Body="&lt;p&gt;As far as I see, there are many repositories for 3D shapes. But in FEM and many other applications, a planar mesh domain is also very common. However, I did not find a mesh repository specially designed for 2D shapes. Since 2D meshes can be generated efficiently by some techniques like 2D alpha shape, Delaunay triangulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is because in my research, I need a 2D dataset(including ~200 or more shapes) to test the performance.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="127" LastEditDate="2012-02-17T10:58:44.197" LastActivityDate="2012-04-11T14:51:42.930" Title="Is there any 2D shape repository?" Tags="|mesh-generation|computational-geometry|data-sets|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1338" PostTypeId="1" AcceptedAnswerId="1476" CreationDate="2012-02-17T06:35:10.127" Score="3" ViewCount="762" Body="&lt;p&gt;I know  for a given matrix $M$, there exists a matrix $U$ over the integers with determinant  $+1$ or $-1$ &#xA;such that $UM=E$.  I know $E$, but $M$ is not a square matrix. Is there any easy way to get $U$?&lt;/p&gt;&#xA;" OwnerUserId="277" LastEditorUserId="127" LastEditDate="2012-02-17T10:57:39.767" LastActivityDate="2012-12-19T20:16:52.010" Title="Unimodular Matrix calculation" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1339" PostTypeId="1" AcceptedAnswerId="1344" CreationDate="2012-02-17T06:56:03.423" Score="4" ViewCount="337" Body="&lt;p&gt;I have so far been using &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow&quot;&gt;VMD&lt;/a&gt; to create visualizations of molecular trajectories. However, I find myself not entirely satisfied with the quality of the resulting animations—when you want to do snapshot-by-snapshot movies, the simulation &quot;moves&quot; too quickly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, I sometimes receive individual graphics images, and need to convert those into an animation. Since I worked on OS X, I've tried both &lt;a href=&quot;http://www.imagemagick.org/script/index.php&quot; rel=&quot;nofollow&quot;&gt;imageMagick&lt;/a&gt; and more recently the built-in &lt;a href=&quot;http://download.cnet.com/Apple-iMovie/3000-2170_4-7469.html&quot; rel=&quot;nofollow&quot;&gt;iMovie&lt;/a&gt;. However, I'd like more flexibility in the available movie formats than imageMagick provides, and more command-line interface control than iMovie offers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what I'd like to know is, are there other visualization tools that can either:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;given a MD trajectory, can produce the snapshots and an animation of the complete trajectory with more control provided than VMD; &lt;em&gt;or&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;given a set of snapshots (in TGA, TIFF, PNG, etc.), can produce animations in multiple formats (MPEG, M4V, etc.)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-02-17T19:04:20.843" Title="Tools/solutions for visualization of molecular simulation trajectories?" Tags="|molecular-dynamics|visualization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1340" PostTypeId="2" ParentId="1339" CreationDate="2012-02-17T09:09:39.653" Score="1" Body="&lt;p&gt;If you want to become a real movie star, you should check: &lt;a href=&quot;http://www.molecularmovies.com/&quot; rel=&quot;nofollow&quot;&gt;http://www.molecularmovies.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the snapshots thing you can play with mpeg encoder&lt;/p&gt;&#xA;" OwnerUserId="891" LastActivityDate="2012-02-17T09:09:39.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1342" PostTypeId="1" AcceptedAnswerId="1345" CreationDate="2012-02-17T14:54:55.917" Score="8" ViewCount="834" Body="&lt;p&gt;I'm looking for a convenient, free C++ library for graphs that include a solver for the maximum common subgraph (MCS) problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of the&#xA;&lt;a href=&quot;http://www.boost.org/doc/libs/1_48_0/libs/graph/doc/index.html&quot; rel=&quot;noreferrer&quot;&gt;Boost Graph Library&lt;/a&gt;&#xA;and&#xA;&lt;a href=&quot;http://lemon.cs.elte.hu/trac/lemon&quot; rel=&quot;noreferrer&quot;&gt;LEMON&lt;/a&gt;&#xA;, but neither includes an MCS solver.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2012-02-17T20:09:22.773" LastActivityDate="2012-02-17T20:39:38.290" Title="C++ library for graphs with maximum common subgraph solver" Tags="|libraries|c++|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1343" PostTypeId="1" CreationDate="2012-02-17T18:52:49.547" Score="7" ViewCount="1202" Body="&lt;p&gt;I'm using &lt;a href=&quot;http://viennacl.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;ViennaCL's&lt;/a&gt; interface to &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt; as a way to leverage OpenCL.  Specifically, I'm using the  &lt;code&gt;::viennacl::linalg::bicgstab_tag&lt;/code&gt; with an Eigen sparse matrix.  However, the performance isn't what I hoped that it would be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What tools on Windows 7/Mac OS X/Linux should I use to understand the performance bottlenecks?&lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="9" LastEditDate="2012-02-17T21:35:49.903" LastActivityDate="2012-02-18T15:26:53.580" Title="Understanding OpenCL performance" Tags="|performance|krylov-method|viennacl|opencl|parallel-computing|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1344" PostTypeId="2" ParentId="1339" CreationDate="2012-02-17T19:04:20.843" Score="2" Body="&lt;p&gt;I like to use &lt;a href=&quot;http://ffmpeg.org&quot; rel=&quot;nofollow&quot;&gt;ffmpeg&lt;/a&gt; for movies from stills. I found this to be better than imovie and imageMagick. Just makes sure on the mac you grab the right codecs (don't remember off hand which I needed to get things working) and restart your computer after install them. Also, ffmpeg is available on &lt;a href=&quot;https://trac.macports.org/browser/trunk/dports/multimedia/ffmpeg/Portfile&quot; rel=&quot;nofollow&quot;&gt;macports&lt;/a&gt; if that's your thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't know much about molecular dynamics visulization, but you could look into &lt;a href=&quot;https://wci.llnl.gov/codes/visit/&quot; rel=&quot;nofollow&quot;&gt;VisIT&lt;/a&gt;. It has some molecular dynamics support according to the &lt;a href=&quot;https://wci.llnl.gov/codes/visit/FAQ.html&quot; rel=&quot;nofollow&quot;&gt;FAQ&lt;/a&gt;, and the developers are pretty responsive and helpful if you need that.&lt;/p&gt;&#xA;" OwnerUserId="880" LastActivityDate="2012-02-17T19:04:20.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1345" PostTypeId="2" ParentId="1342" CreationDate="2012-02-17T20:39:38.290" Score="6" Body="&lt;p&gt;According to &lt;a href=&quot;https://svn.boost.org/trac/boost/ticket/694&quot;&gt;this ticket&lt;/a&gt;, Boost Graph Library added this feature around 3 years ago. The appropriate function (or set of functions) appears to be &lt;a href=&quot;http://www.boost.org/doc/libs/1_40_0/libs/graph/doc/mcgregor_common_subgraphs.html&quot;&gt;this function&lt;/a&gt; (&lt;code&gt;mcgregor_common_subgraphs&lt;/code&gt;), which uses the McGregor algorithm.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-02-17T20:39:38.290" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1346" PostTypeId="1" AcceptedAnswerId="1351" CreationDate="2012-02-17T23:42:24.080" Score="25" ViewCount="4187" Body="&lt;p&gt;I've been reading some resources on the web about Galerkin methods to solve PDEs, but I'm not clear about something. The following is my own account of what I have understood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following boundary value problem (BVP):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L[u(x,y)]=0 \quad \text{on}\quad (x,y)\in\Omega, \qquad S[u]=0 \quad \text{on} \quad (x,y)\in\partial\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L$ is a 2nd order linear differentiation operator, $\Omega\subset\mathbb{R}^2$ is the domain of the BVP, $\partial\Omega$ is the boundary of the domain, and $S$ is a 1st order linear differential operator. Expess $u(x,y)$ as an aproximation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x,y)\approx \sum_{i=1}^N a_i g_i(x,y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $g_i$ are a set of functions that we will use to approximate $u$. Substituting in the BVP:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_i a_i L[g_i(x,y)]=R(a_1,...,a_N,x,y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since our approximation is not exact, the residual $R$ is not exactly zero. In the Galerkin-Ritz-Raleigh method we minimize $R$ with respect to the set of approximating functions by requiring $\langle R,g_i \rangle = 0$. Hence&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\langle R,g_i \rangle = \sum_{j=1}^N a_j \langle L[g_j],g_i \rangle = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, to find the coefficients $a_i$, we must solve the matrix equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(&#xD;&#xA;\begin{array}{ccc}&#xD;&#xA; \left\langle L\left[g_1\right],g_1\right\rangle  &amp;amp; \ldots  &amp;amp; \left\langle L\left[g_N\right],g_1\right\rangle  \\&#xD;&#xA; \ldots  &amp;amp; \ldots  &amp;amp; \ldots  \\&#xD;&#xA; \left\langle L\left[g_1\right],g_N\right\rangle  &amp;amp; \ldots  &amp;amp; \left\langle L\left[g_N\right],g_N\right\rangle &#xD;&#xA;\end{array}&#xD;&#xA;\right)\left(&#xD;&#xA;\begin{array}{c}&#xD;&#xA; a_1 \\&#xD;&#xA; \ldots  \\&#xD;&#xA; a_N&#xD;&#xA;\end{array}&#xD;&#xA;\right)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: How do I incorporate the boundary conditions into this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Originally the question said that $S[u]$ was a 2nd order linear differential operator. I changed it to a 1st order linear differential operator.&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2012-02-19T16:01:04.090" LastActivityDate="2020-09-29T00:01:33.633" Title="How to incorporate the boundary conditions with the Galerkin method?" Tags="|pde|finite-element|boundary-conditions|" AnswerCount="4" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="1348" PostTypeId="1" AcceptedAnswerId="1357" CreationDate="2012-02-18T03:44:21.777" Score="9" ViewCount="375" Body="&lt;p&gt;I have a mesh, faces $F$, edges $E$, and vertices $V$, and I have a list of predefined level set contours. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithm can I use to construct contours in the most efficient manner?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/p3vEQ.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A plot of the contour is shown above. Lines with the same color have the same $z$ value. &lt;/p&gt;&#xA;" OwnerUserId="690" LastEditorUserId="276" LastEditDate="2012-02-19T06:06:27.540" LastActivityDate="2012-02-19T06:06:27.540" Title="Given values on a mesh, what algorithm can I use to construct efficiently level set contours?" Tags="|computational-geometry|visualization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1349" PostTypeId="2" ParentId="1346" CreationDate="2012-02-18T04:05:40.997" Score="6" Body="&lt;p&gt;The general problem of dealing with boundary conditions with the finite element method can get pretty complicated. But if: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$S(u)$ is such that the only imposition $S(u)=0$ makes on the form of $u$  is that it is equal to some $f(x,y)$ on $\delta \Omega$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can finagle your elements so that $\delta \Omega$ is entirely on the boundaries of various elements&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;it's actually very simple. Your equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(&#xD;&#xA;\begin{array}{ccc}&#xD;&#xA; \left\langle L\left[g_1\right],g_1\right\rangle  &amp;amp; \ldots  &amp;amp; \left\langle L\left[g_N\right],g_1\right\rangle  \\&#xD;&#xA; \ldots  &amp;amp; \ldots  &amp;amp; \ldots  \\&#xD;&#xA; \left\langle L\left[g_1\right],g_N\right\rangle  &amp;amp; \ldots  &amp;amp; \left\langle L\left[g_N\right],g_N\right\rangle &#xD;&#xA;\end{array}&#xD;&#xA;\right)\left(&#xD;&#xA;\begin{array}{c}&#xD;&#xA; a_1 \\&#xD;&#xA; \ldots  \\&#xD;&#xA; a_N&#xD;&#xA;\end{array}&#xD;&#xA;\right)=0$$&#xA;needs to be replaced with&#xA;$$\left(&#xD;&#xA;\begin{array}{ccc}&#xD;&#xA; \left\langle L\left[g_1\right],g_1\right\rangle  &amp;amp; \ldots  &amp;amp; \left\langle L\left[g_N\right],g_1\right\rangle  \\&#xD;&#xA; \ldots  &amp;amp; \ldots  &amp;amp; \ldots  \\&#xD;&#xA; \left\langle L\left[g_1\right],g_N\right\rangle  &amp;amp; \ldots  &amp;amp; \left\langle L\left[g_N\right],g_N\right\rangle &#xD;&#xA;\end{array}&#xD;&#xA;\right)\left(&#xD;&#xA;\begin{array}{c}&#xD;&#xA; a_1 \\&#xD;&#xA; \ldots  \\&#xD;&#xA; a_N&#xD;&#xA;\end{array}&#xD;&#xA;\right)=\mathbf{b}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the right hand side vector $\mathbf{b}$ represents the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To determine $\mathbf{b}$, set the elements of your basis that determine the value of $u$ on $\delta \Omega$ to whatever values they need to be to satisfy the boundary conditions. In $\langle L[g_j],g_i \rangle$, you should exclude them from the $g_j$ but not the $g_i$ (the elements of $\mathbf{a}$ that correspond to these functions have already been determined, so they shouldn't be included in the matrix equation) . Then, set up &#xA;$$\langle R,g_i \rangle = \sum_{j=1}^N a_j \langle L[g_j],g_i \rangle = 0$$ as a matrix equation, and the values of the elements of $\mathbf{b}$ should pop right out as the inner products of $L$ operating on your your interior basis with elements of your boundary basis.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-02-18T04:05:40.997" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1350" PostTypeId="2" ParentId="1346" CreationDate="2012-02-18T04:12:40.800" Score="7" Body="&lt;p&gt;One possibility is to assemble the system matrix $A$ and right-hand side vector $b$, with the prescribed degrees of freedom as unknowns, like any other degree of freedom. Then, $A$ and $b$&#xA;are modified by zeroing rows and columns associated with the prescribed dofs, and putting a one into the corresponding diagonal entry, and appropriately modifying the rhs vector $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you zero rows, put one into the diagonal and change the rhs so that you enforce the prescribed value, the system is no longer symmetric. That´s why you zero columns and modify&#xA;the rhs vector $b$ to account for the prescribed value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is to add a very large number $p$ (usually 1e10) to the diagonal of the prescribed dof and then set the rhs entry to p*$\bar{u}$, where $\bar{u}$ is the prescribed value of that dof.&lt;/p&gt;&#xA;" OwnerUserId="484" LastActivityDate="2012-02-18T04:12:40.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1351" PostTypeId="2" ParentId="1346" CreationDate="2012-02-18T08:00:31.673" Score="17" Body="&lt;p&gt;A quick and general answers without mathematical abstractions. There are several options to impose boundary conditions, e.g.&lt;/p&gt;&#xA;&lt;p&gt;Strictly speaking the Galerkin method requires that you choose a set of basis functions which satisfy the BC of the problem (e.g. via basis recombination and/or splitting of the approximation &lt;span class=&quot;math-container&quot;&gt;$u_h=u_0+u_N$&lt;/span&gt; wit &lt;span class=&quot;math-container&quot;&gt;$u_0$&lt;/span&gt; responsible for inhomogenous solutions and &lt;span class=&quot;math-container&quot;&gt;$u_N$&lt;/span&gt; a partial sum which relies on basis functions which satisfies the homogenous conditions)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Penalty methods/Lagrange multiplies where one essentially add a penalty term which incorporated the boundary condition, e.g. &lt;span class=&quot;math-container&quot;&gt;$A + \tau \cdot B = b + \tau\cdot b_p$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is a matrix responsible for the discrete boundary condition and &lt;span class=&quot;math-container&quot;&gt;$b_p$&lt;/span&gt; is responsible for inhomogenous terms. In the limit &lt;span class=&quot;math-container&quot;&gt;$\tau\to\infty$&lt;/span&gt; the conditions is strongly imposed and otherwise it is weakly imposed. Choice of &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; affects conditioning of the system.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Tau method where a number of equations are exchanged (modification of rows in Galerkin system) with discrete versions of boundary conditions which is then enforced explicitly. Note: one option is also to make the systems overdetermined with additional boundary conditions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Before discretization (Ritz Method) rewrite Galerkin formulation via Gauss divergence theorem to transform volume integrals to boundary integrals and then incorporate ( exact or approximately) boundary conditions directly in formulation before discretization.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, by exploiting connection between nodal/modal expansions it is also possible to derive a nodal Galerkin method where the solution to the system is the coefficients of a Lagrange basis rather than a modal basis.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="304" LastEditorUserId="3481" LastEditDate="2020-09-29T00:01:33.633" LastActivityDate="2020-09-29T00:01:33.633" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="1352" PostTypeId="1" AcceptedAnswerId="1358" CreationDate="2012-02-18T10:46:49.717" Score="11" ViewCount="503" Body="&lt;p&gt;In computational science we often encounter large linear systems which we are required to solve by some (efficient) means, e.g. by either direct or iterative methods. If we focus on the latter, how can we establish that an iterative method for solving a large linear systems is convergent in practice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear that we can do trial and error analysis (cf. &lt;a href=&quot;https://scicomp.stackexchange.com/questions/513/why-is-my-iterative-linear-solver-not-converging&quot;&gt;Why is my iterative linear solver not converging?&lt;/a&gt;) and rely on iterative methods which guarantee convergence by proofs or have a sound experience base (e.g. Krylov subspace methods such as CG and GMRES for symmetric and nonsymmetric systems respectively). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, what can be done to establish convergence in practice? and what is done? &lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-02-18T21:39:14.643" Title="How to establish that an iterative method for large linear systems is convergent in practice?" Tags="|linear-algebra|iterative-method|convergence|krylov-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1353" PostTypeId="2" ParentId="1343" CreationDate="2012-02-18T11:07:45.937" Score="2" Body="&lt;p&gt;A very simple means is to compute by hand the amount of &lt;strong&gt;useful&lt;/strong&gt; work that are to be done (i.e. either amount of data to be transferred or the amount of useful floating point operations (flops) or both). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then do some simple timings and average them to get an absolute prediction of performance for a given problem size (time vs. problem size) and from this you can assess overall performance characteristics. This is a simple and useful means for making comparisons with hardware specifications and/or performance of other software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, To do a complete performance breakdown you will need tools such as TotalView for profiling and identifying performance bottlenecks.&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-02-18T11:07:45.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1354" PostTypeId="5" CreationDate="2012-02-18T12:40:22.757" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;http://viennacl.sourceforge.net/viennacl-about.html&quot; rel=&quot;nofollow&quot;&gt;Vienna Computing Library (ViennaCL)&lt;/a&gt; is a scientific computing library written in C++ and based on OpenCL. It allows simple, high-level access to the vast computing resources available on parallel architectures such as GPUs and is primarily focused on common linear algebra operations (BLAS levels 1, 2 and 3) and the solution of large systems of equations by means of iterative methods with optional preconditioner.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-02-18T12:40:22.757" LastActivityDate="2012-02-18T12:40:22.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1355" PostTypeId="4" CreationDate="2012-02-18T12:40:22.757" Score="0" Body="a linear algebra library for computations on GPUs and multi-core CPUs" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-02-18T12:40:22.757" LastActivityDate="2012-02-18T12:40:22.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1356" PostTypeId="2" ParentId="1343" CreationDate="2012-02-18T15:26:53.580" Score="5" Body="&lt;p&gt;General statement:&#xA;For small system sizes, there is little benefit of using OpenCL at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alright, now for the justification: There is a certain amount of overhead associated with each OpenCL kernel launch. Exact timings depend on the underlying hardware, but as a rule of thumb one can use a pessimistic estimate of 10 microseconds for CPUs and 100 microseconds for GPUs as I've once reported in &lt;a href=&quot;http://software.intel.com/en-us/forums/showthread.php?t=81682&quot; rel=&quot;noreferrer&quot;&gt;this thread at the Intel OpenCL forum&lt;/a&gt;. This is A LOT considering that modern hardware provides many GFLOPs of processing power. For example, adding two vectors with 100.000 entries each on a GPU with 100 GB/sec memory bandwidth requires 3 * 8 * 100.000 Bytes (approx. 2MB) of data to be transferred, taking 20 us. Thus, even adding up two vectors of size 100.000 can show significant kernel launch overhead. In practice, kernel launch overhead can be reduced if kernels are enqueued while another kernel is still active - this, however, requires again that kernel execution times are sufficiently large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since most operations inside Krylov solvers are comparable in complexity to vector additions (this often applies to sparse matrix-vector multiplications as well), benchmarks for system sizes below 10.000 by 10.000 essentially measure OpenCL kernel launch overheads only. I thus recommend to run benchmarks again with your upper limit 50k by 50k. For smaller system sizes, just use BiCGStab with the Eigen matrix directly (this is one of the reasons why ViennaCL offers generic implementations...) or give sparse direct solvers a try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For dense systems (matrices), the notion of a 'small system' is certainly shifted to smaller values. Still, below system sizes of about 1000x1000 overheads become significant again.&lt;/p&gt;&#xA;" OwnerUserId="966" LastActivityDate="2012-02-18T15:26:53.580" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1357" PostTypeId="2" ParentId="1348" CreationDate="2012-02-18T17:46:14.247" Score="4" Body="&lt;p&gt;It can be done by a loop over each element and then for each element a loop over the edges. Then for each edge determine the coordinate for a given contour if it cross the edge, i.e. check if the contour takes a value in the interval spanned by the vertices values. If the contour crosses two edges, then draw the a line between the coordinates between detected crossing points at the edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this for all elements will create the contours. It is also an option to refine to a finer mesh via interpolation on each triangle and then use a delaunay triangulations of the refined mesh and then apply the routine as described above for achieving a sufficiently fine resolution.&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-02-18T17:46:14.247" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1358" PostTypeId="2" ParentId="1352" CreationDate="2012-02-18T20:35:31.167" Score="6" Body="&lt;p&gt;For many partial differential equations arising in nature, particularly with strong nonlinearities or anisotropies, the choice of an appropriate preconditioner can have a large effect on whether the iterative method converges rapidly, slowly, or not at all.  Examples of problems that are known to have fast and effective preconditioners include strongly elliptic partial differential equations, where the multigrid method frequently achieves rapid convergence.  There are a number of tests one can use to assess convergence; here I am going to use the functionality from PETSc as an example, as it is arguably the oldest and most mature library for iteratively solving sparse systems of linear (and nonlinear equations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc uses an object called a KSPMonitor for monitoring the progress of an iterative solver, and deciding if the solver has converged or diverged.  The monitor uses four different criterion to decide whether to halt.  More details on the discussion here can be found in the man page for &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPGetConvergedReason.html#KSPGetConvergedReason&quot; rel=&quot;noreferrer&quot;&gt;KSPGetConvergedReason()&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We assume notationally the user is solving the following system of equations for $x$: &#xA;$$Ax=b$$&lt;br&gt;&#xA;We call the current guess $\hat{x}$, and define the residual $\hat{r}$ based on the style of preconditioning:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Left Preconditioning $\left(P^{-1}(Ax − b) \right)$&lt;/strong&gt;&#xA;$$\hat{r}=P^{-1}(A\hat{x}-b)$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;No/Right Preconditioning $\left(AP^{-1}Px = b \right)$&lt;/strong&gt;&#xA;$$\hat{r}=A\hat{x}-b$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Convergence Criteria&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Absolute Tolerance&lt;/strong&gt; - The absolute tolerance criterion is satisfied when the norm of the residual is less than the predefined constant $a_{tol}$:&#xA;$$\|\hat{r}\| \le a_{tol}$$&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Relative Tolerance&lt;/strong&gt; - The relative tolerance criterion is satisfied when the norm of the residual is less than the norm of the right hand side by a factor of predefined constant $r_{tol}$: $$\|\hat{r}\| \le r_{tol}\cdot \|b\|$$ &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Other Criteria&lt;/strong&gt; - The iterative solve can also converge due to detection of a small step length or negative curvature.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Divergence Criteria&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Maximum Iterations&lt;/strong&gt; - The number of iterations a solver can take is capped by maximum iterations.  If none of the other criteria has been met when the maximum number of iterations is reached, the monitor returns as diverged.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Residual NaN&lt;/strong&gt; - If at any point the residual evaluates to NaN, the solver returns as diverged.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Divergence of Residual Norm&lt;/strong&gt; The monitor returns as diverged if at any point the norm of the residual is greater than the norm of the right-hand side by a factor of predefined constant $d_{tol}$:&#xA;$$\|\hat{r}\| \ge d_{tol}\cdot \|b\|$$ &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Solver Breakdown&lt;/strong&gt; The Krylov method itself can signal divergence if it detects a singular matrix or preconditioner.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-02-18T21:39:14.643" LastActivityDate="2012-02-18T21:39:14.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1359" PostTypeId="2" ParentId="1280" CreationDate="2012-02-18T21:15:02.653" Score="5" Body="&lt;p&gt;Okay, the full format of the binary file is &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatLoad.html&quot;&gt;here&lt;/a&gt;, and reduces to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;int              id:  MAT_FILE_CLASSID&lt;/li&gt;&#xA;&lt;li&gt;int              nr:  number of rows&lt;/li&gt;&#xA;&lt;li&gt;int              nc:  number of columns&lt;/li&gt;&#xA;&lt;li&gt;int              nnz: total number of nonzeros&lt;/li&gt;&#xA;&lt;li&gt;int[nr]          nz:  number nonzeros in each row&lt;/li&gt;&#xA;&lt;li&gt;int[nnz]         j:   column indices of all nonzeros (starting index is zero)&lt;/li&gt;&#xA;&lt;li&gt;PetscScalar[nnz] a:   values of all nonzeros&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can also use bin/matlab/PetscBinaryWrite.m to write a Matlab matrix to PETSc binary format.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2012-02-18T21:15:02.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1360" PostTypeId="2" ParentId="1264" CreationDate="2012-02-18T22:04:41.930" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.avglab.com/andrew/soft.html&quot; rel=&quot;nofollow&quot;&gt;There's benchmark code for max cardinality matching&lt;/a&gt;: you could use that and even randomly generated bipartite graphs to establish ground truth and test out your algorithms. &lt;/p&gt;&#xA;" OwnerUserId="1005" LastActivityDate="2012-02-18T22:04:41.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1362" PostTypeId="1" CreationDate="2012-02-19T08:23:08.437" Score="2" ViewCount="1691" Body="&lt;p&gt;If we use the composite trapezoidal rule, then what is the least number of divisions $N$ for which the error of the integral $\int^1_0{e^{-x}}dx$ doesn't exceed $\frac{1}{12}\times10^{-2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is 11 or 5. &#xA;Kindly tell me which of the answer is correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I obtained 11 as the answer by applying the formula &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Big|\frac{(b-a)^3}{12N^2}\times{(e^{-x})^{\prime\prime}_{x=\varepsilon}}\Big| \text{ = } {\frac{10^{-2}}{12}}$$ where $\varepsilon \in [0,1]$ is chosen so that it maximizes the value of $e^{-\varepsilon}$ (which I believe occurs at $\varepsilon = 0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this equation, I get $N = 10$ (i.e. I must have atleast 11 equidistant divisions if I have to keep the error less than  $\frac{10^{-2}}{12}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as 5 is concerned, I just used 5 equidistant intervals i.e $0,\frac{1}{5},\frac{2}{5},\frac{3}{5},\frac{4}{5},1$ and I applied them in the trapezoidal rule. Now here's the problem:- When calculated the answer at $N=5$, I got a value greater that the actual value of the integral. Is it possible? If yes, why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which of my answer is correct because I obtained 11 by a well-established formula and 5 was just an option I hit upon and am not really sure of 5's correctness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note:- I am posting this question because everywhere else nobody is giving any reply at all. I don't know if this question belongs here. If it does, then kindly reply. If it doesn't,then feel free to erase or delete or whatever :)&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="81" LastEditDate="2012-02-19T09:00:25.343" LastActivityDate="2012-02-19T14:29:47.817" Title="Error calculation in trapezoidal rule" Tags="|quadrature|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1363" PostTypeId="2" ParentId="1362" CreationDate="2012-02-19T09:26:38.177" Score="3" Body="&lt;p&gt;One easy way to check your answer is to calculate the integral with the trapezoidal rule and compare it to the definite integral in your problem statement, which you can calculate analytically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Graphing the function and drawing in the appropriate trapezoids (for the trapezoidal rule) will also give you some intuition as to what conditions might produce a trapezoidal rule approximation that is larger than the true definite integral.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-02-19T09:26:38.177" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1365" PostTypeId="5" CreationDate="2012-02-19T18:45:56.510" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-19T18:45:56.510" LastActivityDate="2012-02-19T18:45:56.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1366" PostTypeId="4" CreationDate="2012-02-19T18:45:56.510" Score="0" Body="For questions regarding design and implementation of preconditioners for solving linear systems." OwnerUserId="529" LastEditorUserId="529" LastEditDate="2012-02-19T18:52:39.347" LastActivityDate="2012-02-19T18:52:39.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1367" PostTypeId="5" CreationDate="2012-02-19T18:48:25.540" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-02-19T18:48:25.540" LastActivityDate="2012-02-19T18:48:25.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1368" PostTypeId="4" CreationDate="2012-02-19T18:48:25.540" Score="0" Body="For questions regarding the choice and problems in programming languages within the scope of Scientific Computing." OwnerUserId="529" LastEditorUserId="529" LastEditDate="2012-02-19T18:52:37.320" LastActivityDate="2012-02-19T18:52:37.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1369" PostTypeId="5" CreationDate="2012-02-19T18:50:23.213" Score="0" Body="&lt;p&gt;Homework help&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of homework exercises is to solidify understanding and develop skills. It is in this spirit that homework designated questions will/should be answered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions regarding homework assignments are more than welcome, provided that they:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Briefly explain the problem you are trying to solve—do not post your entire assignment verbatim.&lt;/li&gt;&#xA;&lt;li&gt;Explain what you tried and where you're stuck (showing your work is a good idea).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Don't&lt;/strong&gt; ask for complete solutions to the problem—we're not here to do your homework for you.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This tag is intended for use along with other relevant tags. Please do not use this as the only tag for your homework questions.&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="522" LastEditDate="2012-02-25T04:19:37.700" LastActivityDate="2012-02-25T04:19:37.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1370" PostTypeId="4" CreationDate="2012-02-19T18:50:23.213" Score="0" Body="Do not use this tag." OwnerUserId="529" LastEditorUserId="276" LastEditDate="2017-10-10T00:18:07.560" LastActivityDate="2017-10-10T00:18:07.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1371" PostTypeId="2" ParentId="2" CreationDate="2012-02-19T22:18:49.810" Score="2" Body="&lt;p&gt;Every van-der-Waals bonded molecule like H_2 is not covered by Hartree Fock Theory.&#xA;Electron Correlation is not considered. So HF is just a good starting point for post HF methods like Møller-Plesset perturbation theory, coupled cluster etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Statistically the best available method for the ground state in modern quantum chemistry programs is CCSDT, coupled cluster with single, doublet and triplet exitation.This methodes are time consuming and scale with $N^7$ to $N^8$ where $N$ is the number of gaussian basis functions (GTOs)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explicit correlation (F12) methods are even better, but they scale horrible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In HF the ground state has to be given by a single Slater determinat. So even the ground state may not be computeable with Hartree Fock theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You must have a good guess for the HF starting point. Test your program with e.g. ozone. It has a broken symmetry unrestricted HF singlet wavefunction. Most likely you converge to a higher state in HF.&lt;/p&gt;&#xA;" OwnerUserId="1013" LastActivityDate="2012-02-19T22:18:49.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1372" PostTypeId="1" AcceptedAnswerId="1375" CreationDate="2012-02-20T08:44:03.317" Score="11" ViewCount="735" Body="&lt;p&gt;Is there a speedier way to calculate standard errors for linear regression problems, than by inverting $X&amp;#39;X$? Here I assume we have regression:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y=X\beta+\varepsilon,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $X$ is $n\times k$ matrix and $y$ is $n\times 1$ vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For finding least squares problem solution it is impractical to do anything with $X&amp;#39;X$, you can use QR or SVD decompositions on matrix $X$ directly. Or alternatively you can use gradient methods. But what about standard errors? We really only need the diagonal of $(X&amp;#39;X)^{-1}$ (and naturally LS solution to calculate the estimate of standard error of $\varepsilon$). Are there any specific methods for standard error calculation?&lt;/p&gt;&#xA;" OwnerUserId="128" LastActivityDate="2012-02-20T17:54:06.710" Title="Computing standard errors for linear regression problems without calculating inverse" Tags="|linear-algebra|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1374" PostTypeId="1" CreationDate="2012-02-20T15:09:54.063" Score="10" ViewCount="1247" Body="&lt;p&gt;I have already developed a working solution of the Finite Element Method to solve heat transfer problems using GPU and OpenCL using the Conjugate Gradient method. The main disadvantage of this method is high demand for memory. Moreover, in case of graphics cards memory is often very limited. I see two options: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create subdomains and swap parts of the mesh with host memory&lt;/li&gt;&#xA;&lt;li&gt;Use multifrontal methods&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have to take into account the specific architecture. Swapping could be very expensive. CG method is popular in the context of GPGPU computing but I cannot find any comparison between CG and multifrontal methods (in case of GPGPU). Can multifrontal method be faster then CG? This is a general question, in fact, it still depends on the implementation.&lt;/p&gt;&#xA;" OwnerUserId="1010" LastEditorUserId="880" LastEditDate="2012-02-20T18:22:33.420" LastActivityDate="2014-06-01T07:45:59.903" Title="Selection of linear solver for GPGPU computation (OpenCL)" Tags="|parallel-computing|opencl|linear-solver|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1375" PostTypeId="2" ParentId="1372" CreationDate="2012-02-20T17:54:06.710" Score="5" Body="&lt;p&gt;Let's suppose that you solved your least squares problem using the singular value decomposition (SVD) of $X$, given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X = U\Sigma V',$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ and $V$ are unitary, and $\Sigma$ is diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X'X = V \Sigma^2 V'.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(X'X)^{-1}$ exists iff $X$ is full rank (or has strictly positive singular values), in which case&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(X'X)^{-1} = V \Sigma^{-2} V'.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(See an answer I gave to a &lt;a href=&quot;https://math.stackexchange.com/a/106787/21372&quot;&gt;related question on Math.SE&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you already have $\Sigma$ and $V$, calculating $(X'X)^{-1}$ requires inverting and squaring a diagonal matrix ($n$ operations for an $n \times n$ matrix), scaling the columns (or rows) of a matrix ($n^2$ operations), and a single matrix multiply (unfortunately $\mathcal{O}(n^{3})$). This method will be well-behaved numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are fast methods for obtaining the diagonal elements of the inverse of a sparse matrix (see work by &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/PDF/umsi-2010-42.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yousef Saad's group&lt;/a&gt; and &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/PDF/umsi-2010-42.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;work by Lin Lin, &lt;em&gt;et al&lt;/em&gt;&lt;/a&gt;). However, in your case, $X'X$ is probably not sparse (even if $X$ is), and even if it were, it would likely be ill-conditioned enough that these fast methods would yield inaccurate results. &lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-02-20T17:54:06.710" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1376" PostTypeId="1" CreationDate="2012-02-21T01:02:03.423" Score="7" ViewCount="322" Body="&lt;p&gt;I've written code for an obstructed random walker simulation and I want to run long simulations (6 hours or more on my computer). I don't want to run this simulation on my computer because I will want to run multiple simulations simultaneously and I don't want to have to worry about not having access to my computer's hardware resources, overheating, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I'd like to run the simulation on a supercomputer, but I assume that would be expensive. Are there any companies/tools that would take my code/program run simulations and return the results to me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My source code is in C# (which is obviously part of the reason these simulations take so long).&lt;/p&gt;&#xA;" OwnerUserId="94" LastActivityDate="2013-04-12T00:10:11.790" Title="External hardware resources for running long and computationally intensive simulations" Tags="|hpc|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1377" PostTypeId="2" ParentId="1376" CreationDate="2012-02-21T02:21:02.290" Score="11" Body="&lt;p&gt;Most supercomputing resources in the US are free to use for those associated with a US-based academic or research organization. You must make an application which is reviewed in some way (depending on the center or program that you apply to). For example, the US National Science Foundation funds the &lt;a href=&quot;http://www.xsede.org&quot; rel=&quot;nofollow&quot;&gt;XSEDE&lt;/a&gt; project which federates a number of large-scale resource providers to allocate time on their supercomputers to the national research community. If you're not in the US, there are similar programs in the UK, EU, Australia, and China that I am aware of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One problem in your case is your use of C#. No supercomputing resource provider that I know of supports it. The NSF did fund a project to buy and provide time on Microsoft's Azure cloud platform, but I do not know how time is allocated since it's not part of XSEDE.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2012-02-23T18:13:50.837" LastActivityDate="2012-02-23T18:13:50.837" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1378" PostTypeId="2" ParentId="1376" CreationDate="2012-02-21T04:25:36.047" Score="6" Body="&lt;p&gt;You could consider using the &lt;a href=&quot;http://aws.amazon.com/&quot;&gt;Amazon web services&lt;/a&gt;. There is a &lt;a href=&quot;http://aws.amazon.com/free/&quot;&gt;free tier&lt;/a&gt; of usage which might work for you depending the computational resources required for your application. And if you need something more, stuff really isn't that expensive. There is a bit of a learning curve to get going on these resources.&lt;/p&gt;&#xA;" OwnerUserId="880" LastActivityDate="2012-02-21T04:25:36.047" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1379" PostTypeId="2" ParentId="1376" CreationDate="2012-02-21T08:27:03.637" Score="0" Body="&lt;p&gt;I do not know whether you are in an academic research group or just a curious scientist. In the former, I would search for another research group, with similar interests and access to supercomputing resources and propose them a collaboration.&lt;/p&gt;&#xA;" OwnerUserId="891" LastActivityDate="2012-02-21T08:27:03.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1380" PostTypeId="1" CreationDate="2012-02-21T21:17:13.973" Score="16" ViewCount="27941" Body="&lt;p&gt;Suppose&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align*}&#xD;&#xA;\min A &amp;amp;\mathrm{vec}(U) \\&#xD;&#xA;&amp;amp;\text{subject to } U_{i,j} \leq \max\{U_{i,k}, U_{k,j}\}, \quad i,j,k = 1, \ldots, n&#xD;&#xA;\end{align*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is a symmetric $n\times n$ matrix, and $\mathrm{vec}(U)$ reshapes $U$ into a one-dimensional vector with $n^2$ entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The part of the above program that gives me problems is the $\max\{⋅,⋅\}$. (Restricting solutions to nonnegative symetric matrices seems to be straightforward.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any help or references!&lt;/p&gt;&#xA;" OwnerUserId="1020" LastEditorUserId="276" LastEditDate="2012-02-28T01:28:30.087" LastActivityDate="2018-10-28T08:43:52.323" Title="Constraints involving $\max$ in a linear program?" Tags="|optimization|linear-programming|" AnswerCount="6" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1381" PostTypeId="2" ParentId="1380" CreationDate="2012-02-22T01:09:29.157" Score="14" Body="&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Let's try this explanation again, this time when I'm more awake.&lt;/p&gt;&#xA;&lt;p&gt;There are three big issues with the formulation (in order of severity):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There's no obvious reformulation of the problem that is obviously smooth, convex, or linear.&lt;/li&gt;&#xA;&lt;li&gt;It's nonsmooth.&lt;/li&gt;&#xA;&lt;li&gt;It's not necessarily convex.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;No obvious smooth/convex/linear reformulation&lt;/h2&gt;&#xA;&lt;p&gt;First off, there's no standard, obvious reformulation of each &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; constraint. Aron's suggestion applies to the more common &lt;span class=&quot;math-container&quot;&gt;$\min$&lt;/span&gt; constraint, in which a constraint like &lt;span class=&quot;math-container&quot;&gt;$$U_{ij} \leq \min_{k}\{U_{ik}, U_{kj}\}$$&lt;/span&gt; is replaced by the following two equivalent inequalities:&lt;span class=&quot;math-container&quot;&gt;$$U_{ij} \leq U_{ik}, \quad \forall k$$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$$U_{ij} \leq U_{kj}, \quad \forall k.$$&lt;/span&gt; The reformulation isn't ideal, each &lt;span class=&quot;math-container&quot;&gt;$\min$&lt;/span&gt; constraint has been replaced by &lt;span class=&quot;math-container&quot;&gt;$2n$&lt;/span&gt; linear constraints, but it converts a nonsmooth nonlinear program into a linear program, which is orders of magnitude faster to solve.&lt;/p&gt;&#xA;&lt;p&gt;Wolfgang points out that it might be possible (he doesn't include a proof) to reformulate the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; constraints so that they are linear and smooth by adding slack variables. A slack variable needs to be added for each &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; constraint in the original formulation, which means that we're adding &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt; constraints in this reformulation. In addition, every &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; constraint is replaced by &lt;span class=&quot;math-container&quot;&gt;$2n$&lt;/span&gt; (or so) linear constraints. The real killer is that the nonsmoothness is moved from the constraints to the objective, so Wolfgang's formulation still yields a nonsmooth nonlinear program.&lt;/p&gt;&#xA;&lt;p&gt;There's no standard reformulation of &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; constraints in a minimization problem that I know of, having checked my linear programming textbook and having done a literature search. It doesn't mean that such a reformulation doesn't exist; it just means I haven't come across it. If I had to guess, I'd say an LP formulation doesn't exist.&lt;/p&gt;&#xA;&lt;h2&gt;Nonsmoothness&lt;/h2&gt;&#xA;&lt;p&gt;In this context, nonsmoothness means that at least one of the functions in the formulation (the objective or the constraints) is not twice continuously differentiable. The nonsmooth functions in this formulation are the &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; functions.&lt;/p&gt;&#xA;&lt;p&gt;Nonsmoothness is a huge problem because:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it immediately makes your problem nonlinear&lt;/li&gt;&#xA;&lt;li&gt;most nonlinear programming solvers assume twice continuously differentiable functions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$\max$&lt;/span&gt; functions aren't even once continuously differentiable, you can't even use traditional gradient descent methods without difficulty. Nonsmooth nonlinear programming algorithms are slower than their smooth counterparts.&lt;/p&gt;&#xA;&lt;h2&gt;Possible nonconvexity&lt;/h2&gt;&#xA;&lt;p&gt;Your problem could be nonconvex, because in &amp;quot;standard form&amp;quot; for nonlinear programs (i.e., expressing all constraints in the form &lt;span class=&quot;math-container&quot;&gt;$\mathbf{g}(\mathbf{x}) \leq \mathbf{0}$&lt;/span&gt;), the troublesome constraints in your formulation are&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$U_{ij} - \max_{k}\{U_{ik}, U_{kj}\} \leq 0, \quad \forall i,j,k.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;These functions are concave.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Proof:&lt;/em&gt; In this case, the functions &lt;span class=&quot;math-container&quot;&gt;$-U_{ij}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\max_{k}\{U_{ik}, U_{kj}\}$&lt;/span&gt; are both convex. The sum of convex functions is convex, and multiplying a convex function by -1 results in a concave function. (QED.)&lt;/p&gt;&#xA;&lt;p&gt;As Tim points out, just because &lt;span class=&quot;math-container&quot;&gt;$\mathbf{g}$&lt;/span&gt; is nonconvex doesn't mean that your problem is actually nonconvex, but if you're trying to solve an optimization problem to global optimality, you can only guarantee that a convex optimization solver will return a global optimum if your problem is convex. If you really want a global optimum, it would behoove you to determine if your feasible set is convex (or not). In the absence of such information, you have to assume that your problem might be nonconvex, and use algorithms that do not rely on convexity information. Even then, the nonsmoothness and lack of a good reformulation are much bigger issues.&lt;/p&gt;&#xA;&lt;h2&gt;Options for solving the problem&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Settle for possibly finding a feasible solution. In this case, do what Aron said, and replace &lt;span class=&quot;math-container&quot;&gt;$$U_{ij} \leq  \max_{k}\{U_{ik}, U_{kj}\} , \quad \forall i,j,k$$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$$U_{ij} \leq  \min_{k}\{U_{ik}, U_{kj}\} , \quad \forall i,j,k,$$&lt;/span&gt; which can then be re-expressed as two separate inequalities using a standard LP reformulation. The resulting problem will be an LP restriction of the problem you want to solve; it should solve quickly relative to your original problem, and if it has a solution, that solution will be feasible for your original problem, and its objective function value will be a lower bound on the optimal objective function value of your original problem.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try your luck on your formulation as is with a bundle solver for nonsmooth programs. I don't have a lot of experience with these types of solvers. (A colleague of mine uses them in his research.) They are probably slow, since they can't use derivative information. (I think they use subgradient or Clarke's generalized gradient information instead.) It is also unlikely that you will be able to solve large problem instances with a bundle solver.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-02-26T05:34:15.117" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1385" PostTypeId="2" ParentId="1337" CreationDate="2012-02-22T17:08:06.073" Score="3" Body="&lt;p&gt;There isn't any repository in which a comprehensive collection of 2D shapes are contained that I know of, most of the time, you need to generate the mesh by yourself for your research. While for some applied problems in mechanical engineering, you may need some commercial CAD software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are some free mesh generating packages which could handle arbitrary piecewise defined geometries in 2D. I'll name two here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://persson.berkeley.edu/distmesh/&quot; rel=&quot;nofollow&quot;&gt;DistMesh&lt;/a&gt;, a very compact package written in C++ and MATLAB/Octave by P.-O. Persson, you could generate arbitrary 2D triangulations based on a distance function that describes what the boundary of the domain of your interest is like. In Persson's &lt;a href=&quot;http://persson.berkeley.edu/distmesh/persson04mesh.pdf&quot; rel=&quot;nofollow&quot;&gt;short introductory note&lt;/a&gt;, he showed the MATLAB implementation, also gave many examples of how to cook out a distance function for a certain shape, also a mesh size function that makes your mesh &quot;better&quot;, ie finer in the corners, edges.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.hpfem.jku.at/netgen/&quot; rel=&quot;nofollow&quot;&gt;NETGEN&lt;/a&gt; is a cross-platform mesh generator as a C++ library, it has a UI, lets you import your own geometry/data, here is the &lt;a href=&quot;http://www.hpfem.jku.at/netgen/ng4.pdf&quot; rel=&quot;nofollow&quot;&gt;user's manual&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Last, here is a link of a quite &lt;a href=&quot;http://www.robertschneiders.de/meshgeneration//software.html&quot; rel=&quot;nofollow&quot;&gt;huge list of mesh generators&lt;/a&gt; compiled by Dr. Robert Schneider. Hope my answer would help.&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2012-02-22T17:08:06.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1386" PostTypeId="1" AcceptedAnswerId="1392" CreationDate="2012-02-22T20:15:07.847" Score="5" ViewCount="352" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Possible Duplicate:&lt;/strong&gt;&lt;br&gt;&#xA;  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/152/what-programming-paradigms-should-i-be-investing-in-if-i-want-my-code-to-run-on&quot;&gt;What programming paradigms should I be investing in if I want my code to run on petascale machines in the future?&lt;/a&gt;  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;Having entered the multi-core era (som already refers to as the many-core era) for CPUs, I am wondering what would be a good (the best?) future message interface model to be used for writing portable and scalable massively parallel applications on such future many-core CPUs? &#xA;We can only speculate about it, but, what are the pros/cons?&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-02-23T03:09:35.333" Title="Will MPI continue to be a popular basis writing scalable massively parallel solvers on future many-core CPUs?" Tags="|software|hpc|" AnswerCount="5" CommentCount="3" ClosedDate="2012-02-23T10:54:50.143" ContentLicense="CC BY-SA 3.0" />
  <row Id="1387" PostTypeId="2" ParentId="1386" CreationDate="2012-02-22T20:38:10.933" Score="4" Body="&lt;p&gt;Last year, David Keyes gave a talk at my school titled &quot;Exaflop/s, Seriously!&quot; When I was searching for the title of the talk, I found that &lt;a href=&quot;http://www.youtube.com/watch?v=RYxNVX9SYtk&quot; rel=&quot;nofollow&quot;&gt;a recorded version&lt;/a&gt; of the talk that he gave at another school. &lt;a href=&quot;http://www.cespr.fsu.edu/lighthill/keyes_publiclecture.pdf&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; are the slides from the talk he gave at my school. &lt;a href=&quot;http://ipac.kacst.edu.sa/eDoc/2011/191646_1.pdf&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is the paper (without all the pretty graphics) he references in the slides.&lt;/p&gt;&#xA;" OwnerUserId="739" LastActivityDate="2012-02-22T20:38:10.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1388" PostTypeId="2" ParentId="1386" CreationDate="2012-02-22T21:33:59.377" Score="5" Body="&lt;p&gt;At present, there is no serious contender for distributed memory parallelism, especially when working with libraries. The fact that research languages continue to eschew concepts like communicators and attribute caching (crucial for libraries) suggests that it will continue to dominate for the foreseeable future. My experience has been that MPI consistently performs better and that the MPI Forum has a better understanding of the issues at large scale than any other community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case is much less clear for shared memory. On the one hand, memory locality is crucial to get decent performance on systems with a deep NUMA hierarchy. MPI is good for that. On the other hand, the amount of memory per core (or hardware thread) is decreasing rapidly, so strong scalability is becoming more important. The best solution is dependent on the memory hierarchy and details of the job, but it appears that threads accessing shared memory without always making local copies is frequently important. Whether the eventual solution will be OpenCL kernels or more conventional shared memory models has yet to be determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that for most applications, MPI will continue to be the dominant paradigm for distributed memory. A mostly-separate-memory programming model might be based on future revisions to MPI, but a model with lots of sharing will probably be independent. Library interoperability with threads is an issue that I expect will plague us for at least the next few years.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-22T21:33:59.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1389" PostTypeId="2" ParentId="1386" CreationDate="2012-02-22T21:45:15.480" Score="0" Body="&lt;p&gt;My 2 cents MPI will be popluar and used until some one comes up with a simpler solution , a so called parallel language or open source way to automaticlly parallelize serial code.&lt;/p&gt;&#xA;" OwnerUserId="291" LastActivityDate="2012-02-22T21:45:15.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1391" PostTypeId="1" AcceptedAnswerId="1395" CreationDate="2012-02-16T06:16:17.697" Score="30" ViewCount="5421" Body="&lt;p&gt;Are there any famous problems/algorithms in scientific computing that cannot be sped up by parallelisation? It seems to me whilst reading books on CUDA that most things can be.&lt;/p&gt;&#xA;" OwnerUserId="1017" OwnerDisplayName="RRs_Ghost" LastEditorUserId="276" LastEditDate="2012-02-23T06:33:38.767" LastActivityDate="2017-05-27T14:02:10.677" Title="Are there any famous problems/algorithms in scientific computing that cannot be sped up by parallelisation" Tags="|parallel-computing|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1392" PostTypeId="2" ParentId="1386" CreationDate="2012-02-23T00:11:26.923" Score="6" Body="&lt;p&gt;I think everyone is clear about the fact that MPI can't be the model of the future if we want to use a million-way parallelism. You can read as much in every one of David Keyes' talks, or in any of the many reports on high performance computing over the past decade. It's just that nobody has much else to offer, or just much of an idea how we would structure communication in a better way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My personal hope is that whatever eventually replaces MPI is at a higher level. MPI is good for sending ints and doubles around, but not for objects. That isn't so difficult, though, as the BOOST MPI bindings show. I'd hope that any more modern replacement would have something like this and doesn't, as MPI has, dabble for 20 years on low-level stuff.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-02-23T00:11:26.923" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1393" PostTypeId="2" ParentId="1376" CreationDate="2012-02-23T00:15:51.970" Score="1" Body="&lt;p&gt;As a note, most supercomputers today deal with CPU requirements on the order of hundreds of thousands of CPU hours. I don't know what you need but if you want to run &quot;multiple&quot; 6-hour jobs at the same time, the simplest method may simply be to buy a $800 quad-core desktop without monitor and run things on there. Or find someone who has such a desktop to give away.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-02-23T00:15:51.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1394" PostTypeId="2" ParentId="1386" CreationDate="2012-02-23T03:00:46.297" Score="5" Body="&lt;p&gt;I think I agree with what most others are saying, which is, roughly:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lord, I hope not.&lt;/li&gt;&#xA;&lt;li&gt;But yeah, probably.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The thing is, MPI is a really good middleware for some other language to compile down to; but it's shocking that in 2012 we're still expecting our fellow scientists to write raw MPI calls.   And I'm not convinced that it's going to get any better; people worry about writing million-way parallelism with MPI, but (a) they don't suggest an alternative, and (b) you wouldn't do million way parallelism with a flat MPI decomposition; you'd program your 128-core nodes with OpenMP (or MPI-3 on-node communicators, or whatever) and use tradaditional MPI to do 8192-way parallelism across nodes, which suddenly seems not all that daunting.   Even if you make it 1024-way nodes and 131,072 of them, which gets you up to 130 million, nothing there is clearly impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the magical language that allows you to express parallelism so clearly that it can be efficiently parallelized down to distributed memory for you never materialized.   Frankly, at this point I think it's pretty clear that it's not going to happen, and the best we can hope for is domain-specific languages; but even there, I dispair that the &quot;domains&quot; there aren't going to be, say &quot;fluid dynamics&quot; but rather &quot;regular-grid finite-difference&quot;; eg, still the wrong level of abstraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the recent attempts have been interesting as in &quot;wow, we learned a lot from that effort&quot; but not interesting as in &quot;wow, let's use that for our next project.&quot;   The people who work with it think &lt;a href=&quot;http://charm.cs.uiuc.edu/research/charm&quot;&gt;Charm++&lt;/a&gt; is great, and the idea seems good - specify the finest level of parallelism and let the runtime worry about assembling that and locality - but it still seems to me like yet another in a series of frameworks that are &quot;perfectly general&quot; as long as you restrict yourself to considering problems that are well-suited to that kind of framework.    Stuff like &lt;a href=&quot;http://gcc.gnu.org/wiki/Coarray&quot;&gt;coarray fortran&lt;/a&gt; is certainly better than raw mpi, but not enormously -- you still have to decompose the domain and figure out which sub-domain you need to communicate with, you just don't have to write the message-passing yourself.  So that's something, but...   &lt;a href=&quot;http://upc.lbl.gov/&quot;&gt;UPC&lt;/a&gt; is interesting, in that in some ways its more ambitious -- you're just supposed to pretend you're writing a serial code and have the array decomposition done for you -- but bolting those global distributed array concepts onto a language that &lt;em&gt;barely&lt;/em&gt; has the concept of 1d arrays, and no concept &lt;em&gt;at all&lt;/em&gt; of higher-dimensional arrays, was so obviously doomed to failure that you wonder what people were thinking.   &lt;a href=&quot;http://x10-lang.org/&quot;&gt;X10&lt;/a&gt; is just bizarre; how is a program language where you're manually responsible for doing all the forking and joining of threads going to help you with million-way parallelism?  Also, java.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The one that strikes me the most interesting right now is &lt;a href=&quot;http://chapel.cray.com/&quot;&gt;Chapel&lt;/a&gt;, but that's probably because I'm still trying to find time to play with it.   Chapel seems to have really nice mechanisms for separating the code describing the decomposition of the data from the code describing what you do with the data; and it comes with several very common decompositions &quot;built in&quot;.   It, more than the others, seems like it's approaching the problem the right way around; letting you re-do decompositions without rewriting all of your code, and having several reasonable decompositions baked in.   &lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastEditorDisplayName="user389" LastEditDate="2012-02-23T03:09:35.333" LastActivityDate="2012-02-23T03:09:35.333" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1395" PostTypeId="2" ParentId="1391" CreationDate="2012-02-23T03:46:56.133" Score="34" Body="&lt;p&gt;The central issue is the length of the critical path $C$ relative to the total amount of computation $T$. If $C$ is proportional to $T$, then parallelism offers at best a constant speed-up. If $C$ is asymptotically smaller than $T$, there is room for more parallelism as the problem size increases. For algorithms in which $T$ is polynomial in the input size $N$, the best case is $C \sim \log T$ because very few useful quantities can be computed in less than logarithmic time.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Examples&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$C = T$ for a tridiagonal solve using the standard algorithm. Every operation is dependent on the previous operation completing, so there is no opportunity for parallelism. Tridiagonal problems can be solved in logarithmic time on a parallel computer using a nested dissection direct solve, multilevel domain decomposition, or multigrid with basis functions constructed using harmonic extension (these three algorithms are distinct in multiple dimensions, but can exactly coincide in 1D).&lt;/li&gt;&#xA;&lt;li&gt;A dense lower-triangular solve with an $m\times m$ matrix has $T = N = \mathcal O(m^2)$, but the critical path is only $C = m = \sqrt T$, so some parallelism can be beneficial.&lt;/li&gt;&#xA;&lt;li&gt;Multigrid and FMM both have $T = N$, with a critical path of length $C = \log T$.&lt;/li&gt;&#xA;&lt;li&gt;Explicit wave propagation for a time $\tau$ on a regular mesh of the domain $(0,1)^d$ requires $k = \tau / \Delta t \sim \tau N^{1/d}$ time steps (for stability), therefore the critical path is at least $C = k$. The total amount of work is $T = k N = \tau N^{(d+1)/d}$. The maximum useful number of processors is $P = T/C = N$, the remaining factor $N^{1/d}$ cannot be recovered by increased parallelism.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Formal complexity&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/NC_%28complexity%29&quot;&gt;NC complexity class&lt;/a&gt; characterizes those problems that can be solved efficiently in parallel (i.e., in polylogarithmic time). It is unknown whether $NC = P$, but it is widely hypothesized to be false. If this is indeed the case, then &lt;a href=&quot;http://en.wikipedia.org/wiki/P-Complete&quot;&gt;P-complete&lt;/a&gt; characterizes those problems that are &quot;inherently sequential&quot; and cannot be sped up significantly by parallelism.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-03-03T18:58:09.067" LastActivityDate="2013-03-03T18:58:09.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1396" PostTypeId="1" CreationDate="2012-02-23T05:04:58.107" Score="4" ViewCount="240" Body="&lt;p&gt;Let $H = \triangle + V(x) : \mathbb{R}^2 \rightarrow \mathbb{R}^2$. I am interested in domain decomposition for an eigenproblem involving $H$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lowest 1000 eigenfunctions of $H$,  $ \psi_i $, can be partitioned using a region, $\Omega \subset \mathbb{R}^2$, such that each $\psi_i$ localizes either inside of $\Omega$ or outside of $\Omega$. $\Omega$ is not a subspace of $\mathbb{R}^2$ as it may be an oddly shaped region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Label the inner eigenfunctions $\psi_i^{in}$ and the outer ones $\psi_i^{out}$. There's only about 10 $\psi_i^{in}$s. Given $\Omega$, my goal is to efficiently compute the $\psi_i^{in}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to find the $\psi_i^{in}$ would be to discretize, compute all 1000 $\psi_i$s, and then partition. This is what I do now (5-point stencil for $\triangle$ on a $10^3 \times 10^3$ grid). The problem is that this requires diagonalizing over a 1000 dimensional space in order to get 10 eigenvectors. It seems like there would be a cheaper way to compute the $\psi_i^{in}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know an existing method that solves this problem? Also, this is a repost from &lt;a href=&quot;https://mathoverflow.net/questions/88171/efficiently-computing-a-few-localized-eigenvectors&quot;&gt;https://mathoverflow.net/questions/88171/efficiently-computing-a-few-localized-eigenvectors&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; I think I can solve this if I can at least figure a way to solve&#xA;\begin{equation}&#xA;\max \psi^T H \psi \text{ subject to } P\psi = \psi \text{ and } \psi^T \psi = 1&#xA;\end{equation}&#xA;where $P$ is projection onto the space of functions localized over $\Omega$. If this is doable then something like inverse iteration should be doable which will give me what I want.&lt;/p&gt;&#xA;" OwnerUserId="979" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2012-03-26T19:08:28.037" Title="Efficiently computing a few localized eigenvectors" Tags="|pde|matrix|eigensystem|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1397" PostTypeId="2" ParentId="152" CreationDate="2012-02-23T10:57:15.100" Score="1" Body="&lt;p&gt;I was just about to post this answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1386/will-mpi-continue-to-be-a-popular-basis-writing-scalable-massively-parallel-solv&quot;&gt;this question&lt;/a&gt;, but it was closed as a duplicate of this one, so here goes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may sound a bit Solomonic, but in my experience, the future belongs to &lt;em&gt;hybrid&lt;/em&gt; approaches in which several shared-memory multi-core nodes running multi-threaded kernels are connected via a distributed-memory paradigm such as MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, however, a few problems, and they do not involve the hardware at all. First of all, most parallel programmers are heavily invested in MPI-type codes and are very reluctant to be the first to re-implement parts, or all, of their code-base using a new paradigm. The lack of people using shared-memory approaches leads to slower progress in algorithms for that area, which make any investment seem even more pointless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second problem is that everybody associates shared-memory parallelism with &lt;a href=&quot;http://en.wikipedia.org/wiki/OpenMP&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenMP&lt;/a&gt;. While OpenMP is a nice quick-and-dirty way to solve small, simple problems on a small number of processors, it's an absolutely terrible programming model for &lt;em&gt;real&lt;/em&gt; shared-memory parallelism. Although we've all, at some point or another, learned a number of simple and efficient parallel programming paradigms, e.g. &lt;a href=&quot;http://en.wikipedia.org/wiki/Thread_pool_pattern&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thread pools&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Scheduler_pattern&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schedulers&lt;/a&gt;, these are not easy to implement using OpenMP and, quite frankly, this is not the type of parallelism that OpenMP entices programmers to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, the barrier for moving from a purely distributed-memory to a purely/partially shared-memory paradigm is quite high. If you want to use threads efficiently, you have to forget OpenMP and manage threads and concurrency yourself (hello &lt;a href=&quot;http://en.wikipedia.org/wiki/Pthreads&quot; rel=&quot;nofollow noreferrer&quot;&gt;pthreads&lt;/a&gt;, goodbye Fortran).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why move to a hybrid approach at all? Well, although MPI scales to thousands of cores, the underlying model is one of lock-step synchronicity and static communication patterns. This is good for some problems, e.g. billion-particle simulations, but sub-optimal for more difficult or finer-grained problems. Shared-memory paradigms make dynamic load balancing and/or asynchronous communication much easier, but doing involves a major programming effort.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-02-23T10:57:15.100" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1398" PostTypeId="2" ParentId="1396" CreationDate="2012-02-23T12:59:33.530" Score="0" Body="&lt;p&gt;&lt;strong&gt;EDIT: I misunderstood the question. This answers a different question of spectral locality, not locality in the spatial domain.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know a way to guarantee that you have found all eigenpairs in a particular region, but you can certainly ask for those with certain properties. The table below is from the &lt;a href=&quot;http://www.grycap.upv.es/slepc/documentation/manual.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;SLEPc&lt;/a&gt; manual. Eigenvalues in the interior are more difficult to compute, so you will normally need to use methods like harmonic extension (available with some eigensolver methods, does not change the way the subspace is built) or shift-and-invert (more powerful, uses a transformed problem). Robust, scalable implementations of these methods are available within SLEPc. Note that MATLAB &lt;code&gt;eigs()&lt;/code&gt; can be used for smaller problems. For finding a large number of interior eigenpairs, you might consider a method like &lt;a href=&quot;http://www.mcs.anl.gov/uploads/cels/papers/P1263.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zhang, Smith, Sternberg, and Zapol (2007)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/prjLE.png&quot; alt=&quot;SLEPc eigenvalue selection&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-03-26T19:08:28.037" LastActivityDate="2012-03-26T19:08:28.037" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1399" PostTypeId="2" ParentId="1396" CreationDate="2012-02-23T13:21:16.140" Score="0" Body="&lt;p&gt;As per your edit you should be able to do the following:&#xA;$$ G=H-\|H\|I $$&#xA;Now solve&#xA;$$PGP\psi=\lambda \psi$$&#xA;for the first $n$ eigenpairs, where $\psi\in L^2(\Omega)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The shift ensures that $G$ only has non-positive Eigenvalues. The projector then maps all exterior functions to 0. Hence the lowest eigenvectors are your requested eigenvectors with respective eigenvalues $\lambda+\|H\|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rewrite the problem: $$\max_{\lambda,\psi} \|P\psi\|^2 s.t. \|\psi\|^2=1, \|H\psi-\lambda\psi\|^2=0$$ Now you have a constrained non-linear problem in which all your functions are convex (in each variable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can use a non-linear optimizer (e.g., &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CCoQFjAA&amp;amp;url=http%3A%2F%2Fwww.coin-or.org%2FIpopt%2F&amp;amp;ei=G6JHT5aDEqXt0gGVtoCXDg&amp;amp;usg=AFQjCNGndSOuugso4bEjPUjearFsqRkhtw&quot; rel=&quot;nofollow&quot;&gt;IPOPT&lt;/a&gt;). If you're using a mesh on $\mathbb{R}^2$, all involved operators should be sparse.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2012-02-24T14:47:25.260" LastActivityDate="2012-02-24T14:47:25.260" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1400" PostTypeId="1" CreationDate="2012-01-23T15:01:54.493" Score="11" ViewCount="681" Body="&lt;p&gt;I am trying to implement the fourth order Runge-Kutta method for solving a first order ODE in Python i.e. $\frac{dy}{dx} = f(x,y)$. I understand how the method works, but am trying to write an efficient algorithm that minimises the number of times $f(x,y)$ is calculated as this is quite costly. I have been told that it is possible to reuse data points that were previously calculated as you increment over the steps but can’t see how. Does anyone know how to do this or is it not possible?&lt;/p&gt;&#xA;" OwnerUserId="1158" OwnerDisplayName="Josh" LastEditorUserId="81" LastEditDate="2012-02-23T17:30:33.793" LastActivityDate="2012-03-04T18:18:01.667" Title="Runge-Kutta and Reusing Datapoints" Tags="|ode|python|explicit-methods|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1401" PostTypeId="2" ParentId="1400" CreationDate="2012-01-23T17:03:24.253" Score="8" Body="&lt;p&gt;If you are going from &lt;code&gt;yp_1 = f(x_1, y_1)&lt;/code&gt; to &lt;code&gt;yp_2 = f(x_1+h, y_2)&lt;/code&gt; you are going to need the intermediate points:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K1 = f(x_1+h/2, y_1+h/2*yp_1)&#xA;K2 = f(x_1+h/2, y_1+h/2*K1)&#xA;K3 = f(x_1+h, y_1+h*K2)&#xA;&#xA;x_2 = x_1 + h&#xA;y_2 = y_1 + h/6*(yp_1+2*K1+2*K2+K3)&#xA;yp_2 = f(x_2, y_2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In general none of the intermediate points are useful in the next step. Because  &lt;code&gt;K1&lt;/code&gt;&amp;lt;&gt;&lt;code&gt;K2&lt;/code&gt; and &lt;code&gt;K3&lt;/code&gt;&amp;lt;&gt;&lt;code&gt;yp_2&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1030" OwnerDisplayName="ja72" LastActivityDate="2012-01-23T17:03:24.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1402" PostTypeId="1" CreationDate="2012-02-01T16:31:08.587" Score="8" ViewCount="1008" Body="&lt;p&gt;I know that I should use a tolerance for comparing floating point numbers.  But for comparing vectors, I can think of 3 possible solutions corresponding to different distance metrics:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compare the components of each vector individually: the vectors are equal if all 3 are within tolerance.  This option would behave like the &lt;a href=&quot;http://en.wikipedia.org/wiki/Uniform_norm&quot;&gt;uniform norm&lt;/a&gt;, giving a cube of tolerance.&lt;/li&gt;&#xA;&lt;li&gt;Compare the sum of all the absolute differences to some tolerance.  This would behave like the &lt;a href=&quot;http://en.wikipedia.org/wiki/L1_norm&quot;&gt;taxicab norm&lt;/a&gt;, giving a simplex of tolerance.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the Euclidean length of (vecA - vecB) and see if it's within tolerance.  This would give the standard Euclidean norm with a sphere of tolerance.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But my main concern is numerical stability.  Euclidean norm &quot;feels like&quot; the best option, but I'm worried that all the calculations would induce more rounding errors.  To a lesser extent option 2 could also introduce errors.  (For example, if the &lt;em&gt;x&lt;/em&gt; component of the vectors is much larger than &lt;em&gt;y&lt;/em&gt; and &lt;em&gt;z&lt;/em&gt;, adding together all the differences could swamp any contributions from &lt;em&gt;y&lt;/em&gt; and &lt;em&gt;z&lt;/em&gt;.)  So I'm currently leaning towards option 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone weigh in with an authoritative take on this problem?&lt;/p&gt;&#xA;" OwnerDisplayName="japreiss" LastActivityDate="2012-02-23T18:21:37.003" Title="What's the right way to compare vectors in floating-point?" Tags="|floating-point|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1403" PostTypeId="2" ParentId="1402" CreationDate="2012-02-23T18:21:37.003" Score="5" Body="&lt;p&gt;The answer to this question depends largely on your application.  Instead of sweating the exact numerical implementation think more about what each potential embodiment you suggested means.  For example, does one or more of the computed distances have a physical interpretation?  Are the units and scales of the vector components the same?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the scenario where all parameters have the same units, and different scales of values, you should think more about the best way to dimension your system.   &lt;a href=&quot;http://en.wikipedia.org/wiki/Nondimensionalization&quot;&gt;Nondimensionalization&lt;/a&gt; helps to chase away the evil numerical precision gnomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you use x, y, and z as value names, I suspect you are looking at some sort of position in space.  The two norm has the distinct advantage of having a continuous derivative, so lacking any real problem information I would probably start there.   &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-02-23T18:21:37.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1404" PostTypeId="2" ParentId="1400" CreationDate="2012-02-23T19:25:15.067" Score="4" Body="&lt;p&gt;In general explicit Runge-Kutta methods of order $N$ require &lt;em&gt;at least&lt;/em&gt; $N$ function evaluations, and there is absolutely no way to avoid this. Past $N=4$ they require more than $N$ function evaluations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to re-use past function evaluations, you need to use a multistep method like Adams-Bashforth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case you pay for each strategy. Single step methods requires largest number of function evaluations, but multistep methods have the largest memory requirement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Correction. My statement is true only for explicit methods. The situation is less clear for implicit methods since the number of stages doesn't translate directly to number of function evaluations.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2012-02-25T19:40:26.410" LastActivityDate="2012-02-25T19:40:26.410" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1406" PostTypeId="1" AcceptedAnswerId="1409" CreationDate="2012-02-24T00:06:22.627" Score="4" ViewCount="390" Body="&lt;p&gt;I am just learning about these packages, so forgive me if this is a trivial/silly question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our group is working re-developing our code from the ground up using modern software practices. Currently it is an explicit wave propagation code for earthquake source physics, but more and more our group is working on other coupled seismic source problems. In the future we will need to ability to handle portions of the domain implicitly, use unstructured grids, and (like many others in the numerical PDE world) will consider using GPUs. Thus we really want to make the right design decisions to facilitate this from the get go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the big questions were are looking at is which of the big national lab packages to develop with in mind, PETSc or Trilinos. One of the question that I have been seeking an answer to is how easy it is to overlap computation, I/O, and communication with these packages. For instance, if we used the distributed array data structures of PETSc does this means that we will not be able to exchange our halo cells while applying our stencil to the rest of the domain? (Similarly with the EPetra/TPetra types of Trilinos.)&lt;/p&gt;&#xA;" OwnerUserId="880" LastActivityDate="2012-02-27T03:50:30.967" Title="Overlapping communication and computation in PETSc and/or Trilinos?" Tags="|pde|software|petsc|trilinos|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1408" PostTypeId="2" ParentId="1406" CreationDate="2012-02-24T01:56:20.600" Score="3" Body="&lt;p&gt;Clearly, I need to learn how to search better!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In PETSc this can be accomplished (I believe) with (pg 49 of the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf&quot; rel=&quot;nofollow&quot;&gt;users manual&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   DMGlobalToLocalBegin(DM da,Vec g,InsertMode iora,Vec l);&#xA;   DMGlobalToLocalEnd(DM da,Vec g,InsertMode iora,Vec l);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="880" LastActivityDate="2012-02-24T01:56:20.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1409" PostTypeId="2" ParentId="1406" CreationDate="2012-02-24T02:08:50.627" Score="4" Body="&lt;p&gt;Yes, many communication functions are split into begin/end pairs. A typical example is the code for &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/mat/impls/aij/mpi/mpiaij.c.html#MatMult_MPIAIJ&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;MatMult_MPIAIJ()&lt;/code&gt;&lt;/a&gt; (the standard row-based format).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VecScatterBegin(a-&amp;gt;Mvctx,xx,a-&amp;gt;lvec,INSERT_VALUES,SCATTER_FORWARD);&#xA;(*a-&amp;gt;A-&amp;gt;ops-&amp;gt;mult)(a-&amp;gt;A,xx,yy);            /* multiply the diagonal block */&#xA;VecScatterEnd(a-&amp;gt;Mvctx,xx,a-&amp;gt;lvec,INSERT_VALUES,SCATTER_FORWARD);&#xA;(*a-&amp;gt;B-&amp;gt;ops-&amp;gt;multadd)(a-&amp;gt;B,a-&amp;gt;lvec,yy,yy); /* add contribution from off-diagonal block */&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-24T02:08:50.627" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1410" PostTypeId="2" ParentId="39" CreationDate="2012-02-24T02:51:04.257" Score="1" Body="&lt;p&gt;Here is complete (with the exception of sparse matrices) package for linear algebra in arbitrary precision:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.advanpix.com/&quot; rel=&quot;nofollow&quot; title=&quot;Multiprecision Computing Toolbox for MATLAB&quot;&gt;Multiprecision Computing Toolbox for MATLAB&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It integrates smoothly with Matlab and provides routines for all common operations from determinants to SVD and eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually it also covers other areas - numerical integration, optimization, ode, special functions, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for function minimization you could try Nelder–Mead simplex method (&lt;code&gt;fminsearch&lt;/code&gt;) implemented in the toolbox too. &lt;/p&gt;&#xA;" OwnerUserId="1033" LastActivityDate="2012-02-24T02:51:04.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1412" PostTypeId="2" ParentId="1400" CreationDate="2012-02-24T10:55:47.317" Score="0" Body="&lt;p&gt;Please check on &quot;embedded&quot; methods: the aim in this type of RK methods is to have two methods with different orders, where the high order method uses the same function evaluations as the low order method. This allows for very efficient error estimation. See p.165 and further of &quot;Solving Ordinary Differential Equations I: Nonstiff problems&quot; by Hairer, Norsett and Wanner. Typical examples are Fehlberg methods of order 7(8).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you are looking at solving ODEs in PYTHON, check out &lt;a href=&quot;http://svn.jmodelica.org/assimulo&quot; rel=&quot;nofollow&quot;&gt;assimulo&lt;/a&gt;. I've been playing with this package for a couple of weeks and am quite happy.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-02-24T10:55:47.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1413" PostTypeId="1" AcceptedAnswerId="1418" CreationDate="2012-02-24T15:29:54.893" Score="3" ViewCount="1158" Body="&lt;p&gt;An algorithm I'm writing needs to compute rolling quantiles of a time series. Currently I do this in the naive way: for a window of size &lt;code&gt;W&lt;/code&gt; and a vector &lt;code&gt;X&lt;/code&gt; of size &lt;code&gt;N&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for t from W to N:&#xA;  q[t,:] = quantiles(X[t-W+1:t])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it seems that there should be a faster way, given that I know the previous quantile &lt;code&gt;q[t-1]&lt;/code&gt;, the new data &lt;code&gt;X[t]&lt;/code&gt; and the data &lt;code&gt;X[t-W]&lt;/code&gt; that has just dropped out of the window. I'm thinking of something along the lines of the well (?) known incremental mean algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for t from W to N:&#xA;  m[t] = m[t-1] + ( x[t] - x[t-W] ) / W&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which avoids recalculating the mean at every stage. Even an approximation would be good. I've seen approximations for computing quantiles on &lt;em&gt;streams&lt;/em&gt;, but it is important to me that I have a rolling window, not just an expanding window.&lt;/p&gt;&#xA;" OwnerUserId="991" LastEditorUserId="9" LastEditDate="2012-02-24T16:44:07.190" LastActivityDate="2012-02-24T16:44:07.190" Title="Computing a rolling quantile" Tags="|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1415" PostTypeId="2" ParentId="1380" CreationDate="2012-02-24T16:10:22.177" Score="3" Body="&lt;p&gt;The solution to your question is $-\infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $$U=\left(\begin{matrix} 1&amp;amp;\cdots &amp;amp;1\\ \vdots &amp;amp; &amp;amp; \vdots\\1&amp;amp;\cdots&amp;amp;1\end{matrix}\right).$$&#xA;Since $A\cdot\mbox{vec}(U)$ and your constraints are linear in $U$, any positive multiple $t$ of $\pm U$ satisfies the constraints. Therefore, $\min_V(A\cdot\mbox{vec}(V)) \le min_{t}(A\cdot \mbox{vec}(tU))=-\infty$.&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2012-02-24T16:10:22.177" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1418" PostTypeId="2" ParentId="1413" CreationDate="2012-02-24T16:43:09.733" Score="4" Body="&lt;p&gt;One solution would be to keep your rolling window of data sorted using a &lt;a href=&quot;http://en.wikipedia.org/wiki/Self-balancing_binary_search_tree&quot; rel=&quot;nofollow&quot;&gt;self-balancing binary tree&lt;/a&gt; or a &lt;a href=&quot;http://en.wikipedia.org/wiki/Hash_table&quot; rel=&quot;nofollow&quot;&gt;hash table&lt;/a&gt;.  On every update, you pay a constant insertion cost, a $O(n)$ deletion cost, and a $O(n)$ traversal cost to update the quintiles.  &lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-02-24T16:43:09.733" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1419" PostTypeId="1" AcceptedAnswerId="1422" CreationDate="2012-02-24T20:08:02.527" Score="6" ViewCount="326" Body="&lt;p&gt;Suppose we are dealing with a nonlinear problem, say &#xA;$$&#xD;&#xA;A u := L u + G(u) = f&#xD;&#xA;$$&#xA;the nonlinearity of the operator $A$ is the polynomial type, ie, $L$ is a linear operator, and  $G(u) = u^k$, or more generally we have a Fréchet derivative bound like:&#xA;$$&#xD;&#xA;\big\|A(u) - A(v) -DA(u)(u-v) \big\|\leq C \|A(u) - A(v)\| &#xD;&#xA;$$&#xA;where $D$ is the Fréchet derivative for the operator $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would like to use some numerical method together with an iterative solver to get a finite dimensional approximation to this problem. For a two-grid cycles multigrid, before we compute the residual at a finer level of mesh size $h$(vs coarse level mesh size $2h$), according to the &lt;a href=&quot;http://ec-securehost.com/SIAM/ot82.html&quot;&gt;book written by Saad&lt;/a&gt; chapter 13, we would like to apply a presmoothing operator $\mathcal{S}$ to the initial guess $u^h_0$ at the finer level:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Presmooth: $u^h := \mathcal{S}(A_h,u^h_0,f^h)$&lt;/li&gt;&#xA;&lt;li&gt;Compute the residual: $r^h = f^h - A_h u^h$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then in next few steps we could go back the coarse mesh $2h$ to solve and correct $u^h$ at the finer level. Here $A_h$ is a finite dimensional linearization matrix of the operator $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my questions are: What are the criteria for choosing a &quot;good&quot; presmooth operator for nonlinear multigrid method? and how to construct this presmooth operator? Is there a routine to follow? Appreciate if you could share me some references having examples similar to this type of problem.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-02-24T23:27:37.740" LastActivityDate="2012-02-24T23:31:27.827" Title="Question about the smoothing operators in multigrid methods for nonlinear PDEs" Tags="|linear-algebra|pde|multigrid|iterative-method|reference-request|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1420" PostTypeId="2" ParentId="1380" CreationDate="2012-02-24T20:42:56.033" Score="1" Body="&lt;p&gt;I can't find the comment button...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Geoff pointed out, it is a concave constraint function. However, it doesn't matter if the function itself is concave or not. Concave functions under linear constraints can be convex sets (   e.g. &lt;span class=&quot;math-container&quot;&gt;$log(x)&amp;lt;5$&lt;/span&gt;  ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is a convex set, you could perform gradient descent on your objective function, using something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Dykstra%27s_projection_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dykstra's_projection_algorithm&lt;/a&gt; to project back onto the space of constraints.&lt;/p&gt;&#xA;" OwnerUserId="1042" LastEditorUserId="17088" LastEditDate="2018-10-28T08:43:52.323" LastActivityDate="2018-10-28T08:43:52.323" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="1421" PostTypeId="1" CreationDate="2012-02-24T21:11:13.507" Score="12" ViewCount="11986" Body="&lt;p&gt;Suppose I have two matrices Nx2, Mx2 representing N, M 2d vectors respectively.&#xA;Is there a simple and good way to calculate distances between each vector pair (n, m)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easy but inefficient way is of course:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d = zeros(N, M);&#xA;for i = 1:N,&#xA;  for j = 1:M,&#xA;    d(i,j) = norm(n(i,:) - m(j,:));&#xA;  endfor;&#xA;endfor;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The closest answer I've found is &lt;code&gt;bsxfun&lt;/code&gt;, used like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bsxfun(inline(&quot;x-y&quot;),[1,2,3,4],[3;4;5;6])&#xA;&#xA;ans =&#xA;  -2 -1  0  1&#xA;  -3 -2 -1  0&#xA;  -4 -3 -2 -1&#xA;  -5 -4 -3 -2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1043" LastEditorUserId="9" LastEditDate="2012-04-09T22:09:05.413" LastActivityDate="2012-07-18T16:12:10.807" Title="Octave: calculate distance between two matrices of vectors" Tags="|performance|octave|vectorization|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1422" PostTypeId="2" ParentId="1419" CreationDate="2012-02-24T23:31:27.827" Score="7" Body="&lt;p&gt;Brandt's book &lt;em&gt;Multigrid techniques&lt;/em&gt; has a good discussion of the Full Approximation Scheme, the 1982 &lt;em&gt;Guide to Multigrid Development&lt;/em&gt; also has some discussion. As with linear problems, the requirement is for the smoothers to remove high-frequency components of the residual. The formal spectral analysis only applies to linearized problems, but FAS theory only needs local linearization. Trottenberg, Oosterlee, and Schueller's 2001 book on multigrid is also a good reference and contains local Fourier analysis for nonlinear problems.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-24T23:31:27.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1423" PostTypeId="2" ParentId="1391" CreationDate="2012-02-24T23:34:54.180" Score="10" Body="&lt;p&gt;Start by grocking &lt;a href=&quot;http://en.wikipedia.org/wiki/Amdahl%27s_law&quot;&gt;Amdahl's Law&lt;/a&gt;.  Basically anything with a large number of serial steps will benefit insignificantly from parallelism.  A few examples include parsing, regex, and most high-ratio compression.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aside from that, the key issue is often a bottleneck in memory bandwidth.  In particular with most GPU's your theoretical flops vastly outstrip the amount of floating point numbers you can get to your ALU's, as such algorithms with low arithmetic intensity (flops / cache-miss) will spend a vast majority of time waiting on RAM.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, any time that a piece of code requires branching, it is unlikely to get good performance, as ALU's typically outnumber logic.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, a really simple example of something that would be hard to get a speed gain from a GPU is simply counting the number of zeros in a array of integers, as you may have to branch often, at most perform 1 operation (increment by one) in the case that you find a zero, and make at least one memory fetch per operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example free of the branching problem is to compute a vector which is the cumulative sum of another vector.  ( [1,2,1] -&gt; [1,3,4] ) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if these count as &quot;famous&quot; but there is certainly a large number of problems that parallel computing will not help you with.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-02-24T23:34:54.180" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1424" PostTypeId="2" ParentId="1391" CreationDate="2012-02-25T00:23:32.730" Score="13" Body="&lt;p&gt;To give a theoretical aspect to this, &lt;a href=&quot;https://en.wikipedia.org/wiki/NC_%28complexity%29&quot;&gt;$NC$&lt;/a&gt; is defined as the complexity class that is solvable in $O(log^c n)$ time on a system with $O(n^k)$ parallel processors. It is still unknown whether $P=NC$ (although most people suspect it's not) where $P$ is the set of problems solvable in polynomial time. The &quot;hardest&quot; problems to parallelize are known as $P$-complete problems in the sense that every problem in $P$ can be reduced to a $P$-complete problem via $NC$ reductions. If you show that a single $P$-complete problem is in $NC$, you prove that $P=NC$ (although that's probably false as mentioned above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So any problem that is $P$-complete would intuitively be hard to parallelize (although big speedups are still possible). A $P$-complete problem for which we don't have even very good constant factor speedups is Linear Programming (see &lt;a href=&quot;http://www.or-exchange.com/questions/312/gpu-and-mip-solvers/314&quot;&gt;this&lt;/a&gt; comment on OR-exchange).&lt;/p&gt;&#xA;" OwnerUserId="288" LastEditorUserId="288" LastEditDate="2012-02-25T01:15:29.140" LastActivityDate="2012-02-25T01:15:29.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1425" PostTypeId="2" ParentId="1380" CreationDate="2012-02-25T08:59:37.257" Score="1" Body="&lt;p&gt;Can't you introduce a slack variable? So to reformulate the constraint&#xA;  $$x_i  &amp;lt;=  \max(a_{i1}, a_{i2}, ..., a_{in})$$&#xA;write it as follows:&#xA;$$  x_i &amp;lt;= s_i$$&#xA;$$  s_i &amp;gt;= a_{i1}$$&#xA;$$  s_i &amp;gt;= a_{i2}$$&#xA;$$  ...$$&#xA;$$  s_i &amp;gt;= a_{in} $$&#xA;This will have an infeasible solution with respect to the original problem if you choose s=infinity. But I'm pretty sure you can show that if you add a term&#xA;$$  c*\max(s_i-\max(a_i), 0)$$&#xA;to the objective function (i.e., you want to have $s_i-\max(a_i)$ as small as possible, preferably zero) and $c$ sufficiently large, then you'll get back a feasible solution if the original problem had feasible solutions with objective value less than infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A proof would go along the lines of showing that if $s_i&amp;gt;=\max(a_i)$ and if $x_i=s_i$, then the solution is infeasible; in other words, $s_i-\max(a_i)$ is a measure of infeasibility wrt to the original problem. If the problem is stable, there should be a finite improvement in objective function value for a finite violation of feasibility. If you choose c to be larger than the ratio between change in objective value and violation of feasibility, then the modified objective function would &lt;em&gt;grow&lt;/em&gt; for problems that go into the infeasible region.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-02-25T08:59:37.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1426" PostTypeId="1" CreationDate="2012-02-25T10:45:12.213" Score="14" ViewCount="6726" Body="&lt;p&gt;I have a problem when I want to use the high order center difference approximation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(\frac{-u_{i+2,j}+16u_{i+1,j}-30u_{i,j}+16u_{i-1,j}-u_{i-2,j}}{12}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the Poisson equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(u_{xx}+u_{yy}=0)$$ in a square domain in which the boundary conditions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(0,y)=u(x,0)=u(x,1)=0,u(1,y)=\sin \pi y$$&#xA;$$\Delta{x}=\Delta{y}=0.1$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I want to obtain the value of inside points of domain, considering this approximation some points depend on the outside points of boundary. For example, $u_{1,1}$ needs to have the value of $u_{i-2,j}=u_{-1,0}$ a point which is outside of boundary. Can anybody please help me in this case?  &lt;/p&gt;&#xA;" OwnerUserId="587" LastEditorUserId="9" LastEditDate="2012-02-26T11:02:06.733" LastActivityDate="2015-07-12T13:16:16.210" Title="How to impose boundary conditions in finite difference methods" Tags="|pde|finite-difference|" AnswerCount="3" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1427" PostTypeId="2" ParentId="1426" CreationDate="2012-02-25T16:02:45.590" Score="9" Body="&lt;p&gt;There are other stencils that you can use to obtain a high order accuracy near the boundary points.  Your current stencil is of the form:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Au_{i+2,j} + Bu_{i+1,j} + Cu_{i,j}+Du_{i-1,j}+Eu_{i-2,j}$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, you can also use a different stencil near the boundary like this:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Au_{i+3,j} + Bu_{i+2,j}+Cu_{i+1,j}+Du_{i,j} +Eu_{i-1,j}$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;to compute the value at $u_{1,1}$.  Note that the coefficients in the second stencil will be different from the ones in the first formula.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, you can approximate the value at the opposite boundary by a similar formula.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-02-25T17:39:20.217" LastActivityDate="2012-02-25T17:39:20.217" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1428" PostTypeId="2" ParentId="1426" CreationDate="2012-02-25T16:44:11.580" Score="9" Body="&lt;p&gt;You may want to look into summation-by-parts (SBP) finite difference methods. Ken Mattsson has done a lot of work on these methods. Good place to start is &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2004.03.001&quot;&gt;here&lt;/a&gt; (constant coefficients) and &lt;a href=&quot;http://dx.doi.org/10.1007/s10915-011-9525-z&quot;&gt;here&lt;/a&gt; (variable coefficients).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically the way these methods work is they are the standard central methods in the interior and transition to one sided near the boundary. An important part of the SBP technology, is that the transition to one-sided is such that stability of the method for time dependent problems can be proven even after the inclusion of boundary conditions. (This is possible because the operators themselves &quot;define&quot; a norm, which mimics discretely integration by parts.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say that you are looking at Poisson's equation, I am not totally sure how boundary conditions are stably included with SBP operators and elliptic equations. I have a colleague who has played with these for elliptic problems and seems to indicate it doesn't really matter what you do.&lt;/p&gt;&#xA;" OwnerUserId="880" LastEditorUserId="880" LastEditDate="2012-02-27T16:05:56.587" LastActivityDate="2012-02-27T16:05:56.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1429" PostTypeId="1" AcceptedAnswerId="1448" CreationDate="2012-02-26T02:59:21.007" Score="10" ViewCount="18459" Body="&lt;p&gt;What is a good, common way to express dictionaries (= maps) in pseudocode? I.e. datastructures that basically allow to store values for keys, iterate over all key/value pairs, test for inclusion of a given key, etc. I have something like the following (in this case senseless) Python code in mind:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;D = {}&#xA;D[1] = 2&#xA;for key, value in D.items():&#xA;    # do something with key and value&#xA;if key in D:&#xA;    # do something&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I want to express it as pseudocode in a publication. Thinking mathematically, dictionaries are functions are relations are sets of pairs, so writing something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;D ← ∅&#xA;D[1] ← 2&#xA;for all (k, v) ∈ D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;would actually make sense. But is it understandable? And for the test, I'd use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if k ∈ keys(D)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or is it saver to be more literal, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;D ← empty dictionary&#xA;for all key-value pairs (k, v) in D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there existing good practice / any reference on how to write commonly understandable pseudocode?&lt;/p&gt;&#xA;" OwnerUserId="1048" LastActivityDate="2012-02-27T09:00:24.703" Title="Dictionaries in pseudocode" Tags="|algorithms|publications|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1430" PostTypeId="2" ParentId="861" CreationDate="2012-02-26T13:31:10.187" Score="4" Body="&lt;p&gt;I would try &lt;a href=&quot;http://computation.llnl.gov/casc/SAMRAI/index.html&quot; rel=&quot;nofollow&quot;&gt;SAMRAI&lt;/a&gt; I know at least one code that uses it with success — &lt;a href=&quot;http://code.google.com/p/ibamr/&quot; rel=&quot;nofollow&quot;&gt;IBAMR&lt;/a&gt;, an Immersed Boundary Method code for Fluid-Structure Interaction with AMR.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="1064" LastEditDate="2012-03-02T09:17:16.810" LastActivityDate="2012-03-02T09:17:16.810" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1431" PostTypeId="2" ParentId="1391" CreationDate="2012-02-26T13:41:49.923" Score="8" Body="&lt;p&gt;The (famous) fast marching method for solving the Eikonal equation cannot be sped up by parallelization. There are other methods (for example fast sweeping methods) for solving the Eikonal equation that are more amenable to parallelization, but even here the potential for (parallel) speedup is limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with the Eikonal equation is that the flow of information depends on the solution itself. Loosely speaking, the information flows along the characteristics (i.e. light rays in optics), but the characteristics depend on the solution itself. And the flow of information for the discretized Eikonal equation is even worse, requiring additional approximations (like implicitly present in fast sweeping methods) if any parallel speedup is desired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see the difficulties for parallelization, imagine a nice labyrinth like in some of the examples on &lt;a href=&quot;http://math.berkeley.edu/~sethian/&quot;&gt;Sethian's webpage&lt;/a&gt;. The number of cells on the shortest path through the labyrinth (probably) is a lower bound for the minimal number of steps/iterations of any (parallel) algorithm solving the corresponding problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I write &quot;(probably) is&quot;, because lower bounds are notoriously difficult to prove, and often require some reasonable assumptions on the operations used by an algorithm.) &lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-02-26T13:41:49.923" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1432" PostTypeId="1" AcceptedAnswerId="1462" CreationDate="2012-02-26T19:01:06.327" Score="5" ViewCount="406" Body="&lt;p&gt;I would like to analyze some well-used scientific codes that make heavy use of LAPACK. I.e. I'm looking for codes that both spend a lot of time within LAPACK functions and use LAPACK non-trivially (i.e. calling more than just DGEMM). I would be interested in applications that use either sequential LAPACK or ScaLAPACK or any other implementation that exposes the same &lt;a href=&quot;http://en.wikipedia.org/wiki/LAPACK#Naming_scheme&quot; rel=&quot;nofollow&quot;&gt;interface&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some good examples that are easy to obtain? Ideally they would be open source or available-on-request.&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-02-27T00:22:16.097" LastActivityDate="2013-01-20T21:52:58.803" Title="Easily accessible scientific code that uses LAPACK" Tags="|lapack|" AnswerCount="4" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1433" PostTypeId="1" CreationDate="2012-02-26T19:46:08.947" Score="6" ViewCount="5766" Body="&lt;p&gt;I have a sequence of 2D images (png files) encoding the partitioning (&lt;a href=&quot;http://en.wikipedia.org/wiki/Image_segmentation&quot; rel=&quot;nofollow&quot;&gt;segmentation&lt;/a&gt;) of a large biological 3D volume. In these files, each pixel  has a color, representing the 3D object the pixel belongs to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if anybody knew of tools that I can use, hopefully &lt;em&gt;off-the-shelf&lt;/em&gt;, to visualize this data in 3D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am mostly interested in tools that are free, or free-to-use in academia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can make this community wiki if you wish. &lt;/p&gt;&#xA;" OwnerUserId="13" LastEditorUserId="13" LastEditDate="2012-02-27T03:00:00.570" LastActivityDate="2012-02-27T03:26:17.083" Title="Tools for visualizing large 3D volumes" Tags="|visualization|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1434" PostTypeId="1" AcceptedAnswerId="1438" CreationDate="2012-02-26T22:50:56.320" Score="4" ViewCount="643" Body="&lt;p&gt;I'm having a problem where petsc is complaining about the type of PetscScalar (I get a whole bunch of errors from the c++ standard library that revolve around PetscScalar not defined).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe it is a problem with the way Petsc is compliled (I vaguely remember that Petsc can't seem to have both --with-c-support and --with-c++-support along with --with-scalar-type=complex, but I can't seem to figure out where that is stated.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sample of the errors I get (full version at &lt;a href=&quot;http://pastebin.com/uTujfRUH&quot; rel=&quot;nofollow&quot;&gt;pastebin&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:550: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double std::__complex_abs(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:569: error: declaration of C function ‘double std::__complex_abs(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:566: error: previous declaration ‘float std::__complex_abs(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double std::__complex_abs(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:572: error: declaration of C function ‘long double std::__complex_abs(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:569: error: previous declaration ‘double std::__complex_abs(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:575: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:586: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double std::__complex_arg(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:596: error: declaration of C function ‘double std::__complex_arg(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:593: error: previous declaration ‘float std::__complex_arg(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double std::__complex_arg(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:599: error: declaration of C function ‘long double std::__complex_arg(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:596: error: previous declaration ‘double std::__complex_arg(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:602: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:616: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:628: error: template specialization with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:639: error: template with C linkage&#xA;at-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_cos(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:674: error: declaration of C function ‘double __complex__ std::__complex_cos(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:671: error: previous declaration ‘float __complex__ std::__complex_cos(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_cos(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:677: error: declaration of C function ‘long double __complex__ std::__complex_cos(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:674: error: previous declaration ‘double __complex__ std::__complex_cos(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:680: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:690: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_cosh(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:704: error: declaration of C function ‘double __complex__ std::__complex_cosh(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:701: error: previous declaration ‘float __complex__ std::__complex_cosh(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_cosh(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:707: error: declaration of C function ‘long double __complex__ std::__complex_cosh(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:704: error: previous declaration ‘double __complex__ std::__complex_cosh(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:710: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:720: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_exp(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:730: error: declaration of C function ‘double __complex__ std::__complex_exp(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:727: error: previous declaration ‘float __complex__ std::__complex_exp(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_exp(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:733: error: declaration of C function ‘long double __complex__ std::__complex_exp(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:730: error: previous declaration ‘double __complex__ std::__complex_exp(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:736: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:747: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_log(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:757: error: declaration of C function ‘double __complex__ std::__complex_log(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:754: error: previous declaration ‘float __complex__ std::__complex_log(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_log(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:760: error: declaration of C function ‘long double __complex__ std::__complex_log(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:757: error: previous declaration ‘double __complex__ std::__complex_log(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:763: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:772: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:778: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_exp(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:792: error: declaration of C function ‘double __complex__ std::__complex_sin(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:789: error: previous declaration ‘float __complex__ std::__complex_sin(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_exp(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:795: error: declaration of C function ‘long double __complex__ std::__complex_sin(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:792: error: previous declaration ‘double __complex__ std::__complex_sin(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:798: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:808: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_sinh(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:822: error: declaration of C function ‘double __complex__ std::__complex_sinh(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:819: error: previous declaration ‘float __complex__ std::__complex_sinh(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_sinh(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:825: error: declaration of C function ‘long double __complex__ std::__complex_sinh(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:822: error: previous declaration ‘double __complex__ std::__complex_sinh(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:828: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:839: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_sqrt(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:866: error: declaration of C function ‘double __complex__ std::__complex_sqrt(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:863: error: previous declaration ‘float __complex__ std::__complex_sqrt(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_sqrt(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:869: error: declaration of C function ‘long double __complex__ std::__complex_sqrt(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:866: error: previous declaration ‘double __complex__ std::__complex_sqrt(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:872: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:883: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_tan(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:893: error: declaration of C function ‘double __complex__ std::__complex_tan(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:890: error: previous declaration ‘float __complex__ std::__complex_tan(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_tan(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:896: error: declaration of C function ‘long double __complex__ std::__complex_tan(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:893: error: previous declaration ‘double __complex__ std::__complex_tan(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:899: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:911: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘double __complex__ std::__complex_tanh(double __complex__)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:921: error: declaration of C function ‘double __complex__ std::__complex_tanh(double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:918: error: previous declaration ‘float __complex__ std::__complex_tanh(float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_tanh(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:924: error: declaration of C function ‘long double __complex__ std::__complex_tanh(const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:921: error: previous declaration ‘double __complex__ std::__complex_tanh(double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:927: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:940: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:945: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:960: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: In function ‘long double __complex__ std::__complex_tan(const long double __complex__&amp;amp;)’:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:971: error: declaration of C function ‘double __complex__ std::__complex_pow(double __complex__, double __complex__)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:967: error: previous declaration ‘float __complex__ std::__complex_pow(float __complex__, float __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:976: error: declaration of C function ‘long double __complex__ std::__complex_pow(const long double __complex__&amp;amp;, const long double __complex__&amp;amp;)’ conflicts with&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:971: error: previous declaration ‘double __complex__ std::__complex_pow(double __complex__, double __complex__)’ here&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex: At global scope:&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:979: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:990: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:1001: error: template specialization with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:1104: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:1113: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:1122: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:1131: error: template with C linkage&#xA;/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/complex:1448: error: template with C linkage&#xA;/usr/local/petsc-3.2-p6/include/petscmath.h:92: error: expected initializer before ‘PetscScalar’&#xA;/usr/local/petsc-3.2-p6/include/petscmath.h:191: error: ‘PetscScalar’ does not name a type&#xA;/usr/local/petsc-3.2-p6/include/petscmath.h:347: warning: ‘PetscIsInfOrNanScalar’ initialized and declared ‘extern’&#xA;/usr/local/petsc-3.2-p6/include/petscmath.h:347: error: ‘PetscScalar’ was not declared in this scope&#xA;/usr/local/petsc-3.2-p6/include/petscmath.h:367: error: ‘PetscScalar’ does not name a type&#xA;/usr/local/petsc-3.2-p6/include/petscviewer.h:386: error: expected ‘,’ or ‘...’ before ‘*’ token&#xA;/usr/local/petsc-3.2-p6/include/petscviewer.h:386: error: ISO C++ forbids declaration of ‘PetscScalar’ with no type&#xA;/usr/local/petsc-3.2-p6/include/petscviewer.h:387: error: ‘PetscScalar’ has not been declared&#xA;make: [bin/etom] Error 1 (ignored)&#xA;#-/usr/local/mpich2/bin/mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3  -o testmpi testmpi.o -lgsl  -L/usr/local/petsc-3.2-p6/lib  -lpetsc -lX11 -lpthread -llapack -lblas -L/central/intel/Compiler-11.1.072/mkl/lib/em64t -L/central/intel/Compiler-11.1.072/lib/intel64 -L/central/intel/Compiler-11.1.072/tbb/intel64/cc3.4.3_libc2.3.4_kernel2.6.9/lib -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -ldl -lgcc_s -lifport -lifcore -limf -lsvml -lm -lipgo -lirc -lpthread -lirc_s -lm -lstdc++ -lstdc++ -ldl -lgcc_s -ldl &#xA;#/bin/rm -f -f testmpi.o&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35" LastActivityDate="2012-02-27T01:12:08.883" Title="Petsc not compiling c++ files" Tags="|petsc|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1435" PostTypeId="1" CreationDate="2012-02-26T18:03:27.100" Score="4" ViewCount="592" Body="&lt;p&gt;I was going through my notes on different finite difference methods and came across something I don't quite understand. I have code that will calculate an approximate solution we can call this $U_{nm}$ that I define on a grid using $h$ and $dt$ for the change in $x$ and time, respectively. Now I have written down the global error is just:&#xA;$$e_{nm} =|U_{nm} - u(x_n,t_n)|$$&#xA;where $u(x_n,t_n)$ is the exact solution evaluated at the gird points. From there we can calculate our rate of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I, perhaps naively, just assumed I could take the solution I calculate and subtract the exact solution at every point take the absolute value. However, I have written something about actually just using $e_{nm}$ to calculate the initial values as well as boundary conditions and then plugging them back into the finite difference method to calculate all the grid points for all of $e_{nm}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct or did I perhaps not fully understand what my instructor was saying?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note we are working with forward, backward and crank-nicolson methods)&lt;/p&gt;&#xA;" OwnerUserId="1052" OwnerDisplayName="user25783" LastEditorUserId="1567" LastEditDate="2013-05-08T07:46:55.617" LastActivityDate="2013-05-08T07:46:55.617" Title="finite difference methods and global error" Tags="|pde|error-estimation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1436" PostTypeId="2" ParentId="1433" CreationDate="2012-02-27T00:36:17.243" Score="4" Body="&lt;p&gt;ParaView www.paraview.org &#xA;- also use ImageJ &lt;a href=&quot;http://fiji.sc/wiki/index.php/Fiji&quot; rel=&quot;nofollow&quot;&gt;http://fiji.sc/wiki/index.php/Fiji&lt;/a&gt; if you need to manipulate the data into something ParaView likes (e.g. raw).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd load the stack into ImageJ (it also has 3D viewers - so give them a try), then save it as a raw naming the file with x_y_z_datasize (e.g. my_file_1000x1000x20_8bit.raw).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then load the raw in ParaView using extents of 0-999, 0-999, 0-19 and play about with endiness and data size untill you get something that looks good. A tip is start with slices and then progress to volume. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out the ParaView wiki for more detail.&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2012-02-27T00:36:17.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1437" PostTypeId="2" ParentId="1435" CreationDate="2012-02-27T00:42:08.203" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I have written something about actually just using $e_{nm}$ to&#xA;  calculate the initial values as well as boundary conditions and then&#xA;  plugging them back into the finite difference method to calculate all&#xA;  the grid points for all of $e_{nm}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This statement doesn't make any sense, so you can safely assume you missed something in the lecture :)  In the general case you will not have access to the true solution, your instructor may have described some strategies for evaluating the error or convergence in this situation.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-02-27T00:42:08.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1438" PostTypeId="2" ParentId="1434" CreationDate="2012-02-27T01:12:08.883" Score="5" Body="&lt;p&gt;You have quite a lot of issues to address first. For example, your GSL headers are not being found and your compiler is set up in some strict configuration where it errors on &lt;code&gt;long long&lt;/code&gt;. Perhaps this means you have set &lt;code&gt;-pedantic -Werror&lt;/code&gt;. The environment may also be messed up or you might have a stray open &lt;code&gt;extern &quot;C&quot; {&lt;/code&gt; around some header includes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you will generally have type issues passing complex numbers between C and C++. In practice, C99 complex (see &lt;code&gt;complex.h&lt;/code&gt;) is binary-compatible with C++ complex, so you can cast and still use the same interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you still need help with PETSc, please send configure.log and any relevant build output to petsc-maint@mcs.anl.gov.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-27T01:12:08.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1439" PostTypeId="2" ParentId="1433" CreationDate="2012-02-27T01:39:24.467" Score="6" Body="&lt;p&gt;You can use &lt;a href=&quot;https://wci.llnl.gov/codes/visit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VisIt&lt;/a&gt; to perform this visualization. According to the VisIt FAQ:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;VisIt can read in many image file formats created by popular image editing software including: BMP, JPEG, PNG, PPM, PNM, RGB, TIFF. Stacks of images can also be assembled into 3D volumes using a .imgvol file, which is a text file containing the names of the files to assemble into the 3D volume.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As an example, I created 20 slices of a paraboloid and output each slice into a PNG file. I then created imagelist.imgvol that contains:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Z_START: 0.0&#xA;Z_STEP: 5&#xA;test1.png&#xA;test2.png&#xA;test3.png&#xA;test4.png&#xA;test5.png&#xA;test6.png&#xA;test7.png&#xA;test8.png&#xA;test9.png&#xA;test10.png&#xA;test11.png&#xA;test12.png&#xA;test13.png&#xA;test14.png&#xA;test15.png&#xA;test16.png&#xA;test17.png&#xA;test18.png&#xA;test19.png&#xA;test20.png&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that I used Z_STEP to rescale the z-axis. Then I opened the imgvol file in VisIt, and clicked &lt;code&gt;Add -&amp;gt; Contour -&amp;gt; Intensity&lt;/code&gt;. This results in the following image:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/3nhVl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="739" LastEditorUserId="739" LastEditDate="2012-02-27T01:45:45.490" LastActivityDate="2012-02-27T01:45:45.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1440" PostTypeId="2" ParentId="1433" CreationDate="2012-02-27T01:46:20.947" Score="1" Body="&lt;p&gt;I highly recommend &lt;a href=&quot;http://rsbweb.nih.gov/ij/&quot; rel=&quot;nofollow&quot;&gt;ImageJ&lt;/a&gt;.  It is a great freeware software for visualizing tomographic data or image sequences in any format.  It has tons of pluggins for specialized image processing, such as segmentation, object counting, 3D reconstruction, thresholding, etc.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-02-27T01:46:20.947" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1441" PostTypeId="2" ParentId="1433" CreationDate="2012-02-27T03:26:17.083" Score="4" Body="&lt;p&gt;I would strongly reccomend &lt;a href=&quot;http://mayavi.sourceforge.net/screenshots/index.html&quot; rel=&quot;nofollow&quot;&gt;Python + Mayavi&lt;/a&gt;!  It is cross-platform, fast, powerful, and easy!  Here is a surface plot of in 7 lines of code, including loading files!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy import array, misc&#xA;from mayavi import mlab&#xA;&#xA;directory = &quot;some-data&quot;&#xA;data = array([misc.imread(os.path.join(directory, f)) for f in os.listdir(directory)])&#xA;&#xA;unique_colors = set(list(data))&#xA;for color in unique_colors:&#xA;    mlab.comtour3d(1.*(data==color), contours=[0.5])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will give you an interactable GUI with configurable meshes of your data.  There is nearly infinite room for customization either by hand or programatically! &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-02-27T03:26:17.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1442" PostTypeId="2" ParentId="1429" CreationDate="2012-02-27T03:38:46.470" Score="4" Body="&lt;p&gt;I think it depends on the context of the publication, the intended audience, and how &quot;common&quot; you need &quot;commonly understandable&quot; to be.  I do think your last option is probably safest, i.e. be as literal/explicit and as close to &quot;plain English&quot; as possible.  But if your intended audience has familiarity with set theory, or Python, or Java, or C#, or JavaScript, etc., then a notation similar to one of those might be better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I make this suggestion as someone who conducts job interviews where the candidate is expected to write code or pseudocode.  Most candidates (both good and bad) tend to drift toward the language they're most familiar with in order to make their pseudocode as understandable as possible to themselves and to me.  People familiar with Java write &lt;code&gt;Map&amp;lt;K,V&amp;gt;&lt;/code&gt;.  People familiar with Python write tuples or a dictionary as you did in your first code snippet.  People familiar with C write arrays.  People with less of a programming background tend to write pseudocode that looks more like plain English -- but that often ends up looking like Python anyway. :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I unfortunately can't answer your last question -- I'm not familiar with any reference or common recommendations for writing pseudocode.&lt;/p&gt;&#xA;" OwnerUserId="1054" LastActivityDate="2012-02-27T03:38:46.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1443" PostTypeId="2" ParentId="1406" CreationDate="2012-02-27T03:50:30.967" Score="2" Body="&lt;p&gt;I started out as a Trilinos guy (mostly through the Python bindings), and wrote a big fluids code that worked with both it and PETSc.  In general, I would say that these are matrix libraries, and the communications functions are cludgy and somewhat annoying.  TPetra from the Trilinos folks looks really rad, but seems to cause nothing but heartache on the trilinos-users listserv. . .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend PETSc personally, it seems really active in development and use.  The one caveat I would offer is that if you need to pass something more complex than a vector, using a different channel for communication such as &lt;a href=&quot;http://www.zeromq.org/&quot; rel=&quot;nofollow&quot;&gt;Zero-MQ&lt;/a&gt; can really ease things.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-02-27T03:50:30.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1444" PostTypeId="2" ParentId="1432" CreationDate="2012-02-27T04:01:25.960" Score="1" Body="&lt;p&gt;ATLAS does a pretty brutal group of grinding metrics on LAPACK and BLAS.  You might be able to snag some code out of there that meets your needs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might also suggest checking out Numpy and Scipy as both have very large unit-tests that likely span large parts of that phase-space.  Additionally, they provide hooks down to the LAPACK functions if you want to call those directly on arrays of your creation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another tack to the problem is possibly searching &lt;a href=&quot;https://github.com/search?q=hessenberg%20lapack&amp;amp;repo=&amp;amp;langOverride=&amp;amp;start_value=1&amp;amp;type=Everything&amp;amp;language=&quot; rel=&quot;nofollow&quot;&gt;Github for the LAPACK calls of interest&lt;/a&gt;, and seeing what code surfaces!  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-02-27T04:01:25.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1445" PostTypeId="1" CreationDate="2012-02-27T04:01:53.590" Score="33" ViewCount="5797" Body="&lt;p&gt;I need to get started using Finite Element Methods. I am about to start reading &lt;strong&gt;Numerical solutions of partial differential equations by the finite element method&lt;/strong&gt; by &lt;em&gt;Claes Johnson&lt;/em&gt;, but it's dated 1987. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What newer &lt;em&gt;good&lt;/em&gt; resources/textbooks/e-books/lecture notes on  this subject are out there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) How much am I missing by reading a 1987 book?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="988" LastActivityDate="2016-09-09T18:22:10.707" Title="Modern resources for learning FEM" Tags="|finite-element|reference-request|" AnswerCount="9" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1446" PostTypeId="2" ParentId="1445" CreationDate="2012-02-27T04:30:35.697" Score="21" Body="&lt;p&gt;There are lots of modern finite element references, but I will just comment on a few books that I think are practical and relevant to applications, plus one containing more comprehensive analysis.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=IaV0wgG2jacC&amp;amp;printsec=frontcover&amp;amp;source=gbs_ge_summary_r&amp;amp;cad=0#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;Wriggers &lt;em&gt;Nonlinear Finite Element Methods&lt;/em&gt; (2008)&lt;/a&gt; is a good general reference, but will be most relevant to those concerned with applications in structural mechanics (including contact, shells, and plasticity).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=N9KEgXMrMm8C&amp;amp;printsec=frontcover&amp;amp;source=gbs_ge_summary_r&amp;amp;cad=0#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;Elman, Silvester, and Wathen &lt;em&gt;Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics&lt;/em&gt; (2005)&lt;/a&gt; is less comprehensive on finite element discretization techniques, but has good content on incompressible flow and a certain class of iterative solvers. It also explains the &lt;a href=&quot;http://www.maths.manchester.ac.uk/~djs/ifiss/&quot;&gt;IFISS&lt;/a&gt; package.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=S4URqrTtSXoC&amp;amp;printsec=frontcover&amp;amp;source=gbs_ge_summary_r&amp;amp;cad=0#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;Donéa and Huerta &lt;em&gt;Finite element methods for flow problems&lt;/em&gt; (2003)&lt;/a&gt; covers similar material, but includes ALE moving mesh methods and compressible gas dynamics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=ci4c_R0WKYYC&amp;amp;printsec=frontcover&amp;amp;source=gbs_ge_summary_r&amp;amp;cad=0#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;Brenner and Scott &lt;em&gt;The mathematical theory of finite element methods&lt;/em&gt; (2008 revision)&lt;/a&gt; contains a rigorous theoretical development of discretizations for linear elliptic problems, including associated multigrid and domain decomposition theory. It does not treat transport-dominated problems, &quot;messy&quot; nonlinearities like plasticity, or non-polynomial bases.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These resources fail to cover topics such as discontinuous Galerkin methods or $H(curl)$ problems (Maxwell). I think papers are currently a better resource than books for these topics, although &lt;a href=&quot;http://books.google.com/books?id=APQkDOmwyksC&amp;amp;printsec=frontcover&amp;amp;source=gbs_ge_summary_r&amp;amp;cad=0#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;Hesthaven and Warburton &lt;em&gt;Nodal discontinuous Galerkin methods&lt;/em&gt; (2008)&lt;/a&gt; is certainly worthwhile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also recommend reading the examples from open source finite element software packages such as &lt;a href=&quot;http://fenicsproject.org/&quot;&gt;FEniCS&lt;/a&gt;, &lt;a href=&quot;http://libmesh.sf.net&quot;&gt;Libmesh&lt;/a&gt;, and &lt;a href=&quot;http://dealii.org&quot;&gt;Deal.II&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-02-27T12:25:14.893" LastActivityDate="2012-02-27T12:25:14.893" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1447" PostTypeId="2" ParentId="1445" CreationDate="2012-02-27T07:09:00.340" Score="6" Body="&lt;p&gt;There are numerous textbooks on finite element methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some classical references are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;O. Axelsson, V. A. Barker &quot;Finite Element Solution of Boundary Value Problems&quot; which introduces the fundemenals and includes a preentation and disucssion of useful direct and iterative techniques for solving systems of equation. The perspective is on mechanics and applied mathematics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;S. C. Brenner and L. Ridgway Scotte &quot;The Mathematical Theory of Finite Element Methods&quot; which introduces fundamental mathematical theory for understanding the foundation of FEM. The perspective is that of applied mathematicians. The book put emphasis on mathematical theory, i.e. it is for applied mathematians or engineers who need to dig deeper in theory.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;B. Szabó and I. Babuska &quot;Finite Element Analysis&quot; is a well-written textbook where the history, fundemental theory and principles are presented by two founders of FEM theory. The perspective is that of applied mathematicians and contains applications in structural mechanics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;M. S. Gockenbach &quot;Understanding and Implementing the Finite Element Method&quot; is a good introductory references on the basics and a few advanced topics of FEM, relevant implementation details of FEM, discussion of practical solution strategies. It comes with Matlab examples and is a well-written reference for beginners. It focus on bridging theory of FEM with engineering applications.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I. Babuska, J. R. Whiteman and T. Strouboulis &quot;Finite Elements - An introduction to the method and error estimation&quot; seeks to introduce fundamental mathematical theory of FEM with a focus on engineering applications and practical understanding with specific emphasis on error estimation for use in adaptive FEM. It is well-written and a useful reference on the subjects.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-02-27T07:09:00.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1448" PostTypeId="2" ParentId="1429" CreationDate="2012-02-27T09:00:24.703" Score="8" Body="&lt;p&gt;It looks like you're already using some of the concepts from &lt;a href=&quot;http://mitpress.mit.edu/algorithms/&quot;&gt;Corwin, Leiserson, Rivest, and Stein&lt;/a&gt;, which I've always seen as the &quot;default&quot; introductory text for algorithms, in writing your pseudocode. I don't think there's any more standard reference for pseudocode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would argue that what makes a good pseudocode is to break the problem down into its representative mathematical and logical functions. What conditions have to be true to execute the code? What branches exist? What data is really essential in determining the behavior?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond that, the goal of good pseudocode is to tell the programmer what needs to be done—and not necessarily how to implement it. You might want, for instance, to flatten something like nested loops into a single statement:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;For all elements x in 2-D array A&#xA;    x ← max(0, x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;would probably be better than saying&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;For all i in 1:m&#xA;   For all j in 1:n&#xA;       A(i,j) ← max(0, A(i,j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as the latter already starts to make some assumptions about the language that is being used (row-major versus column-major, etc.).&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-02-27T09:00:24.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1449" PostTypeId="1" AcceptedAnswerId="1870" CreationDate="2012-02-27T10:20:56.733" Score="17" ViewCount="10145" Body="&lt;p&gt;I'm using Valgrind + Callgrind to profile a solver I have written. As the Valgrind user manual states, I've compiled my code with the debugging options for the compiler:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Without debugging info, the best Valgrind tools will be able to do is guess which function a particular piece of code belongs to, which makes both error messages and profiling output nearly useless. With -g, you'll get messages which point directly to the relevant source code lines.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://valgrind.org/docs/manual/manual-core.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Valgrind manual&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When compiled with the debugging option, the codes run much slower. CFD code, becomes REALLY slow, even for small cases when compiled with debugging flags. Valgrind makes it 40x slower (see the manual &lt;a href=&quot;http://valgrind.org/docs/manual/manual-core.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What tools are you using for code profiling (profiling, not benchmarking)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How long do you let the code run (statistics: how many time steps)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How large are the cases (if the case fits in the cache, the solver is orders of magnitude faster, but then I'll miss the memory related processes)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="383" LastEditorUserId="625" LastEditDate="2018-08-18T21:02:20.830" LastActivityDate="2018-08-18T21:02:20.830" Title="Profiling CFD code with Callgrind" Tags="|hpc|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="1450" PostTypeId="1" AcceptedAnswerId="1451" CreationDate="2012-02-27T10:34:04.587" Score="10" ViewCount="307" Body="&lt;p&gt;I started to learn OpenFOAM from the Cavity tutorial which is provided &lt;a href=&quot;http://www.openfoam.org/docs/user/cavity.php&quot;&gt;at the web-site&lt;/a&gt;. When experimenting with different Reynolds numbers, in section &quot;2.1.8.2 Running the code&quot;, tutorial says to rerun the solver because &quot;It is sensible to increase the solution time&quot;. But when I done this I couldn't find any kind of differences between the flow in the cavity with the low (0.2) and the high (0.6) Courant number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I know if I need to rerun the simulation?&lt;/p&gt;&#xA;" OwnerUserId="188" LastEditorUserId="89" LastEditDate="2012-06-28T07:19:03.127" LastActivityDate="2012-06-28T07:19:03.127" Title="Why do we have to rerun the CFD solver for higher Reynolds number?" Tags="|software|fluid-dynamics|convergence|openfoam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1451" PostTypeId="2" ParentId="1450" CreationDate="2012-02-27T12:49:41.090" Score="9" Body="&lt;p&gt;Due to the under-resolved boundary layer near the lid, the velocity in the adjacent cells is significantly lower than the lid. This section is showing you a trick to make the code run faster while still being stable. Increasing the Courant number would normally make the method unstable, but since the velocity in all interior cells is significantly less than the velocity of the boundary layer, you can get away with increasing the time step.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-02-27T12:49:41.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1452" PostTypeId="2" ParentId="1449" CreationDate="2012-02-27T15:31:03.487" Score="9" Body="&lt;p&gt;The &lt;a href=&quot;http://poormansprofiler.org/&quot;&gt;poor man's profiler&lt;/a&gt; is basically a &lt;code&gt;gdb&lt;/code&gt; script that samples the call stack. You'll still need to have debugging symbols. It is still slow, but as it does not implement a virtual machine to run the code on it is often faster than &lt;code&gt;callgrind&lt;/code&gt; and adequate to the task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've run in on particle physics analyzers with modest success (i.e. I demonstrated that the code &lt;em&gt;didn't&lt;/em&gt; any horrible hot spots and optimization was going to require a better algorithm).&lt;/p&gt;&#xA;" OwnerUserId="183" LastActivityDate="2012-02-27T15:31:03.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1453" PostTypeId="2" ParentId="1445" CreationDate="2012-02-27T15:38:32.497" Score="6" Body="&lt;p&gt;Since Jed mentioned discontinuous Galerkin methods, I thought I should mention some other helpful books on spectral methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For theory:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Canuto, C. and Hussaini, M. and Quarteroni, A. and Zang, T. &lt;a href=&quot;http://books.google.com/books?id=DFJB0kiq0CQC&amp;amp;lpg=PR1&amp;amp;dq=Spectral%20Methods%3A%20Fundamentals%20in%20Single%20Domains&amp;amp;pg=PR1#v=onepage&amp;amp;q=Spectral%20Methods%3a%20Fundamentals%20in%20Single%20Domains&amp;amp;f=false&quot;&gt;&lt;em&gt;Spectral Methods: Fundamentals in Single Domains (2006)&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Canuto, C. and Hussaini, M. and Quarteroni, A. and Zang, T. &lt;a href=&quot;http://books.google.com/books?id=7COgEw5_EBQC&amp;amp;lpg=PR1&amp;amp;dq=Spectral%20Methods%3A%20Evolution%20to%20Complex%20Geometries%20and%20Applications%20to%20Fluid%20Dynamics&amp;amp;pg=PR1#v=onepage&amp;amp;q=Spectral%20Methods%3a%20Evolution%20to%20Complex%20Geometries%20and%20Applications%20to%20Fluid%20Dynamics&amp;amp;f=false&quot;&gt;&lt;em&gt;Spectral Methods: Evolution to Complex Geometries and Applications to Fluid Dynamics (2007)&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want a good introduction to implementing spectral methods, I highly recommend:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Kopriva, D. A. &lt;a href=&quot;http://books.google.com/books?id=fZyqWPNjx4AC&amp;amp;lpg=PA310&amp;amp;dq=kopriva%20discontinuous%20galerkin&amp;amp;pg=PA310#v=onepage&amp;amp;q=kopriva%20discontinuous%20galerkin&amp;amp;f=false&quot;&gt;&lt;em&gt;Implementing Spectral Methods for Partial Differential Equations (2009)&lt;/em&gt;&lt;/a&gt;. The errata can be found &lt;a href=&quot;http://www.math.fsu.edu/~kopriva/FSU%20Site/Publications_files/Errata.pdf&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Disclosure: Kopriva is my advisor. The book is light on the highly theoretical results that Canuto, et al. covers, and focuses strictly on implementation.&lt;/p&gt;&#xA;" OwnerUserId="739" LastActivityDate="2012-02-27T15:38:32.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1454" PostTypeId="1" CreationDate="2012-02-27T16:09:43.993" Score="24" ViewCount="4977" Body="&lt;p&gt;Note: the following post may include controversial opinions, so please&#xA;note that they are only my opinions, and not intended to offend&#xA;anyone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm being programming in some form or the other since around 1999. I&#xA;initially used R, and then later, around 2004, mostly switched to&#xA;Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For many scientific applications, for example, simulation,&#xA;including such things as MCMC, both R and Python are too slow and need&#xA;to be sped up. The usual way of doing so is by extending with C or&#xA;C++. For both R and Python, this is what I did, using R's C API with&#xA;C++, and the Boost Python library with Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for various reasons, this combination is not the ideal&#xA;solution. What is important in programming, particularly algorithms?&#xA;Expressiveness and speed, which are of course related. The more&#xA;expressive a language, the faster one can write in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) As far as expressiveness goes, neither R nor Python are really&#xA;ideal for writing scientific algorithms in my opinion. They do not&#xA;closely map to the underlying algorithm. However, they are both&#xA;considerably better than C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) I enjoy writing in Python, which is a pleasant language, though as&#xA;noted above it is not ideal for algorithmic work. However, when one has&#xA;to work with a Python/C++ combination because of speed issues, this&#xA;mix becomes considerably less pleasant to work with. What usually&#xA;happens is that I first write in Python, and once I have something&#xA;that is working well, often discover that it is too slow (for some&#xA;subjective value of too slow). I then face the decision of whether to&#xA;spend some unreasonable amount of time rewriting it in C++, or put up&#xA;with the slowness. In hindsight I often feel I might have been better&#xA;off putting up with the slowness, especially as the speedups obtained&#xA;are unpredictable. Also, the Boost Python interface between the two is&#xA;a significant maintenance headache, and having code in two very&#xA;different languages glued together like this is just distracting. No&#xA;criticism of Boost Python intended, it is as powerful an interface as&#xA;one could imagine, and pretty much just works most of the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in an ideal world, with unlimited time and resources, neither of&#xA;these problems would be a major deal. However, in scientific projects&#xA;I have worked on, I've had the following experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether or not I have collaborators on the project, I always seem to&#xA;wind up doing the vast majority of the computing. In a total of 5&#xA;significant projects, I only had substantial participation from one&#xA;people on one project. That one person did more than pull his weight;&#xA;he did as much as me or more. However, in all other cases, including&#xA;projects with multiple collaborators, I've done (virtually) all the&#xA;computational work. While I can say that I have not been blessed with&#xA;the best collaborators (it seems to be a mixture of laziness and&#xA;incompetence) it is not clear to me whether this state of affairs is&#xA;likely to change in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computational scientific work is an enormous amount of effort, and if&#xA;I can't change how my collaborators behave, I can change the way I&#xA;work. The most important improvement would be to get things done more&#xA;quickly. Which brings me to the main consideration here, which is that&#xA;switching languages to something less orthodox may help. Based on past&#xA;research, the most likely candidates in order of likelihood are Common&#xA;Lisp and Ocaml. I've been thinking about this for years, but recently&#xA;have been thinking about it more seriously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far I can tell, few people use either CL or Ocaml for scientific&#xA;computation. On searching this site, I found two references to CL (one&#xA;was mine) and one to Ocaml (mine). I've had a couple of encouraging&#xA;contacts over the years with adventurous people working on the&#xA;fringe. In 2008 I came across a &lt;a href=&quot;http://www.amazon.com/review/RVC91D51ZA20U&quot;&gt;book&#xA;review&lt;/a&gt; of Peter Seibel's&#xA;&quot;Practical Common Lisp&quot; (which I own), by Tamas K. Papp. This caught&#xA;my attention, since it was one of the few mentions of scientific computing&#xA;for Lisp that I had come across on the net. I wrote to Tamas, who immediately&#xA;replied helpfully and encouragingly. To quote him&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My programming productivity probably increased tenfold with Lisp, but&#xA;  that took about a year to happen and I am still learning (I was doing&#xA;  quite well after 2 months though).  So if you are working on something&#xA;  time-critical, then postpone the switch.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;You should consider asking folks on c.l.l, I am not the only one who&#xA;  knows about these things, others do scientific computing on Lisp.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;He also has a &lt;a href=&quot;http://tkpapp.blogspot.in/&quot;&gt;blog&lt;/a&gt; and &lt;a href=&quot;https://github.com/tpapp&quot;&gt;a GitHub&#xA;page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another person I briefly corresponded with (in December 2006) was &lt;a href=&quot;http://faculty.washington.edu/ikalet/&quot;&gt;Ira&#xA;Kalet&lt;/a&gt;, who has used Common&#xA;Lisp in the context of radiation oncology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps there are others who do scientific computing on Lisp, but I&#xA;don't know of anyone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common problem people cite with CL is the lack of&#xA;libraries. This is a severe problem in general purpose computing, but&#xA;may not be so much so in scientific computing, particularly from the&#xA;ground up implementations of algorithms. Specifically, I can get by&#xA;most of the time with a basic math library, including probability&#xA;distribution functions, a multidimensional array library, and a&#xA;basic set of containers e.g. map, set, list etc. as found in the C++ and&#xA;Python standard libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know even less about Ocaml than I do about CL, but threw that in as an&#xA;alternative. It is supposedly very fast, has one free implementation&#xA;by French researchers, and seems like the most viable of the ML family&#xA;of languages for scientific computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To conclude, I'm wondering if others have experience with this, and&#xA;what thoughts they have, if any. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I'm mostly interested in first hand experience, in the context of the issues I've discussed above. E.g. if you used to use Python and C++ (or R and C++) and moved to a more obscure language, I'd be most interested in hearing about your experiences.&lt;/p&gt;&#xA;" OwnerUserId="38" LastEditorUserId="38" LastEditDate="2014-07-15T13:15:52.697" LastActivityDate="2014-07-15T13:15:52.697" Title="Using unconventional programming languages for scientific computation" Tags="|languages|" AnswerCount="3" CommentCount="13" ClosedDate="2012-02-29T17:42:55.497" ContentLicense="CC BY-SA 3.0" />
  <row Id="1455" PostTypeId="2" ParentId="1454" CreationDate="2012-02-27T16:51:30.703" Score="4" Body="&lt;p&gt;For scientific applications of OCaml, see for example&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ffconsultancy.com/products/ocaml_for_scientists/&quot; rel=&quot;nofollow&quot;&gt;OCaml for scientists&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://romain.bardou.fr/slides/caml060711.pdf&quot; rel=&quot;nofollow&quot;&gt;a presentation of OCaml for scientists&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For Lisp in science, see for example&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.siginf.com/13023.html&quot; rel=&quot;nofollow&quot;&gt;SciCL&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.lrde.epita.fr/~didier/research/verna.06.labri-slides.pdf&quot; rel=&quot;nofollow&quot;&gt;a presentation about scientific computing in Lisp&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://incanter.org/&quot; rel=&quot;nofollow&quot;&gt;Incanter&lt;/a&gt; (statistical computing in Clojure, which is a Lisp dialect)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am sure there are many more references. However, I cannot cite any major research project in which computational work was done in either OCaml or Lisp. Choosing either one means working in relative isolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also be interested in &lt;a href=&quot;http://julialang.org/&quot; rel=&quot;nofollow&quot;&gt;Julia&lt;/a&gt;, a new language for scientific computing currently under development, with clear Lisp influences.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2012-02-27T16:51:30.703" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1456" PostTypeId="2" ParentId="1445" CreationDate="2012-02-27T18:09:03.953" Score="13" Body="&lt;p&gt;For the second question, as a reader of Claes Johnson's book myself, I would say that you didn't miss much as a beginner in the finite element method, that book is pretty well-rounded with every aspect of the FEM except for the implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, lots of developments have been made since the book published 20 years ago, like other people already mentioned: method-wise there are Discontinuous Galerkin FEM and non-conforming FEM, $H(\mathbf{curl})$ and $H(\mathrm{div})$ Conforming Elements, adaptive mesh refining techniques($hp$-FEM), space-time FEM, least-square FEM, finite element exterior calculus, etc; For solving the linear equation system, there are algebraic multigrid methods, various types of nice preconditioners, fast direct solvers, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first question, aside from the references other people already mentioned, I will list some books for some specific topics in FEM:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/Mixed_and_hybrid_finite_element_methods.html?id=yYwZAQAAIAAJ&quot;&gt;Mixed and Hybrid Finite Element Methods&lt;/a&gt; by Brezzi and Fortin: it has the construction of element for $H(\mathrm{div})$ space, also there are lots of examples of various equations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/Finite_element_methods_for_Maxwell_s_equ.html?id=zI7Y1jT9pCwC&quot;&gt;Finite element methods for Maxwell's equations&lt;/a&gt; by Monk: For various $H(\mathbf{curl})$ problems, both theoretical analysis for the Sobolev spaces and a self-contained finite element construction are presented. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=hoyeJbcdvDYC&quot;&gt;Higher-order finite element methods&lt;/a&gt; by Šolín, Segeth, and Doležel: pretty much a complementary book for above two books, it has a comprehensive and explicit construction of the basis functions for the $H(\mathrm{div})$ and $H(\mathbf{curl})$ conforming finite element, ie, Raviart-Thomas element, Brezzi–Douglas–Marini element, and Nédélec element up to arbitrary order in 3D, also the quadrature formulae for these element are presented. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=BVfvAAAAMAAJ&quot;&gt;Finite element methods for Navier-Stokes equations&lt;/a&gt; by Girault and Raviart: Another classic in FEM reference books IMHO, the theoretical analysis for the vector potentials is the gem, if you are dealing with the 3D vector fields FEM computation, then this book pretty much has all the theoretical analysis you need.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=_HligpOKcjoC&quot;&gt;A Posteriori Error Estimation in Finite Element Analysis&lt;/a&gt; by Ainsworth and Oden: this book deals with the core idea in the adaptive mesh refinement: a posteriori error estimation for the FEM, and how to construct various types of local error indicators.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=CCjm79FbJbcC&quot;&gt;Theory and Practice of Finite Elements&lt;/a&gt; by Ern and Guermond: another well-rounded book I would say, but not for beginner, this book is for people who know FEM to some extent, but would like to seek for more ingredients, for example, the author established the Babuška Inf-Sup condition in the general Banach space setting and compared it with the open mapping and closed range theorem in functional analysis; Also this book has a nice presentation of Discontinuous Galerkin method for hyperbolic PDEs; In part III of the book, the author gave us a comprehensive presentation of the implementation, from how to choose the quadrature points to how to efficiently store the sparse matrix, and some pseudo-code for the subroutines needed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="350" LastActivityDate="2012-02-27T18:09:03.953" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1457" PostTypeId="2" ParentId="1454" CreationDate="2012-02-27T20:11:42.203" Score="9" Body="&lt;p&gt;&lt;a href=&quot;http://blog.gmarceau.qc.ca/2009/05/speed-size-and-dependability-of.html&quot;&gt;The speed, size and dependability of programming languages&lt;/a&gt; does a really good job of wrapping up a lot of different concerns expressed in your &quot;question.&quot;  It compares the speed and codebase size of a bunch of implementations of the same benchmarks across 33 languages!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have become a Python lover mainly because it is much more common to have excess computing time than excess time to program.  I am more than willing to squander CPU cycles than sacrifice a wedge of time that could be devoted to something more interesting.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, +1 on Julia.  I think I may switch to it when it becomes a bit more stable and widely supported i.e. when standard modules are wrapped for the work I like doing.&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2012-03-26T21:21:31.947" LastActivityDate="2012-03-26T21:21:31.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1459" PostTypeId="2" ParentId="1001" CreationDate="2012-02-28T00:22:37.407" Score="42" Body="&lt;p&gt;If you want to see what &lt;code&gt;a\b&lt;/code&gt; does for your particular matrix you can set &lt;code&gt;spparms('spumoni',1)&lt;/code&gt; and figure exactly what algorithm you were impressed by. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;spparms('spumoni',1);&#xA;A = delsq(numgrid('B',256));&#xA;b = rand(size(A,2),1);&#xA;mldivide(A,b);  % another way to write A\b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sp\: bandwidth = 254+1+254.&#xA;sp\: is A diagonal? no.&#xA;sp\: is band density (0.01) &amp;gt; bandden (0.50) to try banded solver? no.&#xA;sp\: is A triangular? no.&#xA;sp\: is A morally triangular? no.&#xA;sp\: is A a candidate for Cholesky (symmetric, real positive diagonal)? yes.&#xA;sp\: is CHOLMOD's symbolic Cholesky factorization (with automatic reordering) successful? yes.&#xA;sp\: is CHOLMOD's numeric Cholesky factorization successful? yes.&#xA;sp\: is CHOLMOD's triangular solve successful? yes.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so I can see that &quot;\&quot; ended up using &quot;CHOLMOD&quot; in this case.&lt;/p&gt;&#xA;" OwnerUserId="979" LastEditorUserId="1064" LastEditDate="2012-03-02T09:19:58.227" LastActivityDate="2012-03-02T09:19:58.227" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1460" PostTypeId="2" ParentId="1454" CreationDate="2012-02-28T02:31:43.220" Score="18" Body="&lt;p&gt;We developed &lt;a href=&quot;http://julialang.org/&quot;&gt;Julia&lt;/a&gt; for &lt;em&gt;exactly&lt;/em&gt; the reasons raised. There was just no good high level scientific computing language that also gave good enough performance, that you didn't have to keep rewriting parts of your code in C/Fortran. The design of julia has a fair bit of lisp influence, so you may find it to your liking, while your collaborators can just treat it like a matlab or R if they don't care about the functional parts. The downside is that the language is new, and does not yet have all the libraries one would need for day-to-day usage. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mark, would love to add julia to your benchmark to see how we fare. Do hop on over to our mailing list and let us know what you'd like to see in julia so that it is more useful to you.&lt;/p&gt;&#xA;" OwnerUserId="1060" LastEditorUserId="605" LastEditDate="2012-02-28T11:57:33.023" LastActivityDate="2012-02-28T11:57:33.023" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1461" PostTypeId="2" ParentId="1194" CreationDate="2012-02-28T12:58:42.647" Score="18" Body="&lt;p&gt;I'm coming late to this party, so it's hard for me to follow the back-and-forth from all above. The question is big, and I think if you are interested it could be broken up into smaller pieces. One thing I got interested in was simply the performance of your &lt;code&gt;daxpy&lt;/code&gt; variants, and whether Fortran is slower than C on this very simple code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running both on my laptop (Macbook Pro, Intel Core i7, 2.66 GHz), the relative performance of your hand-vectorized C version and the non-hand vectorized Fortran version depend on the compiler used (with your own options):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Compiler     Fortran time     C time&#xA;GCC 4.6.1    5408.5 ms        5424.0 ms&#xA;GCC 4.5.3    7889.2 ms        5532.3 ms&#xA;GCC 4.4.6    7735.2 ms        5468.7 ms&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, it just seems that GCC got better at vectorizing the loop in the 4.6 branch than it was before.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;On the overall debate, I think one can pretty much write fast and optimized code in both C and Fortran, almost just like in assembly language. I will point out, however, one thing: just like as assembler is more tedious to write than C but gives you finer control over what is executed by the CPU, C is more low-level than Fortran. Thus, it gives you more control over details, which can help optimizing, where the Fortran standard syntax (or its vendor extensions) may lack functionality. One case is the explicit use of vector types, another is the possibility of specifying alignment of variables by hand, something Fortran is incapable of. &lt;/p&gt;&#xA;" OwnerUserId="1064" LastEditorUserId="1064" LastEditDate="2012-02-28T13:10:40.997" LastActivityDate="2012-02-28T13:10:40.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1462" PostTypeId="2" ParentId="1432" CreationDate="2012-02-28T13:08:10.140" Score="3" Body="&lt;p&gt;It's hard to know where to start, as there are so many codes out there using LAPACK and your criteria are a bit… subjective (“use LAPACK non-trivially”). I'll throw in some pointers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you can find &lt;a href=&quot;http://www.netlib.org/utk/people/lapack-survey.ps&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; an old survey of LAPACK users, which gives some information into who uses it, and what are they using it for&lt;/li&gt;&#xA;&lt;li&gt;most quantum chemistry codes out there use LAPACK for their linear algebra (&lt;a href=&quot;http://www.cpmd.org/&quot; rel=&quot;nofollow&quot;&gt;CPMD&lt;/a&gt;, &lt;a href=&quot;http://www.cp2k.org/&quot; rel=&quot;nofollow&quot;&gt;CP2K&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Octopus_%28software%29&quot; rel=&quot;nofollow&quot;&gt;octopus&lt;/a&gt;, to cite only a few open-source codes)&lt;/li&gt;&#xA;&lt;li&gt;you could also go to the packages list of your favorite Linux distro, and check out what packages depend on lapack&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1064" LastActivityDate="2012-02-28T13:08:10.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1463" PostTypeId="1" AcceptedAnswerId="2734" CreationDate="2012-02-28T14:01:21.400" Score="13" ViewCount="1180" Body="&lt;p&gt;Proteins are typically represented in a cartoon form, with β sheets as arrows and α helices as coils:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://kpwu.wordpress.com/2011/10/04/some-cartoon-views-of-proteins-shown-in-pymol/&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AVZiA.png&quot; alt=&quot;Example cartoon representation of a protein&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering, is there somewhere a reference that describes the construction of this representation? That is, what mathematical objects are used to construct these graphics, and on which atoms/directions are they built?&lt;/p&gt;&#xA;" OwnerUserId="1064" LastActivityDate="2012-07-07T21:17:40.813" Title="How can one mathematically describe the &quot;cartoon&quot; type of representation of proteins?" Tags="|computational-chemistry|visualization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1464" PostTypeId="2" ParentId="1432" CreationDate="2012-02-28T14:21:21.647" Score="0" Body="&lt;p&gt;Have a look at the RADAU IIA solver by E. Hairer (&lt;a href=&quot;http://www.unige.ch/~hairer/software.html&quot; rel=&quot;nofollow&quot;&gt;http://www.unige.ch/~hairer/software.html&lt;/a&gt;). It uses LAPACK as a backbone for an implicit Runge-Kutta method for ODEs. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-02-28T14:21:21.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1465" PostTypeId="2" ParentId="223" CreationDate="2012-02-28T15:06:58.077" Score="6" Body="&lt;p&gt;&lt;strong&gt;Profile, don't speculate!&lt;/strong&gt; (also works as &lt;em&gt;“Benchmark, don't speculate!”&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's nothing generic one can say, it depends heavily on the tasks you want to perform (BLAS 1/2/3, for example) and the hardware you're on (obviously, the Intel MKL doesn't run on ARM processors, for example; but even among Intel processors, you can expect performance differences).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Another thing I think is worth mentioning is that AMD also offers mathematical libraries tuned to their processors, the &lt;strong&gt;&lt;a href=&quot;http://developer.amd.com/libraries/acml/pages/default.aspx&quot;&gt;AMD Core Math Library&lt;/a&gt;&lt;/strong&gt;. It's not as feature-rich as Intel's MKL, but it does include BLAS.&lt;/p&gt;&#xA;" OwnerUserId="1064" LastActivityDate="2012-02-28T15:06:58.077" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1467" PostTypeId="2" ParentId="27" CreationDate="2012-02-28T18:01:38.410" Score="9" Body="&lt;p&gt;The following two methods are given in &lt;em&gt;Functions of Matrices: Theory and Computation&lt;/em&gt; by Nicholas Higham, on page 81.  These formulas evaluate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r(X) = b_0 + \frac{a_1 X}{b_1+\frac{a_2 X}{b_2+\cdots + \frac{a_{2m-1} X}{b_{2m-1} + \frac{a_{2m}X}{b_{2m}}}}}&#xA;$$&#xA;where $X$ is a square matrix.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Top-down method:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$P_{−1}=I, Q_{−1}=0,P_0 =b_0 I,Q_0 =I$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;for j=1:2m&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$P_j =b_j P_{j−1}+a_jXP_{j−2}$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$Q_j =b_j Q_{j−1}+a_j X Q_{j−2}$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;end&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$r_m =P_{2m} Q_{2m}^{-1}$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bottom-up method:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$Y_{2m}	= (a_{2m}/b_{2m})X$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;for j=2m−1:−1:1&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Solve $(b_jI+Y_{j+1})Y_j =a_jX$ for $Y_j$.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;end &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$r_m=b_0I+Y_1$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The question asks for evaluation of the more general form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$b_0 + \frac{a_1 X_1}{b_1+\frac{a_2 X_2}{b_2+\cdots + \frac{a_{2m-1} X_{2m-1}}{b_{2m-1} + \frac{a_{2m}X_{2m}}{b_{2m}}}}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be evaluated by a simple generalization of the formulas above; for instance the bottom-up method becomes&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$Y_{2m}	= (a_{2m}/b_{2m})X_{2m}$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;for j=2m−1:−1:1&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Solve $(b_jI+Y_{j+1})Y_j =a_j X_j$ for $Y_j$.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;end &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$r_m=b_0I+Y_1$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-07-05T04:31:33.290" LastActivityDate="2012-07-05T04:31:33.290" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1468" PostTypeId="1" AcceptedAnswerId="1475" CreationDate="2012-02-28T19:03:28.193" Score="5" ViewCount="553" Body="&lt;p&gt;Can someone provide an article or book that explains the principle used in frontal solvers? Some examples also may help understand the frontal methods better.Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="1059" LastEditorUserId="304" LastEditDate="2012-02-29T22:03:42.373" LastActivityDate="2012-02-29T22:03:42.373" Title="Recommendation for a good article/book for frontal methods?" Tags="|linear-algebra|linear-solver|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1469" PostTypeId="1" AcceptedAnswerId="1472" CreationDate="2012-02-28T20:37:06.690" Score="10" ViewCount="2395" Body="&lt;p&gt;How is (generalized) geometric programming different from general convex programming?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A geometric program can be transformed into a convex program, and is typically solved by an interior point method. But what is the advantage over directly formulating the problem as a convex program and solving it by an interior point method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the class of geometric programs only constitutes a subset of the class of convex programs that can be solved especially efficient by interior point methods? Or is the advantage simply that a general geometric program can easily be specified in computer readable form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, are there convex programs that cannot be approximated reasonably well by geometric programs?&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="276" LastEditDate="2012-02-29T17:53:18.650" LastActivityDate="2012-02-29T17:53:18.650" Title="How is geometric programming different from convex programming?" Tags="|optimization|convex-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1470" PostTypeId="2" ParentId="1469" CreationDate="2012-02-28T21:14:35.337" Score="3" Body="&lt;p&gt;Geometric programming can be transformed into a class of convex programs which is a strict subset of the set of all convex programs. However arbitrary convex programs cannot be expressed as geometric programs. The constraints in Geometric Programming are restricted to be of the form $f(x) \leq 1$ where $f(x)$ is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Posynomial&quot; rel=&quot;nofollow&quot;&gt;posynomial&lt;/a&gt; (i.e. a polynomial with positive coefficients). Posynomials are closed under addition, multiplication and &lt;em&gt;positive&lt;/em&gt; scaling. Now if you consider the constraint $-x-y \leq 1$, then $-x-y$ is not itself a posynomial. It cannot be represented as the addition/multiplication of posynomials and it cannot be a positive scaling of another posynomial. Thus it cannot be part of a geometric program. But the constraint above is linear and thus the program is convex. There does exist &lt;em&gt;mixed linear&lt;/em&gt; geometric programming in which one can add in arbitrary linear constraints though which can be also converted into a convex program and solved efficiently. Again more complicated constraints like $-x^2-y^2\leq 1$ cannot be handled even by mixed linear geometric programming for the same reason as given above.&lt;/p&gt;&#xA;" OwnerUserId="288" LastEditorUserId="288" LastEditDate="2012-02-28T21:51:56.037" LastActivityDate="2012-02-28T21:51:56.037" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1471" PostTypeId="2" ParentId="1468" CreationDate="2012-02-28T21:30:23.950" Score="1" Body="&lt;p&gt;Irons, B. M., 1970. A frontal solution scheme for finite element analysis. Int. J. Numer. Methods Eng. 2, 5--32  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Duff, I.S. , Erisman, A. M. , Reid, J.K. 1986. Direct methods for sparse matrices, Oxford University Press, Inc., New York, NY &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Davis, T. (2006). Direct Methods for Sparse Linear Systems.  Fundamentals of Algorithms. SIAM, Philadelphia.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-02-28T21:30:23.950" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1472" PostTypeId="2" ParentId="1469" CreationDate="2012-02-28T21:37:31.967" Score="5" Body="&lt;p&gt;I'd actually never heard of geometric programming until this question. &lt;a href=&quot;http://www.stanford.edu/~boyd/papers/pdf/gp_tutorial.pdf&quot;&gt;Here&lt;/a&gt; is a review paper by Stephen Boyd, &lt;em&gt;et al&lt;/em&gt; (Vandenberghe is a co-author too) that is a tutorial on geometric programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Geometric programs as originally expressed are &lt;em&gt;not&lt;/em&gt; convex. For instance, $x^{1/2}$ is a posynomial, and it is not convex, so geometric programs aren't a strict subset of convex programming. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of transforming a geometric program into a convex program is that the original geometric program is not necessarily convex. If you solved the geometric program as a nonlinear program (NLP), you would need to use methods from non-convex optimization in order to guarantee a global optimal solution. These methods are more expensive than convex optimization methods, require more algorithmic tuning, and require initial guesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, if you use an algorithm from non-convex NLP, you would need to specify your feasible set as a compact set in $\mathbb{R}^{n}$; in geometric programs, $x &amp;gt; 0$ is a valid constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not clear if the set of geometric programs maps (through the log-exponential transformation) to a set of convex programs that solves particularly efficiently. I don't see any advantages to geometric programming beyond the transformation to convex programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your last question, I don't think the set of geometric programs is isomorphic to the set of convex programs, so I suspect that there are convex programs that cannot be expressed as geometric programs, and of these programs, I suspect that there are some that can't be approximated reasonably well by geometric programs. However, I don't have a proof or a counterexample.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-02-28T21:46:54.827" LastActivityDate="2012-02-28T21:46:54.827" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1473" PostTypeId="1" AcceptedAnswerId="1474" CreationDate="2012-02-29T08:52:48.880" Score="11" ViewCount="1762" Body="&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;How would you sort a cloud of points with respect to an unstructured mesh of hexahedral cells? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each cell has a centre and a unique label to represent it. There are two cloud points basically (original point cloud, and a point cloud of the cell centres), but the cell geometry information (bounding box) may be of use, I am not sure. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Results&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I've done some asking around, and searching through the literature:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if the mesh is hexahedral and unstructured, the problem is reduced to an orthogonal range search. For this purpose, k-d trees are most often used. If the mesh is refined based on an octree data structure, the range search algorithm can be built around it. The goal is to avoid dealing with the direct mesh geometry and concentrate on point cloud A - point cloud relationship B. Point cloud A: query points, point cloud B: mesh cell centres. &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="9" LastEditDate="2012-03-08T11:23:59.570" LastActivityDate="2014-02-20T00:29:24.170" Title="Sort a cloud of points with respect to an unstructured mesh of hexahedral cells" Tags="|algorithms|computational-geometry|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1474" PostTypeId="2" ParentId="1473" CreationDate="2012-02-29T10:43:19.587" Score="8" Body="&lt;p&gt;&lt;strong&gt;Important note:&lt;/strong&gt; This answer does &lt;strong&gt;not&lt;/strong&gt; answer the actual question, but it was left undeleted per request. Embarrassingly I confused hexahedral and hexagonal.   The question is about sorting points into arbitrary &lt;a href=&quot;http://en.wikipedia.org/wiki/Hexahedron&quot; rel=&quot;nofollow noreferrer&quot;&gt;hexahedral&lt;/a&gt; cells in 3D while this solution sorts points into regular hexagonal cells in 2D, or irregular ones that correspond to some Voronoi tesselation in any dimension.  This method is applicable only if the mesh was generated as a Voronoi tesselation in the first place (which does seem to be &lt;a href=&quot;http://www.mendeley.com/research/hexahedral-meshing-nonlinear-volumes-using-voronoi-faces-edges/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an occasionally used approach&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am not sure what you mean by sort here, but I assume you want to sort the point into hexagonal bins on the plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica is what I know, so I am going to show you how to do it in Mathematica, but the method can be ported to other systems.  The idea is that a hexagonal lattice is the dual of a triangular one: it can be generated as the Voronoi diagram of a points in triangular arrangement.  A point from the cloud belongs to a given hexagon if it is closer to the centre of that hexagon than to the centre of any other hexagon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method will work for meshes of different shapes too, as long as they can be generates as the Voronoi diagram of some point arrangement.  (E.g. the hexagons don't need to be regular.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let's generate the mesh.  This is a triangle lattice:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pts = Join @@ Table[{x, Sqrt[3] y}, {x, 0, 4}, {y, 0, 2}];&#xA;&#xA;points = Join[pts, TranslationTransform[{1/2, Sqrt[3]/2}] /@ pts];&#xA;&#xA;Needs[&quot;ComputationalGeometry`&quot;]&#xA;PlanarGraphPlot[points, LabelPoints -&amp;gt; False]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xdXrA.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its dual is the hexagonal one we are interested in:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DiagramPlot[points, LabelPoints -&amp;gt; False]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rtC39.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This builds a function &lt;code&gt;nf&lt;/code&gt; which finds the index of the hexagon centre that some cloud point is closest to.  It is the key to the method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nf = Nearest[N[points] -&amp;gt; Range@Length[points]];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now let's generate a cloud of 1000 random points and sort them with &lt;code&gt;nf&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cloud = RandomReal[{-1/2, 5}, {1000, 2}];&#xA;&#xA;indices = First /@ nf /@ cloud;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;indices&lt;/code&gt; contains the indices of the centres that each cloud point is closest to.  This is the information we needed.  Now we can make a histogram out of them ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Histogram[indices]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/v3Ce3.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... or colour each of them ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Show[&#xA; DiagramPlot[points, LabelPoints -&amp;gt; False],&#xA; Graphics@MapThread[{ColorData[3][#1], Point[#2]} &amp;amp;, {indices, cloud}],&#xA; PlotRange -&amp;gt; All, AspectRatio -&amp;gt; Automatic&#xA; ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/MmXFJ.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... or do any sort of fancy visualization we want.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tally = Tally[indices];&#xA;&#xA;ListDensityPlot[Join[points, List /@ Sort[tally][[All, 2]], 2], &#xA; InterpolationOrder -&amp;gt; 0, &#xA; Epilog -&amp;gt; (Text[#2, points[[#1]]] &amp;amp; @@@ tally), &#xA; PlotRange -&amp;gt; {{-.5, 5}, {-.5, 5}}, Mesh -&amp;gt; All, &#xA; ColorFunction -&amp;gt; (ColorData[&quot;BeachColors&quot;][1 - #] &amp;amp;)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fsoON.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The key point here was the function that finds the closest point to something (&lt;a href=&quot;http://reference.wolfram.com/mathematica/ref/Nearest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Nearest&lt;/code&gt;&lt;/a&gt;). Mathematica has this built in, but there's a chance your system doesn't.  If this is the case, please see &lt;a href=&quot;https://stackoverflow.com/q/9463165/695132&quot;&gt;this question&lt;/a&gt; on how to efficiently implement such a function (or just go with the naive linear time implementation if you don't have a huge amount of points to process).&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-02-20T00:29:24.170" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1475" PostTypeId="2" ParentId="1468" CreationDate="2012-02-29T17:01:26.890" Score="2" Body="&lt;p&gt;In my humble opinion, the most well-written paper for frontal method is: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.jstor.org/stable/10.2307/2132786&quot; rel=&quot;nofollow&quot;&gt;The Multifrontal Method for Sparse Matrix Solution: Theory and Practice&lt;/a&gt; by JWH Liu. The paper itself is very user-friendly, shows lots of examples, and explains the idea crystal clear.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Recently, many researchers tried embedding $H$- or $HSS$-matrix structures into the original multifrontal method to further reduce the complexity. For example, &lt;a href=&quot;http://epubs.siam.org/sima/resource/1/sjmael/v31/i3/p1382_s1&quot; rel=&quot;nofollow&quot;&gt;Superfast Multifrontal Method for Large Structured Linear Systems of Equations&lt;/a&gt;&#xA;by Jianlin Xia, et al. In that paper he wrote, the complexity for 2D problem was $\mathcal{O}(n)$, and an $\mathcal{O}(n^{\frac{4}{3}})$ complexity for 3D problem, which is quite astonishing result.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-02-29T17:07:49.597" LastActivityDate="2012-02-29T17:07:49.597" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1476" PostTypeId="2" ParentId="1338" CreationDate="2012-02-29T20:11:40.927" Score="2" Body="&lt;p&gt;So I've thought about this problem a lot, and this answer is the best I can come up with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose that $U$ is a $k \times m$ matrix, $M$ is an $m \times n$ matrix, and $E$ is a $k \times n$ matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $W$ is an $n \times m$ matrix such that $MW = I$ (in this case, $I$ is $m \times m$), then $U = EW$. Matrices having this property are called &lt;em&gt;right inverses&lt;/em&gt; of $M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The language of generalized inverses can be used to determine right inverses. If both matrices are full rank and $MW = I$, then $M$ and $W$ are $\{1,2\}$-inverses of each other, because $MWM = M$ and $WMW = W$. The best reference that I know of that discussed how to calculate $\{1,2\}$-inverses of matrices is &lt;em&gt;Generalized Inverses: Theory and Applications, 2nd edition&lt;/em&gt; by Thomas N.E. Greville and Adi Ben-Israel. Unfortunately, the authors don't really have an eye for computational efficiency, so you'll need to tweak their algorithms accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the converse doesn't necessarily hold: if $M$ and $W$ are $\{1,2\}$-inverses and $M$ and $W$ are full rank, $MW$ may not equal $I$. The reasoning is as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $M$ and $W$ are $\{1,2\}$-inverses, then $MW$ will be a projection matrix with rank equal to $\min\{m,n\}$. $MW$ is an $m \times m$ matrix. These two facts combined means that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If $m \leq n$, then $W$ will be a right inverse because $MW = I$.&lt;/li&gt;&#xA;&lt;li&gt;If $m &amp;gt; n$, then $W$ will not be a right inverse because $MW$ has rank $n$, and the only full rank projection matrix is the identity matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So the approach of determining a right inverse $W$ using $\{1,2\}$-inverses will only work if $m \leq n$ and $M$ is full rank. Otherwise, a right inverse of $M$ may not exist (there are no guarantees that it will, without imposing properties on $M$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't yield a fast algorithm to solve your problem for the matrix dimensions that you've given, but it is at least a partial solution.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-02-29T20:11:40.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1477" PostTypeId="2" ParentId="1463" CreationDate="2012-03-01T02:47:38.333" Score="6" Body="&lt;p&gt;I'll take a stab at it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The protein cartoon (also known as ribbon) representation consist of three parts corresponding to the three types of protein secondary structure.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Random Coil&lt;/strong&gt; (shown in green) - A B-spline, usually of order 2 or 3, passing through the alpha-carbons of each amino acid residue. Occasionally the spline will also pass through the amino-nitrogens to more closely show the conformation of the protein.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alpha Helix&lt;/strong&gt; (shown in red) - Another spline with a flattened 'ribbon' shape which wraps around an imaginary cylinder formed by the residues in the helix.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Beta Sheets&lt;/strong&gt; (shown in yellow) - Splines with the shape of the wide, flat arrow passing through the peptide plane (the plane formed by the alpha-carbon, carbonyl-carbon, and carbonyl-oxygen). The normal vector to the top of the arrow is the normal of the peptide plane. The arrows points from the N-terminus to the C-terminus of the protein chain.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The wikipedia page on &lt;a href=&quot;http://en.wikipedia.org/wiki/Ribbon_diagram&quot;&gt;ribbon diagrams&lt;/a&gt; has additional information on the origins of this type of visualization for showing protein structure.&lt;/p&gt;&#xA;" OwnerUserId="377" LastEditorUserId="377" LastEditDate="2012-03-01T05:41:17.257" LastActivityDate="2012-03-01T05:41:17.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1478" PostTypeId="1" AcceptedAnswerId="1479" CreationDate="2012-03-01T09:24:02.623" Score="7" ViewCount="776" Body="&lt;p&gt;Given certain function $f(X)$ which is quadratic in $X\in\mathbb{R}^{n\times d}$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{2}tr(X^TAX) - tr(Y^TBX)$$&#xA;for positive definite weighted Laplacian matrices $A, B\in\mathbb{R}^{n\times n}$, i.e. the diagonal elements are negative summation of the negative off-diagonal elements plus a constant for strict diagonal dominance:&#xA;$A=\left(\begin{array}{ccccc}&#xD;&#xA; c_1+\sum a_{1s} &amp;amp;   &amp;amp;    &amp;amp;  &amp;amp; \\&#xD;&#xA;   &amp;amp; \ddots &amp;amp;   &amp;amp; -a_{ij} \\&#xD;&#xA;   &amp;amp;  &amp;amp; \ddots &amp;amp; &amp;amp; \\&#xD;&#xA;   &amp;amp; -a_{ij} &amp;amp;   &amp;amp; \ddots &amp;amp; \\&#xD;&#xA;   &amp;amp; &amp;amp; &amp;amp; &amp;amp; c_n+\sum a_{ns}&#xD;&#xA; \end{array}\right) \in\mathbb{R}^{n\times n},$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and an arbitrary $Y\in\mathbb{R}^{n\times d}$. In order to obtain its minimim, I set the gradient to $0$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla f(X) = AX-BY \tag 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so, the solution to the following linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$AX=BY \tag 2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the unknown $X$ would give me the minimizer of $f(X)$. Since $A$ is strictly diagonally dominant, the system is solved by Jacobi iteration known to converge to exact solution in this case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm interested if only $a$ $single$ iteration of Jacobi method on some&#xA;arbitrary initialization $X_0$ would yield result $X_1$ that satisfies $f(X_1)&amp;lt;f(X_0)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I tried many books on Jacobi method, but I was unable to make the connection with the optimization problem. The Jacobi method is known to yield &quot;progressive better results&quot; to the linear system, but I'm not sure what this implies precisely, and what implications it might have on the above optimization attempt. On one place I found that the Jacobi iterands $\{X_0, X_1, \dots, X_{k-1}, X_{k}\}$ satisfy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\lVert X-X_k\rVert_2 &amp;lt; \lVert X-X_{k-1} \rVert_2 \tag 3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $X$ is the true solution to (2). Does the proof on the convergence of Jacobi actually imply (3)? And, if so, does that imply progressively lower values of $f(X_k)$, $k\in\{0, 1, \dots, \}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way I tried is $f(X_1)-f(X_0)&amp;lt;0$, with the replacement as in Jacobi&#xA;$$X_1=D^{-1}(BY-RX_0)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the splitting $A=D+R$, where $D=diag(A)$, but was unable to regroup the terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if the answers would contain supporting references, so that I might explore the connection more deeply.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-03-01T11:10:31.583" LastActivityDate="2012-03-03T13:38:03.573" Title="Jacobi iteration to reduce the quadratic function" Tags="|linear-algebra|optimization|iterative-method|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1479" PostTypeId="2" ParentId="1478" CreationDate="2012-03-01T13:37:18.490" Score="3" Body="&lt;p&gt;Your conjecture is true. Let us define $\Delta x_k = x_{k+1}-x_k$, $x=A\backslash b$, $e_k=x-x_k$.  By the definition of iteration, $\Delta x_k = D^{-1}Ae_k$.&#xA;So &#xA;$$ f(x_{k+1}) - f(x_k) \\&#xD;&#xA;= x_k^TA\Delta x_k + \frac{1}{2}\Delta x_k^TA\Delta x_k - (Ax)^T \Delta x_k\\&#xD;&#xA;= -e_k^TA\Delta x_k + \frac{1}{2}\Delta x_k^T A \Delta x_k\\&#xD;&#xA;= \frac{1}{2}e_k^T A D^{-1}AD^{-1} A e_k - e_k^T A D^{-1}A e_k&#xD;&#xA;= \frac{1}{2}v_k^T D^{-1/2}AD^{-1/2} v_k-v_k^Tv_k&#xD;&#xA;= v_k^T(\frac{1}{2}D^{-1/2}AD^{-1/2}-I)v_k.&#xD;&#xA;$$&#xA;Then, from convergence condition of Jacobi iteration that the spectra of $I-D^{-1}A$ is less than one, you can deduce your conjecture.  To this end, we note that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; $\frac{1}{2}D^{-1/2}AD^{-1/2}-I$ is symmetric so its definiteness is equivalent to&#xA;its spectra lies on one side of zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; $D^{-1/2}AD^{-1/2}$ have the same spectra with $D^{-1}A$ from the fact spectra is invariant under commutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; The spectra of $D^{-1}A$ lies in $(0,2)$.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2012-03-03T13:38:03.573" LastActivityDate="2012-03-03T13:38:03.573" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1480" PostTypeId="2" ParentId="223" CreationDate="2012-03-01T15:31:40.720" Score="1" Body="&lt;p&gt;I think that the main difference between vendor BLAS libraries and open source BLAS libraries is the time that it takes for open source to support the latest hardware features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As BLAS is widely used, it is of the interest of the vendor to support the latest hardware features. For instance, consider &lt;a href=&quot;http://en.wikipedia.org/wiki/Advanced_Vector_Extensions&quot; rel=&quot;nofollow&quot;&gt;Intel's AVX vector extensions&lt;/a&gt; that were introduced with the &quot;sandy-bridge&quot; processor on Jan 2011. MKL had AVX support even before the CPU was available but it was only recently that ATLAS (late 2011) started to rollout support for AVX.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if your application really needs performance and before you start benchmarking different BLAS libraries or start hacking around optimizing anything: profile your application. It is common that human intuition is not a very effective profiler predictor, at least I know mine is not! So instead of spending time randomly optimizing, profile your application and systematically approach each bottleneck.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="1070" LastEditDate="2012-03-03T11:21:42.740" LastActivityDate="2012-03-03T11:21:42.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1481" PostTypeId="1" CreationDate="2012-03-01T16:02:32.517" Score="6" ViewCount="483" Body="&lt;p&gt;DISCLAIMER: I've edited the question repeatedly for clarity and to target the most relevant answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following general problem&#xA;$$ \min \|h_1\cdot h_2\|^2 $$&#xA;such that&#xA;$$\|g_1\wedge g_2-h_1\wedge h_2\|^2 = 0,$$&#xA;where $h_i,g_i\in \mathbb{R}^n\wedge \mathbb{R}^n$ are skew-symmetric matrices, $A\cdot B$ denotes matrix multiplication of matrices $A$ and $B$, and $\lambda\in \mathbb{R}$. $A\wedge B$ is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Geometric_product&quot; rel=&quot;nofollow&quot;&gt;geometric product&lt;/a&gt; of $A$ and $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make the problem solve nicer, i.e., no infeasibility trap at $h_1=0$, for instance, I've reformulated the problem with the following constraints.&#xA;$$\|g_1\wedge g_2-\lambda h_1\wedge h_2\|^2 = 0,\\&#xD;&#xA; \lambda\ge 0,\\&#xD;&#xA;\|h_1\|^2=1,\\&#xD;&#xA;\|h_2\|^2=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My test case is $n=6$ and $g_1=g_2=e_{12}+e_{34}+e_{56}$ (a worst case scenario). Obviously, the problem is degenerate due to symmetry as well as unitarily invariant on $\mathbb{R}^6$. I've introduced $\lambda$ and constraints 2 and 3 in order to exclude some slack in the solution. My expected outcome is $h_1 =\frac{1}{\sqrt{12}}\left(2e_{12}+e_{34}+e_{56}\right)$ and $h_2=\frac{1}{\sqrt{2}}\left(e_{34}+e_{56}\right)$ with a minimum of 1/12 and $\lambda=\sqrt{24}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used IPOPT to implement the minimzation, but it fails to find this solution. Instead a solution is found that is slightly worse. I'm a little perplexed as all functions are convex in each variable, but I'm no expert in NLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: As per @Geoff Oxberry, the problem is not convex. As a matter of fact, the initial guess of $h_1=g_1$ and $h_2=g_2$ is a maximum of the objective function under the constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how do I improve my initial guess or change constraints to improve solutions?&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2012-03-02T13:15:58.367" LastActivityDate="2012-03-02T13:15:58.367" Title="Reducing degeneracy in constrained (convex) optimization problem" Tags="|optimization|nonlinear-programming|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1483" PostTypeId="1" AcceptedAnswerId="1490" CreationDate="2012-03-01T16:51:59.990" Score="6" ViewCount="4008" Body="&lt;p&gt;I am curious how to quickly compute the eigenvalues for arbitrary matrices, sparse or dense, restricted on some given interval of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have an arbitrary $n\times n$ matrix $A$, normally the complexity of the computing all the eigenvalues for $A$ is $O(n^3)$, I wonder if we could find an algorithm that does the same job while only bears an $O(n^2)$ or less complexity, or more specifically, the eigenvalues on an interval $(a,b)$. Is this possible at all? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For symmetric matrix, to my knowledge, we could use bisection method to somewhat get the job done, I would like to know if there are other tricks for general matrices as well.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-03-01T17:26:24.423" LastActivityDate="2012-03-02T22:17:45.053" Title="Fast algorithms to find the eigenvalues of some matrix on intervals of interest" Tags="|linear-algebra|complexity|eigensystem|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1484" PostTypeId="2" ParentId="366" CreationDate="2012-03-01T17:20:12.310" Score="6" Body="&lt;p&gt;One big PRO is the number of vendors behind OpenCL. I have some anecdotal experience about this, having met a research group that spent a large amount of time and effort to develop a fairly complicated CUDA code for an NVIDIA powered system. A year later after the code was developed, the research group got access to a larger and faster AMD based system but they could not use it as they did not have the (human) resources to port the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if the core set of features of CUDA and OpenCL are almost identical (as @JonathanDursi well pointed out), if the original developer is not the one assigned with the task of converting the code, the whole porting project can be quite time consuming. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are some official incompatibilities between CUDA and OpenCL. Most remarkably CUDA support c++ templates while OpenCL does not yet officially support them. However, there is an effort made by AMD to develop an extension to OpenCL with support for templates and other C++ features, more information &lt;a href=&quot;http://blogs.amd.com/developer/2012/01/10/amd-opencl%E2%84%A2-app-sdk-2-6-introducing-opencl%E2%84%A2-1-2-preview-and-static-c-kernel-language/&quot; rel=&quot;nofollow&quot;&gt;in this post from AMD dev central&lt;/a&gt;. I hope a future revision of OpenCL might add this work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point in time (early 2012), the awesome libraries that @MattKnepley links to are closed source or use templates, so they will not become available for hardware other than NVIDIA, at least in the mean time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For someone that is learning gpu-computing, I would say that OpenCL C can be quite difficult, as there are a lot of details that distract the learner from the basic ideas, whereas CUDA is simpler and straight forward. However, there are tools that make OpenCL much more simple to learn and use like PyOpenCL (the python wrapper for opencl) which brings all the python sugar to OpenCL (note that there is also a PyCUDA). For instance, the PyOpenCL demo for adding two arrays is just under 25 lines and it includes: creation of the arrays on host and device, the data transfers, the creation of the context and queue, the kernel, how to build and execute the kernel, getting the results from the GPU and comparing them against numpy (see links below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PyOpenCL -- &lt;a href=&quot;http://mathema.tician.de/software/pyopencl&quot; rel=&quot;nofollow&quot;&gt;http://mathema.tician.de/software/pyopencl&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PyCUDA -- &lt;a href=&quot;http://mathema.tician.de/software/pycuda&quot; rel=&quot;nofollow&quot;&gt;http://mathema.tician.de/software/pycuda&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For experienced gpu-programers, here I agree with @JonathanDursi, CUDA and OpenCL are fundamentally the same and there are really no mayor differences. Moreover, the hard work of developing an efficient algorithm for GPUs is very much language independent, and the OpenCL support from vendors and the documentation is now much more mature than say 2 years ago. The only point that still makes a difference, is that NVIDIA is really doing some great work with their support to the CUDA community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenCL has the added benefit that it can run on CPUs and is already supported by Intel and AMD. So you don't need to change your algorithmic framework if you want to take advantage of any available CPU cores. It is not my opinion that OpenCL is the best solution for a single CPU / multicore oriented application as a CPU optimized kernel might look significantly different than a GPU optimized kernel. However, in my experience CODE development does benefit from being able to run on the CPU.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="1070" LastEditDate="2012-04-16T15:09:16.450" LastActivityDate="2012-04-16T15:09:16.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1485" PostTypeId="1" AcceptedAnswerId="1569" CreationDate="2012-03-01T17:21:25.057" Score="11" ViewCount="1367" Body="&lt;p&gt;I want to know which of the classic linear solvers (e.g Gauss-Seidel, Jacobi, SOR) are guaranteed to converge for the problem $Ax=b$ where $A$ is positive &lt;strong&gt;semi&lt;/strong&gt; definite and of course $b \in im(A)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Notice $A$ is semi definite and not definite)&lt;/p&gt;&#xA;" OwnerUserId="898" LastEditorUserId="123" LastEditDate="2012-03-03T06:50:06.350" LastActivityDate="2018-09-11T14:04:51.567" Title="Which iterative linear solvers converge for positive semidefinite matrices?" Tags="|linear-algebra|iterative-method|convergence|linear-solver|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="1486" PostTypeId="2" ParentId="1483" CreationDate="2012-03-01T18:01:13.570" Score="1" Body="&lt;p&gt;Inverse subspace iteration: &lt;a href=&quot;http://en.wikipedia.org/wiki/Inverse_iteration&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Inverse_iteration&lt;/a&gt; is available in ARPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Filter diagonalization: &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/Harminv&quot; rel=&quot;nofollow&quot;&gt;http://ab-initio.mit.edu/wiki/index.php/Harminv&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rayleigh-Chebyshev: &lt;a href=&quot;ftp://ftp.math.ucla.edu/pub/camreport/cam10-05.pdf&quot; rel=&quot;nofollow&quot;&gt;ftp://ftp.math.ucla.edu/pub/camreport/cam10-05.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are designed for that problem. Inverse iteration is the standard approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I just noticed you're looking for possibly asymmetric eigenvalue problems. Filter diagonalization can still work but it's trickier. See &lt;a href=&quot;http://www.cs.tsukuba.ac.jp/techreport/data/CS-TR-08-13.pdf&quot; rel=&quot;nofollow&quot;&gt;www.cs.tsukuba.ac.jp/techreport/data/CS-TR-08-13.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="979" LastEditorUserId="979" LastEditDate="2012-03-01T22:22:13.073" LastActivityDate="2012-03-01T22:22:13.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1487" PostTypeId="2" ParentId="1481" CreationDate="2012-03-01T18:22:02.373" Score="2" Body="&lt;p&gt;Your problem is not convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, $\|h_{1}\|^{2} = 1$ and $\|h_{2}\|^{2} = 1$ are not convex constraints, even though the functions on the left-hand sides are convex. The easiest way to picture this in a simpler case is to note that the unit circle (or sphere) is not a convex set because the line between any two points on the circle (or sphere) is not contained in the set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the function $\|h_{1} \cdot h_{2}\|^{2}$ is probably not convex because it contains bilinear terms. Bilinear terms are known not to be convex, and I doubt that composing them with a 2-norm changes the non-convexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since IPOPT is a local optimization solver only guaranteed to find the global solution under convexity assumptions, it probably converged to the local minimum nearest your guess (though without knowing more about your guess and the problem iterations, I can't really say for sure).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-03-01T18:22:02.373" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1488" PostTypeId="1" AcceptedAnswerId="1496" CreationDate="2012-03-01T18:24:48.557" Score="4" ViewCount="326" Body="&lt;p&gt;A few of the iterative matrix algorithms (CG,GMRES etc.) I have authored are acting rather &lt;em&gt;funny&lt;/em&gt;. They converge to the right answers but take abnormally long time to run. I am in the process of finding out why. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the first steps I thought is to find out the algorithmic complexity of the methods. For instance, I need to know if CG is indeed taking $O(N^2)$ as expected and similar.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I find out the exact algorithmic complexity of the methods? (I am looking for some way for to get the exact bound on the algorithm like $N^2 + 5N$). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And even before that, is this a valid first step for improving performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of now, I am concentrating on single core performance (which by itself is pretty bad right now). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also looking at trying to determine the memory accesses. Is there any Free software (not necessarily Open Source) which would allow me to do so in a nice GUI? I use VTune for parallel but it is useless for serial.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I have tried googling but all algorithm problems end up in the search/sort portion of computing which is quite different from iterative matrix algorithms.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have tried solving the algorithm for matrix sizes from 1 to 1000, plot the graph of average time taken for that size and curve fit it (quadratic). But I don't seem to be getting anywhere through this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: Just to be clear, I want to &lt;strong&gt;verify&lt;/strong&gt; that the complexity of the algorithm in practice is the same as the one predicted by theory. I want to &lt;strong&gt;verify&lt;/strong&gt; that my algorithm of say, Matrix-Vector is indeed $O(N^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further, I don't mind knowing per iteration complexity.&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="529" LastEditDate="2012-03-02T14:26:59.403" LastActivityDate="2022-10-14T10:38:53.430" Title="Determining the algorithmic complexity" Tags="|algorithms|iterative-method|matrix|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1489" PostTypeId="1" AcceptedAnswerId="1567" CreationDate="2012-03-01T18:42:00.267" Score="5" ViewCount="693" Body="&lt;p&gt;I am looking for preconditioners which don't assume anything about the matrix or its origins.&#xA;I basically want to be able to type in the following in MATLAB and have quick solving time:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = rand(5000,5000);&#xA;b = rand(5000,1);&#xA;precond_a= my_precond_algorithm(a);&#xA;qmr(a,b,1e-8,100,precond_a)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Needless to say, $a$ is dense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked into:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;LU works well. But that's no surprise.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am still to find a good algorithm for ILU for dense matrices but I reckon that should work relatively well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sparse Inverse Approximators (Benzi et. al.) . &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A paper by Prakash and Mittra discusses the use of Multifrontal Preconds for solving dense Maxwell Equations discretization.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Other than LU, I am still a little concerned about the viability of using them as &lt;strong&gt;effective&lt;/strong&gt; preconditioners for large dense matrices. Any resources/comments would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="549" LastEditDate="2012-03-04T17:30:41.103" LastActivityDate="2012-03-08T11:51:13.263" Title="On Vanilla Preconditioners for solving dense $Ax=b$ iteratively" Tags="|preconditioning|iterative-method|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1490" PostTypeId="2" ParentId="1483" CreationDate="2012-03-01T23:28:39.713" Score="6" Body="&lt;p&gt;For a direct general result (as opposed to iterative approximation) you will have to compute the largest eigenvector first to find the smaller ones in the remaining orthonormal sub-space.  Basically compute the largest eigenvector (mean projection of a unit-vector $O(N^2)$), and repeat in the remaining $N$-subspaces, subtracting previous projections against the matrix.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;em&gt;think&lt;/em&gt; 'interval' is a somewhat poorly formed concept here.  Basically if you compute the first eigenvector (and corresponding eigen-value), you may discover that its magnitude is below the desired window, in which case you hit your $N^2$ lower bound of computation.  I think the upper interval bound $a$ will not matter, but the lower will decide what partition of the eigenvalues you must compute, hence leaving you somewhere between $N^2$ and $N^3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sparse matrices checkout &lt;a href=&quot;http://en.wikipedia.org/wiki/Arnoldi_iteration&quot; rel=&quot;nofollow&quot;&gt;Arnoldi Iteration&lt;/a&gt;, and the corresponding math on Krylov subspaces/methods.  These algorithms do a really good job at approximating the largest eigenvalues quickly, but sometimes grind a long time when the matrix has a poor condition number or other undesirable spectral properties.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2012-03-02T22:17:45.053" LastActivityDate="2012-03-02T22:17:45.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1491" PostTypeId="2" ParentId="1207" CreationDate="2012-03-01T23:43:36.780" Score="1" Body="&lt;p&gt;Three easy steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Set out to solve a problem you find interesting.  (I have some if you need more!)&lt;/li&gt;&#xA;&lt;li&gt;Figure out what other OS people have done, try using their software.&lt;/li&gt;&#xA;&lt;li&gt;Find the shortcomings and improve them!  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-03-01T23:43:36.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1492" PostTypeId="2" ParentId="1488" CreationDate="2012-03-02T00:29:50.263" Score="0" Body="&lt;p&gt;Start by looking at the best out there, see: &lt;a href=&quot;http://netlib.org/blas/&quot; rel=&quot;nofollow&quot;&gt;BLAS&lt;/a&gt;/&lt;a href=&quot;http://www.netlib.org/lapack/&quot; rel=&quot;nofollow&quot;&gt;LAPACK&lt;/a&gt;/&lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow&quot;&gt;ARPACK&lt;/a&gt;/&lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/umfpack/&quot; rel=&quot;nofollow&quot;&gt;UMFPACK&lt;/a&gt;/&lt;a href=&quot;http://trilinos.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;Trilinos&lt;/a&gt;/&lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt; (in rough order of level of sophistication)&#xA;These all rely on highly processor-specific, cache-optimized operations, and hence are nearly as fast as can be expected. (These are all c/c++/fortran monsters and there is a reason for that!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume this is related to your previous questions about home-made linear algebra solvers, and I want to reiterate that making your own implementation, while a valuable learning experience, will almost certainly be slower.  In my experience, you are doing really well if you get within a factor (i.e. 1/2) of the performance of these codes!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mostly what causes big performance hits after you have validated the correctness of the algorithim is accidental allocation/copies/deallocation of memory.  A quick run of &lt;a href=&quot;http://valgrind.org/&quot; rel=&quot;nofollow&quot;&gt;Valgrind&lt;/a&gt; might help with this one.  Other than that, make sure you are not storing intermediate products or doing extra work!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, it is worth note that iterative methods act finnicky given certain inputs.  Start by making sure your implementation behaves appropriately given certain sorts of known inputs that have well-defined execution pathways.  An example might be solving a diagonal, or Laplace matrix where the intermediate products have an easily graph-interpertable form.&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2012-03-02T06:51:56.660" LastActivityDate="2012-03-02T06:51:56.660" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1493" PostTypeId="1" AcceptedAnswerId="1494" CreationDate="2012-03-02T02:59:05.353" Score="6" ViewCount="167" Body="&lt;p&gt;Suppose we want to solve the Poisson equation $\Delta u = f$ on a domain $\Omega$ with Dirichlet boundary conditions. One possible way to do is by a domain decomposition method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a condition that the domain has to be decomposed into subdomains such that each subdomain touches the boundary of $\Omega$. Otherwise the scheme is supposed not to be well-posed. I do not understand that. Can you help me?&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-03-02T03:10:37.033" Title="Compability conditions in domain decomposition methods" Tags="|domain-decomposition|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1494" PostTypeId="2" ParentId="1493" CreationDate="2012-03-02T03:10:37.033" Score="5" Body="&lt;p&gt;This result applies only to the most naive one-level Neumann-Neumann domain decomposition methods. If a subdomain does not touch a Dirichlet boundary, it will be &quot;floating&quot;, thus has a null space. Classical Neumann-Neumann and original FETI methods identify the null space to control the sudomain problems and converge with general partitions. The early two-level methods use this null space characterization to build coarse spaces. Those methods tend to be fragile and have fallen out of favor since FETI-DP and BDDC (essentially equivalent dual and primal Neumann-Neumann methods) were developed. These methods use simpler and more general coarse spaces to control the subdomain null spaces.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-02T03:10:37.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1495" PostTypeId="2" ParentId="1488" CreationDate="2012-03-02T03:11:19.330" Score="2" Body="&lt;p&gt;One can compute the complexity of an iteration for iterative sovlers (e.g. number of operations), but this might fluctuate because of compiler magic. However, counting gets a good impression of what happends, and you can do it by yourself (or have the code couting it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A complexity analysis for the total run will be hard, because the cost measure is somehow arbitrary, and, much more important, the total run time for, e.g., conjugate gradient will depend on the spectral properties of your matrix.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-03-02T03:11:19.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1496" PostTypeId="2" ParentId="1488" CreationDate="2012-03-02T16:19:24.247" Score="5" Body="&lt;p&gt;You have the code so implement a global counter for the number of floating point operations and increment the counter on every function call. There is no way around counting operations. You can use a similar global counter to measure the number of memory accesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the performance estimates, measure the time for a single iteration to get the FLOP/TIME. Then you can compare FLOP/TIME against your processor peak performance.&#xA;Of course, you can do the same for the total number of data movements (in Bytes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A piece of software that has counting flops functionality is PETSc, see their documentation for functions &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Profiling/PetscLogFlops.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PetscLogFlops&lt;/a&gt; and the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Profiling/PetscGetFlops.html#PetscGetFlops&quot; rel=&quot;nofollow noreferrer&quot;&gt;PetscGetFlops&lt;/a&gt;. The documentation provides good example codes and link to source code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an instructive discussion on what it involves to get close to peak FLOPS performance on current CPUs, see this &lt;a href=&quot;https://stackoverflow.com/questions/8389648/how-to-achieve-4-flops-per-cycle&quot;&gt;thread&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a book reference you can look into chapter 3 of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0262033844&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Algorithms&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-03-04T18:45:53.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1497" PostTypeId="1" AcceptedAnswerId="1498" CreationDate="2012-03-02T22:02:27.293" Score="9" ViewCount="2208" Body="&lt;p&gt;Does there exist a standard format for finite element meshes which is widely used in the industry?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1081" LastActivityDate="2012-03-03T03:02:25.613" Title="Standard format for finite element meshes" Tags="|finite-element|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1498" PostTypeId="2" ParentId="1497" CreationDate="2012-03-02T22:42:55.820" Score="5" Body="&lt;p&gt;&quot;The nice thing about standards is that you have so many to choose from.&quot; -&lt;a href=&quot;http://en.wikipedia.org/wiki/Andrew_S._Tanenbaum&quot;&gt;Andrew S. Tanenbaum&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience with using 3-4 different pieces of FEM software (and writing a few of my own) is that there is not a silver-bullet gold-standard out there.  Some Creative Googling will give you some options, ranging in implementation from &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/data/mesh/mesh.html&quot;&gt;ascii-driven&lt;/a&gt;, &lt;a href=&quot;http://hpfem.org/hermes/doc-tutorial/src/hermes2d/P01-linear/01-mesh.html&quot;&gt;XML based&lt;/a&gt;, to &lt;a href=&quot;http://www.hdfgroup.uiuc.edu/papers/prototypes/mesh/&quot;&gt;hdf5&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be really pleased to discover that I am wrong about this . . .&lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-03-02T22:42:55.820" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1499" PostTypeId="2" ParentId="1497" CreationDate="2012-03-03T01:15:53.727" Score="4" Body="&lt;p&gt;Exodus II (utilizes NetCDF) is used by many ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.osti.gov/bridge/purl.cover.jsp?purl=/10102115-8&quot; rel=&quot;nofollow&quot;&gt;http://www.osti.gov/bridge/purl.cover.jsp?purl=/10102115-8&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is better pdf somewhere but I cant seem to find it right now&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2012-03-03T01:15:53.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1500" PostTypeId="2" ParentId="1497" CreationDate="2012-03-03T03:02:25.613" Score="1" Body="&lt;p&gt;It all cooks down to &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a coordinate table for all vertices in mesh&lt;/li&gt;&#xA;&lt;li&gt;a connectivity table defining elements in terms of global numbers for vertices&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which completely describes a given mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is easy to construct conversion scripts which can convert to other formats etc. &lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-03-03T03:02:25.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1502" PostTypeId="1" CreationDate="2012-03-03T05:56:31.237" Score="14" ViewCount="386" Body="&lt;p&gt;Case studies in my PhD thesis require that I have interval extensions of Fortran subroutines in &lt;a href=&quot;http://www.sandia.gov/chemkin/FAQs.html&quot;&gt;CHEMKIN-II&lt;/a&gt; (apologies for the link; it's the best one I could find for a package no longer distributed by Sandia National Laboratory). CHEMKIN-II is a chemistry package used in combustion chemistry; since it's the standard package used in combustion, I can't (or rather, would really prefer not to) use a different package out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to get the required interval extensions, I've been using a tool developed in-house called &lt;a href=&quot;http://yoric.mit.edu/DAEPACK&quot;&gt;DAEPACK&lt;/a&gt;. The old binaries I have used to be able to generate the required interval extensions of the CHEMKIN-II subroutines I needed on a 32-bit OS (with a version of libstdc corresponding to gcc 2.95) a couple years ago, but have since fallen out of date, and would be a tremendous ordeal to use (because I'd have to find 32-bit versions of every other piece of software I used). The source code for this package is also proprietary, but since it was originally developed at MIT, and I work at MIT, I was able to (after a great deal of effort) get the source code for DAEPACK. Using the source code I have, I was able to compile a version of DAEPACK compatible with 64-bit versions of Linux and versions of libstdc corresponding to gcc 4.x.x. However, this source code, which I suspect is old, compiles into an executable that does not successfully generate the interval extension code I need in order to complete my thesis. I have contacted the developer in an attempt to get his help in fixing the source code, but from previous contact with the developer, I know that it will take time for him to fix the source code, if he does at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the big problem: &lt;strong&gt;I have 6 months left until I need to graduate because I run out of funding.&lt;/strong&gt; I can't rely on the developer to fix DAEPACK quickly enough for me to successfully complete my PhD thesis, which is where you guys come in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a package that does the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generates interval extensions, Taylor models (using interval arithmetic), and derivatives of almost arbitrary Fortran 77 code; I think the only structures that aren't parsed by DAEPACK are &lt;code&gt;DO&lt;/code&gt; loops and &lt;code&gt;IF&lt;/code&gt; statements. Since I need Taylor models, the code must also have automatic differentiation (AD) capability.&lt;/li&gt;&#xA;&lt;li&gt;Generates these interval extensions and Taylor models by source-to-source translation of the Fortran 77 code (i.e., it parses the source code like a compiler to generate the source code that calculates the interval extensions and Taylor models). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In a pinch, I can use software that does the above two things to C code instead of Fortran 77 code, and then pass everything through &lt;code&gt;f2c&lt;/code&gt;. I'd really prefer not to do it this way, because I have no idea whether or not passing everything through &lt;code&gt;f2c&lt;/code&gt; will even work, but I'm pretty desperate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you guys help a desperate PhD student out?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To anticipate some possible suggestions, here are approaches that will NOT work for me, given the time constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Trying harder to get DAEPACK to work; believe me, I've tried. And I'm bugging the developer frequently.&lt;/li&gt;&#xA;&lt;li&gt;Tools that calculate interval extensions or Taylor models using object-oriented (or object-oriented-like) approaches. I don't have time to recode CHEMKIN-II (or any other similarly large library) with new types; it's too big.&lt;/li&gt;&#xA;&lt;li&gt;Using any language other than Fortran 77/90 or C. The CHEMKIN-II code is in Fortran. I can convert it to C using &lt;code&gt;f2c&lt;/code&gt;, or if need be, use a Sandia-based clone of CHEMKIN-II called TChem. My old results use CHEMKIN-II; my new results should replicate my old results, and I already know that Cantera won't replicate what CHEMKIN-II does based on case studies and unit testing. Furthermore, the graduate student that preceded me did some black magic to the CHEMKIN-II source code to get interval arithmetic to work especially well. It would be difficult for me to replicate that in any other package.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-03-03T06:29:54.300" LastActivityDate="2014-05-22T16:01:30.163" Title="Is there a tool out there that can generate interval extensions of Fortran (or C) functions by parsing Fortran (or C) code?" Tags="|fortran|c|automatic-differentiation|interval-arithmetic|" AnswerCount="1" CommentCount="11" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1503" PostTypeId="1" AcceptedAnswerId="1842" CreationDate="2012-03-03T07:22:34.077" Score="5" ViewCount="491" Body="&lt;p&gt;By good I mean minimal and essential.&#xA;One whose concepts form a minimum spanning tree, and whose words are precious :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A small pdf would be perfect)&lt;/p&gt;&#xA;" OwnerUserId="1083" LastEditorUserId="1083" LastEditDate="2012-03-03T14:01:16.270" LastActivityDate="2022-07-22T06:46:07.120" Title="What is a good introduction to graph theory / algorithm" Tags="|graph-theory|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1504" PostTypeId="2" ParentId="136" CreationDate="2012-03-03T14:35:55.493" Score="8" Body="&lt;p&gt;For the &lt;strong&gt;classical gravitational n-body problem&lt;/strong&gt;, I think the following two papers do a good job at discussing the guts of the parallel implementation for the force evaluation step. Although the papers discuss a GPU implementation, they do a good job at discussing the parallelism and provide details of the algorithms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://http.developer.nvidia.com/GPUGems3/gpugems3_ch31.html&quot;&gt;paper&lt;/a&gt; by Nyland, Harris, and Prins presents the direct n-body algorithm in CUDA for GPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://www.bu.edu/pasi/files/2010/07/YokotaBarba_Gems.pdf&quot;&gt;other paper&lt;/a&gt; by Yokota and Barba has a good discussion on of the treecode and fast multipole algorithm also in the context of GPU-computing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your questions about the &lt;strong&gt;accuracy of n-body numerical simulations&lt;/strong&gt; are a bit more involved and there are so many important details that an answer can spawn several books. I think the best think to do is to give you a couple of book references. I suggest:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://catdir.loc.gov/catdir/samples/cam041/2003046028.pdf&quot;&gt;Gravitational N-body Simulations&lt;/a&gt; by Sverre J. Aarseth&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://books.google.co.uk/books/about/Computer_simulation_using_particles.html?id=nTOFkmnCQuIC&amp;amp;redir_esc=y&quot;&gt;Computer simulations using particles&lt;/a&gt; by Hockney and Eastwood. (Sorry no pdf version)&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-03-03T14:35:55.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1505" PostTypeId="1" AcceptedAnswerId="1590" CreationDate="2012-03-03T14:36:15.063" Score="10" ViewCount="1573" Body="&lt;p&gt;Suppose the following linear system is given&#xA;$$Lx=c,\tag1$$&#xA;where $L$ is the weighted Laplacian known to be positive $semi-$definite with a one dimensional null space spanned by $1_n=(1,\dots,1)\in\mathbb{R}^n$, and the translation variance of $x\in\mathbb{R}^{n}$, i.e., $x+a1_n$ does not change the function value (whose derivative is $(1)$). The only positive entries of $L$ are on its diagonal, which is a summation of the absolute values of the negative off-diagonal entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found in one highly cited academic work in its field that, although $L$ is $not~strictly$ diagonally dominant, methods such as Conjugate Gradient, Gauss-Seidl, Jacobi, could still be safely used to solve $(1)$. The rationale is that, because of translation invariance, one is safe to fix one point (eg. remove the first row and column of $L$ and the first entry from $c$ ), thus converting $L$ to a $strictly$ diagonally dominant matrix. Anyway, the original system is solved in the full form of $(1)$, with $L\in\mathbb{R}^{n\times n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this assumption correct, and, if so, what are the alternative rationale? I'm trying to understand how the convergence of the methods still hold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Jacobi method is convergent with $(1)$, what could one state about the spectral radius $\rho$ of the iteration matrix $D^{-1}(D-L)$, where $D$ is the diagonal matrix with entries of $L$ on its diagonal? Is $\rho(D^{-1}(D-L)\leq1$, thus different from the general convergence&#xA;guarantees for $\rho(D^{-1}(D-L))&amp;lt;1$? I'm asking this since the eigenvalues of the Laplacian matrix $D^{-1}L$ with ones on the diagonal &lt;em&gt;should&lt;/em&gt; be in range $[0, 2]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the original work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;......................................&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each iteration, we compute a new layout (x(t +1), y(t + 1)) by solving the following linear system:&#xA;$$&#xA;L · x(t + 1) = L(x(t),y(t)) · x(t) \\&#xA;L · y(t + 1) = L(x(t),y(t)) · y(t) \tag 8$$&#xA;Without loss of generality we can fix the location of one of the sensors (utilizing the translation&#xA;degree of freedom of the localized stress) and obtain a strictly diagonally dominant&#xA;matrix. Therefore, we can safely use Jacobi iteration for solving (8)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.......................................&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above, the notion of &quot;iteration&quot; is related to the underlying minimization procedure, and is not to be confused with Jacobi iteration. So, the system is solved by Jacobi (iteratively), and then the solution is bought to the right-hand side of (8), but now for another iteration of the underlying minimization. I hope this clarifies the matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I found &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1485/which-iterative-linear-solvers-converge-for-positive-semidefinite-matrices&quot;&gt;Which iterative linear solvers converge for positive semidefinite matrices?&lt;/a&gt; , but am looking for a more elaborate answer.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-03-26T08:24:45.627" Title="Safe application of iterative methods on diagonally dominant matrices" Tags="|linear-algebra|optimization|iterative-method|matrix|linear-solver|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1506" PostTypeId="1" AcceptedAnswerId="1509" CreationDate="2012-03-03T15:08:44.880" Score="6" ViewCount="2471" Body="&lt;p&gt;I am trying to solve the following test problem which is well-known in the community in different variants:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Place N = 15 points in the 3-dim. unit cube such that the minimal distance between them is maximal, e.g. like in the case of repellent but confined electrons.&#xA;Here are Matlab-like non-vectorized/vectorized forms of the function to be optimized (where we assume n = 3*N):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;---- non-vectorized -------------------------------------------------&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function d = balls(x)&#xA;  d = 2.0;&#xA;  for i = 1:14&#xA;    for j = (i+1):15&#xA;      s = (x(i)-x(j))^2 + (x(i+15)-x(j+15))^2 + (x(i+30)-x(j+30))^2;&#xA;      s = sqrt(s);&#xA;      if s &amp;lt; d, d = s; end&#xA;    end&#xA;  end&#xA;  d = -d;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;---- vectorized -----------------------------------------------------&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function d = balls(x)&#xA;X = reshape(x, [], 3);&#xA;d = -min(pdist(X));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The function is continuous, but &lt;em&gt;not&lt;/em&gt; smooth; gradient-based methods will not work. Multi-start and stochastic approaches have a problem, too, because with 45 dimensions the search space is already quite big. The boundary is also a problem, compared for example to the problem of placing electrons on a sphere where there is no boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will have many local minima, e.g. permuting ball indices or interchanging dimensions. I guess all these local minima have the same function value, like an energy level that will always end in a similar configuration (is that true?). I am only interested in this minimal value, so calculating one local minimum &lt;em&gt;exactly&lt;/em&gt; and reliably should be enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From applying Matlab' fmincon() with several restarts I know the minimum will be below -0.62... Still I would like to compute this value more accurately and with open source software only!&#xA;Please no hints to powerful commercial solvers.&lt;/p&gt;&#xA;" OwnerUserId="1084" LastActivityDate="2012-07-26T14:03:20.857" Title="Non-differentiable global optimization problem" Tags="|optimization|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1507" PostTypeId="2" ParentId="1283" CreationDate="2012-03-03T18:26:54.770" Score="4" Body="&lt;p&gt;I do not think there is any case where you want to solve on rank 0. Redundant solve is almost always better since, for small things, allreduce is as efficient as reduce, and redundant computation only has one instead of two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, whether to compute redundantly on all nodes, or on a subset, or redundant subsets depends on the hardware and system size. Thus, you should have a system that can do any of them. The PCREDUNDANT in PETSc can solve redundantly on all processes, some processes, or subsets of processes in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if the global problem is of size $10^6$ as you claim in the comments, it is large enough to benefit significantly from a parallel solve. Parallel assembly is very much the standard and recommended scenario.&lt;/p&gt;&#xA;" OwnerUserId="21" LastEditorUserId="119" LastEditDate="2012-03-03T20:52:56.557" LastActivityDate="2012-03-03T20:52:56.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1508" PostTypeId="2" ParentId="1503" CreationDate="2012-03-04T01:36:16.433" Score="4" Body="&lt;p&gt;A good book for you may be: Dieter Jungnickel, &quot;Graphs, Networks and Algorithms&quot;, Third Edition, Springer, 2008&lt;/p&gt;&#xA;" OwnerUserId="667" LastActivityDate="2012-03-04T01:36:16.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1509" PostTypeId="2" ParentId="1506" CreationDate="2012-03-04T03:02:24.367" Score="8" Body="&lt;h2&gt;Smooth reformulation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As Sid points out, there's no need to treat this problem as non-smooth, since you'd just be making it harder on yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume for the sake of notation that $\mathbf{x}_{1}, \ldots, \mathbf{x}_{15} \in [0,1]^{3} \subset \mathbb{R}^{3}$ are the coordinates of your 15 particles in the unit cube. A smooth formulation, as Sid suggests, presented in standard form (for nonlinear programming), would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{alignat}{1}&#xD;&#xA;&amp;amp;\min_{\mathbf{x}_{1}, \ldots, \mathbf{x}_{15} \in [0,1]^{3}} -E \\&#xD;&#xA;\mathrm{s.t.} &amp;amp; \quad E - \|\mathbf{x}_{i} - \mathbf{x}_{j}\|^{2} \leq 0, \,\, i, j = 1, \ldots, 15, \,\, i \neq j&#xD;&#xA;\end{alignat}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $E$ is a proxy for the minimum distance, which I'm assuming is related to minimizing some sort of energy. There might be a way to reformulate this problem as an equivalent convex problem, but I don't think there is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This formulation probably isn't convex, because the left-hand sides of the nonlinear constraints aren't convex, so you'll need to use a nonconvex nonlinear programming solver to be assured of a global optimal solution (unless you can prove convexity of the feasible set, but I doubt that). Deterministic global solvers that will work for nonconvex problems include (but aren't limited to):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://archimedes.cheme.cmu.edu/baron/baron.html&quot;&gt;BARON&lt;/a&gt; (which is commercial, but you can submit jobs for free via the &lt;a href=&quot;http://neos-server.org/neos/&quot;&gt;NEOS optimization server&lt;/a&gt; run by University of Wisconsin-Madison)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.lindo.com/index.php?option=com_content&amp;amp;view=article&amp;amp;id=1&amp;amp;Itemid=9&quot;&gt;LINDOGlobal&lt;/a&gt; (also commerical, also available through the NEOS optimization server)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.coin-or.org/projects/Couenne.xml&quot;&gt;Couenne&lt;/a&gt; (open-source, part of the &lt;a href=&quot;http://www.coin-or.org/index.html&quot;&gt;COIN-OR&lt;/a&gt; suite of open-source solvers)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.coin-or.org/projects/Bonmin.xml&quot;&gt;Bonmin&lt;/a&gt; (also part of COIN-OR)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.coin-or.org/projects/LaGO.xml&quot;&gt;LaGO&lt;/a&gt; (again, part of COIN-OR)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sites.google.com/site/ylebbah/icos&quot;&gt;icos&lt;/a&gt; (available as open-source, or through NEOS) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's important to note that one solver may work on your problem when others won't; BARON is generally considered the best, but it's fallible, and there are cases where, for example, Couenne will solve a problem to (epsilon) global optimality, but BARON won't (and vice versa).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Solving nonsmooth problems&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Let's suppose for the sake of argument that you (like Hans) want to solve a non-smooth nonlinear programming problem. This type of problem isn't my area of expertise, but I know of a couple references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most famous person in the field (who, as far as I can tell, developed the most important parts of the theory early on) is Frank H. Clarke. The gist of non-smooth optimization seems to be: replace gradients with Clarke's generalized gradients. Using Clarke's generalized gradients, you're supposed to be able to formulate a non-smooth analogue of Newton's method, as well as algorithms for optimization. His textbook on the theory (&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898712564&quot;&gt;&lt;em&gt;Optimization and Nonsmooth Analysis&lt;/em&gt; by Frank H. Clarke&lt;/a&gt;; the link goes to Amazon) is considered a classic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of software, the best links I can find are to &lt;a href=&quot;http://napsu.karmitsa.fi/nsosoftware/&quot;&gt;Napsu Karmitsa's home page&lt;/a&gt;; she's developed a couple non-smooth optimization solvers, and she links to other non-smooth optimization solvers. The methods I've heard of most often are called bundle methods, and should be deterministic. (I favor deterministic methods over stochastic methods.) More links to non-smooth codes can be found &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/software_l.html&quot;&gt;here&lt;/a&gt;; your mileage may vary, because like I said, I don't work with these methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do know that just because a method is developed for non-smooth problems does not mean it will work for non-smooth, non-convex problems, so you will need to make sure that the solver you choose can handle both non-smoothness and non-convexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, as Hans points out in the comments, non-smooth formulations regularly appear in science and engineering. However, my first instinct as someone in the optimization field is to try and find an equivalent smooth reformulation because methods for solving smooth problems are generally &lt;em&gt;much&lt;/em&gt; faster than methods for solving non-smooth methods (a labmate uses non-smooth solvers, and has made this observation). If you can reformulate the problem as a smooth optimization problem, it generally behooves you to do so.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-03-04T17:21:28.600" LastActivityDate="2012-03-04T17:21:28.600" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1511" PostTypeId="2" ParentId="1503" CreationDate="2012-03-04T13:39:23.573" Score="4" Body="&lt;p&gt;If by &amp;quot;good introduction&amp;quot; you mean &amp;quot;just the basics&amp;quot;, then you might want to look at the first chapter of &amp;quot;Graphs, Networks and Algorithms&amp;quot;. Lucky you, springer gives the first chapter as a free preview: &lt;a href=&quot;https://link.springer.com/book/10.1007/978-3-642-32278-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;NB: I wanted to add this as a comment to @Jean-VictorCôté's answer but it seems I'm not reputable enough...&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorDisplayName="user43608" LastEditDate="2022-07-22T06:46:07.120" LastActivityDate="2022-07-22T06:46:07.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1512" PostTypeId="5" CreationDate="2012-03-04T14:28:39.230" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-04T14:28:39.230" LastActivityDate="2012-03-04T14:28:39.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1513" PostTypeId="4" CreationDate="2012-03-04T14:28:39.230" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-04T14:28:39.230" LastActivityDate="2012-03-04T14:28:39.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1514" PostTypeId="1" CreationDate="2012-03-04T15:57:56.437" Score="5" ViewCount="471" Body="&lt;p&gt;Recently, I wrote an &lt;a href=&quot;http://www.math.utep.edu/Faculty/pmdelgado2/courses/adv_algorithms/Project2-ConvexHulls.pdf&quot; rel=&quot;nofollow&quot;&gt;algorithm&lt;/a&gt; to obtain a delaunay triangulation of a random point set in $I=[-10,10]$x$[-10,10] \subset R^2$ by projecting these points onto the 3 dimensional paraboloid $z=x^2+y^2$, computing the convex hull, and then projecting the lower hull back onto $R^2$.  I have verified that my convex hull algorithm is producing the correct result.  Thus, I'm sure the problem is with my approach to obtaining the lower hull.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My lower hull extraction algorithm is similar to the &lt;a href=&quot;http://www.cse.unsw.edu.au/~lambert/java/3d/hull.html&quot; rel=&quot;nofollow&quot;&gt;incremental construction algorithm&lt;/a&gt; for generating convex hulls.  That is, if we chose a point at negative infinity and shined a light from this point up toward the hull, we should be able to see the entire lower hull and be able to extract it completely.  Computationally, of course, we cannot really use negative infinity as a number.  But there must be some optimal point under the hull where the entire lower hull should be visible. (note: visibility here is determined by the signed orientation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my problem, since the point set is close to the origin (x,y)=(0,0), I chose to look for an optimal viewpoint along the negative z-axis.  I figured that since the points on the lower hull are on the paraboloid $z=x^2 + y^2$, we can exploit the fact that the paraboloid is monotonically increasing as the (x,y) input coordinate moves in the radial direction away from the origin (0,0).  Thus, the point in the point set whose x,y coordinate is furthest from the origin (0,0) will not only have the highest z-coordinate on the paraboloid, but also have the steepest tangent plane.  Furthermore, where this tangent plane intersects the negative z-axis should be the optimal point which can see the entire lower hull.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I experimented with this algorithm, it doesn't seem to work perfectly.  That is, it is able to extract almost all the facets of the lower hull.  It only seems to run into problems extracting facets whose projected delaunay triangles (onto $R^2$) are extremely thin and located on the 2D hull (that is, the outer boundary triangles).  In these cases, i have had to lower my initial viewpoint by as much as $10^5$ or more, depending on the thinness of the triangle on the outer boundary of the delaunay triangulation in 2D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there are other approaches to obtaining the lower hull, but I'm really more interested in understanding where the flaw is in my original algorithm.  Since the paraboloid is monotonically concave up, shouldn't the optimal viewpoint be located at a point on the z-axis where the tangent plane from $(x^*,y^*,z^*)$ hits the z-axis, where $(x^*,y^*,z^*)$ is the point projected from the given point set (in $R^2$) onto the paraboloid (in $R^3$ whose (x,y) coordinate is furthest from the origin?  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-07T20:23:33.317" LastActivityDate="2012-03-07T20:23:33.317" Title="Why is my lower convex hull extraction algorithm not working?" Tags="|optimization|computational-geometry|delaunay-triangulation|convex-hull|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1515" PostTypeId="5" CreationDate="2012-03-04T16:07:51.713" Score="0" Body="&lt;p&gt;Consider this analogy:  Imagine a set of nails nailed to a wooden board.  If we stretch a rubber band around the set of nails, and let it contract, the rubber band will form a convex shape that entirely encloses the entire set of nails.  The outer boundary, where the rubber band fits tightly around the nails, is the convex hull of the set of nails.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2D, the convex hull is a set of lines which forms the smallest enclosing convex polygon around the point set.  In 3D, it is a set of faces forming the smallest enclosing convex polyhedron The notion of a convex hull can be extended into arbitrary dimensions as well, where the boundaries are referred to as facets, and the region is referred to as an enclosing convex simplex.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-04T17:04:08.530" LastActivityDate="2012-03-04T17:04:08.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1516" PostTypeId="4" CreationDate="2012-03-04T16:07:51.713" Score="0" Body="The convex hull of a point set is the outer boundary of the smallest convex set that encloses the point set entirely." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-04T17:04:04.457" LastActivityDate="2012-03-04T17:04:04.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1517" PostTypeId="1" AcceptedAnswerId="1534" CreationDate="2012-03-04T16:17:04.323" Score="16" ViewCount="21627" Body="&lt;p&gt;Suppose I'm using a linear congruential pseudo-random number generator (PRNG).  Given a seed $x_0$, the multiplying factor (a), the shift factor (c) and the modulus factor (m), how can I determine the period of my PRNG?  Do I determine it by experimentation / pattern detection algorithms, or is there a direct formula for calculating its period?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although my question is specifically about the linear congruential method, I am open to knowing more about how periods are calculated in practice for other PRNG's as well.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2019-05-10T05:57:42.363" Title="How can I determine the period of my pseudo-random number generator?" Tags="|random-number-generation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1518" PostTypeId="5" CreationDate="2012-03-04T17:31:30.900" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-04T17:31:30.900" LastActivityDate="2012-03-04T17:31:30.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1519" PostTypeId="4" CreationDate="2012-03-04T17:31:30.900" Score="0" Body="Referring the to Quasi-Minimum-Residual method for solving linear systems of equations" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-04T17:32:38.370" LastActivityDate="2012-03-04T17:32:38.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1520" PostTypeId="5" CreationDate="2012-03-04T17:40:02.870" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-04T17:40:02.870" LastActivityDate="2012-03-04T17:40:02.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1521" PostTypeId="4" CreationDate="2012-03-04T17:40:02.870" Score="0" Body="Referring to questions involving the Alternating-Direction-Implicit finite difference methods for solving partial differential equations" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-04T18:40:42.480" LastActivityDate="2012-03-04T18:40:42.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1522" PostTypeId="1" AcceptedAnswerId="1529" CreationDate="2012-03-04T17:55:31.680" Score="9" ViewCount="164" Body="&lt;p&gt;Let $f, g \in \mathbb{R}[x]$ and $\deg f &amp;gt; \deg g$. I am looking for asymptotically fast and &lt;em&gt;numerically stable&lt;/em&gt; algorithms for computing $f \bmod g$. In the applications intended, both $f, g$ are dense polynomials with double precision floating-point coefficients. But, for now, I'm more interested in the algorithms rather than the implementation. References for algorithms for computing GCD of numerical polynomials are also appreciated.&lt;/p&gt;&#xA;" OwnerDisplayName="user182" LastActivityDate="2012-03-05T11:26:08.290" Title="Numerically stable algorithms for computing remainder of polynomials" Tags="|algorithms|reference-request|polynomials|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1523" PostTypeId="2" ParentId="1400" CreationDate="2012-03-04T18:18:01.667" Score="1" Body="&lt;p&gt;I know that you are using Runge-Kutta Methods to solve your ODE, but if you want to reuse old calculated values of your f(x,y), you may want to consider multistep methods, like the Adams-Bashforth or Adams-Moulton methods.  Of course, the disadvantage to these methods is that you cannot use adaptive time-stepping very easily.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-03-04T18:18:01.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1524" PostTypeId="5" CreationDate="2012-03-04T22:45:58.103" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-04T22:45:58.103" LastActivityDate="2012-03-04T22:45:58.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1525" PostTypeId="4" CreationDate="2012-03-04T22:45:58.103" Score="0" Body="For questions regarding the Minimum Residual iterative method for solving linear systems of equations." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-05T07:54:25.117" LastActivityDate="2012-03-05T07:54:25.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1526" PostTypeId="2" ParentId="1488" CreationDate="2012-03-05T00:49:26.587" Score="6" Body="&lt;p&gt;From a computer science point of view, there is no obvious way to &lt;em&gt;automatically infer&lt;/em&gt; a program time/resource complexity. This is a hard problem in the realm of &lt;a href=&quot;http://en.wikipedia.org/wiki/Static_program_analysis&quot;&gt;static analysis&lt;/a&gt;, and is mainly theoretically difficult because of the undecidability of &lt;a href=&quot;http://en.wikipedia.org/wiki/Halting_problem&quot;&gt;the halting problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do the complexity analysis by hand though if your code is of manageable size. Inspect all your iterations and subroutines bottom-up, and calculate the complexity just like &lt;a href=&quot;http://en.wikipedia.org/wiki/Analysis_of_algorithms&quot;&gt;algorithm analysis&lt;/a&gt; books would do with psuedocode. But you have to make sure that the runtime semantics of your program are preserved by the compiler/optimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a practical point of view, you can use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Profiling_%28computer_programming%29&quot;&gt;profiler&lt;/a&gt; (e.g. &lt;a href=&quot;http://sourceware.org/binutils/docs/gprof/&quot;&gt;gprof&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Valgrind&quot;&gt;valgrind&lt;/a&gt;) to diagnose your code performance while/after running. It can tell you about the pressure points in your code, and where your program spends most of its runtime. You can start from this info and diagnose your implementation.&lt;/p&gt;&#xA;" OwnerDisplayName="user182" LastActivityDate="2012-03-05T00:49:26.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1527" PostTypeId="1" AcceptedAnswerId="1828" CreationDate="2012-03-05T09:07:38.010" Score="8" ViewCount="151" Body="&lt;p&gt;Suppose you solve a linear system $Au = f$ by an iterative method, e.g. conjugate gradients or Richardson iteration. Then you try to solve a linear system that is slightly perturbed in the matrix and the right-hand side, say, $\tilde A \tilde u = \tilde f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it make sense to use the old solution as starting value $\tilde u_0 = u$ for the iterative method? &quot;Makes sense&quot; means that there is a relyable gain in the running time of the iterative method. I wonder whether this leads to an improvement in general, such that it can be regarded as advised practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An application that I have in mind comes from adaptive finite elements. If we have computed a solution $u$ on a coarse grid, and want to find a solution $\tilde u$ on a finer grid (which might have been generated based by an adaptive method), the starting value for any iterative algorithm can be the prolongation of $u$ onto the finer grid. Similarly, the Newton-method or Picard-iteration, which is involved in the solution of nonlinear problems, could be &quot;boosted&quot; that way, if it makes any sense at all.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="123" LastEditDate="2012-03-05T10:48:30.927" LastActivityDate="2012-04-01T12:27:31.787" Title="Initial guesses for perturbed linear systems" Tags="|linear-algebra|finite-element|linear-solver|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1528" PostTypeId="1" AcceptedAnswerId="2812" CreationDate="2012-03-05T10:41:38.917" Score="9" ViewCount="3991" Body="&lt;p&gt;It's shown (Yousef Saad, &lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;id=Uoe7xBOhS5AC&amp;amp;pg=PA260&quot; rel=&quot;nofollow&quot;&gt;Iterative methods for sparse linear systems&lt;/a&gt;, p. 260) that $cond(A'A) \approx cond(A)^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this true for $AA'$ as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case $A$ is $N\times M$ with $N \ll M$, I observe that $cond(A'A) \gg cond(AA')$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that mean formulation in terms of $AA'$ is preferable in this case?&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="127" LastEditDate="2012-07-22T15:45:39.710" LastActivityDate="2012-07-25T00:23:30.283" Title="Condition number of A'A and AA' formulations" Tags="|linear-algebra|condition-number|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1529" PostTypeId="2" ParentId="1522" CreationDate="2012-03-05T11:26:08.290" Score="3" Body="&lt;p&gt;Check the book by Dario Bini &amp;amp; Victor Pan: &lt;a href=&quot;http://books.google.be/books/about/Polynomial_and_matrix_computations.html?id=WbFxj-HvaGYC&quot; rel=&quot;nofollow&quot;&gt;&quot;Polynomial and Matrix computations, Volume 1: Fundamental algorithms&quot;, ISBN 0-8176-3786-9,  Birkhäuser, 1994&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-03-05T11:26:08.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1530" PostTypeId="2" ParentId="1528" CreationDate="2012-03-05T12:42:22.093" Score="9" Body="&lt;p&gt;Well, let’s look at why $A^TA$ has approximately the squared condition number of $A$. Using the SVD decomposition of $A=USV^T$, with $U \in \mathbb{R}^{N \times N}$, $S \in \mathbb{R}^{N \times M}$, $V \in \mathbb{R}^{M \times M}$, we can express $A^T A$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A^T A=(USV^T)^T USV^T=VS^T U^T U S V^T=V S^T S V^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which we arrive at by noting that $U$ is orthonormal, such that $U^T U=I$. Further we note that $S$ is a diagonal matrix, such that the final decomposition of $A^TA$ can be expressed as $V S^2 V^T$, with $S^2$ meaning $S^T S$, yielding a diagonal matrix with the first N singular values from $S$ squared in the diagonal. This means that since the condition number is the ratio of the first and the last singular value, $cond(A)=\frac{s_1}{s_N}$ for $A \in \mathbb{R}^{N \times M}$,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$cond(A^T A)=\frac{s_1^2}{s_M^2}=(\frac{s_1}{s_M})^2=cond(A)^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can perform the same exercise with $AA^T$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$AA^T=USV^T (USV^T)^T=USV^T V S^T U^T=U S^2 U^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which means that we get the result $cond(AA^T)=\frac{s_1^2}{s_N^2}$, since $S^2$ here means $SS^T$, a subtle difference from the notation above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But note that subtle difference! For $A^TA$, the condition number has the M'th singular value in the denominator, while $AA^T$ has the N'th singular value. This explains why you are seeing significant differences in the condition number — $AA^T$ will indeed be “better conditioned” than $A^TA$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, David Ketcheson was correct — you are comparing condition numbers between two vastly different matrices. In particular, what you can accomplish with $A^TA$ will not be the same as what you can accomplish with $AA^T$.&lt;/p&gt;&#xA;" OwnerUserId="370" LastEditorUserId="370" LastEditDate="2012-03-05T16:03:59.307" LastActivityDate="2012-03-05T16:03:59.307" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1531" PostTypeId="1" AcceptedAnswerId="1541" CreationDate="2012-03-05T13:20:23.717" Score="15" ViewCount="3777" Body="&lt;p&gt;The Remez algorithm is a well-known iterative routine to approximate a function by a polynomial in the minimax norm. But, as Nick Trefethen [1] says about it: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Most of these [implementations] go back many years and in fact, most of them do not solve the general best approximation problem as posed above but variants involving discrete variables or digital filtering. One can find a few other computer programs in circulation, but overall, it seems that there is no widely-used program at present for computing best approximations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One can compute the minimax solution also by applying least-squares or convex optimization, for example using Matlab and the free CVX toolbox applied to the Runge function on [-1, 1]:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m = 101; n = 11;            % 101 points, polynomial of degree 10&#xA;xi = linspace(-1, 1, m);    % equidistant points in [-1, 1]&#xA;ri = 1 ./ (1+(5*xi).^2);    % Runge function&#xA;&#xA;tic                         % p is the polynomial of degree (n-1)&#xA;cvx_begin                   % minimize the distance in all points&#xA;    variable p(n);&#xA;    minimize( max(abs(polyval(p, xi) - ri)) );&#xA;cvx_end&#xA;toc                         % 0.17 sec for Matlab, CVX and SeDuMi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The approximation with Chebyshev polynomials has a minimax norm of &lt;code&gt;0.1090&lt;/code&gt; while this approach here reaches a minimum of &lt;code&gt;0.0654&lt;/code&gt;, the same value that gets computed with the Remez algorithm in the Matlab &lt;code&gt;chebfun&lt;/code&gt; toolbox.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any advantage in applying the more complicated Remez algorithm if you can compute the minimax solution faster and more accurate with an optimization solver? Are there any reports/articles comparing these two approaches on some difficult problems or test cases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;br&gt;&#xA;[1] R. Pachon and L. N. Trefethen. BIT Numerical Mathematics (2008) Vol. 46.&lt;/p&gt;&#xA;" OwnerUserId="1084" LastActivityDate="2012-03-07T10:41:00.037" Title="The Remez Algorithm" Tags="|optimization|approximation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1532" PostTypeId="2" ParentId="1505" CreationDate="2012-03-05T13:53:31.087" Score="5" Body="&lt;p&gt;Krylov methods never explicitly use the dimensionality of the space they iterate in, therefore you can run them on singular systems so long as you keep the iterates in the non-null subspace. This is normally done by projecting out the null space at each iteration. There are two things that can go wrong, the first is much more common than the second.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The preconditioner is unstable when applied to the singular operator. Direct solvers and incomplete factorization may have this property. As a practical matter, we just choose different preconditioners, but there are more principled ways to design preconditioners for singular systems, e.g. &lt;a href=&quot;http://dx.doi.org/10.1007/s10543-009-0247-7&quot; rel=&quot;nofollow&quot;&gt;Zhang (2010)&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;At some iteration, $x$ is in the non-null subspace, but $A x$ lives entirely in the null space. This is only possible with nonsymmetric matrices. Unmodified GMRES breaks down in this scenario, but see &lt;a href=&quot;http://www.math.kent.edu/~reichel/publications/bfgmres.pdf&quot; rel=&quot;nofollow&quot;&gt;Reichel and Ye (2005)&lt;/a&gt; for breakdown free variants.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To solve singular systems using PETSc, see &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSetNullSpace.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;KSPSetNullSpace()&lt;/code&gt;&lt;/a&gt;. Most methods and preconditioners can solve singular systems. In practice, the small null space for PDEs with Neumann boundary conditions are almost never a problem as long as you inform the Krylov solver of the null space and choose a reasonable preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the comments, it sounds like you are specifically interested in Jacobi. (Why? Jacobi is useful as a multigrid smoother, but there are much better methods to use as solvers.) Jacobi applied to $A x = b$ does not converge when the vector $b$ has a component in the null space of $A$, however, the part of the solution orthogonal to the null space does converge, so if you project the null space out of each iterate, it converges. Alternatively, if you choose a consistent $b$ and initial guess, the iterates (in exact arithmetic) do not accumulate components in the null space.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-03-06T16:00:37.350" LastActivityDate="2012-03-06T16:00:37.350" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1533" PostTypeId="2" ParentId="1527" CreationDate="2012-03-05T14:18:49.437" Score="4" Body="&lt;p&gt;I think it really depends on the condition number of the matrix A.  If it has a large condition number, then perturbing the system  ever-so-slightly may produce a radically different solution.  For Adaptive FEM, it depends on what you expect the behavior of the system to be (and, obviously, the quality of the mesh itself).  If you expect fairly smooth transition from the coarse to the fine grid, then we should expect the perturbed system to have a solution fairly close to the unperturbed system.  If you can expect sudden dramatic changes, there is no real guarantee of the closeness of the perturbed and unperturbed systems.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-03-05T14:18:49.437" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1534" PostTypeId="2" ParentId="1517" CreationDate="2012-03-05T14:29:58.747" Score="14" Body="&lt;p&gt;If you constrain yourself to &lt;a href=&quot;http://en.wikipedia.org/wiki/Full_cycle&quot; rel=&quot;nofollow noreferrer&quot;&gt;full cycle&lt;/a&gt; &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_congruential_generator&quot; rel=&quot;nofollow noreferrer&quot;&gt;LCG&lt;/a&gt; &lt;a href=&quot;http://en.wikipedia.org/wiki/Pseudorandom_number_generator&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRNG&lt;/a&gt;s then the answer is easy, by definition it's simply &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the period of a non full cycle LCG PRNG for a given seed you just need to count the number of iterations of the PRNG until it generates the seed value once more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the referenced &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_congruential_generator#Period_length&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia page&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;Period length&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Periodic_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;period&lt;/a&gt; of a general LCG is at most &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, and for some choices of a much less than that. Provided that &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is nonzero, the LCG will have a full period for all seed values &lt;a href=&quot;http://en.wikipedia.org/wiki/If_and_only_if&quot; rel=&quot;nofollow noreferrer&quot;&gt;if and only if&lt;/a&gt;:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; are &lt;a href=&quot;http://en.wikipedia.org/wiki/Relatively_prime&quot; rel=&quot;nofollow noreferrer&quot;&gt;relatively prime&lt;/a&gt;,&lt;/li&gt;&#xA;  &lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$a - 1$&lt;/span&gt; is divisible by all &lt;a href=&quot;http://en.wikipedia.org/wiki/Prime_factor&quot; rel=&quot;nofollow noreferrer&quot;&gt;prime factors&lt;/a&gt; of &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$a - 1$&lt;/span&gt; is a multiple of 4 if &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is a multiple of 4.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;While LCGs are capable of producing decent &lt;a href=&quot;http://en.wikipedia.org/wiki/Pseudorandom_numbers&quot; rel=&quot;nofollow noreferrer&quot;&gt;pseudorandom numbers&lt;/a&gt;, this is extremely sensitive to the choice of the parameters &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Historically, poor choices had led to ineffective implementations of LCGs. A particularly illustrative example of this is &lt;a href=&quot;http://en.wikipedia.org/wiki/RANDU&quot; rel=&quot;nofollow noreferrer&quot;&gt;RANDU&lt;/a&gt; which was widely used in the early 1970s and lead to many results which are currently being questioned because of the use of this poor LCG.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h1&gt;Why you want to use a full cycle generator&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;If you don't constrain yourself to full cycle LCG PRNGs then you are taking a &lt;strong&gt;huge risk&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't &lt;em&gt;know&lt;/em&gt; that a given LCG is full cycle then you could end up with a generator with an arbitrary number of mutually distinct sequences, some of which could be &lt;em&gt;embarrassingly&lt;/em&gt; small and have appalling randomness, possibly even worse than the infamous &lt;a href=&quot;https://csgillespie.wordpress.com/2016/02/16/randu-the-case-of-the-bad-rng/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RANDU&lt;/a&gt; generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You really don't want to have to check every possible seed value to make sure that it generates a sequence which is long enough for your application.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Further reading&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;For an excellent primer on pseudo random number generators, I would strongly recommend that you read the &lt;a href=&quot;http://www.nr.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Recipes&lt;/a&gt; chapter on Random Numbers.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="31698" LastEditDate="2019-05-10T05:57:42.363" LastActivityDate="2019-05-10T05:57:42.363" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="1535" PostTypeId="1" AcceptedAnswerId="1536" CreationDate="2012-03-05T15:16:31.390" Score="9" ViewCount="4807" Body="&lt;p&gt;In my project I have to solve a couple of tridiagonal matrices at every time step, so it is crucial to have a good solver for those. I did my own implementation, just the classical way to do it described on Wikipedia. I then tried using Lapack instead, and to my surprise it was slower! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, inside Lapack it seems like it does the solving by LU factorization and I wonder why, isn't it more complex than it could be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, I found an algorithm in the &quot;Numerical Recipes&quot; book from nr.com which recursively divides the system into smaller tridiagonal problems. It looked promising. Are there any other goodies out there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: the problem size is about 1000x1000. I used GotoBLAS, it gives you a Lapack 3.1.1 library also. The problem is not symmetric. I used the Lapack routine for general tridiagonal matrices.&lt;/p&gt;&#xA;" OwnerUserId="1088" LastEditorUserId="22284" LastEditDate="2019-03-27T14:10:39.370" LastActivityDate="2019-03-27T14:10:39.370" Title="How does LAPACK solve tridiagonal systems and why?" Tags="|banded-matrix|lapack|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="1536" PostTypeId="2" ParentId="1535" CreationDate="2012-03-05T16:26:35.780" Score="7" Body="&lt;p&gt;You are using a reference implementation that does partial pivoting. Tridiagonal solves do very little work and do not call into the BLAS. It is likely slower than your code because it does partial pivoting. The &lt;a href=&quot;http://www.netlib.org/lapack/double/dgtsv.f&quot;&gt;source code for dgtsv&lt;/a&gt; is straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you will solve with the same matrix multiple times, you may want to store the factors by using &lt;a href=&quot;http://www.netlib.org/lapack/double/dgttrf.f&quot;&gt;dgttrf&lt;/a&gt; and &lt;a href=&quot;http://www.netlib.org/lapack/double/dgttrs.f&quot;&gt;dgttrs&lt;/a&gt;. It is possible that the implementations in an optimized LAPACK such as MKL, ACML, or ESSL will be more performant.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-05T16:26:35.780" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1537" PostTypeId="1" AcceptedAnswerId="1539" CreationDate="2012-03-05T18:19:10.100" Score="5" ViewCount="11655" Body="&lt;p&gt;I was studying Big-Oh notation, and there is apparently a difference between $O(n)$ and $o(n)$. What is it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think $f(n)$ is $o(n)$ means that $$\lim_{n \to \infty} \frac{f(n)}{g(n)} = 0$$ but what does this mean?&lt;/p&gt;&#xA;" OwnerUserId="1091" LastEditorUserId="81" LastEditDate="2012-03-05T19:13:01.363" LastActivityDate="2012-06-30T16:24:33.833" Title="What is the difference between O(n) and o(n)?" Tags="|algorithms|" AnswerCount="2" CommentCount="4" ClosedDate="2012-03-05T21:14:52.037" ContentLicense="CC BY-SA 3.0" />
  <row Id="1538" PostTypeId="2" ParentId="1537" CreationDate="2012-03-05T19:01:03.983" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Big_O_notation#Family_of_Bachmann.E2.80.93Landau_notations&quot; rel=&quot;nofollow&quot;&gt;Wikipedia article on Big O notation&lt;/a&gt;, particularly the section on Bachmann-Landau Notation, is a well-written description of the different notations used in algorithmic complexity analysis.  &lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-03-05T19:01:03.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1539" PostTypeId="2" ParentId="1537" CreationDate="2012-03-05T19:07:58.310" Score="5" Body="&lt;p&gt;Think of it like the difference between &amp;lt;= versus &amp;lt;.  If we say that a &amp;lt;= b, it is possible that a can equal b.  If a &amp;lt; b, then a &amp;amp; b can never be equal in magnitude, and a is always strictly smaller than b.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Big Oh notation, if we say $f(n) = O(g(n))$, then the function g(n) forms an asymptotic bound for f(n), but f(n) can come within a constant factor of g(n) in the limit (we sometimes call this an asymptotically tight bound).  But if $f(n) = o(g(n))$, then g(n) always grows faster than f(n) in the limit.  Thus, f(n) can't come within a constant factor away from g(n).  In little oh notation, g(n) is MUCH LARGER (than a constant factor) away from f(n) in the limit.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-06-30T16:24:33.833" LastActivityDate="2012-06-30T16:24:33.833" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1540" PostTypeId="2" ParentId="1528" CreationDate="2012-03-05T21:16:14.720" Score="1" Body="&lt;p&gt;In exact arithmetic cond(A^2)=cond(A'A)=cond(AA'), see eg. Golub and van Loan, 3rd ed, p70. This is not true in floating point arithmetic if A is nearly rank deficient. The best advise is to follow the above book recipes when solving least square problems, the safest being SVD approach, p257. Use \varepsilon-rank instead when computing SVD, where \varepsilon is the resolution of your matrix data.&lt;/p&gt;&#xA;" OwnerUserId="897" LastActivityDate="2012-03-05T21:16:14.720" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1541" PostTypeId="2" ParentId="1531" CreationDate="2012-03-05T21:19:56.093" Score="6" Body="&lt;p&gt;The &quot;right&quot; answer strongly depends on what you need your approximant for. Do you really need the best approximation for some error bound? Or just a good approximation? Or just a good approximation in the minmax sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nick Trefethen recently gave a &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/mythspaper.pdf&quot; rel=&quot;noreferrer&quot;&gt;nice example&lt;/a&gt; where Remez approximation is a bad idea since it minimizes the maximum error irrespective of the average error over the entire interval, which may not be what you want. Of course, the maximum error may be large, but this is bounded for smooth functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following the discussion in the comments below, I downloaded the CVX Toolbox and did a direct comparison with the Chebfun Remez algorithm (disclaimer: I am part of the Chebfun development team):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Do the convex optimization bit.&#xA;m = 101; n = 11;            % 101 points, polynomial of degree 10&#xA;xi = linspace(-1, 1, m);    % equidistant points in [-1, 1]&#xA;ri = 1 ./ (1+(5*xi).^2);    % Runge function&#xA;&#xA;tic                         % p is the polynomial of degree (n-1)&#xA;cvx_begin                   % minimize the distance in all points&#xA;    variable p(n);&#xA;    minimize( max(abs(polyval(p, xi) - ri)) );&#xA;cvx_end&#xA;toc_or = toc                % 0.17 sec for Matlab, CVX and SeDuMi&#xA;&#xA;% Extract a Chebfun from the result&#xA;x = chebfun( [-1,1] );&#xA;A = [ chebfun(1) , x ];&#xA;for k=3:n, A(:,k) = A(:,k-1).*x; end&#xA;or = A * flipud(p)&#xA;&#xA;% Make a chebfun of Runge's function&#xA;f = chebfun( @(x) 1 ./ ( 1 + 25*x.^2 ) )&#xA;&#xA;% Get the best approximation using Remez&#xA;tic, cr = remez( f , 10 ); toc_cr = toc&#xA;&#xA;% Get the maximum error in each case&#xA;fprintf( 'maximum error of convex optimization: %e (%f s)\n' , norm( f - or , inf ) , toc_or );&#xA;fprintf( 'maximum error of chebfun remez: %e (%f s)\n' , norm( f - cr , inf ) , toc_cr );&#xA;&#xA;% Plot the two error curves&#xA;plot( [ f - cr , f - or ] );&#xA;legend( 'chebfun remez' , 'convex optimization' );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After a lot of output, I get, on my laptop with Matlab 2012a, CVX version 1.22 and Chebfun's latest SVN Snapshot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;maximum error of convex optimization: 6.665479e-02 (0.138933 s)&#xA;maximum error of chebfun remez: 6.592293e-02 (0.309443 s)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the Chebfun &lt;code&gt;f&lt;/code&gt; used to measure the error is accurate to 15 digits. So Chebfun's Remez takes twice as long, but gets a smaller global error. It should be pointed out that CVX uses compiled code for the optimization whereas Chebfun is 100% native Matlab. The minimum error of &lt;code&gt;0.00654&lt;/code&gt; is the minimum error 'on the grid', off that grid, the error can be up to &lt;code&gt;0.00659&lt;/code&gt;. Increasing the grid size to &lt;code&gt;m = 1001&lt;/code&gt; I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;maximum error of convex optimization: 6.594361e-02 (0.272887 s)&#xA;maximum error of chebfun remez: 6.592293e-02 (0.319717 s)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i.e. almost the same speed, but the discrete optimization is still worse as of the fourth decimal digit. Finally, ncreasing the grid size further to &lt;code&gt;m = 10001&lt;/code&gt; I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;maximum error of convex optimization: 6.592300e-02 (5.177657 s)&#xA;maximum error of chebfun remez: 6.592293e-02 (0.312316 s)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i.e. the discrete optimization is now more than ten times slower and is still worse as of the sixth digit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottom line is that Remez will get you the globally optimal result. While the discrete analog may be fast on small grids, it will not give a correct result.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-03-07T10:41:00.037" LastActivityDate="2012-03-07T10:41:00.037" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1542" PostTypeId="1" AcceptedAnswerId="1562" CreationDate="2012-03-06T00:14:57.000" Score="10" ViewCount="884" Body="&lt;p&gt;I have a problem similar in formulation to this post, with a few notable differences:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/923/what-simple-methods-are-there-for-adaptively-sampling-a-2d-function&quot;&gt;What simple methods are there for adaptively sampling a 2D function?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Like in that post:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have a &lt;span class=&quot;math-container&quot;&gt;$f(x,y)$&lt;/span&gt; and evaluation of this function is somewhat expensive to compute&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Unlike in that post:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;I am interested not in the value of the function accurately everywhere, but only in finding a single isocontour of the function.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I can make significant assertions about the autocorrelation of the function, and consequently the scale of smoothness.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Is there an intelligent way to step along/sample this function and find this contour?&lt;/p&gt;&#xA;&lt;h2&gt;More Information&lt;/h2&gt;&#xA;&lt;p&gt;The function is the computation of &lt;a href=&quot;http://murphylab.web.cmu.edu/publications/boland/boland_node26.html&quot; rel=&quot;noreferrer&quot;&gt;Haralick Features&lt;/a&gt; over &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; pixles surrounding the point, and soft classification by some sort of classifier/regressor.  The output of this is a floating point number which indicates which texture/material the point belongs to.  The scaling of this number can be estimated class probablities (SoftSVM or statistical methods etc) or something really simple like the output of a linear/logistic regression.  Classification/regression is accurate and cheap compared to time taken for feature extraction from the image.&lt;/p&gt;&#xA;&lt;p&gt;Statistics surrounding &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; means that the window is typically sampling overlapping regions, and as such there is &lt;em&gt;significant&lt;/em&gt; correlation between nearby samples. (Something I can even approach numerically/symbolically)  Consequently, this can be thought of as a more complex function of &lt;span class=&quot;math-container&quot;&gt;$f(x, y, N)$&lt;/span&gt; where larger &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; will give an estimate more related to the neighborhood (highly correlated), and a smaller &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; will give a more variable, but more local estimate.&lt;/p&gt;&#xA;&lt;h2&gt;Things I Have Tried:&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Brute Computation - Works well.  95% correct segmentation with constant &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;.  The results look fantastic when contoured using any standard method after that.  This takes &lt;em&gt;forever&lt;/em&gt;.   I can simplify the features calculated on a per-sample basis, but ideally I want to avoid this to keep this code general to images with textures who's differences show up in different parts of the feature space.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Dumb Stepping - Take a single pixel &amp;quot;step&amp;quot; in each direction and pick the direction to move based on closeness to iso-line value.  Still pretty slow, and it will ignore bifurcation of an isoline. Also, in areas with a flat gradient it will &amp;quot;wander&amp;quot; or double back on itself.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I am thinking I want to do something like the subdivison proposed in the first link, but pruned for boxes which bound the isoline of interest.  I feel like I should be able to leverage &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; also, but I am not sure how to approach that.&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-03-07T19:39:04.187" Title="Trace An Isoline of an Expensive 2D Function" Tags="|optimization|image-processing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1543" PostTypeId="2" ParentId="126" CreationDate="2012-03-06T00:40:45.953" Score="1" Body="&lt;p&gt;I found an interesting recursive algorithm &lt;a href=&quot;http://astronu.jinr.ru/wiki/upload/e/ef/NumericalRecipesinF90.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; at page 975.&#xA;It looks promising, I wonder what more experienced people say about it.&lt;/p&gt;&#xA;" OwnerUserId="1088" LastEditorUserId="1064" LastEditDate="2012-03-06T08:32:14.153" LastActivityDate="2012-03-06T08:32:14.153" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1545" PostTypeId="1" AcceptedAnswerId="1551" CreationDate="2012-03-06T14:58:54.790" Score="11" ViewCount="888" Body="&lt;p&gt;As someone who has to teach courses in computational science, I am confronted with the age-old question: how do I evaluate the ability of the students to learn a subject that depends on applications that are difficult to test with &quot;standard&quot; testing methods (written or oral exams)? Part of the course does depend on understanding the theory and methods on an abstract level, and for that, I'd like to continue to use a written test for those concepts. However, testing understanding of practical use of these methods requires a different approach&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the natural challenges associated not only with the proliferation of different platforms (for MATLAB, Modelica, Mathematica, and other languages) but also with Internet connectivity and test security, I would be interested in new or original methods for practically evaluating  students' understanding of numerical methods. (Features promoting test security are particularly desirable.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I should also mention that the class I am teaching is an introductory-level course, so the students have a relatively small knowledge base to work from.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-03-06T17:35:56.043" LastActivityDate="2012-03-10T20:35:18.673" Title="How can we evaluate performance of students in computational science courses?" Tags="|education|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1546" PostTypeId="2" ParentId="1545" CreationDate="2012-03-06T17:02:54.620" Score="2" Body="&lt;p&gt;In my honest opinion, I think the ultimate test is to apply your knowledge in unfamiliar territory.  I would propose applied projects designed to test a students' ability to select effective models, discretization, approximation/solver methods, exploitable parallelism, error estimates and numerical analysis, as well as visualization methods to describe a particular physical phenomenon of computational interest.  I would go further as to ask students to justify each choice based upon the problem size / expected accuracy.  The key is to know which methods are appropriate under the constraints of the phenomena under investigation.  Students may opt to select a phenomenon of their choosing.  But if you want to make it even more challenging, assign each student a computational project in an unrelated field to their dissertation research.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-03-06T17:02:54.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1547" PostTypeId="2" ParentId="1545" CreationDate="2012-03-06T20:55:09.620" Score="5" Body="&lt;p&gt;I have lectured, taught, or assisted in a variety of courses relating to numerical and computational methods, from the upper level undergraduate to the advanced graduate level.  Here are the elements I have found to be helpful as an instructor:  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Research Projects&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For advanced classes, a research project (incorporating numerical research, usually some software development, and a write-up) are a very nice way for students to tie in their research with their academic coursework.  I think that a project should be mandatory in graduate level courses, but for undergraduates they are better replaced with more directed work.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Programming Homework Assignments&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The core of any computational science class is accessible programming assignments.  For students with no programming experience you will need to back up your assignments with some introductory sessions to the programming environment, and ideally some sort of &quot;help room&quot; offered either by your department or a student organization such as SIAM.  Allowing multiple frameworks and programming languages can be difficult, I accepted programs written in any language but &lt;em&gt;supported&lt;/em&gt; only one environment, usually freely available in the University computer laboratories (operating system, editor, shell, interpreter, etc...)&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Quizzes&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I really like short 10-15 minute in-class quizzes once every week or every other week.  It is good two-way feedback: the students see how they are doing against my expectations and against each other, and I see which concepts they are hitting and missing. This style of evaluation is not very commonly used in Europe, and I think this is a shame.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Examinations&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Examinations are pencil and paper, with analysis of algorithms, code fragments, and mathematical techniques.  I have never participated in a computer laboratory examination, either as a student or an instructor/evaluator.  I think the closest thing I have seen is requiring a student to demonstrate their homework or project as well as answer questions about design or implementation.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;The Dishonesty Constraint&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Both as a student and as an instructor, I have seen enough dishonesty in the academic system to avoid relying on honor for more than 50% of a student's grade.  This means that evaluations such as projects and homework, where access to external resources could lead to academic dishonesty, do not contribute to more than 50% of the course grade.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-03-06T20:55:09.620" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1548" PostTypeId="2" ParentId="1542" CreationDate="2012-03-06T21:02:03.153" Score="1" Body="&lt;p&gt;You might try applying the core features of the Efficient Global Reliability Analysis (EGRA) method. This method was derived for the efficient computation of a probability of failure, but the guts of it are focused on doing what you describe - creating a model that is accurate only near a specific contour of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might be an interesting starting point, but I'm not sure it will solve your problem. It depends very much on the shape of your function. If it is something that can be approximated well with a &lt;a href=&quot;http://en.wikipedia.org/wiki/Kriging&quot; rel=&quot;nofollow&quot;&gt;kriging&lt;/a&gt; model, then it should perform well. These models are pretty flexible, but generally need a smooth underlying function. I have tried applying EGRA to an image segmentation application in the past, but had little success because it simply doesn't make sense to fit a surrogate model to something that isn't really defined by a functional relationship. Still, I mention it as something you might want to look into in case your application is different than I'm envisioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I haven't talked you out of it, you can read more about EGRA &lt;a href=&quot;http://etd.library.vanderbilt.edu/available/etd-04052010-103718/unrestricted/BichonDissertation.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (PDF link) and &lt;a href=&quot;http://www.mendeley.com/research/efficient-global-reliability-analysis-nonlinear-implicit-performance-functions/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, and there is an existing implementation in Sandia's &lt;a href=&quot;http://dakota.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;DAKOTA&lt;/a&gt; project - to my knowledge, the only open-source implementation of EGRA available.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2012-03-06T21:02:03.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1551" PostTypeId="2" ParentId="1545" CreationDate="2012-03-07T01:37:31.970" Score="5" Body="&lt;p&gt;Here's what I've seen as a student and teaching assistant:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Projects&lt;/strong&gt; in programming and numerical methods were good in terms of bringing lots of concepts together and forcing me and other students to be creative. However, with computational projects, it's probably worth devoting a little bit of time to &lt;a href=&quot;http://software-carpentry.org/&quot;&gt;software carpentry&lt;/a&gt; skills so that students write code that's more organized. When I was an undergrad and didn't know any better, I had cut-and-paste spaghetti code that was probably hell on graders. You may want to avoid that fate by teaching them some good practices. Also, if you give everyone similar enough projects (e.g., calculate all of the thermodynamic properties in a given list for a set of compounds was one term-long project in an undergraduate thermodynamics class; later on, it was a week-long homework assignment in graduate thermodynamics), basically expect them practically copy off of each other and debug each other's code.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Homeworks&lt;/strong&gt;, weekly or biweekly, were the best short-term method for learning new methods and concepts. It's easier to program something up, given a week to do it. Again, expect them to more or less copy off each other and debug each other's code.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Quizzes&lt;/strong&gt; weren't really good for anything other than a couple short methods or analysis questions. You can't do any programming on quizzes, but there should also be less &lt;strike&gt;cheating&lt;/strike&gt;, I mean, collaboration. You could also test pencil-and-paper coding in quizzes, which is good for basic concepts, but probably unfair for advanced concepts, or anything that requires very specialized commands, because students would have access to documentation if they were coding on a computer.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Exams&lt;/strong&gt; were more or less the same as quizzes, if they were given in class, but longer, and more difficult. I have had classes give take-home exams in computational work, in which case you can ask more computationally-oriented questions and expect them to program to solve problems. However, take-home exams have the same sort of problems as homeworks and undergraduate-type projects, in which case, it's probably better if you establish a more draconian collaboration policy for take-home exams. I've had some really good take-home exams, so I think these can work well if the instructor is sufficiently creative.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Computational Labs&lt;/strong&gt; are less effective than the analogous wet labs in science classes, because with a computer in front of you, it's much easier to goof off. There were a few guys in my classes who always spent time in computational labs playing online poker. These labs are probably most effective as demonstrations, or as supervised lessons on practical skills in computational science if you have enough teaching assistants to roam the lab and make sure that people are getting help and no one is screwing around online.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-03-07T01:37:31.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1552" PostTypeId="5" CreationDate="2012-03-07T16:42:34.157" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-07T16:42:34.157" LastActivityDate="2012-03-07T16:42:34.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1553" PostTypeId="4" CreationDate="2012-03-07T16:42:34.157" Score="0" Body="The technique of using a hash function to map large data sets (values) to smaller data sets (keys)." OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-03-07T17:28:31.507" LastActivityDate="2012-03-07T17:28:31.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1554" PostTypeId="5" CreationDate="2012-03-07T16:45:14.120" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-07T16:45:14.120" LastActivityDate="2012-03-07T16:45:14.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1555" PostTypeId="4" CreationDate="2012-03-07T16:45:14.120" Score="0" Body="For questions involving interval arithmetic, a method for determining the uncertainty of a function/operation given the uncertainty of its input variables." OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-12-14T22:01:57.547" LastActivityDate="2021-12-14T22:01:57.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1556" PostTypeId="1" CreationDate="2012-03-07T17:04:31.573" Score="8" ViewCount="1097" Body="&lt;p&gt;What are some good resources (books, articles, sites) about polygon intersection and union algorithms?&lt;/p&gt;&#xA;" OwnerUserId="1109" LastEditorUserId="414" LastEditDate="2012-03-07T19:02:48.163" LastActivityDate="2018-06-05T03:58:18.963" Title="Some good reading on polygon algorithms" Tags="|computational-geometry|reference-request|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1558" PostTypeId="1" AcceptedAnswerId="1563" CreationDate="2012-03-07T17:43:10.730" Score="3" ViewCount="76" Body="&lt;p&gt;Suppose a linear system is given&#xA;$$AX=B,$$&#xA;where $A\in\mathbb{R}^{n\times n}$ is a symmetric strictly diagonal matrix, and $X, B\in\mathbb{R}^{n\times 2}$. Therefore, the 2D Jacobi iterative solver is applicable here,&#xA;$$X_{k+1}=D^{-1}(B-RX_k),$$&#xA;where the splitting corresponds to $A=D+R$, with $D$ containing only the diagonal entries of $A$. Could it be proven that the iterands $X_0, X_1, X_2,  \dots$ are maintaining the line per each row of $X$, meaning that, for $x^i\in\mathbb{R}^2$ being the $i-$th row of $X$, the rows in sequence $x^i_0, x^i_1, x^i_2, \dots$, are colinear (on one line)?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-03-07T19:53:48.130" Title="2D Jacobi line maintenance?" Tags="|linear-algebra|optimization|computational-geometry|linear-solver|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1559" PostTypeId="2" ParentId="1556" CreationDate="2012-03-07T19:22:27.777" Score="4" Body="&lt;p&gt;I am a big fan of Joseph O'Rourke's works.  I highly recommend his book Computational Geometry in C (2nd edition) because it has a particularly good balance of theory and implementation.  Chapter 7 contains direct information pertaining to polygon intersection.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-09T03:59:29.663" LastActivityDate="2012-03-09T03:59:29.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1560" PostTypeId="5" CreationDate="2012-03-07T19:32:22.557" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-07T19:32:22.557" LastActivityDate="2012-03-07T19:32:22.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1561" PostTypeId="4" CreationDate="2012-03-07T19:32:22.557" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-07T19:32:22.557" LastActivityDate="2012-03-07T19:32:22.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1562" PostTypeId="2" ParentId="1542" CreationDate="2012-03-07T19:39:04.187" Score="4" Body="&lt;p&gt;There is a paper in computer graphics called &lt;a href=&quot;http://geometrica.saclay.inria.fr/team/Steve.Oudot/papers/bo-pgsms-05/bo-pgsms-05.pdf&quot; rel=&quot;nofollow&quot;&gt;Provably Good Sampling and Meshing of Surfaces&lt;/a&gt;, which relies on you providing an oracle that determines all the intersections of an isoline with a given line segment. With that, it samples all the contours assuming you can provide a local feature scale (something like the maximum local curvature), and an initial set of line segments that intersects all the contours. It is not the simplest thing to implement, since it relies on computing Delaunay triangulations, but it is implemented in 3D in &lt;a href=&quot;http://www.cgal.org&quot; rel=&quot;nofollow&quot;&gt;CGAL&lt;/a&gt;. It is substantially simpler in 2D, since good triangulation software like &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow&quot;&gt;Triangle&lt;/a&gt; exists. In some sense, this is pretty close to the best you can possibly do.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-03-07T19:39:04.187" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1563" PostTypeId="2" ParentId="1558" CreationDate="2012-03-07T19:53:48.130" Score="3" Body="&lt;p&gt;Let's take the difference between two successive iterations, and define $P$ and $Q$ this way:&#xA;$$ \Delta_k = X_{k+1} - X_{k} = D^{-1}B - (D^{-1}R + I)X_{k} = P-QX_{k} $$&#xA;Then&#xA;$$ \Delta_{k+1} = X_{k+2} - X_{k+1} = P-QX_{k+1} $$&#xA;To be row-wise collinear, we need $\Delta_{k+1} - \Delta_{k} = S \Delta_{k}$ for some diagonal matrix $S$. Doing this, we get&#xA;$$ \Delta_{k+1} - \Delta_{k} = Q(X_{k+1}-X_k) = Q\Delta_k $$&#xA;Since $Q = D^{-1}R + I$ is not diagonal unless $R$ is diagonal, then in general these iterates will not be collinear.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-03-07T19:53:48.130" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1564" PostTypeId="2" ParentId="1556" CreationDate="2012-03-07T21:09:01.637" Score="3" Body="&lt;p&gt;Paul's suggestion is great, I would just like to add two more:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Geometric Tools for Computer Graphics&quot;, Schneider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Computational Geometry&quot; Mark deBerg et al. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On this note, my 2cents (coming from experience): if you are considering coding such algorithms, I advise you kindly to take a look at Boost::Geometry and/or CGAL libray first, there is no need (hopefully) to re-invent the wheel. If you are coding in C++, that is.... &lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-03-07T21:09:01.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1565" PostTypeId="2" ParentId="1556" CreationDate="2012-03-07T23:48:05.010" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://www.cs.man.ac.uk/~toby/alan/software/&quot; rel=&quot;nofollow&quot;&gt;GPC, General Polygon Clipper&lt;/a&gt; is a good implementation for boolean operations on polygons based on &lt;a href=&quot;http://en.wikipedia.org/wiki/Vatti_clipping_algorithm&quot; rel=&quot;nofollow&quot;&gt;Vatti's clipping algorithm&lt;/a&gt;. The page also contains &lt;a href=&quot;http://www.cs.man.ac.uk/~toby/alan/software/#Links&quot; rel=&quot;nofollow&quot;&gt;links&lt;/a&gt; to other solutions.&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2012-03-07T23:48:05.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1566" PostTypeId="2" ParentId="1489" CreationDate="2012-03-08T00:05:32.783" Score="6" Body="&lt;p&gt;I think if there were a &lt;strong&gt;preconditioner&lt;/strong&gt; which&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;was general in the sense of working for an arbitrary dense matrix without defined structure or contents&lt;/li&gt;&#xA;&lt;li&gt;accelerated your solution rate without other drawback&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then you would always use it in your solution algorithim, and thus it would become part of your &lt;strong&gt;solver&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-03-08T00:05:32.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1567" PostTypeId="2" ParentId="1489" CreationDate="2012-03-08T03:15:28.193" Score="10" Body="&lt;p&gt;It is critical to know more about the structure. It matters whether the random entries are uniformly or normally distributed and whether there is a shift or not. If there is no structure at all, then you cannot asymptotically beat a direct solve. Some comments on your proposed approaches&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Incomplete LU is complete LU when applied to a dense matrix. You could consider some thresholding, but it's not likely to work, especially not with a uniform distribution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The inverse is not sparse or have useful decay properties, so a sparse approximate inverse would not be expected to perform well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Integral formulations of Maxwell's equations have very special structure. That paper uses thresholding to create a sparse system. Whether that is beneficial or not (and whether the sparsified matrix is significantly easier to solve with) depends strongly on any special structure in the problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-08T03:15:28.193" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1568" PostTypeId="2" ParentId="1489" CreationDate="2012-03-08T11:51:13.263" Score="6" Body="&lt;p&gt;For dense matrices without structure, polynomial preconditioning is probably the only viable method, though it has limitations (see, e.g., &lt;a href=&quot;http://amath.colorado.edu/pub/iterative/psi-phi.ps.Z&quot;&gt;http://amath.colorado.edu/pub/iterative/psi-phi.ps.Z&lt;/a&gt;). If your matrix has entries of different magnitudes, it may be necessary that you first scale your matrix using a matching &#xA;&lt;a href=&quot;http://www.cerfacs.fr/algor/reports/1997/TR_PA_97_45.ps.gz&quot;&gt;http://www.cerfacs.fr/algor/reports/1997/TR_PA_97_45.ps.gz&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-08T11:51:13.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1569" PostTypeId="2" ParentId="1485" CreationDate="2012-03-08T11:54:15.897" Score="8" Body="&lt;p&gt;The conjugate gradient algorithm works for semidefinite problems and produces the minimal norm solution.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-08T11:54:15.897" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1570" PostTypeId="2" ParentId="1223" CreationDate="2012-03-08T11:57:54.113" Score="2" Body="&lt;p&gt;(i) Just look at the activities of your approximate transformedsolution, and solve the associated linear system. if the approximation was reasonable, you should get the optimal solution of the original system in thisway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ii) Why don't you solve directly the original problem? &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-08T11:57:54.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1571" PostTypeId="1" AcceptedAnswerId="1572" CreationDate="2012-03-08T16:10:06.597" Score="6" ViewCount="3400" Body="&lt;p&gt;Is there an efficient way to numerically compute Stirling numbers of the second kind?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approximate (not exact) method would suffice. Something similar to the connection between factorials and gamma functions would work for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I googled it a bit and didn't find anything too helpful.&lt;/p&gt;&#xA;" OwnerUserId="1118" LastEditorUserId="1118" LastEditDate="2012-03-08T23:19:22.163" LastActivityDate="2021-10-23T20:34:43.367" Title="An efficient way to numerically compute Stirling numbers of the second kind?" Tags="|algorithms|special-functions|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1572" PostTypeId="2" ParentId="1571" CreationDate="2012-03-08T17:01:28.963" Score="8" Body="&lt;p&gt;You can use the following recurrence relation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathop{S}\nolimits\left(n,k\right)=k\mathop{S}\nolimits\left(n-1,k\right)+\mathop{S}\nolimits\!\left(n-1,k-1\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;see&#xA;&lt;a href=&quot;http://dlmf.nist.gov/26.8.E22&quot; rel=&quot;noreferrer&quot;&gt;DLMF equation 26.8.E22&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and build the (triangular) table.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-03-08T17:01:28.963" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1573" PostTypeId="1" AcceptedAnswerId="1606" CreationDate="2012-03-09T03:55:43.817" Score="5" ViewCount="7640" Body="&lt;p&gt;I have a distributed matrix, in block column format.  I know that I can reshape the matrix into one long vector and use an all_gatherv operation.  I just wanted to avoid the trouble of having to reshape the matrix in my code.  So, I was wondering if there's an mpi all gather operation so that in the end, every processor has an exact copy of the full matrix.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="9" LastEditDate="2012-03-09T06:42:52.460" LastActivityDate="2012-03-12T22:49:02.677" Title="Is there an MPI All Gather operation for matrices?" Tags="|matrix|mpi|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1574" PostTypeId="1" AcceptedAnswerId="1586" CreationDate="2012-03-09T04:29:24.620" Score="3" ViewCount="445" Body="&lt;p&gt;I'm simulating two phase immiscible drainage (air displacing water) in a rectangular domain of size .6mm x 2.4mm (2 dimensions) using Ansys FLUENT software.  I am using an implicit Volume of Fluid formulation with a PRESTO pressure solver and PISO pressure-velocity coupling, with 2nd order upwind schemes for momentum and volume fraction.  The problem is that even with a highly refined mesh, my scheme diverges very quickly (within the first few timesteps, using an algebraic multigrid solver).  When I try taking smaller timesteps (about $10^{-6}$ s), the solution takes a lot longer to diverge, but still diverges nonetheless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if the problem is with the choice of solver itself or if the boundary conditions were not set correctly.  I chose to use a constant inlet velocity of .0001m/s, and zero pressure conditions at the outlet.  I used no-slip boundary conditions at the other two walls, with wall adhesion and a constant air-to-water contact angle of 135 degrees.  Any help with this would be greatly appreciated! :)&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-10T06:43:14.123" LastActivityDate="2012-03-10T17:13:00.133" Title="What numerical methods are recommendable for simulating two phase immiscible fluid flow through a pipe with high capillary pressure?" Tags="|linear-solver|fluid-dynamics|boundary-conditions|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1575" PostTypeId="5" CreationDate="2012-03-09T04:36:26.077" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-09T04:36:26.077" LastActivityDate="2012-03-09T04:36:26.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1576" PostTypeId="4" CreationDate="2012-03-09T04:36:26.077" Score="0" Body="For questions regarding the choice and/or appropriateness of conditions necessary to model a particular phenomenon with a partial differential equations." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-09T06:43:32.443" LastActivityDate="2012-03-09T06:43:32.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1577" PostTypeId="5" CreationDate="2012-03-09T04:39:55.103" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-09T04:39:55.103" LastActivityDate="2012-03-09T04:39:55.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1578" PostTypeId="4" CreationDate="2012-03-09T04:39:55.103" Score="0" Body="For questions regarding the modeling and/or simulation of systems with two or more solids and/or fluids." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-09T06:43:47.077" LastActivityDate="2012-03-09T06:43:47.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1579" PostTypeId="5" CreationDate="2012-03-09T04:41:13.730" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-09T04:41:13.730" LastActivityDate="2012-03-09T04:41:13.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1580" PostTypeId="4" CreationDate="2012-03-09T04:41:13.730" Score="0" Body="For questions about modeling phenomena that occur at different spatial or temporal scales." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-09T06:43:49.277" LastActivityDate="2012-03-09T06:43:49.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1581" PostTypeId="5" CreationDate="2012-03-09T04:45:25.177" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-09T04:45:25.177" LastActivityDate="2012-03-09T04:45:25.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1582" PostTypeId="4" CreationDate="2012-03-09T04:45:25.177" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-09T04:45:25.177" LastActivityDate="2012-03-09T04:45:25.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1583" PostTypeId="1" AcceptedAnswerId="1598" CreationDate="2012-03-09T06:03:33.360" Score="9" ViewCount="643" Body="&lt;p&gt;Given a generic sparse matrix $A \in \mathbb{R}^{n\times n}$ with &lt;strike&gt;m &amp;lt;&amp;lt; n&lt;/strike&gt; (correction: $m \ll n^2$) non-zero elements (typically $m \in {\cal O}(n)$). $A$ is generic in the sense that it has no specific properties (e.g. positive definiteness), and no structure (e.g. bandedness) is assumed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some of the good &lt;em&gt;numerical&lt;/em&gt; methods to compute either the characteristic polynomial or the minimal polynomial of $A$?&lt;/p&gt;&#xA;" OwnerDisplayName="user182" LastEditorDisplayName="user182" LastEditDate="2012-03-10T17:15:16.343" LastActivityDate="2012-03-11T18:00:49.763" Title="Computing the characteristic polynomial of real sparse matrix" Tags="|linear-algebra|algorithms|sparse-matrix|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1584" PostTypeId="2" ParentId="1573" CreationDate="2012-03-09T06:37:37.893" Score="5" Body="&lt;p&gt;There are a number of possible solutions to this, I would recommend that you commit a new MPI data type using &lt;a href=&quot;http://dan.lecocq.us/wordpress/2010/08/22/mpi_datatype/&quot;&gt;&lt;code&gt;MPI_Type_struct()&lt;/code&gt;&lt;/a&gt;.  If you have multiple matrices of different sizes, you'll need to commit a new data type for each (the data type size is static) or consider a more flexible approach.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-03-09T06:37:37.893" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1585" PostTypeId="2" ParentId="277" CreationDate="2012-03-09T06:42:22.127" Score="4" Body="&lt;p&gt;My personal experience is that HPC requires such a broad set of skills that is very difficult to do it in just one book, recipe, university course. Consider that HPC is not about programming big machines or building a cluster but rather tackling big science problems at the bleeding edge of computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure everyone would agree but I think that a list of areas involved in HPC could be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Computational &lt;a href=&quot;http://en.wikipedia.org/wiki/Computational_model&quot; rel=&quot;nofollow&quot;&gt;models&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Computer_simulation&quot; rel=&quot;nofollow&quot;&gt;simulations&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_analysis&quot; rel=&quot;nofollow&quot;&gt;numerical analysis&lt;/a&gt;, and &lt;a href=&quot;http://en.wikipedia.org/wiki/Algorithm&quot; rel=&quot;nofollow&quot;&gt;algorithms&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Scientific_modelling&quot; rel=&quot;nofollow&quot;&gt;Scientific modelling&lt;/a&gt; in fields like applied mathematics, physics, engineering&lt;/li&gt;&#xA;&lt;li&gt;Computer &lt;a href=&quot;http://en.wikipedia.org/wiki/Computer_architecture&quot; rel=&quot;nofollow&quot;&gt;architectures&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Computer_programming&quot; rel=&quot;nofollow&quot;&gt;programming&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Scientific_visualization&quot; rel=&quot;nofollow&quot;&gt;Scientific visualization&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Each of this areas are immensely large and there is so much knowledge that is difficult to get hold of everything before attempting to do anything!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, I started from one corner of engineering and slowly developed experience and a collaboration network. At the very beginning I just had one engineering / applied maths problem that I wanted to solve, from there I moved to programming, to getting more performance via more powerful computers, to numerical analysis, to advanced algorithms, to parallel computers, to parallel algorithms, to even larger computers, to a more complete scientific model, and eventually to a new problem  and was iterating again. If I look back, trying to understand and solve my first problem was what crafted my study plan. Having a supervisor that can keep you on the right track also helps a lot, otherwise be careful of not loosing sight of your end goal (in my case, this was the science problem I had at hand).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have to name one thing that really makes a difference while going down the HPC path was to meet people from other areas, exchange ideas, and take advantage of the distributed knowledge of the community. Also, it is important not to be too afraid of being openly naive and doing silly things. The first steps are always difficult and the most important ones! &lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-03-09T06:42:22.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1586" PostTypeId="2" ParentId="1574" CreationDate="2012-03-09T17:33:55.813" Score="2" Body="&lt;p&gt;It sounds like you are using Ansys Fluent. As I recall from some VOF articles most of them were using projection methods with explicit time stepping. If you are using Fluent you could try finding Non-iterative time advancement (NITE) option for time stepping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also suggest you trying to write your own solver! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are really good algorithms that you could try in that case. One of those is Moment of fluid (MOF) interface reconstruction developed by researchers from LANL. Here are some links that might be helpful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/fld.2108/abstract&quot; rel=&quot;nofollow&quot;&gt;A paper from Int. J. Numer. Methods in Fluids&lt;/a&gt;, and &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=3&amp;amp;cts=1331312543286&amp;amp;ved=0CDsQFjAC&amp;amp;url=http://www.lanl.gov/orgs/adtsc/publications/science_highlights_2009/12schofield.pdf&amp;amp;ei=gzdaT9uuHIXAswbU6uXkAQ&amp;amp;usg=AFQjCNFK7Js868XNsKzdt6GB7-VPa-VEcA&amp;amp;sig2=jVtPxaiJNxRCl0hsKBVDEQ&quot; rel=&quot;nofollow&quot;&gt;a short note about the method from LANL web site&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-03-09T17:33:55.813" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1587" PostTypeId="1" AcceptedAnswerId="1589" CreationDate="2012-03-09T17:36:18.573" Score="3" ViewCount="1653" Body="&lt;p&gt;Input&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var('x')&#xA;solve((log((x**2 - x), 6) - log((6*x - 10), 6) == 0), x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[log(x^2 - x) == log(6*x - 10)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But real roots are 5 and 2. What I doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="1127" LastEditorUserId="81" LastEditDate="2012-03-13T19:25:09.983" LastActivityDate="2012-03-13T19:25:09.983" Title="Solve log equations problem [SageMath]" Tags="|python|sage|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1588" PostTypeId="2" ParentId="1583" CreationDate="2012-03-09T17:52:44.470" Score="1" Body="&lt;p&gt;You could use a numerical method like QR Factorization or Power Method and its realtives (inverse power etc) to compute the eigenvalues of your generic matrix. After that, you can compute your characteristic polynomial by factorization as: (λ-λ1)(λ-λ2)...(λ-λn)=0 where λi are the computed eigenvalues. Here is a short presentation about power and QR methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.uio.no/studier/emner/matnat/ifi/INF-MAT4350/h08/undervisningsmateriale/chap20slides.pdf&quot; rel=&quot;nofollow&quot;&gt;QR-Power&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1059" LastActivityDate="2012-03-09T17:52:44.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1589" PostTypeId="2" ParentId="1587" CreationDate="2012-03-10T06:05:38.373" Score="4" Body="&lt;p&gt;You need to use the option &lt;code&gt;to_poly_solve=True&lt;/code&gt; which have the following semantic:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;to_poly_solve&lt;code&gt;- bool (default: False); use Maxima's&#xA;  &lt;/code&gt;to_poly_solver&lt;code&gt;package to search for more possible &#xA;     solutions, but possibly encounter approximate solutions.&#xA;     This keyword is incompatible with&lt;/code&gt;multiplicities=True``.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solve(..., x, to_poly_solve=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;should do.&lt;/p&gt;&#xA;" OwnerDisplayName="user182" LastActivityDate="2012-03-10T06:05:38.373" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1590" PostTypeId="2" ParentId="1505" CreationDate="2012-03-10T10:28:43.953" Score="5" Body="&lt;p&gt;The Jacobi iteration can be proved convergent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing you should make sure is that $c^T \mathbf{1}_n = 0$, which is the condition for existence of solution (I assume $L=L^T$, otherwise you need $c\in (\mathrm{Ker} L^T)^\perp$) because you said $V_0:=\mathrm{Ker} L = \mathrm{span}\{\mathbf{1}_n\}$.  We will use the convention that $V_0$ is also the matrix with columns being orthonormal basis of it. In your case, $V_0:=\mathbf{1}_n/\sqrt{n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, for the errors of the Jacobi iteration on the original system, you have &#xA;$$ e_1 = (I-D^{-1}L)e_0 = (I-D^{-1}L) (P e_0 + V_0a)=(I-D^{-1}L) P e_0 + V_0a,$$&#xA;where $P:=I-V_0V_0&amp;#39;$ is the orthogonal projection onto $V_1:=V_0^\perp$.  From the above iteration, we know  that &#xA;$$ P e_1 = P (I-D^{-1}L) P e_0, $$&lt;br&gt;&#xA;from which we have the iteration matrix $S$ in $V_1$,&#xA;$$ S: = P (I-D^{-1}L) P. $$&#xA;Not that $S$ has the same spectra (except zeros) with the following matrix&#xA;$$ \tilde{S}:= (I-D^{-1}L) P P=(I-D^{-1}L) P=(I-D^{-1}L)(I-V_0V_0&amp;#39;)\\&#xD;&#xA;=I-D^{-1}L-V_0V_0&amp;#39;.$$&#xA;We want the spectral radius of $S$ less than one to prove the convergence.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following quote is old and kept only for reference. See after for the new proof. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In your case, $V_0V_0&amp;#39;=\frac{1}{n}\mathbf{1}_{n\times n}.$  And you&#xA;  can verify that $D^{-1}L+V_0V_0&amp;#39;$ is strictly diagonal-dominant by&#xA;  using the assumption that the entries of $L$ are positive on the&#xA;  diagonal and negative otherwise.  To show the eigenvalues of&#xA;  $D^{-1}L+V_0V_0&amp;#39;$ are real, we note that the matrix is self-adjoint&#xA;  under the inner product $&amp;lt;x,y&amp;gt;:=y^TDx.$  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If $V_0$ is not in your specific form, I have not found an answer to&#xA;  the convergence question.  Could someone clarifies this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note that $V_0$ is the eigen-vector corresponding to the eigenvalue $1$ of $I-D^{-1}L$.  Based on the observation,  we call Theorem 2.1 from &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0893965907000614&quot; rel=&quot;nofollow&quot;&gt;Eigenvalues of rank-one updated matrices with some applications&lt;/a&gt; by Jiu Ding and Ai-Hui Zhou.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theorem 2.1&lt;/strong&gt; &lt;em&gt;Let $u$ and $v$ be two $n$-dimensional column vectors such that $u$ is an eigenvector of $A$ associated with eigenvalue $\lambda_1$. Then, the eigenvalues of $A+uv^T$ &#xA;are $\{\lambda_1+u^Tv,\lambda_2,\ldots,\lambda_n\}$ counting algebraic multiplicity.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, we know that the spectra of $\tilde{S}$ is the same as $I-D^{-1}L$ except that the eigenvalue $1$ in the latter is shifted by $-1$ into the eigenvalue zero in the former.  Since $\rho(I-D^{-1}L)\subset (-1,1]$, we have $\rho(\tilde{S})\subset (-1,1)$.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2012-03-26T08:24:45.627" LastActivityDate="2012-03-26T08:24:45.627" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="1592" PostTypeId="1" CreationDate="2012-03-10T16:15:14.167" Score="6" ViewCount="306" Body="&lt;p&gt;I am going to put this question on math exchange and stack exchange as well (since it doesn't really fit in any of the specific fields, I don't know which it should go in), but here is the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am seeking to evaluate spherical harmonics at a given point, but according to a local coordinate system. Let the actual point be called $(x_g, y_g, z_g)$ (the 'global' point). Now, I have an atom centered at, say, $(x_{at}, y_{at}, z_{at})$ and it has a local coordinate system defined in the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) The z axis is defined by the vector from the given atom center to a chosen atom (so, for example, atom1's z axis is defined as the vector from atom1 to atom5)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) The y axis is defined by the orthogonal vector lieing in the plane defined by a second atom (so, for example, atom1's y axis is defined to lie in the plane defined by the first vector and the vector from atom1 to atom3). This is done via a simple gram schmidt orthogonalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) The x axis is defined by a simple cross product of the previous two vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(note that all vectors are normalized).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, finding this information will leave you with a 3x3 matrix defined the points which define the 'end points' of the given axes. So the Local Coordinate matrix (for a given atom) may be defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left( \begin{array}{ccc}&#xD;&#xA;x_x &amp;amp; x_y &amp;amp; x_z \\&#xD;&#xA;y_x &amp;amp; y_y &amp;amp; y_z \\&#xD;&#xA;z_x &amp;amp; z_y &amp;amp; z_z \end{array} \right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where it is assumed that the 'zero' point (or the center of the local coordinate system) is $(x_{at}, y_{at}, z_{at})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, given this information, what is the best way to evaluate spherical harmonics (or any function really) in the &lt;em&gt;local coordinate system&lt;/em&gt; of a point in the &lt;em&gt;global coordinate system&lt;/em&gt;. So, for example, I want to evaluate a point $(x_g, y_g, z_g)$ but with the 'x' direction defined by $(x_x, x_y, x_z)$ and so on.&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2012-03-16T11:55:27.520" Title="changing from global to local coordinate structure" Tags="|local-coordinates|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1593" PostTypeId="2" ParentId="1583" CreationDate="2012-03-10T17:10:36.220" Score="0" Body="&lt;p&gt;By the way: Did you want to say that you have &#xA;$ m \ll O(n^2) $ entries? If indeed&#xA;$ m \ll O(n) $ then the majority of rows and columns will be completely empty and it is likely that the characteristic polynomial is in fact not of degree $n$ but of degree $O(m)$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-10T17:10:36.220" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1594" PostTypeId="2" ParentId="1574" CreationDate="2012-03-10T17:13:00.133" Score="2" Body="&lt;p&gt;Here's our (open source) entry to the field:&#xA;  &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_43.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_43.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-10T17:13:00.133" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1595" PostTypeId="2" ParentId="1545" CreationDate="2012-03-10T20:35:18.673" Score="5" Body="&lt;p&gt;Some of the other answers suggest individualized projects. I'm doing that in my finite element software class and it's a lot of fun; I believe it's also really instructive for students. At the same time, it's also hugely time intensive: last time I had 18 students and it was practically a full-time job for the entire semester to supervise these projects. So one has to have a small enough class to make that work successfully.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-10T20:35:18.673" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1596" PostTypeId="1" AcceptedAnswerId="1597" CreationDate="2012-03-11T12:53:48.257" Score="7" ViewCount="397" Body="&lt;p&gt;Is there any paper or research concerning float128 arithmetics applied to linear algebra problems(e.g. iterative solvers, decompositions etc.)?&lt;br&gt;&#xA;How much benefit is really there in comparison with double? &#xA;How much slower is it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've discovered this toolbox: &lt;a href=&quot;http://www.advanpix.com/&quot;&gt;http://www.advanpix.com/&lt;/a&gt;&#xA;Unfortunately it is commercial, but results if they are true, look impressive.&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2012-03-11T14:09:14.013" LastActivityDate="2012-03-11T14:09:14.013" Title="float128 in linear algebra" Tags="|linear-algebra|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1597" PostTypeId="2" ParentId="1596" CreationDate="2012-03-11T14:03:36.090" Score="8" Body="&lt;p&gt;The quad-precision type &lt;code&gt;__float128&lt;/code&gt; is available in PETSc. Since most sparse linear algebra methods are limited by memory bandwidth instead of floating point performance, it is often only about half the speed of double precision. It is especially useful when using matrix-free finite differencing (frequently used with Jacobian-free Newton-Krylov methods) or exploring whether a nearly-singular operator is actually singular. For most problems, it is possible and preferable to use a well-scaled and conditioning-friendly discretization so that quad precision is not necessary for production runs.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-11T14:03:36.090" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1598" PostTypeId="2" ParentId="1583" CreationDate="2012-03-11T18:00:49.763" Score="1" Body="&lt;p&gt;If $O(n^3)$ complexity is not a stopper then you might want to look at Danilevskii method. It is pretty well-known in Russian literature on numerical linear algebra, but there's not much information in English. You can start from &lt;a href=&quot;http://www.encyclopediaofmath.org/index.php/Complete_problem_of_eigen_values&quot; rel=&quot;nofollow&quot;&gt;this link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is rather straightforward: the matrix is gradually reduced to &lt;a href=&quot;http://en.wikipedia.org/wiki/Frobenius_normal_form&quot; rel=&quot;nofollow&quot;&gt;Frobenius normal form&lt;/a&gt; by &quot;Gaussian elimination-like&quot; similarity transformations. If you don't find the information I can make the algorithm more elaborate.&lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2012-03-11T18:00:49.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1599" PostTypeId="2" ParentId="1592" CreationDate="2012-03-11T22:08:30.583" Score="3" Body="&lt;p&gt;You will have to transform each of your global points to the local coordinate system. From your description you really just have a translation and rotation.&#xA;$$x_{local} = U(x_{global} -T)$$&#xA;where $U$ is unitary and $T$ is a translation vector. &#xA;Now you have $Y_{m,l}^{local}(\theta(x_{local}),\eta(x_{local}))$. &lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2012-03-11T22:08:30.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1601" PostTypeId="1" AcceptedAnswerId="1611" CreationDate="2012-03-12T11:22:10.813" Score="5" ViewCount="304" Body="&lt;p&gt;For given matrix $A \in R^{n\times n}$, identity matrix $I$ and constant $c &amp;gt; 0$ is this possible to express $cond(A + cI)$ knowing $cond(A)$ and $c$?&lt;/p&gt;&#xA;" OwnerUserId="1089" LastActivityDate="2012-03-14T11:56:58.667" Title="Condition number of (A + cI) matrix" Tags="|condition-number|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1602" PostTypeId="2" ParentId="1592" CreationDate="2012-03-12T12:02:17.543" Score="4" Body="&lt;p&gt;Coordinate transformations between &quot;global&quot; and &quot;local&quot; systems are pretty standard in the field of &lt;a href=&quot;http://en.wikipedia.org/wiki/Geodesy&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geodesy&lt;/a&gt;. There are great &lt;a href=&quot;http://gge.unb.ca/Pubs/LectureNotes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;lecture notes&lt;/a&gt; from the University of New Brunswick on the subject of coordinate systems and least squares. I recommend the Coordinate Systems in Geodesy notes. Around page 52 they explain how to transform between the Local Astronomical system (kind of like your local system) to the Average (or Conventional) Terrestrial system (a global system)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\pmatrix{x_g \\ y_g \\ z_g} = \mathbf{R}_3(180^\circ - \lambda)\mathbf{R}_2(90^\circ - \phi)\mathbf{P}_2\pmatrix{x_x \\ x_y \\ x_z}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\lambda$ is the longitude of $(x_{at}, y_{at}, z_{at})$ and $\theta$ is the latitude. The angles 180 and 90 you will need to adapt to your problem, but the logic is the same. Matrix $\mathbf{P}_2$ is used to reflect the $y$ axis because the global system is right-handed and the Local Astronomical system is left-handed. If that is not your case, then just leave the matrix out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that you can use the &lt;a href=&quot;https://scicomp.stackexchange.com/a/1599/1126&quot;&gt;answer by Deathbreath&lt;/a&gt; to calculate the spherical harmonics.&lt;/p&gt;&#xA;" OwnerUserId="1126" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-03-16T11:55:27.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1603" PostTypeId="1" AcceptedAnswerId="1604" CreationDate="2012-03-12T13:59:47.327" Score="6" ViewCount="3761" Body="&lt;p&gt;I hope this is the right stackexchange for this question; if not, please direct me there!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm on Linux. I've installed IPOPT and AMPL, and all the third-party stuff required: ASL, HSL, Lapack, Metis, Mumps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run the solver from AMPL with binary or integer variables, IPOPT converts them to continuous. Does anyone know why or how to fix it? Here's a minimal example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;file test.mod:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;option solver ipopt;&#xA;var x, integer;&#xA;&#xA;maximize best: x;&#xA;s.t. cons1: x &amp;lt;= 5;&#xA;&#xA;solve;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And when I run it (The second line of output being the problem):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ampl t2.mod&#xA;Ipopt 3.10.2: &#xA;==&amp;gt; Warning: Treating 0 binary and 1 integer variables as continous.&#xA;&#xA;&#xA;******************************************************************************&#xA;This program contains Ipopt, a library for large-scale nonlinear optimization.&#xA; Ipopt is released as open source code under the Eclipse Public License (EPL).&#xA;         For more information visit http://projects.coin-or.org/Ipopt&#xA;******************************************************************************&#xA;&#xA;NOTE: You are using Ipopt by default with the MUMPS linear solver.&#xA;      Other linear solvers might be more efficient (see Ipopt documentation).&#xA;&#xA;&#xA;This is Ipopt version 3.10.2, running with linear solver mumps.&#xA;&#xA;Number of nonzeros in equality constraint Jacobian...:        0&#xA;Number of nonzeros in inequality constraint Jacobian.:        0&#xA;Number of nonzeros in Lagrangian Hessian.............:        0&#xA;// lots more output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help!&lt;/p&gt;&#xA;" OwnerUserId="1136" LastActivityDate="2013-05-10T15:46:51.600" Title="IPOPT solver auto-converts my binary variables to continuous" Tags="|software|optimization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1604" PostTypeId="2" ParentId="1603" CreationDate="2012-03-12T14:14:34.283" Score="6" Body="&lt;p&gt;IPOPT is &lt;strong&gt;not&lt;/strong&gt; for integer programs so this is the rigth thing to do as far as IPOPT is concerned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really have a MINLP problem at hand then try a MINLP solver. IPOPT is an NLP solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I do not solve MINLP (Mixed Integer NonLinear Programming) problems I could only Google too. Hopefully, someone else can suggest a solver here at scicomp.&lt;/p&gt;&#xA;" OwnerUserId="679" LastEditorUserId="679" LastEditDate="2012-03-12T14:20:26.163" LastActivityDate="2012-03-12T14:20:26.163" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1605" PostTypeId="2" ParentId="1603" CreationDate="2012-03-12T14:54:11.830" Score="5" Body="&lt;p&gt;I think the above answer (by Ali) is correct, so I am trying to offer some alternatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mixed Integer, non linear, Problems are difficult to solve. While the terms &quot;non linear&quot; covers a wide view of problems in general, mixed integer linear problems are better researched (as far as I know). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Open source software you could use for both cases is SCIP (www.scip.zib.de), though it emphasizes on mixed integer programming, constraints can be modelled for the &quot;non linear&quot; case as well (though it is in beta-status there).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to solve mixed integer linear problems (only), one could recommend the solvers GAMS, CPLEX  and GUROBI (which are partially free for academic purposes) or the COIN-OR Project (open source, I think). Thought these solvers offer some support for nonlinear constraints (i.e. quadratic), I don't think they can cover the whole aspect of non-linear problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness sake, the following annotations: In general it holds, that mixed integer problems are hard to solve, such that all of these solver can not guarantee an optimal solution in an adequate amount of time for all cases (in theory). Nevertheless I have made good experiences with SCIP, CPLEX and Gurobi, such that I would advise to give them a try, especially if you only want to run small examples (it sounds alike :) )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards&lt;/p&gt;&#xA;" OwnerUserId="772" LastActivityDate="2012-03-12T14:54:11.830" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1606" PostTypeId="2" ParentId="1573" CreationDate="2012-03-12T17:33:18.750" Score="3" Body="&lt;p&gt;If this is a dense matrix, it's pretty straightforward; you use MPI_Type_create_subarray or something similar (you could build it yourself out of MPI_Type_vector or whatever; MPI_Type_struct() is the just about the most general possible option) to define a single column as a datatype.  (In Fortran, you wouldn't even have to do that, you'd just send (nrows) values at a time, but you'd have the same issue if you wanted to send rows instead.)   Then you need to resize the datatype so that they &quot;start&quot; and &quot;end&quot; at the right place, and you're ready to start allgatherv'ing in units of columns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;mpi.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;assert.h&amp;gt;&#xA;&#xA;void printMatrix (float **m, int rows, int cols)&#xA;{&#xA;    for (int i = 0; i &amp;lt; rows; ++i) {&#xA;        printf (&quot;%3d: &quot;, i);&#xA;        for (int j = 0; j &amp;lt; cols; ++j)&#xA;            printf (&quot;%2.0f &quot;, m[i][j]);&#xA;        printf (&quot;\n&quot;);&#xA;    }&#xA;}&#xA;&#xA;float **allocMat (int rows, int cols)&#xA;{&#xA;    float  *data   = (float  *) malloc (rows * cols * sizeof(float));&#xA;    float **matrix = (float **) malloc (rows * sizeof(float *));&#xA;    for (int i = 0; i &amp;lt; rows; i++)&#xA;        matrix[i] = &amp;amp; (data[i * cols]);&#xA;    return matrix;&#xA;}&#xA;&#xA;int main (int argc, char *argv[])&#xA;{&#xA;    int   size, rank;&#xA;    int   i, j;&#xA;    const int root = 0;&#xA;    const int globalncols = 10, globalnrows = 10;&#xA;    int ncols, start;&#xA;    int *allncols, *allstarts;&#xA;    float **matrix;&#xA;    MPI_Datatype columnunsized, column;&#xA;&#xA;    MPI_Init (&amp;amp;argc, &amp;amp;argv);&#xA;    MPI_Comm_size (MPI_COMM_WORLD, &amp;amp;size);&#xA;    MPI_Comm_rank (MPI_COMM_WORLD, &amp;amp;rank);&#xA;&#xA;    /* everyone's number of columns and offsets */&#xA;    allncols = malloc(rank * sizeof(int));&#xA;    allstarts= malloc(rank * sizeof(int));&#xA;&#xA;    /* everyone gets a global matrix */&#xA;    matrix = allocMat(globalnrows, globalncols);&#xA;&#xA;    for (i = 0; i &amp;lt; globalnrows; i++)&#xA;        for (j = 0; j &amp;lt; globalncols; j++)&#xA;            matrix[i][j] = ( i == j? 1. : 0.);&#xA;&#xA;    /* rank 0 print the results */&#xA;    if (rank == 0) {&#xA;        printf(&quot;Before:\n&quot;);&#xA;        printMatrix(matrix, globalnrows, globalncols);&#xA;    }&#xA;&#xA;&#xA;    /* how many columns are we responsble for? */&#xA;    ncols = (globalncols + rank)/size;&#xA;&#xA;    MPI_Allgather(&amp;amp;ncols, 1, MPI_INT, allncols, 1, MPI_INT, MPI_COMM_WORLD);&#xA;&#xA;    start = 0;&#xA;    for (int i=0; i&amp;lt;rank; i++)&#xA;        start += allncols[i];&#xA;&#xA;    MPI_Allgather(&amp;amp;start, 1, MPI_INT, allstarts, 1, MPI_INT, MPI_COMM_WORLD);&#xA;&#xA;    /* create the data type for a column of data */&#xA;    int sizes[2]    = {globalnrows, globalncols};&#xA;    int subsizes[2] = {globalnrows, 1};&#xA;    int starts[2]   = {0,0};&#xA;    MPI_Type_create_subarray (2, sizes, subsizes, starts, MPI_ORDER_C,&#xA;                              MPI_FLOAT, &amp;amp;columnunsized);&#xA;    MPI_Type_create_resized (columnunsized, 0, sizeof(float), &amp;amp;column);&#xA;    MPI_Type_commit(&amp;amp;column);&#xA;&#xA;&#xA;    /* everyone update their columns by adding their rank to all values */&#xA;&#xA;    for (int row=0; row&amp;lt;globalnrows; row++)&#xA;        for (int col=start; col&amp;lt;start+ncols; col++)&#xA;            matrix[row][col] += rank;&#xA;&#xA;    /* gather the updated columns */&#xA;&#xA;    MPI_Allgatherv(&amp;amp;(matrix[0][start]), ncols, column,&#xA;                   &amp;amp;(matrix[0][0]), allncols, allstarts,&#xA;                   column, MPI_COMM_WORLD);&#xA;&#xA;    /* rank 0 print the results */&#xA;    if (rank == 0) {&#xA;        printf(&quot;After:\n&quot;);&#xA;        printMatrix(matrix, globalnrows, globalncols);&#xA;    }&#xA;&#xA;    MPI_Type_free (&amp;amp;column);&#xA;    free (matrix[0]);&#xA;    free (matrix);&#xA;&#xA;    MPI_Finalize();&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ mpirun -np 4 ./columns2&#xA;Before:&#xA;  0:  1  0  0  0  0  0  0  0  0  0 &#xA;  1:  0  1  0  0  0  0  0  0  0  0 &#xA;  2:  0  0  1  0  0  0  0  0  0  0 &#xA;  3:  0  0  0  1  0  0  0  0  0  0 &#xA;  4:  0  0  0  0  1  0  0  0  0  0 &#xA;  5:  0  0  0  0  0  1  0  0  0  0 &#xA;  6:  0  0  0  0  0  0  1  0  0  0 &#xA;  7:  0  0  0  0  0  0  0  1  0  0 &#xA;  8:  0  0  0  0  0  0  0  0  1  0 &#xA;  9:  0  0  0  0  0  0  0  0  0  1 &#xA;After:&#xA;  0:  1  0  1  1  2  2  2  3  3  3 &#xA;  1:  0  1  1  1  2  2  2  3  3  3 &#xA;  2:  0  0  2  1  2  2  2  3  3  3 &#xA;  3:  0  0  1  2  2  2  2  3  3  3 &#xA;  4:  0  0  1  1  3  2  2  3  3  3 &#xA;  5:  0  0  1  1  2  3  2  3  3  3 &#xA;  6:  0  0  1  1  2  2  3  3  3  3 &#xA;  7:  0  0  1  1  2  2  2  4  3  3 &#xA;  8:  0  0  1  1  2  2  2  3  4  3 &#xA;  9:  0  0  1  1  2  2  2  3  3  4 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user389" LastEditorDisplayName="user389" LastEditDate="2012-03-12T22:49:02.677" LastActivityDate="2012-03-12T22:49:02.677" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1607" PostTypeId="2" ParentId="1601" CreationDate="2012-03-12T18:24:55.877" Score="4" Body="&lt;p&gt;It depends on the actual eigenvalues, so no. Counter example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;A_1 = \begin{bmatrix}&#xD;&#xA;1&amp;amp;0\\&#xD;&#xA;0&amp;amp;2&#xD;&#xA;\end{bmatrix}, \quad&#xD;&#xA;A_2 = \begin{bmatrix}&#xD;&#xA;100&amp;amp;0\\&#xD;&#xA;0&amp;amp;200&#xD;&#xA;\end{bmatrix}&#xD;&#xA;$$&#xA;Condition number for both matrices are 2, but when $cI$ is added, $\mbox{cond}(A_1 + cI) = \frac{2+c}{1+c}$ while $\mbox{cond}(A_2 + cI) = \frac{200+c}{100+c}$.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2012-03-12T18:24:55.877" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1608" PostTypeId="1" AcceptedAnswerId="1614" CreationDate="2012-03-12T18:32:04.680" Score="0" ViewCount="113" Body="&lt;p&gt;I am wondering if you can answer a math question that I require answered for an evolutionary program that I am writing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to the phenomenon known as the &quot;space-time tradeoff&quot; (wiki it) - what are the raw numbers for with regards to modern computers for this? IE - what advantages are gained by say - using a 2TB HDD to store frequently used variables VS calculating them on the fly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers&#xA;Martin&lt;/p&gt;&#xA;" OwnerUserId="1137" LastActivityDate="2012-03-13T07:14:21.220" Title="Space Time tradeoff" Tags="|efficiency|" AnswerCount="1" CommentCount="6" ClosedDate="2012-08-21T20:09:17.543" ContentLicense="CC BY-SA 3.0" />
  <row Id="1609" PostTypeId="1" AcceptedAnswerId="1613" CreationDate="2012-03-12T18:48:25.797" Score="2" ViewCount="752" Body="&lt;p&gt;I am currently working on an evolutionary system and most of what I have heard is that a computer like mine at the moment would be able to simulate a bee sized brain (not taking into account the time needed to evolve it)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am wondering about is what is the storage capacity/computations per second of a normal human brain compared to a modern computer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any answers would be awesome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers Martin&lt;/p&gt;&#xA;" OwnerUserId="1137" LastEditorUserId="1070" LastEditDate="2012-03-13T10:58:55.967" LastActivityDate="2012-03-13T22:09:11.770" Title="Modern Computer Efficiency vs Modern Nervous Systems" Tags="|efficiency|neuroscience|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1610" PostTypeId="1" AcceptedAnswerId="1612" CreationDate="2012-03-12T20:10:52.653" Score="1" ViewCount="67" Body="&lt;p&gt;I'm writing a program that uses PETSc and SLEPc, and I was looking for a convienient way to read in options from the command line. The description of &lt;code&gt;PetscBagSetFromOptions()&lt;/code&gt; in &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/PetscBagSetFromOptions.html#PetscBagSetFromOptions&quot; rel=&quot;nofollow&quot;&gt;the official documentation&lt;/a&gt; says it &quot;allows setting options from a bag&quot;. Does this mean that if I create a bag with a variety of &lt;code&gt;PetscFoo&lt;/code&gt; elements  I can read in my program's parameters from the command line without having to use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/PetscOptionsGetInt.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;PetscOptionsGetInt()&lt;/code&gt;&lt;/a&gt; and other functions like it?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-03-12T20:25:27.627" LastActivityDate="2012-03-12T21:36:08.527" Title="What does PetscBagSetFromOptions() do?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1611" PostTypeId="2" ParentId="1601" CreationDate="2012-03-12T21:05:50.837" Score="5" Body="&lt;p&gt;There is hardly any connection. First, the condition number depends on the norm used.&#xA;Second, even in a fixed norm, the eigenvalues don't tell much about the condition number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 2-norm, the condition number is the ration of the largest and the smallest singular value. For &lt;em&gt;symmetric positive definite&lt;/em&gt; matrices, this becomes the ratio of the largest and smallest eigenvalue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I leave it as an easy exercise that if $A$ is symmetric positive definite and you know the condition of $A$ and $A+c_0I$ for some $c_0&amp;gt;0$, you can calculate the condition number of all &#xA;$A+cI$ with $c&amp;gt;0$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="276" LastEditDate="2012-03-14T11:56:58.667" LastActivityDate="2012-03-14T11:56:58.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1612" PostTypeId="2" ParentId="1610" CreationDate="2012-03-12T21:36:08.527" Score="2" Body="&lt;p&gt;It appears from the code, linked on the very page you have described, that PetscBagSetFromOptions literally just goes through the bag's item names and checks to see if they are in the options database (by appending - in front and using PetscOptionsInt, PetscOptionsReal, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's silly that the linked &lt;i&gt;examples&lt;/i&gt;, however, use this redundantly with the PetscOptionsXXX routines, so I can see why this might be confusing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Requests like this may be sent to petsc-users@mcs.anl.gov.&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2012-03-12T21:36:08.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1613" PostTypeId="2" ParentId="1609" CreationDate="2012-03-12T23:07:27.660" Score="2" Body="&lt;p&gt;Brains aren't built for FLOPS. In fact, compared to computers, we suck at them. I can't do a floating point add of the decimal representations of 32-bit (single-precision) numbers in a second or less. I probably couldn't work with the binary representations any faster. So, in terms of FLOPS, my guess is that a bound based on adds would be somewhere around .001 to .1 FLOPS, and if we include multiplies, then maybe less than $10^{-3}$ FLOPS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fairest comparison is to measure the brain's computing power in instructions per second, or IPS, which would be comparable to integer operations for a CPU. Estimates for the computing capacity of a computer vary (see &lt;a href=&quot;http://arstechnica.com/science/news/2011/02/adding-up-the-worlds-storage-and-computation-capacities.ars&quot; rel=&quot;nofollow&quot;&gt;Ars Technica quoting an article in &lt;em&gt;Science&lt;/em&gt;&lt;/a&gt;, &lt;a href=&quot;http://insidehpc.com/2009/03/12/even-supercomputers-not-yet-close-to-the-raw-power-of-human-brain/&quot; rel=&quot;nofollow&quot;&gt;insideHPC quoting a researcher&lt;/a&gt;, &lt;a href=&quot;http://www.ualberta.ca/~chrisw/howfast.html&quot; rel=&quot;nofollow&quot;&gt;a UAlberta professor of psychology doing a back-of-the-envelope calculation&lt;/a&gt;, and &lt;a href=&quot;http://library.thinkquest.org/C001501/the_saga/compare.htm&quot; rel=&quot;nofollow&quot;&gt;an Oracle-sponsored project&lt;/a&gt;). The range seems to be $10^{11}$ to $10^{18}$ IPS, omitting prefactors from the scientific notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the storage capacity of the brain, those same articles quote a range of around $10^{8}$ to $10^{15}$ bits of storage, depending on estimation method. The range of estimates for both processing power and storage tend to try to account for the fact that the brain and computer hardware are designed for fundamentally different sets of tasks. Regardless, for the quoted processing power and storage numbers, the human brain is the more efficient of the two &quot;computers&quot; on a power basis, requiring roughly 2000ish calories per day of sustenance (plus the right micronutrients), whereas a computer typically requires a power supply in tens to hundreds of watts.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-03-13T02:07:26.450" LastActivityDate="2012-03-13T02:07:26.450" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1614" PostTypeId="2" ParentId="1608" CreationDate="2012-03-13T07:14:21.220" Score="4" Body="&lt;p&gt;Whether saving information makes something faster depends on how much time it takes to retrieve from storage and how much time it takes to recalculate the variables on the fly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a general rule, it takes something on the order of fractions of nanoseconds to get something stored in cache or to perform a CPU instruction. Going to RAM can take anywhere from ~1 nanosecond to ~20 nanoseconds, depending on the speed of the memory and if it's being accessed in a contiguous block.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting something from disk can take as much as a millisecond. This time is greatly reduced if you're reading a large amount of contiguous stuff in at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether you can make a program faster by storing intermediate values and precalculating some parts depends on how much time it takes to calculate those values in the first place. There's no simple, general answer to &quot;how much can I gain by storing intermediate values?&quot;.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-03-13T07:14:21.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1615" PostTypeId="1" AcceptedAnswerId="1633" CreationDate="2012-03-13T13:23:16.937" Score="7" ViewCount="230" Body="&lt;p&gt;Assuming that I am running Intel MKL (BLAS, LAPACK), is there any difference in performance if I run it on a 32 or 64 bit OS? (Of course, assuming that my hardware remains the same). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My processor is a 64 bit Intel i3. But am currently running Debian 32 bit. (Long story why)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of my work is shared memory parallelization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I know that the RAM above 3 GB is useless on 32 bit but assume that memory isn't as important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple of our old computers running Debian 32 bit (2 GB RAM) got upgraded to 4 GB RAM. It shows 2.9 GB in our system specs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are wondering whether it makes sense to wipe the OS, install a new one, install all our softwares on the new system, change the link lines of all our automated scripts (Intel MKL link line advisor) and waste 2 days in this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;tl;dr : Is there a performance boost if I run a 64 bit OS (&amp;amp; 64 bit MATLAB/Intel MKL) rather than a 32 bit one?&lt;/p&gt;&#xA;" OwnerUserId="529" LastActivityDate="2012-03-14T16:05:58.763" Title="Will upgrading to a 64 bit OS help me any?" Tags="|parallel-computing|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1616" PostTypeId="1" CreationDate="2012-03-13T12:06:56.813" Score="5" ViewCount="598" Body="&lt;p&gt;I want to use the sample average $(X_1 + .... X_n)/n$ as a substitute for the expectation $\mathbb{E}(X)$. As claimed by the weak law of large numbers, as n increases the sample average should converge to $\mathbb{E}(X)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to use this logic in my project, where the $X_i$ are iid exponential random variables. A simple code, however, does not demonstrate this well. This is because when we add the $n$ numbers, $X_1, ..., X_n$, it usually results in a large number and most of the precision is lost. So when I divide by $n$, the difference $|(X_1 + ... X_n)/n - \mathbb{E}(X)|$ is never very small even when I increase $n$ by a large amount. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried some simple manipulations such as taking sub sums and then taking the total average. Even here I seem to be suffering from the same problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a neat way of achieving errors less than, say, $10^{-8}$. &lt;/p&gt;&#xA;" OwnerDisplayName="Suresh" LastEditorUserId="1113" LastEditDate="2012-04-11T20:59:23.807" LastActivityDate="2012-04-13T17:31:12.103" Title="How can I reduce the error of the sample average?" Tags="|statistics|matlab|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1617" PostTypeId="1" AcceptedAnswerId="1897" CreationDate="2012-03-13T15:54:42.913" Score="31" ViewCount="2353" Body="&lt;p&gt;Can anyone recommend me a good workflow management system (WMS), preferably in Python? So far I have been using GNU Make, but it introduces a layer of complexity that I want to avoid. A good WMS should have the following features:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;integrate easily with command line tools and Python scripts,&lt;/li&gt;&#xA;&lt;li&gt;simple to use and lightweight,&lt;/li&gt;&#xA;&lt;li&gt;handle dependencies,&lt;/li&gt;&#xA;&lt;li&gt;provide command line interface,&lt;/li&gt;&#xA;&lt;li&gt;provide logging mechanism,&lt;/li&gt;&#xA;&lt;li&gt;(optional) provide data provenance.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I know that WMS are very popular in bioinformatics (for example &lt;a href=&quot;http://galaxy.psu.edu/&quot;&gt;Galaxy&lt;/a&gt;), but I am looking for something more general.&lt;/p&gt;&#xA;" OwnerUserId="1142" LastEditorUserId="414" LastEditDate="2012-03-13T20:01:04.587" LastActivityDate="2017-12-18T14:39:44.590" Title="Scientific workflow management system" Tags="|software|python|" AnswerCount="8" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1618" PostTypeId="2" ParentId="1616" CreationDate="2012-03-13T17:10:25.830" Score="5" Body="&lt;p&gt;Knuth (TAOCP Volume 2, 3rd ed., pg. 232) suggests using the formula $M_k = M_{k-1} + (x_k - M_{k-1})/k$ to calculate the mean, where $x$ is a vector of your samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also: &lt;a href=&quot;https://stackoverflow.com/questions/1930454/what-is-a-good-solution-for-calculating-an-average-where-the-sum-of-all-values-e&quot;&gt;a stackoverflow question&lt;/a&gt;, &lt;a href=&quot;http://www.johndcook.com/standard_deviation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article on accurately computing running variance&lt;/a&gt;, and &lt;a href=&quot;http://prod.sandia.gov/techlib/access-control.cgi/2008/086212.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper on computing covariances and arbitrary-order statistical moments&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="739" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-03-13T17:10:25.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1619" PostTypeId="1" CreationDate="2012-03-13T17:35:39.543" Score="2" ViewCount="554" Body="&lt;p&gt;I'm trying to design the frequency response function for a low-pass filter. I need the function to be polynomial and to fulfill the following constraints: the coefficients must sum to 1, the function must lie at the interval [0, 1] and its' value at  1 must be 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inspired by &lt;a href=&quot;https://math.stackexchange.com/questions/60610/polynomial-fitting-where-polynomial-must-be-monotonically-increasing&quot;&gt;this&lt;/a&gt; StackExchange answer, I've tried to solve this problem using an SDP solver. The only non-trivial constraints for the program are the bounds for the polynomial. I implemented them by requiring the polynomial to correspond to a quadratic form defined by a positive semidefinite matrix. Using the same method, I can require the polynomial to be monotonic by imposing a nonnegativity constraint on the polynomial's derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of a solution to the above:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/I8dh0.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the polynomial is required to be monotonic and to pass at two control points which are at the intersections of the dashed-lines on the chart. The latter is supposed to model the transition region of the filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the filter in the chart is very far from the ideal low-pass filter. This is because the SDP solver fails when I try to shrink the transition region. Another thing that happens when I do that is that the quadratic form's matrix condition number skyrockets. I suspect that these two things are related. Raising the polynomial's order doesn't help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions about how to solve this issue?&lt;/p&gt;&#xA;" OwnerUserId="533" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-03-14T12:32:04.647" Title="Polynomial Regression using Semidefinite Programming" Tags="|convex-optimization|regression|condition-number|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1620" PostTypeId="2" ParentId="1617" CreationDate="2012-03-13T19:59:41.703" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://prototype.taverna.org.uk/&quot; rel=&quot;nofollow&quot;&gt;Taverna&lt;/a&gt; is an open-source WMS, not Python but Java. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-03-13T19:59:41.703" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1621" PostTypeId="1" AcceptedAnswerId="1655" CreationDate="2012-03-13T20:15:43.617" Score="3" ViewCount="1083" Body="&lt;p&gt;Do anyone know if there exist any widely used parallel computing programming paradigms/models not based on a master/slave concept?&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-03-19T06:07:01.727" Title="Parallel computing programming paradigms/models not based on a master/slave concept?" Tags="|parallel-computing|hpc|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1623" PostTypeId="2" ParentId="1621" CreationDate="2012-03-13T20:23:07.760" Score="2" Body="&lt;p&gt;Yes, there's the &lt;a href=&quot;http://en.wikipedia.org/wiki/Thread_pool_pattern&quot; rel=&quot;nofollow&quot;&gt;Thread Pool&lt;/a&gt;, in which all threads of a parallel code access a list of tasks and compute them asynchronously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In OpenMP + pseudocode, this could be implemented as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma omp parallel&#xA;while ( there are still tasks ) {&#xA;&#xA;    #pragma omp critical&#xA;    {&#xA;        get a task T from the task list.&#xA;    }&#xA;&#xA;    exectue task T;&#xA;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This model, however, is based on some form or another of shared-memory parallelism. In a distributed-memory system you could have the task list reside on one of the nodes, but then any other node would have to be able to access it without stopping that node. I don't see how this can be done without reverting to a master/slave model though.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-03-13T20:23:07.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1624" PostTypeId="2" ParentId="1616" CreationDate="2012-03-13T20:23:46.557" Score="1" Body="&lt;p&gt;I think you can get around the precision problem by computing it as ((X_1-E(X))+...+(X_n-E(X)))/n.&lt;/p&gt;&#xA;" OwnerUserId="1143" LastActivityDate="2012-03-13T20:23:46.557" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1625" PostTypeId="2" ParentId="1616" CreationDate="2012-03-13T20:43:43.177" Score="2" Body="&lt;p&gt;Since you are summing up exponentials you may want to look at the answer provided &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1122/how-to-add-large-exponential-terms-reliably-without-overflow-errors&quot;&gt;to this SC question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-03-13T20:43:43.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1626" PostTypeId="2" ParentId="1619" CreationDate="2012-03-13T21:09:09.543" Score="3" Body="&lt;p&gt;By forcing your control points close to each other, your Vandermonde matrix becomes nearly singular and hence the condition number is poor and your solver fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, you have $x_2=x_1+\epsilon$, where $x_i$ are your control points. The two corresponding Vandermonde rows are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\begin{align}\left(\begin{matrix}&#xD;&#xA;1 &amp;amp; x_1 &amp;amp; x_1^2 &amp;amp;\cdots &amp;amp; x_1^n\\&#xD;&#xA;1 &amp;amp; x_2 &amp;amp; x_2^2 &amp;amp;\cdots &amp;amp; x_2^n&#xD;&#xA;\end{matrix}\right) &amp;amp;=&#xD;&#xA;\left(\begin{matrix}&#xD;&#xA;1 &amp;amp; x_1 &amp;amp; x_1^2 &amp;amp;\cdots &amp;amp; x_1^n\\&#xD;&#xA;1 &amp;amp; x_1+\epsilon &amp;amp; x_1^2(1 + O(\epsilon)) &amp;amp;\cdots &amp;amp; x_1^n(1+O(\epsilon))&#xD;&#xA;\end{matrix}\right) \\&#xD;&#xA;&amp;amp; \approx \left(\begin{matrix}&#xD;&#xA;1 &amp;amp; x_1 &amp;amp; x_1^2 &amp;amp;\cdots &amp;amp; x_1^n\\&#xD;&#xA;1 &amp;amp; x_1 &amp;amp; x_1^2  &amp;amp;\cdots &amp;amp; x_1^n &#xD;&#xA;\end{matrix}\right) &#xD;&#xA;\end{align}&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Sinc_filter&quot; rel=&quot;nofollow&quot;&gt;ideal low-pass filter&lt;/a&gt; response is discontinuous. A polynomial approximation will invariably exhibit the &lt;a href=&quot;http://en.wikipedia.org/wiki/Gibbs_phenomenon&quot; rel=&quot;nofollow&quot;&gt;Gibbs phenomenon&lt;/a&gt;. Constraining the polynomial to monotonicity potentiates the problem.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2012-03-14T12:32:04.647" LastActivityDate="2012-03-14T12:32:04.647" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1627" PostTypeId="2" ParentId="1621" CreationDate="2012-03-13T21:42:46.517" Score="8" Body="&lt;p&gt;Almost no HPC is done with master-slave, because it doesn't scale.   It's all &quot;peer-to-peer&quot; in some sense.   You design a decomposition of the problem - domain decomposition, frequency-space decomposition, something - and each task is assigned it's piece of the decomposition; they then work on their parts of the problem, exchanging data back and forth between each other as necessary.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest thing you'll see to master-slave is something like &quot;work-stealing&quot; which can be thought of as a distributed, peer-to-peer task queue; but even that isn't something I've seen used at really large scale; usually there's a more complicated load balancing step taken.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2012-03-13T21:42:46.517" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1628" PostTypeId="2" ParentId="1609" CreationDate="2012-03-13T22:09:11.770" Score="1" Body="&lt;p&gt;This has been discussed ad nauseam...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good survey of different estimations can be found in Ray Kurzweil's &quot;Singularity in Near&quot; p89&#xA;Overall, 10^16 - 10^18 flops or bits seems to be the ballpark figure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The various estimations are based on functional neuron behavior * number of neurons, modeling the pattern recognition of the retina and extrapolating for size of brain, etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a different take on simulating the intelligence of a brain, you can try &quot;Jeff Hawkins - On Intelligence&quot;&lt;/p&gt;&#xA;" OwnerUserId="1144" LastActivityDate="2012-03-13T22:09:11.770" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1629" PostTypeId="1" AcceptedAnswerId="1630" CreationDate="2012-03-13T23:44:09.837" Score="2" ViewCount="67" Body="&lt;p&gt;I was looking at the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/PetscBagCreate.html#PetscBagCreate&quot; rel=&quot;nofollow&quot;&gt;documentation for &lt;code&gt;PetscBagCreate()&lt;/code&gt;&lt;/a&gt;, and it says that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The size of the A struct must be small enough to fit in a PetscInt; by default PetscInt is &gt; 4 bytes. The warning about casting to a shorter length can be ignored below unless &#xA;  your A struct is too large.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I assume that this means that a pointer to the struct must be smaller than a PetscInt, the examples use structs that contain PetscInts as well as other things.  That said, I'm under the impression that pointers on 64 bit machines can be larger than the 32 bits in a PetscInt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What does this restriction on the size of the struct mean for users?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can PetscBags be used on 64 bit systems?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-03-14T00:06:59.823" LastActivityDate="2012-03-14T00:47:47.590" Title="Can PetscBags be used with 64 bit versions of petsc?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1630" PostTypeId="2" ParentId="1629" CreationDate="2012-03-14T00:47:47.590" Score="2" Body="&lt;p&gt;&lt;code&gt;PetscBag&lt;/code&gt; is meant for holding parameters or other redundant data, so it's not a place to put huge data like distributed model state. It can be used with 64-bit integers, but unfortunately, the format is not binary-compatible with the 32-bit integer format. It could be fixed to be backward-compatible at the expense of making the format slightly more complicated, but I doubt anyone is volunteering to do that just now. As for your questions:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What does this restriction on the size of the struct mean for users?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You should put less than 2GB in a &lt;code&gt;PetscBag&lt;/code&gt;. Since the data is loaded strictly (instead of lazily), you want this anyway to be able to load it with a reasonable amount of memory.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Can PetscBags be used on 64 bit systems?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Absolutely, and the format is compatible between 32- and 64-bit systems and big versus little endian systems.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-14T00:47:47.590" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1631" PostTypeId="2" ParentId="1617" CreationDate="2012-03-14T10:38:25.340" Score="8" Body="&lt;p&gt;Some month ago, I stumbled upon the highly recommended website of &lt;a href=&quot;http://www.uni-bonn.de/~hmg308/teaching.html&quot; rel=&quot;nofollow&quot;&gt;Hans-Martin v. Gaudecker&lt;/a&gt; who teaches courses like &quot;Effective programming practices for economists&quot;. In his Autumn 2010 course he introduced SCons, in his Autumn 2011 course he switched to &lt;a href=&quot;http://code.google.com/p/waf/&quot; rel=&quot;nofollow&quot;&gt;waf&lt;/a&gt;, which is supposed to be faster than SCons but still Python-based. The slides for both courses are available for download and I (as a social scientist) found them very instructive and enlightening. &lt;/p&gt;&#xA;" OwnerUserId="284" LastEditorUserId="284" LastEditDate="2012-08-21T05:33:10.473" LastActivityDate="2012-08-21T05:33:10.473" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1632" PostTypeId="1" AcceptedAnswerId="1657" CreationDate="2012-03-14T14:37:30.550" Score="11" ViewCount="3708" Body="&lt;p&gt;Given two matrices $A$ and $B$, I'd like to find vectors $x$ and $y$, such that,&#xA;$$ \min \sum_{ij} (A_{ij} - x_i y_j B_{ij})^2. $$&#xA;In matrix form, I'm trying to minimize the Frobenius norm of $A - \mbox{diag}(x) \cdot B \cdot \mbox{diag}(y) = A - B \circ (x y^\top)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, I'd like to find multiple unit vectors $x$ and $y$'s in the form&#xA;$$ \min \sum_{ij} (A_{ij} - \sum_{k=1}^n s_i x_i^{(k)} y_j^{(k)} B_{ij})^2. $$&#xA;where $s_i$'s are positive real coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is equivalent to singular value decomposition (SVD) when $(B)_{ij} = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know what this problem is called? Is there a well-known algorithm like SVD for the solution of such problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(migrated from math.SE)&lt;/p&gt;&#xA;" OwnerUserId="1128" LastEditorUserId="1128" LastEditDate="2012-03-14T17:43:59.197" LastActivityDate="2012-03-17T19:10:43.710" Title="weighted SVD problem?" Tags="|linear-algebra|matrix|reference-request|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1633" PostTypeId="2" ParentId="1615" CreationDate="2012-03-14T16:05:58.763" Score="5" Body="&lt;p&gt;TL;DR: It all depends, and you need to benchmark it, but the big differences besides memory are that you have twice as many registers, pointers are twice as big, and if newer software is limited to one type of architecture, it's probably going to be 64-bit (for the time being).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://superuser.com/questions/56540/32-bit-vs-64-bit-systems&quot;&gt;Super User has a nice post&lt;/a&gt; on the distinctions between 32-bit and 64-bit operating systems and their effect on performance. Ignoring the part about addressing more memory (as you requested), pointers are twice as big (4 bytes in 32-bit, 8 bytes in 64-bit), so you can store half as many pointers in cache, but you have twice as many registers to work with. If the bottleneck in your computations is register memory, then having twice as many can be a big help, and you may see substantial performance gains (the top Super User answer claims as much as 30% from the poster's own experience).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you look at &lt;a href=&quot;http://www.democritos.it/activities/IT-MC/talks/talk-computer.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this presentation on molecular dynamics codes&lt;/a&gt; by Axel Köhlmeyer, the answer isn't so clear. He claims that he saw no difference in performance between 32-bit and 64-bit versions of the codes he was using, though I found the graphs he presented confusing because he doesn't seem to actually compare 32-bit and 64-bit versions of his codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.wadespires.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wade Spires&lt;/a&gt; wrote a &lt;a href=&quot;http://www.cs.ucf.edu/~wspires/32bit_versus_64bit_amd64.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;report&lt;/a&gt; when he was a graduate student at UCF on the differences in performance of the SPEC2000 benchmark on 32-bit and 64-bit operating systems. The 64-bit operating system isn't necessarily more performant overall; there are some tasks where programs execute more quickly on the 32-bit OS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best advice is to benchmark, so that you know for sure what the performance gains will be. It requires some work on your part, and time that you may not have for benchmarking. If you choose to benchmark, I recommend using a virtual machine as a testbed for these sorts of comparisons so that you can easily switch between guest operating systems on the same machine while doing the minimum amount of setup and teardown work.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-03-20T10:18:34.427" LastActivityDate="2012-03-14T16:05:58.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1634" PostTypeId="1" AcceptedAnswerId="1635" CreationDate="2012-03-14T20:51:46.837" Score="25" ViewCount="2723" Body="&lt;p&gt;Suppose that I'm working on a scientific code in C++. In a recent discussion with a colleague, it was argued that expression templates could be a really bad thing, potentially making software compilable only on certain versions of gcc. Supposedly, this problem has affected a few scientific codes, as alluded to in the subtitles of &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=petsc%20diffpack%20deal&amp;amp;source=video&amp;amp;cd=1&amp;amp;sqi=2&amp;amp;ved=0CDIQtwIwAA&amp;amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dj1tbMW_Gxc4&amp;amp;ei=HANhT7eMNarL0QGyuo3VBw&amp;amp;usg=AFQjCNElchLj82toL3Iozi-0L8-zGbEDYw&quot;&gt;this parody of Downfall&lt;/a&gt;. (These are the only examples I know of, hence the link.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, other people have argued that expression templates are useful because they can yield performance gains, as in this &lt;a href=&quot;http://epubs.siam.org/sisc/resource/1/sjoce3/v34/i2/pC42_s1&quot;&gt;paper in SIAM Journal of Scientific Computing&lt;/a&gt;, by avoiding storage of intermediate results in temporary variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know a whole lot about template metaprogramming in C++, but I do know that it is one approach used in automatic differentiation and in interval arithmetic, which is how I got into a discussion about expression templates. Given both the potential advantages in performance and the potential disadvantages in maintenance (if that's even the right word), when should I use C++ expression templates in computational science, and when should I avoid them?&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-09-29T12:53:52.560" Title="When should I use C++ expression templates in computational science, and when should I *not* use them?" Tags="|performance|c++|automatic-differentiation|interval-arithmetic|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1635" PostTypeId="2" ParentId="1634" CreationDate="2012-03-15T01:31:49.383" Score="17" Body="&lt;p&gt;My problem with expression templates is that they are a very leaky abstraction. You spend a lot of work writing very complicated code to do a simple task with nicer syntax. But if you want to change the algorithm, you have to mess with the dirty code and if you slip up with types or syntax, you get completely unintelligible error messages. If your application maps perfectly to a library based on expression templates, then it might be worth considering, but if you aren't sure, I would recommend just writing normal code. Sure, the high level code is less pretty, but you can just do what needs to be done. As a benefit, compilation time and binary sizes will go way down and you won't have to cope with huge variance in performance due to compiler and compilation flag choice.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-15T01:31:49.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1636" PostTypeId="2" ParentId="1634" CreationDate="2012-03-15T08:13:14.757" Score="5" Body="&lt;p&gt;I have experimented a little bit with ET's a long time ago when, as you mentioned, compilers were still struggling with them. I've used the &lt;a href=&quot;http://www.oonumerics.org/blitz/&quot;&gt;blitz&lt;/a&gt; library for linear algebra in some code of mine. The problem then was getting the good compiler and as I'm not a perfect C++ programmer, interpreting the compiler error messages. The latter was simply unmanageable. The compiler would, on average, generate about 1000 lines of error messages. No way I was able to quickly find my programming error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find more information on the &lt;a href=&quot;http://www.oonumerics.org&quot;&gt;oonumerics&lt;/a&gt; webpage (there are the proceedings of two ET workshops).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would stay far away from them....&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-03-15T08:13:14.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1637" PostTypeId="1" CreationDate="2012-03-15T09:02:21.387" Score="7" ViewCount="920" Body="&lt;p&gt;I am interested in the adjoint method for shape optimization problems.&#xA;However, I couldn't find a helpful introduction. So I come here and look forward to some enlightening advices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you direct me to any textbooks, published papers or free online source course materials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And to clearify several things:&#xA;I am familiar with FEM, matrix computation, calculus of variation, etc.; &#xA;I only want to learn the adjoint method for shape optimization in solid mechanics, specifically on continuous level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though the question does not perfectly fit this website, but it seems to be the best choice for a shot amongst stack-websites.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Greetings.&lt;/p&gt;&#xA;" OwnerUserId="1152" LastActivityDate="2020-02-16T19:28:39.877" Title="Adjoint method for optimization problem" Tags="|optimization|reference-request|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1638" PostTypeId="2" ParentId="1637" CreationDate="2012-03-15T11:35:49.733" Score="4" Body="&lt;p&gt;There is a nice chapter on adjoint based optimization in the book &quot;The finite element method for engineers&quot; by Kenneth Huebner. Unlike many other (mostly unhelpful) references that focus purely on theory, it shows the implementation (examples with all math worked out) but only for static solid mechanics problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For quasi-static problems I have no idea (maybe others here can provide pointers). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For dynamic (wave propagation) problems try the paper &lt;a href=&quot;http://www.giseis.alaska.edu/input/carl/research/pubs/TrompTapeLiu05.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2012-03-15T11:35:49.733" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1639" PostTypeId="1" CreationDate="2012-03-15T13:15:04.567" Score="44" ViewCount="20746" Body="&lt;p&gt;Which software provides a good workflow from simple plotting of a few datapoints up to the creation of publication level graphics with detailed styles, mathematical typesetting and &quot;professional quality&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a bit related to the question of David (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/346/what-attributes-make-a-figure-professional-quality&quot;&gt;What attributes make a figure professional quality?&lt;/a&gt;) but the focus is not on the attributes but on the software or general the workflow to get there. I have superficial experience with a number of programs, Gnuplot, Origin, Matplotlib, TikZ/PGFplot, Qtiplot but doing data analysis and nice figures at the same time seems rather hard to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some software that allows this or should I just dig deeper in one of the packages?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: My current workflow is a mix of different components, which more or less work together but in total it is not really efficient and I think this is typical for a number of scientists at an university lab. Typically it is a chain starting from the experiment to the publication like this: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get experimental data (usually in ASCII form, but with different layout, e.g. headers, comments, number of columns)&lt;/li&gt;&#xA;&lt;li&gt;Quick plot of the data to check whether nothing went wrong in Origin, Gnuplot or arcane plot program written 20 years ago.&lt;/li&gt;&#xA;&lt;li&gt;More detailed analysis of the data: subtracting background contributions, analysing dependencies and correlations, fitting with theoretical models. Many scientists use Origin for this task, some Matlab and Python/Scipy/Numpy usage is increasing.&lt;/li&gt;&#xA;&lt;li&gt;Creating professional figures, this involves adjusting to journal guidelines, mathematical typesetting and general editing. At the moment I use Origin for this but it has several drawbacks (just try to get a linewidth of exactly 0.5pt, it is not possible). For combining/polishing figures I mainly use Adobe Illustrator, as it can handle im-/export of PDF documents nicely but I would prefer not having to go through two steps for each diagram. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I added an example of how it might look like in the end (as this has been created mostly by hand changing anything is painful and anything that provides an interface for example to set the linewidth for all elements would be nice):&lt;img src=&quot;https://i.stack.imgur.com/uTQy3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1153" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-06-13T14:53:57.837" Title="Which software and workflow is recommend for publication of scientific data and graphs?" Tags="|visualization|data-management|" AnswerCount="14" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1640" PostTypeId="2" ParentId="1639" CreationDate="2012-03-15T13:26:46.617" Score="34" Body="&lt;p&gt;If you have some experience with Python (or even not), I would recommend using the Python scientific software that is available (&lt;a href=&quot;http://www.scipy.org&quot; rel=&quot;noreferrer&quot;&gt;SciPy&lt;/a&gt;,&lt;a href=&quot;http://pandas.pydata.org/&quot; rel=&quot;noreferrer&quot;&gt;Pandas&lt;/a&gt;),...) together with &lt;a href=&quot;http://matplotlib.sourceforge.net/index.html&quot; rel=&quot;noreferrer&quot;&gt;Matplotlib&lt;/a&gt;. Being a programming environment, you have full control over your data flows, data manipulations and plotting. You can also use the &quot;full applications&quot; &lt;a href=&quot;http://code.enthought.com/projects/mayavi/&quot; rel=&quot;noreferrer&quot;&gt;Mayavi2&lt;/a&gt; or  &lt;a href=&quot;http://veusz.github.io/&quot; rel=&quot;noreferrer&quot;&gt;Veusz&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="1327" LastEditDate="2017-06-13T14:53:57.837" LastActivityDate="2017-06-13T14:53:57.837" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1641" PostTypeId="2" ParentId="1639" CreationDate="2012-03-15T14:39:24.077" Score="5" Body="&lt;p&gt;If you already know TikZ, you might be interested in &lt;a href=&quot;http://www.r-project.org/&quot;&gt;R&lt;/a&gt; and the &lt;a href=&quot;http://cran.r-project.org/web/packages/tikzDevice/index.html&quot;&gt;tikzDevice&lt;/a&gt;; here is an example: &lt;a href=&quot;http://www.texample.net/tikz/examples/tikzdevice-demo/&quot;&gt;tikzDevice - TikZ output from R&lt;/a&gt;, here is another example: &lt;a href=&quot;http://thetarzan.wordpress.com/2011/06/25/tikz-diagrams-with-r-tikzdevice/&quot;&gt;TikZ diagrams with R: tikzDevice&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="284" LastActivityDate="2012-03-15T14:39:24.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1643" PostTypeId="2" ParentId="1639" CreationDate="2012-03-15T15:37:16.377" Score="11" Body="&lt;p&gt;Another software package to consider is &lt;a href=&quot;http://asymptote.sourceforge.net/&quot;&gt;Asymptote&lt;/a&gt;.  Asymptote is actually a programming language (with a C++ like syntax) that produces vector based output.  One nice feature is that text is rendered with LaTeX (including math equations), so the text in your figures is consistent with your document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a Python wrapper to Asymptote called &lt;a href=&quot;http://memmett.github.com/PyAsy/&quot;&gt;PyAsy&lt;/a&gt; so that I could pass NumPy arrays to the Asymptote engine and do some basic plotting.  It is fairly rough and not feature complete, but it might be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, the output quality of Asymptote is really nice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Further to the above, I have found &lt;a href=&quot;http://fabfile.org&quot;&gt;Fabric&lt;/a&gt; very helpful for launching remote jobs, syncing directories and building binaries, fetching run data and/or launching remote analysis scripts.  It is a fairly lightweight Python library that makes running remote commands a bit easier (and scriptable).&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="486" LastEditDate="2012-03-16T22:48:22.487" LastActivityDate="2012-03-16T22:48:22.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1644" PostTypeId="1" AcceptedAnswerId="1650" CreationDate="2012-03-15T16:02:06.077" Score="3" ViewCount="199" Body="&lt;p&gt;what other recent sources of literature on this topic would you recommend? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where I'm starting from: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.jstor.org/stable/10.2307/2158391&quot; rel=&quot;nofollow&quot;&gt;Leveque's article: HRIC scheme&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the related articles seem to be a bit dated (some up to 20 years). Can anyone with experience with such higher order schemes suggest a direction in which I could search?&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-03-15T21:28:21.223" Title="High-resolution finite volume schemes for two phase flow (fields with jumps) literature sources" Tags="|reference-request|finite-volume|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1645" PostTypeId="2" ParentId="1621" CreationDate="2012-03-15T17:29:54.753" Score="3" Body="&lt;p&gt;There's also pipelining (or 'functional parallelism'), where different nodes run different parts of the process. This is generally only useful for problems that are embarrassingly parallel but where each part is still a pretty hefty problem (an example commonly given is astronomical image processing, where you have lots of images and several steps to perform on each one). It also requires careful load balancing.&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2012-03-15T17:29:54.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1646" PostTypeId="2" ParentId="1639" CreationDate="2012-03-15T17:43:12.537" Score="9" Body="&lt;p&gt;I generally use &lt;a href=&quot;http://www.gnuplot.info/&quot;&gt;gnuplot&lt;/a&gt;. It can be hard to learn, but it produces very clean plots. It can also be set to produce plots in &lt;a href=&quot;http://t16web.lanl.gov/Kawano/gnuplot/postproc-e.html&quot;&gt;encapsulated postscript format&lt;/a&gt; (.eps) which allows for easy embedding into documents written in $\LaTeX$.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-03-15T17:43:12.537" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1647" PostTypeId="2" ParentId="1639" CreationDate="2012-03-15T17:55:23.613" Score="4" Body="&lt;p&gt;I find Tecplot to be quite powerful when it comes to data visualization and data manipulation.&lt;/p&gt;&#xA;" OwnerUserId="1156" LastActivityDate="2012-03-15T17:55:23.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1648" PostTypeId="1" AcceptedAnswerId="1744" CreationDate="2012-03-15T19:52:17.057" Score="6" ViewCount="522" Body="&lt;p&gt;I am trying to write computer code which will find the energy and density function for a atom with Z protons and N electrons. I am working in 1D for simplicity and would like to make the overall code as simple as possible (I am using Hartree units aswell). I am using also using the Hartree approximation. This is my overall algorithm:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Finds hydrogen like wave-functions (i.e. no electron-election interaction)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Works out density function via&#xA;$n(z) = \sum_i \left | \phi_i(z)  \right |^2$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Works out Hartree potential&#xA;$ V_H(r) = \int_0^\infty \frac{n(\vec{s})}{\left | \vec{r}-\vec{s} \right |} d\vec{s} $&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Uses Kohn-Sham equation to find new individual wavefuncitons&#xA;$\left (-\tfrac{1}{2}\bigtriangledown^2 + \frac{Z}{\left |r  \right |} + V_H(\vec{r}) + V_{EX}(\vec{r})  \right )\phi_i = \varepsilon_i \phi_i$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Go back to step 2. and repeat&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am having a number of difficulties and would be very grateful for some help with any of my questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly to compute the Hartree potential I need to turn it into a 1D problem, so we assume even spherical distribution of charge:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ V_H(r) = \int_0^\infty 4\pi s^2 \frac{ n(s)}{\left |r-s\right |} ds $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However this blows up when r = s! I have been looking into Gauss's law but had no luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be great.&lt;/p&gt;&#xA;" OwnerUserId="1158" LastEditorUserId="123" LastEditDate="2012-03-16T19:52:27.170" LastActivityDate="2012-03-23T18:18:58.667" Title="Implementing simple atom model using density functional theory (DFT)" Tags="|computational-chemistry|hartree-fock|density-functional-theory|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1649" PostTypeId="2" ParentId="1639" CreationDate="2012-03-15T21:21:28.310" Score="6" Body="&lt;p&gt;Dig deeper into PGF plots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) can do 3d diagrams: python-matplotlib can't do them, gnuplot is ugly in comparison&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) has consistent fonts in &lt;strong&gt;size&lt;/strong&gt; and &lt;strong&gt;family&lt;/strong&gt; with the base document )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) can be templated for specific colorschemes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is nothing else like it on the market. :)&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-03-15T21:21:28.310" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1650" PostTypeId="2" ParentId="1644" CreationDate="2012-03-15T21:28:21.223" Score="4" Body="&lt;p&gt;I take it from your interest in the article that you are interested in incompressible two-phase flows which has quite a bit of literature on it.  One of the more common approaches and arguably most straight forward is to employ a level-set method.  I am not sure what the best paper on this is currently to suggest unfortunately.  Also, if you are interested in allowing the two fluids to mix you may try searching for volume-of-fluid approaches.  If you are actually interested in compressible two-phase flow (such as the Euler gas dynamics equations) then I can suggest some relatively recent papers on the subject:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Saurel, R., Franquet, E., Daniel, E. &amp;amp; Metayer, O. L. A relaxation-projection method for compressible flows. Part I: The numerical equation of state for the Euler equations. Journal of Computational Physics 223, 822–845 (2007).&lt;/li&gt;&#xA;&lt;li&gt;Schwendeman, D., Wahle, C. &amp;amp; Kapila, A. The Riemann problem and a high-resolution Godunov method for a model of compressible two-phase flow. Journal of Computational Physics 212, 490–526 (2006).&lt;/li&gt;&#xA;&lt;li&gt;Abgrall, R. &amp;amp; Karni, S. Computations of Compressible Multifluids. Journal of Computational Physics 169, 594–623 (2001).&lt;/li&gt;&#xA;&lt;li&gt;Shyue, K.-M. A fluid-mixture type algorithm for barotropic two-fluid flow problems. Journal of Computational Physics 200, 718–748 (2004).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="567" LastActivityDate="2012-03-15T21:28:21.223" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1651" PostTypeId="1" AcceptedAnswerId="1653" CreationDate="2012-03-16T02:13:29.917" Score="10" ViewCount="282" Body="&lt;p&gt;I know that we can use mathematical analysis techniques to prove if an IVP or BVP has a solution, is unique, and depends continuously on the boundary / initial values.  For some PDE's, particularly non-linear pde's, it's very difficult, if not impossible, to prove well-posedness.  Is there any kind of numerical technique to verify if a problem is well-posed or not?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-16T12:36:49.970" LastActivityDate="2012-03-16T12:51:19.683" Title="Can a numerical scheme be used to determine the well-posedness of an initial or boundary value problem?" Tags="|pde|well-posedness|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1652" PostTypeId="2" ParentId="74" CreationDate="2012-03-16T09:10:56.217" Score="3" Body="&lt;p&gt;Maple 15 cannot do it. It has no property &quot;Orthogonal&quot; for matrices (although it has Symmetric and PositiveDefinite). &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-03-16T09:10:56.217" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1653" PostTypeId="2" ParentId="1651" CreationDate="2012-03-16T12:51:19.683" Score="10" Body="&lt;p&gt;In general, no. A numerical solution can sometimes be used as a rough measure to indicate whether boundary conditions are sufficient, to identify &quot;floating&quot; domains for example, but there are many cases in which discrete solutions give you downright misleading information about the continuum problem.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Advection-diffusion requires a boundary condition on all boundaries, but discrete systems can use &lt;em&gt;no boundary condition&lt;/em&gt; at the outflow (not a homogeneous Neumann condition, I really mean no boundary condition). Not only that, it's more accurate than the discrete representation of the continuum boundary condition. See &lt;a href=&quot;http://deepblue.lib.umich.edu/bitstream/2027.42/50207/1/1650140506_ftp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Papanastasiou, Malamataris, and Ellwood 1992&lt;/a&gt; and &lt;a href=&quot;http://discovery.dundee.ac.uk/bitstream/handle/10588/1383/NA163_DFG.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Griffiths 1997&lt;/a&gt; for details. A similar boundary condition is also important for slip on curved surfaces, see &lt;a href=&quot;http://www.cats.rwth-aachen.de:8080/publications/Behr2003a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Behr 2004&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &quot;carbuncle phenomenon&quot; afflicts certain methods for compressible flow. It is not very well understood, but seemingly robust numerical schemes may converge to spurious solutions. An example from &lt;a href=&quot;http://oatao.univ-toulouse.fr/2353/1/Gressier_2353.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robinet et al. 2000&lt;/a&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/rNexd.png&quot; alt=&quot;Carbuncle phenomenon&quot;&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Spurious solutions to incompressible Navier-Stokes, within a laminar regime. A simple lid-driven cavity example is given in &lt;a href=&quot;http://dx.doi.org/10.1016/0021-9991(83)90119-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schreiber and Keller 1983&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Systems of hyperbolic conservation laws with non-physical relative size of numerical dissipation. Some numerical dissipation is always required, but otherwise robust (e.g. Godunov) methods can systematically converge to incorrect results if numerical dissipation ends up being non-physical. A simple example is given in &lt;a href=&quot;http://arxiv.org/abs/1109.1446&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mishra and Spinolo 2011&lt;/a&gt; where the standard Godunov method converges to an incorrect result for 1D linearized shallow water. This presents itself in a deeper form in large eddy simulation. The eddy viscosity is a physical manifestation of subgrid scales, but if the (unavoidable) numerical dissipation is larger than the physical dissipation, the simulation can converge to systematically incorrect results. In practice, the subgrid closures for eddy viscosity are very important. This is a matter of taking a singular limit along the correct (physical) path.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Locking effects in elasticity or checkerboard modes in incompressible flow. These are due to choosing an unstable approximation space and are now very well-understood, at least for linear problems, but relying on a numerical solution to deduce well-posedness might lead you to conclude that the incompressible limit was ill-posed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-16T12:51:19.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1654" PostTypeId="2" ParentId="1639" CreationDate="2012-03-16T14:35:36.957" Score="16" Body="&lt;p&gt;Based on what you're saying, a general purpose scripting language is probably your best bet, as long as it has some sort of graphing capability that you can access (whether built-in or imported).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that vein, MATLAB will work, although you'll have to toy with line widths, symbols, and axes for presentation-quality plots. Given your criteria, I'd say the big weakness of MATLAB is detailed mathematical typesetting; MATLAB can use some TeX labels, but there are some LaTeX commands that it can't handle, so I remember having to go back and cut-and-paste some LaTeX labels from a PDF in Adobe Illustrator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python + NumPy will work for data imports, because &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html&quot;&gt;&lt;code&gt;numpy.loadtxt&lt;/code&gt;&lt;/a&gt; makes importing text data painless. At that point, you can choose between matplotlib and Gnuplot (which has a Python interface via &lt;a href=&quot;http://gnuplot-py.sourceforge.net/&quot;&gt;Gnuplot.py&lt;/a&gt;; no active development on the interface since 2008, but then again, does Gnuplot change all that much anymore, even though it keeps adding releases?). MatthewEmmett's PyAsy wrapper could also work, depending on what you'd like to do. Matplotlib has great mathematical typesetting (as noted by Barron), and data manipulations can be done using NumPy, SciPy, or whatever other Python package you want to throw at your data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I had with gnuplot on its own is that it's really built for plotting only. It wasn't apparent to me how to manipulate data within the gnuplot scripting language. I essentially did all of my data manipulation in advance before I output it to text because I couldn't figure out how to do it otherwise. You can use gnuplot to evaluate formulas, and do&#xA;some manipulations, but for me, Python has a lot more natural syntax, and is of greater general utility than gnuplot. I don't want to have to learn another language just to do one thing unless it's a really natural, easy-to-learn language. (Or unless I'm forced to.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tecplot 360 has a similar sort of issue. Tecplot 360 produces amazing plots, especially in 3D. There are things you can do with Tecplot 360 using a relatively easy-to-learn GUI that you'd probably be hard-pressed to do in another plotting package. The last time I used Tecplot was 6 or 7 years ago; apparently, they've added a lot of functionality since. Data manipulation can be done through indexing, slicing, or establishing zones. Derived quantities can be calculated using equations. You can also take the Fourier Transform of your data, perform interpolation (or kriging), and do a number of other things. It's not clear how to do mathematical typesetting. I guess you could hack HTML output from Tecplot and add MathJax or MathML? The disadvantages, in my mind, regarding Tecplot are that it's not free (which means that as a tool, you may not be able to take it with you from job to job), the iffy mathematical typesetting, and that scripting complex data manipulations requires learning Tecplot's macro language. The main advantage is that its 3D plotting capability outstrips matplotlib and MATLAB by a long shot. It's possible to batch plot data using Tecplot's macro/scripting language, and to call external Python scripts (with some limited functionality). Also, similar to MATLAB, there's a GUI you can fall back on if you don't want to learn Tecplot scripting. (Tecplot's GUI is a lot more full-featured than MATLAB's.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LaTeX-based tools like TikZ and PGFPlots don't seem to be made for your use case. The weak point here is the data manipulation; TikZ and PGFPlots are great LaTeX tools. I wish I knew how to use them better. Since they're LaTeX, I'm guessing the workflow is cutting-and-pasting the necessary data into LaTeX and plotting it. It's possible to execute programs inside of LaTeX, but I don't see how that capability would necessarily help you, given that, for a presentation or document, the use case you're describing suggests that you're probably only going to keep the finished product. Maybe you're okay with that workflow; both tools have a reputation for being well-engineered and producing high-quality figures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, Adobe Illustrator is a fine tool for touching up plots, like you said; the deficiencies are also, like you said, lack of scriptability or repeatability, but nothing beats a graphics editing program if you want to make small tweaks. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-03-16T14:35:36.957" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1655" PostTypeId="2" ParentId="1621" CreationDate="2012-03-16T17:47:37.450" Score="6" Body="&lt;p&gt;For parallel models I see them as communication-based or work decomposition-based:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Communication-based models&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Shared memory&lt;/em&gt;: implicit communication via sharing memory.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Message passing&lt;/em&gt;: explicit communication by passing messages between parallel programs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Work partitioning models&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Domain decomposition&lt;/em&gt;. Problem domain is divided into sub-domains, problem is solved by solving all the sub-problems.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Task-based decomposition&lt;/em&gt;: compute work is organised in the form of tasks (it could also be objects). Tasks could be homogeneous or each task could perform different work. Examples for this can be seen in OpenMP, Intel thread building blocks, starPU/Quark.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Implicit dependencies: constructs are used to implicitly represent relations between tasks, think of OpenMP and its &lt;em&gt;#pragma omp parallel for&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Explicit dependencies: relations between task are the core of the representation. A good discussion is given &lt;a href=&quot;http://128.148.6.21/materials/Slides/tw-12-1/Making_Dataflow_Programming_Ubiquitous_for_Scientific_Computing_%5d_Hatem_Ltaief,_King_Abdullah_University_of_Science_&amp;amp;_Technology.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Paradigms examples&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Single program multiple data (SPMD): Domain decomposition.&lt;/li&gt;&#xA;&lt;li&gt;Parallel functional programming: implicit task model.&lt;/li&gt;&#xA;&lt;li&gt;Master-slave paradigm: explicit task-based model.&lt;/li&gt;&#xA;&lt;li&gt;Divide and conquer: domain decomposition.&lt;/li&gt;&#xA;&lt;li&gt;GPU computing: most commonly domain decomposition.&lt;/li&gt;&#xA;&lt;li&gt;Work pool and/or parallel queues: task-based model or domain decomposition.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With the heterogeneity and hierarchical structure of parallel computing architectures currently available, programs tend to mix models and paradigms to better match the architecture. Consider for instance message passing between nodes while using shared memory intra-node, that would match with a divide and conquer approach to distribute work across the nodes while locally each sub-domain could be solved using SPMD over multiple cores, a GPU, or both.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="1070" LastEditDate="2012-03-19T06:07:01.727" LastActivityDate="2012-03-19T06:07:01.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1656" PostTypeId="2" ParentId="74" CreationDate="2012-03-16T18:15:17.713" Score="4" Body="&lt;p&gt;Some symbolic matrix computations (e.g., block matrix completion) can be done with the package NCAlgebra &lt;a href=&quot;http://www.math.ucsd.edu/~ncalg/&quot; rel=&quot;nofollow&quot;&gt;http://www.math.ucsd.edu/~ncalg/&lt;/a&gt; (which runs under mathematica). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bergman &lt;a href=&quot;http://servus.math.su.se/bergman/&quot; rel=&quot;nofollow&quot;&gt;http://servus.math.su.se/bergman/&lt;/a&gt; is a package in Lisp with similar capabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some relevant papers:&lt;BR&gt;&#xA;&lt;a href=&quot;http://math.ucsd.edu/~helton/osiris/COMPALG2000/ohRevisIJC.pdf&quot; rel=&quot;nofollow&quot;&gt;http://math.ucsd.edu/~helton/osiris/COMPALG2000/ohRevisIJC.pdf&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://math.ucsd.edu/~thesis/thesis/dkronewitter/dkronewitter.pdf&quot; rel=&quot;nofollow&quot;&gt;http://math.ucsd.edu/~thesis/thesis/dkronewitter/dkronewitter.pdf&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://www.tandfonline.com/doi/abs/10.1080/00207170600882346&quot; rel=&quot;nofollow&quot;&gt;http://www.tandfonline.com/doi/abs/10.1080/00207170600882346&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-03-16T19:10:48.627" LastActivityDate="2012-03-16T19:10:48.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1657" PostTypeId="2" ParentId="1632" CreationDate="2012-03-16T18:22:35.300" Score="9" Body="&lt;p&gt;This is far from generalized SVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If B is a positive matrix, you can use my package BIRSVD &#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/software/birsvd/&quot;&gt;http://www.mat.univie.ac.at/~neum/software/birsvd/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/software/birsvd/svd_incomplete_data.pdf&quot;&gt;http://www.mat.univie.ac.at/~neum/software/birsvd/svd_incomplete_data.pdf&lt;/a&gt; &#xA;describing the method there also gives references that you may consider to do a literature search.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-16T18:22:35.300" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1658" PostTypeId="1" AcceptedAnswerId="1659" CreationDate="2012-03-16T18:40:59.163" Score="33" ViewCount="34693" Body="&lt;p&gt;Is there a way, using some established Python package (e.g. SciPy) to define my own &lt;strong&gt;probability density function&lt;/strong&gt; (without any prior data, just $f(x) = a x + b$), so I can then make calculations with it (such as obtaining the variance of the continuous random variable)? Of course I could take, say, SymPy or Sage, create a symbolic function and do the operations, but I'm wondering whether instead of doing all this work myself I can make use of an already-implemented package.&lt;/p&gt;&#xA;" OwnerUserId="782" LastEditorUserId="17530" LastEditDate="2016-02-24T20:22:01.517" LastActivityDate="2017-07-31T17:44:37.177" Title="Define custom probability density function in Python" Tags="|python|statistics|probability|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1659" PostTypeId="2" ParentId="1658" CreationDate="2012-03-16T20:01:26.597" Score="35" Body="&lt;p&gt;You have to subclass the &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.rv_continuous.html&quot; rel=&quot;noreferrer&quot;&gt;rv_continuous&lt;/a&gt; class in scipy.stats&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.stats as st&#xA;&#xA;class my_pdf(st.rv_continuous):&#xA;    def _pdf(self,x):&#xA;        return 3*x**2  # Normalized over its range, in this case [0,1]&#xA;&#xA;my_cv = my_pdf(a=0, b=1, name='my_pdf')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;now my_cv is a continuous random variable with the given PDF and range [0,1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in this example &lt;code&gt;my_pdf&lt;/code&gt; and &lt;code&gt;my_cv&lt;/code&gt; are arbitrary names (that could have been anything), but &lt;code&gt;_pdf&lt;/code&gt; is &lt;em&gt;not&lt;/em&gt; arbitrary; it and &lt;code&gt;_cdf&lt;/code&gt; are methods in &lt;code&gt;st.rv_continuous&lt;/code&gt; one of which must be overwritten in order for the subclassing to work.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="17530" LastEditDate="2016-02-25T08:13:52.200" LastActivityDate="2016-02-25T08:13:52.200" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1660" PostTypeId="1" AcceptedAnswerId="1661" CreationDate="2012-03-16T21:19:00.370" Score="2" ViewCount="100" Body="&lt;p&gt;I've been trying to use a PetscBag to read in a number of options I want to pass to a program I'm writing. Before I came across the documentation for PetscBag, I was planning on using a struct and a function that would take the struct and use &lt;code&gt;PetscOptionsGetSometype()&lt;/code&gt; to set all the values in the struct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I figured that PetscBag would save some code, but seeing as each variable in the PetscBag has to use &lt;code&gt;PetscOptionsRegisterSometype()&lt;/code&gt;, it looks to be just as much code to use the PetscBag as not to use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I don't know that much about how the struct I was going to use would work when run with MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should a PetscBag be used for? What advantages do they have over vanilla structs?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-03-17T00:55:03.530" Title="What should a PetscBag be used for?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1661" PostTypeId="2" ParentId="1660" CreationDate="2012-03-17T00:55:03.530" Score="3" Body="&lt;p&gt;Items in a PetscBag are self-documenting though the associated help strings (seen when you run with &lt;code&gt;-help&lt;/code&gt; and with &lt;code&gt;PetscBagView()&lt;/code&gt;, can be overridden at run-time through the options database (&lt;code&gt;-prefix_fieldname 4.2e7&lt;/code&gt;), and are serializable in ASCII and portable binary. With a normal struct, you have to write code to take input, supply help, and serialize (including dealing with backward compatibility as you add new fields).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a reasonable thing to use for a collection of problem-specific data, e.g. specification of a physical model configuration. It's not a good place to put &quot;problem-sized data&quot; such as an unstructured mesh or high-dimensional observational data.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-17T00:55:03.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1662" PostTypeId="2" ParentId="74" CreationDate="2012-03-17T03:12:51.780" Score="4" Body="&lt;p&gt;I think most &lt;code&gt;CAS&lt;/code&gt; systems can show this for &lt;code&gt;2x2&lt;/code&gt; and &lt;code&gt;3x3&lt;/code&gt; matrices given a symbolic orthonormal $\mathbf B$ construct, such as rotation matrices. In the end, you will have to decompose the result to figure out if it is positive definite or not. Symmetry is easier to show.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question then becomes, what about a &lt;code&gt;N&lt;/code&gt; dimensional matrix? Maybe you can come up with an inductive scheme where for &lt;code&gt;N-1 x N-1&lt;/code&gt; is assumed to be true and then construct a new block matrix with overall size &lt;code&gt;N x N&lt;/code&gt; to prove that is positive definite and symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the final question, of which software is better suited for the task (if any), my experience has been with &lt;code&gt;MATLAB/MuPad&lt;/code&gt; and &lt;code&gt;Derive&lt;/code&gt; (still use it) and neither of them handle vectors and matrices very well. &lt;code&gt;MATLAB&lt;/code&gt; breaks everything down into components, and &lt;code&gt;Derive&lt;/code&gt; can declare &lt;em&gt;&lt;code&gt;Non-scalars&lt;/code&gt;&lt;/em&gt; but it does not apply any simplification rules to them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this posting is going to provide some more insight into this kind of &quot;hole&quot; and how to fill it. For me, I want some software that can help me simplify expressions with multiple dot and cross products of vectors, together with rotation matrices use well known identities such as: $\vec a \times (\vec b \times \vec c) = (\vec a \cdot \vec b)\vec c - (\vec a \cdot \vec c) \vec b$&lt;/p&gt;&#xA;" OwnerUserId="1030" LastActivityDate="2012-03-17T03:12:51.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1663" PostTypeId="2" ParentId="1639" CreationDate="2012-03-17T09:07:08.517" Score="8" Body="&lt;p&gt;Let me write a short review of Mathematica's graphics functionality, since that is what I'm familiar with.&lt;/p&gt;&#xA;&lt;h3&gt;What can it do?&lt;/h3&gt;&#xA;&lt;p&gt;To see some example, check out the &lt;a href=&quot;https://www.wolfram.com/language/gallery/&quot; rel=&quot;noreferrer&quot;&gt;Mathematica code gallery&lt;/a&gt; on the Wolfram site.  It's not specifically about visualization, but almost all examples include some plotting.&lt;/p&gt;&#xA;&lt;p&gt;You can also browse the &lt;a href=&quot;https://mathematica.stackexchange.com/questions/tagged/plotting&quot;&gt;plotting&lt;/a&gt; (basic) and  &lt;a href=&quot;https://mathematica.stackexchange.com/questions/tagged/graphics&quot;&gt;graphics&lt;/a&gt; (more advanced) tags of &lt;a href=&quot;https://mathematica.stackexchange.com/&quot;&gt;Mathematica.SE&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is possible to create your example image in a fully automated way (no manual postprocessing needed).&lt;/p&gt;&#xA;&lt;h3&gt;Strengths&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;All graphics are specified in a declarative way&lt;/strong&gt; (like SVG---you don't tell the system to &lt;em&gt;draw&lt;/em&gt; something, you just &lt;em&gt;list&lt;/em&gt; the vector graphics objects).  Graphics are the same kind of Mathematica expressions as anything else in the system, they are just displayed in a special way. This means that graphics can be easily transformed and processed after they have been created---this applies to graphics made by plotting functions as well.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The quick and simple way to create graphics is using &lt;a href=&quot;http://reference.wolfram.com/mathematica/howto/CreatePlots.html&quot; rel=&quot;noreferrer&quot;&gt;high level plotting functions&lt;/a&gt;, and setting lots of options to customize their output.  If there is no function for the precise plot that you need, you can assemble the graphics from &lt;a href=&quot;http://reference.wolfram.com/mathematica/guide/SymbolicGraphicsLanguage.html&quot; rel=&quot;noreferrer&quot;&gt;graphics primitives&lt;/a&gt; (which is also &lt;a href=&quot;https://stackoverflow.com/a/9742617/695132&quot;&gt;pretty easy&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;http://reference.wolfram.com/language/guide/FunctionVisualization.html&quot; rel=&quot;noreferrer&quot;&gt;Mathematical function plotting functionality&lt;/a&gt; is excellent.&lt;/strong&gt; as far as I'm aware it's class leading. Mathematica samples functions &lt;a href=&quot;https://mathematica.stackexchange.com/q/29346/12&quot;&gt;adaptively&lt;/a&gt; (both in 2D and 3D) so a smooth plot can be generated quickly and automatically.  Singularities, discontinuities, etc. are detected symbolically by plotting functions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;You can produce figures for a specific print size,&lt;/strong&gt; and make sure that they fit well with the surrounding text in a publication (e.g. font size and styles match). In 2D, there are two kinds of coordinates: plot coordinates that correspond to your data and absolute coordinates specified in printer's points.  The former scales when resizing the graphic, the latter doesn't.  Text uses absolute coordinates by default which means that 8-point type will be 8 point regardless of whether the same figure is exported at 7 cm or 14 cm size.  This makes it possible to have font sizes consistent with the text in the publication.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Builtin typesetting for mathematical formulae.&lt;/strong&gt;  The &lt;a href=&quot;https://github.com/szhorvat/MaTeX&quot; rel=&quot;noreferrer&quot;&gt;MaTeX&lt;/a&gt; third-party package provides LaTeX integration for excellent quality formula typesetting.  (Disclosure: I'm the author.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can do all the data processing in the same system.  Generally, with a bit of work, the full figure creation process can be automated (even some &lt;a href=&quot;https://mathematica.stackexchange.com/questions/2105/arranging-connector-lines&quot;&gt;advanced aspects&lt;/a&gt;), avoiding the need for manual post processing in software like Adobe Illustrator.  This is important when at the last minute you discover some minor problem and need to re-generate the figure.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Weaknesses&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Having figures with several subplots &lt;em&gt;and&lt;/em&gt; having some precise alignment between the coordinate systems of the subplots can be difficult.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;http://scidraw.nd.edu/&quot; rel=&quot;noreferrer&quot;&gt;SciDraw&lt;/a&gt; (third party) package fixes this and gives detailed control over every aspect of the figure.  The tradeoff of this package is that it takes more effort to set up figures.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;3D graphics don't export well to vector formats.  Exporting a bitmap version is fine though.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your institution does not already have a license, it costs money.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="605" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-11-03T17:11:12.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1664" PostTypeId="2" ParentId="1556" CreationDate="2012-03-17T13:56:02.930" Score="2" Body="&lt;p&gt;One strategy is to look for modern algorithms for related problems, like &lt;a href=&quot;http://en.wikipedia.org/wiki/Collision_detection&quot; rel=&quot;nofollow&quot;&gt;collision detection&lt;/a&gt;, etc.  Often there are good strategies in slightly different application if you have a particular problem you're trying to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for implementation, you could check out the &lt;a href=&quot;http://www.boost.org/doc/libs/1_49_0/libs/polygon/doc/index.htm&quot; rel=&quot;nofollow&quot;&gt;Boost Polygon Library&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple general books for Computational Geometry that are on my shelf are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387961313&quot; rel=&quot;nofollow&quot;&gt;Computational Geometry: An Introduction&lt;/a&gt; by Franco Preparata and Michael Shamos is yet another good introductory book on computational geometry algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0133363635&quot; rel=&quot;nofollow&quot;&gt;Computational Geometry: An Introduction Through Randomized Algorithms&lt;/a&gt; by Ketan Mulmuley  is an excellently constructed book good algorithmic coverage of a wide variety of algorithms for geometric problems; all done through randomized methods.&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2012-03-17T13:56:02.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1665" PostTypeId="1" AcceptedAnswerId="1667" CreationDate="2012-03-17T15:17:39.690" Score="5" ViewCount="1952" Body="&lt;p&gt;For instance, in MUMPS there is option to scale matrix s.t. all rows/columns have the same norm.&#xA;This claims to decrease condition number and improve numerical properties of the matrix: &lt;a href=&quot;ftp://cuter.rl.ac.uk/pub/reports/adruRAL2008013.pdf&quot; rel=&quot;nofollow&quot;&gt;ftp://cuter.rl.ac.uk/pub/reports/adruRAL2008013.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody explain whether it can be used as a preconditioner and how efficient would it be?&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2012-03-18T14:54:08.510" LastActivityDate="2012-03-18T14:54:08.510" Title="How does matrix scaling influence linear solvers?" Tags="|linear-solver|preconditioning|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1666" PostTypeId="1" CreationDate="2012-03-17T17:22:32.077" Score="3" ViewCount="162" Body="&lt;p&gt;A bit of a strange question - but I am developing an application where speed is critical, not memory - I have the ability to blow up to 3 TB on the data storage that this application will use and I am wanting to use a space time trade off to increase processing speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My reasoning goes as such - A compression algorithim utilizies a specific function to decrease a file's size, at the cost of the algorithmic run time to uncompress the specific file - I should be able to do the reverse shouldnt I? Increase the files size to reduce the amount of time taken to process the file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you suggest any simple ways to do this? (Inverse probabilistic fragmentalization was what I was thinking but this issue is causing me some major problems at the moment as I do not know the fundamental basis of file compression)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help/psudocode/suggestions/answers would be amazing...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance,&#xA;Martin&lt;/p&gt;&#xA;" OwnerUserId="1137" LastActivityDate="2012-03-17T19:48:36.983" Title="Inverse compression for space-time trade off" Tags="|efficiency|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1667" PostTypeId="2" ParentId="1665" CreationDate="2012-03-17T18:23:13.843" Score="4" Body="&lt;p&gt;Most preconditioners effectively do this scaling internally. Jacobi scales to put 1 on the diagonal (or for absolute row sums of 1). Scaling is more important when using more sophisticated preconditioners such as geometric multigrid or those based on approximate Schur complements (where matrix entries are not available for algebraic scaling). Additionally, even if the preconditioner corrects the scaling, the unpreconditioned norm (and nonlinear residuals) will still be misleading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For factorization, it can be helpful for numerical stability to apply a pre- and post-scaling before starting the factorization. You can do the same with an iterative solve, but it is relatively more expensive (because scaling--and later unscaling if you want to check a final residual--is similar to the cost of an iteration) compared to direct solvers which do much more work. This scaling also makes it difficult to use &quot;physics-based&quot; preconditioners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever possible, I recommend formulating your initial problem to be well-scaled. If that is not possible, or you want to experiment with linear algebra applied to a legacy application in which it would take significant effort to scale properly, scaling is a useful tool.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-17T18:23:13.843" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1668" PostTypeId="2" ParentId="1666" CreationDate="2012-03-17T18:32:58.510" Score="5" Body="&lt;p&gt;There is no magic way to speed up an arbitrary operation by using more memory. Similarly, there is no way to compress arbitrary data. It so happens that a lot of data has exploitable redundancy, but even then, changing the algorithm (e.g. changing a spatial discretization, applying formal model reduction methods, or using an adjoint model) tends to offer much larger gains than generic compression. Furthermore, many operations are limited by the speed of memory rather than the speed of the CPU. Modern file systems such as ZFS and Btrfs offer transparent compression for &lt;em&gt;speed&lt;/em&gt; as much as to save disk space. With a fast compression algorithm, it's faster to compress by writing to disk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what few details you have given about your application, the most likely way to improve performance at the expense of disk space is to build more/better indexes. Profile carefully and at each point where the algorithm has to do a large read, evaluate the benefits of computing an index so that query could be answered faster. Note that the indexes are not free to compute and that it makes certain changes more expensive (because the indexes have to be updated).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-17T18:32:58.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1669" PostTypeId="2" ParentId="1666" CreationDate="2012-03-17T18:33:34.220" Score="3" Body="&lt;p&gt;If you can retrieve all the information you need from only using 8 bits of data, why store it with 256 bits?  Expanding a file size means that the processors have to read more bits from the hard disk for the same amount of information, and this will definitely slow down performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main reason why expanded memory results in faster performance is that RAM memory access is generally faster than Hard disk access.  In situations where it is absolutely necessary to use 10TB of data to perform a single instruction (not in a loop, but in a single atomic operation).  it makes sense that more memory will be better than less.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you have a dynamic programming problem where there are overlapping subproblems and optimal substructure, then using more memory results in a faster optimal solution.  But again, the speedup occurs from using RAM memory, not main memory access. &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-17T19:48:36.983" LastActivityDate="2012-03-17T19:48:36.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1670" PostTypeId="2" ParentId="1666" CreationDate="2012-03-17T19:01:57.187" Score="3" Body="&lt;p&gt;What you are asking is impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, if you have a data set of that size and are iterating through the whole thing, what's slowing you down is almost certainly the size of your data set.  Making your data set bigger is only going to make thing worse, as then you have to spend more time waiting for your disk and RAM to get back to the CPU with some bit of requested memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, there is usually no &quot;space-time trade off&quot;. It's true that some algorithms save time by storing intermediate results, thus using more space (the FFT is an example), but this only works if different parts of your problem have similar subproblems. This isn't a general property of algorithms. There are, occasionally, very specific ways of doing this for very specific problems, but you're case probably isn't one of them.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-03-17T19:01:57.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1671" PostTypeId="1" CreationDate="2012-03-17T19:47:46.317" Score="31" ViewCount="11184" Body="&lt;p&gt;Are there any tools specifically designed for compressing floating point scientific data?&lt;/p&gt;&#xA;&lt;p&gt;If a function is smooth, there's obviously a lot of correlation between the numbers representing that function, so the data should compress well.  Zipping/gzipping binary floating point data doesn't compress it that well though.  I am wondering if there are methods specifically developed for compressing floating point data.&lt;/p&gt;&#xA;&lt;p&gt;Requirements:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Either lossless compression or the possibility to specify a minimum number of digits to retain (for some applications &lt;code&gt;double&lt;/code&gt; might be more than what we need while &lt;code&gt;float&lt;/code&gt; might not have enough precision).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Well tested working tool (i.e. not just a paper describing a theoretical method).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Suitable for compressing 1D numerical data (such as a time series)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cross platform (must work on Windows)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It must be &lt;em&gt;fast&lt;/em&gt;---preferably not much slower than gzip.  I found that if I have the numbers stored as ASCII, gzipping the file can speed up reading and processing it (as the operation might be I/O bound).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'd especially like to hear from people who have actually used such a tool.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2022-09-15T11:59:24.180" LastActivityDate="2022-09-15T11:59:24.180" Title="Compressing floating point data" Tags="|data-storage|" AnswerCount="10" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="1672" PostTypeId="2" ParentId="1671" CreationDate="2012-03-17T19:58:33.743" Score="6" Body="&lt;p&gt;Arguably, you can interpret regression or transform methods (Fourier transform, Chebyshev transform) methods as &quot;compression&quot; for time-series or 1D function data. Remez's algorithm would be another candidate. In that case, using something like regression, FFT, or Chebyshev via FFT would work for your purposes. That said, none of these methods works on time series data with &lt;em&gt;arbitrary&lt;/em&gt; structure. For example, with FFT, you assume periodicity, and any sort of discontinuities in the data (or lack of periodicity) will lead to &lt;a href=&quot;http://en.wikipedia.org/wiki/Gibbs_phenomenon&quot; rel=&quot;noreferrer&quot;&gt;Gibbs' phenomenon&lt;/a&gt;. Similarly, with Chebyshev transforms, the assumption is that the data describes a function on $[-1,1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the underlying function, you may be able to fit the data to a functional form without error, requiring fewer coefficients to describe the functional form than you have data point (leading to the compression). Error results exist for some of these methods, although I don't know if any of them will give you &lt;em&gt;a priori&lt;/em&gt; (or &lt;em&gt;a posteriori&lt;/em&gt;) bounds or estimates on the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also look at methods developed specifically for the compression of floating point numbers, like FPC and related algorithms. See the papers &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CCcQFjAA&amp;amp;url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.91.7936%26rep%3Drep1%26type%3Dpdf&amp;amp;ei=PPNkT6eJHbPJsQLuvoS2Dw&amp;amp;usg=AFQjCNHW2wskk3IFQbtY6-nmCWpaVRv1bg&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=2&amp;amp;ved=0CDMQFjAB&amp;amp;url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.124.8968%26rep%3Drep1%26type%3Dpdf&amp;amp;ei=PPNkT6eJHbPJsQLuvoS2Dw&amp;amp;usg=AFQjCNER8YHkfe11C8qF113guSmrW6a0gQ&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://users.ices.utexas.edu/~burtscher/papers/dcc07a.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://users.ices.utexas.edu/~burtscher/papers/tr08.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;http://www.ece.neu.edu/groups/nucar/GPGPU4/files/oneil.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, along with a web page containing old source code &lt;a href=&quot;http://www.cs.unc.edu/~isenburg/lcpfpv/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-03-17T20:30:45.513" LastActivityDate="2012-03-17T20:30:45.513" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1673" PostTypeId="1" AcceptedAnswerId="1675" CreationDate="2012-03-17T20:23:16.723" Score="10" ViewCount="2768" Body="&lt;p&gt;I use Intel MKL for BLAS and use the &lt;a href=&quot;http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/&quot; rel=&quot;noreferrer&quot;&gt;Intel MKL Link Line Advisor&lt;/a&gt; for help with the command line options.&lt;/p&gt;&#xA;&lt;p&gt;The advisor provides options for Static, Dynamic and Single Dynamic Library. What do these terms mean?&lt;/p&gt;&#xA;&lt;p&gt;There have been numerous posts on SO regarding this but all of them have been focused on developing softwares rather than scientific/mathematical codes. (&lt;a href=&quot;https://stackoverflow.com/questions/311882/what-do-statically-linked-and-dynamically-linked-mean&quot;&gt;This one&lt;/a&gt; was especially useful)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How are these terms relevant to Parallel Computing?&lt;/li&gt;&#xA;&lt;li&gt;Is one &amp;quot;better&amp;quot; than the other? (I know that better would need to be defined more precisely but I don't know how to as of now)&lt;/li&gt;&#xA;&lt;li&gt;What is used more commonly and why?&lt;/li&gt;&#xA;&lt;li&gt;What does the programmer need to consider when linking?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="529" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2013-11-05T18:45:29.470" Title="What does Static, Dynamic and Single Dynamic linking mean?" Tags="|compiling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1674" PostTypeId="2" ParentId="1665" CreationDate="2012-03-17T20:32:19.533" Score="2" Body="&lt;p&gt;The main reason that sparse direct LU factorisation packages (like MUMPS etc) involve a pre-processing step that scales and permutes large entries onto the diagonal is that it often reduces the number of dynamic pivoting operations required within the factorisation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the pivoting is restricted to the diagonal (or the diagonal blocks in a supernodal sense) it's possible to perform a static symbolic analysis for the factorisation as an initialisation step. This is often faster in general, but especially important for parallel implementations - check out &lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/siam_pp99.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; SuperLU paper that discusses the parallel implications.&lt;/p&gt;&#xA;" OwnerUserId="1139" LastActivityDate="2012-03-17T20:32:19.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1675" PostTypeId="2" ParentId="1673" CreationDate="2012-03-17T20:50:28.593" Score="16" Body="&lt;p&gt;The terms &lt;em&gt;static linking&lt;/em&gt; and &lt;em&gt;dynamic linking&lt;/em&gt; are not directly related to parallel computing, though it has been known for quite some time that dynamic loading (as opposed to loading a statically-compiled executable) does not scale well on network file systems due to the heavy metadata load caused by the dynamic loader searching load paths for target libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is hard to make general statements about whether static or dynamic libraries are better in high performance computing.  Certainly for most supercomputing applications, it is simpler and preferred to statically link.  Why is this?  On the current generation of supercomputers, there is usually only one job running per node, which strongly reduces any benefits of reduced memory consumption due to shared libraries.  Additionally, scientific computing codes tend not to be very sophisticated in terms of language features or program design, and they rarely make use of language features that require dynamic loading (such as plugin modules).  Dynamic libraries have the additional difficulty of being much less portable across operating systems than static libraries.  This has lead to the creation of several tools for automating the installation of dynamic libraries, which can sometimes create more problems than they solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence of all of this, most HPC systems use static compilation when available.  Static libraries are viewed as faster, easier to install and maintain, and generally more robust.  HPC codes based on Python are one of the exceptions to this, but they are still subject to the performance problems associated with dynamic loading (several users on scicomp are actually working on this problem right now!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you are choosing static vs. dynamic linking, you need to consider how and where your code will be deployed, whether the underlying libraries are likely to change or move, and the performance characteristics of your network file system.  You should also evaluate whether you &lt;em&gt;need&lt;/em&gt; dynamic linking, either through a library dependence or to interoperate with a dynamic scripting language such as Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Single Dynamic Library is an Intel-specific term.  It refers to the packaging of their dynamic libraries into a single meta-library to simplify the linking process.  If you will be using dynamic linking with Intel libraries, this form is probably preferred unless you are doing something complicated.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2013-11-05T18:45:29.470" LastActivityDate="2013-11-05T18:45:29.470" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1676" PostTypeId="2" ParentId="1671" CreationDate="2012-03-18T02:52:23.950" Score="27" Body="&lt;p&gt;Try out &lt;a href=&quot;http://www.blosc.org/&quot; rel=&quot;noreferrer&quot;&gt;Blosc&lt;/a&gt;.  It is in many cases &lt;strong&gt;faster than memcopy&lt;/strong&gt;.  Think about that for a second. . . wicked.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is super stable, highly-vetted, cross-platform, and performs like a champ.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="7733" LastEditDate="2016-02-04T17:15:58.550" LastActivityDate="2016-02-04T17:15:58.550" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1677" PostTypeId="1" CreationDate="2012-03-18T03:31:28.787" Score="6" ViewCount="164" Body="&lt;p&gt;(Not sure if that's the right SX site? I don't need actual code, so…)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for an algorithm that generates all cartesian products for a list of sets, but skips tuples that are just rotations of previously issued tuples in a smart way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $(a_0,\dotsc,a_n)$ is a &lt;em&gt;rotation&lt;/em&gt; of $(b_0,\dotsc,b_n)$ iff $\exists x : \forall i \in [0,\dotsc,n]: a_i = b_j \text{ with } j = i + x \mod n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for $f(\{1, 2\}, \{1, 2, 3\})$ the output should be&#xA;$\{(1,1),&#xD;&#xA;(1,2),&#xD;&#xA;(1,3),&#xD;&#xA;(2,2),&#xD;&#xA;(2,3)\}$, i.e. without $(2,1)$ because it's a rotation of $(1,2)$.&lt;/p&gt;&#xA;" OwnerUserId="1168" LastEditorUserId="1168" LastEditDate="2012-03-19T20:30:01.710" LastActivityDate="2012-03-19T20:30:01.710" Title="Algorithm for generating all cartesian products, without rotations" Tags="|algorithms|combinatorics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1678" PostTypeId="1" CreationDate="2012-03-18T16:31:47.833" Score="11" ViewCount="2235" Body="&lt;p&gt;I'm working on a header-only matrix library to provide some reasonable degree of linear algebra capability in as simple a package as possible, and I'm trying to survey what the current state of the art is re: computing the SVD of a complex matrix.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing a two-phase decomposition, bidiagonalization followed by singular value computation.  Right now I'm using the householder method for the bidiagonalization (I believe LAPACK uses this as well), and I think that's about as good as it gets currently (unless someone knows of an $\mathcal{O}(N^2)$ algorithm for it..).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The singular value computation is next on my list, and I'm somewhat out of the loop on what the common algorithms are for doing this.  I read &lt;a href=&quot;http://web.eecs.utk.edu/~dongarra/etemplates/node193.html&quot;&gt;here&lt;/a&gt; that research was heading towards a inverse-iteration method that guarantees orthogonality with $\mathcal{O}(N)$ complexity.  I'd be interested in hearing about that or other advances.&lt;/p&gt;&#xA;" OwnerUserId="1171" LastEditorUserId="9905" LastEditDate="2015-05-14T13:04:48.340" LastActivityDate="2015-05-14T13:04:48.340" Title="What's the current state of the art regarding algorithms for the singular value decomposition?" Tags="|linear-algebra|matrix|svd|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1679" PostTypeId="2" ParentId="1678" CreationDate="2012-03-18T23:18:25.440" Score="6" Body="&lt;p&gt;&quot;Randomized algorithms&quot; have recently become quite popular for partial svds. A header only implementation can be downloaded here: &lt;a href=&quot;http://code.google.com/p/redsvd/&quot;&gt;http://code.google.com/p/redsvd/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A review of the current methods can be found here: &lt;a href=&quot;http://arxiv.org/abs/0909.4061&quot;&gt;http://arxiv.org/abs/0909.4061&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For full svds I am not sure if you can do better than Householder.&lt;/p&gt;&#xA;" OwnerUserId="979" LastEditorUserId="979" LastEditDate="2012-03-19T02:20:00.473" LastActivityDate="2012-03-19T02:20:00.473" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1680" PostTypeId="2" ParentId="1677" CreationDate="2012-03-19T01:49:50.980" Score="2" Body="&lt;p&gt;This is the brute-force way (in Python), maybe it's interesting to somebody:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def rotate_product(*xs):&#xA;    # map integer to tuple&#xA;    def from_int(i):&#xA;        tup = []&#xA;        for x in xs:&#xA;            tup.append(x[i % len(x)])&#xA;            i /= len(x)&#xA;        return tuple(tup)&#xA;&#xA;    # map tuple to integer, or -1 if impossible.&#xA;    def to_int(tup):&#xA;        i = 0&#xA;        mul = 1&#xA;        for t, x in zip(tup, xs):&#xA;            if t not in x:&#xA;                return -1&#xA;            i += mul * x.index(t)&#xA;            mul *= len(x)&#xA;        return i&#xA;&#xA;    # map integer to tuple, if no rotation of smaller tuple&#xA;    def make_tuple(i):&#xA;        tup = from_int(i)&#xA;        for k in range(1, len(xs)):&#xA;            if 0 &amp;lt;= to_int(tup[k:] + tup[:k]) &amp;lt; i:&#xA;                return None&#xA;        return tup&#xA;&#xA;    # total number of possible tuples&#xA;    n = 1&#xA;    for x in xs:&#xA;        n *= len(x)&#xA;&#xA;    # brute force through all possibilities&#xA;    for i in range(n):&#xA;        t = make_tuple(i)&#xA;        if t:&#xA;            yield t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1168" LastActivityDate="2012-03-19T01:49:50.980" CommentCount="0" CommunityOwnedDate="2012-03-19T01:49:50.980" ContentLicense="CC BY-SA 3.0" />
  <row Id="1681" PostTypeId="1" CreationDate="2012-03-19T05:11:05.257" Score="35" ViewCount="33520" Body="&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; I am testing if any eigenvalues have a magnitude of one or greater.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to find the largest absolute eigenvalue of a large sparse, non-symmetric matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been using R's &lt;code&gt;eigen()&lt;/code&gt; function, which uses the QR algo from either EISPACK or LAPACK to find all eigenvalues and then I use &lt;code&gt;abs()&lt;/code&gt; to get the absolute values. However, I need to do it faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also tried using the ARPACK interface in &lt;code&gt;igraph&lt;/code&gt; R package. However, it gave an error for one of my matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final implementation must be accessible from R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will probably be multiple eigenvalues of the same magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;Accuracy only needs to be to &lt;code&gt;1e-11&lt;/code&gt;. A &quot;typical&quot; matrix has so far been $386\times 386$. I have been able to do a QR factorisation on it. However, it is also possible to have much larger ones. I am currently starting to read about the Arnoldi algorithm. I understand that it is related to Lanczsos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT2:&lt;/strong&gt; If I have multiple matrices that I am &quot;testing&quot; and I know that there is a large submatrix that does not vary. Is it possible to ignore/discard it?&lt;/p&gt;&#xA;" OwnerUserId="1175" LastEditorUserId="10134" LastEditDate="2015-10-15T14:07:38.623" LastActivityDate="2021-08-12T20:21:05.750" Title="What is the fastest way to calculate the largest eigenvalue of a general matrix?" Tags="|linear-algebra|algorithms|eigensystem|sparse-matrix|" AnswerCount="6" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1682" PostTypeId="2" ParentId="1678" CreationDate="2012-03-19T05:15:43.410" Score="0" Body="&lt;p&gt;There are the PROPACK and nu-TRLan libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://soi.stanford.edu/~rmunk/PROPACK/&quot; rel=&quot;nofollow&quot;&gt;http://soi.stanford.edu/~rmunk/PROPACK/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://crd-legacy.lbl.gov/~kewu/trlan/&quot; rel=&quot;nofollow&quot;&gt;http://crd-legacy.lbl.gov/~kewu/trlan/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1175" LastEditorUserId="1175" LastEditDate="2012-03-19T06:54:10.213" LastActivityDate="2012-03-19T06:54:10.213" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1683" PostTypeId="2" ParentId="1681" CreationDate="2012-03-19T06:37:14.843" Score="2" Body="&lt;p&gt;It's not the &lt;em&gt;fastest&lt;/em&gt; way, but a reasonably quick way is to just hit an (initially random) vector with the matrix repeatedly, and then normalize every few steps. Eventually it will converge to the largest eigenvector, and the gain in norm for a single step is the associated eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works best when the largest eigenvalue is substantially larger than any other eigenvalue. If another eigenvalue is close in magnitude to the largest, this will take a while to converge, and it may be difficult to determine if it &lt;em&gt;has&lt;/em&gt; converged.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-03-19T06:37:14.843" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1684" PostTypeId="2" ParentId="1639" CreationDate="2012-03-19T10:34:07.380" Score="4" Body="&lt;p&gt;I'll try to give you my view on my limited experience, which only covers a few type of plots: 2D data plots, scatterplots, and graph-based diagrams (trees and graphs, flowcharts); but first allow me to deviate from the question for a bit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I would like to say that the importance of producing high quality plots is often overlooked. Plots are not only for displaying &lt;em&gt;data&lt;/em&gt; but if used effectively they can communicate complex ideas which often can not be expressed clearly in text. It seems to me that this is not the appropriate place to discuss what constitutes, the benefits, and the disadvantages of a good plot but if interested &lt;a href=&quot;http://lilt.ilstu.edu/gmklass/pos138/datadisplay/sections/goodcharts.htm&quot; rel=&quot;nofollow&quot;&gt;look here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, I have more often regretted using a half-baked plot than spending time on a high quality plot. The reason is that a bad plot can &lt;em&gt;lie&lt;/em&gt; to you, for example, if using a half-baked plot for doing a quick look over data that you might have not properly normalized, or by using not-so-good plots to do visual comparisons (the eye can often be very forgiving or very misleading). So I would always recommend to prepare plots as if they were going to be shown to a person that is not familiar with the results. This can save you time and improve your work. Also, you have the added benefit that presentation material will be ready for the next presentation/seminar/paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to &lt;strong&gt;answer the question&lt;/strong&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience it is always better to separate the data processing from the actual plotting tool, and image manipulation tools. For data processing I use python as it is very flexible and I have not found a file format that cannot be easily handled through python. Moreover, the numpy and scipy libraries can be powerful manipulation tools for numerical data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within python I have not seen a plotting tool that gives you the control over all formatting details. I have found that it is the details that make a large difference when you try to integrate images while formatting a paper in latex. For instance, control of the whitespace and proportions of-and-around plots can be a headache with matplotlib (not impossible but not worth my time), this is very important if you have limited space as it is often the case in journal and conference papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, I have found that &lt;a href=&quot;http://gnuplot.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;GNUPLOT&lt;/a&gt; is the best tool for formatting and producing 2D plots and scatterplots. It very much gives you control over all the details of your plot. And you can produce high quality plots (vector images) in black-and-white or colour. The high level of control makes GNUPLOT a bit more difficult to learn, and sometimes simple things can take a bit of work but if you start from an example it can simplify things. I usually look at this two sites for inspiration, see the &lt;a href=&quot;http://t16web.lanl.gov/Kawano/gnuplot/index-e.html&quot; rel=&quot;nofollow&quot;&gt;not-so-FAQ site&lt;/a&gt; or the &lt;a href=&quot;http://gnuplot.sourceforge.net/demo/&quot; rel=&quot;nofollow&quot;&gt;official demos here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes, the plots produced by GNUPLOT can be quite large (in MB range) and the journal might reject the files (this happened to me with some very colourful scatterplots). I have used and I recommend &lt;a href=&quot;http://www.imagemagick.org/script/index.php&quot; rel=&quot;nofollow&quot;&gt;Imagemagick&lt;/a&gt;, a command-line tool that allows you to convert, edit, and compress images (I used it to generate and edit high quality jpegs while significantly reducing the file size). I tried other tools that used automatic data compression, this proved specially problematic due to the delicate balance between image artifacts and compression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also have produced a fair share of diagrams (often for presentations). I found that if I'm going to use a diagram a single time (not a paper) then the Keynote application (OSX) and saving to pdf is good enough. However, when an image is used many times, the presentation quality is more important and &lt;a href=&quot;http://www.texample.net/tikz/&quot; rel=&quot;nofollow&quot;&gt;TikZ&lt;/a&gt; is my preferred tool. I find TikZ a bit difficult to use but if you &lt;a href=&quot;http://www.texample.net/tikz/examples/&quot; rel=&quot;nofollow&quot;&gt;start from an example&lt;/a&gt; it is much more easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I do most of my work from the command-line so to integrate different tools I use with bash or python.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="1070" LastEditDate="2012-03-19T10:39:46.623" LastActivityDate="2012-03-19T10:39:46.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1685" PostTypeId="2" ParentId="1681" CreationDate="2012-03-19T11:55:14.987" Score="9" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Power_iteration&quot;&gt;Power Iteration&lt;/a&gt; (or Power Method), e.g. what Dan is describing, should always converge, albeit at the rate $\left|\lambda_{n-1}/\lambda_{n}\right|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\lambda_{n-1}$ is close to $\lambda_n$, it will be slow, but you can use &lt;a href=&quot;http://ilpubs.stanford.edu:8090/605/1/2003-45.pdf&quot;&gt;extrapolation&lt;/a&gt; to get around that. It may seem complicated, but an implementation in pseudo-code is given in the paper.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-03-19T11:55:14.987" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1686" PostTypeId="2" ParentId="1681" CreationDate="2012-03-19T12:26:15.033" Score="19" Body="&lt;p&gt;It depends a lot on the size of your matrix, in the large-scale case also on whether it is sparse, and on the accuracy you want to achieve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix is too large to allow a single factorization, and you need high accuracy, the Lanczsos algorithm is probably the fastest way. In the nonsymmetric case, the Arnoldi algorithm is needed, which is numerically unstable, so an implementation needs to address this (is somewhat awkward to cure). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not the case in your problem, give more specific information in your question. Then add a comment to this answer, and I'll update it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: [This was for the old version of the question, asling for the largest eigenvalue.] As your matrix is small and apparently dense, I'd do Arnoldi iteration on B=(I-A)^{-1}, using an initial permuted triangular factorization of I-A to have cheap multiplication by B. (Or compute an explicit inverse, but this costs 3 times as much as the factorization.) You want to test whether B has a negative eigenvalue. Working with B in place of A, negative eigenvalues are much better separated, so if there is one, you should converge rapidly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am curious about where your problem comes from. Nonsymmetric matrices usually have complex eigenvalues, so ''largest'' isn't even well-defined. Thus you must know more about your problem, which might help in suggesting how to solve it even faster and/or more reliably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit2: It is difficult to get with Arnoldi a &lt;em&gt;particular&lt;/em&gt; subset of interest. To get the absolutely largest eigenvalues reliably, you'd do subspace iteration using the original matrix, with a subspace size matching or exceeding the number of eigenvalues expected to be close to 1 or larger in magnitude. On small matrices, this will be slower than the QR algorithm but on large matrices it will be much faster.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-03-30T08:38:22.637" LastActivityDate="2012-03-30T08:38:22.637" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1687" PostTypeId="2" ParentId="1681" CreationDate="2012-03-19T12:46:21.390" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://web.eecs.utk.edu/%7Edongarra/etemplates/node138.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; you will find an algorithmic introduction to the Jacobi-Davidson (JD) algorithm, which computes the maximum eigenvalue (by modulus/absolute value).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.win.tue.nl/%7Ehochsten/pdf/jdgamm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this paper&lt;/a&gt; the mathematical aspects are explored. JD allows general (real or complex) matrices and can be used to compute ranges of eigenvalues.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.win.tue.nl/casa/research/topics/jd/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; you can find various library implementations JDQR and JDQZ (including a C interface, which you should be able to link to from R).&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="40004" LastEditDate="2021-08-12T20:21:05.750" LastActivityDate="2021-08-12T20:21:05.750" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="1688" PostTypeId="1" AcceptedAnswerId="1713" CreationDate="2012-03-19T16:35:18.220" Score="6" ViewCount="5446" Body="&lt;p&gt;After going to all possible &quot;Introductions&quot; to ScaLapack, I still can't understand how to carry out a simple &lt;code&gt;PDGEMV&lt;/code&gt; operation using it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I must do :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to generate a matrix using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do i=1,n&#xA;  x(i) = i*i*1.0D+00&#xA;  do j=1,n&#xA;    A(i,j) = (i+j)*j*1.00D+00&#xA;  end do&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then simply multiply &lt;code&gt;Matrix A by Vector x (b=Ax)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Lapack, I simply use &#xA;&lt;code&gt;CALL DGEMV('N',n,n,1.0D+00,a,n,x,1,0.0D+00,b,1)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I achieve this in PBLAS? I have run the sample (&lt;code&gt;example1.f&lt;/code&gt; from the website) program and it works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is partitioning of the matrix? How do I do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everyone (in tutorials) seem to be concentrating on &lt;code&gt;DGESV&lt;/code&gt; while I want &lt;code&gt;DGEMV/DGEMM&lt;/code&gt; i.e. &lt;code&gt;BLAS&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="81" LastEditDate="2012-03-19T18:39:45.103" LastActivityDate="2012-03-21T16:11:35.357" Title="How do I use ScaLapack/PBLAS for Matrix-Vector Multiplication?" Tags="|lapack|blas|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1689" PostTypeId="1" CreationDate="2012-03-19T22:01:27.280" Score="5" ViewCount="97" Body="&lt;p&gt;What are all of the different ways to derive the partial charges, van der Waals interactions, bond lengths, etc. of an amino acid (in other words, all of the parameters that could be used in a molecular mechanics force field or other related method - such as a Poisson-Boltzmann equation solver)? I'm especially interested in the strengths and weaknesses of each method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Complete fitting protocols would be greatly appreciated, although all I'm really looking for is a few sentences describing each method and its broad strengths and weaknesses.&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="81" LastEditDate="2012-03-20T17:58:39.680" LastActivityDate="2012-03-24T12:34:19.317" Title="What are all of the different methods for parameterizing an amino acid (or other small molecule)?" Tags="|regression|computational-biology|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1690" PostTypeId="1" AcceptedAnswerId="1724" CreationDate="2012-03-19T22:34:19.083" Score="4" ViewCount="4387" Body="&lt;p&gt;I know that gnuplot's &lt;code&gt;epslatex&lt;/code&gt; terminal can display superscipts and subscripts in titles and label axes. Is this also possible when using gnuplot's &lt;code&gt;png&lt;/code&gt; terminal?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, how does one do this?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-03-22T17:13:59.940" Title="Can gnuplot's PNG terminal use superscripts and subscipts?" Tags="|visualization|gnuplot|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1691" PostTypeId="2" ParentId="1690" CreationDate="2012-03-20T03:00:56.600" Score="3" Body="&lt;p&gt;It &lt;em&gt;is&lt;/em&gt; possible to use LaTeX syntax for super- and subscripts in axes labels and titles when the terminal is set to &lt;code&gt;png&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To to this, set the terminal with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set term png enhanced font &quot;/full/path/to/font&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then plot normally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the font part isn't included, you may get this error message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gdImageStringFT: Could not find/open font&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This means gnuplot can't find the plots on your system. On linux, fonts are generally located in &lt;code&gt;/usr/share/fonts&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-03-22T17:13:59.940" LastActivityDate="2012-03-22T17:13:59.940" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1692" PostTypeId="1" AcceptedAnswerId="1697" CreationDate="2012-03-20T07:36:07.350" Score="10" ViewCount="2167" Body="&lt;p&gt;I've got a system of ordinary differential equations - 7 equations, and ~30 parameters governing their behavior as part of a mathematical model of disease transmission. I'd &lt;em&gt;like&lt;/em&gt; to find the steady states for those equations Changing &lt;code&gt;dx/dt = rest of the equation&lt;/code&gt; to &lt;code&gt;0 = equation&lt;/code&gt; for each of the equations makes it a straightforward algebra problem. This could be done by hand, but I'm ridiculously bad at that kind of computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried using Mathematica, which can handle smaller versions of this problem (&lt;a href=&quot;https://mathematica.stackexchange.com/questions/2338/very-long-refine-solve-batch-run-is-my-code-broken-or-just-complicated&quot;&gt;see here&lt;/a&gt;), but Mathematica is grinding to a halt on this problem. Is there a more efficient/effective way to approach this? A more efficient symbolic math system? Other suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few updates (March 21st):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The goal is indeed to solve them symbolically - the numerical answers are &lt;em&gt;nice&lt;/em&gt; but for the moment the end-goal is the symbolic version.&lt;/li&gt;&#xA;&lt;li&gt;There is at least one equilibrium. I haven't actually sat down and &lt;em&gt;proved&lt;/em&gt; this, but by design it should have at least one trivial one wherein none is infected at the start. There may not be anything &lt;em&gt;besides&lt;/em&gt; that, but that would make me as content as anything else.&lt;/li&gt;&#xA;&lt;li&gt;Below is the actual set of equations being talked about.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/CHWSX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, I'm looking for symbolic expressions for the solutions of a system of 7 quadratic equations in 7 variables.&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2022-05-12T19:22:16.103" Title="Symbolic solution of a system of 7 nonlinear equations" Tags="|ode|symbolic-computation|computational-biology|" AnswerCount="5" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1693" PostTypeId="2" ParentId="1689" CreationDate="2012-03-20T08:53:42.200" Score="3" Body="&lt;p&gt;To ask about all ways is too ask too much, I guess. For the computationally most efficient way to encode the geometric information, see Section 2 of my survey paper&#xA;&lt;A HREF=&quot;http://www.mat.univie.ac.at/~neum/ms/protein.pdf&quot; rel=&quot;nofollow&quot;&gt;&#xA;Molecular modeling of proteins and mathematical prediction of protein structure&lt;/A&gt;,&#xA;SIAM Rev. 39 (1997), 407-460.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-20T08:53:42.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1694" PostTypeId="1" CreationDate="2012-03-20T09:31:57.013" Score="5" ViewCount="103" Body="&lt;p&gt;Consider an $n \times n$ real, time-dependent matrix $A(t)$ such that $A(t) = A(t)^{T} &amp;gt; 0$ and $A(t)$ is continuous on $[a,b]$. Then it is posible to specify a matrix $S(t) \in SO(n)$ such that $S(t)A(t)S^{T}(t) = \Lambda(t)$, where $\Lambda$ is a diagonal matrix. For instance, we can obtain such $S(t)$ using Jacobi rotation method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for time-continuous on $[a,b]$ matrices $S(t)$ and $\Lambda(t)$. Lets consider a net&#xA;$$&#xD;&#xA;  a = t_0 &amp;lt; t_1 &amp;lt;  \ldots &amp;lt; t_n = b&#xD;&#xA;$$&#xA;Assume that $S(t_k)$, $\Lambda(t_k)$ are computed. How to receive than $S(t_{k+1})$ and $\Lambda(t_{k+1})$ nearest possible to $S(t_{k})$ and $\Lambda(t_k)$? Which algorithm should I use?&lt;/p&gt;&#xA;" OwnerUserId="1180" LastEditorUserId="1180" LastEditDate="2012-03-20T11:22:12.203" LastActivityDate="2012-03-20T11:22:12.203" Title="Time-stable spectral decomposition algorithm" Tags="|eigensystem|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1695" PostTypeId="1" AcceptedAnswerId="1700" CreationDate="2012-03-20T12:04:21.193" Score="8" ViewCount="284" Body="&lt;p&gt;I write codes in Fortran and C for various matrix algorithms. However, when I profile my codes using VTune, I usually run into some terminology that I cannot fully appreciate. Is there a good resource for learning profiling from scratch to a fairly advanced level?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I am looking forward to profiling numerical codes, I don't think (I might be wrong) that profiling other codes is any different. &lt;strong&gt;I want a tutorial on profiling and I would prefer an online (Free) PDF but wouldn't mind books or handbooks.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried reading VTune's handbook but thats like trying to learn chinese by reading a chinese language book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, is VTune the best way to go? I really like a GUI and since I use Intel MKL, I figured it would be better than Valgrind.&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="414" LastEditDate="2012-03-21T20:05:23.783" LastActivityDate="2012-03-23T12:16:36.970" Title="Reference Request for Profiling High Performance Computing Codes" Tags="|parallel-computing|reference-request|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1697" PostTypeId="2" ParentId="1692" CreationDate="2012-03-20T13:54:41.830" Score="14" Body="&lt;p&gt;It looks like the equations you're dealing with are all polynomial after clearing denominators. That's a good thing (transcendental functions are often a little harder to deal with algebraically). However, it's not a guarantee that your equations have a closed-form solution. This is an essential point that many people don't really &quot;get&quot;, even if they know it in theory, so it bears restating: there are fairly simple systems of polynomial equations for which there is no way of giving the solutions in terms of ($n$th) roots etc. A famous example (in one variable) is $x^5-x+1=0$. See also &lt;a href=&quot;http://en.wikipedia.org/wiki/Quintic_function&quot;&gt;this wikipedia page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, of course there are also systems of equations that &lt;em&gt;can&lt;/em&gt; be solved, and it's worthwhile to check if your system is one of those. And even if your system cannot be solved, it might still be possible to find a form for your system of equations that is simpler, in some sense. For example, find one equation involving only the first variable (even if it cannot be solved algebraically), then a second equation involving only the first and second variable, etc. There are a few competing theories for how to find such &quot;normal forms&quot; of polynomial systems; the most well-known is Groebner basis theory, and a competing one is the theory of regular chains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the computer algebra system Maple (full disclosure: I work for them) both of them are implemented. The &lt;code&gt;solve&lt;/code&gt; command typically calls the Groebner basis method, I believe, and that quickly grinds to a halt on my laptop. I tried running the regular chains computation and it takes longer than I have patience for but doesn't seem to blow up as badly memory-wise. In case you're interested, the help page for the command I used is &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=RegularChains/RealTriangularize&quot;&gt;here&lt;/a&gt;, and here is the code I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;restart;&#xA;sys, vars := {theta*H - rho_p*sigma_p*&#xA;       Cp*(Us/N) - rho_d*sigma_d*D*(Us/N)*rho_a*sigma_a*&#xA;       Ca*(Us/N) = 0, &#xA;         rho_p*sigma_p*Cp*(Us/N) + rho_d*sigma_d*&#xA;       D*(Us/N)*rho_a*sigma_a*Ca*(Us/N) + theta*H = 0, &#xA;         (1/omega)*Ua - alpha*Up - rho_p*psi_p*&#xA;       Up*(H/N) - Mu_p*sigma_p*Up*(Cp/N) - &#xA;             Mu_a*sigma_a*Up*(Ca/N) - Theta_p*&#xA;       Up + Nu_up*(Theta_*M + Zeta_*D) = 0, &#xA;         alpha*Up - (1/omega)*Ua - rho_a*psi_a*&#xA;       Ua*(H/N) - Mu_p*sigma_p*Ua*(Cp/N) - &#xA;             Mu_a*sigma_a*Ua*(Ca/N) - Theta_a*&#xA;       Ua + Nu_ua*(Theta_*M + Zeta_*D) = 0, &#xA;         (1/omega)*Ca + Gamma_*Phi_*D + rho_p*psi_p*&#xA;       Up*(H/N) + Mu_p*sigma_p*Up*(Cp/N) + &#xA;             Mu_a*sigma_a*Up*(Ca/N) - alpha*Cp - Kappa_*&#xA;       Cp - Theta_p*Cp + Nu_cp*(Theta_*M + Zeta_*D) = 0, &#xA;         alpha*Cp + Gamma_*(1 - Phi_)*D + rho_a*psi_a*&#xA;       Ua*(H/N) + Mu_p*sigma_p*Ua*(Cp/N) + &#xA;             Mu_a*sigma_a*Ua*(Ca/N) - (1/omega)*&#xA;       Ca - Kappa_*Tau_*Ca - Theta_a*Ca + &#xA;             Nu_ca*(Theta_*M + Zeta_*D) = &#xA;     0, Kappa_*Cp + Kappa_*Tau_*Ca - Gamma_*Phi_*&#xA;       D - Gamma_*(1 - Phi_)*D - &#xA;             Zeta_*D + Nu_d*(Theta_*M + Zeta_*D) = 0, &#xA;    Us + H + Up + Ua + Cp + Ca + D = 0, &#xA;         Up + Ua + Cp + Ca + D = 0}, {Us, H, Up, Ua, Cp, Ca, D, N, &#xA;    M}:&#xA;&#xA;sys := subs(D = DD, sys):&#xA;vars := subs(D = DD, vars):&#xA;params := indets(sys, name) minus vars:&#xA;ineqs := [theta &amp;gt; 0 , rho_p &amp;gt; 0 , sigma_p &amp;gt; &#xA;       0 , rho_d &amp;gt; 0 , sigma_d &amp;gt; 0 , &#xA;            rho_a &amp;gt; 0 , sigma_a &amp;gt; 0 , &#xA;      omega &amp;gt; 0 , alpha &amp;gt; 0 , psi_p &amp;gt; 0 , Mu_p &amp;gt; 0 , &#xA;            Mu_a &amp;gt; 0 , Theta_p &amp;gt; 0 , Nu_up &amp;gt; 0 , Theta_ &amp;gt; &#xA;       0 , Zeta_ &amp;gt; 0 , psi_a &amp;gt; 0 , &#xA;            Theta_a &amp;gt; 0 , Nu_ua &amp;gt; 0 , Gamma_ &amp;gt; 0 , Phi_ &amp;gt; &#xA;       0 , Kappa_ &amp;gt; 0 , Nu_cp &amp;gt; 0 , &#xA;            Tau_ &amp;gt; 0 , Nu_ca &amp;gt; 0]:&#xA;with(RegularChains):&#xA;R := PolynomialRing([vars[], params[]]):&#xA;sys2 := map(numer, map(lhs - rhs, normal([sys[]]))):&#xA;sol := LazyRealTriangularize(sys2,[],map(rhs, ineqs),[],R);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="49" LastActivityDate="2012-03-20T13:54:41.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1698" PostTypeId="2" ParentId="1688" CreationDate="2012-03-20T14:36:10.250" Score="-1" Body="&lt;p&gt;Use the pblas GEMV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/scalapack/pblas_qref.html#PvGEMV&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/scalapack/pblas_qref.html#PvGEMV&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2012-03-20T14:36:10.250" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1699" PostTypeId="1" AcceptedAnswerId="1703" CreationDate="2012-03-20T14:36:15.837" Score="4" ViewCount="1059" Body="&lt;p&gt;Suppose a square, real and symmetric matrix $G\in\mathbb{R}^{n\times n}$ is given, and it is known to have one zero eigenvalue associated with all ones eigenvector, $1_n$. I'm aware that the (possibly) negative spectrum could be shifted to non-negative,&#xA;$$(G + c\cdot I)u = Gu + c\cdot Iu = \lambda\cdot u + c\cdot u = (\lambda + c)\cdot u,$$ &#xA;where $c$ is at least the absolute value of the smallest negative eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) with the above shifting, are the zero eigenvalue affected (I'm asking because of certain contradictory I found in the literature)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) given $J_n=I_n-\frac{1}{n}1_n1_n^T$, what can one say about the spectral decomposition of $$G-c\cdot J_n$$ Is this the same as the above method of shift along the diagonal? I read that that all the eigenvalues are shifted, except for the zero eigenvalue that remains (corresponding to $1_n$), but I wonder how this could be shown.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="276" LastEditDate="2012-03-20T16:54:23.787" LastActivityDate="2012-03-20T16:54:23.787" Title="Spectral decomposition with eigenvalue shift" Tags="|linear-algebra|matrix|eigensystem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1700" PostTypeId="2" ParentId="1695" CreationDate="2012-03-20T14:54:57.527" Score="6" Body="&lt;p&gt;Another common choice is to use PAPI &lt;a href=&quot;http://icl.cs.utk.edu/papi/&quot;&gt;http://icl.cs.utk.edu/papi/&lt;/a&gt; or TAU &lt;a href=&quot;http://www.cs.uoregon.edu/Research/tau/home.php&quot;&gt;http://www.cs.uoregon.edu/Research/tau/home.php&lt;/a&gt;, Valgrind is great for finding memory errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some slides from TACC: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.tacc.utexas.edu/user-services/training/course-materials&quot;&gt;http://www.tacc.utexas.edu/user-services/training/course-materials&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.tacc.utexas.edu/c/document_library/get_file?uuid=fc609b77-b727-4bff-81a4-d30caa4013d4&amp;amp;groupId=13601&quot;&gt;http://www.tacc.utexas.edu/c/document_library/get_file?uuid=fc609b77-b727-4bff-81a4-d30caa4013d4&amp;amp;groupId=13601&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="30" LastEditDate="2012-03-20T15:38:02.863" LastActivityDate="2012-03-20T15:38:02.863" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1702" PostTypeId="2" ParentId="1692" CreationDate="2012-03-20T16:15:11.493" Score="7" Body="&lt;p&gt;The professional way is to write your equations in a modeling language such as AMPL or GAMS, and solve it with a solver such as IPOPT. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;AMPL is a commercial system, but a free student version of AMPL is able to pose problems with up to 300 equations and variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you just want to solve one or a few problems, you can have it solved online freely by using the NEOS server for optimization - simply submit the AMPL description and wait for the answer to be returned to you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to solve such systems repeatedly as part of a bigger study (e.g., varying the parameters), you should download IPOPT (which is software under a very liberal licence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Note that symbolic solutions that are comprehensible are usually restricted to quite small problems - typically the size of a Groebner basis grows explosively with the number of variables or the degree of the polynomials, and the time for processing even more. Thus a waiting time of an hour or more with Mathematica is a sign (though not a proof) that your symbolic solution would be completely incomprehensible. Moreover, evaluating such a long expression is likely to be numerically unstable, so you would need high precision in the evaluation to get meaningful results.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-03-21T18:10:42.733" LastActivityDate="2012-03-21T18:10:42.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1703" PostTypeId="2" ParentId="1699" CreationDate="2012-03-20T16:21:42.733" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When you shift $G$ by $cI$, all eigenvalues are shifted by $c$, including the zero eigenvalue.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When you shift by $G$ by $cJ$, and if $G 1=0$ as you assume, all eigenvalues except one on the zero eigenvalues are shifted by $c$. Thus if $G$ has an $n$fold zero eigenvalue, the shifted matrix has $n-1$ eigenvalues $c$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can prove these statements by extending $1$ to an orthonormal basis consisting of eigenvectors of $G$. Then your problem has been diagonalized in the new basis and all these properties are trivially verified.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-20T16:21:42.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1704" PostTypeId="2" ParentId="1692" CreationDate="2012-03-20T16:53:14.893" Score="3" Body="&lt;p&gt;It depends on the structure of your equations.&lt;/p&gt;&#xA;&lt;p&gt;If you're looking for &lt;em&gt;all&lt;/em&gt; steady states of your set of equations, &lt;em&gt;and&lt;/em&gt; you can rearrange them as ErikP says into polynomials, you can use methods from real algebraic geometry to calculate all numerical solutions to high precision. &lt;a href=&quot;http://www.nd.edu/%7Esommese/bertini/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bertini&lt;/a&gt; is one such package that I know of, but there are others. I went to a conference at Notre Dame a few years ago where Bertini was used to find steady states of ODEs from chemical kinetics; Bertini was developed at Notre Dame.&lt;/p&gt;&#xA;&lt;p&gt;Another possibility is to use the methods proposed in &lt;a href=&quot;https://doi.org/10.1007/s10543-010-0280-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Nonsmooth exclusion test for finding all solutions of nonlinear equations&amp;quot; by M. D. Stuber, V. Kumar, and P. I. Barton, &lt;em&gt;BIT Numerical Mathematics&lt;/em&gt; &lt;strong&gt;50&lt;/strong&gt;(4), 885-917, DOI: DOI: 10.1007/s10543-010-0280-6&lt;/a&gt;; these methods do not require the system of equations to be polynomials. Paul Barton is my adviser, and Matt Stuber is a colleague of mine; if you like, I can ask him for the software and send it to you. The paper uses methods from global optimization and interval arithmetic (it cites ArnoldNeumaier's book), as well as Newton's method. The advantage of this method is that it should locate all solutions; the disadvantage is that it's complicated.&lt;/p&gt;&#xA;&lt;p&gt;In case it isn't clear, ArnoldNeumaier is suggesting that instead of solving &lt;span class=&quot;math-container&quot;&gt;$\mathbf{F}(\mathbf{x}) = \mathbf{0}$&lt;/span&gt; directly using something like Newton's method (which will generally work if you give it a good initial guess, sufficiently close to a solution), you solve&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min_{\mathbf{x} \in S}\|\mathbf{F}(\mathbf{x})\|,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is some feasible set defined by constraints on your problem instead of trying to solve. At a very crude level, using a smooth nonlinear programming solver is a lot like using Newton's method, with additional algorithmic sophistication for robustness and performance. &lt;a href=&quot;https://github.com/coin-or/Ipopt&quot; rel=&quot;nofollow noreferrer&quot;&gt;IPOPT&lt;/a&gt; is a really good piece of software for this purpose; there are a litany of other solvers out there (just look at the available solvers list for &lt;a href=&quot;https://www.gams.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAMS&lt;/a&gt;, &lt;a href=&quot;https://ampl.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AMPL&lt;/a&gt;, or &lt;a href=&quot;https://neos-server.org/neos/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEOS&lt;/a&gt;. If you choose a method like this one, be aware of a few caveats:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It will only locate at most one solution at a time. To find additional solutions, you need to add constraints that exclude all of the previous solutions you've found.&lt;/li&gt;&#xA;&lt;li&gt;If your optimization problem is nonconvex, to use IPOPT or similar solvers, you will either need a good initial guess, close to a solution of your equations (same basic principle as Newton's method), or a nonconvex optimization solver like &lt;a href=&quot;https://minlp.com/baron-solver&quot; rel=&quot;nofollow noreferrer&quot;&gt;BARON&lt;/a&gt;, &lt;a href=&quot;https://www.coin-or.org/Couenne/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Couenne&lt;/a&gt;, &lt;a href=&quot;https://github.com/coin-or/Bonmin&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bonmin&lt;/a&gt;, etc. You should try every solver you get your hands on, since the performance of each nonconvex nonlinear programming solver is problem-dependent.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="22284" LastEditDate="2022-05-12T19:22:16.103" LastActivityDate="2022-05-12T19:22:16.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1705" PostTypeId="2" ParentId="74" CreationDate="2012-03-20T17:42:06.957" Score="30" Body="&lt;h2&gt;Edit: This is now in SymPy&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ isympy&#xA;In [1]: A = MatrixSymbol('A', n, n)&#xA;In [2]: B = MatrixSymbol('B', n, n)&#xA;In [3]: context = Q.symmetric(A) &amp;amp; Q.positive_definite(A) &amp;amp; Q.orthogonal(B)&#xA;In [4]: ask(Q.symmetric(B*A*B.T) &amp;amp; Q.positive_definite(B*A*B.T), context)&#xA;Out[4]: True&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Older answer that shows other work&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;So after looking into this for a while this is what I've found. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current answer to my specific question is &quot;No, there is no current system that can answer this question.&quot; There are however a few things that seem to come close. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, Matt Knepley and Lagerbaer both pointed to &lt;a href=&quot;http://www.aices.rwth-aachen.de:8080/aices/preprint/documents/AICES-2012-01-02.pdf&quot;&gt;work by Diego Fabregat and Paolo Bientinesi&lt;/a&gt;. This work shows both the potential importance and the feasibility of this problem. It's a good read. Unfortunately I'm not certain exactly how his system works or what it is capable of (if anyone knows of other public material on this topic do let me know). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, there is a tensor algebra library written for Mathematica called &lt;a href=&quot;http://xact.es/&quot;&gt;xAct&lt;/a&gt; which handles symmetries and such symbolically. It does some things very well but is not tailored to the special case of linear algebra. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, these rules are written down formally in a couple of libraries for &lt;a href=&quot;http://coq.inria.fr/&quot;&gt;Coq&lt;/a&gt;, an automated theorem proving assistant (Google search for coq linear/matrix algebra to find a few). This is a powerful system which unfortunately seems to require human interaction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After &lt;a href=&quot;https://sympa-roc.inria.fr/wws/arc/coq-club/2012-02/msg00141.html&quot;&gt;talking with some theorem prover people&lt;/a&gt; they suggest looking into logic programming (i.e. Prolog, which Lagerbaer also suggested) for this sort of thing. To my knowledge this hasn't yet been done - I may play with it in the future. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I've implemented this using the &lt;a href=&quot;http://maude.cs.uiuc.edu&quot;&gt;Maude system&lt;/a&gt;. My code is hosted &lt;a href=&quot;http://github.com/mrocklin/matrix-algebra&quot;&gt;on github&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-10-26T00:50:47.277" LastActivityDate="2012-10-26T00:50:47.277" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1706" PostTypeId="2" ParentId="1658" CreationDate="2012-03-20T18:11:09.967" Score="19" Body="&lt;p&gt;You should check out sympy.stats. It provides an interface to deal with random variables. The following example provides a random variable &lt;code&gt;X&lt;/code&gt; defined on the unit interval with density &lt;code&gt;2x&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [1]: from sympy.stats import *&#xA;In [2]: x = Symbol('x')&#xA;In [3]: X = ContinuousRV(x, 2*x, Interval(0, 1))&#xA;&#xA;In [4]: P(X&amp;gt;.5) &#xA;Out[4]: 0.750000000000000&#xA;&#xA;In [5]: Var(X) # variance&#xA;Out[5]: 1/18&#xA;&#xA;In [6]: E(2*cos(X)+X**2) # complex expressions are ok too&#xA;Out[6]: -7/2 + 4⋅cos(1) + 4⋅sin(1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you're interested this abstraction can handle some fairly complex manipulations.&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2015-03-17T17:48:11.333" LastActivityDate="2015-03-17T17:48:11.333" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1707" PostTypeId="2" ParentId="1639" CreationDate="2012-03-21T04:50:54.493" Score="11" Body="&lt;p&gt;I think this is an excellent question, and one that is at the forefront of my own mind right now.  When reading most math-ish journals, Matlab seems to be acceptable, but it just doesn't measure up IMHO to some of the beautiful plots and diagrams that appear in journals like PNAS, Nature, PLoS ONE, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After discussing exactly this issue in my research group for several weeks now, we have still come to no real conclusion on which package is best.  We have found that most comparisons of graphics software in documentation, blogs or other web sources are largely useless in determining which package is best for a given purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that what would really help the average user who is &lt;strong&gt;not an expert&lt;/strong&gt; in all/many of these packages is to have a small set of very well-defined examples that could be used as a &quot;graphical benchmark&quot; of sorts (in a similar sense as &lt;a href=&quot;http://www.featflow.de/en/benchmarks/cfdbenchmarking.html&quot;&gt;CFD benchmarks&lt;/a&gt;).  As far as I know, nothing like this currently exists. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At minimum, I would want to see:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A reasonably straightforward 2D line plot with several line and point types, legend, math in labels/titles, etc.  &lt;/li&gt;&#xA;&lt;li&gt;Ditto for a 3D surface plot.&lt;/li&gt;&#xA;&lt;li&gt;A more sophisticated 3D plot with isosurfaces, cutting planes, and perhaps some other fancy features.&lt;/li&gt;&#xA;&lt;li&gt;A fancy 3D diagram.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Data would be provided for #1-3, and a photo or &quot;original&quot; diagram for #4.  For each example, code (input) and images (output) would be posted.  This would allow the reader to more easily judge which package is right for them, both in terms of image quality and code complexity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My plan is to do something like this in my group over the coming months for #1, 2, 4 and a small number of packages including Matlab, pgf/TikZ, python/scipy.  If there is interest, I could post this publicly.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So while this is not an answer to the originally posted question &lt;strong&gt;yet&lt;/strong&gt; (and I apologize for that) it could be an answer in several months' time.&lt;/p&gt;&#xA;" OwnerUserId="319" LastActivityDate="2012-03-21T04:50:54.493" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1708" PostTypeId="2" ParentId="1695" CreationDate="2012-03-21T07:13:54.483" Score="3" Body="&lt;p&gt;If you are interested in performance analysis of parallel codes I like the book:&#xA;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/069111935X&quot; rel=&quot;nofollow&quot;&gt;Scientific Parallel Computing, L. Ridgway Scott, Terry Clark, Babak Bagheri&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, &lt;a href=&quot;http://people.cs.uchicago.edu/~ridg/newpsc/cp.pdf&quot; rel=&quot;nofollow&quot;&gt;here are some cool slides&lt;/a&gt; used in a summer school course given (PASI. Valparaíso, Chile. January 2011) by one of the authors, the material is based on the book.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-03-21T07:13:54.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1710" PostTypeId="1" AcceptedAnswerId="1714" CreationDate="2012-03-21T13:52:07.593" Score="3" ViewCount="125" Body="&lt;p&gt;Consider an equation $S(t)b(t) = a$, where $a, b(t) \in S^{n-1}$ are given and the vector $b(t)$ is continuous, i.e. its endpoint traces a continuous curve on the unit sphere. The task is to find continuous solution $S(t) \in SO(n)$ (numerically). Consider a time net $t_0 &amp;lt; ... &amp;lt; t_n$. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In 2D case we can count the angle of $b(t)$ on the unit circle and then $S(t)$ will be the rotation matrix by this angle (inversed) corrected by the angle of $a$. &lt;/li&gt;&#xA;&lt;li&gt;In 3D case we can use the Euler theorem and correct the matrix $S(t_k)$ on each step of our iterational method by multiplying it by the rotation matrix $S&amp;#39;$ that sends $b(t_k)$ to $b(t_{k+1})$.&lt;/li&gt;&#xA;&lt;li&gt;In general case assume that $a=(1,0,...,0)^{T}$. I tried to construct some orthonormed basis ${b(t_0),e_2,...,e_n}$ at first step and to let $S(t_0) = [b(t_0),e_2,...,e_n]^{-1}$ using Gram-Schmidt process. Then at each step of our method I constructed the new one using Gram-Schmidt process and the previous basis as initial condition. It works, but it is time-consuming enough. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My question is how to obtain such $S(t)$ in general case more quickly?&lt;/p&gt;&#xA;" OwnerUserId="1180" LastEditorUserId="1180" LastEditDate="2012-03-21T13:57:12.297" LastActivityDate="2012-03-21T19:33:40.257" Title="Time-stable SO(n) matrix synthesis algorithm" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1713" PostTypeId="2" ParentId="1688" CreationDate="2012-03-21T16:11:35.357" Score="5" Body="&lt;p&gt;The data distribution scalapack requires is a &lt;a href=&quot;http://netlib.org/scalapack/slug/node75.html&quot; rel=&quot;noreferrer&quot;&gt;block-cyclic&lt;/a&gt; &lt;a href=&quot;http://www.netlib.org/utk/papers/factor/node3.html&quot; rel=&quot;noreferrer&quot;&gt;distribution&lt;/a&gt;.   (A little interactive calculator is &lt;a href=&quot;http://acts.nersc.gov/scalapack/hands-on/datadist.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;).   Once you get the hang of that, Scalapack is pretty straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a given blocksize (which is a free parameter) and decomposition of the processors into a grid (eg, 4 processors -&gt; 2 rows, 2 columns of processors) you can go from the local indices of your chunk of the matrix to the global indices of the matrix using l2g or g2l from &lt;a href=&quot;http://www.nersc.gov/users/software/programming-libraries/math-libraries/libsci/libsci-example/&quot; rel=&quot;noreferrer&quot;&gt;this NERSC example&lt;/a&gt;; that's a pretty straightforward way to get started.   Row or column vectors are treated as nx1 or 1xn matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is an example of multiplying the diagonal matrix A (1,2,3...) by a unit vector X and getting the results into Y; the only real trick here is that not all processors have a chunk of the X or Y vectors.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program gemv1&#xA;      use mpi&#xA;      implicit none&#xA;&#xA;      integer :: n, nb    ! problem size and block size&#xA;      integer :: myArows, myAcols   ! size of local subset of global matrix&#xA;      integer :: myXrows, myXcols   ! size of local subset of global vector&#xA;      integer :: i,j, myi, myj&#xA;      real, dimension(:,:), allocatable :: myA,myX,myY&#xA;      integer :: ierr&#xA;&#xA;      integer, external :: numroc   ! blacs routine&#xA;      integer :: me, procs, icontxt, prow, pcol, myrow, mycol  ! blacs data&#xA;      integer :: info    ! scalapack return value&#xA;      integer, dimension(9)   :: ides_a, ides_x, ides_y ! matrix descriptors&#xA;      integer, dimension(2) :: dims&#xA;      real :: error, globerror&#xA;&#xA;! Initialize blacs processor grid&#xA;&#xA;      call blacs_pinfo   (me,procs)&#xA;&#xA;! create as square as possible a grid of processors&#xA;&#xA;      dims = 0&#xA;      call MPI_Dims_create(procs, 2, dims, ierr)&#xA;      prow = dims(1)&#xA;      pcol = dims(2)&#xA;&#xA;! create the BLACS context&#xA;&#xA;      call blacs_get     (0, 0, icontxt)&#xA;      call blacs_gridinit(icontxt, 'R', prow, pcol)&#xA;      call blacs_gridinfo(icontxt, prow, pcol, myrow, mycol)&#xA;&#xA;! Construct local arrays&#xA;! Global structure:  matrix A of n rows and n columns&#xA;&#xA;      n = int(25000.*sqrt(dble(procs)))&#xA;&#xA;! blocksize - a free parameter.&#xA;&#xA;      nb = 100&#xA;&#xA;! how big is &quot;my&quot; chunk of matrix A?&#xA;&#xA;      myArows = numroc(n, nb, myrow, 0, prow)&#xA;      myAcols = numroc(n, nb, mycol, 0, pcol)&#xA;&#xA;! how big is &quot;my&quot; chunk of vector x?&#xA;&#xA;      myXrows = numroc(n, nb, myrow, 0, prow)&#xA;      myXcols = 1&#xA;&#xA;! Initialize local arrays    &#xA;&#xA;      allocate(myA(myArows,myAcols)) &#xA;      allocate(myX(myXrows,myXcols)) &#xA;      allocate(myY(myXrows,myXcols)) &#xA;&#xA;      myA = 0.&#xA;      do myj=1,myAcols&#xA;          ! get global index from local index&#xA;          call l2g(myj,mycol,n,pcol,nb,j)&#xA;          do myi=1,myArows&#xA;              ! get global index from local index&#xA;              call l2g(myi,myrow,n,prow,nb,i)&#xA;              if (i == j) myA(myi,myj) = i&#xA;          enddo&#xA;      enddo&#xA;&#xA;      myX = 0.&#xA;      call l2g(1,mycol,n,pcol,nb,j)&#xA;      if (j == 1) then&#xA;          do myi=1,myXrows&#xA;              call l2g(myi,myrow,n,prow,nb,i)&#xA;              myX(myi,1) = 1.&#xA;          enddo&#xA;      endif&#xA;&#xA;      myY = 0.&#xA;&#xA;! Prepare array descriptors for ScaLAPACK &#xA;&#xA;      call descinit( ides_a, n, n, nb, nb, 0, 0, icontxt, myArows, info )&#xA;      call descinit( ides_x, n, 1, nb, nb, 0, 0, icontxt, myXrows, info )&#xA;      call descinit( ides_y, n, 1, nb, nb, 0, 0, icontxt, myXrows, info )&#xA;&#xA;! Call ScaLAPACK library routine&#xA;&#xA;      call psgemv('N',n,n,1.,mya,1,1,ides_a,myx,1,1,ides_x,1,0.,myy,1,1,ides_y,1)&#xA;      if (me == 0) then&#xA;        if (info /= 0) then&#xA;             print *, 'Error -- info = ', info&#xA;        endif&#xA;      endif&#xA;&#xA;! Deallocate the local arrays&#xA;&#xA;      deallocate(myA, myX)&#xA;&#xA;! Print results - Y should be 1,2,3...&#xA;&#xA;      error = 0.&#xA;      call l2g(1,mycol,n,pcol,nb,j)&#xA;      if (j == 1) then&#xA;          do myi=1,myXrows&#xA;              call l2g(myi,myrow,n,prow,nb,i)&#xA;              error = error + (myY(myi,1)-1.*i)**2.&#xA;          enddo&#xA;      endif&#xA;&#xA;      call MPI_Reduce(error, globerror, 1, MPI_REAL, MPI_SUM, 0, MPI_COMM_WORLD, ierr)&#xA;&#xA;      if (me == 0) then&#xA;        print *,'Y l2 error = ', sqrt(globerror/n)&#xA;      endif&#xA;&#xA;      deallocate(myY)&#xA;&#xA;! End blacs for processors that are used&#xA;&#xA;      call blacs_gridexit(icontxt)&#xA;      call blacs_exit(0)&#xA;&#xA;contains&#xA;&#xA;! convert global index to local index in block-cyclic distribution&#xA;&#xA;   subroutine g2l(i,n,np,nb,p,il)&#xA;&#xA;   implicit none&#xA;   integer, intent(in) :: i    ! global array index, input&#xA;   integer, intent(in) :: n    ! global array dimension, input&#xA;   integer, intent(in) :: np   ! processor array dimension, input&#xA;   integer, intent(in) :: nb   ! block size, input&#xA;   integer, intent(out):: p    ! processor array index, output&#xA;   integer, intent(out):: il   ! local array index, output&#xA;   integer :: im1   &#xA;&#xA;   im1 = i-1&#xA;   p   = mod((im1/nb),np)&#xA;   il  = (im1/(np*nb))*nb + mod(im1,nb) + 1&#xA;&#xA;   return&#xA;   end subroutine g2l&#xA;&#xA;! convert local index to global index in block-cyclic distribution&#xA;&#xA;   subroutine l2g(il,p,n,np,nb,i)&#xA;&#xA;   implicit none&#xA;   integer :: il   ! local array index, input&#xA;   integer :: p    ! processor array index, input&#xA;   integer :: n    ! global array dimension, input&#xA;   integer :: np   ! processor array dimension, input&#xA;   integer :: nb   ! block size, input&#xA;   integer :: i    ! global array index, output&#xA;   integer :: ilm1   &#xA;&#xA;   ilm1 = il-1&#xA;   i    = (((ilm1/nb) * np) + p)*nb + mod(ilm1,nb) + 1&#xA;&#xA;   return&#xA;   end subroutine l2g&#xA;&#xA;end program gemv1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2012-03-21T16:11:35.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1714" PostTypeId="2" ParentId="1710" CreationDate="2012-03-21T16:24:23.583" Score="2" Body="&lt;p&gt;Let $A(t_{k}) = [b(t_{k}), e_{2}, \ldots, e_{n}]$. If $b(t_{k+1}) = b(t_{k})$, nothing changes, so without loss of generality, assume that $b(t_{k+1}) - b(t_{k}) \neq 0$, and also assume that $A(t_{k})$ and $A(t_{k+1})$ are both invertible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get $A(t_{k+1})^{-1}$ from $A(t_{k})^{-1}$ by noting that $A(t_{k+1})$ is a rank-one update of $A(t_{k})$, and using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Sherman%E2%80%93Morrison_formula&quot; rel=&quot;nofollow&quot;&gt;Sherman-Morrison formula&lt;/a&gt;, with $u = b(t_{k+1}) - b(t_{k})$, and $v = e_{1}$; this rank-one update would be cheaper than naïvely inverting $A(t_{k+1})$ using an LU decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, since $A(t_{k+1})^{-1}$ is a rank-one update of $A(t_{k})^{-1}$, you can update your QR factorization (at least, that's how I would do Gram-Schmidt numerically) also using a rank-one updating scheme. One algorithm for accomplishing a rank-one update of a QR factorization can be found in Section 12.5.1 of the third edition of &lt;em&gt;Matrix Computations&lt;/em&gt; by Golub and van Loan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of these updates should reduce the complexity of $\mathcal{O}(n^{3})$ operations to $\mathcal{O}(n^{2})$ operations; you can even calculate $A(t_{0})^{-1}$ and its QR factorization in this fashion because $A(t_{0})$ is a rank-one update of the identity matrix.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-03-21T16:24:23.583" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1715" PostTypeId="2" ParentId="1710" CreationDate="2012-03-21T19:26:53.323" Score="2" Body="&lt;p&gt;Rewrite your equation&#xA;$$&#xD;&#xA;b(t)=B(t)a&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B(t+\Delta t)=B^\prime B(t)$ where &#xA;$$B^\prime=\left(\begin{matrix} b/\|b\| &amp;amp; \dot b/\|\dot b\| \end{matrix}\right)\left(\begin{matrix} \cos \theta &amp;amp; \sin \theta \\ -\sin \theta &amp;amp; \cos \theta \end{matrix}\right)\left(\begin{matrix} b/\|b\| &amp;amp; \dot b/\|\dot b\| \end{matrix}\right)^\ast +\\ I-\left(\begin{matrix} b/\|b\| &amp;amp; \dot b/\|\dot b\| \end{matrix}\right)\left(\begin{matrix} b/\|b\|&amp;amp; \dot b/\|\dot b\| \end{matrix}\right)^\ast$$&#xA;and $\theta$ such that $B^\prime$ takes $b(t)$ to $b(t+\Delta t)$.&#xA;Now $S(t)=B(t)^\ast$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B was constructed such that your coordinate frame is defined by $b$ and $\dot b$. Since all other directions are unaffected, we project onto the complement of $\mbox{span }\{b,\dot b\}$ (second term on the right) and then rotate within $b$ and $\dot b$ (first term on right).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also want to consider that your problem can be understood as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ S\dot b +\dot S b = 0$$&#xA;such that&#xA;$$ S\in SO(n).$$&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2012-03-21T19:33:40.257" LastActivityDate="2012-03-21T19:33:40.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1716" PostTypeId="2" ParentId="1695" CreationDate="2012-03-21T20:14:53.287" Score="1" Body="&lt;p&gt;I tool I have used extensively for profiling is &lt;a href=&quot;http://www.valgrind.org/&quot; rel=&quot;nofollow&quot;&gt;valgrind&lt;/a&gt; (in combination with the graphical interface &lt;a href=&quot;http://kcachegrind.sourceforge.net/html/Home.html&quot; rel=&quot;nofollow&quot;&gt;kcachegrind&lt;/a&gt;). Valgrind is a set of dynamic analysis tools: memory error detection, threading bug hunting and profiling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some references on Valgrind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.network-theory.co.uk/valgrind/manual/&quot; rel=&quot;nofollow&quot;&gt;Valgrind 3.3 — Advanced Debugging and Profiling for GNU/Linux applications&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://valgrind.org/docs/manual/manual.html&quot; rel=&quot;nofollow&quot;&gt;Valgrind user manual&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Unfortunately (depends on where you come from :-) ), valgrind doesn't run on Windows, only on Linux and Darwin based machines. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2012-03-23T12:16:36.970" LastActivityDate="2012-03-23T12:16:36.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1717" PostTypeId="1" AcceptedAnswerId="1719" CreationDate="2012-03-22T01:59:11.313" Score="3" ViewCount="551" Body="&lt;p&gt;I wrote a function to read in the runtime parameters my program takes. This function is in a different file than my main program; the two object files are linked after compilation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I compile the file that contains the function I use to read in parameters, I get the following warning:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;io.c(43): warning #120: return value type does not match the function type&#xA;CHKERRQ(ierr);&#xA;^&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This only seems to affect the &lt;code&gt;CHEKRRQ()&lt;/code&gt; macro, and I don't get this warning when I compile the examples included with PETSc.  The only obvious difference is that the example is entirely within one file whereas my program is broken up into multiple files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does PETSc support separate compilation, or does some aspect of the &lt;code&gt;CHKERRQ()&lt;/code&gt; macro cause problems?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="9" LastEditDate="2012-03-22T08:10:57.427" LastActivityDate="2012-03-24T20:49:43.357" Title="Why can't I compile a function using the CHKERRQ macro in PETSc?" Tags="|petsc|compiling|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1718" PostTypeId="2" ParentId="1717" CreationDate="2012-03-22T07:04:44.890" Score="0" Body="&lt;p&gt;&lt;code&gt;CHKERRQ&lt;/code&gt; cannot be used inside functions other than &lt;code&gt;main()&lt;/code&gt; that don't return &lt;code&gt;PetscErrorCode&lt;/code&gt;. This is because the &lt;code&gt;CHKERRQ()&lt;/code&gt; macro is basically &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if(ierr) {PetscError(..,ierr,..);return ierr;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/CHKERRQ.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Although typical usage resembles &quot;void CHKERRQ(PetscErrorCode)&quot; as described above, for certain uses it is highly inappropriate to use it in this manner as it invokes return(PetscErrorCode). In particular, it cannot be used in functions which return(void) or any other datatype. In these types of functions, you can use CHKERRV() which returns without an error code (bad idea since the error is ignored or if (n) {PetscError(....); return(YourReturnType);} where you may pass back a PETSC_NULL to indicate an error. You can also call CHKERRABORT(comm,n) to have MPI_Abort() returned immediately.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Basically, any functions that use &lt;code&gt;CHKERRQ()&lt;/code&gt; should either return &lt;code&gt;PetscErrorCode&lt;/code&gt; themselves or use &lt;code&gt;CHKERRV()&lt;/code&gt; instead.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-03-24T20:49:43.357" LastActivityDate="2012-03-24T20:49:43.357" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1719" PostTypeId="2" ParentId="1717" CreationDate="2012-03-22T08:09:37.117" Score="3" Body="&lt;p&gt;&lt;code&gt;CHKERRQ&lt;/code&gt; can only be used in functions that return &lt;code&gt;PetscErrorCode&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/CHKERRQ.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc documentation&lt;/a&gt; is reasonably clear here:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;CHKERRQ&lt;/code&gt; - Checks error code, if non-zero it calls the error handler and then returns&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;CHKERRQ(n)&lt;/code&gt; is fundamentally a macro replacement for:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;if (n) return(PetscError(...,n,...));&lt;/code&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Although typical usage resembles &lt;code&gt;void CHKERRQ(PetscErrorCode)&lt;/code&gt; as described above, for certain uses it is highly inappropriate to use it in this manner as it invokes &lt;code&gt;return(PetscErrorCode)&lt;/code&gt;. In particular, it cannot be used in functions which &lt;code&gt;return(void)&lt;/code&gt; or any other datatype. In these types of functions, you can use &lt;code&gt;CHKERRV()&lt;/code&gt; which returns without an error code (bad idea since the error is ignored or &lt;code&gt;if (n) {PetscError(....); return(YourReturnType);}&lt;/code&gt; where you may pass back a &lt;code&gt;PETSC_NULL&lt;/code&gt; to indicate an error. You can also call &lt;code&gt;CHKERRABORT(comm,n)&lt;/code&gt; to have &lt;code&gt;MPI_Abort()&lt;/code&gt; returned immediately.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-03-22T08:09:37.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1720" PostTypeId="1" AcceptedAnswerId="1721" CreationDate="2012-03-22T13:13:58.897" Score="10" ViewCount="470" Body="&lt;p&gt;as the title suggests I'm trying to compute the integral of a compactly supported function (Wendland's quintic polynomial) on a triangle. Notice, that the center of the function is somewhere in 3-D space. I integrate this function on an arbitrary, but small triangle ($area &amp;lt; \frac{(radius/4)^2}{2}$). I am currently using the integration described by Dunavant, 1985 (p=19). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems however, that these quadrature rules are not suited towards compactly supported problems. This is supported by the fact that when I integrate $f(r) = [r\leq1]$ (so a function that is 1 inside the circle of radius 1) on a plane which is discretized using triangles, my (normalized) results are between 1.001 and 0.897.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, does a specialized quadrature rule exist for this kind of problem? Would a lower order composite integration rule work better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately this routine is really critical in my code so precision is crucial. On the other hand I need to do this integration &quot;a couple of times&quot; for a single time-step so computational expense should not be too high. Parallelization is not an issue as I will execute the integration itself in serial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for your answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Wendland's quintic polynomial is given by $W(q) = [q\leq2]\frac{\alpha}{h^3}(1-\frac{q}{2})^4(2q+1)$ with $\alpha = \frac{21}{16\pi}$ and $q=\frac{\|r-r_0\|}{h}$ with $r_0$ being an arbitrary vector in $\mathbb{R}^3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2:&#xA;If $\Delta$ is the two-dimensional triangle then I want to calculate $\int_\Delta \omega(r) dr$ with $\omega(r) = W(\frac{\|r-r_0\|}{h})$. So $q$ in $W$ will never be smaller than 0. Note that the integral is a surface integral over a 2-D surface in $\mathbb{R}^3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT3:&#xA;I have an analytical solution for the 1-D (line) problem. Calculating one for 2-D (triangle) might be possible as well.&lt;/p&gt;&#xA;" OwnerUserId="1176" LastEditorUserId="1176" LastEditDate="2012-03-22T19:11:17.197" LastActivityDate="2012-03-24T12:17:54.007" Title="Numerical integration of compactly supported function on a triangle" Tags="|quadrature|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1721" PostTypeId="2" ParentId="1720" CreationDate="2012-03-22T14:02:02.987" Score="4" Body="&lt;p&gt;Since the function is smooth within $q \leq 2$, but not of fixed degree (in the plane, that is), I would suggest using a simple adaptive scheme, e.g. the &lt;a href=&quot;http://en.wikipedia.org/wiki/Trapezoidal_rule&quot; rel=&quot;nofollow&quot;&gt;Trapezoidal Rule&lt;/a&gt; with &lt;a href=&quot;http://en.wikipedia.org/wiki/Romberg%27s_method&quot; rel=&quot;nofollow&quot;&gt;Romberg's method&lt;/a&gt;, in both dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, if your triangle is defined by the vertices $x$, $y$ and $z \in \mathbb R^3$, and you have a routine &lt;code&gt;romb(f,a,b)&lt;/code&gt; which integrates &lt;code&gt;f&lt;/code&gt; along the line from &lt;code&gt;a&lt;/code&gt; to &lt;code&gt;b&lt;/code&gt;, you could do the following (in Matlab notation):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int = romb( @(xi) romb( W , xi , y+(z-y)*(xi-x)./(z-x) ) , x , z );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;romb&lt;/code&gt;, don't use a fixed number of points, but keep growing the table until the difference between two successive diagonals is below your required tolerance. Since your function is smooth, this should be a good error estimate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If parts of the triangle are outside of the domain of $W(q)$, you could try adjusting the limits of integration in the above code accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may not be the most computationally efficient way of solving your problem, but the adaptivity will give you much more robustness than a fixed-degree rule will.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-03-24T12:17:54.007" LastActivityDate="2012-03-24T12:17:54.007" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1722" PostTypeId="2" ParentId="1720" CreationDate="2012-03-22T14:13:49.050" Score="2" Body="&lt;p&gt;Integration rules assume that the function is locally well approximated by a low degree polynomial. Your problem has nothing to do with compact support. Compactly supported radial basis functions are smooth at the support boundary, and quadrature rules up to the order of smoothness can be used without problems. (Higher order rules do not help; thus you should probably not use a rule that integrates degree 5 polynomials exactly.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the inaccuracy comes from the fact that the assumption of good polynomial approximability fails in your case for triangles near $r_0$, even when they do not contain $r_0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$W$ is smooth as a function of $q$, but $q$ is a nonsmooth function of $r$, with a gradient that becomes infinite in the limit $r\rightarrow r_0$. Integration is over $r$, and the composite function is a nonsmooth function of $r$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the triangle does not contain $r_0$, the function is $C^inf$ but this doesn't help as the higher derivative grow very quickly close to $r_0$, and the error of a high order method is proportional to a high order derivative, hence very large!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple remedy is to split each triangle T into a number N_T of subtriangles. You can take $N_T=1$ far away from $r_0$, and $N_T\gg 1$ close to $r_0$. You can figure out offline how big $N_T$ must be for triangles of a given diameter and distance from $r_0$ to reach a desired accuracy. Moreover, you should only use low order formulas close to $r_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you integrate over a triangle, but $r_0$ is 3-dimensional, the triangle is apparently in $R^3$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A faster remedy would therefore tabulate the integral for $r_0=0$ as a function of the triangle coordinates (normalized by rotating it into a 2-dimensional $xy$-plane such that one vertex lies on the $x$-axis, and reflecting it such that a second vertex lies above it). This tabulation must be sufficiently detailed to make a linear or quadratic interpolation accurate enough. But you can use the slow method outlined first to create this table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to get rid of the problem is to use a compactly supported radial basis function that is a polynomial in $q^2$ rather than $q$. This is smooth everywhere, and easy to integrate.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-03-22T17:39:57.023" LastActivityDate="2012-03-22T17:39:57.023" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1723" PostTypeId="2" ParentId="1720" CreationDate="2012-03-22T14:15:46.810" Score="2" Body="&lt;p&gt;For a good overview of cubature rules, see &quot;R. Cools, An Encyclopaedia of Cubature Formulas J. Complexity, 19: 445-453, 2003&quot;. Using a fixed rule, can give you the advantage that some rules integrate polynomials exactly (as Gaussian quadrature does in one-dimension).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cools is also one of the main authors of &lt;a href=&quot;http://nines.cs.kuleuven.be/software/CUBPACK/&quot; rel=&quot;nofollow&quot;&gt;CUBPACK&lt;/a&gt;, a software package for numerical cubature.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-03-22T14:15:46.810" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1724" PostTypeId="2" ParentId="1690" CreationDate="2012-03-22T16:46:09.767" Score="5" Body="&lt;p&gt;You can't just use TeX syntax unless you are using the TiKZ, epslatex, or another terminal that uses LaTeX to process the labels. You can get some fairly crude math layout, including superscripts and subscripts, by using the enhanced text option to many of gnuplot's terminals, including, sometimes, png. But you will have to use the gnuplot enhanced text syntax, which is unrelated to (La)TeX syntax. Fortunately, some of the syntax happens to coincide, including the basic use of superscripts and subscripts. On my system enhanced text works with the pngcairo terminal but not plain png. If you don't have that, you can try others, such as svg. I have plenty of examples of using both LaTeX and enhanced text in my &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/184951724X&quot; rel=&quot;nofollow&quot;&gt;new book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1192" LastActivityDate="2012-03-22T16:46:09.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1725" PostTypeId="1" AcceptedAnswerId="1728" CreationDate="2012-03-22T17:54:44.383" Score="11" ViewCount="523" Body="&lt;p&gt;I'm doing a Lanczos diagonalization of a large sparse matrix (~2 million elements). Almost all of the steps in the Lanzcos algorithm are done in parallel on the GPU, except for diagonalizing the Lanczos matrix to check for convergence. For that, I've been using the TQLI algorithm from Numerical Recipes. Are there methods for finding the eigensystem of a tridiagonal matrix that are parallel or easily parallelizable? Does a parallel version of TQLI exist?&lt;/p&gt;&#xA;" OwnerUserId="1193" LastActivityDate="2022-07-25T13:27:27.107" Title="Parallel algorithm for eigensystem of a tridiagonal matrix" Tags="|linear-algebra|algorithms|eigensystem|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1726" PostTypeId="2" ParentId="1725" CreationDate="2012-03-22T18:26:39.290" Score="10" Body="&lt;p&gt;TQL cannot be parallelized.&lt;/p&gt;&#xA;&lt;p&gt;The standard parallel algorithm is that of Cuppen:&lt;/p&gt;&#xA;&lt;p&gt;JJM Cuppen,&#xA;A divide and conquer method for the symmetric tridiagonal eigenproblem, 1980.&lt;BR&gt;&#xA;&lt;a href=&quot;https://doi.org/10.1007/BF01396757&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1007/BF01396757&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;see also:&lt;/p&gt;&#xA;&lt;p&gt;F. Tisseur,&#xA;A parallel divide and conquer algorithm for the symmetric eigenvalue problem on distributed memory architectures, 1999&lt;BR&gt;&#xA;&lt;a href=&quot;http://eprints.ma.man.ac.uk/981/01/covered/MIMS_ep2007_225.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eprints.ma.man.ac.uk/981/01/covered/MIMS_ep2007_225.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.75.4109&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.75.4109&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www14.in.tum.de/konferenzen/Jass09/courses/2/Kleine_Albers_paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www14.in.tum.de/konferenzen/Jass09/courses/2/Kleine_Albers_paper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorDisplayName="user43608" LastEditDate="2022-07-25T13:27:27.107" LastActivityDate="2022-07-25T13:27:27.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1727" PostTypeId="1" AcceptedAnswerId="1729" CreationDate="2012-03-22T19:58:48.713" Score="15" ViewCount="2684" Body="&lt;p&gt;&lt;em&gt;Sorry for the long post but I wanted to include everything that I thought was relevant in the first go.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;h1&gt;What I want&lt;/h1&gt;&#xA;&lt;p&gt;I am implementing a parallel version of Krylov Subspace Methods for &lt;strong&gt;Dense Matrices.&lt;/strong&gt; Mainly GMRES, QMR and CG. I realized (after profiling) that my DGEMV routine was pathetic. So I decided to concentrate on that by isolating it. I have tried running it on a 12 core machine but the results below are for a 4 core Intel i3 Laptop. There is not much difference in the trend.&lt;/p&gt;&#xA;&lt;p&gt;My &lt;code&gt;KMP_AFFINITY=VERBOSE&lt;/code&gt; output is available &lt;a href=&quot;http://pastebin.com/g2LMNfMu&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I wrote out a small code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;size_N = 15000&#xA;A = randomly_generated_dense_matrix(size_N,size_N); %Condition Number is not bad&#xA;b = randomly_generated_dense_vector(size_N);&#xA;for it=1:n_times %n_times I kept at 50 &#xA; x = Matrix_Vector_Multi(A,b);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I believe this simulates the behaviour of CG for 50 iterations.&lt;/p&gt;&#xA;&lt;h1&gt;What I have tried:&lt;/h1&gt;&#xA;&lt;h3&gt;Translation&lt;/h3&gt;&#xA;&lt;p&gt;I had originally written the code in Fortran. I translated it to C, MATLAB and Python (Numpy). Needless to say, MATLAB and Python were horrible. &lt;strong&gt;Surprisingly, C was better than FORTRAN by a second or two for the above values. Consistently.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Profiling&lt;/h3&gt;&#xA;&lt;p&gt;I profiled my code to run and it ran for &lt;code&gt;46.075&lt;/code&gt; seconds. This was when &lt;strong&gt;MKL_DYNAMIC was set to &lt;code&gt;FALSE&lt;/code&gt;&lt;/strong&gt; and all cores were used. If I used MKL_DYNAMIC as true, only (approx.) half the number of cores were in use at any given point of time. Here are a few details:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Address Line    Assembly                CPU Time&#xA;&#xA;0x5cb51c        mulpd %xmm9, %xmm14     36.591s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The most time consuming process seems to be :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Call Stack                          LAX16_N4_Loop_M16gas_1&#xA;CPU Time by Utilization             157.926s&#xA;CPU Time:Total by Utilization       94.1%&#xA;Overhead Time                       0us&#xA;Overhead Time:Total                 0.0%    &#xA;Module                              libmkl_mc3.so   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here are a few pictures:&lt;img src=&quot;https://i.stack.imgur.com/8133L.png&quot; alt=&quot;enter image description here&quot; /&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/GeMAK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Conclusions:&lt;/h1&gt;&#xA;&lt;p&gt;I'm a real beginner at profiling but I realize that the speed up is still not good. The sequential (1 Core) code finishes in &lt;strong&gt;53 seconds. That is a speed up of less than 1.1 !&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Real Question : What should I do to improve my speedup?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Stuff that I think might help but I can't be sure:&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pthreads implementation&lt;/li&gt;&#xA;&lt;li&gt;MPI (ScaLapack) implementation&lt;/li&gt;&#xA;&lt;li&gt;Manual Tuning (I don't know how. Please recommend a resource if you suggest this)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;em&gt;If anyone needs more (especially regarding memory) details, please let me know what I should run and how. I have never memory profiled before.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-08-19T22:53:44.883" Title="Why isn't my Matrix-Vector Multiplication Scaling?" Tags="|parallel-computing|matrix|lapack|blas|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1728" PostTypeId="2" ParentId="1725" CreationDate="2012-03-22T20:03:54.930" Score="4" Body="&lt;p&gt;I suggest using a library like &lt;a href=&quot;http://www.grycap.upv.es/slepc/&quot; rel=&quot;nofollow&quot;&gt;SLEPc&lt;/a&gt;, which includes interfaces to many different methods for solving eigensystems in serial or parallel. The &lt;a href=&quot;http://www.grycap.upv.es/slepc/documentation/slepc.pdf&quot; rel=&quot;nofollow&quot;&gt;user manual&lt;/a&gt; includes references to several different methods for solving eigenvalue problems.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-03-22T20:03:54.930" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1729" PostTypeId="2" ParentId="1727" CreationDate="2012-03-22T21:52:05.637" Score="21" Body="&lt;p&gt;Your matrix is of size 15,000 x 15,000, so you have 225M elements in the matrix. This makes &#xA;for roughly 2GB of memory. This is much more than the cache size of your processor, so it has to be loaded completely from main memory in every matrix multiplication, making for approximately 100GB of data transfers, plus what you need for the source and destination vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The maximum memory bandwith of the i3 is approximately 21 GB/s based on the Intel specs, but if you look around the web, you'll find that at most half of that is really available in reality. Thus, at the very least, you'd expect your benchmark to last 10 seconds, and your actual measurement of 45 seconds isn't so far off that mark.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the same time, you are also doing some 10 billion floating point multiplies and adds. Considering, say, 10 clock cycles for the combination, and 3 GHz clock rate, you'll come out at ~30 seconds. Of course, they can run concurrently with speculative memory loads if the cache is clever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all, I'd say you're not too far off the mark. What would you have expected?&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-08-19T19:55:11.180" LastActivityDate="2012-08-19T19:55:11.180" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1730" PostTypeId="2" ParentId="1727" CreationDate="2012-03-22T22:04:41.303" Score="4" Body="&lt;p&gt;How are you doing the matrix-vector multiply? A double-loop by hand? Or calls to BLAS? If you're using MKL, I would strongly recommend using the BLAS routines of the threaded version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Out of curiosity, you might also want to compile your own tuned version of &lt;a href=&quot;http://math-atlas.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;ATLAS&lt;/a&gt; and see how that does on your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following the discussion in the comments below, it turns out that your Intel Core i3-330M only has two &quot;real&quot; cores. The two missing cores are emulated with &lt;a href=&quot;http://en.wikipedia.org/wiki/Hyper-threading&quot; rel=&quot;nofollow&quot;&gt;hyperthreading&lt;/a&gt;. Since in hyperthreaded cores both the memory bus and the floating-point units are shared, you won't get any speedup if any of the two are a limiting factor. In fact, using four cores will probably even slow things down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of results do you get on &quot;only&quot; two cores?&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-03-23T15:51:05.047" LastActivityDate="2012-03-23T15:51:05.047" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1731" PostTypeId="1" AcceptedAnswerId="1733" CreationDate="2012-03-23T07:42:09.113" Score="3" ViewCount="1735" Body="&lt;p&gt;Many computational scientists that I know of, including myself for example, are not &lt;em&gt;computer scientists&lt;/em&gt;. As such they are often not very well aware more advanced techniques in OOP. On the other hand, most of advanced programmers are not computational scientists. They may not be well aware of what OOP features are essential or relevant when it comes to write a scientific computing software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering mainly &lt;code&gt;C++&lt;/code&gt; (but maybe also in general), what OOP features do you think are absolutely necessary in a number crunching software and why? What features do you think are generally considered good, but might cause unnecessary overhead? What are some of the &#xA;&quot;myths&quot; when it comes to overhead of using more advanced features?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I generally like to hear things regarding:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1 - templates&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2 - virtual functions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3 - smart pointers and garbage collectors&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;[EDIT:] 4 - operator overloading&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;5 - Boost library&lt;/strong&gt; (technically not an OOP feature but very popular &lt;code&gt;C++&lt;/code&gt; library)&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="414" LastEditDate="2012-03-23T09:13:58.937" LastActivityDate="2012-03-23T17:06:04.890" Title="What modern OOP features should a computational scientist use?" Tags="|libraries|c++|programming-paradigms|" AnswerCount="3" CommentCount="9" ClosedDate="2012-03-23T15:58:29.277" ContentLicense="CC BY-SA 3.0" />
  <row Id="1732" PostTypeId="2" ParentId="1731" CreationDate="2012-03-23T08:19:17.850" Score="1" Body="&lt;p&gt;&quot;Excessive&quot; use of templates in C++ can lead to humongous compilation errors and might lead to frustration with non-computer scientists. In form of &lt;a href=&quot;http://en.wikipedia.org/wiki/Expression_templates&quot; rel=&quot;nofollow&quot;&gt;expression templates&lt;/a&gt; it can significantly speed up your code though, and that's something you should be aware of.&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2012-03-23T08:19:17.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1733" PostTypeId="2" ParentId="1731" CreationDate="2012-03-23T08:41:23.197" Score="8" Body="&lt;p&gt;I'm not a big user of OOP in general, and this has its reasons. My thoughts on some of the features you mention:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Templates: Convenient when they work, a massive nightmare to debug when they don't. Furthermore, you don't really know what's going on &quot;behind the scenes&quot;, which can be a source of errors or inefficiencies.&lt;/li&gt;&#xA;&lt;li&gt;Virtual functions: Can be very convenient and elegant, but keep in mind that &lt;em&gt;calling&lt;/em&gt; that function is more expensive than calling a static function. Don't use these things in the innermost loop of a big calculation.&lt;/li&gt;&#xA;&lt;li&gt;Smart pointers: As with virtual functions, watch out for hidden overheads when assigning and dereferencing. Don't use in performance-critical parts of your code.&lt;/li&gt;&#xA;&lt;li&gt;Garbage collectors: Not necessarily an OOP feature (e.g. the &lt;a href=&quot;http://en.wikipedia.org/wiki/Boehm_garbage_collector&quot;&gt;Boehm GC&lt;/a&gt;), but an awesome thing nevertheless. Again, this can cause your program to pause and wait while cleaning up the memory, but you should avoid excessive creation/allocation and destruction/freeing of objects in performance-critical parts anyway, right?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One thing you left out is &lt;strong&gt;operator overloading&lt;/strong&gt;, which is an extremely convenient way of expressing things in a readable manner. Again, though, watch out for overheads in tight loops, and remember that in general, the compiler will not be able to optimize anything (think common sub-expression elimination, register variables, etc.). A good alternative are &lt;strong&gt;expression templates&lt;/strong&gt;, but writing your own can increase code complexity and give you weird bugs, think the same kind of mess as regular templates &lt;em&gt;can&lt;/em&gt; be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, using other people's templates, assuming they are tried and tested and mostly bug-free, can make life so much sweeter. Remember though: &lt;em&gt;can&lt;/em&gt;, as in not always.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, it's a performance/elegance trade-off. I would definitely recommend OOP, or anything similar, for the user interfaces and/or &quot;big picture&quot; parts of the code. I would, however, stay away from it in kernels or anything performance-critical due to the hidden overheads and the fact that the compiler usually can't optimize much.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-03-23T17:06:04.890" LastActivityDate="2012-03-23T17:06:04.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1734" PostTypeId="2" ParentId="1731" CreationDate="2012-03-23T08:42:14.967" Score="4" Body="&lt;p&gt;I think that the most important lesson for scientists that start programming is &quot;&lt;strong&gt;There is a very high probability that the programming problem that you are struggling with has already been solved by a computer scientist&lt;/strong&gt;&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to refer here to &quot;Design Patterns&quot;: a lot of typical interactions between objects, their behaviour and such, is well captured in a limited number of patterns that have a reference implementation in many OO programming languages like C++, Java, Python, ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The&lt;/em&gt; reference for design patterns is &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0201633612&quot; rel=&quot;nofollow&quot;&gt;Design Patterns&lt;/a&gt; by &quot;The Gang-of-Four&quot; (commonly referred to as the GoF). It's complete, but not easy to read in the evening with a glass of wine. A more tutorial approach is &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0596007124&quot; rel=&quot;nofollow&quot;&gt;Head First Design Patterns&lt;/a&gt;, which I can recommend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when starting a programming job, first do the OOP analysis (identify objects, behaviours, relations), then identify patterns and implement these using their reference implementation. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-03-23T08:42:14.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1736" PostTypeId="1" AcceptedAnswerId="1742" CreationDate="2012-03-23T11:08:16.567" Score="0" ViewCount="1487" Body="&lt;p&gt;As I see it, there are 3 ways to solve unstructured dense system of equations: GMRES, QMR and LU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone done a comparison for these three? As far as I know, LU is the preferred choice and it is fairly popular for solving unstructured systems. I am trying to implement GMRES and QMR as a substitute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: &lt;em&gt;To limit scope of the question&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Has anyone published a comparison between GMRES and QMR? If not, has anyone published a good survey on the convergence of any of the two? Any resources towards this?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="529" LastEditorUserId="529" LastEditDate="2012-03-23T16:40:21.473" LastActivityDate="2012-03-23T17:21:08.770" Title="A Comparison between GMRES, QMR and LU for Dense Matrices" Tags="|linear-algebra|matrix|iterative-method|krylov-method|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1740" PostTypeId="1" CreationDate="2012-03-23T14:52:24.097" Score="1" ViewCount="1229" Body="&lt;p&gt;I am trying to find the minimum of the function using GNU scientific library, package Multidimensional Minimization. The method I am using is Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm which is implemented in function &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gsl_multimin_fdfminimizer_vector_bfgs2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, after the first iteration with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gsl_multimin_fdfminimizer_iterate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I receive an error with code 27: iteration is not making progress towards solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried several initial points and different tolerance/step combinations, but still the same error pops up. Could you please tell me what could be the culprit here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function I am trying to minimize is Gupta-Potential for Aluminum 13 atoms cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also noticed that forces calculated on the first step are the SAME as gradients after first interation. Should it be like that? Shouldn't step come into the picture somehow? &#xA;I use &lt;/p&gt;&#xA;&#xA;&lt;p&gt;gsl_multimin_fdfminimizer_set (s, &amp;amp;my_func, x, 0.001, 1e-10);&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And function (Gupta potential) does NOT change after iteration (I print it before and after) though gradients are big. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;// HERE IS MY GUPTA POTENTIAL//&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double&#xA;      my_f (const gsl_vector *v, void *params)&#xA;     {&#xA;       double *p = (double *)params;&#xA;        int ii,jj;&#xA;        double vv = 0;&#xA;        double sum1 = 0;&#xA;        double sum2 = 0;&#xA;&#xA;          for(ii=1;ii&amp;lt;=NN;ii++){&#xA;          X[ii]=gsl_vector_get (v, 3*ii-3);&#xA;          Y[ii]=gsl_vector_get (v, 3*ii-2);&#xA;          Z[ii]=gsl_vector_get (v, 3*ii-1 );&#xA;         }&#xA;&#xA;        for(ii = 1; ii &amp;lt;= NN; ii++){&#xA;                sum1 = 0;&#xA;                for(jj = 1; jj &amp;lt;= NN; jj++){&#xA;&#xA;                        if(jj!=ii){&#xA;                                sum1 += uA*exp(0 - uP*((rr(ii,jj)/uR0) - 1.));&#xA;                        }&#xA;                }&#xA;                sum2 = 0;&#xA;                for(jj = 1; jj &amp;lt;= NN; jj++){&#xA;                        if(jj!=ii){&#xA;                                sum2 += uXi*uXi*exp(0 - 2*uQ*((rr(ii,jj)/uR0) - 1.));&#xA;                        }&#xA;                }&#xA;                vv += sum1 - sqrt(sum2);&#xA;        }&#xA;        return vv;&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;// HERE ARE MY GRADIENT FUNCTIONS//&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double dVdr(int ii, int jj){&#xA;        double vr;&#xA;        double sum1 = 0;&#xA;        double sum2 = 0;&#xA;        int kk;&#xA;&#xA;        vr = 0. - 2.*uA*(uP/uR0)*exp(0. - uP*((rr(ii,jj)/uR0) - 1.));&#xA;&#xA;        for(kk = 1; kk &amp;lt;= NN; kk++){&#xA;                if(kk != ii){&#xA;                        sum1 += uXi*uXi*exp(0 - 2*uQ*((rr(ii,kk)/uR0) - 1.));&#xA;                }&#xA;                if(kk != jj){&#xA;                        sum2 += uXi*uXi*exp(0 - 2*uQ*((rr(kk,jj)/uR0) - 1.));&#xA;                }&#xA;        }&#xA;&#xA;        vr -= 0.5*uXi*uXi*(0. - 2.*uQ/uR0)*exp(0. - 2.*uQ*((rr(ii,jj)/uR0) - 1.))*( (1./sqrt(sum1)) + (1./sqrt(sum2)) );&#xA;&#xA;        return vr;&#xA;}&#xA;&#xA;//-----------------------------------------------&#xA; void Force(int jj, double &amp;amp; fx, double &amp;amp; fy, double &amp;amp; fz){&#xA;        int mm;&#xA;        double dxx, dyy, dzz, r;&#xA;        fx = 0;&#xA;        fy = 0;&#xA;        fz = 0;&#xA;   for(mm = 1; mm &amp;lt;= NN; mm++){&#xA;&#xA;    if(mm != jj){&#xA;                        dxx = X[mm] - X[jj];&#xA;                        dyy = Y[mm] - Y[jj];&#xA;                        dzz = Z[mm] - Z[jj];&#xA;&#xA;     r = sqrt(dxx*dxx + dyy*dyy + dzz*dzz);&#xA;     fx += dVdr(mm,jj)*(dxx)/r;&#xA;     fy += dVdr(mm,jj)*(dyy)/r;&#xA;     fz += dVdr(mm,jj)*(dzz)/r;&#xA;   }&#xA;   }&#xA;}&#xA;//--------------------------------------------&#xA; void&#xA;     my_df (const gsl_vector *v, void *params,&#xA;            gsl_vector *df)&#xA;     {&#xA;       int ii;&#xA;       double fx,fy,fz;&#xA;       double *p = (double *)params;&#xA;&#xA;          for(ii=1;ii&amp;lt;=NN;ii++){&#xA;&#xA;          X[ii]=gsl_vector_get (v, 3*ii-3);&#xA;          Y[ii]=gsl_vector_get (v, 3*ii-2);&#xA;          Z[ii]=gsl_vector_get (v, 3*ii-1 );&#xA;         }&#xA;&#xA;          for(ii=1;ii&amp;lt;=NN;ii++){&#xA;          Force(ii,fx,fy,fz);&#xA;          gsl_vector_set(df, 3*ii-3, fx );&#xA;          gsl_vector_set(df, 3*ii-2, fy );&#xA;          gsl_vector_set(df, 3*ii-1, fz );&#xA;         }&#xA;     }&#xA;//----------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="808" LastEditorUserId="808" LastEditDate="2012-03-23T18:15:17.430" LastActivityDate="2012-03-25T12:22:26.640" Title="Multidimensional Minimization: GNU GSL C++ error code 27 for - iteration is not making progress towards solution" Tags="|optimization|software|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1741" PostTypeId="2" ParentId="1740" CreationDate="2012-03-23T16:22:17.610" Score="3" Body="&lt;p&gt;Most likely, the programmed gradient does not match the programmed objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check this by choosing a few random $x, p$ (including your starting point as $x$, if you provided one) and printing for $s_i=10^{20-i}$, $i=0:40$ the quotients &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$q_i = (f(x+ s_ip)-f(x))/(s_ig(x)^Tp)$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $g$ is your gradient. The sequence of $q$'s must converge to 1, well, almost, as sooner or later numerical instability sets in and forces the $q_i$ to be zero (and somewhat random one or two steps earlier). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not the case,either the routine for $f$ or that for $g$ is in error. You can check the individual components of $g$ by picking $p$ as the $(0,1)$-unit vectors in turn.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-03-25T12:22:26.640" LastActivityDate="2012-03-25T12:22:26.640" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1742" PostTypeId="2" ParentId="1736" CreationDate="2012-03-23T17:21:08.770" Score="3" Body="&lt;p&gt;There are many more ways to solve unstructured dense system of equations (to vary a well-known title, probably at least 19 different ways).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, CGS and BiCGStab are interesting iterative ones, and QR is a very important direct one - numerically more stable than LU with column pivoting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparisons are moot, as advantagens and disatvantages depend on the accuracy wanted, the level of robustness required, and (in randomized tests) the distribution of the matrix entries assumed. It also depends on the implementation (LU comes in different pivoting variants), whether the matrices are appropriately scaled, etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vanilla GMRES is trivially convergent, as after $n$ steps, it minimizes over the full space. (If you limit memory, then you have lots of different GMRES's, and comparisons are even more questionable.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For superlinear convergence under certain conditions, see &#xA;&lt;a href=&quot;http://ta.twi.tudelft.nl/nw/users/vuik/papers/vdV93V.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ta.twi.tudelft.nl/nw/users/vuik/papers/vdV93V.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The defining paper ''QMR: a quasi-minimal residual method for non-Hermitian linear systems'' on QMR gives in Section 6 a convergence analysis, again under certain conditions.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-23T17:21:08.770" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1743" PostTypeId="2" ParentId="1689" CreationDate="2012-03-23T18:10:40.500" Score="4" Body="&lt;p&gt;A general answer to the question is just impossible. This answer is limited to modeling the electrostatic term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have recently published a detailed comparison of different methods to compute atomic charges, using two sets of 100+ penta-alanine conformers. (&lt;a href=&quot;http://dx.doi.org/10.1021/ct200512e&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;) We tested both the robustness of the charges with respect to conformational changes and the accuracy with which they reproduce the dipole moment (as a simple way of checking the accuracy of the ESP). For the parametrization of a force field, you want a method to assign charges that is successful in both tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://dx.doi.org/10.1063/1.2715563&quot; rel=&quot;nofollow&quot;&gt;Hirshfeld-I method&lt;/a&gt; gives the best trade-off between the two tests. However, we also found that for ionic systems (e.g. oxides) Hirshfeld-I overestimates the ESP surrounding isolated molecules, so make sure you double check this for the system you are interested in. (&lt;a href=&quot;http://dx.doi.org/10.1021/jp210129r&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;) If your molecule is rigid, ESP-fitted charges are a better choice because the erratic conformational dependence is no longer an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, whatever method you use, without including atomic dipoles, one can never parameterize electrostatic interactions up to chemical accuracy, i.e. with errors below 1 kcal mol&lt;sup&gt;-1&lt;/sup&gt;. For rigid molecules, it is possible to make a multipole expansion of each atom-in-molecule density -- with any partitioning scheme such as Hirshfeld-I, Voronio, Mulliken or QTAIM -- to reach any desired level of accuracy for the electrostatic term. (&lt;a href=&quot;http://dx.doi.org/10.1016/0009-2614%2881%2985452-8&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="156" LastEditDate="2012-03-24T12:34:19.317" LastActivityDate="2012-03-24T12:34:19.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1744" PostTypeId="2" ParentId="1648" CreationDate="2012-03-23T18:18:58.667" Score="4" Body="&lt;p&gt;Use the Poisson equation in its differential form to obtain the Hartree potential. It has a nice form in spherical coordinates. See for example &lt;a href=&quot;http://dx.doi.org/10.1063/1.455005&quot; rel=&quot;nofollow&quot;&gt;the seminal paper of Becke&lt;/a&gt;, in which this problem is solved for the general polyatomic case without relying on spherical symmetry. &lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2012-03-23T18:18:58.667" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1745" PostTypeId="1" AcceptedAnswerId="1752" CreationDate="2012-03-23T19:17:11.777" Score="19" ViewCount="3016" Body="&lt;p&gt;The C++ language provides &lt;a href=&quot;https://en.wikipedia.org/wiki/Generic_programming&quot;&gt;generic programming&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Template_meta-programming&quot;&gt;metaprogramming&lt;/a&gt; through templates. These techniques have found their way into many large-scale scientific computing packages (e.g., &lt;a href=&quot;http://www.mpqc.org&quot;&gt;MPQC&lt;/a&gt;, &lt;a href=&quot;http://lammps.sandia.gov&quot;&gt;LAMMPS&lt;/a&gt;, &lt;a href=&quot;http://www.cgal.org&quot;&gt;CGAL&lt;/a&gt;, &lt;a href=&quot;http://trilinos.sandia.gov/&quot;&gt;Trilinos&lt;/a&gt;). But what have they actually contributed to scientific computing in value that goes beyond non-generic, non-meta languages like C or Fortran in terms of overall development time and usability for equal or adequate efficiency? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a scientific computing task, has generic and meta-programming through C++ templates demonstrated an improvement in productivity, expressivity, or usability as measured by any well-understood benchmarks (lines of code, person-effort, etc...)?  Correspondingly, what risks are associated with the use of C++ templates for generic and meta- programming?&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="9" LastEditDate="2012-03-24T09:36:32.830" LastActivityDate="2017-12-14T00:14:27.097" Title="To what extent is generic and meta-programming using C++ templates useful in computational science?" Tags="|performance|c++|programming-paradigms|" AnswerCount="6" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1746" PostTypeId="2" ParentId="1692" CreationDate="2012-03-24T02:55:22.147" Score="6" Body="&lt;p&gt;To write out the entire solution is impossible within reason. But here are some equations to reduce the system down a bit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U_{S}$ doesn't appear in any equations other than equations 1 and 2. Furthermore, these equations are a dependent set (equation 1 is -1 times equation 2), so equation 1 can be solved for $U_{S}$ in terms of all other variables, and equation 2 can be discarded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ U_S = H \frac{N \theta (\gamma+\zeta)}{C_A K_A + C_p +K_D} $$ &#xA;where $K_A = \gamma \rho_A \sigma_A+\kappa \rho_D \sigma_D \tau + \rho_A \sigma_A \zeta$ and $K_D = \gamma \rho_p \sigma_p+\kappa \rho_D \sigma_D  + \rho_p \sigma_p \zeta$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equation 7 is linear in all variables, and can be rearranged to solve for $D$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D = \frac{\kappa ( C_A \tau + C_p ) } {\gamma+\zeta}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting expression suggests that we should try to solve for any remaining variables in terms of $C_{A}$ and $C_{P}$; since we only have 6 independent equations, the best we can do is to reduce the system to one equation in two variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately, adding equations 3 and 5 together yields an equation that's linear in all variables, and can be solved for $U_{A}$ or $U_{P}$. Adding equations 4 and 6 together also yields an equation that's linear in all variables, and can be solved for $U_{A}$ or $U_{P}$ (whichever wasn't solved for when adding equations 3 and 5 together).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, we should have expressions for $U_{A}$ and $U_{P}$ in terms of $H$, $C_{A}$, and $C_{P}$ (because you can eliminate $D$ using the expression above). We've used equations 1, 2, 5, 6, and 7; we'll keep equations 3 and 4 because they're simpler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can use either equation 3 or 4 to solve for $H$ in terms of $C_{A}$ and $C_{P}$. Then, making all of the necessary substitutions, the remaining equation should be only in terms of $C_{A}$ and $C_{P}$. The roots of this equation will determine the steady states of the system; it may or may not be possible to find these roots symbolically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="1030" LastEditorUserId="276" LastEditDate="2012-03-26T21:27:35.153" LastActivityDate="2012-03-26T21:27:35.153" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1747" PostTypeId="2" ParentId="1745" CreationDate="2012-03-24T03:03:59.780" Score="3" Body="&lt;p&gt;What language features like &lt;code&gt;OOP&lt;/code&gt; do for scientific computing is make for more compact code statements which helps in understanding and using code better. For example, &lt;code&gt;FFT&lt;/code&gt; routines need to carry a large number of arguments for each function call making the code cumbersome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By using &lt;code&gt;module&lt;/code&gt; or &lt;code&gt;class&lt;/code&gt; statements only what is needed at the time of the call can be passed, as the rest of the arguments pertain to problem setup (i.e. size of arrays, and coefficients). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, I had &lt;code&gt;SUBROUTINE&lt;/code&gt; calls with 55 arguments (in &amp;amp; out) and I reduced that to 5 making the code better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's value.&lt;/p&gt;&#xA;" OwnerUserId="1030" LastActivityDate="2012-03-24T03:03:59.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1748" PostTypeId="2" ParentId="1745" CreationDate="2012-03-24T11:18:44.937" Score="2" Body="&lt;p&gt;You might find the paper &lt;a href=&quot;http://arxiv.org/abs/1104.1729&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/abs/1104.1729&lt;/a&gt; relevant to your question. It discusses expression templates (a particular application of template meta-programming used in scientific code) from the perspective of performance.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2012-03-24T11:18:44.937" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1749" PostTypeId="1" CreationDate="2012-03-24T16:43:22.853" Score="10" ViewCount="4335" Body="&lt;p&gt;What are recommended ways of doing nonlinear least squares,&#xA;min $\sum err_i(p)^2$,&#xA;with box constraints $lo_j &amp;lt;= p_j &amp;lt;= hi_j$ ?&#xA;It seems to me (fools rush in) that one could&#xA;make the box constraints quadratic, and minimize&#xA;$$ \sum_i err_i(p)^2 + C * \sum_j tub( p_j, lo_j, hi_j )^2 $$&#xA;where $tub( x, lo, hi )$ is the &quot;tub function&quot; shaped like \_&lt;em&gt;_&lt;/em&gt;_/,&#xA;$max( lo - x, 0, x - hi )$.&lt;br&gt;&#xA;Does this work in theory, work in practice ?&lt;br&gt;&#xA;(There seem to be many many theoretical papers on NLS+,&#xA;but my interest is practical &amp;mdash;&lt;br&gt;&#xA;real or realistic test cases would help me to choose among methods.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Experts, please add tags: &quot;least-squares&quot; ?)&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="123" LastEditDate="2012-03-24T18:01:54.720" LastActivityDate="2018-10-08T23:26:44.507" Title="Nonlinear least squares with box constraints" Tags="|optimization|constraints|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1751" PostTypeId="2" ParentId="1749" CreationDate="2012-03-24T17:33:38.827" Score="5" Body="&lt;p&gt;&lt;strong&gt;I would simply use the general-purpose NLP solver&lt;/strong&gt; &lt;a href=&quot;https://projects.coin-or.org/Ipopt&quot; rel=&quot;nofollow&quot;&gt;IPOPT&lt;/a&gt;. It is the most robust solver among those I have tried.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you have some very special requirements, there is no reason why you should insist on a problem specific solver that only works for NLS with box-constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A change in requirements&lt;/strong&gt; (e.g. adding nonlinear constraints) &lt;strong&gt;would cause a major headache with a problem specific solver.&lt;/strong&gt; You will have no such problems if you use the general-purpose IPOPT. &#xA;&lt;hr/&gt;&#xA;&lt;strong&gt;UPDATE: You can try L-BFGS with IPOPT&lt;/strong&gt;, see under Quasi-Newton in the documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution procedure can become faster at the expense of spoiling the remarkable robustness of IPOPT. &lt;em&gt;In my opinion&lt;/em&gt;, use the exact derivatives if they are available. I would start messing with approximations (such as L-BFGS) only if I had proven performance issues.&lt;/p&gt;&#xA;" OwnerUserId="679" LastEditorUserId="679" LastEditDate="2012-05-21T11:24:31.113" LastActivityDate="2012-05-21T11:24:31.113" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1752" PostTypeId="2" ParentId="1745" CreationDate="2012-03-24T20:21:53.067" Score="15" Body="&lt;p&gt;I think by and large, template metaprogramming has been found to be unusable in practice -- it compiles too slow, and the error messages we get are just impossible to decipher. The barrier to entry for newcomers is also just too high when using metaprogramming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, generic programming is an entirely different issue, as witnessed by Trilinos, deal.II (my own library), DUNE, and many other libraries -- expressing the same concept operating on different data types is sort of a no-brainer, and the community has largely accepted it as long as it stays within bounds that avoid the problems of metaprogramming. I think generic programming qualifies as an obvious success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, neither of these topics are immediately connected to OOP. OOP, again, is I would say, universally accepted by the scientific computing community. Even less than generic programming, it isn't a topic of debate: every successful library written in the past 15 years (whether written in C++, C or Fortran) uses OOP techniques.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-24T20:21:53.067" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1753" PostTypeId="2" ParentId="1745" CreationDate="2012-03-24T20:27:29.220" Score="15" Body="&lt;p&gt;Let me give an example based on experience.  Most libraries I use from a day to day basis use OOP in some way.  OOP is able to hide the complexity required for many domains, it is not a mechanism that really helps with performance.  What can happen is that a library is able to use specific optimizations based upon the object hierarchy, but for the most part is about hiding complexity from the user.  Look up Design Patterns, they are the mechanisms often employed to accomplish this complexity hiding.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take PETSc as an example.  PETSc uses an inspector/executor model of OOP where any of its algorithms looks at the available routines in a given object an chooses which to execute to accomplish the routine.  This allows a user to separate concerns, for example the matrix action can include any sort of blocked or optimized routine and be effectively used by numerous iterative solvers.  By giving the user the ability to specify their own data types and evaluations, they get a few important routines sped up and also have the entire library's functionality still available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example I'll give is FEniCS and deal.II. Both of these libraries use OOP to generalize over a large number of Finite Element Methods.  In both everything from element type, element order, quadrature representation, and so on is interchangable.  While both these libraries are &quot;slower&quot; than some special-purpose structured FEM codes, they are able to solve a wide variety of problems with much of the complexity of FEM unknown to the user.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My final example is Elemental.  Elemental is a new dense linear algebra library that has taken the difficulty of managing MPI communicators and data location to a very simple language construct.  The result is that if you have a FLAME serial code, by changing the datatypes you can also have an parallel code via Elemental.  Even more interesting you can play with the data distribution by setting on distribution equal to another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OOP should be thought of as a way to manage complexity, not a paradigm for competing with hand rolled assembly.  Also doing it poorly will result in a lot of overhead thus one must keep timing and updating the mechanisms they use it with.&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2012-03-24T20:27:29.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1754" PostTypeId="2" ParentId="1692" CreationDate="2012-03-24T20:41:48.093" Score="1" Body="&lt;p&gt;I would suggest looking at a homotopy method. While it is not symbolic, it will produce all the solutions of your problem.  For an easy library to check out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://homepages.math.uic.edu/~jan/PHCpack/phcpack.html&quot; rel=&quot;nofollow&quot;&gt;http://homepages.math.uic.edu/~jan/PHCpack/phcpack.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2012-03-24T20:41:48.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1755" PostTypeId="2" ParentId="1634" CreationDate="2012-03-25T14:04:37.230" Score="12" Body="&lt;p&gt;Others have commented on the issue of how difficult it is to write ET programs as well as the complexity of understanding error messages. Let me comment on the issue of compilers: It is true that a while back one of the big issues was finding a compiler that's compliant enough with the C++ standard to make everything work and make it work portably. As a consequence, we found lots of bugs -- I have 2-300 bug reports in my name, distributed over gcc, Intel icc, IBM xlC and Portland's pgicc. Consequently, the deal.II configuration script is a repository of a large number of compiler bug tests, primarily in the area of templates, friend declarations, namespaces, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, it turns out that the compiler makers have really gotten their act together: today, gcc and icc today pass all our tests and it's easy to write code that's portable between the two of them. I would say PGI isn't far behind but it has a number of quirks that don't seem to go away over the years. xlC, on the other hand, is a whole different story -- they fix a bug every 6 months, but despite filing bug reports with them for years, progress is extremely slow and xlC has never been able to compile deal.II successfully.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this all means is this: if you stick with the two big compilers, you can expect that they just work today. Since most computers and OSs today typically have at least one of them, that's enough. The only platform where things are more difficult is the BlueGene, where the system compiler typically is xlC, with all its bugs.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-25T14:04:37.230" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1756" PostTypeId="2" ParentId="1396" CreationDate="2012-03-25T15:43:28.073" Score="2" Body="&lt;p&gt;Discretize by choosing a basis of the subspace of localized functions where you expect the eigenvectors of interest to be. These define the columns of $P$.&#xA;Choose another basis of a slightly bigger subspace also containing slightly less localized functions and slightly more wiggly functions. These define the columns of $Q$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now calculate the matrices $H&amp;#39;:=Q^*HP$ and $K&amp;#39;:=Q^*P$, and solve the overdetermined eigenvalue problem $\|H&amp;#39;\phi-EK&amp;#39;\phi\|=\min$, using OEIG from&#xA;   &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/software/oeig/&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/software/oeig/&lt;/a&gt;&#xA;If the residiuals are not small enough, you can always increase the basis sizes.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-25T15:43:28.073" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1757" PostTypeId="1" AcceptedAnswerId="1760" CreationDate="2012-03-26T08:10:23.590" Score="1" ViewCount="290" Body="&lt;p&gt;I managed to reduce certain computational problem to the Gauss-Seidel solution of the following linear system: $$Ax=Ly,$$ where $A, L\in\mathbb{R}^{n\times n}$ are weighted Laplacian matrices (symmetric, positive semi-definite; negative off-diagonal entries, with rows(collumns) summing (in absolute values) to positive diagonal entries; matrix eigenvalue $0$ corresponds to $1_n$ eigenvector of the nullspace), and $x,y\in\mathbb{R}^{n\times 2}$ are vectors with the unknown $x$. The solution has the form $$x_i^{[k+1]} = \left.\left(b_i  - \sum_{j=1}^{i-1}a_{ij}x_j^{[k+1]} - \sum_{j=i+1}^{n}a_{ij}x_j^{[k]}\right)\middle/a_{ii}\right.,$$ where $b_i$ is the $i^{th}$ entry of $Ly$. Note that, with Gauss-Seidl, the update of $x_i$ takes effect immediately, &lt;em&gt;i.e.&lt;/em&gt;, calculation for the following $x_{i+1}$ is based on the new value of $x_i$ that has been computed just before. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose an &lt;em&gt;iteration&lt;/em&gt; consists of a single update of all $x_i$ in some arbitrary order. In other words, each $x_i$ is considered only once (and is updated only once) in an iteration. My question is: could it be guaranteed that after &lt;em&gt;a single&lt;/em&gt; iteration with initial $x^{[0]}=y$, the solution $x^{[1]}$ has all unique coordinates, &lt;em&gt;i.e.&lt;/em&gt;, there are &lt;strong&gt;not&lt;/strong&gt; two rows of $x^{[1]}$ that are equal?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could assume that the initial $x^{[0]}=y$ has non-unique coordinates. If the uniqueness cannot be resolved this way, I would appreciate a suggestion on the coordinate traversal order to increase the chance of achieving uniqueness (&lt;em&gt;i.e.&lt;/em&gt;, no two coordinates take the same value).&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-03-28T14:36:57.573" LastActivityDate="2012-03-28T14:36:57.573" Title="Unique coordinates (solutions) in a single Gauss-Seidel iteration" Tags="|linear-algebra|matrix|iterative-method|computational-geometry|linear-solver|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1758" PostTypeId="1" CreationDate="2012-03-26T12:20:01.653" Score="9" ViewCount="163" Body="&lt;p&gt;In my master thesis I used an Algorithm called &lt;em&gt;Approximative Dynamic Programming&lt;/em&gt; [1] to solve equations of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\max_{\pi}\mathbb{E}^{\pi}\left\{\sum_{t=0}^{T}\gamma^tC_t^{\pi}(S_t,A_t^{\pi}(S_t))\right\}.&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It uses Monte-Carlo sampling and an approximation $\overline{V}_t$ of the value function to come around the curse of dimensionality. As a decision function serves a convex linear program&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\hat{v}_t^n=\max_{a_t\in A_t^n}\left(C_t(S_t^n,a_t)+\gamma \overline{V}_{t+1}^{n-1}(S^M(S_t^n,a_t))\right),&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the update is done by using a stepsize $\alpha$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\overline{V}_t^n(S_t^n)=(1-\alpha_{n-1})\overline{V}_t^{n-1}(S_t^n)+\alpha_{n-1}\hat{v}_t^n.&#xD;&#xA;$$&#xA;I wondered now if they exists similar algorithms, despite the many flavours of this algorithm, to deliver high quality solutions in reasonable time or if the used principle is the only one to solve such kind of problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers,&#xA;Reza&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] POWELL, W. : Approximate Dynamic Programming: Solving the curses of dimensionality. Bd. 703. Wiley-Blackwell, 2007&lt;/p&gt;&#xA;" OwnerUserId="1208" LastEditorUserId="1208" LastEditDate="2012-03-28T10:17:49.190" LastActivityDate="2012-03-28T10:17:49.190" Title="Fast algorithms to solve Markov Decision Processes" Tags="|optimization|statistics|linear-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1759" PostTypeId="2" ParentId="1678" CreationDate="2012-03-26T14:23:31.547" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I read &lt;a href=&quot;http://web.eecs.utk.edu/~dongarra/etemplates/node193.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; that research was heading towards a inverse-iteration method that guarantees orthogonality with $\mathcal{O}(N)$ complexity.  I'd be interested in hearing about that or other advances.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(I wanted to just make a few comments since I don't have the time to write out details, but it got rather big for the comment box.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That I believe would be the &lt;a href=&quot;http://dx.doi.org/10.1145/1186785.1186788&quot; rel=&quot;nofollow&quot;&gt;MRRR (multiple relatively robust representations) algorithm&lt;/a&gt; of Dhillon and Parlett. This is rooted in previous work by Fernando, which in turn was inspired by a problem posed by Jim Wilkinson in his monumental book on eigenvalue problems. The &quot;inverse iteration&quot; portion for obtaining singular vectors is rooted in the concept of &lt;a href=&quot;http://dx.doi.org/10.1016/S0024-3795%2897%2980053-5&quot; rel=&quot;nofollow&quot;&gt;&quot;twisted factorizations&quot; by Fernando&lt;/a&gt;, which make use of factoring matrices into $\mathbf L\mathbf D\mathbf L^\top$ and $\mathbf U\mathbf D\mathbf U^\top$ decompositions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;singular value&quot; portion of the algorithm, on the other hand, comes from the &lt;a href=&quot;http://dx.doi.org/10.1016/S0024-3795%2800%2900010-0&quot; rel=&quot;nofollow&quot;&gt;(shifted) differential quotient difference (dqd(s)) algorithm&lt;/a&gt;, which is a culmination of previous work by &lt;a href=&quot;http://dx.doi.org/10.1007/s002110050024&quot; rel=&quot;nofollow&quot;&gt;Fernando, Parlett&lt;/a&gt;, &lt;a href=&quot;http://dx.doi.org/10.1137/0911052&quot; rel=&quot;nofollow&quot;&gt;Demmel and Kahan&lt;/a&gt; (with inspiration from Heinz Rutishauser).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you might know SVD methods usually proceed with a bidiagonal decomposition first before the singular values are obtained from the bidiagonal matrix. Unfortunately I'm not too updated on the current best method for the front-end bidiagonal decomposition; last I checked, the usual method is to start with QR decomposition with column pivoting and then apply orthogonal transformations appropriately to the triangular factor to finally obtain the bidiagonal decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that I've been skimpy with the details; I'll try to flesh out this answer further once I have access to my library...&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2012-03-26T14:23:31.547" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1760" PostTypeId="2" ParentId="1757" CreationDate="2012-03-26T16:33:11.960" Score="2" Body="&lt;p&gt;The order in which you update the new solution has a big impact on the values produced from the initial data to the first approximation.  Some updated values will be the same as the jacobi iteration, while others will contain the latest information &lt;em&gt;a la Gauss-Seidel&lt;/em&gt;.  Given a specific ordering, there may be two values that are the same if the initial vector $x_0$ has non-unique values.  But for two different orderings, if the the solution vector is large, and the order of updating the gauss-seidel formula is random, the more unlikely it is that you will produce similar &lt;em&gt;1st iterations&lt;/em&gt;.  I'm not sure if there is a proof for that different orderings produce unique 1st iterations, but even if it is possible for them to be non-unique, the probability is very unlikely.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-26T19:02:47.500" LastActivityDate="2012-03-26T19:02:47.500" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1761" PostTypeId="2" ParentId="1757" CreationDate="2012-03-26T17:03:29.317" Score="-1" Body="&lt;p&gt;This question is ill-posed.  Gauss-Seidel is an iterative method for solving a linear system of equations that is guaranteed to converge for certain matrices.  If the solution to the linear system of equations contains non-unique values, then Gauss-Seidel will converge to non-unique values.  The iteration number and order of applying the updates are irrelevant.  &lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-03-26T17:03:29.317" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1762" PostTypeId="1" AcceptedAnswerId="1776" CreationDate="2012-03-26T18:37:46.773" Score="5" ViewCount="481" Body="&lt;p&gt;What methods are suggested to solve problems of the form $\min || {A} x - y ||_k$, subject to $x^T P x \leq c$, and/or $x^T Q x = d$?&lt;/p&gt;&#xA;" OwnerUserId="1113" LastEditorUserId="1113" LastEditDate="2012-03-27T08:30:46.710" LastActivityDate="2012-03-27T17:00:26.673" Title="Linear regression with quadratic constraints" Tags="|regression|constraints|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1763" PostTypeId="1" CreationDate="2012-03-26T19:40:24.690" Score="20" ViewCount="1238" Body="&lt;p&gt;I often find myself writing very similar code for one, two, and three dimensional versions of a given operation/algorithm.  Maintaining all of these versions can become tedious.  Simple code generation works fairly well, but it seems as thought there must be a better way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a relatively simple way to write an operation once and have it generalize to higher or lower dimensions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One concrete example is: suppose I need to compute the gradient of a velocity field in spectral space.  In three dimensions, the Fortran loops would look something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do k = 1, n&#xA;  do j = 1, n&#xA;    do i = 1, n&#xA;      phi(i,j,k) = ddx(i)*u(i,j,k) + ddx(j)*v(i,j,k) + ddx(k)*w(i,j,k)&#xA;    end do&#xA;  end do&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the &lt;code&gt;ddx&lt;/code&gt; array is appropriately defined.  (One could also do this with matrix multiplies.)  The code for a two-dimensional flow is almost exactly the same, except: the third dimension is dropped from the loops, indexes, and number of components.  Is there a better way of expressing this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example is: suppose I have fluid velocities defined point-wise on a three dimensional grid.  To interpolate the velocity to an arbitrary location (ie, not corresponding to grid points), one can use the one-dimensional &lt;a href=&quot;http://en.wikipedia.org/wiki/Neville_algorithm&quot;&gt;Neville&lt;/a&gt; algorithm successively over all three dimensions (ie, dimensional reduction).  Is there an easy way to do dimensional reduction given a one-dimensional implementation of a simple algorithm?&lt;/p&gt;&#xA;" OwnerUserId="486" LastActivityDate="2021-12-27T22:07:46.683" Title="How do I write dimensionally agnostic code?" Tags="|algorithms|software|fortran|" AnswerCount="7" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1764" PostTypeId="2" ParentId="1757" CreationDate="2012-03-26T19:43:54.937" Score="2" Body="&lt;p&gt;Application of Gauss-Seidel is a nonsingular affine operator (for matrices and orderings in which Gauss-Seidel is convergent), let's call the linear part $G$. That is, with $A = L + D + U$, we have $x_1 = (L+D)^{-1}(b - U x_0) = \tilde b + G x_0$. Without loss of generality, let's take $b = 0$, in which case $\tilde b = (L+D)^{-1} b = 0$, thus $x_1 = G x_0$ with $G = (L+D)^{-1} U$ nonsingular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand your question, you are asking whether observing $G x_0 = G y_0$ (i.e. identical results after one iteration of Gauss-Seidel) implies that $x_0 = y_0$. Well, $G (x_0 - y_0) = 0$ is only possible if $x_0 - y_0 = 0$ or if $G$ is singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that Gauss-Seidel may damp some modes very fast so you can get iterates that are very close even though the initial vectors were different.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-03-27T13:38:58.287" LastActivityDate="2012-03-27T13:38:58.287" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1765" PostTypeId="2" ParentId="1763" CreationDate="2012-03-26T20:06:11.520" Score="5" Body="&lt;p&gt;You can write an algorithm for a $n_1 \times n_2 \times n_3$ grid and then specialize to lower dimensions by setting some $n_j=1$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-26T20:06:11.520" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1766" PostTypeId="2" ParentId="1763" CreationDate="2012-03-26T21:36:39.237" Score="14" Body="&lt;p&gt;You look at how deal.II (&lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dealii.org/&lt;/a&gt;) does it -- there, dimension independence lies at the very heart of the library, and is modeled as a template argument to most data types. See, for example, the dimension-agnostic Laplace solver in the step-4 tutorial program:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_4.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_4.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#why-use-templates-for-the-space-dimension&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#why-use-templates-for-the-space-dimension&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="25840" LastEditDate="2017-10-31T20:14:04.560" LastActivityDate="2017-10-31T20:14:04.560" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1767" PostTypeId="2" ParentId="1763" CreationDate="2012-03-26T21:42:57.517" Score="11" Body="&lt;p&gt;This can be accomplished in any language with the following rough mental prototype:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create a list of the extents of each dimension (something like shape() in MATLAB I think)&lt;/li&gt;&#xA;&lt;li&gt;Create a list of your current location in each dimension.&lt;/li&gt;&#xA;&lt;li&gt;Write a loop over each dimension, containing a loop over whos size changes based on the outer loop.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From there, it is a question of battling the syntax of your certain language to keep your code nd-compliant.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having written an &lt;a href=&quot;https://github.com/meawoppl/ndsolver&quot;&gt;n-dimensional fluid-dynamics solver&lt;/a&gt;, I have found that it is helpful to have a language that supports unpacking a list like object as the arguments of a function.  I.e. a = (1,2,3) f(a*) -&gt; f(1,2,3).  Additionally advanced iterators (such as &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndenumerate.html&quot;&gt;ndenumerate&lt;/a&gt; in numpy) make code an order of magnitude cleaner.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-03-26T21:42:57.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1768" PostTypeId="1" CreationDate="2012-03-26T22:03:08.083" Score="4" ViewCount="283" Body="&lt;p&gt;&lt;em&gt;I asked this question on &lt;a href=&quot;https://stackoverflow.com/q/9711539/199217&quot;&gt;stackoverflow&lt;/a&gt; a few weeks ago, but have not gotten a useable answer, even after a colleague added a 150pt bounty.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to query the doi for each record in a table of citations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a &lt;a href=&quot;https://docs.google.com/spreadsheet/pub?key=0Ai_PDCcY5g2JdGN0VGxCd2NsNk5aQ0puUGlQZGczRkE&amp;amp;output=html&quot; rel=&quot;nofollow noreferrer&quot;&gt;table&lt;/a&gt; (&lt;a href=&quot;https://docs.google.com/spreadsheet/pub?key=0Ai_PDCcY5g2JdGN0VGxCd2NsNk5aQ0puUGlQZGczRkE&amp;amp;output=csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;csv version&lt;/a&gt;) that includes the last name of the first author, the title, journal, year, and page numbers for each citation. I expect most (&gt;90%) of the rows to have a valid doi, but using the [simple query uploader at CrossRef], I get a hit rate of ~7%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also an XML-based query format that allows fuzzy matching, but this seems to have a limit on the number of queries that can be sent at one time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The table is currently in MySQL, but starting with the .csv file would be a great help. &lt;/p&gt;&#xA;" OwnerUserId="452" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-03-28T00:20:10.793" Title="reverse-lookup Digital Object Identifier given table of citations?" Tags="|publications|" AnswerCount="1" CommentCount="6" ClosedDate="2012-03-27T16:36:39.450" ContentLicense="CC BY-SA 3.0" />
  <row Id="1769" PostTypeId="2" ParentId="1768" CreationDate="2012-03-27T01:07:37.833" Score="4" Body="&lt;p&gt;This is an open problem.  There are better and worse ways to attack it but, start by reading &lt;a href=&quot;http://kcoyle.net/jal_34_5.html&quot; rel=&quot;nofollow&quot;&gt;Karen Coyle's summary&lt;/a&gt; of this problem.  The bibliography attached to that article as well is excellent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, the problem of quantifying sameness between two bibliographic records is hard, and a substantial amount machine-learning research has centered around this topic.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2012-03-28T00:20:10.793" LastActivityDate="2012-03-28T00:20:10.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1770" PostTypeId="2" ParentId="1762" CreationDate="2012-03-27T10:08:48.097" Score="3" Body="&lt;p&gt;Without the equality constraint, the problem is convex, and any standard interior point convex optimization package can be applied to solve this efficiently, such as the high level modeling software &lt;a href=&quot;http://cvxr.com/cvx/&quot; rel=&quot;nofollow&quot;&gt;CVX&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the equality constraint, the problem is no longer convex. However, you may approximate the solution by repeatedly solving a bunch of convex problems. One simple approach is to approximate the ellipsoid $x^TQx=d$ as a polyhedron and solve the optimization problem on all the facets of the polyhedron, then taking the best solution among all facets.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-03-27T10:08:48.097" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1771" PostTypeId="1" CreationDate="2012-03-27T12:24:36.693" Score="13" ViewCount="3629" Body="&lt;p&gt;I currently trying to cheaply compute a good rank estimate for a matrix $A$. Therefore I compute a columnt pivoting QR decompostion using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[Q,R,E]=qr(A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in Matlab. I estimate the rank of $A$ using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tol = size(A,n)*eps*norm(A,'fro'); &#xA;r = sum(abs(diag(R))&amp;gt;tol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works fine and a plot over all diagonal entries of R looks like:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/IK0LC.jpg&quot; alt=&quot;plot(sort(abs(diag(R)),1,&amp;#39;descend&amp;#39;),&amp;#39;r*&amp;#39;)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If port the whole algorithm to C/Fortran I replace [Q,R,E]=qr(A) using DGEQP3 from LAPACK, which also computes a column pivoting QR decomposition. But if I use the same estimate for the rank I mostly get something  wrong. The same plot for the $R$ produced from DGEQP3 looks like&#xA;&lt;img src=&quot;https://i.stack.imgur.com/1R0Qt.jpg&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input matrix is exactly the same for both experiments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question now is on which LAPACK function does the column pivoting QR decomposition from Matlab rely on?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help, &#xA;Grisu&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; DGEQPF gives the same wrong result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The input matrix $A$ is dense and build as $E+sign(E,F)$ &lt;/li&gt;&#xA;&lt;li&gt;$A$ is availble here: &lt;a href=&quot;http://www-e.uni-magdeburg.de/makoehle/A.mtx.gz&quot; rel=&quot;noreferrer&quot;&gt;http://www-e.uni-magdeburg.de/makoehle/A.mtx.gz&lt;/a&gt; (MatrixMarket Format)&lt;/li&gt;&#xA;&lt;li&gt;The &lt;em&gt;wrong&lt;/em&gt; $R$:  &lt;a href=&quot;http://www-e.uni-magdeburg.de/makoehle/R_wrong.mtx.gz&quot; rel=&quot;noreferrer&quot;&gt;http://www-e.uni-magdeburg.de/makoehle/R_wrong.mtx.gz&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;I used LAPACK 3.4.0 with OpenBlas/GotoBLAS (64 bit)&lt;/li&gt;&#xA;&lt;li&gt;Matlab 7, 2007b, 2010b Linux 32bit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit3:&#xA; - Using GDB I found out, that Matlab 2010b calls DGEQP3:&#xA; #3  0xaa46ce2f in dgeqp3_ () from /usr/ubuntu10.04/matlabr2010b/bin/glnx86/../../bin/glnx86/../../bin/glnx86/mllapack.so&#xA; Why do I get the wrong result using LAPACK(3.4.0 inlcude the fixes mentioned in Working Note 176)?&lt;/p&gt;&#xA;" OwnerUserId="569" LastEditorUserId="569" LastEditDate="2012-03-28T11:56:39.520" LastActivityDate="2012-07-06T07:04:11.847" Title="What is the corresponding LAPACK function behind Matlab [Q,R,E]=qr(A)?" Tags="|matlab|lapack|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1772" PostTypeId="1" AcceptedAnswerId="1773" CreationDate="2012-03-27T14:48:51.503" Score="11" ViewCount="7008" Body="&lt;p&gt;I'm working on a problem that can be parallelized by using a single mpi_allgather operation or one mpi_scatter and one mpi_gather operation.  These operations are called within a while loop, so they may be called many times.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the implementation with an MPI_allgather scheme, I'm gathering a distributed vector onto all processes for duplicate matrix solving.  In the other implementation, I gather the distributed vector onto a single processor (the root node), solve the linear system on this processor, and then scatter the solution vector back onto all the processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious to know if the cost of an allgather operation is significantly more than the scatter and gather operations combined.  Does the length of the message play a significant role in its complexity?  Does it vary between implementations of mpi?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-03-27T15:59:30.667" LastActivityDate="2018-01-30T06:12:23.793" Title="How does the computational cost of an mpi_allgather operation compare with a gather/scatter operation?" Tags="|algorithms|complexity|mpi|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="1773" PostTypeId="2" ParentId="1772" CreationDate="2012-03-27T15:25:33.297" Score="9" Body="&lt;p&gt;First, the exact answer depends on: (1) usage, i.e. function input arguments, (2) MPI implementation quality and details, and (3) the hardware you're using.  Often, (2) and (3) are related, such as when the hardware vendor optimizes MPI for their network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, fusing MPI collectives is better for smaller messages, since start-up costs can be nontrivial and the synchronization entailed by blocking collectives should be minimized if there is variation in compute time between calls.  For larger messages, the goal should be to minimize the amount of data being sent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in theory, &lt;code&gt;MPI_Reduce_scatter_block&lt;/code&gt; should be better than &lt;code&gt;MPI_Reduce&lt;/code&gt; followed by &lt;code&gt;MPI_Scatter&lt;/code&gt;, although the former is often implemented in terms of the latter, such that there is no real advantage.  There's a correlation between implementation quality and frequency of usage in most implementations of MPI, and vendors obviously optimize those functions for which this is required by the machine contract.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if one is on a Blue Gene, doing &lt;code&gt;MPI_Reduce_scatter_block&lt;/code&gt; using &lt;code&gt;MPI_Allreduce&lt;/code&gt;, which does more communication than &lt;code&gt;MPI_Reduce&lt;/code&gt; and &lt;code&gt;MPI_Scatter&lt;/code&gt; combined, is actually quite a bit faster.  This is something I recently discovered and is an interesting violation of the principle of performance self-consistency in MPI (this principle is described in more detail in &lt;a href=&quot;https://doi.org/10.1109/TPDS.2009.120&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Self-Consistent MPI Performance Guidelines&quot;&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the specific case of scatter+gather versus allgather, consider that in the former, all the data must go to and from a single process, which makes it the bottleneck, whereas in the allgather, data can flow in and out of all ranks immediately, because all ranks have some data to send to all other ranks.  However, sending data from all nodes at once is not necessarily a good idea on some networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the best way to answer this question is to do the following in your code and answer the question by experiment.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#ifdef TWO_MPI_CALLS_ARE_BETTER_THAN_ONE&#xA;  MPI_Scatter(..)&#xA;  MPI_Gather(..)&#xA;#else&#xA;  MPI_Allgather(..)&#xA;#endif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;An even better option is to have your code measure it experimentally during the first two iterations, then use whichever is faster for the remaining iterations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const int use_allgather = 1;&#xA;const int use_scatter_then_gather = 2;&#xA;&#xA;int algorithm = 0;&#xA;double t0 = 0.0, t1 = 0.0, dt1 = 0.0, dt2 = 0.0;&#xA;&#xA;while (..)&#xA;{&#xA;    if ( (iteration==0 &amp;amp;&amp;amp; algorithm==0) || algorithm==use_scatter_then_gather )&#xA;    {&#xA;        t0 = MPI_Wtime();&#xA;        MPI_Scatter(..);&#xA;        MPI_Gather(..);&#xA;        t1 = MPI_Wtime();&#xA;        dt1 = t1-t0;&#xA;    } &#xA;    else if ( (iteration==1 &amp;amp;&amp;amp; algorithm==0) || algorithm==use_allgather)&#xA;    {&#xA;        t0 = MPI_Wtime();&#xA;        MPI_Allgather(..);&#xA;        t1 = MPI_Wtime();&#xA;        dt2 = t1-t0;&#xA;    }&#xA;&#xA;    if (iteration==1)&#xA;    {&#xA;       dt2&amp;lt;dt1 ? algorithm=use_allgather : algorithm=use_scatter_then_gather;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2018-01-30T06:12:23.793" LastActivityDate="2018-01-30T06:12:23.793" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1774" PostTypeId="1" CreationDate="2012-03-27T15:55:23.743" Score="11" ViewCount="7380" Body="&lt;p&gt;The essence of my question is the following: I have a system of two ODEs. One has an initial-value constraint and the other has a final-value constraint. This can be thought of as a single system with an initial-value constraint on some variables and a final-value constraint on the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the details:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to use a continuous-time finite-horizon LQR controller to drive a linear dynamical system. I'd like to continue using the Python ecosystem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system is in the form $\dot{x}(t) = Ax(t) + Bu(t)$, subject to $x(0)=x_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LQR solution generates a matrix $K(t)$ such that the optimal control input u(t), linear in $x(t)$, is $u(t) = K(t)x(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K(t) = R^{-1}  B^T  P(t)$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $P(t)$ is the solution to a continuous time Riccati differential equation (note that this $P(t)$ is a matrix)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot{P}(t) = -A^T  P(t) - P(t) A + P(t)  B R^{-1}  B^T P(t) + Q$ subject to $P(t_f) = Q$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A$, $B$, $x_0$, $Q$, $Qf$, $R$, $t_f$ are all given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In English: you have some dynamical system that starts in state $x_0$. The LQR controller generates a feedback matrix to use between time $0$ and $t_f$ ($t_f$ is commonly called the time-horizon of the problem)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the two ODEs are coupled only in one direction -- the solution to $P(t)$ does not depend on $x(t)$. Therefore one way to solve the problem is to reverse the Riccati equation in order to turn the final-value problem into an initial-value problem and find a numerical solution between time $0$ and $t_f$ using a standard ODE integrator. I can then use this numerical solution to find $x(t)$. This concerns me because the numerical ODE solver for x(t) will not necessarily sample the ODE at the same times as the times in the numerical solution to $P(t). Maybe there is some clever way to enforce this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other way I foresee of solving the problem is to solve the sytem together, but I don't know how to deal with the mix of initial-value and final-value constraints. Are these problems computationally heavy to solve? Can I do it in SciPy / Python?&lt;/p&gt;&#xA;" OwnerUserId="1217" LastActivityDate="2012-07-02T03:04:32.720" Title="solving coupled ODEs with initial-value and final-value constraints" Tags="|python|ode|boundary-conditions|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1775" PostTypeId="2" ParentId="1771" CreationDate="2012-03-27T16:15:18.103" Score="8" Body="&lt;p&gt;There are two issues at hand here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; dense or sparse?&lt;/li&gt;&#xA;&lt;li&gt;Do you have the same software stack as MATLAB's internal libraries?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Dense or sparse?&lt;/h2&gt;&#xA;&lt;p&gt;MATLAB no longer explicitly mentions the LAPACK routines it calls to obtain a QR factorization if &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is dense. If the &lt;a href=&quot;http://www.mathworks.com/help/releases/R2009a/techdoc/index.html?/help/releases/R2009a/techdoc/matlab.html&amp;amp;http://www.mathworks.com/help/releases/R2009a/helpdesk.html&quot; rel=&quot;noreferrer&quot;&gt;information in the documentation for MATLAB R2008b&lt;/a&gt; also holds for later releases, then MATLAB calls &lt;code&gt;DGEQP3&lt;/code&gt; from LAPACK when you call &lt;code&gt;[Q,R,E] = qr(A)&lt;/code&gt;. If &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is sparse, then MATLAB calls &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/SPQR/&quot; rel=&quot;noreferrer&quot;&gt;SuiteSparseQR&lt;/a&gt;, out of Tim Davis' group, which is bundled with UMFPACK in the SuiteSparse library.&lt;/p&gt;&#xA;&lt;h2&gt;Do you have the same software stack as MATLAB's internal libraries?&lt;/h2&gt;&#xA;&lt;p&gt;Probably not, which may be one reason you're getting different results.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/newsreader/view_thread/297843&quot; rel=&quot;noreferrer&quot;&gt;I ran into this issue&lt;/a&gt; when unit testing a library I was writing that used QR factorizations. I used MATLAB to prototype my work and got different results than using LAPACK or NumPy. As far as I can tell, because MathWorks doesn't make this information easy to find, MATLAB uses a verson of LAPACK no earlier than version 3.1.1, and Intel's MKL BLAS library (for Windows, Intel Mac, and Linux) version 9.1 or higher (see &lt;a href=&quot;http://www.mathworks.com/help/techdoc/rn/brhzwro-1.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;). I couldn't find anything about the version of SuiteSparse MATLAB uses. By digging around online or looking at the library files for your system, you may be able to glean additional information. You could try changing the libraries that MATLAB links to in order to be able to compare with the same libraries across software packages; &lt;a href=&quot;http://www.stanford.edu/%7Eechu508/matlab.html&quot; rel=&quot;noreferrer&quot;&gt;Eric Chu provides a nice write-up&lt;/a&gt; that shows at least how you can replace MATLAB's BLAS library with your own (of course, you do this at your own risk). He suggests that you can do the same thing with LAPACK as well. It may even be possible to replace the version of SuiteSparse that MATLAB uses with your own version.&lt;/p&gt;&#xA;&lt;p&gt;LAPACK versions change, and so do BLAS versions; they may use different algorithms from version to version, or different ordering conventions, even though &lt;span class=&quot;math-container&quot;&gt;$A = QR$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; orthogonal and &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is upper triangular, regardless of version. These changes make reproducibility a challenge. Even the tolerance you use for your numerical rank determination is a judgment call; you appear to be using a standard tolerance.&lt;/p&gt;&#xA;&lt;p&gt;I ended up using NumPy to prototype my results for QR factorization, because it uses the system BLAS and LAPACK libraries. NumPy and SciPy isn't a drop-in replacement for MATLAB, because the two libraries combined lack some of MATLAB's functionality, but for this particular linear algebra task, Python + NumPy + SciPy + Matplotlib should work well.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-03-27T16:15:18.103" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1776" PostTypeId="2" ParentId="1762" CreationDate="2012-03-27T16:40:57.773" Score="3" Body="&lt;p&gt;In case of an inequality constraint only and semidefinite $P$, your problem is convex, and there may be better alternatives (CVX mentioned in the answer by Victor Liu, or the methods of arXiv:1009.2065 (which has at thre end a reference to a an implementation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $1&amp;lt;k&amp;lt;\infty$, you can use standard nonlinear programming software (see, e.g. &lt;a href=&quot;http://neos-server.org/neos/&quot; rel=&quot;nofollow&quot;&gt;http://neos-server.org/neos/&lt;/a&gt; ); in case of an equality constraint or an inequality constraint with indefinite $P$, you are very unlikely to do better with other codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, if $k=1$ or $k=\infty$, you need to introduce an extra variable $z$ for an upper bound on the objective, and minimize $z$ subject to your constraints and a bunch of linear constraints equvalent to $\|Ax-b\|_k\le z$,&#xA;and again use standard nonlinear programming software. Alternatively, you might use a nonsmooth optimization code; don't know what to recommend there, but a list of codes is at &#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-03-27T17:00:26.673" LastActivityDate="2012-03-27T17:00:26.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1777" PostTypeId="2" ParentId="1772" CreationDate="2012-03-27T17:40:33.207" Score="5" Body="&lt;p&gt;Jeff's absolutely right about the only way to be sure is to measure - we're scientists, after all, and this is an empirical question - and gives excellent advice about how to implement such measurements.   Let me now offer a contrary (or, maybe, complementary) view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a distinction to be made between writing a code to be widely used, and tuning it to a specific end.  In general we're doing the first - building our code so that a) we can use it on a wide variety of platforms, and b) the code is maintainable and extendable for years to come.  But sometimes we're doing the other - we've got a year's worth of allocation on some big machine, and we are ramping up to some required set of large simulations and we need a certain baseline of performance to get what we need done during the time of the granted allocation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we're writing code, making it widely useable and maintainable is much more important than shaving a few percent off of runtime on a particular machine.   In this case, the right thing to do is almost always to use the routine that best describes what you want to do - this is generally the most specific call you can make that does what you want.  Eg, if a straight allgather or allgatherv does what you want, you should use that rather than rolling your own out of scatter/gatter operations.  The reasons are that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The code now more clearly represents what you're trying to do, making it more understandable to the next person who comes to your code the following year having no idea what the code is supposed to do (that person could well be you);&lt;/li&gt;&#xA;&lt;li&gt;Optimizations are available at the MPI level for this more specific case that aren't in the more general case, so your MPI library can help you; and&lt;/li&gt;&#xA;&lt;li&gt;Trying to roll your own will likely backfire; even if it performs better on machine X with MPI implementation Y.ZZ, it may well perform much worse when you move to another machine, or upgrade your MPI implementation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this fairly common case, if you find out that some MPI collective works unreasonably slowly on your machine, the best thing to do is to file a bug report with the mpi vendor; you don't want to complicate your own software trying to work around in application code what should properly be fixed at the MPI library level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;However&lt;/em&gt;.  If you're in &quot;tuning&quot; mode - you have a working code, you've got to ramp  up to very large scales in a short period of time (eg, a year-long allocation), and you've profiled your code and found out that this particular part of your code is a bottleneck, then it makes sense to start performing these very specific tunings.   Hopefully they won't be long-term parts of your code - ideally these changes will remain in some project-specific branch of your repository - but you may need to do them.   In that case, the coding of two different approaches distinguished by preprocessor directives, or an &quot;autotuning&quot; approach for a specific communication pattern - can make a lot of sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm not disagreeing with Jeff, I just want to add some context about &lt;em&gt;when&lt;/em&gt; you should be concerned enough with such relative performance questions to modify your code to deal with it.&lt;/p&gt;&#xA;" OwnerDisplayName="user389" LastActivityDate="2012-03-27T17:40:33.207" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1778" PostTypeId="5" CreationDate="2012-03-27T19:14:32.170" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-27T19:14:32.170" LastActivityDate="2012-03-27T19:14:32.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1779" PostTypeId="4" CreationDate="2012-03-27T19:14:32.170" Score="0" Body="Questions related to whether the sequence of iterates generated by an iterative method has one or more limit points, and if those limit points have the correct properties." OwnerUserId="1193" LastEditorUserId="276" LastEditDate="2012-03-28T06:50:22.250" LastActivityDate="2012-03-28T06:50:22.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1780" PostTypeId="2" ParentId="1774" CreationDate="2012-03-27T19:47:55.487" Score="5" Body="&lt;p&gt;I don't know how to do it in Python, but the keyword you want to look for in the literature is the &quot;shooting method&quot;. That's the name of a method that solves problems that have both initial and final value constraints.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-27T19:47:55.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1781" PostTypeId="2" ParentId="1771" CreationDate="2012-03-27T19:48:54.327" Score="8" Body="&lt;p&gt;See Leslie Foster's &lt;a href=&quot;http://www.math.sjsu.edu/~foster/rankrevealingcode.html&quot;&gt;page on rank-revealing software&lt;/a&gt;. See also this &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn176.pdf&quot;&gt;LAPACK Working Note analyzing failures of rank-revealing QR &lt;code&gt;xGEQP3&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should be able to find out what routines MATLAB uses by setting breakpoints in a debugger and examining the stack. Last I looked, admittedly several years ago, MATLAB used shared libraries, in which case the symbol names cannot be stripped, so you will see the function names on the call stack (but not arguments because it definitely isn't keeping debugging information).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-03-27T19:48:54.327" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1782" PostTypeId="2" ParentId="1774" CreationDate="2012-03-27T20:19:03.527" Score="6" Body="&lt;p&gt;This is called a two-point boundary value problem and is well studied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The shooting method is very simple to program but may be extremely unstable numerically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard way to solve these problems is using a multiple shooting approach and solving the corresponding nonlinear system of equations by a standard nonlinear solver. For a list of solvers for nonlinear systems of equations, see, e.g.,&lt;br&gt;&#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonlin&quot;&gt;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonlin&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You take as variables the states on a regular grid in time (usually no very fine grid is needed), and as equations the boundary conditions and the mappings that map the time t variables to the time t+h variables. This gives as many equations as variables. You only need to provide the routines for evaluating this mapping for a given configuration of states on the grid, and the nonlinear solver does everything else. (Perhaps you need multiple starting points if your initial guesses are poor.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia &lt;a href=&quot;http://en.wikipedia.org/wiki/Direct_multiple_shooting_method&quot;&gt;http://en.wikipedia.org/wiki/Direct_multiple_shooting_method&lt;/a&gt;&#xA;has a useful description of the process, if the above description isn't detailed enough for you. The book by Stoer/Bulirsch cited there gives complete details.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-27T20:19:03.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1783" PostTypeId="1" AcceptedAnswerId="1790" CreationDate="2012-03-27T21:39:08.353" Score="2" ViewCount="1202" Body="&lt;p&gt;I have some experience with mpmetis from &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/metis/metis/overview&quot; rel=&quot;nofollow&quot;&gt;METIS&lt;/a&gt;. It is pretty good software which offers unstructured mesh grid partitioning. But obtained results always minimize edgecuts or total communication volume. So splitting simple mesh grid into 2 parts gives me about 50 to 50% of mesh nodes in each part. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for software (or better C++ library) which gives me possibility to split mesh into unequal parts - for example 10 to 90% of nodes in each part. It will be nice if I can decide about divide ratios.&lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2013-10-28T16:43:29.527" Title="Mesh domain decompositions / mesh partitioning" Tags="|mesh-generation|domain-decomposition|partitioning|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1784" PostTypeId="1" CreationDate="2012-03-28T07:44:25.567" Score="4" ViewCount="421" Body="&lt;p&gt;how can I turn an ODE equation into a discrete probabilistic model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I take for example the Verhulst equation for the growth of a population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dP}{dt} = rP(1-P/K)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking to simulate the population dynamic simply as a loop (for the time)&#xA;and at every time step, for each member of the population&#xA;evaluate the probability of procreation (in which case I add 1 to the population counter) and the probability that a member has died, in which case&#xA;I remove 1 from the population counter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know though how to use r and K to evaluate such probabilities.&#xA;Can you give me some hints or link me to some good resource that explains how to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, which is the strategy to convert an ODE (or system of ODEs) into it probabilistic equivalent???&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot in advance!&lt;/p&gt;&#xA;" OwnerUserId="1221" LastEditorUserId="522" LastEditDate="2012-03-28T18:39:14.190" LastActivityDate="2012-04-28T19:12:44.760" Title="Convert ODE into discrete probabilistic model" Tags="|ode|probability|stochastic|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1785" PostTypeId="1" AcceptedAnswerId="1787" CreationDate="2012-03-28T09:04:28.673" Score="1" ViewCount="1017" Body="&lt;p&gt;I recently came across the notion of an &lt;strong&gt;accumulation point&lt;/strong&gt; as a result of a certain gradient descent variation. The following definition was found:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;An accumulation point $P$ is such that there are an infinite number of terms of the sequence in any neighborhood of $P$.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if I understand the above definition correctly. &#xA;Since the gradient of the function at this point needs not to be zero ($0$), but is zero at the &lt;em&gt;stationary point&lt;/em&gt;, could one state that a &lt;em&gt;stationary point&lt;/em&gt; is an &lt;em&gt;accumulation point&lt;/em&gt; if it is a result of an iterative approach of minimization of a function $f(\cdot)$, satisfying $$f(P)-f(P_1)=0,$$where $P_1$ is the next iterate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the notion of oscillation adds to the confusion. Namely, it may not hold that $P=P_1$, and the minimization approach switches between the two (if the termination is determined by relative change of $P$ ). How could the accumulation point be defined in this case? (the last is inspired by the cluster point definition I came across; so, a connection between the three would be appreciated: accumulation point, oscillation, cluster point) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is stated &lt;a href=&quot;http://www.mth.kcl.ac.uk/courses/cm115/notes8.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (Theorem 8.5) that a convergent sequence has one and only one limit point (accumulation point), what could be induced from the case with oscillations? In other words, could one state that the sequence is actually non-convergent?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="81" LastEditDate="2012-03-28T16:25:07.370" LastActivityDate="2012-03-28T16:25:07.370" Title="Gradient descent to stationary, or accumulation point" Tags="|optimization|iterative-method|conjugate-gradient|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1786" PostTypeId="2" ParentId="1783" CreationDate="2012-03-28T10:51:10.210" Score="1" Body="&lt;p&gt;Have you tried scotch?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;SCOTCH is a project carried out within the Satanas team of the Laboratoire Bordelais de Recherche en Informatique (LaBRI). It is part of the ScAlApplix project of INRIA Bordeaux - Sud-Ouest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its purpose is to apply graph theory, with a divide and conquer approach, to scientific computing problems such as graph and mesh partitioning, static mapping, and sparse matrix ordering, in application domains ranging from structural mechanics to operating systems or bio-chemistry.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;get it here: &lt;a href=&quot;http://www.labri.fr/perso/pelegrin/scotch/&quot; rel=&quot;nofollow&quot;&gt;scotch site&lt;/a&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;have a look at this as well: &lt;a href=&quot;http://www.google.de/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=2&amp;amp;ved=0CDYQFjAB&amp;amp;url=http://www.hector.ac.uk/cse/reports/unstructured_partitioning.pdf&amp;amp;ei=u-xyT8LgEqX44QS887X7Dg&amp;amp;usg=AFQjCNG8usFVYLcVIYEtrdnYK1bwLxp8Rw&amp;amp;sig2=2_3Jmr-Xh39MU3Ct4dGCvA&quot; rel=&quot;nofollow&quot;&gt;scotch-metis tutorial + comparison&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-03-28T10:51:10.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1787" PostTypeId="2" ParentId="1785" CreationDate="2012-03-28T11:12:48.540" Score="5" Body="&lt;p&gt;accumulation point = cluster point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a sequence oscillates between increasingly narrow neighborhoods of two different points, such as the sequence $x_n=\frac{(-1)^nn}{n+1}$, both points are accumulation points of the sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In optimization, a local solver is usually reliable in practice when it is provable that each accumulation point of the sequence of iterates is a stationary point. In finite precision arithmetic, the gradient will almost never be zero because of rounding errors made. Therefore (if the sequence is bounded) the method will always get stuck close to one of these accumulation points, and hence will return an approximation to some stationary point (typically a minimizer if a descent method is used).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A stationary point is a property of the function optimized, an accummulation point one of the sequence generated in theory if the algorithm were executed exactly and infinitely long. What is computed is an approximation to an accumulation point of this theoretical sequence. Note that an accumulation point has no intrinsic meaning apart from the sequence of which it is the accumulation point. Thus the result has a useful meaning only if every accumulation point is a stationary point, as (only) then the ''solution'' returned will be an approximate stationary point. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-03-28T12:13:44.500" LastActivityDate="2012-03-28T12:13:44.500" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1788" PostTypeId="2" ParentId="1774" CreationDate="2012-03-28T11:27:54.873" Score="8" Body="&lt;p&gt;I disagree with the other answers.  Since you only have one-way coupling between the reverse- and forward-time problems, it will be much more efficient to solve them in sequence, as you first propose.  You simply need a solution $P(t)$ that can be evaluated at any time $t \in [0,t_f]$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this by &lt;strong&gt;interpolating between the output values&lt;/strong&gt;.  I recommend that you use a Runge-Kutta method that supports dense output.  For instance, &lt;code&gt;scipy.integrate.ode.dopri5&lt;/code&gt; is based on such a method.  So you should be able to specify very finely spaced output times without forcing the integrator to take very small steps (assuming that the scipy interface to it is implemented correctly).&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-03-28T11:27:54.873" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1789" PostTypeId="1" AcceptedAnswerId="1793" CreationDate="2012-03-28T13:04:11.663" Score="9" ViewCount="1882" Body="&lt;p&gt;I was trying to run test cases for CG and I need to generate:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;symmetric positive definite matrices &lt;/li&gt;&#xA;&lt;li&gt;of size &gt; 10,000 &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;FULL DENSE&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using only matrix indices and if necessary 1 vector&#xA;(Like $A(i,j) = \dfrac{x(i) - x(j)}{(i+j)}$)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With condition number less than 1000.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have tried:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Generating random matrices using &lt;code&gt;A=rand(N,N)&lt;/code&gt; and then &lt;code&gt;A'A&lt;/code&gt; to make it Sym. PD. [This increases condition number]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using the vector appraoch as shown but I can't seem to get a function of &lt;code&gt;(x,i,j)&lt;/code&gt; which will ensure Sym and PD.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;After much experimentation, I came up with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;a(it,jt) = (vec(it)+vec(jt))/((it-1)^2+(jt-1)^2);&lt;/code&gt; If $it \neq jt$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;a(it,it) = x(it)&lt;/code&gt; if $it=jt$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this is PD till about 500x500.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://math.nist.gov/MatrixMarket/data/misc/xlatmr/xlatmr.html&quot;&gt;XLATMR&lt;/a&gt;. [With all the grading and scaling, its too difficult to understand. Especially since I cannot understand the underlying linear algebra]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Can someone give me a function in x (vector) and i,j (indices) which meets the above requirements?&lt;/p&gt;&#xA;" OwnerUserId="529" LastActivityDate="2012-04-10T16:51:03.497" Title="Generating Symmetric Positive Definite Matrices using indices" Tags="|linear-algebra|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1790" PostTypeId="2" ParentId="1783" CreationDate="2012-03-28T15:29:41.757" Score="2" Body="&lt;p&gt;As can be found in the manual for METIS (and as well for parMETIS), available on &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/metis/metis/download&quot; rel=&quot;nofollow&quot;&gt;http://glaros.dtc.umn.edu/gkhome/metis/metis/download&lt;/a&gt; in section 3.5 &quot;Partitioning for heterogeneous parallel computing architectures&quot;, METIS does support non-equal partitioning of elements on different processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constitutive argument in the function calls &quot;METIS_PartMeshDual&quot; and &quot;METIS_PartMeshNodal&quot; is &quot;tpwgts&quot;. The entries of that array are supposed to be a partition of unity [*], that indicate the weight each process should be assigned eventually. By default, the weight of all elements is assumed to be uniformly 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[*] i.e., the entries are floating-point numbers that sum up to 1.0. The manual seems to ignore floating-point error at this condition.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-03-28T15:29:41.757" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1791" PostTypeId="2" ParentId="1789" CreationDate="2012-03-28T16:51:10.053" Score="2" Body="&lt;p&gt;I'm not sure how you would do it with just one vector, but with two random vectors $x$ and $\theta$ of size $N$, you can produce a positive semi-definite matrix via&#xA;$$&#xD;&#xA;U=\prod_i R_i(\theta_i)\\&#xD;&#xA;A=U\mbox{diag}(\mbox{abs}(x)) U^\ast&#xD;&#xA;$$&#xA;where $R_i(\cdot)$ is a rotation in the plane of the axes $i$ and $i+1 \mbox{ mod } N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to improve the condition number, you can add a fixed positive value to $x$ and rescale if need be.&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2012-03-28T16:51:10.053" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1792" PostTypeId="2" ParentId="1789" CreationDate="2012-03-28T17:55:20.433" Score="3" Body="&lt;p&gt;Try adding a large number $\alpha$ (on the order of the norm of the matrix) to the diagonal entries of your matrix. This is equivalent to adding $\alpha$ to each of your eigenvalues and should improve the condition number by reducing the gap between the largest and smallest eigenvalues.  &lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-03-28T17:55:20.433" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1793" PostTypeId="2" ParentId="1789" CreationDate="2012-03-28T17:56:38.080" Score="10" Body="&lt;p&gt;To get a dense positive definite matrix with condition number $c$ cheaply, pick a diagonal matrix $D$ whose diagonal consists of numbers from $[1,c]$ (which will be the eigenvalues), with $1$ and $c$ chosen at least once, and a vector $u$. Then apply a similarity transformation, via Householder transformations, to form the matrix $A:=(I-tuu^T)D(I-tuu^T)$, where $t=2/u^Tu$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To form this matrix with $O(n^2)$ operations, compute $v:=Du$, $s:=t^2u^Tv/2$, $w:=tv-su$ in $O(n)$ operations and then $A$ as $A=D-uw^T-wu^T$. (If you choose $u$ as the all-one vector, the number of multiplications needed is only $O(n)$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the behavior of CG depends a lot on the eigenvalue distribution, which you can easily confirm by varying $D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Adding $\alpha$ to an arbitrary symmetric matrix $A$ needs $\alpha$ larger than the largest eigenvalue. This is hard to compute; so one would have to choose $\alpha=\|A\|$, but then the condition number will generally be very small, not a realistic test case for CG.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="67" LastEditDate="2012-04-10T16:51:03.497" LastActivityDate="2012-04-10T16:51:03.497" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1795" PostTypeId="2" ParentId="1789" CreationDate="2012-03-28T20:57:04.883" Score="2" Body="&lt;p&gt;An entirely different way to do it would be like this: consider a random vector $x$, then $A=xx^T$ is a rank-one matrix with $N-1$ eigenvalues equal to zero and one strictly positive eigenvalue equal to $\|x\|^2$ with eigenvector $x$. It's also symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To construct a dense SPD matrix, add together many such rank-1 matrices. In other words, if you have $M$ vectors $x_i$ (e.g. random vectors), then&#xA;$$ A = \sum_{i=1}^M x_i x_i^T $$&#xA;is SPD if $M\ge N$ and if the vectors $x_i$ a linearly independent (if they are not linearly independent, then $A$ is positive semidefinite). You can verify if your $M$ vectors (or the first $M\ge N$ vectors you draw from the random process) are linearly independent using Gram-Schmidt successive orthogonalization, but you're also likely to get an SPD matrix if you simply choose $M \gg N$ vectors.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-03-29T17:51:56.613" LastActivityDate="2012-03-29T17:51:56.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1796" PostTypeId="2" ParentId="206" CreationDate="2012-03-28T21:31:12.483" Score="14" Body="&lt;p&gt;In my deal.II classes I teach that software that does not have tests does not work correctly (and go on to stress that I purposefully said &amp;quot;&lt;em&gt;does&lt;/em&gt; not work correctly&amp;quot;, not &amp;quot;&lt;em&gt;may&lt;/em&gt; not work correctly).&lt;/p&gt;&#xA;&lt;p&gt;Of course I live by the mantra -- which is how deal.II has come to run 2,500 tests with every commit ;-) [Update as of 2022: It is now 13,500 tests for each commit.]&lt;/p&gt;&#xA;&lt;p&gt;More seriously, I think Matt already defines the two classes of tests well. We write unit tests for the lower level stuff and it sort of naturally progresses to regression tests for the higher level stuff. I don't think I could draw a clear boundary that would separate our tests to one side or the other, there are certainly many that tread the line where someone has looked at the output and found it to be largely reasonable (unit test?) without having looked at it to the last bit of accuracy (regression test?).&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2022-06-01T23:01:21.997" LastActivityDate="2022-06-01T23:01:21.997" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1797" PostTypeId="2" ParentId="41" CreationDate="2012-03-28T21:41:19.383" Score="5" Body="&lt;p&gt;I think one answer to your question is that certain communities simply &lt;em&gt;always&lt;/em&gt; used conservative schemes and so it has become part of &quot;the way it's done&quot;. One may argue whether that's the best way to do it, but that's about as fruitful as asking the British to drive on the right because it would simply be more convenient to have only on standard side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I do see cases where it's useful. Think, for example, of two-phase porous media flow. This problem is commonly posed in the following way:&#xA;$$&#xD;&#xA;  u + K \nabla p = 0&#xD;&#xA;  \\&#xD;&#xA;  \nabla \cdot u = f&#xD;&#xA;  \\&#xD;&#xA;  \partial_t S + u \cdot \nabla S = q.&#xD;&#xA;$$&#xA;Here, part of the problem is solving the mixed Laplace that makes up the first two equations, a task traditionally done using Raviart-Thomas elements. They are often chosen because of &quot;the importance of ensuring mass conservation&quot;, and in a sense I can understand that: if you end up with a velocity field that's not mass conservative, you'll get a saturation equation that does not conserve the overall mass of the transported fluid. Of course one can argue that that wouldn't be so bad because it would all be the same in the limit $h\rightarrow 0$, but the insistence on ensuring that this property holds even for finite mesh sizes does make some sense.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-28T21:41:19.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1798" PostTypeId="2" ParentId="46" CreationDate="2012-03-28T21:43:38.647" Score="2" Body="&lt;p&gt;Good software must have been tested, and should say how the authors have tested and either provide the test data sets themselves (e.g. in the form of regression tests) or at least provide links to the data it was tested with.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-28T21:43:38.647" CommentCount="1" CommunityOwnedDate="2012-03-28T21:43:38.647" ContentLicense="CC BY-SA 3.0" />
  <row Id="1799" PostTypeId="1" CreationDate="2012-03-29T01:54:25.733" Score="6" ViewCount="1185" Body="&lt;p&gt;I have a symmetric indefinite matrix, $H$. I also have a routine that can compute the algebraically smallest eigenvalues of a symmetric indefinite matrix. I would like to compute the eigenvalues with smallest magnitude using my existing code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way I can do this is to use my routine to compute the algebraically smallest eigenpairs of $H^2$. Then I can use Rayleigh quotients to figure the smallest magnitude eigenvalues of $H$. This works great but is very slow since $cond(H^2) = cond(H)^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way?&lt;/p&gt;&#xA;" OwnerUserId="979" LastEditorUserId="979" LastEditDate="2012-03-30T18:05:14.907" LastActivityDate="2012-03-30T18:05:14.907" Title="Using algebraically smallest eigenvalues to find smallest in magnitude eigenvalues" Tags="|eigensystem|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1800" PostTypeId="2" ParentId="1799" CreationDate="2012-03-29T09:04:48.960" Score="5" Body="&lt;p&gt;If you can afford a single factorization of $H$ you can find the absolute largest eigenvalue of $H^{-1}$ by the Lanczos algorithm, using the factorization to calculate matrix-vector products with $H^{-1}$. The inverse then gives the absolutely smallest eigenvalue of $H$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-29T09:04:48.960" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1801" PostTypeId="1" AcceptedAnswerId="1804" CreationDate="2012-03-29T13:06:28.440" Score="11" ViewCount="10122" Body="&lt;p&gt;In order to estimate the eigenvalues of a real symmetric $n\times n$ matrix, I intend to use the Gershgorin Circle Theorem. Unfortunately, the examples one might find on the internet are a bit confusing; What would be the mathematical&#xA;formula for deriving the eigenvalue estimates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that certain disks are formed, each centered at the diagonal entry, with the radius equal to the summation of absolute values of the associated off-diagonal row entries. (The example from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Gershgorin_circle_theorem&quot;&gt;http://en.wikipedia.org/wiki/Gershgorin_circle_theorem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is clear with the disks, but not with the eigenvalue estimation)&#xA;Which steps to take from this point to get the estimates on the eigenvalues?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From Theorem 2.1 in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://buzzard.ups.edu/courses/2007spring/projects/brakkenthal-paper.pdf&quot;&gt;http://buzzard.ups.edu/courses/2007spring/projects/brakkenthal-paper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one could understand the eigenvalue $ranges$, but the example 2.3 from the above&#xA;paper gives concrete eigenvalue estimates (some of which are negative). I would appreciate if someone explains this. I'm interested in the largest (positive) and smallest (possibly negative) eigenvalue estimates.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2021-12-07T18:37:03.223" Title="Gershgorin Circle Theorem to estimate the eigenvalues" Tags="|linear-algebra|matrix|eigensystem|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1802" PostTypeId="2" ParentId="1801" CreationDate="2012-03-29T13:41:23.033" Score="9" Body="&lt;p&gt;Gershgorin's Theorem provides a bound on where to find each eigenvalue, but it doesn't provide an algorithm to actually calculate them.  You can take an initial guess $x_0$ within each Gershgorin disk, and use the shifted inverse power method to find the eigenvalue closest to $x_{0}$.  But there is no guarantee that using one initial guess per Gershgorin disk will will necessarily lead to finding all eigenvalues, unless you know ahead of time that the disks are disjoint. &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-03-29T13:41:23.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1803" PostTypeId="2" ParentId="530" CreationDate="2012-03-29T15:37:37.613" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;It would be great if gmsh would further include mesquite as 3rd party.  &lt;/li&gt;&#xA;&lt;li&gt;I am from the engineering field, where structured grid is possible and strongly advised. &lt;/li&gt;&#xA;&lt;li&gt;CUBIT seems to be the mesquite's professional version, or commercial version.  &lt;/li&gt;&#xA;&lt;li&gt;mesquite project ended a few year ago, though some other guys seems to continue keep it going. (meshkit?)  &lt;/li&gt;&#xA;&lt;li&gt;I have a very very good GUI experience with gmsh, it's easy going, light weighted, etc.!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1229" LastActivityDate="2012-03-29T15:37:37.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1804" PostTypeId="2" ParentId="1801" CreationDate="2012-03-29T15:42:41.373" Score="15" Body="&lt;p&gt;In Gerschgorin's theorem, the diagonal entries &lt;span class=&quot;math-container&quot;&gt;$A_{ii}$&lt;/span&gt; of the matrix are the eigenvalue estimates, and the radii &lt;span class=&quot;math-container&quot;&gt;$r_i$&lt;/span&gt; of the Gerschgorin disks are corresponding error bounds. Thus &lt;span class=&quot;math-container&quot;&gt;$\min_i A_{ii}-r_i$&lt;/span&gt; is a lower bound on the eigenvalues, and &lt;span class=&quot;math-container&quot;&gt;$\max_i A_{ii}+r_i$&lt;/span&gt; is an upper bound.&lt;/p&gt;&#xA;&lt;p&gt;Note that these bounds are generally poor unless the off-diagonal entries are small. In the latter case, one can get excellent bounds by an appropriate prior similarity transform of the matrix with as suitable scaling matrix.&lt;/p&gt;&#xA;&lt;p&gt;To get good bounds for a general matrix, one must compute an approximate eigensystem and then express the matrix in this basis by a simiarity transform. This doesn't change the eigenvalues but makes the off-diagonal entries small, so that the above applies.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2021-12-07T18:37:03.223" LastActivityDate="2021-12-07T18:37:03.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="1805" PostTypeId="1" CreationDate="2012-03-29T16:34:38.030" Score="5" ViewCount="1430" Body="&lt;p&gt;I'm trying to visualize the propagation of heat in a discrete sphere surface. The sphere is hollow, only the voxels of the boundary have value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Visualization in matlab using isosurface shows correct color over the surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to show iso-contours over this visualization but this is difficult in matlab. Searching for alternatives to visualize the data I've found python + numpy + mayavi2 and immediately fell in love with them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Playing with maiavi I can obtain my desired kind of plots with parametric surfaces but I cannot reproduce the same results with my volumetric, discrete sphere. The following code,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lines = [0.1, 0.3, 0.5, 0.7, 1, 1.5, 1.8, 2.3, 2.8, 3.0, 3.1]&#xA;mlab.contour3d(heatmap,contours=lines)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;only shows uniform color sphere. Enabling contours or not, does not change the result. No iso-heat lines nor even color over the surface&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone knows the correct approach to show the iso-heat contours?&lt;/p&gt;&#xA;" OwnerUserId="1228" LastActivityDate="2012-10-02T10:06:34.180" Title="Drawing isocontour data of 3d discrete volume with mayavi2" Tags="|python|visualization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1806" PostTypeId="2" ParentId="1801" CreationDate="2012-03-29T18:44:25.787" Score="6" Body="&lt;p&gt;If you to learn more about this, you might take a look into Horn and Johnson's book on matrix analysis, they have a whole chapter on eigenvalue estimation with Gershgorin disks.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-03-29T18:44:25.787" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1807" PostTypeId="2" ParentId="1784" CreationDate="2012-03-29T19:02:13.237" Score="1" Body="&lt;p&gt;If I understand you correctly, you're may be looking for some implementation of the Gillespie Algorithm, which takes advantage of the idea that rates and probabilities can be converted into each other. The algorithm works by ordering the system in terms of event times, which seems to be pretty close to what you want to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://press.princeton.edu/titles/8459.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Keeling and Rohani's book&lt;/a&gt;, while focused on mathematical epidemiology, has a decent treatment of several different implementations, and the trade-off between accuracy and computation time between the methods. It shouldn't be terribly difficult to adapt a epidemic model to a population growth model - they work in largely the same fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/338/python-implementations-of-gillespies-direct-method&quot;&gt;This question&lt;/a&gt; (by me) on this site might also be of interest. I know there are also ways to do this in R, and the implementation looks like it might be fairly approachable to code by hand if you're so inclined.&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-03-29T19:02:13.237" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1808" PostTypeId="2" ParentId="866" CreationDate="2012-03-29T19:22:33.577" Score="6" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cuthill%E2%80%93McKee_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cuthill-McKee&lt;/a&gt; is the &lt;em&gt;de facto&lt;/em&gt; standard for what you want to do. If you wanted to play with this method, there's an easy-to-use implementation of the algorithm (and its reverse) in the &lt;a href=&quot;http://www.boost.org/doc/libs/1_66_0/libs/graph/doc/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boost Graph Library&lt;/a&gt; (BGL), and the documentation contains examples how to use it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="20417" LastEditDate="2018-03-15T16:49:28.963" LastActivityDate="2018-03-15T16:49:28.963" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1809" PostTypeId="2" ParentId="873" CreationDate="2012-03-29T19:24:49.097" Score="4" Body="&lt;p&gt;A comparison of reordering algorithms as well as their effect on iterative solvers is also here (a few pages down):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/namespaceDoFRenumbering.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/namespaceDoFRenumbering.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-29T19:24:49.097" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1810" PostTypeId="1" CreationDate="2011-11-30T14:52:24.177" Score="9" ViewCount="179" Body="&lt;p&gt;This question is more about how to tackle a problem numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a small project I wanted to simulate  the  coorbital motion of Janus and Epimetheus. This is basically a three body problem. I choose Saturn to be fixed at the origin, let $r_1$ and $r_2$ be the location vectors of janus and epimetheus, respectively. Since the effect occurs when Janus and Epimetheus are very close together I picked relative coordinates for a better resolution, i.e. $r=r_1-r_2$ and $R=r_1+r_2$. Now I get the following equations of motion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\frac {d^2}{dt^2} \binom{R}{r} = - G (m_2\pm m_1) \frac R {R^3} - 4 M G \left(\frac {r+R}{(r+R)^3} \mp \frac {r-R}{(r-R)^3}\right )&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $m_i$ corresponds to the masses of the moons, $M$ is the mass of Saturn and $G$ the gravitational constant. The problem arises when I try to solve this numerically. One has to deal with values of completely different magnitudes, i.e. $M \sim e^{28}$ and $m_i \sim e^{17}$. And $r$, $R$ are in the regions of 0 to 150,000. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be honest I am not sure if this is the place forum to discuss such numerical problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More Information:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code is written in Matlab and I use a standard ODE solver to obtain the result. However this is breaking down because the step size cannot be reduced under machine precision. (I find this not to be surprising because one has to deal with the already mentioned orders of magnitude).&lt;/p&gt;&#xA;" OwnerDisplayName="bios" LastEditorUserId="414" LastEditDate="2012-03-30T07:36:47.363" LastActivityDate="2012-03-30T07:42:43.440" Title="Numerics: How do I renormalize the following ODE" Tags="|ode|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1811" PostTypeId="2" ParentId="1810" CreationDate="2011-11-30T17:08:38.833" Score="0" Body="&lt;p&gt;How about if you have three steps in your simulation:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;update Janus position by calculating the Janus - Saturn force.&lt;/li&gt;&#xA;&lt;li&gt;update Epimetheus position by calculating the Epimetheus - Saturn force.&lt;/li&gt;&#xA;&lt;li&gt;update Janus and Epimetheus position by calculating the Janus - Epimetheus force.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Possibly using finer timesteps for #3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if this will help. I suppose the real problem is that the magnitude of the force is different in the moon - moon and the moon - Saturn case, except if the moons are close?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;if the moons close, compute an approximate moons - Saturn force using their center of mass vector and update both positions with the same vector.&lt;/li&gt;&#xA;&lt;li&gt;if they are far apart, update them separately.&lt;/li&gt;&#xA;&lt;li&gt;as before.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Best of luck!&lt;/p&gt;&#xA;" OwnerDisplayName="mtrencseni" LastActivityDate="2011-11-30T17:19:09.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1812" PostTypeId="2" ParentId="1810" CreationDate="2012-03-29T19:16:31.543" Score="2" Body="&lt;p&gt;Your current approach ruins numerical stability; in fact you probably lose resolution this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take as coordinates for each satellite its Kepler variables and the angle of the plane containing the position of the satellite, the velocity, and the origin. The differential equations in the absence of interaction between the satellites are then trivially simple, and only the interaction becomes somewhat complicated. As the interaction is tiny if the satellites are far away, the resulting dynamics should be numerically stable.&lt;/p&gt;&#xA;" OwnerUserId="1117" OwnerDisplayName="Arnold Neumaier" LastActivityDate="2012-03-29T19:16:31.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1813" PostTypeId="2" ParentId="1745" CreationDate="2012-03-29T21:24:50.077" Score="3" Body="&lt;p&gt;I am a strong advocate of generic programming and meta-programming for scientific computing. I am actually developing a free software C++  library for Galerkin methods  based on these techniques called Feel++ (http://www.feelpp.org) which is steadily getting momentum. True that there are still difficulties such as slow compilation times and that the learning  curve could be steep if one wants to understand what is going on behind the scene. This is however extremely interesting and mind blowing. If done at the library level and hiding the complexity behind a domain specific language, you get an extremely powerful tool. We have at our disposal a very wide range a methods to use and compare. For teaching purpose of scientific computing this is awesome, for research and new numerical methods too, for large scale applications, well we work on it but so far so good, we can already do some nice stuff. We have engineers, physicists and mathematicians using it: most of them just use the language for variational formulation and they are happy with it. Looking at some of the formulations our physicists colleagues manipulate, I wouldn't want to see them done &quot;by hand&quot; without a high level language to describe the variational formulation. &#xA;I personally consider that these &quot;techniques&quot; or &quot;paradigms&quot; are now necessary to tackle complexity in scientific computing code with having to multiply the code size by a huge factor.&#xA;There is problably a need to improve the support of meta-programming in C++ but it is already in good shape especially since C++11. Compilers support is getting much better and the Boost C++ libraries show the way to proper handling of advanced generic/meta-programming.&lt;/p&gt;&#xA;" OwnerUserId="1230" LastActivityDate="2012-03-29T21:24:50.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1814" PostTypeId="2" ParentId="1763" CreationDate="2012-03-29T23:46:58.653" Score="14" Body="&lt;p&gt;The question highlights that most &quot;plain&quot; programming languages (C, Fortran, at least) do not allow you to do this cleanly. An added constraint is that you want notational convenience &lt;em&gt;and&lt;/em&gt; good performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, instead of &lt;em&gt;writing&lt;/em&gt; a dimension-specific code, consider writing a code that &lt;em&gt;generates&lt;/em&gt; a dimension-specific code. This generator is dimension-independent, even if the compute code is not. In other words, you add a layer of reasoning between your notation and the code expressing the computation. C++ templates amount to the same thing: Upside, they're built right into the language. Downside, they're somewhat cumbersome to write. This reduces the question to how to practically realize the code generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenCL lets you do code generation at run time fairly cleanly. It also makes for a very clean split between 'outer controlling program' and 'inner loops/kernels'. The outer, generating program is far less performance-constrained, and therefore might as well be written in a comfy language, like Python. That's my hope for how &lt;a href=&quot;http://mathema.tician.de/software/pyopencl&quot;&gt;PyOpenCL&lt;/a&gt; will get used--sorry for the renewed shameless plug.&lt;/p&gt;&#xA;" OwnerUserId="299" LastEditorUserId="299" LastEditDate="2012-03-30T01:57:04.747" LastActivityDate="2012-03-30T01:57:04.747" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1815" PostTypeId="2" ParentId="46" CreationDate="2012-03-30T01:00:21.900" Score="7" Body="&lt;p&gt;&lt;a href=&quot;http://www.astro.princeton.edu/~jstone/Athena/tests/index.html&quot;&gt;Athena's tests&lt;/a&gt; if you're solving hyperbolic conservation laws.&lt;/p&gt;&#xA;" OwnerUserId="299" LastActivityDate="2012-03-30T01:00:21.900" CommentCount="0" CommunityOwnedDate="2012-03-30T01:00:21.900" ContentLicense="CC BY-SA 3.0" />
  <row Id="1816" PostTypeId="2" ParentId="771" CreationDate="2012-03-30T02:51:37.693" Score="2" Body="&lt;p&gt;Voronoi and Delaunay graphs are called dual for their graph properties. See &lt;a href=&quot;https://en.wikipedia.org/wiki/Dual_graph&quot; rel=&quot;nofollow&quot;&gt;Dual Graph&lt;/a&gt; on Wikipedia.&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2012-03-30T02:51:37.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1817" PostTypeId="2" ParentId="1810" CreationDate="2012-03-30T07:42:43.440" Score="2" Body="&lt;p&gt;Instead of using a &quot;classical&quot; (stiff) ODE solver, you can use dedicated algorithms for geometric numerical integration. See for example &lt;a href=&quot;http://www.springer.de/cgi-bin/search_book.pl?isbn=3-540-43003-2&quot; rel=&quot;nofollow&quot;&gt;this book&lt;/a&gt; and the GNI codes you can find on &lt;a href=&quot;http://www.unige.ch/~hairer/software.html&quot; rel=&quot;nofollow&quot;&gt;Ernst Hairer's website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-03-30T07:42:43.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1818" PostTypeId="1" CreationDate="2012-03-30T09:40:20.563" Score="7" ViewCount="883" Body="&lt;p&gt;Is there good approximation for largest (in magnitude) eigenvalue for discrete Laplacian ($\nabla^2$) obtained from nonuniform structured grid (like &lt;a href=&quot;http://upload.wikimedia.org/wikipedia/commons/2/23/Rectilinear_grid.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;that&lt;/a&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, one can always use general methods such as Lanczos or Power Iteration, but I am interested in cheaper ways, possibly less accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found exact expressions for eigenvalues and eigenvectors of Laplacian in case of an uniform grid in &quot;&lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite Difference Methods for Ordinary and Partial Differential Equations&lt;/a&gt;&quot;, p.21&#xA;Obviously these formulas can't be extended for nonuniform grid case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried Gerschgorin disks theorem. Downloaded &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/13989&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; matlab script and created 5-point 2D Laplacian for uniform grid using following lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m=10;&#xA;h=10;&#xA;I = speye(m);&#xA;e = ones(m,1);&#xA;T = spdiags([e -4*e e],[-1 0 1],m,m);&#xA;S = spdiags([e e],[-1 1],m,m);&#xA;A = (kron(I,T) + kron(S,I)) / h^2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result looks like that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/YLcdu.png&quot; alt=&quot;Gershgorin Discs Plot for Laplacian on uniform grid&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bounds well enough. &lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2012-03-30T15:02:45.577" LastActivityDate="2012-07-23T14:32:20.773" Title="Largest eigenvalue of FD discrete Laplacian" Tags="|eigensystem|finite-difference|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1819" PostTypeId="2" ParentId="1818" CreationDate="2012-03-30T13:43:48.387" Score="3" Body="&lt;p&gt;For a rectangular grid and homogeneous boundary conditions, one knows the exact eigenfunctions $u_k(x)$ of the Laplacian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if you have a nonuniform mesh with nodes $x_l$ ($l\in L$), you can discretize the eigenfunction for the wanted eigenvalue: Evaluate the eigenfunction at your grid points to get a vector $v$ in your discrete representation, with components $v_l=u(x_l)$. Because the function discretized is an eigenvector of the continuous problem, $v$ is a good approximation of the eigenvector $v$ of the discrete matrix $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then calcuate the corresonding Rayleigh quotient $v^TAv/v^Tv$. The RQ is an $O(\epsilon^2)$ approximation to an eigenvalue if $v$ is an $O(\epsilon)$ approximation to the corresponding eigenvector. &#xA;Thus you shpuld a fairly good value for the corresponding eigenvalue of the discrete Laplacian, the better the finer the grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other cases where exact eigenfunctions are known, one can do the same.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-04-01T15:05:52.977" LastActivityDate="2012-04-01T15:05:52.977" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="1820" PostTypeId="2" ParentId="1799" CreationDate="2012-03-30T14:41:38.643" Score="1" Body="&lt;p&gt;You can use Jacobi-Davidson to find the lowest eigenvalue(s). Also see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1681/what-is-the-fastest-way-to-calculate-the-largest-eigenvalue-of-a-general-matrix&quot;&gt;this question&lt;/a&gt; for related issues.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-03-30T14:41:38.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1821" PostTypeId="5" CreationDate="2012-03-30T18:12:15.947" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-03-30T18:12:15.947" LastActivityDate="2012-03-30T18:12:15.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1822" PostTypeId="4" CreationDate="2012-03-30T18:12:15.947" Score="0" Body="Questions relating to graphing and displaying the results of scientific computations." OwnerUserId="1193" LastEditorUserId="1193" LastEditDate="2012-03-30T22:16:27.070" LastActivityDate="2012-03-30T22:16:27.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1823" PostTypeId="2" ParentId="1818" CreationDate="2012-03-30T20:52:52.250" Score="3" Body="&lt;p&gt;On a uniform mesh, the eigenvalues correspond to functions that are oscillatory. For oscillatory functions of the form $e^{i\vec k \cdot \vec x}$, the eigenvalues are $|\vec k|^2$. On uniform meshes, the highest wave vector that's representable on a mesh satisfies $k_i = \frac{2\pi}{\lambda_i}$ where $\lambda_i=2h_i$ is twice the mesh size in direction $i$. So we get an eigenvalue of $\pi^2\sum_{i=1}^d \frac{1}{h_i^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This construction is not exactly applicable to non-uniform meshes, but I would expect that the largest eigenvalues are still associated with oscillatory eigenfunctions that are localized around the area where the mesh is smallest. If so, for meshes like the one you show, then the largest eigenvalue would be approximately $\pi^2\sum_{i=1}^d \frac{1}{h_{min,i}^2}$, where $h_{min,i}$ is the minimum mesh size in coordinate direction $i$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-03-30T20:52:52.250" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1824" PostTypeId="1" CreationDate="2012-03-31T08:27:03.267" Score="7" ViewCount="233" Body="&lt;p&gt;Can anyone recommend a site that maintains up-to-date reviews of graphics cards for GPGPU use? Most benchmarks focus on gaming performance, whereas I am interested in the performance of scientific applications like FFTs, linear algebra, and image/video processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how the performance varies for scientific applications (as opposed to games) with price so I can budget effectively.&lt;/p&gt;&#xA;" OwnerUserId="1113" LastEditorUserId="1113" LastEditDate="2012-04-01T08:09:05.763" LastActivityDate="2012-04-01T08:33:41.127" Title="Online resources for reviewing graphics cards for GPGPU" Tags="|hpc|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1825" PostTypeId="1" AcceptedAnswerId="1826" CreationDate="2012-03-31T13:42:21.090" Score="8" ViewCount="1195" Body="&lt;p&gt;It is claimed that the time and memory complexities of sparse direct solver are $O(N^2)$ and $O(N^{4/3})$ for 3D problems and $O(N^{1.5})$ and $O(N \log N)$ for 2D, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how does a general-purpose direct solver know about dimensionality? If I call a direct solver with an arbitrary matrix, what is the complexity then? Is it then hidden in constants inside asymptotic estimations?&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="9" LastEditDate="2012-04-01T18:41:37.047" LastActivityDate="2022-05-02T10:02:14.173" Title="How does a Sparse Direct Solver know about dimensionality of a problem being solved?" Tags="|complexity|linear-solver|sparse-matrix|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1826" PostTypeId="2" ParentId="1825" CreationDate="2012-03-31T16:22:42.940" Score="11" Body="&lt;p&gt;A sparse direct solver knows the matrix, and hence its dimensions and its sparsity pattern. Of course it doesn't know the dimension of the problem dimension before discretization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the sparsity patttern reveals the dimension indirectly. In particular, the complexity estimates you report are based on the assumption that you have a sufficiently fine discretization of a 2D or 3D problem, and holds only asymptotically as the refinement goes uniformly to zero. (A 3D model of a long, thin bar is most likely $O(N)$ though in three dimensions, as two of the three dimensionswill hardly be refined in practice.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now direct solvers are based on a tree decomposition of the sparsity graph, and its width determines the complexity. Now nested dissection arguments provide bounds on the treewidth and hence the complexity estimates you mentioned. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-03-31T16:22:42.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1827" PostTypeId="2" ParentId="1824" CreationDate="2012-04-01T06:51:43.310" Score="3" Body="&lt;p&gt;Well, the answer depends on how you define &lt;em&gt;review&lt;/em&gt;. If you mean benchmarks, then there are numerous sites which have up-to-date benchmarks of GPU cards. One of them is &lt;a href=&quot;http://www.videocardbenchmark.net/&quot; rel=&quot;nofollow&quot;&gt;VideoCardBenchMark.com&lt;/a&gt;. A list of GPUs is maintained on that site. Also, there is the typical chart like &lt;a href=&quot;http://www.videocardbenchmark.net/video_lookup.php?gpu=GeForce+GTX+680&quot; rel=&quot;nofollow&quot;&gt;the one on bang-per-buck&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few random website like Toms Hardware do review GPU cards once a while. But those are for gaming and I'm not sure they would meet your requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For scientific computing in GPU, you could probably use &lt;a href=&quot;http://gpgpu.org/forums/viewforum.php?f=4&amp;amp;sid=6a45e3eecb0155d4f8bfcec1eaea3fe0&quot; rel=&quot;nofollow&quot;&gt;forums of gpgpu.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want Double Precision performance with GPU cards, its probably best to use nVidia forums instead. Of course, you may find benchmarks at a lot of places.&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="529" LastEditDate="2012-04-01T08:33:41.127" LastActivityDate="2012-04-01T08:33:41.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1828" PostTypeId="2" ParentId="1527" CreationDate="2012-04-01T12:27:31.787" Score="5" Body="&lt;p&gt;We have tried this out with adaptive finite elements where we take the previous solution over to the new mesh by interpolation. It turns out that starting with this vector has no noticeable effect on the number of CG iterations. In other words, for the CG iteration, a good initial guess is mostly useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the situation is entirely different for nonlinear methods (such as Newton's method) where it is &lt;em&gt;absolutely&lt;/em&gt; worth it to take the last iterate on the coarse mesh as the starting guess for the fine mesh. In practice, one often does 5-10 iterations on the coarsest mesh, but then needs only 1-2 iterations on every successively refined mesh.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-01T12:27:31.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1829" PostTypeId="2" ParentId="1639" CreationDate="2012-04-01T18:40:39.247" Score="2" Body="&lt;p&gt;If you are not looking for scripting take a look at MagicPlot. It's not so feature-rich as Origin (e.g. can produce only 2D plots) but I have never experienced problems with line width and other properties. Plots are nice anti-aliased though redrawing is fast enough. Plus useful fitting.&lt;/p&gt;&#xA;" OwnerUserId="1246" LastActivityDate="2012-04-01T18:40:39.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1830" PostTypeId="1" AcceptedAnswerId="1832" CreationDate="2012-04-01T19:27:37.003" Score="8" ViewCount="615" Body="&lt;p&gt;I need to compute only a small number of low frequency Fourier components of a complex 2-dimensional array. I'll be computing the same Fourier components over and over again as the input array changes. Clearly, in the limit where I only want one Fourier component, it would be fastest to build a DFT matrix that gives the component I'm after, and multiply by that matrix repeatedly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the other limit, if I wanted all Fourier components, it would be faster to use an FFT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At what point does it become faster to compute the FFT of the array and simply pull out the components that I'm after?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it makes a difference, in my particular situation the input array will be something like $256\times256$. I'm using MATLAB, so that means my FFT is done using FFTW, and a matrix multiplication for a matrix DFT is done via whatever matrix multiply algorithm MATLAB uses under the hood.&lt;/p&gt;&#xA;" OwnerUserId="22" LastEditorUserId="127" LastEditDate="2012-07-12T11:05:39.430" LastActivityDate="2012-07-12T11:05:39.430" Title="How many Fourier magnitudes do I have to calculate before an FFT becomes more efficient than a DFT?" Tags="|matlab|fourier-analysis|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1831" PostTypeId="1" AcceptedAnswerId="1839" CreationDate="2012-04-01T22:42:00.667" Score="2" ViewCount="1612" Body="&lt;p&gt;I am a programmer. I am working with Message Passing Interface (MPI) in C. I do a program that consist on finding the contiguous prime from 1 to 10,000,000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already do it! but I do it with trial division, testing the square root of the number to see if it is prime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, to check for a number n, if it is prime:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int Isprime(int n){&#xA;&#xA;for(i = 2; i &amp;lt;= ceil(sqrt(n)); i++)&#xA;&#xA;{ &#xA;   if(n%i == 0) &#xA;       return 0;&#xA;} &#xA;&#xA;return 1; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Meaning that if a number i which is less than or equal to the square root of a specified number, and i divides it, then the specified number is not prime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does someone know something more accurate? I mean more efficient to do this? Is there a more efficient algorithm to determine if a number is prime? Is there some important property of primes that I overlooked?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The run time of my program is good, but I want more! :) Ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also want to mention that I dont want access to a list of numbers because my program does not use the Master-Slave model. I dont have MPI_Send nor MPI_Recv.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also see something about Miller-Rabin primalirity test, but I dont uderstand it...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, Ideas??&lt;/p&gt;&#xA;" OwnerUserId="1248" LastEditorUserId="549" LastEditDate="2012-04-02T03:45:12.933" LastActivityDate="2012-04-02T22:51:19.550" Title="Contiguous prime numbers with MPI (Want more ideas for an efficient algorithm)" Tags="|parallel-computing|optimization|performance|hpc|mpi|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1832" PostTypeId="2" ParentId="1830" CreationDate="2012-04-02T01:31:51.607" Score="9" Body="&lt;p&gt;17.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lots and lots of work has gone into good fft implementations and it's unlikely you'll be able to reliably outperform a good fft library. For example, fftw &quot;automatically adapts itself to your machine, your cache, the size of your memory, the number of registers, and all the other factors that normally make it impossible to optimize a program for more than one machine&quot; &lt;a href=&quot;http://www.fftw.org/fftw2_doc/fftw_1.html&quot;&gt;ref this page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are right that there are situations where it's faster to just compute a few dot products but it's going to be very system dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An experiment:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EDU&amp;gt;&amp;gt; n = 256^2;&#xA;EDU&amp;gt;&amp;gt; x = randn(n,1);&#xA;EDU&amp;gt;&amp;gt; d = randn(1,n); %really, you should take a row from the output of the dftmtx command. But dftmtx(n) won't fit on my laptop...&#xA;EDU&amp;gt;&amp;gt; tic;d*x;toc; %time to compute a single frequency from the dft matrix&#xA;Elapsed time is 0.000225 seconds.&#xA;EDU&amp;gt;&amp;gt; tic;fft(x);toc; %time to compute the entire fft&#xA;Elapsed time is 0.003909 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So when there's 4096 data points computing the whole fft takes only ~17x longer than computing a single dot product.&lt;/p&gt;&#xA;" OwnerUserId="979" LastActivityDate="2012-04-02T01:31:51.607" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1833" PostTypeId="1" AcceptedAnswerId="1834" CreationDate="2012-04-02T07:06:17.430" Score="2" ViewCount="265" Body="&lt;p&gt;Suppose a matrix $X\in\mathbb{R}^{n\times 3}$ is given as a Principal Component Analysis (PCA) projection from some high dimensional space. The 2D PCA solution on X, say $Y\in\mathbb{R}^{n\times 2}$ would simply correspond to the first two columns of $X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose the configuration is shifted such that the origin corresponds to an arbitrary point. I want to &lt;em&gt;mathematically&lt;/em&gt; state (via PCA) that by changing the origin of $X$ (3D data), the new 2D PCA projection $Y&amp;#39;\in\mathbb{R}^{n\times 2}$ simply corresponds to two first columns on $X$ subject to rigid transformation (ie. rotation, reflection, shifting). The reason for the &lt;em&gt;perhaps unnecessary complication&lt;/em&gt; is the PCA assumption on the configuration centered at the origin. &#xA;(In other words, I'm not sure if by getting rid of it, one might loosen the connection with PCA)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To remind you, PCA would be obtained as $$Y=XU_S,$$where $U_S\in\mathbb{R}^{3\times 2}$ would contain eigenvectors of the correlation matrix $S=\frac{1}{n}X^TX$, where $X$ is supposed to be centered at the origin. The origin change would correspond to $X&amp;#39;=PX$, where $P=(I-1_np^T)$ denotes the &lt;em&gt;projector&lt;/em&gt;, and $1_n=[1, \dots, 1]\in\mathbb{R}^n$, and $p^T1_n=1$. The new &lt;em&gt;correlation&lt;/em&gt; matrix $$S&amp;#39;=\frac{1}{n}((PX)^T(PX))=\frac{1}{n}X^TPX=\frac{1}{n}X^T(J-1_np_1^T)X,$$&#xA;where $P$ might be expressed as $P=J-1_np_1^T$, where $J$ is a projector with $p=\frac{1}{n}1_n$, ie. $J=(I-\frac{1}{n}1_n1_n^T)$. So, I would like to state that $Y&amp;#39;=(PX)U_{S&amp;#39;}$ corresponds to a &lt;em&gt;different 2D viewpoint&lt;/em&gt; on the primary $X$. The difficulty, in my interpretation, lies in the effects of spectral decomposition of $S&amp;#39;$, and its &lt;em&gt;possible&lt;/em&gt; effects on rigid transformation on primary $X$. Again, I apologize for the &lt;em&gt;perhaps unnecessary complication&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-04-02T14:42:55.847" Title="Proof continuation for rigid transformation on PCA solution" Tags="|linear-algebra|matrix|eigensystem|high-dimensional|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1834" PostTypeId="2" ParentId="1833" CreationDate="2012-04-02T14:42:55.847" Score="3" Body="&lt;p&gt;In general, the principal components of $X$ and $X&amp;#39;$ with rows shifted by a fixed vector will not have any simple relationship.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In applications it may therefore be meaningful (or even important) to shift all rows of $X$ by subtraction their mean before doing the prinicpal component analysis. This makes the result independent of the placement of the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may even be useful (or necessary) to scale the columns after shifting to have norm 1. This makes the result also independent of the scaling of the variables.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-02T14:42:55.847" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1835" PostTypeId="2" ParentId="1830" CreationDate="2012-04-02T15:18:14.447" Score="4" Body="&lt;p&gt;As an alternative you could look into using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Goertzel_algorithm&quot; rel=&quot;nofollow&quot;&gt;Goertzel Algorithm&lt;/a&gt; to directly compute the frequency components you are interested in.&lt;/p&gt;&#xA;" OwnerUserId="1252" LastActivityDate="2012-04-02T15:18:14.447" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1836" PostTypeId="1" CreationDate="2012-04-02T15:54:41.450" Score="5" ViewCount="2123" Body="&lt;p&gt;I am trying to solve the following equation numerically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dfrac{\partial u}{\partial t}=\dfrac 3 x\dfrac \partial{\partial x}\left(\sqrt x \frac \partial {\partial x}(\nu(u,u^2,\cdots,u^k)u\sqrt x)\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expressed in its expanded form, this is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t=3\nu(u,u^2,\cdots)u_{xx}+\left(6\nu_x(u,u^2,\cdots)+\dfrac{3\nu(u,u^2,\cdots)}{2x}\right)u_x+\left(\nu_x(u,u^2,\cdots)\left(3+\dfrac 1 {2x}\right)+\dfrac{3\nu(u,u^2,\cdots)}{x}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u$ is a function of $u=u(t,x)$.  $\ \nu$ a is a nonlinear function of $u$ and another variable, but for a given $u$ I can obtain it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way in terms of stability, performance, and accuracy to solve this? I was trying method of lines in MATLAB, but I have not been able to make any progress.&lt;/p&gt;&#xA;" OwnerUserId="1253" LastEditorUserId="549" LastEditDate="2012-04-02T18:48:41.247" LastActivityDate="2012-04-03T11:24:17.073" Title="Matlab: Solving $u_t = f(u) u_{xx} + g(u) u_x + h(u) u$" Tags="|pde|matlab|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1837" PostTypeId="2" ParentId="1836" CreationDate="2012-04-02T18:38:38.997" Score="2" Body="&lt;p&gt;If you know for sure that your PDE is well posed, then I would suggest the following:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Try discretizing using an explicit finite difference scheme first.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you find that you get unreasonable / unstable solutions, try to make your temporal stepsizes very small in comparison to your spatial stepsizes (by several orders of magnitude).  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If this still doesn't work, try using an implicit finite difference scheme.  Since your equations are non-linear, it would be best to solve using newton's method.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If all else fails, check that your problem is well-posed.  You may want to try proving/disproving that solutions to your problem exist and are unique for the initial/boundary conditions you impose.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-04-02T19:04:30.650" LastActivityDate="2012-04-02T19:04:30.650" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1838" PostTypeId="2" ParentId="1831" CreationDate="2012-04-02T19:35:23.640" Score="2" Body="&lt;p&gt;The algorithm you implement of course works but it's slow since you need to try lots of long integer divisions: up to sqrt(N) for each number. You will find better algorithms that allow you discard almost all candidate numbers N without having to do these many divisions in any book on prime numbers or algorithmic number theory. I'm sure even wikipedia has suggestions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the simplest algorithm you could implement would be the Sieve of Erathostenes, which altogether avoids all divisions.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-02T19:35:23.640" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1839" PostTypeId="2" ParentId="1831" CreationDate="2012-04-02T22:51:19.550" Score="2" Body="&lt;p&gt;I'm probably stating the obvious here, but there are several things you could do better... For starters, in your &lt;code&gt;for&lt;/code&gt;-loop, you could check if &lt;code&gt;n&lt;/code&gt; is even first and then start at &lt;code&gt;i=3&lt;/code&gt; taking steps of 2, e.g. &lt;code&gt;i += 2&lt;/code&gt;. This saves you testing against all even numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More elaborately, since you're trying to find all contiguous primes, instead of testing &lt;code&gt;n&lt;/code&gt; against all integers &lt;code&gt;&amp;lt; sqrt(n)&lt;/code&gt;, you would be better off testing it against all &lt;em&gt;primes&lt;/em&gt; smaller than &lt;code&gt;sqrt(n)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is, of course, a bit more difficult to parallelize, but the costs grow with &lt;a href=&quot;http://en.wikipedia.org/wiki/Prime_number_theorem&quot; rel=&quot;nofollow&quot;&gt;$\mathcal O(n/\log n)$&lt;/a&gt; vs. $\mathcal O(n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you're serious about this, you should probably read-up a bit more on prime numbers to at least understand the &lt;a href=&quot;http://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test&quot; rel=&quot;nofollow&quot;&gt;Rabin test&lt;/a&gt;, as much better methods are available than what you're proposing, even with the improvements I've described.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-04-02T22:51:19.550" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1840" PostTypeId="2" ParentId="1836" CreationDate="2012-04-02T22:56:12.633" Score="1" Body="&lt;p&gt;It's hard to give concrete advice without knowing how your $\nu(u,u^2,\dots)$ really looks, but if you're in Matlab, you may want to try the &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chebfun&lt;/a&gt; system, which uses collocation spectral methods and automatic differentiation to solve linear and non-linear ODEs and, in some cases, PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you install Chebfun, the fastest way to try your example is to start the &lt;code&gt;chebgui&lt;/code&gt;, a graphical user interface for ODEs and PDEs, and look at some of the PDE examples and modify them for your own function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I am part of the Chebfun developer team :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following Aron Ahmadia's request, I've cooked-up a small example showing how to compute $\partial u/\partial t$ for $\nu(u) = u + u^2$ with boundary conditions $u(1)=u(2)=1$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Make some (any) nu(u)&#xA;nu = @(u) u + u.^2;&#xA;&#xA;% Set-up the differential equation for dudt&#xA;dudt = @(x,u) 3./x.*diff( sqrt(x) .* diff( nu(u).*u.*sqrt(x) ) );&#xA;&#xA;% Create a chebop with that operator on [1,2]&#xA;N = chebop( dudt , [1,2] );&#xA;&#xA;% Set some arbitrary boundary conditions&#xA;N.lbc = 1;&#xA;N.rbc = 1;&#xA;&#xA;% Solve for some righ-hand side&#xA;u = N \ 1;&#xA;plot(u);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/q1hVj.png&quot; alt=&quot;Solution generated using Chebfun&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The syntax may be a bit cryptic, but, again, I'd recommend having a look at &lt;code&gt;chebgui&lt;/code&gt;, which has a more intuitive syntax and is easier (and more fun) to play with.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-04-03T11:24:17.073" LastActivityDate="2012-04-03T11:24:17.073" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1841" PostTypeId="1" AcceptedAnswerId="1915" CreationDate="2012-04-03T01:07:15.327" Score="6" ViewCount="341" Body="&lt;p&gt;I have have multiple large matrices for which I need to find the largest absolute eigenvalue. I know that there is a large submatrix that does not vary. Is it possible to ignore/discard the submatrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is also related to this question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1681/what-is-the-fastest-way-to-calculate-the-largest-eigenvalue-of-a-general-matrix/&quot;&gt;What is the fastest way to calculate the largest eigenvalue of a general matrix?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1175" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-04-12T13:39:18.443" Title="Is it possible to ignore/discard part of a matrix when finding eigenvalues?" Tags="|linear-algebra|algorithms|eigensystem|sparse-matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1842" PostTypeId="2" ParentId="1503" CreationDate="2012-04-03T06:13:19.677" Score="3" Body="&lt;p&gt;Not minimal but fairly good :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've collected some online resources. Pick the one you like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://karussell.wordpress.com/2012/02/19/free-online-graph-theory-books-and-resources/&quot; rel=&quot;nofollow&quot;&gt;http://karussell.wordpress.com/2012/02/19/free-online-graph-theory-books-and-resources/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1254" LastActivityDate="2012-04-03T06:13:19.677" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1843" PostTypeId="1" AcceptedAnswerId="1848" CreationDate="2012-04-03T14:15:14.820" Score="7" ViewCount="358" Body="&lt;p&gt;I know that if I had evenly spaced points, I can use $U_{xx}\approx \frac{U_{i-1}-2U_{i}+U_{i+1}}{dx^2}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if my gridpoints are unevenly spaced, I assume that I can obtain the finite difference formula in the following way:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D^-[D^+(U)]$, where $D^-$ and $D^+$ are the backward and forward difference formulas.  If I define $h_i=x_{i}-x_{i-1}$, then I think I would obtain:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D^-[D^+(U)] = D^-[\frac{U_{i+1}-U_{i}}{h_{i+1}}] = \frac{\frac{U_{i+1}-U_{i}}{h_{i+1}} - \frac{U_{i}-U_{i-1}}{h_{i+1}}}{h_i}=\frac{U_{i-1}-2U_{i}+U_{i+1}}{h_ih_{i+1}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have some doubts about this formula.  Particularly, the term $\frac{U_{i}-U_{i-1}}{h_{i+1}}$ might be wrong.  Any feedback on this would be greatly appreciated :)&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-04-03T17:48:11.950" Title="How can I obtain a one dimensional finite difference formula for $U_{xx}$ with unevenly spaced nodes?" Tags="|finite-difference|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1844" PostTypeId="2" ParentId="1843" CreationDate="2012-04-03T14:25:23.650" Score="7" Body="&lt;p&gt;&lt;a href=&quot;http://www.ams.org/journals/mcom/1988-51-184/S0025-5718-1988-0935077-0/S0025-5718-1988-0935077-0.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt;, Generation of Finite Difference Formulas on Arbitrarily Spaced Grids, by Bengt Fornberg, provides Pseudo-Code for the... well... the title is a bit of a give-away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you pack it into a loop, you can create Differentiation matrices of arbitrary degrees over arbitrary point sets. If you really need it, I've got a Matlab implementation thereof that creates said matrices, which I could provide on request.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/users/1089/alexander&quot;&gt;Alexander&lt;/a&gt; has just pointed-out &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/matlab/fdcoeffF.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;fdcoeffF(k,xbar,x)&lt;/code&gt;&lt;/a&gt;, an implementation of Fornberg's algorithm which computes the weights of the stencil for the &lt;code&gt;k&lt;/code&gt;th derivative at a point &lt;code&gt;xbar&lt;/code&gt; over a set of data at points &lt;code&gt;x&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-04-03T14:41:24.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1845" PostTypeId="1" AcceptedAnswerId="1846" CreationDate="2012-04-03T15:21:15.953" Score="10" ViewCount="1539" Body="&lt;p&gt;Suppose I knew that a random number sequence was generated by a linear congruential generator.  That is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{n+1}=(aX_n+c) \bmod m$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am given the entire period (or at least a large contiguous subsequence of it), how can I reconstruct the parameters $a,c,m$ and $x_0$ that produced this sequence? I'm looking for a general method that will be able to determine the initial parameters if the pseudo-random number generator is known. &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="127" LastEditDate="2012-07-12T09:52:28.307" LastActivityDate="2019-05-10T21:20:15.437" Title="How can I determine the initial values of pseudo-random number generator if the sequence is given?" Tags="|random-number-generation|inverse-problem|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1846" PostTypeId="2" ParentId="1845" CreationDate="2012-04-03T15:33:52.223" Score="11" Body="&lt;p&gt;See the paper &lt;a href=&quot;http://www.reteam.org/papers/e59.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to crack a Linear Congruential Generator&lt;/a&gt;, Haldir (&quot;Reverse Engineering Team&quot;, Dec. 2004):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this paper I will present a method which will solve all values of the LCG including the modulus with six or more consecutive numbers of PRNG output.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The paper includes &quot;proof of concept&quot; source code written in C, using Victor Shoup's NTL for extended precision arithmetic.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2019-05-10T21:20:15.437" LastActivityDate="2019-05-10T21:20:15.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="1847" PostTypeId="1" AcceptedAnswerId="1851" CreationDate="2012-04-03T17:05:47.337" Score="6" ViewCount="149" Body="&lt;p&gt;I have a set of equations of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align*}&#xA;x_1&amp;amp;=(ax_0+c) \bmod (m)\\&#xA;x_2&amp;amp;=(ax_1+c) \bmod (m)\\&#xA;x_3&amp;amp;=(ax_2+c) \bmod (m)\\&#xA;&amp;amp;\vdots\\&#xA;x_{n}&amp;amp;=(ax_{n-1}+c) \bmod (m)&#xA;\end{align*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to compute the sensitivity indices of each equation with respect to the parameters $a$, $c$, and $m$.  That is, compute $\frac{dx_i}{da}$, $\frac{dx_i}{dc}$, and $\frac{dx_i}{dm}$ for each $i=1,\dots,n$.  However, the $\bmod (m)$ is throwing me off... I don't think that we can take a derivative of such an equation... In which case, I don't know if there is any other way to compute the sensitivity index of these equations.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="4244" LastEditDate="2013-07-01T22:11:06.487" LastActivityDate="2013-07-01T22:11:06.487" Title="How can I compute the sensitivity index of an expression with a modulus operator in it?" Tags="|conditioning|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1848" PostTypeId="2" ParentId="1843" CreationDate="2012-04-03T17:48:11.950" Score="8" Body="&lt;p&gt;Since the finite difference approximation to the gradient is&#xA;$$&#xD;&#xA;  u&amp;#39;(x) \approx \frac{\Delta u}{\Delta x}&#xD;&#xA;$$&#xA;it is clear that in your original formula, it needs to be&#xA;$$&#xD;&#xA;  u&amp;#39;&amp;#39;(x) \approx \frac{\frac{U_{i+1}-U_i}{x_{i+1}-x_{i}}&#xD;&#xA;                       -\frac{U_{i}-U_{i-1}}{x_{i}-x_{i-1}}}&#xD;&#xA;                      {\frac 12 (x_{i+1}+x_i)&#xD;&#xA;                      - \frac 12 (x_{i}+x_{i-1})} &#xD;&#xA;$$&#xA;which in your notation is then equal to&#xA;$$&#xD;&#xA;  u&amp;#39;&amp;#39;(x) \approx \frac{\frac{U_{i+1}-U_i}{h_{i+1}}&#xD;&#xA;                       -\frac{U_{i}-U_{i-1}}{h_{i}}}&#xD;&#xA;                      {\frac 12 (h_{i+1}+h_i)} &#xD;&#xA;=&#xD;&#xA; \frac{(U_{i+1}-U_i)h_i&#xD;&#xA;                       -(U_{i}-U_{i-1}){h_{i+1}}}&#xD;&#xA;                      {\frac 12 (h_{i+1}+h_i)h_ih_{i+1}}. &#xD;&#xA;$$&#xA;Conveniently, and not by accident, this reduces to the well-known formula for uniform mesh sizes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-03T17:48:11.950" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1849" PostTypeId="2" ParentId="1841" CreationDate="2012-04-03T17:56:38.150" Score="5" Body="&lt;p&gt;If you know that the second matrix is only a perturbation of the first, then you can do a perturbation argument. For example, if $\tilde A=A+\delta A$ where $\delta A$ is the difference (and is zero in large parts), then assume that the eigenvectors and eigenvalues also have an expansion $\tilde u_k=u_k+\delta u_k$ and $\tilde \lambda_k = \lambda_k + \delta \lambda_k$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the eigenvalues be order in ascending order, so that $\lambda_N$ is the largest. By definition of the Rayleight quotient, it holds:&#xA;$$&#xD;&#xA;  \lambda_N = \frac{u_N^T A u_N}{u_N^T u_N}&#xD;&#xA;$$&#xA;and&#xA;$$&#xD;&#xA;  \tilde \lambda_N = \frac{\tilde u_N^T \tilde A \tilde u_N}{\tilde u_N^T \tilde u_N}.&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you stick the expansion into this expression, you get&#xA;$$&#xD;&#xA;  \tilde \lambda_N = \lambda_N + \frac{\delta u_N^T A u_N + u_N^T A \delta u_N + \delta u_N^T \delta A u_N}{\tilde u_N^T \tilde u_N} + O(\delta\lambda_N^2).&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you happen to know anything about how the eigenvectors may change, you may get something out of this expression about the biggest eigenvalue.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-03T17:56:38.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1850" PostTypeId="2" ParentId="1847" CreationDate="2012-04-03T18:03:16.813" Score="2" Body="&lt;p&gt;The mod operation leads to a discontinuity in the function. That means of course that you can't take a derivative for values $x_i$ where $ax_i+c$ is a multiple of $m$. At all the other places, the mod-operation does nothing but shift its left operand so the derivative is as if the mod-operation wasn't there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see what exactly happens here, simply plot the $x_1$ as a function of $x_0$ for given $a,c$! It's a saw function.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-04-03T18:24:57.210" LastActivityDate="2012-04-03T18:24:57.210" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1851" PostTypeId="2" ParentId="1847" CreationDate="2012-04-03T19:20:30.490" Score="4" Body="&lt;p&gt;My explanation is getting long-winded here, but the answer is, attempting to use a continuous definition of sensitivity index is not going to be meaningful for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to find the sensitivity of a discontinuous function with respect to a parameter at a specific point of discontinuity, so if the problem could be posed with functions having domains over the reals, the problem would be well-posed; there exists mathematics to deal with these types of problems. (I'm having trouble tracking down a good source, since the canonical one appears to be E. N. Rozenvasser, &quot;General sensitivity equations of discontinuous systems,&quot; Automat. Remote Control (1967), 400–404, and every other paper I've seen seems to cite this one.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that moduli have integer-valued arguments, and you're trying to obtain the response to differential (i.e., very small) changes in parameters. It's impossible to meaningfully define what happens in the limit as your change in $m$ goes to zero, so this definition of sensitivity is not going to get you anywhere. I don't believe that switching to a discrete differential operator will remedy that problem in a meaningful way. However, you could search the literature and see if anything comes up; I found some work on sensitivity analysis of discrete stochastic dynamical systems that exists, but upon a quick read, I don't think it generalizes to your work readily. (Your mileage may vary.) If you want to do sensitivity analysis on this problem, there are probably approaches better-suited to your problem, such as sampling.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-04-03T19:20:30.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1852" PostTypeId="1" CreationDate="2012-04-03T21:19:14.610" Score="14" ViewCount="372" Body="&lt;p&gt;I would like to model laminar flow of water from roots to the stem of a plant. At the very end of the roots, the tubes vary from millimeter to centimeter scale in diameter and length. As we get closer to the stem the roots get larger in length and diameter. I want to create random 3D domains that represents the network of roots with varying diameters and lengths. What would be the best way to create this geometry. &lt;/p&gt;&#xA;" OwnerUserId="921" LastEditorUserId="20688" LastEditDate="2019-06-22T22:46:46.547" LastActivityDate="2019-06-22T22:46:46.547" Title="How to create a random 3D domain representing a plant's root structure?" Tags="|fluid-dynamics|computational-geometry|randomized-algorithms|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1853" PostTypeId="2" ParentId="1852" CreationDate="2012-04-03T21:33:22.760" Score="12" Body="&lt;p&gt;Chances are, you don't want something &lt;em&gt;truly&lt;/em&gt; random; you want something that has the same abstract 3D structure as a plant root system, but beyond a certain level of abstraction, you don't care what the root system looks like. I'm guessing you want some way to generate 3D fractal domains of the kind mentioned in &lt;a href=&quot;http://aob.oxfordjournals.org/content/73/3/281.full.pdf&quot;&gt;this paper describing the calculation of fractal dimensions of root systems&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After pulling up &lt;a href=&quot;http://www.ars.usda.gov/SP2UserFiles/Place/53200360/Modelling_applicability_fractal_analysis_efficiency_soil_exploration_roots.pdf&quot;&gt;this paper on fractal analysis of the efficiency of soil exploration by root systems&lt;/a&gt;, I found the &lt;a href=&quot;http://roots.psu.edu/?q=en/node/113&quot;&gt;SimRoot&lt;/a&gt; package that looks like it generates 3D root system geometries that may interest you. Unfortunately, they neglect to provide a way to download their package. However, their web site links to other packages that model root systems, such as &lt;a href=&quot;http://www-sop.inria.fr/virtualplants/wiki/doku.php?id=software%3aplantgl%3aplantgl&quot;&gt;PlantGL&lt;/a&gt; out of INRIA, which is open source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, once you create the geometry, you're going to have to figure out how to extract the relevant data in a compatible format and use it in PDE simulations. I leave that part up to you.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-04-03T21:52:23.983" LastActivityDate="2012-04-03T21:52:23.983" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1854" PostTypeId="2" ParentId="1129" CreationDate="2012-04-03T22:39:45.803" Score="14" Body="&lt;p&gt;Jack has given a good procedure for finding a preconditioner. I will try an address the question, &quot;What makes a good preconditioner?&quot;. The operational definition is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A Preconditioner M accelerates the iterative solution of $A x = b$, and $M^{-1}$ can be applied cheaply compared to $A^{-1}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;however this does not give us any insight into designing a preconditioner. Most preconditioners are based upon manipulation of the operator spectrum. Generically, Krylov methods converge faster when eigenvalues are clustered, see &lt;a href=&quot;http://books.google.com/books/about/Matrix_iterations.html?id=HbFQAAAAYAAJ&quot;&gt;Matrix Iterations&lt;/a&gt; or &lt;a href=&quot;http://books.google.com/books?id=OGSjEy3AM4UC&amp;amp;lpg=PP1&amp;amp;dq=Nevanlinna%20meromorphic%20linear%20algebra&amp;amp;pg=PP1#v=onepage&amp;amp;q=Nevanlinna%20meromorphic%20linear%20algebra&amp;amp;f=false&quot;&gt;Meromorphic Functions and Linear Algebra&lt;/a&gt;. Sometimes we can prove a preconditioner results is just a few unique eigenvalues, e.g. &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.5.982&quot;&gt;A Note on Preconditioning for Indefinite Linear Systems&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common strategy is exemplified by Multigrid. Relaxational preconditioners (here smoothers) like SOR remove high frequency components in the error. When the residual is projected onto a coarse grid, lower frequency error components become higher frequency and can again be attacked by SOR. This basic strategy underlies more complicated versions of MG, such as AMG. Note that at bottom, the solver must resolve the lowest frequencies in the error accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another strategy involves solving the equation in small subspaces, which is exactly what Krylov solvers are doing. In the simplest form, this is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Kaczmarz_method&quot;&gt;Kaczmarz method&lt;/a&gt; or the &lt;a href=&quot;http://en.wikipedia.org/wiki/Additive_Schwarz_method&quot;&gt;Additive Schwarz Method&lt;/a&gt;. The advanced strain of theory here, &lt;a href=&quot;http://books.google.com/books/about/Domain_Decomposition.html?id=dxwRLu1dBioC&quot;&gt;Domain Decomposition&lt;/a&gt;, concentrates on spectral approximation of the error on the interface, since the domains are assumed to be solved fairly accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there are a bunch of things produce operators close to $A$ is some sense, which are easy to invert, but give no guarantees of actually improving convergence of the Krylov iteration, as far as I know. These preconditioners include incomplete factorization (ICC, ILU), sparse approximate inverse (SPAI), and some low rank approximations.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2012-04-03T22:39:45.803" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1855" PostTypeId="2" ParentId="1847" CreationDate="2012-04-04T11:33:52.587" Score="1" Body="&lt;p&gt;You will need to use a trick. In your example, the modulo function merely introduces periodicity on continuous statements. You didn't specify whether your variables were integers or not, and even so you may want to relax your problem to continuous variables. You can transform your function by using an exponential of your equation:&#xA;$$&#xD;&#xA;e^{\Im 2\pi x_i/m} = e^{\Im 2\pi(ax_{i-1}+c)/m}&#xD;&#xA;$$&#xA;Obviously, this equation is differentiable and periodic $\mbox{mod } m$.&#xA;Now your derivatives are&#xA;$$&#xD;&#xA;\frac{dx_i}{da} e^{\Im 2\pi x_i/m} = \left(a\frac{dx_{i-1}}{da}+x_{i-1}\right)e^{\Im 2\pi(ax_{i-1}+c)/m}\\&#xD;&#xA;\left(\frac{dx_i}{dm}-\frac{x_i}{m}\right) e^{\Im 2\pi x_i/m} = \left(a\frac{dx_{i-1}}{dm}-\frac{ax_{i-1}+c}{m}\right)e^{\Im 2\pi(ax_{i-1}+c)/m}\\&#xD;&#xA;\frac{dx_i}{dc} e^{\Im 2\pi x_i/m} = \left(a\frac{dx_{i-1}}{dc}+1\right)e^{\Im 2\pi(ax_{i-1}+c)/m}&#xD;&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2012-04-04T11:33:52.587" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1856" PostTypeId="2" ParentId="1852" CreationDate="2012-04-04T13:44:37.763" Score="2" Body="&lt;p&gt;Something that might be helpful for you is the following paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Olga Wildeotter: &quot;An adaptive numerical method for the Richards equation with root growth&quot;, Plant and Soil, 2003&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They only treat a 2D model and use a cellular automaton to simulate growth. It does not directly relate to your question, however.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-04-04T13:44:37.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1857" PostTypeId="1" AcceptedAnswerId="1858" CreationDate="2012-04-04T14:27:52.160" Score="10" ViewCount="372" Body="&lt;p&gt;Let $\Omega$ be a convex polygonally bounded Lipschitz domain in $\mathbb R^2$, let $f \in L^2(\Omega)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the solution of the Dirichlet problem $\Delta u = f$ in $\Omega$, $\operatorname{trace} u = 0$ on $\partial\Omega$ has a unique solution in $H^2$ and is well-posed, i.e. for some constant $C$ we have $\|u\|_{H^2} \leq C \|f\|_{L^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some finite element approximation $u_h$, say, with nodal elements on a uniform grid, we have the error estimate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\| u - u_h \|_{H^1} \leq C h \| u \|_{H^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems (maybe I am wrong with that) that people usually do not use the obvious error estimate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\| u - u_h \|_{H^1} \leq C h \| f \|_{L^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which we can obtain by combination of the above two inequalities. Instead, a posteriori error estimaters are developed in various forms. The only objection I can imagine against the above equation is that the constant $C$ might in practice be too pessimistic or not reliably estimatable.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="651" LastEditDate="2012-04-04T16:02:32.360" LastActivityDate="2012-04-05T14:15:33.470" Title="What about this simple error estimate for linear PDE?" Tags="|pde|finite-element|error-estimation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1858" PostTypeId="2" ParentId="1857" CreationDate="2012-04-04T17:09:36.377" Score="8" Body="&lt;p&gt;The reason why people prefer to use the first estimate, in my opinion, is that the first one arises naturally from the Galerkin orthogonality of the FEM, interpolation approximation property, and most importantly the coercivity of the bilinear form(for Poisson equation's boundary value problem, it is equivalent with the Poincaré/Friedrichs inequality for $H^1_0$ functions):&#xA;$$&#xD;&#xA;\begin{aligned}&#xD;&#xA;\|u - u_h\|^2_{H^1(\Omega)} &amp;amp;\leq c_1 \| \nabla (u - u_h) \|^2_{L^2(\Omega)}&#xD;&#xA;\\&#xD;&#xA;\| \nabla (u - u_h) \|^2_{L^2(\Omega)} &amp;amp;= \int_{\Omega} \nabla(u- u_h)\cdot \nabla(u- u_h)&#xD;&#xA;\\&#xD;&#xA;&amp;amp;= \int_{\Omega} \nabla(u- u_h)\cdot \nabla(u- \mathcal{I}u)&#xD;&#xA;\\&#xD;&#xA;&amp;amp;\leq \| \nabla (u - u_h) \|_{L^2(\Omega)} \| \nabla (u - \mathcal{I}u) \|_{L^2(\Omega)}&#xD;&#xA;\\&#xD;&#xA;\Rightarrow \| \nabla (u - u_h) \|_{L^2(\Omega)} &amp;amp;\leq \| \nabla (u - \mathcal{I}u) \|_{L^2(\Omega)} \leq c_2 h\| u\|_{H^2(\Omega)}&#xD;&#xA;\end{aligned}&#xD;&#xA;$$&#xA;where $c_1$ depends on the constant in the Poincaré/Friedrichs inequality for $H^1_0$ functions, $\mathcal{I}u$ is the interpolation of $u$ in the finite element space, and $c_2$ depends on the minimum angles of the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While the elliptic regularity estimate $\|u \|_{H^2(\Omega)}\leq c\|f\|_{L^2(\Omega)}$ is solely on the PDE level, has nothing to do with the approximation, plus above argument holds even when $f\in H^{-1}$ is a distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now move on to the reason why a posteriori error estimates are widely used, is mainly because:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It is computable, there is no generic constant in the expression of the estimates.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The estimator has its local form, which could be the local error indicator using in the adaptive mesh refining procedure. Therefore, the problem with singularities or really &quot;bad&quot; geometries could be dealt with.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both of the a priori type estimates you listed are valid, they provide us the information of the orders of convergence, however none of them could be a local error indicator just for one triangle/tetrahedron, because neither of them are computable due to the constant, nor are them defined locally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: For more of a general view of the FEM for elliptic PDEs, I highly recommend reading Chapter 0 in Brenner and Scott's book:&lt;a href=&quot;http://books.google.com/books/about/The_mathematical_theory_of_finite_elemen.html&quot; rel=&quot;nofollow&quot;&gt;The Mathematical Theory of Finite Element Methods&lt;/a&gt;, which consists only 20 pages and covers briefly almost every aspect of finite element methods, from the Galerkin formulation from the PDE, to the motivation why we would like to use adaptive FEM to tackle some problem. Hope this would help you more.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-04-04T20:55:36.647" LastActivityDate="2012-04-04T20:55:36.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1859" PostTypeId="1" CreationDate="2012-04-04T18:28:06.597" Score="3" ViewCount="74" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Possible Duplicate:&lt;/strong&gt;&lt;br&gt;&#xA;  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1861/understanding-how-numpy-does-svd&quot;&gt;Understanding how Numpy does SVD&lt;/a&gt;  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;I have been using different methods to calculate both the rank of a matrix and the solution of a matrix system of equations. I came across the function linalg.svd, which I now learned is in fact a LAPACK routine. Comparing this to my own effort of solving the system with Gaussian Elimination, it appears to be both faster and more precise. I'm trying to understand why. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, the linalg.svd function uses a QR algorithm to calculate the eigenvalues of my matrix. I know how this works mathematically, but I don't know how Numpy manages to do it so quickly and without losing much precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm obviously not saying my script is in any way close to what the LAPACK authors do, but I'm trying to understand the fundamentals behind it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question: How does the numpy.svd function work, and more specifically, how does it manage to do it fast and accurately (compared to gaussian elimination)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time. &lt;/p&gt;&#xA;" OwnerUserId="1264" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-04-04T18:28:06.597" Title="Understanding how Numpy does SVD" Tags="|linear-algebra|matrix|python|" AnswerCount="0" CommentCount="1" FavoriteCount="0" ClosedDate="2012-04-04T19:54:07.420" ContentLicense="CC BY-SA 3.0" />
  <row Id="1860" PostTypeId="2" ParentId="1852" CreationDate="2012-04-04T18:29:03.887" Score="4" Body="&lt;p&gt;You could also borrow code from the folks who do NeuroML for dendrites.  I can upload code that generates tubes from NeuroML if you so desire.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/XwXnK.png&quot; alt=&quot;Dendrites&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2012-04-27T23:54:17.293" LastActivityDate="2012-04-27T23:54:17.293" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1861" PostTypeId="1" AcceptedAnswerId="1863" CreationDate="2012-04-04T17:30:58.047" Score="14" ViewCount="10186" Body="&lt;p&gt;I have been using different methods to calculate both the rank of a matrix and the solution of a matrix system of equations. I came across the function linalg.svd. Comparing this to my own effort of solving the system with Gaussian Elimination, it appears to be both faster and more precise. I'm trying to understand how this is possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, the linalg.svd function uses a QR algorithm to calculate the eigenvalues of my matrix. I know how this works mathematically, but I don't know how Numpy manages to do it so quickly and without losing much precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question: How does the numpy.svd function work, and more specifically, how does it manage to do it fast and accurately (compared to gaussian elimination)?&lt;/p&gt;&#xA;" OwnerUserId="1264" OwnerDisplayName="RobVerheyen" LastEditorUserId="149" LastEditDate="2012-04-05T13:33:38.313" LastActivityDate="2014-11-07T01:46:03.803" Title="Understanding how Numpy does SVD" Tags="|linear-algebra|python|lapack|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1862" PostTypeId="2" ParentId="1861" CreationDate="2012-04-04T17:50:51.543" Score="7" Body="&lt;p&gt;numpy.linalg.svd is a wrapper around {Z,D}GESDD from LAPACK. LAPACK, in turn, is very carefully written by some of the world's foremost experts in numerical linear algebra. Indeed, it'd be very surprising if someone not intimately familiar with the field would succeed in beating LAPACK (either in speed or accuracy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for why QR is better than Gaussian elimination, that is probably more appropriate for &lt;a href=&quot;https://scicomp.stackexchange.com/&quot;&gt;https://scicomp.stackexchange.com/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1077" OwnerDisplayName="janneb" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-04-04T17:50:51.543" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1863" PostTypeId="2" ParentId="1861" CreationDate="2012-04-04T20:18:43.320" Score="19" Body="&lt;p&gt;There are a number of issues in your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do not use Gaussian Elimination (LU factorization) to calculate the numerical rank of a matrix.&lt;/strong&gt; &lt;a href=&quot;http://en.wikipedia.org/wiki/Rank_(linear_algebra)#Computation&quot; rel=&quot;noreferrer&quot;&gt;LU factorization is unreliable for this purpose in floating-point arithmetic.&lt;/a&gt; Instead, use a rank-revealing QR decomposition (such as &lt;code&gt;xGEQPX&lt;/code&gt; or &lt;code&gt;xGEPQY&lt;/code&gt; in LAPACK, where x is C, D, S, or Z, though those routines are difficult to track down; see &lt;a href=&quot;https://scicomp.stackexchange.com/a/1781/276&quot;&gt;JedBrown's answer on a related question&lt;/a&gt;), or use an SVD (singular value decomposition, such as &lt;code&gt;xGESDD&lt;/code&gt; or &lt;code&gt;xGESVD&lt;/code&gt;, where x is again C, D, S, or Z). The SVD is a more accurate, reliable algorithm for the determination of numerical rank, but it requires more floating-point operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for solving a linear system, LU factorization (with partial pivoting, which is the standard implementation in LAPACK) is extremely reliable in practice. There are some pathological cases for which LU factorization with partial pivoting is unstable (see Lecture 22 in &lt;em&gt;Numerical Linear Algebra&lt;/em&gt; by Trefethen and Bau for details). QR factorization is a more stable numerical algorithm for solving linear systems, which is probably why it gives you such precise results. However, it requires more floating-point operations than LU factorization by a factor of 2 for square matrices (I believe; JackPoulson may correct me on that). For rectangular systems, QR factorization is a better choice because it will yield least-squares solutions to overdetermined linear systems. SVD can also be used to solve linear systems, but it will be more expensive than QR factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;janneb is correct that numpy.linalg.svd is a wrapper around &lt;code&gt;xGESDD&lt;/code&gt; in LAPACK. Singular value decompositions proceed in two stages. First, the matrix to be decomposed is reduced to bidiagonal form. The algorithm used to reduce to bidiagonal form in LAPACK is probably the Lawson-Hanson-Chan algorithm, and it does use QR factorization at one point. Lecture 31 in &lt;em&gt;Numerical Linear Algebra&lt;/em&gt; by Trefethen and Bau gives an overview of this process. Then, &lt;code&gt;xGESDD&lt;/code&gt; uses a divide-and-conquer algorithm to calculate the singular values and left and right singular vectors from the bidiagonal matrix. To get background on this step, you'll need to consult &lt;em&gt;Matrix Computations&lt;/em&gt; by Golub and Van Loan, or &lt;em&gt;Applied Numerical Linear Algebra&lt;/em&gt; by Jim Demmel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, &lt;strong&gt;you should not confuse singular values with eigenvalues&lt;/strong&gt;. These two sets of quantities are not the same. The SVD computes the singular values of a matrix. &lt;a href=&quot;http://www.mathworks.com/moler/index_ncm.html&quot; rel=&quot;noreferrer&quot;&gt;Cleve Moler's &lt;em&gt;Numerical Computing with MATLAB&lt;/em&gt;&lt;/a&gt; gives a &lt;a href=&quot;http://www.mathworks.com/moler/eigs.pdf&quot; rel=&quot;noreferrer&quot;&gt;nice overview of the differences between singular values and eigenvalues&lt;/a&gt;. In general, there is no obvious relationship between the singular values of a given matrix and its eigenvalues, except in the case of &lt;a href=&quot;http://en.wikipedia.org/wiki/Normal_matrix&quot; rel=&quot;noreferrer&quot;&gt;normal&lt;/a&gt; matrices, where the singular values are the absolute value of the eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-11-07T01:46:03.803" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1864" PostTypeId="1" AcceptedAnswerId="1871" CreationDate="2012-04-04T23:02:26.987" Score="18" ViewCount="3761" Body="&lt;p&gt;When computing the QR factorization in practice, one uses Householder reflections to zero out the lower portion of a matrix. I know that for computing eigenvalues of symmetric matrices, the best you can do with Householder reflections is getting it to tridiagonal form. Is there an obvious way to see why it can't be fully diagonalized in this way? I am trying to explain this simply but I can't come up with a clear presentation.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="127" LastEditDate="2013-05-11T17:59:23.110" LastActivityDate="2013-05-11T17:59:23.110" Title="Why can't Householder reflections diagonalize a matrix?" Tags="|linear-algebra|matrix|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1865" PostTypeId="2" ParentId="1864" CreationDate="2012-04-04T23:27:49.403" Score="2" Body="&lt;p&gt;For what reason do you assume that this is impossible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any symmetric real matrix $S$ can be orthogonally diagonalized, i.e. $S = G D G^t$, where $G$ is orthogonal and $D$ is diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any orthogonal matrix of size n×n can be constructed as a product of at most n such reflections.&lt;a href=&quot;http://en.wikipedia.org/wiki/Orthogonal_matrix#Primitives&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;. Therefore you have this decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure about the last statement, I just cite it (and I think it is correct). As far as I understand your question, it boils down to whether any orthogonal matrix can be decomposed into a sequence of Householder transforms.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-04-04T23:27:49.403" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1866" PostTypeId="1" CreationDate="2012-04-05T09:36:12.083" Score="5" ViewCount="782" Body="&lt;p&gt;From the &lt;a href=&quot;https://en.wikipedia.org/wiki/Perron%E2%80%93Frobenius_theorem#Perron_root_is_strictly_maximal_eigenvalue_for_positive_(and_primitive)_matrices&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perron-Frobenius theorem&lt;/a&gt;, it might be concluded that the spectral radius &lt;em&gt;is&lt;/em&gt; the largest eigenvalue for &lt;em&gt;positive matrices&lt;/em&gt;, ie, matrices with strictly positive entries. In other words, the largest magnitude eigenvalue of a symmetric, real, positive matrix, is a &lt;em&gt;positive eigenvalue&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm interested if the theorem is applicable to an arbitrary real symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; &lt;em&gt;with no zero entries&lt;/em&gt;. Namely, the particular interest is whether the [power iteration] method&lt;a href=&quot;http://en.wikipedia.org/wiki/Power_iteration&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, might be used to extract the eigenvector and the associated &lt;em&gt;largest positive eigenvalue&lt;/em&gt; of an input matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, I'm concerned with the possibility of largest magnitude eigenvalue being negative. Is the Power Iteration inappropriate in this case? If not in general, does stating that the eigenvector of all ones, &lt;span class=&quot;math-container&quot;&gt;$1_n$&lt;/span&gt;, corresponding to the eigenvalue of &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; spans the null-space of &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="20688" LastEditDate="2019-05-21T19:37:07.457" LastActivityDate="2019-05-21T19:37:07.457" Title="Perron-Frobenius theorem on general real symmetric matrices" Tags="|linear-algebra|matrix|eigensystem|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="1867" PostTypeId="1" AcceptedAnswerId="1876" CreationDate="2012-04-05T11:57:26.857" Score="15" ViewCount="2427" Body="&lt;p&gt;My research group focuses on molecular dynamics, which obviously can generate gigabytes of data as part of a single trajectory which must then be analyzed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several of the problems we're concerned with involve correlations in the data set, which means that we need to keep track of large amounts of data in memory and analyze them, rather than using a more sequential approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'd like to know is what are the most efficient strategies for handling I/O of large data sets into scripts. We normally use Python-based scripts because it makes coding the file I/O much less painful than C or Fortran, but when we have tens or hundreds of millions of lines that need to be processed, it's not so clear what the best approach is. Should we consider doing the file input part of the code in C, or is another strategy more useful? (Will simply preloading the entire array into memory be better than a series of sequential reads of &quot;chunks&quot; (order of megabytes)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some additional notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;We are primarily looking for scripting tools for post-processing, rather than &quot;on-line&quot; tools—hence the use of Python.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As stated above, we're doing MD simulations. One topic of interest is diffusion calculations, for which we need to obtain the Einstein diffusion coefficient:&#xA;$$D = \frac{1}{6} \lim_{\Delta t \rightarrow \infty} \left&amp;lt; \left( {\bf x}(t + \Delta t) - {\bf x}(t) \right)^2 \right&amp;gt;$$&#xA;This means we really need to load all of the data into memory before beginning the calculation—all of the chunks of data (records of individual times) will interact with one another.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-04-06T01:48:29.150" LastActivityDate="2012-04-06T09:52:39.140" Title="I/O Strategies for computational problems with large data sets?" Tags="|python|c|efficiency|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1868" PostTypeId="2" ParentId="1867" CreationDate="2012-04-05T12:02:44.510" Score="1" Body="&lt;p&gt;Perhaps you may use Cython in your file I/O sections and convert this part into C code?&lt;/p&gt;&#xA;" OwnerUserId="1228" LastActivityDate="2012-04-05T12:02:44.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1869" PostTypeId="2" ParentId="1867" CreationDate="2012-04-05T12:13:48.067" Score="7" Body="&lt;p&gt;I've had to deal with similar problems before, and my favourite solution is to use &lt;a href=&quot;http://www.gnu.org/software/libc/manual/html_node/Memory_002dmapped-I_002fO.html&quot; rel=&quot;nofollow&quot;&gt;Memory-mapped I/O&lt;/a&gt;, albeit in C...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principle behind it is quite simple: instead of opening a file and reading from it, you load it directly to the memory and access it as if it were a huge array. The trick that makes it efficient is that the operating system &lt;em&gt;doesn't actually load the file&lt;/em&gt;, it just treats it like &lt;em&gt;swapped-out memory&lt;/em&gt; that needs to be loaded. When you access any given byte in your file, the memory page for that part of the file is swapped into memory. If you keep accessing different parts of the file and memory gets tight, the less-used parts will get swapped back out -- automagically!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick Google search tells me that this is also available for Python: &lt;a href=&quot;http://docs.python.org/library/mmap.html&quot; rel=&quot;nofollow&quot;&gt;16.7. mmap — Memory-mapped file support&lt;/a&gt;, but I don't know enough about Python to tell if it's really the same thing.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="276" LastEditDate="2012-04-05T23:02:35.907" LastActivityDate="2012-04-05T23:02:35.907" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1870" PostTypeId="2" ParentId="1449" CreationDate="2012-04-05T12:37:38.600" Score="13" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Q1: What tools are you using for code profiling (profiling, not&#xA;  benchmarking)?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Q2: How long do you let the code run (statistics: how many time&#xA;  steps)?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Q3: How large are the cases (if the case fits in the cache, the solver&#xA;  is orders of magnitude faster, but then I'll miss the memory related&#xA;  processes) ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/a/927773/23771&quot;&gt;&lt;em&gt;Here's an example of how I do it.&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I separate the benchmarking (seeing how long it takes) from the profiling (identifying how to make it faster). It is not important that the profiler be fast. It is important that it tell you what to fix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't even like the word &quot;profiling&quot; because it conjures an image something like a histogram, where there's a cost-bar for each routine, or &quot;bottleneck&quot; because it implies there's just a little place in the code that needs to be fixed. Both of these things imply some sort of timing and statistics, for which you assume accuracy is important. &lt;em&gt;It is not worth giving up insight for accuracy of timing.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method I use is random pausing, and there's a full case study and slide show &lt;a href=&quot;http://sourceforge.net/projects/randompausedemo/files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;here&lt;/em&gt;&lt;/a&gt;.&#xA;Part of the profiler-bottleneck world view is that if you don't find anything, there's nothing to be found, and if you do find something and get a certain percent speedup, you declare victory and quit. Profiler fans almost never say how much speedup they get,&#xA;and the ads only show artificially contrived problems designed to be easy to find.&#xA;Random pausing finds the problems whether they are easy or hard.&#xA;Then fixing one problem exposes others, so the process can be repeated, to get compounded speedup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience from numerous examples, here's how it goes:&#xA;I can find one problem (by random pausing) and fix it, getting a speedup of some percent, say 30% or 1.3x.&#xA;Then I can do it again, find another problem and fix it, getting another speedup, maybe less than 30%, maybe more.&#xA;Then I can do it again, multiple times until I truly can't find anything else to fix.&#xA;The ultimate speedup factor is the running product of the individual factors, and it can be amazingly large - orders of magnitude in some cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;INSERTED: Just to illustrate this last point. There's a detailed example &lt;a href=&quot;http://sourceforge.net/projects/randompausedemo/files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, with slide show and all files, showing how a speedup of 730x was achieved in a series of problem removals. The first version took 2700 microseconds per unit of work. Problem A was removed, bringing the time down to 1800, and magnifying the percentages of remaining problems by 1.5x (2700/1800). Then B was removed. This process continued through six iterations, resulting in almost 3 orders of magnitude speedup. But the profiling technique has to be really efficacious, because if any of those problems are not found, i.e. if you reach a point where you incorrectly think nothing more can be done, the process stalls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2v1Zs.jpg&quot; alt=&quot;Description of removing multiple problems to get large speedup&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;INSERTED: To put it another way, here's a graph of the total speedup factor as successive problems are removed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/onA8P.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for Q1, for benchmarking a simple timer suffices. For &quot;profiling&quot; I use random pausing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q2: I give it enough workload (or just put a loop around it) so it runs long enough to pause.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q3: By all means, give it realistically large workload so you don't miss cache problems. Those will show up as samples in the code doing the memory fetches.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-09-06T16:29:47.487" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1871" PostTypeId="2" ParentId="1864" CreationDate="2012-04-05T13:12:21.363" Score="16" Body="&lt;p&gt;When computing the eigenvalues of the symmetric matrix $M\in\mathbb{R}^{n\times n}$ the best you can do with Householder reflector is drive $M$ to a tridiagonal form. As was mentioned in a previous answer because $M$ is symmetric there is an orthogonal similarity transformation which results in a diagonal matrix, i.e., $D=S^TMS$. It would be convenient if we could find the action of the unknown orthogonal matrix $S$ strictly using Householder reflectors by computing a sequence of reflectors and applying $H^T$ from the left to $M$ and $H$ from the right to $M$. However this is not possible because of the way the Householder reflector is designed to zero out columns. If we were to compute the Householder reflector to zero out all the numbers below $M_{11}$ we find&#xA;$$&#xD;&#xA;M=\left(\!\!{\begin{array}{ccccc}&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;*  \\&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;\end{array}}\!\!\right)\rightarrow&#xD;&#xA;H^T_1M=\left(\!\!{\begin{array}{ccccc}&#xD;&#xA;	* &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39;  \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;\end{array}}\!\!\right).&#xD;&#xA;$$&#xA;But now the entries $M_{12}-M_{1n}$ have been altered by the reflector $H^T_1$ applied on the left. Thus when we apply $H_1$ on the right it will no longer zero out the first row of $M$ leaving only $M_{11}$. Instead we will obtain&#xA;$$&#xD;&#xA;H^T_1M=\left(\!\!{\begin{array}{ccccc}&#xD;&#xA;	* &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39;  \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;\end{array}}\!\!\right)\rightarrow&#xD;&#xA;H^T_1MH_1=\left(\!\!{\begin{array}{ccccc}&#xD;&#xA;	* &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39; \\&#xD;&#xA;	*&amp;#39; &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39;  \\&#xD;&#xA;	*&amp;#39; &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39; \\&#xD;&#xA;	*&amp;#39; &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39; \\&#xD;&#xA;	*&amp;#39; &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39; \\&#xD;&#xA;\end{array}}\!\!\right).&#xD;&#xA;$$&#xA;Where not only did we not zero out the row but we may destroy the zero structure we just introduced with the reflector $H^T_1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when you opt to drive $M$ to a tridiagonal structure you will leave the first row untouched by the action of $H^T_1$, so&#xA;$$&#xD;&#xA;M=\left(\!\!{\begin{array}{ccccc}&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;*  \\&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;\end{array}}\!\!\right)\rightarrow&#xD;&#xA;H^T_1M=\left(\!\!{\begin{array}{ccccc}&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;	*&amp;#39; &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39;  \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;\end{array}}\!\!\right).&#xD;&#xA;$$&#xA;Thus when we apply the same reflector from the right we obtain&#xA;$$&#xD;&#xA;H^T_1M=\left(\!\!{\begin{array}{ccccc}&#xD;&#xA;	* &amp;amp;* &amp;amp; * &amp;amp; *&amp;amp;* \\&#xD;&#xA;	*&amp;#39; &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39;  \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39; &amp;amp; *&amp;#39; &amp;amp; *&amp;#39;&amp;amp;*&amp;#39; \\&#xD;&#xA;\end{array}}\!\!\right)\rightarrow&#xD;&#xA;H^T_1MH_1=\left(\!\!{\begin{array}{ccccc}&#xD;&#xA;	* &amp;amp;*&amp;#39; &amp;amp; 0 &amp;amp; 0&amp;amp;0 \\&#xD;&#xA;	*&amp;#39; &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39;  \\&#xD;&#xA;	0 &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39; \\&#xD;&#xA;	0 &amp;amp;*&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39; &amp;amp; *&amp;#39;&amp;#39;&amp;amp;*&amp;#39;&amp;#39; \\&#xD;&#xA;\end{array}}\!\!\right).&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applied recursively this allows us to drive $M$ to a tridiagonal matrix $T$. You can complete the diagonalization of $M$ efficiently, as was mentioned previously, using Jacobi or Givens rotations both of which are found in the Golub and Van Loan book &lt;em&gt;Matrix Computations&lt;/em&gt;. The accumulated actions of the sequence of Householder reflectors and Jacobi or Givens rotations allows us to find the action of the orthogonal matrices $S^T$ and $S$ without explicitly forming them.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastEditorUserId="1057" LastEditDate="2012-04-05T13:18:15.210" LastActivityDate="2012-04-05T13:18:15.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1872" PostTypeId="2" ParentId="1857" CreationDate="2012-04-05T14:15:33.470" Score="1" Body="&lt;p&gt;Your estimate is too pessimistic on two fronts. You've identified the first one already ($C$ now not only includes the interpolation constant but also the stability constant). The second one is that the error estimate really reads&#xA;$$&#xD;&#xA;  \|\nabla e\|_{L_2} \le C \;h\; |u|_{H^2}.&#xD;&#xA;$$&#xA;Note that the right hand side has the $H^2$ seminorm, not norm. Of course you can bound the rhs by the full norm, but you lose again this way.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-05T14:15:33.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1873" PostTypeId="1" CreationDate="2012-04-05T14:53:42.763" Score="9" ViewCount="994" Body="&lt;p&gt;I made a strange observation while computing the solution to a simple 1D reaction diffusion equation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\frac{\partial}{\partial t}a=\frac{\partial^2}{\partial x^2}a-ab$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$\frac{\partial}{\partial t}b=-ab$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$\frac{\partial}{\partial t}c = a$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The initial value of $b$ is a constant ($b(0,x)=b_0$), and I'm only interested in the integral over $a$ from $0$ to $1$ ($\int_0^1a(t,x)dt$). The purpose of $c$ and the equation $\frac{\partial}{\partial t}c = a$ is just to evaluate this integral.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I used a Strang splitting scheme for the coupling between diffusion and reaction (a half step reaction, then a full step diffusion, and then again a half step reaction), a Crank Nicholson scheme for the diffusion, and an analytical solution for the reaction (including the equation $\frac{\partial}{\partial t}c = a$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because one step of the analytical solution was more than a factor 3 slower than one step of the Crank Nicholson scheme, I tried to make more than one Crank Nicholson step for each reaction step. I was hoping to get by with fewer step of the Strang splitting scheme, so that I would be faster overall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the opposite effect can be observed, namely that much more steps for the Strang splitting scheme are required if more than one Crank Nicholson step is used. (I'm only concerned with the accuracy of the integral over $a$, which seems to converge faster than $a$ itself.) After wondering for some time, I noticed that the same effect also happens for $b(t,x)=b_0=0$, and that I even understand why for this case. The point is that if I make exactly one Crank Nicholson step, then the overall scheme turns into a trapezoid rule (if $b(t)=0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if I would treat $\frac{\partial}{\partial t}c = a$ as part of the diffusion step, increasing the number of Crank Nicholson steps (probably) wouldn't lead to reduced overall accuracy (as observed). But that seems to defeat the purpose of using an analytical solution for the (non-linear and potentially very stiff) reaction part of the system.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So here is my question: Is there a better way to treat $\frac{\partial}{\partial t}c = a$ in the context of a Strang splitting, than to either treat it as part of the reaction step, or to treat it as part of the diffusion step. I want to avoid being &quot;forced&quot; to use exactly one Crank Nicholson step for the diffusion. (For example in 3D, I would prefer to solve the diffusion analytically by an FFT instead of using Crank Nicholson. Of course I can also combine FFT with Crank Nicholson, so it's not such a big deal.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2012-04-05T23:30:45.487" LastActivityDate="2013-07-23T12:02:09.533" Title="Optimal use of Strang splitting (for reaction diffusion equation)" Tags="|parabolic-pde|operator-splitting|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1874" PostTypeId="2" ParentId="1617" CreationDate="2012-04-05T17:13:09.447" Score="4" Body="&lt;p&gt;Take a look at &lt;a href=&quot;http://www.vistrails.org/index.php&quot; rel=&quot;nofollow&quot;&gt;VisTrails&lt;/a&gt;. I haven't used it (only homebrew stuff around &lt;code&gt;make&lt;/code&gt;), but it looks well thought-out, with good doc, and has real users at NASA etc.&lt;br&gt;&#xA;(Are you looking for tools for 1-2 people, 4-5, more ?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added: not quite your question, but I think worth repeating:&lt;br&gt;&#xA;for uniform, reproducible computer experiments one obviously needs&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;uniform directory structures, e.g. when-what/ in/ out/ scripts/ log/&lt;/li&gt;&#xA;&lt;li&gt;uniform setting and echoing of &lt;em&gt;all&lt;/em&gt; parameters for a run&lt;/li&gt;&#xA;&lt;li&gt;scripts to summarize / plot / evaluate runs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;http://software-carpentry.org&quot; rel=&quot;nofollow&quot;&gt;software-carpentry.org&lt;/a&gt;:&#xA;&quot;The problem we’re trying to solve is that scientists often spend 40% or more of their time wrestling with software, but 95% or more of them are primarily self-taught&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2012-04-08T16:27:38.967" LastActivityDate="2012-04-08T16:27:38.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1875" PostTypeId="1" CreationDate="2012-04-05T17:44:59.840" Score="6" ViewCount="353" Body="&lt;p&gt;Can anyone point me to methods for varying $h$ in gradient estimation&#xA;for noisy numerical optimization?&#xA;Some programs have the user give a fixed $h$,&#xA;which is used for forward-difference or central-difference&#xA;gradient estimates the whole time,&#xA;from initial roughly-linear slopes to roughly-quadratic valleys&#xA;near a minimum. Surely there are adaptive methods that beat fixed $h$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a simple method that's been used in practice,&#xA;not theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added 23 April: a couple of factors that make this interesting:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;noise, scale of roughness, is difficult.&#xA;Imagine hiking in rough country, wanting to get up high,&#xA;but using only local gradients.&#xA;There are obstructions &amp;mdash; noise &amp;mdash;&#xA;at scales of millimeters, meters, kilometers; &#xA;what's a good $h$ to measure roughness,&#xA;not for its own sake, but to get up higher ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;say that each evaluation of f(x) costs \$1,&#xA;so a gradient estimate in 10d costs \$10 or (centered) \$20.&#xA;If you have only \$100 to spend,&#xA;how could you tradeoff accuracy for money ?&#xA;For example, instead of \$100 for 10 accurate gradients at 10 points,&#xA;one could spend \$10 at x$_0$, then update only the biggest gradient component at x$&lt;em&gt;1 \dots $ x$&lt;/em&gt;{10}$,&#xA;then \$10 again ... too many possiblities.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2012-04-23T15:15:14.617" LastActivityDate="2012-04-23T15:15:14.617" Title="Adaptive h for gradient estimation" Tags="|optimization|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1876" PostTypeId="2" ParentId="1867" CreationDate="2012-04-05T21:50:27.563" Score="6" Body="&lt;p&gt;I'm assuming your question comes from the observation that the I/O causes a significant overhead in your whole analysis. In that case, you can try to overlap I/O with computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A successful approach depends on how you access the data, and the computation you perform on that data. If you can identify a pattern, or the access to different regions of the data is known beforehand, you can try to prefetch the &quot;next chunks&quot; of data in the background while processing the &quot;current chunks&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a simple example, if you only traverse your file once and process each line or set of lines, you can divide the stream in chunks of lines (or MBs). Then, at each iteration over the chunks, you can load chunk i+1 while processing chunk i.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your situation may be more complex and need more involved solutions. In any case, the idea is to perform the I/O in the background while the processor has some data to work on. If you give more details on your specific problem, we may be able to take a deeper look into it ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;---- &lt;strong&gt;Extended version after giving more details&lt;/strong&gt; ----&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure I understand the notation, but well, as you said, the idea is an all-to-all interaction. You also mention that the data may fit in RAM. Then, I would start by measuring the time to load all the data and the time to perform the computation. Now,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;if the percent of the I/O is low (low as in you don't care about the overhead, whatever it is: 0.5%, 2%, 5%, ...), then just use the simple approach: load data at once, and compute. You will save time for more interesting aspects of your research.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if you cannot afford the overhead you may want to look into what Pedro suggested. Keep in mind what Aron Ahmadia mentioned, and test it before going for a full implementation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if the previous are not satisfactory, I would go for some &lt;em&gt;out-of-core&lt;/em&gt; implementation[1]. Since it seems that you are performing $n^2$ computations on $n$ data, there is hope :) Some pseudocode (assuming the results of your analysis fit in RAM):&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&#xA;    load chunk1 and chunk2&#xA;    for chunks i = 1 to n&#xA;        asynchronously load chunk i+1&#xA;        for chunks in j = i+1 to n&#xA;            asynchronously load chunk j+1&#xA;            compute with chunks i, j  (* for the first iteration, these are the preloaded chunks 1 and 2 *)&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: this is quick and dirty pseudocode, one would need to adjust the indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To implement this, it is common to use the so-called &lt;em&gt;double-buffering&lt;/em&gt;. Roughly speaking: divide memory in two workspaces; while data is being loaded in the background into workspace 1, processor is computing with the data in workspace 2. At each iteration, exchange the role.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sorry I cannot come up with a good reference right now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] An out-of-core algorithm incorporates some mechanism to (efficiently) deal with data residing on disk. They are called out-of-core as opposed to in-core (&quot;in-RAM&quot;).&lt;/p&gt;&#xA;" OwnerUserId="1267" LastEditorUserId="1267" LastEditDate="2012-04-06T09:52:39.140" LastActivityDate="2012-04-06T09:52:39.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1879" PostTypeId="1" AcceptedAnswerId="1880" CreationDate="2012-04-06T17:52:36.773" Score="25" ViewCount="516" Body="&lt;p&gt;Reproducibility has become more and more important in computational science research. (For instance, see &lt;a href=&quot;http://www.sciencemag.org/content/334/6060/1226.full&quot;&gt;this article by Roger Peng in &lt;em&gt;Science&lt;/em&gt;&lt;/a&gt;; I'm aware of other such articles and web sites also.) However, it's not clear to me how much information I should include with a journal article (or online) to make my computational research reproducible (assuming there are no other obstacles, like intellectual property agreements). Are there any guidelines out there, and if not, could people suggest what steps researchers should take to make their computational science research reproducible? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of particular use in answers would be possible ways to implement those suggestions -- workflows, basically. Workflows that are system-agnostic or Linux-based are preferable. Also, discussing any relevant personal experience you've had would also be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my particular case, I'm writing a theoretical paper with a couple example calculations that are simple enough that they can be done in MATLAB. I'd think that in this case, including the MATLAB script, as well as noting the specific version of MATLAB on my machine, would be enough to ensure reproducibility. However, I'm certain there are more complicated scenarios out there, and advice on how to carry out reproducible research would be very helpful to know for future projects.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-07-30T02:51:42.737" Title="What material should I include with a journal article (or post online) in order to make my computational research reproducible?" Tags="|software|publications|documentation|reproducibility|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1880" PostTypeId="2" ParentId="1879" CreationDate="2012-04-06T19:45:37.603" Score="18" Body="&lt;p&gt;In rough order of importance.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Source Code&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Make the code that implements the key aspects of your algorithm available. Even if the user can't build or run it, they can read exactly what is done. I have several times noticed simple decisions that weren't documented in a paper, but which a couple minutes with the source code answered conclusively.&lt;/li&gt;&#xA;&lt;li&gt;Make it runnable. This involves documenting the versions of dependent libraries and usually requires you to write somewhat portable code. Make sure it builds on at least one machine other than your own (it's easy to have hidden dependencies if you have never built it in a clean environment).&lt;/li&gt;&#xA;&lt;li&gt;Specify the version of the code that was used. If not a formally released version (and sometimes even then), document the SHA1 of the version. (This most naturally applies to DSCMs like Git and Mercurial, but can be used anywhere.) This is a very reliable way to guarantee that someone really has the same version of the code.&lt;/li&gt;&#xA;&lt;li&gt;Include configuration and host parameters including compiler vendor, versions, and optimization flags, system libraries like libc, CPU type, and memory type and topology (especially for performance studies).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;Run-time parameters/input files&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Include the complete input specification. If it was generated by a script, include that script. If it is huge data, document how to obtain and process the data. If your algorithm has randomness, specify the random number generator and seed that was used.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Scripts to generate figures and tables&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It is very helpful to include these scripts, both to clarify any questions about what the figures really show and to let the reader experiment with how things change if they change parameters or modify the algorithm.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-04-06T19:45:37.603" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1881" PostTypeId="2" ParentId="1879" CreationDate="2012-04-06T20:34:23.097" Score="2" Body="&lt;p&gt;At least, the Source code and data you used to perform your experiments should be accesible somewhere. Add instructions to build your code if necessary. Really there are so few open access journals that there is no an open and established rule.&lt;/p&gt;&#xA;" OwnerUserId="1228" LastActivityDate="2012-04-06T20:34:23.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1882" PostTypeId="2" ParentId="1875" CreationDate="2012-04-07T07:34:33.350" Score="4" Body="&lt;p&gt;You are not as free in choosing step size $h$ as you maybe think. If the accuracy with which you can compute your function is $a$, then the proposed step size is $h = a^{1/2}$ for the forward difference formula and $h = a^{1/3}$ for the central difference formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if the accuracy (e.g., internal functions) is &lt;code&gt;1e-15&lt;/code&gt; then $h$ should be not much smaller than &lt;code&gt;1e-7&lt;/code&gt; or &lt;code&gt;1e-5&lt;/code&gt;. The reason is that for very small step sizes the truncation and roundoff errors &quot;step in&quot; and the result will get more inexact. If a larger step size is chosen, one will get less accuracy than is available (like &quot;money left on the table&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing step sizes according to how much the function is &quot;bending&quot; are not promising and have to my knowledge not been considered. The error terms of the computation itself forbid that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: Any textbook on numerical analysis that has a chapter on numerical differentiation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hint: If your function is analytical and the programming language allows complex arithmetics, the &lt;em&gt;complex step derivation&lt;/em&gt; approach might be possible where you get almost the same accuracy for the numerical derivatives as for the function computation itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I forgot to say that if you apply Romberg's method to compute the derivative, as some numerical programs do, it will (try to) settle down with an optimal step size by itself.&lt;/p&gt;&#xA;" OwnerUserId="1084" LastEditorUserId="1084" LastEditDate="2012-04-07T12:50:20.413" LastActivityDate="2012-04-07T12:50:20.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1885" PostTypeId="2" ParentId="1864" CreationDate="2012-04-07T16:16:15.903" Score="21" Body="&lt;p&gt;As the Comments to other Answers clarify, the real issue here is not a shortcoming of Householder matrices but rather a question as to why iterative rather than direct (&quot;closed-form&quot;) methods are used to diagonalize (real) symmetric matrices (via orthogonal similarity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed any orthogonal matrix &lt;a href=&quot;ftp://info.mcs.anl.gov/pub/tech_reports/reports/P431.pdf&quot; rel=&quot;noreferrer&quot;&gt;can be expressed as a product of Householder matrices&lt;/a&gt;, so if we knew the diagonal form of a symmetric matrix (its eigenvalues), we could solve for a complete set of orthonormalized eigenvectors and represent the corresponding change of basis matrix as a product of Householder transformations in polynomial time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's turn to Victor's parenthetical comment &quot;other than Abel's theorem&quot; because we are effectively asking why &lt;em&gt;iterative&lt;/em&gt; methods should be used find the roots of a polynomial rather than a &lt;em&gt;direct&lt;/em&gt; method.  Of course the eigenvalues of a real symmetric matrix are the roots of its characteristic polynomial, and it is possible to go in the other direction as well.  Given a real polynomial with only real roots, it is possible to construct a tridiagonal symmetric companion matrix &lt;a href=&quot;https://mathoverflow.net/questions/48714/newtons-method-for-finding-multiple-roots/48722#48722&quot;&gt;from a Sturm sequence for the polynomial&lt;/a&gt;.  See also that poster Denis Serre's Exercise 92 in &lt;a href=&quot;http://www.umpa.ens-lyon.fr/~serre/DPF/exobis.pdf&quot; rel=&quot;noreferrer&quot;&gt;this set&lt;/a&gt;. This is rather nice for showing the equivalence of those problems since we've seen (@AndrewWinters) the direct application of Householder matrices will tridiagonalize a real symmetric matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analysis of the arithmetic complexity for an iterative (root isolation) method is given in Reif (1999), &lt;a href=&quot;http://www.cs.duke.edu/~reif/paper/roots.pdf&quot; rel=&quot;noreferrer&quot;&gt;An Efficient Algorithm for the Real Root and Symmetric Tridiagonal Eigenvalue Problems&lt;/a&gt;.  Reif's approach improves on tailored versions of &lt;a href=&quot;http://scg.ece.ucsb.edu/publications/oadv2.pdf&quot; rel=&quot;noreferrer&quot;&gt;QR for companion matrices&lt;/a&gt;, giving $O(n \log^3 n)$ instead of $O(n^2)$ complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Abel%E2%80%93Ruffini_theorem&quot; rel=&quot;noreferrer&quot;&gt;Abel-Galois-Ruffini Theorem&lt;/a&gt; says that no general formula for roots of polynomials above degree four can be given in terms of radicals (and usual arithmetic).  However there are closed forms for roots in terms of &lt;a href=&quot;http://en.wikipedia.org/wiki/Bring_radical&quot; rel=&quot;noreferrer&quot;&gt;more exotic operations&lt;/a&gt;.  In principle one might base eigenvalue/diagonalization methods on &lt;a href=&quot;http://library.wolfram.com/examples/quintic/timeline.html#20&quot; rel=&quot;noreferrer&quot;&gt;such approaches&lt;/a&gt;, but one encounters some practical difficulties:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The Bring radical (aka ultraradical) is a function of one variable, in that respect like taking a square root.  Jerrad (c. 1835) showed that solving the general quintic could be reduced to solving $t^5 + t - a = 0$, so that univariate function $t(a)$ (used in addition to radicals and other usual arithmetic) allows all quintics to be solved.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This breaks down with &lt;a href=&quot;http://en.wikipedia.org/wiki/Sextic_equation&quot; rel=&quot;noreferrer&quot;&gt;degree six polynomials&lt;/a&gt; and above, although various ways can be found to solve them using functions of just two variables.  &lt;a href=&quot;http://en.wikipedia.org/wiki/Hilbert%27s_thirteenth_problem&quot; rel=&quot;noreferrer&quot;&gt;Hilbert's 13th Problem&lt;/a&gt; was the conjecture that general degree seven polynomials could not be solved using only functions of at most two variables, but in 1957 V.I. Arnold showed they could.  Among the multivariable function families that can be used to get solutions to arbitrary degree polynomials are Mellin integrals, hypergeometric and Siegel theta functions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Besides implementing somewhat exotic special functions of more than one argument, we need direct methods for solving polynomials which work for general degree $n$ rather than ad hoc or degree specific methods. &lt;a href=&quot;http://www.math.leidenuniv.nl/~psh/CMpapers/Guardia.pdf&quot; rel=&quot;noreferrer&quot;&gt;Guàrdia (2002)&lt;/a&gt; gives &quot;a very simple expression of the roots of a polynomial of arbitrary degree in terms of derivatives of hyperelliptic theta functions.&quot;  However this approach requires making choices of Weierstrass points on hyperelliptic curve $C_f: Y^2 = f(x)$ where all roots of polynomial $f(x)$ are sought.  A good choice leads to expressing less than half of those roots, and it appears this approach requires repeated trials to get all of them.  Each trial involves solving a homogeneous linear system at $O(n^3)$ cost.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Therefore the indirect/iterative methods for isolating real roots (equiv. eigenvalues of symmetric matrices), even to high precision, currently have practical advantages over the known direct/exact methods for these problems.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2012-04-07T16:16:15.903" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1886" PostTypeId="2" ParentId="1879" CreationDate="2012-04-07T17:03:46.160" Score="6" Body="&lt;p&gt;Most journals aren't set up for this in any formal way, but we've recently founded the Archive of Numerical Software that is specifically intended to have the source code and everything else that's necessary be part of the article. Check it out:&#xA;  &lt;a href=&quot;http://journals.tdl.org/ans&quot;&gt;http://journals.tdl.org/ans&lt;/a&gt;&#xA;Submissions are welcome!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-07T17:03:46.160" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1887" PostTypeId="1" AcceptedAnswerId="1889" CreationDate="2012-04-09T09:56:11.933" Score="12" ViewCount="2912" Body="&lt;p&gt;Suppose the following matrix $A$ is given&#xA;$$ \left[\begin{array}{ccc}&#xD;&#xA;0.500 &amp;amp; -0.333 &amp;amp; -0.167\\&#xD;&#xA;-0.500 &amp;amp; 0.667 &amp;amp; -0.167\\&#xD;&#xA;-0.500 &amp;amp; -0.333 &amp;amp; 0.833\end{array}\right]$$&#xA;with its transpose $A^T$. The product $A^TA=G$ yields&#xA;$$ \left[\begin{array}{ccc}0.750 &amp;amp; -0.334 &amp;amp; -0.417\\&#xD;&#xA;-0.334 &amp;amp; 0.667 &amp;amp; -0.333\\&#xD;&#xA;-0.417 &amp;amp; -0.333 &amp;amp; 0.750\end{array}\right]$$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $G$ is a &lt;strong&gt;Laplacian matrix&lt;/strong&gt;. Note that matrices $A$ and $G$ are of rank 2, with the zero &#xA;eigenvalue corresponding to eigenvector $1_n=\left[\begin{array}{ccc}1 &amp;amp; 1 &amp;amp; 1\end{array}\right]^T$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder what would be the way to obtain $A$ if only $G$ is given. I tried&#xA;eigendecomposition $G=UEU^T$, and then set $A&amp;#39;=UE^{1/2}$, but obtained different result. I guess this has to do with rank deficiency. Could someone explain this? Clearly, the above example is for illustration; you could consider general Laplacian matrix decomposition of the above form.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Since, for instance, Cholesky decomposition could be used to find $G=LL^T$, the decomposition on $G$ could yield many solution. I'm interested in the solution that could be expressed as $$A=(I-1_nw^{T}),$$ where $I$ is a $3\times 3$ identity matrix, $1_n=[1~ 1~ 1]$, and $w$ being some vector satisfying $w^T1_n=1$. If it simplifies matters, you could assume that the entries of $w$ are non-negative.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-04-10T14:07:12.363" LastActivityDate="2012-04-11T18:24:06.597" Title="Finding the square root of a Laplacian matrix" Tags="|linear-algebra|matrix|eigensystem|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1888" PostTypeId="2" ParentId="1887" CreationDate="2012-04-09T11:37:02.633" Score="6" Body="&lt;p&gt;In essence, what you're asking is to find the square root A of a matrix G, so that $$G = A^T A.$$ There are many ways to do that if $G$ is a symmetric matrix. For example, if $G$ is symmetric, then the Cholesky decomposition $G=L^TL$ provides you with one answer: $A=L$. But you already found another answer, with the matrix $A$ you already have. What this simply means is that there are many &quot;square roots&quot; of the matrix $G$, and if you want to have one particular one, you need to rephrase the question in such a way that you specify the structural properties of the &quot;branch&quot; of the square root that you're interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would say that this situation is not dissimilar to taking the square root among the real numbers using the complex numbers: there, too, in general you have two roots, and you have to say which one you want to make the answer unique.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-09T11:37:02.633" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1889" PostTypeId="2" ParentId="1887" CreationDate="2012-04-09T15:41:50.173" Score="11" Body="&lt;p&gt;We have the matrix Laplacian matrix $G=A^TA$ which has a set of eigenvalues $\lambda_0\leq\lambda_1\leq\ldots\leq \lambda_n$ for $G\in\mathbb{R}^{n\times n}$ where we always know $\lambda_0 = 0$. Thus the Laplacian matrix is always symmetric positive semi-definite. Because the matrix $G$ is not symmetric positive definite we have to be careful when we discuss the Cholesky decomposition. The Cholesky decomposition exists for a positive semi-definite matrix but it is no longer unique. For example, the positive semi-definite matrix &#xA;$$&#xA; A=\left[\!\!\begin{array}{cc}&#xA;  0 &amp;amp; 0 \\&#xA;  0 &amp;amp; 1&#xA;\end{array}&#xA;\!\!\right],&#xA;$$&#xA;has infinitely many Cholesky decompositions&#xA;$$&#xA; A=\left[\!\begin{array}{cc}&#xA;  0 &amp;amp; 0 \\&#xA;  0 &amp;amp; 1&#xA;\end{array}&#xA;\!\right]&#xA;=&#xA; \left[\!\begin{array}{cc}&#xA;  0 &amp;amp; 0 \\&#xA;  \sin\theta &amp;amp; \cos\theta&#xA;\end{array}&#xA;\!\right]&#xA; \left[\!\begin{array}{cc}&#xA;  0 &amp;amp; \sin\theta \\&#xA;  0 &amp;amp; \cos\theta&#xA;\end{array}&#xA;\!\right]=LL^T.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, because we have a matrix $G$ that is known to be a Laplacian matrix we can actually avoid the more sophisticated linear algebra tools like Cholesky decompositions or finding the square root of the positive semi-definite matrix $G$ such that we recover $A$. For example, if we have the Laplace matrix $G\in\mathbb{R}^{4\times 4}$,&#xA;$$&#xA;G=\left[\!\begin{array}{cccc}&#xA; 3 &amp;amp; -1 &amp;amp;  -1 &amp;amp;  -1\\&#xA;-1 &amp;amp;  1 &amp;amp; 0 &amp;amp;  0 \\&#xA; -1 &amp;amp; 0 &amp;amp;  1 &amp;amp; 0 \\&#xA; -1 &amp;amp;  0 &amp;amp; 0 &amp;amp;  1 \\&#xA;\end{array}\!\right]&#xA;$$&#xA;we can use graph theory to recover the desired matrix $A$. We do so by formulating the oriented incidence matrix. If we define the number of edges in the graph to be $m$ and the number of vertices to be $n$ then the oriented incidence matrix $A$ will be an $m\times n$ matrix given by &#xA;$$&#xA;A_{ev} = \left\{\begin{array}{lc}&#xA;  1 &amp;amp;  \textrm{if }e=(v,w)\textrm{ and }v&amp;lt;w  \\&#xA;  -1 &amp;amp; \textrm{if }e=(v,w)\textrm{ and }v&amp;gt;w \\&#xA;  0 &amp;amp; \textrm{otherwise},&#xA;\end{array}&#xA;\right.&#xA;$$&#xA;where $e=(v,w)$ denotes the edge which connects the vertices $v$ and $w$. If we take a graph for $G$ with four vertices and three edges,&#xA;then we have the oriented incidence matrix &#xA;$$&#xA;A = \left[\!\begin{array}{cccc}&#xA; 1 &amp;amp; -1 &amp;amp; 0 &amp;amp;  0\\&#xA; 1 &amp;amp;  0 &amp;amp; -1 &amp;amp;  0 \\&#xA; 1 &amp;amp; 0 &amp;amp;  0 &amp;amp; -1 \\&#xA;\end{array}\!\right],&#xA;$$&#xA;and we can find that $G=A^TA$. For the matrix problem you describe you would construct a graph for $G$ with the same number of edges as vertices, then you should have the ability to reconstruct the matrix $A$ when you are only given the Laplacian matrix $G$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we define the diagonal matrix of vertex degrees of a graph as $N$ and the adjacency matrix of the graph as $M$, then the Laplacian matrix $G$ of the graph is defined by $G=N-M$. For example, in the following graph&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nhdnt.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PfenV.png&quot; alt=&quot;&quot; title=&quot;Hosted by imgur.com&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we find the Laplacian matrix is&#xA;$$&#xA;G=\left[\!\begin{array}{cccc}&#xA; 3 &amp;amp; 0 &amp;amp;  0 &amp;amp;  0\\&#xA; 0 &amp;amp;  1 &amp;amp; 0 &amp;amp;  0 \\&#xA; 0 &amp;amp; 0 &amp;amp;  1 &amp;amp; 0 \\&#xA; 0 &amp;amp;  0 &amp;amp; 0 &amp;amp;  1 \\&#xA;\end{array}\!\right] - \left[\!\begin{array}{cccc}&#xA; 0 &amp;amp; 1 &amp;amp;  1 &amp;amp;  1\\&#xA; 1 &amp;amp;  0 &amp;amp; 0 &amp;amp;  0 \\&#xA; 1 &amp;amp; 0 &amp;amp;  0 &amp;amp; 0 \\&#xA; 1 &amp;amp;  0 &amp;amp; 0 &amp;amp;  0 \\&#xA;\end{array}\!\right].&#xA;$$&#xA;Now we relate the $G$ to the oriented incidence matrix $A$ using the edges and nodes given in the pictured graph. Again we find the entries of $A$ from &#xA;$$&#xA;A_{ev} = \left\{\begin{array}{lc}&#xA;  1 &amp;amp;  \textrm{if }e=(v,w)\textrm{ and }v&amp;lt;w  \\&#xA;  -1 &amp;amp; \textrm{if }e=(v,w)\textrm{ and }v&amp;gt;w \\&#xA;  0 &amp;amp; \textrm{otherwise},&#xA;\end{array}&#xA;\right..&#xA;$$ &#xA;For example, edge $e_1$ connects the nodes $v_1$ and $v_2$. So to determine $A_{e_1,v_1}$ we note that the index of $v_1$ is less than the index of $v_2$ (or we have the case $v&amp;lt;w$ in the definition of $A_{ev}$). Thus, $A_{e_1,v_1} = 1$. Similarly by the way of comparing indices we can find $A_{e_1,v_2} = -1$. We give $A$ below in a more explicit way referencing the edges and vertices pictured.&#xA;$$&#xA;A = \begin{array}{c|cccc}&#xA; &amp;amp; v_1 &amp;amp; v_2 &amp;amp; v_3 &amp;amp; v_4 \\ \hline&#xA; e_1 &amp;amp; 1 &amp;amp; -1 &amp;amp; 0 &amp;amp;  0\\&#xA; e_2 &amp;amp; 1 &amp;amp;  0 &amp;amp; -1 &amp;amp;  0 \\&#xA; e_3 &amp;amp; 1 &amp;amp; 0 &amp;amp;  0 &amp;amp; -1 \\&#xA;\end{array}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, we generalize the concept of the Laplacian matrix to a weighted undirected graph. Let $Gr$ be an undirected finite graph defined by $V$ and $E$ its vertex and edge set respectively. To consider a weighted graph we define a weight function &#xA;$$&#xA;w: V\times V\rightarrow \mathbb{R}^+,&#xA;$$&#xA;which assigns a non-negative real weight to each edge of the graph. We will denote the weight attached to edge connecting vertices $u$ and $v$ by $w(u,v)$. In the case of a weighted graph we define the degree of each vertex $u\in V$ as the sum of all the weighted edges connected to $u$, i.e.,&#xA;$$&#xA;d_u = \sum_{v\in V}w(u,v).&#xA;$$&#xA;From the given graph $Gr$ we can define the weighted adjacency matrix $Ad(Gr)$ as an $n\times n$ with rows and columns indexed by $V$ whose entries are given by $w(u,v)$. Let $D(Gr)$ be the diagonal matrix indexed by $V$ with the vertex degrees on the diagonal then we can find the weighted Laplacian matrix $G$ just as before&#xA;$$&#xA;G = D(Gr) - Ad(Gr).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the problem from the original post we know &#xA;$$&#xA;G=\left[\!\begin{array}{ccc}&#xA; \tfrac{3}{4} &amp;amp; -\tfrac{1}{3} &amp;amp;  -\tfrac{5}{12} \\&#xA;-\tfrac{1}{3} &amp;amp;  \tfrac{2}{3} &amp;amp; -\tfrac{1}{3}  \\&#xA; -\tfrac{5}{12} &amp;amp; -\tfrac{1}{3} &amp;amp;  \tfrac{3}{4}  \\&#xA;\end{array}\!\right].&#xA;$$ &#xA;From the comments we know we seek a factorization for $G$ where $G=A^TA$ and specify $A$ is of the form $A=I-1_nw^T$ where $w^T1_n=1$. For full generality assume the matrix $A$ has no zero entries. Thus if we formulate the weighted oriented incidence matrix to find $A$ we want the weighted adjacency matrix $Ad(Gr)$ to have no zero entries as well, i.e., the weighted graph will have loops. Actually calculating the weighted oriented incidence matrix seems difficult (although it may simply be a result of my inexperience with weighted graphs). However, we can find a factorization of the form we seek in an ad hoc way if we assume we know something about the loops in our graph. We split the weighted Laplacian matrix $G$ into the degree and adjacency matrices as follows&#xA;$$&#xA;G=\left[\!\begin{array}{ccc}&#xA; \tfrac{5}{4} &amp;amp; 0 &amp;amp;  0 \\&#xA;0 &amp;amp;  1 &amp;amp; 0  \\&#xA; 0 &amp;amp; 0 &amp;amp;  \tfrac{11}{12}  \\&#xA;\end{array}\!\right]-\left[\!\begin{array}{ccc}&#xA; \tfrac{1}{2} &amp;amp; \tfrac{1}{3} &amp;amp;  \tfrac{5}{12} \\&#xA;\tfrac{1}{3} &amp;amp;  \tfrac{1}{3} &amp;amp; \tfrac{1}{3}  \\&#xA; \tfrac{5}{12} &amp;amp; \tfrac{1}{3} &amp;amp;  \tfrac{1}{6}  \\&#xA;\end{array}\!\right] = D(Gr)-Ad(Gr).&#xA;$$&lt;br&gt;&#xA;Thus we know the loops on $v_1$, $v_2$ and $v_3$ have weights $1/2$, $1/3$, and $1/6$ respectively. If we put the weights on the loops into a vector $w$ = $[\frac{1}{2}$ $\frac{1}{3}$ $\frac{1}{6}]^T$ then we can recover the matrix $A$ we want in the desired form&#xA;$$&#xA;A = I-1_nw^T = \left[\!\begin{array}{ccc}&#xA; \tfrac{1}{2} &amp;amp; -\tfrac{1}{3} &amp;amp;  -\tfrac{1}{6} \\&#xA;-\tfrac{1}{2} &amp;amp;  \tfrac{2}{3} &amp;amp; -\tfrac{1}{6}  \\&#xA; -\tfrac{1}{2} &amp;amp; -\tfrac{1}{3} &amp;amp;  \tfrac{5}{6}  \\&#xA;\end{array}\!\right].&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears if we have knowledge of the loops in our weighted graph we can find the matrix $A$ in the desired form. Again, this was done in an ad hoc manner (as I am not a graph theorist) so it may be a hack that worked just for this simple problem.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastEditorUserId="1057" LastEditDate="2012-04-10T13:44:25.627" LastActivityDate="2012-04-10T13:44:25.627" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1890" PostTypeId="1" AcceptedAnswerId="7007" CreationDate="2012-04-09T18:32:11.123" Score="5" ViewCount="950" Body="&lt;p&gt;How does one discretize the classical convective term in a transport equation using finite differences? I know the finite volume schemes out ther i.e. upwind, central differencing etc. Are there direct finite difference analogs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am modelling gas combustion in porous media. There are several PDEs involved and my question is about the energy equation for the gas. Here it is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\epsilon \rho_g C_{pg} \frac{\partial T_g}{\partial t}+ C_{pg} \nabla( \rho_g \bar u T_g) = -\epsilon \rho_g Q \frac{dC}{dt}+\alpha_\nu(T_s-T_g)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_g$ is the gas temperature, $T_s$ is the solid. The term of interest is $\nabla( \rho_g \bar u T_g)$. &lt;/p&gt;&#xA;" OwnerUserId="1088" LastEditorUserId="1113" LastEditDate="2012-04-09T20:54:28.743" LastActivityDate="2013-05-27T18:24:22.590" Title="Finite-difference discretization for a convective term" Tags="|finite-difference|fluid-dynamics|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="1891" PostTypeId="2" ParentId="1421" CreationDate="2012-04-09T19:57:44.483" Score="6" Body="&lt;p&gt;Vectorizing is straightforward in these situations using a strategy like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eN = ones(N,1);&#xA;eM = ones(M,1);&#xA;d  = sqrt(eM*n.^2' - 2*m*n' + m.^2*eN');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's an example that vectorizes the for loop with a 15x speedup for M=1000 and  N=2000.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = rand(N,2);&#xA;m = rand(M,2);&#xA;eN = ones(N,2);&#xA;eM = ones(2,M);&#xA;&#xA;tic;&#xA;d_vect  = sqrt(eN*m.^2' - 2*n*m' + n.^2*eM);&#xA;vect_time = toc;&#xA;&#xA;tic;&#xA;for i=1:N&#xA;  for j=1:M&#xA;     d_for(i,j) = norm(n(i,:)-m(j,:));&#xA;  end&#xA;end&#xA;for_time = toc; &#xA;&#xA;assert(norm(d_vect-d_for) &amp;lt; 1e-10*norm(d_for)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1290" LastEditorUserId="9" LastEditDate="2012-04-09T21:52:14.060" LastActivityDate="2012-04-09T21:52:14.060" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1892" PostTypeId="2" ParentId="1129" CreationDate="2012-04-10T06:55:44.487" Score="16" Body="&lt;p&gt;Others have already commented on the issue of preconditioning what I will call &quot;monolithic&quot; matrices, i.e. for example the discretized form of a scalar equation such as the Laplace equation, the Helmholtz equation or, if you want to generalize it, the vector-valued elasticity equation. For these things, it is clear that multigrid (either algebraic or geometric) is the winner if the equation is elliptic, and for other equations it isn't quite so clear -- but something like SSOR often works reasonably well (for some meaning of &quot;reasonable&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, the big revelation has been what to do about problems that are not monolithic, for example for the Stokes operator&#xA;$$&#xD;&#xA;  \begin{pmatrix} A &amp;amp; B \\ B^T &amp;amp; 0 \end{pmatrix}.&#xD;&#xA;$$&#xA;When I started with numerical analysis some 15 years ago, I think people had the hope that the same techniques could be applied to such matrices as above, and the direction of research was to either try multigrid directly or to use generalizations of SSOR (using &quot;point smoothers&quot; like Vanka) and similar methods. But this has faded since it doesn't work very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What has come to replace this was what was initially called &quot;physics-based preconditioners&quot; and later simply (and maybe more accurately) &quot;block preconditioners&quot; like the one by Silvester and Wathen. These are often based on block eliminations or Schur complements and the idea is to build a preconditioner in such a way that one can reuse preconditioners for individual blocks that are known to work well. In the case of the Stokes equation, for example, the Silvester/Wathen preconditioner uses that the matrix&#xA;$$&#xD;&#xA;  \begin{pmatrix} A &amp;amp; B \\ 0 &amp;amp; B^T A^{-1} B \end{pmatrix}^{-1}&#xD;&#xA;$$&#xA;when used as a preconditioner with GMRES would result in convergence in exactly two iterations. Since it is triangular, the inversion is also much simpler, but we still have the problem of what to do with the diagonal blocks, and there one uses approximations:&#xA;$$&#xD;&#xA;  \begin{pmatrix} \widetilde{A^{-1}} &amp;amp; B \\ 0 &amp;amp; \widetilde{(B^T A^{-1} B)^{-1}} \end{pmatrix}&#xD;&#xA;$$&#xA;where the tilde means to replace the exact inverse by an approximation. This is often much simpler: because the $A$ block is an elliptic operator, $\widetilde{A^{-1}}$ is well approximated by a multigrid V-cycle, for example, and it turns out that here, $\widetilde{(B^T A^{-1} B)^{-1}}$ is well approximated by an ILU of a mass matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea of working with the individual blocks that comprise the matrix and re-using preconditioners on individual ones has proven to be enormously powerful and has completely changed how we think of preconditioning systems of equations today. Of course, this is relevant because most actual problems are, in fact, systems of equations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-10T06:55:44.487" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1893" PostTypeId="2" ParentId="1421" CreationDate="2012-04-10T15:47:49.997" Score="2" Body="&lt;p&gt;From Octave 3.4.3 and later the operator - does automatic broadcasting (uses bsxfun internally).&#xA;So you can proceed in this way.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Dx = N(:,1) - M(:,1)';&#xA;Dy = N(:,2) - M(:,2)';&#xA;D = sqrt (Dx.^2 + Dy.^2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can do the same using a 3d matrix but I guess like this is more clear. D is a NxM matrix of distances, every vector in N against every vector in M.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps&lt;/p&gt;&#xA;" OwnerUserId="1298" LastEditorUserId="1298" LastEditDate="2012-07-18T16:12:10.807" LastActivityDate="2012-07-18T16:12:10.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1895" PostTypeId="2" ParentId="1887" CreationDate="2012-04-10T16:31:35.127" Score="9" Body="&lt;p&gt;I think that you are confusing the &lt;em&gt;unique&lt;/em&gt; matrix square-root of Hermitian positive semi-definite matrix $A$, i.e., a Hermitian positive semi-definite matrix $B$ satisfying,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;B^2 = A,&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the &lt;em&gt;non-unique&lt;/em&gt; problem of finding a matrix $C$ satisfying&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;C^H C = A,&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where clearly the mapping $C \mapsto Q C$, for &lt;em&gt;any&lt;/em&gt; unitary $Q$, preserves the identity. As you noticed, a Cholesky factorization provides one possible solution. However, &lt;strong&gt;note that Cholesky only works for Hermitian positive-definite matrices&lt;/strong&gt; (with the possible exception of a Hermitian positive semi-definite matrix which is positive-definite if the last row and column are removed). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, one can constructively define the &lt;em&gt;unique&lt;/em&gt; matrix square-root of a Hermitian positive semi-definite matrix through its eigenvalue decomposition, say&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;A = U \Lambda U^H,&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is unitary and $\Lambda$ is diagonal with non-negative entries due to $A$ being positive semi-definite. The Hermitian matrix square-root can easily be identified as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;B = U \sqrt{\Lambda} U^H.&#xD;&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-04-10T23:52:58.643" LastActivityDate="2012-04-10T23:52:58.643" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1896" PostTypeId="2" ParentId="1861" CreationDate="2012-04-10T16:47:20.960" Score="8" Body="&lt;p&gt;Due to the wording of your question, I am assuming that your matrix is square. LAPACK's SVD routines, such as &lt;em&gt;zgesvd&lt;/em&gt;, essentially proceed in three stages for square matrices:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Implicitly computing unitary matrices $U_A$ and $V_A$, as products of Householder transforms, such that the general matrix $A$ is reduced to a real, upper bidiagonal matrix $B := U_A^H A V_A$. On exit of this subroutine, the Householder vectors (normalized so that their first entry is implicitly one) for $U_A$ and $V_A$ are respectively stored in the portions of $B$ below and to the right of the main and super-diagonal. This step requires $O(n^3)$ work.&lt;/li&gt;&#xA;&lt;li&gt;A variation on an algorithm for computing the eigenvalue decomposition of a real symmetric tridiagonal matrix is used to compute a bidiagonal SVD. Though the best-known algorithm for real symmetric tridiagonal EVP's (MRRR) is not yet, to my knowledge, stable for the bidiagonal SVD, there is an interesting discussion &lt;a href=&quot;http://www.eecs.berkeley.edu/Pubs/TechRpts/2005/5648.html&quot;&gt;here&lt;/a&gt;. LAPACK currently uses a &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=207472&quot;&gt;&lt;em&gt;divide and conquer&lt;/em&gt; approach&lt;/a&gt; for the bidiagonal SVD. The bidiagonal SVD yields $\{U_B,V_B,\Sigma\}$ such that $B = U_B \Sigma V_B^H$. This step will require $O(n^2)$ work when MRRR is made stable for SVD, but currently requires as much as $O(n^3)$ work.&lt;/li&gt;&#xA;&lt;li&gt;This step is trivial. Since $U_A B V_A^H = A$, $A = (U_A U_B) \Sigma (V_A V_B)^H$, so that the SVD can be explicitly formed after applying the Householder transforms which implicitly define $U_A$ and $V_A$ to $U_B$ and $V_B$, respectively. This step required $O(n^3)$ work.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-04-10T16:47:20.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1897" PostTypeId="2" ParentId="1617" CreationDate="2012-04-10T17:45:03.277" Score="12" Body="&lt;p&gt;For logging that allows full reproducibility, I highly recommend the &lt;a href=&quot;https://pypi.python.org/pypi/Sumatra&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sumatra&lt;/a&gt; python package. It nicely links the version control commit number, machine state, and output files to each program run and has a django web interface to interact with the database of run info. The python API makes it very easy to include logging in my scripts.&lt;/p&gt;&#xA;" OwnerUserId="1303" LastEditorUserId="414" LastEditDate="2017-12-18T14:39:44.590" LastActivityDate="2017-12-18T14:39:44.590" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1898" PostTypeId="1" AcceptedAnswerId="2766" CreationDate="2012-04-10T20:12:01.763" Score="5" ViewCount="561" Body="&lt;p&gt;I have a small FEM implementation program. And I want to add imposing multifreedom constraints (MFC) feature to it. The theory of master-slave method is given &lt;a href=&quot;http://www.colorado.edu/engineering/cas/courses.d/IFEM.d/IFEM.Ch08.d/IFEM.Ch08.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (page 10 for general case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stiffness matrix (static truss) is given in a rearranged block form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xD;&#xA;\begin{bmatrix}&#xD;&#xA;       K_{uu} &amp;amp; K_{um} &amp;amp; K_{us} \\&#xD;&#xA;       K^{T}_{um} &amp;amp; K_{mm} &amp;amp; K_{ms} \\&#xD;&#xA;       K^{T}_{us} &amp;amp; K^{T}_{ms} &amp;amp; K_{ss}&#xD;&#xA;\end{bmatrix}&#xD;&#xA;\begin{bmatrix}&#xD;&#xA;       u_{u}\\&#xD;&#xA;       u_{m}\\&#xD;&#xA;       u_{s}&#xD;&#xA;\end{bmatrix}&#xD;&#xA;=&#xD;&#xA;\begin{bmatrix}&#xD;&#xA;       f_{u}\\&#xD;&#xA;       f_{m}\\&#xD;&#xA;       f_{s}&#xD;&#xA;\end{bmatrix}&#xD;&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The form of the general constraints equations is a linear combination of DOFs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{m}u_{m} + A_{s}u_{s} = g_{A}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can find $u_{s}$ from this as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{s} = -A^{-1}_{s}A_{m}u_{m} + A^{-1}_{s}g_{A} = Tu_{m} + g$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I should &quot;insert ($u_{s}$) into the partitioned stiffness matrix and &lt;strong&gt;symmetrize&lt;/strong&gt;&quot; to obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xD;&#xA;\begin{bmatrix}&#xD;&#xA;       K_{uu} &amp;amp; K_{um}+K_{us}T \\&#xD;&#xA;       symm &amp;amp; K_{mm}+T^{T}K^{T}_{ms}+K_{ms}T+T^{T}K_{ss}T&#xD;&#xA;\end{bmatrix}&#xD;&#xA;\begin{bmatrix}&#xD;&#xA;       u_{u}\\&#xD;&#xA;       u_{m}&#xD;&#xA;\end{bmatrix}&#xD;&#xA;=&#xD;&#xA;\begin{bmatrix}&#xD;&#xA;       f_{u}-K_{us}g\\&#xD;&#xA;       f_{m}-K_{ms}g&#xD;&#xA;\end{bmatrix}&#xD;&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to reproduce the last step but couldn't get the result. What is &quot;to symmetrize&quot;?&lt;/p&gt;&#xA;" OwnerUserId="188" LastEditorUserId="188" LastEditDate="2012-04-10T21:39:10.937" LastActivityDate="2012-07-29T14:06:40.547" Title="What does symmetrize mean? (imposing multifreedom constraints to stiffness matrix)" Tags="|finite-element|constraints|block-decomposition|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1899" PostTypeId="1" AcceptedAnswerId="1900" CreationDate="2012-04-10T23:08:56.113" Score="2" ViewCount="646" Body="&lt;p&gt;The version of PETSc installed on my machine has &lt;code&gt;PetscScalar&lt;/code&gt; set to be complex. I am making a matrix which has all real entries.&lt;/p&gt;&#xA;&lt;p&gt;Something like the following code compiles:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PetscScalar value_for_matrix;&#xA;value_for_matrix=some_expression*returns_PetscReal;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is the real value returned by the right hand side put into the real part of the PetscScalar value? If this doesn't work, how should I put a PetscReal value into the real part of a complex PetscScalar?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="23791" LastEditDate="2022-01-11T15:37:10.253" LastActivityDate="2022-01-11T15:37:10.253" Title="Implicitly casting PetscReal to the real part of PetscComplex" Tags="|petsc|c|complex-arithmetic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1900" PostTypeId="2" ParentId="1899" CreationDate="2012-04-11T00:43:18.257" Score="4" Body="&lt;p&gt;Yes, this does exactly what you would expect. If you want to create a complex number explicitly, you can multiply real values by &lt;code&gt;PETSC_i&lt;/code&gt;. Note that you can use &lt;code&gt;PetscRealPart()&lt;/code&gt;, &lt;code&gt;PetscImaginaryPart()&lt;/code&gt;, &lt;code&gt;PetscAbsScalar()&lt;/code&gt;, and &lt;code&gt;PetscConj()&lt;/code&gt; to manipulate complex numbers.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-04-11T00:43:18.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1901" PostTypeId="1" AcceptedAnswerId="1902" CreationDate="2012-04-11T03:21:29.343" Score="10" ViewCount="566" Body="&lt;p&gt;I am wondering how to find the eigenvalues of some sparse matrix in given interval [a, b] by iterative method. To my personal understanding, it is more obvious to use Krylov subspace method to find the extreme eigenvalues rather than the interior ones.&lt;/p&gt;&#xA;" OwnerUserId="866" LastActivityDate="2012-11-22T14:58:53.100" Title="How to find the interior eigenvalues by krylov subspace method?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1902" PostTypeId="2" ParentId="1901" CreationDate="2012-04-11T03:59:24.883" Score="10" Body="&lt;p&gt;The following strategy is called &lt;em&gt;shift and invert&lt;/em&gt; and depends upon two important facts:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$A-\tau I$ has the same spectrum as $A$, but shifted down by $\tau$, i.e., if $\lambda \in \sigma(A)$ then $\lambda-\tau \in \sigma(A-\tau I)$.&lt;/li&gt;&#xA;&lt;li&gt;Assuming that $A$ is invertible, the matrix $A^{-1}$ has a spectrum which is equal to the element-wise inverse of the spectrum of $A$, i.e., if $\lambda \in \sigma(A)$ then $1/\lambda \in \sigma(A^{-1})$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since $A-\frac{a+b}{2}I$ will have shifted the portion of $A$'s spectrum which is close to $\frac{a+b}{2}$ near the origin, the eigenvalues of $A$ near $\frac{a+b}{2}$ will be very large in $(A-\frac{a+b}{2}I)^{-1}$, and so it is reasonable to expect a Krylov algorithm to pick them up.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-11-22T13:53:08.840" LastActivityDate="2012-11-22T13:53:08.840" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1903" PostTypeId="2" ParentId="46" CreationDate="2012-04-11T05:57:22.177" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://plato.asu.edu/bench.html&quot; rel=&quot;nofollow&quot;&gt;Hans Mittelman's website&lt;/a&gt; is an excellent resource for navigating the current software options in numerical optimization.  He includes his &lt;a href=&quot;http://plato.asu.edu/sub/benchm.html&quot; rel=&quot;nofollow&quot;&gt;own benchmarks, as well as links to other benchmarks for test problems in optimization&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-04-11T05:57:22.177" CommentCount="0" CommunityOwnedDate="2012-04-11T05:57:22.177" ContentLicense="CC BY-SA 3.0" />
  <row Id="1904" PostTypeId="2" ParentId="1337" CreationDate="2012-04-11T14:51:42.930" Score="1" Body="&lt;p&gt;Adding to the previous answer. There is also &lt;a href=&quot;http://www.geuz.org/gmsh/&quot; rel=&quot;nofollow&quot;&gt;Gmsh&lt;/a&gt; and Octave package &lt;a href=&quot;http://octave.sourceforge.net/geometry/overview.html&quot; rel=&quot;nofollow&quot;&gt;geometry&lt;/a&gt; has a simple interface function data2geo.m that allows to create meshes from polygons you have in your Octave workspace (creation of polygons is quite simple with the package itself). Additionally, the package geometry also has a SVG class that allows you to read shapes from SVG files created with Inkscape or similar software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this sounds like propaganda, please forgive me, it is not. Your question seemed simple and this package is simple.&lt;/p&gt;&#xA;" OwnerUserId="1298" LastActivityDate="2012-04-11T14:51:42.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1905" PostTypeId="2" ParentId="1852" CreationDate="2012-04-11T15:00:58.287" Score="3" Body="&lt;p&gt;I think the answer from Geoff Oxberry is very good. It provides out-of-the-box solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to go on your own:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The mentioned L-systems may generate root-like structures if you provide the right rules. there is this book about &lt;a href=&quot;http://algorithmicbotany.org/papers/#webdocs&quot; rel=&quot;nofollow&quot;&gt;&quot;The algorithmic beauty of plants&quot;&lt;/a&gt;, but it doesn't cover root systems.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Diffusion-limited_aggregation&quot; rel=&quot;nofollow&quot;&gt;Diffusion limited aggregation&lt;/a&gt; processes may also generate root-like structures. If you aggregate spheres and once aggregated you produce the boolean union of the structure you will get the volume to mesh directly (smoothing needed, almost for sure).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As I said, not many solutions but maybe the ideas help you. If you implement something do not forget to release it with a free license! :D&lt;/p&gt;&#xA;" OwnerUserId="1298" LastActivityDate="2012-04-11T15:00:58.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1906" PostTypeId="1" AcceptedAnswerId="1914" CreationDate="2012-04-11T15:14:51.000" Score="7" ViewCount="259" Body="&lt;p&gt;Let's assume we have a planar domain whose boundary can be described with a polynomial curve (like Bezier curves).&#xA;Now assume that you want to produce a discretization of the boundary, i.e. you want to produce a polygon by sampling the polynomial defining the boundary of the domain. How can I produce a boundary that incurs in a error lower than some user provided tolerance? A measure of the error could be something like.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum_{\text{sides}}\int_0^1 ||\operatorname{line}(x_j,x_{j+1},t) - \partial\Omega_j(t)||^2 dt&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;That is, the square error of the polygon (whose sides are straight lines from vertex &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$x_{i+1}$&lt;/span&gt;) with respect to the polynomial domain whose sides are represented by the parametrization &lt;span class=&quot;math-container&quot;&gt;$\partial\Omega_j(t)$&lt;/span&gt;. Each side is parametrized in the interval &lt;span class=&quot;math-container&quot;&gt;$t \in [0,1)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to improve the shape2polygon function of the GNU/Octave package &lt;a href=&quot;http://octave.sourceforge.net/geometry/overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;geometry&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: The idea is to produce a polygonal representation of the original domain with the given error, with as few vertices as possible.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have implemented an GNU/Octave non-recursive version of the Ramer-Douglas-Peucker algorithm to simplify polylines and polygons. &lt;a href=&quot;https://sourceforge.net/p/octave/geometry/ci/default/tree/inst/simplifyPolyline_geometry.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;simplifyPolyline_geometry.m&lt;/a&gt; and &lt;a href=&quot;https://sourceforge.net/p/octave/geometry/ci/default/tree/inst/simplifyPolygon_geometry.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;simplifyPolygon_geometry.m&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I have implemented an GNU/Octave non-recursive version of the algorithm in &lt;em&gt;L. H. de Figueiredo (1993). &amp;quot;Adaptive Sampling of Parametric Curves&amp;quot;. Graphic Gems III&lt;/em&gt;. This function performs adaptive sampling of a parametrized planar curve. &lt;a href=&quot;https://sourceforge.net/p/octave/geometry/ci/default/tree/inst/curve2polyline.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;curve2polyline.m&lt;/a&gt;. The extension to polynomial domains is in &lt;a href=&quot;https://sourceforge.net/p/octave/geometry/ci/default/tree/inst/shape2polygon.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;shape2polygon.m&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1298" LastEditorUserId="1298" LastEditDate="2021-05-26T22:40:27.870" LastActivityDate="2021-05-26T22:40:27.870" Title="Converting from planar polynomial domain to planar polygon" Tags="|computational-geometry|matlab|octave|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="1907" PostTypeId="1" CreationDate="2012-04-11T17:46:48.347" Score="3" ViewCount="170" Body="&lt;p&gt;I'm using a PetscBag to store the input parameters of my program. At some point, I'm going to need to use python to plot these parameters against some output parameters, and &lt;code&gt;dict&lt;/code&gt;s along with matplotlib seem to be the way to go with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I want to use &lt;code&gt;PetscBagView()&lt;/code&gt; or something like it to write a petsc bag to a file, and then use some python thing (built-in part of the standard library, external package especially for this purpose, whatever works) to read it into a &lt;code&gt;dict&lt;/code&gt;. The idea is to make it easy to write matplotlib code that plots various input parameters against certain output parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;code&gt;dict&lt;/code&gt;s are the wrong thing to use for plotting the inputs of several simulations against their outputs, what should I use? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there an easy way to read a (binary or ascii) serialized PetscBag into python as a &lt;code&gt;dict&lt;/code&gt; ?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-04-22T16:35:10.847" LastActivityDate="2012-04-22T16:35:10.847" Title="Is there an easy way to read a PetscBag into a python dict?" Tags="|petsc|python|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="1908" PostTypeId="2" ParentId="1887" CreationDate="2012-04-11T18:24:06.597" Score="4" Body="&lt;p&gt;I think you can apply $LDL^{T}$ factorization for your matrix A. Since your matrix has non-negative eigenvalues, the diagonal matrix D will have non-negative entries along the diagonl. Then you can easily factorize $\hat{D} = \sqrt{D}$. And you get the matrix $G = L\hat{D}$. The eigendecomposition is not numerically stable, so I think you should avoid this kind of decomposition.&lt;/p&gt;&#xA;" OwnerUserId="866" LastActivityDate="2012-04-11T18:24:06.597" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1909" PostTypeId="1" AcceptedAnswerId="1910" CreationDate="2012-04-11T20:39:08.870" Score="11" ViewCount="6746" Body="&lt;p&gt;I'm solving the differential equation&#xA;$$&#xD;&#xA;   \left( \sigma^{2}(x) u &amp;#39;&amp;#39;(x) \right)&amp;#39;&amp;#39; = f(x), \;\;\; 0 \leqslant x \leqslant 1&#xD;&#xA;$$&#xA;with initial conditions $u(0) = u(1) = 0$, $u&amp;#39;&amp;#39;(0) = u&amp;#39;&amp;#39;(1) = 0$. Here $\sigma(x) \geqslant \sigma_{0} &amp;gt; 0$ is parameter. In operator form we can rewrite the differential equation as $Au = f$, where operator $A$ is positive definite. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following FEM scheme, I reduce my problem to an optimisation problem &#xA;$$&#xD;&#xA;   J(u) = (Au,u) - 2(f,u) \to \min_{u}&#xD;&#xA;$$&#xA;I introduce finite elements $h_{k}(x)$ as&#xA;$$&#xD;&#xA;   v_{k}(x) = \left\{ \begin{array}{rl} 1 - \left( \frac{x-x_{k}}{h} \right)^2, &amp;amp; x \in [x_{k-1},x_{k+1}] \\ 0, &amp;amp; \text{otherwise} \end{array} \right.&#xD;&#xA;$$&#xA;for any $k = 1,\ldots,n-1$, where $x_{k} = hk$, $h = \frac{1}{n}$. Finite elements $v_{0}(x)$ and $v_{n}(x)$ are introduced similarly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to find numericaly the vector $\alpha$ such that $u(x) = \sum_{k=0}^{n} \alpha_{k} v_{k}(x)$ solves the optimisation problem. We have&#xA;$$&#xD;&#xA;  J(u) = \sum\limits_{i=0}^{n} \sum\limits_{j=0}^{n} \alpha_{i} \alpha_{j} (Av_{i},v_{j}) - \sum\limits_{i=0}^{n} 2\alpha_{i} (v_{i},f) = \alpha^{T} V \alpha - 2\alpha^{T} b \to \min\limits_{\alpha},&#xD;&#xA;$$ &#xA;where $b_{i} = (f,v_{i})$ and $V_{i,j} = (Av_{i},v_{j})$. After differentiation with respect to $\alpha$ I receive&#xA;$$&#xD;&#xA;   V\alpha = b,&#xD;&#xA;$$&#xA;but here the stiffness matrix $V$ is singular. So what I have to do? Maybe I have to choose other finite elements?&lt;/p&gt;&#xA;" OwnerUserId="1180" LastActivityDate="2013-04-05T16:42:06.180" Title="FEM: singularity of the stiffness matrix" Tags="|finite-element|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1910" PostTypeId="2" ParentId="1909" CreationDate="2012-04-11T21:03:34.260" Score="13" Body="&lt;p&gt;In decreasing order of likelihood&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Incorrect basis.&lt;/strong&gt; From you description, it appears that you have exactly two quadratic functions with support on each element. That space is not a partition of unity and is not $C^1$ (continuous first derivatives). To discretize your fourth order problem directly (instead of reducing it to a system of second order equations, for example), you will need a $C^1$ basis. Note that the $C^1$ basis should be able to exactly reproduce all linear functions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Insufficient boundary conditions.&lt;/strong&gt; This will be blatantly obvious if you compute and plot the null space.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Incorrect assembly.&lt;/strong&gt; Check the map from elements to assembled ordering to confirm that it is what you expected, for example that it isn't reversing the orientation of elements.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Incorrect local assembly.&lt;/strong&gt; In 1D, you can analytically compute what the element stiffness matrix looks like (perhaps for a simplified case) and check that the code reproduces it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-04-12T13:32:12.723" LastActivityDate="2012-04-12T13:32:12.723" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1911" PostTypeId="1" AcceptedAnswerId="1912" CreationDate="2012-04-11T21:27:06.663" Score="1" ViewCount="267" Body="&lt;p&gt;I've made a matrix, and now I want to draw it on the screen to make a basic check of correctness. The &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatAssemblyEnd.html#MatAssemblyEnd&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; for &lt;code&gt;MatAssemblyEnd()&lt;/code&gt; states that I can pass an option to do this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-mat_view_draw - PetscDraws nonzero structure of matrix, using MatView() and PetscDrawOpenX().&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But when I run my program with this option, I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0]PETSC ERROR: --------------------- Error Message ------------------------------------&#xA;[0]PETSC ERROR: No support for this operation for this object type!&#xA;[0]PETSC ERROR: Viewer type draw not supported!&#xA;[0]PETSC ERROR: ------------------------------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does PETSc still support drawing to X terminals? Or is this just an indication of a bug somewhere in my code?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-04-16T16:54:42.153" LastActivityDate="2012-04-16T16:54:42.153" Title="Error message when trying to get PETSc to draw to an X terminal by passing -mat_view_draw" Tags="|petsc|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1912" PostTypeId="2" ParentId="1911" CreationDate="2012-04-11T23:31:41.690" Score="3" Body="&lt;p&gt;Configure probably did not find X11. You can check whether it was found using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep '\&amp;lt;PETSC_HAVE_X\&amp;gt;' $PETSC_DIR/$PETSC_ARCH/include/petscconf.h &#xA;#ifndef PETSC_HAVE_X&#xA;#define PETSC_HAVE_X 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you don't see &lt;code&gt;PETSC_HAVE_X&lt;/code&gt; defined, try reconfiguring &lt;code&gt;--with-x&lt;/code&gt; (perhaps with a path, see &lt;code&gt;./configure --help&lt;/code&gt; for details on what to specify).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-04-11T23:31:41.690" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1913" PostTypeId="2" ParentId="1906" CreationDate="2012-04-11T23:57:01.860" Score="3" Body="&lt;p&gt;The problem of simplifying a polygon is well-studied.&#xA;The Douglas-Peucker algorithm is among the most popular methods.&#xA;It can be implemented to run in time O(n log n).&#xA;See &lt;a href=&quot;http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm&quot; rel=&quot;nofollow&quot;&gt;the Wikipedia article on this algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="673" LastActivityDate="2012-04-11T23:57:01.860" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1914" PostTypeId="2" ParentId="1906" CreationDate="2012-04-12T11:25:54.250" Score="3" Body="&lt;p&gt;You want to perform an adaptive sampling of the boundary curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Bézier curves, use the de Casteljau algorithm recursively until the control points are approximately collinear. See for instance &lt;a href=&quot;http://www.antigrain.com/research/adaptive_bezier/index.html&quot; rel=&quot;nofollow&quot;&gt;Adaptive Subdivision of Bezier Curves&lt;/a&gt; from the Anti-Grain Geometry project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For general curves, see for instance &lt;a href=&quot;http://www.tecgraf.puc-rio.br/~lhf/ftp/doc/g5011.ps.gz&quot; rel=&quot;nofollow&quot;&gt;Adaptive sampling of parametric curves&lt;/a&gt;, in Graphics Gems V, 1995. &lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="640" LastEditDate="2012-04-12T11:31:05.960" LastActivityDate="2012-04-12T11:31:05.960" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1915" PostTypeId="2" ParentId="1841" CreationDate="2012-04-12T13:39:18.443" Score="3" Body="&lt;p&gt;If the part that is unchanged does not cover most of the matrix there is little you can do to save work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the part that is unchanged can be permuted by a symmetric permutation to occupy the top left corner of yopur matrix, you can first similarity transform it to (tri)diagonal form. Extending the transform to the whole matrix leaves a matrix of the form $\pmatrix{ T &amp;amp; A\cr B &amp;amp; C}$ with (tri)diagonal $T$, whose eigenvalues are those of the original matrix. Its eigenvalues are relatively cheap to find by some subspace iteration steps followed by inverse iteration for extracting the absolutely largest eigenvalues once reasonable approximations are available.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-12T13:39:18.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1916" PostTypeId="1" CreationDate="2012-04-13T02:30:04.087" Score="8" ViewCount="148" Body="&lt;p&gt;In both the potential reduction and primal path following interior point methods for linear programming, a barrier function is constructed which contains the terms $-\sum \log x_j$ where $x_j$ are the variables. This is to keep the variables from being 0 and thus keep the current solution feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in potential reduction, the new direction to move the current solution to is found by solving a linear program which contains as the single inequality constraint $||\mathbf{X^{-1}d}|| \leq \beta$ where $\beta &amp;lt; 1$ and $\mathbf{d}$ is the direction. This constraint is so that the new solution $\mathbf{x}+\mathbf{d}$ remains $&amp;gt; 0$ and thus feasible. $\mathbf{X}$ is a diagonal matrix with the elements of the variable vector $\mathbf{x}$ forming the diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in the primal path method (&lt;a href=&quot;https://secure.wikimedia.org/wikipedia/en/wiki/Interior_point_method&quot;&gt;related approach&lt;/a&gt;), one can just solve the Lagrangian dual of the barrier function problem and the new solution is guaranteed to be feasible - i.e. the property $\mathbf{||X^{-1}d}|| \leq \beta$ is guaranteed to be satisfied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some good intuition as to why primal path can get away with just solving the Lagrangian dual (and thus not have to deal with inequalities) while potential reduction cannot? The best I can think of is that the barrier function in the potential reduction algorithm contains the term $\log \mathbf{s&amp;#39;x}$ where $\mathbf{s&amp;#39;x}$ is the duality gap ($\mathbf{s}$ is the slack vector for slacks in the dual version of the problem). This term thus strongly pulls $\mathbf{x+d}$ into a potentially infeasible region. The barrier function for the primal path only has the term $\mathbf{c&amp;#39;x}$ (where $\mathbf{c&amp;#39;x}$ is the objective function of the original problem) in addition to the log terms in the barrier function so it does not exert as strong a pull (as it will not for instance go to $-\infty$ when $\mathbf{c&amp;#39;x}$ goes to 0 unlike $\log \mathbf{s&amp;#39;x}$ which will). However I don't know how to formalize this.&lt;/p&gt;&#xA;" OwnerUserId="288" LastEditorUserId="288" LastEditDate="2012-04-13T05:42:47.873" LastActivityDate="2012-04-13T05:42:47.873" Title="Potential Reduction and Primal Path following methods" Tags="|linear-programming|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1918" PostTypeId="2" ParentId="1906" CreationDate="2012-04-13T15:34:17.037" Score="1" Body="&lt;p&gt;The question to me is simply whether you want to have the &lt;em&gt;minimal&lt;/em&gt; number of vertices, or whether you're just interested in a &lt;em&gt;small&lt;/em&gt; number of vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd use the obvious, greedy adaptation algorithm: you start with a small number of vertices and get a poor boundary approximation. Let's say you have $N$ segments in this step. Then for each segment $i,i=1\ldots N$ of your polygonal approximation, you compute the error $e_i$ of this one segment, and if $e_i&amp;gt;\frac{tol}{N}$ then you insert a new vertex along the boundary and split the segment into two. Iterate this until all segments have an error less than $\frac{tol}{N}$ and you have a boundary approximation with an overall error equal to or less than $tol$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-13T15:34:17.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1919" PostTypeId="2" ParentId="1616" CreationDate="2012-04-13T17:31:12.103" Score="2" Body="&lt;p&gt;If you would compute in exact arithmetic, there is no way of rewriting your formula to get better accuracy; you cannot do better than $O(\sigma/\sqrt{N})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you get low accuracy just because of roundoff errors, you can use the formula $E(X)=x+ N^{-1}\sum (X_k-x)$, using for $x$ an arbitrary approximation of $E(X)$, e.g., the mean of the first few terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can reduced the rounding errors further by summing the entries of this sum in the ordering by their absolute values, starting with the smallest.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-13T17:31:12.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1920" PostTypeId="1" AcceptedAnswerId="1924" CreationDate="2012-04-13T17:48:14.107" Score="9" ViewCount="554" Body="&lt;p&gt;I know that in order to solve symmetric eigenvalue problem $Ax = \lambda x$, we can use the Sylvester Inertia Law, that is the number of eigenvalues of $A$ less than $a$ equals the number of negative entries of $D$ where diagonal matrix $D$ comes from the LDL factorization of $A-aI = LDL^{T}$. Then, by bisection method, we can find all or some eigenvalues as desired. I wish to know if there exists a generalization of the Sylvester Inertia Law for symmetric generalized eigenvalue problems, that is solving $Ax= \lambda Bx$, where $A$ and $B$ are symmetric matrices. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="866" LastEditorUserId="9" LastEditDate="2012-04-13T19:05:27.117" LastActivityDate="2012-04-14T19:16:01.793" Title="Is there a generalization of the Sylvester Inertia Law for the symmetric generalized eigenvalue problem?" Tags="|linear-algebra|eigensystem|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1921" PostTypeId="2" ParentId="1920" CreationDate="2012-04-13T17:55:41.047" Score="4" Body="&lt;p&gt;In the case where $B$ is Hermitian and positive-definite, a Cholesky factorization of $B$, say $B = L L^H$, gives that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;Ax=LL^H x \lambda,&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this equation can be manipulated to show that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;(L^{-1} A L^{-H})(L^H x) = (L^H x) \lambda,&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where it should be clear that $C \equiv L^{-1} A L^{-H}$ preserves the symmetry of $A$, and also has the same spectrum as the pencil $(A,B)$. Thus, after forming $C$, with a Cholesky factorization followed by a &lt;a href=&quot;http://code.google.com/p/elemental/downloads/detail?name=FLAWN56.pdf&quot; rel=&quot;nofollow&quot;&gt;two-sided triangular solve&lt;/a&gt;, you can directly apply the Sylvester inertia law to $C$ to glean information about the eigenvalues of the pencil $(A,B)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, since Sylvester's Law of Inertia is invariant with respect to &lt;em&gt;congruence&lt;/em&gt; transformations, e.g., $S \cdot S^H$, then the matrix $C$ is congruent to $A$ through the transformation $L^{-1} \cdot L^{-H}$, and so $C$ has the same inertia as $A$. However, if the inertia of $C-\sigma I$ is desired, for some nonzero shift $\sigma$, then we can no longer simply consider $A$.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-04-13T18:25:37.617" LastActivityDate="2012-04-13T18:25:37.617" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1922" PostTypeId="1" AcceptedAnswerId="3525" CreationDate="2012-04-14T14:53:24.800" Score="4" ViewCount="863" Body="&lt;p&gt;According to &lt;a href=&quot;http://surface.syr.edu/cgi/viewcontent.cgi?article=1012&amp;amp;context=npac&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Ideally, a pseudorandom number generator would produce a stream of&#xA;  numbers that:  &lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;are uniformly distributed,&lt;/li&gt;&#xA;  &lt;li&gt;are uncorrelated,&lt;/li&gt;&#xA;  &lt;li&gt;never repeats itself,&lt;/li&gt;&#xA;  &lt;li&gt;satisfy any statistical test for randomness,&lt;/li&gt;&#xA;  &lt;li&gt;are reproduceable (for debugging purposes),&lt;/li&gt;&#xA;  &lt;li&gt;are portable (the same on any computer),&lt;/li&gt;&#xA;  &lt;li&gt;can be changed by adjusting the initial &quot;seed&quot; value,&lt;/li&gt;&#xA;  &lt;li&gt;can easily be split into many independent subsequences&lt;/li&gt;&#xA;  &lt;li&gt;can be generated rapidly using limited computer memory&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But, as the paper suggests, it is impossible to satisfy all criteria simultaneously.  I'm curious to know how well different PRNG satisfy each criteria.  I'd really like to find a reference to a thorough discussion of each PRNG's strengths and weaknesses, but I'm open to any heuristics as well :)&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-10-21T06:48:40.340" Title="Quality Measures for Various Pseudo-Random Number Generators" Tags="|reference-request|random-number-generation|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1923" PostTypeId="2" ParentId="1637" CreationDate="2012-04-14T19:03:56.050" Score="2" Body="&lt;p&gt;A short explanation of the underlying principle is given in slides 12 and 13 of my lecture &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/ms/slopeslides.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/ms/slopeslides.pdf&lt;/a&gt;&#xA;(It is in terms of an ODE and slopes, but the PDE case is easy to derive once the principle is understood, and the usual derivative case is obtained by specializing the slope to two equal arguments.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-14T19:03:56.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1924" PostTypeId="2" ParentId="1920" CreationDate="2012-04-14T19:16:01.793" Score="7" Body="&lt;p&gt;Yes, if the pencil is definite, i.e., if $A$ and $B$ are Hermitian and $B$ is positive definite. Then the signature of $A-\sigma B$ has the same interpretation for the eigenvalue problem $(A-\lambda B)x=0$ as in the case $B=I$. A more general result of this kind holds for any definite nonlinear eigenvalue problem $A(\lambda)x=0$. See Section 5.3 of my book &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Arnold Neumaier, &#xA;Introduction to numerical analysis,&#xA;Cambridge Univ. Press, Cambridge 2001.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $(A-\lambda B)x=0$, the proof of my assertion can be deduced from  the argument given by Jack Poulson upon noting that $C-\sigma I$ and $A-\sigma B$ are congruent, hence have the same inertia. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, one can directly compute the inertia of $A-\sigma B$, and doesn't need a Cholesky factorization of $B$ to form $C$. Indeed, if $B$ is ill-conditioned then the numerical formation of $C$ degrades the quality of the inertia test.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-14T19:16:01.793" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1925" PostTypeId="1" CreationDate="2012-04-14T19:22:11.037" Score="12" ViewCount="6099" Body="&lt;p&gt;Please list the Python package (petsc4py, etc...) and the sparse direct solvers it supports.  One (community-wiki) answer per package, please.&lt;/p&gt;&#xA;" OwnerUserId="1327" LastEditorUserId="9" LastEditDate="2012-04-14T22:31:22.003" LastActivityDate="2014-07-08T11:25:57.313" Title="What are the best Python packages/interfaces to sparse direct solvers?" Tags="|python|linear-solver|sparse-matrix|" AnswerCount="4" CommentCount="3" CommunityOwnedDate="2012-04-14T22:30:14.470" ContentLicense="CC BY-SA 3.0" />
  <row Id="1926" PostTypeId="2" ParentId="1925" CreationDate="2012-04-14T19:51:36.943" Score="10" Body="&lt;p&gt;All direct solvers supported by PETSc are available in Python under a common interface via &lt;a href=&quot;https://bitbucket.org/petsc/petsc4py/&quot; rel=&quot;nofollow&quot;&gt;petsc4py&lt;/a&gt;. Supported sparse direct solver packages include the PETSc native direct solvers, MUMPS, PasTiX, SuperLU, SuperLU_DIST, Umfpack, CHOLMOD, Spooles, LUSOL, MATLAB, and ESSL. See the &lt;code&gt;MATSOLVER*&lt;/code&gt; man pages &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/index.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="1553" LastEditDate="2014-07-08T11:25:57.313" LastActivityDate="2014-07-08T11:25:57.313" CommentCount="0" CommunityOwnedDate="2012-04-14T22:30:26.090" ContentLicense="CC BY-SA 3.0" />
  <row Id="1928" PostTypeId="2" ParentId="1925" CreationDate="2012-04-14T21:09:22.150" Score="4" Body="&lt;p&gt;All direct solvers supported by &lt;a href=&quot;http://trilinos.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;Trilinos&lt;/a&gt; through &lt;a href=&quot;http://trilinos.sandia.gov/packages/amesos/&quot; rel=&quot;nofollow&quot;&gt;Amesos&lt;/a&gt; are available in Python under a common interface via &lt;a href=&quot;http://trilinos.sandia.gov/packages/pytrilinos/&quot; rel=&quot;nofollow&quot;&gt;PyTrilinos&lt;/a&gt;. Supported sparse direct solver packages include UMFPACK, TAUCS, PARDISO, SuperLU, and MUMPS. See &lt;a href=&quot;http://trilinos.sandia.gov/packages/pytrilinos/UsersGuide.pdf&quot; rel=&quot;nofollow&quot;&gt;this technical report&lt;/a&gt; discussing sparse linear algebra with PyTrilinos for some examples.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-04-14T21:09:22.150" CommentCount="0" CommunityOwnedDate="2012-04-14T22:30:55.933" ContentLicense="CC BY-SA 3.0" />
  <row Id="1929" PostTypeId="2" ParentId="1925" CreationDate="2012-04-14T21:12:58.370" Score="8" Body="&lt;p&gt;SciPy supports sparse linear algebra via &lt;code&gt;scipy.sparse.linalg&lt;/code&gt; (see the &lt;a href=&quot;http://scipy-lectures.github.com/advanced/scipy_sparse/solvers.html&quot;&gt;SciPy Documentation&lt;/a&gt;). SciPy supports the sparse direct solver packages SuperLU and UMFPACK.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-04-14T21:12:58.370" CommentCount="0" CommunityOwnedDate="2012-04-14T22:30:47.370" ContentLicense="CC BY-SA 3.0" />
  <row Id="1930" PostTypeId="2" ParentId="1925" CreationDate="2012-04-14T21:35:59.763" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://pysparse.sourceforge.net/index.html&quot;&gt;PySparse&lt;/a&gt; supports the sparse direct solver packages SuperLU and UMFPACK; see the &lt;a href=&quot;http://pysparse.sourceforge.net/fact.html&quot;&gt;solver documentation here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-04-14T21:35:59.763" CommentCount="0" CommunityOwnedDate="2012-04-14T22:30:37.370" ContentLicense="CC BY-SA 3.0" />
  <row Id="1931" PostTypeId="2" ParentId="1922" CreationDate="2012-04-15T03:46:56.360" Score="3" Body="&lt;p&gt;I believe one of the Art of Computer Programming books by Knuth is devoted exclusively to random numbers. It's probably not up to date, but it would be the place where I'd start reading.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-15T03:46:56.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1932" PostTypeId="1" AcceptedAnswerId="1933" CreationDate="2012-04-15T10:42:50.260" Score="10" ViewCount="4787" Body="&lt;p&gt;I have implemented CG in FORTRAN by linking it to Intel MKL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When there are statements like:&#xA;(&lt;a href=&quot;http://en.wikipedia.org/wiki/Conjugate_gradient_method#Example_code_in_GNU_Octave&quot; rel=&quot;nofollow noreferrer&quot;&gt;Refer Wikipedia&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; p=r; &#xA; x=x+alpha*p&#xA; r=r-alpha*Ap;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or similar ones in &lt;a href=&quot;http://www.netlib.org/templates/matlab/qmr.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;QMR&lt;/a&gt; (in much greater quantity)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v_tld = r;&#xA;y = v_tld;&#xA;rho = norm( y );&#xA;w_tld = r;&#xA;z = w_tld;&#xA;xi = norm( z ); (and more)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does it make sense to use BLAS Level 1 implementations such as DAXPY, DCOPY, DSCAL?&lt;/strong&gt;&#xA;The motivation for my question is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I have 2 implementations of the algorithms. One wherein I have only linked Norms and MatVecs to MKL; copying, scaling and adding is done by Fortran's intrinsic functions and another where every possible subroutine is carried out by BLAS.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I was of the notion that nothing can get faster than BLAS. But, it turns out that my code using Fortran's intrinsic functions ran 100% faster than one with BLAS Level 1 subroutines (FWIW, This wasn't a small problem, it was solving a dense system of size 13k x 13k which filled up my 4 GB RAM). I was running both on 2 threads (on a 2 core machine) &lt;code&gt;ifort QMR.f90 -mkl&lt;/code&gt; with &lt;code&gt;MKL_DYNAMIC=TRUE&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I had &lt;a href=&quot;https://stackoverflow.com/questions/10156487/how-do-you-extend-blas-subroutines&quot;&gt;asked a question on SO&lt;/a&gt; regarding the extension of BLAS but as I tried to include BLAS Level 1 into my code, my code kept getting slower and slower.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Am I doing something wrong or is this expected?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, Does it make sense to try extend BLAS to do non-obvious operations like &lt;code&gt;y = 2.89*x&lt;/code&gt; by &lt;code&gt;DCOPY(n,2.89*x,1,y,1) or even DSCAL then DCOPY&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What is also interesting is, &lt;code&gt;DDOT&lt;/code&gt; and &lt;code&gt;DNRM2&lt;/code&gt; improve performance. I attributed it to the fact that since they carry out double precision multiplications, putting them in parallel might help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supplementary Question : &lt;strong&gt;When do you decide whether a BLAS Level 1 operation is actually going to aid the performance?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding : &#xA;Currently, I am running on a i3 2.13 GHz Laptop with 4 GB RAM and Debian 64 bit &lt;a href=&quot;http://www.pastebin.com/8rVqvxPm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proc info here&lt;/a&gt;. But, I get similar answers on an Intel Xeon 12 core Workstation with 24 GB RAM.&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-01-25T21:40:48.903" Title="Are DAXPY, DCOPY, DSCAL overkills?" Tags="|performance|fortran|blas|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1933" PostTypeId="2" ParentId="1932" CreationDate="2012-04-15T16:27:23.227" Score="6" Body="&lt;p&gt;If your goal is really to squeeze as much performance out as possible, then it is important to remember:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The (BLAS) library might not have been tuned for your exact system/configuration. &lt;/li&gt;&#xA;&lt;li&gt;Library developers make mistakes. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A vendor-tuned BLAS library should certainly be your default approach, but if you have taken the time to time individual kernels and noticed that some other implementation is faster, then, by all means, use the other implementation. Missing out on the usage of vector intrinsics could possibly lead to the large performance difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible that your best bet for simple routines like &lt;strong&gt;daxpy&lt;/strong&gt; and &lt;strong&gt;dscal&lt;/strong&gt; is a hand-written loop which exploits vector intrinsics.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-04-15T16:27:23.227" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1934" PostTypeId="1" CreationDate="2012-04-15T20:18:22.110" Score="4" ViewCount="189" Body="&lt;p&gt;Assume I am running an iterative method, I have a rough estimate of how many iterations it will need, How do best estimate the time it will run for in serial?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, If I have Conjugate Gradient (for Ax=b), how do I estimate how long it will run for given that I know the matrix dimension?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I wish to ask :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;One can carry out a rough calculation by ignoring caches and compiler optimizations but is there any way to account for them? Of course, accounting for each one of them is impossible but at least the significant ones. I was intrigued by the &lt;a href=&quot;https://scicomp.stackexchange.com/a/1729/529&quot;&gt;analysis posted on my other question&lt;/a&gt; and wished to be able to find out estimates for other algorithms but with a tighter bound.&lt;/li&gt;&#xA;&lt;li&gt;How do you find out cost of optimized operations? For instance, MatVecs are $O(N^2)$. When calculating for BLAS &lt;code&gt;DGEMV&lt;/code&gt;, what leading constant do you assume? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;3. Given that I know the time it ran in serial mode, how I can estimate the time for parallel mode? I've looked into Amdahl's law but I'm not sure how to use it for iterative methods.&lt;/em&gt; &lt;strong&gt;Answered&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S : An online PDF (or any reference) for finding out more on this would be great!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Update :&#xA;For solving a 10000x10000 CG Problem with the DGEMV as bottleneck, I did the following:&#xA;I took matrices of size from 2k x 2k to 19k x 19k and plotted a graph of Time (for random matvec) vs Size^2. I multiplied time with 2.13E9 (for GHz) to give the graph of Total Operations vs Size^2. Then I approximate the slope of the line which gives me Operations = (slope) * N^2. In my case, the constant was 45.45454545 and the graph was dead straight. For a 10k x 10k CG problem, it takes 1 matvec per iteration (no preconditioner) and 550 such iterations. That's a total of 550 matvecs $\approx$ 550*45.45*N^2 operations $\approx$ 550*45.45*1e8 / 2.13e9 seconds $\approx$ 1000 seconds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/10215939/whats-wrong-with-my-running-time-calculation&quot;&gt;It turns out that&lt;/a&gt; (I have requested the moderator at SO to delete that question):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I am not accounting for Instructions per Second (and per clock). Consequently, my running time is 120 seconds in reality (for 10k x 10k CG) while my calculations are at 1000 seconds! How do I account for IPS/IPC?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;45 is too huge a leading constant for MatVecs. Ideally, it should be around 2. Where did I go wrong in this calculation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How do I improve my theoretical predictions?&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-04-21T08:13:05.333" Title="Estimating time for running serial/parallel codes" Tags="|linear-algebra|blas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1935" PostTypeId="2" ParentId="1932" CreationDate="2012-04-15T20:31:21.687" Score="3" Body="&lt;p&gt;Given the state of optimizing compilers now a days, I don't think there's much voodoo in the linear BLAS routines, e.g. &lt;code&gt;DAXPY&lt;/code&gt;, &lt;code&gt;DCOPY&lt;/code&gt;, and &lt;code&gt;DSCAL&lt;/code&gt;, that your compiler won't do already, e.g. SSE-vectorization and loop unrolling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the code is the same, the only difference between your routine and a call to MKL's BLAS is the overhead of the function call and whatever extra magic MKL might be trying to do in there. If this is the case, the difference between your code and MKL's code should be a constant, independent of the problem/vector size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question has interesting echoes of &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1194/how-much-better-are-fortran-compilers-really&quot;&gt;this question&lt;/a&gt;, which also uses &lt;code&gt;DAXPY&lt;/code&gt; as an example.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-04-15T20:31:21.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1936" PostTypeId="2" ParentId="1934" CreationDate="2012-04-15T20:32:57.467" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;It's pretty hard to get a meaningful analysis of cache effects and optimizations since the benefit you get from them depends on the implementation. (For example, are you accessing memory row-wise or column-wise? This will affect memory locality and cache hits. Are the inner loops possible to unroll? What intrinsics are available for your platform?)&lt;/li&gt;&#xA;&lt;li&gt;Finding constants for BLAS implementations is very platform specific so you should probably try some benchmarks (try some datasets with a size N, N^2, N^3 etc)&lt;/li&gt;&#xA;&lt;li&gt;A good introduction to Amdahl's law with some modifications for heterogenous environments is the paper &quot;Amdahl's law in the multicore era&quot; found &lt;a href=&quot;http://research.cs.wisc.edu/multifacet/papers/ieeecomputer08_amdahl_multicore.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. Generally, for rough estimates, try to divide the operation into parallel operations, communication costs (not always applicable) and serial operations. You can then use Amdahl's law to see what kind of speedup you might expect.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;While run time analysis a priori is a useful tool, determining constants and the like is better done empircally, in my opinion. There are a lot of subtle stuff going on in the background when compiling and running scientific apllications.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastEditorUserId="1273" LastEditDate="2012-04-15T20:45:02.897" LastActivityDate="2012-04-15T20:45:02.897" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1937" PostTypeId="2" ParentId="1932" CreationDate="2012-04-16T01:13:54.857" Score="2" Body="&lt;p&gt;The BLAS standard actually has several checks for correctness of the function arguments that are unnecessary in many situations.  See this reference implementation of &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d9/dcd/daxpy_8f_source.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;daxpy.f&lt;/code&gt;&lt;/a&gt;.  Additionally, constants like &lt;code&gt;INCX&lt;/code&gt; are usually known to you at compile-time, but may not be assumed by the implementation.  BLAS calls cross compilation units, and I am not aware of any compilers that will be able to optimize these out without turning on whole program optimization.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A funny sidenote to this is that the Intel Compiler now recognizes BLAS 3 matrix-matrix multiply loops, and will transform this code into the equivalent &lt;em&gt;x&lt;/em&gt;&lt;code&gt;gemm&lt;/code&gt; call, with enough optimization enabled.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-04-16T01:13:54.857" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1938" PostTypeId="1" CreationDate="2012-04-16T05:34:30.560" Score="2" ViewCount="2351" Body="&lt;p&gt;I wrote a function to search a Binary Search Tree, but I have logic problems: When I insert some values, and I have a tree of 2 levels, and the final level (2 in this case) is not full (full is that it does not have all child nodes) it returns 3, not 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// --- Definition of depth()&#xA;template&amp;lt;typename DataType&amp;gt;&#xA;int BST&amp;lt;DataType&amp;gt;::depth()&#xA;{&#xA;&#xA;  int counter = 0; &#xA;&#xA;  return Auxdepth(counter, myRoot);&#xA;&#xA;}&#xA;&#xA;&#xA;// --- Definition of Auxdepth()&#xA;template&amp;lt;typename Datatype&amp;gt;&#xA;int BST&amp;lt;Datatype&amp;gt;::Auxdepth(int &amp;amp;counter, BinNodePointer subtree)&#xA;{&#xA; //BinNodePointer parameter is myRoot, which is the root node&#xA;int a;&#xA;int b; &#xA;&#xA;if(empty()) return -1;// garbage value&#xA;&#xA;if(subtree-&amp;gt;left == 0 &amp;amp;&amp;amp; subtree-&amp;gt;right == 0)&#xA;    return counter;&#xA;&#xA;else if(subtree-&amp;gt;left == 0){&#xA;    int x = counter++;&#xA;    a= Auxdepth(x,subtree-&amp;gt;right);&#xA;    return a;&#xA;}&#xA;&#xA;else if(subtree-&amp;gt;right == 0){&#xA;    int x = counter++;&#xA;    a = Auxdepth(x,subtree-&amp;gt;left);&#xA;    return a;&#xA;}&#xA;&#xA;if(subtree != 0) { // Recursion&#xA;    int x = counter + 1;&#xA;    cout &amp;lt;&amp;lt; &quot;This is x &quot; &amp;lt;&amp;lt; x &amp;lt;&amp;lt; endl;&#xA;    a = Auxdepth(x, subtree-&amp;gt;left);&#xA;    b = Auxdepth(x, subtree-&amp;gt;right);&#xA;&#xA;            if(a &amp;gt; b) return a;&#xA;            else return b;&#xA;    }&#xA;} &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: The depth function is there just to be more friendly.  Can anyone tell me where is my error? &lt;/p&gt;&#xA;" OwnerUserId="1248" LastEditorUserId="9" LastEditDate="2012-04-16T10:42:16.007" LastActivityDate="2012-04-16T12:23:18.757" Title="Depth of a Binary Search Tree" Tags="|algorithms|graph-theory|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1939" PostTypeId="1" AcceptedAnswerId="1949" CreationDate="2012-04-16T06:42:09.657" Score="6" ViewCount="494" Body="&lt;p&gt;I have the positions of a large number of charges (the strengths are known, but are also variables). Are there any tools that will allow me to visualize the electric field induced by these charges (or by a subset of these charges)? Are there such tools that are connected to visualization routines, so that these can be converted into figures for presentations or articles? &lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-04-21T19:24:50.030" Title="Tools for computing an electric field based on location of charges?" Tags="|visualization|electromagnetism|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1940" PostTypeId="1" AcceptedAnswerId="1946" CreationDate="2012-04-16T08:08:09.480" Score="4" ViewCount="151" Body="&lt;p&gt;Suppose a set of $n$ points,  $n\in M$, is given in some $d-$dimensional space,  $X\in\mathbb{R}^{n\times d}$. Among these $n$ points, some $k\in K$ are selected, so $k&amp;lt;n$, and the distances from these $k$ points to a all other points are calculated and stored in $D\in\mathbb{R}^{n\times k}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, given only this distance information, $D$, following the &lt;em&gt;cosine law&lt;/em&gt; one is able to construct&#xA;the matrix of inner-products between all points from $M$ and points from $K$, with respect to an origin $s\in K$, $$x_ik_j=d_{si}d_{sj}\cdot cos(\alpha)= -\frac{1}{2}\left(d_{ij}^2 - d_{si}^2 - d_{sj}^2\right),~~~~~~i\in\{1, \dots, n\}, ~~j\in\{\&#xD;&#xA;1, \dots, k\} \tag 1.$$ I'm interested in the restrictions of the origin positions in this case when only distances are supplied. With the above approach, for instance, I could not specify some point that is not in $K$ to be the origin, since not all points would have distance information to it. I think that I could specify a linear combination of points from $K$ to be the origin (or?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case only coordinates are given, then, by a simple shift, I could specify an arbitrary origin position. But, with the distance information to only a subset of points, I suppose the set of origin choices is more restricted. Is it, and if so, it what way?  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To simplify the matters: given the above distance info, what would be the restrictions on the choice of origin? Am I right to state that not all points could be chosen as the origin, because of insufficient distance information?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-04-17T18:44:53.890" LastActivityDate="2012-04-17T18:44:53.890" Title="Constructing the origin position by transforming distance information" Tags="|linear-algebra|matrix|high-dimensional|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1941" PostTypeId="2" ParentId="1938" CreationDate="2012-04-16T10:37:37.450" Score="0" Body="&lt;p&gt;I'm not sure if this is your problem, but you are passing around counter by reference, and that is surely going to cause errors the way you have implemented things currently.  Try switching from a call-by-reference to a call-by-value and returning the value of counter to make a copy.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-04-16T10:37:37.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1942" PostTypeId="2" ParentId="1938" CreationDate="2012-04-16T12:23:18.757" Score="2" Body="&lt;p&gt;What about this implementation....(the code is C, but you could easily translate to your C++ code).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int depth(Tree *t)&#xA;{&#xA;  if(t==NULL)&#xA;    return -1;&#xA;  else&#xA;    return 1 + max( depth(t-&amp;gt;subleft), depth(t-&amp;gt;subright) );&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know if your version is more efficient, but this one works fine for me.&lt;/p&gt;&#xA;" OwnerUserId="484" LastActivityDate="2012-04-16T12:23:18.757" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1943" PostTypeId="2" ParentId="451" CreationDate="2012-04-16T13:36:06.323" Score="2" Body="&lt;p&gt;Monte Carlo has a very attractive property:  it has the same convergence rate independent of the dimension of the problem.  So, even for problems of dimension $d=4,...,100$, they will converge at the same rate as problems with dimension $d=100,101,...\infty$.  So, even as your dimension increases from 4 to 100, monte carlo will probably be faster anyways.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-04-16T13:36:06.323" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1944" PostTypeId="1" AcceptedAnswerId="1948" CreationDate="2012-03-19T16:07:20.843" Score="5" ViewCount="695" Body="&lt;p&gt;So, we are solving the Biharmonic equation ($\Delta^2 u = f$) on a rectangle by solving the Poisson equation ($\nabla^2 u = f$) two times. We have nice boundary conditions, $u = 0$ and $\Delta u = 0$ on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use a 5 point scheme where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{h^2} \delta_x^2u_p + \frac{1}{k^2} \delta_y^2u_p = f_p + \tau_p$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we get the following expression for the truncation error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tau_p = \frac{1}{12}h^2\partial_x^4u_p + \frac{1}{12}k^2\partial_y^4u_p$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming &quot;nice&quot; initial value function $f$ we want to prove that the method is convergent, ie. that the global error somehow goes to zero when the step sizes $h$ and $k$ goes to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any clues on how to proceed to show this?&lt;/p&gt;&#xA;" OwnerUserId="1333" OwnerDisplayName="burk" LastEditorUserId="9" LastEditDate="2012-04-16T17:02:19.990" LastActivityDate="2012-04-16T19:50:15.383" Title="Proving convergence of 5 point scheme for the Poisson equation" Tags="|pde|convergence|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1945" PostTypeId="2" ParentId="1939" CreationDate="2012-04-16T16:43:15.180" Score="5" Body="&lt;p&gt;If the number of charges isn't too large, hand-coding the formula for the potential and then taking the derivative symbolically with &lt;a href=&quot;http://sagemath.org/&quot; rel=&quot;noreferrer&quot;&gt;Sage&lt;/a&gt; or Mathematica should be easy enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an enormous number of charges (several tens or more, in a file to long to convert to something by hand), you could read the charges and their positions into a python list, calculate the potential for each of them separately, and then add all the potentials together.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-04-16T16:43:15.180" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1946" PostTypeId="2" ParentId="1940" CreationDate="2012-04-16T16:52:45.883" Score="5" Body="&lt;p&gt;If you know the Gram matrix $G_{ik}=(x_i-x_0,x_k-x_0)$ of inner products with respect to the origin $x_0$, you can get the shifted Gram matrix $G_{ik}&amp;#39;=(x_i-x,x_k-x)$ with respect to any affine combination $x=x_0+\sum a_j (x_j-x_0)$ by expanding the inner products using $x_i-x=x_i-x_0-\sum a_j (x_j-x_0)$  and expressing the result in terms of the $G_{ik}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If only the part $G_{K:}$ of $G$ is known and the points indexed by $K$ span togeher with $x_0$ the full space then one can reconstruct the whole of $G$. Indeed, under these assumptions, $G_{KK}$ has the same rank $d$ as $G_{K:}$,&#xA;and they span the same column space. The full Gram matrix is now $G=G_{K:}^TG_{KK}^+G_{K:}$, with $+$ denoting the pseudo inverse.&#xA;(Numerically, one must think about ho2w to get this numerically stable.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the above recipe applies. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-04-17T14:38:15.020" LastActivityDate="2012-04-17T14:38:15.020" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1947" PostTypeId="2" ParentId="1944" CreationDate="2012-04-16T17:09:57.563" Score="1" Body="&lt;p&gt;Have you tried some good books about finite differences? Pretty sure that they show how to prove that a given method is convergent or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;J. Strikwerda, Finite Difference Schemes and Partial Differential Equations, SIAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://pages.cs.wisc.edu/~strik/strik.html&quot; rel=&quot;nofollow&quot;&gt;http://pages.cs.wisc.edu/~strik/strik.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R. LeVeque, Finite Difference Methods for Ordinary and Partial Differential Equations: Steady State and Time Dependent Problems, SIAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/index.html&quot; rel=&quot;nofollow&quot;&gt;http://faculty.washington.edu/rjl/fdmbook/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="484" LastActivityDate="2012-04-16T17:09:57.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1948" PostTypeId="2" ParentId="1944" CreationDate="2012-04-16T19:50:15.383" Score="2" Body="&lt;p&gt;From what I can remember, proving stability for the five point stencil has a classic proof involving the spectral radius because the eigenvalues are a simple trigonometric function. While I haven't got my numerical analysis books handy, I was able to find the proof &lt;a href=&quot;http://www.math.wisc.edu/~rossmani/math712_Fall06/am585winter06.pdf&quot; rel=&quot;nofollow&quot;&gt;in p. 43&lt;/a&gt; of this note (this is probably the notes which ended up as the previously mentioned LeVeque book)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you probably want to look at the 1D case first since the note is very brief on the 2D case. I suppose it depends if you want to take the eigenvalue identity for granted, the rest is quite straightforward.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2012-04-16T19:50:15.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1949" PostTypeId="2" ParentId="1939" CreationDate="2012-04-16T20:22:03.267" Score="6" Body="&lt;p&gt;-Short answer for small molecules&lt;br&gt;&#xA;APBS + VMD (&lt;a href=&quot;http://www.poissonboltzmann.org/apbs/examples/visualization/apbs-electrostatics-in-vmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/n4J1c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Longer answer&lt;br&gt;&#xA;First, you need to get your molecules into &lt;a href=&quot;http://www.poissonboltzmann.org/file-formats/biomolecular-structurw/pqr&quot; rel=&quot;nofollow noreferrer&quot;&gt;PQR format&lt;/a&gt;. PQR is fairly straightforward and is whitespace delimited:   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Field_name Atom_number Atom_name Residue_name Chain_ID Residue_number X Y Z Charge Radius&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;PQR is one line per atom. Field_name should always be &lt;code&gt;ATOM&lt;/code&gt;. The next five fields all relate back to the residue naming scheme used by the older PDB format. If the only thing that you want to be able to do is to visualize the field, you can just use &lt;code&gt;0001 H DUM A 001&lt;/code&gt; for those five fields for every atom, as long as you increment the Atom_number field by 1 on each line (&lt;code&gt;0001, 0002, etc.)&lt;/code&gt;. You'll want to pay special attention to the X Y Z fields, which are (obviously) the x y z coordinates of the atom, and to the Charge field, which (also obviously) is the partial charge of the atom. Since you have no implicit solvent, the Radius field is irrelevant and should probably be set to &lt;code&gt;0&lt;/code&gt; for all atoms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you already have partial charges (I'm going to assume you already have coordinates), it should be straightforward to write a script that will create your PQR file. If you don't already have partial charges, or if your molecule happens to be in certain accepted formats (MOL2, PDB, or XML), you should probably figure out how to use the &lt;a href=&quot;http://www.poissonboltzmann.org/pdb2pqr&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB2PQR tool&lt;/a&gt; in order to make your PQR file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Install &lt;a href=&quot;http://www.poissonboltzmann.org/apbs/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;APBS&lt;/a&gt; and &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VMD&lt;/a&gt;, and then just follow the instructions &lt;a href=&quot;http://www.poissonboltzmann.org/apbs/examples/visualization/apbs-electrostatics-in-vmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. One thing to note: since you have no implicit solvent, in step 3 of &quot;Running the electrostatics calculation&quot; you will need to set your solvent and solute dielectric constant equal to whatever dielectric constant you used in your MD simulation (if you don't know what it was, setting them both to 1 should be fine).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That should be it. If you have any questions, or if it doesn't work, please add some comments and I'll see what I can do to answer/fix it.&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="34" LastEditDate="2012-04-21T19:24:50.030" LastActivityDate="2012-04-21T19:24:50.030" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1950" PostTypeId="1" AcceptedAnswerId="1961" CreationDate="2012-04-17T17:12:30.850" Score="7" ViewCount="336" Body="&lt;p&gt;Given large $x \in \mathbb{R}$, I want to know whether or not $2^x$ is an integer. Is there any fast way to answer the question for $x&amp;gt;2^{500}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also asked a slightly different form of this question on math.se: &lt;a href=&quot;https://math.stackexchange.com/questions/133390/how-many-significant-figures-are-needed-in-base-2&quot;&gt;How many significant figures are needed in base 2?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1338" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-04-21T19:05:37.543" Title="Given large $x \in \mathbb{R}$, How to determine if $2^x$ is an integer?" Tags="|algorithms|" AnswerCount="3" CommentCount="22" ContentLicense="CC BY-SA 3.0" />
  <row Id="1951" PostTypeId="2" ParentId="1950" CreationDate="2012-04-17T18:05:47.573" Score="-1" Body="&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;br&gt;&#xA;You can determine if $2^x$ is integer by checking how close $x$ is to the log of an integer. If $|x-log_2(N)|&amp;lt;\epsilon$ for some small $\epsilon$ for some integer N, then we can say that $2^x$ is approximately an integer.  You can use a taylor expansion of $log_2(x)$ to evaluate it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tricky part is knowing which integer N has a log closest to x.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-04-17T19:24:53.090" LastActivityDate="2012-04-17T19:24:53.090" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1952" PostTypeId="1" CreationDate="2012-04-17T22:33:47.133" Score="4" ViewCount="74" Body="&lt;p&gt;There is a model embedded in a web browser &lt;a href=&quot;http://web.archive.org/web/20011118202519/http://www.msstate.edu/Entomology/PGjava/ILSImodel.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Caprio 1998)&lt;/a&gt; that I would like to use in an MCMC algorithm. What is the best way to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could implement the model in my favorite language but I would like to know if it is possible to either write a web-scraping / mechanize script to evaluates the model and dumps the output into a file, or somehow capture the existing model implementation as a desktop executable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interface is fairly straightforward:&lt;img src=&quot;https://i.stack.imgur.com/FqxRT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I envision a script that takes the parameters in the fields on the left side as inputs, and returns the results provided in the right column of fields as outputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; ./runmodel.sh 0.001 0.002 1 0.04 0.001 1.0 1.0 0.5 0 &amp;gt; model.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can I efficiently do this using the current implementation (in Java 1.0), or should I just start from scratch?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Caprio, MA (1998) The evolution of resistance: a simple deterministic model. &lt;a href=&quot;http://web.archive.org/web/20011118202519/http://www.msstate.edu/Entomology/PGjava/ILSImodel.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://web.archive.org/web/20011118202519/http://www.msstate.edu/Entomology/PGjava/ILSImodel.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="452" LastActivityDate="2012-04-19T06:29:00.803" Title="How to use a web-embedded model in a computational workflow?" Tags="|algorithms|monte-carlo|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1953" PostTypeId="1" AcceptedAnswerId="2058" CreationDate="2012-04-17T22:53:55.733" Score="9" ViewCount="147" Body="&lt;p&gt;Let $V$ be a finite-dimensional vector space with norm $\|\cdot\|$ and let $F : V \rightarrow \mathbb R$ be a bounded linear functional. It is only given as black-box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to estimate the norm of $F$ (from above and below). As $F$ is a black-box, the only way to do so is to test it with unit vectors from $V$ and, based on the result, find $v \in S^1 V$ that maximizes $|F(v)|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know such an algorithm? In the application that I have in mind, $V$ is a finite-element space and $F$ is a complicated functional on that space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;My first idea is to choose $v \in S^1 V$ randomly, perturb it into several directions, say, $v_1,\dots,v_k$, and then repeat the procedure with the $v_i$ that got the biggest $F(v_i)$. I do not know where to find algorithms and analysis for this problem.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="238" LastEditDate="2012-04-18T08:07:12.887" LastActivityDate="2012-04-26T05:44:30.390" Title="Estimate Norm of a black-box functional" Tags="|linear-algebra|algorithms|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1954" PostTypeId="1" CreationDate="2012-04-18T01:47:05.427" Score="8" ViewCount="1566" Body="&lt;p&gt;I'm trying to do MD on water vapor. As I know there exists some water models for liquid water, such as SPC,SPC/E,TIP3P, but will they also apply to vapor state of water? And what's the difference of simulations of liquid and vapor water? &lt;/p&gt;&#xA;" OwnerUserId="279" LastActivityDate="2022-02-04T15:26:44.043" Title="Molecular dynamics simulation of water vapor?" Tags="|molecular-dynamics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1955" PostTypeId="2" ParentId="1954" CreationDate="2012-04-18T06:57:23.490" Score="7" Body="&lt;p&gt;It is certainly true that most of the standard water models are &quot;tuned&quot; according to properties of the liquid phase—they are intended, after all, to be used as models for &lt;em&gt;bulk&lt;/em&gt; water, rather than modeling interfacial properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, you &lt;em&gt;can&lt;/em&gt; use these liquid-phase models to simulate vapor-liquid equilibrium, so long as you're willing to live with the substantial inadequacies that are associated with them. A few years back, I did a study of &lt;a href=&quot;http://link.aip.org/link/?jcp/125/014702&quot;&gt;surface tension of common water models&lt;/a&gt; &lt;a href=&quot;http://arxiv.org/abs/cond-mat/0605104&quot;&gt;(arXiv link)&lt;/a&gt;, and found that none of them gave surface tensions higher than 62 mN/m (the experimental value is closer to 72 mN/m). Even the best water model currently out there, the TIP4P/2005 model, only gives values of approximately 65 mN/m (according to Gromacs, NAMD, and LAMMPS simulations). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principal thing to remember about the vapor phases is that at low temperatures, there are very few water molecules in the vapor phase, since the density is so low. That means you'll need to have either a large system or a long sampling time to get enough statistics for accurate calculations.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-04-18T06:57:23.490" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1956" PostTypeId="2" ParentId="1954" CreationDate="2012-04-18T08:06:39.143" Score="3" Body="&lt;p&gt;Generally, you should choose the forcefield that most accurately reproduces the quantity of greatest interest to you (dielectric constant, dipole moment, surface tension, etc.). &lt;a href=&quot;http://www.lsbu.ac.uk/water/models.html&quot; rel=&quot;nofollow&quot;&gt;This&lt;/a&gt; is an excellent guide to all of the different water models and their respective strengths and weaknesses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several dozen different water models out there, and they all suck in their own special way, so eventually you just have to close your eyes and pick one. My experience is that most non-experts just end up using the defacto standard, i.e. TIP3P/TIP4P, which is probably just fine for most applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the difference in how accurate forcefields are in gas vs liquid phases, one of the weaknesses of all MM forcefields is that they are very context sensitive. A forcefield which is accurate for gas phase will always be less accurate for liquid phase, and vice versa. Some forcefield features, such as polarization, allow for some degree of dynamic response to changes in molecular context, but even so there will never be an MM forcefield that has anywhere near chemical accuracy across two phases. I think that most simulators learn to live with a certain level of these kinds of inaccuracies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do want a very accurate multi-phase simulation, you'll really have to start looking into the &lt;a href=&quot;http://chem.hamilton.edu/~gshields/publications/McGrathSKMVHMK2006.pdf&quot; rel=&quot;nofollow&quot;&gt;quantum&lt;/a&gt; methods.&lt;/p&gt;&#xA;" OwnerUserId="34" LastActivityDate="2012-04-18T08:06:39.143" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1957" PostTypeId="1" CreationDate="2012-04-18T14:38:49.043" Score="3" ViewCount="184" Body="&lt;p&gt;Background: I'm trying to make a system that tracks a number of bubbles in a video&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm implementing the bubble detection in the single image case using the Circular Hough Transform.  Due to occlusion, blur and other factors, this detection will never be 100% accurate.  I am tuning the detection procedure for high recall, possibly at the expense of precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once this is done and applied to a sequence of frames from a video, I will have a number of detections that can be characterised as points in 4D space - x-position, y-position, radius and frame index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a procedure that can recover curves from this 4D point cloud?&lt;/p&gt;&#xA;" OwnerUserId="1339" LastEditorUserId="20688" LastEditDate="2019-06-22T19:21:49.783" LastActivityDate="2019-06-22T19:21:49.783" Title="Recover curves from noisy collection of points" Tags="|regression|computer-vision|pattern-recognition|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1958" PostTypeId="2" ParentId="1953" CreationDate="2012-04-18T17:06:51.627" Score="0" Body="&lt;p&gt;Maybe you can modify Hager's condition number estimator (see, e.g., the paper&#xA;&lt;a href=&quot;http://eprints.ma.man.ac.uk/321/01/35608.pdf&quot; rel=&quot;nofollow&quot;&gt;http://eprints.ma.man.ac.uk/321/01/35608.pdf&lt;/a&gt;), which bounds $\|A^{-1}\|$ when a factorization of $A$ is known, to work for your particular case.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-18T17:06:51.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1959" PostTypeId="2" ParentId="1957" CreationDate="2012-04-18T17:18:44.247" Score="1" Body="&lt;p&gt;As long as the curve is not too curly, one way of doing it is to first use a SVD of the points (centered to mean zero) to get the dominant direction in space. If your centered points are the rows of a matrix $X=USV^T$, and $u$ is the first column of $U$, you can do an error in variables fit to a stochastic model of the form $X_{k:}=p(u_k+\eta)+\epsilon$ using a univariate low degree polynomial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the more curly case, solve instead the problem $P(X_{k:},u_k)\approx 0$ with a bivariate polynomial ansatz for $P$, using the smallest singular vector of the matrix of the resulting linear problem $A z=b$ for the coefficients $z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the curve is curled up in a complex way, you'd have to disentangle it first by using a heuristics to extract from the cloud shorter and more well-behaved pieces of the curve. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-18T17:18:44.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1960" PostTypeId="1" AcceptedAnswerId="2190" CreationDate="2012-04-18T19:36:40.357" Score="22" ViewCount="5451" Body="&lt;p&gt;What would be a good finite difference discretization for the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \rho}{\partial t} + \nabla \cdot \left(\rho u\right)=0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can take the 1D case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \rho}{\partial t} + \frac{d}{dx}\left(\rho u\right)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some reason all schemes I can find is for the formulation in Lagrangian coordinates.&#xA;I came up with this scheme for the time being (disregard the &lt;em&gt;j&lt;/em&gt; index):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\rho^{n+1}_{i,j}-\rho^{n}_{i,j}}{\tau} + \frac{1}{h_x}\left(\frac{\rho^{n+1}_{i+1,j}+\rho^{n+1}_{i,j}}{2}u^{n}_{_xi+1/2,j}- \frac{\rho^{n+1}_{i,j}+\rho^{n+1}_{i-1,j}}{2}u^{n}_{_xi-1/2}\right)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But seems to be really unstable or have some horrible stability condition. Is that so?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The velocity is actually calculated through the darcy law $u=-\frac{k}{\mu}\nabla p$. Plus we have the equation of state. The full system consists also of an energy equation and the equation of state for the ideal gas. &lt;strong&gt;The velocities can turn negative&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1088" LastEditorUserId="1088" LastEditDate="2012-05-11T13:52:31.250" LastActivityDate="2013-03-14T16:33:08.760" Title="A good finite difference for the continuity equation" Tags="|finite-difference|fluid-dynamics|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1961" PostTypeId="2" ParentId="1950" CreationDate="2012-04-18T21:45:59.160" Score="4" Body="&lt;p&gt;(This is a partial response to OP's question that covers the case where $x$ is rational and is not being used to approximate an irrational number)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $x\in\mathbb{Q}$, testing whether $2^x$ is an integer is equivalent to testing whether $x$ is a non-negative integer.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Proof&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;proposition 1&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Given $Log_2(y)=x$ where $x\in\mathbb{Q}$, if $y$ is a positive integer there must be some $c$ such that $2^m=y^n=c$ and $x=\frac{m}{n}$ where $m,n\in\mathbb{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(see comments of &lt;a href=&quot;https://math.stackexchange.com/a/133210/13983&quot;&gt;this answer&lt;/a&gt; for the proof)&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;proposition 2&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If $y&amp;gt;2$, and $m,n, y\in\mathbb{N}$, if $2^m=y^n$ then $m$ must equal $ni$ where $i$ is some integer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the set of prime factors of $2^m$. In order for $2^m$ to have an integer $n^{th}$ root, the set of prime factors must be able to be split into $n$ identical subsets. Since the prime factors are a set of $m$ twos, this condition can only be fulfilled if $m=ni$ where $i\in\mathbb{N}$.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;putting it together&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Taken together, propositions 1 and 2 imply that if $2^x\in\mathbb{N}$ then $x=\frac{m}{n}=\frac{ni}{n}=\{i|i\in\mathbb{N}\}$. It is taken as obvious that if $x\in\mathbb{N}$ then  $2^x\in\mathbb{N}$. Therefore, $2^x\in\mathbb{N}\Longleftrightarrow x\in\mathbb{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory this would mean that all you need to do is test whether the fractional part of $x=0$. Of course, OP's application skirts this proof by virtue of the fact that his test is not that $2^x\in\mathbb{N}$ but rather that $2^x$ should be arbitrarily close to some positive integer.&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-04-21T19:05:37.543" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1962" PostTypeId="1" CreationDate="2012-04-19T02:10:18.437" Score="3" ViewCount="444" Body="&lt;p&gt;I am trying to generate a 2 tuple using maple. Can anyone give me the command to generate this? Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="1341" LastActivityDate="2012-04-19T03:14:05.470" Title="Generating a tuple in Maple" Tags="|algorithms|software|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1963" PostTypeId="2" ParentId="1962" CreationDate="2012-04-19T03:14:05.470" Score="2" Body="&lt;p&gt;What do you mean by a 2-tuple? If the two values you want to put into it are called &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt;, then you can create a&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;sequence&lt;/em&gt; as &lt;code&gt;x, y&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;list&lt;/em&gt; as &lt;code&gt;[x, y]&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;set&lt;/em&gt; as &lt;code&gt;{x, y}&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Vector&lt;/em&gt; as &lt;code&gt;&amp;lt;x, y&amp;gt;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Array&lt;/em&gt; as &lt;code&gt;Array([x, y])&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;table&lt;/em&gt; as e.g. &lt;code&gt;table([1 = x, 2 = y])&lt;/code&gt; (or use something else as keys - or use &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; as keys and something else as values)&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;record&lt;/em&gt; as e.g. &lt;code&gt;Record(x, y)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;mutable set&lt;/em&gt; as &lt;code&gt;MutableSet(x, y)&lt;/code&gt; (I don't believe this is documented, so you're on your own for forward compatibility)&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;deque&lt;/em&gt; as &lt;code&gt;DEQueue(x, y)&lt;/code&gt; (same remarks as for the mutable set)&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;unevaluated function call&lt;/em&gt; as e.g. &lt;code&gt;f(x, y)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Each of these can be used as a 2-tuple (with some fantasy). What do you want to do with the tuple?&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2012-04-19T03:14:05.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1964" PostTypeId="2" ParentId="1952" CreationDate="2012-04-19T06:29:00.803" Score="3" Body="&lt;p&gt;Extracting data from a Java applet on the web seems complicated (see &lt;a href=&quot;https://stackoverflow.com/questions/6948814/how-can-i-extract-data-from-a-java-applet-inside-the-browser&quot;&gt;this answer on SO&lt;/a&gt; and &lt;a href=&quot;http://objectmix.com/jdbc-java/41680-extract-data-java-applet.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this forum post&lt;/a&gt;), but if, for example, the data was available as ordinary web forms or, in general, in a more accesible way, there are some solutions in Python for web scraping:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://scrapy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scrapy&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.python.org/library/urllib2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;urllib&lt;/a&gt; to grab de data + &lt;a href=&quot;http://lxml.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;lxml&lt;/a&gt; / &lt;a href=&quot;http://www.crummy.com/software/BeautifulSoup/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BeautifulSoup&lt;/a&gt; to parse it  (the latter option depending on its well-formedness)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="782" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-04-19T06:29:00.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1965" PostTypeId="1" AcceptedAnswerId="1978" CreationDate="2012-04-19T08:27:11.183" Score="5" ViewCount="1257" Body="&lt;p&gt;Below there are two algorithms for solving tridiagonal linear systems of the form&#xA;$$&#xD;&#xA;\left[&#xD;&#xA;\begin{array}{ccccc|c}&#xD;&#xA;b_1 &amp;amp; c_1 &amp;amp; &amp;amp; &amp;amp; &amp;amp;d_1\\&#xD;&#xA;a_2 &amp;amp; b_2 &amp;amp; c_2 &amp;amp; &amp;amp; &amp;amp; d_2\\ &#xD;&#xA; &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; &amp;amp; \vdots\\&#xD;&#xA;&amp;amp; &amp;amp;  a_{n-1} &amp;amp; b_{n-1} &amp;amp; c_{n-1} &amp;amp; d_{n-1}\\&#xD;&#xA;&amp;amp; &amp;amp; &amp;amp; a_n &amp;amp; b_n &amp;amp; d_n&#xD;&#xA;\end{array}&#xD;&#xA;\right].&#xD;&#xA;$$&#xA;I called them Algorithms A and B.&#xA;Both of them are equivalent to Gaussian elimination, but with important difference in the form of the resulting triangular &#xA;(bidiagonal) matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is: &lt;strong&gt;which one of them is more preferrable?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithm A is the one that described in &lt;a href=&quot;http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt; and many textbooks, it is called Thomas algorithm and is implemented, for example, in Numerical Recipes in some tricky form. Algorithm B is more &#xA;straightforward and, in my opinion, is more numerically stable in cases when $|b_i|\gg|a_i|+|c_i|$ . &#xA;Though I haven't seen Algorithm B in texbooks, note that exactly this &#xA;algorithm is implemented in the mentioned Wikipedia article, see &quot;Implementation in Fortran 90&quot;, while &quot;Implementation in &#xA;Matlab&quot; deals with Algorithm A (&quot;Implementation in C&quot; in its current state is a mess that does not seem to work at all). &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\begin{array}{|c|c|}\hline&#xD;&#xA;\mathbf{Algorithm\ A} &amp;amp; \mathbf{Algorithm\ B}\\\hline&#xD;&#xA;\textit{% Elimination}&amp;amp;\textit{% Elimination}\\&#xD;&#xA;\begin{array}{l}&#xD;&#xA;\tilde c_1=c_1/b_1\\&#xD;&#xA;\tilde  d_1=d_1/b_1\\&#xD;&#xA;\mathbf{for }\quad i=2 \quad \mathbf{to}\quad n-1 \quad  \textbf{do}\\&#xD;&#xA;\quad q=b_i-a_i c_{i-1}\\&#xD;&#xA;\quad \tilde c_i=c_i/q\\&#xD;&#xA;\quad \tilde d_i=(d_i-a_i \tilde d_{i-1})/q\\&#xD;&#xA;\mathbf{end do}\\&#xD;&#xA;\tilde d_n=(d_n-a_n \tilde d_{n-1})/(b_{n}-a_n \tilde c_{n-1})\\&#xD;&#xA;\\&#xD;&#xA;\end{array}&#xD;&#xA;&amp;amp; &#xD;&#xA;\begin{array}{l}&#xD;&#xA;\\&#xD;&#xA;\\&#xD;&#xA;\hat b_1=b_1\\&#xD;&#xA;\mathbf{for }\quad i=2 \quad \mathbf{to}\quad n \quad  \textbf{do}\\&#xD;&#xA;\quad q=a_i/\hat b_{i-1}\\&#xD;&#xA;\quad \hat b_i=b_i-q c_{i-1}\\&#xD;&#xA;\quad \hat d_i=d_i-q \hat d_{i-1}\\&#xD;&#xA;\mathbf{end do}\\&#xD;&#xA;\\&#xD;&#xA;\\&#xD;&#xA;\end{array}\\ \hline&#xD;&#xA;\textit{% Resulting system} &amp;amp; \textit{% Resulting system}\\&#xD;&#xA;\left[&#xD;&#xA;\begin{array}{ccccc|c}&#xD;&#xA;1 &amp;amp; \tilde c_1 &amp;amp; &amp;amp; &amp;amp; &amp;amp;\tilde d_1\\&#xD;&#xA; &amp;amp; 1 &amp;amp; \tilde c_2 &amp;amp; &amp;amp; &amp;amp; \tilde d_2\\ &#xD;&#xA; &amp;amp;  &amp;amp; \ddots &amp;amp; \ddots &amp;amp; &amp;amp; \vdots\\&#xD;&#xA;&amp;amp; &amp;amp;   &amp;amp; 1 &amp;amp; \tilde c_{n-1} &amp;amp; \tilde d_{n-1}\\&#xD;&#xA;&amp;amp; &amp;amp; &amp;amp;  &amp;amp; 1 &amp;amp; \tilde d_n&#xD;&#xA;\end{array}&#xD;&#xA;\right]&#xD;&#xA;&amp;amp;&#xD;&#xA;\left[&#xD;&#xA;\begin{array}{ccccc|c}&#xD;&#xA;\hat b_1 &amp;amp; c_1 &amp;amp; &amp;amp; &amp;amp; &amp;amp;\hat d_1\\&#xD;&#xA; &amp;amp; \hat b_2 &amp;amp; c_2 &amp;amp; &amp;amp; &amp;amp; \hat d_2\\ &#xD;&#xA; &amp;amp; &amp;amp; \ddots &amp;amp; \ddots &amp;amp; &amp;amp; \vdots\\&#xD;&#xA;&amp;amp; &amp;amp;  &amp;amp; \hat b_{n-1} &amp;amp; c_{n-1} &amp;amp; \hat d_{n-1}\\&#xD;&#xA;&amp;amp; &amp;amp; &amp;amp; &amp;amp; \hat b_n &amp;amp; \hat d_n&#xD;&#xA;\end{array}&#xD;&#xA;\right]\\&#xD;&#xA;\hline&#xD;&#xA;\textit{% Backsubtitution} &amp;amp; \textit{% Backsubtitution}\\&#xD;&#xA;\begin{array}{l}&#xD;&#xA;\\&#xD;&#xA;x_n=\tilde d_n\\&#xD;&#xA;\mathbf{for }\quad i=n-1 \quad \mathbf{downto}\quad 1 \quad  \textbf{do}\\&#xD;&#xA;\quad x_i=\tilde d_i-\tilde c_i x_{i+1}\\&#xD;&#xA;\\&#xD;&#xA;\end{array}&#xD;&#xA;&amp;amp;&#xD;&#xA;\begin{array}{l}&#xD;&#xA;\\&#xD;&#xA;x_n=\hat d_n/\hat b_n\\&#xD;&#xA;\mathbf{for }\quad i=n-1 \quad \mathbf{downto}\quad 1 \quad  \textbf{do}\\&#xD;&#xA;\quad x_i=(\hat d_i-c_i x_{i+1})/\hat b_i\\&#xD;&#xA;\\&#xD;&#xA;\end{array}\\\hline&#xD;&#xA;\end{array}&#xD;&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="9" LastEditDate="2012-04-20T13:58:44.590" LastActivityDate="2012-04-20T13:58:44.590" Title="Comparing algorithms for tridiagonal linear systems solution" Tags="|linear-algebra|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1966" PostTypeId="1" AcceptedAnswerId="1974" CreationDate="2012-04-19T12:51:21.293" Score="11" ViewCount="1961" Body="&lt;p&gt;(I hope this question fits this site; if not, accept my apologies).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran a certain simulation, and got a time series y(t), t = 0, 1, ... 20. After trying some functions, I found that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y(t) =~ 1 / (A t + B)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where A and B are coefficients I calculated using linear regression, with R^2 &gt; 0.99.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the standard way to report such results in a scientific paper? Specifically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. I have no theoretic explanation, why the output looks like this (I know it should be decreasing, and that it's bounded from below, but not much more). It was just a successful guess. Should I describe all other unsuccessful guesses that I tried?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B. Whenever I run the simulation, I get slightly different values of A and B. Should I just report a random run, or should I run the simulation many times and average the results? If so, how many times is enough?&lt;/p&gt;&#xA;" OwnerUserId="1330" LastActivityDate="2014-02-24T07:54:58.590" Title="Reporting curve-fit results in a scientific paper" Tags="|regression|publications|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1967" PostTypeId="2" ParentId="616" CreationDate="2012-04-19T15:09:21.240" Score="3" Body="&lt;p&gt;I'd calculate the mean $\overline x$ and the covariance matrix $C$ of the joint data set, and then do a K/S test on the univariate quantity $V(x):=(x-\overline x)^TC^{-1}(x-\overline x)$ evaluated on the parts. If the K/S test give a significant difference between the parts, there is one. If it gives no significant difference, the test is to be regarded as unconclusive.  &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-19T15:09:21.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1968" PostTypeId="2" ParentId="952" CreationDate="2012-04-19T16:32:26.537" Score="-3" Body="&lt;p&gt;I do not understand the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why the minus?&lt;/li&gt;&#xA;&lt;li&gt;These four conditions holds for $r=0$ AND $r=1$ AND $\varphi=0$ AND $\varphi=\pi$? Why?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Could you give more details, please?&lt;/p&gt;&#xA;" OwnerUserId="1345" LastEditorUserId="764" LastEditDate="2012-07-02T09:25:29.093" LastActivityDate="2012-07-02T09:25:29.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1969" PostTypeId="1" AcceptedAnswerId="1971" CreationDate="2012-04-19T16:45:36.147" Score="6" ViewCount="5060" Body="&lt;p&gt;While trying to find cell-centered gradients in finite volume method computation of incompressible fluid flow I get over-determined linear system. This is a well known &quot;cell based least-square&quot; gradient reconstruction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know in theory that by using least squares we search for a solution which minimizes error vector in 2-norm, and that we can solve the problem by calculating 'pseudo-inverse'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x=(A^TA)^{-1}A^Tb$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I read that it is not typically how the problem is solved.&#xA;Rather $QR$ decomposition is performed on original system matrix, and eventually we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Rx=Q^Tb$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm looking for is a fast algorithm for $QR$ decomposition of rectangular matrices that I can implement in Fortran, or any other suggestion related to particular problem that I mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;System matrix is usually 6x3. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-04-19T21:00:23.387" Title="How to find QR decomposition of a rectangular matrix in overdetermined linear system solution?" Tags="|linear-algebra|algorithms|linear-solver|fluid-dynamics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1970" PostTypeId="2" ParentId="1969" CreationDate="2012-04-19T16:58:33.300" Score="7" Body="&lt;p&gt;If you're using Fortran and don't know much about linear algebra, I'd suggest you use &lt;a href=&quot;http://en.wikipedia.org/wiki/LAPACK&quot;&gt;LAPACK&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relevant routine in there for solving an over-determined system of equations using a QR-decomposition is &lt;a href=&quot;http://www.netlib.org/lapack/double/dgels.f&quot;&gt;DGELS&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your matrix is quite small, you may get significantly more preformance with a custom QR-decomposition, but LAPACK is probably the more robust solution, and definitely the fastest in terms of development time.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-04-19T16:58:33.300" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1971" PostTypeId="2" ParentId="1969" CreationDate="2012-04-19T18:14:49.240" Score="6" Body="&lt;p&gt;If you are looking for an efficient and numerically stable algorithm to solve the least squares problem you can use Householder reflectors. Golub and van Loan &quot;Matrix Computations'' gives a very nice treatment on Householder reflectors (in fact one of the dedications in the book is for Alston Scott Householder). We'll outline the basic idea behind the linear least squares problem and how we can use Householder reflectors to solve it. Suppose we have an overdetermined matrix $A\in\mathbb{R}^{n\times k}$ with $n\geq k$ and full rank&#xA;$$&#xD;&#xA;A=\left(\!\!{\begin{array}{cccc}&#xD;&#xA;	* &amp;amp; * &amp;amp; * &amp;amp; * \\&#xD;&#xA;	* &amp;amp; * &amp;amp; * &amp;amp; * \\&#xD;&#xA;	* &amp;amp; * &amp;amp; * &amp;amp; * \\&#xD;&#xA;	* &amp;amp; * &amp;amp; * &amp;amp; * \\&#xD;&#xA;	* &amp;amp; * &amp;amp; * &amp;amp; * \\&#xD;&#xA;	* &amp;amp; * &amp;amp; * &amp;amp; * \\	&#xD;&#xA;\end{array}}\!\!\right).&#xD;&#xA;$$&#xA; Now let us take the vector $v$ to be the $i$th column of the matrix $A$ running from the diagonal elements $i$ to $n$. Then we calculate the $i$th Householder reflector $H_i$ to zero out all the entries below $A_{ii}$. We define $\gamma = sign(e_1^Tv)\|v\|_2$, where $e_1^T = [1\;0\;\cdots\;0]$,&#xA; $$&#xD;&#xA;x = v+\gamma e_1\qquad and \qquad \alpha = -\frac{2}{\|x\|_2^2}.&#xD;&#xA;$$&#xA;The $sign$ function on $\gamma$ is chosen for numerical stability. Then the reflector $H_i$ is given by &#xA;$$&#xD;&#xA;H_i = I+\alpha xx^T.&#xD;&#xA;$$&#xA;Note that the reflector $H_i$ will be an orthogonal matrix which will zero every element in the vector $v$ except the first value, i.e., &#xA;$$&#xD;&#xA;H_iv = (I+\alpha xx^T)v = -\gamma e_1.&#xD;&#xA;$$&#xA;This can be done in a computationally efficient way because of the structure of the reflector. We do not need to form the orthogonal matrix for each $H_i$ instead we can compute its action as&#xA;$$&#xD;&#xA;H_iv = (I+\alpha xx^T)v = v+\alpha x(x^Tv) = -\gamma e_1,&#xD;&#xA;$$&#xA;which is a single dot product and then a scaled vector addition. So we have turned an $\mathcal{O}(n^3)$ operation into an $\mathcal{O}(n)$ operation. We can then create a sequence of $k$ reflectors to drive the rectangular matrix $A$ to an upper trapezoidal form. For example,&#xA;$$&#xD;&#xA;H_4H_3H_2H_1A = \left(\!\!{\begin{array}{cccc}&#xD;&#xA;	* &amp;amp; * &amp;amp; * &amp;amp; * \\&#xD;&#xA;	0 &amp;amp; * &amp;amp; * &amp;amp; * \\&#xD;&#xA;	0 &amp;amp; 0 &amp;amp; * &amp;amp; * \\&#xD;&#xA;	0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * \\&#xD;&#xA;	0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * \\&#xD;&#xA;	0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\	&#xD;&#xA;\end{array}}\!\!\right) = \left(\!{\begin{array}{c} R \\ 0 \end{array}}\!\right).&#xD;&#xA;$$ &#xA;The product of these $k$ Householder reflectors is an $n\times n$ orthogonal matrix which we denote $H=H_kH_{k-1}\cdots H_1$. We wish to find the minimizing solution $x_{min}$ to the overdetermined system $Ax=b$. We transform the system using the matrix $H$ into the system $HAx = Hb$ or &#xA;$$&#xD;&#xA;\left(\!{\begin{array}{c} R \\ 0 \end{array}}\!\right)x = \left(\!{\begin{array}{c} c \\ d \end{array}}\!\right).&#xD;&#xA;$$&#xA;Then the minimal solution with respect to the $2$-norm is $x_{min} = R^{-1}c$ which can be solved efficiently using a back substitution method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more details on the geometry of the least squares problem (given in Golub and van Loan) and how the Householder reflectors project onto two spaces, the range of $A$ and the perp space of $A$. One can prove interesting results on the residual of linear least squares problems, it is related to the value $d$ above. But the original post seemed more an implementation question versus a theoretical linear algebra question so we'll omit such discussion.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastActivityDate="2012-04-19T18:14:49.240" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1972" PostTypeId="2" ParentId="1969" CreationDate="2012-04-19T21:00:23.387" Score="1" Body="&lt;p&gt;Modified Gram-Schmidt is fastest for your problem in terms of implementation time and performance. To avoid loss of orthogonality you must append the matrix with the right hand side. See Golub, van Loan for details.&lt;/p&gt;&#xA;" OwnerUserId="897" LastActivityDate="2012-04-19T21:00:23.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1973" PostTypeId="2" ParentId="1950" CreationDate="2012-04-20T00:16:43.860" Score="-1" Body="&lt;p&gt;You just need to check that $x \in \mathbb Z_{\ge 0}$ given $x \in \mathbb{Q}$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the following, let &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\mathbb{I} = \mathbb{R}\setminus\mathbb{Q}$&lt;/li&gt;&#xA;&lt;li&gt;$E(n) \iff n$ is even&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Lemma 1: $n \in \mathbb{Z_{\ge 2}} \implies 2^{1/n} \in \mathbb{I}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case $n = 2$: $\sqrt{2} \in \mathbb{I}$ is a well-known fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case $n \gt 2$: Proof is similar to the $\sqrt{2} \in \mathbb{I}$ found in Euclid's Elements. Suppose for a contradiction that $2^{1/n} \in \mathbb{Q}$. Then by definition, $2^{1/n} = a/b$, where $a$ and $b$ are coprime. Then $2 = (a/b)^n = a^n/b^n \implies 2b^n = a^n \implies E(a^n) \implies E(a) \implies a = 2k, k \in \mathbb{Z} \implies 2b^n = (2k)^n = 2^nk^n \implies b^n = 2^{n-1}k^n \implies E(b^n) \because 2 \mid 2^{n-1} \because n \ge 2 \implies E(b) \implies b = 2l, l \in \mathbb{Z}$. But since $2 \mid a$ and $2 \mid b$, $a$ and $b$ are not coprime $\implies$ contradiction.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Lemma 2: If $x \in \mathbb{Q} \cap (0,1)$ then $2^x \in \mathbb{I}.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof: Note that $2^0=1$ and $2^1 = 2$. Since $f(x) = 2^x$ is strictly monotonic, it follows that $0&amp;lt;x&amp;lt;1 \implies f(0) &amp;lt; f(x) &amp;lt; f(1)$. But this is the same as saying $1 &amp;lt; f(x) &amp;lt; 2$. Thus $f(x) \notin \mathbb{Z}$ because there are no integers between $1$ and $2$. Since $x \in \mathbb{Q}, x = a/b$, where $a \in \mathbb{z}$ and $b \in \mathbb{Z}$. But $b &amp;gt; 1 \because f(x) \notin \mathbb{Z}$. Thus $2^x = 2^{a/b} = 2^{a(1/b)} = 2^{(1/b)a} = (2^{1/b})^a = w^a, w \in \mathbb{I}$ by lemma 1. Since $a &amp;lt; b \because x \in (0,1), w^a \in \mathbb{I} \because $ [fuck it, i'm stuck w/e'vs].&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Theorem: If $x \in \mathbb{Q}$ then $2^x \in \mathbb{Z} \iff x \in \mathbb{Z_{\ge 0}}.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof: Let $x \in \mathbb{Q}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case $x &amp;lt; 0$: $2^x \notin \mathbb{Z}$ because $0 &amp;lt; 2^x &amp;lt; 1$ by exponential properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case $x \ge 0$: By rules of exponentiation, $2^x = 2^{n + \epsilon} = 2^n2^\epsilon, n \in \mathbb{Z_{\ge 0}}$ and $ \epsilon \in [0, 1).$ By lemma 2 and $2^0 = 1$, $2^\epsilon \in \mathbb{Z} \iff \epsilon = 0.$ Hence $2^n2^\epsilon \in \mathbb{Z} \iff \epsilon = 0$ because $2^n \in \mathbb{Z}$ and $2^0 = 1$. Since $x = n+\epsilon$ by definition, it follows that $2^x \in \mathbb{Z} \iff x \in \mathbb{Z_{\ge 0}}.$&lt;/p&gt;&#xA;" OwnerUserId="1340" LastEditorUserId="1340" LastEditDate="2012-04-21T00:57:02.383" LastActivityDate="2012-04-21T00:57:02.383" CommentCount="2" CommunityOwnedDate="2012-04-21T00:44:31.167" ContentLicense="CC BY-SA 3.0" />
  <row Id="1974" PostTypeId="2" ParentId="1966" CreationDate="2012-04-20T00:38:15.350" Score="5" Body="&lt;p&gt;You're trying to fit a &lt;a href=&quot;http://en.wikipedia.org/wiki/Power_law&quot; rel=&quot;nofollow&quot;&gt;power law&lt;/a&gt; to your distribution. Very interesting. These show up all the time in &lt;a href=&quot;http://en.wikipedia.org/wiki/Scale-free_network&quot; rel=&quot;nofollow&quot;&gt;graph theory&lt;/a&gt;, &lt;a href=&quot;http://arxiv.org/abs/cs/0103016&quot; rel=&quot;nofollow&quot;&gt;social networks&lt;/a&gt;, and a &lt;a href=&quot;http://arxiv.org/abs/0706.1062&quot; rel=&quot;nofollow&quot;&gt;slew&lt;/a&gt; of other places.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's some tutorials on fitting your data &lt;a href=&quot;http://tuvalu.santafe.edu/~aaronc/powerlaws/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://www.physics.pomona.edu/sixideas/labs/LRM/LR05.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in reference to question A., how does the probability of a person buying land depend on how much land they already have? You may be able to use &lt;a href=&quot;http://en.wikipedia.org/wiki/Barab%C3%A1si%E2%80%93Albert_model&quot; rel=&quot;nofollow&quot;&gt;Barbasi's model&lt;/a&gt; to explain why a power law is a reasonable fit to your data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;update:&lt;/strong&gt; I've used this and it works great: &lt;a href=&quot;https://pypi.python.org/pypi/powerlaw&quot; rel=&quot;nofollow&quot;&gt;https://pypi.python.org/pypi/powerlaw&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="979" LastEditorUserId="979" LastEditDate="2014-02-24T07:54:58.590" LastActivityDate="2014-02-24T07:54:58.590" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1975" PostTypeId="1" AcceptedAnswerId="1980" CreationDate="2012-04-20T00:58:31.593" Score="5" ViewCount="72" Body="&lt;p&gt;I'm just getting into climate research, and am amazed at how much poorly documented legacy code is used. This kind of thing means that models more or less have to be maintained by the people who wrote them originally. If there was a good open standard for generic data exchange between components of a spatio-temporal model like this, most models could be split up into modules that could more easily be switched in and out, and more easily maintained (you wouldn't have to have such a broad grasp of what was going on inside other modules, and could concentrate on your own area of expertise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, are there any such open standards?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data for climate models might include latent and sensible heat, fluid densities and velocities, vegetation and snow cover, snow and ice depth, etc. etc. It'd be a bit pointless to have a standard just for climate modelling, so ideally the standard would be applicable to any modelling of this type.&lt;/p&gt;&#xA;" OwnerUserId="1232" LastActivityDate="2012-04-20T06:28:36.563" Title="Are there any open standards for data exchange between components of a spatial model?" Tags="|data-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1976" PostTypeId="2" ParentId="1966" CreationDate="2012-04-20T01:23:55.477" Score="7" Body="&lt;p&gt;A few thoughts on your question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;How&lt;/em&gt; you report your model fit will depend very much on your audience, and your field. For example, in my field, model fit statistics like R^2 are very rarely reported - regarded as neither impressive nor particularly useful. Instead, some criteria for how you arrived at the model you arrived at tends to be described, and then you report your model results - we all assume you actually fit the model correctly.&lt;/li&gt;&#xA;&lt;li&gt;&quot;I happened across this form&quot; is a bad explanation. A really bad one. Despite a fondness for stories of accidental genius, like the discovery of penicillin or quinine, &quot;blind dumb luck&quot; is not a reliable scientific process. For example, you have shown that that form is good at fitting your data, but you have not yet shown it is &lt;em&gt;best&lt;/em&gt; at fitting your data. R^2 alone is not a sufficient metric for evaluating how well your model fits the data. See &lt;a href=&quot;http://en.wikipedia.org/wiki/Anscombe%27s_quartet&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anscombe's quartet&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;As @rcompton mentioned, it looks like you're trying to fit a power law distribution without knowing it, but even if you do manage to well fit a power law, it's really best if you found some reason why you think its a power law. It may be enough to plot Y over time, head over to &lt;a href=&quot;https://stats.stackexchange.com/&quot;&gt;CrossValidated&lt;/a&gt; (or a college/department more comfortable with statistics) and systematically go over distributions that might give you roughly that look. There are others besides the power law distribution that might give you a superior fit.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="110" LastEditorUserId="-1" LastEditDate="2017-04-13T12:44:17.923" LastActivityDate="2013-02-16T10:48:25.980" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1978" PostTypeId="2" ParentId="1965" CreationDate="2012-04-20T04:32:52.260" Score="3" Body="&lt;p&gt;Both algorithms compute $LU$ decompositions (solving against $L$ while it is being formed) and then solve against the resulting $U$. The difference is that Algorithm A forces $U$ to have a diagonal of all ones (we say that $U$ is &lt;em&gt;unit-diagonal&lt;/em&gt;), while Algorithm B forces $L$ to have a unit diagonal (this is the usual convention).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of whether or not one is more stable than the other, both are a bad idea; as @AronAhmadia mentioned, you should use an algorithm which performs partial pivoting. I would go with the &lt;a href=&quot;http://www.netlib.org/lapack&quot; rel=&quot;nofollow&quot;&gt;LAPACK&lt;/a&gt; routine &lt;a href=&quot;http://www.netlib.org/lapack/double/dgtsv.f&quot; rel=&quot;nofollow&quot;&gt;dgtsv&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-04-20T04:32:52.260" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1979" PostTypeId="2" ParentId="1960" CreationDate="2012-04-20T04:56:25.480" Score="12" Body="&lt;p&gt;In the 1D case, you don't want to use a forward or central difference scheme for the spatial derivative term $(\frac{d}{dx})$ because they are numerically unstable.  Instead, it is better to discretize the equation with an explicit backwards (upwind) finite difference for the spatial derivative:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\rho_i^{k+1}-\rho_i^k}{\Delta t} + \frac{\rho_i^k U_i^k-\rho_{i-1}^k U_{i-1}^k}{\Delta x} = 0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the velocities are positive, then this backward scheme is stable.  If they are negative, then a forward difference will work.  Regardless, there is always a constraint on your choice of $\Delta x$ and $\Delta t$ (courant number) to make the scheme stable.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-04-28T18:49:26.693" LastActivityDate="2012-04-28T18:49:26.693" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1980" PostTypeId="2" ParentId="1975" CreationDate="2012-04-20T06:28:36.563" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.itaps.org/team.html&quot; rel=&quot;nofollow&quot;&gt;Interoperable Technologies for advanced Petascale Scientific Computing, or ITAPS&lt;/a&gt;, has defined several interfaces in an effort to standardize data communication between large-scale software components.  In particular, take a look at Tim Tautges' &lt;a href=&quot;http://trac.mcs.anl.gov/projects/ITAPS/wiki/MOAB&quot; rel=&quot;nofollow&quot;&gt;MOAB&lt;/a&gt; software package, which implements the ITAPS &lt;a href=&quot;http://www.itaps.org/software/iMesh_html/index.html&quot; rel=&quot;nofollow&quot;&gt;iMesh&lt;/a&gt; interface.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-04-20T06:28:36.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1981" PostTypeId="1" CreationDate="2012-04-20T09:19:16.090" Score="4" ViewCount="223" Body="&lt;p&gt;I'm trying to use MATMPIDENSE matrices to solve a system of type Ax=b, but I have some problems. The KSP documentation says that KSPSolve for dense matrices requires to set a 'gmres' solver and a 'lu' preconditioner, the problem is that a 'lu' preconditioner doesn't work in parallel and the corresponding 'asm' doesn't work with dense matrices. How can I solve this &quot;loop&quot; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I need to do some operations with these dense matrices, like call a MatMatMult that seems to not work with MATMPIDENSE. Is it the case ? What can I do ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have even tried to compute a gather of my matrices, to allow these operations, but I have an error message if run with more than 1 processor on the calls SetMatType(A,MATSEQDENSE) and MatGetRow(A,...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance,&#xA;Elisa&lt;/p&gt;&#xA;" OwnerUserId="1351" LastEditorUserId="119" LastEditDate="2012-04-20T14:11:25.383" LastActivityDate="2012-04-20T14:11:25.383" Title="MatMatMult and KSPSolve for MATMPIDENSE matrices" Tags="|parallel-computing|petsc|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1982" PostTypeId="1" CreationDate="2012-04-20T11:24:33.947" Score="3" ViewCount="6589" Body="&lt;p&gt;Which search/optimization problems can be reduced to the famous &quot;Traveling Salesman Problem&quot;? For instance, I have a collection of N particles, in 3D, and there is a function (Van der Waals energy) which depends on their coordinates. I want to find which configuration of the system minimizes the value of the function. Can this problem or type of problems by reduced to TSP? Could you point me to some bibliography?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: How could this problem be reduced to TSP?&lt;/p&gt;&#xA;" OwnerUserId="891" LastEditorUserId="276" LastEditDate="2012-07-05T03:41:31.073" LastActivityDate="2012-07-05T08:53:26.083" Title="Problems that can be reduced to the Traveling Salesman Problem" Tags="|optimization|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="1984" PostTypeId="2" ParentId="1981" CreationDate="2012-04-20T13:42:46.263" Score="5" Body="&lt;p&gt;Configure with PLAPACK support and those operations will work. We are hoping to also add Elemental support this summer.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-04-20T13:42:46.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1985" PostTypeId="2" ParentId="503" CreationDate="2012-04-20T15:25:23.503" Score="6" Body="&lt;p&gt;A first order Taylor expansion can be used to improve convergence over simple lagging. Suppose we have a preconditioner (or factors for a direct solve) available for $A+D$, and we want to use it for preconditioning $A$. We can compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xD;&#xA;A^{-1} &amp;amp;= (A+D-D)^{-1} (A+D) (A+D)^{-1} \\&#xD;&#xA;      &amp;amp;= [(A+D)^{-1} (A+D-D)]^{-1} (A+D)^{-1} \\&#xD;&#xA;      &amp;amp;= [I - (A+D)^{-1} D]^{-1} (A+D)^{-1} \\&#xD;&#xA;     &amp;amp;\approx [I + (A+D)^{-1} D] (A+D)^{-1}&#xD;&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the Taylor expansion was used to write the last line. Application of this preconditioner requires two solves with $A+D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works fairly well when the preconditioner is shifted away from 0 by a similar or larger amount than the operator we are trying to solve with (e.g. $D\gtrsim 0$). If the shift in the preconditioner is smaller ($D \lesssim \min \sigma(A)$), the preconditioned operator becomes indefinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the shift in the preconditioner is much larger than in the operator, this method tends to produce a condition number about half that of preconditioning by the lagged operator (in the random tests I ran, it could be better or worse for a specific class of matrices). That factor of 2 in condition number gives a factor of $\sqrt 2$ in iteration count. If the iteration cost is dominated by the solves with $A+D$, then this is not a sufficient factor to justify the first order Taylor expansion. If matrix application is proportionately expensive (e.g. you only have an inexpensive-to-apply preconditioner for $A+D$), then this first order method may make sense.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-04-20T15:25:23.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1986" PostTypeId="1" AcceptedAnswerId="1989" CreationDate="2012-04-20T15:44:19.043" Score="2" ViewCount="46" Body="&lt;pre&gt;&lt;code&gt;In[2]:= Solve[sqrt(2x-9) == sqrt(4x+3), x]&#xA;&#xA;Out[2]= {{x -&amp;gt; -6}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But mathematically there is no solution, since sqrt (-21) is not defined. There is a flag that is responsible for this?&lt;/p&gt;&#xA;" OwnerUserId="1127" LastActivityDate="2012-04-20T16:09:03.490" Title="The region of allowed values ​​for solving the equation in Mathematica" Tags="|mathematica|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1987" PostTypeId="2" ParentId="1986" CreationDate="2012-04-20T15:55:34.757" Score="2" Body="&lt;p&gt;Try using the &lt;code&gt;Reduce&lt;/code&gt; equation while requiring &lt;code&gt;x&lt;/code&gt; to belong to the &lt;code&gt;Reals&lt;/code&gt;.  See &lt;a href=&quot;http://reference.wolfram.com/mathematica/tutorial/EquationsAndInequalitiesOverDomains.html&quot; rel=&quot;nofollow&quot;&gt;the Mathematica documentation&lt;/a&gt; for an example.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-04-20T15:55:34.757" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1988" PostTypeId="1" AcceptedAnswerId="1992" CreationDate="2012-04-20T16:03:56.120" Score="8" ViewCount="1647" Body="&lt;p&gt;It appears that matlab's &lt;code&gt;eigs&lt;/code&gt; is giving me bad approximations of the smallest eigenvectors of a matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume I can use some slower methods which would also be more accurate...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to find the 2nd smallest eigenvector of a lapalcian matrix (known as the &quot;fiedler&quot; vector). I know of course that the smallest eigenvector of a laplacian matrix is the constant vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions for a more accurate method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S&#xA;In all the above, when I say &quot;smallest eigenvector&quot; I mean the eigenvector associated with the eigenvalue of smallest magnitude. &lt;/p&gt;&#xA;" OwnerUserId="898" LastEditorUserId="9" LastEditDate="2012-04-20T16:19:57.067" LastActivityDate="2012-04-20T19:12:30.567" Title="Compute smallest eigenvectors of a matrix" Tags="|eigensystem|matlab|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1989" PostTypeId="2" ParentId="1986" CreationDate="2012-04-20T16:09:03.490" Score="5" Body="&lt;p&gt;By default, Mathematica assumes that all variables are complex numbers, and when working in the set of complex numbers, &lt;code&gt;Sqrt[-21]&lt;/code&gt; is well defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can tell Mathematica (version 8) that you are working on the set of reals using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Solve[Sqrt[2 x - 9] == Sqrt[4 x + 3], x, Reals]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives no solution. For versions earlier than 8, you need to use &lt;code&gt;Reduce&lt;/code&gt; if you want to specify a domain.&lt;/p&gt;&#xA;" OwnerUserId="605" LastActivityDate="2012-04-20T16:09:03.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1990" PostTypeId="2" ParentId="1988" CreationDate="2012-04-20T16:19:26.863" Score="1" Body="&lt;p&gt;You can get the full eigendecomposition with &lt;code&gt;eig(full(A))&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-04-20T16:19:26.863" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1992" PostTypeId="2" ParentId="1988" CreationDate="2012-04-20T17:16:37.120" Score="5" Body="&lt;p&gt;There is a straightforward way to exploit your a priori knowledge of the smallest eigenpair: you could simply project out the component of the current eigenvector estimate in the direction of the constant vector in each iteration of, say, inverse iteration. You should then expect the iterate to converge to the eigenvector corresponding to the second smallest eigenvalue, your desired Fiedler vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the connectivity of your graph, you may or may not want to run a sparse-direct factorization before inverse iteration in order to accelerate the applications of $A^{-1}$. Once the inverse iteration approach is working, you could also consider replacing it with a Krylov algorithm, which takes a little more work to code but should converge faster.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-04-20T17:16:37.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1993" PostTypeId="2" ParentId="1988" CreationDate="2012-04-20T19:12:30.567" Score="3" Body="&lt;p&gt;You can take the eigenvalue approximation computed by Matlab as the shift $\sigma$ in inverse iteration. If you accumulate the residual to high accuracy, you get in this way full accuracy of the eigenvector, using techniques such as that of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ogita, Rump and Oishi,&#xA;Accurate sum and dot product with applications,&#xA;&lt;a href=&quot;http://oishi.info.waseda.ac.jp/~oishi/papers/OgRuOi05.pdf&quot; rel=&quot;nofollow&quot;&gt;http://oishi.info.waseda.ac.jp/~oishi/papers/OgRuOi05.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An exception is the case where the eigenvalue is close to defective; then you need to use a subspace method applied with $(A-\sigma I)^{-1}$ as the matrix.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-20T19:12:30.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1994" PostTypeId="1" AcceptedAnswerId="1997" CreationDate="2012-04-20T20:26:00.473" Score="7" ViewCount="305" Body="&lt;p&gt;I have an 2D elastodynamics problem, that is a problem which is driven by the Cauchy equation:&#xA;$$\rho\ddot u-\mathrm{div}\sigma=\rho f$$&#xA;where $u$ is the displacement, $\sigma$ the Cauchy stress tensor, $f$ the density of volumetric forces, and $\rho$ the mass density. The constituve law is Hook's law from linear elasticty (homogeneous and isotropic medium). To solve it numerically, I use the finite element method. After space discretization I will have the following equation:&#xA;$$M\ddot u+Ku=F$$&#xA;where $M$ is the mass matrix, $K$ the stiffness matrix and $F$ the load vector. The solution of such an equation oscillate around an equilibrium position and my aim is to add a viscous damping term $C\dot u$ that will &lt;em&gt;critically damp&lt;/em&gt; the system, by which I mean I would like to avoid the oscillatory behaviour as much as possible. The equation now reads:&#xA;$$M\ddot u+C\dot u+Ku=F$$&#xA;The problem is &lt;strong&gt;how to choose&lt;/strong&gt; the matrix $C$ that will achieve the desired effect?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In modal analysis of dynamic structures, the same term $C\dot u$ is added to the discretized Cauchy equation to take damped vibrations into account but for a different purpose: we search for proper modes of vibration. Here we already know the two proper modes of vibration: S-waves and P-waves. So taking into account this information, I would like to build a matrix $C$ that will reflect this physical fact and that contains only constants like $E$, $\mu$ (two classical elasticity modulus) and $\rho$, rather than assuming a Rayleigh damping term $C=\alpha M+\beta K$ and finding the relevant values of $\alpha$ and $\beta$ by hand (numerically).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for any idea!&lt;/p&gt;&#xA;" OwnerUserId="1081" LastActivityDate="2012-04-21T00:03:48.270" Title="Forced viscous damping in elastodynamics" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1995" PostTypeId="1" CreationDate="2012-04-20T22:46:33.887" Score="7" ViewCount="2320" Body="&lt;p&gt;Doing so strikes me as a waste of time. Consider&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;std::complex&amp;lt;double&amp;gt; *a = new std::complex&amp;lt;double&amp;gt;[1&amp;lt;&amp;lt;28];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This could be near-instantaneous and only grab pages once they're used, except it isn't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears I'm not the only one bothered by this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://listengine.tuxfamily.org/lists.tuxfamily.org/eigen/2011/01/msg00124.html&quot;&gt;http://listengine.tuxfamily.org/lists.tuxfamily.org/eigen/2011/01/msg00124.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GCC's libstdc++ (4.6.3) even gloats with this fact:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  ///  Default constructor.  First parameter is x, second parameter is y.&#xA;  ///  Unspecified parameters default to 0.&#xA;  _GLIBCXX_CONSTEXPR complex(const _Tp&amp;amp; __r = _Tp(), const _Tp&amp;amp; __i = _Tp())&#xA;  : _M_real(__r), _M_imag(__i) { }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Worse, since the C++ standard doesn't specify a default constructor, this behavior may even be required. (Is this true?)&lt;/p&gt;&#xA;" OwnerUserId="299" LastEditorUserId="739" LastEditDate="2012-04-20T23:44:37.540" LastActivityDate="2012-04-22T15:12:45.723" Title="Why does std::complex&lt;&gt; initialize its value to 0 upon default construction?" Tags="|c++|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1997" PostTypeId="2" ParentId="1994" CreationDate="2012-04-20T23:54:40.933" Score="8" Body="&lt;p&gt;I'm a bit rusty on structural dynamics, but pages 87-92 of Meirovitch's &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0070413452&quot;&gt;&quot;Fundamentals of Vibration&quot;&lt;/a&gt; explain that a system &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;m\ddot{x} + c\dot{x} + kx = 0&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is critically damped when $c=2m\omega$, where $\omega=\sqrt{\frac{k}{m}}$ is the natural frequency of vibration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a natural generalization to the system of ODEs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;M \ddot{x} + C \dot{x} + Kx = 0.&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an eigenpair of the pencil $(K,M)$, say $(\omega^2,u)$, where $u$ is a unit vector, we have that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;K u = M u \omega^2,&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and thus we could construct a $C$ which critically damps this mode of vibration if we set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;C := 2 \omega (Mu)u^T,&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the matrix $(Mu)u^T$ transforms $u \mapsto Mu$, so that $(Mu)u^T$ is analogous to the $m$ term in the equation $c = 2 \omega m$, and the $2$ and $\omega$ require no generalization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some set of modes $\{u_j\}_j$, with corresponding eigenvalues $\{\omega^2_j\}_j$, you could critically damp those modes by setting&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;C := 2 \sum_j \omega_j (Mu_j)u_j^T.&#xD;&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-04-21T00:03:48.270" LastActivityDate="2012-04-21T00:03:48.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1998" PostTypeId="1" AcceptedAnswerId="2000" CreationDate="2012-04-21T06:21:46.143" Score="16" ViewCount="1521" Body="&lt;p&gt;Many of the better-known C++ libraries in computational science such as &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigen&lt;/a&gt;, &lt;a href=&quot;https://trilinos.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trilinos&lt;/a&gt;, and &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;deal.II&lt;/a&gt; use the standard C++ template header library object, &lt;code&gt;std::complex&amp;lt;&amp;gt;&lt;/code&gt;, to represent complex floating-point numbers.&lt;/p&gt;&#xA;&lt;p&gt;In Jack Poulson's &lt;a href=&quot;https://scicomp.stackexchange.com/a/1996/9&quot;&gt;answer&lt;/a&gt; to a question about default constructors, he points out that he has his own implementation of &lt;code&gt;std::complex&lt;/code&gt; in &lt;a href=&quot;https://github.com/elemental/Elemental&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elemental&lt;/a&gt; &amp;quot;for a number of reasons&amp;quot;.  What are those reasons?  What are the advantages and disadvantages to this approach?&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="23791" LastEditDate="2022-01-11T15:37:05.107" LastActivityDate="2022-01-11T16:15:47.263" Title="Why would a computational scientist need to implement their own version of std::complex?" Tags="|linear-algebra|c++|programming-paradigms|complex-arithmetic|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="1999" PostTypeId="1" CreationDate="2012-04-21T06:30:40.840" Score="3" ViewCount="195" Body="&lt;p&gt;I have an iterative method for computing the Moore-Penrose generalized inverse of matrices, that is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X_{k+1} = ((I-\beta X_{k}A)^t) + X_{k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial approximation: &#xA;$$X_{0} = \beta AA^t$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$A$ is a given $m \times n$ matrix&lt;/li&gt;&#xA;&lt;li&gt;$\beta \in [0,1]$&lt;/li&gt;&#xA;&lt;li&gt;$X_{k}$ is the $k$th iterate generated by the algorithm, approximating the Moore-Penrose generalized inverse of matrix $A$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can this method be applied for higher-order matrices?&lt;/li&gt;&#xA;&lt;li&gt;What is the advantage of this method over other algorithms like singular value decomposition and QR factorization?&lt;/li&gt;&#xA;&lt;li&gt;How would I determine if the iterative algorithm runs faster than the other methods mentioned above?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1359" LastEditorUserId="34" LastEditDate="2012-04-23T08:13:48.773" LastActivityDate="2012-04-23T08:13:48.773" Title="How to establish that an iterative method can be applied to large matrices whose size may reach 10^3?" Tags="|linear-algebra|iterative-method|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2000" PostTypeId="2" ParentId="1998" CreationDate="2012-04-21T06:34:44.487" Score="19" Body="&lt;p&gt;I believe this discussion has come up a number of times on the PETSc list. My main reasons are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The C++ standard states that std::complex is only defined for the float, double, and long double datatypes. Thus it cannot be used for other datatypes, such as quad-precision.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The standard makes no guarantees about the stability of the complex arithmetic.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The standard makes no guarantee that the data in an std::complex is stored as the real component followed by the imaginary component. This is crucial for interfaces with external libraries, such as BLAS and LAPACK. It is true for all major implementations, but I would prefer to be able to ensure it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I prefer being able to directly manipulate the real and imaginary components. std::complex makes this unnecessarily difficult.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I would like to eventually have a more general version which only requires the datatype to be a ring instead of requiring a field. This would include the Gaussian integers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-04-21T06:34:44.487" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2002" PostTypeId="2" ParentId="1982" CreationDate="2012-04-21T07:12:41.103" Score="4" Body="&lt;p&gt;Depending on which version of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow&quot;&gt;Traveling Salesman Problem (TSP)&lt;/a&gt; you're looking at, it's either &lt;a href=&quot;http://en.wikipedia.org/wiki/NP-hard&quot; rel=&quot;nofollow&quot;&gt;NP-hard&lt;/a&gt; (finding the shortest possible route), or &lt;a href=&quot;http://en.wikipedia.org/wiki/NP-complete&quot; rel=&quot;nofollow&quot;&gt;NP-complete&lt;/a&gt; (called the &lt;em&gt;decision version&lt;/em&gt; of TSP; given a length $L$, determine if &lt;em&gt;any&lt;/em&gt; tour of a given graph has length less than or equal to $L$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between the two is subtle, but important: NP-hard problems are &lt;em&gt;at least as hard&lt;/em&gt; as any problem in &lt;a href=&quot;http://en.wikipedia.org/wiki/NP_%28complexity%29&quot; rel=&quot;nofollow&quot;&gt;NP&lt;/a&gt;. The way you show a problem is at least as hard as another problem is to use what's called a &lt;a href=&quot;http://en.wikipedia.org/wiki/Reduction_%28complexity%29&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;reduction&lt;/em&gt;&lt;/a&gt;. A reduction is a way of transforming your given problem into the problem you'd like to compare it with. In this case, if you want to compare TSP to, say, a known NP problem like &lt;a href=&quot;http://en.wikipedia.org/wiki/K-SAT#3-satisfiability&quot; rel=&quot;nofollow&quot;&gt;3-SAT&lt;/a&gt; (satisfiability of a Boolean statement consisting of clauses containing 3 literals), you would try to reduce TSP to 3-SAT. The idea is to convert an instance of a problem $a$ that we &lt;em&gt;don't&lt;/em&gt; yet know how to solve into a problem $b$ that we &lt;em&gt;do&lt;/em&gt; know how to solve, such that the solution to $b$ can be used to determine a solution to $a$. Then, by converting $a$ to $b$ and solving $b$, we've constructed an algorithm to solve $a$. If we know how long the conversion process -- that is, the reduction -- takes, and we know how long it takes to solve problem $b$, then we can figure out an upper bound on how long it takes to solve the problem $a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common and useful kind of reduction I saw in my introductory graduate-level theory of computation course was called a &lt;a href=&quot;http://en.wikipedia.org/wiki/Polynomial-time_Turing_reduction&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;polynomial-time reduction&lt;/em&gt;&lt;/a&gt;, which is a reduction process whose algorithm takes polynomial time, as a function of some metric of problem size, to produce its output. This concept is then used to define what &lt;em&gt;NP-complete&lt;/em&gt; means. A problem $p$ is NP-complete if it is in NP, and for every problem in NP, there exists a polynomial-time reduction from that problem to problem $p$. &lt;em&gt;NP-hard&lt;/em&gt; can be defined a similar way: a problem $p$ is NP-hard if there exists a polynomial time reduction from an NP-complete problem to $p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that all of that terminology is out of the way, to answer your question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you're talking about the NP-complete decision version of TSP, then any problem in NP can be reduced in polynomial time to TSP. Global optimization problems tend to be NP-hard (though I don't know for sure that all of them are, I do know that nonconvex optimization problem are NP-hard). Since NP-hard problems by definition are polynomial time reductions of NP-complete problems, TSP can be polynomial time reduced to NP-hard global optimization problems.&lt;/li&gt;&#xA;&lt;li&gt;If you're talking about the NP-hard version of TSP, then no statements can be made about reductions of NP-hard global optimization problems.&lt;/li&gt;&#xA;&lt;li&gt;There is nothing to suggest that NP-hard optimization problems can be reduced to either version of TSP. However, it is known that &lt;a href=&quot;http://en.wikipedia.org/wiki/Integer_linear_programming#Integer_unknowns&quot; rel=&quot;nofollow&quot;&gt;0-1 integer programming&lt;/a&gt; is an NP-complete problem, so it can be reduced to TSP.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A good, but dense, reference is &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/B002L6GJG0&quot; rel=&quot;nofollow&quot;&gt;Michael Sipser's &lt;em&gt;Introduction to the Theory of Computation, 2nd edition&lt;/em&gt;&lt;/a&gt;. (&lt;em&gt;Disclaimer:&lt;/em&gt; Professor Sipser taught my theory of computation class.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-04-21T07:38:16.233" LastActivityDate="2012-04-21T07:38:16.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2003" PostTypeId="2" ParentId="1998" CreationDate="2012-04-21T12:54:10.950" Score="8" Body="&lt;p&gt;I use &lt;code&gt;std::complex&amp;lt;&amp;gt;&lt;/code&gt; in my programs, and have to fight with compiler flags and workaround for each new compiler or compiler upgrade. I will try to recount these fights in chronological order:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Performance measurements showed that a a step involving only computing the square of the absolute value of a field of complex numbers took more time than a preceding FFT for gcc-4.x. Digging into the generated assembler code showed that &lt;code&gt;std::norm&lt;/code&gt; ($|z|^2$) computed the absolute value ($|z|$) in a way avoiding overflow, and then squared the result. This problem could be fixed by the compile flag &lt;code&gt;-ffast-math&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The intel icc compiler on linux (or linker) compiled &lt;code&gt;std::arg&lt;/code&gt; to a non-opt under certain configurations (link compatibility with a specific gcc-version). The problem resurfaced too often, so &lt;code&gt;std::arg&lt;/code&gt; had to be replaced by &lt;code&gt;atan2(imag(),real())&lt;/code&gt;. But it was all too easy to forget this when writing new code.&lt;/li&gt;&#xA;&lt;li&gt;The type &lt;code&gt;std::complex&lt;/code&gt; uses different call conventions (=ABI) than the build-in C99 complex type, and the built-in Fortran complex type for newer gcc versions.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;code&gt;-ffast-math&lt;/code&gt; compile flag interacts with the handling of floating point exceptions in unexpected ways. What happens is that the compiler pulls divisions out of loops, thereby causing &lt;code&gt;division by zero&lt;/code&gt; exceptions at runtime. These exceptions would have never happened inside the loop, because the corresponding division didn't take place due to the surrounding logic. That one was really bad, because it was a library that was compiled separately from the program which used the floating point exception handing (using different compile flags) and run into these issues (the corresponding teams were sitting at opposite parts of the world, so this issue really caused bad trouble). This was solved by doing the optimization used by the compiler by hand with more care.&lt;/li&gt;&#xA;&lt;li&gt;The library became part of the program and no longer used the &lt;code&gt;-ffast-math&lt;/code&gt; compile flag. After an upgrade to a newer gcc version, performance dropped by a huge factor. I haven't investigated this issue in detail yet, but I fear it is related to &lt;a href=&quot;http://www.softintegration.com/docs/whitepaper/j_ddj.pdf&quot;&gt;C99 Annex G&lt;/a&gt;. I have to admit that I'm completely confused by this strange definition of multiplication for complex numbers, and there even seems to exist different versions of this with claims that the other versions are  misguided. I hope that the &lt;a href=&quot;http://old.nabble.com/performance-question-with-std%3a%3acomplex%3Cfloat%3E-in-new-g++-versions-td27187415.html&quot;&gt;&lt;code&gt;-fcx-limited-range&lt;/code&gt;&lt;/a&gt; compile flag will solve the issue, because there seems to be another problem related to &lt;code&gt;-ffast-math&lt;/code&gt; for this newer gcc version.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;code&gt;-ffast-math&lt;/code&gt; compile flag makes the behavior of &lt;code&gt;NaN&lt;/code&gt; completely unpredictable for newer versions of gcc (even &lt;code&gt;isnan&lt;/code&gt; is affected). The only workaround seems to be to avoid any occurrence of &lt;code&gt;NaN&lt;/code&gt; in the program, which defeats the purpose for the existence of &lt;code&gt;NaN&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now you may ask whether I plan to abandon the built-in complex types and &lt;code&gt;std::complex&lt;/code&gt; for these reasons. I will stay with the built-in types, as long as I stay with C++. In case C++ should manage to become completely unusable for scientific computing, I would rather considering to switch to a language that takes more care of the issues relevant to scientific computing.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-04-21T12:54:10.950" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2004" PostTypeId="1" AcceptedAnswerId="2005" CreationDate="2012-04-21T13:47:26.967" Score="10" ViewCount="25270" Body="&lt;p&gt;I am getting the warning in the post subject when attempting to optimize a function in Python with the &lt;strong&gt;scipy.optimize.fmin_bfgs&lt;/strong&gt; function.&#xA;The complete output:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Warning: Desired error not necessarily achieveddue to precision loss&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     Current function value: nan&#xA;     Iterations: 1&#xA;     Function evaluations: 18&#xA;     Gradient evaluations: 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is not a fatal error, I am getting answers, but they are far from the optimum I search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any typical &quot;noob&quot; errors that might generate this error? I started working with this package only yesterday; as it is, I'm not even sure where to start looking.&#xA;Any help is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="1354" LastEditorUserId="9" LastEditDate="2012-04-21T15:24:07.113" LastActivityDate="2023-12-11T08:01:44.010" Title="scipy.optimize.fmin_bfgs: &quot;Desired error not necessarily achieved due to precision loss&quot;" Tags="|optimization|python|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2005" PostTypeId="2" ParentId="2004" CreationDate="2012-04-21T15:51:50.930" Score="12" Body="&lt;p&gt;It is helpful in situations like this to take a look at the &lt;a href=&quot;https://github.com/scipy/scipy/blob/master/scipy/optimize/optimize.py&quot; rel=&quot;noreferrer&quot;&gt;source code&lt;/a&gt;.  This is easy because everything in &lt;code&gt;scipy&lt;/code&gt; is open source!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see from reading the source code, the warning message is printed when &lt;code&gt;warnflag==2&lt;/code&gt;.  This gets set elsewhere in the code when the &lt;code&gt;linesearch&lt;/code&gt; function returns None (it fails).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why does linesearch fail?  The goal of an optimization algorithm is to find the minima of some objective function through a successive set of iterations.  The line search, in this case, is trying to find a step size where the approximations in BFGS are still valid.  When the Hessian of your function or its gradient are ill-behaved in some way, the bracketed step size could be computed as zero, even though the gradient is non-zero.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess my suggestion is to either go to the literature (Nocedal and Wright has a good discussion of line search and the BFGS method) or to your function and make sure that it is well-behaved in the region you are searching.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2017-09-04T20:28:38.290" LastActivityDate="2017-09-04T20:28:38.290" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2008" PostTypeId="1" CreationDate="2012-04-21T19:54:07.967" Score="30" ViewCount="2862" Body="&lt;p&gt;As someone who holds a BA in physics I was somewhat scandalized when I began working with molecular simulations. It was a bit of a shock to discover that even the most detailed and computationally expensive simulations can't quantitatively reproduce the full behavior of water from first principles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previously, I had been under the impression that the basic laws of quantum mechanics were a solved problem (aside from gravity, which is usually assumed to be irrelevant at molecular scale). However, it seems that once you try to scale those laws up and apply them to anything larger or more complex than a hydrogen atom their predictive power begins to break down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a mathematics point of view, I understand that the wave functions quickly grow too complicated to solve and that approximations (such as Born-Oppenheimer) are required to make the wave functions more tractable. I also understand that those approximations introduce errors which propagate further and further as the time and spatial scales of the system under study increase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the nature of the largest and most significant of these approximation errors? How can I gain an intuitive understanding of those errors? Most importantly, how can we move towards an ab-initio method that will allow us to accurately simulate whole molecules and populations of molecules? What are the biggest unsolved problems that are stopping people from developing these kinds of simulations?&lt;/p&gt;&#xA;" OwnerUserId="34" LastActivityDate="2012-06-23T14:19:25.937" Title="Where do the laws of quantum mechanics break down in simulations?" Tags="|quantum-mechanics|simulation|" AnswerCount="7" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2009" PostTypeId="2" ParentId="1999" CreationDate="2012-04-21T22:39:56.200" Score="2" Body="&lt;p&gt;From the phrasing of question 2, I'm assuming that this is a homework question, so I'm going to give vague answers and not feel too bad about it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) A 1000 X 1000 matrix isn't that big, so as long as your computer has enough memory (and any computer made in the last ten years probably would), yes you can run this algorithm on such matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 and 3) These are related questions. Your best bet would be to start with an &lt;a href=&quot;http://courses.cs.vt.edu/~cs2604/Summer2000/Notes/C03.Asymptotics.pdf&quot; rel=&quot;nofollow&quot;&gt;order of magnitude analysis&lt;/a&gt; for each of your algorithms. Basically, using this kind of analysis you can roughly figure out how many computational steps are required for each algorithm as a function of the size of matrix $A$. You can probably assume that if an algorithm takes fewer steps to produce an answer then it will run faster (although in reality this depends on the computational cost of each step. Cost is very implementation dependent, however, so you may not want to get into it). Other possible advantages one algorithm can have over another include requiring less memory or being easier to parallelize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in terms of running speed alone, you can make a set of test cases and then time how long it takes for each algorithm to finish the set. &lt;/p&gt;&#xA;" OwnerUserId="34" LastActivityDate="2012-04-21T22:39:56.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2010" PostTypeId="2" ParentId="1995" CreationDate="2012-04-22T02:59:31.207" Score="2" Body="&lt;p&gt;I'd say initialization to zero follows the principle of least surprise. Having uninitialized values around is a prescription for errors, and surely initialization is a far cheaper process than almost anything else you're going to do later on with these complex numbers. For example, allocating and initializing to zero 1&amp;lt;&amp;lt;28 complex values can be done with one malloc and one memset.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-22T02:59:31.207" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2011" PostTypeId="2" ParentId="1999" CreationDate="2012-04-22T09:56:47.490" Score="2" Body="&lt;p&gt;Program it in Matlab, say, and test it against SVD for increrasing dimensions and matrices generated by $A=BDC$ with random square B and nonrandom diagonal $D&amp;gt;0$ with increasing ratio of $\max_i D_{ii}/\min_i D_{ii}$ Most likely, your method is much slower for the same accuracy.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-22T09:56:47.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2012" PostTypeId="1" AcceptedAnswerId="2025" CreationDate="2012-04-22T17:24:35.053" Score="9" ViewCount="1545" Body="&lt;p&gt;In the paper &lt;a href=&quot;http://epubs.siam.org/sinum/resource/1/sjnaam/v29/i6/p1610_s1&quot;&gt;Hierarchical Conforming Finite Element Methods for the Biharmonic Equation&lt;/a&gt;, P. Oswald claimed Clough-Tocher type elements has $C^1$-continuity while being a cubic polynomial on each triangle. He didn't give a set of explicit basis functions just the standard degrees of freedom on the quadrature points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, in the book &lt;a href=&quot;http://books.google.com/books?id=ci4c_R0WKYYC&quot;&gt;The Mathematical Theory of Finite Element Methods&lt;/a&gt; Chapter 3, the authors give us the construction of cubic Hermite finite elements, but they didn't mention the continuity of the cubic Hermite elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the paper &lt;a href=&quot;http://www.ima.umn.edu/~arnold/papers/icm2002.pdf&quot;&gt;Differential complexes and numerical stability&lt;/a&gt;, Doulgas Arnold proposed that for $C^1$/$H^2$-conforming discrete space we should use the Hermite quintic(or rather Argyris) finite elements, which is very complicated to express explicitly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here are my questions: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Is there any paper that comes up with an explicit formula for the $C^1$/$H^2$-conforming finite elements on triangular or tetrahedral mesh? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Should piecewise cubic be the minimal degree of polynomials requirement for $C^1$-continuity?&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2012-11-16T13:58:55.050" Title="Construction of $C^1$/$H^2$-conforming finite element basis for triangular or tetrahedral mesh" Tags="|finite-element|reference-request|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2013" PostTypeId="1" AcceptedAnswerId="2014" CreationDate="2012-04-22T23:34:37.663" Score="10" ViewCount="2551" Body="&lt;p&gt;Let $T$ be a triangle and let $f$ be a smooth function on $T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can use mid-point quadrature $\int f dx \approx |T|\cdot f(x_M)$, where $x_M$ is the middle-point of $T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you provide me with (a reference for) higher-order formulas on a simplex?&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2016-12-08T15:02:16.203" Title="Higher-order numerical integration on a triangle/tetrahedron/simplex" Tags="|quadrature|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2014" PostTypeId="2" ParentId="2013" CreationDate="2012-04-23T01:40:37.000" Score="7" Body="&lt;p&gt;Is it important to have the most efficient rules possible? Do you require all weights to be positive and/or all points to be inside the region? Two general resources:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://nines.cs.kuleuven.be/ecf/&quot;&gt;Encyclopaedia of Cubature Formulas&lt;/a&gt; and referenced papers.&lt;/li&gt;&#xA;&lt;li&gt;The Libmesh source contains good choices of cubature rules for all orders, with and without negative weights, thanks to John Peterson. An example &lt;a href=&quot;http://libmesh.sourceforge.net/doxygen/quadrature__gauss__3D_8C_source.php&quot;&gt;src/quadrature/quadrature_gauss_3D.C&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-04-23T01:40:37.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2015" PostTypeId="1" AcceptedAnswerId="2020" CreationDate="2012-04-18T18:47:47.410" Score="4" ViewCount="138" Body="&lt;p&gt;I am running a linear algebra iterative method (PCG) for solving Ax=b, the dimension of the matrix is 10000x10000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I did 2 preliminary analyses:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Memory Analysis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The size of the matrix dominates the total storage required. Thats about 1E4 x 1E4 = 1E8 elements of double precision which is approximately 0.8 GB of data. The number of iterations required for convergence was 450. Since this won't fit in cache, I assume no cache benefits, that would mean 450 x 0.8 = 360 GB of data transfer. With a memory bandwidth of 10 GB/s, thats approximately 36 seconds for memory transfers.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Flop Analysis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I calculated that I will be carrying out 1 matrix vector (dominant operation) per iteration for 450 iterations. That is &lt;code&gt;cN^2&lt;/code&gt; operations/iterations x 450 iterations = &lt;code&gt;450c N^2&lt;/code&gt; operations with a 2.13GHz processor (ensured to work on a single processor only). That is &lt;code&gt;21.12c&lt;/code&gt; for N=10000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find c, I carried out MatVecs for all sizes from 1 to 19000, plotted the graph for &lt;strong&gt;No. of Operations vs (Dimension)^2 (Operations = slope x Dim^2)&lt;/strong&gt; where &lt;code&gt;No. of Operations = Time x 2.13 GHz&lt;/code&gt;. I used the slope of this (linear) graph as &lt;code&gt;c&lt;/code&gt;&#xA;. Which came out to be &lt;code&gt;50&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, total time = 21.12c = 1000 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, assuming both memory transfer and operations happen concurrently, theoretically, it should take 1000 seconds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in reality, the code took 120 seconds maximum. Where did I go wrong? My calculation is fairly off.&lt;/p&gt;&#xA;" OwnerUserId="529" OwnerDisplayName="Nunoxic" LastActivityDate="2012-04-23T04:40:03.243" Title="Whats wrong with my running time calculation?" Tags="|performance|linear-algebra|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2016" PostTypeId="2" ParentId="2015" CreationDate="2012-04-18T19:07:19.450" Score="2" Body="&lt;p&gt;Must be in this : No. of Operations = Time x 2.13 GHz , operations in a process can't be caculated this way. There are numbers of variables that will influence this notably the kind of cpu that you are using, heat and parellism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The performance or speed of a processor depends on the clock rate (generally given in multiples of hertz) and the instructions per clock (IPC), which together are the factors for the instructions per second (IPS) that the CPU can perform.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only variable you have here is clock rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go read this here : &lt;a href=&quot;http://en.wikipedia.org/wiki/Central_processing_unit#Design_and_implementation&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Central_processing_unit#Design_and_implementation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="Machinegon" LastActivityDate="2012-04-18T19:07:19.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2017" PostTypeId="2" ParentId="2015" CreationDate="2012-04-19T08:39:46.433" Score="4" Body="&lt;p&gt;I find it hard to believe that c is 50. Theoretically, a matrix-vector product requires 2N^2 flops if the matrix is N-by-N, which gives c = 2. In practice, stuff like indexing operations, vectorization and quality of implementation influence this, but c = 50 seems very high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, assuming that PCG = preconditioned conjugate gradient, which preconditioner do you use? Are you sure that the cost of preconditioning can be neglected?&lt;/p&gt;&#xA;" OwnerUserId="153" OwnerDisplayName="Jitse Niesen" LastActivityDate="2012-04-19T08:39:46.433" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2018" PostTypeId="1" CreationDate="2012-04-23T03:38:24.153" Score="3" ViewCount="2103" Body="&lt;p&gt;I am working on an assignment where I am determining the temperature distribution of a chip on a substrate. When I decrease the nodal spacing the results change drastically. The smaller the nodal spacing the larger the temperatures i get. However when the nodal spacing is really small as in .00001m the temperature is nearly 0. Should this happen? What should i look for in the code to possibly remedy this situation? I am using MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok. Guess I need to add more info. I think the best way to show my equations is to show my code. I am fairly confident about the equations used at each node. I derived them all by hand. When I use a nodal spacing, delta of 0.003 I get really low temperatures. At a delta of 0.001 I get reasonable temperatures. I think think the reason is that at 0.003 meters there are not enough node with heat generation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    %**********************************************&#xA;%--------Thermo Lab Numerical Project---------|&#xA;%-----------------Paul Fjare------------------|&#xA;%**********************************************&#xA;clear; clc&#xA;%Cooling of a silicon chip mounted in a dielectric&#xA;%substrate. &#xA;&#xA;ks=5; % W/m-K&#xA;kc=50;&#xA;ksc=(ks+kc)/2;&#xA;&#xA;h=500; % W/m^2-K&#xA;qgen=10^7; %W/m^3&#xA;L=.027; % meters&#xA;H=.012;&#xA;Tamb=293.15; % temperature of coolant in Kelvin&#xA;delta=.003; % meters&#xA;N=100000; %iterations&#xA;B=h*delta/ks; %dimensionless&#xA;sectionL=L/3;&#xA;sectionH=H/4;&#xA;%Determine size of matrix&#xA;cols = round(L/delta + 1)    % Columns&#xA;rows = round(H/delta + 1)    % Rows&#xA;sectioncols=round(sectionL/delta+1)&#xA;sectionrows=round(sectionH/delta+1)&#xA;&#xA;% Build beginning zeros matrix from size above&#xA;T = zeros([rows,cols]);&#xA;%T=80*ones([rows,cols]);&#xA;&#xA;for i=1:N&#xA;    % 4 nodes conduction no heat gen&#xA;    for m = sectionrows+1:rows-1&#xA;        for n = 2:cols-1&#xA;            T(m,n) = 0.25 *(T(m,n+1) + T(m,n-1) + T(m+1,n) + T(m-1,n));&#xA;        end&#xA;    end&#xA;    %upper space between the chip and walls&#xA;        %left&#xA;        for m=2:sectionrows&#xA;            for n=2:sectioncols-1&#xA;                T(m,n)=0.25 *(T(m,n+1) + T(m,n-1) + T(m+1,n) + T(m-1,n));&#xA;            end&#xA;        end&#xA;&#xA;        %right&#xA;        for m=2:sectionrows&#xA;            for n=2*sectioncols:cols-1&#xA;                T(m,n)=0.25 *(T(m,n+1) + T(m,n-1) + T(m+1,n) + T(m-1,n));&#xA;            end&#xA;        end&#xA;&#xA;    %------chip------&#xA;    %top &#xA;        for n=sectioncols+1:2*sectioncols-2&#xA;            T(1,n)=(T(m,n-1)+T(m,n+1)+2*T(m+1,n)+qgen*delta^2/kc+2*h*delta/kc)/(4+2*h*delta/kc);&#xA;        end&#xA;        %top corners&#xA;        for n=sectioncols&#xA;            T(1,n)=(ks*T(1,n-1)+kc*T(1,n+1)+ksc*T(2,n)+2*qgen*delta^2/4+2*h*delta*Tamb)/(ks+kc+ksc+2*h*delta);&#xA;        end&#xA;        for n=2*sectioncols-1&#xA;            T(1,n)=(ks*T(1,n+1)+kc*T(1,n-1)+ksc*T(2,n)+2*qgen*delta^2/4+2*h*delta*Tamb)/(ks+kc+ksc+2*h*delta);&#xA;        end&#xA;&#xA;&#xA;    %bottom and bottom corners &#xA;    for m=sectionrows&#xA;        for n=sectioncols+1:2*sectioncols-2&#xA;            T(m,n)=(ks*T(m+1,n)+ksc*(T(m,n+1)+T(m,n-1))+kc*T(m-1,n)+qgen*delta^2/2)/(ks+2*ksc+kc);&#xA;        end&#xA;        %bottom corners&#xA;        for n=sectioncols&#xA;            T(m,n)=(ks*T(m,n-1)+ks*T(m+1,n)+ksc*T(m,n+1)+ksc*T(m-1,n)+qgen*delta^2/4+h*delta*Tamb)/(ks+2*ksc+ks+delta*h);&#xA;        end&#xA;        for n=2*sectioncols-1&#xA;            T(m,n)=(ks*T(m,n+1)+ks*T(m+1,n)+ksc*T(m,n-1)+ksc*T(m-1,n)+qgen*delta^2/4+h*delta*Tamb)/(ks+2*ksc+ks+delta*h);&#xA;        end&#xA;    end&#xA;    if sectionrows&amp;gt;2&#xA;        for m=2:sectionrows-1&#xA;            for n=sectioncols+1:2*sectioncols-2&#xA;                T(m,n) = 0.25 *(T(m,n+1) + T(m,n-1) + T(m+1,n) + T(m-1,n)+qgen*delta^2) ;&#xA;            end&#xA;        end&#xA;    end&#xA;    %chip sides&#xA;   if sectionrows&amp;gt;2&#xA;&#xA;       for m=2:sectionrows-1&#xA;        for n=2*sectioncols-1&#xA;            T(m,n)=(ks*T(m,n+1)+ksc*(T(m+1,n)+T(m-1,n))+kc*T(m,n-1)+qgen*delta^2/2)/(ks+2*ksc+kc);&#xA;        end&#xA;        for n=sectioncols&#xA;             T(m,n)=(ks*T(m,n-1)+ksc*(T(m+1,n)+T(m-1,n))+kc*T(m,n+1)+qgen*delta^2/2)/(ks+2*ksc+kc);&#xA;        end&#xA;       end&#xA;    end&#xA;    %------end of chip-----&#xA;&#xA;&#xA;    %top of substrate between chip and walls&#xA;        %left&#xA;        for n=2:sectioncols-1&#xA;        T(1,n)=(T(1,n-1)+T(1,n+1)+2*T(1+1,n)+2*h*delta/ks)/(4+2*h*delta/ks);&#xA;        end&#xA;        %right&#xA;        for n=2*sectioncols:cols-1&#xA;        T(1,n)=(T(1,n-1)+T(1,n+1)+2*T(1+1,n)+2*h*delta/ks)/(4+2*h*delta/ks);&#xA;        end&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;    %upper left corner of substrate: node 1,1&#xA;        T(1,1)=(T(1,2)+T(2,1)+Tamb*B)/(2+B);&#xA;    %upper right corner of substrate: node 1,cols&#xA;        T(1,cols)=(T(1,cols-1)+T(2,cols)+Tamb*B)/(2+B);&#xA;&#xA;    %left side of substrate&#xA;    for m=2:rows-1&#xA;        T(m,1)=(T(m-1,1)+T(m+1,1)+2*T(m,2))/4;&#xA;    end&#xA;    %right side of substrate&#xA;    for m=2:rows-1&#xA;        T(m,cols)=(T(m-1,cols)+T(m+1,cols)+2*T(m,cols-1))/4;&#xA;    end&#xA;&#xA;    %lower left corner: node 5,1&#xA;    T(rows,1)=(T(rows-1,1)+T(rows,2))/2;&#xA;    %lower right corner: node 5,10&#xA;    T(rows,cols)=(T(rows-1,cols)+T(rows,cols-1))/2;&#xA;&#xA;&#xA;    %bottom of substrate&#xA;    for n=2:cols-1&#xA;    T(rows,n)=(T(rows,n-1)+T(rows,n+1)+2*T(rows-1,n))/4;&#xA;    end&#xA;end&#xA;&#xA; T=T; %temperature distribution in Kelvin&#xA; TC=T-273.15 %temperature distribution in degrees Celsius&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A picture of the physical situation is shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XJIEQ.png&quot; alt=&quot; &quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1364" LastEditorUserId="1364" LastEditDate="2012-04-23T15:11:32.263" LastActivityDate="2012-04-23T15:11:32.263" Title="Gauss-Seidel iterations node spacing" Tags="|matlab|convergence|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2019" PostTypeId="1" CreationDate="2012-04-23T03:41:08.277" Score="3" ViewCount="114" Body="&lt;p&gt;It was suggested I should try posting this question here from &lt;a href=&quot;https://math.stackexchange.com/questions/135447/efficient-principal-pivots&quot;&gt;Mathematics&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on a numerical linear algebra package in C#.  I'm trying to implement a variety of &quot;principal pivoting&quot; methods to solve optimization problems (specifically linear complementarity problems, but I think similar schemes are used in linear and quadratic programming).  These methods require the ability to multiply a column vector against the &quot;principal pivotal transform&quot; of a matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principal pivot transform (PPT) of a matrix $M$ is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$PM^'P^T = \left (\begin{array}{cc} M_{\alpha\alpha}^{-1} &amp;amp;\ -M_{\alpha\alpha}^{-1} M_{\alpha\beta} \\ M_{\beta\alpha}M_{\alpha\alpha}^{-1} &amp;amp;\ M_{\beta\beta} - M_{\beta\alpha} M_{\alpha\alpha}^{-1}M_{\alpha\beta} \end{array} \right )$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\alpha$ is the set of pivoted row/column indices, and $\beta$ is the set of unpivoted row/column indices.  $P$ is a permutation matrix that keeps the pivoted and unpivoted rows/columns together to make the block matrix form above cleaner.  $M_{\alpha\beta}$ is a submatrix of $M$ with the rows in $\alpha$ and the columns in $\beta$, and the other submatrices of $M$ are defined similarly.  Also, $M_{\alpha\alpha}^{-1} = (M_{\alpha\alpha})^{-1}$ and not $(M^{-1})_{\alpha\alpha}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each iteration of a principal pivot method swaps a single index from $\alpha$ to $\beta$ or vice-verse.  This single pivoting operation can be expressed as a low rank update of the matrix.  Say we're pivoting index $r$ and using $M^'$ to represent the updated matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{matrix}&#xA;M_{rr}^' &amp;amp; = &amp;amp; &amp;amp; \frac{1}{M_{rr}} &amp;amp; \\ \\ \\&#xA;M_{ir}^' &amp;amp; = &amp;amp; &amp;amp; \frac{M_{ir}}{M_{rr}}, &amp;amp; (i \neq r) \\ \\ \\&#xA;M_{ri}^' &amp;amp; = &amp;amp; &amp;amp; \frac{-M_{ri}}{M_{rr}}, &amp;amp; (i \neq r) \\ \\ \\&#xA;M_{ij}^' &amp;amp; = &amp;amp; M_{ij} - &amp;amp; \frac{M_{ir} * M_{ri}}{M_{rr}}, &amp;amp; (i \neq r, j \neq r)&#xA;\end{matrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I'm currently doing.  Unfortunately, this is basically calculating $M_{\alpha\alpha}^{-1}$ using something like Gaussian elimination.  I have a few test cases where the results are suffering from numerical problems even on on small problems, even using double precision floating point.  To help combat this, if I remove an index from $\alpha$ (that is, if I unpivot an index), I recalculate the matrix from scratch using one pivot at a time with the above scheme.  But this makes the Principal Pivot algorithm I have quite slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there are reasonable ways to calculate the product $M_{\alpha\alpha}^{-1} z$ for some column vector $z$ without calculating $M_{\alpha\alpha}^{-1}$ directly (such as LU decomposition or QR decomposition).  I could calculate this decomposition fresh every iteration from the current pivot set ($\alpha$), and probably get a pretty robust method.  But then each iteration of the pivoting algorithm would take $O(n^3)$; that's too slow.  I could do a low rank update of the LU or QR decomposition, but since most of the algorithms pivot a single index at a time, I lose most of the benefit of these decomposition schemes after a few iterations.  I could perform the full decomposition only every few iterations, but then the question becomes how often should I do so?  And it still would be quite slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something more clever that can be done?  I think I basically need to find the decomposition of principal submatrices.  Is there any literature on the subject?  Any other ways to approach the problem?&lt;/p&gt;&#xA;" OwnerUserId="1365" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-04-23T03:41:08.277" Title="Efficient principal pivots" Tags="|linear-algebra|optimization|nonlinear-programming|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2020" PostTypeId="2" ParentId="2015" CreationDate="2012-04-23T04:40:03.243" Score="3" Body="&lt;p&gt;As Jitse pointed out, the mistake is clearly in your estimate for the cost of $c$.  As you probably noticed, the graph for estimating almost any computation as a function of array size is not particularly linear, there's a lot of jitter due to weird cache effects, and there are several clear regions where L1, L3, and main memory dominate.  I noticed that Mystical also pointed this out to you in &lt;a href=&quot;http://chat.stackoverflow.com/transcript/10242&quot;&gt;your chat discussion&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, if you have a good estimate of the cost per mat-vec from direct measurement, the $c$ you are calculating takes into account both memory and flop/s.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overall picture for this sort of computation is messy.  It seems like you are trying to get into very fine performance analysis details without a good high-level understanding of computer architecture and pipelined performance.  Have you read Hennessy and Patterson's &lt;a href=&quot;http://store.elsevier.com/product.jsp?isbn=9780123838728&quot; rel=&quot;nofollow&quot;&gt;Computer Architecture: A Quantitative Approach&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-04-23T04:40:03.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2021" PostTypeId="2" ParentId="2018" CreationDate="2012-04-23T04:46:19.450" Score="5" Body="&lt;p&gt;Your question is hard to answer because it doesn't provide enough information for a complete answer and also the labeling seems to indicate that you are using the Gauss-Seidel method to solve a linear system but you don't provide details about the problems you are having with it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may be experiencing difficulty with the Gauss-Seidel method. As the grid spacing decreases, there is an increase in the size of the linear system which means that the Gauss-Seidel method may take more iterations before it begins to converge. If you are using a fixed number of iterations, then you might try increasing the number of iterations to make sure that you aren't experiencing problems. Another option is to approximate the error by comparing the solution from the Gauss-Seidel method with the solution using Matlab's backslash operator which solves linear systems also. If they are different then there may be a problem with your Gauss-Seidel solver or you may not be running until you converge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also be having problems with the problem that you are setting up. I suspect that you are modeling the temperature using &lt;a href=&quot;http://en.wikipedia.org/wiki/Poisson%27s_equation&quot;&gt;Poisson's equation&lt;/a&gt; which might be written $-\Delta u = f$. The advantage of this form is that upon discretization, you get a positive definite system (which the Gauss-Seidel method will converge for). If you have a negative sign on the wrong side of your equation, I recommend multiplying so that you have this form. When you discretize this, you should have a $\frac{1}{h^2}$ factor and a matrix with twos on the diagonal (assuming 1D) and -1s above and below the diagonal. If you have the wrong number of $h$ factors appearing in your equation, then you might have the solution grow as $h$ decreases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps you can clarify where your difficulties are and get a better answer. &lt;/p&gt;&#xA;" OwnerUserId="1366" LastActivityDate="2012-04-23T04:46:19.450" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2024" PostTypeId="2" ParentId="2013" CreationDate="2012-04-23T07:45:37.583" Score="4" Body="&lt;p&gt;If you are looking for some implementations (in Fortran) then I can recommend &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/f_src/f_src.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;. There you can find multiple integration rules for different geometries like triangles, simplices, etc. Of course you will also find references there to the original papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classical triangle paper is the one by Dunavant, Symmetrical Gaussian Quadrature Rules, 1985. Where you have rules up to order 20. An implementation can be found &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/f_src/triangle_dunavant_rule/triangle_dunavant_rule.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1176" LastEditorUserId="19791" LastEditDate="2016-12-08T15:02:16.203" LastActivityDate="2016-12-08T15:02:16.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2025" PostTypeId="2" ParentId="2012" CreationDate="2012-04-23T15:29:40.303" Score="5" Body="&lt;p&gt;The cubic Hermite elements have a continuous normal derivative but not full $ C^1 $ continuity.  In particular, the normal derivatives may not match at the boundary of two elements, away from the vertices.  If you want full $ C^1 $ continuity you will have to use the Argyris element or Hsieh-Clough-Tucker or something.  I recommend the discussion in chapter 6 of Ciarlet's finite element book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The degree of polynomial required for $ C^1 $ continuity will depend on your spatial dimension, but in 2D or 3D I don't think you can get away with less than cubic polynomials.  You could consider some kind of nonconforming method which may allow a simpler finite element space.&lt;/p&gt;&#xA;" OwnerUserId="336" LastEditorUserId="336" LastEditDate="2012-06-08T14:51:27.903" LastActivityDate="2012-06-08T14:51:27.903" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2026" PostTypeId="1" CreationDate="2012-04-23T16:20:12.600" Score="29" ViewCount="20579" Body="&lt;p&gt;Which is the fastest library for performing delaunay triangulation of sets with millions if 3D points? Are there also GPU versions available? From the other side, having the voronoi tessellation of the same set of points, would help (in terms of performance) for getting the delaunay triangulation?&lt;/p&gt;&#xA;" OwnerUserId="891" LastEditorUserId="549" LastEditDate="2012-04-23T17:54:18.660" LastActivityDate="2018-03-30T16:36:36.187" Title="Fastest Delaunay triangulation libraries for sets of 3D points" Tags="|computational-geometry|delaunay-triangulation|voronoi-diagrams|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2027" PostTypeId="1" AcceptedAnswerId="2029" CreationDate="2012-04-23T16:25:49.207" Score="9" ViewCount="467" Body="&lt;p&gt;I am doing research on the structure in the Schur complements and find an interesting phenomenon:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that A is from 5--pt laplacian. If I use nested dissection ordering and multifrontal method to compute the LU factorization and then check the last schur complement block, it has low-rank for the off-diagonal blocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, when I use the same method to factorize $A - \lambda I$, where $\lambda$ is some positive value near the eigenvalues of A, then the last schur complement does not have the low-rank property. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know whether the indefinite will change the structure in the schur complement or not. Can anyone provide some reference for this topic?&lt;/p&gt;&#xA;" OwnerUserId="866" LastActivityDate="2012-04-23T17:28:16.197" Title="Rank structure in the Schur complement" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2028" PostTypeId="1" CreationDate="2012-04-23T16:27:15.963" Score="18" ViewCount="5113" Body="&lt;p&gt;When memory bandwidth limited computations are performed in shared memory environments (e.g. threaded via OpenMP, Pthreads, or TBB), there is a dilemma of how to ensure that the memory is correctly distributed across &lt;em&gt;physical&lt;/em&gt; memory, such that each thread mostly accesses memory on a &quot;local&quot; memory bus. Although the interfaces are not portable, most operating systems have ways to set thread affinity (e.g. &lt;code&gt;pthread_setaffinity_np()&lt;/code&gt; on many POSIX systems, &lt;code&gt;sched_setaffinity()&lt;/code&gt; on Linux, &lt;code&gt;SetThreadAffinityMask()&lt;/code&gt; on Windows). There are also libraries such as &lt;a href=&quot;http://www.open-mpi.org/projects/hwloc/&quot;&gt;hwloc&lt;/a&gt; for determining the memory hierarchy, but unfortunately, most operating systems do not yet provide ways to set NUMA memory policies. Linux is a notable exception, with &lt;a href=&quot;http://linux.die.net/man/3/numa&quot;&gt;libnuma&lt;/a&gt; allowing the application to manipulate memory policy and page migration at page granularity (in mainline since 2004, thus widely available). Other operating systems expect users to observe an implicit &quot;first touch&quot; policy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Working with a &quot;first touch&quot; policy means that the caller should create and distribute threads with whatever affinity they plan to use later when first writing to the freshly allocated memory. (Very few systems are configured such that &lt;code&gt;malloc()&lt;/code&gt; actually finds pages, it just promises to find them when they are actually faulted, perhaps by different threads.) This implies that allocation using &lt;code&gt;calloc()&lt;/code&gt; or immediately initializing memory after allocation using &lt;code&gt;memset()&lt;/code&gt; is harmful since it will tend to fault all the memory onto the memory bus of the core running the allocating thread, leading to worst-case memory bandwidth when the memory is accessed from multiple threads. The same applies to the C++ &lt;code&gt;new&lt;/code&gt; operator which insists on initializing many new allocations (e.g. &lt;code&gt;std::complex&lt;/code&gt;). Some observations about this environment:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Allocation can be made &quot;thread collective&quot;, but now allocation becomes mixed into the threading model which is undesirable for libraries which may have to interact with clients using different threading models (perhaps each with their own thread pools).&lt;/li&gt;&#xA;&lt;li&gt;RAII is considered to be an important part of idiomatic C++, but it seems to be actively harmful for memory performance in a NUMA environment. Placement &lt;code&gt;new&lt;/code&gt; can be used with memory allocated via &lt;code&gt;malloc()&lt;/code&gt; or routines from &lt;code&gt;libnuma&lt;/code&gt;, but this changes the allocation process (which I believe is necessary).&lt;/li&gt;&#xA;&lt;li&gt;EDIT: My earlier statement about operator &lt;code&gt;new&lt;/code&gt; was incorrect, it can support multiple arguments, see Chetan's reply. I believe there is still a concern of getting libraries or STL containers to use specified affinity. Multiple fields may be packed and it may be inconvenient to ensure that, e.g., an &lt;code&gt;std::vector&lt;/code&gt; reallocates with the correct context manager active.&lt;/li&gt;&#xA;&lt;li&gt;Each thread can allocate and fault its own private memory, but then indexing into neighboring regions is more complicated. (Consider a sparse matrix-vector product $y \gets A x$ with a row partition of the matrix and vectors; indexing the unowned part of $x$ requires a more complicated data structure when $x$ is not contiguous in virtual memory.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Are any solutions to NUMA allocation/initialization considered idiomatic? Have I left out other critical gotchas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I don't mean for my C++ examples to imply an emphasis on that language, however the C++ &lt;em&gt;language&lt;/em&gt; encodes some decisions about memory management that a language like C does not, thus there tends to be more resistance when suggesting that C++ programmers do those things differently.)&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-04-24T21:45:47.030" LastActivityDate="2013-12-19T10:13:35.893" Title="Portable multicore/NUMA memory allocation/initialization best practices" Tags="|performance|memory-management|multicore|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2029" PostTypeId="2" ParentId="2027" CreationDate="2012-04-23T17:28:16.197" Score="8" Body="&lt;p&gt;Welcome to the wonderful world of Helmholtz equations. Replace $\lambda \ge 0$ with $\omega^2$ and you are describing a factorization of the Helmholtz equation. You may be interested in &lt;a href=&quot;http://www.ma.utexas.edu/users/lexing/publications/sweephmf.pdf&quot;&gt;this paper&lt;/a&gt;, which covers this exact issue. There is also &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/HelmholtzReview.ps.gz&quot;&gt;a nice review paper&lt;/a&gt; which explains why Helmholtz equations are hard.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-04-23T17:28:16.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2030" PostTypeId="2" ParentId="2028" CreationDate="2012-04-23T18:30:26.737" Score="7" Body="&lt;p&gt;One solution to this problem that I tend to prefer is to disaggregate threads and (MPI) tasks at the, effectively, memory controller level. I.e., remove the NUMA aspects from your code by having one task per CPU socket or memory controller and then threads under each task. If you do it that way, then you should be able to bind all memory to that socket/controller safely either via first-touch or one of the available APIs no matter which thread actually does the work of allocation or initialization. Message passing between sockets is usually quite well optimized, in MPI at the very least. You can always have more MPI tasks than this, but due to the issues you raise, I rarely recommend folks have less. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-04-23T18:30:26.737" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2031" PostTypeId="2" ParentId="2008" CreationDate="2012-04-23T19:40:59.500" Score="11" Body="&lt;p&gt;The fundamental challenge of quantum mechanical calculations is that they do not scale very well—from what I recall, the current best-case scaling is approximately $O(N_e^{3.7})$, where $N_e$ is the number of &lt;em&gt;electrons&lt;/em&gt; contained in the system. Thus, 13 water molecules will scale as having $N_e = 104$ electrons instead of just $N = 39$ atoms. (That's a factor of nearly 40.) For heavier atoms, the discrepancy becomes even greater.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main issue will be that, in addition to increased computational horsepower, you will need to come up with better algorithms that can knock down the 3.7 exponent to something that is more manageable.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-04-23T19:40:59.500" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2032" PostTypeId="2" ParentId="2012" CreationDate="2012-04-23T20:06:11.050" Score="3" Body="&lt;p&gt;I refer you to the book &lt;a href=&quot;http://books.google.com.sa/books?id=6hvqGgbBmEoC&amp;amp;dq=Spline%20Functions%20on%20Triangulations&amp;amp;printsec=frontcover&amp;amp;source=bn&amp;amp;hl=en&amp;amp;ei=hg_HSYazN82ktwf_vrTJCg&amp;amp;sa=X&amp;amp;oi=book_result&amp;amp;ct=result&amp;amp;redir_esc=y#v=onepage&amp;amp;q=Spline%20Functions%20on%20Triangulations&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Splines on Triangulations&lt;/a&gt;. I cannot locate my copy at the moment to give you a better answer, but I recall a discussion/theorems on the polynomial order required for $C^1$ spaces. If I recall correctly, Lai proves that under certain conditions $p=3$ is OK, but $p=5$ is always sufficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I also remember that Lai does not then show how to construct $C^1$ spaces, only prove they exist given a triangulation and a spline space. Once he has this proof, he solves his application with additional linear constraint equations to enforce the $C^1$ condition.&lt;/p&gt;&#xA;" OwnerUserId="1370" LastActivityDate="2012-04-23T20:06:11.050" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2034" PostTypeId="1" CreationDate="2012-04-23T21:02:00.123" Score="5" ViewCount="118" Body="&lt;p&gt;I am not an xray crystallographer, but from what I've heard there's often a good deal of guess work and intuition involved in the process of fitting a ball-and-stick molecular model to a crystal-derived electron density map. This guess work tends to introduce a substantial amount of error into the the final set of atomic coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quantum mechanical simulation programs that I have tried (Gaussian, Gamess) use atomic coordinates as their input. Are there any QM methods that directly utilize the information contained in an electron density map, either in conjunction with or instead of atomic coordinates? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there isn't currently a QM software package that can directly incorporate electron density data, would it be theoretically possible to make one? It seems like it would be a good way to eliminate some of the errors introduced durring atomic fitting, and to move the source of the simulation's input one step closer to the actual experimental data. Could electron density data be useful for simulations, or am I just being delusional?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are any structural biologists and/or QM macromolecule simulators in the house, please help me sort this out.&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="34" LastEditDate="2012-04-24T04:44:45.543" LastActivityDate="2012-05-08T12:27:50.207" Title="Are there any QM macromolecule simulation methods that can use an electron density map as input?" Tags="|quantum-mechanics|simulation|computational-biology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2035" PostTypeId="1" AcceptedAnswerId="2037" CreationDate="2012-04-23T21:40:09.453" Score="3" ViewCount="244" Body="&lt;p&gt;I'm writing a petsc code using a sequential matrix type. Since I want it to be easy to parallelize, I put some stuff in the code that is useless now, but will make it easier to parallelize later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the things I added was &lt;code&gt;MatGetOwnershipRange()&lt;/code&gt; to define the range of iterators that I'm using to build a matrix. The matrix is of type MatSeqAIJ, and has not had any values assigned to it, though memory has been allocated with &lt;code&gt;MatSeqAIJSetPreallocation()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run &lt;code&gt;MatGetOwnershipRange()&lt;/code&gt;, though, the two PetscInts I get for the beginning and end of the range are both 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does &lt;code&gt;MatGetOwnershipRange()&lt;/code&gt; only work for MPI matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it only work for matrices that have had their values assigned with &lt;code&gt;MatSetValues()&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-04-24T00:51:06.857" LastActivityDate="2012-04-24T00:51:06.857" Title="What does MatGetOwnershipRange() do for sequential matrices?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2036" PostTypeId="1" AcceptedAnswerId="2054" CreationDate="2012-04-23T21:47:30.113" Score="15" ViewCount="457" Body="&lt;p&gt;The meta seemed to suggest that &lt;a href=&quot;https://scicomp.meta.stackexchange.com/questions/255/are-the-questions-about-the-career-path-as-a-computational-scientist-appropriate&quot;&gt;career advice&lt;/a&gt; is ok . . . so here goes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a couple of close friends in the ML and mathematical modeling fields just finishing PhD's and starting out on the job hunt.  While both are quite qualified and talented, neither are interested in staying in Academia. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides personal referrals and communication, which I have always had the best luck with, are there any resources/websites/aggregators that can help someone looking for jobs in the area?&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="20688" LastEditDate="2019-06-22T19:53:28.080" LastActivityDate="2019-06-22T19:53:28.080" Title="SciComp Modeling Jobs" Tags="|career-development|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2037" PostTypeId="2" ParentId="2035" CreationDate="2012-04-24T00:25:00.260" Score="3" Body="&lt;p&gt;It works for any matrix that has had its size set, returning the ownership range of a &lt;code&gt;Vec&lt;/code&gt; that can be the result of a matrix-vector product. For most matrix types, this matches the row-based storage, though that is not strictly necessary. For sequential matrices, the ownership range is all the rows.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-04-24T00:25:00.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2038" PostTypeId="2" ParentId="2036" CreationDate="2012-04-24T06:23:55.950" Score="9" Body="&lt;p&gt;As an aggregator, I think the best option is to go looking on the all-over &lt;a href=&quot;http://linkedin.com&quot;&gt;LinkedIn&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as a nice solution (at least for me) in finding a mid-way between academia and industry there are also (partly or fully) government funded research centers. In the US you have (among others) ORNL, LANL, LLNL, Sandia,... And if your friends want to dip their toes in some international waters, in Europe every country has one or more of these research centers. And in my case (shameless  free advertisement), that is &lt;a href=&quot;http://www.sckcen.be/en/About-SCK-CEN/Career-opportunities/Vacancies&quot;&gt;SCK-CEN&lt;/a&gt;. Next to regular contracts, we have a &lt;a href=&quot;http://www.sckcen.be/en/Education-Training/Guidance-for-young-researchers&quot;&gt;PhD and PostDoc programme&lt;/a&gt; in different fields of science. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the &lt;a href=&quot;http://www.netlib.org/na-digest-html/&quot;&gt;NA-Digest&lt;/a&gt; usually has some job announcements (but most of the time PhD, PostDoc or professorships at universities).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck to your friends in finding their dream job!&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2012-04-24T09:31:34.497" LastActivityDate="2012-04-24T09:31:34.497" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2039" PostTypeId="2" ParentId="2034" CreationDate="2012-04-24T13:27:03.320" Score="2" Body="&lt;p&gt;I do not think this would be useful and am not aware of it being implemented anywhere, but as they say, absence of evidence is not evidence of absence. I saw a lecture a while back about research by Reimers and coworkers concerning the use of &lt;em&gt;ab initio&lt;/em&gt; methods in the final refinement of XRD fits, in addition to the restrained MD often used in refinement, however this is quite distinct from using the ESD map to prime a calculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In XRD, the electron density map is a proxy for the nuclear coordinates, whereas in QM calculations, the electron density is a function of the nuclear coordinates and can be obtained fairly easily* by convergence from a guess wavefunction. Using the electron density as the sole input for a QM calculation would still require an initial guess at the nuclear coordinates, as without those you do not have a nuclear potential with which to make the QM calculation meaningful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;QM calculations can of course be extremely useful in attempting to elucidate contentious XRD structures, where several interpretations of a structure or structures may exist. The ability to garner extra electronic structure information from a QM calculation means that you can also attempt to reconcile your QM model with other experimental techniques like EPR, NMR, etc. This allows researchers to connect suites of data that would otherwise be independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this is interesting to you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*YMMV!&lt;/p&gt;&#xA;" OwnerUserId="512" LastActivityDate="2012-04-24T13:27:03.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2040" PostTypeId="2" ParentId="2028" CreationDate="2012-04-24T14:05:40.460" Score="6" Body="&lt;p&gt;This answer is in response to two C++ related misconceptions in the question.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&quot;The same applies to the C++ new operator which insists on initializing new allocations (including PODs)&quot;&lt;/li&gt;&#xA;&lt;li&gt;&quot;C++ operator new takes only one parameter&quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It is not a direct answer for multi-core issues you mention.  Just responding to comments that classify C++ programmers as C++ zealots so that the the reputation is maintained ;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To point 1. C++ &quot;new&quot; or stack allocation does not insist on initializing new objects, whether PODs or not.  The class's default constructor, as defined by the user, has that responsibility. The first code below shows junk printed whether the class is POD or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To point 2. C++ allows overloading &quot;new&quot; with multiple arguments.  The second code below shows such a case for allocating single objects.  It should give an idea and perhaps be useful for the situation you have.  operator new[] can be modified appropriately too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;// Code for point 1.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;struct A&#xA;{&#xA;    // int/double/char/etc not inited with 0&#xA;    // with or without this constructor&#xA;    // If present, the class is not POD, else it is.&#xA;    A() { }&#xA;&#xA;    int i;&#xA;    double d;&#xA;    char c[20];&#xA;};&#xA;&#xA;int main()&#xA;{&#xA;    A* a = new A;&#xA;    std::cout &amp;lt;&amp;lt; a-&amp;gt;i &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; a-&amp;gt;d &amp;lt;&amp;lt; '\n';&#xA;    for(int i = 0; i &amp;lt; 20; ++i)&#xA;        std::cout &amp;lt;&amp;lt; (int) a-&amp;gt;c[i] &amp;lt;&amp;lt; '\n';&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Intel's 11.1 compiler shows this output (which is of course&#xA;uninitialized memory pointed by &quot;a&quot;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;993001483 6.50751e+029&#xA;105&#xA;108&#xA;... // skipped&#xA;97&#xA;108&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;// Code for point 2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;cstddef&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;new&amp;gt;&#xA;&#xA;// Just to use two different classes.&#xA;class arena { };&#xA;class policy { };&#xA;&#xA;struct A&#xA;{&#xA;    void* operator new(std::size_t, arena&amp;amp; arena_obj, policy&amp;amp; policy_obj)&#xA;    {&#xA;        std::cout &amp;lt;&amp;lt; &quot;special operator new\n&quot;;&#xA;        return (void*)0x1234; //Just to test&#xA;    }&#xA;};&#xA;&#xA;void* operator new(std::size_t, arena&amp;amp; arena_obj, policy&amp;amp; policy_obj)&#xA;{&#xA;    std::cout &amp;lt;&amp;lt; &quot;special operator new (global)\n&quot;;&#xA;    return (void*)0x5678; //Just to test&#xA;}&#xA;&#xA;int main ()&#xA;{&#xA;    arena arena_obj;&#xA;    policy policy_obj;&#xA;    A* ptr = new(arena_obj, policy_obj) A;&#xA;    int* iptr = new(arena_obj, policy_obj) int;&#xA;    std::cout &amp;lt;&amp;lt; ptr &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;    std::cout &amp;lt;&amp;lt; iptr &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user335" LastActivityDate="2012-04-24T14:05:40.460" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2041" PostTypeId="1" CreationDate="2012-04-24T19:03:08.013" Score="8" ViewCount="5385" Body="&lt;p&gt;Basically the formulation of the problem I'd like to solve is very simple. Given 2 simple polygons (without self-intersections) report all intersecting edge pairs in &lt;em&gt;O(n+k)&lt;/em&gt; time, where n - is a total number of edges, k - number of intersections between two polygons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is very important to stay within mentioned time hardness. What does surprise me, that I could hardly find information on this subject. Polygon intersections seems to be so natural and important problem. Anyway at the moment I don't have a clue if it is even possible to do it in &lt;em&gt;O(n+k)&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can please someone help with lower bounds for this problem?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Some additional points on my question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Polygon clipping with set operations and intersection reporting (about which my question was) seems to be slightly different problems - Having found intersecting edges you would have to constract a polygon which is a result of clipping/set operation.&lt;/li&gt;&#xA;&lt;li&gt;An approach based on segment intersection algorithm won't work for me. As it is proved that worst case takes O(nlogn+k).&lt;/li&gt;&#xA;&lt;li&gt;I'm not insisting that an algorithm with such hardness exist - it may not. But in this case I would really appreciate if someone could provide a lower bound proof for my problem - tons of papers provide some very interesting algorithm (usually with O(nlogn+k) complexity) but for some reason don't mention the lower bound.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1374" LastEditorUserId="9" LastEditDate="2012-04-25T13:24:50.767" LastActivityDate="2020-03-29T00:08:30.610" Title="How to get all intersections between two simple polygons in O(n+k)" Tags="|computational-geometry|constraints|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2042" PostTypeId="2" ParentId="2008" CreationDate="2012-04-24T19:05:31.167" Score="-1" Body="&lt;p&gt;This problem is solved by Density Functinal Theory. The essence is replacing many body degrees of freedom by several fields one of them beeing the density of electrons. For a grand exposition see the nobel lecture of one of the founders of DFT: &lt;a href=&quot;http://www.nobelprize.org/nobel_prizes/chemistry/laureates/1998/kohn-lecture.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.nobelprize.org/nobel_prizes/chemistry/laureates/1998/kohn-lecture.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="897" LastEditorUserId="897" LastEditDate="2012-04-25T05:00:03.523" LastActivityDate="2012-04-25T05:00:03.523" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2043" PostTypeId="2" ParentId="2041" CreationDate="2012-04-24T22:34:39.160" Score="8" Body="&lt;p&gt;The $\mathcal{O}(n+k)$ algorithms seem to be limited to convex polygons. The fastest algorithms that I have found &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0098300499000710&quot; rel=&quot;noreferrer&quot;&gt;  here &lt;/a&gt; and &lt;a href=&quot;ftp://ftp.geoinfo.tuwien.ac.at/frank/4756_Intersection_Nonconvex_Polygons_Using_Alternate_Hierarchical_Decomposition.pdf&quot; rel=&quot;noreferrer&quot;&gt; here&lt;/a&gt; have approximately $\mathcal{O}(n\log(n))$ algorithms.  According to &lt;a href=&quot;http://www.tcs.tifr.res.in/~ghosh/lec-coimbatore-jan2011.pdf&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt;, the problem of simple polygon intersection is linear time&#xA;transformable to line-segment intersection testing, which has an optimal bound of approximately $\mathcal{O}(n\log(n) + k)$ according to &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=147511&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; article.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="183" LastEditDate="2012-04-25T02:16:28.063" LastActivityDate="2012-04-25T02:16:28.063" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2045" PostTypeId="2" ParentId="2008" CreationDate="2012-04-25T13:14:34.493" Score="6" Body="&lt;p&gt;The problem is broadly equivalent to the difference between classical computers and quantum computers. Classical computers work on single values at once, as only one future/history is possible for one deterministic input. However, a quantum computer can operate on every possible input simultaneously, because it can be put in a superposition of all the possible states. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same way, a classical computer has to calculate every property individually, but the quantum system it is simulating has all the laws of the universe to calculate all the properties simultaneously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is exacerbated by the way we have to pass data almost serially through a CPU, or at most a few thousand CPUs. By contrast, the universe has a nearly unlimited set of simultaneous calculations going on at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider as an example 3 electrons in a box. A computer has to pick a timestep (first approximation), and keep recalculating the interactions of each electron with each other electron, via a limited number of CPUs. In reality, the electrons have an unknowable number of real and virtual exchange particles in transit, being absorbed and emitted, as a continuous process. Every particle and point in space has some interaction going on, which would need a computer to simulate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simulation is really the art of choosing your approximations and your algorithms to model the subject as well as possible with the resources you have available. If you want perfection, I'm afraid it's the mathematics of spherical chickens in vacuums; we can only perfectly simulate the very simple.&lt;/p&gt;&#xA;" OwnerUserId="1372" LastActivityDate="2012-04-25T13:14:34.493" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2046" PostTypeId="2" ParentId="2034" CreationDate="2012-04-25T14:05:01.997" Score="1" Body="&lt;p&gt;You could perhaps try &lt;a href=&quot;http://leonardo.inf.um.es/macromol/programs/programs.htm&quot; rel=&quot;nofollow&quot;&gt;Hydromic&lt;/a&gt;, or &lt;a href=&quot;http://chaconlab.org/imodfit/index.html&quot; rel=&quot;nofollow&quot;&gt;iMODFIT&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;HYDROMIC&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Solution properties of macromolecules and macromolecular complexes with a 3D density map, obtained by cryo-electron microscopy or other techniques&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;iMODFIT&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;iMODFIT is an efficient tool for flexible fitting of atomic structures into EM maps based on Normal Mode Analysis in internal Coordinates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="891" LastEditorUserId="149" LastEditDate="2012-05-08T12:27:50.207" LastActivityDate="2012-05-08T12:27:50.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2047" PostTypeId="1" AcceptedAnswerId="2048" CreationDate="2012-04-25T14:46:36.720" Score="16" ViewCount="1955" Body="&lt;p&gt;I've had a glimpse of Numerical Analysis (majorly, Numerical Methods like root finding, quadratic equations and other preliminary stuff) in my Calculus class but now, I find myself wanting more sophistication in my work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good book which will help me understand concepts such as &lt;em&gt;stability of algorithms, designing stable algorithms, error propogation, convergence analysis etc.&lt;/em&gt; from a more general point of view?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, I want to be able to understand and analyze Krylov Subspace Methods (QMR, GMRES and CG) and a few Nonlinear Optimization algorithms better. Especially, how floating point approximation makes a difference to the algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with most books I've seen is that they start off assuming that the reader knows nothing about Linear Algebra and go on into basics of LU, Gaussian Elimination, QR etc. which I don't need. What I want is more of a &quot;bird's eye view&quot; of Numerical Analysis without going into the details of specific methods. Brevity would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="529" LastActivityDate="2014-10-20T14:48:03.430" Title="Book reference for Numerical Analysis" Tags="|reference-request|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2048" PostTypeId="2" ParentId="2047" CreationDate="2012-04-25T15:08:30.690" Score="8" Body="&lt;p&gt;My favorite book on this topic is &lt;a href=&quot;http://www.maths.manchester.ac.uk/~higham/asna/index.php&quot;&gt;Accuracy and Stability of Numerical Algorithms&lt;/a&gt; by Nick Higham. The first few chapters are on general principles of stability, floating point arithmetic etc. Then starting from simple problems (summation, polynomial evaluation), Higham proceeds to the stability analysis of more elaborate numerical methods. I would highly recommend this book, even for the first few chapters. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-04-25T15:08:30.690" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2049" PostTypeId="2" ParentId="2047" CreationDate="2012-04-25T15:15:46.247" Score="6" Body="&lt;p&gt;Quite recently I've discovered Trefethen and Bau's &lt;a href=&quot;http://books.google.by/books/about/Numerical_Linear_Algebra.html?id=bj-Lu6zjWbEC&amp;amp;redir_esc=y&quot; rel=&quot;nofollow&quot;&gt;Numerical Linear Algebra&lt;/a&gt;. I really like the style and it seems to me this book satisfies almost all your criteria.&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="123" LastEditDate="2014-10-20T14:48:03.430" LastActivityDate="2014-10-20T14:48:03.430" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2050" PostTypeId="2" ParentId="952" CreationDate="2012-04-25T15:29:38.290" Score="1" Body="&lt;p&gt;Having skimmed over most of the analysis and looked at the pretty pictures, they do tell a convincing story: the analytical solution does not have any circularity to it, but your numeric solution does; the central region is curved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I would focus squarely on your finite difference scheme. I can't see what's wrong yet, but perhaps some part of the angular dependence of the coordinates is not quite completely taken care of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I doubt it is the boundary condition; both solutions seem to have zero at all the boundaries. It could conceivably be the nature of the condition though, being a derivative rather than value condition.&lt;/p&gt;&#xA;" OwnerUserId="1372" LastEditorUserId="1372" LastEditDate="2012-04-25T16:19:20.723" LastActivityDate="2012-04-25T16:19:20.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2051" PostTypeId="1" CreationDate="2012-04-25T16:01:35.527" Score="8" ViewCount="750" Body="&lt;p&gt;I would like to master Inverse PDE Problems particularly with the use of Finite Elements. My problem is I don't know where to start. Should I begin by reading a book on Inverse Problems or on PDE-constrianed Optimization!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could I get some book recommendations and/or route to follow.&lt;/p&gt;&#xA;" OwnerUserId="1376" LastActivityDate="2013-11-06T18:16:27.213" Title="Recommended Route for Mastering Inverse PDE Problems" Tags="|pde|optimization|finite-element|inverse-problem|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2052" PostTypeId="2" ParentId="2008" CreationDate="2012-04-25T16:20:31.860" Score="16" Body="&lt;p&gt;As far as I'm aware, the most accurate methods for static calculations are Full Configuration Interaction with a fully relativistic four-component Dirac Hamiltonian and a &quot;complete enough&quot; basis set. I'm not an expert in this particular area, but from what I know of the method, solving it using a variational method (rather than a Monte-Carlo based method) scales shockingly badly, since I think the number of Slater determinants you have to include in your matrix scales something like $O(^{n_{orbs}}C_{n_e})$. (There's an article on the computational cost &lt;a href=&quot;http://vergil.chemistry.gatech.edu/notes/ciscale/ciscale.html&quot;&gt;here&lt;/a&gt;.) The related Monte-Carlo methods and methods based off them using &quot;walkers&quot; and networks of determinants can give results more quickly, but as implied above, aren't variational. And are still hideously costly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Approximations currently in practical use just for energies for more than two atoms include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Born Oppenheimer, as you say: this is almost never a problem unless your system involves hydrogen atoms tunneling, or unless you're very near a state crossing/avoided crossing. (See, for example, conical intersections.) Conceptually, there are non-adiabatic methods for the wavefunction/density, including CPMD, and there's also Path-Integral MD which can account for nuclear tunneling effects.&lt;/li&gt;&#xA;&lt;li&gt;Nonrelativistic calculations, and two-component approximations to the Dirac equation: you can get an exact two-component formulation of the Dirac equation, but more practically the Zeroth-Order Regular Approximation (see Lenthe et al, JChemPhys, 1993) or the Douglas-Kroll-Hess Hamiltonian (see Reiher, ComputMolSci, 2012) are commonly used, and often (probably usually) neglecting spin-orbit coupling. &lt;/li&gt;&#xA;&lt;li&gt;Basis sets and LCAO: basis sets aren't perfect, but you can always make them more complete.&lt;/li&gt;&#xA;&lt;li&gt;DFT functionals, which tend to attempt to provide a good enough attempt at the exchange and correlation without the computational cost of the more advanced methods below. (And which come in a few different levels of approximation. LDA is the entry-level one, GGA, metaGGA and including exact exchange go further than that, and including the RPA is still a pretty expensive and new-ish technique as far as I'm aware. There are also functionals which use differing techniques as a function of separation, and some which use vorticity which I think have application in magnetic or aromaticity studies.) (B3LYP, the functional some people love and some people love to hate, is a GGA including a percentage of exact exchange.)&lt;/li&gt;&#xA;&lt;li&gt;Configuration Interaction truncations: CIS, CISD, CISDT, CISD(T), CASSCF, RASSCF, etc. These are all approximations to CI which assume the most important excited determinants are the least excited ones.&lt;/li&gt;&#xA;&lt;li&gt;Multi-reference Configuration Interaction (truncations): Ditto, but with a few different starting reference states.&lt;/li&gt;&#xA;&lt;li&gt;Coupled-Cluster method: I don't pretend to properly understand how this works, but it obtains similar results to Configuration Interaction truncations with the benefit of size-consistency (i.e. $E(H_2) \times 2 = E((H_2)_2$ (at large separation)).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For dynamics, many of the approximations refer to things like the limited size of a tractable system, and practical timestep choice -- it's pretty standard stuff in the numerical time simulation field. There's also temperature maintenance (see Nose-Hoover or Langevin thermostats). This is mostly a set of statistical mechanics problems, though, as I understand it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, if you're physics-minded, you can get a pretty good feel for what's neglected by looking at the formulations and papers about these methods: most commonly used methods will have at least one or two papers that aren't the original specification explaining their formulation and what it includes. Or you can just talk to people who use them. (People who study periodic systems with DFT are always muttering about what different functionals do and don't include and account for.) Very few of the methods have specific surprising omissions or failure modes. The most difficult problem appears to be proper treatment of electron correlation, and anything above the Hartree-Fock method, which doesn't account for it at all, is an attempt to include it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, getting to the accuracy of Full relativistic CI with complete basis sets is &lt;em&gt;never going to be cheap&lt;/em&gt; without dramatically reinventing (or throwing away) the algorithms we currently use. (And for people saying that DFT is the solution to everything, I'm waiting for your pure density orbital-free formulations.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also the issue that the more accurate you make your simulation by including more contributions and more complex formulations, the harder it is to actually do anything with. For example, spin orbit coupling is sometimes avoided solely because it makes everything more complicated to analyse (but sometimes also because it has negligable effect), and the canonical Hartree-Fock or Kohn-Sham orbitals can be pretty useful for understanding qualitative features of a system without layering on the additional output of more advanced methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I hope some of this makes sense, it's probably a bit spotty. And I've probably missed someone's favourite approximation or niggle.)&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2012-04-25T16:20:31.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2053" PostTypeId="2" ParentId="2051" CreationDate="2012-04-25T17:19:54.530" Score="12" Body="&lt;p&gt;I will use parameter estimation problem as an example to briefly introduce what inverse problems are and then list some of my recommendations on learning this topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One famous example would be the &lt;a href=&quot;http://en.wikipedia.org/wiki/Electrical_impedance_tomography&quot; rel=&quot;nofollow&quot;&gt;Calderón Problem&lt;/a&gt;, basically what it concerns is how to use the boundary measurements data to determine the material related parameters of the interior domain. The finite element methods you mentioned are used to solve the &lt;strong&gt;&lt;em&gt;forward problem&lt;/em&gt;&lt;/strong&gt;(!) not the inverse problem, a class of problems can be formulated as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given data $d$ on boundary, the material parameter $\beta$ you would like to estimate, the forward problem is:&#xA;$$\mathcal{A}(\beta) u = f $$&#xA;where the differential operator $\mathcal{A}$ depends on the parameter, also accompanied with some boundary condition(see Remark 1). Forward problem actually &lt;em&gt;can&lt;/em&gt; be ill-posed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now your inverse problem would be using this PDE as a constraint, to solve a functional minimization problem like the following:&#xA;$$ \min_{u,\beta} \mathfrak{F}(u,\beta,d,u_h,\beta_h) $$&#xA;for example, $\mathfrak{F}$ is your objective functional, could be Tikhonov functional, and others.(see Remark 2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The prerequisites to conduct research on this field would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you would like to do some theoretical work on inverse problems, Functional Analysis is a must, because you will use Fréchet derivatives as your daily snacks. Also some PDE theory is needed. For this purpose, reading Lars Hörmander's four-volume series &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3540499377&quot; rel=&quot;nofollow&quot;&gt;The Analysis of Linear Partial Differential Operators&lt;/a&gt; would be more than nice, but it costs too much time. A rather concise but difficult to read book is Victor Isakov's &lt;a href=&quot;http://books.google.com/books?id=K_aNMWE5O38C&quot; rel=&quot;nofollow&quot;&gt;Inverse problems for partial differential equations&lt;/a&gt;, btw despite the fact that Isakov works in a not-so-famous university, he is always considered to be one of the great mathematician working on inverse problems by Gunther Uhlmann, Stanley Osher, etc.(more book recommendation see Remark 3)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since you mentioned finite element, I am guessing you would like to learn this topic from a more numerical point of view, then Stig Larsson's book &lt;a href=&quot;http://books.google.com/books?id=vZGCaLoEqwgC&quot; rel=&quot;nofollow&quot;&gt;Partial Differential Equations with Numerical Methods&lt;/a&gt; is not hard to read even you are not a numerical analyst, it would get you started in finite element methods. For the connection between the finite element methods and inverse problems, there isn't a dedicated book about this that I know of, however there are many research articles on this. For example the current development in inverse problems Gunther Uhlmann edited: &lt;a href=&quot;http://books.google.com/books?id=HJ5vVVrq8sMC&quot; rel=&quot;nofollow&quot;&gt;Inside Out: Invserse Problems and Applications&lt;/a&gt; shall be a neat introduction to some advanced research area.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To sum up, my suggestion would be: Know what exactly the PDE constraint optimization you are gonna do first, do not bother with the lengthy mathematical theory behind it, then go learning the corresponding finite element methods for the forward problem, lastly go back to the inverse problem to see what you need there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;SOME UPDATES&lt;/code&gt;&#xA;Remark 1: If we just have a single boundary measurement, we could treat the data $d$ as the Dirichlet boundary condition for the forward problem; If what we know is a mapping(!), aka Dirichlet-Neumann map, ie given input electromagnetic wave of certain frequency, we could get certain feedback on the boundary, then the boundary data $d$ is not the boundary condition of the forward problem, but rather that Dirichlet-Neumann map, working like a black-box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remark 2: A potential difficulty here, due to the property of high frequency waves, the objective functional may have lots and lots of local minima, and this problem is still an open research area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remark 3: I consulted my officemate today who is working on Dirichlet-Neumann mapping, and he recommended two more books, the first one is the yet to be published &lt;a href=&quot;http://www.math.ubc.ca/~feldman/ibook/&quot; rel=&quot;nofollow&quot;&gt;lecture notes by Gunter Uhlmann and Joel Feldman&lt;/a&gt;, this is a more user-friendly read than Isakov's. Another book is by Colton and Kress, &lt;a href=&quot;http://www.springer.com/mathematics/dynamical+systems/book/978-3-540-62838-5&quot; rel=&quot;nofollow&quot;&gt;Inverse Acoustic and Electromagnetic Scattering Theory&lt;/a&gt;, which is even more user-friendly assuming you know some PDE and/or functional analysis and/or finite element. &lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-04-26T21:55:09.063" LastActivityDate="2012-04-26T21:55:09.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2054" PostTypeId="2" ParentId="2036" CreationDate="2012-04-25T18:08:19.507" Score="10" Body="&lt;p&gt;From my own experience, I found that it is very important to develop a network of contacts while you are working on your own projects, and not leave it to the very last moment. This means to actively meet other people at every chance you have, openly share your ideas, and expose yourself to other people ideas. So take every opportunity to meet visiting professors, postdocs, and other students.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that attending and actively participating at conferences is the most efficient way to develop a personal network of contacts. However, this requires quite a bit of effort, as besides of delivering your own talk you need to effectively interact with lots of people (this is different from just having a casual conversations and can be quite exhausting if you do this non-stop for a whole week). Going for the after-conference dinner and drinks is probably one of the most important parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that socialising at conferences is one of the most fun and effective ways to meet future collaborators and even land a job (in the long term).&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-04-25T18:08:19.507" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2055" PostTypeId="2" ParentId="2047" CreationDate="2012-04-25T18:34:14.013" Score="5" Body="&lt;p&gt;With respect to floating point arithmetic, I think a good starting point is the paper of D. Golberg &lt;a href=&quot;http://www-users.math.umd.edu/~jkolesar/mait613/floating_point_math.pdf&quot;&gt;&quot;What every computer scientist should know about floating-point arithmetic&quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other fun books to read, besides the already suggested, are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;Matrix computations&quot; by Golub, and van Loan.&lt;/li&gt;&#xA;&lt;li&gt;&quot;Numerical methods that usually work&quot; by Acton.&lt;/li&gt;&#xA;&lt;li&gt;&quot;The art of computer programming&quot; by Knuth.&lt;/li&gt;&#xA;&lt;li&gt;&quot;Domain decomposition: parallel multilevel methods for elliptic partial differential equations&quot; by Smith, Bjørstad, Gropp.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Every book has remarkable chapters but how good is a book at helping to develop  the reader's understanding of a topic depends on the reader background and interests. I found these books useful for my work and I recommend you to have a look at them at the library.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-04-25T18:34:14.013" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2056" PostTypeId="2" ParentId="2008" CreationDate="2012-04-25T19:21:06.283" Score="2" Body="&lt;p&gt;One problem is that quantum mechanics suffers from the &quot;curse of dimensionality&quot;: for most methods of solving PDEs, the number of basis functions needed to get a certain accuracy scales exponentially with the number of dimensions.  Since an $n$-electron atom has $3n$ degrees of freedom, even relatively small systems require an enormous number of dimensions to simulate exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Monte Carlo can be used to get around this problem, as the error scales like $\text{points}^{-\frac{1}{2}}$ regardless of the number of dimensions, but convergence is slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Density functional theory is another way to deal with this problem, but it's an approximation. It's a very good approximation in some cases, but in other cases it can be surprisingly bad.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-04-25T19:21:06.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2057" PostTypeId="1" AcceptedAnswerId="2061" CreationDate="2012-04-25T19:45:01.607" Score="11" ViewCount="214" Body="&lt;p&gt;Suppose A is a general sparse matrix, and I want to compute the eigenvalues. I do not know how to detect the multiplicity for the eigenvalues. As far as I know, for a special case, finding the polynomial roots by companion matrix method, we can apply RRQR to detect the multiplicity for the roots.&lt;/p&gt;&#xA;" OwnerUserId="866" LastActivityDate="2012-04-26T11:29:31.373" Title="How to detect the multiplicity for the eigenvalues?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2058" PostTypeId="2" ParentId="1953" CreationDate="2012-04-26T05:44:30.390" Score="2" Body="&lt;p&gt;If your space $V$ is a Hilbert space, then Riesz theorem says that you can represent $F(v)=\left&amp;lt; f, v \right&amp;gt;$ and you can compute $f$ as you mention by trying out unit vectors. If the space is higher dimensional, then this becomes impractical, but you can at least compute estimates of $f$ by computing $F(v)$ for a sequence of random vectors $v$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-04-26T05:44:30.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2059" PostTypeId="1" CreationDate="2012-04-26T08:46:16.250" Score="4" ViewCount="3364" Body="&lt;p&gt;I have a similarity matrix which is symmetric and sparse. How can I parallelize the computation of the eigenvalues of this matrix in MATLAB?&lt;/p&gt;&#xA;" OwnerUserId="1381" LastEditorUserId="9" LastEditDate="2012-04-29T11:33:47.407" LastActivityDate="2012-04-29T11:33:47.407" Title="How to parallelize the computation of eigenvalues of a sparse symmetric matrix in MATLAB?" Tags="|eigensystem|matlab|sparse-matrix|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2060" PostTypeId="1" AcceptedAnswerId="2066" CreationDate="2012-04-26T11:17:09.677" Score="3" ViewCount="1239" Body="&lt;p&gt;I have a class of matrices $A$ which are created by a domain decomposition method. Each matrix represents several subproblems of equal size, and I know that for some permutation matrix $P$,&#xA;$PAP^T$ will be a block diagonal system, i.e.&#xA;$$&#xD;&#xA;A = &#xD;&#xA;\begin{bmatrix}&#xD;&#xA;A_1 &amp;amp; 0 &amp;amp; 0 \\&#xD;&#xA;0 &amp;amp; A_2 &amp;amp; 0 \\&#xD;&#xA;0 &amp;amp; 0 &amp;amp; A_3 \\&#xD;&#xA;\end{bmatrix}&#xD;&#xA;$$&#xA;Solving this system for some right hand side is obviously just a matter of inverting each block and permuting back to he original ordering. The question is if there are any methods for such systems when the ordering required to permute the system into block diagonal form is not known.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;One approach would be to interpret the matrix system as a graph, and try to find cycles. The system can then be permuted by ordering according to each cycle. The question is then what algorithms are fast and suitable for the purpose?&lt;/li&gt;&#xA;&lt;li&gt;The condition number of such a system is often much lower than that of an similar system where the blocks are connected because the eigenvalues are that of each subblock. A question of general interest could be what algorithms are good at solving such systems without actually permuting the system - CG would probably converge faster for a more decoupled system because of the lowered condition number. Are some algorithms better for this type of problem?&lt;/li&gt;&#xA;&lt;li&gt;Is there a known nomenclature for such systems? When searching for inversion/permutation of block diagonal matrices, what I usually find is systems which are block diagonal except for the last row - a different problem, really.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;While I have no problems solving each system, I thought this question was interesting. The submatrices can be thought of as similar to the five point stencil for the heat equation in terms of matrix properties - they arise from a similar mass conservation system. (I hope this is of general interest, so focusing on the structure of the submatrices is not very important)&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2012-05-03T01:22:48.987" Title="Solving sparse matrix systems which can be reordered to block diagonal form" Tags="|linear-algebra|sparse-matrix|block-decomposition|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2061" PostTypeId="2" ParentId="2057" CreationDate="2012-04-26T11:29:31.373" Score="9" Body="&lt;p&gt;Strictly speaking, the problem of computing multiplicities is ill-posed, as arbitrarily small perturbations may change the multiplicities (usually reducing them to 1). However, to some approximation, the following works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a close eigenvalue approximation $\sigma$ and can afford to factor $A-\sigma I$ then you can apply a subspace method with the matrix $B=(A-\sigma I)^{-1}$ to find the eigenspace of eigenvalues close to $\sigma$. Projecting to an orthonormal basis of that space and computing the Schur decomposition then gives the numerical decomposition into eigenspaces and their multiplicities, as far a a numerical method can determine them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you cannot a afford a single factorization, one can do similar things with a direct subspace method, but with much poorer resolution.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-26T11:29:31.373" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2062" PostTypeId="2" ParentId="2059" CreationDate="2012-04-26T17:45:54.587" Score="2" Body="&lt;p&gt;If you only need the largest and smallest eigenvalues, you can use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Power_iteration&quot; rel=&quot;nofollow&quot;&gt; power&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Inverse_iteration&quot; rel=&quot;nofollow&quot;&gt;inverse power&lt;/a&gt; methods, respectively.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The power method requires a matrix-vector product at every iteration, which can be executed in parallel.  If you are using a distributed computer, you can explicity allocate the vector and a certain number of rows of the matrix to each processor and calculate a portion of the resulting vector (this is called row-wise decomposition).  These partial results can be combined into a single result using an all-gather operation.  Alternatively, if you are using a symmetric computer, Matlab's Parallel Computing Toolbox has a parallel-for loop operation which can allow you do do this wihout altering your sequential code too much.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inverse method requires a linear system of equations to be solved. Again, you can implement virtually any linear system solver method that you want in parallel.  If you choose to use the conjugate gradient method, it too requires a matrix-vector product at every iteration, which can be executed in parallel using Matlab's PCT's parallel for-loop operation.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-04-28T19:14:52.823" LastActivityDate="2012-04-28T19:14:52.823" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2063" PostTypeId="1" CreationDate="2012-04-26T17:56:59.143" Score="0" ViewCount="288" Body="&lt;p&gt;I have a job which stops unexpectedly with Exit_status=15. What is the most probable error ?&lt;/p&gt;&#xA;" OwnerUserId="1382" LastActivityDate="2012-04-28T21:38:23.380" Title="Exit_status=15 , what does it mean?" Tags="|hpc|pbs|" AnswerCount="1" CommentCount="2" ClosedDate="2012-04-28T21:47:17.970" ContentLicense="CC BY-SA 3.0" />
  <row Id="2064" PostTypeId="1" AcceptedAnswerId="2065" CreationDate="2012-04-26T19:49:24.870" Score="9" ViewCount="1186" Body="&lt;p&gt;As far as I understand, the order in which non-blocking point-to-point MPI messages (Isend and Irecv) are received is consistent with the order in which they are sent.  Are there any techniques for giving certain messages priority over others?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we have a multi-level algorithm in which the high resolution solutions are sent with non-blocking calls and computations on the coarse levels are done while the fine messages are being sent.  However, when it comes time to send the low resolution solutions, we'd like those to take priority (they are essentially blocking).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can also imagine that this might be useful for other algorithms as we move to the exascale: some messages are in the &quot;critical path&quot; while others are not.&lt;/p&gt;&#xA;" OwnerUserId="486" LastActivityDate="2013-04-13T19:31:51.633" Title="Can MPI messages be prioritized?" Tags="|mpi|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2065" PostTypeId="2" ParentId="2064" CreationDate="2012-04-26T20:10:13.237" Score="12" Body="&lt;p&gt;I think the answer to this is no. Once you've pushed them into the MPI stack, they're out of your control, and the MPI semantics govern the way the messages are sent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could certainly prioritize messages by queuing them in your code before sending them, and then checking frequently which are the most important to send. But I'm not at all convinced that you'll get any benefit. Is there evidence that your fine messages are not complete when you get ready to send the coarse ones? If there isn't, then you might want to investigate whether it is necessary at all in the first place.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-04-26T20:10:13.237" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2066" PostTypeId="2" ParentId="2060" CreationDate="2012-04-27T04:32:08.857" Score="5" Body="&lt;p&gt;&lt;code&gt;Q1&lt;/code&gt; To find the individual blocks in the matrix (and hence recover the block permutation) you could find the connected components in the underlying graph. This can be done efficiently ( in &lt;code&gt;O(|E|)&lt;/code&gt; for a graph with &lt;code&gt;E&lt;/code&gt; edges) via repeated breadth-first-searches from unvisited vertices:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i = all vertices in graph G)&#xA;    if (vertex i is unvisited)&#xA;        // start a BFS from vertex i&#xA;        // all of the vertices visited during this BFS will be &#xA;        // part of the same diagonal block&#xA;    endif&#xA;endfor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this only identifies the block structure in the matrix - there are still a whole family of perumtations that can be applied locally within each block. Depending on your choice of linear solver you would then choose an appropriate ordering scheme (&lt;code&gt;AMD&lt;/code&gt;, &lt;code&gt;RCM&lt;/code&gt;, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Q2&lt;/code&gt; I don't understand what you're getting at here - since the diagonal blocks are fully decoupled from each other I don't see why you wouldn't just solve them individually - in parallel if you're interested in speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Q3&lt;/code&gt; Block diagonal is the correct name - as far as I know.&lt;/p&gt;&#xA;" OwnerUserId="1139" LastActivityDate="2012-04-27T04:32:08.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2067" PostTypeId="1" AcceptedAnswerId="2075" CreationDate="2012-04-27T09:20:44.100" Score="11" ViewCount="1016" Body="&lt;p&gt;Given the system $$Ax=b,$$ where $A\in\mathbb{R}^{n\times n}$, I read that, in case Jacobi iteration is used as a solver, the method will not converge if $b$ has a non-zero component in the null-space of $A$.  So, how could one formally state that, provided that $b$ has a non-zero component spanning the null-space of $A$, Jacobi method is non-convergent? I wonder how could that be mathematically formalized, since part of the solution orthogonal to the null-space does converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, by projecting the null-space of $A$ out of each iterate, it converges (or?). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;.........&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm particularly interested in the case of $$Lx=b,$$&#xA;where $L$ is a symmetric Laplacian matrix with the null-space spanned by a vector $1_n=[1\dots 1]^T\in\mathbb{R}^n$, and $b$ has a &lt;em&gt;zero&lt;/em&gt; component in the null-space of $L$, $$Jb=b,$$ where $J=I-\frac{1}{n}1_n1_n^T$ is the centering matrix. Does that imply that each Jacobi iterate will have the null-space of $L$ projected out, ie., each iterate will be &lt;em&gt;centered&lt;/em&gt;? I'm asking this since then there would be no need to project out the null-space of $L$ from Jacobi iterates (or, in other words, to &lt;em&gt;center&lt;/em&gt; the iterates).&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-04-30T12:03:10.613" LastActivityDate="2012-04-30T12:03:10.613" Title="Projecting out the null-space of $A$ from $b$ in $Ax=b$" Tags="|linear-algebra|optimization|matrix|iterative-method|linear-solver|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2068" PostTypeId="1" AcceptedAnswerId="2069" CreationDate="2012-04-27T16:40:58.817" Score="4" ViewCount="247" Body="&lt;p&gt;For a general matrix A, there are many properties to describe it: symmetric positive definite or indefinite, condition number, spectrum and so on. I am curious about how these properties affect the solving of matrix A for both direct method and iterative method like multigrid, krylov subspace methods. Thanks. &lt;/p&gt;&#xA;" OwnerUserId="866" LastEditorUserId="549" LastEditDate="2012-04-27T19:10:56.133" LastActivityDate="2012-04-27T19:45:39.787" Title="How do the properties of a matrix affect the linear system solving" Tags="|linear-algebra|linear-solver|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2069" PostTypeId="2" ParentId="2068" CreationDate="2012-04-27T17:46:34.117" Score="2" Body="&lt;p&gt;That's a rather big question :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would probably advise you to look up a book on numerical linear algebra - for example Iterative Methods for Sparse Linear Systems by Y. Saad. This is a big and complex subject, with a lot of different quirks to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, however, low condition numbers is good, because it means that the solution is less sensitive to changes in the right hand side of the system. Positive definiteness is important for a multitude of reasons (and some methods, for example Conjugate Gradient won't work without it). The more you know about your matrix in terms of symmetry, structure and properties, the easier it is to find a linear solver, and more importantly, know how good your results will be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sorry if this answer isn't what you hoped for, but whole books could be written on the subject. Perhaps you could narrow the question down a bit - are you solving a particular problem, or are you just out to learn numerical linear algebra?&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2012-04-27T17:46:34.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2070" PostTypeId="2" ParentId="2068" CreationDate="2012-04-27T19:45:39.787" Score="1" Body="&lt;p&gt;To add a bit on the direct solver side, matrix symmetry can be exploited to save you a factor of 2 in memory (regardless of definiteness). Symmetric positive definite is even better, because you don't even need to pivot and can use a BLAS3/GEMM based cholesky decomposition. Symmetric indefinite matrices still save the memory, but should be factored using LDL, which requires 2x2 pivoting to be stable. That yields just BLAS2 speed (I think you can block these algorithms for BLAS3, but you end up with weaker stability guarantees).&lt;/p&gt;&#xA;" OwnerUserId="1388" LastActivityDate="2012-04-27T19:45:39.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2071" PostTypeId="2" ParentId="2063" CreationDate="2012-04-28T21:38:23.380" Score="3" Body="&lt;p&gt;As mentioned by @fcruz, that is a SIGTERM.  i.e the kernel asked your program to stop for some reason.&lt;br&gt;&#xA;   SIGTERM can come for a whole lot of reasons.  Have you tried instrumenting your code to see where you are at when the error crops up?  I most often get that when I blow my memory ceiling.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-04-28T21:38:23.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2072" PostTypeId="2" ParentId="2059" CreationDate="2012-04-29T11:27:58.063" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/support/solutions/en/data/1-9O7RYJ/index.html?solution=1-9O7RYJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;MathWorks doesn't think this is a good idea&lt;/a&gt;.  Their basic defense for not multithreading &lt;code&gt;eigs&lt;/code&gt; is that the MATLAB sparse matrix-vector products will not significantly benefit from multithreading.  I recommend that you look into some of the libraries discussed in &lt;a href=&quot;https://scicomp.stackexchange.com/a/1279/9&quot;&gt;another answer on scicomp&lt;/a&gt; if you are interested in computing eigenvalues more efficiently.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-04-29T11:27:58.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2073" PostTypeId="1" AcceptedAnswerId="2074" CreationDate="2012-04-29T13:51:04.510" Score="8" ViewCount="178" Body="&lt;p&gt;I have a Hermitian matrix $\mathbf{H}$ which depends on two parameters say $x$ and $y$. &#xA;When I diagonalize it at two close points $(x_1,y_1)$ and $(x_2,y_2)$ I get two close eigenvalues ($\varepsilon_1$ and $\varepsilon_2$) and two corresponding eigenspaces ($S_1$ and $S_2$) of the same dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that they are not eigenvalues of the same matrix. There are two different matrices: $\mathbf{H}_1=\mathbf{H}(x_1,y_1)$ and $\mathbf{H}_2=\mathbf{H}(x_2,y_2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a mesh of points $(x_i,y_i)$ and want to find the eigenvalue and the eigenspace at any point using interpolation. The problem is that since the matrices are diagonalized numerically the bases of $S_1$ and $S_2$ are completely independent. Even if $(x_1,y_1)$ and $(x_2,y_2)$ are very close the basis vectors can have very different components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For interpolation I need a basis that depends on $x$ and $y$ continuously, i.e. the closer the eigenspaces $S_1$ and $S_2$ the closer should be the basis vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $S_1$ and $S_2$ are plains in 3-dimensional Euclidean space then a good way to select a basis in S2 is to rotate the basis of S1 around the line which is the intersection of the plains. Is there something analogous to this in complex multidimensional space?&lt;/p&gt;&#xA;" OwnerUserId="1025" LastActivityDate="2012-04-30T11:18:23.363" Title="Eigenspace basis continuously depending on parameters" Tags="|eigensystem|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2074" PostTypeId="2" ParentId="2073" CreationDate="2012-04-29T15:56:56.593" Score="8" Body="&lt;p&gt;For simplicity, assume that there is only one parameter $t$ rather than your two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order that you can have continuous eigenspaces, you need to assume that the associated eigenvalues do not nearly cross. (For nearly crossing eigenvalues it may very well happen that the eigenspaces are essentially exchanged though the eigenvalue curves do not touch. This happens regularly in symmetric parametric eigenvalue problems and is called the phenomenon of avoided crossings.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the eigenspace of $H(t)$ to the continuous eigenvalue $\lambda(t)$ is the column space of $Q_k$ when $t=t_k$, you need to assume a form $Q(t)=Q_0+\sum_j \Phi_j(t) Z_j$ with fixed basis functions $\Phi_j(t)$ (e.g., B-splines) and $Q_0$ being one of the $Q_k$, and then fit the coefficient matrices $Z_j$ and (low-dimensional eigenspace transforms $Y_k$ to the approximate equation $Q_k Y_k - \sum_j \Phi_j(t_k) Z_j \approx Q_0$. This is a linear least squares problem&#xA;that is easy to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With 2 parameters, replace the B-splines by FEM shape functions. Your least squares problem might now become large and suitable additional trickery is needed to make the problem solvable if the direct solution is infeasible.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-04-30T11:18:23.363" LastActivityDate="2012-04-30T11:18:23.363" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2075" PostTypeId="2" ParentId="2067" CreationDate="2012-04-29T16:05:06.507" Score="7" Body="&lt;p&gt;The correct condition for solvability has nothing to do with the null space of $A$ (unless $A$ is symmetric) but with the null space of $A^T$. If $A^Tu=0$ then $Ax=b$ implies that $u^Tb=u^TAx=0$, hence $b$ must be orthogonal to any null vector of $A^T$ (otherwise there is no solution, and the Jacobi iteration has no reason to converge).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if this is the case, a solution exists, and in the square case there are infinitely many. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the singular case, as one never knows whether this condition is satisfied (and it would be spoiled by roundoff anyway), one would typically solve the problem as a least squares problem. To find the minimum norm solution, use conjugate gradients on the normal equations; this requires that you code multiplication by $A$ and by $A^T$. (Given only a routine for multiplying with $A$, one could use GMRES instead, with less predictable convergence properties.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-29T16:05:06.507" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2076" PostTypeId="1" CreationDate="2012-04-30T03:10:02.437" Score="10" ViewCount="152" Body="&lt;p&gt;I'm trying to match items. Given a set of $n$ items I can rank on a scale from 0 to 100 of how similar they are to one another. For instance, if item $n_1$ is milk and item $n_2$ is also milk, then the similarity between $n_1$ and $n_2$ would be 100%. If $n_3$ is soda, then the similarity between $n_1$ and $n_3$ would be, perhaps, 80%, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to figure out how to group the $n$ items into groups of similar items, but it's a challenging problem. I run into the following issue: A horse is similar to a cow, which is similar to a goat, which is similar in part to goat cheese, which is similar to cheese, which is correlated with crackers. However, I wouldn't expect a horse to be in the same group as crackers. Each item might be pair-wise correlated, but the beginning and end might not be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="1392" LastEditorUserId="739" LastEditDate="2012-05-01T04:37:02.563" LastActivityDate="2012-05-01T04:37:02.563" Title="Matching Similar Items from a Set" Tags="|regression|data-analysis|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2077" PostTypeId="2" ParentId="2076" CreationDate="2012-04-30T06:38:58.740" Score="5" Body="&lt;p&gt;I think what you are looking for is called &quot;cluster analysis&quot; or &quot;clustering&quot;. Many different algorithms exist. In your case, you would want some &quot;connectivity clustering&quot;, i.e. group elements together based on a property that links each two. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at the &lt;a href=&quot;http://scikit-learn.org/dev/modules/clustering.html&quot;&gt;clustering algorithms in scikits.learn&lt;/a&gt; (Python code) and the references mentioned there.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-04-30T06:38:58.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2078" PostTypeId="2" ParentId="2076" CreationDate="2012-04-30T06:56:30.963" Score="7" Body="&lt;p&gt;What you want to do here is partition N observations into K clusters who exhibit similar properties. This is called &lt;strong&gt;clustering&lt;/strong&gt; and you can find more info &lt;a href=&quot;http://en.wikipedia.org/wiki/Cluster_analysis&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you already have a numerical similarity measure, this makes me think about using the &lt;a href=&quot;http://en.wikipedia.org/wiki/K-means_clustering&quot;&gt;K-Means&lt;/a&gt; algorithm, in which you operate in several steps:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Initialize cluster centroids randomly&lt;/li&gt;&#xA;&lt;li&gt;Assign each observation to the cluster corresponding to the closest centroid.&lt;/li&gt;&#xA;&lt;li&gt;Update the centroids as the new mean of the elements in the cluster.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can check for convergence when the centroids have stopped moving or within a certain threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would ensure that the items in each cluster are somewhat correlated, and you can have more fine-grained clusters by increasing the number of clusters in the algorithm (the &quot;K&quot;). Finding the number of clusters depends on each problem, and I advise you try a bunch of values for your problems, look at what comes out grouped together, and see what makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="1380" LastActivityDate="2012-04-30T06:56:30.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2079" PostTypeId="2" ParentId="2008" CreationDate="2012-04-30T07:24:05.610" Score="2" Body="&lt;p&gt;I think a highly-accurate simulation of water was the topic of one of the very first and large simulations performed using the &lt;a href=&quot;http://www.nccs.gov/jaguar/&quot; rel=&quot;nofollow&quot;&gt;Jaguar supercomputer&lt;/a&gt;. You might want to look into this paper and their follow-up work (which, by the way, was a finalist for the &lt;a href=&quot;http://awards.acm.org/homepage.cfm?awd=160&quot; rel=&quot;nofollow&quot;&gt;Gordon-Bell&lt;/a&gt; prize in 2009):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://secure3.computer.org/csdl/proceedings/sc/2009/8744/00/87440066-abs.html&quot; rel=&quot;nofollow&quot;&gt;&quot;Liquid water: obtaining the right answer for the right reasons&quot;&lt;/a&gt;, Aprà, Rendell, Harrison, Tipparaju, deJong, Xantheas.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="1070" LastEditDate="2012-04-30T08:37:04.427" LastActivityDate="2012-04-30T08:37:04.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2080" PostTypeId="1" AcceptedAnswerId="2083" CreationDate="2012-04-30T15:16:11.800" Score="7" ViewCount="456" Body="&lt;p&gt;For computing the gradient of a scalar field, one can use the weighted least squares method as described in the paper &lt;a href=&quot;http://www.google.de/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CCwQFjAA&amp;amp;url=http://ns1.nianet.org/technicalreports/pdfs/2003/2003-06.pdf&amp;amp;ei=aaueT7jIB4Wv8gODibT0Dg&amp;amp;usg=AFQjCNHL5Tz5dxebdStJt3NqzUc30lmbwA&amp;amp;sig2=NHq9hkwHGc2wf90jmi0djg&quot; rel=&quot;nofollow&quot;&gt;Revisiting the Least-squares Procedure for Gradient&#xA;Reconstruction on Unstructured Meshes&lt;/a&gt; by Dimitri Mavriplis (pg. 23).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: &lt;strong&gt;How can I reconstruct a gradient of a vector field?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;http://www.google.de/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CC8QFjAA&amp;amp;url=http://www.ann-geophys.net/25/971/2007/angeo-25-971-2007.pdf&amp;amp;ei=zaueT7bANIHd8AOq8LiDDw&amp;amp;usg=AFQjCNFZ1Xqg3oST4UC5zEkM9RfPuqGRhw&amp;amp;sig2=8aXJloAidDVhufrdisDo5g&quot; rel=&quot;nofollow&quot;&gt;Least-squares gradient calculation from multi-point observations of scalar and vector fields: methodology and applications with Cluster in the plasmasphere&lt;/a&gt; by J. De Keyser, et al., it seems that one could consider the vector field components as scalar fields and feed this to the gradient calculation, under some assumptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm writing currently a generic C++ method that is supposed to compute this, where the result rank is determined by outer product trait classes defined for all combinations available (scalar vector, vector-tensor, tensor-vector, vector-vector, etc). Any thoughts on how to approach this?&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="739" LastEditDate="2012-05-01T04:37:52.140" LastActivityDate="2012-05-01T04:37:52.140" Title="Generalized least squares gradient of a vector field" Tags="|c++|visualization|data-sets|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2081" PostTypeId="1" AcceptedAnswerId="2082" CreationDate="2012-04-30T15:22:17.490" Score="12" ViewCount="1308" Body="&lt;p&gt;I am new to programming and I am trying to solve my first big problem and write my first big program. I have looked for open source examples of code to learn from, but so far have only found code in languages I don't fully understand or that does related but still-too-distant things for me to really learn from. I am having trouble taking some conceptual steps here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want make a simple piece of software to build, modify, and later represent small organic molecules. This is mainly a learning exercise. A user would give a SMILES string or choose from a basic set of starter molecules, and then could build on that molecule either graphically or via a text input syntax. However, I am not even at that point of complexity yet. I can't even fully comprehend how to make classes/object to store the molecules. So, my question succinctly: How do I use classes/objects to build molecules while retaining all levels of information, and what data structures should I use as attributes to which objects? And, can objects be attributes to other objects?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my train of thought so far: I was thinking to have a &quot;Molecule&quot; class, then a &quot;Atom&quot; class/subclass and a &quot;Bond&quot; subclass, and maybe a &quot;FunctionalGroup&quot; subclass also. That seems like a good place to start, but maybe I'm misunderstanding OOP and this is bad. But then my problem really gets confusing (for me). Even though I have all these concepts/ideas/classes, I don't fully grasp what datastructures are needed to represent the molecule. A list of atoms would be a good thing to have. Could that list be a list of Atom objects? I would also need to have a way to store connectivity. A 2D matrix seems like a good idea, with bond order as integers in the matrix positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point I'm starting to get overwhelmed with the task. Is everything I'm doing so far make sense? Attaching a display/drawing aspect on top of this may mean that I need to rewrite/rework a lot of these things, but I'm just trying to get to a point where I can at least store molecules with the relevant data and then access that data to check/modify. I was thinking of doing this in Python, so the code/classes would perhaps look something like this: &lt;a href=&quot;http://pastebin.com/uUi1BMzr&quot;&gt;http://pastebin.com/uUi1BMzr&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps this is really a programming questions for StackOverflow, but I thought it was specific enough to go here. Any assistance would be greatly appreciated, even if you just point out where I have made conceptual blunders. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="1397" LastActivityDate="2012-07-07T20:58:01.773" Title="Making a Molecular editor/visualizer: Object oriented programming, data structures, and molecules" Tags="|computational-chemistry|python|visualization|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2082" PostTypeId="2" ParentId="2081" CreationDate="2012-04-30T16:00:33.090" Score="5" Body="&lt;p&gt;writing software is an iterative process - write code -&gt; see how far you can get then plan next steps -&gt; write code -&gt; repeat.   At this stage in learning the art I'd suggest you jump in soon just to test the waters.  No need to plan the entire system up front.  Yes python would be a good first language.   For visualization try out MatPlotLib, also NumPy and SciPy are handy.  Industrial scale software always relies on pulling in pre-build libraries, instead of writing everything yourself, yet simple solutions can and should be written yourself, especially when learning to program. Your OO layout seems OK for now.  When/if your object relationships need to change later, the act of re-factoring your code itself is a worthy experience to master.  Welcome aboard !&lt;/p&gt;&#xA;" OwnerUserId="1399" LastActivityDate="2012-04-30T16:00:33.090" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2083" PostTypeId="2" ParentId="2080" CreationDate="2012-04-30T16:44:27.407" Score="6" Body="&lt;p&gt;No assumption is needed to reduce the vector least squares problem to scalar least squares problems. But you can take advantage of the fact that the matrix is the same for all vector components, so you can solve a single least squares problem with multiple right hand sides, saving factorizations.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-04-30T16:44:27.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2085" PostTypeId="1" AcceptedAnswerId="2093" CreationDate="2012-05-01T08:17:16.683" Score="4" ViewCount="166" Body="&lt;p&gt;Suppose two Gram matrices are given $A, B\in\mathbb{R}^{n\times n}$, such that $$A=XX^T,~~~~~~~~~~~~~B=YY^T,$$ for some $X, Y\in\mathbb{R}^{n\times k}$, $k\ll n$. Also, suppose a Gram matrix based on linear interpolation $X+Y$ is denoted by $C=(X+Y)(X+Y)^T$. I'm aware that $$C=A+B+XY^T+YX^T,$$ but I wonder how I could (approximately) get $C$ by only operating on $A$ and $B$. Clearly, by decomposing to $X$ and $Y$ it is possible, but I'm interested in restricted operation on $A$ and $B$ only. So, could one approximate $XY^T$ and $YX^T$ by $A$ and $B$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know of some academic work exploring the above relation, please post it.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="127" LastEditDate="2012-07-22T15:48:11.470" LastActivityDate="2012-07-22T15:48:41.517" Title="Simplifying some operations on Gram matrices" Tags="|linear-algebra|optimization|matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2086" PostTypeId="1" AcceptedAnswerId="2087" CreationDate="2012-05-01T10:53:06.203" Score="20" ViewCount="5748" Body="&lt;p&gt;It was suggested that this might be a better place for this question than Mathematics Stack Exchange &lt;a href=&quot;https://math.stackexchange.com/q/139100/9281&quot;&gt;where I asked it before&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose one has a black-box function which can be evaluated anywhere (cheaply) on a specified interval $[a,b]$ and has no noise (except floating point granularity, say). What would be the best way to find the discontinuities of this function? I don't know how many discontinuities there might be and there may be none.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of some straightforward methods (uniform sampling, refine where there are large differences between samples, ...), but perhaps there is a better way? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function is &quot;reasonable&quot; in that one could assume that it has at most finitely many discontinuities, the same for higher derivatives, I don't mind if small pathological discontinuities are missed ... (the application is automated plotting of 1d functions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to all who replied, particularly Pedro; the method described in Pachón, Platte and Trefethen seems to be the best approach to me, so I go now to implement it&lt;/p&gt;&#xA;" OwnerUserId="1400" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2017-05-31T16:36:57.230" Title="What is the best way to find discontinuities of a black-box function?" Tags="|interpolation|signal-processing|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2087" PostTypeId="2" ParentId="2086" CreationDate="2012-05-01T11:05:34.263" Score="18" Body="&lt;p&gt;If you're using Matlab, you may be interested in the &lt;a href=&quot;http://www.chebfun.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chebfun project&lt;/a&gt;. Chebfun takes a function, samples it and tries to represent it as a polynomial interpolant. If your function has discontinuities, Chebfun should be able to detect them with the &lt;code&gt;splitting on&lt;/code&gt; command. You can find some examples &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/guide/html/guide8.shtml#6&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in the underlying algorithms, a good reference is Pachón, Platte and Trefethen's paper &quot;&lt;a href=&quot;https://www.cs.ox.ac.uk/files/717/NA-08-07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Piecewise Smooth Chebfuns&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="24422" LastEditDate="2017-05-31T16:36:57.230" LastActivityDate="2017-05-31T16:36:57.230" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2088" PostTypeId="2" ParentId="2086" CreationDate="2012-05-01T14:06:07.253" Score="5" Body="&lt;p&gt;Along with @Pedro, I would look at edge detection algorithms. A discontinuity is an infinity on the derivative, so consider looking at an increasingly fine mesh and targeting regions of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite difference approximation to the derivative of a continuous function should reduce as the mesh is refined. Comparing the finite difference result for the derivative between meshes could then reveal divergences in the gradient which signal discontinuities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;Ok, so although a discontinuity has an infinite derivative, an infinite derivative is not always a discontinuity (thanks @n00b for pointing out an example). For an example like $f(x) = \mathrm{sign}(x) \sqrt{|x|}$ at $x=0$, the finite difference will reduce as the step size $h_x \rightarrow 0$, as I described, even though the analytic derivative is not continuous. &lt;/p&gt;&#xA;" OwnerUserId="1372" LastEditorUserId="1372" LastEditDate="2012-05-02T09:23:17.537" LastActivityDate="2012-05-02T09:23:17.537" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2089" PostTypeId="2" ParentId="2086" CreationDate="2012-05-01T14:51:50.300" Score="10" Body="&lt;p&gt;I suspect that chebfun algorithm must appear to be more practical, but it is neccesary to mention one more way to detect discontinuities, namely the discrete wavelet transform. You can get an idea how it works by looking at &lt;a href=&quot;http://reference.wolfram.com/mathematica/ref/DiscreteWaveletTransform.html&quot;&gt;this Mathematica documentation page&lt;/a&gt;, see section &gt; Applications &gt; Detect Discontinuities and Edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly speaking, you can take DWT of $f$ at uniformly sampled points and look at high-frequency coefficients. The largest of them most probably correspond to discontinuity points. &lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2012-05-01T14:51:50.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2090" PostTypeId="2" ParentId="2085" CreationDate="2012-05-01T15:13:55.777" Score="7" Body="&lt;p&gt;This problem is not well-defined without constraints on $X$ and $Y$. Consider $X=-I$ and $Y=I$. Then $A=B=I$, but $C=0$. &lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-05-01T15:13:55.777" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2091" PostTypeId="1" AcceptedAnswerId="2092" CreationDate="2012-05-01T18:39:04.410" Score="6" ViewCount="1139" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The fact that the back substitution is not done in parallel is not&#xA;  important, because it uses a negligible amount of computer time when N&#xA;  is large, compared to the forward elimination.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a citation from &lt;a href=&quot;http://books.google.ca/books?id=DzfKHO1jalsC&amp;amp;printsec=copyright&amp;amp;source=gbs_pub_info_r&quot;&gt;this book&lt;/a&gt; (page 204) when discussing the implementation of parallel solution of linear systems using Gaussian elimination. I suspect that in practice people do parallelize back substitution and a brief googling shows that such algorithms exist. So the questions are: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When parallel back substitution is reasonable and when it is not? &lt;/li&gt;&#xA;&lt;li&gt;What is the most effective parallel back substitution algorithm so far?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="720" LastActivityDate="2012-05-01T18:53:38.100" Title="Practical efficacy of parallel back substitution" Tags="|linear-algebra|parallel-computing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2092" PostTypeId="2" ParentId="2091" CreationDate="2012-05-01T18:53:38.100" Score="6" Body="&lt;p&gt;You are right that, if there are only $O(1)$ right-hand sides available to be simultaneously solved, then triangular solves are not very scalable. However, if there are $p$ processes and $O(p)$ right-hand sides, then scalable algorithms exist since each right-hand side is independent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that the current gold standard for parallel triangular solves with a single right-hand side is &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.2.9349&quot;&gt;this paper by Joshi et al.&lt;/a&gt;. Fast scalable triangular solves are important to many algorithms, especially when an approximate factorization is embedded in a Krylov algorithm, as one or more triangular systems will need to be solved at every iteration. One effective trick is to explicitly invert the triangular matrices of the factors so that the triangular solves become triangular matrix-vector multiplications (see &lt;a href=&quot;http://www.cse.psu.edu/~raghavan/Papers/solve1.ps&quot;&gt;selective inversion&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-05-01T18:53:38.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2093" PostTypeId="2" ParentId="2085" CreationDate="2012-05-02T01:51:41.280" Score="6" Body="&lt;p&gt;In the restated problem both $X$ and $Y$ have rank at most $k \ll n$, and the same is true of Gram matrices $A$ and $B$.  Also $C=(X+Y)(X+Y)^T$ will have rank at most $k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of forming $C$ from $A$ and $B$, even approximately, seems unattainable.  To briefly enlarge upon Jack Poulson's example, let $X$ and $Y$ be any two matrices such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ XX^T = YY^T = \begin{pmatrix} I &amp;amp; 0 \\ 0 &amp;amp; 0 \end{pmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the given nonzero block $I$ is $k \times k$.  Thus $X,Y$ might be of this form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ X = \begin{pmatrix} P \\ 0 \end{pmatrix} $$&#xA;$$ Y = \begin{pmatrix} Q \\ 0 \end{pmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $P,Q$ are $k \times k$ orthogonal matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking $Q = I$ for simplicity, we see that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ C = (X+Y)(X+Y)^T = 2 \begin{pmatrix} I+P &amp;amp; 0 \\ 0 &amp;amp; 0 \end{pmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So Jack's point is that the problem is not well-defined without further constraints.  Knowing $A = XX^T$ and $B = YY^T$ is insufficient to approximate $C$, as $P$ might be any orthogonal matrix of the right size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me suggest that computing $C$ from $A$ and $B$ might be the wrong goal.  After all, storing $A$ rather than $X$ (resp. $B$ than $Y$) takes more space, since $X$ is only $n \times k$, but $A$ would generally be a full $n \times n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ordinarily a programmer would be pleased to store the &lt;em&gt;factors&lt;/em&gt; $X$ (resp. $Y$) and to do the fairly easy update $X+Y$ rather than compute and store $C$.  Saves time, saves space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course one might fear that further operations done &lt;em&gt;implicitly&lt;/em&gt; with $C$ using $X+Y$ are less efficient.  This is not so, at least with a wide range of studied numeric applications.  Consider for example forming a matrix-vector product $Cv$.  With explicitly formed $C$ this takes $n^2$ multiplications (and roughly equal number of adds).  Doing the implicit product by associativity $(X+Y)(X+Y)^Tv$ only takes $2kn$ multiplies (resp. adds).&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="127" LastEditDate="2012-07-22T15:48:41.517" LastActivityDate="2012-07-22T15:48:41.517" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2094" PostTypeId="1" AcceptedAnswerId="2098" CreationDate="2012-05-02T02:16:05.003" Score="6" ViewCount="115" Body="&lt;p&gt;I'm currently getting very long propagation times when attempting to use the Time Stepping propagators in Petsc 3.2, and in the interest of speeding things up, I'm curious how I can reduce the accuracy (or see what it is in the first place) to speed it up.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2012-05-03T10:58:41.580" Title="How do you change the desired accuracy of a TS object in PETSc?" Tags="|petsc|time-integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2095" PostTypeId="1" AcceptedAnswerId="2101" CreationDate="2012-05-02T03:58:36.073" Score="10" ViewCount="4033" Body="&lt;p&gt;I'm trying to write a full &lt;a href=&quot;http://en.wikipedia.org/wiki/Support_vector_machine&quot;&gt;SVM&lt;/a&gt; implementation in Python and I have a few issues computing the Lagrange coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First let me rephrase what I understand from the algorithm to make sure I'm on the right path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $x_1, x_2, ..., x_n$ is a dataset and $y_i \in \{-1, 1\}$ is the class label of $x_i$, then $$\forall i, y_i(w^Tx_i + b) \geq 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we just need to solve an optimization problem to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;minimize $\|w\|^2$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to $y_i(w^Tx_i + b) \geq 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of Lagrange coefficients, this translates into finding $w$, $b$ and $\alpha=(\alpha_1, \alpha_2, ... \alpha_n) \neq0$ and $\geq0$ minimizing: $$L(\alpha, w, b) = \frac12 \|w\|^2 - \sum \alpha_i(y_i(w^Tx + b)-1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now since $$\frac{\partial L}{\partial w}=0 \implies w=\sum \alpha_i y_i x_i$$ and $$\frac{\partial L}{\partial b}=0 \implies \sum y_i \alpha_i = 0$$ we can rewrite it as $$L(\alpha, w, b) = Q(\alpha)=\sum \alpha_i - \frac12\sum \sum \alpha_i \alpha_j y_i y_j x_i^T x_j$$ with constraints $$\alpha_i \geq 0 \ \text{and} \ \sum \alpha_i y_i = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm trying to solve the optimization problem using Python, and the only free package I could find is called &lt;a href=&quot;http://abel.ee.ucla.edu/cvxopt/&quot;&gt;cvxopt&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like some help to solve this, I couldn't find any good example about this, and while I understand the theory, I'm having a hard time translating it into code (I would have expected the opposite since I'm more from a programming background).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that at some point I'll want to solve it using Kernels $$L(\alpha, w, b) = Q(\alpha)=\sum \alpha_i - \frac12\sum \sum \alpha_i \alpha_j y_i y_j K(x_i,x_j)$$ but I'm not sure what the implications are regarding solving this in code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated, I'm really lost on how to implement this in Python. If you have a better module to solve the optimization problem I'd like to read about it as well.&lt;/p&gt;&#xA;" OwnerUserId="1380" LastEditorUserId="9" LastEditDate="2012-05-02T05:54:26.580" LastActivityDate="2012-05-02T17:26:16.340" Title="Calculating Lagrange coefficients for SVM in Python" Tags="|optimization|machine-learning|support-vector-machines|quadratic-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2096" PostTypeId="2" ParentId="2086" CreationDate="2012-05-02T04:26:15.923" Score="8" Body="&lt;p&gt;Weighted Essentially Non-Oscillatory (WENO) methods use &quot;smoothness indicators&quot; to detect discontinuities in finite volume and difference methods.  From the description of Chebfun that Pedro gave, it seems as though the general idea is the same: construct a set of interpolating polynomials and use them to compute some measure of smoothness.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See G.S. Jiang, and C.W. Shu, Efficient implementation of weighted ENO schemes, J.Comput.Phys., vol. 126, pp. 202--228, 1996.&lt;/p&gt;&#xA;" OwnerUserId="486" LastActivityDate="2012-05-02T04:26:15.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2097" PostTypeId="2" ParentId="2095" CreationDate="2012-05-02T06:04:32.463" Score="2" Body="&lt;p&gt;The general form of your optimization problem is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Quadratic_programming&quot; rel=&quot;nofollow&quot;&gt;Quadratic Program&lt;/a&gt;, regardless of whether you are using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Kernel_trick&quot; rel=&quot;nofollow&quot;&gt;kernel trick&lt;/a&gt; or a linear kernel.  It sounds like &lt;a href=&quot;http://abel.ee.ucla.edu/cvxopt/userguide/coneprog.html#quadratic-programming&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;cvxopt&lt;/code&gt;&lt;/a&gt; will be sufficient for what you're trying to do, but other pythonauts on here have had luck with &lt;a href=&quot;http://openopt.org/Welcome&quot; rel=&quot;nofollow&quot;&gt;OpenOpt&lt;/a&gt; as well.  &lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-05-02T06:04:32.463" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2098" PostTypeId="2" ParentId="2094" CreationDate="2012-05-02T06:18:58.047" Score="4" Body="&lt;h3&gt;nonlinear systems of equations&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In general, the &lt;code&gt;PETSc TS&lt;/code&gt; propagators solve a nonlinear system of equations at each time step, so you would need to access the &lt;code&gt;SNES&lt;/code&gt; object and modify its accuracy tolerance there.  See the documentation on &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/SNES/SNESSetFromOptions.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;SNESSetFromOptions&lt;/code&gt;&lt;/a&gt; for a list of the command line options available to you for adjusting the solver tolerances.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;linear system of equations&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;PETSc doesn't expose the &lt;code&gt;KSP&lt;/code&gt; object belonging to &lt;code&gt;TS&lt;/code&gt; in the same way at the command-line, @JedBrown or @MattKnepley may be able to comment on why.  Programmatically, you can:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSGetKSP.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;TSGetKSP(ts, &amp;amp;ksp);&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSetTolerances.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;KSPSetTolerances(ksp, rtol, abstol, dtol, maxits;&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-05-03T10:58:41.580" LastActivityDate="2012-05-03T10:58:41.580" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2100" PostTypeId="1" CreationDate="2012-05-02T11:33:22.543" Score="21" ViewCount="2296" Body="&lt;p&gt;I can't seem to find any literature on algorithms which can be used to solve a many-to-many generalized assignment problem (GAP), i.e. models where not only can more tasks be assigned to one agent, but multiple agents can also be assigned to one task (one-to-one and one-to-many AP's are discussed in a paper by Pentico).  I know next-to-nothing of assignment problems, but I came upon a problem like this during my research and would like to know more about how to solve them. Is it possible that such a many-to-many GAP is known under another name, or is there a different reason why so few literature on it can be found?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pentico, D. &lt;a href=&quot;https://www.ida.liu.se/~TDDD10/papers_to_read/survey.pdf&quot; rel=&quot;nofollow&quot;&gt;Assignment Problems: A Golden Anniversary Survey&lt;/a&gt;. &lt;em&gt;European Journal Of Operational Research&lt;/em&gt; (2007); &lt;strong&gt;176&lt;/strong&gt;(2):774-793.&lt;/p&gt;&#xA;" OwnerUserId="1408" LastEditorUserId="16731" LastEditDate="2015-08-19T19:41:47.503" LastActivityDate="2015-08-19T19:41:47.503" Title="Algorithms for a many-to-many generalized assignment problem" Tags="|algorithms|data-analysis|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2101" PostTypeId="2" ParentId="2095" CreationDate="2012-05-02T15:14:35.887" Score="4" Body="&lt;p&gt;I have used cvxopt to implement an SVM before, however in matlab not python.  It will definitely serve your purpose, whether its efficient enough will depend on what you are using it for.  The most efficient SVMs do not use a QP solver package, they take advantage of some optimizations unique to SVM.  Many use an &lt;a href=&quot;http://cognet.mit.edu/library/books/mitpress/0262194163/cache/chap12.pdf&quot; rel=&quot;nofollow&quot;&gt;SMO style&lt;/a&gt; algorithm to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.csie.ntu.edu.tw/~cjlin/libsvm/&quot; rel=&quot;nofollow&quot;&gt;LibSVM&lt;/a&gt; is an SVM package which uses the algorithm in &lt;a href=&quot;http://www.csie.ntu.edu.tw/~cjlin/papers/quadworkset.pdf&quot; rel=&quot;nofollow&quot;&gt;Working Set Selection Using Second Order Information for Training Support Vector Machines&lt;/a&gt;. The code is open source, if you are interested in looking at how its implemented.  It also does have a python interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://svmlight.joachims.org/&quot; rel=&quot;nofollow&quot;&gt;SVMLight&lt;/a&gt; is another package, they use a different algorithm (see their site for references).  It is also open source and has a python interface.&lt;/p&gt;&#xA;" OwnerUserId="1410" LastEditorUserId="1410" LastEditDate="2012-05-02T17:26:16.340" LastActivityDate="2012-05-02T17:26:16.340" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2102" PostTypeId="1" AcceptedAnswerId="2112" CreationDate="2012-05-02T19:26:18.350" Score="-1" ViewCount="170" Body="&lt;p&gt;Could the runtime of methods within a system potentially be reduced via refactoring during development (pre-public release)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. Let's assume that &lt;code&gt;methodX()&lt;/code&gt; takes 2 seconds to complete &lt;strong&gt;without&lt;/strong&gt; having been refactored.  It has been discovered that &lt;code&gt;methodX()&lt;/code&gt; is suffering from severe code smells.  After refactoring (for example, such as using global variables where appropriate or reducing the parameter list for &lt;code&gt;methodX()&lt;/code&gt;), could &lt;code&gt;methodX()&lt;/code&gt; potentially run faster (i.e. to 0.5 seconds, for example)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could the speed of the method potentially be increased if refactoring (i.e. eradication of code smells such as using global variables instead of creating &lt;code&gt;x&lt;/code&gt; amount of individual variables, etc) has been applied?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few of you have provided some good answers, but unfortunately appear to be somewhat missing the point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am fully aware that the system's speed could be improved by changing the algorithm used; however, my question is &lt;strong&gt;not&lt;/strong&gt; focussed on this and is more focussed on whether or not refactoring alone could potentially speed up the system (i.e. splitting up large methods/classes, reducing parameter lists, using static and/or global variables, addressing unreachable code (i.e. removing it altogether if it isn't serving a purpose or making it &quot;reachable&quot;), etc).&lt;/p&gt;&#xA;" OwnerUserId="1211" LastEditorUserId="1211" LastEditDate="2012-05-03T08:43:59.840" LastActivityDate="2012-05-03T20:35:56.680" Title="System Speed - Refactoring" Tags="|software|" AnswerCount="4" CommentCount="8" ClosedDate="2012-05-03T18:42:39.063" ContentLicense="CC BY-SA 3.0" />
  <row Id="2103" PostTypeId="2" ParentId="2102" CreationDate="2012-05-02T22:20:36.933" Score="3" Body="&lt;p&gt;Any refactoring you do will possibly affect the speed of the method, either making the method slower or faster. A lot of times refactoring is done by changing the actual algorithm used inside the method. Take as a simple example something learned in basic CS classes: sorting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you have a method &lt;code&gt;sort&lt;/code&gt; which takes in a list of things and outputs that same list, but sorted. You initially use a bubble sort, but later refactor the method to use a quicksort. Your method is now generally faster.&lt;/p&gt;&#xA;" OwnerUserId="739" LastActivityDate="2012-05-02T22:20:36.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2104" PostTypeId="2" ParentId="2051" CreationDate="2012-05-03T00:12:12.897" Score="4" Body="&lt;p&gt;As a shameless plug, if you're familiar with the finite element method and want to learn about modern numerical methods to solve inverse problems, I've got a paper on that:&#xA;  &lt;a href=&quot;http://epubs.siam.org/sisc/resource/1/sjoce3/v30/i6/p2965_s1?isAuthorized=no&quot; rel=&quot;nofollow&quot;&gt;http://epubs.siam.org/sisc/resource/1/sjoce3/v30/i6/p2965_s1?isAuthorized=no&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-03T00:12:12.897" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2105" PostTypeId="2" ParentId="2028" CreationDate="2012-05-03T01:17:13.970" Score="1" Body="&lt;p&gt;In deal.II we've got the software infrastructure to parallelize assembly on each cell onto multiple cores using the Threading Building Blocks (in essence, you have one task per cell and need to schedule these tasks onto available processors -- that's not how it's implemented but it's the general idea). The problem is that for the local integration you need a number of temporary (scratch) objects and you need to provide at least as many as there are tasks that can run in parallel. We see poor speedup, presumably because when a task is put onto a processor it grabs one of the scratch objects that will typically be in some other core's cache. We had two questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(i) Is this really the reason? When we run the program under cachegrind I see that I'm using basically the same number of instructions as when running the program on a single thread, yet the total run-time accumulated over all threads is much larger than the single-thread one. Is it really because I continually fault the cache?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ii) How can I find out where I am, where each of the scratch objects are, and which scratch object I'd need to take to access the one that's hot in my current core's cache?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, we haven't found answers to either of these solutions and after a couple of works decided that we lack the tools to investigate and solve these problems. I do know how to at least in principle solve problem (ii) (namely, using thread-local objects, assuming that threads remain pinned to processor cores -- another conjecture that's not trivial to test), but I have no tools to test problem (i).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, from our perspective, dealing with NUMA is still an unsolved question.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-03T01:17:13.970" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2106" PostTypeId="2" ParentId="2060" CreationDate="2012-05-03T01:22:48.987" Score="2" Body="&lt;p&gt;On Q1: In the FEM/FDM language, you would apply the Cuthill-McKee algorithm to number all degrees of freedom that are connected to each other. Once the algorithm stops without having numbered all degrees of freedom, you know that there's another as yet unnumbered component of the matrix, and you would continue with one of its degrees of freedom, with the next available index. This way, you end up with the block diagonal structure of the matrix -- in other words, the Cuthill-McKee algorithm will compute the permutation for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Q2: All Krylov-space methods have convergence properties independent of the numbering of degrees of freedom (i.e., they are invariant under permutations) as long as you have a preconditioner that is invariant.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-03T01:22:48.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2107" PostTypeId="2" ParentId="2102" CreationDate="2012-05-03T01:27:44.690" Score="5" Body="&lt;p&gt;If by &quot;refactoring&quot; you mean just renaming things, breaking code out of one function and putting it into a different one, etc, then the answer is that the result is probably in the noise. There will be differences of course since the compiler compiles different code, but it's not going to be a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You get significant speed improvements by changing the &lt;em&gt;algorithm&lt;/em&gt; behind what you do. But that isn't usually called &quot;refactoring&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, it all depends what exactly it is you mean when you use the word &quot;refactoring&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-03T01:27:44.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2108" PostTypeId="2" ParentId="2102" CreationDate="2012-05-03T02:43:48.563" Score="1" Body="&lt;p&gt;If you have any concern about the speed of the software, then I would recommend concentrating on that first, as in &lt;em&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/1870/1262&quot;&gt;this post&lt;/a&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anything you do to improve speed is going to change the code, so it will look like refactoring.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any changes you make to clean up the code might make it a little faster, and might make it a lot slower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So by all means clean it up, but if performance is an issue (maybe it's not), make that your primary concern.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-05-03T02:43:48.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2109" PostTypeId="1" AcceptedAnswerId="2118" CreationDate="2012-05-03T13:38:43.300" Score="6" ViewCount="1536" Body="&lt;p&gt;We have a flow that is dominated by adhesion forces from the substrate and surface tension from the free surface. The material is nearly solid and at rest first, and gets a bit less solid by heating. So we have a creeping flow, which can be modeled by Stokes equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of the difficulties in simulating the free surface and the surface tension it generates, somebody proposed to use a lattice Boltzmann approach for simulation. However it's unclear to me whether the lattice Boltzmann scheme can be used at all for an incompressible Stokes flow, and even if it should be possible to use it, whether it would be grossly inefficient. Is my gut-feeling about lattice Boltzmann correct, or do I actually undervalue its potential for that type of problem?&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2015-10-29T11:19:24.587" Title="Is lattice Boltzmann suitable for simulation of incompressible Stokes flow?" Tags="|pde|fluid-dynamics|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2110" PostTypeId="1" CreationDate="2012-05-02T15:55:05.330" Score="9" ViewCount="2215" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Hill_climbing&quot;&gt;Hill climbing&lt;/a&gt; seems to be a very powerful tool for optimization. However, how to generate the &quot;neighbors&quot; of a solution always puzzles me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I am optimizing a solution $(x_1, x_2, x_3)$. Here $x_1$ is in range $(0, 0.1)$, $x_2$ is in range $(0, 100)$, $x_3$ is in range $(0, 1000000)$. What's the best way to generate &quot;neighbors&quot;? I can't really pick a &quot;step size&quot; here, because a step size of 1 is huge to $x_1$, but very minor to $x_3$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best generic way to generate &quot;neighbors&quot; in hill climbing algorithms? &lt;/p&gt;&#xA;" OwnerUserId="1418" OwnerDisplayName="CodeNoob" LastActivityDate="2022-07-23T15:00:43.097" Title="How to generate neighbors in hill climbing algorithm?" Tags="|algorithms|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2111" PostTypeId="2" ParentId="2110" CreationDate="2012-05-03T07:33:11.957" Score="2" Body="&lt;p&gt;The easiest (and safest)&#xA;is to reparametrize your problem &#xA;so that all parameters have a comparable range, say [0,1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can use a different step size for each parameter.&lt;/p&gt;&#xA;" OwnerDisplayName="Vincent Zoonekynd" LastActivityDate="2012-05-03T07:33:11.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2112" PostTypeId="2" ParentId="2102" CreationDate="2012-05-03T14:49:16.787" Score="3" Body="&lt;p&gt;In general, people refers to &lt;strong&gt;bad code&lt;/strong&gt;, &lt;strong&gt;code smells&lt;/strong&gt;, or whatever name you like, as code with low programmer productivity. I think we can agree that bad code is complex and difficult to maintain. Refactoring are just a guidelines to improve code quality from a human perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer performance is a different issue, and in a nutshell is how much useful work is performed by a computer relative to its maximum capacity. Therefore, performance is not concerned about if a person can easily read the code, how easy is to maintain or extend the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can a messy code achieve higher performance than a readable, maintainable, and less complex code when both codes implement the same functionality or even the same algorithm? the answer depends on how well these codes map to the underlying architecture so that the generated programs are able to effectively and efficiently use the hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, consider that &lt;em&gt;hand-tuned high-performance&lt;/em&gt; code (that tightly maps to the underlying hardware) is most commonly not easy to read, maintain, or extend... at all! If you are curious about this, you can look into the source code of the high-performance (until recently) &lt;a href=&quot;http://www.tacc.utexas.edu/tacc-projects/gotoblas2&quot; rel=&quot;nofollow&quot;&gt;GotoBLAS library&lt;/a&gt; and see how much it &lt;em&gt;smells&lt;/em&gt;. The down side to hand-tuned high-performance code is that is not easy to read or maintain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Mike Dunlavey mentioned in a comment, for &lt;em&gt;most software&lt;/em&gt;, good code and productivity are not necessarily opposite goals and code can be simultaneously improved in both directions. However, the closer you get to the maximum performance of the hardware (this is not the case for &lt;em&gt;most software&lt;/em&gt;, hence the reference to high-performance GotoBLAS) the more important is the mapping from the expressions in the code to the the instructions that effectively run on the hardware. At this point you probably worry about hardware specific features that are hard to maintain and tune for, such as cache sizes, thread affinity, vector instructions, page faults, etc. I should add that good code practices make developing high-performance applications a much easier job, as you most likely need to maintain and extend such code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The balance between programmer productivity and computer performance is one of the main struggles in &lt;em&gt;high-performance scientific computing&lt;/em&gt;. There is a long list of initiatives to try solve this problem: advanced compilers, auto-tuning code, new programming languages, domain specific languages, programming patterns and guidelines, etc.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="1070" LastEditDate="2012-05-03T20:35:56.680" LastActivityDate="2012-05-03T20:35:56.680" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2113" PostTypeId="1" CreationDate="2012-05-03T15:39:46.817" Score="2" ViewCount="227" Body="&lt;p&gt;This is a follow up question to one I made previously about non-linear equations and &lt;a href=&quot;https://stackoverflow.com/questions/10315437/on-working-with-ranges-of-real-numbers-in-microsoft-solver-foundation&quot;&gt;ranged real numbers in Solver Foundation.&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I acknowledge that where possible, rewriting a problem that is non-linear into linear could avoid some issues, however it is likely for various reasons this won’t be a viable option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The requirements for the project I am involved with call for a set of constraints to be generated from a data source. The issue is the constraints could be a mix of linear or non-linear problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to come up with a set of example constraints and variables to test against that will hopefully give a clearer picture as to what I am trying to achieve. Does anyone know of a solver capable of handling a model of the kind to be generated by this combination of variables and constraints? We are currently using Solver foundation and have been trying various configurations and solver types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some parameters are created and have been bound to some arbitrary values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   P1 = 1 &#xA;   P2 = 2&#xA;   P3 = 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Some decisions with a domain specified as a finite range with a defined step value between each value in the range:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;D1 (1..5) step of 0.1  //(e.g. 1, 1.1, 1.2 … 4.9, 5)&#xA;D2 (1..5) step of 1&#xA;D3 (10..20) step of 0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The following example constraints simulate the kinds of constraints likely to be acquired from the data source. Next to each constraint is the expected result of running a solver with a model containing the one constraint, given the values of the above parameters and decision variables. Note that there is no concern with the quality of a solution, simply if there is a solution (feasible) or not (unfeasible). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C1: P1 = P2 * P3 (infeasible) &#xA;C2: P3 = P1 + P2 (feasible)&#xA;C3: D1 = P1 * P2 (feasible)&#xA;C4: D3 = P1 * P2 (infeasible)&#xA;C5: P1 = D1 * D2 (feasible)&#xA;C6: P1 = D3 * D3 (infeasible)&#xA;C7: D3 = D1 * D2 (feasible)&#xA;C8: D2 = D3 * D3 (infeasible)&#xA;C9: P3 = P2 / P1 (infeasible)&#xA;C10: -P3 = P1 * D1 (infeasible)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any pointers or advice would be most welcome :) moving from a UI background into an optimisation project has been character building to say the least.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&#xA;d.&lt;/p&gt;&#xA;" OwnerUserId="1415" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-05-03T15:39:46.817" Title="Is it possible to run a Solver Foundation solver against a model containing linear and non-linear elements?" Tags="|optimization|linear-solver|nonlinear-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2114" PostTypeId="1" AcceptedAnswerId="2132" CreationDate="2012-05-03T17:25:09.627" Score="15" ViewCount="8023" Body="&lt;p&gt;There are numerous FD schemes for the advection equation $\frac{\partial T}{\partial t}+u\frac{\partial T}{\partial x}=0$ discuss in the web. For instance here:&#xA;&lt;a href=&quot;http://farside.ph.utexas.edu/teaching/329/lectures/node89.html&quot;&gt;http://farside.ph.utexas.edu/teaching/329/lectures/node89.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I haven't seen anyone propose an &quot;implicit&quot; upwind scheme like this:&#xA;$\frac{T^{n+1}_i-T^{n}_i}{\tau}+u\frac{T^{n+1}_i-T^{n+1}_{i-1}}{h_x}=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the upwind schemes I've seen were dealing with data on the previous time-step in the spacial derivative. What is the reason for that? How does the classical upwind scheme compare to the one I wrote above?&lt;/p&gt;&#xA;" OwnerUserId="1088" LastEditorUserId="123" LastEditDate="2012-08-20T14:25:14.600" LastActivityDate="2012-08-20T14:25:14.600" Title="Implicit finite difference schemes for advection equation" Tags="|finite-difference|implicit-methods|advection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2115" PostTypeId="2" ParentId="2110" CreationDate="2012-05-04T02:29:12.693" Score="3" Body="&lt;p&gt;Vincent's solution will get you over the first hurdle. You might still be wondering how to select the step size even if all the dimensions have the same range. In that case it helps to understand the behavior of your function so you can select an appropriate value. Another way is to adaptively set the step size based on gradient (tread lightly if the function is changing a lot).&lt;/p&gt;&#xA;&lt;p&gt;This paper might have some interesting ideas too: &lt;a href=&quot;https://doi.org/10.1007/3-540-34783-6_56&quot; rel=&quot;nofollow noreferrer&quot;&gt;On a Hill-Climbing Algorithm with Adaptive Step Size: Towards a Control Parameter-Less Black-Box Optimisation Algorithm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1113" LastEditorDisplayName="user43608" LastEditDate="2022-07-23T15:00:43.097" LastActivityDate="2022-07-23T15:00:43.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2116" PostTypeId="2" ParentId="2114" CreationDate="2012-05-04T02:34:09.733" Score="12" Body="&lt;p&gt;There is no reason that you cannot do what you wrote. One of the reasons that this is uncommon is that there for hyperbolic (advection) type problems the domain of dependence is finite. Thus an explicit methods makes sense from a computational efficiency standpoint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implicit scheme you have written will require solving a linear system, albeit in the case you have written triangular, and thus fairly simple to solve. Of course when you go to systems, and multiple dimensions, the system will likely no be triangular, though sometimes this can result with a proper ordering of you unknowns (see for instance &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2007.08.012&quot;&gt;Kwok and Tchelepi, JCP 2007&lt;/a&gt; and  &lt;a href=&quot;http://dx.doi.org/10.1007/s10915-006-9079-7&quot;&gt;Gustafsson and Khalighi, JSC, 2006&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes in the hopes of taking large time steps people will use implicit time stepping as you have written, but you must be careful here. When using an implicit method, you will introduce a large amount of diffusion thus you will smear out your solution significantly.&lt;/p&gt;&#xA;" OwnerUserId="880" LastActivityDate="2012-05-04T02:34:09.733" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2117" PostTypeId="2" ParentId="2109" CreationDate="2012-05-04T03:07:21.167" Score="2" Body="&lt;p&gt;A (very) quick literature review shows that people do use it for low Reynolds number and creeping incompressible flows. See &lt;a href=&quot;http://journals.cambridge.org/action/displayAbstract;jsessionid=4966A33B7FB93127D62525D77E5A2C19.journals?fromPage=online&amp;amp;aid=291422&quot; rel=&quot;nofollow&quot;&gt;this paper from JFM&lt;/a&gt; and &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0142727X99000818&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;. The second paper shows a range of Reynolds numbers and indicates that the results match well between LBM and FVM with no mention of inefficiencies anywhere in the Reynolds number spectrum.&lt;/p&gt;&#xA;" OwnerUserId="391" LastActivityDate="2012-05-04T03:07:21.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2118" PostTypeId="2" ParentId="2109" CreationDate="2012-05-04T03:14:04.593" Score="9" Body="&lt;p&gt;As a matter of general observation, it is certainly true than one can solve a great many problems using the Lattice Boltzmann Method. Specifically, since they have been used for the Navier-Stokes equations, they will also be applicable to the Stokes equations, much as for most any other flow problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it is also true that LBMs are generally very very slow. There is little one can do about it if you have, say, a non-thermal equilibrium dilute gas flow problem for which there may not be any other efficient solver. But it is exceptionally silly to use such an expensive method for a problem for which there are any number of exceptionally efficient methods around, such as multigrid-preconditioned higher-order finite element discretizations of the Stokes problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, yes: it's possible. And, no: it's not a good idea. :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-05-04T10:51:55.093" LastActivityDate="2012-05-04T10:51:55.093" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2119" PostTypeId="1" AcceptedAnswerId="2122" CreationDate="2012-05-04T07:00:12.130" Score="9" ViewCount="2206" Body="&lt;p&gt;I was wondering what differences and relations are between &quot;search methods&quot; and &quot;optimization methods&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially when solving an optimization problem? I emphasize the context of solving optimization problems, because I guess &lt;a href=&quot;http://en.wikipedia.org/wiki/Search_algorithm&quot;&gt;search methods&lt;/a&gt; are not just for solving optimization problems, but also non-optimization  problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My confusion comes from the following facts:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There are some optimization methods, named &quot;xxx search&quot;, such as&#xA;&lt;a href=&quot;http://www.results-page.net/redirect?URL=http%253A%252F%252Fen.wikipedia.org%252Fwiki%252FLocal_search_%2528optimization%2529&amp;amp;HitID=mPKIb38AAAEAAB2FeSYAAACZ&amp;amp;ParticipantID=euekiz39ksg8nwp7iqj2fp5wzfwi5q76&amp;amp;Keyword=local%20search&amp;amp;Position=0&amp;amp;LinkID=mPKIb38AAAEAAB2FeSYAAACZ&amp;amp;Affiliate=12&amp;amp;KeywordType=1&amp;amp;FailureMode=4&amp;amp;HitDateTime=2012-05-04%2007:09:12&amp;amp;rand=0107780ac743bcb278ba2d24682e7ce6&quot;&gt;local search&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Stochastic_optimization#Randomized_search_methods&quot;&gt;stochastic search&lt;/a&gt;, .... What does &quot;search&quot; mean actually? I wonder if there are&#xA;optimization methods that are not &quot;search&quot;?&lt;/li&gt;&#xA;&lt;li&gt;Also in this book &lt;a href=&quot;http://books.google.com/books?id=f66OIvvkKnAC&amp;amp;source=gbs_navlinks_s&quot;&gt;Introduction to Stochastic Search and&#xA;Optimization by Spall&lt;/a&gt;, I don't quite understand the difference&#xA;between &quot;Search&quot; and &quot;Optimization&quot; in its title as well as in its&#xA;content. Why needs to distinguish between &quot;Search&quot; and&#xA;&quot;Optimization&quot;, if they mean the same? Or does &quot;Optimization&quot; mean&#xA;stochastic optimization tasks/problems instead of optimization&#xA;methods, as opposed to &quot;search&quot; means the methods to solve the&#xA;optimization tasks/problems?&lt;/li&gt;&#xA;&lt;li&gt;Also &lt;a href=&quot;http://en.wikipedia.org/wiki/No_free_lunch_in_search_and_optimization&quot;&gt;No free lunch in search and optimization&lt;/a&gt; distinguishes&#xA;search and optimization again.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="258" LastEditDate="2012-05-04T07:18:32.903" LastActivityDate="2012-05-07T21:18:23.690" Title="Meaning of search methods and optimization methods" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2120" PostTypeId="1" AcceptedAnswerId="2123" CreationDate="2012-05-04T08:23:03.650" Score="10" ViewCount="3785" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For optimization, from &lt;a href=&quot;http://en.wikipedia.org/wiki/Metaheuristic&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In computer science, &lt;strong&gt;metaheuristic&lt;/strong&gt; designates a computational method&#xA;  that &lt;strong&gt;optimizes&lt;/strong&gt; a problem by iteratively trying to improve a candidate&#xA;  solution with regard to a given measure of quality. Metaheuristics&#xA;  make few or no assumptions about the problem being optimized and can&#xA;  search very large spaces of candidate solutions. However,&#xA;  metaheuristics do not guarantee an optimal solution is ever found.&#xA;  Many metaheuristics implement some form of stochastic optimization.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Other terms having a similar meaning as metaheuristic, are:&#xA;  derivative-free, direct search, black-box, or indeed just heuristic&#xA;  optimizer. Several books and survey papers have been published on the&#xA;  subject.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I wonder how to tell whether an optimization method is metaheuristic or not? For example, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Is the simplex method for linear&#xA;programming metaheuristic?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Are the majority of nonlinear&#xA;programming methods such as gradient descent, Lagrangian multiplier&#xA;method, penalty methods, Interior point methods (barrier methods),&#xA;metaheuristic? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Are all gradient-free methods, such as Nelder–Mead method or downhill simplex method, metaheuristic?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are some optimization methods that are not metaheuristic?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;More generally (going beyond optimization) for problem solving&#xA;techniques, from &lt;a href=&quot;http://en.wikipedia.org/wiki/Heuristic&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Heuristic&lt;/strong&gt; refers to experience-based &lt;strong&gt;techniques for problem solving,&#xA;  learning, and discovery&lt;/strong&gt;. Where an exhaustive search is impractical,&#xA;  heuristic methods are used to speed up the process of finding a&#xA;  satisfactory solution. Examples of this method include using a rule of&#xA;  thumb, an educated guess, an intuitive judgment, or common sense.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In more precise terms, heuristics are strategies using readily&#xA;  accessible, though loosely applicable, information to control problem&#xA;  solving in human beings and machines.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I wonder how to understand the meaning of &quot;heuristic&quot;?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;how can I tell whether a &quot;problem solving,learning, and discovery&quot; technique is heuristic or not? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are some &quot;problem solving,learning, and discovery&quot; techniques that are not heuristic?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastActivityDate="2012-05-04T17:13:14.163" Title="Meaning of (meta)heuristic methods" Tags="|optimization|heuristics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2121" PostTypeId="1" AcceptedAnswerId="3145" CreationDate="2012-05-04T08:41:01.267" Score="13" ViewCount="1357" Body="&lt;p&gt;The paper &quot;Expression Templates Revisited: A Performance Analysis of Current Methodologies&quot; in SIAM Journal of Scientific Computing references the &quot;Blaze&quot; linear algebra library. I haven't heard of it before, and can't seem to find online references. (The obvious google searches are giving the above paper back.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is this library and where I can I learn more about it? &lt;/p&gt;&#xA;" OwnerUserId="1420" LastEditorUserId="1420" LastEditDate="2012-05-04T15:46:17.790" LastActivityDate="2012-08-28T09:19:32.940" Title="Blaze linear algebra library?" Tags="|linear-algebra|reference-request|c++|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2122" PostTypeId="2" ParentId="2119" CreationDate="2012-05-04T11:49:00.787" Score="12" Body="&lt;p&gt;search = attempt to find a feasible point that satisfies all constraints (and for optimization a better point than found so far), generally using function values only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;local search: improving a feasible point (or a distance to feasibility measure) by searching among neighboring points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;stochastic search: searching using a nondeterministic criterion for choosing trial points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is independent of whether an optimization criterion is given. In particular, in ''No free lunch in search and optimization'' search refers to search for feasibility, while optimization refers to search for optimality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a general sense, for optimization problem, search and optimization are equivalent. However, they have connotations that make a difference in the usage of the term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;optimization method = a method for solving an optimization problem, often (but not necessarily) using gradient (or subgradient or even Hessian) information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being able to use gradients drastically increases the efficiency of optimization methods. One uses in this context (i.e., with gradients known) to the term search only in the combination ''line search'' which means searching for a better point along a chosen direction.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-04T11:49:00.787" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2123" PostTypeId="2" ParentId="2120" CreationDate="2012-05-04T11:56:38.343" Score="8" Body="&lt;p&gt;Heuristic is something that works in many cases in practice, though there is no detailed argument for why it should work well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Metaheuristics is not an algorithm but a general heuristic scheme or idea that can be used inside specific algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the simplex algorithm for linear programming is neither heuristics nor metaheuristics, as it has a well-established convergence theory. The sqame holds for sequential quadtatic programming or interior point methods. &#xA;(Interior point methods are a general scheme, but not heuristic and hence not a metaheuristics, as there is a quite strong theory associated with it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Nelder-Mead = downhill simplex algorithm for minimizing a function is heuristics (it actually may fail on quite simple problems in higher dimensions), and tabu search is metaheuristics (as quite a lot of diverse algorithms can be written that employ tabu search, but are otherwise of quite different quality.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-04T11:56:38.343" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2124" PostTypeId="1" CreationDate="2012-05-04T14:16:56.773" Score="0" ViewCount="171" Body="&lt;p&gt;I am looking for recommendations for a C++ math library, with a permissive licence, well suited to calculating a wide variety of statistics on segmentations of timebased parameter data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be particularly interested in anything that can quantify properties of the curve shapes as well as the raw data set.&lt;/p&gt;&#xA;" OwnerUserId="1421" LastActivityDate="2012-05-04T14:16:56.773" Title="Permissive Math Library for Parameter Statistics in C++" Tags="|c++|statistics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2125" PostTypeId="2" ParentId="2120" CreationDate="2012-05-04T14:32:43.927" Score="8" Body="&lt;p&gt;I won't iterate over simplex and Nelder-Mead since @ArnoldNeumaier gave already a very good explanation, but wanted to add my 2 cents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the best quote I've heard some time ago to describe the difference between heuristic and metaheuristic: &lt;strong&gt;A heuristic is a pretty good rule. A metaheuristic is a pretty good rule for finding pretty good rules.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should just see it as a way to find good heuristics for specific problems; basically if you ask yourself one of the following questions you are talking about a metaheuristic:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How should I tweak the parameters of this heuristic to improve performance on that problem?&lt;/li&gt;&#xA;&lt;li&gt;Is this heuristic better than that heuristic?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is a bunch of metaheuristics you can use for &lt;em&gt;problem solving,learning, and discovery&lt;/em&gt;, namely:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow&quot;&gt;Simulated annealing&lt;/a&gt;: i've used this one in a problem to optimize the placement of connected components on a grid to minimize the total connection cost for example.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Particle_swarm_optimization&quot; rel=&quot;nofollow&quot;&gt;Particle swarm&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms&quot; rel=&quot;nofollow&quot;&gt;Ant colony&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Tabu_search&quot; rel=&quot;nofollow&quot;&gt;Tabu search&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I find that most metaheuristics are somewhat inspired by natural phenomenons, which are hard to rigorously explain, but have good convergence properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is &lt;a href=&quot;http://www.scholarpedia.org/article/Metaheuristic_Optimization&quot; rel=&quot;nofollow&quot;&gt;a good link&lt;/a&gt; if you want to read more about some other metaheuristic techniques&lt;/p&gt;&#xA;" OwnerUserId="1380" LastEditorUserId="1380" LastEditDate="2012-05-04T17:13:14.163" LastActivityDate="2012-05-04T17:13:14.163" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2126" PostTypeId="1" AcceptedAnswerId="2130" CreationDate="2012-05-04T18:17:23.910" Score="5" ViewCount="2656" Body="&lt;p&gt;From &lt;a href=&quot;http://en.wikipedia.org/wiki/Hill_climbing&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In computer science, &lt;strong&gt;hill climbing&lt;/strong&gt; is a mathematical optimization&#xA;  technique which belongs to the family of local search. It is an&#xA;  iterative algorithm that starts with an arbitrary solution to a&#xA;  problem, then attempts to find a better solution by &lt;strong&gt;incrementally&#xA;  changing a single element of the solution&lt;/strong&gt;. If the change produces a&#xA;  better solution, an incremental change is made to the new solution,&#xA;  repeating until no further improvements can be found.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I wonder if there are differences between hill climbing method and coordinate descent/ascent method? They seem the same to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastActivityDate="2012-05-04T21:50:18.523" Title="Hill climbing and coordinate descent/ascent" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2128" PostTypeId="2" ParentId="2121" CreationDate="2012-05-04T19:01:46.180" Score="4" Body="&lt;p&gt;I have never heard of Blaze. However, after reading a bit in the article and searching for the namespace used, I found the physics library at one of the author's &lt;a href=&quot;http://www10.informatik.uni-erlangen.de/~iglberger/&quot; rel=&quot;nofollow&quot;&gt;home page&lt;/a&gt;. As the paper says on p. 2&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This “smart” ET&#xA;  methodology is implemented in the Blaze library that was developed in context of the pe physics engine&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which I take to mean that Blaze is the part of this PE library which does matrix computations. While the website says that the library has &quot;complete documentation&quot;, I was unable to find neither library nor documentation on the web site. I think your only option is sending an email.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2012-05-04T19:01:46.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2129" PostTypeId="2" ParentId="2126" CreationDate="2012-05-04T19:18:47.673" Score="1" Body="&lt;p&gt;Going by the definitions of &lt;a href=&quot;http://en.wikipedia.org/wiki/Hill_climbing&quot; rel=&quot;nofollow&quot;&gt;hill climbing&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Coordinate_descent&quot; rel=&quot;nofollow&quot;&gt;coordinate descent&lt;/a&gt; in wikipedia, they're both very similar: each operates on every input variable of the function of interest separately. The main difference between the two methods appears to be in how they adjust those input variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In basic hill climbing, each variable is first adjusted by a predefined quantity, and then the function value is tested for improvement, like a very simple Monte Carlo algorithm. On the other hand, in coordinate descent a &lt;a href=&quot;http://en.wikipedia.org/wiki/Line_search&quot; rel=&quot;nofollow&quot;&gt;line search&lt;/a&gt; is performed along each variable in order to directly find the local minima/maxima of the function value with respect to that variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, this means that in a single iteration of hill climbing each variable will be incremented by either 0 or a fixed quantity, whereas in coordinate descent each variable will be incremented by some value between 0 and the endpoint of the line search. In their vanilla forms, it seems like coordinate descent is best applied to functions on continuous variables, while hill climbing can be applied to both continuous and discrete variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the literature, the two methods seem to get conflated somewhat, although it appears that coordinate descent is most often used to descrive this kind of search when it is performed along the axes of a Euclidean space.&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="34" LastEditDate="2012-05-04T20:34:24.943" LastActivityDate="2012-05-04T20:34:24.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2130" PostTypeId="2" ParentId="2126" CreationDate="2012-05-04T19:48:09.767" Score="6" Body="&lt;p&gt;Hill climbing = optimization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coordinate descent is ''search in turn along each coordinate'', changing one variable at a time, and is a very specific optimization technique. (Very slow, too, except for separable problems.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But please read more substantial stuff than wikipedia to see how optimization is done and how the terminology is actually used, rather than asking here about everything that wikipedia leaves open. This site is not a language course in optimization!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Online references to some excellent books/surveys/papers/websites on optimization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Non-heuristic (Usually requiring gradients or subgradients):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;J. Nocedal and S.J. Wright,Numerical Optimization, 2006.&lt;BR&gt;&#xA;&lt;a href=&quot;http://home.agh.edu.pl/~pba/pdfdoc/Numerical_Optimization.pdf&quot;&gt;http://home.agh.edu.pl/~pba/pdfdoc/Numerical_Optimization.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;S. Boyd and L. Vandenberghe, Convex Optimization, 2004.&lt;BR&gt;&#xA;&lt;a href=&quot;http://www.stanford.edu/~boyd/cvxbook&quot;&gt;http://www.stanford.edu/~boyd/cvxbook&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yu. Nesterov, Introductory lectures on convex optimization, 2003.&lt;BR&gt;&#xA;&lt;a href=&quot;http://www.core.ucl.ac.be/~nesterov/Courses/INMA2460/Intro-nl.pdf&quot;&gt;http://www.core.ucl.ac.be/~nesterov/Courses/INMA2460/Intro-nl.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Neumaier, Complete Search in Continuous Global Optimization and Constraint &#xA;Satisfaction, 2004&lt;BR&gt; &#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/papers.html#glopt03&quot;&gt;http://www.mat.univie.ac.at/~neum/papers.html#glopt03&lt;/a&gt; (this is by me)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B. Derivative-free optimization (mostly heuristic)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L.M. Rios and N.V. Sahinidis,&#xA;Derivative-free optimization: A review of algorithms and comparison &#xA;of software implementations. &lt;BR&gt;&#xA;&lt;a href=&quot;http://thales.cheme.cmu.edu/dfo/dfo.pdf&quot;&gt;http://thales.cheme.cmu.edu/dfo/dfo.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;N. Hansen et al. Comparing Results of 31 Algorithms from the Black-Box&#xA;Optimization Benchmarking BBOB-2009.&lt;BR&gt;&#xA;&lt;a href=&quot;http://www.lri.fr/~hansen/gecco09-results-2010.pdf&quot;&gt;http://www.lri.fr/~hansen/gecco09-results-2010.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;M. Lozano,  D. Molina,  C. García-Martínez,  F. Herrera&#xA;Evolutionary Algorithms and other Metaheuristics for Continuous Optimization Problems.&lt;BR&gt;&#xA;&lt;a href=&quot;http://sci2s.ugr.es/eamhco/&quot;&gt;http://sci2s.ugr.es/eamhco/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C: Repository of papers on optimization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A repository of eprints about optimisation and related topics.&#xA;&lt;a href=&quot;http://www.optimization-online.org/&quot;&gt;http://www.optimization-online.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;D: Online access to mostly excellent solvers&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NEOS Solvers - NEOS Server for Optimization&#xA;&lt;a href=&quot;http://www.neos-server.org/neos/solvers/index.html&quot;&gt;http://www.neos-server.org/neos/solvers/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-05-04T21:50:18.523" LastActivityDate="2012-05-04T21:50:18.523" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2131" PostTypeId="2" ParentId="2121" CreationDate="2012-05-04T21:00:36.153" Score="9" Body="&lt;h3&gt;Original answer (May 2012)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;As far as I am concerned, the Blaze library has not been publicly released.  A link to the software as well as the license for its use should have both been in the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in a modern, freely available, numerical linear algebra library that heavily leverages expression templates, I recommend &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigen&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Update (August 2012)&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Blaze has been released, see &lt;a href=&quot;https://scicomp.stackexchange.com/a/3145/9&quot;&gt;Jack's answer&lt;/a&gt; for more details.  I still personally recommend Eigen for those interested in working in numerical linear algebra with C++, but I'm glad to see a new openly released package in this space.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-08-28T09:19:32.940" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2132" PostTypeId="2" ParentId="2114" CreationDate="2012-05-05T18:34:00.703" Score="15" Body="&lt;p&gt;It's quite common in computational fluid dynamics to use implicit schemes similar to what you propose.  The ones I know of are based on compact finite difference formulas (not simply on replacing $n$ with $n+1$ in existing schemes).  For instance, one of the most widely used schemes was developed by &lt;a href=&quot;http://www.math.colostate.edu/~yzhou/course/math750_fall2009/Lele_1992JCP.pdf&quot;&gt;Lele in 1992 in this paper&lt;/a&gt; with &gt;2500 citations.  Such schemes can be made to have better dispersive properties than typical explicit schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upwinding is usually less important when using implicit methods and large time step sizes, because the huge amount of diffusion (mentioned by Jeremy) means you can't resolve shocks anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the particular scheme you propose: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It can be obtained from a method-of-lines discretization by using a backward difference in space and the backward (implicit) Euler method in time.  &lt;/li&gt;&#xA;&lt;li&gt;It is unconditionally stable as long as $u\ge0$ (interestingly, it's also stable for $u&amp;lt;0$ if the time step is not too &lt;strong&gt;small&lt;/strong&gt;!)  &lt;/li&gt;&#xA;&lt;li&gt;It is more dissipative than the traditional explicit upwind scheme.&lt;/li&gt;&#xA;&lt;li&gt;Unlike the explicit upwind scheme, it does not satisfy the unit CFL condition (i.e., it is not exact in the case that $\tau u/h = 1$).  Instead it satisfies the anti-unit CFL condition (it is exact if $\tau u/h = -1$).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-05-05T18:43:20.377" LastActivityDate="2012-05-05T18:43:20.377" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2133" PostTypeId="1" AcceptedAnswerId="2581" CreationDate="2012-05-05T20:32:42.550" Score="12" ViewCount="1353" Body="&lt;p&gt;I was wondering if anybody had any suggestions for texts or survey articles on decomposition methods (e.g. primal, dual, Dantzig–Wolfe decompositions) for solving large mathematical programming problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I liked Stephen Boyd's &lt;a href=&quot;http://see.stanford.edu/materials/lsocoee364b/08-decomposition_notes.pdf&quot;&gt;&quot;Notes on Decomposition Methods&quot;&lt;/a&gt;, and it would be great to find for example a textbook that covers this topic in more detail.&lt;/p&gt;&#xA;" OwnerUserId="13" LastActivityDate="2013-06-21T11:29:02.430" Title="Decomposition methods for solving large optimization problems" Tags="|optimization|linear-programming|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2134" PostTypeId="1" AcceptedAnswerId="2135" CreationDate="2012-05-05T22:00:25.670" Score="7" ViewCount="1283" Body="&lt;p&gt;I need to do a large number of matrix operations in Matlab, but one of the matrices is 4-D and my normal instincts for correct vectorization are failing me. Right now I'm using a loop. Maybe somebody can fix my dumb brain?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k=1:kNum&#xA;   for l=1:lNum&#xA;      L(k,l,:) = squeeze(M(k,l,:,:))\squeeze(P(k,l,:));&#xA;   end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22" LastEditorUserId="22" LastEditDate="2012-05-05T22:28:22.423" LastActivityDate="2012-05-07T08:29:35.407" Title="How do I vectorize this 4-D matrix operation in Matlab?" Tags="|matlab|vectorization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2135" PostTypeId="2" ParentId="2134" CreationDate="2012-05-06T20:50:13.183" Score="8" Body="&lt;p&gt;I don't think that it is possible to vectorize this expression very much. Generally, vectorization is used with assignments to vectors along with functions which output seperate results for each item, for instance&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i = 1:N&#xA;   A(i) = sin(i)&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is equal to &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;i = 1:N&#xA;A(i) = sin(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;because sin does elementwise sine of the elements. When solving linear equation sets using mldivide/&quot;backslash operator&quot; the solution depends on all the elements in the vector - you can't just do element wise mldivide unless the matrix has special properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For optimizing this loop, you could consider using parfor-loops (the looping variables are normal numbers incremented by one, so it should be as simple as doing &quot;parfor&quot; instead of &quot;for). This requires the parallel computing toolbox. You could also unroll the loops to get better performance, if the numbering of the indices is somewhat predictible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, you could try to write an .mex-file doing the same operation. In C/Fortran looping is as fast as / faster than MATLAB vectorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing to consider is reordering the memory access: Since matrices are just long arrays accessed using a [i + j*dim1 + k*dim1*dim2] pattern it is important to ensure that the inner loop corresponds to the memory which is stored sequentially. For 2D matrices, at least, doing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k=1:kNum&#xA;   for l=1:lNum&#xA;       L(k,l,:) = squeeze(M(k,l,:,:))\squeeze(P(k,l,:));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is slower than&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for l= 1:lNum&#xA;   for k=1:kNum&#xA;       L(k,l,:) = squeeze(M(k,l,:,:))\squeeze(P(k,l,:));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;since the elements corresponding to the k-index are stored sequentially. &lt;a href=&quot;http://www.mathworks.se/company/newsletters/news_notes/june07/patterns.html&quot;&gt;Mathworks has some information on it&lt;/a&gt;, but it is not really something specific to MATLAB per se, any somewhat advanced text on arrays for C or FORTRAN should go into more details. There is also some cache stuff going on under the hood here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: While not useful here as the right hand side and left hand side is different in each solution, if you are doing something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i = 1:N&#xA;    solution(:,i) = A\b(:,i)&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you might as well do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solution = A\b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1273" LastEditorUserId="1273" LastEditDate="2012-05-07T08:29:35.407" LastActivityDate="2012-05-07T08:29:35.407" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2137" PostTypeId="1" AcceptedAnswerId="2140" CreationDate="2012-05-07T06:20:19.230" Score="11" ViewCount="2359" Body="&lt;p&gt;How much slower is petsc4py vs c/c++/fortran?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize it will depend significantly on the code being executed, but what about something simple like a matrix-vector product?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2012-05-07T10:47:29.093" Title="Efficiency of using petsc4py vs. c/c++/fortran" Tags="|petsc|python|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2138" PostTypeId="2" ParentId="816" CreationDate="2012-05-07T06:21:10.273" Score="2" Body="&lt;p&gt;The over-powered convex analysis way to show this is just take subgradients. In fact this is equivalent to the reasoning used in some of the other answers involving slopes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimization problem is convex (because the objective is convex and there are no constraints.) Also, the subgradient of $\left|m-x_i\right|$ is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-1 if $m&amp;lt;x_i$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[-1,1] if $m=x_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;+1 if $m&amp;gt;x_i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since a convex function is minimized if and only if it's subgradient contains zero, and the subgradient of a sum of convex functions is the (set) sum of the subgradients, you get that 0 is in the subgradient if and only if $m$ is the median of $x_1,\ldots x_k$. &lt;/p&gt;&#xA;" OwnerUserId="1420" LastActivityDate="2012-05-07T06:21:10.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2139" PostTypeId="2" ParentId="2137" CreationDate="2012-05-07T09:01:24.833" Score="7" Body="&lt;p&gt;Petsc4py is just another way to acces PETSc but &lt;em&gt;from python&lt;/em&gt;, or is the same to say that, petsc4py provides the bindings so that, from python, you can access PETSc data structures and routines that are meant to reduce the effort of developing parallel PDE solvers (that scale).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc provides several levels of abstractions to their solvers, and you can even use PETSc to implement your own solver. At the lowest level of software abstraction, PETSc uses BLAS, LAPACK, and MPI, and at best it will be as fast as the implementation of these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, pets4py uses cython to implement the bindings to PETSc. The overhead of using cython is relative to how much computations are going to be done from PETSc. If you use the high level PDE solvers from PETSc, the overheads should be small enough that you don't need to worry about them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A maybe more important question than the performance comparison of PETSc vs GEMV is if PETSc is the right tool for your job. If you need to implement non-trivial parallel PDE solvers, then most probably, PETSc will really help you. However, if you need to do a bunch of GEMV, you want a BLAS library. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="1070" LastEditDate="2012-05-07T09:19:00.833" LastActivityDate="2012-05-07T09:19:00.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2140" PostTypeId="2" ParentId="2137" CreationDate="2012-05-07T10:47:29.093" Score="11" Body="&lt;p&gt;This is a widely-held concern in the scientific programming community, and I would consider the performance uncertainty to be one of the major &quot;myths&quot; in computational science.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As &lt;a href=&quot;https://scicomp.stackexchange.com/a/2139/9&quot;&gt;@fcruz&lt;/a&gt; discusses, &lt;code&gt;petsc4py&lt;/code&gt; is a wrapper to the PETSc libraries, not a reimplementation of PETSc in Python.  Therefore, you can expect any performance penalties to come from either copying arrays to and from PETSc, or from overhead in your driver code/function calls. &lt;code&gt;petsc4py&lt;/code&gt; is very carefully implemented, and as long as you understand the &lt;code&gt;numpy&lt;/code&gt; multi-dimensional array interfaces, you can avoid copy overhead. For the majority of use cases I work in, the performance penalty in working in Python is on the order of 10-40%, and I often gain substantially in other ways that more than make up for this performance hit.  In fact, several more experienced HPC Python developers I have talked with hold the opinion that this performance difference can usually be reduced even further, and when Python is driving computationally expensive codes, this will certainly be the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;petsc4py&lt;/code&gt; repository itself features a number of useful examples to illustrate the performance/flexibility tradeoff.  Look in the &lt;code&gt;petsc4py&lt;/code&gt; source repository for the demo called &lt;a href=&quot;http://code.google.com/p/petsc4py/source/browse/demo/perftest&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;perftest&lt;/code&gt;&lt;/a&gt;, which solves a nonlinear system of equations using both a Python driver and a C driver (over a Fortran kernel provided in &lt;code&gt;App.f90&lt;/code&gt; in that directory).  The performance overhead here is on the order of 10%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a concrete example, I am part of a team of scientists working on &lt;a href=&quot;http://numerics.kaust.edu.sa/pyclaw/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyClaw&lt;/a&gt;, a software package that interfaces into PETSc for parallel grid management and legacy Fortran kernels for solving Riemann problems on cell interfaces.  We performed a fairly careful study of the performance degradation from switching over from a Fortran driver, and you can see the results on the bottom of page 5 in Table 1 in the &lt;a href=&quot;http://numerics.kaust.edu.sa/papers/petclaw2011/petclaw2011.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;conference paper&lt;/a&gt;.  In our case, we traded a little bit of on-core performance for the capability to easily interface our code to PETSc and Fortran and run efficiently in parallel on tens of thousands of cores.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-05-07T10:47:29.093" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2141" PostTypeId="1" AcceptedAnswerId="2142" CreationDate="2012-05-07T10:51:39.043" Score="2" ViewCount="242" Body="&lt;p&gt;I have functions defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f1(A) = \sum\|x_i-x_j\|_A = \sum\sqrt{(x_i-x_j)^TA(x_i-x_j)}$ and $f2(A) = \sum\|x_k-x_l\|^2_A$ where A is PSD matrix, x are number vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Task is to minimize function $f(A) = f2(A)$ subject to: $f1(A) \geq 1$ and $A \succeq 0$ . I can compute its gradient and hessian to use in Newton's method for minimization (which i want to use). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how can i incorporate above constraints (to be satisfied) to the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can be first constraint solved by rewriting the main function $f(A)$ to contain slack variable -&gt; something like: $f(A) = f2(A) + 1 - f1(A)$ ?&lt;/p&gt;&#xA;" OwnerUserId="1428" LastActivityDate="2012-05-07T20:46:18.487" Title="Solving PSD matrix in Newton's method" Tags="|optimization|linear-programming|convex-optimization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2142" PostTypeId="2" ParentId="2141" CreationDate="2012-05-07T15:21:08.700" Score="1" Body="&lt;p&gt;insert $A=R^TR$ with upper traingual $R$ to get rid of the semidefinite constraint (and reformulate $f_2(A)$ as the trace of $R BR^T$ to save function evaluation cost). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then solve the resulting constraint problem using a constrianed optimization routine, as you cannot eliminate the constraint on $f_1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For optimization software, see my page &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/software_l.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/glopt/software_l.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-07T15:21:08.700" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2143" PostTypeId="2" ParentId="2064" CreationDate="2012-05-07T17:28:58.340" Score="7" Body="&lt;p&gt;Currently MPI has no provisions for prioritisation of messages and neither has the upcoming MPI 3.0 standard. It is up to the MPI implementation to decide how to transmit the messages. E.g. smaller messages &lt;em&gt;might&lt;/em&gt; get sent faster because of certain bypasses in the communication machinery (highly implementation and system dependent). You &lt;em&gt;might&lt;/em&gt; be able to exploit the fact that most MPI implementations break large messages into chunks and smaller messages &lt;em&gt;might&lt;/em&gt; be able to slip between the chunks of the large ones. But, once again, this is highly implementation dependent and I wouldn't rely on that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a simple experiment using Open MPI 1.5.3 over InfiniBand connection. The program sends one very large message (1 GiB) with &lt;code&gt;MPI_Isend&lt;/code&gt; and then two short messages (16 bytes) with &lt;code&gt;MPI_Send&lt;/code&gt;, and afterwards it waits for the big send to complete with &lt;code&gt;MPI_Wait&lt;/code&gt;. On the other side an &lt;code&gt;MPI_Irecv&lt;/code&gt; is first posted for the big receive and then two subsequent &lt;code&gt;MPI_Recv&lt;/code&gt; operations, followed by &lt;code&gt;MPI_Wait&lt;/code&gt; for the big receive. I was consistently able to receive the two short messages before the receive of the big message has been completed. Here is the output of my test:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0] Rank 0 running on host1&#xA;[0] Starting big send at 0.000019s&#xA;[0] Starting small send at 0.215448s&#xA;[0] Starting small send 2 at 0.224105s&#xA;[0] Starting wait at 0.224114s&#xA;[0] Finished wait at 0.935843s&#xA;[1] Rank 1 running on host2&#xA;[1] Starting big receive at 0.000020s&#xA;[1] Starting small recv at 0.000037s&#xA;[1] Starting small recv 2 at 0.548396s&#xA;[1] Starting wait at 0.548418s&#xA;[1] Finished wait at 0.935780s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both small sends succeed before the async send has completed as evident from the waiting time of ~700 ms. I would say that the first small receive succeeds some time (~300 ms) after the big receive has started in the background. I tried this using only &lt;code&gt;MPI_COMM_WORLD&lt;/code&gt; or using a separate communicator for the small messages - results are the same. Nodes have one QDR IB HCA each and running with &lt;code&gt;--mca btl_base_verbose 50&lt;/code&gt; confirms that there are no alternative communication channels in use.&lt;/p&gt;&#xA;" OwnerUserId="1423" LastActivityDate="2012-05-07T17:28:58.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2145" PostTypeId="2" ParentId="2119" CreationDate="2012-05-07T21:18:23.690" Score="4" Body="&lt;p&gt;The difference in terminology between &quot;search&quot; and &quot;optimize&quot; comes from the fact that searching refers to the process of finding an $x^\ast$ so that for a given $g(x)$ we have $g(x^\ast)=0$, i.e. we search for a root. In optimization, we want to find an $x^\ast$ so that $f(x) \rightarrow \textrm{min!}$. At least if $f$ is smooth, then finding this minimum is typically converted to the problem of finding a root for $g(x)=\nabla f(x)$. In other words, the term &quot;searching&quot; comes from a more general problem, but for optimization problems things that deal with optimization are often reduced to things that deal with searching.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-07T21:18:23.690" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2146" PostTypeId="1" AcceptedAnswerId="2148" CreationDate="2012-05-07T22:52:10.687" Score="25" ViewCount="5127" Body="&lt;p&gt;An Amazon EC2 compute cluster costs about \$800-\$1000 (depending on duty cycle) per physical CPU core over the course of 3 years. In our last round of hardware acquisition, my lab picked up 48 cores worth of hardware very similar to that of Amazon's clusters for about ~$300 a core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something here? Are there any situations in which it makes economic sense to build a cluster in the cloud for high CPU tasks such as molecular dynamics simulations? Or am I always better off just building and baby-sitting the dang machine myself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I should mention that my lab doesn't pay for electricity in our server room (at least not directly), but even with this benefit Amazon still seems extremely expensive).&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="34" LastEditDate="2012-05-07T23:04:50.703" LastActivityDate="2012-05-09T19:02:46.140" Title="When is building a cluster in the cloud cheaper than building one in my lab for MD simulations?" Tags="|cloud-computing|hpc|" AnswerCount="5" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2147" PostTypeId="2" ParentId="2146" CreationDate="2012-05-08T08:40:49.487" Score="7" Body="&lt;p&gt;There are some things to worry about when doing cloud computing with MD simulations. For instance, you need to worry about the physical layout of the processors in the server farm where these jobs will be running. The reason is that, depending upon the size of your simulations, and the kinds of calculations you're running (for instance, systems with electrostatics), you may be heavily reliant on FFT's—and pushing electrons around to different processors in a gigantic cluster could become a very time-consuming part of the total compute time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for something as data-intensive as MD, you'll want to make sure that you have fast upload and download connections to the servers, as well as sensible limits on data storage. Otherwise, a lot of the cost savings could get sucked away in lost productivity and storage fees. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, our institute just purchased about ~240 cores for our local cluster at a cost of under 500€ per core. That cost includes hosting and administration, plus service, on our campus for four years. On an annualized basis that seems ridiculously cheap. I think that's probably the best of both worlds—local access, but professionally maintained without needing our own IT team.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-05-08T08:40:49.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2148" PostTypeId="2" ParentId="2146" CreationDate="2012-05-08T08:51:52.350" Score="16" Body="&lt;p&gt;The main advantage, in my opinion, of using Cloud-based resources is flexibility, i.e. if you have a fluctuating workload, you only pay for what you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not the case in your application, i.e. you know you will have a quantifiable and constant workload, then you're probably better-off building your own cluster. In the Cloud, you pay for flexibility, and if you don't need flexibility, you would be paying for something you don't need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your workload is flexible but somewhat intense and relies on certain hardware features (see aeismail's answer), you may want to try sharing a cluster with other people in your university to amortize the idle cycles. My old university runs such a shared cluster with a &quot;Shareholder Model&quot; in which every group is guaranteed a share of the computing power proportional to their investment in the hardware and idle cycles can be used by anyone. The only difficulty is centralizing the cluster administration.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-05-08T08:51:52.350" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2149" PostTypeId="2" ParentId="2146" CreationDate="2012-05-08T13:04:25.390" Score="6" Body="&lt;p&gt;I have no firsthand experience with cloud services like Amazon's EC2, but the actual cost per core is likely much greater than you cite: it's the cost of initial purchase, electricity, cooling, space in a building, replacement hardware. Plus the cost of administration: setting up the OS and cluster services, keeping the OS up to date, troubleshooting the queue, etc. I would not at all be surprised if the sum of all of this is twice the cost of initial purchase. Of course you gain flexibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, the model comes down to scale: If you have a truly large cluster (1000 cores or more) then you can amortize work time, repairs, system administration because there is enough to do to keep a professional busy. If you have a small cluster where it's not worth having a dedicated person do it, then it's likely that you make someone do it whose first job should be to do science, and in that case this person's time is poorly spent on such adminstratorial jobs. This is where services-on-demand such as cloud servers shine.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-08T13:04:25.390" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2150" PostTypeId="1" AcceptedAnswerId="2151" CreationDate="2012-05-08T13:59:57.923" Score="6" ViewCount="1211" Body="&lt;p&gt;From &lt;a href=&quot;http://en.wikipedia.org/wiki/Stochastic_approximation#Robbins.E2.80.93Monro_algorithm&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;, assume that we have a function $M(x)$, and we want to solve the equation $M(x) = 0$. But we cannot directly observe the function $M(x)$, we can instead obtain measurements of the random variable $N(x)$ where $E[N(x)] = M(x)$.  The Robbins–Monro algorithm is to solve this problem by generating iterates of the form:&#xA;$$&#xD;&#xA;        x_{n+1}=x_n-a_n N(x_n)&#xD;&#xA;$$&#xA;where $a_1, a_2, \dots$ is a sequence of positive step sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If considering solving the deterministic version of the equation instead, ie solving $M(x)=0$ when $M(x)$ can be observed directly, I wonder:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there already a type of algorithm similar to the Robbins–Monro&#xA;algorithm? Is it $$&#xD;&#xA;            x_{n+1}=x_n-a_n M(x_n) $$ where $a_1, a_2, \dots$ is a sequence of positive step sizes? I couldn't find such an algorithm&#xA;in the resources that I can access.&lt;/li&gt;&#xA;&lt;li&gt;If such an algorithm in 1 can work, what is its&#xA;rationale/intuition/motivation of $x_{n+1}=x_n-a_n M(x_n)$? By&#xA;rationale/intuition/motivation, I mean, for example, tangent line&#xA;approximation in Newton's method for solving equation, and steepest&#xA;descent in gradient descent method for optimization.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The reason of asking this question is that I think most, if not all, stochastic approximation algorithms are inspired from some algorithms for the similar deterministic cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="258" LastEditDate="2012-05-09T01:34:42.220" LastActivityDate="2017-01-02T08:20:35.007" Title="What is the deterministic counterpart of Robbins-Monro algorithm?" Tags="|optimization|stochastic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2151" PostTypeId="2" ParentId="2150" CreationDate="2012-05-08T15:49:53.233" Score="5" Body="&lt;p&gt;In the deterministic case, you can of course run the same algorithm, but it is very inefficient compared to quasi-Newton (Broyden type) methods. There is little point to investigate the properties of so poor an algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, broyden's method is quite sensitive to noise, hence cannot be easily adapted to the stochastic case. Moreover, if the amount of noise is large then convergence is dictated by the law of large numbers anyway, and there is little to be gained from trying to adapt algorithms with superlinear convergence.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-08T15:49:53.233" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2152" PostTypeId="1" AcceptedAnswerId="2153" CreationDate="2012-05-08T22:17:15.090" Score="30" ViewCount="1709" Body="&lt;h3&gt;The problem&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I'm currently working on a Finite Element Navier Stokes simulation and I would like to investigate the effects of a variety of parameters. Some parameters are specified in an input file or via a command line options; other parameters are provided as flags in a Makefile so my code has to be recompiled whenever I change those options. I would be interested to get some advice about a good way to systematically explore the parameter space.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Are there useful C++/Python libraries/frameworks that can help with this sort of thing?&lt;/em&gt; For example discovering &lt;a href=&quot;http://www.boost.org/doc/libs/1_49_0/doc/html/program_options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;boost.Program_options&lt;/a&gt; was a big help since it's possible to overload input file options with command line arguments. I have also seen some people use a job file describing each case quite effectively and a collegue suggested that writing parameters into vtu files as comment blocks could work too.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Perhaps it isn't worth investing much time in this at all?&lt;/em&gt; Is it just a distraction and a time-drain and it's best to just muscle through the testing process brute force and ad hoc?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Some thoughts&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I am currently doing things mostly by hand and I have encountered the following problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Naming test cases&lt;/em&gt;. I tried storing results in folders named with the run parameters separated with underscores e.g. &lt;code&gt;Re100_dt02_BDF1...&lt;/code&gt;. These quickly become long or difficult to read/cryptic if they are abbreviated too much . Also, real number parameters include a &lt;code&gt;.&lt;/code&gt; which is awkward/ugly.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Logging run data&lt;/em&gt;. Sometimes I would like to see the results written to the terminal and also saved to a text file. &lt;a href=&quot;https://stackoverflow.com/questions/937805/how-to-redefine-clog-to-tee-to-original-clog-and-a-log-file&quot;&gt;This answer&lt;/a&gt; from StackOverflow for instance is somewhat helpful but the solutions seem to be a bit intrusive.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Plotting data according to parameter&lt;/em&gt;. It takes quite some time collect relevant data from a variety of log files into a single file which I can then plot, with a better system perhaps this would become easier.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Recording comments on the data&lt;/em&gt;. After examining results I write some comments in a text file but keeping this is sync with the results folders is sometimes difficult.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="482" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2017-05-17T15:06:41.743" Title="What is a good way to run parameter studies in C++" Tags="|software|finite-element|python|c++|" AnswerCount="9" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2153" PostTypeId="2" ParentId="2152" CreationDate="2012-05-08T22:25:17.863" Score="10" Body="&lt;p&gt;Just some comments on two of your points:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Logging run data&lt;/em&gt;: Your best bet is probably piping output through the &lt;a href=&quot;http://en.wikipedia.org/wiki/Tee_%28command%29&quot;&gt;tee&lt;/a&gt; command, which should be available in most shells.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Plotting data according to parameter&lt;/em&gt;: I guess it's a matter of taste, but when I have to do complex data aggregation, I store the results in plain text, read them into Matlab as matrices, and do all the computations, plotting and even LaTeX output from there. Obviously, whatever programming/scripting language you're most familiar with will give you best results.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-05-08T22:25:17.863" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2154" PostTypeId="2" ParentId="2152" CreationDate="2012-05-09T06:18:51.313" Score="11" Body="&lt;p&gt;If you want to write something general-purpose, you can do it either with shell scripts if it is something very simple, as &lt;a href=&quot;https://scicomp.stackexchange.com/a/2153/9&quot;&gt;Pedro suggests&lt;/a&gt;, or aggregate in a higher-level mathematical programming language such as Python or MATLAB.  I agree that plain text files are useful for smaller amounts of data, but you should probably switch to binary data for anything larger than a few megabytes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you are &lt;strong&gt;just&lt;/strong&gt; doing parameter estimation, I would recommend using a piece of software specifically suited for this.  Several researchers at my University have had good luck with &lt;a href=&quot;http://www.cs.sandia.gov/optimization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAKOTA&lt;/a&gt;, an Uncertainty Quantification toolbox out of Sandia National Laboratories (&lt;a href=&quot;http://dakota.sandia.gov/license.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;available under a GNU Lesser General Public License&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an excerpt from the Sandia page describing DAKOTA:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We provide a variety of methods to allow a user to run a collection of computer simulations to assess the sensitivity of model outputs with respect to model inputs. Common categories include parameter studies, sampling methods and design of experiments. In parameter studies one steps some input parameters through a range while keeping other input parameters fixed and evaluates how the output varies. In sampling methods, one generates samples from an input space distribution and calculates the output response at the input values. Specific sampling methods available within DAKOTA include Monte Carlo, Latin Hypercube, and (coming soon) quasi-Monte Carlo. In design of experiments the output is evaluated at a set of input &quot;design&quot; points chosen to sample the space in a representative way. Specific design of experiment methods available within DAKOTA include Box-Behnken, Central Composite, and Factorial designs. Sensitivity metrics are a mathematical way of expressing the dependence of outputs on inputs. A variety of sensitivity metrics are available within Dakota, such as simple and partial correlation coefficients, and rank correlations. Our current research focuses on methods to generate sensitivity metrics with a minimal number of runs, and on optimal estimation of parameters in computer models using Bayesian analysis techniques. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-05-09T06:18:51.313" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2156" PostTypeId="2" ParentId="2146" CreationDate="2012-05-09T09:08:58.340" Score="0" Body="&lt;p&gt;There are a lot of god answers already. I just want to add two small points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) I know that IBM is offering something similar. So depending on your application it might be worth comparing the different offers. Not only with respect to the cost but also the equipment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) Of course most labs are too small to be able to invest into their own cluster. Thus the question is whether it is not possible to gather several labs together and invest in a cluster that is shared between the different labs. This has besides the obvious cost sharing effect also the advantage of a more even load distribution.&lt;/p&gt;&#xA;" OwnerUserId="1176" LastActivityDate="2012-05-09T09:08:58.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2157" PostTypeId="2" ParentId="2146" CreationDate="2012-05-09T09:28:38.720" Score="6" Body="&lt;p&gt;As a supplement to some of the already excellent answers, there's another factor to consider:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Regardless of the costs, how are you going to pay for it?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I've encountered a non-trivial number of grants that &lt;em&gt;will not&lt;/em&gt; under any circumstances pay for hardware expenses, but will pay for computing time on something like EC2. So under some funding circumstances, while you might be able to fund a small &quot;testbed&quot; cluster with unstructured funds or a lab startup package, for larger-scale projects it may be the only way to have your computing costs funded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the NIH:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ADP/Computer Services: The services you include here should be research specific computer services- such as reserving computing time on supercomputers or getting specialized software to help run your statistics. This section should not include your standard desktop office computer, laptop, or the standard tech support provided by your institution. Those types of charges should come out of the F&amp;amp;A costs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;While it's possible to put cluster machines down under the $5,000+ equipment heading, and you can make a good argument for it, I've found both reviewers who are skittish about it, and universities that are hesitant about the ongoing costs of maintaining such a system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some grants are even more strict. One grant I currently have reads as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Funds may also not be used for computer hardware&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's often simply easier to get a cluster paid for by direct costs if its EC2-based or one of its many analogs than actually buying the hardware, especially if your institution is stingy with the indirect costs. This may not be the case for you, but its the case for some.&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="110" LastEditDate="2012-05-09T19:02:46.140" LastActivityDate="2012-05-09T19:02:46.140" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2158" PostTypeId="1" CreationDate="2012-05-09T12:33:20.920" Score="3" ViewCount="390" Body="&lt;p&gt;Given some data points in 3D, $X\in\mathbb{R}^{n\times 3}$, could one say that&#xA;$$Y=XP,$$ for some $P\in\mathbb{R}^{3\times 2}$ actually corresponds to a &lt;em&gt;particular viewpoint&lt;/em&gt; on a 3D data? Basically, does any $P$ correspond to a &lt;em&gt;viewpoint&lt;/em&gt;, which is a 2D projection?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to understand the relation, and I'm interested in whether one is left with more degrees of freedom by choosing $P$, or simply by choosing a &lt;em&gt;viewpoint&lt;/em&gt;. What is the relation between the two? &lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-05-09T16:30:27.643" LastActivityDate="2012-05-09T17:23:35.447" Title="3D to 2D projections, a generalization" Tags="|linear-algebra|matrix|high-dimensional|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2159" PostTypeId="1" CreationDate="2012-05-09T15:01:20.820" Score="5" ViewCount="659" Body="&lt;p&gt;How to estimate $$\max_i |\mathrm{Arg}(-\lambda_i)|,$$ where  $\{\lambda_i\}$ are eigenvalues of a large sparse matrix $A$ all lying in the left complex half-plane?&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="720" LastEditDate="2012-05-09T17:04:44.907" LastActivityDate="2012-05-11T05:47:54.253" Title="How to estimate the maximum argument of eigenvalues?" Tags="|linear-algebra|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2160" PostTypeId="2" ParentId="2158" CreationDate="2012-05-09T17:23:35.447" Score="3" Body="&lt;p&gt;In order to answer your question, I need to clarify a few things, especially because I've noticed that in at least three of your previous questions, the word &quot;projection&quot; tends to be used very loosely. Projection is a technical term, and as such, it's vitally important that it have a precise meaning, so here it is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Definition&lt;/em&gt;: A &lt;em&gt;projection matrix&lt;/em&gt; $P$ is a matrix $\mathbb{R}^{n \times n}$ for some $n$ such that $P^2 = P$. (cf. &lt;a href=&quot;http://en.wikipedia.org/wiki/Projection_%28linear_algebra%29&quot; rel=&quot;nofollow&quot;&gt;Wikipedia, projection (linear algebra)&lt;/a&gt; and also &lt;em&gt;Generalized Inverses: Theory and Applications, Second Edition&lt;/em&gt;, p.58-64; Strang's books on linear algebra supposedly have a nice discussion of projection matrices also.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usage of &quot;projection&quot; in the sense of mapping from $\mathbb{R}^{n}$ to $\mathbb{R}^{m}$ also occurs in the literature, but its use is confusing, at best, especially since that notion is strongly related to (and thus conflated with) the definition above. (See &lt;a href=&quot;http://en.wikipedia.org/wiki/Projection_%28mathematics%29&quot; rel=&quot;nofollow&quot;&gt;Wikipedia, projection (mathematics)&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a projection matrix $P$, then its range $\mathcal{R}(P)$ and nullspace $\mathcal{N}(P)$ are complementary subspaces, which is to say that $\mathcal{R}(P) \cap \mathcal{N}(P) = \{0\}$, and any vector in $\mathbb{R}^{n}$ can be decomposed &lt;em&gt;uniquely&lt;/em&gt; into the sum of a component in $\mathcal{R}(P)$ and a component in $\mathcal{N}(P)$. This relationship is denoted notationally as a direct sum: $\mathcal{R}(P) \oplus \mathcal{N}(P) = \mathbb{R}^{n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you take a vector $x$ and project it onto $\mathcal{R}(P)$ along $\mathcal{N}(P)$, you are calculating $y = Px$. Let $\mathcal{R}(P)$ have dimension $m \leq n$. If we change coordinate systems, points in the subspace $\mathcal{R}(P)$ can be represented such that only $m$ coordinates have nonzero values; that is, we could represent them as a point in $\mathbb{R}^{m}$ instead, leading to the irritatingly confusing abuse of terminology &quot;projecting from $n$-D to $m$-D&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $P$ can be decomposed as $P = XY^{*}$ (see G. W. Stewart's paper &lt;a href=&quot;http://epubs.siam.org/sima/resource/1/sjmael/v32/i1/p309_s1&quot; rel=&quot;nofollow&quot;&gt;&quot;On the Numerical Analysis of Oblique Projectors&quot;&lt;/a&gt;, &lt;em&gt;SIAM Journal of Matrix Analysis and Applications&lt;/em&gt;, Vol. 32, pp. 309-342)), such that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$X \in \mathbb{R}^{n \times m}$&lt;/li&gt;&#xA;&lt;li&gt;$\mathcal{R}(X) = \mathcal{R}(P)$&lt;/li&gt;&#xA;&lt;li&gt;$Y \in \mathbb{R}^{n \times m}$&lt;/li&gt;&#xA;&lt;li&gt;$Y \in \mathcal{R}(P^{*})$&lt;/li&gt;&#xA;&lt;li&gt;$Y^{*} \in \mathcal{N}(P)$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Such a decomposition is not unique; for given values of $X$ and $Y$ satisfying the relationships above, $Y^{*}$ is the mapping from $\mathbb{R}^{n}$ to $\mathbb{R}^{m}$ loosely referred to as a &quot;projection from $n$-D to $m$-D&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specific relationship you're talking about would be where $n = 3$ and $m = 2$; depending on how you pick the range and nullspace, you'll get a different mapping of your 3D points onto 2D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the range and nullspace only refer to perspective (the direction from where you're viewing) and not the viewpoint (the exact location you're viewing from, along that direction). The viewpoint is more like a shift of origin (if you can shift your coordinate system however you want, then this shift doesn't matter, but in a lot of concrete applications, it does). For a given $x \in \mathbb{R}^{n}$, the map you want would look something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\tilde{x} = Y^{*}(x - x_{0}),&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\tilde{x} \in \mathbb{R}^{m}$, and $x_{0} \in \mathbb{R}^{n}$ corresponds to the shift in origin from $0$ to $x_{0}$.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-05-09T17:23:35.447" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2161" PostTypeId="2" ParentId="2152" CreationDate="2012-05-09T18:24:34.440" Score="0" Body="&lt;p&gt;You can check out &lt;a href=&quot;http://sfepy.org&quot; rel=&quot;nofollow&quot;&gt;sfepy&lt;/a&gt; which is a finite element program almost entirely coded in python. It also has sample Navier Stokes problem. The operating procedure of sfepy is very easy.&lt;/p&gt;&#xA;" OwnerUserId="1327" LastActivityDate="2012-05-09T18:24:34.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2162" PostTypeId="1" AcceptedAnswerId="2225" CreationDate="2012-05-09T18:58:31.263" Score="11" ViewCount="1624" Body="&lt;p&gt;I am implementing the paper &quot;&lt;a href=&quot;http://www.math.ucdavis.edu/~saito/data/emd/zhu-ijcv2004.pdf&quot;&gt;Optimal Mass Transport for Registration and Warping&lt;/a&gt;&quot;, my goal being to put it online as I just cannot find any eulerian mass transportation code online and this would be interesting at least for the research community in image processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper can be summarized as follows :&lt;br&gt;&#xA;- find an initial map $u$ using 1D histogram matchings along the x and y coordinates&lt;br&gt;&#xA;- solve for the fixed point of $u_t = \frac{1}{\mu_0} Du \nabla^\perp\triangle^{-1}div(u^\perp)$ , where $u^\perp$ stands for a 90 degrees counter clockwise rotation, $\triangle^{-1}$ for the solution of the poisson equation with Dirichlet boundary conditions (=0), and $Du$ is the determinant of the Jacobian matrix.&lt;br&gt;&#xA;- stability is guaranteed for a timestep $dt&amp;lt;\min|\frac{1}{\mu_0}\nabla^\perp\triangle^{-1}div(u^\perp)|$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For numerical simulations (performed on a regular grid), they indicate using matlab's &lt;em&gt;poicalc&lt;/em&gt; for solving the poisson equation, they use centered finite differences for spatial derivatives, except for $Du$ which is computed using an upwind scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using my code, the energy functional and curl of the mapping are properly decreasing for a couple iterations (from a few tens to a few thousands depending on the time step). But after that, the simulation explodes : the energy increases to reach a NAN in very few iterations. I tried several orders for the differentiations and integrations (a higher order replacement to cumptrapz can be found &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/9734-inverse-integrated-gradient/content/integratedgradient/intgrad1.m&quot;&gt;here&lt;/a&gt;), and different interpolation schemes, but I always get the same issue (even on very smooth images, non-zero everywhere etc.).&lt;br&gt;&#xA;Anyone would be interested in looking at the code and/or the theoretical problem I am facing ? The code is rather short.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Code with debugging functions    &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pastebin.com/VDesvexz&quot;&gt;registration function&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pastebin.com/yrnidQmn&quot;&gt;test code provided you have two images of the same size to register&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://pastebin.com/4rATNveT&quot;&gt;Just the necessary function without test stuffs (&amp;lt; 100 lines)&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please replace gradient2() at the end by gradient(). This was a higher order gradient but doesn't solve things either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am only interested in the optimal transport part of the paper for now, not the additional regularization term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks !&lt;/p&gt;&#xA;" OwnerUserId="1440" LastActivityDate="2012-05-15T07:08:00.217" Title="Optimal transport warping implementation in Matlab" Tags="|pde|finite-difference|matlab|fluid-dynamics|image-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2163" PostTypeId="2" ParentId="2159" CreationDate="2012-05-09T20:41:42.727" Score="1" Body="&lt;p&gt;You could multiply a random vector by the matrix repeatedly. The magnitude of a given eigenvalue will tend to cause the vector's magnitude to grow, and the argument will tend to cause the vector to oscillate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you store the vector after each step, and then take the componentwise FFT, the highest frequency components should correspond to the largest arguments of eigenvalues of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can show this as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let each eigenvalue $\lambda_k=r_ke^{i\theta_k}$. Then, for a given eigenvector $x_k$, the effect of the matrix applied $n$ times would be $r_k^ne^{in\theta_k}x_k$, which clearly has a peak in the fourier transform with respect to $n$ around $\theta_k$. If one has a vector which is a linear combination of eigenvectors, there will be one peak per eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method has some problems: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If the magnitude of the eigenvector with the greatest argument is small, the oscillation may be so suppressed that you are unable to detect it. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It's not a standard method as far as I know (just something I came up with), so it probably has thorny bits I'm not aware of. Also, I haven't actually tried it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The $r^n$ term will mix in some low frequency components, so if the largest argument is small enough it could be hidden by this term.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-05-10T20:59:41.443" LastActivityDate="2012-05-10T20:59:41.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2164" PostTypeId="2" ParentId="2152" CreationDate="2012-05-09T21:01:15.323" Score="3" Body="&lt;p&gt;I get to know PyTables when installing PETSC. And I guess the table (or database) method&#xA;is well-suited for exploring parameter space, though I have not tried yet.  We can record&#xA;every run with specific parameters and then we can consult any aggregations satisfying some&#xA;conditions, say, we can fix dt, BDF1 and look up all relevant records to study the variation&#xA;due to the other parameters.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to hear from people who are actually using the table (or database) method for&#xA;exploring parameter space. I will appreciate for detailed expamples.&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2012-05-09T21:01:15.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2165" PostTypeId="1" AcceptedAnswerId="2166" CreationDate="2012-05-10T01:26:25.507" Score="5" ViewCount="253" Body="&lt;p&gt;What is the difference between MATSEQMAIJ and MATSEQAIJ in PETSc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, where can I find more information on each of the MatTypes? I went to the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatType.html&quot; rel=&quot;noreferrer&quot;&gt;MatType&lt;/a&gt; documentation, but it didn't have anything but the definitions.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-05-10T02:01:51.567" Title="What is the difference between MATSEQMAIJ and MATSEQAIJ in PETSc?" Tags="|matrix|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2166" PostTypeId="2" ParentId="2165" CreationDate="2012-05-10T02:01:51.567" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MATMAIJ.html&quot; rel=&quot;noreferrer&quot;&gt;MATMAIJ&lt;/a&gt; is a matrix type that is mostly used to represent interpolation/restriction operators in multigrid methods. It represents the Kronecker product of a sparse matrix with a small block identity, so that all fields associated with a given discretization node are interpolated in the same way, without needing any redundant storage.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-10T02:01:51.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2167" PostTypeId="2" ParentId="2152" CreationDate="2012-05-10T02:31:18.233" Score="8" Body="&lt;p&gt;For my doctoral work, I am running into similar issues as you are. Since it is not my code that I am using, though, I do not have quite the same flexibility as you do. That said, I do have a few suggestions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Pedro suggested, there is the tee command. But, if it is not available, or you would like something built into your software itself, I would suggest looking at the &lt;a href=&quot;http://www.boost.org/doc/libs/1_49_0/libs/iostreams/doc/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;boost::iostreams&lt;/code&gt;&lt;/a&gt; library. It provides mechanisms for defining input sources and output sinks which the standard library does not do. In particular, there is the &lt;a href=&quot;http://www.boost.org/doc/libs/1_49_0/libs/iostreams/doc/functions/tee.html#tee_device&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tee_device&lt;/code&gt;&lt;/a&gt; which allows you to connect two output sinks to your stream, and &lt;a href=&quot;https://stackoverflow.com/a/8685609/198315&quot;&gt;other streams&lt;/a&gt; can act as sinks. This would allow you to make the simultaneous output to &lt;code&gt;stdout&lt;/code&gt; and a log-file config dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I agree that &lt;a href=&quot;http://www.boost.org/doc/libs/1_49_0/doc/html/program_options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;boost::program_options&lt;/code&gt;&lt;/a&gt; can be very helpful in configuring your software. However, it has a couple flaws which may impact how you do things, though. First, if you need a hierarchical configuration,$^1$ then &lt;code&gt;ini&lt;/code&gt; files are a painful way of accomplishing it. Second, and more significantly, &lt;code&gt;boost::program_options&lt;/code&gt; has no output capabilities, so you cannot save your state as a configuration file for later inspection or resuming a stopped code. As an alternative, I would suggest using &lt;a href=&quot;http://www.boost.org/doc/libs/1_49_0/doc/html/property_tree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;boost::property_tree&lt;/code&gt;&lt;/a&gt; which supports hierarchical configuration &lt;a href=&quot;http://www.boost.org/doc/libs/1_49_0/doc/html/boost_propertytree/parsers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;files&lt;/a&gt; and saving the trees for later re-use. This has the added benefit that if you need to checkpoint your code, you can save out its current state as input when you restart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For gathering the data from the different calculations, I loop over all the data files I would like to include in a set, I then use &lt;a href=&quot;http://en.wikipedia.org/wiki/AWK&quot; rel=&quot;nofollow noreferrer&quot;&gt;awk&lt;/a&gt; to produce a single line in the file, and pipe all the results into my output. This can take a couple of minutes, but unfortunately, I do not have a better method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to processing/commenting on your data, I can not stress the usefulness of the Mathematica notebook format enough. It allows me to organize my observations, speculations, and visualizations all in one place. My notebooks regularly top 100 MB, though. For good measure, &lt;a href=&quot;https://mathematica.stackexchange.com/a/103/52&quot;&gt;Mathematica&lt;/a&gt; performs just as well as Matlab on matrix tasks. Additionally, it can be used to &lt;a href=&quot;https://stackoverflow.com/a/5058892/198315&quot;&gt;take notes&lt;/a&gt; with full mathematical formatting in real time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish I had a better solution to the naming problem, and it is rather pernicious. It may be worthwhile considering outputting some of your data into a database because of this. However, if you do not wish to do that, either consider using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Xfs#Extended_attributes&quot; rel=&quot;nofollow noreferrer&quot;&gt;extended attributes&lt;/a&gt; in XFS to capture more complete information about your simulation, and store your config file in with the data that it was used to generate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;1. As an example where hierarchical configuration files is needed, a friend of mine was examining the effects of different tip geometries in &lt;a href=&quot;http://en.wikipedia.org/wiki/Atomic_force_microscopy&quot; rel=&quot;nofollow noreferrer&quot;&gt;AFM&lt;/a&gt; and each geometry had a different set of parameters. Additionally, alongside this, he was testing several calculation schemes so he could compare them to experiment, and they had vastly different parameters.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="85" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-05-10T03:10:00.083" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2168" PostTypeId="1" AcceptedAnswerId="2170" CreationDate="2012-05-10T06:17:41.610" Score="27" ViewCount="10428" Body="&lt;p&gt;One of the major issues that we have to deal with in molecular simulations is the calculation of distance-dependent forces. If we can restrict the force and distance functions to have even powers of the separation distance $r$, then we can just compute the square of the distance $r^2 = {\bf r \cdot r}$ and not have to worry about $r$. If there are odd powers, however, then we need to deal with $r = \sqrt{r^2}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how expensive is computing $\sqrt{x}$ as implemented in the libraries of common languages (C/C++, Fortran, Python), etc.? Is there really a lot of performance improvements to be had by hand-tuning the code for specific architectures?&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2013-10-20T13:05:25.567" Title="What is the computational cost of $\sqrt{x}$ in standard libraries?" Tags="|efficiency|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2169" PostTypeId="2" ParentId="2168" CreationDate="2012-05-10T07:55:02.713" Score="16" Body="&lt;p&gt;The square root is implemented in hardware on most processors, that is, there are specific assembly instructions and the performance should be comparable in most languages because it is very hard to muck up the implementation. You will probably never be able to beat the FSQRT instruction, since it was designed by some smart hardware designer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How it is implemented in hardware may vary, but it is probably some kind of fixed point iteration, for example Newton-Raphson's method which does a specific number of iterates until the number of digits required are computed. Iterative methods in hardware are in general much slower than other operations, since several cycles have to be completed before the result is ready.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also some &lt;a href=&quot;http://en.wikipedia.org/wiki/Streaming_SIMD_Extensions&quot;&gt;Streaming SIMD Instructions&lt;/a&gt; which can be used on the XMM registers for fast vector computations &lt;a href=&quot;http://software.intel.com/sites/products/documentation/studio/composer/en-us/2011/compiler_c/intref_cls/common/intref_bk_sqrcube_ops.htm&quot;&gt;found here&lt;/a&gt;. These registers are fairly small, but if you have a known number of coordinates (say, a three dimensional Cartesian coordinate system) they can be quite a bit faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your language is low level enough, you could always typecast to a lower precision or use a lower precision number for your coordinates. Single precision is often more than good enough, and from what I remember will be faster when computing square roots since the iterations can be terminated earlier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be easy enough to benchmark different languages: Just write a long series of random numbers to a file, load it using different languages and then time the square roots.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2012-05-10T07:55:02.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2170" PostTypeId="2" ParentId="2168" CreationDate="2012-05-10T10:04:37.740" Score="43" Body="&lt;p&gt;As an extension to &lt;a href=&quot;https://scicomp.stackexchange.com/a/2169/149&quot;&gt;moyner's answer&lt;/a&gt;, the on-chip &lt;code&gt;sqrt&lt;/code&gt; is usually an &lt;code&gt;rsqrt&lt;/code&gt;, i.e. a reciprocal square root that computes $a \rightarrow 1/\sqrt{a}$. So if in your code you're only going to use $1/r$ (if you're doing molecular dynamics, you are), you can compute &lt;code&gt;r = rsqrt(r2)&lt;/code&gt; directly and save yourself the division. The reason why &lt;code&gt;rsqrt&lt;/code&gt; is computed instead of &lt;code&gt;sqrt&lt;/code&gt; is that its Newton iteration has no divisions, only additions and multiplications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side-note, divisions are also computed iteratively and are almost just as slow as &lt;code&gt;rsqrt&lt;/code&gt; in hardware. If you're looking for efficiency, you're better off trying to remove superfluous divisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more modern architectures such as &lt;a href=&quot;http://en.wikipedia.org/wiki/IBM_POWER&quot; rel=&quot;noreferrer&quot;&gt;IBM's POWER architectures&lt;/a&gt; do not provide &lt;code&gt;rsqrt&lt;/code&gt; per-se, but an estimate accurate to a few bits, e.g. &lt;a href=&quot;http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.aixassem/doc/alangref/frsqrte.htm&quot; rel=&quot;noreferrer&quot;&gt;FRSQRTE&lt;/a&gt;. When a user calls &lt;code&gt;rsqrt&lt;/code&gt;, this generates an estimate and then one or two (as many as required) iterations of Newton's or Goldschmidt's algorithm using regular multiplications and additions. The advantage of this approach is that the iteration steps may be pipelined and interleaved with other instructions without blocking the FPU (for a very nice overview of this concept, albeit on older architectures, see &lt;a href=&quot;http://e-collection.library.ethz.ch/eserv/eth:23265/eth-23265-02.pdf&quot; rel=&quot;noreferrer&quot;&gt;Rolf Strebel's PhD Thesis&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For interaction potentials, the &lt;code&gt;sqrt&lt;/code&gt; operation can be avoided entirely by using a polynomial interpolant of the potential function, but &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2009.09.028&quot; rel=&quot;noreferrer&quot;&gt;my own work&lt;/a&gt; (implemented in &lt;a href=&quot;http://mdcore.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;mdcore&lt;/code&gt;&lt;/a&gt;) in this area show that, at least on x86-type architectures, the &lt;code&gt;sqrt&lt;/code&gt; instruction is fast enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this answer seems to be getting quite a bit of attention, I would also like to address the second part of your question, i.e. is it really worth it to try to improve/eliminate basic operations such as &lt;code&gt;sqrt&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of Molecular Dynamics simulations, or any particle-based simulation with cutoff-limited interactions, there is a &lt;em&gt;lot&lt;/em&gt; to be gained from better algorithms for neighbour finding. If you're using &lt;a href=&quot;http://en.wikipedia.org/wiki/Cell_lists&quot; rel=&quot;noreferrer&quot;&gt;Cell lists&lt;/a&gt;, or anything similar, to find neighbours or create a &lt;a href=&quot;http://en.wikipedia.org/wiki/Verlet_list&quot; rel=&quot;noreferrer&quot;&gt;Verlet list&lt;/a&gt;, you will be computing a large number of spurious pairwise distances. In the naive case, only 16% of particle pairs inspected will actually be within the cutoff distance of each other. Although no interaction is computed for such pairs, accessing the particle data and computing the spurious pairwise distance carries a large cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own work in this area (&lt;a href=&quot;http://scholar.google.com/scholar?oi=bibs&amp;amp;hl=en&amp;amp;cluster=7853340637852591905&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://scholar.google.com/scholar?oi=bibs&amp;amp;hl=en&amp;amp;cluster=9234270314286200467&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;http://scholar.google.com/scholar?oi=bibs&amp;amp;hl=en&amp;amp;cluster=9613968361121364060&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;), as well as that of others (e.g. &lt;a href=&quot;http://scholar.google.ch/scholar?cluster=10991615015002105436&amp;amp;hl=en&amp;amp;as_sdt=0,5&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;), show how these spurious computations can be avoided. These neighbour-finding algorithms even out-perform Verlet lists, as described &lt;a href=&quot;http://scholar.google.ch/scholar?cluster=16816384859077213474&amp;amp;hl=en&amp;amp;as_sdt=0,5&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point I want to emphasize is that although there may be some improvements to gain from better knowing/exploiting the underlying hardware architecture, there are also potentially larger gains to be had in re-thinking the higher-level algorithms.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-10-20T13:05:25.567" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2173" PostTypeId="1" AcceptedAnswerId="2179" CreationDate="2012-05-10T20:28:16.900" Score="69" ViewCount="16116" Body="&lt;p&gt;I work in computational science, and as a result, I spend a non-trivial amount of my time trying to increase the scientific throughput of many codes, as well as understanding the efficiency of these codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume I have evaluated the performance vs. readability/reusability/maintainability tradeoff of the software I am working on, and I have decided that it's time to go for performance.  Let's also assume that I know I don't have a better algorithm for my problem (in terms of flop/s and memory bandwidth).  You can also assume my code base is in a low-level language like C, C++, or Fortran.  Finally, let's assume that there is no parallelism to be had in the code, or that we're only interested in performance on a single core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the most important things to try first?  How do I know how much performance I can get?  &lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="89" LastEditDate="2012-06-28T07:18:57.523" LastActivityDate="2019-07-24T21:05:51.463" Title="What are some good strategies for improving the serial performance of my code?" Tags="|performance|" AnswerCount="10" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2174" PostTypeId="1" AcceptedAnswerId="2181" CreationDate="2012-05-10T20:32:00.817" Score="3" ViewCount="932" Body="&lt;p&gt;Trying to explain my problem better, I need to start with the diffusion Equation, which is the one that I am trying to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Diffusion Equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\dfrac{\partial u}{\partial t}=\dfrac 3 x\dfrac \partial{\partial x}\left(\sqrt x \frac \partial {\partial x}(\nu(T(u,x))u\sqrt x)\right)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $u$ is the surface density of a gas disk. $T$, is the temperature of that disk. My aim is calculate the evolution of that disk using an initial condition.&#xA;After treat this equation with MOL (Method of lines) in Matlab I am able to solve this equation. But for that I need to use another equations that will relate some of the variables in my problem, so finally I have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\nu\propto \frac{T}{\Omega }&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Omega$ is just the keplarian velocity of the disk, therefore is a function that depends on $x$, and also I have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\Xi(T,\Omega)T^3=u^2\Omega&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\Xi$ is another function which depend also in $T$ and $\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, since I have an initial condition for $u$ and of course I define my dominion in $x$ ($x=1..10^3$ for instance). I just need to solve the last equation and find the value for T given some position and a value of $u$. I do this with a method to find zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOW MY PROBLEM:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to add advection to my problem. In terms of equations I have this now:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\Xi(T,\Omega)T^3=u^2\Omega + \frac{T^4\;u\;H(T,\Omega)}{\Omega x^2}\left|\frac{d\;\log\;H(T,\Omega)}{d\;\log\;x} - \frac{d\;\log\;T}{d\;\log\;x} - \frac{d\;\log\;u}{d\;\log\;x} \right|&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;H^2\Omega^2-H\frac{T^4}{u}-T=0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I only need one root (the real and positive one), and of course the method to solve this (the one that I'm trying to find) will be the same using any root of H.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem here is the derivatives, without them, I can use the same method that before to find T. But now I can't do that. Because I need to calculate those derivates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please ask me anything about the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;%%%UPDATE%%%%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, after doing a &quot;little&quot; bit of algebra, I realize that the equation is of this type:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left|T'+A(T)+B(T)\right|=f(T)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it can be solve numerically as a regular ODE. With two possible solutions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T'+A(T)+B(T)=f(T)$$&#xA;and,&#xA;$$T'+A(T)+B(T)=-f(T)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for all the help!&lt;/p&gt;&#xA;" OwnerUserId="1253" LastEditorUserId="1253" LastEditDate="2012-05-22T12:10:27.877" LastActivityDate="2012-05-22T12:10:27.877" Title="How to solve a nonlinear differential equation of this type" Tags="|pde|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2175" PostTypeId="2" ParentId="2174" CreationDate="2012-05-10T20:39:50.777" Score="2" Body="&lt;p&gt;Instinctively, I'd say use &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot; rel=&quot;nofollow&quot;&gt;Chebfun&lt;/a&gt;, which uses spectral collocation and automatic differentiation to solve non-linear BVPs. In any case, I'd replace $R$ with $\exp(r)$ to make the derivatives more reasonable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd try it myself, but there is quite a bit of information missing, e.g. what boundary conditions are you using and what are your constants? &lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-05-10T20:52:41.787" LastActivityDate="2012-05-10T20:52:41.787" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2176" PostTypeId="2" ParentId="2159" CreationDate="2012-05-10T21:11:16.237" Score="0" Body="&lt;p&gt;If you just want an upper bound, another way of estimating it is to use &lt;a href=&quot;http://en.wikipedia.org/wiki/Gershgorin_circle_theorem&quot; rel=&quot;nofollow&quot;&gt;Gershgorin's circle theorem&lt;/a&gt;. You could simply determine the maximum argument of any point on the union of the Gershgorin disks, and this would be guaranteed to be larger than the argument of any of your eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method has a serious problem, though: if the Gershgorin region contains some neighborhood of the origin, no restrictions on the largest argument of any of your eigenvalues can be derived.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-05-10T21:11:16.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2177" PostTypeId="2" ParentId="2173" CreationDate="2012-05-10T21:20:53.840" Score="8" Body="&lt;p&gt;The very first thing you should do is profile your code. You want to find out &lt;em&gt;which&lt;/em&gt; parts of your program are slowing you down before you begin to optimize, otherwise you could end up optimizing a part of your code that wasn't eating up much of the execution time anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Linux&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://www.cs.utah.edu/dept/old/texinfo/as/gprof_toc.html&quot; rel=&quot;noreferrer&quot;&gt;gprof&lt;/a&gt; is pretty good, but it only tells you how much time is taken by each function rather than each line. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Apple OS X&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You might want to try out &lt;a href=&quot;https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/SharkUserGuide/Introduction/Introduction.html&quot; rel=&quot;noreferrer&quot;&gt;Shark&lt;/a&gt;. It is available in Apple Developer site under Downloads &gt; Developer Tools &gt; CHUD 4.6.2, the older version &lt;a href=&quot;https://www.macupdate.com/app/mac/8506/apple-chud-tools&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. CHUD also contains other profiling tools such as BigTop frontend, PMC Index search tool, Saturn function-level profiler and a lot of  other commands. Shark will come with a commandline version.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="81" LastEditorUserId="4355" LastEditDate="2013-05-21T13:20:57.623" LastActivityDate="2013-05-21T13:20:57.623" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2178" PostTypeId="2" ParentId="2173" CreationDate="2012-05-10T22:15:33.913" Score="7" Body="&lt;p&gt;As for how much performance you can get, take the results from the profiling of your code and let's say you identify a piece that takes &quot;p&quot; fraction of the time.  If you were to improve the performance of that piece only by a factor of &quot;s&quot;, your overall speedup will be 1/((1-p) + p/s). Therefore you can maximally increase your speed by a factor of 1/(1-p).  Hopefully you have areas of high p! This is the equivalent of &lt;a href=&quot;http://en.wikipedia.org/wiki/Amdahl%27s_law&quot;&gt;Amdahl's Law&lt;/a&gt; for serial optimization. &lt;/p&gt;&#xA;" OwnerUserId="1439" LastActivityDate="2012-05-10T22:15:33.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2179" PostTypeId="2" ParentId="2173" CreationDate="2012-05-10T22:44:48.507" Score="69" Body="&lt;p&gt;First of all, as &lt;a href=&quot;https://scicomp.stackexchange.com/a/2178/541&quot;&gt;skillman&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/a/2177/541&quot;&gt;Dan&lt;/a&gt; have pointed out, profiling is essential. I personally use &lt;a href=&quot;http://software.intel.com/en-us/articles/intel-vtune-amplifier-xe/&quot; rel=&quot;noreferrer&quot;&gt;Intel's VTune Amplifier&lt;/a&gt; on Linux as it gives me a very fine-grained overview of where time was spent doing what.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not going to change the algorithm (i.e. if there will be no major changes that will turn all your optimizations obsolete), then I'd suggest looking for some common implementation details that can make a big difference:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Memory locality&lt;/strong&gt;: is data that is read/used together also stored together, or are you picking up bits and pieces here and there?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Memory alignment&lt;/strong&gt;: are your doubles actually aligned to 4 bytes? How did you pack your &lt;code&gt;structs&lt;/code&gt;? To be pedantic, use &lt;code&gt;posix_memalign&lt;/code&gt; instead of &lt;code&gt;malloc&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Cache efficiency&lt;/strong&gt;: Locality takes care of most cache efficiency issues, but if you have some small data structures that you read/write often, it helps if they are an integer multiple or fraction of a cache line (usually 64 bytes). It also helps if your data is aligned to the size of a cache line. This can drastically reduce the number of reads necessary to load a piece of data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Vectorization&lt;/strong&gt;: No, don't go mental with hand-coded assembler. &lt;code&gt;gcc&lt;/code&gt; offers vector types that get translated to SSE/AltiVec/whatever instructions automagically. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Instruction-Level Parallelism&lt;/strong&gt;: The bastard son of vectorization. If some often-repeated computation does not vectorize well, you can try accumulating input values and computing several values at once. It's kind of like loop unrolling. What you're exploiting here is that your CPU will usually have more than one floating-point unit per core.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Arithmetic precision&lt;/strong&gt;: Do you really need double-precision arithmetic in everything you do? E.g. if you're computing a correction in a Newton iteration, you usually don't need all the digits you're computing. For a more in-depth discussion, see &lt;a href=&quot;http://www.lri.fr/~baboulin/CPC.pdf&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some of these tricks are used in the &lt;code&gt;daxpy_cvec&lt;/code&gt; &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1194/how-much-better-are-fortran-compilers-really&quot;&gt;this&lt;/a&gt; thread. Having said that, if you're using Fortran (not a low-level language in my books), you will have very little control over most of these &quot;tricks&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're running on some dedicated hardware, e.g. a cluster you use for all your production runs, you may also want to read-up on the specifics of the CPUs used. Not that you should write stuff in assembler directly for that architecture, but it might inspire you to find some other optimizations that you may have missed. Knowing about a feature is a necessary first step to writing code that can exploit it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's been a while since I wrote this and I hadn't noticed that it had become such a popular answer. For this reason, I'd like to add one important point:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Talk to your local Computer Scientist&lt;/strong&gt;: Wouldn't it be cool if there were a discipline which dealt exclusively with making algorithms and/or computations more efficient/elegant/parallel, and we could all go ask them for advice? Well, good news, that discipline exists: Computer Science. Chances are, your institution even has a whole department dedicated to it. Talk to these guys.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm sure to a number of non-Computer Scientists this will bring back memories of frustrating discussions with said discipline that led to nothing, or memories of other people's anecdotes thereof. Don't be discouraged. Interdisciplinary collaboration is a tricky thing, and it takes a bit of work, but the rewards can be massive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, as a Computer Scientist (CS), the trick is in getting both the expectations and the communication right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Expectation&lt;/strong&gt;-wise, a CS will only help you if he/she thinks your problem is interesting. This pretty much excludes trying to optimize/vectorize/parallelize a piece of code you've written, but not really commented, for a problem they don't understand. CSs are usually more interested in the underlying problem, e.g. the algorithms used to solve it. Don't give them your &lt;em&gt;solution&lt;/em&gt;, give them your &lt;em&gt;problem&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, be prepared for the CS to say &quot;&lt;em&gt;this problem has already been solved&lt;/em&gt;&quot;, and just give you a reference to a paper. A word of advice: Read that paper and, if it really does apply to your problem, implement whatever algorithm it suggests. This is not a CS being smug, it's a CS that just helped you. Don't be offended, remember: If the problem is not computationally interesting, i.e. it has already been solved and the solution shown to be optimal, they won't work on it, much less code it up for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Communication&lt;/strong&gt;-wise, remember that most CSs are not experts in your field, and explain the problem in terms of &lt;em&gt;what&lt;/em&gt; you are doing, as opposed to &lt;em&gt;how&lt;/em&gt; and &lt;em&gt;why&lt;/em&gt;. We usually really don't care about the &lt;em&gt;why&lt;/em&gt;, and the &lt;em&gt;how&lt;/em&gt; is, well, what we do best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I'm currently working with a bunch of Computational Cosmologists on writing a better version of their simulation code, based on &lt;a href=&quot;http://en.wikipedia.org/wiki/Smoothed-particle_hydrodynamics&quot; rel=&quot;noreferrer&quot;&gt;SPH&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Fast_multipole_method&quot; rel=&quot;noreferrer&quot;&gt;Multipoles&lt;/a&gt;. It took about three meetings to stop talking in terms of &lt;a href=&quot;http://en.wikipedia.org/wiki/Dark_matter&quot; rel=&quot;noreferrer&quot;&gt;dark matter&lt;/a&gt; and galaxy haloes (huh?) and to drill down to the core of the computation, i.e. that they need to find all the neighbours within a given radius of each particle, compute some quantity over them, and then run over all said neighbours again and apply that quantity in some other computation. Then move the particles, or at least some of them, and do it all again. You see, while the former may be incredibly interesting (it is!), the latter is what I need to understand to start thinking about algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm diverging from the main point: If you're really interested in making your computation fast, and you're not a Computer Scientist yourself, go talk to one.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-10-08T13:06:14.203" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2180" PostTypeId="2" ParentId="2173" CreationDate="2012-05-10T23:42:29.937" Score="3" Body="&lt;p&gt;I am not a computational scientist as many others here (so I could be wrong :)) but these days there is little point in spending too much time on serial performance as long as we use standard libs. It might be more worthwhile to spend any additional time/effort on making the code more scalable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case here are two examples (if you havent already read them) on how performance was improved (for unstructured FE problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.agu.org/journals/gc/gc0804/2007GC001719/&quot; rel=&quot;nofollow&quot;&gt;Serial&lt;/a&gt;: See 2nd half of abstract and related text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://climate.ornl.gov/~rmills/pubs/mills-PFLOTRAN_CUG2009.pdf&quot; rel=&quot;nofollow&quot;&gt;Parallel&lt;/a&gt;: Specially the initialization phase, in sec 4.2.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-05-10T23:52:38.050" LastActivityDate="2012-05-10T23:52:38.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2181" PostTypeId="2" ParentId="2174" CreationDate="2012-05-11T00:18:09.720" Score="3" Body="&lt;p&gt;Your equation is an implicit ordinary differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you have an initial value problem, you could use a code that solves differential-algebraic equations like DASSL, DASPK, or IDA. These problems all use BDF methods; in particular, IDA has a MATLAB interface, since it is part of SUNDIALS. In the community of people I work with, these methods tend to be the most &quot;standard,&quot; because BDF methods handle stiffness well, and I tend to work with stiff problems.&lt;/li&gt;&#xA;&lt;li&gt;If you have an initial value problem, you could use also use other numerical methods, like Radau collocation, or spectral collocation.&lt;/li&gt;&#xA;&lt;li&gt;If you have a boundary value problem, in principle, you could adapt methods for explicit ordinary differential equations to implicit ordinary differential equations, according to &lt;a href=&quot;http://epubs.siam.org/ebooks/siam/classics_in_applied_mathematics/cl13&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Numerical Solution of Boundary Value Problems for Ordinary Differential Equations&lt;/em&gt; (See section 11.2 for details)&lt;/a&gt;. It's not entirely clear how to follow the authors' advice; presumably, a literature search would turn up appropriate methods.&lt;/li&gt;&#xA;&lt;li&gt;You could treat it as a boundary value problem for differential-algebraic equations; again, you'd have to do a literature search to find appropriate methods. Some type of collocation should work here also. One pertinent code is &lt;a href=&quot;http://www.cs.ubc.ca/spider/ascher/coldae.f&quot; rel=&quot;nofollow&quot;&gt;COLDAE&lt;/a&gt;, by Ascher, et al.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You'd have to supply more information for me to give you more specific advice.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-05-11T00:18:09.720" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2183" PostTypeId="2" ParentId="2173" CreationDate="2012-05-11T10:27:51.523" Score="3" Body="&lt;p&gt;This is perhaps more of a meta-answer than an answer ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You must develop an intimate familiarity with your compiler.  You can most efficiently acquire this by reading the manual and experimenting with the options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Much of the good advice that @Pedro dispenses can be implemented by adjusting the compilation rather than the program.&lt;/p&gt;&#xA;" OwnerUserId="1003" LastActivityDate="2012-05-11T10:27:51.523" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2184" PostTypeId="1" CreationDate="2012-05-11T11:40:35.240" Score="2" ViewCount="1400" Body="&lt;p&gt;I try to use the matlab fminunc functionality in java and found the optimisation functions in commons-math. I have absolutely no clue how to use them right because the example tells nothing about the usage of gradients or how to implement them. Can anybody show me some examples?&lt;/p&gt;&#xA;" OwnerUserId="310" LastEditorUserId="310" LastEditDate="2012-05-11T13:16:03.780" LastActivityDate="2012-05-14T14:51:27.730" Title="Usage of DifferentiableMultivariateFunction and NonLinearConjugateGradientOptimizer" Tags="|optimization|matlab|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2185" PostTypeId="2" ParentId="2173" CreationDate="2012-05-11T12:42:18.493" Score="24" Body="&lt;p&gt;Not only do you have to have intimate knowledge of your &lt;em&gt;compiler&lt;/em&gt;, you also have intimate knowledge of your &lt;strong&gt;&lt;em&gt;target architecture&lt;/em&gt;&lt;/strong&gt; and &lt;em&gt;operating system&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What can affect performance?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you want to squeeze every last ounce of performance, then every time you change your target architecture, you will have to tweak and re-optimise your code. Something which was an optimisation with one CPU may become sub-optimal in the very next revision of that same CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An excellent example of this would be CPU cache's. Move your program from a CPU with a fast small cache to one with a &lt;em&gt;slightly&lt;/em&gt; slower, &lt;em&gt;slightly&lt;/em&gt; larger cache and your profiling could change significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if the target architecture doesn't change, low level changes to an operating system can also affect performance. The Spectre and Meltdown mitigation patches had a huge impact in some workloads, so these might force a re-evaluation of your optimisations.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;How can I keep my code optimised?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;When developing highly optimised code, you need to keep it modular and make it easy to swap different versions of the same algorithm in and out, possibly even selecting the specific version used at run time, depending on the resources available and the size/complexity of data to be processed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modularity also means being able to use the same test suite on all of your optimised and unoptimised versions, allowing you to verify that they all behave the same &lt;em&gt;and&lt;/em&gt; profile each one quickly in a &lt;em&gt;like-for-like&lt;/em&gt; comparison. I go into a little more detail in my answer to &lt;a href=&quot;https://softwareengineering.stackexchange.com/a/121535/22493&quot;&gt;How to document and teach others “optimized beyond recognition” computationally intensive code?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In addition, I would highly recommend taking a look at Ulrich Drepper's excellent paper &lt;a href=&quot;http://www.akkadia.org/drepper/cpumemory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Every Programmer Should Know About Memory&lt;/a&gt;, whose title is an homage to David Goldberg’s equally fantastic &lt;a href=&quot;http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Every Computer Scientist Should Know About Floating-Point Arithmetic&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember every &lt;em&gt;optimisation&lt;/em&gt; has the potential to become a future &lt;em&gt;anti-optimisation&lt;/em&gt;, so should be considered a possible code smell, to be kept to a minimum. My answer to &lt;a href=&quot;https://softwareengineering.stackexchange.com/a/99463/22493&quot;&gt;Is micro-optimisation important when coding?&lt;/a&gt;&#xA;provides a concrete example of this from personal experience.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2018-08-20T14:32:42.283" LastActivityDate="2018-08-20T14:32:42.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2186" PostTypeId="1" AcceptedAnswerId="2189" CreationDate="2012-05-11T16:17:59.487" Score="4" ViewCount="265" Body="&lt;p&gt;What I did is&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create some PC named pc, and PCSetOperators(pc,A,A,..); &lt;/li&gt;&#xA;&lt;li&gt;Create KSP named ksp;&lt;/li&gt;&#xA;&lt;li&gt;KSPSetPC(ksp,pc);&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;After the above process, am I still obliged to KSPSetOperators(ksp,A,A,..), or I can skip this?  From the source code of Petsc, it seems I do not need. Just want to make sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2012-05-11T16:26:01.223" LastActivityDate="2012-05-12T05:05:22.550" Title="Do I need KSPSetOperators after PCSetOperators?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2187" PostTypeId="1" AcceptedAnswerId="2191" CreationDate="2012-05-11T18:15:34.970" Score="16" ViewCount="7687" Body="&lt;p&gt;I am running molecular dynamics (MD) simulations using several software packages, like Gromacs and DL_POLY.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.gromacs.org/&quot;&gt;Gromacs&lt;/a&gt; now supports both the particle decomposition and domain decomposition algorithms.  By default, Gromacs simulations use domain decomposition, although for many years, until recently, particle decomposition was the only method implemented in Gromacs.  In one of the Gromacs papers (DOI 10.1002/jcc.20291), the authors give a reason for their initial choice of particle decomposition:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;An early design decision was the choice to work with &lt;em&gt;particle&#xA;  decomposition&lt;/em&gt; rather than &lt;em&gt;domain decomposition&lt;/em&gt; to distribute work&#xA;  over the processors.  In the latter case, spatial domains are assigned&#xA;  to processors, which enables finding spatial neighbors quickly by&#xA;  local communication only, but complications due to particles that move&#xA;  over spatial boundaries are considerable. Domain decomposition is a&#xA;  better choice only when linear system size considerably exceeds the&#xA;  range of interaction, which is seldom the case in molecular dynamics.&#xA;  With particle decomposition each processor computes the forces and&#xA;  coordinate/velocity updates for an assigned fraction of the particles,&#xA;  using a precomputed neighborlist evenly distributed over processors.&#xA;  The force $F_{ij}$ arising from the pair interaction between particles $i$&#xA;  and $j$, which is needed for the velocity update of both particles $i$ and&#xA;  $j$, is computed only once and communicated to other processors. Every&#xA;  processor keeps in its local memory the complete coordinate set of the&#xA;  system rather than restricting storage to the coordinates it needs.&#xA;  This is simpler and saves communication overhead, while the memory&#xA;  claim is usually not a limiting factor at all, even for millions of&#xA;  particles. The neighborlist, on the other hand, which can contain up&#xA;  to 1000 times the number of particles, is distributed over the&#xA;  processors. Communication is essentially restricted to sending&#xA;  coordinates and forces once per time step around the processor ring.&#xA;  These choices have proven to be robust over time and easily applicable&#xA;  to modern processor clusters.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What do they mean by &quot;linear system size&quot; in the sentence &quot;Domain decomposition is a better choice only when linear system size considerably exceeds the range of interaction, which is seldom the case in molecular dynamics&quot;?  From the paragraph above, I get the idea that particle decomposition has the advantage that one does not have to deal with particles moving across domain boundaries; rather, you just have to have enough memory for each processor to store the &lt;strong&gt;total&lt;/strong&gt; system configuration.  So particle decomposition is looking very favorable, whereas domain decomposition is looking very unfavorable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sure that this is a very complicated question (and probably the subject of many books), but just basically, &lt;strong&gt;if particle decomposition seems so favorable, why would anyone need to use domain decomposition?&lt;/strong&gt;  Is domain decomposition just favorable if the system's size  is very large (making it difficult or impossible to store the total configuration in each processor)?  Based on the quoted paragraph above, I am not sure why domain decomposition is now, just recently, the default parallelization algorithm in Gromacs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that &lt;a href=&quot;http://www.stfc.ac.uk/CSE/randd/ccg/software/DL_POLY/25526.aspx&quot;&gt;DL_POLY&lt;/a&gt; now (version 4) also uses domain decomposition.  From the version 4 manual:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The division of the conguration data in this way is based on the&#xA;  location of the atoms in the simulation cell, such a geometric&#xA;  allocation of system data is the hallmark of DD algorithms. Note that&#xA;  in order for this strategy to work efficiently, the simulated system&#xA;  must possess a reasonably uniform density, so that each processor is&#xA;  allocated almost an equal portion of atom data (as much as possible).&#xA;  Through this approach the forces computation and integration of the&#xA;  equations of motion are shared (reasonably) equally between processors&#xA;  and to a large extent can be computed independently on each processor.&#xA;  The method is conceptually simple though tricky to program and is&#xA;  particularly suited to large scale simulations, where efficiency is&#xA;  highest. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;... &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In the case of the DD strategy the SHAKE (RATTLE) algorithm is simpler than for the&#xA;  Replicated Data method of DL_POLY Classic), where global updates of the atom positions&#xA;  (merging and splicing) are required.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This makes it sound as if domain decomposition is good because it may be more efficient, even though perhaps more difficult to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, a previous version (DL_POLY Classic) used replicated data parallelization, which seems to be another name for particle decomposition.  From that version's manual:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Replicated Data (RD) strategy is one of several ways to&#xA;  achieve parallelisation in MD. Its name derives from the replication&#xA;  of the configuration data on each node of a parallel computer (i.e.&#xA;  the arrays defining the atomic coordinates $\textbf{r}_i$, velocities&#xA;  $\textbf{v}_i$, and forces $\textbf{f}_i$, for all $N$ atoms in the&#xA;  simulated system, are reproduced on every processing node). In this&#xA;  strategy most of the forces computation and integration of the&#xA;  equations of motion can be shared easily and equally between nodes and&#xA;  to a large extent be processed independently on each node. The method&#xA;  is relatively simple to program and is reasonably efficient. Moreover,&#xA;  it can be “collapsed” to run on a single processor very easily.&#xA;  However the strategy can be expensive in memory and have high&#xA;  communication overheads, but overall it has proven to be successful&#xA;  over a wide range of applications.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This paragraph seems generally consistant with the first paragraph in this question, except that it says that replicated data/particle decomposition has &quot;high communication overheads.&quot;  The paragraph from the Gromacs paper seems to say the opposite -- that particle decomposition is preferable because it has lower communication overhead than domain decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any thoughts?&lt;/p&gt;&#xA;" OwnerUserId="1454" LastActivityDate="2014-05-29T09:59:18.597" Title="What are the advantages and disadvantages of the particle decomposition and domain decomposition parallelization algorithms?" Tags="|parallel-computing|efficiency|molecular-dynamics|simulation|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2189" PostTypeId="2" ParentId="2186" CreationDate="2012-05-12T05:05:22.550" Score="4" Body="&lt;p&gt;While not strictly necessary in certain use cases, please call &lt;code&gt;KSPSetOperators()&lt;/code&gt; anyway (perhaps instead of &lt;code&gt;PCSetOperators()&lt;/code&gt;). This ensures that null space and setup information is consistent and manages initial guesses for repeated solution of linear systems (&quot;Fischer guess&quot;). Calling &lt;code&gt;KSPSetOperators()&lt;/code&gt; with the same operators is essentially free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you can always &lt;code&gt;KSPGetPC()&lt;/code&gt; and then configure the &lt;code&gt;PC&lt;/code&gt;, instead of creating it yourself and using &lt;code&gt;KSPSetPC()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-12T05:05:22.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2190" PostTypeId="2" ParentId="1960" CreationDate="2012-05-12T06:09:13.370" Score="21" Body="&lt;p&gt;You are looking at the mass conservation equation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{dm}{dt}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When considering mass evolution per unit volume, this boils down to the density advection equation in flux form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial \rho}{\partial t} = -\nabla \cdot (\rho u)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good thing about this is that it is just the advection equation of an arbitrary scalar field (in our case, this happens to be density $\rho$) and it is (relatively) easy to solve, provided adequate time and space differencing schemes, and initial and boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When designing a finite differencing scheme, we worry about convergence, stability and accuracy. A scheme is converging if $\dfrac{\Delta A}{\Delta t} \rightarrow \dfrac{\partial A}{\partial t}$ when $\Delta t \rightarrow 0$. Stability of the schemes ensures that the quantity $A$ remains finite when $t \rightarrow \infty$. Formal accuracy of the scheme tells where the truncation error in the Taylor expansion series of the partial derivative lies. Look into a CFD textbook for more details on these fundamental properties of a differencing scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the simplest approach is to go straight to 1st order upstream differencing. This scheme is positive-definite, conservative and computationally efficient. The first two properties are especially important when we model the evolution of a quantity which is always positive (i.e. mass or density).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, let's look at 1-D case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial \rho}{\partial t} = -\dfrac{\partial(\rho u)}{\partial x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is convenient now to define the flux $\Phi = \rho u$, so that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial(\rho u)}{\partial x} = \dfrac{\partial \Phi}{\partial x} \approx \dfrac{\Delta \Phi}{\Delta x} \approx \dfrac{\Phi_{i+1/2}-\Phi_{i-1/2}}{\Delta x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a schematic of what we are simulating:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            u           u&#xA;|          --&amp;gt;         --&amp;gt;          |&#xA;|    rho    |    rho    |    rho    |&#xA;x-----o-----x-----o-----x-----o-----x&#xA;     i-1  i-1/2   i   i+1/2  i+1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We are evaluating the evolution of $\rho$ at cell $i$. The net gain or loss comes from the difference of what comes in, $\Phi_{i-1/2}$ and what goes out, $\Phi_{i+1/2}$. This is where we start to diverge from Paul's answer. In true conservative upstream differencing, the quantity at the cell center is being carried by velocity at its cell edge, in the direction of its motion. In other words, if you imagine you are the advected quantity and you are sitting at the cell center, you are being carried into the cell in front of you by the velocity at the cell edge. Evaluating the flux at the cell edge as a product of density and velocity, both at the cell edge, is not correct and does not conserve the advected quantity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Incoming and outgoing fluxes are evaluated as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Phi_{i+1/2} = \dfrac{u_{i+1/2}+|u_{i+1/2}|}{2}\rho_{i} &#xA;              + \dfrac{u_{i+1/2}-|u_{i+1/2}|}{2}\rho_{i+1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Phi_{i-1/2} = \dfrac{u_{i-1/2}+|u_{i-1/2}|}{2}\rho_{i-1} &#xA;              + \dfrac{u_{i-1/2}-|u_{i-1/2}|}{2}\rho_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above treatment of flux differencing ensures upstream-definiteness. In other words, it adjusts the differencing direction according to the sign of velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Courant-Friedrichs-Lewy (CFL) stability criterion, when doing time differencing with simple first order, forward Euler differencing is given as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mu = \dfrac{u\Delta t}{\Delta x} \leq 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in 2 dimensions, the CFL stability criterion is more strict:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mu = \dfrac{c\Delta t}{\Delta x} \leq \dfrac{1}{\sqrt{2}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c$ is velocity magnitude, $\sqrt{u^{2}+v^{2}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some things to consider. This scheme may or may not be appropriate for your application depending on what kind of process you are simulating. This scheme is highly diffusive, and is appropriate for very smooth flows without sharp gradients. It is also more diffusive for shorter time steps. In the 1-D case, you will obtain an almost exact solution if the gradients are very small, and if $\mu = 1$. In the 2-D case, this is not possible, and diffusion is anisotropic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your physical system considers shock waves or high gradients of other sort, you should look into upstream differencing of higher order (e.g. 3rd or 5th order). Also, it may be worthwhile looking into the Flux Corrected Transport family of schemes (Zalesak, 1979, JCP); anti-diffusion correction for the above scheme by Smolarkiewicz (1984, JCP); MPDATA family of schemes by Smolarkiewicz (1998, JCP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For time differencing, 1st order forward Euler differencing may be satisfactory for your needs. Otherwise, look into higher-order methods such as Runge-Kutta (iterative), or Adams-Bashforth and Adams-Moulton (multi-level). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be worthwhile looking into some CFD graduate-level textbook for a summary of above mentioned schemes and many more.&lt;/p&gt;&#xA;" OwnerUserId="264" LastEditorUserId="3914" LastEditDate="2013-03-14T16:33:08.760" LastActivityDate="2013-03-14T16:33:08.760" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2191" PostTypeId="2" ParentId="2187" CreationDate="2012-05-12T10:22:28.140" Score="12" Body="&lt;p&gt;Particle and domain decomposition are directly connected to the two main methods of speeding up force calculations for systems with limited-range interactions - Verlet neighbour lists and cell linked lists. If you'd like to get into details, there is a pretty nice book from Allen and Tildesley, called &lt;em&gt;Computer Simulation of Liquids&lt;/em&gt;, considered by many to be the &quot;bible&quot; of Molecular Dynamics and Monte Carlo studies. Then there is the &lt;em&gt;Numerical Simulation in Molecular Dynamics&lt;/em&gt; from Griebel, Knapek and Zumbusch, that goes deep into the various techniques for parallel implementation of MD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically Verlet lists build a list of all the neighbours of a given atom/molecule (or particles in general) within a given radius. Later when pairs of atoms are being examined in order to compute the force, the list is consulted. Once that you have the list constructed it is obvious which particles are close to which other and they can be distributed among different processors for evaluation. The list is constructed only every now and then with some clever techniques to keep it up to date, since it takes $O(N^2)$ to build (all possible paris of particles are examined). Force evaluation given the already constructed list takes $O(N)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cell linked lists divide the space in equally sized cells that are larger than the cut-off distance of the interaction potential and then each particle is put on a list associated with the cell it falls into. This process takes $O(N)$. Then neighbours of a given particle are only searched for in the same cell or in its neighbouring cells. Since each cell has a constant number of neighbours (e.g. 26 in the 3-D case), forces are evaluated in $O(N)$. But here the constant multiplier could be large enough so to make this algorithm to scale worse than the Verlet list method. But for &lt;em&gt;large enough&lt;/em&gt; $N$ it scales better. Hence the linear size argument. The domain decomposition method is a straightforward extension of the cell linked lists method - cells are divided among different CPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with domain decomposition is that it has to communicate when particles move from one cell to another one that is taken care of by another CPU. This could become problematic at higher simulation temperatures where particles tend to move further away than their equilibrium position, or when there is a flow of particles. Also information from cells on the domain border has to be transfered on each iteration to the neighbouring domain(s). But all this is locally synchronous communication and could be done very efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The replicated data is the easiest approach but unfortunately it requires that at every step all position and velocity information is synced globally. This really doesn't scale well and for very large system the global ammount of memory is the size of the data structure times the number of CPUs used, while one of the goals of the parallel processing is distribution of data such that each CPU holds &lt;em&gt;less&lt;/em&gt; than the global ammount of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, there exists no &quot;one size fits all&quot; method, suitable for all systems being simulated. Most of the time the best parallelisation strategy can be deduced from the system geometry and the appropriate for that case MD code could be picked - they all implement more of less the same underlying force fields and integrators after all.&lt;/p&gt;&#xA;" OwnerUserId="1423" LastActivityDate="2012-05-12T10:22:28.140" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2192" PostTypeId="1" CreationDate="2012-05-12T12:45:31.020" Score="7" ViewCount="5652" Body="&lt;p&gt;What is the best platform for high performance computing (HPC)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Windows is long gone, I think. So only Unix and Linux stand the chance. What platform will strongly back my interest on computational mechanics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The application field is large dynamic FEM problem, the approach will be Multi-physics, Multi-scale modeling. Clustering with consumer hardware will be the focus (To reduce the cost).&lt;/p&gt;&#xA;" OwnerUserId="1327" LastEditorUserId="1327" LastEditDate="2012-05-13T14:31:23.703" LastActivityDate="2012-05-13T14:31:23.703" Title="What is the best high performance computing platform for engineering simulations?" Tags="|parallel-computing|hpc|" AnswerCount="3" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2193" PostTypeId="2" ParentId="2192" CreationDate="2012-05-12T14:21:41.367" Score="6" Body="&lt;p&gt;Linux takes &lt;a href=&quot;http://www.top500.org/&quot;&gt;all ten of the top 10 spots&lt;/a&gt; in petaflops/supercomputing, but the operating system is perhaps not the only aspect you should focus on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most developers will never get &quot;time&quot; on the highest end machines until they have something promising to show for their efforts.  So you probably want to keep an eye on the evolving standards for &lt;a href=&quot;http://occi-wg.org/&quot;&gt;cloud computing APIs&lt;/a&gt;.  At this point vendors like Amazon have a lead in &lt;a href=&quot;http://gigaom.com/cloud/5-takeaways-from-the-cloudstack-openstack-dustup/&quot;&gt;establishing defacto standards&lt;/a&gt;, but the open standards community is hard at work in promoting &lt;a href=&quot;http://searchcloudcomputing.techtarget.com/news/1516320/CloudAudit-goes-to-10-with-new-API-standards&quot;&gt;interoperable standards&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2012-05-12T14:21:41.367" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2194" PostTypeId="2" ParentId="2192" CreationDate="2012-05-12T14:28:41.670" Score="1" Body="&lt;p&gt;That depends on the computer architecture. When you say HPC platform,  I assume a multi-processor computing cluster. Most of these would come with vendor specific flavor of UNIX, e.g. IBM AIX or HPUX. If you are building your cluster from scratch, most common choice is some flavor of Linux. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linux is fully customizable, which is important for resource management and performance optimization. This also makes it most convenient for building/setting up large, multi-processor clusters. In addition, most scientific and engineering software and compilers are readily available for Linux through package managers, or their source is available and they are easily configured and compiled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which flavor of Linux exactly is usually a matter of preference. &lt;/p&gt;&#xA;" OwnerUserId="264" LastActivityDate="2012-05-12T14:28:41.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2195" PostTypeId="2" ParentId="2192" CreationDate="2012-05-12T18:09:22.290" Score="2" Body="&lt;p&gt;Here is a slightly different view: Windows is not gone. On the contrary, it's making its way in the HPC again these days. Microsoft has done excellent work with their cluster tools in Windows 2008 HPC Server. Their MPI implementation (based on MPICH2) is as capable as its free sibling and the InfiniBand stack on Windows is quite performant. There is also a great choice of commercial software that is only available on Windows and is now gaining distributed computing capabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I would strongly disagree with anyone that can suggest &quot;the best&quot; HPC platform. There is no &quot;one size fits all&quot; platform and it depends on variety of factors. Most machines in Top 500 come with Linux or other Unix-like OS/kernel but that doesn't mean that you have to stick with Un*x for life. Big machines are all alike - learn to use MPI and one batch system and you can use them all (there is a caveat though - efficient usage of the hardware is highly platform dependent but there are some general rules).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the real HPC user the platform is just a tool that executes his or her code in production mode. It's the development environment that counts. You can develop with MPI and/or OpenMP on any major OS, be it Windows or Mac, or Linux, or Solaris, or even some obscure OS and then run your code on the Linux supercomputer in your computing centre (or in the cloud) if you adhere to some language standards and don't use platform specific APIs. There are several people in our department that use extensively Visual Studio to develop code that runs on our enormous Linux cluser just because they don't find the development tools on Linux advanced or convenient enough. It's up to you to find the most convenient environment that suits your style and your work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are building your own system then probably the price will be a serious factor and the choice of OS is almost predetermined to be Linux. It might also be your platform if your work depends on software available for Linux only. Besides, Intel provide their development tools for that OS for free if you are going to use them for personal stuff only. Linux also comes with its default GNU compiler collection and large selection of software and libraries for almost anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to run Windows software and you could afford the extra costs, then Windows 2008 HPC Server is a nice option. It integrates the HPC envrionment with ActiveDirectory and makes it really easy for the system to be centrally administered. Intel provides great tools for HPC development on Windows but they come at cost, even for private usage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Intel development tools are available on both platforms and since they support the same variety of programming languages, it is quite easy to migrate code between the two OSes. There are also other vendors like Portland Group that provide multi-platform high performant compilers and scientific libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the connectivity - in HPC nowadays it is either InfiniBand or multigigabit Ethernet (e.g. 10 GigE). Both provide low latency and high bandwidths which are a necessity for fast message passing. InfiniBand is faster but costs more.&lt;/p&gt;&#xA;" OwnerUserId="1423" LastEditorUserId="1423" LastEditDate="2012-05-12T18:16:56.110" LastActivityDate="2012-05-12T18:16:56.110" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2196" PostTypeId="2" ParentId="2152" CreationDate="2012-05-12T19:18:01.347" Score="3" Body="&lt;p&gt;Exploring parameter space like you are trying to do can very quickly become unwieldy. There are so many different ways of doing this that there is no one real solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually when you reach this limit in your work, you might want to investigate hierarchical data formats &lt;a href=&quot;http://www.hdfgroup.org/HDF5/&quot; rel=&quot;nofollow&quot; title=&quot;HDF5&quot;&gt;HDF5&lt;/a&gt;. HDF5 allows you to store complex output of your simulation in a well defined file format. Advantages are that your data is stored in a single well defined file format. You can add multiple simulation runs, identified by different parameters, to your file, and manipulate them afterwards. The data can be compressed and is fairly easy to extract out using a variety of tools. There are easy to apis for c/c++/python etc and plenty of command line tools to manipulate the files. A disadvantage is that writing to hdf5 is not quite as simple as writing to the console. There are many example programs at &lt;a href=&quot;http://www.hdfgroup.org/ftp/HDF5/examples/misc-examples/&quot; rel=&quot;nofollow&quot;&gt;HDF5 examples&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2012-05-12T19:18:01.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2197" PostTypeId="1" AcceptedAnswerId="2198" CreationDate="2012-05-13T06:59:04.997" Score="13" ViewCount="2159" Body="&lt;p&gt;The conclusion here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/1194/how-much-better-are-fortran-compilers-really&quot;&gt;How much better are Fortran compilers really?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is that gfortran and gcc are as fast for simple code. So I wanted try something more complicated. I took the spectral norm shootout example. I first precalculate the 2D matrix A(:, :), and then calculate the norm. (This solution is not allowed on the shootout I think.) I have implemented Fortran and C version. Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/certik/spectral_norm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/certik/spectral_norm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fastest gfortran versions are spectral_norm2.f90 and spectral_norm6.f90 (one uses the Fortran's built-in matmul and dot_product, the other implements these two functions in the code -- with no difference in speed). The fastest C/C++ code that I was able to write is spectral_norm7.cpp. Timings as of the git version 457d9d9 on my laptop are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time ./spectral_norm6 5500&#xA;1.274224153&#xA;&#xA;real    0m2.675s&#xA;user    0m2.520s&#xA;sys 0m0.132s&#xA;&#xA;&#xA;$ time ./spectral_norm7 5500&#xA;1.274224153&#xA;&#xA;real    0m2.871s&#xA;user    0m2.724s&#xA;sys 0m0.124s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So gfortran's version is a little faster. Why is that? If you send a pull request with a faster C implementation (or just paste a code), I'll update the repository.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Fortran I pass a 2D array around, while in C I use a 1D array. Feel free to use a 2D array or any other way that you see fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to compilers, let's compare gcc vs gfortran, icc vs ifort, and so on. (Unlike the shootout page, which compares ifort vs gcc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: using the version 179dae2, which improves matmul3() in my C version, they are now as fast:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time ./spectral_norm6 5500&#xA;1.274224153&#xA;&#xA;real    0m2.669s&#xA;user    0m2.500s&#xA;sys 0m0.144s&#xA;&#xA;$ time ./spectral_norm7 5500&#xA;1.274224153&#xA;&#xA;real    0m2.665s&#xA;user    0m2.472s&#xA;sys 0m0.168s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Pedro's vectorized version below is faster:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time ./spectral_norm8 5500&#xA;1.274224153&#xA;&#xA;real    0m2.523s&#xA;user    0m2.336s&#xA;sys 0m0.156s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, as laxxy reports below for Intel compilers, there doesn't seem to be a big difference there and even the simplest Fortran code (spectral_norm1) is among the fastest.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-05-18T15:45:02.843" Title="Is C slower than Fortran on the spectral norm shootout (using gcc, intel and other compilers)?" Tags="|fortran|c|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2198" PostTypeId="2" ParentId="2197" CreationDate="2012-05-13T12:25:56.613" Score="12" Body="&lt;p&gt;First of all, thanks for posting this question/challenge! As a disclaimer, I'm a native C programmer with some Fortran experience, and feel most at home in C, so as such, I will focus only on improving the C version. I invite all Fortran hacks to have their go too!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to remind newcomers about what this is about: The basic premise in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1194/how-much-better-are-fortran-compilers-really&quot;&gt;this&lt;/a&gt; thread was that gcc/fortran and icc/ifort should, since they have the same back-ends respectively, produce equivalent code for the same (semantically identical) program, irrespective of it being in C or Fortran. The quality of the result depends only on the quality of the respective implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I played around with the code a bit, and on my computer (ThinkPad 201x, Intel Core i5 M560, 2.67 GHz), using &lt;code&gt;gcc&lt;/code&gt; 4.6.1 and the following compiler flags:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GCCFLAGS= -O3 -g -Wall -msse2 -march=native -funroll-loops -ffast-math -fomit-frame-pointer -fstrict-aliasing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also went ahead and wrote a SIMD-vectorized C-language version of the C++ code, &lt;code&gt;spectral_norm_vec.c&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;string.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;&#xA;/* Define the generic vector type macro. */  &#xA;#define vector(elcount, type)  __attribute__((vector_size((elcount)*sizeof(type)))) type&#xA;&#xA;double Ac(int i, int j)&#xA;{&#xA;    return 1.0 / ((i+j) * (i+j+1)/2 + i+1);&#xA;}&#xA;&#xA;double dot_product2(int n, double u[], double v[])&#xA;{&#xA;    double w;&#xA;    int i;&#xA;    union {&#xA;        vector(2,double) v;&#xA;        double d[2];&#xA;        } *vu = u, *vv = v, acc[2];&#xA;&#xA;    /* Init some stuff. */&#xA;    acc[0].d[0] = 0.0; acc[0].d[1] = 0.0;&#xA;    acc[1].d[0] = 0.0; acc[1].d[1] = 0.0;&#xA;&#xA;    /* Take in chunks of two by two doubles. */&#xA;    for ( i = 0 ; i &amp;lt; (n/2 &amp;amp; ~1) ; i += 2 ) {&#xA;        acc[0].v += vu[i].v * vv[i].v;&#xA;        acc[1].v += vu[i+1].v * vv[i+1].v;&#xA;        }&#xA;    w = acc[0].d[0] + acc[0].d[1] + acc[1].d[0] + acc[1].d[1];&#xA;&#xA;    /* Catch leftovers (if any) */&#xA;    for ( i = n &amp;amp; ~3 ; i &amp;lt; n ; i++ )&#xA;        w += u[i] * v[i];&#xA;&#xA;    return w;&#xA;&#xA;}&#xA;&#xA;void matmul2(int n, double v[], double A[], double u[])&#xA;{&#xA;    int i, j;&#xA;    union {&#xA;        vector(2,double) v;&#xA;        double d[2];&#xA;        } *vu = u, *vA, vi;&#xA;&#xA;    bzero( u , sizeof(double) * n );&#xA;&#xA;    for (i = 0; i &amp;lt; n; i++) {&#xA;        vi.d[0] = v[i];&#xA;        vi.d[1] = v[i];&#xA;        vA = &amp;amp;A[i*n];&#xA;        for ( j = 0 ; j &amp;lt; (n/2 &amp;amp; ~1) ; j += 2 ) {&#xA;            vu[j].v += vA[j].v * vi.v;&#xA;            vu[j+1].v += vA[j+1].v * vi.v;&#xA;            }&#xA;        for ( j = n &amp;amp; ~3 ; j &amp;lt; n ; j++ )&#xA;            u[j] += A[i*n+j] * v[i];&#xA;        }&#xA;&#xA;}&#xA;&#xA;&#xA;void matmul3(int n, double A[], double v[], double u[])&#xA;{&#xA;    int i;&#xA;&#xA;    for (i = 0; i &amp;lt; n; i++)&#xA;        u[i] = dot_product2( n , &amp;amp;A[i*n] , v );&#xA;&#xA;}&#xA;&#xA;void AvA(int n, double A[], double v[], double u[])&#xA;{&#xA;    double tmp[n] __attribute__ ((aligned (16)));&#xA;    matmul3(n, A, v, tmp);&#xA;    matmul2(n, tmp, A, u);&#xA;}&#xA;&#xA;&#xA;double spectral_game(int n)&#xA;{&#xA;    double *A;&#xA;    double u[n] __attribute__ ((aligned (16)));&#xA;    double v[n] __attribute__ ((aligned (16)));&#xA;    int i, j;&#xA;&#xA;    /* Aligned allocation. */&#xA;    /* A = (double *)malloc(n*n*sizeof(double)); */&#xA;    if ( posix_memalign( (void **)&amp;amp;A , 4*sizeof(double) , sizeof(double) * n * n ) != 0 ) {&#xA;        printf( &quot;spectral_game:%i: call to posix_memalign failed.\n&quot; , __LINE__ );&#xA;        abort();&#xA;        }&#xA;&#xA;&#xA;    for (i = 0; i &amp;lt; n; i++) {&#xA;        for (j = 0; j &amp;lt; n; j++) {&#xA;            A[i*n+j] = Ac(i, j);&#xA;        }&#xA;    }&#xA;&#xA;&#xA;    for (i = 0; i &amp;lt; n; i++) {&#xA;        u[i] = 1.0;&#xA;    }&#xA;    for (i = 0; i &amp;lt; 10; i++) {&#xA;        AvA(n, A, u, v);&#xA;        AvA(n, A, v, u);&#xA;    }&#xA;    free(A);&#xA;    return sqrt(dot_product2(n, u, v) / dot_product2(n, v, v));&#xA;}&#xA;&#xA;int main(int argc, char *argv[]) {&#xA;    int i, N = ((argc &amp;gt;= 2) ? atoi(argv[1]) : 2000);&#xA;    for ( i = 0 ; i &amp;lt; 10 ; i++ )&#xA;        printf(&quot;%.9f\n&quot;, spectral_game(N));&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All three versions were compiled with the same flags and the same &lt;code&gt;gcc&lt;/code&gt; version. Note that I wrapped the main function call in a loop from 0..9 to get more accurate timings.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time ./spectral_norm6 5500&#xA;1.274224153&#xA;...&#xA;real    0m22.682s&#xA;user    0m21.113s&#xA;sys 0m1.500s&#xA;&#xA;$ time ./spectral_norm7 5500&#xA;1.274224153&#xA;...&#xA;real    0m21.596s&#xA;user    0m20.373s&#xA;sys 0m1.132s&#xA;&#xA;$ time ./spectral_norm_vec 5500&#xA;1.274224153&#xA;...&#xA;real    0m21.336s&#xA;user    0m19.821s&#xA;sys 0m1.444s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So with &quot;better&quot; compiler flags, the C++ version out-performs the Fortran version and hand-coded vectorized loops only provide a marginal improvement. A quick look at the assembler for the C++ version shows that the main loops have also been vectorized, albeit unrolled more aggressively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also had a look at the assembler generated by &lt;code&gt;gfortran&lt;/code&gt; and here's the big surprise: no vectorization. I attribute the fact that it's only marginally slower to the problem being bandwidth limited, at least on my architecture. For each of the matrix multiplications, 230MB of data are traversed, which pretty-much swamps all levels of cache. If you use a smaller input value, e.g. &lt;code&gt;100&lt;/code&gt;, the performance differences grow considerably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side-note, instead of obsessing about vectorization, alignment and compiler flags, the most obvious optimization would be to compute the first few iterations in single-precision arithmetic, until we have ~8 digits of the result. The single-precision instructions are not only faster, but the amount of memory that has to be moved around is also halved.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-05-13T16:52:19.513" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2200" PostTypeId="2" ParentId="2184" CreationDate="2012-05-13T17:35:09.433" Score="1" Body="&lt;p&gt;In this context, gradients are usually evaluated from derivatives of your function and are associated with the direction that your function is most rapidly decreasing or increasing.  If you are evaluating a reasonably well-defined mathematical function, you might be able to derive the gradient yourself or using a tool like Mathematica or SymPy.  A textbook on vector calculus will help you a great deal here in understanding what you are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know how to compute the Jacobian, follow the &lt;a href=&quot;http://commons.apache.org/math/userguide/optimization.html&quot; rel=&quot;nofollow&quot;&gt;comprehensive instructions and example at Commons Math&lt;/a&gt; for deriving a class that implements the &lt;code&gt;jacobian&lt;/code&gt; and &lt;code&gt;value&lt;/code&gt; methods.&#xA;If on the other hand, you cannot say much analytically about the function, you will need an algorithm that does not rely on the gradient, like Levenberg-Marquardt, which is a good starting point if you are trying to do something that looks like nonlinear least-squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are &lt;a href=&quot;http://www.androidadb.com/class/le/LevenbergMarquardtOptimizer.html&quot; rel=&quot;nofollow&quot;&gt;several examples of using the Java LevenbergMarquardtOptimizer class&lt;/a&gt; which hopefully clarify things a bit for you.  &lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-05-14T09:11:50.063" LastActivityDate="2012-05-14T09:11:50.063" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2201" PostTypeId="2" ParentId="507" CreationDate="2012-05-13T18:32:23.193" Score="18" Body="&lt;p&gt;In low dimensions, a well implemented BFGS method is generally both faster and more robust than CG, especially if the function is not very far from a quadratic.&lt;/p&gt;&#xA;&lt;p&gt;Neither BFGS nor CG need any assumption about convexity; only the initial Hessian approximation (for BFGS) resp. the preconditioner (for CG) must be positive definite. But these can always be chosen to be the identity matrix, in low dimensions without much harm. See also &lt;a href=&quot;https://scicomp.stackexchange.com/a/3213/1117&quot;&gt;https://scicomp.stackexchange.com/a/3213/1117&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the absence of a programmed gradient, it is a big waste of effort to use numerical gradients, especially when function values are expensive. Rather, one should use a derivative-free algorithm. See &lt;a href=&quot;https://doi.org/10.1007/s10898-012-9951-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rios and Sahinidis 2013&lt;/a&gt; and &lt;a href=&quot;https://sahinidis.coe.gatech.edu/?q=dfocomp&quot; rel=&quot;nofollow noreferrer&quot;&gt;its accompanying webpage&lt;/a&gt; for a recent survey.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="39409" LastEditDate="2021-06-05T05:56:03.557" LastActivityDate="2021-06-05T05:56:03.557" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2202" PostTypeId="1" CreationDate="2012-05-13T23:38:15.767" Score="2" ViewCount="481" Body="&lt;p&gt;In Octave, how do I specify that the solution to a matrix equation should be over integers?&#xA;I.e., Given matrix $A$, vectors $x$ and $b$; $Ax=b$. Find vector $x=A^{-1}b$ such that all its entries are integers. While I'm primarily concerned with Octave, your answer may alternatively consider Wolfram|Alpha, or any other software available with a free online interface. If discussing Wolfram|Alpha or other systems, please also explain how to load a sparse matrix, which is currently in row, col, value format (i.e., I would prefer to do minimal format conversions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A$, in sparse format (row, col, value):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 1 1&#xA;2 1 1&#xA;3 2 1&#xA;4 3 1&#xA;5 3 1&#xA;6 4 1&#xA;7 4 1&#xA;8 4 1&#xA;9 5 1&#xA;10 5 1&#xA;11 6 1&#xA;12 6 1&#xA;13 7 1&#xA;14 9 1&#xA;15 9 1&#xA;16 10 1&#xA;17 11 1&#xA;18 12 1&#xA;19 12 1&#xA;20 13 1&#xA;21 13 1&#xA;22 14 1&#xA;23 14 1&#xA;24 14 1&#xA;25 15 1&#xA;26 15 1&#xA;27 16 1&#xA;28 17 1&#xA;29 17 1&#xA;30 17 1&#xA;31 18 1&#xA;32 19 1&#xA;33 20 1&#xA;34 20 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Coefficients:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;30&#xA;27&#xA;26&#xA;26&#xA;24&#xA;25&#xA;25&#xA;20&#xA;17&#xA;21&#xA;13&#xA;14&#xA;17&#xA;18&#xA;17&#xA;13&#xA;14&#xA;13&#xA;12&#xA;12&#xA;11&#xA;6&#xA;2&#xA;3&#xA;3&#xA;2&#xA;4&#xA;2&#xA;3&#xA;0&#xA;2&#xA;1&#xA;4&#xA;4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1463" LastEditorUserId="34304" LastEditDate="2023-03-05T16:23:07.170" LastActivityDate="2023-03-05T16:23:07.170" Title="In Octave, how do I specify that the solution to a matrix equation should be over integers?" Tags="|linear-algebra|matrix|octave|mixed-integer-programming|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2203" PostTypeId="2" ParentId="2202" CreationDate="2012-05-14T09:25:04.073" Score="7" Body="&lt;p&gt;If $A$ is full-rank, there is only one solution to $Ax=b$, and it is either integer or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is not full-rank, this problem starts to look like a search for least common multiples in the null space or integer programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't something that a general numerical linear algebra solver like those in Octave will be able to do on its own.  If your problem is large, you could try entering this as a feasibility problem into an integer programming solver like &lt;a href=&quot;http://lpsolve.sourceforge.net/5.1/Octave.htm&quot; rel=&quot;noreferrer&quot;&gt;lpsolve&lt;/a&gt; or &lt;a href=&quot;https://projects.coin-or.org/Cbc&quot; rel=&quot;noreferrer&quot;&gt;COIN-CBC&lt;/a&gt;.  If $A$ is not so big or nearly full-rank, then you might try an eigenvalue decomposition and searching in the null space for the nearest set of integers. &lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-05-14T23:04:41.083" LastActivityDate="2012-05-14T23:04:41.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2204" PostTypeId="1" CreationDate="2012-05-14T10:14:13.653" Score="10" ViewCount="5585" Body="&lt;p&gt;I have to make a coordinates transformation between two reference systems (axes). For that, three matrices ($3\times3$) have to be multiplied due to some intermediate axes being used. &#xA;I have thought about two approaches to resolve this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method #1&lt;/strong&gt;: Making the multiplication directly, that is,&#xA;$$v_f = R_1\ R_2\ R_3\ v_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method #2&lt;/strong&gt;: Split into steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$v_{3i} = R_3\ v_i$&lt;/li&gt;&#xA;&lt;li&gt;$v_{23} = R_2\ v_{3i}$&lt;/li&gt;&#xA;&lt;li&gt;$v_f = R_1\ v_{23}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;where: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R_1$, $R_2$ and $R_3$ are $3\times3$ matrices&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_f$,$v_i$, $v_{3i}$, $v_{23}$ are $3\times1$ vectors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know what method is more efficient computationally (less time) to do the transformation (this will be made a lot of times).&lt;/p&gt;&#xA;" OwnerUserId="1469" LastEditorUserId="9667" LastEditDate="2017-04-16T00:49:34.167" LastActivityDate="2017-04-16T00:49:34.167" Title="MATLAB matrix multiplication (the best computational approach)" Tags="|matrix|performance|matlab|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2205" PostTypeId="2" ParentId="2204" CreationDate="2012-05-14T10:24:53.960" Score="14" Body="&lt;p&gt;For starters, I wouldn't use intermediate variables, but brackets. Unless, of course, you're interested in the intermediate results, but I'm guessing not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following in Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; N = 500;                                             &#xA;&amp;gt;&amp;gt; A = rand(N); B = rand(N); C = rand(N); v = rand(N,1);&#xA;&#xA;&amp;gt;&amp;gt; tic, for k=1:100, A*B*C*v; end; toc&#xA;Elapsed time is 3.207299 seconds.&#xA;&#xA;&amp;gt;&amp;gt; tic, for k=1:100, A*(B*(C*v)); end; toc&#xA;Elapsed time is 0.108095 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have to say, though, that this is quite frightening. I've always assumed that Matlab would be smart about the matrix multiplication order, as this is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Matrix_chain_multiplication&quot;&gt;known problem&lt;/a&gt; with a simple and efficient solutions.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-05-14T10:24:53.960" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2206" PostTypeId="2" ParentId="2204" CreationDate="2012-05-14T11:17:35.850" Score="14" Body="&lt;p&gt;Since the matrices are so small, all of the cost is going to be in call overhead.  If you will do the transformation many times, it will be faster to precompute &lt;code&gt;D=A*B*C&lt;/code&gt; once and then for each vector apply &lt;code&gt;v_f=D*v_i&lt;/code&gt;.  You could also consider bringing this out to a mex file.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-05-14T13:38:43.273" LastActivityDate="2012-05-14T13:38:43.273" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2207" PostTypeId="2" ParentId="2184" CreationDate="2012-05-14T12:49:05.153" Score="1" Body="&lt;p&gt;After some help and the provided answers i came up with the following code for my target function with gradients :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;public class GaussianProcessRegressionMarginalLikelihood implements&#xA;    DifferentiableMultivariateRealFunction {&#xA;&#xA;RealMatrix K;&#xA;RealMatrix invK;&#xA;&#xA;RealMatrix y;&#xA;CholeskyDecomposition L;&#xA;double L_diag;&#xA;RealMatrix alpha;&#xA;&#xA;RealMatrix alpha2;&#xA;&#xA;private double[][] cov;&#xA;private double [][][] paramCov;&#xA;&#xA;private double[] targets;&#xA;private double[] X;&#xA;&#xA;&#xA;&#xA;public GaussianProcessRegressionMarginalLikelihood(double[] targets, double[] X) {&#xA;&#xA;    this.targets = targets;&#xA;    this.X = X;&#xA;&#xA;&#xA;    this.cov = new double[targets.length][targets.length];&#xA;&#xA;}&#xA;&#xA;private void initCovarianceAndGradients(double[] parameter)&#xA;{&#xA;    double mean = StatUtils.mean(this.targets);     &#xA;    double[] y_array = new double[this.targets.length];&#xA;&#xA;    paramCov = new double[parameter.length][][];&#xA;&#xA;    for(int i = 0; i &amp;lt; parameter.length; i++)&#xA;    {&#xA;        paramCov[i] = new double[targets.length][targets.length];&#xA;    }&#xA;&#xA;    for(int i = 0; i &amp;lt; this.targets.length;i++)&#xA;    {&#xA;        y_array[i] = (targets[i]- mean);&#xA;        //y_array[i] = targets[i]- mean;&#xA;    }&#xA;&#xA;    for(int i = 0; i &amp;lt; this.X.length;i++)&#xA;    {&#xA;&#xA;        for(int j = 0; j &amp;lt; this.X.length;j++)&#xA;        {&#xA;&#xA;            double covar = (Double)squaredExponential.cov(X[i], X[j],parameter[0],parameter[1],parameter[2]);&#xA;            this.cov[i][j] = covar;&#xA;&#xA;            double dSigmaF = (Double)squaredExponential.dSigmaF(X[i], X[j],parameter[0],parameter[1],parameter[2]);&#xA;            this.paramCov[0][i][j] = dSigmaF;&#xA;&#xA;            double dSigmaN = (Double)squaredExponential.dSigmaN(X[i], X[j],parameter[0],parameter[1],parameter[2]);&#xA;            this.paramCov[1][i][j] = dSigmaN;&#xA;&#xA;            double dL = (Double)squaredExponential.dL(X[i], X[j],parameter[0],parameter[1],parameter[2]);&#xA;            this.paramCov[2][i][j] = dL;&#xA;&#xA;        }&#xA;    }&#xA;&#xA;    y = MatrixUtils.createColumnRealMatrix(y_array);&#xA;&#xA;    K =  MatrixUtils.createRealMatrix(cov);&#xA;&#xA;    //identity matrix for I&#xA;    RealMatrix k_eye = MatrixUtils.createRealIdentityMatrix(cov.length);&#xA;&#xA;    //k_eye = k_eye.scalarMultiply(Math.pow(Float.MIN_VALUE * 1000, 2));&#xA;    k_eye = k_eye.scalarMultiply(Math.pow(parameter[2], 2));&#xA;&#xA;    //RealMatrix choleskyInput = K.add(k_eye.scalarMultiply(Math.pow(parameter[2], 2)));&#xA;    RealMatrix choleskyInput = K.add(k_eye);&#xA;&#xA;    CholeskyDecomposition L = null;&#xA;&#xA;    try {&#xA;            L = new CholeskyDecompositionImpl(choleskyInput);&#xA;        } catch (NonSquareMatrixException e) {&#xA;                e.printStackTrace();&#xA;        } catch (NotSymmetricMatrixException e) {&#xA;                e.printStackTrace();&#xA;            } catch (NotPositiveDefiniteMatrixException e) {&#xA;                e.printStackTrace();&#xA;            }&#xA;&#xA;    DecompositionSolver solverLTransponse = new LUDecompositionImpl(L.getLT()).getSolver();&#xA;    DecompositionSolver solverL = new LUDecompositionImpl(L.getL()).getSolver();&#xA;&#xA;    //inverse of Ltranspose for left devision&#xA;    RealMatrix L_transpose_1 = solverLTransponse.getInverse();&#xA;    //inverse of Ltranspose for left devision&#xA;    RealMatrix L_1 = solverL.getInverse();&#xA;&#xA;    //L\y&#xA;    RealMatrix L_y = L_1.multiply(y);&#xA;&#xA;    //alpha = L'\(L\y)&#xA;    alpha = L_transpose_1.multiply(L_y);&#xA;&#xA;    double L_diag = 0.0;&#xA;&#xA;    for(int i = 0; i &amp;lt; L.getL().getColumnDimension();i++)&#xA;    {&#xA;        L_diag += Math.log(L.getL().getEntry(i, i)); &#xA;    }&#xA;&#xA;&#xA;    DecompositionSolver solverK = new LUDecompositionImpl(K).getSolver();&#xA;    this.invK = solverK.getInverse();&#xA;&#xA;    alpha2 = invK.multiply(y);&#xA;}&#xA;&#xA;/* log p(y|X,theta) = -(1/2) * y^T*Ky^(-1) * y - 1/2 * log * |Ky| - (n/2) * log(2*pi)&#xA; * &#xA; */&#xA;@Override&#xA;public double value(double[] parameter) throws FunctionEvaluationException,&#xA;        IllegalArgumentException {&#xA;&#xA;    this.initCovarianceAndGradients(parameter);&#xA;&#xA;    double logpyX = - y.transpose().multiply(alpha).getData()[0][0] * 0.5&#xA;                    - L_diag&#xA;                    - X.length * Math.log(2 * Math.PI) * 0.5;&#xA;&#xA;    System.out.println(&quot;logPYX: &quot; +logpyX + &quot;param: &quot; + parameter);&#xA;&#xA;    return logpyX;&#xA;}&#xA;&#xA;@Override&#xA;public MultivariateVectorialFunction gradient() {&#xA;    return new MultivariateVectorialFunction() {&#xA;        public double[] value( double[] parameter) {&#xA;&#xA;            initCovarianceAndGradients(parameter);&#xA;&#xA;            RealMatrix innerMatrix = alpha2.multiply(alpha2.transpose()).subtract(invK);&#xA;&#xA;            double[] result = new double[paramCov.length];&#xA;&#xA;            for(int i = 0; i &amp;lt; paramCov.length; i++)&#xA;            {&#xA;                result[i] = parameter[i] * innerMatrix.multiply(MatrixUtils.createRealMatrix(paramCov[i])).getTrace() * 0.5;&#xA;            }&#xA;&#xA;            return result;&#xA;        }&#xA;    };&#xA;}&#xA;&#xA;@Override&#xA;public MultivariateRealFunction partialDerivative(final int k) {&#xA;    return new MultivariateRealFunction() {&#xA;        public double value(double[] parameter) {&#xA;            initCovarianceAndGradients(parameter);&#xA;&#xA;            RealMatrix innerMatrix = alpha2.multiply(alpha2.transpose()).subtract(invK);&#xA;&#xA;            double[] result = new double[paramCov.length];&#xA;&#xA;            for(int i = 0; i &amp;lt; paramCov.length; i++)&#xA;            {&#xA;                result[i] = parameter[i] * innerMatrix.multiply(MatrixUtils.createRealMatrix(paramCov[i])).getTrace() * 0.5;&#xA;            }&#xA;&#xA;            return result[k];&#xA;        }&#xA;    };&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;initCovarianceAndGradients(): initialisation of matrices and calculations which are needed by both marginal likelihood calculation and gradient calculation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within this function i calculate some things globally which are strongly reused by the value() and gradient() functions. What i do not really understand is the passing of the double[] argument to the value() function and the value() function of the gradient() method. Are those methods called by the optimizer with the updated parameters? If this is the case i have to recalculate the global calculations with each call to the value() and gradient() methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for clarification&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: The Class is now ok and i run it within the following process:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;parameter[2] = 1.0;&#xA;parameter[1] = 1.0;&#xA;parameter[0] = 1.0;&#xA;&#xA;NonLinearConjugateGradientOptimizer underlying =&#xA;                        new NonLinearConjugateGradientOptimizer(ConjugateGradientFormula.FLETCHER_REEVES);&#xA;&#xA;JDKRandomGenerator g = new JDKRandomGenerator();&#xA;                 g.setSeed(753289573253l);&#xA;&#xA;RandomVectorGenerator generator =&#xA;                            new UncorrelatedRandomVectorGenerator(3, new GaussianRandomGenerator(g));&#xA;&#xA;MultiStartDifferentiableMultivariateRealOptimizer optimizer =&#xA;                         new MultiStartDifferentiableMultivariateRealOptimizer(underlying, 10, generator);&#xA;&#xA;GaussianProcessRegressionMarginalLikelihood gprml = new GaussianProcessRegressionMarginalLikelihood(input, X);&#xA;&#xA;RealPointValuePair pair = null;&#xA;&#xA;try {&#xA;    pair = optimizer.optimize(gprml, GoalType.MAXIMIZE, parameter);&#xA;} catch (OptimizationException e) {&#xA;    // TODO Auto-generated catch block&#xA;    e.printStackTrace();&#xA;} catch (FunctionEvaluationException e) {&#xA;    // TODO Auto-generated catch block&#xA;    e.printStackTrace();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This procedure is actually running but it converges at some negative parameters (impossible by method). If the implementation is alright i guess there might be some other issues but to be sure it would be very helpful to have a statement about the solution. &lt;/p&gt;&#xA;" OwnerUserId="310" LastEditorUserId="310" LastEditDate="2012-05-14T14:51:27.730" LastActivityDate="2012-05-14T14:51:27.730" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2209" PostTypeId="2" ParentId="2204" CreationDate="2012-05-14T16:10:34.900" Score="17" Body="&lt;p&gt;Matlab interprets sequences of multiplications and/or divisions from left to right. Hence $A*B*C*v$ is much more expensive than $A*(B*(C*v))$, as you have two matrix products and one matrix-vecor product in place of three matrix-vector products. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, $A*(B*(C*v))$ should be slightly faster than if you save the intermediates in separate vectors, as your second method suggests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find out in general how to measure the impact of small programming &#xA;differences on large-scale computations, write at the Matlab prompt ''help profile''.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-14T16:10:34.900" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2213" PostTypeId="1" AcceptedAnswerId="2914" CreationDate="2012-05-14T20:43:36.097" Score="4" ViewCount="206" Body="&lt;p&gt;I'm trying to generate all graphs with &lt;em&gt;n&lt;/em&gt; or fewer vertices that can be embedded in some lattice, eg square, triangular, Kagome. Do there exist algorithms to both enumerate &lt;em&gt;and&lt;/em&gt; draw these graphs? What I am looking for is a technique to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start with a listing of graphs (probably in adjacency matrix format) of size &lt;em&gt;k&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;From this list generate/draw all graphs of size &lt;em&gt;k+1&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Eliminate those graphs which are disconnected/isomorphic to another graph&lt;/li&gt;&#xA;&lt;li&gt;Keep going until I get graphs with &lt;em&gt;n&lt;/em&gt; vertices &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Which is either already parallel or reasonably easy to parallelize. I've seen suggestions to use a canonical labelling scheme to cut down on the number of graphs that need to be eliminated through isomorphism at the end. All the graphs I would like to draw are planar and no disconnected graphs are allowed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit] I'm also wondering whether it would be better to use an adjacency matrix or list to store my graphs - since there will be many hundreds of thousands of them, I'd like a compact storage method. &lt;/p&gt;&#xA;" OwnerUserId="1193" LastEditorUserId="1193" LastEditDate="2012-05-25T21:27:38.233" LastActivityDate="2012-07-24T20:07:57.597" Title="Generating lattice clusters/graphs in parallel" Tags="|algorithms|parallel-computing|graph-theory|combinatorics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2214" PostTypeId="1" CreationDate="2012-05-14T21:40:44.333" Score="11" ViewCount="312" Body="&lt;p&gt;This is a simple symmetry enumeration problem. I give the full background here, but no knowledge of quantum chemistry is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two particle integral $\langle ij|kl\rangle$ is:&#xA;$$&#xD;&#xA;    \langle ij|kl\rangle =&#xD;&#xA;        \int {\psi_i^*({\bf x})\psi_j^*({\bf x}&amp;#39;)\psi_k({\bf x})\psi_l({\bf x}&amp;#39;)&#xD;&#xA;            \over | {\bf x} - {\bf x}&amp;#39; |}\, d^3 x \,d^3 x&amp;#39;&#xD;&#xA;$$&#xA;And it has the following 4 symmetries:&#xA;$$\langle ij|kl\rangle = \langle ji|lk\rangle = \langle kl|ij\rangle = \langle lk|ji\rangle$$&#xA;I have a function that calculates the integral and stores them in a 1D array &lt;code&gt;int2&lt;/code&gt;, indexed as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int2(ijkl2intindex2(i, j, k, l))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the function &lt;code&gt;ijkl2intindex2&lt;/code&gt; returns a unique index, taking the above symmetries into account. The only requirement is that if you loop over all combinations of i, j, k, l (from 1 to n each), it will fill the &lt;code&gt;int2&lt;/code&gt; array consecutively and it will assign the same index to all ijkl combinations that are related by the above 4 symmetries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current implementation in Fortran is &lt;a href=&quot;https://gist.github.com/2695880&quot;&gt;here&lt;/a&gt;. It is very slow. Does anybody know how to do this effectively? (In any language.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hint: if the orbitals $\psi_i({\bf x})$ are real, then in addition to the above symmetries, one can exchange $i\leftrightarrow k$ and $j\leftrightarrow l$ so we get 8 symmetries total:&#xA;$$\langle ij|kl\rangle = \langle ji|lk\rangle = \langle kj|il\rangle = \langle il|kj\rangle = $$&#xA;$$= \langle kl|ij\rangle = \langle lk|ji\rangle =  \langle il|kj\rangle&#xD;&#xA;= \langle kj|il\rangle$$&#xA;and then one can implement a very fast function for indexing it, see&#xA;my implementation &lt;a href=&quot;https://gist.github.com/2695874&quot;&gt;here&lt;/a&gt;. I would like to find some efficient indexing scheme for the cases when the orbitals are not real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: the functions that I implemented actually accept the four numbers $i$, $j$, $k$, $l$ in a so called &quot;chemistry&quot; notation $(ij|kl) = \langle ik|jl\rangle$, i.e. the $j$ and $k$ arguments are interchanged, but this is not important.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="9" LastEditDate="2012-05-14T22:56:27.880" LastActivityDate="2013-04-14T20:22:53.407" Title="How to implement efficient indexing function for two particle integrals &lt;ij|kl&gt;?" Tags="|algorithms|performance|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2216" PostTypeId="1" CreationDate="2012-05-14T21:44:41.500" Score="5" ViewCount="1025" Body="&lt;p&gt;Given the function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal{M} = g + Ah + Bh^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ and $B$ are constants and $g$ and $h$ are random variables with their distributions $f_G(g)$ and $f_H(h)$ known, is it possible to compute the probability density function $f_\mathcal{M}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can obviously generate samples of it by creating realizations of $g$ and $h$ and then computing $\mathcal{M}$ (and then possibly form the empirical CDF, etc.), but I'm wondering if it is possible to &quot;write down&quot; an expression for $f_\mathcal{M}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few notes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If it is helpful, we can assume that $g$ and $h$ have normal distributions.&lt;/li&gt;&#xA;&lt;li&gt;I'd be very interested in a solution that could accommodate the more complex case of $\mathcal{M} = g + \sum_i^n A_i h_i + B_i h_i^2$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="37" LastActivityDate="2012-05-15T18:13:25.157" Title="Computing the PDF of a quadratic function of two random variables" Tags="|statistics|probability|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2217" PostTypeId="2" ParentId="2216" CreationDate="2012-05-14T22:16:02.173" Score="1" Body="&lt;p&gt;For the simple case of $\mathcal M = g + Ah$, the result is a convolution of the PDFs $f_G$ and $f_H$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{\mathcal M}(x) = \int_{-\infty}^\infty f_G(t)\bar{f}_H(x-t)\,\mbox{d}t$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\bar{f}_H$ is $f_H$ normalized such that $A$ disappears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiple sums are convolutions of convolutions. This can quickly get quite messy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way of computing them numerically (in Matlab) is to use &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot; rel=&quot;nofollow&quot;&gt;Chebfun&lt;/a&gt; (just as a disclaimer, I'm a member of the Chebfun development team):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; normdist = @(m,s) @(x) 1/(s*sqrt(2*pi))*exp(-0.5*((x-m)/s).^2);&#xA;&amp;gt;&amp;gt; G = chebfun( normdist(0,1) , [-10,10] )    &#xA;G = &#xA;   chebfun column (1 smooth piece)&#xA;       interval       length   endpoint values   &#xA;[     -10,      10]       93  7.7e-23  7.7e-23   &#xA;vertical scale = 0.4 &#xA;&amp;gt;&amp;gt; H = chebfun( normdist(2,0.5) , [-10,10] )  &#xA;H = &#xA;   chebfun column (1 smooth piece)&#xA;       interval       length   endpoint values   &#xA;[     -10,      10]      169 6.7e-126  2.1e-56   &#xA;vertical scale = 0.79 &#xA;&amp;gt;&amp;gt; M = conv(G,H)&#xA;M = &#xA;   chebfun column (2 smooth pieces)&#xA;       interval       length   endpoint values   &#xA;[     -20,       0]       59  1.2e-34    0.072   &#xA;[       0,      20]       70    0.072  9.1e-35   &#xA;Total length = 129   vertical scale = 1.6 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can verify this result by comparing it to the exact result&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; Mex = chebfun( normdist( 2 , sqrt(1^2+0.5^2) ) , [-20,20] )&#xA;Mex = &#xA;   chebfun column (1 smooth piece)&#xA;       interval       length   endpoint values   &#xA;[     -20,      20]      154    3e-85  1.9e-57   &#xA;vertical scale = 0.36 &#xA;&amp;gt;&amp;gt; norm(M-Mex,inf)&#xA;ans =&#xA;   5.5569e-16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not using an infinite domain, since this seems to be broken in my current version of Chebfun. The result seems to be correct to about machine precision though.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-05-15T08:22:14.387" LastActivityDate="2012-05-15T08:22:14.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2218" PostTypeId="5" CreationDate="2012-05-14T22:48:32.350" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-05-14T22:48:32.350" LastActivityDate="2012-05-14T22:48:32.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2219" PostTypeId="4" CreationDate="2012-05-14T22:48:32.350" Score="0" Body="A field of combinatorics relating to the study of vertices and the edges that connect them" OwnerUserId="1193" LastEditorUserId="1193" LastEditDate="2012-05-14T22:55:14.637" LastActivityDate="2012-05-14T22:55:14.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2220" PostTypeId="5" CreationDate="2012-05-14T23:13:40.493" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-05-14T23:13:40.493" LastActivityDate="2012-05-14T23:13:40.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2221" PostTypeId="4" CreationDate="2012-05-14T23:13:40.493" Score="0" Body="Questions about the theory and numerical algorithms for optimizing (minimizing or maximizing) nonlinear functions, possibly subject to equality and/or inequality constraints.&#xD;&#xA;" OwnerUserId="127" LastEditorUserId="1804" LastEditDate="2016-09-10T16:02:00.370" LastActivityDate="2016-09-10T16:02:00.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2222" PostTypeId="2" ParentId="2216" CreationDate="2012-05-15T03:56:44.187" Score="2" Body="&lt;p&gt;Using the assumption of normality, we can get an expression for this in Maple, but it's not in closed form (it still has an integral sign in it). Here's what I did:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with(Statistics):&#xA;h := RandomVariable(Normal(mu[H], sigma[H]));&#xA;g := RandomVariable(Normal(mu[G], sigma[G]));&#xA;PDF(g + A * h + B * h^2, z) assuming A &amp;gt; 0, B &amp;gt; 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Without the assumptions on A and B I get an error, but with them, I get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;\int _{-1/4\,{\frac {{A}^{2}}{B}}}^{\infty }\!1/2\,\sqrt {2}{{\rm e}^{-1/2\,{\frac { \left( z-{\it &#xD;&#xA;\_t}-\mu_{{{\it G}}} \right) ^{2}}{{\sigma_{{{\it G}}}}^{2}}}}} \left( 1/2\,\sqrt {2}{{\rm e}&#xD;&#xA;^{-1/8\,{\frac { \left( A+\sqrt {{A}^{2}+4\,B{\it \_t}}+2\,\mu_{{{\it H}}}B \right) ^{2}}{{B}^{2}&#xD;&#xA;{\sigma_{{{\it H}}}}^{2}}}}}{\frac {1}{\sqrt {\pi }}}{\sigma_{{{\it H}}}}^{-1}{\frac {1}{\sqrt &#xD;&#xA;{{A}^{2}+4\,B{\it \_t}}}}+1/2\,\sqrt {2}{{\rm e}^{-1/8\,{\frac { \left( A-\sqrt {{A}^{2}+4\,B{\it &#xD;&#xA;\_t}}+2\,\mu_{{{\it H}}}B \right) ^{2}}{{B}^{2}{\sigma_{{{\it H}}}}^{2}}}}}{\frac {1}{\sqrt {&#xD;&#xA;\pi }}}{\sigma_{{{\it H}}}}^{-1}{\frac {1}{\sqrt {{A}^{2}+4\,B{\it \_t}}}} \right) {\frac {1}{&#xD;&#xA;\sqrt {\pi }}}{\frac {1}{\sqrt {{\sigma_{{{\it G}}}}^{2}}}}{d{\it \_t}}&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is in an internal development version, so your results may be different. Full disclosure: I work for these guys.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, what might work even better is to see (by completing the square) that &#xA;$$A h + B h^2 = B \left(\left(h + \frac{A}{2B}\right)^2 - \frac{A^2}{4 B^2}\right).$$&#xA;Suppose $h$ is normally distributed with parameters $\mu_H$ and $\sigma_H$, and let $C = \frac{A}{2B \sigma_H}$. Then so is $h /\sigma_H + C$ is normally distributed with variance 1 and mean $\mu_H/\sigma_H + C$. Its square is then distributed as a &lt;a href=&quot;http://en.wikipedia.org/wiki/Noncentral_chi-squared_distribution&quot; rel=&quot;nofollow&quot;&gt;noncentral chi squared distribution&lt;/a&gt; with parameters $k = 1$ (a.k.a. $\nu = 1$) and $\delta = \mu_H/\sigma_H + C$. So we define $\tilde h = (h / \sigma_H + C)^2$, which is distributed in this way, and we examine&#xA;$$A h + B h^2 = B \sigma_H^2((h+\frac{A}{2B})^2/\sigma_H^2 - \frac{A^2}{4B^2\sigma_H^2}) = B \sigma_H^2(\tilde h + C^2).$$&#xA;Now the PDF of that can easily be derived from the PDF of $\tilde h$. Unfortunately, &lt;a href=&quot;http://en.wikipedia.org/wiki/Noncentral_chi-squared_distribution&quot; rel=&quot;nofollow&quot;&gt;that PDF&lt;/a&gt; involves a nasty hypergeometric function (a modified Bessel function of the first kind), so it doesn't make the evaluation of the PDF too much easier. However, it might make the &lt;em&gt;theoretical&lt;/em&gt; analysis easier...&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2012-05-15T03:56:44.187" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2224" PostTypeId="1" AcceptedAnswerId="2234" CreationDate="2012-05-15T06:43:18.457" Score="4" ViewCount="1616" Body="&lt;p&gt;I'm making a simple eigenvalue solver with SLEPc, using a 5-point stencil and the finite difference method.  I want to be able to assemble the matrix in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thought was just to use &lt;code&gt;MatGetOwnershipRange()&lt;/code&gt; to get &lt;code&gt;range_begin&lt;/code&gt; and  &lt;code&gt;range_end&lt;/code&gt;, then use tests inside the loop to determine if that row represents a point on the edge. All those conditionals make the loop extremely slow, and elsewhere in the PETSc documentation it recommends building separate loops for the corners and edges. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this be done in a parallel program?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way that works for parallel code not mentioned in the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.1/src/ksp/ksp/examples/tutorials/ex13.c.html&quot;&gt;laplacian example code&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-05-15T13:50:16.480" Title="How should I build a 2D 5-point stencil Laplacian matrix in parallel?" Tags="|parallel-computing|petsc|performance|finite-difference|banded-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2225" PostTypeId="2" ParentId="2162" CreationDate="2012-05-15T07:08:00.217" Score="5" Body="&lt;p&gt;My good friend &lt;a href=&quot;http://www.getreuer.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pascal&lt;/a&gt; made this a few years ago (it's &lt;em&gt;almost&lt;/em&gt; in Matlab):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#! /usr/bin/env python&#xA;&#xA;#from scipy.interpolate import interpolate&#xA;from pylab import *&#xA;from numpy import *&#xA;&#xA;&#xA;def GaussianFilter(sigma,f):&#xA;    &quot;&quot;&quot;Apply Gaussian filter to an image&quot;&quot;&quot;&#xA;    if sigma &amp;gt; 0:&#xA;        n = ceil(4*sigma)&#xA;        g = exp(-arange(-n,n+1)**2/(2*sigma**2))&#xA;        g = g/g.sum()&#xA;&#xA;        fg = zeros(f.shape)&#xA;&#xA;        for i in range(f.shape[0]):&#xA;            fg[i,:] = convolve(f[i,:],g,'same')&#xA;        for i in range(f.shape[1]):&#xA;            fg[:,i] = convolve(fg[:,i],g,'same')&#xA;    else:&#xA;        fg = f&#xA;&#xA;    return fg&#xA;&#xA;&#xA;def clamp(x,xmin,xmax):&#xA;    &quot;&quot;&quot;Clamp values between xmin and xmax&quot;&quot;&quot;&#xA;    return minimum(maximum(x,xmin),xmax)&#xA;&#xA;&#xA;def myinterp(f,xi,yi):&#xA;    &quot;&quot;&quot;My bilinear interpolator (scipy's has a segfault)&quot;&quot;&quot;&#xA;    M,N = f.shape&#xA;    ix0 = clamp(floor(xi),0,N-2).astype(int)&#xA;    iy0 = clamp(floor(yi),0,M-2).astype(int)&#xA;    wx = xi - ix0&#xA;    wy = yi - iy0&#xA;    return ( (1-wy)*((1-wx)*f[iy0,ix0] + wx*f[iy0,ix0+1]) +&#xA;        wy*((1-wx)*f[iy0+1,ix0] + wx*f[iy0+1,ix0+1]) )&#xA;&#xA;&#xA;def mkwarp(f1,f2,sigma,phi,showplot=0):&#xA;    &quot;&quot;&quot;Image warping by solving the Monge-Kantorovich problem&quot;&quot;&quot;&#xA;    M,N = f1.shape[:2]&#xA;&#xA;    alpha = 1&#xA;    f1 = GaussianFilter(sigma,f1)&#xA;    f2 = GaussianFilter(sigma,f2)&#xA;&#xA;    # Shift indices for going from vertices to cell centers&#xA;    iUv = arange(M)             # Up&#xA;    iDv = arange(1,M+1)         # Down&#xA;    iLv = arange(N)             # Left&#xA;    iRv = arange(1,N+1)         # Right&#xA;    # Shift indices for cell centers (to cell centers)&#xA;    iUc = r_[0,arange(M-1)]&#xA;    iDc = r_[arange(1,M),M-1]&#xA;    iLc = r_[0,arange(N-1)]&#xA;    iRc = r_[arange(1,N),N-1]&#xA;    # Shifts for going from centers to vertices&#xA;    iUi = r_[0,arange(M)]&#xA;    iDi = r_[arange(M),M-1]&#xA;    iLi = r_[0,arange(N)]&#xA;    iRi = r_[arange(N),N-1]&#xA;&#xA;&#xA;    ### The main gradient descent loop ###      &#xA;    for iter in range(0,30):&#xA;        ### Approximate derivatives ###&#xA;        # Compute gradient phix and phiy at pixel centers.  Array phi has values&#xA;        # at the pixel vertices.&#xA;        phix = (phi[iUv,:][:,iRv] - phi[iUv,:][:,iLv] + &#xA;            phi[iDv,:][:,iRv] - phi[iDv,:][:,iLv])/2&#xA;        phiy = (phi[iDv,:][:,iLv] - phi[iUv,:][:,iLv] + &#xA;            phi[iDv,:][:,iRv] - phi[iUv,:][:,iRv])/2&#xA;        # Compute second derivatives at pixel centers using central differences.&#xA;        phixx = (phix[:,iRc] - phix[:,iLc])/2&#xA;        phixy = (phix[iDc,:] - phix[iUc,:])/2&#xA;        phiyy = (phiy[iDc,:] - phiy[iUc,:])/2&#xA;        # Hessian determinant&#xA;        detD2 = phixx*phiyy - phixy*phixy&#xA;&#xA;        # Interpolate f2 at (phix,phiy) with bilinear interpolation&#xA;        f2gphi = myinterp(f2,phix,phiy)&#xA;&#xA;        ### Update phi ###&#xA;        # Compute M'(phi) at pixel centers&#xA;        dM = alpha*(f1 - f2gphi*detD2)&#xA;        # Interpolate to pixel vertices&#xA;        phi = phi - (dM[iUi,:][:,iLi] + &#xA;            dM[iDi,:][:,iLi] + &#xA;            dM[iUi,:][:,iRi] + &#xA;            dM[iDi,:][:,iRi])/4&#xA;&#xA;&#xA;    ### Plot stuff ###      &#xA;    if showplot:&#xA;        pad = 2&#xA;        x,y = meshgrid(arange(N),arange(M))&#xA;        x = x[pad:-pad,:][:,pad:-pad]&#xA;        y = y[pad:-pad,:][:,pad:-pad]&#xA;        phix = phix[pad:-pad,:][:,pad:-pad]&#xA;        phiy = phiy[pad:-pad,:][:,pad:-pad]&#xA;&#xA;        # Vector plot of the mapping&#xA;        subplot(1,2,1)&#xA;        quiver(x,y,flipud(phix-x),-flipud(phiy-y))&#xA;        axis('image')&#xA;        axis('off')&#xA;        title('Mapping')&#xA;&#xA;        # Grayscale plot of mapping divergence&#xA;        subplot(1,2,2)  &#xA;        divs = phixx + phiyy # Divergence of mapping s(x)&#xA;        imshow(divs[pad:-pad,pad:-pad],cmap=cm.gray)&#xA;        axis('off')&#xA;        title('Divergence of Mapping')&#xA;        show()&#xA;&#xA;    return phi&#xA;&#xA;&#xA;if __name__ == &quot;__main__&quot;:  # Demo&#xA;    from pylab import *&#xA;    from numpy import * &#xA;&#xA;    f1 = imread('brain-tumor.png')&#xA;    f2 = imread('brain-healthy.png')&#xA;    f1 = f1[:,:,1]&#xA;    f2 = f2[:,:,1]&#xA;&#xA;    # Initialize phi as the identity map&#xA;    M,N = f1.shape&#xA;    n,m = meshgrid(arange(N+1),arange(M+1))&#xA;    phi = ((m-0.5)**2 + (n-0.5)**2)/2&#xA;&#xA;    sigma = 3&#xA;    phi = mkwarp(f1,f2,sigma,phi)&#xA;    phi = mkwarp(f1,f2,sigma/2,phi,1)&#xA;#   phi = mkwarp(f1,f2,sigma/4,phi,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DvYWD.png&quot; alt=&quot;Test run, takes about 2 seconds.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient descent approach is explained here: &lt;a href=&quot;http://people.clarkson.edu/~ebollt/Papers/quadcost.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;people.clarkson.edu/~ebollt/Papers/quadcost.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="979" LastActivityDate="2012-05-15T07:08:00.217" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2226" PostTypeId="1" AcceptedAnswerId="2233" CreationDate="2012-05-15T07:35:19.860" Score="11" ViewCount="3014" Body="&lt;p&gt;Does matrix multiplication (both Mat*Mat, and Mat*Vec) scale with number of non-zeros, or with the size of the matrix?  Or some combination of the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about with shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I have a 100 x 100 matrix with 100 values in it, or a 1000 x 1000 matrix with 100 values in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When squaring these matrices (or multiplying them by similar matrices with similar sparsity), is the first (100x100) going to be faster than the second (1000x1000)?  Does it depend on where the values are?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is implementation dependent, I am interested in the answer for PETSc.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2015-09-16T11:13:40.913" Title="What is the overhead in sparse matrix multiplication" Tags="|linear-algebra|performance|sparse-matrix|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2227" PostTypeId="2" ParentId="2216" CreationDate="2012-05-15T08:11:16.740" Score="2" Body="&lt;p&gt;First of all, we can decorrelate the linear and quadratic terms by completing the square:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g + \sum A_i h_i + B_i h_i^2 \equiv g' + \sum B_i (h_i')^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sum of squared &lt;em&gt;nonstandard&lt;/em&gt; normal random variables appears to have no name or neat density, so I will compute it numerically, assuming that all the random variables are independent. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Remember that the density of the sum of two random variables is the convolution of their densities, and that convolution is a Fourier/Laplace transform pair with multiplication. Thus, we can easily convolve multiple pdfs by multiplying their Laplace- or Fourier transforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Pedro gave you an example in MATLAB, let me provide a complementary one in Mathematica. This toy example only has one quadratic term, but extension is easy.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mu = RandomReal[{0, 20}, 2]&#xA;sigma = RandomReal[{1, 5}, 2]&#xA;weights = RandomReal[{0, 1}, 2]&#xA;list1 = Table[PDF[NormalDistribution[mu[[1]], sigma[[1]]], x/weights[[1]]]/&#xA; weights[[1]], {x, 0, 50, 0.1}]&#xA;list2 = Table[&#xA;PDF[NoncentralChiSquareDistribution [1, mu[[2]]^2], &#xA; x/(sigma[[2]] weights[[2]])]/(sigma[[2]] weights[[2]]), {x, 0, 50, 0.1}];&#xA;ListPlot[{list1, list2}, PlotRange -&amp;gt; {{0, 300}, {0, 0.4}}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This plot shows the discretizations of the normal and non-central chi-square random variables:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bPSLQ.png&quot; alt=&quot;The random variables to be convolved&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ListPlot[InverseFourier[ Times[##] &amp;amp;@&#xA;Apply[Sequence, Fourier[#] &amp;amp; /@ {list1, list2}]],&#xA;PlotRange -&amp;gt; {{0, 400}, {0, 0.06}}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This plot shows the convolution; the final result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bhqiN.png&quot; alt=&quot;The final result&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1113" LastEditorUserId="1113" LastEditDate="2012-05-15T18:13:25.157" LastActivityDate="2012-05-15T18:13:25.157" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2228" PostTypeId="2" ParentId="2224" CreationDate="2012-05-15T11:22:35.220" Score="2" Body="&lt;p&gt;Consider defining a boundary and setting its values to zero. In 2d:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 0 0 0 0&#xA;0 . . . 0&#xA;0 . . . 0&#xA;0 . . . 0&#xA;0 0 0 0 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This way the inner loop can be free of conditionals, and you can carve up your matrix as desired for parallelisation.&lt;/p&gt;&#xA;" OwnerUserId="1372" LastActivityDate="2012-05-15T11:22:35.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2229" PostTypeId="2" ParentId="2226" CreationDate="2012-05-15T11:38:11.660" Score="5" Body="&lt;p&gt;Firstly, it is implementation dependent. If you implement a sparse matrix as a dense matrix and fill in the non-zeroes, it will scale with the overall size of the matrix. If it is stored as nonzeroes, it will scale as the access time scales with the matrix size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf&quot; rel=&quot;noreferrer&quot;&gt;In the PETSc documentation&lt;/a&gt;, it explains that the default storage for sparse matrices is compressed row storage, which scales with the number of rows and the number of non-zero values per row. So I would expect a MatMat to scale broadly with the square of this measure; i.e. $O(r^2 n^2)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing to note, however, is that there is no point storing what isn't there; if you care about this performance, why are you storing a 100 values for a 1000x1000 matrix? That means that at least 90% of the rows/columns have no nonzero values at all, and could be entirely removed from the matrix. If the pattern of non-zero values does not change, consider removing the always-all-zero rows from both this and the target matrix; it will remove some 90% of the effort, leaving the performance of the two matrices (100&lt;sup&gt;2&lt;/sup&gt;, 1000&lt;sup&gt;2&lt;/sup&gt;) broadly equivalent.&lt;/p&gt;&#xA;" OwnerUserId="1372" LastActivityDate="2012-05-15T11:38:11.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2230" PostTypeId="2" ParentId="2214" CreationDate="2012-05-15T12:01:13.847" Score="5" Body="&lt;p&gt;[Edit: 4th time's the charm, at last something sensible]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got to this backwards: I started with another answer showing the filter-based approach, and used that to generate all the valid combinations for a series of values of $n$, and looked the sequence up on the online integer sequence database. The number of combinations is $n^2(n^2+3)$, which looks unlikely (why the 3?). It is also $t(t(n))+t(t(n-1))$ where $t(a)$ is the triangular number of $a$, $t(a)=a(a+1)/2$. Having got this, we need to know why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first term is simpler - the pairs of pairs where the $i \ge j$ and $tid(i,j) \ge tid(k,l)$, where $tid(a,b)$ is the triangular index of $a,b$. That is fulfilled by a function like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def ascendings(n):&#xA;    idx = 0&#xA;    for i in range(1,n+1):&#xA;        for j in range(1,i+1):&#xA;            for k in range(1,i):&#xA;                for l in range(1,k+1):&#xA;                    idx = idx + 1&#xA;                    print(i,j,k,l)&#xA;            k=i&#xA;            for l in range(1,j+1):&#xA;                idx = idx + 1&#xA;                print(i,j,k,l)&#xA;    return idx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the second $l$ loop is because we cannot nest the $l$ loop entirely inside the $k$ loop without an if/skip to check the triangular indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second term $t(t(n-1))$ is where the first pair is ascending, and the second pair is descending (note, no ==, as they are handled above).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def mixcendings(n):&#xA;    idx = 0&#xA;    for j in range(2,n+1):&#xA;        for i in range(1,j):&#xA;            for k in range(1,j):&#xA;                for l in range(1,k):&#xA;                    print(i,j,k,l)&#xA;                    idx = idx + 1&#xA;            k=j&#xA;            for l in range(1,i+1):&#xA;                print(i,j,k,l)&#xA;                idx = idx + 1&#xA;    return idx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The combination of both of these gives the complete set, so putting both loops together gives us the complete set of indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A significant problem is that these patterns are difficult to calculate for an arbitrary i,j,k,l. So I would suggest a map that yields the index given i,j,k,l. Frankly, if you are doing this at all, you might as well use the generate+filter approach, because you only need to do that once for a given $n$. The plus side of the above method is that you at least have a predictable loop structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In python we can write the following iterator to give us the idx and i,j,k,l values for each different scenario:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def iterate_quad(n):&#xA;    idx = 0&#xA;    for i in range(1,n+1):&#xA;        for j in range(1,i+1):&#xA;            for k in range(1,i):&#xA;                for l in range(1,k+1):&#xA;                    idx = idx + 1&#xA;                    yield (idx,i,j,k,l)&#xA;                    #print(i,j,k,l)&#xA;            k=i&#xA;            for l in range(1,j+1):&#xA;                idx = idx + 1&#xA;                yield (idx,i,j,k,l)&#xA;&#xA;    for i in range(2,n+1):&#xA;        for j in range(1,i):&#xA;            for k in range(1,i):&#xA;                for l in range(1,k):&#xA;                    idx = idx + 1&#xA;                    yield (idx,i,j,k,l)&#xA;            k=i&#xA;            for l in range(1,j+1):&#xA;                idx = idx + 1&#xA;                yield (idx,i,j,k,l)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In fortran we would just have to run the loop and store the values. We can use a simple index to store the i,j,k,l combination as a single value ($i n^3 + j n^2 + k n + l$), and store these values in an array whose index is the same as the index above. We can then iterate over this array and retrieve the i,j,k,l from the values. To get the idx for arbitrary i,j,k,l would require a reverse map and a filter to handle the symmetry, although we could probably construct a function from the above structure. The idx array generation function in fortran would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integer function squareindex(i,j,k,l,n)&#xA;    integer,intent(in)::i,j,k,l,n&#xA;    squareindex = (((i-1)*n + (j-1))*n + (k-1))*n + l&#xA;end function&#xA;&#xA;integer function generate_order_array(n,arr)&#xA;    integer,intent(in)::n,arr(*)&#xA;    integer::total,idx,i,j,k,l&#xA;    total = n**2 * (n**2 + 3)&#xA;    reshape(arr,total)&#xA;    idx = 0&#xA;    do i=1,n&#xA;      do j=1,i&#xA;        do k=1,i-1&#xA;          do l=1,k&#xA;            idx = idx+1&#xA;            arr(idx) = squareindex(i,j,k,l,n)&#xA;          end do&#xA;        end do&#xA;        k=i&#xA;        do l=1,j&#xA;          idx = idx+1&#xA;          arr(idx) = squareindex(i,j,k,l,n)&#xA;        end do&#xA;      end do&#xA;    end do&#xA;&#xA;    do i=2,n&#xA;      do j=1,i-1&#xA;        do k=1,i-1&#xA;          do l=1,j&#xA;            idx = idx+1&#xA;            arr(idx) = squareindex(i,j,k,l,n)&#xA;          end do&#xA;        end do&#xA;        k=i&#xA;        do l=1,j&#xA;          idx = idx+1&#xA;          arr(idx) = squareindex(i,j,k,l,n)&#xA;        end do&#xA;      end do&#xA;    end do&#xA;&#xA;    generate_order_array = idx&#xA;  end function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then loop over it thus:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;maxidx = generate_order_array(n,arr)&#xA;do idx=1,maxidx&#xA;  i = idx/(n**3) + 1&#xA;  t_idx = idx - (i-1)*n**3&#xA;  j = t_idx/(n**2) + 1&#xA;  t_idx = t_idx - (j-1)*n**2&#xA;  k = t_idx/n + 1&#xA;  t_idx = t_idx - (k-1)*n&#xA;  l = t_idx&#xA;&#xA;  ! now have i,j,k,l, so do stuff&#xA;  ! ...&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1372" LastEditorUserId="1372" LastEditDate="2012-05-23T10:45:57.327" LastActivityDate="2012-05-23T10:45:57.327" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="2231" PostTypeId="1" CreationDate="2012-05-15T12:05:19.817" Score="2" ViewCount="78" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Possible Duplicate:&lt;/strong&gt;&lt;br&gt;&#xA;  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/866/how-to-reorder-variables-to-produce-a-banded-matrix-of-minimum-bandwidth&quot;&gt;How to reorder variables to produce a banded matrix of minimum bandwidth?&lt;/a&gt;  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;What is the time and space complexity of direct sparse solvers (e.g., UMFPACK, SUPERLU, PARDISO, etc.) for finite element matrices in 2D and 3D? How does the complexity splits into the creation of the inverse and the forward/backward substitution? Can anybody make some notes on this topic and/or post here some references?&lt;/p&gt;&#xA;" OwnerUserId="1090" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-05-15T12:05:19.817" Title="Complexity of direct solvers?" Tags="|finite-element|linear-solver|complexity|" AnswerCount="0" CommentCount="2" ClosedDate="2012-05-15T14:12:37.653" ContentLicense="CC BY-SA 3.0" />
  <row Id="2232" PostTypeId="2" ParentId="2197" CreationDate="2012-05-15T12:07:31.053" Score="3" Body="&lt;p&gt;Checked this with Intel compilers. With 11.1 (-fast, implying -O3), and with 12.0 (-O2) the fastest ones are 1,2,6,7,and 8 (i.e. the &quot;simplest&quot; Fortran and C codes, and the hand-vectorized C) -- these are indistinguishable from each other at ~1.5s.&#xA;Tests 3 and 5 (with array as a function) are slower; #4 I couldn't compile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quite notably, if compiling with 12.0 and -O3, rather than -O2, the first 2 (&quot;simplest&quot;) Fortran codes slow down A LOT (1.5 -&gt; 10.2 sec.) -- this is not the first time I see something like this, but this may be the most dramatic example. If this still is the case in the current release, I think it would be a good idea to report it to Intel, as there is clearly something going very wrong with their optimizations in this rather simple case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise I agree with Jonathan that this is not a particularly informative exercise :)&lt;/p&gt;&#xA;" OwnerUserId="1464" LastActivityDate="2012-05-15T12:07:31.053" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2233" PostTypeId="2" ParentId="2226" CreationDate="2012-05-15T12:55:13.780" Score="12" Body="&lt;p&gt;The cost of sparse matrix-vector multiplication scales linearly with the number of nonzero entries, as each entry is multiplied once by some entry in the vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cost of sparse matrix-matrix multiplication is highly dependent on the structure of the nonzeros. For instance, consider squaring a sparse matrix $A$ which is of an &lt;em&gt;arrowhead&lt;/em&gt; structure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;A = \left(\begin{array}{ccccc} &#xD;&#xA;  \delta_1 &amp;amp;          &amp;amp;        &amp;amp;              &amp;amp; \beta_1  \\&#xD;&#xA;           &amp;amp; \delta_2 &amp;amp;        &amp;amp;              &amp;amp; \beta_2  \\&#xD;&#xA;           &amp;amp;          &amp;amp; \ddots &amp;amp;              &amp;amp; \vdots   \\&#xD;&#xA;           &amp;amp;          &amp;amp;        &amp;amp; \delta_{n-1} &amp;amp; \beta_{n-1} \\&#xD;&#xA;  \gamma_1 &amp;amp; \gamma_2 &amp;amp; \cdots &amp;amp; \gamma_{n-1} &amp;amp; \delta_n&#xD;&#xA; \end{array}\right),                     &#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then $A$ has $O(n)$ nonzeros, but $A^2$ is dense. There is a well-known graph interpretation of this phenomenon: every path of length 1 or 2 in the graph of $A$ becomes an edge in the graph of $A^2$ (i.e., a nonzero entry in $A^2$).&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-05-15T12:55:13.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2234" PostTypeId="2" ParentId="2224" CreationDate="2012-05-15T13:50:16.480" Score="5" Body="&lt;p&gt;Those conditionals are the least of your worries. If matrix insertion/assembly is slow, it almost always means that the matrix was not correctly preallocated. After that, check that the parallel distribution is good (e.g. use a 2D decomposition instead of 1D) and that your assembly loop mostly sets values in rows owned by the given process. The 2D decomposition is somewhat tricky to program yourself, so you might want to look at an example like &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/src/ksp/ksp/examples/tutorials/ex45.c.html&quot;&gt;this&lt;/a&gt; which uses a &lt;code&gt;DMDA&lt;/code&gt; to manage the parallel decomposition. Finally, prefer using one call to &lt;code&gt;MatSetValues()&lt;/code&gt; per row or per element, instead of inserting scalar entries.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-15T13:50:16.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2235" PostTypeId="1" AcceptedAnswerId="2236" CreationDate="2012-05-15T16:55:32.110" Score="4" ViewCount="547" Body="&lt;p&gt;In petsc, there are already SetValues(Local) methods.  But when does one need GetArray methods? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related question: to copy a portion of a Mat/Vec to part of another Mat/Vec, it is usually suggested that one use a Scatter object.  Is using scatter always better than the GetArray method?&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="549" LastEditDate="2012-05-15T21:46:05.483" LastActivityDate="2012-05-15T21:46:05.483" Title="When is MatGetArray/VecGetArray useful?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2236" PostTypeId="2" ParentId="2235" CreationDate="2012-05-15T17:38:17.113" Score="6" Body="&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetArray.html&quot;&gt;&lt;code&gt;MatGetArray()&lt;/code&gt;&lt;/a&gt; is an advanced (should be developer) level routine that depends on the matrix format and is best avoided. It is only used with dense matrices within PETSc; there is no reason to use it otherwise. It's existence comes from users that stubbornly insist on having the function, not from meaningful use cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecGetArray.html&quot;&gt;&lt;code&gt;VecGetArray()&lt;/code&gt;&lt;/a&gt;, on the other hand, is the recommended way to write nonlinear residuals and other functions that need direct access to vector entries. You can use a &lt;code&gt;VecScatter&lt;/code&gt; to obtain a &quot;local&quot; vector that contains ghost points needed during assembly, and to assemble contributions from element-partitioned subdomains (as with finite element methods). There are many examples of this in PETSc, many use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMDAVecGetArray.html&quot;&gt;&lt;code&gt;DMDAVecGetArray()&lt;/code&gt;&lt;/a&gt; to also provide convenient indexing.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-15T17:38:17.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2237" PostTypeId="1" AcceptedAnswerId="2238" CreationDate="2012-05-15T18:09:53.043" Score="8" ViewCount="1121" Body="&lt;p&gt;When I think of solving non-linear equations, I generally think of linearizing first, then applying a preconditioner to the linear matrix.  The thought occurred to me that it might be possible to condition the non-linear equations BEFORE they are linearized.  Are there any general strategies to precondition non-linear equations that are independent of the structure of the system of equations?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-05-15T18:40:18.670" Title="How can I precondition a non-linear problem before linearization?" Tags="|preconditioning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2238" PostTypeId="2" ParentId="2237" CreationDate="2012-05-15T18:28:48.533" Score="7" Body="&lt;p&gt;Sure, any approximate nonlinear solver can be used as a nonlinear preconditioner. For example, if your system has directional stiffness, you could use a 1D shooting method as a preconditioner. &lt;a href=&quot;http://www.cs.colorado.edu/~cai/papers/pin.pdf&quot;&gt;Cai and Keyes' (2002) &lt;em&gt;Nonlinearly preconditioned inexact Newton algorithms&lt;/em&gt;&lt;/a&gt; provide another useful example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In PETSc, nonlinear methods such as &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/SNES/SNESQN.html&quot;&gt;quasi-Newton&lt;/a&gt;, &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/SNES/SNESNCG.html&quot;&gt;nonlinear conjugate gradients&lt;/a&gt;, &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/SNES/SNESNGMRES.html&quot;&gt;nonlinear GMRES&lt;/a&gt; (aka. Anderson acceleration), and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/SNES/SNESNRICHARDSON.html&quot;&gt;nonlinear Richardson&lt;/a&gt; can be used with a nonlinear preconditioner. The nonlinear preconditioner is any other &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/SNES/SNES.html&quot;&gt;&lt;code&gt;SNES&lt;/code&gt;&lt;/a&gt; implementation, perhaps a &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/SNES/SNESSHELL.html&quot;&gt;user-defined &quot;shell&quot;&lt;/a&gt; implementation.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-15T18:28:48.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2239" PostTypeId="2" ParentId="2237" CreationDate="2012-05-15T18:40:18.670" Score="4" Body="&lt;p&gt;When using a nonlinear conjugate gradient method to solve a square system $F(x)=0$ by minimizing the residual squared norm $\|F(x)\|_2^2$, it usually pays to precondition the system by means of a linear transformation,&#xA;e.g., computed by an approximate factorization $LU$ of an approximate Jacobian at the starting point. (Often such an approximate Jacobian is available from a simplification of the problem.) Thus one minimizes instead $\|B F(x)\|_2^2$,&#xA;where $B=(LU)^{-1}$, and $r=BF(x)$ is computed by solving $LUr=F(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At more expense (one new approximate factorization per iteration), one can also minimize $B(x)F(x)$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-15T18:40:18.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2240" PostTypeId="2" ParentId="2197" CreationDate="2012-05-15T20:50:35.187" Score="5" Body="&lt;p&gt;It turns out that I can write a Python code (using numpy to do the BLAS operations) faster than the Fortran code compiled with my system's gfortran compiler.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ gfortran -o sn6a sn6a.f90 -O3 -march=native&#xD;&#xA;    &#xD;&#xA;    $ ./sn6a 5500&#xA;1.274224153&#xA;1.274224153&#xA;1.274224153&#xA;   1.9640001      sec per iteration&#xA;&#xA;$ python ./foo1.py&#xA;1.27422415279&#xA;1.27422415279&#xA;1.27422415279&#xA;1.20618661245 sec per iteration&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;foo1.py:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;import scipy.linalg&#xA;import timeit&#xA;&#xA;def specNormDot(A,n):&#xA;    u = numpy.ones(n)&#xA;    v = numpy.zeros(n)&#xA;&#xA;    for i in xrange(10):&#xA;        v  = numpy.dot(numpy.dot(A,u),A)&#xA;        u  = numpy.dot(numpy.dot(A,v),A)&#xA;&#xA;    print numpy.sqrt(numpy.vdot(u,v)/numpy.vdot(v,v))&#xA;&#xA;    return&#xA;&#xA;n = 5500&#xA;&#xA;ii, jj = numpy.meshgrid(numpy.arange(1,n+1), numpy.arange(1,n+1))&#xA;A  = (1./((ii+jj-2.)*(ii+jj-1.)/2. + ii))&#xA;&#xA;t = timeit.Timer(&quot;specNormDot(A,n)&quot;, &quot;from __main__ import specNormDot,A,n&quot;)&#xA;ntries = 3&#xA;&#xA;print t.timeit(ntries)/ntries, &quot;sec per iteration&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and sn6a.f90, a very lightly modified spectral_norm6.f90:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program spectral_norm6&#xA;! This uses spectral_norm3 as a starting point, but does not use the&#xA;! Fortrans&#xA;! builtin matmul and dotproduct (to make sure it does not call some&#xA;! optimized&#xA;! BLAS behind the scene).&#xA;implicit none&#xA;&#xA;integer, parameter :: dp = kind(0d0)&#xA;real(dp), allocatable :: A(:, :), u(:), v(:)&#xA;integer :: i, j, n&#xA;character(len=6) :: argv&#xA;integer :: calc, iter&#xA;integer, parameter :: niters=3&#xA;&#xA;call get_command_argument(1, argv)&#xA;read(argv, *) n&#xA;&#xA;allocate(u(n), v(n), A(n, n))&#xA;do j = 1, n&#xA;    do i = 1, n&#xA;        A(i, j) = Ac(i, j)&#xA;    end do&#xA;end do&#xA;&#xA;call tick(calc)&#xA;&#xA;do iter=1,niters&#xA;    u = 1&#xA;    do i = 1, 10&#xA;        v = AvA(A, u)&#xA;        u = AvA(A, v)&#xA;    end do&#xA;&#xA;    write(*, &quot;(f0.9)&quot;) sqrt(dot_product2(u, v) / dot_product2(v, v))&#xA;enddo&#xA;&#xA;print *, tock(calc)/niters, ' sec per iteration'&#xA;&#xA;contains&#xA;&#xA;pure real(dp) function Ac(i, j) result(r)&#xA;integer, intent(in) :: i, j&#xA;r = 1._dp / ((i+j-2) * (i+j-1)/2 + i)&#xA;end function&#xA;&#xA;pure function matmul2(v, A) result(u)&#xA;! Calculates u = matmul(v, A), but much faster (in gfortran)&#xA;real(dp), intent(in) :: v(:), A(:, :)&#xA;real(dp) :: u(size(v))&#xA;integer :: i&#xA;do i = 1, size(v)&#xA;    u(i) = dot_product2(A(:, i), v)&#xA;end do&#xA;end function&#xA;&#xA;pure real(dp) function dot_product2(u, v) result(w)&#xA;! Calculates w = dot_product(u, v)&#xA;real(dp), intent(in) :: u(:), v(:)&#xA;integer :: i&#xA;w = 0&#xA;do i = 1, size(u)&#xA;    w = w + u(i)*v(i)&#xA;end do&#xA;end function&#xA;&#xA;pure function matmul3(A, v) result(u)&#xA;! Calculates u = matmul(v, A), but much faster (in gfortran)&#xA;real(dp), intent(in) :: v(:), A(:, :)&#xA;real(dp) :: u(size(v))&#xA;integer :: i, j&#xA;u = 0&#xA;do j = 1, size(v)&#xA;    do i = 1, size(v)&#xA;        u(i) = u(i) + A(i, j)*v(j)&#xA;    end do&#xA;end do&#xA;end function&#xA;&#xA;pure function AvA(A, v) result(u)&#xA;! Calculates u = matmul2(matmul3(A, v), A)&#xA;! In gfortran, this function is sligthly faster than calling&#xA;! matmul2(matmul3(A, v), A) directly.&#xA;real(dp), intent(in) :: v(:), A(:, :)&#xA;real(dp) :: u(size(v))&#xA;u = matmul2(matmul3(A, v), A)&#xA;end function&#xA;&#xA;subroutine tick(t)&#xA;    integer, intent(OUT) :: t&#xA;&#xA;    call system_clock(t)&#xA;end subroutine tick&#xA;&#xA;! returns time in seconds from now to time described by t &#xA;real function tock(t)&#xA;    integer, intent(in) :: t&#xA;    integer :: now, clock_rate&#xA;&#xA;    call system_clock(now,clock_rate)&#xA;&#xA;    tock = real(now - t)/real(clock_rate)&#xA;end function tock&#xA;end program&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user389" LastEditorUserId="9" LastEditorDisplayName="user389" LastEditDate="2012-05-16T06:33:55.790" LastActivityDate="2012-05-16T06:33:55.790" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2242" PostTypeId="2" ParentId="1506" CreationDate="2012-05-16T08:25:47.467" Score="3" Body="&lt;p&gt;You might wish to try one of the (local) nonsmooth solvers at my web page&#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&lt;/a&gt;&#xA;using multiple starting points to globalize the search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found CMA-ES quite robust in dimensions up to 50. (It gets very slow though when the dimension  is large.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-16T08:25:47.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2243" PostTypeId="2" ParentId="1329" CreationDate="2012-05-16T08:29:49.583" Score="19" Body="&lt;p&gt;As $\det(kA)=k^n\det A$, the determinant can be made arbitrarily large or small by simple rescaling (which doesn't change the condition number). Especially in high dimensions, even scaling by an innocent factor of 2 changes the determinant by a huge amount.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus never use the determinant to assess condition or closeness to singularity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, for almost all well-posed numerical problems, the condition is closely related to the distance to singularity, in the sense of the smallest relative perturbation needed to make the problem ill-posed. In particular, this holds for linear systems.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-16T08:29:49.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2244" PostTypeId="1" AcceptedAnswerId="2245" CreationDate="2012-05-16T08:46:38.387" Score="25" ViewCount="923" Body="&lt;p&gt;I have matrices $A$ and $G$. $A$ is sparse and is $n\times n$ with $n$ very large (can be on the order of several million.) $G$ is an $n\times m$ tall matrix with $m$ rather small ($1 \lt m \lt 1000$) and each column can only have a single $1$ entry with the rest being $0$'s, such that $G^TG = I$. $A$ is huge, so it is really tough to invert, and I can solve a linear system such as $Ax = b$ iteratively using a Krylov subspace method such as $\mathrm{BiCGStab}(l)$, but I do not have $A^{-1}$ explicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve a system of the form: $(G^TA^{-1}G)x = b$, where $x$ and $b$ are $m$ length vectors. One way to do it is to use an iterative algorithm within an iterative algorithm to solve for $A^{-1}$ for each iteration of the outer iterative algorithm. This would be extremely computationally expensive, however. I was wondering if there is a computationally easier way to go about solving this problem.&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="10134" LastEditDate="2015-10-15T14:07:33.330" LastActivityDate="2015-10-15T14:07:33.330" Title="Solving $(G^TA^{-1}G)x = b$ without inverting $A$" Tags="|linear-algebra|linear-solver|sparse-matrix|krylov-method|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2245" PostTypeId="2" ParentId="2244" CreationDate="2012-05-16T08:59:41.700" Score="23" Body="&lt;p&gt;Introduce the vector $y:=-A^{-1}Gx$ and solve the large coupled system $Ay+Gx=0$, $G^Ty=-b$ for $(y,x)$ simultaneously, using an iterative method. If $A$ is symmetric (as seems likely though you don't state it explicitly) then the system is symmetric (but indefinite, though quasidefinite if $A$ is positive definite), which might help you to choose an appropriate method. (relevant keywords: KKT matrix, quasidefinite matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: As $A$ is complex symmetric, so is the augmented matrix, but there is no quasidefiniteness. You can however use the $Ax$ routine to compute $A^*x=\overline{A\overline x}$; therefore you could adapt a method such as QMR &lt;a href=&quot;ftp://ftp.math.ucla.edu/pub/camreport/cam92-19.pdf&quot;&gt;ftp://ftp.math.ucla.edu/pub/camreport/cam92-19.pdf&lt;/a&gt; (designed for real systems, but you can easily rewrite it for complex systems, using the adjoint in place of the transpose) to solve your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit2: Actually, the (0,1)-structure of $G$ means that you can eliminate $x$ amd the components of $G^Ty$ symbolically, thus ending up with a smaller system to solve. This means messing with the structure of $A$, and pays only when $A$ is given explicitly in sparse format rather than as a linear operator.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-05-16T19:45:37.823" LastActivityDate="2012-05-16T19:45:37.823" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2246" PostTypeId="1" AcceptedAnswerId="2247" CreationDate="2012-05-16T12:52:48.220" Score="25" ViewCount="15951" Body="&lt;p&gt;For a project I am working on (in hyperbolic PDEs) I would like to get some rough handle on the behavior by looking at some numerics. I am, however, not a very good programmer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can you recommend some resources for learning how to effectively &lt;em&gt;code&lt;/em&gt; finite difference schemes in Scientific Python (other languages with small learning curve also welcome)?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you an idea of the audience (me) for this recommendation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I am a pure mathematician by training, and am somewhat familiar with the theoretical aspects of finite difference schemes&lt;/li&gt;&#xA;&lt;li&gt;What I need help with is how to make the computer compute what I want it to compute, especially in a way that I don't duplicate too much of the effort already put in by others (so as to not re-invent the wheel when a package is already available). (Another thing I would like to avoid is to stupidly code something by hand when there are established data structures fitting the purpose.)&lt;/li&gt;&#xA;&lt;li&gt;I have had some coding experience; but I have had none in Python (hence I don't mind if there are good resources for learning a different language [say, Octave for example]). &lt;/li&gt;&#xA;&lt;li&gt;Books, documentation would both be useful, as would collections of example code. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1481" LastActivityDate="2021-08-05T11:03:42.893" Title="Recommendation for Finite Difference Method in Scientific Python" Tags="|python|finite-difference|reference-request|hyperbolic-pde|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2247" PostTypeId="2" ParentId="2246" CreationDate="2012-05-16T14:13:09.233" Score="13" Body="&lt;p&gt;Here is a 97-line example of solving a simple multivariate PDE using finite difference methods, contributed by &lt;a href=&quot;http://web.kaust.edu.sa/faculty/davidketcheson/Home.html&quot;&gt;Prof. David Ketcheson&lt;/a&gt;, from the &lt;a href=&quot;https://github.com/ahmadia/py4sci/&quot;&gt;py4sci repository&lt;/a&gt; I maintain.  For more complicated problems where you need to handle shocks or conservation in a finite-volume discretization, I recommend looking at &lt;a href=&quot;http://numerics.kaust.edu.sa/pyclaw/&quot;&gt;pyclaw&lt;/a&gt;, a software package that I help develop.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;&quot;&quot;Pattern formation code&#xA;&#xA;    Solves the pair of PDEs:&#xA;       u_t = D_1 \nabla^2 u + f(u,v)&#xA;       v_t = D_2 \nabla^2 v + g(u,v)&#xA;&quot;&quot;&quot;&#xA;&#xA;import matplotlib&#xA;matplotlib.use('TkAgg')&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.sparse import spdiags,linalg,eye&#xA;from time import sleep&#xA;&#xA;#Parameter values&#xA;Du=0.500; Dv=1;&#xA;delta=0.0045; tau1=0.02; tau2=0.2; alpha=0.899; beta=-0.91; gamma=-alpha;&#xA;#delta=0.0045; tau1=0.02; tau2=0.2; alpha=1.9; beta=-0.91; gamma=-alpha;&#xA;#delta=0.0045; tau1=2.02; tau2=0.; alpha=2.0; beta=-0.91; gamma=-alpha;&#xA;#delta=0.0021; tau1=3.5; tau2=0; alpha=0.899; beta=-0.91; gamma=-alpha;&#xA;#delta=0.0045; tau1=0.02; tau2=0.2; alpha=1.9; beta=-0.85; gamma=-alpha;&#xA;#delta=0.0001; tau1=0.02; tau2=0.2; alpha=0.899; beta=-0.91; gamma=-alpha;&#xA;#delta=0.0005; tau1=2.02; tau2=0.; alpha=2.0; beta=-0.91; gamma=-alpha; nx=150;&#xA;&#xA;#Define the reaction functions&#xA;def f(u,v):&#xA;    return alpha*u*(1-tau1*v**2) + v*(1-tau2*u);&#xA;&#xA;def g(u,v):&#xA;    return beta*v*(1+alpha*tau1/beta*u*v) + u*(gamma+tau2*v);&#xA;&#xA;&#xA;def five_pt_laplacian(m,a,b):&#xA;    &quot;&quot;&quot;Construct a matrix that applies the 5-point laplacian discretization&quot;&quot;&quot;&#xA;    e=np.ones(m**2)&#xA;    e2=([0]+[1]*(m-1))*m&#xA;    h=(b-a)/(m+1)&#xA;    A=np.diag(-4*e,0)+np.diag(e2[1:],-1)+np.diag(e2[1:],1)+np.diag(e[m:],m)+np.diag(e[m:],-m)&#xA;    A/=h**2&#xA;    return A&#xA;&#xA;def five_pt_laplacian_sparse(m,a,b):&#xA;    &quot;&quot;&quot;Construct a sparse matrix that applies the 5-point laplacian discretization&quot;&quot;&quot;&#xA;    e=np.ones(m**2)&#xA;    e2=([1]*(m-1)+[0])*m&#xA;    e3=([0]+[1]*(m-1))*m&#xA;    h=(b-a)/(m+1)&#xA;    A=spdiags([-4*e,e2,e3,e,e],[0,-1,1,-m,m],m**2,m**2)&#xA;    A/=h**2&#xA;    return A&#xA;&#xA;# Set up the grid&#xA;a=-1.; b=1.&#xA;m=100; h=(b-a)/m; &#xA;x = np.linspace(-1,1,m)&#xA;y = np.linspace(-1,1,m)&#xA;Y,X = np.meshgrid(y,x)&#xA;&#xA;# Initial data&#xA;u=np.random.randn(m,m)/2.;&#xA;v=np.random.randn(m,m)/2.;&#xA;plt.hold(False)&#xA;plt.pcolormesh(x,y,u)&#xA;plt.colorbar; plt.axis('image'); &#xA;plt.draw()&#xA;u=u.reshape(-1)&#xA;v=v.reshape(-1)&#xA;&#xA;A=five_pt_laplacian_sparse(m,-1.,1.);&#xA;II=eye(m*m,m*m)&#xA;&#xA;t=0.&#xA;dt=h/delta/5.;&#xA;plt.ion()&#xA;&#xA;#Now step forward in time&#xA;for k in range(120):&#xA;    #Simple (1st-order) operator splitting:&#xA;    u = linalg.spsolve(II-dt*delta*Du*A,u)&#xA;    v = linalg.spsolve(II-dt*delta*Dv*A,v)&#xA;&#xA;    unew=u+dt*f(u,v);&#xA;    v   =v+dt*g(u,v);&#xA;    u=unew;&#xA;    t=t+dt;&#xA;&#xA;    #Plot every 3rd frame&#xA;    if k/3==float(k)/3:&#xA;        U=u.reshape((m,m))&#xA;        plt.pcolormesh(x,y,U)&#xA;        plt.colorbar&#xA;        plt.axis('image')&#xA;        plt.title(str(t))&#xA;        plt.draw()&#xA;&#xA;plt.ioff()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-05-16T14:40:51.957" LastActivityDate="2012-05-16T14:40:51.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2248" PostTypeId="2" ParentId="2246" CreationDate="2012-05-16T14:18:52.980" Score="10" Body="&lt;p&gt;You could have a look at &lt;a href=&quot;http://fenicsproject.org&quot;&gt;Fenics&lt;/a&gt;, which is a python/C framwork which allows quite general equations to be solved using a special markup language. It mostly uses finite elements though, but worth a look. The &lt;a href=&quot;http://fenicsproject.org/about/features.html#features&quot;&gt;tutorial&lt;/a&gt; should give you an impression of how easy it can be to solve problems.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2012-05-16T14:18:52.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2249" PostTypeId="1" AcceptedAnswerId="2307" CreationDate="2012-05-16T14:39:07.697" Score="17" ViewCount="2326" Body="&lt;p&gt;I have a bounded non-convex 2-D function which I'd like to find the minimum of. The function is quite smooth. Evaluating it is costly. An acceptable error is about 3% of the function's domain in each axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried running the implementation of the DIRECT algorithm in the NLOPT library, but it didn't give a considerable improvement over brute force search in terms of the amount of function evaluations needed for the required accuracy and there were some outliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which other global optimization solvers should I consider?&lt;/p&gt;&#xA;" OwnerUserId="533" LastEditorUserId="9" LastEditDate="2012-05-16T18:07:00.597" LastActivityDate="2016-10-30T20:35:58.727" Title="Finding a global minimum of a smooth, bounded, non-convex 2D function that is costly to evaluate" Tags="|optimization|" AnswerCount="5" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2250" PostTypeId="2" ParentId="2244" CreationDate="2012-05-16T15:46:23.673" Score="0" Body="&lt;p&gt;But we know $G$, $G^T$ and $A$, so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$G^T A^{-1} G x = b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$G G^T A^{-1} G x = G b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $G^T G = I$, then $G^T = G^{-1}$, so $G G^T=I$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A^{-1} G x = G b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A A^{-1} G x = A G b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$G x = A G b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$G^T G x = G^T A G b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x = G^T A G b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless I've missed something, you don't need any iteration, or any solver to calculate x given $G$, $A$ and $b$.&lt;/p&gt;&#xA;" OwnerUserId="1372" LastActivityDate="2012-05-16T15:46:23.673" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2251" PostTypeId="2" ParentId="2249" CreationDate="2012-05-16T15:53:32.307" Score="8" Body="&lt;p&gt;See &lt;/p&gt;&#xA;&#xA;&lt;p&gt;L.M. Rios and N.V. Sahinidis,&#xA;&lt;A HREF=&quot;http://thales.cheme.cmu.edu/dfo/dfo.pdf&quot; rel=&quot;nofollow&quot;&gt;&#xA;Derivative-free optimization: A review of algorithms and&#xA;comparison of software implementations&lt;/A&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for a very useful recent comparison of solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DOI: &lt;a href=&quot;http://dx.doi.org/10.1007/s10898-012-9951-y&quot; rel=&quot;nofollow&quot;&gt;10.1007/s10898-012-9951-y&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="59" LastEditDate="2013-02-18T14:43:30.647" LastActivityDate="2013-02-18T14:43:30.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2252" PostTypeId="2" ParentId="2249" CreationDate="2012-05-16T19:06:00.953" Score="3" Body="&lt;p&gt;For a smooth function, the Efficient Global Optimization method should perform quite well and be dramatically more efficient than DIRECT. Implementations are available in &lt;a href=&quot;http://tomopt.com/tomlab/products/cgo/solvers/ego.php&quot; rel=&quot;nofollow&quot;&gt;TOMLAB&lt;/a&gt; (haven't used it myself) and &lt;a href=&quot;http://dakota.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;DAKOTA&lt;/a&gt; (which I've had some success with).&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2012-05-16T19:06:00.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2254" PostTypeId="1" CreationDate="2012-05-16T22:58:01.637" Score="8" ViewCount="151" Body="&lt;p&gt;Anyone knows references summarizing the history and ideas behind supercomputing including mentioning of developments in parallel programming languages, applications, startup companies (some was establish but didn't survive), needs of industry, etc.&lt;/p&gt;&#xA;" OwnerUserId="304" LastActivityDate="2012-06-02T19:57:52.227" Title="Anyone knows references summarizing the history of supercomputing?" Tags="|parallel-computing|hpc|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2255" PostTypeId="1" AcceptedAnswerId="2280" CreationDate="2012-05-17T03:51:25.653" Score="15" ViewCount="1547" Body="&lt;p&gt;As much as I try to find a concise explanation on the internet, I can't seem to grasp the concept of a mimetic finite difference, or how it even relates to standard finite differences.  It would be really helpful to see some simple examples of how they are implemented for classic linear PDE's (hyperbolic, elliptic and parabolic).  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-05-18T04:50:38.850" LastActivityDate="2015-03-10T04:16:05.720" Title="Illustrative examples of mimetic finite difference methods" Tags="|pde|finite-difference|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2256" PostTypeId="1" CreationDate="2012-05-17T03:52:01.577" Score="5" ViewCount="244" Body="&lt;p&gt;I am interested in the solutions of a very large quadratic programming (QP) problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\min_{x \in \mathbb{R}^n} &amp;amp; x^T Q x\\&#xA;\mathrm{subject\ to} &amp;amp; A x = b\\&#xA;&amp;amp; x \in \{0,1\}^n&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $n=10^7$, $Q$ is a dense, positive-semidefinite matrix whose entries are natural numbers that can be computed rapidly, and $A \in \mathbb{N}^{n \times 20}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a suitable algorithm for such a large problem, and what is a good implementation?&lt;/p&gt;&#xA;" OwnerUserId="1484" LastEditorUserId="276" LastEditDate="2012-06-16T18:25:01.863" LastActivityDate="2012-06-16T18:25:01.863" Title="What is a suitable algorithm for solving a large mixed-integer quadratic program?" Tags="|optimization|quadratic-programming|mixed-integer-programming|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2257" PostTypeId="2" ParentId="2214" CreationDate="2012-05-17T08:04:44.723" Score="3" Body="&lt;p&gt;Here is an idea of using a simple space filling curve modified to return same key for the symmetry cases (all code snippets are in python).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Simple space-filling curve&#xA;def forge_key(i, j, k, l, n): &#xA;  return i + j*n + k*n**2 + l*n**3&#xA;&#xA;# Considers the possible symmetries of a key&#xA;def forge_key_symmetry(i, j, k, l, n): &#xA;  return min(forge_key(i, j, k, l, n), &#xA;             forge_key(j, i, l, k, n), &#xA;             forge_key(k, l, i, j, n), &#xA;             forge_key(l, k, j, i, n)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The example is python but if you inline the functions into your fortran code and unroll your inner loop for (i, j, k, l), you should get decent performance.&lt;/li&gt;&#xA;&lt;li&gt;You could compute the key using floats and then convert the key into integer to use as an index, this would allow the compiler to use the floating point units (e.g. AVX is available).&lt;/li&gt;&#xA;&lt;li&gt;If N is a power of 2 then the multiplications would be just bit shifts.&lt;/li&gt;&#xA;&lt;li&gt;The treatment for the symmetries is not efficient in memory (i.e. it does not produce a continuous indexing) and uses around of 1/4 of the total index array entries.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is a test example for n = 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(n):&#xA;  for j in range(n):&#xA;    for k in range(n):&#xA;      for l in range(n):&#xA;        key = forge_key_symmetry(i, j, k, l, n)&#xA;        print i, j, k , l, key&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output for n = 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;i j k l key&#xA;0 0 0 0 0&#xA;0 0 0 1 1&#xA;0 0 1 0 1&#xA;0 0 1 1 3&#xA;0 1 0 0 1&#xA;0 1 0 1 5&#xA;0 1 1 0 6&#xA;0 1 1 1 7&#xA;1 0 0 0 1&#xA;1 0 0 1 6&#xA;1 0 1 0 5&#xA;1 0 1 1 7&#xA;1 1 0 0 3&#xA;1 1 0 1 7&#xA;1 1 1 0 7&#xA;1 1 1 1 15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If of interest, the inverse function of forge_key is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Inverse of forge_key&#xA;def split_key(key, n): &#xA;  d = key / n**3&#xA;  c = (key - d*n**3) / n**2&#xA;  b = (key - c*n**2 - d*n**3) / n &#xA;  a = (key - b*n - c*n**2 - d*n**3)&#xA;  return (a, b, c, d)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1070" LastEditorUserId="1070" LastEditDate="2012-05-17T14:09:09.433" LastActivityDate="2012-05-17T14:09:09.433" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2258" PostTypeId="2" ParentId="2249" CreationDate="2012-05-17T08:37:01.683" Score="1" Body="&lt;p&gt;Since the function is smooth, Newton's method will be the overwhelmingly most efficient method to find a minimum. Since the function is not convex you will have to apply the usual tricks to make Newton's method converge (Levenberg-Marquardt modification, line search or trust region to globalize). If you can't get derivatives of your function try either computing it via finite differences or using a BFGS update. If you suspect that the problem has more than one local minimum, one would simply start Newton's method from a bunch of randomly or not quite so randomly chosen points and see where they converge.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-17T08:37:01.683" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2259" PostTypeId="2" ParentId="2197" CreationDate="2012-05-17T09:12:21.997" Score="9" Body="&lt;p&gt;user389's answer has been deleted but let me state that I'm firmly in his camp: I fail to see what we learn by comparing micro-benchmarks in different languages. It doesn't come as much of a surprise to me that C and Fortran get pretty much the same performance on this benchmark given how short it is. But the benchmark is also boring since it can easily be written in both languages in a couple dozen lines. From a software viewpoint, that is not a representative case: we should care about software that has 10,000 or 100,000 lines of code and how compilers do on that. Of course, at that scale, one will quickly find out other things: that language A requires 10,000 lines while language B requires 50,000. Or the other way around, depending on what you want to do. And suddenly it's not so much about the difference in milliseconds upon execution anymore, but a difference of months of developer time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, it doesn't matter much to me that maybe my application could be 50% faster if I developed it in Fortran 77 if instead it will only take me 1 month to get it to run correctly while it would take me 3 months in F77. The problem with the question here is that it focuses on an aspect (individual kernels) that isn't relevant in practice in my view.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-17T09:12:21.997" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2260" PostTypeId="2" ParentId="2173" CreationDate="2012-05-17T09:23:42.857" Score="8" Body="&lt;p&gt;I think you phrase the question too narrowly. In my view, a useful attitude is to live under the assumption that only changes to the data structures and algorithms can yield significant performance gains on codes that are more than a few 100 lines, and I believe that I have yet to find a counterexample to this claim.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-17T09:23:42.857" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2261" PostTypeId="2" ParentId="2256" CreationDate="2012-05-17T12:35:35.747" Score="1" Body="&lt;p&gt;This is a mixed-integer QP, not a positive semidefinite QP. (One can make any QP with binary constraints psd by adding a large multiple of $x^Tx-e^Tx$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Couenne ( &lt;a href=&quot;https://projects.coin-or.org/Couenne&quot; rel=&quot;nofollow&quot;&gt;https://projects.coin-or.org/Couenne&lt;/a&gt; ) might be an appropriate solver for this.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-17T12:35:35.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2262" PostTypeId="2" ParentId="2244" CreationDate="2012-05-17T14:02:32.947" Score="7" Body="&lt;p&gt;Following Arnold's reply, there is something you can do to simplify the problem. Specifically, rewrite the system as $Ay+Gx=0, G^Ty=-b$. Then note that from the statement that $G$ is tall and narrow and each row has only one 1 and zeros otherwise, then the statement $G^Ty=-b$ means that a subset of the elements of $y$ have a fixed value, namely the elements of $-b$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us say that for simplicity that $G$ has $m$ columns and $n$ rows and that exactly the first $m$ rows have ones in them and that be reordering the elements of $x$ I can make it so that $G$ has the $m \times m$ identity matrix at the top and a $n-m \times m$ zero matrix at the bottom. Then&#xA;I can partition $y=(y_c,y_f)$ into $m$ &quot;constrained&quot; and $n-m$ &quot;free&quot; elements so that $y_c=-b$. I can also partition $A$ so that $A=\begin{pmatrix} A_{cc} &amp;amp; A_{cf} \\ A_{fc} &amp;amp; A_{ff} \end{pmatrix}$. From the equation $Ay+Gx=0$ I then get the following:&#xA;$$&#xD;&#xA;  A_{cc} y_c + A_{cf} y_f + x = 0, \\&#xD;&#xA;  A_{fc} y_c + A_{ff} y_f = 0&#xD;&#xA;$$&#xA;and using what we know about $y_c$ we have from the second of these equations&#xA;$$&#xD;&#xA;  A_{ff} y_f = A_{fc} b &#xD;&#xA;$$&#xA;and consequently&#xA;$$&#xD;&#xA;  x = A_{cc} b - A_{cf} A_{ff}^{-1} A_{fc} b.&#xD;&#xA;$$&#xA;In other words, the only matrix you have to invert is the subset of $A$ whose rows and columns are not mentioned in $G$ (the null space of $G$). This you can easily do: (i) compute  $z=A_{fc} b$; (ii) use whatever solver you have to solve $A_{ff} h = z$; (iii) compute $x = A_{cc} b - A_{cf} h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, given the structure of $G$, solving the linear system you have is really not more difficult than solving a single linear system with $A$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-17T14:02:32.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2263" PostTypeId="1" AcceptedAnswerId="2264" CreationDate="2012-05-17T19:54:20.830" Score="4" ViewCount="142" Body="&lt;p&gt;I have a $p \times n$ matrix $B$ (where $n &amp;lt; p$) with orthonormal columns and would like to find a numerically efficient way to extend this matrix to get a complete $p$-dimensional orthonormal basis. In other words I want to compute a $p$ by $p-n$ complementary matrix $W$ so that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$[B|W]$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;is orthonormal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I am computing the QR factorisation of the rank $p-n$ projection matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$P=I-BB^T= QR$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the LAPACK routines &lt;code&gt;dgeqp3&lt;/code&gt; and &lt;code&gt;dorgqr&lt;/code&gt; and discarding the trailing $p$ columns of $Q$. I feel sure that there must be a more efficient method because in my application $B$ is almost a complete basis i.e $p-n$ is small.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have wondered about building a complementary basis $R$ recursively by working through the columns of $P$. At each step adding a column $C$ to the current $R$ only if $\det([R|C]^T[R|C)\neq0$. After I have $p-n$ independent vectors in $R$ I could form $W$ as $W=(R^TR)^{-1/2} R$. However, I'm not sure if this is a numerically stable solution, or if there is another more efficient method.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally I would like a method I can implement using standard LAPACK routines. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Originally asked on stackoverflow &lt;a href=&quot;https://stackoverflow.com/questions/10642023/efficient-computation-of-the-extension-of-a-linear-basis-to-completion-when-the&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1485" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-05-17T21:59:22.873" Title="Efficient computation of the extension of a linear basis to completion when the basis is almost complete (ideally using LAPACK routines)" Tags="|linear-algebra|basis-set|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2264" PostTypeId="2" ParentId="2263" CreationDate="2012-05-17T20:09:17.690" Score="2" Body="&lt;p&gt;Just compute a QR factorization of $B$. The last $p-n$ columns then have he required property of forming a basis complement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large $n$ and small $p-n$, you can take a random $p\times k$ matrix $X$ with $k=p-n$ or slightly larger (for an increased likelihood of having numerical stability), and compute a QR factorization of $X-B(B^TX)$, which, with probability one, spans the complementary subspace.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1485" LastEditDate="2012-05-17T21:59:22.873" LastActivityDate="2012-05-17T21:59:22.873" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2265" PostTypeId="1" AcceptedAnswerId="2270" CreationDate="2012-05-17T23:39:55.150" Score="2" ViewCount="36" Body="&lt;p&gt;I have a PetscBag in my program that I want to write to a filename I have stored as a string. If I want to write to &lt;code&gt;stdout&lt;/code&gt;, there is a pre-made PetscViewer macro for that. How do I create a viewer for a PetscBag that writes to a specified file?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-05-18T06:02:33.887" Title="How do I print a PetscBag to a specific file?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2266" PostTypeId="1" CreationDate="2012-05-18T00:33:14.070" Score="8" ViewCount="603" Body="&lt;p&gt;My challenge is to solve the following system of equations, which describe gas combustion in porous media:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Continuity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\varepsilon \frac{\partial \rho_g}{\partial t} +\frac{\partial}{\partial x} \left(\rho_g u_x\right)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Darcy law (momentum)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_x=-\frac{k}{\mu} \frac{\partial p}{\partial x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Equation of state, note the variable temperature&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho_g=\frac{M_Rp}{RT_g(x)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Energy equation for the gas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) Energy equation for the solid phase&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have succesfully descritised and solved the case where the velocity, pressure and density are assumed constant, i.e the first three equations drop out. But solving the gasodynamical part proved to be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying an upwind scheme to 1) (as was suggested here: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1960/a-good-finite-difference-for-the-continuity-equation&quot;&gt;A good finite difference for the continuity equation&lt;/a&gt;)  yeilds a really harsh stability criteria on the timestep, I am forced to have it as low as 1e-6 with a 1e-2 spacial timestep, even when I take the isothermal case, disregarding the combustion for the time being. And I need at least 1e-3 to resolve the energy equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first three equations can be also coupled together to form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;6) $ \frac{\partial p}{\partial t} +C\frac{\partial^2}{\partial x^2} \left(p^2 \right)=0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;but &lt;strong&gt;only in the isothermal case&lt;/strong&gt;, so that is of little help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that people have solved 1)-5) and 6) before, but I couldn't find a description of the schemes they used. I tryed searching articles on compressible flow in porous media specifically, but those all deal with way more complex models (multiphase, deformable solids etc) and use very complicated solving methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone suggest a good FD scheme for (1)-(3) or say how the stability criteria is formed if one just uses upwinding like I did?&lt;/p&gt;&#xA;" OwnerUserId="1088" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-05-20T01:36:09.827" Title="Finite difference scheme for compressible nonisothermal flow in porous media" Tags="|finite-difference|fluid-dynamics|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2267" PostTypeId="1" AcceptedAnswerId="2295" CreationDate="2012-05-18T01:56:28.877" Score="20" ViewCount="14446" Body="&lt;p&gt;I'm looking at using OpenFOAM for solving basic internal flows in CFD. What is the best way to get started, and could anyone please point me to a good online reference to go to with any questions I might have once I dive in? I've heard that it is quite niche as of now and many solver functions are not that well documented. Obviously the documentation that comes with the package itself is a start but is there a third-party reference (something like an OpenFOAM wiki)?&lt;/p&gt;&#xA;" OwnerUserId="1387" LastEditorUserId="383" LastEditDate="2012-05-22T12:30:23.737" LastActivityDate="2016-03-23T15:02:09.717" Title="How to get started with OpenFOAM for CFD" Tags="|fluid-dynamics|openfoam|" AnswerCount="6" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2268" PostTypeId="1" AcceptedAnswerId="2272" CreationDate="2012-05-18T03:33:26.823" Score="7" ViewCount="1144" Body="&lt;p&gt;when optimizing my code, i find myself often writing something like the following ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do i = 1,n&#xA;    r = t(i)&#xA;    y(i) = r*r*2.0&#xA;    f(i) = r*3.5&#xA;enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;what i am doing with my variable, &lt;code&gt;r&lt;/code&gt; is really irrelavent except that it is used more than once, and is most likely used as many as 5 or 6 times. now, in my head, i do this because i figure that accessing an array is slow and i want to store that variable so that i can retrieve it faster the next time. is this a correct line of thinking though? i wonder if, since i have used t(i), it is just as easy to reference in short succession as it is to reference &lt;code&gt;r&lt;/code&gt; again. does storing a value from an array into a temporary variable make it quicker to us that value again, or is it just the same as if i had written&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do i = 1,n&#xA;    r = t(i)&#xA;    y(i) = t(i)*t(i)*2.0&#xA;    f(i) = t(i)*3.5&#xA;enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;? if they are the same, then i am actually slowing things down by writing an extra assignment (&lt;code&gt;r = t(i)&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2012-05-18T13:25:31.513" Title="is accessing an element in an array slower than accessing a variable?" Tags="|optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2269" PostTypeId="2" ParentId="2268" CreationDate="2012-05-18T04:22:21.483" Score="8" Body="&lt;p&gt;Provided there is not possible aliasing (prohibited by Fortran), any modern compiler should interpret these as being identical. You should use whichever you find easier to read or examine in a debugger.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-18T04:22:21.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2270" PostTypeId="2" ParentId="2265" CreationDate="2012-05-18T06:02:33.887" Score="2" Body="&lt;p&gt;Viewers are discussed in the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf&quot; rel=&quot;nofollow&quot;&gt;PETSc user guide&lt;/a&gt;.  Create one using &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Viewer/PetscViewerASCIIOpen.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;PetscViewerASCIIOpen&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Viewer/PetscViewerBinaryOpen.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;PetscViewerBinaryOpen&lt;/code&gt;&lt;/a&gt;, then call &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/PetscBagView.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;PetscBagView&lt;/code&gt;&lt;/a&gt; on your viewer.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-05-18T06:02:33.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2271" PostTypeId="1" AcceptedAnswerId="2273" CreationDate="2012-05-18T08:28:51.980" Score="3" ViewCount="1722" Body="&lt;p&gt;Suppose a 3D configuration of points is given, $X\in\mathbb{R}^{n\times 3}$, and a matrix $Q\in\mathbb{3\times 2}$, with &lt;em&gt;orthonormal&lt;/em&gt; columns. Now, suppose a mapping to 2D is obtained as&#xA;$$Y=XQ.$$ Could it be safely assumed that $Y$ simply corresponds to a rotated or reflected configuration $X$ restricted to first 2 axes? I read that the application of orthogonal matrices, as in the above, preserves isometries in Euclidean space, meaning that it should correspond a rotation, reflection and translation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www2.bc.edu/~reederma/Linalg17.pdf&quot; rel=&quot;nofollow&quot;&gt;https://www2.bc.edu/~reederma/Linalg17.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(p.4), it is stated that &quot;Orthogonal matrices with determinant $-1$ are not rotations, but most of them are not reﬂections either&quot;. On the other hand, the statement from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.utk.edu/~freire/teaching/m251f10/m251s10orthogonal.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.utk.edu/~freire/teaching/m251f10/m251s10orthogonal.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(p.1) makes a general statement on orthogonal matrices corresponding to a rotation and reflection. A claim that a $3\times 3$ orthogonal matrix $Q$ with determinant $-1$ corresponds to a &lt;em&gt;rotation + reflection&lt;/em&gt; is given in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ocw.nthu.edu.tw/ocw/upload/20/201/AP1-Operator.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ocw.nthu.edu.tw/ocw/upload/20/201/AP1-Operator.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(p. 3). As you may observe, the claims are not the same, i.e., there is no certainty that the application of any orthogonal $Q$ corresponds to a rigid transformation of on a configuration (rotation and/or reflection). So, I'm interested if a general statement that the application of an orthogonal matrix simply corresponds to a &lt;em&gt;rigid transformation&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.....................&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another concept that adds to confusion is the &lt;em&gt;orthogonal projector matrix&lt;/em&gt;, $P\in\mathbb{R}^{n\times n}$. What does $$Z=PX$$ imply, and how does it differ from the above $Y=XQ$? &lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-05-18T13:11:32.067" LastActivityDate="2012-05-18T13:37:34.477" Title="Application of an orthogonal matrix to a 3D configuration of point" Tags="|linear-algebra|matrix|computational-geometry|geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2272" PostTypeId="2" ParentId="2268" CreationDate="2012-05-18T13:25:31.513" Score="4" Body="&lt;p&gt;The best thing to do is to first write the code in the most readable form&#xA;(helps understanding and debugging, and making changes long after the code was written), and then use a profiler (on the machine where you intend to use the code) to find out where the most costly statements are, and how the time is affected by altering these. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pieces of code that are not among the most costly need not be optimized at all, as the savings is minor, and readability suffers. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-18T13:25:31.513" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2273" PostTypeId="2" ParentId="2271" CreationDate="2012-05-18T13:37:34.477" Score="3" Body="&lt;p&gt;(i) Think of $Q$ to be the first two columns of an orthogonal matrix $U$. The rows of $XU$ are the points viewed in a rotated and/or reflected coordinate system, &#xA;(i.e., a rotation or a rotation followed by a reflection) and distances are preserved in this transformation. (As 0 is preserved, there is no translation involved.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discarding the last column of the transformed data means that you look at a 2-dimensional projection of the rotated/reflected point set. The projection generally changes distances. Thus multiplication with rectangular orthogonal matrices need not be an isometry, and in your case it isn't. Thus your transformation is not rigid. (You can check this easily by comuting a few random distences before and after the transformation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as you can change the sign of the third column of $U$, you can always make the determinant 1, you may interpret your transformation as the result of a rotation only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ii)  $Z=PX$ is a completely unrelated transformation, as it mixes different points to create a new set of points.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-18T13:37:34.477" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2275" PostTypeId="1" CreationDate="2012-05-18T18:15:06.690" Score="6" ViewCount="511" Body="&lt;p&gt;When we calculate products of multiple matrices, e.g., $ABC$, do you think it can be done in a cheaper way than as two consecutive multiplications? Note that I'm not talking about applying matrices to vectors but exactly matrix multiplication.  &lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2012-05-18T18:29:09.963" LastActivityDate="2012-05-21T03:17:56.620" Title="Multiply Multiple Sparse Matrices" Tags="|linear-algebra|sparse-matrix|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2276" PostTypeId="2" ParentId="2275" CreationDate="2012-05-18T21:30:57.133" Score="2" Body="&lt;p&gt;If $A$,$B$, and $C$ are the same size and don't have interactions between their sparsity patterns, then it should take the same amount of time to multiply them in any order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, however, the matrices are different sizes, then you can gain something my multiplying in a specific order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After multiplying one of the matrices by one of the the others, you will have to store an intermediate matrix. The multiplication will be faster if you pick the order of multiplications such that the size of intermediate matrices is minimized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different sparsity patterns can interact, though, so watch out. In general you want to minimize the number of entries in the intermediate matrices, not the overall size.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-05-18T21:30:57.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2277" PostTypeId="2" ParentId="2275" CreationDate="2012-05-19T00:58:04.970" Score="5" Body="&lt;p&gt;The answer completely depends upon how you've chosen to represent your matrices (e.g., unstructured dense, sparse, analytic, low-rank, hierarchically low-rank, etc.), whether or not there is a priori knowledge of properties of the product, and whether or not you are happy with forming an approximation to $M=ABC$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since most people seem to be thinking of the sparse case, I'll provide a different viewpoint. Suppose that it is known that $M$ is low-rank, with rank $r$, but that $A$, $B$, and $C$ are dense matrices, each of size $n \times n$, where $n \gg r$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One &lt;em&gt;could&lt;/em&gt; form $M=A(BC)$ with $4n^3$ flops, but a smarter method would start by generating a random matrix $\Omega \in \mathbb{F}^{n \times f(r)}$, where $f(r)$ is slightly greater than $r$, and recognize that $M \Omega$ almost certainly spans the column space of $M$. The $QR$ factorization can be computed in $O(nr^2)$ work (since $f(r)$ is $O(r)$), yielding a unitary matrix $Q$, and thus we can approximate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xD;&#xA;Mx \approx QQ&amp;#39;Mx = QQ&amp;#39;ABCx&#xD;&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for any vector $x$. Then one can compute $Q&amp;#39;M$ such that $Q(Q&amp;#39;M)$ is a low-rank approximation to $M$. Thus, once $M \Omega$ has been formed, only $O(n r^2)$ work is required to form $M$ with high probability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case where $A$, $B$, and $C$ are just dense matrices, forming $M \Omega$ requires $O(n^2 r)$ work, which dominates the $O(n r^2)$ work required afterward, but this is still much better than the $O(n^3)$ naive method. Since your title claims that $A$, $B$, and $C$ are sparse, $M \Omega$ can be formed in $O(nr)$ work, so the overall complexity would be $O(nr^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the &lt;a href=&quot;http://arxiv.org/abs/0909.4061&quot;&gt;review paper by Halko et al.&lt;/a&gt; for a more complete description of the ideas.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-05-19T01:16:36.120" LastActivityDate="2012-05-19T01:16:36.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2278" PostTypeId="2" ParentId="2275" CreationDate="2012-05-19T01:02:26.310" Score="1" Body="&lt;p&gt;It is likely that some algorithms for matrix multiply could benefit by considering a single 3-multiply rather than two 2-multiplies. For example one might be able to better optimize memory access if one considers all of the work at once. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this will be a substantially more complex and less general algorithm. There is a trade-off between writing your own fine-tuned algorithm and restricting yourself to a set of well understood and established operations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question was &quot;can we improve upon the standard solution in this specific case?&quot;&#xA;The answer to this question is almost always yes. A good followup question is &quot;is it worth it?&quot;&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-05-21T03:17:56.620" LastActivityDate="2012-05-21T03:17:56.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2279" PostTypeId="1" AcceptedAnswerId="2364" CreationDate="2012-05-17T23:11:49.477" Score="15" ViewCount="14820" Body="&lt;p&gt;I am very interested in computational physics and it is great lot of fun studying these topics. Since I am planning to go one semester abroad, I was wondering what universities are known for computational physics? Especially in regard to universities in the USA?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that computational physics is not a huge and independent branch of physics, but often integrated in the theoretical physics departments. But nevertheless I would very much appreciate any hints and suggestions on active and growing research groups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I was asked to add a little more detail on which topic I am an interested in. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is a bit difficult, since I am in my 4.th Bachelor semester and know near to nothing about i.e. lattice quantum field theory. So its hard for me to say that I would prefer a specific research field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping to find a kind of university/department, that is giving several lectures on computational physics or offers a broad spectrum of possibilities, so that I can specialize as soon as I obtain the theoretical background.&lt;/p&gt;&#xA;" OwnerUserId="1544" OwnerDisplayName="ftiaronsem" LastEditorUserId="123" LastEditDate="2012-05-28T06:30:58.110" LastActivityDate="2013-04-01T08:35:03.700" Title="Universities known for computational physics" Tags="|education|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2280" PostTypeId="2" ParentId="2255" CreationDate="2012-05-19T08:29:15.407" Score="4" Body="&lt;p&gt;Not sure if it is the answer you wanted, but seeing as no-one else have answered I can mention the GPL'd &lt;a href=&quot;http://www.sintef.no/MRST&quot; rel=&quot;nofollow&quot;&gt;MATLAB Reservoir Toolbox&lt;/a&gt;, which uses mimetic solvers for pressure equations in reservoir simulation. Seeing as this equation,&#xA;$$-\nabla \frac{K}{\mu} \nabla p = q$$&#xA;reduces to the typical elliptic test equation,&#xA;$\Delta p = 0$ (Poisson) for constant permeability/viscosity ratio, you can probably get some understanding out of the MRST solvers. MRST supports fully unstructured grids using different mimetic methods, where mimetic here refers to a mimicking of the inner product required for setting up mass balance equations. You will probably not need any understanding of reservoir simulations to understand this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good example to start with is described &lt;a href=&quot;http://www.sintef.no/Projectweb/MRST/Tutorials/Flow-Solver-Tutorial/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. The examples included use MATLAB's block script functionality, where you can use shift-enter to step through the steps and inspect the data at each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relevant articles can be found &lt;a href=&quot;http://www.sintef.no/Projectweb/MRST/Publications/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. The first paper goes through the formulation of the mimetic inner product so you can have a readalong with the code. If you haven't got MATLAB or is unfamiliar with the language, this is probably not very helpful - but I think the simple examples should be compatible with Octave as well.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastEditorUserId="4912" LastEditDate="2014-04-15T01:26:46.737" LastActivityDate="2014-04-15T01:26:46.737" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2281" PostTypeId="1" CreationDate="2012-05-19T12:06:33.567" Score="2" ViewCount="3205" Body="&lt;p&gt;If there are any COMSOL gurus here, I want to simulate a 3D RF system infinite in 1 direction. I realize I need to use PML's, but am unsure how to configure them. it seems that simply scaling them by lambda isn't enough - anyone know if I need to configure the material itself in a special way?&#xA;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I want to solve a problem containing a infinite (in +z direction) cylindrical waveguide filled with vacuum with PEC walls. How can I replace (some of) the infinite waveguide with a PML?&lt;/p&gt;&#xA;" OwnerUserId="1494" LastEditorUserId="1494" LastEditDate="2012-05-19T21:32:23.497" LastActivityDate="2012-06-17T07:04:56.007" Title=" COMSOL - Implementing Perfectly Matched Layers (3D)" Tags="|finite-element|comsol|electromagnetism|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2282" PostTypeId="2" ParentId="2266" CreationDate="2012-05-20T01:36:09.827" Score="2" Body="&lt;p&gt;For hyperbolic problems solved explicitly, you must satisfy a CFL (Courant, Friedrichs, Lewy) condition. This ensures that the scheme will use data only from the domain of dependence for the differential equation. For more information on the CFL condition and why it is necessary, you can read e.g. pp.215-218 in Finite Difference Methods for Ordinary and Partial Differential Equations by LeVeque. Also in that chapter, there are some alternative methods presented but the stability criteria is still there. The CFL condition for the upwind method in 1D is $\frac{u\Delta t}{\Delta x} &amp;lt; 1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Discontinuous Galerkin (DG) methods the use of strong-stability preserving (SSP) Runge-Kutta methods is becoming popular. Though these are explicit methods, they often allow some multiple of the usual Courant number to be used unlike a simple forward Euler method. That means you can take longer time steps but at a greater cost per time step. It might be possible to adapt SSPRK methods to your problem, but I have only seen it done for DG methods and my understanding of their applicability is limited. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be possible to use an implicit in time method since they are unconditionally stable. In order to keep the accuracy at an acceptable level, you may end up back with the original restriction on the time step. LeVeque's book seems to suggest that using backward Euler or an Adams method for the time discretization and central differencing for the spatial derivative might work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I second the vote for Finite Volume methods or, if you want a challenge, DG methods. &lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2012-05-20T01:36:09.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2283" PostTypeId="1" AcceptedAnswerId="2287" CreationDate="2012-05-20T13:36:53.997" Score="28" ViewCount="6748" Body="&lt;p&gt;I know of, and have used &lt;a href=&quot;http://cens.ioc.ee/projects/f2py2e/&quot; rel=&quot;noreferrer&quot;&gt;f2py2e&lt;/a&gt; to wrap some old Fortran 77 code, but my understanding is that it does not work with newer Fortran 95 code. I've researched what I should use, and have come across fwrap and G3 f2py, neither of which seem to give any explanation of their current state, or how to use them (beyond basic usage).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also seen that the version of f2py has the option to use the third generation f2py, but it is commented as being non-functional. Given this, I don't know which project I should use for a uni project. Which one should I use for new code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. This is basically the same question as &lt;a href=&quot;https://stackoverflow.com/questions/10665717/current-best-method-for-wrapping-modern-fortran-code-with-python&quot;&gt;https://stackoverflow.com/questions/10665717/current-best-method-for-wrapping-modern-fortran-code-with-python&lt;/a&gt;, it was suggested that asking here might give better answers.&lt;/p&gt;&#xA;" OwnerUserId="1437" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-06-23T02:48:02.837" Title="Which package should I use to wrap Modern Fortran Code with Python?" Tags="|python|fortran|" AnswerCount="8" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2284" PostTypeId="1" CreationDate="2012-01-18T23:36:57.493" Score="3" ViewCount="1291" Body="&lt;p&gt;I'm trying to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax = b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;given a vector $b$ and a large, symmetric positive definite, sparse, banded matrix $A$ that has a very poor condition number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know several iterative methods that could be used for this task (but haven't implemented them all): &lt;br&gt;&#xA;a) (Preconditioned) Conjugate Gradient. &lt;br&gt;&#xA;b) Simple Gradient Descent. &lt;br&gt;&#xA;c) SSOR / Gauss-Seidel. &lt;br&gt;&#xA;d) Multigrid flavors of the above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I know that there are sparse direct solvers but I have been reluctant to try because they seem very complex and hard to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's the method of choice here?&lt;/p&gt;&#xA;" OwnerDisplayName="Rafael" LastActivityDate="2012-05-20T20:04:43.250" Title="What is the best way to solve Ax = b (with A large, spd, sparse, banded and poorly conditioned)?" Tags="|linear-algebra|optimization|convex-optimization|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2285" PostTypeId="2" ParentId="2284" CreationDate="2012-01-18T23:46:45.527" Score="1" Body="&lt;p&gt;There are several other better and more complicated iterative methods (gmres, bicgstab, etc.), for example here is a list of methods that Matlab &lt;a href=&quot;http://www.mathworks.com/help/techdoc/ref/f16-5872.html#f16-6565&quot; rel=&quot;nofollow&quot;&gt;supports&lt;/a&gt;. If the matrix is very large, direct methods could do very poorly, depending on the sparsity structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why do you want to implement the method yourself in the first place? If the system is large and poorly conditioned it is unlikely that you will be able to do better yourself than to use a highly-optimized method already included in a mature library.&lt;/p&gt;&#xA;" OwnerUserId="713" OwnerDisplayName="Kirill" LastActivityDate="2012-01-18T23:46:45.527" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2286" PostTypeId="2" ParentId="2283" CreationDate="2012-05-20T15:34:53.537" Score="3" Body="&lt;p&gt;You could use &lt;a href=&quot;http://fortwrap.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;FortWrap&lt;/a&gt; to get you halfway there. This will create a C++ interface to your Fortran code, which could then be wrapped with &lt;a href=&quot;http://www.swig.org/&quot; rel=&quot;nofollow&quot;&gt;SWIG&lt;/a&gt; to get a Python interface. We do this on a somewhat large base of Fortran code and it works quite well (in fact, FortWrap was developed by a member of our team for this exact purpose).&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2012-05-20T15:34:53.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2287" PostTypeId="2" ParentId="2283" CreationDate="2012-05-20T15:39:24.033" Score="20" Body="&lt;p&gt;You can use the Python builtin &lt;a href=&quot;http://docs.python.org/library/ctypes.html&quot;&gt;ctypes&lt;/a&gt; module as described on &lt;a href=&quot;http://fortran90.org/src/best-practices.html#interfacing-with-python&quot;&gt;fortran90.org&lt;/a&gt;.  It is pretty straight forward and doesn't require any external dependencies.  Also, the &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/routines.ctypeslib.html&quot;&gt;ndpointer&lt;/a&gt; arg type helper is very handy.&lt;/p&gt;&#xA;" OwnerUserId="486" LastActivityDate="2012-05-20T15:39:24.033" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2288" PostTypeId="2" ParentId="1749" CreationDate="2012-05-20T16:54:46.980" Score="11" Body="&lt;p&gt;Adding squared penalty terms to get rid of constraints is a simple approach giving an accuracy of order 1/penalty factor only. Hence it is not recommended for high accuracy unless you let the penalty go to infinity during the computation. But a high penalty factor makes the Hessian very ill-conditioned,&#xA;which limits the total accuracy achievable without taking into account the constraints explicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that bound constraints are &lt;em&gt;much&lt;/em&gt; easier to handle than general constraints, whence they are virtually never converted to penalties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solver L-BFGS-B (used with an about 5-dimensional history) usually solves bound constrained problems very reliably and fast in both low high dimensions.&#xA;Exceptions are misconvergence on problems that can become very flat far off the solutions, where it is easy to get stuck with a descent method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We made lots of experiments on very diverse functions in many different dimensions, with many different solvers available, as we needed a very robust bound-constrained solver as part of our global optimization software.&#xA;L-BFGS-B clearly stands out as general purpose method, though of course on sme problems other solvers perform significantly better. Thus I'd recommend L-BFGS-B as a first choice, and would try alternative techniques just in case L-BFGS-B handles your particular class of problems poorly.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-20T16:54:46.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2290" PostTypeId="2" ParentId="2284" CreationDate="2012-05-20T17:12:46.363" Score="2" Body="&lt;p&gt;If your bandwidth is small, a direct method will be by far the fastest, and if implemented with iterative refinement can provide the solution to almost working accuracy unless the condition is so poor that no method provides any accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As your matrix is positive definite, it is easy to implement a direct banded method - because of poor condition, increase all pivots smaller than 13-8 times the largest diagonal element to this value before continuing the factorization to increase numerical stability. I would use the resulting factorization as the preconditioner of a conjugate gradient method, then you get the best of both worlds.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-20T17:12:46.363" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2291" PostTypeId="2" ParentId="2284" CreationDate="2012-05-20T20:04:43.250" Score="0" Body="&lt;p&gt;Best method would be MG preconditioned CG. I believe that is the ultimate solver for this case. If you know how the system is generated, i.e  the geometry is known, you can apply geometric multigrid. If not, algebraic multigrid should be used. Surely, for small problems - direct methods could be favored yet even in that case MG+PCG is really fast -if properly implemented, of course.&lt;/p&gt;&#xA;" OwnerUserId="1499" LastActivityDate="2012-05-20T20:04:43.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2292" PostTypeId="2" ParentId="2267" CreationDate="2012-05-20T20:14:02.503" Score="6" Body="&lt;p&gt;Actually, there is a wiki for openfoam: &lt;a href=&quot;http://openfoamwiki.net&quot;&gt;http://openfoamwiki.net&lt;/a&gt; It is a helpful site for openfoam in general yet the tutorials are not enough. Starting point should always be the manuals of openfoam. I started with the pdf's provided by the package around five years ago and I found them pretty handy. Real community of openfoam can be found in &lt;a href=&quot;http://www.cfd-online.com/Forums/openfoam/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1499" LastActivityDate="2012-05-20T20:14:02.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2293" PostTypeId="1" AcceptedAnswerId="2324" CreationDate="2012-05-21T05:30:47.843" Score="5" ViewCount="336" Body="&lt;p&gt;Summary: Are there good algorithms for out-of-core dense matrix transpose if each row of the matrix is separately compressed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Details: The matrix is about 1 TB uncompressed, and is roughly but not exactly square.  An uncompressed row is less than 6 MB, so many of them fit in RAM at once.  On disk, I would like to separately compress each row of the matrix (with a domain-specific, non-random access method), so I need a transpose algorithm that reads from compressed form, recompresses each column, and writes out the compressed columns as the new transposed matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are any existing out-of-core transpose algorithms compatible with this setup?&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="127" LastEditDate="2013-05-26T16:52:12.670" LastActivityDate="2013-05-26T16:52:12.670" Title="Out-of-core matrix transpose of row compressed data" Tags="|matrix|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2294" PostTypeId="1" AcceptedAnswerId="2302" CreationDate="2012-05-21T08:50:05.307" Score="6" ViewCount="576" Body="&lt;p&gt;Assume that we have generalized eigenvalue problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B^HB\textbf{x} = \lambda A\textbf{x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is an nxn Hermitian sparse matrix (n is very large, so we do not have $A^{-1}$ but can solve using iterative methods) and full-rank, and $B$ is a 2xn matrix such that $B^HB$ is also nxn but only rank 2. Thus, we know that this problem can only have 2 non-zero eigenvalues. Is there any simple way for finding the two eigenpairs corresponding to nonzero eigenvalues by taking advantage of the very low rank of $B^HB$? Assume that we have the two eigenvectors of $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am only interested in the eigenvector corresponding to the largest eigenvalue, is there a faster way of finding it than using simple power iteration on the transformed standard eigenvalue problem: $A^{-1}B^HB\textbf{x} = \lambda\textbf{x}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="1480" LastEditDate="2012-05-21T09:04:04.010" LastActivityDate="2012-05-22T02:07:06.453" Title="Large-scale generalized eigenvalue problem with low rank LHS matrix" Tags="|linear-algebra|eigensystem|sparse-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2295" PostTypeId="2" ParentId="2267" CreationDate="2012-05-21T09:55:55.700" Score="17" Body="&lt;p&gt;For running of  CFD simulations, I would suggest to start with the UserGuide and ProgrammersGuide. The ProgrammersGuide document features case examples as well, and it explains additional stuff such as boundary conditions. Using OpenFOAM on the top level is fairly easy, once you get a hang of the configuration files. Both documents are available in the &lt;em&gt;/doc&lt;/em&gt; directory. Once you are done with this, you can find a lot of additional material from the Chalmers university: &lt;a href=&quot;http://www.tfd.chalmers.se/~hani/kurser/OS_CFD/&quot;&gt;Chalmers OpenFOAM course&lt;/a&gt; organized by prof. Håkan Nilsson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you are done with this, dig through the &lt;a href=&quot;http://openfoamwiki.net/index.php/Main_Page&quot;&gt;Wiki&lt;/a&gt;. Concentrate on the task you want to accomplish: the library is &lt;em&gt;huge&lt;/em&gt; so there is no chance in getting to know it if you have a wide approach to learning how it operates. Work on specific problems, try to find documentation, engage in discussions on the forum and the extend project (see below). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in a specific problem, try to find out which solver is used to simulate this, e.g. for turbulent/laminar stationary solution, use &lt;em&gt;simpleFoam&lt;/em&gt;, for transiert laminar solution, use &lt;em&gt;icoFoam&lt;/em&gt;, etc. All of the solvers are saved in a directory structure that corresponds to their use like &lt;em&gt;/multiphase/incompressible/les/interFoam&lt;/em&gt; would be a vof-like solver used for two phase Large Eddy Simulations (les). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start with running the cases from the tutorial directories. Always make a copy of the simulation case, so you can have the original (if you use git, and make a mistake, checkout the directory to revert to its original state). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have made a successful instalation of OpenFOAM on your machine, execute &lt;em&gt;alias&lt;/em&gt; on the comnand line and see that there are some nifty commands that help you get started: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;tut&lt;/em&gt; brings you to the tutorial directories&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;run&lt;/em&gt; brings you to the run directory: this is where you run your simulations&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;sol&lt;/em&gt; brings you to the directories of the solvers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can ask questions on the &lt;a href=&quot;http://www.cfd-online.com/Forums/openfoam/&quot;&gt;OpenFOAM Forum&lt;/a&gt; when you run into a problem, and there is a directory of PhD and MSc theses and articles on OpenFOAM available at the FAMENA (Faculty of Mechanical Engineering, Uni. Zagreb, Croatia) &lt;a href=&quot;http://powerlab.fsb.hr/ped/kturbo/OpenFOAM/&quot;&gt;File Server&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.fsb.unizg.hr/?OpenFOAM_Summer_School_2012&quot;&gt;OpenFOAM Summer School&lt;/a&gt; is a really cool course organized by prof. Hrvoje Jasak at FAMENA, it takes place sometime in September each year, and it's a great opportunity to learn and get to know researchers working with OF. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there is a site organized by the OpenFOAM community: &lt;a href=&quot;http://www.extend-project.de/&quot;&gt;OpenFOAM extend project&lt;/a&gt;, where you can engage into discussion with people as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck! :) &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2012-12-13T08:22:30.300" LastActivityDate="2012-12-13T08:22:30.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2296" PostTypeId="1" AcceptedAnswerId="2297" CreationDate="2012-05-21T10:05:40.163" Score="11" ViewCount="1927" Body="&lt;p&gt;I have sets of multipoint field data, each point data set relates to a single cell of an unstructured mesh. The goal is to interpolate the data to the cell centre, directly or indirectly, in the most accurate way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use Inverse Distance Weighted interpolation, in the case when the distance between the source and the target (cell centre) is very small, I may end up with a floating point exception. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this kind of interpolation on a structured mesh, a volume weighted interpolation is used. This does not translate directly to an arbitrarily shaped mesh cell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introducing a tolerance for a IDW interpolation to circumvent the SIGFPE makes sense only if I do not introduce any tests that could render the interpolation inefficient. Is adding a sufficiently small $\delta$ to the denominator for every weight a possible option with the IDW interpolation? What interpolation methods suitable for this problem do you know? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional info: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the interpolation from the mesh to the points, I am using an interpolation based on the &lt;a href=&quot;http://en.wikipedia.org/wiki/Barycentric_coordinate_system_%28mathematics%29&quot;&gt;barcycentric coordinates&lt;/a&gt;. Each polyhedral cell of the mesh is decomposed into tetrahedra. Cell centred field is interpolated to the cell points using &lt;a href=&quot;http://en.wikipedia.org/wiki/Inverse_distance_weighting&quot;&gt;IDW interpolation&lt;/a&gt;. A search is conducted for each point to find the tetrahedron within which it lies, and the values are interpolaed using &lt;a href=&quot;http://en.wikipedia.org/wiki/Barycentric_coordinate_system_%28mathematics%29&quot;&gt;the barycentric interpolation&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the interpolation from the points to the mesh, this is not possible. The cell centred values are unknown. There is no way to assemble a tetrahedral composition that would enforce $\sum_p W_{PC} = 1$, where $W_{PC}$ is the weight related to a point &lt;em&gt;P&lt;/em&gt; and a cell centre &lt;em&gt;C&lt;/em&gt;. This comes from the fact that the point configuration is arbitrary. So, I am currently using IDW for this, making sure that I don't get a floating point exeption. Are there any better suited interpolation methods for this problem? &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2012-05-21T11:46:06.053" LastActivityDate="2012-05-31T14:36:51.210" Title="How to interpolate multipoint data to the cell centres of an unstructured mesh?" Tags="|interpolation|unstructured-mesh|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2297" PostTypeId="2" ParentId="2296" CreationDate="2012-05-21T16:47:51.877" Score="7" Body="&lt;p&gt;Links to diverse software packages for scattered data interpolation are on my web page &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/stat.html#fit&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/stat.html#fit&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book&lt;BR&gt;&#xA;G.E. Fasshauer, Meshfree Approximation Methods using MATLAB, &#xA;World Scientiﬁc 2007.&lt;BR&gt;&#xA;gives a comprehensive state of the art (as of 2006).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few more recent papers on scattered data interpolation:&lt;BR&gt;&#xA;&lt;a href=&quot;http://www.stanford.edu/group/uq/pdfs/journals/jcp_scattered_2010.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.stanford.edu/group/uq/pdfs/journals/jcp_scattered_2010.pdf&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://www.math.auckland.ac.nz/~waldron/Preprints/Box-splines/box-splines.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.auckland.ac.nz/~waldron/Preprints/Box-splines/box-splines.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which method to use depends a lot on the use made of the resulting interpolant.&#xA;Kriging methods are based on a stochastic model, hence are good if the data to be interpolated are somewhat noisy. Radial basis functions are to be preferred if (implemented stably) and a visually pleasing result is wanted (low curvature interpolation).&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-05-22T14:09:44.847" LastActivityDate="2012-05-22T14:09:44.847" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2298" PostTypeId="2" ParentId="2294" CreationDate="2012-05-21T17:43:32.097" Score="3" Body="&lt;p&gt;If $B$ is $2\times n$, then the only two non-trivial eigenvectors (i.e. the eigenvectors corresponding to the two non-zero eigenvalues) can be written as linear combinations of the vectors that form the two rows of $B$. Let's call these two vectors $b_1, b_2$ so that $B=\left[\begin{matrix}b_1^T\\b_2^T\end{matrix}\right]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let $P \in {\mathbb R}^{2\times n}$ be the projector from ${\mathbb R}^n$ onto the two-dimensional space spanned by $b_1,b_2$. Since we are only interested in vectors in this space, we know that the two non-trivial eigenvectors must satisfy $x = P^TPx$. The eigenvalue problem can then be written as &#xA;$$&#xA;  B^H B P^T P x = \lambda A P^T P x.&#xA;$$&#xA;Even though this linear system has $n$ rows, it is really only a two-dimensional problem since we can only determine only two components of $x$. The remainder of the linear system is over-determined, but we can select the two independent equations by projecting onto the non-trivial subspace:&#xA;$$&#xA;  P B^H B P^T P x = \lambda P A P^T P x.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, you only have to solve the $2 \times 2$ eigenvalue problem&#xA;$$&#xA;  P B^H B P^T y = \lambda (P A P^T) y.&#xA;$$&#xA;This is easy to solve since the matrices involved are only $2\times 2$ and the matrix on the right can easily be computed using just two matrix-vector and two vector-vector products.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-21T17:43:32.097" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2299" PostTypeId="2" ParentId="2293" CreationDate="2012-05-21T17:48:24.117" Score="0" Body="&lt;p&gt;I'd implement it using blocks of rows. If the matrix is $A$, write it as a sum $A=\sum_{b=1}^B A_b$ where each block $A_b$ consists of only some of the rows of $A$. You choose each block to be so that it fits into memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then $A^T=\sum_{b=1}^B A_b^T$: compute the transpose of $A_1$, store it to disk. Read $A_2$, compute its transpose and add the new columns you get from $A_2^T$ to the ones already computed for $A_1^T$, and so on and so forth. As long as you can store $A_b^T$ and the compressed form of the already treated transposes, it should work.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-21T17:48:24.117" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2300" PostTypeId="1" AcceptedAnswerId="2301" CreationDate="2012-05-21T18:26:59.547" Score="8" ViewCount="2586" Body="&lt;p&gt;what would be good methods (and/or software packages) to try for solving a problem minimizing a quadratic function $f(x) = \sum_{i=1}^N{(x_i - y_i)^2}$, s.t. $0 \leq x_i \leq 1$, and there are more constraints some of which are non-linear (and non-differentiable), e.g. $\sum_i x_i \mathbf{1}_{x_i&amp;gt;a} &amp;lt; b$ ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking about $N \approx 100$. &#xA;FWIW, Matlab is apparently using an &quot;active set method, similar to that of Gill et al.&quot;, which has somewhat uneven performance.&lt;/p&gt;&#xA;" OwnerUserId="1464" LastActivityDate="2012-06-26T14:00:26.930" Title="Minimizing a quadratic function with nonlinear constraints" Tags="|optimization|nonlinear-programming|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2301" PostTypeId="2" ParentId="2300" CreationDate="2012-05-21T19:12:47.787" Score="6" Body="&lt;p&gt;If you have nonsmooth constraints, it is no help that you have a convex quadratic objective. You need a constrained nonsmooth solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See my web page&lt;BR&gt; &#xA;    &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&quot;&gt;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&lt;/a&gt;&lt;BR&gt; &#xA;for suitable software.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-21T19:12:47.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2302" PostTypeId="2" ParentId="2294" CreationDate="2012-05-22T01:46:38.333" Score="5" Body="&lt;p&gt;This answer is essentially a fix of the approach suggested by @WolfgangBangerth, as there is not enough space in the comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting from&#xA;$$&#xA;B^H B x = \lambda A x,&#xA;$$&#xA;if we are interested in eigenpairs corresponding to nonzero eigenvalues, then we must have that $B^H B x$ lies in the range of $A$, and $Ax$ lies in the range of $B^H B$, which is to say that, since $A$ is invertible,&#xA;$$&#xA;B^H B x \in \mathrm{Range}(A) = \mathbf{C}^n,&#xA;$$&#xA;and&#xA;$$&#xA;Ax \in \mathrm{Range}(B^H B) = \mathrm{span}(B^H).&#xA;$$&#xA;Now, the first constraint is trivially satisfied, but we must ensure that $Ax \in \mathrm{span}(B^H)$, which is equivalent to the constraint&#xA;$$&#xA;x \in \mathrm{span}(A^{-1} B^H).&#xA;$$&#xA;Then if the columns of a unitary matrix $Q$ span the columns of $A^{-1}B^{H}$, we have that&#xA;$$&#xA;x = Q Q^H x&#xA;$$&#xA;for any eigenvector corresponding to a nonzero eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are now ready to use the mechanism from Wolfgang's approach: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute $W := A^{-1} B^H$ through two (preconditioned) Krylov solves&lt;/li&gt;&#xA;&lt;li&gt;Compute $[Q,R]=\mathrm{qr}(W)$&lt;/li&gt;&#xA;&lt;li&gt;Form $K := (B Q)^H (B Q)$ and $M := Q^H (A Q)$&lt;/li&gt;&#xA;&lt;li&gt;Solve the $2 \times 2$ eigenvalue problem $K U = M U \Lambda$&lt;/li&gt;&#xA;&lt;li&gt;Form the interesting global eigenvectors, $Z := Q U$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="67" LastEditorUserId="276" LastEditDate="2012-05-22T02:07:06.453" LastActivityDate="2012-05-22T02:07:06.453" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2303" PostTypeId="1" AcceptedAnswerId="2312" CreationDate="2012-05-22T08:06:11.980" Score="3" ViewCount="217" Body="&lt;p&gt;In PETSC, there are many xxxSetyyy methods, e.g. MatSetLocalToGlobalMapping(A,rmap,cmap).  I wonder whether contents of rmap and cmap (or generally yyy, that set to xxx) pointing to are copied to somewhere in A (or xxx).  In other words, shall I destroy 'yyy' after set to 'xxx' or I should not before finish using 'yyy' inside 'xxx'?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other examples include KSPSetPC, PCCompositeAddPC(PC pc,PCType type), ISCreateGeneral, PCASMSetLocalSubdomains. I found that for PCASMSetLocalSubdomains the IS defining the subdomains are &lt;strong&gt;copied&lt;/strong&gt; so we can destroy it afterwards. On the other hand, ISCreateGeneral forward a parameter to let user specify PetscCopyMode (Own_Pointer or Copy_Values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any general rule for these things?  Thanks!&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2012-05-22T08:13:55.037" LastActivityDate="2012-05-23T11:56:48.440" Title="Petsc's xxxSetxxx methods: Own Pointer or Copy Values?" Tags="|petsc|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2304" PostTypeId="1" AcceptedAnswerId="2305" CreationDate="2012-05-22T14:39:21.170" Score="6" ViewCount="1472" Body="&lt;p&gt;What is the best software for quickly implement and test a Reynolds Averaged Navier-Stokes turbulence model ?&lt;/p&gt;&#xA;" OwnerUserId="1383" LastEditorUserId="383" LastEditDate="2012-05-22T18:17:08.340" LastActivityDate="2015-10-07T11:37:44.213" Title="How to quickly implement and test a turbulence model?" Tags="|libraries|reference-request|fluid-dynamics|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2305" PostTypeId="2" ParentId="2304" CreationDate="2012-05-22T14:46:47.230" Score="5" Body="&lt;p&gt;OpenFOAM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's really easy to implement models there, even if you do not know how to program in C++, everything is set up, and you can just follow the pattern. If you do know something about similar class interfaces/dynamic polymorphism using virtual functions, then it is exceptionally easy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Model, explicit, or implicit, will involve manipulations on fields, and in OpenFOAM, field manipulation is really simple (Open Field Operaton And Manipulation), as you can see on e.g. page 21 of the &lt;a href=&quot;http://www.google.de/url?sa=t&amp;amp;rct=j&amp;amp;q=openfoam%20developing%20models%20.pdf&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CE4QFjAA&amp;amp;url=http://cfd.iut.ac.ir/files/Tutorial,%2520University%2520of%2520California,%2520San%2520Diego%2520.pdf&amp;amp;ei=v6W7T_OWMOj74QSHyeC1CQ&amp;amp;usg=AFQjCNHYUbVPL3sEek62L8oe7TWIhRJGxw&amp;amp;cad=rja&quot; rel=&quot;nofollow&quot;&gt;OpenFOAM programming slides&lt;/a&gt;. Generic programming and function overloading allows for top level coding when it comes to model development: your code looks exactly as the governing equations you are solving, or the calculation you are doing on the fields. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On this level, the resulting code is automatically parallel, with a choice between various types of parallel libraries (flavorus of MPI and recently OpenMP). Testing the model is easy as well, the model will have (if you have just followed the structure of other models, existing turbulence models, viscosity models, etc, the code structure is always the same) automatically enabled &lt;em&gt;run-time selectivity&lt;/em&gt;, which means that you can change parameters durign a run. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's from the top of my head.... there are other advantages: the library is modular, supports highly advanced topological changes on arbirary unstructured meshes (sliding interfaces, refinement, mixing planes, etc), has top notch numerical solvers... all in all: it's really cool. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additonal info: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can examine the hierarchy of of models &lt;a href=&quot;http://foam.sourceforge.net/docs/cpp/a02160.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, then find a model in the source code using the locate command : &lt;code&gt;locate turbulenceModel.H&lt;/code&gt; brings you to the source directory of the abstract base class (root class, basic type of the turbulence model), in my case it is (&lt;em&gt;src/turbulenceModels/incompressible/RAS&lt;/em&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, find the one that resembles your model, copy the code, rename the classes (if you are using a vim editor, try &lt;em&gt;%s/oldName/newName/g&lt;/em&gt; in all the source files), and compile it as a part of your new library. Et voila, a new turbulence model, parallel, shiny, run-time modifiable, etc. &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2012-05-22T17:46:05.837" LastActivityDate="2012-05-22T17:46:05.837" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2306" PostTypeId="1" CreationDate="2012-05-22T16:27:03.850" Score="3" ViewCount="1515" Body="&lt;p&gt;I am currently developing structural FEM solver in FORTRAN. My question is about Rigid Body Elements (Multi Point Constraints). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In NASTRAN there is RBE2 element defined by one independent and one or more dependent nodes. Given degrees of freedom of the independent nodes, defined by the user, are rigidly transferred to the dependent nodes. During the solution of the system of equations, degrees of freedom of dependent nodes are eliminated since they can be recovered from the defined Multi Point Constraint Equations. If there is a node which is in RBE2 definition defined as independent, and if there are no applied loads or constraints or finite element attached to that node, what degrees of freedom will be transferred to dependent node? Since there are no associated coefficients of stiffness matrix to these degrees of freedom, they can not be defined. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there first check of finite elements association to the independent node and if there is no association, or specific degree of freedom as Constraint, RBE2 will not be considered?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks, Vladimir  &lt;/p&gt;&#xA;" OwnerUserId="1471" LastEditorUserId="541" LastEditDate="2012-05-24T19:41:08.087" LastActivityDate="2012-05-24T19:41:08.087" Title="Rigid Body Elements" Tags="|finite-element|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2307" PostTypeId="2" ParentId="2249" CreationDate="2012-05-22T20:21:58.313" Score="12" Body="&lt;p&gt;I would like to suggest a somewhat different approach compared to the other answers, though @barron has indirectly discussed the same thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of optimizing your function directly, i.e. by evaluating it at a series of points $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_k$ points that (hopefully) converge to a (local) optimum, you could use the concept of $\textit{surrogate modelling}$, which is very well suited for problems of the type you describe (high cost, smooth, bounded, low dimensional i.e. less than 20 unknowns).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, surrogate modelling works by setting up a model function $c \in \mathbb{R}^d \rightarrow \mathbb{R}$ of your true function $f \in \mathbb{R}^d \rightarrow \mathbb{R}$. The key is that while $c$ of course does not perfectly represent $f$, it is far cheaper to evaluate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, a typical optimization process would be as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Evaluate $f$ at a set of $j$ initial points $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_j$. Note that derivatives are not needed. Also note that these points should be distributed evenly throughout the search space, e.g. by Latin Hypercube Sampling or a similar space-filling design.&lt;/li&gt;&#xA;&lt;li&gt;Based on this original dataset, create a model function $c$. You could use cross validation to validate your model (i.e. use only a subset of the original $j$ points to create $c$, and then use the remainder of the dataset to check how well $c$ predicts those values)&lt;/li&gt;&#xA;&lt;li&gt;Use a criterion such as the Expected Improvement (EI) criterion to find out where to ''fill in'' more samples to make $c$ more accurate by sampling $f$. This is actually far better studied theoretically than it might seem, and the EI criterion is very well researched. The EI criterion is also not a greedy criterion, so you both get good overall improvement of the model accuracy, whilst prioritizing accuracy near potential optima.&lt;/li&gt;&#xA;&lt;li&gt;If your model is not accurate enough, repeat step 3, else use your favourite optimization routine to find the optimum of $c$, which will be very cheap to evaluate (so you could use any routine you want, even ones that requires derivatives, or simply evaluate the function in a fine mesh). &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In general, this is what is meant by EGO, Efficient Global Optimization, as @barron suggested. I would like to stress that for your application, this seems perfectly suitable&amp;nbsp;— you get a surprisingly accurate model based on relatively few evaluations of $f$, and can then use any optimization algorithm you want. What's often also interesting is that you can now evaluate $c$ on a mesh and plot it, thereby gaining insight into the general appearance of $f$. Another interesting point is that most surrogate modelling techniques also provide statistical error estimates, thereby allowing uncertainty estimation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to construct $c$ is of course an open question, but often Kriging or so-called space-mapping models are used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this is all quite a bit of coding work, but a lot of other people have done very good implementations. In Matlab, I only know of the &lt;a href=&quot;http://www2.imm.dtu.dk/projects/dace/&quot; rel=&quot;nofollow&quot;&gt;DACE software toolbox&lt;/a&gt; DACE is free. TOMLAB might also offer a Matlab package, but costs money&amp;nbsp;— however, I believe it also works in C++ and has far more capabilities than DACE will ever have. (Note: I am one of the developers of the new version of DACE, soon to be released, which will offer added support for EGO.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that this rough overview has helped you, please ask questions if there are points that can be made more clear or stuff I've missed, or if you would like further material on the subject.&lt;/p&gt;&#xA;" OwnerUserId="370" LastEditorUserId="370" LastEditDate="2016-08-06T21:49:01.910" LastActivityDate="2016-08-06T21:49:01.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2308" PostTypeId="2" ParentId="1179" CreationDate="2012-05-22T21:34:29.013" Score="1" Body="&lt;p&gt;There is also an interesting program called &lt;a href=&quot;http://fold.it/portal/&quot; rel=&quot;nofollow&quot;&gt;foldit&lt;/a&gt; which looks like a puzzle game with a nice GUI but in the background it actually studies protein folding.&lt;/p&gt;&#xA;" OwnerUserId="1499" LastActivityDate="2012-05-22T21:34:29.013" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2309" PostTypeId="1" CreationDate="2012-05-23T01:25:18.507" Score="11" ViewCount="2085" Body="&lt;p&gt;I am attempting to solve an equation of the type:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \left( -\tfrac{\partial^2}{\partial x^2} - f\left(x\right) \right) \psi(x) = \lambda \psi(x) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $f(x)$ has a simple pole at $0$, for the smallest $N$ eigenvalues and eigenvectors.  The boundary conditions are: $\psi(0) = 0$ and $\psi(R)=0$, and I'm only looking at the function over $(0,R]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I do a very simple, evenly spaced finite difference method, The smallest eigenvalue is very inaccurate, (sometimes there is a &quot;false&quot; eigenvalue that is several orders of magnitude more negative than the one I know should be there, the real &quot;first eigenvalue&quot; becomes the second, but is still poor).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What affects the accuracy of such a finite difference scheme?  I assume that the singularity is what is causing the problem, and that an unevenly spaced grid would improve things significantly, are there any papers that can point me towards a good non-uniform finite difference method?  But perhaps a higher order difference scheme would improve it more?  How do you decide (or is it just &quot;try both and see&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;note:  my finite difference scheme is symmetric tridiagonal where the 3 diagonals are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( -\frac{1}{2 \Delta^2}, \; \frac{1}{\Delta^2} - f(x), \; -\frac{1}{2 \Delta^2} \right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\Delta$ is the grid spacing.  And I am solving the matrix using a direct symmetric solver (I am assuming that the accuracy is not affected drastically by the solver, am I wrong?)&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2013-05-29T15:53:59.623" LastActivityDate="2013-05-29T15:53:59.623" Title="How do you improve the accuracy of a finite difference method for finding the eigensystem of a singular linear ODE" Tags="|finite-difference|ode|accuracy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2310" PostTypeId="2" ParentId="2309" CreationDate="2012-05-23T09:14:23.973" Score="6" Body="&lt;p&gt;If you want to increase the accuracy of a finite difference scheme, you can always try increasing the degree of your stencil. On equidistant points, though, this can lead to numerical instabilities. To avoid these problems and still get high accuracy, I would suggest using &lt;a href=&quot;http://en.wikipedia.org/wiki/Spectral_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spectral Methods&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem has fixed poles, you can try to get around them by splitting your domain and solving two coupled problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chebfun&lt;/a&gt; system (disclaimer: of which I am one of the developers), uses the above mentioned techniques and you could give your problem a quick spin in the &lt;code&gt;chebgui&lt;/code&gt; interface. I'd try it myself, but I don't know what your domain or $f(x)$ is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've attached a screenshot of the &lt;code&gt;chebgui&lt;/code&gt; window for computing the first six eigenvalues and eigenmodes of $-u''(x) - u(x)x = \lambda u$ on $[-1,1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZDdPn.png&quot; alt=&quot;Using &amp;lt;code&amp;gt;chebgui&amp;lt;/code&amp;gt; to compute the eigenvalues and eigenmodes of a simple second-order differential equation.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to solve this problem without getting too much into Chebfun, all the details should be in Chapter 9 of Nick Trefethen's book &quot;&lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/spectral.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spectral Methods in Matlab&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-05-23T16:00:14.013" LastActivityDate="2012-05-23T16:00:14.013" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2311" PostTypeId="2" ParentId="2303" CreationDate="2012-05-23T11:22:49.353" Score="4" Body="&lt;p&gt;In general, the &lt;code&gt;Set&lt;/code&gt; methods (and PETSc) pass-by-reference, so it is not safe to destroy the object that you have passed to the method.  Methods such as &lt;code&gt;PCASMSetLocalSubdomains&lt;/code&gt; are the exceptions, not the rules.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-05-23T11:22:49.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2312" PostTypeId="2" ParentId="2303" CreationDate="2012-05-23T11:56:48.440" Score="5" Body="&lt;p&gt;When there is a &lt;strong&gt;&lt;em&gt;PetscCopyMode&lt;/em&gt;&lt;/strong&gt; parameter, the behavior is explicit. We would like to be explicit in every instance, but that would become very cumbersome in the interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Create&lt;/em&gt;&lt;/strong&gt;() gives back a new reference, whereas &lt;strong&gt;&lt;em&gt;Get&lt;/em&gt;&lt;/strong&gt;() returns a borrowed reference and should be given back using &lt;strong&gt;&lt;em&gt;Restore&lt;/em&gt;&lt;/strong&gt;().&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exceptions come up in &lt;strong&gt;&lt;em&gt;Set&lt;/em&gt;&lt;/strong&gt;() methods. They should all reference objects, and most do such as KSPSetPC(), however since this change was made relatively recently (and sometimes does not make as much sense), there are a few exceptions left. They will all eventually be documented/fixed. Please report any inconsistencies.The &lt;strong&gt;&lt;em&gt;Get&lt;/em&gt;&lt;/strong&gt;() methods generally return a borrowed reference since it is so cumbersome to destroy the object each time.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2012-05-23T11:56:48.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2313" PostTypeId="1" AcceptedAnswerId="2316" CreationDate="2012-05-23T12:21:02.553" Score="16" ViewCount="4959" Body="&lt;p&gt;I am doing a text classification task with R, and I obtain a document-term matrix with size 22490 by 120,000 (only 4 million non-zero entries, less than 1% entries). Now I want to reduce the dimensionality by utilizing PCA (Principal Component Analysis). Unfortunately, R cannot handle this huge matrix, so I store this sparse matrix in a file in the &quot;Matrix Market Format&quot;, hoping to use some other techniques to do PCA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So could anyone give me some hints for useful libraries (whatever the programming language), which could do PCA with this large-scale matrix with ease, or do a longhand PCA by myself, in other words, &lt;strong&gt;&lt;em&gt;calculate the covariance matrix at first, and then calculate the eigenvalues and eigenvectors for the covariance matrix&lt;/em&gt;&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want is to &lt;strong&gt;&lt;em&gt;calculate all PCs (120,000), and choose only the top N PCs, who accounts for 90% variance&lt;/em&gt;&lt;/strong&gt;. Obviously, in this case, I have to give a threshold a priori to set some very tiny variance values to 0 (in the covariance matrix), otherwise, the covariance matrix will not be sparse and its size would be 120,000 by 120,000, which is impossible to handle with one single machine. Also, the loadings (eigenvectors) will be extremely large, and should be stored in sparse format. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks very much for any help !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am using a machine with 24GB RAM and 8 cpu cores. &lt;/p&gt;&#xA;" OwnerUserId="1518" OwnerDisplayName="Ensom Hodder" LastEditorUserId="1518" LastEditDate="2012-05-23T15:35:08.733" LastActivityDate="2014-09-18T23:57:42.630" Title="Apply PCA on very large sparse matrix" Tags="|machine-learning|" AnswerCount="5" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2314" PostTypeId="2" ParentId="2313" CreationDate="2012-05-23T12:58:02.210" Score="8" Body="&lt;p&gt;I suggest using &lt;a href=&quot;http://www.grycap.upv.es/slepc/&quot;&gt;SLEPc&lt;/a&gt; to compute a partial SVD. See Chapter 4 of the &lt;a href=&quot;http://www.grycap.upv.es/slepc/documentation/slepc.pdf&quot;&gt;User's Manual&lt;/a&gt; and the &lt;a href=&quot;http://www.grycap.upv.es/slepc/documentation/current/docs/manualpages/SVD/index.html&quot;&gt;SVD man pages&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-23T12:58:02.210" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2315" PostTypeId="2" ParentId="2313" CreationDate="2012-05-23T13:22:46.053" Score="2" Body="&lt;p&gt;I vote for &lt;a href=&quot;https://cwiki.apache.org/confluence/display/MAHOUT/Algorithms&quot; rel=&quot;nofollow&quot;&gt;mahout&lt;/a&gt; which is also good for other NLP/TA tasks and implements map/reduce.&lt;/p&gt;&#xA;" OwnerUserId="1517" LastActivityDate="2012-05-23T13:22:46.053" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2316" PostTypeId="2" ParentId="2313" CreationDate="2012-05-23T16:10:56.910" Score="4" Body="&lt;p&gt;I suggest the irlba package - it produces virtually the same results as svd, yet you can define a smaller number of singular values to solve for. An example, using sparse matrices to solve the Netflix prize, can be found here: &lt;a href=&quot;http://bigcomputing.blogspot.de/2011/05/bryan-lewiss-vignette-on-irlba-for-svd.html&quot; rel=&quot;nofollow&quot;&gt;http://bigcomputing.blogspot.de/2011/05/bryan-lewiss-vignette-on-irlba-for-svd.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1519" LastActivityDate="2012-05-23T16:10:56.910" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2317" PostTypeId="2" ParentId="2304" CreationDate="2012-05-23T17:00:40.960" Score="2" Body="&lt;p&gt;&lt;strong&gt;Which code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would propose something different than previous post...&#xA;I suggest you take some lightweight code like 'CAFFA' from Ferziger and Peric CFD book, available in 2dgt folder at &lt;a href=&quot;ftp://ftp.springer.de/pub/technik/peric/&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; site. Everything is laid out in less then 2000 lines in front of you, no need to know anything about OOP. You will get more confidence because you know every line of the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What does it mean to implement a new turbulence model?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having this source code, you will learn that in case of most RANS models, implementing a new turbulence model means doing following few things in subroutines dedicated to solving transport equation of a scalar:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;unsteady term - do nothing&lt;/li&gt;&#xA;&lt;li&gt;convection term - do nothing&lt;/li&gt;&#xA;&lt;li&gt;diffusion - change diffusion coefficients ($\sigma$'s in $k-\epsilon$)&lt;/li&gt;&#xA;&lt;li&gt;volume source terms (production, destruction)- you might have to change them. This is where coefficients of your new turbulence model come into play as well. You calculate them (source terms) explicitly using known values and mid-point rule for integration. It's really easy.&lt;/li&gt;&#xA;&lt;li&gt;eddy viscosity - go to routine where it's calculated, in worst case you will have to change one line of code to get new form of equation for it.&lt;/li&gt;&#xA;&lt;li&gt;boundary conditions - you usually change them too, except in a case when you're solving same transport equation as the one you already have implemented.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That's about it if you are working with RANS models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Testing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Set up a channel case of $Re_\tau = 395$ for which there is available DNS database, and try to get matching in logarithmic velocity profile. Even without DNS there is well-known &quot;Law of the wall&quot; to help you. Every well tuned (in terms of coefficient values) turbulence model should reproduce it correctly. This will be validation or the answer to question are you solving the equations right.&#xA;Then setup a test case intended for turbulence model verification, like some from ERCOFTAC database. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="1050" LastEditDate="2012-05-23T20:08:52.737" LastActivityDate="2012-05-23T20:08:52.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2318" PostTypeId="5" CreationDate="2012-05-23T19:59:35.747" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-05-23T19:59:35.747" LastActivityDate="2012-05-23T19:59:35.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2319" PostTypeId="4" CreationDate="2012-05-23T19:59:35.747" Score="0" Body="Questions relating to graphing and displaying numerical data" OwnerUserId="1193" LastEditorUserId="1193" LastEditDate="2012-05-23T20:31:05.827" LastActivityDate="2012-05-23T20:31:05.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2320" PostTypeId="2" ParentId="2309" CreationDate="2012-05-23T20:18:45.360" Score="0" Body="&lt;p&gt;One way to quickly make things better (though likely not &lt;em&gt;much&lt;/em&gt; better) is to consider the similarity between the lowest order finite difference methods you use and the lowest order finite element method. If you compute the tri-diagonal matrix you get from using linear finite element shape functions in 1d, then the discretization of the second derivatives will look exactly the same (up to a factor $\Delta x$ but you'll get a different term for what comes off $f(x)\psi(x)$. I don't know how $f(x)$ looks in your case, but where right now you use $f(x_i)$, it will instead be something like $\int_{x_{i-1}}^{x_{i+1}} f(x) \varphi_i(x)$ where $\varphi_i(x)$ is the hat function that peaks at $x_i$. If $f(x)$ is sufficiently simple, then you can compute this integral exactly, and it will provide a more accurate matrix of which you have to find the eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you already do finite elements, you might as well invest in using higher order elements which aren't that much more difficult in 1d either.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-23T20:18:45.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2321" PostTypeId="5" CreationDate="2012-05-23T20:37:29.240" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-05-23T20:37:29.240" LastActivityDate="2012-05-23T20:37:29.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2322" PostTypeId="4" CreationDate="2012-05-23T20:37:29.240" Score="0" Body="A high-dimensionality space is one that can only be spanned by a basis set with a large number of elements. High-dimensional problems often suffer from the *Curse of Dimensionality*, which is exponential growth in the problem size as a function of the number of dimensions." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2013-01-04T00:09:04.837" LastActivityDate="2013-01-04T00:09:04.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2323" PostTypeId="1" AcceptedAnswerId="2326" CreationDate="2012-05-23T21:43:35.397" Score="5" ViewCount="505" Body="&lt;p&gt;suppose solving sequential generalized eigenvalue problems&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_i x= \lambda Bx, i=1,2,3,\ldots $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general setting, we always need to perform LU for matrix B&#xA;(preconditioned) before to apply the rest iterative algorithm. Is there a&#xA;numerical library(I have programming experiences with PETSc+SLEPc) or a toolkit that can allow me to separate those two parts, thus to perform LU only once? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default, LU factorization of $B$ is by direct solver, whose costs may be somewhat comparable, I suppose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: thanks to Arnold, but I want to modify my problem a little, where $B$ has a null vector s.t.&#xA;$B\mathbf{1}=\mathbf{0},\quad \mathrm{rank}(B) = n-1$&#xA;where $A_i,B$ are both $n\times n$ sparse symmetric matrix&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2012-05-24T13:31:33.333" LastActivityDate="2012-05-27T09:55:27.940" Title="solving generalized eigenvalue problems with the same precondition" Tags="|software|eigensystem|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2324" PostTypeId="2" ParentId="2293" CreationDate="2012-05-24T00:24:16.107" Score="4" Body="&lt;p&gt;If you do not need fast random access to single rows, you can store tiles instead of storing rows. For example, generate $500$ rows at a time and store them in $500\times 500$ tiles (match to a disk block size for best possible performance). These tiles are big enough that reading them off the disk in a different order (e.g. by columns) will still perform well.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-24T00:24:16.107" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2325" PostTypeId="2" ParentId="1671" CreationDate="2012-05-24T12:04:53.403" Score="7" Body="&lt;p&gt;I got good results using &lt;a href=&quot;http://www.hdfgroup.org/HDF5/&quot;&gt;HDF5&lt;/a&gt; and its GZIP filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The HDF5 also provides an &lt;a href=&quot;http://www.hdfgroup.org/doc_resource/SZIP/&quot;&gt;SZIP&lt;/a&gt; filter &#xA;which achieves better results for some scientifica data-sets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience the choice of compressions depends heavily on the kind of data and benchmarking is probably the only way to make a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, &lt;a href=&quot;http://www.hdfgroup.org/services/contributions.html&quot;&gt;third-party filters&lt;/a&gt; for HDF5 include BLOSC, BZIP2, LZO, LZF, MAFISC.&lt;/p&gt;&#xA;" OwnerUserId="1526" LastEditorUserId="1526" LastEditDate="2012-05-25T07:28:29.067" LastActivityDate="2012-05-25T07:28:29.067" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2326" PostTypeId="2" ParentId="2323" CreationDate="2012-05-24T12:25:48.383" Score="5" Body="&lt;p&gt;Factor $PB=LU$ yourself and write a routine for evaluating $L^{-1}PAU^{-1}x$ given $x$ (using two backsolves). Then you can solve the problem $(L^{-1}PAU^{-1}-\lambda I)z=0$ with a standard iterative solver for the ordinary eigenvalue problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $B$ is singular, compute a left null space basis consisting of the rows of $M$, and a right null space basis consisting of the columns of $N$, so that $MB=0$ and $BN=0$. Then you can replace the eigenvalue problem by the modified problem with the matrices $A'=\pmatrix{A &amp;amp; sAN\\MA &amp;amp; sC}$ and $B'=\pmatrix{B &amp;amp; AN\\MA &amp;amp; C}$, with $C$ and $s$ arbitrary. If $x$ solves the original eigenvalue problem then $x'=\pmatrix{x\\0}$ is an eigenvector of the new problem with the same eigenvalue. Now the kernel of the matrix $\pmatrix{B\\MA}$ is trivial since otherwise the eigenvalue problem is ill-posed. This implies that $B'$ is a nonsingular matrix. Thus one can apply the preceding to the modified problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The new eigenvalue problem also has the eigenvalue $s$, attained for all vectors of the form $x'=\pmatrix{0\\z}$. Therefore one should choose $s$ such that it lies somewhere in the middle of the  expected spectrum.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-05-27T09:55:27.940" LastActivityDate="2012-05-27T09:55:27.940" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2327" PostTypeId="2" ParentId="2306" CreationDate="2012-05-24T14:08:29.847" Score="1" Body="&lt;p&gt;I'm not sure I understand exactly what is being asked here. Different FEM packages use different methods to enforce constraints, e.g. Lagrange multipliers versus condensation at assembly time. If you have an element contribution involving slave dofs, you can condense (transform) them to be in terms of the master dofs before adding the element matrix to a load vector or sparse matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feel free to update the question with more details.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-24T14:08:29.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2328" PostTypeId="1" AcceptedAnswerId="2332" CreationDate="2012-05-24T17:25:02.053" Score="17" ViewCount="1877" Body="&lt;p&gt;I've spent the last couple of months on coding a Fortran program for solving a particular PDE system (describes fluid flow/combustion). I tryed to use latest-standard Fortran and the new OOP capabilites modern Fortran has. I am working on my own and have no Fortran guru beside me to ask questions, so a nataral way to learn for me would be to look at other libraries/solvers that use modern Fortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately all the Fortran libraries out there seem to be written in pretty old Fortran, Fortran90 tops. Therefore I had to think through the class-design and interaction myself. And I am not at all certian that I did it right, especially if one looks from a preformance perspective. &lt;strong&gt;But perhaps I have missed something and there are modern scientific packages written if Fortran and using OOP?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of good C++ libraries to learn from (OpenFOAM, deal.II and more) and also Python libraries. Those languages have a bigger community in general as well. Is it perhaps better to drop Fortran and switch language if I want to learn by example?&lt;/p&gt;&#xA;" OwnerUserId="1088" LastActivityDate="2020-11-12T07:20:34.003" Title="Are there open-source scientific libraries which use modern Fortran with OOP?" Tags="|fortran|" AnswerCount="6" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2329" PostTypeId="2" ParentId="2328" CreationDate="2012-05-24T18:09:11.480" Score="4" Body="&lt;p&gt;I agree with the comment by @JackPoulson. I am not aware of any open source libraries written in modern Fortran. So the answer to your specific question about needing to switch to C++ to learn by example is an unfortunate &quot;yes.&quot; But, to be clear, I do know that at least one library exists that was written almost exclusively in modern Fortran with an eye towards OOP, so it is possible (and worthwhile). This library is not open source, though, so it will not help you achieve your learning goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you are also likely to be reluctant to throw away what you have already completed. If you are still interested in pursuing Fortran for this and have specific development questions, I'd encourage you to ask them on StackOverflow. There are a few highly skilled Fortran developers active there that are more than happy to help.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2012-05-24T18:09:11.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2330" PostTypeId="2" ParentId="2328" CreationDate="2012-05-24T18:47:16.157" Score="7" Body="&lt;p&gt;If you want to learn by example I would recommend &lt;a href=&quot;http://rosettacode.org/&quot;&gt;rosettacode&lt;/a&gt;, a programming chrestomathy site. There you can find many example problems comparatively written in quite a few different languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe even more interesting for you would be &lt;a href=&quot;http://fortranwiki.org&quot;&gt;Fortran wiki&lt;/a&gt;, and &lt;a href=&quot;http://fortranwiki.org/fortran/show/Object-oriented+programming&quot;&gt;here&lt;/a&gt; is a section on OOP in Fortran which might be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortan is doing just fine - let's not give up on it! &lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="541" LastEditDate="2012-05-24T22:12:20.473" LastActivityDate="2012-05-24T22:12:20.473" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2332" PostTypeId="2" ParentId="2328" CreationDate="2012-05-24T21:40:37.223" Score="14" Body="&lt;p&gt;There are modern CFD codes which you can look into. For example ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://amcg.ese.ic.ac.uk/index.php?title=Fluidity&quot;&gt;Fluidity&lt;/a&gt;: General purpose multiphase CFD (FE) code; Even does fully unstructured AMR&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mmm.ucar.edu/wrf/users/download/get_source2.html&quot;&gt;WRF&lt;/a&gt;: Next-generation numerical model weather prediction system from NCAR&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://research.edf.com/research-and-the-scientific-community/software/code-saturne/introduction-code-saturne-80058.html&quot;&gt;Code Saturne&lt;/a&gt;: General purpose CFD (FV) code; Some features listed on &lt;a href=&quot;http://en.wikipedia.org/wiki/Code_Saturne&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are similar modern Fortran codes for structural analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But they may not use full F2003 capabilities due to lack of widespread compiler support. Most new compilers do support large parts of F2003 (see the ACM SIGPLAN Fortran Forum papers) but it takes time to have a wide user base. Debian stable for example has GCC 4.4.5 so generally speaking you should not expect your users to have anything better (i.e., 4.5/4.6/4.7).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When F2003 compilers become widespread you will see libs/code that use the F2003 capabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now you can look at this &lt;a href=&quot;http://books.google.com/books/about/Scientific_Software_Design.html?id=xedE5KkHVn4C&quot;&gt;book&lt;/a&gt; by Damian Rouson on how to use the OOP capabilities.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-05-27T14:22:40.447" LastActivityDate="2012-05-27T14:22:40.447" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2333" PostTypeId="1" CreationDate="2012-05-25T02:55:59.847" Score="5" ViewCount="13149" Body="&lt;p&gt;For the purpose of model fitting in a large time series dataset, I am using stochastic gradient descent of the negative log likelihood. The model is nonlinear and non-convex. Is there a thumb rule for choosing a good step size? I could choose a very small step size for stable but painfully slow convergence, but I would like to be able to choose a big enough step size for faster convergence and anneal it.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2013-03-22T14:04:48.943" Title="How to choose a good step size for stochastic gradient descent?" Tags="|optimization|stochastic|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2334" PostTypeId="2" ParentId="2333" CreationDate="2012-05-25T03:45:30.873" Score="1" Body="&lt;p&gt;Have you considered a &lt;a href=&quot;http://en.wikipedia.org/wiki/Line_search&quot; rel=&quot;nofollow&quot;&gt;line search&lt;/a&gt;?  Typically, convergence proofs and anecdotes of convergence for optimization problems both rely heavily on having an appropriate line search in place.&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2012-05-25T03:45:30.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2335" PostTypeId="2" ParentId="2333" CreationDate="2012-05-25T06:57:41.437" Score="3" Body="&lt;p&gt;Depending on your specific system and the size, you could try a line search method as suggested in the other answer such as Conjugate Gradients to determine step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if your data size is really large, this might become very inefficient and time consuming. For large datasets people often choose a fixed step size and stop after a certain number of iterations and/or decrease the step size by a certain percentage after each pass through the data so that you can effectively take big &quot;jumps&quot; when you are first starting out and slow down once you are getting closer to your solution. You can determine the step size by using a validation method such as cross-validation and choose a step-size which minimizes your cross-validation error. If your training set is huge and your model (number of free parameters) is not terribly complicated, then a step size which works well for the in-sample will likely work well for out-of-sample (test data set) as well. Even so, regularization may be important for achieving a good generalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Netflix competition is a great example of a huge dataset (480,189 users and 17,770 movies, and several million ratings for the training set) where stochastic gradient descent was used as the workhorse optimization algorithm for training most of the prediction models used. A good paper to read regarding an algorithm used in the Netflix competition is &lt;a href=&quot;http://public.research.att.com/~volinsky/netflix/kdd08koren.pdf&quot; rel=&quot;nofollow&quot;&gt;Factorization Meets the Neighborhood&lt;/a&gt; by one of the winners of the competition, Y. Koren. For training their models, they typically used a fixed learning rate, and empirically a learning rate $\eta = 0.001$ seemed to work well for the Netflix problem. This is highly application specific, however!&lt;/p&gt;&#xA;" OwnerUserId="1480" LastActivityDate="2012-05-25T06:57:41.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2336" PostTypeId="1" AcceptedAnswerId="2340" CreationDate="2012-05-25T10:38:14.110" Score="4" ViewCount="1529" Body="&lt;p&gt;I am trying to rewrite some MATLAB/Octave code in Python, and I don't know what would be the nicest or most intuitive way of writing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;octave:10&amp;gt; dt = 0.1;&#xA;octave:12&amp;gt; T = 0:dt:1&#xA;T =&#xA;&#xA;    0.00000    0.10000    0.20000    0.30000    0.40000    0.50000    0.60000    0.70000    0.80000    0.90000    1.00000&#xA;octave:15&amp;gt; dt = 0.17;&#xA;octave:16&amp;gt; T = 0:dt:1&#xA;T =&#xA;&#xA;    0.00000    0.17000    0.34000    0.51000    0.68000    0.85000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which creates a discretization of the interval [0, 1] with step 0.1, as it's seen. I referred to the NumPy/MATLAB mathesaurus and it uses &lt;code&gt;arange&lt;/code&gt; function, but it's not suitable for non-integer values as it's stated in &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.arange.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation&lt;/a&gt; and shown in &lt;a href=&quot;https://stackoverflow.com/q/10011302/554319&quot;&gt;this SO question&lt;/a&gt;. On the other hand, playing with &lt;code&gt;linspace&lt;/code&gt; is not appealing to me because it takes care of endpoints, not spacing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which would be a straight-forward, one-line way of doing this in Python?&lt;/p&gt;&#xA;" OwnerUserId="782" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-04-15T10:22:08.197" Title="Python syntax for MATLAB/Octave colon operator a:dx:b" Tags="|matlab|discretization|octave|numpy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2338" PostTypeId="1" CreationDate="2012-05-25T13:10:44.593" Score="4" ViewCount="286" Body="&lt;p&gt;I have a question about the memory usage within the graphics card. What does happen when the used application needs more memory than the graphic card does contain? let`s say the reserved main memory is also not enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scenario I am talking about is something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a graphics card with, let's say, 50 MB memory usable for texture information. The reserved main memory space for texture data is also around 50 MB. The application I want to run contains five data sets of each 25 MB that are needed at the same time, so you have a total of 100 MB memory, but 125 MB is needed.&lt;/p&gt;&#xA;" OwnerUserId="1535" LastEditorUserId="276" LastEditDate="2012-05-25T21:05:57.750" LastActivityDate="2012-05-25T21:59:27.637" Title="Memory usage within graphics cards" Tags="|data-management|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2340" PostTypeId="2" ParentId="2336" CreationDate="2012-05-25T14:36:44.167" Score="4" Body="&lt;p&gt;MATLAB and Octave are susceptible to the same subtle floating-point issues that Python is, where you can get a slightly unexpected result if you do not anticipate rounding issues.  On the other hand, this is quite convenient syntax to have!  You can easily &quot;bake&quot; a simple function to do this from the existing functionality in &lt;code&gt;numpy.linspace&lt;/code&gt;, which is a very Pythonic way to do things!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# use Python3 syntax in Python2 to get clean, consistent integer division  &#xA;from __future__ import division   &#xA;import numpy as np&#xA;&#xA;def lrange(r1, inc, r2):&#xA;    &quot;&quot;&quot;Provide spacing as an input and receive samples back by wrapping `numpy.linspace`&quot;&quot;&quot;&#xA;    n = ((r2-r1)+2*np.spacing(r2-r1))//inc&#xA;    return np.linspace(r1,r1+inc*n,n+1)&#xA;&#xA;&amp;gt; lrange(0,0.1,1)&#xA;array([ 0. ,  0.1,  0.2,  0.3,  0.4,  0.5,  0.6,  0.7,  0.8,  0.9,  1. ])&#xA;&#xA;&amp;gt; lrange(0,0.17,1)&#xA;array([ 0.  ,  0.17,  0.34,  0.51,  0.68,  0.85])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: I've added a floating point epsilon to the computation that will handle small floating-point errors in the operands/representation per &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2336/python-syntax-for-matlab-octave-adxb/2340#comment4457_2340&quot;&gt;Juanlu001's comment&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-06-08T13:53:38.013" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2341" PostTypeId="1" CreationDate="2012-05-25T14:52:13.557" Score="2" ViewCount="100" Body="&lt;p&gt;Has anybody stumbled upon any kind of application of high-frequency high-dimensional problems ($d\geq 4$)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My interest comes from the following: there is quite a decent amount of papers where people consider efficient representation of discretized integral operators in $d$ dimensions that allows to avoid a curse of dimensionality (tensor decomposition methods); namely, $d$-dimensional Laplace operator seems to be of an importance for quantum mechanics community. Is there any interest in working with $d$-dimensional Helmholtz equation, for example? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I heard as well that statistical applications require efficient representation of high-dimensional functions, but still, I have never seen anybody dealing with low-rank representations of multidimensional oscillatory functions unless it was a question of mainly theoretical interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="1493" LastActivityDate="2012-05-25T17:09:26.630" Title="application of oscillatory high-dimensional functions" Tags="|high-dimensional|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2343" PostTypeId="2" ParentId="2283" CreationDate="2012-05-25T17:03:58.210" Score="6" Body="&lt;p&gt;The documentation of f2py is really not that great. I'm in the process of moving my own projects from f2py to &lt;a href=&quot;http://cython.org/&quot;&gt;Cython&lt;/a&gt;. Even though Cython can be used to wrap C code, I found that it is most useful for creating new high performance code. If your intent is to write new Fortran code, wrap it, and call it in Python, I recommend that you use Cython instead. That way you can write Python code, and if there are performance bottlenecks, you transfer that over to Cython with minor modifications.&lt;/p&gt;&#xA;" OwnerUserId="1126" LastActivityDate="2012-05-25T17:03:58.210" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2344" PostTypeId="2" ParentId="2341" CreationDate="2012-05-25T17:09:26.630" Score="1" Body="&lt;p&gt;Semiclassical methods in quantum mechanics based on the path integral (Herman--Kluk formula) lead to highly oscillatory integrals. See, e.g., &lt;a href=&quot;http://arxiv.org/pdf/0908.0847v1.pdf&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/pdf/0908.0847v1.pdf&lt;/a&gt; ,&#xA;which looks at stationary path approximations to such integrals. For related papers, look into &lt;a href=&quot;http://scholar.google.com&quot; rel=&quot;nofollow&quot;&gt;http://scholar.google.com&lt;/a&gt; for citations of reference [19] in this paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I never saw low rank approximations to such integrals, and do not know whether these would be useful.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-25T17:09:26.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2345" PostTypeId="2" ParentId="2338" CreationDate="2012-05-25T17:45:49.927" Score="4" Body="&lt;p&gt;The most common case is that you cannot allocate a memory block larger than what is physically available for your GPU card. However, depending on your hardware and  driver, the maximum allocatable size might be a fraction of the total physically installed memory (e.g. old AMD opencl drivers only allowed a maximum of 500 MB), so it is better if you read into the specs of the vendor (NVIDIA, AMD, or Intel).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case that a GPU kernel of your implementation needs to access more of the &lt;em&gt;global&lt;/em&gt;, or &lt;em&gt;local&lt;/em&gt; / &lt;em&gt;shared&lt;/em&gt; memory than what one can allocate at once, then you have to manually manage the memory by allocating and freeing memory segments (smaller than maximum memory) and transfer them between host and GPU as needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as my experience goes, memory management is crucial for achieving high performance for GPU applications. The reason for this is that for most applications, transferring data through the PCIe between host and GPU can become a big bottleneck. To illustrate the PCIe problem, consider that the maximum bandwidth of PCIe 2.0 (x16 lanes) is 8 GB/s bidirectional, whereas the bandwidth from on board GPU memory to GPU chip is +140 GB/s. Moreover, all the tiny cores of a GPU can collectively process several terabytes of data per second, thus most high performance GPU kernels also need to make efficient use of shared / local and register memory (which is orders of magnitude faster than accessing global memory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative to explicit memory management, is to use extensions to opencl where a GPU kernel can directly access, from the discrete GPU chip, memory segments that reside on host (as fas as I know, AMD allows this). In such a case, the GPU would halt execution until data is received from host, these can be a very long time from the perspective of the GPU thread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more interesting scenario happens when the CPU and GPU are integrated in the same chip. In such a case, CPU and GPU share the host memory (consider the fusion architecture of AMD) so there should not be necessary to copy data between host and device.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-05-25T17:45:49.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2346" PostTypeId="2" ParentId="2338" CreationDate="2012-05-25T19:08:17.537" Score="1" Body="&lt;p&gt;On NVIDIA cards at least, it is possible to allocate pinned memory on the CPU for use by the graphics card. Pointers to this block of memory are in the GPU's global memory space (so they can be accessed by your kernels) but reads/writes to them must still go through the PCI-E bus. The drawback is that you cannot use too much of this memory or your OS will become very angry with you. Is this what you meant by &quot;reserved main memory&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also possible to handle compute/memory transfer at once, so if an application doesn't need all the data at once you can have the kernel process a slice of data while the next batch is transferring, do something with the result, and then continue processing and transferring. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: referring to your specific example, it may also be possible to store 50 kB in texture memory, 50 kB in global (I assume this is what you meant by main?), and then find somewhere else to squeeze the last 25kB in, like constant memory. You would only be able to read these last 25 kB, so this may not work in all cases. Another option would be to use the pinned memory, as I discussed earlier. The advantage of constant memory is that the data is actually on the device, but the read-only restriction may be too onerous for some applications.&lt;/p&gt;&#xA;" OwnerUserId="1193" LastEditorUserId="1193" LastEditDate="2012-05-25T21:59:27.637" LastActivityDate="2012-05-25T21:59:27.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2347" PostTypeId="1" AcceptedAnswerId="2349" CreationDate="2012-05-25T19:09:21.160" Score="8" ViewCount="1500" Body="&lt;p&gt;I learned that for BiCGStab solver for sparse linear systems it's pretty much always necessary to use a preconditioner. I realized by now that choosing a good one is problem dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Surfing the web I found out there are many ILU based preconditioners (ILUT, MILU,etc.) This is where I got confused. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if someone can briefly explain taxonomy of ILU preconditioners, give some literature sources for particular ones, and which of those are suited for BiCGStab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The setting in which I'm working is CFD, and code is based on unstructured finite volume discretization. Probably different preconditioners will be needed for momentum equation and transport equations for turbulence scalars.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-05-25T21:50:00.233" Title="Taxonomy of ILU preconditioners" Tags="|linear-solver|fluid-dynamics|sparse-matrix|preconditioning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2348" PostTypeId="1" AcceptedAnswerId="2354" CreationDate="2012-05-25T19:30:11.273" Score="10" ViewCount="1080" Body="&lt;p&gt;Does compiling PETSc with an external BLAS/LAPACK library significantly affect performance on sparse matrices, or does it only use those libraries for dense matrix math?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-05-25T23:25:01.147" Title="Does PETSc ever make use of LAPACK libraries for sparse matrix math?" Tags="|petsc|sparse-matrix|lapack|blas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2349" PostTypeId="2" ParentId="2347" CreationDate="2012-05-25T20:23:17.583" Score="11" Body="&lt;p&gt;A preconditioner, say M, is an approximation on the system matrix, say A that changes the problem into another problem with improved eigenvalue spectrum. A perfect preconditioner would be inverse of A i.e inv(M) = A. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this inverse is normally not avaiable, too complicated to compute, requires more space to store because of the fill-in's introduced during the factorization and also might suffer from round-off errors. Hence, a preconditioner should be easy to compute and apply yet effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides the basic preconditioner like Jacobi or Gauss-Seidel (or SOR), one of the most frequrently preconditioners is ILU (IC in symmetric problems - you are working on CFD so the systems or non-symmetric)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Selection of the preconditioner normally depends on your problem. ILU has many variants like ILUT,ILUS,MILU etc. You can consult the literature I added at then end. For problems with mild difficulty, ILU(0) can be used but as the problems get harder i.e. Reynolds number is increasing more fill-ins (eg. ILU(1)) with thresholding strategies (ILUT) should be used. The problem is, advanced uses of ILU requires more memory and the determination of the sparsity pattern which is now different than your system matrix. In this case, you have to compute the spartisity pattern symbolically first and numerically later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The are several ideas to reduce the computation work,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use of lagged preconditioners at which you avoid recomputing the preconditioner even if the linear system varies.&lt;/li&gt;&#xA;&lt;li&gt;use of LU-SGS like preconditioner which might be implemented efficiently with flux splitting algorithms.&lt;/li&gt;&#xA;&lt;li&gt;use of matrix-free methods - the one I preferred in my PhD work. Newton-Krylov solvers where the jacobian is not needed expect for preconditioner which is normally calculated with a low-order approximation and probably with color-based algorithms (more difficult in unstructured problems, though).&lt;/li&gt;&#xA;&lt;li&gt;use of only diffusion operators like laplacian and avoiding convective terms (not efficient in reducing number of iterations)&lt;/li&gt;&#xA;&lt;li&gt;multigrid, where you use a simple smoother like w-Jacobi or SOR in a grid hierarchy. for unstructed problems you should use Algebraic multigrid (AMG) instead of geometric.&lt;/li&gt;&#xA;&lt;li&gt;and many others (multiply the number of my entries with 10)..&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since the problem is non-symmetric, there are mainly two solvers to use: GMRES or BiCGStab. (QMR or TFQMR are other alternative but I believe its performance is below those two). GMRES is normally a better solver if you have no storage problem yet because of the stored vectors a restart is needed - this is a problem is the preconditioner is poor or you have very large dof. BiCGStab requires only four Matrix-Vector product which is nice for large problems but normally inferior to GMRES. (I preferred GMRES but I like BiCGStab very much!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this preconditioner, linear solver issue is way complex. I can suggest some books to read. Your starting point should be &lt;a href=&quot;http://www.netlib.org/linalg/html_templates/Templates.html&quot;&gt;Templates for the solution of linear systems&lt;/a&gt; This is a free book. Smoothers both as standalone solvers and preconditioners on Krylov solvers is explained in this book. You can also consult to Yousef Saad's &quot;Iterative Methods for Sparse Linear Systems&quot;. It is definitive in the library of your institution. First edition is also avaiable &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/books.html&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before concluding, I recommend you to look at frameworks like &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot;&gt;Petsc&lt;/a&gt;, &lt;a href=&quot;http://trilinos.sandia.gov/&quot;&gt;Trilinos&lt;/a&gt; or even &lt;a href=&quot;http://acts.nersc.gov/hypre/&quot;&gt;Hypre&lt;/a&gt; and also the files provided by &lt;a href=&quot;http://www.netlib.org/linalg/html_templates/Templates.html&quot;&gt;1&lt;/a&gt;. They provide preconditioners with some programming. There are actually more books to offer but also take a look in Ke Chen's &quot;Matrix Preconditioning Techniques and Applications&quot;. Matlab codes are avaiable with the book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck in your journey, you'll need it.&lt;/p&gt;&#xA;" OwnerUserId="1499" LastActivityDate="2012-05-25T20:23:17.583" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2350" PostTypeId="2" ParentId="21" CreationDate="2012-05-25T20:30:42.597" Score="8" Body="&lt;p&gt;Aesin already answered part of your question. I can provide you with some more information on GAMESS(US).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to make GAMESS(US) use the same 'type' of B3LYP as Gaussian 03. For this, you need to specify &quot;DFT=B3LYP1&quot; as you already mentioned in your question. This selects B3LYP with VWN formula 1 RPA local correlation which, to the best of my knowledge, is identical to what they call VWN formula III in some other programs (like Gaussian 03).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, choosing the same functional in both programs is not the only requirement to get identical results. Some other things to consider are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Basis set. Both programs have to use the exact same basis set. If you're using an internally stored basis set in Gaussian (like, for example, 6-31G(d,p)), you can make Gaussian print the basis set details by adding the keyword GFINPUT to the route section. GAMESS(US) prints basis set details in its main output.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Grid size. By default, Gaussian 03 uses a (75,302) grid while GAMESS(US) uses a (96,302) grid. In Gaussian, the grid size can be controlled by the INT keyword. In GAMESS(US), you should take a look at the NRAD and NLEB keywords in the $DFT group. The type of grid will also make a difference, but to the best of my knowledge, GAMESS(US) and Gaussian make use of similar grids.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Integral cutoff. Both programs neglect very small integrals, as this will speed up the calculation without having a significant impact on accuracy. However, the cutoff factors between the two programs can be different, which can lead to slightly different results. In Gaussian 03, you can control the cutoff factor with IOP(3/27). In GAMESS(US), you can use the ICUT keyword in $CONTRL.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SCF convergence. Gaussian normally uses EDIIS and CDIIS for the SCF procedure while GAMESS(US) uses DIIS or SOSCF. Both methods should converge towards the same solution, provided that your case is not too complex for DFT. However, you should specify very tight convergence criteria if you'd like to compare energies obtained with both programs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With regard to geometry optimizations: Gaussian and GAMESS(US) use very different coordinate systems, geometry optimizers, and convergence criteria. Making both programs optimize to the exact same geometry is difficult, maybe even impossible.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There might be other minor differences you've got to take into account. Maybe it is best to start with a Hartree-Fock calculation, and see if the two programs yield the same SCF energy -- this takes differences in functional and DFT grid out of the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="1537" LastEditorUserId="1537" LastEditDate="2012-05-25T21:05:24.843" LastActivityDate="2012-05-25T21:05:24.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2352" PostTypeId="2" ParentId="2347" CreationDate="2012-05-25T21:50:00.233" Score="3" Body="&lt;p&gt;The book by Y. Saad is also one of the standard references for solvers and preconditioners.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-25T21:50:00.233" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2353" PostTypeId="2" ParentId="2328" CreationDate="2012-05-25T21:59:31.487" Score="2" Body="&lt;p&gt;I don't know which Fortran dialect it's written in, but there is William Mitchell's PHAML code that is a general purpose Fortran FEM code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will admit that I'm not impartial in the discussion of Fortran vs C++ but I do like to point out that you difficulty in finding a modern Fortran-based FEM code while at the same time finding a whole lot of C++-based codes tells something of what the community as a whole thinks is the way to go. There is of course nothing wrong with trying to build your own code in a language you like and are familiar with, but your brief survey already shows you that you're building an island. You may want to take that into account.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-25T21:59:31.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2354" PostTypeId="2" ParentId="2348" CreationDate="2012-05-25T23:25:01.147" Score="10" Body="&lt;p&gt;PETSc uses BLAS for a few vector primitives, but these are generally limited by memory bandwidth and there isn't much variance in &quot;optimization&quot;, so it tends not to make much performance difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also uses Lapack for some analysis such as Lanczos or Arnoldi estimates of eigenvalues and singular values, but these are generally not performance-sensitive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dense &quot;level 3&quot; operations generally only appear in a performance-sensitive context when using sparse direct solvers from third-party libraries (e.g. MUMPS, SuperLU, UMFPACK), in which case fill eventually leads to dense problems that are large enough to benefit from calling BLAS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you rely on these sparse direct solvers applied to large problems, then it's worth building with a tuned BLAS implementation, otherwise it makes very little difference.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-25T23:25:01.147" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2355" PostTypeId="1" AcceptedAnswerId="2356" CreationDate="2012-05-26T11:50:25.750" Score="3" ViewCount="1110" Body="&lt;p&gt;When indexing PETSc.Mat &lt;code&gt;A&lt;/code&gt; with an array &lt;code&gt;c&lt;/code&gt; ( &lt;code&gt;numpy.ndarray&lt;/code&gt; with &lt;code&gt;dtype('int64')&lt;/code&gt;), I run into the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  File &quot;/Users/cls/workspace/LAMG-Python/src/lamg/amg/setup.py&quot;, line 945, in buildInternal&#xA;    Acc = A[c, c]&#xA;  File &quot;Mat.pyx&quot;, line 185, in petsc4py.PETSc.Mat.__getitem__ (src/petsc4py.PETSc.c:71407)&#xA;  File &quot;petscmat.pxi&quot;, line 862, in petsc4py.PETSc.mat_getitem (src/petsc4py.PETSc.c:22442)&#xA;  File &quot;petscmat.pxi&quot;, line 773, in petsc4py.PETSc.matgetvalues (src/petsc4py.PETSc.c:21226)&#xA;  File &quot;arraynpy.pxi&quot;, line 123, in petsc4py.PETSc.iarray_i (src/petsc4py.PETSc.c:5284)&#xA;  File &quot;arraynpy.pxi&quot;, line 117, in petsc4py.PETSc.iarray (src/petsc4py.PETSc.c:5192)&#xA;TypeError: Cannot cast array data from dtype('int64') to dtype('int32') according to the rule 'safe'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assume there is a simple solution (allowing the cast, making everything  32 or 64 bit...). Can you point me in the right direction? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="1360" LastEditorUserId="9" LastEditDate="2012-05-26T13:34:35.367" LastActivityDate="2012-05-26T13:34:35.367" Title="32bit/64bit issue when working with Numpy and petsc4py" Tags="|petsc|numpy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2356" PostTypeId="2" ParentId="2355" CreationDate="2012-05-26T13:15:54.367" Score="3" Body="&lt;p&gt;There are two solutions, depending on what you want.  As a point of information, PETSc &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/faq.html#with-64-bit-indices&quot; rel=&quot;nofollow noreferrer&quot;&gt;defaults to 32-bit indexing&lt;/a&gt; on normal builds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you &lt;strong&gt;always want&lt;/strong&gt; 64-bit indices, then you should configure and build PETSc with 64-bit indices (pass &lt;code&gt;--with-64-bit-indices&lt;/code&gt; to configure) and everything will work fine.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you would like to explicitly force a copy if needed every time you hand a data structure to PETSc, you can do this by  &lt;code&gt;c.astype(petsc4py.PETSc.IntType)&lt;/code&gt;.  See &lt;a href=&quot;https://stackoverflow.com/questions/4389517/in-place-type-conversion-of-a-numpy-array&quot;&gt;this question on StackOverflow for more on the intricacies of doing/avoiding a Numpy copy&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-05-26T13:15:54.367" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2357" PostTypeId="1" AcceptedAnswerId="2358" CreationDate="2012-05-27T01:19:58.020" Score="17" ViewCount="4025" Body="&lt;p&gt;The other day, my computational fluid dynamics instructor was absent and he sent in his PhD candidate to substitute for him.  In the lecture he gave, he seemed to indicate several disadvantages associated with various discretization schemes for fluid flow simulations:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Finite Difference Method:&lt;/strong&gt;  It is difficult to satisfy conservation and to apply for irregular geometries  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Finite Volume Method:&lt;/strong&gt;  It tends to be biased toward edges and one-dimensional physics.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Finite Element Method:&lt;/strong&gt;  It is difficult to solve hyperbolic equations using FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discontinuous Galerkin:&lt;/strong&gt;  It is the best (and worst) of all worlds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fluctuation Splitting:&lt;/strong&gt;  They are not yet widely applicable.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the lecture, I tried asking him where he got this information but he did not specify any source.  I also tried to get him to clarify what he meant by DG being the &quot;best and worst of all worlds&quot;, but couldn't get a clear answer.  I can only assume that he came to these conclusions from his own experience.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my own experience, I can only verify the first claim that FDM is difficult to apply to irregular geometries.  For all other claims, I don't have sufficient experience to verify them.  I'm curious how accurate these claimed 'disadvantages' are for CFD simulations in general.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-05-27T22:06:55.717" Title="Disadvantages of common discretization schemes for CFD simulations" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2358" PostTypeId="2" ParentId="2357" CreationDate="2012-05-27T04:22:29.603" Score="18" Body="&lt;p&gt;The proposed characteristics are reasonable in the sense that they roughly represent popular opinion. This question has massive scope, so I'll just make a few observations now. I can elaborate in response to comments. For more detailed related discussion, see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/290/what-are-criteria-to-choose-between-finite-differences-and-finite-elements/303&quot;&gt;What are criteria to choose between finite-differences and finite-elements?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Low order conservative finite difference methods are readily available for unstructured grids. High order non-oscillatory FD methods are another matter. In Finite Difference WENO schemes, the physics appears in a flux splitting that is not available for all Riemann solvers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finite volume methods work fine in multiple dimensions, but to go higher than second order for general flow structures, you need extra face quadrature points and/or transverse Riemann solves, greatly increasing the cost relative to FD methods. However, these FV methods can be applied to non-smooth and unstructured meshes and can use arbitrary Riemann solvers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Continuous finite element methods can be used for CFD, but stabilization becomes delicate. It is not usually practical to have strictly non-oscillatory methods and stabilization often needs additional information like entropy. When the consistent mass matrix is used, explicit time stepping becomes much more expensive. Continuous Galerkin methods are not locally conservative, which causes problems for strong shocks. See also &lt;a href=&quot;https://scicomp.stackexchange.com/questions/41/why-is-local-conservation-important-when-solving-pdes&quot;&gt;Why is local conservation important when solving PDEs?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Discontinuous Galerkin methods can use any Riemann solver to connect elements. They have better inherent nonlinear stability properties than the other common methods. DG is also rather complicated to implement and is not generally monotone inside an element. There are limiters for DG that ensure positivity or a maximum principle.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are other methods like Spectral Difference (e.g. &lt;a href=&quot;http://dx.doi.org/10.1007/s10915-006-9113-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wang et al 2007&lt;/a&gt; or &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2008.12.038&quot; rel=&quot;nofollow noreferrer&quot;&gt;Liang et al 2009&lt;/a&gt;) that have the potential to be very efficient (like Finite Difference), while having more geometric flexibility and high order accuracy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;High Reynolds number flows have thin boundary layers, requiring highly anisotropic elements to solve efficiently. For incompressible or nearly incompressible elements, this causes significant trouble for many discretizations. For additional discussion, mostly from the perspective of finite element methods, see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/43/what-spatial-discretizations-work-for-incompressible-flow-with-anisotropic-bound&quot;&gt;What spatial discretizations work for incompressible flow with anisotropic boundary meshes?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For steady problems, the ability to efficiently use nonlinear multigrid (FAS) is attractive. FD, FV, and DG methods can generally use FAS efficiently because, roughly speaking,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{(\text{cost per pointwise residual}) \cdot (\text{number of points})}{\text{cost of global residual}} \lesssim 2 . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This ratio is often more than 10 for continuous finite element methods. This ratio is not sufficient for efficient FAS with pointwise or elementwise smoothers, however. It is also necessary to have an $h$-elliptic discretization to use for defect correction, or otherwise modify the multigrid cycle. For further discussion, see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/60/is-there-a-multigrid-algorithm-that-solves-neumann-problems-and-has-a-convergenc&quot;&gt;Is there a multigrid algorithm that solves Neumann problems and has a convergence rate independent of the number of levels?&lt;/a&gt; A positive answer to this research question would potentially offer an efficient FAS for continuous finite elements.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-05-27T16:30:46.647" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2359" PostTypeId="2" ParentId="2357" CreationDate="2012-05-27T11:20:44.840" Score="3" Body="&lt;p&gt;In short for DG:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A consequence of relaxing the continuity requirements across element boundaries is that number of variables in DG-FEM is larger than for continuous counterpart for the same number of elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand because of local formulation (in terms of elements) we have following advantages:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Non-stationary and source terms are fully decoupled between elements. Mass matrices can be inverted at element level.&lt;/li&gt;&#xA;&lt;li&gt;Easier parallelization.&lt;/li&gt;&#xA;&lt;li&gt;Adaptive refinements (h-, p- and hp) are made easy - there is no need for global node renumbering.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="1050" LastEditDate="2012-05-27T22:06:55.717" LastActivityDate="2012-05-27T22:06:55.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2360" PostTypeId="1" AcceptedAnswerId="2362" CreationDate="2012-05-27T15:12:22.873" Score="9" ViewCount="908" Body="&lt;p&gt;Suppose I had the following periodic 1D advection problem:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} + c\frac{\partial u}{\partial x} = 0$ in $\Omega=[0,1]$&lt;br&gt;&#xA;$u(0,t)=u(1,t)$&lt;br&gt;&#xA;$u(x,0)=g(x)$&lt;br&gt;&#xA;where $g(x)$ has a jump discontinuity at $x^*\in (0,1)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is my understanding that for linear finite difference schemes of higher than first order, spurious oscillations occur near the discontinuity as it is advected over time, resulting in a distortion of the solution from its expected wave shape.  According to &lt;a href=&quot;http://en.wikipedia.org/wiki/Gibbs_phenomenon#Explanation&quot; rel=&quot;nofollow&quot;&gt;wikipedia explanation&lt;/a&gt;, it seems that these oscillations typically occur when a discontinuous function is approximated with a finite fourier series.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some reason, I can't seem to grasp how a finite fourier series can be observed in the solution of this PDE.  In particular, how can I estimate a bound on the &quot;over-shoot&quot; analytically?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="123" LastEditDate="2012-08-20T14:25:31.057" LastActivityDate="2019-04-18T18:04:40.500" Title="How can I derive a bound on the spurious oscillations in the numerical solution of the 1D advection equation?" Tags="|pde|finite-difference|advection|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2361" PostTypeId="2" ParentId="2360" CreationDate="2012-05-27T15:51:45.243" Score="2" Body="&lt;p&gt;Not all spurious oscillations are Gibbs phenomena. They look similar, but there are Gibbs oscillations for all finite Fourier approximations of discontinuous functions (they just get smaller as you add more terms). Whereas, there are non-oscillatory representations of discontinuous functions resulting from the solution of finite difference approximations to PDEs that don't require infinite series. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bathe (&lt;a href=&quot;http://www-dimat.unipv.it/sangalli/infsup1d.pdf&quot; rel=&quot;nofollow&quot;&gt;Inf–sup testing of upwind methods&lt;/a&gt;, PDF)  has a paper on this for finite element methods (convection-diffusion, IIRC) in 1-D that involves computing the constant for the $\inf$-$\sup$ condition and relating that to oscillations. You might gain some insight from that. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-05-27T15:51:45.243" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2362" PostTypeId="2" ParentId="2360" CreationDate="2012-05-27T18:10:46.187" Score="11" Body="&lt;p&gt;The first order upwind method is monotone; it does not introduce spurious oscillations. But it is only first order accurate, resulting in so much numerical diffusion as to be unusable for many purposes. &lt;a href=&quot;http://en.wikipedia.org/wiki/Godunov%27s_theorem&quot;&gt;Godunov's Theorem&lt;/a&gt; states that linear spatial discretizations of higher than first order cannot be monotone. To rigorously control oscillations, we use &lt;a href=&quot;http://en.wikipedia.org/wiki/Total_variation_diminishing&quot;&gt;Total Variation Diminishing (TVD)&lt;/a&gt; schemes. TVD methods are typically limited to second order accuracy. For higher order, we must either relax our request, leading to Total Variation Bounded (TVB) methods like (Weighted) Essential Non-Oscillatory ((W)ENO), or we must relax the definition of TVD to &quot;maximum-principle preserving&quot; or similar, where the initial extrema are in terms of an initial reconstructed solution, resulting in &lt;a href=&quot;http://dx.doi.org/10.1007/s10915-011-9472-8&quot;&gt;special limiting schemes&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-27T18:10:46.187" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2363" PostTypeId="2" ParentId="2360" CreationDate="2012-05-27T20:34:22.697" Score="2" Body="&lt;p&gt;As for your last question about the connection between finite Fourier series and finite element approximation: In general, if you try to &lt;em&gt;project&lt;/em&gt; a function with a jump onto a finite dimensional space whose basis functions are continuous, you get Gibbs phenomenon. This is true if the basis is a finite Fourier series (where the basis functions are the sines and cosines) or if the basis are the usual finite element hat functions -- it's a property of the projection plus the unsuitability of the basis functions.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-27T20:34:22.697" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2364" PostTypeId="2" ParentId="2279" CreationDate="2012-05-28T05:13:56.970" Score="11" Body="&lt;p&gt;You can check my links to computational programs in the USA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know one here must provide direct information and not links. So, what I'll do is copy what I have in my links page, but provide a link to the page itself, since the actual links to the programs will not get copied when I copy the page here since the links are encoded. So, if you want the actual links, there are here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://12000.org/my_notes/schools_ranking/index.htm&quot;&gt;http://12000.org/my_notes/schools_ranking/index.htm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or you can google them from the information below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the content of my page as of today&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Introduction&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;These are my collection of links to degrees in computational physics, computational science, computational engineering, Scientific computing, computational science engineering (CSE), or Applied and Computational Mathematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The differences is not that large, I think, since all these fields are really interdisciplinary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it seems the term CSE (computational science and engineering) is becoming the most common term used, at least in the US, to describe programs of this nature.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1 Computational Physics specific&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.physics.ku.edu/~physics/graduate/comp.shtml&quot;&gt;http://www.physics.ku.edu/~physics/graduate/comp.shtml&lt;/a&gt; University of Kansas has MS in computational physics and astronomy&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.physics.oregonstate.edu/~rubin/CPUG/&quot;&gt;http://www.physics.oregonstate.edu/~rubin/CPUG/&lt;/a&gt; Oregon state univ. has computational physics but for undergraduate degree&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://undergrad-catalog.buffalo.edu/academicprograms/comphys.shtml&quot;&gt;http://undergrad-catalog.buffalo.edu/academicprograms/comphys.shtml&lt;/a&gt; Univ. of Buffalo, NY state, has computational physics but undergraduate degree.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.csp.uga.edu/&quot;&gt;http://www.csp.uga.edu/&lt;/a&gt; Center for simulation physics at Univ. of Georgia has degree program    &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://info.sjsu.edu/web-dbgen/catalog/departments/PHYS-section-8.html&quot;&gt;http://info.sjsu.edu/web-dbgen/catalog/departments/PHYS-section-8.html&lt;/a&gt;California state univ. San Jose has MS in computational physics&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://iacs.cua.edu/&quot;&gt;http://iacs.cua.edu/&lt;/a&gt; The Catholic University of America, Institute for&#xA;Astrophysics and Computational Sciences within the Physics dept has graduate program&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://earthscience.rice.edu/centers/ccg/&quot;&gt;http://earthscience.rice.edu/centers/ccg/&lt;/a&gt; Rice university Computational Geophysics in the dept of earth science. They have MS and PhD programs.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.chapman.edu/scst/computational-sciences/academic-programs/bs-physics-computational-science.aspx&quot;&gt;http://www.chapman.edu/scst/computational-sciences/academic-programs/bs-physics-computational-science.aspx&lt;/a&gt; Chapman University, ORANGE, CA. Has a Bachelor of Science in&#xA;Physics and Computational Science in the School of Computational Sciences.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.uark.edu/misc/aaron5/index.html&quot;&gt;http://www.uark.edu/misc/aaron5/index.html&lt;/a&gt; University of Arkansas, Computational Condensed Matter Physics (CCMP) Group. Seems to be just for research and visiting Scientists.No degrees offered.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cmu.edu/physics/research/computational/&quot;&gt;http://www.cmu.edu/physics/research/computational/&lt;/a&gt; Carnegie Mellon. Concentration area in Computational Physics.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.uaf.edu/catalog/current/programs/physics_comp.html&quot;&gt;http://www.uaf.edu/catalog/current/programs/physics_comp.html &lt;/a&gt; UNIVERSITY OF ALASKA FAIRBANKS, MS in Computational physics.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.memphis.edu/physics/grad_physics.htm&quot;&gt;http://www.memphis.edu/physics/grad_physics.htm&lt;/a&gt; University of Memphis,Tennessee. MS degree in physics with concentration in Computational physics&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;2 Other non-physics specific computational degrees&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;HTML New graduate program at UC Berkeley in Computational Science&#xA;    and Engineering.&lt;/li&gt;&#xA;&lt;li&gt;HTML George Mason Univ. has graduate computational science program&#xA;within the Physics dept now&#xA;older site (before merging with the physics dept) HTML   &lt;/li&gt;&#xA;&lt;li&gt;HTML Stanford university. Mathematical and Computational Science program at Stanford offers programs leading to a B.S. degree, an honors option, and a minor. MCS is a multidisciplinary.&#xA;Standford also has ICME (Institute for Computational and Mathematical Engineering) has its own M.S. and Ph.D. programs. interdisciplinary programs. &lt;/li&gt;&#xA;&lt;li&gt;HTML State univ. of NY, Brockport, has undergraduate in&#xA;computational science&lt;/li&gt;&#xA;&lt;li&gt;HTML Univ. of Utah has MS degree in Computational Engineering and&#xA;Science Program CSE. Univ. of Utah has also the HTML Scientific Computing and Imaging Institute. Accepts graduate students. interdisciplinary program. They also have a Computational Bioimaging Certificate Program. Univ. of Utah also has M.S. and Ph.D. in Computing, Graphics Track in the school of computing.  &lt;/li&gt;&#xA;&lt;li&gt;HTML Rice univ. has MS in computational science and engineering&lt;/li&gt;&#xA;&lt;li&gt;HTML Rice univ. has also graduate program in computational and applied&#xA;mathematics.&lt;/li&gt;&#xA;&lt;li&gt;HTML The University of Southern Mississippi has graduate program in&#xA;computational science&lt;/li&gt;&#xA;&lt;li&gt;HTML Syracuse University has MS computational science&#xA;    and also certificates in computational science HTML&lt;/li&gt;&#xA;&lt;li&gt;HTML Purdue Univ. Indiana, has graduate program in computational&#xA;science and engineering CSE. Purdue also has Center for Computational &amp;amp; Applied Mathematics (CCAM), in the Math dept. Seems to accept graduate students.&lt;/li&gt;&#xA;&lt;li&gt;HTML Department of Mathematics and Computer Science at the Courant&#xA;Institute of Mathematical Sciences, New York University has MS in&#xA;scientific computing&lt;/li&gt;&#xA;&lt;li&gt;HTML HTML University of Michigan, College of Engineering, Lab for Scientific Computation, LaSC, has PhD in Scientific Computing.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Michigan also has a Phd program in Scientific&#xA; Computing The doctoral program in scientific computing was&#xA; established by the University of Michigan in 1989.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Illinois at Urbana Champaign (UIUC) has graduate program in Computational Science and Engineering CSE. HTML UIUC also has Scientific Computing Group in the dept of computer science. Graduate level. &lt;/li&gt;&#xA;&lt;li&gt;HTML HTML Penn State has a graduate minor in Computational Science. Also Penn State has CCMA, the Center for Computational Mathematics and Applications, HTML. Not sure if they offer direct study programs. &lt;/li&gt;&#xA;&lt;li&gt;HTML University of Texas At Austin has MS in Computational and&#xA; Applied Mathematics&lt;/li&gt;&#xA;&lt;li&gt;HTML I think the above program is now replaced by this: The&#xA; university of Texas at Austin, Institute for Computational&#xA; Engineering and Sciences (ICES), has a Computational Science,&#xA; Engineering, and Mathematics (CSEM) graduate program. Formerly&#xA; Computational and Applied Mathematics. An interdisciplinary&#xA; program.&lt;/li&gt;&#xA;&lt;li&gt;HTML San Diego State/Claremont Graduate University: PhD degree in&#xA; computational science&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Minnesota has graduate program in scientific&#xA; computation&lt;/li&gt;&#xA;&lt;li&gt;HTML UC Davis has graduate program in CSE&lt;/li&gt;&#xA;&lt;li&gt;HTML Michigan Tech university, MI, Has PhD in CSE&lt;/li&gt;&#xA;&lt;li&gt;HTML HTML Indiana University Bloomington has minor in Scientific&#xA; Computing&lt;/li&gt;&#xA;&lt;li&gt;HTML HTML Florida state univ. has graduate Degrees In APPLIED AND COMPUTATIONAL MATHEMATICS and also a program in the Department of Scientific Computing. &lt;/li&gt;&#xA;&lt;li&gt;HTML Princeton has a graduate program in Applied and Computational&#xA; Mathematics (PCAM). There is also Princeton Institute for Computational Science and Engineering (PICSciE) but I am not sure it offers direct programs. It seems only for research.&lt;/li&gt;&#xA;&lt;li&gt;HTML RPI has graduate program in CSE&lt;/li&gt;&#xA;&lt;li&gt;HTML Univ. Of California, San Diego UCSD new graduate program called CSME (computational science mathematics and engineering. UCSD has also HTML the center for Computational Mathematics (CCoM). &lt;/li&gt;&#xA;&lt;li&gt;HTML Caltech Computing + Mathematical Sciences Department offers an&#xA; interdisciplinary program of graduate study in Applied and&#xA; Computational Mathematics leading to a PhD degree.&lt;/li&gt;&#xA;&lt;li&gt;HTML Clark Univ. Computational Science program&lt;/li&gt;&#xA;&lt;li&gt;HTML Colorado state univ. Master of Science Degree in Mathematics&#xA; with Specialization in Applied and Computational Mathematics&lt;/li&gt;&#xA;&lt;li&gt;HTML Cornell Univ. Program in Computational Science and Engineering&lt;/li&gt;&#xA;&lt;li&gt;HTML Duke Univ Computational Science, Engineering and Medicine&#xA; program (CSEM)&lt;/li&gt;&#xA;&lt;li&gt;HTML Georgia Institute of Technology, Computational Science and&#xA; Engineering Division (CSE)&lt;/li&gt;&#xA;&lt;li&gt;HTML Georgia state Univ. Dept of Mathematics has MS degree with&#xA; concentrations in scientific computing.&lt;/li&gt;&#xA;&lt;li&gt;HTML Louisiana State University, Center for Computation and&#xA; Technology. Graduate level program.&lt;/li&gt;&#xA;&lt;li&gt;HTML Louisiana Tech University, PhD program in Computational&#xA; Analysis and Modeling (CAM)&lt;/li&gt;&#xA;&lt;li&gt;HTML Loyola university has an MS in Scientific and Technical&#xA; Computing&lt;/li&gt;&#xA;&lt;li&gt;HTML MIT has MS degree program in in Computation for Design and&#xA; Optimization in the center of computational engineering.&lt;/li&gt;&#xA;&lt;li&gt;HTML Mississippi State University has graduate studies in&#xA; Computational Engineering and high performance computing&lt;/li&gt;&#xA;&lt;li&gt;HTML College of Engineering at North Carolina Agricultural and&#xA; Technical State University has graduate program in Computational&#xA; Science and Engineering (CSE)&lt;/li&gt;&#xA;&lt;li&gt;HTML Center for Research in Scientific Computation (CRSC) at North&#xA; Carolina state univ. has graduate studies program.&lt;/li&gt;&#xA;&lt;li&gt;HTML Northwestern univ. has a graduate Engineering and applied&#xA; mathematics program (ESAM) which seems to be computational based.&lt;/li&gt;&#xA;&lt;li&gt;HTML Ralph Regula School of Computational Science at the Ohio&#xA; Supercomputer Center has an undergraduate program in computational&#xA; science.&lt;/li&gt;&#xA;&lt;li&gt;HTML Ohio university has Master of Science in Mathematics -&#xA; Computational Track&lt;/li&gt;&#xA;&lt;li&gt;HTML Old Dominion University has a graduate program in modeling and&#xA; simulation and has certificate program in computational science&#xA; HTML&lt;/li&gt;&#xA;&lt;li&gt;HTML Southern Methodist University Center for Scientific&#xA; Computation has graduate studies.&lt;/li&gt;&#xA;&lt;li&gt;HTML Stony brrok university, NY, has graduate program in&#xA; COMPUTATIONAL APPLIED MATHEMATICS in the applied Math dept.&lt;/li&gt;&#xA;&lt;li&gt;HTML Texas A&amp;amp;M Institute for Scientific Computation. Interesting&#xA; courses, FEM and numerical HTML do not know what degrees they&#xA; offer.&lt;/li&gt;&#xA;&lt;li&gt;HTML UCLA Computational &amp;amp; Applied Mathematics, in the Math dept.&#xA; Has graduate programs.&lt;/li&gt;&#xA;&lt;li&gt;HTML Univ. Of California, Santa Barbra (UCSB) has CSE program for&#xA; MS and PhD. multidisciplinary program.&lt;/li&gt;&#xA;&lt;li&gt;HTML Univ of Chicago, dept of computer science, has a Computational&#xA; Mathematics track of the Ph.D. program in Computer Science.&lt;/li&gt;&#xA;&lt;li&gt;HTML The University of Arizona, Computational Science and Numerical&#xA; Analysis program, in the Applied Math dept.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Delaware. Graduate Program in Mathematical&#xA; Computation, within the computer science dept. MS/PhD level.&lt;/li&gt;&#xA;&lt;li&gt;HTML Also at University of Delaware, in the Math dept, a Numerical&#xA; Analysis and Scientific Computing program. Courses seem really&#xA; interesting. Graduate level.&lt;/li&gt;&#xA;&lt;li&gt;HTML The Computational Sciences Initiative at the University of&#xA; Houston (CSIUH) interdisciplinary graduate certificate program.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Illinois at Chicago, Applied Mathematics and&#xA; Computational Science (AMCS). Seems to be an MS program in the Math&#xA; dept.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Iowa. Applied Mathematical and Computational&#xA; Sciences program (AMCS) is an interdisciplinary Ph.D. program in&#xA; the Graduate College.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Kentucky, dept of Math, has an Applied and&#xA; Computational Mathematics graduate level program.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Maryland. Applied Mathematics and Scientific&#xA; Computation (AMSC) Graduate Program They also have a Scientific&#xA; Computation Graduate Certificate Program.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Maryland also has Center for Scientific&#xA; Computation and Mathematical Modeling (CSCAMM). See the above AMSC&#xA; program.&lt;/li&gt;&#xA;&lt;li&gt;HTML Univ. of Massachusetts, darthmouth, has Mathematics Major with&#xA; Computational Mathematics concetration.&lt;/li&gt;&#xA;&lt;li&gt;HTML Univ. of Massachusetts, Lowell. Show a PhD program in&#xA; Computational Mathematics&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Michigan-Dearborn has Master of Science Degree&#xA; in Applied and Computational Mathematics in the department of&#xA; Mathematics and Statistics.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Minnesota Duluth, has Master of Science in&#xA; Applied and Computational Mathematics in the dept of Mathematics&#xA; and statistics.&lt;/li&gt;&#xA;&lt;li&gt;HTML The University of Tennessee, Knoxville. Has Interdisciplinary&#xA; Graduate Minor in Computational Science (IGMCS)&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Tennessee at Knoxville, Joint Institute for&#xA; Computational Sciences has graduate program in Applied and&#xA; Computational Mathematics through the math dept.&lt;/li&gt;&#xA;&lt;li&gt;HTML University of Washington, seattle, College of Arts and&#xA; Sciences, The Applied and Computational Mathematical Sciences&#xA; (ACMS) program, multidisciplinary Bachelor of Science degree&#xA; program in the College of Arts and Sciences.&lt;/li&gt;&#xA;&lt;li&gt;HTML Western Michigan University, Math dept has Master of Science&#xA; in Applied and Computational Mathematics&lt;/li&gt;&#xA;&lt;li&gt;HTML Johns Hopkins University, Whiting School of Engineering, has&#xA; Applied and Computational Mathematics at the MSc level.&lt;/li&gt;&#xA;&lt;li&gt;HTML Chapman University, ORANGE, CA. offers MS and PhD in&#xA; Computational Science.&lt;/li&gt;&#xA;&lt;li&gt;HTML Central Washington Univ. WA. Has Computational Science Masters&#xA; Program. Offered by the Computer Science Department.&lt;/li&gt;&#xA;&lt;li&gt;HTML Boston University, The Boston University Center for&#xA; Computational Science (CCS). CCS offers a Certificate in&#xA; Computational Science to graduate students in engineering and&#xA; science pursuing a Ph.D.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="727" LastEditorUserId="727" LastEditDate="2013-04-01T08:35:03.700" LastActivityDate="2013-04-01T08:35:03.700" CommentCount="0" CommunityOwnedDate="2012-05-28T07:33:33.100" ContentLicense="CC BY-SA 3.0" />
  <row Id="2365" PostTypeId="2" ParentId="2360" CreationDate="2012-05-28T06:13:51.237" Score="5" Body="&lt;p&gt;Linear finite difference discretization of a 1D problem with periodic boundaries leads to a discretization of the form&#xA;$$U^{n+1} = LU^n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L$ is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Circulant_matrix&quot; rel=&quot;noreferrer&quot;&gt;circulant matrix&lt;/a&gt;.  The eigenvectors of any circulant matrix are discrete Fourier modes&#xA;$$v_j = \exp(ijh\xi)$$&#xA;(here $h$ is the grid spacing and $\xi$ is the wavenumber, which ranges from zero up to the highest wavenumber representable on the grid).  These eigenvectors form a basis for all functions that can be represented on the grid.  If you express the solution in terms of these discrete Fourier modes, then the numerical method is diagonalized, i.e. each Fourier component is multiplied by a (generally complex) scalar factor at each step.  The scalar factor is often referred to as the amplification factor, and what I have just described is known as &lt;a href=&quot;http://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&quot; rel=&quot;noreferrer&quot;&gt;von Neumann analysis&lt;/a&gt;.  It is analogous to Fourier analysis of linear PDEs, in which one uses a Fourier basis, to &quot;diagonalize&quot; the linear differential operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find nice explanations, for instance, in the text of &lt;a href=&quot;http://www.ec-securehost.com/SIAM/ot88.html&quot; rel=&quot;noreferrer&quot;&gt;Strikwerda&lt;/a&gt; or &lt;a href=&quot;http://www.ec-securehost.com/SIAM/OT98.html&quot; rel=&quot;noreferrer&quot;&gt;LeVeque&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-05-28T06:13:51.237" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2368" PostTypeId="1" AcceptedAnswerId="2413" CreationDate="2012-05-28T07:33:12.850" Score="3" ViewCount="375" Body="&lt;p&gt;Suppose I had a stochastic partial differential equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^2U=F(x,D)$, where $x\in\Omega\equiv [0,1]$ &#xA;and $F(x,D)$ is a function which depends on position $x$ and a uniform random coefficient D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approximate the stochastic solution can be easily obtained by monte-carlo simulations.  That is, I can choosing different values of D randomly, obtaining the approximate solution of the PDE by some numerical scheme, then accumulate these solutions in histogram bins in the $X-U$ space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now then, let's suppose that the function $F$ depends not on a single random parameter, but on a vector of random parameters $\vec{D}$.  For simplicity, let's assume that each component of $\vec{D}$ has the same probability distribution.  Let's further assume that each component of F is a function of a single, unique random parameter only.  Then,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Would a monte-carlo approach require substantially more samples than in the 1 random parameter case?  If so, how can I quantify the rate of convergence in terms of the sample size and number of random parameters?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a more efficient approach to approximate the solution of the stochastic pde which requires fewer random samples in the case of the vector of random parameters? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/presentations/fsu_2007.pdf&quot; rel=&quot;nofollow&quot;&gt;this presentation&lt;/a&gt; (slide #50), which proposes a similar problem to the one I proposed above.   Slide #52 proposes a sparse grid approach to resolving this problem, but does not give too much detail about this approach.  I'm really curious about the details of its implementation and whether it can be applied to the problem I proposed above.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-06-01T02:18:11.197" LastActivityDate="2012-06-01T02:59:10.920" Title="Sampling strategies to solve a stochastic partial differential equation" Tags="|pde|stochastic|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2369" PostTypeId="1" AcceptedAnswerId="2371" CreationDate="2012-05-28T10:06:35.077" Score="10" ViewCount="1549" Body="&lt;p&gt;I'm diving into the fascinating world of finite element analysis and would like to solve a large thermo-mechanical problem (only thermal $\rightarrow$ mechanical, no feedback). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the mechanical problem, I already grasped from &lt;a href=&quot;https://scicomp.stackexchange.com/a/659/86&quot;&gt;Geoff's answer&lt;/a&gt;, that I'll need to use iterative solver due to the size of my mesh. I further read in &lt;a href=&quot;https://scicomp.stackexchange.com/a/84/86&quot;&gt;Matt's reply&lt;/a&gt;, that the choice of the correct iterative algorithm is a daunting task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking here if there exist any experience on large 3-d linear-elastic problems that would help me narrow down my search for the best performance? In my case, it's a structure with &lt;strong&gt;thin, patterned films&lt;/strong&gt;, and &lt;strong&gt;irregularly placed materials&lt;/strong&gt; (both high-CTE and low-CTE). There are no large deformations in this thermo-mechanical analysis. I can use my university's HPC [1.314 nodes, with 2 AMD Opteron processors (each 2.2 GHz/ 8 cores)].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think &lt;code&gt;PETSc&lt;/code&gt; could contain something interesting, especially the algorithms which do some sort of domain decomposition (FETI, multigrid) but I'm a bit overwhelmed by the options and have no experience. I also like the phrase &lt;em&gt;&quot;geometrically informed preconditioners&quot;&lt;/em&gt;, but am unsure if this helps me. I have not yet found something focussing on linear continuum mechanics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strong scaling (Amdahl) is very important in my application because my industrial partners can't wait a long time for simulation results. I definitely not only appreciate answers, but also recommendations for further reading in the comments. &lt;/p&gt;&#xA;" OwnerUserId="86" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-09-30T04:17:36.357" Title="What is a robust, iterative solver for large 3-d linear-elastic problems?" Tags="|pde|parallel-computing|petsc|hpc|iterative-method|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2370" PostTypeId="1" CreationDate="2012-05-25T19:33:49.520" Score="2" ViewCount="151" Body="&lt;h1&gt;Modification&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Is a continuous adjoint approach a reliable/promising method for optimizing the shape of an airfoil under conditions of unsteady turbulence flow?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Original Question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I saw lots of people is doing shape optimization of airfoil using optimization theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But my feeling is that its effectiveness is very limited, and the formulation in continuous adjoint and programming in discrete adjoint method are all very complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I do not think it is very promising after a few month's tests and programming. It seems the &quot;improved&quot; shape of surface is very noisy. And many of its assumptions only valid in frozen turbulence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone share more his experience on this subject?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="1229" OwnerDisplayName="Daniel" LastActivityDate="2012-05-28T13:32:16.710" Title="Would anyone give some review on adjoint methods in shape optimization method &lt;in fluid dynamics&gt;?" Tags="|fluid-dynamics|optimization|" AnswerCount="0" CommentCount="11" ClosedDate="2019-06-06T19:38:52.510" ContentLicense="CC BY-SA 3.0" />
  <row Id="2371" PostTypeId="2" ParentId="2369" CreationDate="2012-05-28T13:37:21.407" Score="6" Body="&lt;p&gt;Assuming that your structures are actually 3D (rather than only thin features, perhaps discretized with shell elements) and that the model is larger than a few hundred thousand dofs, direct solvers become impractical, especially if you only need to solve each problem once. Additionally, unless the structure is always &quot;close&quot; to a Dirichlet boundary, you will need a multilevel method to be efficient. The community is divided between &quot;multigrid&quot; and &quot;multilevel domain decomposition&quot;. For a comparison of the mathematics, see &lt;a href=&quot;https://scicomp.stackexchange.com/a/546/119&quot;&gt;my reply to: What is the advantage of multigrid over domain decomposition preconditioners, and vice versa?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The multigrid community has generally been more successful at producing general purpose software. For elasticity, I suggest using smoothed aggregation which requires an approximate &quot;near null space&quot;. In PETSc, this is done by choosing PCGAMG or PCML (configure with &lt;code&gt;--download-ml&lt;/code&gt;) and calling &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatSetNearNullSpace.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatSetNearNullSpace()&lt;/code&gt;&lt;/a&gt; to provide the rigid body modes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Domain decomposition methods offer an opportunity to coarsen faster than smoothed aggregation, thus possibly being more latency tolerant, but the &quot;sweet spot&quot; in terms of performance tends to be narrower than smoothed aggregation. Unless you want to do research on domain decomposition methods, I suggest just using smoothed aggregation, and perhaps try a domain decomposition method when the software becomes better.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-05-28T13:37:21.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2372" PostTypeId="2" ParentId="2369" CreationDate="2012-05-28T16:46:44.697" Score="5" Body="&lt;p&gt;I would say the canonical choice for this problem would be the Conjugate Gradient solver plus an algebraic multigrid preconditioner. For PETSc, hypre/boomeramg or ML would be the obvious preconditioner choices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these components when used through PETSc scale very well to thousands or tens of thousands of processors if the problem is large enough (at least ~100,000 degrees of freedom per MPI process).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-28T16:46:44.697" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2373" PostTypeId="1" AcceptedAnswerId="2375" CreationDate="2012-05-29T05:58:15.410" Score="8" ViewCount="482" Body="&lt;p&gt;Different numerical linear algebra and numerical optimization methods have different size regimes where they're a 'good idea', in addition to their own properties. For example, for very large optimization problems, gradient, stochastic gradient and coordinate descent methods are used instead of Newton or Interior Point methods because you don't have to deal with the Hessian. Similarly, dense linear solver methods stop being feasible after a certain size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So given that both the algorithms and the computer hardware are changing constantly, what's a good way to know, and keep up with, how big is too big for standard linear algebra and optimization solvers? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm thinking about this because when you're an end-user of numerics algorithms it's important to have a vague idea when those algorithms can be applied. Part of that is problem structure and the sort of solution desired, but part of it is also just the size of the problem.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: For more concreteness, what got me thinking about this was varying rules of thumb on the upper bounds for how large a problem interior point algorithms could solve. Earlier papers said the dimensionality should be around 1000 while later papers had revised upwards to 5000 and even more recent papers allow for even larger depending on if you can take advantage of sparsity. That's a rather large range, so I'm curious what is large for state of the art interior point methods. &lt;/p&gt;&#xA;" OwnerUserId="1420" LastEditorUserId="1420" LastEditDate="2012-05-29T08:30:08.513" LastActivityDate="2012-05-29T20:01:37.237" Title="What is too big for standard linear algebra/optimization methods?" Tags="|linear-algebra|optimization|nonlinear-programming|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2375" PostTypeId="2" ParentId="2373" CreationDate="2012-05-29T12:33:12.340" Score="7" Body="&lt;p&gt;If sparsity is preserved, optimal preconditioners are available, and inequality constraints can be resolved by a multiscale method (or the number of active constraints is not too large), the overall algorithm can be $\mathcal O(n)$ time and space. Distributing across a parallel machine adds a logarithmic term to time. If enough sparsity is available, or if matrix-free methods are used, on the order of one million degrees of freedom can be solved per core. That puts the problem size for today's largest machines at around one trillion degrees of freedom. Several groups have run PDE simulations at this scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that it is still possible to use Newton-based optimization with large design spaces, you just need to solve iteratively with the Hessian. There are many approaches to doing so efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it all depends how you define &quot;standard methods&quot;. If your definition includes multilevel structure-preserving methods, then extremely large problems are tractable. If your definition is limited to unstructured dense methods, the feasible problem sizes are much smaller because the algorithms are not &quot;scalable&quot; in either time or space.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-05-29T12:51:53.987" LastActivityDate="2012-05-29T12:51:53.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2376" PostTypeId="1" CreationDate="2012-05-29T14:06:51.337" Score="5" ViewCount="252" Body="&lt;p&gt;What is the largest problem (in DOF) that has been solved till date using a fully unstructured mesh. I know about PFLOTRAN but I am not sure if it is the largest. If someone can point to relevant papers then that would be great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually I need some pointers on creating and reading in large meshes on 4K plus cores. PFLOTRAN papers have some ideas but I am wondering if there is anything else.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2012-05-29T17:36:55.483" Title="Largest unstructured mesh problem solved till date" Tags="|unstructured-mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2377" PostTypeId="1" AcceptedAnswerId="2408" CreationDate="2012-05-29T15:05:16.083" Score="24" ViewCount="4278" Body="&lt;p&gt;I am looking for algorithms to draw standard 2d-graphs for functions that may or may not have singularities. The purpose is to write a &quot;Mini-CAS&quot;, so I have no a priori knowledge of the types of functions, the users want to graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is very old, so I imagine there must be some standard algorithms in the literature. For once I did not have much succes finding references via Google.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did find one interesting algorithm, namely this one from&#xA;&lt;a href=&quot;http://yacas.sourceforge.net/Algochapter4.html#c4s1&quot;&gt;&quot;YACAS - Book of algorithms&quot;&lt;/a&gt; named &quot;Adaptive function plotting&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in short:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there standard algorithms?&lt;/li&gt;&#xA;&lt;li&gt;Is there a test suite for known difficult-to-plot functions?&lt;/li&gt;&#xA;&lt;li&gt;What are the interesting papers to read?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1541" LastEditorUserId="1541" LastEditDate="2012-05-30T10:48:34.110" LastActivityDate="2017-06-19T03:44:44.323" Title="Algorithms for (adaptive?) function plotting" Tags="|algorithms|visualization|" AnswerCount="4" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2379" PostTypeId="2" ParentId="2376" CreationDate="2012-05-29T17:36:55.483" Score="5" Body="&lt;p&gt;We have solved problems with 2 billion unknowns in deal.II, so this is a lower bound. The problem wasn't the size but that we ran out of signed integers. I've heard anecdotally that others have solved problems with 100 billion unknowns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think that any of these computations happen on &lt;em&gt;completely&lt;/em&gt; unstructured meshes. Instead, a coarse mesh with maybe a few 100,000 or millions of cells is generated and then refined in some structured way to obtain the final mesh.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-29T17:36:55.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2380" PostTypeId="2" ParentId="2373" CreationDate="2012-05-29T17:52:46.243" Score="3" Body="&lt;p&gt;The limit is primarily given by the memory it takes to store the matrix representation, and the time to retrieve it from memory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes a few thousand the limit for dense matrix method in simple environments. For sparse problems the limit for direct solvers is much higher, but depends on the sparsity pattern, as fill-in must be accomodated. The limit for iterative methods for linear solvers is essentiall the cost of a matrix vector multiply. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The limit for solving the linear subproblems directly translates into corresponding limits for local solvers for nonlinear systems of equations and optimization problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Global solvers have much more severe limits, as they are limited by the number of subproblems that need to be solved in a branch and bound framework, or by the curse of dimensionality in stochastic search methods. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-05-29T17:52:46.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2381" PostTypeId="2" ParentId="2373" CreationDate="2012-05-29T20:01:37.237" Score="3" Body="&lt;p&gt;For a given class of problem, good ways to figure out what constitutes how big is &quot;too big&quot; are to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;search the literature in your field; understand what factors will limit execution time (these include memory usage and communication)&lt;/li&gt;&#xA;&lt;li&gt;look at test problem sets (better for serial algorithms than parallelized algorithms, since I don't know of many test problem sets for parallelized algorithms)&lt;/li&gt;&#xA;&lt;li&gt;break the algorithm down into smaller, constituent parts, and use what you know about the parts to make inferences about the whole (example: Krylov subspace methods boil down to loops of matrix-vector products, so then you have to figure out how many iterations is &quot;too many&quot; and what makes a matrix-vector product &quot;too big&quot;)&lt;/li&gt;&#xA;&lt;li&gt;try it out yourself (on a serial machine, or if you have the resources, a parallel machine). See how large of an instance you can run, or run several instances of different sizes, and do an empirical scaling analysis (time vs. problem size).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To give an example, in global optimization, the concrete answer is extremely structure-dependent. As Arnold Neumaier notes, deterministic global optimization algorithms tend to be limited by the number of subproblems that must be solved in a branch-and-bound (or branch-and-cut) framework. I have solved mixed-integer linear programs (MILPs) containing thousands of binary variables, but I suspect that the reason I can solve such large problems (comparatively speaking, for MILPs) is because the problem structure was such that few subproblems were required to solve some critical set of binary variables, and the rest could be set to zero. I know that my problem is &quot;large&quot;; I've constructed other MILPs of similar size that solve tens of thousands of times more slowly.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are global optimization test sets that give you an idea of what is &quot;run-of-the-mill,&quot; and the literature can give you ideas of what problems are &quot;large&quot;. Similar tactics exist for figuring out the state-of-the-art in problem sizes in other fields, which is how Jed Brown and Arnold Neumaier can quote these figures. It's great to get these numbers, but it's far more valuable to be able to figure out how to get them yourself when the time comes.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-05-29T20:01:37.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2382" PostTypeId="1" CreationDate="2012-05-29T20:39:22.667" Score="4" ViewCount="85" Body="&lt;p&gt;Is there any reference (preferably available online as PDF, Free would be best) which summarizes the various matrix decomposition with their conditions for use, usage, algorithm, complexity and underlying math? Of course, a single resource might not have all of them but more the merrier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would love something which covers SVD, Cholesky, LU and LDU.&lt;/p&gt;&#xA;" OwnerUserId="1549" LastActivityDate="2012-05-30T00:13:06.980" Title="Any reference which summarizes decompositions?" Tags="|matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2383" PostTypeId="1" CreationDate="2012-05-29T20:44:49.820" Score="3" ViewCount="649" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For identification of macromolecular complexes in cryoelectron tomo- &#xA;  grams covering cytoplasmic regions, we used a combination of&#xA;  subtomogram  averaging and classification. In a first step, the cross&#xA;  correlation function  between two times binned tomograms and a 25 nm&#xA;  spherical template  was computed. From the corresponding correlation&#xA;  peaks subvolumes  (24 × 24 × 24 voxels) were extracted and classified&#xA;  into 12 classes via a &lt;strong&gt;missing-wedge corrected multi reference&#xA;  classification procedure&lt;/strong&gt;, using the    initial template with different&#xA;  representations of Gaussian noise added as    starting references.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1550" LastActivityDate="2012-05-30T17:58:49.540" Title="What does &quot;missing-wedge corrected&quot; mean?" Tags="|image-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2384" PostTypeId="1" AcceptedAnswerId="2386" CreationDate="2012-05-29T21:21:49.163" Score="20" ViewCount="1205" Body="&lt;p&gt;A parallel reduction assumes that the corresponding operation is associative. This assumption is violated for addition of floating point numbers. You might ask why I care about this. Well, it makes results less reproducible. And it gets worse when simulated annealing is used to optimize (or fit parameters) over subroutines producing such non-reproducible results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the common ways to deal with this problem? What can be said about the following strategies?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Don't care about the non-reproducibility.&lt;/li&gt;&#xA;&lt;li&gt;Don't use parallel reduction with floating point numbers and addition.&lt;/li&gt;&#xA;&lt;li&gt;Create suitably sized work-packages in a reproducible way, and do the final reduction by hand.&lt;/li&gt;&#xA;&lt;li&gt;Use higher precision for the addition (but not all compilers offer higher precision floating point types).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="534" LastEditorUserId="123" LastEditDate="2012-07-23T15:21:50.740" LastActivityDate="2019-06-24T16:13:02.893" Title="How to address numerical non-associativity for parallel reduction?" Tags="|parallel-computing|reproducibility|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2385" PostTypeId="2" ParentId="2382" CreationDate="2012-05-29T21:29:02.963" Score="2" Body="&lt;p&gt;I've occasionally looked online for such things, but my favorite reference--for just the basics--is Trefethen and Bau's Numerical Linear Algebra textbook. It is, as such things go, very short and eminently readable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's one of the standard introductory books to algorithmic material on matrix decompositions and for a short introduction I know of nothing better. Unfortunately, this also includes free online material for comparison. &lt;/p&gt;&#xA;" OwnerUserId="1420" LastActivityDate="2012-05-29T21:29:02.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2386" PostTypeId="2" ParentId="2384" CreationDate="2012-05-29T22:28:10.160" Score="18" Body="&lt;p&gt;A reduction implemented using &lt;code&gt;MPI_Allreduce()&lt;/code&gt; is reproducible as long as you use the same number of processors, provided the implementation observed the following note appearing in Section 5.9.1 of the MPI-2.2 standard.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Advice to implementors&lt;/em&gt;. It is strongly recommended that &lt;code&gt;MPI_REDUCE&lt;/code&gt; be implemented so that the same result be obtained whenever the function is applied on the&#xA;  same arguments, appearing in the same order. Note that this may prevent optimizations that take advantage of the physical location of processors. (&lt;em&gt;End of advice to&#xA;  implementors&lt;/em&gt;.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you need to guarantee reproducibility at all costs, you can follow the guidelines in the next paragraph:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Advice to users&lt;/em&gt;. Some applications may not be able to ignore the non-associative nature of floating-point operations or may use user-defined operations (see Section 5.9.5)&#xA;  that require a special reduction order and cannot be treated as associative. Such&#xA;  applications should enforce the order of evaluation explicitly. For example, in the&#xA;  case of operations that require a strict left-to-right (or right-to-left) evaluation order, this could be done by gathering all operands at a single process (e.g., with&#xA;  &lt;code&gt;MPI_GATHER&lt;/code&gt;), applying the reduction operation in the desired order (e.g., with&#xA;  &lt;code&gt;MPI_REDUCE_LOCAL&lt;/code&gt;), and if needed, broadcast or scatter the result to the other&#xA;  processes (e.g., with &lt;code&gt;MPI_BCAST&lt;/code&gt;). (&lt;em&gt;End of advice to users&lt;/em&gt;.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the broader scheme of things, efficient algorithms for most applications take advantage of locality. Since the algorithm is really different when run on a different number of processes, it is just not practical to exactly reproduce results when run on a different number of processes. A possible exception is multigrid with damped Jacobi or polynomial (e.g. Chebyshev) smoothers, where it is possible for this simple method to perform very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the same number of processes, it is often beneficial to performance to process messages in the order that they are received (e.g. using &lt;code&gt;MPI_Waitany()&lt;/code&gt;), which introduces non-determinism. In such cases, you can implement two variants, the fast one that receives in any order and a &quot;debug&quot; one that receives in a static order. This requires that all underlying libraries are also written to offer this behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For debugging in some cases, you can isolate part of a computation that does not offer this reproducible behavior and perform that redundantly. Depending on how the components were designed, that change may be a tiny amount of code or very intrusive.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-05-29T22:28:10.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2387" PostTypeId="2" ParentId="2382" CreationDate="2012-05-30T00:13:06.980" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://orion.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf&quot; rel=&quot;nofollow&quot;&gt;The Matrix Cookbook&lt;/a&gt; and &lt;a href=&quot;http://faculty.uml.edu/cbyrne/bookDraft.htm&quot; rel=&quot;nofollow&quot;&gt;Applied and Computational Linear Algebra: A First Course&lt;/a&gt; cover decompositions in chapter five.&lt;/p&gt;&#xA;" OwnerUserId="1113" LastActivityDate="2012-05-30T00:13:06.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2388" PostTypeId="2" ParentId="2384" CreationDate="2012-05-30T06:30:14.833" Score="4" Body="&lt;p&gt;For addressing the issue in the context of threads on a shared memory system, I've written this page that explains what we do in deal.II:&#xA;&lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/group__threads.html#MTWorkStream&quot; rel=&quot;nofollow&quot;&gt;http://dealii.org/developer/doxygen/deal.II/group__threads.html#MTWorkStream&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-05-30T06:30:14.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2389" PostTypeId="2" ParentId="2283" CreationDate="2012-05-30T07:48:27.443" Score="7" Body="&lt;p&gt;I personally use f2py for this. However, the Fortran I write for f2py is not really modern Fortran, but rather free-form F90 with modules. I do not use assumed shapes or custom types. This method still works very well for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a &lt;a href=&quot;http://fortran90.org/src/best-practices.html#interfacing-with-python&quot;&gt;resource&lt;/a&gt; I want to share concerning the use of C as part of the wrapping. The rest of this page is recommend, too.&lt;/p&gt;&#xA;" OwnerUserId="1553" LastActivityDate="2012-05-30T07:48:27.443" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2391" PostTypeId="1" AcceptedAnswerId="2392" CreationDate="2012-05-30T09:12:52.003" Score="6" ViewCount="294" Body="&lt;p&gt;The context is I have a big matrix, 20K * 50K, and I want reduce the dimensionality. In R, it's impossible to apply PCA with more variables(columns) than observations(rows). Therefore, I am trying a partial SVD with this matrix, for instance, I calculate the top 100 left and right singular vectors. But with these singular vectors, how can I obtain the new data. For example, for PCA, we have loadings for calculating the scores in the new transformed coordinate system, can we do the same thing with SVD?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your kind response !&lt;/p&gt;&#xA;" OwnerUserId="1518" LastActivityDate="2012-05-31T07:12:02.523" Title="Derive PCA with SVD" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2392" PostTypeId="2" ParentId="2391" CreationDate="2012-05-30T13:02:33.183" Score="7" Body="&lt;p&gt;Suppose your data matrix is $\mathbf{X} \in \mathbb{R}^{n \times m}$, where $m$ is the number of data points, and $n$ is the dimensionality of the data. Let the SVD of $\mathbf{X}$ be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mathbf{X} = \mathbf{U}\boldsymbol{\Sigma}\mathbf{V}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume for the sake of simplicity that the data points that make up $\mathbf{X}$ have been not been translated (for example, if you normally subtract the mean, assume it's zero). Also, assume that the singular values are in decreasing order from left to right. Let $\mathbf{U}_{k}$ be the matrix consisting of the leftmost $k$ columns of $\mathbf{U}$ (&lt;em&gt;i.e.&lt;/em&gt;, the left singular vectors corresponding to the $k$ largest singular values). Then PCA defines the following transformations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mapping into a lower dimensional coordinate system: given $\mathbf{x} \in \mathbb{R}^{n}$, where $n$ corresponds to the dimensionality of your original data, the mapped (or transformed) data will be $\mathbf{y} \in \mathbb{R}^{k}$, where $\mathbf{y} = \mathbf{U}_{k}^{T}\mathbf{x}$&lt;/li&gt;&#xA;&lt;li&gt;Mapping the lower dimensional, transformed data back to the original host space and coordinate system: given the transformed data point $\mathbf{y} \in \mathbb{R}^{k}$, the representation of that point in the original host space and coordinate system is $\tilde{\mathbf{x}} = \mathbf{U}_{k}\mathbf{y}$.&lt;/li&gt;&#xA;&lt;li&gt;Projecting onto a lower dimensional subspace: given $\mathbf{x} \in \mathbb{R}^{n}$, PCA defines an orthogonal projector $\mathbf{P}_{k} = \mathbf{U}_{k}\mathbf{U}_{k}^{T}$, so that the projected data point $\tilde{\mathbf{x}} \in \mathbb{R}^{n}$ is defined by $\tilde{\mathbf{x}} = \mathbf{P}_{k}\mathbf{x}$. Projection is equal to the second mapping in this list composed with the first mapping in this list.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you translate all of your data by a point $\mathbf{x}_{0} \in \mathbb{R}^{n}$, just replace $\mathbf{x}$ in the above list with the expression $(\mathbf{x} - \mathbf{x}_{0})$, and the same expressions will hold.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-05-30T14:59:19.200" LastActivityDate="2012-05-30T14:59:19.200" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2393" PostTypeId="1" AcceptedAnswerId="2395" CreationDate="2012-05-30T14:35:37.203" Score="9" ViewCount="1042" Body="&lt;p&gt;I'm trying to optimize a flow distributor in a tank such that the velocity and temperature distribution across any cross-section is relatively uniform.  There are many parameters I can adjust to the maximum cross-sectional uniformity, such as the number of inlet pipes, their position, orientation, and direction.  I know that I can create a number of different geometries and test each one individually, but this is very time consuming.  I'd like to be able to write a program that can iteratively test several cases at once (in parallel), and adaptively choose a new set of geometries to test based upon the previous results.  How can I best do this?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2015-12-24T10:50:28.237" Title="How can I automate the process of optimizing the design of a physical object?" Tags="|optimization|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2394" PostTypeId="2" ParentId="2393" CreationDate="2012-05-30T14:47:41.320" Score="5" Body="&lt;p&gt;If you parameterize your geometry construction part appropriately, this is a problem of black box optimization with mixed discrete and continuous parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;DAKOTA &lt;a href=&quot;http://dakota.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;http://dakota.sandia.gov/&lt;/a&gt; and NOMAD &lt;a href=&quot;http://www.gerad.ca/NOMAD/Project/Home.html&quot; rel=&quot;nofollow&quot;&gt;http://www.gerad.ca/NOMAD/Project/Home.html&lt;/a&gt; are two useful packages that allow you to choose automatically best parameter selections. (DAKOTA has better application support, but NOMAD probably has the better optimizers.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To vary the geometry, introduce a discrete or continuous parameter for each control with which you want to influence the geometry, and automatize the construction of the geometry from the collection of controls. Note that derivative-free methods are quite slow in high dimensions, so keep the number of parameters reasonably small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After finishing exploring the space with one of the above packages, you may refine the analysis by doing a more accurate optimization in which all discrete parameters and all continuous parameters are fixed for which you can't get an analytic derivative. But you may increase the number of continuous shape parameters with respect to which you can compute analytic derivatives, as a gradient-based optimizer (such as IPOPT &lt;a href=&quot;https://projects.coin-or.org/Ipopt&quot; rel=&quot;nofollow&quot;&gt;https://projects.coin-or.org/Ipopt&lt;/a&gt; ) can efficiently handle far bigger problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't know how to get the derivative but the dependence is smooth, you may consider using an automatic differentiation program, or coding your continuous problem in AMPL, in which case the solver interface will take care of the derivatives. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the basics on shape optimization see, e.g., &#xA;Haftka, R.T. and Grandhi, R.V., tructural shape optimization--A survey, &#xA;Computer Methods in Applied Mechanics and Engineering 57 (1986), 91-106.&#xA;(Trust the description about the modeling; but don't use the solvers they recommend, as optimization technology has much improved since that time.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-05-31T14:41:17.683" LastActivityDate="2012-05-31T14:41:17.683" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2395" PostTypeId="2" ParentId="2393" CreationDate="2012-05-30T14:59:09.077" Score="4" Body="&lt;p&gt;What you want to do is shape optimization using gradient based methods. It essentially means that you need to calculate the gradient of the objective function w.r.t. to your model parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a small number of parameters you can use FD but for large number of parameters you need to look into adjoint methods. If you are using a commercial code or someone else's code that cannot solve the adjoint equations then FD is your only option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look into basic basic shape optimization books. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: For structural FE problems you can check out the book by Choi and Kim I and II&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-05-31T18:00:00.080" LastActivityDate="2012-05-31T18:00:00.080" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2396" PostTypeId="2" ParentId="2383" CreationDate="2012-05-30T17:58:49.540" Score="4" Body="&lt;p&gt;I found the answer myself :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In transmission electron microscopy one obtains a 3D image by tilting the sample stage. Usually it's 180°-x and x is called &quot;missing wedge&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Transmission_electron_microscopy#Three-dimensional_imaging&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Transmission_electron_microscopy#Three-dimensional_imaging&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1558" LastActivityDate="2012-05-30T17:58:49.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2397" PostTypeId="1" AcceptedAnswerId="2398" CreationDate="2012-05-30T18:57:12.040" Score="6" ViewCount="2221" Body="&lt;p&gt;Take $n$ points ${\bf r} =\{ r_1 = (x_1,y_1,z_1), r_2=(x_2,y_2,z_2) \ldots r_n=(x_n,y_n,z_n) \}$ enclosed in a &lt;a href=&quot;http://en.wikipedia.org/wiki/Periodic_boundary_conditions&quot;&gt;periodic box&lt;/a&gt; of length $L$, such that that the point $(0,0,0)=(L,0,0)$, $(0,0,0)=(L,L,L)$, etc... The difference between two points is the minimum distance of the first point and any of the images of the second point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a point $p$ inside the box, how can I find the $k$ nearest neighbors without brute force? I'm not sure how to apply a standard space partition (like a kd-tree) as I don't see how I can &quot;split&quot; the space.&lt;/p&gt;&#xA;" OwnerUserId="1559" LastEditorUserId="183" LastEditDate="2013-03-13T21:37:15.867" LastActivityDate="2013-03-13T21:37:15.867" Title="$k$ nearest neighbors for $n$ points with periodic boundary conditions in R^3" Tags="|algorithms|nearest-neighbors|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2398" PostTypeId="2" ParentId="2397" CreationDate="2012-05-30T20:42:20.230" Score="5" Body="&lt;p&gt;Do you have to compute the nearest neighbours of all points, or just of a single point? And do you have to compute it/them once, or several times?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you cannot assume anything about your points, then a spatial &lt;a href=&quot;http://en.wikipedia.org/wiki/Octree&quot; rel=&quot;nofollow&quot;&gt;Octree&lt;/a&gt; is probably your best bet, cutting the space into eight equal cubes recursively until you have a small number of points per cell. You can look for neighbours of a point by inspecting all points in the same cell and successively climbing up the tree and inspecting neighbouring cells as you go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can assume a maximum distance $r_c$ within which all $k$ neighbours will lie, then you could use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Cell_lists&quot; rel=&quot;nofollow&quot;&gt;Cell List&lt;/a&gt;, which consists of partitioning the space into cells of edge length of at least $r_c$ in all dimensions. The neighbours of any given point will then be in any of the 26 cell surrounding that point's cell, so these are the only candidate points you need to consider. This approach is of course only useful if $r_c$ is reasonably bounded and if you are trying to find the neighbours of more than one point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dealing with boundary conditions is not much of an added difficulty, since in both cases you are dealing with cells of points which can be wrapped periodically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're only considering a single target point, you may be better off just computing the $k$ nearest brute-force.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have to compute the $k$ neighbours continuously, and the points only move by a bit, you can get some improvement by using a &lt;a href=&quot;http://en.wikipedia.org/wiki/Verlet_list&quot; rel=&quot;nofollow&quot;&gt;Verlet list&lt;/a&gt; (a.k.a. Neighbour list): Instead of finding the $k$ nearest neighbours, find the $K&amp;gt;k$ nearest. If $r_0$ is your target point, compute $s = \|r_0-r_K\| - \|r_0-r_k\|$, i.e. the difference in distance from the $k$th and $K$th nearest neighbours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By doing this, in every iteration you only have to re-check these $K$ nearest neighbours and pick out the $k$ nearest. That is, until any of your points has moved more than $s$ since you first looked for the $K$ neighbours. If that happens, just re-start the whole procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may already know this, but if you're aiming for efficiency, you should try using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Median_algorithm#Linear_general_selection_algorithm_-_Median_of_Medians_algorithm&quot; rel=&quot;nofollow&quot;&gt;Median of Medians algorithm&lt;/a&gt; to find the $k$ or $K$ smallest point distances.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-05-31T08:18:01.733" LastActivityDate="2012-05-31T08:18:01.733" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2400" PostTypeId="1" AcceptedAnswerId="2412" CreationDate="2012-05-30T23:00:29.810" Score="8" ViewCount="900" Body="&lt;p&gt;I am interested in solving a sequence of shifted linear systems $(A+\sigma I)x = b$ for various values of $\sigma$. The matrix $A$ is sparse and not too large, so I have its LU factorization available. What is the best way to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is for calculating a frequency spectrum; $\sigma$ is roughly the frequency so it will vary smoothly in an interval. I expect that solutions to nearby $\sigma$ will be somewhat similar. My first thought was to use an iterative solver like GMRES with the LU factors as a preconditioner, but I'm wondering if there is any better method.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-11-05T16:05:51.333" Title="Solving shifted linear systems with LU factorization" Tags="|linear-algebra|linear-solver|sparse-matrix|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2401" PostTypeId="2" ParentId="2400" CreationDate="2012-05-31T03:24:30.497" Score="3" Body="&lt;p&gt;You can use the direct solve with $A + \sigma_0 I$ as a preconditioner for $A + \sigma_i I$ as long as $\sigma_i$ is reasonably close to $\sigma_0$. This tends to work well if the spectrum is away from the origin. You can improve the approximation somewhat using the first order correction described in &lt;a href=&quot;https://scicomp.stackexchange.com/a/1985/119&quot;&gt;this answer&lt;/a&gt;. If the spectrum crosses the origin between the two shifts $\sigma_0$ and $\sigma_i$, as occurs for Helmholtz, I do not know a way to utilize the old factorization.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-05-31T03:24:30.497" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2402" PostTypeId="2" ParentId="2377" CreationDate="2012-05-31T06:49:45.437" Score="9" Body="&lt;p&gt;&lt;a href=&quot;http://mathworld.wolfram.com/FunctionGraph.html&quot; rel=&quot;noreferrer&quot;&gt;The MathWorld web page on Function Graphs&lt;/a&gt; contains references to several papers which seem to be relevant on adaptative function plotting. Quoting the page:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Good routines for plotting graphs use adaptive algorithms which plot more points in regions where the function varies most rapidly (Wagon 1991, Math Works 1992, Heck 1993, Wickham-Jones 1994). Tupper (1996) has developed an algorithm [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On the other hand, on Google I stumbled upon a paper&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.uic.edu/~wilkinson/Publications/plotfunc.pdf&quot; rel=&quot;noreferrer&quot;&gt;www.cs.uic.edu/~wilkinson/Publications/plotfunc.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that explains how to properly choose the domain and other things. I hope they are useful to you.&lt;/p&gt;&#xA;" OwnerUserId="782" LastEditorUserId="1541" LastEditDate="2012-06-17T18:21:48.110" LastActivityDate="2012-06-17T18:21:48.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2403" PostTypeId="2" ParentId="2397" CreationDate="2012-05-31T07:09:11.560" Score="2" Body="&lt;p&gt;There is a simple answer where you just replicate and translate the points into your acceleration structure.  Requires 8x more ram, but 1/8th the code of a real &quot;solution.&quot;  For less than 1e7 points or so this works pretty well actully reasonably.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a nice KD-tree implementation in scipy.spatial that makes this easy.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used it once to calculate the Renyi parking coefficient for high-dimensional spaces.  If you are interested, I can see if I can dig up code.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a single point query, there is no beating brute-force.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-05-31T07:09:11.560" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2404" PostTypeId="2" ParentId="2391" CreationDate="2012-05-31T07:12:02.523" Score="1" Body="&lt;p&gt;Checkout &lt;a href=&quot;http://radimrehurek.com/gensim/&quot; rel=&quot;nofollow&quot;&gt;gensim&lt;/a&gt;, Latent Semantic Analysis is the field of linguistics code-word for iterative SVD.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-05-31T07:12:02.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2405" PostTypeId="2" ParentId="2393" CreationDate="2012-05-31T08:14:17.407" Score="3" Body="&lt;p&gt;There is also adjoint space optimization, which seems to be a lot faster than standard parametric optimization in CFD. Recently it has had a large increase in popularity within the CFD community in general, and in OpenFOAM especially. We are organizing a workshop on OpenFOAM currently, and we received a lot of abstract submissions regarding this method. If you are interested, check &lt;a href=&quot;http://www.google.de/url?sa=t&amp;amp;rct=j&amp;amp;q=adjoint%20shape%20optimization%20foam&amp;amp;source=web&amp;amp;cd=1&amp;amp;sqi=2&amp;amp;ved=0CFMQFjAA&amp;amp;url=http://www.wire.tu-bs.de/mitarbeiter/cothmer/docs/vdi.pdf&amp;amp;ei=hyfHT6K7Ocza4QSa8q2KDw&amp;amp;usg=AFQjCNHUHTUTGW1IRxkAjffpbs-8i8rcjg&amp;amp;cad=rja&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; out, for other info, just google &quot;adjoint space shape optimization in CFD&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional info: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you could use OpenFOAM, there is a Python based library that is used for just such thing, to manipulate large amount of cases and change their parameters called &lt;a href=&quot;http://openfoamwiki.net/index.php/Contrib_PyFoam&quot; rel=&quot;nofollow&quot;&gt;PyFoam&lt;/a&gt;. For a simple geometry, you could define a mesh as a simple &lt;em&gt;blockMesh&lt;/em&gt; and iterate over whatever you like. For a simple case, this is a question of writing a few loops in Python. &lt;a href=&quot;http://pastebin.com/zQ0eUd9P&quot; rel=&quot;nofollow&quot;&gt;Here's&lt;/a&gt; how the script looks like if you change the &quot;inlet&quot; boundary condition velocity. Simple mesh geometry changes would be a few more lines of code... &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2012-06-03T18:08:34.977" LastActivityDate="2012-06-03T18:08:34.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2406" PostTypeId="2" ParentId="2377" CreationDate="2012-05-31T09:23:02.763" Score="15" Body="&lt;p&gt;Knowing how other CAS do this might help you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge, Mathematica uses a variation on the following basic algorithm for plotting a one-variable function $f(x)$ or a parametric curve $(x(t), y(t))$ (I'm going to assume $f(x)$ for this description).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Start with a regularly spaced grid of points on the plotting domain.  （In Mathematica, there's a parameter to control how many to take, called &lt;code&gt;PlotPoints&lt;/code&gt;.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Look at each pair of successive line segments (defined by three successive points, $(x_1, f(x_1)),\; (x_2, f(x_2)),\; (x_3, f(x_3))$) and insert a new sampling point in the middle of both segments ($\frac{x_1+x_2}{2}$ and $\frac{x_2+x_3}{2}$) if their angle is larger than a threshold.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If we haven't reached the iteration limit yet (set by &lt;code&gt;MaxRecursion&lt;/code&gt; in Mathematica), repeat from step 2.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some of this is discussed in the book Mathematica in Action by Stan Wagon, which you can &lt;a href=&quot;http://books.google.com.mt/books?id=EbVrWLNiub4C&amp;amp;lpg=PA30&amp;amp;pg=PA28#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;see here on Google Books&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented this algorithm before to have better control over how many times my expensive to calculate function was evaluated.  Here's the Mathematica code for step 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nd[{points_, values_}] :=&#xA;Transpose@{(Drop[points, 1] + Drop[points, -1])/2,&#xA;Differences[values]/Differences[points]}&#xA;&#xA;subdivide1d[result_, resolution_, maxAngle_: 10] :=&#xA;  Module[&#xA;    {deriv, angle, dangle, pos, nf},&#xA;    deriv = nd[result\[Transpose]];&#xA;    angle = ArcTan[#2] &amp;amp; @@@ deriv;&#xA;    dangle = Differences[angle];&#xA;    pos = Flatten@Position[dangle, d_ /; Abs[d] &amp;gt; maxAngle/180 Pi];&#xA;    pos = Union[pos, pos + 1];&#xA;    nf = Nearest[result[[All, 1]]];&#xA;    Select[deriv[[pos, 1]], Abs[# - First@nf[#]] &amp;gt; resolution &amp;amp;]&#xA;  ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="605" LastActivityDate="2012-05-31T09:23:02.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2407" PostTypeId="2" ParentId="2026" CreationDate="2012-05-31T09:37:15.883" Score="14" Body="&lt;p&gt;For computing three-dimensional Delaunay triangulations (tetrahedralizations, really), &lt;a href=&quot;http://www.wias-berlin.de/software/index.jsp?id=TetGen&amp;amp;lang=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;TetGen&lt;/a&gt; is a commonly used library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your convenience, here's a little benchmark on how long it takes to compute the terehedralization of a number of random points from the unit cube.  For 100,000 points it takes 4.5 seconds on an old Pentium M.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/17nQj.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;(This was done with Mathematica's TetGen interface.  I don't know how much overhead it introduces.)&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your other question: if you already have the Voronoi tessellation, then getting the Delaunay triangulation is a &lt;a href=&quot;http://en.wikipedia.org/wiki/File:Delaunay_Voronoi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;relatively simple transformation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="27333" LastEditDate="2018-03-30T16:36:36.187" LastActivityDate="2018-03-30T16:36:36.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2408" PostTypeId="2" ParentId="2377" CreationDate="2012-05-31T09:50:35.033" Score="13" Body="&lt;p&gt;I have implemented Mathematica's adaptive sampling routine &lt;a href=&quot;https://github.com/victorliu/S4/blob/master/modules/function_sampler_1d.c&quot; rel=&quot;noreferrer&quot;&gt;here on GitHub&lt;/a&gt; (It's a single C file, go up to the source tree for the header file). I found a description of the routine in a big book on Mathematica a long time ago, and I've been using variations on this implementation for some time now. It basically does a rough linear sample over the domain of interest, then goes back through to refine regions of high curvature. It is possible that some very sharp features are missed, but in practice I find this exceedingly rare. This file also contains the parallel version.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2014-06-22T01:42:32.430" LastActivityDate="2014-06-22T01:42:32.430" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2409" PostTypeId="2" ParentId="2296" CreationDate="2012-05-31T14:36:51.210" Score="3" Body="&lt;p&gt;Below I will give an example how do I interpolate from one set of points to another one, on finite volume mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have collocated arrangement of variables - the data I store in memory represent values at cell-centers. I store field variables and their gradients. Gradients are found from surrounding values solving a least-squares problem (with QR via Householder reflections).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your arrangement may differ but the principle is the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then if I'm looking for $\phi_f$ - a value at cell face center I may get it from:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \phi_{nb1} + \nabla\phi_{nb1}\mathbb{r}_{nb1,f} = \phi_f $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \phi_{nb2} + \nabla\phi_{nb2}\mathbb{r}_{nb2,f} = \phi_f $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \phi_{nbn} + \nabla\phi_{nbn}\mathbb{r}_{nbn,f} = \phi_f $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, $nb$ designates a neighboring cell center,they go from 1 to n (very often just 1 suffices, I use 2, i.e. I use neighboring cells that share the face). $\mathbb{r}_{nbn,f}$ is a distance vector from n-th neighbors cell's center to face center $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi_f = \frac{1}{n}( \sum_{i=1}^{n} \phi_{nbi} + \sum_{i=1}^n (\nabla \phi_{nbi}\mathbb{r}_{nbi,f}) )$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you need one set of field values and gradients at those points. You need to decide which surrounding points will contribute to your interpolated point, as well as distance vectors from these points to point to which we interpolate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: If one stores data representative of values at cell vertices you use this  equation to find cell-center values, etc., all depending of what situation you have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this is based on Taylor series around the point. One can use also second derivatives to derive a more accurate expression.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-05-31T14:36:51.210" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2410" PostTypeId="2" ParentId="2393" CreationDate="2012-05-31T14:53:45.643" Score="3" Body="&lt;p&gt;As far as parameterizing the geometry is in question (as Geoff pointed out a no trivial one) - I can sincerely recommend &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=6&amp;amp;sqi=2&amp;amp;ved=0CGgQjBAwBQ&amp;amp;url=http://mx1.brendakulfan.com/docs/CST6.pdf&amp;amp;ei=dYPHT62wGYnP4QTCgf2wDg&amp;amp;usg=AFQjCNHSzjRwIZKxYvnnKBgCv9TS_bKyOA&amp;amp;sig2=DcSNBFzgZDSm0DPf6tTwYA&quot; rel=&quot;nofollow&quot;&gt;Brenda Kulfan - Universal Parametric Geometry Representation Method, J. Aircraft, Vol.45,  No.1,2008&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Described approach is applicable in aerodynamic optimization of aircrafts. &lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-05-31T14:53:45.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2411" PostTypeId="2" ParentId="2400" CreationDate="2012-05-31T15:02:06.893" Score="5" Body="&lt;p&gt;The solution $x$ (and hence any simple response computed from it) is an analytic function of $\sigma$, hence it can be well approximated locally by rational functions. The response has poles at the eigenvalues of $-A$, which is why rational interpolation is needed. The following interpolation technique should work quite well with a number of factorizations approximately equal to the number of peaks in the responses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compute factorizations for three values of $\sigma$, namely the endpoints and the midpoint of your interval, and then calculate the responses of interest and their first two (or more) derivatives with respect to $\sigma$. Note that given the factorization, you can easily get the derivatives by multiple backsolves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then use piecewise rational Hermite interpolation, first a single rational, then two rational pieces, one in each subinterval. Assuming that the responses of interest decay to zero for $\sigma\to\infty$ (if not one needs to change the degrees in the interpolant), $d$ derivatives at the endpoints of a subinterval allow one to interpolate by a quotient of a degree $d-1$ polynomial divided by a degree $d$ polynomial. For $d=2$, this captures a single Lorentz peak exactly, for $d=4$ it captures a superposition of two. (If this is not enough, one can also consider multipoint Hermite interpolation, making better use of the computed data, at the expense of more programming work.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Test the accuracy of the current interpolant by an additional exact calculation at the point where the last two interpolants differed most, using a new factorization and a new rational fit in the two intervals created by the new point, to decide when you can quit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless your spectrum is very complicated, you'll need few factorizations only.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-05-31T15:50:55.460" LastActivityDate="2012-05-31T15:50:55.460" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2412" PostTypeId="2" ParentId="2400" CreationDate="2012-05-31T20:12:40.990" Score="8" Body="&lt;p&gt;There is a method called Automated MultiLevel Substructuring (AMLS) which was originally designed for a similar problem in vibration analysis, where the solution of the linear system with a particular shift corresponds to the frequency response problem at a frequency which is the square-root of the shift. The basic idea is to use nested dissection in order to generate a tree of separators and substructures, and to use extensions of the low-frequency modes of substructures (which can be found cheaply) as a means of reducing the global model. It has a setup cost proportional to a single factorization, but the cost grows with the highest frequency that you would like to solve to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.cs.sandia.gov/~rlehoucq/amls-rev.pdf&quot; rel=&quot;nofollow&quot;&gt;original paper is here&lt;/a&gt;, and &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782508003605&quot; rel=&quot;nofollow&quot;&gt;an extension to Maxwell's equations is here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer: I used to work in the lab that invented AMLS&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-06-01T22:24:53.737" LastActivityDate="2012-06-01T22:24:53.737" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2413" PostTypeId="2" ParentId="2368" CreationDate="2012-06-01T02:59:10.920" Score="2" Body="&lt;p&gt;If you adopt the sparse grid approach, you will have to choose between &quot;intrusive&quot; stochastic Galerkin and &quot;unintrusive&quot; stochastic collocation. The former performs &quot;mixing&quot; of your approximation of uncertainty using the operators in the model, thus potentially converging faster than the latter which performs independent model evaluations. The cost of this faster convergence is that you have to solve much larger coupled systems and that code often has to be rewritten.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For background on sparse grids, see the replies and references in these questions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/451/119&quot;&gt;PDEs in Many Dimensions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/1042/119&quot;&gt;Working with multi-dimensional functions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-06-01T02:59:10.920" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2414" PostTypeId="1" CreationDate="2012-06-01T17:15:31.673" Score="2" ViewCount="187" Body="&lt;p&gt;As motivation, consider a function which is smooth and continuous but for some reason it is very expensive to perform routine calculations of finding the Laplacian on it (maybe because it is over a large domain, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given an original function, say, in one dimension, is it possible to estimate (or slightly overestimate) the maximum absolute value (i.e., the magnitude) of the Laplacian of that function anywhere over its given domain (without actually computing the Laplacian at all points and just finding the maximum)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start discussion, perhaps this can be done my taking a subset of points along the known function and using them as a basis for the estimate.&lt;/p&gt;&#xA;" OwnerUserId="1565" LastActivityDate="2012-06-01T17:15:31.673" Title="Estimating the maximum absolute value (magnitude) of the Laplacian for a given function?" Tags="|algorithms|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2415" PostTypeId="1" AcceptedAnswerId="2422" CreationDate="2012-06-02T01:33:40.900" Score="10" ViewCount="514" Body="&lt;p&gt;Suppose I have the original large, sparse linear system: $A\textbf{x}_0=\textbf{b}_0$. Now, I do not have $A^{-1}$ as A is too large to factor or any sort of decomposition of $A$, but assume that I have the solution $\textbf{x}_0$ found with an iterative solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I wish to apply a small rank update to the diagonal of A (change a few of the diagonal entries): $(A+D)\textbf{x}_1=\textbf{b}_0$ where $D$ is a diagonal matrix with mostly 0's on its diagonal and a few nonzero values. If I had $A^{-1}$ I would be able to take advantage of the Woodbury formula to apply an update to the inverse. However, I do not have this available. Is there anything I can do short of just resolving the whole system all over again? Is there some way perhaps that I can come up with a preconditioner $M$ which is easy\easier to invert, such that $MA_1 \approx A_0$, so that all I would have to do if I have $\textbf{x}_0$ is apply $M^{-1}$ and an iterative method would converge in a couple/few iterations?&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="1480" LastEditDate="2012-06-02T01:46:26.323" LastActivityDate="2012-06-06T12:50:49.290" Title="Solving a system with a small rank diagonal update" Tags="|linear-algebra|iterative-method|sparse-matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2416" PostTypeId="1" AcceptedAnswerId="2418" CreationDate="2012-06-02T06:18:24.367" Score="6" ViewCount="1022" Body="&lt;p&gt;I am considering &lt;a href=&quot;http://en.wikipedia.org/wiki/MINPACK&quot; rel=&quot;nofollow noreferrer&quot;&gt;minpack&lt;/a&gt; software package to solve my optimization problem ( this is the kind of &lt;a href=&quot;https://math.stackexchange.com/questions/65462/solving-a-set-of-3-nonlinear-equations-with-constraints&quot;&gt;question&lt;/a&gt; that I am facing), but I don't quite know what is the memory requirement and the speed of this package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea?&lt;/p&gt;&#xA;" OwnerUserId="690" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-06-05T05:40:56.557" Title="The speed and memory requirement of minpack" Tags="|optimization|software|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2417" PostTypeId="2" ParentId="2254" CreationDate="2012-06-02T19:41:56.853" Score="1" Body="&lt;p&gt;I will collect here some interesting references:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Landscape of Parallel Computing Research: A View from Berkeley (2006)&#xA;&lt;a href=&quot;http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="304" LastEditDate="2012-06-02T19:57:52.227" LastActivityDate="2012-06-02T19:57:52.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2418" PostTypeId="2" ParentId="2416" CreationDate="2012-06-02T20:13:17.773" Score="7" Body="&lt;p&gt;Don't use MINPACK. It's over 30 years old, and better, more modern optimization software exists out there. More importantly, I've searched the MINPACK source code and perused the documentation (the PDFs are scanned images, and can't be searched), I don't see any options to accommodate the constraints in your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not clear to me that your problem is convex. However, before considering global optimization algorithms, I'd try more modern optimization packages. The standard open-source constrained nonlinear optimization package is &lt;a href=&quot;https://projects.coin-or.org/Ipopt&quot; rel=&quot;nofollow&quot;&gt;IPOPT&lt;/a&gt;, which has an excellent reputation in the optimization community. Its authors (primarily Andreas Wächter and Carl Laird) have won an award from INFORMS based on the quality of the software. It's probably overkill for your particular problem, but if you have any interest in solving similar case studies that are larger, it's well worth your while to use IPOPT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also several other solvers out there. Arnold Neumaier has a &lt;a href=&quot;https://www.mat.univie.ac.at/~neum/glopt/software_l.html&quot; rel=&quot;nofollow&quot;&gt;web page&lt;/a&gt; listing many solvers; you should also check the solvers used by &lt;a href=&quot;http://www.gams.com/docs/document.htm&quot; rel=&quot;nofollow&quot;&gt;GAMS&lt;/a&gt; and &lt;a href=&quot;http://www.ampl.com/solvers.html&quot; rel=&quot;nofollow&quot;&gt;AMPL&lt;/a&gt;, since Professor Neumaier's list does not contain some of those solvers. Since your problem is a constrained nonlinear least squares problem, you should also consider solvers that exploit that structure as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GAMS is an excellent modeling language for optimization problems; if you just need to solve the optimization problem, or you want to do some prototyping, it is a very good tool. The trial license should suffice for your small problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the size of your example, it's unlikely that any optimization package will have trouble with your problem with respect to memory and processing power requirements. Memory will definitely not be an issue, and processing power could only be an issue in cases where the problem is nonconvex and has nonzero local minima, or if your objective function is expensive to evaluate. Given that you're considering solving an optimization problem with this objective function using MINPACK, I'm going to assume that the objective function is not expensive to evaluate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for large examples, more modern packages like IPOPT take advantage of sparsity, whereas MINPACK does not. Taking advantage of sparsity in the optimization algorithm reduces memory requirements and generally decreases the time it takes to solve a given problem.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-06-05T05:40:56.557" LastActivityDate="2012-06-05T05:40:56.557" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2419" PostTypeId="1" AcceptedAnswerId="2423" CreationDate="2012-06-03T08:57:24.170" Score="3" ViewCount="821" Body="&lt;p&gt;For non linear system of hyperbolic PDE, The finite volume methods work well (because of inherent conservation). Godunov scheme is a very elegant solution philosophy. For linear system, it is nothing but upwinding, but for nonlinear system, there are entropy issues.&#xA;Also the Riemann solution at the local Riemann problem can not be determined if the global solution is the centered expansion fan. &#xA;For this issue, we apply entropy fix. (or local linearisation e.g. Roe's)&#xA;There are lot of solutions provided for this.&#xA;Which one gives most consistent and accurate results?&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2014-06-15T13:31:02.430" Title="Entropy fix for godunov scheme" Tags="|fluid-dynamics|hyperbolic-pde|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2420" PostTypeId="2" ParentId="2246" CreationDate="2012-06-03T09:08:22.973" Score="3" Body="&lt;p&gt;This reference might be very useful for you.&#xA;This is an open book on internet. I learned (still am learning), python from this book. I found it very good resource indeed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.openbookproject.net/thinkcs/python/english2e/&quot; rel=&quot;nofollow&quot;&gt;http://www.openbookproject.net/thinkcs/python/english2e/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For numerical calculation, One should definitely go for 'numpy'.&#xA;(just make sure that you have understood the 'array' and 'matrix' and 'list' properly)&#xA;(refer numpy documentation for that)&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2012-06-03T09:08:22.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2421" PostTypeId="1" CreationDate="2012-06-03T09:27:40.203" Score="10" ViewCount="2100" Body="&lt;p&gt;As an assignment in college, I did a 1d simulation.&#xA;The problem statement was to solve 1d shock tube problem involving compressible ideal gas as working fluid.&#xA;For this problem, I solved system of Eulers equations using Roe's Riemann solver.&#xA;I want to know, &#xA;to solve the Euler's equations in 2 or 3 dimensions, where should I start?&#xA;Which is the test problem, i should consider first?&#xA;(Please don't suggest commercial solvers. I want to write my own code)&#xA;just I need some help in writing my own code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the good resources that introduce 2d problem in the most practical way?&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2012-06-06T11:38:35.080" Title="Euler equations in 2d" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2422" PostTypeId="2" ParentId="2415" CreationDate="2012-06-03T12:09:10.317" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Save in the columns of two matrices $B$ and $C$ all vectors $b_j$ to which you applied the matrix in the previous iterations and the results $c_j=Ab_j$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each new system $(A+D)x'=b'$ (or $Ax=b'$, which is the special case $D=0$), approximately solve the overdetermined linear system $(C+DB)y\approx b'$, e.g., by selecting a subset of the rows (possibly all) and using a dense least square method. Note that only the selected part of $C+DB$ needs to be assembled; so this is a fast operation!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Put $x_0=By$. This is a good initial approximation with which to start the iteration for solving $(A+D)x'=b'$. In case further systems must be processed, use the matrix vector products in this new iteration to extend the matrices $B$ and $C$ on the resulting subsystem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If the matrices $B$ and $C$ do not fit into main memory, store $B$ on disk, and select the subset of rows in advance. This allows you to keep in core the relevant part of $B$ and $C$ needed to form the least squares system, and the next $x_0$ can be computed by one pass through $B$ with little use of core memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rows should be selected in such a way that they approximately correspond to a coarse discretization of the full problem. Taking five times more rows than the total number of expected matrix vector multiplies should be enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Why does this work? By construction, the matrices $B$ and $C$ are related by $C=AB$. If the subspace spanned by the columns of $B$ contains the exact solution vector $x'$ (a rare but simple situation) then $x'$ has the form $x'=By$ for some $y$. Substituting this into the equation defining $x'$ gives the equation $(C+DB)y= b'$. Thus in this case, the above process gives as starting point $x_0=By=x'$, which is the exact solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, one cannot expect $x'$ to lie in the column space of $B$, but the starting point generated will be the point in this cloumn space closest to $x'$, in a metric determined by the selected rows. Thus it is likely to be a sensible approximation. As more systems are processed, the column space grows and the approximation will be likely to improve a lot, so that one can hope to converge in fewer and fewer iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit2: About the subspace generated: If one solves each system with a Krylov method, the vectors used to get the starting point for the second system span the Krylov subspace of the first right hand side. Thus one gets a good approximation whenever this Krylov subspace contains a vector close to the solution of your second system.&#xA;In general, the vectors used to get the starting point for the $(k+1)$st system span a space containing the Krylov subspace of the first $k$ right hand sides.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-06-06T12:50:49.290" LastActivityDate="2012-06-06T12:50:49.290" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2423" PostTypeId="2" ParentId="2419" CreationDate="2012-06-03T13:08:30.940" Score="5" Body="&lt;p&gt;Godunov's method has an exact Riemann solver so no entropy fix is needed. A Roe solver (of which there are a few variants) uses a local linearization which has no diffusion to &quot;fill in&quot; the rarefaction fan, so it needs an entropy fix. Other approximate Riemann solvers, including Lax-Friedrichs, Rusanov, and the HLL family are inherently diffusive and do not need an entropy fix. Not surprisingly, an exact Riemann solver will give the best results for a given spatial discretization, but may be much more computationally expensive. Approximate Riemann solvers are more attractive with high order methods like WENO. Two books that you may find useful are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;lr=&amp;amp;id=SqEjX0um8o0C&quot;&gt;Toro, &lt;em&gt;Riemann Solvers And Numerical Methods for Fluid Dynamics&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot;&gt;LeVeque, &lt;em&gt;Finite Volume Methods for Hyperbolic Problems&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-06-03T13:08:30.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2424" PostTypeId="1" AcceptedAnswerId="2425" CreationDate="2012-06-03T17:59:23.073" Score="8" ViewCount="7927" Body="&lt;p&gt;I want to solve the &lt;a href=&quot;http://www.astro.psu.edu/users/rbc/a534/lec15.pdf&quot;&gt;Lane-Emden isothermal equation [PDF, eq. 15.2.9]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2 \!\psi}{d \xi^2} + \frac{2}{\xi} \frac{d \psi}{d \xi} = e^{-\psi}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the initial conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(\xi = 0) = 0 \quad \left.\frac{d\psi}{d \xi}\right|_{\xi = 0} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html&quot;&gt;SciPy &lt;code&gt;odeint()&lt;/code&gt;&lt;/a&gt; but, as it can be seen, the equation is singular at the origin. The documentation states it uses ODEPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already know the power series of the solution in a neighbourhood of $\xi = 0$ (&lt;a href=&quot;http://www.physics.uc.edu/~sitko/Astrophysics-II/18-Polytropes&amp;amp;Isothermals/18-Polytropes&amp;amp;Isothermals.htm&quot;&gt;ref&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(\xi) \simeq \frac{\xi^2}{6} - \frac{\xi^4}{120} + \frac{\xi^6}{1890}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried setting &lt;code&gt;tcrit&lt;/code&gt; to &lt;code&gt;np.array([0.0])&lt;/code&gt;, but didn't work: I get a warning about invalid values and then my solution is all &lt;code&gt;NaN&lt;/code&gt;. Should I integrate starting from 0.01 maybe? Or is there any other solution?&lt;/p&gt;&#xA;" OwnerUserId="782" LastEditorUserId="782" LastEditDate="2012-06-04T05:46:55.920" LastActivityDate="2012-06-04T11:37:14.897" Title="Solving non-linear singular ODE with SciPy odeint / ODEPACK" Tags="|python|ode|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2425" PostTypeId="2" ParentId="2424" CreationDate="2012-06-03T23:01:38.943" Score="7" Body="&lt;p&gt;All right, this answer is a shot in the dark, but here goes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, transform the second-order ODE into a system of two ODEs. Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\varphi_{1} &amp;amp;= \psi, \\&#xA;\varphi_{2} &amp;amp;= \dot{\psi},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the dots atop functions correspond to differentiation with respect to the independent variable (in this case, $\xi$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the second-order implicit ODE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\ddot{\psi}(\xi) + 2\xi^{-1}\dot{\psi}(\xi) &amp;amp;= e^{-\psi(\xi)} \\&#xA;\psi(0) &amp;amp;= 0 \\&#xA;\dot{\psi}(0) &amp;amp;= 0&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can be expressed as the first-order explicit ODE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot{\varphi}_{1}(\xi) &amp;amp;= \varphi_{2}(\xi) \\&#xA;\dot{\varphi}_{2}(\xi) &amp;amp;= -2\xi^{-1}\varphi_{2}(\xi) + e^{-\varphi_{1}(\xi)} \\&#xA;\varphi_{1}(0) &amp;amp;= 0 \\&#xA;\varphi_{2}(0) &amp;amp;= 0.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first, it would appear that we cannot evaluate the right-hand side of this explicit ODE system at $\xi = 0$, like a numerical integrator requires. If a solution to this system exists, then it must be differentiable. On the assumption that a solution exists, take the limit of the right-hand side as $\xi \rightarrow 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we know that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\lim_{\xi \rightarrow 0} \varphi_{2}(\xi) = 0,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because we've assumed that a solution exists, so $\varphi_{2}$ is differentiable, which means it must be continuous. The limit of a continuous function at a point is its value at that point, and we know the value of $\varphi_2(0)$ because it is an initial condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also know that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\lim_{\xi \rightarrow 0} e^{-\varphi_{1}(\xi)} = 1&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for similar reasons; we've assumed that $\varphi_{1}$ is differentiable, so it is continuous, and $\varphi_{1}(0) = 0$ because it is an initial condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\lim_{\xi \rightarrow 0} \frac{-2\varphi_{2}(\xi)}{\xi} = &#xA;\lim_{\xi \rightarrow 0} -2\dot{\varphi}_{2}(\xi),&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by using L'Hôpital's rule on the indeterminate form $0/0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To proceed further, we have to make another assumption: $\dot{\varphi}_{2}$ is continuous at $\xi = 0$. Then it follows that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\lim_{\xi \rightarrow 0} -2\dot{\varphi}_{2}(\xi) = -2\dot{\varphi_{2}}(0).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Revisiting the first-order ODE, and evaluating the right-hand side at $\xi = 0$, we can see that we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot{\varphi}_{1}(0) = 0 \\&#xA;\dot{\varphi}_{2}(0) = -2\dot{\varphi}_{2}(0) + 1,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from which it follows that $\dot{\varphi}_{2}(0) = 1/3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this analysis, you could plug in an &lt;code&gt;if&lt;/code&gt; statement that returns these values of the right-hand side function at $\xi = 0$, which should get you past the singularity. That said, this analysis requires a couple assumptions about continuity that may or may not hold, so take the resulting solution with a grain of salt.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-06-04T05:53:42.517" LastActivityDate="2012-06-04T05:53:42.517" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2426" PostTypeId="1" CreationDate="2012-06-04T00:16:00.670" Score="1" ViewCount="106" Body="&lt;p&gt;I'm working on an algorithm for which I would like to cover an $n$-dimensional unit cube by a set of $n$-cuboids (i.e., $n$-dimensional rectangles).  The size and orientation of these cuboids is determined solely by the position of their centrepoint -- so given a point in the cube, one has a set of coordinate axies based at that point, a set $a_1,\ldots,a_n$ of positive numbers, and the cuboid for that point is $-a_1 \leq x_1 \leq a_1,\ldots,-a_n \leq x_n \leq a_n$ with respect to these coordinate axes.  The cuboids may overlap, but I'd like the overlap to be small if possible (for reasons off efficiency).  The unit cube must be contained in the union of the cuboids. The dimenison $n$ is small and odd, say $n=3,5,\ldots,11$. Any idea about how one could construct such a covering?  Computational cost is a minor issue here, a solution needing serious resources is acceptable (probably inevitable). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Motivation: This problem comes from a subdivide-and-reject method for a global optimisation problem with a non-convex objective.  I have a bound on a quantity of interest whose domain of validity is an $n$-ellipsoid at each point, but the shape and orientation of the ellipsoid varies over the cube.  The idea would be to put a cuboid inside the ellipsoid (so the bound is valid across it), this cube might then be rejected or subdivided uniformly, the subdivided cuboids would then have an a smaller covering ellipsoid, an improved bound, and so on. This is the reason that the cuboids must cover the cube, so that the union of these bounds covers the entire cube.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A key point is that the anisotropy is &quot;localy almost constant&quot;, once we get to a small enough size, the ellipsoids covering the cuboids are practically the same shape as the ones covering the subdivided cuboids.  One could express this in terms of a bound away from zero of the Jacobian of the anisotropy tensor or such-like, suffice it to say that the anisotropy is &quot;well behaved&quot;. &lt;/p&gt;&#xA;" OwnerUserId="1400" LastActivityDate="2012-06-04T00:16:00.670" Title="Anisotropic cover with n-cuboids" Tags="|optimization|computational-geometry|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2427" PostTypeId="1" AcceptedAnswerId="2439" CreationDate="2012-06-04T05:01:26.273" Score="7" ViewCount="382" Body="&lt;p&gt;In some hyperbolic PDEs the flux is nonconvex.&#xA;One example is equations in magnetohydrodynamics. What are the complications in the wave structures of such problems? What general precautions one should take while coding such a system? Which numerical schemes work well for such systems? &lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="123" LastEditDate="2012-06-04T17:58:23.020" LastActivityDate="2012-06-04T17:58:23.020" Title="Numerical solution of hyperbolic PDEs with nonconvex flux" Tags="|fluid-dynamics|hyperbolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2429" PostTypeId="1" AcceptedAnswerId="2431" CreationDate="2012-06-04T07:32:13.400" Score="17" ViewCount="2507" Body="&lt;p&gt;I have started using python as the programming language for doing all my assignments in CFD. I have a very little experience in programming. I am from mechanical engineering background and am pursuing higher education in Aerospace engineering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sometimes the computing aspect of CFD becomes more tedious than manipulating the equations or doing the math.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are general guidelines that make run our program faster? What are the tricks to do things parallely? How to write the codes that run faster?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where do I get resources (easy to understand for a layperson like me) that answer above questions?&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2022-12-02T23:01:42.927" Title="(how to) write simulations that run faster?" Tags="|algorithms|python|fluid-dynamics|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2430" PostTypeId="1" AcceptedAnswerId="2435" CreationDate="2012-06-04T08:00:06.533" Score="8" ViewCount="4769" Body="&lt;p&gt;I am new to the field of CFD. When should one go for structured grid and when should one go for unstructured?&#xA;(Yes, it depends a lot on the geometry of the problem)&#xA;More specifically, I want to know the difference in the computational power required, accuracy achieved and efforts involved in both types of grid.&#xA;What are good resources that explain structured and unstructured grids in most simple language?&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2012-06-04T09:58:13.980" Title="structured grid and unstructured grid" Tags="|fluid-dynamics|discretization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2431" PostTypeId="2" ParentId="2429" CreationDate="2012-06-04T08:17:40.197" Score="20" Body="&lt;p&gt;I will try to answer your question considering that you are asking for Python specifically. I will describe my own method of tackling a simulation problem. Strategies for faster simulations are given in this description.&lt;/p&gt;&#xA;&lt;p&gt;First, I prototype new simulations in Python. Of course, I try to make use of &lt;a href=&quot;https://numpy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NumPy&lt;/a&gt; and &lt;a href=&quot;https://scipy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciPy&lt;/a&gt; as much as I can. Whereas NumPy provides a suitable array data type for numerical simulations, SciPy offers a wide numerical routines working with NumPy arrays.&lt;/p&gt;&#xA;&lt;p&gt;Once the prototypes work more or less, I try to learn which parts of the program or script are the bottleneck. There are typical candidates for that:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Loops in Python are slow. Very slow.&lt;/li&gt;&#xA;&lt;li&gt;As Python uses &lt;a href=&quot;https://en.wikipedia.org/wiki/Duck_typing&quot; rel=&quot;nofollow noreferrer&quot;&gt;duck typing&lt;/a&gt;, calling functions can be slow.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I use a simple profiling strategy to learn where all the run time is spent. Using the &lt;a href=&quot;https://ipython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IPython&lt;/a&gt; shell (which I cannot recommend enough), I run my script with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;%timeit script.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This &amp;quot;magic command&amp;quot; will do the profiling (using &lt;a href=&quot;https://docs.python.org/3/library/timeit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;timeit&lt;/a&gt;) for you and present you a list with times once your script has terminated. Use this list to find out where your code is too slow.&lt;/p&gt;&#xA;&lt;p&gt;Once you nailed down the parts that need to be speeded up, you may consider using compiled languages. I will point to two solutions.&lt;/p&gt;&#xA;&lt;p&gt;First, there is the &lt;a href=&quot;https://cython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cython&lt;/a&gt; language. Cython is a programming language very similar to Python (in fact, Python code is often valid Python code, too); however, the Cython compiler converts the Cython files to C code, which then may be compiled into a module usable from Python. Cython understands NumPy arrays. There are two ways in which using Cython can help you: first, you may introduce data types. This will speed up function calls. Also, if you iterate over arrays, your loop will run faster (in fact, if you type both the dummy variable and the array, you get a plain C loop!). Second, in my experiments even the untyped scripts run a bit faster due the fact that they're compiled instead of interpreted.&lt;/p&gt;&#xA;&lt;p&gt;The other compiled language that will be useful for you is Fortran. There are different ways to use Fortran with Python (&lt;a href=&quot;https://web.archive.org/web/20140822061353/http://cens.ioc.ee:80/projects/f2py2e/&quot; rel=&quot;nofollow noreferrer&quot;&gt;f2py&lt;/a&gt;, &lt;a href=&quot;https://fortwrap.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fortwrap&lt;/a&gt;, &lt;a href=&quot;https://www.fortran90.org/src/best-practices.html#interfacing-with-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cython&lt;/a&gt;). As to me personally f2py seems to be the easiest way, I'll quickly describe what it does. f2py can compile Fortran code to Python modules. It will allow you to use NumPy arrays as input and output variables from Python space. In Fortran space, these will be ordinary Fortran arrays. You can operate on those at full Fortran speed.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I tend to use Cython where the number of function calls is the bottleneck. For loop-heavy stuff, I prefer f2py (maybe because I have a strong Fortran background).&lt;/p&gt;&#xA;&lt;p&gt;On additional note on Fortran: modern Fortran reads and writes very similar to NumPy - the syntax is very close. This makes it easy to convert NumPy code to Fortran code.&lt;/p&gt;&#xA;&lt;p&gt;Note that both Cython and f2py support paralleism in some way. For Cython, you'll find help &lt;a href=&quot;https://docs.cython.org/en/latest/src/userguide/parallelism.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, whereas for Fortran, there are the standard techniques such as &lt;a href=&quot;http://www.openmp.org/presentations/miguel/F95_OpenMPv1_v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenMP&lt;/a&gt; or MPI. Furthermore, there are P&lt;a href=&quot;http://code.google.com/p/mpi4py/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ython wrappers for MPI&lt;/a&gt;, too. Personally, I use mpi4py on the Python level as well as OpenMP in Fortran.&lt;/p&gt;&#xA;&lt;p&gt;Let me recommend a bit of literature: the book &lt;a href=&quot;https://link.springer.com/book/10.1007/978-3-540-73916-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python Scripting For Computational Science&lt;/a&gt; by H.-P. Langtangen is a great resource on Python in general as well as on strategies to make Python a bit faster. Unfortunately, AFAIR, it does not mention anything on Cython. As I second resource you may look at &lt;a href=&quot;https://github.com/aeberspaecher/PythonForScientists/raw/master/talk.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these slides&lt;/a&gt;. These give examples for everything I mentioned in this post (see also the code and sources &lt;a href=&quot;https://github.com/aeberspaecher/PythonForScientists&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). There are many other good set of slides on the internet.&lt;/p&gt;&#xA;&lt;p&gt;If you have more specific questions, we are all happy to help!&lt;/p&gt;&#xA;" OwnerUserId="1553" LastEditorUserId="22284" LastEditDate="2022-12-02T23:01:42.927" LastActivityDate="2022-12-02T23:01:42.927" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2432" PostTypeId="2" ParentId="2430" CreationDate="2012-06-04T08:36:01.923" Score="2" Body="&lt;p&gt;As you have mentioned yourself, this is greatly dependent on the geometry of the problem involved but also on the computational framework used (i.e. FEM, FDM or FVM). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite Difference Methods (FDM) are often restricted to structured grids that do not require special data structures for recording of grid information. That being said, FDM can be extended to semi-structured grids such as adaptive quadtree or octree based grids by using more sophisticated data structures. Nevertheless, structured grids are much simpler to code and easier to develope. Besides, with this type of grids, domain decomposition and parallelization are often trivial. They are usually suitable for simple domains, but certain methods, such as immersed boundary or immersed interface methods, have been developed that utilize this type of grids for even non-trivial geometries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite Volume Methods (FVM) and Finite Element Methods (FEM), on the other hand, are often more general and can (uniformly) handle various geometries. This, however, comes at the cost of using more complicated data structures which results in more complex algorithms and more development-time. They are usually harder to parallelize since the the grid now should be partitioned into sub-domains before they are sent to different processors. That being said, the existence of well-written software packages for specific tasks (such as preconditioners, linear solvers and graph partitioners) along with their robustness and versatility, makes them an excellent choice to consider if you have non-trivial geometries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, no matter which type of method (and thus grid) you choose, there both high (expensive) and low (cheap) order methods in all three different families that you can choose from for your specific problem. &lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2012-06-04T08:36:01.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2433" PostTypeId="2" ParentId="2081" CreationDate="2012-06-04T08:48:41.540" Score="1" Body="&lt;p&gt;Another approach to get started might be to have a look at some code that already does something related to your problem. In that case your exercise might even end up in some other program, and that would be nice, wouldn't it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programs that might be of interest for you are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The MD simulation and molecular modeling package &lt;a href=&quot;http://dirac.cnrs-orleans.fr/MMTK/&quot; rel=&quot;nofollow&quot;&gt;MMTK&lt;/a&gt; (as already suggested by Nate above)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The visualization package &lt;a href=&quot;http://www.pymol.org/&quot; rel=&quot;nofollow&quot;&gt;PyMol&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1568" LastActivityDate="2012-06-04T08:48:41.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2434" PostTypeId="2" ParentId="1805" CreationDate="2012-06-04T09:04:25.967" Score="1" Body="&lt;p&gt;Probably you should have a look at the &lt;a href=&quot;http://docs.enthought.com/mayavi/mayavi/example_heart.html&quot; rel=&quot;nofollow&quot;&gt;example in the Mayavi2 docs&lt;/a&gt;. This should help you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't done it in a while, but I remember that one of the important issues was to get the data into a useful structured grid.&lt;/p&gt;&#xA;" OwnerUserId="1568" LastActivityDate="2012-06-04T09:04:25.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2435" PostTypeId="2" ParentId="2430" CreationDate="2012-06-04T09:14:34.253" Score="6" Body="&lt;p&gt;I am currently implementing a VoF method (a geometrical method for two phase flow simulation on Eulerian mesh) that is native to structured grid, on an unstructured grid, so here are my experiences so far (please note that what I'm writing comes from working with a specific implementation):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;unstructured mesh&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;pros&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;fast generation of meshes for complex geometries&lt;/li&gt;&#xA;&lt;li&gt;straightforward operations on mesh topology &#xA;&lt;ul&gt;&#xA;&lt;li&gt;tetrahedral mesh: edge swapping, refinement&lt;/li&gt;&#xA;&lt;li&gt;hexahedral  mesh: octree based refinement (split the cell with 2 planes)&lt;/li&gt;&#xA;&lt;li&gt;sliding mesh interfaces (rotating geometries, etc)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;finite volume method built on top of this kind of mesh: it is robust, parallelization of the method is straightforward, implementation of boundary conditions is very easy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;cons&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;reduced accuracy because of a smaller cell stencil: you can only access face neighbours of a cell (this may differ for different mesh implementations, but in my case it is so)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;because of a smaller stencil, implementation of higher order interpolation schemes (WENO, ENO) is very difficult (problems in parallelization)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;reconstruction of gradients for sharp fields that propagate in the skew direction (involve information coming from &lt;em&gt;point neighbours&lt;/em&gt;) is not straightforward&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;structured mesh&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;pros&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;higher accuracy than for unstructured mesh: you can access points in all directions and build large stencils &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;octree based mesh refinement: the mesh is represented using an octree data structure, so the top level geometry is a box &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;refinement is much faster than for unstructured  (on an unstructured mesh, the complete mesh is copied and inflated)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;cons&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;for dealing with relative motion of bodies, complex immersed grid (chimera mesh) are used (most are not mass conservative)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if you need a boundary conform mesh, you can do it for curved boundaries, but the discretization is then translated into the curvilinear coordinate system&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;mostly used for flow domains in the shape of boxes (however, octree refinement and cell cut methods allow for fully complex geometries within the boxed domains)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, if you have a boxed domain, and a complex geometry within, and you require high accuracy, use the structured mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if the geometry of your &lt;em&gt;domain bounary&lt;/em&gt; is complex (like metal alloy casting into complex moulds), use the unstructured mesh. Also, if the simulation requires relative motion of bodies, unstructured mesh is the choice, simply because the chimera libraries are very difficult to get (military based research). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question is what is available to you at what cost, like: licence fees, time needed to learn an open source library, etc. &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2012-06-04T09:58:13.980" LastActivityDate="2012-06-04T09:58:13.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2436" PostTypeId="1" AcceptedAnswerId="2440" CreationDate="2012-06-04T10:10:38.250" Score="0" ViewCount="1907" Body="&lt;p&gt;I have solved 1d shock tube problem. (Euler's equations). Using following steps:&lt;br&gt;&#xA;1) Define Riemann problem over the domain&lt;br&gt;&#xA;2) Carry out local linearisation&lt;br&gt;&#xA;3) Based on linearisation, write eigen values and eigenvectors&lt;br&gt;&#xA;4) Use upwinding, i.e. find out numerical flux and use update equation    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I implemented above strategy in Matlab, it worked fine. But 'same' problem was when coded in python, it showed problem for some values of the left and right states.&#xA;When the left and right states were altered to some new parameters, it worked fine and got the expected results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why should this happen?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a problem with python, or the 'thinking' strategy of python and matlab are different?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used numpy for defining arrays and matrices. I used numpy.linalg.solve() function for finding the vector of characteristic variables and hence the vector of flux differences. I used pylab.plot(X,Y) for plotting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I cross checked the program several times. I guess there is no problem with it. And it runs well... its just that the graphs given are not correct for some values of the left and right states)&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="1567" LastEditDate="2012-06-04T18:31:48.643" LastActivityDate="2012-06-04T18:31:48.643" Title="1d shock tube problem, and the solution" Tags="|fluid-dynamics|python|" AnswerCount="1" CommentCount="1" ClosedDate="2012-07-03T06:13:33.570" ContentLicense="CC BY-SA 3.0" />
  <row Id="2437" PostTypeId="2" ParentId="2424" CreationDate="2012-06-04T11:37:14.897" Score="3" Body="&lt;p&gt;If you want more options for your ODE solver, have a look at the &lt;a href=&quot;http://www.jmodelica.org/assimulo&quot; rel=&quot;nofollow&quot;&gt;Assimulo&lt;/a&gt; package that implements bindings to the CVODE package (and RADAU and some simple integrators for that matter). &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-06-04T11:37:14.897" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2438" PostTypeId="2" ParentId="2421" CreationDate="2012-06-04T13:18:30.217" Score="4" Body="&lt;p&gt;Yes, the 2D problem more-or-less becomes solving a 1D problem in the X direction followed by a 1D problem in the Y direction.  The data structures are a little more complicated, the Roe matrix needs to be adjusted for characteristic projection in the second direction, and your CFL condition has to be modified as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clawpack, and in particular Clawpack's 2D examples (http://depts.washington.edu/clawpack/users-4.6/claw/doc/gallery/gallery_2d.html) may be of use to you if you want to peek at someone else's code.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2012-06-04T13:18:30.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2439" PostTypeId="2" ParentId="2427" CreationDate="2012-06-04T17:57:11.430" Score="6" Body="&lt;p&gt;A nonconvex flux function means that the characteristic velocity associated with a given characteristic field may not be a monotone function of the conserved variables.  This can lead to non-classical Riemann solutions; for instance the total number of (shock, rarefaction, and contact) waves arising in the solution may be different from the number of equations.  Another important difference from the numerical point of view is that the maximum and minimum wave speeds occurring in the Riemann problem do not generally correspond to the maximum/minimum over the input states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A full answer to your question would be very long.  This is an area of active research, but you might start by reading the relevant sections of &lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot;&gt;LeVeque's text&lt;/a&gt; (see Chapter 16) and experimenting with the Buckley-Leverett Clawpack example in the code that accompanies the book.  The Buckley-Leverett equation is one of the most well-examined non-convex fluxes and there are several good references to the literature available in that book chapter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting example for which some successful and unsuccessful schemes are known is the KPP problem introduced in &lt;a href=&quot;http://dx.doi.org/10.1137/040614189&quot;&gt;this paper&lt;/a&gt; (be sure to look at subsequent papers that cite it).  It seems that schemes without sufficient dissipation tend to generate a shock in a region that should instead have a rarefaction.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-06-04T17:57:11.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2440" PostTypeId="2" ParentId="2436" CreationDate="2012-06-04T17:57:33.833" Score="1" Body="&lt;p&gt;I guess the problem is with the pressure conditions I gave.&lt;br&gt;&#xA;1) condition 1&lt;br&gt;&#xA;Domain divided equally in 101 number of points. Two halves are defined as&lt;br&gt;&#xA;Pressure[left]=10,&lt;br&gt;&#xA;pressure[right]=0.1;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;density[left]=1&lt;br&gt;&#xA;density[right]=0.1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;velocity=0 everywhere:&#xA;results&lt;img src=&quot;https://i.stack.imgur.com/vg1HA.png&quot; alt=&quot;enter image description here&quot;&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Condition 2:&lt;br&gt;&#xA;pressure[left]=1.&lt;br&gt;&#xA;pressure[right]=0.1;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;density and velocity same as above  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/MQvb7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas on matlab, I checked only on 2nd case (but with the same code). which on python is also working fine.&#xA; So i guess my question was not mature, and I should have done this work before asking it.Here is high resolution picture taking 501 equally spaced points on domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/kNMDZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the nature of graph in first case is because the method can not handle this big jump in pressure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(surprisingly when pressure and density both were set to [10:1], it worked well!)&#xA;(any inputs on this are welcome)&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2012-06-04T17:57:33.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2441" PostTypeId="1" AcceptedAnswerId="2445" CreationDate="2012-06-04T18:51:18.980" Score="2" ViewCount="552" Body="&lt;p&gt;I am implementing a machine learning algorithm for which I need to solve an integer linear program. To get the solution in polynomial time, the authors of the algorithm have dropped the integral constraints and instead solve the corresponding linear program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not too aware of the theory of optimization, so using the Mosek optimization tool-kit as a black-box to solve the LP. Now obviously I have to add back the integral constraints once the solution of LP is obtained. Any ideas how to go about it? I am sure Mosek and other popular LP solvers would have an option for the same but can't seem to find it in their documentation or elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="1570" LastActivityDate="2012-06-05T08:11:47.257" Title="How to add back integral constraints to linear program solution" Tags="|optimization|linear-programming|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2442" PostTypeId="2" ParentId="2441" CreationDate="2012-06-04T19:58:40.517" Score="2" Body="&lt;p&gt;As a first attempt, try to solve your problem with the Mosek optimization toolkit you are using but do &lt;strong&gt;not&lt;/strong&gt; remove the integral constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the Mosek website, their solver can handle mixed-integer problems.&lt;/p&gt;&#xA;" OwnerUserId="679" LastActivityDate="2012-06-04T19:58:40.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2443" PostTypeId="2" ParentId="2429" CreationDate="2012-06-04T20:46:36.637" Score="7" Body="&lt;p&gt;For CFD+Python there is a solution: &lt;a href=&quot;http://pythonflu.wikidot.com/&quot;&gt;http://pythonflu.wikidot.com/&lt;/a&gt; These are Python-bindings on top of OpenFOAM (which was already mentioned in the comments to the questions). These bindings allow programming on &quot;solver level&quot; (there are examples where original OpenFOAM-solvers are replicated in Python and they are not slower than the originals - the slow loops mentioned in another answer are not a problem here as the &quot;inner loops&quot; happen in the C++-code of OpenFOAM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of these bindings is also that all the parallelization in OpenFOAM happens beneath the solver-level, so you don't have to bother with it (as well as the other stuff that the OpenFOAM-core takes care of: input/output, linear solver, operator discretization)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you just want to write a new solver and not add new features to the OF-core (boundary conditions, linear solver, etc) then PythonFlu might be sufficient for you and you can avoid C++ (which has a much higher learning cure than Python)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: originally wanted to add this as a comment to the discussion of the original question, but my reputation doesn't allow me this&lt;/p&gt;&#xA;" OwnerUserId="1556" LastActivityDate="2012-06-04T20:46:36.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2444" PostTypeId="1" AcceptedAnswerId="2447" CreationDate="2012-06-04T23:06:57.583" Score="12" ViewCount="2945" Body="&lt;p&gt;I asked for clarification about a recent &lt;a href=&quot;https://scicomp.stackexchange.com/q/2416/534&quot;&gt;question about minpack&lt;/a&gt;, and got the following comment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any system of equations is equivalent to an optimization problem, which is why Newton-based methods in optimization look a lot like Newton-based methods for solving systems of nonlinear equations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What confuses me about this comment (and related &lt;a href=&quot;https://scicomp.stackexchange.com/q/1749/534&quot;&gt;negative opinions&lt;/a&gt; about specialized nonlinear least squares solvers like minpack) might be best explained on the example of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;conjugate gradient method&lt;/a&gt;. This method is applicable to systems $Ax=b$ with a symmetric positive definite matrix $A$. It could also be used to solve the general least square problem $\operatorname{min}_x||Ax-b||^2$ for an arbitrary matrix $A$, but doing so is not recommended. One explanation why we shouldn't do this is that the condition number of the system would increase significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if turning a systems of equations into an optimization problem is considered problematic even for the linear case, why should it be less problematic for the general case? It seems to be somehow related to using a state of the art optimization algorithm, instead of using a slightly aged nonlinear least squares solver. But aren't the problems related to throwing away information and increasing the condition number of the system relatively independent of the actually used optimization algorithm?&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-06-15T23:00:56.550" Title="Newton-based methods in optimization vs. solving systems of nonlinear equations" Tags="|optimization|least-squares|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2445" PostTypeId="2" ParentId="2441" CreationDate="2012-06-05T03:02:42.777" Score="2" Body="&lt;p&gt;A lot of linear programming (LP) software packages will also solve mixed-integer linear programs (MILPs) with varying degrees of effectiveness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to &quot;add back the integral constraints&quot; is to warm-start an MILP solver with the solution from the LP relaxation of the integer linear program (ILP) that the authors of the machine learning algorithm use. However, Ali is right that it would be more efficient to solve the original ILP. Certain algorithms (such as branch-and-bound) will solve the LP relaxation in the process of computing an optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, any MILP solver worth using will implement MILP solution algorithms, such as branch-and-bound and branch-and-cut, with much more sophisticated algorithmic heuristics and variants than most people could code themselves. Even in problems with special structure, researchers often modify and tweak existing solvers rather than write their own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're at a university, I highly suggest using &lt;a href=&quot;http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/&quot; rel=&quot;nofollow&quot;&gt;CPLEX&lt;/a&gt; or &lt;a href=&quot;http://www.gurobi.com/&quot; rel=&quot;nofollow&quot;&gt;Gurobi&lt;/a&gt;, both of which are top-of-the-line MILP solvers that have free academic licenses. In some cases, these solvers will solve MILP instances significantly faster than their competition. That said, if your problem is small enough, speed may not really matter.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-06-05T03:02:42.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2446" PostTypeId="2" ParentId="2444" CreationDate="2012-06-05T03:14:47.987" Score="14" Body="&lt;p&gt;&lt;em&gt;If&lt;/em&gt; a given nonlinear system is the first order optimality condition for an optimization problem, then we can often produce a more robust algorithm by using that information. For example, consider the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.wolframalpha.com/input/?i=x%5E2+-+exp%28-4*%28x-2%29%5E2%29&quot; rel=&quot;noreferrer&quot;&gt;$$ f(x) = x^2 - \exp\big(-4(x-2)^2 \big) \qquad \text{[click for Wolfram Alpha]}$$&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gpDDu.png&quot; alt=&quot;Plot of original objective&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This clearly has a unique minimum and we expect our optimization method to find it regardless of starting point. But if we only look at the first order optimality conditions, we are looking for a solution $x$ of &lt;a href=&quot;http://www.wolframalpha.com/input/?i=diff%28x%5E2+-+exp%28-4*%28x-2%29%5E2%29%2Cx%29&quot; rel=&quot;noreferrer&quot;&gt;$\nabla f(x) = 0$ [Wolfram Alpha]&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ero8x.png&quot; alt=&quot;gradient&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which has a unique solution, but many rootfinding methods can get stuck at the local minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we reformulate a new optimization problem to minimize the norm of the gradient squared, we are looking for a global minimum $x$ of &lt;a href=&quot;http://www.wolframalpha.com/input/?i=abs%5Bdiff%28x%5E2+-+exp%28-4*%28x-2%29%5E2%29%2Cx%29%5D%5E2&quot; rel=&quot;noreferrer&quot;&gt;$\lVert\nabla f(x)\rVert^2$ [Wolfram Alpha]&lt;/a&gt; which has multiple local minima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xYjTD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize, we started with an optimization problem that had a unique solution that we could guarantee that a method would find. We reformulated as a nonlinear root finding problem that had a unique solution that we could identify locally, but a rootfinding method (like Newton) might stagnate before reaching it. We then reformulated the root finding problem as an optimization problem that had multiple local solutions (no local measure can be used to identify that we are not at the global minimum).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, each time we convert a problem from optimization to rootfinding or vice-versa, we make the available methods and associated convergence guarantees weaker. The actual mechanics of the methods are often very similar so it is possible to reuse a lot of code between nonlinear solvers and optimization.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-06-15T23:00:56.550" LastActivityDate="2012-06-15T23:00:56.550" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2447" PostTypeId="2" ParentId="2444" CreationDate="2012-06-05T05:38:08.917" Score="10" Body="&lt;p&gt;Since one of my answers has been cited, I'll try to clarify why I suggested using IPOPT instead of MINPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My objections to using MINPACK have nothing to do with the algorithms that MINPACK uses and everything to do with their implementation. My main objection is that the software dates back to 1980, and was last updated in 1999. Jorge Moré is retired; I doubt he or any of the other authors of the software keep tabs on it anymore, and there's no team of people actively supporting it. The only documentation I can find on the software is the original, 1980 Argonne technical report written by Jorge Moré and the other MINPACK authors. (Chapters 1-3 can be found &lt;a href=&quot;http://www.mcs.anl.gov/~more/ANL8074a.pdf&quot;&gt;here&lt;/a&gt;, and Chapter 4 can be found &lt;a href=&quot;http://www.mcs.anl.gov/~more/ANL8074b.pdf&quot;&gt;here&lt;/a&gt;.) After searching the MINPACK source code and perusing the documentation (the PDFs are scanned images, and can't be searched), I don't see any options to accommodate constraints. Since the original poster of the nonlinear least-squares problem wanted to solve a constrained nonlinear least-squares problem, MINPACK won't even solve that problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the IPOPT mailing list, some users indicate that performance of the package on nonlinear least squares (NLS) problems is mixed relative to Levenberg-Marquardt algorithms and more specialized NLS algorithms (see &lt;a href=&quot;http://list.coin-or.org/pipermail/ipopt/2006-December/000691.html&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://list.coin-or.org/pipermail/ipopt/2005-December/000427.html&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;http://list.coin-or.org/pipermail/ipopt/2006-June/000577.html&quot;&gt;here&lt;/a&gt;). The performance of IPOPT relative to NLS algorithms, is, of course, problem dependent. Given that user feedback, IPOPT seems like a reasonable recommendation relative to NLS algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you make a good point that NLS algorithms should be investigated. I agree. I just think that a package more modern than MINPACK should be used because I believe it will perform better, be more usable, and be supported. &lt;a href=&quot;https://code.google.com/p/ceres-solver/&quot;&gt;Ceres&lt;/a&gt; seems like an interesting candidate package, but it can't handle constrained problems right now. &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/tao/index.html&quot;&gt;TAO&lt;/a&gt; would work on box-constrained least-squares problems, although it doesn't implement the classic Levenberg-Marquardt, but instead implements a derivative-free algorithm. A derivative-free algorithm would probably work well for large-scale problems, but I wouldn't use it for small-scale problems. I couldn't find any other packages that inspired a great deal of confidence in their software engineering. For instance, GALAHAD doesn't seem to use version control or any automated testing, at first glance. MINPACK doesn't seem to do those things either. If you have experience with MINPACK or recommendations regarding better software, I'm all ears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With all of that in mind, getting back to the quote of my comment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any system of equations is equivalent to an optimization problem, which is why Newton-based methods in optimization look a lot like Newton-based methods for solving systems of nonlinear equations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A better comment is probably something to the effect of:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When we want to solve a system of $n$ equations with $n$ unknowns $g(x) = 0$, we can formulate this as a least squares optimization problem. (Paraphrase of last paragraph of p.102 of &lt;em&gt;Nonlinear Programming&lt;/em&gt;, 2nd edition, by Dmitri Bertsekas.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This statement holds even for solving systems of equations under constraints. I don't know of any algorithms that are considered &quot;equation solvers&quot; for the case where there are constraints on the variables. The common approach I know of, perhaps jaundiced by several semesters of optimization courses and research in an optimization lab, is to incorporate the constraints on the system of equations into an optimization formulation. If you were to try to use the constraints in a Newton-Raphson-like scheme for equation solving, you'd probably end up with a projected gradient or projected trust-region method, much like methods used in constrained optimization.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-06-05T05:38:08.917" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2448" PostTypeId="2" ParentId="2441" CreationDate="2012-06-05T08:11:47.257" Score="3" Body="&lt;p&gt;If you delete the integer constraints and then tamper with the soultion to get an integral approximation, you are just reinventing the wheel of mixed integer linear programming (MILP). This is a highly nontrivial matter and you are unlikely to come up with anything better than most MILP solvers already implement - unless you first do a lot of research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the term ''mixed integer'' also covers the case where the number of continuous variables is zero. Because of that there is hardly any special software for the pure integer case. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-05T08:11:47.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2449" PostTypeId="1" AcceptedAnswerId="2500" CreationDate="2012-06-05T19:20:07.933" Score="4" ViewCount="81" Body="&lt;p&gt;I am studying $SU(2)$ lattice field theory, and I am attempting to use migdal recursion for renormalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main equation for Migdal recursion for my case is &#xA;$$e^{-S_p(U,\lambda a)}=\left[ \sum_r F_r(a)^{\lambda^2}d_r \chi_r (U) \right]^{\lambda^{d-2}}$$&#xA;with $d_r$ as the dimension of the representation and $\chi_r$ the trace around a plaquette of a lattice, and $U$ the product of the group matrices assigned to each link in the plaquette.  Here $\lambda$ is an expansion factor for the lattice links.  We can write &#xA;$$&#xA;e^{-S_p(U, a)}=\sum_r F_r(a)d_r \chi_r (U)&#xA;$$&#xA;with&#xA;$$F_r (a)=\frac{1}{d_r}\int dU\, e^{-S_p(U,a)}\chi^{\ast}_r (U)&#xA;$$&#xA;as the &quot;Fourier&quot; coefficients.  The trace can also be written as the sum of the eigenvalues&#xA;$$&#xA;\chi_r=\frac{\sin[(2r+1)\theta /2]}{\sin(\theta /2)}&#xA;$$&#xA;Now normalize the coefficients with $e^{-S_p(\mathbb{1},a)}=\sum_r F_r (d_r)^2$  Okay, so far I have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dim[r_]:=2r+1;&#xA;chi[r_]:=Sin[dim[r]theta/2]/Sin[theta/2];&#xA;F[r_]:=(1/dim[r])NIntegrate[Sin^2[theta/2] act chi[r],{theta,0,4 pi}];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with &lt;code&gt;act&lt;/code&gt; defined by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;act=Exp[betaf (chi[1/2]-2)+betaa(chi[1]-3)+beta3(chi[3/2]-4)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I define my new normalized coefficients with&#xA;$$&#xA;C_r(a)=F_r (a)/e^{-S_p (1,a)}&#xA;$$&#xA;so&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;norm=Sum[F[r] dim[r]^2,{r,0,9,1/2}];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c[r_]:=F[r]/norm;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now&#xA;$$&#xA;e^{-S_p(U,\lambda a)}=\left[ \sum_r \frac{F_{r}^{\lambda^2} d_r \chi_r (U)}{(\sum_r F_r (d_r)^2)^{\lambda^2}} \right]^{\lambda^{d-2}}=\left[ \sum_r \frac{\left( \frac{1}{d_r}\int dU\, e^{-S_p(U,a)}\chi^{\ast}_r (U)\right)^{\lambda^2} d_r \chi_r (U)}{\left(\sum_r \left( \frac{1}{d_r}\int dU\, e^{-S_p(U,a)}\chi^{\ast}_r (U)\right) (d_r)^2\right)^{\lambda^2}} \right]^{\lambda^{d-2}}&#xA;$$&#xA;Is this my recursion relation?  I have an initial action to start with, so I can generate subsequent $e^{S}$s.  Please ask me to clarify if you think you can help!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2012-06-12T20:53:26.887" Title="Migdal Recursion and Mathematica" Tags="|mathematica|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2450" PostTypeId="1" AcceptedAnswerId="2451" CreationDate="2012-06-05T19:25:09.423" Score="8" ViewCount="373" Body="&lt;p&gt;Pretty much the question. Given a general sparse, non-symmetric (both numerically and structurally) matrix, how important is the sparsity pattern (i.e. row/column permutation of matrix/vector) for iterative solvers? I can see it becomes important for direct solvers (LU) or preconditioners (ILU) by directly affecting the number of fill-ins. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For iterative solvers, however, it seems that the most important part is the MatVec operation which does not seem to care about the actual matrix pattern. Is there some component that could be depended on the pattern that I'm not considering here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How about in parallel? I suspect the pattern could become important in the way the matrix and vectors are distributed and thus determines the communication volume/overhead but would like to see other thoughts and inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking this both in general and also regarding PETSc's KSP solvers.&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="485" LastEditDate="2012-06-05T19:35:32.157" LastActivityDate="2012-06-05T21:12:20.803" Title="Is the sparsity pattern of a linear system important for iterative (KSP) solvers?" Tags="|linear-algebra|petsc|iterative-method|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2451" PostTypeId="2" ParentId="2450" CreationDate="2012-06-05T21:12:20.803" Score="6" Body="&lt;p&gt;Ordering is only significant in load balance/communication and suitability for preconditioning. The Krylov method does not care about the order or even whether the matrix entries are stored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, a bad ordering may require much more communication than otherwise necessary when multiplying by a matrix. See the section of the PETSc User's Manual on &quot;Natural ordering&quot; versus &quot;PETSc ordering&quot; for an example. Additionally, an ordering corresponding to a bad parallel partition might make domain decomposition preconditioners less effective. Note that it is the partition that matters here, though a partition induces an (equivalence class of) orderings.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-06-05T21:12:20.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2452" PostTypeId="2" ParentId="2421" CreationDate="2012-06-06T11:38:35.080" Score="5" Body="&lt;p&gt;As David Ketcheson suggests, LeVeque's book is a great resource; however, it naturally focuses on the Riemann-solving methods used in CLAWPACK. An alternative to these methods are the non-oscillatory central schemes (e.g. that of &lt;a href=&quot;http://www.cscamm.umd.edu/~tadmor/centralstation/KT_semi-discrete.JCP00.pdf&quot;&gt;Kurganov &amp;amp;  Tadmor&lt;/a&gt;), which are straightforward to implement and extend naturally to 2-D or 3-D (rather than using dimensional splitting). There's open-source code and large resource of papers at &lt;a href=&quot;http://www.cscamm.umd.edu/centpack/&quot;&gt;http://www.cscamm.umd.edu/centpack/&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;http://www.cscamm.umd.edu/centpack/publications/files/Liska-Wendroff_1-2D_comparison_tests_SISC2003-6-centpack.pdf&quot;&gt;paper&lt;/a&gt; by Liska &amp;amp; Wendroff (SIAM Journal on Scientific Computing 25(3), 2003, 995-1017) compares various schemes (both Riemann-solving and central) for the Euler equations in 1-D and 2-D, and has a number of test problems that may be useful.&lt;/p&gt;&#xA;" OwnerUserId="1235" LastActivityDate="2012-06-06T11:38:35.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2454" PostTypeId="1" AcceptedAnswerId="2457" CreationDate="2012-06-06T17:13:50.630" Score="3" ViewCount="1547" Body="&lt;p&gt;The library itself &lt;em&gt;compiles&lt;/em&gt; just fine with icc, but when I try to link to it (using icc for both the driver code and the linker), I get the same error that &lt;a href=&quot;https://stackoverflow.com/q/10889655/789750&quot;&gt;this question on stackoverflow&lt;/a&gt; is asking about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard that gsl is very specific to the gnu compiler. Can gsl be compiled and linked to with icc? What about other compilers, like Clang?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-01-20T09:23:39.823" Title="Can gsl be compiled with the intel C compiler?" Tags="|compiling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2455" PostTypeId="5" CreationDate="2012-06-06T17:18:19.780" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-06-06T17:18:19.780" LastActivityDate="2012-06-06T17:18:19.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2456" PostTypeId="4" CreationDate="2012-06-06T17:18:19.780" Score="0" Body="The GNU Scientific Library. A C library with a wide array of scientific functions, including linear algebra, special functions, ODE solvers, physical constants, functional transforms, numerical integration, random number generators, and more." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-06-06T18:02:20.350" LastActivityDate="2012-06-06T18:02:20.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2457" PostTypeId="2" ParentId="2454" CreationDate="2012-06-06T18:42:47.803" Score="4" Body="&lt;p&gt;We've built it with no problems with both GCC and the Intel compilers at TACC. According to ldd, libgsl.so is linked to libintlc.so which is the library that contains the SSE versions of the string routines. You might try explicitly linking to it with something like: &lt;code&gt;-L/path/to/your/intel/libraries -lintlc&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-06-06T18:42:47.803" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2458" PostTypeId="5" CreationDate="2012-06-06T19:40:38.323" Score="0" Body="&lt;p&gt;It can be shown that the energy of a given multi-electron quantum state can be expressed as a sum of a term depending only on the external potential and another which is a universal functional of the electron density. Density functional theory is the development and use of these universal functionals to calculate properties of quantum many-body systems.&lt;/p&gt;&#xA;" OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-12T13:15:15.460" LastActivityDate="2021-11-12T13:15:15.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2459" PostTypeId="4" CreationDate="2012-06-06T19:40:38.323" Score="0" Body="For questions about applying density functional theory, a method for predicting the energy and other properties of multi-electron systems (e.g. molecules and materials)" OwnerUserId="81" LastEditorUserId="23791" LastEditDate="2021-11-11T23:35:29.663" LastActivityDate="2021-11-11T23:35:29.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2460" PostTypeId="5" CreationDate="2012-06-07T09:06:40.023" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-06-07T09:06:40.023" LastActivityDate="2012-06-07T09:06:40.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2461" PostTypeId="4" CreationDate="2012-06-07T09:06:40.023" Score="0" Body="A command line plotting program with a surprising amount of depth. Unrelated to the free software foundation's GNU project." OwnerUserId="81" LastEditorUserId="81" LastEditDate="2012-06-07T09:23:54.540" LastActivityDate="2012-06-07T09:23:54.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2462" PostTypeId="1" CreationDate="2012-06-07T09:15:49.657" Score="11" ViewCount="5423" Body="&lt;p&gt;Are there any algorithms for community detection for bipartite graphs (2-mode networks) implemented in igraph, networkX, R or Python etc.? In particular, is there such an implementation in which one would be able to restrict the detection of communities just on one of the two modes?&lt;/p&gt;&#xA;" OwnerUserId="1585" LastActivityDate="2012-07-17T13:39:29.463" Title="Algorithms for community detection for bipartite graphs?" Tags="|python|graph-theory|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2463" PostTypeId="1" CreationDate="2012-06-07T14:32:12.177" Score="8" ViewCount="1052" Body="&lt;p&gt;I am faced with the problem of finding the ellipsoid $B$ ($B$ is a symmetric positive definite matrix) of maximal volume within a convex set $C$ given as a set of linear inequalities $C=\{x| a_i^T x \leq b_i, i=1,\dots,m\}$. &#xA;I understood how it is formalized as a convex optimization problem&#xA;$$&#xA;\min_{B,d}\quad[\log\det B^{-1}]\\&#xA;\mbox{s.t.:}\quad ||Ba_i||_2+a_i^Td\leq b_i, \qquad i=1,\dots, m&#xA;$$&#xA;as is given in &quot;Convex Optimization,Stephen Boyd and Lieven Vandenberghe,&#xA;Cambridge University Press, 2004&quot; &lt;a href=&quot;http://www.stanford.edu/~boyd/cvxbook/&quot;&gt;[pdf version]&lt;/a&gt;.&#xA;My approach would be to use interior point methods, introduce an accuracy parameter $t&amp;gt;0$ and incorporate the constrains into the objective via a logarithmic barrier function as explained in chapter 11 of the above book and try to minimize the resulting uncontrained problem&#xA;$$&#xA;\min_{B,d}\quad \underbrace{\left[\log\det B^{-1} - \frac{1}{t}\sum_{i=1}^m\log(b_i-||Ba_i||_2-a_i^Td)\right]}_{= f(B,d)}.&#xA;$$&#xA;Therefore I would take partial derivatives of $f$:&#xA;$$&#xA;\frac{\partial f}{\partial B} = B^{-1}+\frac{1}{t}\sum_{i=1}^m\left(\frac{\frac{Ba_ia_i^T}{||Ba_i||}}{b_i-||Ba_i||_2-a_i^Td}\right)&#xA;$$&#xA;which is a matrix and &#xA;$$&#xA;\frac{\partial f}{\partial d}=\frac{1}{t}\sum_{i=1}^m\left(\frac{a_i}{b_i-||Ba_i||_2-a_i^Td}\right)&#xA;$$&#xA;which is a vector. And then starting from an initial (feasible) point $(B^0,d^0)$ I would iteratively update the actual solution $(B^k,d^k)$ according the negative partial derivates:&#xA;$$&#xA;B^{k+1} = B^k - s_B \frac{\partial f(B^k,d^k)}{\partial B}\\&#xA;d^{k+1} = d^k - s_d \frac{\partial f(B^k,d^k)}{\partial d}&#xA;$$&#xA;where $s_B&amp;gt;0$ and $s_d&amp;gt;0$ are step size parameters until a predefined stoping criterum is fullfilled.\&#xA;I am not sure whether this is a correct way to solve the problem? It seems to me very awkward and not very elegant. I am not an expert in optimization techniques and I am not sure whether I put all the ingredients (partial derivatives, interior-point-method, unconstrained minization, etc.) together in the right way. I wonder how an expert would solve this problem. In the above mentioned book this task was shown as an example for a convex problem, but as far as I can see there was so explicit algorithm given for solving the task. Although I think Mr. Boyd has somewhere a Matlab-script on his pages for solving the task, but I want to understand the basic techniques first before using a &quot;black-box&quot;-algorithm.&#xA;There seem to be other approaches in &quot;Interior-Point Polynomial Algorithms in Convex Programming; Yurii Nesterov and Arkadii Nemirovskii, SIAM studies in applied mathematics; vol.13, 1994&quot; and &quot;On the complexity of approximating the maximal inscribed ellipsoid for a polytope, Leonid G. Khachiyan and Michael J. Todd, Mathematical Programming 61 (1993), 137-159&quot;  but I don't understand them because they are written to technical for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way: How does the dual problem of the first problem look like? And how is it derived?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="1586" LastActivityDate="2012-06-26T17:03:08.297" Title="How to calculate the maximal ellipsoid in a given polyhedron" Tags="|optimization|convex-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2464" PostTypeId="1" AcceptedAnswerId="2465" CreationDate="2012-06-07T19:19:39.890" Score="7" ViewCount="266" Body="&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I was advised to replace the question with a more specific one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming from a very theoretical background, I'm pretty ignorant about what practical &lt;strong&gt;matrix solvers&lt;/strong&gt; exist. (I have been, and will continue to scour the web for information, but I figured I would get direct and concise answers here as well.)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Currently, what are the most important matrix solvers? (implemented in any software)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have a rudimentary programming background, and I'm familiar with classical theoretical methods for solving matrix systems. However, I'm well aware computer scientists have been at work finding ingenious ways to solve larger systems more quickly, especially for specialized classes of matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking at applications of matrix solvers in industry, in particular for use in simulation programs.&lt;/p&gt;&#xA;" OwnerUserId="1588" LastEditorUserId="1588" LastEditDate="2012-06-07T20:18:10.000" LastActivityDate="2012-06-08T22:08:41.137" Title="Wanting to learn about matrix solvers" Tags="|matrix|reference-request|linear-solver|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2465" PostTypeId="2" ParentId="2464" CreationDate="2012-06-07T20:13:21.853" Score="8" Body="&lt;p&gt;The best high-level overview that I know of is &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/text.html&quot; rel=&quot;nofollow&quot;&gt;Trefethen and Bau&lt;/a&gt;. If I had to boil it down to a list, it would be (somewhat in pedagogical order):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Dense $QR$ factorization&lt;/li&gt;&#xA;&lt;li&gt;Dense symmetric/Hermitian Eigenvalue Decomposition (EVD)&lt;/li&gt;&#xA;&lt;li&gt;Dense Singular Value Decomposition (SVD)&lt;/li&gt;&#xA;&lt;li&gt;The Conjugate Gradient Method (CG)&lt;/li&gt;&#xA;&lt;li&gt;Generalize Minimum Residual method (GMRES)&lt;/li&gt;&#xA;&lt;li&gt;Sparse Cholesky and $LU$ factorization&lt;/li&gt;&#xA;&lt;li&gt;&quot;Fast&quot; methods, such as multigrid, the Fast Fourier Transform (FFT), and the Fast Multipole Method (FMM)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-06-07T21:59:01.940" LastActivityDate="2012-06-07T21:59:01.940" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2466" PostTypeId="1" CreationDate="2012-06-07T21:15:44.460" Score="10" ViewCount="3542" Body="&lt;p&gt;At the moment I am working on a differential equation solving method called basis-spline collocation. What I am having trouble with is building a method to build an arbitrary order spline, with the relationship &#xA;$$&#xA;B^{k+1}_{i}(x)= \frac{x-x_i}{x_{k+i}-x_i}B^k_i + \frac{x_{k+i+1}-x}{x_{k+i+1}-x_{i+1}}B^k_{i+1}(x) &#xA;$$&#xA;with the initial condition&#xA;$$B^1_i(x)=\begin{cases}&#xA;            1 &amp;amp; \; \text{for } \; x_i \leq x &amp;lt; x_{i+1} \\&#xA;            0 &amp;amp; \; \text{otherwise}&#xA;        \end{cases}$$&#xA;and I am having trouble even starting with this problem, as it is recursive on could start from either the &quot;top&quot; or the &quot;bottom&quot;, and I am running into a general writers block type of thing, where I can't get my mind around what I'm needing to do.&lt;/p&gt;&#xA;" OwnerUserId="1590" LastEditorUserId="127" LastEditDate="2013-05-23T17:59:25.943" LastActivityDate="2013-05-23T18:03:34.133" Title="How to build a recursive spline function in C++" Tags="|c++|b-spline|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2467" PostTypeId="1" AcceptedAnswerId="2479" CreationDate="2012-06-08T00:58:10.747" Score="16" ViewCount="4692" Body="&lt;p&gt;Single precision floating point numbers take up half the memory and on modern machines (even on GPUs it seems) operations can be done with them at almost twice the speed compared to double precision. Many FDTD codes that I have found exclusively use single precision arithmetic and storage. Is there a rule of thumb of when it is acceptable to use single precision for solving large-scale sparse systems of equations? I assume it must heavily depend on the matrix condition number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, is there any effective technique which uses double precision where necessary and single where the accuracy of double is not required. For instance, I would think that for a matrix vector multiplication or a vector dot product, it might be a good idea to accumulate the results in a double precision variable (to avoid cancellation error), but that individual entries to be multiplied with each other can be multiplied using single precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do modern FPU's seamlessly allow conversion from single precision (float) to double precision (double) and vice versa? Or are these costly operations?&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="149" LastEditDate="2012-06-08T10:28:15.893" LastActivityDate="2012-06-08T22:04:01.937" Title="Single versus double floating-point precision" Tags="|matrix|linear-solver|precision|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2468" PostTypeId="1" AcceptedAnswerId="2471" CreationDate="2012-06-08T11:00:33.493" Score="12" ViewCount="526" Body="&lt;p&gt;I am interested in setting up calculations to check if a distance criterion is satisfied: that is, the distance between a vector ${\bf x}_i$ and anther vector ${\bf x}_j$ should be less than some value $r_{\rm max}$. My data is partitioned according to an orthogonal grid of coordinates. Since my cutoff is smaller than the distance between the endpoints of nearest-neighbor coordinates, I'd like to add an &quot;octant&quot; variable to check if things are set up correctly:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;if octant[j] in allowed_list continue&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as a &quot;short-circuit&quot; to &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;if dist(x[i], x[j]) &amp;lt; r_max&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is: how efficient computationally are boolean lookups and comparisons relative to floating-point operations? Is this worth doing on modern architectures?&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="276" LastEditDate="2012-06-08T11:49:49.073" LastActivityDate="2012-06-08T14:51:04.550" Title="Costs of lookups versus calculations" Tags="|efficiency|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2469" PostTypeId="1" AcceptedAnswerId="2470" CreationDate="2012-06-08T11:35:03.473" Score="28" ViewCount="57026" Body="&lt;p&gt;Related question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/559/state-of-the-mac-os-in-scientific-computing-and-hpc&quot;&gt;State of the Mac OS in Scientific Computing and HPC&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A significant number of software packages in computational science are written in Fortran, and Fortran isn't going away. A Fortran compiler is also required to build other software packages (one notable example being &lt;a href=&quot;https://www.scipy.org/Installing_SciPy/Mac_OS_X&quot; rel=&quot;noreferrer&quot;&gt;SciPy&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, Mac OS X does not include a Fortran compiler. How should I install a Fortran compiler on my machine?&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-01-07T17:31:44.900" Title="How should I install a Fortran compiler on a Mac? (OS X 10.x, x &gt;= 4)" Tags="|software|fortran|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2470" PostTypeId="2" ParentId="2469" CreationDate="2012-06-08T11:35:03.787" Score="35" Body="&lt;p&gt;Pick your poison. I recommend using Homebrew. I have tried all of these methods except for &quot;Fink&quot; and &quot;Other Methods&quot;. Originally, I preferred MacPorts when I wrote this answer. In the two years since, Homebrew has grown a lot as a project and has proved more maintainable than MacPorts, which can require a lot of &lt;code&gt;PATH&lt;/code&gt; hacking.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Installing a version that matches system compilers&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you want the version of gfortran to match the versions of gcc, g++, etc. installed on your machine, download the appropriate version of gfortran from &lt;a href=&quot;http://r.research.att.com/tools/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. The &lt;a href=&quot;http://cran.r-project.org/bin/macosx/RMacOSX-FAQ.html#Quick-start&quot; rel=&quot;noreferrer&quot;&gt;R developers&lt;/a&gt; and &lt;a href=&quot;https://www.scipy.org/Installing_SciPy/Mac_OS_X&quot; rel=&quot;noreferrer&quot;&gt;SciPy developers&lt;/a&gt; recommend this method.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Advantages&lt;/strong&gt;: Matches versions of compilers installed with XCode or with &lt;a href=&quot;http://kennethreitz.com/xcode-gcc-and-homebrew.html&quot; rel=&quot;noreferrer&quot;&gt;Kenneth Reitz's installer&lt;/a&gt;; unlikely to interfere with OS upgrades; coexists nicely with MacPorts (and probably Fink and Homebrew) because it installs to &lt;code&gt;/usr/bin&lt;/code&gt;. Doesn't clobber existing compilers. Don't need to edit &lt;code&gt;PATH&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Disadvantages&lt;/strong&gt;: Compiler stack will be really old. (GCC 4.2.1 is the latest Apple compiler; it was released in 2007.) Installs to &lt;code&gt;/usr/bin&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Installing a precompiled, up-to-date binary from HPC Mac OS X&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hpc.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;HPC Mac OS X&lt;/a&gt; has binaries for the latest release of GCC (at the time of this writing, 4.8.0 (experimental)), as well as g77 binaries, and an f2c-based compiler. The PETSc developers recommend this method on their &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/faq.html#gfortran&quot; rel=&quot;noreferrer&quot;&gt;FAQ&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Advantages&lt;/strong&gt;: With the right command, installs in &lt;code&gt;/usr/local&lt;/code&gt;; up-to-date. Doesn't clobber existing system compilers, or the approach above. Won't interfere with OS upgrades.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Disadvantages&lt;/strong&gt;: Need to edit &lt;code&gt;PATH&lt;/code&gt;. No easy way to switch between versions. (You could modify the PATH, delete the compiler install, or kludge around it.) Will clobber other methods of installing compilers in &lt;code&gt;/usr/local&lt;/code&gt; because compiler binaries are simply named 'gcc', 'g++', etc. (without a version number, and without any symlinks).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Use MacPorts&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.macports.org/&quot; rel=&quot;noreferrer&quot;&gt;MacPorts&lt;/a&gt; has a number of versions of compilers available for use.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Advantages&lt;/strong&gt;: Installs in &lt;code&gt;/opt/local&lt;/code&gt;; &lt;code&gt;port select&lt;/code&gt; can be used to switch among compiler versions (including system compilers). Won't interfere with OS upgrades.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Disadvantages&lt;/strong&gt;: Installing ports tends to require an entire &quot;software ecosystem&quot;. Compilers don't include debugging symbols, which can pose a problem when using a debugger, or installing PETSc. (&lt;a href=&quot;https://bitbucket.org/seanfarley/scienceports/wiki/Installation&quot; rel=&quot;noreferrer&quot;&gt;Sean Farley&lt;/a&gt; proposes some workarounds.) Also requires changing &lt;code&gt;PATH&lt;/code&gt;. Could interfere with Homebrew and Fink installs. (See &lt;a href=&quot;https://superuser.com/questions/181337/is-it-safe-to-install-both-homebrew-and-macports-on-the-same-machine&quot;&gt;this post on SuperUser&lt;/a&gt;.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Use Homebrew&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mxcl/homebrew&quot; rel=&quot;noreferrer&quot;&gt;Homebrew&lt;/a&gt; can also be used to install a Fortran compiler.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Advantages&lt;/strong&gt;: Easy to use package manager; installs the same Fortran compiler as in &quot;Installing a version that matches system compilers&quot;. Only install what you need (in contrast to MacPorts). Could install a newer GCC (4.7.0) stack using the alternate repository homebrew-dupes. &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Disadvantages&lt;/strong&gt;: Inherits all the disadvantages from &quot;Installing a version that matches system compilers&quot;. May need to follow the Homebrew paradigm when installing other (non-Homebrew) software to &lt;code&gt;/usr/local&lt;/code&gt; to avoid messing anything up. Could interfere with MacPorts and Fink installs. (See &lt;a href=&quot;https://superuser.com/questions/181337/is-it-safe-to-install-both-homebrew-and-macports-on-the-same-machine&quot;&gt;this post on SuperUser&lt;/a&gt;.) Need to change &lt;code&gt;PATH&lt;/code&gt;. Installs could depend on system libraries, meaning that dependencies for Homebrew packages could break on an OS upgrade. (See &lt;a href=&quot;http://terrychay.com/article/macports-vs-homebrew.shtml&quot; rel=&quot;noreferrer&quot;&gt;this article&lt;/a&gt;.) I wouldn't expect there to be system library dependencies when installing gfortran, but there could be such dependencies when installing other Homebrew packages.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Use Fink&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In theory, you can use &lt;a href=&quot;http://www.finkproject.org/&quot; rel=&quot;noreferrer&quot;&gt;Fink&lt;/a&gt; to install gfortran. I haven't used it, and I don't know anyone who has (and was willing to say something positive).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Other methods&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Other binaries and links are listed on the &lt;a href=&quot;http://gcc.gnu.org/wiki/GFortranBinaries#MacOS&quot; rel=&quot;noreferrer&quot;&gt;GFortran wiki&lt;/a&gt;. Some of the links are already listed above. The remaining installation methods may or may not conflict with those described above; use at your own risk.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-03-20T10:18:34.427" LastActivityDate="2014-01-07T17:31:44.900" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2471" PostTypeId="2" ParentId="2468" CreationDate="2012-06-08T12:53:23.907" Score="15" Body="&lt;p&gt;My rule of thumb is that if you can compute some quantity efficiently (good utilization of the FPU) in less than 50 flops per double precision value, it's better to recompute than to store. The trend, which has been steady for decades, is for floating point capability to improve faster than memory performance, and is not likely to relent due to physical constraints and energy requirements of fast memory. The value of 50 is of the right magnitude for all popular computing platforms (Intel/AMD, Blue Gene, and GPUs).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Approximate cost estimates per core&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[guidelines for 2011/2012 Intel- and AMD-based machines]&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$0.05$ ns: time to do one double precision floating point operation as part of a vectorized code without data dependencies and interleaved multiply/add&lt;/li&gt;&#xA;&lt;li&gt;$0.2$ ns: time to do one non-vectorized floating point operation without vectorization or data dependencies&lt;/li&gt;&#xA;&lt;li&gt;$0.4$ ns: time to do one non-vectorized floating point operation without vectorization or data dependencies, but without interleaved multiply/add (1 clock cycle)&lt;/li&gt;&#xA;&lt;li&gt;$0.4$ to $0.8$ ns: latency to reference L1 cache&lt;/li&gt;&#xA;&lt;li&gt;$2$ ns: latency of a floating point operation (vectorized or not)&lt;/li&gt;&#xA;&lt;li&gt;$3$ to $5$ ns: time to load one double precision value from memory as part of perfectly prefetched and fully utilized stream&lt;/li&gt;&#xA;&lt;li&gt;$3$ to $5$ ns: indirect function call (virtual method or function pointer, without register pressure)&lt;/li&gt;&#xA;&lt;li&gt;$5$ ns: conditional branch mis-predict&lt;/li&gt;&#xA;&lt;li&gt;$4$ to $8$ ns: time for one division (vectorized or not, cannot run concurrently with other other instructions)&lt;/li&gt;&#xA;&lt;li&gt;$12$ ns: L1 cache miss satisfied from local L2&lt;/li&gt;&#xA;&lt;li&gt;$12$ ns: best case compare-and-swap or fetch-and-add atomic instruction&lt;/li&gt;&#xA;&lt;li&gt;$30$-$50$ ns: L1 write cache miss or atomic instruction in which the cache line is available locally, but the current core has to obtain exclusive access&lt;/li&gt;&#xA;&lt;li&gt;$100$ ns: cache miss or atomic instruction satisfied from off-socket or memory&lt;/li&gt;&#xA;&lt;li&gt;$10^3$ ns ($1$ $\mu$s): hardware network latency for a fancy network&lt;/li&gt;&#xA;&lt;li&gt;$10^4$ ns ($10$ $\mu$s): low-data neighbor exchange when well-mapped to the network&lt;/li&gt;&#xA;&lt;li&gt;$10^6$ ns (1 ms): time (per process) to create a file on a parallel file system&lt;/li&gt;&#xA;&lt;li&gt;$2\cdot 10^6$ ns (2 ms): small data &lt;code&gt;MPI_Allreduce&lt;/code&gt; (e.g. a norm or dot product) on a large machine&lt;/li&gt;&#xA;&lt;li&gt;$10^7$ ns (10 ms): local disk seek&lt;/li&gt;&#xA;&lt;li&gt;$5\cdot 10^8$ ns (500 ms): time to rewrite all available memory&lt;/li&gt;&#xA;&lt;li&gt;$1.8\cdot 10^{12}$ ns (0.5 hour): time to checkpoint entire machine state to disk&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.agner.org/optimize/instruction_tables.pdf&quot;&gt;Agner Fog's instruction tables&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gist.github.com/2841832&quot;&gt;Latency Numbers Every Programmer Should Know&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www2.rdrop.com/users/paulmck/scalability/paper/Validation.2012.04.04a.pdf&quot;&gt;Paul McKenney slides on threading&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-06-08T14:51:04.550" LastActivityDate="2012-06-08T14:51:04.550" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2472" PostTypeId="2" ParentId="2467" CreationDate="2012-06-08T13:14:15.067" Score="3" Body="&lt;p&gt;My advice would be to focus mainly on the memory consumption for the decision when to use single precision (float). So the bulk data for a FDTD computation should use float, but I would keep the problem description itself (like geometry, material parameters, excitation conditions) and all related meta data in double.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would keep all performance uncritical and not deeply analyzed computations in double. Especially, I would keep polygonal data and other description of geometry in double (perhaps even integer if possible), since experience tells that you will never get the computational geometric parts of your code fully robust, even if it would be possible in theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third part I would keep in double are analytic computations, including shortcuts using non-symmetric eigenvalue decompositions. As an example, I have a piecewise defined rotational symmetric 2D functions, and I need its Fourier transform. There would be various numerical ways involving FFTs and appropriate &quot;analytical low pass filters&quot; to get it &quot;efficiently&quot;. Because it's performance uncritical, I used an &quot;exact&quot; analytical expression involving Bessel functions instead. Since this was a shortcut to begin with, and I won't spend any time analyzing the error propagation of my complicated formula, I better use double precision for that computation. (It still turned out that only some of the analytical equivalent expressions for the formula were usable, because some lost accuracy too quick even with double precision).&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-06-08T13:14:15.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2473" PostTypeId="2" ParentId="2464" CreationDate="2012-06-08T13:30:35.417" Score="4" Body="&lt;p&gt;I second Jack Poulson's recommendation, but also want to recommend Yousef Saad's Iterative Methods for Sparse Linear Systems (available online, see &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/books.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main advantage of Saad is its discussion of the very important topic of preconditioning, something Trefethen and Bau does not touch.  It also has pseudocode implementations of the standard algorithms, and talks about implementation details.&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2012-06-08T13:30:35.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2474" PostTypeId="1" CreationDate="2012-06-08T16:10:36.800" Score="3" ViewCount="666" Body="&lt;p&gt;I am fitting a curve to some instrument data. The data is a pulse with a particular functional form, which starts from and returns to a constant (with noise) baseline level before and after the pulse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find that if I fit the whole dataset, I get one set of fitted parameters for the pulse, but if I crop out the part of the data containing the pulse and fit only that, I get a different set of fitted parameters. In both cases the fit looks good and the parameters are reasonable, but not equivalent at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this represent a problem in my fit function or methodology in particular? Or is this something which is generally possible and expected?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the problem isn't with my fit function in particular, then how might I improve my fitting method to make the results correct, and invariant w.r.t. the amount of baseline samples present in the data set?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using &lt;code&gt;lsqcurvefit()&lt;/code&gt; in Matlab, with an error function that provides an analytical Jacobian, so the optimizer being used under the hood is a Trust-Region method. I'm not completely familiar with this algorithm, but as far as I know it is in the same category as BFGS or other quasi-Newton methods, with some additional magic to improve robustness and efficiency. I get similar results with other solvers (eg. Conjugate Gradient).&lt;/p&gt;&#xA;" OwnerUserId="22" LastActivityDate="2012-06-08T17:10:39.330" Title="When fitting a Gaussian-like function, how does the amount of baseline datapoints affect the fit?" Tags="|regression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2475" PostTypeId="2" ParentId="2474" CreationDate="2012-06-08T16:43:10.780" Score="1" Body="&lt;p&gt;If you fit only data from a window around the peak, then what you're implicitly saying is that data points outside the window don't matter. It would thus not be a surprise if the curve doesn't describe points outside the window well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you don't use a window, you say that &lt;em&gt;every&lt;/em&gt; point matters equally much, and so the curve has to fit &lt;em&gt;all&lt;/em&gt; points. In particular, this means that if you have a &lt;em&gt;lot&lt;/em&gt; of points, far more than are in your window, then you might end up with a curve that fits the vast majority of points on the fringes, but provides little fit for the points you really care about, i.e., the ones around the peak.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fundamentally, what you want to do is more of a philosophical question than a mathematical one: which data points do you want the curve to describe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, from what you describe the curve you get actually &lt;em&gt;does&lt;/em&gt; fit all data points, in either case. That means at the least that the model you try to fit to the data points is an accurate representation of the true process that produced these data points. The question then becomes: if you get two different curves from your two approaches, which one is better? That, again, is a question of interpretation: what is it that you care about? Obviously, for the points inside the window, the curve created with the window is the best fit. But, how good is the fit of the curve generated from the window with regards to the points outside the window? And how good is the globally fitted curve for the points inside the window? The standard tool to consider for these questions is to look at the standard deviation; i.e., if the data points are $(t_i,x_i)$ and you fit a curve $f(t)$ to them, the standard deviation would be $\sqrt{\frac 1N \sum_i (x_i-f(t_i))^2}$. It may be that with this measure, both curves are approximately the same, in which case they'd be equally good approximations. Or they're not, on a given subset of points (e.g. all, or only the ones inside the window), in which case you'd know which curve is &quot;better&quot; for your purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll find a lot of methods in this regard in books on parameter estimation and statistical fitting. Among the more important ones are those that fit using only a subset of points and then compare by computing the residual for the remainder of the points. If you repeat this procedure with several, different subsets, you'll get a feeling for how sensitive the fitted curve is to the choice of measurement points.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-08T16:43:10.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2476" PostTypeId="2" ParentId="2474" CreationDate="2012-06-08T17:10:39.330" Score="3" Body="&lt;p&gt;If very different parameter sets give quite similar curves, you are most likely fitting too many parameters, which means that some of the fitted parameters will be sensitive to changes in the data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look at the inverse Hessian of the objective function of the two fits to see how well the estimated parameters are determined by the data. Large diagonal entires imply poor conditioning, and hence unreliability of the parameters (even though the fit is good).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is harmless if you only want to use the smoothed curve to replace your data set. But it is disastrous if you want to treat the parameters as representing substantial information. In this case you need to add a regularization term reflecting qualitative knowledge about your parameters to improve the quaity of your parameter estimates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See my paper ''Solving ill-conditioned and singular linear systems: A tutorial on regularization'' &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/papers.html#regtutorial&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/papers.html#regtutorial&lt;/a&gt;&#xA;for regularization in fitting by linear least squares, but qualitatively everything said there also applies to nonlinear objectives.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-08T17:10:39.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2477" PostTypeId="2" ParentId="2467" CreationDate="2012-06-08T20:21:13.163" Score="5" Body="&lt;p&gt;A nice paper on this topic is &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0010465508003846&quot; rel=&quot;noreferrer&quot;&gt;Accelerating scientific computations with mixed precision algorithms&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-06-08T20:21:13.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2478" PostTypeId="2" ParentId="2466" CreationDate="2012-06-08T20:34:06.330" Score="8" Body="&lt;p&gt;I can recommend consulting &lt;a href=&quot;http://dx.doi.org/10.1007/978-3-642-97385-7&quot; rel=&quot;nofollow&quot;&gt;The NURBS book&lt;/a&gt;, which seems to be a classic text on this subject. The algorithm itself is given on &lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;id=7dqY5dyAwWkC&amp;amp;pg=PA72&quot; rel=&quot;nofollow&quot;&gt;page 72&lt;/a&gt;, it is available for online viewing.&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="127" LastEditDate="2013-05-23T18:03:34.133" LastActivityDate="2013-05-23T18:03:34.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2479" PostTypeId="2" ParentId="2467" CreationDate="2012-06-08T22:04:01.937" Score="9" Body="&lt;p&gt;For all non-trivial problems (i.e., for those where performance matters) almost all of the memory you have will be in the matrix, and relatively little in vectors. For example, for 3d Taylor-Hood elements for the Stokes equation, you have a few hundred elements per row in the matrix, and this vastly outweighs the amount of memory needed for vectors. We have thus played with the idea of storing the matrix as floats and the vectors as doubles. I don't recall our timing results but I know for sure that we haven't seen any problems with round-off etc. So this approach definitely works.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-08T22:04:01.937" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2480" PostTypeId="2" ParentId="2464" CreationDate="2012-06-08T22:08:41.137" Score="2" Body="&lt;p&gt;From a practical viewpoint, and in addition to the texts referenced in the other answers, there's the book that everyone just calls the &quot;&lt;a href=&quot;http://www.netlib.org/linalg/html_templates/Templates.html&quot; rel=&quot;nofollow&quot;&gt;Templates book&lt;/a&gt;&quot;. It has no theory, but it tells you a bit about how to implement iterative solvers.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-08T22:08:41.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2481" PostTypeId="2" ParentId="285" CreationDate="2012-06-09T07:07:30.783" Score="5" Body="&lt;p&gt;There's no need to use transfer to emulate &lt;code&gt;void *&lt;/code&gt; in a modern Fortran code.  Instead, just use the &lt;a href=&quot;http://gcc.gnu.org/onlinedocs/gfortran/ISO_005fC_005fBINDING.html&quot;&gt;ISO_C_BINDING&lt;/a&gt; intrinsic module, which is supported by all mainstream Fortran compilers.  This module makes it very simple to interface between Fortran and C, with some very minor caveats.  One can use the &lt;code&gt;C_LOC&lt;/code&gt; and &lt;code&gt;C_FUNLOC&lt;/code&gt; functions to get C pointers to Fortran data and procedures, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regard to the PETSC example above, I assume the context is typically a pointer to a user-defined structure, which is equivalent to a derived data type in Fortran.  That shouldn't be a problem to deal with using &lt;code&gt;C_LOC&lt;/code&gt;.  The opaque TSIFunction handle is also very simple to deal with: just use the ISO_C_BINDING data type &lt;code&gt;c_ptr&lt;/code&gt;, which is equivalent to &lt;code&gt;void *&lt;/code&gt; in C.  A library written in Fortran can use &lt;code&gt;c_ptr&lt;/code&gt; if it needs to work around modern Fortran's strict type checking.&lt;/p&gt;&#xA;" OwnerUserId="1598" LastActivityDate="2012-06-09T07:07:30.783" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2482" PostTypeId="1" AcceptedAnswerId="2483" CreationDate="2012-06-09T10:37:31.170" Score="5" ViewCount="562" Body="&lt;p&gt;A very basic question but i couldn't find another post about it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $p$ non parallel hyper-plane in $\mathbb{R}^p$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left(\begin{array}{cccc}&#xA;c_{11} &amp;amp; a_{11} &amp;amp; .... &amp;amp; a_{1p} \\&#xA;... &amp;amp; ... &amp;amp; ... &amp;amp; .... \\&#xA;c_{p1} &amp;amp; a_{p1} &amp;amp; .... &amp;amp; a_{pp} \end{array} \right)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$||a_{i.}||=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best (from a numerical standpoint of view) &#xA;way to get the $p$ vector of coordinates $x$ of their &#xA;intersection?&lt;/p&gt;&#xA;" OwnerUserId="479" LastEditorUserId="549" LastEditDate="2012-06-09T11:48:04.813" LastActivityDate="2012-06-09T11:48:04.813" Title="Intersection of hyperplanes" Tags="|linear-algebra|performance|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2483" PostTypeId="2" ParentId="2482" CreationDate="2012-06-09T11:47:11.647" Score="7" Body="&lt;p&gt;If the matrix A is non-singular, the intersection of the hyperplanes is simply the the solution of the linear system of equations $Ax=b$, where  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A= \left(\begin{array}{cccc}&#xA;a_{11} &amp;amp; .... &amp;amp; a_{1p} \\&#xA;... &amp;amp; ... &amp;amp; .... \\&#xA;a_{p1} &amp;amp; .... &amp;amp; a_{pp} \end{array} \right)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b= -\left(\begin{array}{cccc}&#xA;c_{11} \\&#xA;... \\&#xA;c_{p1} \end{array} \right)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;best&quot; way to solve this system largely depends on the matrix A itself (see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2068/how-do-the-properties-of-a-matrix-affect-the-linear-system-solving&quot;&gt;this&lt;/a&gt; , &lt;a href=&quot;https://scicomp.stackexchange.com/questions/81/what-guidelines-should-i-follow-when-choosing-a-sparse-linear-system-solver&quot;&gt;this&lt;/a&gt;, or &lt;a href=&quot;https://scicomp.stackexchange.com/questions/513/why-is-my-iterative-linear-solver-not-converging&quot;&gt;this&lt;/a&gt; for more information on a choice of solver).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case when A is singular, we cannot describe the intersection as a single point.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-06-09T11:47:11.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2484" PostTypeId="1" AcceptedAnswerId="2485" CreationDate="2012-06-10T03:37:28.000" Score="4" ViewCount="382" Body="&lt;p&gt;Why does DMDA objects require so much memory on PETSc 3.2-p7. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When running the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  ...&#xA;  N = 8000;&#xA;  ierr = DMDACreate2d(PETSC_COMM_WORLD, DMDA_BOUNDARY_PERIODIC, DMDA_BOUNDARY_PERIODIC, &#xA;            DMDA_STENCIL_STAR, N, N, size, PETSC_DECIDE, PETSC_DECIDE, &#xA;            0, PETSC_NULL, PETSC_NULL, &amp;amp;da);&#xA;  //ierr = DMCreateGlobalVector(da, &amp;amp;x); CHKERRQ(ierr);&#xA;  ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;using the command &lt;strong&gt;./prog -log_summary&lt;/strong&gt;, I get the following memory usage summary:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Distributed Mesh: 512004856 bytes&#xA;   Vector: 2944 bytes&#xA;   Vector Scatter: 1256 bytes&#xA;   Index Set: 256003752 bytes&#xA;   IS L to G Mapping: 256000572 bytes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By uncommenting the DMCreateGlobalVector line, the vector size in the memory summary increases to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Vector: 512004384&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It appears that DMDA objects have a size proportional to the vector data size. According to the documentation, the data is not stored in the Distributed Mesh. Therefore, why does the DMDA object require this much memory? Is there a way to reduce the memory overhead used by DMDA?&lt;/p&gt;&#xA;" OwnerUserId="1031" LastActivityDate="2012-06-10T04:23:36.463" Title="Memory footprint for DMDA objects in PETSc" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2485" PostTypeId="2" ParentId="2484" CreationDate="2012-06-10T04:23:36.463" Score="3" Body="&lt;p&gt;That storage is in the scatters and local-to-global maps. The DMDA also creates (and destroys) a vector during setup; it likely doesn't count toward peak memory usage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have an application where this limits your capability in any way or is it an academic interest? There are normally a few vectors in a simulation (and for some applications, a matrix) so the DMDA memory usage is fairly insignificant and worth it in exchange for the faster data structures.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-06-10T04:23:36.463" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2486" PostTypeId="1" AcceptedAnswerId="2487" CreationDate="2012-06-10T15:28:13.793" Score="19" ViewCount="2777" Body="&lt;p&gt;I want to develop a parallel scientific computation software from scratch. I want some thoughts on which language to start. The program involves reading/writing data to txt files and doing heavy computations in parallel, with many LU factorizations and the use of sparse linear solvers. The candidate solutions I was thinking are Fortran 2003/2008 with OpenMP or co-array, C++ with openmp cilk+ or TBB, python. Any other, documented, suggestions are welcome! I know very well C, Fortran and Java (in that order). I've done some scripting in python but basic stuff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know fortran is very fast, but, hard to maintain and parallelize. C++ is said to be slow unless you use external libraries etc Python I like, but is it realistic to write a full scale, industrial level software upon?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The software needs to be able to handle big amounts of data and be effective with scientific computations. The performance is of the essence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the background, I already have a working software written in Fortran. Many people were involved in development over many years and the code is really dirty. Maintaining and parallelizing the code has proved a nightmare and I'm thinking of alternatives.&lt;/p&gt;&#xA;" OwnerUserId="1602" LastEditorUserId="1602" LastEditDate="2020-02-21T19:56:49.693" LastActivityDate="2020-02-21T19:56:49.693" Title="Parallel Scientific Computation Software Development Language?" Tags="|python|hpc|c++|fortran|simulation|" AnswerCount="5" CommentCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="2487" PostTypeId="2" ParentId="2486" CreationDate="2012-06-10T17:36:15.960" Score="19" Body="&lt;p&gt;Let me try and break down your requirements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Maintainability&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Reading/writing text data&lt;/li&gt;&#xA;&lt;li&gt;Strong interfaces/capability for LU factorizations&lt;/li&gt;&#xA;&lt;li&gt;Sparse linear solvers&lt;/li&gt;&#xA;&lt;li&gt;Performance and scalability to large data&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From this list, I would consider the following languages:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C, C++, Fortran, Python, MATLAB, Java&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Julia is a promising new language, but the community is still forming around it and it has not been deployed in any major new codes.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Reading/writing text data&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This is easy to get right in any programming language.  Make sure you are appropriately buffering and coalescing your I/O access, and you will get good performance from any of the languages you should consider.  Avoid the stream objects in C++ unless you know how to use them performantly.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Strong interfaces/capability for LU factorizations&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you are performing dense LU factorizations, you will want to use LAPACK, or ScaLAPACK/Elemental for parallel functionality.  LAPACK and ScaLAPACK are written in Fortran, Elemental is written in C++.  All three libraries are performant and well-supported and documented.  You can interface into them from any of the languages you should consider.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Sparse linear solvers&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The premier freely available sparse linear solvers are &lt;a href=&quot;http://www.mcs.anl.gov/petsc/miscellaneous/external.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;almost all available&lt;/a&gt; through &lt;a href=&quot;http://www.mcs.anl.gov/petsc/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc&lt;/a&gt;, written in C, which is well-documented and supported.  You can interface into PETSc from any of the languages you should consider.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Performance and scalability to large data&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The only parallel programming paradigms you mention are shared memory based, which means you are not considering an MPI-based (message-passing), distributed-memory computing approach.  In my experience, it is much easier to write code that scales well beyond a dozen cores using a distributed-memory solution.  Almost all University &quot;clusters&quot; are MPI-based these days, large shared-memory machines are expensive, and correspondingly rare.  You should consider MPI for your approach, but my advice will apply regardless of the programming paradigm you choose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to on-node performance, if you are writing numerical routines yourself, it is easiest to get good serial performance in Fortran.  If you have a little bit of experience in C, C++, or Python, you can get very comparable performance (C and C++ are dead-even with Fortran, Python and MATLAB come within about a 25% time overhead without much effort).  MATLAB does this through a JIT compiler and very good linear algebra expressivity.  You will likely need to use either Cython, numpy, numexpr, or embed numerical kernels to get the claimed performance from Python.  I can't comment on Java's performance, because I don't know the language very well, but I suspect it is not far from Python's if written by an expert.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;A note on interfaces&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I hope I've convinced you that you are going to be able to do everything you want in any of the programming languages you are considering.  If you are using Java, the C interfaces will be a little challenging.  Python has excellent C and Fortran interface support through ctypes, Cython, and f2py.  LAPACK is already wrapped and available through scipy.  MATLAB has all of the functionality you need in its native libraries, but is not readily scalable or particularly easy to run on clusters.  Java can support C and Fortran interfaces with the &lt;a href=&quot;http://en.wikipedia.org/wiki/Java_Native_Interface&quot; rel=&quot;nofollow noreferrer&quot;&gt;JNI&lt;/a&gt;, but is not commonly found on clusters and in parallel software for scientific computing.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Maintainability&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;A lot of this is going to come down to personal flavor, but the general consensus on maintainability is that you want to minimize the number of lines of code in your software, write modular code with well-defined interfaces, and for computational software, provide tests that verify the correctness and functionality of the implementation.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Recommendation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I &lt;a href=&quot;http://numerics.kaust.edu.sa/pyclaw/&quot; rel=&quot;nofollow noreferrer&quot;&gt;personally have had a lot of luck with Python&lt;/a&gt; and I recommend it for many computational projects.  I think you should strongly consider it for your project.  Python and MATLAB are probably the most expressive of the languages available for scientific computing.  You can easily interface Python to any other programming language, you can use f2py to wrap your current Fortran implementation and piece-by-piece rewrite whichever parts you wish in Python while verifying that you are maintaining functionality.  At this time, I would recommend a combination of &lt;a href=&quot;http://www.python.org/getit/releases/2.7/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the official Python 2.7 implementation&lt;/a&gt; with &lt;a href=&quot;http://www.scipy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy&lt;/a&gt;.  You can get very easily started with this stack from the &lt;a href=&quot;http://www.enthought.com/products/epd_free.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;freely available Enthought Python Distribution&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also do most of this in C, C++, or Fortran.  C and C++ are very appealing languages for professional developers with a lot of experience, but frequently trip new developers and are in this sense probably not a great idea for a more academic code.  Fortran and MATLAB are popular in academic computation, but are weak at the advanced data structures and expressivity Python offers (think of a Python dict object, for example).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Related Questions:&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2656/recommendations-for-a-usable-fast-java-matrix-library&quot;&gt;Recommendations for a usable, fast Java matrix library?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-06-30T19:52:47.330" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2488" PostTypeId="2" ParentId="2486" CreationDate="2012-06-10T23:03:32.590" Score="3" Body="&lt;p&gt;In addition to Aron's very comprehensive answer, I'd take a look at the various threads on scicomp.stackexchange that dealt with the question which programming language to take -- both regarding the speed of programs as well as the question of how easy or hard it is to write and maintain software in these languages. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, in addition to what has been written there, let me make a few observations: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(i) You include co-array Fortran in your list. To my knowledge, the number of compilers that actually support it is very small -- and my, in fact, be zero. The most widely available Fortran compiler is GNU gfortran, and while the current development sources parse a subset of co-arrays, I believe that it doesn't actually support any of it (i.e., it accepts the syntax but implements none of the semantics). This is of course a general observation about newer Fortran standards: that the lag with which compilers actually support new standards is measured in several years -- compilers have only fully implemented Fortran 2003 in the last couple of years, and only partially support Fortran 2008. This shouldn't stop you from using any of it if you have a compiler that happens to support what you use, but you must known that you put yourself on a portability island.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ii) The same is certainly true with C++/Cilk+: Yes, Intel is developing this on a branch of GCC but it's not available in any of the GCC releases and will, likely, not be for a while. You can expect it to take another 2-3 years at least till you will find Cilk+ with the GCC versions installed on typical linux machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(iii) C++/TBB is a different story: The TBB has been around for a while, has a very stable interface and is compilable with most any C++ compiler that has existed for the past several years (on linux as well as on windows). We have been using it in &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal.II&lt;/a&gt; for several years already with good results. There is also a very good book on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(iv) I have my very own opinion on OpenMP, namely that it's a solution in search of a problem. It works well for parallelizing the inner loops which is what might be of interest if you have very regular data structures. But it's rarely what you want to do if you need to parallelize something -- because what you really want to do is to parallelize the &lt;em&gt;outer&lt;/em&gt; loops. And for that, solutions such as the TBB are &lt;em&gt;much&lt;/em&gt; better solutions because they use the mechanisms of the programming language rather than trying to describe what happens &lt;em&gt;outside&lt;/em&gt; the language (via #pragmas) and in such a way that you have no access to thread handles, result status indicators, etc, from within your program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(v) If you're experimental, you might also take a look at the new programming languages that are designed for parallel programming and, in particular, for tasks like the ones you describe. There are essentially two I'd take a look at:&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/X10_%28programming_language%29&quot; rel=&quot;nofollow&quot;&gt;X10&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Chapel_programming_language&quot; rel=&quot;nofollow&quot;&gt;Chapel&lt;/a&gt;. I've seen nice tutorials on Chapel, and it seems well designed, though both of course today are insular solutions as well.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-10T23:03:32.590" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2489" PostTypeId="2" ParentId="2486" CreationDate="2012-06-11T08:27:13.853" Score="2" Body="&lt;p&gt;Generally, if you are really serious about this software project, I would suggest a complete re-write in whatever language you yourself feel most comfortable with. It sounds like you will be doing the work alone, and therefore you will get the best results in the language you feel most at home with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, though, regarding parallelism, I would encourage you to try to think a bit outside of the box. OpenMP has its strengths, but is stuck in a mindset of taking a sequential code and slapping-on parallelism here and there. The same goes, in essence, for Intels TBB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Cilk&quot; rel=&quot;nofollow&quot;&gt;Cilk&lt;/a&gt; is definitely a step in the right direction, i.e. it forces you to re-think your problem/solution in an inherently parallel setup. What I don't like about it, though, is that it is &lt;em&gt;yet another language&lt;/em&gt;. Also, since it can only roughly infer relations between parallel tasks, the scheduler can be quite conservative and may not scale well for certain problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The good news is, though, that, again, if you're serious about your implementation, you can do what Cilk does, e.g. re-write your problem as a set of inter-dependent tasks and distribute them over a number of processors/cores, all on your own either using &lt;a href=&quot;http://en.wikipedia.org/wiki/Pthreads&quot; rel=&quot;nofollow&quot;&gt;pthreads&lt;/a&gt; or misusing OpenMP to spawn processes. A nice example of how this can be done is the &lt;a href=&quot;http://icl.cs.utk.edu/quark/&quot; rel=&quot;nofollow&quot;&gt;QUARK&lt;/a&gt; scheduler used in the &lt;a href=&quot;http://icl.cs.utk.edu/plasma/index.html&quot; rel=&quot;nofollow&quot;&gt;PLASMA&lt;/a&gt; library. A nice comparison of its performance vs. Cilk is given &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn213.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-06-11T08:27:13.853" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2490" PostTypeId="1" AcceptedAnswerId="2492" CreationDate="2012-06-11T10:29:01.560" Score="4" ViewCount="1710" Body="&lt;p&gt;If we want to solve nonlinear minimization problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{x} f(x),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;making least-squares assumption and using Gauss-Newton method so that at k$th$ iteration we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J_k^T J_k p_k = - J_k^T r_k,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with vector $r$ being residual and matrix $J$ Jacobian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then update $x$ in the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_{k+1} = x_k + \alpha p_k,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha \in (0, 1]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is how to find $\alpha$ such that $f(x_{k+1}) = min$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, what step length calculating strategy would be good enough taking into account $f$ and $f'$ are expensive to compute and highly nonlinear?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of methods that approximate this problem with polynomial, e.g. in case of quadratic approximation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p_0 + p_1 \alpha + p_2 \alpha^2 = min$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $p_0 = f(x_k), p_1 = f'(x_k), p_2 = f(x_k + \alpha p_k)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm wondering what are the other options to try? Can somebody point me to a good overview or shortly write down different techniques. &lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2012-06-11T14:00:25.157" LastActivityDate="2022-06-29T16:00:27.620" Title="Line search for Newton method" Tags="|optimization|nonlinear-programming|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2491" PostTypeId="2" ParentId="2490" CreationDate="2012-06-11T15:13:33.413" Score="7" Body="&lt;p&gt;Popular, simple to implement line search strategies are doubling and backtracking, but they need often more function values than strictly needed.&#xA;Interpolation schemes as you describe them need safeguards to be efficient, but all (or at least most) currently used schemes are based on some form of interpolation.&lt;/p&gt;&#xA;&lt;p&gt;The most used high quality line search (enforcing the Wolfe condition) is the one by More and Thuente,&#xA;which uses safeguarded cubic Hermite interpolation and is accompanied by a thorough theoretical analysis. &lt;a href=&quot;http://www.ii.uib.no/%7Elennart/drgrad/More1994.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ii.uib.no/~lennart/drgrad/More1994.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;More-Thuente line search implementations:&lt;/p&gt;&#xA;&lt;p&gt;in R:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://rdrr.io/github/jlmelville/rcgmin/man/more_thuente.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/github/jlmelville/rcgmin/man/more_thuente.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;in C++:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ZJU-FAST-Lab/LBFGS-Lite&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ZJU-FAST-Lab/LBFGS-Lite&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.extremeoptimization.com/Documentation/Reference/Extreme.Mathematics.Optimization.LineSearches.MoreThuenteLineSearch.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.extremeoptimization.com/Documentation/Reference/Extreme.Mathematics.Optimization.LineSearches.MoreThuenteLineSearch.aspx&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;in Julia:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.juliapackages.com/p/linesearches&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.juliapackages.com/p/linesearches&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JuliaNLSolvers/LineSearches.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JuliaNLSolvers/LineSearches.jl&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;in Rust:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.rs/argmin/latest/argmin/solver/linesearch/morethuente/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.rs/argmin/latest/argmin/solver/linesearch/morethuente/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2022-06-29T16:00:27.620" LastActivityDate="2022-06-29T16:00:27.620" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="2492" PostTypeId="2" ParentId="2490" CreationDate="2012-06-11T19:39:36.483" Score="6" Body="&lt;p&gt;This is discussed in great detail in the excellent book by Nocedal and Wright on nonlinear optimization. I can't summarize the method better than they describe it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-11T19:39:36.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2493" PostTypeId="1" AcceptedAnswerId="2496" CreationDate="2012-06-12T02:30:34.230" Score="32" ViewCount="1855" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt; I think I might want to port some code that calculates matrix exponential-vector products using a Krylov subspace method from MATLAB to Python. (Specifically, Jitse Niesen's &lt;a href=&quot;http://www.maths.leeds.ac.uk/~jitse/expmvp.m&quot;&gt;expmvp&lt;/a&gt; function, which uses an algorithm described in &lt;a href=&quot;https://www1.maths.leeds.ac.uk/~jitse/abstract_options.html&quot;&gt;this paper&lt;/a&gt;.) However, I know that unless I make heavy use of functions from modules derived from compiled libraries (i.e., I only use raw Python, and not many built-in functions), then it could be quite slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; What tools or approaches are available to help me speed up code I write in Python for performance? In particular, I'm interested in tools that automate the process as much as possible, though general approaches are also welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; I have an older version of Jitse's algorithm, and haven't used it in a while. It could be very easy to make this code fast, but I felt like it would make a good concrete example, and it is related to my own research. Debating my approach for implementing this particular algorithm in Python is another question entirely.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-06-12T02:38:24.673" LastActivityDate="2022-12-02T23:02:01.060" Title="What tools or approaches are available to speed up code written in Python?" Tags="|performance|python|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2494" PostTypeId="2" ParentId="2493" CreationDate="2012-06-12T06:34:58.013" Score="7" Body="&lt;p&gt;First of all, if there is a C or Fortran implementation available (MATLAB MEX function?), why don't you write a Python wrapper? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want your own implementation not only a wrapper, I would strongly suggest to use the numpy module for linear algebra stuff. Make sure it is linked to an optimized blas (like ATLAS, GOTOblas, uBLAS, Intel MKL, ...). And use Cython or weave. Read this &lt;a href=&quot;http://www.scipy.org/PerformancePython&quot;&gt;Performance Python&lt;/a&gt; article for a good introduction and benchmark. The different implementations in this article are available for download &lt;a href=&quot;https://github.com/scipy/speed&quot;&gt;here&lt;/a&gt; courtesy of Travis Oliphant (Numpy-guru).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2012-06-12T06:47:02.333" LastActivityDate="2012-06-12T06:47:02.333" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2495" PostTypeId="1" AcceptedAnswerId="2530" CreationDate="2012-06-08T21:17:19.303" Score="2" ViewCount="913" Body="&lt;p&gt;I am looking at the finite difference methods to solve simple $u_t=a(x,t)u_{xx}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are explicit, implicit, Crank Nicolson.&lt;br&gt;&#xA;The latter is said to be more accurate since the local truncation error is of second order provided all expansions are done around point $t^{n+0.5}$. However, local truncation error basically tells us how well the difference equation approximates the p.d.e. Thus, is I do expansion of CN scheme around any other point than I don't have second order anymore.&lt;br&gt;&#xA;&lt;strong&gt;Question1&lt;/strong&gt; How I can trust the results of the scheme if there is only a single point where second order occurs?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand if I have explicit scheme, regardless around which point I do Taylor I keep having first order, so I get why it is first order but what I don't understand is:&lt;br&gt;&#xA;&lt;strong&gt;Question2&lt;/strong&gt; Why is CN supposed to give global error at the point on the grid of second order when the local error of second order is estimated at the point which is not even on the mesh?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;br&gt;&#xA;EDIT:&#xA;Any scheme can be written as $\frac{u^{n+1}-u^n}{\tau}=L_hu^{\theta}$. So, regardless for which $\theta$ we have RHS=$u_{xx}(x_i,t^{\theta})+(h^2)...$ So it really boils down to approximation of the first derivative on the left hand side. And here we have choice. We can take $\theta=1$ and have implicit scheme with a derivative approximated at the point $t^{n+1}$, we can take the midpoint and have a derivative being approximated at that point with higher accuracy. However, the point is not on the grid! What I don't understand is that we expand around the point which is not on the grid but measure the error at the point which is on the grid and where the expansion of the difference equation gives only first order. I am confused about it.&lt;/p&gt;&#xA;" OwnerUserId="1611" OwnerDisplayName="Medan" LastEditorUserId="1611" LastEditDate="2012-06-14T16:55:43.320" LastActivityDate="2012-06-15T07:19:22.003" Title="I don't understand how to correctly calculate truncation error" Tags="|pde|finite-difference|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2496" PostTypeId="2" ParentId="2493" CreationDate="2012-06-12T09:38:49.950" Score="41" Body="&lt;p&gt;I'm going to break up my answer into three parts.  Profiling, speeding up the python code via c, and speeding up python via python.  It is my view that Python has some of the best tools for looking at what your code's performance is then drilling down to the actual bottle necks.  Speeding up code without profiling is about like trying to kill a deer with an uzi.&lt;/p&gt;&#xA;&lt;p&gt;If you are really only interested in mat-vec products, I would recommend &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/sparse.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.sparse&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Python tools for profiling&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://docs.python.org/3/library/profile.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;profile and cProfile modules&lt;/a&gt;&lt;/strong&gt;: These modules will give you your standard run time analysis and function call stack.  It is pretty nice to save their statistics and using the pstats module you can look at the data in a number of ways.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://pythonhosted.org/line_profiler/&quot; rel=&quot;nofollow noreferrer&quot;&gt;kernprof&lt;/a&gt;&lt;/strong&gt;: this tool puts together many routines for doing things like line by line code timing&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://pypi.org/project/memory-profiler/&quot; rel=&quot;nofollow noreferrer&quot;&gt;memory_profiler&lt;/a&gt;&lt;/strong&gt;: this tool produces line by line memory foot print of your code.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://ipython.org/ipython-doc/dev/interactive/tutorial.html#magic-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;IPython timers&lt;/a&gt;&lt;/strong&gt;: The &lt;code&gt;timeit&lt;/code&gt; function is quite nice for seeing the differences in functions in a quick interactive way.&lt;/p&gt;&#xA;&lt;h2&gt;Speeding up Python&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://cython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cython&lt;/a&gt;&lt;/strong&gt;: cython is the quickest way to take a few functions in python and get faster code.  You can decorate the function with the cython variant of python and it generates c code.  This is very maintable and can also link to other hand written code in c/c++/fortran quite easily.  It is by far the preferred tool today.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://docs.python.org/3/library/ctypes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ctypes&lt;/a&gt;&lt;/strong&gt;: ctypes will allow you to write your functions in c and then wrap them quickly with its simple decoration of the code.  It handles all the pain of casting from PyObjects and managing the gil to call the c function.&lt;/p&gt;&#xA;&lt;p&gt;Other approaches exist for writing your code in C but they are all somewhat more for taking a C/C++ library and wrapping it in Python.&lt;/p&gt;&#xA;&lt;h2&gt;Python-only approaches&lt;/h2&gt;&#xA;&lt;p&gt;If you want to stay inside Python mostly, my advice is to figure out what data you are using and picking correct data types for implementing your algorithms. It has been my experience that you will usually get much farther by optimizing your data structures then any low level c hack. For example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://numpy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;numpy&lt;/a&gt;&lt;/strong&gt;: a contingous array very fast for strided operations of arrays&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://github.com/pydata/numexpr&quot; rel=&quot;nofollow noreferrer&quot;&gt;numexpr&lt;/a&gt;&lt;/strong&gt;: a numpy array expression optimizer.  It allows for multithreading numpy array expressions and also gets rid of the numerous temporaries numpy makes because of restrictions of the Python interpreter.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://pypi.org/project/blist/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blist&lt;/a&gt;&lt;/strong&gt;: a b-tree implementation of a list, very fast for inserting, indexing, and moving the internal nodes of a list&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://pandas.pydata.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pandas&lt;/a&gt;&lt;/strong&gt;: data frames (or tables) very fast analytics on the arrays.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.pytables.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pytables&lt;/a&gt;&lt;/strong&gt;: fast structured hierarchical tables (like hdf5), especially good for out of core calculations and queries to large data.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="22284" LastEditDate="2022-12-02T23:02:01.060" LastActivityDate="2022-12-02T23:02:01.060" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="2497" PostTypeId="1" CreationDate="2012-06-12T18:33:19.310" Score="3" ViewCount="668" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Possible Duplicate:&lt;/strong&gt;&lt;br&gt;&#xA;  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/861/is-there-a-general-purpose-library-for-structured-grid-adaptive-mesh-refinement&quot;&gt;Is there a general-purpose library for structured grid adaptive mesh refinement?&lt;/a&gt;  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;I'm looking for a quality, open source, maintained, scalable automated mesh refinement library  to integrate with a FEM library.  It would be best if it could do both simplices and quads/hexes.  I have a list of about a dozen but have little experience using any of them.  What are the communities recommendations?&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-06-13T17:03:52.863" Title="Quality open source AMR libraries" Tags="|mesh|" AnswerCount="1" CommentCount="1" ClosedDate="2012-06-13T11:18:06.583" ContentLicense="CC BY-SA 3.0" />
  <row Id="2498" PostTypeId="1" CreationDate="2012-06-12T20:32:46.770" Score="9" ViewCount="187" Body="&lt;p&gt;I would like to predict runtimes for dense linear algebra operations on a specific architecture using a specific library. I would like to learn a model that approximates the function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F_{op} \;::\; $input sizes$ \rightarrow $runtime&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for operations like matrix-multiply, element-wise add, triangular solve, etc....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that these runtimes are mostly predictable due to the regularity of the operations once you get beyond problem sizes that fit comfortably in cache.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this assumption realistic? Is the runtime function likely to be nearly deterministic?&lt;/li&gt;&#xA;&lt;li&gt;Can I assume that this function will be polynomial in the sizes of the inputs? (i.e. I expect dense matrix multiply to look something like $\alpha n\times k\times m$ for $A_{nk}\times B_{km}$ and $\alpha$ some scalar coefficient)&lt;/li&gt;&#xA;&lt;li&gt;Is there preexisting work on this somewhere?&lt;/li&gt;&#xA;&lt;li&gt;My current plan is to do least squares regression with an $L_1$ regularizer. Any other suggestions?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Edit: To be clear I'm looking for runtimes, not FLOPs or any other common performance metric. I'm willing to restrict myself to one particular architecture. &lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-06-14T18:58:30.380" LastActivityDate="2012-09-26T21:02:42.920" Title="Predict runtimes for dense linear algebra" Tags="|linear-algebra|machine-learning|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2500" PostTypeId="2" ParentId="2449" CreationDate="2012-06-12T20:53:26.887" Score="1" Body="&lt;p&gt;Well, I figured I'd close this with what I ended up using.  I ended up using the final relation without the normalization&#xA;$$&#xA;e^{-S_p(U,\lambda a)}=\left[ \sum_r F_{r}^{\lambda^2} d_r \chi_r (U) \right]^{\lambda^{d-2}}=&#xA;\left[ \sum_r \left( \frac{1}{d_r}\int dU\, e^{-S_p(U,a)}\chi^{\ast}_r (U)\right)^{\lambda^2} d_r \chi_r (U) \right]^{\lambda^{d-2}}&#xA;$$&#xA;then taking the log of both sides I am able to grab the coefficients from the different order terms in the sum using the orthogonality of the $\chi_r(U)$s.&#xA;$$&#xA;\chi_r(WU)\delta_{rs}/d_r =\int dV\chi_r(WV)\chi_{s}^*(UV)&#xA;$$&#xA;That way I iterate as many times as necessary and then integrate again after multiplying through to knock off the coefficients.  Now I have been plotting these coefficients against each other and I have a definite trend, and I suspect I can match a fit to the data points, but I need some help with that too, which will be in a follow-up question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks anyways,&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2012-06-12T20:53:26.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2501" PostTypeId="2" ParentId="2493" CreationDate="2012-06-12T21:50:28.083" Score="4" Body="&lt;p&gt;Basically I agree with the other answers. The best options for speedy numerical &lt;code&gt;python&lt;/code&gt;-code are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use specialized libraries like &lt;code&gt;numpy&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;wrap your existing code so that your &lt;code&gt;python&lt;/code&gt;-program can call it directly&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But if you want to program the whole algorithm from scratch (I quote: &quot;I only use raw Python&quot;) then you might want to consider &lt;a href=&quot;http://pypy.org/&quot; rel=&quot;nofollow&quot;&gt;http://pypy.org/&lt;/a&gt; a JIT (Just In Time) implementation of &lt;code&gt;python&lt;/code&gt;. I haven't been able to use it for my project (because that relies on &lt;code&gt;numpy&lt;/code&gt;and the &lt;code&gt;pypy&lt;/code&gt;guys are corrently working on supporting that) but the benchmarks are quite impressive (&lt;a href=&quot;http://speed.pypy.org/&quot; rel=&quot;nofollow&quot;&gt;http://speed.pypy.org/&lt;/a&gt;) &lt;/p&gt;&#xA;" OwnerUserId="1556" LastActivityDate="2012-06-12T21:50:28.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2502" PostTypeId="2" ParentId="2497" CreationDate="2012-06-12T22:31:19.450" Score="2" Body="&lt;p&gt;Well, I'm biased, but I would recommend &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal.II&lt;/a&gt;. It's got the AMR plus all of the finite element stuff you'll ever need plus more documentation than you can read in a lifetime. It also scales to 10,000+ processors. It only supports quads/hexes, though. (Disclaimer: I'm the principal author of it, along with a couple dozen other co-authors from around the world.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other libraries that have other combinations of feature sets:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;libmesh: Very similar in many regards.&lt;/li&gt;&#xA;&lt;li&gt;FEniCS: More targeted to higher level problem descriptions. Meant to be used on a more intuitive level. I don't know how well it supports AMR or scales to large machines.&lt;/li&gt;&#xA;&lt;li&gt;There are a number of packages that are more targeted to individual kinds of problems (e.g. LifeV and OpenFOAM for fluid dynamics problems).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I will gladly let others join in to add more links to more libraries. I think that even taking into account my own bias, the ones above are what a neutral person today would want to look at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if the goal is to simply have a library that can manage the adaptive mesh, possibly widely distributed, then I would recommend &lt;a href=&quot;http://www.p4est.org/&quot; rel=&quot;nofollow&quot;&gt;p4est&lt;/a&gt;. It's occasionally a bit clumsy in its use, but it's lightweight, well written, and scales further than any other mesh project I can think of.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-06-13T17:03:52.863" LastActivityDate="2012-06-13T17:03:52.863" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2504" PostTypeId="2" ParentId="2498" CreationDate="2012-06-13T02:34:25.107" Score="6" Body="&lt;p&gt;There is lots of preexisting work. Most linear algebra library developers publish performance results in terms of floating-point performance which can be converted into run times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Googling for &quot;DGEMM performance&quot; for example, yields the following: &lt;a href=&quot;http://math-atlas.sourceforge.net/timing/3_5_10/index.html&quot;&gt;http://math-atlas.sourceforge.net/timing/3_5_10/index.html&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, you can expect the answers to be non-smooth. There will be jumps or spikes in the vicinity of certain problem sizes (which relate to cache sizes). You should also expect plateaus in rates, and, therefore, linear-ish regions for a broad range of problem sizes. I don't expect polynomial fits to be very helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a broad-based benchmarking effort, it might be easier to tabulate results and interpolate as necessary. What's your goal?&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2012-06-13T02:42:27.033" LastActivityDate="2012-06-13T02:42:27.033" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2505" PostTypeId="2" ParentId="2495" CreationDate="2012-06-13T06:15:00.620" Score="2" Body="&lt;p&gt;The truncation error of a numerical scheme is a property of the method and (of course) is independent of how you do the analysis.  If you believe you can show that the truncation error for Crank-Nicolson is first order at the grid points, then write your analysis here and we can explain where you're going wrong.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-06-13T06:15:00.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2506" PostTypeId="1" CreationDate="2012-06-13T08:27:17.740" Score="4" ViewCount="278" Body="&lt;p&gt;In a finite difference (FD) based electromagnetic formulation based on a Yee cell grid, one can define electric current source excitations ($J$) on the $E$ field grid points. At a distance, the fields generated from these excitations (in free-space) closely match with those of an analytical Hertzian dipole (which is infinitesimally thin). A Hertzian dipole has a constant current $I$ over its small length $l$, but since it is infinitesimally thin, its effective current area density $J = \frac{I}{dxdy} = \infty$, so the fields are singular at the origin and do not match up with those of the &quot;real&quot; current in a FD formulation (which is kind of like a cube of current rather than a line and thus has finite $J$ and finite $E$ everywhere including the near-field) close to the source. If I have multiple of these sources in my system at different locations, I can compute the power radiated from each individual one by integrating the quantity $J\cdot E$ over a small volume enclosing each source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an integral equation (IE) formulation, instead of directly specifying current excitations, one just specifies incident fields and the corresponding currents over dielectric/conducting surfaces and scattered fields are computed. How could I achieve the same thing as a current excitation with finite dimensions with an IE formulation (like with a FD method)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess I could use the closed-form fields from an analytical Hertzian dipole as an incident wave source and calculate scattered fields, however, this would not have the same result as a finite current &quot;cube&quot; in an FD formulation. Is there any straightforward way to define electric current excitations with finite current density J (or at least surface current density) in a surface integral equation formulation? Further, how would you go about calculating the incident power from each source when using an IE method?&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="1480" LastEditDate="2012-06-16T00:47:14.933" LastActivityDate="2012-11-05T14:35:29.680" Title="Defining electric current source excitations for surface integral equation formulations" Tags="|finite-difference|integral-equations|electromagnetism|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2507" PostTypeId="1" CreationDate="2012-06-13T11:27:57.417" Score="3" ViewCount="996" Body="&lt;p&gt;I am looking for a description of algorithm which allows me to detect key turning points on the road amongs a set of all given points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've ilustrated my problem on the below image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Green spots: those are the starting and ending point of a route.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Blue spots: those are the key points which should be detected by the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Red dots: This is the data about the road that has been driven.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone point me to the algorithm name or publication that deals with this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/N1zMG.png&quot; alt=&quot;Road&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1613" LastActivityDate="2012-10-29T06:37:05.427" Title="How to detect key turning points on a driven road?" Tags="|algorithms|approximation|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2508" PostTypeId="1" AcceptedAnswerId="2509" CreationDate="2012-06-13T14:23:32.423" Score="4" ViewCount="365" Body="&lt;p&gt;I have formulated an integer linear program (ILP). The constraint matrix for the ILP is &lt;a href=&quot;http://en.wikipedia.org/wiki/Totally_unimodular&quot; rel=&quot;nofollow&quot;&gt;totally unimodular&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I solve it as an LP without the integral constraints, or should I keep the integral constraints and let the optimization-toolbox's (MOSEK) mixed integer solver handle this special case?&lt;/p&gt;&#xA;" OwnerUserId="1570" LastEditorUserId="123" LastEditDate="2012-06-17T19:42:24.707" LastActivityDate="2012-06-17T19:42:24.707" Title="Should I include integral constraints in a integer linear program with a totally unimodular constaint matrix?" Tags="|optimization|linear-programming|mixed-integer-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2509" PostTypeId="2" ParentId="2508" CreationDate="2012-06-13T14:29:57.857" Score="4" Body="&lt;p&gt;It won't matter to any worthwhile mixed-integer linear programming (MILP, also known as MIP) solver if you keep or omit the integral constraints, as long as your constraint matrix is truly totally unimodular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be on the safe side, keep the integral constraints and call your MILP solver. Any solver utilizing a branch-and-bound or branch-and-cut framework should start by solving the linear programming (LP) relaxation of your ILP, in which case, it will omit the integral constraints. Since your constraint matrix is totally unimodular, an optimal solution of the LP relaxation should be satisfy the integral constraint because the polytope those constraints define consists of vertices with integer coordinates. Since the solution to the relaxation satisfies the integral constraints, the MILP solver will stop and return the solution obtained from the LP relaxation with little to no additional overhead beyond calling the LP solver (which is what you would have done if you just solved the LP relaxation).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-06-13T14:29:57.857" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2510" PostTypeId="1" AcceptedAnswerId="2511" CreationDate="2012-06-13T14:30:09.730" Score="20" ViewCount="11278" Body="&lt;p&gt;Given a dense matrix $$A \in R^{m \times n}, m &amp;gt;&amp;gt; n; max(m) \approx 100000 $$ what is the best way to find its null-space basis within some tolerance $\epsilon$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on that basis can I then say that certain cols are linearly dependent within $\epsilon$? In other words, having null space basis computed, what columns of $A$ have to be removed in order to get nonsingular matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2012-06-13T15:15:34.103" LastActivityDate="2022-09-22T18:06:56.727" Title="Null-space of a rectangular dense matrix" Tags="|linear-algebra|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2511" PostTypeId="2" ParentId="2510" CreationDate="2012-06-13T14:51:26.677" Score="17" Body="&lt;p&gt;Standard methods for determining the null space of a matrix are to use a QR decomposition or an SVD. If accuracy is paramount, the SVD is preferred; the QR decomposition is faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the SVD, if $A = U\Sigma V^{H}$, then columns of $V$ corresponding to small singular values (i.e., small diagonal entries of $\Sigma$) make up the a basis for the null space. The relevant tolerance here is what one considers a &quot;small&quot; singular value. MATLAB, for instance, takes small to be $\max(m,n) \cdot \varepsilon$, where $\varepsilon$ is related to machine accuracy (see &lt;a href=&quot;https://www.mathworks.com/help/techdoc/ref/rank.html&quot;&gt;here in MATLAB's documentation&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the QR decomposition, if $A^{T} = QR$, and the rank of $A$ is $r$, then the last $n-r$ columns of $Q$ make up the nullspace of $A$, assuming that the QR decomposition is rank revealing. To determine $r$, calculate the number of entries on the main diagonal of $R$ whose magnitude exceeds a tolerance (similar to that used in the SVD approach).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't use LU decomposition. In exact arithmetic, it is a viable approach, but with floating point arithmetic, the accumulation of numerical errors makes it inaccurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia covers these topics &lt;a href=&quot;https://en.wikipedia.org/wiki/Kernel_%28matrix%29#Numerical_computation_of_null_space&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-06-13T14:51:26.677" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2512" PostTypeId="2" ParentId="2510" CreationDate="2012-06-13T15:06:21.183" Score="10" Body="&lt;p&gt;If $m\gg n$, as your question indicates, you can save some work by first picking an index set $I$ of $p\approx 5n$ (say) random rows and using the orthogonal factorization $A_{I:}^T=QR$. (The QR-factorization is the one where $Q$ is sqare and $R$ is rectangular of rank $r$, and the remaining $n-r$ columns of $R$ are zero. Using a permuted QR factorization will enhance stability; the permutation must then be accounted for in a more detailed recipe.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, this will give you a much lower dimensional subspace spanned by the columns of $N$, the last $n-r$ columns of $Q$. This subspace contains the null space of $A$. Now pick another, disjoint random index set and compute the QR factorization of $(A_{I:}N)^T$. Multiply the resulting null space on the left by $N$ to get an improved $N$ of probably even lower dimension. Iterate until the dimension of $N$ no longer decreases. Then you probably have the correct null space and can check by computing $AN$. If this is not yet negligible, do further iterations with the most significant rows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Once you have $N$, you can find a maximal set $J$ of linearly independent columns of $A$ by an orthogonal factorization of $N^T=QR$ with pivoting. Indeed, the set $J$ of indices not chosen as pivots  will have this property.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-06-14T14:44:26.373" LastActivityDate="2012-06-14T14:44:26.373" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="2513" PostTypeId="1" AcceptedAnswerId="2514" CreationDate="2012-06-13T15:22:03.390" Score="2" ViewCount="81" Body="&lt;p&gt;This question is aimed at cleaning up an implementation detail of an in-house sparse direct solver. It uses METIS to reorder $A$ into $PAP^{T}$ for reduced fill-in. Inside the $Lx=b$ and $L^{T}x=b$ backsolution routines, the solver needs to apply the same permutation matrix to $b$ upon entry, then undo the permutation on $x$ just before returning. This is currently done with a temporary vector $t$, by copying $b$ with permutation into $t$, performing the $L$ and $L^{T}$ backsolves in-place on $t$, then copying $t$ with permutation back into $x$. (actually $b$ and $x$ are aliases for each other, but there's still the $t$-temporary)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to redo this operation to be completely in-place, eliminating the temporary $t$. I think that in order do so, I need to take the permutation vector as computed by METIS and determine a sequence of row swaps that yields it. (Is this correct?) The algorithm that I was thinking to implement would basically be like a customized quicksort with a customized comparator. (The comparator would be customized so that it will correctly sort $\{0, 1, 2 ,3,\ldots,N\}$ into METIS's answer, the quicksort would be customized to record the atomic swap operations so that they can later be applied in-place to $b$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is - is this the best way to accomplish what I am trying to do? I can tell already that the answer (a sequence of row swaps to realize $P$) is non-unique, because any sorting algorithm will yield an answer, but might perform completely different swaps. For example, bubblesort vs. quicksort will obviously yield different sequences - one with $O(n^2)$ swaps, the other with $O(n \log n)$, but both would work. I think finding the minimum number of swaps would be better for performance, hence the bias towards $O(n \log n)$ sort algorithms, but is there a different approach that would be guaranteed to find a minimum length sequence of swaps? I would think there would be some way to find a sequence that was $O(n)$ in length.&lt;/p&gt;&#xA;" OwnerUserId="1388" LastEditorUserId="276" LastEditDate="2012-06-13T15:31:26.097" LastActivityDate="2012-06-13T16:23:00.090" Title="Computing a sequence of row interchanges that realizes a given permutation matrix?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2514" PostTypeId="2" ParentId="2513" CreationDate="2012-06-13T16:23:00.090" Score="2" Body="&lt;p&gt;Can't you simply do it by swapping in the $i$th step the $i$th component with the component that contains the $i$th target component? &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-13T16:23:00.090" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2515" PostTypeId="1" CreationDate="2012-06-13T18:16:20.137" Score="3" ViewCount="868" Body="&lt;p&gt;I'm having some troubles implementing a collocation method to solve a stochastic partial differential equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla (a(x,w)\nabla u(x,w))=f(x,w)$ in $D$,&lt;br&gt;&#xA;$u=g$ in $\partial D$&lt;br&gt;&#xA;where $w$ is a vector of uniformly distributed random variables and a(x,w) is a randomly distributed coefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read a few papers on the subject, and I have a very basic understanding of the theory behind using smolyak sampling in the collocation method.  I think I could make a lot more sense of it by seeing how a particular example is coded from start to finish.  I found &lt;a href=&quot;http://www.stanford.edu/~paulcon/projects/SGS_Primer/sgs_primer.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; reference with code provided &lt;a href=&quot;http://www.stanford.edu/~paulcon/projects/SGS_Primer&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for the galerkin method.  I can't seem to find another example with the collocation method implemented.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-06-14T01:17:21.403" LastActivityDate="2013-05-29T19:48:26.100" Title="Where can I find coded examples of stochastic collocation applied to an elliptical PDE using smolyak sampling?" Tags="|pde|stochastic|collocation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2516" PostTypeId="1" AcceptedAnswerId="2518" CreationDate="2012-06-13T19:17:17.297" Score="9" ViewCount="4594" Body="&lt;p&gt;As exemplified by Jed Brown's answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2468/costs-of-lookups-versus-calculations/2471#2471&quot;&gt;Costs of lookups versus calculations&lt;/a&gt;, using vectorized vs non-vectorized floating point operations results in much faster code.  Many modern compilers claim that they can perform automatic vectorization.  How do I which parts of my code are being successfully vectorized?&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-06-14T08:42:43.683" Title="How do I know if my code is being vectorized by the compiler?" Tags="|fortran|c|compiling|vectorization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2517" PostTypeId="2" ParentId="2469" CreationDate="2012-06-13T19:23:21.003" Score="0" Body="&lt;p&gt;I would suggest Intel Fortran Composer on MacOS. It's good, up-to-date and works. One negative thing, it's not free. I think there is a 30 days free evaluation though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under Linux they offer a free, non-commercial version.&lt;/p&gt;&#xA;" OwnerUserId="1602" LastActivityDate="2012-06-13T19:23:21.003" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2518" PostTypeId="2" ParentId="2516" CreationDate="2012-06-13T19:39:29.570" Score="10" Body="&lt;p&gt;With the Intel compiler of any modern vintage, &lt;code&gt;-O3 -vec-report3&lt;/code&gt;. Optimization level three guarantees that it's trying to vectorize, and the vector report will tell you what it's doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GNU page on vectorization says that it's on by default at optimization level 3, but I can't find the equivalent of vec-report.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-06-13T19:39:29.570" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2519" PostTypeId="2" ParentId="2516" CreationDate="2012-06-13T19:53:19.310" Score="5" Body="&lt;p&gt;With GNU compilers, adding &lt;code&gt;-Wa,-ahl=asm.s&lt;/code&gt; will dump the generated assembly code to &lt;code&gt;asm.s&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Intel compilers, adding &lt;code&gt;-fcode-asm -Faasm.s&lt;/code&gt; will dump the generated code to &lt;code&gt;asm.s&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then inspect the assembly code and look for vector float point operations.&lt;/p&gt;&#xA;" OwnerUserId="486" LastActivityDate="2012-06-13T19:53:19.310" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2521" PostTypeId="1" AcceptedAnswerId="2524" CreationDate="2012-06-14T07:45:58.257" Score="5" ViewCount="4242" Body="&lt;p&gt;I have a mesh produced from scanning a real 3D object (I &lt;em&gt;don't&lt;/em&gt; have a geometry). What is the most convenient way to specify inlets, outlets, etc. for CFD in OpenFOAM? The mesh consists of thousands of faces, so defining each of them manually is not feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further information if that helps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The scanner produced a VRML file of the object surface.&lt;/li&gt;&#xA;&lt;li&gt;I transformed this (using Chisel) into STL, edited it and produced a 3D mesh in Salome.&lt;/li&gt;&#xA;&lt;li&gt;I exported it as UNV and converted with ideasUnvToFoam into an OpenFOAM mash.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1616" LastActivityDate="2012-06-14T22:22:58.137" Title="Specifying boundary conditions for imported mesh in OpenFOAM" Tags="|fluid-dynamics|openfoam|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2522" PostTypeId="2" ParentId="2516" CreationDate="2012-06-14T08:42:43.683" Score="8" Body="&lt;p&gt;Within the GNU compiler collection, you have the option &lt;code&gt;-ftree-vectorizer-verbose=n&lt;/code&gt; where &lt;code&gt;n&lt;/code&gt; is a &lt;a href=&quot;http://gcc.gnu.org/ml/gcc-patches/2005-01/msg01247.html&quot;&gt;number between 0 and 6&lt;/a&gt; which will print information similar to &lt;code&gt;icc&lt;/code&gt;/&lt;code&gt;ifort&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-06-14T08:42:43.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2523" PostTypeId="2" ParentId="2507" CreationDate="2012-06-14T09:07:44.657" Score="2" Body="&lt;p&gt;You can combine the turning information (derivative of x/y over time) with speed information (derivative of position over time). The rationale is that on critical points you need to slow down and then accelerate again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the best way would be, as Aron said, to compare your path with a map and detect when you have passed a crossing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, do your key points really need to be turning points? The GPS navigator should be able to find the turning points itself when you supply it your vias.&lt;/p&gt;&#xA;" OwnerUserId="1616" LastActivityDate="2012-06-14T09:07:44.657" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2524" PostTypeId="2" ParentId="2521" CreationDate="2012-06-14T09:53:33.133" Score="3" Body="&lt;p&gt;The best way to do this, since you have already exported the mesh in Salome is to generate &lt;strong&gt;groups of faces&lt;/strong&gt; in your Salome mesh. Since you have no geometry to base your choices on (e.g. group all the faces that belong to a circle A), you will have to use &lt;strong&gt;filters&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go to Mesh-&gt;Create Group-&gt;Group on Filter, and set such filters that will enable you to isolate the faces of interest. For boundary faces, start with the filter &lt;em&gt;Free faces&lt;/em&gt;. You may create additional geometry, such as planes, or rectangles, that you know will determine your set of faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though all you have is a Mesh in Salome, you can create geometrical entities that you will then use in the Set_filter environment to filter out the faces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of filters: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Free faces + Belong to Geom &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where Geom is e.g. a rectangle that you have drawn in the Geom module. This is the simplest and most elegant way, you just need to know the dimensions of your geometry (which you must know, if you are running a simulation on it). Once you have created the Geoms, they will be selectable: just click on the &lt;em&gt;Threshold value&lt;/em&gt; of the Belong to Geom filter and select the geometrical entity from the Geometry tree on the left. &lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-06-14T09:53:33.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2525" PostTypeId="1" AcceptedAnswerId="2592" CreationDate="2012-06-14T18:14:46.890" Score="34" ViewCount="965" Body="&lt;p&gt;In one of my papers, I list some numerical results in addition to some figures. What I'd like to do is make sure that the numerical results in my paper always agree with the code. Right now, I just directly copy the numerical results from my simulation output into the paper, which is extremely simple and low-tech, but error-prone, because I could mis-copy the results, or forget to sync the paper results with the code output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good way to keep the numerical results I quote in my papers in sync with the results generated by my code? (Here, I assume that executing the code is easy and practical to do whenever I want to update my paper.) These numerical results don't necessarily lend themselves to tabular form. Sometimes, I have tables in manuscripts, but more commonly, I have simulation parameters listed as numbers in &lt;em&gt;equations&lt;/em&gt;. An example would be something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mathbf{y}^{*} = (y_{1}^{*}, \ldots, y_{n}^{*})&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I'd like to replace the elements of the initial condition $\mathbf{y}^{*}$ with the actual parameters I use in a simulation that numerically integrates an system of ordinary differential equations. Using a table for one-off data such as this example seems like overkill and more ink than necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I presume that figures are an easier case: whenever the document is &quot;built&quot; (from LaTeX source, Markdown, RST, etc.), start the build process by executing the code. However, if people have better suggestions for keeping figures generated by my simulations in sync with my paper, I'd love to hear them.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-06-19T21:44:53.113" LastActivityDate="2012-07-27T13:09:13.343" Title="How do I make sure that the results of my simulations and the results in my paper are always in sync?" Tags="|software|documentation|reproducibility|" AnswerCount="8" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2526" PostTypeId="2" ParentId="2525" CreationDate="2012-06-14T18:40:04.530" Score="20" Body="&lt;p&gt;What you are asking for is the Elsivier grand challenge of the &lt;a href=&quot;http://www.executablepapers.com/&quot;&gt;&quot;Executable Paper&quot;&lt;/a&gt;.  While many approaches have been tried, none are as compelling as the authors might suggest.  Here are a few examples of techniques used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;http://www.reproducibility.org/wiki/Main_Page&quot;&gt;Madagascar Project&lt;/a&gt;&lt;/strong&gt; takes your approach, inside the make script have the simulations run that produce the figures and paper simultaneously.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;http://ipython.org/ipython-doc/dev/interactive/htmlnotebook.html&quot;&gt;IPython Notebook&lt;/a&gt;&lt;/strong&gt; provides a document that one can execute as you read and produce figures to your hearts content.  (I've seen word plugins, Mathematica, and numerous other solutions used in the same fashion)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;http://www.vistrails.org/index.php/Main_Page&quot;&gt;VisTrails&lt;/a&gt;&lt;/strong&gt; uses a service oriented  architecture approach and provides a &quot;providence&quot; or &quot;workflow&quot; manager.  Basically you register hooks to code then design a work flow or experiment that reproduces your work. It has been used on many types of codes, even HPC clusters.  With this approach you will have a way to replay the experiments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are tons of these type solutions out there, but those are three I was impressed with. Its a hard problem and one I believe we really aren't even close to addressing.  We can't even get people to release their code with their papers, how can we expect them to reproduce the results =P &lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2012-06-14T18:40:04.530" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2528" PostTypeId="2" ParentId="2521" CreationDate="2012-06-14T22:22:58.137" Score="0" Body="&lt;p&gt;Two workarounds that only need OpenFOAM itself&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Regular expressions in boundary conditions: if patches are consistently named (for instance &quot;wall01&quot;, &quot;wall02&quot; ... and &quot;inlet01&quot;, &quot;inlet02&quot;) then OpenFOAM allows to specify boundary conditions where instead of a name a regular expression (OpenFOAM recognizes this by the &quot;&quot;) is given and all patches whose names match this regular expression receive the same boundary condition (for instance &quot;wall.+&quot; matches wall01, wall02 etc)&lt;/li&gt;&#xA;&lt;li&gt;the createPatch-utility: this utility allows creating a new patch from a list of existing patches. The list of patches is specified in a dictionary file (again for specifying the patches regular expressions can be used)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Both methods here require that the names of the patches have some system&lt;/p&gt;&#xA;" OwnerUserId="1556" LastActivityDate="2012-06-14T22:22:58.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2529" PostTypeId="2" ParentId="65" CreationDate="2012-06-15T05:29:37.473" Score="2" Body="&lt;p&gt;Splitting error can, at least in principle, be reduced by spectral deferred correction methods.  However, this seems to be an area of active research and not really something ready for general use.&lt;/p&gt;&#xA;" OwnerUserId="1598" LastActivityDate="2012-06-15T05:29:37.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2530" PostTypeId="2" ParentId="2495" CreationDate="2012-06-15T07:19:22.003" Score="1" Body="&lt;p&gt;I will simplify and take $a$ to be constant. We know that the scheme&#xA;can be written &#xA;\begin{eqnarray*}&#xA;u_{i}^{n+1} &amp;amp; = &amp;amp; u_{i}^{n}+\frac{a\Delta t}{2}\left(F_{i}^{n+1}+F_{i}^{n}\right)\\&#xA;F_{i}^{k} &amp;amp; \approx &amp;amp; u_{xx}\left(x_{i},t_{k}\right)&#xA;\end{eqnarray*}&#xA;assuming that we can approximate $u_{xx}$ to high enough order (which&#xA;is probably done using the usual three point discretization). We want&#xA;to find $\left|u\left(x_{i},t_{n+1}\right)-u_{i}^{n+1}\right|$. We&#xA;Taylor expand about a &lt;strong&gt;point on the grid&lt;/strong&gt;&#xA;\begin{eqnarray*}&#xA;u\left(x_{i},t_{n+1}\right) &amp;amp; = &amp;amp; u_{i}^{n}+\Delta t\frac{\partial u}{\partial t}+\frac{\Delta t^{2}}{2}\frac{\partial^{2}u}{\partial t^{2}}+\frac{\Delta t^{2}}{6}\frac{\partial^{3}u}{\partial t^{3}}+\cdots\\&#xA;F\left(x_{i},t_{n+1}\right) &amp;amp; = &amp;amp; F_{i}^{n}+\Delta t\frac{\partial F}{\partial t}+\frac{\Delta t^{2}}{2}\frac{\partial^{2}F}{\partial t^{2}}+\frac{\Delta t^{2}}{6}\frac{\partial^{3}F}{\partial t^{3}}+\cdots&#xA;\end{eqnarray*}&#xA;then &#xA;\begin{eqnarray*}&#xA;u\left(x_{i},t_{n+1}\right)-u_{i}^{n+1} &amp;amp; = &amp;amp; u_{i}^{n}+\Delta t\frac{\partial u}{\partial t}+\frac{\Delta t^{2}}{2}\frac{\partial^{2}u}{\partial t^{2}}+\frac{\Delta t^{2}}{6}\frac{\partial^{3}u}{\partial t^{3}}+\cdots\\&#xA; &amp;amp;  &amp;amp; -u_{i}^{n}+\frac{a\Delta t}{2}\left(2F_{i}^{n}+\Delta t\frac{\partial F}{\partial t}+\frac{\Delta t^{2}}{2}\frac{\partial^{2}F}{\partial t^{2}}+\frac{\Delta t^{2}}{6}\frac{\partial^{3}F}{\partial t^{3}}+\cdots\right)\\&#xA; &amp;amp; = &amp;amp; \Delta t\left[u_{t}+au_{xx}\right]_{i}^{n}+\frac{\Delta t^{2}}{2}\frac{\partial^{2}u}{\partial t^{2}}+\frac{\Delta t^{2}}{6}\frac{\partial^{3}u}{\partial t^{3}}+\cdots\\&#xA; &amp;amp;  &amp;amp; +\frac{a\Delta t}{2}\left(\Delta t\frac{\partial F}{\partial t}+\frac{\Delta t^{2}}{2}\frac{\partial^{2}F}{\partial t^{2}}+\frac{\Delta t^{2}}{6}\frac{\partial^{3}F}{\partial t^{3}}+\cdots\right)&#xA;\end{eqnarray*}&#xA;So we have the $u_t + au_{xx}$ term which is  of order $\Delta t^2$ since it is the PDE at the previous iteration and the other terms have $\Delta t^2$ at least. &lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2012-06-15T07:19:22.003" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2531" PostTypeId="1" AcceptedAnswerId="2532" CreationDate="2012-06-15T07:41:09.453" Score="7" ViewCount="1046" Body="&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2152/what-is-a-good-way-to-run-parameter-studies-in-c&quot;&gt;Somewhat related&lt;/a&gt;, but I think the question is distinct enough to justify a separate question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a bit of background, I come from a observational/statistical Epidemiology background, working with data that's been collected already, so even our large datasets tend to be in discrete, inseparable &lt;em&gt;chunks&lt;/em&gt;. As such, I've never actually learned how to handle large amounts of data coming out of a simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on a compartmental model of an infection system, involving a system of ~10 differential equations and ~40 parameters shared among them. Most of these parameters aren't constants, but rather statistical estimates, and as such could be drawn from a distribution. That's one of the things I'd like to do - see how much the system varies purely as the result of uncertainty in the parameter estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which involves &lt;em&gt;running&lt;/em&gt; the numerical solution to the model sampling from each distribution many many times to cover the parameter space. If it was just a stochastic simulation, I might be fine with outputting one huge, or thousands of small data files that I could churn through with a script. My current problem is how to manage the output data given I need to know what parameter values were drawn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, my very quick and dirty method is to output the parameter value alongside the numerical results, meaning that if I ran the system for 100 steps, I end up with a 100 row, 50 column data set - but where 40 of those columns are the same number repeated over and over again. That seems hugely wasteful, and is resulting in really large files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Surely there's a better way to do this? Most of this is, currently, being implemented in Python.&lt;/p&gt;&#xA;" OwnerUserId="110" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-06-17T11:20:48.183" Title="How to handle large numbers of output data sets from a simulation/sensitivity analysis?" Tags="|python|data-sets|random-number-generation|simulation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2532" PostTypeId="2" ParentId="2531" CreationDate="2012-06-15T08:24:08.043" Score="2" Body="&lt;p&gt;You could try using something like a very basic &lt;a href=&quot;http://en.wikipedia.org/wiki/Relational_database&quot; rel=&quot;nofollow&quot;&gt;Relational database&lt;/a&gt;. You could label every output file with a separate &lt;em&gt;key&lt;/em&gt;, e.g. a sequential number, and then maintain a separate file which, in every row, contains the keys and the parameters used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're processing the data automatically, you will have to use one level of indirection, but that still saves you quite a few table entries compared to your model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a side note, it sounds like you're doing &lt;a href=&quot;http://en.wikipedia.org/wiki/Sensitivity_analysis&quot; rel=&quot;nofollow&quot;&gt;parameter sensitivity analysis&lt;/a&gt; by just sampling the parameter space. If you're trying to minimize some function of the results of your ODE integration, you may want to look at approaches which also integrate the derivative of the system variables with respect to the parameters. I have recently published &lt;a href=&quot;http://www.biomedcentral.com/1752-0509/6/46/abstract&quot; rel=&quot;nofollow&quot;&gt;a paper&lt;/a&gt; on this (sorry for the shameless plug), which you might find interesting, if not for the references to other methods.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-06-15T08:24:08.043" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2533" PostTypeId="2" ParentId="65" CreationDate="2012-06-15T08:49:15.450" Score="7" Body="&lt;p&gt;If you consider general operators A and B and if you only want to make positive time steps (which is what you usually require when solving parabolic problems), there is an order barrier of 2, i.e., using &lt;strong&gt;any&lt;/strong&gt; kind of splitting, you cannot obtain a rate of convergence higher than two. An elementary proof is given in a recent paper by S. Blanes and F. Casas, &lt;a href=&quot;http://www.gicas.uji.es/Fernando/MyPapers/2005APNUM.pdf&quot;&gt;http://www.gicas.uji.es/Fernando/MyPapers/2005APNUM.pdf&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are several ways out if you know a little bit more about your problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assume that you can solve your equations backward in time (which is common for, e.g., Schrödinger equations), then there are many splittings available, see the book &quot;Geometric Numerical Integration&quot; by Hairer, Lubich, and Wanner.&lt;/li&gt;&#xA;&lt;li&gt;If your operators generate analytic semigroups, i.e., you can insert complex values for t (typical for parabolic equations), it was recently observed that you can obtain higher order splittings by going into the complex plane. The first articles in that direction are by E. Hansen and A. Ostermann, &lt;a href=&quot;http://www.maths.lth.se/na/staff/eskil/dataEskil/articles/Complex.pdf&quot;&gt;http://www.maths.lth.se/na/staff/eskil/dataEskil/articles/Complex.pdf&lt;/a&gt; , and F. Castella, P. Chartier, S. Descombes, and G. Vilmart. The choice of complex splittings that are &quot;optimal&quot; in some sense is a topic of current research, you can find several papers on the topic on arxiv.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Summing up: If you put in some assumptions on your problem, you can get something, but if not, then order 2 is the maximum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS.: I had to take the link to the Castella et al-paper out due to spam prevention, but you can easily find it on google.&lt;/p&gt;&#xA;" OwnerUserId="3628" LastActivityDate="2012-06-15T08:49:15.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2534" PostTypeId="1" AcceptedAnswerId="2536" CreationDate="2012-06-15T09:39:44.933" Score="15" ViewCount="1681" Body="&lt;p&gt;I need to calculate the following integral:&#xA;$$&#xA;{1\over 2\pi i} \int_C f(E) \, d E&#xA;$$&#xA;$$&#xA;f(E) = {\rm Tr}\,\left(({\bf h} + E)\,{\bf G}(E) \right)&#xA;$$&#xA;Where $\bf h$ is a matrix (one particle kinetic and potential energy expressed in a basis), $\bf G$ is a matrix which depends on $E$ (one-particle many-body Green's function) and the contour integral is a left semicircle. The integrand $f(E)$ has poles on the negative real axis and it is expensive to evaluate. What is the most effective way to calculate such integral?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my research so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I use Gaussian integration, my integration path is a rectangle. I fixed the left and right side (i.e. width) and played with height (above and below the real axis) such that for the given integration order I get the highest accuracy. For example for order 20, if the height is too large, the accuracy goes down (obviously), but if it is too small, it also goes down (my theory is that it needs more and more points around the poles as height goes to 0). I settled with optimal height 0.5 for my function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Then I set the right side of the rectangle at E0, typically E0=0, but it could be E0=-0.2 or something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) I start moving the left side of the rectangle to the left and for each step I do integration order convergence to make sure my integral is fully converged for each rectangle. By increasing the width, I eventually get a converged value in the limit of the infinite left semicircle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computation is really slow and also not very accurate for large widths. One improvement is to simply partition the long width into &quot;elements&quot; and use Gaussian integration on each element (just like in FE).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option would be to integrate a small circle around each pole and sum it up. Problems:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) How to numerically find the poles of the function $f(E)$? It should be robust. The only thing I know is that they are on the negative real axis. For some of them (but not all) I also know a pretty good initial guess. Does there exist a method that works for any analytic function $f(E)$? Or does it depend on the actual form of $f(E)$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Once we know the poles, what numerical scheme is the best for integrating the small circle around it? Should I use Gaussian integration on a circle? Or should I use some uniform distribution of the points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option might be that once I know the poles thanks to a), there might be some semi-analytic way to get the Residues without the need of the complex integration. But for now I'd be happy to just optimize the contour integration.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="123" LastEditDate="2012-06-23T16:17:08.227" LastActivityDate="2012-06-23T16:17:08.227" Title="How to numerically calculate residues?" Tags="|complex-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2535" PostTypeId="1" AcceptedAnswerId="2548" CreationDate="2012-06-15T09:58:43.120" Score="8" ViewCount="12177" Body="&lt;p&gt;I am new to linear programming and have formulated a linear program (LP) with order of $10^{13}$ variables and $10^{13}$ constraints, although the constraint matrix is extremely sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to know if an LP of this scale is tractable or not?&lt;/p&gt;&#xA;" OwnerUserId="1570" LastEditorUserId="276" LastEditDate="2012-06-15T11:23:47.800" LastActivityDate="2016-11-02T01:44:43.183" Title="Is there an in practice limit on the number of constraints on a linear programming problem?" Tags="|linear-programming|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2536" PostTypeId="2" ParentId="2534" CreationDate="2012-06-15T09:59:28.283" Score="7" Body="&lt;p&gt;I can offer a suggestion for your first question: If you know your poles are somewhere along the real axis, you could localize them quite efficiently using &lt;a href=&quot;http://en.wikipedia.org/wiki/Rational_function&quot;&gt;Rational interpolation/approximation&lt;/a&gt;. This amounts to finding polynomials $p(x)$ and $q(x)$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) \approx \frac{p(x)}{q(x)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $x$ in some interval. The poles of $f(x)$ &lt;em&gt;should&lt;/em&gt; then match the roots of $q(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rational interpolation/approximation can be a tricky thing, but I've recently co-authored &lt;a href=&quot;http://www.emis.ams.org/journals/ETNA/vol.38.2011/pp146-167.dir/pp146-167.pdf&quot;&gt;a paper&lt;/a&gt; on a stable algorithm to compute them using the SVD. The paper contains Matlab code implementing the algorithm, and a more extensive version thereof is available as the function &lt;code&gt;ratinterp&lt;/code&gt; in the &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot;&gt;Chebfun project&lt;/a&gt;, of which I am one of the developers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your second question, &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=355979&quot;&gt;this paper&lt;/a&gt; may be useful.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-06-15T09:59:28.283" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2537" PostTypeId="2" ParentId="2531" CreationDate="2012-06-15T11:13:00.203" Score="2" Body="&lt;p&gt;If you want to analyze the sensitivity of your outcome on changes in the input space, you would want something like &lt;a href=&quot;http://dakota.sandia.gov/index.html&quot; rel=&quot;nofollow&quot;&gt;Dakota&lt;/a&gt; or SUSA (which is described in &lt;a href=&quot;http://www.hindawi.com/journals/stni/2008/798901/&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;). These codes allow you to run your simulation as a black box a number of times while sampling the parameters from the probability distribution you assign to them. The output is a statistical result: you get a mean and standard deviation on your output value. SUSA will also rank the input parameters according to sensitivity etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of ODEs, you might want to go for the &lt;a href=&quot;https://computation.llnl.gov/casc/sundials/main.html&quot; rel=&quot;nofollow&quot;&gt;Sundials&lt;/a&gt; package that has specific solvers that allow sensitivity analysis on parameters in the ODE. The &lt;a href=&quot;http://www.jmodelica.org/assimulo&quot; rel=&quot;nofollow&quot;&gt;Assimulo&lt;/a&gt; package has Python bindings for these solvers. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2012-06-15T11:22:18.650" LastActivityDate="2012-06-15T11:22:18.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2538" PostTypeId="2" ParentId="2535" CreationDate="2012-06-15T11:39:12.133" Score="6" Body="&lt;p&gt;A huge number of inequality constraints is generally tractable by constraint generation techniques, processing at each time only a limited number of constraints, and adding constraints violated at the current solution to the constraint set (while deleting strongly satisfied ones). But in the cases I have seen, this requires that the number of non-slack variables is limited, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is whether you can reformulate your problem in such a way that the problem itself or its dual has many fewer variables than constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: See also recent work by Nesterov on ''Subgradient methods for huge-scale optimization problems'', &#xA;&lt;a href=&quot;http://dial.academielouvain.be/vital/access/services/Download/boreal:107876/PDF_01&quot; rel=&quot;nofollow&quot;&gt;http://dial.academielouvain.be/vital/access/services/Download/boreal:107876/PDF_01&lt;/a&gt; .&#xA;His technique works under favorable circumstances with a complexity of $O(n\log n)$ if the accuracy requirements are moderate.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-06-17T11:26:42.903" LastActivityDate="2012-06-17T11:26:42.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2540" PostTypeId="2" ParentId="65" CreationDate="2012-06-15T15:07:20.930" Score="5" Body="&lt;p&gt;The &lt;a href=&quot;https://ccse.lbl.gov/&quot; rel=&quot;noreferrer&quot;&gt;CCSE&lt;/a&gt; group at LBNL have recently used Spectral Deferred Correction (SDC) methods in a low mach number flow with complex chemistry.  They compare the SDC results with Strang splitting, and the results are very promising.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a draft paper with the details: &lt;a href=&quot;https://ccse.lbl.gov/Publications/nonaka/LMC_SDC.pdf&quot; rel=&quot;noreferrer&quot;&gt;A Deferred Correction Coupling Strategy for Low Mach Number Flow with Complex Chemistry&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the SDC scheme is an iterative scheme that converges to a high-order accurate collocation solution, but it is built from first order methods.&lt;/p&gt;&#xA;" OwnerUserId="486" LastActivityDate="2012-06-15T15:07:20.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2541" PostTypeId="1" AcceptedAnswerId="2543" CreationDate="2012-06-15T19:53:25.280" Score="7" ViewCount="10182" Body="&lt;p&gt;I generated a cartesian grid in Python using NumPy's &lt;code&gt;linspace&lt;/code&gt; and &lt;code&gt;meshgrid&lt;/code&gt;, and I obtained some data over this 2D grid from an unknown function. I want to get an approximation of the value of the function over some points inside the boundaries of the grid which are not part of it. I do not have some other unstructured grid, I just want to know the value in certain points. I assume I have to &lt;em&gt;interpolate&lt;/em&gt; the data somehow, but I am very clueless on this and reading the documentation about the &lt;code&gt;interpolate&lt;/code&gt; module of SciPy and some related components doesn't help. How can I find out this interpolated data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: I know what I want to accomplish, but I have never done a task like this before and I have some problems formulating this question in terms of proper concepts and vocabulary. If I am not clear enough please help me improve my question.&lt;/p&gt;&#xA;" OwnerUserId="782" LastActivityDate="2013-05-23T14:57:59.233" Title="Interpolate 2D data" Tags="|python|interpolation|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2542" PostTypeId="2" ParentId="2541" CreationDate="2012-06-15T19:58:20.387" Score="7" Body="&lt;p&gt;You can find a good overview of methods and vocabulary on interpolation in two dimensions at &lt;a href=&quot;http://en.wikipedia.org/wiki/Multivariate_interpolation&quot;&gt;http://en.wikipedia.org/wiki/Multivariate_interpolation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-15T19:58:20.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2543" PostTypeId="2" ParentId="2541" CreationDate="2012-06-15T23:41:14.347" Score="8" Body="&lt;p&gt;I particularly like the &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.RectBivariateSpline.html#scipy.interpolate.RectBivariateSpline&quot;&gt;bivariate spline class&lt;/a&gt; for what I think you are describing. You can use it to make a function (i.e. it is callable at any point) which interpolates the data using a spline. If you want just an interpolation then you simply set the &lt;code&gt;kx&lt;/code&gt; and &lt;code&gt;ky&lt;/code&gt; values to 1. If you want a smoother function then increasing the order of the spline (arguably 3 is a good choice) and you can even use a smoothing factor which I never do. The &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; values you would use are the ones that &lt;code&gt;linspace&lt;/code&gt; gave you and the &lt;code&gt;z&lt;/code&gt; value would be the function values. &lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2012-06-15T23:41:14.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2544" PostTypeId="1" CreationDate="2012-06-16T05:29:17.360" Score="3" ViewCount="390" Body="&lt;p&gt;Is there any type of function that when graphed would show a curve which intersects the x axis multiple times, with each point being an arbitrary distance from the last? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, not like a trig function where each intersect is the same distance from the last. (2,0); (4,0); (6,0); (8,0).&#xA;And not like a spiral where the distance gets bigger and bigger (or smaller) (2,0); (4,0); (8,0); (16,0);    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for example, some curve which intersects x-axis at (2,0); (6,0); (14,0); (15,0); (20,0); (122,0)...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that type function exist? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, is it possible to solve/get the equation, given only those intersect points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't need the exact equation of any particular curve. Just the equation of any curve that happens to intersect x-axis at whatever given arbitrary x values. Is that at least that possible to do? &lt;/p&gt;&#xA;" OwnerUserId="1621" LastActivityDate="2012-06-16T10:01:02.227" Title="Get equation for a curve which intersects x at seemingly randomly distributed points?" Tags="|algorithms|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2545" PostTypeId="2" ParentId="2544" CreationDate="2012-06-16T07:43:01.353" Score="2" Body="&lt;p&gt;For a set zero points $x_i$ you can always find a function which is zero in all the points, but there is not just one function which fullfills this. This is obvious because any solution which is zero at any point can be multiplied with an arbitrary constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different approaches exist, depending on ease of implementation and desired properties of the function (smoothness, boundedness, oscillatory behavior and so on). &lt;a href=&quot;http://en.wikipedia.org/wiki/Polynomial_interpolation&quot; rel=&quot;nofollow&quot;&gt;Polynomial interpolation&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Spline_%28mathematics%29&quot; rel=&quot;nofollow&quot;&gt;Splines&lt;/a&gt; would be the best starting points.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2012-06-16T07:43:01.353" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2546" PostTypeId="2" ParentId="2544" CreationDate="2012-06-16T10:01:02.227" Score="7" Body="&lt;p&gt;If the number of zeros you're interested in is relatively small (say, less than 20), then use a random number generator to draw these zeros $x_i$ and then use the function $f(x)=\prod_i (x-x_i)$ -- and voila, you have a function that is zero at a collection of random points $x_i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the number $N$ of zeros becomes large, then a high order polynomial becomes instable to evaluate, and you'll probably want to use a different approach. For example, you could put a low-order polynomial through each set of $n \ll N$ adjacent random zeros and connect them in a $C^1$ way as you would do with splines, for example.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-16T10:01:02.227" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2547" PostTypeId="2" ParentId="2541" CreationDate="2012-06-16T11:17:57.550" Score="6" Body="&lt;p&gt;There is a nice video made by Travis Oliphant where he discusses 2D interpolation using python: see the youtube video &lt;a href=&quot;http://www.youtube.com/watch?v=_cJLVhdj0j4&quot;&gt;Python Interpolation 3 of 4: 2d interpolation with Rbf and interp2d&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-06-16T11:17:57.550" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2548" PostTypeId="2" ParentId="2535" CreationDate="2012-06-16T12:40:09.470" Score="9" Body="&lt;p&gt;$10^{13}$ is beyond tractable on today's largest supercomputers due mostly to memory limits.  The largest problems I've seen practically solved have had on the order of $10^5$ rows and columns, but the most important factor tends to be the number of nonzeros, where we are just crossing into solving problems with $10^6$ nonzeros.  See &lt;a href=&quot;http://plato.asu.edu/ftp/lpcom.html&quot;&gt;Mittelman's parallel benchmarks page&lt;/a&gt; to get a feel for what the best freely available and commercial solvers can do on a range of problems at this size.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-06-16T12:40:09.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2549" PostTypeId="2" ParentId="2535" CreationDate="2012-06-16T15:58:01.360" Score="4" Body="&lt;p&gt;My comment on Aron's answer got too long, but I would like to augment his answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I like bringing up the example of parallel benchmarks. Several comments to add here. All four of the solvers tested are commercial, but have free academic licenses available. Furthermore, the test cuts off the run time at 25000s, or ~8h, which is arbitrary, and depends heavily on external resource constraints (i.e., in a company, people may be unwilling to wait more than a day for results; in academia, some people can run their simulations for months). The test is run on a single quad-core machine, which is not indicative, to me, of bleeding edge performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I was hunting around for data to answer this question, I found papers 10+ years ago that were solving problems of roughly the same size, which suggests to me that we might be able to do better now with the infrastructure we have. Certainly not $10^{13}$ variables, but based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Karmarkar%27s_algorithm&quot; rel=&quot;nofollow&quot;&gt;$n^{3.5}$ scaling of interior point methods&lt;/a&gt;, if the linear algebra and parallelism were implemented well, and you had the time and a modestly sized cluster, I don't see why you couldn't attempt to solve a problem with $10^{7}$ or &lt;em&gt;maybe&lt;/em&gt; $10^{8}$ variables (only if you had special structure you could exploit with decomposition methods like Benders' decomposition or Dantzig-Wolfe, plus cutting-plane generation algorithms). (I will add that I am ignoring the effect of constraints, which complicate matters depending on how many bits are stored; this effect only makes the estimates below more pessimistic.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe GAMS has a parallel implementation, and since it uses solvers like CPLEX, Gurobi, MOSEK, and Xpress (i.e., the four solvers in the benchmark Aron cites), I don't see why one couldn't run parallel instances of those solvers (in fact, I know this is possible for CPLEX and Gurobi). The limiting factors will be memory (because memory is expensive) and communication more than processing power, since a linear program reduces, ultimately, to constructing and solving a system of linear equations repeatedly (a massive oversimplification, yes, but linear algebra is something we know how to parallelize).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But $10^{13}$ variables is too much. Assuming that memory and communication were no object, you'd need to take the largest problem in the benchmark, and scale the run time on that machine by a factor of roughly $10^{24}$ before possibly exploiting special structure that your particular problem may have. That's not to say that you couldn't try to solve it approximately using the methods that Professor Neumaier has suggested, but a solution to optimality is likely impossible without waiting a really long time, using a huge computer, and having a scalable implementation of an LP solver tuned to that huge computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a very rough order of magnitude estimate, the Intel Core 2 Quad used in the benchmark Aron cites can operate at a peak speed of 40 gigaflops. Assuming you were to get on Jaguar, Oak Ridge National Lab's supercomputer, and you could use the whole machine (extremely unlikely, but let's dream big), you'd have roughly 2 petaflops at your fingertips (based on the TOP 500 numbers &lt;a href=&quot;http://i.top500.org/system/176544&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, or roughly 50000 times the computing power, not counting penalties due to communication, memory limitations, or the fact that no one ever runs at peak speed (not even the LINPACK benchmark).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going from $10^{6}$ to $10^{7}$ variables means roughly a factor of 10,000 increase, which you could conceivably split among a cluster of 50-100 machines, and waiting a month (assuming you're willing to wait, you have the machines, and again, memory and communication aren't limiting, all of which are big &quot;ifs&quot;). Going from $10^{6}$ to $10^{13}$ variables means going from your desktop to using all of Jaguar, and waiting roughly $10^{17}$ to $10^{18}$ years. (And again, these effects also ignore the fact that you're going to have more constraints!)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-06-16T16:04:53.643" LastActivityDate="2012-06-16T16:04:53.643" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2550" PostTypeId="2" ParentId="2256" CreationDate="2012-06-16T16:09:40.113" Score="1" Body="&lt;p&gt;You could also try &lt;a href=&quot;http://helios.princeton.edu/GloMIQO/&quot; rel=&quot;nofollow&quot;&gt;GloMIQO&lt;/a&gt;. It's available in GAMS (as of version 23.8) and designed specifically to solve mixed-integer quadratic programs with linear or quadratic constraints, and has some promising results. You could also try contacting the authors of the software directly, if purchasing a GAMS license is not a viable option.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-06-16T16:09:40.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2551" PostTypeId="1" CreationDate="2012-06-17T02:36:17.217" Score="3" ViewCount="238" Body="&lt;p&gt;I am relatively new to integral equations for solving time-harmonic EM scattering problems. I have read a decent number of papers on the subject, and it seems that for formulations that can support 3D closed-volume dielectric scatterers that people use EFIE (Electric-Field Integral Equations), MFIE (Magnetic Field Integral Equations), CFIE (a linear combination of EFIE and MFIE to deal with some resonance problems leading to singularities in dielectrics), MPIE (Mixed-Potential Integral Equations), or PMCHW (Poggio, Miller, Chang, Harrington, and Wu) formulations. In terms of basis functions for Method of Moments (MoM) solutions, it seems that most people use RWG functions over triangle elements. I am wondering what the advantages\disadvantages of each of these methods are, and how to go about choosing what is most appropriate for my needs.&lt;/p&gt;&#xA;" OwnerUserId="1480" LastActivityDate="2012-06-17T10:16:30.390" Title="Different kinds of Integral Equation Methods" Tags="|integral-equations|electromagnetism|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2555" PostTypeId="2" ParentId="2551" CreationDate="2012-06-17T10:16:30.390" Score="4" Body="&lt;p&gt;You should definitely use CFIE over EFIE or MFIE, due to the problem of internal resonances. Basically, if you are at/near a frequency where there is a resonance in one of your scattering objects, then the EFIE or MFIE matrix may become singular/ill-conditioned. Using a linear combination of them greatly reduces the chances of singularity. I don't know much about the other formulations. I know someone that implemented CFIE with success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of basis functions, RWG (or Whitney) elements on triangular meshes is definitely the way to go, since it enforces the divergence constraints (this applies to FEM, finite differences, and other settings). It is in many ways the &quot;right&quot; choice of basis on the conceptual level (in frameworks like discrete exterior calculus), and it can also be generalized to quad meshes or general polygonal meshes.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-06-17T10:16:30.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2556" PostTypeId="2" ParentId="2466" CreationDate="2012-06-17T10:54:59.420" Score="6" Body="&lt;p&gt;I second the NURBS book and also would like to highlight that while this recursive formula is typically how one expresses the B-spline basis (as in a paper) it is not how you implement the basis functions. Since all higher order functions are based on lower order, you can compute all $p+1$ nonzero functions at once and reuse the lower order evaluations. That is how Piegl's algorithm does this.&lt;/p&gt;&#xA;" OwnerUserId="1625" LastActivityDate="2012-06-17T10:54:59.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2557" PostTypeId="2" ParentId="2531" CreationDate="2012-06-17T11:20:48.183" Score="0" Body="&lt;p&gt;As the main goal is to compute afterwards some statistics, the correct approach is to decide on a genertous set of additive sufficient statistics for what you possibly want to compute afterwards, and then to update thes statistics with each batch. Then you never need to store old data, though you misgt store some as backup or for independent hypothesis testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relevant statistics typically consist of a collection of numbers computed as the sum or max or min of selected functionals (key features, products of key features, key features in the presence of key indicators, etc.) together with the total number of items summed (maxxed, minned) over. This allows you to do at any instant correspondng means, covariance matrices, and probability tables of interest for subsequent statistical interpretation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to do some sort of 5-fold cross-validation, say, to validate your statistical conclusions, make 5 such collections, assigning each new item randomly to one of the five collections. Then combine the statistics for 4 out of the 5 collections to make predictions, and check their accuracy on the fifth one (each one taking this role in turn). This gives 5 accuracy estimates.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-17T11:20:48.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2558" PostTypeId="2" ParentId="2525" CreationDate="2012-06-17T11:34:54.737" Score="13" Body="&lt;p&gt;I have not had a lot of success in using other people's solutions to this problem. I usually just want something simple that works for me and gets the job done. To this end, I generally try to write one python script which is in charge of running all the results, parsing the output, as well as building the figures/tables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I write my codes to generate datafiles which contain the results in some text format. You can avoid rerunning these results in your script by first testing for the existence of the output file (in python using os.path.isfile() for example). If you want to rerun your results, simply remove the datafiles. If the datafiles exist, then I run a parser of these files. For this, the python module for regular expressions is very useful (re). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then from the parsed output I create the figures or tables. For tables in latex you can write the code to generate the table into a separate file (I use a .tbl extension) and then include this into your latex file. They key for me is to use 1 python script. If I have many, then I am later wondering which one is which and what they do. If this description is too vague I can send you some examples. &lt;/p&gt;&#xA;" OwnerUserId="1625" LastActivityDate="2012-06-17T11:34:54.737" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2559" PostTypeId="1" AcceptedAnswerId="2561" CreationDate="2012-06-17T15:21:06.900" Score="6" ViewCount="309" Body="&lt;p&gt;I have a large number of systems of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=b_i$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve for a large numbers of such $b_i\;1\leq i \leq k$ &#xA;but where $A$ is fixed (A is a rank $p$ general &#xA;--i.e. non sparse, non PSD-- matrix). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can solve them individually using an LU decomposition &#xA;(costs $O(p^3)$) but was wondering whether there is a&#xA; more efficient way to get all of the $k$ vector of &#xA;solution $x^*_i$ than solving these $k$ systems independently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical range of values of $p$, $k$ i'm considering are in the &#xA;10-100 (typically, i'm expecting $k\approx p$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A pointer to a c++ implementation of whatever method is proposed &#xA;would also be greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;" OwnerUserId="479" LastEditorUserId="4244" LastEditDate="2013-07-03T06:56:18.953" LastActivityDate="2013-07-03T06:56:18.953" Title="efficiently solving a low rank linear parametric systems?" Tags="|linear-solver|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2560" PostTypeId="2" ParentId="2559" CreationDate="2012-06-17T15:53:12.897" Score="7" Body="&lt;p&gt;First, you need to use LU factorization for the first system of equations.  This is an $\Theta(p^3)$ operation.  But since you are not changing the matrix coefficients and only changing the RHS vector, you can reuse the factors L and U to &quot;resolve&quot; the new systems in $\Theta(p^2)$ operations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Algorithm:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find the LU factorization of A.  &lt;/li&gt;&#xA;&lt;li&gt;For i = 1 to N&lt;/li&gt;&#xA;&lt;li&gt;Solve the system $Ly_i=b_i$ (forward substitution)&lt;/li&gt;&#xA;&lt;li&gt;Use the solution $y_i$ to solve the system $Ux_i=y_i$&lt;/li&gt;&#xA;&lt;li&gt;The solution to the system with RHS vector $b_i$ is then $x_i$.&lt;/li&gt;&#xA;&lt;li&gt;End For&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-06-17T15:53:12.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2561" PostTypeId="2" ParentId="2559" CreationDate="2012-06-17T16:26:39.800" Score="5" Body="&lt;p&gt;If you know all right hand sides in advance, you can combine them into a matrix and solve $AX=B$. Routines for this (in C and Fortran) can be found in the LAPACK library, which represents the state of the art. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $k\gg p$ it is probably more efficient (than using Paul's standard way of proceeding) to compute the inverse matrix and then multiply each right hand side with it. This vectorizes (and parallelizes) much better, and the small initial overhead in computing the inverse may be ignored. Unlike in the case $k\le p$.&#xA;For example, if $k=p$ then the overhead in arithmetic operations is 50%, and the gain from vectorization may not be sufficient to make up for this. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-17T16:26:39.800" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2562" PostTypeId="2" ParentId="2525" CreationDate="2012-06-17T19:38:08.363" Score="7" Body="&lt;p&gt;Even more important, in my opinion, is making sure that you can figure out how to re-generate all your results from scratch in a month or a year (for instance, when referees ask you to add or modify something).  For that purpose, what I do is include a text file with very detailed directions on how to reproduce all results.  It's best if you test these out by having someone else (like a co-author) try them.  I recommend that you also provide these instructions (and all your code) to referees and readers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bitbucket.org/ahmadia/pyclaw-sisc-rr/src/29282260fe48/README.txt&quot;&gt;Here is an example&lt;/a&gt; (actually prepared by my co-author, Aron Ahmadia).&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-06-17T19:38:08.363" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2563" PostTypeId="1" AcceptedAnswerId="2577" CreationDate="2012-06-17T19:45:57.113" Score="8" ViewCount="825" Body="&lt;p&gt;I want to solve a relatively small system of stiff ODEs (&amp;lt; 10 first-order equations) using high precision floating point arithmetic (using MPFR or alike). What would be the easiest algorithm to implement/port? I'm not only interested in the &quot;stepper&quot; algorithm, but in the combination of stepper/error estimation/step size control. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2012-06-18T20:12:21.097" LastActivityDate="2022-05-18T10:42:31.733" Title="What algorithm for solving a set of stiff ODEs would be easiest to port to high precision floating point arithmetic?" Tags="|ode|floating-point|precision|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2564" PostTypeId="2" ParentId="2563" CreationDate="2012-06-17T19:54:32.050" Score="6" Body="&lt;p&gt;This may be a bit outdated, but Hairer and Wanner's book recommends their own &lt;code&gt;radau5&lt;/code&gt; code. &lt;a href=&quot;http://www.biomedcentral.com/1752-0509/6/46/abstract&quot;&gt;In my own experience&lt;/a&gt;, this code is both extremely robust and efficient. It is also a rather straight-forward Runge-Kutta scheme, and thus not too difficult to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a Matlab implementation of &lt;code&gt;radau5&lt;/code&gt; by Christian Engstler somewhere out there.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-06-17T19:54:32.050" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2565" PostTypeId="2" ParentId="2563" CreationDate="2012-06-17T21:26:38.503" Score="5" Body="&lt;p&gt;Obviously, using multiple precision data types like those in MPFR only makes sense if numerical roundoff is at least around the same order of magnitude as the discretization error. You can only achieve this with high order integrators, so anything that has less than, say, 4th order will not be a useful candidate, and even higher order is better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not an expert in ODEs to recommend an integration package, but the considerations above may narrow your choice.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-17T21:26:38.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2566" PostTypeId="2" ParentId="2559" CreationDate="2012-06-17T21:30:39.793" Score="0" Body="&lt;p&gt;If $k&amp;lt;p$ then it might be worthwhile to use one of the &quot;subspace recycling&quot; methods you get by recording the iterates of a Krylov subspace method such as GMRES. If $k\approx p$ or $k&amp;gt;p$ I think it should be simple to show that these subspace recycling methods won't be effective.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-17T21:30:39.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2567" PostTypeId="1" AcceptedAnswerId="2570" CreationDate="2012-06-18T01:47:52.573" Score="3" ViewCount="319" Body="&lt;p&gt;I need to compute the &lt;a href=&quot;http://en.wikipedia.org/wiki/Permanent&quot; rel=&quot;nofollow&quot;&gt;Matrix Permanents&lt;/a&gt; of several $64 \times 64$, zero-one matrices. I have tried using the built in functions in both Sage and Maple, but both programs return out of memory errors. I have also tried several MatLab implementations, but they tend to run very slowly, on the order of days or weeks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering whether anyone has any suggestions (functions, programs, methods) of how I might compute these matrix permanents. I have access to Sage, Maple, and MatLab, and I also do a fair amount of work in Python. The matrices themselves have between 16-24 ones per row/column in the densest case and 6-8 ones per row/column in the sparsest case.&lt;/p&gt;&#xA;" OwnerUserId="603" LastActivityDate="2012-06-18T13:08:10.643" Title="Computing Permanents of $64 \times 64$ Matrices" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2568" PostTypeId="1" CreationDate="2012-06-18T07:32:06.527" Score="2" ViewCount="1800" Body="&lt;p&gt;I'm trying to install a Python package that relies on extensions built from Fortran 90 using f2py, but I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;error: f90 not supported by GnuFCompiler needed for source_file.f90&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have both gfortran and g77 installed, what is going wrong?&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-06-18T08:57:39.103" Title="f2py: error f90 not supported by GnuFCompiler needed for source_file.f90" Tags="|numpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2569" PostTypeId="2" ParentId="2568" CreationDate="2012-06-18T07:32:06.777" Score="4" Body="&lt;p&gt;By default, f2py prefers the GNU Fortran 77 compiler &lt;code&gt;--fcompiler=gnu&lt;/code&gt; over the GNU Fortran 95 compiler &lt;code&gt;--fcompiler=gnu95&lt;/code&gt;.  If you have both &lt;code&gt;g77&lt;/code&gt; and &lt;code&gt;gfortran&lt;/code&gt; on your path, then f2py will, use &lt;code&gt;g77&lt;/code&gt;, which cannot build Fortran 90 extensions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would prefer for numpy to use &lt;code&gt;gfortran&lt;/code&gt; over &lt;code&gt;g77&lt;/code&gt;, you can either pass the &lt;code&gt;--fcompiler=gnu95&lt;/code&gt; flag to setup.py when building your code, or you can modify your numpy installation's &lt;code&gt;numpy/distutils/fcompiler/__init__.py&lt;/code&gt; file [*] from:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_default_compilers = (&#xA;# sys.platform mappings&#xA;('win32', ('gnu','intelv','absoft','compaqv','intelev','gnu95','g95',&#xA;           'intelvem', 'intelem')),&#xA;('cygwin.*', ('gnu','intelv','absoft','compaqv','intelev','gnu95','g95')),&#xA;('linux.*', ('gnu','intel','lahey','pg','absoft','nag','vast','compaq',&#xA;            'intele','intelem','gnu95','g95','pathf95')),&#xA;('darwin.*', ('nag', 'absoft', 'ibm', 'intel', 'gnu', 'gnu95', 'g95', 'pg')),&#xA;('sunos.*', ('sun','gnu','gnu95','g95')),&#xA;('irix.*', ('mips','gnu','gnu95',)),&#xA;('aix.*', ('ibm','gnu','gnu95',)),&#xA;# os.name mappings&#xA;('posix', ('gnu','gnu95',)),&#xA;('nt', ('gnu','gnu95',)),&#xA;('mac', ('gnu','gnu95','pg')),&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_default_compilers = (&#xA;# sys.platform mappings&#xA;('win32', ('gnu95','intelv','absoft','compaqv','intelev','gnu','g95',&#xA;           'intelvem', 'intelem')),&#xA;('cygwin.*', ('gnu95','intelv','absoft','compaqv','intelev','gnu','g95')),&#xA;('linux.*', ('gnu95','intel','lahey','pg','absoft','nag','vast','compaq',&#xA;            'intele','intelem','gnu','g95','pathf95')),&#xA;('darwin.*', ('nag', 'absoft', 'ibm', 'intel', 'gnu95', 'gnu', 'g95', 'pg')),&#xA;('sunos.*', ('sun','gnu95','gnu','g95')),&#xA;('irix.*', ('mips','gnu95','gnu',)),&#xA;('aix.*', ('ibm','gnu95','gnu',)),&#xA;# os.name mappings&#xA;('posix', ('gnu95','gnu',)),&#xA;('nt', ('gnu95','gnu',)),&#xA;('mac', ('gnu95','gnu','pg')),&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will prioritize gfortran over g77, allowing you to build Fortran 90 extensions for your Python code using f2py.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[*] &lt;em&gt;If you've never modified an installed Python module before, you will probably we wondering where to find it.  From your Python prompt:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy; print numpy.__file__&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-06-18T08:57:39.103" LastActivityDate="2012-06-18T08:57:39.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2570" PostTypeId="2" ParentId="2567" CreationDate="2012-06-18T10:11:42.783" Score="4" Body="&lt;p&gt;The wikipedia article you link to has several references to computing the permanent for unweighted graphs, which is exactly what you have in your case. It might be worthwhile exploring these directions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, looking at the definition of the permanent, the difficulty is that there are &lt;em&gt;many&lt;/em&gt; permutations for 64 indices -- 64! in fact, quite an unwieldy number, about $10^{89}$. If we ignore this fact for a moment, computing the weight of each permutation is probably difficult for Matlab/Sage/Maple/... if you store the matrix as floating point numbers, but it should be reasonably straightforward if you wrote such a code in C/C++ and store the 0/1 entries as booleans. You then only need to compute the product of each weight as long as the factor is 'true', and terminate the computation if you hit a 'false', continuing with the next permutation. This is akin to a branch and bound algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that doesn't get you around the problem that there are so many permutations. The only approaches I can see working for this are (i) use symmetries in and other knowledge about your matrices to reduce the number of &quot;interesting&quot; permutations to a more manageable size, and (ii) following up on the connection to graphs, in particular that the permanent of a 0/1 matrix equals the number of cycle covers. I'm not a graph expert but there may be specialized algorithms for computing cycle covers.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-18T10:11:42.783" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2571" PostTypeId="2" ParentId="2563" CreationDate="2012-06-18T12:16:53.330" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is DLSODE an option?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is the system linear? If so, DGPADM is probably the best.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;They are relatively self-contained, so not too torturous to re-code if that's what you need to do.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastActivityDate="2012-06-18T12:16:53.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2572" PostTypeId="2" ParentId="2567" CreationDate="2012-06-18T13:08:10.643" Score="3" Body="&lt;p&gt;Computing prmanents is an NP-hard problem (for a sharper version of this, see&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Permanent_is_sharp-P-complete&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Permanent_is_sharp-P-complete&lt;/a&gt;), hence (unless P=NP) there are no fast general algorithms. This explains why you run out of resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you need the permanents for? Perhaps one can reformulate the underlying problem to use computationally more amenable stuff rather than permanents!&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-18T13:08:10.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2573" PostTypeId="2" ParentId="2563" CreationDate="2012-06-18T14:33:47.730" Score="3" Body="&lt;p&gt;There are existing implementations, though I don't know of any that are publicly available.  You could ask Glaser and Rohklin for their implementation of the methods described in &lt;a href=&quot;http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA471824&quot; rel=&quot;nofollow&quot;&gt;this paper.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-06-18T14:33:47.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2574" PostTypeId="1" AcceptedAnswerId="2575" CreationDate="2012-06-18T15:14:11.453" Score="14" ViewCount="1675" Body="&lt;p&gt;I've got a program that computes the largest eigenvalue of many real symmetric 50x50 matrices by performing singular-value decompositions on all of them. The SVD is a bottleneck in the program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there algorithms that are much faster in finding the largest eigenvalue, or would optimizing this part not give much return on investment?&lt;/p&gt;&#xA;" OwnerUserId="1627" LastEditorUserId="123" LastEditDate="2012-06-23T16:13:44.940" LastActivityDate="2012-06-27T14:06:33.507" Title="SVD for finding the largest eigenvalue of a 50x50 matrix -- am I wasting significant amounts of time?" Tags="|linear-algebra|eigensystem|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2575" PostTypeId="2" ParentId="2574" CreationDate="2012-06-18T15:25:32.190" Score="13" Body="&lt;p&gt;Depending on the precision you require for the largest eigenvalue, you could try using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Power_iteration&quot;&gt;Power Iteration&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your specific example, I would go as far as to not form $A=XX^\mathsf{T}$ explicitly, but compute $x \leftarrow X(X^\mathsf{T}x)$ in each iteration. Computing $A$ would require $\mathcal O(n^3)$ operations whereas the matrix-vector product requires only $\mathcal O(n^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convergence rate depends on the separation between the largest two eigenvalues, so this may not be a good solution in all cases,&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-06-18T15:36:10.227" LastActivityDate="2012-06-18T15:36:10.227" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2576" PostTypeId="2" ParentId="2574" CreationDate="2012-06-18T16:33:28.157" Score="5" Body="&lt;p&gt;If only 5 eigenvalues are very significant, the Lanczsos algorithm with $X(X^Tx)$ as matrix-vector multiply should give fast linear convergence after 5 initial steps, hence a fairly accurate largest eigenvalue with few iterations.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-06-27T14:06:33.507" LastActivityDate="2012-06-27T14:06:33.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2577" PostTypeId="2" ParentId="2563" CreationDate="2012-06-18T17:01:16.313" Score="3" Body="&lt;p&gt;I don't know if your problem would be tractable using a high order Taylor series method, but if so, and you're comfortable with implementing a solution to your problem in Python, then you could try a combination of &lt;a href=&quot;https://code.google.com/archive/p/mpmath&quot; rel=&quot;nofollow noreferrer&quot;&gt;mpmath&lt;/a&gt;, &lt;a href=&quot;https://code.google.com/archive/p/sympy&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt;, and possibly &lt;a href=&quot;https://web.archive.org/web/20130521223040/http://gitorious.org/pytaylor#more&quot; rel=&quot;nofollow noreferrer&quot;&gt;pytaylor&lt;/a&gt; (which is not as well-established as the first two Python modules, but does implement 4th order RK in addition to Taylor series methods). Based on a quick read of a &lt;a href=&quot;http://www.springerlink.com/content/p21220875812275k/&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper by Nedialkov in BIT&lt;/a&gt;, it seems as though a sufficiently high Taylor series (of order 20-30) could be used for highly accurate, efficient numerical integration of mildly to moderately stiff problems. (Nedialkov studies the differential-algebraic case; presumably, these results must also apply to ODEs.)&lt;/p&gt;&#xA;&lt;p&gt;Whatever solution method you choose, I encourage you open-source your work, since it sounds like it'd be a useful contribution to the ODE software landscape.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="22284" LastEditDate="2022-05-15T19:33:24.770" LastActivityDate="2022-05-15T19:33:24.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2578" PostTypeId="1" CreationDate="2012-06-18T19:33:27.207" Score="6" ViewCount="3151" Body="&lt;p&gt;I need to compute the determinant of a matrix that is calculated as $B \circ A$, with $B$ and $A$ being square matrices and $\circ$ representing their Hadamard product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way of doing this is through the eigenvalues of $B \circ A$, since a determinant is equal to the product of eigenvalues. This calculation is part of a simulation, where $A$ remains constant and $B$ changes during the simulation. Since $A$ does not change, its eigenvalues will not change, but $B$'s eigenvalues will.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to calculate this determinant by only updating the eigenvalues of $B$? Or, similarly, how can I efficiently determine the eigenvalues of the Hadamard product of two matrices (perferably in R)?&lt;/p&gt;&#xA;" OwnerUserId="1630" OwnerDisplayName="Peter Verbeet" LastEditorUserId="276" LastEditDate="2012-06-18T21:09:03.460" LastActivityDate="2018-03-18T01:02:16.630" Title="eigenvalues (and determinant) of a hadamard product of matrices" Tags="|linear-algebra|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2581" PostTypeId="2" ParentId="2133" CreationDate="2012-06-19T19:38:09.170" Score="3" Body="&lt;p&gt;Lately I've been working with Decomposition Techniques in Mathematical Programming: Engineering and Science Applications by Conejo, Castillo, Minguez and Garcia-Bertrand (http://www.springer.com/engineering/computational+intelligence+and+complexity/book/978-3-540-27685-2).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It covers several different techniques and when they are applicable, including Dantzig-Wolfe and Benders, and I find it has a good balance of theory and application.  I particularly like the examples, because I think they closely resemble real problems I might wish to formulate and solve.  &lt;/p&gt;&#xA;" OwnerUserId="1635" LastActivityDate="2012-06-19T19:38:09.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2582" PostTypeId="2" ParentId="2525" CreationDate="2012-06-19T21:05:25.200" Score="6" Body="&lt;p&gt;Emacs's &lt;em&gt;orgmode&lt;/em&gt; in combination with Babel achieves that. Babel can execute code snippets from various programming and scripting languages, e.g., it could open the file containing the simulation data and put it into a table in orgmode, which can be exported to LaTeX (and many other formats). It takes quite a while getting used to all the key combos in orgmode, but once it's running everything is automatic. &lt;/p&gt;&#xA;" OwnerUserId="1636" LastActivityDate="2012-06-19T21:05:25.200" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2583" PostTypeId="1" CreationDate="2012-06-20T01:50:03.840" Score="14" ViewCount="1430" Body="&lt;p&gt;I want to compute the &lt;a href=&quot;http://en.wikipedia.org/wiki/Spectrum_of_a_matrix&quot;&gt;spectrum&lt;/a&gt; (&lt;em&gt;all&lt;/em&gt; the eigenvalues) of a large sparse matrix (hundreds of thousands of rows). This is hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am willing to settle for an approximation. Are there approximation methods to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I hope for a general answer to this question I would also be satisfied with an answer in the following specific case. My matrix is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors#Eigenvalues_of_a_graph&quot;&gt;Normalized Laplacian&lt;/a&gt; of a large graph. Eigenvalues will be between 0 and 2 with a large number of them clustered around 1.&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2012-06-20T13:11:27.307" LastActivityDate="2018-09-18T16:22:07.900" Title="Approximate spectrum of a large matrix" Tags="|linear-algebra|eigensystem|graph-theory|approximation|" AnswerCount="6" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2584" PostTypeId="2" ParentId="2525" CreationDate="2012-06-20T02:06:26.793" Score="4" Body="&lt;p&gt;If running all your code is cheap then you could do something low-tech like the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could template your documents with formatted strings so that they look like this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;we observed a %(fractional_improvement)s increase in ...&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Have python scripts that look like this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;results = {&quot;alpha&quot;                  : run_alpha_computation(...),&#xA;           &quot;fractional_improvement&quot; : run_fi_computation(...), &#xA;           ...}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then do something like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for fn in filenames:&#xA;    file = open(fn);      s = file.read();       file.close()&#xA;    file = open(fn, 'w'); file.write(s%results); file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You could then wrap this up in a Makefile.&lt;/p&gt;&#xA;" OwnerUserId="19" LastActivityDate="2012-06-20T02:06:26.793" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2586" PostTypeId="2" ParentId="2583" CreationDate="2012-06-20T11:35:16.770" Score="15" Body="&lt;p&gt;If your graph is undirected (as I suspect), the matrix is symmetric, and you cannot do anything better than the Lanczsos algorithm (with selective reorthogonalization  if necessary for stability). As the full spectrum consists of 100000 numbers, I giess you are mainly interested in the spectral density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get an approximate spectral density, take the spectrum of the leading Krylov subspace of dimension 100 or so, and replace its discrete density by a smoothed version. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The leading Krylov spectrum will have nearly resolved well-isolated eigenvalues (should these exist), approximates the eigenvalues at the end of the nonisolates spectrum, and is somewhat random in-between, with a distribution whose cumulative distribution function resembles that of the true spectrum. It would converge to it in exact arithmetic if the dimension grows. (If your operator were infinite-dimensional, this would still be the case, and you'd get the integral of the true spectral density function on the continuous spectrum.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-06-21T08:36:15.870" LastActivityDate="2012-06-21T08:36:15.870" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2587" PostTypeId="1" AcceptedAnswerId="2588" CreationDate="2012-06-17T18:24:47.123" Score="17" ViewCount="3872" Body="&lt;p&gt;Suppose I have some function $f$ and I want to find $x$ such that $f(x)\approx 0$. I might use the Newton-Raphson method. But this requires that I know the derivative function $f'(x)$. An analytic expression for $f$ may be unavailable.  For example, $f$ may be defined by a complicated piece of computer code that consults a database of experimental values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even if $f'$ is complicated, I can approximate $f'(a)$ for any particular $a$ by choosing a small number $\epsilon$ and calculting $f'(a) \approx {f(a+\epsilon) - f(a)\over\epsilon}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have heard that there are distinct disadvantages to this approach, but I don't know what they are. Wikipedia hints that &quot;Using this approximation would result in something like the secant method whose convergence is slower than that of Newton's method.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please elaborate on this, and provide a reference that particularly discusses the problems with this technique?&lt;/p&gt;&#xA;" OwnerUserId="1640" OwnerDisplayName="Mark Dominus" LastEditorUserId="1640" LastEditDate="2012-06-20T13:45:58.000" LastActivityDate="2020-09-11T03:21:53.153" Title="Drawbacks of Newton-Raphson approximation with approximate numerical derivative" Tags="|reference-request|approximation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2588" PostTypeId="2" ParentId="2587" CreationDate="2012-06-20T13:38:58.847" Score="13" Body="&lt;p&gt;For the sake of notation, let's suppose that $f: \mathbb{R}^{n} \rightarrow \mathbb{R}^{n}$ (i.e., it's a vector-valued function that takes a vector as input and outputs a vector of the same size). There are two concerns: computational cost and numerical accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculating the derivative $\mathrm{D}f(x)$ (the Jacobian matrix, $J(x)$, or $(\nabla f(x))^{T}$, or whatever you prefer) using finite differences is going to require $n$ function evaluations. If you could calculate the derivative using floating point arithmetic directly from the definition, you would have to calculate the difference quotient&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mathrm{D}f(x)e_{i} = \lim_{\varepsilon \rightarrow 0} \frac{f(x + \varepsilon e_{i}) - f(x)}{\varepsilon}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for each $i = 1, \ldots, n$, assuming you don't do any sort of &quot;smart finite differencing&quot; (like Curtis-Powell-Reid) because you know (or can detect) the sparsity pattern of $\mathrm{D}f$. If $n$ is large, that could be a lot of function evaluations. If you have an analytical expression for $\mathrm{D}f$, then calculating it could be cheaper. Automatic (also known as algorithmic) differentiation methods can also be used in some cases to calculate $\mathrm{D}f$ at roughly 3 to 5 times the cost of a function evaluation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also numerical concerns. Obviously, on a computer, we can't take the limit of a scalar as it goes to zero, so when we approximate $\mathrm{D}f$, we're really picking $\varepsilon$ to be &quot;small&quot; and calculating&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mathrm{D}f(x)e_{i} \approx \frac{f(x + \varepsilon e_{i}) - f(x)}{\varepsilon},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\approx$ means it's an approximation, and we hope it's a really good approximation. Calculating this approximation in floating point arithmetic is tough because if you pick $\varepsilon$ too large, your approximation could be bad, but if you pick $\varepsilon$ too small, there could be significant rounding error. These effects are covered in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_differentiation#Practical_considerations_using_floating_point_arithmetic&quot;&gt;Wikipedia article on numerical differentiation&lt;/a&gt; in superficial detail; more detailed references can be found within the article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the error in the Jacobian matrix $\mathrm{D}f$ isn't too large, Newton-Raphson iterations will converge. For a detailed theoretical analysis, see Chapter 25 of &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898718027&quot;&gt;&lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; by Nick Higham&lt;/a&gt;, or &lt;a href=&quot;http://eprints.ma.man.ac.uk/979/01/tisseur3.pdf&quot;&gt;the paper by Françoise Tisseur&lt;/a&gt; on which it is based.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Libraries generally take care of these algorithmic details for you, and usually, library implementations of the Newton-Raphson algorithm (or variants thereof) will converge quite nicely, but every so often, there will be a problem that causes some trouble due to the drawbacks above. In the scalar case $(n = 1)$, I'd use &lt;a href=&quot;https://en.wikipedia.org/wiki/Brent%27s_method&quot;&gt;Brent's method&lt;/a&gt;, owing to its robustness and good convergence rate in practice.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-06-20T13:47:20.397" LastActivityDate="2012-06-20T13:47:20.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2589" PostTypeId="1" AcceptedAnswerId="2961" CreationDate="2012-06-20T15:37:23.373" Score="9" ViewCount="1673" Body="&lt;p&gt;I am interested in local AMR on unstructured meshes. Currently, I'm working with the OpenFOAM library - it supports completely unstructured local AMR:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;cell refinement criteria determine a list of cells that are cut &lt;/li&gt;&#xA;&lt;li&gt;selected cells are refined: entire mesh is re-constructed &lt;/li&gt;&#xA;&lt;li&gt;a map is created from the old mesh to a new one&lt;/li&gt;&#xA;&lt;li&gt;connectivity is recomputed (face-cells, edge-faces, etc)&lt;/li&gt;&#xA;&lt;li&gt;fields are mapped onto the new mesh&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since the data structures involved are basically C++ vectors, the mesh gets inflated, copied. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to learn about alternative approaches that can be built upon a mesh that uses static data structures. One of them is the parallel Octree Forrest local AMR, present in &lt;a href=&quot;http://users.ices.utexas.edu/~carsten/papers/BursteddeWilcoxGhattas11.pdf&quot;&gt;p4est&lt;/a&gt; and &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1413370.1413389&quot;&gt;Dendro&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me to a &lt;em&gt;recent&lt;/em&gt; review paper on local adaptive AMR strategies for unstructured mesh? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experience based advice would be even better: which local AMR engine is the optimal choice for fixed data structure based unstructured mesh? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need an overview &lt;em&gt;before&lt;/em&gt; reading about balancing inter-tree communication on the first page of a paper. :) &lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2013-07-02T17:30:11.527" Title="What are the strategies for local Adaptive Mesh Refinement (local AMR) on unstructured meshes?" Tags="|libraries|reference-request|adaptive-mesh-refinement|openfoam|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2590" PostTypeId="1" AcceptedAnswerId="2621" CreationDate="2012-06-20T16:39:00.127" Score="5" ViewCount="256" Body="&lt;p&gt;What are typical (or promising) techniques and methods in CFD to achieve high-detailed turbulence and non-linear waves interaction? And with &quot;active&quot; geometry (when bodies interacts with fluid in both ways)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I've been recommended SPH, but I am very unsure, that SPH is smart choice for my task. Especially considering my attempts to achieve nice parallel behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it would be interesting to see some promising and good papers about some recommended techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update on motivation&lt;/strong&gt;: the reason to write our own code is because there is no high-order discretization with low dissipation and dispersion in most comercial codes. Also, software like ANSYS is known to be not very scalable.&lt;/p&gt;&#xA;" OwnerUserId="1631" LastEditorUserId="1631" LastEditDate="2012-06-21T04:01:15.650" LastActivityDate="2012-06-25T08:03:47.897" Title="CFD for high-detailed turbulence and non-linear waves" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2591" PostTypeId="1" CreationDate="2012-06-20T17:17:44.073" Score="16" ViewCount="1666" Body="&lt;h2&gt;TL,DR&lt;/h2&gt;&#xA;&lt;p&gt;What is the accepted best practice in scientific computing circles for storing large quantities of hierarchically structured data? For example, SQL does not play nicely with large sparse matrices. Is there a good tool out there for structuring, warehousing, and analyzing this kind of data? What do the guys at the LHC use?&lt;/p&gt;&#xA;&lt;h2&gt;Use case details&lt;/h2&gt;&#xA;&lt;p&gt;I want to store data from protein simulations according to the following hierarchy:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;protein&#xA;  |__simulation conditions&#xA;  |____|__residues&#xA;  |____|____|__conformers&#xA;  |____|____|____|__atoms&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Every protein should be aware of each of its residues, every atom should know the conditions used for its simulation, etc. and vice versa.&lt;/p&gt;&#xA;&lt;p&gt;Originally I figured that a relational database would be perfect for this application, and so I wrote a program using python and sqlalchemey that stores the data in an SQL database. In practice, however, this program is not working so well.&lt;/p&gt;&#xA;&lt;p&gt;The biggest issue relates to the fact that there is an N x N matrix at the conformer data level that stores the potential energy due to the pairwise interactions between every possible pair of conformers. Most of the entries in the matrix are zeros, so I'm storing the matrix in a separate table in the database in a kind of sparse format, one row per entry. Unfortunately, for a simulation involving several thousand conformers the pairwise table still ends up with several hundred thousand rows and:&lt;/p&gt;&#xA;&lt;p&gt;a) builds and queries very slowly (hours)&lt;br /&gt;&#xA;b) takes up an order of magnitude more space on my hard drive than an equivalent plain text representation of the data as a non-sparse matrix&lt;br /&gt;&#xA;c) takes up more than ten gigabytes of memory when the table is read into memory&lt;/p&gt;&#xA;&lt;p&gt;My ultimate goal is to store tens of thousands of runs (derived from thousands of proteins under several dozen simulation conditions) in the database so they can all be analyzed together. This would mean that the table representing the pairwise matrices would likely grow to around a billion rows. Currently it's seeming like I'm going to need a Cray or some other shared-memory monster in order to even run a single query on this database.&lt;/p&gt;&#xA;&lt;p&gt;Do I have any better options here? What do the guys at the LHC use?&lt;/p&gt;&#xA;&lt;h3&gt;update&lt;/h3&gt;&#xA;&lt;p&gt;One PhD, two jobs, and eight years later and I still don't have a good answer to this question (even though I did publish &lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btw614&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper about my own custom format for comp bio data&lt;/a&gt;). HDF5 is okay up to 10's of GBs, but is incompatible with Hadoop/Spark and so is unsuitable for truly large data. The most promising new thing is &lt;a href=&quot;https://zarr.readthedocs.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;zarr&lt;/a&gt;, a Spark-compatible HDF5 alternative&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="34" LastEditDate="2021-01-12T00:12:37.393" LastActivityDate="2021-01-12T00:12:37.393" Title="Best practice for storing hierarchical simulation data" Tags="|data-management|data-storage|data-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2592" PostTypeId="2" ParentId="2525" CreationDate="2012-06-20T18:18:06.730" Score="15" Body="&lt;p&gt;As some comments have suggested, this approach has long been developed in the R community by building on &lt;code&gt;Sweave&lt;/code&gt; and more recently, &lt;code&gt;knitr&lt;/code&gt;.  Obviously this approach has the disadvantage of being language-specific at the moment, but the advantage that its regularly used in academic papers.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Use of Sweave in real publications&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://biostatistics.oxfordjournals.org/content/10/3/405.full&quot;&gt;Journal of Biostatistics&lt;/a&gt; encourages these submissions, and places a kitemark letter &quot;R&quot; on academic papers in which the reproduciblity editor has been able to run the code and data and get these results.  &lt;/li&gt;&#xA;&lt;li&gt;Essentially all publications in the &lt;a href=&quot;http://journal.r-project.org/current.html&quot;&gt;R-Journal&lt;/a&gt; are built on Sweave. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course more commonly Sweave/knitr users submit just the final output to most journals, with the personal confidence that the methods indeed reproduce the results.  Likewise sweave is often used to create slides for professional talks, etc. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Useful features&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;To be useful in practice, such a system should have certain features.  With a critical mass of users, many of these are well-developed in &lt;code&gt;knitr&lt;/code&gt;.  A few highlights: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Caching.  Writing in a format such as markdown or latex that must be compiled to see the output makes this including the code impossible when results cannot be cached.  Clever caching in &lt;code&gt;knitr&lt;/code&gt; actually makes debugging intensive code much easier than working in pure &lt;code&gt;R&lt;/code&gt;, since it is not necessary to re-run successful chunks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Toggle code display.  In a formal publication one may not want any of the underlying code to be visible on the output.  Meanwhile it is often valuable to show the (nicely formatted, syntax-highlighted) code that produces the results in the output format when you want the reader to see exactly what you are typing.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Community.  Perhaps the most compelling advantage of this approach over a homebrewed solution is the number of folks familiar with the model and trying to improve it.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lots of nice examples of other features can be found on the &lt;a href=&quot;http://yihui.name/knitr/demos&quot;&gt;knitr webpage demos&lt;/a&gt;, including tools for &quot;lightweight publishing&quot; or sharing on the web.  Many of these features help make this approach more compatible with one's general workflow, rather than something to do only when writing the manuscript.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Historical footnote.&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;While rooted in Knuth's &quot;literate programming&quot;, as the name suggests, the emphasis is quite different, as literate programming focuses on connecting software code and documentation (in the R community that role is played by &lt;code&gt;Roxygen&lt;/code&gt;, which traces its roots to a different branch of the &quot;literate programming tree&quot;).  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Going beyond&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In principle we might ask much more from a dynamic document, such as the ability of the reader to change the inputs and see the outputs without having to edit and recompile the entire document, for instance, using an interactive online platform.  &lt;a href=&quot;http://www.omegahat.org/XDynDocs/&quot;&gt;XDynDocs&lt;/a&gt; may be a step in this direction.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Other approaches&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Might want to keep an eye on &lt;a href=&quot;http://www.dexy.it&quot;&gt;dexy.it&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;In a slightly different approach: A growing number of economics papers host code associated with publications on &lt;a href=&quot;http://www.runmycode.org/&quot;&gt;http://www.runmycode.org/&lt;/a&gt;, which will re-run the analyses on the cloud, and allow custom input parameters or custom input data.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="551" LastActivityDate="2012-06-20T18:18:06.730" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2593" PostTypeId="2" ParentId="2591" CreationDate="2012-06-20T18:25:24.753" Score="13" Body="&lt;p&gt;Consider using the &lt;a href=&quot;http://www.hdfgroup.org/HDF5/&quot; rel=&quot;noreferrer&quot;&gt;HDF5&lt;/a&gt; file format.  HDF5 is a hierarchical data storage format with several nice features:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;platform independent storage: the library takes care of little/big endianness for you&lt;/li&gt;&#xA;&lt;li&gt;hierarchical layout of datasets: like a filesystem within a file&lt;/li&gt;&#xA;&lt;li&gt;large, growable n-dimensional array storage&lt;/li&gt;&#xA;&lt;li&gt;mixed dataset types can exist within one file (ie, integers, floats etc)&lt;/li&gt;&#xA;&lt;li&gt;automatic compression is available&lt;/li&gt;&#xA;&lt;li&gt;binary storage&lt;/li&gt;&#xA;&lt;li&gt;parallel i/o&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are C and Fortran interfaces, as well as Python (&lt;a href=&quot;http://code.google.com/p/h5py/&quot; rel=&quot;noreferrer&quot;&gt;h5py&lt;/a&gt; and &lt;a href=&quot;http://www.pytables.org/moin&quot; rel=&quot;noreferrer&quot;&gt;pytables&lt;/a&gt;) wrappers.  MATLAB can also read HDF5.  HDF5 is fairly flexible, almost to a fault, ie, it is not &quot;self-describing&quot;, hence the creation of &lt;a href=&quot;http://www.xdmf.org&quot; rel=&quot;noreferrer&quot;&gt;XDMF&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not exactly sure what you mean by &quot;conditions used for its simulation&quot;, but if these are just small collections of parameters you could store them as &lt;a href=&quot;https://support.hdfgroup.org/HDF5/doc1.6/UG/13_Attributes.html&quot; rel=&quot;noreferrer&quot;&gt;attributes&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="9667" LastEditDate="2017-11-02T13:43:57.740" LastActivityDate="2017-11-02T13:43:57.740" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2594" PostTypeId="2" ParentId="2541" CreationDate="2012-06-20T19:28:10.193" Score="5" Body="&lt;p&gt;Let's say you have a 2D grid with the X-axis running from ${0,1,...,i,...,M}$ and the Y-axis running from ${0,1,...,j,...,N}$. Each $i,j$ in a non-negative integer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your data over the grid can be viewed as a function of the grid locations $(i,j)$. In effect, the data $z = f(i,j)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you want the value of this function at $(i',j')$, where $i'=i+\delta{i}$ and $j'=j+\delta{j}$, such that $\delta{i}$ and $\delta{j}$ are the fractional decimal values between $0$ and $1$. Your problem is to find $z' = f(i',j') = f(i+\delta{i}, j+\delta{j})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several options for interpolating on such a grid. One of the simplest methods is the nearest neighbor interpolation. In this kind of interpolation, you simply assign to $(i',j')$, the value of the closest grid point. A naive way of doing this is to round $i'$ and $j'$ to the nearest integer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A slightly better interpolation scheme would use a weighted combination of its closest neighbors that lie on the grid. For example, with linear interpolation, you would use the four closest grid points $(i,j)$, $(i+1,j)$, $(i, j+1)$ and $(i+1,j+1)$ to find the appropriate interpolate value at $(i',j')$.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2012-06-20T19:28:10.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2595" PostTypeId="1" AcceptedAnswerId="2598" CreationDate="2012-06-20T20:21:38.873" Score="2" ViewCount="532" Body="&lt;p&gt;What is the quickest way to  find a point inside a linear feasible space? (Defined by the intersection of several hyperplanes and halfspaces). I want to be able to choose an initial point in the original convex space, discover a certain neighborhood around it (not convex but can be written as the union of some convex spaces defined by by the intersection of several halfspaces) using a procedure that depends on the point I have, and then I need to choose another point in the original space but not in the neighborhood already explored. I need to keep doing that until the space is exhausted (It should be exhausted eventually).&#xA;Basically, I have a convex space $S$, I need loop until $S=\phi$ while doing the following: Choose $x\in S$, Find $N$ around $x$, and then $S \leftarrow S - N$&#xA;Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1644" LastEditorUserId="123" LastEditDate="2012-06-23T16:15:15.063" LastActivityDate="2013-04-03T04:01:58.557" Title="Exploring feasible points in a linearly defined space" Tags="|linear-algebra|linear-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2596" PostTypeId="2" ParentId="2590" CreationDate="2012-06-20T21:33:56.987" Score="1" Body="&lt;p&gt;This is not my area, but I would recommend never the less to take a look into interDyMFoam solver of the OpenFOAM library. If I understood you right, you need body dynamics + two phase flow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A pseudo VoF two phase flow solver coupled with a sliding mesh interface and a 6DoF solver for the solid body motion == a box that floats on the water. Turbulence for two phase flow is still in development (the modelling part, and I mean in general: there are no rigorous turbulence models for two phase flows out there yet that I know of) but you can use the standard turbulence models with this solver without a problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the &lt;a href=&quot;http://powerlab.fsb.hr/ped/kturbo/openfoam/movies/VOF+6-DOF/box3dGGI.mpg&quot; rel=&quot;nofollow&quot;&gt;movie&lt;/a&gt;, it is large so it will break when you play it in your web browser. The parent directory containing papers, PhD theses, and other results is &lt;a href=&quot;http://powerlab.fsb.hr/ped/kturbo/openfoam/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-06-20T21:33:56.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2597" PostTypeId="2" ParentId="2583" CreationDate="2012-06-21T06:48:39.660" Score="4" Body="&lt;p&gt;If you're ok with thinking about things that are not eigenvalues but functions that in some sense still tell you something about the spectrum, then I think you should check out some of the work by Mark Embree at Rice University.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-21T06:48:39.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2598" PostTypeId="2" ParentId="2595" CreationDate="2012-06-21T08:46:07.310" Score="4" Body="&lt;p&gt;Finding a single feasible point is traditionally done by phase 1 of the simplex algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Simplex_algorithm#Finding_an_initial_canonical_tableau&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Simplex_algorithm#Finding_an_initial_canonical_tableau&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that you can do it by calling any routine for linear programming, just by putting the objective function to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Covering the feasible domain by balls of fixed radius $r$ whose midpoints are feasible is much harder, though, as exccluding a ball constitutes a nonconvex domain. There are algorithms for enumerating all vertices of a bounded polyhedron given by equations and inequalities (their number typically grows exponentially with the dimension, though). After having all vertices, the feasible set consists just of their convex combinations, so this can be used to samle inside. But unless the polyhedron is a simplex, different convex combinations may give the same point, so one would need to add a reject facility when generating a minimal covering.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-21T08:46:07.310" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2599" PostTypeId="2" ParentId="2525" CreationDate="2012-06-21T13:04:34.760" Score="4" Body="&lt;p&gt;If you're using LaTeX, a relatively low-tech solution is to have your code spit out a file (or else use a script to filter from your code's output) containing a whole bunch of lines like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;\newcommand{\myresults1}{&amp;lt;value&amp;gt;}&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can use the &lt;code&gt;\input&lt;/code&gt; command to add that file into your document, and use the defined commands to place the values.&lt;/p&gt;&#xA;" OwnerUserId="70" LastActivityDate="2012-06-21T13:04:34.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2600" PostTypeId="2" ParentId="83" CreationDate="2012-06-21T14:58:46.367" Score="1" Body="&lt;p&gt;for global problems you could be interested in &lt;a href=&quot;http://openopt.org/interalg&quot; rel=&quot;nofollow&quot;&gt;http://openopt.org/interalg&lt;/a&gt; and other openopt global solvers (http://openopt.org/GLP )&#xA;for local optimization openopt also provides variety of solvers: &lt;a href=&quot;http://openopt.org/NLP&quot; rel=&quot;nofollow&quot;&gt;http://openopt.org/NLP&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="1646" LastActivityDate="2012-06-21T14:58:46.367" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2601" PostTypeId="1" CreationDate="2012-06-21T16:15:48.800" Score="1" ViewCount="85" Body="&lt;p&gt;Consider a node $s$. Let's assume that there are three outgoing arcs from node $s$ namely $(s,i)$,$(s,j)$ and $(s,k)$. Corresponding to each of these arcs, there is a flow proportion value $t_{sj}\in (0,1]$. It means that the amount of flow along the arc $(s,j)$ should be $t_{sj} *f_{ps}$ in which $f_{ps}$ is flow over the single arc going to node $s$. Now in my LP I have the following constraints regarding node $s$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$f_{ps}  =  f_{si}+f_{sj}+f_{sk}$&lt;/li&gt;&#xA;&lt;li&gt;$ f_{si} =  t_{si} *f_{ps}$ &lt;/li&gt;&#xA;&lt;li&gt;$ f_{sj} =  t_{sj} *f_{ps}$ &lt;/li&gt;&#xA;&lt;li&gt;$ f_{sk} =  t_{sk} *f_{ps}$ &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In a special case, just assume that $\sum_j {t_{sj}}=1$. For example assume that $t_{sj}$ are probability values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my question is that how I can getting rid of these proportional equality constraints, i.e. constraints  2-4 in our given example. I am looking for a way to modify the structure of the network in such a way to include these constraints implicitly. &lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="486" LastEditDate="2012-06-21T16:43:45.987" LastActivityDate="2012-06-21T16:43:45.987" Title="Proportional equality constraints" Tags="|optimization|algorithms|linear-programming|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2602" PostTypeId="1" CreationDate="2012-06-21T18:32:44.347" Score="1" ViewCount="210" Body="&lt;p&gt;A linear program (LP)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{alignat}{1}&#xA;&amp;amp; \min_{x} {c}^{T}x, \\&#xA;\mathrm{s.t.} &amp;amp; \quad Ax = b, \\&#xA;&amp;amp; x \geq 0.&#xA;\end{alignat}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is called combinatorial if the size of entries of matrix $A \in \mathbb{R}^{m \times n}$ is bounded by a polynomial of dimension of the LP. (See &lt;a href=&quot;http://www.jstor.org/discover/10.2307/170819?uid=1008&amp;amp;uid=29684&amp;amp;uid=3739696&amp;amp;uid=2&amp;amp;uid=4&amp;amp;uid=62&amp;amp;uid=3739256&amp;amp;sid=47699100322467&quot; rel=&quot;nofollow&quot;&gt;Eva Tardós, &quot;A strongly polynomial algorithm to solve combinatorial linear programs&quot;, &lt;em&gt;Operations Research&lt;/em&gt; (1985)&lt;/a&gt;) The size of a rational number is described as the length of its binary representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems as though any entry size is a polynomial of the LP dimension. In other words, size($a_{ij}$) = size($a_{ij}$) * $m^0$ $n^0$ where size of LP is $m\times n$. So it seems like any LP is combinatorial. Why isn't this statement true?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="276" LastEditDate="2012-06-23T22:00:59.070" LastActivityDate="2012-06-26T05:27:10.980" Title="Why isn't every linear program combinatorial?" Tags="|optimization|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2603" PostTypeId="2" ParentId="2583" CreationDate="2012-06-21T19:12:37.783" Score="-1" Body="&lt;p&gt;You can always use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Gershgorin_circle_theorem&quot; rel=&quot;nofollow&quot;&gt;Gershgorin circle Theorem&lt;/a&gt; bounds to approximate the eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the off-diagonal terms are small, the diagonal itself is a good approximation of the spectrum. Otherwise if you end up with an approximation of the eigenspace (by other methods) you could try to express the diagonal entries in this system. This will lead to a matrix with smaller off-diagonal terms and the new diagonal will be a better approximation of the spectrum.&lt;/p&gt;&#xA;" OwnerUserId="1495" LastActivityDate="2012-06-21T19:12:37.783" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2604" PostTypeId="1" CreationDate="2012-06-21T19:50:41.517" Score="9" ViewCount="1614" Body="&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/870/how-to-find-conditional-lyapunov-exponents&quot;&gt;Answer&lt;/a&gt;  gives a software for calculating &lt;a href=&quot;https://en.wikipedia.org/wiki/Lyapunov_exponent#Conditional_Lyapunov_exponent&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional Lyapunov exponent&lt;/a&gt; (CLE) for coupled oscillators in chaos synchronization. However, it is hard to follow and there is no graphical output of the plot (and it is in C, more complex). Does anyone know how to modify the &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/233&quot; rel=&quot;nofollow noreferrer&quot;&gt;LET Toolbox&lt;/a&gt; which is very good for uncoupled system but I do not understand how to work with synchronized system so as to accommodate the CLE.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I have a confusion regarding how to include the driver signal while calculating the Jacobian matrix for CLE since theory says that CLE is found for the response system,so don't we have to find the Jacobian for the driver as well as the response system for similar oscillators(drive and response). Or should we consider both the drive and response system in the software and proceed as if it is a single system?&lt;/li&gt;&#xA;&lt;li&gt;How to accommodate an external forcing like a random process in a state equation if any in CLE.&lt;/li&gt;&#xA;&lt;li&gt;Is there any other implementation for CLE?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;thank you&lt;/p&gt;&#xA;" OwnerUserId="1628" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-11-29T14:35:03.300" Title="How to find Lyapunov exponent for coupled system" Tags="|matlab|ode|simulation|quantum-mechanics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2605" PostTypeId="2" ParentId="2574" CreationDate="2012-06-21T21:25:38.413" Score="5" Body="&lt;p&gt;For a positive semi-definite matrix such as $A = XX^T$ it may be worth the effort to accelerate convergence with &lt;a href=&quot;https://math.stackexchange.com/q/76439/3111&quot;&gt;a spectrum shift&lt;/a&gt;.  That is, a suitable scalar $\mu$ is chosen and the power method is applied to $A - \mu I$ instead of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few iterations of the basic power method should give you a rough estimate $||Ax||/||x||$ of the largest eigenvalue $\lambda_1$.  Assuming the dominant eigenvalue has multiplicity 1 and that all the others are in $[0,\frac{5}{6} \lambda_1]$, then $A - \frac{5}{12} \lambda_1 I$ would have a largest eigenvalue $\frac{7}{12} \lambda_1$ and the rest in $[\frac{-5}{12} \lambda_1, \frac{5}{12} \lambda_1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words you would increase the dominance of the largest eigenvalue from 20% over the next largest to 40% over the next largest (absolute value of an) eigenvalue.  Geometric convergence of the power method would accelerate accordingly.  Once the largest eigenvalue of $A - \mu I$ is found to sufficient accuracy, $\lambda_1$ is estimated by adding back the shift $\mu$ that had been taken away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you need not explicitly form $A - \mu I$ because $(A - \mu I)x = X(X^Tx) - \mu x$ can still be computed with $O(n^2)$ effort.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2012-06-21T21:25:38.413" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2606" PostTypeId="1" CreationDate="2012-06-22T05:33:37.160" Score="4" ViewCount="1120" Body="&lt;p&gt;I am seeking applications in the industry for the Moore-Penrose generalized inverse $A^\dagger$ of a matrix $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse&quot; rel=&quot;nofollow&quot;&gt;Moore-Penrose Inverse&lt;/a&gt; of $A\in \mathbb{C}^{m\times n}$, denoted&#xA;by $A^\dagger$, is the unique matrix $X$ satisfying the following four&#xA;Penrose equations.&#xA;\begin{eqnarray}&#xA; (i)~  AXA = A,~   (ii)~ XAX = X, ~ (iii)~ (AX)^* = AX,~ (iv)~  (XA)^* =&#xA; XA&#xA; \end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A^*$ denotes the conjugate transpose of a metrix.  Associated with $A^\dagger$  we can define orthogonal projection operators $AA^\dagger$ and $AA^\dagger$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references would be welcome.&lt;/p&gt;&#xA;" OwnerUserId="1580" LastEditorUserId="119" LastEditDate="2012-06-23T13:58:07.813" LastActivityDate="2012-06-23T14:33:31.013" Title="Applications of Moore - Penrose generalized inverse of a matrix and associated projection?" Tags="|linear-algebra|reference-request|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2607" PostTypeId="1" CreationDate="2012-06-22T09:59:01.357" Score="6" ViewCount="4416" Body="&lt;p&gt;I have a linear programming problem. I want to know if this LP is feasible. What is the best known algorithm for checking feasibility of an LP or a linear system of equations?&lt;/p&gt;&#xA;" OwnerUserId="917" LastActivityDate="2022-09-16T20:00:44.397" Title="LP feasibility checking" Tags="|optimization|linear-programming|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2608" PostTypeId="2" ParentId="2607" CreationDate="2012-06-22T10:23:15.337" Score="5" Body="&lt;p&gt;In general it is &lt;strong&gt;Phase I of the simplex method.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using a black-box solver, remove the objective function or set all the coefficients to zero in it (same effect as removing the objective) then call the LP solver. The solver will stop after finding the first feasible solution or it will conclude that the problem is infeasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The wording &lt;em&gt;&quot;the best known algorithm&quot;&lt;/em&gt; is a little unfortunate. Unless you specify the context, it is impossible to give you an accurate, general answer. Well, I've tried...&lt;/p&gt;&#xA;" OwnerUserId="679" LastActivityDate="2012-06-22T10:23:15.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2609" PostTypeId="2" ParentId="2607" CreationDate="2012-06-22T11:06:07.643" Score="11" Body="&lt;p&gt;Checking feasibility of an LP and solving an LP are essentially equivalent problems, as one can be transformed into the other by standard methods changing the complexity by a constant factor only. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best-known algorithms are the simplex method and interior point methods. For both, there are numerous variations. Which of the two approaches is most useful depends on the size and sparsity pattern of the matrix defining the LP.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="20688" LastEditDate="2019-06-19T00:01:57.363" LastActivityDate="2019-06-19T00:01:57.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2610" PostTypeId="1" CreationDate="2012-06-22T12:20:25.467" Score="2" ViewCount="685" Body="&lt;p&gt;I have a system of linear equations in form of $AX=b$ where $A_{m\times n}$, $X_{n\times 1}$ and $b_{m\times 1}$. Coefficient matrix $A$ is quite sparse. However, using a practical LP solver like LINGO it is clear that after permutation of rows, it turns out to be like a lower triangular matrix. I do not know what the permutation function should be. Since the matrix dimension is not square, I cannot use the LU decomposition to solve the system efficiently. Can you please let me know an efficient method for linear systems with non-square coefficient matrix?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="123" LastEditDate="2012-06-22T15:54:58.240" LastActivityDate="2012-06-22T15:54:58.240" Title="Finding a permutation that makes a matrix lower triangular" Tags="|linear-algebra|optimization|linear-programming|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2611" PostTypeId="2" ParentId="2606" CreationDate="2012-06-22T16:08:19.003" Score="3" Body="&lt;p&gt;Some uses of the pseudoinverse are in &lt;a href=&quot;http://www.siam.org/search/?type=1&amp;amp;terms=pseudoinverse&amp;amp;search.x=0&amp;amp;search.y=0&quot; rel=&quot;nofollow&quot;&gt;http://www.siam.org/search/?type=1&amp;amp;terms=pseudoinverse&amp;amp;search.x=0&amp;amp;search.y=0&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But one can use the singular-value decomposition (SVD) whenever one can use the pseudoinverse, and the SVD is much more flexible to apply, and therefore much more used. Moreover, the pseudoinverse is numerically unstable and must be regularized in practice, and this is usually done via the SVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The SVD is ubiquitous in applications, too numerous to even give a sensible partial list.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-22T16:08:19.003" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2612" PostTypeId="2" ParentId="2606" CreationDate="2012-06-22T17:07:32.260" Score="3" Body="&lt;p&gt;The standard references for learning about generalized inverses (not just Moore-Penrose generalized inverses) are &lt;em&gt;Generalized Inverses: Theory and Applications&lt;/em&gt; by Ben-Israel and Greville and &lt;em&gt;Generalized Inverses of Linear Transformations&lt;/em&gt; by Campbell and Meyer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One typically uses a QR decomposition (such as the $XY$ representation of projectors in &lt;a href=&quot;http://epubs.siam.org/action/showAbstract?page=309&amp;amp;volume=32&amp;amp;issue=1&amp;amp;journalCode=sjmael&quot; rel=&quot;nofollow&quot;&gt;&quot;On the Numerical Analysis of Oblique Projectors&quot;, G. W. Stewart, SIMAX, 2011&lt;/a&gt;) or an SVD to calculate a pair of generalized inverses in practice, since the methods suggested in the textbooks above (normally some variant of LU decomposition) are, as Professor Neumaier points out, numerically unstable. In contrast, the QR- and SVD-based methods are far more stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although not typically identified as such, generalized inverses play a role in any method identified as &quot;Petrov-Galerkin&quot; (or &quot;Galerkin&quot;). The orthogonality relation between the two relevant subspaces (these end up being the range and nullspace of a projector) is induced by a bilinear form; the two orthogonal subspaces determine a (nonunique) pair of generalized inverses. Such methods play a role in the numerical solution of partial differential equations, numerical sparse linear algebra (which you can see in the popular textbook &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Iterative Methods for Sparse Linear Systems&lt;/em&gt; by Yousef Saad&lt;/a&gt;, and model (order) reduction methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ben-Israel and Greville discuss applications of generalized inverses such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;integral solutions of linear equations&lt;/li&gt;&#xA;&lt;li&gt;linear programmming&lt;/li&gt;&#xA;&lt;li&gt;least-squares solutions of inconsistent linear systems&lt;/li&gt;&#xA;&lt;li&gt;Tikohonov Regularization&lt;/li&gt;&#xA;&lt;li&gt;Difference equations&lt;/li&gt;&#xA;&lt;li&gt;and others...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-06-22T17:07:32.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2613" PostTypeId="2" ParentId="2466" CreationDate="2012-06-22T20:46:14.143" Score="4" Body="&lt;p&gt;I honestly don't know how efficient this is, but one way to do it is with c++ templates:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The order is k, t is the knot structure, and x is the value you want.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;int k&amp;gt; &#xA;real BSpline(real x, real *t)&#xA;{&#xA;    if (*t &amp;lt;= x &amp;amp;&amp;amp; x &amp;lt; *(t+k))&#xA;    {&#xA;        real a = (x - *t) / (*(t+k-1) - *t);&#xA;        real b = (*(t+k) - x) / (*(t+k) - *(t+1));&#xA;&#xA;        return a * BSpline&amp;lt;k-1&amp;gt;(x, t) + b * BSpline&amp;lt;k-1&amp;gt;(x, (t+1));&#xA;    }&#xA;    else&#xA;        return 0;&#xA;};&#xA;&#xA;template &amp;lt;&amp;gt;&#xA;real BSpline&amp;lt;1&amp;gt;(real x, real *t)&#xA;{&#xA;    if (*t &amp;lt;= x &amp;amp;&amp;amp; x &amp;lt; *(t+1))&#xA;        return 1.;&#xA;    else&#xA;        return 0.;&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35" LastActivityDate="2012-06-22T20:46:14.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2614" PostTypeId="2" ParentId="2583" CreationDate="2012-06-22T20:52:44.737" Score="2" Body="&lt;p&gt;Here's yet another way to characterize the spectrum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given an eigenvalue problem $\mathbf{A} v_k = \lambda_k v_k$ (assume real&#xA;symmetric $\mathbf{A}$ and separated eigenvalues; although the latter is probably not&#xA;necessary), let's attempt to estimate the smeared spectral density&#xA;$$ &#xA;S(\omega) &#xA;= \sum_k \frac{\pi^{-1}\sigma}{\sigma^2 + (\lambda_k - \omega)^2}&#xA;= \frac{\sigma}{\pi} \mathop{\mathrm{Tr}} [\sigma^2 + (\omega - \mathbf{A})^2]^{-1}&#xA;$$&#xA;After hitting e.g. &lt;a href=&quot;http://dx.doi.org/10.1016/0377-0427(96)00018-0&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1016/0377-0427(96)00018-0&lt;/a&gt; in a&#xA;literature search, we know that an unbiased Monte Carlo estimator to the trace is &#xA;$$&#xA;S(\omega)&#xA;= \frac{\sigma}{\pi}\langle z^T [\sigma^2 + (\omega - \mathbf{A})^2]^{-1} z \rangle&#xA;$$&#xA;where each entry of the random vector $z$ contains either $+1$ or $-1$ with&#xA;probability 0.5 for each. For given $\sigma$ and $\omega$, the inverse product&#xA;$[\sigma^2 + (\omega - \mathbf{A})^2]^{-1} z$ can be computed for example with&#xA;the conjugate gradient method, or sparse LU on $[\omega + i \sigma - \mathbf{A}]^{-1} [\omega - i \sigma - \mathbf{A}]^{-1}$ to minimize fill-in. This allows estimation of $S(\omega)$ also for large matrices. In practice, it seems the CG solution doesn't need to be very&#xA;accurate, and neither are many vectors necessary in computing the average. This&#xA;may depend on the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above appears to weigh parts of the spectrum more evenly than a similarly&#xA;smeared Krylov spectral density --- try diag(linspace(0, 1, 150000)) --- although maybe there is a way to correct for this?.  This is somewhat&#xA;similar to the pseudospectral approach, but the result indicates the (smeared) number of eigenvalues in the vicinity to point $\omega$, rather than &#xA;the inverse distance to the nearest eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: A better performing alternative for computing the above quantity is to compute Chebyshev moments (via similar stochastic evaluation as above) and then reconstruct the spectral density from them. This requires neither matrix inversions nor separate computations for each $\omega$. See  &lt;a href=&quot;http://theorie2.physik.uni-greifswald.de/downloads/publications/LNP_chapter19.pdf&quot; rel=&quot;nofollow&quot;&gt;http://theorie2.physik.uni-greifswald.de/downloads/publications/LNP_chapter19.pdf&lt;/a&gt; and references therein.&lt;/p&gt;&#xA;" OwnerUserId="1645" LastEditorUserId="1645" LastEditDate="2012-06-25T20:24:42.333" LastActivityDate="2012-06-25T20:24:42.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2615" PostTypeId="2" ParentId="2008" CreationDate="2012-06-23T14:19:25.937" Score="6" Body="&lt;p&gt;I don't know if the following helps, but for me it was very insightful to visualize the scaling behavior of quantum systems:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problem comes from the fact that the Hilbert space of quantum states grows exponentially with the number of particles. This can be seen very easily in discrete systems. Think of a couple of potential wells that are conected to each other, may just two: well 1 and well 2. Now add bosons (e.g., Rubidium 87, just as an example), at first only one. How many possible basis vectors are there? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;basis vector 1: boson in well 1&lt;/li&gt;&#xA;&lt;li&gt;basis vector 2: boson in well 2&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;They can be written like $\left|1,0 \right\rangle$ and $\left|0,1 \right\rangle$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose the boson can hop (or tunnel) from one well to the other. The Hamiltonian that describes the system can then be written is matrix notation as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\hat{H}=\pmatrix{ \epsilon_{1} &amp;amp; t \\ t &amp;amp; \epsilon_{2}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\epsilon_{1,2}$ are just the energies of the boson in well 1 and 2, respectively, and &lt;em&gt;t&lt;/em&gt; the tunneling amplitude. &#xA;The complete solution of this system, i.e., the solution containing all the information necessary to compute the system's state at any given point of time (given an initial condition), is given by the eigenstates and eigenvalues. The eigenstates are linear superpositions of the basis vectors (in this case $\left|1,0 \right\rangle$ and $\left|0,1 \right\rangle$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is so simple that it can be solved by hand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose we have more potential wells and more bosons, e.g., in the case of four wells with two bosons there are 10 different possibilities to distribute the bosons among the wells. Then the Hamiltonian would have 10x10=100 elements and 10 eigenstates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can quickly see that the number of eigenstates is given by the binomial coefficient:&#xA;$$&#xA;\text{number of eigenstates}=\pmatrix{\text{number of wells} + \text{number of bosons} - 1 \\ \text{number of bosons}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So even for &quot;just&quot; ten bosons and ten different potential wells (a very small system), we'd have 92,378 eigenstates. The size of the Hamiltonian is then $92,378^2$ (approximately 8.5 &lt;em&gt;billion&lt;/em&gt; elements). In a computer they'd occupy (depending on your system) about 70 gigabytes of RAM and is therefore probably impossible to solve on most computers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's assume we have a continuous system (i.e. no potential wells, but free space) and 13 water molecules (for simplicity I treat them each of them as a particle). Now in computer we can still model free space using many tiny potential wells (we discretize space... which is ok, as long as the relevant physics takes place on larger length scales then the discretization length). Let's say there are 100 different possible positions for each of the molecules in each of the x, y and z directions. So we end up with 100*100*100 = 1,000,000 little boxes. Then we'd have more than $2.7 \cdot 10^{53}$ basis vectors, the Hamiltonian would have almost $10^{107}$ elements, occupying so much space that we'd need all the particles from 10 million universes like ours just to encode that information.&lt;/p&gt;&#xA;" OwnerUserId="1636" LastActivityDate="2012-06-23T14:19:25.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2616" PostTypeId="2" ParentId="2606" CreationDate="2012-06-23T14:33:31.013" Score="2" Body="&lt;p&gt;Another application of the pseudoinverse is in preconditioning saddle point problems. Given a saddle point matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = \begin{pmatrix} F &amp;amp; B^T \\ B &amp;amp; 0 \end{pmatrix},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;arising in incompressible flow, &lt;a href=&quot;http://chess.cs.umd.edu/~elman/papers/bfbt.pdf&quot; rel=&quot;nofollow&quot;&gt;Elman's &quot;BFBt&quot; preconditioner&lt;/a&gt; approximates the inverse of the Schur complement&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S = B F^{-1} B^T$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S_{BFBt}^{-1} = (BB^T)^{-1} B F B^T (BB^T)^{-1} .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is essentially application of the Moore-Penrose pseudoinverse considering the identities (when $BB^T$ is nonsingular)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align} B^\dagger &amp;amp;= B^T (BB^T)^{-1} \\ (B^T)^\dagger &amp;amp;= (BB^T)^{-1} B .\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This preconditioner was subsequently improved by moving the least squares argument from the discrete to the continuous setting, resulting in new diagonal scaling terms that significantly improve performance, see &lt;a href=&quot;https://www.cs.umd.edu/~elman/papers/spac.pdf&quot; rel=&quot;nofollow&quot;&gt;Elman et al (2006), &lt;em&gt;Block preconditioners based on approximate commutators&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-06-23T14:33:31.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2618" PostTypeId="1" CreationDate="2012-06-23T23:15:20.147" Score="4" ViewCount="3442" Body="&lt;p&gt;I have a problem where I need to diagonalize a large number of small Hermitian matrices. Typically the matrices are between 4 and 64 in size (skewed towards the low end) and the number of matrices is on the order of millions. This is then repeated many times, so I have been trying to figure out the best way to go about this.&#xA;Most of the performance information I've been able to gather seems to focus on &#xA;the case for matrices that are significantly larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is:&#xA;Is there an algorithm (or implementation thereof) that is particularly suited for diagonalizing small (Hermitian) matrices?&lt;/p&gt;&#xA;" OwnerUserId="1654" LastEditorUserId="1654" LastEditDate="2012-06-24T01:43:37.877" LastActivityDate="2012-06-24T18:08:32.980" Title="What is the most efficient way to diagonalize small matrices?" Tags="|linear-algebra|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2619" PostTypeId="2" ParentId="2618" CreationDate="2012-06-24T09:44:19.043" Score="7" Body="&lt;p&gt;The most efficient package for platform adapted dense linear algebra (including eigensolvers) is the LAPACK library. &lt;a href=&quot;http://en.wikipedia.org/wiki/LAPACK&quot;&gt;http://en.wikipedia.org/wiki/LAPACK&lt;/a&gt;&#xA;Every improvement in the state of the art is fairly soon reflected in this library, and a lot of effort is spent in tuning the kernels to optimal performance on all platforms.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-06-24T09:44:19.043" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2620" PostTypeId="2" ParentId="2618" CreationDate="2012-06-24T18:08:32.980" Score="8" Body="&lt;p&gt;Let me add a few remarks explaining why there is no better method for &quot;small&quot; matrices in the range described (order 4 to 64) than the usual approach: &lt;a href=&quot;http://elemental.googlecode.com/hg/doc/build/html/lapack-like/condensed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tridiagonalize the Hermitian matrix&lt;/a&gt; with a unitary similarity transform (a direct step), then iteratively apply QR-like operations to approach a diagonal limit.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the eigenvalues (diagonalized matrices) alone are needed, the reduction to tridiagonal form can be carried out with Householder transformations using $2n^3/3$ (complex) multiplications, or $4n^3/3$ multiplications if the eigenvectors (unitary similarity transforms) are also needed.  This reduction step's complexity dominates that of the iterative steps, which is $O(n \log^3 n)$ &quot;even in the case of high precision&quot; (&lt;a href=&quot;http://www.cs.duke.edu/~reif/paper/roots.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reif,1999&lt;/a&gt;) if only eigenvalues are needed (increase by a factor of $n$ if eigenvectors too are needed).  So in any case the usual combined complexity is $O(n^3)$ with a modest leading coefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding eigenvalues is of course theoretically equivalent to getting roots of a characteristic polynomial, and their effective equivalence is outlined (with links to references) in the early portion of &lt;a href=&quot;https://scicomp.stackexchange.com/a/1885/651&quot;&gt;this previous Answer&lt;/a&gt;.  The rest of that Answer explains why direct methods for orders 6 and above are impractical compared with the tried-and-true iterative approach(es).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 2x2 matrix it certainly makes sense to use &lt;a href=&quot;http://en.wikipedia.org/wiki/Quadratic_equation#Floating_point_implementation&quot; rel=&quot;nofollow noreferrer&quot;&gt;the stable form of the quadratic formula&lt;/a&gt; to get the (real) eigenvalues directly.  Probably a case can be made for using &lt;a href=&quot;http://www.math.vanderbilt.edu/~schectex/courses/cubic/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cardano's formula for roots of a cubic&lt;/a&gt; to solve eigenvalues of a 3x3 matrix, or perhaps &lt;a href=&quot;http://en.wikipedia.org/wiki/Cubic_function#Trigonometric_.28and_hyperbolic.29_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;the formulas of Viète&lt;/a&gt; if we are to avoid complex arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there the &quot;garden path&quot; of direct methods comes to a sudden narrowing!  Although the general quartic can be solved by radicals (&lt;a href=&quot;http://en.wikipedia.org/wiki/Quartic_equation#The_general_case.2C_along_Ferrari.27s_lines&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ferrari's method&lt;/a&gt;) and the general quintic polynomial by &lt;a href=&quot;http://en.wikipedia.org/wiki/Bring_radical#Solution_of_the_general_quintic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bring radicals&lt;/a&gt;, these methods require solving &quot;auxiliary polynomials&quot; of lower degrees.  Such cumbersomeness is compounded by complex arithmetic and monitoring for &lt;a href=&quot;http://en.wikipedia.org/wiki/Loss_of_significance&quot; rel=&quot;nofollow noreferrer&quot;&gt;loss of significance&lt;/a&gt; in a floating point implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation for order 6 and above is even more inauspicious because in place of the single argument Bring radical, exotic functions of at least two arguments are needed.  Moreover the known methods that could be coded for variable degrees $n$ involve an element of trial-and-error to locate &lt;em&gt;all&lt;/em&gt; roots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast the usual approach adapts to variable orders $n$ with no apparent difficulty and takes efficient advantage of the smallness of $n$ in both time and space.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-06-24T18:08:32.980" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2621" PostTypeId="2" ParentId="2590" CreationDate="2012-06-25T08:03:47.897" Score="3" Body="&lt;p&gt;I'm working with SPH and what you describe will not be suitable for the following reasons&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SPH is a low order method (1st at most)&lt;/li&gt;&#xA;&lt;li&gt;2 phase flow can be done with SPH but is generally very costly due to the restrictions coming from the different speed of sound&lt;/li&gt;&#xA;&lt;li&gt;With 1000 cores you will not be doing a RE 10k DNS&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;other than that LES is probably the way forward.&lt;/p&gt;&#xA;" OwnerUserId="1176" LastActivityDate="2012-06-25T08:03:47.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2623" PostTypeId="1" AcceptedAnswerId="2628" CreationDate="2012-06-25T22:11:58.097" Score="3" ViewCount="126" Body="&lt;p&gt;I have a a system of bivariate polynomials as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ E(u,v): e_2(u) v^2 + e_1(u) v + e_0(v) = 0 \\&#xA;  F(u,v): f_2(u) v^2 + f_1(u) v + f_0(v) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e_n(u) = e_{n_2}u^2 + e_{n_1} u + e_{n_0}$ and similarly $f_n(u) = f_{n_2}u^2 + f_{n_1} u + f_{n_0}$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Thus far, I have solved via a Sylvester Matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the Sylvester Matrix $\mathbf{G}$ be defined as follows:&#xA;\begin{align}&#xA; \mathbf{G} = \begin{bmatrix}&#xA;               e_{2}(u) &amp;amp; e_{1}(u) &amp;amp; e_{0}(u) &amp;amp;        0 \\&#xA;                      0 &amp;amp; e_{2}(u) &amp;amp; e_{1}(u) &amp;amp; e_{0}(u) \\&#xA;               f_{2}(u) &amp;amp; f_{1}(u) &amp;amp; f_{0}(u) &amp;amp;        0 \\&#xA;                      0 &amp;amp; f_{2}(u) &amp;amp; f_{1}(u) &amp;amp; f_{0}(u)&#xA;              \end{bmatrix}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possible solutions for $u$ may be found by finding the determinant of $\mathbf{G}$, which will be a polynomial of degree 8, i.e.:&#xA;\begin{align}&#xA; G_u &amp;amp;= \det\left( \mathbf{G} \right) \\&#xA;     &amp;amp;= \sum_{i=0}^{8} G_{u_i} u^i&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Real roots of $G_u$ may or may not be solutions of $E$ and $F$. To determine whether each real root, $u_i$ is a solution, substitute $u_i$ into $E$ and $F$ and solve for roots $v_{E_{i,j}}, j \in \{1, 2\}$ and $v_{F_{i,j}}, j \in \{1, 2\}$ respectively. Where there is a common real solution to $v$, i.e. &#xA;\begin{equation}&#xA; v_{E_{i,j}} = v_{F_{i,j}}&#xA;\end{equation}&#xA;then the pair $\left( u_i, v_{E_{i,j}} \right)$ is a solution.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The problem with this method is that is appears to be horribly numerically unstable. Ultimately, I needed to used &lt;a href=&quot;http://www.holoborodko.com/pavel/mpfr/&quot; rel=&quot;nofollow&quot;&gt;MPFR&lt;/a&gt; and &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt; with a crazy number of bits (I think 500 or so) to get correct answers - double was nowhere near good enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, fine variations of the coefficients of $G(u)$ caused by rounding or other numerical stability issues significantly moved the zeros of the polynomial, or in some instances, caused no real solutions at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this instance, the real solutions are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Solutions: &#xA;u = 283.118152022138386048081228332, v = 253.676766248666064015102940087&#xA;u = 283.531719376538811255222357595, v = 251.92153962772290946172986145&#xA;u = 305.922640195704152315990688064, v = 277.085127660836580117988944869&#xA;u = 307.305710607492739248674271296, v = 275.328781581725038016656158251&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the original bivariate quadratic is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;e_2_2 = 2.96184510308460152308099163079e-05&#xA;e_2_1 = -0.0174923195188134962339249161373&#xA;e_2_0 = 2.58392910782947268103166838601&#xA;&#xA;e_1_2 = -0.0156683206373072309167532302994&#xA;e_1_1 = 9.24288932470428612919593805961&#xA;e_1_0 = -1363.76987336669518990414286879&#xA;&#xA;e_0_2 = 2.07344822818614405777141090841&#xA;e_0_1 = -1221.73998971149758113804476542&#xA;e_0_0 = 180057.979202842850602362010181&#xA;&#xA;f_2_2 = 0.0057280951738335050897549908978&#xA;f_2_1 = -3.37584828933150293352939673891&#xA;f_2_0 = 497.643484709582863520217198114&#xA;&#xA;f_1_2 = -3.02706498792933011045380441656&#xA;f_1_1 = 1781.95310671648910949906364811&#xA;f_1_0 = -262381.909585190823129099681101&#xA;&#xA;f_0_2 = 400.16454659182969104246501941&#xA;f_0_1 = -235295.984529511565861184026453&#xA;f_0_0 = 34606200.151155688437914904126&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the $G(u)$ coefficients are (from $G_8 u^8 + ... + G_0$):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;3.96025792973163717278031784693e-13&#xA;-9.33800975227555950436364008271e-10&#xA;9.63557996744215866033970248179e-07&#xA;-0.000568300516844107153566292037895&#xA;0.209542890052501016806589147067&#xA;-49.4609340406527708387460620101&#xA;7298.7082158003621329697479114&#xA;-615609.87547890187406435557892&#xA;22722530.2050891103731763098963&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What techniques (or code ;) ) will help me out?&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="761" LastEditDate="2012-06-25T23:08:29.530" LastActivityDate="2012-06-27T14:04:32.997" Title="Numerically stable real solution(s) to a system of bivariate quadratics" Tags="|stability|polynomials|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2624" PostTypeId="1" AcceptedAnswerId="2627" CreationDate="2012-06-26T02:16:27.510" Score="15" ViewCount="5671" Body="&lt;p&gt;Suppose I have a 2D mesh consisting of nonoverlapping triangles $\{T_k\}_{k=1}^N$, and a set of points $\{p_i\}_{i=1}^M \subset \cup_{k=1}^N T_K$. What is the best way to determine which triangle each of the points lies in? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the following image we have $p_1 \in T_2$, $p_2 \in T_4$, $p_3 \in T_2$, so I would like a function $f$ that returns the list $f(p_1,p_2,p_3)=[2,4,2]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/dAMnt.png&quot; alt=&quot;enter image description here&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab has the function &lt;a href=&quot;http://www.mathworks.com/help/techdoc/ref/delaunaytri.pointlocation.html&quot; rel=&quot;noreferrer&quot;&gt;pointlocation&lt;/a&gt; which does what I want for Delaunay meshes, but it fails for general meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first (dumb) thought is, for all nodes $p_i$, loop through all the triangles to find out which triangle $p_i$ is in. However, this is is extremely inefficient - you might have to loop through every triangle for every point, so it could take $O(N \cdot M)$ work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My next thought is, for all points $p_i$, find the nearest mesh node via nearest-neighbor search, then look through triangles attached to that nearest node. In this case, the work would be $O(a\cdot M\cdot log(N))$, where $a$ is the maximum number of triangles attached to any node in the mesh. There are a couple solvable but annoying issues with this approach,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It requires implementing an efficient nearest-neighbor search (or finding a library that has it), which could be a nontrivial task. &lt;/li&gt;&#xA;&lt;li&gt;It requires storing a list of which triangles are attached to each node, which my code is currently not set up for - right now there is just a list of node coordinates and a list of elements.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Altogether it seems inelegant, and I think there should be a better way. This must be a problem that arises a lot, so I was wondering if anyone could recommend the best way to approach finding what triangles the nodes are in, either theoretically or in terms of available libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2012-07-02T18:58:16.977" Title="Finding which triangles points are in" Tags="|finite-element|computational-geometry|unstructured-mesh|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2625" PostTypeId="2" ParentId="2602" CreationDate="2012-06-26T05:27:10.980" Score="4" Body="&lt;p&gt;This is certainly true for any individual problem, but this sort of statement is typically made about a &lt;em&gt;family&lt;/em&gt; of LPs. In that case, the dimensions of the matrix &lt;em&gt;and&lt;/em&gt; the entries of the matrix are typically expressed in terms of other parameters $\theta = (\theta_1, \theta_2, \dotsc, \theta_k)$ for some $k \geq 1$ - i.e., we have $n(\theta)$ and $m(\theta)$. If the size of each $A_{ij}$ can be given as an expression (in $\theta$) that is bounded by a polynomial in the expressions $n(\theta)$ and $m(\theta)$, then we have a combinatorial LP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your argument is somewhat similar to saying, &quot;Once I am given a graph on $n$ vertices, $n$ is a constant, so I can enumerate all $n!$ tours in constant time. Therefore I can solve the travelling salesman problem on that graph in polynomial time. But this holds for every graph. Hence P=LP.&quot;&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2012-06-26T05:27:10.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2626" PostTypeId="2" ParentId="2624" CreationDate="2012-06-26T05:42:40.670" Score="8" Body="&lt;p&gt;I'm not convinced your solution is actually correct. Consider the situation where you have these nodes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A: (-3, 1)&lt;/li&gt;&#xA;&lt;li&gt;B: (0, 2)&lt;/li&gt;&#xA;&lt;li&gt;C: (3, 1)&lt;/li&gt;&#xA;&lt;li&gt;D: (0, -5)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are triangles ABC and ACD. Now B is the closest point to the origin, but the origin is in triangle ACD, which doesn't contain B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the rest, this answer provides a solution that may in degenerate cases be as slow as $O(N \cdot M)$, but will typically be faster - especially for Delaunay triangulations and triangulations close to that in some sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would consider the option of building a &lt;a href=&quot;http://en.wikipedia.org/wiki/Quadtree&quot;&gt;quadtree&lt;/a&gt; that contains the triangles themselves. I.e., you have a quaternary tree that stores in each node (which corresponds to a bounding box):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The coordinates at which the box is being split, or alternatively, the bounding boxes of the four subtrees;&lt;/li&gt;&#xA;&lt;li&gt;Pointers to the subtrees;&lt;/li&gt;&#xA;&lt;li&gt;The set of triangles that fall completely within the bounding box of this rectangle, but not completely within any of the four subtrees. In other words, the triangles that intersect with any of the two subdividing line segments of the quadtree.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When given a point P, traverse all nodes on the path from the root of the quadtree to the smallest box containing P. You'll need to examine all triangles that you encounter in those nodes. For a 'well-behaved' triangulation, there should only be something like $\sqrt{n}$ triangles that need to be examined at the level of a node that contains $n$ triangles in its subtree, and the depth should be limited by $\log n$. For a 'badly behaved' triangulation, you may get the $O(N\cdot M)$ work.&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2012-06-26T05:42:40.670" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2627" PostTypeId="2" ParentId="2624" CreationDate="2012-06-26T05:51:42.610" Score="16" Body="&lt;p&gt;The usual randomized edge hopping method should work. Basically, start with any triangle of the mesh, then determine which of the edges the target point lies on the opposite side of. That is, determine which of the edges, when extended out to a line, separate the point from the interior of the triangle. When there are two possibilities, choose one at random, and consider the triangle that is adjacent to that shared edge, and repeat. The randomization should make this method converge with probability 1 for Delaunay triangulations, and I can't think of a reason it would not work for arbitrary triangulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: I should add that edge hopping should be $O(\log N)$ with a reasonable constant for a single point, so it would be $O(M \log N)$ for $M$ points. However, if you sort your points by locality (like using a Hilbert curve ordering first), you can initialize each new query with the triangle of the previous query, to further reduce the runtime (I'm not a CS theorist so I can't tell you what the big-O would be there).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit2&lt;/strong&gt;: Found &lt;a href=&quot;http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-728.pdf&quot;&gt;this PDF&lt;/a&gt; describing such a &quot;walking&quot; scheme that is guaranteed to terminate, and reviews the more naive approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative to using quadtrees is using a Triangulation Hierarchy. See&#xA;Olivier Devillers. Improved incremental randomized Delaunay triangulation. In Proc. 14th Annu. ACM Sympos. Comput. Geom., pages 106-115, 1998.&#xA;It works best for Delaunay triangulations, but can also work for non-Delaunay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically whatever you do to speed up point location will require the construction of an auxiliary data structure. In the case of quadtrees or some other spatial subdivision, you need to build the subdivision tree. In the case of edge-hopping, you need to build the triangle adjacent topological structure. The triangulation hierarchy also requires building a tree of coarser triangulations.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2012-06-26T06:05:02.733" LastActivityDate="2012-06-26T06:05:02.733" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2628" PostTypeId="2" ParentId="2623" CreationDate="2012-06-26T12:14:47.057" Score="2" Body="&lt;p&gt;To find all the solutions of a polynomial system of equations with the usual accuracy expectable from floating-point calculations, use a homotopy algorithm that exploits the polynomial structure. See, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Morgan and A. Sommese, &#xA;Computing all solutions to polynomial systems using homotopy continuation, Applied Mathematics and Computation 24 (1987), 115-138.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and there is an associated software package called HOMPACK: &#xA;(source: &lt;a href=&quot;http://www.netlib.org/hompack/&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/hompack/&lt;/a&gt; ;&#xA;paper: &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=279235&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=279235&lt;/a&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, use a branch and bound package such as ICOS &lt;a href=&quot;https://sites.google.com/site/ylebbah/icos&quot; rel=&quot;nofollow&quot;&gt;https://sites.google.com/site/ylebbah/icos&lt;/a&gt; .&#xA;This gives guarantees even in the presence of rounding errors, and you can add inequality constraints to restrict to a desired domain.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-06-27T14:04:32.997" LastActivityDate="2012-06-27T14:04:32.997" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2629" PostTypeId="2" ParentId="2300" CreationDate="2012-06-26T14:00:26.930" Score="5" Body="&lt;p&gt;You say in the comment that you cannot get it to work as it is not quadratic enough. I don't see any reason for that. The problem is easily coded as a mixed-integer quadratic program. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand your problem definition, you want to constraint the sum of the varables larger than a threshold. Introduce a binary variable indicating if x is larger than a, and introduce another variable z which should be equal to x when this holds, and zero otherwise, and use the sum of the new variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the MATLAB toolbox YALMIP to interface CPLEX (or Gurobi or any other MIQP solver), the problem is trivially solved in fractions of a second. Here, a random example, implemented using both a manually derived model, and a model which exploits the high-level modelling capabilities in YALMIP&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Create random data&#xA;N = 100;&#xA;y = rand(N,1);&#xA;a = rand(N,1);&#xA;b = rand(1);&#xA;&#xA;% Decision variables&#xA;x = sdpvar(N,1);&#xA;z = sdpvar(N,1);&#xA;d = binvar(N,1);&#xA;&#xA;% Define objective&#xA;Objective = (x-y)'*(x-y)&#xA;&#xA;% High-level model&#xA;Con1 = [0 &amp;lt;= x &amp;lt;= 1,0 &amp;lt;= z &amp;lt;=1];&#xA;Con2 = [implies(x-a&amp;gt;=0,d), implies(d,z==x), implies(1-d,z==0)]&#xA;Con3 = [sum(z) &amp;lt;= b];&#xA;&#xA;% Solve problem&#xA;solvesdp([Con1,Con2,Con3],Objective)&#xA;&#xA;% display solution&#xA;[double(x) a double(z)]&#xA;&#xA;% Manually derived model&#xA;Con1 = [0 &amp;lt;= x &amp;lt;= 1,0 &amp;lt;= z &amp;lt;=1];&#xA;Con2 = [x-a &amp;lt;= d, -(1-d) &amp;lt;= x-z &amp;lt;= 1-d, z &amp;lt;= d];&#xA;Con3 = [sum(z) &amp;lt;= b];&#xA;Objective = (x-y)'*(x-y)&#xA;solvesdp([Con1,Con2,Con3],Objective)&#xA;[double(x) a double(z)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2012-06-26T14:00:26.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2630" PostTypeId="1" CreationDate="2012-06-26T15:28:16.117" Score="12" ViewCount="1330" Body="&lt;p&gt;The following matrix equation &#xA;$$B\Sigma + \Sigma B^T + C = 0$$&#xA;in $\Sigma$ $-$ for given $B$ and $C$ matrices $-$ appears in my work as a characterization of a covariance matrix. I have learned that this equation is known, in particularly in continuous time control theory, as &lt;strong&gt;Lyapunov's equation&lt;/strong&gt;, and that there are various well known algorithms for solving it that exploit the special nature of this linear equation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From googling I have also learned that there exist Matlab and Fortran implementations. I have found SLICOT and RECSY. Due to licensing issues access to SLICOT source has been stopped, though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of my work is implemented in R, and as I have been unable to find an R interface to a solver, I consider writing one myself. My question is then if SLICOT is the best available Fortran (or C) library with an implementation of a solver of Lyapunov's equation? I am also interested in implementations that can handle large sparse $B$ matrices.  &lt;/p&gt;&#xA;" OwnerUserId="44" LastEditorUserId="4244" LastEditDate="2013-05-29T13:00:00.467" LastActivityDate="2022-06-30T19:19:38.350" Title="Libraries for solving Lyapunov's equation" Tags="|matrix|matrix-equations|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2631" PostTypeId="2" ParentId="2463" CreationDate="2012-06-26T17:03:08.297" Score="1" Body="&lt;p&gt;Well, you are sort of on the right track. There is no magic to solving these problems using interior-point methods, they are iterative in nature and based on linearizations etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a typical interior-point algorithm for these problems would not take a step in the gradient direction, but compute second-order information too, and thus take a Newton step. Hence, a line-search is performed along the Newton direction, an optimal step is taken, and the procedure is repeated. Once converged (too some suitably defined precision), the barrier parameter is decreased (with some suitable factor), and the procedure is repeated again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get it to work in practice, you have to carefully think about how you do the parameter updates, and most often you would perform everything in the primal-dual space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you google for sillysdp, you find a simple implementation of a SDP solver implementing, essentially, example 11.9 in the Boyd &amp;amp; Vandenberghe reference which you are reading. Maybe a start for inspiration (since semidefinite programming is a generalization of an SOCP which you are solving, neglecting the logdet term in the objective, which doesn't complicate things really)&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2012-06-26T17:03:08.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2632" PostTypeId="2" ParentId="2624" CreationDate="2012-06-27T03:56:37.817" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://www.cgal.org&quot; rel=&quot;nofollow&quot;&gt;CGAL&lt;/a&gt; handles triangulations and point location (and a great deal more).  In particular, the &lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Triangulation_2/Chapter_main.html&quot; rel=&quot;nofollow&quot;&gt;2D triangulation&lt;/a&gt; module can do what you want.&lt;/p&gt;&#xA;" OwnerUserId="1665" LastActivityDate="2012-06-27T03:56:37.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2633" PostTypeId="1" CreationDate="2012-06-27T11:14:41.063" Score="0" ViewCount="88" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Possible Duplicate:&lt;/strong&gt;&lt;br&gt;&#xA;  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2544/get-equation-for-a-curve-which-intersects-x-at-seemingly-randomly-distributed-po&quot;&gt;Get equation for a curve which intersects x at seemingly randomly distributed points?&lt;/a&gt;  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;I tried &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2544/get-equation-for-a-curve-which-intersects-x-at-seemingly-randomly-distributed-po&quot;&gt;asking something similar to this before&lt;/a&gt;, but I guess I didn't explain very well. Hopefully this will be more clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to get an equation which describes a curve with certain points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I have a list of 5-10 different points along the x-axis where a curve either touches or crosses the x-axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how can I get an equation to describe any curve (at least one of the possible curves) which satisfies this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm only concerned with x values within the range of the given values. So, if the curve interacts with the x-axis at other points outside this range, that would be ok too, it doesn't matter....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to put it another way,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if y = 0; and 0 &amp;lt; x &amp;lt; 100; also x = (list of given points eg. 0,4,17,19,56,67,100)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't matter what the other x values are when y ≠ 0. And it doesn't matter what other x values might be outside the given range (in this case when x &gt; 100).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any website or free software which can take the coordinates and spit out an equation? Or is there a method you can explain, which I can use to write my own program for doing so?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more clear, I'm given a list of points all where y=0, How can I get an equation such that, someone else could look at the equation alone, without also seeing the points I used to produce the equation, but then be able to use that equation to get those same points.... to tell me where the line crosses x within the range I specify?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also,&#xA;If multiple possibilities exist, the shorter length of equation the better.&lt;/p&gt;&#xA;" OwnerUserId="1621" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-06-27T16:08:39.270" Title="Produce equation of curve, given some coordinates" Tags="|algorithms|" AnswerCount="0" CommentCount="10" ClosedDate="2012-06-28T14:20:23.573" ContentLicense="CC BY-SA 3.0" />
  <row Id="2634" PostTypeId="1" AcceptedAnswerId="2635" CreationDate="2012-06-27T12:19:30.723" Score="13" ViewCount="5496" Body="&lt;p&gt;I have a function $f(x,y,z)$ such that&lt;br&gt;&#xA;$\int_{R^3} f(x,y,z)dV$&lt;br&gt;&#xA;is finite, and I want to approximate this integral.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm familiar with quadrature rules and monte carlo approximations of integrals, but I see some difficulties implementing them with in an infinite domain.  In the monte carlo case, how does one go about sampling an infinite region (especially if the regions that contribute more significantly to the integral are unknown)?  In the quadrature case, how do I find the optimal points?  Should I simply fix an arbitrarily large region centered around the origin and apply sparse quadrature rules?  How can I go about approximating this integral?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-06-28T08:24:34.123" LastActivityDate="2012-06-30T01:35:46.093" Title="How can I approximate an improper integral?" Tags="|monte-carlo|quadrature|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2635" PostTypeId="2" ParentId="2634" CreationDate="2012-06-27T12:30:22.910" Score="21" Body="&lt;p&gt;In one dimension, you can map your infinite interval to a finite interval using integration by substitution, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_a^b f(x)\,\mathrm dx \quad=\quad \int_{u^{-1}(a)}^{u^{-1}(b)}f(u(t))u'(t)\,\mathrm dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u(x)$ is some function that goes off to infinity in some finite range, e.g. $\tan(x)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{-\infty}^{\infty}f(x)\,\mathrm dx \quad=\quad 2\int_{-\pi/2}^{\pi/2} f(\tan(t))\frac{1}{\cos(2t)+1} \,\mathrm dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then use any regular numerical quadrature routine for the modified, finite integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substitution for multiple variables is a bit trickier, but is quite well described &lt;a href=&quot;http://en.wikipedia.org/wiki/Integration_by_substitution#Substitution_for_multiple_variables&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="127" LastEditDate="2012-06-30T01:35:46.093" LastActivityDate="2012-06-30T01:35:46.093" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2636" PostTypeId="2" ParentId="2634" CreationDate="2012-06-27T14:10:59.603" Score="11" Body="&lt;p&gt;The standard way of doing it is to extract from the expression for $f(x)$ an exponential prefactor, transform that to $e^{-x^2}$, and then use Gaussian quadrature rules (or Gauss Kronrod) with this as a weight. If $f$ is smooth, this usually gives excellent results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In $R^3$, the same works with weight $e^{-|x|^2}$, and appropriate cubature formulas can be found, e.g., in the book by Engels, numerical quadrature and cubature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Online formulas are at &lt;a href=&quot;http://nines.cs.kuleuven.be/ecf/&quot;&gt;http://nines.cs.kuleuven.be/ecf/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-06-27T14:17:11.357" LastActivityDate="2012-06-27T14:17:11.357" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2637" PostTypeId="2" ParentId="2634" CreationDate="2012-06-27T14:23:56.150" Score="7" Body="&lt;p&gt;For one-dimensional quadrature, you can check the book on &lt;a href=&quot;http://www.netlib.org/quadpack/&quot;&gt;Quadpack&lt;/a&gt; (a golden oldie but still very relevant in one-dimensional quadrature) and the techniques used in the algorithm QAGI, an automatic integrator for an infinite range. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another technique is the double-exponential quadrature formula, nicely implemented for an infinite interval by &lt;a href=&quot;http://www.kurims.kyoto-u.ac.jp/~ooura/intde.html&quot;&gt;Ooura&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For cubature, you can consult the &lt;a href=&quot;http://nines.cs.kuleuven.be/ecf/&quot;&gt;Encyclopedia of cubature formulas&lt;/a&gt; by Ronald Cools. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-06-27T14:23:56.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2639" PostTypeId="2" ParentId="2634" CreationDate="2012-06-27T16:02:29.183" Score="4" Body="&lt;p&gt;If you want to use Monte Carlo integration, you could start by using &lt;a href=&quot;http://en.wikipedia.org/wiki/Importance_sampling&quot; rel=&quot;nofollow&quot;&gt;importance sampling&lt;/a&gt; with a sampler that roughly approximates your integrand. The better your sampler matches your integrand, the less variance in your integral estimates. It doesn't matter than your domain is infinite as long as your sampler has the same domain.&lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2012-06-27T16:02:29.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2640" PostTypeId="2" ParentId="2634" CreationDate="2012-06-27T19:53:30.400" Score="6" Body="&lt;p&gt;If you remember how quadrature worked, then you'll also know one way how to approximate infinite integrals. Namely: for quadrature, you approximate the function $f(x)$ you want to integrate by something similar, say a polynomial $\tilde f(x)$ (or a piecewise polynomial) for which you can write down the integral analytically. You get $\tilde f$ from $f$ by interpolation at interpolation points -- which will then be your quadrature points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For infinite integrals, one approach uses the exact same thinking. For example, you could try to approximate $f(x)$ on the entire line using a simpler function, e.g. $\tilde f(x)=e^{-x^2}p(x)$ with a polynomial $p(x)$ that interpolates $f(x)e^{x^2}$ at a number of points. There are then simple formulas for computing the integral $\int_{-\infty}^{\infty} \tilde f(x) dx$. The choice of interpolation points follows similar logic as is done for the usual quadrature derivation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-27T19:53:30.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2641" PostTypeId="1" AcceptedAnswerId="2650" CreationDate="2012-06-27T22:06:30.893" Score="13" ViewCount="6840" Body="&lt;p&gt;I would like to know a good method for interpolating data between two unstructured grids, where one grid is a coarser version of the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Efficiency is very important to me since I'm solving a transient PDE problem where I need to transfer data between the grids at every time step of the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about using kd-tree for searching the nearest node of a given point, then I would use the shape functions of that element (FEM simulation) to interpolate the data. Is this a good solution? Are there better ones?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you also know any robust and reliable library in C/C++ for this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*I know there is a similar question, but it asks for the most accurate method on a structured grid.&lt;/p&gt;&#xA;" OwnerUserId="484" LastEditorUserId="549" LastEditDate="2012-06-28T08:29:04.210" LastActivityDate="2012-06-28T18:09:57.460" Title="Efficient interpolation method for unstructured grids?" Tags="|interpolation|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2642" PostTypeId="2" ParentId="2641" CreationDate="2012-06-27T22:55:16.683" Score="6" Body="&lt;p&gt;If I understand you correctly, you want to fill in the values of the finer grid by interpolating on the coarser grid.  One way to do linear interpolation on an unstructured grid is with Delaunay triangulations (this is how Matlab's griddata and TriScatteredInterp commands are implemented).  After constructing a triangulation of your grid points, interpolation boils down to locating the triangle containing the target point, calculating its barycentric coordinates, and using the function values at the vertices to compute the interpolated value.  &lt;a href=&quot;http://www.cgal.org/&quot;&gt;CGAL&lt;/a&gt; can construct n-dimensional triangulations (for medium n), and also has a built-in &lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Interpolation/Chapter_main.html&quot;&gt;2d interpolation&lt;/a&gt; module.&lt;/p&gt;&#xA;" OwnerUserId="1665" LastActivityDate="2012-06-27T22:55:16.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2643" PostTypeId="2" ParentId="2641" CreationDate="2012-06-28T00:12:39.517" Score="2" Body="&lt;p&gt;This is the kind of job for which you really want to avoid unstructured meshes since there is no way to do this efficiently for any significant number of points. You should consider using meshes that are at least somehow related to each. For example, if they are both obtained from hierarchical refinement of a coarse mesh, then you can relatively easily and efficiently find out where the interpolation points of one mesh are located on the other mesh.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-06-28T00:12:39.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2644" PostTypeId="5" CreationDate="2012-06-28T10:42:37.603" Score="0" Body="" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-06-28T10:42:37.603" LastActivityDate="2012-06-28T10:42:37.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2645" PostTypeId="4" CreationDate="2012-06-28T10:42:37.603" Score="0" Body="Regression analysis is the process of measuring and establishing a relationship between a dependent variable and one or more independent variables." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-06-28T10:42:37.603" LastActivityDate="2012-06-28T10:42:37.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2646" PostTypeId="5" CreationDate="2012-06-28T10:48:39.093" Score="0" Body="" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-06-28T10:48:39.093" LastActivityDate="2012-06-28T10:48:39.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2647" PostTypeId="4" CreationDate="2012-06-28T10:48:39.093" Score="0" Body="For questions regarding the numerical treatment of processes whose behaviors are determined by both deterministic (predictable) and non-deterministic (random) actions." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-06-28T10:48:39.093" LastActivityDate="2012-06-28T10:48:39.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2648" PostTypeId="1" AcceptedAnswerId="2649" CreationDate="2012-06-28T12:36:37.803" Score="7" ViewCount="708" Body="&lt;p&gt;Where can I find a library to solve a sparse complex matrix equation iteratively in C.&#xA;So far I've only found libraries for direct solution to complex systems, and libraries for iterative solutions to real systems, but no solvers for iterative solutions to complex systems&lt;/p&gt;&#xA;" OwnerUserId="1671" LastActivityDate="2012-06-28T12:56:13.083" Title="C library - iterative sparse complex linear equation solver?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2649" PostTypeId="2" ParentId="2648" CreationDate="2012-06-28T12:56:13.083" Score="6" Body="&lt;p&gt;PETSc (http://www.mcs.anl.gov/petsc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you have to build PETSc specifically for solving complex systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at installation instructions for complex version:&#xA;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/installation.html&quot;&gt;http://www.mcs.anl.gov/petsc/documentation/installation.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="484" LastActivityDate="2012-06-28T12:56:13.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2650" PostTypeId="2" ParentId="2641" CreationDate="2012-06-28T17:47:24.743" Score="7" Body="&lt;p&gt;Unstructured grids have their place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to look at the Earth System Modeling Framework (ESMF). They have some code for re-gridding -- specifically for this purpose -- and they've done some nifty stuff with parallel code, too. The whole system is designed to couple models, so there may be other useful stuff there as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other notes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;no way to do this efficiently for any significant number of points&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;well, efficient is a relative thing -- once you've got the grid in a tree structure, you can search it in O(logn), which can be pretty darn fast, though not O(1), as searching a regular grid is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it sounds like while the interpolation needs to be done at every time step, if the grids aren't adapting, then the mapping from one grid to another remains constant. So you can compute that mapping (i.e. which element in each grid corresponds to which element in the other) in whatever way is convenient, store it, and then you never need to compute it agin (until the grids change).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That leaves you with the interpolation code -- where you will want to balance accuracy with performance -- simple linear interpolation across a triangle is fast, and may be good enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;I thought about using kd-tree for searching the nearest node of a given point, then I would use the shape functions of that element&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;remember that the nearest node doesn't get you the element -- so you'll want to do a bit more to find the element you want. One option would be to use an rtree instead, which stores/searches by bounding box -- you'll get more than one element with each search, but you can then check which of those is correct directly.&lt;/p&gt;&#xA;" OwnerUserId="1676" LastActivityDate="2012-06-28T17:47:24.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2651" PostTypeId="2" ParentId="2641" CreationDate="2012-06-28T18:09:57.460" Score="3" Body="&lt;p&gt;This is what I'm doing at the moment, except I'm transferring function values at quadrature points, not nodes. I'm implementing the technique explained in the chosen answer to my question here: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2624/finding-which-triangles-points-are-in&quot;&gt;Finding which triangles points are in&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, say you have 2 grids $A$ and $B$, and want to transfer information from grid $A$ to grid $B$&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Based on node or quadrature points in grid $B$, construct a list of points $p_i$ to be evaluated in grid $A$,&lt;/li&gt;&#xA;&lt;li&gt;order the evaluation points $p_i$ based on a Hilbert curve,&lt;/li&gt;&#xA;&lt;li&gt;construct the constrained Delaunay extension of grid $A$ (all triangles in $A$, plus some more exterior triangles to make the shape convex)&lt;/li&gt;&#xA;&lt;li&gt;starting at a triangle in the extended version of $A$, randomly walk from triangle to triangle &quot;in the general direction&quot; of point $p_1$ until you get there. Then keep walking to $p2$, $p3$, etc. until you have found which triangles in $A$ contain all the evaluation points.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you have $N$ evaluation points and $M$ triangles in the extended version of $A$, and if you ignore the Hilbert curve ordering, and you walk deterministically, worst case the time could be $O(N \sqrt{M})$. Including the Hilbert curve ordering and walking with randomness, I'm seeing much more efficient results in practice, more like $O(max(N,M))$.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-06-28T18:09:57.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2654" PostTypeId="1" AcceptedAnswerId="2709" CreationDate="2012-06-29T10:30:38.203" Score="3" ViewCount="3274" Body="&lt;p&gt;When the absolute pressure becomes negative or $U$ exceeds the speed of light, things have pretty obviously gone wrong (be that bad boundary conditions, a too coarse mesh, a too large timestep etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is there any way to have a simulation abort with an error message when values exceed definable non-physical ranges?&lt;/p&gt;&#xA;" OwnerUserId="89" LastActivityDate="2012-07-06T07:54:33.993" Title="How to let OpenFOAM abort a simulation when values exceed a given range?" Tags="|openfoam|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2655" PostTypeId="2" ParentId="2654" CreationDate="2012-06-29T16:44:41.470" Score="7" Body="&lt;p&gt;This is basic C++ - just add whatever you are interested in to the code and recompile. For example, copy and paste the following code right before the &lt;code&gt;runTime.write();&lt;/code&gt; call (which is at the end of the &lt;code&gt;main&lt;/code&gt; function of every solver) to execute the sanity check at the end of every time step:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// sanity check&#xA;if (min(p) &amp;lt; 0)&#xA;{&#xA;    FatalError&#xA;        &amp;lt;&amp;lt; &quot;Negative pressure&quot; &amp;lt;&amp;lt; nl&#xA;        &amp;lt;&amp;lt; exit(FatalError);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="59" LastEditorUserId="59" LastEditDate="2012-07-02T07:48:16.063" LastActivityDate="2012-07-02T07:48:16.063" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2656" PostTypeId="1" AcceptedAnswerId="2657" CreationDate="2012-06-30T19:20:43.430" Score="7" ViewCount="6752" Body="&lt;p&gt;This complements an earlier question on &lt;a href=&quot;https://scicomp.stackexchange.com/questions/351/recommendations-for-a-usable-fast-c-matrix-library&quot;&gt;usable, fast C++ matrix libraries&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked at the &lt;a href=&quot;http://code.google.com/p/java-matrix-benchmark/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Java Matrix Benchmark&lt;/a&gt;, and it seems like the performance of java matrix libraries is all over the place. The Apache Commons Math library seems like a simple solution, but I'm curious for others recommendations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I'm curious which libraries are much easier/more performant than the others, and/or if Java library writers are better off just using JNI to call to an existing, better developed native matrix libraries. (Though that seems like it could make the code less portable or more of a hassle to setup.) &lt;/p&gt;&#xA;" OwnerUserId="1420" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-11-21T01:37:01.207" Title="Recommendations for a usable, fast Java matrix library?" Tags="|linear-algebra|software|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2657" PostTypeId="2" ParentId="2656" CreationDate="2012-06-30T19:48:31.750" Score="7" Body="&lt;p&gt;In general, you are almost always better off using the JNI if you are concerned about performance.  A JNI interface to a fast BLAS is implemented by the &lt;a href=&quot;http://software.intel.com/en-us/articles/performance-tools-for-software-developers-how-do-i-use-intel-mkl-with-java/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel Math Kernel Library&lt;/a&gt;, which would be option if you are considering a cross-platform commercial offering.  If you are looking for a fast open source offering, you might want to consider &lt;a href=&quot;http://jblas.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jblas&lt;/a&gt;, which tries to be multi-platform (with a few holes), and wraps ATLAS, a reasonably fast open source implementations of the BLAS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your mileage will vary based on the specific matrix operations you are performing and how big the matrices are.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Related Questions:&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2486/parallel-scientific-computation-software-development-language&quot;&gt;Parallel Scientific Computation Software Development Language?&lt;/a&gt; - &lt;em&gt;A discussion of comparisons and considerations for using Java against other programming languages.&lt;/em&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-06-30T19:57:17.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2658" PostTypeId="5" CreationDate="2012-06-30T20:00:28.303" Score="0" Body="&lt;p&gt;Although Java is not commonly used in high performance scientific computing, it remains one of the most popular programming languages for implementing large-scale software and is found in many industrial applications.  Although general questions about Java are not on-topic here, any questions relating to technical computing with Java are more than welcome!&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-06-30T20:00:28.303" LastActivityDate="2012-06-30T20:00:28.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2659" PostTypeId="4" CreationDate="2012-06-30T20:00:28.303" Score="0" Body="Questions about technical computing with the Java programming language" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-06-30T20:00:28.303" LastActivityDate="2012-06-30T20:00:28.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2660" PostTypeId="1" AcceptedAnswerId="2661" CreationDate="2012-06-30T15:47:35.410" Score="5" ViewCount="1271" Body="&lt;p&gt;I need to compute the helicity modulus as a function of temperature for a three-dimensional X-Y model (see &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0375960196006822&quot; rel=&quot;nofollow&quot;&gt;N.K. Kultanov, Yu.E. Lozovik, &quot;The critical behavior of the 3D X-Y model and its relation with fractal properties of the vortex excitations&quot;&lt;/a&gt;). I am resorting to using a Monte Carlo simulation because I cannot find a closed expression for the superfluid density in a 3-D MC model in the literature. I have written a simple Monte Carlo simulation and I am experiencing some problems, i.e. long thermalization times and noisy results even when averaging on a huge number of simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since my code is simple, are there any tricks I could use to make the computation more efficient? References and source code would be very much appreciated. I am primarily interested in the 3-D case, since my code seems to work well for the 2-D case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone is interested I've uploaded my code (in C) here: &lt;a href=&quot;http://pastebin.com/ScrMEaQ8&quot; rel=&quot;nofollow&quot;&gt;http://pastebin.com/ScrMEaQ8&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7560" OwnerDisplayName="zakk" LastEditorUserId="276" LastEditDate="2012-06-30T21:14:13.010" LastActivityDate="2012-07-01T15:55:17.013" Title="Monte Carlo simulation of 3D X-Y model" Tags="|monte-carlo|simulation|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2661" PostTypeId="2" ParentId="2660" CreationDate="2012-06-30T17:28:32.993" Score="9" Body="&lt;p&gt;The long thermalization time that you're running into is a generic problem that typically goes under the name &quot;critical slowing down&quot; and is common to the &lt;em&gt;local-update&lt;/em&gt; scheme that you're using (you update by locally changing a single spin at a time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you realize that, the way out is to do better sampling - local updates are out so you have to invent global updates. Two great ways of doing this are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Cluster updates using the Wolff algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://prl.aps.org/abstract/PRL/v62/i4/p361_1&quot;&gt;http://prl.aps.org/abstract/PRL/v62/i4/p361_1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Sorry, no free version available)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Find a &quot;dual&quot; model to the one you're studying, with the same partition function but new degrees of freedom. For Ising and XY models if you play this game you end up with &quot;bond-current&quot; models where the currents are updated globally using a worm algorithm detailed nicely in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/cond-mat/0103146v1&quot;&gt;http://arxiv.org/abs/cond-mat/0103146v1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oh! And as I googled for that Prokofiev reference, I see that someone even implemented it with cute graphics in Mathematica!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://demonstrations.wolfram.com/WormAlgorithmForJCurrentModel/&quot;&gt;http://demonstrations.wolfram.com/WormAlgorithmForJCurrentModel/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1689" OwnerDisplayName="wsc" LastActivityDate="2012-06-30T17:28:32.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2662" PostTypeId="1" AcceptedAnswerId="2666" CreationDate="2012-06-30T21:23:01.120" Score="14" ViewCount="598" Body="&lt;p&gt;In the numerical solution of initial boundary value PDEs, it is very common to employ &lt;em&gt;parallelism in space&lt;/em&gt;.  It is much less common to employ some form of &lt;em&gt;parallelism in the time discretization&lt;/em&gt;, and that parallelism is usually much more limited.  I'm aware of an increasing number of codes and published works demonstrating temporal parallelism, but none of them include spatial parallelism.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there examples of implementations that include parallelism in both space and time?  I'm interested in both publications and available codes.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-08-23T02:53:43.567" Title="Examples of PDE computations using parallelism in both space and time" Tags="|pde|parallel-computing|time-integration|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2663" PostTypeId="2" ParentId="2662" CreationDate="2012-06-30T22:38:41.767" Score="1" Body="&lt;p&gt;Have a look at the Parareal algorithm and its related work like spectral deferred correction (a simple Google search turns up quite a lot of material). The basic idea is to use a coarse &quot;mesh&quot; in time and do a rough time stepping, but then go back over it and perform corrections on a finer time scale. It seems to be used mostly in fluid simulations, but I'm in the area of electromagnetics, so I can't really say much more about it. The only reason I know about it is because I attended a seminar on the deferred correction approach and it seemed very interesting that any kind of parallelization could be done in time.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-06-30T22:38:41.767" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2664" PostTypeId="2" ParentId="2662" CreationDate="2012-06-30T23:40:46.823" Score="4" Body="&lt;p&gt;There are also space-time DG and continuous Galerkin methods. After choice of quadrature, space-time DG with a structured grid in the time direction is equivalent to an implicit Runge-Kutta method. The space-time DG method, however, allows for different step sizes in different parts of the domain, a case that is difficult to analyze for implicit RK methods. Space-time multigrid methods can also be applied in this context.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-06-30T23:40:46.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2665" PostTypeId="1" AcceptedAnswerId="2672" CreationDate="2012-07-01T03:11:54.483" Score="4" ViewCount="432" Body="&lt;p&gt;I am reading the following &lt;a href=&quot;http://math.fullerton.edu/mathews/n2003/neldermead/NelderMeadProof.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;, that explain the Nelder-Mead optimization Algorithm.(Algorithm Below) Where $B$ is the best point, $G$ second best point, $W$ is the worst point, $R$ reflection point. Why in Case(i) $W$ is replaced by $R$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that $f(R)&amp;lt;f(G)$ than we go in $Case(i)$. If $f(B) &amp;lt; f(R)$ is true, than $f(B)&amp;lt;f(R)&amp;lt;f(G)$, in other words $R$ is better choice than $G$, but worst than $B$. According to the algorithm, $R$ should replace $W$, which doesn't make sense.&#xA;Either the paper has an error, or I don't understand something. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7V9lJ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1575" LastActivityDate="2016-09-20T16:10:12.687" Title="Nelder-Mead optimization Algorithm" Tags="|optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2666" PostTypeId="2" ParentId="2662" CreationDate="2012-07-01T03:19:05.220" Score="8" Body="&lt;p&gt;The &lt;a href=&quot;http://www.unc.edu/~mwemmett/pfasst/&quot; rel=&quot;nofollow&quot;&gt;PFASST&lt;/a&gt; (Parallel Full Approximation Scheme in Space and Time) and &lt;a href=&quot;http://wwwgsb.fz-juelich.de/ias/jsc/EN/AboutUs/Organisation/ComputationalScience/Simlabs/slpp/SoftwarePEPC/_node.html&quot; rel=&quot;nofollow&quot;&gt;PEPC&lt;/a&gt; (Pretty Efficient Parallel Coulomb) algorithms have recently been used together to achieve parallelism in both space and time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PFASST does the time parallelism, PEPC does the space parallelism.  The results of this were recently presented at the &lt;a href=&quot;http://dd21.inria.fr/&quot; rel=&quot;nofollow&quot;&gt;DD21&lt;/a&gt; conference, and we have prepared a submission for &lt;a href=&quot;http://sc12.supercomputing.org/&quot; rel=&quot;nofollow&quot;&gt;SC12&lt;/a&gt; describing the combination of PFASST+PEPC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;small&quot; problem consisting of 4 million particles (PEPC is a parallel N-body solver) was shown to scale well up to 8192 cores on &lt;a href=&quot;http://www2.fz-juelich.de/jsc/jugene&quot; rel=&quot;nofollow&quot;&gt;JUGENE&lt;/a&gt; using only PEPC (ie, only parallel in space).  Beyond this, communication costs became significant and the parallel efficiency began to degrade.  The addition of PFASST allows this fixed sized problem to be run on 262,144 cores (ie, we filled up JUGENE) by using 32 &quot;time&quot; processors (each of which consists of 8192 &quot;spatial&quot; cores).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the parallel efficiency of time-parallel algorithms isn't 100%, we were able to obtain speedups of about 6.5x using 32 PFASST processors with this PFASST+PEPC configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a link to a preprint: &lt;a href=&quot;http://icsweb.inf.unisi.ch/cms/index.php/publications.html?view=publication&amp;amp;task=show&amp;amp;id=415&quot; rel=&quot;nofollow&quot;&gt;A massively space-time parallel N-body solver&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="486" LastEditDate="2012-08-23T02:53:43.567" LastActivityDate="2012-08-23T02:53:43.567" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2667" PostTypeId="2" ParentId="2662" CreationDate="2012-07-01T08:44:44.420" Score="2" Body="&lt;p&gt;Once you consider space-time parallelsim, the subdomain is space-time over multiple time levels.  A method called waveform relaxation exploits space-time subdomains but parallelize only in space (no partition in time dimension).  So a cartesian of the space partition and time partition gives a kind of space-time parallelism.  You can find a paper on such a cartesian method &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/gander_m_mini_17.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.  As Jed Brown mentioned in his answer,  space-time method not only gives more flexible parallelsim but also adaptivity for discretization.  On the latter topic, you can google works of Schwab, see also their &lt;a href=&quot;http://www.sam.math.ethz.ch/projects/schwab_space_time&quot; rel=&quot;nofollow&quot;&gt;project&lt;/a&gt;.  For the work exploiting both parallelism and adaptivity, you can watch at R. Haynes' &lt;a href=&quot;http://www.math.mun.ca/~rhaynes/research.html&quot; rel=&quot;nofollow&quot;&gt;homepage&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2012-07-01T08:44:44.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2668" PostTypeId="2" ParentId="2486" CreationDate="2012-07-01T09:04:40.113" Score="2" Body="&lt;p&gt;There's been little discussion of coarray fortran in the above comments. At this time, and to my limited knowledge, coarray support in compilers are roughly as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cray has a compiler which supports at least the basic coarray features. I've used it to write code that was intended to be &quot;educational&quot;, but I'd say that you could write real code in coarray fortran. The syntax and concepts are mostly much simpler than MPI, but as always, there are lotsa traps, and the traps are different from MPI.&lt;/li&gt;&#xA;&lt;li&gt;Intel fortran has coarray support built on top of their MPI library. Supposedly this limits their theoretical peak performance, but I haven't seen any metrics.&lt;/li&gt;&#xA;&lt;li&gt;Gfortran supports coarrays, but only for a single image (or single rank, in MPI speak). Hence, no real parallelization is available until gfortran 4.8 or 4.9 is out.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Generally, I'd be careful if starting a coarray based code. The syntax is simple and much more convenient than Fortran/C/C++ with MPI, but then, it's just not as full-featured. For instance, MPI supports a &lt;em&gt;lot&lt;/em&gt; of reduction operations etc. which could be very convenient for you. It would really depend on your need for a lot of communication. If you want an example, let me know and I can provide you with a few, if I can dig up the files.&lt;/p&gt;&#xA;" OwnerUserId="1691" LastActivityDate="2012-07-01T09:04:40.113" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2669" PostTypeId="1" AcceptedAnswerId="2682" CreationDate="2012-07-01T04:36:33.950" Score="6" ViewCount="4030" Body="&lt;p&gt;How can I solve the following Linear Programming problem in the easiest way?&lt;/p&gt;&#xA;&lt;p&gt;How can the summation terms be modeled in there?&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/doTPD.png&quot; alt=&quot;From page 187 of Mathematical Programming Approaches&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;From page 187 of &lt;a href=&quot;https://doi.org/10.1007/978-0-387-68155-9_8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematical Programming Approaches&lt;/a&gt; by Charu Chandra and Janis Grabis.&lt;/p&gt;&#xA;" OwnerUserId="1692" OwnerDisplayName="Reza" LastEditorDisplayName="user43608" LastEditDate="2022-07-24T15:48:29.107" LastActivityDate="2022-07-24T15:48:29.107" Title="Is there a convenient solver for linear programming with many summations?" Tags="|optimization|" AnswerCount="5" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2670" PostTypeId="2" ParentId="2669" CreationDate="2012-07-01T05:49:30.217" Score="1" Body="&lt;p&gt;The problem you have is more an OR problem. As I see, you have a target function and some constraints. IBM ILOG CPLEX Optimization Studio, also known as CPLEX, is a professional software to solve these types of questions. SAS also does optimization and simulation.&#xA;Hope this answer helps. &lt;/p&gt;&#xA;" OwnerDisplayName="Sepehr" LastActivityDate="2012-07-01T05:49:30.217" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2671" PostTypeId="2" ParentId="2669" CreationDate="2012-07-01T11:14:37.390" Score="5" Body="&lt;p&gt;Any Linear Program (LP) can be solved by a variety of freely available software. The sigma notation can be converted to linear algebra notation $Ax=b$, which is what will be expected by the LP solver unless you are using a more advanced optimization design framework. You asked about MATLAB, which contains a Linear Programming solver in its &lt;a href=&quot;http://www.mathworks.com/help/toolbox/optim/ug/linprog.html&quot; rel=&quot;noreferrer&quot;&gt;Optimization Toolbox&lt;/a&gt;, which is available under separate license.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be interested in &lt;a href=&quot;http://cvxr.com/cvx/download/&quot; rel=&quot;noreferrer&quot;&gt;CVX&lt;/a&gt;, a freely available toolbox for MATLAB that can solve LPs amongst a more general class of optimization problems.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to pursue a completely open source route, the two best options I'm aware of are &lt;a href=&quot;http://www.coin-or.org/projects/Clp.xml&quot; rel=&quot;noreferrer&quot;&gt;CLP&lt;/a&gt; and the &lt;a href=&quot;http://www.gnu.org/software/glpk&quot; rel=&quot;noreferrer&quot;&gt;GLPK&lt;/a&gt;.  If you have MATLAB, I recommend starting with CVX.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-07-01T11:14:37.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2672" PostTypeId="2" ParentId="2665" CreationDate="2012-07-01T12:58:11.243" Score="4" Body="&lt;p&gt;R isn't the &quot;reflection point&quot; but the &quot;reflected point&quot;, i.e., it lies on the other side of the simplex as viewed from the worst point. It should then all make sense.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-01T12:58:11.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2673" PostTypeId="2" ParentId="2662" CreationDate="2012-07-01T13:01:13.033" Score="1" Body="&lt;p&gt;The multiple shooting method used in optimal control is designed in such a way that the sub-problems on each shooting interval can be solved in parallel. I don't know of papers that couple this with spatial parallelism (there are not that many optimal control problems that have been solved in the past where the equation is a time-dependent spatial PDE) but it would be obvious how to do parallelism in both space and time.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-01T13:01:13.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2674" PostTypeId="2" ParentId="2660" CreationDate="2012-07-01T15:55:17.013" Score="3" Body="&lt;p&gt;Just taking a quick look at your code, there are a few things that could be improved.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You use three random number calls to pick a site, when you could just use modulo arithmetic with one number from 1 to $N = N_x \times N_y \times N_z$, and compute the values of $i$, $j$, and $k$. &lt;/li&gt;&#xA;&lt;li&gt;Similarly, in &lt;code&gt;update_configuration&lt;/code&gt;, you are performing loops that lead to a function call; you'd probably be better off if you reorganized the function to contain the loops instead. That way you can take better advantage of memory structures and data parallelism than you have now.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-07-01T15:55:17.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2676" PostTypeId="2" ParentId="2669" CreationDate="2012-07-01T05:19:14.013" Score="1" Body="&lt;p&gt;Please take a look at &lt;a href=&quot;http://lpsolve.sourceforge.net/5.5/distribution.htm&quot; rel=&quot;nofollow&quot;&gt;lpsolve&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;lp_solve is a free (see LGPL for the GNU lesser general public license) linear (integer) programming solver based on the revised simplex method and the Branch-and-bound method for the integers.&#xA;  It contains full source, examples and manuals.&#xA;  lp_solve solves pure linear, (mixed) integer/binary, semi-continuous and special ordered sets (SOS) models.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerDisplayName="knightrider" LastEditorUserId="123" LastEditDate="2012-07-23T15:25:44.603" LastActivityDate="2012-07-23T15:25:44.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2677" PostTypeId="2" ParentId="1774" CreationDate="2012-07-02T03:04:32.720" Score="1" Body="&lt;p&gt;AUTO can solve two point BVPs and has a python interface and is relatively easy to install. &lt;a href=&quot;http://www.ma.hw.ac.uk/~gabriel/auto07/node6.html&quot; rel=&quot;nofollow&quot;&gt;http://www.ma.hw.ac.uk/~gabriel/auto07/node6.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you go the route of wanting to solve P(t) first and feed it to the other ODE as an input, then an efficient way to set that up is using PyDSTool. PyDSTool is very easy to install on any platform, see &lt;a href=&quot;http://pydstool.sf.net&quot; rel=&quot;nofollow&quot;&gt;http://pydstool.sf.net&lt;/a&gt;. It will, by default, only use linear interpolation though for your previously computed solution (so compute that at fine time resolution). However, you can force PyDSTool to step to exactly the desired time points even with an adaptive integrator (although that might be inefficient and lead to inaccuracies). But with small enough max time steps, the linear interpolation and a fast integrator (Dopri is built in) for the second system means you'll be fine for &quot;regular&quot; systems like this.&lt;/p&gt;&#xA;" OwnerUserId="1693" LastActivityDate="2012-07-02T03:04:32.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2678" PostTypeId="1" AcceptedAnswerId="2686" CreationDate="2012-07-02T15:46:52.097" Score="9" ViewCount="4080" Body="&lt;p&gt;I would like to do evolving factor analysis using the SVD:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $m \times n$ data matrix $\mathcal{A}$, and for each $i$ from 1 to $m$, I want to calculate the singular values of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{A}\left[1:i,\ 1:n\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be much faster if there was a way to update the computed SVD after adding a row to $\mathcal{A}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am quite sure such algorithms do exist: is there a open implementation available for Python? Pointers to an open implementation in C or Fortran would also be useful.&lt;/p&gt;&#xA;" OwnerUserId="1695" LastEditorUserId="3740" LastEditDate="2013-02-27T00:23:19.537" LastActivityDate="2018-08-13T01:29:53.460" Title="Updatable SVD implementation in Python, C, or Fortran?" Tags="|linear-algebra|python|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2679" PostTypeId="1" AcceptedAnswerId="2684" CreationDate="2012-07-02T15:50:24.913" Score="7" ViewCount="1027" Body="&lt;p&gt;Let $B$ and $I$ denote finite sets, let &#xA;$$ E: I \times B \to \mathbb{R}_{&amp;gt; 0} $$ &#xA;be a function, and let $s_b \in \mathbb{N}$ for $b \in B$ be given.&#xA;Find $x_{i, b} \in \{ 0, 1 \}$, for $b \in B$ and $i \in I$ maximizing&#xA;$$ \sum_{i \in I} \sum_{b \in B} E(i, b) $$&#xA;subject to the constraints:&#xA;$$ \sum_{i \in I} x_{i, b} \leqslant s_b \quad \forall b \in B, \quad \text{and} \quad \sum_{b \in B} x_{i, b} \leqslant 1 \quad \forall i \in I.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how I might approach this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interpretation: we are assigning items $I$ to buckets $B$ where each bucket $b$ has a limited capacity $s_b$ (all items have the same size) and there is a value (benefit) $E(i, b)$ in assigning item $i$ to bucket $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: a school has several courses $B$; each student $i \in I$ can be assigned to only one course; student $i$ has preference $E(i, b)$ for course $b$.  Which students should be allocated to which courses to maximize happiness (the sum)?&lt;/p&gt;&#xA;" OwnerUserId="1704" OwnerDisplayName="Benjamin W." LastEditorUserId="1704" LastEditDate="2012-07-03T12:35:19.290" LastActivityDate="2012-07-03T17:59:33.287" Title="objects in buckets -- assignment optimization problem" Tags="|optimization|algorithms|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2682" PostTypeId="2" ParentId="2669" CreationDate="2012-07-03T00:49:47.140" Score="3" Body="&lt;p&gt;I generally agree with Aron's answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the modeling software you use, the matrix notation $Ax = b$ may be more or less convenient than sigma notation. For instance, with the C++ interfaces to CPLEX, CLP, &lt;a href=&quot;https://projects.coin-or.org/Cbc/&quot; rel=&quot;nofollow&quot;&gt;CBC&lt;/a&gt;, and the &lt;a href=&quot;https://projects.coin-or.org/Osi/&quot; rel=&quot;nofollow&quot;&gt;Open Solver Interface (OSI)&lt;/a&gt;, the linear algebra notation is definitely easier to work with, since the interfaces specifically ask you to set up the problem in (more or less) the form of $Ax = b$ (these interfaces are slightly more flexible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, some software packages treat defining a linear program as a more declarative process. Examples include &lt;a href=&quot;http://www.gams.com/&quot; rel=&quot;nofollow&quot;&gt;GAMS&lt;/a&gt;, and the Python packages &lt;a href=&quot;https://code.google.com/p/pulp-or/&quot; rel=&quot;nofollow&quot;&gt;PuLP&lt;/a&gt; and &lt;a href=&quot;https://software.sandia.gov/trac/coopr&quot; rel=&quot;nofollow&quot;&gt;Coopr&lt;/a&gt;. These packages focus on making the process of defining the LP look much more like the summation (or sigma) notation used in many problems. It is, of course, possible to force a problem coded in one of these packages to   look like $Ax = b$, but it's nice not to have to resort to that notation if you find it cumbersome. Furthermore, these packages are not set up as standalone solvers, and thus are much more likely to have interfaces to multiple solvers, allowing for increased flexibility in implementation (since solvers vary in speed) and distribution (not everyone has licenses for closed-source solvers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reasons, I'd recommend, in decreasing order of preference:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;PuLP because it's easy-to-use, lightweight, open source, and in Python&lt;/li&gt;&#xA;&lt;li&gt;Coopr because it's open source and in Python (but more flexible than PuLP)&lt;/li&gt;&#xA;&lt;li&gt;OSI because it's open source, and is an interface to multiple (mixed-integer) linear programming solvers, but it requires that you work in a compiled language (C++), which is a usability drawback only necessary for large-scale problems where PuLP and Coopr might run out of memory or take too long&lt;/li&gt;&#xA;&lt;li&gt;GAMS because it's easy-to-use, and uses math-like notation, but is closed-source and the trial version can only solve smaller LPs &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It took me only about a day or so to learn enough about PuLP to solve small research-scale problems. It took me a couple weeks to get the hang of OSI for research, because compiled languages require more time for development; first, there was the issue of installation, then making sure that I could compile the example files and set up Makefiles correctly, and finally, programs written in C++ are generally a few times longer than programs written in Python to accomplish the same task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really must use MATLAB, Aron's recommendations are probably best. To those, I'd add the MATLAB interface to &lt;a href=&quot;http://lpsolve.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;lpsolve&lt;/a&gt;, &lt;a href=&quot;https://research.cs.wisc.edu/math-prog/lpbook/&quot; rel=&quot;nofollow&quot;&gt;the MATLAB scripts accompanying the book &lt;em&gt;Linear Programming with Matlab&lt;/em&gt;&lt;/a&gt;, the &lt;a href=&quot;https://pic.dhe.ibm.com/infocenter/cosinfoc/v12r2/index.jsp?topic=%2Filog.odms.cplex.help%2FContent%2FOptimization%2FDocumentation%2FCPLEX%2F_pubskel%2FCPLEX1133.html&quot; rel=&quot;nofollow&quot;&gt;CPLEX for MATLAB toolbox&lt;/a&gt;, and the &lt;a href=&quot;https://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.HomePage&quot; rel=&quot;nofollow&quot;&gt;YALMIP toolbox&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of these, the YALMIP toolbox looks most promising because it's free and open source, uses declarative syntax similar to GAMS, PuLP, and Coopr, and has interfaces to a wide variety of free and nonfree solvers for many types of problems (not just linear programming). I haven't used it yet, since I typically do most of my work in the other tools I've mentioned.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-07-03T00:49:47.140" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2683" PostTypeId="2" ParentId="2679" CreationDate="2012-07-03T13:09:05.133" Score="1" Body="&lt;p&gt;Isnt it a standard mixed-integer program (assume you multiply the objective coefficients with the corresponding variables)? You should be able to solve it with any MIP solver like &lt;a href=&quot;http://scip.zib.de/&quot; rel=&quot;nofollow&quot; title=&quot;scip&quot;&gt;scip (&lt;/a&gt;open source) or &lt;a href=&quot;http://www.gurobi.com/&quot; rel=&quot;nofollow&quot; title=&quot;Gurobi&quot;&gt;Gurobi&lt;/a&gt; (academic trial version) - at last for &quot;not to big&quot; instances.&lt;/p&gt;&#xA;" OwnerUserId="772" LastActivityDate="2012-07-03T13:09:05.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2684" PostTypeId="2" ParentId="2679" CreationDate="2012-07-03T15:20:25.233" Score="1" Body="&lt;p&gt;As other people have noted, your problem is a special case of a mixed-integer linear program, as long as your objective function is linear. (If your objective function is not linear, then it would be a mixed-integer nonlinear program, and I'd make different recommendations below.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation is to use a software package like &lt;a href=&quot;http://www.gams.com/&quot; rel=&quot;nofollow&quot;&gt;GAMS&lt;/a&gt; (free trial version for small problems), &lt;a href=&quot;http://www.ampl.com/&quot; rel=&quot;nofollow&quot;&gt;AMPL&lt;/a&gt; (free trial version for small problems), &lt;a href=&quot;https://code.google.com/p/pulp-or/&quot; rel=&quot;nofollow&quot;&gt;PuLP&lt;/a&gt; (Python, free), &lt;a href=&quot;https://software.sandia.gov/trac/coopr/&quot; rel=&quot;nofollow&quot;&gt;Coopr&lt;/a&gt; (Python, free), &lt;a href=&quot;https://projects.coin-or.org/Osi/&quot; rel=&quot;nofollow&quot;&gt;OSI&lt;/a&gt; (C++, free), or &lt;a href=&quot;https://users.isy.liu.se/johanl/yalmip/&quot; rel=&quot;nofollow&quot;&gt;YALMIP&lt;/a&gt; (MATLAB, free) instead of a direct solver interface, unless you have a very clear idea of the solver you want to use (and that you may want to force potential users to use) for your application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these packages focus much more on modeling your problem, and offer interfaces to multiple free and non-free mixed-integer programming solvers, which gives you much more freedom to experiment with choice of solver and high-level parameters, at the cost of speed and memory. This approach also gives you considerable flexibility to try alternative formulations and higher-level algorithms (things like Bender's decomposition, Dantzig-Wolfe decompositions, and so on) that solvers won't attempt for you with heuristics. (Mixed-integer programming solvers tend to focus more on heuristics for generating cutting planes, generating additional valid constraints, and exploring branch-and-bound trees.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have set your formulation, and you have an idea of the speed and memory requirements for your application, you can make the decision to choose among the many mixed-integer linear programming solvers out there for your problem. Furthermore, a lot of these solvers have multiple language interfaces; again, multiple factors (convenience, familiarity, performance requirements, project deadlines) will influence your choice of language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of particular note are the free, open-source solvers in the &lt;a href=&quot;http://www.coin-or.org/&quot; rel=&quot;nofollow&quot;&gt;COIN&lt;/a&gt; collection (&lt;a href=&quot;https://projects.coin-or.org/Cbc/&quot; rel=&quot;nofollow&quot;&gt;CBC&lt;/a&gt;, &lt;a href=&quot;https://projects.coin-or.org/SYMPHONY/&quot; rel=&quot;nofollow&quot;&gt;SYMPHONY&lt;/a&gt;, &lt;a href=&quot;https://projects.coin-or.org/Bcp/&quot; rel=&quot;nofollow&quot;&gt;BCP&lt;/a&gt;, and others), &lt;a href=&quot;https://www.gnu.org/software/glpk/&quot; rel=&quot;nofollow&quot;&gt;GLPK&lt;/a&gt;, &lt;a href=&quot;http://scip.zib.de/&quot; rel=&quot;nofollow&quot;&gt;SCIP&lt;/a&gt;, and &lt;a href=&quot;http://sourceforge.net/projects/lpsolve/&quot; rel=&quot;nofollow&quot;&gt;LPSOLVE&lt;/a&gt;, as well as the closed-source, professional packages &lt;a href=&quot;http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/&quot; rel=&quot;nofollow&quot;&gt;CPLEX&lt;/a&gt; and &lt;a href=&quot;http://www.gurobi.com/&quot; rel=&quot;nofollow&quot;&gt;Gurobi&lt;/a&gt;. You may also be able to find a solver that exploits the structure of the mixed-integer linear program instance(s) you've written above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The professional packages tend to be considerably faster than the open-source solvers, but the professional packages are also expensive for non-academics. For CPLEX and Gurobi, it is possible to obtain fully functional licenses for free if you are at an academic institution; these will enable you to solve very large problem instances. CPLEX and Gurobi compete for the title of &quot;fastest (mixed-integer) linear programming solver,&quot; whatever that means. The free solvers can also handle somewhat large instances, depending on problem structure; I've used free solvers to solve problems with a few thousand binary variables and a few thousand constraints, for instance.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-07-03T17:59:33.287" LastActivityDate="2012-07-03T17:59:33.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2685" PostTypeId="1" CreationDate="2012-07-03T04:04:54.817" Score="2" ViewCount="79" Body="&lt;p&gt;Can similarity criterion help reducing calculations in &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_element_method&quot; rel=&quot;nofollow&quot;&gt;Finite Element Analysis&lt;/a&gt; (FEA) problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using NX (a thermal and CFD simulation software) to simulate the temperature and fluid field of a LED application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It takes a long time each time i change something.&#xA;I'm wondering if I can reduce the calculations by scaling down the object I'm calculating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if cooling solution A is best for an application of size 10. Is the cooling solution A also the best if I scale down the application to size 5?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is true, then i can base my calculations on size 5 and reduce drastically the time of calculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe this is not so simple. For example heat pipes behave differently with different lengths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But maybe with some corrective parameters, this can be done.&#xA;Any advices ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I realize that this might not be the place to ask this question, please also advice where i can post this question)&lt;/p&gt;&#xA;" OwnerUserId="1710" OwnerDisplayName="osager" LastActivityDate="2012-07-03T23:14:27.060" Title="Can similarity criterion help reducing calculations in Finite Element Analysis (FEA) problems?" Tags="|fluid-dynamics|simulation|software|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2686" PostTypeId="2" ParentId="2678" CreationDate="2012-07-03T19:12:35.677" Score="15" Body="&lt;p&gt;Algorithms for rank-1 updates of the SVD (also called incremental SVD) do exist, but I haven't been able to find a LAPACK-like implementation anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The one I've seen mentioned repeatedly is that of Brand (2003). Judging from &lt;a href=&quot;http://www.cs.carleton.edu/cs_comps/0607/recommend/recommender/svd.html&quot;&gt;this website&lt;/a&gt;, it seems as though Brand's algorithm is relatively simple to implement using existing LAPACK and BLAS routines as building blocks, which could perhaps explain why none of the big names has bothered to write a specialized implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a MATLAB implementations of various algorithms &lt;a href=&quot;https://web.mit.edu/~wingated/www/resources.html&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://jblevins.org/research/centroid/&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;https://www.math.fsu.edu/~cbaker/IncPACK/&quot;&gt;here&lt;/a&gt;. The last link, to IncPACK, also mentions a C++ implementation buried within a development branch of Trilinos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Python implementation of Brand's algorithm (from a 2006 paper of his) can be found buried in the &lt;code&gt;svdAddCols&lt;/code&gt; method of the &lt;code&gt;LsiModel&lt;/code&gt; class of the &lt;code&gt;gensim&lt;/code&gt; package, version 0.5.0. (Note: I could &lt;strong&gt;not&lt;/strong&gt; find this method in &lt;code&gt;gensim&lt;/code&gt; version 0.8.4; a similar method, &lt;code&gt;svdUpdate&lt;/code&gt; is present in &lt;code&gt;gensim&lt;/code&gt; version 0.7.4.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different algorithm, by Gu, is implemented in the package &lt;a href=&quot;http://sourceforge.net/projects/isvd/&quot;&gt;isvd&lt;/a&gt;, which is in C or C++ (haven't looked at the source to tell which; I arrived at this project from the link &lt;a href=&quot;https://sites.google.com/site/recheliunew/researchblog/softwareincrementalsvdpackage&quot;&gt;here&lt;/a&gt;, which discusses compilation flags for isvd). Another C++ (unfortunately, Windows-based) implementation is buried within a Netflix recommendation algorithm &lt;a href=&quot;http://www.timelydevelopment.com/demos/NetflixPrize.aspx&quot;&gt;here&lt;/a&gt;; Netflix recommendation algorithms seem to use incremental SVD frequently, and might be another potential source of implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's the best I could do with my Google-fu.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-07-03T19:21:25.727" LastActivityDate="2012-07-03T19:21:25.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2689" PostTypeId="1" CreationDate="2012-07-03T23:22:03.227" Score="3" ViewCount="234" Body="&lt;p&gt;While simulating motion of nonlinear inelastic wire one meets the following equations&#xA;\begin{align}&#xA; &amp;amp;{\partial^2\varphi\over\partial t^2}=2{\partial F\over\partial s}{\partial\varphi\over\partial s}+F{\partial^2\varphi\over\partial s^2} \\&#xA; &amp;amp;{\partial^2F\over\partial s^2}+\left({\partial\varphi\over\partial t}\right)^2-F\left({\partial\varphi\over\partial s}\right)^2=0\;.&#xA;\end{align}&#xA;Here coordinates of wire are given by&#xA;\begin{equation}&#xA; {\partial x\over\partial s}=\cos\varphi\qquad\qquad{\partial y\over\partial s}=\sin\varphi \;.&#xA;\end{equation}&#xA;What is the best discretization scheme to use for this elliptic equation with regard to this specific application? How can this be determined?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following finite difference schemes on nonstaggered grid&#xA;\begin{align}&#xA; {\partial^2\varphi\over\partial t^2} &amp;amp;\approx \frac{\varphi(t+\Delta t) - 2 \varphi(t) + \varphi(t-\Delta t)}{{\Delta t}^{2}}+\mathcal{O}\left(\Delta t^2\right)\\&#xA; {\partial^2\varphi\over\partial s^2} &amp;amp;\approx \frac{\varphi(s+\Delta s) - 2 \varphi(s) + \varphi(s-\Delta s)}{{\Delta s}^{2}}+\mathcal{O}\left(\Delta s^2\right) \\&#xA; {\partial^2F\over\partial s^2} &amp;amp;\approx \frac{F(s+\Delta s) - 2 F(s) + F(s-\Delta s)}{{\Delta s}^{2}}+\mathcal{O}\left(\Delta s^2\right) \\&#xA; {\partial \varphi\over\partial s} &amp;amp;\approx \frac{\varphi(s+\Delta s) - \varphi(s-\Delta s)}{2{\Delta s}}+\mathcal{O}\left(\Delta s^2\right) \\&#xA; {\partial \varphi\over\partial t} &amp;amp;\approx \frac{3\varphi(t) - 4\varphi(t-\Delta t) + \varphi(t-2\Delta t)}{2{\Delta t}}+\mathcal{O}\left(\Delta t^2\right) \\&#xA; {\partial F\over\partial s} &amp;amp;\approx \frac{F(s+\Delta s) - F(s-\Delta s)}{2{\Delta s}}+\mathcal{O}\left(\Delta s^2\right)&#xA;\end{align}&#xA;From here the following equations are obtained, for elliptic equation&#xA;\begin{equation}&#xA; F_{m+1}+F_{m}\left[-2-{1\over4}\left(\varphi_{m+1,n}-\varphi_{m-1,n}\right)^2\right]+F_{m-1}=-\left({\Delta s\over2\Delta t}\right)^2\left(3\varphi_{m,n}-4\varphi_{m,n-1}+\varphi_{m,n-2}\right)^2&#xA;\end{equation}&#xA;and for explicit timestep&#xA;\begin{equation}&#xA; \varphi_{m,n+1}=2\varphi_{m,n}-\varphi_{m,n-1}+\left({\Delta t\over\Delta s}\right)^2\bigg[{1\over2}\left(F_{m+1}-F_{m-1}\right)\left(\varphi_{m+1,n}-\varphi_{m-1,n}\right)+F_{m}\left(\varphi_{m+1,n}-2\varphi_{m,n}+\varphi_{m-1,n}\right)\bigg]\;.&#xA;\end{equation}&#xA;Here index $m$ denotes variable $s$ and $n$ stands for different times, we thus need to keep track of $\varphi$ at three different times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if initial position of wire is near horizontal (far from equilibrium), then the algorithm becomes unstable after some time, are there ways to improve stability? The problem seems to be that when the free part of wire gets too curved the forces raise and the triangular matrix from elliptic equation becomes near to singular.&lt;/p&gt;&#xA;" OwnerUserId="764" LastActivityDate="2012-07-12T20:16:32.537" Title="Which discretization scheme to use for elliptic PDE?" Tags="|pde|discretization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2691" PostTypeId="1" CreationDate="2012-07-04T10:27:50.280" Score="6" ViewCount="7783" Body="&lt;p&gt;The tutorial case &lt;code&gt;incompressible/pimpleDyMFoam/movingCone&lt;/code&gt; describes a cone moving through a &quot;tube&quot; (if you change the far-field patch into a wall) of constant diameter. This is the most related tutorial case I can think of, since what I want to simulate is the same geometry but with a jump in diameter. If any of the following turns this into a &lt;a href=&quot;https://meta.stackexchange.com/q/66377/189871&quot;&gt;XY problem&lt;/a&gt; please interrupt me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the tutorial case, instead of moving only the cone, which would distort the surrounding mesh too much, the outer patch (representing the far-field, which I'd replace by a &lt;code&gt;type wall&lt;/code&gt;) is split into a part moving along with the cone (&lt;code&gt;farFieldMoving&lt;/code&gt;) and two other parts (&lt;code&gt;farField&lt;/code&gt;) with a &lt;code&gt;slip&lt;/code&gt; movement (see &lt;code&gt;0/pointMotionUx&lt;/code&gt;). This results in three dynamic tube segments, where the middle one contains the cone, is moving but fixed in dimensions, while the outer segments expand/compress correspondingly. This method can however not be directly applied to the modification I would like to put on the geometry, namely a jump in diameter along the cone's motion axis - the wall segment moving along with the cone is no longer a constant. (Please correct me if I'm wrong there)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the Ansätze &lt;sup&gt;(that's the German plural of Ansatz, please edit in a better wording...)&lt;/sup&gt; I could think of, each of which lacks some knowledge from my part:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Just move a cone-shaped hole through the fixed tube mesh  &lt;/li&gt;&#xA;&lt;li&gt;Move the cone inside a smaller virtual tube and connect that tube's outer boundary to the physical tube via AMIs  &lt;/li&gt;&#xA;&lt;li&gt;Move that entire virtual tube (make it longer than the physical one) and treat the non-overlapping AMI parts somehow  &lt;/li&gt;&#xA;&lt;li&gt;Simulate the cone-fixed inertial system where the wall dynamically expands/contracts&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some more details:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The ideal solution would probably be a fixed tube mesh wherein a cone-shaped hole is moving and distorting only the cells on the cone boundary. I suspect this can be achieved with &lt;code&gt;topoChangerFvMesh&lt;/code&gt; or more precisely &lt;code&gt;movingConeTopoFvMesh&lt;/code&gt;, but I can't find any tutorial using it nor any documentation. I am also not familiar enough (yet) with the OpenFOAM code to understand the usage by simply looking at the source code. So, &lt;strong&gt;are there any resources you could point me to?&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The other approach I consider is using the &lt;a href=&quot;http://www.openfoam.org/version2.1.0/ami.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arbitrary Mesh Interface (AMI)&lt;/a&gt;: Put the cone in a smaller tube inside the physical one and connect the virtual interfaces via &lt;code&gt;cyclicAMI&lt;/code&gt;, assuming these AMIs do not actually have to be cyclic. Unfortunately, the connection using the &lt;code&gt;neighbourPatch&lt;/code&gt; keyword requires both partner boundaries to be equal in area. This means I cannot simply let the AMI touch the moving hole since there is no face to match. If I add a small cell layer, the AMI can match, but I have the problem of distorting cells again. So I think I could use the &lt;code&gt;movingCone&lt;/code&gt;-tutorial approach of moving part of the inner AMI parallel to the cone-hole. But since I can only connect &lt;em&gt;two&lt;/em&gt; equally sized faces via the &lt;code&gt;neighbourPatch&lt;/code&gt; keyword, I cannot split the inner AMI into two slipping parts embracing a moving part. Can this be achieved some other way? So, &lt;strong&gt;either how can AMIs that are split up in multiple boundaries be coupled, or how can one AMI be split into multiple sub-boundaries that have different boundary conditions&lt;/strong&gt; (in &lt;code&gt;pointMotionUx&lt;/code&gt;)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Moving the virtual tube means only part of the inner AMI actually matches the outer one, the remainder would have to be of some other type. Again the problem is that the AMIs have to be matching faces.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I didn't investigate much on changing the inertial system. I suspect this would have the advantage that &quot;only&quot; the outer walls have to move towards/away from the tube center while the cells would only slightly deform, the disadvantage would be the change of reference frame, where walls might require special treatment (as is hinted at &lt;a href=&quot;http://usna.academia.edu/CarlMungan/Papers/615827/The_Bernoulli_equation_in_a_moving_reference_frame&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, although that is for the Bernoulli equation).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I hope some of you have ideas how to either complete these attempts to a full solution or suggest another way (which may turn out to be more simple...).&lt;/p&gt;&#xA;" OwnerUserId="89" LastEditorUserId="-1" LastEditDate="2017-03-20T10:29:50.967" LastActivityDate="2012-07-09T09:17:55.577" Title="How to set up the dynamic mesh for a piston moving through a tube of variable diameter?" Tags="|mesh-generation|openfoam|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2692" PostTypeId="1" CreationDate="2012-07-04T13:48:50.577" Score="3" ViewCount="112" Body="&lt;p&gt;I am about to start research for my thesis. I am trying to find public datasets of images mostly dominated by contour meaning (An outline, esp. one representing or bounding the shape or form of something.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest any? Or give directions please?&lt;/p&gt;&#xA;" OwnerUserId="1712" LastActivityDate="2012-10-09T01:54:10.403" Title="Where can I find public datasets of images mostly dominated by contour?" Tags="|data-sets|computer-vision|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2693" PostTypeId="2" ParentId="2692" CreationDate="2012-07-04T14:18:24.080" Score="0" Body="&lt;p&gt;You can have a look at the wikimedia categories &lt;a href=&quot;http://commons.wikimedia.org/wiki/Category%3aBlueprints&quot; rel=&quot;nofollow&quot;&gt;Blueprints&lt;/a&gt;, &lt;a href=&quot;http://commons.wikimedia.org/wiki/Category%3aFloor_plans&quot; rel=&quot;nofollow&quot;&gt;Floor plans&lt;/a&gt;, &lt;a href=&quot;http://commons.wikimedia.org/wiki/Category%3aTechnical_drawings&quot; rel=&quot;nofollow&quot;&gt;Technical Drawings&lt;/a&gt; etc. if you are looking for images which are not dominated by but consist entirely of contours...&lt;/p&gt;&#xA;" OwnerUserId="89" LastActivityDate="2012-07-04T14:18:24.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2695" PostTypeId="2" ParentId="2692" CreationDate="2012-07-04T21:27:20.373" Score="1" Body="&lt;p&gt;You can create contours (not necessarily closed) by running A Canny edge detector on various high contrast images. Just grab a set of images and run a Canny edge detector on them. The output will be a set of contours. You can smooth the images beforehand to make sure you get clean images.&lt;/p&gt;&#xA;" OwnerUserId="1641" LastActivityDate="2012-07-04T21:27:20.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2696" PostTypeId="1" AcceptedAnswerId="2699" CreationDate="2012-07-05T00:43:21.143" Score="13" ViewCount="8426" Body="&lt;p&gt;I need to solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{alignat}{1}&#xA;&amp;amp; \min_{x}\|Ax - b\|^2_{2}, \\&#xA;\mathrm{s.t.} &amp;amp; \quad\sum_{i}x_{i} = 1, \\&#xA;&amp;amp; \quad x_{i} \geq 0, \quad \forall{i}. &#xA;\end{alignat}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;strong&gt;think&lt;/strong&gt; it is a quadratic problem which should be solvable with &lt;a href=&quot;https://cvxopt.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVXOPT&lt;/a&gt;, but I can't work out how.&lt;/p&gt;&#xA;" OwnerUserId="1695" LastEditorUserId="20417" LastEditDate="2018-07-11T18:03:02.350" LastActivityDate="2018-07-11T18:03:02.350" Title="Solving a least squares problem with linear constraints in Python" Tags="|optimization|python|convex-optimization|least-squares|quadratic-programming|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2697" PostTypeId="2" ParentId="2696" CreationDate="2012-07-05T03:40:32.883" Score="4" Body="&lt;p&gt;Instead of the problem you solved, solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{alignat}{1}&#xA;&amp;amp; \min_{x}\|Ax - b\|^{2}_{2}, \\&#xA;\mathrm{s.t.} &amp;amp; \quad\sum_{i}x_{i} = 1, \\&#xA;&amp;amp; \quad x_{i} \geq 0, \quad \forall{i}. &#xA;\end{alignat}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is a differentiable, convex, nonlinear optimization problem that can be solved in CVXOPT, IPOPT, or any other convex optimization solver.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-07-05T03:40:32.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2698" PostTypeId="1" CreationDate="2012-07-05T03:53:27.377" Score="7" ViewCount="894" Body="&lt;p&gt;I am trying to solving a poisson equation in structured grid with Geometric Multigrid method.   However, when coarsening the fine grid, I simply double the grid spacing at each successive level.  That means that the number of points on the finest grid must be a power of 2. This is an undesirable constraint when doing some numerical simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any strategy to avoid this constraint (i.e., to use grids whose sizes are not powers of 2?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.  &lt;/p&gt;&#xA;" OwnerUserId="1715" LastEditorUserId="9" LastEditDate="2012-07-08T18:33:29.070" LastActivityDate="2013-05-01T14:28:53.020" Title="How to use grid sizes that are not powers of two in Geometric Multigrid" Tags="|multigrid|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2699" PostTypeId="2" ParentId="2696" CreationDate="2012-07-05T04:56:38.737" Score="12" Body="&lt;p&gt;I wrote a full answer (below the line) before discovering &lt;a href=&quot;http://www.cvxpy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVXPY&lt;/a&gt;, which (like CVX for MATLAB) does all the hard stuff for you and has a very short example almost identical to yours &lt;a href=&quot;http://www.stanford.edu/~ttinoco/cvxpy/overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  You only need to replace the relevant line with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; p = program(minimize(norm2(A*x-b)),[equals(sum(x),1),geq(x,0)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My old answer, doing it the harder way with CVXOPT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following Geoff's suggestion to square your objective function gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\|Ax-b\|^2_2 = \langle x^TA^T - b^T,Ax-b\rangle = x^TA^TAx - b^TAx - x^TAb - b^Tb$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, all terms are scalars, so you can transpose the third one and drop the last one (as it doesn't depend on $x$ and therefore won't change which $x$ gives you a minimum, though you will need to add it back in after solving in order to get the correct value of your objective) to obtain&#xA;$$ x^TA^TAx - b^T(A + A^T)x$$&#xA;This (including your constraints) has the form of a quadratic program, as given in the CVXOPT documentation &lt;a href=&quot;http://abel.ee.ucla.edu/cvxopt/userguide/coneprog.html#quadratic-programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, where there is also example code for solving such a problem.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="3980" LastEditDate="2018-03-02T20:25:24.783" LastActivityDate="2018-03-02T20:25:24.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2700" PostTypeId="2" ParentId="1982" CreationDate="2012-07-05T08:53:26.083" Score="1" Body="&lt;p&gt;What you're trying to compute, if I understood your question correctly, is usually known as &lt;em&gt;Lennard-Jones cluster minimization&lt;/em&gt;. A quick search in Google Scholar will give you a &lt;a href=&quot;http://scholar.google.co.uk/scholar?hl=en&amp;amp;q=lennard-jones%20cluster%20minimization&quot; rel=&quot;nofollow&quot;&gt;list&lt;/a&gt; of the most relevant publications on the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having worked on this problem myself years ago, I don't think that this can be modelled as a TSP. If you consider a path over all particles using the Lennard-Jones/Van der Waals potential as the distance measure, then, for each particle, only the energy to two other particles is taken into account, although it interacts with all neighbours. Furthermore, your interaction potential may permit negative values, leading to negative distances, which may cause problems for most TSP algorithms.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-07-05T08:53:26.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2706" PostTypeId="1" AcceptedAnswerId="2722" CreationDate="2012-07-05T10:11:12.680" Score="5" ViewCount="761" Body="&lt;p&gt;I have a matrix which is complex symmetric. It is around 50,000 elements per side. It is a Method of Moments matrix. Is it feasible to use a standard direct solver such as Lapack to do a matrix decomposition such as $LU$ or $LDL^T$. I need to solve approximately 7200 right hand side vectors with this matrix, so if I were to take an iterative approach I would need to run it 7200 times.&lt;/p&gt;&#xA;" OwnerUserId="1480" LastActivityDate="2012-07-06T16:49:58.593" Title="Largest invertible dense matrix with standard solvers such as Lapack" Tags="|dense-matrix|linear-system|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2707" PostTypeId="1" CreationDate="2012-07-05T14:04:27.777" Score="2" ViewCount="465" Body="&lt;p&gt;I am trying to solve the following differential equations on matlab. (They are the equations obtained from the yang-mills-higgs lagrangian for the hoofy polyakov monopole ansatz). This is my function file. I have two variables h and k and their derivatives w.r.t to a variable t. My x(1)=h, x(2)=k, x(3)=dh\dt, x(4)=dk\dt. All the functions have initial value 0.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  function xprime = monopole( t,x )&#xA;    %UNTITLED Summary of this function goes here&#xA;    %   Detailed explanation goes here&#xA;&#xA;    xprime(1)=x(3);&#xA;    xprime(2)=x(4);&#xA;    xprime(4)=(1/(t.^2)).*((x(2).^2)-1).*x(2) + 4.*(x(1).^2).*x(2);&#xA;    xprime(3)=(2/(t.^2)).*(x(2).^2).*x(1)-(1-(x(1)).^2).*x(1)-(2/t).*x(3);&#xA;    xprime=xprime(:);&#xA;&#xA;&#xA;&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now when I run the following code &gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; t0=0;&#xA;    &amp;gt;&amp;gt; tf=10;&#xA;    &amp;gt;&amp;gt; x0=[0 0 0 0];&#xA;    &amp;gt;&amp;gt; [t,s]=ode45(@monopole,[t0,tf],x0);&#xA;    &amp;gt;&amp;gt; plot(t,s(:,1));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not getting anything. The graph window appears but it doesnt contain anything. This equations are supposed to have solutions. The dotted curves is what one should get with the curve starting from 1 is k, and from 0 is h.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/74MJH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is my mistake?&lt;/p&gt;&#xA;" OwnerDisplayName="user1717" LastActivityDate="2012-07-05T20:30:17.637" Title="problem in output solving two coupled second order differential equations in matlab" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2708" PostTypeId="2" ParentId="2707" CreationDate="2012-07-05T14:45:14.853" Score="2" Body="&lt;p&gt;The value $x=[0,0,0,0]^T$ is an equilibrium solution of your system of ODEs.  So MATLAB is computing and plotting the answer -- all zeros.  It appears that some entries in your initial vector $x_0$ should be 1 rather than 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judging from your expected solution, x0 is likely supposed to be $[0,1,1,0]^T$.  That will produce the same initial values (assuming the slope of the h curves is 1 at the origin).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="1552" LastEditDate="2012-07-05T20:30:17.637" LastActivityDate="2012-07-05T20:30:17.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2709" PostTypeId="2" ParentId="2654" CreationDate="2012-07-05T18:57:29.453" Score="4" Body="&lt;p&gt;If you don't want to write C++: &#xA;In the &lt;a href=&quot;http://openfoamwiki.net/index.php/Contrib_simpleFunctionObjects&quot; rel=&quot;nofollow&quot; title=&quot;simpleFunctionObjects&quot;&gt;simpleFunctionObjects&lt;/a&gt; there is a functionObject called panicDump. Something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;functions&#xA;{&#xA;    pressureOurOfRange {&#xA;        type panicDump;&#xA;        functionObjectLibs&#xA;        (&#xA;            &quot;libsimpleFunctionObjects.so&quot;&#xA;        );&#xA;        fieldName p;&#xA;        minimum 0;&#xA;        maximum 1e7;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in &lt;code&gt;system/controlDict&lt;/code&gt; would stop the simulation if the pressure blows up. Together with the expressionField-functionObject from &lt;a href=&quot;http://openfoamwiki.net/index.php/Contrib/swak4Foam&quot; rel=&quot;nofollow&quot;&gt;swak4Foam&lt;/a&gt; you can also test for non-scalar fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I'm the original developer of &lt;a href=&quot;http://openfoamwiki.net/index.php/Contrib_simpleFunctionObjects&quot; rel=&quot;nofollow&quot; title=&quot;simpleFunctionObjects&quot;&gt;simpleFunctionObjects&lt;/a&gt; so this answer could be seen as advertisement&lt;/p&gt;&#xA;" OwnerUserId="1556" LastEditorUserId="89" LastEditDate="2012-07-06T07:54:33.993" LastActivityDate="2012-07-06T07:54:33.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2710" PostTypeId="1" AcceptedAnswerId="2718" CreationDate="2012-07-05T20:12:34.820" Score="2" ViewCount="141" Body="&lt;p&gt;On several places I found different claims on a sufficient number of distances to recover relative positions of $n$ points in $d$-dimensional space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, work from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dimitris-agrafiotis.com/Papers/jcc20078.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.dimitris-agrafiotis.com/Papers/jcc20078.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(page 5, right column) claims that only $\left(\frac{n-d/2-1}{d+1}\right)$ distances are sufficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In another paper I found that $n(d+1) -(d+1)(d+2)/2$ distances are sufficient. Plugging in $n=3$ and $d=2$ yields different results (what about the result with the first paper?) What is the correct answer? A reference containing a more elaborate account would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-07-05T21:31:16.830" LastActivityDate="2012-07-06T13:25:14.027" Title="A sufficient number of distances to recover relative positions of n points" Tags="|matrix|computational-geometry|high-dimensional|machine-learning|geometry|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2711" PostTypeId="2" ParentId="2710" CreationDate="2012-07-05T22:08:17.677" Score="2" Body="&lt;p&gt;The formula from the paper is a factor of $(d+1)^2$ times smaller than the second formula you mention.  Since the paper is only citing the equation as a well known fact I am going to have to argue that he simply mis-typed it, replacing the multiplication by $d+1$ with a division by $d+1$.  Also, the fact that it says $1/3$ of a distance is required for the 3 point, 2-D case doesn't make any sense at all.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-07-05T22:08:17.677" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2713" PostTypeId="1" CreationDate="2012-07-05T13:50:30.290" Score="2" ViewCount="631" Body="&lt;p&gt;I have an energy functional that I want to optimize using the calculus of variations. Is there any software package that will do this automatically?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell, Maple and Mathematic have some relevant functionality, and they can generate C code as well. However, I could not find a working example. Are there any available working examples that use Mathematica?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other specialized packages that will solve these sorts of problems?&lt;/p&gt;&#xA;" OwnerUserId="1725" OwnerDisplayName="Shan" LastEditorUserId="276" LastEditDate="2012-07-06T08:02:12.267" LastActivityDate="2012-07-06T21:57:28.833" Title="Is there a package for computing and discretizing Euler-Lagrange equations?" Tags="|optimization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2714" PostTypeId="2" ParentId="2713" CreationDate="2012-07-05T15:24:22.903" Score="1" Body="&lt;p&gt;Mathematica has the package &lt;code&gt;VariationalMethods&lt;/code&gt; which you may want to take a look at:&#xA;&lt;a href=&quot;http://reference.wolfram.com/mathematica/VariationalMethods/tutorial/VariationalMethods.html&quot; rel=&quot;nofollow&quot;&gt;http://reference.wolfram.com/mathematica/VariationalMethods/tutorial/VariationalMethods.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example of how to set up the Euler-Lagrange equations for a problem and then use Mathematica to solve them, see the first example here:&#xA;&lt;a href=&quot;http://reference.wolfram.com/mathematica/VariationalMethods/ref/EulerEquations.html&quot; rel=&quot;nofollow&quot;&gt;http://reference.wolfram.com/mathematica/VariationalMethods/ref/EulerEquations.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="nullUser" LastActivityDate="2012-07-05T15:24:22.903" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2715" PostTypeId="1" AcceptedAnswerId="2729" CreationDate="2012-07-06T10:31:26.600" Score="5" ViewCount="6559" Body="&lt;p&gt;I know using &lt;code&gt;topoSet&lt;/code&gt; and &lt;code&gt;subsetMesh&lt;/code&gt; along the lines of the &lt;code&gt;mesh/moveDynamicMesh/simpleHarmonicMotion&lt;/code&gt; tutorial, one can cut holes into a mesh and obtain new patches. How can something similar be achieved with a vanishing hole, i.e. can a set of internal faces be turned into two touching patches with opposite orientation?&lt;/p&gt;&#xA;" OwnerUserId="89" LastActivityDate="2012-07-09T09:58:32.060" Title="How to create zero thickness internal walls in a given mesh in OpenFOAM?" Tags="|openfoam|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2716" PostTypeId="2" ParentId="2081" CreationDate="2012-07-06T11:44:45.637" Score="1" Body="&lt;p&gt;Learning the details of a molecular object system is a wonderful way for chemists to learn object oriented programming.  You'll find that implementation of such a system will refine your molecular intuitions.  You should do some hard thinking about atoms, molecules, and perhaps collections of molecules in terms of attributes and methods.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some nice python (a bit older) slides that may help: &#xA;&lt;a href=&quot;http://www.wag.caltech.edu/home/rpm/python_course/Lecture_4.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.wag.caltech.edu/home/rpm/python_course/Lecture_4.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For checking your work:&#xA;in addition to openbabel (which has python bindings!) and MMTK, there is ELBOW in phenix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the polyglot in you, there is also PerlMol (Perlmol.org).  PerlMol is written in object oriented perl and can be downloaded from CPAN.    &lt;/p&gt;&#xA;" OwnerUserId="1726" LastActivityDate="2012-07-06T11:44:45.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2717" PostTypeId="2" ParentId="2706" CreationDate="2012-07-06T11:49:58.187" Score="3" Body="&lt;p&gt;It will be slow but feasible. (You can find out the likely time it takes by solving first for sevaral $k$ reduced problems with dimension $50000/k$ and $7200/k$ right hand sides, and multiply the resulting time with $k^3$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to do pivoting for accuracy (as $LDL^T$ may be unstable &lt;a href=&quot;http://eprints.ma.man.ac.uk/344/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eprints.ma.man.ac.uk/344/&lt;/a&gt;), and you should process all right hand sides simultaneously (i.e., solve $AX=B$ with a matrix $B$ composed of all right hand sides) to take maximal advantage of the BLAS3 kernel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it is still a brute force approach, and a multipole method is the more elegant choice. Moreover, you can combine the latter with the method in &lt;a href=&quot;https://scicomp.stackexchange.com/a/2422/1117&quot;&gt;https://scicomp.stackexchange.com/a/2422/1117&lt;/a&gt; to avoid having to solve 7200 independent problems.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-07-06T11:49:58.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2718" PostTypeId="2" ParentId="2710" CreationDate="2012-07-06T13:25:14.027" Score="2" Body="&lt;p&gt;For $d&amp;gt;1$, you need to determine all distances of a simplex consisting of $d+1$ points, and then the distances from each other point to these points. ($d$ points and distances already reduce the possibilities to a finite number, but then one needs additional disambiguation information). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes a total of $d(d+1)/2+(n-d-1)(d+1) = n(d+1)-(d+2)(d+1)/2$ distances.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-06T13:25:14.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2719" PostTypeId="2" ParentId="2173" CreationDate="2012-07-06T13:57:32.660" Score="39" Body="&lt;p&gt;Scientific software is not that much different from other software, as far as how to know what needs tuning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method I use is &lt;a href=&quot;https://stackoverflow.com/a/378024/23771&quot;&gt;&lt;em&gt;random pausing&lt;/em&gt;&lt;/a&gt;.&#xA;Here are some of the speedups it has found for me:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a large fraction of time is spent in functions like &lt;code&gt;log&lt;/code&gt; and &lt;code&gt;exp&lt;/code&gt;, I can see what the arguments to those functions are, as a function of the points they are being called from. Often they are being called repeatedly with the same argument. If so, memoizing produces a massive speedup factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm using BLAS or LAPACK functions, I may find that a large fraction of time is spent in routines to copy arrays, multiply matrices, choleski transform, etc.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The routine to copy arrays is not there for speed, it is there for convenience. You may find there is a less convenient, but faster, way to do it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Routines to multiply or invert matrices, or take choleski transforms, tend to have character arguments specifying options, such as 'U' or 'L' for upper or lower triangle. Again, those are there for convenience. What I found was, since my matrices were not very big, the routines were spending more than half their time calling the subroutine for &lt;em&gt;comparing characters&lt;/em&gt; just to decipher the options. Writing special-purpose versions of the most costly math routines produced massive speedup.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If I can just expand on the latter: matrix-multiply routine DGEMM calls LSAME to decode its character arguments. Looking at inclusive percent time (the only statistic worth looking at) profilers regarded as &quot;good&quot; could show DGEMM using some percent of total time, like 80%, and LSAME using some percent of total time, like 50%. Looking at the former, you would be tempted to say &quot;well it must be heavily optimized, so not much I can do about that&quot;. Looking at the latter, you would be tempted to say &quot;Huh? What's that all about? That's just a teeny little routine. This profiler must be wrong!&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not wrong, it's just not telling you what you need to know.&#xA;What random pausing shows you is that DGEMM is on 80% of stack samples, and LSAME is on 50%. &#xA;(You don't need a lot of samples to detect that. 10 is usually plenty.)&#xA;What's more, on many of those samples, DGEMM is &lt;em&gt;in the process of calling LSAME&lt;/em&gt; from a couple different lines of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now you know why &lt;em&gt;both&lt;/em&gt; routines are taking so much inclusive time.&#xA;You also know where in &lt;em&gt;your code&lt;/em&gt; they are being called from to spend all this time.&#xA;That's why I use random pausing and take a jaundiced view of profilers, no matter how well-made they are.&#xA;They're more interested in getting measurements than in telling you what's going on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easy to assume the math library routines have been optimized to the nth degree, but in fact they have been optimized to be usable for a wide range of purposes.&#xA;You need to see what's &lt;em&gt;really&lt;/em&gt; going on, not what is easy to assume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADDED: So to answer your last two questions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are the most important things to try first?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Take 10-20 stack samples, and don't just summarize them, understand what each one is telling you. Do this first, last, and in-between. &lt;em&gt;(There is no &quot;try&quot;, young Skywalker.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I know how much performance I can get?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The stack samples will give you a very rough estimate of what fraction $x$ of time will be saved.&#xA;(It follows a $\beta(s+1,(n-s)+1)$ distribution, where $s$ is the number of samples that displayed what you are going to fix, and $n$ is the total number of samples. That doesn't count the cost of the code that you used to replace it, which will hopefully be small.)&#xA;Then the speedup ratio is $1/(1-x)$ which can be large.&#xA;Notice how this behaves mathematically.&#xA;If $n=10$, and $s=5$, the mean and mode of $x$ is 0.5, for a speedup ratio of 2.&#xA;Here's the distribution:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/byABj.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;br&gt;&#xA;If you are risk-averse then, yes, there is a small probability (.03%) that $x$ is less than 0.1, for a speedup of less than 11%.&#xA;But balancing that is an equal probability that $x$ is greater than 0.9, for a speedup ratio of greater than 10!&#xA;If you're getting money in proportion to program speed, that's not bad odds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I've pointed out to you before, you can repeat the whole procedure until you can't any more, and the compounded speedup ratio can be quite large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADDED: In response to Pedro's concern about false positives, let me try to construct an example where they might be expected to occur.&#xA;We never act on a potential problem unless we see it two or more times, so we would expect false positives to occur when we see a problem the fewest possible times, especially when the total number of samples is large.&#xA;Suppose we take 20 samples and see it twice. That estimates its cost is 10% of total execution time, the mode of its distribution.&#xA;(The mean of the distribution is higher - it is $(s+1)/(n+2)=3/22=13.6\%$.)&#xA;The lower curve in the following graph is its distribution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EQIw5.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider if we took as many as 40 samples (more than I ever have at one time) and only saw a problem on two of them.&#xA;The estimated cost (mode) of that problem is 5%, as shown on the taller curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a &quot;false positive&quot;?&#xA;It is that if you fix a problem you realize such a smaller gain than expected, that you regret having fixed it.&#xA;The curves show (if the problem is &quot;small&quot;) that, while the gain &lt;em&gt;could&lt;/em&gt; be less than the fraction of samples showing it, on average it will be larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a far more serious risk - a &quot;false negative&quot;. That is when there is a problem, but it is not found. (Contributing to this is &quot;confirmation bias&quot;, where absence of evidence tends to be treated as evidence of absence.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you get with a profiler (a good one) is you get much more precise measurement (thus less chance of false positives), at the expense of much less precise information about what the problem actually &lt;em&gt;is&lt;/em&gt; (thus less chance of finding it and getting &lt;em&gt;any&lt;/em&gt; gain).&#xA;That limits the overall speedup that can be achieved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would encourage users of profilers to report the speedup factors they actually get in practice.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There's another point to be made re. Pedro's question about false positives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He mentioned there could be a difficulty when getting down to small problems in highly optimized code. (To me, a small problem is one that accounts for 5% or less of total time.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is entirely possible to construct a program that is totally optimal except for 5%, this point can only be addressed empirically, as in &lt;a href=&quot;https://scicomp.stackexchange.com/a/1870/1262&quot;&gt;&lt;em&gt;this answer&lt;/em&gt;&lt;/a&gt;.&#xA;To generalize from empirical experience, it goes like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A program, as written, typically contains several opportunities for optimization.&#xA;(We can call them &quot;problems&quot; but they are often perfectly good code, simply capable of considerable improvement.)&#xA;This diagram illustrates an artificial program taking some length of time (100s, say), and it contains problems A, B, C, ... that, when found and fixed, save 30%, 21%, etc. of the original 100s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8RmbL.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that problem F costs 5% of the original time, so it is &quot;small&quot;, and difficult to find without 40 or more samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the first 10 samples easily find problem A.**&#xA;When that is fixed, the program takes only 70s, for a speedup of 100/70 = 1.43x.&#xA;That not only makes the program faster, it magnifies, by that ratio, the percentages taken by the remaining problems.&#xA;For example, problem B originally took 21s which was 21% of the total,&#xA;but after removing A, B takes 21s out of 70s, or 30%, so it is easier to find when the entire process is repeated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the process is repeated five times, now the execution time is 16.8s, out of which problem F is 30%, not 5%, so 10 samples find it easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So that's the point. Empirically, programs contain a series of problems having a distribution of sizes, and any problem found and fixed makes it easier to find the remaining ones.&#xA;In order to accomplish this, none of the problems can be skipped because, if they are, they sit there taking time, limiting the total speedup, and failing to magnify the remaining problems.&#xA;&lt;em&gt;That's why it is very important to find the problems that are &lt;strong&gt;hiding&lt;/strong&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If problems A through F are found and fixed, the speedup is 100/11.8 = 8.5x. If one of them is missed, for example D, then the speedup is only 100/(11.8+10.3) = 4.5x.&#xA;&lt;em&gt;That's the price paid for false negatives.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, when the profiler says &quot;there doesn't seem to be any significant problem here&quot; (i.e. good coder, this is practically optimal code), maybe it's right, and maybe it's not.&#xA;(A &lt;em&gt;false negative&lt;/em&gt;.)&#xA;You don't know for sure if there are more problems to fix, for higher speedup, unless you try another profiling method and discover that there are.&#xA;In my experience, the profiling method does not need a large number of samples, summarized, but a small number of samples,  where each sample is understood thoroughly enough to recognize any opportunity for optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;** It takes a minimum of 2 hits on a problem to find it, unless one has prior knowledge that there is a (near) infinite loop.&#xA;(The red tick marks represent 10 random samples);&#xA;The average number of samples needed to get 2 or more hits, when the problem is 30%, is $2/0.3=6.67$ (&lt;a href=&quot;http://en.wikipedia.org/wiki/Negative_binomial_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;negative binomial distribution&lt;/a&gt;).&#xA;10 samples find it with 85% probability, 20 samples - 99.2% (&lt;a href=&quot;http://en.wikipedia.org/wiki/Binomial_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;binomial distribution&lt;/a&gt;).&#xA;To get the probability of finding the problem, in R, evaluate &lt;code&gt;1 - pbinom(1, numberOfSamples, sizeOfProblem)&lt;/code&gt;, for example: &lt;code&gt;1 - pbinom(1, 20, 0.3) = 0.9923627&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADDED: The fraction of time saved, $x$, follows a Beta distribution $\beta(s+1,(n-s)+1)$, where $n$ is the number of samples, and $s$ is the number that display the problem.&#xA;However, the speedup ratio $y$ equals $1/(1-x)$ (assuming all of $x$ is saved), and it would be interesting to understand the distribution of $y$.&#xA;It turns out $y-1$ follows a &lt;a href=&quot;http://en.wikipedia.org/wiki/Beta_prime_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;BetaPrime&lt;/em&gt;&lt;/a&gt; distribution.&#xA;I simulated it with 2 million samples, arriving at this behavior:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         distribution of speedup&#xA;               ratio y&#xA;&#xA; s, n    5%-ile  95%-ile  mean&#xA; 2, 2    1.58    59.30   32.36&#xA; 2, 3    1.33    10.25    4.00&#xA; 2, 4    1.23     5.28    2.50&#xA; 2, 5    1.18     3.69    2.00&#xA; 2,10    1.09     1.89    1.37&#xA; 2,20    1.04     1.37    1.17&#xA; 2,40    1.02     1.17    1.08&#xA;&#xA; 3, 3    1.90    78.34   42.94&#xA; 3, 4    1.52    13.10    5.00&#xA; 3, 5    1.37     6.53    3.00&#xA; 3,10    1.16     2.29    1.57&#xA; 3,20    1.07     1.49    1.24&#xA; 3,40    1.04     1.22    1.11&#xA;&#xA; 4, 4    2.22    98.02   52.36&#xA; 4, 5    1.72    15.95    6.00&#xA; 4,10    1.25     2.86    1.83&#xA; 4,20    1.11     1.62    1.31&#xA; 4,40    1.05     1.26    1.14&#xA;&#xA; 5, 5    2.54   117.27   64.29&#xA; 5,10    1.37     3.69    2.20&#xA; 5,20    1.15     1.78    1.40&#xA; 5,40    1.07     1.31    1.17&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first two columns give the 90% confidence interval for the speedup ratio.&#xA;The mean speedup ratio equals $(n+1)/(n-s)$ except for the case where $s=n$.&#xA;In that case it is undefined and, indeed, as I increase the number of simulated $y$ values, the empirical mean increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a plot of the distribution of the speedup factors, and their means, for 2 hits out of 5, 4, 3, and 2 samples.&#xA;For example, if 3 samples are taken, and 2 of them are hits on a problem, and that problem can be removed, the average speedup factor would be 4x.&#xA;If the 2 hits are seen in only 2 samples, the average speedup is undefined - conceptually because programs with infinite loops exist with non-zero probability!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pZ9Q9.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1262" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-10-01T16:27:36.597" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="2720" PostTypeId="1" AcceptedAnswerId="2727" CreationDate="2012-07-06T15:40:13.533" Score="6" ViewCount="979" Body="&lt;p&gt;I have a method that is supposed to be second order accurate based on asymptotic analysis/Taylor series expansion which assumes that the solution is smooth. I am solving a PDE which has very rough coefficients (one of them is a step function) and thus the solution of the PDE is not smooth, though it does exist and is unique. As a result, I observe only linear convergence. Will further mesh refinement eventually produce quadratic convergence? How should I formulate results of the numerical method in such case? Thanks!&#xA;Edit: I am solving an equation of the type:&#xA;$$u_t=x^2u_{xx}+1_{\{x\geq C\}}u_y$$ for some constant $C$ which is in the range of domain of computation $[0,x_{max}]$. I am solving that with Semi-lagrangian approach.&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2012-07-06T17:57:19.070" LastActivityDate="2012-07-06T22:17:55.657" Title="Why is my second order accurate method only converging at first order when the coefficients are rough?" Tags="|pde|finite-difference|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2721" PostTypeId="2" ParentId="2720" CreationDate="2012-07-06T16:15:00.467" Score="2" Body="&lt;p&gt;Unless your mesh is extremely coarse right now, refining it will not change the order of your convergence.  If you are getting good linear convergence, then it is likely your mesh is fine enough for the method to be asymptotic, and it will never change, no matter how fine your mesh gets (until you bottom out your residuals).  I would recommend looking at the analysis of your derivative approximation and see if the assumption is made that the derivative being approximated is continuous.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I may be mistaken about this fact, but if it is based on some polynomial approximation of the function, this may cause the quadratic convergence to fail at singularities.  This will drop your convergence from second order to some lower order at the point where you state your coefficient is a step function.  Dropping the order at one point, be it an internal singularity or a boundary condition reduces the overall order to that same value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, another possibility I thought of, are you sure that the method is second order global, and not just second order local?  It is a simple mistake to make, and would explain the one order lower convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you give more detail as to what your PDE is and what method you are using to approximate your derivatives, it may help us give a more specific answer as to what could be causing the incorrect convergence rate.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2012-07-06T16:29:00.870" LastActivityDate="2012-07-06T16:29:00.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2722" PostTypeId="2" ParentId="2706" CreationDate="2012-07-06T16:49:58.593" Score="4" Body="&lt;p&gt;&lt;em&gt;[Moving the comment thread down here]&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A direct solve will be pretty expensive, likely on the order of an hour for the sort of machine you describe. If you want invest time in something faster using more specialized methods, I suggest using FMM and a block Krylov method. FMM applies the operator in $\mathcal O(N)$ memory and time. FMM has a big constant, but less than $N=50000$, so it should be faster than dense matrix multiply. I don't know if you'll find a library implementation that applies to multiple vectors at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Block Krylov methods apply to $k$ right hand sides, converging as much as $k$ times faster than with a single right hand side. The cost per iteration is $\mathcal O(N k)$ to apply the operator using FMM plus $\mathcal O(N k^2)$ to orthogonalize (with an additional factor for the restart length if using GMRES) and $\mathcal O(k^3)$ &quot;scalar&quot; work. You can choose $k=7200$ or some smaller number, to reduce the $Nk^2$ and $k^3$ cost at the expense of (slightly) slower convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given its source in integral equations, I assume that the continuum operator associated with your problem is of the form $I + K$ where $K$ is a compact operator. Being compact, $K$ has essentially finite rank, therefore the discretized system has an essentially constant number of eigenvalues that are not &quot;near&quot; $1$, thus we expect the block Krylov method to converge very fast.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-06T16:49:58.593" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2723" PostTypeId="2" ParentId="2720" CreationDate="2012-07-06T17:44:06.503" Score="6" Body="&lt;p&gt;If your problem is elliptic with variable coefficients, a discontinuous coefficient will make the gradient discontinuous. If the solution space has a continuous gradient at that point, the method cannot be better than first order accurate. No amount of refinement will fix that unless refinement causes the solution space to allow a discontinuity at that point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite element methods allow such discontinuities most naturally. If you use a standard $C^0$ conforming finite element method and align the grid with the coefficient discontinuity, you should see second (or higher, depending on the basis) order accuracy. Unfortunately, corners in coefficients cause additional trouble because the solution appears on coarse grids to have a singularity similar to that near a reentrant corner in the domain. The apparent singularity will eventually be resolved with grid refinement, but the required resolution depends on the ratio of coefficients, which may be impractically fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways to enrich solution spaces so the internal discontinuity is represented without needing to conform to a background mesh. These papers provide a good perspective on classical treatment of material discontinuities.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1002/nme.1620240209&quot; rel=&quot;noreferrer&quot;&gt;MacKinnon and Carey, Treatment of material discontinuities in finite element computations, 1987&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1016/0021-9991(88)90104-0&quot; rel=&quot;noreferrer&quot;&gt;MacKinnon and Carey, Analysis of material interface discontinuities and superconvergent fluxes in finite difference theory, 1988&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;More recent methods in the finite element context allowing much more general enrichments go by names like Generalized Finite Element Method (GFEM), eXtended FEM (XFEM), and Discontinuous Enrichment Method. In case the coefficient structure is oscillatory, random, or otherwise unresolvable and not possible to conform to, there are homogenization techniques, though first order accuracy is optimal for such multiscale problems.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-06T17:44:06.503" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2724" PostTypeId="2" ParentId="2698" CreationDate="2012-07-06T18:22:24.487" Score="4" Body="&lt;p&gt;All you need is a hierarchy of meshes that reduces sufficiently fast. You can use a 3:1 coarsening ratio, or you could use unstructured meshes, etc. There is no such restriction as the one you cite, even though it's common to implement the multigrid method with a 2:1 coarsening ratio.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-06T18:22:24.487" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="2725" PostTypeId="1" AcceptedAnswerId="2732" CreationDate="2012-07-06T18:45:51.280" Score="11" ViewCount="4958" Body="&lt;p&gt;I have experience in coding OpenMP for Shared Memory machines (in both C and FORTRAN) to carry out simple tasks like matrix addition, multiplication etc. (Just to see how it competes with LAPACK). I know OpenMP enough to carry out simple tasks without the need to look at documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently, I shifted to Python for my projects and I don't have any experience with Python beyond the absolute basics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 2 questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a good guide (online PDF) for describing shared memory parallel computing for Python?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the best way of achieving it? I have seen a bit of &lt;code&gt;ctypes&lt;/code&gt; and I'm not sure if that's the best way.&#xA;(By best I mean one which has a good tradeoff between programmer time and system time. It shouldn't be too tedious to code neither should the execution be slow)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="529" LastEditorUserId="123" LastEditDate="2012-07-07T06:21:47.460" LastActivityDate="2012-07-07T18:57:15.383" Title="Guides on Python for shared-memory Parallel Programming" Tags="|parallel-computing|python|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2726" PostTypeId="2" ParentId="2713" CreationDate="2012-07-06T21:57:28.833" Score="4" Body="&lt;p&gt;The question of course all depends on what kind of problem you have. If you have an energy functional that describes the position of a weight suspended from a spring, then the Euler-Lagrange equations are just an algebraic equation. On the other hand, if you have an energy equation that is described using an ODE, e.g. the position of a chain suspended from its end points as gravity acts on it, then you will get a two-point boundary value problem. Finally, if you have a multi-dimensional problem, then the Euler-Lagrange equations will be partial differential equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point I want to make is that the type of problem you get in the Euler-Lagrange equations depends very much on the situation you are considering, and so the question of what discretization method to use can not be answered without seeing more context.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-06T21:57:28.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2727" PostTypeId="2" ParentId="2720" CreationDate="2012-07-06T22:17:55.657" Score="10" Body="&lt;p&gt;Like Jed says, if your coefficient is a step function, then the exact solution will be continuous but have a &quot;kink&quot; (i.e., the gradient will be discontinuous). Now, take the Laplace equation as an example (your heat equation behaves the same way, but the analysis is a bit more involved). There, you have the following analysis ($e=u-u_h$ is the error):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \|\nabla e\|^2 = (\nabla e,\nabla e) = (\nabla e, \nabla (e-\varphi_h))&#xA;$$&#xA;where the last step is simply an application of Galerkin orthogonality, for any discrete function $\varphi_h$. If you choose $\varphi_h=I_h u - u_h$ where $I_hu$ is the interpolation of the exact solution onto the finite element space, then you see the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \|\nabla e\|^2 = (\nabla e, \nabla (u-u_h-I_hu+u_h))&#xA;  = (\nabla e, \nabla (u-I_hu))&#xA;  \le&#xA;  \|\nabla e\| \|\nabla (u-I_hu)\|.&#xA;$$&#xA;In other words,&#xA;$$&#xA;  \|\nabla e\|&#xA;  \le&#xA;  \|\nabla (u-I_hu)\|.&#xA;$$&#xA;What this means is that your finite element solution can not be better in the $H^1$ seminorm than the interpolant. Using duality arguments, you will get estimates from this that are in the $L_2$ norm and look like this:&#xA;$$&#xA;  \|e\|&#xA;  \le&#xA;  C h&#xA;  \|u-I_hu\|,&#xA;$$&#xA;i.e. you now need the interpolation estimate in $L_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you think about approximating a function with a &quot;kink&quot; with finite element shape functions using interpolation, it quickly becomes clear that if you make the mesh fine enough, then the error will ultimately be concentrated in the one element in which the kink is located (unless the kink happens to be located on a mesh line). In any case, you will find in the literature the following estimates:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If $u\in H^1$ (e.g., if it has a kink), then $\|u-I_hu\| \le h \|\nabla u\|$.&lt;/li&gt;&#xA;&lt;li&gt;If $u\in H^2$ (e.g., if it has no kinks), then $\|u-I_hu\| \le h^2 \|\nabla^2 u\|$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In your case, the function has a kink and so is only in $H^1$, so you can't expect to be better than ${\cal O}(h)$ because that is the order you get from the interpolation estimate, and the error estimate for the finite element solution can not be better than the interpolation estimate.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-06T22:17:55.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2728" PostTypeId="2" ParentId="2725" CreationDate="2012-07-07T10:00:16.253" Score="3" Body="&lt;p&gt;It depends on the level you're trying to program parallel. For instance for matrix/vector stuff the first stop with python would be NumPy/SciPy (they provide an interface to numeric libraries which gives you the full speed of the libraries with the convenience of Python) and from &lt;a href=&quot;http://www.scipy.org/ParallelProgramming&quot; rel=&quot;nofollow&quot;&gt;what they write about parallelism&lt;/a&gt; it seems that if the libraries are compiled for parallel use then the programs take advantage of multicores for certain operations. (seems like this article is a bit older, os things might have improved in the meantime. Also there are links to other means of parallel programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And of course there is &lt;a href=&quot;http://mpi4py.scipy.org/docs/usrman/index.html&quot; rel=&quot;nofollow&quot;&gt;mpi4py&lt;/a&gt; to directly program MPI (included in the above article)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My bottom line is: if your main interest is vector/matrix-operations and parallelism is something you &quot;only&quot; need to do it fast then you should have a look at the NumPy/SciPy-ecosystem and only if you don't find the stuff there that you need should you consinder writing your own libraries&lt;/p&gt;&#xA;" OwnerUserId="1556" LastActivityDate="2012-07-07T10:00:16.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2729" PostTypeId="2" ParentId="2715" CreationDate="2012-07-07T10:43:05.343" Score="4" Body="&lt;p&gt;I think what you're looking for is the &lt;code&gt;createBaffles&lt;/code&gt; utility:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use &lt;code&gt;topoSet&lt;/code&gt; to create a &lt;code&gt;faceZone&lt;/code&gt;, then run something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;createBaffles nameOfFaceZone '(nameOfMasterPatch nameOfSlavePatch)' -overwrite&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The patches have to exist already, for which &lt;code&gt;createPatches&lt;/code&gt; can come in handy.&lt;/p&gt;&#xA;" OwnerUserId="1556" LastEditorUserId="89" LastEditDate="2012-07-09T09:58:32.060" LastActivityDate="2012-07-09T09:58:32.060" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2730" PostTypeId="2" ParentId="2691" CreationDate="2012-07-07T12:09:07.783" Score="1" Body="&lt;p&gt;If you consider introducing a topological change to the mesh, there is a tutorial explaining exactly the problem that you are facing. Here you can find the background of the case: &lt;a href=&quot;http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2008/ErikBjerklund/OpenFoamBjerklundE3.pdf&quot; rel=&quot;nofollow&quot;&gt;Modification of a movingConeTopoFvMesh&lt;/a&gt;. Because of cell layer removal, I think that it should be possible to have a cylinder mesh with variable radius without a problem: removed cell layer is created using the cone patch that is translated to deflate, and destroy, the cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A tutorial case is also present in the 1.6-ext version of OpenFOAM in the folder &lt;em&gt;incompressible/icoDyMFoam/movingConeTopo/&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of new &quot;cylinder&quot; width:&#xA;In the tutorial case, in the blockMeshDict file, &lt;strong&gt;add another block above block 4&lt;/strong&gt;, with the appropriate density to match the block 4, and be sure to change the patches accordingly (the top patch of the 4th block in the direction of the Y axis needs to be deleted, because its faces are now internal). &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="89" LastEditDate="2012-07-09T09:17:55.577" LastActivityDate="2012-07-09T09:17:55.577" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2731" PostTypeId="2" ParentId="2462" CreationDate="2012-07-07T14:42:06.327" Score="5" Body="&lt;p&gt;The phrase &quot;community detection&quot; is &lt;a href=&quot;http://www.inf.u-szeged.hu/~pluhar/ExtendedModularity.pdf&quot; rel=&quot;nofollow&quot;&gt;loosely defined&lt;/a&gt; as partitioning the vertices of a graph into &quot;communities&quot; such that each has members more densely linked to one another than to members of other &quot;communities&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our first task is to ascertain what this should mean in the case of a bipartite graph, which by definition consists of two &quot;modes&quot; such that members of one mode are linked only to members of the other mode.  It may be expressed, at least for simple graphs, as having an adjacency matrix of special block structure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = \begin{pmatrix} 0 &amp;amp; B \\ B^T &amp;amp; 0 \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that the most pertinent interpretation of &quot;restrict the detection of communities just on one of the two modes&quot; would apply said algorithms to the &quot;projected&quot; graphs corresponding to blocks of $A^2$, i.e. the first mode with adjacency matrix $BB^T$ and the second mode with adjacency matrix $B^TB$.  Note that even if the original bipartite graph is simple (so that $A$ is binary), the projected graphs will generally be multi-graphs.  Fortunately &lt;a href=&quot;http://igraph.sourceforge.net/doc/R/bipartite.projection.html&quot; rel=&quot;nofollow&quot;&gt;igraph has a method to construct these&lt;/a&gt; for us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are equally fortunate in that &lt;a href=&quot;http://igraph.sourceforge.net/doc-0.5/R/fastgreedy.community.html&quot; rel=&quot;nofollow&quot;&gt;the igraph community detection algorithms&lt;/a&gt; and &lt;a href=&quot;http://igraph.sourceforge.net/doc-0.6/python/igraph.Graph-class.html&quot; rel=&quot;nofollow&quot;&gt;related&lt;/a&gt; have been &lt;a href=&quot;http://igraph.sourceforge.net/relnotes-0.5.html&quot; rel=&quot;nofollow&quot;&gt;&quot;updated to handle weighted graphs&quot;&lt;/a&gt; (such as multi-graphs).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;S. Fortunato (2010) surveys community detection criteria (&lt;a href=&quot;http://arxiv.org/pdf/0906.0612.pdf&quot; rel=&quot;nofollow&quot;&gt;Community detection in graphs&lt;/a&gt;) and their use with bipartite and multipartite networks.  The interpretation I suggest above is articulated on page 8:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Multipartite graphs are usually reduced to unipartite projections&#xA;  of each vertex class. For instance, from the bipartite network of&#xA;  scientists and papers one can extract a network of scientists&#xA;  only, who are related by coauthorship.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2012-07-17T13:39:29.463" LastActivityDate="2012-07-17T13:39:29.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2732" PostTypeId="2" ParentId="2725" CreationDate="2012-07-07T14:45:51.800" Score="8" Body="&lt;p&gt;[This is my first post and I hope I have not completely misunderstood the use of SE - if so I apologize in advance]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I agree with &quot;bgschaid&quot; that the question is very difficult to answer based on the information provided.  It makes a huge difference if you want low-level routines to exploit a multi-core architecture or if you need to exploit parallelism for embarrassingly parallel problems - or something in-between. An overview of different parallel computing possibilities in Python can be found &lt;a href=&quot;http://wiki.python.org/moin/ParallelProcessing&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the former case I for sure recommend to use tools such as &lt;strong&gt;NumPy&lt;/strong&gt; / &lt;strong&gt;SciPy&lt;/strong&gt; which at least in the MKL compiled version from Enthought supports multi-core architectures. Here you can control the number of cores to use via the environment variable &quot;MKL_NUM_THREADS&quot;.  This relies on highly optimized libraries that we can hardly expect to beat performance wise.  I believe it is generally advised to use these high quality and highly optimized libraries whenever possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wish to exploit parallelism at a coarse level, the Python standard tool &lt;strong&gt;multiprocessing&lt;/strong&gt; is easy to use - and it also supports shared data objects.  There are different tools to use as part of the &lt;strong&gt;multiprocessing&lt;/strong&gt; package.  I have used &lt;strong&gt;map_async&lt;/strong&gt; (SIMD like) and &lt;strong&gt;apply_async&lt;/strong&gt; (MIMD like) for several problems with good results.  The &lt;strong&gt;multiprocessing&lt;/strong&gt; package is quite easy to use and being a standard part of Python means that you can expect other potential users of your code to easily be able to use it.  &lt;strong&gt;multiprocessing&lt;/strong&gt; also links to NumPy data objects directly.  When using &lt;strong&gt;multiprocessing&lt;/strong&gt; I would recommend you to set the environment variable &quot;MKL_NUM_THREADS&quot; to 1 such that &lt;strong&gt;NumPy&lt;/strong&gt; is only allowed one core for each process/worker - otherwise you could end up in a resource contention between NumPy parallel and &lt;strong&gt;multiprocessing&lt;/strong&gt; which leads to a performance degradation.  &lt;strong&gt;multiprocessing&lt;/strong&gt; works fine for a multi-CPU/multi-core architecture under the same operating system.  I have used &lt;strong&gt;multiprocessing&lt;/strong&gt; on a shared memory computer with 4 x Xeon E7-4850 CPUs (each 10 cores) and 512 GB memory and it worked extremely well.  Shared arrays can be handled by &lt;strong&gt;multiprocessing.Array&lt;/strong&gt; or &lt;strong&gt;sharedctypes&lt;/strong&gt;.  You can find the Python documentation &lt;a href=&quot;http://docs.python.org/download.html&quot;&gt;here&lt;/a&gt; - check the &lt;strong&gt;library.pdf&lt;/strong&gt; file.  I have some slides explaining some of the basic parts of this - PM me if you want those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a cluster configuration with distributed memory I believe &lt;strong&gt;mpi4py&lt;/strong&gt; is likely the preferred tool.  I have not used this myself but I know that it is being used a lot in Python parallel programming.&lt;/p&gt;&#xA;" OwnerUserId="1678" LastEditorUserId="1678" LastEditDate="2012-07-07T18:57:15.383" LastActivityDate="2012-07-07T18:57:15.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2733" PostTypeId="2" ParentId="2081" CreationDate="2012-07-07T20:58:01.773" Score="9" Body="&lt;p&gt;Your goal has a lot of challenges. I'll break them down into parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SMILES is not a trivial language to parse, and the rules for aromaticity perception are not well defined. The detailed grammar definition from the OpenSMILES project should help some.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SMILES defines the topology, but does not give 2D or 3D information. Making either is hard. (That is, if you want it to look nice.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really, you should look to the RDKit cheminformatics toolkit (or OpenBabel, but I prefer RDKit). It has a SMILES parser built in, as well as 2D layout, and I believe 3D conformation generation. OpenBabel does as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then for display you'll have to figure out the GUI system. Actually, here the CDK cheminformatics toolkit, in Java, is the most advanced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you're in the basics of how to represent a molecule. There are differences between small-molecule and large-molecule (protein, DNA) data models, but since you're interested in SMILES that means you're small-molecule oriented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can look at the API docs for RDKit, OpenBabel, CDK, OEChem, and Indigo, among many others. That will give you an idea of the diverse ways that people develop their class API. Of these, I prefer OEChem the most, followed by RDKit. Even though OEChem is open source, the API is online and freely readable, along with examples of use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, have a Molecule class, with a list of Atom and Bond instances. &quot;mol.AddAtom(element number)&quot; creates a new atom, with no bonds, &quot;mol.AddBond(atom1, atom2, bond_type)&quot; make a bond connection. Each bond needs to know the atoms its connected to, and each atom needs a list of bonds. This leads to a lot of cycles in the data structure, but it's needed so that various algorithms, like connectivity search, can be done in linear time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do not use a 2D matrix. While feasible for a small molecule, it doesn't scale that well, and there's no need for it. Few algorithms need the connectivity matrix, and it's easily generated if/when needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't have a &quot;FunctionalGroup&quot;. It's too specialized. Use something like a &quot;Subset&quot; or &quot;Fragment&quot;, which contains the list of atoms and bonds you're interested in. That way you can also handle things like &quot;selected atoms&quot; and &quot;ring substructure&quot; and &quot;scaffold&quot; by referring to the specific subset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked at your pastebin. The parser shouldn't work that way. You should separate parsing from your actual molecule structure. Try something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Molecule(object):&#xA;    def __init__(self):&#xA;        self.atoms = []&#xA;        self.bonds = []&#xA;        self._atom_id = 0&#xA;        self._bond_id = 0&#xA;    def _next_atom_id(self):&#xA;        atom_id = self._atom_id&#xA;        self.atom_id += 1&#xA;        return atom_id&#xA;    def AddAtom(self, eleno):&#xA;        self.atoms.append(Atom(self, self._next_atom_id(), eleno))&#xA;    def AddBond(self, atom1, atom2, bondtype):&#xA;        assert atom1.molecule is atom2.molecule&#xA;        self.bonds.append(Bond(self, self._next_bond_id(),&#xA;                               atom1, atom2, bondtype))&#xA;&#xA;class Atom(object):&#xA;    def __init__(self, molecule, id, eleno):&#xA;        self.molecule = molecule&#xA;        self.id = id&#xA;        self.eleno = eleno&#xA;        self.charge = 0&#xA;        self.isotope = 0&#xA;   ..&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then the parser for a simple linear chain like &quot;C C O&quot; is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def parse_linear_chain(text):&#xA;   mol = Molecule()&#xA;   prev_atom = None&#xA;   for atom_symbol in text.split():&#xA;     eleno = lookup_symbol[atom_symbol]&#xA;     atom = mol.NewAtom(eleno)&#xA;     if pre_atom is not None:&#xA;       mol.AddBond(prev_atom, atom, 1)&#xA;     prev_atom = atom&#xA;   return mol&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course, a full SMILES parser is much more complicated than this, and the full data model has to handle things like hydrogen counts, which are often implicit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The OpenBabel, RDKit, and CDK mailing lists are also good places to go, should you decide to use one of those toolkits. There's also the &quot;Blue Obelisk&quot; Q&amp;amp;A site, hosted by Shapado.&lt;/p&gt;&#xA;" OwnerUserId="1735" LastActivityDate="2012-07-07T20:58:01.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2734" PostTypeId="2" ParentId="1463" CreationDate="2012-07-07T21:17:40.813" Score="9" Body="&lt;p&gt;Some of the algorithms are available from the source code for the different packages. PyMol is one such, and VMD's source is also accessible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented VMD's ribbon algorithm in the 1990s. The first step is the structure determination - where are the amino acids? which are connected into a chain? where are the C-alpha atoms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, as Kyle said, is the spline. VMD uses a Catmull–Rom spline, with the C-alphas as control points. This is a 3rd order spline, and the splines goes through the C-alphas. If you work out the math, there's a single free parameter, which corresponds to how stiff the spline is around the control point. I tried a few values until I found the one which was esthetically pleasing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also some trickiness about how to handle the end, which don't have enough C-alphas. I extrapolated to get the other points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That gives the path. A circular extrusion along the path gives a tube. You can vary the cross-section radii to give an ellipse, and with a bit more work define the ribbon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is to find the correct norm, so that the ribbons are aligned with an alpha helix. I tried various things then gave up, looked at the Raster3D implementation, got permission to use it, and added that to VMD. It's a cumulative sum of the previous vector norm and the current norm defined by the C-alpha trace. I would have to look at the source for how it works again. Interestingly, Ethan Merritt, the Raster3D author, pointed out that he got that bit of code from FRODO, so it has a long history.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VMD now has &quot;NewRibbons&quot;, which was implemented after my time. I don't know how it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest way to do an alpha helix is to draw a line from the first to the last residue; extrude a circle along the line and you have a cylinder. You can also do a linear best-fit to the helix, but I think that caused problems for short helices. There's likely more clever ways to do it, including ways like Kyle suggested, which allows for soft bends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beta strands are easy. There are two control paths, one for each side. Those define the strand path, and the normal. You do have to be a bit careful with twists, so that your strand doesn't twist 290 degrees when it should twist 70 degrees, but that wasn't hard to handle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A hard part, which you didn't mention, is how to detect where the alpha-helix and beta-strands are located. Some PDB records contain that, but not all. I punted and used a 3rd-party tool, STRIDE, for that. Warren implemented his own algorithm. Roger Sayle implemented his own version of DSSP for Raster3D.&lt;/p&gt;&#xA;" OwnerUserId="1735" LastActivityDate="2012-07-07T21:17:40.813" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2735" PostTypeId="1" AcceptedAnswerId="2741" CreationDate="2012-07-07T23:30:10.840" Score="7" ViewCount="251" Body="&lt;p&gt;I would like to do the analysis for the Crank-Nicolson method on a non-uniform grid for the parabolic equation with variable coefficients. I was able to prove everything for a uniform grid by energy methods for the results of stability, where I have used summation by parts to do estimates. Now that I turn to a non-uniform grid, I can't carry over the same proofs, since the summation by parts is different. Is there any reference on it? What arguments do people make to extend the proofs to that case?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="276" LastEditDate="2012-07-09T01:32:49.537" LastActivityDate="2012-07-09T01:32:49.537" Title="How do I analyze the error for the Crank-Nicolson method on a parabolic PDE?" Tags="|pde|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2736" PostTypeId="1" AcceptedAnswerId="2827" CreationDate="2012-07-08T02:45:55.570" Score="0" ViewCount="120" Body="&lt;p&gt;I am looking at the pde of the type $u_t=\mathcal{L}u$ for some elliptic operator $\mathcal{L}$ and on some domain $D$. Assume I am solving that with a finite difference method and want to estimate the error. Thus, I create a mesh over $D$ and compute the error in discrete $L^2$ norm, that is at every point on the grid I take the difference between the function itself and an approximate solution, square it and sum up. This sum includes all the points, including the boundary points. By Lax theorem the rate of convergence is the rate of local truncation error PROVIDED the problem is well posed and the derivatives in local truncation error are bounded. However, assume it can be shown that the weak solution exist and unique and is sufficiently smooth inside of the domain but on the boundaries it is only satisfied in a weak sense, and lacks regularity. So there is no hope that the terms in local truncation error are bounded on the boundary, they are not even defined there. Thus, how do all these points on the boundary contribute to the error? They exist even if we refine a mesh, more than that we add points with it. Please let me know should I take into account irregularity on the boundary.  &lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2012-07-15T20:57:40.780" Title="Impact of irregularity at the boundary on the error analysis" Tags="|pde|finite-difference|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2737" PostTypeId="1" AcceptedAnswerId="2763" CreationDate="2012-07-08T02:55:14.190" Score="4" ViewCount="1716" Body="&lt;p&gt;When I read books on finite differences they often end up using discrete $L^2$ norm for estimating the error as it naturally arises from weak formulation. I was wondering if people do that in Sobolev norm and when it is useful. I have not seen any at least in finite-differences book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific look at the $$Au=f,$$ where assume $A_h$ is some approximation for $A$ and $U$ is the numerical solution for the system. Then if we plug the actual function $u$ into $A_hU=f$ and substruct we have $$A_h(u-U)=\tau$$ for $\tau$ being a local error. Thus I have an error equation $$e=A_h^{-1}\tau$$ What are the problems I am facing If I use discrete Sobolev norm? What would that be then, it should involve derivatives estimates, but can I do one for the local truncation error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2012-07-10T12:44:58.737" Title="choice of the norm for the error of the numerical method" Tags="|pde|finite-difference|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2738" PostTypeId="1" AcceptedAnswerId="2739" CreationDate="2012-07-08T03:12:38.370" Score="4" ViewCount="88" Body="&lt;p&gt;I am solving the following equation $u_t=x^2u_{xx}+\frac{x-y}{T-t}u_y$ with an initial data $u_0=max(y-C,0)$ for some $C$ in the domain of the numerical method. In time I am solving that on $[0,T]$. Before I implement a numerical method I have to make sure the solution exist and unique. Assume it is done and the next question is the regularity since the error analysis is based on the Taylor expansion and to do so I need enough regularity in the solution. So, I would like to know what I can have in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In $y$ dimension this equation is just transport and thus this non differentiability might propagate, but does the diffusion in the first variable help to smooth it at all? If not, then I don't even have $u_y$ in classical sense and only in weak, so why any numerical method would ever converge because if the local truncation error includes $u_{yy}$ or any higher order terms I can never have boundedness of it and at the point of the kink, and thus no mesh refinement can achieve decreasing error at that bad point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2012-07-08T19:19:02.320" Title="regularity of a solution and its affect on the global error" Tags="|pde|finite-difference|parabolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2739" PostTypeId="2" ParentId="2738" CreationDate="2012-07-08T19:19:02.320" Score="1" Body="&lt;p&gt;Your diffusion only acts in $x$ direction and so, consequently, any kinks in the solution will only be smoothed away if the are along lines in $x$ direction. But, in your case, the discontinuity is only in $y$ direction, and consequently, the kink will never disappear -- it will only be transported. The easiest way to see this is to think of the solution as the limit $\Delta t \rightarrow 0$ of an operator splitting approach in which you first apply the diffusion step in $x$ direction and then the transport step in $y$ direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on this hypothetical operator splitting scheme, you will realize that any numerical method will only converge if you make the transport part converge. There are of course many ways of doing this, most notably things like SUPG or artificial diffusion. To understand why convergence for these methods does indeed happen, local truncation error analysis is not a very useful approach. You will need to read the literature on discretizations of transport equations and adapt the reasoning there.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-08T19:19:02.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2740" PostTypeId="2" ParentId="2736" CreationDate="2012-07-08T19:22:52.940" Score="1" Body="&lt;p&gt;This is one of the many reasons why the analysis of finite element methods is so much simpler than that of finite difference methods...&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-07-15T18:30:07.750" LastActivityDate="2012-07-15T18:30:07.750" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2741" PostTypeId="2" ParentId="2735" CreationDate="2012-07-08T19:24:14.077" Score="3" Body="&lt;p&gt;I am 100% sure that there is literature from the 1970s and 1980s that show what you need. Take a look and see whether there's anything in Vidar Thomee's papers that helps.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-08T19:24:14.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2742" PostTypeId="1" AcceptedAnswerId="2744" CreationDate="2012-07-08T21:14:53.640" Score="3" ViewCount="318" Body="&lt;p&gt;I'm looking into integrals of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_a^b {f(x)g(x)dx}$$&#xA;Where $f(x)$ is unknown, but it's integral is:&#xA;$$\int_a^b {f(x)dx}=F$$&#xA;It's been suggested to me that one could approximate this integral by:&#xA;$$\int_a^b {f(x)g(x)dx}\sim F \cdot\overline{g(x)}=\frac{F}{b-a}\int_a^b {g(x)dx}$$&#xA;This of course being just a &quot;good guess&quot; form of the integral mean value theorem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is the following:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would one go about calculating the error bounds on this approximation? are there any conditions on $f(x)$ that would tighten the said error bounds?&lt;/p&gt;&#xA;" OwnerUserId="1699" LastActivityDate="2012-07-08T22:31:57.473" Title="Integral average approximation and error bounds" Tags="|approximation|error-estimation|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2744" PostTypeId="2" ParentId="2742" CreationDate="2012-07-08T22:31:57.473" Score="4" Body="&lt;p&gt;For simplicity's sake, let's just say that $a=0,b=1$. Then you want to estimate&#xA;$$&#xA;  \int_0^1 f(x) g(x) dx - \left(\int_0^1 f(x) dx \right) \left(\int_0^1 g(x) dx\right)&#xA;$$&#xA;from above and below. Furthermore, let me consider two cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If $g(x)$ has mean value zero, i.e., $\int_0^1 g(x) dx = 0$, then you know a priori that your approximation isn't a good one. Let's not consider this case any more then.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If $g(x)$ has a mean value different than zero. Let's only consider this case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since in this second case it doesn't make a difference in the problem, let us assume for simplicity that $g(x)$ has been scaled in such a way that $\int_0^1 g(x) dx = 1$. Then you are looking for estimates of the term&#xA;$$&#xA;  \int_0^1 f(x) g(x) dx - \int_0^1 f(x) dx&#xA;$$&#xA;from above and below where $g(x)$ has mean value one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider a sequence of functions $f_n(x)=(n+1)x^n$. Then, assuming that $g(x)$ is bounded,&#xA;$$&#xA; \int_0^1 f_n(x) g(x) dx \rightarrow 0, \qquad \int_0^1 f_n(x) dx =1&#xA;$$&#xA;so your relative error can become arbitrarily large, and by tweaking the sign of $f_n,g$ you can make this an error that can be positive or negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, all this shows is that you need to say in which class (or function space) you are looking for your function $f(x)$. Each function $f_n$ above is $C^\infty$, and its limit is in $L_1$, so you can't expect an error bound to hold in $L_1$. But the limit function is not in in $L_p,p&amp;gt;1$ and so it may be that you can find, for example, an estimate in $L_\infty$. In other words, more information is necessary to answer your question.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-08T22:31:57.473" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2745" PostTypeId="1" CreationDate="2012-07-08T23:01:33.913" Score="10" ViewCount="852" Body="&lt;p&gt;I am trying to diagonalize some dense, ill-conditioned matrices. In machine precision, results are inaccurate (returning negative eigenvalues, eigenvectors do not have the expected symmetries). I switched over to Mathematica's Eigensystem[] function to take advantage of arbitrary precision, but computations are extremely slow. I am open to any number of solutions. Are there packages/algorithms that are well suited to ill-conditioned problems? I am not an expert on preconditioning, so I am not sure how much this could help. Otherwise, all I can think of are parallelized arbitrary precision eigenvalue solvers, but I am not familiar with anything beyond Mathematica, MATLAB and C++. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give some background on the problem, the matrices are large, but not huge (4096x4096 to 32768x32768 at the most). They are real, symmetric, and the eigenvalues are bounded between 0 and 1 (exclusive), with many eigenvalues being very close to 0 and none close to 1. The matrix is essentially a convolution operator. I do not need to diagonalize all of my matrices, but the larger I can go, the better. I have access to computing clusters with many processors and distributed computing capabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="1739" LastEditorUserId="127" LastEditDate="2012-07-12T05:35:02.213" LastActivityDate="2012-07-12T05:35:02.213" Title="Diagonalization of Dense Ill Conditioned Matrices" Tags="|linear-algebra|parallel-computing|eigensystem|precision|dense-matrix|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2746" PostTypeId="1" AcceptedAnswerId="2751" CreationDate="2012-07-09T03:55:30.520" Score="2" ViewCount="1117" Body="&lt;p&gt;I have two points $x_1, x_2$ between which I would like to have a linear interpolation $P_1$. Those two points are just points where I know the value of the underlying function $f$. I know that the error at any point between the two will be bounded by the second derivative of the function $\sup_{x\in[x_1,x_2]}f^{''}(x)$ and some multiplying constant. Thus, I have to require that the second derivative exist and bounded. However, assume my underlying function is only continuous, for example $|x|$, so the second derivative exist only in a weak sense, it is defined a.e. except this bad point $0$. So, what is the resulting error then? Does it depend on the fact that $0$ is one of those points $x_1$ or $x_2$, but what if $0\in[x_1,x_2]$? Can I still state some error estimates in this case?  &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="127" LastEditDate="2012-07-12T05:35:46.890" LastActivityDate="2012-07-12T05:35:46.890" Title="error of linear interpolation" Tags="|interpolation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2747" PostTypeId="2" ParentId="2746" CreationDate="2012-07-09T06:46:10.103" Score="2" Body="&lt;p&gt;While I believe this question would get a more in-depth answer on math.stackexchange.com I'll try to give you a sufficient answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your example with the absolute value may also not be capturing your question entirely, since its first derivative also doesn't exist at the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only have a single discontinuity in any of the derivatives (including the first), you can get a rather loose error bound of $M |x_i - x|$ where M is the bound on $f'(x)$ on the interval $[x_i, x]$ and $x_i$ is the endpoint of your interpolating interval chosen so that $[x_i, x]$ does not contain the discontinuity.  An analysis showing this is below if you are interested, however it only holds for the single discontinuity case, and is a pretty horrible error bound, considering just using $f(x_0)$ or $f(x_1)$ would give approximately the same provable error bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's start with a function $f(x)$ from which we know $f(x_0) = y_0$ and $f(x_1) = y_1$.  We also know that $f'(x)$ does not exist at a single point, $x_d \in (x_0, x_1)$.  We approximates $f(x)$ with the formula $p = y_0 + (y_1 + y_0)/(x_1 + x_0)*(x - x_0)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we want to estimate $|f(x) - p|$.  With a little bit of algebra and the triangle inequality we can show that this value is less than or equal to $|f(x) - y_0| + |-(y_1 + y_0)/(x_1 + x_0)*(x - x_0)|$.  The second term here is well known, since we know $x$.  All we need to know now is how far $f(x)$ deviates from $f(x_0)$ in the displacement from $x_0$ to $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also know that $f(x) - f(x_0)= \int_{x_0}^x f'(x)$.  If we have that $f'(x)$ is bounded on the interval $[x_0 x]$ by some value $M$, we then know that the maximum value of $\int_{x_0}^x f'(x)$ could be is $M |x - x_0|$.  Of course if the discontinuity is in the interval $[x_0, x]$, we then know that it is not in the interval $[x, x_1]$, so you could perform a similar analysis from the $x_1$ point rather than the $x_0$ point.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-07-09T06:46:10.103" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2748" PostTypeId="1" AcceptedAnswerId="2753" CreationDate="2012-07-09T09:43:44.937" Score="26" ViewCount="1284" Body="&lt;p&gt;Generally speaking, I've heard numerical analysts utter the opinion that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Of course, mathematically speaking, time is just another dimension, but still, time &lt;em&gt;is&lt;/em&gt; special&quot; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How to justify this? In what sense is time special for computational science? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, why do we so often prefer to use finite differences, (leading to &quot;time-stepping&quot;), for the time dimension, while we apply finite differences, finite elements, spectral methods, ..., for the spatial dimensions? One possible reason is that we tend to have an IVP in the time dimension, and a BVP in the spatial dimensions. But I don't think this fully justifies it.&lt;/p&gt;&#xA;" OwnerUserId="1740" LastEditorUserId="1740" LastEditDate="2012-07-11T10:11:07.830" LastActivityDate="2012-07-12T17:21:37.347" Title="Why is the time dimension special?" Tags="|pde|discretization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2749" PostTypeId="2" ParentId="2745" CreationDate="2012-07-09T13:34:58.980" Score="7" Body="&lt;p&gt;Calculate the SVD in place of the spectral decomposition. The results are the same in exact arithmetic, as your matrix is symmetric positve definite, but in finite precision arithmetic, you'll get the small eigenvalues with much more accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: See Demmel &amp;amp; Kahan, Accurate Singular Values of Bidiagonal Matrices,&#xA;SIAM J. Sci. Stat. Comput. 11 (1990), 873-912.&lt;BR&gt;&#xA;&lt;a href=&quot;ftp://netlib2.cs.utk.edu/lapack/lawnspdf/lawn03.pdf&quot; rel=&quot;nofollow&quot;&gt;ftp://netlib2.cs.utk.edu/lapack/lawnspdf/lawn03.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit2; Note that no method will be able to resolve eigenvalues smaller than about the norm times the machine accuracy used, as changing a single entry by one ulp may already change a small eigenvalue by this much. Thus getting zero eigenvalues in place of very tiny ones is appropriate, and no method (except working with higher precision) will disentangle the corresponding eigenvectors, but just return a basis for the common numerical null space.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-07-11T07:17:19.687" LastActivityDate="2012-07-11T07:17:19.687" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2750" PostTypeId="2" ParentId="2737" CreationDate="2012-07-09T13:49:43.383" Score="2" Body="&lt;p&gt;The discrete $H^1$ norm of the error is simply $\sqrt{e^T A_h e}$, so you immediately get the equation&#xA;$$&#xA;  e^T A_h e = e^T \tau = e^T A_h^{T/2} A_h^{-1/2} \tau&#xA;$$&#xA;and thus&#xA;$$&#xA;  e^T A_h e \le \sqrt{e^T A_h e } \sqrt{\tau^T A_h^{-1} \tau}&#xA;$$&#xA;and&#xA;$$&#xA;  \|e\|_{H^1 \textrm{discrete}} = \sqrt{e^T A_h e} \le \sqrt{\tau^T A_h^{-1} \tau}&#xA;$$&#xA;You treat the term on the right as always with truncation errors.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-09T13:49:43.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2751" PostTypeId="2" ParentId="2746" CreationDate="2012-07-09T13:56:17.783" Score="6" Body="&lt;p&gt;Yes, the error can be estimated without assuming twice differentiability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $f$ is Lipschitz continuous with Lipschitz constant $L$, the maximal error for linear interpolation of $f$ in $x_1$ and $x_2$ is $Lh/2$, where $h=|x_2-x_1|$.&#xA;(The worst case is easily seen to be that of interpolating a linear function with a single kink and slopes $\pm L$, and a little geometry then produces the stated optimal bound.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-09T13:56:17.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2752" PostTypeId="2" ParentId="2748" CreationDate="2012-07-09T15:35:08.140" Score="6" Body="&lt;p&gt;While there are some exceptions (e.g. fully discrete finite element methods), temporal discretization generally implies an inherently sequential dependence in flow of information.  This dependency restricts semi-discrete algorithms (BVP in space, IVP in time) to compute solutions to subproblems in sequential manner.  This discretization is usually preferred for its simplicity and because it offers the analyst many well-developed algorithms for higher accuracy both in space and time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible (and simpler) to use finite differences in spatial dimensions as well, but finite element methods offer easier flexibility in the type of domain on interest (e.g. non-regular shapes) than finite difference methods.  A &quot;good&quot; choice of spatial discretization is often very problem dependent.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-07-09T15:35:08.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2753" PostTypeId="2" ParentId="2748" CreationDate="2012-07-09T16:22:46.870" Score="27" Body="&lt;p&gt;Causality indicates that information only flows forward in time, and algorithms should be designed to exploit this fact. Time stepping schemes do this, whereas global-in-time spectral methods or other ideas do not. The question is of course why everyone insists on exploiting this fact -- but that's easy to understand: if your spatial problem already has a million unknowns and you need to do 1000 time steps, then on a typical machine today you have enough resources to solve the spatial problem by itself one timestep after the other, but you don't have enough resources to deal with a coupled problem of $10^9$ unknowns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation is really not very different from what you have with spatial discretizations of transport phenomena either. Sure, you can discretize a pure 1d advection equation using a globally coupled approach. But if you care about efficiency, then the by far best approach is to use a downstream sweep that carries information from the inflow to the outflow part of the domain. That's exactly what time stepping schemes do in time.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-07-12T17:21:37.347" LastActivityDate="2012-07-12T17:21:37.347" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2754" PostTypeId="2" ParentId="2748" CreationDate="2012-07-09T18:42:26.853" Score="9" Body="&lt;p&gt;Similar to the causality Wolfgang mentioned in his post, we could see the reason why time dimension is special from Minkowski spacetime point of view:$&#xA;\newcommand{\rd}{\mathrm{d}}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $(3+1)$-dimensional spacetime has an inner product defined as&#xA;$$&#xA;(A,B) = A_x B_x + A_y B_y + A_z B_z - \dfrac{1}{c^2}A_t B_t&#xA;$$&#xA;if $A$ and $B$ are two 1-form in Minkowski spacetime:&#xA;$A = A_x \rd x + A_y \rd y + A_z \rd z + A_t \rd t$, $B$ is defined in a similar fashion, the intuition behind defining an inner product(or rather to say, metric) is to imposing the idea of absolute light speed, such that two different points(events) in the spacetime have zero distance(happens at the &quot;same time&quot;, like we are observing the motion of galaxies billions of lightyears away as if they are moving right now) if they are on the same light cone. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, this inner product is not positive definite due to the presence of the time dimensional scaled by the light speed $c$, therefore intuitively speaking, when treating a problem concerning a quantity propagating in the spacetime, we cannot simply apply theorems in 3-dimensional Euclidean metric to a $(3+1)$-dimensional spacetime, just think of 3-dimensional elliptic PDE theories and their corresponding numerical methods differ drastically from the hyperbolic PDE theories.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Maybe off-topic, but another major difference of space vs spacetime(elliptic vs hyperbolic) is that most elliptic equations model the equilibrium and ellipticity gives us &quot;nice&quot; regularity, while there are all kinds of discontinuities in hyperbolic problems(shock, rarefaction, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I don't know there is a dedicated article about the difference other than giving you the definition, based what I learned before, typical elliptic equation like Poisson equation or elasticity, models a static phenomenon, has &quot;smooth&quot; solution if data and boundary of domain of interest are &quot;smooth&quot;, this is due the ellipticity(or rather to say positive definite property) of the governing differential operator, this type of equations leads us to a very intuitive Galerkin type approach(multiply a test function and integration by parts), typical continuous finite element works well. Similar things apply to parabolic equation like heat equation, which is essentially an elliptic equation marching in time, has a similar &quot;smoothing&quot; property, an initial sharp corner will be smoothened out over time, we call this &quot;diffusive&quot; or &quot;dissapative&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a hyperbolic problem, normally derived from a conservation law, is &quot;conservative&quot; or &quot;dispersive&quot;. For example, linear advection equation, describing the certain quantity flows with a vector field, conserves how this specific quantity is like initially, just it moves spacially along this vector field, the discontinuities will propagate. Schrodinger equation, another hyperbolic equation, however, is dispersive, it is the propagation of a complex quantity, a non-oscillatory initial state will become different oscillatory wave packets over time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you mentioned &quot;time-stepping&quot;, you could think the quantity &quot;flows&quot; in the time &quot;fields&quot; with a certain velocity as the causality, very similar to the linear advection equation BVP, we only have to impose the inflow boundary condition, ie, what the quantity is like when flowing into the domain of interest, and the solution would tell us what the quantity is like when flowing out, an idea very similar to every method that uses time-stepping. Solving a 2D advection equation in space is like solving a 1D one-sided propagation problem in  spacetime. For numerical schemes, you could google about spacetime FEM.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-07-11T17:56:37.337" LastActivityDate="2012-07-11T17:56:37.337" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2755" PostTypeId="1" AcceptedAnswerId="2759" CreationDate="2012-07-09T18:47:53.227" Score="6" ViewCount="618" Body="&lt;p&gt;I have a low speed flow with a high voltage discharge occurring within it between two spherical electrodes.  We have quite a bit of data from the experiment and have performed 0-D modeling of the chemical kinetics within the plasma discharge, however our next step is to expand the modeling with 1 or 2 spacial dimensions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One issue with this is that we have a shockwave propagating from the plasma and one of the electrodes that moves downstream.  To properly capture this shockwave at all time with a single mesh would require a very fine meshing for the first couple centimeters around the plasma.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, since we have Schlieren images of the shockwave, and know where it is in time, would it be worth while to recompute a mesh for each time step with refinement only where the shock should be, interpolate onto it, and then advance the time rather than using a static mesh.  The static mesh would avoid two of the previous steps, however would likely require time advancing on meshes with significantly more points than the dynamic case.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="123" LastEditDate="2012-07-10T10:20:41.107" LastActivityDate="2012-07-10T10:20:41.107" Title="Efficiency of a Dynamic Mesh vs. a Static Mesh for a Propagating Shockwave" Tags="|adaptive-mesh-refinement|mesh|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2756" PostTypeId="2" ParentId="2755" CreationDate="2012-07-09T20:13:24.717" Score="2" Body="&lt;p&gt;I think your suggestion makes a lot of sense. It even avoids that you have to define a refinement criterion to decide where to refine and where to coarsen.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-09T20:13:24.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2757" PostTypeId="1" CreationDate="2012-07-09T21:46:02.483" Score="5" ViewCount="130" Body="&lt;p&gt;My friend was working on this, and he asked me about the stability of &lt;a href=&quot;http://en.wikipedia.org/wiki/Perfectly_matched_layer&quot; rel=&quot;nofollow&quot;&gt;PML&lt;/a&gt; while applying on pseudo-spectral method, I believe his concern was how to choose the difference(if the difference should be chosen with a directional preference, forward/backward, or inflow/outflow) such that the numerical solution is stable over time. I am nowhere an expert on this topic and do not even know how to choose the keywords to google, so here I am requesting some references. Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="549" LastEditDate="2012-07-11T02:48:13.577" LastActivityDate="2012-07-11T02:48:13.577" Title="How to choose a stable PML for pseudo-spectral method with strongly varying velocity" Tags="|finite-difference|reference-request|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2758" PostTypeId="2" ParentId="2745" CreationDate="2012-07-09T21:56:35.887" Score="1" Body="&lt;p&gt;Thank you for this suggestion. I tried Mathematica's SVD command, but I get no noticeable improvement (still missing appropriate symmetries, 'eigenvalues' are incorrectly zero where they were incorrectly coming out negative before). Maybe I would need to implement one of the algorithms you describe above instead of a built-in function? I would probably want to avoid going to the trouble of using a specific method like this unless I was sure ahead of time that it would offer a significant improvement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@JackPoulson, I skimmed the paper on Jacobi's method you referenced, and it looks promising. Can you or anyone recommend a good way to implement Jacobi's method for finding eigensystems? I am guessing that if I coded it up myself (in MATLAB), it would be extremely slow.&lt;/p&gt;&#xA;" OwnerUserId="1741" LastActivityDate="2012-07-09T21:56:35.887" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2759" PostTypeId="2" ParentId="2755" CreationDate="2012-07-09T22:42:47.707" Score="6" Body="&lt;p&gt;Adaptive mesh refinement is a very useful technique for improving accuracy around shocks, since any method will be at most first order accurate near solution discontinuities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside of implementing AMR is that it will add substantial complexity to your code and thus require significant additional development and maintenance time.  I would only do the work of implementing AMR if your target problems cannot be solved in an acceptable length of time without it.  I think it makes sense to &lt;strong&gt;implement a uniform grid solver first and see just how slow it is&lt;/strong&gt;, since you would need a verified uniform grid solver as a starting point for your adaptive code anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably already know this: you'll need to be careful about maintaining conservation on the adaptive mesh.  For that (and for a basic understanding of AMR), consult the seminal &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0021999184900731&quot;&gt;paper of Berger and Oliger&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-07-09T22:42:47.707" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2760" PostTypeId="1" CreationDate="2012-07-10T08:13:04.907" Score="4" ViewCount="2087" Body="&lt;p&gt;What is the best software for solving and simulating the 3D Navier-Stokes equation for incompressible laminar non-Newtonian fluid flow?&lt;/p&gt;&#xA;" OwnerUserId="1743" LastEditorUserId="3980" LastEditDate="2013-05-08T20:09:30.357" LastActivityDate="2013-05-08T20:09:30.357" Title="Software for 3D Navier-Stokes equation" Tags="|pde|software|fluid-dynamics|navier-stokes|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2761" PostTypeId="2" ParentId="2760" CreationDate="2012-07-10T08:27:52.610" Score="7" Body="&lt;p&gt;3D Navier-Stokes problems are already difficult to analyze, not to mention rheological concerns. I could have suggested &lt;a href=&quot;http://www.ansys.com/Products/Simulation+Technology/Fluid+Dynamics/ANSYS+Polyflow&quot; rel=&quot;nofollow&quot;&gt;Polyflow&lt;/a&gt; of Ansys but I will not because I believe it is a poorly executed idea. GUI is not good, solver options are tricky to adjust. Actually, I suggest not to rely on commercial software. Solvers in this area are not that general, they are mostly developed for specific industries, like &lt;a href=&quot;http://usa.autodesk.com/moldflow/&quot; rel=&quot;nofollow&quot;&gt;Moldflow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, I would go with &lt;a href=&quot;http://openfoam.org&quot; rel=&quot;nofollow&quot;&gt;OpenFOAM&lt;/a&gt;. The number of non-newtonian &lt;a href=&quot;https://github.com/OpenFOAM/OpenFOAM-2.1.x/tree/master/src/transportModels/incompressible/viscosityModels&quot; rel=&quot;nofollow&quot;&gt;models&lt;/a&gt; might not be enough for you but you still have the opportunity to adapt your own model since the code is freely available. Also, there are some efforts to implement some advanced problems, an example is in &lt;a href=&quot;http://powerlab.fsb.hr/ped/kturbo/openfoam/slides/viscoelasticFluidFoam.pdf&quot; rel=&quot;nofollow&quot;&gt;viscoelasticity&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1499" LastEditorUserId="1499" LastEditDate="2012-07-12T12:21:50.387" LastActivityDate="2012-07-12T12:21:50.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2762" PostTypeId="1" CreationDate="2012-07-10T11:02:53.900" Score="1" ViewCount="6788" Body="&lt;p&gt;I currently searching for a subroutine from BLAS or LAPACK which realizes the following operation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = alpha*A + beta * B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where A and B have different leading dimensions, i.e. A is stored as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; REAL*8 A(N,N)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and B is stored as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; REAL*B B(LDB,N)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with LDB &gt; N. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to use existing BLAS or LAPACK operation to compute this or must I create an own subroutine for this kind of problems? &lt;/p&gt;&#xA;" OwnerUserId="569" LastActivityDate="2014-08-17T05:37:26.350" Title="BLAS/LAPACK subroutine to add two matrices with different offsets and leading dimensions" Tags="|linear-algebra|lapack|blas|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2763" PostTypeId="2" ParentId="2737" CreationDate="2012-07-10T12:44:58.737" Score="4" Body="&lt;p&gt;Maybe a note on the motivation to use the $L^2$-error. It is natural to estimate the error in the input of a partial differential equation in the $L^2$-norm. The mathematical motivation of this is the Gauss-Markov theorem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem#Generalized_least_squares_estimator&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem#Generalized_least_squares_estimator&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It states that, if your measurement satisfies intuitive probability laws, then a least square approximation based on measurements will provide the best approximation with high probability. This choice of norm propagates within partial differential equations, so you end with $L^2$-Sobolev norms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we can use this norm, too, for the error estimates in Galerkin methods, mainly because we have no better idea how to estimate the error.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-07-10T12:44:58.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2764" PostTypeId="1" AcceptedAnswerId="3011" CreationDate="2012-07-10T16:21:19.927" Score="2" ViewCount="506" Body="&lt;p&gt;I would like to solve for the electric potential and magnetic vector potential using the finite volume method (collocated grid). My equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla\cdot(\sigma\nabla\phi)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \cdot \nabla A = \mu_m \sigma \nabla\phi$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The domain is a wedge, with the following boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A fixed gradient for the electric potential $\phi$ on the top, zero gradient at the bottom and a zero value at the side boundary&lt;/li&gt;&#xA;&lt;li&gt;Zero gradient for the magnetic vector potential $A$ on all boundaries&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I didn't expect this would be a tough problem, but the linear solver (GAMG) only converges with under-relaxation, and the result is dependent on the value of the under-relaxation factor. Can anyone give me a hint where things are going wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional information: The software package I use is OpenFOAM. The gradients are discretized using the Gauss linear scheme, and the laplacian with &quot;Gauss linear corrected&quot; (explicit non-orthogonal correction).&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="59" LastEditDate="2012-07-11T07:18:19.307" LastActivityDate="2012-09-21T14:51:09.850" Title="Finite volume solution of electrostatics using magnetic vector potential" Tags="|convergence|finite-volume|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2765" PostTypeId="2" ParentId="2762" CreationDate="2012-07-10T18:07:53.300" Score="2" Body="&lt;p&gt;The best you can do is to use the BLAS &lt;code&gt;AXPY&lt;/code&gt; on each column, but that doesn't let you apply &lt;code&gt;beta&lt;/code&gt;. You really might as well write your own since this is basically a level 1 or 2 BLAS operation, which for large matrices is memory bandwidth limited.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-07-10T18:07:53.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2766" PostTypeId="2" ParentId="1898" CreationDate="2012-07-10T21:17:06.727" Score="2" Body="&lt;p&gt;It's the old trick...&#xA;(first seen &lt;a href=&quot;http://www.citeulike.org/user/miccoli/article/9471000&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; as far as I know.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For brevity I will introduce $Ku = f$ with&#xA;\begin{align}&#xA;K &amp;amp;= \begin{bmatrix}&#xA;       K_{uu} &amp;amp; K_{um} &amp;amp; K_{us} \\&#xA;       K^{T}_{um} &amp;amp; K_{mm} &amp;amp; K_{ms} \\&#xA;       K^{T}_{us} &amp;amp; K^{T}_{ms} &amp;amp; K_{ss}&#xA;\end{bmatrix}&#xA;&amp;amp;&#xA;u &amp;amp;= \begin{bmatrix}&#xA;       u_{u}\\&#xA;       u_{m}\\&#xA;       u_{s}&#xA;\end{bmatrix}&#xA;&amp;amp;&#xA;f = \begin{bmatrix}&#xA;       f_{u}\\&#xA;       f_{m}\\&#xA;       f_{s}&#xA;\end{bmatrix}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let&#xA;\begin{align}&#xA;v &amp;amp;= \begin{bmatrix}&#xA;u_u \\ u_m&#xA;\end{bmatrix}&#xA;&amp;amp;&#xA;B &amp;amp;= \begin{bmatrix}&#xA;I &amp;amp; 0 \\&#xA;0 &amp;amp; I \\&#xA;0 &amp;amp; T \\&#xA;\end{bmatrix}&#xA;&amp;amp;&#xA;b &amp;amp;= \begin{bmatrix}&#xA;0 \\ 0 \\ g&#xA;\end{bmatrix}&#xA;\end{align}&#xA;so that&#xA;\begin{equation}&#xA;u = Bv + b&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;first insert $u_s$ (which by this notation is substituting $u$ from the equation above)&#xA;\begin{equation}&#xA;K(Bv+b) = f&#xA;\end{equation}&lt;/li&gt;&#xA;&lt;li&gt;then &lt;em&gt;symmetrize&lt;/em&gt; by premultiplication by $B^T$&#xA;\begin{equation}&#xA;B^T K(Bv+b) = B^T f&#xA;\end{equation}&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;leading to&#xA;\begin{equation}&#xA;B^T K B \:v = B^T (f - Kb)&#xA;\end{equation}&#xA;which, with a few substitutions, is the result you are looking for.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-07-10T21:23:28.283" LastActivityDate="2012-07-10T21:23:28.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2767" PostTypeId="1" AcceptedAnswerId="2768" CreationDate="2012-07-10T21:47:39.493" Score="6" ViewCount="684" Body="&lt;p&gt;I have a question regarding stability analysis for constant coefficient pde. Suppose I am looking at the pde $$u_t= au_{xx}$$ So the first approach is to compute the amplification factor, and this is so-called Fourier approach. Call it $Q(\zeta)$ and for explicit method plugging the test function $e^{ijhw}$ I have for $\zeta=hw$ that $$Q(\zeta)=1-4a\tau/h^2sin^2(\zeta/2)$$ &#xA;On the other hand, the eigenvalue analysis for matrix $I+\tau A$, where $A$ is the discrete aproximation of $au_{xx}$ gives the eigenvalues of the matrix $I+\tau A$ that are exactly the same as the amplification factor $Q(\zeta)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are they always the same, are these approaches equivalent? So, I can calculate either one and make sure it is bounded by $1$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And second question. Since $A$ is symmetric, it implies $\max \lambda_i=||I+\tau A||_2$. However, if I use non uniform grid then $\max |\lambda_i| \leq ||I+\tau A||_2$ and thus neither of the two would guarantee stability because I can only show that $\max |\lambda_i|\leq 1$. What would be a sufficient condition for stability then in the case of a nonuniform grid?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2012-07-11T08:24:15.530" Title="eigenvalue analysis vs fourier analysis for stability and their equivalence" Tags="|pde|finite-difference|parabolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2768" PostTypeId="2" ParentId="2767" CreationDate="2012-07-11T08:24:15.530" Score="3" Body="&lt;p&gt;Answer to question 1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Circulant_matrix&quot; rel=&quot;nofollow&quot;&gt;circulant matrix&lt;/a&gt;, then &lt;a href=&quot;http://en.wikipedia.org/wiki/Circulant_matrix#Eigenvectors_and_eigenvalues&quot; rel=&quot;nofollow&quot;&gt;its eigenvectors are of the form $v_{j,w} = e^{ijhw}$; i.e., they are discrete Fourier modes&lt;/a&gt;.  So Fourier analysis (or &lt;a href=&quot;http://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&quot; rel=&quot;nofollow&quot;&gt;von Neumann analysis&lt;/a&gt;, as it's usually called) and eigenvalue analysis are equivalent if and only if you're dealing with a &lt;strong&gt;circulant matrix&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Answer to question 2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The maximum eigenvalue modulus and the Euclidean norm are equal if $A$ is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Normal_matrix&quot; rel=&quot;nofollow&quot;&gt;normal matrix&lt;/a&gt;.  If $A$ is non-normal, then the Euclidean norm can be larger, as you say.  A sufficient condition for stability is, of course, $\|I+\tau A\|_2 \le 1$.  There are other sufficient conditions, depending on what kind of stability you mean.  The most important kind of stability for PDE discretizations is &lt;strong&gt;Lax-Richtmeyer stability&lt;/strong&gt;, which in your case would mean&#xA;$$\|(I+\tau A)^n\|&amp;lt; C$$&#xA;for all $n\tau&amp;lt;T$, where $C$ is a constant that may depend on $T$ but not on $n$ or $\tau$.  For a discussion, see e.g. Chapter 9 of &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow&quot;&gt;Leveque's text&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proving this kind of stability for a non-uniform grid discretization will probably be difficult.  In practice, it is usually sufficient to consider the stability criterion based on a uniform grid, e.g. $\Delta t \le (\Delta x)^2/2$ and apply it but with $\Delta x$ replaced by the minimum of $\Delta x$ over the whole grid.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-07-11T08:24:15.530" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2769" PostTypeId="1" AcceptedAnswerId="2770" CreationDate="2012-07-11T12:36:51.420" Score="4" ViewCount="1137" Body="&lt;p&gt;Given SPD matrix $$A \in \mathbb{R}^{N \times N} $$ and positive diagonal matrix $$D \in \mathbb{R}^{N\times N}.$$ What is then spectrum of the product $$D^TAD.$$ Is there a closed-form relationship between spectra of $A, D$ and that of the product? Or at least can we find bounds of it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g., one can see that if $D = \alpha I$ then $cond(A) = cond(D^TAD)$ for any $\alpha &amp;gt; 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: In practice I observe that for arbitrary positive $D$ the $cond(A) &amp;lt; cond(D^TAD)$. However, I can not explain or understand it. Any clues are appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;:&#xA;The following MATLAB script generates $k$ random matrices $A$ and $D$ and then $i$ times decreases the smallest diagonal element of $D$ by factor $r &amp;lt; 1$, calculating at each iteration condition number of the product $D^TAD$ with updated $D$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sz = 100;&#xA;&#xA;v = [];&#xA;cw = []; ch = [];&#xA;&#xA;r = 0.75;&#xA;&#xA;for k=1:100&#xA;&#xA;    A = 4 + 1.*randn(sz);&#xA;    A = A'*A;&#xA;    W = diag(4 + 1.*randn(sz,1));&#xA;&#xA;    cw(1) = cond(W);&#xA;    ch(1) = cond(W'*A*W);&#xA;&#xA;    [c, j] = min(diag(W));&#xA;    for i=1:25&#xA;        W(j,j) = W(j,j) * r;&#xA;        cw(i+1) = cond(W);&#xA;        ch(i+1) = cond(W'*A*W);&#xA;    end&#xA;&#xA;    v(:,k) = (ch(2:end)./ch(1:end-1))';&#xA;end&#xA;plot(v);&#xA;axis([1 i 1 1/r^2]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I plot then ratios of condition numbers between iterations it approaches $1/r^2$. E.g., for $r = 0.75$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8lBDX.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it is never larger than $1/r^2$. It tells that perhaps there should be bound for condition number of the product $D^TAD$ or we can even compute it knowing spectrum of $A$, but I can't figure it out. Can anyone help?&#xA;Also, this effect seems to be independent of random generator and its parameters. &lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2012-07-12T07:13:20.507" LastActivityDate="2012-07-12T15:00:00.830" Title="Spectrum of the product of two matrices" Tags="|linear-algebra|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2770" PostTypeId="2" ParentId="2769" CreationDate="2012-07-11T14:18:38.083" Score="4" Body="&lt;p&gt;There is no useful relationship. You can see this by looking at the $2\times 2$ case, where explicit formulas exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit; You can see from your plot that the worst cases in you sample curves are already very far from the typical cases. The absolutely worst would be even further off. Thus though one can perhaps prove optimal bounds (by writing down the optimatity conditions and analysing them), they would tell nothing useful about the typcal situation. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-07-11T15:42:24.163" LastActivityDate="2012-07-11T15:42:24.163" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2771" PostTypeId="1" AcceptedAnswerId="2772" CreationDate="2012-07-11T14:21:24.323" Score="5" ViewCount="258" Body="&lt;p&gt;I'm dealing with a set of points which are already placed on the 2D hull boundary: a convex polygon. I know this for sure. However, the point set is not ordered, and I need the polygon points to be ordered counter-clockwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of all the 2D algorithms for convex hull generation from the books/literature, but I'm interested if there is a faster, specialized, algorithm for this special case? &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2012-07-11T14:44:22.413" LastActivityDate="2012-07-11T17:39:12.047" Title="Is there a special algorithm for computing the convex hull ordering when the candidate points are on the hull?" Tags="|algorithms|reference-request|computational-geometry|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2772" PostTypeId="2" ParentId="2771" CreationDate="2012-07-11T14:29:33.623" Score="5" Body="&lt;p&gt;Take the mean of all your points as origin and transform to polar coordinates. This gives the desired ordering in $O(n\log n)$ operations. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-07-11T14:52:29.007" LastActivityDate="2012-07-11T14:52:29.007" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2773" PostTypeId="2" ParentId="2769" CreationDate="2012-07-11T15:24:38.243" Score="0" Body="&lt;p&gt;Unless $A$ and $D$ are commutable, where the spectrum of the product is the product of spectra, there is no simple relationship.&lt;/p&gt;&#xA;" OwnerUserId="1614" LastActivityDate="2012-07-11T15:24:38.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2775" PostTypeId="1" CreationDate="2012-07-11T16:15:11.043" Score="1" ViewCount="3430" Body="&lt;p&gt;Suppose I am solving a pde with a solution known with a finite-difference method. I can represent it as $A_hu_h=f$ for some approximating matrix $A_h$. And I define the discrete norm in which I will analyze the convergence to be $$||e||^2=h\sum e_i^2$$. I have the estimate for the error $$||e||\leq C ||\tau||$$ for $\tau$ local error if first order in space and $C$ a stability bound for $A^{-1}$. Then, I want to see which error gives my program to confirm the theory. I pick a point on the grid and watch what happens as a double the mesh. Assume I see the error gets decreased by 2, so I suspect it is a linear convergence at that point. Thus, I can see only rate of convergence at one point. How can I guarantee that the convergence is uniform &quot;everywhere&quot; on the grid? to confirm the theory results, should not I measure the error at each point and see I have a linear convergence in order to claim that the error goes to zero linearly in a discrete norm defined above? Because I could state results in a similar norms such as discrete $L^{\infty}$ or discrete $L^1$, however, what I measure by the computer is the same: difference between numerical solution and the function value at the point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What confuses me is the fact that before I implement the method, I can state error estimates in a variety of norms, however, how do I relate these theoretical approximations to the error that I can actually measure, since this is just a value at the point on the grid and is independent from the way I do my theoretical analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I should rephrase the question: There are a number of papers, where the convergence is measured for a particular point on the grid and the results are stated to confirm the theoretical estimates that are done in discrete $L^{\infty}$ or $L^{2}$ for the error vector. Is this sufficient to measure only at one point to obtain the confirmation and why the results presented only for the error at one point?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2012-07-14T20:01:18.437" LastActivityDate="2012-07-15T05:48:02.843" Title="how to measure the error of a finite difference method" Tags="|pde|finite-difference|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2776" PostTypeId="2" ParentId="2771" CreationDate="2012-07-11T17:39:12.047" Score="2" Body="&lt;p&gt;Since the points are in 2D, you can use &lt;a href=&quot;http://en.wikipedia.org/wiki/Graham_scan&quot; rel=&quot;nofollow&quot;&gt;Graham's Scan&lt;/a&gt; to order the points instead of a sorting algorithm.  This will also give you a $\Theta(nlog(n))$ complexity without the need for any transformations.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-07-11T17:39:12.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2777" PostTypeId="1" AcceptedAnswerId="2793" CreationDate="2012-07-11T22:59:35.333" Score="2" ViewCount="653" Body="&lt;p&gt;I'm working from &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/src/ksp/ksp/examples/tutorials/ex2.c.html&quot; rel=&quot;nofollow&quot;&gt;ex2.c&lt;/a&gt; of PETSc's example codes.  On line 65, the code specifies:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MatMPIAIJSetPreallocation(A,5,PETSC_NULL,5,PETSC_NULL);  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I looked up the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; for MatMPIAIJSetPreallocation(), and understand that its purpose is to allocate memory for the matrix in a compressed sparse row format in parallel.  That is, to allocate memory only for the maximum number of non-zeros expected in each row.  According to this documentation, the second argument is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;d_nz - number of nonzeros per row in DIAGONAL portion of local submatrix&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not quite sure what is meant by &quot;diagonal portion of local submatrix&quot;.  Furthermore, why are 5 values needed for both the diagonal (d_nz) and off-diagonal (o_nz) portions of each row?  As I understand, the 5-point stencil laplacian problem being solved has a linear system of equations which requires a maximum of 5 non-zero entries per row.  The d_nz and o_nz values specified here seem to indicate to me that memory has been allocated for 10 values per row.  I'm sure I'm missing something in the interpretation of d_nz and o_nz. &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-07-12T20:16:58.373" Title="What are d_nz and o_nz in PETSc's MatMPIAIJSetPreallocation function?" Tags="|petsc|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2778" PostTypeId="2" ParentId="203" CreationDate="2012-07-12T02:04:10.233" Score="8" Body="&lt;p&gt;Be careful of urban myths here.&#xA;If two compilers generate the same assembly code, then the resulting programs will have the same performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any given piece of logic, there is a program in assembly language that minimizes its execution time. That program doesn't care which compiler generated it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, compiled languages exist to make life easier for the programmer.&#xA;Part of the cost of this is they may tempt the user into using features that don't result in minimal execution time.&#xA;The prime example of this is &lt;code&gt;new&lt;/code&gt; in C++.&#xA;(How slow could it be - it's only three characters?)&#xA;It practically begs you to dynamically allocate memory, and pay no attention to the runtime cost. If that's what you want to do, that's terrific, but Fortran could be faster just because it didn't suck you into doing that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But way beyond that, I've never seen a program that, as first written, didn't have &lt;em&gt;major&lt;/em&gt; room for performance improvement of the kind that the compiler could never clean up for you.&#xA;As one example, spending a large fraction of time calling &lt;code&gt;exp&lt;/code&gt; and/or &lt;code&gt;log&lt;/code&gt; repeatedly with the same argument.&#xA;As another example, calling DGEMM to multiply matrices and finding out that a large fraction of time goes into calling LSAME just to decipher its input character arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is at the same time as people are saying Fortran is faster because of pointer aliasing or loop unrolling.&#xA;That's like saying a bus made by Porsche would certainly be faster than a bus made by Chevrolet.&#xA;There needs to be a little common sense.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastEditorUserId="1262" LastEditDate="2015-12-14T13:29:06.473" LastActivityDate="2015-12-14T13:29:06.473" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2779" PostTypeId="1" AcceptedAnswerId="2790" CreationDate="2012-07-12T05:23:54.847" Score="13" ViewCount="511" Body="&lt;p&gt;Many algorithms used in scientific computing have a different inherent structure than algorithms commonly considered in less math-intensive forms of software engineering. In particular, individual mathematical algorithms tend to be highly complex, often involving hundreds or thousands of lines of code, yet nonetheless involve no state (i.e. are not acting upon a complex data structure) and can often be boiled down -- in terms of programmatic interface -- to a single function acting on an array (or two).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This suggests that a function, and not a class, is the natural interface to most algorithms encountered in scientific computing. Yet this argument offers little insight regarding how the implementation of complex, multi-part algorithms should be handled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While the traditional approach has been to simply have one function that calls a number of other functions, passing the relevant arguments along the way, OOP offers a different approach, wherein algorithms can be encapsulated as classes. For clarity, by encapsulating an algorithm in a class, I mean creating a class wherein the algorithm inputs are entered into the class constructor, and then a public method is called to actually invoke the algorithm. Such an implementation of multigrid in C++ psuedocode might look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class multigrid {&#xA;    private:&#xA;        x_, b_&#xA;        [grid structure]&#xA;&#xA;        restrict(...)&#xA;        interpolate(...)&#xA;        relax(...)&#xA;    public:&#xA;        multigrid(x,b) : x_(x), b_(b) { }&#xA;        run()&#xA;}&#xA;&#xA;multigrid::run() {&#xA;     [call restrict, interpolate, relax, etc.]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is then as follows: what are the benefits and drawbacks of this kind of practice as compared to a more traditional approach without classes? Are there issues of extensibility or maintainability? To be clear, I am not intending to solicit opinion, but rather to better understand the downstream effects (i.e. those that might not arise until a codebase becomes quite large) of adopting such a coding practice.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastActivityDate="2016-01-27T10:28:39.950" Title="What are the benefits and drawbacks inherent to using classes to encapsulate numerical algorithms?" Tags="|algorithms|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2780" PostTypeId="2" ParentId="2779" CreationDate="2012-07-12T07:47:07.070" Score="16" Body="&lt;p&gt;Encapsulation and data hiding are extremely important for &lt;em&gt;extensible&lt;/em&gt; libraries in scientific computing. Consider matrices and linear solvers as two examples. A user just needs to know that an operator is linear, but it may have internal structure such as sparsity, a kernel, a hierarchical representation, a tensor product, or a Schur complement. In all cases, Krylov methods do not depend on the details of the operator, they only depend on the action of the &lt;code&gt;MatMult&lt;/code&gt; function (and perhaps its adjoint). Similarly, the user of a linear solver interface (e.g. a nonlinear solver) only cares that the linear problem is solved, and should not need or want to specify the algorithm that is used. Indeed, specifying such things would impede the capability of the nonlinear solver (or other outer interface).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Interfaces&lt;/em&gt; are good. Depending on an &lt;em&gt;implementation&lt;/em&gt; is bad. Whether you achieve this using C++ classes, C objects, Haskell typeclasses, or some other language feature is inconsequential. The capability, robustness, and extensibility of an interface is what matters in scientific libraries.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-12T07:47:07.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2781" PostTypeId="1" CreationDate="2012-07-12T07:52:49.777" Score="1" ViewCount="176" Body="&lt;p&gt;What is the best known algorithm for exactly solving a large sparse system of linear equations? The system I'm working on is not symmetric, not positive definite and integer. The only benefit is being sparse. I also need to point out that the matrix is not square. The dimension is m×n and it is not generally either underestimate or overestimate.&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="549" LastEditDate="2012-07-12T13:45:28.467" LastActivityDate="2012-07-12T14:57:35.513" Title="Sparse non-square system of linear equations in exact arithmetic" Tags="|linear-algebra|" AnswerCount="2" CommentCount="9" ClosedDate="2012-07-13T00:59:25.567" ContentLicense="CC BY-SA 3.0" />
  <row Id="2782" PostTypeId="2" ParentId="2779" CreationDate="2012-07-12T08:12:06.653" Score="8" Body="&lt;p&gt;Classes should be used only if the structure of the code is hierarchical. Since you are mentioning Algorithms, their natural structure is a flow chart, not a hierarchy of objects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of OpenFOAM, the algorithmic part is implemented in terms of generic operators (div, grad, curl, etc) that are basically abstract functions operating on different types of tensors, using different types of numerical schemes. This part of the code is basically built from a lot of generic algorithms operating on classes. This allows the client to write something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solve(ddt(U) + div(phi, U)  == rho*g + ...);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hierarchies such as transport models, turbulence models, differencing schemes, gradient schemes, boundary conditions, etc are implemented in terms of C++ classes (again, generic on the tensor quantities).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've noticed a similar structure in the CGAL library, where the various algorithms are packed together as groups of function objects bundled with geometrical information to form geometrical Kernels (classes), but this is again done to separate operations from geometry (point removal from a face, from a point data type). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hierarchical structure ==&gt; classes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Procedural, flow chart ==&gt; algorithms&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-07-12T08:12:06.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2783" PostTypeId="5" CreationDate="2012-07-12T11:01:51.713" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2012-07-12T11:01:51.713" LastActivityDate="2012-07-12T11:01:51.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2784" PostTypeId="4" CreationDate="2012-07-12T11:01:51.713" Score="0" Body="Questions on the computational aspects of Fourier analysis, including the various applications of the fast Fourier transform (FFT)." OwnerUserId="127" LastEditorUserId="127" LastEditDate="2012-07-12T12:12:07.607" LastActivityDate="2012-07-12T12:12:07.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2785" PostTypeId="2" ParentId="2781" CreationDate="2012-07-12T12:32:49.523" Score="0" Body="&lt;p&gt;Krylov Iterative methods are a usual choice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you happen to have access to Mathematica, it offers a good way to test for different method: if A is your matrix, write B=SparseArray[A]; Then use the LinearSolve function with Method-&gt;&quot;Krylov&quot;. You can also test to see if there are advantages to retaining integer digits. Converting to real numbers may yield faster results, possibly at the cost of accuracy. &lt;/p&gt;&#xA;" OwnerUserId="1751" LastActivityDate="2012-07-12T12:32:49.523" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2786" PostTypeId="2" ParentId="2781" CreationDate="2012-07-12T14:57:35.513" Score="1" Body="&lt;p&gt;The exact solution of linear equations with rational coefficients belongs to the field of computer algebra. For an entry to the literature, see&lt;BR&gt;&#xA;    &lt;a href=&quot;http://www2.isye.gatech.edu/~dsteffy/papers/OSLifting.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www2.isye.gatech.edu/~dsteffy/papers/OSLifting.pdf&lt;/a&gt;&lt;BR&gt;&#xA;    &lt;a href=&quot;http://www2.isye.gatech.edu/~dsteffy/papers/rationalsolver.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www2.isye.gatech.edu/~dsteffy/papers/rationalsolver.pdf&lt;/a&gt;&lt;BR&gt;&#xA;    &lt;a href=&quot;http://www.eecis.udel.edu/~youse/post/itersolve.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.eecis.udel.edu/~youse/post/itersolve.pdf&lt;/a&gt;&lt;BR&gt;&#xA;    &lt;a href=&quot;http://www.lirmm.fr/~giorgi/issac06.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.lirmm.fr/~giorgi/issac06.pdf&lt;/a&gt;&lt;BR&gt;&#xA;You can do a literature search based on this and the citation facilities of &lt;a href=&quot;http://scholar.google.com&quot; rel=&quot;nofollow&quot;&gt;http://scholar.google.com&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-12T14:57:35.513" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2787" PostTypeId="2" ParentId="2769" CreationDate="2012-07-12T15:00:00.830" Score="2" Body="&lt;p&gt;If you are interested only in the $\kappa_2$ matrix condition number and not in the full spectrum, then your question is linked to linear system perturbation theory, where matrix $D$ is often used as a preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A well known result is &#xA;\begin{equation}&#xA;\kappa_2(D_* A D_*) \leq N\min_{D \in \mathcal{D}_N} \kappa_2(DAD)&#xA;\end{equation}&#xA;where $D_* = \mathrm{diag}(a_{ii}^{-1/2})$, so that $(D_*AD_*)$ has unit diagonal, see corollary 7.6 (van der Sluis) in N. J. Higham, Accuracy and Stability of Numerical Algorithms (2nd ed.), SIAM 2002.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2012-07-12T15:00:00.830" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2788" PostTypeId="1" AcceptedAnswerId="2834" CreationDate="2012-07-12T15:08:07.193" Score="0" ViewCount="297" Body="&lt;p&gt;Suppose I have a sparse stochastic matrix $M$ (with thousands or millions of stochastic column vectors), possibly encoding some links in a web graph. Now I split it into two matrices: $D$ containing only the diagonal entries of $M$, and $R$ containing the remaining entries of $M$. What would be a fast way to compute $D(I−R)^{−1}$ (or a good approximation)? Instead of low computational complexity, I'm looking for fast practical performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I actually care about is $D(I−R)^{−1}$ for a continuous stream of vectors $x$'s, though $M$ may also change (both nodes and edges), but less often. Looks similar to PageRank, but still quite different. What would be a fast implementation? - Thanks, Michelle&lt;/p&gt;&#xA;" OwnerUserId="1752" LastActivityDate="2012-07-16T01:50:45.500" Title="Quickly computing inversion of a large sparse partial stochastic matrix" Tags="|linear-algebra|matrix|sparse-matrix|graph-theory|approximation|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2789" PostTypeId="2" ParentId="2779" CreationDate="2012-07-12T15:41:45.830" Score="1" Body="&lt;p&gt;Two advantages of the OO approach could be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a long calculation from which you might or might not want different results. For example, if $\beta$ is the final output, but it depends on an intermediate result $\alpha$, you could have a &lt;code&gt;calculate_alpha()&lt;/code&gt; method that caches the $\alpha$ result inside the instance. Then when you call &lt;code&gt;calculate_beta()&lt;/code&gt;, it also calls &lt;code&gt;calculate_alpha()&lt;/code&gt; if no $\alpha$ result has been cached yet.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a calculation with several inputs, where if one input changes then the whole calculation doesn't necessarily have to be done over. For example, the &lt;code&gt;calculate_f()&lt;/code&gt; method returns $f(x,y,z)$. If you then decide to redo the calculation for another value of $z$, you can call &lt;code&gt;set_z()&lt;/code&gt; and the $z$ parameter is marked internally as 'dirty', so that when you call &lt;code&gt;calculate_f()&lt;/code&gt; again, only the portion of the calculation that depends on $z$ is redone.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1753" LastActivityDate="2012-07-12T15:41:45.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2790" PostTypeId="2" ParentId="2779" CreationDate="2012-07-12T17:42:53.843" Score="13" Body="&lt;p&gt;Having done numerical software for 15 years, I can unambiguously state the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Encapsulation is important. You do not want to pass around pointers to data (as you suggest) since it exposes the data storage scheme. If you expose the storage scheme, you can never change it again because you will access the data all over the entire program. The only way to avoid this is to encapsulate the data into private member variables of a class and let only member functions act on it. If I read your question, you think of a function that computes the eigenvalues of a matrix as stateless, taking a pointer to the matrix entries as argument and returning the eigenvalues in some way. I think this is the wrong way to think about it. In my view, this function should be a &quot;const&quot; memberfunction of a class -- not because it changes the matrix, but because it is one that operates with the data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Most OO programming languages allow you to have private member functions. This is your way to break apart one large algorithm into a smaller one. For example, the various helper functions you need for the eigenvalue computation still operate on the matrix, and so would naturally be private member functions of a matrix class.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compared to many other software systems, it may be true that class hierarchies are often less important than, say, in graphical user interfaces. There are certainly places in numerical software where they are prominent -- Jed outlines one in another answer to this thread, namely the many ways one can represent a matrix (or, more generally, a linear operator on a finite dimensional vector space). PETSc does this very consistently, with virtual functions for all operations that act on matrices (they don't call it &quot;virtual functions&quot;, but that's what it is). There are other areas in typical finite element codes where one uses this design principle of OO software. The ones that come to mind are the many kinds of quadrature formulas and the many kinds of finite elements, all of which are naturally represented as one interface/many implementations. Material law descriptions would also fall into this group. But it may be true that that's about it and that the rest of a finite element code doesn't use inheritance as pervasively as one may use it, say, in GUIs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From only these three points, it should be clear that object oriented programming is most definitely applicable to numerical codes as well, and that it would be foolish to ignore the many benefits of this style. It may be true that BLAS/LAPACK don't use this paradigm (and that the usual interface exposed by MATLAB doesn't either) but I would venture the guess that every successful numerical software written in the past 10 years is, in fact, object oriented.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-12T17:42:53.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2791" PostTypeId="2" ParentId="2777" CreationDate="2012-07-12T17:48:02.150" Score="1" Body="&lt;p&gt;If you split a matrix so that it is stored across processes in an MPI program (say, the matrix is $N\times N$, and we have $M$ processors), my understanding is that PETSc stores it in such a way that every processor stores a matrix that represents the diagonal block (size $N/M \times N/M$) and one that represents the other entries of the $N/M$ rows of the matrix that are stored on the local processor. It's done this way since every processor stores $N/M$ entries of a vector, and doing a matvec operation is simplest if you can just do a matvec on the diagonal block of the matrix with the locally stored part of the matrix, and then fix up the rest with the remaining matrix entries stored locally and the vector entries imported from the other processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this storage scheme, I believe that the diagonal and non-diagonal matrix entries you preallocate correspond to the diagonal $N/M \times N/M$ matrix and the $N/M \times N-N/M$ remainder block.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-12T17:48:02.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2792" PostTypeId="2" ParentId="2689" CreationDate="2012-07-12T20:16:32.537" Score="-1" Body="&lt;p&gt;I can't give you a discretization solution that &lt;em&gt;WILL&lt;/em&gt; work, but you might consider implicit in the space dimensions rather than explicit. Also from what I can tell you can likely use von neumann analysis to determine the steps sizes such that you will have a stable solution with the explicit space dimension steps.&lt;/p&gt;&#xA;" OwnerUserId="777" LastActivityDate="2012-07-12T20:16:32.537" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2793" PostTypeId="2" ParentId="2777" CreationDate="2012-07-12T20:16:58.373" Score="3" Body="&lt;h3&gt;Distributed vector spaces and matrix storage formats&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;PETSc vectors are distributed across the communicator with each process owning a contiguous block of rows. The most common case is for all the parts to be the same size, but that is not necessary. Any partition of the $N$ entries into $P$ contiguous parts (some possibly empty) with the parts labeled sequentially is valid. The &lt;code&gt;Mat&lt;/code&gt; class represents a linear transformation between two finite dimensional vector spaces, $A: X \to Y$. The spaces $X$ and $Y$ may be different (even different sizes), though Krylov methods and most linear solvers require that $X = Y$. There are routines for looking at the ownership range of the current process:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecGetOwnershipRange.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;VecGetOwnershipRange(x,&amp;amp;start,&amp;amp;end)&lt;/code&gt;&lt;/a&gt; returns range of owned entries for any vector $x$.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetOwnershipRangeColumn.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;MatGetOwnershipRangeColumn(A,&amp;amp;cstart,&amp;amp;cend)&lt;/code&gt;&lt;/a&gt; returns the owned part of the domain $X$.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;MatGetOwnershipRange(A,&amp;amp;rstart,&amp;amp;rend)&lt;/code&gt;&lt;/a&gt; returns the owned part of the range $Y$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The storage format used by PETSc &lt;code&gt;MPI*AIJ&lt;/code&gt; matrices distributes the entries according to ownership of the range space $Y$. In this model, there are generally some entries of the domain $X$ that affect the owned part of $Y$. To be precise, let $X_i \subset X$ be the part of the domain space $X$ that is owned by process (MPI rank) $i$ and similarly for $Y_i \subset Y$. The &quot;diagonal block&quot; of the matrix owned by rank $i$ is exactly the effect of $X_i$ on $Y_i$. When performing multiplication, the part of a vector in $X_i$ is available locally without any communication. The &quot;off-diagonal block&quot; represents the effect of $X \setminus X_i$ on $Y_i$, which cannot be applied without communicating entries. The operation $y \gets A x$ is computed as&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start communicating off-process entries of $x$ to $x_i^{\text{off}}$.&lt;/li&gt;&#xA;&lt;li&gt;Apply the diagonal block of the matrix $y_i \gets A_i^{\text{diag}} x_i$.&lt;/li&gt;&#xA;&lt;li&gt;Finish communicating off-process entries.&lt;/li&gt;&#xA;&lt;li&gt;Add the action of the off-diagonal block $y_i \gets y_i + A_i^{\text{off}} x_i^{\text{off}}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Geometrically, the off-diagonal part corresponds to the influence of the &quot;ghost points&quot; on the subdomain.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Preallocation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Since the diagonal part $A_i^{\text{diag}}$ is stored separately from $A_i^{\text{off}}$, we need separate preallocation information to avoid dynamic data structures. This can be done by providing one parameter indicating the maximum number of nonzeros in any row (&lt;code&gt;*_nz&lt;/code&gt;) or by providing an array indicating the number of nonzeros in each row (&lt;code&gt;*_nnz&lt;/code&gt;). Note that rows corresponding to &quot;interior&quot; points will generally have no nonzero entries in the off-diagonal part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for &lt;code&gt;ex2.c&lt;/code&gt;, there are indeed never more than 5 nonzeros per row of the matrix, of which at least one (the diagonal entry) must fall in the diagonal block. Therefore it would have been safe to pass &lt;code&gt;o_nz=4&lt;/code&gt;. Note that this extreme case &lt;code&gt;o_nz=4&lt;/code&gt; is only realized by this example for a single-point subdomain.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-12T20:16:58.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2794" PostTypeId="1" AcceptedAnswerId="2796" CreationDate="2012-07-13T09:46:12.683" Score="2" ViewCount="2368" Body="&lt;p&gt;As stated in the title, I encountered a proof with the final statement of the form &#xA;&lt;em&gt;&quot;the eigenvalues of A are then $\{\lambda_1+c, \lambda_2, \dots, \lambda_n \},$ counting algebraic multiplicity.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does &quot;Counting algebraic multiplicity&quot; mean in general? I'll then try to apply the &quot;general&quot; to my specific problem.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-07-13T11:22:15.757" Title="What does &quot;Counting algebraic multiplicity&quot; mean?" Tags="|linear-algebra|matrix|eigensystem|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2795" PostTypeId="2" ParentId="2794" CreationDate="2012-07-13T10:09:02.403" Score="2" Body="&lt;p&gt;How do you find eigenvalues of a linear map? There are two conceptual ways:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) You find the roots of the characteristic polynomial of you map. There you can have roots with higher multiplicity like in $(x-1)^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) You can identify eigenspaces and then derive the eigenvalues. Here eigenspaces can have higher dimensions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the algebraic multiplicity of an eigenvalue is the multiplicity of the respective root of the characteristic polynomial in case 1. The geometric multiplicity is the dimension of the respective eigenspace (and is always smaller than the algebraic multiplicity).&lt;/p&gt;&#xA;" OwnerUserId="36" LastActivityDate="2012-07-13T10:09:02.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2796" PostTypeId="2" ParentId="2794" CreationDate="2012-07-13T11:22:15.757" Score="2" Body="&lt;p&gt;It means that an eigenvalue appears in the list $m$ times if its algebraic multiplicity is $m$. (What the algebraic multiplicity is can be seen from the other comments/answers.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-13T11:22:15.757" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2797" PostTypeId="1" AcceptedAnswerId="2798" CreationDate="2012-07-13T11:35:26.030" Score="5" ViewCount="666" Body="&lt;p&gt;I want to solve the following expression (used to obtain an analytic solution to a current distribution inside a workpiece):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_{mn} = -\frac{\frac{4}{ab} \int_0^a \int_0^b f(x',y')\sin(px')\sin(qy')\mathrm{d}x'\mathrm{d}y'}{t\sinh(tc)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here. $a$,$b$ are scalar constants and $p = \frac{m\pi}{a}$, $q = \frac{n\pi}{b}$, $t=\sqrt{p^2+q^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $f$ is a Gaussian distribution:&#xA;$f(x,y) = \frac{I_0d}{\pi\sigma^2} \exp(-\frac{r^2d}{\sigma^2})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if using the Simpson's method is the smartest way to solve the double integral. Are there alternative solutions which would be more computationally efficient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional information:&#xA;The resulting matrix $a_{mn}$ is used in the following infinite series to obtain the final result:&#xA;$\sum_{m=1}^\infty \sum_{n=1}^\infty a_{mn}p\cos(px)\sin(qy)\cosh(tz)$&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="103" LastEditDate="2012-09-04T02:12:43.920" LastActivityDate="2012-09-04T02:12:43.920" Title="Methods to solve a double integral" Tags="|quadrature|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2798" PostTypeId="2" ParentId="2797" CreationDate="2012-07-13T12:02:34.363" Score="7" Body="&lt;p&gt;It depends on how big $m$ and $n$ are (assuming that $x'$ ranges from $0$ to $a$, though the notation suggests the opposite if read literally). You should probably split the intervals into at least $m$ resp $n$ subintervals, integrate the pieces separately, and sum the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $f$ is smooth, a Gaussian rule is probably more accurate for the same effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $m$ and $n$ are large, it is better to rewrite the integral in terms of Fourier transforms and solve it by FFT.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-07-13T14:36:19.127" LastActivityDate="2012-07-13T14:36:19.127" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2799" PostTypeId="1" AcceptedAnswerId="2813" CreationDate="2012-07-13T12:57:16.353" Score="4" ViewCount="211" Body="&lt;p&gt;Suppose I have a symmetric linear operator $A:\mathbb{R}^k \rightarrow \mathbb{R}^k$ where $k$ is &quot;small&quot; (eg., $k=100$), and I want to find it's first few eigenvectors, (eg., $10$ eigenvectors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we had a matrix representation of $A$ this would be a standard problem in dense numerical linear algebra. However, the catch is that we don't have the entries of $A$, only a code that can evaluate the action of $A$ on a vector. Furthermore, evaluating $Av$ is moderately costly - it involves solving a PDE behind the scenes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to find the first few eigenvectors of $A$ in this situation? My first thought was the Lanczos algorithm, but that's usually good for large sparse systems, whereas my system is small and dense.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2012-07-14T02:48:23.880" Title="Finding dominant eigenvectors of an operator that is small but costly to evaluate" Tags="|eigensystem|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2800" PostTypeId="1" AcceptedAnswerId="2845" CreationDate="2012-07-13T14:02:19.313" Score="7" ViewCount="955" Body="&lt;p&gt;Assume you have a stability constraint between the space distance in time and space, for example, with an explicit Euler method for $u_t=u_{xx}$ we know $\tau\leq h^2/2$. That is, one can do stability analysis for a uniform grid, obtain a constraint and then take the minimum over a nonuniform grid for the spacing to set a constraint for the spacing in time in the case of a non-uniform grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I use the same argument for &quot;unconditionally stable&quot; method on non-uniform grid? My goal is to obtain a stability in the case of a nonuniform grid. Is that sufficient to consider uniform grid for the method, such as Crank-Nicolson and somehow extend the argument to the non uniform grid? What are my options obtaining stability in this case?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="123" LastEditDate="2012-07-16T18:40:08.487" LastActivityDate="2012-07-17T12:28:31.787" Title="analyze stability on a nonuniform grid" Tags="|finite-difference|stability|parabolic-pde|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2801" PostTypeId="2" ParentId="2799" CreationDate="2012-07-13T14:41:38.450" Score="4" Body="&lt;p&gt;To find with reasonable accuracy the 10 dominant eigenvectors of a general 100 x 100 matrix you can hardly do better than to build the matrix using 100 matrix vector multiplies and then find its eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An exception is when you are confident that the rank is much smaller than $n$. In that case, Lanczsos with full reorthogonalization is the way to go.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-13T14:41:38.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2802" PostTypeId="1" AcceptedAnswerId="2893" CreationDate="2012-07-13T15:40:30.590" Score="6" ViewCount="622" Body="&lt;p&gt;This question is not very deep. Suppose I have a small rectangular matrix $A$, with number of rows and columns between $50$-$100$, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a right-hand side $b$, I want to solve the least-squares problem $Ax = b$, i.e., I want to find $x$ with norm-minimal residual $r = b - Ax$ and $x$ having minimum norm among these minimizers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the system is very small, and $A$ has a good generalized condition number, I consider solving the normal equations $A^t A x = A^t b$ instead, say, by Cholesky decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this stage, efficiency is less important than getting accurate solutions. Would you argue, from your experience, that this is a good choice, rather than, say, computing the SVD?&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="127" LastEditDate="2013-05-14T17:26:35.943" LastActivityDate="2013-05-14T17:26:35.943" Title="How to solve a small least-squares problem" Tags="|linear-algebra|least-squares|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2803" PostTypeId="1" CreationDate="2012-07-13T12:44:17.260" Score="3" ViewCount="3184" Body="&lt;p&gt;I want to invert a 10X10 antisymmetric matrix in Python around 10,000 - 20,000 times. Is there a faster way to do it other than to use the built-in inverse function in Python?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerDisplayName="tut_einstein" LastEditorUserId="276" LastEditDate="2012-07-14T10:04:33.880" LastActivityDate="2012-07-14T10:43:12.177" Title="Inverse of a 10X10 antisymmetric matrix" Tags="|linear-algebra|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2804" PostTypeId="2" ParentId="2803" CreationDate="2012-07-13T13:04:50.823" Score="1" Body="&lt;p&gt;You should really be asking in the Maths or IT Stack Exchanges, however if you want a layman's view:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Python's matrix inversion is likely to be written in a high level language and compiled so it will be a lot faster than interpreted Python code&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;as far as I know there aren't any cunning short-cut algorithms specifically for anti-symmetric matrices&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So I would use the Python implementation rather than writing your own (in Python).&lt;/p&gt;&#xA;" OwnerUserId="33714" OwnerDisplayName="John Rennie" LastActivityDate="2012-07-13T13:04:50.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2805" PostTypeId="2" ParentId="2803" CreationDate="2012-07-13T13:42:56.593" Score="2" Body="&lt;p&gt;Yes, this is the wrong place to post the question, but can't resist answering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python doesn't have a built-in matrix inverse. Numpy does. Numpy's algorithm is written in a low-level language, and written by matrix-inversion experts, so it's about as fast as possible. Unless you are a matrix-inversion expert yourself, you cannot write one that is faster.&lt;/p&gt;&#xA;" OwnerUserId="1761" OwnerDisplayName="Steve B" LastActivityDate="2012-07-13T13:42:56.593" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2807" PostTypeId="2" ParentId="2802" CreationDate="2012-07-13T16:15:21.067" Score="3" Body="&lt;p&gt;Sure, go with inverting $A^TA$. That matrix is symmetric and positive definite, so you can do that with CG as well. Alternatively, you can solve the original problem by computing a decomposition of the original, rectangular matrix, using an algorithm like QR.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-13T16:15:21.067" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2808" PostTypeId="2" ParentId="2802" CreationDate="2012-07-13T16:18:26.000" Score="4" Body="&lt;p&gt;I'd guess a &lt;a href=&quot;http://en.wikipedia.org/wiki/QR_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;QR decomposition&lt;/a&gt; is better than solving the normal equations and faster than SVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some &lt;a href=&quot;http://classes.soe.ucsc.edu/cmps290c/Spring04/paps/lls.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;class notes&lt;/a&gt; that compare the three approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Also:&lt;/strong&gt; The QR decomposition for least-squares solution of overdetermined linear systems has been &lt;a href=&quot;https://scicomp.stackexchange.com/q/1969/651&quot;&gt;previously discussed here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-07-13T16:33:34.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2809" PostTypeId="2" ParentId="2797" CreationDate="2012-07-13T16:20:50.140" Score="2" Body="&lt;p&gt;If you solve this only for a single pair $m,n$, then high order Gaussian integration is likely the fastest way to do this. But if you want to build the elements of a matrix $a_{mn}$ for many different values $m,n$, then it is best to recognize that what you're computing here is a double Fourier transform, and it would likely be best to compute the double integral exploiting this fact using, for example, the FFT.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-13T16:20:50.140" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2812" PostTypeId="2" ParentId="1528" CreationDate="2012-07-13T22:23:35.300" Score="8" Body="&lt;p&gt;If $A\in\mathbb{R}^{N\times M}$ with $N&amp;lt;M$, then &#xA;$$&#xA;\mathop{\mathrm{rank}}(A^TA) = \mathop{\mathrm{rank}}(AA^T) = \mathop{\mathrm{rank}}(A) \leq N &amp;lt; M&#xA;$$&#xA;so that $A^TA \in \mathbb{R}^{M\times M}$ &lt;em&gt;cannot&lt;/em&gt; be full rank, i.e. it is &lt;em&gt;singular.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accordingly the condition number is $\kappa_2(A^TA)=\infty$. Due to finite precision arithmetic, if you compute &lt;code&gt;cond(A'A)&lt;/code&gt; in matlab you obtain a large number, not &lt;code&gt;Inf&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-07-14T19:22:20.160" LastActivityDate="2012-07-14T19:22:20.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2813" PostTypeId="2" ParentId="2799" CreationDate="2012-07-14T02:48:23.880" Score="6" Body="&lt;p&gt;Is the matrix also positive, such that eigenvalues are equal to singular values? You can estimate $k$ singular values of a general matrix (possibly nonsquare) with several digits of accuracy provided they are at least somewhat isolated in $8k$ matrix multiplies. The algorithm, as described in the excellent survey by &lt;a href=&quot;http://dx.doi.org/10.1137/090771806&quot;&gt;Halko, Martinsson, and Tropp (2012)&lt;/a&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;choose a random $n\times 2k$ matrix $X$&lt;/li&gt;&#xA;&lt;li&gt;compute $Y = (A^* A)^q A X$ where $q=1$ is typically sufficient&lt;/li&gt;&#xA;&lt;li&gt;orthogonalize $Y = QR$ with $Q$ orthogonal&lt;/li&gt;&#xA;&lt;li&gt;compute $B = Q^* A$&lt;/li&gt;&#xA;&lt;li&gt;compute an SVD of the small matrix: $\tilde U \Sigma V^* = B$&lt;/li&gt;&#xA;&lt;li&gt;If you want the left singular vectors, let $U = Q \tilde U$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The first $k$ singular values and singular vectors typically provide several digits of accuracy. This algorithm is often more accurate than Lanczos or Arnoldi iteration and has fewer synchronization points. The advantage is more significant for large problems and if applying the operator to several vectors at once is significantly cheaper than applying it to one at a time. This could be a good choice if you only want a couple values.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-14T02:48:23.880" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2814" PostTypeId="1" AcceptedAnswerId="2819" CreationDate="2012-07-14T06:20:49.573" Score="11" ViewCount="2046" Body="&lt;p&gt;I'm &lt;em&gt;very&lt;/em&gt; new to GPGPU programming so please forgive me if the question is not particularly appropriate. From what I understand GPU programming is a very intricate piece of engineering work when compared to usual CPU programming. One has to be very careful about divergence issues, tiling, pinned memory allocation, and host-device communication/device computation overlapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After doing a little bit of research I found the &lt;a href=&quot;https://code.google.com/p/thrust/&quot; rel=&quot;nofollow noreferrer&quot;&gt;thrust library&lt;/a&gt; which seems to try to mimic C++ STL. This is quite nice. However, based on my very limited experience and having seen all the micro-managing required to get good performance, I'm a little bit skeptical about the performance. Can thrust efficiently handle all the intricate programming part internally? Some very well-known libraries, such as PETSc, seem to use this package which makes me believe it somehow should.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if people with more experience on CUDA and thrust could say a word or two about the performance of the package when compared to the low level CUDA programming. When can I use thrust and when should I switch back to CUDA? &lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="24110" LastEditDate="2017-04-30T20:08:43.683" LastActivityDate="2017-04-30T20:08:43.683" Title="Thrust for GPU programming" Tags="|parallel-computing|petsc|gpu|cuda|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2815" PostTypeId="2" ParentId="2775" CreationDate="2012-07-14T09:42:12.940" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I guarantee that the convergence is uniform &quot;everywhere&quot; on the grid? to confirm the theory results, should not I measure the error at each point and see I have a linear convergence in order to claim that the error goes to zero linearly in a discrete norm defined above?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.  Do that.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There are a number of papers, where the convergence is measured for a particular point on the grid and the results are stated to confirm the theoretical estimates that are done in discrete L∞ or L2 for the error vector. Is this sufficient to measure only at one point to obtain the confirmation &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;and why the results presented only for the error at one point?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is no way we can tell you that without a link to the paper(s) you're referring to.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-07-15T05:48:02.843" LastActivityDate="2012-07-15T05:48:02.843" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2816" PostTypeId="2" ParentId="2803" CreationDate="2012-07-14T10:43:12.177" Score="3" Body="&lt;p&gt;I picked this trick up from Jack Poulson when he answered &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1234/matrix-exponential-of-a-skew-hermitian-matrix-with-fortran-95-and-lapack&quot;&gt;this related question on antisymmetric (or skew-symmetric) matrix exponentials&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An antisymmetric (more commonly called skew-symmetric matrix) $A$ is one in which $A^{T} = -A$. Since the matrix wasn't called skew-Hermitian, I'm assuming that the matrix is real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conveniently, $(iA)^{H} = -iA^{H}$, where $H$ denotes the Hermitian transpose, so you could compute $iA$, and invert it using the LAPACK routine ZHESV (or CHESV; unless it is also positive definite, in which case you could use ZPOSV or CPOSV). At this point, you have $(iA)^{-1} = -iA^{-1} = B$. It follows that $iB = A^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, NumPy and SciPy don't implement those functions (you'd have to call them from another language, like Fortran, C, C++, Java, etc.; there could be other libraries that provide a Python interface to LAPACK, but I don't know any that implement all of it). Based on the module &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/linalg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scipy.linalg&lt;/code&gt;&lt;/a&gt;, your best option is probably to call &lt;code&gt;scipy.linalg.solve&lt;/code&gt; (and ignore any symmetry), or if $iA$ is Hermitian positive definite, call &lt;code&gt;scipy.linalg.solveh_banded&lt;/code&gt; after rearranging your data appropriately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on looking at the source, at a high level, it doesn't matter whether you call &lt;code&gt;scipy.linalg.solve&lt;/code&gt; or &lt;code&gt;scipy.linalg.inv&lt;/code&gt;; both are ultimately LAPACK calls. If any conceivable speed difference matters, you may as well test the two, but before you do so, you're probably better off making sure you use a high-quality BLAS implementation (ATLAS, Intel MKL (if appropriate), GotoBLAS) and making sure you build LAPACK, NumPy, and SciPy accordingly. Also, if there's any task parallelism with these matrices, you could potentially exploit that as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this assumes that you want to invert 10,000 - 20,000 different matrices. I presume that if you wanted to invert the same matrix that many times, you know to just calculate the LU decomposition once, and use it to solve a linear system with 10,000 - 20,000 different right hand sides (each with the same coefficient matrix), in which case, the appropriate functions are marked in &lt;code&gt;scipy.linalg&lt;/code&gt;. I can add further details if need be.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-07-14T10:43:12.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2818" PostTypeId="1" AcceptedAnswerId="2831" CreationDate="2012-07-14T13:47:42.483" Score="1" ViewCount="312" Body="&lt;p&gt;I have been reading a book by Thomee and he considers the case of $u_t=(au_x)_x$, for the case of $a$ possibly being discontinuous. Then he says that the problems with convergence might occur, and thus to fix that one can smooth the coefficient $a$ and restore convergence rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please correct me if I am wrong but discontinuous coefficients affect the smoothness of the solution of a pde at the first place, and therefore affects the local truncation error of a numerical method. How does smoothing of the coefficient affects convergence if the convergence depends on local truncation error, which depends on the properties of the function itself and its derivatives by the original stated problem which we can't control, because it is an input to the problem? &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="123" LastEditDate="2012-07-14T18:29:14.720" LastActivityDate="2012-07-16T01:28:01.357" Title="Smoothing the diffusion coefficient to improve convergence" Tags="|finite-difference|parabolic-pde|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2819" PostTypeId="2" ParentId="2814" CreationDate="2012-07-14T17:08:29.980" Score="2" Body="&lt;p&gt;I don't have personal experienced with thrust, but I do use ViennaCL, which is another high level GPU library that hides almost all of the details.  From my own personal benchmarking I can see speed-ups of 2x - 40x on the actual computation if you ignore the time it take to move around memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you should use the CPU vs. thrust vs. CUDA all depends on the problem you are solving, your skill, and the time you have available.  I would recommend starting by solving simple problems with all 3 methods to see their relative performance.  Then you can write your actual software in a quick manner, benchmark it, and apply the appropriate gpu method in the areas that &lt;strong&gt;need&lt;/strong&gt; the speed-up, rather than wasting your time writing CUDA software that will only gain you a couple minutes of execution time.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-07-14T17:08:29.980" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2820" PostTypeId="1" AcceptedAnswerId="2821" CreationDate="2012-07-14T20:34:27.270" Score="1" ViewCount="100" Body="&lt;p&gt;I am trying to solve complex valued Poisson equation &#xA;$$(C + \nabla. D \nabla )u = f \text{  ;where C, D, u and f are complex numbers.} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am breaking this eqn into real valued problem, which is of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  \underbrace{\begin{pmatrix} C_r + \nabla. D_r \nabla &amp;amp;  -C_i - \nabla. D_i \nabla \\ C_i + \nabla. D_i \nabla &amp;amp;  C_r + \nabla. D_r \nabla \end{pmatrix} }_{A}  \begin{pmatrix} u_r \\ u_i \end{pmatrix} =  \begin{pmatrix} f_r \\ f_i \end{pmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where 'r' and 'i' denote the real and imaginary parts. I wrote a Preconditioner (PC) which has the same form as the above equation by defining MAT &lt;code&gt;A&lt;/code&gt; and Vec. I verified the PC was working fine by solving the above equation for an example.&#xA;I have also defined a shell matrix operator  which computes the action of &lt;code&gt;A&lt;/code&gt; operator on the vector and registered the shell matrix and PC with ksp. When I try to solve &#xA;equations using shell matrix and PC I am getting wrong solution. I am using Dirichlet boundary conditions(bcs) for $u_r$ and $u_i$ and homogeneous bcs for residual equation passed to PC. As a sanity check I did &lt;code&gt;-ksp_type preonly&lt;/code&gt; with Dirichlet bcs for PC and I do get a correct solution.  The ksp for both PC and Shell matrix is &lt;code&gt;gmres&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on what could be going wrong?&lt;/p&gt;&#xA;" OwnerUserId="1763" LastEditorUserId="1763" LastEditDate="2012-07-15T16:54:40.310" LastActivityDate="2012-07-15T16:54:40.310" Title="Debugging Shell matrix" Tags="|linear-algebra|petsc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2821" PostTypeId="2" ParentId="2820" CreationDate="2012-07-15T01:06:46.533" Score="1" Body="&lt;p&gt;Run with &lt;code&gt;-ksp_monitor_true_residual&lt;/code&gt;. Does the residual go to zero? If so, then the definition of the matrix is incorrect (or the right hand side or your meaning of &quot;correct&quot;). If it does not converge, try a small well-conditioned case with &lt;code&gt;-pc_type none&lt;/code&gt; to rule out bugs in the preconditioner. If the problem is in the preconditioner, try &lt;code&gt;-ksp_type fgmres&lt;/code&gt; to see if the preconditioner is nonlinear (FGMRES converging when GMRES does not usually means the preconditioner is nonlinear).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: A couple more things to try:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Check that matrix-multiply is correct by setting the &quot;correct solution&quot; in $x$, then computing $b \gets A x$, then solving $A y = b$. If $x = y$, the matrix multiply is likely correct.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compare the true residual to the preconditioned residual in &lt;code&gt;-ksp_monitor_true_residual&lt;/code&gt;. Are they similar? Try the same with &lt;code&gt;-ksp_norm_type unpreconditioned&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-07-15T02:21:18.173" LastActivityDate="2012-07-15T02:21:18.173" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2822" PostTypeId="1" AcceptedAnswerId="2830" CreationDate="2012-07-15T12:25:50.513" Score="9" ViewCount="11635" Body="&lt;p&gt;Recently, I saw this question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2775/how-to-measure-the-error-of-a-finite-difference-method&quot;&gt;how to measure the error of a finite difference method&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am student of simulation sciences and unfortunately, for me, it's totally unclear, what norm to use in what context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quite often, we use the Euclidian norm or the L2 norm, but why does one choose different norms, what's their meaning besides the numerical / mathematical definition?&#xA;Or more precise:&#xA;What is the reason to use a specific norm in a specific context?&lt;/p&gt;&#xA;" OwnerUserId="1738" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-07-16T08:03:50.570" Title="What norm to choose when?" Tags="|error-estimation|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2823" PostTypeId="2" ParentId="2822" CreationDate="2012-07-15T12:51:05.497" Score="9" Body="&lt;p&gt;The Euclidean norm is often used based on the assumption that the Euclidean distance of two points is a reasonable measure of distance. But unless this is the case, this choice is not preferable to a problem-adapted choice. For example if the typical size of the components of a vector are very different (since they mean very different things), the Euclidean norm is very poor as it hardly takes into account the effects of changes in the small-size components.&#xA;In such a case, one either needs to first scale the vectors to have similar sized components before applying norms, or one must use a norm that scales different components differently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The norm $\|x\|$ of a vector $x$ (and similarly for matrices and functions) is a measure of its size; this measure must be adapted to the meaning of the problem you are solving. In finite dimensions, all norms are equivalent, in the sense that they describe the same topology; but the numerical values may depend quite a lot on the particular norm. (For the topology, the only thing of interest is the limit. In finite D this is independentent of the norm, i.e., $\|x_k−x\|\to 0$ in any norm if $\lim x_k=x$. But how close one is to the limit depends a lot on in which norm you measure it.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore one must choose a meaningful norm to get meaningful results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In infinite-dimensional spaces (which in particular includes the common function spaces), norms are no longer equivalent, and different norms may lead to different topologies. Now one must choose a suitable norm even to get finite results, and bounding terms may be impossible wwithout a good choice of the norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an exercise, I'd like to suggest that you compare the values of the $p$-norm for $p=1,2,\infty$ for a variety of vectors in $R^n$ parameterized by $n$, and do the same in various spaces of sequences $x=(x_1,x_2,\dots)$. You'll then appreciate the differences. A good example is the vector with $i$the entry $x_i=\epsilon/i^s$, where $s&amp;gt;0$. Here for tiny $\epsilon$ and large $n$ (approximate the sum by an integral) &#xA;$\|x\|_p\approx \epsilon\frac{1-1/n^{ps-1}}{ps-1}$, which becomes infinitely large as $n\to\infty$ when $p\le 1/s$ but remains tiny when $p&amp;gt;1/s$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-07-15T14:31:19.613" LastActivityDate="2012-07-15T14:31:19.613" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2824" PostTypeId="1" CreationDate="2012-07-15T15:40:09.030" Score="6" ViewCount="32049" Body="&lt;p&gt;The following MATLAB function produces the Eigenvalues and Eigenvectors of matrix X.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[V,D] = eig(X) produces a diagonal matrix D of eigenvalues and a&#xA;full matrix V whose columns are the corresponding eigenvectors so&#xA;that X*V = V*D.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does this mean that the first (or principal or dominant) eigenvector lay on the last column of V? NOTE: the author says that, all the coefficients of the dominant eigenvector are positive and that the remaining eigenvectors (the rest of columns) must have components that are negative, in order to be orthogonal (what does this mean) to u^(i);&lt;/li&gt;&#xA;&lt;li&gt;Regarding the &quot;corresponding eigenvecrtors&quot;, do we read them &quot;column-by-column&quot; OR &quot;row-by-row&quot;?&lt;/li&gt;&#xA;&lt;li&gt;Do eigenvalues-eigenvectors come in pairs? If yes, and considering the above, then does the corresponding eigenvalue lay on the bottom-right of matrix D?&lt;/li&gt;&#xA;&lt;li&gt;Actually, I want eigenvalues and their corresponding eigenectors in decreasing order, and then select the, 2 say, &quot;most significant&quot; ones. What I should do?&lt;/li&gt;&#xA;&lt;li&gt;Out of curiosity, but what does it mean &quot;the two times-series Fi and Fi' are uncorrelated in the sense that their empirical correlation vanishes for i != i' ??? How to check that in MATLAB?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;LOTS of questions, I know, but I would &lt;em&gt;REALLY&lt;/em&gt; appreciate if you could help me answer some of them!&lt;/p&gt;&#xA;" OwnerUserId="1829" OwnerDisplayName="eualin" LastActivityDate="2014-12-08T14:14:05.213" Title="Ordering of Eigenvalues and Eigenvectors in MATLAB" Tags="|matlab|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2825" PostTypeId="2" ParentId="2824" CreationDate="2012-07-15T15:52:27.703" Score="0" Body="&lt;p&gt;Given the formula, you should read the vectors column-by-column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value associated with the i-th vector is the i-th value on the diagonal of D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure the eigenvalues or egenvectors are ordered, and the &quot;order&quot; you are talking about doesn't seem clear to me...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I assume you want to order the vectors by decreasing eigenvalues, then simply switch the right columns in V and the right rows and columns in D.&lt;/p&gt;&#xA;" OwnerDisplayName="Flav" LastActivityDate="2012-07-15T15:52:27.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2826" PostTypeId="2" ParentId="2824" CreationDate="2012-07-15T15:56:26.883" Score="2" Body="&lt;p&gt;Actually each diagonal element &lt;code&gt;(i,i)&lt;/code&gt; of matrix &lt;code&gt;D&lt;/code&gt; (i.e. eigenvalue) corresponds to &lt;code&gt;i&lt;/code&gt;th column of matrix &lt;code&gt;V&lt;/code&gt;. That is the the higher value of &lt;code&gt;D(i,i)&lt;/code&gt; the more important the corresponding eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MatLab function &lt;code&gt;eig(X)&lt;/code&gt; sorts eigenvalues in the acsending order, so you need to take the last two colmns of matrix &lt;code&gt;V&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also do remember that if you try to perform factor analysis you can simply use MatLab's &lt;code&gt;princomp&lt;/code&gt; function or center the data before using &lt;code&gt;eig&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The signs of components for the first eigenvector is not defines anyhow (in case X is nonsingular and symmetric, e.g. correlation matrix as in factor analysis) &lt;/p&gt;&#xA;" OwnerUserId="1766" OwnerDisplayName="Paul" LastActivityDate="2012-07-15T15:56:26.883" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2827" PostTypeId="2" ParentId="2736" CreationDate="2012-07-15T20:57:40.780" Score="5" Body="&lt;p&gt;Elliptic problems have weak singularities at reentrant corners, therefore you &lt;em&gt;must&lt;/em&gt; interpret solutions weakly, making strong form discretizations like finite differences less natural than weak form discretizations (like finite element methods) to analyze and compute with. The canonical analysis text is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Grisvard (1985) &lt;em&gt;Elliptic problems in nonsmooth domains&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Solution of elliptic systems in domains with reentrant corners is the &lt;em&gt;canonical&lt;/em&gt; problem in adaptive finite element methods, thus the topic of thousands of papers. A good place to get started on the background theory is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Brenner and Scott (1994,2002,2008) &lt;em&gt;The mathematical theory of finite element methods&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are numerous open source software packages that use adaptive refinement guided by a posteriori error estimators to solve elliptic problems in domains with reentrant corners.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-15T20:57:40.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2828" PostTypeId="2" ParentId="2822" CreationDate="2012-07-15T21:29:05.210" Score="6" Body="&lt;p&gt;A few remarks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, which norm you choose depends on what you want to measure. It is that simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For numerical pde, the $L^2$-norm has the convenient property of providing a Hilbert space structure. A natural reason to use this norm comes from the treatment of measurement errors, as described in &lt;a href=&quot;https://scicomp.stackexchange.com/a/2763/238&quot;&gt;https://scicomp.stackexchange.com/a/2763/238&lt;/a&gt;. I do not know whether there are other reasons that lie beyond mathematical feasibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $L^\infty$-norm is used when you want to assert a maximum bound on the error &quot;pointwise&quot;. It is natural to represent the dual space of $L^\infty$ by functions with finite $L^1$-norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other $L^p$-norms are used in nonlinear PDE, and Sobolev norms are the simple generalization of $L^p$-spaces if you want to control a function and its generalized derivatives.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-07-15T21:29:05.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2829" PostTypeId="1" AcceptedAnswerId="3241" CreationDate="2012-07-15T22:14:17.670" Score="4" ViewCount="1103" Body="&lt;p&gt;I have been reading &lt;a href=&quot;http://pubs.rsc.org/en/Content/ArticleLanding/2012/FD/C1FD00086A&quot; rel=&quot;nofollow&quot;&gt;a recent paper&lt;/a&gt;.  In it, the authors performed molecular dynamics (MD) simulations of parallel-plate supercapacitors, in which liquid resides between the parallel-plate electrodes.  To simplify the situation, let us suppose that the liquid between the electrodes is argon liquid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system has a &quot;slab&quot; geometry, so the authors are only interested in variations of the liquid structure along the $z$ direction.  Thus, the authors compute the particle number densities averaged over $x$ and $y$: $\bar{n}_\alpha(z)$, where $\alpha$ is a solvent species. (That is, in my simplified example, $\alpha$ is argon -- an argon atom.)  $\bar{n} _\alpha(z)$ has dimensions of $\frac{\text{number}}{\text{length}^3}$ or simply $\text{length}^{-3}$, I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $xy$-plane is given by the inequalities $-x_0 &amp;lt; x &amp;lt; x_0$ and $-y_0 &amp;lt; y &amp;lt; y_0$.  The area $A_0$ of the $xy$-plane is thus given by $A_0 = 4x_0y_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the authors define the particle number density averaged over $x$ and $y$ as follows: $$\bar{n}_\alpha(z) = A_0^{-1} \int_{-x_0}^{x_0} \int_{-y_0}^{y_0} dx^\prime dy^\prime n_\alpha(x^\prime, y^\prime, z)$$ where $A_0 = 4x_0y_0$ and $n_\alpha(x, y, z)$ is the local number density of $\alpha$ at $(x, y, z)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, $\bar{n}_\alpha(z)$ is simply &lt;em&gt;proportional&lt;/em&gt; to $n_\alpha$ integrated over $x$ and $y$.  But, my question is, what is $n_\alpha(x, y, z)$?  How is $n_\alpha(x, y, z)$ determined in practice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the computer is concerned, the argon atoms are point particles; they are modeled as having zero volume (although they interact by Lennard-Jones interactions).  So how is it possible to define a number density?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we simply &quot;cut&quot; the &quot;slab&quot; in &quot;slices&quot; along $z$ and then assign the particles to these slices?  There might be 5 particles in the first $z$ slice, 10 in the second, 7 in the third, and so on.  If I then divide 5, 10, and 7 by the volume of the respective slice, then I have a sort of number density, with units of $\frac{\text{number}}{\text{length}^3}$ or simply $\text{length}^{-3}$.  But how do I now integrate this $n_\alpha(x^\prime, y^\prime, z)$ over $x$ and $y$?  Do I have to additionally perform binning in the $x$ and $y$ directions?&lt;/p&gt;&#xA;" OwnerUserId="1454" LastActivityDate="2012-09-07T20:30:21.823" Title="In molecular dynamics (MD) simulations, how is particle number density computed in practice?" Tags="|algorithms|computational-chemistry|statistics|molecular-dynamics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2830" PostTypeId="2" ParentId="2822" CreationDate="2012-07-15T23:04:07.667" Score="12" Body="&lt;p&gt;For measuring the error in the solution of PDE, it is quite natural to choose the norm of the space in which the solution lies. For example, for elliptic PDEs, the solution lies in $H^1$ and so it is natural to choose the $H^1$ norm to measure the error. This makes sense because, for example, the solution does &lt;em&gt;not&lt;/em&gt; lie in the space $W^{1,\infty}$ and so it does not make sense to compute the maximum error in the gradient simply because you can't measure this error if even the &lt;em&gt;exact&lt;/em&gt; solution has points where the gradient is not finite. In other words, it doesn't make sense to measure the error in the norm of a space $X$ (e.g., $X=W^{1,\infty}$) if the exact solution lies in $Y$ (e.g., $Y=H^1$) and $X\subset Y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, we frequently measure the error in the norm of a space $Z$ if $Z\supset Y$, e.g., when we measure the error in $L_2$. For these other norms, it is sometimes because of physical importance but equally often simply a matter of convenience. The $L_2$ norm sometimes has some physical meaning: for example, the integral of the square of the electric field $\int E(x)^2 \; dx$, i.e., the square of the $L_2$ norm, is the energy stored in the electric field; likewise, the square of the norm of the solution of the wave equation is the potential energy stored in the solution. At other times, it is just a conveniently chosen norm. For example measuring the $L_2$ norm of the error in the time-dependent heat equation is almost always the wrong choice since the physically relevant quantities (the total thermal energy, the amount of material) is in reality the $L_1$ norm of the solution; in this case, measuring the error in the $L_2$ norm has no meaning other than being convenient.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-15T23:04:07.667" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2831" PostTypeId="2" ParentId="2818" CreationDate="2012-07-15T23:08:10.873" Score="3" Body="&lt;p&gt;I don't know what exactly Thomee had in mind, but if you can't get solution at all for a discontinuous coefficient and a particular discretization scheme, you can smooth the coefficient &lt;em&gt;on a lengthscale proportional to the mesh size $h$&lt;/em&gt; and obtain a problem that's solvable. Then you make the smoothing distance go to zero as you refine the mesh and you obtain a sequence of numerical solutions that approximate a sequence of exact solutions of perturbed problems that converge to the solution of the original problem. That's a common trick.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-15T23:08:10.873" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2832" PostTypeId="1" AcceptedAnswerId="2841" CreationDate="2012-07-15T23:47:21.660" Score="12" ViewCount="452" Body="&lt;p&gt;The journal &lt;a href=&quot;http://toms.acm.org/&quot;&gt;Association for Computing Machinery Transactions on Mathematical Software (ACM TOMS)&lt;/a&gt; publishes many articles on numerical algorithms that include software implementations. According to their &lt;a href=&quot;http://toms.acm.org/AlgPolicy.html&quot;&gt;editorial policy&lt;/a&gt;, submission of an algorithm paper includes the source code for an implementation of the algorithm described in said paper. This source code is subject to the &lt;a href=&quot;http://www.acm.org/publications/policies/softwarecrnotice&quot;&gt;ACM Software Copyright and License Agreement&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I am interested in releasing software implementations of my own work, I'm concerned about the legal implications of this license. Specifically, how does this software license interact with common open source licenses (such as the GPLv3, BSD, MIT/X11, and Apache licenses)?&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-07-16T13:50:04.613" Title="For software submitted to ACM TOMS, how does the ACM software license agreement interact with other licenses?" Tags="|software|documentation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2833" PostTypeId="2" ParentId="2818" CreationDate="2012-07-16T01:28:01.357" Score="1" Body="&lt;p&gt;Strong form discretizations are extremely cumbersome for the case of discontinuous coefficients. It is better to use weak form methods that do not require differentiating the solution. Furthermore while the product $a u_x$ is continuous even for discontinuous coefficients, $u_x$ alone is not. Therefore even evaluating one derivative of the solution is something for a numerical method to be very careful with. The standard (robust) approaches are $H(\textrm{div})$ mixed finite elements, mimetic finite differences, and primal finite volume/difference methods with a choice of quadrature making them equivalent to certain mixed methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started my reply with the note above because a numerical discretization must be stable and conserve the correct properties for it to be worth trying to solve the resulting algebraic system. Make sure the spatial discretization is robust for the chosen purpose before proceeding further. Local conservation, for example, is critical for any problem exhibiting non-smoothness. Also confirm that your quadrature respects the non-smoothness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the coefficient has discontinuities, but when the jumps conform to the grid, you can use discrete spaces that have a discontinuous gradient at the interface, thus allowing accurate approximation without elaborate tricks. This is most natural in the finite element context, which partly explains its popularity for structural mechanics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case of the coefficient $a$ being rough at a subgrid scale, there are numerous homogenization techniques. For problems with scale separation in 1D, the simple approach of replacing the rough coefficient with its harmonic mean on each element is optimal. For multiple dimensions or for problems that do not have separated scales, more sophisticated techniques are needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all cases, simple smoothing of the coefficients degrades accuracy significantly, and you will have no hope of accurate large-scale solutions until the mesh resolves the fine scale structure and the scheme will generally not be convergent in the energy norm.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-16T01:28:01.357" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2834" PostTypeId="2" ParentId="2788" CreationDate="2012-07-16T01:50:45.500" Score="2" Body="&lt;p&gt;Here is an idea. How about developing $(I-R)^{-1}$ in a geometric series? It only converges if all the eigenvalues of R are smaller than 1. But if it does then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D(I-R)^{-1} = D(I+R+R^2+\ldots)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem would be reduced to (hopefully) not many sparse matrix multiplications. &lt;/p&gt;&#xA;" OwnerUserId="1751" LastActivityDate="2012-07-16T01:50:45.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2835" PostTypeId="2" ParentId="2824" CreationDate="2012-07-16T07:20:59.450" Score="4" Body="&lt;p&gt;At least in the older versions of Matlab, eig didn't sort the eigenvalues. (For complex eigenvalues there is not even a natural ordering.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus it is safer to pick the wanted eigenvalue indices $i$ by inspecting all eigenvalues, and get the corresponding eigenvectors as the columns $V(:,i)$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-16T07:20:59.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2837" PostTypeId="1" CreationDate="2012-07-16T08:17:35.233" Score="4" ViewCount="105" Body="&lt;p&gt;Assume the optimal value of a primal problem is bounded. Is the following statement true? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the primal problem is bounded, then its dual problem is bounded as well.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="917" LastEditorUserId="549" LastEditDate="2012-07-16T13:50:05.190" LastActivityDate="2012-07-16T17:10:27.197" Title="Linear programming boundedness" Tags="|linear-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2838" PostTypeId="2" ParentId="2837" CreationDate="2012-07-16T08:21:37.113" Score="4" Body="&lt;p&gt;No. The primal LP min $x+y$ subject to $x-y\ge 0$ has no bounded objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, one must &lt;em&gt;assume&lt;/em&gt; boundedness of the primal and dual problem as a hypothesis, and then gets the result that both problems are solvable and their values agree.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-16T08:21:37.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2839" PostTypeId="2" ParentId="2814" CreationDate="2012-07-16T10:23:47.970" Score="3" Body="&lt;p&gt;The purpose of thrust (as most template libraries) is to provide a high-level abstraction, while preserving good, or even excellent, performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest not to worry to much about performance, but to ask yourself if &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;your application can be described in terms of the algorithms implemented in thrust, and if&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;you like the possibility of writing &quot;generic&quot; parallel code, without the need of going into the gory details of finding an efficient mapping to the given hardware/software architecture.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you respond positively to both questions you should be able to implement your program with less effort with respect to a CUDA only implementation. Then you can profile your application and decide if it is worthwhile to try improve performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This said, I have to confess that I do not like &quot;generic&quot; programming, because I'm willing to learn something new, when I write a program. I would follow another route: write a prototype implementation in python+numpy+scipy, then add CUDA kernels for those 1%--2% of the code that really needs optimization and is suitable to be run on a GPU. Of course by doing so you need some sort of pre-science, since a wrong decision in the prototyping phase (e.g a data structure unsuited for CUDA kernels) may have terrible results on performance. Usually more iterations are needed for obtaining a good code and there is no assurance of doing better than thrust.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-07-16T10:54:10.643" LastActivityDate="2012-07-16T10:54:10.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2840" PostTypeId="1" CreationDate="2012-07-16T12:59:42.043" Score="0" ViewCount="574" Body="&lt;p&gt;Consider a min cost flow problem in a directed graph $G=(V, E)$ as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(*)      Min $\sum {c_{ij}f_{ij}}$        &lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t.:  $\sum_{j\in out(i)}{f_{ij}} - \sum_{j\in in(i)}{f_{ji}} =b(i)$ for each $i\in V$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_{ij} \geq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$in(i)$ and $out(i)$ are the set of arcs coming into node i and going out of node i, respectively. In the formulation, $f_{ij}$ is the amount of flow routing on arc $(ij)$ and $c_{ij}$ is the unit cost of routing flow on arc $(ij)$ . For each node i, b(i) is the node demand/supply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Claim:&lt;/em&gt;&lt;/strong&gt; If * is feasible and $c_{ij} \geq 0$ for all arcs $(i,j)\in E$, the optimum value is zero.&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="549" LastEditDate="2012-07-16T13:57:08.803" LastActivityDate="2012-07-16T15:37:57.020" Title="Min-cost flow problem" Tags="|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2841" PostTypeId="2" ParentId="2832" CreationDate="2012-07-16T13:50:04.613" Score="4" Body="&lt;p&gt;Normally the author of a work can license it under more than one of the open source licenses you mention (so called dual licensing).  However there seems to be an issue with doing so under the referenced ACM editorial policy, which states that you would be obliged to transfer copyright ownership on published &quot;algorithms&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Authors of copyrightable algorithms (or their employers) are required&#xA;  to transfer the copyright to ACM upon acceptance of the algorithm for&#xA;  publication, in accordance with ACM policy to own copyright on ACM&#xA;  published material.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The policy (and the ACM Software and Copyright and License Agreement) go on to state that it &quot;grants authors the right to reuse their material, and also grants liberal permission for the reuse of the associated software for noncommercial purposes.&quot;  This restriction to noncommercial purposes would then prevent the author (IMHO, IANAL) from relicensing under the open source licenses mentioned in your question, as these do not permit a distinction between commercial and noncommercial use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible resolution may be found in the detailed discussion of &lt;a href=&quot;http://www.acm.org/publications/policies/copyright_policy&quot; rel=&quot;nofollow&quot;&gt;ACM Copyright Policy&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In special cases where an author or author's employer must retain&#xA;  copyright, or when ACM does not wish to give its imprimatur to a&#xA;  particular work, ACM may accept a release from the owner that grants&#xA;  ACM the permission it needs to publish the work.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have no idea how often such exceptions might be granted for publication.  If I were trying to seek such an exception, I'd probably argue that I'd already licensed the software under one of those open source licenses, and that the ACM could instead have copyright in the derivative work created by publication of the article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did mention IANAL (I am not a lawyer)?&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2012-07-16T13:50:04.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2842" PostTypeId="2" ParentId="2840" CreationDate="2012-07-16T15:37:57.020" Score="1" Body="&lt;p&gt;Your claim is wrong in general. Take all $c_{ij}=1$ and define the $b(i)$ by in-degree minus out-degree at $i$. Take as graph one in which the $b(i)$ are not all zero. Then the problem is feasible (with all $f_{ij}$ taken to be 1 giving a feasible point). For an arbitrary feasible point, not all $f_{ij}$ can vanish, so the minimum must be strictly positive.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-16T15:37:57.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2844" PostTypeId="1" AcceptedAnswerId="2908" CreationDate="2012-07-16T20:56:41.000" Score="4" ViewCount="3024" Body="&lt;p&gt;I'm using &lt;a href=&quot;http://dx.doi.org/10.1023/A:1008115206617&quot; rel=&quot;nofollow&quot;&gt;Richard Hartley's rectification algorithm&lt;/a&gt; to rectify a pair of images before performing stereo disparity computation. The problem is I'm observing shearing in one of the rectified images and it's causing problems on disparity computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider these two input images (the points pairs were produced using SURF):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://dl.dropbox.com/u/4713941/input_pair_plus_surf.png&quot; alt=&quot;Input images plus the SURF features&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hartley's algorithm produced these two rectified images:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://dl.dropbox.com/u/4713941/rectified_hartley_alg.png&quot; alt=&quot;Rectified images&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The epipolar lines look fine. The following images shows some epipolar lines and a few point pairs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://dl.dropbox.com/u/4713941/rectified_epipolar_lines.png&quot; alt=&quot;Rectified epipolar lines&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rectified SURF keypoints look fine too, consider the following small sample for inspection, &lt;em&gt;d&lt;/em&gt; is disparity and &lt;em&gt;erro&lt;/em&gt; is the difference between the &lt;em&gt;y&lt;/em&gt; coordinate (zero for a perfect mapping):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(306.28, 139.00) &amp;lt;-&amp;gt; (284.15, 138.48): d = -22.13, erro = -0.52&#xA;(259.84, 150.72) &amp;lt;-&amp;gt; (234.34, 150.51): d = -25.50, erro = -0.21&#xA;(423.93, 151.01) &amp;lt;-&amp;gt; (425.24, 150.71): d = 1.30, erro = -0.30&#xA;(220.98, 151.05) &amp;lt;-&amp;gt; (190.53, 151.05): d = -30.45, erro = -0.00&#xA;(354.21, 157.88) &amp;lt;-&amp;gt; (346.19, 157.91): d = -8.02, erro = 0.04&#xA;(304.17, 161.58) &amp;lt;-&amp;gt; (289.66, 161.80): d = -14.51, erro = 0.22&#xA;(229.47, 162.44) &amp;lt;-&amp;gt; (203.86, 162.27): d = -25.61, erro = -0.17&#xA;(406.54, 262.40) &amp;lt;-&amp;gt; (442.38, 262.91): d = 35.84, erro = 0.50&#xA;(361.67, 290.02) &amp;lt;-&amp;gt; (399.54, 289.98): d = 37.87, erro = -0.04&#xA;(356.44, 293.49) &amp;lt;-&amp;gt; (394.51, 292.96): d = 38.07, erro = -0.53&#xA;(340.01, 339.44) &amp;lt;-&amp;gt; (318.47, 339.75): d = -21.54, erro = 0.31&#xA;(245.47, 360.89) &amp;lt;-&amp;gt; (204.93, 360.18): d = -40.55, erro = -0.71&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, the problem. The following image shows the two rectified images &lt;em&gt;overlaped&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://dl.dropbox.com/u/4713941/overlaping_rectified_images.png&quot; alt=&quot;image description&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;ZOTAC&lt;/em&gt; word printed in the box is a good example. The word is in the same plane and, ideally, should present similar disparity. But the observed shearing will produce small disparities for &quot;Z pixels&quot; compared to disparities for &quot;C pixels&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm computed the rectification using two different implementations of the algorithm: the OpenCV implementation, &lt;code&gt;stereoRectifyUncalibrated&lt;/code&gt;, and an implementation coded by myself, from scratch, using Python and NumPy (following section 11.12 in Hartley and Zisserman's book). Both implementations got the same results. What is the matter with Hartley's algorithm? Can stereo algorithms handle this problem? Or have I made some mistake?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[This question was asked at &lt;a href=&quot;http://answers.opencv.org/question/418/heavy-shearing-effects-using-hartleys/&quot; rel=&quot;nofollow&quot;&gt;OpenCV Q&amp;amp;A&lt;/a&gt;. Because, apparently, it is not an OpenCV issue but an algorithm issue, it is being asked here as an general computer vision question.]&lt;/p&gt;&#xA;" OwnerUserId="599" LastActivityDate="2012-07-23T16:30:43.827" Title="Shearing and Hartley's rectification" Tags="|computer-vision|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2845" PostTypeId="2" ParentId="2800" CreationDate="2012-07-16T23:22:18.017" Score="8" Body="&lt;p&gt;It all boils down to finding the largest or smallest eigenvalue of the stiffness matrix. You can do that analytically for a uniform mesh, and you then arrive at the traditional formula $\tau \le ch^2$. For non-uniform meshes, the relationship between mesh size and eigenvalue is not obvious any more; in particular, there is no analytic formula for the eigenvalue any more, and there is of course no longer a single mesh size $h$ either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, I'm not familiar with an approach to come up with a definite bound on the time step that makes the scheme stable in the case of non-uniform meshes. That said, the formula everyone uses is to replace $\tau \le ch^2$ by the generalization $\tau \le c \min_K h_K^2$ where $h_K$ is the mesh size of cell $K$ and you simply take the minimum over all mesh sizes. That appears to be working well for practical cases. As I said, I don't know if there's a formal proof that this works -- my suspicion is that there is no such proof for general 2d/3d meshes but I'm willing to put down a conjecture that the statement is nevertheless true.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-16T23:22:18.017" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2846" PostTypeId="1" AcceptedAnswerId="2850" CreationDate="2012-07-17T05:22:21.083" Score="6" ViewCount="1243" Body="&lt;p&gt;I have a MPI computation with the following structure: each processor has a large region of read-only memory divided into chunks.  During a compute epoch, each processor performs a (different) number of steps of the form:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Gather several chunks of data different processors.&lt;/li&gt;&#xA;&lt;li&gt;Do some computation and store the results.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Different steps can take wildly different amounts of time between different processors, although the total load is well balanced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This structure seems &lt;em&gt;almost&lt;/em&gt; perfect for one-sided communication using MPI_Get.  However, I do not see how to fit this structure into any of the available synchronization models, at least using a single window.  Here are the options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;MPI_Win_fence: doesn't work since different processors execute different numbers of steps.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MPI_Win_start/complete/post/wait: doesn't work since it requires a 1-1 match between access epochs on origin processes and exposure epochs on target processes.  In other words, the origins need to synchronize multiple times incoherently between processes, so the targets can't choose a single consistent set of exposure epochs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MPI_Win_lock/unlock: doesn't work since you can't only target one rank at a time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I also don't see any solution using multiple windows, since I believe windows aren't allowed to overlap, and here multiple origins are frequently accessing the same chunks of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this situation is easy to handle using MPI_Isend, MPI_Irecv, and MPI_Wait_all, as long as the target processes poll or have a communication thread that responds to requests.  When a process wants to gather data, it sends off a bunch of MPI_Isend requests, posts a corresponding number of MPI_Irecv's to accept the data, and does an MPI_Wait_all until the set of data is gathered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way around to make one-sided communication work here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: This seems related: &lt;a href=&quot;http://www.cs.berkeley.edu/~bonachea/upc/mpi2.html&quot;&gt;http://www.cs.berkeley.edu/~bonachea/upc/mpi2.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2012-07-17T06:23:27.747" LastActivityDate="2012-07-23T00:02:31.357" Title="Simulating the MPI_Isend/Irecv/Wait model with one-sided communication" Tags="|mpi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2847" PostTypeId="1" AcceptedAnswerId="2858" CreationDate="2012-07-17T05:40:19.127" Score="4" ViewCount="164" Body="&lt;p&gt;The comments on the accepted answer of my previous question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2755/efficiency-of-a-dynamic-mesh-vs-a-static-mesh-for-a-propagating-shockwave&quot;&gt;here&lt;/a&gt; have left me with a more general question about accurately capturing shockwaves in fluid calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of having an example, let's say we have a standing shockwave with a pressure or density ratio of 10:1 measured at two points one unit length apart.  Let's also assume that I am attempting to capture it with 1D fluid calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, is the shockwave a true singularity, where infinite mesh refinement at the location of the shockwave always leading to one cell having 10 times the flow variable value of an adjacent cell?  Or is it the case that the gradients of flow variables may be several orders of magnitude larger within the area of the shockwave than anywhere else, but are still continuous?  Does the answer to the above question depend on the source of the shockwave?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And as a tangent question, assuming that the second case is true, over the one unit length thickness of the shockwave, what would be a good rule of thumb for an initial mesh density?  100 cell?  1000?&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-07-17T20:14:16.317" Title="Properties of a Shockwave in Fluid Calculations" Tags="|fluid-dynamics|mesh|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2849" PostTypeId="2" ParentId="2800" CreationDate="2012-07-17T07:37:24.723" Score="4" Body="&lt;p&gt;Your question may have multiple answers, depending on context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will give a simple one, which completes and clarify the previous answer by Wolfang Bangerth, so I'm not claiming the bounty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within a FEM approach you may in a first step discretize only the spatial domain:&#xA;\begin{equation}&#xA;u^h(x,t) = \sum_i \eta_i(x) U_i(t)&#xA;\end{equation}&#xA;so that&#xA;\begin{equation}&#xA;\frac{\partial u(x,t)}{\partial t} \approx &#xA;\frac{\partial u^h(x,t)}{\partial t} = \sum_i \eta_i(x) \dot U_i(t)&#xA;\end{equation}&#xA;where $\dot U_i$ are the time derivatives of the nodal unknowns. Applying the classical FEM method we end up, for linear problems, with an ODE's system like&#xA;\begin{equation}&#xA;A \dot U + K U = b&#xA;\end{equation}&#xA;Now we integrate with respect to time, forgetting the original PDE and the underlying mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this simple approach, stability with respect to time increment is defined at the ODE level, so that in the case of unconditionally stable methods we are not concerned about mesh. (Or at least, very bad meshes give raise to badly conditioned $K$ matrices, but this is another problem.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of explicit time integration, stability is linked to a generalized eigenvalue problem involving $A$ and $B$. Here for uniform meshes we have simple relations between $h$ and the stability limit, for non uniform meshes only estimates and bounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some experience in the field of non-linear continuum mechanics, where the ODE to be solved is&#xA;\begin{equation}&#xA;M(t)\ddot U + f_i(U, \dot U, \ldots \text{state variables}) = f_e(t)&#xA;\end{equation} &#xA;Here the stable time increment at time $t_0$ is linked to highest natural frequency $\omega_\text{max}$ (eigenvalue) of the linearized problem&#xA;\begin{equation}&#xA;M(t_0) \ddot U + K_{T} U = 0, \qquad U = U_\alpha \sin(\omega_\alpha t) &#xA;\end{equation}&#xA;where $K_T$ is the tangent stiffness matrix. Finding a good (but fast to compute) estimator of $\omega_\text{max}$ is crucial for efficiency and far from trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This treatment (which is the most common in commercial non linear continuum mechanics FEM codes) does not directly addresses the problem of convergence in the time-space domain, since it relies on the simplistic assumption that convergence for the stationary problem &#xA;($t = t_0 = \text{constant}$) and a good time integrator solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the time-space convergence problem has been addressed, but as said above, this is only a partial answer to the question above&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2012-07-17T07:37:24.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2850" PostTypeId="2" ParentId="2846" CreationDate="2012-07-17T11:55:30.520" Score="4" Body="&lt;p&gt;Indeed, passive target RMA is over-synchronizing in MPI-2 because only one &lt;code&gt;MPI_Win_lock&lt;/code&gt; can be active at a given time, thus only one target can be accessed in an access epoch. The MPI-3 draft standard adds&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int MPI_Win_lock_all(int assert, MPI_Win win);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which does what you want. There are many new RMA routines and generally more permissive semantics for MPI-3. A particular routine that might be useful is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int MPI_Rget(void *origin_addr, int origin_count,&#xA;             MPI_Datatype origin_datatype, int target_rank,&#xA;             MPI_Aint target_disp, int target_count,&#xA;             MPI_Datatype target_datatype, MPI_Win win,&#xA;             MPI_Request *request);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;MPI_Request&lt;/code&gt; allows completion without a new access epoch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can get the targets to participate, then you could implement everything over point-to-point with all possible target ranks calling &lt;code&gt;MPI_Iprobe&lt;/code&gt; periodically and servicing any requests. Since this does not use RMA, it will be higher latency on some systems, but completion ordering is more flexible.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-17T11:55:30.520" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2851" PostTypeId="2" ParentId="39" CreationDate="2012-07-17T12:28:24.423" Score="2" Body="&lt;p&gt;''I don't know how to choose a polynomial basis that is well conditioned with respect to arbitrary sets of points in the complex plane.'' &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the set of points is bounded, a good basis of polynomials of degree $d$ to use is the Lagrange polynomials of $d+1$ reasonably spaced point along an enclosing contour. This will give you far better results than a multiprecision solution of a problem involving a Vandermonde matrix.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-17T12:28:24.423" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2852" PostTypeId="2" ParentId="2800" CreationDate="2012-07-17T12:28:31.787" Score="3" Body="&lt;p&gt;Let's do the analysis by discretizing in space first, then in time (method of lines). First, we take some sequence of spatial discretizations $L^i, i=\{1,2,\dotsc\}$ approximating the continuous operator $L:u \to u_{xx}$. For the sequence of discretizations $L^i$ to be a convergent discretization, we need that the sequence be consistent and stable. Since the continuum operator $L$ has all negative eigenvalues, this implies that all eigenvalues of each discrete operator $L^i$ will be in the left half plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we turn our attention to the temporal discretization of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t^i = L^i u^i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where all eigenvalues of $L^i$ are in the left half plane. But &lt;a href=&quot;http://en.wikipedia.org/wiki/A-stable#A-stability&quot; rel=&quot;nofollow&quot;&gt;$A$-stable&lt;/a&gt; methods are &lt;em&gt;defined&lt;/em&gt; by exactly this property. ($A$-stability is just a more precise/less misleading term for unconditional stability. There are many other kinds of stability, e.g. $L$-stability and forms of nonlinear stability.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that we have not used any property of grid spacing, time step size, or even used the properties of the original PDE except that it must be stable.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-17T12:28:31.787" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2853" PostTypeId="1" AcceptedAnswerId="2857" CreationDate="2012-07-17T15:54:58.153" Score="18" ViewCount="8296" Body="&lt;p&gt;I was wondering whether there is a fast and efficient method to find the number of non zeros in advance for sparse matrix multiplication operation assuming both matrices are in CSC or CSR format. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there is one in smmp package but I need something that is already implemented in C or C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated.&#xA;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2012-07-17T19:43:42.250" LastActivityDate="2012-12-18T01:16:15.947" Title="What is the best way to determine the number of non zeros in sparse matrix multiplication?" Tags="|matrix|sparse-matrix|" AnswerCount="5" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2854" PostTypeId="2" ParentId="2853" CreationDate="2012-07-17T16:22:44.970" Score="0" Body="&lt;p&gt;For CSR or CSC, are you guaranteed that your array of matrix elements already has no zeros? In that case it is simple to figure out how many non-zero elements there are, using something similar to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int nnz = sizeof(My_Array)/sizeof(long int);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However if this is not the case (seems a bit too easy) what you could try is a &lt;a href=&quot;http://www.drdobbs.com/architecture-and-design/parallel-pattern-7-reduce/222000718&quot; rel=&quot;nofollow&quot;&gt;reduction&lt;/a&gt;. If your array of matrix elements is very large, this may be the most efficient way to compute the number of nonzero elements. Many parallel C/C++ libraries such as Thrust (a CUDA library) or OpenCL (which you don't need a GPU to use) have support for conditional reductions - for each element, add the result of &lt;code&gt;Condition(Element)&lt;/code&gt;. If you set the condition to &lt;code&gt;Element != 0&lt;/code&gt; then you'll add up the number of nonzero elements. You may also want to remove the zero valued elements from your array of elements, array of row/column indices, and adjust your column/row pointers. &lt;/p&gt;&#xA;" OwnerUserId="1193" LastActivityDate="2012-07-17T16:22:44.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2855" PostTypeId="2" ParentId="2814" CreationDate="2012-07-17T16:33:03.463" Score="5" Body="&lt;p&gt;I have used Thrust in my linked cluster expansion project. Depending on the situation, Thrust can perform as well as or better than a low level implementation that you roll yourself (in particular, the &lt;code&gt;reduce&lt;/code&gt; kernel has been working quite well for me). However Thrust's generic nature and flexibility means it sometimes has to do a lot of extra copying, array padding, etc. which can slow it down quite a bit in a few nasty edge cases. Last time I used &lt;code&gt;sort&lt;/code&gt; it was quite slow compared to other libraries such as b40c or mgpu. However, NVIDIA has been working on improving Thrust's algorithmic performance so that may be less of an issue in the future. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should try writing your code using both Thrust and CUDA and then using the Visual Profiler to determine which is better for the specific task you are interested in. If it's likely that memory transfer will take up the most running time of your program and you don't want to have to worry about optimizing your own kernels for bank conflicts, instruction count, etc. then I would use Thrust. It also has the side benefit of making your code much less verbose and easier for people who are not familiar with GPU programming to read. &lt;/p&gt;&#xA;" OwnerUserId="1193" LastActivityDate="2012-07-17T16:33:03.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2856" PostTypeId="2" ParentId="2853" CreationDate="2012-07-17T19:43:43.153" Score="6" Body="&lt;p&gt;&lt;a href=&quot;http://arxiv.org/pdf/1006.4173.pdf&quot; rel=&quot;noreferrer&quot;&gt;This&lt;/a&gt; paper describes an algorithm to approximate the size of a resultant from the matrix product of two sparse matrices.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with finding an exact number of non-zero entries in a sparse matrix multiplication is that each element in the resultant depends on the interaction of two vectors, both of which are likely to contain at least a few non-zero elements.  Therefore, to calculate the number you need to evaluate logical operations on a pair of vectors for every element in the resultant.  The problem with this is that it requires a number of operations similar to the number of operations needed to calculate the matrix product itself.  In my comments I mentioned the possibility to exploit certain structures in the non-zero elements of the original matrices, however those same exploits could be used to reduce the work done in the matrix multiplication as well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would likely be better off to use the above paper to over-estimate the memory requirements, do the multiplication and then truncate the allocated memory, or move the resultant matrix to a more appropriately sized array.  Also, sparse matrix products are not a rare occurrence, and I would almost guarantee that this problem has been solved before.  A little digging into some open source, sparse matrix libraries should lead you to the algorithms they use to preallocate memory.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-07-17T19:43:43.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2857" PostTypeId="2" ParentId="2853" CreationDate="2012-07-17T20:09:19.733" Score="16" Body="&lt;p&gt;You can just simulate the matrix-matrix product by forming the product of the two sparsity patterns -- i.e., you consider the sparsity pattern (that is stored in separate arrays in CSR format) as a matrix that contains either a zero or a one in each entry. Performing this simulated product only requires you to form the &lt;em&gt;and&lt;/em&gt; operation on these zeros and ones and is thus much faster than the actual matrix-matrix product -- in fact, all you have to do is go through the rows and columns of the two matrices and verify that there is at least one entry in a row and the column you multiply with where both matrices are non-zero. This is a cheap operation -- much cheaper in any case than actually having to do the floating point multiplication in the actual product which not only requires you to do floating point arithmetic (expensive) but also read in the actual floating point numbers from memory (even more expensive, but you don't need that when multiplying the sparsity pattern because the non-zero values of the matrix are stored separately in CSR).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-17T20:09:19.733" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2858" PostTypeId="2" ParentId="2847" CreationDate="2012-07-17T20:14:16.317" Score="4" Body="&lt;p&gt;This depends on the equations in case. If your equations are first-order hyperbolic equations such as the Euler equations of gas dynamics, then the equations allow solutions that have truly discontinuous shocks -- in that case, there is no amount of mesh refinement that can resolve the shock.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if your equations are transport dominated but still have a diffusion term (like, I believe, what you call the Navier-Stokes equations, by which I believe you mean the compressible equations that still contain a certain amount of viscosity with a term that contains second order derivatives) then in general the solutions may have large gradients but are still continuous. In that case, at least conceptually, shocks are continuous and have a finite, positive width that can be resolved using mesh refinement -- although oftentimes the length scale of the shock is so small that you won't be able to resolve it with any practically usable mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in essence, it all boils down to which equations exactly you are considering, and if it has a diffusion/viscosity term.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-17T20:14:16.317" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2860" PostTypeId="2" ParentId="699" CreationDate="2012-07-18T12:29:26.007" Score="1" Body="&lt;p&gt;I work for Elsevier. My company has started using the Collage framework (developed in response to the Executable Paper Grand Challenge) in journal issues to enable authors to publish chunks of executable code with their article. This feature makes it easier for readers to reproduce results reported in the article and to reuse published material for their own research. Collage supports a wide variety of proprietary software, in addition to open-source software; more information can be found in the informational video &lt;a href=&quot;http://www.elsevier.com/wps/find/P05.cws_home/executablepaper&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and at the &lt;a href=&quot;https://collage.elsevier.com&quot; rel=&quot;nofollow&quot;&gt;Collage Authoring Environment Website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1776" LastEditorUserId="276" LastEditDate="2012-07-30T02:56:05.303" LastActivityDate="2012-07-30T02:56:05.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2861" PostTypeId="2" ParentId="2525" CreationDate="2012-07-18T12:32:52.810" Score="2" Body="&lt;p&gt;I work for Elsevier. My company has started using the Collage framework (developed in response to the Executable Paper Grand Challenge) in journal issues to enable authors to publish chunks of executable code with their article. This feature makes it easier for readers to reproduce results reported in the article and to reuse published material for their own research. Collage supports a wide variety of open-source and proprietary software; more information can be found in the informational video &lt;a href=&quot;http://www.elsevier.com/wps/find/P05.cws_home/executablepaper&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and at the &lt;a href=&quot;https://collage.elsevier.com&quot; rel=&quot;nofollow&quot;&gt;Collage Authoring Environment Website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1776" LastEditorUserId="549" LastEditDate="2012-07-27T13:09:13.343" LastActivityDate="2012-07-27T13:09:13.343" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2862" PostTypeId="2" ParentId="1879" CreationDate="2012-07-18T12:37:43.030" Score="2" Body="&lt;p&gt;I work for Elsevier. My company has started using the Collage framework (developed in response to the Executable Paper Grand Challenge) in journal issues to enable authors to include all of the data and code needed to reproduce the results and figures in their papers. This feature makes it easier for readers to reproduce results reported in the article and to reuse published material for their own research. Collage supports a wide variety of open-source and proprietary software; more information can be found in the informational video &lt;a href=&quot;http://www.elsevier.com/wps/find/P05.cws_home/executablepaper&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and at the &lt;a href=&quot;https://collage.elsevier.com&quot; rel=&quot;nofollow&quot;&gt;Collage Authoring Environment Website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1776" LastEditorUserId="276" LastEditDate="2012-07-30T02:51:42.737" LastActivityDate="2012-07-30T02:51:42.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2863" PostTypeId="1" AcceptedAnswerId="2865" CreationDate="2012-07-18T17:44:09.003" Score="6" ViewCount="146" Body="&lt;p&gt;Suppose I have N different airplanes traveling on a two dimensional rectangular plane of size 400km x 400km (i.e. it is as if all planes travel at the same altitude).  Assume each airplane has a randomly generated starting and ending points.  Also, assume that the airplanes can have different speeds.  We assume that two airplanes will &quot;collide&quot; if they come within 10 km of each other.  Based upon this information, how can I efficiently determine which pairs of airplanes will inevitably collide?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;:  As far as I can tell, it would require something similar to a line segment intersection algorithm, but instead of a line having zero width, it has a thickness of 20km.  I'm just not sure how to incorporate the time dynamics into the detection algorithm.  I suppose I could use a polygon-intersection at each (small) time step to determine if they collide, but I wanted to see if there is a more efficient way to detect it without going through pair-wise polygon intersection at each time step.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-07-18T21:12:21.987" LastActivityDate="2012-07-18T22:40:25.173" Title="How can I detect which among N bodies with different velocities will collide?" Tags="|computational-geometry|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2864" PostTypeId="2" ParentId="2863" CreationDate="2012-07-18T20:04:42.233" Score="3" Body="&lt;p&gt;Consider the trajectory of a plane as a line in three-dimensional $(x,y,t)$ space given by the segment $x_{start},y_{start},t_{start}$ to $x_{end},y_{end},t_{end}$. To make things like computing distances meaningful it is probably useful to measure time with the same units as distances, i.e., for example, in kilometers using the constant speed $v$ of planes. For example a time of $t=940$ km is then equivalent to $t=1$ hour assuming a speed of $v=940$ km/h.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each pair of planes it is then a simple exercise to compute the minimal Euclidean distance in this three-dimensional space. This already gives an indication whether planes come close to each other but it isn't quite what you want because the distance is not spatial but spatio-temporal. You'll have to work a bit to see if one can be converted into the other. (Essentially it is a question of which norm you compute the distance in, but that's not useful since it may not be convenient to compute the distance between two lines in anything but the Euclidean norm.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another idea that can work in this regard is to consider the &quot;safety zone&quot; of each plane as a cylinder or tube with spatial radius 20 kilometers around the space-time lines of each plane. Detecting collision then simply becomes an exercise to see if two such cylinders have an intersection.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-07-18T22:40:25.173" LastActivityDate="2012-07-18T22:40:25.173" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2865" PostTypeId="2" ParentId="2863" CreationDate="2012-07-18T20:59:36.103" Score="4" Body="&lt;p&gt;At a point in time, each airplane has a position and a velocity vector.&#xA;For each airplane A, subtract its position and velocity vector from every other airplane B.&#xA;That puts each other B airplane into an A-relative coordinate system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then for each B airplane, do a little trigonometry to see if its velocity vector brings it within a distance of 10 from the origin.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastActivityDate="2012-07-18T20:59:36.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2866" PostTypeId="2" ParentId="2863" CreationDate="2012-07-18T22:18:19.980" Score="2" Body="&lt;p&gt;Here is one way to perform the computation for any two planes $p_1$ and $p_2$, moving at velocities $v_1$ and $v_2$.  You could repeat this computation for every pair.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subtract $v_1$ from both velocities, so that now $p'_1$ is fixed and $p'_2$ is moving&#xA;with velocity $v_2-v_1$.&#xA;Imagine drawing the line $L$ that $p'_2$ follows.  You want the closest approach of $L$ to the fixed point $p'_1$.  This is a straightforward calculation, which could be organized as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assign a parameter $t$ to points $p'_2(t)$ on $L$.  Then compute $|| p'_1 - p'_2(t) ||$.&#xA;Minimize this w.r.t. $t$.  You will obtain the radius of the sphere centered on $p'_1$ that&#xA;is tangent to $L$.&lt;/p&gt;&#xA;" OwnerUserId="673" LastActivityDate="2012-07-18T22:18:19.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2867" PostTypeId="1" CreationDate="2012-07-19T02:42:41.103" Score="8" ViewCount="2124" Body="&lt;p&gt;I have part of a  problem that is described by the momentum conservation equation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \rho}{\partial t} + \frac{1}{\sin\theta} \frac{\partial}{\partial \theta}(\rho u \sin \theta) =0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u=f(\theta)$ and $ \rho = f(\theta,t) $ (constant velocity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively one could apply one of the solutions listed &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1960/a-good-finite-difference-for-the-continuity-equation&quot;&gt;here&lt;/a&gt;. The problem at hand is best described in spherical polar coordinates (thin spherical shell) and those solutions in Cartesian. &lt;strong&gt;must I make some kind of coordinate transformation before discretizing this equation&lt;/strong&gt; or can I discretize it directly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly &lt;strong&gt;is there any reason one should first expand the derivative in $\theta$&lt;/strong&gt; and then attempt to discretize?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a note - I have done several of the above and have obtained solutions that do not seem to be consistent (physically a couple seem to make sense). I am interested in if there is a proper coordinate transformation that should be made or if any of the previously mentioned methods will suffice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I define flux as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Phi_{i+1/2} = \dfrac{u_{i+1/2}+|u_{i+1/2}|}{2}\rho_{i} \sin{\theta_{i}} &#xA;              + \dfrac{u_{i+1/2}-|u_{i+1/2}|}{2}\rho_{i+1} \sin{\theta_{i+1}} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Phi_{i-1/2} = \dfrac{u_{i-1/2}+|u_{i-1/2}|}{2}\rho_{i-1} \sin{\theta_{i-1}}&#xA;              + \dfrac{u_{i-1/2}-|u_{i-1/2}|}{2}\rho_{i} \sin{\theta_{i}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm guessing the 'proper' way to define the flux would be to evaluate $\sin{\theta}$ at the $\pm\frac{1}{2}$ 'cell boundaries' rather than at the cell center. This would be more in line with the definition of flux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One final question - what should I do at the boundaries ($\theta = 0 $ is particularly a problem and I just avoid that point altogether).&lt;/p&gt;&#xA;" OwnerUserId="1779" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-07-20T14:31:36.223" Title="Finite difference coordinate transformation for spherical polar coordinates" Tags="|finite-difference|fluid-dynamics|hyperbolic-pde|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2868" PostTypeId="1" AcceptedAnswerId="2869" CreationDate="2012-07-19T05:22:23.657" Score="31" ViewCount="9030" Body="&lt;p&gt;The Journal of Computational Physics has been an important outlet for computational science in the past, and I have published there before.  For the benefit of those (like me) who have signed &lt;a href=&quot;http://thecostofknowledge.com&quot;&gt;the Elsevier boycott&lt;/a&gt;, &lt;strong&gt;what non-Elsevier journals would be appropriate places to publish articles that could have been submitted to the Journal of Computational Physics?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good alternative should:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Overlap (at least partially) in subject matter with JCP&lt;/li&gt;&#xA;&lt;li&gt;Have a good reputation&lt;/li&gt;&#xA;&lt;li&gt;Not be published by Elsevier&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: When I say &quot;reputation&quot;, I don't mean impact factor.  Please see &lt;a href=&quot;http://arxiv.org/abs/1010.0278&quot;&gt;this article&lt;/a&gt; that demonstrates that the two are not well-correlated in this field.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-08-12T08:36:54.157" LastActivityDate="2016-04-13T08:28:39.067" Title="Alternatives to Journal of Computational Physics" Tags="|publications|" AnswerCount="7" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2869" PostTypeId="2" ParentId="2868" CreationDate="2012-07-19T05:37:01.797" Score="30" Body="&lt;p&gt;The SIAM Journals, especially &lt;a href=&quot;http://www.siam.org/journals/sisc.php&quot;&gt;SISC (Scientific Computing)&lt;/a&gt; and &lt;a href=&quot;http://www.siam.org/journals/mms.php&quot;&gt;MMS (Multiscale Modeling and Simulation)&lt;/a&gt; are obvious established and high-quality choices.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="123" LastEditDate="2012-08-12T08:34:26.280" LastActivityDate="2012-08-12T08:34:26.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2870" PostTypeId="1" AcceptedAnswerId="2871" CreationDate="2012-07-19T07:06:27.590" Score="8" ViewCount="291" Body="&lt;p&gt;I've seen a lot of publications in Computational Physics journals use different metrics for the performance of their code. Especially for GPGPU code, there seems to be a great variety of timing results people publish. In particular, I've seen:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Comparisons of (essentially) running &lt;code&gt;time&lt;/code&gt; on the GPU and CPU versions and reporting an average&lt;/li&gt;&#xA;&lt;li&gt;Comparisons of profiling data on each GPU/CPU function called (so timing the run of &lt;code&gt;main&lt;/code&gt; and all functions called from it, but ignoring driver initialization time)&lt;/li&gt;&#xA;&lt;li&gt;Comparisons of profiling data for several relevant functions, ignoring things like memory transfers (across the PCI-E bus in this case), I/O to the disk, transforming the data from one format to another, etc &lt;/li&gt;&#xA;&lt;li&gt;Comparisons of profiling data for only one function (for instance, comparing only the time to do spin updates in a CPU vs GPU Quantum Monte Carlo) and ignoring things like memory transfer time, I/O to disk, setting up the Hamiltonian/diagonalizing it, etc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My feeling is that the first option is the most 'honest' of the four, although I can see the merits of the second and third as well. It's a bit ridiculous to report a runtime difference of 5s when 4.99s of it was the difference in I/O implementations between languages or Infiniband vs Gigabit. The last option seems a bit &quot;sketchy&quot; to me, since unless this function is the pain point of the whole program reporting information about it only won't reflect the performance someone replicating my results would see. Which of these approaches is more forthright? Ideally a paper would contain all of these but in the case of a limitation on figures/length, which is/are most valuable, honest, and relevant to provide?&lt;/p&gt;&#xA;" OwnerUserId="1193" LastActivityDate="2012-07-19T08:01:51.957" Title="How should I report profiling/timing information about my code?" Tags="|performance|publications|documentation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2871" PostTypeId="2" ParentId="2870" CreationDate="2012-07-19T07:33:51.437" Score="6" Body="&lt;p&gt;Total running time (wall clock) is the only metric that matters in industry or real life applications: this figure should never be omitted, even if embarrassing. Of course this metric is very dependent on the test environment, so this should be described in detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All other metrics can (or should be) reported if they provide insight into the problem, or interesting conclusions (e.g. verification of some theoretical bound) can be drawn from them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that &lt;a href=&quot;http://lemire.me/blog/archives/2012/01/27/citogenesis-in-science-and-the-importance-of-real-problems/&quot;&gt;this blog entry&lt;/a&gt; addresses this point, especially the last paragraph.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2012-07-19T07:33:51.437" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2872" PostTypeId="2" ParentId="2870" CreationDate="2012-07-19T08:01:51.957" Score="4" Body="&lt;p&gt;It is often the case that one can only report the tip of the iceberg of all the work and compromises that went into a piece of software. Reporting performance is nice but the real deal is when the code is made freely accesible on internet, this way, anyone interested can evaluate and reproduce the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, if you release the software, you can also make available the tests that generate the data presented in a paper.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-07-19T08:01:51.957" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2874" PostTypeId="2" ParentId="2867" CreationDate="2012-07-19T18:00:13.743" Score="3" Body="&lt;p&gt;There is no reason you have to transform the coordinates before discretizing the equation.  However, since you are using spherical coordinates, you will end up with a non-linear system rather than the linear system that would appear in cartesian coordinates.  If you want to work with a linear system, you will have to do variable transformations (much like the creation of $\Phi$ that appears in the first answer of your linked question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as expanding the derivative in $\theta$, I would recommend not to, unless you must for the above transformations.  The equation in its current state is in the conservative form, with all $u$'s being coupled with $\rho$.  When you expand the derivative you will lose this coupling.  The two solutions will converge to each other with finer and finer meshes, however there is a non-zero difference between them.  Also, someone else with more experience may be able to elaborate on the advantage/disadvantages of the two forms, since even though it is called the conservative form of the equation, finite difference methods are, by their nature, non-conservative on some level.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-07-19T18:00:13.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2875" PostTypeId="2" ParentId="2868" CreationDate="2012-07-19T22:37:24.993" Score="4" Body="&lt;p&gt;Although not as good as SIAM journals, &lt;a href=&quot;http://www.global-sci.com/&quot; rel=&quot;nofollow&quot;&gt;Communications in Computational Physics&lt;/a&gt; (CiCP) also seems promising.&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="123" LastEditDate="2012-08-11T20:32:21.143" LastActivityDate="2012-08-11T20:32:21.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2876" PostTypeId="1" AcceptedAnswerId="2879" CreationDate="2012-07-20T04:03:53.300" Score="8" ViewCount="2298" Body="&lt;p&gt;I have a bunch of MPI processes exchanging request messages back and forth.  Processes do not know which other processes will send them messages, or how many.  Given this situation, I want an efficient way to know whether all other processes consider themselves done sending messages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be accomplished perfectly by the following nonblocking version of MPI_Barrier, which we'll call MPI_Ibarrier:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int MPI_Ibarrier(MPI_Comm comm, MPI_Request* request);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;MPI_Ibarrier&lt;/code&gt; would return immediately, and standard operations on the request object would let us know when the barrier has been reached by everyone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to simulate this behavior efficiently in MPI 2 (i.e., without official nonblocking collectives)?&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-09-17T16:35:36.153" Title="Nonblocking version of MPI_Barrier in MPI 2" Tags="|mpi|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2877" PostTypeId="2" ParentId="2868" CreationDate="2012-07-20T08:01:29.883" Score="7" Body="&lt;p&gt;I think there are several alternatives, but to some extent, it may depend upon the field. For fluid mechanics, there's obviously &lt;a href=&quot;http://journals.cambridge.org/action/displayJournal?jid=FLM&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;J. Fluid Mech.&lt;/em&gt;&lt;/a&gt;. For many areas in microscopic physics, you could send your work to &lt;a href=&quot;http://jcp.aip.org&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;J. Chem. Phys.&lt;/em&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="123" LastEditDate="2016-04-13T08:28:39.067" LastActivityDate="2016-04-13T08:28:39.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2879" PostTypeId="2" ParentId="2876" CreationDate="2012-07-20T12:27:28.510" Score="12" Body="&lt;p&gt;Remarkably, &lt;code&gt;MPI_Ibarrier&lt;/code&gt; is a very useful routine. For example, you can deliver an unstructured round of messages to ranks that do not know how many messages to receive by sending with &lt;code&gt;MPI_Issend&lt;/code&gt; (yes, a rare use of synchronous send), then entering a loop of alternating &lt;code&gt;MPI_Testall&lt;/code&gt; (to see if the sends completed) and &lt;code&gt;MPI_Iprobe&lt;/code&gt; (to process incoming messages). When the sends complete, you post &lt;code&gt;MPI_Ibarrier&lt;/code&gt; and alternative testing the barrier an probing for incoming messages. Torsten Hoefler has a paper on this where he proves communication optimality, see Algorithm 2: &lt;a href=&quot;http://unixer.de/publications/img/hoefler-dsde-protocols.pdf&quot;&gt;http://unixer.de/publications/img/hoefler-dsde-protocols.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that a barrier does not ensure that point-to-point messages or other nonblocking collectives posted before the barrier have completed. If you want them to complete, you have to ensure that they have completed before posting the barrier. As Bill says, (blocking) &lt;code&gt;MPI_Barrier&lt;/code&gt; is incorrect/unnecessary in most cases. An exception is communicating through side channels such as the file system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although there is not a similarly performant way to simulate &lt;code&gt;MPI_Ibarrier&lt;/code&gt; with MPI-2, MPICH2 provides &lt;code&gt;MPIX_Ibarrier&lt;/code&gt; in the 1.5 branch (current). Vendor networks generally support these operations so the vendor implementations (which are generally derived from MPICH2) just need an interface. As soon as their patches move to 1.5, &lt;code&gt;MPI_Ibarrier&lt;/code&gt; and other nonblocking collectives should be supported. The Open MPI development branch has an implementation of &lt;code&gt;MPI_Ibarrier&lt;/code&gt; based on libNBC.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-09-17T16:35:36.153" LastActivityDate="2012-09-17T16:35:36.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2881" PostTypeId="1" CreationDate="2012-07-20T16:21:32.610" Score="5" ViewCount="1167" Body="&lt;p&gt;I have troubles imagining how to compute a volume and centroid of an n-dimesional convex polytope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a polygon (especially for convex polygon) the area and centroid are described in (&lt;a href=&quot;http://en.wikipedia.org/wiki/Polygon#Area_and_centroid&quot; rel=&quot;noreferrer&quot;&gt;wiki&lt;/a&gt;) by &#xA;$$&#xA;A= \frac{1}{2} \sum \limits_{i=0}^n (x_iy_{i+1}-x_{i+1}y_i)&#xA;$$&#xA;For k-dimensional case the volume (&lt;a href=&quot;http://en.wikipedia.org/wiki/Volume&quot; rel=&quot;noreferrer&quot;&gt;Volume&lt;/a&gt;) is &#xA;$$&#xA;\int\limits_a^bA(h)\mathrm{d}h&#xA;$$&#xA;Still, maybe I need another cup of coffee to be able to transfer this into code. So i was thinking about this&#xA;$$&#xA;A= \sum\limits_{d=1}^k \sum\limits_{\forall d' \neq d} \frac{1}{k}\sum \limits_{i=0}^n (p^{(d)}_i p^{(d')}_{i+1}-p^{(d)}_{i+1}p^{(d')}_i)&#xA;$$&#xA;with $p^{(d)}$ beeing a variable in dimesion $d$ &#xA;But I do not trust myself ;-) Do you have an idea ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for centroids, I do not understand where the factor $\frac{1}{6}$ comes from&#xA;$$&#xA;C_x = \frac{1}{6 A} \sum_{i = 0}^{n - 1} (x_i + x_{i + 1}) (x_i y_{i + 1} - x_{i + 1} y_i)&#xA;$$&#xA;and would blindly guess&#xA;$$&#xA;C_{(d)} = \frac{1}{(k+1)! A} \sum\limits_{\forall d' \neq d}\sum_{i = 0}^{n - 1} (p_i^{(d)} + p_{i + 1}^{(d)}) (p_i^{(d)} p_{i + 1}^{(d')} - p_{i + 1}^{(d)} p_i^{(d')})&#xA;$$&#xA;Do you think this is right? I appriciate any feedback!&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1795" LastActivityDate="2012-07-20T21:14:58.790" Title="Convex polytope volume and centroid calculation" Tags="|computational-geometry|high-dimensional|convex-hull|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2882" PostTypeId="2" ParentId="2881" CreationDate="2012-07-20T21:14:58.790" Score="3" Body="&lt;p&gt;If the dimension is not too high, you can use a triangulation library (e.g. &lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Convex_hull_d/Chapter_main.html&quot; rel=&quot;nofollow&quot;&gt;CGAL&lt;/a&gt;) to decompose the polytope into n-dimensional simplices; calculating the volume of a simplex has a simple &lt;a href=&quot;http://en.wikipedia.org/wiki/Simplex#Geometric_properties&quot; rel=&quot;nofollow&quot;&gt;formula.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1665" LastActivityDate="2012-07-20T21:14:58.790" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2883" PostTypeId="2" ParentId="2868" CreationDate="2012-07-20T21:43:46.900" Score="5" Body="&lt;p&gt;Along the same lines as aeismail's answer, &lt;a href=&quot;http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291097-0207/&quot;&gt;International Journal of Numerical Methods in Engineering&lt;/a&gt;, and &lt;a href=&quot;http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291097-0363&quot;&gt;International Journal of Numerical Methods in Fluids&lt;/a&gt; could be candidate journals for papers that would otherwise go into Journal of Computational Physics. (These journals are both Wiley journals.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-07-20T21:43:46.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2884" PostTypeId="1" AcceptedAnswerId="2885" CreationDate="2012-07-21T11:19:59.637" Score="11" ViewCount="1246" Body="&lt;p&gt;Is there any hope in solving the following linear system efficiently with an iterative method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A \in \mathbb{R}^{n \times n}, x \in \mathbb{R}^n, b \in \mathbb{R}^n \text{, with } n &amp;gt; 10^6$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;A=(\Delta - K)&#xA;$, where $\Delta$ is a very sparse matrix with a few diagonals, arising from the discretization of the Laplace Operator. On it's main diagonal there is $-6$ and there are $6$ other diagonals with $1$ on it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K$ is a full $\mathbb{R}^{n \times n}$ matrix that consists completely of ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving $A=\Delta$ works fine with iterative methods like Gauss-Seidel, because it's a sparse diagonally dominant matrix. I suspect that the problem $A=(\Delta - K)$ is pretty much impossible to solve efficiently for large numbers of $n$, but is there any trick to maybe solve it, exploiting the structure of $K$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Would doing something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta x^{k+1} = b + Kx^{k}$ // solve for $x^{k+1}$ with Gauss-Seidel&lt;/p&gt;&#xA;&#xA;&lt;p&gt;converge to the correct solution? I read that such a Splitting Method converges if $\rho(\Delta^{-1} K) &amp;lt; 1$, where $\rho$ is the spectral norm. I manually calculated the eigenvalues of $\Delta^{-1} K$ for some different small values of $n$ and they're all zero except the one which has a pretty high negative value. (about ~500 for $n=256$) So I guess that wouldn't work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT: More information about $\Delta$&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta \in \mathbb{R}^{n \times n}$ is symmetric and is negative definite and diagonally dominant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is created the following way in matlab &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;n=W*H*D;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;e=ones(W*H*D,1);&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;d=[e,e,e,-6*e,e,e,e];&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;delta=spdiags(d, [-W*H, -W, -1, 0, 1, W, W*H], n, n);&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1798" LastEditorUserId="1798" LastEditDate="2012-07-21T15:46:14.203" LastActivityDate="2012-07-21T15:46:14.203" Title="Solving huge dense linear system?" Tags="|linear-solver|dense-matrix|linear-system|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2885" PostTypeId="2" ParentId="2884" CreationDate="2012-07-21T15:44:17.513" Score="14" Body="&lt;p&gt;There are two choices which, if you use reasonable data structures, can solve the problem with $n&amp;gt;10^6$ on a laptop and $n \approx 10^{12}$ on a supercomputer. Note that for efficiency, you should use multigrid to solve with $\Delta$. The cost in either case will be a small factor more expensive than just solving with $\Delta$. The two approaches are equivalent via a Schur complement argument, but I describe them separately because the implementation is different.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use the bordered system&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$ \newcommand\ee{\mathbf{e}} M = \begin{pmatrix} \Delta &amp;amp; \ee \\ \ee^T &amp;amp; 1 \end{pmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\ee$ is a column vector consisting of all ones and solve the system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ M \begin{pmatrix} \mathbf{x} \\ y \end{pmatrix} = \begin{pmatrix} \mathbf{b} \\ 0 \end{pmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using an iterative or direct solver.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use a Krylov method and apply the matrix $A$ as $\Delta - \ee \ee^T$ (i.e. sparse matrix plus rank-1 correction. Use your existing preconditioner $P^{-1} \approx \Delta^{-1}$, or, especially if you wanted to use a direct solve with $\Delta$, update it with the &lt;a href=&quot;http://en.wikipedia.org/wiki/Sherman%E2%80%93Morrison_formula&quot;&gt;Sherman-Morrison formula&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-21T15:44:17.513" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2887" PostTypeId="1" CreationDate="2012-07-21T23:30:03.890" Score="10" ViewCount="1178" Body="&lt;p&gt;I am solving Turing's reaction-diffusion system with following C++ code. It is too slow: for 128x128 pixel texture, acceptable number of iterations is 200 – which results in 2.5 seconds of delay. I need 400 iterations to obtain interesting image – but 5 seconds of waiting is too much. Also, size of the texture should be in fact 512x512 – but this results in huge waiting time. The devices are iPad, iPod.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any chance to do this faster? Euler method converges slowly (wikipedia) – having quicker method would allow to drop number of iterations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; As Thomas Klimpel pointed out, the lines: &quot;if( m_An[i][j] &amp;lt; 0.0 ) { ... }&quot;, &quot;if( m_Bn[i][j] &amp;lt; 0.0 ) { ... }&quot; are delaying convergence: after removing, meaningful image appears after &lt;strong&gt;75 iterations&lt;/strong&gt;. I have commented out the lines in code below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void TuringSystem::solve( int iterations, double CA, double CB ) {&#xA;    m_iterations = iterations;&#xA;    m_CA = CA;&#xA;    m_CB = CB;&#xA;&#xA;    solveProcess();&#xA;}&#xA;&#xA;void set_torus( int &amp;amp; x_plus1, int &amp;amp; x_minus1, int x, int size ) {&#xA;    // Wrap &quot;edges&quot;&#xA;    x_plus1 = x+1;&#xA;    x_minus1 = x-1;&#xA;    if( x == size - 1 ) { x_plus1 = 0; }&#xA;    if( x == 0 ) { x_minus1 = size - 1; }&#xA;}&#xA;&#xA;void TuringSystem::solveProcess() {&#xA;    int n, i, j, i_add1, i_sub1, j_add1, j_sub1;&#xA;    double DiA, ReA, DiB, ReB;&#xA;&#xA;    // uses Euler's method to solve the diff eqns&#xA;    for( n=0; n &amp;lt; m_iterations; ++n ) {&#xA;        for( i=0; i &amp;lt; m_height; ++i ) {&#xA;            set_torus(i_add1, i_sub1, i, m_height);&#xA;&#xA;            for( j=0; j &amp;lt; m_width; ++j ) {&#xA;                set_torus(j_add1, j_sub1, j, m_width);&#xA;&#xA;                // Component A&#xA;                DiA = m_CA * ( m_Ao[i_add1][j] - 2.0 * m_Ao[i][j] + m_Ao[i_sub1][j]   +   m_Ao[i][j_add1] - 2.0 * m_Ao[i][j] + m_Ao[i][j_sub1] );&#xA;                ReA = m_Ao[i][j] * m_Bo[i][j] - m_Ao[i][j] - 12.0;&#xA;                m_An[i][j] = m_Ao[i][j] + 0.01 * (ReA + DiA);&#xA;                // if( m_An[i][j] &amp;lt; 0.0 ) { m_An[i][j] = 0.0; }&#xA;&#xA;                // Component B&#xA;                DiB = m_CB * ( m_Bo[i_add1][j] - 2.0 * m_Bo[i][j] + m_Bo[i_sub1][j]   +   m_Bo[i][j_add1] - 2.0 * m_Bo[i][j] + m_Bo[i][j_sub1] );&#xA;                ReB = 16.0 - m_Ao[i][j] * m_Bo[i][j];&#xA;                m_Bn[i][j] = m_Bo[i][j] + 0.01 * (ReB + DiB);&#xA;                // if( m_Bn[i][j] &amp;lt; 0.0 ) { m_Bn[i][j]=0.0; }&#xA;            }&#xA;        }&#xA;&#xA;        // Swap Ao for An, Bo for Bn&#xA;        swapBuffers();&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1799" LastEditorUserId="1799" LastEditDate="2012-07-22T22:16:13.590" LastActivityDate="2012-07-22T22:16:13.590" Title="Explicit Euler method too slow for reaction-diffusion problem" Tags="|pde|stiffness|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2888" PostTypeId="2" ParentId="2887" CreationDate="2012-07-22T00:11:34.353" Score="0" Body="&lt;p&gt;Euler does converge slowly in relation to other methods, however I don't think that is what you are interested.  If you are just looking for &quot;interesting&quot; images, increase the size of your time step and take less iterations.  The problem, as Jed points out, is that the explicit euler method has stability issues with large time steps in relation to grid size.  the smaller your grid is (i.e. the higher resolution your image is), the smaller your time step must be to account for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, by using implicit euler instead of explicit, you don't gain any orders of convergence, but the solution will have unconditional stability, allowing much larger time steps.  Implicit methods are more complicated to implement and take more computation per time-step, but you should see gains well beyond that by taking less steps in total.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2012-07-22T01:02:34.887" LastActivityDate="2012-07-22T01:02:34.887" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2889" PostTypeId="2" ParentId="2887" CreationDate="2012-07-22T00:49:00.553" Score="9" Body="&lt;p&gt;You seem to be limited by stability, which is expected since diffusion is stiff as you refine the grid. Good methods for stiff systems are at least partly implicit. It will take some effort, but you can implement a simple multigrid algorithm (or use a library) to solve this system with a cost of less than ten &quot;work units&quot; (essentially the cost of one of your time steps). When you refine the grid, the number of iterations will not increase.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-22T00:49:00.553" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2890" PostTypeId="2" ParentId="2853" CreationDate="2012-07-22T06:09:25.007" Score="0" Body="&lt;p&gt;The simplest way to implements CSR is to try &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;std::vector&amp;lt; std::map&amp;lt;int, complex&amp;lt;float&amp;gt;&amp;gt; &amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to represent your matrix. In that case you will not really worry about number of non zero elements, all is accessed via&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;std::map&amp;lt; int, complex&amp;lt;float&amp;gt; &amp;gt;::iterator&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;on each row. &#xA;Best ..&lt;/p&gt;&#xA;" OwnerDisplayName="user814" LastActivityDate="2012-07-22T06:09:25.007" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2891" PostTypeId="2" ParentId="2887" CreationDate="2012-07-22T12:26:53.543" Score="2" Body="&lt;p&gt;From a practical point of view: the A5 processor is not that much powerful, so you can wait a few HW iterations, or if your ipod / ipad are going to be connected to the internet, solve your problem remotely or in the cloud.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-07-22T12:26:53.543" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2892" PostTypeId="2" ParentId="2868" CreationDate="2012-07-22T13:19:10.267" Score="9" Body="&lt;p&gt;&lt;strong&gt;Physical Review E&lt;/strong&gt;&lt;br&gt;&#xA;&lt;a href=&quot;http://pre.aps.org/about&quot;&gt;http://pre.aps.org/about&lt;/a&gt;&lt;br&gt;&#xA;Physical Review E (PRE), interdisciplinary in scope, focuses on many-body phenomena, including recent developments in quantum and classical chaos and soft matter physics. It has sections on statistical physics, equilibrium and transport properties of fluids, liquid crystals, complex fluids, polymers, chaos, &lt;strong&gt;fluid dynamics&lt;/strong&gt;, plasma physics, classical physics, and &lt;strong&gt;computational physics&lt;/strong&gt;. In addition, the journal features sections on two rapidly growing areas: biological physics and granular materials.&lt;/p&gt;&#xA;" OwnerUserId="1802" LastActivityDate="2012-07-22T13:19:10.267" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2893" PostTypeId="2" ParentId="2802" CreationDate="2012-07-22T15:43:31.973" Score="3" Body="&lt;p&gt;If need be, you can use the &quot;seminormal equations&quot;; the key here is that the $\mathbf R$ factor in the QR decomposition of $\mathbf A$ is the same (up to changes in signs of entries) as the Cholesky triangle of $\mathbf A^\top\mathbf A$, since&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf A^\top\mathbf A=(\mathbf Q\mathbf R)^\top(\mathbf Q\mathbf R)=\mathbf R^\top\mathbf Q^\top\mathbf Q\mathbf R=\mathbf R^\top\mathbf R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is especially convenient if you are unable or unwilling to store the orthogonal matrix, either explicitly or as its decomposition in Householder vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, you can solve the equations $\mathbf R^\top\mathbf R\mathbf x=\mathbf A^\top\mathbf b$ to obtain your least-squares solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, nothing beats the diagnostic ability afforded by the singular-value decomposition; if you can afford to use it, you should, unless you know well in advance that your least-squares problems will always be well-conditioned.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2012-07-22T15:43:31.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2894" PostTypeId="2" ParentId="1528" CreationDate="2012-07-22T17:10:01.210" Score="2" Body="&lt;p&gt;The claim that $\DeclareMathOperator{\cond}{cond} \cond A^2 \approx \cond A^T A$ (for square matrices) &lt;strike&gt;in the question and&lt;/strike&gt; [Edit: I misread] in Artan's answer is nonsense. Counter-example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\newcommand\bigO{\mathcal{O}}A = \begin{pmatrix} \epsilon &amp;amp; 1 \\ 0 &amp;amp; \epsilon \end{pmatrix}, \quad \epsilon \ll 1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for which you can easily check that $\cond A^T A = \bigO(\epsilon^{-4})$ while $\cond A^2 = \bigO(\epsilon^{-2})$.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-07-25T00:23:30.283" LastActivityDate="2012-07-25T00:23:30.283" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2895" PostTypeId="1" AcceptedAnswerId="2903" CreationDate="2012-07-22T23:00:38.813" Score="3" ViewCount="1090" Body="&lt;p&gt;I have an MPI application that needs to asynchronously respond to both incoming messages and request completions inside a dedicated communication thread.  The obvious way to do this is a busy wait that alternately calls MPI_Iprobe and MPI_Testsome.  Should I be worried about performance if I do this?  Is it better to just use two threads and eat the context switching time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an architecture-unspecific answer is unreasonable, I'm shooting for either BlueGene/Q or Cray XE6.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately (in terms of MPI_Testsome costs), I expect to have O(100) requests active at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I'm memory constrained, so it's important to detect completed requests as soon as possible.  A completed request might free up enough space to schedule more compute on other threads.&lt;/li&gt;&#xA;&lt;li&gt;The incoming messages have varying sizes, and I don't have nearly enough memory to allocate buffers for all of them, so I can't switch to blanket MPI_Irecvs instead of the MPI_Iprobes.&lt;/li&gt;&#xA;&lt;li&gt;It's a shame MPI doesn't let you do an MPI_Irecv with only an upper bound on the size of the incoming message, since this would solve my problem perfectly.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2012-07-23T17:07:54.960" LastActivityDate="2012-07-25T14:03:26.763" Title="Is busy waiting on both MPI_Iprobe and MPI_Testsome efficient?" Tags="|mpi|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2896" PostTypeId="2" ParentId="2876" CreationDate="2012-07-22T23:53:51.070" Score="8" Body="&lt;p&gt;See &lt;a href=&quot;http://www.unixer.de/research/nbcoll/&quot;&gt;http://www.unixer.de/research/nbcoll/&lt;/a&gt;, especially &lt;a href=&quot;http://www.unixer.de/research/nbcoll/libnbc/&quot;&gt;http://www.unixer.de/research/nbcoll/libnbc/&lt;/a&gt;, which is the reference implementation that sits on top of MPI-1.  As Jed said, MPICH2 1.5 already has NBC and the implementation is similar to Torsten's (although almost certainly faster due to lower overhead).&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2012-07-22T23:53:51.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2897" PostTypeId="2" ParentId="2846" CreationDate="2012-07-23T00:02:31.357" Score="5" Body="&lt;p&gt;Overlapping windows are permitted in MPI-2.  In fact, the RMA working group actually believes that this is a sufficiently reasonable usage that they refused better mechanisms for switching between usage modes (via window attributes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is unrelated, but I'm the person who got request-based RMA into MPI-3.  And I had to fight like hell for them.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2012-07-23T00:02:31.357" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2898" PostTypeId="2" ParentId="2895" CreationDate="2012-07-23T01:34:39.500" Score="2" Body="&lt;p&gt;If you could use &lt;code&gt;MPI_Irecv&lt;/code&gt; with an upper bound on the message size, why not just send that number of bytes every time (i.e. pad messages)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's hard to say whether 1 thread doing &lt;code&gt;Iprobe&lt;/code&gt; and &lt;code&gt;Testsome&lt;/code&gt; is better or worse than a thread for each.  This is going to be incredibly dependent on the usage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How hard is it to implement both approaches and compare them in the wild?&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="119" LastEditDate="2012-07-23T02:34:51.860" LastActivityDate="2012-07-23T02:34:51.860" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2899" PostTypeId="2" ParentId="2064" CreationDate="2012-07-23T02:00:15.380" Score="5" Body="&lt;p&gt;This is not supported by MPI nor any other communication middleware I know of.  This is probably because it's not supported by any hardware I know of, with the exception of Blue Gene, where there are high-priority packets for control messages that will overtake other messages under some conditions.  However, these are not for general use since they only allow one to communicate 64 bytes (at least on Blue Gene/P).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The good news is that you don't need this.  The overhead to implement it is not going to be worth it and you will find - assuming you ever investigate the low-level details - that not implementing priorities in the network allows MPI to deliver the best performance in most usages.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2013-04-13T19:31:51.633" LastActivityDate="2013-04-13T19:31:51.633" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2900" PostTypeId="2" ParentId="976" CreationDate="2012-07-23T02:13:59.903" Score="0" Body="&lt;p&gt;Why bother counting flops?  Just count cycles for every operation and you'll have something that is universal.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2012-07-23T02:13:59.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2901" PostTypeId="2" ParentId="2384" CreationDate="2012-07-23T02:17:05.360" Score="6" Body="&lt;p&gt;You can implement a numerically stable reduction algorithm in MPI the same as you can do in serial.  There may be a performance hit, of course.  If you can afford to replicate the vector, just use MPI_Gather and do the numerically stable reduction in serial on the root.  In some cases, you may find the performance hit is not a big deal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another solution is to use wide accumulators as described &lt;a href=&quot;http://sites.utexas.edu/jdm4372/2012/02/15/is-ordered-summation-a-hard-problem-to-speed-up/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. You can do this with MPI as a user-defines reduction, although it will use a lot more bandwidth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A compromise for the above is to use compensated summation.  See references “Kahan summation” for details. Higham’s “&lt;a href=&quot;http://bookstore.siam.org/ot80&quot; rel=&quot;nofollow noreferrer&quot;&gt;Accuracy and Stability of Numerical Algorithms&lt;/a&gt;” is an excellent resource on this topic.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="17088" LastEditDate="2019-06-24T16:13:02.893" LastActivityDate="2019-06-24T16:13:02.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2903" PostTypeId="2" ParentId="2895" CreationDate="2012-07-23T13:40:40.597" Score="4" Body="&lt;p&gt;As Jeremiah W. mentioned in a note above, your &quot;Note 3&quot; actually is explicitly supported by MPI.  You can always post a larger receive buffer than the message that you will actually be sent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this is fine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (rank == 0) {&#xA;    MPI_Request req;&#xA;    MPI_Status status;&#xA;    int num_received;&#xA;    MPI_Irecv(recvbuf, 100, MPI_DOUBLE, 1, 1234, MPI_COMM_WORLD, &amp;amp;req);&#xA;    MPI_Wait(&amp;amp;req, &amp;amp;status);&#xA;    MPI_Get_count(&amp;amp;status, MPI_DOUBLE, &amp;amp;num_received);&#xA;    /* num_received should now contain 5 */&#xA;}&#xA;else if (rank == 1) {&#xA;    MPI_Send(sendbuf, 5, MPI_DOUBLE, 0, 1234, MPI_COMM_WORLD);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1807" LastEditorUserId="119" LastEditDate="2012-07-23T18:19:11.910" LastActivityDate="2012-07-23T18:19:11.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2904" PostTypeId="2" ParentId="1818" CreationDate="2012-07-23T14:32:20.773" Score="0" Body="&lt;p&gt;If you are looking for a strictly numerical way to estimate the largest eigenvalue of a symmetric matrix you can use Jacobi rotations. A nice treatment of the algorithm is given in Chap. 8 of &lt;em&gt;Matrix Computations&lt;/em&gt; by Golub and van Loan (for non-symmetric eigenvalue problems you basically have to resort to something like a power method which are covered in Chap. 7 of the same book). Essentially the algorithm pushes the &quot;energy&quot; of the matrix onto the main diagonal, causing the symmetric matrix to converge to the diagonal matrix of eigenvalues. Because the matrix is already diagonally dominent the convergence of this method should be relatively fast even though it will cause fill in of the pentadiagonal matrix to a dense matrix. The fill in entries will be small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The form of the Jacobi rotation allows the application of the rotation to be cheap even to a dense matrix. The rotation is an identity matrix with four off-diagonal entries which are non-zero. You won't have to run the algorithm to full convergence if you just want a descent estimate of the largest eigenvalue. You could run the algorithm a few times until the off-diagonal entries are &quot;small enough&quot; for your purposes, then sweep through the main diagonal of the matrix and pick off the largest value. &lt;/p&gt;&#xA;" OwnerUserId="1057" LastActivityDate="2012-07-23T14:32:20.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2905" PostTypeId="1" AcceptedAnswerId="2906" CreationDate="2012-07-23T14:47:58.110" Score="9" ViewCount="1880" Body="&lt;p&gt;Newton's method for solving nonlinear equations is known to converge quadratically when the starting guess is &quot;sufficiently close&quot; to the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is &quot;sufficiently close&quot;?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there literature about the structure of this basin of attraction?&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-12-30T23:01:17.337" Title="Basin of attraction for Newton's method" Tags="|iterative-method|convergence|nonlinear-equations|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2906" PostTypeId="2" ParentId="2905" CreationDate="2012-07-23T16:11:22.490" Score="8" Body="&lt;p&gt;For a single rational equation in the complex domain, the basin of attraction is fractal, the compelement of a so-called Julia set. &lt;a href=&quot;http://en.wikipedia.org/wiki/Julia_set&quot;&gt;http://en.wikipedia.org/wiki/Julia_set&lt;/a&gt; . For theory with some nice online figures, see, e.g., &lt;BR&gt;&#xA;&lt;a href=&quot;http://mathlab.mathlab.sunysb.edu/~scott/Papers/Newton/Published.pdf&quot;&gt;http://mathlab.mathlab.sunysb.edu/~scott/Papers/Newton/Published.pdf&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://hera.ugr.es/doi/15019160.pdf&quot;&gt;http://hera.ugr.es/doi/15019160.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even the ''globalized'' damped Newton method for $x^3-1=0$ has a fractal basin of attraction; see &lt;a href=&quot;http://www.jstor.org/stable/10.2307/2653002&quot;&gt;http://www.jstor.org/stable/10.2307/2653002&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus there is little point in specifying in detail what is &quot;sufficiently close&quot; to the solution. If one knows bounds on the second derivatives, there is the Newton--Kantorovich theorem which gives lower bounds on the radius of a ball in which Newton's method converges, but except in 1D, these tend to be quite pessimistic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computationally useful bounds can be obtained using interval arithmetic; see, e.g., my paper &lt;BR&gt;&#xA;Shen Zuhe and A. Neumaier, &#xA;The Krawczyk operator and Kantorovich's theorem, &#xA;J. Math. Anal. Appl. 149 (1990), 437-443. &lt;BR&gt;&#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/scan/61.pdf&quot;&gt;http://www.mat.univie.ac.at/~neum/scan/61.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-07-23T16:17:45.430" LastActivityDate="2012-07-23T16:17:45.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2907" PostTypeId="2" ParentId="2905" CreationDate="2012-07-23T16:13:33.673" Score="4" Body="&lt;p&gt;&quot;Sufficiently close&quot; is difficult to characterize considering that it gives rise to a &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton_fractal&quot; rel=&quot;nofollow&quot;&gt;class of fractals&lt;/a&gt;. Newton methods with globalization strategies such as line search and trust region extend the basin of attraction. If additional problem structure is available, such as in optimization, the assumptions necessary for convergence can be further weakened.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-23T16:13:33.673" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2908" PostTypeId="2" ParentId="2844" CreationDate="2012-07-23T16:30:43.827" Score="5" Body="&lt;p&gt;Loop &amp;amp; Zhang present a solution in &lt;a href=&quot;http://dx.doi.org/10.1109/CVPR.1999.786928&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt;. They use a shearing transform to reduce the distortion introduced by the projective transform that mapped the epipoles to infinity (ie, that made the epipolar lines parallel). Consider the shearing transform &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     | k1 k2 0 |&#xA;S =  | 0   1 0 |.&#xA;     | 0   0 1 |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Let &lt;em&gt;w&lt;/em&gt; and &lt;em&gt;h&lt;/em&gt; be image width and height respectively. Consider the four midpoints of the image edges:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = ((w-1)/2, 0, 1),&#xA;b = (w-1, (h-1)/2, 1),&#xA;c = ((w-1)/2, h-1, 1) and&#xA;d = (0, (h-1)/2, 1).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;According to Loop &amp;amp; Zhang:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(...) we attempt to preserve&#xA;  &lt;strong&gt;perpendicularity&lt;/strong&gt; and &lt;strong&gt;aspect ratio&lt;/strong&gt; of&#xA;  the lines bd and ca&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let &lt;code&gt;H&lt;/code&gt; be the rectification homography and let &lt;em&gt;a' = Ha&lt;/em&gt; be a point in the affine plane by dividing through so that &lt;em&gt;a'&lt;a href=&quot;https://i.stack.imgur.com/zRgkU.png&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt; = 1&lt;/em&gt; (note a'&lt;a href=&quot;https://i.stack.imgur.com/zRgkU.png&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt; is the &lt;strong&gt;third&lt;/strong&gt; component, ie, &lt;em&gt;a' = (a'[0], a'&lt;a href=&quot;https://i.stack.imgur.com/zI6S6.png&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;, a'&lt;a href=&quot;https://i.stack.imgur.com/zRgkU.png&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;)&lt;/em&gt;). Let&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = b' - d',&#xA;y = c' - a'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;According to Loop &amp;amp; Zhang:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As the difference of affine points, &lt;strong&gt;x&lt;/strong&gt;&#xA;  and &lt;strong&gt;y&lt;/strong&gt; are vectors in the euclidean&#xA;  image plane. Perpendicularity is&#xA;  preserved when (Sx)^T(Sy) = 0, and&#xA;  aspect ratio is preserved if&#xA;  [(Sx)^T(Sx)]/[(Sy)^T(Sy)] = w²/h².&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The real solution presents a &lt;strong&gt;closed-form&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k1 = (h²x[1]² + w²y[1]²)/(hw(x[1]y[0] - x[0]y[1])) and&#xA;k2 = (h²x[0]x[1] + w²y[0]y[1])/(hw(x[0]y[1] - x[1]y[0]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;up to sign (the positive is preferred).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the result after applying shearing &lt;em&gt;S&lt;/em&gt; on the left image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zI6S6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the left and right images overlaped:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zRgkU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, OpenCV stereo method &lt;code&gt;StereoBM&lt;/code&gt; can compute a reasonable result for disparity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/NxOCP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="599" LastActivityDate="2012-07-23T16:30:43.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2909" PostTypeId="1" CreationDate="2012-07-24T09:40:59.423" Score="2" ViewCount="1570" Body="&lt;p&gt;I would like to use &lt;a href=&quot;https://github.com/OpenFOAM/OpenFOAM-2.1.x/blob/master/applications/utilities/postProcessing/patch/patchAverage/patchAverage.C&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;patchAverage&lt;/code&gt;&lt;/a&gt; to obtain the average pressure on an object that consists of multiple patches (due to different boundary conditions), however &lt;code&gt;patchAverage&lt;/code&gt; only averages over single patches. As long as the constituent patches aren't shared with other objects, I could simply take the area-weighted average of each patch's average.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, due to a dynamic geometry where the individual patches change area in time, this would become quite messy. So instead, my idea is to use &lt;a href=&quot;https://github.com/OpenFOAM/OpenFOAM-2.1.x/blob/master/applications/utilities/mesh/manipulation/topoSet/topoSetDict&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;topoSet&lt;/code&gt;&lt;/a&gt; to define a faceSet and turn that into a new object patch via &lt;a href=&quot;https://github.com/OpenFOAM/OpenFOAM-2.1.x/blob/master/applications/utilities/mesh/manipulation/createPatch/createPatchDict&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;createPatch&lt;/code&gt;&lt;/a&gt; (replacing the old ones, which are no longer needed after simulating).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would &lt;code&gt;createPatch&lt;/code&gt; correctly map the simulated fields to the new patch or would I end up with values being mapped to the wrong vertices? Or is there another way to achieve this? (I also considered using &lt;a href=&quot;https://github.com/OpenFOAM/OpenFOAM-2.1.x/blob/master/applications/utilities/postProcessing/sampling/sample/sampleDict&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;sample&lt;/code&gt;&lt;/a&gt;, which can obtain the values for multiple surfaces at once, but then I'd have to manually average, and I'm not sure this way I'd weigh the patch faces correctly.)&lt;/p&gt;&#xA;" OwnerUserId="89" LastEditorUserId="1552" LastEditDate="2012-07-24T19:28:30.210" LastActivityDate="2012-07-24T19:28:30.210" Title="Can post-processing use topoSet and createPatch without screwing up the results?" Tags="|openfoam|data-analysis|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2910" PostTypeId="1" CreationDate="2012-07-24T13:44:11.337" Score="6" ViewCount="250" Body="&lt;p&gt;I am investigating the accuracy and stability properties of a non-symmetric discretization of a Poisson problem. The problem originates from a ghost fluid discretization of the projection step of a fractional step Navier-Stokes solver. In this type of approach, grid points away from the geometry surface are discretized using standard 5-point stencils. For points adjacent to the geometry, a boundary condition preserving extrapolation is used to fill in a set of ghost points right outside the geometry. These points are then used to fill in the 5 point stencil of the points immediately interior to the geometry surface. The act of extrapolating to the 'ghost points' introduces a non-symmetric character to the matrix, and I'm pretty much trying to figure out how much doing this ruins the matrix properties. One particular example of such an approach is shown in the paper by Mittal et al.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mittal R, Dong H, Bozkurttas M, Najjar FM, Vargas A, von Loebbecke A., A VERSATILE SHARP INTERFACE IMMERSED BOUNDARY METHOD FOR INCOMPRESSIBLE FLOWS WITH COMPLEX BOUNDARIES. J Comput Phys. 2008;227(10):4825-4852.&lt;/p&gt;&#xA;" OwnerUserId="1453" LastEditorUserId="1453" LastEditDate="2012-07-24T20:02:10.940" LastActivityDate="2012-08-03T08:04:40.573" Title="How far is a non-symmetric discretization of an elliptic operator from the continuous operator itself?" Tags="|discretization|poisson|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2911" PostTypeId="1" AcceptedAnswerId="2913" CreationDate="2012-07-24T14:18:45.557" Score="10" ViewCount="1859" Body="&lt;p&gt;I'd like to know if there is an algorithm that given a set o points and an angle computes the convex-hull if the angle is $\alpha = 0$ and given an $\alpha &amp;gt; 0$ computes an envelope that follows more closely the &quot;perimeter&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xSA15.jpg&quot; alt=&quot;Illustration of $\alpha$ size effect&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if there is a definition of a non intersecting perimeter of a set of points, in this case the resulting polygon when $\alpha$ is big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another view of the problem can be to find an algorithm that can be parametrized to find for $\alpha = 0$ the minimum perimeter solution (convex-hull) and for $\alpha = 1$ (normalized) the minimum area polyline enclosing all the points.&lt;/p&gt;&#xA;" OwnerUserId="1812" LastEditorUserId="1812" LastEditDate="2012-07-25T10:53:49.010" LastActivityDate="2021-11-10T16:20:20.737" Title="Is there an algorithm to find an almost-convex hull given a tolerance angle?" Tags="|algorithms|computational-geometry|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2913" PostTypeId="2" ParentId="2911" CreationDate="2012-07-24T18:07:39.640" Score="4" Body="&lt;p&gt;You might investigate the so-called &lt;em&gt;alpha-hull&lt;/em&gt;, for example: &#xA;&lt;a href=&quot;http://cran.r-project.org/web/packages/alphahull/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CRAN package&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Alpha_shape&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia on alpha shapes&lt;/a&gt;:&#xA;&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/psJbH.gif&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;sub&gt;[Image from &lt;a href=&quot;http://cgm.cs.mcgill.ca/~athens/cs644/2002projects/PavelDimitrov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;.]&lt;/sub&gt;&#xA;&lt;br /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alpha-hull has very nice geometric properties, and has been heavily&#xA;studied, but it still might not serve your purposes.&lt;/p&gt;&#xA;" OwnerUserId="673" LastActivityDate="2012-07-24T18:07:39.640" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2914" PostTypeId="2" ParentId="2213" CreationDate="2012-07-24T20:07:57.597" Score="0" Body="&lt;p&gt;It turned out my original question was somewhat orthogonal to what I actually needed to do. Since starting with a list (E,V) and mapping it to a particular lattice is &quot;hard&quot;, we opted to instead start off with the lattice type assumed. In this case, we specify graphs using coordinates on the lattice, so a 4-cycle on the square lattice would be: (0,0), (0,1), (1,0), (1,1). Using this representation, it's easy to add new sites to a graph (there are four possible positions, check if a site's already there). This also prevents us from generating disconnected graphs. We define two vertices as connected by an edge if they are lattice nearest-neighbours. Since we have several different meanings for &quot;distinct&quot; graphs (depending on the Hamiltonian we are using, it could be distinct under the dihedral group, distinct under x-reflection but not y-reflection, different adjacency lists, etc), we implemented several functions to check for different kinds of isomorphism, depending on which Hamiltonian we are using. We canonise distinct graphs using a simple lexicographic ordering scheme, which allows us to eliminate a huge number of similar graphs. To check if one graph is a subgraph of another, we check if its lattice coordinates can all be shifted by a constant vector so that the resulting list's elements can all be found in the lattice coordinates of the larger graph. Although we haven't implemented any parallelization yet, it seems likely that checking each graph for subgraphs is parallelizable (since a graph with &lt;code&gt;n + 1&lt;/code&gt; vertices can only have graphs of &lt;code&gt;n&lt;/code&gt; vertices or below for subgraphs, if all the graphs are non-isomorphic), as is attempting to add sites in all possible positions for a graph and canonising the distinct graphs. &lt;/p&gt;&#xA;" OwnerUserId="1193" LastActivityDate="2012-07-24T20:07:57.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2915" PostTypeId="1" AcceptedAnswerId="2920" CreationDate="2012-07-25T08:22:46.507" Score="6" ViewCount="4989" Body="&lt;p&gt;I want to have an estimation, that my solution has an error, let's say less than 1e-8.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, I stop the Gauss-Seidel algorithm, when the residual is &quot;small enough&quot; and this is already the problem. How do I know when the residual is small enough, because even when the residual is small, the solution may still have too much error.&#xA;So this is no good method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you use as stopping criterion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On another website (&lt;a href=&quot;http://www.math-linux.com/spip.php?article48&quot; rel=&quot;nofollow&quot;&gt;math-linux.com&lt;/a&gt;) I found a stopping criterion:&#xA;$$&#xA;\|r\|/\|b\|\leq \epsilon&#xA;$$&#xA;But again, what theory is behind that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, by the way, is the code I used in my last project, just for information, how I did it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void relax(double epsilon, vector&amp;lt;double&amp;gt; &amp;amp;x, SparseMatrix &amp;amp;A, const vector&amp;lt;double&amp;gt; &amp;amp;f) {&#xA;    int maxIter = 100;&#xA;    int iter = 0;&#xA;    double residual = 1.0;&#xA;    double minResidual 0.000001; //I also tried 1e-14;&#xA;&#xA;    while (iter &amp;lt; maxIter &amp;amp;&amp;amp; residual &amp;gt;= minResidual) {&#xA;        for (int i = 0; i &amp;lt; A.dim; ++i) {&#xA;            double ls = A.lineScalar(i, x);&#xA;            x[i] = (1.0/A(i,i)) * (f[i] - ls);&#xA;        }&#xA;        vector&amp;lt;double&amp;gt; temp = A.multiply(x);&#xA;        residual = L2(temp - f);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1738" LastEditorUserId="1738" LastEditDate="2012-07-26T17:38:27.593" LastActivityDate="2012-07-26T17:38:27.593" Title="When to stop Gauss-Seidel-iterations?" Tags="|linear-solver|convergence|error-estimation|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2916" PostTypeId="2" ParentId="882" CreationDate="2012-07-25T08:41:56.893" Score="1" Body="&lt;p&gt;You can separate the problem along the direction with Dirichlet conditions and then solve the 2D periodic problems. Exactly yours combination of boundary conditions is covered by &lt;a href=&quot;http://dx.doi.org/10.1016/0021-9991%2877%2990001-8&quot; rel=&quot;nofollow&quot;&gt;Wilhelmson, Ericksen, JCP 1976&lt;/a&gt; and it is easy to implement. You could also use FISHPACK, but it is old and buggy. (I'm working on a small solver for similar cases, but it is not ready for release yet and it will not be a big MPI thing, just for shared memory machines.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Actually, my code is now an MPI thing and it solves this problem too: &lt;a href=&quot;https://github.com/LadaF/PoisFFT&quot; rel=&quot;nofollow&quot;&gt;https://github.com/LadaF/PoisFFT&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1244" LastEditorUserId="1244" LastEditDate="2015-05-12T16:15:32.800" LastActivityDate="2015-05-12T16:15:32.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2917" PostTypeId="1" AcceptedAnswerId="2921" CreationDate="2012-07-25T09:13:38.973" Score="11" ViewCount="3464" Body="&lt;p&gt;Suppose $A$ is a real symmetric matrix and its eigenvalue decomposition $V \Lambda V^T$ is given. It is easy to see what happens with the eigenvalues of the sum $A + cI$ where $c$ is a scalar constant (see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1699/spectral-decomposition-with-eigenvalue-shift&quot;&gt;this question&lt;/a&gt;). Can we draw any conclusion in the general case $A + D$ where $D$ is an arbitrary diagonal matrix? Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ivan&lt;/p&gt;&#xA;" OwnerUserId="1815" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-05-06T03:27:25.440" Title="Eigenvalue decomposition of the sum: A (symmetric) + D (diagonal)" Tags="|linear-algebra|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2920" PostTypeId="2" ParentId="2915" CreationDate="2012-07-25T10:25:43.727" Score="7" Body="&lt;p&gt;One cannot conclude from a residual how accurate the solution is. Between the best and the worst case in norm, there is a factor of exactly the condition number. More precisely, if the residual norm is r and the error norm is e then&#xA;$\|A\|^{-1}\le e/r \le \|A^{-1}\|$, and both bounds are attainable. Taking the quotient of the bounds proves the claim.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The order of magnitude of the norm of the inverse can often be estimated from a few steps of the Lanczos/Arnoldi method. Thus one can apply the bounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, generally, model equations are not really exact anyway, and getting the residual smaller than the accuracy with which uncertain data determine the residual is considered the scientifically corrrect procedure. &#xA;Depending on the data, this may be far earlier that stagnation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The theory behind this is called backward error analysis. For example, if the only source of uncertainty are measurement errors in the right hand side of 1e-4 in some norm, and your residuum norm is smaller than 1e-4, it cannot be distinguished from the residuum the exact solution of your solved problem would have when used as approximation in a problem with a different rhs of the same accuracy.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-07-25T14:27:55.670" LastActivityDate="2012-07-25T14:27:55.670" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2921" PostTypeId="2" ParentId="2917" CreationDate="2012-07-25T13:15:46.273" Score="3" Body="&lt;p&gt;One can say very little, except for generalities such as that the eigenvalues change continuously with the entries of $D$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see by symbolic computation in the 2 by 2 case that nothing strong can be expected.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-25T13:15:46.273" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2922" PostTypeId="2" ParentId="2895" CreationDate="2012-07-25T14:03:26.763" Score="0" Body="&lt;p&gt;Just a further optimization to get rid of using an upper bound of the receive buffer (as described previously):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are using MPI_Iprobe, that also returns an MPI_Status of your message. Why not use MPI_Get_count() with this status and the MPI_datatype to query the size of the message to receive and allocate a buffer of the exact size?&lt;/p&gt;&#xA;" OwnerUserId="1605" LastActivityDate="2012-07-25T14:03:26.763" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2924" PostTypeId="1" CreationDate="2012-07-25T17:06:35.060" Score="2" ViewCount="88" Body="&lt;p&gt;In the worst case complexity analysis of all the polynomial algorithms in linear programming such as ellipsoid method and interior point method, there is an assumption that the input data must be integer. Using this assumption, then they bound the bitlength of input data by $L$. Now, the question is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q: How can this assumption be met for a linear programming with rational data. In other words, does there exist a way to convert the LP with rational data to an integer one? To me, it seems as though there is a way since data integrality is an assumption for these algorithms. &lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="917" LastEditDate="2012-07-25T17:23:43.323" LastActivityDate="2012-07-26T14:48:07.853" Title="Rational LP to integer LP" Tags="|optimization|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2926" PostTypeId="1" CreationDate="2012-07-25T17:42:36.780" Score="1" ViewCount="324" Body="&lt;p&gt;For solving a very sparse LP: {min $cx$: s.t.: $A_{m \times n}x=b$ , $x\geq 0$}, which one of the following algorithms is faster?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Logarithmic barrier method &lt;/li&gt;&#xA;&lt;li&gt;Other variants of the interior point&#xA;method &lt;/li&gt;&#xA;&lt;li&gt;Simplex method&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What is the best worst case complexity? In general, which algorithm performs well in practice for sparse matrices?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="123" LastEditDate="2012-07-25T22:01:57.500" LastActivityDate="2012-07-25T22:01:57.500" Title="Fast Algorithms for solving sparse LP problems" Tags="|optimization|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2927" PostTypeId="1" AcceptedAnswerId="2936" CreationDate="2012-07-25T18:36:59.730" Score="4" ViewCount="11971" Body="&lt;p&gt;I am familiar with the Courant-Friedrich-Lewy Condition in as far as it applies to the stability of explicit finite difference schemes for standard parabolic and hyperbolic PDEs.  However, when solving the Navier-Stokes Equations, the non-linearity of the PDEs make it difficult to derive a CFL condition analytically.  I see that the &quot;Courant number&quot; is often &lt;a href=&quot;http://www.cfd-online.com/Forums/fluent/34736-courant-number.html&quot; rel=&quot;nofollow&quot;&gt; estimated by the CFL condition of the 1st order wave equation or the residence time&lt;/a&gt;.  Is it common practice to do so for all Reynolds numbers?  Are there some Reynolds numbers under which an alternative formula is used to estimate the Courant number?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="9" LastEditDate="2012-07-29T11:37:10.483" LastActivityDate="2012-11-16T18:34:46.600" Title="Estimating the Courant number for the Navier-Stokes Equations under differing Reynolds number regimes" Tags="|fluid-dynamics|stability|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2928" PostTypeId="1" CreationDate="2012-07-25T19:10:43.860" Score="5" ViewCount="1411" Body="&lt;p&gt;I am trying to write an implementation of a color quantization algorithm in order to find an image's dominant colors (say 5) and then find contrasting colors based on the colors found in the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried implementing a k-means algorithm but the result ends up differently every time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you suggest? I need help mostly with the color segmentation. What algorithm/method do you suggest? I can play with picking contrasting colors on my own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am able to iterate through an image's individual rgb values on a per-pixel basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side note: The language I am using is objective C, so there are no libraries (that I know of) that can perform this task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How I used the K-Means algorithm:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;create array of pixels with an R, G, B value.&lt;/li&gt;&#xA;&lt;li&gt;choose 5 random pixels as clusters (c)&lt;/li&gt;&#xA;&lt;li&gt;assign each pixel to the closest c in 3 dimensional space using 3d distance equation&lt;/li&gt;&#xA;&lt;li&gt;change cluster locations (c) to the average of the r, g, and b values of it's assigned points&lt;/li&gt;&#xA;&lt;li&gt;repeat steps 3&amp;amp;4 for a while&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1818" LastEditorUserId="1818" LastEditDate="2012-07-25T21:05:00.470" LastActivityDate="2012-08-28T11:13:40.307" Title="Algorithmically selecting colors that will contrast with their background" Tags="|algorithms|image-processing|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2929" PostTypeId="2" ParentId="2926" CreationDate="2012-07-25T19:26:48.887" Score="2" Body="&lt;p&gt;The speed of all three methods (which are in fact generic patterns for algorithms rather than algorithms) depends a lot on who implemented which details. A sensible implementation of each method can be far better than a poor implementation of any of the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complexity in the sparse case depends a lot on the sparsity pattern, and there are no results on the general sparse case. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="549" LastEditDate="2012-07-25T20:01:10.220" LastActivityDate="2012-07-25T20:01:10.220" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2930" PostTypeId="1" AcceptedAnswerId="2933" CreationDate="2012-07-25T20:05:51.367" Score="5" ViewCount="762" Body="&lt;p&gt;I work on non-graded quadtree grids where the entire grid is a hierarchy of cells specified using a quadtree data structure, where, in general, there is no constraint regarding the relative size of neighboring cells (or any cells for that matter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://dl.dropbox.com/u/58836692/mesh.png&quot; alt=&quot;Valid XHTML&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My library is mainly based on finite difference schemes that sample the unknowns at each cell vertex, i.e. the &quot;nodes&quot;. Recently, however, I have found that I need to switch to finite volume schemes (for mass conservation properties), for certain type of equations, where I sample the values at the cell centers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I still like to reuse many of functions/class I've written for vertex-based values so I need to be able to interpolate data back and forth between the cells and vertices. Going from vertices to centers is easy; one could use a simple averaging operator ... let's call this $V2C$ operator. What I need now, is to construct the reverse operator, $C2V$, with the property that $V2C\:(C2V) \equiv I_{c}$ and $C2V\:(V2C) \equiv I_{v}$ where $I_c$ and $I_v$ are identity operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked looked at &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2296/how-to-interpolate-multipoint-data-to-the-cell-centres-of-an-unstructured-mesh&quot;&gt;this question&lt;/a&gt;. However the general schemes, either mentioned there or those I can think of myself, e.g. Inverse Distance Weighted (IDW) or Radial Basis Function (RBF) methods do not seem to satisfy this property (at least its not obvious to me if they do; I'm glad to be proven wrong). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the best way to approach this problem? I'd appreciate if anyone could refer me to possible sources? Needles to say, I need something accurate (be able to at least approximate the identity operator, $I$, with a reasonable accuracy) and fast since this has to be done at every time-step.&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-07-25T21:56:07.743" Title="Interpolation schemes to move data between cells and nodes" Tags="|finite-difference|adaptive-mesh-refinement|interpolation|finite-volume|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2931" PostTypeId="2" ParentId="2915" CreationDate="2012-07-25T20:15:16.990" Score="1" Body="&lt;p&gt;Usually, when I'm estimating a solution of a system of linear equations, I save the approximation $\vec x^{n-1}$ and use it to compute $x_{err}=max|x_i^n-x_i^{n-1}|$ over each component i.  This is faster than calculating the residual, since it doesn't require a matrix-vector product.  Although, if your system is ill-conditioned, it still may not give you the correct solution for a given tolerance $\epsilon$.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-07-25T20:15:16.990" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2932" PostTypeId="2" ParentId="2927" CreationDate="2012-07-25T20:24:28.447" Score="6" Body="&lt;p&gt;This estimate is not okay when the viscous terms become large, either at low Reynolds number or in (possibly very stretched) boundary layers. For linear stability, you can use&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min \Big[ C_1 \frac{\Delta x}{u}, C_2  \frac{(\Delta x)^2}{\eta} \Big]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is velocity, $C_1$ depends on the spatial discretization of the transport terms (usually $\frac 1 2$ or $1$ for the simplest spatial and temporal discretizations), $\eta$ is the viscosity, and $C_2 = \frac 1 2$ for the simplest spatial and temporal discretizations. Note that CFL is only a necessary condition for linear stability, it may not be sufficient. Preservation of nonlinear stability may impose more stringent time step constraints, e.g. in the case of strong stability preserving methods.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="3319" LastEditDate="2012-11-16T18:34:46.600" LastActivityDate="2012-11-16T18:34:46.600" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2933" PostTypeId="2" ParentId="2930" CreationDate="2012-07-25T21:35:08.627" Score="6" Body="&lt;p&gt;There are no such operators so that $V2C(C2V)=I$ and $C2V(V2C)=I$ simultaneously. This is already easy to see if you only have a 1d situation with 2 vertices and 1 cell. In that case, $C2V$ is a $2 \times 1$ matrix, and $V2C$ is a $1 \times 2$ matrix. It is easy to verify that you can't find entries for these two matrices that satisfy the criteria you ask. The situation does not become different if you go from a structured 1d mesh to a hierarchic 2d mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, the $C2V$ operation is a smoothing operation because you start with something that is piecewise constant on each cell and you find yourself vertex values that are something like averages of the surrounding cells. On the other hand, the $V2C$ operation computes for each cell a value that is something like the average of the vertex values surrounding it. In other words, $V2C$ is a smoothing operation as well! Clearly, then, $C2V(V2C)$ and $V2C(C2V)$ are operations that start and end with a cell-based field (or, in the second case, start and end with a vertex-based field) and has as output a smoothed version of the input in both cases. It is simply not possible to have an operation that reconstructs in the second step the input to an averaging operation that has been done in the first step.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-07-25T21:35:08.627" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2934" PostTypeId="2" ParentId="2924" CreationDate="2012-07-25T21:43:45.450" Score="5" Body="&lt;p&gt;Why not multiply the data (i.e., the matrix $A$ and vector $b$ in the constraint $Ax \ge b$ and the vector $c$ in the objective function $c^Tx$) by the greatest common denominator of all entries? If you do so, you end up with a problem that has only integer constraints. Of course, you could do that for each individual constraint separately if you want to use smaller multipliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all you want to do is estimate the bit size you need, then the bits you need to represent a fraction $\alpha/\beta$ in a matrix or vector entry is simply $bits(\alpha/\beta)=bits(\alpha)+bits(\beta^*)$ where $\beta^*\ge \beta$ is the smallest common multiplier, and this yields the same complexity bound for the linear program if you adjust the bit count for rational numbers. Of course, this yields the very same bound as you get if you just scale the entire problem by the largest common denominator $\beta^*$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-07-26T14:48:07.853" LastActivityDate="2012-07-26T14:48:07.853" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2935" PostTypeId="2" ParentId="2930" CreationDate="2012-07-25T21:56:07.743" Score="2" Body="&lt;p&gt;You cannot do this as a local operation for the reasons that Wolfgang says. As you say, $V2C$ is just integration. For the &quot;inverse&quot; $C2V$, you can perform an $L^2$ projection. Depending on the boundary conditions, the number of vertices can be smaller or larger than the number of cells, in which case you will have to choose compatible fields. The $L^2$ projection is globally conservative (modulo boundary conditions and mesh refinement), but not locally conservative. The projection $C2V$ is also dense which means you have to solve it iteratively (it is well-conditioned and CG or Kaczmarz converge quickly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a practical matter, it's worth your time to modify your routines to work with cell-centered fields rather than relying on frequent projections between cells and vertices.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-07-25T21:56:07.743" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2936" PostTypeId="2" ParentId="2927" CreationDate="2012-07-25T22:00:47.020" Score="5" Body="&lt;p&gt;Strictly speaking, &lt;em&gt;it doesn't make sense to speak of a Courant number&lt;/em&gt; for the Navier-Stokes equations at finite Reynolds number.  The Courant number is defined in terms of a characteristic velocity, and &lt;em&gt;solutions of parabolic equations (like Navier-Stokes) aren't described by characteristics&lt;/em&gt;.  The CFL condition implies that no explicit solver with finite stencil width can be convergent (for parabolic problems like Navier-Stokes) when the ratio $\Delta t/\Delta x$ is fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonlinearity doesn't cause any problem in defining the Courant number for &lt;strong&gt;hyperbolic PDEs&lt;/strong&gt;, since you can still find the maximum characteristic velocity.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-07-25T22:00:47.020" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2937" PostTypeId="2" ParentId="2152" CreationDate="2012-07-25T23:31:29.107" Score="1" Body="&lt;p&gt;I tend to agree in the following implementation, which I developed on the course of my investigation work, as can be found &lt;a href=&quot;https://sites.google.com/site/jorgecarleitao/home/tools/submit&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://sites.google.com/site/jorgecarleitao/home/tools/logbook&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/7993082/logbook-for-scientific-simulations&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To pass variables to the program and be able to change then, I use the paradigm of using a bash script where I define&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;export aValue=10&#xA;export bValue=2&#xA;export idName=test&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then use in C/C++&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;char *env_aValue = getenv(&quot;aValue&quot;);&#xA;char *env_bValue = getenv(&quot;bValue&quot;);&#xA;char *env_idName = getenv(&quot;idName&quot;);&#xA;&#xA;aValue = atoi(env_aValue)&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The big advantages of this is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it can be accessed in a global scope,&lt;/li&gt;&#xA;&lt;li&gt;it is portable to sun grid engine (clusters),&lt;/li&gt;&#xA;&lt;li&gt;can be easily changed on the bash script,&lt;/li&gt;&#xA;&lt;li&gt;it is platform independent,&lt;/li&gt;&#xA;&lt;li&gt;the number of parameters can be very large (potentially infinite)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Besides, I always pass an idName, on which every file written by that executable will have a initial identification of it (can be followed by other parameters if you want), and they also receive a export directory=idName, which is created on the bash script, and all the files of that executable are saved on it. This way the results are organized by directories (optional).&lt;/p&gt;&#xA;" OwnerUserId="1819" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-07-25T23:31:29.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2938" PostTypeId="2" ParentId="2064" CreationDate="2012-07-26T03:31:26.283" Score="2" Body="&lt;p&gt;It's a little odd that you mention this in the context of message order.  Quoting you:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As far as I understand, the order in which non-blocking point-to-point&#xA;  MPI messages (Isend and Irecv) are received is consistent with the&#xA;  order in which they are sent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's worth pointing out here that MPI only guarantees that &lt;em&gt;matching&lt;/em&gt; messages between processes will be received in the order they were sent.  You really don't want this type of ordering to change, because it makes your code more understandable and takes a huge burden off of you as the application programmer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you sent messages with different tags, that changes the matching criteria, and you could easily receive the second before the first.  See the second example in the &lt;a href=&quot;http://www.mpi-forum.org/docs/mpi22-report/node54.htm#Node54&quot; rel=&quot;nofollow&quot;&gt;relevant portion of the standard&lt;/a&gt; for details.  I &lt;em&gt;hope&lt;/em&gt; that if you have two pieces of your code sending simultaneously that you're already separating the coarse and fine messages using tags, and not trying to implement some protocol of your own on top of message ordering.  This is second nature to most MPI programmers I know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, assuming you're doing that, you're probably concerned that high-volume fine-grained messages are going to clog your network when you want to send the coarse ones.  My general advice on this is that if it's not a performance problem that you can actually measure right now, then you really shouldn't bother addressing it yet.  You seem to confirm that it's not an issue yet in one of the comments above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible solution you &lt;em&gt;might&lt;/em&gt; consider would be to use a non-blocking collective (NBC) like Bcast or Barrier to notify everyone that the coarse phase is done and ready to send its solution.  In all likelihood, the NBC traffic isn't going to get prioritized, but notified processes can at least stop sending gobs of fine solutions until the coarse sends are done.  NBC's &lt;a href=&quot;http://www.cs.uiuc.edu/homes/wgropp/projects/parallel/MPI/mpi-errata/discuss/nbcoll/nbcoll.pdf&quot; rel=&quot;nofollow&quot;&gt;will be in MPI-3&lt;/a&gt; or you could try using &lt;a href=&quot;http://www.unixer.de/research/nbcoll/libnbc/&quot; rel=&quot;nofollow&quot;&gt;libNBC&lt;/a&gt; if you can't wait that long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, though, this seems like a lot of work for something that doesn't sound like it's a performance problem yet.&lt;/p&gt;&#xA;" OwnerUserId="1721" LastActivityDate="2012-07-26T03:31:26.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2939" PostTypeId="2" ParentId="2152" CreationDate="2012-07-26T07:22:40.177" Score="0" Body="&lt;p&gt;Have you thought about using a MySQL database? I've never done it, but I could imagine, that you can query this system very good! Perhaps other systems like MongoDB are better.&#xA;So, this is just an idea.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-07-26T07:22:40.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2941" PostTypeId="1" AcceptedAnswerId="2943" CreationDate="2012-07-26T09:09:57.927" Score="0" ViewCount="76" Body="&lt;p&gt;Given $d$-dimensional coordinates residing in a matrix $X\in\mathbb{R}^{n\times d}$, the Euclidean distance between items $i$ and $j$ is denoted as $g_{ij}$. Let $c\in\mathbb{R}^d$ denote the centroid of the configuration $X$. Is it possible to establish a relation between $$\sum_{i,\, j} g_{ij}^2,$$ and $$\sum_{i=1}^n g_{ic}^2,$$where $g^2$ is a squared Euclidean distance? My solution shows that the first is $2n$ times the second. Is this correct?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-07-26T10:38:52.097" LastActivityDate="2012-07-26T15:00:25.340" Title="Relation to all-pairs Euclidean distances" Tags="|linear-algebra|matrix|high-dimensional|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2943" PostTypeId="2" ParentId="2941" CreationDate="2012-07-26T15:00:25.340" Score="1" Body="&lt;p&gt;Yes. Assume wlog that the centroid is at zero. Then $\sum_i x_i=0$, whence&#xA;$\sum_{i,j} g_{ij}^2 = \sum_{i,j}\|x_i-x_j\|^2=\sum_{i,j}\|x_i\|^2-2\sum_{i,j}x_i^Tx_j+\sum_{i,j}\|x_j\|^2$. This simplified to &#xA;$n\sum_i|x_i\|^2-0+n\sum_j\|x_i\|^2=2n\sum_{i}\|x_i\|^2$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-26T15:00:25.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2944" PostTypeId="1" CreationDate="2012-07-26T19:14:23.357" Score="5" ViewCount="598" Body="&lt;p&gt;Suppose I have a matrix $A$  of size&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_1 \times n_2 \times n_3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I have another matrix $B$ of size $n_1 \times n_2 \times n_3 \times N$ where $N&amp;lt;n_3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to create the following matrix $C = [A(m,l,B(m,l,k,:))]_{1 \le m \le n_1, 1 \le l \le n_2, 1 \le k \le n_3}$.&#xA;What is the most efficient way of doing this without having to create loops?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B essentially tells me which elements to pick out of the 3-rd dimension given an m,l and k. So a loop would go along the lines of: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for m=1:n1; &#xA;    for l=1:n2; &#xA;       for k=1:n3 &#xA;          C(m,l,:,k)=A(m,l,B(m,l,k,:)) &#xA;       end &#xA;    end &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks!!!&lt;/p&gt;&#xA;" OwnerUserId="1824" LastEditorUserId="1824" LastEditDate="2012-07-27T01:51:34.863" LastActivityDate="2012-07-31T03:22:47.937" Title="Manipulating Matrices in Matlab" Tags="|matlab|vectorization|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2945" PostTypeId="2" ParentId="2927" CreationDate="2012-07-26T19:30:46.667" Score="3" Body="&lt;p&gt;If you're looking for a time step stability criterion in the nondimensional compressible Navier--Stokes setting, Guarini derives an estimate of it using scalar stability theory in section 4.2.2 of his thesis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out Guarini, Stephen. Direct numerical simulation of supersonic turbulent boundary layers. PhD thesis, Stanford University, 1998. &lt;a href=&quot;http://proquest.umi.com/pqdweb?did=732826501&amp;amp;Fmt=7&amp;amp;clientId=48776&amp;amp;RQT=309&amp;amp;VName=PQD&quot; rel=&quot;nofollow&quot;&gt;http://proquest.umi.com/pqdweb?did=732826501&amp;amp;Fmt=7&amp;amp;clientId=48776&amp;amp;RQT=309&amp;amp;VName=PQD&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2012-07-26T19:30:46.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2946" PostTypeId="1" AcceptedAnswerId="2948" CreationDate="2012-07-26T21:08:37.387" Score="3" ViewCount="1360" Body="&lt;p&gt;I am seeking for an efficient algorithm to compute a multivariate polynomial of a fixed structure, but different coefficients and evaluation points. The question is the same as &lt;a href=&quot;http://www.mathworks.com/matlabcentral/newsreader/view_thread/264760&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt;, which apparently did not draw much attention. I am working in MATLAB, so &quot;efficient&quot; probably in this situation means to vectorize as much as possible. Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ivan&lt;/p&gt;&#xA;" OwnerUserId="1815" LastActivityDate="2012-07-27T15:42:35.413" Title="Evaluation of multivariate polynomials" Tags="|matlab|polynomials|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2947" PostTypeId="2" ParentId="2944" CreationDate="2012-07-27T03:03:51.050" Score="0" Body="&lt;p&gt;I am assuming that you want to vectorize the operation for efficiency.  While a vectorized assignment may be possible (I personally have tried several things and have failed up to this point) I am going to have to argue that you would be better off not using a vectorized matlab for this application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for my argument is that the clarity of your code will go out the window.  Even with your single line assignment&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C(m,l,:,k)=A(m,l,B(m,l,k,:))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;takes a moment to wrap your head around, and the complex vector notation required to perform the three loops in a single line would completely obscure the assignment process to anyone else reading the code (this includes you a month from now).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you must have the efficiency boost you would see from the vectorization, you could likely get better performance and more clarity by utilizing a short fortran or C function.  The above loops could be implemented with minimal effort, and both languages would likely beat the vectorized matlab in performance while being called from within your matlab script.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this does not directly answer your question, but hopefully it helps in the case that no one here finds an acceptable vectorization for your loops.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-07-27T03:03:51.050" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2948" PostTypeId="2" ParentId="2946" CreationDate="2012-07-27T09:34:07.800" Score="2" Body="&lt;p&gt;Your question is answered in the paper&lt;BR&gt;&#xA;W.C. Rheinboldt, C.K. Mesztenyi and J.M. Fitzgerald,&#xA;On the evaluation of multivariate polynomials and their derivatives,&#xA;BIT 17 (1977), 437-450.&lt;BR&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper contains pseudocode, but was written at a time where vectorization was not yet a common problem. But to simultaneously evaluate pol(coef,x) in Matlab for many values of (coef,x), you can apply the algorithm at is is with pointwise operations applied to vectors coef(i,:) and x(i,:). This vectorizes perfectly.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-27T09:34:07.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2949" PostTypeId="1" CreationDate="2012-07-27T10:10:03.020" Score="1" ViewCount="390" Body="&lt;p&gt;I am working on a paper part of which is the application of validation rules to find how many principal components give us the least generalization error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept goes more or less like this: &quot;Given that the dimension of the model is reduced, we reset our window size to 60 days to avoid overfitting problem. After running multivariate linear regression within the first 20 components using 60 days training set, we find that the first 12 components give the smallest generalization error on the 30 testing days. Finally, we compute the in-sample and out-of-sample residuals.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, the dimension of the full principal components matrix is 483 (days) X 482 (values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation is trivial indeed, yet I am strangling on how to &quot;choose&quot; the correct sub-matrices for any of these cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All suggestions (including matlab code) are welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;Using the hint of &lt;em&gt;cross validation&lt;/em&gt; as Arnold Neumaier mentioned below, and provided that I have already implemented the function that splits the initial dataset into parts, does the following solution solve partially the problem? What should I do next?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:1:10&#xA;  training_set = ex1_data_txt(find(split_assignments(:,i)==0),:);&#xA;  test_set = ex1_data_txt(find(split_assignments(:,i)==1),:);&#xA;&#xA;  % determine weights from the training set&#xA;  phi_train=[training_set(:,1).^(0) training_set(:,1).^(1)];&#xA;  w=pinv(phi_train)*training_set(:,2);&#xA;  phi_test=[test_set(:,1).^(0) test_set(:,1).^(1)];&#xA;  % apply learned weights to the test set and compute MSE&#xA;  MSE(i)=sum((test_set(:,2)-phi_test*w).^2)/size(test_set,1);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1829" LastEditorUserId="1829" LastEditDate="2012-07-27T17:13:34.917" LastActivityDate="2012-07-27T17:13:34.917" Title="How to find the number of principal components that lead to the smallest generalization error?" Tags="|matlab|machine-learning|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2950" PostTypeId="1" AcceptedAnswerId="2954" CreationDate="2012-07-27T11:26:21.437" Score="1" ViewCount="98" Body="&lt;p&gt;For a given centered configuration of points $X\in\mathbb{R}^{n\times 3}$, the covariance matrix is denoted by $S=\frac{1}{n}X^TX$. Recall that the 2D PCA solution is obtained by $Y=X\cdot U$, where $U\in\mathbb{R}^{3\times 2}$ is a matrix of orthonormal eigenvectors of $S$ associated with its 2 dominant eigenvalues $\lambda_1, \lambda_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose that $Y_1\in\mathbb{R}^{n\times 2}$ is obtained by normalizing each axis of $Y$. The normalization factor would be $1/\sqrt{\lambda_i}$, $i\in\{1,2\}$ (please correct me if I'm wrong here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, suppose that $Z$ is obtained by normalizing &#xA; each axis of $X$ to unit length. A 2D PCA solution on $Z$ is denoted by $Y_2$.&#xA; Are $Y_1$ and $Y_2$ the same? I obtain that the results are different, but it hold that the axes of both $Y_1$ and $Y_2$ are orthonormal. If so, what is the distinguishing characteristic of $Y_2$ relative to $Y_1$?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-07-27T19:52:57.923" LastActivityDate="2012-07-27T19:52:57.923" Title="Normalizing axes prior to PCA" Tags="|linear-algebra|matrix|high-dimensional|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2951" PostTypeId="2" ParentId="2949" CreationDate="2012-07-27T12:00:48.507" Score="1" Body="&lt;p&gt;By cross validation. Divide your data set into 5 random batches of about equal size. Apply the estimation procedure 5 times with always one batch left out, and work out for each choice of effective dimension the generalization error by testing it on the remainder. Average these generalization errors .(If you want to be cautios, consider tin place of the averages the averages plus $k$ times the standard deviations, for some $k$ that you feel addresses your level of risk aversion.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives you a curve from which you can read off the answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the curve has a pronounced minimum, it gives you the effective dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, instead, the curve has an essentially flat bottom region, it gives you an indifference interval of effective dimensions, and by Ockahm's razor, you'd choose the smallest value in the indifference interval as the effective dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Afterwards, you can do with this effective dimension the PCA on the full data set, and can expect a similar generalization errro on unseen data. But of course only when the original data were representative of the unseed data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Window sizes, etc. can be chosen along similar lines.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-27T12:00:48.507" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2952" PostTypeId="2" ParentId="2927" CreationDate="2012-07-27T15:15:12.517" Score="7" Body="&lt;p&gt;For the compressible Navier-Stokes equations, a &lt;em&gt;convenient&lt;/em&gt; possibility is to choose the time step by considering the stability criterion of the inviscid (hyperbolic, Euler) and the viscous (parabolic, diffusive) terms &lt;em&gt;independently&lt;/em&gt; from one another:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Euler term you have &quot;something like&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \Delta t_\mathrm{Hyperbolic} \leq \displaystyle\min_{\mathbf{x}_i\in\Omega} \left( \min_{d=1,2,3} \frac{C \Delta x_{i,d}}{\lvert u_{i,d} \rvert + a_i} \right ) \; ,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;depending on your grid, spatial discretization and temporal discretization schemes, where $u_{i,d}$ is the d-th velocity component in the i-th cell/element, $a_i$ is the speed of sound, $\Delta x_i$ is the i-th cell/element size, and $C$ is the Courant number. Remember that in 1D the eigenvalues of the flux Jacobian for the inviscid term are $u + a$, $u - a$, and $a$ (and that you can compute them at every time step for every cell/element).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the diffusive term you will have &quot;something like&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \Delta t_\mathrm{Parabolic} \leq \displaystyle\min_{\mathbf{x}_i\in\Omega} \left( \min_{d=1,2,3} \frac{ \Delta x_{i,d}^2}{\lambda \; \mu} \right ) \; ,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mu$ is your &lt;em&gt;diffusivity&lt;/em&gt; and $\lambda$ is a factor that depends on your spatial and time discretization schemes. Although it seems counterintuitive, explicit time integration schemes for the linear diffusion equation satisfying this criterion are stable for a sufficiently large $\lambda$. Implicit schemes allow you to take a larger $\Delta t_\mathrm{Parabolic}$ from the stability point-of-view, but if it is not small enough your scheme won't be very accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So one could choose $\Delta t = \min ( \Delta t_{\mathrm{Hyperbolic}}, \Delta t_{\mathrm{Parabolic}} )$ and &lt;em&gt;hope&lt;/em&gt; to remain stable for a sufficiently large class of problems if $C$ and $\lambda$ are choosen appropiately (they depend on the spatial and temporal discretization schemes!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, don't forget that: &lt;strong&gt;both terms are actually coupled&lt;/strong&gt;, and as @Jed Brown said above, the CFL condition is a &lt;em&gt;necessary&lt;/em&gt; condition (which is &lt;em&gt;not the same as sufficient&lt;/em&gt;!). If you are using a first order finite volume method, a consistent and monotone numerical flux ensures that your scheme is both &lt;em&gt;stable and convergent&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know from Godunov theorem that no linear monotone scheme of higher than first order accuracy exist. This means that a higher order scheme has to be non-linear in order to be monotone. Total Variation Diminishing (TVD) methods can be constructed through limiting in FV, limiting and filtering in DG, and non-linear stabilization in FE. For higher than second order accuracy, it is worth relaxing the TVD constraint in order to retain the global order of accuracy of the method when discontinuities are present in the solution. Alternatives are Total Variation Bounded (TVB) and Total Variation Bounded in the Mean (TVBM). These methods do not remove oscillations completely, but &lt;em&gt;kind of ensure&lt;/em&gt; stability while maintaining your global order of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you satisfy both the CFL condition and some non-linear stability condition (TVD,TVB,TVBM) your scheme will be &lt;em&gt;very likely&lt;/em&gt; to remain stable. If you do not, you are on  your own, which does not mean that your scheme will be unstable (see for example second-order FV schemes for the Euler equations using &lt;em&gt;unlimited&lt;/em&gt; weighted least-squares reconstruction).&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-08-22T08:06:00.317" LastActivityDate="2012-08-22T08:06:00.317" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2953" PostTypeId="2" ParentId="2946" CreationDate="2012-07-27T15:30:34.290" Score="2" Body="&lt;p&gt;Which kind of multivariate polynomial? For what application (interpolation, curve fitting, computing derivatives...)? Structured Cartesian data points or scattered data? Do you require continuous derivatives? How many derivatives?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing this is hard to give you a &quot;good&quot; accurate answer. A general one would be: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For structured Cartesian data take a look at the matlab Spline Toolbox of deBoor.  You can easily, and very efficiently compute tensor-product B-Splines for as many variables as you want, in as many dimensions as you want, and with as many continuous derivatives as you want (althoug you shouldnt use more than 15-20). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For multidimensional scattered data there are a couple of implementations of radial basis functions for matlab, a google search for &quot;radial basis functions matlab&quot; yields a couple of them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another possibility is to approximate your data using the Finite Element Toolbox (on an unstructured grid). This can be fast, however your derivatives might be discontinuous across elements.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Furthermore, matlab includes very fast implementations for a wide range of polynomial functions and is used frequently for approximation, interpolation and curve fitting so if you need an specific type of polynomial is very likely that someone has already implemented it. For example, a search for &quot;Chebyshev polynomials matlab&quot; yields also a lot of results.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-07-27T15:42:35.413" LastActivityDate="2012-07-27T15:42:35.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2954" PostTypeId="2" ParentId="2950" CreationDate="2012-07-27T16:07:39.603" Score="1" Body="&lt;p&gt;Normalizing the axes of $X$ by the singular values of $Y$ is arbitrary and hence meaningless. ($X$ might just be $Y$ with the ordering of the three reversed.)&#xA;No sensible relation can be expected, neither is the use of the second recipe sound.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-07-27T16:07:39.603" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2955" PostTypeId="2" ParentId="159" CreationDate="2012-07-27T16:08:34.250" Score="2" Body="&lt;p&gt;&quot;Re-use is primarily a social phenomenon.&#xA;I can use someone else's software provided that&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;it works&lt;/li&gt;&#xA;&lt;li&gt;it is comprehensible&lt;/li&gt;&#xA;&lt;li&gt;it can co-exist&lt;/li&gt;&#xA;&lt;li&gt;it is supported (or I'm willing to support it myself, mostly I'm not)&lt;/li&gt;&#xA;&lt;li&gt;it is economical&lt;/li&gt;&#xA;&lt;li&gt;I can find it.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&quot; &amp;mdash; B. Stroustrup, The C++ Programming Language 2 ed. (1991) p. 383.&lt;/p&gt;&#xA;" OwnerDisplayName="user1832" LastActivityDate="2012-07-27T16:08:34.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2956" PostTypeId="2" ParentId="2911" CreationDate="2012-07-27T22:46:14.310" Score="1" Body="&lt;p&gt;This may be too simple to be of interest, but one approach would be to find the convex hull and use the polygonal boundary segment by segment to locate additional points that satisfy the $\alpha$-angle criterion, stopping once a full circuit has been completed without adding further vertices.  More than once pass may be required to reach &quot;convergence&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $\alpha$-angle criterion may be formulated for a given pair of consecutive boundary vertices as lying in a region between a circular arc and its chord = boundary segment.  One might term this a circular segment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We'd want to give some thought to a data structure that would make finding the specified points efficient.  One idea would be to compute a bounding box for each segment and check it against a sorted list of the points.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2012-07-27T22:46:14.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2957" PostTypeId="2" ParentId="1879" CreationDate="2012-07-29T03:54:14.617" Score="3" Body="&lt;p&gt;In&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stodden, V. 2009. “The Legal Framework for Reproducible Scientific Research.” &lt;em&gt;CiSE&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Victoria Stodden recommends publishing the full &quot;research compendium&quot;, and lists the following components on p. 38:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;The research paper&lt;/strong&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;The data&lt;/strong&gt; - including documentation and code for processing the data&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;The experiment&lt;/strong&gt; — all source code; documentation, parameters, settings, and operating system dependencies&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;The results of the experiment&lt;/strong&gt; — figures, data, illustration source files; and documentation and explanation of the processing of the experimental results&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Any auxiliary material&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-07-29T12:25:30.770" LastActivityDate="2012-07-29T12:25:30.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2959" PostTypeId="2" ParentId="1898" CreationDate="2012-07-29T14:06:40.547" Score="0" Body="&lt;p&gt;Just as an addendum to Stefano's answer, this is known in the recent structural mechanics literature as kinematic condensation. By adding the constraint K becomes linearly dependent. You remove one equation by expresing your vector of dependent degrees of freedom $u$ as vector of independent degrees of freedom  $\nu$ using stefano's matrix $\; B$ (sometimes called $A$ in the literature: $\; A^T K A \nu = A^T f \;$).&lt;/p&gt;&#xA;" OwnerUserId="1831" LastActivityDate="2012-07-29T14:06:40.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2960" PostTypeId="2" ParentId="924" CreationDate="2012-07-29T14:17:11.750" Score="0" Body="&lt;p&gt;If you are using an isoparametric formulation, the determinant of the Jacobian of your mapping has to be greater than zero in order to have a bijective mapping from the reference to the real elements. This generalizes to arbitrary dimensions, and this is &lt;em&gt;the&lt;/em&gt; regularity criterion (if its not fulfilled its game over). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other regularity criterion serve multiple purposes. You can have elements in which the determinant of the Jacobian is close to, but not, zero, which might lead to large errors. You might want to have a regular distribution of elements to achieve an even distribution of your error in &quot;single-scale&quot; problems, ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two possibilities are: using the diameter of the lagest n-sphere that fits inside your n-th dimensional element and trying to make this value regular across your mesh, and, restricting the minimum/maximum angles between your element faces, which are (n-1)-dimensional manifolds. Both criterion generalize to arbitrary dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a discussion about this in Bathe's Finite Element Procedures. I don´t remember exactly if this was also covered in Frey's and George's Mesh Generation or in Lisekin's Mesh Generation Methods.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-07-31T12:16:46.430" LastActivityDate="2012-07-31T12:16:46.430" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2961" PostTypeId="2" ParentId="2589" CreationDate="2012-07-29T14:37:50.990" Score="4" Body="&lt;p&gt;From the comments above, I understand that you want to avoid to copy the vector when you add more cells. The easiest approach is to reserve space for the maximum number of cells that you might want to have: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;std::vector&amp;lt;YourCellType&amp;gt; myVectorOfCells;&#xA;vectorOfCells.reserve(maxNoCells);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your vector has allocated space for creating maxNoCells cells but no cells have been created yet. You can now add maxNoCells to your vector, each operation in &lt;code&gt;O(1)&lt;/code&gt; time, without the vector copying itself. However, the C++ Standard requires the push_back operation to be &lt;em&gt;amortized&lt;/em&gt; &lt;code&gt;O(1)&lt;/code&gt; time. If you add more than maxNoCells, the vector will copy itself, reserving space for &lt;em&gt;k-times&lt;/em&gt; as many cells as it previously had (typical implementations choose a k between 1.4 and 2), so that you can keep adding cells to the vector in &lt;code&gt;O(1)&lt;/code&gt; time. This resizing operation is &lt;em&gt;not&lt;/em&gt; &lt;code&gt;O(1)&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An AMR strategy begs for a dynamic data structure, like a linked-list of cells that supports insertion/deletion in &lt;code&gt;O(1)&lt;/code&gt;. A linked-list is, however, much slower than a vector because traversing it requires to randomly jump in memory from cell to cell, generating &lt;em&gt;a lot&lt;/em&gt; of cache misses: in modern processors it is faster to swap $n/2$ elements of a vector in order to insert an element than traverse $n/2$ elements of a linked list. So in practice, vectors are the way to go: they are customizable (you can supply them your own allocator), your cells are aligned in memory, you have random acces in &lt;code&gt;O(1)&lt;/code&gt; time... as far as you reserve memory first, you can add elements in O(1) time too! As Prof. Bangerth mentioned above, hierarchical data structures like trees also use vectors internally for storing their data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I think it is better practice to allocate memory &lt;em&gt;at the beginning&lt;/em&gt; of the simulation. You have to know how many cells you can possibly need in order to check if you have enough memory available. You do not want your simulation in 200.000 processors having to reallocate your data structure or running out of memory and having to swap to disk. In case that happens my opinion is that your program should fail loudly because of an user input error.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2013-07-02T17:30:11.527" LastActivityDate="2013-07-02T17:30:11.527" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2962" PostTypeId="1" AcceptedAnswerId="3012" CreationDate="2012-07-29T16:16:14.330" Score="10" ViewCount="2788" Body="&lt;p&gt;The Heat equation is discretized in space with FV (or FEM), and a semi-discrete equation is obtained (system of ODEs). This approach, known as the &lt;a href=&quot;http://en.wikipedia.org/wiki/Method_of_lines&quot;&gt;method of lines&lt;/a&gt;, allows to easily switch from one temporal discretization to another, without code duplication. In particular, you can reuse any time integrator for ODEs without much effort. This is very convenient because if you decide to change your spatial discretization from FV to say FE, you still get a semi-discrete equation and your time integrators still work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am trying to implement the &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_23.html#Timediscretization&quot;&gt;method of rothe&lt;/a&gt; for the same problem. However, discretizing in time first forces me to rewrite the spatial discretization for every temporal discretization scheme I might want to use. This eliminates the reuse of time integrators that I previously had, and makes it very complicated to write modular software that can discretize a PDE using both the method of lines or the method of Rothe. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way of implementing both approaches, without code duplication? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In convection dominated problems, the FE discretization needs stabilization both in time and space, making the method of Rothe the &quot;best&quot; choice. However, this is not the case for FV/DG methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the method of lines, the PDE is discretized first in space, and then in time. In the method of Rothe, the PDE is discretized first in time, and then in space. The third possibility is to discretize both in space and time simultaneously (also known as space-time  discretizations). A discussion about the method of lines and the method of Rothe can be found &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_23.html#Timediscretization&quot;&gt;here&lt;/a&gt;. For more information the book &quot;Finite Element Methods for Flow Problems&quot; from Donea and Huerta is a good resource.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="6775" LastEditDate="2018-11-13T13:56:09.397" LastActivityDate="2018-11-13T13:56:09.397" Title="PDE discretization with the method of rothe and the method of lines (Modular implementation)" Tags="|finite-element|pde|finite-volume|software|rothe-method|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2963" PostTypeId="1" CreationDate="2012-07-29T19:46:40.050" Score="3" ViewCount="1016" Body="&lt;p&gt;I found R function &lt;a href=&quot;http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=parcor%3aridge.cvunction&quot; rel=&quot;nofollow&quot;&gt;ridge.cv&lt;/a&gt; very useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to implement the equivalent function in MATLAB. As a starting point, I used MATLAB function &lt;code&gt;b0 = ridge(y,X,k,scale)&lt;/code&gt;, however it gives completely different results. Why does this might happen? What value should I set for variable &quot;scale&quot; (1 or 0 and what's their difference?)? And how could I implement it from the scratch in MATLAB?&lt;/p&gt;&#xA;" OwnerUserId="1829" LastActivityDate="2012-08-06T17:12:03.367" Title="How to compute the optimal ridge regression model" Tags="|matlab|regression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2964" PostTypeId="2" ParentId="2944" CreationDate="2012-07-29T22:38:23.973" Score="5" Body="&lt;p&gt;If I got the question right, the problem is given $a_{ijk}$ and $b_{ijkl}$ to form&lt;br&gt;&#xA;$$ c_{ijkl} = a_{ijb_{ijlk}} $$&#xA;or in &lt;code&gt;matlab&lt;/code&gt; notation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C(i,j,k,l) = A(i,j,B(i,j,l,k));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;4 nested loops&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Without colon notation this would require 4 nested loops over variables &lt;code&gt;i,j,k,l&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C(n1,n2,N,n3) = 0.0;          % equivalent to C = zeros(n1,n2,N,n3);&#xA;for i = 1:n1,&#xA;  for j = 1:n2,&#xA;    for k = 1:N,&#xA;      for l = 1:n3,&#xA;        C(i,j,k,l) = A(i,j,B(i,j,l,k));&#xA;      end,&#xA;    end,&#xA;  end,&#xA;end,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note the preallocation of array &lt;code&gt;C&lt;/code&gt;: having variable sized arrays inside a &lt;code&gt;for&lt;/code&gt; loop is normally the main reason for inefficiency. In fact &lt;code&gt;matlab&lt;/code&gt; is pretty good (compared to python) in evaluating for loops, provided that the l.h.s. does not change size during the iterations.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;2 nested loops&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Now let's go for the easy vectorization, eliminating the loops over &lt;code&gt;k&lt;/code&gt; and &lt;code&gt;l&lt;/code&gt;. The main problem here is that &lt;code&gt;k,l&lt;/code&gt; are swapped in &lt;code&gt;C&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; so we have to &lt;code&gt;permute&lt;/code&gt; the array dimensions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C(n1,n2,n3,N) = 0.0;&#xA;for i = 1:n1,&#xA;  for j = 1:n2,&#xA;    C(i,j,:) = A(i,j,B(i,j,:));&#xA;  end,&#xA;end,&#xA;C = permute(C, [1 2 4 3]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Remember that a trailing colon in an array squeezes all remaining dimension into a single one: &lt;code&gt;size(C(i,j,:)) == [1 1 n3*N]&lt;/code&gt; ; correct stride is preserved, so that &lt;code&gt;C(i,j,:)&lt;/code&gt; is OK as a l.h.s.) &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;no loops&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edited&lt;/strong&gt;: &lt;em&gt;corrected error and inserted final solution.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for the hard part of the question: how to eliminate the loops over &lt;code&gt;i&lt;/code&gt; and &lt;code&gt;j&lt;/code&gt;. The idea is simple: find an integer array &lt;code&gt;indx&lt;/code&gt; such that &lt;code&gt;C == A(indx)&lt;/code&gt;: in fact &lt;code&gt;A(indx)&lt;/code&gt; is an array with the same shape of &lt;code&gt;indx&lt;/code&gt; and elements taken from &lt;code&gt;A&lt;/code&gt; regarded as a single column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to construct this array, we have to remember that &lt;code&gt;matlab&lt;/code&gt; follows fortran convention:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;size(A) == [n1 n2 n3]&#xA;A(i,j,k) == A(i + (j-1)*n1 + (k-1)*n1*n2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So here is how to construct this matrix with 2 nested loops:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;indx(n1,n2,n3,N) = 0.0;&#xA;for j=1:n2,&#xA;  for i=1:n1,&#xA;    indx(i,j,:) = i+(j-1)*n1+(B(i,j,:)-1)*n1*n2;&#xA;  end,&#xA;end,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or with two instructions&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;indx = (B-1)*n1*n2;&#xA;indx(:) = indx(:) + repmat(1:n1*n2, n3*N)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The no loops code reads&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;indx = (B-1)*n1*n2;&#xA;indx(:) = indx(:) + repmat(1:n1*n2, 1, n3*N)';&#xA;indx = permute(indx, [1 2 4 3]);&#xA;C = A(indx);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;conclusions&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In my opinion the most efficient way of computing &lt;code&gt;C&lt;/code&gt; should be the &lt;em&gt;2 nested loops&lt;/em&gt; version. In the &lt;em&gt;no loops&lt;/em&gt; version we put a great effort in computing &lt;code&gt;indx&lt;/code&gt; instead of letting &lt;code&gt;matlab&lt;/code&gt; do his own pointer arithmetic, so I would discourage it, unless the same array &lt;code&gt;B&lt;/code&gt; and therefore the same vector &lt;code&gt;indx&lt;/code&gt; can be used multiple times with different &lt;code&gt;A&lt;/code&gt; arrays. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is the case I would still recommend to compute &lt;code&gt;indx&lt;/code&gt; by two nested loops, instead of the no loops version. (Here Godric Seer is right: it is almost impossible to understand the logic behind.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me stress out an important point: if array &lt;code&gt;C&lt;/code&gt; is pre-allocated, two nested loops will be very fast in &lt;code&gt;matlab&lt;/code&gt;, and there will be no substantial advantage in going to a no loops version, unless we have a fast method of computing the &lt;code&gt;indx&lt;/code&gt; vector. (E.g. by using integer arithmetic.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this lengthy reply will be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edited&lt;/strong&gt; &lt;em&gt;previous wrong no loops version:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C(n1,n2,n3,N) = 0.0;&#xA;indx =  ... % to be defined later&#xA;C(:) = A(indx);&#xA;C = permute(C, [1 2 4 3]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here we have to exploit the fact that &lt;code&gt;C(:)&lt;/code&gt; is the multi-dimensional array viewed as a single column, and &lt;code&gt;A(indx)&lt;/code&gt; is a single column of elements picked from &lt;code&gt;A&lt;/code&gt;. We have to know how pointer arithmetic works in &lt;code&gt;matlab&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;size(C) == [n1 n2 n3 N]&#xA;C(i,j,k,l) == C(i + (j-1)*n1 + (k-1)*n1*n2 + (l-1)*n1*n2*n3 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C(i,j,k,l) = A(i,j,B(i,j,k,l))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;can be written as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C(i,j,k,l) = A(i + (j-1)*n1 + (B(i,j,k,l)-1)*n1*n2 );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or even&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C(i,j,:) = A(i + (j-1)*n1 + (B(i,j,:)-1)*n1*n2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;therefore we can compute &lt;code&gt;indx&lt;/code&gt; as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;indx = [];&#xA;for j=1:n2,&#xA;  for i=1:n1,&#xA;    indx = [ indx, reshape( i+(j-1)*n1+(B(i,j,:)-1)*n1*n2, 1, n3*N ) ];&#xA;  end,&#xA;end,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With some effort it should be possible to compute &lt;code&gt;indx&lt;/code&gt; without loops: however I have no time to derive it now. By the way I fear that it would be neither elegant nor efficient, but who knows for sure!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;/strike&gt;&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-07-30T14:09:53.977" LastActivityDate="2012-07-30T14:09:53.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2965" PostTypeId="2" ParentId="2928" CreationDate="2012-07-30T17:07:08.950" Score="5" Body="&lt;p&gt;RGB distance is not at all close to &lt;a href=&quot;http://en.wikipedia.org/wiki/Color_difference&quot; rel=&quot;nofollow noreferrer&quot;&gt;Color difference&lt;/a&gt; (but don't be put off by the formulas there).&lt;br&gt;&#xA;Basically you want to transform your RGB triples to &lt;a href=&quot;http://en.wikipedia.org/wiki/Lab_color_space&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lab color space&lt;/a&gt;, and use distances there:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def rgb_deltae76( rgb, rgb2 ):&#xA;    return distance( rgb_lab(rgb), rgb_lab(rgb2) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;rgb_lab&lt;/code&gt; is a couple of pages of Python, somebody must have code in objective C;&lt;br&gt;&#xA;try &lt;a href=&quot;https://stackoverflow.com/questions/tagged/colors&quot;&gt;https://stackoverflow.com/questions/tagged/colors&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added: &lt;a href=&quot;http://www.compuphase.com/cmetric.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;this note&lt;/a&gt; gives the approximation &#xA;$\sqrt { 3 (R - R')^2 + 4 (G - G')^2 + 2 (B - B')^2 }$&lt;br&gt;&#xA;and another better one.&lt;br&gt;&#xA;See also &lt;a href=&quot;http://en.wikipedia.org/wiki/Munsell_color_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Munsell color system&lt;/a&gt;&#xA;and &lt;a href=&quot;http://www.triplecode.com/munsell&quot; rel=&quot;nofollow noreferrer&quot;&gt;triplecode.com/munsell&lt;/a&gt;:&lt;br&gt;&#xA;&quot;The Munsell system is different because it is based on how people &lt;em&gt;perceive&lt;/em&gt; colors.&quot;&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-08-28T11:13:40.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2966" PostTypeId="2" ParentId="2963" CreationDate="2012-07-30T17:44:58.660" Score="1" Body="&lt;p&gt;One form of ridge regression is the problem:&lt;br&gt;&#xA;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; minimize |Ax - b|, and keep x near a given point x0 (often 0).&lt;br&gt;&#xA;If we cast this as:&lt;br&gt;&#xA;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; minimize |Ax - b|^2 + w^2 |x - x0|^2 ,&lt;br&gt;&#xA;we can solve it by running least squares on these extended inputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ A ]    [ b ]&#xA;[ w I ]  [ w x0 ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here w is a weight factor that balances minimizing |Ax - b|, and keeping x near x0.&lt;br&gt;&#xA;How should one choose it if one has no idea ?&#xA;A rule of thumb is to make |Ax - b| and |x - x0| roughly equal -- print those values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added:&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Least_squares#Weighted_least_squares&quot; rel=&quot;nofollow&quot;&gt;Weighted least squares&lt;/a&gt;,&#xA;with different weights w$_i$, is a powerful method.&lt;br&gt;&#xA;For example, w$_i \sim \frac{1}{\sqrt |\text{x}_i|}$&#xA;pushes small x$_i$ towards 0 (think iPhone fingers), &quot;sparsifies&quot; &amp;mdash;&lt;br&gt;&#xA;a poor man's approach to L1-norm regularization;&#xA;see &lt;a href=&quot;http://en.wikipedia.org/wiki/Lasso_%28statistics%29#LASSO_method&quot; rel=&quot;nofollow&quot;&gt;LASSO&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not Matlab, but hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2012-08-06T17:12:03.367" LastActivityDate="2012-08-06T17:12:03.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2967" PostTypeId="2" ParentId="2944" CreationDate="2012-07-31T03:22:47.937" Score="0" Body="&lt;p&gt;Thanks for all your response. I actually worked it out over the weekend but haven't had the chance to check this forum (To be frank, I'm quite surprised by how helpful the people are in this community. I actually just assumed that my questioned would be shrugged off given that no answers were given that night.) In any case, I used the following command, which I think is the same as the first answer, but I haven't had the chance to thoroughly go over the generously long response. Again, MANY thanks to the two responders for your time and insights. I'll read over your suggestions and incorporate them in my code. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A(repmat(reshape(1:n1*n2,n1,n2),[1,1,n3,N])+n1*n2*(B-1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1844" LastActivityDate="2012-07-31T03:22:47.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2968" PostTypeId="2" ParentId="2963" CreationDate="2012-07-31T05:52:00.337" Score="1" Body="&lt;p&gt;You can get the source and documentation for the &lt;a href=&quot;http://cran.r-project.org/web/packages/parcor/index.html&quot; rel=&quot;nofollow&quot;&gt;parcor package&lt;/a&gt; which includes the  ridge.cv function on &lt;a href=&quot;http://cran.r-project.org/web/packages/parcor/index.html&quot; rel=&quot;nofollow&quot;&gt;CRAN&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can implement the function in matlab by looking at the source. Alternatively, I think it should be possible to call R from Matlab.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-07-31T05:52:00.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2969" PostTypeId="1" AcceptedAnswerId="2970" CreationDate="2012-07-31T10:44:12.983" Score="7" ViewCount="686" Body="&lt;p&gt;I am dealing with optimization of functions for which I do not have derivatives available, and the optimization is not constrained.  I am searching for a high quality GNU Public License-compatible optimization library compatible with the C++ language. So far I have managed to find &lt;a href=&quot;http://acts.nersc.gov/opt++/&quot; rel=&quot;nofollow&quot;&gt;OPT++&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone recommend this or some other library? &lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="9" LastEditDate="2012-08-03T11:59:38.790" LastActivityDate="2012-08-03T11:59:38.790" Title="Recommendations for a usable, fast GPL-compatible derivative-free numerical optimization library that can be interfaced to C++" Tags="|optimization|reference-request|libraries|c++|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2970" PostTypeId="2" ParentId="2969" CreationDate="2012-07-31T11:23:27.333" Score="11" Body="&lt;p&gt;&lt;a href=&quot;https://software.sandia.gov/opt++/&quot;&gt;OPT++&lt;/a&gt; is used internally by &lt;a href=&quot;http://dakota.sandia.gov/index.html&quot;&gt;Dakota&lt;/a&gt; (Sandia), which is much more than an optimization library and is released under the LGPL. You should also take a look at &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/tao/&quot;&gt;TAO&lt;/a&gt; (ANL), released under a BSD-like license. An introduction to both OPT++ and TAO can be found &lt;a href=&quot;http://acts.nersc.gov/events/SIAMCSE05/docs/OPT++TAO.pdf&quot;&gt;here&lt;/a&gt;. Other alternatives are &lt;a href=&quot;http://trilinos.sandia.gov/packages/moocho/&quot;&gt;MOOCHO&lt;/a&gt;, &lt;a href=&quot;http://www.gerad.ca/NOMAD/Project/Home.html&quot;&gt;NOMAD&lt;/a&gt;, and &lt;a href=&quot;https://software.sandia.gov/trac/hopspack/&quot;&gt;HOPSPACK&lt;/a&gt;, which are, as far as I know, also LGPL licensed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both Sandia and ANL offer automatic differentiation packages, like &lt;a href=&quot;http://trilinos.sandia.gov/packages/docs/r10.12/packages/sacado/doc/html/index.html&quot;&gt;Sacado&lt;/a&gt; and &lt;a href=&quot;http://web.anl.gov/techtransfer/Software_Shop/ADIFOR/adifor.html&quot;&gt;ADIFOR&lt;/a&gt;/&lt;a href=&quot;http://www.mcs.anl.gov/research/projects/adic/&quot;&gt;ADIC&lt;/a&gt;, that should play well with their general purpose numerical libraries &lt;a href=&quot;http://trilinos.sandia.gov/packages/&quot;&gt;Trilinos&lt;/a&gt;/&lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot;&gt;PETSc&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-07-31T12:06:40.010" LastActivityDate="2012-07-31T12:06:40.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2971" PostTypeId="1" CreationDate="2012-07-31T13:38:39.147" Score="2" ViewCount="591" Body="&lt;p&gt;Recently, I came across an optimization problem with binary decision variables which was constrained with a set of first-order differential equations (resulting from a continuous-time Markov chain model). The objective function is the sum of a partial set of probabilities which are obtained via the set of first-order differential equations. The binary variables are imposing whether a transition is possible from one state to another states according to a specific state transition diagram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As there are solution approaches for solving a first-order differential equation system, a simple solution approach to this problem would be to search on the feasible region of binary decision variables and solve the resulting first-order differential equation system to obtain the objective function value. However, I'm still wondering if there are any exact solution method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is two-folds:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the other (preferably, exact) available solution approaches for tackling such problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any solver (open-source or commercial) for this kind of problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any reference on introduction to modeling and solving these kind of problems is highly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ps&lt;/strong&gt;. I have seen some sources on optimization problems constrained with partial differential equations, which seems to be rooted in control theory. Are there any relation between these two types of problems?&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2013-05-10T15:45:26.280" Title="Binary Integer Programming Problem Subject to a Set of First-Order DEs" Tags="|mixed-integer-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2972" PostTypeId="2" ParentId="2969" CreationDate="2012-07-31T22:13:42.757" Score="5" Body="&lt;p&gt;The library &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt&quot; rel=&quot;nofollow&quot;&gt;NLopt&lt;/a&gt; meets your requirements.  I have used it in the past and I am satisfied with its interface and results.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2012-08-03T04:51:01.897" LastActivityDate="2012-08-03T04:51:01.897" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2973" PostTypeId="2" ParentId="2971" CreationDate="2012-08-01T07:47:38.650" Score="2" Body="&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I'm not an expert in this field; I've just worked with people who do this sort of thing.&lt;/p&gt;&#xA;&lt;h1&gt;Binary decisions are always solved using a glorified guess-and-check algorithm&lt;/h1&gt;&#xA;&lt;p&gt;Setting aside the differential equations for a moment, any integer component of a mixed-integer linear (or nonlinear) programming solver is probably going to use some sort of branch-and-bound or branch-and-cut approach to solving the mixed-integer part of the problem. These approaches are sophisticated ways to do guess-and-check, and there are pathological cases where every possible value of the binary decision variable will be tested.&lt;/p&gt;&#xA;&lt;p&gt;The first approach you mentioned is exact (provided you can solve the differential equations exactly), and solves the problem. In practice, I think as long as you can solve the differential equations numerically to high accuracy, you will probably obtain the correct optimal objective function value.&lt;/p&gt;&#xA;&lt;h1&gt;What are the other available solution approaches for tackling such a problem?&lt;/h1&gt;&#xA;&lt;p&gt;There are a few that I know of:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Discretize, then optimize&lt;/strong&gt;: Differential equations are typically solved numerically (unless you can construct a solution otherwise), and these numerical solutions are constructed from discrete equations. Also known as a collocation approach, you would essentially pick a quadrature rule (for example, something like a Radau method) along with specified time steps for your differential equations. The quadrature rule and the time steps transform the system of differential equations into a system of nonlinear algebraic equations, so your optimization problem becomes a mixed-integer nonlinear program that you then solve. This approach is probably not exact unless the time step is sufficiently small so that the numerical solution of the ODE is accurate. It generally creates a large system of constraints. Larry Biegler's papers are probably good references for this approach. This approach is also referred to as &amp;quot;collocation&amp;quot;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Optimize, then discretize&lt;/strong&gt;: Replace your optimization problem with necessary optimality conditions under constraint qualification. One reformulation uses the Karush-Kuhn-Tucker (KKT) conditions and a Slater point constraint qualification; there are others. Then discretize the resulting KKT conditions and solve. For binary decision variables, I'm not sure how this approach would work, because the KKT conditions don't apply, but if your decision variables were continuous, this approach would make a lot more sense. The KKT conditions are not sufficient for optimality unless the objective function and constraints are convex; generally, ODE-constrained optimization problems do not satisfy these convexity conditions.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Optimize without discretizing&lt;/strong&gt;: Construct a sequence of relaxations and restrictions of your original problem while using a branch-and-bound approach similar to that in nonconvex optimization. Paul Barton publishes papers using this approach; he is one of my PhD advisers. This approach is exact, but requires a lot of mathematics and programming infrastructure.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h1&gt;Relationship to PDE-constrained optimization&lt;/h1&gt;&#xA;&lt;p&gt;The first two approaches I mentioned in the list above are also used in PDE-constrained optimization. The third has not yet been developed for the PDE case. I suspect it may be viable, but I don't know if it is.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-08-01T07:47:38.650" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2974" PostTypeId="1" AcceptedAnswerId="2976" CreationDate="2012-08-01T13:00:45.220" Score="0" ViewCount="218" Body="&lt;p&gt;I have a complex surface (real dimension 2) in $\mathbb{C}^2$ with coordinates $(z,w)$ given explicitely: for any $\xi \in \mathbb{C}$ I know points $w(\xi)$ of intersection of surface with complex line $z = \xi$. I have to draw it's projection on fixed 3d plane. Please help me with algorithm.&lt;/p&gt;&#xA;" OwnerUserId="1180" LastActivityDate="2012-08-02T04:29:59.970" Title="Drawing 3d projection of complex surface" Tags="|algorithms|matlab|complex-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2975" PostTypeId="1" CreationDate="2012-08-01T23:58:30.090" Score="7" ViewCount="1020" Body="&lt;p&gt;I know there are &quot;superfast&quot; $O(n \log^p n)$ algorithms for solving Toeplitz linear systems.  Is it possible to compute all eigenvalues of such a matrix with the same complexity?&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="127" LastEditDate="2013-05-25T01:53:04.040" LastActivityDate="2013-05-25T01:53:04.040" Title="Can all eigenvalues of a Hermitian Toeplitz matrix be computed in $\tilde{O}(n)$ time?" Tags="|linear-algebra|eigenvalues|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2976" PostTypeId="2" ParentId="2974" CreationDate="2012-08-02T04:29:59.970" Score="2" Body="&lt;p&gt;Since you have an explicit parametric representation, the easiest way is direct rasterization of a rectilinear patch.  You don't say what kind of projection you want, so for concreteness say we want an orthographic (orthogonal) projection onto a 3D hyperplane defined by a linear function $A : \mathbb{R}^3 \to \mathbb{R}^4 = \mathbb{C}^2$.  Given a point $y \in \mathbb{R}^4$, the projection is defined by minimizing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|y - Ax|^2 = |y|^2-2y^TAx+x^TA^TAx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;over $x \in \mathbb{R}^3$.  The minimum is attained at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x = (A^T A)^{-1} A^T y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now pick a 2D grid of points $z_i \in \mathbb{C}=\mathbb{R}^2$, map them to 4D with your function, and project them back to 3D with the above formula.  The result can be passed to a suitable plotting function (I don't know Matlab so I don't know which).&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-08-02T04:29:59.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2977" PostTypeId="1" CreationDate="2012-08-02T11:46:47.217" Score="3" ViewCount="692" Body="&lt;p&gt;I'm installing PETSc for complex numbers with the C99 standard. I'd like to have it installed using a higher precision than the default (double, I presume) since that will most likely make the LU preconditioner run better. I've tried several different commands that I could google, but it doesn't work. I've searched the manual but haven't found anything on the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried invoking &lt;code&gt;./configure&lt;/code&gt; with the &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;precision=longdobule&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;precision=__float128&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;download-f-blas-lapack&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;download-f2cblaslapack&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;in all four different combinations. The one time I used &lt;code&gt;download-f2cblaslapack&lt;/code&gt; and &lt;code&gt;precision=__float128&lt;/code&gt; it did configure successfully but at the next stage when I compiled the libraries I got the following error for every library being compiled: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;error: unknown type name ‘PetscScalar’&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What do I have to write to have it configured at a higher precision?&lt;/p&gt;&#xA;" OwnerUserId="1849" LastEditorUserId="20688" LastEditDate="2019-06-22T20:27:42.100" LastActivityDate="2019-06-22T20:27:42.100" Title="How do I configure PETSc to run long double precision or some other precision that is greater than default?" Tags="|petsc|" AnswerCount="0" CommentCount="1" ClosedDate="2019-05-21T20:11:02.130" ContentLicense="CC BY-SA 4.0" />
  <row Id="2978" PostTypeId="1" AcceptedAnswerId="3002" CreationDate="2012-08-02T11:54:22.543" Score="2" ViewCount="189" Body="&lt;p&gt;In order to achieve a 2D representation $X\in\mathbb{R}^{n\times 2}$ of some high-dimensional data residing in $Y\in\mathbb{R}^{n\times k}$, I use PCA:$$X=Y\cdot U,$$where $U\in\mathbb{R}^{k\times 2}$ contains eigenvectors of $Y^TY$ corresponding to its dominant eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in case there are multiple occurrences of, e.g., first dominant eigenvalue, my PCA solution, as defined above, will be inconsistent: it would depend on the actual eigenvalue that is declared as 'the first dominant' by the&#xA;method that I use for the eigendecomposition. What is the recipe to allow for a consistent solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps more important is the following. Namely, PCA provides guarantees on maximal variance along axes; what impact does the above problem have on the solution with maximal variance along axes? Will maximal variance be retained with each solution, regardless on which eigenvector corresponding to dominant first eigenvalue is used?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-08-04T20:23:22.657" Title="Handling inconsistent solutions obtained by PCA" Tags="|linear-algebra|matrix|high-dimensional|machine-learning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2979" PostTypeId="2" ParentId="2910" CreationDate="2012-08-02T12:36:40.363" Score="2" Body="&lt;p&gt;There is a field of research that can be identified by the keywords &quot;mimetic discretizations&quot; and &quot;compatible spatial discretizations&quot; that starts from your same assumption: algebraic equations arising from the discretization of partial differential equations &lt;em&gt;should&lt;/em&gt; mimic the properties of the continuum operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good starting point could be &lt;a href=&quot;http://dx.doi.org/10.1007/0-387-38034-5_5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principles of Mimetic Discretizations of Differential Operators&lt;/a&gt; by Bochev and Hyman, which appeared in &lt;a href=&quot;http://dx.doi.org/10.1007/0-387-38034-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an interesting framework, but I would not focus to much on &quot;matrix properties&quot; &lt;em&gt;per se:&lt;/em&gt;  finally what matters is obtaining a &quot;good solution&quot; (robust, accurate, not to much expensive, ...) and not having a &lt;em&gt;nice&lt;/em&gt; matrix at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you to start asking more narrow questions: e.g. how can I deal efficiently with the unsymmetric terms? Can I trade accuracy with speed by using a symmetric solver?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jed Brown correctly points out that sometimes lack of symmetry is only apparent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me explain this point with a very trivial example. Suppose that $\mathcal{L}u=f$ + b.c (continuous, with $\mathcal{L}$ self adjoint) becomes $Ax=b$ (discrete, with $A=A^T$) where the b.c. are expressed as $u = Bv$ (see &lt;a href=&quot;https://scicomp.stackexchange.com/a/2766/1744&quot;&gt;this answer&lt;/a&gt; for a concrete example, sorry for the shameless self citation). Now by substitution we have $ABv = b$ where $AB$ is &lt;em&gt;not&lt;/em&gt; symmetric, but symmetry can be restored by premultiplication by $B^T$ to obtain $B^TABv = B^Tf$. If matrix $B$ is square (and note that usually it is an identity matrix plus some low rank correction), problem $ABv=f$ is perfectly equivalent to $B^TABv = B^Tf$. If premultiplication by $B^T$ is not feasible, sometimes a symmetric problem can be obtained by augmenting the system, i.e. by introducing extra unknown variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[disclaimer: what follows is only a conjecture since I had no time to work out your problem in detail]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case at hand b.c. are introduced by adding ghost points outside the problem domain: unknown fields at the ghost points are obtained by extrapolation from the domain points, taking into account boundary geometry and b.c. (The algorithm actually goes the other way: from the ghost point an image point in the problem domain is derived, and its value is obtained by interpolation of the surrounding problem points. For this discussion however the approaches are equivalent.) The source of &quot;unsymmetry&quot; is clear, and the situation is slightly different from my trivial example above, so that there is no easy recipe for fixing the symmetry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion is to investigate in these two directions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this a sensible way of introducing the boundary  conditions? If the answer is yes, lack of symmetry is acceptable at the formulation level.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is lack of symmetry acceptable at the computational level? I.e. are you able to solve the unsymmetric problem? If no, try to modify the system for restoring symmetry in a way that does not introduces further approximations (like the trivial example above) or trade accuracy with speed by introducing some extra approx. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-08-03T08:04:40.573" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2980" PostTypeId="2" ParentId="2978" CreationDate="2012-08-02T16:47:39.347" Score="2" Body="&lt;p&gt;If the largest eigenvalue has multiplicity, the PCA solution really depends on your choice of the eigenvectors as 'the first two dominant'. Different choice will give you different representation of the original high-dimensional data, but they all will have the same maximal variance.&lt;/p&gt;&#xA;" OwnerUserId="1614" LastActivityDate="2012-08-02T16:47:39.347" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2981" PostTypeId="2" ParentId="2928" CreationDate="2012-08-02T19:15:02.870" Score="0" Body="&lt;p&gt;Can you transform your RGB colors to some other color notation, like &lt;a href=&quot;http://en.wikipedia.org/wiki/HSL_and_HSV&quot; rel=&quot;nofollow&quot;&gt;HSV&lt;/a&gt; ? The &lt;em&gt;H&lt;/em&gt; would be the color, be it a radial measure ( 0 -360 degrees) or some value. Then use the value that maximizes the distance to the colors you have.&lt;/p&gt;&#xA;" OwnerUserId="730" LastActivityDate="2012-08-02T19:15:02.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2982" PostTypeId="2" ParentId="2975" CreationDate="2012-08-03T08:02:04.377" Score="4" Body="&lt;p&gt;My first idea for computing the eigenvalues of a Hermitian Toeplitz matrix would be to use the fast $O(n \log n)$ matrix multiplication for Toeplitz matrices together with the Lanczos algorithm to get a tridiagonal matrix. The complexity of this procedure is $O(n^2 \log n)$, and it isn't even numerically robust. I vaguely remember that &lt;a href=&quot;http://na.math.kit.edu/marlis/research/publications.php&quot; rel=&quot;nofollow&quot;&gt;Marlis Hochbruck&lt;/a&gt; has written a paper on look-ahead algorithms related to Toeplitz matrices, but I think these were intended to ensure robustness for non-Hermitian Toeplitz systems, not to reduce the complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My conclusion is that already a proof that there exists a numerically robust $\tilde{O}(n^2)$ algorithm for computing all eigenvalues (and eigenvectors) of a Hermitian Toeplitz matrix would be a nice thing to have. I also wonder how useful it would be to able to compute the eigenvalues in $\tilde{O}(n)$ time, if computing the corresponding eigenvectors takes $O(n^2)$ time.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-08-03T08:02:04.377" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2983" PostTypeId="2" ParentId="2975" CreationDate="2012-08-03T08:17:44.780" Score="5" Body="&lt;p&gt;The results in &lt;a href=&quot;http://comet.lehman.cuny.edu/vpan/pdf/165.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Complexity of the Matrix Eigenproblem&lt;/a&gt; (STOC '99, Proceedings of the thirty-first annual ACM symposium on theory of computing, p. 507-516) indicate that the best known algorithms for the Hermitian Toeplitz case are &lt;span class=&quot;math-container&quot;&gt;$\tilde{O}(n^{2})$&lt;/span&gt;, based on Section 1.2. In this section, the eigenproblem is divided into three stages (Section 1.2, pages 3 and 4 of the article):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a) the reduction of the eigenproblem for a given matrix to the one for a matrix in a canonical form (that is, for a Frobenius matrix, for a block triangular or block diagonal matrix with Frobenius diagonal blocks or for a tridiagonal matrix); as a by-product of this stage, the coefficients of the characteristic polynomial &lt;span class=&quot;math-container&quot;&gt;$c_A(x)$&lt;/span&gt; of the original input matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; are output or become available readily, in &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; additional ops&lt;/p&gt;&#xA;&lt;p&gt;b) the approximation of the eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; as the zeros of &lt;span class=&quot;math-container&quot;&gt;$c_A(x)$&lt;/span&gt;, and&lt;/p&gt;&#xA;&lt;p&gt;c) the computational of the eigenspaces of the approximated eigenvalues, by exploiting the available reduction of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to the canonical form.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The lower bound on the arithmetic complexity of stage (a) stated in the article fror general matrices is &lt;span class=&quot;math-container&quot;&gt;$\Omega(M(n))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$M(n)$&lt;/span&gt; is the arithmetic complexity of matrix-matrix multiplication. I know that the arithmetic complexity of LU decomposition is equivalent to the arithmetic complexity of matrix-matrix multiplication, but I don't know if it is true for individual linear solves. I don't believe it's true. Since &lt;span class=&quot;math-container&quot;&gt;$M(n)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\Omega(n^{2})$&lt;/span&gt; (even for Toeplitz matrices, from what I can tell), calculating the eigenvalues (stages (a) and (b)) must be &lt;span class=&quot;math-container&quot;&gt;$\Omega(n^{2})$&lt;/span&gt;, based on the results of the article.&lt;/p&gt;&#xA;&lt;p&gt;In order to obtain a &lt;span class=&quot;math-container&quot;&gt;$\tilde{O}(n)$&lt;/span&gt; algorithm, the limiting step is calculating the characteristic polynomial in &lt;span class=&quot;math-container&quot;&gt;$\tilde{O}(n)$&lt;/span&gt; time. How one does that, I'm not sure, but the output of stage (a) can't be the limiting step because Frobenius, block Frobenius, and tridiagonal matrices can all be specified with &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; data. I don't have access to the source cited along with the &lt;span class=&quot;math-container&quot;&gt;$\Omega(n^{2})$&lt;/span&gt; bound, so I can only speculate that the limiting step there is a sequence of matrix-matrix multiplies associated with elementary matrix operations to reduce the given matrix to one of the listed canonical forms, in which case the &amp;quot;output-limiting step&amp;quot; is in intermediate calculations.&lt;/p&gt;&#xA;&lt;p&gt;The fastest known algorithm is one by &lt;a href=&quot;https://cs.anu.edu.au/techreports/1997/TR-CS-97-14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ng and Trench (1997 technical report)&lt;/a&gt;, which calculates the eigenvalues in &lt;span class=&quot;math-container&quot;&gt;$\tilde{O}(n^{2})$&lt;/span&gt; time in serial and &lt;span class=&quot;math-container&quot;&gt;$\tilde{O}(n)$&lt;/span&gt; time in parallel.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-08-04T04:49:20.223" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="2984" PostTypeId="1" CreationDate="2012-08-03T11:20:33.520" Score="4" ViewCount="917" Body="&lt;p&gt;I have a linear programming problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;min $c^T x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax\leq b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in my problem, $A$ contains also some variables $y$, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = \begin{pmatrix} y_1 &amp;amp; 4 \\ 3 &amp;amp; y_2 \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find a value of $y$ such that the solution $x$ of the LP, for that fixed choice of $y$, is positive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question is different than normal formulation of parametric LP which typically only involves parameters in the cost vector or right-hand side of the constraints, and I do not want to simply use nonlinear programing. Any good solution?&lt;/p&gt;&#xA;" OwnerUserId="1851" LastEditorUserId="9" LastEditDate="2012-08-07T17:11:31.357" LastActivityDate="2012-08-07T19:46:26.927" Title="parametric linear programming" Tags="|linear-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2986" PostTypeId="1" CreationDate="2012-08-03T16:34:07.447" Score="0" ViewCount="7580" Body="&lt;p&gt;I want to access a GPU on a computer remotely, and share this GPU with different desktops. I envision this sort of access as being similar to using a VNC. Is it possible to do something like what I've just described, and if so, how would I go about doing it?&lt;/p&gt;&#xA;" OwnerUserId="217" LastEditorUserId="276" LastEditDate="2012-08-06T09:02:23.053" LastActivityDate="2020-04-08T09:18:32.290" Title="Is it possible to access a GPU remotely (i.e., from another machine)?" Tags="|gpu|cloud-computing|" AnswerCount="1" CommentCount="4" ClosedDate="2012-08-10T18:27:55.737" ContentLicense="CC BY-SA 3.0" />
  <row Id="2987" PostTypeId="1" AcceptedAnswerId="2988" CreationDate="2012-08-03T16:56:50.377" Score="37" ViewCount="134750" Body="&lt;p&gt;I don't want to deal with &lt;a href=&quot;https://stackoverflow.com/questions/4324558/whats-the-proper-way-to-install-pip-virtualenv-and-distribute-for-python&quot;&gt;virtualenv&lt;/a&gt; for a local Python installation, I just want to install a few packages locally without dealing with the &lt;code&gt;PYTHONPATH&lt;/code&gt; environment variable, how do I do that?&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="20688" LastEditDate="2019-06-22T20:27:32.030" LastActivityDate="2019-06-22T20:27:32.030" Title="What is the simplest way to do a user-local install of a python package?" Tags="|python|software|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2988" PostTypeId="2" ParentId="2987" CreationDate="2012-08-03T16:56:50.377" Score="56" Body="&lt;p&gt;Python (as of &lt;a href=&quot;http://www.python.org/dev/peps/pep-0370/&quot;&gt;2.6 and 3.0&lt;/a&gt;) now searches in the ~/.local directory for local installs, which do not require administrative privileges to install, so you just need to point your installer to that directory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have already downloaded the package &lt;code&gt;foo&lt;/code&gt; and would like to install it manually, type:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd path/to/foo&#xA;python setup.py install --user&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you are using easy_install and would like the package downloaded and installed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;easy_install --prefix=$HOME/.local/ foo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update by RafiK&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install --user foo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The following answer is provided for historical purposes:&#xA;&lt;em&gt;It's a little more work if you are using pip to download and install:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install --install-option=&quot;--prefix=$HOME/.local&quot; foo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2015-03-12T17:36:59.523" LastActivityDate="2015-03-12T17:36:59.523" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2990" PostTypeId="2" ParentId="943" CreationDate="2012-08-03T21:33:38.777" Score="4" Body="&lt;p&gt;From a metaphorical point of view, the gpu can be seen as a person lying on a bed of nails. The person lying on top is the data and in the base of each nail there is a processor, so the nail is actually an arrow pointing from processor to memory. All nails are in a regular pattern, like a grid.&#xA;If the body is well spread, it feels good (performance is good), if the body only touches some spots of the nail bed, then the pain is bad (bad performance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be taken as a complementary answer to the excellent answers above.&lt;/p&gt;&#xA;" OwnerUserId="1853" LastEditorUserId="317" LastEditDate="2013-11-03T08:17:22.853" LastActivityDate="2013-11-03T08:17:22.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2991" PostTypeId="2" ParentId="2026" CreationDate="2012-08-03T21:37:59.613" Score="1" Body="&lt;p&gt;If you already have the voronoi diagram of a set of points, then computing the Delaunay triangulation will only take you O(n). Equivalently, given a voronoi point, you can obtain its Delaunay triangle in O(1). They are dual so try exploiting this situation whenever is possible.&lt;/p&gt;&#xA;" OwnerUserId="1853" LastActivityDate="2012-08-03T21:37:59.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2992" PostTypeId="2" ParentId="2986" CreationDate="2012-08-03T23:16:34.430" Score="2" Body="&lt;p&gt;VirtualGL/TurboVNC may help &lt;a href=&quot;https://sourceforge.net/projects/virtualgl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/projects/virtualgl/&lt;/a&gt;. I haven't tried 3 VNC sessions at once but I think it should work. There are cloud solutions coming but are proprietory and you will have pay for them, it requires the hardware to handle sharing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.gputechconf.com/gtcnew/on-demand-gtc.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.gputechconf.com/gtcnew/on-demand-gtc.php&lt;/a&gt; has some presentations on the tech.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that might help is rCUDA &lt;a href=&quot;http://www.rcuda.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.rcuda.net/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="564" LastEditorUserId="35560" LastEditDate="2020-04-08T09:18:32.290" LastActivityDate="2020-04-08T09:18:32.290" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2993" PostTypeId="1" CreationDate="2012-08-04T03:42:59.410" Score="10" ViewCount="522" Body="&lt;p&gt;I am beginning my graduate studies in engineering and will be working on computational science projects. I noticed that there has been some discussion about the advantages and disadvantages of implementing your own algorithms &lt;a href=&quot;https://scicomp.stackexchange.com/questions/159/is-it-common-not-to-use-libraries-for-standard-numerical-algorithms-and-why&quot;&gt;here&lt;/a&gt;. Is the cost-benefit analysis of implementing your own algorithms versus using libraries different when you're a beginning graduate student?&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-08-08T13:15:46.270" Title="Priorities for learning computational methods, when should I write my own code vs. using libraries as a beginning graduate student?" Tags="|software|education|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2996" PostTypeId="1" AcceptedAnswerId="3001" CreationDate="2012-08-04T08:10:59.357" Score="5" ViewCount="121" Body="&lt;p&gt;Say I would like to generate a pseudo-random number, which only contains the digits 1, 4, 7 (this is arbitrary).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first guess would be to create and array &quot;147&quot; and create random numbers in the range [0..2] and use these numbers as an index into this array to concatenate a number in a loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is a more &quot;mathematics&quot;-like answer?&lt;/p&gt;&#xA;" OwnerUserId="1856" LastEditorUserId="1804" LastEditDate="2013-05-25T12:49:13.337" LastActivityDate="2013-05-25T12:49:13.337" Title="How to assure that a number only contains a certain range of digits?" Tags="|random-number-generation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2997" PostTypeId="2" ParentId="2984" CreationDate="2012-08-04T08:11:58.850" Score="5" Body="&lt;p&gt;As (almost) always in parametric programming, you have to encode optimality of $x$, for a given $y$, using optimality conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c + A(y)^T\lambda=0$ (stationarity)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lambda^T(b-A(y)x)=0$ (complementarity)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lambda\geq 0,b-A(y)x\geq 0$ (dual and primal feasibility)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve your problem, you solve a feasibility problem with the KKT conditions above, and the additional constraint $x\geq 0$. In other words, effectively solving a &lt;strong&gt;bilevel program&lt;/strong&gt;, where the outer program has no objective and a simple bound constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parametric/bilevel programming is hard, and as you probably realized, with parametric $A$ is even harder as it doesn't retain the polytopic properties of parametric linear programming. I don't think there is any straightforward way to solve the problem beyond simply attacking the problem just posed using global nonlinear programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; By using the stationarity constraint in the complementarity constraint, bilinear constraints arise (still nonconvex of-course)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c + A(y)^T\lambda=0$ (stationarity)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lambda^Tb + c^Tx=0$ (rewritten complementarity)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lambda\geq 0,b-A(y)x\geq 0$ (dual and primal feasibility)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, in case someone is missing the point with optimality conditions, we are not looking for a solution in $(x,y)$ to the problem minimize $c^Tx$ subject to $A(y)x\leq b, x\geq 0$. As an example (using parametric $b$ instead of $A$ for simplicity), let us find a fixed value $y$ such that the optimal solution $x$ to minimize $x$ subject to $x\geq y$ is non-negative. The solution to this problem is any $y\geq 0$ (since with these fixed values of $y$ the LP will return the non-negative solution $x=y\geq 0$). However, if we simply merge the constraints and minimize $x$ subject $x\geq y, x\geq 0$, over $x$ and $y$, the optimal solution is $x=0$ and $y$ any non-positive number. A completely wrong solution, since using this non-positive number (-2 for example) in the LP will lead to an infeasible $x$ (-2).&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2012-08-07T13:33:39.117" LastActivityDate="2012-08-07T13:33:39.117" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="2998" PostTypeId="2" ParentId="2987" CreationDate="2012-08-04T10:02:53.547" Score="2" Body="&lt;p&gt;@Aron: Be sure to add the local site-packages path to the environment variable $PYTHONPATH&lt;/p&gt;&#xA;" OwnerUserId="829" LastActivityDate="2012-08-04T10:02:53.547" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2999" PostTypeId="1" AcceptedAnswerId="3000" CreationDate="2012-08-04T16:11:42.953" Score="5" ViewCount="1049" Body="&lt;p&gt;For example I consider a heat equation that I want to solve numerically : $$u_t=u_{xx},$$ In order to have a uniqueness on a computational bounded domain I have to have boundary condition specified for both $x_{min}$ and $x_{max}$. Ideally I can specify the value of the function, if it is not known I can specify the value of the derivative. However, even if I use the exact solution to specify value of the derivative I noticed that the error at the boundaries is larger than in the middle of the grid. It might be a bug in the code or a fundamental error that I don't get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, my question is: does Neumann boundary condition maintains uniqueness of a solution or it is only valid for Dirichlet? And when I can safely use gradient value on the boundary compare to the value of the function itself? Sometimes, I have no idea about the value of the function on the boundary, however, it is more natural to set the boundary values though the derivatives.    &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="123" LastEditDate="2012-08-05T09:38:45.953" LastActivityDate="2012-08-05T09:38:45.953" Title="Is the heat equation with Neumann boundary conditions well-posed?" Tags="|parabolic-pde|well-posedness|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3000" PostTypeId="2" ParentId="2999" CreationDate="2012-08-04T17:13:13.013" Score="8" Body="&lt;p&gt;Yes, for the heat equation Neumann boundary conditions all around the boundary is sufficient to maintain uniqueness of the solution. Whether you choose Neumann or Dirichlet conditions is dictated by the physical situation you try to model: if you know the temperature, then you need Dirichlet conditions; if you know the heat flux, then you need Neumann conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the error: in a 1d situation I would expect the error to be roughly uniformly distributed after a few time steps. That may mean that you have an error, but it's of course hard to tell without more details.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-04T17:13:13.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3001" PostTypeId="2" ParentId="2996" CreationDate="2012-08-04T17:24:59.783" Score="7" Body="&lt;p&gt;Your proposal appears to make sense but what would be important to ask first is what probability distribution you want on those numbers. The set of numbers with only certain digits is rather peculiar as it is specific to the base-10 number system you choose. It would be interesting to hear what you intend to do with those random numbers!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-04T17:24:59.783" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3002" PostTypeId="2" ParentId="2978" CreationDate="2012-08-04T20:23:22.657" Score="3" Body="&lt;p&gt;PCA is called this way since it picks the &lt;em&gt;principal components&lt;/em&gt;. If you happen to have several components with the same or almost the same eigenvalue and you pick one but not the other, then you can't claim that you picked the &lt;em&gt;principal components&lt;/em&gt;. You picked a &lt;em&gt;subset&lt;/em&gt; of the principal components. In other words, if your second eigenvalue is doubled, then it's not a useful strategy to choose only two principal components, but you need to consider three.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-04T20:23:22.657" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3003" PostTypeId="2" ParentId="2993" CreationDate="2012-08-04T20:50:25.647" Score="10" Body="&lt;p&gt;In my opinion, being a beginning graduate student doesn't change the answer by David Ketcheson &lt;a href=&quot;https://scicomp.stackexchange.com/a/170/276&quot;&gt;here&lt;/a&gt; to the question you've linked in your post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code minimal versions of algorithms you want to learn. Then set them aside. Coding your own algorithms is most useful for learning, but for research (or production) code, unless your research goals are to write software that improves upon the state-of-the-art libraries out there (if any even exist), you're better off using libraries. Libraries are likely to be better documented, more scalable, and more robust than what you code yourself, unless you are (or become) really good at coding. Also, libraries are likely to be tested and debugged for you (though of course, that depends on who wrote the library...). You will be responsible for supporting, debugging, and testing any code you write for your thesis, and to save time, it helps to minimize the amount of code you need to write.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only other scenarios that I can think of (i.e., there may be others) are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There aren't any libraries out there that provide the features you need. Consider contributing to existing open-source libraries or writing your own open-source library, so that others may benefit.&lt;/li&gt;&#xA;&lt;li&gt;You need additional performance and can leverage special problem structure to get it. Then document and modify an existing library, or write a higher performance implementation yourself.&lt;/li&gt;&#xA;&lt;li&gt;A supervisor insists that you roll your own software. Well, you're on your own there, but I suggest bringing up the points above to try to convince them that you'd be better off using libraries (if that's at all possible in your situation).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-08-04T20:50:25.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3004" PostTypeId="1" AcceptedAnswerId="3005" CreationDate="2012-08-04T22:59:11.440" Score="7" ViewCount="1474" Body="&lt;p&gt;Reading an implementation of the golden section search, I came across the following termination test:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$| a - d | &amp;lt; \varepsilon ( |b| + |c| )$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a &amp;lt; b &amp;lt; c &amp;lt; d$ are four points at which the function has been probed and $\varepsilon$ is a user-provided tolerance. Why is this a good termination criterion? If I take a function and an initial bracket and translate them along the abscissa, the algorithm should probe the function in the same fashion, but this test will terminate the runs at different iterations...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers!&lt;/p&gt;&#xA;" OwnerUserId="1862" LastEditorUserId="127" LastEditDate="2013-05-08T14:39:51.820" LastActivityDate="2013-05-08T14:39:51.820" Title="Termination Criterion of Golden Section Search" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3005" PostTypeId="2" ParentId="3004" CreationDate="2012-08-05T00:12:09.377" Score="8" Body="&lt;p&gt;$|b|+|c|$ has the same order of magnitude as the extrema.  Therefore, however many significant digits you have in $|b|+|c|$ is as accurately as you can calculate the location of the extrema.  By ensuring your bracket is some fraction of this, you are ensuring you have a reachable condition while still getting as much accuracy as you can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's say that your extrema is at $10^9$ and you are working in single precision.  With 7 sig. figs. in single precision, you can only hope to get an answer to within $100$ of the actual value (single precision simply doesn't have the accuracy to do any better).  In this case you would have to set your error tolerance (the r.h.s. of the inequality) to at least $100$ or your algorithm would never converge.  In another case you could have a function with an extrema at $10^{-3}$.  If you used the parameters from the last run, you would find a number somwhere between -100 and 100, and be nowhere close.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you set epsilon to $10^{-7}$, you will measure your extrema to an accuracy of about $100$ in the first case and an accuracy of about $10^{-10}$ in the second case before convergence.  This is using the exact same parameters in both cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a work around for when you have unknown locations of extrema in floating point arithmetic.  If you had arbitrary precision, or knew the approximate location of the solution you could set a fixed error bound and the algorithm would work perfectly well.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-08-05T00:12:09.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3006" PostTypeId="2" ParentId="2993" CreationDate="2012-08-05T23:33:20.623" Score="10" Body="&lt;p&gt;I'd like to give some more breadth to &lt;a href=&quot;https://scicomp.stackexchange.com/a/3003/9&quot;&gt;Geoff's thoughtful answer&lt;/a&gt;.  In particular, I want to give you a little more perspective on the value of your programming efforts as opposed to your research efforts in your early career as an academic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will find that being able to write software to augment your scientific research will make you a valuable member of almost any research team.  However, this time will not be necessarily be considered &quot;valuable&quot; by your academic peers or those hiring for academic positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a 2011 research survey conducted at Princeton, &lt;a href=&quot;http://liberty.princeton.edu/Publications/sc11_survey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;A Survey of the Practice of Computational Science&quot;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Scientists spend substantial amount of research time&#xA;  programming.  On average, scientists estimate that 35% of their research&#xA;  time is spent in programming/developing software.  While initially some&#xA;  time is spent on writing code afresh, a considerable portion of time is spent in many tedious activities. For example, researchers in Politics and Sociology who used R/Stata had to do considerable programming to retrofit census&#xA;  data into formats that individual packages in R/Stata understood. Some researchers in Chemical Engineering had to reverse engineer undocumented legacy code that performed flame simulation, long after the original authors had graduated, in order to adapt the code to newer fuels... Despite this, a vast majority of these researchers felt that &quot;they spend more time programming than they should,&quot; and&#xA;  that research time was better spent in focusing on scientic&#xA;  theories or on experiments (&quot;more concerned about physics,&quot;&#xA;  said one researcher).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That doesn't mean that it is not a good idea to implement or redesign a core library or applications, but if you are going to engage in any serious software development (more than 25% of your time working with code), keep these three thoughts in mind.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Complexity and risk grow exponentially with project size and the number of developers.  Until you have written or worked with larger pieces of software or teams of developers that extend beyond your lab, it will be difficult for you to gain a good appreciation of this and properly forecast effort.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You need to be good.  It takes a certain amount of maturity, both &lt;a href=&quot;http://software-carpentry.org/about/ninety-second-pitch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;as a programmer&lt;/a&gt; and as an application scientist, to write useful software.  You have to know what the important features are, where the numerical risks are, and be able to forecast programming effort for a given set of features and robustness.  Of course, the only way to get good is to spend time on projects that you aren't the lead on or that can safely fail or be delayed, which brings me to my final point.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Although many research laboratories and industrial positions highly value programming experience, scientific programming can act as a potential detriment to your academic career, even if your software benefits science more than your papers do.  All that time you spend learning how to program well, programming, documenting your code, and making it robust translates into papers that are not being written.  An advisor will not always have their student's best interests in mind here, as this is one of those cases where the student can provide work that benefits the advisor's group without benefiting the student's citation count.  Seek out one or more trusted mentors in the field you are interested in and make sure you have a clear understanding of what contributions are considered valuable.  &lt;a href=&quot;http://academia.stackexchange.com&quot;&gt;academia.stackexchange.com&lt;/a&gt; is an excellent place to ask a follow-up question on this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As a footnote: the number of one-man effort projects that significantly advance any computational field is steadily diminishing, be it an application area or something more technical such as dense linear algebra.  An increasing number of the software packages that form the &quot;bread-and-butter&quot; of computational research are 10 years older or more.  Scientific code that has not reached this level of maturity tends to have more bugs, less features, and sparse documentation.  Try to avoid working with immature code that is not actively supported, regardless of how old it is.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-08-05T23:46:02.370" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3007" PostTypeId="1" AcceptedAnswerId="3008" CreationDate="2012-08-06T08:37:17.403" Score="3" ViewCount="314" Body="&lt;p&gt;Suppose a matrix $A\in\mathbb{R}^{n\times n}$ is given. Faced with a proof for $$x^TAx&amp;gt;0,$$&#xA;for a non-zero vector $x\in\mathbb{R}^{n}$, I was thinking to use the information of the spectrum of $A$ (note that the proof of the above is the proof of positive definiteness of $A$). Namely, it is known that the spectrum of $A$ is strictly positive. Does that mean that $A$ is positive definite? Or, does that mean that only for symmetric $A$, $A$ is positive definite if and only if the spectrum of $A$ is positive?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-08-07T07:08:36.710" LastActivityDate="2012-08-07T07:08:36.710" Title="Positive semi-definiteness of a (symmetric) matrix" Tags="|linear-algebra|matrix|eigensystem|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3008" PostTypeId="2" ParentId="3007" CreationDate="2012-08-06T08:56:16.867" Score="2" Body="&lt;p&gt;Usually, the term &quot;positive definite&quot; refers to any square, Hermitian matrix $A$ such that $x^{H}Ax &amp;gt; 0$ for all nonzero vectors $x$, where the superscript $H$ denotes the Hermitian transpose. An equivalent definition states that a positive definite matrix $A$ is Hermitian and has strictly positive spectrum (this statement is well-posed because Hermitian matrices have real spectra); see &lt;a href=&quot;https://en.wikipedia.org/wiki/Positive-definite_matrix&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition is sometimes extended to include matrices that are not Hermitian, but such extended definitions may be inappropriate for the applications you're considering. It's probably safer to require that $A$ be Hermitian.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-08-06T09:14:41.790" LastActivityDate="2012-08-06T09:14:41.790" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3009" PostTypeId="2" ParentId="3007" CreationDate="2012-08-06T14:15:01.257" Score="8" Body="&lt;p&gt;According to &lt;a href=&quot;http://mathworld.wolfram.com/PositiveDefiniteMatrix.html&quot;&gt;MathWorld&lt;/a&gt; a matrix $A \in \mathbb{R}^{n \times n}$ is positive definite iff&#xA;$$&#xA;(x^T A x) &amp;gt; 0&#xA;$$&#xA;for all non zero vectors $x\in\mathbb{R}^n$. It is trivial to obtain that &#xA;$$&#xA;x^T\,A\,x = x^T\, \left [ \frac12(A+A^T) \right ]\, x&#xA;$$&#xA;and to recognize that positive definiteness is linked to the spectrum of the symmetric part of matrix $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A general real square matrix $A$ is positive definite iff its symmetric part $\frac12(A+A^T)$ has all positive eigenvalues.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A positive spectrum for $A$ does not implies that $\frac12 (A+A^T)$ has positive spectrum, as it can seen for&#xA;$$&#xA;A = \begin{pmatrix}a &amp;amp; 1\\ 0 &amp;amp;b\end{pmatrix}&#xA;$$&#xA;If we assume $a&amp;gt;0$ and $b&amp;gt;0$ this matrix has positive spectrum, but &#xA;$$&#xA;\mathrm{det} \frac12 (A+A^T) = ab-\frac14&#xA;$$&#xA;now if $ab &amp;lt; 1/4$, $\mathrm{det} \frac12(A+A^T)&amp;lt;0$ and $A$ is not positive definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottom line: for unsymmetric matrices $A$, if you are interested in the sign of $x^TAx$ you have to study the eigenvalues of $(A+A^T)/2$.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2012-08-06T14:15:01.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3011" PostTypeId="2" ParentId="2764" CreationDate="2012-08-06T16:11:08.347" Score="2" Body="&lt;p&gt;The solution required two changes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Change the boundary conditions for $A$ to fixedValue 0, which follows from $B\cdot n = 0$ at the boundaries.&lt;/li&gt;&#xA;&lt;li&gt;Disable under-relaxation. Even though the system converged without under-relaxation, I was not able to reach convergence no matter which value I assigned to the under-relaxation coefficient. This seems to be a problem with the implementation of matrix relaxation in OpenFOAM.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="59" LastActivityDate="2012-08-06T16:11:08.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3012" PostTypeId="2" ParentId="2962" CreationDate="2012-08-07T06:42:26.883" Score="8" Body="&lt;p&gt;I don't really have much more to say than I already did on the pages you linked to, but for me the primary arguments go like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In many problems, one needs to adapt the mesh between time steps. The conceptual framework for doing this is the Rothe method where one can choose the spatial discretization independently at each time step whereas the method of lines a priori assumes that the PDE is converted into a system of ODEs -- which is incompatible with mesh adaptation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On the other hand, if you don't want to adapt your mesh, then it doesn't matter: in most cases, if the spatial discretization is going to be the same between time steps, then it doesn't matter whether you want to first discretize in space and then in time using your favorite time integrator, or the other way around -- you will come out with the same discrete problem to solve in every time step. In such cases, the Rothe method and the method of lines are then the same.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This extends to the case where you only want to adapt the mesh every once in a while: you can consider this as the method of lines applied to a number of time steps, then adapting the mesh, then one more set of time steps where you apply the method of lines. Or you can think of this as the Rothe method where you just happen to only adapt the mesh every once in a while. It will essentially come out to the same numerical scheme, just a different philosophical viewpoint.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It might be worthwhile adding one more point: In the ODE world, one often uses high order schemes with several stages or multiple steps. Thus, there is significant benefit to bottling up these algorithms into packages that you only have to hand an ODE system in one way or another. On the other hand, for time dependent PDEs, most of the time one uses rather simple time stepping methods (with the notable exception of some hyperbolic solvers): for example, Crank-Nicolson, BDF-2 or just the forward or backward Euler schemes. For these simple time integrators, it isn't particularly difficult to hand-code the time integration since it is so much simpler than the spatial discretization. What this means is that the price to pay for thinking in terms of the Rothe method -- not being able to use an ODE solver package -- is a small one, whereas the price to pay for using the method of lines -- not being able to adapt the mesh between time steps -- is a large one. That might explain why most people in the adaptive finite element world prefer to think along the Rothe method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a corollary to the last point, and coming back to the original question: It is indeed true that it is difficult in the Rothe method to package everything up nicely in an object oriented way. However, (i) as long as you stick with a single class of ODE integrators, you can of course still tabulate the coefficients of the various stages of the integrator in a class and have the code that computes them be entirely encapsulated; and (ii) the fact that one commonly uses relatively simple time integrators for time dependent PDEs means that the amount of code necessary to implement the time integration is usually vastly smaller than the amount of code that deals with the spatial discretization. In other words, I know of no way how to nicely separate the spatial from the temporal discretization using the Rothe method, but I haven't usually found this to be too much of an obstacle since so little code is necessary to write, say, a BDF-2 or RK-4 scheme.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-08-08T06:58:56.783" LastActivityDate="2012-08-08T06:58:56.783" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3013" PostTypeId="2" ParentId="2993" CreationDate="2012-08-07T08:36:46.490" Score="6" Body="&lt;p&gt;I think the cost-benefit analysis depends upon the desired scientific computing literacy that you want to achieve during your studies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For most scientist who work with computers being proficient in scientific computing is enough, this requires: a high-level (abstract) understanding of the main algorithms used, and programming skills that allow you to effectively use software libraries (building software, linking, using mailing lists).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, if you plan to become an expert in the scientific computing field, you will need a deep understanding of numerical methods, floating-point arithmetic, and computing technology. You can learn theory of these topics from books however experience through practice is needed to build and maintain advanced skills. Hence, it might be a great idea to program all you use while you learn (e.g. if you want to cook like a Chef: you learn by eating what you cook, and by cooking often!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the correct level of proficiency depends on your career. See what level of skills are used by working people in your field.&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-08-07T08:36:46.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3014" PostTypeId="2" ParentId="83" CreationDate="2012-08-07T10:22:14.673" Score="8" Body="&lt;p&gt;&lt;a href=&quot;https://software.sandia.gov/trac/coopr/wiki/Pyomo&quot;&gt;pyomo&lt;/a&gt; is a full GAMS/AMPL-like modeling environment for optimization in python. It is extremely powerful, has interfaces to all solvers that are supported by AMPL, and generates Jacobians etc. automatically. However, due to it running in a 'virtual python environment', it might not be trivial to link it to existing code.&lt;/p&gt;&#xA;" OwnerUserId="1873" LastActivityDate="2012-08-07T10:22:14.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3015" PostTypeId="1" AcceptedAnswerId="3017" CreationDate="2012-08-07T11:54:14.920" Score="3" ViewCount="232" Body="&lt;p&gt;I am looking for an efficient method to compute $$\sum_{i=1}^\left|B\right|\left|Ax_i-b_i\right|^2\rightarrow min$$ under the condition $$\forall i, x_i\ge 0,$$&#xA;where $A$ is an n-by-m matrix and $B$ is a set of n-dimensional vectors. Only $B$, $n$ and $m$ are given. In other words the goal is to find $A$ so that the sum of squares of the orthogonal distance of the subspace defined by the columns of $A$ (under the non-negativity constraint) to every point in $B$ is minimal. &#xA;$B$ is a finite set with approximately 10 to 1000 elements and $n$ is in the range of 50 to 1000 and $m$ will be rather small, i.e. $\le 10$.&lt;/p&gt;&#xA;" OwnerUserId="1874" LastEditorUserId="1874" LastEditDate="2012-08-08T08:56:32.273" LastActivityDate="2012-08-08T12:37:28.267" Title="How to efficiently compute the total least squares with an inequality constraint" Tags="|linear-algebra|algorithms|high-dimensional|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3016" PostTypeId="2" ParentId="3015" CreationDate="2012-08-07T19:22:24.753" Score="3" Body="&lt;p&gt;You can transform this into a standard form &lt;a href=&quot;http://en.wikipedia.org/wiki/Quadratic_programming&quot; rel=&quot;nofollow&quot;&gt;quadratic program&lt;/a&gt; by expanding the square term, then distributing the sum (assuming your set B is finite).  You can then use one of the standard solvers (see the links at the bottom of the wikipedia page for several options).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking for more specific advice, I'd suggest adding more detail about your specific problem, including the size of your problem (how big n and m are), and if there is any structure to your matrix A.&lt;/p&gt;&#xA;" OwnerUserId="944" LastActivityDate="2012-08-07T19:22:24.753" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3017" PostTypeId="2" ParentId="3015" CreationDate="2012-08-08T07:59:17.570" Score="3" Body="&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; Rewritten after clearing up some notational confusion in the comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $X = \left[x_1, x_2, \ldots x_r\right]$ and $B = \left[b_1, b_2, \ldots b_r\right]$ where $r$ is the numbers of vectors. Your problem can then be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  \text{minimize}_{A,X\geq 0} ||AX-B||$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This looks very much like a version of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Non-negative_matrix_factorization&quot; rel=&quot;nofollow&quot;&gt;non-negative matrix factorization problem&lt;/a&gt;, and is called semi-nonnegative matrix factorization. See &lt;a href=&quot;http://www.cs.berkeley.edu/~jordan/papers/ding-li-jordan-pami.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for an example.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2012-08-08T12:37:28.267" LastActivityDate="2012-08-08T12:37:28.267" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3019" PostTypeId="1" CreationDate="2012-08-08T16:20:21.860" Score="16" ViewCount="1769" Body="&lt;p&gt;The thing I dislike most about MPI is dealing with datatypes (i.e. data maps/masks) because they don't fit that nicely with object oriented C++. &lt;code&gt;boost::mpi&lt;/code&gt; only supports MPI 1.1, however, from their website:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;boost::mpi is a C++ friendly interface to the standard Message Passing Interface… Boost.MPI can build MPI data types for user-defined types using the Boost.Serialization library&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Has anyone had any experience with &lt;code&gt;boost::mpi&lt;/code&gt; for serious scientific computing?  Would you recommend it? Did you had any issues (scaling problems, compiler problems, errors, not implemented features, the need for some mpi 2.2 features)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you comment on using &lt;code&gt;boost::mpi&lt;/code&gt; instead of using the MPI C implementation from C++? Can you combine both (use boost::mpi when you can, C-MPI elsewhere)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know of any large scientific code using &lt;code&gt;boost::mpi&lt;/code&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-08-08T18:22:44.253" LastActivityDate="2019-09-03T14:02:21.120" Title="Boost::mpi or C MPI for high performance scientific applications?" Tags="|hpc|c++|mpi|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3020" PostTypeId="1" CreationDate="2012-08-08T17:04:24.900" Score="13" ViewCount="1045" Body="&lt;p&gt;We are comparing the performance of various numerical methods that can be used to solve the Schrodinger's Equation for the Hydrogen Atom interacting with a strong laser pulse (too strong to use perturbation methods). When using discretization schemes for the radial part, it seems like most (all) people put the atom in a box, just chopping the radius off at some large value and solving for those basis sets. How does this compare with mapping the radial variable to a finite domain, and then discretizing that domain (in the process, throwing out most of the basis sets available)?   Is there a reason nobody seems to do that?&lt;/p&gt;&#xA;" OwnerUserId="1883" LastActivityDate="2012-10-07T07:08:13.763" Title="Numerical Methods for the Schrodinger Equation" Tags="|quantum-mechanics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3021" PostTypeId="1" CreationDate="2012-08-08T23:12:37.080" Score="2" ViewCount="141" Body="&lt;p&gt;I need to write MATLAB program (or use an existing one) to obtain Fourier series coefficients. Let's say the series is going to approximate a 1D curve. The boundary conditions are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;value of the curve's function in a few places&lt;/li&gt;&#xA;&lt;li&gt;curve's first derivative known in a few places&lt;/li&gt;&#xA;&lt;li&gt;maximum allowed value of curve's second derivative&lt;/li&gt;&#xA;&lt;li&gt;number of harmonics&lt;/li&gt;&#xA;&lt;li&gt;the value of curve's integral over a specified interval should be as big as possible &amp;lt;- optimization criterion.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All the functions and tools I've seen so far can obtain series coefficients in much more simple cases. Any hints will be helpful, especially:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;literature, articles with descriptions of useful methods  &lt;/li&gt;&#xA;&lt;li&gt;hints on objective functions useful for Fourier Series  &lt;/li&gt;&#xA;&lt;li&gt;info on useful MATLAB Tools&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1885" LastEditorUserId="9" LastEditDate="2012-08-10T13:38:37.237" LastActivityDate="2012-08-11T08:52:38.867" Title="Numerically designing a periodic 1D curve  that maximizes an integral area objective and satisfies value, derivative, and frequency constraints" Tags="|optimization|matlab|approximation|fourier-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3022" PostTypeId="2" ParentId="3019" CreationDate="2012-08-08T23:19:30.787" Score="6" Body="&lt;p&gt;As far as I know, &lt;code&gt;boost::mpi&lt;/code&gt; is just a &lt;code&gt;c++&lt;/code&gt; wrapper around the &lt;code&gt;C&lt;/code&gt; API. As such, you should be able to use &lt;code&gt;boost::mpi&lt;/code&gt; and switch to the &lt;code&gt;C&lt;/code&gt; API whenever some functionality is not implemented. Indeed, from their webpage:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The thin abstractions in Boost.MPI allow one to easily combine it with calls to the underlying C MPI library.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have not used it myself, and do not know of a major library that does, but I'd expect it to be just a lightweight wrapper and as such one should not worry about the performance compared to &lt;code&gt;C&lt;/code&gt; API.&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2012-08-08T23:19:30.787" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3023" PostTypeId="2" ParentId="3019" CreationDate="2012-08-09T07:23:47.537" Score="13" Body="&lt;p&gt;I have always thought that we should use it in our own project, deal.II, because it is higher level than pure MPI and can save a few lines of code here and there. That said, what I learned over the years is that most high-level code doesn't actually have that much MPI code to begin with -- the 600,000 lines of code in deal.II have only ~50 calls to MPI. That's certainly much much less than a package such as PETSc, but I think it's nevertheless true that most codes have fewer MPI calls than one would expect at first and that, consequently, the benefit of using something that's higher level is not as great as one may think at first glance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this then boils down to is for you to consider what the trade-offs are. How much MPI will you need to use, and how does that compare to the additional effort required to build and link with an external library.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2019-09-03T14:02:21.120" LastActivityDate="2019-09-03T14:02:21.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3025" PostTypeId="1" CreationDate="2012-08-09T13:55:28.663" Score="2" ViewCount="148" Body="&lt;p&gt;I have a non-symmetric matrix, where the non-symmetry only appears at a subset of points. This arises due to the particular manner on which boundary conditions are applied in a Cartesian grid method. I'm looking for a way to factor the matrix into the product of a symmetric sparse matrix and a diagonal matrix with a low rank correction so I can symmetrize the system as suggested here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2910/how-far-is-a-non-symmetric-discretization-of-an-elliptic-operator-from-the-conti&quot;&gt;How far is a non-symmetric discretization of an elliptic operator from the continuous operator itself?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1453" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-08-09T17:35:29.503" Title="Factor a non-symmetric matrix into the product of a sparse symmetric matrix and a diagonal matrix plus a low rank correction" Tags="|linear-algebra|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3026" PostTypeId="2" ParentId="3025" CreationDate="2012-08-09T14:15:13.123" Score="1" Body="&lt;p&gt;There is no way you really mean product since the product of a low rank matrix with anything is also low rank. Given non-symmetric $A$, perhaps you are looking for&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S = \frac 1 2 (A + A^T)$$&#xA;$$N = \frac 1 2 (A - A^T).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A = S + N$ is symmetric in most of the domain, then the non-symmetric part $N$ will be low rank &lt;em&gt;and&lt;/em&gt; sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course a sum is much less useful than a product if you were interested in solving this system. In particular, if $N$ is nonzero at boundaries, the rank is not small enough for specialized tricks to apply. What was really intended in the question you reference is to reformulate your problem, not to brute-force decompose or symmetricze.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-09T14:15:13.123" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3027" PostTypeId="1" CreationDate="2012-08-09T20:19:32.723" Score="4" ViewCount="109" Body="&lt;p&gt;I am looking for existing software or scripts to generate spiral graphs from cyclical (time) data, as presented by &lt;a href=&quot;http://ieg.ifs.tuwien.ac.at/~aigner/teaching/ws06/infovis_ue/papers/spiralgraph_weber01visualizing.pdf&quot; rel=&quot;nofollow&quot;&gt;Webber and Muller&lt;/a&gt;.  The graphs shown in the paper look like a great means of visualizing and identifying periodicity in cyclical data, but I haven't been able to Google up any existing solutions for creating graphs like these.&lt;/p&gt;&#xA;" OwnerUserId="1889" LastEditorUserId="276" LastEditDate="2012-08-09T20:52:42.923" LastActivityDate="2012-08-09T20:52:42.923" Title="Existing software/scripts for spiral graphs?" Tags="|software|visualization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3028" PostTypeId="2" ParentId="1445" CreationDate="2012-08-09T22:02:08.777" Score="8" Body="&lt;p&gt;My personal favorite for linear structural mechanics and dynamics has not been mentioned yet:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0133014584&quot;&gt;Finite Element Procedures&lt;/a&gt;&lt;/em&gt;, from KJ Bathe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a structural engineering background this book is the best introduction to the FEM that I have seen. It discusses the formulation of structural elements in depth, the inf-sup condition, error estimation, and modal analysis. It also discusses non linearities, heat flow, and fluid flow problems, but I can't recommend it for these topics (there are simply better books for them)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My other favorites have already been mentioned (e.g Ern and Guermond, Donea and Huerta). However I'd like to also add: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0980232708&quot;&gt;An Analysis of the Finite Element Method&lt;/a&gt;&lt;/em&gt;, from Strang and Fix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as an introduction to the theory behind the FEM. &lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-08-09T22:24:09.263" LastActivityDate="2012-08-09T22:24:09.263" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3029" PostTypeId="1" CreationDate="2012-08-10T12:06:40.997" Score="15" ViewCount="1938" Body="&lt;p&gt;There seem to be two main kinds of test function&#xA;for no-derivative optimizers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;one-liners like the&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Rosenbrock_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosenbrock function&lt;/a&gt; ff., with start points  &lt;/li&gt;&#xA;&lt;li&gt;sets of real data points, with an interpolator&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is it possible to compare say 10d Rosenbrock&#xA;with any real 10d problems ?&lt;br&gt;&#xA;One could compare in various ways:&#xA;describe the structure of local minima,&lt;br&gt;&#xA;or run optimizers A B C on Rosenbrock and on some real problems;&lt;br&gt;&#xA;but both of these seem difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Maybe theorists and experimenters are just two quite different cultures,&#xA;so I'm asking for a chimera ?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;scicomp.SE question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/46&quot;&gt;Where can one obtain good data sets/test problems for testing algorithms/routines?&lt;/a&gt;  &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://repository.cmu.edu/tepper/197&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hooker, &quot;Testing Heuristics: We Have It All Wrong&quot;&lt;/a&gt; is scathing:&#xA;&quot;the emphasis on competition ... tells us which algorithms are better but not why.&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;(Added in September 2014):&lt;br&gt;&#xA;The plot below compares 3 DFO algorithms on 14 test functions in 8d&#xA;from 10 random start points:&#xA;BOBYQA PRAXIS SBPLX from &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLOpt&lt;/a&gt;&lt;br&gt;&#xA;$\times$ 14 N-dimensional test functions, Python under &lt;a href=&quot;https://gist.github.com/denis-bz&quot; rel=&quot;nofollow noreferrer&quot;&gt;gist.github&lt;/a&gt;&#xA;from this &lt;a href=&quot;http://www-optima.amp.i.kyoto-u.ac.jp/member/student/hedar/Hedar_files/TestGO.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matlab&lt;/a&gt; by A. Hedar&lt;br&gt;&#xA;$\times$ 10 uniform-random startpoints in each function's bounding box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Ackley, for example, the top row shows that SBPLX is best and PRAXIS terrible;&#xA;on Schwefel, the bottom right panel shows SBPLX finding a minimum on the 5 th random start point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, BOBYQA is best on 1, PRAXIS on 5, and SBPLX (~ Nelder-Mead with restarts) on 7&#xA;of 13 test functions, with Powersum a tossup.&#xA;YMMV !  In particular, Johnson says,&#xA;&quot;I would advise you not to use function-value (ftol) or parameter tolerances (xtol) in global optimization.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conclusion: don't put all your money on one horse, or on one test function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/WhY1W.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-06-09T18:36:26.517" Title="Testing numerical optimization methods: Rosenbrock vs. real test functions" Tags="|optimization|data-sets|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3030" PostTypeId="1" AcceptedAnswerId="3054" CreationDate="2012-08-10T15:22:26.003" Score="3" ViewCount="157" Body="&lt;p&gt;Currently, I have some experience with classical molecular dynamics simulations, and I've had undergraduate course in quantum mechanics (the course was &quot;analytical&quot; one, no approaches to computer simulations were covered, so I'm familiar with Schroedinger equation, but not with Liouville's equation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried reading some papers, but I often can't just relate some concepts to each other, probably due to lack of some basic knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a good reading to get some basic background in mixed quantum-classical simulations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is &quot;QM/MM&quot; a synonym for mixed quantum classical methods or just one approach?&lt;/p&gt;&#xA;" OwnerUserId="463" LastEditorUserId="463" LastEditDate="2012-08-13T03:23:08.567" LastActivityDate="2012-08-13T20:37:00.443" Title="What is a good introduction to mixed quantum-classical modelling" Tags="|reference-request|quantum-mechanics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3031" PostTypeId="1" CreationDate="2012-08-10T18:45:37.557" Score="1" ViewCount="10998" Body="&lt;p&gt;Below is my own approach to implement the Least Squares Regression algorithm in MATLAB. Could you please take a look and tell me if it makes sense; if it does exactly what is supposed to do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Please, pay attention to the commented commands as well.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [MSE_train_mean,MSE_train_std,MSE_test_mean,MSE_test_std,w_star] = perform_lsr(X,Y,Z)&#xA;&#xA;% 604 days for the training set (29/05/2009 - 18/10/2011)&#xA;% 145 days for the evaluation set (19/10/2011 - 16/05/2012)&#xA;Xtran = X(1:end-145,:);&#xA;Ytran = Y(1:end-145,:);&#xA;Ztran = Z(1:end-145,:);&#xA;Xeval = X(end-144:end,:);&#xA;Yeval = Y(end-144:end,:);&#xA;Zeval = Z(end-144:end,:);&#xA;&#xA;% Set number of runs.&#xA;runs = 20;&#xA;&#xA;% Set number of folds.&#xA;folds = 5;&#xA;&#xA;% Initialize auxiliary matrices.&#xA;MSE_train = zeros(runs,folds);&#xA;MSE_test = zeros(runs,folds);&#xA;w_star = zeros(runs,folds,size(Xtran,2));&#xA;&#xA;% Perform runs.&#xA;for r=1:runs,&#xA;&#xA;% Split original dataset into training and test set.&#xA;split_assignments = cross_val_split(folds,size(Xtran,1));&#xA;&#xA;% Perform folds.&#xA;for f=1:folds,&#xA;&#xA;% Assign explanatory variables (x).&#xA;x_train = Xtran((split_assignments(:,f)==0),:);&#xA;x_test = Xtran((split_assignments(:,f)==1),:);&#xA;&#xA;% Assign respond variable (y).&#xA;y_train = Ytran((split_assignments(:,f)==0),:);&#xA;y_test = Ytran((split_assignments(:,f)==1),:);&#xA;&#xA;% Retrieve size of matrices.&#xA;[l_train_n,l_train_m] = size(x_train);&#xA;[l_test_n,l_test_m] = size(x_test);&#xA;&#xA;% Estimate parameters (w0,w1,...) of the 1st order model from training set.&#xA;w_star(r,f,:) = (x_train' * x_train) \ (x_train' * y_train);&#xA;w_star_temp = w_star(r,f,:);&#xA;w_star_temp = w_star_temp(:);&#xA;&#xA;% Apply the learned weights on both training and test sets and compute the&#xA;% corresponding MSEs.&#xA;MSE_train(r,f) = (1 / l_train_n) * (w_star_temp' * (x_train') * x_train * w_star_temp - 2 * y_train' * x_train * w_star_temp + y_train' * y_train);&#xA;MSE_test(r,f) = (1 / l_test_n) * (w_star_temp' * (x_test') * x_test * w_star_temp - 2 * y_test' * x_test * w_star_temp + y_test' * y_test);&#xA;&#xA;end&#xA;end&#xA;&#xA;% % Plot both training and test sets' MSEs as a function of runs and folds.&#xA;% figure, mesh(MSE_train), title('Training Set`s MSEs vs Runs and Folds');&#xA;% figure, mesh(MSE_test), title('Testing Set`s MSEs vs Runs and Folds');&#xA;% &#xA;% % Plot both training and test sets' mean of MSEs accompanied by their&#xA;% % corresponding std of MSEs.&#xA;% figure, errorbar(1:runs,mean(MSE_train,2),std(MSE_train,0,2));&#xA;% figure, errorbar(1:runs,mean(MSE_test,2),std(MSE_test,0,2));&#xA;&#xA;% Average over folds and then over runs.&#xA;MSE_train_mean = mean(mean(MSE_train,2),1);&#xA;MSE_train_std = std(mean(MSE_train,2),0,1);&#xA;MSE_test_mean = mean(mean(MSE_test,2),1);&#xA;MSE_test_std = std(mean(MSE_test,2),0,1);&#xA;w_star_temp = mean(mean(w_star,2),1);&#xA;w_star = w_star_temp(:);&#xA;&#xA;% % Calculate in-sample residuals.&#xA;% y_hat_tran = Xtran * w_star;&#xA;% e_tran = Ytran - y_hat_tran;&#xA;% % Check that they sum up to zero.&#xA;% fprintf('In-sample residuals sum up to %i.\n',sum(e_tran));&#xA;% % Finally, plot them.&#xA;% figure, scatter(1:size(e_tran,1),e_tran);&#xA;% &#xA;% % Calculate out-of-sample residuals.&#xA;% y_hat_eval = Xeval * w_star;&#xA;% e_eval = Yeval - y_hat_eval;&#xA;% % Check that they sum up to zero.&#xA;% fprintf('Out-of-sample residuals sum up to %i.\n',sum(e_eval));&#xA;% % Finally, plot them.&#xA;% figure, scatter(1:size(e_eval,1),e_eval);&#xA;&#xA;% Print summary statistics.&#xA;fprintf('Over %i runs, %i folds, and regarding the training set, the mean and standard deviation is %.10f and %.10f respectively.\n',runs,folds,MSE_train_mean,MSE_train_std);&#xA;fprintf('Over %i runs, %i folds, and regarding the testing set, the mean and standard deviation is %.10f and %.10f respectively.\n',runs,folds,MSE_test_mean,MSE_test_std);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1829" LastEditorUserId="1829" LastEditDate="2012-08-10T23:43:43.737" LastActivityDate="2015-09-13T08:54:05.153" Title="MATLAB Code Evaluation for Least Squares Regression (LSR)" Tags="|algorithms|matlab|least-squares|" AnswerCount="1" CommentCount="1" ClosedDate="2012-08-21T20:08:21.943" ContentLicense="CC BY-SA 3.0" />
  <row Id="3032" PostTypeId="2" ParentId="1445" CreationDate="2012-08-11T02:42:59.657" Score="4" Body="&lt;p&gt;The book &lt;a href=&quot;http://books.google.de/books?id=PizECgOWoGgC&amp;amp;lpg=PP1&amp;amp;hl=de&amp;amp;pg=PP1#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Dietrich Braess - Finite elements. Theory, Fast Solvers, and Applications in Solid Mechanics&lt;/a&gt; gives a good perspective onto several standard and advanced topics. In particular, Ch. 3 offers introductions into many very different topics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, I think are two recommandable references for problems in vector analysis, although these are very long papers rather than text books:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.sam.math.ethz.ch/~hiptmair/Courses/CEM/HIP02.pdf&quot; rel=&quot;nofollow&quot;&gt;Finite elements in computational electromagnetism&lt;/a&gt; by Ralf Hiptmair. The paper introduces the relevant tools for $H(\operatorname{curl})$-methods. The reader should regard it as a starting point for further study in the topic.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.sam.math.ethz.ch/~hiptmair/Seminars/STRUCPRES/Articles/AFW06.pdf&quot; rel=&quot;nofollow&quot;&gt;Finite element exterior calculus, homological techniques, and applications&lt;/a&gt; is suitable for those (and IMO only those) who want to go into the mathematical theory of FEM (for non-scalar problems). There is still a large array of smaller and bigger gaps to close, but it is a good starting point into the relevant methods, too.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-08-11T02:42:59.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3033" PostTypeId="1" AcceptedAnswerId="3089" CreationDate="2012-08-11T07:32:11.263" Score="6" ViewCount="186" Body="&lt;p&gt;I have got a large sparse matrix $A \in \mathbb R^{n \times n}$ and I want to find non-trivial elements in the kernel/nullspace of this matrix. How can this be done? I would like to learn more about a particular method that seems to be folklore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most people that I have talked to regard this problem as an instance of an eigenvalue problem and recommend, e.g., the power method for this. For general $A$, this might a very good recommendation. But assume $A$ is symmetric-positive semidefinite, like, e.g., the stiffness matrix of the Poisson system with natural boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us choose some random vector $x_0 \in \mathbb R^n$, which can be assembled by a probability distribution. Let us use the Conjugate Residual method (CRM) with $x_0$ as a starting vector and right-hand side $0 \in \mathbb R^n$ (!). The CRM converges to a solution $x \in \mathbb R^n$. This solution is the projection of $x_0$ along the range of $A$ onto the nullspace of $A$. With high probability, we have found a non-trivial nullspace vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar ideas can be applied to many other Krylow space iterative methods. Whereas I have been told that this trick is generally known, it seems to be folklore. But it seems to be too useful for me to not even have a proper name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this &quot;trick&quot; have a proper name, and why does it lead such a shadowy existence?&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="9" LastEditDate="2012-08-17T20:11:49.153" LastActivityDate="2012-08-17T20:11:49.153" Title="Identifying the name/provenance of a technique to find the nullspace vectors of a matrix by random sampling and the conjugate residual method" Tags="|linear-algebra|eigensystem|krylov-method|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3034" PostTypeId="2" ParentId="3021" CreationDate="2012-08-11T08:52:38.867" Score="3" Body="&lt;p&gt;Write your function as a linear combination of harmonics and treat the coefficients as variables. This gives a semi-infinite linear program. &#xA;The Matlab optimization toolbox has a routine linprog to solve linear programs (LPs) only; so you need to discretize the curvature constraint. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replace the curvature constraint by $N$ constraints evaluated at $N$ equidistant points, solve the LP, add more points close to the points where near-worst case curvature occurred, and repeat until the desired accuracy is reached.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-08-11T08:52:38.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3035" PostTypeId="2" ParentId="3029" CreationDate="2012-08-11T09:03:45.390" Score="13" Body="&lt;p&gt;Simple functions like Rosenbrock's are used to debug and pre-test newly written algorithms: They are fast to implement and to execute, and a method that cannot solve the standard problems well is unlikely to work well on real life problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a recent thorough comparison of derivative-free methods for expensive functions, see &lt;strong&gt;Derivative-free optimization: A review of algorithms and comparison of software implementations&lt;/strong&gt;. L.M. Rios, N.V. Sahinidis - doi 10.1007/s10898-012-9951-y Journal of Global Optimization, 2012. (See also the accompanying webpage: &lt;a href=&quot;http://archimedes.cheme.cmu.edu/?q=dfocomp&quot;&gt;http://archimedes.cheme.cmu.edu/?q=dfocomp&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="2155" LastEditDate="2013-01-11T22:04:25.127" LastActivityDate="2013-01-11T22:04:25.127" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3036" PostTypeId="2" ParentId="3029" CreationDate="2012-08-11T10:22:10.943" Score="8" Body="&lt;p&gt;The advantage of synthetic testcases like the Rosenbrock function is that there is existing literature to compare with, and there is a sense in the community how good methods behave on such testcases. If everyone used their own testcase it would be much harder to come to a consensus which methods work and which don't.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-11T10:22:10.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3037" PostTypeId="1" CreationDate="2012-08-12T06:06:56.080" Score="6" ViewCount="230" Body="&lt;p&gt;Finding the roots of a complex polynomial is in general extremely numerically unstable, as discussed in &lt;a href=&quot;http://en.wikipedia.org/wiki/Wilkinson%27s_polynomial&quot; rel=&quot;nofollow&quot;&gt;(1)&lt;/a&gt;.  According to Pan (&lt;a href=&quot;http://algo.inria.fr/seminars/sem97-98/pan.html&quot; rel=&quot;nofollow&quot;&gt;(2)&lt;/a&gt;, &lt;a href=&quot;http://scholar.google.com/scholar?q=Algorithm%20for%20Approximating%20Complex%20Polynomial%20Zeros&amp;amp;hl=en&amp;amp;lr=&quot; rel=&quot;nofollow&quot;&gt;(3)&lt;/a&gt;), this produces a cubic complexity lower bound, and he presents a near-optimal algorithm which is quasilinear in arithmetic and quasicubic in boolean (control flow) complexity.  His algorithm is an improvement to the &lt;a href=&quot;http://en.wikipedia.org/wiki/Splitting_circle_method&quot; rel=&quot;nofollow&quot;&gt;splitting circle method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this lower bound applies only if we want forward stable (i.e., correct) answers.  If we're satisfied with backward stable answers, where it's okay to find correct roots for a slightly modified input problem, I would expect much faster complexities to be possible.  Does anyone know if such an algorithm exists, and specifically if a splitting circle variant can run in quasilinear time if we only need backward stability?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: $\epsilon$ in $\tilde{O}(n \log(1/\epsilon))$ vaguely refers to the relative backward accuracy of the result (the amount you have to shift the coefficients to move to the roots to the computed answers).  I'm not sure exactly what the correct definition of $\epsilon$ would be, since different coefficients can change the polynomial values by dramatically different amounts.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2012-08-12T16:18:08.240" LastActivityDate="2013-01-17T17:48:09.993" Title="Is there a backward stable $\tilde{O}(n \log(1/\epsilon))$ algorithm to factor a complex polynomial?" Tags="|polynomials|roots|stability|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3038" PostTypeId="1" CreationDate="2012-08-12T07:32:00.400" Score="4" ViewCount="217" Body="&lt;p&gt;Algorithms for edit distance give a measure of the distance between two strings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: which of these measures would be most relevant to detect two different persons names which are actually the same? (different because of a mispelling). The trick is that it should minimize false positives. Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obaama&#xA;Obama&#xA;=&gt; should probably be merged&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obama&#xA;Ibama&#xA;=&gt; should not be merged.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just an oversimple example. Are their computer scientists out there who worked out this issue in more detail?&lt;/p&gt;&#xA;" OwnerUserId="1894" LastActivityDate="2012-08-12T16:41:53.703" Title="most adequate edit distance for misspellings in names?" Tags="|algorithms|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3039" PostTypeId="2" ParentId="3037" CreationDate="2012-08-12T11:11:14.187" Score="6" Body="&lt;p&gt;I don't know about the theoretical complexity. But the practically fastest algorithm for finding all zeros of a polynomial given as a finite power series is the algorithm by Traub [1]. It factors a degree $n$ polynomial in typically $O(n^2)$ time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also [2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For factoring a polynomial accessible only through its function values, see, e.g., my paper [3].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1145/361254.361262&quot; rel=&quot;nofollow&quot;&gt; &lt;strong&gt;Jenkins MA, Traub JF&lt;/strong&gt;. 1972. Algorithm 419: zeros of a complex polynomial [C2]. Communications of the ACM 15: 97–99.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/S0036144595288554&quot; rel=&quot;nofollow&quot;&gt; &lt;strong&gt;Pan VY&lt;/strong&gt;. 1997. Solving a Polynomial Equation: Some History and Recent Progress. SIAM Review 39: 187.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1016/S0377-0427%2803%2900380-7&quot; rel=&quot;nofollow&quot;&gt; &lt;strong&gt;Neumaier A&lt;/strong&gt;. 2003. Enclosing clusters of zeros of polynomials. Journal of Computational and Applied Mathematics 156: 389–401.&lt;/a&gt; [&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/ms/polzer.pdf&quot; rel=&quot;nofollow&quot;&gt;pdf&lt;/a&gt;]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="463" LastEditDate="2012-08-13T00:18:03.043" LastActivityDate="2012-08-13T00:18:03.043" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3040" PostTypeId="2" ParentId="3038" CreationDate="2012-08-12T15:29:56.040" Score="4" Body="&lt;p&gt;I recommend looking into this,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://norvig.com/spell-correct.html&quot; rel=&quot;nofollow&quot;&gt;http://norvig.com/spell-correct.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the correct training data, you can likely taylor the solution to just formal names.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2012-08-12T15:29:56.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3041" PostTypeId="2" ParentId="3038" CreationDate="2012-08-12T16:41:53.703" Score="1" Body="&lt;p&gt;Aron is right in his comment, that the corpus is important, not just the distance metric between a pair of strings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the link that Carleton mentions is on the right track, in terms of having a rule set for defining the allowable misspellings.&#xA;I'm a little unclear how the probability measures help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few decades ago I wrote a rule-driven spelling corrector (in Pascal, and also in Fortran, if you can believe it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corpus was a character-based trie containing all the words.&#xA;In addition to being easy to search, it's easy to build-in arbitrary prefixes and suffixes by just connecting together parts of the trie into a FSM.&#xA;So for example it could recognize &quot;nationalizational&quot;, or other words that would not really be in any dictionary but still are words - sort of.&#xA;The advantage of that is you can dramatically reduce the size of the corpus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm was similar to branch-and-bound, but it consisted of a recursive tree walk with an error budget.&#xA;When called with an error budget of 0, it would only &quot;print&quot; exact matches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When called with an error budget of 1, it would print all entries in the trie that could be gotten from the input key by applying 1 misspelling rule, such as insertion, deletion, transposition, or simple replacement of a character of group of characters by another that could &quot;sound alike&quot;.&#xA;You can tune the rule set for the kinds of misspellings appropriate for the particular subject matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If no match was found with a budget of 0 or 1, then the budget was increased to 2, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The performance was of course exponential in the error budget, for small budgets.&#xA;What that means is that the effort spent at budget level N only differed by a constant factor from the sum of the effort spent at lower levels of budget.&#xA;So it's OK to just call repeatedly with successively larger budgets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large budgets, the performance is linear in the corpus size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found it much easier to write the rule-based search as a recursive tree-walk than as a kind of breadth-first or branch-and-bound algorithm, because at any moment in time it only has to manage a single hypothetical misspelling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I did not do, and I should have, is to precompile both the corpus trie and the rule set into a compiled language (like C), because the corpus and rule set change very seldom.&#xA;This would have given higher speed, by a couple orders of magnitude.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastActivityDate="2012-08-12T16:41:53.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3042" PostTypeId="2" ParentId="1725" CreationDate="2012-08-12T21:28:21.953" Score="3" Body="&lt;p&gt;I recommend using Parallel Multiple Relatively Robust Representations (PMRRR) for the parallel tridiagonal eigensolve. It can compute all of the eigenpairs of the tridiagonal matrix in $O(n^2)$ work in parallel. You can find an overview of the method &lt;a href=&quot;http://www.aices.rwth-aachen.de:8080/~pauldj/talks/ILAS11.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. There is also the implementation in ScaLAPACK, which is discussed &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1644002&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-08-12T21:28:21.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3043" PostTypeId="1" AcceptedAnswerId="3052" CreationDate="2012-08-13T10:42:36.253" Score="14" ViewCount="8962" Body="&lt;p&gt;I am trying to find some resources to help explain how to choose boundary conditions when using finite difference methods to solve PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The books and notes which I currently have access to all say similar things:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The general rules governing stability in the presence of boundaries are far too complicated for an introductory text; they require sophisticated mathematical machinery&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(A. Iserles A First Course in the Numerical Analysis of Differential Equations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when trying to implement the 2-step leapfrog method for the advection equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_i^{n+1} = u_i^{n-1} + \mu (u_{i+1}^n - u_{i-1}^n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using MATLAB&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M = 100; N = 100;&#xA;&#xA;mu = 0.5;&#xA;&#xA;c = [mu 0 -mu];&#xA;f = @(x)(exp(-100*(x-0.5).^2));&#xA;&#xA;u  = zeros (M, N);&#xA;x = 1/(M+1) * (1:M);&#xA;&#xA;u(:,1) = f(x);&#xA;u(:,2) = f(x + mu/(M+1));&#xA;&#xA;for i = 3:N&#xA;    hold off;&#xA;    u(:,i) = conv(u(:,i-1),c,'same') + u(:,i-2);&#xA;    plot(x, u(:,i));&#xA;    axis( [ 0 1 0 2] )&#xA;    drawnow;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The solution behaves nicely until it reaches the boundary, when it very suddenly starts behaving badly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I learn how to handle boundary conditions like this?&lt;/p&gt;&#xA;" OwnerUserId="1898" LastEditorUserId="123" LastEditDate="2012-08-19T19:36:36.727" LastActivityDate="2018-03-28T08:37:53.887" Title="Boundary conditions for the advection equation discretized by a finite difference method" Tags="|pde|finite-difference|boundary-conditions|advection|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3044" PostTypeId="1" CreationDate="2012-08-13T10:52:34.153" Score="-1" ViewCount="160" Body="&lt;p&gt;I have a system of equations &#xA;$$&#xA;   S_{m}(\xi) +P_{m}(\xi)=f(\xi)&#xA;$$&#xA;where $\xi$ can be choosen arbitrary in some domain in $\mathbb{C}$, $f$ is known, $P_m$ is a polynomial of degree at most $m$. Here $S_{m}$ is a generic function from $\mathbb{C}$ to $\mathbb{C}$ but for me it is sufficient to know it's values in only $(m+1)$ mutually distinct points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking $(m+1)$ mutually distinct points $\xi_0, \ldots, \xi_m$ we can invert the Vandermonde matrix and hence write coefficients of $P_m$ as linear combination of $S_m(\xi_k)$,$k=0,\ldots,m$. Introducing this result back in equations we get a linear system on $S_{m}(\xi_k)$. So theoretically we can find $S_{m}(\xi_k)$. But is there some way to do it practically?&lt;/p&gt;&#xA;" OwnerUserId="1180" LastEditorUserId="1180" LastEditDate="2012-08-13T12:10:21.903" LastActivityDate="2012-08-13T14:00:59.343" Title="How to solve numerically such system of equations" Tags="|linear-algebra|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3045" PostTypeId="2" ParentId="3044" CreationDate="2012-08-13T11:20:42.367" Score="0" Body="&lt;p&gt;Unfortunately I can't comment yet, so I'm putting this as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think there are enough constraints in the problem. $S_m = f - P_m$ where $P_m$ is any polynomial of degree $\leq m$ is going to be a possible solution, so there isn't going to be a way to find out which polynomial we're talking about.&lt;/p&gt;&#xA;" OwnerUserId="1898" LastActivityDate="2012-08-13T11:20:42.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3047" PostTypeId="2" ParentId="3044" CreationDate="2012-08-13T13:31:26.713" Score="0" Body="&lt;p&gt;There are far better polynomial basis to use than the Vandermonde, which turns out to be really poorly conditioned for a lot of real-world fits.  If you know something about your solution, you'll be able to pick a better basis, but I would start with the Chebyshev basis appropriately scaled by the range of $\xi$.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-08-13T13:31:26.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3048" PostTypeId="2" ParentId="3044" CreationDate="2012-08-13T14:00:59.343" Score="2" Body="&lt;p&gt;If you know $f$ and $S$ in $m+1$ points, you have the standard polynomial interpolation problem with $f-S$ as right hand side. Thus newton or lagrange interpolation give stable answers.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-08-13T14:00:59.343" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3049" PostTypeId="2" ParentId="3043" CreationDate="2012-08-13T14:40:35.793" Score="5" Body="&lt;p&gt;&lt;strong&gt;General answer&lt;/strong&gt;&lt;br&gt;&#xA;Your problem is that you do not set (or even specify) the boundary conditions at all - your numerical problem is ill-defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, there are two possible ways to specify the boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Set the boundary conditions by specifying $u_{0}$ and $u_{101}$ externally, e.g. through the exact solution.&lt;/li&gt;&#xA;&lt;li&gt;Change the numerical stencil so it will use only interior information at the boundary.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Which way you go heavily depends on the physics of your problem. For wave-equation type problems one usually determines the eigenvalues of the flux Jacobian in order to decide whether external boundary conditions are needed, or whether the interior solution is to be used (this method is commonly called 'upwinding').&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Quick fix&lt;/strong&gt;&lt;br&gt;&#xA;In your scheme, the numerical stencil needs the values of $u_{i-1}^n$ and $u_{i+1}^n$ to calculate a value at location $i$ for time $n+1$. Thus at the left domain boundary (i.e. $i = 1$) it would need the value of $u_{0}^n$, which you have not set in your code example. The same problem arises for $u_{100}^{n+1}$, which needs $u_{101}^n$ to build the stencil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An easy way to get around would be to set $u_{1}^n$ and $u_{100}^n$ manually at each time step, by interpolating linearly from the interior solution. This is not really physical (as it does not take into account the actual wave speed), but will in your case lead to a first satisfactory solution behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a modified version of your source code below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M = 100; N = 100;&#xA;&#xA;mu = 0.5;&#xA;&#xA;c = [mu 0 -mu];&#xA;f = @(x)(exp(-100*(x-0.5).^2));&#xA;&#xA;u  = zeros (M, N);&#xA;x = 1/(M+1) * (1:M);&#xA;&#xA;u(:,1) = f(x);&#xA;u(:,2) = f(x + mu/(M+1));&#xA;&#xA;for i = 3:N&#xA;    hold off;&#xA;    %u(:,i) = conv(u(:,i-1),c,'same') + u(:,i-2);&#xA;&#xA;    % Apply the numerical stencil to all interior points&#xA;    for j = 2:M-1&#xA;        u(j,i) = u(j,i-2) + mu*(u(j+1,i-1) - u(j-1,i-1));&#xA;    end&#xA;&#xA;    % Set the boundary values by interpolating linearly from the interior&#xA;    u(1,i) = 2*u(2,i) - u(3,i);&#xA;    u(M,i) = 2*u(M-1,i) - u(M-2,i);&#xA;&#xA;    plot(x, u(:,i));&#xA;    axis( [ 0 1 0 2] )&#xA;    drawnow;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1724" LastActivityDate="2012-08-13T14:40:35.793" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3050" PostTypeId="1" AcceptedAnswerId="3051" CreationDate="2012-08-13T17:18:16.203" Score="3" ViewCount="224" Body="&lt;p&gt;Ignoring eigenvectors, the shifted QR algorithm for computing eigenvalues in the symmetric tridiagional case costs $O(n)$ per iteration, converges globally, and converges cubically near the end.  What is its worst case complexity (which presumably depends on the desired error)?&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-08-14T07:02:21.083" Title="What is the worst case complexity of the symmetric tridiagonal QR eigenvalue algorithm?" Tags="|complexity|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3051" PostTypeId="2" ParentId="3050" CreationDate="2012-08-13T18:06:49.117" Score="4" Body="&lt;p&gt;With the right shift stategy (mixing RQ and W shift appropriately), it is globally cubically convergent, with a guaranteed linear convergence factor, which makes its complexity of the order of $N$ times the logarithm of the off-diagonal entries. Unfortunately, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://www.springerlink.com/content/f1w76x84463l52t7/&quot; rel=&quot;nofollow&quot;&gt;http://www.springerlink.com/content/f1w76x84463l52t7/&lt;/a&gt; for local cubic convergence.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-08-14T07:02:21.083" LastActivityDate="2012-08-14T07:02:21.083" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3052" PostTypeId="2" ParentId="3043" CreationDate="2012-08-13T19:25:46.407" Score="12" Body="&lt;p&gt;Sloede's response is very thorough and correct. I just wanted to add a few points to make it easier to grasp. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, any wave equation has an inherent wave speed and direction. For a one-dimensional wave equation:  $$ u_t + a u_x = 0 $$ the wave speed is the constant $a$ which determines not only the speed at which the information is propagating in the domain but also its direction. If $a&amp;gt;0$, information is going from left to right and if $a&amp;lt;0$ it's the other way around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the leap frog method, when you discretize the equations you get: $$\frac{u_i^{n} - u_i^{n-2}}{2 \Delta t} + a \frac{u_{i+1}^{n-1} - u_{i-1}^{n-1}}{2 \Delta x} = 0$$&#xA;or:&#xA;$$u_i^{n} = u_i^{n-2} + \mu (u_{i+1}^{n-1} - u_{i-1}^{n-1}) $$ where $\mu = -a \Delta t/\Delta x$. In your case, $\mu &amp;gt; 0$ which translates to a wave going to the left. Now if you think about it, a wave that is traveling to the left, will only need a boundary condition at the right boundary since all the values to the left are being updated via their right neighbors. In fact specifying any value at the left boundary is inconsistent with the nature of the problem. In certain methods, like simple upwind, this is taken care of automatically since the scheme also involves only the correct neighbors in it's stencil. In other methods, like the leap frog, you have to specify some &quot;correct&quot; value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is usually done via extrapolation from the inside domain to find the missing value. For multi-dimensional and non-canonical  problems, this involves finding all the eigen-vectors of the flux Jacobian to determine which parts of the boundary actually need boundary conditions and which parts require extrapolation. &lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="20688" LastEditDate="2018-03-28T08:37:53.887" LastActivityDate="2018-03-28T08:37:53.887" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3053" PostTypeId="2" ParentId="83" CreationDate="2012-08-13T20:28:04.620" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://esa.github.io/pygmo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyGMO&lt;/a&gt; contains several solvers, providing the same interface to them. IPOPT and scipy slsqp are included in case you compile the code and download/install the third party code independently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a bonus, parallel use of the solver is made really easy (multistart) via the archipelago class!&lt;/p&gt;&#xA;" OwnerUserId="1903" LastEditorUserId="26237" LastEditDate="2017-12-05T21:31:44.250" LastActivityDate="2017-12-05T21:31:44.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3054" PostTypeId="2" ParentId="3030" CreationDate="2012-08-13T20:37:00.443" Score="1" Body="&lt;p&gt;I am not exactly familiar with the standard chemistry curriculum, so I don't really know what people are taught in a molecular dynamics course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a &lt;em&gt;very&lt;/em&gt; good read is &lt;a href=&quot;http://www.weizmann.ac.il/chemphys/tannor/&quot; rel=&quot;nofollow&quot;&gt;David Tannor&lt;/a&gt;'s book &quot;Introduction to Quantum Mechanics - a time-dependent perspective&quot;. I haven't read the full book, however, all parts I went through are excellent. Tannor introduces numerical methods (focusing on the time-dependent Schrödinger equation) and gives a chapter on molecular dynamics. However, I cannot say how the chapter on molecular dynamics relates the numerical methods chapter. The book also goes into semiclassical methods (in an excellent way).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in conclusion, the book has good material on all keywords that you gave in your question, however, as someone how does not work in this field, I cannot reliably judge how valuable Tannor's exposition will be for you specifically.&lt;/p&gt;&#xA;" OwnerUserId="1553" LastActivityDate="2012-08-13T20:37:00.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3055" PostTypeId="1" AcceptedAnswerId="3060" CreationDate="2012-08-13T21:31:07.127" Score="12" ViewCount="1115" Body="&lt;p&gt;I am producing 1d and 2d images using Matlab right now for comparison of accuracy against a given model. I need to compare my methods with the standard Gaussian .wfn model and I am going to do that by analyzing the density of a molecule as well as the laplacian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in producing 2-D images of differences in my approximation to the .wfn approximation (possibly via contour map?) and am also interested in properties (density, laplacian, etc) along the bond path and direct line between two atoms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I previously stated, I am currently using Matlab to produce my images, though these images are mostly for me and my associates. They do not seem to be the types of images I often see in papers and other publications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of software should I be using (or do others use) to produce publication-quality graphs and images?&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2015-07-31T02:59:18.173" Title="good (free) software for producing publishable images?" Tags="|matlab|computational-chemistry|publications|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3056" PostTypeId="2" ParentId="3055" CreationDate="2012-08-13T22:21:46.900" Score="7" Body="&lt;p&gt;I oftem use Matlab and then save the result as an &lt;code&gt;eps&lt;/code&gt; file via &lt;code&gt;print -depsc2 -f1 fig&lt;/code&gt;. This saves figure 1 as an &lt;code&gt;eps&lt;/code&gt; file named &lt;code&gt;fig.eps&lt;/code&gt;. There is also the &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/7401&quot;&gt;Plot2Svg&lt;/a&gt; from matlab file exchange that can save as &lt;code&gt;svg&lt;/code&gt;. This is really nice since you can then open the files in &lt;code&gt;Inkscape&lt;/code&gt; or other vector graphics software and edit them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are willing to learn new tools, there is also &lt;a href=&quot;http://matplotlib.sourceforge.net/&quot;&gt;matplotlib&lt;/a&gt; which is basically a python package. I have not used it personally, but many friends of mine do use it and tend to produce really nice graphs. You can check their gallery.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you are willing to pay for the software, you could use &lt;a href=&quot;http://www.tecplot.com/&quot;&gt;TecPlot&lt;/a&gt; but that'll cost you an arm and a leg!&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="485" LastEditDate="2012-08-13T22:29:11.573" LastActivityDate="2012-08-13T22:29:11.573" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3057" PostTypeId="1" CreationDate="2012-08-14T04:22:13.233" Score="5" ViewCount="4585" Body="&lt;p&gt;I've been working in R but sometimes switching to python. I'd like a more inter-language portable way of storing a large array than a csv file. (The particular csv file I'm dealing with is about 10^6 rows by 10^3 columns, but only about 1% of the entries are non-zero.) Processing a large csv file everytime I start up R or Python takes far too long. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard HDF5 is a great solution for this, but I have limited experience with it. Is HDF5 appropriate for storing non-hierarchical array data? What about for a single sparse array? I also am unsure which HDF5 R package to use. &lt;/p&gt;&#xA;" OwnerUserId="1420" LastEditorUserId="9" LastEditDate="2012-08-14T08:30:09.897" LastActivityDate="2012-08-15T21:37:29.907" Title="Storing a large, sparse array for R and Python" Tags="|python|sparse-matrix|data-management|data-storage|r|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3058" PostTypeId="2" ParentId="3057" CreationDate="2012-08-14T05:43:40.213" Score="2" Body="&lt;p&gt;Assuming your sparse array is 2-dimensional, you can decompose it into three vectors of column (index), row (index), and value fairly easily with a single traversal of the matrix.  You can then store these vectors in whatever file format you want, no need to switch to HDF5 for that reason alone.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-08-14T05:43:40.213" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3059" PostTypeId="2" ParentId="3055" CreationDate="2012-08-14T07:19:59.853" Score="7" Body="&lt;p&gt;If you're willing to overcome the learning curve of &lt;a href=&quot;http://www.gnuplot.info&quot;&gt;Gnuplot&lt;/a&gt;, you can generate top-quality figures with it (as exemplified in &lt;a href=&quot;http://www.gnuplotting.org/&quot;&gt;http://www.gnuplotting.org/&lt;/a&gt;).  In particular, I'd recommend using the epslatex terminal as it allows you to use LaTeX to render all the text in the figure (including any mathematical notation that you may wish to input).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to stick to MATLAB, you may want to try &lt;a href=&quot;http://win.ua.ac.be/~nschloe/content/matlab2tikz/&quot;&gt;matlab2tikz&lt;/a&gt;.  This is a script that converts MATLAB figures to &lt;a href=&quot;http://www.texample.net/tikz/examples/all/&quot;&gt;Tikz illustrations&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2012-08-14T07:25:59.253" LastActivityDate="2012-08-14T07:25:59.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3060" PostTypeId="2" ParentId="3055" CreationDate="2012-08-14T07:35:12.327" Score="15" Body="&lt;p&gt;I personally have come the way from &lt;a href=&quot;http://www.gnuplot.info/&quot;&gt;Gnuplot&lt;/a&gt; to &lt;a href=&quot;http://matplotlib.sourceforge.net/&quot;&gt;Matplotlib&lt;/a&gt; with &lt;a href=&quot;http://pgfplots.sourceforge.net/&quot;&gt;PGFPlots&lt;/a&gt; as an intermediate step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will try to name all aspects of Matplotlib that I like.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It is very versatile. You are not limited to line or scatter plots, you can easily do bar plots, images (matrix visualization!), basic 3D plotting and even some animation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use Matplotlib as a GUI for your Python applications.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;LaTeX is supported.&lt;/li&gt;&#xA;&lt;li&gt;It supports many output formats (try to get pain-free PDF with Gnuplot).&lt;/li&gt;&#xA;&lt;li&gt;Matplotlib's user base is broad. You get reasonably much help on the internet.&lt;/li&gt;&#xA;&lt;li&gt;It reasonably easy to learn.&lt;/li&gt;&#xA;&lt;li&gt;Being a Python library, you have the power of Python at hand (which means for example that you can easily load text or HDF5 data or, using wrappers such as Cython/f2py, load binary data written by compiled programs written in C/Fortan).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, I have always loved figures that integrate seamlessly into LaTeX documents. Here, in principle, PGFPlots excels. However, it is slow and (compared to Matplotlib) limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's why I have come to prefer a new solution: the (experimental) &lt;a href=&quot;https://github.com/pwuertz/matplotlib-backend-pgf/&quot;&gt;PGF backend&lt;/a&gt; for Matplotlib. Using this, the plots themselves are created by Matplotlib, whereas the axis and labels are set up by PGFPlots and finally drawn by LaTeX. You can use this for either PDF output without an embedding LaTeX document or for PGF output you can include in your LaTeX document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the feeling that the solution I described above is &lt;em&gt;the&lt;/em&gt; solution for my use cases (for the time being).&lt;/p&gt;&#xA;" OwnerUserId="1553" LastEditorUserId="1553" LastEditDate="2013-04-23T14:21:46.023" LastActivityDate="2013-04-23T14:21:46.023" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3061" PostTypeId="2" ParentId="3057" CreationDate="2012-08-14T07:44:50.977" Score="4" Body="&lt;p&gt;If you are interested in a &quot;standardized&quot; format, you should have a look at the &lt;a href=&quot;http://math.nist.gov/MatrixMarket/formats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;matrix market exchange formats&lt;/a&gt;. The &quot;coordinate format&quot; (which is good for sparse matrices) simply adds metadata to the format suggested by Aron in his &lt;a href=&quot;https://scicomp.stackexchange.com/a/3058/1744&quot;&gt;answer&lt;/a&gt; and specifies how the data has to be formatted.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-08-14T07:53:02.357" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3062" PostTypeId="1" AcceptedAnswerId="3065" CreationDate="2012-08-14T09:41:45.303" Score="15" ViewCount="2437" Body="&lt;p&gt;I would like to visualize simulation results, obtained using the discontinuous Galerkin (DG) approach, within ParaView. Similarly to finite volume methods, the problem domain is divided into cube-shaped cells (&quot;elements&quot;). As opposed to finite volume methods, within each cell there is not just one value for the solution vector $\mathbf{u}$, but each cell contains the solution $\mathbf{u}$ at multiple Gauss integration points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is whether anyone has experience with visualizing such data efficiently with ParaView/VTK, and what approach you chose to represent the data in VTK. Several possible ways come to my mind, but I do not know which one is the most promising:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(1) Use voxels&lt;/strong&gt;&lt;br&gt;&#xA;Use one voxel for each integration point.&lt;br&gt;&#xA;&lt;em&gt;Pro:&lt;/em&gt; All plugins that work with the standard VTK unstructured cell types will continue to work without changing anything.&lt;br&gt;&#xA;&lt;em&gt;Con:&lt;/em&gt; Since the integration points are not distributed evenly, it might be difficult to find the correct location of the vertices. Also, the solution can be defined twice on the cell surfaces, as the DG framework allows discontinuous solutions. Also, the hierarchical information (domain divided into elements, each element contains several points) is lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(2) Use polyvertices&lt;/strong&gt;&lt;br&gt;&#xA;Use one vertex per integration point.&lt;br&gt;&#xA;&lt;em&gt;Pro:&lt;/em&gt; Easiest to implement, easy to specify multiple points at the same location with different solutions.&lt;br&gt;&#xA;&lt;em&gt;Con:&lt;/em&gt; Capability to visualize data as &quot;cells&quot; is lost, plus the same disadvantages as above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(3) Use VTK quadrature scheme&lt;/strong&gt;&lt;br&gt;&#xA;Use the built-in support for quadrature schemes.&lt;br&gt;&#xA;&lt;em&gt;Pro:&lt;/em&gt; Rather straightforward implementation, preserves all relations and properties of the original solution.&lt;br&gt;&#xA;&lt;em&gt;Con:&lt;/em&gt; Since this is a completely new cell type, many (most) of the existing plugins will not work anymore and will probably have to be rewritten.&lt;/p&gt;&#xA;" OwnerUserId="1724" LastEditorUserId="9" LastEditDate="2012-08-15T12:42:08.430" LastActivityDate="2018-05-31T13:33:31.757" Title="Visualizing discontinuous Galerkin/finite element data" Tags="|finite-element|visualization|discontinuous-galerkin|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3063" PostTypeId="2" ParentId="3031" CreationDate="2012-08-14T10:10:12.887" Score="2" Body="&lt;p&gt;Chapter 44, &quot;Approximation of Discrete Data&quot;, in Advanced Engineering Mathematics by Robert J. Lopez gives the following algorithm for least squares regression:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;d = 6;&#xA;x = [1, 2, 3, 4, 5, 6, 7]&#xA;y = [3, 4, 1, 3, 2, 4, 9]&#xA;p = [0:1:d]&#xA;for i=1:length(y)&#xA;    for j=1:length(p)&#xA;        A1(i,j)=x(i)^p(j);&#xA;    end&#xA;end&#xA;A1&#xA;A2 = A1'&#xA;B0=y&#xA;for k=1:length(y)-1&#xA;    B0=cat(1,B0,y);&#xA;end&#xA;B0&#xA;B1=B0'&#xA;A3 = A2*B1&#xA;C1 = A2*A1&#xA;C2 = inv(C1)&#xA;C3 = C2*A3&#xA;C4 = C3'&#xA;pol = C4(1,1:length(p))&#xA;C5 = A1*C4&#xA;yreg = sum(C5,2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which I have written in Scilab. Should work in Matlab since Scilab is a Matlab clone.&#xA;&quot;pol&quot; is the coefficients of the polynomial of degree &quot;d&quot; that best fits the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Mathematica:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(*&quot;d&quot; is the degree of the polynomial to be fitted to x,y pairs*)&#xA;d = 6;&#xA;x = {1, 2, 3, 4, 5, 6, 7};&#xA;y = {3, 4, 1, 3, 2, 4, 9};&#xA;p = Range[d + 1] - 1;&#xA;A1 = Table[Table[x[[i]]^p[[j]], {j, 1, Length[p]}], {i, 1, Length[y]}];&#xA;A2 = Transpose[A1];&#xA;B1 = Transpose[Table[y, {i, 1, Length[y]}]];&#xA;A3 = A2.B1;&#xA;C1 = A2.A1;&#xA;C2 = Inverse[C1];&#xA;C3 = C2.A3;&#xA;C4 = Transpose[C3];&#xA;Print[&quot;polynomial&quot;]&#xA;pol = Total[C4[[1, All]]*(z^p)]&#xA;g1 = ListPlot[&#xA;   Prepend[Table[{x[[i]], y[[i]]}, {i, 1, Length[x]}], {0, 0}], &#xA;   PlotMarkers -&amp;gt; Automatic];&#xA;g2 = Plot[pol, {z, 0, Length[x] + 1}];&#xA;Show[g1, g2, PlotRange -&amp;gt; {Min[y] - 1, Max[y] + 1}, ImageSize -&amp;gt; Full]&#xA;C5 = A1*C4;&#xA;yreg = Total[Transpose[C5]];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to plot the polynomials for different values of the polynomial degree parameter &quot;d&quot; we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d=0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;polynomial of degree 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 26/7 = 3.71429&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2rL4F.png&quot; alt=&quot;degree zero polynomial or average of y values of data&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d=1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;polynomial of degree 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 1 + (19 z)/28&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 1. + 0.678571 z&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nputk.png&quot; alt=&quot;polynomial of degree 1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d=2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;polynomial of degree 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 46/7 - (85 z)/28 + (13 z^2)/28&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 6.57143 - 3.03571 z + 0.464286 z^2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gMHt5.png&quot; alt=&quot;polynomial of degree 2&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d=3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;polynomial of degree 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 11/7 + (335 z)/126 - (101 z^2)/84 + (5 z^3)/36&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 1.57143 + 2.65873 z - 1.20238 z^2 + 0.138889 z^3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/f6o7x.png&quot; alt=&quot;polynomial of degree 3&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d=4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;polynomial of degree 4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 2 + (2743 z)/1386 - (7 z^2)/8 + (31 z^3)/396 + z^4/264&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= 2. + 1.97908 z - 0.875 z^2 + 0.0782828 z^3 + 0.00378788 z^4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/3L0DJ.png&quot; alt=&quot;polynomial of degree 4&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d=5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;polynomial of degree 5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= -(107/7) + (7947 z)/220 - (571 z^2)/24 + (3631 z^3)/528 - (241 z^4)/264 + (11 z^5)/240&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= -15.2857 + 36.1227 z - 23.7917 z^2 + 6.87689 z^3 - 0.912879 z^4 + 0.0458333 z^5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FnM5j.png&quot; alt=&quot;polynomial of degree 5&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;d=6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;polynomial of degree 6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= -110 + (5093 z)/20 - (4179 z^2)/20 + (3965 z^3)/48 - (815 z^4)/48 + (419 z^5)/240 - (17 z^6)/240&lt;/p&gt;&#xA;&#xA;&lt;p&gt;= -110. + 254.65 z - 208.95 z^2 + 82.6042 z^3 - 16.9792 z^4 + 1.74583 z^5 - 0.0708333 z^6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/MYjhO.png&quot; alt=&quot;polynomial of degree 6&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevermind the dummy point {0,0} in origo. It is just there to make the plots start at zero which I did not know how to do in a better way. {x,y} = {0,0} is not part of the data used in the least squares regression.&lt;/p&gt;&#xA;" OwnerUserId="1899" LastEditorUserId="1899" LastEditDate="2015-09-13T08:54:05.153" LastActivityDate="2015-09-13T08:54:05.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3064" PostTypeId="2" ParentId="3043" CreationDate="2012-08-14T11:03:05.517" Score="2" Body="&lt;p&gt;So I've looked at this in some more detail, and it seems that this (at least in the basic cases which I'm handling) depends on the group velocity of the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The leapfrog method (for instance) is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{i}^{n+1} = u_{i}^{n-1} + \mu (u_{i+1}^n - u_{i-1}^n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying solutions of the form $u_k^n = e^{i( \zeta k \Delta x + \omega(\zeta) n \Delta t)}$ we find:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e^{2 i \omega \Delta t} = 1 + \mu e^{i\omega \Delta t} ( e^{i \zeta \Delta x} - e^{-i \zeta \Delta x} )$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sin(\omega \Delta t) = \mu \sin (\zeta \Delta x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{d\omega}{d\zeta} = \frac{\cos (\zeta \Delta x)}{\sqrt{1-\mu^2 sin^2(\zeta \Delta x)}} \in [-1,1]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we need to find out the group velocity of the boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My method&lt;/strong&gt;: $u_1^{n+2} = u_1^n + \mu u_2^{n+1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can compute the boundary group velocity as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$2 i \sin(\omega \Delta t) = \mu e^{i \zeta \Delta x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so to find some group velocities which the boundaries allow we need to find:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega = c \zeta$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\cos(\zeta \Delta x) = 0, \mu \sin(\zeta \Delta x) = 2 \sin (\zeta c \Delta t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\zeta = \frac{\pi}{2 \Delta x}$ would give $\mu = 2 \sin (\frac{c \mu \pi}{2})$ for which a solution for $c \in [-1,1]$ will exist. (For most choices of $\mu$ at least)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The solution which I've found in the literature is to take $u_0^{n+1} = u_1^{n}$ since this has a boundary wave number which lies outside $[-1,1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've still quite quite a bit more to read up about this before I understand it completely. I think the key words I'm looking for are GKS theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source for all this A Iserles Part III notes&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A clearer calculation of what I've done can be found here: &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/publication/PDF/1983_7.pdf&quot; rel=&quot;nofollow&quot;&gt;http://people.maths.ox.ac.uk/trefethen/publication/PDF/1983_7.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1898" LastEditorUserId="1898" LastEditDate="2012-08-14T11:08:31.377" LastActivityDate="2012-08-14T11:08:31.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3065" PostTypeId="2" ParentId="3062" CreationDate="2012-08-14T17:33:32.423" Score="7" Body="&lt;p&gt;deal.II uses option (2) for a long time already with very good success. In other words, in 2d, every vertex appears 4 times in the output file which means there is more data to be written but fields can be discontinuous. The only other drawback I am aware of is that some visualization programs can't follow interfaces between cells that don't share vertices when doing streamlines, though today they seem to have become better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also want to look at this entry in our FAQs:&#xA;&lt;a href=&quot;http://dealii.sourceforge.net/index.php/Deal.II_Questions_and_Answers#The_graphical_output_files_don.27t_make_sense_to_me_.E2.80.94_they_seem_to_have_too_many_degrees_of_freedom.21&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dealii.sourceforge.net/index.php/Deal.II_Questions_and_Answers#The_graphical_output_files_don.27t_make_sense_to_me_.E2.80.94_they_seem_to_have_too_many_degrees_of_freedom.21&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update: May 31, 2018&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The link has been moved to: &lt;a href=&quot;https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#the-graphical-output-files-dont-make-sense-to-me----they-seem-to-have-too-many-degrees-of-freedom&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#the-graphical-output-files-dont-make-sense-to-me----they-seem-to-have-too-many-degrees-of-freedom&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="9667" LastEditDate="2018-05-31T13:33:31.757" LastActivityDate="2018-05-31T13:33:31.757" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3066" PostTypeId="1" AcceptedAnswerId="3069" CreationDate="2012-08-14T21:31:50.200" Score="9" ViewCount="1171" Body="&lt;p&gt;Leafing through a few textbooks, I've noticed that the problem of initially bracketing a minimum during a line search tends be an afterthought (at least in my undergraduate texts). Are there well-established techniques or best practices for this type of problem, or are solutions typically application dependent? Can anyone recommend some references on the topic?&lt;/p&gt;&#xA;" OwnerUserId="1862" LastEditorUserId="9" LastEditDate="2012-08-15T07:52:23.003" LastActivityDate="2012-08-15T09:39:11.513" Title="Initially Bracketing Minimum for Line Search" Tags="|optimization|reference-request|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3067" PostTypeId="2" ParentId="3066" CreationDate="2012-08-15T07:56:53.060" Score="5" Body="&lt;p&gt;In my experience establishing the bracket is very often application-dependent.  If you had real constraints or an algebraic derivation for your bracket, you'd use it of course!  Usually there's an appeal to either &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;nature&lt;/em&gt; this physically makes no sense outside this bracket&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;computability&lt;/em&gt; this would be too hard to compute outside of the bracket&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;objective&lt;/em&gt; solutions outside of this region are otherwise undesirable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm hoping someone else can come in with a more algorithmic approach, which is what I think you're looking for here.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-08-15T09:37:53.700" LastActivityDate="2012-08-15T09:37:53.700" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3068" PostTypeId="1" AcceptedAnswerId="3070" CreationDate="2012-08-15T09:12:41.983" Score="6" ViewCount="319" Body="&lt;p&gt;Lapack contains a driver routine to solve dense generalized Hermitian positive definite eigenvalue problems of the form $Ax=\lambda Bx$, where $A$ and $B$ are both Hermitian, and $B$ is positive definite. I am wondering if there is a method (either code or a modification of &lt;code&gt;zhegv&lt;/code&gt;) when $B$ is indefinite (specifically, $B$ will almost certainly be non-singular). I know that the driver routine first performs a Cholesky decomposition of positive definite $B$, and transforms it into a standard eigenvalue problem, but does this still work when $B$ is indefinite? I would rather use a specialized method instead of the fully general solver because I want the realness of the eigenvalues and orthogonality relations for the eigenvectors to be guaranteed.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-08-16T07:27:59.117" Title="Dense generalized hermitian indefinite eigenvalue problem" Tags="|eigensystem|lapack|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3069" PostTypeId="2" ParentId="3066" CreationDate="2012-08-15T09:39:11.513" Score="9" Body="&lt;p&gt;Usually one doubles the initial step until the Goldstein condition is violated or (in a feasible point method) the boundary is reached. Then one has a bracket. (If no such step exists, the objective function is unbounded below.) One can also use less conservative extrapolation procedures, but these require good tuning to be robust enough in a general purpose solver.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-08-15T09:39:11.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3070" PostTypeId="2" ParentId="3068" CreationDate="2012-08-15T09:40:57.440" Score="5" Body="&lt;p&gt;When $B$ is indefinite, the eigenvalues may be complex, and there is little advantage exploiting the symmetry. Instead, one generally uses the QZ algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: If you know a priori that all your eigenvalues are real, it is probably because you can establish a priori that some linear combination $C=sA+tB$ is positive definite. In this case, you should consider your problem with either $A$ or $B$ replaced by $C$. It has the same eigenvectors, and the eigenvalues are related by a Moebius transform.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-08-16T07:27:59.117" LastActivityDate="2012-08-16T07:27:59.117" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3071" PostTypeId="1" AcceptedAnswerId="3072" CreationDate="2012-08-15T12:25:15.863" Score="3" ViewCount="98" Body="&lt;p&gt;Given a symmetric matrix $A\in\mathbb{R}^{n\times n}$ with positive entries and zero diagonal, is it always possible to construct a high-dimensional configuration in Euclidean space, such that these coordinates give rise to distances from $A$?&#xA;(high-dimensional configuration implies dimensionality at most $n-1$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is generally not possible, what would be the conditions that have to be met in order to achieve a high-dimensional configuration giving rise to distances in $A$?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-09-12T12:50:23.237" LastActivityDate="2012-09-12T12:50:23.237" Title="High-dimensional representation of arbitrary input" Tags="|linear-algebra|matrix|high-dimensional|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3072" PostTypeId="2" ParentId="3071" CreationDate="2012-08-15T13:27:27.453" Score="3" Body="&lt;p&gt;From suitable linear combinations of the entries of the distance matrix (the matrix of &lt;em&gt;squared&lt;/em&gt; distances) one can construct the Gram matrix of the direction vectors from one point (taken as zero) to the others. To be embeddable into a Euclidean space of dimension $r$, this matrix must be positive semidefinite of rank $r$; then the Cholesky factor provides an explicit embedding. (This is old stuff by Schoenberg, &lt;a href=&quot;http://www.jstor.org/stable/10.2307/1989894&quot; rel=&quot;nofollow&quot;&gt;http://www.jstor.org/stable/10.2307/1989894&lt;/a&gt;; see also my paper &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/scan/16.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/scan/16.pdf&lt;/a&gt; and &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/scan/combcon.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/scan/combcon.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one allows a monotone transformation of the distances, an embedding is always possible: Simply add a large constant to all distances between distinct elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related is the technique of multidimensional scaling;&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Multidimensional_scaling&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Multidimensional_scaling&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-08-15T13:34:28.723" LastActivityDate="2012-08-15T13:34:28.723" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3075" PostTypeId="2" ParentId="3057" CreationDate="2012-08-15T21:37:29.907" Score="2" Body="&lt;p&gt;I happened to have this little Python function kicking around:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def storeSparseProblem(sp_mat, rhs, filename):&#xA;    # Convert the sparse matrix to lil&#xA;    sp_mat = sp_mat.tocoo()&#xA;&#xA;    # Make a record array out of the above&#xA;    rcv = recarray((3, len(sp_mat.row)), dtype=[(&quot;row&quot;,&quot;&amp;lt;i8&quot;), (&quot;col&quot;,&quot;&amp;lt;i8&quot;), (&quot;val&quot;,&quot;&amp;lt;f8&quot;)])&#xA;&#xA;    # Set the values&#xA;    rcv[&quot;row&quot;] = sp_mat.row&#xA;    rcv[&quot;col&quot;] = sp_mat.col&#xA;    rcv[&quot;val&quot;] = sp_mat.data&#xA;&#xA;    # Open the output h5&#xA;    h5 = openFile(filename, &quot;w&quot;)&#xA;&#xA;    # Save the matrix&#xA;    h5.createTable(&quot;/&quot;, &quot;M&quot;, rcv, title=&quot;The Matrix&quot;)&#xA;&#xA;    # Add the rhs&#xA;    ary = h5.createArray(&quot;/&quot;, &quot;rhs&quot;, rhs, title=&quot;the right hand side&quot;)&#xA;&#xA;    # Close it out&#xA;    h5.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You could make this faster by turning on tables's magic &lt;a href=&quot;http://blosc.pytables.org/trac&quot; rel=&quot;nofollow&quot;&gt;blosc&lt;/a&gt; compression, and switching from a table to a carray, but this should at least get you started.  The load function is really simple, but I can't seem to locate it.  If someone has interest, I will code it up and post it.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-08-15T21:37:29.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3076" PostTypeId="2" ParentId="1864" CreationDate="2012-08-16T09:38:18.690" Score="2" Body="&lt;p&gt;If the eigenvalues are already known (from a preliminary calculation based on the usual approach), one can use them to triangulize a nonsymmetric matrix (or diagonalize a symmetric matrix) by a product on $n-1$ Householder reflections. In the $k$th step the $k$th column is brought to triangular form. (This also provides a simple inductive proof of the existence of the Schur factorization.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is actually useful for methods where one repeatedly needs the orthoginal matrix in a numerically stable factored form.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-08-16T09:38:18.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3077" PostTypeId="1" AcceptedAnswerId="3079" CreationDate="2012-08-16T12:26:03.410" Score="3" ViewCount="152" Body="&lt;p&gt;I have calculated estimates of the daily supply of empty trucks and daily demand for empty trucks at the US postal code level. I would like to optimize the routing of supply to demand to minimize the total distance travelled. I would use that result to calculate the average repositioning miles required for a unit of demand in each postal code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some additional info about the problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are approximately ~29,000 postal codes with either supply or demand.&lt;/li&gt;&#xA;&lt;li&gt;I'd like to vary the minimum demand that must be met, but it will always be close to 100%.&lt;/li&gt;&#xA;&lt;li&gt;A 100% optimal solution is not necessary.&lt;/li&gt;&#xA;&lt;li&gt;I'd like to solve this problem monthly from within a ruby program.&lt;/li&gt;&#xA;&lt;li&gt;Solving speed isn't extremely important.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How should I formulate the problem?&lt;/p&gt;&#xA;" OwnerUserId="1914" LastActivityDate="2012-08-16T13:54:59.727" Title="How should I calculate the average repositioning distance for empty trucks given known supply and demand at the postal code level?" Tags="|optimization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3078" PostTypeId="1" AcceptedAnswerId="3088" CreationDate="2012-08-16T13:52:01.977" Score="3" ViewCount="132" Body="&lt;p&gt;I trying to detecting whether a data set of time series has a global change in frequencies. Calculating the average (or median) pairwise coherence, I feel, misses the point because I am trying to get a measure of network (i.e. multivariate, more than pairwise) coherence. I found a group[1] that uses the relative magnitude of the first singular value to all others to calculate what they call &quot;global coherence&quot;. The singular values come from a singular value decomposition (SVD) of a matrix of pairwise coherences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching online didn't show up much more. Any thoughts?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1109/IEMBS.2011.6091170&quot; rel=&quot;nofollow&quot;&gt; &lt;strong&gt;Wong KFK, Mukamel EA, Salazar AF, Pierce ET, Harrell PG, Walsh JL, Sampson A, Brown EN, Purdon PL&lt;/strong&gt;. 2011. Robust time-varying multivariate coherence estimation: application to electroencephalogram recordings during general anesthesia. Conference proceedings : Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Conference 2011: 4725–8.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1867" LastEditorUserId="463" LastEditDate="2012-08-19T18:35:19.663" LastActivityDate="2012-08-19T18:35:19.663" Title="Calculation of Multivariate Coherence" Tags="|matrix|data-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3079" PostTypeId="2" ParentId="3077" CreationDate="2012-08-16T13:54:59.727" Score="2" Body="&lt;p&gt;Since you said you have the distances between all the nodes, one possibility would be linear programming.  Formulating the program would be fairly straight forward.  Your objective would be to minimize $\sum{c_{ij}t_{ij}}$ where $t_{ij}$ is the number of trucks sent from node i to node j and $c_{ij}$ is the distance.  You would only have two constraints.  The first constraint would be that $\sum_{\forall i}t_{ij}\ge D_{j}$ which states that the number of trucks from all sources to node j is greater than the demand at j.  You could formulate a similar constraint for supply at each node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear programming approach would also guarantee a global optimum for you, but there is a glaring issue with the above formulation.  Your decision variables are an integer for every pair of nodes.  That turns your 29,000 nodes into just shy of 900,000,000 decision variables, which isn't exactly a trivial computational problem.  Of course you could do things like remove pairs whose distance is above some threshold, which could greatly reduce the problem size, but make the implementation more complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for linear programming, I do not know what interfaces there are in ruby.  I personally learned using glpk (a free solver) and pyomo (a python framework).  The &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_programming#Solvers_and_scripting_.28programming.29_languages&quot; rel=&quot;nofollow&quot;&gt;wikipedia page&lt;/a&gt; has a nice list of different free libraries.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-08-16T13:54:59.727" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3080" PostTypeId="1" CreationDate="2012-08-16T18:48:19.733" Score="2" ViewCount="86" Body="&lt;p&gt;I need to create a scoring model out of estimated conditional probability functions for two events, A and B. Let 0.5 be the threshold value. Ideally, the probability is in the interval $[0,0.5)$ for A and in the interval $[0.5,1]$ for B. However, my probabilities are estimated with errors. Therefore, for a certain number of events A the probabilities fall into the interval $[0.5,1]$, and for B into $[0,0.5)$.&#xA;Let the minimum score &lt;code&gt;min_score&lt;/code&gt; $=0$ ($P = 0$) and maximum score `max_score' $= 10$ ($P = 1$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to map those probabilities to scores with a function that would emphasize and enlarge difference in probabilities of A and B in terms of scores (for example, a sigmoidal function). That is the higher the probability (likely event B) the higher the score, the lower the probability (likely event A), the smaller the score. Though for the intersection interval, which I can identify as $[a,b]$ the scores should be around &lt;code&gt;(max_score + min_score)/2&lt;/code&gt;. In the interval $[t_1,t_2]$, which is symmetric about $1/2$, scores should be smoothing, while in $(0, t_1)$ and $(t_2, 1)$, the scores should be &quot;more extreme&quot; with a higher confidence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is $[a,b]$ can be spanned over $[0,1]$, and that effect should be smoothed. Can I apply some kind of regularization here? What function would I use and how would I establish the coefficients?&lt;/p&gt;&#xA;" OwnerUserId="1916" LastEditorUserId="348" LastEditDate="2012-08-20T16:02:38.293" LastActivityDate="2012-08-20T16:02:38.293" Title="function over conditional probability" Tags="|probability|error-estimation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3081" PostTypeId="1" CreationDate="2012-08-17T00:05:47.033" Score="8" ViewCount="408" Body="&lt;p&gt;I appologize in advance if this question is silly.&#xA;I need to compute the root of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u -f(u) =0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u$ is a real vector and $f(u)$ is a real-vector valued function. &#xA;I started with Newton's method (which worked), but then realized a much simpler method would be an iterative solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_{i+1} = f(u_{i})&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is much quicker and apparently as accurate/stable as Newton's method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this the correct approach or should I use a different method?&lt;/li&gt;&#xA;&lt;li&gt;Is there anything that can be said about it's convergence rate, stability, acc, etc?&lt;/li&gt;&#xA;&lt;li&gt;Is it globally convergent?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you all in advance for the attention. &lt;/p&gt;&#xA;" OwnerUserId="1751" LastEditorUserId="1909" LastEditDate="2012-08-17T13:31:59.913" LastActivityDate="2012-10-11T18:15:54.717" Title="Iterative solution to a nonlinear equation" Tags="|iterative-method|nonlinear-equations|roots|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3082" PostTypeId="2" ParentId="3081" CreationDate="2012-08-17T00:44:07.320" Score="5" Body="&lt;p&gt;The Feigenbaum fractal is a good example of how strange fixpoint iteration can be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Feigenbaum_fractal&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Feigenbaum_fractal&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/File:Logistic_Bifurcation_map_High_Resolution.png&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/File:Logistic_Bifurcation_map_High_Resolution.png&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second link plots the behavior of fixpoint iteration applied to the logistic map as one of the parameters varies.  For certain values it converges, though only linearly.  For other values it converges to a cycle of varying length.  For yet another class of values, it behaves completely chaotically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the behavior of fixpoint iteration depends entirely on the function in question.  Even functions that look similar may exhibit radially different behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: As Jed points out, Newton iteration &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton_fractal&quot; rel=&quot;nofollow&quot;&gt;can be equally weird&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2012-08-19T03:54:33.173" LastActivityDate="2012-08-19T03:54:33.173" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3083" PostTypeId="2" ParentId="3081" CreationDate="2012-08-17T00:55:06.803" Score="1" Body="&lt;p&gt;This method is correct and it is called &quot;Successive Substitution&quot;. Please, look at page 189 of &lt;a href=&quot;http://samples.jbpub.com/9780763714994/Linz07.pdf&quot; rel=&quot;nofollow&quot;&gt;this reference&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="1909" LastActivityDate="2012-08-17T00:55:06.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3084" PostTypeId="1" CreationDate="2012-08-17T02:27:53.167" Score="4" ViewCount="1938" Body="&lt;p&gt;I am trying to iterate the following equation&#xA;$$&#xA;x_{k}(n+1)=x_k (n)-\epsilon (x_{k+1}(n)-2x_k(n) +x_{k-1}(n))+\sqrt{\epsilon}\; \eta_{k}(n)&#xA;$$&#xA;where $n$ denotes which time step I'm on and $k$ is the location on the string with periodic boundary conditions, $\epsilon$ is the size of the time step and $\eta$ is a random variate from a gaussian distribution with mean 0 and variance 1.  $\eta$ is re-picked after each time step and is varies from location to location. I generated my initial $x$ randomly with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x[0] = RandomReal[{0, π}, 10]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives me 10 real numbers between 0 and $\pi$, which are my 1D lattice site variables.  I then define $\eta$ by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;η[0] = RandomVariate[NormalDistribution[0, 1], 10]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives me 10 random numbers from my distribution. I am taking the time step to be $\epsilon =1/100$ Then to define my iteration equation I write&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x[[j]][n_ + 1] := x[[j]][n] - ϵ (x[[j + 1]][n] - 2 x[[j]][n] + x[[j + 1]][n]) + Sqrt[ϵ] η[[j]][n]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this failed to iterate when I tried when just looking at a sigle site and its neighbors.  If someone could be so kind as to help me put in periodic boundary conditions, as well as to get the iteration to work, I would be grateful.  I am using Mathematica 8.  Thank you.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="123" LastEditDate="2012-08-19T19:31:41.850" LastActivityDate="2013-01-16T21:48:32.320" Title="Implementing a finite difference method in Mathematica" Tags="|finite-difference|mathematica|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3085" PostTypeId="2" ParentId="3081" CreationDate="2012-08-17T08:22:43.623" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Banach_fixed-point_theorem&quot; rel=&quot;nofollow&quot;&gt;Banach fixed-point theorem&lt;/a&gt; describes the standard situation when a fixed-point iteration is globally convergent. Especially the uniqueness part of the theorem indicates that you can only expect local convergence if the solution is not unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most situations of local convergence can be explained by this theorem, at least in theory. This is even true for the convergence occurring in some of the fractals mentioned above. It's just that the theorem has to be applied to $f^n=f\circ \ldots\circ f$ instead of $f$ at some basins of attraction.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-08-17T08:22:43.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3086" PostTypeId="2" ParentId="3084" CreationDate="2012-08-17T09:35:54.193" Score="1" Body="&lt;p&gt;It's been a while since I coded something in mathematica so I'll write in fortranish pseudocode instead&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ntimesteps = 20&#xA;nx = 10&#xA;x = array(0:nx+1, 0:ntimesteps)&#xA;x[1:nx,0] = RandomReal(0,pi)&#xA;// boundary conditions:&#xA;x[0,0] = x[nx,0]&#xA;x[nx+1,0] = x[1,0]&#xA;do i=1,ntimesteps&#xA;  eta[1:nx] = RandomNormalDistribution(0,1)&#xA;  do j=1,nx&#xA;    x[j][i] = x[j,i-1] - eps(x[j-1,i-1]-2x[j,i-1]+x[j+1,i-1]) + sqrt(eps)eta[j]&#xA;  enddo&#xA;  x[0,i] = x[nx,i]&#xA;  x[nx+1,i] = x[1,i]&#xA;enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that in the formula of the iteration you gave there are two mistakes:&#xA;First use n_ instead of n_+1, secondly you have two j+1 instead of one j+1 and one j-1&lt;/p&gt;&#xA;" OwnerUserId="1176" LastActivityDate="2012-08-17T09:35:54.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3087" PostTypeId="2" ParentId="3081" CreationDate="2012-08-17T10:22:35.097" Score="7" Body="&lt;p&gt;If $q:=|f'(x^*)|&amp;lt;1$, where $x^*$ is the solution, the fixed point iteration you talk about is locally linearly convergent with convergence rate $q$. Thus if $q$ is small or zero, the method is competitive with Newton's method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Far away from the solution, convergence is difficult to predict in the absence of global information (such as a Lipschitz constant $&amp;lt;1$, which produces a contraction). &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-08-17T10:22:35.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3088" PostTypeId="2" ParentId="3078" CreationDate="2012-08-17T11:30:32.933" Score="0" Body="&lt;p&gt;It seems that, at the risk of writing a truism, any multivariate measure must be sensitive to all the independent components of a system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relative ratio of the first singular values to the sum of all singular values is one such measure. Another is the determinant of the matrix of singular values. Using SVD does assume that the system can be completely described by parts that are separable into linearly interacting entities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In sum, in as much as global coherence captures the essence of a multivariate measurement and is not too sensitive to noise other ones will just be variations on a theme rather than refinements.&lt;/p&gt;&#xA;" OwnerUserId="1867" LastActivityDate="2012-08-17T11:30:32.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3089" PostTypeId="2" ParentId="3033" CreationDate="2012-08-17T12:32:52.000" Score="0" Body="&lt;p&gt;Ilic et al (2007) wrote a paper to describe an alternative technique to solve a system of linear equations. The paper can be found at &lt;a href=&quot;http://dx.doi.org/10.1002%2fnla.509&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1002%2fnla.509&lt;/a&gt;. A portion of this algorithm is the computation of the nullspace of a large sparse matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I recall, the technique described is only exhibited when the nullity of $A$ is one. I do believe that the theoretical results hold for higher nullities.&lt;/p&gt;&#xA;" OwnerUserId="1918" LastActivityDate="2012-08-17T12:32:52.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3095" PostTypeId="2" ParentId="3084" CreationDate="2012-08-17T22:01:56.477" Score="2" Body="&lt;p&gt;In Mathematica I recommend do this with Sparse Matrix-Vector products as follows. First I will define a circulant matrix $C$ (with the periodic BCs) which appears on the RHS of your equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In matrix notation your equation reads&#xA;\begin{equation}&#xA;x(n+1) = x(n) + \epsilon C x(n) + \sqrt{\epsilon} \eta(n)&#xA;\end{equation}&#xA;or &#xA;\begin{equation}&#xA;x(n+1) = (I + \epsilon C) x(n) + \sqrt{\epsilon} \eta(n)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, here are the matrix definitions. The classical (-1 2 -1) stiffness matrix is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;StiffnessMatrix[n_] := SparseArray[{Band[{1, 1}] -&amp;gt; 2., Band[{1, 2}] -&amp;gt; -1., &#xA;   Band[{2, 1}] -&amp;gt; -1.}, {n, n}, 0.];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From this definition the circulant matrix is constructed with a simple adjustment.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CirculantMatrix[n_] := Block[{K = StiffnessMatrix[n]}, &#xA;   ReplacePart[K, {{1, n} -&amp;gt; -1., {n, 1} -&amp;gt; -1.}]];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a given $n$ (size of your system) we the define &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C = CirculantMatrix[n];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The identity matrix can be computed with IdentityMatrix[n]. Conversely, a sparse implementation reads &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eye[n_] := SparseArray[{{i_, i_} -&amp;gt; 1.}, {n, n}, 0.];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, to implement $A = (I + \epsilon C)$ we write &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = Eye[n] + eps C;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We are now ready to iterate: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = x + A.x + sqe eta;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $sqe = Sqrt[eps]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following implementation is for a noise $\eta$ which is independent of position. You can adjust it for your needs. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Iteration[nmax_, eps_, size_] := With[{&#xA;   A = Eye[size] + eps CirculantMatrix[size],&#xA;   x0 = RandomReal[{0, \[Pi]}, size],&#xA;   sqe = N@Sqrt[eps]&#xA;  },&#xA;    NestList[A.# + sqe RandomVariate[NormalDistribution[0, 1], size] &amp;amp;, x0, nmax]];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Input is the number of iterations &lt;em&gt;nmax&lt;/em&gt;, the $\epsilon$ guy &lt;em&gt;eps&lt;/em&gt; and the size of your system (which in you example is 10). Output is a matrix where each entry (e.g. &lt;code&gt;sol[[1]]&lt;/code&gt;) is the state of your system at a given time. &#xA;If you want only the last output, either use &lt;code&gt;Last&lt;/code&gt; in your output or replace &lt;code&gt;NestList&lt;/code&gt; with &lt;code&gt;Nest&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="1751" LastActivityDate="2012-08-17T22:01:56.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3097" PostTypeId="2" ParentId="351" CreationDate="2012-08-18T13:01:48.250" Score="2" Body="&lt;p&gt;Basically the same question popped up on SO:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/q/1380371/341970&quot;&gt;What are the most widely used C++ vector/matrix math/linear algebra libraries, and their cost and benefit tradeoffs?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(It adds some value to Geoff's answer.)&lt;/p&gt;&#xA;" OwnerUserId="679" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-08-18T13:01:48.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3099" PostTypeId="1" AcceptedAnswerId="3103" CreationDate="2012-08-18T14:43:00.347" Score="2" ViewCount="445" Body="&lt;p&gt;Consider a vector $\mathbf{g} \in \mathbb{R}^{m}$ and a matrix $\mathbf{A} \equiv \mathbf{A(g)} \in \mathcal{M}_{p\times q} [\mathbb{R}]$, a function of $\mathbf{g}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, let $\mathbf{S} \in \mathcal{M}_{p \times r} [\mathbb{R}]$, independent of $\mathbf{g}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it a way to calculate $\displaystyle \frac{\partial L(\mathbf{A}, \mathbf{S})}{\partial g_i}$, where $L(\cdot)$ is the &lt;code&gt;linsolve&lt;/code&gt; Matlab function? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can suppose that $\displaystyle \frac{\partial \mathbf{A}}{\partial g_i}$ is known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: $X =$ &lt;code&gt;linsolve&lt;/code&gt;$(A,B)$ solves the linear system $AX = B$ using LU factorization with partial pivoting when $A$ is square and QR factorization with column pivoting otherwise.&lt;/p&gt;&#xA;" OwnerUserId="1922" LastEditorUserId="123" LastEditDate="2012-08-19T07:22:28.717" LastActivityDate="2012-08-26T10:45:36.730" Title="derivative of linsolve" Tags="|matrix|matlab|linear-solver|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3101" PostTypeId="1" CreationDate="2012-08-18T18:10:52.573" Score="9" ViewCount="252" Body="&lt;p&gt;I want to minimize a complicated objective function, and I'm not sure if it is convex.  Is there a nice algorithm that attempts to prove that it is not convex?  Of course the algorithm could fail to prove this, in which case I would not know if it is convex or not, and this is OK; I just want to try to rule out convexity before I spend a lot of time trying to analytically determine whether the objective function is convex, for example by trying to rewrite the problem in a standard form known to be convex.  One quick test would be to try to minimize from various starting points and if multiple local minima are found in this manner then it is not convex.  But I was wondering if there is a better algorithm that was designed with this goal in mind.&lt;/p&gt;&#xA;" OwnerUserId="1921" LastActivityDate="2012-08-27T11:44:32.430" Title="How to intellligently attempt to rule out convexity?" Tags="|convex-optimization|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3102" PostTypeId="2" ParentId="3037" CreationDate="2012-08-18T18:20:45.237" Score="3" Body="&lt;p&gt;As mentioned in [1], Reif [2] gives an $O(n \log^2 n (\log n + \log \epsilon))$ algorithm for the case when all roots are real, which is $O(n \log^3 n)$ as long as $\epsilon = n^{O(1)}$.  He analyzes only the case of exact arithmetic, but after going through the algorithm I believe it is (or could be modified to be) backward stable.  His method is an improved variant of the splitting circle method specialized to the real line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the exact arithmetic case, this gives $O(n \log^3 n)$ algorithm for the Hermitian tridiagonal eigenproblem.  However, since the first step is to compute the characteristic polynomial, I believe the stability properties of this approach would be poor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He states that a similar efficient algorithm for the general complex root case is unknown, at least by 1999.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/1864/why-cant-householder-reflections-diagonalize-a-matrix&quot;&gt;Why can&amp;#39;t Householder reflections diagonalize a matrix?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cs.duke.edu/~reif/paper/roots.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reif, J. 1999. An efficient algorithm for the real root and symmetric tridiagonal eigenvalue problems.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="447" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-01-17T17:48:09.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3103" PostTypeId="2" ParentId="3099" CreationDate="2012-08-18T18:25:30.233" Score="4" Body="&lt;p&gt;Let's simplify slightly. Suppose the overdetermined $m\times n$ system $A$ admits the factorization $Q R = A$ with $R$ full rank. Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A^\dagger = R^{-1} Q^T$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;be the &lt;code&gt;linsolve&lt;/code&gt; solution operator and observe that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I = R^{-1} Q^T Q R  = A^\dagger A.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Differentiating the above, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 0 = \frac{\partial (A^\dagger A)}{\partial g} = \frac{\partial A^\dagger}{\partial g} A + A^\dagger \frac{\partial A}{\partial g} . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We cannot solve for $\frac{\partial A^\dagger}{\partial g}$ algebraically because it is not uniquely determined in this form, but we can compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial A^\dagger}{\partial g} Q Q^T = - A^\dagger \frac{\partial A}{\partial g} R^{-1} Q^T = - R^{-1} Q^T \frac{\partial A}{\partial g} R^{-1} Q^T = - A^\dagger \frac{\partial A}{\partial g} A^\dagger $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $QQ^T$ is the identity on the range of $A$.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Edit&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If we need the rest, $\frac{\partial A^\dagger}{\partial g} (I - Q Q^T)$, we can formally differentiate the QR factorization,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial A^\dagger}{\partial g} = \frac{\partial R^{-1}}{\partial g} Q^T + R^{-1} \frac{\partial Q^T}{\partial g},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and decompose into the parts in the range of $A$ (which yields expressions equivalent to that derived above) and in the left null space of $A$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align} \frac{\partial A^\dagger}{\partial g} (I - Q Q^T) &amp;amp;= \left[ \frac{\partial R^{-1}}{\partial g} Q^T + R^{-1} \frac{\partial Q^T}{\partial g} \right] (I - Q Q^T) \\&#xA;&amp;amp;= R^{-1} \frac{\partial Q^T}{\partial g} (I - Q Q^T) .&#xA;\end{align} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that only perturbations $dg$ that change $Q$ can contribute to this second term. We can rewrite in terms of $\frac{\partial A}{\partial g}$ by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;  \frac{\partial A^\dagger}{\partial g} (I - Q Q^T) &amp;amp;= R^{-1} R^{-T} \left[ R^T \frac{\partial Q^T}{\partial g} + \frac{\partial R^T}{\partial g} Q^T \right] (I - Q Q^T) \\&#xA;&amp;amp;= R^{-1} R^{-T} \frac{\partial A^T}{\partial g} (I - Q Q^T) \\&#xA;&amp;amp;= (A^T A)^{-1} \frac{\partial A^T}{\partial g} (I - A A^\dagger)&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the last line is the second term in Stefano's derivation, though the QR form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial A^\dagger}{\partial g} = R^{-1} \left[ -Q^T \frac{\partial A}{\partial g} R^{-1} Q^T + R^{-T} \frac{\partial A^T}{\partial g} (I - Q Q^T) \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is preferable to compute with for numerical stability and because it reuses more computation and operates in a smaller space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can apply similar arguments to rank-deficient and under-determined cases.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-08-19T17:07:23.873" LastActivityDate="2012-08-19T17:07:23.873" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3105" PostTypeId="2" ParentId="3101" CreationDate="2012-08-18T20:31:49.710" Score="2" Body="&lt;p&gt;A function can be non-convex without having multiple minima. There are a variety of optimization methods that apply (linear or nonlinear) conjugate gradient iterations, truncated when a negative operator norm is computed. The negative value indicates a direction of negative curvature (which cannot happen for convex functionals). If negative curvature is rarely encountered, these methods converge in a small number of optimization iterations. (If a quality preconditioner is available, the inner steps should also converge quickly.)&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-18T20:31:49.710" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3106" PostTypeId="2" ParentId="3101" CreationDate="2012-08-19T00:59:35.417" Score="10" Body="&lt;p&gt;A function that is convex needs to satisfy $f(\alpha x + (1-\alpha)y) \le \alpha f(x) + (1-\alpha)f(y)$ for all $\alpha\in(0,1)$ and $x,y$ in the domain of definition. You could simply try to verify this formula for a large number of pairs $x,y$ and a couple of values $\alpha$, e.g. $\alpha=\{1/4,1/2,3/4\}$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-08-19T14:44:39.993" LastActivityDate="2012-08-19T14:44:39.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3107" PostTypeId="1" CreationDate="2012-08-19T06:57:16.947" Score="8" ViewCount="2841" Body="&lt;p&gt;The minimum image convention (MIC), see for example &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.57.1696&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;noreferrer&quot;&gt;a short note of W. Smith&lt;/a&gt;, is often used in molecular dynamics or monte carlo simulations of periodic systems with an orthorhombic unit cell. For this special case, it is rather trivial to implement the MIC correctly. How does one apply the MIC to systems with a more general triclinic unit cell? The naive idea to simply generalize the algorithm for orthorhomic cells to triclinic cells, does not seem to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could find &lt;a href=&quot;http://www.tcm.phy.cam.ac.uk/~mdt26/talks/zoltan/min_image_slides.ps.gz&quot; rel=&quot;noreferrer&quot;&gt;this document&lt;/a&gt;, which provides some clues, such as the importance of the reduced basis of the lattice vectors, but it does not contain enough details to make a computer implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will implement the answers below in a test code to check which algorithms work (under given assumptions). The Python test program can be found here: &lt;a href=&quot;https://gist.github.com/3566972&quot; rel=&quot;noreferrer&quot;&gt;https://gist.github.com/3566972&lt;/a&gt;. It tests the lammps implementation and one naive attempt of mine. Both fail. If one finds a bug in this test program, please get in touch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. For those who get worried about MD or MC codes that use a naive algorithm for triclinic cells: please read &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.57.1696&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;noreferrer&quot;&gt;the note written by W. Smith&lt;/a&gt;. It explains a naive algorithm that works as long as the distance cutoff is shorter than half the shortest distance between the opposite faces of the unit cell. For a given cutoff, one may always construct a triclinic supercell for which naive algorithms will work fine.&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="156" LastEditDate="2012-09-01T08:27:37.397" LastActivityDate="2022-04-11T10:56:46.290" Title="Minimum image convention for triclinic unit cell" Tags="|monte-carlo|molecular-dynamics|molecular-mechanics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3108" PostTypeId="2" ParentId="3099" CreationDate="2012-08-19T10:06:22.240" Score="4" Body="&lt;p&gt;&lt;code&gt;X = linsolve(A,B)&lt;/code&gt; is a convenience function of MATLAB that tries to find a sensible numerical solution to the linear system $$AX = B$$ in the general case, $A\in\mathbb{C}^{m\times n}$ with $m \gtreqqless n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For (numerical) full rank &lt;code&gt;A&lt;/code&gt;, &lt;code&gt;linsolve&lt;/code&gt; operates in such a way that the result is linear in the second argument, so we can assume that the solution to the above problem is defined as $$X = A^\dagger B,$$ where $A^\dagger$ is a function of $A$ alone. The MATLAB approximation to $A^\dagger$ can be computed as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ad = linsolve(A, eye(size(A,1)));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the question at hand is to compute $\frac{\partial X}{\partial g_i}$, the the proble reduces to computing $$\frac{\partial A^\dagger}{\partial g_i}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will discuss only the case of real $A$, with $m&amp;gt;n$ and $\mathrm{rank}(A) = n$, i.e. an &lt;a href=&quot;http://en.wikipedia.org/wiki/Overdetermined_system&quot; rel=&quot;nofollow&quot;&gt;overdetermined&lt;/a&gt; full rank system. In this case MATLAB computes the least square solution, and we have $$A^\dagger = (A^TA)^{-1}A^T.$$ Now &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{multline}&#xA;\frac{\partial A^\dagger}{\partial g_i} = &#xA;\frac{\partial (A^TA)^{-1}}{\partial g_i} A^T +&#xA;(A^TA)^{-1} \frac{\partial A^T}{\partial g_i} &#xA;= \\ =&#xA;- (A^TA)^{-1} \left[  \frac{\partial A^T}{\partial g_i}A +  A^T \frac{\partial A}{\partial g_i} \right] (A^TA)^{-1} A^T&#xA;+ (A^TA)^{-1} \frac{\partial A^T}{\partial g_i}&#xA;= \\ =&#xA;-A^\dagger \frac{\partial A}{\partial g_i} A^\dagger + (A^TA)^{-1} \frac{\partial A^T}{\partial g_i} (I-AA^\dagger).&#xA;\end{multline} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analogous calculations can be performed in the remaing cases, with reference to the actual problem MATLAB is solving.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2012-08-19T10:06:22.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3109" PostTypeId="2" ParentId="1727" CreationDate="2012-08-19T22:53:44.883" Score="0" Body="&lt;p&gt;I have the impression that row-major ordering is optimal for this problem with respect to memory access times, cache lines usage and TLB misses. I guess your FORTRAN version used column-major ordering instead, which could explain why it is consistently slower than the C version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already pointed out before, you're memory bandwidth limited here. What could possibly go wrong is that the vector $b$ isn't kept in the cache. You could test whether you observe the same (effective) memory bandwidth for size_N = 15000 than for size_N = 5000. If you do, there is a pretty good chance that the code is already optimal, and that the memory bandwidth of your system is simply not that great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also test the speed if you just sum up all elements of the matrix in a single loop instead of the matrix vector multiplication. (You might want to unroll the loop by a factor 4, because non-associativity of addition could prevent the compiler from doing this optimization for you.)&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-08-19T22:53:44.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3110" PostTypeId="1" CreationDate="2012-08-19T23:21:06.587" Score="2" ViewCount="1327" Body="&lt;p&gt;I am trying to build a Runge-Kutta code to integrate the equations of motion for a simple harmonic oscillator. However, when I run the code, I only see first order improvement in the error as I decrease the step size. What could be causing this issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I start with the second order equation x''+x=0 and introduce a variable v=x'. Then we have a system of two first order equations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x'=v&#xA;v'=-x.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Written in matrix form as &#xA;    (x',v')^T=M(x,v)^T (T stands for transposition), where M=(0,1;-1,0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we compute A,B,C,D in the RK formula x(i+1)=x(i)+h/6*(A+2B+2C+2D) using (ts=timestep)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A=M(x0,v0)&#xA;B=M[(x0,v0)+ts/2*A]&#xA;C=M[(x0,v0)+ts/2*B]&#xA;D=M[(x0,v0)+ts*C]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and apply the RK formula written above. But this gives only linear improvement (ie halving the timestep increases the accuracy by a factor of 2).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program rk4&#xA;implicit none&#xA;&#xA;integer :: i,j,k&#xA;integer :: nos&#xA;&#xA;double precision, allocatable :: y(:,:)&#xA;double precision :: x0,x1,x2,x3,x4,v0,v1,v2,v3,v4&#xA;double precision :: ts&#xA;&#xA;allocate(y(2,10000))&#xA;&#xA;x0=1.D0 !initial position&#xA;v0=0.D0 !initial velocity&#xA;ts=.025D0 !timestep&#xA;nos=1000 !number of steps&#xA;&#xA;do i=1,nos&#xA;&#xA;x1=v0&#xA;v1=-x0&#xA;&#xA;x2=v0+ts/(2.D0)*v1 !a&#xA;v2=-x0-ts/(2.D0)*x1&#xA;&#xA;x3=v0+ts/(2.D0)*v2 !b&#xA;v3=-x0-ts/(2.D0)*x2&#xA;&#xA;x4=v0+ts/(2.D0)*v3 !c&#xA;v4=-x0-ts/(2.D0)*x3&#xA;&#xA;x0=x0+ts/(6.D0)*(x1+(2.D0)*x2+(2.D0)*x3+x4) !d&#xA;v0=v0+ts/(6.D0)*(v1+(2.D0)*v2+(2.D0)*v3+v4)&#xA;&#xA;y(1,i)=x0&#xA;y(2,i)=v0&#xA;&#xA;end do&#xA;&#xA;open(1,file='rk4.dat', status='unknown')&#xA;do i=1,nos&#xA;write(1,*) y(1,i), dble(i)*ts !use to plot a graph of the solution&#xA;end do&#xA;write(*,*) cos(6.3D0)-y(1,252) !error&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1926" LastEditorUserId="123" LastEditDate="2012-08-20T14:24:25.367" LastActivityDate="2012-08-20T14:24:25.367" Title="Where is the bug in my fourth order Runge-Kutta method implementation?" Tags="|fortran|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3111" PostTypeId="2" ParentId="3110" CreationDate="2012-08-19T23:49:26.040" Score="7" Body="&lt;p&gt;It appears you are only taking half a time step on the fourth stage (where you should take a full one).  Since you aren't taking the proper time step, this is turning into a complicated but relatively useless explicit scheme.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2012-08-19T23:57:09.820" LastActivityDate="2012-08-19T23:57:09.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3112" PostTypeId="1" CreationDate="2012-08-20T08:45:03.807" Score="1" ViewCount="149" Body="&lt;p&gt;I have the following Problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I know the range within the results of a division must lie.&#xA;[ quotientrange ] Additionally, the quotient should not exceed a certain number of fractions.&lt;/li&gt;&#xA;&lt;li&gt;I know, that both dividend and divisor must consist of a certain set of digits. Dividend and divisor are further restricted by a range.&#xA;Edit: Dividend and divisor are only integers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My question is: How can I determine a set of dividends and divisors, which satisfis this assumption?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Dividend should be within the rage [100.00; 999.99] and only consist of the digits 1, 2, 3, 4, 5&lt;/li&gt;&#xA;&lt;li&gt;Divisor should be within the range [10; 99] and only consist of the digits 1, 2, 5.&lt;/li&gt;&#xA;&lt;li&gt;Thus, the maximum range of the result is clearly determined by [100.00 / 99; 999.99 / 10]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Question: How can I determine a set [dividend1,dvisor1;....;dividend_n,divisor_n] which satisfies my assumptions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Brute force is an alternative, certainly, by calculating from an assumed result (the range is known), constructing a valid divisor, performing a multiplication and chcecking, wheater the result satisfies the constraint for the dividend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I wonder if there is a more intelligent approach?&lt;/p&gt;&#xA;" OwnerUserId="1856" LastEditorUserId="1856" LastEditDate="2012-08-21T19:08:57.173" LastActivityDate="2012-08-21T19:08:57.173" Title="Determine set of dividend and divisor for quotient" Tags="|algorithms|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3113" PostTypeId="1" AcceptedAnswerId="3114" CreationDate="2012-08-20T18:13:09.323" Score="5" ViewCount="228" Body="&lt;p&gt;I have an optimization problem that has a linear objective function. The constraints are of the form: $(Ax \leq b) \wedge (Cx \nless d)$. &#xA;In other words, I have: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\min &amp;amp;f^T x &#xA;\notag \\&#xA;\text{s.t.}&#xA;&amp;amp;Ax \leq b \\&#xA;&amp;amp;Cx \nless d\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to solve the problem would be to decompose the constraint $Cx \nless d$ into $m$ constraints (assuming $C\in R^{m\times n}$): &#xA;\begin{align}&#xA;     &amp;amp; c_1^T x &amp;amp;\geq &amp;amp;d_1 \\&#xA;\vee &amp;amp; c_2^T x &amp;amp;\geq &amp;amp;d_2 \\&#xA;     &amp;amp;         &amp;amp; \vdots &amp;amp; \\&#xA;\vee &amp;amp; c_m^T x &amp;amp;\geq &amp;amp;d_m \\&#xA;\end{align}&#xA;and we end up solving $m$ linear programs that are exactly the same except for one constraint that changes from one LP to another. Global optimum is simply the best among the $m$ optimums obtained. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone think of a faster way to perform this optimization? What about convex relaxation? How would I relax my problem to a single linear program? How good a convex relaxation solution would be? &lt;/p&gt;&#xA;" OwnerUserId="1932" LastEditorUserId="996" LastEditDate="2012-08-20T20:20:37.030" LastActivityDate="2012-08-20T20:20:37.030" Title="Linear Programming with constraints of the form $Cx \nless d$ where $C\in R^{m\times n}$" Tags="|optimization|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3114" PostTypeId="2" ParentId="3113" CreationDate="2012-08-20T19:56:44.267" Score="6" Body="&lt;p&gt;The constraint $Cx \nless d$ is can be expressed as a mixed-integer program under certain conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;C_i x - M_i y_i    &amp;amp;\ge&amp;amp; d_i \hspace{0.3 in} \forall i \in I \\&#xA;\sum_{i \in I} y_i &amp;amp;\le&amp;amp; m-1 \\&#xA;y_i   &amp;amp;\in&amp;amp; \{ 0, 1 \} \hspace{0.2 in} \forall i \in I &#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $I = \{1, \ldots m\}$, $M_i$ is a constant that represents an a-priori upper-bound on $c_i x$.  If A good mixed-integer solver should do better than solving the $m$ separate continuous problems.&lt;/p&gt;&#xA;" OwnerUserId="996" LastActivityDate="2012-08-20T19:56:44.267" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3115" PostTypeId="1" AcceptedAnswerId="3119" CreationDate="2012-08-20T22:36:49.500" Score="2" ViewCount="218" Body="&lt;p&gt;I have an iterative sequence for optimizing an EM (Expectation Maximization) algorithm based loss function $L(X)$  with $t$ being the iteration number as:&#xA;$X_t=ABX_{t-1}+CX_{t-1}+X_{t-1}$ where $A$ is a diagonal matrix, $B$ and $C$ are positive semi-definite matrices. Also the diagonal entries in $A$ are the inverse of the diagonal elements of $C$. i.e, $A=Diag^{-1}(C)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to compute the convergence rate (root-convergence rate) of this algorithm as $t\to \infty$. Am assuming it has got to do with taylor expansions, spectral radii and fixed point theorems. How is this approached or done for iterative schemes?&lt;/p&gt;&#xA;" OwnerUserId="1934" LastEditorUserId="1934" LastEditDate="2012-08-22T04:09:53.940" LastActivityDate="2012-08-22T12:18:05.663" Title="Root Convergence rate of Iterative Scheme " Tags="|optimization|matrix|eigensystem|statistics|convergence|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3116" PostTypeId="1" AcceptedAnswerId="3117" CreationDate="2012-08-21T08:57:09.120" Score="6" ViewCount="189" Body="&lt;p&gt;I am currently trying to code the &lt;a href=&quot;http://www.cs.berkeley.edu/~mhoemmen/pubs/thesis.pdf&quot;&gt;CA-CG method&lt;/a&gt; within the PETSc framework. A mandatory step in this process is the implementation of the &quot;matrix powers kernel&quot; algorithm for a generic sparse matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This requires the computation of the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/IS/IS.html&quot;&gt;index set&lt;/a&gt; obtained exploring the graph associated with the sparse matrix for s-steps, starting from the index set of the local rows &quot;owned&quot; by each process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a clean way to retrieve this kind of information using the PETSc API?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="305" LastActivityDate="2012-08-21T14:45:36.267" Title="Is there a way to inspect the graph of a sparse matrix with PETSc?" Tags="|matrix|petsc|sparse-matrix|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3117" PostTypeId="2" ParentId="3116" CreationDate="2012-08-21T14:45:36.267" Score="3" Body="&lt;p&gt;Use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatIncreaseOverlap.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;MatIncreaseOverlap()&lt;/code&gt;&lt;/a&gt; just like &lt;code&gt;PCASM&lt;/code&gt; (additive Schwarz preconditioner) to get the overlapping region, then &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSubMatrices.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;MatGetSubMatrices()&lt;/code&gt;&lt;/a&gt; (also like &lt;code&gt;PCASM&lt;/code&gt;) to pull out the overlapping part of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further technical discussion of implementation, we encourage you to subscribe to &lt;code&gt;petsc-dev@mcs.anl.gov&lt;/code&gt; or just email &lt;code&gt;petsc-maint@mcs.anl.gov&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-21T14:45:36.267" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3118" PostTypeId="2" ParentId="3107" CreationDate="2012-08-21T18:49:41.490" Score="5" Body="&lt;p&gt;The key is to take the differences $\Delta x$, $\Delta y$, and $\Delta z$ separately before beginning. Given the edge vectors ${\bf a}$, ${\bf b}$, and ${\bf c}$ that define a unit cell that has one corner at the origin, then your tilt factors are $b_1$, $c_1$, and $c_2$, where $n_m$ defines the $m^{th}$ component of vector ${\bf n}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for each component:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If $|\Delta x| &amp;gt; 0.5(a_{max} - a_{min})$, then subtract off $a_{max} - a_{min}$ if $\Delta x &amp;gt; 0$, and add it for values less than zero.&lt;/li&gt;&#xA;&lt;li&gt;If $|\Delta y| &amp;gt; 0.5(b_{max} - b_{min})$, then subtract off $b_{max} - b_{min}$ if $\Delta y &amp;gt; 0$, and add it for values less than zero. &lt;em&gt;Also&lt;/em&gt; add or subtract $b_1$ from $\Delta x$.&lt;/li&gt;&#xA;&lt;li&gt;If $|\Delta z| &amp;gt; 0.5(c_{max} - c_{min})$, then subtract off $c_{max} - c_{min}$ if $\Delta z &amp;gt; 0$, and add it for values less than zero. &lt;em&gt;Also&lt;/em&gt; add or subtract $c_1$ from $\Delta x$ and $c_2$ from $\Delta y$. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My source for this is taken from the code for triclinic domains found in &lt;code&gt;domain.cpp&lt;/code&gt; in the &lt;a href=&quot;http://lammps.sandia.gov&quot;&gt;LAMMPS molecular dynamics package&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2012-08-21T18:49:41.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3119" PostTypeId="2" ParentId="3115" CreationDate="2012-08-22T12:18:05.663" Score="2" Body="&lt;p&gt;This is just a linear fixed-point iteration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ X_t = M X_{t-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\DeclareMathOperator{\diag}{diag} M = AB + C + I = (\diag C)^{-1} B + C + I .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convergence rate of this iteration is linear with worst-case convergence factor equal to the spectral radius of $M$ (therefore, if $\rho(M) &amp;gt; 1$, the method does not converge). Furthermore, if $\rho(M) = 1$ and has any eigenvalues of magnitude 1 that are not exactly 1, the iteration also does not converge. Clearly, if $B$ and $C$ can vary independently, the iteration is not generally convergent. Note that if $\rho(M) &amp;lt; 1$, it always converges to 0.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-22T12:18:05.663" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3120" PostTypeId="1" CreationDate="2012-08-23T08:34:35.137" Score="12" ViewCount="1403" Body="&lt;p&gt;I have a (2-dimensional) improper integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I=\int_A \frac{W(x,y)}{F(x,y)}\,\mbox{d}x\mbox{d}y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the domain of integration $A$ is smaller than $x=[-1,1]$, $y=[-1,1]$ but further restricted by $F(x,y)&amp;gt;0$ . Since $F$ and $W$ are smooth and $W \ne 0$ at the boundaries, the later relation implies that the integrand can be singular at the boundaries. The integrand is finite though. I, so far, compute this integral with nested numerical integration. This is successfull but slow. I search a more appropriate (faster) method to address the integral, maybe a Monte-Carlo method. But I need one which does not put points on the boundary of the non-cubic domain A and takes the limit of the improper integral correctly. Can a Integral transformation help for this general expression? Note that I can solve $F(x,y)$ for $y$ as a function of $x$ and even compute $I$ for a few special weight functions $W(x,y)$.&lt;/p&gt;&#xA;" OwnerUserId="1943" LastEditorUserId="541" LastEditDate="2012-08-23T08:40:34.643" LastActivityDate="2012-08-27T07:11:38.607" Title="Numeric integration of multi-dimensional integral with known boundaries" Tags="|monte-carlo|quadrature|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3121" PostTypeId="2" ParentId="3120" CreationDate="2012-08-23T09:10:01.177" Score="8" Body="&lt;p&gt;Disclaimer: I wrote my PhD thesis on adaptive quadrature, so this answer will be severely biased towards my own work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GSL's QAGS is the old &lt;a href=&quot;http://en.wikipedia.org/wiki/QUADPACK&quot;&gt;QUADPACK&lt;/a&gt; integrator, and it is not entirely robust, especially in the presence of singularities. This usually leads to users requesting far more digits of accuracy than they actually need, thus making the integration quite expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're using GSL, you might want to try my own code, &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/CQUAD-doubly_002dadaptive-integration.html&quot;&gt;CQUAD&lt;/a&gt;, described in &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1824804&quot;&gt;this paper&lt;/a&gt;. It is designed to cope with singularities, both at the interval edges and within the domain. Note that the error estimate is quite robust, so only ask for as many digits as you actually require.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to Monte-Carlo integration, it depends on what kind of accuracy you're looking for. I'm also not quite sure of how well it will work near singularities.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-08-23T09:10:01.177" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="3122" PostTypeId="1" CreationDate="2012-08-23T15:48:23.673" Score="12" ViewCount="1091" Body="&lt;p&gt;I have the impression, from very different resources and talks with researches, that there is a growing demand for high precision computations in numerical partial differential equations. Here, high precision means more precision than just the standard 64bit double precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder about the state of the art of this topic. By way of comparision, there are communitites in numerical PDE which specifically target, e.g., multicore methods, large-scale parallelization or GPU-computing. I wonder whether a similar community is exists or is growing for high precision methods in numerical PDE, and I would be particulary interested (and this is the actual point of the question) in introductory or survey papers on high precision, which also provide an impression of the actual relevance of the topic. &lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="238" LastEditDate="2012-08-23T16:04:42.693" LastActivityDate="2021-10-21T15:51:47.583" Title="Higher precision floating-point arithmetic in numerical PDE" Tags="|reference-request|precision|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3123" PostTypeId="2" ParentId="3122" CreationDate="2012-08-23T17:04:46.137" Score="24" Body="&lt;p&gt;Discretization of the continuum PDEs usually commits much more error than the finite precision. I find that about 90% of the people requesting higher precision have just been lazy with problem formulation and are trying to solve a problem using poor scaling, bad discretizations, or bad continuum modeling. The remaining 10% may have justifiably ill-conditioned systems for which increased precision really makes sense. Even in those cases, we mostly use quad precision as a debugging tool (especially in conjunction with methods using finite differenced Frechet derivatives, and to investigate the cause of &quot;spurious&quot; numerical null spaces) or locally for a very sensitive operation rather than in the large scale for production.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GCC has provided &lt;code&gt;__float128&lt;/code&gt; since version 4.6, so it's very easy to try. (Earlier implementations were generally much more intrusive and less portable.) PETSc has supported &lt;code&gt;--with-precision=__float128&lt;/code&gt; since version 3.2 so it's just a matter of recompiling.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-23T17:04:46.137" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3124" PostTypeId="1" AcceptedAnswerId="3125" CreationDate="2012-08-23T17:56:59.053" Score="3" ViewCount="303" Body="&lt;p&gt;I'd like to place as many random points as possible in a 2D square $S=[0,1]x[0,1]$ such that the euclidean distance $d$ between any two points $d$ is greater than a given value $b$ (b is small).  I'm interested in an iterative construction algorithm that successively limits the remaining space where points can be placed.  In such a case, I'm curious how to efficiently characterize the available space and how to check the stopping criteria &quot;until no more points can be placed&quot;.  Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="20688" LastEditDate="2019-06-22T22:42:02.870" LastActivityDate="2019-06-22T22:42:02.870" Title="Random placement of euclidean points with constrained inter-point distances in a fixed area" Tags="|computational-geometry|random-sampling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3125" PostTypeId="2" ParentId="3124" CreationDate="2012-08-23T18:29:06.477" Score="4" Body="&lt;p&gt;This is called Poisson disk sampling, and there are a lot of papers on the subject.  Here are a few:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cs.virginia.edu/~gfx/pubs/antimony&quot; rel=&quot;nofollow&quot;&gt;D Dunbar, G Humphreys (2006). A spatial data structure for fast Poisson-disk sample generation.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://jesper.kalliope.org/blog/library/JonesPoissonPreprint.pdf&quot; rel=&quot;nofollow&quot;&gt;L. Wei (2008). Parallel Poisson disk sampling.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.ubc.ca/~rbridson/docs/bridson-siggraph07-poissondisk.pdf&quot; rel=&quot;nofollow&quot;&gt;R. Bridson (2007). Fast Poisson disk sampling in arbitrary dimensions.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://modsim.gist.ac.kr/thesis/SIGGRAPH2011/disk2/content/papers/049-ebeida.pdf&quot; rel=&quot;nofollow&quot;&gt;M. Ebeida et al. (2011). Efficient maximal Poisson-disk sampling&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The last one appears to be what you what.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-08-23T18:29:06.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3126" PostTypeId="2" ParentId="3122" CreationDate="2012-08-23T21:18:05.883" Score="17" Body="&lt;p&gt;In the 15 years that we have provided FEM software in the form of the deal.II project (&lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;noreferrer&quot;&gt;http://www.dealii.org/&lt;/a&gt;), I don't think that we've ever had a genuine request to solve PDEs to higher accuracy than double precision. The reason is as Jed suggests in the other answer: The error one makes discretizing the PDE is much larger than the 16 digits of accuracy one gets from double precision floating point arithmetic. Thus, you'd have to have an incredibly fine mesh to get to the point where you need more accuracy in the arithmetic to affect the overall error.&lt;/p&gt;&#xA;&lt;p&gt;I think that actually the opposite is true: People are thinking about (and working on) what happens when you, for example, use single precision to store the elements of the matrix or of preconditioners. In general, this does not significantly reduce their accuracy, but it increases performance by roughly a factor of two because you need to move only half as much data from memory into the processor.&lt;/p&gt;&#xA;&lt;p&gt;So, my sense is that quad precision (or even higher) is something that may be relevant to the ODE solver community but not to the PDE community.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2021-10-21T15:51:47.583" LastActivityDate="2021-10-21T15:51:47.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3127" PostTypeId="2" ParentId="3120" CreationDate="2012-08-23T21:42:54.713" Score="5" Body="&lt;p&gt;Monte Carlo methods can in general not compete with adaptive quadrature unless you have a high dimensional integral where you cannot afford the combinatorial explosion of quadrature points with the dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is relatively easy to understand. Take, for example, just $\int_{[0,1]^n} f(x)\; d^nx$ where $n$ is the dimension of the problem. Let's say, for simplicity, that you subdivide every dimension into $M$ sub-intervals, i.e., you get $M^n$ hypercube cells in total. Let's assume further that you use a Gauss formula with $k$ Gauss points, just as an example. Then you have $N=(kM)^n$ quadrature points in total, and because the $k$ Gauss points provide you with order $(2k-1)$ accuracy, $e = {\cal O}(h^5)={\cal O}(M^{-(2k-1)})$, your overall accuracy as a function of evaluation points will be&#xA;$$&#xA;  e = {\cal O}(N^{-(2k-1)/n}).&#xA;$$&#xA;On the other hand, Monte Carlo methods generally only provide you with error convergence as&#xA;$$&#xA;  e = {\cal O}(N^{-1/2})&#xA;$$&#xA;which is worse than for any Gauss formula with at least $k&amp;gt;n/4+1/2$ points per interval. The reason is relatively simple to understand: Gauss quadrature chooses the interpolation points in some sort of smart way, Monte Carlo isn't. You can't expect anything useful to come of the latter. (Of course there are situations where Gaussian quadrature is difficult: for example in your case where the integration domain is irregularly shaped; but in that case you're likely still better off doing adaptive integration or similar.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there are practical (stability) problems with integration with more than, say, 8 or 10 points per interval. So if you want $k\le 8$, then you can't go beyond $n=30$. On the other hand, in that case, even choosing a single interval per direction ($M=1$) yields $N=8^{30}$ integration points, far more than you could ever in a lifetime evaluate. In other words, as long as you can evaluate enough integration points, quadrature on subdivisions of your integration domain is always the more efficient approach. It's cases where you have have a high dimensional integral for which you can't evaluate the integration points on even a single subdivision any more that people use Monte Carlo methods despite their worse convergence order.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="276" LastEditDate="2012-08-24T06:15:16.187" LastActivityDate="2012-08-24T06:15:16.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3128" PostTypeId="1" AcceptedAnswerId="3129" CreationDate="2012-08-24T05:26:08.620" Score="11" ViewCount="453" Body="&lt;p&gt;A simple enough question: to do a multidimensional integral, given that one has decided that some sort of Monte Carlo method is appropriate, is there any advantage that a regular MC integration using pseudorandom numbers has over a quasi-Monte Carlo integration using a quasirandom sequence? If so, how would I recognize situations where this advantage would come into play? (And if not, why does anyone ever use plain old Monte Carlo integration?)&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2018-01-03T20:36:51.373" Title="Under what circumstances is Monte Carlo integration better than quasi-Monte Carlo?" Tags="|monte-carlo|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3129" PostTypeId="2" ParentId="3128" CreationDate="2012-08-24T06:55:36.027" Score="4" Body="&lt;p&gt;Monte Carlo simulations are the method of choice for computation of electron scattering. Tricks like importance sampling are used sometimes, so you might say it's not plain old Monte Carlo. But the main point is probably that an inherently stochastic process is simulated here, while you are asking only about using Monte Carlo for integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because nobody else tried to offer an answer, let me try to expand my answer a bit. Assume we have an electron scattering simulation, where only a single number, like a backscattering coefficient, is computed. If we would reformulate this as a multidimensional integral, it would probably be an infinite dimensional integral. On the other hand, during the simulation of a single trajectory, only a finite number of random numbers is required (this number can become quite large, if secondary electron generation is taken into account). If we would use a quasirandom sequence like latin hypercube sampling, we would have to use an approximation with a fixed number of dimensions, and generate a random number for every dimension for each sample point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think the difference is whether some kind of high dimensional unit-hypercube is sampled, versus an infinite dimensional probability cloud around the origin.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2012-08-28T00:06:08.890" LastActivityDate="2012-08-28T00:06:08.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3130" PostTypeId="1" AcceptedAnswerId="3133" CreationDate="2012-08-25T13:55:45.357" Score="5" ViewCount="283" Body="&lt;p&gt;For a discontinuous Galerkin-based simulation I need to store cell-based simulation data in memory. Since the order of the polynomial approximation $N_p$ may vary between cells, I wonder what the most efficient data structure would be, if the typical access pattern is a loop over all cells? These are the ideas I had so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) Use an array of cell objects, each object with a pointer to its (individually allocated) simulation data array&lt;/strong&gt;&lt;br&gt;&#xA;&lt;em&gt;Pro:&lt;/em&gt; Very easy to handle algorithmically, no headaches when pre-allocating memory for data storage, very easy to store topological relationships between cells (i.e. neighbor information), straightforward direct access of specific cells.&lt;br&gt;&#xA;&lt;em&gt;Con:&lt;/em&gt; Very (extremely?) slow when looping over all cells as there will probably cache misses every single time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) Use an array of cell objects, each object with a pointer to its simulation data array (memory pre-allocated and cell data stored contiguously)&lt;/strong&gt;&lt;br&gt;&#xA;&lt;em&gt;Pro:&lt;/em&gt; Same as above. Also probably faster since data is now contiguous in memory.&lt;br&gt;&#xA;&lt;em&gt;Con:&lt;/em&gt; Still slower than just using arrays, since each cell access still requires the knowledge of $N_p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3) Use an  array (or an STL container) for each set of cells with common $N_p$&lt;/strong&gt;&lt;br&gt;&#xA;&lt;em&gt;Pro:&lt;/em&gt; Each container can contain the data contiguously, thus there will be many cache hits if iterating over $N_p$ first, cell second.&lt;br&gt;&#xA;&lt;em&gt;Con:&lt;/em&gt; Difficult if memory pre-allocation is desired. Random cell access using only the cell id becomes very expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have experience with one of the aforementioned methods, or has an even better alternative in mind? If you need more information on e.g. the data access patterns, please let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: This question is similar to &lt;a href=&quot;https://stackoverflow.com/questions/11413031/what-is-the-most-efficient-yet-sufficiently-flexible-way-to-store-multi-dimens&quot;&gt;this one&lt;/a&gt; on Stackoverflow, but rephrased and simplified.&lt;/p&gt;&#xA;" OwnerUserId="1724" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-08-31T14:33:19.940" Title="How to efficiently structure simulation data in memory for cells with varying degrees of freedom?" Tags="|hpc|efficiency|discontinuous-galerkin|data-structures|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3131" PostTypeId="1" AcceptedAnswerId="3132" CreationDate="2012-08-25T18:41:42.650" Score="8" ViewCount="191" Body="&lt;p&gt;I've been reading about implementing dense matrix multiplication when the matrix doesn't fit in cache. One of the graphs I've seen (slide 9 from &lt;a href=&quot;http://vuduc.org/teaching/standalone-talks/mem-hier-tutorial-1.pdf&quot;&gt;these slides&lt;/a&gt;) shows sudden drops in performance using the naive algorithm. This drops are to around 50% of the speed, occur while the matrix still fits in cache, and only occur for one or two sizes. I'm not planning on using the naive algorithm, but I would like to know where the sudden drops in performance come from. (The drops also occur with blocked algorithms, but are much smaller.) &lt;/p&gt;&#xA;" OwnerUserId="1420" LastActivityDate="2012-08-25T19:28:45.880" Title="Sudden drops in matrix multiplication performance" Tags="|linear-algebra|matrix|blas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3132" PostTypeId="2" ParentId="3131" CreationDate="2012-08-25T19:28:45.880" Score="6" Body="&lt;p&gt;That is a classic example of cache associativity. The stride associated with that problem size is filling up certain sets causing cache eviction despite there being lots of space in other sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DV1Q4.png&quot; alt=&quot;Cache associativity&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Figure from &lt;a href=&quot;http://duartes.org/gustavo/blog/post/intel-cpu-caches&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gustavo Duarte's excellent blog post&lt;/a&gt; on the topic&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;http://www.akkadia.org/drepper/cpumemory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drepper's &lt;em&gt;What every programmer should know about memory&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-25T19:28:45.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3133" PostTypeId="2" ParentId="3130" CreationDate="2012-08-26T01:30:12.657" Score="4" Body="&lt;p&gt;deal.II uses variants of your option (2). This is of course also essentially the way you store sparse matrices and is very efficient since you only have two arrays (the array that for each cell stores the beginning index of the data that corresponds to this cell, and the array that contiguously stores all the data of all cells). You traverse both arrays sequentially, producing few cache misses. Both of your options are either terribly inefficient (option 1) or rather wasteful with non-consecutive access (option 3).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-26T01:30:12.657" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="3135" PostTypeId="2" ParentId="3099" CreationDate="2012-08-26T10:45:36.730" Score="1" Body="&lt;p&gt;The question asks for the derivative of the solution of $X$ of $A^*(AX-S)=0$ with respect to a parameter $t=g_i$ on which $A$ (but not $S$) depends. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Differentiation of the defining equation with respect to $t$ (denoted by a dot) gives $\dot A^*(AX-S) +A^*(\dot A X+A\dot X)=0$, hence $\dot X$ is the solution of the system of equations $A^*A\dot X=B:=\dot A^*(S-AX)-A^*\dot A X$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The right-hand side $B$ can be computed after $X$ has been found, typically from&#xA;an orthogonal factorization $A=QR$ (but a Cholesky factor $R$ of the normal equations matrix $A^*R$ would also do). Then $\dot X$ can be found without refactorization by solving $R^*R\dot X=B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case a $QR$ factorization is used, a slightly more numerically stable version first solves $R^*Y=\dot A^*(S-AX)$ and then $R\dot X=Y-Q^*\dot A X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Derivatives with respect to all $g_i$ can be obtained in both cases more efficiently by stacking the right hand sides, using BLAS 3 routines.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-08-26T10:45:36.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3136" PostTypeId="2" ParentId="3101" CreationDate="2012-08-26T10:51:24.483" Score="6" Body="&lt;p&gt;For a number of practically useful convexity/nonconvexity verification tests, see (self-disclaimer, I am the third author on this paper):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/ms/convexity.pdf&quot; rel=&quot;nofollow&quot;&gt;R. Fourer, C. Maheshwari, A. Neumaier, D. Orban, and H. Schichl, Convexity and Concavity Detection in Computational Graphs. Tree Walks for Convexity Assessment, INFORMS J. Computing 22 (2010), 26-43.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that there are many functions that are convex in the domain of interest but acannot easily be ''disciplined'', i.e., written in one of the forms required by structured convex solvers such as &lt;a href=&quot;http://cvxr.com/cvx&quot; rel=&quot;nofollow&quot;&gt;CVX&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="9" LastEditDate="2012-08-27T11:44:32.430" LastActivityDate="2012-08-27T11:44:32.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3137" PostTypeId="1" CreationDate="2012-08-26T18:51:47.920" Score="6" ViewCount="1594" Body="&lt;p&gt;Is it possible to simulate transitional flow in OpenFOAM? Or, alternatively, does it make sense to use k-epsilon or k-omega model with low Reynolds number? If not, why? What kind of errors would appear?&lt;/p&gt;&#xA;" OwnerUserId="1616" LastActivityDate="2012-08-31T18:58:27.290" Title="Transitional flow in OpenFOAM" Tags="|fluid-dynamics|openfoam|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3138" PostTypeId="2" ParentId="3137" CreationDate="2012-08-26T21:01:29.373" Score="6" Body="&lt;p&gt;There seems to be a misunderstanding of the purpose of a turbulence model. Turbulence models attempt to solve a homogenized problem involving time- and space-averaged quantities. When simulating transitional flows, however, you are typically interested in exactly those finest scales that Reynolds Averaged Navier-Stokes (RANS) would homogenize. Also note that RANS is a steady-state model averaging over all scales of turbulence (because turbulence is inherently transient). &quot;Unsteady&quot; RANS (URANS) allows transient features, but still does not attempt to resolve turbulent structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenFOAM can do LES and (to some extent) DNS, which is more appropriate for transitional flow.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-26T21:01:29.373" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3139" PostTypeId="2" ParentId="3120" CreationDate="2012-08-27T07:11:38.607" Score="1" Body="&lt;p&gt;Try a nested Double-Exponential Quadrature (see the implementations of &lt;a href=&quot;http://www.kurims.kyoto-u.ac.jp/~ooura/intde.html&quot; rel=&quot;nofollow&quot;&gt;Ooura&lt;/a&gt;). This technique use a variable transformation that makes the transformed integrand to behave very smoothly at the boundaries and is very efficient for handling singularities at the boundary. There is also a very good list of references on the DE quadrature on his website. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-08-27T07:11:38.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3141" PostTypeId="1" AcceptedAnswerId="3144" CreationDate="2012-08-27T12:31:11.467" Score="3" ViewCount="365" Body="&lt;p&gt;I am studying a research paper on iterative methods to compute generalized inverses of an arbitrary matrix $A$. I am studying the following iterative method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Y_{k+1} = Y_{k} + Y_{k}(I - AY_{k}),$$  given an initial approximation $Y_{0}$. I am having difficulties understanding how to measure the computational time taken by this method over 15 iterations. I made MATLAB code and used &lt;code&gt;tic&lt;/code&gt; and &lt;code&gt;toc&lt;/code&gt; to compute this. But every time I am clicking on run program it is showing me different computational time values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is the computational time different on different runs?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = [1 4 0 &#xA;     2 3 0 &#xA;     2 0 1&#xA;     0 0 0]; % given matrix&#xA;&#xA;Y0 = [0.0101 0.0202 0.0202 0&#xA;      0.0404 0.0303 0      0&#xA;      0      0      0.0101 0]; % initial approximation&#xA;&#xA;I = eye(4);     &#xA;&#xA;tic&#xA;&#xA;for n=1:15    &#xA;    Y1 = zeros(4,3);&#xA;    % compute sequence of approximations&#xA;    Y1 = Y0+Y0*(I-A*Y0) &#xA;    Y0 = Y1;&#xA;end&#xA;&#xA;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1580" LastEditorUserId="9" LastEditDate="2012-08-27T15:23:49.813" LastActivityDate="2012-08-28T07:30:13.543" Title="Understanding wall time jitter in MATLAB computations" Tags="|matlab|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3142" PostTypeId="1" AcceptedAnswerId="3143" CreationDate="2012-08-27T14:22:22.000" Score="3" ViewCount="698" Body="&lt;p&gt;Assume we want to solve the Poisson equation &#xA;$$&#xA;\Delta u = f&#xA;$$&#xA;with free (Neumann) boundary conditions. So, the right hand side function $f$ must fulfill the compatibility condition to integrate to zero. When using linear finite elements and a uniform refined mesh, the rhs vector has also nodal average of zero. So an iterative solver will find the solution. But this does not hold anymore for adaptively refined mesh. Thus, on the level of linear algebra the rhs vector is not orthogonal to the null space of the operator and we cannot assume the solver to find a solution in this case. Is it still valid to project out the null space from the rhs vector? For the constant null space this is the same as the substruction of the rhs nodal average from each vector entry. In this case, is there any interpretation on the finite element space? For me, this does not seem to be correct. &lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2012-08-28T01:47:21.773" Title="Solving Poisson equation with free boundaries and adaptively refined mesh" Tags="|finite-element|linear-solver|poisson|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3143" PostTypeId="2" ParentId="3142" CreationDate="2012-08-27T14:42:56.743" Score="3" Body="&lt;p&gt;Remove the discrete null space, solve the discrete problem, then postprocess (shift by a constant) to correct for the difference between the continuum null space and discrete null space. You can compute the difference by integrating the discrete null space using quadrature.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-27T14:42:56.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3144" PostTypeId="2" ParentId="3141" CreationDate="2012-08-27T17:50:05.170" Score="10" Body="&lt;p&gt;There are several issues in your code. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There is no semi-colon after the &lt;code&gt;Y1 = Y0+Y0*(I-A*Y0)&lt;/code&gt; statement. In fact you are timing the screen output of &lt;code&gt;Y1&lt;/code&gt;, &lt;em&gt;not&lt;/em&gt; the computational time&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The iteration should be written as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for n=1:15    &#xA;    Y0 = Y0+Y0*(I-A*Y0);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;allocation of  &lt;code&gt;Y1&lt;/code&gt;, assignment to &lt;code&gt;Y1&lt;/code&gt;, and then to &lt;code&gt;Y0&lt;/code&gt; are useless.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally you could even simplify further &lt;code&gt;Y0 = 2*Y0 - Y0*(A*Y0) ;&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After applying these modification you will see that the measured time becomes very small ($\approx 100 \, \mathrm{\mu s}$ on my machine). Usually for such small elapsed times the &lt;code&gt;tic, toc&lt;/code&gt; approach is not considered accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Timing small snippets of code is rather difficult, so please consider my answer here only a simple hint, not an accurate and in depth answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially you have to measure a bigger interval (about one second):&#xA;if you are interested in estimating the time of a single iteration, you can simply increase the number of iterations in your loop and divide the total time by the iteration number. (To be more precise you should also time an empty loop, to estimate the overhead associated to the loop itself.) If you are interested in estimating the total time (init &lt;code&gt;Y0&lt;/code&gt;, loop overhead, and 15 iterations) you should repeat your &lt;code&gt;for, end&lt;/code&gt; a sizable number of times, and measure the total time. Here is an example adapted from the MATLAB &lt;code&gt;help toc&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;REPS = 1000; minTime = Inf; nsum = 10;&#xA;tic;&#xA;for i=1:REPS&#xA;  tstart = tic;&#xA;  % init Y0&#xA;  Y0 = ...&#xA;  for n=1:15    &#xA;      Y0 = Y0+Y0*(I-A*Y0);&#xA;  end&#xA;  telapsed = toc(tstart);&#xA;  minTime = min(telapsed,minTime);&#xA;end&#xA;averageTime = toc/REPS;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You have to experiment with both the number of iterations (first approach) and &lt;code&gt;REPS&lt;/code&gt; (second approach) to find a sensible number, in which the &lt;code&gt;tic, toc&lt;/code&gt; errors are small enough.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-08-28T07:30:13.543" LastActivityDate="2012-08-28T07:30:13.543" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3145" PostTypeId="2" ParentId="2121" CreationDate="2012-08-27T19:19:44.937" Score="7" Body="&lt;p&gt;It seems that the library was just &lt;a href=&quot;http://www.netlib.org/na-digest-html/12/v12n35.html#1&quot;&gt;officially released on NA Digest&lt;/a&gt;. The &lt;a href=&quot;http://blaze-lib.googlecode.com/files/blaze-1.0.tar.gz&quot;&gt;source code&lt;/a&gt; and &lt;a href=&quot;http://blaze-lib.googlecode.com/files/blaze-docu-1.0.tar.gz&quot;&gt;documentation&lt;/a&gt; are available on &lt;a href=&quot;http://code.google.com/p/blaze-lib/&quot;&gt;Google Code&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-08-27T19:19:44.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3146" PostTypeId="1" AcceptedAnswerId="3149" CreationDate="2012-08-27T19:27:57.347" Score="7" ViewCount="1710" Body="&lt;p&gt;Often in computational science, we talk about the scaling or order of a particular method ($\mathcal{O}(N)$, $\mathcal{O}(N^2)$, $\mathcal{O}(N \log N)$, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having a really difficult time finding a resource (book, journal article, or website) that gives an order $\mathcal{O}$ (with respect to $N$, the number of particles) for molecular dynamics (MD) simulations.  Can you please help me?  Thank you!&lt;/p&gt;&#xA;" OwnerUserId="1454" LastEditorUserId="1454" LastEditDate="2012-08-27T19:51:42.030" LastActivityDate="2012-08-27T20:24:26.323" Title="What is the scaling or order of molecular dynamics (MD) simulations?" Tags="|reference-request|simulation|molecular-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3149" PostTypeId="2" ParentId="3146" CreationDate="2012-08-27T20:24:26.323" Score="10" Body="&lt;p&gt;Since the size of each type of atom is fixed, for a given level of accuracy the asymptotic cost is dominated by far field electrostatic interactions.  These are $O(n)$ using multigrid and $O(n \log n)$ using FFTs.  Thus the optimal complexity is $O(n)$ per time step as a function of the number of atoms simulated.  The time step is also asymptotically constant, so the total complexity of simulating $n$ particles for time $t$ is $O(nt)$.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-08-27T20:24:26.323" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3150" PostTypeId="1" AcceptedAnswerId="3152" CreationDate="2012-08-27T22:18:11.437" Score="3" ViewCount="216" Body="&lt;p&gt;I have to perform a [complex] basis transformation on a large number of [real] diagonal matrices:&#xA;$$ \langle b_i | A | b_j \rangle = \sum_k \langle b_i | \bar{b}_k\rangle \langle\bar{b}_k | A | \bar{b}_k \rangle \langle\bar{b}_k | b_j \rangle $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most efficient way to perform this operation, ideally relying on BLAS/LAPACK or similar libraries?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be important that the $\bar{B}$ basis is much larger than the $B$ basis (that's why we're doing it, after all).  Also, only the upper or lower triangular of the matrix  $\langle b_i | A | b_j \rangle$  is actually needed, as its will end up in an eigenvalue solver.  $\langle\bar{b}_k | b_j \rangle$ is stored with $ k $ as the fast index and $ j $ as the slow one.  On distributed systems, the $k$ index is distributed very close to evenly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll put my current implementation as an answer below.  I don't think it is bad, but this accounts for a large enough fraction of run-time that any improvement would be awesome.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="103" LastEditDate="2012-08-28T20:38:33.083" LastActivityDate="2012-08-28T20:38:33.083" Title="Efficiently changing basis on many diagonal matrices " Tags="|linear-algebra|performance|blas|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3151" PostTypeId="2" ParentId="3150" CreationDate="2012-08-27T22:18:11.437" Score="0" Body="&lt;p&gt;I do the right-side of the transformation $b_j$ by $b_j$ and then the left-side through a matrix-vector call.  After accumulating in a condensed vector, and summing on distributed systems, I re-expand into the upper triangular.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  ! trans(k,j) is $&amp;lt;\bar{b}_k | b_j&amp;gt;$&#xA;  ij = 1&#xA;  do j = 1, N1&#xA;    ! right-side of transformation&#xA;    jtmp(1:N2) = A(1:N2) * conjg(trans(1:N2, j))&#xA;    ! left-size of transformation&#xA;    call ZGEMV('T', N2, j, one, &amp;amp;&#xA;                trans(:,:), N2, &amp;amp;&#xA;                jtmp, 1, zero, &amp;amp;&#xA;                res(ij), 1)&#xA;    ij = ij + j&#xA;  enddo&#xA;&#xA;  ! global sum if needed&#xA;  [global sum(res)]&#xA;&#xA;  ! expand to upper triangular&#xA;  ij=0&#xA;  do j = 1, N2; do i = 1,j&#xA;    ij = ij + 1&#xA;    A_new(i,j) = conjg(res(ij))&#xA;  enddo; enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am unhappy with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;BLAS not knowing that the right and left side of the transformations are the same (though the cache might, on machines with cache).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The element-wise multiply not being a library call.  I've noticed that BLAS's ZDOTC noticeably outperforms Fortran's intrinsic dot_product (mkl, Intel compilers), so I don't have a ton of confidence in that line being optimally vectorized.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="521" LastActivityDate="2012-08-27T22:18:11.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3152" PostTypeId="2" ParentId="3150" CreationDate="2012-08-28T00:12:12.983" Score="5" Body="&lt;p&gt;Unfortunately the desired routine is not part of BLAS but is very similar to zherk, which performs a Hermitian rank-k update. In particular, zherk supports operations which look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A := \alpha B B^H + \beta A,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where only one triangle of $A$ is updated (the traditional variable names are different in the zherk prototype, where usually $C$ is the matrix to be updated). What you want is the generalization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A := \alpha B D B^H + \beta A,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ is a real diagonal matrix. I needed something similar for computing Hermitian matrix functions and wrote my own version which computes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A := \alpha B (D B)^H + \beta A,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the user can specify that, even though the left and right matrices for the outer product are different, the result will still be Hermitian. You can see &lt;a href=&quot;https://github.com/poulson/Elemental/blob/master/include/elemental/lapack-like/HermitianFunction.hpp#L38&quot;&gt;my distributed-memory implementation here&lt;/a&gt;. I call the generalization a Triangular Rank K update (Trrk). It is also useful for $LDL$ factorizations.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2012-08-28T00:12:12.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3153" PostTypeId="2" ParentId="3142" CreationDate="2012-08-28T01:47:21.773" Score="1" Body="&lt;p&gt;The correct compatibility condition is that $\int_\Omega f(x) \; dx = 0$. If you are on a uniform mesh, this means that the average of the right hand side vector entries must be zero, but on a non-uniform mesh (such as an adaptively refined one) that is no longer true. You need to ensure the analogue to this for your right hand side vector which means that you'll need to weigh the entries of your right hand side vector appropriately when taking their average.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-28T01:47:21.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3154" PostTypeId="2" ParentId="3130" CreationDate="2012-08-28T07:40:12.123" Score="1" Body="&lt;p&gt;I use a variant of option 3, that is: N independent containers, one for each cell/element type/order such that all cells of the same type are aligned in memory. When I want to perform operation x on all cells (integrate, quadrature, evolve, numerical flux...) I do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each container&amp;lt;cell_type&amp;gt; in cell_containers&#xA;  for each cell in container&amp;lt;cell_type&amp;gt;&#xA;     do x&amp;lt;cell_type&amp;gt;(cell)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where x is a different, specialized and optimized function for each cell type. To allow random access a small vector of pointers to all cell containers is stored, inside each container random access iterators are available (which should be at least as fast as the second approach but here your jumps in memory are constant inside each container because the elements of the same type are aligned sequentially). If you need global ordering for say a space-filling curve I can't see how implementing it to iterate over elements[order][id] should be more difficult than elements[id] but I haven't tried it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;main advantage&lt;/strong&gt; is that the compiler knows a lot about what is going on and can safely optimize (no virtual dispatch, no aliasing, no breaks out of the loop, the types and element order inside your numerical kernels are all compile-time constants...). As a nice side effect, hybridizing an MPI code using this approach is just as easy as loading the std parallel library(*) and compiling with OpenMP, no extra code required. For boundary elements one has usually to perform other operations and store different data. I store them separately using this same approach and define specialized kernels for them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;main drawback&lt;/strong&gt; is that you need to perform memory allocation very carefully. The problem is, before hand the only thing you know is how much memory is available, but you don't know how many elements of which order you are going to need. In particular, imagine that you have allocated a certain amount of memory for a certain type of cell, and you want to exceed it. This will require that you have &lt;em&gt;at least&lt;/em&gt; enough free memory to copy the whole vector of cells plus one extra cell. It also involves copying all the cells from that vector to the new one but that is fast. This is not smart, you probably want to allocate memory for more than one extra cell, but for how many? You probably dont want to allocate memory for twice as many cells. If you want to do this in a smart perfomant way you really need your own allocator that knows about your data structures, your system, and the on going simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Operations that need to be performed on all cells (numerical kernels) are very fast. &lt;/li&gt;&#xA;&lt;li&gt;Random access is available and is as fast as with the &quot;one vector of cell data/one vector of cell ptrs&quot; approach.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cons&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Memory allocation has to be performed &lt;em&gt;very&lt;/em&gt; carefully (but isn't this always the case?).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;* using TBB requires replacing your for_each loops with parallel_for_each or you can just implement your own for_each that takes a parallelization policy (or macro it: please dont do this).&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-08-29T14:32:27.060" LastActivityDate="2012-08-29T14:32:27.060" CommentCount="6" CommunityOwnedDate="2012-08-28T14:24:56.643" ContentLicense="CC BY-SA 3.0" />
  <row Id="3155" PostTypeId="2" ParentId="3128" CreationDate="2012-08-28T17:35:54.973" Score="5" Body="&lt;p&gt;Some of my research involves solving large scale stochastic partial differential equations.  In which case, traditional monte carlo approximation of the integral of interest converges too slowly for it to be worthwhile in a practical sense... i.e. I don't want to have to run 100 times more simulations just to get a decimal point more accuracy to the integral.  Instead, I tend to use other methods like sparse smolyak grids because they offer better accuracy in fewer function evaluations.  This is only possible because I can assume a certain degree of smoothness in the function.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is reasonable to conjecture that if you expect that the function you are integrating to have certain structure (like smoothness), it would be best to use the quasi-monte carlo scheme that exploits it.  If you really can't make very many assumptions about the function, then monte carlo is the only tool I can think of to handle it.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2018-01-03T20:05:49.050" LastActivityDate="2018-01-03T20:05:49.050" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3156" PostTypeId="1" AcceptedAnswerId="3158" CreationDate="2012-08-28T20:54:29.207" Score="6" ViewCount="967" Body="&lt;p&gt;I want this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mathworld.wolfram.com/Erfi.html&quot;&gt;http://mathworld.wolfram.com/Erfi.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But apparently scipy does not have this in its extensive special functions library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/special.html&quot;&gt;http://docs.scipy.org/doc/scipy/reference/special.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not the same as erfinv.  Right now I am using an obscure identity involving the hyp1f1 hypergeometric function but I would rather use a nicer function if possible.  Also I'll pre-emptively say that I would rather not deal with complex numbers, because the inputs and outputs are all real.&lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2012-12-21T18:52:08.240" Title="What is the best way to get erfi with scipy?" Tags="|python|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3157" PostTypeId="1" CreationDate="2012-08-29T00:23:00.200" Score="3" ViewCount="399" Body="&lt;p&gt;For an assignment i need to program an application to schedule conversations. Something similar to speeddating or Pta meeting.&#xA;The problem is that i know that this is hard to solve, but i dont know if its np-hard.&#xA;What can I do to proof that this problem is np-hard? I read that i need to reduce a known problem to my problem. But how do I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an event where Respresentatives of the elementary schools talks with the representatives of a highschool. They will talk about the students that will be transferred to the highschool.&#xA;There are approximately 200 elementary schools and 40 highschools that will be participating in this event. And this event last 2 days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rules are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The duration of each conversation is based on the ammount of students per representatives. Each conversation last 5 minutes per student. If a group consist of 1 student, this conversation last 10 minutes.&lt;/li&gt;&#xA;&lt;li&gt;No timeclashes&lt;/li&gt;&#xA;&lt;li&gt;All the students of the same group will be scheduled together, so, a representatives will only face the same representative once.&lt;/li&gt;&#xA;&lt;li&gt;Timespan is 13.00-19.00&lt;/li&gt;&#xA;&lt;li&gt;The waiting time of a representative is at most 20% of his time. A waiting time is an empty timeslot between the 1st and last conversation. &lt;/li&gt;&#xA;&lt;li&gt;Schedules for 2 days&lt;/li&gt;&#xA;&lt;li&gt;Each representatives participate for 1 day.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Based on this i will decide if i will use a heuristic algorithm to solve this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for my english&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="1901" LastEditorUserId="123" LastEditDate="2012-08-29T08:21:19.360" LastActivityDate="2012-08-29T12:56:51.053" Title="How to prove that my problem is np-hard" Tags="|complexity|approximation|heuristics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3158" PostTypeId="2" ParentId="3156" CreationDate="2012-08-29T01:05:48.473" Score="5" Body="&lt;p&gt;Erfi is available from the mpmath library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See documentation here:&#xA;&lt;a href=&quot;http://mpmath.googlecode.com/svn/trunk/doc/build/functions/expintegrals.html#mpmath.erfi&quot;&gt;http://mpmath.googlecode.com/svn/trunk/doc/build/functions/expintegrals.html#mpmath.erfi&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a link to the library:&#xA;&lt;a href=&quot;https://code.google.com/p/mpmath/&quot;&gt;https://code.google.com/p/mpmath/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1070" LastActivityDate="2012-08-29T01:05:48.473" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3159" PostTypeId="1" AcceptedAnswerId="3162" CreationDate="2012-08-29T05:11:18.273" Score="58" ViewCount="22143" Body="&lt;p&gt;Is it a good idea to use &lt;code&gt;vector&amp;lt;vector&amp;lt;double&amp;gt;&amp;gt;&lt;/code&gt; (using std) to form a matrix class for high performance scientific computing code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the answer is no. Why? Thanks&lt;/p&gt;&#xA;" OwnerUserId="1965" LastEditorUserId="11951" LastEditDate="2016-05-19T15:59:18.967" LastActivityDate="2019-05-29T20:20:28.220" Title="Is it a good idea to use vector&lt;vector&lt;double&gt;&gt; to form a matrix class for high performance scientific computing code?" Tags="|hpc|c++|" AnswerCount="7" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3160" PostTypeId="2" ParentId="3159" CreationDate="2012-08-29T07:02:43.880" Score="11" Body="&lt;p&gt;No, use one of the free available linear algebra libraries. A discussion about different libraries can be found here: &lt;a href=&quot;https://scicomp.stackexchange.com/q/351/1090&quot;&gt;Recommendations for a usable, fast C++ matrix library?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1090" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-08-29T07:02:43.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3161" PostTypeId="2" ParentId="3156" CreationDate="2012-08-29T07:49:29.850" Score="5" Body="&lt;p&gt;The correct answer in SciPy is given by datageist in his comment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(-1.0j)*scipy.special.erf((-1.0j*x))&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-08-29T07:49:29.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3162" PostTypeId="2" ParentId="3159" CreationDate="2012-08-29T11:06:17.033" Score="60" Body="&lt;p&gt;It's a bad idea because vector needs to allocate as many objects in space as there are rows in your matrix. Allocation is expensive, but primarily it is a bad idea because the data of your matrix now exists in a number of arrays scattered around memory, rather than all in one place where the processor cache can easily access it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also a wasteful storage format: std::vector stores two pointers, one to the beginning of the array and one to the end because the length of the array is flexible. On the other hand, for this to be a proper matrix, the lengths of all rows must be the same and so it would be sufficient to store the number of columns only once, rather than letting each row store its length independently.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-29T11:06:17.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3163" PostTypeId="2" ParentId="3157" CreationDate="2012-08-29T11:13:43.887" Score="3" Body="&lt;p&gt;Showing that one problem can be reduced to another to demonstrate that it is NP-hard is in general not a trivial matter. It means that you have to show that you can reformulate the problem as another one for which it is known that it is NP-hard and then obtain the solution of the original problem from the solution of the reformulated problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more common way is to see if someone has already looked at your particular problem. What you have here is a typical resource allocation problem. Consider, for example, your high school reps as a resource that the elementary school reps want to have, subject to certain rules (each elementary school rep needs the resource for a given length). This is no different than, say, considering buses as a resource that the routes want to have (for different amounts of time). It is, in other words, a rather common problem and there should be plenty of literature on finding efficient algorithms for this problem, as well as on which complexity class this problem falls into. (And no, sorry, I don't know the answer.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-29T11:13:43.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3164" PostTypeId="1" CreationDate="2012-08-29T11:54:36.280" Score="9" ViewCount="398" Body="&lt;p&gt;I have a problem where I need to find all positive (as in the eigenvalue is positive) eigenpairs of a small (usually smaller than 60x60) nonsymmetric matrix. I can stop calculating when the eigenvalue is smaller than a certain threshold. I know that the eigenvalues are real. Any suggestions on algorithms I could use to try to squeeze out the best performance? I have to do several thousand of these decompositions, so speed is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I need to do this on the GPU in shared memory. The matrices are also not necessarily the same size. I'm not aware of any libraries that do this at the moment. Suggestions of algorithms that would be well suited to the problem would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1966" LastEditorUserId="9" LastEditDate="2012-08-30T13:14:32.323" LastActivityDate="2012-08-30T13:14:32.323" Title="Fastest way to find eigenpairs of a small nonsymmetric matrix on a GPU in shared memory" Tags="|performance|eigensystem|gpu|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3165" PostTypeId="2" ParentId="3157" CreationDate="2012-08-29T12:20:47.563" Score="0" Body="&lt;p&gt;Your description reminds me of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Knapsack_problem&quot; rel=&quot;nofollow&quot;&gt;Knapsack problem&lt;/a&gt;, where the decision and optimization parts are NP-complete and NP-hard, respectively. The typical approach is to solve it using &lt;a href=&quot;http://en.wikipedia.org/wiki/Dynamic_programming&quot; rel=&quot;nofollow&quot;&gt;dynamic programming&lt;/a&gt; (&lt;a href=&quot;http://en.wikipedia.org/wiki/Memoization&quot; rel=&quot;nofollow&quot;&gt;memoization&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Greedy_algorithm&quot; rel=&quot;nofollow&quot;&gt;greedy algorithms&lt;/a&gt;). Googling for &lt;em&gt;knapsack scheduling&lt;/em&gt; also returns a lot of results (at least from my machine/ip). In wikipedia there is also a &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_knapsack_problems&quot; rel=&quot;nofollow&quot;&gt;list of knapsack-like problems&lt;/a&gt;. Reducing your problem to one of those in the list is enough for proving NP-hardness.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-08-29T12:56:51.053" LastActivityDate="2012-08-29T12:56:51.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3166" PostTypeId="2" ParentId="3164" CreationDate="2012-08-29T15:59:19.440" Score="2" Body="&lt;p&gt;Use the functions in LAPACK, it's unlikely that you can beat them in your own implementation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-08-29T15:59:19.440" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3167" PostTypeId="1" AcceptedAnswerId="3169" CreationDate="2012-08-29T21:15:18.223" Score="12" ViewCount="1641" Body="&lt;p&gt;I have the following optimization problem where I have absolute value in my constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\mathbf{x} \in \mathbb{R}^n$ and $\mathbf{f}_0, \mathbf{f}_1, \ldots, \mathbf{f}_m$ be column vectors of size $n$ each. We would like to solve the following:&#xA;\begin{align}&#xA;\min &amp;amp;\mathbf{f}_0^T \mathbf{x} &#xA;\notag \\&#xA;\text{s.t.}&#xA;&amp;amp;|\mathbf{f}_1^T \mathbf{x}| \leq |\mathbf{f}_2^T \mathbf{x}| \leq \ldots \leq |\mathbf{f}_m^T \mathbf{x}|&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the feasible space will not be convex and I will probably need an MILP to solve the problem. I'm looking for the least number of binary variables that I would need and the setup that would solve the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dealing with absolute values is generally easy when only one side of the inequality has an absolute value (http://lpsolve.sourceforge.net/5.1/absolute.htm); this case however seems to be more complicated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="1932" LastActivityDate="2012-08-30T10:20:27.833" Title="Absolute Value in Linear Constraints" Tags="|optimization|linear-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3168" PostTypeId="2" ParentId="3159" CreationDate="2012-08-29T21:32:58.500" Score="24" Body="&lt;p&gt;In addition to the reasons Wolfgang mentioned, if you use a &lt;code&gt;vector&amp;lt;vector&amp;lt;double&amp;gt; &amp;gt;&lt;/code&gt;, you'll have to dereference it twice every time you want to retrieve an element, which is more computationally costly than a single dereferencing operation. One typical approach is to allocate a single array (a &lt;code&gt;vector&amp;lt;double&amp;gt;&lt;/code&gt; or a &lt;code&gt;double *&lt;/code&gt;) instead. I've also seen people add syntactic sugar to matrix classes by wrapping around this single array some more intuitive indexing operations, to reduce the amount of &quot;mental overhead&quot; needed to invoke the proper indices.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-08-29T21:32:58.500" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3169" PostTypeId="2" ParentId="3167" CreationDate="2012-08-29T23:59:02.990" Score="5" Body="&lt;p&gt;The easiest way is to add $m$ binary values $s_i \in {0,1}$, and solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\min &amp;amp;\mathbf{f}_0^T \mathbf{x} &#xA;\notag \\&#xA;\text{s.t.} &amp;amp; 0 \le (2s_i-1) \mathbf{f}^T_i \mathbf{x} \le (2s_{i+1}-1) \mathbf{f}^T_{i+1} \mathbf{x}  &amp;amp; \forall i\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that either (1) nothing substantially faster exists or (2) there is a special trick to reformulate as a convex program.  Probably (1).&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-08-29T23:59:02.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3171" PostTypeId="2" ParentId="3164" CreationDate="2012-08-30T10:06:52.817" Score="3" Body="&lt;p&gt;Without doing a lot of search I recommend you to look at the &lt;a href=&quot;http://icl.cs.utk.edu/magma/&quot; rel=&quot;nofollow&quot;&gt;MAGMA&lt;/a&gt; library. Freely available code with continuous support. NVIDIA &lt;a href=&quot;http://icl.cs.utk.edu/magma/news/news.html?id=292&quot; rel=&quot;nofollow&quot;&gt;recognized&lt;/a&gt; MAGMA as a &quot;A Breakthrough in Solvers for Eigenvalue Problems&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also &lt;a href=&quot;http://www.culatools.com/&quot; rel=&quot;nofollow&quot;&gt;CULA&lt;/a&gt; library, which is generally commercial product, although recently it has been made free for academic usage (see details &lt;a href=&quot;http://www.culatools.com/academic-faq/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2012-08-30T10:12:01.637" LastActivityDate="2012-08-30T10:12:01.637" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3172" PostTypeId="2" ParentId="3167" CreationDate="2012-08-30T10:20:27.833" Score="3" Body="&lt;p&gt;You may use a solver for nonsmooth problems. See &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-08-30T10:20:27.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3175" PostTypeId="1" CreationDate="2012-08-30T15:42:06.330" Score="5" ViewCount="1884" Body="&lt;p&gt;Say I need to solve a PDE numerically. Depending on the problem and the numerical method chosen, I can usually see some issues coming: implementation issues (e.g. boundary conditions, parallelization), discretization issues (e.g. the stabilization of finite elements for hyperbolic problems is not funny while finite volumes &quot;just work&quot;, the &quot;opposite&quot; for elliptic problems), time integration...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I usually base the initial choice of the discretization on these &quot;intuitions&quot; only. Still, I would love to base them on facts and proof regarding &lt;em&gt;the&lt;/em&gt; question: how much computational effort would I have to put to solve my problem to a desired solution error (i.e. dollars/error)? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there any resource I could use to weight the suitability of a method with this criterion?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Although one could compute the complexity of the methods one still would need a priori error estimators that, if they exist at all, would have different &quot;sharpness&quot;, depending on the method. Is there a non-empirical way of finding this out? If testing all possible methods cannot be avoided, how would one go on about it? There are standard numerical testcases for most fields, would they be meaningful? Are there any public databases comparing implementations for these testcases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: This should not be the only criterion, implementing finite difference for complex boundaries is &quot;doable&quot; but a pain. Still, is it worth it to go through that pain? If the cost of using FD would be much lower than the alternatives, for some people, it might.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2012-08-30T22:56:08.140" LastActivityDate="2016-01-21T23:39:20.383" Title="Computational cost of numerical methods for PDEs" Tags="|pde|complexity|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3176" PostTypeId="2" ParentId="3175" CreationDate="2012-08-30T20:01:58.800" Score="9" Body="&lt;p&gt;You are asking for something overly ambitious if you hope to be able to reliably get within a constant factor (e.g. a couple orders of magnitude) of an optimal method, even just on one type of computer. If someone claimed to have a business analysis technique that could tell you the value of any company in five years, they would be accused of selling snake oil. A uniform optimality analysis for PDE solution methods is similarly unattainable. You will find that as you construct a classification methodology, you will have to severely limit the scope of problems and methods that you can treat, or pay the price of such huge uncertainty factors as to make the method analysis useless. The lack of a simple answer is one reason why &quot;optimal&quot; PDE solution methods is still an active research area.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-08-30T20:01:58.800" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3177" PostTypeId="1" CreationDate="2012-08-30T22:20:34.650" Score="19" ViewCount="768" Body="&lt;p&gt;I am having a bit of difficulty in trying to understand a paper. The paper uses spectral method to solve for an eigenvalue that comes from a system of coupled ODEs. I will write out only one equation now, because it is enough to get to the crux of my question(s). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V[r] = \frac{e^{-(\nu[r] +\lambda[r])}}{\epsilon[r] + p[r]} *\biggr[ (\epsilon[r] + p[r])( e^{\nu[r] +\lambda[r]})r W[r] \biggr]'$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I carry out the derivative and get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(Eq1)&lt;/strong&gt; $V = \biggr[ \frac{\epsilon' +p'}{\epsilon + p}  + r(\nu'+\lambda') +1 \biggr] W + r W'$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now according to the paper I should be able to expand equilibrium quantities $(\epsilon ,p ,\nu ,\lambda$) of the system as Chebyshev Polynomials of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B[r] = \Sigma_{i=0}^{\infty}b_i T_i[y] - \frac{1}{2} b_0 $, where $T_i[y]$ are the polynomials. I know how to get the $b_i$ using code I wrote in Mathematica. Also $y = 2(r/R) -1$, and the domain of $r$ is $(0,R)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper also states that the functions ($V,W$) can be expanded as&#xA;$F[r] = (\frac{r}{R})^l \Sigma_{i=0}^{\infty}f_i T_i[y] - \frac{1}{2} f_0 $, and that in general &#xA;a term like $B[r]F[r]$ can be expressed as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B[r]F[r] = \frac{1}{2} (\frac{r}{R})^l \Sigma_{i=0}^{\infty} \pi_i T_i[y] - \frac{1}{2} \pi _0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\pi_i = \Sigma_{j=0}^{\infty}[b_{i+j} + \Theta(j-1)b_{|i-1|} ] f_l $&#xA;and $\Theta(k) = 0$ for $k&amp;lt;0$ and &lt;strong&gt;equals 1&lt;/strong&gt; for $k\geq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With all that being said, lets say  I make the following equilibrium functions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\epsilon' +p'}{\epsilon + p} = B_1[r] $ and $ r(\nu'+\lambda') = B_2[r]$, Then Eq1 becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(Eq2)&lt;/strong&gt; $ \bigg((\frac{r}{R})^l \Sigma_{i=0}^{\infty}v_i T_i[y] - \frac{1}{2} v_0 \bigg) =  \biggr[ B_1[r]  + B_2[r] +1 \biggr] \biggr( (\frac{r}{R})^l \Sigma_{i=0}^{\infty}w_i T_i[y] - \frac{1}{2} w_0 \biggr)  + r W' $. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question1:&lt;/strong&gt; What do I do with the $(\frac{r}{R})^l $terms? The polynomials are functions of $[y]$ so how can I even have an expansion like $B[r]=  (\frac{r}{R})^l$ X function of [y]? Also it seems like I can just divide them out on each side of the equation, so what was the point of introduction that term? I mean, according to the paper this term is supposed to impose the boundary condition that $V,W$ go to zero as $r$ goes to zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;*&lt;em&gt;Question2:&lt;/em&gt;*How am I supposed to deal with the $r$ in the $r*W'$ term. The paper gives a description of how to handle derivative terms, but what about the $r$ itself. Am I supposed to treat it like an equilibrium value  and use the rule for terms like $B[r]F[r]$ or should I express this $r$ in terms of $y$. Or should I do something else altogether?&lt;/p&gt;&#xA;" OwnerUserId="1970" LastEditorUserId="1280" LastEditDate="2015-05-02T01:23:32.100" LastActivityDate="2015-06-03T03:57:59.993" Title="Difficulty with Spectral Method using Chebyshev Polynomials" Tags="|eigenvalues|polynomials|spectral-method|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3178" PostTypeId="2" ParentId="3175" CreationDate="2012-08-30T22:38:09.113" Score="2" Body="&lt;p&gt;The computational cost depends not just on the number of DOF, but also on how efficient you can solve the resulting (nonlinear?) equation system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an extreme case, the time harmonic Maxwell equations for a structure periodic in x- and y-direction can be expanded into Fourier modes (in x- and y-direction). The resulting system has very few degrees of freedom. It can be solved by various methods, but it ultimately boils down to solving a two point boundary value problem by some sort of shooting method. If you resort to matrix methods for the stable solution of this problem, the runtime scales with the third power of the area of the periodic unit cell. If you adapt a simple FDTD scheme to the solution of the time harmonic Maxwell equation instead, you end up with many degrees of freedom. You don't need to solve an equation system in this case, and the runtime scales linear with the area of the periodic unit cell. (Both approaches can be modified to overcome their main limitations, but these modifications are also not without issues.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the time harmonic Maxwell equations are elliptic, but time domain problem solved by FDTD is hyperbolic.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-08-30T22:38:09.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3180" PostTypeId="2" ParentId="3130" CreationDate="2012-08-31T14:33:19.940" Score="1" Body="&lt;p&gt;Also think about the loops over faces where you compute the flux once and update the left and right cell (contributions LL,LR, RL and RR in the matrix). &lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2012-08-31T14:33:19.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3181" PostTypeId="2" ParentId="3137" CreationDate="2012-08-31T18:58:27.290" Score="1" Body="&lt;p&gt;RANS models, more precisely those based on transport equations of $\omega$ may be used as a rudimentary transition models.&#xA;Transition due to growing instabilities though, cannot be predicted by RANS models. By-pass transitions (with turbulence intensity $Tu &amp;gt; 1$ % )can be predicted with Low-Re RANS models owing to action of diffusion terms in transport equations which bring turbulence from free stream to near wall region, leading to production of more turbulence and then to transition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method however is considered unreliable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More advanced models are those like Menter and Langtry (based on SST model) which solve additional transport equation for intermitency $\gamma$. When gamma increases it multiplies $P_k$, the production of turbulence kinetic energy. Intermitency factor itself tells you what persentage of measured time fluid flow is turbulent at certain fixed position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you still want to use RANS models for predictions of transition in OpenFOAM, either search for implementation of Menter&amp;amp;Langtry model or implement it yourself. Word goes arround that implementing new transport equation in this code is easy.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-08-31T18:58:27.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3182" PostTypeId="2" ParentId="3159" CreationDate="2012-08-31T19:49:41.267" Score="7" Body="&lt;p&gt;Is it really such a bad thing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@Wolfgang: Depending on the size of the dense matrix, two additional pointer per row might be negligable. Concerning scattered data one could think of using a custom allocator that makes sure that the vectors are in contiguous memory. As long as memory is not recycled even the standard allocator will us contiguous memory with a two pointer size gap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@Geoff: If you are doing random access and use just one array you still have to calculate the index. Might not be faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let us do a small test:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;vectormatrix.cc:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include&amp;lt;vector&amp;gt;&#xA;#include&amp;lt;iostream&amp;gt;&#xA;#include&amp;lt;random&amp;gt;&#xA;#include &amp;lt;functional&amp;gt;&#xA;#include &amp;lt;sys/time.h&amp;gt;&#xA;&#xA;int main()&#xA;{&#xA;  int N=1000;&#xA;  struct timeval start, end;&#xA;&#xA;  std::cout&amp;lt;&amp;lt; &quot;Checking differenz between last entry of previous row and first entry of this row&quot;&amp;lt;&amp;lt;std::endl;&#xA;  std::vector&amp;lt;std::vector&amp;lt;double&amp;gt; &amp;gt; matrix(N, std::vector&amp;lt;double&amp;gt;(N, 0.0));&#xA;  for(std::size_t i=1; i&amp;lt;N;i++)&#xA;    std::cout&amp;lt;&amp;lt; &quot;index &quot;&amp;lt;&amp;lt;i&amp;lt;&amp;lt;&quot;: &quot;&amp;lt;&amp;lt;&amp;amp;(matrix[i][0])-&amp;amp;(matrix[i-1][N-1])&amp;lt;&amp;lt;std::endl;&#xA;  std::cout&amp;lt;&amp;lt;&amp;amp;(matrix[0][N-1])&amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt;&amp;amp;(matrix[1][0])&amp;lt;&amp;lt;std::endl;&#xA;  gettimeofday(&amp;amp;start, NULL);&#xA;  int k=0;&#xA;&#xA;  for(int j=0; j&amp;lt;100; j++)&#xA;    for(std::size_t i=0; i&amp;lt;N;i++)&#xA;      for(std::size_t j=0; j&amp;lt;N;j++, k++)&#xA;        matrix[i][j]=matrix[i][j]*matrix[i][j];&#xA;  gettimeofday(&amp;amp;end, NULL);&#xA;  double seconds  = end.tv_sec  - start.tv_sec;&#xA;  double useconds = end.tv_usec - start.tv_usec;&#xA;&#xA;  double mtime = ((seconds) * 1000 + useconds/1000.0) + 0.5;&#xA;&#xA;  std::cout&amp;lt;&amp;lt;&quot;calc took: &quot;&amp;lt;&amp;lt;mtime&amp;lt;&amp;lt;&quot; k=&quot;&amp;lt;&amp;lt;k&amp;lt;&amp;lt;std::endl;&#xA;&#xA;  std::normal_distribution&amp;lt;double&amp;gt; normal_dist(0, 100);&#xA;  std::mt19937 engine; // Mersenne twister MT19937&#xA;  auto generator = std::bind(normal_dist, engine);&#xA;  for(std::size_t i=1; i&amp;lt;N;i++)&#xA;    for(std::size_t j=1; j&amp;lt;N;j++)&#xA;      matrix[i][j]=generator();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And now using one array:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;arraymatrix.cc&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #include&amp;lt;vector&amp;gt;&#xA;#include&amp;lt;iostream&amp;gt;&#xA;#include&amp;lt;random&amp;gt;&#xA;#include &amp;lt;functional&amp;gt;&#xA;#include &amp;lt;sys/time.h&amp;gt;&#xA;&#xA;int main()&#xA;{&#xA;  int N=1000;&#xA;  struct timeval start, end;&#xA;&#xA;  std::cout&amp;lt;&amp;lt; &quot;Checking difference between last entry of previous row and first entry of this row&quot;&amp;lt;&amp;lt;std::endl;&#xA;  double* matrix=new double[N*N];&#xA;  for(std::size_t i=1; i&amp;lt;N;i++)&#xA;    std::cout&amp;lt;&amp;lt; &quot;index &quot;&amp;lt;&amp;lt;i&amp;lt;&amp;lt;&quot;: &quot;&amp;lt;&amp;lt;(matrix+(i*N))-(matrix+(i*N-1))&amp;lt;&amp;lt;std::endl;&#xA;  std::cout&amp;lt;&amp;lt;(matrix+N-1)&amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt;(matrix+N)&amp;lt;&amp;lt;std::endl;&#xA;&#xA;  int NN=N*N;&#xA;  int k=0;&#xA;&#xA;  gettimeofday(&amp;amp;start, NULL);&#xA;  for(int j=0; j&amp;lt;100; j++)&#xA;    for(double* entry =matrix, *endEntry=entry+NN;&#xA;        entry!=endEntry;++entry, k++)&#xA;      *entry=(*entry)*(*entry);&#xA;  gettimeofday(&amp;amp;end, NULL);&#xA;  double seconds  = end.tv_sec  - start.tv_sec;&#xA;  double useconds = end.tv_usec - start.tv_usec;&#xA;&#xA;  double mtime = ((seconds) * 1000 + useconds/1000.0) + 0.5;&#xA;&#xA;  std::cout&amp;lt;&amp;lt;&quot;calc took: &quot;&amp;lt;&amp;lt;mtime&amp;lt;&amp;lt;&quot; k=&quot;&amp;lt;&amp;lt;k&amp;lt;&amp;lt;std::endl;&#xA;&#xA;  std::normal_distribution&amp;lt;double&amp;gt; normal_dist(0, 100);&#xA;  std::mt19937 engine; // Mersenne twister MT19937&#xA;  auto generator = std::bind(normal_dist, engine);&#xA;  for(std::size_t i=1; i&amp;lt;N*N;i++)&#xA;      matrix[i]=generator();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On my system there is now clear winner (Compiler gcc 4.7 with -O3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time vectormatrix prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;index 997: 3&#xA;index 998: 3&#xA;index 999: 3&#xA;0xc7fc68 0xc7fc80&#xA;calc took: 185.507 k=100000000&#xA;&#xA;real    0m0.257s&#xA;user    0m0.244s&#xA;sys     0m0.008s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We also see, that as long as the standard allocator does not recycle freed memory, the data is contiguous. (Of course after some deallocations there is no guarantee for this.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time arraymatrix prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;index 997: 1&#xA;index 998: 1&#xA;index 999: 1&#xA;0x7ff41f208f48 0x7ff41f208f50&#xA;calc took: 187.349 k=100000000&#xA;&#xA;real    0m0.257s&#xA;user    0m0.248s&#xA;sys     0m0.004s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1605" LastActivityDate="2012-08-31T19:49:41.267" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3183" PostTypeId="1" AcceptedAnswerId="3184" CreationDate="2012-09-01T15:38:16.263" Score="7" ViewCount="174" Body="&lt;p&gt;I have formulated a linear program with binary indicator variables $z_i(a)$ which is equal to $1$ if the $i^{th}$ document is of rank $a$ and $0$ otherwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other variables in the linear program, $z^1_{ij}(a), z^2_{ij}(a)$ are defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;z^1_{ij}(a) \equiv z_i(a)  \sum_{b&amp;lt;a} z_j(b), &#xA; \\&#xA;z^2_{ij}(a) \equiv z_i(a)  \sum_{b\geq a} z_j(b).&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to convert the above non-linear constraint to the following set of equivalent linear constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$z^1_{ij}(a) + z^2_{ij}(a) = z_i(a), \forall i, j, a$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I am facing is that, the above set of linear constraints are clearly not equivalent to the definition of $z^1_{ij}(a), z^2_{ij}(a)$. Any idea if it is possible to represent non-linear ranking type constraints as equivalent linear constraints?&lt;/p&gt;&#xA;" OwnerUserId="1570" LastEditorUserId="1570" LastEditDate="2012-09-01T18:57:07.213" LastActivityDate="2012-09-01T20:24:56.567" Title="Is it possible to represent non-linear ranking type constraints as equivalent linear constraints?" Tags="|optimization|linear-programming|convex-optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3184" PostTypeId="2" ParentId="3183" CreationDate="2012-09-01T19:02:34.330" Score="3" Body="&lt;p&gt;There are several ways to convert your model to linear constraints.  For example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;z^1_{ij}(a) + z^2_{ij}(a) &amp;amp;=&amp;amp; z_i(a) \ \ \forall i,j,a \\ &#xA;z^1_{ij}(a) &amp;amp;\le&amp;amp; 1- \sum_{b \ge a} z_j(b) \ \ \forall i,j,a \\&#xA;z^2_{ij}(a) &amp;amp; \le&amp;amp; 1- \sum_{b &amp;lt; a} z_j(b) \ \ \forall i,j,a &#xA;\end{eqnarray}&lt;/p&gt;&#xA;" OwnerUserId="996" LastActivityDate="2012-09-01T19:02:34.330" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3185" PostTypeId="2" ParentId="3183" CreationDate="2012-09-01T20:24:56.567" Score="3" Body="&lt;p&gt;In general, whenever you have a mixed-integer program where the only nonlinearities are polynomials of binary variables, it is possible to reformulate the program so that it is a mixed-integer linear program, using the work of Fred Glover, and subsequent related work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;F. Glover. Further reduction of zero-one polynomial programming problems to zero-one linear programming problems. &lt;em&gt;Operations Research&lt;/em&gt;, Volume 21, pages 156-161, 1971.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;F. Glover, E. Woolsey. Converting the 0-1 polynomial programming problem to a 0-1 linear program. &lt;em&gt;Operations Research&lt;/em&gt;, Volume 22, pages 180-182, 1974.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;F. Glover. Improved linear integer programming formulations of nonlinear integer problems. &lt;em&gt;Management Science&lt;/em&gt;, Volume 22, pages 455-460, 1975.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;F. E. Torres. Linearization of mixed-integer products. &lt;em&gt;Mathematical Programming&lt;/em&gt;, Volume 49, pages 427-428, 1991.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;O. Kettani, M. Oral. Equivalent formulations for nonlinear integer problems for efficient optimization. &lt;em&gt;Management Science&lt;/em&gt;, Volume 36, pages 115-119, 1990.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-09-01T20:24:56.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3186" PostTypeId="5" CreationDate="2012-09-01T22:40:22.617" Score="0" Body="&lt;p&gt;Convex Optimization is a special case of mathematical optimization where the feasible region is convex and the objective is to either minimize a convex function or maximize a concave function. &lt;a href=&quot;https://scicomp.stackexchange.com/tags/linear-programming/info&quot;&gt;Linear Programming&lt;/a&gt; is a special case. Convex Optimization problems as a class are easier to solve numerically than general mathematical optimization problems.&lt;/p&gt;&#xA;" OwnerUserId="996" LastEditorDisplayName="user9927" LastEditDate="2017-08-27T17:21:58.267" LastActivityDate="2017-08-27T17:21:58.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3187" PostTypeId="4" CreationDate="2012-09-01T22:40:22.617" Score="0" Body="Convex Optimization is a special case of mathematical optimization where the feasible region is convex and the objective is to either minimize a convex function or maximize a concave function." OwnerUserId="996" LastEditorUserId="996" LastEditDate="2012-09-01T22:42:14.533" LastActivityDate="2012-09-01T22:42:14.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3188" PostTypeId="1" AcceptedAnswerId="3195" CreationDate="2012-09-02T17:23:18.223" Score="29" ViewCount="19112" Body="&lt;p&gt;In statistics and its various applications, we often calculate the &lt;a href=&quot;http://en.wikipedia.org/wiki/Covariance_matrix&quot;&gt;covariance matrix&lt;/a&gt;, which is positive definite (in the cases considered) and symmetric, for various uses. Sometimes, we need the inverse of this matrix for various computations (quadratic forms with this inverse as the (only) center matrix, for example). Given the qualities of this matrix, and the intended uses, I wonder:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best, in terms of numerical stability, way to go about computing or using (let's say for quadratic forms or matrix-vector multiplication in general) this inverse? Some factorization that can come in handy?&lt;/p&gt;&#xA;" OwnerUserId="1979" LastEditorUserId="386" LastEditDate="2013-07-03T15:56:34.790" LastActivityDate="2017-03-29T18:46:13.090" Title="Dealing with the inverse of a positive definite symmetric (covariance) matrix?" Tags="|linear-algebra|matrix|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3189" PostTypeId="1" AcceptedAnswerId="3196" CreationDate="2012-09-02T18:00:27.287" Score="13" ViewCount="640" Body="&lt;p&gt;I have been reading the book &lt;a href=&quot;http://books.google.com/books/about/Computer_Simulation_of_Liquids.html?id=O32VXB9e5P4C&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Computer Simulation of Liquids&lt;/em&gt;&lt;/a&gt; by Allen and Tildesley.  Starting on page 71, the authors discuss the various algorithms that are used to integrate Newton's equations of motion in molecular dynamics (MD) simulations.  Starting on page 78, the authors discuss the Verlet algorithm, which is perhaps the canonical integration algorithm in MD.  They state:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Perhaps the most widely used method of integrating the equations of motion is that initially adopted by Verlet (1967) and attributed to Stormer (Gear 1971).  This method is a direct solution of the second-order equation &lt;span class=&quot;math-container&quot;&gt;$m_i \ddot{\textbf{r}}_i = \textbf{f}_i$&lt;/span&gt;.  The method is based on postions &lt;span class=&quot;math-container&quot;&gt;$\textbf{r}(t)$&lt;/span&gt;, accelerations &lt;span class=&quot;math-container&quot;&gt;$\textbf{a}(t)$&lt;/span&gt;, and the positions &lt;span class=&quot;math-container&quot;&gt;$\textbf{r}(t - \delta t)$&lt;/span&gt; from the previous step.  The equation for advancing the positions reads as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\tag{3.14}\textbf{r}(t + \delta t) = 2\textbf{r}(t) - \textbf{r}(t - \delta t) + \delta t^2 \textbf{a}(t).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are several points to note about eqn (3.14).  It will be seen that the velocities do not appear at all.  They have been eliminated by addition of the equations obtained by Taylor expansion about &lt;span class=&quot;math-container&quot;&gt;$\textbf{r}(t)$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\textbf{r}(t + \delta t) = \textbf{r}(t) + \delta t \textbf{v}(t) + (1/2) \delta t^2 \textbf{a}(t) + ...$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\tag{3.15}\textbf{r}(t - \delta t) = \textbf{r}(t) - \delta t \textbf{v}(t) + (1/2) \delta t^2 \textbf{a}(t) - ... .$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then, later (on page 80), the authors state:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Against the Verlet algorithm, ... the form of the algorithm may needlessly introduce some numerical imprecision.  This arises because, in eqn (3.14), a small term (&lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\delta t^2)$&lt;/span&gt;) is added to a difference of large terms (&lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\delta t^0)$&lt;/span&gt;), in order to generate the trajectory.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I guess that the &amp;quot;small term&amp;quot; is &lt;span class=&quot;math-container&quot;&gt;$\delta t^2 \textbf{a}(t)$&lt;/span&gt;, and the &amp;quot;difference of large terms&amp;quot; is &lt;span class=&quot;math-container&quot;&gt;$2\textbf{r}(t) - \textbf{r}(t - \delta t)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My question is, why does numerical imprecision result from adding a small term to a difference of large terms?&lt;/p&gt;&#xA;&lt;p&gt;I am interested in a rather basic, conceptual reason, since I am not familiar at all with details of floating point arithmetic.  Also, do you know of any &amp;quot;overview-type&amp;quot; references (books, articles, or websites) that would introduce me to basic ideas of floating point arithmetic related to this question?  Thanks for your time.&lt;/p&gt;&#xA;" OwnerUserId="1454" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-09-03T17:27:49.963" Title="In floating point arithmetic, why does numerical imprecision result from adding a small term to a difference of large terms?" Tags="|floating-point|computer-arithmetic|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3190" PostTypeId="2" ParentId="3189" CreationDate="2012-09-02T18:35:36.043" Score="10" Body="&lt;p&gt;If you're looking for a good introduction, I would suggest David Goldberg's &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.22.6768&quot;&gt;What Every Computer Scientist Should Know About Floating Point Arithmetic&lt;/a&gt;. It may be a bit too detailed, but it's available online for free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you've got a good library, I'd suggest Michael Overton's &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898714826&quot;&gt;Numerical Computing with IEEE Floating Point Arithmetic&lt;/a&gt;, or the first few chapters of Nick Higham's &lt;a href=&quot;http://www.maths.manchester.ac.uk/~higham/asna/index.php&quot;&gt;Accuracy and Stability of Numerical Algorithms&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What Allen and Tildesley are referring to specifically is numerical &lt;a href=&quot;http://en.wikipedia.org/wiki/Loss_of_significance&quot;&gt;Cancellation&lt;/a&gt;. The short of it is that if you have, say, only three digits and you subtract &lt;code&gt;100&lt;/code&gt; from &lt;code&gt;101&lt;/code&gt;, you get &lt;code&gt;1.00&lt;/code&gt; (in three digits). The number looks like it is accurate to three digits, but actually, only the first digit is true and the trailing &lt;code&gt;.00&lt;/code&gt; are garbage. Why? Well, &lt;code&gt;100&lt;/code&gt; and &lt;code&gt;101&lt;/code&gt; are only inexact representations of, say &lt;code&gt;100.12345&lt;/code&gt; and &lt;code&gt;101.4321&lt;/code&gt;, but you could only store them as three-digit numbers.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-09-02T18:35:36.043" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3191" PostTypeId="1" CreationDate="2012-09-02T21:07:40.743" Score="5" ViewCount="10322" Body="&lt;p&gt;I'm experimenting with OpenFOAM. In my simulation of a time-dependent, transitional flow the Courant number remains well below 1 for a certain period but then rises, eventually leading to a floating point exception. In order to avoid it, I reduced the time step, with the consequence of an unacceptable slowdown of the simulation. Is there a possibility of dynamically adjusting the time step in dependence of the Courant number?&lt;/p&gt;&#xA;" OwnerUserId="1616" LastEditorUserId="1616" LastEditDate="2012-09-03T20:15:27.473" LastActivityDate="2012-09-03T20:15:27.473" Title="Adjusting the time step dynamically in OpenFOAM?" Tags="|fluid-dynamics|openfoam|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3192" PostTypeId="2" ParentId="3189" CreationDate="2012-09-02T21:33:49.023" Score="3" Body="&lt;p&gt;Pedro already gives the important fact, namely cancellation. The point is that every number you compute with has an associated accuracy; for example, a single precision floating point number can only represent things up to approximately 8 digits of accuracy. If you have two numbers that are almost exactly the same but differ in the 7th digit, then the difference will again be an 8-digit single precision floating point number and it looks like it is accurate to 8 digits, but in reality only the first 1 or 2 digits are accurate because the quantities you computed it from are not accurate beyond this first 1 or 2 digits of the difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the book you cite is from 1989. Back then, computations were most often done in single precision and round-off and cancellation were serious problems. Today, most computations are done using double precision with 16 digits of accuracy, and this is far less a problem today than it used to be. I think it is worthwhile reading the paragraphs you cite with a grain of salt and take them in the context of their time.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-02T21:33:49.023" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3193" PostTypeId="2" ParentId="3188" CreationDate="2012-09-02T22:09:21.710" Score="2" Body="&lt;p&gt;If you know that the matrix has an inverse (i.e., if it is indeed positive definite) and if it isn't too large, then the Cholesky decomposition gives an appropriate means to characterize the inverse of a matrix.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-02T22:09:21.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3194" PostTypeId="1" AcceptedAnswerId="3200" CreationDate="2012-09-03T08:55:35.163" Score="5" ViewCount="209" Body="&lt;p&gt;I'm using the cell-centered finite-volume method (for example Morton, Numerical solution of convection-diffusion problems, Chapman&amp;amp;Hall, 1996) to discretize the advection-diffusion equation and the incompressible Navier-Stokes equations. To check the implementation, I'm using the method of manufactured solutions (Roache, Fundamentals of verification and validation, Hermosa publishers, 2009). So I need to know the theoretical order of accuracy to compare with the observed order in my numerical experiments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper by Weiser and Wheeler (SIAM Journal on Numerical Analysis, 25(2), 1988) proves 2nd order of accuracy for the advection-diffusion equation, both for the solution and its gradient on Cartesian but not necessarily uniform grids. I do get 2nd order for Cartesian grids including stretching, also for the incompressible Navier-Stokes equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For non-Cartesian grids however, the velocity is still 2nd order, but the velocity gradient drops to 1st order and so does the pressure. In some cases, it even drops to half-order. Is there any theoretical analysis that covers this situation? I know that there are many papers proposing all kinds of corrections for non-Cartesian grids based on numerical evidence, but I'm looking specifically for analysis. The nicest thing would be an overview paper that summarizes a number of propositions and analyzes whether they actually improve the order of accuracy instead of reducing the constant.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2012-09-04T01:40:22.483" Title="a priori error analysis of cell-centered finite-volume methods" Tags="|error-estimation|finite-volume|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3195" PostTypeId="2" ParentId="3188" CreationDate="2012-09-03T09:18:29.497" Score="17" Body="&lt;p&gt;The Cholesky factorization $C=R^TR$ leads to a Cholesky-like factorization of the inverse $C^{-1}=SS^T$ with the upper triangular matrix $S=R^{-1}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, is best to keep the inverse factored. If $R$ is sparse then it is usually even better to keep $S$ implicit, as matrix-vector products $y=C^{-1}x$ can be computed by solving the two triangular systems $R^Tz=x$ and $Ry=z$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-03T09:18:29.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3196" PostTypeId="2" ParentId="3189" CreationDate="2012-09-03T09:33:22.537" Score="9" Body="&lt;p&gt;Their observation ''the form of the algorithm may needlessly introduce some numerical imprecision'' is correct. But their explanation&#xA;''This arises because, in eqn (3.14), a small term ($O(δt^2)$) is added to a difference of large terms ($O(δt^0)$), in order to generate the trajectory.''&#xA;is spurious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The true reason for the slight numerical instability of Verlet's algorithm is that it is only marginally stable, because the difference equation $x_{k+1}=2x_k-x_{k-1}$ (essentially the case where you neglect $a$ in Verlet) has a parasitic solution proportional to $k$, which causes errors introduced to grow linearly in $k$ whereas for a fully stable multistep method applied to a dissipative differential equation, error growth is bounded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Note that the book is about numerical simulation of molecular dynamics, and for getting a reasonable accuracy of the resulting expectations one needs a huge number$N$ of steps, as the accuracy scales with $O(N^{-1/2})$ only. (Often the time step is in the picoseconds to follow the intrinsic oscillation scale. But biologically relevant time scales are in the milliseconds or larger ($N\sim 10^9$), though usually one doesn't compute that far.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details, see&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_multistep_method#Stability_and_convergence&quot;&gt;http://en.wikipedia.org/wiki/Linear_multistep_method#Stability_and_convergence&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-03T17:27:49.963" LastActivityDate="2012-09-03T17:27:49.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3197" PostTypeId="2" ParentId="3189" CreationDate="2012-09-03T14:57:11.383" Score="5" Body="&lt;p&gt;To apply Pedro's example to the equation $(3.14)$, assume your variables are stored with the following values:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{r}(t) = 101&#xA;$$&#xA;$$&#xA;\mathbf{r}(t - \delta t) = 100&#xA;$$&#xA;$$&#xA;\delta t^2 \mathbf{a}(t) = 1.49&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From $(3.14)$ it should follow that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{r}(t + \delta t) = 103.49&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but, since we can only use three digits, the result becomes truncated to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{r}(t + \delta t) = 103&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This error will propagate, so that after 20 steps, assuming $\mathbf{a}(t)$ remains unchanged, you get $\mathbf{r}(t + 20\delta t) = 331$ instead of $433.90$, &lt;/p&gt;&#xA;" OwnerUserId="1616" LastActivityDate="2012-09-03T14:57:11.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3198" PostTypeId="2" ParentId="3191" CreationDate="2012-09-03T15:41:54.603" Score="6" Body="&lt;p&gt;pimpleFoam is the OpenFOAM application designated for transient, incompressible flow which can be either laminar or turbulent. The solver does support dynamic adjustment of the time step based on the Courant number as well as relaxation of the transport equations to improve performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check if a solver supports adaptive time stepping by having a look at the program code, specifically the main file of a solver. If the line &lt;em&gt;#include &quot;setDeltaT.H&quot;&lt;/em&gt; is present within the time loop &lt;em&gt;while (runTime.run())&lt;/em&gt;, the solver application is enabled to work with adaptive time stepping (compare e.g. pimpleFoam and pisoFoam).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see how to use adaptive time stepping, have a look at the tutorials included with OpenFOAM (tutorials/incompressible/pimpleFoam/).&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="59" LastEditDate="2012-09-03T15:46:58.747" LastActivityDate="2012-09-03T15:46:58.747" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3199" PostTypeId="1" AcceptedAnswerId="3201" CreationDate="2012-09-03T19:22:08.957" Score="8" ViewCount="748" Body="&lt;p&gt;I recently inherited a large body of legacy code that solves a very stiff, transient problem.  I would like to demonstrate that the spatial and temporal step sizes are small enough that the qualitative nature of the computed solution would not change if they were decreased.  In other words, I'd like to show that the solution is &quot;converged&quot; in a qualitative sense.  Since I can explicitly set the spatial mesh size, that part is straightforward.  However, as the code uses automatic time-step size control, I cannot set the time step size directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm changes the time step between two bounds based on the number of jacobian iterations needed to reach an error tolerance during the last $n$ time steps.  The fact that it uses jacobian iteration makes me fairly certain that it is some sort of implicit scheme, but I cannot be absolutely certain.  It does not account for the error it is seeing in the current time step, which leads to it running into the iteration limit on occasion (maybe a dozen times over the course of several thousand time steps, almost always during the most dynamic portions of the simulation).  The current runs I am completing I am setting the time-step bounds two and a half orders of magnitude apart ($10^{-13}$ to $5 \cdot 10^{-11}$).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the runs, I have control over the time-step bounds, the number of past time-steps it looks at to choose the current time step, the maximum change in the time step (ratios), the target number of jacobian iterations, the maximum number of iterations, and the error bound.  I would like if someone could put me on the right path to analyzing the time-step independence, or at the very least figuring out what algorithm is used.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="123" LastEditDate="2012-09-04T05:25:36.733" LastActivityDate="2012-09-04T18:02:33.293" Title="Demonstrating that the time step size is small enough in a code with automatic step size selection" Tags="|convergence|accuracy|time-integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3200" PostTypeId="2" ParentId="3194" CreationDate="2012-09-04T01:40:22.483" Score="1" Body="&lt;p&gt;You are looking for the term &quot;superconvergence&quot;. One frequently observes (and can sometimes prove) that on regular enough meshes the convergence order is higher than it should really be (e.g. that the gradient converges at the same order as the values, at least at certain points). This property is typically lost if you destroy certain symmetry properties of the mesh.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-04T01:40:22.483" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3201" PostTypeId="2" ParentId="3199" CreationDate="2012-09-04T05:23:08.050" Score="5" Body="&lt;p&gt;The purpose of automatic error estimation and step size control is to free you from the problem of determining manually what a sufficiently small step size is.  So your question is a bit like asking &quot;somebody gave me this automatic transmission car; how can I tell what gear I'm in?&quot;  The point is that you shouldn't need to know.  Of course, if the transmission is faulty, then you might need to take it apart and fix it, but that's a much bigger problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, typically the right approach is to determine what kind of error is acceptable and impose that through the automatic step size control.  It's imperfect because error control in this sense is usually only &lt;strong&gt;local error control&lt;/strong&gt;, so you don't directly control the global error, which is what you probably care about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you could easily do if you're in doubt, is to run the simulation with a sequence of increasingly tight (i.e. small) error tolerances.  Once the solution seems insensitive (in whatever your metric is) to decreasing the tolerance, you can stop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum&lt;/strong&gt;: Regarding the issue of the maximum iteration limit being reached (which leads to a local error exceeding the specified tolerance), I suggest the following.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently the code thinks that if it exceeds the maximum number of iterations, the &lt;em&gt;right&lt;/em&gt; thing to do is to accept the step.  I would say that is the wrong thing.  A better approach is to reject the step and start that step again with a reduced step size.  Of course, there is the danger of the step being reduced below the minimum step size.  In that case, the &lt;em&gt;right&lt;/em&gt; thing is to abort the simulation.  But if you believe that a wrong solution is better than no solution, you could just accept the step and go on if &lt;strong&gt;both&lt;/strong&gt; conditions are met: the minimum step size is reached &lt;strong&gt;and&lt;/strong&gt; the maximum number of iterations is exceeded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a well-designed code, making these types of changes is trivial, but in an arbitrary code it may be arbitrarily difficult.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-09-04T18:02:33.293" LastActivityDate="2012-09-04T18:02:33.293" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3202" PostTypeId="1" AcceptedAnswerId="3203" CreationDate="2012-09-04T10:44:28.513" Score="11" ViewCount="4775" Body="&lt;p&gt;Is there a open C-implementation for the solution of quartic equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ax⁴+bx³+cx²+dx+e=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking of an implementation of Ferrari's solution. On Wikipedia I read that the solution is computational stable only for some of the possible sign combinations of the coefficients. But maybe I am lucky ... I got a pragmatic solution by solving analytically using a computer algebra system and exporting to C. But if there is a tested implementation I would prefer to use this. I search a fast method and prefer not to use a general root finder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need only real solutions.&lt;/p&gt;&#xA;" OwnerUserId="1943" LastEditorUserId="123" LastEditDate="2012-09-04T18:11:33.547" LastActivityDate="2017-01-28T00:49:54.143" Title="Solution of quartic equation" Tags="|polynomials|nonlinear-equations|roots|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3203" PostTypeId="2" ParentId="3202" CreationDate="2012-09-04T11:14:57.213" Score="20" Body="&lt;p&gt;I would strongly advice against using closed form solutions since they tend to be numerically very unstable. You need to take extreme care in the way and order of your evaluations of the discriminant and other parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classical example is the one for the quadratic equation $ax^2+bx+c=0$. Calculating the roots as $$x_{1,2} = \frac{-b \pm \sqrt{b^2-4ac}}{2a}$$ will get you into trouble for polynomials where $b\gg 4ac$ since then you get cancellation in the numerator. You need to calculate $$x_1 = \frac{-(b+sign(b)\sqrt{b^2-4ac})}{2a}; x_2 = \frac{c}{a}\frac{1}{x_1}$$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Higham in his masterpiece &quot;Accuracy and Stability of Numerical Algorithms&quot; (2nd ed, SIAM) uses a direct search method to find coefficients of a cubic polynomial for which the classical analytical cubic solution gives very inaccurate results. The example he gives is $[a,b,c] = [1.732, 1, 1.2704]$. For this polynomial the roots are well-separated and hence the problem is not ill-conditioned. However, if he calculates the roots using the analytical approach, and evaluates the polynomial in these roots, he obtains a residue of $\mathcal{O}(10^{-2})$ while using a stable standard method (the companion matrix method), the residue is of order $\mathcal{O}(10^{-15})$. He proposes a slight modification to the algorithm, but even then, he can find a set of coefficients leading to residues of $\mathcal{O}(10^{-11})$ which is definitely not good. See p480-481 of the above mentioned book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, I would apply &lt;a href=&quot;http://en.wikipedia.org/wiki/Bairstow%27s_method&quot;&gt;Bairstow's method&lt;/a&gt;. It uses an iterative combination of Newton iteration on quadratic forms (and then the roots of the quadratic are solved) and deflation. It is easily implemented and there are even some implementations available on the web.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2012-09-05T12:57:01.407" LastActivityDate="2012-09-05T12:57:01.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3204" PostTypeId="1" AcceptedAnswerId="3205" CreationDate="2012-09-04T12:39:12.567" Score="5" ViewCount="3969" Body="&lt;p&gt;I have a data set containing the recorded data from a car's motion (latitude,longitude, and heading).  I'd like to plot the (lat,lon) points on a 2D plot with a unit vector pointing in the direction of the car's heading at each point.  The heading is given in azimuth degrees so I think I would first have to convert that to radians then calculate the (x,y) point that represents the head of the unit vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have any trouble plotting the (lat,lon) points but I have no idea about the vectors.  I'm pretty new to Octave but I do know at least basic C/C++.  An animated plot would be even better if anyone knows how to set that up.  &lt;/p&gt;&#xA;" OwnerUserId="1984" LastActivityDate="2012-09-04T12:49:10.180" Title="Plotting x,y coordinates and vectors with octave" Tags="|octave|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3205" PostTypeId="2" ParentId="3204" CreationDate="2012-09-04T12:49:10.180" Score="4" Body="&lt;p&gt;The command you are likely looking for is quiver.  It only does 2D vector field plots, and you may have to normalize your vector lengths yourself.  It also requires that all of your points be on a rectangular grid, although there is likely a work around for that.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;quiver(x, y, u, v);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For an animated plot, you want to draw a single location and vector(still using quiver) inside a loop and use the pause command for animation.  Since this sounds a little like a homework problem I will leave the details to you.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-09-04T12:49:10.180" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3206" PostTypeId="1" CreationDate="2012-09-04T14:10:17.633" Score="8" ViewCount="5276" Body="&lt;p&gt;I want to make my python program fast by using cython, but my inner loop is still making slow python calls to the random number generator!  Several years ago this same issue was raised by someone on sage-support and there seemed to be no good solution at that time.  It is not convenient for me to pre-generate a long list of random samples because I am actually sampling from various distributions in a way that is conditional on previous samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a blog post explaining how this was kludged by connecting from cython to gsl:&lt;br&gt;&#xA;&lt;a href=&quot;http://pyinsci.blogspot.com/2010/12/efficient-mcmc-in-python-errata-and.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pyinsci.blogspot.com/2010/12/efficient-mcmc-in-python-errata-and.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And a stackoverflow post by someone trying to implement the gsl kludge:&lt;br&gt;&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/8177446/random-number-generators-to-work-on-x86-64&quot;&gt;https://stackoverflow.com/questions/8177446/random-number-generators-to-work-on-x86-64&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-04-13T22:17:53.900" Title="random number generation from cython" Tags="|python|random-number-generation|sage|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3208" PostTypeId="1" AcceptedAnswerId="3240" CreationDate="2012-09-04T15:55:11.227" Score="8" ViewCount="261" Body="&lt;p&gt;Assuming that a direct numerical simulation is performed, what is a good method for estimate the impact of small scale on large scale in fluid dynamics ? For example is it pertinent to compare two run with different grid size or two run with different viscosity ? Is there some relevant statistical tools for this ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The large scale field can be defined as a coarse-grained field&#xA;\begin{equation}&#xA;\overline{q}_l(t,\mathbf{x})=\int G_l(\mathbf{y}) q(t,\mathbf{y}+\mathbf{x})d\mathbf{y}&#xA;\end{equation}&#xA;where $G_l$ is a normalized convolution kernel of scale $l$. For example the forme of $G_l$ can be $G_l(y)=l^{-3}/\sqrt{2 \pi} \exp(-((y/l)^2/2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The small scales field is defined as&#xA;\begin{equation}&#xA;q'_l=q-\overline{q}_l&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If at some scale $l$ we can remove the small scale of the dynamic, the impact of the small scale on the large scale, will be the difference between the field of the full dynamical system with the field of the truncated dynamical system.&lt;/p&gt;&#xA;" OwnerUserId="1383" LastEditorUserId="1383" LastEditDate="2012-09-04T20:02:10.393" LastActivityDate="2012-09-08T15:20:26.423" Title="How to estimate the impact of small scale on large scale in fluid dynamics?" Tags="|fluid-dynamics|statistics|simulation|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3209" PostTypeId="2" ParentId="3208" CreationDate="2012-09-04T16:00:59.770" Score="3" Body="&lt;p&gt;It is imperative to compare many runs with refined meshes until you detect convergence. A single solution with no mesh refinement study shouldn't give you much confidence in your results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing runs with different fluid properties tells you something different. If you think that a set of runs with different viscosities is relevant to your ultimate science/engineering question, then you should also perform such a study. You should, of course, make sure that each point in this study is mesh-refined as well.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="123" LastEditDate="2012-09-04T18:05:23.400" LastActivityDate="2012-09-04T18:05:23.400" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3210" PostTypeId="1" AcceptedAnswerId="3211" CreationDate="2012-09-04T16:55:36.747" Score="4" ViewCount="129" Body="&lt;p&gt;I am trying to solve a problem of finding a max repetitive sub-tree in an object tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the object tree I mean a tree where each leaf and node has a name. Each leaf has a type and a value of that type associated with that leaf. Each node has a set of leaves / nodes in certain order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given an arbitrary object tree that - we know - has a repetitive sub-tree in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By repetitive I mean 2 or more sub-trees that are similar in everything (names/types/order of sub-elements) but the values of leaves. No nodes/leaves can be shared between sub-trees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem is to identify these sub-trees of the max height.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the exhaustive search can do the trick. I am rather looking for more efficient approach.&lt;/p&gt;&#xA;" OwnerUserId="1985" LastActivityDate="2012-09-04T17:24:40.970" Title="Efficient way to find max height repetitive sub-trees in an object tree" Tags="|algorithms|performance|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3211" PostTypeId="2" ParentId="3210" CreationDate="2012-09-04T17:24:40.970" Score="4" Body="&lt;p&gt;For each non-leaf node in the tree make a hash of its names/order and of the hashes of its ordered sub-nodes.  If a node is a leaf then its hash is its name/type.  This gives an O(N) algorithm to get all the heights and subtree hashes.  Next you can traverse the tree in order of decreasing height until you find a hash that matches one you've seen earlier -- this is your target match.  So the whole algorithm is O(N).&lt;/p&gt;&#xA;" OwnerUserId="1963" LastActivityDate="2012-09-04T17:24:40.970" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3212" PostTypeId="1" AcceptedAnswerId="3213" CreationDate="2012-09-04T18:02:16.757" Score="11" ViewCount="2262" Body="&lt;p&gt;I'm trying to implement the Broyden-Fletcher-Goldfarb-Shanno method to find the minimum of a function.  I need two initial guesses $x_{-1}$ &amp;amp; $x_0$ and an initial Hessian Matrix approximation $B_0$.  The only requirements I find for $B_0$ is that if the Hessian is symmetric positive definite, so too should $B_0$.  Looking at wikipedia, I see that a typical initial approximation is $B_0=I$ (the identity matrix).  Is this always a good initial $B_0$?  Is there any reason why I might want to choose anything other than $I$?  Would other choices of B, satisfying the same matrix properties, greatly affect the convergence of the method?  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-09-07T07:03:56.547" Title="Sensitivity of BFGS to initial Hessian approximations" Tags="|optimization|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3213" PostTypeId="2" ParentId="3212" CreationDate="2012-09-04T18:35:58.807" Score="8" Body="&lt;p&gt;If you have a justified Hessian approximation, it is better to use it rather than the arbitrary $B_0=I$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The rationale is that if you start close to the solution $x^*$, the initial rate of convergence is (for any $r&amp;gt;0$) $r+1$-step linear with a $r+1$-step convergence factor of $q=\|B_0^{-1}f''(x^*)-G\|$ if this is $&amp;lt;1$ for some rank $r$ correction $G$ of the identity matrix. Thus trying to make this small is very valuable. (This is equivalent to preconditioning the system.) The convergence factor improves with time and ultimately approaches zero (superlinear convergence), but in many real problems (especially high-dimensional ones), one never makes enough iterations to reach the superlinear regime. Thus the initial speed is quite important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One important case is when solving nonlinear least squares problems (minimize $\|F(x)\|_2^2$), where the Gauss-Newton approximation $B_0=F'(x_0)^TF'(x_0)$ of the initial Hessian can be computed without the need for second derivatives. Using it makes the BFGS method affine invariant, i.e., invariant under linear transformations of $x$ like Newton's method, which is usually very beneficial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another important case is when you solve a sequence of related problems. Often, restarting the solver with the final Hessian approximation of the previous problem significantly reduces the number of iterations needed.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-07T07:03:56.547" LastActivityDate="2012-09-07T07:03:56.547" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3214" PostTypeId="1" CreationDate="2012-09-04T20:41:12.943" Score="13" ViewCount="4166" Body="&lt;p&gt;nVidia, for example, has CUBLAS, which promises 7-14x speedup. Naively, this is nowhere near the theoretical throughput of any of nVidia's GPU cards. What are the challenges in speeding up linear algebra on GPUs, and are there faster linear algebra routings already available?&lt;/p&gt;&#xA;" OwnerUserId="1450" LastEditorUserId="9" LastEditDate="2012-09-05T12:01:11.187" LastActivityDate="2022-10-01T17:21:21.917" Title="What are the fastest available implementations of BLAS/LAPACK or other linear algebra routines on GPU systems?" Tags="|linear-algebra|lapack|blas|gpu|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3215" PostTypeId="2" ParentId="3214" CreationDate="2012-09-04T20:57:04.527" Score="12" Body="&lt;p&gt;I can't answer the second half of your question as far as other implementations out there but I can provide some insight as to the challenges.  For reference, I personally used ViennaCL on a nVidia GTX 560 Ti with 2GB of memory for my benchmarks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over serial code on a mid-range i5, I saw speed-ups for dense matrix multiplications of approximately 40x.  For operations such as a vector-scalar multiply I saw as high as 1000x speed-ups.  The 800 pound gorilla in the room, however, is memory bandwidth.  For most commercial GPUs, you will be using something like PCIe which limits you to about 6GB/s of throughput.  In my case, while the computation was 40x faster, the three matrix copies (two to the GPU, and one back) each took about as much time as just doing the computation on the CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem then with any general library for GPU linear algebra is going to be that they can't really re-use objects on the GPU, because they don't know what you are going to do with them.  So every call to a compute kernel will likely require copying to the GPU, then copying the result back.  This will eat up a large portion of the gains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can reuse objects such as matrices, then you could write the higher level algorithms to avoid as much memory management as possible, but a library would be hard pressed to do this efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope that this helps, and I am sure there are other people here who are much more experienced in this, but these are the experiences and impressions I got during my short foray into GPU computing.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-09-04T20:57:04.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3216" PostTypeId="1" AcceptedAnswerId="3217" CreationDate="2012-09-05T00:28:55.823" Score="12" ViewCount="288" Body="&lt;p&gt;I wonder: what is the best algorithm to solve &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\frac{du}{dt} = Au&#xA;\end{equation}&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a real &lt;span class=&quot;math-container&quot;&gt;$n\times n$&lt;/span&gt; matrix. A is not explicitly time-dependent, usually sparse but not necessarily banded.  Its eigenvalues have non-positive real parts.  A is also diagonalizable but may be too big for a full diagonalization to be computationally efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is the implicit Trapezoidal rule which I have had a good experience. &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\left(I-\frac{\Delta t}{2} A\right) u_{n+1} = \left(I+\frac{\Delta t}{2} A\right) u_{n}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about explicit methods or Pade approximants? Also, how does this changes if a forcing term is added to the RHS?&lt;/p&gt;&#xA;" OwnerUserId="1751" LastEditorUserId="22314" LastEditDate="2019-11-28T14:09:49.980" LastActivityDate="2019-11-28T14:09:49.980" Title="Algorithms for linear system of ODEs" Tags="|linear-algebra|ode|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3217" PostTypeId="2" ParentId="3216" CreationDate="2012-09-05T08:15:36.737" Score="15" Body="&lt;p&gt;As your matrix is independent of $u$ the result is a matrix exponential times the intial vector. The standard discussion of relevent method can be found from &lt;a href=&quot;http://scholar.google.at&quot; rel=&quot;noreferrer&quot;&gt;http://scholar.google.at&lt;/a&gt; by searching for ''Nineteen dubious ways''.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the scaling-and-squaring algorithm (the least dubious one), see also&#xA;&lt;a href=&quot;http://blogs.mathworks.com/cleve/2012/07/23/a-balancing-act-for-the-matrix-exponential/&quot; rel=&quot;noreferrer&quot;&gt;http://blogs.mathworks.com/cleve/2012/07/23/a-balancing-act-for-the-matrix-exponential/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-18T18:13:08.543" LastActivityDate="2012-09-18T18:13:08.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3218" PostTypeId="2" ParentId="3214" CreationDate="2012-09-05T09:15:44.427" Score="6" Body="&lt;p&gt;Let me focus only on CUDA and BLAS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speedup over an host BLAS implementation is not a good metric to assess throughput, since it depends on too many factors, although I agree that speedup is usually what one cares about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the &lt;a href=&quot;http://developer.nvidia.com/cuda/cublas&quot; rel=&quot;nofollow&quot;&gt;benchmarks published by NVIDIA&lt;/a&gt; and take into account that the Tesla M2090 has 1331 Gigaflops (single precision) and 665 Gigaflops (double prec.) peak performance, you will see that for SGEMM and DGEMM we have a measured throughput nearly at 60% of the theoretical one, which is pretty good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do you define the flops performance? Flop count / elapsed time, where flop count is $2\,mnk$ ($m\times k$ and $k\times n$ are the matrix dimensions), and elapsed time can include or not the transfer time from host to GPU memory and back. (As Godric correctly points out, this make a big difference!) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As what regards sustained floating point throughput, I think that flops should be computed without taking into account data and result transfer times, and this makes speedup comparisons difficult. Furthermore you have to take into account the matrix size, since best performance is for big matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottom line: speedup of a real life application can be very different from peak measured performance on linear algebra routines, since you have to take into account GPU initialization, data transfer times, etc. etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I won't answer your question about the fastest library, since the question makes no sense unless a precise metric and problem is defined. All this said, i think that &lt;a href=&quot;http://developer.nvidia.com/cuda/cublas&quot; rel=&quot;nofollow&quot;&gt;cuBLAS&lt;/a&gt; and &lt;a href=&quot;http://icl.cs.utk.edu/magma/&quot; rel=&quot;nofollow&quot;&gt;MAGMA&lt;/a&gt; are a very good starting point.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-09-05T12:08:10.223" LastActivityDate="2012-09-05T12:08:10.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3219" PostTypeId="2" ParentId="3188" CreationDate="2012-09-05T13:07:07.303" Score="28" Body="&lt;p&gt;A Cholesky factorization makes the most sense for the best stability and speed when you are working with a covariance matrix, since the covariance matrix will be positive semi-definite  symmetric matrix. Cholesky is a natural here. BUT...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IF you intend to compute a Cholesky factorization, before you ever compute the covariance matrix, do yourself a favor. Make the problem maximally stable by computing a QR factorization of your matrix. (A QR is fast too.) That is, if you would compute the covariance matrix as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C = A^{T} A&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ has had the column means removed, then see that when you form $C$, it squares the condition number. So better is to form the QR factors of $A$ rather than explicitly computing a Cholesky factorization of $A^{T}A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A = QR&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Q is orthogonal, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;C &amp;amp;= (QR)^{T} QR \\&#xA;&amp;amp;= R^T Q^T QR \\&#xA;&amp;amp;= R^T I R \\&#xA;&amp;amp;= R^{T} R&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus we get the Cholesky factor directly from the QR factorization, in the form of $R^{T}$. If a $Q$-less QR factorization is available, this is even better since you don't need $Q$. A $Q$-less QR is a fast thing to compute, since $Q$ is never generated. It becomes merely a sequence of Householder transformations. (A column pivoted, $Q$-less QR would logically be even more stable, at the cost of some extra work to choose the pivots.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The great virtue of using the QR here is it is highly numerically stable on nasty problems. Again, this is because we never had to form the covariance matrix directly to compute the Cholesky factor. As soon as you form the product $A^{T}A$, you square the condition number of the matrix. Effectively, you lose information down in the parts of that matrix where you originally had very little information to start with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, as another response points out, you don't even need to compute and store the inverse at all, but use it implicitly in the form of backsolves on triangular systems.&lt;/p&gt;&#xA;" OwnerDisplayName="user840" LastEditorUserId="462" LastEditorDisplayName="user840" LastEditDate="2017-03-29T18:46:13.090" LastActivityDate="2017-03-29T18:46:13.090" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3220" PostTypeId="1" CreationDate="2012-09-05T18:40:43.797" Score="11" ViewCount="14090" Body="&lt;p&gt;I would like to perform a PCA on a dataset composed of approximately 40 000 samples, each sample displaying about 10 000 features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Matlab princomp function consistently takes over half an hour at which point I kill the process.  I would like to find an implementation/algorithm that runs in less than 10 minutes. What would be the fastest algorithm ? How long would it take on a i7 dual core / 4GB Ram ?&lt;/p&gt;&#xA;" OwnerUserId="1990" LastEditorUserId="1552" LastEditDate="2012-09-05T19:09:19.910" LastActivityDate="2013-11-15T14:26:54.887" Title="Fastest PCA algorithm for high-dimensional data" Tags="|high-dimensional|data-analysis|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3221" PostTypeId="2" ParentId="3188" CreationDate="2012-09-05T22:50:21.930" Score="4" Body="&lt;p&gt;I did this for the first time recently, using suggestions from mathSE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SVD was recommended by most I think, but I opted for the simplicity of Cholesky:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If matrix $M = A A^\top$, then I decompose $M$ to a triangular matrix $L$ using Cholesky, such that $M = L L^\top$.  I then use backsubstitution or forwardsubstitution (depending on whether I choose L to be upper or lower triangular), to invert $L$, such that I have $L^{-1}$.  From this, I can quickly calculate $M^{-1} = \left(L L^\top\right)^{-1} = L^{-\top}L^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Start with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M = A A^\top$, where $M$ is known and is implicitly symmetric and is also positive-definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cholesky factorisation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M \rightarrow L L^\top$, where $L$ is square and non-singular&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back-substitution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L \rightarrow L^{-1}$, probably the fastest way to invert $L$ (don't quote me on that though)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiplication:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M^{-1} = \left(L L^\top\right)^{-1} = L^{-\top} L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notation used:&lt;/strong&gt;&#xA;Lower indices are rows, upper indices are columns and $L^{-\top}$ is the transpose of $L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My Cholesky algorithm (probably from Numerical Recipes or Wikipedia)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L_i^j = \frac{M_i^j - M_i \cdot M_j}{M_i^i - M_i \cdot M_i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can almost be done in-place (you only need temporary storage for the diagonal elements, an accumulator and some integer iterators).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My back-substitution algorithm (from Numerical Recipes, check their version as I may have made a mistake with the LaTeX markup)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left(L^{-1}\right)_i^j = \left\{\begin{array}{11}&#xA;1 / {L_i^i} &amp;amp; \mbox{if } i = j\\&#xA;\left(-L_i \cdot \left(L^{-T}\right)_j\right) / L_i^i &amp;amp; \mbox{otherwise}&#xA;\end{array}\right.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As $L^{-T}$ appears in the expression, the order that you iterate over the matrix is important (some parts of the result matrix depend on other parts of it that must be calculated beforehand).  Check the Numerical Recipes code for a complete example in code.&#xA;[Edit]: Actually, just check the Numerical Recipes example.  I've over-simplified too much by using dot-products, to the point that the above equation has a cyclic dependency no matter what order you iterate...&lt;/p&gt;&#xA;" OwnerUserId="1992" LastEditorUserId="1992" LastEditDate="2012-09-05T23:01:49.010" LastActivityDate="2012-09-05T23:01:49.010" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3222" PostTypeId="2" ParentId="3206" CreationDate="2012-09-06T01:37:44.957" Score="7" Body="&lt;p&gt;Cython makes code faster by removing the type ambiguity. Since random.py is a pure python module, you can just copy it and add the types to the functions you need.  Then cython can optimize the dynamic overhead away.  &lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2012-09-06T01:37:44.957" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3223" PostTypeId="2" ParentId="3220" CreationDate="2012-09-06T08:05:37.100" Score="11" Body="&lt;p&gt;First of all, you should specify whether you want all components or the most significant ones?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denote your matrix $A \in \mathbb{R}^{N \times M}$ with $N$ being number of samples and $M$ dimensionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case you want all components the classical way to go is to compute covariance matrix $C \in \mathbb{R}^{M\times M}$ (which has time complexity of $O(NM^2)$) and then apply SVD to it (additional $O(M^3)$). In terms of memory this would take $O(2M^2)$ (covariance matrix + singular vectors and values forming orthogonal basis) or $\approx 1.5$ GB in double precision for your particular $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could apply SVD directly to the matrix $A$ if you normalize each dimension prior to that and take left singular vectors. However, practically I would expect SVD of the matrix $A$ to take longer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need only a fraction of (perhaps most significant) components you may want to apply &lt;a href=&quot;http://arxiv.org/pdf/0811.1081.pdf&quot;&gt;iterative PCA&lt;/a&gt;. As far as I know all these algorithms are closely related to Lanczos process thus you are dependent on the spectrum of the $C$ and practically it will be difficult to achieve accuracy of SVD for obtained vectors and it will degrade with the number of singular vector. &lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2012-09-06T12:47:22.693" LastActivityDate="2012-09-06T12:47:22.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3224" PostTypeId="2" ParentId="3202" CreationDate="2012-09-06T10:56:45.073" Score="3" Body="&lt;p&gt;See these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://donhe.topcities.com/pubs/solving.html&quot; rel=&quot;nofollow&quot;&gt;Solving Quartics and Cubics for Graphics&lt;/a&gt;, originally published in &lt;em&gt;Graphics Gems V&lt;/em&gt;. The original code is &lt;a href=&quot;http://tog.acm.org/resources/GraphicsGems/gems/Roots3And4.c&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. See also &lt;a href=&quot;http://rtiac.googlecode.com/svn-history/r74/trunk/src/herbison-evans/&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;http://fftjet.hepforge.org/svn/trunk/fftjet/quartic_lib.cc&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.ijpam.eu/contents/2011-71-2/7/7.pdf&quot; rel=&quot;nofollow&quot;&gt;A Universal Method Of Solving Quartic Equations&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="640" LastActivityDate="2012-09-06T10:56:45.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3225" PostTypeId="1" AcceptedAnswerId="3226" CreationDate="2012-09-06T11:08:01.097" Score="12" ViewCount="610" Body="&lt;p&gt;Assume I have a real valued function $f(x_1,\ldots ,x_N)$ of some variables $x_i$ which I want to evaluate numerically. In general the formula for $f$ can contain products, rationals, trancendental functions etc. and  will be to long to investigate its numerical stability analytically. Or it will at least be to time consuming to do it in practice. Assume I do not have a shorter equivalent with guaruanteed stability. Is there a methodical procedure to analyse the numerical stability of $f$. I think of comparing it to arbitrary precission results obtained using a computer algebra system. Say the function will be implemented in C using stdlib functions and single or double precision. Which quantities should I compare to quantify the quality of the approximation at finite precission? How do I determine critical values of the variables? How can I choose the compiler and the compiler optimizations so other people can easily reproduce the results? ... I know that the problem setting is probably to generic to give good answers. But I still think that this is a common problem in computer science and wonder if there are references which propose standards to perform such analysis.&lt;/p&gt;&#xA;" OwnerUserId="1943" LastEditorUserId="549" LastEditDate="2012-09-06T15:14:29.117" LastActivityDate="2012-09-11T17:07:03.820" Title="Heuristic check of numerical stability" Tags="|stability|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3226" PostTypeId="2" ParentId="3225" CreationDate="2012-09-06T11:28:27.757" Score="7" Body="&lt;p&gt;What you are looking for is what is called &quot;Automatic error analysis&quot; and is the subject of Chapter 26 of Higham's book &quot;Accuracy and Stability of Numerical Algorithms&quot;, 2nd ed., SIAM Publishers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One technique he describes is using direct search optimization: try to formulate your problem as an optimisation problem and use the optimization algorithm to find coefficients or parameter values that maximize or minimize a quantity related to the accuracy of your algorithm/formula. He uses the example of the growth factor in Gaussian Elimination (what matrix maximizes this growth factor) or the roots of a cubic (as I answered in one of your previous questions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest that you obtain a copy of this book, read the introductory chapters and this chapter 26 and the references therein. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-09-06T11:28:27.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3227" PostTypeId="1" CreationDate="2012-09-06T17:07:06.130" Score="5" ViewCount="10587" Body="&lt;p&gt;I have encountered the following system of differential equations in lagrangian mechanics. Can you suggest a numerical method, with relevant links and references on how can I solve it, and the implementation in C (if possible) Also, is there a shorter implementation on Matlab or Mathematica?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;mx \dot y^2 + mg\cos(y) - Mg - (m+M)\,\ddot x &amp;amp;= 0 &#xA;\\&#xA;g\sin(y) + 2\dot x\dot y + x \,\ddot y &amp;amp;= 0&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\dot x$ or $\dot y$ are time derivatives, and the double dots indicate a 2nd derivative wrt time.&lt;/p&gt;&#xA;" OwnerDisplayName="user1717" LastEditorUserId="1744" LastEditDate="2012-09-06T21:27:32.980" LastActivityDate="2012-09-07T06:20:59.157" Title="Solving two coupled non-linear second order differential equations numerically" Tags="|matlab|mathematica|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3228" PostTypeId="2" ParentId="3227" CreationDate="2012-09-06T17:30:41.247" Score="5" Body="&lt;p&gt;You could use &lt;a href=&quot;http://www.physics.buffalo.edu/phy410-505-2009/topic3/lec-3-2.pdf&quot; rel=&quot;nofollow&quot;&gt;Runge-Kutta method&lt;/a&gt; to solve this system numerically, first rewrite your second order equation as a first order system by doing following substitution trick:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\{&#xA;\begin{aligned}&#xA;x_1' &amp;amp;= x_2&#xA;\\&#xA;y_1' &amp;amp; = y_2&#xA;\\&#xA;x_2' &amp;amp;= x_1 y_2 + g \cos y_1 -(M+m)g/m&#xA;\\&#xA;y_2' &amp;amp;= -g(\sin y_1) /x_1 - 2x_2 y_2/x_1&#xA;\end{aligned}&#xA;\right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you could use &lt;code&gt;MATLAB&lt;/code&gt;'s &lt;a href=&quot;http://www.mathworks.com/help/techdoc/ref/ode23.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;ode45&lt;/code&gt; or &lt;code&gt;ode23&lt;/code&gt;&lt;/a&gt; to solve it, if you wanna implement the method on C, I believe there are many pkgs available there on the internet, like &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/cpp_src/rkf45/rkf45.html&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-09-06T20:45:09.147" LastActivityDate="2012-09-06T20:45:09.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3229" PostTypeId="1" CreationDate="2012-09-06T18:27:27.753" Score="31" ViewCount="29400" Body="&lt;p&gt;I want to modify a dense square transition matrix in-place by changing the order of several of its rows and columns, using python's numpy library.  Mathematically this corresponds to pre-multiplying the matrix by the permutation matrix P and post-multiplying it by P^-1 = P^T, but this is not a computationally reasonable solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I am manually swapping rows and columns, but I would have expected numpy to have a nice function f(M, v) where M has n rows and columns, and v has n entries, so that f(M, v) updates M according to the index permutation v.  Maybe I am just failing at searching the internet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like this might be possible with numpy's &quot;advanced indexing&quot; but my understanding is that such a solution would not be in-place.  Also for some simple situations it may be sufficient to just separately track an index permutation, but this is not convenient in my case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added:&lt;br&gt;&#xA;Sometimes when people talk about permutations, they only mean the sampling of random permutations, for example as part of a procedure to obtain p-values in statistics.  Or they mean counting or enumerating all possible permutations.  I'm not talking about these things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added:&lt;br&gt;&#xA;The matrix is small enough to fit into desktop RAM but big enough that I do not want to copy it thoughtlessly.  Actually I would like to use matrices as large as possible, but I don't want to deal with the inconvenience of not being able to hold them in RAM, and I do O(N^3) LAPACK operations on the matrix which would also limit the practical matrix size.  I currently copy matrices this large unnecessarily, but I would hope this could be easily avoided for permutation.&lt;/p&gt;&#xA;" OwnerUserId="1963" LastEditorUserId="4405" LastEditDate="2016-11-07T16:30:22.193" LastActivityDate="2016-11-07T16:30:22.193" Title="Permute a matrix in-place in numpy" Tags="|linear-algebra|python|numpy|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3230" PostTypeId="2" ParentId="3225" CreationDate="2012-09-06T18:56:55.157" Score="0" Body="&lt;p&gt;What GertVdE describes is the numerical error. This may be what you are looking for, but it's not the same as &lt;em&gt;numerical stability&lt;/em&gt; as indicated in the title of the question. Numerical stability essentially asks whether nearby values of your input variables yield nearby values of the output. In other words, whether a formula like $$|f(x+\varepsilon)-f(x)| \le C |\varepsilon|$$ holds for some moderate constant $C$. For this, you can analyze the derivative of $f$ or, if your function does not vary in its properties wildly over its domain, simply try this for a bunch of $x,\epsilon$ pairs.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-06T18:56:55.157" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3231" PostTypeId="2" ParentId="3227" CreationDate="2012-09-06T19:01:04.763" Score="5" Body="&lt;p&gt;Why implement it by hand? Matlab, Maple and Mathematica all have tools builtin to solve differential equations numerically, and they use far better methods than you could implement yourself in finite time. In Matlab, you want to look at &lt;em&gt;ode45&lt;/em&gt;. In Maple it's called &lt;em&gt;dsolve&lt;/em&gt; (with the 'numeric' option set), in Mathematica it is &lt;em&gt;NDSolve&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-06T19:01:04.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3232" PostTypeId="2" ParentId="3229" CreationDate="2012-09-06T23:19:54.877" Score="12" Body="&lt;p&gt;According to the docs, there is no in-place permutation method in numpy, something like &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.sort.html#numpy.ndarray.sort&quot; rel=&quot;noreferrer&quot;&gt;ndarray.sort&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your options are (assuming that &lt;code&gt;M&lt;/code&gt; is a $N\times N$ matrix and &lt;code&gt;p&lt;/code&gt; the permutation vector)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;implementing your own algorithm in C as an extension module (but &lt;a href=&quot;https://stackoverflow.com/q/2352542/1499402&quot;&gt;in-place algorithms&lt;/a&gt; are hard, at least for me!)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$N$ memory overhead&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(N):&#xA;    M[:,i] = M[p,i]&#xA;for i in range(N):&#xA;    M[i,:] = M[i,p]&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$N^2$ memory overhead&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M[:,:] = M[p,:]&#xA;M[:,:] = M[:,p]&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hope that these suboptimal hacks are useful.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-09-06T23:42:39.497" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3233" PostTypeId="1" AcceptedAnswerId="3243" CreationDate="2012-09-07T00:45:33.237" Score="7" ViewCount="776" Body="&lt;p&gt;I want to compare two floating point numbers for equality relative to a known absolute tolerance. However, this is inside an algorithm I wrote quite some time ago, and I believe the logic of that algorithm would get corrupted if the equality relation is not transitive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some false negatives are no problem, i.e. if two &quot;equal&quot; numbers compare unequal, all that will happen is that the algorithm will use a bit more time and memory. However, I now got input data preprocessed by another algorithm (to smooth out corners), but that algorithm added noise to every single straight line, which leads to memory consumption issues (&gt; 4GB) during my algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see essentially two options, how I could fix the issue:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I could try to remove the &quot;noise&quot; from the results of the preprocessing algorithm.&lt;/li&gt;&#xA;&lt;li&gt;I could try to find a way to do tolerance based equality comparison in a transitive way.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first approach looks easier to me. I would basically have a fixed set of doubles and would need to pick a set of representatives such that every double in the set is within epsilon of a representative. The only idea I have for the second approach is to snap the values to a grid for the comparison. However, I vaguely remember having implemented such a grid snapping approach before, but it broke down as soon as the (c++) compiler started to inline the corresponding code. I fixed this by moving the snapping code to a different translation unit, but later rewrote the code to make the snapping obsolete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible to do tolerance based equality comparisons (in c++) without violating transitivity?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is a good way to implement a &quot;noise removal&quot; algorithm? My approach would probably be to keep a sorted list of representatives, and look up each new double value by bisection in that list, leading to a $O(n \log n)$ runtime and $O(n)$ (additional) memory consumption for the &quot;noise removal&quot; algorithm.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-09-07T22:13:01.283" Title="transitive floating point comparison with (absolute) tolerance" Tags="|c++|floating-point|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3234" PostTypeId="2" ParentId="3233" CreationDate="2012-09-07T01:59:03.553" Score="3" Body="&lt;p&gt;You can't do equality comparison transitively with a tolerance. In the past I've used methods that snap points onto a grid by casting a double to a float and back to double. This is not failsafe, but it often gets rid of some of the more annoying cases.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-07T01:59:03.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3235" PostTypeId="2" ParentId="3227" CreationDate="2012-09-07T06:20:59.157" Score="1" Body="&lt;p&gt;Runge-Kutta methods such as (4,5) are also available in the &lt;a href=&quot;http://www.gnu.org/software/gsl&quot; rel=&quot;nofollow&quot;&gt;GNU Scientific Library&lt;/a&gt; (which is written in C). They also include adaptive time-stepping.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2012-09-07T06:20:59.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3236" PostTypeId="2" ParentId="3225" CreationDate="2012-09-07T07:16:26.997" Score="3" Body="&lt;p&gt;Evaluate your function a few times (3 is typically enough) with all inputs slightly randomly perturbed by $\pm 1$ulp. The standard deviation of the three results will give you a crude (but usually sufficient) measure of numerical sensitivity. You can compare this to the expected sensitivity from linearization, and form the quotient to get a stability estimate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that numerical stability asks for how much worse the actual error at evaluation of a particular $x$ is compared to the error expected from sensitivity analysis when changing the inputs by $\pm1$ ulp; the latter error expressed in ulps defines the problem condition. Condition can be very poor for a stable algorithm (example: $1/x$ near $x=0$) and stability can be poor for a very well-conditioned function (example: $1/(1-x)-1/(1+x)$ near $x=0$).&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-11T17:07:03.820" LastActivityDate="2012-09-11T17:07:03.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3237" PostTypeId="2" ParentId="3220" CreationDate="2012-09-07T07:35:13.217" Score="2" Body="&lt;p&gt;I guess you only need a few (or a few hundred) dominant singular value/vector pairs. Then it is best to use an iterative method, which will be much faster and consume far less memory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab, see&lt;/p&gt;&#xA;&#xA;&lt;p&gt;help svds&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-07T07:35:13.217" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3238" PostTypeId="1" AcceptedAnswerId="3244" CreationDate="2012-09-07T10:15:11.103" Score="4" ViewCount="1376" Body="&lt;p&gt;it is all about valley detection in image processing.&#xA;I would like to find, for a given pixel, direction for higher second order derivative. I am not quite sure what discrete mask/filter I can use to compute directional derivative, along xx, yx, xy, yy. &#xA;What should I understand with 'direction of higher second order derivative'? is it one of the four xx, yx, xy, yy with maximum filter value? &#xA;In the end, I am to found loci of extremal height for the signal, in direction along which second order derivative is of greatest magnitude.&#xA;How to do so? Could you hint at relevant doc? &lt;/p&gt;&#xA;" OwnerUserId="1184" LastActivityDate="2012-09-07T23:18:11.397" Title="Second order directional derivative in image processing" Tags="|optimization|image-processing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3239" PostTypeId="1" AcceptedAnswerId="3257" CreationDate="2012-09-07T10:16:01.360" Score="10" ViewCount="823" Body="&lt;p&gt;Consider the following problem &#xA;$$ W_{uv} = F $$&#xA;where the forcing term can depend on $u,v$ (see &lt;em&gt;Edit 1&lt;/em&gt; below for the formulation), and $W$ and its first derivatives. This is a 1+1 dimensional wave equation. We have initial data prescribed at $\{u+v = 0\}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in the solution inside the domain of dependence of an interval $$\{ u+v = 0, u \in [- u_M,u_M]\}$$&#xA;and am considering the following finite difference scheme.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The goal is to evolve $W_u$ by $W_u(u,v+1) - W_u(u,v) = F(u,v)$ and similarly $W_v(u+1,v) - W_v(u,v) = F(u,v)$. This scheme is integrable in the sense that $$ W(u,v) + W_u(u,v) + W_v(u+1,v) = W(u+1,v+1) = W(u,v) + W_v(u,v) + W_u(u,v+1)$$ so I can consistently compute $W$ from the initial data by integrating upwards; hence I only really need to look at the evolution equations for $W_v$ and $W_u$. &lt;/li&gt;&#xA;&lt;li&gt;For the initial data, we need the compatibility condition $W_u(u,v) - W_v(u+1,v-1) = W(u+1,v-1) - W(u,v)$. Which suggests that I can compute the initial data by using the forward (in $u$) finite difference of $W$ on the initial time with the values of given $W_t$ at half-integer points $(u+0.5,v-0.5)$. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this a well known scheme? In particular, where can I find analysis of this scheme?&lt;/li&gt;&#xA;&lt;li&gt;Any thing obvious I should look out for? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;:&#xA;Pretend I know next to nothing (which is probably true, as I am a pure mathematician trying to learn a little bit of computation machinery). &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit 1&lt;/em&gt;: Just to clarify (to address some comments): the equation in $x$ $t$ coordinates would be &#xA;$$ W_{tt} - W_{xx} = F $$&#xA;and $u$ and $v$ are ¨null coordinates¨ given by (up to some renormalising factors of 2) $u = t+x$ and $v = t-x$. So the initial data at $\{u+v = 0\}$ is in fact at $\{t = 0\}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So instead of a mesh adapted to $(t,x)$ I consider a mesh adapted to $(u,v)$ which is ¨rotated 45 degrees¨. Compared to the $(t,x)$ where $t,x$ take integer values, one can think of the $u,v$ mesh as having additional points where &lt;strong&gt;both&lt;/strong&gt; (but not just one of) $t$ and $x$ take half-integer values. &lt;/p&gt;&#xA;" OwnerUserId="1481" LastEditorUserId="1481" LastEditDate="2012-09-10T08:45:22.560" LastActivityDate="2012-09-14T15:26:44.337" Title="Finite difference scheme for &quot;wave equation&quot;, method of characteristics" Tags="|finite-difference|reference-request|hyperbolic-pde|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3240" PostTypeId="2" ParentId="3208" CreationDate="2012-09-07T19:24:33.007" Score="4" Body="&lt;p&gt;There are &lt;em&gt;many&lt;/em&gt; reasons that a coarser simulation would give different results that a finer grained simulation.  A few examples: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;are boundary layers being resolved differently?&lt;/li&gt;&#xA;&lt;li&gt;am I resolving new features (vortices / blockages to flow)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thinking about a grid result as purely a convolution of a fine-grid result with a Gaussian will work very well in viscous dominated flows (where there is already a large-scale smoothness imposed), but may be remarkably wrong where that assumption breaks down (higher Reynolds number)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can pose a situation with a known symbolic solution, simulate that at several different scales.  If your algorithm/implementation is good, there should be a (roughly linear) convergence in log(error) vs. log(grid size), the slope of which is the &quot;order&quot; of your accuracy.  There are some examples of this in &lt;a href=&quot;http://arizona.openrepository.com/arizona/bitstream/10150/193422/1/azu_etd_11053_sip1_m.pdf&quot; rel=&quot;nofollow&quot;&gt;my thesis&lt;/a&gt;, and I suggest a bunch more reading if you are so inclined.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-09-07T19:24:33.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3241" PostTypeId="2" ParentId="2829" CreationDate="2012-09-07T20:30:21.823" Score="2" Body="&lt;p&gt;As is often the case in simulation papers, the mathematical description of the reported quantities are not literally describing the algorithm used to compute these quantities. (This typically happens when the main author and the compute monkey are not the same person.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, there is no point to first compute $n_\alpha(x',y',z)$ and then integrate out $x$ and $y$. As you suggest in the question, one may estimate $\bar{n}_\alpha(z)$ directly by only constructing bins along the Z-axis. Just compute (the time-average of) the number of particles in a bin and divide it by the volume of the bin. That will give you an estimate of $\bar{n}_\alpha(z)$ in the bin.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2012-09-07T20:30:21.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3242" PostTypeId="1" AcceptedAnswerId="24521" CreationDate="2012-09-07T21:49:17.450" Score="3" ViewCount="200" Body="&lt;p&gt;I have a decision tree model -- that I've trained with some data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just got a new data set in and I want to run it through the model. Is there some way I can measure a &quot;confidence interval&quot; for the classes that new observations are being sorted into?&lt;/p&gt;&#xA;" OwnerUserId="1998" LastActivityDate="2016-07-23T02:10:54.060" Title="How do I measure the confidence of each new observation according to a decision tree learning model?" Tags="|optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3243" PostTypeId="2" ParentId="3233" CreationDate="2012-09-07T22:13:01.283" Score="4" Body="&lt;p&gt;If you have all the numbers up front, you can efficiently compute the transitive closure of tolerance-based comparison with a &lt;a href=&quot;http://en.wikipedia.org/wiki/Union_find&quot; rel=&quot;nofollow&quot;&gt;union-find structure&lt;/a&gt;.  First loop through all pairs of nearby points (e.g., using a bounding box hierarchy), and for each pair within your tolerance mark them as merged in the union-find.  Later on, compare points for equality by checking if they're in the same union-find component.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-09-07T22:13:01.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3244" PostTypeId="2" ParentId="3238" CreationDate="2012-09-07T23:18:11.397" Score="4" Body="&lt;p&gt;You are looking for the directions of the eigenvectors of the Hessian matrix (i.e., the matrix of second derivatives) that correspond to the largest and smallest eigenvalues. At a local minimum, any function can be developed into a Taylor series&#xA;$$&#xA;  f(x) \approx f(x_0) + \frac 12 (x-x_0)^T H (x-x_0) = m_2(x)&#xA;$$&#xA;where $H=\nabla^2 f(x_0)$ and where $m_2(x)$ is this local approximation. The directions where $m_2$ curves up the quickest are the directions that correspond to the eigenvectors of the largest eigenvalue of $H$. The directions where $m_2$ grows the slowest (the &quot;valley floor&quot;) correspond to the direction of the eigenvector with the smallest eigenvalue of $H$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-07T23:18:11.397" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3245" PostTypeId="1" AcceptedAnswerId="3256" CreationDate="2012-09-08T04:11:41.130" Score="7" ViewCount="2004" Body="&lt;p&gt;Suppose I have a high-dimensional vector space $X$, a subspace $V \subset X$, and a collection of $n$ vectors $\{x_i\}_{i=1}^n \subset X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: How can I choose a small collection $k &amp;lt; n$ of the vectors $x_i$ so that the span of this smaller collection &quot;well-approximates&quot; the subspace $V$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notion of &quot;well-approximation&quot; is intentionally left vague  since, although it's intuitive that some subspaces approximate each other better than others, it's not clear to me the best way to introduce definitions that make this precise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For concreteness, in my scenario the sizes of the various objects are of the following orders $dim(X)\approx 10000$, $dim(V)\approx 20$, $n\approx 5000$, and $k$ can be varied but has a target of $k \approx 100$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like this should be well studied, but I'm having trouble finding the right terms to search for. In particular, the subject of &quot;subspace approximation&quot; appears to deal with the opposite problem of choosing a subspace to approximate vectors, and the topic of &quot;basis selection&quot; appear to be interested with choosing linear combinations of basis vectors that make certain things sparse - both very different problems from this (as far as I can tell).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; &lt;em&gt;some clarifications based on discussion below&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The dimension of the space $X$ is larger than the number of candidate basis vectors $x_i$, and the subspace $V$ does not necessairily lie in the span of the $x_i$'s.&lt;/li&gt;&#xA;&lt;li&gt;As an illustrative example of where it might be useful to consider more basis vectors than the dimension of the space being approximated, consider the following situation: $X=\mathbb{R}^4$, $V=span((1,0,0,0))$, $x_1=(1,1,\epsilon,0)$, $x_2=(1,-1,\epsilon,0)$, $x_3=(0,0,0,1)$. It would be useful to choose 2 vectors $x_1$ and $x_2$, even though the space to be approximated, $V$, has dimension 1. &lt;/li&gt;&#xA;&lt;li&gt;Or in 3D, consider the situation in the following picture. You can approximate the space 1D $V$ perfectly with 3 vectors $x_1,x_2,x_3$, very well with 2 vectors $x_1,x_2$, and poorly with only one vector.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/ZlFBG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/li&gt;&#xA;&lt;li&gt;One possible measure of how well a candidate space $\tilde V$ approximates the target space $V$ would be the expected value of the size of the projection of a random unit vector in $V$ onto $\tilde V$. Ie, for a uniformly distributed random unit vector $v \in V$, maximize $\mathbb{E}||\Pi_{\tilde V} v||$. If the approximation is exact this will be 1, otherwise it will be less than 1. Other definitions of &quot;well approximation&quot; may be better, this is just the first thing I thought of.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2012-09-09T18:53:55.733" LastActivityDate="2022-05-18T09:20:05.677" Title="Choosing subset of vectors to approximate a subspace" Tags="|linear-algebra|optimization|approximation|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3246" PostTypeId="2" ParentId="3245" CreationDate="2012-09-08T12:07:05.033" Score="2" Body="&lt;p&gt;I would generate vectors $y_i = \Pi_V x_i, i=1\ldots n$ where $\Pi_V$ is the projector into the subspace $V$. Not all of these $y_i$ will be mutually linearly independent, but you can of course apply the Gram-Schmidt orthogonalization procedure to the $y_i$ which will produce a set of mutually orthogonal vectors $z_i,i=1\ldots n'$ where $n'&amp;lt;\textrm{min}\{n,\textrm{dim}(V)\}$. These vectors $z_i$ are then the best basis for $V$ you can find from the subspace spanned by the $x_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you don't want to apply the Gram-Schmidt procedure, you can consider pairs of vectors $x_i,x_j$ and eliminate one of them if the angle $\theta_{ij}$ they form in $V$, with $\cos \theta_{ij} = \frac{(\Pi_V x_i) \cdot (\Pi_V x_j)}{\|\Pi_V x_i\| \|\Pi_V x_j\|}$, is too small. You also need to eliminate vectors that are mostly orthogonal to $V$, i.e. for which $\|\Pi_V x_i\|$ is below a certain threshold.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-08T12:07:05.033" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3247" PostTypeId="1" AcceptedAnswerId="3248" CreationDate="2012-09-08T13:02:20.517" Score="9" ViewCount="1175" Body="&lt;p&gt;Consider the general FD &lt;strong&gt;implicit&lt;/strong&gt; time stepping scheme&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{x_{t+1} - x_t}{\Delta t}  =   f(x_{t+1})$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x$ is the vector variable of interest and $f$ is some function, generally non-linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can advance from $x_{t}$ to $x_{t+1}$ using Newton's method to solve the above non-linear equation.&#xA;However, this can get quite costly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If instead we write $f(x_{t+1}) \approx f(x_t) + J_f (x_t)(x_{t+1}-x_t) $, where $J_f$ denotes the Jacobian of $f$, we obtain the time-stepping scheme&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{t+1} - x_t  = \left(I - \Delta t \, J_f(x_t) \right)^{-1} \, \Delta t f(x_t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody tell me anything about this very general scheme, any references? What's its name? Stability properties?&lt;/p&gt;&#xA;" OwnerUserId="1740" LastActivityDate="2012-09-08T14:44:35.867" Title="Linearized implicit time stepping" Tags="|finite-difference|stability|implicit-methods|time-integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3248" PostTypeId="2" ParentId="3247" CreationDate="2012-09-08T14:44:35.867" Score="7" Body="&lt;p&gt;This particular example is often called &lt;em&gt;linearly implicit Euler&lt;/em&gt;. Its linear stability is identical to nonlinearly implicit Euler, but the nonlinear stability can be a limiting factor, especially for larger time steps. You can find some discussion for reaction-diffusion systems in Ropp, Shadid, and Ober 2004.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More formally, this is the simplest example of a Rosenbrock method. There is a good chapter in Hairer and Wanner's &lt;em&gt;Solving Ordinary Differential Equations II: Stiff and Differential Algebraic Problems&lt;/em&gt;, including work-precision diagrams for a suite of benchmark problems. For most problems, Rosenbrock methods are very efficient for moderate tolerances. Fully implicit methods can typically take slightly larger time steps for strongly nonlinear problems and extremely high order approaches are good when very tight tolerances are needed. Note that there are also Rosenbrock-W methods which only require an approximation of the Jacobian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For software, you can find the RODAS Fortran 77 implementation from Hairer and Wanner. I also implemented this family of methods in PETSc, which may be appropriate if you have a use for parallel linear algebra or sparse direct solvers. To experiment, you might start by looking at this &lt;a href=&quot;http://code.google.com/p/petsc4py/source/browse/demo/ode/orego.py&quot; rel=&quot;noreferrer&quot;&gt;ODE example written in Python&lt;/a&gt; which produces &lt;a href=&quot;https://i.stack.imgur.com/q1Wzx.png&quot; rel=&quot;noreferrer&quot;&gt;this figure&lt;/a&gt;. You can compare different methods using run-time options, e.g. &lt;code&gt;-ts_type sundials&lt;/code&gt; will produce &lt;a href=&quot;https://i.stack.imgur.com/wKbTd.png&quot; rel=&quot;noreferrer&quot;&gt;this figure&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-09-08T14:44:35.867" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3249" PostTypeId="2" ParentId="3245" CreationDate="2012-09-08T14:59:47.457" Score="3" Body="&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The following greedy algorithm should do what you want reasonably fast.&#xA;$\DeclareMathOperator{\argmax}{arg\,max} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume the original vectors are normalized. Compute&#xA;$$ y_1 = \argmax_{x_i} \lVert \Pi_V x_i \rVert . $$&#xA;Now, given $Y_j = [y_1|\dotsb| y_j]$, let&#xA;$$y_{j+1} = \argmax_{x_i \notin Y_j} \lVert (1 - \Pi_{Y_j}) \Pi_V x_i \rVert . $$&#xA;At each step, this algorithm selects the $y_j$ that &quot;fills&quot; the most of the remaining part of $V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that even though $\hat X = [x_1| x_2| \dotsb | x_n]$ may be full rank (implying that all $x_i$ are linearly independent, $\Pi_V \hat X$ has rank at most $\dim(V)$. This means that if the above algorithm selects $\dim(V)$ vectors, then $V$ has been spanned. Given that you don't care about basis conditioning, there is no use in considering more than $\dim(V)$ vectors.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Original&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;How is the subspace $V$ defined? In terms of data consisting of vectors that live in $V$? You state that $\dim V \approx 20$, but you are willing to use $k \approx 100$ vectors to &quot;approximately span&quot; it? Are you really insisting on using a subset of the vectors $x_i$ or would linear combinations do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have you looked at &lt;a href=&quot;http://en.wikipedia.org/wiki/Principle_component_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;principle component analysis&lt;/a&gt;? Computationally, this will involve computing a partial SVD of the data matrix (the vectors mentioned above). For your problem size, the SVD can be computed directly, but if the size grows much, you can approximate it using iterative methods (e.g. in SLEPc) or the randomized methods discussed in &lt;a href=&quot;https://scicomp.stackexchange.com/a/2813/119&quot;&gt;this question&lt;/a&gt;. (PCA is not quite what you asked for, but there was enough ambiguity in the question that it may be what you want.)&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-09-08T23:25:34.050" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3250" PostTypeId="2" ParentId="3208" CreationDate="2012-09-08T15:20:26.423" Score="1" Body="&lt;p&gt;You are essentially describing a large eddy simulation (LES). The key word in LES is &quot;large&quot;. You must &lt;em&gt;resolve&lt;/em&gt; the &quot;large&quot; eddies for LES to deliver meaningful results. &quot;Large&quot; in this context refers to the energy-containing scales so you must resolve into the inertial subrange for LES to be valid. For many industrial flows, the Reynolds number is much too high to resolve into the inertial subrange, in which case we must resort to turbulence modeling (RANS; examples include $k-\epsilon$, $k-\omega$, Reynolds stress models, etc). These RANS models explicitly model the effect of the unresolved scales on the larger scales.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under-resolved models that do not explicitly model these subgrid scales deliver vastly inferior results. Although some people do this anyway, mostly to avoid the complexity of an actual turbulence model, the results have almost no predictive value. You will rarely see this degree of laziness in fields with rigorous validation standards.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-09-08T15:20:26.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3253" PostTypeId="1" AcceptedAnswerId="3255" CreationDate="2012-09-09T02:34:00.403" Score="10" ViewCount="372" Body="&lt;p&gt;I'm trying to optimize an MPI application with a highly asynchronous communication pattern.  Each rank has a list of things to compute, and sends messages as necessary if the inputs or outputs reside on a different rank.  In addition, each rank is threaded (currently with one communication thread and 5 workers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've instrumented the code with timers around the different performance critical portions of code, which gives me a list of (start,end,type) triples for each thread.  Plotted in the obvious way, with time as the horizontal axis, rank and thread as the vertical, and color indicating what each thread is currently doing, I get an image like this for 16 ranks with 6 threads/rank:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/3lRO5.jpg&quot; alt=&quot;Pentago rank and thread history&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what are other ways of visualizing this data that might help pin down performance issues?  Does anyone have a favorite type of plot they use when profiling asynchronous applications?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This data set is limited in that it doesn't know the dataflow structure, but I'd like to get as much insight out of it as possible before trying to collect something more complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The uncompressed image is &lt;a href=&quot;http://naml.us/random/pentago-mpi-history.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; in case anyone wants to look around (it failed to upload through the normal route).  Unfortunately, Firefox doesn't accept it even though I believe it's valid, possibly because it's simply too large.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2012-09-10T06:01:57.967" LastActivityDate="2012-09-12T12:59:41.597" Title="Ways of visualizing event data in search of performance issues" Tags="|mpi|visualization|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3255" PostTypeId="2" ParentId="3253" CreationDate="2012-09-09T08:44:27.603" Score="4" Body="&lt;p&gt;I spend a lot of time writing and debugging parallel code, both with shared and/or distributed memory, but without knowing your specific problem, I can only tell you what works best for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing which routines take how much time is an important thing if you're looking at computational efficiency, but if you're worried about parallel efficiency, then you should be more worried about what your code is doing when it's &lt;em&gt;not&lt;/em&gt; doing any computation. Kind of like worrying what the kids are doing when it's too quiet...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you're using a hybrid shared/distributed memory approach, I guess your code is, in the blanks, either waiting on an MPI call or on a mutex/condition variable. You can wrap these calls in timers too, and that will give you a better picture of what is slowing you down, e.g. if it's always the same conditional or always the same &lt;code&gt;MPI_REDUCE&lt;/code&gt; that your threads get stuck on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One piece of software I use quite often is the &lt;a href=&quot;http://software.intel.com/en-us/intel-vtune-amplifier-xe&quot; rel=&quot;nofollow&quot;&gt;Intel Vtune Amplifier XE&lt;/a&gt;. It has a nice plotting feature/option that visualizes thread concurrency. The program will draw a plot very similar to yours, yet when a thread waits on a mutex or condition variable, it draws a diagonal line from the waiting thread, at the time it started waiting, to the thread that actually released the mutex or signalled the condition it was waiting for, at the time that it was released/signalled. This can be quite messy, but it makes bottlenecks appear immediately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I also collect bulk statistics, e.g. for each mutex/signal/MPI-call, what were the average and maximum wait times? What is the histogram of the collected wait times? While the plot gives you a nice overview, it can get quite messy when it comes down to the fine details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, one question that should not be underestimated: How are you collecting your timings? Is your timer non-intrusive enough to not influence your code? I use the CPU instruction count whenever possible, i.e. &lt;a href=&quot;http://en.wikipedia.org/wiki/Time_Stamp_Counter&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;RDTSC&lt;/code&gt;&lt;/a&gt; on x86 architectures. This usually just adds a single instruction to your code.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-09-09T08:44:27.603" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3256" PostTypeId="2" ParentId="3245" CreationDate="2012-09-09T13:04:34.807" Score="2" Body="&lt;p&gt;Form the matrix &lt;span class=&quot;math-container&quot;&gt;$X:=[x_1 ... x_N]$&lt;/span&gt; and the matrix &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; whose columns define the subspace of interest. Thus I use &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; to denote matrices rather than their column spaces.&lt;/p&gt;&#xA;&lt;p&gt;An appropriate goal seems to be the solution of the problem &lt;span class=&quot;math-container&quot;&gt;$XU \approx V$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; being of reasonable size and having at most &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; nonzero rows. Without the restriction on the number of rows, the problem is both under- and overdetermined, and can be represented as the least squares problem &lt;BR&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$~~~~~~~~~~$&lt;/span&gt; (1) &lt;span class=&quot;math-container&quot;&gt;$~~~~$&lt;/span&gt; minimize &lt;span class=&quot;math-container&quot;&gt;$\|XU-V\|_F^2$&lt;/span&gt; in the Frobenius norm, subject to a suitable regularization.&lt;BR&gt;&#xA;In the present case, the restriction on the number of rows serves to regularize the problem. This problem can be solved as follows.&lt;P&gt;&lt;BR&gt;&lt;/p&gt;&#xA;&lt;p&gt;Compute an orthogonal factorization &lt;span class=&quot;math-container&quot;&gt;$V=:Q_0\pmatrix{R_0\\0}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$Q_0$&lt;/span&gt; a product of reflections, and conformally partition &lt;span class=&quot;math-container&quot;&gt;$Y:=Q_0^TX=:\pmatrix{Y_1\\Y_2}$&lt;/span&gt;. Then the normal equations reduce to solving &lt;BR&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$~~~~~~~~~~$&lt;/span&gt; (2) &lt;span class=&quot;math-container&quot;&gt;$~~~~$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$Y U = \pmatrix{R_0\\0}~~~~~~~~$&lt;/span&gt;.&lt;BR&gt;&#xA;The permutation &lt;span class=&quot;math-container&quot;&gt;$P_1$&lt;/span&gt; of an orthogonal factorization &lt;span class=&quot;math-container&quot;&gt;$Y_1=Q_1R_1P_1$&lt;/span&gt; with column pivoting defines a set of &lt;span class=&quot;math-container&quot;&gt;$dim~ V$&lt;/span&gt; indices of columns of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; needed so that the projected vectors form a good basis of &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Permute &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; accordingly and denote the result again by &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. Conformally partition &lt;span class=&quot;math-container&quot;&gt;$Y=\pmatrix{Y_{11}&amp;amp; Y_{12}\\Y_{21}&amp;amp; Y_{22}}$&lt;/span&gt;. By construction, &lt;span class=&quot;math-container&quot;&gt;$Y_{11}$&lt;/span&gt; is nonsingular (and usually well-conditioned).&lt;/p&gt;&#xA;&lt;p&gt;Write conformally &lt;span class=&quot;math-container&quot;&gt;$U = \pmatrix{U_1\\U_2}$&lt;/span&gt;, solve the first block of the normal equations (2) for &lt;span class=&quot;math-container&quot;&gt;$U_1$&lt;/span&gt;, and substitute it into the second block of equations.&#xA;This gives a smaller least squares problem of the form (1). Thus we can proceed recursively until the desired number of vectors has been determined.&lt;P&gt;&lt;BR&gt;&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, one can solve the optimization problem&lt;BR&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$~~~~~~~~~~$&lt;/span&gt; (3) &lt;span class=&quot;math-container&quot;&gt;$~~~~$&lt;/span&gt; minimize &lt;span class=&quot;math-container&quot;&gt;$\|XU-V\|_F^2+\lambda\sum_i\|U_{i:}\|_2$&lt;/span&gt;.&lt;BR&gt;&#xA;with a suitable regularization parameter &lt;span class=&quot;math-container&quot;&gt;$\lambda&amp;gt;0$&lt;/span&gt;. The lack of the square in the penalty terms tends to force unneeded rows of &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; to zero. One starts with a fairly large value of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; (forcing few nonzero rows) and uses the result as a starting point for a problem with smaller &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; (if &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; was too sparse) or with larger &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; (if &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; was not sparse enough).&lt;/p&gt;&#xA;&lt;p&gt;Problem (3) is nonsmooth but convex, and there are a number of recent efficient (''first order convex'') methods for handling these at the scale given in the original problem.&lt;/p&gt;&#xA;&lt;p&gt;Edit: The modern era of efficient convex first order methods started with Nesterov's paper ''Smooth minimization of nonsmooth functions'' &lt;a href=&quot;https://doi.org/10.1007/s10107-004-0552-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt; . Some of the later work done in this area is dedicated to group regularization (as problem (3) is called). See &lt;a href=&quot;http://scholar.google.at/scholar?q=group&amp;amp;btnG=&amp;amp;hl=en&amp;amp;as_sdt=2005&amp;amp;sciodt=0%2C5&amp;amp;cites=14895856039931556754&amp;amp;scipsc=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://scholar.google.at/scholar?q=group&amp;amp;btnG=&amp;amp;hl=en&amp;amp;as_sdt=2005&amp;amp;sciodt=0%2C5&amp;amp;cites=14895856039931556754&amp;amp;scipsc=1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="22284" LastEditDate="2022-05-18T09:20:05.677" LastActivityDate="2022-05-18T09:20:05.677" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="3257" PostTypeId="2" ParentId="3239" CreationDate="2012-09-10T10:47:48.067" Score="6" Body="&lt;p&gt;There is definitely literature on schemes like this.  Two keywords are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Modified method of characteristics&lt;/li&gt;&#xA;&lt;li&gt;Semi-Lagrangian schemes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;After 20 minutes of googling:  some possibly important papers are &lt;a href=&quot;http://dx.doi.org/10.1137/0719063&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1137/0719063&lt;/a&gt; and &lt;a href=&quot;http://dx.doi.org/10.1137/0728024&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1137/0728024&lt;/a&gt; (search forward from there).  Those probably aren't the &lt;em&gt;best&lt;/em&gt; references out there, but they should be a starting point to get you into the right literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think of this as a rotated method of lines with dimensional splitting.  Presumably you're very well aware of the equivalence of your equation and the usual form of the wave equation&#xA;$$W_{tt}-W_{xx} = F$$&#xA;under the transformation &#xA;$$u = t+x, \ \ \ \ v = t-x.$$&#xA;For me it is useful to think of your scheme in terms of this traditional form of the wave equation.  What the scheme does is integrate first along one set of characteristics, then along the other.  The integration is done using &lt;strong&gt;dimensional splitting&lt;/strong&gt; and &lt;strong&gt;Euler's method&lt;/strong&gt;, both of which are first order accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, since you're integrating along characteristics, your scheme would be exact in the case $F=0$.  That is, the numerical errors in your scheme will be due only to numerical integration of $F$ (this may be obvious, but is perhaps useful to point out to those who are accustomed to more traditional numerical methods).  Furthermore, your scheme is unconditionally stable for the case $F=0$.  Nothing more can be said about its stability without knowing some properties of $F$.  In general, the scheme will be stable only under some finite step size restriction (since Euler's method is explicit).  If the Jacobian of $F$ has any purely imaginary eigenvalues, the scheme will be unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general discretization approach of reducing a PDE to a system of ODEs (as in your method) is known as the method of lines.  As with any method of lines discretization, you could increase the order of accuracy by using a higher-order ODE solver and you could improve the stability by using an appropriate implicit ODE solver (with the attendant increase in computational cost per step).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-09-10T19:43:50.980" LastActivityDate="2012-09-10T19:43:50.980" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3258" PostTypeId="2" ParentId="3253" CreationDate="2012-09-10T15:20:19.470" Score="5" Body="&lt;p&gt;Sometimes you can get an alternative view on performance problems via a high-level resource analysis: Is there a relevant bottleneck such as memory bandwidth? Does every worker thread do the same amount of work? This data might be collected easily with likwid-perfctr from the LIKWID tool suite &lt;a href=&quot;http://code.google.com/p/likwid&quot;&gt;LIKWID Google code project&lt;/a&gt;. If the profile is such that many different hot spots exist, you may be required to tackle them one by one. There may also be different issues, depending on how many threads/processes are used.&lt;/p&gt;&#xA;" OwnerUserId="2006" LastActivityDate="2012-09-10T15:20:19.470" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3259" PostTypeId="1" CreationDate="2012-09-10T16:01:17.613" Score="8" ViewCount="1765" Body="&lt;p&gt;I need to solve system of two coupled partial differential equations numerically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial x_1}{\partial t} &amp;amp;= c_1\nabla ^2 x_1 + f_1(x_1,x_2)\\ &#xA;\frac{\partial x_2}{\partial t} &amp;amp;= c_2\nabla ^2 x_2 + K\frac{\partial x_1}{\partial t}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The domain of system is a square region. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Boundary condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;x &amp;amp;= \text{constant} \implies \frac{\partial x_1}{\partial x} = \frac{\partial x_2}{\partial x} = 0\\&#xA;y &amp;amp;= \text{constant} \implies \frac{\partial x_1}{\partial y} = \frac{\partial x_2}{\partial y} = 0&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to solve this system with Fourier transform. Solution becomes unstable after few iterations. I have solved this system earlier with finite difference scheme and it worked well so I know that constants of system are perfectly fine.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My question is can Fourier transform be used to solve these equations? &lt;/li&gt;&#xA;&lt;li&gt;I read somewhere that it because of Neumann boundary condition one cannot apply Fourier transform.  Is this correct? &lt;/li&gt;&#xA;&lt;li&gt;If yes what is alternative?(I have read that cosine transform should be used but want to confirm). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2007" LastEditorUserId="10134" LastEditDate="2016-08-28T16:46:29.983" LastActivityDate="2016-08-28T16:46:29.983" Title="Fourier transform for Neumann boundary condition" Tags="|pde|fourier-analysis|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3260" PostTypeId="1" AcceptedAnswerId="3265" CreationDate="2012-09-10T19:27:45.937" Score="3" ViewCount="266" Body="&lt;p&gt;I have constructed a local and global distributed array in Petsc 3.2 using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DMCreateGlobalVector(da, &amp;amp;v);&#xA;DMCreateLocalVector(da, &amp;amp;lv);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In order to retrieve the ghost cell values from neighbouring processes, I simply use the command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DMGlobalToLocalBegin(da, v, INSERT_VALUES, lv);&#xA;DMGlobalToLocalEnd(da, v, INSERT_VALUES, lv);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From what I understand, the local and global vectors use separate memory spaces. For any processor, the local portion of the global vector (Vec v) is duplicated in another memory space allocated to the local vector (Vec lv). I am attempting to construct a local/global vector that use the same memory space. I would then need to use some sort of scatter routine to only fill the appropriate ghost cells. I notice that there is a &lt;strong&gt;VecCreateGhost&lt;/strong&gt; function that seems to provide this functionality. However, it does not seem to be associated with the DMDA functions (e.g. DMCreateGhostVector). &lt;/p&gt;&#xA;" OwnerUserId="1031" LastEditorUserId="1031" LastEditDate="2012-09-10T19:34:01.667" LastActivityDate="2012-09-11T13:27:07.393" Title="Using same memory space for global and local Petsc DA Vectors" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3261" PostTypeId="1" AcceptedAnswerId="3295" CreationDate="2012-09-10T21:15:15.523" Score="14" ViewCount="1144" Body="&lt;p&gt;$\newcommand{\v}[1]{\boldsymbol{#1}}$&#xA;Suppose we have following Stokes flow model equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tag{1}&#xA;\left\{&#xA;\begin{aligned}&#xA;-\mathrm{div}(\nu \nabla \v{u}) + \nabla p &amp;amp;= \v{f}&#xA;\\&#xA;\mathrm{div} \v{u} &amp;amp;= 0&#xA;\end{aligned}&#xA;\right.$$&#xA;where the viscosity $\nu(x)$ is a function, for the standard mixed finite element, say we use the stable pair: Crouzeix-Raviart space $\v{V}_h$ for the velocity $\v{u}$ and element-wise constant space $S_h$ for the pressure $p$, we have the following variational form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{L}([\v{u},p],[\v{v},q]) = \int_{\Omega} \nu \nabla\v{u}:\nabla \v{v}&#xA;-\int_{\Omega} q\mathrm{div} \v{u}  -\int_{\Omega} p\mathrm{div} \v{v} &#xA;=\int_{\Omega} \v{f}\cdot \v{v} \quad \forall \v{v}\times q\in \v{V}_h\times S_h&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we know that since the Lagrange multiplier $p$ can be determined up to a constant, the finally assembled matrix should have &lt;em&gt;nullspace $1$&lt;/em&gt;, to circumvent this we could enforce the pressure $p$ on some certain element be zero, so that we don't have to solve a singular system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here is my question 1:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(Q1) Is there other way than enforcing $p=0$ on some element to eliminate the kernel for standard mixed finite element? or say, any solver out there that be able to solve the singular system to get a compatible solution?(or some references are welcome)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And about the compatibility, for (1) it should be&#xA;$$&#xA;\int_{\Omega} \nu^{-1} p = 0&#xA;$$&#xA;and the nice little trick is to compute $\tilde{p}$ be the $p$ we got from the solution of the linear system subtracted by its weighted average:&#xA;$$&#xA;\tag{2}&#xA;\tilde{p} = p - \frac{\nu}{|\Omega|}\int_{\Omega} \nu^{-1} p&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, recently I have just implemented &lt;a href=&quot;http://www.cs.sandia.gov/~pbboche/papers_pdf/2006SINUM_PPS.pdf&quot; rel=&quot;noreferrer&quot;&gt;a stabilized $P_1-P_0$ mixed finite element for Stokes equation by Bochev, Dohrmann,and Gunzberger&lt;/a&gt;, in which they added a stabilized term to the variational formulation (1):&#xA;$$&#xA;\widetilde{\mathcal{L}}([\v{u},p],[\v{v},q]) =\mathcal{L}([\v{u},p],[\v{v},q])&#xA;-\int_{\Omega} (p - \Pi_1 p)(q -\Pi_1 q)&#xA;=\int_{\Omega} \v{f}\cdot \v{v} \quad \forall \v{v}\times q\in \v{V}_h\times S_h&#xA;$$&#xA;where $\Pi_1$ is the projection from piecewise constant space $P_0$ to continuous piecewise $P_1$, and the constant kernel of the original mixed finite element is gone, however, weird things happened, (2) doesn't work anymore, I coined the test problem from &lt;a href=&quot;http://www.springerlink.com/content/j7bqfnh1eccpcuwx/&quot; rel=&quot;noreferrer&quot;&gt;an interface problem for diffusion equation&lt;/a&gt;, this is what I got for pressure $p$, the right one is the true solution and the left one is the numerical approximation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wSzFO.png&quot; alt=&quot;Stokes Test 1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;however if $\nu$ is a constant, the test problem performs just fine:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/xmRDD.png&quot; alt=&quot;Stokes Test 2&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am guessing it is because the way I am imposing the compatibility condition, since it is linked with the inf-sup stability of the whole system, here is my second question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(Q2): is there any way other than (2) to impose the compatibility for pressure $p$? or while coining the test problem, what kind of $p$ should I use?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2012-09-10T21:59:39.303" LastActivityDate="2012-09-15T15:51:51.217" Title="Impose the compatibility conditions for mixed finite elements method in Stokes equation" Tags="|finite-element|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3262" PostTypeId="1" AcceptedAnswerId="3302" CreationDate="2012-09-11T02:59:38.553" Score="40" ViewCount="17914" Body="&lt;p&gt;To my knowledge, there are 4 ways to solving a system of linear equations (correct me if there are more):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the system matrix is a full-rank square matrix, you can use Cramer’s Rule;&lt;/li&gt;&#xA;&lt;li&gt;Compute the inverse or the pseudoinverse of the system matrix;&lt;/li&gt;&#xA;&lt;li&gt;Use matrix decomposition methods (Gaussian or Gauss-Jordan elimination is considered as LU decomposition);&lt;/li&gt;&#xA;&lt;li&gt;Use iterative methods, such as the conjugate gradient method.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In fact, you almost never want to solving the equations by using Cramer's rule or computing the inverse or pseudoinverse, especially for high dimensional matrices, so the first question is when to use decomposition methods and iterative methods, respectively. I guess it depends on the size and properties of the system matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second question is, to your knowledge, what kind of decomposition methods or iterative methods are most suitable for certain system matrix in terms of numerical stability and efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the conjugate gradient method is used to solve equations where the matrix is symmetric and positive definite, although it can also be applied to any linear equations by converting $\mathbf{A}x=b$ to $\mathbf{A}^{\rm T}\mathbf{A}x=\mathbf{A}^{\rm T}b$. Also for positive definite matrix, you can use Cholesky decomposition method to seek the solution. But I don't know when to choose the CG method and when to choose Cholesky decomposition. My feeling is we'd better use CG method for large matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For rectangular matrices, we can either use QR decomposition or SVD, but again I don't know how to choose one of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other matrices, I don't now how to choose the appropriate solver, such Hermitian/symmetric matrices, sparse matrices, band matrices etc.&lt;/p&gt;&#xA;" OwnerUserId="1614" LastEditorUserId="1804" LastEditDate="2012-09-16T12:51:10.717" LastActivityDate="2015-12-17T18:33:44.070" Title="How to choose a method for solving linear equations" Tags="|linear-algebra|linear-solver|iterative-method|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="3265" PostTypeId="2" ParentId="3260" CreationDate="2012-09-11T13:27:07.393" Score="2" Body="&lt;p&gt;If you have a structured grid then this is unquestionably premature optimization. Direct access in more than one dimension requires that some of the ghost values are &quot;between&quot; owned values, thus a copy is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use Krylov methods, accelerated nonlinear methods (NGMRES), or multistage time integrators, then at the end of the day use of &lt;code&gt;VecGhost&lt;/code&gt; in place of normal global vectors will almost always require &lt;em&gt;more&lt;/em&gt; memory because the extra space is stored as part of every vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So unless you have a very unique application, you really shouldn't try to use &lt;code&gt;VecGhost&lt;/code&gt; for structured grids. For unstructured grids, it's arguably a natural representation and typically not harmful unless you have larger stencils, but it's also not a performance optimization. If you have have a very special use case, you can implement &lt;code&gt;DMCreateGlobalVector_XXX()&lt;/code&gt; to return a &lt;code&gt;VecGhost&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-09-11T13:27:07.393" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3266" PostTypeId="1" AcceptedAnswerId="3274" CreationDate="2012-09-11T17:10:27.930" Score="2" ViewCount="1321" Body="&lt;p&gt;Suppose a system $$Ax=b$$ is given, with $A\in\mathbb{R}^{n\times n}$ being a symmetric positive-definite matrix, and some non-zero $b\in\mathbb{R}^n$. The gradient method with optimum step length can be written as&#xA;$$x_{k+1}=x_k-\alpha_k\cdot g_k,$$ with $g_k=Ax_k-b$ and $\alpha_k=\frac{g_k^Tg_k}{g_k^TAg_k}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can it be proven that the above iterative procedure converges to $\bar{x}=A^{-1}b$, regardless of the initialization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can the above iterative scheme be regarded as a &lt;em&gt;linear&lt;/em&gt; fixed-point iteration? In fact, what is the precise meaning of &lt;em&gt;linear&lt;/em&gt; in the term &lt;em&gt;linear fixed-point iteration&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-09-13T12:17:23.813" Title="Convergence of the gradient descent and linear vs non-linear fixed point iteration" Tags="|linear-algebra|matrix|linear-solver|iterative-method|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3267" PostTypeId="1" AcceptedAnswerId="3269" CreationDate="2012-09-12T00:09:32.993" Score="1" ViewCount="133" Body="&lt;p&gt;After reading the first answer &lt;a href=&quot;https://scicomp.stackexchange.com/questions/81/what-guidelines-should-i-follow-when-choosing-a-sparse-linear-system-solver&quot;&gt;here&lt;/a&gt; about how the best way to find the most performant sparse solver is to try almost everything, I began to wonder if there was any past work on libraries or research for adaptive libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by an adaptive library is one that implements (or links to) a wide range of iterative and direct solvers, attempting different ones over the course of a simulation to find the best performer automatically.  While simple simulations likely wouldn't gain from this sort of system, I am currently working with simulations that involve on the order of a million solves.  Even if the first few dozen of my solves were a couple orders of magnitude slower than the rest as it converged to a near optimum solver setup speed-ups of only a few percent over what I would have chosen myself would easily make up for the extra time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously such a system would require that each matrix being solved would have to share characteristics with previous ones on some level, but this is usually the case in FD, FV, or FE models.  So my question is, do such libraries exist, and what are some of the pitfalls an implementation of one may experience with regards to performance?&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-09-12T08:54:17.110" Title="Adaptive Linear Algebra Libraries" Tags="|linear-algebra|machine-learning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3268" PostTypeId="2" ParentId="3266" CreationDate="2012-09-12T03:22:21.227" Score="4" Body="&lt;p&gt;This is a damped Richardson method. Since one step of this operation, $x_k \mapsto x_{k+1}$ is not a linear operation, it is not a stationary iteration. The presence of the extra matrix multiply and extra reduction per iteration make the method very unattractive compared to Krylov subspace methods which need onyl one operator application and have better convergence properties.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-09-13T12:17:23.813" LastActivityDate="2012-09-13T12:17:23.813" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3269" PostTypeId="2" ParentId="3267" CreationDate="2012-09-12T03:46:42.887" Score="7" Body="&lt;p&gt;You shouldn't get the impression from the cited question that there is no theory, such that brute force testing of &quot;all methods&quot; (whatever that means; it's easy to produce infinite sequences of methods that are hard to exclude a priori) is the only viable approach. The problem is that if you don't know enough about your problem to formulate a coherent question, there are very few definitive statements that we can make. Frankly, much of the time that people ask these overly general questions, it is due to a combination of laziness and lack of familiarity with the field. There are also a significant number of cases where the problem changes very rapidly due to changing physics, discretization, materials, domains, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any given class, we can usually design good solvers, but for non-standard problems, it requires some effort and some experience (as with any skilled trade). In designing a method, we often use an iterative process where we look at convergence and diagnostics obtained by trying a few classes of methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for automatic solver configuration adaptivity, it is much like &quot;auto-tuning&quot;, but at a higher level that involves more mathematics. Projects such as SALSA that Jack referenced have not been successful at finding good solvers for problems that were not already well-understood. This is not surprising considering that general-purpose &quot;autotuning&quot; systems tend to only be effective at finding local minima that are eventually shown to be not particularly good. If you inject a lot of domain knowledge, the autotuning system can &quot;fine-tune&quot; some details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it's mostly a waste of time to ask for a black-box iterative solver for a non-standard system. General auto-configuration would be a black box if it yielded good quality results. Instead, I recommend not trying to take a short-cut, understanding the problems and understanding the properties of a few classes of methods, then using that knowledge to design a good solver.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="276" LastEditDate="2012-09-12T08:54:17.110" LastActivityDate="2012-09-12T08:54:17.110" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3270" PostTypeId="2" ParentId="3253" CreationDate="2012-09-12T12:59:41.597" Score="2" Body="&lt;p&gt;When I have a problem in a network of highly asynchronous processes governed by messages or events, I use a method which is not easy but is effective.&#xA;It involved getting time-stamped logs of the processes, merging them into a common timeline, and tracking the progress of some messages as they trigger activities, triggering further messages. What I'm looking for is delay between the time a message is received and the time it is acted upon, and understanding the reason for the delay.&#xA;When a problem is found, it is fixed, and the process is repeated.&#xA;This way, you can get really satisfying performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's important to see how this differs from approaches where you measure, measure, measure.&#xA;The only thing measuring can conceivably tell you is where &lt;em&gt;not&lt;/em&gt; to look.&#xA;Real performance tuning requires looking carefully at the details, from a time perspective.&#xA;What you're looking for is not where time is spent, but where it is spent unnecessarily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastActivityDate="2012-09-12T12:59:41.597" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3271" PostTypeId="1" AcceptedAnswerId="3273" CreationDate="2012-09-12T22:22:44.810" Score="14" ViewCount="9904" Body="&lt;p&gt;According to Nocedal &amp;amp; Wright's Book &lt;em&gt;Numerical Optimization&lt;/em&gt; (2006), the Wolfe's conditions for an inexact line search are, for a descent direction $p$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sufficient Decrease: $f(x+\alpha p)\le f(x)+c_1\alpha_k\nabla f(x)^T p$&lt;br&gt;&#xA;Curvature Condition: $\nabla f(x+\alpha p)^Tp\ge c_2 \nabla f(x)^T p$&lt;br&gt;&#xA;for $0&amp;lt;c_1&amp;lt;c_2&amp;lt;1$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can see how the sufficient decrease condition states that the function value at the new point $x+\alpha p$ must be under the tangent at $x$.  But I'm not sure what the curvature condition is telling me geometrically.  Also, why must the relation $c_1&amp;lt;c_2$ be imposed?  What does this accomplish, geometrically?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-09-13T02:14:07.020" LastActivityDate="2012-09-13T02:14:07.020" Title="Understanding the Wolfe Conditions for an Inexact line search" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3272" PostTypeId="2" ParentId="3266" CreationDate="2012-09-13T00:17:45.367" Score="1" Body="&lt;p&gt;It is easy to verify that $x=A^{-1}b$ is indeed a fixed point of the iteration. The question is whether the iteration operator&#xA;$$&#xA;  I - \alpha A&#xA;$$&#xA;is a contraction. If $A$ is SPD, this is the case if $\alpha$ were less than or equal $\frac{1}{\lambda_{\textrm{max}}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, we can only infer with simple arguments that $\alpha\le\frac{1}{\lambda_{\textrm{min}}(A)}$. This is not enough to guarantee convergence using the usual arguments of linear iterations. In other words, it is the nonlinearity of the iteration that ensures that this actually converges, if in fact it does. One would need a more sophisticated analysis to then prove convergence.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-13T00:17:45.367" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3273" PostTypeId="2" ParentId="3271" CreationDate="2012-09-13T00:28:30.963" Score="14" Body="&lt;p&gt;The curvature condition essentially says this: We know that $\nabla f(x)\cdot p &amp;lt; 0$ (because $p$ is a descent direction). So in direction $p$, it goes downhill. Now, we're looking for a minimum, i.e. a point where $\nabla f=0$. That means that we don't want to accept step lengths $x+\alpha p$ where the gradient in direction $p$, i.e., $\nabla f(x+\alpha p) \cdot p$ is still as negative as it is at x. Rather, we want to stop at a place where the gradient is less negative or even positive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the right hand side of the curvature condition is negative, a common variant of the condition is to require $$|\nabla f(x+\alpha p) \cdot p| \le c_2 |\nabla f(x) \cdot p|$$ which I usually find easier to understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding this will allow you to easily construct cases where you can't satisfy both conditions unless $c_1&amp;lt;c_2$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-13T00:28:30.963" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3274" PostTypeId="2" ParentId="3266" CreationDate="2012-09-13T07:33:25.037" Score="2" Body="&lt;p&gt;This is known as the steepest descent method, which converges for any symmetric and positive definite matrix (with a linear rate that depends on its condition number) independent of the initialization; see chapter 5.3.1 of Saad's &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot; rel=&quot;nofollow&quot;&gt;Iterative Methods for Sparse Linear Systems&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a nonlinear iterative method, since the new iterate $x_{k+1}$ is not a linear function of $x_k$ (as, e.g., in the Jacobi or Gauss-Seidel iteration) since it appears (by way of $g_k$) in the definition of $\alpha_k$. It should be pointed out that the conjugate gradient method (see top of page 200 in Saad's book) needs essentially the same amount of work and has (in practice much) better convergence properties. &lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2012-09-13T09:45:58.187" LastActivityDate="2012-09-13T09:45:58.187" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3275" PostTypeId="1" AcceptedAnswerId="3277" CreationDate="2012-09-13T13:12:34.520" Score="1" ViewCount="4714" Body="&lt;p&gt;A method to orthogonalize a set of vectors (vectors of unit length that are mutually orthogonal) is the Gram-Schmidt process:&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the the nested &lt;em&gt;for&lt;/em&gt; loops imply that the difference is between vector $v_i$ and vectors $\{v_1, \dots, v_{i-1}\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I wonder about the behavior of the process in case the set of input vectors is linearly dependent. Namely, suppose the process is initialized on vectors $\{v_1, v_2, v_3, v_4, v_5\}$. In case $v_4$ can be expressed as a linear combination of $\{v_1, v_2, v_3\}$, the process will set $v_4=0_n$ (i.e, zero vector).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose that $v_2$ can be expressed as a linear combination of $\{v_3, v_4, v_5\}$. Would such &lt;em&gt;ordering&lt;/em&gt; of vectors imply that the Gram-Schmidt process will fail to set $v_2=0_n$ (fail to report linear dependence)? If so, what is the common way to ensure that Gram-Schmidt would yield orthonormal vectors and report linearly dependent ones? &lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-09-14T15:17:51.407" Title="Gram-Schmidt method to identify linearly dependent vectors" Tags="|linear-algebra|matrix|basis-set|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3276" PostTypeId="1" CreationDate="2012-09-13T13:47:02.033" Score="10" ViewCount="457" Body="&lt;p&gt;In recent years, several libraries/software projects have appeared that offer some form or another of general-purpose data-driven shared-memory parallelism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main idea is that instead of writing an explicitly threaded code, programmers implement their algorithms as inter-dependent tasks which are then scheduled dynamically by a general-purpose middleware on a shared-memory machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of such libraries are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://icl.cs.utk.edu/projectsfiles/plasma/pubs/56-quark_users_guide.pdf&quot;&gt;QUARK&lt;/a&gt;: Originally designed for the &lt;a href=&quot;http://icl.cs.utk.edu/magma/&quot;&gt;MAGMA&lt;/a&gt; parallel linear algebra library, seems to have been used for a &lt;a href=&quot;http://arxiv.org/abs/1203.0889&quot;&gt;parallel Fast Multipole Method&lt;/a&gt; too.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Cilk&quot;&gt;Cilk&lt;/a&gt;: Originally an MIT-based project, now supported by Intel, implemented as language/compiler extensions to C, used in the &lt;a href=&quot;http://supertech.csail.mit.edu/chess/&quot;&gt;Cilkchess&lt;/a&gt; computer chess software and &lt;a href=&quot;http://www.fftw.org/fftw2_doc/fftw_4.html&quot;&gt;experimentally in FFTW&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.bsc.es/computer-sciences/programming-models/smp-superscalar&quot;&gt;SMP superscalar&lt;/a&gt;: Developed at the Barcelona Supercomputing Center, similar to Cilk in many respects, based on &lt;code&gt;#pragma&lt;/code&gt; extensions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://runtime.bordeaux.inria.fr/StarPU/&quot;&gt;StarPU&lt;/a&gt;: Similar library based &quot;codelets&quot; which can be compiled for and scheduled on several different architectures, including GPUs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;OpenMP tasks: As of version 3.0, OpenMP introduced &quot;tasks&quot; that can be scheduled asynchronously (see Section 2.7 of the specification).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://threadingbuildingblocks.org/&quot;&gt;Intel's Threading Building Blocks&lt;/a&gt;: Uses C++ classes to create and launch asynchronous tasks, see Section 11 of the Tutorial.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/OpenCL&quot;&gt;OpenCL&lt;/a&gt;: Supports task-based parallelism on multi-cores.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While there is a lot of literature describing the inner working of these libraries/language extensions and their application to specific problems, I have only come across very few examples of them being used in practice in scientific computing applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So here's the question:&lt;/strong&gt; Does anybody know of scientific computing codes using any of these libraries/language extensions, or similar, for shared-memory parallelism?&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2013-10-10T08:45:00.667" LastActivityDate="2013-10-10T08:45:00.667" Title="Task-based shared-memory parallel libraries in Scientific Computing" Tags="|parallel-computing|software|hpc|libraries|multicore|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3277" PostTypeId="2" ParentId="3275" CreationDate="2012-09-13T13:48:46.400" Score="5" Body="&lt;p&gt;If $v_2$ can be expressed as a linear combination of $\{ v_3, v_4, v_5 \}$, but not of a subset of these vectors, then, e.g., $v_5$ is in the linear span of $\{ v_2, v_3, v_4 \}$. So your algorithm would likely map $v_5$ onto $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, if you apply the Gram-Schmidt proecess to the sequence of $v_i$, $1 \leq i \leq L$, a vector $v_k$ is mapped onto $0$ if it is in linear span of the $v_1, \dots, v_{k-1}$, in which case you probably drop it from further considerations. Eventually, your algorithm will produce an orthonormal basis $\{ w_1, \dots, w_l \}$ of the linear span of $\{ v_1, \dots, v_L \}$, $l \leq L$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this only holds up to rounding errors. Furthermore, you would probably use the stabilized Gram-Schmidt process. An interesting follow-up question is when a vector should be regarded as numerically $0$.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-09-13T13:48:46.400" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3278" PostTypeId="1" CreationDate="2012-09-13T17:15:21.707" Score="1" ViewCount="63" Body="&lt;p&gt;I have an $n \times m$ matrix of data $\mathbf{D}$ as well as a $k$-partition $P$ of $n$ indices each representing a row in a dataset. Assuming an arbitrary clustering algorithm $A$, I would like to find a subset $F$ of $\{1,\ldots, m\}$, representing indices of the columns of $D$, such that applying $A$ to only the columns indexed by $F$, an $n \times |F|$ matrix, minimizes the average variation of information (VI) between $P$ and the output of $A$. Another way of saying this is that I would like to be able to find the subspace on which $\mathbf{D}$ is most informative about $P$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(That the metric is VI or that the statistic taken over multiple runs is an average is not necessarily important. I'm looking for a general solution, but if a solution that specifies either the metric or the clustering algorithm exists, I'd be interested.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attempting to build the distribution of (normalized) variation of information values by brute force is not practical since the number of possible subsets of $\{1,\ldots, m\}$ is the Stirling number (of the second kind) $S(m, 2)$, which obviously gets monstrous as $m$ gets to any interesting size.&lt;/p&gt;&#xA;" OwnerUserId="1165" LastActivityDate="2012-09-14T09:25:01.167" Title="Finding most informative feature subsets given dataset, clustering algorithm and gold standard partition" Tags="|machine-learning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3279" PostTypeId="1" AcceptedAnswerId="3289" CreationDate="2012-09-13T20:01:55.017" Score="3" ViewCount="129" Body="&lt;p&gt;I am reading a paper on stability of CG, and I came across the following statement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\|A\|\,\|p\|^2}{\langle p,Ap\rangle} \leq \kappa(A)&#xA;\end{equation}&#xA;where $\kappa(\cdot)$ is the condition number and $\langle \cdot, \cdot \rangle$ the inner product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you, please, help me understand this bound? I cannot see how it was derived.&#xA;The authors simply state this as a fact. What obvious fact am I missing?&#xA;$p$ is the search direction vector, and $A$ is the SPD matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;here is the paper: &lt;a href=&quot;http://www.cs.cas.cz/tichy/download/public/StTi2004.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.cas.cz/tichy/download/public/StTi2004.pdf&lt;/a&gt;&#xA;The statement is on page 13, at the bottom, on the first line after equation 4.27: &quot;Using (4.5)–(4.8) and ....&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2029" LastEditorUserId="1744" LastEditDate="2012-09-14T14:11:20.783" LastActivityDate="2012-09-14T14:11:20.783" Title="How can you explain the following bound on the inner product?" Tags="|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3280" PostTypeId="2" ParentId="3276" CreationDate="2012-09-13T21:05:54.337" Score="4" Body="&lt;p&gt;In my opinion, these systems have been relatively unsuccessful due primarily to the following reasons.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The naive perspective that parallel computation is about parallelizing the computation (e.g. flops) more than exposing memory locality and removing synchronization points. Even though some problems, such as dense matrix algorithms, are still FP-limited, that only occurs after careful consideration of the memory subsystem and most computational kernels (especially in the PDE world) are more memory-sensitive. Work queues tend to trade memory locality for better naive balance of flops and more atomic memory operations (due to synchronization through the queue).&lt;/li&gt;&#xA;&lt;li&gt;Reliance on over-decomposition for dynamic load balance at the expense of strong scalability. Tasks generally have overlapping data dependencies (ghost values). As the size of the interior shrinks, the ghost/interior ratio increases. Even when this does not imply redundant work, it implies increased memory movement. Significant reductions in memory bandwidth requirements can be had by approaches such as cooperative prefetch by which multiple threads share an L1 or L2 cache by software-prefetching for their neighbor (which implicitly keeps the group of threads approximately coherent). This is exactly the opposite of over-decomposition.&lt;/li&gt;&#xA;&lt;li&gt;Unpredictable performance, mostly due to the memory-related issues above.&lt;/li&gt;&#xA;&lt;li&gt;Lack of library-friendly components. This can almost be summarized as not having an analogue of an &lt;code&gt;MPI_Comm&lt;/code&gt; which allows different libraries to perform rich operations without colliding, as well as to pass the context between libraries and recover necessary attributes. The abstraction provided by the &quot;communicator&quot; is important for library composition regardless of whether shared or distributed memory is used.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-09-13T21:05:54.337" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="3282" PostTypeId="1" CreationDate="2012-09-14T03:31:57.460" Score="2" ViewCount="195" Body="&lt;p&gt;I'm trying to teach myself FEM. The problem I have in mind is to completely investigate a plastic (e.g rubber band) being stretched (therefore, boundary conditions move as you stretch). I imagine this is a very simple problem but I can't find it covered in common FEA books. Does anybody know a good source and software I can play around with?&lt;/p&gt;&#xA;" OwnerUserId="2031" LastEditorUserId="1744" LastEditDate="2012-09-14T09:46:40.240" LastActivityDate="2012-09-14T09:46:40.240" Title="FEM simulation of a material being stretched" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3283" PostTypeId="1" AcceptedAnswerId="3309" CreationDate="2012-09-14T06:43:15.577" Score="16" ViewCount="1824" Body="&lt;p&gt;what are state of art methods for numerical solution of ODEs with discontinuous right side? I'm mostly interested piecewise-smooth right side functions, e.g. sign.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to solve the equation of a following type: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;      \dot x &amp;amp;= v\\&#xA;      \dot v &amp;amp;= &#xA;      \begin{cases}&#xA;        (|F_\text{external}| - |F_\text{friction}|) \mathop{\rm sign} (F_\text{external}) &amp;amp; :|F_\text{external}| &amp;lt; |F_\text{friction}|\\&#xA;        0 &amp;amp; : \text{otherwise}&#xA;      \end{cases}&#xA;  \end{align*}&lt;/p&gt;&#xA;" OwnerUserId="2032" LastEditorUserId="1744" LastEditDate="2012-09-17T22:09:20.657" LastActivityDate="2013-05-07T06:57:48.970" Title="Numerical methods for discontinuous r.s. ODEs" Tags="|ode|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3284" PostTypeId="1" CreationDate="2012-09-14T07:14:41.977" Score="1" ViewCount="89" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Multilayer_perceptron#Activation_function&quot; rel=&quot;nofollow&quot;&gt;This article&lt;/a&gt; says that all any multilayer perceptron with a linear on-off functions for all the neurons can be reduced to a two-layered perceptron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, consider a two input/one output perceptron. The theory says that it impossible to model XOR function with this kind of perceptron, because it can represent only linearly separable function and XOR is not linearly separable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://www.heatonresearch.com/online/introduction-neural-networks-java-edition-2/chapter-1/page4.html&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt; (at the bottom of the page) the author shows a 3-layer on-off perceptron which does represent XOR.&lt;br&gt;&#xA;According to the wikipedia article, this perceptron can reduced to a two-layer perceptron and according to the another part of theory, it cannot model XOR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone spread some light on this? Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm not sure this the right place to ask this question. The AI site in Area 51 was closed.)&lt;/p&gt;&#xA;" OwnerUserId="2033" LastActivityDate="2012-09-14T15:12:52.430" Title="neural networks: multilayer on-off perceptrons" Tags="|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3285" PostTypeId="2" ParentId="3275" CreationDate="2012-09-14T09:16:22.860" Score="5" Body="&lt;p&gt;While it doesn't answer your explict question, I offer the following important remark that might answer the intentions behind the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The anomalies of the Gram-Scmidt process in case of dependence are avoided if one computes the orthogonalization instead by the standard Householder $QR$ factorization (or in the sparse case the Givens variant). The latter is also much more stable and hence usually the method of choice. Dependence is then simply reflected by one or more zero (or in finite precision arithmetic) tiny diagonal entry in $R$. Column pivoting helps by moving these tiny entries towards the bottom of $R$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-14T15:17:51.407" LastActivityDate="2012-09-14T15:17:51.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3286" PostTypeId="2" ParentId="3278" CreationDate="2012-09-14T09:25:01.167" Score="1" Body="&lt;p&gt;This problem is called subset selection and is nowadays usually solved by appropriate nonsmooth convex optimization techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two recent papers describing subset selection methods are &#xA;&lt;BR&gt;&#xA;&lt;a href=&quot;http://users.cms.caltech.edu/~jtropp/reports/Tro04-Just-Relax-TR.pdf&quot; rel=&quot;nofollow&quot;&gt;http://users.cms.caltech.edu/~jtropp/reports/Tro04-Just-Relax-TR.pdf&lt;/a&gt;&#xA;&lt;BR&gt;&#xA;&lt;a href=&quot;http://automatica.dei.unipd.it/public/Schenato/PSC/2010_2011/gruppo4-Building_termo_identification/Bibliografia%20Casuale/Sensor%20Selection%20via%20Convex%20Optimization.pdf&quot; rel=&quot;nofollow&quot;&gt;http://automatica.dei.unipd.it/public/Schenato/PSC/2010_2011/gruppo4-Building_termo_identification/Bibliografia%20Casuale/Sensor%20Selection%20via%20Convex%20Optimization.pdf&lt;/a&gt;&#xA;&lt;BR&gt;&#xA;but this is just the tip of an iceberg.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-14T09:25:01.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3287" PostTypeId="2" ParentId="3282" CreationDate="2012-09-14T09:25:34.610" Score="4" Body="&lt;p&gt;Modelling the stretching of objects made of rubber-like material is quite a complex task: you need to take into account &lt;a href=&quot;http://en.wikipedia.org/wiki/Hyperelastic_material&quot; rel=&quot;nofollow&quot;&gt;hyperelastic constitutive equations&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_strain_theory&quot; rel=&quot;nofollow&quot;&gt;large strains&lt;/a&gt;, and write &lt;a href=&quot;http://en.wikipedia.org/wiki/Continuum_mechanics#Balance_laws&quot; rel=&quot;nofollow&quot;&gt;momentum conservation equations&lt;/a&gt; in the current configuration and not in the reference one (large displacements).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This results in a highly non-linear problem that can be solved (if mesh distortion does not becomes to large) by a purely lagrangian approach in FEM. (If mesh distortion becomes a problem, remeshing techniques or ALE approaches are to be considered.) In a Lagrangian approach you do not have &quot;moving boundaries&quot;, since Dirichlet conditions are still expressed as displacements with respect to the reference configuration. (Of course if you remesh or switch to an ALE formulation, tracking moving boundaries is a problem to deal with.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This said I think that this problem is not well suited to learn basic FEM. Linear problems (like heat conduction under some simplifying assumptions or small strain/small displacement linear elastic solid mechanics) are more useful.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2012-09-14T09:25:34.610" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3288" PostTypeId="2" ParentId="3279" CreationDate="2012-09-14T09:27:50.397" Score="2" Body="&lt;p&gt;This assumes that the matrix $A$ is symmetric. Transform it to diagonal form, and observe that the singular values are the absolute values of the eigenvalues. THen the derivation essentially reduces to the definition of the condition number as the quotient of the extreme singular values.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-14T09:27:50.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3289" PostTypeId="2" ParentId="3279" CreationDate="2012-09-14T10:26:13.617" Score="6" Body="&lt;p&gt;The bound is a consequence of the following three facts:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The condition number is defined as the ratio of the largest eigenvalue $\lambda_\textrm{max}$ and the smallest eigenvalue $\lambda_\textrm{min}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The spectral norm $\|A\|_2$ of a symmetric positive definite matrix is $\lambda_\textrm{max}$ (since, as Arnold Neumaier pointed out, it is the square root of the largest singular value).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;By the Courant-Fischer theorem, the smallest eigenvalue of a symmetric matrix $A$ satisfies&#xA;$$&#xA;\lambda_\textrm{min} = \min_{x\neq 0} \frac{\langle x,Ax\rangle}{\langle x,x\rangle}.&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Putting this together, we have for any $p\neq 0$ that&#xA;$$&#xA;\kappa(A) = \frac{\lambda_\textrm{max}}{\lambda_\textrm{min}} &#xA;= \frac{\|A\|}{\min_x \frac{\langle x,Ax\rangle}{\|x\|^2}}&#xA;\geq \frac{\|A\|}{\frac{\langle p,Ap\rangle}{\|p\|^2}}&#xA;= \frac{\|A\|\|p\|^2}{\langle p,Ap\rangle}&#xA;$$&#xA;since the fraction can only become smaller for any $p$ that is not the minimizer of the denominator.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2012-09-14T10:26:13.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3290" PostTypeId="2" ParentId="1048" CreationDate="2012-09-14T11:01:48.587" Score="16" Body="&lt;p&gt;Iterative methods in a nutshell:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Stationary methods are in essence &lt;em&gt;fixed point iterations&lt;/em&gt;: To solve $Ax=b$, you pick an invertible matrix $C$ and find a fixed point of &#xA;$$&#xA;x = x + Cb- CAx&#xA;$$&#xA;This converges by Banach's fixed point theorem if $\|I-CA\|&amp;lt;1$. The various methods then correspond to a specific choice of $C$ (e.g., for Jacobi iteration, $C=D^{-1}$, where $D$ is a diagonal matrix containing the diagonal elements of $A$).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Krylov methods subspace methods are in essence &lt;em&gt;projection methods&lt;/em&gt;: You pick subspaces $U,V\subset \mathbb{C}^n$ and look for a $\tilde x \in U$ such that the residual $b-A\tilde x$ is orthogonal to $V$. For Krylov methods, $U$ of course is the space spanned by powers of $A$ applied to an initial residual. The various methods then correspond to specific choices of $V$ (e.g., $V=U$ for CG and $V=AU$ for GMRES). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convergence properties of these methods (and projection methods in general) follow from the fact that due to the respective choice of $V$, the $\tilde x$ are optimal over $U$ (e.g., they minimize the error in the energy norm for CG or the residual for GMRES). If you increase the dimension of $U$ in every iteration, you are guaranteed (in exact arithmetic) to find the solution after finitely many steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by Reid Atcheson, using Krylov spaces for $U$ allows you to prove rates of convergence in terms of the eigenvalues (and thus the condition number) of $A$. In addition, they are crucial for deriving efficient algorithms for computing the projection $\tilde x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is nicely explained in Youcef Saad's &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot;&gt;book on iterative methods&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2012-09-14T11:01:48.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3291" PostTypeId="2" ParentId="3276" CreationDate="2012-09-14T11:20:05.573" Score="8" Body="&lt;p&gt;&lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal.II&lt;/a&gt; uses the Threading Building Blocks throughout the library and by and large we're reasonably happy with it. We've looked at a few alternatives, in particular OpenMP since everyone seems to be using that for simpler codes, but found them lacking. In particular, OpenMP has the huge disadvantage that its task model does not allow you to get a handle for a task you started, and consequently it is difficult to access the state of a task (e.g. to wait for it finishing) or return values of functions you run on a separate task. OpenMP is primarily good for parallelizing the &lt;em&gt;innermost&lt;/em&gt; loops but you gain parallel efficiency by parallelizing the &lt;em&gt;outermost&lt;/em&gt;, complex loops, and OpenMP is not the tool for that while the TBB are reasonably good for that.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="67" LastEditDate="2012-09-18T14:39:03.197" LastActivityDate="2012-09-18T14:39:03.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3292" PostTypeId="2" ParentId="3284" CreationDate="2012-09-14T15:12:52.430" Score="3" Body="&lt;p&gt;The neural net in the second article doesn't use linear activation functions.  It uses thresholded on-off functions (e.g. f(x) = 1 if x &gt; threshold, else 0), hence can model XOR.  (A linear activation function would just be f(x) = x).  &lt;/p&gt;&#xA;" OwnerUserId="944" LastActivityDate="2012-09-14T15:12:52.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3293" PostTypeId="2" ParentId="3239" CreationDate="2012-09-14T15:26:44.337" Score="5" Body="&lt;p&gt;Starting from where David Ketcheson left me in his answer, a little bit more search revealed some historical notes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scheme I outlined above was considered already back in &lt;strong&gt;1900&lt;/strong&gt; by J. Massau, in &lt;em&gt;Mémoire sur l'intégration graphique des équations aux dérivées partielles&lt;/em&gt;. The work is republished in 1952 by G. Delporte, Mons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first (albeit brief) modern analysis of its convergence and such was given by Courant, Friedrichs, and Lewy's in their classic 1928 paper in Math. Ann. &lt;/p&gt;&#xA;" OwnerUserId="1481" LastActivityDate="2012-09-14T15:26:44.337" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3294" PostTypeId="2" ParentId="332" CreationDate="2012-09-14T17:23:58.317" Score="7" Body="&lt;p&gt;The Galerkin approach (seeking an approximation from a given subspace &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; such that the residual is orthogonal to another given subspace &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;) is indeed very general (and not restricted to finite-dimensional spaces). In the context of the numerical solution of partial differential equations, there are essentially two conditions that &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; have to satisfy:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The discrete problem must have a unique solution; this often requires verifying so-called &lt;em&gt;inf-sup-conditions&lt;/em&gt;. (In the standard Ritz-Galerkin method, this is basically the &lt;a href=&quot;https://en.wikipedia.org/wiki/Weak_formulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lax-Milgram theorem&lt;/a&gt;; for linear systems &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt;, this amounts to &lt;span class=&quot;math-container&quot;&gt;$\dim U = \dim V$&lt;/span&gt; and that no vector in &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;-orthogonal to &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The discretization error must become smaller as the dimension of &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; are increased. This requires certain &lt;em&gt;approximation properties&lt;/em&gt; for the subspaces. Usually, one takes spaces of piecewise polynomials (as in the standard finite element method), but other choices are possible (e.g., spectral methods). (Similarly, projection methods for linear systems are often based on Krylov spaces, since these have nice approximation properties.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In fact, (some) finite volume methods can be described as &lt;a href=&quot;https://en.wikipedia.org/wiki/Discontinuous_Galerkin_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;discontinuous Galerkin methods&lt;/a&gt; (where &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; and/or &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; consist of piecewise constant functions.&lt;/p&gt;&#xA;&lt;p&gt;Most modern mathematical textbooks on finite element methods follow this approach. Two good examples are&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1007/978-0-387-75934-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brenner, Scott: &lt;em&gt;The Mathematical Theory of Finite Element Methods&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1007/978-1-4757-4355-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ern, Guermond: &lt;em&gt;Theory and Practice of Finite Elements&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(I particularly like the latter, as takes a very general approach to Galerkin methods, including mixed and hybrid finite elements and discontinuous Galerkin methods.)&lt;/p&gt;&#xA;&lt;p&gt;For linear systems, a good general discussion of the projection approach is given in Saad's &lt;a href=&quot;https://www-users.cse.umn.edu/%7Esaad/IterMethBook_2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorDisplayName="user43608" LastEditDate="2022-07-25T13:28:38.620" LastActivityDate="2022-07-25T13:28:38.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3295" PostTypeId="2" ParentId="3261" CreationDate="2012-09-14T19:48:48.197" Score="10" Body="&lt;p&gt;The compatibility condition concerns velocity, not pressure. It states that if you &lt;em&gt;only&lt;/em&gt; have Dirichlet boundary conditions for the velocity, then these should be compatible with the divergence-free constraint, i.e. $\int_{\partial \Omega} u \cdot n = 0$ with $\partial \Omega$ the boundary of the computational domain (not the cell).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case $\nabla p$ cannot be distinguished from $\nabla (p + c)$ with $c$ an arbitrary constant because you have no boundary condition on $p$ to fix the constant. Thus there are infinitely many solutions for the pressure and in order to compare solutions, a convention is needed. Mathematicians prefer choosing $c$ such that $\overline{p}=p_\mathrm{ref}$ (because they can integrate) while physicist prefer $p(x_\mathrm{ref}) = p_\mathrm{ref}$ (because they can measure in a point). If $Bp$ is your discrete equivalent of $\nabla p$, it implies that $B$ has a null space consisting of the identity vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Krylov subspace methods can solve a singular system by removing the null space from the Krylov subspace in which they look for the solution. However, that does not mean you will get the solution $p$ that matches a given convention, you will always need to determine the constant yourself in a postprocessing step, no solver can do it for you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some suggestions to tackle your problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Equation (2) seems strange. If $\nu$ is a function of $x$ how can it be outside the integral?&lt;/li&gt;&#xA;&lt;li&gt;Does your velocity field satisfy the compatibility constraint?&lt;/li&gt;&#xA;&lt;li&gt;Try not to do anything for the pressure, just let the solver free to come up with a $p$, then look at $p-p_\mathrm{exact}$. Is it a constant? &lt;/li&gt;&#xA;&lt;li&gt;If not, are you sure that the null space of $B$ is indeed the identity vector and nothing more? Both on paper and in the code? The problem seems small enough to actually compute the null space.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1973" LastEditorUserId="1973" LastEditDate="2012-09-15T06:25:26.997" LastActivityDate="2012-09-15T06:25:26.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3296" PostTypeId="1" AcceptedAnswerId="3299" CreationDate="2012-09-15T04:46:13.957" Score="3" ViewCount="370" Body="&lt;p&gt;problem comes from &#xA;&quot;Numerical solution of fractional integro-differential , equations by collocation method , E.A. Rawashdeh, Department of Mathematics, Yarmouk University, Irbid 21110, Jordan&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D^qy(t)=p(t)y(t)+f(t)+\int_{0}^{1}{K(t,s)y(s)\,ds} , t\in I=[0,1]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to create a maple code to check if the results in given article is valid or not but I do not have any idea about collocation method!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;any reference to collocation method solution are welcome!&lt;/p&gt;&#xA;" OwnerUserId="2040" LastActivityDate="2012-09-17T17:03:03.433" Title="Numerical solution of fractional integro-diffrential equ. using collocation method?" Tags="|finite-difference|matlab|mathematica|integral-equations|collocation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3297" PostTypeId="2" ParentId="3261" CreationDate="2012-09-15T15:51:51.217" Score="3" Body="&lt;p&gt;As for (Q1), you can choose a solver for saddle-point problems that computes a least squares solution for your system. Then an additional condition can be imposed on the multiplier, like setting a specific degree of freedom, are imposing a specific average.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, and I think this answers (Q1), you may use a linear constraint that can distinguish different constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This constraint may imposed in a post-processing step, or by a appropiate choice of the trial space (e.g., if you leave out one degree of freedom).&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-09-15T15:51:51.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3298" PostTypeId="1" AcceptedAnswerId="3300" CreationDate="2012-09-15T19:34:27.940" Score="7" ViewCount="1584" Body="&lt;p&gt;I'm working with the following mixed inhomogeneous boundary value problem:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla(\kappa\nabla u)=f$ in $\Omega$&lt;br&gt;&#xA;with $\partial\Omega = \Omega_1 \bigcup\Omega_2$ such that&lt;br&gt;&#xA;$u=g$ on $\partial\Omega_1$&lt;br&gt;&#xA;$\kappa\nabla u\cdot n = h$ on $\Omega_2$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note 1:&lt;/strong&gt;&lt;br&gt;&#xA;If the entire boundary $\partial \Omega$ only consisted of the inhomogenous neumann boundary condition, I can clearly see that the weak formulation is:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \kappa\nabla u \nabla v = \int_\Omega fv + \int_{\partial\Omega} vh$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there is no &quot;essential&quot; boundary condition and the weak formulation only consists of first derivatives, u and v can be elements of the sobolev space $H^1(\Omega)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note 2:&lt;/strong&gt;&lt;br&gt;&#xA;I understand if the entire boundary $\partial \Omega$ only consisted of the inhomogeneous dirichlet condition, I can take test functions $v\in H_0^1$ but $u\notin H_0^1$.  To correct for this, I can &quot;homogenize&quot; the boundary condition by proposing a new variable $w=u-G$ where G is some function in $H^1(\Omega)$ such that $G=g$ on $\partial\Omega$.  Using $u=w+G$, I can see that the weak formulation becomes  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \kappa\nabla w\cdot\nabla v=\int_\Omega fv -\int_{\partial\Omega}\kappa \nabla G\cdot\nabla v$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $w=u-G$, the boundary values of $w$ must be zero.  Hence, $w\in H_0^1(\Omega)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt;&lt;br&gt;&#xA;Following this pattern, I conjecture that for the mixed inhomogenous boundary value problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The test function $v\in H_0^1(\Omega)$&lt;/li&gt;&#xA;&lt;li&gt;The neumann boundary condition on $\partial\Omega_2$ is naturally satisfied.&lt;/li&gt;&#xA;&lt;li&gt;The dirichlet boundary condition on $\partial\Omega_1$ can be satisfied by proposing a new variable $w$ such that $w=u-G$ for some $G\in H^1(\Omega)$ and leading to the weak formulation:  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega k\nabla w \nabla v = \int_\Omega fv + \int_{\partial \Omega_2} hv -\int_\Omega k\nabla G \nabla v$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm almost completely sure that this is correct, but I just want to get some feedback on my conjectures.  If I'm completely wrong, please let me know! :)&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2023-08-05T23:29:45.840" Title="Appropriate space for weak solutions to an elliptical pde with mixed inhomogeneous boundary conditions" Tags="|finite-element|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3299" PostTypeId="2" ParentId="3296" CreationDate="2012-09-15T20:18:53.083" Score="3" Body="&lt;p&gt;As I understand it, collocation method for partial differential equations is something akin to interpolation.  First we characterize the solution space as a linear combination of some set of linearly independent functions $\phi_i(x)$.  The appropriate choice of $\phi_i(x)$ depends on the problem.  But assuming that the anticipated solution is sufficiently smooth, the simplest and easiest case to consider is the space of polynomials on $x$. Thus, the solution is of the form $u(x) = \sum \alpha_i \phi_i(x)$ where each $\phi_i(x)$ is a different (linearly independent) polynomial function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to determine which coefficients $\alpha_i$ make the best approximation.  We do so by substituting the linear combination directly into the partial differential equation.  Once you do this, you obtain expressions in terms of the functions $\phi_i(x)$ (lets call this expression EQ1).  Now, you need to force this expression to be exact at a finite number of points x.  So, we pick a set of points $\{x_i\}$ and evaluate EQ1 at these points.  This results in a linear system of equations whose unknowns are the $\alpha_i$'s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put the process into context, let's consider the 1 dimensional boundary value problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U_{xx} = f$ on $\Omega = (0,1)$&lt;br&gt;&#xA;$u=0$ on $\partial \Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I expect the solution to be smooth enough that it can be characterized by polynomials.  I can choose $\phi_i=x^i$.  Then my linear combination would be $u(x)=\sum \alpha_i x^i$ for $i=0,...,n$.  I can substitute this linear combination directly into the pde and obtain:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U_{xx} = f$$&#xA;$$\frac{\partial^2}{\partial x^2}[\sum_{i=1}^n \alpha_i \phi_i(x)]=f(x)$$&#xA;$$\sum_{i=0}^n \alpha_i \phi_i''(x)=f(x)$$ (lets call this last equation EQ1).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we need to choose points in our domain $\Omega = (0,1)$ to force EQ1 to be true.  Since there are n+1 coefficients ($\alpha_i$), we need to choose n+1 points.  While theoretically we can choose any points in the domain, certain choices of points produce better accuracy than others.  In my experience on smooth problems, the points are chosen as the n+1 gaussian quadrature points on (0,1).  Keep in mind that this is probably not the optimal choice for all problems and that each specific problem may have a better choice of points.  Once we know the points $x_k$, we evaluate EQ1 at each point $x_k$ and obtain:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{i=1}^n \alpha_i \phi_i''(x_k)=f(x_k)$$ for $k=0,...,n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, we have a system of equations for the unknowns $\alpha_i$, which we can solve by whatever method we see fit.  Once we know the exact values of the $\alpha_i$'s, the final solution, again, is just their linear combination:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x) = \sum_{i=0}^n\alpha_i \phi_i(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I can't access the journal where your article is, so I can't say with certainty that this is how it works for an integro-differential equation.  But this is the general framework of the collocation method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After considering the comments, I realized there are 2 other important conditions on the choice of functions $\phi_i(x)$:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;They must all satisfy the boundary conditions.  &lt;/li&gt;&#xA;&lt;li&gt;Ideally, they should have small support.   &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My initial choice of basis functions $\phi_i(x)=x^i$ do not all satisfy the boundary condition at $u(1)=0$.  Furthermore, polynomial functions are non-zero over a large region, leading to a dense matrix system which can be costly to solve.  Often, cubic hermite functions are used in place of the usual polynomial functions because:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;They can naturally be adjusted to satisfy the boundary conditions.&lt;/li&gt;&#xA;&lt;li&gt;They lead to sparse linear systems of equations &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-09-16T14:43:47.710" LastActivityDate="2012-09-16T14:43:47.710" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3300" PostTypeId="2" ParentId="3298" CreationDate="2012-09-16T04:45:18.010" Score="8" Body="&lt;p&gt;Your setup is fine, but it is often inconvenient to always formulate in terms of the general &quot;lifting&quot; function $G$, especially for nonlinear problems, structured grids, and simply for IO with solutions and forcing terms for the linear problem. If you find lifting convenient, then by all means, keep using it. It will never cause a problem for linear solvers provided you can create auxiliary operators with respect to the lifted system (if you have a system and solver that requires this; common with mixed methods for Stokes, $H(\mathrm{div})$, and $H(\mathrm{curl})$ problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand and implement general boundary conditions, I like to start with how I would formulate the nonlinear problem. The linear problem then falls out as the Jacobian of the nonlinear problem. For that, define the spaces&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$V_D$ : the discrete ansatz space with inhomogeneous Dirichlet conditions implicitly built in (so this is an affine space)&lt;/li&gt;&#xA;&lt;li&gt;$V_0$ : the corresponding homogeneous space (also the test function space for Galerkin discretizations)&lt;/li&gt;&#xA;&lt;li&gt;$V_\Gamma$ : the trace space on the Dirichlet boundary $\partial\Omega_1$ extended by zero into the domain&lt;/li&gt;&#xA;&lt;li&gt;$\bar V = V_0 \times V_\Gamma$ : all discrete functions on $\bar\Omega$ (this is what you want for visualization). This notation implies that the discrete space can be partitioned, but this is true of most practical compact discretizations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We introduce the projections&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$R_D : \bar V \to V_D$, the affine projector that imposes &quot;correct&quot; boundary conditions on the Dirichlet boundary $\partial D_1$&lt;/li&gt;&#xA;&lt;li&gt;$R_0 : \bar V \to V_0$, the projector that imposes zero boundary conditions on the Dirichlet boundary $\partial D_1$&lt;/li&gt;&#xA;&lt;li&gt;$R_\Gamma : \bar V \to V_\Gamma$, the projector into the trace space&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now we can write the discrete residual $F$ defined on $\bar V$ in terms of the &quot;natural&quot; residual $\tilde F$ that does nothing special for Dirichlet boundaries (e.g. your normal finite element assembly),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(u) = R_0 \tilde F(R_D u) + \alpha R_\Gamma (u - R_D 0) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha$ is a scaling parameter that can be used to improve conditioning (most relevant if you will use rediscretized geometric multigrid). The Jacobian isolates the Dirichlet degrees of freedom and the implementation is straightforward. Local element residuals and Jacobians are evaluated with correct Dirichlet values imposed on the state $u$, the result is inserted with contributions to Dirichlet nodes discarded. Then a loop over the boundary places the $\alpha$-scaled difference from the correct boundary values into the residual vector and inserts $\alpha$ on the diagonal of the Jacobian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will solve $F(u) = 0$ so we arrange terms as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ v^T \tilde F(u) = \int_\Omega (\nabla v \cdot \kappa \cdot \nabla u - v f) - \int_{\partial \Omega_2} v h . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no need for special mechanics since all discrete solutions live in $\bar V$ and the lifting function $G$ does not appear explicitly.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-09-16T05:12:13.863" LastActivityDate="2012-09-16T05:12:13.863" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3301" PostTypeId="1" AcceptedAnswerId="3303" CreationDate="2012-09-16T11:30:48.860" Score="6" ViewCount="12089" Body="&lt;p&gt;I implemented an algorithm to find the alpha shape of a set of points. The alpha shape is a concave hull for a set of points, whose shape depends on a parameter alpha deciding which points make up the hull. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have resolved the set of points in concave hull. These points make up a concave polygon. I would like to order these points in a clockwise manner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ordering points in a clockwise manner is straightforward when it is a convex shape. With a concave thing, I really don't know what to do. What algorithm is behind? I looked at 'non crossing shortest path' algorithms, 'shortest non crossing Hamiltonian path' algorithms, but I am still not convinced of any of these solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simple solution?&lt;/p&gt;&#xA;&#xA;EDIT&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/k6Bu1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1184" LastEditorUserId="1184" LastEditDate="2012-09-18T16:25:52.667" LastActivityDate="2013-11-07T15:49:40.477" Title="Concave polygon 'hull' finding" Tags="|algorithms|computational-geometry|graph-theory|convex-hull|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3302" PostTypeId="2" ParentId="3262" CreationDate="2012-09-16T12:27:01.557" Score="58" Body="&lt;p&gt;Your question is a bit like asking for which screwdriver to choose depending on the drive (slot, Phillips, Torx, ...): Besides there being &lt;a href=&quot;http://en.wikipedia.org/wiki/Screw_drive&quot;&gt;too many&lt;/a&gt;, the choice also depends on whether you want to just tighten one screw or assemble a whole set of library shelves. Nevertheless, in partial answer to your question, here are some of the issues you should keep in mind when choosing a method for solving the linear system $Ax=b$.&#xA;I will also restrict myself to invertible matrices; the cases of over- or underdetermined systems are a different matter and should really be separate questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you rightly noted, option 1 and 2 are right out: Computing and applying the inverse matrix is a tremendously bad idea, since it is much more expensive and often numerically less stable than applying one of the other algorithms. That leaves you with the choice between direct and iterative methods. The first thing to consider is not the matrix $A$, but what you expect from the numerical solution $\tilde x$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How &lt;em&gt;accurate&lt;/em&gt; does it have to be? Does $\tilde x$ have to solve the system up to machine precision, or are you satisfied with $\tilde x$ satisfying (say) $\|\tilde x - x^*\| &amp;lt; 10^{-3}$, where $x^*$ is the exact solution?&lt;/li&gt;&#xA;&lt;li&gt;How &lt;em&gt;fast&lt;/em&gt; do you need it? The only relevant metric here is clock time on &lt;em&gt;your&lt;/em&gt; machine - a method which scales perfectly on a huge cluster might not be the best choice if you don't have one of those, but you do have one of those shiny new Tesla cards.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As there's no such thing as a free lunch, you usually have to decide on a trade-off between the two. After that, you start looking at the matrix $A$ (and your hardware) to decide on a good method (or rather, the method for which you can find a good implementation). (Note how I avoided writing &quot;best&quot; here...) The most relevant properties here are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;em&gt;structure&lt;/em&gt;: Is $A$ symmetric? Is it dense or sparse? Banded?&lt;/li&gt;&#xA;&lt;li&gt;The &lt;em&gt;eigenvalues&lt;/em&gt;: Are they all positive (i.e., is $A$ positive definite)? Are they clustered? Do some of them have very small or very large magnitude?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With this in mind, you then have to trawl the (huge) literature and evaluate the different methods you find for your specific problem. Here are some general remarks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you really need (close to) machine precision for your solution, or if your matrix is small (say, up to $1000$ rows), it is hard to beat direct methods, especially for dense systems (since in this case, every matrix multiplication will be $\mathcal{O}(n^2)$, and if you need a lot of iterations, this might not be far from the $\mathcal{O}(n^3)$ a direct method needs). Also, LU decomposition (with pivoting) works for any invertible matrix, as opposed to most iterative methods. (Of course, if $A$ is symmetric and positive definite, you'd use Cholesky.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is also true for (large) sparse matrices &lt;em&gt;if&lt;/em&gt; you don't run into memory problems: Sparse matrices in general do not have a sparse LU decomposition, and if the factors do not fit into (fast) memory, these methods becomes unusable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, direct methods have been around for a long time, and very high quality software exists (e.g., UMFPACK, MUMPS, SuperLU for sparse matrices) which can automatically exploit the band structure of $A$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you need less accuracy, or cannot use direct methods, choose a Krylov method (e.g., CG if $A$ is symmetric positive definite, GMRES or BiCGStab if not) instead of a stationary method (such as Jacobi or Gauss-Seidel): These usually work much better, since their convergence is not determined by the spectral radius of $A$ but by (the square root) of the condition number and does not depend on the structure of the matrix. However, to get really good performance from a Krylov method, you need to choose a good preconditioner for your matrix - and that is more a craft than a science...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you repeatedly need to solve linear systems with the same matrix and different right hand sides, direct methods can still be faster than iterative methods since you only need to compute the decomposition once. (This assumes sequential solution; if you have all the right hand sides at the same time, you can use block Krylov methods.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, these are just very rough guidelines: For any of the above statements, there likely exists a matrix for which the converse is true...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you asked for references in the comments, here are some textbooks and review papers to get you started. (Neither of these - nor the set - is comprehensive; this question is much too broad, and depends too much on your particular problem.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://books.google.com/books?id=X5YfsuCWpxMC&quot;&gt;Golub, van Loan: &lt;em&gt;Matrix Computations&lt;/em&gt;&lt;/a&gt; (still the classical reference on matrix algorithms; the much expanded fourth edition now also discusses sparse matrices and has Matlab code in place of Fortran as well as an extensive bibliography)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ec-securehost.com/SIAM/FA02.html&quot;&gt;Davis: &lt;em&gt;Direct Methods for Sparse Linear Systems&lt;/em&gt;&lt;/a&gt; (a good introduction on decomposition methods for sparse matrices)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.99.4466&quot;&gt;Duff: &lt;em&gt;Direct Methods&lt;/em&gt;&lt;/a&gt; (review paper; more details on modern &quot;multifrontal&quot; direct methods for sparse matrices)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot;&gt;Saad: &lt;em&gt;Iterative methods for sparse linear systems&lt;/em&gt;&lt;/a&gt; (the theory and - to a lesser extent - practice of Krylov methods; also covers preconditioning)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-12-17T18:33:44.070" LastActivityDate="2015-12-17T18:33:44.070" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3303" PostTypeId="2" ParentId="3301" CreationDate="2012-09-16T13:15:30.600" Score="6" Body="&lt;p&gt;I'm not sure what is your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alpha shape code starts with a delaunay tessellation (at least mine does.) It erodes any simplex that fails the alpha shape criterion, one simplex at a time. What remains is still a tessellation of the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I would find the outer boundary of that complex, by listing all facets (in 2-d, these are simply edges) of each simplex. These edges are defined by pairs of indexes into the list of vertices. Look for duplicate facets, and exclude all edges that appear twice in the list. (Make sure that you catch edge [1,2] as a duplicate of edge [2,1].) What remains in the list must be only those on the outer surface since they are unshared facets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus in 2-d, we now have a list of edges that comprise the enclosing polygon, which may be concave depending on the value of alpha to generate that alpha shape. Connect the edges into a polygon, traversing one to the next in order. Again, since the list of edges is really a list of references into the list of vertices, that ordering will be simple. If your choice ends up the counter-clockwise polygon, flip them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps the only way to explain this is by an example. I'm using my own set of tools to compute the alpha shape, etc., but the ideas are the same no matter what tools you use. All computations were done in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll start with 121 lattice points in the plane, then exclude a few, by taking a bite out of one edge, thus excluding 34 of those points.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xy = lattice({0:.1:1,0:.1:1})&#xA;xy =&#xA;            0            0&#xA;            0          0.1&#xA;            0          0.2&#xA;            0          0.3&#xA;            0          0.4&#xA;            0          0.5&#xA;            0          0.6&#xA;            0          0.7&#xA;            0          0.8&#xA;            0          0.9&#xA;            0            1&#xA;          0.1            0&#xA;          0.1          0.1&#xA;&#xA;...&#xA;&#xA;            1          0.7&#xA;            1          0.8&#xA;            1          0.9&#xA;            1            1&#xA;&amp;gt;&amp;gt; k = sqrt(sum(bsxfun(@minus,[.8 .5],xy).^2,2)) &amp;lt; .35;&#xA;&amp;gt;&amp;gt; sum(k)&#xA;ans =&#xA;    34&#xA;&amp;gt;&amp;gt; xy(k,:) = [];&#xA;&amp;gt;&amp;gt; plot(xy(:,1),xy(:,2),'o')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zbCDF.jpg&quot; alt=&quot;points&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I can form a triangulation of that set. Again, I'll use my own tools, but in matlab, delaunay.m does exactly this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; tri = delaunayn(xy)&#xA;ans =&#xA;    84    79    83&#xA;    86    80    85&#xA;    75    79    84&#xA;    84    66    75&#xA;    75    66    71&#xA;    85    80    76&#xA;    76    67    85&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The tessellation itself is simply a set of references. Each row of that array is one triangle, so we have one of the triangles as [84 79 83], which are references into those points from our original point set. The triangulation is like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/IlkWh.jpg&quot; alt=&quot;delaunay&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the long facets that span the hole along the right edge. Now, compute an alpha shape. The maximum nearest neighbor distance is 0.1, so here I'll use an alpha ball of radius 0.2 (twice the maximum nearest neighbor distance.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; sc = alphashape(xy,.2);&#xA;&amp;gt;&amp;gt; tria = sc.tessellation;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/qzDxk.jpg&quot; alt=&quot;alphashape&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the alpha shape is in yellow, with black edges for its triangles. I've overlaid the delaunay triangulation on top, with blue edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alpha shape has 130 triangles in it, since each triangle has 3 edges, I will have a list of 130x3 = 390 edges, some of which are replicates. There really are only 216 edges in that list.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; edgelist = [tria(:,[1 2]);tria(:,[1 3]);tria(:,[2 3])];&#xA;&amp;gt;&amp;gt; size(unique(sort(edgelist,2),'rows'))&#xA;ans =&#xA;   216     2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What we wish to see however, are only the edges on the boundary of that alpha shape. These edges are those that appear in the list exactly once. Some careful sorting can resolve the boundary edges.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; sortrows(sort(edgelist,2))&#xA;ans =&#xA;     1     2&#xA;     1    12&#xA;     2     3&#xA;     2    12&#xA;     2    12&#xA;     2    13&#xA;     2    13&#xA;     2    14&#xA;     2    14&#xA;     3     4&#xA;     3    14&#xA;     3    14&#xA;     3    15&#xA;     3    15&#xA;     4     5&#xA;     4    15&#xA;     4    15&#xA;     5     6&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thus we see that edge [1 2] appeared once in the list, but edge [2 12] shows up twice, as does edge [2 13]. These latter edges are interior edges that we wish to remove if we want to see the boundary. In MATLAB, I'd do it like this to remove those that appeared twice in the list:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; k = find(all(diff(sortrows(sort(edgelist,2))) == 0,2));&#xA;&amp;gt;&amp;gt; edgelist(k,:) = []&#xA;edgelist =&#xA;    79    83&#xA;    80    85&#xA;    75    79&#xA;    76    80&#xA;    50    51&#xA;    60    61&#xA;    49    50&#xA;    67    68&#xA;    58    59&#xA;    78    82&#xA;     1     2&#xA;     2    12&#xA;    10    21&#xA;    10    20&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;42 edges remain in the 1-manifold representing the boundary polygon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/kxgim.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still am not sure where you got stuck, but that should clear up some issues. Just use those 42 edges as your polygon, connecting the dots. How you should do this in your favorite language is nothing I can solve, since I don't even know what that is.&lt;/p&gt;&#xA;" OwnerDisplayName="user840" LastEditorDisplayName="user840" LastEditDate="2012-09-18T18:09:08.087" LastActivityDate="2012-09-18T18:09:08.087" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="3304" PostTypeId="2" ParentId="160" CreationDate="2012-09-16T16:38:26.897" Score="2" Body="&lt;p&gt;I wanted to chime in to say that, I've recently gone through this process with a simulation and I should note that using Fortuna is not out of the question if it is really necessary.  In our case, we were concerned that MT's entropy wasn't high enough which would translate in our simulation to a bias.  So for our simulation we used Fortuna pulling about 65 billion random numbers from that algo.  Point being, computers are fast, if you &lt;em&gt;really need to&lt;/em&gt; you can use it if you have a reason.  If you are just doing something like a monte carlo integration, stick with MT.&lt;/p&gt;&#xA;" OwnerUserId="2045" LastActivityDate="2012-09-16T16:38:26.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3305" PostTypeId="2" ParentId="3283" CreationDate="2012-09-17T06:48:56.920" Score="10" Body="&lt;p&gt;The most significant reference I know of is David Stewart's thesis, which is more than 20 years old:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1017/S000497270002829X&quot;&gt;High Accuracy Numerical Methods for Ordinary Differential Equations with Discontinuous Right-hand Side&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The abstract references several significant earlier works.  A keyword here is &lt;strong&gt;differential inclusion&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-09-17T06:48:56.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3306" PostTypeId="1" AcceptedAnswerId="3627" CreationDate="2012-09-17T08:29:12.250" Score="8" ViewCount="598" Body="&lt;p&gt;I'm trying to learn more about volume meshing and have decided to try to implement a simple volume mesher. The strategy I have chosen is to subdivide my space using an octree, refined based on some criteria. I have made sure that my octree is balanced, such that the difference between two neighbouring cells is no more than 1 (the so-called 2:1 rule).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now have to cut my cells into tetrahedrons. I have read that Body Centered Cubic (BCC) is a popular way to do this but unfortunately I have not found any literature that explains well how to handle differences between two neighbouring cubes of different levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also seen that cutting cubes into 6 tetras as shown below is common, but again I have found nothing explaining how to handle cubes of different levels:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ics.uci.edu/~eppstein/projects/tetra/sixcube.gif&quot;&gt;http://www.ics.uci.edu/~eppstein/projects/tetra/sixcube.gif&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate it if someone could explain how Octree cubes should be cut into tetras or perhaps refer me to a paper/presentation about this subject. &lt;/p&gt;&#xA;" OwnerUserId="2047" LastActivityDate="2012-11-05T14:57:27.563" Title="Octree cubes to tetrahedrons" Tags="|finite-element|mesh-generation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3307" PostTypeId="1" AcceptedAnswerId="3311" CreationDate="2012-09-17T11:26:13.670" Score="2" ViewCount="213" Body="&lt;p&gt;Suppose an Euclidean distance $D\in\mathbb{R}^{n\times n}$ matrix between a set of $n$ objects is given. To obtain inner-products (which will be further be used to recover coordinates), entries of $D$ are squared, and the matrix is double-centered and scaled, ie., $K=-\frac{1}{2}JD^{(2)}J$, where matrix $J=I-\frac{1}{n}11^T$ defines the origin wrt which the inner-products are formed. So, the aim is to reconstruct coordinates $X$ that give rise to inner products $K$,&#xA;$$K=-\frac{1}{2}JD^{(2)}J=JXX^TJ.$$ This is done by eigendecomposition of $K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given  the above equation, I wonder if one could use a shortcut $-\frac{1}{2}D^{(2)}=XX^T$, ie, obtain coordinates $X$ without double-centering $-\frac{1}{2}D^{(2)}$ (matrix $J$ is removed from the left and from the right).&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-09-18T17:34:21.640" Title="Recovering coordinates by eigendecomposition without double-centering" Tags="|linear-algebra|matrix|computational-geometry|high-dimensional|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3308" PostTypeId="2" ParentId="3306" CreationDate="2012-09-17T12:49:24.297" Score="2" Body="&lt;p&gt;You can do it in two different ways:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You accept that your mesh has hanging nodes: just like the cells of your octree have a 2:1 relationship where it is refined, you can have the same kind of 2:1 relationship for tet meshes. If you do the isotropic (BCC) refinement of hexahedra into tets, then the 2:1 relationship between adjacent cells of the octree is inherited to the tets as well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you don't want hanging nodes, then you can do a postprocessing step. Assume you have done the BCC refinement, then there may be adjacent tets that differ in size by a factor of 2. You only have to refine the larger ones in such a way that they now match their smaller neighbors. How to do this is easy to understand on a piece of paper in the 2d case. In 3d it's a bit more tricky but you'll see that you can refine a tet anisotropically in such a way that it matches its neighbors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-17T12:49:24.297" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3309" PostTypeId="2" ParentId="3283" CreationDate="2012-09-17T12:56:51.497" Score="15" Body="&lt;p&gt;See &lt;a href=&quot;http://www.math.uiowa.edu/~dstewart/&quot;&gt;David Stewart&lt;/a&gt;'s new (2011) book on this topic, &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9781611970715&quot;&gt;Dynamics with Inequalities: Impacts and Hard Constraints&lt;/a&gt;. Coulomb friction problems are mentioned several times in the analysis chapters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chapter 8 is devoted to numerical methods for non-smooth ODEs and DAEs. It mostly advocates fully implicit Runge-Kutta methods with special treatment of nonsmoothness. Note Section 8.4.4 which points out that if you do not accurately locate the points of non-smoothness, all methods degrade to first order $\mathcal{O}(h)$ accuracy, therefore implicit Euler (with modifications for nonsmoothness) are popular in practice. Furthermore, solutions of problems with infinite dimensional inequalities are generally not piecewise smooth, therefore the theory provides only $\mathcal{O}(h^{1/2})$ convergence, though in practice, $\mathcal{O}(h)$ is often observed.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-09-17T12:56:51.497" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3310" PostTypeId="2" ParentId="3296" CreationDate="2012-09-17T17:03:03.433" Score="1" Body="&lt;p&gt;If you want to experiment and play with collocation method - I can suggest you to try my own code &lt;a href=&quot;http://code.google.com/p/bernstein-poly/&quot; rel=&quot;nofollow&quot;&gt;bernstein-poly&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an implementation of a collocation method using Bernstein polynomials. These are pretty interesting. I'm still working on a paper - haven't published anything yet, except at &lt;a href=&quot;http://www.laturbolenza.com/&quot; rel=&quot;nofollow&quot;&gt;my blog&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now it is intended for two-point BVP's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are also welcome to send some suggestions for improvement.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-09-17T17:03:03.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3311" PostTypeId="2" ParentId="3307" CreationDate="2012-09-18T07:44:47.133" Score="3" Body="&lt;p&gt;Your proposal will give meaningless results. This can be seen already with simple examples consisting of two points only. Indeed, the matrix that you want to decompose is not even positive semidefinite as it is nonzero but has zero diagonal entries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you can't cancel $J$ since it is singular. In general, $Ju=Jv$ implies $u=v$ if and only if $J$ has a trivial kernel, i.e., iff the rank of $J$ equals the number of columns of $J$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-18T17:34:21.640" LastActivityDate="2012-09-18T17:34:21.640" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3312" PostTypeId="1" AcceptedAnswerId="3323" CreationDate="2012-09-18T15:23:48.090" Score="8" ViewCount="1839" Body="&lt;p&gt;Is there an open-source, thread-safe implementation of convex optimization solvers in C/C++?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some libraries such as &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt&quot; rel=&quot;noreferrer&quot;&gt;NLopt&lt;/a&gt;, &lt;a href=&quot;https://projects.coin-or.org/Ipopt&quot; rel=&quot;noreferrer&quot;&gt;Ipopt&lt;/a&gt;, &lt;a href=&quot;https://software.sandia.gov/opt++/&quot; rel=&quot;noreferrer&quot;&gt;OPT++&lt;/a&gt; don't meet my requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OPT++ and Ipopt aren't thread-safe, and NLopt doesn't seem to have a specific/efficient way to solve convex programming problems with equality/inequality constraints.&lt;/p&gt;&#xA;" OwnerUserId="1118" LastEditorUserId="1118" LastEditDate="2012-09-19T12:46:59.373" LastActivityDate="2012-09-20T03:05:42.617" Title="Open-source, thread-safe implementation of convex optimization solvers in C/C++?" Tags="|optimization|c|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3313" PostTypeId="1" AcceptedAnswerId="3316" CreationDate="2012-09-18T17:01:30.493" Score="3" ViewCount="175" Body="&lt;p&gt;For study 3D instabilities problem i.e a base state (velocity field and temperature field) and an additional perturbation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method must be independent of the type of instabilities considered but as an example, let's take the well known Kelvin-Helmholtz instability. The dynamical equation for the system can be defined as the Boussinesq equation&#xA;\begin{equation}&#xA;\nabla\cdot\mathbf{u}=0&#xA;\end{equation}&#xA;\begin{equation}&#xA;\frac{d\mathbf{u}}{dt}=-\frac{1}{\rho}\nabla p + b \mathbf{n} + \nu_\mathbf{u} \Delta \mathbf{u}&#xA;\end{equation}&#xA;\begin{equation}&#xA;\frac{d b}{dt}= \nu_{b} \Delta b&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathbf{u}=(u,v,w)$ is the velocity vector, $b$ is the buoyancy proportional to the temperature, $p$ is the pressure, $\frac{d~}{dt}$ is the lagrangian derivative, $\nu_\mathbf{u}$ is the kinematic viscosity, $\nu_{b}$ is the diffusion coefficient of buoyancy, $\Delta$ is the Laplacien operator and $\nabla$ is the gradient operator. The problem is define on $[0,Lx]\times[0,Ly]\times[0,Lz]$. The problem is $Lx$ periodic.&#xA;$\mathbf{u}$ and $b$ are the computational variables. We initialize the field with &#xA;\begin{equation}&#xA;\mathbf{u}(\mathbf{x},t=0)=\mathbf{u}_B(\mathbf{x})+\mathbf{u}_P(\mathbf{x})&#xA;\end{equation}&#xA;\begin{equation}&#xA;b(\mathbf{x},t=0)=b_B(\mathbf{x})+b_P(\mathbf{x})&#xA;\end{equation}&#xA;where $\mathbf{x}=(x,y,z)$ is the spacial coordonate, $t$ is the time coordinate, the sub-script $B$ is for the base state and the sub-script $P$ is for the perturbed field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if $z&amp;gt;Lz/2$ then $u_B(\mathbf{x})=\alpha_1$ else $u_B(\mathbf{x})=\alpha_2$. if $z&amp;gt;Lz/2$ then $u_B(\mathbf{x})=\beta_1$ else $u_B(\mathbf{x})=\beta_2$. $v_B=0$ and $w_B=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically we add a perturbation $(\mathbf{u}_P,b_P)$ for accelerate the developmental of the instability. We expect the perturbed field to be Gaussian and negligible in front of the base state. By negligeable let's said $||(\mathbf{u}_P,b_P)||\approx 10^{-3}||(\mathbf{u}_B,b_B)||$. The flow is discretized on a regular structured grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to implement properly the perturbed field $(\mathbf{u}_P,b_P)$ in the physical grid space ?&lt;/p&gt;&#xA;" OwnerUserId="1383" LastEditorUserId="27" LastEditDate="2012-09-19T00:02:44.113" LastActivityDate="2012-09-19T15:26:19.660" Title="How to add perturbation to a base state in physical grid space?" Tags="|algorithms|fluid-dynamics|statistics|simulation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3314" PostTypeId="1" CreationDate="2012-09-18T17:57:06.577" Score="9" ViewCount="448" Body="&lt;p&gt;I am looking for a numerical software package to help me solve the 2-dimensional &quot;free boundary&quot; PDEs that arise in optimal stopping problems.  In one dimension a standard optimal stopping problem in Economics is when to exercise an expansion option.  In the end, solving this problem comes down to solving the following ODE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mu x u' + \frac{1}{2}\sigma^2 x^2 u'' + x k_0= r u&#xA;$$&#xA;such that &#xA;\begin{eqnarray}&#xA;u(0) &amp;amp; =&amp;amp; 0 \\&#xA;u(\overline{x}) &amp;amp; =&amp;amp;  \frac{\overline{x}k_1}{r-\mu} - p\\&#xA;u'(\overline{x}) &amp;amp; = &amp;amp;  \frac{k_1}{r-\mu}.&#xA;\end{eqnarray}&#xA;A solution to this problem is a function $u$ together with the location of the boundary $\overline{x}$ - this is the so called &quot;free boundary&quot;. This problem has a closed form solution.  In two dimensions, the problem no longer has a closed form solution.  The problem I want to solve is &#xA;$$&#xA; x k_0 + \mu x \frac{\partial u}{\partial x}+ rw\frac{\partial u}{\partial w} + \frac{1}{2} x^2 \left(\sigma_x^2\frac{\partial^2 u}{\partial x^2}  + \sigma_w^2 \frac{\partial^2 u}{\partial w^2} + 2\sigma_x\sigma_w \frac{\partial^2 u}{\partial x\partial w}\right)  = r u&#xA;$$&#xA;such that &#xA;\begin{eqnarray}&#xA;u(0,w) &amp;amp; = &amp;amp; 0\\&#xA;u(\overline{x}(w),w) &amp;amp; = &amp;amp; v(\overline{x}(w),w) - p \\&#xA;\frac{\partial}{\partial x}u(\overline{x}(w),w) &amp;amp; = &amp;amp; \frac{\partial}{\partial x}v(\overline{x}(w),w) \\&#xA;u(x,0) &amp;amp;= &amp;amp; \ell x \\&#xA;\frac{\partial}{\partial w}u(x,\overline{w}) &amp;amp; = &amp;amp; -1 \\&#xA;\frac{\partial^2}{\partial w^2}u(x,\overline{w}) &amp;amp; = &amp;amp; 0 \\&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $v$ is some known function of $x$ and $w$.  The solution of this problem will then be a function $u$ together with functions $\overline{x}(w)$ and $\overline{w}(x)$ that give the locations of the boundaries.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know of any software packages that can solve these free boundary problems?  If so, can someone point me towards a reference or tutorial that might help?&lt;/p&gt;&#xA;" OwnerUserId="2053" LastEditorUserId="9" LastEditDate="2012-09-18T18:53:57.660" LastActivityDate="2012-09-18T18:53:57.660" Title="What's a good numerical/optimization software package for solving the 2-D optimal stopping problem?" Tags="|optimization|pde|reference-request|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3315" PostTypeId="1" CreationDate="2012-09-18T22:54:08.523" Score="27" ViewCount="9622" Body="&lt;p&gt;I am looking for a tool to visualize very large directional link graphs.&#xA;I currently have ~2million nodes with ~10million edges. I have tried a few different things, but most take hours to even do 100k node graphs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have tried:&lt;br&gt;&#xA;I spent a day with gephi, but 80K nodes take about an hour to add and the application becomes mostly useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interactive visualization would be a plus.&lt;/p&gt;&#xA;" OwnerUserId="115" LastEditorUserId="115" LastEditDate="2012-09-19T03:29:35.747" LastActivityDate="2015-11-04T18:48:35.910" Title="Visualizing very large link graphs" Tags="|visualization|" AnswerCount="6" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3316" PostTypeId="2" ParentId="3313" CreationDate="2012-09-19T00:18:48.033" Score="4" Body="&lt;p&gt;My experience in these types of problems is that almost any perturbation will do. I suggest you do not try to perturb $u$ or $p$, because generating a divergence-free $u$ with a perturbed component is more complicated than necessary. You can easily perturb $b$ with lots of different things including a small random field or a randomly-distributed set of Gaussian bubbles or small sinusoids with randomly-chosen frequencies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem in my experience is that you need to choose to kick your problem with something appropriate to the physics and the boundary conditions. If the perturbation in the Kelvin-Helmholtz problem is too lopsided, you might find yourself with a very long transient while the the problem settles into a more symmetric pattern. Also, depending on the dynamics of landscape, you may land in one kind of semi-stable flow pattern rather than another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&#xA;Here's some math that might be used as initial conditions on $u$ and $b$:&#xA;$$&#xA;u_P(\mathbf{x})=0 \\&#xA;b_P(\mathbf{x})=\epsilon{\rm random}(\mathbf{x})&#xA;$$&#xA;where ${\rm random}(\mathbf{x})$ gives you a random number between 0 and 1 at each point in space, and $\epsilon$ is a small parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other cases proceed similarly.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2012-09-19T15:26:19.660" LastActivityDate="2012-09-19T15:26:19.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3317" PostTypeId="1" AcceptedAnswerId="3319" CreationDate="2012-09-19T09:52:48.950" Score="3" ViewCount="396" Body="&lt;p&gt;I want to know methods which are fully parallelizable on CUDA architecture.&#xA;I have implemented the Jacobi and Conjugate Gradient methods and now Im thinking about the Bi-Conjugate gradient method. I was using the cublas library to simplify some basic algebra operations. Someone said me that Gauss-Seidel is semi-parallelizable. I am new on CUDA, so I cannot understand some issues in the Cholesky descomposition or Gaussian-Elimination papers like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://gauss.cs.ucsb.edu/~aydin/talks/PMAA-062008-Aydin.pdf&quot; rel=&quot;nofollow&quot;&gt;Gaussian Elimination&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.umd.edu/Grad/scholarlypapers/papers/jjung.pdf&quot; rel=&quot;nofollow&quot;&gt;Cholesky Descomposition&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't understand the new algorithms, so I am saying that I have to use the classic numerical algorithms(I am not able to &quot;create&quot; that kind of algorithms), something like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is, does there exist any &quot;theory&quot; about parallelizing of those algorithms? Because it was easy for me the Jacobi parallelization and conjugate gradient methods, there was some saxpy and gemv operations inside the algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in the Cholesky's case the inner loop depends of the first, how we can solve this? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k = 1 to n&#xA;...&#xA;   for j = k + 1 to n&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;PS: I always developed the algorithms on C++ and from there I started to parallelizing process( it means the process of &quot;discovering&quot; how to parallelize )&lt;/p&gt;&#xA;" OwnerUserId="2057" LastEditorUserId="238" LastEditDate="2012-09-19T10:01:30.160" LastActivityDate="2012-09-19T12:49:44.863" Title="Parallelization of LSE solvers using CUDA" Tags="|algorithms|parallel-computing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3318" PostTypeId="1" AcceptedAnswerId="3321" CreationDate="2012-09-19T10:23:18.503" Score="9" ViewCount="10173" Body="&lt;p&gt;I have two variables &lt;code&gt;k&lt;/code&gt; and &lt;code&gt;t&lt;/code&gt; as functions of two other variables &lt;code&gt;p1&lt;/code&gt; and &lt;code&gt;p2&lt;/code&gt;. I also know their maximum values. I do not have any analytic expression for this. I want to find the values of &lt;code&gt;k&lt;/code&gt; and &lt;code&gt;t&lt;/code&gt; which are the closest to their maximum values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to optimise the &lt;code&gt;k = f1(p1, p2)&lt;/code&gt; and &lt;code&gt;t = f2(p1, p2)&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can try to check the product &lt;code&gt;k0 * t0&lt;/code&gt; or the product of the squares &lt;code&gt;k0^2 * t0^2&lt;/code&gt; or some other relation of the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this efficient and which way to go?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerDisplayName="user1639" LastEditorUserId="1804" LastEditDate="2012-09-23T10:45:05.017" LastActivityDate="2020-04-15T09:02:15.007" Title="Simultaneous maximization of two functions without available derivatives" Tags="|optimization|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3319" PostTypeId="2" ParentId="3317" CreationDate="2012-09-19T11:11:35.873" Score="3" Body="&lt;p&gt;If I understand your question correctly you are interested in the GPU implementation of direct methods for the solution of dense linear systems of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me first say that implementing efficient, accurate, and robust linear algebra routines requires considerable expertise, so your first option should be using an existing library, e.g. &lt;a href=&quot;http://icl.cs.utk.edu/magma/index.html&quot; rel=&quot;nofollow&quot;&gt;MAGMA&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in your own implementation &lt;strike&gt;for didactical purposes&lt;/strike&gt;, I would suggest you to focus on a model problem, e.g. Cholesky factorization ($A = C^T C$ where $A$ is symmetric positive definite and $C$ is upper triangular), and study the data dependencies between the $c_{ij}$ elements of $C$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will soon learn that a &quot;full&quot; parallelization is impossible, but that are several ways to lay out the computation exploiting the GPU parallelism. Googling for &quot;GPU parallel dense cholesky&quot; will show you a vast literature on the argument, so you should not pretend to catch up in a small amount of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;remember that there are several algorithmic variants of the Cholesky factorization, a good book on the subject should show you more than one, &lt;/li&gt;&#xA;&lt;li&gt;do not focus only on the computations but also on the memory layout to reduce latencies&lt;/li&gt;&#xA;&lt;li&gt;pay attention on how you implementation scales for increasing matrix sizes&lt;/li&gt;&#xA;&lt;li&gt;start by comparing simple variants, and do not try to optimize from the very first try&lt;/li&gt;&#xA;&lt;li&gt;don't be disappointed if your implementation is much less efficient than an established library, but rather study their code to learn.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As what regards CUDA, it adopts a programming model that is tailored to a very specific architecture, with less abstraction than other frameworks like OpenCL or API's like MPI. Programming has to be &quot;hardware aware&quot; to obtain good efficiency, since there is no intermediate layer, doing optimization for you. Again memory and arithmetic latency hiding is the key factor, so there is no simple theoretical result or &quot;parallelization model&quot; to invoke, as far as I know. The &quot;CUDA C Best Practices Guide&quot; is a good (and almost necessary) starting point.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-09-19T12:49:44.863" LastActivityDate="2012-09-19T12:49:44.863" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3320" PostTypeId="2" ParentId="3318" CreationDate="2012-09-19T11:20:24.387" Score="2" Body="&lt;p&gt;With multi-objective optimization, there are many ways you can combine/compare the objective variables in your analysis.  The problem is that there isn't a &quot;right&quot; way to do it.  It depends entirely on what the problem actually is and what the variables represent.  Your best bet is likely to maximize something like $k+a*t$ where $a$ is an arbitrary positive value.  Once you have an answer, see if you like the resultant $k$ and $t$, and modify $a$ as necessary until you find a solution you are happy with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the actual optimization, not having an analytical expression for the functions isn't the end of the world, but not having &lt;em&gt;any&lt;/em&gt; information is going to make it rough.  If you can assume some level of smoothness/continuity, even if it is only piece-wise, you can use a root finding algorithm on a derivative approximation to find local maxima (there are many more sophisticated methods than this, but I am unfamiliar with them.  Other people here could likely point you in the right direction).  If you can establish convexity, you could extend it to global optimality.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;True black box multi-objective optimization isn't exactly an easy problem, but a few assumptions and an iterative process with a objective reduction should get you an answer that is acceptable (assuming one exists).&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-09-19T11:20:24.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3321" PostTypeId="2" ParentId="3318" CreationDate="2012-09-19T12:07:17.770" Score="16" Body="&lt;p&gt;There are two issues here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your optimization problem has two competing objectives: maximizing &lt;span class=&quot;math-container&quot;&gt;$k=f_1(p_1,p_2)$&lt;/span&gt; and maximizing &lt;span class=&quot;math-container&quot;&gt;$t = f_2(p_1,p_2)$&lt;/span&gt;. This is known as &lt;em&gt;multi-objective&lt;/em&gt; (or &lt;em&gt;multi-criteria&lt;/em&gt;) optimization, and such problems have an infinite number of solutions, each based on a specific choice of the relative weight of the objectives (i.e., is it more important for &lt;span class=&quot;math-container&quot;&gt;$f_1$&lt;/span&gt; to be close to the maximal value than for &lt;span class=&quot;math-container&quot;&gt;$f_2$&lt;/span&gt;?). If both have the same importance for you, then you can simply minimize the function&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;F(p_1,p_2) = (f_1(p_1,p_2)-K)^2 + (f_2(p_1,p_2)-T)^2,&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; are the known maximal values of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, respectively. Otherwise you would add a corresponding weight before each term. (If the maximal values were not known, you'd instead minimize &lt;span class=&quot;math-container&quot;&gt;$-f_1^2-f_2^2$&lt;/span&gt;.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To find a minimizer of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;, you can only use function values of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; at a given point &lt;span class=&quot;math-container&quot;&gt;$(p_1,p_2)$&lt;/span&gt;. This is known as &lt;em&gt;derivative-free&lt;/em&gt; optimization; see, e.g., &lt;a href=&quot;http://www.ec-securehost.com/SIAM/MP08.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Derivative-Free Optimization&lt;/a&gt; by Conn, Scheinberg and Vicente or chapter 9 in &lt;a href=&quot;http://books.google.at/books?id=eNlPAAAAMAAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Optimization&lt;/a&gt;. The most of these use approximate derivatives based on finite differences or derivatives of interpolating functions. Since &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is a function of only two variables, building finite difference approximations of the full Hessian is not too expensive (or unstable). The idea is the following: given a point &lt;span class=&quot;math-container&quot;&gt;$p^k=(p_1^k,p_2^k)$&lt;/span&gt;, you construct a local quadratic model&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;m_k(p^k + d) = F(p^k) + (g^k)^T d + \frac{1}{2} d^TH^kd,&#xA;$$&lt;/span&gt;&#xA;compute its minimizer &lt;span class=&quot;math-container&quot;&gt;$d^k$&lt;/span&gt; and set &lt;span class=&quot;math-container&quot;&gt;$p^{k+1} = p^k+d^k$&lt;/span&gt;. Here, for a small (but not too small, see below) &lt;span class=&quot;math-container&quot;&gt;$\epsilon&amp;gt;0$&lt;/span&gt;,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;g^k = (g_1,g_2)^T,\quad g_i = \frac{F(p^k+\epsilon e_i)-F(p^k-\epsilon e_i)}{2\epsilon}&#xA;$$&lt;/span&gt; &#xA;with &lt;span class=&quot;math-container&quot;&gt;$e_1 = (1,0)^T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$e_2 = (0,1)^T$&lt;/span&gt;, is the approximate gradient&#xA;and &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H^k = \begin{pmatrix}h_{11}&amp;amp;h_{12}\\h_{21}&amp;amp;h_{22}\end{pmatrix},&#xA;\quad&#xA;h_{ij} = \frac{F(p^k+\epsilon e_i + \epsilon e_j) - F(p^k + \epsilon e_i) - F(p^k+\epsilon e_j) + F(p^k)}{\epsilon^2}&#xA;$$&lt;/span&gt;&#xA;is a Taylor approximation of the Hessian. This requires evaluating &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; at 5 additional points in every iteration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important issue in any finite difference approximation is the choice of &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;: If it is too large, you have a poor approximation of the derivative; if it is too small, you run into the danger of cancellation and hence numerical instability. A good rule of thumb is to take &lt;span class=&quot;math-container&quot;&gt;$\epsilon = {u}^{1/3}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is the unit roundoff (about &lt;span class=&quot;math-container&quot;&gt;$10^{-16}$&lt;/span&gt; for double precision).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, you would want to combine this with a trust-region strategy, where you would require &lt;span class=&quot;math-container&quot;&gt;$d^k$&lt;/span&gt; to inside a ball whose radius you adapt during the iteration (see the books mentioned above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A comparison of algorithms and implementations for derivative-free optimization can be found on this &lt;a href=&quot;http://archimedes.cheme.cmu.edu/?q=dfocomp&quot; rel=&quot;nofollow noreferrer&quot;&gt;webpage&lt;/a&gt;, which accompanies the paper &quot;Derivative-free optimization: A review of algorithms and comparison of software implementations&quot; by Luis Miguel Rios and Nikolaos V. Sahinidis&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2020-04-15T09:02:15.007" LastActivityDate="2020-04-15T09:02:15.007" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="3322" PostTypeId="1" CreationDate="2012-09-19T13:15:13.563" Score="1" ViewCount="543" Body="&lt;p&gt;I am working on something that involves solving an advection equation $\partial{x}/\partial{t}+\vec{u}\cdot\nabla{x}=0$ in 3D. I discretized the space into 3d cartesian grid and used the Semi-Lagrangian method to solve the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By 'Semi-Lagrangian method' I mean the method that Jos Stam proposed in his paper 'Stable Fluids'(People in the computer graphics community might be familiar with him). The method states that to get the new value of $x$ at current position, we trace back in time and use the value of $x$ at the 'traced-back' position. It is based on 'the method of characteristic'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine I am advecting some objects with a velocity field and $x$ is some attribute of the objects. I update $x$ according to the advection equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have one question: What if I get the value of $x$ outside the boundary of the object while tracing back? I can't use this value to update the value of $x$ inside the object since it's undefined. Is there any method to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="1976" LastActivityDate="2012-09-19T13:15:13.563" Title="How to solve advection equation using semi-lagrangian method?" Tags="|pde|advection|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3323" PostTypeId="2" ParentId="3312" CreationDate="2012-09-20T03:05:42.617" Score="5" Body="&lt;p&gt;After I contacted &lt;a href=&quot;https://projects.coin-or.org/OBOE&quot; rel=&quot;noreferrer&quot;&gt;OBOE&lt;/a&gt;'s developers, they said &lt;a href=&quot;http://list.coin-or.org/pipermail/oboe/2012-September/000015.html&quot; rel=&quot;noreferrer&quot;&gt;it's thread-safe&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="1118" LastActivityDate="2012-09-20T03:05:42.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3324" PostTypeId="1" AcceptedAnswerId="3325" CreationDate="2012-09-20T08:43:39.303" Score="3" ViewCount="258" Body="&lt;p&gt;Is there any visualizer for parMetis (mpmetis), which can visualize FEM mesh grids after partitioning?&lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2012-09-20T12:16:49.710" Title="Looking for parMetis visualizer?" Tags="|finite-element|visualization|partitioning|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3325" PostTypeId="2" ParentId="3324" CreationDate="2012-09-20T12:16:49.710" Score="2" Body="&lt;p&gt;We used to use &lt;a href=&quot;http://www.generalmeshviewer.com/&quot; rel=&quot;nofollow&quot; title=&quot;GMV&quot;&gt;GMV&lt;/a&gt; back in the day. If you label each partition as though it was a constant material property, you can have GMV display and &quot;explode&quot; your mesh on that property. It works pretty well for modest partitions.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-09-20T12:16:49.710" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3326" PostTypeId="1" AcceptedAnswerId="3328" CreationDate="2012-09-20T23:06:12.977" Score="1" ViewCount="12261" Body="&lt;p&gt;How do you deal with data that you need to be monotonically increasing in order to work with interpolation libs and other functions, when it is in fact not monotonically increasing? &lt;/p&gt;&#xA;" OwnerUserId="291" LastEditorUserId="447" LastEditDate="2012-09-21T01:36:09.390" LastActivityDate="2013-07-11T14:19:18.020" Title="Dealing with non-monotonically increasing data" Tags="|data-management|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3327" PostTypeId="1" AcceptedAnswerId="3332" CreationDate="2012-09-20T23:15:36.293" Score="15" ViewCount="336" Body="&lt;p&gt;Say I have a linear system $A x = b$, which converges quickly using a suitable Krylov method (such as CG or GMRES) for all $b$.  If $B$ is a matrix with low rank $r$, will the same Krylov method on the system $(A + B) x = b$ also converge quickly (ideally with an extra number of iterations that roughly depends only on $r$)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of such a system would be well preconditioned membrane elasticity and bending plus unpreconditioned air pressure terms with dense outer product structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the question is the same with or without preconditioning, since $P(A + B)Q = PAQ + PBQ$ is a rank $r$ modification of $PAQ$.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2015-05-29T14:25:45.117" Title="How do low rank modifications affect Krylov method convergence?" Tags="|krylov-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3328" PostTypeId="2" ParentId="3326" CreationDate="2012-09-21T00:39:31.243" Score="5" Body="&lt;p&gt;If you want to remove the nonmonotonic structure without changing anything else, you can do an isotonic least squares fit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Isotonic_regression&quot; rel=&quot;noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Isotonic_regression&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-09-21T00:39:31.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3329" PostTypeId="2" ParentId="3326" CreationDate="2012-09-21T00:53:18.903" Score="4" Body="&lt;p&gt;I'm confused. Data needs not be monotonic to interpolate. (Yes, there are monotonic curve fitting tools, but that seems not to be what you are asking for.) For example (in matlab)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = 0:20;&#xA;y = sin(x);&#xA;xi = 0:.01:20;&#xA;yi = spline(x,y,xi);&#xA;plot(x,y,'o',xi,yi,'r-')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gh3d4.jpg&quot; alt=&quot;sine function&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps you have a problem where the data comes from a relationship that is not single valued. As an example, points that lie around the circumference of a circle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is any function at a given value of x must yield a single prediction. But if you have a relationship that is not single valued, What can you do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general idea is to create a third variable, that allows you to predict x(t) and y(t) parametrically. You really need that third piece of information, as the sequence of the points is crucial. Typically one uses the cumulative chordal arc length between the points as the parameter t. Of course, in the case of a circle, one could have used polar angle as the parameter, but for the general problem the cumulative linear arc length works well.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = rand(10,1);&#xA;y = rand(10,1);&#xA;t = [0;cumsum(diff(x).^2 + diff(y).^2)];&#xA;ti = linspace(0,t(end),1000);&#xA;xi = spline(t,x,ti);&#xA;yi = spline(t,y,ti);&#xA;plot(x,y,'o',xi,yi,'r-')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/B3jZq.jpg&quot; alt=&quot;random curve&quot;&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user840" LastActivityDate="2012-09-21T00:53:18.903" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3330" PostTypeId="2" ParentId="3326" CreationDate="2012-09-21T04:20:11.400" Score="1" Body="&lt;p&gt;You can use parametric interpolation methods to fit a curve through a sequence of given data points $(x_i,y_i)$ where $x_i$ does not necessarily have to be less than $x_{i+1}$:&#xA;&lt;a href=&quot;http://www.ams.org/journals/mcom/1997-66-217/S0025-5718-97-00796-5/&quot; rel=&quot;nofollow&quot;&gt;http://www.ams.org/journals/mcom/1997-66-217/S0025-5718-97-00796-5/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-09-21T04:20:11.400" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3331" PostTypeId="1" AcceptedAnswerId="3347" CreationDate="2012-09-21T10:21:44.000" Score="0" ViewCount="187" Body="&lt;p&gt;This question is related to recently posted one, but I guess it deserves a separate attention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose a symmetric matrix $L\in\mathbb{R}^{n\times n}$ is given, and a rectangular matrix $A\in\mathbb{R}^{n\times m}$, $m&amp;lt;n$. A solution to the system $$LAx=b, \tag 1$$ is sought, for known $b\in\mathbb{R}^n$ and unknown $x\in\mathbb{R}^m$. Since $LA\in\mathbb{R}^{n\times m}$, a solution $x$ should be calculated in a least squares sense. Now, suppose that one premultiplies the above system by $A^T$, ie, $$A^TLAx=A^Tb \tag 2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could one replace solving (1) by solving (2), or vice versa? Why couldn't one replace solving (2) by solving (1)? If a solution to (1) exists, then it is also a solution to (2). Could it be that the solution to (1) does not exits (ie, exists only in LS sense), but the exact solution to (2) exists?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-09-22T19:57:16.970" Title="Equivalence of linear systems, solving one instead of the other" Tags="|linear-algebra|matrix|linear-solver|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3332" PostTypeId="2" ParentId="3327" CreationDate="2012-09-21T10:36:48.897" Score="7" Body="&lt;p&gt;If your Krylov subspace is based on powers of $A$, convergence will be delayed by a number of iterations at most the rank of the correction. If it is based on powers of $A^TA$ then at most twice this number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't seen such a statement in the literature. But to see the validity in the first case, it is sufficient to show that the $k$th Krylov space of the matrix $A+USV^T$ where $U,V$ have $r$ columns is contained in the corresponding space without low rank corrections but with a correspondingly higher index $k+r$. This is straightforward to verify.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2015-05-29T14:25:45.117" LastActivityDate="2015-05-29T14:25:45.117" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3333" PostTypeId="1" AcceptedAnswerId="3355" CreationDate="2012-09-21T14:48:50.993" Score="11" ViewCount="7401" Body="&lt;ul&gt;&#xA;&lt;li&gt;CVXOPT: &lt;a href=&quot;http://abel.ee.ucla.edu/cvxopt/index.html&quot;&gt;http://abel.ee.ucla.edu/cvxopt/index.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;OpenOpt: &lt;a href=&quot;http://openopt.org/Welcome&quot;&gt;http://openopt.org/Welcome&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What's the relation between them?&lt;/li&gt;&#xA;&lt;li&gt;What are the advantages/disadvantages of them, respectively?&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;BTW, is there any other high quality general purpose convex optimization library for Python/C++ worth noting?&lt;/p&gt;&#xA;" OwnerUserId="2064" LastActivityDate="2012-09-22T20:16:43.747" Title="CVXOPT VS. OpenOpt" Tags="|python|convex-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3334" PostTypeId="2" ParentId="2764" CreationDate="2012-09-21T14:51:09.850" Score="1" Body="&lt;p&gt;OpenFOAM tends to follow [Ferziger &amp;amp; Peric, Computational Methods for Fluid Dynamics, Springer, 2002] so my guess is that they solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp; \text{outer iter: do $k=0,K-1$}  \\&#xA;&amp;amp; \qquad \text{inner iter: solve} \quad \left( A_k + \frac{1-\omega}{\omega} \mathrm{diag}(A_k) \right) x_{k+1} = b + \frac{1-\omega}{\omega} \mathrm{diag}(A_k) x_k \\&#xA;&amp;amp; \text{end outer iter}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upon outer-loop convergence $x_{k+1}=x_k$ so the correct system is being solved. But in your linear case, you don't have an outer iter (k=0 with initial guess $x_0=0$) so you get the solution of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left( A + \frac{1-\omega}{\omega} \mathrm{diag}(A) \right) x = b &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is only correct for relaxation parameter $\omega=1$. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;To confirm my guess, try to construct $A$ and substract $\frac{1-\omega}{\omega} \mathrm{diag}(A)$ before passing it to the linear solver.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To actually use their relaxation scheme, try to add the outer iterations (take for example K=10 and solve the linear system with a relative tolerance of 1e-2 instead of taking K=1 and solving with a relative tolerance of 1e-12)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2012-09-21T14:51:09.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3337" PostTypeId="2" ParentId="3333" CreationDate="2012-09-21T15:33:17.673" Score="4" Body="&lt;p&gt;OpenOpt is, for the most part, just an interface to other solvers.  CVXOPT is one of the open source solvers underlying OpenOpt, so unless you have a commercial solver you want to use, you'll likely want to install CVXOPT in any case.  AFAIK, CVXOPT is state-of-the-art among open source libraries, at least for second-order cone programming and semi-definite programming.&lt;/p&gt;&#xA;" OwnerUserId="944" LastEditorUserId="944" LastEditDate="2012-09-22T16:47:03.803" LastActivityDate="2012-09-22T16:47:03.803" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3338" PostTypeId="2" ParentId="3331" CreationDate="2012-09-21T18:42:58.150" Score="2" Body="&lt;p&gt;You'd look at simple cases of your statements before bothering others.&#xA;Take $2\times 1$ matrices: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A=0$ and $b\ne 0$, (1) has no solution but (2) has infinitely many.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-21T18:42:58.150" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3339" PostTypeId="1" AcceptedAnswerId="3349" CreationDate="2012-09-21T21:16:05.423" Score="7" ViewCount="1697" Body="&lt;p&gt;I'm working on modeling two phase immiscible flow in a porous medium.  When I setup the system of equations, I obtain a non-linear system of equations that can be expressed in the form:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A(x)x=b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the matrix $A(x)$ is a function of the unknown $x$.  I know that I can use Newton's method to solve this system, but it is a bit tedious to explicitly write the the derivatives of the left hand side of this equation.  Instead, I'd like to use a fixed point iteration of the form:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A(x_{n})x_{n+1}=b$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not sure that the iteration will converge to the correct solution.  I'm familiar with the contractive mapping theorem as it applies to fixed point iterations in a single dimension.  Does this theory extend into higher dimensions as well?  Are there any specific conditions on $A(x)$ that guarantee this fixed point iteration will converge to the solution?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-09-28T04:42:32.743" Title="Convergence of fixed point iterations of a non-linear matrix system" Tags="|nonlinear-equations|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3340" PostTypeId="2" ParentId="3339" CreationDate="2012-09-21T22:12:49.003" Score="8" Body="&lt;p&gt;The Banach fixed point theorem is extremely general, needing only a metric space. If the map $x \to A(x)^{-1} b$ is globally contractive, the fixed point iteration converges. For your problem (if the formulation is typical of the problem class you describe), it will likely converge linearly, with a basin of attraction that is somewhat larger than basic Newton (without line search, trust region, or continuation methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you can still write the system as a defect correction method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;  \tilde A(x_n) \delta x &amp;amp;= - (A(x_n) x_n - b) = -F(x) \\&#xA;  x_{n+1} &amp;amp;= x_n + \delta x&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(where $\tilde A = A$ at this point is still your &quot;Picard&quot; linearization). This has identical convergence properties in exact arithmetic, but has a few attractive features in practice:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;An explicit residual $F(x) = A(x) x - b$ is available for nonlinear convergence tests, thus you don't have to use the ad-hoc &quot;my solution didn't change much&quot; test which chronically misdiagnoses stagnation for convergence.&lt;/li&gt;&#xA;&lt;li&gt;The convergence tolerance of the linear solve can usually be greatly relaxed since you are only solving for the defect.&lt;/li&gt;&#xA;&lt;li&gt;It is also more natural to apply line search or trust region globalization in this context.&lt;/li&gt;&#xA;&lt;li&gt;It has the structure of a Newton or modified Newton method, thus you can incrementally enrich your approximation $\tilde A$. If you add all terms from Newton linearization, you arrive at precisely a Newton iteration.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As a practical matter, I recommend starting with a Jacobian-free Newton-Krylov method preconditioned by this Picard linearization. If you are using PETSc, just pass your Picard $A$ in both slots of &lt;code&gt;SNESSetJacobian()&lt;/code&gt;, then pass &lt;code&gt;-snes_mf_operator&lt;/code&gt;. Compare the convergence rate (which should be quadratic) with the (linear) convergence rate you see without &lt;code&gt;-snes_mf_operator&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also be aware that there are many nonlinear solution methods, most notably nonlinear GMRES and quasi-Newton, that can accommodate approximate Jacobians such as your Picard linearization. This latter class does not suffer from the finite differencing error of matrix-free finite difference Jacobian application.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-09-21T22:12:49.003" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3341" PostTypeId="1" AcceptedAnswerId="24039" CreationDate="2012-09-21T23:20:53.420" Score="10" ViewCount="3834" Body="&lt;p&gt;What is the state of the art way of implementing double precision special functions? I need the following integral:&#xA;$$&#xA;    F_m(t) = \int_0^1 u^{2m} e^{-tu^2} d u&#xA;        = {\gamma(m+{1\over 2}, t)\over 2 t^{m+{1\over 2}}}&#xA;$$&#xA;for $m=0, 1, 2, ...$ and $t&amp;gt;0$, which can be written in terms of the lower incomplete gamma function. Here is my Fortran and C implementation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/3764427&quot;&gt;https://gist.github.com/3764427&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which uses series expansion, sums up the terms until the given accuracy, and then uses recursion relations to efficiently obtain values for lower $m$. I tested it well and I get 1e-15 accuracy for all values of parameters that I need, see the Fortran version's comments for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to implement it? Here is a gamma function implementation in gfortran:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mirrors/gcc/blob/master/libgfortran/intrinsics/c99_functions.c#L1781&quot;&gt;https://github.com/mirrors/gcc/blob/master/libgfortran/intrinsics/c99_functions.c#L1781&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it is using rational function approximation instead of summing up some infinite series that I am doing. I think that's a better approach, because one should obtain uniform accuracy. Is there some canonical way to approach these things, or does one have to figure out a special algorithm for each special function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 1&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the comments, here is the implementation using SLATEC:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/3767621&quot;&gt;https://gist.github.com/3767621&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it reproduces values from my own function, roughly on the level of 1e-15 accuracy. However, I noticed a problem that for t=1e-6 and m=50, the $t^{m+{1\over2}}$ term gets equal to 1e-303 and for higher &quot;m&quot; it simply starts giving incorrect answers. My function doesn't have this problem, because I use a series expansion/recurrence relations directly for $F_m$. Here is an example of a correct value:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F_{100}$&lt;code&gt;(1e-6)=4.97511945200351715E-003&lt;/code&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I can't get this using SLATEC because the denominator blows up. As you can see, the actual value of $F_m$ is nice and small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 2&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid the above problem, one can use the function &lt;code&gt;dgamit&lt;/code&gt; (Tricomi's incomplete Gamma function), then &lt;code&gt;F(m, t) = dgamit(m+0.5_dp, t) * gamma(m+0.5_dp) / 2&lt;/code&gt;, so there is no problem with $t$ anymore, but unfortunately the &lt;code&gt;gamma(m+0.5_dp)&lt;/code&gt; blows up for $m\approx 172$. This however might be high enough $m$ for my purposes.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="236" LastEditDate="2012-09-22T20:25:28.477" LastActivityDate="2016-06-06T17:51:19.773" Title="Fast and accurate double precision implementation of incomplete gamma function" Tags="|efficiency|accuracy|special-functions|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3343" PostTypeId="2" ParentId="3341" CreationDate="2012-09-22T00:11:10.937" Score="4" Body="&lt;p&gt;I'd take a look at Abramowicz &amp;amp; Stegun's book, or the newer revision that NIST has published a couple of years ago and that's available online I believe. They also discuss ways to implement things in a stable way.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-22T00:11:10.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3346" PostTypeId="2" ParentId="3341" CreationDate="2012-09-22T09:26:35.490" Score="1" Body="&lt;p&gt;It doesn't seem to be &lt;em&gt;state-of-the-art&lt;/em&gt; but &lt;a href=&quot;http://en.wikipedia.org/wiki/SLATEC&quot; rel=&quot;nofollow&quot;&gt;SLATEC&lt;/a&gt; at Netlib offers &quot;1400 general purpose mathematical and statistical routines.&quot; The incomplete Gamma is available under the &lt;a href=&quot;http://www.netlib.org/slatec/fnlib/&quot; rel=&quot;nofollow&quot;&gt;special functions here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementing such functions is time consuming and error prone so I wouldn't do it myself unless absolutely necessary. SLATEC has been around for quite a while now and is widely used, at least based on the &lt;a href=&quot;http://www.netlib.org/master_counts2.html&quot; rel=&quot;nofollow&quot;&gt;download counts&lt;/a&gt;, so I would expect the implementation to be mature.&lt;/p&gt;&#xA;" OwnerUserId="679" LastEditorUserId="679" LastEditDate="2012-09-22T09:57:19.143" LastActivityDate="2012-09-22T09:57:19.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3347" PostTypeId="2" ParentId="3331" CreationDate="2012-09-22T10:02:55.163" Score="3" Body="&lt;p&gt;Weirdly enough, this sort of scenario can arise in orthogonal model reduction methods. In that situation, $A$ is unitary, and therefore, $A$ can take the interpretation of a basis set used for model reduction, and $A^{T}$ is a pseudoinverse (to be precise, a $\{1,2\}$-inverse, in the language of generalized inverses) of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Arnold Neumaier points out, the two sets of solutions are, in general, not the same. If $x \in \mathbb{R}^{m}$ is a solution to (1), then it will also be a solution to (2). However, the  converse of that statement is not true in the general case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, for the case where $A$ is &lt;em&gt;unitary&lt;/em&gt;, such a transformation can be quite useful. Suppose you want to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;Ly = b, \tag 0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but for whatever reason (for instance, memory, CPU time), it is impractical to solve this equation using standard dense or sparse (direct or iterative) linear algebra methods. One approach is to assume that $y \in \mathbb{R}^{n}$ is a linear combination of some set of orthonormal basis vectors $A \in \mathbb{R}^{n}$. In other words, assuming that $y = Ax$, replace $y$ with $Ax$ so that you can solve for $x \in \mathbb{R}^{m}$, which has the advantage of solving for fewer unknowns. So now you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;LAx = b, \tag 1&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as in equation (1), but this system is nonsquare, and underdetermined. It may not have a solution, because $b$ may not be in the range of $A$. A classic assumption in the model reduction literature made to obtain a square system is to assume that the quantity $LAx - b$ (you can think of it as the residual of a value of $x$ substituted into equation (1)) is orthogonal to each column of $A$, in which case, $A^{T}(LAx - b) = 0$ or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;A^{T}LAx = A^{T}b. \tag 2&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This assumption may or may not be a good one. If the solution $y$ to (0) is in the range of $A$, then solving (2) does not induce any approximation error in the computed solution to (0). Solving (0) can yield substantial computational savings compared to solving (2) directly, provided $m$ is small enough (there is an overhead of two matrix multiplies to reap savings in the linear solve).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the solution to (0) is not in the range of $A$, then solving (2) instead of (0) will only yield an approximation of the solution to (0), and the error associated with such an approximation may be substantial. Approximate solutions can be acceptable for many real-world applications (for example, problem parameters may only be known to limited precision).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ isn't unitary, all bets are off. The model reduction interpretation no longer holds, although it is still true that if $x$ is a solution to (1), it is also a solution to (2). It's possible to generalize the model reduction argument for general $A$, but then $A^{T}$ is typically replaced in that argument by another matrix, so the analogy to the question above no longer holds.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-09-22T10:02:55.163" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3348" PostTypeId="2" ParentId="3315" CreationDate="2012-09-22T11:13:50.857" Score="14" Body="&lt;p&gt;Graphviz should work. I believe that the images associated with the matrices in the &lt;a href=&quot;https://www2.research.att.com/~yifanhu/GALLERY/GRAPHS/search.html&quot;&gt;University of Florida sparse matrix collection&lt;/a&gt; were visualized using sfdp, a force-directed graph visualization algorithm developed by Yifan Hu. Most of the matrices in the collection have a computational time associated with generating a corresponding visualization, so you might be able to search for matrices whose graphs have characteristics similar to the ones you wish to visualize. For instance, &lt;a href=&quot;https://www2.research.att.com/~yifanhu/GALLERY/GRAPHS/GIF_SMALL/DIMACS10@hugebubbles-00020.html&quot;&gt;a graph with ~2.1 million nodes and ~3 million edges took Hu ~36000s to generate, or 10 hours&lt;/a&gt;. While it's not clear what hardware was used to generate the graph, it's probably a reasonable guess that a desktop or laptop was used, and the times would at least give you a rough idea of how much time rendering the graph may take. Hu's algorithm appears to be one of the state-of-the-art visualization algorithms (he published it in 2005), but not being an expert in the field, I can't speak to whether or not better algorithms exist. This algorithm is included with Graphviz as an option, and is designed to be used on large graphs such as the one you describe.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-09-22T11:13:50.857" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="3349" PostTypeId="2" ParentId="3339" CreationDate="2012-09-22T12:01:37.503" Score="5" Body="&lt;p&gt;The sequence generated by $A(x_n)x_{n+1}=b$ can be written in the form&#xA;$x_{n+1}=F(x_n):=A(x_n)^{-1}b$, and contracts locally to a solution $x^*$ iff the spectral radius $r$ of the matrix $F'(x^*)=-A(x^*)^{-1}A'(x^*)x^*$ is smaller than one. In this case, $r$ gives the rate of linear convergence. Thus your method will be fast if the $A(x_n)$ are reasonably well-conditioned and depend only weakly on $x_n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all other cases, it is probably far better if you compute a suitable preconditioner $B\approx A(x_0)$ and then solve by one of the standard methods&#xA;(e.g., Broyden's good method &lt;a href=&quot;http://en.wikipedia.org/wiki/Broyden%27s_method&quot; rel=&quot;noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Broyden%27s_method&lt;/a&gt;)  the system of equations $F(x)=0$, where $F(x):=B^{-1}(A(x)x-b)$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-22T12:01:37.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3350" PostTypeId="1" AcceptedAnswerId="3357" CreationDate="2012-09-22T13:25:05.253" Score="6" ViewCount="1892" Body="&lt;p&gt;Recently I am taking advantage of the &lt;code&gt;ode45&lt;/code&gt;, as well as &lt;code&gt;ode23s&lt;/code&gt;, &lt;code&gt;ode15s&lt;/code&gt; solvers in &lt;code&gt;Matlab&lt;/code&gt; to find numerical solutions of a system of dynamic equations (differential equations) which are developed by Lagrange method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is hard for me to get a result even though I have run the program using &lt;code&gt;ode45&lt;/code&gt; as long as 8 hours. No error appears during this process while it displays &quot;busy&quot;. I suspect the system is stiff, accordingly I have tried to solve it by &lt;code&gt;ode23s&lt;/code&gt; and &lt;code&gt;ode15s&lt;/code&gt;. However, it still keep busy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I deal with this problem? Could any expert give me some suggestions? or which other methods I could use to solve dynamic equations derived by Lagrange method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2066" LastActivityDate="2012-09-28T04:37:17.393" Title="What other method can be used to solve differential equations except ode functions in Matlab?" Tags="|matlab|ode|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3351" PostTypeId="2" ParentId="3341" CreationDate="2012-09-22T14:27:54.123" Score="5" Body="&lt;p&gt;You might take a look at &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898716349&quot; rel=&quot;noreferrer&quot;&gt;Numerical Methods for Special Functions&lt;/a&gt; by &#xA;Amparo Gil, Javier Segura, and Nico M. Temme.&lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2012-09-22T14:27:54.123" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3352" PostTypeId="2" ParentId="3350" CreationDate="2012-09-22T17:35:46.367" Score="3" Body="&lt;p&gt;You could try solvers written in a different language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within Matlab, you can have the code print something (see help odeprint), and then investigate the output. Or add options (help odeset), or shorten the time interval. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab ode solvers don't seem to have a time-limiting option, but you can easily alter the source code to return after a given number of time steps or a given cpu time difference of two tocs). Then you can check with the profiler (help profile) where the time goes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;From another comment, I gather that you use complex Mathematica-generated code. Such code is sometimes extremely unstable numerically, which might cause the solver to impose extremely small time steps. To check this, run your matlab code for a shorter and shorter time interval (in powers of 10) until it returns after at most a minute; this will give you a check on this particular potential source of trouble.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-23T18:58:37.320" LastActivityDate="2012-09-23T18:58:37.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3353" PostTypeId="2" ParentId="3331" CreationDate="2012-09-22T19:42:57.017" Score="1" Body="&lt;p&gt;Suppose $\mathop{\rm rank}(A)=m$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $L$ is &lt;em&gt;positive definite&lt;/em&gt; then the solution of $A^TLAx = A^Tb$ (2) is the least squares solution of $LAx = b$ (1), provided that the residual $r = LAx -b$ is minimized in the norm $\| r\|^2_* = r^T L^{-1}r$:&#xA;\begin{equation}&#xA;\min_x \| LAx - b \|^2_* =&#xA;\min_x (LAx-b)^TL^{-1}(LAx-b) \quad \Leftrightarrow \quad A^TLAx = A^Tb&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $L$ is &lt;em&gt;not definite&lt;/em&gt; then no simple general relation holds between the solutions of (2) and (1).&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-09-22T19:57:16.970" LastActivityDate="2012-09-22T19:57:16.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3354" PostTypeId="2" ParentId="3350" CreationDate="2012-09-22T19:49:21.307" Score="3" Body="&lt;p&gt;What tolerances and other options are you using? Are you passing any options with &lt;a href=&quot;http://www.mathworks.co.uk/help/matlab/ref/odeset.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;odeset&lt;/code&gt;&lt;/a&gt; or just using the defaults?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good start would be to increase the values for &lt;code&gt;RelTol&lt;/code&gt; and &lt;code&gt;AbsTol&lt;/code&gt; until you get a solution, and start trying to figure out, from there, what goes wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, make sure it's not your function that is hanging. You can check where all the computational time is going by using the &lt;a href=&quot;http://www.mathworks.co.uk/help/matlab/ref/profile.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;profile&lt;/code&gt;&lt;/a&gt; command.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-09-23T10:45:57.870" LastActivityDate="2012-09-23T10:45:57.870" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3355" PostTypeId="2" ParentId="3333" CreationDate="2012-09-22T20:07:45.897" Score="11" Body="&lt;p&gt;CVXOPT only solves (smooth and nonsmooth) convex problems, giving access to several third party convex solvers with guaranteed state of the art worst case complexity. You may pose linear, convex quadratic, linear semidefinite, and many other convex types of constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenOpt solves general (smooth and nonsmooth) nonlinear programs, including problems with integer constraints. Unlike CVXOPT, it has no software for solving semidefinite programs. The solvers were all written by Dmitrey Kroshko himself and don't have a long history, so testing was probably limited.&#xA;OpenOpt itself does _not- interface to general third party solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is the ''OpenOpt Framework'' (http://openopt.org/OOFramework) that interfaces to IPOPT (mentioned in another comment; it cannot solve nonsmooth problems), CVXOPT, and some other solvers available in Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from this interface, there is no relation between these packages. The methods are completely different; the only common things may well be the Python language and the access to LAPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convex problems, I'd also like to recommend&lt;BR&gt;&#xA;CVX: Matlab Software for Disciplined Convex Programming &#xA;&lt;a href=&quot;http://cvxr.com/cvx&quot;&gt;http://cvxr.com/cvx&lt;/a&gt;,&lt;BR&gt;&#xA;for which the authors obtained this year's very prestigious &#xA;Beale-Hays-Orchard Prize for Excellence in Computational Mathematical Programming. It is written in Matlab, though.&lt;BR&gt;&#xA;They also have a Python ''version'' called CVXPY&#xA;&lt;a href=&quot;http://www.stanford.edu/~ttinoco/cvxpy/&quot;&gt;http://www.stanford.edu/~ttinoco/cvxpy/&lt;/a&gt;, but CVXPY is effectively only a&#xA;very nice and useful ''disciplined'' interface to CVXOPT, in which one can make sure that when a program is syntactically correct it is a convex optimization problem.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-22T20:16:43.747" LastActivityDate="2012-09-22T20:16:43.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3356" PostTypeId="2" ParentId="3350" CreationDate="2012-09-23T03:58:53.413" Score="4" Body="&lt;p&gt;Judging from your question and the source code you posted earlier, I'm guessing that using a symplectic integrator might benefit you if you are integrating over long periods of time, because it sounds like the equations you're solving are derived from mechanics. Symplectic integrators are designed to integrate Hamiltonian systems; I don't know if they work well for Lagrangian mechanics. Also, I am not an expert on this topic; my only source for this advice is the book by Petzold and Ascher. The suggestions made by Arnold Neumaier and Pedro are also good places to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB does not include a symplectic integrator built in, so if you decide to use one, you will have to find one written in MATLAB that you trust, or implement your problem in another language (as Arnold Neumaier suggests).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your source code seems to indicate that you have 10 state variables, and from the little I could tell from your code, the right-hand side function evaluations are extraordinarily complicated for a problem of 10 state variables (compared, say, to examples I've seen in the ODE literature, which tend to be simple). My guess (without having executed your code) is that right-hand side evaluations take a non-negligible amount of time; this guess could be tested by profiling the code, although MATLAB's profiler is not always the most reliable (&#xA;see, for instance &lt;a href=&quot;https://abandonmatlab.wordpress.com/2011/06/20/profiling/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post at Abandon MATLAB&lt;/a&gt;), and profilers may not always give accurate information (sampling profilers are supposed to yield more accurate information than instrumenting profilers). If a profiler doesn't seem to help you much (i.e., it doesn't give you a good indication of why your program is taking so damn long), you could try &quot;random pausing&quot; (see &lt;a href=&quot;https://scicomp.stackexchange.com/a/2719/276&quot;&gt;this post on SciComp by Mike Dunlavey&lt;/a&gt;). I don't know how one would pull it off in MATLAB, exactly, so hopefully you won't need to resort to that tactic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judging from the way you've written your code, it seems like the way it is written could be very bug-prone (it may not be), and it may be worth checking your code by stepping it through a debugger or unit testing it. I strongly suggest writing your code in a way that makes it easier for other people to read, if only so that it's easier to debug, and thus makes it easier to exclude typos as a possible source of error, so that it's clear that the numerical methods are the primary issue causing the problems you've observed.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-09-23T03:58:53.413" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3357" PostTypeId="2" ParentId="3350" CreationDate="2012-09-23T15:40:38.330" Score="9" Body="&lt;p&gt;You seem to think that the matlab integrator is the problem and by this hypothesis you should try a different integrator. But before you do this I think it would be useful to check whether that's actually the case. For example, it may be that your user-provided function hangs. Or that it is just very slow. Or that it produces wrong results. As programmers we always tend to believe that our own code is correct but experience says that that's most of the time not true and that in fact the external code is well tested and it is our own code that has problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way you can test this by using the function you pass to ode45 is to run a few iterations of a hand-written forward or backward Euler method. This would require maybe 10 lines of code. Then do 1000 time steps and verify that it should be done in essentially no significant time. If it &lt;em&gt;does&lt;/em&gt; take significant time, you've already narrowed down the problem. When you have this working, verify that the trajectory you get for these first few time steps looks reasonable by having matlab plot them. Etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-23T15:40:38.330" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3358" PostTypeId="1" CreationDate="2012-09-23T19:53:55.323" Score="2" ViewCount="165" Body="&lt;p&gt;There is a class of stiff initial value problems for ODEs that have small Lipschitz constants, slowly-changing solutions, but very long interval of integration. The only practical example of such a problem that I am aware of is the Becker-Döring equation from Hairer and Wanner's book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there any other interesting examples of such problems?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="720" LastEditDate="2012-09-23T21:19:34.807" LastActivityDate="2012-09-24T12:25:59.043" Title="What are good examples of problems which are stiff due to very long interval of integration?" Tags="|reference-request|time-integration|stiffness|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3359" PostTypeId="2" ParentId="3358" CreationDate="2012-09-23T20:15:43.203" Score="3" Body="&lt;p&gt;A differential equation $y'=F(y)$ may be called stiff in $[0,T]$ if, for some $y$, the matrix $TF'(y)$ has some huge eigenvalues with negative real part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the simplest example is $y'=-y$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, the equation $y'=-10^6 y$ for $t\in[0,1]$ is obviously stiff, but $y'=-y$ is as stiff for $t\in[0,10^6]$. An explicit method such as Euler's needs the same number of integration steps to achieve the same accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether the example is interesting depends on your interests. But by the same argument, &lt;B&gt;all dissipative ODEs become stiff&lt;/B&gt; when integrated over huge time intervals. As all ODEs are (by definition) dissipative if the trajectory approaches a lower-dimensional stable manifold, there is no paucity of realistic examples.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-24T07:46:59.187" LastActivityDate="2012-09-24T07:46:59.187" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3360" PostTypeId="2" ParentId="3315" CreationDate="2012-09-23T21:49:32.870" Score="7" Body="&lt;p&gt;The &lt;a href=&quot;http://philogb.github.com/jit/index.html&quot; rel=&quot;nofollow&quot;&gt;JavaScript InfoVis Toolkit&lt;/a&gt; has a neat interactive interface for annotated &lt;em&gt;local&lt;/em&gt; views of graphs. These demos may be relevant to you:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://web.archive.org/web/20120428223834/http://demos.thejit.org/lpkgd/&quot; rel=&quot;nofollow&quot;&gt;Debian package dependencies&lt;/a&gt; (note: this is an archived page without scripts)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://philogb.github.com/jit/static/v20/Jit/Examples/RGraph/example3.html&quot; rel=&quot;nofollow&quot;&gt;Band tree&lt;/a&gt; (here is another &lt;a href=&quot;http://philogb.github.com/jit/static/v20/Jit/Examples/Hypertree/example1.html&quot; rel=&quot;nofollow&quot;&gt;local explorable version&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="2155" LastEditDate="2013-03-14T12:33:40.343" LastActivityDate="2013-03-14T12:33:40.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3361" PostTypeId="1" AcceptedAnswerId="3369" CreationDate="2012-09-24T02:40:21.900" Score="3" ViewCount="77" Body="&lt;p&gt;I remember seeing in the book by Kreiss &quot;Time-dependent partial differential equations and their numerical solution&quot; that if some elliptic differential operator satisfies $$(Lu,u)\leq K(u.u)$$ for the equation $u_t=Lu+f$ with some boundary condition then it can be shown that, the equation is stable, that is continuously depends on the initial data. However, when I think of $L:=u''$ and $u=-\sin(nx)$ on $(0,\pi)$ as a solution of $u_t=u''+n^2\sin(nx)$, I have boundary conditions as zero at both ends independently of $n$.&#xA;Then we can calculate&#xA;\begin{equation}&#xA;(Lu,u)=(u^{\prime\prime},u) = \int u^{\prime\prime} udx = n^2 \int \sin^2(nx)dx&#xA;\end{equation}&#xA;and &#xA;\begin{equation}&#xA;(u,u)= \int u^2dx = \int \sin^2(nx)dx &#xA;\end{equation}&#xA;Clearly, the inequality $(Lu,u)\leq K(u,u)$ doesn't hold as $n$ increases, even though second derivative is a proper elliptic operator. What am I missing here? I did not put the proof here but it is just a few lines, however the example above contradicts to the statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When does the estimate $(Lu,u)\leq K(u.u)$ hold then? &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2012-09-25T00:44:50.873" LastActivityDate="2012-09-25T00:44:50.873" Title="semiboundedness of the operator and it is affect on stability" Tags="|pde|parabolic-pde|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3362" PostTypeId="1" AcceptedAnswerId="3363" CreationDate="2012-09-24T03:39:04.527" Score="11" ViewCount="1531" Body="&lt;p&gt;I have a numerical function &lt;code&gt;f(x, y)&lt;/code&gt; returning a double floating point number that implements some formula and I want to check that it is correct against analytic expressions for all combination of the parameters &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; that I am interested in. What is the proper way to compare the computed and analytical floating point numbers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the two numbers are &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;. So far I've been making sure that both absolute (&lt;code&gt;abs(a-b) &amp;lt; eps&lt;/code&gt;) as well as relative (&lt;code&gt;abs(a-b)/max(abs(a), abs(b)) &amp;lt; eps&lt;/code&gt;) errors are less than eps. That way it will catch numerical inaccuracies even if the numbers are let's say around 1e-20.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, today I discovered a problem, the numerical value &lt;code&gt;a&lt;/code&gt; and analytic value &lt;code&gt;b&lt;/code&gt; were: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [47]: a                                                                     &#xA;Out[47]: 5.9781943146790832e-322&#xA;&#xA;In [48]: b                                                                     &#xA;Out[48]: 6.0276008792632078e-322&#xA;&#xA;In [50]: abs(a-b)                                                              &#xA;Out[50]: 4.9406564584124654e-324&#xA;&#xA;In [52]: abs(a-b) / max(a, b)                                                  &#xA;Out[52]: 0.0081967213114754103&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the absolute error [50] is (obviously) small, but the relative error [52] is large. So I thought that I have a bug in my program. By debugging, I realized, that these numbers are &lt;a href=&quot;http://en.wikipedia.org/wiki/Denormal_number&quot;&gt;denormal&lt;/a&gt;. As such, I wrote the following routine to do the proper relative comparison:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real(dp) elemental function rel_error(a, b) result(r)&#xA;real(dp), intent(in) :: a, b&#xA;real(dp) :: m, d&#xA;d = abs(a-b)&#xA;m = max(abs(a), abs(b))&#xA;if (d &amp;lt; tiny(1._dp)) then&#xA;    r = 0&#xA;else&#xA;    r = d / m&#xA;end if&#xA;end function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;tiny(1._dp)&lt;/code&gt; returns 2.22507385850720138E-308 on my computer. Now everything works and I simply get 0 as the relative error and all is ok.&#xA;In particular, the above relative error [52] is wrong, it's simply caused by insufficient accuracy of the denormal numbers. Is my implementation of the &lt;code&gt;rel_error&lt;/code&gt; function correct? Should I just check that &lt;code&gt;abs(a-b)&lt;/code&gt; is less than tiny (=denormal), and return 0? Or should I check some other combination, like&#xA;&lt;code&gt;max(abs(a), abs(b))&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would just like to know what the &quot;proper&quot; way is.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2012-09-24T15:21:42.903" Title="Relative comparison of floating point numbers" Tags="|floating-point|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3363" PostTypeId="2" ParentId="3362" CreationDate="2012-09-24T04:32:41.053" Score="11" Body="&lt;p&gt;You can directly check for denormals using &lt;code&gt;isnormal()&lt;/code&gt; from &lt;code&gt;math.h&lt;/code&gt; (C99 or later, POSIX.1 or later). In Fortran, if the module &lt;a href=&quot;http://fortranwiki.org/fortran/show/ieee_arithmetic&quot;&gt;&lt;code&gt;ieee_arithmetic&lt;/code&gt;&lt;/a&gt; is available, you can use &lt;code&gt;ieee_is_normal()&lt;/code&gt;. To be more precise about fuzzy equality, you have to consider the floating point representation of denormals and decide what you mean for the results to be good enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More to the point, to believe that either result is correct, you have to be sure that you didn't lose too many digits at an intermediate step. Computing with denormals is generally unreliable and should be avoided by having your algorithm rescale internally. To ensure that your internal scaling was successful, I recommend activating floating point exceptions using &lt;code&gt;feenableexcept()&lt;/code&gt; with C99 or the &lt;code&gt;ieee_arithmetic&lt;/code&gt; module in Fortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although you can have your application catch the signal that is raised on floating point exceptions, all kernels I have tried reset the hardware flag so &lt;code&gt;fetestexcept()&lt;/code&gt; does not return a useful result. When run with &lt;code&gt;-fp_trap&lt;/code&gt;, PETSc programs will (by default) print a stack trace when a floating point error is raised, but will not identify the offending line. If you run in a debugger, the debugger preserves the hardware flag and breaks on the offending expression. You can check the precise reason by calling &lt;code&gt;fetestexcept&lt;/code&gt; from the debugger where the result is a bitwise OR of the following flags (values may vary by machine, see &lt;code&gt;fenv.h&lt;/code&gt;; these values are for x86-64 with glibc).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;FE_INVALID = 0x1&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;FE_DIVBYZERO = 0x4&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;FE_OVERFLOW = 0x8&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;FE_UNDERFLOW = 0x10&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;FE_INEXACT = 0x20&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-09-24T04:32:41.053" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3364" PostTypeId="2" ParentId="3362" CreationDate="2012-09-24T09:01:37.803" Score="5" Body="&lt;p&gt;Donald Knuth has a proposal for a floating point comparison algorithm in volume 2 &quot;Seminumerical algorithms&quot; of &quot;The Art of Computer Programming&quot;. It was implemented in C by Th. Belding (see &lt;a href=&quot;http://fcmp.sourceforge.net/&quot;&gt;fcmp package&lt;/a&gt;) and is available in the &lt;a href=&quot;http://www.gnu.org/software/gsl/&quot;&gt;GSL&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-09-24T09:01:37.803" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3366" PostTypeId="2" ParentId="3358" CreationDate="2012-09-24T11:24:15.290" Score="3" Body="&lt;p&gt;There is the case of production and destruction of radioactive isotopes in nuclear fuel. You typically use nuclear fuel for more than 3 years (hence, you fission isotopes into radioactive daughter products that decay with certain half-lives that can go from microseconds to years or more). So during the irradiation, if you want to track all these isotopes (to get an idea on decay heat and radioactivity level), you are solving problems that have very large differences in time-constants. And then, if you need to evaluate the impact of long term storage (or even geological storage) of this spent fuel, you want to simulate this radioactive decay for $10^2$-$10^6$ years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My apologies for the self-advertisement, but you can have a look at &lt;a href=&quot;http://www.hindawi.com/journals/stni/2012/545103/&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; for more information on the application of an implicit Runge-Kutta method (RADAU-IIA) to these types of problems. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2012-09-24T12:25:59.043" LastActivityDate="2012-09-24T12:25:59.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3367" PostTypeId="1" CreationDate="2012-09-24T13:02:22.523" Score="6" ViewCount="4840" Body="&lt;p&gt;I wonder what  relation and difference are between combinatorial optimization and discrete optimization? Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Originally by reading Wikipedia, I thought discrete optimization consists of combinatorial optimization and integer optimization, where the combinatorial one is to search over a finite set of solutions, and the integer one is to search over a countably infinite set of solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But after reading the table of content of Bernhard Korte and Jens Vygen&#xA;&lt;em&gt;Combinatorial Optimization: Theory and Algorithms&lt;/em&gt;, I saw they include integer programming in their combinatorial optimization book. Now I am confused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="258" LastActivityDate="2019-06-26T17:36:48.100" Title="Relation and difference between combinatorial optimization, discrete optimization and integer programming" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3368" PostTypeId="2" ParentId="3362" CreationDate="2012-09-24T15:21:42.903" Score="5" Body="&lt;p&gt;Optimally rounded denormalized numbers may indeed have a high relative error.&#xA;(Flushing that to zero while still calling it a relative error is misleading.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But close to zero, computing relative erros is meaningless. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, even before you reach denormalized numbers, you should probably switch to absolute accuracy (namely the one you want to guarantee in this case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd therefore suggest to test the computed $y$ against the true $x$ by checking the validity of a formula such as $|y-x|\le absacc+relacc*\max(|x|,|y|)$.&#xA;say with relacc=1e-12 and absacc=1e-150.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then users of your code know precisely how much accuracy they really have. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-24T15:21:42.903" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3369" PostTypeId="2" ParentId="3361" CreationDate="2012-09-24T15:27:46.920" Score="5" Body="&lt;p&gt;I get $u''=-n^2 u$, and the condition holds with $K=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, Kreiss' conclusion holds for any linear operator; no ellipticity must be assumed.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-24T15:27:46.920" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3371" PostTypeId="1" AcceptedAnswerId="4913" CreationDate="2012-09-25T13:23:23.867" Score="5" ViewCount="459" Body="&lt;p&gt;I am using a program that utilizes the PARDISO solver as part of the Intel math kernel library.  I am currently in the process of deciding on a new computer to run the simulations on.  I have a question or two on how benchmarks with the program will transfer across systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current machine runs older dual quad-core xeons (E5345's) at 2.3GHz with DDR2 RAM at 667 MHz in 2 channels.  On this system, running with 8 threads, the program in memory bound with small cases (3000 cells or so, about 1/3 the usual size) running at about 3/4 of the cpu cap and larger cases bottoming out around 1/2 of the cpu cap.  I don't have any way to benchmark it on a more representative system before buying the new one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can very easily benchmark my code on the smaller case on 1, 2, 4, and 8 cores on my current machine, which is what I am currently doing.  I will use that information to decide between less faster cores or more slower cores in the future computer.  The issue is that in going to the new computer will be upgrading to RAM at 1333 MHz, in 4 channels, and the sandy bridge architecture.  I have heard that the sandy bridge architecture is very strong in memory-throughput benchmarks, but have not be able to locate any benchmarks comparing it to older architectures.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I have any reason to expect a change from memory boundedness to cpu boundedness of my model as I upgrade to 12 or 16 cores of comparable clock speeds with the faster RAM and architecture?  I understand this is likely hard to answer due to a high level of model dependence of the solvers performance but I am hoping someone has experience benchmarking across architectures and can give some insight as to what to expect.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-12-27T20:05:02.467" Title="Effects of memory speed/architecture on Pardiso scaling" Tags="|parallel-computing|benchmarking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3372" PostTypeId="1" AcceptedAnswerId="3376" CreationDate="2012-09-25T13:43:37.477" Score="6" ViewCount="3820" Body="&lt;p&gt;I implemented downhill simplex simulated annealing algorithm. Algorithm is very hard to tune, w.r.t. parameters including cooling schedule, starting temperature... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first question is about convergence. In general, independently from choice of initial parameters, is it sure that algorithm converges to global optimal solution?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely: I use SA with downhill simplex based on Num.recipes, however, I am adding some specific actions for observing constraints. Do you think my additional conditions for constraints are cause for bad convergence? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I would like your advice on how to go on with solver conception: for instance, my algo is tested on data in 10-D space (10 parameters). It goes as follows: a preprocessing (monte carlo) picks up a good starting point. Then I run simplex downhill --&gt; does not converge. I run simulated annealing (with simplex downhill to go to next step), and it remains far to solution (RMSE == 0.001). Excel solver on these data does the job very good. Excel method is generalized reduced gradient. Should i go on with fine tuning for SA, or go to non-derivative-free methods s.a. GRG ?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, to speak the same language: with my example, convergence is not reach AT ALL. i don't have time to reach a local minimum either, since I have put some exit conditions when the simplex gets stuck. what does one mean in general with 'solver does not converge'? does it mean that termination condition (s.a. go below a tolerance value) is met and solution is sub optimal, or can it mean that termination condition is never met?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks &amp;amp; regards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My objective function is given explicitly. It can, for instance, look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F(p) = \sum_{j=1}^{n}\Bigg(y_{j} - (p_{1} + p_{2}\exp{\Big(-\frac{(x_{i}-p_{3})^{2}}{p_{4}}\Big)} + p_{5}\exp{\Big(-\frac{(x_{j}-p_{6})^{2}}{p_{7}}\Big)} + p_{8}\exp{\Big(-\frac{(x_{j}-p_{9})^{2}}{p_{10}}\Big))}\Bigg)^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so, the squared residuals with a 'three Gaussians + step' model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an example, solver should solve for any combination of functions among gaussians, exponential, power, polynomial, linear, rational with 'reasonable' number of parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Box constraints on parameters are specified, with form $l_{i} \leq p_{i} \leq u_{i}$.&lt;/p&gt;&#xA;" OwnerUserId="1184" LastEditorUserId="20688" LastEditDate="2019-06-22T22:44:43.940" LastActivityDate="2019-06-22T22:44:43.940" Title="Simulated Annealing proof of convergence" Tags="|optimization|algorithms|iterative-method|stochastic|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3373" PostTypeId="2" ParentId="3372" CreationDate="2012-09-25T13:49:26.860" Score="4" Body="&lt;p&gt;No, it is not.  Very few methods are provably convergent for nonconvex optimization problems.  If you're looking for such a method, you might look into branch-and-bound methods.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-09-25T13:49:26.860" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3374" PostTypeId="2" ParentId="3367" CreationDate="2012-09-25T14:20:13.997" Score="4" Body="&lt;p&gt;The three topics are roughly the same (as one can reformulate the same problem in many ways), just looked at from different perspective, and hence typically treating different parts of theory and practice in different depth.&#xA;&lt;UL&gt;&#xA;&lt;LI&gt;Combinatorial optimization emphasizes the combinatorial origin, formulation or solution algorithm of a problem.&#xA;&lt;LI&gt;Discrete optimization emphasizes the difference to continuous optimization.&#xA;&lt;LI&gt;Integer programming emphases the usage of integer (or binary integer)-valued variables in formulation or solution.&#xA;&lt;/UL&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-25T14:20:13.997" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3375" PostTypeId="2" ParentId="3371" CreationDate="2012-09-25T14:27:05.597" Score="3" Body="&lt;p&gt;My stance on this is that you are spending your time on the wrong topic. The difference between fast and slow cores is going to be 10 or 15% in raw FPU capacity, but in practice you may see a 5% difference. The same is going to be true for the memory architecture. In other words, unless you do incredibly long computations, you are likely not going to see a lot of difference between cores of different speeds if you have the same number of cores. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I find that more cores are more better (TM) because they can be used in other contexts as well -- e.g. in compiling stuff in parallel. Finally, you may simply want to invest the time in profiling your code to see where there are bottlenecks: finding a spot where you can save 5% of the overall run time of a program is not usually very difficult.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-25T14:27:05.597" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3376" PostTypeId="2" ParentId="3372" CreationDate="2012-09-25T15:46:35.220" Score="9" Body="&lt;p&gt;There is a theorem that syas that a black box algorithm is guaranteed to find the global minimum of an arbitrary smooth (i.e., twice continuously differentiable) function if and only if it samples points densely in the search space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here dense is meant in the topological sense, i.e., it must sample points in arbirarily small neighborhoods of every point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the worst case complexity is &#xA;$O(\Big(\frac{d}{\delta}\Big)^{-n})$, [&lt;I&gt;the Latex parser chokes with nested big brackets&lt;/I&gt;] where $d$ is the diameter of the search space and $\delta$ the guaranteed error in $x$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: While this looks like being the worst case complexity for locating an $\delta$-accurate $x$ rather than for locating a point for a $\epsilon$-accurate $f$, the complexity for the latter is as bad (even for the rather big value of $\epsilon=(f_{first}-f_{global})/2$), as one can easily construct a smooth function which interpolates all data points so far but takes much smaller values in the center of the largest open ball not containing one of the points evaluated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus guaranteed convergence to a global minimum is worthless in practice. &#xA;(For example, uniformly random search has a convergence guarantee to the global minimizer, whereas most practically fast algorithms don't have one.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that simulated annealing usually performs much worse than modern methods. &#xA;Rather use code recommended in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;A HREF=&quot;http://archimedes.cheme.cmu.edu/?q=dfocomp&quot; rel=&quot;noreferrer&quot;&gt;&#xA;Comparison of derivative-free optimization algorithms&lt;/A&gt; &#xA;(2012, by Nick Sahinidis)&#xA;&lt;P&gt;&#xA;&lt;A HREF=&quot;http://coco.gforge.inria.fr/doku.php?id=bbob-2012&quot; rel=&quot;noreferrer&quot;&gt;&#xA;Black-Box Optimization Benchmarking (BBOB) 2012&lt;/A&gt;&#xA;(by Auger, Hansen, et al.)&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-09-26T13:40:40.250" LastActivityDate="2012-09-26T13:40:40.250" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3377" PostTypeId="2" ParentId="3372" CreationDate="2012-09-25T22:36:48.657" Score="2" Body="&lt;p&gt;Do you have box constraints, or do your constraints at least define a convex region? If not, they certainly can cause bad convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked up SA with downhill simplex in my copy of Num.recipes. This is not a normal SA implementation, and probably behaves more like an improved downhill simplex than like an &quot;improved&quot; SA implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my own experience, the convergence of a simple SA implementation can be surprisingly robust, even so you will probably need at least 100000 steps for 10 parameters. A simple SA implementation would randomly select an axis, and then do a &quot;small&quot; random move along that axis. My own attempts to do something more clever than this always slowed down convergence significantly or even completely ruined it. Perhaps something similar happens for SA with downhill simplex, even so it should still converge better than a pure downhill simplex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check for convergence, I found it helpful to always do three independent SA optimization runs, and see whether they produce similar results. I have to admit that most of the time when the three runs produced significantly different results, I actually had a bug somewhere in my implementation of the target function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The (implicit) suggestion of Arnold Neumaier to try more than one optimization algorithm on your problem (for example from the DAKOTA project) is probably also a good idea. But these algorithms also have tuning parameters, so being familiar with the real world behavior of some optimization algorithms and their tuning parameters still seems useful.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2012-09-25T22:52:20.000" LastActivityDate="2012-09-25T22:52:20.000" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3378" PostTypeId="1" CreationDate="2012-09-26T00:02:12.127" Score="7" ViewCount="862" Body="&lt;p&gt;I have been reading Numerical recipes about how to create a power series approximation to a function once you have a Chebyshev approximation to the function. However it is still very unclear to me how one can go from the coefficients $c_k$ in $\Sigma c_k T_k - c_0/2 $ to the $d_k$ in $\Sigma d_k x_k$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand just how one can get the $c_k$'s and I am very confident that I can get those correctly, but I'm getting confused about how to implement this so called Clenshaw recurrence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks. &lt;/p&gt;&#xA;" OwnerUserId="1970" LastActivityDate="2012-09-26T08:07:48.660" Title="Polynomial Fitting from Chebyshev Coefficients" Tags="|approximation|polynomials|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3379" PostTypeId="2" ParentId="3378" CreationDate="2012-09-26T08:07:48.660" Score="6" Body="&lt;p&gt;One way to get the $d_k$ is to expand each Chebyshev polynomial in powers, and then take linear combinations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, using the Chebyshev representation itself (as detailed below) is numerically far more stable than using the power series expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can evaluate a Chebyshev sum at any pareticular $x$ in a Horner-like fashion by using the 3-term recurrence to eliminate recursively the highest term until you end up with a linear function.&#xA;By differentiating the resulting recurrence you also get the values of the first derivative.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-09-26T08:07:48.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3380" PostTypeId="2" ParentId="3372" CreationDate="2012-09-26T14:13:35.533" Score="2" Body="&lt;p&gt;You're probably better off switching to a deterministic method instead of simulated annealing. Provided you can import the data to a text file, you can learn the basics of the algebraic modeling language &lt;a href=&quot;http://www.gams.com/&quot; rel=&quot;nofollow&quot;&gt;GAMS&lt;/a&gt; without too much effort, and use the &lt;a href=&quot;http://archimedes.cheme.cmu.edu/baron/baron.html&quot; rel=&quot;nofollow&quot;&gt;BARON&lt;/a&gt; solver to attempt to solve your problem. BARON is a very good (though not foolproof) nonconvex nonlinear programming solver developed by Nick Sahinidis and Mohit Tawarmalani. Ten decision variables and box constraints should still be within the capabilities of a free GAMS license, but if that is not the case, you can submit a GAMS job (using the BARON solver) on the &lt;a href=&quot;http://www.neos-server.org/neos/&quot; rel=&quot;nofollow&quot;&gt;NEOS optimization server&lt;/a&gt; and obtain results that way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stochastic optimization solvers, while useful in some circumstances, can be very finicky when it comes to tuning parameters. You may even be better off attempting to solve your least-squares problem with a traditional convex or least-squares solver; even though it may get stuck in a local minimum, using judicious initial guesses or multistart could give you a sufficiently good solution for your purposes.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-09-26T14:13:35.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3381" PostTypeId="1" AcceptedAnswerId="3386" CreationDate="2012-09-26T18:04:58.793" Score="2" ViewCount="525" Body="&lt;p&gt;I'm working on a problem given in Nocedal &amp;amp; Wright's Numerical Optimization 2nd Edition, pg 303 Exercise 11.7:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider a line-search newton method in which the step length&#xA;  $\alpha_k$ is chosen to be the exact minimizer of the merit function&#xA;  $f(\cdot)$; that is,  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$\alpha_k = argmin_\alpha [f(x_k-\alpha J^{-1}(x_k)r(x_k))]$$  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Show that if $J(x)$ is non-singular at the solution $x^*$, then&#xA;  $\alpha_k\rightarrow 1$ as $x_k\rightarrow x^*$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this problem, we seek $x^*$ such that $r(x^*)=0$, where $r:R^n\rightarrow R^n$ and use a merit function $f(x)$ to globalize the convergence of newton's method with a line search.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to work this proof out on my own, but I've run into some difficulties.  First of all, I understand that the newton direction $\Delta x_k =-J^{-1}(x_k)r(x_k)$ is a descent direction.  Also, if we (theoretically) chose the step length $\alpha_k$ as above, then the new step should satisfy the sufficient decrease condition (Armijo condition):  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_k+\alpha_k\Delta x)\le f(x_k)+c\alpha_k\nabla f(x_k)^T\Delta x$$ for some $0&amp;lt;c\le1$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that in practice, we try using the full newton step $\alpha_k=1$ first.  If the newton step doesn't produce sufficient decrease, then we search back until sufficient decrease is met.  I'm thinking that the fact that the jacobian being non-degenerate implies that there exists a ball around $x^*$ such that the full newton step always satisfies armijo's condition.  Thus, as we get closer to the root, the newton step is enough to ensure sufficient decrease.  However, I'm not entirely sure that such a ball exists.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried assuming that $\alpha_k\rightarrow a\ne1$ as $x_k\rightarrow x^*$, but the search for a contradition is somewhat mysterious to me.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help with this would be greatly appreciated! :)&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-09-27T17:37:34.463" Title="Convergence of step-length in a globally-convergent newton line search method with non-degenerate Jacobian" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3382" PostTypeId="2" ParentId="2498" CreationDate="2012-09-26T21:02:42.920" Score="10" Body="&lt;p&gt;I have recently been working on exactly this topic. You may want to take a look at our paper: &lt;a href=&quot;http://arxiv.org/abs/1209.2364&quot;&gt;http://arxiv.org/abs/1209.2364&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why are you interested in the runtime prediction of linear algebra routines? Do you intend to use the model for a certain purpose?&lt;/p&gt;&#xA;" OwnerUserId="2086" LastActivityDate="2012-09-26T21:02:42.920" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3383" PostTypeId="1" CreationDate="2012-05-04T11:49:16.683" Score="6" ViewCount="9060" Body="&lt;p&gt;For the numerical solution of &lt;a href=&quot;http://en.wikipedia.org/wiki/Reynolds-averaged_Navier%E2%80%93Stokes_equations&quot; rel=&quot;nofollow&quot;&gt;Reynolds equations&lt;/a&gt; (a non-linear partial differential equation), the Newton-Raphson method is generally proposed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After getting algebraic equations from a finite difference discretization, the Newton-Raphson method is applied to those non-linear algebraic equations. This yields a square system of linear equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a two-dimensional mesh with $m\times n$ grid points ($m\neq n$) system, is it necessary to have $n&amp;gt;m$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how do we solve those linear equations?  &lt;/p&gt;&#xA;" OwnerDisplayName="P. S." LastEditorUserId="1804" LastEditDate="2012-09-27T14:13:00.373" LastActivityDate="2012-09-27T14:13:00.373" Title="Newton-Raphson method for nonlinear partial differential equations" Tags="|pde|nonlinear-equations|newton-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3385" PostTypeId="2" ParentId="3383" CreationDate="2012-09-27T09:57:43.833" Score="7" Body="&lt;p&gt;For solving nonlinear partial equations, say $-\Delta y + f(y) = 0$, there are two common approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Fixed point iteration&lt;/em&gt;: Pick $y^0$ and for $k=1,\dots$ solve&#xA;$$ -\Delta y^{k+1} + f(y^k) = 0.$$&#xA;(That is, you replace $y$ by $y^{k+1}$ in all linear terms and by $y^k$ in all nonlinear terms.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Newton's method&lt;/em&gt;: Pick $y^0$ and for $k=1,\dots$ solve&#xA;$$ -\Delta \delta y + f'(y^k)\delta y = -(\Delta y^k+f(y^k)) \tag{1} $$ &#xA;and set $y^{k+1} = y^k + \delta y$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which to use depends on the nonlinear term $f$; in the context of Reynolds equation, $f$ is quadratic, and this means that Newton's method is a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to the main question: How to actually compute $\delta y$. The easiest (and, at least for me, least error-prone way) is to apply your finite-difference discretization to $(1)$ and solve the discretized problem like you would any linear PDE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, let's say you already have a discretization of the nonlinear problem and want to solve the resulting nonlinear finite-dimensional system. From your question, I take it that you have a two-dimensional domain which you have discretized using $N:=mn$ grid points, resulting in matrices $K_h\in\mathbb{R}^{N\times N}$ (representing the linear part) and $F_h(y_h)\in\mathbb{R}^{N\times N}$ (representing the nonlinear part), where $y_h\in\mathbb{R}^N$ is the discrete solution. Since the dimensions only depend on the product $N$, this means that Newton's method does not care whether $n&amp;gt;m$ or not (your discretization will, of course).&#xA;Applying Newton's method to $K_h y_h + F_h(y_h) = 0$, you have to solve for $\delta y_h \in \mathbb{R}^N$ in&#xA;$$(K_h + \nabla F_h(y_h^k)) \delta y_h = - K_h y_h^k - F_h(y_h^k),$$&#xA;where $\nabla F_h(y_h)\in\mathbb{R}^{N\times N}$ is the Jacobi matrix of $F_h$ with entries $(\nabla F_h(y))_{ij} = \frac{\partial F_h(y_h)_i}{\partial (y_h)_j}$. This is a standard linear system, which you can solve with any of the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3262&quot;&gt;usual methods&lt;/a&gt;. There is one additional point: If you pick an iterative method, you can exploit the fact that for $y^k$ far from the solution $y$, your linearization error will be large and likely dominate the approximation error, and thus there is no need to solve the linear system very accurately in the beginning. (Of course, you then need to increase the accuracy as you get closer to $y$.) This is known as an &lt;em&gt;inexact Newton(-Krylov) method&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-09-27T12:07:42.447" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3386" PostTypeId="2" ParentId="3381" CreationDate="2012-09-27T17:37:34.463" Score="4" Body="&lt;p&gt;You misread the question. It isn't talking about any Armijo or sufficient decrease condition. It says assume that we compute $\alpha$ as in the formula. Then show that if the iteration converges, i.e., $x_k\rightarrow x^\ast$ that necessarily $\alpha\rightarrow 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proof of this statement is a simple application of Taylor expansion. If $x_k\rightarrow x^\ast$ then $f(x)$ is approximated better and better in a neighborhood of $x_k$ by a quadratic Taylor approximation whose minimum is attained at a point that corresponds to exactly $\alpha=1$. Simply form the Taylor expansion of $f(x_k+\alpha p_k)$ around $x_k$ with the given $p_k$ and see what happens.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-09-27T17:37:34.463" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="3387" PostTypeId="2" ParentId="3350" CreationDate="2012-09-28T03:15:24.950" Score="0" Body="&lt;p&gt;Now I have found the problem which leads to long time computation. Just as the code I posted here &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3344/how-to-solve-the-following-differential-equations-using-ode-functions-of-matlab?lq=1&quot;&gt;https://scicomp.stackexchange.com/questions/3344/how-to-solve-the-following-differential-equations-using-ode-functions-of-matlab?lq=1&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3345/how-to-solve-the-following-differential-equations-using-ode-functions-of-matlab&quot;&gt;https://scicomp.stackexchange.com/questions/3345/how-to-solve-the-following-differential-equations-using-ode-functions-of-matlab&lt;/a&gt;, I have utilized symbols to found the differential equations in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3345/how-to-solve-the-following-differential-equations-using-ode-functions-of-matlab&quot;&gt;https://scicomp.stackexchange.com/questions/3345/how-to-solve-the-following-differential-equations-using-ode-functions-of-matlab&lt;/a&gt;. However, it is difficult for matlab to deal with symbolic computation rather than numerical ones. Now I have made use of &lt;code&gt;y(1)&lt;/code&gt;, &lt;code&gt;y(2)&lt;/code&gt;,...&lt;code&gt;y(10)&lt;/code&gt; instead of using &lt;code&gt;subs()&lt;/code&gt; function to substibute the symbols such as &lt;code&gt;q1t&lt;/code&gt;, &lt;code&gt;q2t&lt;/code&gt;,...&lt;code&gt;q7tp&lt;/code&gt;. In this way, the equations can be solved within a few seconds.&lt;/p&gt;&#xA;" OwnerUserId="2066" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-09-28T04:37:17.393" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3388" PostTypeId="2" ParentId="3339" CreationDate="2012-09-28T04:42:32.743" Score="1" Body="&lt;p&gt;Your problem is reminiscent of the self-consistent field (SCF) procedure in electronic structure methods. although these are eigenvalue problems, there are some similarities. Most SCF implementations use the DIIS [1] method to accelerate or enforce convergence. It turns out that DIIS is applicable to nearly any iterative procedure in which convergence is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://en.wikipedia.org/wiki/DIIS&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/DIIS&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2012-09-28T04:42:32.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3389" PostTypeId="2" ParentId="3259" CreationDate="2012-09-28T09:43:03.990" Score="4" Body="&lt;p&gt;The FFT can be used for periodic boundary conditions. Because von Neumann boundary conditions are effectively &quot;mirror&quot; boundary conditions, you have to do a &quot;mirrored continuation&quot;, before you can apply an FFT. One drawback of this approach is that you will blow up the data volume by a factor 4 (which is not important if you are just interested in experimenting a bit). The use of the cosine transform implicitly does the &quot;mirrored continuation&quot; and avoids the factor 4 overhead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that depending on where the grid points near the boundary are located, there are two different ways to do a &quot;discrete mirrored continuation&quot;. Hence you will find that libraries like FFTW offer different variants of the cosine transform (corresponding to these different &quot;discrete mirrored continuations&quot;).&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2012-09-28T09:43:03.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3390" PostTypeId="1" CreationDate="2012-09-28T07:33:27.767" Score="0" ViewCount="491" Body="&lt;p&gt;I'm attempting to implement some code that moves particles and then calculated the acceptance/rejection of the move, but I'm stuck in a rut. Here is my Python code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def VMC(WF,numSteps):&#xA;     seed=5   ### my random number seed&#xA;     EnergyList=[]  &#xA;     R=numpy.zeros((2,3),float)   &#xA;     movesAttempted=0.0&#xA;     movesAccepted=0.0&#xA;     print &quot;num-steps: &quot;,numSteps&#xA;     for i in range(numSteps):  &#xA;&#xA;            OldPos= R.copy()&#xA;            oldWfc= WF.WaveFunction(R)&#xA;            for ptcl in xrange(0,len(R)):   &#xA;             R[ptcl] = numpy.add( R[ptcl], (numpy.random.rand(3) - 0.5)*3.0 ) &#xA;            newWfc= WF.WaveFunction(R)  &#xA;&#xA;            ratio = (newWfc**2/oldWfc**2)&#xA;&#xA;            rander = numpy.random.rand(1)&#xA;            if ratio &amp;gt; rander:&#xA;             Eloc = WF.LocalEnergy(R)&#xA;             EnergyList.append(Eloc)&#xA;             movesAttempted+=1&#xA;             movesAccepted+=1&#xA;            else:&#xA;             movesAttempted+=1&#xA;             R = OldPos         &#xA;     print &quot;movesAcepted: &quot;,movesAccepted&#xA;         print &quot;  movesAttempted: &quot;, movesAttempted             &#xA;     print &quot;Acceptance Ratio&quot;, movesAccepted/movesAttempted&#xA;     return EnergyList&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is a pastebin of more of the code: &lt;a href=&quot;http://pastebin.com/7nTLDesy&quot; rel=&quot;nofollow&quot;&gt;http://pastebin.com/7nTLDesy&lt;/a&gt;&#xA;I'm  getting a ridiculous value of the acceptance ratio, which makes me think I'm not getting the concept of how the electrons should be moved at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically: When I run this code I get an acceptance ratio of 0.478, while for VMC calculations, moving a particle in a box of length 1.5 (in each direction) around its current position (moving in 3d) the acceptance ratio is supposed to be   0.323.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is mainly to know how atoms are supposed to be moved in a VMC calculation and whether how I'm doing it is o.k.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen an &lt;a href=&quot;http://scienze-como.uninsubria.it/bressanini/src/var/var&quot; rel=&quot;nofollow&quot;&gt;implementation in Fortran&lt;/a&gt; (metropolis), and I was wondering how different mine is and how I can correct it to give better results.&lt;/p&gt;&#xA;" OwnerUserId="2093" OwnerDisplayName="mike" LastEditorDisplayName="user10435" LastEditDate="2015-03-13T23:36:17.457" LastActivityDate="2015-03-13T23:36:17.457" Title="How to move particles in a variational Monte Carlo simulation?" Tags="|software|python|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3391" PostTypeId="1" AcceptedAnswerId="3396" CreationDate="2012-09-28T13:20:54.333" Score="3" ViewCount="154" Body="&lt;p&gt;I was checking partitioning capabilities of Metis (mpmetis) when I noticed, that it leaves two single nodes. I have marked them in red &#xA;&lt;a href=&quot;https://i.stack.imgur.com/TgN7O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TgN7O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have you seen something similar or maybe it is my mistake?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entire mesh&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5mSPe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5mSPe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Elements are typical, 8-nodal quadrilaterals. I used mpmetis without any options to create 2 partitions. It produces two partitions, where the first part contains single nodes - not connected to any elements while the second partition is fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Partition 1:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PYWbF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PYWbF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Partition 2:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/b2rgk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b2rgk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1010" LastEditorUserId="20688" LastEditDate="2019-08-25T23:40:04.773" LastActivityDate="2019-08-25T23:40:04.773" Title="Single nodes after mpmetis partitioning" Tags="|mesh-generation|partitioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3392" PostTypeId="2" ParentId="351" CreationDate="2012-09-29T11:38:25.743" Score="11" Body="&lt;p&gt;If you want&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Matrix classes with an intuitive interface&lt;/li&gt;&#xA;&lt;li&gt;All the LAPACK and BLAS features&lt;/li&gt;&#xA;&lt;li&gt;Easy to learn and use API&lt;/li&gt;&#xA;&lt;li&gt;Easy to install&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then I recommend you to have a look at my library &lt;a href=&quot;http://flens.sf.net&quot;&gt;FLENS&lt;/a&gt;.  I designed it for exactly these kind of tasks.  However, it requires a C++11 conform compiler (e.g. gcc 4.7 or clang).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FLENS gives you exactly the same performance as the underlying BLAS implementation.  &lt;a href=&quot;http://flens.sourceforge.net/legacy/session1/tut9.html&quot;&gt;There are some (rather old) benchmarks showing this&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same can be said about FLENS-LAPACK it just gives you exactly the&#xA;same performance as Netlib's LAPACK if the same BLAS implementation&#xA;is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding new benchmarks let me go into more into detail ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some time ago I asked Clint Whaley (the author of ATLAS) what he thinks about the benchmarks published on the Eigen site.  He just confirmed my suspect that these benchmarks are most likely not reliable.  In the meantime some other realized benchmarks the way Clint suggested.  Details can be found on the ATLAS site and the Eigen mailing list.  The benchmarks are not nicely presented in graphs but they show that ATLAS is always about 40% faster than Eigen.  This contradicts the&#xA;benchmarks from the Eigen site but confirms other benchmarks (e.g. those from blaze-lib).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that for dense numerical linear algebra the matrix-matrix products&#xA;are most relevant.  Personally I don't care whether Eigen or ATLAS is&#xA;faster.  If Eigen was faster than ATLAS then I would use Eigen as BLAS-backend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: Yes, FLENS is my baby! That means I coded about 95% of it and every line of code was worth it :-)&lt;/p&gt;&#xA;" OwnerUserId="5160" LastEditorUserId="5160" LastEditDate="2012-10-05T01:01:09.067" LastActivityDate="2012-10-05T01:01:09.067" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3393" PostTypeId="2" ParentId="1634" CreationDate="2012-09-29T12:53:52.560" Score="4" Body="&lt;p&gt;The problem already starts with the term 'expression templates (ET)'.  I don't know if there is a precise definition for it.  But in its common usage it somehow couples 'how you code linear algebra expressions' and 'how it gets computed'.  For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You code the vector operation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v = 2*x + 3*y + 4*z;                    // (1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And it gets computed by a loop&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (int i=0; i&amp;lt;n; ++i)                 // (2)&#xA;    v(i) = 2*x(i) + 3*y(i) + 4*z(i);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my opinion this are two different things and need to be decoupled: (1) is a interface and (2) one possible implementation.  I mean this is common practice in programming.  Sure (2) might be a good default implementation, but in general I want to be able to utilize a specialized, dedicated implementation.  For instance, I want that a function like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myGreatVecSum(alpha, x, beta, y, gamma, z, result);    // (3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;get called when I am coding (1).  Maybe (3) just uses internally a loop like in (2).  But depending on the vector size other implementations might be more efficient.  Anyway, some expert in high performance can implement and tune (3) as much as possible.  So if (1) can not be mapped to a call of (3) then I rather avoid the syntactic sugar of (1) and directly call (3) right away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I describe is nothing new.  On the contrary, it's the idea behind BLAS/LPACK:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All performance critical operations in LAPACK are done by calling BLAS functions.&lt;/li&gt;&#xA;&lt;li&gt;BLAS just defines a interface for those linear algebra expressions that are commonly needed.&lt;/li&gt;&#xA;&lt;li&gt;For BLAS different optimized implementations exist.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If the scope of BLAS is not sufficient (e.g. it does not provide a function like (3)) then one can extend the scope of BLAS.  So this dinosaur from the 60s and 70s realizes with its stone age tool a clean and orthogonal separation of interface and implementation.  It's kind of funny that (most) numerical C++ libraries do not achieve this level of software quality.  Although the programming language itself is so much more sophisticated.  So it is no surprise that BLAS/LAPACK is still alive and actively developed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in my opinion ETs are not evil per se.  But how they are commonly used in numerical C++ libraries earned them a very bad reputation in the scientific computing circles.&lt;/p&gt;&#xA;" OwnerUserId="5160" LastActivityDate="2012-09-29T12:53:52.560" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3394" PostTypeId="2" ParentId="2100" CreationDate="2012-09-29T13:23:20.837" Score="1" Body="&lt;p&gt;Your problem doesn't seem to be, &quot;that the sum of the &quot;agents&quot; have to supply exactly a discrete portion of energy or nothing for each single demand ...&quot;, right? Or you did not understand me. So I'll try to describe my problem better, also because I found a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my problem, I have a set of agents where each one has a budget of certain resources, who can share the cost of tasks, which should be &quot;executed&quot; 1 time or not (many-to-many-assignment without the need to &quot;execute&quot; every task). &#xA;It means: the sum of partial solutions of agents for task x should be less or equal to cost of task x. The objective is to find the set of tasks with most value which agents can pay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working with gams software so i describe it in gams-style:&#xA;    set a agents, t tasks&#xA;    parameter cost(t), value(t)&#xA;    parameter resources(a)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;positive variable y(a,t) (non-int), part of agent a for cost of task t&#xA;objective: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;maxvalue =e= sum((a,t), value(t) * y(a,t) / cost(t) );&#xA;agentresource_max_constraint(a).. sum(t, y(a,t)) =l= resources(a);&#xA;taskcost_max_constraint.. sum(a, y(a,t)) =l= cost(t);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As I wrote, I had a solution but didn't know how to separate partial task solutions. But now I found out that I can build a constraint with a&lt;/p&gt;&#xA;&#xA;&lt;p&gt;binary variable &lt;code&gt;z(t)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;taskcost_bin_constraint z(t) =e= sum(a, y(a,t)) / cost(t);&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sum(a, y(a,t)) / cost(t)&lt;/code&gt; in the equation formulation is something between 0 and 1 and by this constraint, &lt;code&gt;z&lt;/code&gt; is 0 for all less than 1 and 1 for 1. &#xA;with this &lt;code&gt;taskcost_bin_constraint&lt;/code&gt; objective would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;maxvalue =e= sum(t, value(t) * z(t));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I was wondering but this works and gives me better solutions under the constraint, to build a task full or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you can also add such a constraint? A Constraint to fullfill the demands exactly, expressed in a expression with value between 0 and 1.&lt;/p&gt;&#xA;" OwnerUserId="2098" LastEditorUserId="391" LastEditDate="2012-10-20T02:57:34.153" LastActivityDate="2012-10-20T02:57:34.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3396" PostTypeId="2" ParentId="3391" CreationDate="2012-09-29T17:24:27.660" Score="4" Body="&lt;p&gt;Metis does not guarantee that the partitions are non-empty, so your code must deal with this situation.&lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2012-09-29T17:24:27.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3397" PostTypeId="1" AcceptedAnswerId="3398" CreationDate="2012-09-30T22:48:01.963" Score="4" ViewCount="3934" Body="&lt;p&gt;In my c language program, I have to store multiple dense $m\times m$ matrices corresponding to gridpoints $x_i$ with $i=1,...,n$.  I decided to create a three dimensional array $A\in R^{n\times m\times m}$ as follows:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double ****A;&#xA;A=(double****)malloc(sizeof(double***)*n);  &#xA;for (i=0; i&amp;lt;n; i++) {&#xA;     A[i]=(double***)malloc(sizeof(double**)*m);&#xA;     for (j=0; j&amp;lt;m; j++) {&#xA;          A[i][j]=(double**)malloc(sizeof(double*)*m);&#xA;     }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this way, I'm indexing my matrix as &lt;code&gt;A[i][j][k]&lt;/code&gt; such that at each index &lt;code&gt;[i]&lt;/code&gt; corresponds to a unique $m\times m$ matrix.  However, I now have to input these 2 dimensional ($m\times m$)matrices into a solver which assumes they are stored as only two dimensional arrays.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If i were writing the program in matlab, I could extract the 2D matrix by simply coding:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;B=A(i,:,:);&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm not sure how to do this in c.  There must be some clever trick using pointers.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-10-01T02:08:26.813" Title="Extracting a 2D matrix from a multidimensional array in C" Tags="|matrix|c|data-structures|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3398" PostTypeId="2" ParentId="3397" CreationDate="2012-09-30T23:55:54.980" Score="7" Body="&lt;p&gt;As Aron notes in his comment, rarely are matrices stored in nested pointer data structures in practice. Multiple levels of indirection (I'm told) cause considerable performance penalties, and require multiple allocations and frees (more performance issues, plus more potential to screw up and segfault). I've seen nested pointers used to store matrices only a few times (in CHEMKIN, and I think also in Cantera). Far more common practice is to store the data in memory associated with a single pointer; the C interfaces to BLAS and LAPACK use this practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably want to do something more like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double *A;&#xA;A = (double *) malloc(sizeof(double) * m * m * n);&#xA;// A[(i - 1) * m * m + (j - 1) * m + k] == A[i][j][k]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;i, j, k&lt;/code&gt; refer to traditional one-based mathematical indices; this sort of arrangement of data is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Row-major&quot;&gt;row-major ordering&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, to do the slicing you'd like, you will need to copy the data in &lt;code&gt;A&lt;/code&gt; to another pointer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double *B;&#xA;B = (double *) malloc(sizeof(double) * m * m);&#xA;// Here, zero-based indexing is used instead of the one-based indexing in the&#xA;// last code snippet.&#xA;for (j = 0; j++; j &amp;lt; m) {&#xA;    for (k =0 ; k++; k &amp;lt; m) {&#xA;        B[j * m + k] = A[i * m * m + j * m + k]; // B == A[i, :, :]&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There might be slicker ways of doing this operation, using something like &lt;code&gt;memcpy&lt;/code&gt;, but you'll still need to copy the data to another data structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose that you proceed with your current code, and suppose further that you continue with your strategy of storing a matrix in nested pointers. To slice &lt;code&gt;A&lt;/code&gt;, you would do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double **B;&#xA;B = (double **) malloc(sizeof(double *) * m);&#xA;// Zero-based indexing also used in this code snippet.&#xA;for (j = 0; j++; j &amp;lt; m) {&#xA;    B[j] = (double *) malloc(sizeof(double *) * m);&#xA;    for (k = 0; k++; k &amp;lt; m) {&#xA;        B[j][k] = A[i][j][k];&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, there might be slicker ways of doing this copying, such as using &lt;code&gt;memcpy&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to MATLAB, languages such as Fortran 90/95/2003/2008 and Python support array slicing syntax natively.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-09-30T23:55:54.980" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3399" PostTypeId="2" ParentId="3397" CreationDate="2012-10-01T02:08:26.813" Score="5" Body="&lt;p&gt;If you don't want to litter your program with this kind of copying loop then maybe C isn't your language. Assuming this is a common operation, C++ provides far nicer ways to do this sort of thing (by making the array a class and, possibly, making the sub-array a view of it). Even Fortran has a nicer syntax for this kind of operation. What this illustrates is that you can express everything in every Turing-complete language but that it isn't true that everything can be expressed with equal ease in each language.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-01T02:08:26.813" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3400" PostTypeId="1" CreationDate="2012-10-01T05:11:32.587" Score="13" ViewCount="1518" Body="&lt;p&gt;I'm working on solving the coupled one-dimensional &lt;a href=&quot;http://en.wikipedia.org/wiki/Poromechanics&quot;&gt;poroelasticity&lt;/a&gt; equations (biot's model), given as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-(\lambda+ 2\mu) \frac{\partial^2 u}{\partial x^2} + \frac{\partial p}{\partial x} = 0$$&#xA;$$\frac{\partial}{\partial t} \left[ \gamma p + \frac{\partial u}{\partial x}\right] -\frac{\kappa}{\eta}\left[\frac{\partial^2 p}{\partial x^2}\right]  =q(x,t)$$&#xA;on the domain $\Omega=(0,1)$ and with the boundary conditions:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p=0, (\lambda + 2\mu)\frac{\partial u}{\partial x}=-u_0$ at $x=0$ and $u=0, \frac{\partial p}{\partial x} = 0$ at $x=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I discretized these equations using a centered finite difference scheme:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\lambda + 2\mu)\frac{u^{t+1}_{i+1}-2u^{t+1}_i+u^{t+1}_{i-1}}{\Delta x^2} + \frac{p^{t+1}_{i+1}-p^{t+1}_{i-1}}{2\Delta x} = 0$$&#xA;$$\gamma \frac{p^{t+1}_i-p^t_i}{\Delta t} + \frac{u^{t+1}_{i+1}-u^{t+1}_{i-1}}{2\Delta x \Delta t} -\left[ \frac{u^t_{i+1}-u^t_{i-1}}{2\Delta x \Delta t}\right] - \frac{\kappa}{\eta}\left[ \frac{p^{t+1}_{i+1} -2p^{t+1}_i + p^{t+1}_{i-1}}{\Delta x^2}\right]=q^{t+1}_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently working out the details of the scheme's convergence by analyzing its consistency and stability.  The consistency part seems fairly straightforward to me, but I'm already foreseeing some difficulties with the stability analysis.  First of all, there are two variables and two equations.  Secondly, there is also a mixed spatiotemporal derivative term in the second equation.  I'm familiar with von neumann stability analysis and can see that it will be very tough to establish stability with this method.  Are there any alternatives to von neumann analysis that I can use?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="4244" LastEditDate="2013-07-01T09:29:38.073" LastActivityDate="2013-07-01T09:29:38.073" Title="Alternatives to von neumann stability analysis for finite difference methods" Tags="|pde|finite-difference|stability|numerical-analysis|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3403" PostTypeId="1" CreationDate="2012-10-01T15:08:31.877" Score="0" ViewCount="52" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Possible Duplicate:&lt;/strong&gt;&lt;br&gt;&#xA;  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2837/linear-programming-boundedness&quot;&gt;Linear programming boundedness&lt;/a&gt;  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;Consider the following LP:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max$  $\sum_{i=1}^N b_i \pi_i$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t. $\;\;$   $\pi_i-\pi_j\leq 1 $ $\quad$   for each $(i,j) \in \tilde{A}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i=1}^N a_s^i \pi_i \leq 1$ for each $s \in S$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This LP is always feasible, since zero is a feasible solution. Now, in order to find the optimum value, we relax the LP by multiplying second set of constraints by nonnegative Lagrangian multipliers $\lambda_s, s\in S$ and add them to the objective&#xA;function. Thereafter, we get the following relaxed dual LP:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L(\lambda)$: $\max$ $\sum_{i=1}^N (b_i+\sum_{s\in S}\lambda_s a_s^i)\pi_i - \sum_{s\in S}\lambda_s$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t.  $\pi_i-\pi_j\leq 1$ $\quad$ for each $(i,j) \in \tilde{A}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us consider the Lagrangian dual LP. This LP can be formulated&#xA;as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LD: $\min$ $L(\lambda)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t. $\lambda \geq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the question is as follows. Both LP and LD problems are feasible (e.g. zero vecor). Moreover, if the LD is bounded so is the LP. What cab be said if the LD is unbounded. Consider that in my case LP is always feasible.&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-10-01T15:43:31.897" Title="Lagrangian Duality" Tags="|optimization|linear-programming|" AnswerCount="0" CommentCount="4" ClosedDate="2012-10-01T20:56:24.770" ContentLicense="CC BY-SA 3.0" />
  <row Id="3404" PostTypeId="1" AcceptedAnswerId="3405" CreationDate="2012-10-02T08:07:49.820" Score="3" ViewCount="747" Body="&lt;p&gt;I am using a stable version of SQP algorithm from a lib. Parameters setting is left to the developer, althought default values are at hands. I launch solver on very simple optimization problems s.a. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min{( p_{1} * \exp^{ (p_{2} * x)})}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s.t.     0 \leq p_{1}, p_{2} \leq 10$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feed optimization with a well chosen initial point, using Monte Carlo sampling in the allowed space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears that optimization output is very sensitive to initial point. I tried to play with some parameters such as precision on gradient calculation (using finite difference scheme)... but still: even on simple examples like this one, it does not converge at all time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also would like optimization to be possible on any function such as elementary functions of matlab optimization toolbox, and their linear combination (exp, power, gaussian, sin, cos, fourier, polynomial, linear, rational as a start).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If somebody has some experience with tuning such algorithms, I would be glad that he shares that knowledge. Also, would you have good hint at literature where tuning strategies are unveiled? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;regards.&lt;/p&gt;&#xA;" OwnerUserId="1184" LastActivityDate="2012-10-02T10:16:54.527" Title="SQP optimization algorithm tuning advice" Tags="|optimization|c++|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3405" PostTypeId="2" ParentId="3404" CreationDate="2012-10-02T08:18:38.620" Score="3" Body="&lt;p&gt;The right thing is to scale your variables before calling the solver. Then you can probably use the default version. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good scaling is often problem-dependent and therefore difficut to do automatically. This is why most solvers leave it to the user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your variables are well-scaled if the Hessian divided by its absolutely largest entry has small entries only where the corresponding interactions are truly negligible. This may give you an idea how to scale even if you cannot calculate an explicit Hessian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example, it means to use $p_2x$ or even $e^{p_2x}$ as variable in place of $p_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, there are no fixed rules for scaling; it is an art and depends on the details of your objective function and your constraints. One way to find out which variables need scaling is to compute the Hessian at the starting point, equilibrate it using a scaling method for symmetric matrices, and check all variables that were scaled heavily. (If the Hessian is positive definite, you may simply divide the variables by the square root of the Hessian diagonal elements, or their order of magnitude; but if the Hessian is indefinite, this may be dangerous!) This gives you also scaling factors for linear scaling. If this is not sufficient, you either need nonlinear scaling&#xA;(inspect your function to find out what is natural), or you need to use the solver iteratively, restarting with the previous output and a new scale determined at the new starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you still need to tune, the standard recipe is to prepare a set of test problems typical for the application (but simpler if the application functions are expensive to evaluate), run all examples with the solver, and compute from the results a performance measure of interest. Then systematically change the tuning parameters to improve the performance. This is itself a black-box optimization problem, for which you can use solvers such as SNOBFIT (a solver developed by my group, works for continuous tuning parameters only) or NOMAD (also accepts discrete tuning parameters).&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-10-02T10:16:54.527" LastActivityDate="2012-10-02T10:16:54.527" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3406" PostTypeId="1" AcceptedAnswerId="3408" CreationDate="2012-10-02T13:20:17.917" Score="6" ViewCount="1417" Body="&lt;p&gt;To compute the eigenvector corresponding to dominant eigenvalue of a symmetric matrix $A\in\mathbb{R}^{n\times n}$, one used Power Iteration, i.e., given some random initialization, $u_1\in\mathbb{R}^n$, one iteratively computes $$u_1\leftarrow Au_1,$$after which a normalization is applied to $u_1$. Now, suppose that eigenvectors $u_1, u_2$ are computed in advance, and one wants to compute the eigenvector $u_3$ associated with the third dominant eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case the initial $u_3$ is orthogonal to both $u_1$ and $u_2$, can it be shown that the series $$u_3\leftarrow Au_3$$ converges in the direction of the eigenvector of $A$ corresponding to third dominant eigenvalue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the question is motivated by an observation that most implementations of the Power Iteration that computes higher eigenvectors provides Gram-Schmidt (GS) orthonormalization in each iteration, i.e., after each $u_3\leftarrow Au_3$, orthogonalization is applied wrt $v_1, v_2$. In case the orthonogonality of $u_3$ is imposed with its initialization (ie. from the start), is the GS necessary after each matrix-vector multiplication?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-10-02T14:12:34.147" Title="Eigenvectors with the Power Iteration" Tags="|linear-algebra|matrix|eigensystem|iterative-method|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3407" PostTypeId="2" ParentId="3406" CreationDate="2012-10-02T14:04:29.273" Score="8" Body="&lt;p&gt;In theory, yes. In practice, rounding errors will usually result in (initially slow) convergence to $u_1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At essentially the same cost one can run the Lanczos algorithm, which will have much faster convergence, and produce the three dominant eigenvalues unless two of these eigenvalues are essentially the same. For Lanczos, selective reorthogonalization is sufficient to get good results.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-02T14:04:29.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3408" PostTypeId="2" ParentId="3406" CreationDate="2012-10-02T14:11:47.970" Score="11" Body="&lt;p&gt;In exact arithmetic you shouldn't need to reorthogonalize regularly, but practically you do. Your u1 and u2 are close to (but not exactly) the true eigenvectors, so your initial deflation almost (but not entirely) removed the true eigenvectors from u3. The tiny components you left behind will be amplified by repeated multiplication by A, you will need to continue damping it out by gram-schmidt subtraction.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2012-10-02T14:11:47.970" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3409" PostTypeId="2" ParentId="3406" CreationDate="2012-10-02T14:12:34.147" Score="5" Body="&lt;p&gt;Yes. If $A\in\mathbb{R}^{n\times n}$ is symmetric, there exists an orthogonal basis of eigenvectors $v_1,\dots,v_n$ (sorted by decreasing magnitude) of $A$. If you expand your starting vector $u_3$ with respect to this basis, you get &#xA;$$ u_3 = \alpha_1 v_1 + \alpha_2 v_2 + \dots + \alpha_n v_n,$$&#xA;where $\alpha_1 = \alpha_2 =0$ because $u_3$ is orthogonal to $v_1$ and $v_2$. If $u_3$ is not orthogonal to $v_3$, you can now apply the usual argument for the convergence of the power method (with $\lambda_3$ and $v_3$ in place of $\lambda_1$ and $v_1$) to show convergence to $v_3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is true in exact arithmetic only; in floating point arithmetic, rounding errors will lead to loss of orthogonality, and orthogonalizing $u^k := A^k u_3$ against $v_1$ and $v_2$ is required (although it is not necessary to apply this after every single iteration).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2012-10-02T14:12:34.147" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3410" PostTypeId="1" AcceptedAnswerId="3411" CreationDate="2012-10-03T06:50:03.180" Score="1" ViewCount="260" Body="&lt;p&gt;what is your preferred test set to test quality of non linear solver? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;this could be set of data, model and results obtained with some benchmark solver, or simply a panel of test functions that could be used to generate synthetic data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks!&lt;/p&gt;&#xA;" OwnerUserId="1184" LastActivityDate="2012-10-03T09:18:35.450" Title="Test set non linear solver" Tags="|optimization|algorithms|nonlinear-programming|data-sets|testing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3411" PostTypeId="2" ParentId="3410" CreationDate="2012-10-03T08:01:13.360" Score="3" Body="&lt;p&gt;It depends a lot on which nonlinear problem you want to solve. &lt;BR&gt;&#xA;    &lt;a href=&quot;http://plato.asu.edu/bench.html&quot; rel=&quot;nofollow&quot;&gt;http://plato.asu.edu/bench.html&lt;/a&gt;&lt;BR&gt;&#xA;is a good place to start for local constrained optimization,&lt;BR&gt;&#xA;    &lt;a href=&quot;http://www.mat.univie.ac.at/~dferi/testenv.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~dferi/testenv.html&lt;/a&gt;&lt;BR&gt;&#xA;for global constrained optimization, and &lt;BR&gt;&#xA;    &lt;a href=&quot;http://archimedes.cheme.cmu.edu/?q=dfocomp&quot; rel=&quot;nofollow&quot;&gt;http://archimedes.cheme.cmu.edu/?q=dfocomp&lt;/a&gt;&lt;BR&gt;&#xA;    &lt;a href=&quot;http://coco.gforge.inria.fr/doku.php?id=bbob-2012&quot; rel=&quot;nofollow&quot;&gt;http://coco.gforge.inria.fr/doku.php?id=bbob-2012&lt;/a&gt;&lt;BR&gt;&#xA;for unconstrained black box optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not aware of special test sets for nonlinear least squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know the kind of fitting problem you want to apply it to, you can generate from any known model simulated data and then compare the solutions obtained with the input for the simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you want to have a general-purpose constrained least square solver, &#xA;then you can take anyway hardly any advantage of the least squares structure.&#xA;Thus it would be sensible to use (or develop) a general purpose nonlinear program solver, and test it on the test sets from one of the above sites.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-10-03T09:18:35.450" LastActivityDate="2012-10-03T09:18:35.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3412" PostTypeId="1" CreationDate="2012-10-03T14:23:57.143" Score="7" ViewCount="2981" Body="&lt;p&gt;We would like to reach Molecular Dynamics simulation of proteins with around 20000 atoms in explicit water with trajectories of around 1 microsecond each. We are looking at different options for computer resources to complete these simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we (in Europe) can not apply for supercomputing time in Anton (from D.E.Shaw Research) and we have some funds (up to 500k €) we wonder which would be the best cluster or HPC infrastructure to buy for such calculations.&lt;/p&gt;&#xA;" OwnerUserId="891" LastEditorUserId="9" LastEditDate="2012-10-06T03:21:41.917" LastActivityDate="2013-10-10T03:04:48.757" Title="Best hardware solution for microsecond Molecular Dynamics" Tags="|computational-chemistry|hpc|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3413" PostTypeId="2" ParentId="3412" CreationDate="2012-10-03T16:11:18.807" Score="12" Body="&lt;p&gt;For such a small simulation, I would strongly suggest looking into GPU-based solutions. This is probably what will get you the most ns/day/Euro.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, the fastest fully-featured GPU-based Molecular Dynamics (MD) software out there is &lt;a href=&quot;http://multiscalelab.org/acemd&quot;&gt;ACEMD&lt;/a&gt; (see &lt;a href=&quot;http://multiscalelab.org/gianni/publications?action=AttachFile&amp;amp;do=get&amp;amp;target=acemd.pdf&quot;&gt;here&lt;/a&gt; for timings). The software, however, is commercial, but has a single-GPU free version that can be used for evaluation purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other fully-featured, yet open-source, GPU-enabled MD packages include &lt;a href=&quot;http://www.ks.uiuc.edu/Research/gpu/#md&quot;&gt;NAMD&lt;/a&gt; and &lt;a href=&quot;http://www.gromacs.org/Developer_Zone/Roadmap/GROMACS_4.6&quot;&gt;GROMACS 4.6&lt;/a&gt;. Other projects include &lt;a href=&quot;http://gcl.cis.udel.edu/projects/fenzi/index.php&quot;&gt;FenZi&lt;/a&gt;, but they don't seem to make their code available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the Joint Amber-Charmm (JAC) Benchmark, which consists of 23 558 atoms, but with a relatively short cutoff of 0.9nm, all these codes will get a handful of ns per day on a commodity GPU. That's still a few days of computing to get 1ms, but not bad considering that it's just one single machine.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-10-05T09:49:40.630" LastActivityDate="2012-10-05T09:49:40.630" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="3414" PostTypeId="1" AcceptedAnswerId="3423" CreationDate="2012-10-03T16:59:55.477" Score="3" ViewCount="242" Body="&lt;p&gt;Let $X\in\mathbb{R}^{n\times p}$ denote a matrix with $p$ linearly-independent columns, and let $L\in\mathbb{R}^{n\times n}$ denote a symmetric matrix. Furthermore, let $D\in\mathbb{R}^{n\times n}$ denote a diagonal matrix with positive-entries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose that we are interested in computing solutions to the &lt;em&gt;symmetric generalized-definite eigenproblem&lt;/em&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  A v = \lambda B v,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the symmetric matrices $A$ and $B$ (where $B$ is also positive-definite) are implicitly given by the matrices $X$, $L$, and $D$ above, where $A = X^T L X$, and $B = X^T D X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran into a paper which claimed that we may solve an equivalent standard eigenvalue problem involving a modification of $X$. In particular, if we orthogonalize $X$ with respect to $D$, say $\tilde X^T D \tilde X = I$, &#xA;then eigenpairs of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde X^T L \tilde X u = \lambda u&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are also eigenpairs for the original generalized eigenproblem. How can we show this to be true?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="67" LastEditDate="2012-10-04T18:26:41.110" LastActivityDate="2012-10-04T18:55:51.463" Title="Manipulating a generalized eigenvector problem to plain eigenvector problem" Tags="|linear-algebra|matrix|eigensystem|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3415" PostTypeId="1" AcceptedAnswerId="3416" CreationDate="2012-10-03T20:14:00.337" Score="9" ViewCount="2344" Body="&lt;p&gt;Is there a preferred way how to implement a fast (approximate) evaluation of the Chebyshev interpolation polynomial on uniform grid (given the function values at the Chebyshev nodes)? My problem is that the interpolation becomes slow when the degree of the interpolating polynomial increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following ideas came to my mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Try to adapt non-uniform FFT (NFFT) techniques&lt;/li&gt;&#xA;&lt;li&gt;Use FFT to compute the derivates at the Chebyshev nodes, potentially after first going to a finer (Chebyshev) grid. Then use a piecewise cubic interpolation for (approximative) evaluation.&lt;/li&gt;&#xA;&lt;li&gt;Use some formula that only uses function values (and potentially derivatives) at &quot;nearby&quot; Chebyshev nodes (this is related a specific NFFT technique).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="534" LastActivityDate="2017-10-12T17:15:18.387" Title="Fast (approximate) evaluation of Chebyshev polynomial" Tags="|interpolation|fourier-analysis|fftw|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3416" PostTypeId="2" ParentId="3415" CreationDate="2012-10-03T22:39:06.743" Score="11" Body="&lt;p&gt;Have you thought of using &lt;a href=&quot;https://en.wikipedia.org/wiki/Lagrange_polynomial#Barycentric_form&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barycentric Interpolation&lt;/a&gt;? A detailed description on how to do it efficiently for Chebyshev nodes is given in Section 5 of &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/S0036144502417715&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is actually an exact evaluation of the Chebyshev interpolant. If you're evaluating a polynomial of degree $n$ at $m$ nodes, the cost is in $\mathcal O(nm)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative, if you have the Chebyshev coefficients of your interpolatory polynomial, is to use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Clenshaw_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clenshaw algorithm&lt;/a&gt;. If you only have the function values at the Chebyshev nodes, but have to evaluate the polynomial several times, you could compute the coefficients with an FFT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Clenshaw algorithm is somewhat faster than Barycentric interpolation as it requires only additions and multiplications, and that it also vectorizes quite nicely.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="9667" LastEditDate="2017-10-12T17:15:18.387" LastActivityDate="2017-10-12T17:15:18.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3417" PostTypeId="1" AcceptedAnswerId="3421" CreationDate="2012-10-04T05:06:12.257" Score="16" ViewCount="1721" Body="&lt;p&gt;I am looking for some open source implementation (any of Python, C, C++, Fortran is fine) of rational approximation to a function. Something along the article [1].&#xA;I give it a function and it gives me back two polynomials, whose ratio is the approximation on the given interval and the error is oscillating with the same amplitude and it is an optimal approximation, or close to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I found:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It looks like chebfun can do that, but I don't have access to Matlab *.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a simple program in the section 5-13 &quot;Rational Chebyshev Approximation&quot; in Numerical Recipes (NR).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Mathematica has EconomizedRationalApproximation and MiniMaxApproximation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I was wondering whether there is something newer (possibly better tested) than the NR code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My application is that I have a set of special functions, about 10, which are given either as hypergeometric series, or some formula which has numerical cancellations, and I want to have a robust, fast and accurate evaluation function, that is being called in the inner most loop of calculating two particle matrix elements in Hartree Fock calculations. I put a simple example of a function that works for me at [2]. As you can see, it's either using a direct formula, or a series around x=0, that I calculated using SymPy. It sort of works, but the accuracy is not great, around x=1 about half of the significant digits are lost (but for x=0.1 as well as x=1e5 it is accurate to almost all significant digits). I am looking for a better approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Deun, J., &amp;amp; Trefethen, L. N. (2011). A robust implementation of the Carathéodory-Fejér method for rational approximation. BIT Numerical Mathematics, 51(4), 1039–1050. doi:10.1007/s10543-011-0331-7 (&lt;a href=&quot;http://eprints.maths.ox.ac.uk/926/1/NA-10-03.pdf&quot; rel=&quot;noreferrer&quot;&gt;June 2010 e-print&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://gist.github.com/3831580&quot; rel=&quot;noreferrer&quot;&gt;https://gist.github.com/3831580&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(*) Nor to the chebfun website, which gives me 404, but Pedro suggested that must be my provider issue.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="651" LastEditDate="2015-02-22T19:25:11.053" LastActivityDate="2015-10-23T12:08:06.717" Title="Open source implementation of rational approximation to a function" Tags="|performance|special-functions|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3418" PostTypeId="1" CreationDate="2012-10-04T07:25:22.003" Score="12" ViewCount="419" Body="&lt;p&gt;I want to solve a non-linear problem with non-linear equality constrains and I'm using a augmented Lagrangian with a penalty regularization term that, as well known, spoils the condition number of my linearized systems (at each Newton iteration I mean). The bigger the penalty term, the worse the condition number is. Would someone know an efficient way to get rid of this bad conditioning in that specific case ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, I'm using the classical augmented lagrangian because I have lots of constraints which may generally be  redundant. So blindly incorporating the constraints direclty into the primal variables is very convenient. I tried other more sophisticated approaches based on variable eliminations or efficient preconditioners directly on the KKT system but, because of constraints redundancy, I have some troubles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with regard to $\mathbf u =[u_1,\cdots,u_n]$ variables is formulated as follow &#xA;my Lagrangian as the form&#xA;$$\mathcal L(\mathbf u,\lambda):= \mathcal W(\mathbf u) + \rho \lambda^T \,c(\mathbf u) + \frac{\rho}{2} c^2(\mathbf u) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So generally The goal at each Newton iteration is to solve a problem of the form&#xA;$$A \Delta u = b$$&#xA;With (we drop hessian of the constraint)&#xA;$$A(\mathbf u,\rho): =  \nabla_{\mathbf u}^2 \mathcal W(\mathbf u) + \rho C^T(\mathbf u)C(\mathbf u) $$ &#xA;and&#xA;$$b(\mathbf u,\rho) :=- \big(\nabla_{\mathbf u}\mathcal W(\mathbf u) +(\rho +\lambda^Tc(\mathbf u)) \nabla_{\mathbf u}(\mathbf u)\big)$$&#xA;and the capital $C$ is meant for $C(\mathbf u) := \nabla_{\mathbf u} c(\mathbf u)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="1804" LastEditDate="2013-06-26T13:54:24.123" LastActivityDate="2013-06-26T13:54:24.123" Title="Efficient preconditioner for Augmented Lagrangian" Tags="|linear-solver|preconditioning|constrained-optimization|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3419" PostTypeId="2" ParentId="3418" CreationDate="2012-10-04T13:56:25.873" Score="4" Body="&lt;p&gt;Introduce extra variables for the spoiling terms in the KT condition, and you can find a bigger symmetric system that is numerically well-behaved, with only the inverse of the penalty factor entering the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve the ill-conditioned system $(A+\rho C^TC)x=b~$ when $\rho$ is large, introduce $y=\rho Cx$ and recast your problem in the form $Ax+C^Ty=b$, $Cx-\rho^{-1}y=0$, which is generically well-conditioned. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-10-04T15:16:07.023" LastActivityDate="2012-10-04T15:16:07.023" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3420" PostTypeId="1" CreationDate="2012-10-04T15:15:02.473" Score="9" ViewCount="1594" Body="&lt;p&gt;Most of my programming is one-off research codes in C for my own use. I have never distributed any code to other than close collaborators. I have developed an algorithm that I am publishing in a scientific journal. I want to provide the source code and perhaps executable code in the online supplement to the article. A colleague requested that I make a generalization to the algorithm which required me to write in C++ (ack!) and which requires that I solve small dense linear systems. If I succeed in getting a user base for the algorithm it will be partly because the entry bar to using it is low (like on the floor). Potential users won't install libraries, etc. in order to use the code. I want the code to be fully stand alone and unencumbered by any license at all. I might simply write my own solver by taking something out of Golub and van Loan but I'd rather use a vanilla solver that someone else has already written if there are any out there. Suggestions appreciated. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="2119" LastEditorUserId="9" LastEditDate="2012-10-04T23:31:40.363" LastActivityDate="2012-10-05T01:50:32.273" Title="Recommendations for a lightweight/no-install C or C++ based dense linear algebra solver" Tags="|linear-solver|libraries|c++|c|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3421" PostTypeId="2" ParentId="3417" CreationDate="2012-10-04T15:36:52.383" Score="5" Body="&lt;p&gt;Might this be of any help? &lt;a href=&quot;http://www.alglib.net/interpolation/rational.php&quot; rel=&quot;noreferrer&quot;&gt;http://www.alglib.net/interpolation/rational.php&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-10-04T15:36:52.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3422" PostTypeId="1" AcceptedAnswerId="3424" CreationDate="2012-10-04T15:56:20.950" Score="15" ViewCount="878" Body="&lt;p&gt;I wonder what has happened to polynomial preconditioners. I am interested in them, because they appear to be comparatively elegant from a mathematical perspective, but as far as I have read in surveys on krylov methods, they generally perform very poor as preconditioners. In the words of Saad and van der Host, &quot;Current interest in these techniques has all but vanished&quot; &lt;a href=&quot;http://igitur-archive.library.uu.nl/math/2001-0227-152732/ithistory.pdf&quot;&gt;(Here)&lt;/a&gt;. Nevertheless, there have been uses for multicore- and GPU-calculations in the recent past.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me or rather explain me in which contexts these methods are still alive, and where to find a good survey on the current state of the art?&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="238" LastEditDate="2012-12-19T13:59:35.927" LastActivityDate="2012-12-19T13:59:35.927" Title="What is the current state of polynomial preconditioners?" Tags="|iterative-method|preconditioning|krylov-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3423" PostTypeId="2" ParentId="3414" CreationDate="2012-10-04T17:21:56.280" Score="5" Body="&lt;p&gt;Let $X=\tilde X Z$, where $\tilde X^T D \tilde X=I$. Then we may rewrite &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  (X^T L X)v = \lambda (X^T D X) v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  (Z^T \tilde X^T L \tilde X Z) v = \lambda (Z^T Z)v.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, put $u=Z v$, and premultiply both sides of the previous equation by $Z^{-T}$ to find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \tilde X^T L \tilde X u = \lambda u.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth noting that, contrary to your last sentence, the $u$ and $v$ are not the same, but are related through the equation $u=Z v$. Thus, after solving the standard eigenvalue problem with $\tilde X^T L \tilde X$, linear systems will need to be solved with $Z$ in order to find the original eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also important to notice that the fact that $D$ is positive is crucial to the ability to orthonormalize $X$ with respect to $D$. Consider the case where $D=-I$. Then requiring that $\tilde X^T D \tilde X = I$ is equivalent to requiring that $\tilde X^T \tilde X = -I$, which is not possible for real $\tilde X$ (nor in the complex case, when we replace transposes with Hermitian-transposes). When $D$ is positive, we may define&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA; Z = \sqrt{X^T D X},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which will exist because $X^T D X$ will be positive-definite.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-10-04T18:55:51.463" LastActivityDate="2012-10-04T18:55:51.463" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3424" PostTypeId="2" ParentId="3422" CreationDate="2012-10-04T18:56:26.957" Score="12" Body="&lt;p&gt;To perform reasonably, polynomial preconditioners need fairly accurate spectral estimates. For ill-conditioned elliptic problems the smallest eigenvalues are usually separated such that methods like Chebyshev are far from optimal. The most interesting property of polynomial methods is that they do not require any inner products.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's actually quite popular to use polynomial &lt;em&gt;smoothers&lt;/em&gt; in multigrid. The main difference from a preconditioner is that the smoother is only supposed to target &lt;em&gt;part&lt;/em&gt; of the spectrum. A polynomial smoother is currently the default in PETSc's multigrid, for example. See also &lt;a href=&quot;http://www.columbia.edu/~ma2325/adams_poly.pdf&quot;&gt;Adams et al, &lt;em&gt;Parallel multigrid smoother: polynomial versus Gauss-Seidel&lt;/em&gt; (2003)&lt;/a&gt; for a comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Polynomial preconditioners can be used purely to reduce the frequency of reductions. Although they have to be retuned for each matrix, the savings can be significant on hardware in which reductions are expensive (common on large supercomputers). See &lt;a href=&quot;http://www.mcs.anl.gov/uploads/cels/papers/P2097-0612.pdf&quot;&gt;McInnes, Smith, Zhang, and Mills, &lt;em&gt;Hierarchical and Nested Krylov Methods for Extreme-Scale Computing&lt;/em&gt; (2012)&lt;/a&gt; for more on this.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-10-04T18:56:26.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3425" PostTypeId="2" ParentId="3418" CreationDate="2012-10-04T20:39:56.230" Score="6" Body="&lt;p&gt;Depending on the problem structure, you can solve the ill-conditioned Augmented Lagrangian system directly. For example, BDDC/FETI-DP can solve almost-incompressible elasticity in primal form with a convergence rate independent of the Poisson ratio (piecewise constant on subdomains, but with arbitrary jumps). Similarly, multigrid methods that exactly reproduce the volumetric mode can have this property. Such methods are problem-specific and in general, large penalties result in systems that are difficult to precondition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To allow more flexibility in preconditioner choice, I recommend introducing explicit dual variables and writing the larger saddle point system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{pmatrix}A &amp;amp; C^T \\ C &amp;amp; -\rho^{-1} \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} b \\ 0 \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as suggested by Arnold Neumaier. This system is much better conditioned and allows you to accurately evaluate a residual. If a preconditioner for some penalized system $A - \tilde\rho C^T C$ exists (where $\tilde \rho \le \rho$), you can use it as a block preconditioner for the saddle point system. For an example of this, see &lt;a href=&quot;http://www.cs.sandia.gov/~rblehou/pbp.pdf&quot;&gt;Dohrmann and Lehoucq (2006)&lt;/a&gt; which preconditions incompressible elasticity in mixed form using BDDC applied to compressible problems. Another popular class of methods are based on approximating the Schur complement $-\rho^{-1} - C A^{-1} C^T$ using &quot;approximate commutator&quot; arguments. There is an extremely diverse range of methods for solving saddle point problems, see &lt;a href=&quot;http://dx.doi.org/10.1017/S0962492904000212&quot;&gt;Benzi, Golub, and Liesen, &lt;em&gt;Numerical Solution of Saddle Point Problems&lt;/em&gt; (2005)&lt;/a&gt; for a review. If you are using PETSc, many of the methods described in the review above can be constructed using run-time options via the &lt;code&gt;PCFIELDSPLIT&lt;/code&gt; component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can be more specific about the source of your problem (what are you minimizing and what is the constraint), I may be able to suggest more specific references.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-10-04T20:39:56.230" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3426" PostTypeId="1" AcceptedAnswerId="3427" CreationDate="2012-10-04T20:57:27.640" Score="13" ViewCount="1271" Body="&lt;p&gt;A Poisson equation with all Neumann boundary conditions has a single constant dimensional null space.  When solving via a Krylov method, the null space can be removed either by subtracting the mean of the solution each iteration or by pinning the value of a single vertex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pinning a single vertex has the benefit of simplicity, and also avoids an extra global reduction per projection.  However, it is typically viewed as bad due to its effect on conditioning.  Therefore, I've always subtracted means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the two methods differ from each other by at most a rank 2 correction, so according to (1) they should converge in nearly the same number of iterations (at least in exact arithmetic).  Is this reasoning correct, or is there an additional reason that point pinning is bad (perhaps inexact arithmetic)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1): &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3327/how-do-low-rank-modifications-affect-krylov-method-convergence&quot;&gt;How do low rank modifications affect Krylov method convergence?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-10-04T22:37:19.260" Title="Why is pinning a point to remove a null space bad?" Tags="|krylov-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3427" PostTypeId="2" ParentId="3426" CreationDate="2012-10-04T22:05:27.213" Score="9" Body="&lt;p&gt;Your arguments apply naturally to the unpreconditioned case. The reason that I don't recommend pinning is that it confuses norms and preconditioning. If you know the size of a typical diagonal value, you can scale the trivial equation for the pinned node so that norms become reasonable again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see the consequence on preconditioning, we have to distinguish between different methods of enforcing the pinning. I consider two of the most popular.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the pinning is achieved by &quot;zeroing a row&quot; (setting a row equal to a scaled row of the identity), it introduces asymmetry which restricts the choice of Krylov method and can confuse preconditioners (e.g. make algebraic multigrid choose a poor aggregate).&lt;/li&gt;&#xA;&lt;li&gt;If the corresponding column is also zeroed (with the contribution&#xA;&quot;lifted&quot; to the right hand side), the effect is pretty benign.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that interpolation operators for multigrid may have to be adjusted&#xA;    to do the pinning in a compatible way on each level.&#xA;If you don't&#xA;    mind the complexity introduced by implementing the pinning with a&#xA;    good scaling, it is a fine approach. In most cases, we find that it is more intrusive and error-prone to implement pinning in a non-disruptive way than to provide the near-null space. By having the original (singular) matrix around, the solver library can also verify that the provided null space is indeed a null space, thus protecting against a common mistake.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-10-04T22:05:27.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3428" PostTypeId="2" ParentId="3420" CreationDate="2012-10-04T23:26:35.127" Score="5" Body="&lt;p&gt;A very early mistake that many people make when getting started in scientific computing is assuming that you need to write all of your code in the same language.  I think this is due largely to historical reasons, when it wasn't clear how to make compiled programs communicate with each other across even versions of the same compiler.  That said, in this case, if you are going to be using C++ anyway, there are several very good C++ header-only template libraries that might fit your needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are distributing your code for academic reasons, and you would like to embed a dense linear algebra solver into your code, I would strongly recommend that you consider &lt;a href=&quot;http://eigen.tuxfamily.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigen&lt;/a&gt;.  Eigen has been licensed under the Mozilla Public License and is a header-only library.  This means that you can distribute Eigen with your code in source form (this does not impose any licensing restrictions on your code), and you will receive access to its general capabilities, including extremely efficient dense linear solvers.  As GertVdE mentions, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/351/recommendations-for-a-usable-fast-c-matrix-library&quot;&gt;you have several other options&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-10-04T23:26:35.127" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3429" PostTypeId="2" ParentId="3420" CreationDate="2012-10-05T00:23:26.697" Score="4" Body="&lt;p&gt;If you want a reliable solver for systems of linear equations I would recommend &lt;a href=&quot;http://flens.sf.net&quot; rel=&quot;nofollow&quot;&gt;FLENS&lt;/a&gt;.  It contains an exact re-implementation of LAPACK (it even reproduces the same roundoff errors as LAPACK if a single-threaded BLAS implementation is used).  This is true for all &lt;a href=&quot;http://apfel.mathematik.uni-ulm.de/~lehn/FLENS/flens/lapack/lapack.html&quot; rel=&quot;nofollow&quot;&gt;FLENS-LAPACK&lt;/a&gt; functions (together with the utility functions about 100 routines).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;FLENS is under a BSD License and therefore allows to be incorporated into proprietary products.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FLENS is header only and if you only need a subset of FLENS I can give you a stripped-down version containing only those functions you need.  FLENS comes with its own reference BLAS implementation.  But optionally your users can link against optimized BLAS libraries like ATLAS, OpenBLAS or GotoBALS.  For large matrices this gives a performance gain of about 40% compared to Eigen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And yes, Eigen also uses the LAPACK test suite to check their results.  They do this for 3 functions (Lu, Cholesky and Eigenvalues/-vectors of a symmetric matrix).  However, their computation of eigenvalues/-vectors of a non-symmetric matrix would fail the LAPACK test suite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: Yes, FLENS is my baby!  That means I coded about 95% of it and every line of code was worth it.&lt;/p&gt;&#xA;" OwnerUserId="5160" LastEditorUserId="5160" LastEditDate="2012-10-05T00:50:07.147" LastActivityDate="2012-10-05T00:50:07.147" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="3431" PostTypeId="2" ParentId="326" CreationDate="2012-10-05T01:34:27.830" Score="0" Body="&lt;p&gt;If you're running out of memory, just run on more nodes so you have more memory.  You're wasting a very valuable resource (the human brain) to solve a problem that has a very easy solution.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2012-10-05T01:34:27.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3433" PostTypeId="2" ParentId="3420" CreationDate="2012-10-05T01:50:32.273" Score="7" Body="&lt;p&gt;I would suggest to exactly duplicate the Lapack interface to the function that you need, most probably you just need &lt;code&gt;dgesv&lt;/code&gt;. That way people that have Lapack installed can simply link to it and it will just work. For people that don't have Lapack installed, you provide your own simple implementation of this function, or possibly implement it using Eigen or FLENS as others suggested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Fortran land, the Lapack library is such a standard, that most people simply use it and that's it, instead of providing their own implementations. &lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2012-10-05T01:50:32.273" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3434" PostTypeId="2" ParentId="3412" CreationDate="2012-10-05T04:42:38.970" Score="4" Body="&lt;p&gt;Even though you have so much money available to spend on computing resources, the bigger issue, as Pedro points out, is that your problem is relatively small. With roughly 100,000 atoms, your &quot;sweet spot&quot; on CPU's will likely be about 100 cores. If you try to use more than that, you'll end up spending a lot of time communicating information between processors, which can be much more expensive. You could try to purchase multithreaded processors, but they might not help nearly as much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you are relatively speaking in the sweet spot for GPU computing. So your best bet would be either to use a package like &lt;a href=&quot;http://codeblue.umich.edu/hoomd-blue/&quot; rel=&quot;nofollow&quot;&gt;HOOMD&lt;/a&gt;, or to take advantage of shared-memory machines using the multithreaded package options in codes such as &lt;a href=&quot;http://lammps.sandia.gov&quot; rel=&quot;nofollow&quot;&gt;LAMMPS&lt;/a&gt; or &lt;a href=&quot;http://www.ks.uiuc.edu/Research/namd/&quot; rel=&quot;nofollow&quot;&gt;NAMD&lt;/a&gt; to avoid some of the internal message passing.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2012-10-05T19:41:29.123" LastActivityDate="2012-10-05T19:41:29.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3435" PostTypeId="1" AcceptedAnswerId="3449" CreationDate="2012-10-05T14:58:13.213" Score="2" ViewCount="329" Body="&lt;p&gt;What is the best tool to sample rotamers in side chain amino acids in a protein? More exactly, I want to sample the different rotamers only for lysine residues while keeping backbone rigid. I've tried to use the fixed backbone design application (&quot;fixbb&quot;) in Rosetta3.4 but this program give only one (optimized?) solution. Here are my input files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-s file.pdb&#xA;-resfile RESFILE&#xA;-nstruct 50&#xA;-ex1&#xA;-ex2&#xA;-database /path/to/rosetta_database&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and for the RESFILE:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NATAA&#xA;start&#xA;2 E NATRO&#xA;3 E NATRO&#xA;4 E NATRO&#xA;5 E NATRO&#xA;[...]&#xA;212 E NATRO&#xA;213 E NATRO&#xA;214 E NATRO&#xA;215 E NATRO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The header before start &lt;code&gt;NATAA&lt;/code&gt; indicates that I want to only repack the input sequence and the, e.g. &lt;code&gt;2 E NATRO&lt;/code&gt;, command indicates to freeze the considered residue (here the second residue of chain E). For the output I obtain 50 (&lt;code&gt;nstruct&lt;/code&gt;) &lt;strong&gt;identical structures&lt;/strong&gt;, with all the residue in the native conformation given by the &lt;code&gt;file.pdb&lt;/code&gt; except for the lysine residues. My question is how to obtain &lt;code&gt;nstruct&lt;/code&gt; different models for the lysine side chain rotamers?&lt;/p&gt;&#xA;" OwnerUserId="2121" LastEditorUserId="1552" LastEditDate="2012-10-05T15:31:11.923" LastActivityDate="2012-10-09T14:44:55.600" Title="Side chain rotamers sampling in protein" Tags="|software|computational-biology|molecular-mechanics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3436" PostTypeId="2" ParentId="3435" CreationDate="2012-10-05T16:19:14.973" Score="1" Body="&lt;p&gt;Estimate from the PDB the covariance matrix of the vector of atomic distances in the side chains minus the vector of template distances. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then simulate random corrections to the template distances using the multivariate Gaussian with this covariance matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convert each vector of distances to a distance matrices.&#xA;Then convert the randomly corrected distance matrices to Gram matrices, cf. &lt;a href=&quot;https://scicomp.stackexchange.com/a/1946/1117&quot;&gt;https://scicomp.stackexchange.com/a/1946/1117&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then truncate the gram matrices to rank 3 (using an SVD) to get coordinates of random side chain structures that have approximately the correct distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, rotate thesee structures to match the backbone.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-10-05T16:19:14.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3437" PostTypeId="1" CreationDate="2012-10-05T16:34:45.040" Score="7" ViewCount="850" Body="&lt;p&gt;CG may be used to estimate the extremal eigenvalues of a SPD matrix (by computing eigenvalues of tridiagonal matrix associated with the Lanczos algorithm). After a few iterations the largest eigenvalue is generally well approximated whereas the smallest is generally overestimated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to have some estimate of the quality of those approximate eigenvalues?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="127" LastEditDate="2013-05-14T15:51:46.157" LastActivityDate="2018-12-07T03:02:05.340" Title="Estimate extreme eigenvalues with CG" Tags="|linear-algebra|conjugate-gradient|eigenvalues|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3438" PostTypeId="2" ParentId="3435" CreationDate="2012-10-06T10:16:57.887" Score="1" Body="&lt;p&gt;You can use the &lt;code&gt;set_residue_to_rotamer_number()&lt;/code&gt; function in Coot to do this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m = read_pdb(&quot;test.pdb&quot;)&#xA;for irot in range(n_rotamers(m, &quot;A&quot;, 30, &quot;&quot;)):&#xA;    set_residue_to_rotamer_number(m, &quot;A&quot;, 30, &quot;&quot;, &quot;&quot;, irot)&#xA;    fn = &quot;sample-A-30-rot-&quot; + str(irot) + &quot;.pdb&quot;&#xA;    write_pdb_file(m, fn)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2126" LastEditorUserId="2126" LastEditDate="2012-10-07T11:38:33.753" LastActivityDate="2012-10-07T11:38:33.753" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3439" PostTypeId="1" AcceptedAnswerId="3441" CreationDate="2012-10-07T06:51:54.047" Score="12" ViewCount="379" Body="&lt;p&gt;I want to integrate a polynomial expression over a 4-node element in 3D. Several books on FEA cover the case where integrating is performed over an arbitrary flat 4-noned element.&#xA;The usual procedure in this case is to find Jacobi matrix and use it's determinant to change the integration basis to the normalized one in which I have the simpler integration limits [-1;1] and the Gauss-Legendre quadrature technique is used easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words $\displaystyle\int_S f(x,y)\ \mathrm{d}x\,\mathrm{d}y\,$  is reduced to the form of $\displaystyle\int^{-1}_{1}\int^{-1}_{1} \tilde{f}(e,n)\ \left|\det(J)\right|\,\mathrm{d}e\,\mathrm{d}n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in 2D case I change the flat arbitrary element to the flat one but well-shaped square 2 by 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3D 4-noded element isn't flat in general but I suppose&#xA;it still can be mapped with 2D coordinate system which is somehow&#xA;related to cartesian coordinate system. I can't figure out how&#xA;to express {x,y,z} in terms of {e,n} and what would be the size&#xA;of the Jacobi matrix in this case (it's supposed to be square).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fqrKG.png&quot; alt=&quot;2D and 3D domains&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="188" LastEditorUserId="276" LastEditDate="2012-10-08T06:30:39.283" LastActivityDate="2012-10-08T06:30:39.283" Title="How to integrate polynomial expression over 3D 4-node element?" Tags="|finite-element|quadrature|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3440" PostTypeId="2" ParentId="3020" CreationDate="2012-10-07T07:08:13.763" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1063/1.468081&quot; rel=&quot;nofollow&quot;&gt;Baker et al.&lt;/a&gt; proposed such a mapping for a radial grid for atomic &amp;amp; molecular electronic structure computations in 1994. It is still used in modern electronic structure codes, e.g. &lt;a href=&quot;https://aimsclub.fhi-berlin.mpg.de/&quot; rel=&quot;nofollow&quot;&gt;FHI-AIMS&lt;/a&gt; uses them, as described in a &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0010465509002033&quot; rel=&quot;nofollow&quot;&gt;recent paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even with such a mapping, the same problems still remains: if something interesting should happen beyound the outermost grid point, you will miss it. However, these mappings do have the advantage that the grid can be systematically improved toward the inclusion of distant grid points. (This is explained in section 4.1 of the &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0010465509002033&quot; rel=&quot;nofollow&quot;&gt;recent FHI-AIMS paper&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2012-10-07T07:08:13.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3441" PostTypeId="2" ParentId="3439" CreationDate="2012-10-07T08:01:29.017" Score="8" Body="&lt;p&gt;You're integrating a function on a 2-dimensional manifold embedded in $\mathbb{R}^{3}$; books in analysis on manifolds (like Munkres' accessible book, or Lee's books on manifolds) are helpful in discussing the theory defining this type of integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose that $f$ is a real-valued function defined on the manifold $\mathcal{M}$, which is your 4-node 3-D element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You want to calculate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\int_{\mathcal{M}} f \,\mathrm{d}S.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that $\varphi$ is a function that maps $[-1,1]^{2}$ to $\mathcal{M}$. Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\int_{\mathcal{M}} f \,\mathrm{d}S = \int_{[-1,1]^{2}} f(\varphi(x,y))\Big(\det\big(\mathrm{D}\varphi^{\mathrm{T}}(x,y)\mathrm{D}\varphi(x,y)\big)\Big)^{1/2}\, \mathrm{d}x \,\mathrm{d}y&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I used &lt;a href=&quot;http://www.owlnet.rice.edu/~fjones/chap11.pdf&quot; rel=&quot;nofollow&quot;&gt;this set of notes&lt;/a&gt; to refresh my memory.) Above, $\mathrm{D}\varphi$ is the Jacobian matrix of $\varphi$, and $\mathrm{D}\varphi^{\mathrm{T}}$ is its transpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you can write the integral over $[-1,1]^{2}$, then you can use numerical methods to evaluate it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some comments:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I'm pretty sure your 4-node 3-D element is a manifold. If it is, the function $\varphi$ exists (by definition), is piecewise continuous (for topological manifolds), and is invertible. It's up to you to find a function with those properties.&lt;/li&gt;&#xA;&lt;li&gt;The argument above assumes that $\mathcal{M}$ is a smooth manifold, which implies that there exists a $\varphi$ that is continuously differentiable. In your case, the element you describe may not be continuously differentiable. If that is true, you could probably still partition your manifold into two smooth manifolds, and then the argument above still holds. Again, you must find $\varphi$ satisfying the properties of invertibility and continuous differentiability.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-10-08T06:27:35.410" LastActivityDate="2012-10-08T06:27:35.410" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3442" PostTypeId="1" CreationDate="2012-10-07T14:31:02.743" Score="3" ViewCount="161" Body="&lt;p&gt;Suppose $A$ is a directed graph adjacency matrix. Is there any good interpration of the $(i,j)-$entry of the matrix $(A^{32}\cdot (A^T)^{32})$ ?&lt;/p&gt;&#xA;" OwnerUserId="2131" LastActivityDate="2012-10-07T15:09:03.033" Title="multiplications of graph adjacency matrix" Tags="|matrix|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3443" PostTypeId="2" ParentId="3442" CreationDate="2012-10-07T15:09:03.033" Score="3" Body="&lt;p&gt;Start with equal edge weight, so $A$ is a boolean matrix. Think about what happens when you apply $A$ to the $k$th column of the identity, $\mathbf e_k$. Are the nonzeros in $A \mathbf e_k$ upstream or downstream of node $k$? What happens when you apply $A$ again, $A(A \mathbf e_k)$? What happens when there are two length-2 paths from node $i$ to node $k$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try the same experiment with $A^T$. You should now have an interpretation for the $k$th column of $A^{32} (A^T)^{32}$.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-10-07T15:09:03.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3444" PostTypeId="2" ParentId="1374" CreationDate="2012-10-08T16:46:08.780" Score="1" Body="&lt;p&gt;I don't know whether it helps you. Here, you'll find a link to &lt;a href=&quot;http://www.libgeodecomp.org&quot; rel=&quot;nofollow&quot; title=&quot;libgeodecomp&quot;&gt;libgeodecomp&lt;/a&gt;, a tool which employs &lt;em&gt;customizable domain decomposition techniques&lt;/em&gt; (from the site). It can be used with GPUs as far as I know. If it helps you, vote me up ;-)&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-10-08T16:46:08.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3445" PostTypeId="1" AcceptedAnswerId="3446" CreationDate="2012-10-07T18:13:14.020" Score="4" ViewCount="4077" Body="&lt;p&gt;My problem is something similar to &lt;a href=&quot;https://stackoverflow.com/questions/10258094/correct-usage-of-scipy-optimize-fmin-bfgs&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to implement a (Neural Network) Cost function, Back propogation algorithm in Python.&lt;/p&gt;&#xA;&lt;p&gt;The Neural Network has 3 layers. Hence 2 parameters to optimize (Theta1 and Theta2; depicted as T1 and T2 in the program).&lt;/p&gt;&#xA;&lt;p&gt;I tried the solution offered in above link, but doesn't work for me.&lt;/p&gt;&#xA;&lt;p&gt;my Cost Function is defined with following parameters.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def CostFunction(Thetas, X, y, Lambda = None):&#xA;&#xA;    T1 = Thetas[0]&#xA;    T2 = Thetas[1]&#xA;&#xA;    if(Lambda == None):&#xA;        Lambda = 0&#xA;    &#xA;    m = X.shape[0]&#xA;    # so on.......&#xA;    return J, [Theta1_grad, Theta2_grad]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How would I optimize the above function using &lt;code&gt;Scipy.optimize.fmin_bfgs&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I have the following code, doesn't seem to work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;initial_values = np.zeros([0]))&#xA;myargs = (Thetas ,X, y)&#xA;def decorated_cost(Thetas):&#xA;    return NeuralNetwork.CostFunction(Thetas, X, y, Lambda=1)&#xA;print scipy.optimize.fmin_bfgs(decorated_cost, Thetas , maxiter=400)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to optimize both Theta1 and Theta2 but the API only takes 1 parameter as input.&lt;/p&gt;&#xA;&lt;p&gt;Also, for some reason the above code (after a couple of iterations) the order of parameters of input to Cost function seem to change when operated by optimize function.&lt;/p&gt;&#xA;" OwnerUserId="2156" OwnerDisplayName="raul_w" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-10-09T09:05:54.477" Title="Using scipy.optimize to implement a neural network with back propagation" Tags="|optimization|python|machine-learning|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3446" PostTypeId="2" ParentId="3445" CreationDate="2012-10-08T11:45:40.717" Score="3" Body="&lt;p&gt;The objective function should return only the cost value, not the gradient.  Something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def decorated_cost(Thetas, X, y, Lambda):&#xA;    return NeuralNetwork.CostFunction(Thetas, X, y, Lambda=Lambda)[0]&#xA;&#xA;initial_values = np.array([0.1, 0.2])&#xA;print scipy.optimize.fmin_bfgs(decorated_cost, initial_values, maxiter=400, args=(X, y, 1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that I added &lt;code&gt;[0]&lt;/code&gt; after the call to &lt;code&gt;CostFunction&lt;/code&gt; in &lt;code&gt;decorated_cost&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def decorated_cost(Thetas, X, y, Lambda):&#xA;    return NeuralNetwork.CostFunction(Thetas, X, y, Lambda=Lambda)[0]&#xA;&#xA;def decorated_gradient(Thetas, X, y, Lambda):&#xA;    return NeuralNetwork.CostFunction(Thetas, X, y, Lambda=Lambda)[1]&#xA;&#xA;initial_values = np.array([0.1, 0.2])&#xA;print scipy.optimize.fmin_bfgs(decorated_cost, initial_values, maxiter=400, args=(X, y, 1), fprime=decorated_gradient)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but that is inefficient.  Instead &lt;code&gt;CostFunction&lt;/code&gt; should be split into two functions, one for the objective function and one for the gradient.&lt;/p&gt;&#xA;" OwnerDisplayName="Warren Weckesser" LastActivityDate="2012-10-08T11:45:40.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3447" PostTypeId="2" ParentId="2692" CreationDate="2012-10-09T01:54:10.403" Score="2" Body="&lt;p&gt;My answer might still be of interest for others searching large image data-sets or for Light Alchemist in particular if his thesis is running over the course of the entire year. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently found &lt;a href=&quot;http://www.cs.utexas.edu/~grauman/courses/spring2008/datasets.htm&quot; rel=&quot;nofollow&quot;&gt;this list of image data-sets&lt;/a&gt; on the University of Texas at Austin website. It's a collection of links to image data-sets provided by different Universities and Research Institutes. These data-sets can be enormous, for example the &lt;a href=&quot;http://lms.comp.nus.edu.sg/research/NUS-WIDE.htm&quot; rel=&quot;nofollow&quot;&gt;NUS-WIDE data-set&lt;/a&gt; alone contains +269K images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This won't provide you with the specific contour images you've asked for. The list does however contain many data-sets of objects on distinct backgrounds. Use edge detection algorithms as stated above and batch process the images to get the desired result.  &lt;/p&gt;&#xA;" OwnerUserId="2138" LastActivityDate="2012-10-09T01:54:10.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3448" PostTypeId="1" AcceptedAnswerId="3452" CreationDate="2012-10-09T13:48:05.110" Score="4" ViewCount="234" Body="&lt;p&gt;This is often required in the stiffness matrix evaluation. Thanks to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3439/how-to-integrate-polynomial-expression-over-3d-4-node-element&quot;&gt;this&lt;/a&gt; thread I found out how to perform integrals over 2D surface. I've tried to evaluate the integral of the single shape function and got 0.3333.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to have the expression which is being integrated to be more complicated. Namely I want to perform this integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\displaystyle\int_S {\frac{\partial{H_1(e,n)}}{\partial{x}}}^2 + {\frac{\partial{H_1(e,n)}}{\partial{y}}}^2 + {\frac{\partial{H_1(e,n)}}{\partial{z}}}^2\ ds$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivatives of the first shape function with respect to {x,y,z} are not the same as the derivatives with respect to {e,n}. I should use the inversed Jacobian matrix to translate the values of the derivatives with respect to {e,n} at the integration point to the derivatives with respect to {x,y,z}.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the Jacobian matrix isn't square in this case.&lt;/p&gt;&#xA;" OwnerUserId="188" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-10-10T01:30:25.183" Title="How to express the derivatives with respect to the initial coordinates {x,y,z} when 3D coordinate system is mapped into 2D manifold {e,n}?" Tags="|finite-element|local-coordinates|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3449" PostTypeId="2" ParentId="3435" CreationDate="2012-10-09T14:44:55.600" Score="1" Body="&lt;p&gt;I've actually used &lt;a href=&quot;http://basilisk-dbn.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BASILISK&lt;/a&gt; to solve my problem. I've written &lt;a href=&quot;https://stackoverflow.com/q/12785836/1679629&quot;&gt;python functions that compute dihedral angles&lt;/a&gt; and python module to rotate side chains according to the distribution given by basilisk.&lt;/p&gt;&#xA;" OwnerUserId="2121" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-10-09T14:44:55.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3450" PostTypeId="1" CreationDate="2012-10-09T14:53:54.670" Score="10" ViewCount="907" Body="&lt;p&gt;I'm interested in maximizing a function $f(\mathbf \theta)$, where $\theta \in \mathbb R^p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I don't know the analytic form of the function, or of its derivatives. The only thing that I can do is to evaluate the function point-wise, by plugging in a value $\theta_*$ and get a NOISY estimate $\hat{f}(\theta_*)$ at that point. If I want I can decrease the variability of these estimates, but I have to pay increasing computational costs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I have tried so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Stochastic steepest descent with finite differences: it can work but it requires a lot of tuning (ex. gain sequence, scaling factor) and it is often very unstable. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Simulated annealing: it works and it is reliable, but it requires lots of function evaluations so I found it quite slow.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So I'm asking for suggestions/idea about possible alternative optimization method that can work under these conditions. I'm keeping the problem as general as possible in order to encourage suggestions from research areas different from mine.&#xA;I must add that I would be very interested in a method that could give me an estimate of the Hessian at convergence. This is because I can use it to estimate the uncertainty of the parameters $\theta$. Otherwise I'll have to use finite differences around the maximum to get an estimate. &lt;/p&gt;&#xA;" OwnerUserId="2140" LastEditorUserId="2140" LastEditDate="2012-10-11T16:18:31.320" LastActivityDate="2015-09-25T20:05:24.153" Title="Maximizing unknown noisy function" Tags="|optimization|monte-carlo|simulation|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3451" PostTypeId="1" AcceptedAnswerId="3461" CreationDate="2012-10-09T16:05:14.243" Score="41" ViewCount="1527" Body="&lt;p&gt;In my field of research the specification of experimental errors is commonly accepted and publications which fail to provide them are highly criticized. At the same time I often find that results of numerical computations are provided without any account of numerical errors, even though (or maybe because) often questionable numerical methods are at work. I am talking about errors which result from discretization and finite precision of numerical computations etc. Sure, these error estimates are not always easy to obtain, such as in the case of hydro-dynamical equations but often it seems to result from laziness while I believe that the specification of numerical error estimates should be standard just as much as they are for experimental results. Hence my question: Are there resources which discuss in some detail the treatment of numerical errors or propose scientific standards for the specification of numerical errors which result from typical approximations such as discretization?&lt;/p&gt;&#xA;" OwnerUserId="1943" LastActivityDate="2012-11-20T16:00:01.207" Title="Scientific standards for numerical errors" Tags="|error-estimation|numerics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3452" PostTypeId="2" ParentId="3448" CreationDate="2012-10-09T16:55:55.047" Score="3" Body="&lt;p&gt;This post uses the same notation and preliminaries as &lt;a href=&quot;https://scicomp.stackexchange.com/a/3441/276&quot;&gt;an answer I wrote to an earlier post&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's again suppose that you're integrating over a smooth manifold $\mathcal{M}$, and that you have a diffeomorphism $\varphi: [-1,1]^{2} \rightarrow \mathcal{M}$  that maps $[-1,1]^{2}$ to your manifold. Since $\varphi$ is a diffeomorphism, it has the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it is continuously differentiable&lt;/li&gt;&#xA;&lt;li&gt;it is invertible&lt;/li&gt;&#xA;&lt;li&gt;its inverse is continuously differentiable&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We will need all of these properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to assume that $H_{1}$ is a scalar-valued function, and for simplicity of notation, I'm going to drop the subscript and refer to it as $H$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose further that the arguments $(e,n)$ refer to coordinates in $[-1,1]^{2}$, and the coordinates $(x,y,z)$ refer to points in $\mathbb{R}^{3}$, where $\mathcal{M} \subset \mathbb{R}^{3}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your question, it looks like you have $H: [-1,1]^{2} \rightarrow \mathbb{R}$, which is to say that the function you've stated is given in terms of the arguments $(e,n)$. We'll assume that $H$ is continuously differentiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $G: \mathcal{M} \rightarrow \mathbb{R}$ be defined by $G = H \circ \varphi^{-1}$, or $G(x,y,z) = H(\varphi^{-1}(x,y,z))$. We're using $\varphi^{-1}$ as a &quot;change of coordinates&quot; here, mapping $(x,y,z)$ coordinates of points in $\mathcal{M}$ to $(e,n)$ coordinates of points in $[-1,1]^{2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't make sense to differentiate $H$ with respect to $(x,y,z)$, but it &lt;em&gt;does&lt;/em&gt; make sense to differentiate $G$ with respect to $(x,y,z)$. I think the integral you want to express above is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\int_{\mathcal{M}} \left(\frac{\partial{G}}{\partial{x}}\right)^{2}(x,y,z) + \left(\frac{\partial{G}}{\partial{y}}\right)^{2}(x,y,z) + \left(\frac{\partial{G}}{\partial{z}}\right)^{2}(x,y,z)\,\mathrm{d}S,&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;more compactly expressed as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\int_{\mathcal{M}} \|\mathrm{D}G^{T}(x,y,z)\|^{2}\,\mathrm{d}S = \int_{\mathcal{M}} \|\nabla G(x,y,z)\|^{2}\,\mathrm{d}S.&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the chain rule,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\mathrm{D}G(x,y,z) = (\mathrm{D}H \circ \varphi^{-1})(x,y,z) \cdot \mathrm{D}(\varphi^{-1})(x,y,z),&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $\cdot$ denotes matrix multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sanity check is in order here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$\mathrm{D}H$ is a 1 by 2 matrix (the Jacobian of a scalar function with respect to multiple variables is a row matrix)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$\mathrm{D}(\varphi^{-1})$ is a 2 by 3 matrix&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$\mathrm{D}G$ is a 1 by 3 matrix&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the right hand side of the equation is a 1 by 3 matrix (the product of a 1 by 2 matrix and a 2 by 3 matrix&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given all of this information (it's quite a long post already), you should be able to use &lt;a href=&quot;https://scicomp.stackexchange.com/a/3441/276&quot;&gt;my answer to your previous post&lt;/a&gt; to calculate your integral. I can fill in the additional steps, if need be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You make an excellent point in your question that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The derivatives of the first shape function with respect to $(x,y,z)$ are not the same as the derivatives with respect to $(e,n)$. I should use the inverse Jacobian matrix to translate the values of the derivatives with respect to $(e,n)$ at the integration point to the derivatives with respect to $(x,y,z)$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If $f: X \rightarrow X$, where $X \subset \mathbb{R}^{n}$ for some natural number $n$, and $f$ is a diffeomorphism (again, continuously differentiable function with continuously differentiable inverse), it is true by the chain rule that $\mathrm{D}(f^{-1}) = \mathrm{D}f^{-1}$. The problem with this concept for your problem (as you rightly point out) is that $\varphi$ is a diffeomorphism, but doesn't have the same number of input arguments as output values, so we can't state that $\mathrm{D}(\varphi^{-1}) = \mathrm{D}\varphi^{-1}$ unless we interpret the &quot;inverse&quot; of the nonsquare matrix as some sort of generalized matrix inverse. I hope it is clear from the reasoning above how you get from the &quot;inverse matrix&quot; idea to the more general case.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-10-10T01:30:25.183" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3453" PostTypeId="2" ParentId="3451" CreationDate="2012-10-09T17:14:13.417" Score="13" Body="&lt;p&gt;Sort of. There are theoretical error bounds that have been derived by numerical analysts that are usually overestimates, and may not be as useful in practice, because they may involve information that is difficult to obtain for problems in practice. A good example would be the bounds on numerical errors in the solution of ordinary equations, which you can find in Hairer and Wanner's books. Nick Higham's book, &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; (I may be slightly off about the title) also provides some error bounds on common numerical operations and linear algebra algorithms. The numerical analysis literature is rife with such bounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interval analysis methods have also been used to calculate error bounds; these methods are rigorous, and tend to provide stronger error bounds than theoretical error bounds, but these methods may still grossly overestimate the error in a numerical calculation. These methods have been best exploited (to my knowledge) in global optimization, but are also finding use in uncertainty quantification. Arnold Neumaier has written at least one book on interval analysis methods, and is better qualified to comment on this topic in detail. In addition to potential overestimation issues, interval analysis methods suffer from requiring additional computational infrastructure that requires a retrofitting of existing large numerical simulation packages (such as PETSc, Trilinos, CLAWPACK/PyClaw, etc.) to include interval arithmetic and automatic differentiation (for Taylor-based methods). From what I've seen, there aren't many permissively licensed interval arithmetic and automatic differentiation packages out there, although there are some. Even then, sometimes, these libraries have limited functionality; it's been hard to find a permissively-licensed (LGPL, or BSD-like) interval arithmetic library with BLAS-like functionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;A posteriori&lt;/em&gt; error estimates can be obtained more easily, but aren't rigorous. I'm most familiar with these estimates from work on ordinary differential equations, but they also exist for many methods used to calculate solutions to partial differential equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More broadly, methods from uncertainty quantification, such as the use of polynomial chaos expansions, Monte Carlo methods, or other sampling methods can be used to quantify the uncertainty in calculations due to variations in input parameters. These methods should be able to provide some sort of heuristic &quot;error bar&quot; due to variations in parameters, but won't give rigorous bounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that you're absolutely right on when it comes to specification of numerical errors: computational science should be just as rigorous about presenting its results as experiment- based physical sciences. There is a great deal of work being done in this area (under the umbrella terms &quot;uncertainty quantification&quot; and &quot;numerical analysis&quot;), and it is my hope that error bars will be included when discussing most computational results at some point in the future.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-10-09T17:14:13.417" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3454" PostTypeId="2" ParentId="3450" CreationDate="2012-10-09T17:28:59.263" Score="7" Body="&lt;p&gt;Our Matlab package SnobFit was created precisely for this purpose. No assumption about the distribution of the noise is needed. Moreover, function values can be supplied through text files, thus you can apply it to functions implemented in any system able to write a text file. &#xA;See &lt;BR&gt;&#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/software/snobfit/&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/software/snobfit/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SnobFit had been developed for an application where the function to be optimized didn't even exist, and function values (a measure of manufacturing quality) were obtained by specialized, expensive equipment creating sample products and measuring these by hand, resulting in about 50 function evaluations per day.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-10-10T07:41:23.270" LastActivityDate="2012-10-10T07:41:23.270" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3456" PostTypeId="1" CreationDate="2012-10-09T17:54:07.567" Score="5" ViewCount="150" Body="&lt;p&gt;Suppose a matrix $D\in\mathbb{R}^{n\times n}$ of Euclidean distances between $n$ points is given. To obtain a Gram matrix (matrix of inner-products of points that give rise to distances in $D$), one performs $$G=-1/2\cdot PD^{(2)}P^T,$$ where $D^{(2)}$ contains squared entries of $D$, and $P=(I_n-1_nw^T)$, $w^T1_n=1$. It is known (see &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0024379585901879&quot;&gt;http://www.sciencedirect.com/science/article/pii/0024379585901879&lt;/a&gt;) that in case $w^T1_n=1$, the rank of all Gram matrices is identical, and corresponds to true dimensionality of the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I wonder what happens when $w^T1_n\neq 1$. Could there be a choice of two different $P$ for which the ranks of associated $G$ are different?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;also, why is the constraint $w^T1_n=1$ imposed?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2013-08-05T22:00:36.993" Title="Configuration shift to change the rank of a Gram matrix" Tags="|linear-algebra|matrix|high-dimensional|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3457" PostTypeId="2" ParentId="3451" CreationDate="2012-10-09T17:59:08.303" Score="20" Body="&lt;p&gt;No such standards exist, as reliable error estimates often cost &lt;em&gt;much&lt;/em&gt; more than the approximate calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically there are four kinds of error estimates: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(i) Theoretical analyses proving that a numerical method is numerically stable. This doesn't really give an error bar as the analysis only guarantees that the error made is not worse than a quantified error in the input arguments. It suffices for most scientific calculations as the inputs are also approximate only, so the error made with a numerically stable method is not worse than having used a slightly different (but unknown) input. Most highly regarded numerical methods are accompanied by a numerical stabitly analysis, though one&#xA;finds hardly any implementation that reports upon request the resulting so-called backward error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ii) Asymptotic error estimates. These assume that products of all errors (input errors, rounding errors or discretization errors being the most common sources) can be neglected (questionable if functions are very nonlinear), and use sensitivity analysis to propagate input errors. Together with a numerical stability analysis, this can also capture the effect of rounding errors or discretization errors. The resulting error bars are as realiable as the validity of the assumptions on which they are based. Using automatic differentiation tools, the cost of the error estimate is typically a factor of 1 or 2 in addition to the cost for the approximation. Thus this kind of error estimate is fairly frequent in practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit] For example, the Oettli-Prager theorem gives easily computable backward error estimates for the solution of linear systems. Sensitivity analysis tells that these errors must be multiplied by the norm of the matrix inverse, which can be estimated using Hager's estimator (built in into modern condition number estimators).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(iii) Stochastic error analysis: (CESTAC, &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0378475488900705&quot;&gt;http://www.sciencedirect.com/science/article/pii/0378475488900705&lt;/a&gt;)&#xA;This is done by overloading all operations with a corresponding stochastic variant that evaluates three sets of arguments and afterwards adds an artificial random rounding error. the final three results are used to compute a mean and a standard deviation the square root of (sum of squares of deviations from the mean divided by 2 = 3-1). This gives a fairly useful accuracy estimate of the rounding error part. This does not account of discretization error, however, which is typically the dominating error in ODE and PDE computations. The cost depends on the programming language because of the overhead in executing overloaded operations. Assuming (which is almost never the case) the overloading carries no time penalty, the cost for result plus error estimate is a factor of 3 compared to computing the approximation only. With a special hardware chip for the stochastic operation, &#xA;this method could be made as fast as evaluating the approximation itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(iv) Interval analysis: This gives rigorous bounds for all error sources if done properly, but except in simple cases it requires lots of experience (or software embodying it) to do it in a way that the bounds do not severely overestimate the true errors. Good interval software is available among others for linear algebra (e.g., IntLab &lt;a href=&quot;http://www.ti3.tu-harburg.de/rump/intlab/&quot;&gt;http://www.ti3.tu-harburg.de/rump/intlab/&lt;/a&gt;; cost a factor of about 6 if the dimension is large) and global optimization (e.g., COCONUT &lt;a href=&quot;http://www.mat.univie.ac.at/~coconut/coconut-environment/&quot;&gt;http://www.mat.univie.ac.at/~coconut/coconut-environment/&lt;/a&gt;; may be far more expensive or even be cheaper than approximate global optimization, depending on the problem features). But many other classes of problems easy to treat accurrately approximately (e.g., enclosing the trajectories of the big planets of the solar system over 10 years) are completely out of reach for the current generation of interval methods.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-11-20T16:00:01.207" LastActivityDate="2012-11-20T16:00:01.207" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3458" PostTypeId="2" ParentId="3456" CreationDate="2012-10-09T18:04:45.817" Score="1" Body="&lt;p&gt;The only formulas that give the correct Gram matrix is the one derived from the relation $d_{ik}^2=g_{ii}-2g_{ik}+g_{kk}$. The simplest version declares a point (wlog with index $0$) to be the zero point; thus $g_{0k}=0$ for all $i$, which gives $g_{kk}=d_{i0}^2$ and then $g_{ik}=(g_{ii}-d_{ik}^2+g_{kk})/2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this you can probably answer your questions by yourself.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-09T18:04:45.817" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3459" PostTypeId="2" ParentId="1310" CreationDate="2012-10-09T21:51:41.827" Score="-1" Body="&lt;p&gt;I have a simple Fortran subroutine that computes the exponent of an&#xA;arbitrary matrix. I checked it up against Matlab command and it is fine.&#xA;It is based on scaling and squaring. I wrote it a few years back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would have like to fins another subroutine, like the ones I download from&#xA;gams.nist.gov. But no luck yet.&lt;/p&gt;&#xA;" OwnerUserId="2142" LastActivityDate="2012-10-09T21:51:41.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3460" PostTypeId="2" ParentId="3451" CreationDate="2012-10-09T22:20:27.263" Score="7" Body="&lt;p&gt;In addition to the other answers, there are a few additional points to consider.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Numerical discretization errors, or at least the order of the schemes, can be determined analytically. The discussion of these errors may be omitted from papers if they use a commonly known scheme. &lt;/li&gt;&#xA;&lt;li&gt;Grid refinement studies where the same problem, usually something simple, is run on progressively finer grids. These are compared with an exact solution, or the solution on a ridiculously fine grid, to find the L-norm, typically L2. The slope of this error estimate gives the order of accuracy.&lt;/li&gt;&#xA;&lt;li&gt;In problems where different numerical schemes are available but grid refinement or exact solutions are not, another method called Richardson Extrapolation will give bounds on the error terms. A good review describing these methods can be found &lt;a href=&quot;http://www.annualreviews.org/doi/abs/10.1146/annurev.fluid.29.1.123?journalCode=fluid&quot;&gt;in this paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Lastly, each journal sets its own standards for acceptance. Some are strict, others are not. For example, the AIAA set out its standards &lt;a href=&quot;http://arc.aiaa.org/doi/abs/10.2514/1.10195?journalCode=ja&quot;&gt;here&lt;/a&gt;. Other journals have similar information for authors.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="391" LastActivityDate="2012-10-09T22:20:27.263" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3461" PostTypeId="2" ParentId="3451" CreationDate="2012-10-09T22:47:07.660" Score="27" Body="&lt;p&gt;Your question is asking about model Verification. You can find numerous resources on methods and standards by searching for Verification and Validation (&lt;a href=&quot;http://www.stanford.edu/group/uq/docs/roache.pdf&quot; rel=&quot;noreferrer&quot;&gt;Roache 1997&lt;/a&gt;, &lt;a href=&quot;http://dx.doi.org/10.1115/1.1436090&quot; rel=&quot;noreferrer&quot;&gt;2002&lt;/a&gt;, &lt;a href=&quot;http://dx.doi.org/10.1109/MCSE.2004.33&quot; rel=&quot;noreferrer&quot;&gt;2004&lt;/a&gt;, &lt;a href=&quot;http://dx.doi.org/10.1016/S0376-0421(02)00005-2&quot; rel=&quot;noreferrer&quot;&gt;Oberkampf &amp;amp; Trucano 2002&lt;/a&gt;, &lt;a href=&quot;http://prod.sandia.gov/techlib/access-control.cgi/2000/001444.pdf&quot; rel=&quot;noreferrer&quot;&gt;Salari &amp;amp; Knupp 2000&lt;/a&gt;, &lt;a href=&quot;http://dx.doi.org/10.1016/j.cma.2004.03.002&quot; rel=&quot;noreferrer&quot;&gt;Babuska &amp;amp; Oden 2004&lt;/a&gt;), as well as the broader topic of &lt;a href=&quot;http://en.wikipedia.org/wiki/Uncertainty_quantification&quot; rel=&quot;noreferrer&quot;&gt;Uncertainty Quantification&lt;/a&gt;. Rather than elaborate on methods, I would like to highlight a community that took a firm stand on the issue.&lt;/p&gt;&#xA;&lt;p&gt;In 1986, Roache, Ghia, and White established the &lt;a href=&quot;http://dx.doi.org/10.1115/1.3242537&quot; rel=&quot;noreferrer&quot;&gt;Journal of Fluids Engineering Editorial Policy Statement on the Control of Numerical Accuracy&lt;/a&gt; which opens with&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A professional problem exists in the computational fluid dynamics community and also in the broader area of computational physics. Namely, there is a need for higher standards on the control of numerical accuracy.&lt;/p&gt;&#xA;&lt;p&gt;[...] The problem is certainly not unique to the JFE and came into even sharper focus at the 1980-81 AFOSRHTTM-Stanford Conference on Complex Turbulent Flows. It was a conclusion of that conference's Evaluation Committee that, in most of the submissions to that conference, it was impossible to evaluate and compare the accuracy of different turbulence models, since one could not distinguish physical modeling errors from numerical errors related to the algorithm and grid. This is especially the case for first-order accurate methods and hybrid methods.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They conclude with very direct guidelines:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Journal of Fluids Engineering will not accept for publication any paper reporting the numerical solution of a fluids engineering problem that fails to address the task of systematic truncation error testing and accuracy estimation.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;[...] we must make it clear that &lt;strong&gt;a single calculation in a fixed grid will not be acceptable&lt;/strong&gt;, since it is impossible to infer an accuracy estimate from such a calculation. Also, the editors will not consider a reasonable agreement with experimental data to be sufficient proof of accuracy, especially if any adjustable parameters are involved, as in turbulence modeling.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;http://journaltool.asme.org/Content/JFENumAccuracy.pdf&quot; rel=&quot;noreferrer&quot;&gt;current version&lt;/a&gt; contains a comprehensive set of criteria and represents a standard that, in my opinion, other fields should aspire to match. It is shameful that even today, awareness about the importance of model verification is absent in so many fields.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-10-09T22:47:07.660" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3462" PostTypeId="1" CreationDate="2012-10-10T16:27:13.787" Score="11" ViewCount="354" Body="&lt;p&gt;Is there a published list of best practices to ensure longevity of code, with an eye towards reproducible scientific results?  (e.g. open source, documentation practices, selecting dependencies, selecting a language, virtual machines, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Know of any studies (or lacking that, examples/anecdotes) that have tried to estimate the half-life of typical scientific code or other software (if that's even a reasonable question?)  &lt;/p&gt;&#xA;" OwnerUserId="551" LastActivityDate="2012-10-11T16:23:26.170" Title="Increasing the archival longevity of code" Tags="|software|publications|reproducibility|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3463" PostTypeId="2" ParentId="661" CreationDate="2012-10-10T17:13:03.717" Score="8" Body="&lt;p&gt;Great question and great replies, but I think none addresses the question of persistence adequately, if the goal is to achieve the same standard accorded to the publication itself.  (Which may be silly given &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3462/increasing-the-archival-longevity-of-code&quot;&gt;the chances the code still runs&lt;/a&gt;, but may still be at least as useful as the publication all the same).  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;University websites journal supplements aren't persistent&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;University websites are unlikely to provide the stability or the redundancy to preserve hosted content.  Content is more difficult to cite and typically lacks machine readable metadata.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately it appears that journals are not doing much better in maintaining their supplemental materials (see &lt;a href=&quot;http://www.biomedcentral.com/1471-2105/7/260/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anderson et al. 2006&lt;/a&gt;), and may not accept the necessary formats, or even accept supplemental material at all (see &lt;a href=&quot;http://www.jneurosci.org/content/30/32/10599.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;one notable example&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reasons, folks concerned with long term archiving of &lt;em&gt;data&lt;/em&gt; have unanimously turned towards advocating for the use of dedicated repositories rather than websites or supplemental materials, and many journals now &lt;a href=&quot;http://datadryad.org/jdap&quot; rel=&quot;nofollow noreferrer&quot;&gt;mandate this practice&lt;/a&gt;.  It seems only fair that code be held to this standard.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;The solution of many copies?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Github and related sites have yet to prove longevity over the 100s of years scale achieved by university libraries and established publishers. By facilitating widespread distribution it may provide a solution others have echoed in the comments, including one fellow who couldn't comment on stackexchange,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...let us save what remains: not by vaults and locks which fence them&#xA;  from the public eye and use in consigning them to the waste of time,&#xA;  but by such a multiplication of copies, as shall place them beyond the&#xA;  reach of accident.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;— Thomas Jefferson, February 18, 1791&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;Figshare &amp;amp; the CLOCKSS standard&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The only archival standard I am aware of is &lt;a href=&quot;http://figshare.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;figshare&lt;/a&gt;, which can accept full code repositories (as &quot;filesets&quot; for the moment, but I believe will soon have the option to be listed as type &quot;code&quot;).  The key piece to figshare is not only the citable DOI with programmatic metadata, but the backing of &lt;a href=&quot;http://clockss.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CLOCKSS&lt;/a&gt; archival service, which maintains copies of all its content at 12 geographically and geo-politically distributed nodes around the world.  Should figshare go out of business or cease to exist, this will trigger all its content to be freely available from CLOCKSS.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, I'd suggest using Github for distribution of code, but also providing an archival copy to figshare at the time of publication.  &lt;/p&gt;&#xA;" OwnerUserId="551" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-10-10T17:13:03.717" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3464" PostTypeId="2" ParentId="3462" CreationDate="2012-10-10T18:59:01.317" Score="8" Body="&lt;p&gt;the planned longevity of TeX comes to mind:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;“Ever since those beginnings in 1977, the TeX research project that I embarked on was driven by two major goals. The first goal was quality: we wanted to produce documents that were not just nice, but actually the best. (…) The second major goal was archival: to create systems that would be independent of changes in printing technology as much as possible. When the next generation of printing devices came along, I wanted to be able to retain the same quality already achieved, instead of having to solve all the problems anew. I wanted to design something that would be still usable in 100 years. ” – Donald E. Knuth: Digital Typography, p. 559 (quoted from &lt;a href=&quot;http://de.wikipedia.org/wiki/TeX&quot;&gt;http://de.wikipedia.org/wiki/TeX&lt;/a&gt; )&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Based on Knuth's books about digital typography, even a complete&#xA;reimplementation of TeX and METAFONT should be possible. They include&#xA;annotations and explanations for all the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By demanding that your results should be stable over decades you get&#xA;into a kind of freezing dilemma. On one hand, you want to make it easy&#xA;reproduce your results 100%, so you freeze your software /&#xA;environment. On the other hand, someone who is interested in&#xA;reproducing your results in the future will certainly want to build on&#xA;it. This person will be stuck with very old software, making it very&#xA;hard to change anything. For anything that builds on several external&#xA;packages, already a few years are enough to make things practically&#xA;unchangeable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For TeX, freezing is announced in the 1990 article&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The future of TEX and METAFONT&#xA;  &lt;a href=&quot;http://www.ntg.nl/maps/05/34.pdf&quot;&gt;http://www.ntg.nl/maps/05/34.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;I strongly believe that an unchanging system has great value, even&#xA;  though it is axiomatic that any complex system can be improved.&#xA;  Therefore I believe that it is unwise to make further “improvements”&#xA;  to the systems called TEX and METAFONT. Let us regard these systems as&#xA;  ﬁxed points, which should give the same results 100 years from now&#xA;  that they produce today.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The ideal system would combine reproducibility with changeability.&#xA;Trying to be as self-contained, simple and well-tested as possible&#xA;certainly helps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pardon me if I was disgressing too much from the original question.&#xA;[cross posted from 'Scientists for Reproducible Research', reproducible-research@googlegroups.com ]&lt;/p&gt;&#xA;" OwnerUserId="2148" LastActivityDate="2012-10-10T18:59:01.317" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3465" PostTypeId="2" ParentId="3450" CreationDate="2012-10-11T01:49:32.487" Score="4" Body="&lt;p&gt;James Spall's SPSA algorithm (short for Stochastic Perturbation Simulated Annealing, if I recall correctly) has been designed for exactly this kind of problem. He has a couple of papers where he uses it for problems like the one you describe.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-11T01:49:32.487" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3466" PostTypeId="2" ParentId="3462" CreationDate="2012-10-11T02:05:22.590" Score="2" Body="&lt;p&gt;There have been many attempts at making reproducibility happen and there is a whole literature on this topic. My personal opinion from 15 years of scientific software is that it's unrealistic, as unsatisfactory as I find that answer. The problems are that (i) complex software has bugs and so can't be frozen; (ii) software is never feature complete and so development continues; (iii) what is the value of delivering with a paper several hundred thousands of lines of code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I say, I find this answer unsatisfactory. I believe that as a field, computational science has not been very successful in producing literature that instills trust that the results we publish are correct and reproducible. At the same time, I can't really come up with ways to do things better. For sure, releasing source code that goes with a paper is useful. At the same time, everyone who is honest will agree that the results in a paper will typically be produced by different versions of the code that, in most cases contain hacks describing different boundary conditions, different right hand sides, etc. A paper would then come with different versions of the same code. This is awkward for the reader to begin with, but it is outright unproductive if the code is large as it often happens today -- my two most recent papers used codes that are about 20,000 lines of code and that build on deal.II (600,000 lines of code) and Trilinos (1.5M lines of code). What information does that provide to a potential reader? (I should say that my codes are nevertheless available.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-11T02:05:22.590" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3467" PostTypeId="2" ParentId="3462" CreationDate="2012-10-11T03:18:00.783" Score="4" Body="&lt;p&gt;There are many technical challenges that make exact bit-for-bit reproducibility of computational results extremely hard to achieve.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the software level, changes to the code &lt;em&gt;or any of the libraries used by the code&lt;/em&gt; can obviously cause different results to be produced.  You'd be surprised by the number of support libraries that can end up linked into a typical scientific code.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a lower level, recompiling any of the code or any of the libraries used by the code with a new compiler or with different compiler optimizations turned on can also cause problems.  One reason is that various operations in the code might be performed in a different order when the code is recompiled.  Since floating point addition is not associative (a+b)+c &amp;lt;&gt; a+(b+c), this can give different results.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK, so what if we preserve the entire software environment (OS, libraries, and compiled code) by (e.g.) burning it on to a bootable CD-Rom that will run the code.  Now can we be sure that we'll get the same results if we run this code on a different computer?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Surprisingly, some codes actually vary the order of computations based on aspects of the particular processor model that they're running on.  For example, optimized linear algebra libraries typically break up matrix multiplications to work on blocks that will fit into cache.  When Intel releases a new microprocessor with a bigger cache the code might dynamically adjust the block size, resulting in arithmetic that is performed in a different order and giving different results.  Other codes dynamically adjust the order of computations based on the amount of available memory- if you run the code on a computer with more memory that could well cause the arithmetic to be done in a different order and thus give different results.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things get amazingly more complicated when you throw in multithreaded code, since the exact execution history of the different threads is often non-deterministic and this can again cause arithmetic operations to be performed in a different order from one run to the next.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice the most that you can really hope for are results that are similar from one machine to the next, up to the accuracy tolerances of the algorithms used.  e.g. if I have a root finding problem and use bisection to get a root to within +-1.0e-10, then I should be happy as long as different machines are producing answers that agree within that tolerance.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2012-10-11T03:18:00.783" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3470" PostTypeId="1" AcceptedAnswerId="3473" CreationDate="2012-10-11T10:27:16.313" Score="4" ViewCount="5398" Body="&lt;p&gt;To compute the eigenvector corresponding to a dominant eigenvalue of a matrix $A\in\mathbb{R}^{n\times n}$, one could apply the Power Iteration: $$v_1=\frac{Av_1}{\|Av_1\|}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) in case $A$ is symmetric, eigenvectors are orthonormal. However, suppose that there are, e.g, two occurrences of the dominant eigenvalue $\lambda_1$ corresponding to different eigenvectors. Does that mean that the method would yield inconsistent results on different invocation? The &quot;inconsistency&quot; means that the method could (assuming random initialization on each invocation) change direction of convergence (since the eigenvalues are the same)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case one needs the following dominant eigenvector, one usually performs Gram Schmidt orthonormalization, ie, removes component of the first eigenvector from the initialization to the second. Would this second vector converge to the eigenvector corresponding to &quot;the other&quot; occurence of dominanant eigenvalue $\lambda_1$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) in case of a general $A$, eigenvectors are not orthonormal. So, what would be the way to extract subsequent eigenvectors. In other words, would GS orthonormalization now make sense? It removes component from the first eigenvector, but, since the eigenvectors are not orthogonal, I'm not sure it the following matrix-vector multiplication adds the component back.&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-10-11T13:31:50.670" LastActivityDate="2012-11-01T16:49:31.393" Title="Power Iteration on general matrices (with higher multiplicity of dominant eigenvalue)" Tags="|linear-algebra|matrix|eigensystem|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3471" PostTypeId="2" ParentId="3462" CreationDate="2012-10-11T11:09:24.077" Score="2" Body="&lt;p&gt;For a possible solution to this problem, see my &lt;a href=&quot;http://dirac.cnrs-orleans.fr/plone/software/activepapers&quot; rel=&quot;nofollow&quot;&gt;ActivePapers&lt;/a&gt; project. In summary, it describes how data and code can be packaged together with explicit dependencies on specific versions of each software component. This makes it possible to exactly reproduce a computation, while also permitting to run updated software on the same data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should add that ActivePapers is no more than a proof of concept and unlikely to be of any practical use in the near future. The reason is that it is based on the principle that all executable code must exist as JVM bytecode. At the moment, this excludes too many popular scientific libraries. However, once reproducibility is recognized as important, priorities in programming tools may change as well.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2012-10-11T11:09:24.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3472" PostTypeId="1" AcceptedAnswerId="3474" CreationDate="2012-10-11T12:53:44.533" Score="1" ViewCount="119" Body="&lt;p&gt;I have a 6-sided equilateral polyhedral cone in $R^3$  defined by a symmetric set of &#xA;equations: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C=\{(X_1,X_2,X_3)\ |\ K X_i\leq X_j \forall i,j\in\{1,2,3\}\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given an point $P=(P_1,P_2,P_3)$, I would like to know what is the closest point on the cone to $P$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will need to implement this as an algorithm, so the more elegant a solution the better. &lt;/p&gt;&#xA;" OwnerUserId="898" LastActivityDate="2012-10-11T13:52:21.877" Title="closest point to a polyhedral cone" Tags="|closest-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3473" PostTypeId="2" ParentId="3470" CreationDate="2012-10-11T13:43:18.050" Score="5" Body="&lt;p&gt;1) In case of a multiple dominant eigenvalue (and no other of the same absolute value), the power itieration converges to the vector obtained by projecting the starting vector to the dominant eigenspace (if this vector is nonzero). These projections are orthogonal if the matrix is symmetric. Of course, if you start with different starting vectors you'll typically get different such projections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If the matrix is nondefective, the starting vector can be written in a unique way as a linear combination of eigenvectors to distinct eigenvalues. In this decomposition, it is easy to see what happens when you iterate. If the matrix is nondefective, the result is the same but the proof needs an additional limiting step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit1] Note that in the nonsymmetric, nondefective case, the left and right eigenvectors form a biorthogonal system, and one must orthogonalize with a left eigenvector to get a particular right eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit2]&lt;BR&gt;&#xA;3) If the matrix has precisely two dominant eigenvalues, each of algebraic multiplicity 1, one has convergence if and only the starting vector is&#xA;orthogonal to exactly one of the corresponding left eigenvectors, and then converges to the other. I leave it as an exercise to figure out what happens in the other degenerate cases possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why are you so concerned about the power iteration? it is generally a poor method, and it fails to converge if there are two different eigenvalues with (equal) maximal absolute value. Lanczos (in the symmetric case) or Arnoldi (in the nonsymmetric case) are far better.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-11-01T16:49:31.393" LastActivityDate="2012-11-01T16:49:31.393" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3474" PostTypeId="2" ParentId="3472" CreationDate="2012-10-11T13:52:21.877" Score="1" Body="&lt;p&gt;You can minimize the squared distance to $X$ subject to the six linear constraints. this is a strictly convex quadratic program. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If calling a general-purpose convex QP-solver (such as my MINQ in Matlab) is too slow, you can work out the Kuhn-Tucker optimality conditions (here necessary and sufficient), and make a case analysis. Because of symmetry you only need to distinguish three cases: Solution at the vertex, on a ray, and on a face. Then you can program it, taking also account of the need to permute the points, giving additional cases (but just permuted solution formulas).&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-11T13:52:21.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3475" PostTypeId="2" ParentId="3462" CreationDate="2012-10-11T14:57:20.183" Score="1" Body="&lt;p&gt;I believe that as far as the choice of language goes, using a standardized one (e.g. C/Fortran/C++) would qualify as &quot;best practice&quot;. If a package depends on 10 other libs/packages, especially those written in obscure languages then that obviously is bad for longevity. Many projects end up being orphaned after some time. I dont think major libs/api's like BLAS/LAPACK, PETSc, FFTW, MPI etc. would disappear anytime soon. BLAS is already pretty old.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following piece of code (stolen from &lt;a href=&quot;http://www.math.utah.edu/software/c-with-fortran.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.utah.edu/software/c-with-fortran.html&lt;/a&gt;) predates Fortran 77, uses Hollerith constants for char manipulation but compiles just fine 40-50 years later with the GNU Fortran Compiler:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;stali@x61:~$ cat olde.f&#xA;&#xA;       CALL S(12HHello, world, 12)&#xA;       END&#xA;       SUBROUTINE S(MSG,N)&#xA;       INTEGER K, N, M&#xA;       INTEGER MSG(1)&#xA;       M = (N + 3) / 4&#xA;       WRITE (6,'(20A4)') (MSG(K), K = 1,M)&#xA;       END&#xA;&#xA;stali@x61:~$ gfortran -std=legacy olde.f; ./a.out&#xA;Hello, world&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Open sourcing/putting it up somewhere like googlecode which is less likely to disappear soon (though they did shut down code search) is a no brainer.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-10-11T16:23:26.170" LastActivityDate="2012-10-11T16:23:26.170" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3476" PostTypeId="2" ParentId="3081" CreationDate="2012-10-11T18:15:54.717" Score="2" Body="&lt;p&gt;You may consider useful this reference: &lt;strong&gt;A Homotopy for Solving Large, Sparse and Structured Fixed Point Problems. R. Saigal. Mathematics of Operations Research, Vol. 8, No. 4 (Nov., 1983), pp. 557-578.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2155" LastActivityDate="2012-10-11T18:15:54.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3477" PostTypeId="2" ParentId="661" CreationDate="2012-10-11T19:31:29.510" Score="0" Body="&lt;p&gt;University libraries could be a place for this or the hosting center of the university.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-10-11T19:31:29.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3478" PostTypeId="1" CreationDate="2012-10-11T22:39:07.653" Score="15" ViewCount="544" Body="&lt;p&gt;I'm writing a reproducible paper, and the paper has computational results that are generated by a Python script (a similar MATLAB script generates nearly identical results). I feel that the paper would be easier to understand for readers if they could match up the calculations in the paper with calculations in the code. The work proposes an abstract formalism, and the examples in the paper are supposed to make this formalism more concrete for readers (many of whom will be engineers); the code will probably be the most detailed record of how to perform the calculations, and making it clear could help us during the review process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any suggestions on how to make the correspondence between code and computational results (figures, equations) more clear?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, I was thinking that when it came to lines of code implementing various steps in the paper, I could cite equation numbers (it would be amazing if I could cross reference between the code and LaTeX, but hand-labeling them is fine), and I could write functions corresponding to the various examples and figures, such as&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def example_1():&#xA;    # Insert code corresponding to first example&#xA;    pass&#xA;&#xA;def figure_1():&#xA;    # Insert code that generates Figure 1&#xA;    pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the code were large, and I weren't trying to explain how a bunch of different mathematical methods used in engineering were actually the same, I probably wouldn't bother so much with making the code clear, but given the abstract nature of the paper and the small code base, it seems as though there could be value in this exercise.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="414" LastEditDate="2019-04-10T13:58:40.863" LastActivityDate="2019-04-10T13:58:40.863" Title="What would be the most helpful way to write code for a paper so that readers can clearly match the results to the code that generates them?" Tags="|python|software|reproducibility|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3479" PostTypeId="2" ParentId="3478" CreationDate="2012-10-12T01:31:38.427" Score="7" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You might consider writing the entire paper in &lt;a href=&quot;http://www.cs.tufts.edu/~nr/noweb/&quot;&gt;Noweb&lt;/a&gt;. It's a bit tedious to set up, but it's a very powerful way to mix code and LaTeX-formatted text, equations, and figures. For long programs, it tends to turn your code into more of a book than an article, but for short programs, it might work out pretty well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you don't want to go that far, it still ought to be reasonably straightforward to format the comment sections of your code listings using LaTeX. The &lt;code&gt;listings&lt;/code&gt;package can help you pull this off. Here's a short example:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;&#xA;\documentclass{article}&#xA;\usepackage{amsmath}&#xA;\usepackage{listings}&#xA;\begin{document}&#xA;\begin{equation}&#xA;  \label{eq:one}&#xA;  Ax=b&#xA;\end{equation}&#xA;\begin{lstlisting}[escapechar=\%]&#xA;  # Comment with a reference to Equation%~\eqref{eq:one}%&#xA;  def f(a):&#xA;    return a+1&#xA;\end{lstlisting}&#xA;\end{document}&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With some additional manipulations, you ought to be able to get your referenced equation numbers to appear in the monospace font that it uses for listing the equation.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-10-12T01:31:38.427" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3480" PostTypeId="1" CreationDate="2012-10-12T02:55:20.260" Score="4" ViewCount="2487" Body="&lt;p&gt;As far as I can tell, finite volume methods are primarily for eulerian reference frames (such as in fluid dynamics simulations) while finite element methods are easily ameanable to lagrangian reference frames (such as in solid mechanics simulations).  I've always taken it for granted, but I never really thought about why this is the case.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is it about finite volume method that makes it favorable for eulerian but not lagrangian frames of references?  Conversely, what is it about the lagrangian frame of reference that make finite element methods better suited than finite volume methods?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-10-12T03:30:32.290" Title="Finite Volume Method vs. Finite Element Method for Eulerian and Lagrangian Reference Frames" Tags="|finite-element|finite-volume|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3481" PostTypeId="2" ParentId="3480" CreationDate="2012-10-12T03:30:32.290" Score="4" Body="&lt;p&gt;Where did you get that impression? I think your observation is more a side-effect of the classical problem domains for each class of methods. Classical applications of FEM, such as large-deformation elasticity, are indeed well-suited to Lagrangian formulations, while many classical applications of FVM, such as fluid dynamics, are most commonly treated in Eulerian reference frames. Even so, there have been many publications on Eulerian FEM methods for problems with moving boundaries (e.g., immersed boundary), as well as moving mesh methods with finite volume methods. Note that in general, ALE methods are expected to satisfy a geometric conservation law (GCL). The GCL is perhaps more natural to enforce in a FV setting, but FE is usually more forgiving of inexact satisfaction. Note that FV methods are commonly used in turbo-machinery, which usually has multiple moving meshes (e.g., one fixed and one rotating) with fluxes evaluated across interfaces between the moving meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, methods that conform to interfaces allow use of fewer degrees of freedom to accurately represent the nonsmoothness at the interface than interface tracking methods. Thus, when it is practical to conform to an interface, that is often preferable. FE methods for solid mechanics are almost always written for unstructured meshes, and the interfaces usually do not become too deformed. We may simulate until some structure fails, but many analyses do not continue simulating the dynamic crumpling after failure. In the fluids world, interfaces more frequently change topology very irregularly, such as when simulating vigorously rising bubbles. Meshing all those interfaces and transfering solutions to new meshes would be a headache, so is usually avoided. Other interfaces, such as along the edge of a wing on an aircraft, are extremely anisotropic (aspect ratio up to $\sim 10^6$) and either static or with static topology (though certainly moving in applications such as aeroacoustics). CFD simulations usually conform to boundaries like the wing, but use interface capturing or cut-cell technology for interfaces like the rising bubbles.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-10-12T03:30:32.290" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3482" PostTypeId="1" AcceptedAnswerId="3489" CreationDate="2012-10-12T08:45:18.590" Score="5" ViewCount="556" Body="&lt;p&gt;I know that long back Dr. Leszek Demkowicz finite elements codes(1Dhp,2Dhp,3Dhp) were available in his website. I'm finding it difficult to locate it now. Is there any alternatives available to these codes in FORTRAN(77/90) only. &lt;/p&gt;&#xA;" OwnerUserId="1106" LastEditorUserId="123" LastEditDate="2012-10-12T13:13:36.347" LastActivityDate="2012-10-14T00:39:29.353" Title="Where to find Leszek Demkowicz's finite element codes or alternatives?" Tags="|finite-element|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3483" PostTypeId="2" ParentId="3482" CreationDate="2012-10-12T12:10:41.183" Score="2" Body="&lt;p&gt;I think the code was also distributed with his books. But you may just want to write him an email :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-12T12:10:41.183" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3484" PostTypeId="2" ParentId="3482" CreationDate="2012-10-12T13:11:48.420" Score="2" Body="&lt;p&gt;Dr. Demkowicz's codes are pretty unique, so you aren't going to find a complete alternative, in any language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are just looking for a Fortran code, you can get &lt;a href=&quot;http://www.ce.berkeley.edu/projects/feap/feappv/&quot; rel=&quot;nofollow&quot;&gt;FEAPpv&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I put this &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_finite_element_software_packages&quot; rel=&quot;nofollow&quot;&gt;Wikipedia page&lt;/a&gt; together a long time ago  and someone liste &lt;a href=&quot;http://www.csc.fi/english/pages/elmer&quot; rel=&quot;nofollow&quot;&gt;Elmer&lt;/a&gt;.  I have no experience using it.&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2012-10-12T13:11:48.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3485" PostTypeId="2" ParentId="3482" CreationDate="2012-10-13T07:03:00.690" Score="0" Body="&lt;p&gt;I have these codes on my computer (files: tar1d.tgz, tar2d.tgz, and tar3d.tgz). I downloaded them from his website a year ago, or so.  I just like collecting codes, I'm also looking for some (if someone could help me find Orszag's CENTIKUBE it would be great).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may contact me and I'll forward them to you.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-10-13T07:03:00.690" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3486" PostTypeId="1" AcceptedAnswerId="3491" CreationDate="2012-10-13T11:45:51.537" Score="14" ViewCount="6312" Body="&lt;p&gt;Suppose we have a linear system and we know nothing about its conditioning and have no preliminary information about the solution. We blindly apply Gaussian elimination and obtain some solution $x$. Is it possible to determine whether this solution is trustworthy (i.e. that the system is well-conditioned) without &lt;a href=&quot;https://scicomp.stackexchange.com/q/34/720&quot;&gt;thorough preliminary analysis of the matrix&lt;/a&gt;? Does the magnitude of pivots give reliable information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And generally, what are the main guidelines for detecting ill-conditioning &quot;on the fly&quot;?&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2018-06-21T15:43:50.077" Title="What are the symptoms of ill-conditioning when using direct methods?" Tags="|linear-solver|condition-number|conditioning|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3487" PostTypeId="2" ParentId="3486" CreationDate="2012-10-13T13:25:08.130" Score="5" Body="&lt;p&gt;It is nearly impossible to to tell if the your system is ill conditioned from just one result.  Unless you have some foresight into the behavior of your system (i.e. know what the solution SHOULD be), there's not much you can say from a single solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said this, you can gain more information if you solve more than one system with the same $A$.  Suppose you have a system of the form $Ax=b$. For a specific A which you have no prior knowledge about its conditioning, you can perform the following test:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve $Ax=b$ for a specific right hand side vector $b$.  &lt;/li&gt;&#xA;&lt;li&gt;Perturb your right hand side vector by $b_{new}=b+\mathbf{\varepsilon}$ where $||\mathbf{\epsilon}||$ is very small in comparison to $||b||$.&lt;/li&gt;&#xA;&lt;li&gt;Solve $Ax_{new}=b_{new}$.&lt;/li&gt;&#xA;&lt;li&gt;If your system is well-conditioned, your new solution should be fairly close to your old solution (i.e. $||x-x_{new}||$ should be small).  If you observe a dramatic change to your new solution (i.e. $||x-x_{new}||$ is large), then your system is probably ill-conditioned.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You may need to solve several linear systems with different right hand side vectors to give you a better indication of whether the system is ill-conditioned.  Of course, this process is a bit expensive ($\Theta(n^3)$operations for the first solution and $\Theta(n^2)$ operations for each successive solution, assuming your direct solver saves its factors).  If your matrix A is fairly small, this is not a problem.  If it is large, you may not want to do this.  Instead, you may be better off calculating the condition number $||A||\cdot||A^{-1}||$ in a convenient norm. &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-10-14T13:18:19.450" LastActivityDate="2012-10-14T13:18:19.450" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3488" PostTypeId="1" AcceptedAnswerId="3490" CreationDate="2012-10-13T23:27:51.890" Score="2" ViewCount="633" Body="&lt;p&gt;I'm trying to find the best approximation to the function $e^x$ in the finite dimensional polynomial space $P_4$ with respect to the standard basis vectors $B=\{1,x,x^2,x^3,x^4\}$ with inner product $$(f,g)=\int_0^1 \frac{f(x)g(x)}{\left(x(1-x)\right)^{\frac{1}{2}}}dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $w=\sum_{j=0}^4a_j\phi_j $ where $\phi_j=x^j$.  Then, the best approximation of $w$ to $e^x$ satisfies the orthogonality property:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(e^x-w,\phi_i)=0$$ for $i=0,...,4$.  Hence,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j=0}^4 a_j (\phi_j,\phi_i) = (e^x,\phi_i)$$ for $i=0,...,4.$.  This generates linear system of equations which I'm trying to solve.  Unfortunately, the integrals are a bit difficult to evaluate by hand.  So, I tried using a three point gaussian quadrature rule to approximate the integrals and solve the system.  Unfortunately, I keep getting a nearly singular system.  When I observed this, I tried using a higher order quadrature rule, but the matrix remains ill-conditioned.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked at my code over and over again and I can't seem to find any errors in the implementation.  I'm tempted to believe that these inner products simply can't be evaluated well with a gaussian quadrature rule.  Is this common for Gram Matrices?  Should I use a different quadrature rule?  Or is it a bug in my code?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have provided my matlab code assembing this matrix system below for reference.  Any help would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n=5;&#xA;%Quadrature points on [-1,1]&#xA;z1=-sqrt(3/5);&#xA;z2=0;&#xA;z3=sqrt(3/5);&#xA;&#xA;%Quadrature points on [0,1]&#xA;x1=z1/2+1/2;&#xA;x2=z2/2+1/2;&#xA;x3=z3/2+1/2;&#xA;&#xA;A=zeros(n,n);&#xA;b=zeros(n,1);&#xA;&#xA;%assemble gramm matrix&#xA;for i=1:n&#xA;    for j=1:n&#xA;        F1=x1^(i-1+j-1)/sqrt(x1*(1-x1));&#xA;        F2=x2^(i-1+j-1)/sqrt(x2*(1-x2));&#xA;        F3=x3^(i-1+j-1)/sqrt(x3*(1-x3));&#xA;        A(i,j)=(1/2)*((5/9)*F1+(8/9)*F2+(5/9)*F3);&#xA;    end&#xA;    G1=x1^(i-1)*exp(x1)/sqrt(x1*(1-x1));&#xA;    G2=x2^(i-1)*exp(x2)/sqrt(x2*(1-x2));&#xA;    G3=x3^(i-1)*exp(x3)/sqrt(x3*(1-x3));&#xA;    b(i)=(1/2)*((5/9)*G1+(8/9)*G2+(5/9)*G3);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2012-10-14T03:03:39.330" LastActivityDate="2012-10-15T08:29:34.580" Title="Ill-conditioned Gram Matrix Assembly" Tags="|finite-element|condition-number|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3489" PostTypeId="2" ParentId="3482" CreationDate="2012-10-14T00:39:29.353" Score="6" Body="&lt;p&gt;Prof. Demkowicz has asked me to post the following reply (I have slightly edited it so that fits better with the site's format):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;1D and 2D codes come with &lt;a href=&quot;http://books.google.com/books/about/Computing_with_hp_ADAPTIVE_FINITE_ELEMEN.html?id=Xc3yGiekgEUC&quot;&gt;the first volume of my book&lt;/a&gt;. I can E-mail both codes following a &lt;a href=&quot;http://www.ices.utexas.edu/people/80/&quot;&gt;personal request&lt;/a&gt; but it is impossible to read them w/o my book...&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;The (old) 3D code has been described in &lt;a href=&quot;http://books.google.com/books/about/Computing_with_hp_ADAPTIVE_FINITE_ELEMEN.html?id=03B-5AH7D10C&quot;&gt;the second volume of our books&lt;/a&gt;. It is still in use by my colleagues in Cracow including Dr. Rachowicz. Please  &lt;a href=&quot;https://www.ices.utexas.edu/people/59/&quot;&gt;contact him directly&lt;/a&gt; for a working copy of that code. Again, it is impossible to use that code w/o reading the book first.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;We have a new, very powerful 3Dhp code (developed by current students) that it is accessible to my collaborators. Interested parties should get in touch with me directly...&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15" LastActivityDate="2012-10-14T00:39:29.353" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3490" PostTypeId="2" ParentId="3488" CreationDate="2012-10-14T03:31:57.623" Score="5" Body="&lt;h3&gt;Preliminary observations&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have a basis $\{\phi_i(x)\}$ of size $n$, you must have at least $n$ quadrature points. You can see this by writing the inner product as $(u,v) = u^T B^T W B v$ where $W$ is a diagonal weight matrix and $B_{ij} = \phi_j(x_i)$ is the basis evaluation matrix. If there are fewer than $n$ quadrature points, $B$ has fewer than $n$ rows, thus $\DeclareMathOperator{\rank}{rank} \rank(B^T W B) &amp;lt; n$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your weight function cannot be integrated exactly using Gauss quadrature, depending on the order and desired accuracy, may have unacceptable errors. You can find papers on special quadratures for all sorts of specific functional forms. Note that there is also quadrature error integrating the right hand side. Fortunately, these functions are smooth and the weight effectively asks for more points near the endpoints, so Gauss quadrature is not &lt;em&gt;that&lt;/em&gt; bad. See &lt;a href=&quot;http://eprints.maths.ox.ac.uk/1116/1/NA-06-07.pdf&quot; rel=&quot;nofollow&quot;&gt;Trefethen (2008) &lt;em&gt;Is Gauss Quadrature Better Than Clenshaw-Curtis&lt;/em&gt;&lt;/a&gt; for more on generic quadratures.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The representation $A = B^T W B$ is a more convenient way to write your code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It is considered good Matlab style to use matrix and vector notation when convenient. It substantially compresses and simplifies the code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Refactored and working code&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 5;&#xA;qpoints = 10;&#xA;&#xA;% Quadrature points on [-1,1]; see Golub &amp;amp; Welsch (1969) and Trefethen (2008)&#xA;beta = .5 ./sqrt(1-(2*(1:qpoints-1)).^(-2)); % 3-term recurrence coefficients&#xA;T = diag(beta,1) + diag(beta,-1);            % Jacobi matrix&#xA;[V,D] = eig(T);                              % eigenvalue decomposition&#xA;[x,i] = sort(diag(D));                       % Gauss-Legendre points&#xA;w = 2*V'(i,1).^2;                            % weights&#xA;% Map quadrature to interval [0,1]&#xA;x = 0.5 + 0.5*x; w = 0.5*w;&#xA;&#xA;% assemble Gram matrix&#xA;B = fliplr(vander(x,n));         % basis evaluation matrix&#xA;W = diag(w ./ sqrt(x.*(1-x)));   % diagonal quadrature weight matrix&#xA;A = B'*W*B;                      % Gram matrix&#xA;b = B'*W*exp(x);                 % exponential integrated against test functions&#xA;&#xA;taylor = 1./factorial(0:n-1)';&#xA;ceoffs = [A \ b, taylor]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Further observations&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The monomial basis is notoriously ill-conditioned. If you would like to work with higher degree polynomials, you should choose a well-conditioned basis such as (a) Legendre/Chebyshev polynomials, (b) Lagrange interpolants on good point sets like Gauss-Lobatto, or (c) Newton-form (divided differences) polynomials associated with a good point set. Of these, (b) is the most popular in PDE computations.&lt;/li&gt;&#xA;&lt;li&gt;The computed coefficients do not match the leading terms in the exponential because they are weighted-$L^2$ optimal on the interval $[0,1]$ rather than only in the limit as $x \to 0$. In contrast, the Taylor polynomials only apply to the $x\to 0$ limit and provide no bounds on a finite interval in any norm.&lt;/li&gt;&#xA;&lt;li&gt;You can produce optimal polynomials with respect to non-smooth norms such as $L^\infty(\Omega)$. Approximation in $L^\infty$, sometimes called &quot;Chebyshev approximation&quot;, gives you optimal pointwise error bounds across the entire interval. &lt;a href=&quot;http://www.pvk.ca/Blog/2012/10/07/the-eight-useful-polynomial-approximations-of-sinf-3/&quot; rel=&quot;nofollow&quot;&gt;This blog post&lt;/a&gt; has some nice figures, an introduction to the algorithms, and further links to the literature.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-10-14T04:57:43.787" LastActivityDate="2012-10-14T04:57:43.787" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3491" PostTypeId="2" ParentId="3486" CreationDate="2012-10-14T09:35:10.307" Score="13" Body="&lt;p&gt;When is a matrix &lt;em&gt;ill conditioned&lt;/em&gt;? It depends on the accuracy of the solution you are looking for, as much as &quot;beauty is in the eye of the beholder&quot;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;May be your question should better rephrased as &lt;strong&gt;are there cheap and robust condition number estimators based on the $LU$ factorization?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you are interested in the real general (dense, non symmetric) problem in double precision arithmetic I would suggest you to use LAPACK expert solver &lt;a href=&quot;http://www.netlib.org/lapack/double/dgesvx.f&quot;&gt;DGESVX&lt;/a&gt; which provides a condition estimate in the form of its reciprocal, $\text{RCOND}\approx 1/\kappa(A)$. As a bonus you have also other goodies like equation equilibration/balancing, iterative refinement, forward and backward error bounds. By the way, pathological ill conditioning ($\kappa(A) &amp;gt; 1/\epsilon$) is signaled as an error by &lt;code&gt;INFO&amp;gt;0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going into more detail, LAPACK estimates the condition number in the 1-norm (or $\infty$-norm if you are solving $A^T x = b$) via &lt;a href=&quot;http://www.netlib.org/lapack/double/dgecon.f&quot;&gt;DGECON&lt;/a&gt;. The underlying algorithm is described in lawn 36: &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn36.pdf&quot;&gt;&quot;Robust Triangular Solves for Use in Condition Estimation&quot;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to confess that I'm not an expert in the area, but my philosophy is: &quot;if it is good enough for LAPACK, it is for me&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2012-10-14T09:35:10.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3492" PostTypeId="1" AcceptedAnswerId="3494" CreationDate="2012-10-14T19:45:51.193" Score="3" ViewCount="158" Body="&lt;p&gt;Any error analysis is based on the Taylor expansions. So, if I take a finite difference scheme, I can calculate the value of the function at any point using the known value at another node via Taylor. However, if I am &quot;at the edge&quot; of the domain, the value of the function is fixed by the boundary condition, so how to deal with that? For example, inside of the domain I can say $$\frac{f(x+h)-f(x-h)}{2h}=f'(x)+Ch^2$$ So my question is how the boundary condition affects the Taylor expansion for the calculation of the local error?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1552" LastEditDate="2012-10-15T00:20:09.510" LastActivityDate="2012-10-15T11:37:06.353" Title="Effect of boundary condition on the local error" Tags="|pde|finite-difference|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3493" PostTypeId="2" ParentId="3492" CreationDate="2012-10-14T20:58:20.400" Score="2" Body="&lt;p&gt;You can use the boundary conditions very easy in your scheme. If you have Dirichlet boundary conditions, which means specifying the values at the boundaries, you just use them instead of $f(x+h)$ or $f(x-h)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The accuracy of your scheme might degrade only if you have complex formed boundaries (i.e. curvy boundaries in 2D). Then you must use a Shortley-Weller scheme, which is only first order accurate.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastEditorUserId="1738" LastEditDate="2012-10-14T21:03:31.143" LastActivityDate="2012-10-14T21:03:31.143" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3494" PostTypeId="2" ParentId="3492" CreationDate="2012-10-14T23:24:37.983" Score="3" Body="&lt;p&gt;As you correctly pointed out, the two point central scheme for FD is second order local, which makes it first order globally.  The important point I think you are missing, is that in a FD scheme with specified values at the edges of your spacial mesh, you are &lt;em&gt;always&lt;/em&gt; inside the domain.  You should never have to approximate a first derivative at the boundary since the value is known.  If you aren't evaluating your DE at the boundary, you don't have to worry about your error since there isn't any.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you have second order spacial derivatives, for other types of boundary conditions, things can get more complicated since you have to make sure a biased scheme at the edge is at least as high of order as your interior schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Looking at your comments on other answers, here is where I think you are being hung up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you are analyzing a standard interior node, you end with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{f(x+h)-f(x-h)}{2h}=f'(x)+O(h^2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so you have 2nd-order local error.  You got to this point by using the taylor expansion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x+h) = f(x) + f'(x)h + \frac{f''(x)h^2}{2} + H.O.T.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a similar expansion for the left hand neighbor.  Now imagine you are at a node directly next to the boundary where you have a specified function value.  The expansion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g =f(x+h) = f(x) + f'(x)h + \frac{f''(x)h^2}{2} + H.O.T.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;still&lt;/em&gt; holds true.  Your previous analysis still applies with the added information, because your approximation of the first derivative assumed you could accurately evaluate the function at the left and right neighbors.  The fact that you are given $g$ is irrelevant, because you are already acting like you were given $f(x+h)$ and $f(x-h)$.  Hopefully this is a bit clearer for you.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2012-10-15T11:37:06.353" LastActivityDate="2012-10-15T11:37:06.353" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3495" PostTypeId="1" AcceptedAnswerId="3501" CreationDate="2012-10-15T07:33:00.173" Score="2" ViewCount="223" Body="&lt;p&gt;I want to understand how does the simulation identifies and models when two bodies are colliding with each other. For example car crash simulation. The car moves towards the wall or another crash. Until they collide some equations of moving works, but how do they change and what are the methods of dealing with the situation when the collision occures? Do the algorithms for collision detection for games work the same way (physical engines etc)?&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="123" LastEditDate="2012-10-18T09:11:15.857" LastActivityDate="2012-10-18T09:11:15.857" Title="How are collisions detected in simulations?" Tags="|education|simulation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3496" PostTypeId="2" ParentId="3488" CreationDate="2012-10-15T08:29:34.580" Score="2" Body="&lt;p&gt;The gram matrix of the powers is known to be very ill-conditioned. It is far better if you expand in terms of Chebyshev polynomials for an integral in [-1,1],&#xA;and transform your integral to that range. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-15T08:29:34.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3497" PostTypeId="2" ParentId="3486" CreationDate="2012-10-15T08:33:28.727" Score="8" Body="&lt;p&gt;The solution of an ill-conditioned system of equations with a matrix of norm 1 a random right hand side of norm 1 will have with high probability a norm of the order of the condition number. Thus computing a few such solutions will tell you what is going on.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-15T08:33:28.727" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3498" PostTypeId="1" AcceptedAnswerId="3499" CreationDate="2012-10-15T11:24:55.150" Score="5" ViewCount="63" Body="&lt;p&gt;I have written an algorithm where the 2 input arguments are a file and a list of values. I would like to say the algorithm complexity is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;O(max{n,m}) where n is the size of the file and m is the length of the list&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But maybe I'm mixing apples and oranges here. What is the correct way to write or express this?&lt;/p&gt;&#xA;" OwnerUserId="3174" LastActivityDate="2012-10-15T11:32:18.413" Title="correct complexity notation" Tags="|algorithms|complexity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3499" PostTypeId="2" ParentId="3498" CreationDate="2012-10-15T11:32:18.413" Score="4" Body="&lt;p&gt;Writing $O(m)+O(n)$ avoids the problem of having to address units. Note that $\max(m,n)$ and $m+n$ have the same order, and having max in a Landau symbol is quite uncommon.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-15T11:32:18.413" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3500" PostTypeId="1" CreationDate="2012-10-15T11:35:02.690" Score="3" ViewCount="216" Body="&lt;p&gt;In &lt;a href=&quot;http://www.pauldebevec.com/Research/HDR/debevec-siggraph97.pdf&quot; rel=&quot;nofollow&quot;&gt;Debevec and Malik&lt;/a&gt; (mentioned similarly in Forsyth and Ponce's &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/013608592X&quot; rel=&quot;nofollow&quot;&gt;Computer Vision: A Modern Approach&lt;/a&gt;) they highlight a method of solving the camera response function using linear least-squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We collect image intensity data for a number of points with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_{pk} = f(E_{p}\Delta t_{k})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $I_{pk}$ is the image intensity for pixel $p$ for the $k$-th exposure time&lt;br&gt;&#xA;$t_{k}$ is the $k$-th exposure time&lt;br&gt;&#xA;$E_{p}$ is the intensity of the surface projected onto the image pixel&lt;br&gt;&#xA;$f$ is the camera response function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$I_{pk}$ and $t_{k}$s are known and we are using least-squares to solve for the unknown $E_{p}$'s and $f$. This problem is turned linear by taking $g = \ln f^{-1}$ and taking the logs we can formulate the above as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g(I_{pk}) = \ln(E_{p}) + ln(\Delta t_{k}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then minimizing this as the following objective function by choice of $g$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum(g(I_{pk}) - \ln(E_{p}) + ln(\Delta t_{k}))^2 + \sum_{z} g''(z)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $z$ is the &lt;strong&gt;discrete&lt;/strong&gt; domain of all pixel intensities, and $g''(z) = g(z - 1) - 2g(z) +g(z + 1)$ creates a smoothness penalty on $g$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://www.pauldebevec.com/Research/HDR/debevec-siggraph97.pdf&quot; rel=&quot;nofollow&quot;&gt;Debevec and Malik&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Because it is quadratic in the $E_{i}$’s and $g(I)$’s, minimizing [the above] is&#xA;  a straightforward linear least squares problem&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't understand how we've reached the following conclusion. As mentioned by the paper, we should be able to solve the following by creating the correct matrix and performing singular-value decomposition. What is the construction of this matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are we saying that $g = \ln f^{-1}$ is a linear function? &lt;/p&gt;&#xA;" OwnerUserId="3173" LastEditorUserId="3173" LastEditDate="2012-10-15T12:35:42.217" LastActivityDate="2012-11-15T23:51:24.763" Title="How do you formulate the linear least-squares method for radiometric calibration?" Tags="|linear-algebra|linear-solver|computer-vision|least-squares|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3501" PostTypeId="2" ParentId="3495" CreationDate="2012-10-15T12:48:34.437" Score="3" Body="&lt;p&gt;Collision is typically modeled through equations like the &lt;em&gt;obstacle problem&lt;/em&gt;, i.e., you have a partial differential equation with an inequality constraint. This leads to &lt;em&gt;variational inequalities&lt;/em&gt;. An important step is indeed to find out whether two objects are in contact, and there you do use similar techniques as for games.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-15T12:48:34.437" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3502" PostTypeId="1" AcceptedAnswerId="3504" CreationDate="2012-10-15T19:42:35.447" Score="10" ViewCount="3150" Body="&lt;p&gt;Performing a direct numerical simulation of isotropic turbulence with Fourier pseudo-spectral method (Orzag &amp;amp; Patterson, PRL, 1972) using FFT.&#xA;For a background of the method, which is widely used in turbulence community, you can see this course: &lt;a href=&quot;http://www.math.ualberta.ca/%7Ebowman/m655/lab3d.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.ualberta.ca/~bowman/m655/lab3d.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using the so called &lt;span class=&quot;math-container&quot;&gt;$2/3$&lt;/span&gt; rules for dealiasing which consists of doing&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\hat{u}(\mathbf{k},t)=0~~~~~~~~~~~~~~if~~~k &amp;gt; \frac{2}{3} k_\text{MAX}&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the time, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{k}$&lt;/span&gt; is the wave number, &lt;span class=&quot;math-container&quot;&gt;$k_\text{MAX}$&lt;/span&gt; is the maximum wave number and &lt;span class=&quot;math-container&quot;&gt;$\hat{u}$&lt;/span&gt; is the spectral amplitude of the velocity.&lt;/p&gt;&#xA;&lt;p&gt;Does the dealiasing act as a numerical dissipation? In other words, is there a leak of energy due to the dealiasing?&lt;/p&gt;&#xA;" OwnerUserId="1383" LastEditorUserId="20688" LastEditDate="2021-02-24T22:22:43.127" LastActivityDate="2021-02-24T22:22:43.127" Title="Fourier pseudo-spectral method and numerical dissipation" Tags="|fluid-dynamics|simulation|fourier-transform|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3504" PostTypeId="2" ParentId="3502" CreationDate="2012-10-15T23:02:19.780" Score="5" Body="&lt;p&gt;The dealiasing of the convolution doesn't act as numerical dissipation.  In fact, energy is conserved only if you kick out the aliased terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea behind dealiasing FFT-based convolutions is to get rid of extra terms that are added by the FFT.  A convolution is just a sum, and you can compute it by just calculating the sum. However, this is really slow, so it's better to Fourier-transform the inputs and multiply the result and then invert the Fourier transform, which, by the convolution theorem, is the same thing as a convolution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the convolution theorem only works when the inputs are of infinite length; for finite length inputs, extra terms (aliases) show up which aren't physical.  The point of dealiasing is to get back the original equation that you were trying to calculate, while still allowing you to use FFTs to speed up the computation.&lt;/p&gt;&#xA;" OwnerUserId="3182" LastActivityDate="2012-10-15T23:02:19.780" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3505" PostTypeId="1" AcceptedAnswerId="3508" CreationDate="2012-10-16T11:10:12.030" Score="2" ViewCount="136" Body="&lt;p&gt;I was wondering whether there was a efficient procedure for solving &#xA;constrained quadratic approximations of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\underset{k\in \mathbb{R}}{\min}\;||x_i-kx_0||_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for fixed values of $x_0,x_i\in \mathbb{R}^d$. This needs to be solved recursively for a large number of values of $x_i$ for fixed $x_0$ (so if there is a way to parametrize the solution in terms of $x_i$, that will be helpful). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I'm using an iterative algorithm (in R):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(123)&#xA;d&amp;lt;-100&#xA;n&amp;lt;-50&#xA;xi&amp;lt;-matrix(rnorm(n*d),n,d)&#xA;x0&amp;lt;-rnorm(d)&#xA;ff&amp;lt;-function(k,xi,x0,ll){&#xA;    crossprod(xi[ll,]-k*x0)&#xA;}&#xA;optimize(ff,lower=-1000,upper=1000,xi=xi,x0=x0,ll=13)$min&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="479" LastEditorUserId="479" LastEditDate="2012-10-17T07:21:01.090" LastActivityDate="2012-10-17T07:21:01.090" Title="efficient mean of solving constrained OLS problems?" Tags="|optimization|least-squares|constrained-optimization|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3506" PostTypeId="2" ParentId="3505" CreationDate="2012-10-16T11:22:34.910" Score="1" Body="&lt;p&gt;You can do it by Restricted Least Squares. If your constraints on $k$ are linear you should even have a closed form solution for $\hat{k}$.&lt;/p&gt;&#xA;" OwnerUserId="2140" LastEditorUserId="2140" LastEditDate="2012-10-16T16:06:49.150" LastActivityDate="2012-10-16T16:06:49.150" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3507" PostTypeId="2" ParentId="3502" CreationDate="2012-10-16T13:38:56.050" Score="5" Body="&lt;p&gt;Understanding what the dealiasing does requires understanding why you're taking FFTs in the first place.  For me, the method of weighted residuals approach provides the simplest framework:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You're expanding a solution as a sum of Fourier modes and dropping it into the Navier--Stokes equations.  Your test functions are also a Fourier basis.&lt;/li&gt;&#xA;&lt;li&gt;When you multiply the trial and test functions together and take the inner product (i.e. integrate with an appropriate conjugation), you'll see that you have one integral you cannot simply evaluate because it is nonlinear.&lt;/li&gt;&#xA;&lt;li&gt;You can approximate that integral using a uniform quadrature scheme where you simply sum the value at uniform collocation points.&lt;/li&gt;&#xA;&lt;li&gt;Notice your quadrature is exact for a quadratic nonlinearity if you use 1.5 times as many quadrature points as you had Fourier modes.  This is the three-halves factor.&lt;/li&gt;&#xA;&lt;li&gt;Finally, notice the quadrature can be efficiently computed as a FFT.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All the FFT magic can be thought of as an efficient, approximate quadrature scheme which can be made exact for quadratic nonlinearities.  Other dealiasing variants play other tricks to make the quadrature exact for the same class of nonlinearities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your question, dealiasing adds no dissipation.  This can be seen because every step in the procedure was exact.&lt;/p&gt;&#xA;" OwnerUserId="327" LastEditorUserId="327" LastEditDate="2012-10-16T13:44:19.337" LastActivityDate="2012-10-16T13:44:19.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3508" PostTypeId="2" ParentId="3505" CreationDate="2012-10-16T14:10:24.747" Score="3" Body="&lt;p&gt;By squaring the expression and differentiating w.r.t $k$, it follows that the unconstrained solution is $x_0^Tx_i/(x_0^Tx_0)$. If $k$ is bounded, the optimal solution is obtained by clipping the solution at the bound.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2012-10-16T14:10:24.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3509" PostTypeId="1" AcceptedAnswerId="3515" CreationDate="2012-10-16T18:14:25.517" Score="2" ViewCount="1365" Body="&lt;p&gt;Say I had the choice of choosing one out of the following two optimization problems which I could use to solve my problem. Which choice is the fastest? How much of a trade-off would it be-as in - Is the improvement in speed by many factors!?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Minimizing a convex function L(X) in one matrix variable with orthogonality constraints over the matrix-essentially in my case this ends up to solving an eigen-decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Minimizing the same convex function L(X) with a single linear constraint in X.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that 2) should be faster. But what is the direction of work I need to do- to compare the improvement in speed-especially in terms of using the fastest available eigen solver for 1)-what would be the corresponding fastest approach to solve 2)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Details: &#xA;Example Formulation 1) Minimize $Tr(X^TAX)$ over $X$ under a constraint that $X^TX=I$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example Formulation 2) Minimize $Tr(X^TAX)$ over $X$ under a single linear constraint, $Tr~X^TC=b$ over $X$ where $A$ is a known p.s.d matrix , $b$ is a constant (scalar) $\in \mathbb{R^+}$ and $C$ is a constant matrix with real-entries.  Hence making $Tr(X^TAX)$ convex. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dimension of $X$ in this problem setting varies from 5000 by 2 and up until 50000 by 3. So, the number of columns are not many. $A$ is a sparse matrix, with the amount of sparsity dependent on a tuning parameter of a kernel function that generates the matrix $A$. On a holistic sense the sparsity does range a lot from very sparse to not too sparse and is data and problem dependent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which would be the fastest to solve and by what factor!? And what are the example -fastest possible methods you would use for each individual problem-while coming to this conclusion. Would you come to this conclusion from a theoretical aspect- in terms of how the problem were formulated? If so, please go over that too. &lt;/p&gt;&#xA;" OwnerUserId="1934" LastEditorUserId="1934" LastEditDate="2012-10-18T17:15:13.827" LastActivityDate="2012-10-18T18:55:55.440" Title="Quadratic program With Linear Constraint vs. Eigen Decomposition Time Complexity-Comparison. Which is faster?" Tags="|optimization|complexity|convex-optimization|constraints|quadratic-programming|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3510" PostTypeId="2" ParentId="3500" CreationDate="2012-10-16T23:32:06.087" Score="1" Body="&lt;p&gt;This is a Penalized Least Squares problem, with penalty on the second derivative of $g()$ (so we are not saying that the function is linear). &#xA;In practice you have one unknown parameter for each pixel intensity $I_{pk}$, that represents the value of the function at that point $g(I_{pk})$.&#xA;You just have to express your penalty in a matrix form $g(I)^T M g(I)$, where $M$ is a tridiagonal matrix with -2s on the main diagonal, and 1s above and below the main diagonal (looks right, but not completely sure about it). Then either just give it to an optimizer to estimate the values $g(I)$ and $E$ at the same time or you estimate them one at the time. For example for fixed $E$, the closed form solution for $g(I)$ should be $\hat{g}(I)=(I^TI + M)^{-1}I^T(ln(\Delta t) -ln(E))$. Then for $E$ you can just use an optimizer (with fixed $\hat{g}(I)$) and alternate until convergence.&lt;/p&gt;&#xA;" OwnerUserId="2140" LastActivityDate="2012-10-16T23:32:06.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3511" PostTypeId="2" ParentId="3509" CreationDate="2012-10-17T02:17:53.113" Score="1" Body="&lt;p&gt;Minimizing $Tr(X^TAX)$ subject to a linear constraint on $X$ is clearly a simpler problem than minimizing it subject to a quadratic constraint. You can do the former in a single step -- the solution is simply the solution of a single, linear, saddle point problem.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-17T02:17:53.113" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3513" PostTypeId="1" AcceptedAnswerId="3516" CreationDate="2012-10-17T08:33:39.813" Score="7" ViewCount="457" Body="&lt;p&gt;Is there any (open source) FEM toolbox that allows the direct discretization of higher order PDEs without the need to split them up into systems of second order?&lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2012-10-17T20:14:34.373" Title="FEM toolbox for discretization of higher order PDEs" Tags="|pde|software|finite-element|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3514" PostTypeId="2" ParentId="3513" CreationDate="2012-10-17T09:07:23.607" Score="2" Body="&lt;p&gt;Have a look at the &lt;a href=&quot;http://fenicsproject.org&quot; rel=&quot;nofollow&quot;&gt;FENICS&lt;/a&gt; project. I have not used it yet, but it seems one can do a lot with it, with respect to finite element methods.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-10-17T09:07:23.607" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3515" PostTypeId="2" ParentId="3509" CreationDate="2012-10-17T09:29:03.397" Score="3" Body="&lt;p&gt;For a symmetric and positive definite matrix $A$, the problem $\min Tr~X^TAX$ subject to $Tr~X^TC=b [\in R_+]$ can be solved by introducing a Lagrange multipliers and setting the gradient of the Lagrangian to zero. The result is&#xA;$X=\lambda Z$, where $Z=A^{-1}C$. Inserting this into the constraints gives the multiplier $\lambda=b/Tr~C^TA^{-1}C$. As $C$ has only a few columns, the dominant work is therefore that for computing $Z$, which means a solve for a fixed positive definite matrix with a few right hand sides. (If there were $s$ linear constraints, one would end up with an $s\times s$ system for the multiplier vector.)&lt;BR&gt;&#xA;If the sparsity of $A$ is such that upon reordering you can compute a Cholesky factor $R$, so that $A=R^TR$ then solving $R^TY=C$ and $RZ=Y$ gives the solution $X=\lambda Z$, where $\lambda=b/||Y||^2$ (in the Frobenius norm).&#xA;If a factorization is too expensive, you need to employ conjugate gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a symmetric and positive definite matrix $G$, the problem $\min Tr~X^TGX$ subject to $X^TX=1$ is solved (for an $m\times n$ matrix $X$) by taking as columns of $X$ orthogonal eigenvectors corresponding to the $n$ smallest eigenvalues of $G$. This is more expensive to compute when $A$ can be factored, but if a factorization is not feasible, the Lanczos iteration will have complexity comparable to that for the other problem, and this will become better the more columns $X$ has.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-10-18T18:55:55.440" LastActivityDate="2012-10-18T18:55:55.440" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3516" PostTypeId="2" ParentId="3513" CreationDate="2012-10-17T20:09:00.837" Score="11" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://libmesh.sf.net&quot;&gt;LibMesh&lt;/a&gt; supports Clough-Tocher and tensor product Hermite $C^1$ elements, see the &lt;a href=&quot;http://libmesh.sourceforge.net/adaptivity_ex4.php&quot;&gt;biharmonic example&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://fenicsproject.org&quot;&gt;FEniCS&lt;/a&gt; has a &lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/1.0.0/cpp/demo/pde/biharmonic/cpp/documentation.html&quot;&gt;biharmonic example&lt;/a&gt; that does uses a mixed &lt;a href=&quot;http://dx.doi.org/10.1016/j.cma.2007.03.008&quot;&gt;continuous-discontinuous Galerkin formulation&lt;/a&gt;. Any package with $C^0$ elements that can compute second derivatives and can integrate DG jump terms can also use this approach.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://bitbucket.org/dalcinl/petiga&quot;&gt;PetIGA&lt;/a&gt; Supports isogeometric elements with arbitrary continuity, but currently only on one &quot;patch&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://geopdes.sourceforge.net/&quot;&gt;GeoPDEs&lt;/a&gt; appears to be capable, but I haven't used it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.oofem.org&quot;&gt;OOFEM&lt;/a&gt; supports multi-patch isogeometric analysis with T-splines.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that it is not always better to use methods with higher continuity than to reduce to second order form. The bandwidth and vertex separators for these higher continuity methods are typically much larger, sometimes leading to more expensive solves than if you had reduced to second order form. You have to consider inherent heterogeneity/nonsmoothness in your problem, practical orders of accuracy, dimension, solution algorithm, and problem scale to determine which methods to prefer. Comparison studies using appropriate algorithms in each case would be a welcome contribution to the literature.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-10-17T20:14:34.373" LastActivityDate="2012-10-17T20:14:34.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3518" PostTypeId="1" AcceptedAnswerId="3520" CreationDate="2012-10-18T08:10:08.813" Score="6" ViewCount="435" Body="&lt;p&gt;Consider general system &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt;. Does convergence of the Krylov subspace methods depend on actual vector &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; assuming the initial guess is zero? I mean such factors as the locality of the source (with the limit case where RHS represents delta source) or the dynamic range of the values in there. Does finite precision play any role here?&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="20688" LastEditDate="2020-09-04T14:41:01.530" LastActivityDate="2020-09-04T14:41:01.530" Title="Does right-hand side influence convergence rate of a Krlylov subspace method?" Tags="|linear-algebra|iterative-method|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3519" PostTypeId="2" ParentId="3518" CreationDate="2012-10-18T08:36:46.643" Score="2" Body="&lt;p&gt;Just a note. I suppose you are talking about the special case of the Poisson problem. (?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The behaviour of Krylov subspace methods depends on the initial residual. This is only the right-hand side if the initial guess is zero, which is not necessarily what you want in numerical pde, because you might have good approximations through computations on coarser grids available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you regard symmetric systems, then it suffices to regard a diagonal matrix to study all relevant cases (in exact arithmetics). So the convergence behaviour depends on the decomposition of the initial residual into eigenmodes of the Laplacian on your ansatz space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Eventually, I suppose it gets worse because the Dirac delta, as a distribution, is the superposition of a &quot;large share&quot; of eigenmodes of the Laplacian. Take a look at the Fourier transform on the real line.)&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-10-18T08:36:46.643" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3520" PostTypeId="2" ParentId="3518" CreationDate="2012-10-18T11:38:14.020" Score="13" Body="&lt;p&gt;Of course. The common Krylov space methods build the solution of $Ax=b$ as an approximation first in the space $\{r\}$ (where $r=b-Ax_0$ with the initial guess $x_0$), then $\{r,Ar\}$, then $\{r,Ar,A^2r\}$, etc. So the space in which the solution is sought depends on the right hand side vector. If your solution happens to be, say, in $\{r,Ar,A^2r\}$, then the iteration will terminate after 3 iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, if you ask about this from a purely practical perspective, then the answer is no. In practice this kind of lucky breakdown of the iteration rarely happens, and it is a common observation that the number of iterations you need does not depend strongly on the right hand side, just as it doesn't depend strongly on the starting guess of your iteration. For example, in adaptive finite element methods one might think that it's worthwhile starting a CG iteration on one mesh with the solution from the previous mesh interpolated to the current mesh -- because it should already be close to the solution on the current mesh. However, it's not worth it: the number of iterations might be slightly lower but not enough to justify the effort of interpolating the solution. What this shows is that in practice, convergence of Krylov methods is by and large given by the matrix, not the right hand side or the starting guess.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-18T11:38:14.020" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3521" PostTypeId="1" CreationDate="2012-10-19T10:44:52.093" Score="4" ViewCount="2113" Body="&lt;p&gt;For a problem I am trying to solve it appears MOSEK's Quadratic Program solver is 100 times slower than MATLAB's Interior Point solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone encountered this behavior in the past, or maybe could guess what sort of problem might cause this behavior?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align} &#xA;\text{min }&amp;amp; 0.5 x^T Q x + c^T x \\&#xA;\text{s.t. }&amp;amp; A x \leq b&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With more linear constraints than variables.&lt;/p&gt;&#xA;" OwnerUserId="898" LastEditorUserId="276" LastEditDate="2012-10-24T04:44:39.513" LastActivityDate="2012-10-24T04:44:39.513" Title="Why does MATLAB's quadprog outperform MOSEK for my problem?" Tags="|matlab|quadratic-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3523" PostTypeId="1" CreationDate="2012-10-20T13:05:20.983" Score="8" ViewCount="724" Body="&lt;p&gt;I'm looking for examples of Helmholtz and Biharmonic equations in Cartesian co-ordinates with exact solutions, in order to compare my numerical solutions with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was able to find quite a few examples on the internet, where the problem with boundary conditions were precisely defined. Those were, unfortunately only illustrative examples and exact solutions were not shown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was encouraged about manufacturing the solution (like on &lt;a href=&quot;https://math.stackexchange.com/questions/216417/helmholtz-and-biharmonic-equation-examples-with-exact-solution&quot;&gt;math.stackexchange.com&lt;/a&gt;, and I did that successfully). I was afraid in that case some interesting examples the specialists in PDEs are aware of wouldn't be treated, like some solutions given by infinite series (which I would truncate when some level of accuracy is reached). For example, the one given on &lt;a href=&quot;http://en.wikipedia.org/wiki/Elliptic_boundary_value_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipeda article on elliptic BVPs&lt;/a&gt; is interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any particular example, or a useful link to a web-page or a paper is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-08-27T17:26:49.943" Title="Helmholtz and Biharmonic equation examples with exact solution" Tags="|pde|finite-element|reference-request|elliptic-pde|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3524" PostTypeId="2" ParentId="3315" CreationDate="2012-10-20T20:06:18.383" Score="5" Body="&lt;p&gt;Here are some recommendations and links collected over time:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For 2M nodes it is hard to recommend anything not knowing your hardware, and possibly some data reduction is in order, but taking stuff that is freely available, &lt;a href=&quot;http://zvtm.sourceforge.net/zgrviewer.html&quot; rel=&quot;nofollow&quot;&gt;zGrViewer&lt;/a&gt; may fit your needs for visualization (requires GraphViz).&lt;/li&gt;&#xA;&lt;li&gt;Following @pyCthon 's idea, suggest you also have a look at &lt;a href=&quot;https://wci.llnl.gov/codes/visit/&quot; rel=&quot;nofollow&quot;&gt;VisIt&lt;/a&gt; for some interactivity in plotting.&lt;/li&gt;&#xA;&lt;li&gt;I am re-visiting the &lt;a href=&quot;http://igraph.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;igraph&lt;/code&gt;&lt;/a&gt; package for the &lt;a href=&quot;http://cran.r-project.org&quot; rel=&quot;nofollow&quot;&gt;R&lt;/a&gt; statistical language, which includes neat layout algorithms (&lt;a href=&quot;https://en.wikipedia.org/wiki/Force-based_algorithms_%28graph_drawing%29&quot; rel=&quot;nofollow&quot;&gt;Fruchterman-Reingold and Kamada-Kawai&lt;/a&gt;), among others.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://lgl.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;Large Graph Layout&lt;/a&gt; library is now on SourceForge.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2155" LastEditorUserId="2155" LastEditDate="2013-03-14T07:22:08.597" LastActivityDate="2013-03-14T07:22:08.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3525" PostTypeId="2" ParentId="1922" CreationDate="2012-10-21T04:12:48.163" Score="3" Body="&lt;p&gt;I think the book &lt;a href=&quot;http://www.nr.com/oldverswitcher.html&quot; rel=&quot;nofollow&quot;&gt;Numerical Recipes--The Art of Scientific Computing,William H et al&lt;/a&gt; talks about each PRNG's strengths and weaknesses in chapter 7. If you want to test different PRNGs by yourself, you can try the &lt;a href=&quot;http://www.iro.umontreal.ca/~simardr/testu01/tu01.html&quot; rel=&quot;nofollow&quot;&gt;TestU01&lt;/a&gt; suite, which is developed by L'Ecuyer and Richard Simard, this suite contains 3 predefined batteries(SmallCrush, Crush, and BigCrush), provides different levels of thoroughness for testing. Most PRNGs even like LCGs can pass SmallCrush, which is roughly equivalent to the Diehard Battery, some may pass Crush, but very few of them will pass BigCrush. It also summarized the testing result for different popular PRNGs in a table.&lt;/p&gt;&#xA;" OwnerUserId="3188" LastEditorUserId="3188" LastEditDate="2012-10-21T04:25:25.010" LastActivityDate="2012-10-21T04:25:25.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3526" PostTypeId="2" ParentId="1922" CreationDate="2012-10-21T06:48:40.340" Score="1" Body="&lt;p&gt;I have had a lot of luck with this package:&#xA;&lt;a href=&quot;http://www.phy.duke.edu/~rgb/General/dieharder.php&quot; rel=&quot;nofollow&quot;&gt;http://www.phy.duke.edu/~rgb/General/dieharder.php&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is the diehard tests + a large suite of others all ramped up to paranoid levels, and it outputs ACTUAL METRICS!  As suggested by the question.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran a ways down this rabbit hole sometime during my masters thesis, which addressed how viscous flow was affected by large numbers of random obstructions.  During my investigation I discovered that random-sphere packing is actually part of the diehard test-suite, getting good randoms is hard . . . on a related note, I would also reccomend this read, and associated troll Amazon comments:&#xA;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0833030477&quot; rel=&quot;nofollow&quot;&gt;http://www.amazon.com/Million-Random-Digits-Normal-Deviates/dp/0833030477&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-10-21T06:48:40.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3527" PostTypeId="2" ParentId="3502" CreationDate="2012-10-21T08:43:36.710" Score="0" Body="&lt;p&gt;@Malcolm gave an expert explanation of the dealiasing of the FFT based convolution sums. My intuition tells me however that this is still not the complete answer @abberation needed, so I'll give few more sentences that reflect my own struggle for understanding of numerical methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dealiasing doesn't act as a numerical dissipation, but it does the same thing as numerical dissipation - it prevents your simulation to blow-up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does your simulation blow-up? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an unphysical transfer of energy from high to low wavenumbers which causes, first the instability (often followed by a simptome - 2h waves, h-grid node distance, the smallest waves that can be represented on a grid). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason behind this is aliasing of high wave numbers to those low wavenumbers that can be represented by the grid. This causes &quot;spectral blocking&quot; - the unphysical increase of the amplitudes (read energy) of the high wavenumbers, which gets worse as you approach the upper limit of the represented wavenumbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First remedy (Philips, 1959): Filter out, or make equal to zero, upper half of the represented wavenumber spectrum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second remedy (Orszag, 1971): The remedy known as the Orszag's Two-Thirds rule. Steven said &quot;filtering out half of the wavenumbers is a waste, we should keep 2/3N and filter out only the upper 1/3N.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conclusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our simulations sometimes tend to blow-up. If you live in second order, finite volume universe (like the author of these lines did), and know nothing about Spectral Methods, they tell you &quot;Oh your simulation is blowing up? Try using first order upwind instead of central scheme, or blend your central scheme with 10% of upwind in deffered correction approach. That will add dissipation but will prevent your simulation from blowing-up! Oh you're doing LES? Then you shouldn't use upwind, unless you are performing ILES, use just central scheme and Smagorinsky will model the physical dissipation that happens on lowest scales.&quot; &#xA;If you live in the Spectral Methods universe, you learn how unphysical transfer of energy takes place, and you find the remedy in dealiasing by Two-Thirds Rule or &quot;padding&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My inspiration for this post is a wonderfull book by J. Boyd &quot;Chebyshev and Fourier Spectral Methods&quot;, section 11.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Act physically or spectrally, but think spectrally!&#xA;(To paraphrase well known &quot;Think globally, act locally.&quot;)&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="1050" LastEditDate="2012-10-21T08:53:19.117" LastActivityDate="2012-10-21T08:53:19.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3528" PostTypeId="1" AcceptedAnswerId="3529" CreationDate="2012-10-21T13:23:10.300" Score="12" ViewCount="549" Body="&lt;p&gt;I'm looking for a library that performs matrix operations on large sparse matrices w/o sacrificing numerical stability. Matrices will be 1000+ by 1000+ and values of the matrix will be between 0 and 1000. I will be performing the &lt;a href=&quot;http://en.wikipedia.org/wiki/Index_calculus_algorithm&quot; rel=&quot;noreferrer&quot;&gt;index calculus algorithm&lt;/a&gt; so I will be generating (sparse) row vectors of the matrix serially. As I develop each row, I will need to test for linear independence. Once I fill my matrix with the desired number of linearly independent vectors, I will then need to transform the matrix into reduced row echelon form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem now is that my implementation uses Gaussian elimination to determine linear independence (ensuring row echelon form once all my row vectors have been found). However, given the density and size of the matrix, this means the entries in each new row become exponentially larger over time, as the lcm of the leading entries must be found in order to perform cancellation. Finding the reduced form of the matrix further exacerbates the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, is there an algorithm, or better yet an implementation, that can test linear independence and solve the reduced row echelon form while keeping the entries as small as possible? An efficient test for linear independence is especially important since in the index calculus algorithm it is performed by far the most.&lt;/p&gt;&#xA;" OwnerUserId="3199" LastEditorUserId="391" LastEditDate="2012-10-23T08:36:44.407" LastActivityDate="2012-10-23T08:36:44.407" Title="Algorithms for Large Sparse Integer Matrices" Tags="|linear-algebra|algorithms|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3529" PostTypeId="2" ParentId="3528" CreationDate="2012-10-21T13:39:54.670" Score="5" Body="&lt;p&gt;You can work modulo a number of large primes to get the results modulo these primes, then check whether there are rationals with few enough digits satisfying these congruences. If yes, you can check by a matrix-vector multiply whether the approximation found is exact. This can be turned into an exact decision algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the determiant of the matrix has a size of the order of $10^{1000}$ (quite possible in your scenario), you'll generically not find solutions whose components need less that a few thousand digits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;related links:&lt;BR&gt;&#xA;&lt;a href=&quot;http://cs.ucsb.edu/~koc/docs/j21.pdf&quot; rel=&quot;nofollow&quot;&gt;http://cs.ucsb.edu/~koc/docs/j21.pdf&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=355767&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=355767&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=355765&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=355765&lt;/a&gt;&lt;BR&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-10-21T16:44:00.463" LastActivityDate="2012-10-21T16:44:00.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3530" PostTypeId="2" ParentId="3315" CreationDate="2012-10-21T16:58:46.060" Score="5" Body="&lt;p&gt;See Graphinsight 1.2, can handle with million of nodes easily and it's interactive and in 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also layout graphs with million of nodes and edges with high efficient algebraic methods or force directed methods. It's available in trial version for evaluation (&lt;strong&gt;Disclaimer: I am one of the authors of the program&lt;/strong&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.graphinsight.com&quot;&gt;www.graphinsight.com&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3186" LastEditorUserId="2155" LastEditDate="2012-10-21T19:57:14.050" LastActivityDate="2012-10-21T19:57:14.050" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3531" PostTypeId="1" CreationDate="2012-10-22T11:06:23.403" Score="9" ViewCount="5234" Body="&lt;p&gt;I want to solve $K u = b$ where $K$ is my stiffness matrix. However some constraints may be missing an therefore some rigid body motion may be still present in the system (due to eigenvalue zero). Since I'm using CG for solving the linear system this is not acceptable since sometimes CG doesn't converge on semi-positive problems (but I may sometimes converges).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually I'm using a penalized displacement approach in the sense that I'm adding a penalty of the form $ \alpha ||u||^2$ to the elastic energy. So the energy reads&#xA;\begin{equation}&#xA;\mathcal W(u) := \frac{1}{2} u^T (K + \alpha I) u - b^t u \end{equation} where $\alpha$ taken as a proportional to some diagonal entry of the stiffness matrix. But actually this has the effect to damp some deformation mode that I would sometime like to have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) could I transform the original system so has to make it free of singularity and positive definite (such as coordinate transformation or congruence transformation or whatever) ? My idea is to use such transformation to still use CG on the transformed problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Is there any standard way to deal with those singularities ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kind regards, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tom&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2012-10-22T13:56:28.457" Title="How to remove Rigid Body Motions in Linear Elasticity?" Tags="|finite-element|iterative-method|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3532" PostTypeId="2" ParentId="3531" CreationDate="2012-10-22T11:47:48.180" Score="6" Body="&lt;p&gt;The standard way is to add the constraint $u(x_0)=0$ for an arbitrarily chosen node $x_0$. This makes sure that your body can't translate or rotate and therefore takes away the zero eigenvalue. The resulting system with this constraint is positive definite even without your penalty term.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-22T11:47:48.180" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3533" PostTypeId="2" ParentId="3531" CreationDate="2012-10-22T13:02:23.680" Score="6" Body="&lt;p&gt;If you know the null space, you can make the right hand side compatible and have the Krylov method prevent the preconditioner from causing pollution, see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3426&quot;&gt;Why is pinning a point to remove a null space bad?&lt;/a&gt; for further discussion. In PETSc, this is done using the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatNullSpace.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatNullSpace&lt;/code&gt;&lt;/a&gt; object. Note that you can &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatNullSpaceSetFunction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;provide your own function&lt;/a&gt; to project out the null space, which would be useful to reduce projection cost when you have many floating structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do not know the null space and cannot avoid an incompatible right hand side, there are specialized Krylov methods like &lt;a href=&quot;http://www.stanford.edu/group/SOL/software/minresqlp/MINRESQLP-SISC-2011.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MINRES-QLP&lt;/a&gt; that can find the minimum norm solution despite. This approach may be useful if you have hinges and single-point connections that only couple some modes. Note that you must still be careful about the preconditioner causing pollution (e.g., due to LU factorization finding zero pivots, perhaps on a coarse level of multigrid).&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-10-22T13:56:28.457" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3534" PostTypeId="1" AcceptedAnswerId="3536" CreationDate="2012-10-22T14:04:20.437" Score="9" ViewCount="5178" Body="&lt;p&gt;From the definition of condition number it seems that a matrix inversion is needed to compute it, I'm wondering if for a generic square matrix (or better if symmetric positive definite) is possible to exploit some matrix decomposition to compute the condition number in a faster way.&lt;/p&gt;&#xA;" OwnerUserId="3186" LastActivityDate="2012-10-23T04:15:21.507" Title="Fastest algorithm to compute the condition number of a large matrix in Matlab/Octave" Tags="|linear-algebra|matrix|condition-number|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3535" PostTypeId="2" ParentId="3534" CreationDate="2012-10-22T14:16:44.330" Score="5" Body="&lt;p&gt;It's certainly easy to use the eigenvalue/eigenvector decomposition of a symmetric matrix or the SVD of a general matrix to compute the condition number, but these aren't particularly fast ways to proceed.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are iterative algorithms that can compute an estimate of the condition number that is useful for most purposes without going to all of the work of computing $A^{-1}$.  See for example the &lt;code&gt;condest&lt;/code&gt; function in MATLAB.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="541" LastEditDate="2012-10-22T14:19:39.413" LastActivityDate="2012-10-22T14:19:39.413" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3536" PostTypeId="2" ParentId="3534" CreationDate="2012-10-22T14:35:39.920" Score="8" Body="&lt;p&gt;Computing the condition number (even approximating it within a factor of 2) seems to have the same complexity as computing a factorization, though there are no theorems in this direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a sparse Cholesky factor $R$ of a symmetric positive definite matrix,&#xA;or from a sparse $QR$ factorization (with implicit $Q$) of a general square matrix, one can obtain the condition number in the Frobenius norm by computing the sparse inverse subset of $(R^TR)^{-1}$, which is much faster than computing the full inverse. (Related to this is my paper: Hybrid norms and bounds for overdetermined linear systems, Linear Algebra Appl. 216 (1995), 257-266. &#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/scan/74.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/scan/74.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:  If $A=QR$ then with respect to any unitarily invariant norn, $$cond(A)=cond(R)=\sqrt{cond(R^TR)}.$$ For the computation of sparse QR factorizations see, e.g., &lt;BR&gt;&#xA;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=174408&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=174408&lt;/a&gt;.&lt;BR&gt;&#xA;For the computation of the sparse inverse, see, e.g., my paper: &#xA;Restricted maximum likelihood estimation of covariances in sparse linear models,&#xA;Genetics Selection Evolution 30 (1998), 1-24.&lt;BR&gt;&#xA;&lt;a href=&quot;https://www.mat.univie.ac.at/~neum/ms/reml.pdf&quot; rel=&quot;nofollow&quot;&gt;https://www.mat.univie.ac.at/~neum/ms/reml.pdf&lt;/a&gt;&#xA;The cost is about 3 times the cost for the factorization.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-10-22T20:23:23.600" LastActivityDate="2012-10-22T20:23:23.600" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3537" PostTypeId="2" ParentId="3523" CreationDate="2012-10-22T16:28:49.003" Score="1" Body="&lt;p&gt;Does an axis aligned rectangular box (length/width/height = a/b/c) with dirichlet boundary conditions ($\phi=0$) on the walls admit a closed form / exact solution? Maybe a tensor product of sinusoids, e.g. $\phi(x,y,z) = sin(k_x x)sin(k_y y)sin(k_z z)$. Pick $k_x/k_y/k_z$ judiciously to realize the dirichlet condition, e.g. $k_x = n\pi/a$, $k_y = m\pi/b$, $k_z = p\pi/c$, for some integers (n,m,p). Plug that solution into the $div \, grad$ operator, then the resulting equation, $div\,grad \, \phi = k^2 \phi$, should give you the separation condition for $k_x,k_y,k_z$ (probably $k_x^2+k_y^2+k_z^2=k^2$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is kinda standard fare for the vector wave equation / maxwell's equations (electromagnetics), I've not messed around much with the scalar helmholtz equation but I'd expect it to work very similarly. For electromagnetic resonators / the VWE, I'd recommend Balanis' &quot;Advanced Engineering Electromagnetics&quot;. There's probably a comparable reference for the the scalar Helmholtz equation (a graduate level acoustics text, perhaps?) but I wouldn't know what it is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no experience with the Biharmonic equation.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="1050" LastEditDate="2012-10-22T17:32:09.093" LastActivityDate="2012-10-22T17:32:09.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3538" PostTypeId="1" AcceptedAnswerId="3539" CreationDate="2012-10-22T16:50:47.663" Score="19" ViewCount="534" Body="&lt;p&gt;I'm trying to implement the following function in double-precision floating point with low &lt;em&gt;relative&lt;/em&gt; error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{logsum}(x,y) = \log(\exp(x) + \exp(y))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is used extensively in statistical applications to add probabilities or probability densities that are represented in log space. Of course, either $\exp(x)$ or $\exp(y)$ could easily overflow or underflow, which would be bad because log space is used to avoid underflow in the first place. This is the typical solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{logsum}(x,y) = x + \mathrm{log1p}(\exp(y - x))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cancellation from $y-x$ does happen, but is mitigated by $\exp$. Worse by far is when $x$ and $\mathrm{log1p}(\exp(y - x))$ are close. Here's a relative error plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/0tZtu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plot is cut off at $10^{-14}$ to emphasize the shape of the curve $\mathrm{logsum}(x,y) = 0$, about which the cancellation occurs. I've seen error up to $10^{-11}$ and suspect that it gets much worse. (FWIW, the &quot;ground truth&quot; function is implemented using MPFR's arbitrary-precision floats with 128-bit precision.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried other reformulations, all with the same result. With $\log$ as the outer expression, the same error occurs by taking a log of something near 1. With $\mathrm{log1p}$ as the outer expression, cancellation happens in the inner expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the &lt;em&gt;absolute&lt;/em&gt; error is very small, so $\exp(\mathrm{logsum}(x,y))$ has very small relative error (within an epsilon). One might argue that, because a user of $\mathrm{logsum}$ is really interested in probabilities (not log probabilities), this terrible relative error isn't a problem. It's likely that it usually isn't, but I'm writing a library function, and I'd like its clients to be able to count on relative error not much worse than rounding error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems I need a new approach. What might it be?&lt;/p&gt;&#xA;" OwnerUserId="3203" LastEditorUserId="3203" LastEditDate="2012-10-22T17:44:43.993" LastActivityDate="2015-09-03T18:57:15.537" Title="Catastrophic cancellation in logsum" Tags="|floating-point|stability|numerics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3539" PostTypeId="2" ParentId="3538" CreationDate="2012-10-22T17:02:27.337" Score="13" Body="&lt;p&gt;The formula &#xA;$$\mathrm{logsum}(x,y)=\max(x,y)+\mathrm{log1p}(\exp(-\operatorname{abs}(x-y))$$&#xA;should be numerically stable. It generalizes to a numerically stable computation of &#xA;$$\log \sum_i e^{x_i} = \xi+ \log\sum_i e^{x_i-\xi},~~~\xi=\max_i x_i$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case the logsum is very close to zero and you want high relative accuracy, you can probably use&#xA;$$\mathrm{logsum}(x,y)=\max(x,y)+\mathrm{lexp}(x-y)$$&#xA;using an accurate (i.e., more than double precision) implementation of &#xA;$$\mathrm{lexp}(z):=\log(1+e^{-|z|})$$&#xA;which is nearly linear for small $z$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="9667" LastEditDate="2015-09-03T18:57:15.537" LastActivityDate="2015-09-03T18:57:15.537" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="3540" PostTypeId="1" CreationDate="2012-10-22T23:24:13.937" Score="8" ViewCount="5116" Body="&lt;p&gt;I'm trying to re-implement Neural Networks in Python. I implemented the cost function and the backpropagation algorithm correctly. I have checked them by executing its Octave equivalent code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I'm trying to use the &lt;code&gt;scipy.optimize.fmin_cg&lt;/code&gt; function, the iterations take a very long time to execute. It exits with a warning and gives me an error saying that the &quot;desired error rate wasn't achieved&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same program in Octave executes fine. However, it has its own &lt;code&gt;fmincg&lt;/code&gt; function defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong? &lt;/p&gt;&#xA;" OwnerUserId="2156" LastEditorUserId="6923" LastEditDate="2013-12-28T22:05:27.797" LastActivityDate="2018-10-01T19:35:58.957" Title="fmincg implementation in Python" Tags="|optimization|python|machine-learning|octave|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3541" PostTypeId="2" ParentId="3540" CreationDate="2012-10-22T23:58:35.800" Score="6" Body="&lt;p&gt;As a future hint, use the double question mark ?? to pull the source of the function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the scipy sources this error message comes up if the internal parameter alpha_k is zero or None.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This value in turn is tied to the internal Wolfe Line search algorithm.  In particular it is called when the search doesn't find a better value along the search trajectory.  Your function likely has a linear contour somewhere along it that the optimizer falls into and gets confused.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps try adding a callback, and see where the failing search is generated?&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="276" LastEditDate="2013-12-29T19:06:09.633" LastActivityDate="2013-12-29T19:06:09.633" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3542" PostTypeId="2" ParentId="3534" CreationDate="2012-10-23T04:15:21.507" Score="1" Body="&lt;p&gt;For sparse Hermitian matrices $H$, you can use Lanczos algorithm to compute its eigenvalues. If $H$ is not Hermitian, you can compute its singular values by computing the eigenvalues of $H^TH$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the largest and smallest eigenvalues/singularvalues can be found very fast (long before the tridiagonalization is complete), the Lanczos method is particular useful to compute the condition number.&lt;/p&gt;&#xA;" OwnerUserId="1614" LastActivityDate="2012-10-23T04:15:21.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3543" PostTypeId="2" ParentId="3521" CreationDate="2012-10-23T11:49:28.123" Score="6" Body="&lt;p&gt;For problems of this form, you should solve the dual problem using MOSEK. In some cases this can provide several orders of magnitudes of speedup.&#xA;MOSEK is tuned for the more common case&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\text{min }&amp;amp; 0.5 x^T Q x + c^T x \\&#xA;\text{s.t. }&amp;amp;  A x = b \\   &#xA;      &amp;amp; x &amp;gt;= 0   &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where there are many more variables than constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you contact MOSEK support at support@mosek.com and are willing to give us your problem then we can most likely tell what you should change to get a better performance of MOSEK. If you are not willing to provide any information (Size and density of $Q$ and $A$, other special structure to the problem) then it is hard to help you.&lt;/p&gt;&#xA;" OwnerUserId="3207" LastEditorUserId="9" LastEditDate="2012-10-24T01:25:07.347" LastActivityDate="2012-10-24T01:25:07.347" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3544" PostTypeId="1" CreationDate="2012-10-23T13:41:50.450" Score="7" ViewCount="580" Body="&lt;p&gt;I am trying to put down some code to get numerically the solution of the following PDE:&#xA;$$&#xA;  \partial^2_t\phi-\partial^2_x\phi+\lambda\phi^3=\delta(x)\delta(t).&#xA;$$&#xA;Of course, there are several problems already starting with a proper numerical definition of the Dirac deltas and a proper setting of the initial conditions. Last but not least this equation is nonlinear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this: Is there any library of known numerical recipes for this kind of problems, also for the linear case? I have no prejudice about the code and also well-known packages as Matlab or Mathematica are good for my aims.&lt;/p&gt;&#xA;" OwnerUserId="3208" LastEditorUserId="391" LastEditDate="2012-10-25T03:17:50.253" LastActivityDate="2012-10-25T03:17:50.253" Title="Numerical Green functions for a nonlinear wave equation" Tags="|pde|matlab|hyperbolic-pde|mathematica|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3545" PostTypeId="2" ParentId="3544" CreationDate="2012-10-23T14:03:23.227" Score="2" Body="&lt;p&gt;To begin, I would probably advice to consider the steady problem by looking for ground states $\phi = \Phi(x)e^{i\omega t}$. In this way you reduce your equation to the nonlinear Helmholtz-type problem: the elliptic operator $\omega\phi + \phi_{xx}$ + nonlinear part $\lambda\phi^3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I don't see what you can do theoretically with the nonlinear term. Since the works of L. Schwartz (50s) we know that the product of distributions cannot be consistently defined...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jon, what is your initial problem? What would you like to do?&lt;/p&gt;&#xA;" OwnerUserId="3209" LastActivityDate="2012-10-23T14:03:23.227" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3546" PostTypeId="1" AcceptedAnswerId="3550" CreationDate="2012-10-23T16:52:42.323" Score="2" ViewCount="105" Body="&lt;p&gt;I have a time series representing the result of a complex calculation (physical simulation). Due to round-off errors and approximation errors, there will be some &quot;noise&quot; on the data series. In some cases, the physical simulation represents a &quot;steady state&quot; situation meaning that, in theory, the time series should be a flat line. However, if the steady state is not well calculated there will be a drift in the time series. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best algorithm to check whether there is drift or not? Some smoothing filter followed by a kind of linear function approximation $ax+b$ and check for a small $a$ (define small?)&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-10-24T01:53:49.963" Title="Estimate (non-)drift in noisy data" Tags="|approximation|data-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3547" PostTypeId="1" AcceptedAnswerId="3549" CreationDate="2012-10-23T19:29:43.550" Score="8" ViewCount="2492" Body="&lt;p&gt;What is the most efficient way to get a complex sparse matrix from my Fortran code to PETSc? I understand that this is problem dependent, so I tried to give as many relevant details as possible below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been playing with the FEAST eigenvalue solver [1] for problems of the type $A x = \lambda B x$, the dimension of the matrices $A$ and $B$ is $N$, and pretty much all the time is spent solving $N \times N$ complex linear system with M0 right hand sides. N is large (number of FE basis functions, in 3D), M0 is small (in my case I am interested in M0 ~ 20). The matrices $A$ and $B$ are real, symmetric and sparse, and the complex problem that needs solving is $zA-B$, where $z$ is a complex number. The author of FEAST seems to suggest, that the accuracy of the solution to this linear system doesn't have to be very high in order to get high accurate eigenvalues and eigenvectors, so some fast iterative solvers might be a great solution to this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I've been just using Lapack for the complex system, and that works great for $N &amp;lt; 1500$ on my computer. For larger $N$, I don't know yet what the best solver is, so I wanted to just use PETSc and play with the iterative solvers there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a simple C driver, and call it from Fortran, see [2] for all the code, but the problem is just with this part (update: I've put here all the lines to create the matrix, as I just realized that this is relevant):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ierr = MatCreate(PETSC_COMM_WORLD,&amp;amp;A);CHKERRQ(ierr);&#xA;ierr = MatSetSizes(A,PETSC_DECIDE,PETSC_DECIDE,n,n);CHKERRQ(ierr);&#xA;ierr = MatSetFromOptions(A);CHKERRQ(ierr);&#xA;ierr = MatSetOption(A,MAT_IGNORE_ZERO_ENTRIES,PETSC_TRUE);CHKERRQ(ierr);&#xA;ierr = MatSetUp(A);CHKERRQ(ierr);&#xA;for (i=0; i&amp;lt;n; i++) col[i] = i;&#xA;ierr = MatSetValues(A,n,col,n,col,A_,INSERT_VALUES);CHKERRQ(ierr);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which is extremely slow (i.e. for N~1500, this takes maybe 2s, while actually solving it is immediate), in fact the &lt;code&gt;MatSetValues&lt;/code&gt; line takes pretty much 100% of all the time for the whole eigenvalue calculation... The matrix A_ is a 2D matrix that comes from Fortran. I tried to disable the &lt;code&gt;MAT_IGNORE_ZERO_ENTRIES&lt;/code&gt; but it didn't make any difference. So I think that the problem simply is that even for moderate N like 1500, I need to use some sparse matrix format, is that correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I implemented the CSR format in my Fortran code for the $A$ and $B$ matrices (or for the $zA-B$) and now I am trying to figure out how to efficiently give it to PETSc. For now, I just want to get something working sequentially, that beats Lapack. Should I use the &lt;code&gt;MatCreateSeqAIJWithArrays&lt;/code&gt; function for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the most efficient way to do that? Since the matrices $A$ and $B$ don't change, only the complex number $z$ changes in the FEAST algorithm, and in an FE calculation, I think that both $A$ and $B$ have the same sparse structure, one can probably improve things further by preallocating the sparse structure and then just quickly evaluate $zA_x - B_x$ in each FEAST iteration (the $A_x$, $B_x$ are the values arrays of the CSR format), I can do this easily in Fortran, but maybe it's slow to always call &lt;code&gt;MatCreateSeqAIJWithArrays&lt;/code&gt;, so it might be faster to do all this in PETSc once the matrices $A$ and $B$ are transferred over.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know whether this CSR approach to this problem is correct, or whether I am doing it wrong (clearly my original approach with the &lt;code&gt;MatSetValues&lt;/code&gt; is not optimal). Thanks for any tips.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.ecs.umass.edu/~polizzi/feast/&quot;&gt;http://www.ecs.umass.edu/~polizzi/feast/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://github.com/certik/hfsolver/pull/14&quot;&gt;https://github.com/certik/hfsolver/pull/14&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.1/docs/manualpages/Mat/MatCreateSeqAIJWithArrays.html&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-3.1/docs/manualpages/Mat/MatCreateSeqAIJWithArrays.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2012-10-23T20:59:24.177" Title="How to get sparse complex matrices from my code to PETSc efficiently" Tags="|matrix|petsc|performance|fortran|sparse-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3548" PostTypeId="2" ParentId="3546" CreationDate="2012-10-23T19:48:48.960" Score="0" Body="&lt;p&gt;You should try a Kalman filter, if possible for your problem. It's used in accelerometers, which also tend to drift.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-10-23T19:48:48.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3549" PostTypeId="2" ParentId="3547" CreationDate="2012-10-23T20:44:24.473" Score="7" Body="&lt;p&gt;It is important to preallocate correctly. This is almost certainly the reason why your assembly was slow. If you are starting with a dense matrix representation, just scan through it once counting the number of nonzeros per row, then call &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSeqAIJSetPreallocation.html&quot;&gt;&lt;code&gt;MatSeqAIJSetPreallocation()&lt;/code&gt;&lt;/a&gt;. See &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/faq.html#efficient-assembly&quot;&gt;this FAQ&lt;/a&gt;. The option &lt;code&gt;MAT_IGNORE_ZERO_ENTRIES&lt;/code&gt; is really intended to be used when there is some mild sparsity in those otherwise dense blocks rather than building the entire matrix in one call from a dense matrix. For this reason, it doesn't automatically do preallocation based on the sparsity of that one block.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Creating a dense intermediate matrix is not memory scalable, so you will eventually want to avoid it. &lt;code&gt;MatSetValues()&lt;/code&gt; is really meant to be used for &lt;em&gt;logically dense&lt;/em&gt; blocks in a sparse matrix. You typically either call once per row (or group of rows, typical of FD methods) or once per element (typical of FEM methods). If you are translating an existing assembled sparse matrix, just call &lt;code&gt;MatSetValues()&lt;/code&gt; once per row. If you would prefer to skip the intermediate matrix (better performance and lower memory), just call &lt;code&gt;MatSetValues()&lt;/code&gt; once per element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you can call PETSc directly from Fortran, though there are users of every Fortran dialect between basic Fortran 77 and the most recent versions. The interfaces will look rather crude to you as an eager adopter of the latest features. Suggestions for maintainable ways to improve support for the latest dialects would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-10-23T20:59:24.177" LastActivityDate="2012-10-23T20:59:24.177" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3550" PostTypeId="2" ParentId="3546" CreationDate="2012-10-24T01:53:49.963" Score="3" Body="&lt;p&gt;I'd follow your suggestion of fitting a line $ax+b$ to your data. You don't even have to smooth it first -- fitting a line already takes care of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, however, two questions: first, how do you fit the line? This is a question of the statistics of your noise. If noise is Gaussian then you'd use least squares. If your noise has large outliers then you'd want to use a L1-minimization (least absolute values) to fit $a,b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part is to determine whether your drift is significant. To this end, after fitting your predicted drift is $d=a(x_1-x_0)$ if your $x$ values are in the interval $[x_0,x_1]$. $d$ is significant if it is larger than, say, one standard deviation $\sigma$ of your noise where you can compute $\sigma$ as&#xA;$\sigma=\sqrt{\frac 1N \sum_i (y_i-(ax_i+b))^2}$. If $d\gg \sigma$ you can be sure that there is drift.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-24T01:53:49.963" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3551" PostTypeId="1" AcceptedAnswerId="3553" CreationDate="2012-10-24T05:40:13.430" Score="6" ViewCount="325" Body="&lt;p&gt;Is there any work that considers Krylov subspace iterative methods in floating point arithmetic? I'm especially interested in how rounding errors influence the convergence and the accuracy of the solution. &lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2018-06-12T20:26:13.273" Title="Krylov subspace iterative methods in floating point arithmetic" Tags="|iterative-method|floating-point|krylov-method|computer-arithmetic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3552" PostTypeId="2" ParentId="3544" CreationDate="2012-10-24T10:05:24.793" Score="2" Body="&lt;p&gt;Create some mesh in $(x,t)$-space, take out the elements containing the origin, and impose nonzero boundary conditions on the resulting boundary that you believe is appropriate for the solution near the singularity. Then solve the problem without the forcing term in a domain with the resulting hole. This will give you an idea how the solution will look like. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quality will be as good or bad as your guess of the boundary. Maybe your theoretical published work mentioned in the discussion helps to tell you what this guess should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could work with the Fourier transform of your equation in space-time, and try to solve it by collocation in frequency-momentum space.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-24T10:05:24.793" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3553" PostTypeId="2" ParentId="3551" CreationDate="2012-10-24T10:17:11.707" Score="10" Body="&lt;p&gt;Some references on rounding error analysis of Krylov methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Gutknecht, Martin H., and Zdenvek Strakos. &lt;a href=&quot;ftp://ftp.sam.math.ethz.ch/pub/sam-reports/reports/reports99/99-21.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Accuracy of two three-term and three two-term recurrences for Krylov space solvers.&quot;&lt;/a&gt; SIAM Journal on Matrix Analysis and Applications 22.1 (2000): 213-229.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Paige, Christopher C., and Zdenvek Strakos. &lt;a href=&quot;https://www.cs.mcgill.ca/~chris/pub/PaiS02c.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Residual and backward error bounds in minimum residual Krylov subspace methods.&quot;&lt;/a&gt; SIAM Journal on Scientific Computing 23.6 (2002): 1898-1923.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sleijpen, Gerard LG, and Henk A. Van der Vorst. &lt;a href=&quot;http://www.staff.science.uu.nl/~sleij101/Reprints/SVo95b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Maintaining convergence properties of BiCGstab methods in finite precision arithmetic.&quot;&lt;/a&gt; Numerical algorithms 10.2 (1995): 203-223.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Strakoš, Zdenek, and Petr Tichý. &lt;a href=&quot;http://emis.ams.org/journals/ETNA/vol.13.2002/pp56-80.dir/pp56-80.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;On error estimation in the conjugate gradient method and why it works in finite precision computations.&quot;&lt;/a&gt; Electron. Trans. Numer. Anal 13.56-80 (2002): 8.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="9667" LastEditDate="2018-06-12T20:26:13.273" LastActivityDate="2018-06-12T20:26:13.273" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3554" PostTypeId="1" CreationDate="2012-10-24T15:15:26.927" Score="4" ViewCount="589" Body="&lt;p&gt;I have a 2D surface in 3D that I want to advect under a velocity field.  More precisely, I have a surface $S$ and a velocity field $v$ and I want to advect $S$ under $v$ using the flow map of $v$, i.e. I want to compute $F^t(S_0)$, where $F^t$ is the flow of $v$. The velocity field is chaotic, so a direct computation leaves a bunch of scattered points.  $S_0$ is very smooth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been told that there are the so-called level set methods to do what I'd like.  The idea sounds very nice, but this is not my background and it would be great if there are any codes available that do this.&lt;/p&gt;&#xA;" OwnerUserId="1496" LastActivityDate="2012-10-25T11:33:06.260" Title="code for surface advection (e.g. level set advection)" Tags="|pde|advection|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3555" PostTypeId="1" AcceptedAnswerId="3558" CreationDate="2012-10-24T19:11:20.480" Score="12" ViewCount="539" Body="&lt;p&gt;I am trying to simulate a rope object.  The formulation I understand is an array of particles, connected by springs.  These springs have very large k-values, so that the line deforms, but stretches very little.  I have concluded that solving this as a function of time is not possible in closed-form because a rope is a generalization of a pendulum (which is not closed-form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Settling for approximate solutions, then.  I need an algorithm that scales well.  Examples I have seen use either explicit or implicit Eulerian integration to move the particles.  This does not scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see this, consider a rope with n nodes.  Apply a large force to one end.  Because the rope should not stretch very much, the acceleration at the other end must be immediate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, with Eulerian integration, to get &lt;em&gt;any&lt;/em&gt; force to the other end requires n steps.  I notice an exponential falloff: if the first node accelerates a certain amount, then the adjacent nodes accelerate less (if they accelerate at the same rate, then the algorithm is not stable).  Consequently, the nodes adjacent to &lt;em&gt;that&lt;/em&gt; node accelerate even slower!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for n nodes away, the acceleration is almost negligible.  This leads to rope that stretches significantly.  If you want to only double the simulation's resolution, you suddenly need to take time steps that are tens or hundreds of times smaller to get similar behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a simple method that solves this problem--i.e., higher resolution simulations converge to the solution with only polynomial time extra computation.  A full library of matrix and linear algebra techniques is available.  My knowledge of classical mechanics is very good, and I know some numerical analysis.&lt;/p&gt;&#xA;" OwnerUserId="3214" OwnerDisplayName="Ian Mallett" LastActivityDate="2012-10-25T13:23:23.843" Title="Arbitrary Accuracy Scalable Rope Simulation" Tags="|simulation|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3556" PostTypeId="2" ParentId="3555" CreationDate="2012-10-25T04:42:25.717" Score="5" Body="&lt;p&gt;You have a &lt;em&gt;stiff&lt;/em&gt; system with the current formulation. The dynamic stretching and vibration in the string are (presumably) uninteresting, but they control the explicit time step. This indicates using an implicit time integration method. You can use damping to prevent the oscillations which will tend to mess up adaptive error control for the implicit method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the fine-scale oscillations are important to model despite wanting to step over them (e.g., for fatigue modeling), then you may want to check out new multiscale methods such as the Heterogeneous Multiscale Method (Engquist, Tsai, etc) or semi-spectral in time methods. Use of such methods is a research-level topic and you have to understand your problem and the capabilities of the method well to decide whether it may be appropriate.  If you want to conserve energy, for example that certain vibrational modes should not dissipate, then you should look at symplectic integrators such as Verlet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also solve the zero-stretch limit if you like. With inertial terms, the model can be reformulated in terms of angles, leading to a non-stiff ODE system. As faleichik pointed out, this is the &lt;code&gt;ROPE&lt;/code&gt; test problem considered in Hairer, Nørsett, and Wanner's book. If you discard inertia of the rope itself, but allow slack (light, low-stretch rope with discrete loading; not a common model), the problem becomes a differential variational inequality (DVI) and you cannot generally get better than first order accuracy in time.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-10-25T13:23:23.843" LastActivityDate="2012-10-25T13:23:23.843" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3557" PostTypeId="2" ParentId="3555" CreationDate="2012-10-25T08:42:15.943" Score="2" Body="&lt;p&gt;If you're interested in a fast, approximate solution, then the methods used in digital effects such as discrete differential geometry may be of interest to you.  I'm aware of a quasistatic formulation in &lt;a href=&quot;http://www.cs.columbia.edu/cg/rods/&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Discrete Elastic Rods&lt;/em&gt;&lt;/a&gt;, a 2008 paper from Grinspun's group at Columbia University, but there is probably more recent literature in this area.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-10-25T08:42:15.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3558" PostTypeId="2" ParentId="3555" CreationDate="2012-10-25T09:05:24.613" Score="4" Body="&lt;p&gt;First of all, as &lt;a href=&quot;https://scicomp.stackexchange.com/a/3556/541&quot;&gt;Jed Brown&lt;/a&gt; has mentioned, you should use an implicit time-stepping scheme as your problem seems quite stiff, or at least a more stable, yet equally simple scheme such as &lt;a href=&quot;http://en.wikipedia.org/wiki/Leapfrog_integration&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leapfrog integration&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Verlet_integration&quot; rel=&quot;nofollow noreferrer&quot;&gt;Verlet integration&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the physical problem, how interested are you in the stretching? Instead of connecting the particles with stiff springs, you could use &lt;a href=&quot;http://en.wikipedia.org/wiki/Holonomic_constraints&quot; rel=&quot;nofollow noreferrer&quot;&gt;holonomic constraints&lt;/a&gt;, e.g. ensure that the distance between pairs of particles remains constant. The constraints need to be solved for at each time step, and there exist efficient algorithms for exactly your setup, i.e. a long linked chain of constraints. See, for instance, &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2008.07.002&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just out of curiosity, are you also using angular potentials along the length of the rope to model its flexibility?&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-10-25T09:05:24.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3559" PostTypeId="2" ParentId="3555" CreationDate="2012-10-25T10:21:22.933" Score="2" Body="&lt;p&gt;The movement of hanging rope is a beloved test problem of Hairer and Wanner which appeared in the second (stiff) volume of &quot;Solving Ordinary Differential Equations&quot; and in the second edition of the first volume (1993). I recommend the last option, page 247. The equations are tricky to derive and the algorithm of numerical solution is not very straightforward. Though at the end conventional explicit time steppers like DOPRI, RK45 or ODEX are applied and behave pretty well, so the problem is not really stiff.&lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2012-10-25T10:21:22.933" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3560" PostTypeId="2" ParentId="3554" CreationDate="2012-10-25T11:33:06.260" Score="5" Body="&lt;p&gt;Level set methods are really only one particular case of advection PDEs which you can solve with any of the modern PDE toolboxes such as deal.II (disclaimer: that's my own library), fenics, libmesh, ... What makes it a level set method is that you the evaluate the solution function to find that surface where, for example, the solution equals zero. I'd try to look into one of these PDE toolboxes and see what you can use to solve your problem. They certainly all can solve advection equations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-25T11:33:06.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3561" PostTypeId="1" AcceptedAnswerId="3562" CreationDate="2012-10-25T15:21:30.120" Score="8" ViewCount="733" Body="&lt;p&gt;I implemented a Navier Stokes based on FEM discretization and PETSc for solving the linear system of equations. To create an efficient solution procedure, I follow the paper &lt;a href=&quot;http://dx.doi.org/10.1016/S0377-0427%2800%2900515-X&quot; rel=&quot;noreferrer&quot;&gt;&amp;quot;Efficient preconditioning of the linearized Navier-Stokes equations for incompressible flow&amp;quot;&lt;/a&gt; (Silvester et al.), which propose a Schur complement approach. It works quite fine in the sense that I have a nearly constant number of iterations independently of mesh size and timestep for the simple benchmarks that are also presented in this paper (2D driven cavity flow and backward facing step). But at the moment, I solve the upper velocity block with a parallel direct solver (MUMPS). The pressure Schur block is solved with inexact solvers as proposed in the paper.&lt;/p&gt;&#xA;&lt;p&gt;In the paper, the authors propose to run in each outer iteration a single multigrid V-cycle and using a point Gauss-Seidel smoother to approximate the inverse of this discrete convection-diffusion operator. As I cannot easily use a geometric multigrid method, I thought to replace the direct solver by one algebraic multigrid V-cycle (boomeramg from the hypre package). But than I loose the constant number of iteratons while making the mesh finer.&lt;/p&gt;&#xA;&lt;p&gt;Does any of you has an idea how to create a spectrally equivalent and efficient preconditioner for the inverse of the velocity matrices based on algebraic multigrid? Is there something inherent which does not allow to make use of algebraic multigrid in this case? If not, what could be the source of loosing the constant iteration scaling?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I added some benchmarks for different solvers of the velocity block. The problem is solved is the standard 2D driven cavity flow, discretization with Taylor-Hood and a uniform refinement of the unit box&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Exakt solver (MUMPS)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{32}$&lt;/span&gt;: 25 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{64}$&lt;/span&gt;: 25 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{128}$&lt;/span&gt;: 25 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{256}$&lt;/span&gt;: 22 iter&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;one V-AMG (algebraic, boomeramg)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{32}$&lt;/span&gt;: 30 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{64}$&lt;/span&gt;: 30 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{128}$&lt;/span&gt;: 39 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{256}$&lt;/span&gt;: 48 iter&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;FGMRES with preconditioner V-AMG (algebraic, boomeramg), rtol: &lt;span class=&quot;math-container&quot;&gt;$10^{-6}$&lt;/span&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{32}$&lt;/span&gt;: 30 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{64}$&lt;/span&gt;: 29 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{128}$&lt;/span&gt;: 30 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{256}$&lt;/span&gt;: 47 iter&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;FGMRES with preconditioner V-AMG (algebraic, boomeramg), atol: &lt;span class=&quot;math-container&quot;&gt;$10^{-8}$&lt;/span&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{32}$&lt;/span&gt;: 27 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{64}$&lt;/span&gt;: 27 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{128}$&lt;/span&gt;: 28 iter&#xA;&lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{256}$&lt;/span&gt;: 26 iter&lt;/p&gt;&#xA;" OwnerUserId="1090" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-10-26T06:10:05.690" Title="Using algebraic multigrid for preconditioning convection-diffusion operators" Tags="|finite-element|petsc|linear-solver|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3562" PostTypeId="2" ParentId="3561" CreationDate="2012-10-26T03:12:45.657" Score="2" Body="&lt;p&gt;What you did is to replace the exact (direct) solver on the top left by one where you only do a single V-cycle. The way to go is to solve the top left block in your preconditioner using an iterative solver (e.g. GMRES) using the V-cycle AMG as a preconditioner to this inner solver. Experience shows that if you use F-GMRES as the outer solver, then the inner GMRES solver does not need to be terribly accurate. Just accurate enough to allow for it to yield an almost constant number of outer iterations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-10-26T03:12:45.657" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3563" PostTypeId="2" ParentId="3478" CreationDate="2012-10-26T20:50:31.190" Score="5" Body="&lt;p&gt;The noweb approach mentioned by Bill has evolved quite a bit, both in it's original spirit of documenting code (rather than scientific publication) under the term &lt;em&gt;literate programming&lt;/em&gt; and now comes in many flavors (I guess noweb was a generalization of cweb initially), of which &lt;code&gt;doxygen&lt;/code&gt; and various language specific versions can generate documentation in TeX, HTML, and other formats.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More to your point, noweb has been developed for some time in the &lt;code&gt;R&lt;/code&gt; community (well originally the &lt;code&gt;S&lt;/code&gt; community, hence the name) under the title &quot;Sweave&quot; with the goal of providing a &quot;reproducible research&quot; paper, where the code is actually run when the latex file is compiled (and optionally displayed as well).  Quite a number of academic papers are written in Sweave (including, I believe, all of the R-journal; but see also the journal of biostatistics and it's policy on reproducible papers).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While Sweave is still part of any base R installation, it is being replaced by &lt;a href=&quot;http://yihui.name/knitr&quot; rel=&quot;noreferrer&quot;&gt;knitr&lt;/a&gt; which is now &lt;a href=&quot;http://yihui.name/knitr/demo/engines/&quot; rel=&quot;noreferrer&quot;&gt;language agnostic&lt;/a&gt;, making it a possible choice for your python code.  Knitr supports writing in LaTeX or markdown, supporting syntax highlighting, caching, externalization of the code from the source latex and many other desirable features for this kind of work.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python has it's own solutions that are similar, &lt;a href=&quot;http://lighthouseinthesky.blogspot.com/2011/09/review-ipython-notebooks.html&quot; rel=&quot;noreferrer&quot;&gt;ipython notebooks&lt;/a&gt;, which can render to HTML, maybe LaTeX, but I know less about this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another project definitely worth a look is &lt;em&gt;&lt;a href=&quot;http://www.dexy.it/&quot; rel=&quot;noreferrer&quot;&gt;dexyit&lt;/a&gt;&lt;/em&gt;, another language-agnostic program that works very nicely with LaTeX and HTML.  While it has more examples in documenting code than in writing scientific papers, working in LaTeX it should be straight forward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both &lt;code&gt;knitr&lt;/code&gt; and &lt;code&gt;dexyit&lt;/code&gt; will do exactly what you describe in the LaTeX, including pointing to external python script and reading in the code.  Similar things can be accomplished in DocBook and XML, though I'm less familiar with this approach.   &lt;/p&gt;&#xA;" OwnerUserId="551" LastActivityDate="2012-10-26T20:50:31.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3564" PostTypeId="1" CreationDate="2012-10-27T14:45:38.113" Score="1" ViewCount="176" Body="&lt;p&gt;Let certain configuration of $n$ points exist in $d-$dimensional space, $X\in\mathbb{R}^{n\times d}$, $d&amp;lt;&amp;lt;n$. Also, let the corresponding Gram matrix be defined as $G=XX^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $X$ exists in Euclidean space, rank of $G$ is, $rank(G)=rank(X)=d$. Now, suppose that the configuration $X$ is &lt;em&gt;shifted&lt;/em&gt; to matrix $X'$ (&lt;em&gt;shift&lt;/em&gt; corresponds to the origin translation). Could it be possible that the Gram matrix, $G'=X'(X')^T$, has rank different from $G$, ie, $rank(G)\neq rank(G')$?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-10-30T10:50:46.620" LastActivityDate="2012-10-30T10:50:46.620" Title="Affecting the rank of a Gram matrix by configuration shift" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3565" PostTypeId="2" ParentId="2507" CreationDate="2012-10-27T18:13:18.293" Score="3" Body="&lt;p&gt;@Wodzu: this is a problem from radar data processing for tracking generally unfriendly items. Try searching for algorithms like IMM (interacting multiple models). Book &quot;Adaptive Filtering and Change Detection&quot; by Gustafsson (ISBN 0471492876) gives quite a bit of useful background on the field in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general (zero apriori information on likely routes) case you would use Kalman filter; however, for a road network you can deduce several hypotheses on where the car is going, and IMM (and its sister algos) based on continuous pathfinding solutions would be of more value here. If you don't have to be real-time and have the whole dataset handy, there exist even better statistical techniques (see a bunch of &lt;a href=&quot;https://stats.stackexchange.com/search?q=break%20detection&quot;&gt;Q/As at Cross-Validated SE&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: What I have left out at first: GPS does not have a neat error distribution, please be aware that there are some nasty properties (both in position and velocity errors).&lt;/p&gt;&#xA;" OwnerUserId="2155" LastEditorUserId="-1" LastEditDate="2017-04-13T12:44:17.923" LastActivityDate="2012-10-27T18:28:30.400" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3566" PostTypeId="1" CreationDate="2012-10-27T20:02:05.390" Score="15" ViewCount="970" Body="&lt;p&gt;I'm creating a simple astronomy simulator that should use Newtonian physics to simulate movement of planets in a system (or any objects, for that matter). All the bodies are circles in an Euclidean plane, that have properties such as position, velocity, mass, radius and the resultant force.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to update the universe in small time steps, usually a few milliseconds, but I'm not sure how to correctly calculate the changes in position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The force is simple: &lt;code&gt;fr = sum(G * body.m * bodyi.m / dist(body, bodyi)^2)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do I go on from there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = Fr/body.m&#xA;v += a*dt&#xA;position += v*dt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But that would, of course, be false. Maybe if I added 0.5 as a factor in position calculation?&lt;/p&gt;&#xA;" OwnerUserId="3223" LastEditorUserId="3223" LastEditDate="2012-10-27T23:50:22.477" LastActivityDate="2012-10-27T23:50:22.477" Title="What is the correct way of integrating in astronomy simulations?" Tags="|time-integration|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3567" PostTypeId="2" ParentId="3564" CreationDate="2012-10-27T21:07:16.657" Score="2" Body="&lt;p&gt;As @AronAhmadia mentioned, shifting one of the points to the origin provides a simple example where the rank changes. Consider the following case where $n=2$ and $d=2$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;X = \begin{pmatrix} 1 &amp;amp; 0 \\ 0 &amp;amp; 1 \end{pmatrix} = I,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where clearly $\text{rank}(X)=2$, and $G=XX^T=I$ has the same rank. But, if we shift the origin to the point $[1,0]^T$ to define&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde X = \begin{pmatrix} 0 &amp;amp; 0 \\ -1 &amp;amp; 1 \end{pmatrix},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then $\text{rank}(\tilde X)=\text{rank}(\tilde G)=1$.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-10-28T01:50:40.057" LastActivityDate="2012-10-28T01:50:40.057" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3568" PostTypeId="2" ParentId="3566" CreationDate="2012-10-27T22:41:17.453" Score="17" Body="&lt;p&gt;You've essentially got the answer - no need for the factor of 0.5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially you have a two-dimensional system of first-order ODEs:&#xA;\begin{align}&#xA;\dot{x} &amp;amp; = v \\&#xA;\dot{v} &amp;amp; = \frac{F}{m},&#xA;\end{align}&#xA;where everything is a function of time except presumably $m$, and dots denote time-derivatives. If you do a simple, forward-Euler-esque first-order differencing of these, you find&#xA;\begin{align}&#xA;\frac{x_{n+1}-x_n}{\Delta t} &amp;amp; = v_n \\&#xA;\frac{v_{n+1}-v_n}{\Delta t} &amp;amp; = \frac{F_n}{m},&#xA;\end{align}&#xA;or&#xA;\begin{align}&#xA;x_{n+1} &amp;amp; = x_n + \Delta t \cdot v_n \\&#xA;v_{n+1} &amp;amp; = v_n + \Delta t \frac{F_n}{m}.&#xA;\end{align}&#xA;Here I'm indexing the timestep with $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, forward-Euler is inherently unstable. Fortunately, there's a &lt;a href=&quot;http://en.wikipedia.org/wiki/Symplectic_integrator&quot;&gt;symplectic method&lt;/a&gt; right around the corner. (That linked article is more of a stub, but it might contain some useful links.) The key is to advance positions from $t_n$ to $t_{n+1}$ using velocities at $t_{n+1/2}$. That is, suppose you were given $x_0$ and $v_{1/2}$ for each particle. Then you could use&#xA;\begin{align}&#xA;x_{n+1} &amp;amp; = x_n + \Delta t \cdot v_{n+1/2} \\&#xA;v_{n+1/2} &amp;amp; = v_{n-1/2} + \Delta t \frac{F_n}{m}&#xA;\end{align}&#xA;to integrate forward in time. This is known as the &lt;a href=&quot;http://en.wikipedia.org/wiki/Leapfrog_integration&quot;&gt;leapfrog method&lt;/a&gt;. With this, your system conserves an energy of a sort, and orbits are less likely to fly off to infinity or some such thing due to exponential growth of roundoff error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only catch is how to get $v_{1/2}$, since presumably you start with $v_0$. There you should just use as accurate a scheme as you are up to writing, fourth-order Runge-Kutta being a popular choice. It may be long-term unstable, but there's only so much error you'll introduce in half a timestep, and that error will be kept small afterward by the leapfrog scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, this answer applies to any general Newtonian gravity sim. If you really want &lt;em&gt;perfect&lt;/em&gt; circles, as mentioned in passing in the question, then you won't get those except in an idealized system in which planets do not interact with each other and the initial conditions are chosen just right. If that's the case, then, you don't need to integrate at all, since the angular velocity (radians per unit time) of such an object is simply&#xA;$$ \omega = \sqrt{\frac{GM}{r^3}}, $$&#xA;where $M$ is the mass of the central object and $r$ is the radius of the orbit. This can be used to test the accuracy of your simulation.&lt;/p&gt;&#xA;" OwnerDisplayName="user3224" LastActivityDate="2012-10-27T22:41:17.453" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3572" PostTypeId="1" CreationDate="2012-10-29T13:17:50.710" Score="3" ViewCount="4134" Body="&lt;p&gt;I have a quadratic form $\mathbf{x}^T A \mathbf{x}$ (where $A\in \mathbb{R}^{n\times n}$ is symmetric matrix and $\mathbf{x}\in \mathbb{R}^n$) that I want to &lt;strong&gt;minimize&lt;/strong&gt; given the normalization constraint $\mathbf{x}^T\mathbf{x}=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because $\mathbf{A}$ is a adjacency matrix of an undirected graph then I know that it is symmetric and real and also sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is an appropriate memory conservative algorithm to solve this kind of problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it good to solve the eigensystem $\mathbf{A}\mathbf{x}=\lambda \mathbf{x}$ and then taking as solution the first smallest nonzero eigenvalue and its related eigenvector?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is the way to proceed how is it possible to get the smallest eigenvalue with subspace iteration?&lt;/p&gt;&#xA;" OwnerUserId="3186" LastEditorUserId="3186" LastEditDate="2012-10-30T10:17:06.067" LastActivityDate="2012-10-30T10:17:06.067" Title="An eigenvalue algorithm to solve constrained quadratic form minimization" Tags="|optimization|eigensystem|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3573" PostTypeId="2" ParentId="3572" CreationDate="2012-10-29T13:53:59.287" Score="4" Body="&lt;p&gt;The optimal objective function value will be the minimal eigenvalue of $\mathbf{A}$, &lt;em&gt;including&lt;/em&gt; zero eigenvalues, and the optimal solution will be the eigenvector of $\mathbf{A}$ corresponding to that eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eigenvectors, by definition, must be nonzero vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to derive this answer is to note that a symmetric matrix $\mathbf{A}$ will always have the eigendecomposition $\mathbf{A}$ = $\mathbf{Q}^{T}\boldsymbol{\Lambda}\mathbf{Q}$, where $\mathbf{Q}$ is an orthogonal matrix and $\boldsymbol{\Lambda}$ is a diagonal matrix consisting of the eigenvalues of $\mathbf{A}$ (such that they correspond correctly to the columns of $\mathbf{Q}$). Using the transformation $\mathbf{y} = \mathbf{Qx}$, and recalling that orthogonal matrices preserve the standard inner product norm, the problem $\min_{\mathbf{x}^{T}\mathbf{x} = 1}\mathbf{x}^{T}\mathbf{A}\mathbf{x}$ is equivalent to $\min_{\mathbf{y}^{T}\mathbf{y} = 1} \mathbf{y}^{T}\boldsymbol{\Lambda}\mathbf{y}$. The solution to the latter problem is a vector whose only nonzero entry is a one, at the index corresponding to the minimal eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Arnold Neumaier states, Lanczos would work.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-10-29T14:01:58.933" LastActivityDate="2012-10-29T14:01:58.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3574" PostTypeId="2" ParentId="3572" CreationDate="2012-10-29T13:57:13.807" Score="3" Body="&lt;p&gt;$x=0$ does not satisfy the constraint, hence is irrelevant. The solution to your problem is any normalized eigenvector belonging to the smallest eigenvalue of $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An eigenvector to the smallest eigenvalue of a large and sparse matrix can be found efficiently by the Lanczos algorithm, see &lt;a href=&quot;http://en.wikipedia.org/wiki/Lanczos_algorithm&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Lanczos_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-29T13:57:13.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3575" PostTypeId="2" ParentId="3564" CreationDate="2012-10-29T15:23:53.770" Score="1" Body="&lt;p&gt;With your updated question, the answer still remains the same, you can still lose rank due to configuration shift if the points are trivially distributed.  Appealing back to &lt;a href=&quot;https://scicomp.stackexchange.com/a/3567/9&quot;&gt;Jack Poulson's answer&lt;/a&gt;, imagine now a case where $n &amp;gt;&amp;gt; 2$, but every point except for one is along the axis $(1,0)$.  Assume this last point is on the other axis $(0,1)$.  If your linear shift moves the point on $(0,1)$ to $(0,0)$, the rank of the shifted Gram operator becomes one-dimensional. &lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-10-29T15:23:53.770" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3576" PostTypeId="1" AcceptedAnswerId="3580" CreationDate="2012-10-30T11:00:36.267" Score="4" ViewCount="59" Body="&lt;p&gt;What would then be the way to determine a true dimensionality of a configuration of points $X\in\mathbb{R}^{n\times k}$ based on its Gram matrix $G=XX^T$? The &quot;true&quot; dimensionality refers to the minimum number of dimensions needed to represent a configuration (eg, a line is 1D, but can be represented in 2D). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different shifts would imply different ranks of $G$ that correspond to different dimensionalities. Consider, for instance, the paper &#xA;&lt;a href=&quot;http://convexoptimization.com/TOOLS/Gower1.pdf&quot; rel=&quot;nofollow&quot;&gt;http://convexoptimization.com/TOOLS/Gower1.pdf&lt;/a&gt;  (Sect. 3, very short, please consider)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly, as long as the shifts are of the form&#xA;$X'=PX$, $P=I_n-1_nw^T$, where $1_n^Tw=1$, the rank of $G'=X'(X')^T$ is the true dimensionality of $X$? If you consider the 2x2 example from the above paper, a shift where one point is the origin reduces the dimensionality to 1D, which is the true dimensionality (same holds for a centroid origin)&#xA;Does that mean that, in order to recover the &quot;true&quot; configuration, one is restricted to the above form of the shift?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-10-30T15:51:24.520" LastActivityDate="2012-10-30T16:22:53.537" Title="Configuration shift for determination of a true dimensionality" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3577" PostTypeId="1" CreationDate="2012-10-30T14:00:42.477" Score="4" ViewCount="219" Body="&lt;p&gt;The E-M algorithm does not guarantee convergence to global maxima. Following Applet shows this nicely: &lt;a href=&quot;http://www.cs.cmu.edu/~alad/em/&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.cmu.edu/~alad/em/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to generate data set, that guarantees reaching global maximum? Maybe few simple constraints imposed on generation process is enough to guarantee, that E-M will find global maximum?&lt;/p&gt;&#xA;" OwnerUserId="3246" LastActivityDate="2012-10-30T14:38:36.667" Title="Expectation-Maximization and local maxima" Tags="|heuristics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3578" PostTypeId="2" ParentId="924" CreationDate="2012-10-30T14:23:54.500" Score="3" Body="&lt;p&gt;The correct answer was already given by Jed Brown in a comment, but a brief explanation might help. &lt;em&gt;Shape regularity&lt;/em&gt; enters into finite element methods when stitching together local interpolation errors to get a global interpolation error (which is an upper bound for the approximation error and hence, by Céa's lemma, for the discretization error). Basically, you get local interpolation errors by estimating the interpolation error on a reference element (using the Bramble-Hilbert lemma) and then transforming it to each local element. This transformation gives you &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the required power of the element size $h$ and&lt;/li&gt;&#xA;&lt;li&gt;a power of the condition number of the Jacobian of the transformation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;on the right hand side of the estimate. In order to obtain a global error estimate with only this power of (the maximal occuring) $h$ on the right hand side, you thus need a uniform bound for this condition number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicial meshes, you can show (independently of the dimension!) that this condition number can be estimated by the ratio of the diameters of the inscribed and the circumscribed (or minimal containing) ball. (In two dimensions, this ratio can in turn be estimated by $2/\sin(\vartheta)$, where $\vartheta$ is the smallest angle of the triangle.) If this ratio is uniformly bounded for all elements, the mesh is called shape regular. Hence in this case the usual definition is already dimension independent. You can find a detailed treatment in Chapter 1.5 of &lt;a href=&quot;http://www.springer.com/mathematics/applications/book/978-0-387-20574-8&quot; rel=&quot;nofollow&quot;&gt;Ern and Guermond, &lt;em&gt;Theory and Practice of Finite Elements&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(For rectangular meshes, you would probably use bounds of the condition number in terms of ratios of maximal and minimal edge lengths, which also makes sense for arbitrary dimensions, although I don't have any reference for that.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is a separate issue from &lt;em&gt;mesh quality&lt;/em&gt;, which also takes into account the conditioning of the linear system arising from the discretization, and ignores the magnitude of the constants in the mentioned estimates.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2012-10-30T14:34:44.460" LastActivityDate="2012-10-30T14:34:44.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3579" PostTypeId="2" ParentId="3577" CreationDate="2012-10-30T14:38:36.667" Score="1" Body="&lt;p&gt;Increasing the number of data points often helps reducing the number of local maxima. There is no other general recipe to avoid converging to nonglobal maxima with EM, except to ensure that these don't exist. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or switch the method and use a branch and bound technique. It will find a globab maximum, if it has enough space and time for running to completion.&#xA;But for typical statistical estimation problems this tends to happen only for fairly small problems.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-30T14:38:36.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3580" PostTypeId="2" ParentId="3576" CreationDate="2012-10-30T16:22:53.537" Score="2" Body="&lt;p&gt;Yes. You need to move the affine subspace to the origin by subtracting from the points (rows of $X$) some point in this affine subspace, i.e., a linear combination of the given points. This leads to the form of $P$. The rank of the matrix $PX$ formed by the shifted points then gives the dimension of the subspace.&#xA;As the rank of $X$ and $G=XX^T$ is the same, the rank of $PGP^T$ gives the correct dimension.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-10-30T16:22:53.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3581" PostTypeId="1" AcceptedAnswerId="3582" CreationDate="2012-10-30T19:58:13.010" Score="8" ViewCount="2465" Body="&lt;p&gt;If I want to use large array say mpz_t A[100000], &#xA;I got &quot;Segmentation fault (core dumped)&quot; during my&#xA;compilation. Is there any easier way to solve this? &lt;/p&gt;&#xA;" OwnerUserId="277" LastActivityDate="2012-11-06T04:29:03.917" Title="Large array in GMP" Tags="|programming-paradigms|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3582" PostTypeId="2" ParentId="3581" CreationDate="2012-10-31T00:17:35.447" Score="6" Body="&lt;p&gt;tpg2114's comment is spot on.  Try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/* at top */&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;&#xA;/* definition */&#xA;mpz_t *A;&#xA;&#xA;/* initialization of A */&#xA;A = (mpz_t *) malloc(100000 * sizeof(mpz_t));&#xA;if (NULL == A) {&#xA;    printf(&quot;ERROR: Out of memory\n&quot;);&#xA;    return 1;&#xA;}&#xA;&#xA;/* no longer need A */&#xA;free(A);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the &lt;code&gt;malloc&lt;/code&gt; call here triggers an error, you don't have enough memory available in your system.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in using a static array on the stack, then you can try increasing the stack limit size in Linux with the &lt;code&gt;ulimit&lt;/code&gt; command.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2012-10-31T12:08:41.977" LastActivityDate="2012-10-31T12:08:41.977" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3583" PostTypeId="1" AcceptedAnswerId="3624" CreationDate="2012-10-31T15:58:47.467" Score="9" ViewCount="838" Body="&lt;p&gt;I'm using the Matlab pde toolbox to solve a certain elliptic equation in 2D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution is fine, although I do need to plot it along a given line, i.e. to cut a planar slice from the 3D mesh representing the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't figure out a way that smartly involves the toolbox functions (i.e. not involving low level interpolation on the triangular mesh).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3248" LastEditorUserId="3248" LastEditDate="2012-10-31T18:11:35.553" LastActivityDate="2012-11-11T20:59:37.723" Title="Matlab Pde Toolbox: Plot solution on a line or on a submanifold" Tags="|pde|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3584" PostTypeId="1" CreationDate="2012-10-31T16:26:53.440" Score="5" ViewCount="66" Body="&lt;p&gt;I have the intuition that the minimal point (in the sense of having the lowest value in one of the Euclidean space dimensions)  of a intersection of $n$ convex sets is always the minimal point of the intersection of two convex sets in the family. Has this been proven before?&lt;/p&gt;&#xA;" OwnerUserId="3249" LastActivityDate="2012-10-31T17:16:30.483" Title="Minimal point of a intersection of $n$ convex sets is always the minimal point of the intersection of two convex sets?" Tags="|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3585" PostTypeId="2" ParentId="3584" CreationDate="2012-10-31T17:16:30.483" Score="3" Body="&lt;p&gt;In $d$ dimensions, you need at least $d$ convex sets, not two.  For example, in a suitably rotated polytope in $d$ dimensions, the minimal point will be a vertex, which is defined by the intersection of $d$ planes.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-10-31T17:16:30.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3587" PostTypeId="1" CreationDate="2012-10-31T23:23:15.873" Score="6" ViewCount="401" Body="&lt;p&gt;We're dealing with country borders, that is the set of multiple disjoint domains that is made of polygons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To extract the different point on the map by a given country we've been said to implement the &lt;a href=&quot;http://en.wikipedia.org/wiki/Point_in_polygon&quot; rel=&quot;nofollow noreferrer&quot;&gt;ray casting algorithm&lt;/a&gt; which is absolutely fine, but the country is hardly to be the simply-connected domain. For example, the Italy consists of many disjoint domains (islands). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case the crossing number algorithm or the even-odd rule algorithm is still a valid approach to validate if the point in question is within the country borders or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After days of research finally found one discussion: &lt;a href=&quot;https://stackoverflow.com/questions/7573881/mysql-implementation-of-ray-casting-algorithm/13196590&quot;&gt;MySQL implementation of ray-casting Algorithm?&lt;/a&gt; which gives the exhausting number of links, answers and code snippets to implement the algorithm in code (Pseudo-code: &lt;a href=&quot;http://rosettacode.org/wiki/Ray-casting_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rosettacode.org/wiki/Ray-casting_algorithm&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3252" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-11-02T14:13:22.447" Title="Ray casting algorithm for multiple disjoint polygons is still valid?" Tags="|algorithms|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3588" PostTypeId="2" ParentId="3587" CreationDate="2012-10-31T23:46:54.590" Score="3" Body="&lt;p&gt;Yes, the even-odd rule works for any closed set of polygons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing to be careful about is the ray hitting a vertex, since code which is not equipped to handle degeneracies may return incorrect results in that case.  The lowest effort solution is usually to (1) first check if you are on the boundary of the polygon and (2) try rays with random directions until no vertex is hit.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-10-31T23:46:54.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3589" PostTypeId="1" CreationDate="2012-11-01T12:09:05.870" Score="4" ViewCount="196" Body="&lt;p&gt;I'm using following code to generate a picture of the reaction-diffusion process:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2887/explicit-euler-method-too-slow-for-reaction-diffusion-problem&quot;&gt;Explicit Euler method too slow for reaction-diffusion problem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to obtain an animation effect. When I draw the solution step by step, it does give good animation. The problem is that the solution quickly converges to uniform, solid color (i.e. uniform concentration of chemical species).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to sustain the animation? The solution should reverse the Euler method to some degree? Adding random noise sustains the animation, but also adds visible noise&lt;/p&gt;&#xA;" OwnerUserId="3246" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-01-21T20:59:23.000" Title="Reaction-diffusion system animation" Tags="|pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3590" PostTypeId="2" ParentId="3589" CreationDate="2012-11-01T13:25:43.560" Score="4" Body="&lt;p&gt;If I remember correctly from the original question, the main purpose is to have an &quot;interesting&quot; visual output.  While this somewhat lines up with your noise comment, likely the best way to maintain a transient solution is to have time dependent boundary conditions.  As time goes on and your animation stabilizes, choose a random point on the edge of the image and impose a non-equilibrium concentration (how much effort you want to put into making it appear smooth the current solution is up to you).  This will cause your solution to remain transient while avoiding the visible noise you mentioned.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-11-01T13:25:43.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3591" PostTypeId="1" CreationDate="2012-11-01T13:40:52.303" Score="6" ViewCount="168" Body="&lt;p&gt;I would like to compute the flow rate (mL/s) of a pipe flow given the 3D velocity field $\mathbf{v} = (v_x, v_y, v_z)$ over the computational domain (a curved pipe). The field is represented in the form of a voxel grid with uniform sampling distances along each dimension. The flow field is a result of a numerical simulation of the flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general the flow rate can be computed as the flux of the velocity field over a cut plane of the 3D domain (pipe).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q = \int_S v \mathbf{n} dA$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this approach however the solution only depends on samples from a subset of the available results, and in order to improve the accuracy of the computation I would like to use all available data. This could be achieved by computing the flow rate as an average of multiple flux values based on different sampling planes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that the considered domain is a pipe, it is possible to estimate the centerline in the form of a curve $z$, and then using test surfaces that are planes perpendicular to the curve along equally spaced locations $z_i, i\in(1,\dots, N)$ and taking $N\to\infty$, we get the following expression &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q = \frac{1}{L}\int_Z\int_S v \mathbf{n} dAdz$, with L the length of the curve z&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make the computations simpler I would like to avoid the explicit computations of the flux values for the test surfaces (cutting planes). What I am aiming for is to express the solution as a volume integral of some kind, that could be trivially evaluated without any knowledge of the topology of the domain in 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was trying to use the divergence theorem first, however when applied to the computational domain (pipe interior) it would basically contain the difference of fluxes at the inlet and outlet (assuming no flux through the walls) and I could not get anything meaningful out of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas / hints are very much appreciated!   &lt;/p&gt;&#xA;" OwnerUserId="3229" LastEditorUserId="276" LastEditDate="2017-10-10T00:46:07.490" LastActivityDate="2017-10-10T00:46:07.490" Title="Computing flux of vector field numerically with regular grids" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3592" PostTypeId="1" CreationDate="2012-11-01T14:17:07.443" Score="13" ViewCount="6252" Body="&lt;p&gt;In this sample program I'm doing the same thing (at least I think so) in two different ways. I'm running this on my Linux pc and monitoring the memory usage with top. Using gfortran I find that in the first way (between &quot;1&quot; and &quot;2&quot;) the memory used is 8.2GB, while in the second way (between &quot;2&quot; and &quot;3&quot;) the memory usage is 3.0GB. With the Intel compiler the difference is even larger: 10GB versus 3GB. This seems an excessive penalty for using pointers. Why does this happen?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program test&#xA;implicit none&#xA;&#xA;  type nodesType&#xA;    integer:: nnodes&#xA;    integer,dimension(:),pointer:: nodes &#xA;  end type nodesType&#xA;&#xA;  type nodesType2&#xA;    integer:: nnodes&#xA;    integer,dimension(4):: nodes &#xA;  end type nodesType2&#xA;&#xA;  type(nodesType),dimension(:),allocatable:: FaceList&#xA;  type(nodesType2),dimension(:),allocatable:: FaceList2&#xA;&#xA;  integer:: n,i&#xA;&#xA;  n = 100000000&#xA;&#xA;  print *, '1'&#xA;  read(*,*)&#xA;  allocate(FaceList(n))&#xA;  do i=1,n&#xA;    FaceList(i)%nnodes = 4&#xA;    allocate(FaceList(i)%nodes(4))&#xA;    FaceList(i)%nodes(1:4) = (/1,2,3,4/)&#xA;  end do&#xA;  print *, '2'&#xA;  read(*,*)&#xA;&#xA;  do i=1,n&#xA;    deallocate(FaceList(i)%nodes)&#xA;  end do&#xA;  deallocate(FaceList)&#xA;&#xA;  allocate(FaceList2(n))&#xA;  do i=1,n&#xA;    FaceList2(i)%nnodes = 4&#xA;    FaceList2(i)%nodes(1:4) = (/1,2,3,4/)&#xA;  end do&#xA;  print *, '3'&#xA;  read(*,*)&#xA;&#xA;end program test&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The background is local grid refinement. I chose the linked list to easily add and remove faces. The number of nodes is 4 by default but can become higher depending on the local refinements.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastEditorUserId="1973" LastEditDate="2012-11-01T19:56:35.287" LastActivityDate="2019-09-24T14:53:38.203" Title="Memory usage in fortran when using an array of derived type with pointer" Tags="|performance|fortran|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3593" PostTypeId="2" ParentId="3592" CreationDate="2012-11-01T15:01:46.883" Score="6" Body="&lt;p&gt;I don't actually know how fortran compilers work, but based on the language features, I can take a guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dynamic arrays in fortran come with meta-data to work with intrinsic functions like shape, size, lbound, ubound, and allocated or associated (allocatable vs pointers).  For large arrays, the size of the meta-data is negligible, but for tiny arrays, like in your case, it can add up.  In your case, the size 4 dynamic arrays likely have more meta-data than real data, which is leading to your memory usage balloon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd strongly recommend against dynamic memory at the bottom of your structures.  If you're writing a code that deals with physical systems in some number of dimensions, you can set it as a macro and recompile.  If you dealing with graphs, you can statically allocate an upper bound on the number of edges or the likes.  If you are dealing with a system that actually needs fine-grain dynamic memory control, then it is probably best to switch to C.&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2012-11-01T15:01:46.883" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3594" PostTypeId="1" CreationDate="2012-11-01T16:17:47.420" Score="5" ViewCount="1585" Body="&lt;p&gt;CPU benchmarks available online mostly focus on desktop apps/games and rarely on serial/parallel numerical kernels, specially sparse ones (e.g., MatMult). Some benchmarks like NAS/SciMark exists but are rarely used by popular benchmarking sites on lastest/greatest CPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to codes that use sparse/dense numerical libraries such as PETSc/ScaLAPACK how does one decide which machine to buy/use specially w.r.t. to multicore performance? E.g. would I be better off with an AMD FX8350 or a Intel Core i7-3770K as my Desktop or should I request allocation on a Xeon or Opteron based cluster, specially for small/medium (i.e., less than 4000 cores) type jobs where one has more options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know from personal experience that Intel chips have performed significantly better in the last few years but what about the latest AMD stuff?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I am specifically interested in PETSc's MatMult kernel (for unstructured FE matrices) which I have been told is memory bound. Performance in terms of percentage of peak FLOPS using all cores I think would be useful.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="20688" LastEditDate="2021-12-22T13:40:53.863" LastActivityDate="2021-12-22T13:40:53.863" Title="CPU benchmarks for numerical kernels" Tags="|linear-solver|sparse-matrix|petsc|benchmarking|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3595" PostTypeId="2" ParentId="3594" CreationDate="2012-11-01T18:36:47.727" Score="2" Body="&lt;p&gt;Raw benchmarks are correlated with performance for certain, but to have a real answer about what is optimal for your particular problem, &lt;strong&gt;you need to profile your code on the system in question.&lt;/strong&gt;  IMHO it is impossible to say authoritatively without some actual performance testing on your code.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, as mentioned in the question thread, most matrix/vector operations are memory bound, and become fairly dependent on so many factors, like structure of the matrix, how your compiler optimizes memory access, cache-hits, ram latency, OS foolery, the list goes on and on.  I think it is &lt;em&gt;very&lt;/em&gt; hard to make any substantive claims without actual testing on the system in question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have had similar experience with Intel's hardware as you mention.  Especially when using the icc tool-chain, as it seems to really know how to write machine code that uses the full capabilities of the underlying metal.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2012-11-01T18:36:47.727" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3596" PostTypeId="2" ParentId="3592" CreationDate="2012-11-01T21:32:19.660" Score="5" Body="&lt;p&gt;As &lt;a href=&quot;https://scicomp.stackexchange.com/a/3593/541&quot;&gt;maxhutch&lt;/a&gt; has pointed out, the problem is probably the sheer number of separate memory allocations. On top of the metadata, though, there's probably whatever additional data and alignment the memory manager needs, i.e. it is probably rounding each allocation up to some multiple of 64 bytes or more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid allocating a small chunk for each node, you could try allocating each node a &lt;em&gt;portion&lt;/em&gt; of a pre-allocated array:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integer :: finger&#xA;indeger, dimension(8*n) :: theNodes&#xA;&#xA;finger = 1&#xA;do i=1,n&#xA;    FaceList(i)%nodes =&amp;gt; theNodes(finger:finger+FaceList(i)%nnodes-1)&#xA;    finger = finger + FaceList(i)%nnodes&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My Fortran is a bit rusty, but the above should work, if not in principle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'd still have the overheads of whatever your Fortran compiler thinks it needs to store for a POINTER type, but you won't have the memory manager overheads.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2012-11-02T10:41:08.963" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3597" PostTypeId="2" ParentId="3594" CreationDate="2012-11-01T21:59:19.597" Score="3" Body="&lt;p&gt;There are benchmarks that are derived from typical scientific computing codes doing sparse matrix vector stuff in SPEC CPU 2006, see &lt;a href=&quot;http://www.spec.org/cpu2006/&quot; rel=&quot;nofollow&quot;&gt;http://www.spec.org/cpu2006/&lt;/a&gt; . In particular, the 447.dealII benchmark in this suite derives from the deal.II library and spends most of its time in exactly the kind of operations that you describe for your workload. The advantage of this is that you can query the SPEC database for the peak rates on this benchmark to see how various processors (variants) perform on this one benchmark. Pretty much every CPU built over the past 10 years is in that database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I'm the author of 447.dealII.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-01T21:59:19.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3598" PostTypeId="1" AcceptedAnswerId="3601" CreationDate="2012-11-02T00:02:10.553" Score="4" ViewCount="1496" Body="&lt;p&gt;I'm working on an finite element code to solve the boundary value problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\frac{d}{dx}\left[ k \frac{du}{dx} \right] = f $$ &#xA;$$u(0)=u(1)=0$$&#xA;The matlab code is available &lt;a href=&quot;http://www.math.utep.edu/Faculty/pmdelgado2/FEM1D.m&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm testing this code in the case where $k=1$ and the exact solution is:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x) = x(1-x)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x)=-ku''=2$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this information, the stiffness matrix was created using piecewise linear basis functions (hat functions) and with equispaced nodes.  Checking the plot of the graphs of the approximate and exact solutions, I see that they are encouragingly close to each other.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also calculate the discrete $L^{\infty}$ norm error, according to the following formula:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$||U_{exact}-U_{approx}||_{L^{\infty}}=max_{x_i} \{ U_{exact}(x_i)-U_{approx}(x_i)\}$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tested this code varying the number of elements as $xnel=10,20,40,...$ (i.e. successive doubling).  In doing so, I noticed that this error is actually increasing as the number of element increases (i.e. the size of each element decreases).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've combed the code for mistakes, but I haven't found any thus far.  Could it really be possible that the error in the discrete $L^{\infty}$ norm actually increases as the element size decreases?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-11-02T13:00:39.927" Title="Should the discrete $L^{\infty}$ norm error increase as the mesh refines?" Tags="|finite-element|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3599" PostTypeId="1" CreationDate="2012-11-02T00:11:57.173" Score="1" ViewCount="123" Body="&lt;p&gt;A current trend in cognitive science is to view the mind as a dynamical system (e.g., &lt;em&gt;Continuity of Mind&lt;/em&gt; by Spivey, in which cognition is understood as a &quot;continuous and often recurrent trajectory through a state space&quot;). Although I'd like to critically evaluate this trend, I'm embarresed to admit that &lt;em&gt;I've never taken even a basic calculus course.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet since I don't intend to build dynamical systems models myself, what is the bare minimum of maths learning that I need to accomplish in order to understand dynamical systems in the context of psychology? Remember, I'm a total novice!&lt;/p&gt;&#xA;" OwnerUserId="3258" LastActivityDate="2012-11-02T03:07:27.987" Title="What kinds of maths to learn for understanding dynamical systems in cognitive science?" Tags="|high-dimensional|neuroscience|" AnswerCount="1" CommentCount="4" ClosedDate="2012-11-04T02:48:28.713" ContentLicense="CC BY-SA 3.0" />
  <row Id="3600" PostTypeId="2" ParentId="3594" CreationDate="2012-11-02T00:31:51.073" Score="6" Body="&lt;p&gt;First, sparse direct is &lt;em&gt;completely&lt;/em&gt; different from sparse iterative. You cannot reliably predict performance if you don't have a good understanding of what your code is doing. For sparse &lt;code&gt;MatMult&lt;/code&gt;, &lt;code&gt;MatSolve&lt;/code&gt;, &lt;code&gt;MatSOR&lt;/code&gt;, and similar kernels, you have an arithmetic intensity of no more than 1 flop/4 bytes of memory bandwidth. Meanwhile, most recent multicore chips can perform about 5 flops/byte. This means that even if cache is reused perfectly, you cannot get better than $1/20 = 5\%$ of floating point peak. This is a hard barrier that no amount of &quot;tuning&quot; can circumvent. In practice, it is very important to &lt;em&gt;order&lt;/em&gt; your matrices so that cache is reused well. If you choose an ordering that produces poor reuse, the realized peak will be several times lower. In practice, with decent orderings and sufficiently large subdomains, PETSc sparse matrix kernels frequently get $80\%$ of memory bandwidth peak or better. Look at the STREAM benchmarks to determine the realizable bandwidth. Some machines, such as Ranger at TACC, cannot realize better than $50\%$ of peak bandwidth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you resort to assembled sparse matrices to be solved using iterative methods, there is absolutely no point buying cores. Buy memory bandwidth. I recommend testing algorithms with assembled matrices, then switching to mostly-unassembled methods (perhaps matrix-free multigrid), when appropriate, if you want to get high machine utilization. A better choice of algorithm can give an order of magnitude speedup in so many cases that there's no point micro-tuning a suboptimal method.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-11-02T00:31:51.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3601" PostTypeId="2" ParentId="3598" CreationDate="2012-11-02T01:01:46.150" Score="4" Body="&lt;p&gt;If you use linear finite elements, then theory tells us that the $L^\infty$ error decreases as&#xA;$$&#xA;  \|e\|_{L^\infty} \le C \;h^2 \; |\log h| \; \|u\|_{H^2}&#xA;$$&#xA;where $h$ is the mesh size. In fact, in 1d, the finite element approximation &lt;em&gt;equals&lt;/em&gt; the interpolant of the solution, and so the error should even decay as $h^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I could imagine is that the definition above applies to the $L^\infty$ norm whereas you only evaluate it at the nodes. You should evaluate it at other points as well in order to get the &lt;em&gt;real&lt;/em&gt; $L^\infty$ error.&#xA;Do you see the correct error orders in the other norms?&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-02T01:01:46.150" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="3602" PostTypeId="2" ParentId="3599" CreationDate="2012-11-02T03:07:27.987" Score="3" Body="&lt;p&gt;That all depends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not having read the psychology literature at all, the minimum you probably need is 2-3 semesters of calculus and 1 semester of differential equations, and that's to understand something on the level of &lt;em&gt;Nonlinear Dynamics And Chaos: With Applications To Physics, Biology, Chemistry, And Engineering (Studies in Nonlinearity)&lt;/em&gt; by Steven Strogatz. Even then, you'd probably understand the big picture, but not any of the subtleties. You could try reading that book without the math background and see how far you get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To really start getting into the maths of dynamical systems, you'd need 1-2 semesters of real analysis on top of that (at the level of Rudin's &lt;em&gt;Principles of Mathematical Analysis&lt;/em&gt; and, say, Munkres' &lt;em&gt;Analysis on Manifolds&lt;/em&gt;) to be able to tackle something like &lt;em&gt;Nonlinear Systems (3rd Edition)&lt;/em&gt; by Hassan Khalil, which would delve into much more advanced ways of quantifying and proving properties of dynamical systems. It is probably not worth you getting that much background to study dynamical systems, but maybe it is of interest to other people.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-11-02T03:07:27.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3603" PostTypeId="2" ParentId="3220" CreationDate="2012-11-02T05:46:11.993" Score="1" Body="&lt;p&gt;You can try the Fast PCA algorithm which is based on an iterative way of computing a few eigenvectors. See, &lt;a href=&quot;https://maxwell.ict.griffith.edu.au/spl/publications/papers/prl07_alok_pca.pdf&quot; rel=&quot;nofollow&quot;&gt;A.Sharma and K.K. Paliwal, Fast principal component analysis using fixed-point analysis, Pattern Recognition Letters, 28, 1151-1155, 2007&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3259" LastEditorUserId="95" LastEditDate="2012-11-11T20:25:17.907" LastActivityDate="2012-11-11T20:25:17.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3605" PostTypeId="2" ParentId="3598" CreationDate="2012-11-02T13:00:39.927" Score="1" Body="&lt;p&gt;Just to riff on Wolfgang's post, if on each element the error takes the form:&#xA;$$&#xA;e_h(x)=x(1-x) -(a_h x +b_h)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then this error has its extreme value where its derivative is zero, i.e. at&#xA;$$&#xA;x_h=\frac{1-a_h}{2}&#xA;$$&#xA;Assuming I've done the math right, you ought to be able to evaluate the $L_\infty$ error without having to search each element for the maximum value of its error.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-11-02T13:00:39.927" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3610" PostTypeId="1" AcceptedAnswerId="3612" CreationDate="2012-11-03T17:46:38.303" Score="9" ViewCount="531" Body="&lt;p&gt;For my research work I have been developing a small C++ library aimed at facilitating the communication between C++ computational codes and Octave/Matlab (when the latter is used for post-processing purposes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like release such a library under GPL on some free hosting site such as Sourceforge or Github. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Honestly, though, I'm lacking in experience to choose which site better suits a (small) library as mine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On one hand, github looks to me more immediate, on the other hand, Sourceforge is a reference site for scientific codes, many of them are hosted there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; I enlarge my question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The library I am talking about is a small project ($\approx 15K$ CL, currently in a private repo on bitbucket), almost personal, which followed 2 years of boring technical requests by my supervisors and 6 months of code design (mostly by trial and error).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some inquires I noticed that sourceforge has a nice mechanism allowing one to make statistics about visits of the project page and downloads, thus people (users) just downloading the lib to &lt;em&gt;use it&lt;/em&gt; (if ever any) can be somehow counted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Github, on the other hand, looks way easier from the point of view of developers (here I mean someone that is interested also in analyzing,extending,&lt;em&gt;forking&lt;/em&gt; the code).&lt;/p&gt;&#xA;" OwnerUserId="3248" LastEditorUserId="3248" LastEditDate="2012-11-19T12:35:02.260" LastActivityDate="2012-11-19T12:35:02.260" Title="Hosting site for a small scientific library" Tags="|matlab|libraries|c++|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3612" PostTypeId="2" ParentId="3610" CreationDate="2012-11-03T18:09:26.970" Score="9" Body="&lt;p&gt;Whatever site is &lt;em&gt;better&lt;/em&gt; will depend on your own appreciation of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although it might be tempting to promote whatever system I prefer personally, I believe that the correct choice has more to do with your personal preferences and style of working...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many developers work on your project? How often will you be updating it? How likely is it that somebody will want to fork it? What revision system are you more comfortable with? Which user interface do you prefer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Honestly, both &lt;a href=&quot;http://sourceforge.net&quot; rel=&quot;nofollow&quot;&gt;SourceForge&lt;/a&gt; and &lt;a href=&quot;http://github.com&quot; rel=&quot;nofollow&quot;&gt;Github&lt;/a&gt; are mature sites and both offer some awesome tools. But if you're not comfortable using them, then however amazing other users may say they are, they will be of limited benefit for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both sites are equally good, they just choose to do things differently. They are both well referenced by search engines and downloading software from one is just as easy as from the other&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would strongly suggest you look at both and see what you're most comfortable with. Again: Which site is better will strongly depend on your own style of working and personal preferences.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="123" LastEditDate="2012-11-09T15:07:58.003" LastActivityDate="2012-11-09T15:07:58.003" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3613" PostTypeId="1" CreationDate="2012-11-03T18:26:49.553" Score="6" ViewCount="4867" Body="&lt;p&gt;I'm following the derivation from &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0849300967&quot;&gt;Finite Element Method using Matlab 2nd Edition&lt;/a&gt;, pg 311-315, which derives of the local stiffness matrix for planar isotropic linear elasticity as follows:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Force Balance Equations&lt;br&gt;&#xA;$\frac{\partial\sigma_x}{\partial x}+\frac{\tau_{xy}}{\partial y} + f_x=0$&lt;br&gt;&#xA;$\frac{\partial\tau_{xy}}{\partial x}+\frac{\sigma_y}{\partial y} + f_y=0$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using galerkin method, we multiply the first and second equation by test functions $w_1$ and $w_2$, respectively and integrate over the domain $\Omega$.  Integrating by parts, I see that we obtain the weak formulation:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega \begin{bmatrix} \frac{\partial w_1}{\partial x} &amp;amp; 0 &amp;amp; \frac{\partial w_1}{\partial y}\\ 0 &amp;amp; \frac{\partial w_2}{\partial y} &amp;amp; \frac{\partial w_2}{\partial x} \end{bmatrix} \begin{bmatrix} \sigma_x \\ \sigma_y \\ \tau_{xy} \end{bmatrix} = \int_\Omega \begin{bmatrix} w_1 f_x \\ w_2 f_y \end{bmatrix} + \int_{\partial\Gamma} \begin{bmatrix} w_1 \Phi_x \\ w_2 \Phi_y \end{bmatrix}$$&#xA;where $\Gamma$ is the portion of the boundary with the neumann (traction) boundary condition in the x and y directions $\Phi_x$ and $\Phi_y$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's just consider the integral on the left hand side of this equation.  Using the linear isotropic stress strain relationship in two dimensions we can rewrite this equation as  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega M D \epsilon$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M=\begin{bmatrix} \frac{\partial w_1}{\partial x} &amp;amp; 0 &amp;amp; \frac{\partial w_1}{\partial y}\\ 0 &amp;amp; \frac{\partial w_2}{\partial y} &amp;amp; \frac{\partial w_2}{\partial x}\end{bmatrix}$, $D=\frac{E}{1-\nu^2} \begin{bmatrix} 1 &amp;amp; \nu &amp;amp; 0 \\ \nu &amp;amp; 1 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; \frac{1-\nu}{2} \end{bmatrix}$, and $\epsilon = \begin{bmatrix} \frac{\partial u}{\partial x} \\ \frac{\partial v}{\partial y} \\ \frac{\partial u}{\partial y} + \frac{\partial v}{\partial x} \end{bmatrix}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose the domain is tesselated into triangular elements.  Consider a single element $e$ and the three basis functions (unit hat functions) on this element as $H_i(x)$ for i=1,2,3.  We can charaterize the displacement functions on this element as $u(x,y)=\sum_{i=1}^3 u_iH_i$ and $v(x,y)=\sum_{i=1}^3 v_iH_i$, then we can rewrite &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\epsilon=Bd$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $B=\begin{bmatrix} \frac{\partial H_1}{\partial x} &amp;amp; 0 &amp;amp; \frac{\partial H_2}{\partial x} &amp;amp; 0 &amp;amp; \frac{\partial H_3}{\partial x} &amp;amp; 0 \\ 0 &amp;amp; \frac{\partial H_1}{\partial y} &amp;amp; 0 &amp;amp; \frac{\partial H_2}{\partial y} &amp;amp; 0 &amp;amp; \frac{\partial H_3}{\partial y} \\ \frac{\partial H_1}{\partial y} &amp;amp; \frac{\partial H_1}{\partial x} &amp;amp; \frac{\partial H_2}{\partial y} &amp;amp; \frac{\partial H_2}{\partial x} &amp;amp; \frac{\partial H_3}{\partial y} &amp;amp; \frac{\partial H_3}{\partial x}\end{bmatrix}$ and $d=\begin{bmatrix} u_1 \\ v_1 \\ u_2 \\ v_2 \\ u_3 \\ v_3 \end{bmatrix}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, we can write the integral over each element as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_e M D \epsilon= \int_e MDBd$$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author claims that the matrix $M$ becomes $B^T$ when we only consider the test functions equivalent to basis functions with support on element $e$.  That is, when $w_1,w_2 = H_i$ for $i=1,2,3$ we obtain  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_e B^TDBd$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not immediately obvious to me why the matrix $M$ becomes $B^T$ over the element $e$.  How can I arrive at this conclusion just by letting the test functions be the basis functions?  Any help with this would be greatly appreciated! :)&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-11-11T23:33:07.887" Title="Deriving the element stiffness matrix for 2D linear elasticity" Tags="|finite-element|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3614" PostTypeId="2" ParentId="3610" CreationDate="2012-11-03T20:22:46.377" Score="5" Body="&lt;p&gt;Google code is another option as they offer svn/git/hg and almost everyone has a Google ID.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever site you choose I would recommend putting a tarball somewhere right at the top (some sites already do this) as most people are not familiar with revision control and departmental servers across universities often run a 5 year old OS which if one is lucky may have svn installed.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-11-03T20:32:02.847" LastActivityDate="2012-11-03T20:32:02.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3615" PostTypeId="2" ParentId="3551" CreationDate="2012-11-03T21:24:14.667" Score="5" Body="&lt;p&gt;I think, this one &lt;a href=&quot;http://doku.b.tu-harburg.de/volltexte/2003/48/&quot; rel=&quot;noreferrer&quot;&gt;Krylov Subspace Methods in Finite Precision: A Unified Approach, Jens-Peter M. Zemke&lt;/a&gt; is also worth reading.&lt;/p&gt;&#xA;" OwnerUserId="369" LastActivityDate="2012-11-03T21:24:14.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3617" PostTypeId="2" ParentId="3613" CreationDate="2012-11-04T00:10:25.163" Score="5" Body="&lt;p&gt;My answer will address plane strain (instead of plane stress) using a different notation. I use Lamé parameters instead of Young's modulus and Poisson ratio for notational convenience. Consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\DeclareMathOperator{\trace}{tr} -\nabla\cdot \big[2 \mu D u + \lambda \trace(Du) I \big]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D u = \frac 1 2 \Big[ \nabla u + (\nabla u)^T \Big]$ is the strain tensor (frequently denoted by $\varepsilon$, but my notation will be cleaner here; note that the cross-term is scaled different from your $\epsilon$) written as a linear differential operator $D$ (&quot;symmetric gradient&quot;) applied to the displacement vector $u$. Now multiply by test functions $v$, integrate by parts, and look at the interior term&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \nabla v : \Big[2 \mu D u + \lambda \trace(Du) I\Big] &#xA;  = \int_\Omega 2\mu \nabla v:D u + \lambda \trace(Du) \nabla v:I \\&#xA;  = \int_\Omega 2\mu Dv : Du + \lambda \trace(Dv) \trace(Du)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we have used the property that a tensor contraction of nonsymmetric tensor with a symmetric tensor $N : S$ is equivalent to the symmetrized form $\frac 1 2 (N + N^T) : S$ to replace instances of $\nabla v$ with $D v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This latter form is obviously symmetric and what I recommend computing with. To get back to your notation, choose your ordering convention for the three entries in the symmetric tensors $Du$ and $Dv$, discretize the test and trial functions $v_h$ and $u_h$ as well as the differential operator $D_h$, and expand the integral as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 2 \mu v_h^T D_h^T \begin{pmatrix} 1 &amp;amp; 0 &amp;amp; 0 \\ 0 &amp;amp; 1 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 2\end{pmatrix} D_h u_h + \lambda v_h^T D_h^T \begin{pmatrix} 1 &amp;amp; 1 &amp;amp; 0 \\ 1 &amp;amp; 1 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 0 \end{pmatrix} D_h u_h $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Differentiating yields the Jacobian&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D_h^T \begin{pmatrix} 2\mu + \lambda &amp;amp; \lambda &amp;amp; 0 \\ \lambda &amp;amp; 2\mu + \lambda &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 4 \mu \end{pmatrix} D_h $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is the stiffness matrix contribution associated with a quadrature point (single-point quadrature integrates this exactly for $P_1$ triangular elements).&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2012-11-04T00:28:50.873" LastActivityDate="2012-11-04T00:28:50.873" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3618" PostTypeId="1" AcceptedAnswerId="3619" CreationDate="2012-11-04T11:33:49.373" Score="3" ViewCount="89" Body="&lt;p&gt;I understand how to generate a random sequence of binary variables where &lt;code&gt;1&lt;/code&gt; occurs with probability &lt;code&gt;p&lt;/code&gt; and &lt;code&gt;0&lt;/code&gt; occurs with probability &lt;code&gt;1-p&lt;/code&gt;. Namely, at each step you choose &lt;code&gt;1&lt;/code&gt; if some random number is less than &lt;code&gt;p&lt;/code&gt;  and otherwise you choose &lt;code&gt;0&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does this generalize to more than two alternatives?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have three mutually exclusive choices &lt;code&gt;{0,1,2}&lt;/code&gt;. How do I proceed? I found an explanation &lt;a href=&quot;http://cameroncounts.wordpress.com/2010/05/21/a-fair-coin/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. It says to generate the random number from a &lt;em&gt;sequence&lt;/em&gt; of binary tosses that represents the binary approximation of probabilities. However, this only works if the events have sufficiently different probabilities and all have decimal expansions of the same length. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there another way? &lt;/p&gt;&#xA;" OwnerUserId="1867" LastActivityDate="2012-11-04T14:10:48.860" Title="Randomly choose among N alternatives" Tags="|algorithms|monte-carlo|random-number-generation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3619" PostTypeId="2" ParentId="3618" CreationDate="2012-11-04T14:10:48.860" Score="6" Body="&lt;p&gt;If I understood your question correctly: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Subdivide the interval $[0,1]$ in $N$ segments, each having a width proportional to your probabilities $p_i$ (where $\sum_{i=1}^N p_i = 1$). &lt;/li&gt;&#xA;&lt;li&gt;Use a pseudo-random-number-generator using the uniform probability density distribution on $[0,1]$ to generate the number $x$. &lt;/li&gt;&#xA;&lt;li&gt;Determine in which segment $x$ is located; this segment corresponds to the choice. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Example: apples (p=0.5), pears (p=0.2), lemons (p=0.3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your intervals are then $[0,0.5]$, $]0.5,0.7]$ and $]0.7, 1.0]$. If $x=0.76$, you go for lemons; if $x=0.34$, you go for apples. And so on.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-11-04T14:10:48.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3620" PostTypeId="1" AcceptedAnswerId="3621" CreationDate="2012-11-05T02:58:04.630" Score="7" ViewCount="6818" Body="&lt;p&gt;I am a bit confused of the qualitative behavior of the two methods. Consider quadratic case, start by having points $x_i$, where I know the value and points $y_j$, where the values to be found. If I want to do Lagrange interpolation at any point $y_j$, I find three points from $x_i$ with values close by and fit the parabola between those three points with values $f(x_i)$. Once I have the parabola I estimate the value at the unknown point as a value of the parabola at that point. Since the polynomial is of second order I will match all the terms up to 2 in the Taylor expansion and locally will have $O(h^3)$ error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, for the Hermite cubic spline I fit the parabola between every two points  keeping the derivatives equal at all nodes from both sides. Again, I expect the local error to be $O(h^3)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is there a difference between the two in the error? If so, why one would prefer one method to the other? Thanks! &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="123" LastEditDate="2012-11-05T05:13:09.600" LastActivityDate="2012-11-07T08:50:08.917" Title="Piecewise polynomial interpolation: Hermite vs Lagrange" Tags="|pde|interpolation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3621" PostTypeId="2" ParentId="3620" CreationDate="2012-11-05T03:05:45.477" Score="5" Body="&lt;p&gt;The convergence order is the same. My gut feeling is that the magnitude of the error is going to be larger for the Hermite interpolation than for Lagrange interpolation, but the real reason why one would use the former is that you get an interpolation that is $C^1$, i.e., that is continuously differentiable. On the other hand, the Lagrange interpolation is only locally a quadratic polynomial but is not continuously differentiable at the nodes. There are situations where that is not enough and we really does need a continuously differentiable approximation of a function.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-05T03:05:45.477" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3622" PostTypeId="1" AcceptedAnswerId="4647" CreationDate="2012-11-05T04:41:51.833" Score="6" ViewCount="2162" Body="&lt;p&gt;I am using some finite difference algorithm to solve the problem of a parabolic equation. Reading the Leveque's book on finite differences he suggests to test convergence of the method by considering the ratio of differences between solutions, that is $$\frac{u_h-u_{h/2}}{u_{h/2}-u_{h/4}}$$. I understand that it is valid at any given point on the grid but can I estimate in the same way the convergence of the whole solution which is the vector with a dimension $N$, the number of grid points? My goal is to show convergence of the method where the error $e$ is defined as a difference between numerical solution and the function projected on the grid. That is I want to find $p$ s.t. $||e||=Ch^p$. Let me know if the approach is still valid, thus I would find the ratio not the differences but the ration of those norms, i.e. $$\frac{||u_h-u_{h/2}||}{||u_{h/2}-u_{h/4}||}$$&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="123" LastEditDate="2012-11-05T05:17:37.793" LastActivityDate="2012-11-13T05:48:52.730" Title="how to test convergence of the solution vector" Tags="|pde|finite-difference|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3623" PostTypeId="1" AcceptedAnswerId="4644" CreationDate="2012-11-05T07:26:06.137" Score="4" ViewCount="359" Body="&lt;p&gt;I gain some introductory knowledge from the materials I read. I feel Ok with the numerical implementation part of boundary element method when the integral equation has been formulated. &#xA;But the concept of fundamental solution confuses me. They appear in the integral equation during the deduction. I wonder what does fundamental solution stand for? I tried to read some materials on bem, but they just don't help. I'm a beginner in BEM, thus I think I need some intuitive explanation, which I could not find in books.&lt;/p&gt;&#xA;" OwnerUserId="1976" LastActivityDate="2012-11-06T21:19:44.857" Title="What does fundamental solutions stand for in boundary element method?" Tags="|pde|integral-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3624" PostTypeId="2" ParentId="3583" CreationDate="2012-11-05T12:05:48.043" Score="3" Body="&lt;p&gt;I posted the same question on &lt;a href=&quot;http://www.mathworks.it/matlabcentral/&quot; rel=&quot;nofollow&quot;&gt;MATLAB central&lt;/a&gt;, and &lt;a href=&quot;http://www.mathworks.it/matlabcentral/answers/contributors/2963969-bill-greene&quot; rel=&quot;nofollow&quot;&gt;Bill Greene&lt;/a&gt; kindly provided an &lt;a href=&quot;http://www.mathworks.it/matlabcentral/answers/52403-plot-solution-of-pde-toolbox-on-a-line-or-submanifold&quot; rel=&quot;nofollow&quot;&gt;answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I briefly report and extend it so that it can be useful to others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution by Bill:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here is one way to create such a plot. Assume you have the point matrix created by the PDE Toolbox mesher, p, and a solution vector, u. The function below will create a plot of that solution along a line defined by the x and y locations of the two end points. My example is for a solution on a unit square and I want a plot along the line (0,.5) to (1,.5). I want to include 25 points in the plot. As you can see, the real work is being done by the &lt;strong&gt;&lt;code&gt;TriScatteredInterp&lt;/code&gt;&lt;/strong&gt; function from core MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plotAlongLine(p, u, [0,.5], [1,.5], 25);&#xA;&#xA;function plotAlongLine(p, u, xy1, xy2, numpts)&#xA;    x = linspace(xy1(1),xy2(1),numpts);&#xA;    y = linspace(xy1(2),xy2(2),numpts);&#xA;    F = TriScatteredInterp(p(1,:)', p(2,:)', u);&#xA;    uxy = F(x,y);&#xA;    figure; plot(x, uxy);  //REM: x is chosen here as a curvilinear coordinate&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'd like to further remark that the previous function allows one to plot the solution &lt;code&gt;u&lt;/code&gt; or its functions &lt;code&gt;f(u)&lt;/code&gt;, provided &lt;code&gt;u&lt;/code&gt; is defined on mesh nodes (as typically happens for FEM approximated solutions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one needs to plot sections of functions defined on mesh centers (e.g. functions of &lt;code&gt;grad u&lt;/code&gt;), she may use in advance the function &lt;code&gt;pdeprtni&lt;/code&gt; which produces &lt;i&gt;node-valued-functions&lt;/i&gt; out of &lt;i&gt;center-valued-functions&lt;/i&gt;. &lt;/p&gt;&#xA;" OwnerUserId="3248" LastEditorUserId="3248" LastEditDate="2012-11-11T20:59:37.723" LastActivityDate="2012-11-11T20:59:37.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3625" PostTypeId="2" ParentId="3622" CreationDate="2012-11-05T12:19:48.843" Score="-2" Body="&lt;p&gt;Isn't it possible to use the root mean square differences between the $u$ values between iterations. The RMS values computed this way should decrease rapidly during the iterations. You could terminate your iterations if the value drops below a threshold.&lt;/p&gt;&#xA;" OwnerUserId="3229" LastActivityDate="2012-11-05T12:19:48.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3626" PostTypeId="2" ParentId="3620" CreationDate="2012-11-05T13:05:48.597" Score="4" Body="&lt;p&gt;Cubic Hermite interpolation requires different data (function value and derivative at two end points) than quadratic polynomial fit (three function values). Also, cubic Hermite interpolation fits a cubic to 4 dof, hence is order $O(h^4)$, while a quadratic polynomial fits 3 dof only, hence is order $O(h^3)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a cubic polynomial were fitted by Lagrange (or Newton interpolation), the error would be $O(h^4)$ as for a cubic Hermite spline, but still the data used are different, so (unless the function interpolated is exactly cubic) the interpolants will typically be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have function values and derivative values at multiple points, cubic Hermite spline interpolation will typically be more robust than polynomial interpolation to the function values only by Lagrange. (You get higher order if your original function ishighly differentiable, but you could get even higher order if you'd use confluent newton interpolation through the hermite data.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit] Note that cubic Hermite splines do not need the solution of a large linear system. If you have the derivatives they are as easy to apply as Lagrange interpolation. To interpolate between two points $x_i$ and $x_{i+1}$, define $h:=x_{i+1}-x_i$ and get for $t\in[0,1]$&#xA;$$f(x_i+th)=(1-t)f_i+tf_{i+1}+ t(1-t)(u +tv),$$&#xA;where $u$ and $v$ determined by matching the derivatives at both end points. You get one order of accuracy higher than with quadratic Lagrange interpolation.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-11-07T08:50:08.917" LastActivityDate="2012-11-07T08:50:08.917" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3627" PostTypeId="2" ParentId="3306" CreationDate="2012-11-05T14:32:15.267" Score="1" Body="&lt;p&gt;I found the following papers which describes clearly how to do a BCC mesh, and how to handle the hanging nodes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.imr.sandia.gov/papers/imr19/RNwang.pdf&quot; rel=&quot;nofollow&quot;&gt;Adaptive and Quality Tetrahedral Mesh Generation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.berkeley.edu/~jrs/papers/stuffing.pdf&quot; rel=&quot;nofollow&quot;&gt;Isosurface Stuffing&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following presentation, in addition to describing BCC, also describes Face Centred Cubic (FCC): &lt;a href=&quot;http://science.unitn.it/~gabbrielli/mesh.pdf&quot; rel=&quot;nofollow&quot;&gt;Unbound Optimal Tetrahedral Mesh&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2047" LastEditorUserId="2047" LastEditDate="2012-11-05T14:57:27.563" LastActivityDate="2012-11-05T14:57:27.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3628" PostTypeId="2" ParentId="2506" CreationDate="2012-11-05T14:35:29.680" Score="1" Body="&lt;p&gt;I think your question is fine. Depends upon what exactly you're trying to do - seems like it could be 1 of 2 scenarios:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you are interested in illuminating an arbitrary meshed object by a nearby dipole source with electrically small but nonzero length, I think you could safely assume some current distribution over the length of the dipole (e.g. half sinusoid or maybe hat function) and then impose its incident field through quadrature. That is, when integrating E-inc over a surface mesh element, evaluate it by perfoming an auxiliary integral over the length of the driven wire, looking up the current density at that position on the wire, then push that through your analytical Hertzian dipole formula.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you are interested in driving your pure MoM model with an electrically small ideal current source (e.g. modeling a bowtie antenna), you should look into/google &quot;delta gap&quot; feed models. Unfortunately I have never implemented this and don't know what the best original reference is for it. All the antenna codes I've worked were either FEM or FEM-BEM hybrids and were fed by either an FEM lumped element model or an FEM waveguide boundary condition. I would bet that &quot;The Method of Moments In Electromagnetics&quot; by Walton C. Gibson would contain some discussion - at least there's a picture of it on the cover. (on a bowtie, no less)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2012-11-05T14:35:29.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3629" PostTypeId="1" CreationDate="2012-11-05T15:45:34.560" Score="2" ViewCount="118" Body="&lt;p&gt;Assuming there exist two 2D grids on the same domain, one of them is a cartesian one while the other is curvilinear (body fitted regular grid, with curved coordinate lines).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a way to transfer values (interpolate) between these grids. Assuming the body fitted grid is given in the form of coordinates of the cell centers in the cartesian system doing the interpolation from the cartesian to the body fitted grid is relatively easy, as simple bicubic etc. interpolation can be used on the cell center coordinates to find the corresponding cartesian coordinates and get the values from the grid with these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand for the other direction, body fitted to cartesian, this cannot be achieved so easily, as it would essentially require the inversion of the body fitted to cartesian diffeomorphism (mapping).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if someone may have some suggestions for this problem. The numerical inversion of the mapping can be carried out, however I am a little worried about the compound effect of the inaccuracies during this process with all the interpolation involved.&lt;/p&gt;&#xA;" OwnerUserId="3229" LastActivityDate="2012-11-05T15:45:34.560" Title="Resampling of values between body fitted and cartesian grids" Tags="|finite-difference|interpolation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3630" PostTypeId="1" CreationDate="2012-11-05T16:15:25.727" Score="8" ViewCount="2248" Body="&lt;p&gt;Even if quad precision is not directly supported by most CPUs, many Compilers (GNU, Intel) support them. Also some software packages allow to compile with quad precision, e.g. PETSc. But is there really a need for this? I asked some days ago about Krylov subspace method in finite arithmetic, see &lt;a href=&quot;https://scicomp.stackexchange.com/q/3551/1090&quot;&gt;Krylov subspace iterative methods in floating point arithmetic&lt;/a&gt;. I read the mentioned papers, and it seems to me that these methods are well understood in floating point arithmetic. Does iterative solvers benefit from using higher precision if the matrices are well conditioned? I.e., does the iteration counter goes down with higher precision? Are there, may be, some studies on this topic?&#xA;Are there problems in scientific computing, where double precision is not enough to formulate and discretize them? &lt;/p&gt;&#xA;" OwnerUserId="1090" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2021-01-22T14:19:09.167" Title="Need for quad precision in scientific computing?" Tags="|floating-point|precision|computer-arithmetic|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3631" PostTypeId="1" AcceptedAnswerId="4747" CreationDate="2012-11-05T18:31:09.873" Score="2" ViewCount="5852" Body="&lt;p&gt;I am trying to implement the PageRank algorithm described in &lt;a href=&quot;http://pubs.doc.ic.ac.uk/hypergraph-fast-pagerank/hypergraph-fast-pagerank.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (Fig. 1). Here is the breakdown of the steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.louismullie.com/algo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.louismullie.com/algo.png&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;pT&lt;/code&gt; is a probability distribution for the random walk (typically, each element is &lt;code&gt;1/N&lt;/code&gt; where N is the total number of elements)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;P&lt;/code&gt; is the connectivity matrix where &lt;code&gt;p(i,j) = {1 or 0} / # outbound links&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I understand the rationale for the first two steps well. In the following equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/math/5/c/9/5c9a6cc4c1381791153f64891d34b00e.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;step 2 solves for the second term. However, I don't understand the rationale behind steps 3-4. In particular, I don't see how adding &lt;code&gt;omega * pT&lt;/code&gt; in the description of the algorithm amounts to adding &lt;code&gt;(1-d)/N&lt;/code&gt; in the equation shown above. As a result, the output I am getting with the two methods differ. Can anybody enlighten me?&lt;/p&gt;&#xA;" OwnerUserId="3270" LastEditorUserId="-1" LastEditDate="2017-03-10T09:42:37.780" LastActivityDate="2012-11-26T17:35:30.637" Title="Implementing PageRank using the Power Method" Tags="|linear-algebra|algorithms|eigensystem|sparse-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3632" PostTypeId="1" AcceptedAnswerId="3633" CreationDate="2012-11-05T19:25:19.043" Score="11" ViewCount="309" Body="&lt;p&gt;Most math libraries have a number of versions of logarithm functions.&#xA;Most of the time we assume them to be perfect, but actually quite a lot of them just offer a certain number of digits of precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some functions, there are numerically more stable variants. For example Fortran, R, Java and C both have &lt;code&gt;Math.log1p&lt;/code&gt;, for computing &lt;code&gt;log(1.0+x)&lt;/code&gt; (which offers higher precision for small values of x), and the counterpart &lt;code&gt;expm1&lt;/code&gt;. Here the numeric problems arise from a loss in precision - if &lt;code&gt;x&lt;/code&gt; is really small, &lt;code&gt;1.0 + x&lt;/code&gt; loses digits in order to preserve the 1 at the beginning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen such functions for increased precision in a number of situations. This seems to be quite common whenever you are implementing distribution functions (Gamma, Beta, Poisson etc.) with high numeric precision. For example the Gamma function seems to be most of the time used as &lt;code&gt;logGamma&lt;/code&gt;. In general, going to &quot;logspace&quot; can improve precision a lot, and so R seems to have a &quot;logspace&quot; flag on most functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example, in R, there exists &lt;code&gt;log1mexp&lt;/code&gt; for &lt;code&gt;log(1 - exp(p))&lt;/code&gt;:&#xA;&lt;a href=&quot;http://cran.r-project.org/web/packages/Rmpfr/vignettes/log1mexp-note.pdf&quot;&gt;http://cran.r-project.org/web/packages/Rmpfr/vignettes/log1mexp-note.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been playing around with entropy and information theoretic measures. A very common term there is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p * -log(p)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where usually, one would want the base of the logarithm to be 2, not e; but just as often this is only a linear factor, and you can as well use the natural logarithm (so this is not of key importance to me). Anyway, do you know if there is a faster / more direct / more precise way of computing this term? I'm having it all over the place, so it could really pay off to make it a bit more precise and fast (save me the usual &quot;premature optimization&quot; stuff, thanks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see any obvious reason that would cause a loss in precision. So I'm mostly interested if there is any nice trick to speed up this computation. That maybe even saves me treating the &lt;code&gt;p=0&lt;/code&gt; corner case (which sensibly is &lt;code&gt;0&lt;/code&gt;, although &lt;code&gt;log(0)&lt;/code&gt; does not exist) or gives me base 2 for free (although a single multiplication with a constant obviously is not killer expensive). Thanks.&lt;/p&gt;&#xA;" OwnerUserId="3271" LastEditorUserId="348" LastEditDate="2012-11-07T21:46:45.360" LastActivityDate="2012-11-07T21:46:45.360" Title="Are there Improved ways of computing $p \log(p)$?" Tags="|floating-point|accuracy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3633" PostTypeId="2" ParentId="3632" CreationDate="2012-11-05T19:49:44.737" Score="11" Body="&lt;p&gt;$p \log p$ won't suffer from precision loss anywhere in $[0,1]$, and won't suffer from exponent overflow near $0$ either.  Thus, the fast, accurate way is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p ? p * -log(p) : 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-11-05T19:49:44.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3634" PostTypeId="2" ParentId="3630" CreationDate="2012-11-05T20:09:32.797" Score="0" Body="&lt;p&gt;This does not answer your question, but there are people - interested in the reproducibility and reliability of large scale simulations, e.g., climate simulation - who investigate implementational details of floating-point algorithms in double precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Behavioural differences have been observed, e.g., when the scalar products in the CGM are computed not by naive summation, but by more sophisticated algorithm as compensated summation or summation in order of ascending magnitude. This is particular relevant for parallel summation on many cores, because the summation (say, in MPI) is usually black-boxed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A search like [&quot;conjugate gradient&quot; &quot;compensated summation&quot;] in a well-known search engine for scientific papers will give a few (!) publications on that. As far as I remember, the iteration count might vary slightly, and might even increase when improving the numerical accuracy. It is probably less relevant on your local quad-core work station.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-11-05T20:09:32.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3635" PostTypeId="1" CreationDate="2012-11-05T21:24:41.250" Score="2" ViewCount="336" Body="&lt;p&gt;Consider the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start with a $2\times 2$ matrix $W_{ij}$.  I then take this $W$ matrix and make a new tensor, $T$, by doing the following:&#xA;$$&#xA;T_{ijkl}=\sum_{a}W_{ai}W_{aj}W_{ak}W_{al}&#xA;$$&#xA;which in my code I am using the (slick) ``einsum'' tool in numpy:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tt = np.einsum('ai, aj, ak, al', w, w, w, w)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with &lt;code&gt;import numpy as np&lt;/code&gt; for np.  Next I preform the following:&#xA;$$&#xA;M_{XX'y''y'}=\sum_{y}T_{xx'yy'}T_{x''x'''y''y}&#xA;$$&#xA;with $X=x\otimes x''$ and $X'=x'\otimes x'''$.  Now when I do a naive reshape in numpy &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mtensor = np.einsum('ikma, jlan', tt, tt).reshape(4, 4, 2, 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This simply unfolds the array and then recollects in order.  However, I would like to have the power to combine certain indices (states) into a single index (like an outer product but for indices).  That is, take my two, $2$ state indices, and combine them into a $4$ state index that is all the possible combinations of their states.  If someone knows a slick way to do this in numpy I would greatly appreciate some documentation or an answer.  Thanks in advance.  &lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2012-12-05T23:54:12.123" Title="Reshape and Index (State) Products in Numpy" Tags="|python|numpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4635" PostTypeId="2" ParentId="3635" CreationDate="2012-11-05T22:59:53.590" Score="2" Body="&lt;p&gt;Turns out if the indices are input in the alphabetical order as they are up above, the reshape actually is the correct product of states into a single index.  Sorry, for this question, I honestly didn't know it was already like this.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2012-11-05T22:59:53.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4636" PostTypeId="1" CreationDate="2012-11-06T03:53:31.700" Score="0" ViewCount="78" Body="&lt;p&gt;I was given a problem by my professor as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve the&lt;/p&gt;&#xA;&#xA;&lt;p&gt;System &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$pV=S$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$pcT=kT+BS\frac{dG}{dt}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dS}{dt}=\mu(V-\frac{dG}{dt})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dG}{dt}=f(S,T)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $p$, $c$, $B$, $\mu$ are constant and $f(S,T)$ is a function to be reasonably assumed&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where do I even begin... I know I have to use a numerical solution but Id love some helps and tips if possible.&lt;/p&gt;&#xA;" OwnerUserId="3272" LastEditorUserId="119" LastEditDate="2012-11-06T14:46:20.983" LastActivityDate="2012-11-06T14:46:20.983" Title="Thermoplastic Equation solving" Tags="|computational-chemistry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4637" PostTypeId="2" ParentId="3630" CreationDate="2012-11-06T04:10:52.660" Score="1" Body="&lt;p&gt;I think this is actually a duplicate of this question:&#xA;  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3122/higher-precision-floating-point-arithmetic-in-numerical-pde&quot;&gt;Higher precision floating-point arithmetic in numerical PDE&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I stated on that question, quad precision is certainly not widely used in scientific computing. Moreover, I do not believe that quad precision is actually necessary in any significant number of applications.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-11-06T04:10:52.660" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4638" PostTypeId="2" ParentId="3623" CreationDate="2012-11-06T04:17:32.250" Score="0" Body="&lt;p&gt;The fundamental solution is the Green's function of the PDE you are trying to solve.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-06T04:17:32.250" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4639" PostTypeId="2" ParentId="3581" CreationDate="2012-11-06T04:29:03.917" Score="2" Body="&lt;p&gt;You might also consider using &lt;code&gt;mpz_class&lt;/code&gt; in C++ (&lt;a href=&quot;http://gmplib.org/manual/C_002b_002b-Interface-Integers.html&quot; rel=&quot;nofollow&quot;&gt;ref&lt;/a&gt;.) rather than &lt;code&gt;mpz_t&lt;/code&gt;.  It can make arbitrary precision arithmetic straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a random example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;gmp.h&amp;gt;&#xA;#include &amp;lt;gmpxx.h&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;using namespace std;&#xA;&#xA;int main() {&#xA;  mpz_class A[100000];&#xA;  for(int i=0;i&amp;lt;100000;i++) A[i]=142412+i;&#xA;&#xA;  for(int i=0;i&amp;lt;100000;i++) cout &amp;lt;&amp;lt; i &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; A[i] &amp;lt;&amp;lt; endl;&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Compiled with e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g++ [[filename]] -lgmp -lgmpxx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="732" LastActivityDate="2012-11-06T04:29:03.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4640" PostTypeId="1" AcceptedAnswerId="4645" CreationDate="2012-11-06T08:36:56.207" Score="3" ViewCount="2077" Body="&lt;p&gt;When trying to solve a magneto-static boundary-value problem (BVP) ($\nabla \times \mathbf{H} = \mathbf{J}$ and $\nabla \cdot \mathbf{B} = 0$), one can use either the magnetic vector potential $\mathbf{B} = \nabla \times \mathbf{A} $ or the scalar potential $\mathbf{H} = - \nabla \psi$ to fulfil one of the equations. What advantages and disadvantages does each of them have, especially regarding applicability to a wide range of problems (Do they introduce restrictions on the type of problem to be solved?) and possible implementation difficulties (e.g., when trying to impose gauge conditions)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the question is too broad, maybe you could give me a pointer to a paper or book I should read.&lt;/p&gt;&#xA;" OwnerUserId="20" LastEditorUserId="348" LastEditDate="2012-11-07T21:44:37.953" LastActivityDate="2022-04-18T12:30:15.480" Title="Scalar vs. vector potential for magnetostatics" Tags="|pde|finite-element|reference-request|electromagnetism|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4641" PostTypeId="2" ParentId="4640" CreationDate="2012-11-06T13:38:09.357" Score="5" Body="&lt;p&gt;The fundamental problem with going with the potentials is that they are non-physical entities (at least in the classical sense, leaving out quantum mechanical oddities such as the Aharonov-Bohm effect) and as such you have to deal with two problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the gauge? The potentials are not uniquely defined but only up to a gauge condition and you have to decide how you want that condition to look and how to introduce that into your numerical method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What to do with boundary conditions? You will always be given boundary conditions in physical form, such as $n \times E = 0$ but it's not always easy to incorporate such conditions into numerical schemes because they're not natural to the equations you get if you write these equations with potentials whereas they're natural (and, thus, typically easy to incorporate) when writing equations in fields.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-06T13:38:09.357" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4642" PostTypeId="2" ParentId="4640" CreationDate="2012-11-06T14:24:40.250" Score="1" Body="&lt;p&gt;I would go with the magnetic vector potential formulation, discretizing with nedelec's curl conforming elements and using a tree-cotree gauge. There's a description of this approach in&#xA;&lt;a href=&quot;https://ieeexplore.ieee.org/document/43865&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Solution of three-dimensional eddy current problems by integral and differential methods&amp;quot;&lt;/a&gt; by&#xA;R. Albanese and G. Rubinacci (&lt;em&gt;IEEE Transactions on Magnetics&lt;/em&gt;), but it might be a little terse. A more thorough description of the tree-cotree gauge can be found in &lt;a href=&quot;https://ieeexplore.ieee.org/document/1215668&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Hierarchical vector finite elements for analyzing waveguiding structures&amp;quot;&lt;/a&gt; by Seong Cheol Lee (&lt;em&gt;IEEE Transactions on Microwave Theory and Techniques&lt;/em&gt;), and it also has some stuff on higher order basis functions which you could use. Although that latter paper is about the wave equation, the gauging process is the same as what you'd do for magnetostatics.&lt;/p&gt;&#xA;&lt;p&gt;The downside is that the Nédélec elements are less common than the scalar nodal ones that you'd use to model the scalar potential - might have to search a bit for a OSS implementation / roll your own. I thought one of the bigger OSS packages (deal.ii?) had them - I know libmesh does not.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="22284" LastEditDate="2022-04-18T12:30:15.480" LastActivityDate="2022-04-18T12:30:15.480" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4643" PostTypeId="2" ParentId="4636" CreationDate="2012-11-06T14:37:01.560" Score="1" Body="&lt;p&gt;There are a few issues I can see with the formulation of your problem.  First, as a set of ODE's you need initial conditions to completely pose the problem.  Another issue is that to solve it numerically you need to know $f(S,T)$ (or at the very least be able to tell your computer to evaluate it).  Both of these issues can be ignored if the solution is supposed to be analytical and general.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this is a homework and I am not sure what exactly you are being asked to do, I am not going to put any equations in this answer for the time being.  I will tell you that it is possible to reduce your system to a single ODE with all of the unknowns eliminated except for $S$.  The algebra to get there is not too difficult.  Assuming stability the ODE can be solved with time-stepping such as Explicit Euler or an RK method.  Once you have $S$ as a function of time, you can post-process for the rest of your unknowns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you let us know what type of class you received this in and what tools/methods you have learned you may get a bit more information.  &lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-11-06T14:37:01.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4644" PostTypeId="2" ParentId="3623" CreationDate="2012-11-06T18:10:19.910" Score="4" Body="&lt;p&gt;In the framework of mathematical physics, the fundamental solution is the response of an infinite domain to a point source. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. in &lt;a href=&quot;http://en.wikipedia.org/wiki/Poisson%27s_equation#Electrostatics&quot; rel=&quot;nofollow&quot;&gt;electrostatics&lt;/a&gt; the electric potential field $\varphi$ satisfies the Poisson equation&#xA;\begin{equation}&#xA;\nabla^2 \varphi = -\frac{\rho_f}{\epsilon_0}&#xA;\end{equation}&#xA;where $\epsilon_0$ is permittivity and $\rho_f$ is the free charge &lt;em&gt;density&lt;/em&gt; (i.e. charge per unit of volume). What happens if we try to obtain the electric potential of a point charge $Q$ at $y$? $\rho_f(x)$ is everywhere zero, except at point $y$, where it becomes infinite—i.e. a &lt;a href=&quot;http://en.wikipedia.org/wiki/Dirac_delta_function&quot; rel=&quot;nofollow&quot;&gt;Dirac-delta function&lt;/a&gt; in the generalized functions sense.&#xA;The solution to this problem is the well known &lt;a href=&quot;http://en.wikipedia.org/wiki/Electrical_potential#Electric_potential_due_to_a_point_charge&quot; rel=&quot;nofollow&quot;&gt;Coulomb potential&lt;/a&gt;&#xA;\begin{equation}&#xA;\varphi(r) = \frac{1}{4\pi\epsilon_0}\frac{Q}{r}&#xA;\end{equation}&#xA;where $r=\| x-y \|$ is the euclidean distance. And indeed, if we take $Q/\epsilon_0=1$, the Coulomb potential coincides with the &lt;a href=&quot;http://en.wikipedia.org/wiki/Fundamental_solution#Laplace_equation&quot; rel=&quot;nofollow&quot;&gt;fundamental solution&lt;/a&gt; of the Laplace/Poisson equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same holds for elasto-statics: the fundamental solution (Kelvin) is the response of the elastic space to a point force; and so on for other interesting equations.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-11-06T21:19:44.857" LastActivityDate="2012-11-06T21:19:44.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4645" PostTypeId="2" ParentId="4640" CreationDate="2012-11-07T02:51:36.920" Score="3" Body="&lt;p&gt;This is really a huge topic...For simplicity suppose we have linear isotropic media so that$\newcommand{\v}[1]{\boldsymbol{#1}}$ $\v{B} = \mu \v{H}$.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Like Wolfgang mentioned, the formulation you choose first depends on the &lt;strong&gt;boundary conditions&lt;/strong&gt; you would like to impose: either $\v{B}\cdot\v{n} = 0$ or $\v{H}\times\v{n} = 0$. According to Demkowicz's paper &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782597001849&quot; rel=&quot;nofollow&quot;&gt;Modeling of electromagnetic absorption/scattering problems using hp-adaptive finite elements&lt;/a&gt;, $\v{B}\cdot\v{n} = 0$ is used in modeling the region enclosing &lt;em&gt;a perfect conductor&lt;/em&gt;, while $\v{H}\times\v{n} = 0$ is used in modeling &lt;em&gt;a magnetic symmetry wall&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A systematic treatment of magnetostatic BVP using scalar or vector potential can be found in Bossavit's book &lt;a href=&quot;http://books.google.com/books/about/Computational_Electromagnetism.html?id=RQ_9hc8U7MsC&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Computational Electromagetism&lt;/em&gt;&lt;/a&gt;, chapter 2 and chapter 6 respectively. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The choice sometimes depends on the solenoidal nature of the current $\v{J}$, but in a slight different model than yours, in which there is a conductor and the equation is: &#xA;$$\nabla \times \v{H} - \sigma \v{E} = \v{J}\tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(a) If $\v{J}$ is divergence-free, then we get a div-curl system: you could introduce the vector potential $\v{A}$ such that $\mu \v{H} = \nabla \times \v{A}$ and a gauge condition $\nabla \cdot \v{A} = 0$. We could also impose the natural boundary condition weakly in the finite-element variational formulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b) If $\v{J}$ is not divergence-free, we could introduce an electric potential $\phi$: $\v{E} = \nabla \phi$ and apply the divergence operator to both sides of the equation (1). We then have the well-known div-grad electrostatic system:&#xA;$$-\nabla \cdot (\sigma \nabla \phi) = \nabla \cdot \v{J}$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Substituting $\v{H} = -\nabla \psi$ would lead us to a harmonic function $\psi$ that has proper jump conditions on the medium interfaces. Normally this can be done when $\v{B}\cdot\v{n} = 0$ is imposed, and we don't have to impose any gauge condition, merely a compatiblity condition should be taken care of for the pure Neumann problem. This is the relatively easy path.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The gauge condition for the vector potential can really be a pain, especially for people who would like to use $\v{H}^1$-conforming elements or are reluctant to switch to the Nédélec elements which preserves the de Rham cohomology at a discrete level. Imposing the divergence-free condition exactly is too tricky so people use the mixed formulation to impose it weakly; this leads to a solvable saddle-point problem. There are several ways to formulate the mixed variational problem, see &lt;a href=&quot;http://www.asc.tuwien.ac.at/~schoeberl/wiki/lva/notes/maxwell.pdf&quot; rel=&quot;nofollow&quot;&gt;Schoberl's notes&lt;/a&gt; page 13 for one formulation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I will edit more into answer when more relevant issues popping into my mind.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="348" LastEditDate="2012-11-07T21:39:59.363" LastActivityDate="2012-11-07T21:39:59.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4646" PostTypeId="2" ParentId="3613" CreationDate="2012-11-07T08:15:14.827" Score="2" Body="&lt;p&gt;The derivation presented &lt;strike&gt;by the OP&lt;/strike&gt; in the question is, in my opinion, if not wrong, at least highly confusing. In fact after introducing test functions and a weak formulation, it fails to sum the two residuum equations obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we define the force-balance residuum components as&#xA;\begin{align*}&#xA;r_x &amp;amp;= \frac{\partial\sigma_x}{\partial x}+\frac{\tau_{xy}}{\partial y} + f_x, \\&#xA;r_y &amp;amp;= \frac{\partial\tau_{xy}}{\partial x}+\frac{\sigma_y}{\partial y} + f_y,&#xA;\end{align*}&#xA;we note that the weak formulation is written in the question above as &lt;em&gt;two&lt;/em&gt; distinct scalar equations:&#xA;\begin{equation}&#xA;\int_\Omega \begin{bmatrix} w_1\,r_x \\ w_2 \,ry\end{bmatrix}\,d\Omega = &#xA;\begin{bmatrix} 0 \\ 0 \end{bmatrix}&#xA;\end{equation}&#xA;or more simply&#xA;\begin{align*}&#xA;\int_\Omega w_1\,r_x \: d\Omega &amp;amp;= 0,&#xA;&amp;amp;&#xA;\int_\Omega w_2\,r_y \: d\Omega &amp;amp;= 0.&#xA;\end{align*}&#xA;Since we are interested in obtaining a formulation in which an energy norm is defined, the weak formulation should be written instead as a &lt;em&gt;single&lt;/em&gt; scalar equation:&#xA;\begin{equation}&#xA;\int_\Omega w_1\,r_x \: d\Omega&#xA;+&#xA;\int_\Omega w_2\,r_y \: d\Omega = 0&#xA;\end{equation}&#xA;or in vector notation&#xA;\begin{equation}&#xA;\int_\Omega &#xA;\begin{bmatrix}&#xA;w_1 \\ w_2&#xA;\end{bmatrix}^T&#xA;\begin{bmatrix}&#xA;r_x \\ r_y&#xA;\end{bmatrix}&#xA;\: d\Omega&#xA;= 0.&#xA;\end{equation}&#xA;It is easy to show now that after integration by parts we have&#xA;\begin{equation}&#xA;-\int_\Omega&#xA;\begin{bmatrix} \frac{\partial w_1}{\partial x} \\ \frac{\partial w_2}{\partial y} \\ \frac{\partial w_1}{\partial y} + \frac{\partial w_2}{\partial x} \end{bmatrix}^T&#xA;\; D \;&#xA;\begin{bmatrix} \frac{\partial u}{\partial x} \\ \frac{\partial v}{\partial y} \\ \frac{\partial u}{\partial y} + \frac{\partial v}{\partial x} \end{bmatrix}&#xA;\: d\Omega + \dots = 0&#xA;\end{equation}&#xA;From this symmetric expression ($D=D^T$ and positive definite) it is clear why the expression of the element stiffness matrix results in $\int_e B^TDB \,d\Omega$ if both $(w_1, w_2)$ and $(u, v)$ are discretized by the same basis (shape) functions $H_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note also that &#xA;\begin{equation}&#xA;a(w,u) = &#xA;\int_\Omega&#xA;\begin{bmatrix} \frac{\partial w_1}{\partial x} \\ \frac{\partial w_2}{\partial y} \\ \frac{\partial w_1}{\partial y} + \frac{\partial w_2}{\partial x} \end{bmatrix}^T&#xA;\; D \;&#xA;\begin{bmatrix} \frac{\partial u}{\partial x} \\ \frac{\partial v}{\partial y} \\ \frac{\partial u}{\partial y} + \frac{\partial v}{\partial x} \end{bmatrix}&#xA;\: d\Omega &#xA;\end{equation}&#xA;is the inner product in the strain energy norm, so that $\frac12 a(u,u)$ is the strain energy associated with displacement field $u$. &lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2012-11-11T23:33:07.887" LastActivityDate="2012-11-11T23:33:07.887" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4647" PostTypeId="2" ParentId="3622" CreationDate="2012-11-07T14:01:22.277" Score="5" Body="&lt;p&gt;The usual procedure for verifying convergence rates is to take a problem where you know the exact solution analytically. The easiest way to do this is to start from the solution: Pick a function that satisfies the boundary data and then plug it into the differential equation to get the corresponding right-hand side. (This is sometimes called the &lt;em&gt;method of manufactured solution&lt;/em&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A (very simple) example: Consider the one-dimensional parabolic problem&#xA;$$ u_t - u_{xx} = f $$&#xA;with boundary conditions $u(t,0) = u(t,1) = 0$ and initial conditions $u(0,x) = 0$. Then you could choose, say, &#xA;$$ u(t,x) = \sin(\pi x)t $$&#xA;and compute&#xA;$$ f(t,x) = u_t(t,x) - u_{xx}(t,x) = (1+\pi^2t)\sin(\pi x)$$&#xA;and use this as the right hand side in your finite difference code. This is a bit more complicated if you need to consider, e.g., jumping coefficients, but it can be done by choosing the coefficients together with the right-hand side. &lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2012-11-07T14:43:53.387" LastActivityDate="2012-11-07T14:43:53.387" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4648" PostTypeId="2" ParentId="3622" CreationDate="2012-11-07T15:21:57.097" Score="1" Body="&lt;p&gt;The book by Patrick Roach is probably a good reference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Roache, Patrick J. &quot;Verification and validation in computational science and engineering&quot;, Hermosa publishers, 1998.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2012-11-07T15:21:57.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4649" PostTypeId="2" ParentId="3631" CreationDate="2012-11-07T16:59:09.987" Score="2" Body="&lt;p&gt;The PageRank method is basically the &lt;a href=&quot;http://en.wikipedia.org/wiki/Power_iteration&quot; rel=&quot;nofollow&quot;&gt;Power iteration&lt;/a&gt; for finding the eigenvector corresponding to the largest eigenvalue of the transition matrix.  The algorithm you quote is coming directly from equations (4) and (5) of the paper you reference, and this is just a way of implementing the power iteration for a matrix with a particular structure.  I have no idea where your equation involving $ PR(A) &#xA; $ is coming from or what it means or how it relates to the rest of your question.&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2012-11-07T16:59:09.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4651" PostTypeId="1" AcceptedAnswerId="4654" CreationDate="2012-11-07T23:35:29.137" Score="2" ViewCount="123" Body="&lt;p&gt;I would like to know how to unroll loop-carried dependency inside the cholesky algorithm. What are the techniques that I should know to accomplish this work?&#xA;I need to know it because I want to parallelize it. I was reading about from the following link: &lt;a href=&quot;http://cnx.org/content/m32782/latest/&quot; rel=&quot;nofollow&quot;&gt;http://cnx.org/content/m32782/latest/&lt;/a&gt;, Should I to pay attention to that page? Is the last correct?&lt;/p&gt;&#xA;" OwnerUserId="2057" LastActivityDate="2012-11-08T08:29:00.717" Title="Cholesky Algorithm loop-Carried" Tags="|linear-algebra|parallel-computing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4652" PostTypeId="2" ParentId="3591" CreationDate="2012-11-08T05:12:18.267" Score="2" Body="&lt;p&gt;If your flow-rate is constant, there is no source/sink terms and your flow is incompressible, then it should not matter at which plane you compute your flow-rate, it will be the same in any plane cutting your pipe (I don't even think it needs to be normal to your mean flow). Any discrepancy between different plane should be attributed to numerical errors. I assume you are performing a steady-state computation right?&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2012-11-08T05:12:18.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4653" PostTypeId="1" AcceptedAnswerId="4657" CreationDate="2012-11-08T07:45:39.683" Score="4" ViewCount="110" Body="&lt;p&gt;Could someone explain me (roughly) the interest of Bounded Variation (BV) Spaces for PDEs ?&#xA;Is there any numerical application of those space to real problems or is it just a theoretic way to formulate specific problems ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2012-11-10T06:38:14.733" Title="Bounded Variation Spaces" Tags="|pde|numerics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4654" PostTypeId="2" ParentId="4651" CreationDate="2012-11-08T08:29:00.717" Score="2" Body="&lt;p&gt;The link is correct. To parallelize an algorithm, you have to handle its dependencies. However, this is not a trivial task, even if you know this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are plenty of articles to be found on how to parallelize Cholesky (just google for &quot;parallel Cholesky&quot;). If you do not have experience in parallelizing algorithms, then I would highly recommend to simply implement one of those algorithms, or to use a library that can do so. When you implement it yourself, you will for sure not come up with a good solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of articles:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://runtime.bordeaux.inria.fr/shenry/papers/HS_Cholesky.pdf&quot; rel=&quot;nofollow&quot;&gt;http://runtime.bordeaux.inria.fr/shenry/papers/HS_Cholesky.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://gpgpu.org/2012/06/13/parallel-incomplete-lu-and-cholesky-factorization&quot; rel=&quot;nofollow&quot;&gt;http://gpgpu.org/2012/06/13/parallel-incomplete-lu-and-cholesky-factorization&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1568" LastActivityDate="2012-11-08T08:29:00.717" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4655" PostTypeId="2" ParentId="3610" CreationDate="2012-11-08T11:10:26.563" Score="4" Body="&lt;p&gt;Yet one other option is to use &lt;a href=&quot;https://bitbucket.org/&quot; rel=&quot;nofollow&quot;&gt;Bitbucket&lt;/a&gt;. It is very well compatible with &lt;a href=&quot;http://mercurial.selenic.com/&quot; rel=&quot;nofollow&quot;&gt;Mercurial&lt;/a&gt; (Hg). My personal experience with Hg is that it is a simple yet powerful version control system. I believe it is much more suitable for personal software development. For large groups, I would also suggest git and consequently github. Still, I'm pretty satisfied with bitbucket and its features. I started the project with a private repo (where you can have infinitely many, including a project for your thesis and scientific papers, as well) and when the project was mature enough - I converted it to a public release.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are not familiar with version control, especially in Mercurial, I suggest &lt;a href=&quot;http://hginit.com/01.html&quot; rel=&quot;nofollow&quot;&gt;HgInit&lt;/a&gt;. A colleague of mine showed me this page when I first wanted to start to use revision control. I found it useful.&lt;/p&gt;&#xA;" OwnerUserId="1499" LastActivityDate="2012-11-08T11:10:26.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4656" PostTypeId="2" ParentId="3622" CreationDate="2012-11-09T15:23:09.300" Score="6" Body="&lt;p&gt;Yes, what you're proposing is certainly a valid approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will attempt to also answer what I think you have asked in the comments.  If this is the answer you're looking for, then we should clarify the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error is of the form&#xA;$$\|e_h\| = Ch^p + {\mathcal O}(h^{p+1})$$&#xA;where, importantly, $C$ is independent of $h$.  Note that if we take the log of the above, we get&#xA;$$\log(\|e_h\|) \approx \log(C) + p\log(h),$$&#xA;so if you plot error versus $h$ on a log-log plot, you should see a straight line with slope $p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can compute some values of $e_h$ for different $h$ using your method.  Given two pairs $(h_1, e_{h_1})$ and $(h_2,e_{h_2})$, you can approximate $p$ as follows.  Note that&#xA;$$\frac{\|e_{h_1}\|}{\|e_{h_2}\|} \approx \left(\frac{h_1}{h_2}\right)^p$$&#xA;Thus&#xA;$$p \approx \log\left(\frac{\|e_{h_1}\|}{\|e_{h_2}\|}\right)/\log\left(\frac{h_1}{h_2}\right).$$&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2012-11-13T05:48:52.730" LastActivityDate="2012-11-13T05:48:52.730" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4657" PostTypeId="2" ParentId="4653" CreationDate="2012-11-10T06:38:14.733" Score="8" Body="&lt;p&gt;$BV$ spaces are exactly what you want in many inverse problem. The point is that in many inverse problems you try to determine a function $q(x)$ that describes the internal properties of a body -- say, the water content (MRI), the density (X-ray) or elastic coefficients (ultrasound). A good approximation is that this function $q(x)$ is constant (or, at least, varying little) within each part of the body (e.g. body vs. soft tissue, one organ vs. the next) but that it is discontinuous between different parts. In other words, $q(x)$ is discontinuous but the area of the surfaces where it can be discontinuous is bounded. Functions that satisfy such properties are exactly in the space $BV$ but not in any of the usual Sobolev spaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A separate application for $BV$ is in plasticity with weakening -- think of materials like rocks that, when they break due to excessive strain, become dramatically weaker and keep deforming in the very same places. Tectonic fault lines are examples of this. In such cases, the displacement is discontinuous along these lines but continuous in the rest of the domain. Again, $BV$ is the right space to seek solutions in in such cases.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-10T06:38:14.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4658" PostTypeId="1" CreationDate="2012-11-10T08:51:25.283" Score="7" ViewCount="463" Body="&lt;p&gt;I have a nonlinear system of algebraic equations of special kind:&#xA;$$&#xA;   \begin{array}{rcl}  &#xA;   x_{i}+y_{i}+z_{0,1}+c_{i,1}z_{1,1} &amp;amp; = &amp;amp; d_{i,1}, \\&#xA;   x_{i}^2 + y_{i}^2 + z_{0,2} + c_{i,1} z_{1,2} + c_{i,2} z_{2,2} &amp;amp; = &amp;amp; d_{i,2} \\&#xA;   \ldots &amp;amp; = &amp;amp; \ldots \\&#xA;   x_{i}^{m} + y_{i}^{m} +z_{0,m}+c_{i,1} z_{1,m} + \ldots + c_{i,m} z_{m,m} &amp;amp; = &amp;amp; d_{i,m}&#xA;   \end{array}&#xA;$$&#xA;where $i = 1,\ldots.k$ and number of equations is not less than number of unknowns.  I have to find $(x_{i},y_{i},z_{i,j})$; $c_{i,j}$ and $d_{i,j}$ are known. Is there some numerical method that take in account specificity of this system? If not, please tell me which numerical method will work better with this system?&lt;/p&gt;&#xA;" OwnerUserId="1180" LastEditorUserId="276" LastEditDate="2012-11-10T20:38:35.663" LastActivityDate="2019-07-21T19:10:29.140" Title="Numerical method for nonlinear system of algebraic equations of special type" Tags="|linear-algebra|nonlinear-equations|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4660" PostTypeId="2" ParentId="1617" CreationDate="2012-11-10T20:48:25.043" Score="4" Body="&lt;p&gt;All the requirements you mentioned in your question are fulfilled by the Swift parallel scripting system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've spent a year with Swift group as a postdoctoral researcher (PhD in scientific workflows). We've been helping scientists and researchers from different domains address their computational needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Swift is an open source framework for running workflows in parallel manner. It is called parallel scripting mainly to highlight the fact that it provides a scripting interface to creating workflows as opposed to the GUI box-arrow interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can personally help you getting started and running your application with Swift. To know more about Swift, please take a look &lt;a href=&quot;http://www.ci.uchicago.edu/swift/main&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2091" LastEditorUserId="2091" LastEditDate="2012-11-10T22:57:56.053" LastActivityDate="2012-11-10T22:57:56.053" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4662" PostTypeId="1" AcceptedAnswerId="4667" CreationDate="2012-11-11T18:34:04.807" Score="3" ViewCount="125" Body="&lt;p&gt;In my research I have come across the following integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;f = \int_0^{2\pi} \text{d}\theta \exp\left\{\frac{3}{2}(h_1 \cos^2\theta + h_2 \sin^2\theta + 2 h_{12} \sin\theta \cos\theta)\right\}&#xA;\end{equation}&#xA;where $h_1$, $h_2$ and $h_{12}$ are constants of order unity, but which can be either positive or negative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually, I will have to evaluate this integral numerically a large number of times. Hence, it would be quite interesting if it could be written in terms of special functions (if $h_2 = h_{12} = 0$, for instance, it can be written in terms of Bessel Functions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I was unable to figure out how. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, do you have any tips on how to implement it numerically efficiently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much in advance.&#xA;Best regards, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gabriel &lt;/p&gt;&#xA;" OwnerUserId="1751" LastActivityDate="2012-11-12T00:23:24.053" Title="Representing an integral as a special function" Tags="|special-functions|integral-equations|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4663" PostTypeId="1" CreationDate="2012-11-11T18:41:57.617" Score="2" ViewCount="328" Body="&lt;p&gt;Is there a way to partition a mesh consisting of 20 noded hexahedral elements for parallel processing? I used METIS for partitioning mesh with 8 noded hexahedron elmements, which works fine but i don't know how to extend this to 20 noded element mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Fortran code is as follows, which works fine for 4 node quad and 8 node hex meshes but when i used 20 noded element it gives error &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Segmentation fault&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subroutine split_domain(cells,nodes,topol,nprocs, epart)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;implicit none &lt;/p&gt;&#xA;&#xA;&lt;p&gt;!========================================================&#xA;integer,intent(in)::cells,nodes,topol(cells,20),nprocs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;integer,intent(out)::epart(cells)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;integer::ne, nn&lt;/p&gt;&#xA;&#xA;&lt;p&gt;integer::eptr(cells+1),eind(20*cells),npart(nodes) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;integer::ncommon,nparts,objval   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;integer, pointer ::vwgt,vsize,tpwgts,options     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;integer::j &lt;/p&gt;&#xA;&#xA;&lt;p&gt;!========================================================&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ne=cells; nn=nodes; eptr(1)=1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;do j=1,ne    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eind(20*j-19:20*j)=topol(j,:)&#xA;&#xA;eptr(j+1)=eptr(j)+20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;enddo&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ncommon=1; nparts=nprocs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;call METIS_PartMeshDual (ne,nn,eptr,eind,vwgt,vsize,ncommon,nparts,tpwgts,options,objval,epart,npart)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;end subroutine split_domain&lt;/p&gt;&#xA;" OwnerUserId="3295" LastEditorUserId="3295" LastEditDate="2012-11-11T20:11:14.223" LastActivityDate="2012-12-11T20:30:54.493" Title="Partitioning of mesh with 20 noded hexahedral elements" Tags="|finite-element|parallel-computing|" AnswerCount="1" CommentCount="1" ClosedDate="2012-12-14T04:00:33.390" ContentLicense="CC BY-SA 3.0" />
  <row Id="4664" PostTypeId="2" ParentId="4663" CreationDate="2012-11-11T18:53:24.180" Score="0" Body="&lt;p&gt;METIS supports partitioning of meshes which elements consist of arbitrary number of nodes. The number of nodes per element may also vary within the mesh. You have just to create the corresponding arrays &lt;code&gt;eptr&lt;/code&gt; and &lt;code&gt;eind&lt;/code&gt; in the appropriate way. Check the manual, sections 5.6 and 4.1.2 for a detailed description and some small examples.&lt;/p&gt;&#xA;" OwnerUserId="1090" LastEditorUserId="1090" LastEditDate="2012-11-11T19:12:31.847" LastActivityDate="2012-11-11T19:12:31.847" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4665" PostTypeId="2" ParentId="4662" CreationDate="2012-11-11T23:14:53.040" Score="1" Body="&lt;p&gt;Do you get anywhere by noticing that $h_1\cos^2\theta + h_2\sin^2\theta +2h_{12}\sin\theta\cos\theta$ can be written in the form $(h_1\cos(\theta)+a\sin\theta)(\cos(\theta)+b\sin\theta))$? A second step would be to realize that $h_1 \cos(\theta)+a\sin\theta = \cos(\theta-\psi)$ for some $\psi$. This helps you because the integral is over $\theta=0\ldots 2\pi$, so you can shift the entire integration interval by $\psi$ and hopefully come out with something that has a better structured exponent.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-11T23:14:53.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4666" PostTypeId="1" CreationDate="2012-11-11T23:26:33.157" Score="8" ViewCount="3671" Body="&lt;p&gt;Strassen’s Matrix Multiplication algorithm has theoretical performance of $ O( n^{log_2 7}) $. Regular MM algorithm has performance of $ O( n^{3}) $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At certain sizes of matrices (lets call it $n*n$), Regular MM works better. However, once size $n*n$ is breached, Strassen’s MM works better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: at what size of the matrix, Strassen’s MM algorithm works better ?&lt;br&gt;&#xA;Thanks !&lt;/p&gt;&#xA;" OwnerUserId="1575" LastActivityDate="2012-11-12T15:40:28.447" Title="What is the criteria for switching between Strassen's and Regular matrix multiplication Algorithms" Tags="|algorithms|matrix|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4667" PostTypeId="2" ParentId="4662" CreationDate="2012-11-12T00:23:24.053" Score="5" Body="&lt;p&gt;Using the half angle formulas you can convert the exponent into the form&#xA;$$a + b \cos 2\theta + c \sin 2\theta$$&#xA;which then integrates nicely into Bessel functions.  Mathematica gives&#xA;$$\int_0^{2\pi}e^{a + b \cos 2\theta + c \sin 2\theta} d\theta = e^a \pi I_0\left(\sqrt{b^2 + c^2}\right)$$&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2012-11-12T00:23:24.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4668" PostTypeId="2" ParentId="4666" CreationDate="2012-11-12T01:56:54.750" Score="2" Body="&lt;p&gt;The crossover point is highly dependent on the particular machine and architecture. For modern linear algebra libraries, the largest impact on performance is the cache hierarchy within CPUs, and so therefore it depends on how you structure and order the operations for either method. This aside, there are also accuracy issues, as I recall, Strassen's algorithm produces larger errors.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-11-12T01:56:54.750" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4669" PostTypeId="2" ParentId="4666" CreationDate="2012-11-12T03:33:37.593" Score="4" Body="&lt;p&gt;For a relatively recent (2010) benchmarking study of Strassen vs. conventional matrix multiplication, see:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ics.uci.edu/~fastmm/FMM-Reference/paoloA.ishp-vi.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.ics.uci.edu/~fastmm/FMM-Reference/paoloA.ishp-vi.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic conclusions are that the the point at which it becomes worthwhile to use Strassen's algorithm depends on the machine and that in no case were the authors able to get significant (the best improvement was only 15%) performance improvements by using Strassen's algorithm for matrices of reasonable size.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2012-11-12T03:33:37.593" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4670" PostTypeId="1" CreationDate="2012-11-12T05:07:09.530" Score="3" ViewCount="5652" Body="&lt;p&gt;I'm beginning to learn Comsol Multiphysics as part of a modelling course in my masters degree. Currently, we are modelling a circuit with a spark gap, which presents some problems for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using a 0D domain with some global equations defined, and I'd like to model the spark gap as a resistor with a very high resistance (10^8 Ohm), except when the voltage across the resistor exceeds the breakdown voltage U_break - when that happens, I'd like to use the value 1 kOhm for the resistance of the spark gap for 1 ms, and then go back to the high value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could just run the simulation without switching the resistance, find where the voltage exceeds U_break the first time and next time I run the simulation simply enter the switch points manually - but I'd like to do a parametric sweep over a variable that changes how long it will take to build up the voltage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab, you can have &quot;event&quot; functions, which cause something to happen whenever a condition is fulfilled (usually just terminating the integration, but you could do other stuff as well). Is it possible to do anything similar in Comsol Multiphysics? If so, how?&lt;/p&gt;&#xA;" OwnerUserId="3297" LastActivityDate="2012-11-15T14:09:30.217" Title="In Comsol Multiphysics, is there a way to change parameter values when something happens in the solution?" Tags="|comsol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4671" PostTypeId="2" ParentId="4666" CreationDate="2012-11-12T15:40:28.447" Score="6" Body="&lt;p&gt;Others have commented on the machine dependence of it. What I'd like to point out is that Strassen's algorithm only concerns dense matrices and that the matrices need to be fairly large for it to be worthwhile. However, this is a case one doesn't encounter all that often in applications:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If your matrix really is dense and large, then you probably have all sorts of other problems as well. For example, you are probably not just multiplying it but you want to compute principle components, eigenvalues, inverses, etc. Accelerating the matrix vector product might help you but there are multiple other layers of algorithms that will prove to be a bottleneck to the size of the problems you can solve that you will hit before matrix products become important.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Matrix-matrix products aren't that common to begin with. Most of the time we consider a matrix a linear operator on vector spaces and so when we see the product of two matrices we think of it is one operator applied to a vector and then another operator applied to the result. Reflecting this in software is almost always the more efficient way compared to actually multiplying out the operators.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Very large problems are often formulated in terms of sparse matrices for which Strassen's algorithm does not apply. This is certainly true for everything that comes from partial differential equations (including boundary integral equations) but also for things like most graph based algorithms (e.g. social graphs, which are also sparse).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-12T15:40:28.447" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4672" PostTypeId="1" AcceptedAnswerId="4673" CreationDate="2012-11-12T17:01:22.747" Score="4" ViewCount="177" Body="&lt;p&gt;I need to simulate the movement of a large number of agents. The processes that govern the agents' movement are complex and so the entire process requires parallelisation. The output from this simulation needs to be visualised in 3D. As I will be running this simulation across many different nodes (MPI or even MPI+GPGPU) I do not want the simulation to run in real time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a rendering library that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can render offline&lt;/li&gt;&#xA;&lt;li&gt;Is easy to use&lt;/li&gt;&#xA;&lt;li&gt;Can handle soft body deformation and collision&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(i'm not look for awesome AAA video game quality graphics, in addition the movement code will take up enough CPU time so I don't want to further slow the application down by adding heavy weight rendering code) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem MUST have been solved before - there's plenty of visualisation in HPC&lt;/p&gt;&#xA;" OwnerUserId="1017" LastActivityDate="2012-11-13T04:52:56.007" Title="Offline (visualisation) rendering in scientific computing" Tags="|hpc|visualization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4673" PostTypeId="2" ParentId="4672" CreationDate="2012-11-12T18:16:55.527" Score="4" Body="&lt;p&gt;Check for paraview (&lt;a href=&quot;http://www.paraview.org&quot; rel=&quot;nofollow&quot;&gt;www.paraview.org&lt;/a&gt;), which is a widly used open source visualization software package for HPC. It should fit all your requirements.&lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2012-11-12T18:16:55.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4674" PostTypeId="2" ParentId="4672" CreationDate="2012-11-12T19:37:36.297" Score="2" Body="&lt;p&gt;Perhaps &lt;a href=&quot;https://wci.llnl.gov/codes/visit/home.html&quot; rel=&quot;nofollow&quot;&gt;VisIt&lt;/a&gt; is a good fit to your requirements.&lt;/p&gt;&#xA;" OwnerUserId="2091" LastActivityDate="2012-11-12T19:37:36.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4675" PostTypeId="2" ParentId="3610" CreationDate="2012-11-12T23:07:20.830" Score="6" Body="&lt;p&gt;Go for Github.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The publication model of Github is the future. Github's model of forking, pull requests, and merging is very close to the model of scientific publishing. Many scientific communities are using Github to host data and code of their research projects. There are open access journals that use Github as their sole means of submission and publication. Plus, you will get a better Google ranking and thus more prospective users. Sourceforge is slow and mainly hosts dead projects. Many actively maintained projects have moved from Sourceforge to Github in recent years.&lt;/p&gt;&#xA;" OwnerUserId="3300" LastActivityDate="2012-11-12T23:07:20.830" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4676" PostTypeId="1" AcceptedAnswerId="4679" CreationDate="2012-11-12T23:52:09.050" Score="2" ViewCount="487" Body="&lt;p&gt;I am solving $u_t=u_{xx}+u_{yy}$ with a solver which is locally $O(dx^2+dy^2+dt^2)$. I use the following norm for the error between the numerical vector solution and the analytical solution $$||e||_2=\sqrt{dxdy\sum_{i,j}^{N_1,N_2}e_{ij}}=\sqrt{dxdyN_1,N_2O(dt^2+dx^2+dy^2)}$$ Now I want to test the rate of convergence, that is I increase the number of points in all dimensions by a factor of $2$ and thus new mesh sizes become $dx/2,dy/2,dt/2$ then the new norm of the error is $$||\bar{e}||:=\sqrt{dxdy/4*2N_12N_2O(dx^2/4+dy^2/4+dt^2/4)}=1/2||e||$$. So for the second order method discrete $L^2$ norm decreases by $2$, not by $4$? Even though at each point the local error will decrease by $4$. Did I do something wrong this is expected? I thought the total $L^2$ discrete norm is like averaged norm at each point, so the average should decrease by $4$ as well? &lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2012-11-13T05:45:45.607" Title="rate of convergence for the second order accurate method on two dimensional grid" Tags="|pde|finite-element|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4677" PostTypeId="2" ParentId="4672" CreationDate="2012-11-13T04:52:56.007" Score="0" Body="&lt;p&gt;Both paraview and visit support data formats that allow you to write whatever data you have on each processor of a parallel computation into one file per processor and then later visualize all the files at the same time. For paraview, the collection of files that make up a single time step is the .pvtu file; for visit, the &quot;Getting Data into Visit&quot; manual describes the format of a .visit file. In either case, every processor would produce a .vtk or .vtu file and you would later (offline) visualize the whole bunch.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-13T04:52:56.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4679" PostTypeId="2" ParentId="4676" CreationDate="2012-11-13T05:45:45.607" Score="2" Body="&lt;p&gt;Yes, you've done something wrong.  The appropriate 2-norm of a vector that approximates a grid function is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\|e\|_2 = \sqrt{dx dx \sum_{i,j} e_{ij}^2}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are incorrectly using $e_{ij}$ instead of $e_{ij}^2$.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-11-13T05:45:45.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4680" PostTypeId="1" CreationDate="2012-11-13T11:13:35.263" Score="4" ViewCount="263" Body="&lt;p&gt;Does some of you know a mesh partitioner that assures non empty subdomains? For METIS, ParMETIS and Zoltan this is not the case. &lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2020-03-18T02:04:03.013" Title="Mesh partitioner that assures non empty subdomain?" Tags="|partitioning|mesh|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4682" PostTypeId="1" AcceptedAnswerId="4683" CreationDate="2012-11-13T18:37:03.653" Score="5" ViewCount="2036" Body="&lt;p&gt;I have the following problem in Finite Element Method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ -(\alpha u')' + \beta u' + \gamma u = f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $ \Omega = (0, 1)$, $ u(0) = 0 $ and $ u'(1) = 3 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to be able to write the &lt;strong&gt;weak formulation&lt;/strong&gt; of the problem do I need a lifting function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e., do I need to define something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \tilde u = u - R$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm stuck after integrating by parts, I got something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ - \Big [(\alpha u' v) _0^1 - \int_0^1 \alpha u' v' \Big ] + \beta \int_0^1 u' v + \gamma \int_0^1 u v = \int_0^1 f v  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \forall v \in V = H_{\Gamma_D}^1 (0,1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get rid of the first term $$ (\alpha u' v) _0^1 $$ and have the bilinear form $a(u,v)$, when $$v(0) = 0$$ the lower limit dissapears, but for the upper limit I get $$ - \alpha u' (1) v(1)$$ and it does not disappear, how to proceed?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2012-11-13T18:52:05.547" LastActivityDate="2012-11-13T20:12:57.013" Title="Mixed boundary conditions Finite Element Method" Tags="|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4683" PostTypeId="2" ParentId="4682" CreationDate="2012-11-13T20:12:57.013" Score="5" Body="&lt;p&gt;For the second term $\alpha u'(1)v(1)$, you can insert the Neumann boundary condition $u'(1) = 3$ to obtain $3\alpha v(1)$. Since this term can be evaluated (for a given test function $v$), it becomes part of the right hand side: The weak formulation is to find $u\in H^1_{\Gamma_D}(0,1)$ such that&#xA;$$&#xA;\int_0^1 u'(x)v'(x) dx + \beta \int_0^1 u'(x)v(x) dx + \int_0^1 u(x)v(x) dx = \int_0^1 f(x) v(x) dx + 3\alpha v(1)&#xA;$$&#xA;for all $v\in H^1_{\Gamma_D}(0,1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in two and more dimensions, the term arising from integration by parts is&#xA;$$&#xA;\alpha(x)\nabla u(x)\cdot \nu(x) v(x), \qquad x\in\Gamma_N,&#xA;$$&#xA;where $\alpha$ can be a matrix and $\nu(x)$ is the unit outward normal at $x$. To handle that term, you need to specify &#xA;$$&#xA;\alpha\nabla u\cdot \nu = g&#xA;$$&#xA;on $\Gamma_N$, and not just the normal derivative $\nabla u\cdot \nu$.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2012-11-13T20:12:57.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4684" PostTypeId="1" AcceptedAnswerId="4685" CreationDate="2012-11-14T04:07:44.687" Score="6" ViewCount="730" Body="&lt;p&gt;I would like to implement a Numerical Optimizer in my favorite language Go.  It shall find solution(s) to this problem: maximize a function &lt;code&gt;f(x)&lt;/code&gt; where &lt;code&gt;f&lt;/code&gt; is non-linear and &lt;code&gt;x&lt;/code&gt; is a real vector of dimension 10 or 20.  &lt;code&gt;f&lt;/code&gt; is real-valued.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best method in terms of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;simplicity to implement&lt;/li&gt;&#xA;&lt;li&gt;opportunities to paralellize&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The algorithm is supposed to run on a x64 CPU with multiple cores.  Maybe one needs to look at different cases, so it would be okay for me to implement 2 or 3 algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interesting cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;f&lt;/code&gt; is rational &lt;em&gt;or&lt;/em&gt; &lt;code&gt;f&lt;/code&gt; is a composition of rational functions, &lt;code&gt;exp&lt;/code&gt; and maybe &lt;code&gt;log&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;f&lt;/code&gt; has multiple maximums&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is not intended as an open question.  So I am not looking for the answer that lists as many algorithms as possible.  Or an endless discussion of what algorithm might be the nicest.  Choosing an algorithm can be somehow subjective, so I want to emphasize on a decent solution, rather than a perfect solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If have heard that simulated annealing is considered state-of-the-art when it comes to multiple maximums and a fixed domain.  Although this sounds to me like: if I do not fix the domain, although I &lt;em&gt;could&lt;/em&gt; fix it a priori, simulated annealing might be inferior to some other algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; suboptimal solutions are fine, however I want to make sure the algorithm doesn't concentrate to easily on a single maximum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the function: I want to do maximum likelihood fits on functions that estimate time series.  Something like &lt;code&gt;y_{n+1} = a_1 * y_n + a_2 + y_{n-1} + ... + b_1 * epsilon_n + ... + epsilon_n&lt;/code&gt; in a simple case.  (&lt;code&gt;epsilon&lt;/code&gt; is i.i.d. noise, &lt;code&gt;y&lt;/code&gt; some time series, &lt;code&gt;a_i&lt;/code&gt; and &lt;code&gt;b_i&lt;/code&gt; real parameters .)  Currently I'm doing this by assuming &lt;code&gt;epsilon&lt;/code&gt; to be normal distributed but I want to change to the Cauchy-distribution.  Moreover I want to add some non-linear terms and extend this from y real value to y being a real vector.  Unfortunately I don't know yet exactly what I want, so the optimizer should not be too scoped.&lt;/p&gt;&#xA;" OwnerUserId="3307" LastEditorUserId="123" LastEditDate="2012-11-14T14:16:31.000" LastActivityDate="2012-11-14T16:48:32.823" Title="Implementing a numerical method for unconstrained optimization in Go" Tags="|optimization|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4685" PostTypeId="2" ParentId="4684" CreationDate="2012-11-14T09:18:22.040" Score="6" Body="&lt;p&gt;If you have explicit gradients, BFGS with More-Thuente line searches is the method of choice. It is a serial algorithm, fairly easy to implement, and quite robust. (A Matlab implementation of a variant of BFGS/More-Thuente is described in &lt;a href=&quot;http://prod.sandia.gov/techlib/access-control.cgi/2010/101422.pdf&quot;&gt;http://prod.sandia.gov/techlib/access-control.cgi/2010/101422.pdf&lt;/a&gt;. The code is publicly available and can guide your reimplementation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parallelization needs some modifications, that you can easily invent yourselves after having understood how the algorithm works. You need multiple start options (with far away starting points) to find multiple mimina with wide domain of attraction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Branch and bound using convex relaxations would give you assurance to find the global minimum, but it is far from easy to make efficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have gradients, I propose that you reimplement my SNOBFIT package, being a very robust robust package for derivative-free optimization of expensive functions, with excellent parallelization capability. See&#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/software/snobfit/&quot;&gt;http://www.mat.univie.ac.at/~neum/software/snobfit/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-11-14T16:48:32.823" LastActivityDate="2012-11-14T16:48:32.823" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4686" PostTypeId="2" ParentId="4684" CreationDate="2012-11-14T09:22:04.520" Score="3" Body="&lt;p&gt;Your problem is called estimation by maximum likelihood of an ARMA(p,q) model. You are going to extend it to a general VAR (vector auto-regression) case. This problem has been solved many times, there are existing Time Series Analysis software suites that do just that and what's more, output statistics for related tests (ML, Wald and likelihood ratio, for starters). For instance, there exist R packages for this; commercial statistics software has various implementations of ML routines. Quite unfortunately, to tap into the wealth of existing knowledge you have to name your problem in the most specific way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, a few ideas on the implementation. First off, I have absolutely no idea on computational peculiarities and IEEE-754 conformance in the Go programming language. With (say) Fortran, the nitty-gritty details are well-known. There are some languages/math libraries that make my head ache on the computational front (Java, I'm looking at you!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, parallelization is the last step you'd like to take from an already-working code. The natural approach to parallelize here would be running identical routines from widely spaced starting points to convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, if you are keen on implementing an algorithm on your own and not getting a pre-packaged solution, would suggest experimenting with Powell's algorithm for differentiable functions (BOBYQA). You can port it from &lt;a href=&quot;http://ab-initio.mit.edu/nlopt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLOPT&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fourth, your idea on Cauchy errors sounds &lt;strong&gt;very&lt;/strong&gt; interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some books and articles to keep you busy:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hamilton, J.D. Time Series Analysis, 1994. Princeton.&lt;/li&gt;&#xA;&lt;li&gt;Box, G.E.P., Jenkins, G.M. and Reinsel, G.C., 1994. Time Series Analysis:&#xA;Forecasting and Control. Holden-Day.&lt;/li&gt;&#xA;&lt;li&gt;Box, G.E.P. and Luceño, A., 1997. Statistical Control by Monitoring and Feedback Adjustment. Wiley.&lt;/li&gt;&#xA;&lt;li&gt;E. J. Hannan and A. J. McDougall. Regression Procedures for ARMA Estimation. Journal of the American Statistical Association  Vol. 83, No. 402, Jun., 1988. pp.490-498&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please note this sort of questions could be better answered at &lt;a href=&quot;https://stats.stackexchange.com/&quot;&gt;Stats SE&lt;/a&gt; (Cross-Validated).&lt;/p&gt;&#xA;" OwnerUserId="2155" LastEditorUserId="-1" LastEditDate="2017-04-13T12:44:17.923" LastActivityDate="2012-11-14T09:22:04.520" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4688" PostTypeId="1" CreationDate="2012-11-15T01:36:30.853" Score="11" ViewCount="586" Body="&lt;p&gt;When we have a numerical model that represents a real physical system, and that exhibits chaos (e.g. fluid dynamics models, climate models), how can we know that the model is performing as it should? We cannot compare two sets of model output directly, because even small changes in initial conditions will dramatically change the outputs of individual simulations. We cannot compare the model output directly to observations, either because we can never know with enough detail the initial conditions of the observations, and numerical approximation would anyway cause minor differences that would propagate through the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question is partly inspired by David Ketcheson's &lt;a href=&quot;https://scicomp.stackexchange.com/questions/206/is-it-worthwhile-to-write-unit-tests-for-scientific-research-codes&quot;&gt;question on unit testing scientific code&lt;/a&gt;: I'm particularly interested in how regression tests for such models could be implemented. If a minor initial conditions change can lead to major output changes (which may well still be adequate representations of reality), then how can we separate those changes from changes caused by modifying parameters, or implementing new numerical routines?&lt;/p&gt;&#xA;" OwnerUserId="1232" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-11-16T14:09:35.873" Title="Regression testing of chaotic numerical models" Tags="|testing|numerics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4689" PostTypeId="2" ParentId="4688" CreationDate="2012-11-15T03:22:29.427" Score="2" Body="&lt;p&gt;First, I am going to focus on your last sentence, as you touch on a few different things in you question, but I feel it adequately captures what you are asking.  If you are changing numerical routines, you shouldn't be changing initial conditions or system parameters until you have validated the new routine from the old one.  On the weakest level I see this as comparing some time averaged values over your solution, and them being in agreement (even if the transient behaviors diverged from each other within the chaos).  On the strongest level, you would expect the two routines to reproduce the full transient behavior.  Which of these you want, and which is acceptable depends on what questions you are asking and what conclusions you are drawing from the solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as telling whether a model is &quot;performing as it should&quot;, that is an entirely different question.  This has nothing to do with the numerical routines you choose.  How you build your model, from your simplifying assumptions to your measurements/calculations of parameters, you should be basing all of your decisions on the physics of the problem, and hopefully prior work done on similar cases.  You may be able to validate a model with a simple case reproduced in a lab setting, but there are times when even that is non-trivial.  If you can't determine an important system parameter to within an order of magnitude, you can't expect anyone to trust the small details you are calculating in the transient local behavior.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-11-15T03:22:29.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4690" PostTypeId="2" ParentId="4688" CreationDate="2012-11-15T03:24:41.127" Score="7" Body="&lt;p&gt;All you can compare in such cases are the statistics of your solution: averages, higher moments, heat fluxes across the boundary, and other integral quantities. Take a look at one of the many papers discussing turbulence models for the Navier-Stokes equations, for example: they're full to the brim with plots of power spectra, entalpies, entropies, enstrophies, and other words you have never before heard of. All are some integral quantity of the flow and they are compared against the same integral quantities computed from other simulations and/or experiments.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-15T03:24:41.127" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4691" PostTypeId="1" AcceptedAnswerId="4729" CreationDate="2012-11-15T06:45:56.310" Score="5" ViewCount="168" Body="&lt;p&gt;The theory behind computing a single Householder reflector to zero out part of a column of a matrix is pretty well described in Matrix Computations by Golub and Van Loan. However, the blocked algorithm in Lapack's xDLARFB routines do not match the description in Matrix Computations. The book describes computation of a non symmetric reflector of the form $I-WY^T$, where as Lapack computes something of the form $I-VTV^T$. Where is the algorithm as implemented in Lapack described? Or alternatively, can anyone provide a high level picture of what is going on?&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-11-22T17:54:49.997" Title="What is the algorithm for computing block reflectors in xDLARFB" Tags="|matrix|reference-request|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4693" PostTypeId="1" AcceptedAnswerId="4698" CreationDate="2012-11-15T13:32:45.640" Score="1" ViewCount="95" Body="&lt;p&gt;Given a convex quadratic function $f(x)$, to obtain a solution for which $f(x)$ has minimal value one sets $\nabla f(x)=0$, and solves for $x$. Suppose that the result of differentiation of convex quadratic $f(x)$ is $$Ax=b.$$ Does the solution $x$ satisfying the above always exist? Statement that a solution can be obtained in a least-squares sense imply that $Ax=b$ is not exactly solvable, which means that a global minimum of convex quadratic does not exists (which is not true)(?) &lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-11-15T15:50:41.223" Title="Existence of a solution at a stationary equation for quadratics" Tags="|linear-algebra|optimization|matrix|convex-optimization|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4694" PostTypeId="2" ParentId="4670" CreationDate="2012-11-15T14:09:30.217" Score="3" Body="&lt;p&gt;off the cuff, this should be possible. Event handling is available in COMSOL. To change a parameter value through reinitialization when a certain voltage is reached should not be a problem. There is an example model using events that demonstrates how to integrate the temperature of a body. The integral is periodically reset to zero using an explicit event. This is different physics than yours, but the principle is the same. I would suggest that you contact support@comsol.com and ask for model id 14191.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To change the value back after certain time is a little more difficult but should be doable through an extra equation that integrates the time and the event after it occured, but zero else (which also needs reinitialization).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;regards&#xA;Niklas Rom (COMSOL employee)&lt;/p&gt;&#xA;" OwnerUserId="3310" LastActivityDate="2012-11-15T14:09:30.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4695" PostTypeId="1" CreationDate="2012-11-15T14:10:05.393" Score="16" ViewCount="1405" Body="&lt;p&gt;I read about computational science on Wikipedia, but my understanding is not very clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does computational science involve programming? How different is computational science from computational &lt;strong&gt;&lt;em&gt;_&lt;/em&gt;____&lt;/strong&gt;, where the blank could be any discipline (materials science, engineering, chemistry, biology, and so on)? (I will be doing computational materials science.)&lt;/p&gt;&#xA;" OwnerUserId="3267" LastEditorUserId="276" LastEditDate="2013-02-28T04:27:41.420" LastActivityDate="2013-03-11T21:01:45.530" Title="Does Computational Science involve programming?" Tags="|computational-chemistry|education|" AnswerCount="7" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4696" PostTypeId="2" ParentId="4695" CreationDate="2012-11-15T14:25:08.187" Score="3" Body="&lt;p&gt;I am not aware of text-book versions of definition or description of computational science but here are my two-cents based on my experience:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computational science involves dealing with computational processes. One of the crucial process is programming. So, yes, it involves programming. It is a different issue wether the computational scientist is a writer of a program or not but she will most likely be the user of programs written for specific scientific domains. And it is increasingly likely that she will have to write &quot;glue&quot; code to make things suited to her needs. Hope this answers your first question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not aware of computational material science but assume it is one branch of science which has computational needs, e.g. for simulations of energy and interactions between elements etc. So, yes, it will involve same general principles of computational science like other scientific domains: algorithms, procedures, glue-codes, file manipulation, configuration, visualization and so on. Hope this is close to answering your second question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computer science background of yours should definitely help. Wether it is in the scope or not is a bit subjective. However, I would say, it should be very much in your comfort zone given the subjects you have studied. &lt;/p&gt;&#xA;" OwnerUserId="2091" LastActivityDate="2012-11-15T14:25:08.187" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4697" PostTypeId="1" AcceptedAnswerId="4699" CreationDate="2012-11-15T15:15:11.293" Score="5" ViewCount="1451" Body="&lt;p&gt;I am facing a problem where I want to find the complex roots of $f(z)=z-sin(z)=0$ numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are infinitely many roots of the function, but I am only interested in the $N$ closest to the origin, or to put it more general, within a certain domain in the complex plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Newton-Raphson method also works on complex functions, so given some initial $z_0$, I end up in some root of $f(z)$. However, this is not giving me all roots. I know that the roots will have a basin of attraction, so I will basically need a $z_0$ in every basin of attraction, except that I don't have this basin to start with. Smartest I can come up with, is starting with a grid of $z_0$'s, and to select the uniques. This seems very inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefor my question: is there an efficient method to find &lt;em&gt;all&lt;/em&gt; complex roots of $f(z)$ within a domain of the complex plane?&lt;/p&gt;&#xA;" OwnerUserId="504" LastActivityDate="2012-11-16T03:27:57.280" Title="How to find all complex roots of an equation in a domain" Tags="|algorithms|reference-request|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4698" PostTypeId="2" ParentId="4693" CreationDate="2012-11-15T15:50:41.223" Score="4" Body="&lt;p&gt;$f(x):=\frac{1}{2}x^TGx+c^Tx+\gamma~~~~$ with symmetric Hessian $G$ is bounded below iff $G$ is positive semidefinite and $c$ lies in the column space of $G$. In this case, the equation fior a stationary point is $Gx+c=0~$ and has at least one solution, and all solutions (infinitely many exists if $G$ is singular) are global minimizers of $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the function is not bounded below $Gx+c=0~$ may or may not have a solution, but if it has one, the solution will never be a local minimizer.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-15T15:50:41.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4699" PostTypeId="2" ParentId="4697" CreationDate="2012-11-15T15:56:30.960" Score="2" Body="&lt;p&gt;To find all complex roots in some domain you need to employ a branch and bound method. Typically one begins with an enclosing box and splits it recursively into subboxes, then applies some constructive method for getting lower and/or upper bounds on the number of solutions in the box. If no solution exists in a box it can be discarded. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some applicable test are for example in my paper&lt;BR&gt;&#xA;A. Neumaier, Enclosing clusters of zeros of polynomials, J. Comput. Appl. Math. 156 (2003), 389-401. &lt;BR&gt;&#xA;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/ms/polzer.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/ms/polzer.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-15T15:56:30.960" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4700" PostTypeId="1" CreationDate="2012-11-15T20:47:10.927" Score="5" ViewCount="3252" Body="&lt;p&gt;Is there public code in Matlab for solving the Laplace equation in polar coordinates in a circular domain? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried a lot but my level of Matlab and Mathematica is not good enough, but still not quite understand the scheme.&#xA;Excuse my English but I'm from Peru.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the equation of Laplace&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{r}\frac{\partial}{\partial r}\left(r\frac{\partial u}{\partial r}\right)+\frac{1}{r^2}\frac{\partial^2u}{\partial\theta^2}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $0\leq r\leq R$ y $0\leq \theta \leq 2\pi$. We will use a mesh $r_i=i\Delta R$ and $\theta_j=j\Delta\theta$. We approximate the equation with the squeme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{r_i}\left(r_{i+1/2}\frac{u_{i+1j}-u_{ij}}{\Delta r}-r_{j-1/2}\frac{u_{ij}-u_{i-1j}}{\Delta r}\right)\frac{1}{\Delta r}+\frac{1}{r_i^2}\frac{u_{ij+1}-2u_{ij}+u_{ij-1}}{\Delta\theta^2}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{ij}M$ and $f_{ij}$ are mesh functions $(r_i,\theta_j)=(i\Delta r,j\Delta\theta)$. The functions are periodic in the angular index $j$ with period $J=2\pi/\Delta\theta$ and $u_{0j}$ is independent of the value of $j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To derive the condition of the source Laplace equation integrate on the disk $D$ with radius $\varepsilon$, obtaining&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int\int_D frdrd\theta=\int\int_D\frac{1}{r}\left[\frac{\partial}{\partial r}\left(r\frac{\partial u}{\partial r}\right)\right] + \frac{1}{r^2}\frac{\partial^2u}{\partial\theta^2}rdrd\theta$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_0^2\pi \frac{\partial u}{\partial r}\varepsilon d\theta$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, choosing $\varepsilon$ = $\Delta j/2$ and approximating this relation by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(0)\left(\frac{\Delta r}{2}\right)^2\pi=\sum\frac{u_{1j}-u_0}{\Delta r}\frac{\Delta r}{2}\Delta\theta$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{0j}$ is independent $j$, let's call $u_0$ we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_0=\frac{1}{J}\sum u_{ij}-f(0)\left(\frac{\Delta r}{2}\right)^2$&#xA;as $f(0)=0$&#xA;then&#xA;$u_0=\frac{1}{J}\sum u_{ij}$&lt;/p&gt;&#xA;" OwnerUserId="3314" LastEditorUserId="9667" LastEditDate="2014-08-10T22:38:50.840" LastActivityDate="2014-08-10T22:38:50.840" Title="Laplace's equation problem in Polar Coordinates (Edit)" Tags="|pde|finite-difference|matlab|mathematica|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4701" PostTypeId="1" AcceptedAnswerId="4704" CreationDate="2012-11-15T21:07:55.113" Score="3" ViewCount="121" Body="&lt;p&gt;I have some C++ code that links to matlab2008b. Are matlab 2012a and 2012b backwards compatible with 2008b?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's not trivially compatible, are there some simple steps to make it compatible?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2012-11-24T09:47:56.393" Title="Are matlab C library versions backwards compatible?" Tags="|matlab|c++|c|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4702" PostTypeId="2" ParentId="4697" CreationDate="2012-11-16T03:27:57.280" Score="2" Body="&lt;p&gt;Arnold has already given a good answer. In general, what you are asking for is similar to a global optimization algorithm for finding all the minima of a function (in your case, the minima would be those of $|f(z)|^2=f(z)\overline{f(z)}$. There is a vast amount of literature on this issue if you know the keyword (global optimization) to look for.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-16T03:27:57.280" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4703" PostTypeId="2" ParentId="4695" CreationDate="2012-11-16T03:37:21.313" Score="14" Body="&lt;p&gt;The link Godric pointed out under the question is exactly right. It gives a good overview of the many talents a computational scientist has to have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the difference between being a Computational X (X=mathematician, civil engineer, materials scientist) and being a Theoretical X or Experimental X is that the former tackle problems with computers rather than experiment or theory. Obviously, the ability to use computers -- in particular to program them -- is an important part of this, as are skills and a good understanding of the X itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computational Scientists are the people who are not immediately part of the X community but often have a background in mathematics or computer science. I would count myself as one. I would say that most of them are not as much interested in any particular application (i.e., the X) in itself but rather consider solving problems in field X as a way to develop and apply new numerical and computational methods -- i.e., as testcases for methods that are more broadly applicable. Many computational scientists are excellent programmers and, maybe more importantly, software designers and engineers. Being good at that is certainly an important part of the job description of a computational scientist.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-16T03:37:21.313" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4704" PostTypeId="2" ParentId="4701" CreationDate="2012-11-16T12:34:33.590" Score="3" Body="&lt;p&gt;In each version some functions are added, changed and removed.&#xA;As such newer versions are never fully backwards compatible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, not that much has changed from 2008 to 2012, so there is a good chance that you can get your code to work with no or few adjustments.&lt;/p&gt;&#xA;" OwnerUserId="3315" LastActivityDate="2012-11-16T12:34:33.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4705" PostTypeId="2" ParentId="2012" CreationDate="2012-11-16T12:48:47.043" Score="3" Body="&lt;p&gt;You can refer to the following pages for a full listing of the basis functions for Argyris: &lt;a href=&quot;https://docs.google.com/viewer?a=v&amp;amp;q=cache%3acfhKf4TBHjQJ%3adl.gna.org/getfem/doc/getfemlist.pdf%20&amp;amp;hl=en&amp;amp;gl=us&amp;amp;pid=bl&amp;amp;srcid=ADGEEShFFbsULh4EhCil4EAA9eszMUUsAP4V49EJuma29lp4FGTBLDtFyolmO_dluDlfcAd1MUPTjKfGxrmkg_RhPb_HQFzkY8LATLTibe22PJo9Z1QwV5RWC0iy9ChB6mJXq_UXLpGq&amp;amp;sig=AHIEtbTVogm9ZLw4lXgTvsRp1sf2nL3R2w&quot; rel=&quot;nofollow&quot;&gt;FEMList.pdf&lt;/a&gt; &lt;a href=&quot;http://fr.wikipedia.org/wiki/Fonctions_d%27Argyris&quot; rel=&quot;nofollow&quot;&gt;Wikipedia entry (French)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you can use the &lt;a href=&quot;https://github.com/VT-ICAM/ArgyrisPack&quot; rel=&quot;nofollow&quot;&gt;VT-ICAM ArgyrisPack&lt;/a&gt; that a colleague and I developed.&lt;/p&gt;&#xA;" OwnerUserId="3316" LastEditorUserId="1804" LastEditDate="2012-11-16T13:58:55.050" LastActivityDate="2012-11-16T13:58:55.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4706" PostTypeId="2" ParentId="4688" CreationDate="2012-11-16T14:09:35.873" Score="5" Body="&lt;p&gt;If your code can run in non-chaotic regimes of your underlying problem, especially non-chaotic regimes where you can use the method of manufactured solutions, you should write regression tests that run in these regimes even if they aren't otherwise interesting to you. If these tests fail, then you immediately know that something has gone wrong in your latest code changes. Then you can move on to more physically relevant problems.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2012-11-16T14:09:35.873" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4707" PostTypeId="1" CreationDate="2012-11-16T16:09:16.240" Score="8" ViewCount="282" Body="&lt;p&gt;I am trying to understand the rebinning algorithm of the VEGAS (&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0021999178900049&quot; rel=&quot;nofollow&quot; title=&quot;A New Algorithm for Adaptive Multidimensional Integration, G. Peter Lepage, Journal of Computational Physics 27, 192-203 (1978)&quot;&gt;original publication&lt;/a&gt; (&lt;a href=&quot;http://slac.stanford.edu/pubs/slacpubs/1750/slac-pub-1839.pdf&quot; rel=&quot;nofollow&quot;&gt;preprint&lt;/a&gt; from LKlevin) and &lt;a href=&quot;http://ccdb5fs.kek.jp/cgi-bin/img_index?8006210&quot; rel=&quot;nofollow&quot; title=&quot;VEGAS - An Adaptive Multi-dimensional Integration Program, G. Peter Lepage, CLNS-80/447&quot;&gt;implementation notes&lt;/a&gt;) Monte Carlo integration. I will try to explain first what I think I understood and then pose my questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, let's assume we have a 1-dimensional function $f(x)$ that is positive over the whole interval $[0,1]$. This interval is seperated into, let's say, $n$ bins. These bins are initially equally-sized. The bin sizes $\Delta x_i$ define a probability density&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho(x) = \begin{cases} 0 \le x &amp;lt; \Delta x_1 : \frac{1}{n \Delta x_1} \\ \vdots \\ \Delta x_{n-1} \le x &amp;lt; \Delta x_n : \frac{1}{n \Delta x_n} \end{cases} .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bin sizes must add up to the length of the interval in order to make $\rho(x)$ properly normalized:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{i=1}^n \Delta x_i = 1 \quad \Rightarrow \quad \int\limits_0^1 \mathrm{d}x \, \rho(x) = 1.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using $N$ randomly chosen numbers $\{ x_i \}$ from the distribution $\rho(x)$ VEGAS computes an approximation $S^{(1)}$ of the integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S^{(1)} = \frac{1}{N} \sum_{\{x_i\}} \frac{f(x_i)}{\rho(x_i)} \approx \int\limits_0^1 \mathrm{d}x \, f(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to now this is just importance sampling (I am not interested in stratified sampling) using a variably-sized grid. The interesting piece in VEGAS is the rebinning algorithm, i.e. the algorithm which recomputes the bin sizes depending on the function values accumulated in the iteration before:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For each bin the squared function values (?) are summed up (in the original publication the absolute values are summed up).&lt;/li&gt;&#xA;&lt;li&gt;There is also a dampening function applied to each value, to &quot;avoid rapid, destablizing changes&quot;.&lt;/li&gt;&#xA;&lt;li&gt;After that, each value is smoothed with the neighboring bins. I guess this also adds some stability to the rebinning algorithm for certain functions (but I can not explain why). Let's call the final values $b_i$.&lt;/li&gt;&#xA;&lt;li&gt;The bin sizes are now set such that every new bin contains approximately the average:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$\overline{b} = \frac{1}{n} \sum_{i=1}^n b_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This algorithm makes the bins grow where the function is &quot;small&quot; and shrink where the function is &quot;big&quot;. Small and big are understood in relation to each other, e.g. the function's maxima are considered &quot;big&quot; and everything else would be considered &quot;smaller&quot;. Since the probability for a point $x$ to end up in any bin is equal (VEGAS is written to exhibit this behavior), the function is sampled most where it is biggest and thereby the error is reduced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is it written that way? Why does one not tackle the problem more directly, for example by taking the averaged and binned function as a probability density for the next iteration?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3317" LastEditorUserId="3317" LastEditDate="2015-04-15T21:38:53.997" LastActivityDate="2015-04-15T21:38:53.997" Title="Rebinning algorithm in VEGAS" Tags="|monte-carlo|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4708" PostTypeId="1" AcceptedAnswerId="4876" CreationDate="2012-11-16T20:04:50.727" Score="5" ViewCount="423" Body="&lt;p&gt;I am working on estimating the position and orientation (pose) of a model (rigid object) from its silhouette in an image. For this, I have constructed an error measure between the model in its pose and the silhouette, which looks roughly like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\epsilon ( \bar{x} ) = \sum_{\forall i} \| f(\bar{x}, m_i) - s_i \|^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\bar{x}$ is a six-dimensional vector describing the 3D translation and rotations as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f( \bar{x}, p ) = R_{\bar{x}} \cdot p + t_{\bar{x}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ordinarily, this could be nonlinear least squares, however there is a catch: An assignment needs to be made between model-points $m_i$ and silhouette points $c_i$, which complicates the evaluation of the error measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am approaching the problem as a general nonlinear optimization problem. I already know that this error measure is continous, but not continously differentiable due to the aforementioned assignment. I do have gradient information however, but this does not take the assignment into account and therefore is not completely accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The question:&lt;/strong&gt; Is there a method which can calculate/approximate and visualize the basins of attractions in this six-dimensional space?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is absolutely not feasible, is there a method which can calculate/approximate the number of local minima within a &quot;bounded&quot; region?&lt;/p&gt;&#xA;" OwnerUserId="1789" LastActivityDate="2012-12-18T17:05:51.767" Title="Approximating and visualizing basins of attraction" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4709" PostTypeId="1" CreationDate="2012-11-18T06:48:21.313" Score="1" ViewCount="143" Body="&lt;p&gt;I am trying to fit the log-log plot of the cumulative distribution of a network to one of three models: Exponential (EXP) ($P(k)\sim e^{-ak}$), Exponentially truncated power law (TRU) ($P(k)\sim k^{a-1}e^{k/k_c}$), and Power law (POWER) ($P(k)\sim k^{-a}$). I know this is a low-information test, but I am simply trying to determine which of the three models is the best fit (or possibly the least terrible fit!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize that TRU has 2 degrees of freedom, while EXP and POWER have only 1 degree of freedom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following data in R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;logCPK&amp;lt;-c(0, -0.0014, -0.0038, -0.0056, -0.007, -0.013, -0.0165, -0.0222,     &#xA;-0.0261, -0.0354, -0.0478, -0.0581, -0.0714, -0.0846, -0.1068, -0.1205, &#xA;-0.1403, -0.1638, -0.1837, -0.2091, -0.2265, -0.2517, -0.2766, -0.3121, &#xA;-0.3381, -0.3744, -0.4079, -0.4485, -0.4869, -0.5326, -0.5881, -0.6383, &#xA;-0.6959, -0.7586, -0.8199, -0.8853, -0.9624, -1.0352, -1.1341, -1.2294, &#xA;-1.3521, -1.4982, -1.6824, -1.892, -2.182)&#xA;&#xA;dataPOW&amp;lt;-c(0.3387, 0.2175, 0.1197, 0.0441, -0.0198, -0.0777, -0.1262, &#xA;-0.1717, -0.2108, -0.2482, -0.281, -0.3113, -0.341, -0.3673, -0.3921, &#xA;-0.4167, -0.4387, -0.4607, -0.4806, -0.4996, -0.5187, -0.536, -0.5535, &#xA;-0.5695, -0.5856, -0.6004, -0.6154, -0.6292, -0.6426, -0.6561, -0.6686, &#xA;-0.6808, -0.6932, -0.7046, -0.7158, -0.7272, -0.7383, -0.7486, -0.7586, &#xA;-0.7689, -0.7785, -0.7883, -0.7974, -0.8064, -0.8155)&#xA;&#xA;dataEXP&amp;lt;-c(0.1981, 0.168, 0.1364, 0.1063, 0.0762, 0.0445, 0.0144, -0.0172, &#xA;-0.0473, -0.0789, -0.109, -0.1391, -0.1707, -0.2008, -0.2309, -0.2625, &#xA;-0.2926, -0.3243, -0.3544, -0.3845, -0.4161, -0.4462, -0.4778, -0.5079, &#xA;-0.5395, -0.5696, -0.6012, -0.6313, -0.6614, -0.6931, -0.7232, -0.7533, &#xA;-0.7849, -0.815, -0.8451, -0.8767, -0.9083, -0.9384, -0.9685, -1.0001, &#xA;-1.0302, -1.0619, -1.092, -1.1221, -1.1537)&#xA;&#xA;dataTRU&amp;lt;-c(-0.1867, -0.0857, -0.0193, 0.0204, 0.0443, 0.057, 0.0602, &#xA;0.0562, 0.0467, 0.0319, 0.0139, -0.0073, -0.0327, -0.0592, -0.088, -0.1202, &#xA;-0.1525, -0.1881, -0.2234, -0.26, -0.2995, -0.3383, -0.38, -0.4205, -0.464, &#xA;-0.5062, -0.5512, -0.5947, -0.6388, -0.6858, -0.731, -0.7767, -0.8253, &#xA;-0.8719, -0.919, -0.9689, -1.0191, -1.0674, -1.116, -1.1673, -1.2165, &#xA;-1.2685, -1.3183, -1.3684, -1.4212)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So my question is, how can I get the AIC values for the three models?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read, but not fully understand, the documentation for &lt;code&gt;AIC()&lt;/code&gt;, which has a general syntax of: &lt;code&gt;AIC(object, ..., k = 2)&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not understanding what I am doing, I have tried commands in R, like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; AIC(logCPK~dataPOW)&#xA;Error in UseMethod(&quot;logLik&quot;) : &#xA;  no applicable method for 'logLik' applied to an object of class &quot;formula&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I calculate the &lt;code&gt;AIC&lt;/code&gt; with the above data?&lt;/p&gt;&#xA;" OwnerUserId="3326" LastEditorUserId="391" LastEditDate="2012-12-08T03:40:22.500" LastActivityDate="2014-08-26T11:12:50.447" Title="Calculating AIC (in R or any other statistical software)" Tags="|computational-chemistry|statistics|computational-biology|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4710" PostTypeId="1" AcceptedAnswerId="4713" CreationDate="2012-11-18T13:10:20.910" Score="3" ViewCount="882" Body="&lt;p&gt;For positive semi-definite symmetric matrices $A,B$, how can I find an $X$ that minimizes $\text{Trace}(AX^TBX$) under the constraint:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that $X$ is orthogonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the matrices have real entries and $A,B$ are square while $X$ is rectangular. Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define $B=F^{T}F$. Define $Y=FX$. You get the above problem as &#xA;\begin{align}&#xA;\min_{Y}~ \text{trace}(AY^{T}Y)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now I want $X^*$ that minimizes the original problem. This is what is confusing me! &lt;/p&gt;&#xA;" OwnerUserId="3328" LastEditorUserId="3328" LastEditDate="2012-11-19T15:33:17.950" LastActivityDate="2012-11-19T15:33:17.950" Title="minimization with orthogonality constraint" Tags="|linear-algebra|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4711" PostTypeId="2" ParentId="4695" CreationDate="2012-11-18T13:24:25.207" Score="6" Body="&lt;p&gt;To put it crudely, you will not be a good professional if you claim to do 'computational science' but do not know programming. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do computation, you need a tool. And computer is such a tool. Programming is the only way to teach computer how to do a thing. Therefore programming is an essential part of computational science. This is not to say that it is the only part of it. Theory has its own sacred place. If you are very good at theory, you will design better, and what is more important correct algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I see it, you can survive without knowing programming too : by designing algorithms theoretically. But then you have to hire someone else who knows programming to verify if your algorithms work in real time or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides, it is fact of modern world that people who know programming are paid better. IMHO!&lt;/p&gt;&#xA;" OwnerUserId="3329" LastEditorUserId="3329" LastEditDate="2012-11-28T12:12:14.637" LastActivityDate="2012-11-28T12:12:14.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4712" PostTypeId="2" ParentId="4708" CreationDate="2012-11-18T13:55:44.860" Score="2" Body="&lt;p&gt;You could use a nonsmooth solver such as solvopt or ralg, run it with many random starting values and limited number of function evaluations, and make a cluster of the resulting approximate solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give you an idea of the basins of attraction. Since randomness is involved, there are no guarantees, but I do not think that rigorous methods (branch and bound) would be efficient in your context.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-18T13:55:44.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4713" PostTypeId="2" ParentId="4710" CreationDate="2012-11-18T14:09:02.050" Score="4" Body="&lt;p&gt;I guess that $A$ and $B$ are also assumed symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your second optimization problem is not equivalent to the first, for two reasons:&lt;BR&gt;&#xA;(1) You forgot the constraint $YY^T=FF^T$, which corresponds to the old orthogonality constraints. &lt;BR&gt;&#xA;(2) $F$ may be singular, in which case you cannot reconstruct $X$ from $Y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result, your transformation gained nothing.&#xA;Indeed, there doesn't seem to be a closed form solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can simplify the problem by transforming $A$ and $B$ to diagonal form (spectral factorization), and transforming $X$ accordingly: Factor $A=QaQ^T$, $B=UbU^T$ with $a,b$ diagonal, $Q,U$ square orthogonal, and define &#xA;$Z=U^TXQ$. Then $Z$ is orthogonal, $tr~AX^TbX=tr~ aZ^TbZ$, and $X$ can be recovered as $X=UZQ^T$. Thus this preserves the orthogonality constraint, and simplifies the objective function to $\sum_{i,k} a_ib_k Z_{ik}^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you have two options: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(i) Pose the problem as a constrained optimization problem to a standfard constrained optimization problem solver such as IPOPT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ii) Parameterize $Z$ as a product of reflections or rotations. (This is the Householder or givens version of the QR factorization, applied to an orthogonal matrix $Z$. Use the reflection vectors respective rotation angles as new variables.)&#xA;Then pose the problem of finding the optimal parameters as an unconstrained optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that (i) is significantly simpler to prepare, as for high solution quality, one should provide the solver with explicit first derivative information.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-11-18T19:47:10.173" LastActivityDate="2012-11-18T19:47:10.173" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4714" PostTypeId="1" CreationDate="2012-11-18T16:50:41.200" Score="5" ViewCount="281" Body="&lt;p&gt;I have a symmetric positive semidefinite covariance matrix $A$, which is approximately computed as the output of a quadratic regression. I then need to invert $A$, but often it is close to singular. I've reduced the problem by using scaling. That is, I create a diagonal matrix $D$, with elements $D_{ii} = 1/\sqrt{A_{ii}}$. Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A^{-1} = D(DAD)^{-1}D$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $DAD$ has a lower conditioning number then $A$. Unfortunately in some iterations this is not enough. The size of $A$ is quite small, say maximum $50 \times 50$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need the inverse of $A$ because I have to use it in a long calculation, where terms such as $x^TA^{-1}x$ and $A^{-1}B$ appear lots for time. Also: $A^{-1}$ represents a covariance matrix, so it has to be symmetric and positive definite. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some better way to make $A$ invertible?&lt;/p&gt;&#xA;" OwnerUserId="2140" LastEditorUserId="1117" LastEditDate="2012-11-19T19:00:00.553" LastActivityDate="2012-11-19T19:00:00.553" Title="Imposing invertibility on a Matrix" Tags="|linear-algebra|matrix|condition-number|numerics|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4715" PostTypeId="2" ParentId="4714" CreationDate="2012-11-19T00:32:45.813" Score="4" Body="&lt;p&gt;The question to ask is why you need to invert the matrix. If a matrix is near-singular, it's true that you can define something like a pseudo-inverse in some stable way but it's nevertheless true that for near-singular matrices, solving linear systems in any way is unlikely to result in anything useful because the result is so strongly dependent on small changes in the right hand side.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-19T00:32:45.813" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4716" PostTypeId="1" CreationDate="2012-11-19T01:53:20.900" Score="6" ViewCount="549" Body="&lt;p&gt;I am looking for a method to solve the matrix equation&#xA;$$ DXa = Xb $$&#xA;where $D\in \mathbb{R}^{n\times n}$ is diagonal, $a, b\in \mathbb{R}^{n}$ and $X$ is the unknown orthogonal $n\times n$ matrix such that $X^TX = I$. The matrices $a$ and $b$ have unit norm, and $D$ has diagonal entries of unit magnitude, so that the matrix equation is consistent.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2018-07-06T16:11:26.303" Title="Computing an orthogonal matrix subject to linear constraints" Tags="|reference-request|matrix-equations|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4717" PostTypeId="2" ParentId="4714" CreationDate="2012-11-19T11:55:20.577" Score="5" Body="&lt;p&gt;Multiply the diagonal elements by a factor $q&amp;gt;1$ but close to 1. It will usually do the job. (I'd first try $q=1.0001$, but one can experiment with the number of zeros in this expression; e.g., use $q=1.01$ for very noisy data.)&#xA;This is justified under certain conditions, as it is a specific form of regularization. For more on regularization, see my tutorial&lt;BR&gt; &lt;a href=&quot;http://mat.univie.ac.at/~neum/ms/regtutorial.pdf&quot; rel=&quot;nofollow&quot;&gt;http://mat.univie.ac.at/~neum/ms/regtutorial.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important exception is when some diagonal element is tiny or negative, in which case your data were insufficent for the attempted estimation.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-11-19T18:55:19.940" LastActivityDate="2012-11-19T18:55:19.940" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4718" PostTypeId="1" CreationDate="2012-11-19T16:16:08.357" Score="3" ViewCount="367" Body="&lt;p&gt;Solutions to PDEs over irregular domains can be computed using the finite difference method by the introduction of so called body fitted coordinate systems where the coordinate lines are aligned to the domain boundary. The benefit of the idea is that the finite difference method can be applied directly to the problem in a new coordinate system (with certain modifications regarding the computation of the derivatives).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/E9hah.png&quot; alt=&quot;Body fitted coordinate system&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As these modifications seem ridiculously complicated for practical consideration (&lt;a href=&quot;http://hdl.handle.net/2060/19800017591&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hdl.handle.net/2060/19800017591&lt;/a&gt; p. 18) I am looking for ways to make the problem easier by having a less generic coordinate representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is if using orthogonal coordinate systems has any advantages over a generic 2D curvilinear coordinate system (one to one mapping)? Orthogonal coordinate systems can be generated for a specific domain based on the Scwarz-Christoffel mapping (conformal mapping), what I would be interested in is if this approach would have any practical benefits, i.e. if orthogonality can somehow be exploited for this problem?&lt;/p&gt;&#xA;" OwnerUserId="3229" LastActivityDate="2012-11-19T16:16:08.357" Title="Orthogonal vs general curvilinear coordinates" Tags="|finite-difference|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4719" PostTypeId="2" ParentId="4716" CreationDate="2012-11-19T20:41:11.883" Score="1" Body="&lt;p&gt;Your belief that there should always be a solution is wrong for $n=1$ if $D=a=1$ and $b=-1$, which satisfies all your requirements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus it seems unlikely that one can say anything in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Posing the linear and the orthogonality constraint as a least squares problems in the Frobenius norm, and submitting it to an unconstrained optimization routine is probably the only thing one can do. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-19T20:41:11.883" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4720" PostTypeId="1" CreationDate="2012-11-20T11:50:24.233" Score="2" ViewCount="227" Body="&lt;p&gt;I want to perform &lt;a href=&quot;http://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm&quot; rel=&quot;nofollow&quot;&gt;$k$-Nearest Neighbor Search&lt;/a&gt; in multidimensional space, but not using for example $L_2$-distance. I want the user to specify some &quot;similar&quot;-pairs examples and then perform a search using this information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithm can I use for this?&lt;/p&gt;&#xA;" OwnerUserId="3290" LastEditorUserId="183" LastEditDate="2013-03-13T21:36:44.743" LastActivityDate="2013-03-13T21:36:44.743" Title="$k$-Nearest Neighbor Search using examples" Tags="|statistics|high-dimensional|nearest-neighbors|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4721" PostTypeId="2" ParentId="4720" CreationDate="2012-11-20T15:07:51.750" Score="1" Body="&lt;p&gt;This functionality is implemented in the libAGF library. Check here: &lt;a href=&quot;http://libagf.sourceforge.net/download.html&quot; rel=&quot;nofollow&quot;&gt;http://libagf.sourceforge.net/download.html&lt;/a&gt; To get an overview (the webpage is not very informative), download the library and check for the &lt;code&gt;agf_paper&lt;/code&gt; in &lt;code&gt;doc&lt;/code&gt; directory.&lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2012-11-20T15:07:51.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4722" PostTypeId="1" CreationDate="2012-11-21T15:24:08.320" Score="11" ViewCount="7138" Body="&lt;p&gt;Hello scicomp community,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have worked in the area of graph algorithms using frameworks such as &lt;a href=&quot;http://networkx.lanl.gov/&quot;&gt;NetworkX&lt;/a&gt; (Python), &lt;a href=&quot;http://jung.sourceforge.net/&quot;&gt;JUNG&lt;/a&gt; and &lt;a href=&quot;http://www.yworks.com/en/index.html&quot;&gt;YFiles&lt;/a&gt; (Java). I am now entering the area of parallel and high perfomance computing. For a new project, I am looking for a C++ graph library with the following features:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;has an intuitive interface that enables algorithm development&lt;/li&gt;&#xA;&lt;li&gt;supports dynamic operations: e.g. arbitrary node/edge insertions and deletions&lt;/li&gt;&#xA;&lt;li&gt;supports parallelization: e.g. shields the programmer from issues arising with multithreading&lt;/li&gt;&#xA;&lt;li&gt;has a low memory overhead and is suitable for high performance computing&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please suggest some libraries and discuss these criteria as well as pros and cons.&lt;/p&gt;&#xA;" OwnerUserId="1360" LastActivityDate="2014-02-24T07:10:44.100" Title="I am looking for a parallel dynamic graph library in C++" Tags="|algorithms|parallel-computing|libraries|graph-theory|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4723" PostTypeId="2" ParentId="4691" CreationDate="2012-11-21T23:30:02.170" Score="5" Body="&lt;p&gt;Jack's right, AFAIK, that what LAPACK uses is the &quot;compact $WY$ form:&quot;  $I - \tau V T V^H$ where $T$ is triangular.   It's equal to a product of elementary reflectors.  And a product of reflectors is not in general a reflector.&lt;br&gt;&#xA;($R$ is a reflector if $R$ is its own inverse.&lt;br&gt;&#xA;$R_1 R_2 R_1 R_2$ is not $I$, even if $R_1 R_1$ and $R_2 R_2$ are.)   But even though the compact $WY$ form&#xA;is not a reflector, it gets the usual jobs done, efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The theory of genuine (orthogonal, symmetric) block reflectors of the form&#xA;$I - \tau V V^H$&#xA;(which can be used to make a matrix block upper triangular, for example) &#xA;is contained in a paper by Parlett and me, published in &lt;a href=&quot;http://dx.doi.org/10.1137/0725014&quot; rel=&quot;nofollow&quot;&gt;SINUM 25:1 (1988), p. 189-205&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3340" LastEditorUserId="1804" LastEditDate="2012-11-22T13:56:07.343" LastActivityDate="2012-11-22T13:56:07.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4724" PostTypeId="2" ParentId="4716" CreationDate="2012-11-21T23:58:24.583" Score="3" Body="&lt;p&gt;Generalizing Arnold's example, suppose $D = I$.   Then the problem is to find an orthogonal $X$ satisfying $X(a-b) = 0$, which is possible only when $a = b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, in 2D with $D = \text{diag}(1, -1)$, one can always find a plane rotation $X$ that does the job.  Evidently the solvability for arbitrary unit-norm $a$, $b$ depends only on $D$.  What is the general condition that makes the problem solvable?  Is it something like $D$ has negative determinant, i.e., it has an odd number of -1's on the diagonal?&lt;/p&gt;&#xA;" OwnerUserId="3340" LastEditorUserId="67" LastEditDate="2012-12-19T20:07:03.037" LastActivityDate="2012-12-19T20:07:03.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4725" PostTypeId="2" ParentId="4700" CreationDate="2012-11-22T00:06:32.203" Score="1" Body="&lt;p&gt;You could use &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/pdepe.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;pdepe&lt;/code&gt;&lt;/a&gt; and turn Laplace's equation into a BVP, then solve it with a multiple shooting method. This solution is not a turnkey solution, but it does use MATLAB built-in functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately, you could set up the equations you've written in matrix form; these are algebraic and you should be able to solve them using an LU decomposition (if they're linear; use the matrix backslash operator) or a Newton-Raphson-like approach (if they're nonlinear; use &lt;code&gt;fsolve&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-11-22T00:06:32.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4726" PostTypeId="2" ParentId="485" CreationDate="2012-11-22T10:04:00.763" Score="3" Body="&lt;p&gt;You can also use &lt;a href=&quot;http://cython.org/&quot; rel=&quot;nofollow&quot;&gt;Cython&lt;/a&gt; for your algorithms. It's essentially Python with added type information for some variables that need to be &quot;fast&quot;. It translates Python code to C code, which can subsequently be compiled by your favorite C compiler. Careful adding of this type information can make your code up to 150 times faster than naive Python code.&lt;/p&gt;&#xA;" OwnerUserId="1238" LastActivityDate="2012-11-22T10:04:00.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4727" PostTypeId="1" CreationDate="2012-11-22T13:36:56.377" Score="6" ViewCount="701" Body="&lt;p&gt;For the 2D Poisson equation, there exist on finite-difference mesh, some code taking &lt;span class=&quot;math-container&quot;&gt;$O(n \log(n))$&lt;/span&gt; operations to solve it on a mesh with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; nodes. They rely on Fast Fourier Transform or Block Cyclic Reduction.&lt;/p&gt;&#xA;&lt;p&gt;Is there anything as fast on unstructured meshes? I am afraid no, but fast solution of Poisson equation on unstructured meshes is a common problem. Any known efficient solution method?&lt;/p&gt;&#xA;" OwnerUserId="3343" LastEditorUserId="20688" LastEditDate="2021-12-18T15:11:54.123" LastActivityDate="2021-12-18T15:11:54.123" Title="Poisson solver on unstructured mesh" Tags="|poisson|complexity|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4729" PostTypeId="2" ParentId="4691" CreationDate="2012-11-22T17:32:05.967" Score="4" Body="&lt;p&gt;I suppose I should give an actual answer instead of just a comment. The reason for the upper-triangular structure of the matrix $T$ in the accumulated Householder transformation $I - V T V^H$ can be seen from considering the product of two Householder transformations, say $I - \tau_1 v_1 v_1^H$ and $I - \tau_2 v_2 v_2^H$ (if $v_1$ and $v_2$ are unit vectors, then $\tau_1=\tau_2=2$, but it is often useful to scale the vectors such that their first entry is one and then to store the remaining entries directly within the matrix being modified). We see that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;(I - \tau_1 v_1 v_1^H) (I - \tau_2 v_2 v_2^H) = I - \tau_1 v_1 v_1^H - \tau_2 v_2 v_2^H + \tau_1 \tau_2 (v_1^H v_2) v_1 v_2^H,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which, as an exercise for the reader, can be rearranged into the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  I - \begin{pmatrix} v_1 &amp;amp; v_2 \end{pmatrix}&#xA;      \begin{pmatrix} \tau_1 &amp;amp; -\tau_1 \tau_2 v_1^H v_2 \\ &#xA;                      0      &amp;amp; \tau_2 \end{pmatrix}&#xA;      \begin{pmatrix} v_1 &amp;amp; v_2 \end{pmatrix}^H. &#xA;$$&#xA;This insight in fact generalizes to an arbitrary product of Householder reflections, and I recommend reading &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1141886&quot; rel=&quot;nofollow&quot;&gt;Accumulating Householder transformations, revisited&lt;/a&gt;, as well as &lt;a href=&quot;http://dx.doi.org/10.1137/0725014&quot; rel=&quot;nofollow&quot;&gt;the paper mentioned by Schreiber&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-11-22T17:54:49.997" LastActivityDate="2012-11-22T17:54:49.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4730" PostTypeId="1" AcceptedAnswerId="4731" CreationDate="2012-11-22T19:41:11.270" Score="12" ViewCount="1004" Body="&lt;p&gt;I am dealing with a tricky integral that exhibits NaNs at certain values near zero and at the moment I am dealing with them quite crudely using an ISNAN statement which sets the integrand to zero when this occurs. I have tried this with the NMS library in FORTRAN (the q1da routine - q1dax is no different) and with the GSL library in C (using the QAGS routine).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked into CQUAD (part of the GSL library for C) which is specifically designed to handle NaNs and INF in the integrand, but there is very little useful info in the reference and no example programs online that I could find. Does anyone know any other numerical integration routine for either C or FORTRAN which could do the job?&lt;/p&gt;&#xA;" OwnerUserId="3344" LastActivityDate="2012-11-26T12:57:06.763" Title="Numerical Integration - handling NaNs (C / Fortran)" Tags="|quadrature|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4731" PostTypeId="2" ParentId="4730" CreationDate="2012-11-22T20:27:22.267" Score="10" Body="&lt;p&gt;I am the author of &lt;code&gt;CQUAD&lt;/code&gt; in the GSL. The interface is almost identical to that of &lt;code&gt;QAGS&lt;/code&gt;, so if you've used the latter, it should not be difficult at all to try the former. Just remember not to convert your &lt;code&gt;NaN&lt;/code&gt;s and &lt;code&gt;Inf&lt;/code&gt;s to zeros in the integrand -- the code will deal with these itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The routine is also available in Octave as &lt;a href=&quot;http://octave.sourceforge.net/octave/function/quadcc.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;quadcc&lt;/code&gt;&lt;/a&gt;, and in Matlab &lt;a href=&quot;http://www.mathworks.co.uk/matlabcentral/fileexchange/35489-quadcc&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you provide an example of the integrands you are dealing with? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example of using &lt;code&gt;CQUAD&lt;/code&gt; to integrate a function with a singularity at one of the endpoints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_integration.h&amp;gt;&#xA;&#xA;/* Our test integrand. */&#xA;double thefunction ( double x , void *param ) {&#xA;    return sin(x) / x;&#xA;    }&#xA;&#xA;/* Driver function. */&#xA;int main ( int argc , char *argv[] ) {&#xA;&#xA;    gsl_function f;&#xA;    gsl_integration_cquad_workspace *ws = NULL;&#xA;    double res, abserr;&#xA;    size_t neval;&#xA;&#xA;    /* Prepare the function. */&#xA;    f.function = &amp;amp;thefunction;&#xA;    f.params = NULL;&#xA;&#xA;    /* Initialize the workspace. */&#xA;    if ( ( ws = gsl_integration_cquad_workspace_alloc( 200 ) ) == NULL ) {&#xA;        printf( &quot;main: call to gsl_integration_cquad_workspace_alloc failed.\n&quot; );&#xA;        abort();&#xA;        }&#xA;&#xA;    /* Call the integrator. */&#xA;    if ( gsl_integration_cquad( &amp;amp;f, 0.0 , 1.0 , 1.0e-10 , 1.0e-10 , ws , &amp;amp;res , &amp;amp;abserr , &amp;amp;neval ) != 0 ) {&#xA;        printf( &quot;main: call to gsl_integration_cquad failed.\n&quot; );&#xA;        abort();&#xA;        }&#xA;&#xA;    /* Print the result. */&#xA;    printf( &quot;main: int of sin(x)/x in [0,1] is %.16e +/- %e (%i evals).\n&quot; ,&#xA;        res , abserr , neval );&#xA;&#xA;    /* Free the workspace. */&#xA;    gsl_integration_cquad_workspace_free( ws );&#xA;&#xA;    /* Bye. */&#xA;    return 0;&#xA;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which I compiled with &lt;code&gt;gcc -g -Wall cquad_test.c -lgsl -lcblas&lt;/code&gt;. The output is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;main: int of sin(x)/x in [0,1] is 9.4608307036718275e-01 +/- 4.263988e-13 (63 evals).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which, given the result computed in Maple to 20 digits, $0.94608307036718301494$, is correct to 14 digits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that there is nothing special here, neither to tell &lt;code&gt;CQUAD&lt;/code&gt; where the singularity is, or any special treatment within the integrand itself. I just let it return &lt;code&gt;NaN&lt;/code&gt;s, and the integrator takes care of them automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that there is a bug in the latest GSL version 1.15 which can affect the treatment of singularities. It has been fixed, but has not made it to the official distribution. I used the most recent source, downloaded with &lt;code&gt;bzr branch http://bzr.savannah.gnu.org/r/gsl/trunk/&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2012-11-26T12:57:06.763" LastActivityDate="2012-11-26T12:57:06.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4732" PostTypeId="2" ParentId="4730" CreationDate="2012-11-23T09:39:45.273" Score="3" Body="&lt;p&gt;You could also check out the double-exponential quadrature formulas. They do an (implicit) change of variables, making sure that they &quot;ease-away&quot; boundary singularities. A very nice (Fortran77 and C) implementation can be found on &lt;a href=&quot;http://www.kurims.kyoto-u.ac.jp/~ooura/intde.html&quot; rel=&quot;nofollow&quot;&gt;Ooura's website&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2012-11-23T09:39:45.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4733" PostTypeId="1" AcceptedAnswerId="4734" CreationDate="2012-11-23T15:20:27.990" Score="13" ViewCount="1302" Body="&lt;p&gt;I'm interested in an advice for efficient data structures for cell browsing in unstructured cell-based finite volume CFD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example that I encountered (in &lt;em&gt;dolfyn&lt;/em&gt; cfd code) goes like this (I'll show relevant segment)&#xA;\begin{listing}&#xA;do ip=1,Ncel&#xA;         ...&#xA;         do j=1,NFaces(ip)&#xA;           k   = CFace(ip,j)&#xA;           ipp = Face(k)%cell1&#xA;           inn = Face(k)%cell2&#xA;           if( inn &amp;gt; 0 )then&#xA;             ! internal&#xA;\end{listing}&#xA;So we have an array NFaces where the number of faces for each cell is stored. Then CFace array which maps local-to-cell face number to global face number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is face based therefore there is a face data type which stores the serial number of two cells it lies between Face(k)%cell1 and Face(k)%cell2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any comments on this or suggestions for alternative approach are welcome.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2012-12-06T11:18:10.620" Title="What are some good data-types for unstructured cell-centered FVM CFD code?" Tags="|fluid-dynamics|finite-volume|data-structures|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4734" PostTypeId="2" ParentId="4733" CreationDate="2012-11-23T16:07:16.253" Score="10" Body="&lt;p&gt;The structure you show is a common choice and equivalent to storing cell-face adjacencies in a CSR matrix format, with the boundary ghost cells in a special place. However, note that FV methods can also be formulated to consist entirely or almost-entirely of face traversal where each face is only visited once (reconstruct to face centroid/quadrature point from both sides, solve Riemann problem, distribute flux back into residual on cells). You can &quot;fake&quot; that by using your cell-based traversal and skipping any two cells that are below the &quot;diagonal&quot; in the sparse matrix, but a popular alternative is to store &lt;code&gt;(leftCell, rightCell) = support(face)&lt;/code&gt;, in which case faces become first-class entities. That is useful because you typically need a place to store face quadrature points (centroids), face normals. You can also put reconstruction parts (like least squares) into the face-based data structures. The face traversal is seemingly vectorization-friendly because all the sizes are regular, but on the other hand, there are overlapping outputs so you need to organize the traversal to avoid putting it in an inner loop. With this more face-oriented data structure, it is natural to order the face numbers so that each boundary condition type can be applied using a contiguous traversal of faces (also vectorization-friendly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you choose this data structure, remember to sort the faces so that the traversal reuses cell data in cache as much as possible. See any of the PETSc-FUN3D papers for performance analysis of face ordering and related optimizations.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-11-23T16:07:16.253" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4735" PostTypeId="2" ParentId="4701" CreationDate="2012-11-24T09:47:56.393" Score="3" Body="&lt;p&gt;Depends. If the C code is too old, it might not compile or link anymore because of name changes in the eng or mx library. I ran into this when i tried to compile some code from 1992 that supposedly provided a Matlab interface to Mathematica. The reason: it used a datatype called &lt;code&gt;Matrix&lt;/code&gt; which has turned out to be &lt;code&gt;mxArray&lt;/code&gt; in newer versions of Matlab.&lt;/p&gt;&#xA;" OwnerUserId="1238" LastActivityDate="2012-11-24T09:47:56.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4736" PostTypeId="1" CreationDate="2012-11-25T01:56:48.810" Score="1" ViewCount="134" Body="&lt;p&gt;Some background: define a process $Y_t=\frac{1}{t}\int_0^tX_sds$, where $X_t$ is a standard Brownian motion. Then I define a function $u(t,X_t,Y_t)$ and require it to be a martingale. Thus, by applying Ito formula and setting drift to zero I end up with a following p.d.e.: $$u_t+u_{xx}+\frac{1}{t}(x-y)u_y=0$$$x,y\in \mathbb{R}$, with some terminal condition $u(T,x,y)=f(s,y)$. I approach to solve this problem numerically with finite differences but I have a hard time understanding what is going on at time $0$. At the moment I can't even apply any implicit scheme due to singularity at zero. One can observe though, that due to Lebesgue integration theorem $\lim_{t-&amp;gt;0} Y_t=X_0$, however, I divide by $t$, which also approaches zero and thus it is unclear how the pde behaves at that time. Any suggestions how to deal with time zero? This is where I am looking for the solution of this pde. Thanks! &lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2012-11-25T23:37:59.220" Title="how to approach time zero when the equation is not defined at that point" Tags="|pde|finite-element|finite-difference|parabolic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4737" PostTypeId="2" ParentId="4736" CreationDate="2012-11-25T02:52:56.780" Score="1" Body="&lt;p&gt;If you multiply it all through by t, you get rid of the singularity. You can then also merge terms using the chain-rule to get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(tu)_t+(tu)_{xx}+(x-y)u_y=u$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which may or may not be an easier form than if you didn't merge them together. Note at $t = 0$, this further simplifies (since t is not a function of x):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(tu)_t + (x-y)u_y = u$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve numerically, which form is better depends on what you are doing. Consider the form where $t$ is not inside the derivatives and superscript $n$ indicates the current time step and $n+1$ indicates the time step to be solved. If you use a simple first order in time, second order central in space, you would get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t^n\frac{u^{n+1}_{ij}-u^{n}_{ij}}{dt} + t^n\frac{u^n_{i+1,j}-2u^n_{ij}+u^n_{i-1,j}}{dx^2} + (x_{ij}-y_{ij})\frac{u^n_{i,j+1}-u^n_{i,j-1}}{2dy} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;keeping in mind that $dt &amp;lt; 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note how we discretized $t$. It's $t^n$ which means the solution of $u(t=0,x,y)$ is found with all the terms included. Another option is to use $t^{n+1/2}$ which is still not zero and all your terms are included (this is the average of $t^n$ and $t^{n+1}$ which in this case would be $0.5t^n$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the case where $t=0$ and $x\neq y$ gives the solution $u_y = 0$. This means that any distribution in $x$ is valid provided it is constant in $y$. On it's own, this is not unique. However, when you solve the PDE and march in time, you will get a unique distribution in $x$ approaching $t=0$ and your final answer should then be constant in $y$.&lt;/p&gt;&#xA;" OwnerUserId="391" LastEditorUserId="391" LastEditDate="2012-11-25T21:46:39.243" LastActivityDate="2012-11-25T21:46:39.243" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4738" PostTypeId="1" AcceptedAnswerId="4741" CreationDate="2012-11-25T09:03:45.343" Score="5" ViewCount="324" Body="&lt;p&gt;In principle, I know finite differences. In university, we discussed it and derived consistency and boundary conditions. &#xA;But I am still left with a big question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to design a good finite difference scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do, that my solution does not oscillate or makes other weird things.&#xA;When I look at a scheme, how can I decide, whether it's a good or bad one?&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-11-28T17:48:16.687" Title="How to design good finite difference schemes?" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ClosedDate="2012-11-30T07:15:10.367" ContentLicense="CC BY-SA 3.0" />
  <row Id="4739" PostTypeId="2" ParentId="4722" CreationDate="2012-11-25T11:37:15.873" Score="5" Body="&lt;p&gt;Perhaps, the &lt;a href=&quot;http://www.boost.org/doc/libs/1_52_0/libs/graph/doc/index.html&quot;&gt;Boost Graph Library&lt;/a&gt; is what you are looking for. It has a parser to read graphs specified in GraphViz's DOT format. While i don't really know about memory overhead, it does provide a variant for &lt;a href=&quot;http://www.boost.org/doc/libs/1_52_0/libs/graph_parallel/doc/html/index.html&quot;&gt;parallelization&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another graph library is &lt;a href=&quot;http://lemon.cs.elte.hu/trac/lemon&quot;&gt;LEMON&lt;/a&gt; but i don't really know it and if it has support for parallelization, it's not advertised. It's website makes a good impression though ;)&lt;/p&gt;&#xA;" OwnerUserId="1238" LastActivityDate="2012-11-25T11:37:15.873" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4740" PostTypeId="2" ParentId="4722" CreationDate="2012-11-25T17:30:37.033" Score="11" Body="&lt;h3&gt;Boost Graph Library and LEMON&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;As Daniel mentions in his &lt;a href=&quot;https://scicomp.stackexchange.com/a/4739/9&quot;&gt;comprehensive answer&lt;/a&gt;, the most full-featured general C++ library is the &lt;a href=&quot;http://www.boost.org/doc/libs/1_52_0/libs/graph/doc/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boost Graph Library&lt;/a&gt;.  There is a new distributed-memory extension capable of doing some basic algorithms such as breadth-first and depth-first search, minimum spanning trees, and connected components search, but I am not very familiar with the new project.  The Boost Graph Library itself is well-reputed and used in many projects around the world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are doing basic HPC graph work, you might want to start with the Boost Graph Library, but be aware that many HPC C++ compilers have difficulty with Boost (despite its fairly strict adherence to C++ standards), and you may need to use an older version of Boost or a non-vendor compiler such as GCC to get it working on HPC systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick browse of LEMON's repositories shows that there is involvement from the IBM BlueGene supercomputing team, but I don't see any dependencies or configuration for MPI, so this is likely to only be a serial graph library at the moment.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Load-balancing and Dynamic graph (re)-partitioning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you are interested in load-balancing and dynamic graph partitioning, you have several more options.  Perhaps the most well-known library is &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;ParMETIS&lt;/a&gt;, which was updated to version 4 last year.  ParMETIS features vertex-based weighting, which is important for multi-physics simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ParMETIS's European competitor is &lt;a href=&quot;http://www.labri.fr/perso/pelegrin/scotch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PT-Scotch&lt;/a&gt;, which has had better performance for certain types of problems, but, similar to ParMETIS, is not frequently updated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also be interested in &lt;a href=&quot;http://www.cs.sandia.gov/Zoltan/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zoltan&lt;/a&gt;, which is part of the Sandia National Laboratories Trilinos meta-package for scientific computing in C++.  Zoltan features its own hierarchical partitioners and interfaces into both ParMETIS and PT-Scotch.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Graph500&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you are working on the bleeding edge of concurrent search, optimization (single source shortest path), and edge-oriented (maximal independent set), you will also be interested in the &lt;a href=&quot;http://www.graph500.org/specifications&quot; rel=&quot;nofollow noreferrer&quot;&gt;freely available Graph500 benchmark&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-02-24T07:10:44.100" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4741" PostTypeId="2" ParentId="4738" CreationDate="2012-11-25T20:36:41.253" Score="10" Body="&lt;p&gt;The quality of the scheme depends on the type of equation to be solved. You can check for dispersion and dissipation: the dominant error term for odd-order schemes is typically dissipative, while the dominant error term for even-ordered schemes is typically dispersive; it is true that some schemes, such as symmetric centered schemes, have only dispersive error terms (&lt;em&gt;thanks to David Ketcheson for pointing this out&lt;/em&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your solution is smooth, ie. no discontinuities, then central schemes are better than biased schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your solution not smooth, then upwind or upwind-biased schemes are required. A central scheme will generate oscillations. Filtering or artificial dissipation could be used to offset the oscillations though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Downwind biasing is unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are running a problem that has smooth and non-smooth regions, then you can look at schemes that adjust order and/or bias such as WENO or ENO schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach to determine stability of a method is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&quot; rel=&quot;nofollow&quot;&gt;Von Neumann Stability Analysis&lt;/a&gt; where you substitute a solution in the form of a Fourier series into your equation and see whether the disturbances are amplified or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Almost all stability methods only work on linear PDE's, but some insight can be made into non-linear stability from those approaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other issue that often arises is grid quality. Some schemes will tolerate a non-uniform grid just fine while others go unstable if used on a non-uniform grid. Typically higher order schemes are more sensitive to grid issues than lower order schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last thing to check is conservation, if that matters. You may come up with a numerical scheme that is consistent but not conservative. If it is conservative then it is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Telescoping_series&quot; rel=&quot;nofollow&quot;&gt;telescoping series&lt;/a&gt; and you should be left with only the boundary terms in your series when summed over a grid. Which makes sense intuitively, what comes in goes out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In aerodynamics, the classic example of this is the 1970 Murman-Cole scheme for transonic flow calculations. The scheme is used to solve the transonic potential equations. But the original formulation actually gave answers closer to real life than it should, it showed viscous effects and non-linear effects that it shouldn't have shown. In 1973, Murman found the issue and published a correction to make the scheme conservative. The answer got &quot;better&quot; in the sense that it showed the solution expected for the equation solved, but that solution was further from the real, physical solution. So practitioners still use the non-conservative form because it happened to give a more physical answer, despite such an answer not being a solution to the equation it is supposed to solve. Sometimes being wrong is lucky.&lt;/p&gt;&#xA;" OwnerUserId="391" LastEditorUserId="391" LastEditDate="2012-11-28T17:48:16.687" LastActivityDate="2012-11-28T17:48:16.687" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4742" PostTypeId="2" ParentId="4727" CreationDate="2012-11-25T23:27:26.573" Score="5" Body="&lt;p&gt;Multigrid solves the Poisson equation with complexity $O(n)$ which is the optimal possible complexity.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-25T23:27:26.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4743" PostTypeId="2" ParentId="4736" CreationDate="2012-11-25T23:37:59.220" Score="1" Body="&lt;p&gt;In order to understand what happens at $t=0$ you shouldn't look at this as a point value in time -- i.e., you shouldn't assume that the PDE holds pointwise in time in the same way as, say, the Laplace equation doesn't hold pointwise in space. Rather, the PDE will hold in the weak form with regard to all possible test functions from appropriate test spaces. The weak form contains integrals over both space and time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you discretize the equation in the strong form, using for example finite differences in time, you get into trouble because of the singularity at $t=0$ (in the original form) or because certain terms disappear (after multiplying through with $t$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you discretized the weak form using test functions that exist in both space and time, you would realize that you have integrals of the form&#xA;$$&#xA;  \int_0^T \int_\Omega (tu(x,t))_t \phi(x,t) \; dx \; dt&#xA;$$&#xA;where it doesn't make a difference what exactly happens at $t=0$. The same would be true for the term with the factor $1/t$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-25T23:37:59.220" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4744" PostTypeId="1" AcceptedAnswerId="4746" CreationDate="2012-11-26T15:59:49.643" Score="12" ViewCount="725" Body="&lt;p&gt;So the Cholesky decomposition theorem states that &#xA;that any &lt;strong&gt;real symmetric positive-definite&lt;/strong&gt; matrix $M$ has a Cholesky decomposition $M= LL^\top$ where $L$ is a lower triangular matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $M$, we already know there are fast algorithms to calculate its Cholesky factor $L$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose I was given a rectangular $m\times n$ matrix $A$, and I knew that $A^\top A$ was positive definite. Is there a way to calculate the Cholesky factor $L$ of $A^\top A$ without computing $A^\top A$ explicitly and then applying Cholesky factorization algorithms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is a very large rectangular matrix performing $A^\top A$ explicitly seems very expensive and hence the question.&lt;/p&gt;&#xA;" OwnerUserId="3352" LastEditorUserId="127" LastEditDate="2013-05-14T18:09:14.920" LastActivityDate="2013-05-14T18:09:14.920" Title="Computation of Cholesky factor" Tags="|linear-algebra|algorithms|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4745" PostTypeId="2" ParentId="4744" CreationDate="2012-11-26T16:34:39.807" Score="8" Body="&lt;p&gt;Yes. Compute the $QR$ factorization and take $L=R^T$; rescale the rows of $R$ if necessary (by changing some of their signs) to make the sign of the diagonal nonnegative (as the Cholesky factor is defined to have a nonnegative diagonal). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sparse QR factorizations see, e.g., &#xA;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=174408&quot;&gt;http://dl.acm.org/citation.cfm?id=174408&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-26T16:34:39.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4746" PostTypeId="2" ParentId="4744" CreationDate="2012-11-26T16:35:01.417" Score="9" Body="&lt;p&gt;Yes, you can obtain the factor (up to the signs of entries) using QR decomposition; see &lt;a href=&quot;https://scicomp.stackexchange.com/a/2893/1804&quot;&gt;this answer&lt;/a&gt;. Note that if all you're interested in is solving the least squares problem that lead to the normal equations involving $A^T A$, you can use the QR decomposition directly.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-11-26T16:35:01.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4747" PostTypeId="2" ParentId="3631" CreationDate="2012-11-26T17:24:23.783" Score="4" Body="&lt;p&gt;As page 5 of the paper you linked to explains, the random surfer model of the PageRank algorithm interprets one step in the power iteration as a surfer looking at a given page following one of the links with probability $c$ (which of these is determined by their relative weight, i.e., the components of the current iterate $x^k$) and jumping to an unrelated page (e.g., by using a bookmark) with probability $1-c$ (which if these is determined by the components of the vector $p$). As explained, writing this as a matrix-vector product $pA$ with an explicit matrix $A$ is horrendously expensive since jumping to an arbitrary page leads to a fully populated matrix. So one writes $A=cP+(1-c)E$ and calculates $xA = cxP + (1-c)xE$ separately. In the algorithm above, step 2 corresponds to the first term and steps 3 and 4 correspond to the second term. The specific form is derived in detail in the paper (equations (4) and (5)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the Wikipedia article only includes a uniform jumping probability (confusingly called &lt;em&gt;damping factor&lt;/em&gt;), while the paper uses a non-uniform probability (given by the personalization vector $p$) and includes an additional term $D$ to prevent getting stuck on pages without outgoing links.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I should also remark that the Wikipedia article is not a terribly good explanation of what is going on, especially mathematically. A better source is this &lt;a href=&quot;http://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.pdf&quot; rel=&quot;nofollow&quot;&gt;SIAM Review article&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2012-11-26T17:35:30.637" LastActivityDate="2012-11-26T17:35:30.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4748" PostTypeId="1" CreationDate="2012-11-26T21:55:41.827" Score="4" ViewCount="119" Body="&lt;p&gt;Suppose I have the following MATLAB code. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms a b c1 c2&#xA;&#xA;c1 = a + b + pi*b&#xA;c2 = a + b + 0.5*b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then c&lt;sub&gt;1&lt;/sub&gt; gets evaluated to &lt;code&gt;a + b + pi*b&lt;/code&gt; and c&lt;sub&gt;2&lt;/sub&gt; to &lt;code&gt;a + (3*b)/2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I need MATLAB to calculate double-precision values for the coefficients for &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; and not just symbolic expressions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I do this? e.g. I want c&lt;sub&gt;1&lt;/sub&gt; to be evaluated as &lt;code&gt;a + 4.1416*b&lt;/code&gt; and c&lt;sub&gt;2&lt;/sub&gt; as &lt;code&gt;a + 1.5*b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, suppose I am interested in doing arithmetic like c&lt;sub&gt;1&lt;/sub&gt;*c&lt;sub&gt;2&lt;/sub&gt;, c&lt;sub&gt;1&lt;/sub&gt;*5, I would like the &#xA;coefficients of &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; to evaluated as numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I do this? &lt;/p&gt;&#xA;" OwnerUserId="3352" LastEditorUserId="22380" LastEditDate="2016-11-18T11:17:40.107" LastActivityDate="2016-11-18T11:17:40.107" Title="Simple substitutions using symbolic computing in MATLAB" Tags="|matlab|symbolic-computation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4749" PostTypeId="1" CreationDate="2012-11-27T00:08:18.907" Score="9" ViewCount="292" Body="&lt;p&gt;Aside from extra computational cost due to having to compute both fluxes over a certain region, is there any downside to blend two flux evaluations for a hybrid scheme in a finite volume method? The flux evaluation would look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{F}_{i+\frac12} = \Lambda_{i+\frac12} \mathbf{F}^c_{i+\frac12} &#xA;                       + (1 - \Lambda_{i+\frac12}) \mathbf{F}^u_{i+\frac12}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The switch is based on a pressure and/or density gradient sensor depending on your application. $\mathbf{F}^c$ is a central scheme (McCormack, compact, ...) and $\mathbf{F}^u$ is an upwind scheme like a flux-difference splitting with a MUSCL reconstruction. Are there any issues in terms of numerics, conservative properties if I am blending the two schemes using a continuous function for $\Lambda$ as opposed to simply switching between schemes with $\Lambda$ valued as either 0 or 1?&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2014-01-08T09:57:36.797" Title="Hybrid spatial schemes for CFD: any downside to blending versus switching?" Tags="|fluid-dynamics|finite-volume|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4750" PostTypeId="1" CreationDate="2012-11-27T08:42:23.177" Score="6" ViewCount="1602" Body="&lt;p&gt;I am looking for flexible and easy to learn mesh framework which provides data structure for representing and manipulating meshes in 2D i 3D. I've already found a few:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;MSTK&lt;/strong&gt; &lt;a href=&quot;https://software.lanl.gov/MeshTools/trac&quot;&gt;https://software.lanl.gov/MeshTools/trac&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;OpenMesh&lt;/strong&gt; &lt;a href=&quot;http://openmesh.org/&quot;&gt;http://openmesh.org/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It should work both on windows and linux in C++ and be open source or free. Do you have any suggestion?&lt;/p&gt;&#xA;" OwnerUserId="1010" LastEditorUserId="1010" LastEditDate="2012-11-27T12:05:19.310" LastActivityDate="2012-11-27T12:28:45.897" Title="Flexible Mesh Framework" Tags="|c++|mesh|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4751" PostTypeId="2" ParentId="4748" CreationDate="2012-11-27T10:22:05.603" Score="4" Body="&lt;p&gt;To evaluate a symbolic expression numerically, you can use &lt;a href=&quot;http://www.mathworks.com/help/symbolic/subs.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;subs&lt;/code&gt;&lt;/a&gt; to insert numerical values for symbolic variables:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; subs(c1,{a,b},{1.1,1.2})&#xA;&#xA;ans =&#xA;&#xA;    6.0699&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Note that as long as &lt;code&gt;a&lt;/code&gt; and/or &lt;code&gt;b&lt;/code&gt; are symbolic variables, I don't see a  point in evaluating &lt;code&gt;pi&lt;/code&gt; or &lt;code&gt;3/2&lt;/code&gt; numerically. However, &lt;a href=&quot;http://www.mathworks.com/help/symbolic/vpa.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;vpa&lt;/code&gt;&lt;/a&gt; seems to do what you want.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2012-11-27T10:22:05.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4752" PostTypeId="2" ParentId="4722" CreationDate="2012-11-27T10:40:44.647" Score="5" Body="&lt;p&gt;I'd also like to mention &lt;a href=&quot;http://www.cc.gatech.edu/stinger/index.php&quot; rel=&quot;noreferrer&quot;&gt;STINGER&lt;/a&gt;, a dynamic graph data structure designed for parallelism. According to the website, it is designed for the following objectives:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Portability: Algorithms written for STINGER can easily be translated/ported between multiple languages and frameworks&lt;/p&gt;&#xA;&lt;p&gt;Productivity: STINGER should provide a common abstract data structure such that the large graph community can quickly leverage&#xA;each others' research developments. This is similar in philosophy to&#xA;the numerical algorithms community implicit use of sparse and dense&#xA;matrices.&lt;/p&gt;&#xA;&lt;p&gt;Performance: It is recognized that no single data structure is optimal for every graph algorithm. The objective of STINGER is to&#xA;configure a sensible data structure that can run most algorithms well.&#xA;There should be no significant performance reduction for using STINGER&#xA;when compared with another general data structure across a broad set&#xA;of typical graph algorithms. STINGER should assume a shared memory&#xA;address space, and allow both sequential or parallel algorithms. The&#xA;data structure should allow parallel algorithms to exploit concurrency&#xA;where possible.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is not as generic as LEMON or Boost Graph Library and in an earlier stage of development. If you check it out, I'd be interested in your comments.&lt;/p&gt;&#xA;" OwnerUserId="1360" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-11-27T10:40:44.647" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4753" PostTypeId="1" AcceptedAnswerId="4755" CreationDate="2012-11-27T11:57:35.140" Score="1" ViewCount="575" Body="&lt;p&gt;I have the following boundary value problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-(\alpha u')' + \gamma u = f $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in $\Omega = (a,b)$ with b.c. $u(a) = u(b) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\alpha &amp;gt; 0, \gamma ≥ 0$ and $f:(a,b) \to \Re $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weak formulation I got is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_a^b \alpha u' v' + \int_a^b \gamma uv = \int_a^b fv~~~~~\forall v\in V=H^1_0(a,b)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then to discretize this, using Galerkin's method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ find~~u_h \in V_h:a(u_h,v_h)=F(v_h)~~~\forall v \in V_h \subset V $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And define $V_h$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$V_h = \Big\{ v_h \in X^1_h:v_h(a)=v_h(b)=0\Big\} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have a basis $ \phi_i(x)  = \cases{ \frac{x-x_{i-1}}{x_i-x_{i-1}},~~~~x_{i-1} ≤ x ≤ x_i \cr \cr \frac{x_{i+1}-x}{x_{i+1}-x_i},~~~~x_i ≤ x ≤ x_{i+1} \cr \cr 0,~~~~~~~~~~~~otherwise}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now after all this, how can I build the problem in the form $Au = f$ ?? I'm really confused when trying to do this, can someone explain to me in a clearly and basic way, it's the first time I see FEM. If you have a link to any reference (books, pdfs, videos) that can help would be perfect.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="123" LastEditDate="2012-11-28T17:30:53.127" LastActivityDate="2012-11-28T17:30:53.127" Title="How to apply a Galerkin finite element method to a linear, one-dimensional boundary value problem" Tags="|matrix|finite-element|ode|discretization|polynomials|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4754" PostTypeId="2" ParentId="4750" CreationDate="2012-11-27T12:28:45.897" Score="1" Body="&lt;p&gt;If quadrangles are fine, check &lt;a href=&quot;http://www.p4est.org&quot; rel=&quot;nofollow&quot;&gt;www.p4est.org&lt;/a&gt;, a C++ open source mesh library which has shown very good scalability to very large number of cores. You may also want to check for the DUNE project at &lt;a href=&quot;http://www.dune-project.org/&quot; rel=&quot;nofollow&quot;&gt;www.dune-project.org&lt;/a&gt;. They have a very nice abstract grid interface which can be used to make use of several different mesh libraries dependent on your specific requirement. &lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2012-11-27T12:28:45.897" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4755" PostTypeId="2" ParentId="4753" CreationDate="2012-11-27T12:31:39.887" Score="2" Body="&lt;p&gt;Constructing the matrix $A$ and vector $f$ from the (bi)linear forms $a$ and $F$ is called &lt;em&gt;assembly&lt;/em&gt;. The key observation is that since $a$ is bilinear and $V_h$ is finite-dimensional, you can substitute &#xA;$$ u_h(x) = \sum_{i=1}^N u_i \phi_i(x) $$ &#xA;(where $u_i$ is the $i$th component of the solution vector $u$ and $N$ is the number of nodes $a=x_1&amp;lt;\dots&amp;lt;x_N=b$)&#xA;and only need to test with all basis functions $\phi_1,\dots,\phi_N$ instead of all $v_h$. Doing this and pulling the sums out of the bilinear form yields&#xA;$$&#xA;\sum_{i=1}^N a(\phi_i,\phi_j)u_i = F(\phi_j) \qquad \text{for all }j=1,\dots,N.&#xA;$$&#xA;This is a linear system for $u=(u_1,\dots,u_N)^T$ with the matrix $A=(a_{ij})$ and right hand side $F=(F_j)$ where&#xA;$$&#xA;a_{ij} = \int_a^b \alpha\phi_i'\phi_j'\,dx + \int_a^b\gamma \phi_i\phi_j\,dx,\qquad F_j = \int_a^b f \phi_j\,dx.&#xA;$$&#xA;Since the $\phi_j$ are given analytically, you can compute these integrals explicitly for constant $\alpha$ and $\gamma$. If $\alpha$ or $\gamma$ is a function, you would approximate the integrals using Gauss quadrature.&#xA;Note that the product $\phi_i\phi_j$ is zero unless $i-1 \leq j \leq i +1$, which you can use to reduce the number of entries to compute. For homogeneneous Dirichlet conditions, you can just fix the values of $u_1 = u_N = 0$ (since these are the only functions which are nonzero at the boundary).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above procedure is called &lt;em&gt;node-based assembly&lt;/em&gt;, which should suffice for your one-dimensional model problem. A better alternative in higher dimensions is &lt;em&gt;element-based assembly&lt;/em&gt;, where instead of looping over all (non-trivial) pairs of $i$ and $j$, you loop over all &lt;em&gt;elements&lt;/em&gt; $[x_i,x_{i+1}]$ and compute all non-zero integrals over this interval. Summing all contributions for each pair $(i,j)$ gives you $a_{ij}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is explained in any standard textbook on finite element methods; one with a particular focus on the algorithmic side (and less on the mathematical theory) is &lt;a href=&quot;http://www.math.mtu.edu/~msgocken/fembook/index.html&quot; rel=&quot;nofollow&quot;&gt;Mark Gockenbach, &lt;em&gt;Understanding and Implementing the Finite Element Method&lt;/em&gt;, SIAM, 2006&lt;/a&gt;. For an even simpler test problem in 1D, both approaches are explained in Chapter 1.4 of these &lt;a href=&quot;http://www.uni-graz.at/~clason/teaching/numpde12/NumPDESkript.pdf&quot; rel=&quot;nofollow&quot;&gt;lecture notes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other good (mathematical) references which address implementation are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/mathematics/applications/book/978-0-387-20574-8&quot; rel=&quot;nofollow&quot;&gt;Guermond Ern, &lt;em&gt;Theory and Practice of Finite Elements&lt;/em&gt;, Springer 2004&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://homepage.ruhr-uni-bochum.de/dietrich.braess/FEM_corr_eng3.htm#NEW&quot; rel=&quot;nofollow&quot;&gt;Dietrich Braess, &lt;em&gt;Finite Elements&lt;/em&gt;, Cambridge University Press, 2007&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Between these three books, you should get a good coverage of material. For self-study, I'd particularly recommend the last book, since it is quite readable and nicely presents the theory without too much technical detail (and there are worked solutions to some of the exercises - available from the link above - so you can check your answers).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2012-11-27T13:20:07.077" LastActivityDate="2012-11-27T13:20:07.077" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4756" PostTypeId="1" AcceptedAnswerId="4760" CreationDate="2012-11-28T00:00:44.150" Score="5" ViewCount="2010" Body="&lt;p&gt;I'm having a difficult time understanding the difference between the linear algebra packages &lt;a href=&quot;http://icl.cs.utk.edu/magma/&quot;&gt;MAGMA&lt;/a&gt; and &lt;a href=&quot;http://icl.cs.utk.edu/plasma/index.html&quot;&gt;PLASMA&lt;/a&gt; from just a quick glance. It looks like MAGMA is oriented towards GPU's and vector coprocessors, while PLASMA is oriented towards a vanilla multicore system. But I'd appreciate an explanation comparing them, and what they do differently from BLAS/LAPACK/ScaLAPACK. &lt;/p&gt;&#xA;" OwnerUserId="1420" LastActivityDate="2012-11-28T15:06:47.707" Title="Magma vs. Plasma" Tags="|linear-algebra|lapack|blas|gpu|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4757" PostTypeId="1" AcceptedAnswerId="4758" CreationDate="2012-11-28T02:36:43.657" Score="8" ViewCount="698" Body="&lt;p&gt;When evaluating cylindrical harmonics, one needs to evaluate trigonometric functions $\cos(m\theta)$ and $\sin(m\theta)$, potentially for large integer $m$ and $\theta\in[-\pi,\pi]$. What is the best way to do this in C code? Currently, I just evaluate at the angle $m\theta$, but I would suspect that standard libraries lose accuracy at large arguments. I was considering using double angle formulas and the like to recursively reduce the magnitude of the arguments, but I'm wondering if that ends up incurring more error.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="127" LastEditDate="2018-04-13T06:58:45.830" LastActivityDate="2018-04-13T06:58:45.830" Title="Evaluating sine and cosine of an integer multiple of an angle" Tags="|special-functions|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4758" PostTypeId="2" ParentId="4757" CreationDate="2012-11-28T03:15:35.503" Score="4" Body="&lt;p&gt;If you do it iteratively, by computing $\sin(n \theta)$ and $\cos(n \theta)$ from $\sin((n - 1) \theta)$ and $\cos((n - 1) \theta)$, then floating point errors will not blow up by accumulation. This is because the transition matrix is orthonormal.  It is a simple rotation matrix.&lt;/p&gt;&#xA;" OwnerDisplayName="user335" LastActivityDate="2012-11-28T03:15:35.503" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4759" PostTypeId="2" ParentId="4757" CreationDate="2012-11-28T10:03:41.927" Score="7" Body="&lt;p&gt;I've encountered a similar problem before, but I was more worried about speed than accuracy (see paper &lt;a href=&quot;http://eprints.maths.ox.ac.uk/1401/1/NA-11-12.pdf&quot;&gt;here&lt;/a&gt;). If your angle $\vartheta$ is the result of an $\arccos(\cdot)$, which is often the case in geometric computations, you can use &lt;a href=&quot;http://en.wikipedia.org/wiki/Chebyshev_polynomials&quot;&gt;Chebyshev polynomials&lt;/a&gt; which are defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_k(x) = \cos(k\arccos(x)), \quad \mbox{or} \quad T_k(\cos(\vartheta)) = \cos(k\vartheta)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and can be evaluated quickly ans stably using the three-term recurrence relation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ T_k(x) = 2xT_{k-1}(x) - T_{k-2}(x), \quad T_0(x) = 1, \quad T_1(x) = x. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, for your problem, you can evaluate $\cos(m\vartheta)$ in $m+1$ multiplications and additions, provided you have $\cos(\vartheta)$.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2012-11-28T10:03:41.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4760" PostTypeId="2" ParentId="4756" CreationDate="2012-11-28T14:13:01.823" Score="1" Body="&lt;p&gt;See &lt;a href=&quot;http://icl.cs.utk.edu/news_pub/submissions/plasma-scidac09.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; or the PLASMA README. LAPACK is serial and parallelism enters only via multi threaded BLAS3. ScaLAPACK is for distributed memory machines and requires BLACS/MPI. PLASMA targets shared memory parallelism on multicore machines.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2012-11-28T15:06:47.707" LastActivityDate="2012-11-28T15:06:47.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4761" PostTypeId="1" CreationDate="2012-11-28T15:02:05.513" Score="1" ViewCount="638" Body="&lt;p&gt;What is a good way to check if the any numerical error is occured in conjugate gradient algorithm. Additionally why is it not suggested to check error by checking A-orthogonality of search direction or checking orthogonality of residuals?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: here by error I mean error from floating point unit of CPU because of incorrect computation (which can be due to corrupt data in cache etc.) not due to rounding error. In some cases the errors can be due incorrect computation of matrix vector product (in cases where matrix A is not explicitly available).&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2012-11-28T22:42:33.627" Title="Checking for error in conjugate gradient algorithm" Tags="|linear-algebra|linear-solver|iterative-method|conjugate-gradient|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4762" PostTypeId="2" ParentId="4761" CreationDate="2012-11-28T16:23:23.050" Score="0" Body="&lt;p&gt;If you try to use the CG to seek the least square solution of $Ax=b$, where $A$ is not symmetric or even square, you will solve the normal equation $A^TAx=A^Tb$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, from my experience, it's very important to check if $A^T$ is indeed the adjoint of $A$ by using the definition of adjoint (inner product), especially when you implicitly compute $A$ and $A^T$. Every time I found my CG didn't converge, it's due to the error from $A^T$, i.e. it's not the strictly adjoint of $A$.&lt;/p&gt;&#xA;" OwnerUserId="1614" LastActivityDate="2012-11-28T16:23:23.050" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4763" PostTypeId="2" ParentId="4761" CreationDate="2012-11-28T16:57:21.113" Score="3" Body="&lt;p&gt;What you are looking for is called &lt;em&gt;fault tolerant computing&lt;/em&gt;. For highly sensitive routines, one traditionally used redundant computation: Perform every elementary step twice, and if the results agree, continue; otherwise repeat again and choose the result which occurred twice (often on independent machines).&#xA;There are more recent approaches which have less overhead, but except for very specific types of errors, some sort of extra computation (checksums, probabilistic repeats together with checkpointing) are required to catch errors. This is especially the case for iterative linear solvers, since these are quite vulnerable (they could fail to converge or even calculate wrong solutions). For conjugate gradients, this is addressed in detail in &lt;a href=&quot;http://www.emcl.kit.edu/preprints/emcl-preprint-2011-10.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.emcl.kit.edu/preprints/emcl-preprint-2011-10.pdf&lt;/a&gt;. Here, the conjugate gradient (in defect correction form) is frequently restarted, and iterations where the residual has failed to decrease are discarded as corrupted by errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand (and since you referred to this case in your question), the error due to incorrect computation of the matrix vector product has also been investigated; the keyword is &lt;em&gt;inexact conjugate gradient&lt;/em&gt;. See, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/S1064827502406415&quot; rel=&quot;nofollow&quot;&gt;Theory of Inexact Krylov Subspace Methods and Applications to Scientific Computing, Simoncini, V. and Szyld, D., SIAM Journal on Scientific Computing 2003 25:2, 454-477&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/S0895479802403459&quot; rel=&quot;nofollow&quot;&gt;Inexact Krylov Subspace Methods for Linear Systems,&#xA;van den Eshof, J. and Sleijpen, G.,&#xA;SIAM Journal on Matrix Analysis and Applications 2004 26:1, 125-153 &lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2012-11-28T22:42:33.627" LastActivityDate="2012-11-28T22:42:33.627" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4764" PostTypeId="2" ParentId="4749" CreationDate="2012-11-28T17:37:00.340" Score="4" Body="&lt;p&gt;The approach you are using will maintain conservation either way.  There are other obvious approaches that are not conservative, and can cause problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible (and even likely) that you will lose an order of accuracy in the region where you switch, if you examine the local truncation error.  But typically this error is localized so that the global error is still of the expected order.  So in my experience, you will see essentially the same behavior whether you use a hard switch or a transition region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a manuscript on (more or less) this topic: &lt;a href=&quot;https://arxiv.org/abs/1310.7168&quot; rel=&quot;nofollow&quot;&gt;Error Analysis of Explicit Partitioned Runge-Kutta Schemes for Conservation Laws &lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be very interested in knowing what you see when trying the two approaches, if it is different from what I'm suggesting.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-01-08T09:57:36.797" LastActivityDate="2014-01-08T09:57:36.797" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4765" PostTypeId="1" CreationDate="2012-11-28T18:09:44.123" Score="5" ViewCount="764" Body="&lt;p&gt;I was wondering, before trying to do that myself, has anyone attempted to do orthonormalization of Bernstein polynomials using Gram-Schmidt?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I discussed this with several people and have been told that Bernstein polynomials don't make a good basis for FEM because they are not orthogonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't use FEM, instead I made a (pseudospectral-like) collocation method formulation, and documented my attempts to solve elliptic problems in 2D domains in &lt;a href=&quot;http://arxiv.org/abs/1211.3567&quot; rel=&quot;nofollow&quot;&gt;an arXiv article&lt;/a&gt;. I had exponential convergence with polynomial orders $n&amp;lt;20$. After that approximation become worse as $n$ was increased. One of the reasons may be non-orthogonality of Bernstein polynomial basis functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code discussed is &lt;a href=&quot;http://code.google.com/p/bernstein-poly/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to make a new orthogonal basis using Gram-Schmidt and try again.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="9" LastEditDate="2012-11-29T04:37:50.163" LastActivityDate="2012-11-29T04:37:50.163" Title="Orthonormalized Bernstein polynomials using Gram-Schmidt" Tags="|finite-element|basis-set|collocation|spectral-method|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4766" PostTypeId="1" AcceptedAnswerId="4767" CreationDate="2012-11-28T18:17:38.753" Score="13" ViewCount="10201" Body="&lt;p&gt;According to &lt;a href=&quot;http://en.wikipedia.org/wiki/Rate_of_convergence&quot;&gt;Wikipedia&lt;/a&gt; the rate of convergence is expressed as a specific ratio of vector norms. I'm trying to understand the difference between &quot;linear&quot; and &quot;quadratic&quot; rates, at different points of time (basically, &quot;at the beginning&quot; of the iteration, and &quot;at the end&quot;). Could it be stated that: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;with linear convergence, the norm of the error $e_{k+1}$ of the iterate $x_{k+1}$ is bounded by $\|e_k\|$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;with quadratic convergence, the norm of the error $e_{k+1}$ of the iterate $x_{k+1}$ is bounded by $\|e_k\|^2$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Such interpretation would mean that, with a few (small number of) iterates of linearly convergent algorithm A1 (random initialization assumed), smaller error would be achieved that with a few iterates of quadraticaly convergent algorithm A2. However, since the error diminishes, and due to squaring, later iterates would mean smaller error with A2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the above interpretation valid? Note that it disregards the rate coefficient $\lambda$. &lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="276" LastEditDate="2012-11-30T07:06:37.247" LastActivityDate="2012-11-30T07:06:37.247" Title="Understanding the &quot;rate of convergence&quot; for iterative methods" Tags="|iterative-method|convergence|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4767" PostTypeId="2" ParentId="4766" CreationDate="2012-11-28T18:38:04.240" Score="9" Body="&lt;p&gt;In practice, yes. While $e_k$ is still large, the rate coefficient $\lambda$ will dominate the error rather than the q-rate. (Note that these are &lt;em&gt;asymptotic&lt;/em&gt; rates, so the statements you linked to only hold for the limit as $k\to\infty$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for first order methods in optimization you often observe an initially fast decrease in error, which then levels out. For Newton's method on the other hand, it can take a while before superlinear (or quadratic) convergence kicks in (it's only locally superlinearly convergent after all). For that reason, it is common to either start with a few gradient steps to get going before switching to a Newton method, or use homotopy or quasi-Newton methods which behave as first order methods initially and turn into a Newton method as you approach the target.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2012-11-28T18:38:04.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4768" PostTypeId="2" ParentId="4765" CreationDate="2012-11-28T22:50:54.370" Score="4" Body="&lt;p&gt;The paper [1] gives an explicit construction of the Bernstein form of a set of  orthogonal polynomials on simplices based on Legendre polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.maths.dundee.ac.uk/tgoodman/rida.pdf&quot; rel=&quot;nofollow&quot;&gt;Farouki, R.T., Goodman, T.N.T and Sauer, T: &lt;em&gt;Construction of orthogonal bases for polynomials in Bernstein form on triangular and simplex domains&lt;/em&gt;, Computer Aided Geometric Design 20 (2003), 209-230, DOI: 10.1016/S0167-8396(03)00025-6&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2012-11-28T22:50:54.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4769" PostTypeId="2" ParentId="4765" CreationDate="2012-11-29T03:54:09.390" Score="3" Body="&lt;p&gt;I think Rob Kirby told me once that he had written something on using Bernstein polynomials for FEM. Take a look at his web site at Texas Tech (or now at Baylor).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-29T03:54:09.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4770" PostTypeId="2" ParentId="4766" CreationDate="2012-11-29T03:58:38.623" Score="12" Body="&lt;p&gt;In addition to Christian's answer, it's also worth noting that for linear convergence you have $e_{k+1} \le \lambda_1 e_k$ where you have $\lambda_1&amp;lt;1$ if the method converges. On the other hand, for quadratic convergence you have $e_{k+1} \le \lambda_2 e_k^2$ and the fact that a method converges does not necessarily imply that $\lambda_2$ must be smaller than one. Rather, the condition for convergence is that $\lambda_2 e_1&amp;lt;1$ -- i.e., that your starting guess is close enough. This is commonly observed behavior: that quadratically convergent algorithms need to be started &quot;close enough&quot; from the solution to converge whereas linearly convergent algorithms are typically more robust. This is another reason why one often starts with a few steps of a linearly convergence algorithm (e.g., the steepest descent method) before switching to more efficient ones (e.g., Newton's method).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-29T03:58:38.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4771" PostTypeId="2" ParentId="3206" CreationDate="2012-11-29T06:38:21.297" Score="5" Body="&lt;p&gt;Following aterrel's suggestion, you could use &lt;code&gt;pyximport&lt;/code&gt; to &lt;a href=&quot;http://docs.cython.org/src/userguide/tutorial.html#pyximport-cython-compilation-the-easy-way&quot;&gt;automatically compile the &lt;code&gt;random&lt;/code&gt; module&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pyximport&#xA;pyximport.install(pyimport=True)&#xA;&#xA;import random&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, this still will not make it as fast as it would be if you declared static types for the variables in Cython.&lt;/p&gt;&#xA;" OwnerUserId="3363" LastActivityDate="2012-11-29T06:38:21.297" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4772" PostTypeId="1" CreationDate="2012-11-29T06:55:33.347" Score="5" ViewCount="309" Body="&lt;p&gt;I've been doing FORTRAN programming for 10 years and I've started using python for a few years now, mostly for data processing.  I've been lucky enough to work with people that are knowledgeable python programmers and so I've learned quite a bit of practical skill. However, I feel like I'm rarely if ever thinking, if not in a pythonic way, at least in a truly OOP way, probably because I've had little formal training in OOP. Also I'm rarely doing such computationally intensive stuff that I feel optimization is critical but it has been a problem on a few search algorithms. Finally, to list a few specific packages that I'm actively using or would like to know more about. I'd say I know numpy and matplotlib fairly well. I've done a little bit of R but wouldn't mind a fully pythonic way like pandas or some of the scipy stuff (which I've used very little). Finally, GUI has never been necessary but why not if it makes other people more likely to use the tools...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So overall I feel like I'm missing something. I know the basics, I can understand most programs even with complex classes but when it's time to write my own classes, it feels clunky. Any good book/resource out there with a focus on science/engineering applications to help me think python? Like I tried to say, not looking for a reference book but more a python and science way of life. Anybody with similar background and/or experience?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will add below books that I'm considering:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://goo.gl/x8DKv&quot; rel=&quot;nofollow&quot;&gt;Python Scripting for Computational Science by Hans Petter Langtangen&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://goo.gl/SXO1l&quot; rel=&quot;nofollow&quot;&gt;Python for Data Analysis by Wes McKinney&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="95" LastEditorUserId="95" LastEditDate="2012-11-29T07:02:57.727" LastActivityDate="2012-11-29T08:46:35.757" Title="Recommendation for a python book for data processing" Tags="|python|education|programming-paradigms|data-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4773" PostTypeId="2" ParentId="4772" CreationDate="2012-11-29T08:46:35.757" Score="2" Body="&lt;p&gt;I haven't read Wes McKinney's book, but he's a smart guy and pandas is an awesome package. So I suspect it's good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility for a quick start is to just go through the &lt;a href=&quot;http://scikit-learn.org/stable/&quot; rel=&quot;nofollow&quot;&gt;scikits learn&lt;/a&gt; documentation and code. It's nice package with a consistent interface, good examples, and nicely structured code. It's not &quot;the answer&quot; for data analysis using python, but it's a good starting place. &lt;/p&gt;&#xA;" OwnerUserId="1420" LastActivityDate="2012-11-29T08:46:35.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4774" PostTypeId="1" AcceptedAnswerId="4777" CreationDate="2012-11-29T11:12:24.527" Score="3" ViewCount="100" Body="&lt;p&gt;I have two tables at my disposal, one work dataset and one reference dataset. Each dataset has got two columns, lets say these are fields A and B. I would like to associate the rows in the reference dataset with the rows in work dataset that are 'closest' w.r.t. some distance. I have more rows in the work dataset than in the reference dataset, and I will match all rows in the reference dataset to some rows in the work dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can define distance between two rows in the reference and work datasets as: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ d_{i,j} = (A_{w}(i) - A_{r}(j))^{2} + (B_{w}(i) - B_{r}(j))^{2} $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $A_{w}, A_{r}, B_{w}, B_{r} $ the A and B columns in the work and reference datasets with obvious notation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other word, I want to minimize over all permutations of rows from the work dataset (with the same number of rows as in the reference dataset) the sum of distances between one row in reference dataset and one row in work dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know how to proceed other than by brute-force examination of all permutations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is combinatorial. What about stochastic methods: genetic algorithm, swarm... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you hint at some idea for a start?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1184" LastEditorUserId="276" LastEditDate="2012-11-29T19:17:15.873" LastActivityDate="2012-11-29T19:17:15.873" Title="Improve optimization over 'mapping' of indices" Tags="|optimization|algorithms|machine-learning|complexity|combinatorics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4775" PostTypeId="1" CreationDate="2012-11-29T14:49:31.427" Score="3" ViewCount="1957" Body="&lt;p&gt;I'm trying to solve the steady state of a heat equation problem in 2D  $\Delta u = 0$ (3D also), with the method of solving the huge system of equations that arises from the discretization of the domain.&#xA;My boundary conditions are different from any example that I've been able to find.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If my domain is a 2D square, I want to put a fixed temperature (say $T_e$) all around the square edges. However, I want to force a point in the center of the square to be at a different temp from the edges, $T_c, T_c &amp;gt; T_e$.&#xA;Think of a square metal plate being heated at the center by a soldering iron.&#xA;I want to calculate the steady state heat in such a scenario.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To the best of my knowledge it seems that this problem is not really a heat equation problem: having a point at the center of the domain with a local maxima temperature goes against the properties of the heat equation...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm obtaining results, but it seems that heat does not propagate linearly from the center to the edges, but rapidly decaying. The problem is the same if moved into the 3D domain (a sphere/cube that is heated from it's center to its exterior edges, and my results are the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe I'm using a wrong model? or a wrong tool to solve this steady state problem?&#xA;Any hints or directions will be highly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards&lt;/p&gt;&#xA;" OwnerUserId="1228" LastActivityDate="2012-12-09T02:32:21.283" Title="Heat equation (steady state) with boundary conditions at domain edges and inside the domain" Tags="|pde|boundary-conditions|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4776" PostTypeId="2" ParentId="4775" CreationDate="2012-11-29T15:23:44.973" Score="5" Body="&lt;p&gt;You need to add the condition that $u(c) = T_c$ as an explicit constraint via a Lagrange multiplier. For simplicity, assume that $T_e=0$ (otherwise construct a $u_e$ such that $u_e|_{\partial\Omega} = T_e$ and add it to the solution with homogeneous boundary conditions, shifting the constraint by $T_c-u_e(c)$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You know that the solution of $-\Delta u = 0$ solves the minimization problem&#xA;$$&#xA;\min_{u\in H^1_0(\Omega)} \int_\Omega |\nabla u|^2 \,dx.&#xA;$$&#xA;To enforce the constraint $u(c)=T_c$, you consider the Lagrange functional&#xA;$$&#xA;\max_{\lambda\in\mathbb{R}} \min_{u\in H^1_0(\Omega)} \int_\Omega |\nabla u|^2 \,dx + \lambda(u(c)-T_c).&#xA;$$&#xA;The necessary optimality conditions for this problem are obtained (formally) by differentiating with respect to $u$ and $\lambda$:&#xA;\begin{align}&#xA;\int_\Omega \nabla u\cdot \nabla v + \lambda v(c)&amp;amp;= 0 \quad\text{for all } v\in H^1_0(\Omega)\\&#xA;(u(c) - T_c)\mu &amp;amp;=0 \quad\text{for all }\mu\in\mathbb{R}.&#xA;\end{align}&#xA;(Note here that except in 1D, functions in $H^1(\Omega)$ are not continuous and thus do not have point values. You probably should replace the point value with an average value over a small subdomain. For a finite element discretization, you move to a subspace of piecewise polynomials, where point values make sense.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For details on saddle point problems for partial differential equations, see &#xA;&lt;a href=&quot;http://homepage.ruhr-uni-bochum.de/dietrich.braess/FEM_corr_eng3.htm#NEW&quot;&gt;Dietrich Braess, Finite Elements, Cambridge University Press, 2007&lt;/a&gt;, Chapter III.4.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2012-11-29T15:23:44.973" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4777" PostTypeId="2" ParentId="4774" CreationDate="2012-11-29T15:49:09.727" Score="3" Body="&lt;p&gt;If I interpret the problem right, the task is to assign to each $j$ in the column index set $J$ of the distance matrix an index $x_j$ from the row index set $I$&#xA;such that the $x_j$ are all different and the $\sum_j d(x_j,j)$ is minimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be posed as a combinatorial constraint satisfaction problem to a number of constraint solvers. See, e.g., &lt;BR&gt;&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Constraint_programming&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Constraint_programming&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://argo.matf.bg.ac.rs/publications/2010/alldiff-smt2010.pdf&quot; rel=&quot;nofollow&quot;&gt;http://argo.matf.bg.ac.rs/publications/2010/alldiff-smt2010.pdf&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://www-circa.mcs.st-and.ac.uk/Preprints/alldiff-paper.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www-circa.mcs.st-and.ac.uk/Preprints/alldiff-paper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether the problem is tractable depends on thestructure and size of the distance matrix derived from the data sets.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-29T15:49:09.727" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4778" PostTypeId="2" ParentId="4766" CreationDate="2012-11-29T15:56:23.020" Score="6" Body="&lt;p&gt;The interpretation is qualitatively correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that linear and quadratic convergence are with respect to the worst case, the situation in a particular algorithm can be better than what you get from the worst case analysis given by Wolfgang Bangerth, though the qualitative situation usually corresponds to this analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In concrete algorithms (e.g., in optimization) it makes often sense to first iterate with a cheap but only linearly convergent method until progress gets slow, and then finish with a quadratically (or at least superlinearly) convergent method. In practice, superlinear convergence tends to be as good as quadratic convergence just because the initial, slowly converging part tends to dominate the overall work.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-29T15:56:23.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4781" PostTypeId="1" AcceptedAnswerId="4787" CreationDate="2012-11-29T20:22:24.610" Score="13" ViewCount="6684" Body="&lt;p&gt;I'm trying to solve the following system of equations for the variables $P,x_1$ and $x_2$ (all else are constants):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{A(1-P)}{2}-k_1x_1=0 \\ \frac{AP}{2}-k_2x_2=0 \\ \frac{(1-P)(r_1+x_1)^4}{L_1}-\frac{P(r_1+x_2)^4}{L_2}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can see that I can turn this system of equations into a single equation of a single variable $(P)$ by solving equations 1 and 2 for $x_1$ and $x_2$ respectively and substituting them into equation 3.  In doing so, I am able to use matlab's &lt;code&gt;fzero&lt;/code&gt; command to find the solution.  Using the parameters $k_1=k_2=1$, $r_1=r_2=0.2$, and $A=2$, I found the true solution to be $P=x_1=x_2=0.5$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I use newton's method applied to the original 3 variate - 3 equation system, the iterations never converge to the solution, no matter how close I begin to the true solution $x^*=(P^*,x_1^*,x_2^*)=(0.5,0.5,0.5)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first, I suspected my a bug in my implementation of newton's method.  After checking several times, I found no bug.  Then I tried using an initial guess $x_0=x^*$, and lo &amp;amp; behold: the Jacobian is singular.  I know that a singular jacobian can reduce the order of convergence, but I don't think it necessarily prevents convergence to the true solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is, Given that the jacobian of the system at the true solution is singular:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What other conditions are necessary to prove that newton's method will not converge to the root?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Would a globalization strategy (e.g. line-search) guarantee convergence despite the singular jacobian?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-12-07T14:46:30.710" Title="Strategies for Newton's Method when the Jacobian at the solution is singular" Tags="|optimization|convergence|newton-method|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4784" PostTypeId="1" CreationDate="2012-11-30T03:58:18.110" Score="2" ViewCount="2184" Body="&lt;p&gt;Does anyone know a good library that implements basic sparse matrix operations such as transpose, SpMV eigenvalues, etc. in GPU (cuda/OpenCL)?&lt;/p&gt;&#xA;" OwnerUserId="3360" LastEditorUserId="20688" LastEditDate="2024-02-10T00:36:03.577" LastActivityDate="2024-02-10T00:36:03.577" Title="Sparse Matrix Library for GPU" Tags="|sparse-matrix|gpu|software-recommendation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4785" PostTypeId="1" CreationDate="2012-11-30T11:28:13.370" Score="2" ViewCount="108" Body="&lt;p&gt;In my studies in computability I have come across the notion of the &quot;machine&quot;, an abstract representation of a device that essentially computes. I have read about Turing Machines and Wolfram's binary cellular automata and I understand them.ie. the rules, states, colours, etc. and how they work. What I really don't get however is how they are viewed as an abstraction of computing. Why are automata even used to represent computers?? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They begin in some state and run forever(unless a terminating state is specified). The critical point I'm trying to make here is that they receive no external input and are essentially functions of themselves:their current state is some function of its starting state.&lt;/p&gt;&#xA;" OwnerUserId="3368" LastActivityDate="2012-11-30T14:48:08.677" Title="How does an automaton actually &quot;compute&quot;?" Tags="|complexity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4786" PostTypeId="2" ParentId="4785" CreationDate="2012-11-30T12:17:44.970" Score="1" Body="&lt;p&gt;The Turing Machine is just a simple model of a computer. On the one hand one has only a restricted set of operations. On a modern PC, you have more operations. On the other hand one has infinite space and time, which is not true for a modern PC. &#xA;So, the goal is just, that you can analyze the Turing Machine easily. So you say, you could combine these simple operations to get the complex operations of modern PCs and you assume that memory limitations aren't an issue, which is true for &lt;strong&gt;most&lt;/strong&gt; software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then it's easy to analyze, wether an algorithm can run on a computer or whether a programming language is Turing-complete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can program a Turing Mashine, if you write some bits on this infinite large band of memory ;-). I think you will find some Turing simulators in the web or you watch this nice example video of a &lt;a href=&quot;http://aturingmachine.com/examples.php&quot; rel=&quot;nofollow&quot;&gt;counting program&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-11-30T12:17:44.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4787" PostTypeId="2" ParentId="4781" CreationDate="2012-11-30T14:38:54.947" Score="5" Body="&lt;p&gt;(1): This depends on the behavior of derivatives of the Jacobian (sic!) in the null space of the Jacobian at the solution. In practice, nobody calculates these derivatives, and i didn't even bother to remember the precise conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) works, though convergence is only linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get superlinear convergence (at least in the majority of cases), one may use tensor methods. See, e.g., &lt;BR&gt;&#xA;&lt;a href=&quot;https://cfwebprod.sandia.gov/cfdocs/CCIM/docs/SAND2004-1944.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://cfwebprod.sandia.gov/cfdocs/CCIM/docs/SAND2004-1944.pdf&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://www.jstor.org/stable/10.2307/2156931&quot; rel=&quot;noreferrer&quot;&gt;http://www.jstor.org/stable/10.2307/2156931&lt;/a&gt;&lt;BR&gt;&#xA;&lt;a href=&quot;http://www.springerlink.com/index/X5G827367G548327.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.springerlink.com/index/X5G827367G548327.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-30T14:38:54.947" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4788" PostTypeId="2" ParentId="4775" CreationDate="2012-11-30T14:41:48.203" Score="2" Body="&lt;p&gt;From a mathematical viewpoint you can't do what you are trying to do: the problem&#xA;$$&#xA;  \Delta u = 0 \qquad \text{in } \Omega, \\&#xA;  u = T_e \qquad \text{on } \partial\Omega, \\&#xA;  u(c) = T_c \qquad \qquad\qquad\text{}&#xA;$$&#xA;does not, in general, have a solution. This is because, mathematically speaking, the space of solutions of the Laplace equation, $H^1$, consists of functions that may not be continuous and so it doesn't make sense to take point values such as $u(c)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative explanation of this problem would be to realize that if $T_c&amp;gt;T_e$ then there will be heat flowing from $c$ to the boundary, and consequently someone will have to put that heat into the body at $c$. But putting a finite amount of heat into a single point (of area zero) means to have a device that can produce an infinite heat flux -- hardly realistic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to what you are trying to do is to assume that the area at which you are able to impose a temperature $T_c$ is finite, i.e., to consider a domain that has a hole of finite size at the center.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-11-30T14:41:48.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4789" PostTypeId="2" ParentId="4785" CreationDate="2012-11-30T14:48:08.677" Score="1" Body="&lt;p&gt;The simplest model of a computer consists of an automaton plus its tape(s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input is on the tape at the time the automaton starts. The program is in (i.e., defines) the automaton, or it is part of the input (for a universal Turing machine).And the output is on the tape at the time the automaton stops (if it stops). Not stopping also happens in real programs if there are infinite loops. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus a (general enough) automaton captures all features of a real program execution. Indeed, one can map any algorithm onto a Turing machine, and this gives the standard, mathematically rigorous definition of the term ''algorithm''. &lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2012-11-30T14:48:08.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4790" PostTypeId="2" ParentId="4775" CreationDate="2012-11-30T15:43:30.860" Score="1" Body="&lt;p&gt;I am not sure about interpreting $\nabla u=0$ as heat spreading, given that it corresponds to the steady state of the time dependent (parabolic) equation $u_t=\nabla u$. For me the meaning is rather closer to an interpolation of the values specified at the boundary of the domain of definition $\partial \Omega$. These values definitely determine the function $u$, so if they are set to zero on a ball of radius 1, the function is identically zero. This is consistent with the formula of the solution to $\nabla u=0$ with $u=f$ at the boundary, given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x)=\frac{-1}{\omega_n} \int_{|s|=1} \frac{-1+|x|^2}{|s-x|^n} f(s) ds, x \in \Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $| \cdot |$ the norm of the Euclidean space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, having an interior point with maximum value is against the maximum principle holding for all armonic functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, I am not sure if a boundary problem on a unit ball with constant temperature at its boundary and another temperature at an interior point it is well formulated for $\nabla u=0$. At least the interior point should be removed, so the domain becomes a sort of spherical band. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, given that the solution to $\nabla u=0$ depends on the function values set on the boundary of the domain and on the boundary shape itself, what are these in the case of the fundamental solution (given by a radial solution)?. Further, is there any categorization of generic (fundamental) solutions depending on boundary shapes and values?&lt;/p&gt;&#xA;" OwnerUserId="3369" LastEditorUserId="391" LastEditDate="2012-12-08T03:39:46.730" LastActivityDate="2012-12-08T03:39:46.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4791" PostTypeId="1" AcceptedAnswerId="4793" CreationDate="2012-11-30T19:48:51.100" Score="4" ViewCount="1660" Body="&lt;p&gt;I am trying to create a series of randomly distributed spheres in Matlab, three will be three sub-sets. I would like each subset to have a different color. I am using a structure because each sphere will also have different properties associated with it. I tried using colormap, and set(...'FaceColor'..) but am having some difficulty. Please Help. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for n = 1:100&#xA;    Bslice.cell(n).index = n; &#xA;    Bslice.cell(n).type = 'Tyep1'; % Type2, Type3&#xA;     Bslice.cell(n).location = round(rand(1, 3)*10);&#xA;end &#xA;&#xA;%%&#xA;[x,y,z] = sphere;&#xA;&#xA;for n = 1:10&#xA;    hold on &#xA;    grid on&#xA;    surfl(x-Bslice.cell(n).location(1), y-Bslice.cell(n).location(2), z-        Bslice.cell(n).location(3), ones(size(); &#xA;    shading interp&#xA;    % colormap(hot(100))&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3371" LastActivityDate="2012-11-30T21:55:50.733" Title="Creating spheres In Matlab that change surface color depending data" Tags="|matlab|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4793" PostTypeId="2" ParentId="4791" CreationDate="2012-11-30T21:55:50.733" Score="1" Body="&lt;p&gt;Let me be clear, &lt;code&gt;FaceColor&lt;/code&gt; assigns a specific &lt;em&gt;single&lt;/em&gt; color to every surface in that &lt;code&gt;patch&lt;/code&gt; object drawn by &lt;code&gt;surfl&lt;/code&gt;, and you would lose the lighting info. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to modify &lt;code&gt;FaceColor&lt;/code&gt; for each ball, you could add a &lt;code&gt;color&lt;/code&gt; to indicate the color(here I use the distance to the origin), rescale it to an RGB vector(also I assume &lt;code&gt;Bslice.cell(n).location&lt;/code&gt; is the direction of the lighting source in &lt;code&gt;surfl&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;close all;&#xA;for n = 1:100&#xA;    Bslice.cell(n).index = n; &#xA;    Bslice.cell(n).type = 'Tyep1'; % Type2, Type3&#xA;    Bslice.cell(n).location = round(rand(1, 3)*10);&#xA;    color(n) = sqrt(sum(Bslice.cell(n).location.^2));&#xA;end &#xA;color = repmat(color,3,1)/max(color);&#xA;&#xA;[x,y,z] = sphere;&#xA;&#xA;for n = 1:10&#xA;    hold on &#xA;    grid on&#xA;    h(n) = surfl(x-Bslice.cell(n).location(1), ...&#xA;          y-Bslice.cell(n).location(2), ...&#xA;          z-Bslice.cell(n).location(3), ...&#xA;          Bslice.cell(n).location); &#xA;set(h(n),'FaceColor',color(:,n))&#xA;view(3);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7u1IE.png&quot; alt=&quot;sphere1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you mean for each set you are using a different &lt;code&gt;colormap&lt;/code&gt; like this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/RDch8.png&quot; alt=&quot;sphere2&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/X5vYE.png&quot; alt=&quot;sphere3&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/lUbfI.png&quot; alt=&quot;sphere4&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2012-11-30T21:55:50.733" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4794" PostTypeId="2" ParentId="4775" CreationDate="2012-12-01T07:54:05.897" Score="2" Body="&lt;p&gt;Based on some of the previous answers, I think you are faced with a few choices in your modeling. If you are forced to use a cartesian unskewed grid (not clear from your question what discretization you are using), it's going to be a little tougher because as said previously, you need to cut off a finite area of a few cells in the center and you might get corner issues once you try to solve this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can do arbitrary grids, may I suggest a circular one (not a square) which means you can use the symmetry properties to reduce the dimensionality of your problem. You then can compare your results with an analytical solution of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T(r) = (T_1 - T_0)\frac{\ln{\frac{r}{r_0}}}{\ln{\frac{r_1}{r_0}}}+ T_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And you can see that as $r_0 \rightarrow 0$, the temperature at the origin becomes undefined because your heat flux (positive or negative) would become infinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TnkV7.png&quot; alt=&quot;One solution&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="95" LastEditorUserId="95" LastEditDate="2012-12-09T02:32:21.283" LastActivityDate="2012-12-09T02:32:21.283" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4795" PostTypeId="2" ParentId="593" CreationDate="2012-12-01T07:57:45.220" Score="1" Body="&lt;p&gt;Of interest may be the Date Compensated Discrete Fourier Transform:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://adsabs.harvard.edu/cgi-bin/bib_query?1981AJ.....86..619F&quot; rel=&quot;nofollow&quot;&gt;Ferraz-Mello, S., 1981, &lt;em&gt;Estimation of periods from unequally spaced observations&lt;/em&gt;, The Astronomical Journal, 302:757-763&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="3375" LastEditorUserId="1804" LastEditDate="2012-12-01T09:56:41.393" LastActivityDate="2012-12-01T09:56:41.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4796" PostTypeId="1" CreationDate="2012-12-01T21:18:02.487" Score="19" ViewCount="8545" Body="&lt;p&gt;In programming dense matrix computations, is there any reason&#xA;to choose a row-major layout of the over the column-major layout?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that depending on the layout of the matrix chosen, we need to write the appropriate code to use the cache memories effectively for speed purposes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The row-major layout seems more natural and simpler (at least to me). But major libraries like LAPACK which are written in Fortran use the column major layout, so there must be some reason for having made this choice.  &lt;/p&gt;&#xA;" OwnerUserId="3376" LastActivityDate="2019-01-01T17:10:27.090" Title="Row major versus Column major layout of matrices" Tags="|matrix|fortran|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4797" PostTypeId="2" ParentId="4796" CreationDate="2012-12-01T22:53:13.487" Score="19" Body="&lt;p&gt;The column major layout is the scheme used by Fortran and that's why it's used in LAPACK and other libraries.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general it is much more efficient in terms of memory bandwidth usage and cache performance to access the elements of an array in the order in which they're laid out in memory.  Depending on how your matrices are stored, you'll want to pick algorithms that take advantage of this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/hHZ2I.png&quot; alt=&quot;internal storage&quot;&gt; &#xA;Internal storage of column major format&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="22284" LastEditDate="2019-01-01T17:10:27.090" LastActivityDate="2019-01-01T17:10:27.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4798" PostTypeId="2" ParentId="4796" CreationDate="2012-12-02T02:52:13.970" Score="11" Body="&lt;p&gt;In vacuum without considering any existing software, there's no reason to prefer column major over row major from the code point of view. However, most mathematical literature is written in a way that groups vectors into a matrix by storing them as columns instead of rows. For example when you write the full eigenvalue equation $AX=X\Lambda$, the $X$ matrix contains all the eigenvectors written out in columns. You never really see it written the other way (though I hear that statistics folks like row vectors). Therefore, it was natural that the earliest software assumed column major format so that if you have a matrix which is a set of vectors, the storage of any single vector is contiguous. Thus, I imagine that tradition has just been carried forward to the present day, and if you want to interact with the ye olde Fortran, you want to use column major. So pretty much all highly efficient numerical linear algebra is done in column major.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason C is row major is somewhat of a consequence of its array syntax; you declare a 3 row by 2 column array as &lt;code&gt;double a[3][2]&lt;/code&gt;, and later indices vary faster than earlier indices, which for 2D arrays makes it row major. Combine this with the natural Western reading order from left to right, it makes row major seem more natural.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-12-02T02:52:13.970" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4800" PostTypeId="2" ParentId="4784" CreationDate="2012-12-03T05:41:17.217" Score="3" Body="&lt;p&gt;Check &lt;a href=&quot;http://viennacl.sourceforge.net/viennacl-about.html&quot; rel=&quot;nofollow&quot;&gt;ViennaCL&lt;/a&gt;. I use that library in many projects to improve performance of sparse operations and I am very pleased with the results. Remember that using GPU makes sense only for relatively large size of the jobs. Otherwise, it is not cost-effective.&lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2012-12-03T05:41:17.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4801" PostTypeId="1" CreationDate="2012-12-03T20:46:47.447" Score="1" ViewCount="182" Body="&lt;p&gt;I could not get accurate description that aids to my understanding of inner blocking size parameter that is used in many LAPACK routines(such as DGEQRT, DLARFB). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2012-12-05T17:14:34.357" Title="Understanding Inner blocking size in lapack/plasma" Tags="|lapack|blas|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4802" PostTypeId="2" ParentId="4801" CreationDate="2012-12-03T21:20:10.543" Score="1" Body="&lt;p&gt;You achieve ideal blocking, if your block fits in your cache. If it does not, it will cost you way more performance. So you have to fit the block size to cache size.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-12-03T21:20:10.543" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4803" PostTypeId="1" AcceptedAnswerId="4804" CreationDate="2012-12-04T09:21:47.530" Score="5" ViewCount="232" Body="&lt;p&gt;First of all, I am quite new to this field and I excuse myself in advance for any stupid content in this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the field of compressed sensing or deblurring I have a nonlinear optimization problem of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min R(x)$ s.t. $Ax-b&amp;lt;\epsilon$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I solve the optimization problem using the Langrangian form and the non-linear conjugated gradient method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x) = |Ax-b|^2 +\lambda R(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the gradient&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla f(x) = 2A^TAx-A^Tb +\lambda \nabla R(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax$ and $A^Tx$ are expensive to evaluate since they contain a non-uniform FFT. However, I am able to evaluate $A^TAx$ quite fast on a cartesian grid by convolution with the point spread function of $A^TA$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way I am able to calculate the gradient fast, but for the line search of the nonlinear CG I still have to evaluate $Ax$ frequently. I am looking for a nonlinear method that only needs the evaluation of $A^TAx$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a first attempt, somehow similar to linear CGNR, I tried to minimize the residual of the normal equations by using the following (in the line search only):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f2(x) = |A^TAx-A^Tb|^2 +\lambda R(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without $\lambda Rx$ this method obviously reduces to linear CGNR (with an unnecessary line-search).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method seems to converge to a solution. However, it converges to a different solution than the nonlinear CG, since the constant value of $\lambda$ needs to be adapted .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is this total nonsense what I tried? ;-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is there something like a &quot;nonlinear CGNR&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Or some other suitable method that solves my nonlinear optimization problem using only $A^TA$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) What is the gradient of $f2x()$?&lt;/p&gt;&#xA;" OwnerUserId="2026" LastEditorUserId="2026" LastEditDate="2012-12-06T16:39:12.693" LastActivityDate="2012-12-06T17:02:23.067" Title="Is there a nonlinear solver similar to CGNR evaluating only $A^TAx$?" Tags="|nonlinear-programming|conjugate-gradient|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4804" PostTypeId="2" ParentId="4803" CreationDate="2012-12-04T13:43:14.810" Score="3" Body="&lt;p&gt;Your problem is equivalent to minimizing $f(x)=x^T(Bx-c)+\lambda R(x)~~~~$&#xA;where $B=A^TA$, and $c=2A^Tb~~$. The gradient is $g(x)=2Bx-c+\lambda\nabla R(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus if you precompute $c$, any gradient-based method with line search only needs multiplications with $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit] In the complex case, you have $f(x)=x^*Bx-Re~(x^*c)+\lambda R(x)~~~~~$&#xA;where $B=A^*A$, and $c=2A^*b~~$, and things are essentially as in the real case.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2012-12-06T17:02:23.067" LastActivityDate="2012-12-06T17:02:23.067" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4805" PostTypeId="1" AcceptedAnswerId="4810" CreationDate="2012-12-04T19:53:08.227" Score="7" ViewCount="3883" Body="&lt;p&gt;I have been slowly learning C++ and ROOT for over a year now, in order to debug a program made with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I reached a point where I can really understand that this will never become easier. Therefore I'm looking for some advices in alternatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I need is something that can do, plots, histogram, 2D and 3D, and that has good minimization routines, and specially, fast learning curve and easy to write and debug.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also need it do be fast, as fore example I now need run to over 600k fits, this takes around 1h30. But I spend &lt;strong&gt;far&lt;/strong&gt; more time writing it, so if I double the running time and cut in half the programming time, this will be worth the effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the moment I know, Matlab/octave and C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="3382" LastActivityDate="2022-12-02T23:01:03.790" Title="ROOT (CERN C++ Libraries) alternatives" Tags="|c++|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4806" PostTypeId="2" ParentId="4805" CreationDate="2012-12-04T20:13:10.057" Score="0" Body="&lt;p&gt;Have you tried R: &lt;a href=&quot;http://www.r-project.org&quot; rel=&quot;nofollow&quot;&gt;http://www.r-project.org&lt;/a&gt; ? I don't know how fast it is ... and I don't know whether it has minimization routines ...&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-12-04T20:13:10.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4807" PostTypeId="2" ParentId="4805" CreationDate="2012-12-04T22:08:49.573" Score="10" Body="&lt;p&gt;You might want to look at Python with &lt;a href=&quot;https://numpy.org/doc/stable/reference/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NumPy&lt;/a&gt;/&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciPy&lt;/a&gt;. (There are many more packages available; for example, &lt;a href=&quot;https://pandas.pydata.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;pandas&lt;/a&gt; is a dedicated data analysis library).&#xA;The syntax is somewhere between Matlab and C and so should not be too hard to pick up. There's a helpful &lt;a href=&quot;https://numpy.org/devdocs/user/numpy-for-matlab-users.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NumPy for Matlab users&lt;/a&gt; page, too. One of the advantages is that it is fairly easy to &amp;quot;outsource&amp;quot; parts of the code to C/C++, e.g., using &lt;a href=&quot;https://cython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cython&lt;/a&gt;. This means you can prototype and then profile your algorithm, and only move the most compute-intensive routines to C, keeping all the input/output and driver routines in Python.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="22284" LastEditDate="2022-12-02T23:01:03.790" LastActivityDate="2022-12-02T23:01:03.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4810" PostTypeId="2" ParentId="4805" CreationDate="2012-12-05T14:32:00.170" Score="6" Body="&lt;p&gt;Well... for tools used by the particle physics community there's &lt;a href=&quot;http://paw.web.cern.ch/paw/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAW&lt;/a&gt; (part of &lt;a href=&quot;http://cernlib.web.cern.ch/cernlib/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CERNLIB&lt;/a&gt;, the thing root replaced and no longer supported except as legacy) and Physica (which I haven't seen in years and can't even find a website for), which are probably not what you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you have been working in the cint interface to root your might consider the Python interface before changing tools completely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can go outside of the particle physics community as suggest by &lt;a href=&quot;https://scicomp.stackexchange.com/a/4806/183&quot;&gt;vanCompute&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/a/4807/183&quot;&gt;Christian&lt;/a&gt;. The big risk here--if you are doing nuclear or particle physics--is that your colleagues won't be able to help you with any problems you encounter and may not understand what the tool has done for you making talks a little more awkward. I've seen a fair number of questions in talks answered with &lt;em&gt;&quot;I used the Foo facility of root&quot;&lt;/em&gt; to everyone's satisfaction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes--when what you want to do is really unusual--it is worth taking the processing away from the black box: code it all by hand where you have complete control (and write a test suite as you go, because you need to be able to show that it is correct!) and only go back to the package for plotting. But as with all roll-your-own solution this is not a step to be taken lightly.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-12-05T14:32:00.170" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4811" PostTypeId="2" ParentId="4733" CreationDate="2012-12-05T15:00:03.897" Score="3" Body="&lt;p&gt;I know this question is already answered, but here's a similar single-face based looping storage which is implemented in the OpenFOAM C++ library: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each cell has an index (ID) in a cellList. &#xA;Two lists are defined for all faces: &quot;face internal owner&quot; and &quot;face neighbour&quot;.&#xA;The length of both face lists corresponds to the number of internal faces in the mesh. &#xA;A face owner will be the cell with the lower ID in the cellList (opposite for face neighbour). &#xA;Boundary faces are written last, and they have outward oriented normals (from the solution domain), and of course, only one owner cell. &#xA;The face area normal is oriented so that it looks outwards from the owner cell to the neighbour cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works well for e.g. flux calculation. The flux is evaluated once per face, and it is added to the sum of total faces for the owner cells, and deducted from the neighbour cells (the summation/deduction is decided based on the orientation of the face area normal). The boundary faces are sorted and stored at the bottom of the face list, allowing boundary conditions to be defined as slices of the face list (begining label, end label of the boundary patch),simplifying thus the implementation of the boundary conditions, as well as inhancing efficiency of the updating process for the boundary conditions, since it is relying on the solution provided by the operations on internal faces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the boundary faces are agglomerated into patches, the inter-process communication is defined for coupled (processor) patches, and pre-defined. This means that as soon as there is a loop over the boundary mesh, the top level access functions envoke wrapped MPI calls, making such code &quot;automatically&quot; parallelized, if it relies on the above explained face-based connectivity.&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2012-12-06T11:18:10.620" LastActivityDate="2012-12-06T11:18:10.620" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4812" PostTypeId="1" AcceptedAnswerId="11116" CreationDate="2012-12-05T22:37:03.600" Score="15" ViewCount="4683" Body="&lt;p&gt;I'm trying to simulate basic semiconductor models for pedagogical purposes--starting from the Drift-diffusion model. Although I don't want to use an off-the-shelf semiconductor simulator--I'll be learning other (common, recent or obscure) models, I do want to use an off-the-shelf PDE solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even for the simple 1D case, the drift-diffusion model consists of a number of coupled nonlinear PDEs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current density equations&#xA;$$J_n = q n(x) \mu_n E(x) + qD_n \nabla n$$&#xA;$$J_p = q p(x) \mu_p E(x) + qD_p \nabla p$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuity equation&#xA;$$\frac{\partial{n}}{\partial{t}} = \frac{1}{q} \nabla \cdot J_n + U_n $$&#xA;$$\frac{\partial{p}}{\partial{t}} = \frac{1}{q} \nabla \cdot J_p + U_p $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Poisson equation&#xA;$$\nabla \cdot (\epsilon \nabla V) = -(p - n + N_D^+ - N_A^-) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a number of boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried some python FEM solvers, &lt;a href=&quot;http://www.fenicsproject.org&quot;&gt;FEniCS/Dolfin&lt;/a&gt; and &lt;a href=&quot;http://www.sfepy.org&quot;&gt;SfePy&lt;/a&gt;, but with no luck, due to being unable to formulate them in the weak variational form with test functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is of course the option of implementing the numerical solution from scratch but I haven't studied FEM/Numerical in depth yet, so I hope it's not my only option as I don't want to be overwhelmed with numerical issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is there a package (pref. open source) that would take these equations, in that form, and solve them? Or perhaps is the variational form required by the tools is not as hard? In any case, what are my options?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit: Attempt of formulating the weak variational form for FEniCS/Dolfin or SfePy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using three PDEs (Poisson + two continuity equations with J substituted), we are looking for V, n, and p. The Poisson equation (using a test function $u_V$) is straight forward. I'm having difficulty, however, with the continuity equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second PDE (strong form) $$\frac{\partial{n}}{\partial{t}} = \nabla \cdot (C_1 n \nabla V +C_2\nabla n) + U $$ &#xA;where  $C_1, C_2$ are constants, $U, n, p, V$ are scalar functions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $f_n$ denote a test function for the second PDE. Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Omega}f_n\frac{n - n_1}{\Delta t}d\Omega - C_1 \int_{\Omega}f_n \nabla \cdot ( n \nabla V) d\Omega - C_2 \int_{\Omega}f_n \nabla^2 n d\Omega - \int_{\Omega}f_n U d\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially worrying is the integral:&#xA;$$C_1 \int_{\Omega} f_n \nabla \cdot ( n \nabla V) d\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But $\mathbf{\nabla V}$ is a vector, and $V, u_n, n$ are scalars. Then using the identity&#xA;$\nabla \cdot \phi \mathbf{A} = \mathbf{A} \cdot \mathbf{\nabla \phi} + \phi \nabla \cdot \mathbf{A}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C_1 \int_{\Omega} f_n \nabla \cdot ( n \nabla V) d\Omega = C_1 \int_{\Omega} f_n (\nabla V \cdot \nabla n) +  C_1 \int_{\Omega} f_n n \nabla \cdot \nabla V$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since V is solved by Poisson equation, can we use the recently computed value as allowed in software Dolfin/FEniCS and simplify how we treat V in this second coupled equation? These sort of techniques work in while discretizing (e.g. Gummel, ...), which I don't do in these ready solvers!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the boundary conditions are given in terms of $J_n$ not $n$, how do you implement this? Should I solved for the five variables $J_n, J_p, n, p, V$, even though $J_n$ is determined by V and n?&lt;/p&gt;&#xA;" OwnerUserId="3390" LastEditorUserId="3390" LastEditDate="2012-12-07T18:45:58.303" LastActivityDate="2014-03-17T15:27:19.060" Title="PDE solvers for Drift-diffusion and related models" Tags="|pde|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4813" PostTypeId="1" AcceptedAnswerId="4817" CreationDate="2012-12-06T00:37:57.523" Score="8" ViewCount="1436" Body="&lt;p&gt;I have the following linear program:&#xA;$$&#xA;\begin{array}{cc}&#xA;\text{Maximize}   &amp;amp; a^T x \\&#xA;\text{Subject to} &amp;amp; x_{\min} \leq x \leq x_{\max} \\&#xA;                  &amp;amp; \mathbf{1}^T x = 1 &#xA;\end{array}&#xA;$$&#xA;where $x \in \mathbb{R}^n$, $\mathbf{1}^T x$ denotes the sum of entries of $x$,&#xA;and $a$ is known and has distinct strictly positive entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a quick way to solve the above without using an LP solver. Is there a fast procedure to follow? (besides Simplex).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you! &lt;/p&gt;&#xA;" OwnerUserId="1932" LastActivityDate="2012-12-07T19:12:12.377" Title="Max of a convex combination over a convex hull of real variables" Tags="|optimization|convex-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4814" PostTypeId="2" ParentId="4813" CreationDate="2012-12-06T01:37:32.713" Score="7" Body="&lt;p&gt;The greedy algorithm works: start with some solution that satisfies your constraints, and then iteratively increase the $x_i$ with largest $a_i$ that isn't at $x_{\max}$ and shrink the $x_j$ with smallest $a_j$ that isn't at $x_{\min}$ as far as possible.  Stop when no further pair motions are possible.  This takes $O(n \log n)$ to sort $a$ and $O(n)$ time for the rest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the algorithm completes, there will be a set of $x_i$'s stuck at $x_{\max}$ (those with large $a_i$), a set of $x_j$'s stuck at $x_{\min}$ (those with small $a_j$), and at most one $x_k$ with $x_{\min} &amp;lt; x_k &amp;lt; x_{\max}$ and $a_i &amp;gt; a_k &amp;gt; a_j$.  A differential change to $x$ starting from such a position cannot increase the objective, since any gains due to increasing $x_j$ or $x_k$ would be more than offset by losses due to decreasing $x_i$ or $x_k$.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2012-12-06T21:49:30.470" LastActivityDate="2012-12-06T21:49:30.470" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4815" PostTypeId="1" CreationDate="2012-12-06T10:07:57.093" Score="4" ViewCount="185" Body="&lt;p&gt;I have a list of numbered elements 1 to N that fit into positions on a number line starting with 1. I also have constraints for these elements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The element 1 is in position 1, and element N must be at a position &gt;= the position of element N-1. (i.e. element 2 could be at position 1, element 3 at position 7, and element 4 at position 8 (but not position 5))&lt;/li&gt;&#xA;&lt;li&gt;Some elements must be within a certain distance from each other on the line.&lt;/li&gt;&#xA;&lt;li&gt;Some elements must be at least a certain distance from other on the line.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My objective is to return an integer that represents the maximum span between element 1 and element N. If no lineup is possible, return -1, and if the elements can be any distance apart, return -2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am given:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The number of elements&lt;/li&gt;&#xA;&lt;li&gt;A withinArray[][] where withinArray[x][y] = the distance elements x and y must be within on the line. Any zero values represent no constraints.&lt;/li&gt;&#xA;&lt;li&gt;An atLeastArray[][] where atLeastArray[x][y] = the distance elements x and y must be apart on the line. Any zero values represent no constraints.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An example input would be: 4 elements, withinArray&lt;a href=&quot;https://stackoverflow.com/questions/13714903/find-maximum-distance-between-elements-given-constraints-on-some&quot;&gt;1&lt;/a&gt;[3] = 10, withinArray[2][4] = 20, and atLeastArray[2][3] = 3. (all other array values are zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The return value for this input would be 27. (element 1 at position 1, element 2 at position 8, element 3 at position 11, and element 4 at position 28)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem was first posted &lt;a href=&quot;https://stackoverflow.com/questions/13714903/find-maximum-distance-between-elements-given-constraints-on-some&quot;&gt;here&lt;/a&gt; by someone else. I'd like to figure out an elegant solution to it programmatically. Though I've been working on it for a whole day, I still have no luck coming up with a good solution. I feel that I need to use dynamic programming techniques, but have a hard time finding a good substructure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not able to locate the same example on the web. Any pointer to such materials online would be appreciated. It's even better if you are an expert for such kind of question and can outline the solution in detail. Executable code is a plus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: I know how to formulate this as a linear programming problem and I know how to solve it using LP solver. I just prefer an end-to-end solution with input and output as specified. General answers might not help me much. &lt;/p&gt;&#xA;" OwnerUserId="3392" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-12-09T06:33:23.113" Title="Find maximum distance between elements given constraints on some" Tags="|optimization|linear-programming|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4816" PostTypeId="1" CreationDate="2012-12-06T20:36:16.107" Score="1" ViewCount="331" Body="&lt;p&gt;I have a data-set that has four columns [X Y Z C]. I would like to find all the C values that are in a given sphere centered at [X, Y, Z] with a radius r. What is the best approach to address this problem? Should I use the clusterdata command?&lt;/p&gt;&#xA;" OwnerUserId="3371" LastActivityDate="2012-12-07T00:04:18.913" Title="Clustering data based on the radius of a sphere" Tags="|matlab|statistics|data-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4817" PostTypeId="2" ParentId="4813" CreationDate="2012-12-06T22:04:09.633" Score="5" Body="&lt;p&gt;The optimal solution is the following: Set all variables equal to their minimums.  Then, starting from the largest $a_i$ to the smallest, iteratively set the corresponding $x_i$ as large as possible until you hit $\sum_i x_i = 1$.  If $\sum_i x_{i,min} &amp;gt; 1$ or $\sum_i x_{i,max} &amp;lt; 1$ then the problem is infeasible.  I believe that this is the same solution that Geoffrey Irving's algorithm outputs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason this works is that you can transform your problem into the LP relaxation of the 0-1 knapsack problem via $$y_i = \frac{x_i - x_{i,min}}{x_{i,max} - x_{i,min}}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In $y$-variable space, the problem becomes&#xA;\begin{array}{cl}&#xA;\text{Maximize}   &amp;amp; \sum_i c_i y_i \\&#xA;\text{Subject to} &amp;amp; 0 \leq y_i \leq 1, \text{ for each } i \\&#xA;                  &amp;amp; \sum_i b_i y_i = d, &#xA;\end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c_i = a_i (x_{i,max} - x_{i,min})$, $b_i = (x_{i,max} - x_{i,min})$, and $d = 1 - \sum_i x_{i,min}$.  If the original problem is feasible then $d \geq 0$.  The $c_i$'s and $b_i$'s are nonnegative, so we do have the LP relaxation of 0-1 knapsack.   (The expression $\sum_i a_i x_{i,min}$ technically appears in the objective, too, but since it is a constant we can drop it.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the variables are sorted by the ratio $\frac{c_i}{b_i} = a_i$ from largest to smallest, the known optimal solution is the greedy one: Set $y_1 = y_2 = \cdots = y_k = 1$ for as large a $k$ as possible, set $y_{k+1} = d - \sum_{i=1}^k b_i$, and set $y_{k+2} = \cdots = y_n = 0$.  Transforming this solution back into the $x$-variable problem space gives the solution I just described.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, 0-1 knapsack does have a $\leq$ constraint rather than an $=$ constraint.  If you can fit all the items in the knapsack with space left over, then the original $x$-variable problem is infeasible because $\sum_i x_{i,max} &amp;lt; 1$.&lt;/p&gt;&#xA;" OwnerUserId="657" LastEditorUserId="657" LastEditDate="2012-12-07T19:12:12.377" LastActivityDate="2012-12-07T19:12:12.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4818" PostTypeId="2" ParentId="4816" CreationDate="2012-12-07T00:04:18.913" Score="1" Body="&lt;p&gt;To collect the required C values I'd make a single pass through the data-set (array?) with some quick checks to eliminate most of the points outside the sphere, and then (only for the points that seem eligible) a check of actual radius.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, for a candidate point &lt;code&gt;(x_i,y_i,z_i,c_i)&lt;/code&gt;, test:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1) Is x_i between X-r and X+r?&#xA;2) If so, is y_i between Y-r and Y+r?&#xA;3) If so, is z_i between Z-r and Z+r?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Only if all three tests are true is it possible for the point to be in the desired sphere.  For the final test, use precomputed r^2 and ask:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4) Is (x_i - X)^2 + (y_i - Y)^2 + (z_i - Z)^2 less than or equal r^2 ?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the point passes the final test, include &lt;code&gt;c_i&lt;/code&gt; in your collected C values.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2012-12-07T00:04:18.913" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4819" PostTypeId="1" AcceptedAnswerId="4821" CreationDate="2012-12-07T05:05:08.223" Score="6" ViewCount="278" Body="&lt;p&gt;I would like to know about the effect of using a higher order interpolator for the methods of characteristics.  I am solving $$u_t+a(x,t)u_x=0$$ with some nonsmooth initial data $u_0(x)$ by the method of characteristics. It seems to be a great method, since I need not deal with any of the difficulties that arise from trying to solve this problem discretely by finite differences. The numerical solution will not have any oscillations at all! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, that is for the case of &quot;infinitely-refined grid&quot; that is, if I know value of the function at any value of $x$ at the previous time step. Instead, consider a grid $x_i:i\in\{1,N\}$ and march forward in time. Since the characteristics don't cross the nodes perfectly I have to interpolate. I can choose linear interpolation but would like to take a higher order to increase the accuracy of interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume I choose a piecewise quadratic polynomial.  Then the value at the interpolated point might be higher or smaller than the value at the end points of the interval and thus the extrema can get magnified from step to step. I can artificially restrict the interpolated value to be between $\min_{x\in I} u$ and $\max_{x\in I} u$ to prevent such growth (where $I$ is the interval the characteristic crosses at the previous known time). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two questions I would like to know the answer to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Looks like if I take higher order than linear interpolator, methods of characteristics lose stability as the value can grow because the maximum or minimum of the parabola that I am fitting can go outside of the range of the endpoint values (Assume I have something like $\sin(x)$) as the initial profile? So it is not as effective in this case and if I keep the interpolation linear I can't do better than first order convergence.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does the artificial restriction on the range of values affect the accuaracy of quadratic interpolation? If before the restriction the accuracy is second order, do I lower that by &quot;squaeezing&quot; the value to be between max and min value over the interval? I WOULD LIKE TO KNOW IF I CAN DO METHOD OF CHARACTERISTICS WITH SECOND ORDER ACCURACY?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="123" LastEditDate="2012-12-08T02:11:04.610" LastActivityDate="2012-12-08T02:11:04.610" Title="interpolation combined with methods of characteristics can cause oscillations for the transport equation?" Tags="|pde|hyperbolic-pde|stability|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4820" PostTypeId="2" ParentId="4781" CreationDate="2012-12-07T14:46:30.710" Score="4" Body="&lt;p&gt;Instead of a line-search you could try modifying your Newton's method to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm&quot; rel=&quot;nofollow&quot;&gt;Levenberg-Marquardt algorithm&lt;/a&gt;. The implementation is simple enough if you're using Matlab.&lt;/p&gt;&#xA;" OwnerUserId="1126" LastActivityDate="2012-12-07T14:46:30.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4821" PostTypeId="2" ParentId="4819" CreationDate="2012-12-08T02:07:33.477" Score="6" Body="&lt;p&gt;This is an important and challenging issue.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, using quadratic interpolation means that your solution values may lie outside the interval in which the initial data lie.  This is not what we usually mean when we refer to numerical instability, but it is a potentially undesirable feature.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, forcing the interpolated values to lie in an interval destroys the accuracy of your solution.  Nevertheless, it is a commonly-used technique.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One may use slope-limiter methods or ENO/WENO interpolants to mitigate this problem, but with either of those approaches, you still have to give up either the formal 2nd-order accuracy or the strict maximum principle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One very recent proposed approach that claims to give high order accurate, data-bounded interpolants is in &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.190.218&amp;amp;rep=rep1&amp;amp;type=pdf&quot;&gt;this paper of Martin Berzins&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you want to do can also be done if you advect cell averages instead of point values; see &lt;a href=&quot;http://math.mit.edu/~zhangxx/thesis.pdf&quot;&gt;Xiangxiong Zhang's thesis&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A final remark, unrelated to the focus of your question: for general functions $a(x,t)$, you'll need to use a Newton iteration to implement the method of characteristics.  This is a major drawback compared to traditional finite difference methods.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-12-08T02:07:33.477" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4822" PostTypeId="1" CreationDate="2012-12-08T12:33:20.073" Score="10" ViewCount="275" Body="&lt;p&gt;I am searching for the most important literature and slide references for modeling current and future energy costs of floating-point operations and data transfers across the CPU, memory, network, and storage.  I have marked this question as a community wiki, and I would prefer that you limit each answer to the following format:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Title, Authors, Location/Conference/Journal, Year, DOI/URL, Summary of what information the paper or presentation provides relevant to my request.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2015-09-22T14:40:14.977" Title="Literature references for modeling current and future energy costs of floating-point operations and data transfers" Tags="|performance|architecture|exascale|" AnswerCount="3" CommentCount="1" CommunityOwnedDate="2012-12-08T12:33:20.073" ContentLicense="CC BY-SA 3.0" />
  <row Id="4824" PostTypeId="1" CreationDate="2012-12-09T06:32:41.010" Score="5" ViewCount="979" Body="&lt;p&gt;I tried to search the answer and I found that method cs_multiply from &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898716136&quot; rel=&quot;noreferrer&quot;&gt;this book&lt;/a&gt; has been adopted for the purpose of multiplication of two general sparse matrices in MATLAB. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the book it says that &#xA;&quot;The time taken by this method &lt;code&gt;(A * B)&lt;/code&gt; is &lt;code&gt;O(n + f +|B|)&lt;/code&gt;, where &lt;code&gt;f&lt;/code&gt; is the number of floating-point operations performed (&lt;code&gt;f&lt;/code&gt; dominates the run time unless &lt;code&gt;A&lt;/code&gt; has one or more columns with no entries, in which case either &lt;code&gt;n&lt;/code&gt; or &lt;code&gt;|B|&lt;/code&gt; can be greater than f)&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what does this &lt;code&gt;|B|&lt;/code&gt; stand for? What should be the runtime complexity in Big-O notation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated.Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2012-12-09T20:56:31.797" Title="What is the runtime complexity of MATLAB operation A*B where A and B are general sparse matrices?" Tags="|matlab|sparse-matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4825" PostTypeId="2" ParentId="4822" CreationDate="2012-12-09T19:06:51.237" Score="1" Body="&lt;p&gt;There have been numerous panels and reports about exascale computing that should have these sorts of numbers. I would look for reports that came out from DoE (or were written for DoE). Also, there may be relevant stuff at &lt;a href=&quot;http://www.exascale.org&quot; rel=&quot;nofollow&quot;&gt;http://www.exascale.org&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-09T19:06:51.237" CommentCount="0" CommunityOwnedDate="2012-12-09T19:06:51.237" ContentLicense="CC BY-SA 3.0" />
  <row Id="4826" PostTypeId="1" AcceptedAnswerId="4870" CreationDate="2012-12-09T19:51:00.353" Score="5" ViewCount="2967" Body="&lt;p&gt;I am trying to code up logistic regression in Python using the SciPy &lt;code&gt;fmin_bfgs&lt;/code&gt; function, but am running into some issues. I wrote functions for the logistic (sigmoid) transformation  function, and the cost function, and those work fine (I have used the optimized values of the parameter vector found via canned software to test the functions, and those match up). I am not that sure of my implementation of the gradient function, but it looks reasonable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# purpose: logistic regression &#xA;import numpy as np&#xA;import scipy as sp&#xA;import scipy.optimize&#xA;&#xA;import matplotlib as mpl&#xA;import os&#xA;&#xA;# prepare the data&#xA;data = np.loadtxt('data.csv', delimiter=',', skiprows=1)&#xA;vY = data[:, 0]&#xA;mX = data[:, 1:]&#xA;intercept = np.ones(mX.shape[0]).reshape(mX.shape[0], 1)&#xA;mX = np.concatenate((intercept, mX), axis = 1)&#xA;iK = mX.shape[1]&#xA;iN = mX.shape[0]&#xA;&#xA;# logistic transformation&#xA;def logit(mX, vBeta):&#xA;    return((1/(1.0 + np.exp(-np.dot(mX, vBeta)))))&#xA;&#xA;# test function call&#xA;vBeta0 = np.array([-.10296645, -.0332327, -.01209484, .44626211, .92554137, .53973828, &#xA;    1.7993371, .7148045  ])&#xA;logit(mX, vBeta0)&#xA;&#xA;# cost function&#xA;def logLikelihoodLogit(vBeta, mX, vY):&#xA;    return(-(np.sum(vY*np.log(logit(mX, vBeta)) + (1-vY)*(np.log(1-logit(mX, vBeta))))))&#xA;logLikelihoodLogit(vBeta0, mX, vY) # test function call&#xA;&#xA;# gradient function&#xA;def likelihoodScore(vBeta, mX, vY):&#xA;    return(np.dot(mX.T, &#xA;                  ((np.dot(mX, vBeta) - vY)/&#xA;                   np.dot(mX, vBeta)).reshape(iN, 1)).reshape(iK, 1))&#xA;&#xA;likelihoodScore(vBeta0, mX, vY).shape # test function call&#xA;&#xA;# optimize the function (without gradient)&#xA;optimLogit = scipy.optimize.fmin_bfgs(logLikelihoodLogit, &#xA;                                  x0 = np.array([-.1, -.03, -.01, .44, .92, .53,&#xA;                                            1.8, .71]), &#xA;                                  args = (mX, vY), gtol = 1e-3)&#xA;&#xA;# optimize the function (with gradient)&#xA;optimLogit = scipy.optimize.fmin_bfgs(logLikelihoodLogit, &#xA;                                  x0 = np.array([-.1, -.03, -.01, .44, .92, .53,&#xA;                                            1.8, .71]), fprime = likelihoodScore, &#xA;                                  args = (mX, vY), gtol = 1e-3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The first optimization (without gradient)  ends with a whole lot of stuff about division by zero.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The second optimization (with gradient) ends with a matrices not aligned error, which probably means I have got the way the gradient is to be returned wrong.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any help with this is appreciated. If anyone wants to try this, the data is included below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;low,age,lwt,race,smoke,ptl,ht,ui&#xA;0,19,182,2,0,0,0,1&#xA;0,33,155,3,0,0,0,0&#xA;0,20,105,1,1,0,0,0&#xA;0,21,108,1,1,0,0,1&#xA;0,18,107,1,1,0,0,1&#xA;0,21,124,3,0,0,0,0&#xA;0,22,118,1,0,0,0,0&#xA;0,17,103,3,0,0,0,0&#xA;0,29,123,1,1,0,0,0&#xA;0,26,113,1,1,0,0,0&#xA;0,19,95,3,0,0,0,0&#xA;0,19,150,3,0,0,0,0&#xA;0,22,95,3,0,0,1,0&#xA;0,30,107,3,0,1,0,1&#xA;0,18,100,1,1,0,0,0&#xA;0,18,100,1,1,0,0,0&#xA;0,15,98,2,0,0,0,0&#xA;0,25,118,1,1,0,0,0&#xA;0,20,120,3,0,0,0,1&#xA;0,28,120,1,1,0,0,0&#xA;0,32,121,3,0,0,0,0&#xA;0,31,100,1,0,0,0,1&#xA;0,36,202,1,0,0,0,0&#xA;0,28,120,3,0,0,0,0&#xA;0,25,120,3,0,0,0,1&#xA;0,28,167,1,0,0,0,0&#xA;0,17,122,1,1,0,0,0&#xA;0,29,150,1,0,0,0,0&#xA;0,26,168,2,1,0,0,0&#xA;0,17,113,2,0,0,0,0&#xA;0,17,113,2,0,0,0,0&#xA;0,24,90,1,1,1,0,0&#xA;0,35,121,2,1,1,0,0&#xA;0,25,155,1,0,0,0,0&#xA;0,25,125,2,0,0,0,0&#xA;0,29,140,1,1,0,0,0&#xA;0,19,138,1,1,0,0,0&#xA;0,27,124,1,1,0,0,0&#xA;0,31,215,1,1,0,0,0&#xA;0,33,109,1,1,0,0,0&#xA;0,21,185,2,1,0,0,0&#xA;0,19,189,1,0,0,0,0&#xA;0,23,130,2,0,0,0,0&#xA;0,21,160,1,0,0,0,0&#xA;0,18,90,1,1,0,0,1&#xA;0,18,90,1,1,0,0,1&#xA;0,32,132,1,0,0,0,0&#xA;0,19,132,3,0,0,0,0&#xA;0,24,115,1,0,0,0,0&#xA;0,22,85,3,1,0,0,0&#xA;0,22,120,1,0,0,1,0&#xA;0,23,128,3,0,0,0,0&#xA;0,22,130,1,1,0,0,0&#xA;0,30,95,1,1,0,0,0&#xA;0,19,115,3,0,0,0,0&#xA;0,16,110,3,0,0,0,0&#xA;0,21,110,3,1,0,0,1&#xA;0,30,153,3,0,0,0,0&#xA;0,20,103,3,0,0,0,0&#xA;0,17,119,3,0,0,0,0&#xA;0,17,119,3,0,0,0,0&#xA;0,23,119,3,0,0,0,0&#xA;0,24,110,3,0,0,0,0&#xA;0,28,140,1,0,0,0,0&#xA;0,26,133,3,1,2,0,0&#xA;0,20,169,3,0,1,0,1&#xA;0,24,115,3,0,0,0,0&#xA;0,28,250,3,1,0,0,0&#xA;0,20,141,1,0,2,0,1&#xA;0,22,158,2,0,1,0,0&#xA;0,22,112,1,1,2,0,0&#xA;0,31,150,3,1,0,0,0&#xA;0,23,115,3,1,0,0,0&#xA;0,16,112,2,0,0,0,0&#xA;0,16,135,1,1,0,0,0&#xA;0,18,229,2,0,0,0,0&#xA;0,25,140,1,0,0,0,0&#xA;0,32,134,1,1,1,0,0&#xA;0,20,121,2,1,0,0,0&#xA;0,23,190,1,0,0,0,0&#xA;0,22,131,1,0,0,0,0&#xA;0,32,170,1,0,0,0,0&#xA;0,30,110,3,0,0,0,0&#xA;0,20,127,3,0,0,0,0&#xA;0,23,123,3,0,0,0,0&#xA;0,17,120,3,1,0,0,0&#xA;0,19,105,3,0,0,0,0&#xA;0,23,130,1,0,0,0,0&#xA;0,36,175,1,0,0,0,0&#xA;0,22,125,1,0,0,0,0&#xA;0,24,133,1,0,0,0,0&#xA;0,21,134,3,0,0,0,0&#xA;0,19,235,1,1,0,1,0&#xA;0,25,95,1,1,3,0,1&#xA;0,16,135,1,1,0,0,0&#xA;0,29,135,1,0,0,0,0&#xA;0,29,154,1,0,0,0,0&#xA;0,19,147,1,1,0,0,0&#xA;0,19,147,1,1,0,0,0&#xA;0,30,137,1,0,0,0,0&#xA;0,24,110,1,0,0,0,0&#xA;0,19,184,1,1,0,1,0&#xA;0,24,110,3,0,1,0,0&#xA;0,23,110,1,0,0,0,0&#xA;0,20,120,3,0,0,0,0&#xA;0,25,241,2,0,0,1,0&#xA;0,30,112,1,0,0,0,0&#xA;0,22,169,1,0,0,0,0&#xA;0,18,120,1,1,0,0,0&#xA;0,16,170,2,0,0,0,0&#xA;0,32,186,1,0,0,0,0&#xA;0,18,120,3,0,0,0,0&#xA;0,29,130,1,1,0,0,0&#xA;0,33,117,1,0,0,0,1&#xA;0,20,170,1,1,0,0,0&#xA;0,28,134,3,0,0,0,0&#xA;0,14,135,1,0,0,0,0&#xA;0,28,130,3,0,0,0,0&#xA;0,25,120,1,0,0,0,0&#xA;0,16,95,3,0,0,0,0&#xA;0,20,158,1,0,0,0,0&#xA;0,26,160,3,0,0,0,0&#xA;0,21,115,1,0,0,0,0&#xA;0,22,129,1,0,0,0,0&#xA;0,25,130,1,0,0,0,0&#xA;0,31,120,1,0,0,0,0&#xA;0,35,170,1,0,1,0,0&#xA;0,19,120,1,1,0,0,0&#xA;0,24,116,1,0,0,0,0&#xA;0,45,123,1,0,0,0,0&#xA;1,28,120,3,1,1,0,1&#xA;1,29,130,1,0,0,0,1&#xA;1,34,187,2,1,0,1,0&#xA;1,25,105,3,0,1,1,0&#xA;1,25,85,3,0,0,0,1&#xA;1,27,150,3,0,0,0,0&#xA;1,23,97,3,0,0,0,1&#xA;1,24,128,2,0,1,0,0&#xA;1,24,132,3,0,0,1,0&#xA;1,21,165,1,1,0,1,0&#xA;1,32,105,1,1,0,0,0&#xA;1,19,91,1,1,2,0,1&#xA;1,25,115,3,0,0,0,0&#xA;1,16,130,3,0,0,0,0&#xA;1,25,92,1,1,0,0,0&#xA;1,20,150,1,1,0,0,0&#xA;1,21,200,2,0,0,0,1&#xA;1,24,155,1,1,1,0,0&#xA;1,21,103,3,0,0,0,0&#xA;1,20,125,3,0,0,0,1&#xA;1,25,89,3,0,2,0,0&#xA;1,19,102,1,0,0,0,0&#xA;1,19,112,1,1,0,0,1&#xA;1,26,117,1,1,1,0,0&#xA;1,24,138,1,0,0,0,0&#xA;1,17,130,3,1,1,0,1&#xA;1,20,120,2,1,0,0,0&#xA;1,22,130,1,1,1,0,1&#xA;1,27,130,2,0,0,0,1&#xA;1,20,80,3,1,0,0,1&#xA;1,17,110,1,1,0,0,0&#xA;1,25,105,3,0,1,0,0&#xA;1,20,109,3,0,0,0,0&#xA;1,18,148,3,0,0,0,0&#xA;1,18,110,2,1,1,0,0&#xA;1,20,121,1,1,1,0,1&#xA;1,21,100,3,0,1,0,0&#xA;1,26,96,3,0,0,0,0&#xA;1,31,102,1,1,1,0,0&#xA;1,15,110,1,0,0,0,0&#xA;1,23,187,2,1,0,0,0&#xA;1,20,122,2,1,0,0,0&#xA;1,24,105,2,1,0,0,0&#xA;1,15,115,3,0,0,0,1&#xA;1,23,120,3,0,0,0,0&#xA;1,30,142,1,1,1,0,0&#xA;1,22,130,1,1,0,0,0&#xA;1,17,120,1,1,0,0,0&#xA;1,23,110,1,1,1,0,0&#xA;1,17,120,2,0,0,0,0&#xA;1,26,154,3,0,1,1,0&#xA;1,20,106,3,0,0,0,0&#xA;1,26,190,1,1,0,0,0&#xA;1,14,101,3,1,1,0,0&#xA;1,28,95,1,1,0,0,0&#xA;1,14,100,3,0,0,0,0&#xA;1,23,94,3,1,0,0,0&#xA;1,17,142,2,0,0,1,0&#xA;1,21,130,1,1,0,1,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1969" LastActivityDate="2012-12-16T00:16:38.717" Title="Logistic regression with Python" Tags="|python|machine-learning|numpy|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4827" PostTypeId="2" ParentId="4824" CreationDate="2012-12-09T20:56:31.797" Score="8" Body="&lt;p&gt;As a general rule, the by far dominant aspect of multiplying two sparse matrices is not actually any floating point computations, but it is forming and allocating the sparsity pattern of the resulting sparse matrix. If $A,B$ are two sparse matrices, then $AB$ is in general sparse as well, but far less so -- it will have many more entries per row, and there is significant work involved in determining which ones these are and in allocating the resulting memory. This will take far longer than actually multiplying out the respective elements unless $A,B$ were already pretty dense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second rule is to not form the product between two sparse matrices unless you can't help it. For example, if $X=AB$ and you later need $X$ to form products of the kind $b=Xa$ where $a,b$ are vectors, then you will almost always be better off by computing $b$ as $b=A(Ba)$, i.e., by using two matrix-vector products rather than one matrix-vector product with the product matrix $X$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-09T20:56:31.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4828" PostTypeId="1" CreationDate="2012-12-10T00:34:50.140" Score="6" ViewCount="1147" Body="&lt;p&gt;I am modeling fracture propagation using a 2-D dynamic unstructured grid. As the fracture propagates over time, the elements move accordingly. For a given time step, I would like to interpolate the data of my unstructured grid $z_i=f(x_i,y_i)$ (where $(x_i,y_i)$ are the nodes of the unstructured triangles) to obtain the value of the function $z$ on the Cartesian grid (where $x$ and $y$ regularly spaced). My grid size is about $100 \times 100$ nodes. There are many ways of doing this interpolation scheme but I do not know enough about the topic to decide which scheme is robust and simple enough. I am using Fortran 90 (do not ask why...). Is there any open-source algorithm available? I do not want to reinvent the wheel!&lt;/p&gt;&#xA;" OwnerUserId="3410" LastEditorUserId="276" LastEditDate="2012-12-10T19:41:46.170" LastActivityDate="2019-01-10T19:06:11.570" Title="Grid mapping from an unstructured triangular mesh to a regular rectangular mesh" Tags="|fortran|interpolation|unstructured-mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4829" PostTypeId="1" AcceptedAnswerId="4834" CreationDate="2012-12-10T01:08:17.053" Score="17" ViewCount="1374" Body="&lt;p&gt;I'm curious to know what good numerical algorithms exist for evaluation of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Generalized_hypergeometric_function&quot;&gt;generalized hypergeometric function&lt;/a&gt; (or series), defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${}_pF_q(a_1,\ldots,a_p;b_1,\ldots,b_q;z) = \sum_{k=0}^{\infty} \frac{(a_1)_k\cdots(a_p)_k}{(b_1)_k\cdots(b_q)_k}\frac{z^k}{k!}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, this series is not necessarily going to converge very fast (or at all), so summing up terms one by one seems less than ideal. Is there some alternate method that works better? To be specific, I'm looking for something that will give 4 or 5 digits of precision with a reasonable number of calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common cases that I usually see used are $p=1,q=1$ and $p=2,q=1$, but in the particular project I'm working on, I have a need for $p=1,q=2$. Obviously a general algorithm for any $p$ and $q$ is ideal, but I'll take what I can get.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2012-12-10T21:09:35.207" Title="What are the efficient, accurate algorithms for evaluation of hypergeometric functions?" Tags="|special-functions|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4830" PostTypeId="2" ParentId="4828" CreationDate="2012-12-10T05:32:06.713" Score="3" Body="&lt;p&gt;I have always used &lt;a href=&quot;https://people.sc.fsu.edu/~jburkardt/f_src/toms526/bivar.f90&quot; rel=&quot;nofollow noreferrer&quot;&gt;bivar.f90&lt;/a&gt; for interpolating irregular data. It is Fortran and very simple to use.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="7472" LastEditDate="2019-01-10T19:06:11.570" LastActivityDate="2019-01-10T19:06:11.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4831" PostTypeId="1" CreationDate="2012-12-10T09:01:58.580" Score="5" ViewCount="213" Body="&lt;p&gt;I am interested in learning firsthand about graph data structures but have no data to put into the database. Can anyone recommend a source for a beginner to find graph data? (Preferably a source that has nothing to do with Facebook.)&lt;/p&gt;&#xA;" OwnerUserId="3412" LastEditorUserId="276" LastEditDate="2012-12-14T02:56:04.947" LastActivityDate="2012-12-14T02:56:04.947" Title="Where do I find data to start a graph data structure to practice on?" Tags="|libraries|graph-theory|data-structures|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4832" PostTypeId="1" CreationDate="2012-12-10T11:00:22.583" Score="5" ViewCount="230" Body="&lt;p&gt;I managed to build the Voronoï diagram of n points using Fortune's algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives me a set of half-edges, some of which being infinite (no starting point and/or no end point).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to restrict this diagram to a specific polygon P, by creating new vertices at the intersections of the polygon and the inifinite half-edges and connecting them in order to close all cells of the diagram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If P is convex, I think I can perform it in O(n log p) where p is the size of P, by finding the intersecting edge of P for each half-edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there some way to do better in the case of a convex polygon ?&lt;/li&gt;&#xA;&lt;li&gt;Can we do something in the general case (any polygon, not convex)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3414" LastActivityDate="2013-02-02T00:15:36.120" Title="Restrict Voronoï diagram to a polygon" Tags="|computational-geometry|complexity|voronoi-diagrams|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4833" PostTypeId="1" AcceptedAnswerId="7284" CreationDate="2012-12-10T13:51:13.417" Score="5" ViewCount="551" Body="&lt;p&gt;For a kinetic Monte Carlo simulation of solids that crystallize in the diamond lattice structure, I need some efficient representation of the diamond lattice as integer(s), to store it in some array-like structure, where each array index can uniquely be mapped to a certain atom in real-space:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;array[i] \rightarrow (x,y,z)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The diamond lattice consists of two face centered lattices, that are shifted against each other:&lt;img src=&quot;https://i.stack.imgur.com/O0VOi.jpg&quot; alt=&quot;Diamond lattice&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have not found very suitable solutions. I have some demands to this mapping that must be met:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I need to know the nearest and second nearest neighbors of each atom in integer space. If possible, I want to pre-calculate the numbers that I need to add or substract from the array index of my current atom to get to the 4 nearest and 12 second nearest neighbors: &lt;span class=&quot;math-container&quot;&gt;$j^k = i+\Delta i^k, k = 1,\dots,4 (12)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;I need to fill the system from the bottom (in real space) up, so I would be happy to have some simple relation at least for the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; coordinate &lt;span class=&quot;math-container&quot;&gt;$z(i)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The simpler the mapping is, the happier I would be. It would be a big plus if it 'd be somewhat intuitive, so I can imagine it in my head. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What I tried so far: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I found &lt;a href=&quot;http://www.hermetic.ch/compsci/lattgeom.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website&lt;/a&gt; (and recently also &lt;a href=&quot;http://www.physik.fu-berlin.de/en/einrichtungen/ag/ag-eisert/research/software/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website&lt;/a&gt;, on which a &lt;span class=&quot;math-container&quot;&gt;$(i,j,k)$&lt;/span&gt; representation for the diamond lattice is given including the &lt;span class=&quot;math-container&quot;&gt;$(\Delta i, \Delta j, \Delta k)$&lt;/span&gt; for the nearest neighbors. However, I couldn't find the mapping to get from there to real space. (Any idea how I could do that?)&lt;/li&gt;&#xA;&lt;li&gt;An option that works is to use an fcc lattice geometry, with basis vectors pointing from the origin to the atoms in the face centers. With that basis, each atom of the lattice can be reached with a set of 3 integers. However, since the diamond lattice consists of two shifted fcc lattices, I would need a fourth index to determine to which lattice each atom belongs to. Also, I didn't succeed in finding a simple mapping &lt;span class=&quot;math-container&quot;&gt;$(x,y,z) \rightarrow (i,j,k,l)$&lt;/span&gt; that allows me to determine the &lt;span class=&quot;math-container&quot;&gt;$z=0$&lt;/span&gt; plane or so. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am asking, because this should be a common problem. There exist so many simulations/publications of simulations, in which the diamond lattice was simulated (Si, GaS,...) that it surprises me, I can't find a simple solution to my problem on the internet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be very happy if you could point me into some direction or know any literature on how to do this. &lt;/p&gt;&#xA;" OwnerUserId="3415" LastEditorUserId="22284" LastEditDate="2018-12-31T09:34:38.017" LastActivityDate="2018-12-31T09:34:38.017" Title="index representation of the diamond lattice" Tags="|monte-carlo|simulation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4834" PostTypeId="2" ParentId="4829" CreationDate="2012-12-10T20:37:31.753" Score="17" Body="&lt;p&gt;In a single application, it is rather likely that you only will need a small subset of all the possible extremes of the generalized hypergeometric function. It is a very general function, after all. Having an idea about the range of $z$ and of the parameters $a_i, b_i$ would allow giving more specific advice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the standard method, assuming $p \le q + 1$, is of course to use the defining power series when $|z|$ is small. If $p &amp;lt; q + 1$, it is best to switch to an asymptotic expansion when $|z|$ is large, either because the Taylor series converges too slowly and/or because it becomes too inaccurate due to catastrophic cancellation. The best cutoff between these algorithms depends on the parameters and the accuracy requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For ${}_1F_2$ the asymptotic series is given by &lt;a href=&quot;http://functions.wolfram.com/HypergeometricFunctions/Hypergeometric1F2/06/02/03/&quot;&gt;http://functions.wolfram.com/HypergeometricFunctions/Hypergeometric1F2/06/02/03/&lt;/a&gt; It looks rather horrible, but if your $a_1, b_1, b_2$ are fixed, you can compute numerical values for the coefficients in advance. General formulas are found in the DLMF: &lt;a href=&quot;http://dlmf.nist.gov/16.11&quot;&gt;http://dlmf.nist.gov/16.11&lt;/a&gt; (Note that some care is required to select the correct branch cuts.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is a range where neither the Taylor series nor the asymptotic series works well enough, &quot;exponentially-improved expansions&quot; might be useful. Another possibility worth mentioning is that you could just plug the hypergeometric differential equation into a general-purpose ODE solver. This should work quite well especially if you only need 4-5 digits. This can be used to do analytic continuation from a small $z$ (where the power series works well) to a larger one, or in reverse from a value obtained via an asymptotic series (you might need to do a bit more work to get all the derivatives needed as initial values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need functions with $p = q + 1$ on the whole complex plane then $1/z$ transformation formulas can be used to map the exterior of the unit disk to the interior. Some convergence acceleration algorithm or other method, like numerical integration of the ODE, must be used close to the unit circle. If $p &amp;gt; q + 1$ the radius is convergence is zero, so if the function you want to evaluate is given by such a divergent series, you might need to apply a Borel transform (numerically or symbolically) to reduce it to a convergent series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a complete implementation, there are other issues to consider as well (for example, dealing with parameters which are extremely large or very close to negative integers). For sufficiently bad parameters, it will be very difficult to obtain accurate values with double precision, no matter what you do, so arbitrary-precision arithmetic might be needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should note that I have written a nearly complete numerical implementation of the generalized hypergeometric function for the mpmath library (it is currently missing asymptotic series for functions higher than ${}_2F_3$), which might be useful to study or run tests against (assuming it is not fast enough already for your purposes).&lt;/p&gt;&#xA;" OwnerUserId="3420" LastActivityDate="2012-12-10T20:37:31.753" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4835" PostTypeId="2" ParentId="4829" CreationDate="2012-12-10T21:09:35.207" Score="2" Body="&lt;p&gt;The canonical reference for all special functions is Abramowicz and Stegun. This is a book that's been around for about half a century soon and if there's something you can't find in it, take a look at the &quot;updated second edition&quot; which is in fact a website organized by the National Institute of Standards (NIST). I don't have the exact URL but it shouldn't be very difficult to find.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-10T21:09:35.207" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4836" PostTypeId="2" ParentId="4831" CreationDate="2012-12-11T02:32:57.680" Score="6" Body="&lt;p&gt;When I was in the same position as you, I was very happy to find Knuth's &lt;a href=&quot;http://www-cs-faculty.stanford.edu/~uno/sgb.html&quot;&gt;Stanford Graphbase (SGB)&lt;/a&gt;. He supplies not only a library for working with graphs, but also some data sets to play with. My favorite one was &lt;a href=&quot;http://www-cs-faculty.stanford.edu/~uno/sgb-words.txt&quot;&gt;the 5-letter English words dataset&lt;/a&gt;. You can use your programming language of choice to generate an undirected graph where two words are neighbors if and only if they differ by exactly one letter. So, for example, &quot;bread&quot; and &quot;broad&quot; are neighbors, but not &quot;bread&quot; and &quot;bleak&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As proposed in the book/documentation (you can buy a printed copy on Amazon, it's quite interesting), you can then play games like &quot;Word Ladders&quot;, where players try to find the shortest path between words. Implementing these games will give you an excuse to experiment with shortest-path algorithms, breadth-first searches, etc. Knuth actually provides an implementation of such a game (using his library) in the file &lt;code&gt;ladders.w&lt;/code&gt; in the SGB tarball.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another data set provided on the SGB webpage: a graph with the adjacencies between the contiguous United States and DC. I cannot include the hyperlink because I'm new to this site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: the library was written using CWEB, a tool for literate programming (again, cannot include link, so google it). If you want to actually use the SGB library, you will need to install CWEB; if you want the documentation, you will need CWEB and TeX.&lt;/p&gt;&#xA;" OwnerUserId="1917" LastActivityDate="2012-12-11T02:32:57.680" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4838" PostTypeId="1" AcceptedAnswerId="4839" CreationDate="2012-12-11T08:47:48.040" Score="8" ViewCount="147" Body="&lt;p&gt;I have need to compute the functions:&#xA;$$ f(x) = \frac{\sin^{-1}x}{x}$$&#xA;and&#xA;$$ g(x) = \frac{\sin a x}{\sin x} $$&#xA;where $a\in[0,1]$ and $ x\in[0,\frac{\pi}{2}]$ and is often very small ($x\ll 1$). Are there any general ways of generating highly accurate algorithms for &quot;special&quot; functions like these?&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-12-11T21:00:47.487" Title="Computing ratio of trigonometric functions" Tags="|special-functions|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4839" PostTypeId="2" ParentId="4838" CreationDate="2012-12-11T11:37:56.177" Score="9" Body="&lt;p&gt;Do a polynomial expansion (a la l'Hopital's rule) for both enumerator and denominator and you get a rational function that, for small $x$, will approximate the function well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example:&#xA;$$&#xA;  \frac{\sin ax}{\sin x} \approx&#xA;  \frac{ax-\tfrac 1{3!}(ax)^3 + \ldots}{x-\tfrac 1{3!}x^3 + \ldots}&#xA;  =&#xA;  \frac{a-\tfrac 1{3!}a^3x^2 + \ldots}{1-\tfrac 1{3!}x^2 + \ldots}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do better if you have an idea of the range of values $x$ at which &#xA;you want to evaluate. You can then replace the Taylor expansion around $x_0=0$&#xA;above by a better suited interpolation of projection approach for &#xA;the range of $x$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-12-11T14:58:02.670" LastActivityDate="2012-12-11T14:58:02.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4841" PostTypeId="1" CreationDate="2012-12-11T16:26:52.630" Score="6" ViewCount="286" Body="&lt;p&gt;I was wondering where I could get a detailed account (either in print or online) on applying a Neumann/mixed Boundary condition along the &lt;span class=&quot;math-container&quot;&gt;$r=0$&lt;/span&gt; axis in an axially symmetric geometry. Though this is a very common task I have always struggled to find a reasonable account of it. Piecing together sparse and poorly referenced notes I believe that in 1D polar coordinates where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla^2 u = \frac{\partial^2 u}{\partial r^2} + \frac{1}{r} \frac{\partial u}{\partial r}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one gets, by Taylor expansion at the origin,:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla^2 u \rvert_{r=0} \simeq 2\frac{\partial^2 u}{\partial r^2}\rvert_{r=0}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is symmetric about the &lt;span class=&quot;math-container&quot;&gt;$r=0$&lt;/span&gt; axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to more information on higher order derivatives, best practice, non-symmetric distributions ... etc. Also, I would be very keen to find some info on best practice for applying this boundary condition in FEM analysis.&lt;/p&gt;&#xA;" OwnerUserId="1880" LastEditorUserId="9667" LastEditDate="2019-06-25T20:34:32.410" LastActivityDate="2019-06-25T20:34:32.410" Title="Neumann BCs in cylindrical geometry (FEM)" Tags="|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4842" PostTypeId="1" AcceptedAnswerId="4846" CreationDate="2012-12-11T17:24:27.363" Score="6" ViewCount="237" Body="&lt;p&gt;I am looking for efficient algorithm to interpolate values from one unstructured 2D mesh grid to another. Both grids are constructed using the same type of elements (triangles or quadrilaterals). Both meshes are created for the same geometry.&lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2012-12-11T18:15:26.157" Title="What is the most efficient approach to interpolate values between two FEM meshes in 2D?" Tags="|interpolation|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4844" PostTypeId="1" CreationDate="2012-12-11T18:02:58.460" Score="4" ViewCount="759" Body="&lt;p&gt;I have an mathematical theorem on the QR decomposition, which relies on the QR decomposition of an invertible square complex matrix always constructing a triangular matrix with real diagonal entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While at least in Octave this seems to be true, I wonder this can be relied upon in practice. I am looking for an implementation that does not produce such a triangular factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS:&#xA;Actually the theorem in question is the correctness of the double shifted QR iteration, as described in section 3.5 of&#xA;&lt;a href=&quot;http://people.inf.ethz.ch/arbenz/ewp/Lnotes/chapter3.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://people.inf.ethz.ch/arbenz/ewp/Lnotes/chapter3.pdf&lt;/a&gt; .&#xA;The presentation seems to rely on a particular implementation of the QR algorithm.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2012-12-12T16:40:45.223" Title="On implementations of the complex QR decomposition" Tags="|eigensystem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4845" PostTypeId="2" ParentId="4841" CreationDate="2012-12-11T18:13:12.930" Score="7" Body="&lt;p&gt;This is the wrong boundary condition. If your solution is smooth, then the correct condition at $r=0$ is&#xA;$$&#xA;  \frac{du}{dr} = 0.&#xA;$$&#xA;You can see this by thinking about what would happen if you cut a line through the origin through the entire domain, i.e., you look at the solution not only for $r\ge 0$ but also for $r\le 0$. To the left of $r=0$ you of course have a mirror image of the solution at the right. If you don't want the solution to have a kink at the origin, you need to require the condition above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is, of course, also the natural boundary condition so it is easy to implement.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-11T18:13:12.930" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4846" PostTypeId="2" ParentId="4842" CreationDate="2012-12-11T18:15:26.157" Score="2" Body="&lt;p&gt;Take a look at the recent manuscripts by Caroline Japhet and Martin Gander. They have a few papers in this direction.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-11T18:15:26.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4847" PostTypeId="2" ParentId="4844" CreationDate="2012-12-11T19:20:02.803" Score="6" Body="&lt;p&gt;A slight generalization of a Householder reflector, as seen in &lt;a href=&quot;http://www.netlib.org/lapack/complex16/zlarfg.f&quot; rel=&quot;noreferrer&quot;&gt;LAPACK's zlarfg&lt;/a&gt;, can be used to define a sequence of unitary transformations. In particular, the $j$'th transformation zeros the portion of the matrix below the $j$'th diagonal entry, and also ensures that the $j$'th diagonal entry becomes real. The unblocked algorithm &lt;a href=&quot;http://www.netlib.org/lapack/complex16/zgeqr2.f&quot; rel=&quot;noreferrer&quot;&gt;zgeqr2&lt;/a&gt; and the blocked algorithm &lt;a href=&quot;http://www.netlib.org/lapack/complex16/zgeqrf.f&quot; rel=&quot;noreferrer&quot;&gt;zgeqrf&lt;/a&gt; make use of this approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, suppose that you have an algorithm that provides $A=QR$, where $R$ is triangular but with complex diagonal entries. Then define the diagonal matrix $\Omega$ to have diagonal entries drawn from the unit circle such that $\Omega R$ has real diagonal entries, which would imply that the decomposition $A=(Q \bar \Omega)(\Omega R)$ is a QR decomposition with your desired properties.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-12-12T16:40:45.223" LastActivityDate="2012-12-12T16:40:45.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4848" PostTypeId="2" ParentId="4838" CreationDate="2012-12-11T21:00:47.487" Score="4" Body="&lt;p&gt;My approach is to use software like SymPy as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sympy import var, sin, S&#xA;var(&quot;x a&quot;)&#xA;g = sin(a*x)/sin(x)&#xA;gseries = g.series(x, 0, 10).removeO()&#xA;s = {x: S(1)/100, a: S(1)/2}&#xA;print gseries.subs(s).n(30)&#xA;print g.subs(s).n(30)&#xA;print &quot;%.17f&quot; % g.subs({x: 1./100, a: 1./2})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.500006250065104828565736800905&#xA;0.500006250065104828565736868886&#xA;0.50000625006510480&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first number is a Taylor series expansion truncated at 10 terms, the second number is the exact evaluation. SymPy uses exact arithmetic, in this example I used x = 1/100 and a = 1/2, but you can play with that. Finally, I evaluate it to 30 decimal digits so that one can easily compare the numbers.&#xA;The third number is a double precision evaluation using Python's floats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, it seems to me that there is no cancellation. But for other expressions the direct double precision evaluation might no be accurate enough and then series expansion is one way to evaluate it. The other is rational approximation, I have used &lt;code&gt;MiniMaxApproximation&lt;/code&gt; in Mathematica in the past with great success.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2012-12-11T21:00:47.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4849" PostTypeId="1" AcceptedAnswerId="4861" CreationDate="2012-12-12T16:15:07.233" Score="5" ViewCount="310" Body="&lt;p&gt;In finite difference theory, you learn, that you have to use upwinding for equations with high convection, like Burgers' equation. What does the finite volume equivalent look like? What if the convection is nonlinear like in Burgers' equation?&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-12-14T00:34:37.193" Title="How to do upwinding in finite volume schemes for nonlinear equations?" Tags="|finite-volume|nonlinear-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4850" PostTypeId="1" CreationDate="2012-12-12T16:19:56.747" Score="3" ViewCount="233" Body="&lt;p&gt;Where is the divide between a computational scientist and a computational X (X = some natural science or discipline)? What questions could someone ask themself to decide which is the best route for them?  &lt;/p&gt;&#xA;" OwnerUserId="3433" LastActivityDate="2012-12-13T07:37:51.263" Title="Difference between Computational science and Computational X (X = some natural science or discipline)" Tags="|education|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4851" PostTypeId="2" ParentId="4850" CreationDate="2012-12-12T16:28:56.980" Score="2" Body="&lt;p&gt;There is only an &lt;em&gt;artificial&lt;/em&gt; difference. Computational X solves problems in X, i. e. physics.&#xA;Most often, you specialize later in Computational science, so you get an X, lately. Computational science without the X doesn't make much sense, so you would have to study computer science or mathematics, to get the pure math or the pure informatics.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2012-12-12T16:28:56.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4852" PostTypeId="2" ParentId="4850" CreationDate="2012-12-12T16:31:15.993" Score="3" Body="&lt;p&gt;Your question implies that there is a firm line between them, but this is far from the case.  I will be answering from my own position, where I see myself as a computational scientist, but I work in computational plasmas.  Hopefully the difference I draw between them will be insightful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a computational scientist, I enjoy the mathematics and methodology behind solving the many equations that appear in my field (Poisson's eq, compressible N-S, Boltzmann eq, etc.).  My advisor, on the other hand, couldn't really care less how the equations are solved, only that the solutions are found quickly and are &quot;correct&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this, the main difference I see between the two groups is whether the application or the methodology is the focus.  I personally see the methodology as my focus, since after graduating with my PhD, I likely won't stay in plasmas.  It is really more of a spectrum, with CS/Mathematics on one end, next to computational science, followed by computational X, and then the pure X, whatever that may be.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2012-12-12T16:31:15.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4853" PostTypeId="1" CreationDate="2012-12-13T06:10:02.140" Score="3" ViewCount="182" Body="&lt;p&gt;I have $n$  arbitrary $p\times 1$ vectors $x_i$, and $p\times k$ matrices $A_i$, and $n$ $p \times p$ positive semidefinite matrices $S_i$, where some (often most) of the $S_i$'s are same (for example only two different $S$ matrices, one positive definite which applies to $i=1,\ldots n-1$ and semidefinite $S$ for $i=n$). I want to make following linear transformations for all $x_i$ and $A_i$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_i^* = L_i^{-1}x_i$ and&#xA;$A_i^* = L_i^{-1}A_i$, where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L_i$ is a lower triangular matrix so that $L_i L'_i=S_i$ (or even better $L_i D_i L'_i=S_i$ where $D_i$ is diagonal matrix.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n$ can be range from few to thousands or even more, $p$ is usually less than 10 and $k$ is usually less than 100. $S$ can contain rows and columns with zeros, and it is originally formed as $BB'$ where $B$ is a lower tridiagonal matrix with nonnegative diagonal elements.These $B$'s are not available though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering what would be the optimal way of accomplishing this in terms of speed and accuracy, with more weight on accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I'm using my self written LDL decomposition function for different $S$, invert $L$ and compute the transformations, as I have beem dealing with cases with small number of different $S$ and large $n$. If I have understood correctly, it would be wiser to just solve the linear equations without explicitly inverting the matrix in terms of accuracy, but for speed it seems to be better option?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Fortran, and I would rather use some LAPACK function for the factorization instead of my own LDL decomposition which I cannot guarantee to be stable etc., but I'm not sure which would be the good way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: This may be better suited for stackoverflow, so this can be removed by whoever has the power to do so.&lt;/p&gt;&#xA;" OwnerUserId="3432" LastEditorUserId="3432" LastEditDate="2012-12-13T07:33:46.593" LastActivityDate="2012-12-13T07:33:46.593" Title="How to accurately decompose positive semidefinite matrix and use the lower triangular part in linear equations" Tags="|linear-algebra|matrix|fortran|accuracy|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4854" PostTypeId="2" ParentId="4850" CreationDate="2012-12-13T07:37:51.263" Score="2" Body="&lt;p&gt;I've given an answer to this question here:&#xA;  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/4695/does-computational-science-involve-programming&quot;&gt;Does Computational Science involve programming?&lt;/a&gt;&#xA;I still think it to be true :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2012-12-13T07:37:51.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4855" PostTypeId="2" ParentId="4849" CreationDate="2012-12-13T07:54:40.943" Score="2" Body="&lt;p&gt;Read any textbook on FV, for example Ferziger and Peric, or Wesseling &lt;img src=&quot;https://i.stack.imgur.com/ruDqR.jpg&quot; alt=&quot;&quot;&gt;!&lt;img src=&quot;https://i.stack.imgur.com/lbVR2.jpg&quot; alt=&quot;&quot;&gt;!&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2012-12-13T07:54:40.943" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4856" PostTypeId="1" CreationDate="2012-12-13T08:32:55.760" Score="3" ViewCount="87" Body="&lt;p&gt;Some background: the &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999101967269&quot; rel=&quot;nofollow&quot;&gt;Front-Tracking&lt;/a&gt; method uses a triangular surface mesh to describe the boundary between two immiscible fluids. To deal with the breakup and coalescence of the fluid interface, direct topological operations are executed on the surface mesh. Since in this case, there exists a history of topological operations between the old and the new mesh, it is possible to conservatively map a field stored on the surface mesh before and after topological operations. This is used e.g. to solve transport equations for a &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S002199910700438X&quot; rel=&quot;nofollow&quot;&gt;surfactant&lt;/a&gt; (a field that describes surface-active agents whose concentration on the fluid interface changes the surface tension). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to know if there is an established algorithm that can logically connect two completely topologically disconnected surface meshes that could serve as a mapping basis? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. something like the nearest element search algorithm: for each triangle of surface mesh A, find the nearest triangle of surface mesh B.&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-12-13T08:32:55.760" Title="Conservative field mapping between two topologically disconnected surface meshes" Tags="|reference-request|computational-geometry|mapping-strategy|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4857" PostTypeId="2" ParentId="4831" CreationDate="2012-12-13T10:43:48.987" Score="1" Body="&lt;p&gt;Try &lt;a href=&quot;http://www.boost.org/doc/libs/1_52_0/libs/graph/doc/index.html&quot; rel=&quot;nofollow&quot;&gt;the boost graph library&lt;/a&gt;, it's extensively documented and, it is used by &lt;a href=&quot;http://www.cgal.org/&quot; rel=&quot;nofollow&quot;&gt;CGAL&lt;/a&gt;, which is a computational geometry library, which allows for graph visualization (e.g. &lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Segment_Delaunay_graph_2/Chapter_main.html&quot; rel=&quot;nofollow&quot;&gt;2D Segment Delaunay graph&lt;/a&gt; or &lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Apollonius_graph_2/Chapter_main.html&quot; rel=&quot;nofollow&quot;&gt;2D Apollonius Graphs visualisation&lt;/a&gt;). If you want to visualize your graphs, CGAL might be a good thing to try. It works with OFF file format, but you can convert it to STL or something else using &lt;a href=&quot;http://www.cs.princeton.edu/~min/meshconv/&quot; rel=&quot;nofollow&quot;&gt;meshconv&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2012-12-13T10:43:48.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4858" PostTypeId="2" ParentId="4831" CreationDate="2012-12-13T15:47:53.843" Score="1" Body="&lt;p&gt;The data for &lt;em&gt;Algorithms&lt;/em&gt; by Sedgewick and Wayne is available at &lt;a href=&quot;http://introcs.cs.princeton.edu/java/data/&quot; rel=&quot;nofollow&quot;&gt;http://introcs.cs.princeton.edu/java/data/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2012-12-13T15:47:53.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4859" PostTypeId="1" AcceptedAnswerId="4860" CreationDate="2012-12-13T16:16:49.020" Score="4" ViewCount="217" Body="&lt;p&gt;I recently read Wolfgang's answer to the question found &lt;a href=&quot;https://scicomp.stackexchange.com/questions/4824/what-is-the-runtime-complexity-of-matlab-operation-ab-where-a-and-b-are-general&quot;&gt;here&lt;/a&gt; and found myself wondering about a related followup question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume you have two sparse matrices $A$ and $B$.  You need to do the matrix-vector multiplication $a = Xb$ where $X=AB$.  He states in his answer that forming X is much more intensive than just doing two multiplies in the form $a=A(Bb)$, which is for the most part intuitive.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what if you have many $b$'s?  Let's assume you have $n$ different $b$'s to multiply.  How large does $n$ have to be in relation to the size/sparsity of $A$ and $B$ to make forming $X$ worthwhile or does forming $X$ never become more efficient?  If nothing can be said in general, are there specific cases where conclusions can be drawn?&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2012-12-14T11:17:45.720" Title="Efficiency of Repeated Sparse Matrix-Vector Products" Tags="|linear-algebra|sparse-matrix|efficiency|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4860" PostTypeId="2" ParentId="4859" CreationDate="2012-12-14T00:28:32.507" Score="7" Body="&lt;p&gt;With your choice of fuzzy language in your comment, you are carefully skirting the answer to your question. The number of nonzeros in a matrix is a decent proxy for cost to apply it to a vector, since the cost is dominated by the time to load its entries from cache. Roughly speaking, if $$\mathrm{nnz}(AB) &amp;gt; \mathrm{nnz}(A) + \mathrm{nnz}(B),$$ then you should apply them separately. This is typically the case for PDE matrices and (even more so) for power-law graphs such as appear in network analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are applying the matrix to many vectors at once, you get to amortize the cost to load the matrix entries. Once again, you can look at the performance model for a matrix of your chosen density and number of vectors to decide whether it is better to explicitly form the product. For square matrices, it rarely is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One place where explicit matrix products &lt;em&gt;are&lt;/em&gt; useful is in &quot;Galerkin&quot; coarse grid operators. There, the coarse grid operator $A_c = P^T A_f P$ resides in a much smaller space than the fine grid operator $A_f$, so the explicit product is much cheaper to apply.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-12-14T00:28:32.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4861" PostTypeId="2" ParentId="4849" CreationDate="2012-12-14T00:34:37.193" Score="6" Body="&lt;p&gt;You need to solve a &lt;strong&gt;Riemann problem&lt;/strong&gt;, perhaps approximately. For a linear system of equations, the solution to the Riemann problem is just upwinding applied to the characteristics. An &quot;exact&quot; Riemann solver for nonlinear problems resolves the full wave structure (consisting of &lt;em&gt;shocks&lt;/em&gt;, &lt;em&gt;rarefactions&lt;/em&gt;, and possibly linearly degenerate &lt;em&gt;contact discontinuities&lt;/em&gt;). An approximate Riemann solver does not resolve all waves, which implies (some) excess diffusion, but can be much simpler to implement. Details are discussed in any book on finite volume methods, or in Toro's book on Riemann Solvers.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-12-14T00:34:37.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4862" PostTypeId="1" CreationDate="2012-12-14T06:32:36.967" Score="1" ViewCount="171" Body="&lt;p&gt;I have a partial differential equation of the form&#xA;$$ \frac{d}{dt}f(x,t) = \Theta(x) f(x,t) \qquad \Theta(x) \sim \left[\frac{d^2}{dx^2} + k^2(x)\right] $$&#xA;subject to $f(x,t=0) = f_0(x)$, and $f(x=0,t) = f(x=1,t) = 0$. The function $k(x)$ is piecewise constant over $x\in[0,1]$. Call the $x$ intervals where $k$ is constant $I_j$. I would like to locally represent the solution $f$ as a sum of (say) Chebyshev polynomials within each interval $I_j$ and enforce continuity conditions at the boundaries of the intervals (continuity of $f$ and $\frac{\partial f}{\partial x}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, since the RHS operator is time independent, I could just solve this using an exponential: $f=e^{\Theta t}f_0$. My question is how I would compute the matrix elements of $\Theta$ to take into account the subdomain continuity conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, I could use a time marching method and make this a true domain decomposition method, but I still don't know how I would enforce the continuity conditions. I don't want to use a penalty method to enforce the constraints only approximately; I want them enforced as exactly as possible, and I really don't care about parallelism (so perhaps calling this DD is misleading). The literature on this probably exists, but I simply have no idea what I'm looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only give a rough description of $\Theta$, but it comes from the Helmholtz equation so it's hyperbolic, and a $2\times2$ block operator, with $f$ actually being a 2-vector of a function with its dual. This isn't terribly relevant to the question, since I just want to know how to enforce internal consistency.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2012-12-14T06:32:36.967" Title="Enforcing continuity conditions in pseudospectral domain decomposition methods for time dependent PDEs" Tags="|pde|hyperbolic-pde|domain-decomposition|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4863" PostTypeId="2" ParentId="4859" CreationDate="2012-12-14T11:17:45.720" Score="6" Body="&lt;p&gt;As Jed already remarked, it all depends on the matrices. But you can get an idea by considering, for example, the 5-point stencil of the Laplace equation in 2d. There, each row of the stiffness or mass matrix $A$ has five entries. But the product of two such matrices has 13 entries (apply the five point stencil to every location of the five point stencil). In 3d, $A$ has 7 entries per row, but $A^2$ has 33. The situation becomes even worse if you have, say, a $Q_2-Q_1$ discretization of the Stokes equation in 3d.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I mentioned in the other post, however, a significant cost in forming the product of a matrix is not actually the floating point operations but instead the cost of determining the sparsity pattern and resulting memory allocations. You will need to take that into account when forming a cost model.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-14T11:17:45.720" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4864" PostTypeId="1" CreationDate="2012-12-14T14:29:51.403" Score="4" ViewCount="1705" Body="&lt;p&gt;First I'm not 100% sure I'm on the good stack for asking my question. I would like to get a bilinear form for linear elasticity that separate a rotational part from a pure divergence part, so starting from the Navier equation &#xA;\begin{equation} \mu \nabla^2 \mathbf u +(\mu+\lambda)\nabla(\nabla \cdot \mathbf u) + \mathbf f =0 \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I use the vector Laplacian identity &#xA;$\nabla(\nabla \cdot \mathbf A)= \nabla^2 \mathbf A +\nabla  \times \nabla \times\mathbf A$  to write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation} (\lambda +2\mu) \nabla^2 \mathbf u +(\mu+\lambda)\nabla  \times (\nabla \times\mathbf u) + \mathbf f =0 \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So multiplying by a test function $\mathbf v$ I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\int_\Omega \bigg((\lambda +2\mu) \nabla^2 \mathbf u \cdot \mathbf v +(\mu+\lambda)\nabla  \times (\nabla \times\mathbf u)\cdot \mathbf v  + \mathbf f \cdot \mathbf v  \bigg) d\Omega=0 &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and using Green formula, &#xA;\begin{equation}&#xA;\int_\Omega \bigg((\lambda +2\mu) (\nabla \mathbf u :\nabla \mathbf v) +(\mu+\lambda)(\nabla  \times \mathbf v)\cdot(\nabla \times\mathbf u) + \mathbf f \cdot \mathbf v  \bigg) d\Omega=0 &#xA;\end{equation}&#xA;I'm skeptical about this for, since when I compare the bilinear form&#xA;\begin{equation}&#xA;a(\mathbf u,\mathbf v) := \int_\Omega \bigg((\lambda +2\mu) (\nabla \mathbf u :\nabla \mathbf v) +(\mu+\lambda)(\nabla  \times \mathbf v)\cdot(\nabla \times\mathbf u) \bigg) d\Omega&#xA;\end{equation}&#xA;With the classical one&lt;br&gt;&#xA;\begin{equation}&#xA;a(\mathbf u,\mathbf v) := \int_\Omega \sigma(\mathbf u):\varepsilon(\mathbf v) d\Omega&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got few different terms. For instance, in 2D the &quot;classical&quot; form (noted $a_1$) gives &#xA;\begin{eqnarray}&#xA;a_1(\mathbf u,\mathbf v) &amp;amp;=&amp;amp; \int_\Omega \sigma(\mathbf u):\varepsilon(\mathbf v) d\Omega \\&amp;amp;=&amp;amp; \bigg(2\mu \frac{\partial u_x}{\partial x}+\lambda(\frac{\partial u_x}{\partial x}+\frac{\partial u_y}{\partial y}) \bigg)\frac{\partial v_x}{\partial x}&#xA;+\mu\bigg(\frac{\partial u_y}{\partial x}+\frac{\partial u_x}{\partial y}\bigg)\bigg(\frac{\partial v_x}{\partial y}+\frac{\partial v_y}{\partial x}\bigg)&#xA;+\bigg(2\mu \frac{\partial u_y}{\partial y}+\lambda(\frac{\partial u_x}{\partial x}+\frac{\partial u_y}{\partial y})\bigg) \frac{\partial v_y}{\partial y}&#xA;\\&amp;amp;=&amp;amp;(\lambda+2\mu) \bigg(\frac{\partial u_x}{\partial x} \frac{\partial v_x}{\partial x} +\frac{\partial u_y}{\partial y}\frac{\partial v_y}{\partial y}\bigg) &#xA;+ \mu\bigg(\frac{\partial u_y}{\partial x}+\frac{\partial u_x}{\partial y}\bigg)\bigg(\frac{\partial v_x}{\partial y}+\frac{\partial v_y}{\partial x}\bigg)&#xA;+\lambda\bigg(\frac{\partial u_y}{\partial y}\frac{\partial v_x}{\partial x} +\frac{\partial u_x}{\partial x}\frac{\partial v_y}{\partial y}\bigg)\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas, the &quot;rotational&quot; form gives&#xA;\begin{eqnarray}&#xA;a_2(\mathbf u,\mathbf v) &amp;amp;=&amp;amp; \int_\Omega \bigg((\lambda +2\mu) (\nabla \mathbf u :\nabla \mathbf v) +(\mu+\lambda)(\nabla  \times \mathbf v)\cdot(\nabla \times\mathbf u) \bigg) d\Omega\\&#xA;&amp;amp;=&amp;amp; (2\mu+\lambda)\bigg(\frac{\partial u_x}{\partial x}\frac{\partial v_x}{\partial x} +&#xA;\frac{\partial u_x}{\partial y}\frac{\partial v_x}{\partial y}+\frac{\partial u_y}{\partial x}\frac{\partial v_y}{\partial x} +&#xA;\frac{\partial u_y}{\partial y}\frac{\partial v_y}{\partial y}\bigg)&#xA;+ (\lambda+\mu)\bigg(\frac{\partial u_y}{\partial x} -\frac{\partial u_x}{\partial y}\bigg)&#xA;\bigg(\frac{\partial v_x}{\partial y} -\frac{\partial v_y}{\partial x}\bigg)&#xA;\end{eqnarray}&#xA;The $(\lambda +\mu)$ terms may be developped and some terms may be factorized with some other terms of $(2\mu+\lambda)$&#xA;\begin{eqnarray}&#xA;a_2(\mathbf u,\mathbf v) &amp;amp;=&amp;amp; (2\mu+\lambda)\bigg(\frac{\partial u_x}{\partial x}\frac{\partial v_x}{\partial x}  +&#xA;\frac{\partial u_y}{\partial y}\frac{\partial v_y}{\partial y}\bigg)&#xA;+\mu\bigg(\frac{\partial u_x}{\partial y}\frac{\partial v_x}{\partial y} + \frac{\partial u_y}{\partial x}\frac{\partial v_y}{\partial x}\bigg)&#xA;+ (\lambda+\mu)\bigg(\frac{\partial u_y}{\partial x}\frac{\partial v_x}{\partial y} +\frac{\partial u_x}{\partial y}\frac{\partial v_y}{\partial x}\bigg)&#xA;\end{eqnarray}&#xA;The $\mu$ terms in two and second expressions may be factorized &#xA;\begin{eqnarray}&#xA;a_2(\mathbf u,\mathbf v) &amp;amp;=&amp;amp; (2\mu+\lambda)\bigg(\frac{\partial u_x}{\partial x}\frac{\partial v_x}{\partial x} + \frac{\partial u_y}{\partial y}\frac{\partial v_y}{\partial y}\bigg)&#xA;+\mu\bigg(\frac{\partial u_y}{\partial x}+\frac{\partial u_x}{\partial y}\bigg)\bigg(\frac{\partial v_x}{\partial y}+\frac{\partial v_y}{\partial x}\bigg)&#xA;+ \lambda\bigg(\frac{\partial u_y}{\partial x}\frac{\partial v_x}{\partial y} +\frac{\partial u_x}{\partial y}\frac{\partial v_y}{\partial x}\bigg)&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation is almost the same as $a_1$ except for the $\lambda$ term.&#xA;Could someone correct me ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: the rotational being defined by&#xA;\begin{eqnarray}&#xA;\nabla \times \mathbf u =&#xA;\left|&#xA;\begin{array}{l}&#xA;\partial_x\\&#xA;\partial_y \\&#xA;\partial_z&#xA;\end{array}&#xA;\right.&#xA;\times&#xA;\left|&#xA;\begin{array}{l}&#xA;u_x\\&#xA;u_y \\&#xA;u_z&#xA;\end{array}&#xA;\right.&#xA;=&#xA;\left|&#xA;\begin{array}{l}&#xA;\partial_y u_z -\partial_z u_y\\&#xA;\partial_z u_x -\partial_x u_z \\&#xA;\partial_x u_y -\partial_y u_x&#xA;\end{array}&#xA;\right.&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have&#xA;\begin{eqnarray}&#xA;(\nabla \times \nabla \times \mathbf u ) \cdot \mathbf v =&#xA;\left|&#xA;\begin{array}{l}&#xA;\partial_y (\partial_x u_y -\partial_y u_x) - \partial_z(\partial_z u_x -\partial_x u_z)\\&#xA;\partial_z (\partial_y u_z -\partial_z u_y) - \partial_x(\partial_x u_y -\partial_y u_x)\\&#xA;\partial_x (\partial_z u_x -\partial_x u_z) - \partial_y (\partial_y u_z -\partial_z u_y)&#xA;\end{array}&#xA;\right.&#xA;\cdot &#xA;\left|&#xA;\begin{array}{l}&#xA;v_x \\&#xA;v_y\\&#xA;v_z&#xA;\end{array}&#xA;\right.&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And using integration by part (but I suspect that the flaw stands here), &#xA;\begin{eqnarray}&#xA;\int_\Omega (\nabla \times \nabla \times \mathbf u ) \cdot \mathbf v &amp;amp;=&amp;amp;&#xA;-\int_\Omega &#xA;\partial_y v_x(\partial_x u_y -\partial_y u_x) &#xA;- \partial_z v_x (\partial_z u_x -\partial_x u_z)\\&#xA;&amp;amp;&amp;amp;+\partial_z  v_y(\partial_y u_z -\partial_z u_y) &#xA;- \partial_x v_y (\partial_x u_y -\partial_y u_x)\\&#xA;&amp;amp;&amp;amp;&#xA;+\partial_x v_z(\partial_z u_x -\partial_x u_z) &#xA;- \partial_y v_z(\partial_y u_z -\partial_z u_y)\\&#xA;&amp;amp;=&amp;amp; - \int_\Omega (\partial_y v_x - \partial_x v_y )(\partial_x u_y -\partial_y u_x) +&#xA;(\partial_z  v_y-\partial_y v_z) (\partial_y u_z -\partial_z u_y) &#xA;+(\partial_x v_z- \partial_z v_x )(\partial_z u_x -\partial_x u_z)\\&#xA;&amp;amp;=&amp;amp; \int_\Omega (\nabla \times \mathbf v )\cdot (\nabla \times \mathbf u)&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="2116" LastEditDate="2012-12-17T17:42:03.353" LastActivityDate="2014-11-17T08:12:23.823" Title="variational formulation of linear elasticity" Tags="|pde|well-posedness|elliptic-pde|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4865" PostTypeId="1" AcceptedAnswerId="4899" CreationDate="2012-12-14T16:04:25.200" Score="2" ViewCount="86" Body="&lt;p&gt;It is stated in &#xA;&lt;a href=&quot;http://users.eecs.northwestern.edu/~yingwu/teaching/EECS510/Reading/Williams_NIPS01.pdf&quot; rel=&quot;nofollow&quot;&gt;http://users.eecs.northwestern.edu/~yingwu/teaching/EECS510/Reading/Williams_NIPS01.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that the PCA mapping from $h$-dimensional data to low $k$-dimensional space minimizes $$\sum_{i,~j} d_{ij}^2-g_{ij}^2,$$ subject to orthonormality of direction vectors. Distances $d_{ij}$ and $g_{ij}$ correspond to distances between items $i,j$ in a high and low-dimensional space, respectively. Can it be shown that PCA also minimizes $$\sum_{i,~j} (d_{ij}-g_{ij})^2,$$subject to orthonormality of direction vectors? &lt;/p&gt;&#xA;" OwnerUserId="1069" LastActivityDate="2012-12-21T22:39:44.183" Title="Optimality criterion of PCA via recovered distances" Tags="|linear-algebra|matrix|machine-learning|high-dimensional|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4866" PostTypeId="1" CreationDate="2012-12-14T17:18:35.293" Score="2" ViewCount="145" Body="&lt;p&gt;I'm solving a simple 1D heat diffusion problem $$u_t=u_{xx},\quad \Omega\times[0,T]$$ $$u=0\quad, \partial\Omega\times [0,T]$$ $$u(x,0)=f$$using a fully discrete galerkin finite element method.  This effectively reduces solution of the parabolic problem into solving a recursive sequence of elliptic problems $\{U_h\}^t$, each using the solution from the previous timestep $\{U_h\}^{t-1}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been told that it is not a good idea to use the initial condition $f$ as $\{U_h\}^0$.  Instead, it has been recommended to me to use the projection of $f$ onto the variational space chosen for the weak formulation as $\{U_h\}^0$.  I have run some numerical experiments using $f$ instead of its projection and I observe that &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The errors still approach zero as the spatial and temporal step sizes approach zero.&lt;/li&gt;&#xA;&lt;li&gt;The orders of convergence are still preserved.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Heuristically, it seems to me that projecting the initial condition would introduce more error into the problem than using the $f$ as is.  These numerical experiments seem to indicate that it really isn't necessary to obtain consistent, stable solutions (at least, for this problem).  Are there situations in which it is crucial to project the initial condition, or is more a matter of taste/mathematical formality?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2012-12-16T00:43:33.110" Title="Is it necessary to project the initial condition onto the variational space in a fully discrete galerkin method?" Tags="|pde|finite-element|parabolic-pde|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4868" PostTypeId="2" ParentId="4866" CreationDate="2012-12-15T19:21:19.597" Score="3" Body="&lt;p&gt;It depends on how you define &quot;projection&quot; and what scheme you are using. But let's investigate the backward Euler method. There, you need to solve the following discrete problem in every time step:&#xA;$$&#xA;  (v_h,U_h^n) + \Delta t (\nabla v_h, \nabla u_h^n)&#xA;  =&#xA;  (v_h, U_h^{n-1}).&#xA;$$&#xA;The question is what to use in the first time step: Either&#xA;$$&#xA;  (v_h,U_h^1) + \Delta t (\nabla v_h, \nabla u_h^1)&#xA;  =&#xA;  (v_h, f)&#xA;$$&#xA;or &#xA;$$&#xA;  (v_h,U_h^1) + \Delta t (\nabla v_h, \nabla u_h^1)&#xA;  =&#xA;  (v_h, \Pi f)&#xA;$$&#xA;where $\Pi$ is the projection onto the finite element space. Let's assume that you mean the $L_2$ projection, then $\Pi f$ is defined as follows:&#xA;$$&#xA;  (v_h, \Pi f) = (v_h, f) \qquad \forall v_h\in V_h.&#xA;$$&#xA;In other words, for this time discretization and the $L_2$ norm, the two alternatives for the first time step are exactly the same!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not always the case. For example, for the explicit Euler scheme (unsuitable for the heat equation, but instructive here), alternative 1 would yield the following for the first time step:&#xA;$$&#xA;  (v_h,U_h^1)&#xA;  =&#xA;  - \Delta t (\nabla v_h, \nabla f)+&#xA;  (v_h, f).&#xA;$$&#xA;Alternative 2 would be&#xA;$$&#xA;  (v_h,U_h^1)&#xA;  =&#xA;  - \Delta t (\nabla v_h, \nabla \Pi f)+&#xA;  (v_h, \Pi f).&#xA;$$&#xA;This is obviously not the same. Sometimes, however, you can define a different projector in such a way that the two match again. (Here, this is not possible due to the sign of the first term on the right; if it had been positive, you could just consider the two right hand sides as defining a projector $\Pi$.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2012-12-16T00:43:33.110" LastActivityDate="2012-12-16T00:43:33.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4869" PostTypeId="2" ParentId="4695" CreationDate="2012-12-15T22:47:33.470" Score="1" Body="&lt;p&gt;Interesting question, I'm studying computational material science, and I'm a grad student from a physics major background. So far the amount of programming I've done is minimal, but that's speaking for myself. Often, the specific goals of your project will determine the exact amount of coding that you will do. If it's a look at a particular compound using standard techniques and software, at worst you will end up writing Bash/Python scripts to automate things around the code your are using. I'm of the opinion that you will do some programming possibly to implement the stuff you seek to validate, so short answer: can't speak for computational science, but for computational material science, the amount of programming if any will be determined by your project. &lt;/p&gt;&#xA;" OwnerUserId="2093" LastEditorUserId="276" LastEditDate="2013-02-28T04:30:36.240" LastActivityDate="2013-02-28T04:30:36.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4870" PostTypeId="2" ParentId="4826" CreationDate="2012-12-15T23:04:11.627" Score="4" Body="&lt;p&gt;Here's the&#xA;&lt;a href=&quot;https://github.com/argriffing/algopy/blob/master/documentation/sphinx/examples/logistic_regression.py&quot; rel=&quot;nofollow&quot;&gt;code&lt;/a&gt;&#xA;and the&#xA;&lt;a href=&quot;https://github.com/argriffing/algopy/blob/master/documentation/sphinx/examples/logistic_regression.rst&quot; rel=&quot;nofollow&quot;&gt;output&lt;/a&gt;&#xA;of a solution that uses the development version of&#xA;&lt;a href=&quot;http://packages.python.org/algopy/&quot; rel=&quot;nofollow&quot;&gt;algopy&lt;/a&gt; for&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Automatic_differentiation&quot; rel=&quot;nofollow&quot;&gt;automatic differentiation&lt;/a&gt;.  As someone else said in the comments, you could also try other python packages.  Theano and statsmodels come to mind.&lt;/p&gt;&#xA;" OwnerUserId="3443" LastEditorUserId="1804" LastEditDate="2012-12-16T00:16:38.717" LastActivityDate="2012-12-16T00:16:38.717" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4871" PostTypeId="1" AcceptedAnswerId="4872" CreationDate="2012-12-16T22:36:54.510" Score="3" ViewCount="1179" Body="&lt;p&gt;I often see that it is common to put &quot;more points&quot; in the region of interest in the computational domain of the numerical method, i.e. use non-uniform grid. The proofs are usually done for the uniform grid though and then using some arguments like &quot;the proof can be extended to non-uniform grid as well&quot; the results show good errors obtained. Ok, even if the theory behind non-uniform grid is the same, I was thinking how to generate those non-uniform grids and I did not find that many articles. The simplest way, perhaps, is to use some sort of &quot;generating&quot; function, something like a cubic polynomial that does create a desired non-uniformity controlled by the polynomial coefficients. However, there are spectral methods such as Chebyshev discretization(I don't know much about) that can also be utilized. Thus, I wonder if there is any advantage of using it over a cubic function mapping? In either case I would approximate derivatives as on a non-uniform grid so the local error of the first and second derivative would be bounded by $O((\max_{i\in I}\{x_{i+1}-x_i\})^2)$.  &lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2012-12-16T23:31:26.283" Title="generating a non-uniform grid with Chebyshev discretization" Tags="|pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4872" PostTypeId="2" ParentId="4871" CreationDate="2012-12-16T23:31:26.283" Score="6" Body="&lt;p&gt;The main use for Chebyshev points is with a (pseudo)spectral method. The Chebyshev points are good (&quot;optimal&quot;) for $L^\infty$ approximation and derivatives can be evaluated in $O(n \log n)$ by FFT. If you are going to use a compact spatial discretization (such as FD, FV, or FE) then there is no advantage to Chebyshev points unless the &quot;interesting&quot; regions just happen to coincide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To create topologically structured grids that place high resolution in interesting regions, you should check out &lt;strong&gt;elliptic mesh generation&lt;/strong&gt;. In this approach, you provide a &quot;monitor function&quot; that indicates the desired resolution (and anisotropy) and the grid generator solves a (typically nonlinear) elliptic PDE to compute the location of the nodes in the mesh. There are several good books and review articles, e.g.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Huang and Russell, &lt;a href=&quot;http://books.google.com/books/about/Adaptive_Moving_Mesh_Methods.html?id=Pn9_zDG6qYUC&quot;&gt;&lt;em&gt;Adaptive Moving Mesh Methods&lt;/em&gt;&lt;/a&gt;, Springer, 2011.&lt;/li&gt;&#xA;&lt;li&gt;Liseikin, &lt;a href=&quot;http://books.google.com/books/about/Grid_Generation_Methods.html?id=Kj1LhJxcF3sC&quot;&gt;&lt;em&gt;Grid Generation Methods&lt;/em&gt;&lt;/a&gt;, Springer, 2009.&lt;/li&gt;&#xA;&lt;li&gt;Budd, Huang, and Russell, &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CDUQFjAA&amp;amp;url=http%3A%2F%2Fpeople.bath.ac.uk%2Fmascjb%2FPapers09%2Fbudd090403.pdf&amp;amp;ei=7FjOUJrJHJSBrQGfm4HQBw&amp;amp;usg=AFQjCNFCifAcj95iyNidHgTl0pu_sWkXRg&amp;amp;bvm=bv.1355325884,d.aWM&quot;&gt;&lt;em&gt;Adaptivity with moving grids&lt;/em&gt;&lt;/a&gt;, Acta Numerica, 2009.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-12-16T23:31:26.283" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4873" PostTypeId="2" ParentId="1338" CreationDate="2012-12-18T00:55:46.563" Score="6" Body="&lt;p&gt;You are talking about computation of the Hermite Normal Form (HNF), which is covered in books by &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0471982326&quot; rel=&quot;nofollow&quot;&gt;Schrijver&lt;/a&gt; and by &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0471359432&quot; rel=&quot;nofollow&quot;&gt;Nemhauser and Wolsey&lt;/a&gt;.    In the theory of lattices and integer matrices, the HNF plays a big role.  It is like an integer QR factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A square integer matrix with determinant $\pm 1$ is called a unimodular matrix.&#xA;By Cramer's rule, the inverse of a unimodular matrix is integer and unimodular.  Thus, the unimodular matrices are isomorphisms of $\mathbb{Z}^m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that $M$ is $m \times n$, with linearly independent rows, $m \le n$, and integer entries so that its rows are a basis of a sub-lattice of $\mathbb{Z}^n$. (A lattice in general is the set of integer linear combinations of a set of linearly independent vectors.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E$ is the HNF of the lattice determined by the rows of $M$.  It will have an upper triangular form, with the additional property that the first nonzero of each row is bigger than the elements above it.   It is a normal form in the sense that if two lattices are identical, then they have the same $E$, the same HNF.   The HNF can be useful in answering a variety of lattice related questions.  The most basic one is the question of whether a given vector $b$ in $\mathbb{Z}^n$ is in the lattice of $M$'s rows: It is iff $\text{inv}(U)b$ is in the lattice of $E$'s rows, and the triangular structure of $E$ makes this easy to check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The elements of $E$ and $U$ may be very large integers, and controlling the growth of the elements is the key algorithmic challenge.   If one ignores this, its easy to derive an algorithm by finding a unimodular &quot;Givens&quot; type transformation -- a $2 \times 2$ unimodular transformation that can zero one element of an integer 2-vector.  Integer gcd is a subroutine of the construction of one of these.  With them, one can introduce zeros so as to create the desired upper triangular form.&lt;/p&gt;&#xA;" OwnerUserId="3340" LastEditorUserId="67" LastEditDate="2012-12-19T20:16:52.010" LastActivityDate="2012-12-19T20:16:52.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4874" PostTypeId="2" ParentId="2853" CreationDate="2012-12-18T01:16:15.947" Score="18" Body="&lt;p&gt;I actually wrote the original code in Matlab for A*B, both A and B sparse.  Pre-allocation of space for the result was indeed the interesting part.   We observed what Godric points out -- that knowing the number of nonzeros in AB is as costly as computing AB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We did the initial implementaion of sparse Matlab around 1990, before the Edith Cohen paper that gave the first practical, fast way to estimate the size of AB accurately.&#xA;We put together an inferior size estimator, and if we ran out of space in mid-computation, doubled the allocation and copied the partially computed result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what's in Matlab now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility would be to compute AB one column at a time.   Each column can be stored temporarily in a sparse accumulator (see the sparse Matlab paper for an explanation of these), and space allocated to hold the exactly known size of the result column.   The result would be in scattered compressed sparse column form -- each column in CSC but no intercolumn contiguity -- using 2 vectors of length numcols (col start, col length), rather than one, as meta-data.   Its a storage form that may be worth a look; &#xA;it has another strength -- you can grow a column without reallocating the whole matrix.&lt;/p&gt;&#xA;" OwnerUserId="3340" LastActivityDate="2012-12-18T01:16:15.947" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4875" PostTypeId="1" CreationDate="2012-12-18T13:56:52.397" Score="1" ViewCount="154" Body="&lt;p&gt;I have the following eliptic 1-D problem&#xA;$$-\mu u'' + \beta u' = 1$$&#xA;$$u(0) = u'(1) = 1$$&#xA;where $\mu = 10e^{-5}$ and $\beta = 1$. For this specific problem I am using the following space steps $h=[0.1, 0.01]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What surprises me is that this problem doesn't present oscillations. I think it is related to the Péclet number, $\mathrm{Pe}$. When the problem had a source term (right-hand side function) equals to zero I know that the Péclet number should be less than 1 to avoid this oscillations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this lack of oscillations related to the local Péclet number of my problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use finite differences to see if I can find why this does not present oscillations but I don't know how to proceed from here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\textrm{Pe}-1)u_{i+1} + 2u_i - (\textrm{Pe}+1)u_{i-1} = \frac{h^2}{\mu}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how to proceed or if I'm actually in the right direction?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="276" LastEditDate="2012-12-18T15:29:06.653" LastActivityDate="2013-01-18T08:48:36.253" Title="FEM oscillations for polynomials of degree 1" Tags="|finite-element|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4876" PostTypeId="2" ParentId="4708" CreationDate="2012-12-18T17:05:51.767" Score="3" Body="&lt;p&gt;Visualizing 6 dimensional domains is simply not easy. Unless of course, your uber-dimensional monitor is back form the repairman. Getting parts from the future is never a quick thing to do however, so mine languishes in a back room with my busted Holodeck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kidding aside, visualizing a basin in 6-d really is not easy. Even computing the limits of a basin of attraction will be difficult. The curse of dimensionality hounds you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, even in a lower number of dimensions, identifying the boundaries of such a basin requires solving MANY optimization problems. After all, a basin of attraction need not be a convex set. It need not be connected. And, since an optimizer, starting from distinct starting values will yield results that are still distinct, you must now do some clustering, testing that the multiple solutions truly are the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other issues of course. Suppose I ask to minimize the function (x-y)^2 in the (x,y) plane? Clearly any point on the line y=x is a solution, and all are equally good. But clustering will have problems here, as they will on any such degeneracies, and identifying degeneracies in 6-d is not always trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, you ask about identifying the NUMBER of local minimizers in any bounded region. The is too is quite difficult for a general black box problem. The field of global optimization has been working on problems like this for many years, though I don't think they can give you any hard, easy to compute answers in general.&lt;/p&gt;&#xA;" OwnerDisplayName="user840" LastActivityDate="2012-12-18T17:05:51.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4879" PostTypeId="2" ParentId="4875" CreationDate="2012-12-19T08:24:52.930" Score="1" Body="&lt;p&gt;The finite difference / volume case is covered for example in Wesseling, Principles of Computational Fluid Dynamics, Springer 2001. The main idea is to take a solution of the form $u_i=A+Bz^i$, fill in and see whether it allows solutions with $z&amp;lt;0$ so that $z^i$ alternates in sign (wiggles). It is important to include boundary conditions as well.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2012-12-19T08:24:52.930" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4880" PostTypeId="1" CreationDate="2012-12-19T09:58:25.580" Score="1" ViewCount="103" Body="&lt;p&gt;Consider a set of 7D vectors. Each vector belongs to one of four classes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After mapping to 3D with PCA and coloring each point according its class the dataset looks like as shown below: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/3p7zq.jpg&quot; alt=&quot;pca3-4&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the animated GIF version, click here &lt;a href=&quot;https://docs.google.com/open?id=0B26-1RYIJR_oOWRmSXZyWUl4M2M&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/open?id=0B26-1RYIJR_oOWRmSXZyWUl4M2M&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What classification technique would be suitable for this dataset (e.g. decision trees, neural networks, etc) ?&lt;/p&gt;&#xA;" OwnerUserId="1922" LastEditorUserId="3299" LastEditDate="2012-12-27T03:24:05.227" LastActivityDate="2013-01-10T01:20:37.990" Title="classification machinery needed" Tags="|machine-learning|data-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4882" PostTypeId="1" CreationDate="2012-12-19T10:48:40.697" Score="2" ViewCount="1014" Body="&lt;p&gt;I have a comsol model. I want to draw a few plots for different values of a global parameter called &lt;code&gt;plot_z&lt;/code&gt;, e.g. if &lt;code&gt;plot_z&lt;/code&gt; can have 5 different values I want to draw 5 different plots, one for each value. In each plot I need to write the value of &lt;code&gt;plot_z&lt;/code&gt; parameter in the plot title. I have typed many expressions such as &lt;code&gt;plot_z&lt;/code&gt;, &lt;code&gt;$plot_z&lt;/code&gt;, &lt;code&gt;%plot_z%&lt;/code&gt;, etc. in the plot title text box, but unfortunately none of them worked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it even possible to have the value of a global parameter in plot title?&lt;/p&gt;&#xA;" OwnerUserId="3457" LastActivityDate="2014-06-08T05:32:28.780" Title="How to include parameters in plot title in comsol?" Tags="|finite-element|software|comsol|" AnswerCount="1" CommentCount="4" ClosedDate="2014-06-09T15:23:51.577" ContentLicense="CC BY-SA 3.0" />
  <row Id="4883" PostTypeId="1" CreationDate="2012-12-19T16:20:53.850" Score="11" ViewCount="1207" Body="&lt;p&gt;I am solving $Ax=b$ for a huge sparse positive definite matrix $A$ using the conjugate gradient (CG) method. It is possible to compute the determinant of $A$ using the information produced during the solve?&lt;/p&gt;&#xA;" OwnerUserId="3459" LastEditorUserId="67" LastEditDate="2012-12-19T18:56:04.353" LastActivityDate="2014-10-06T06:48:26.453" Title="Calculating determinant while solving $Ax=b$ using CG" Tags="|linear-algebra|optimization|krylov-method|conjugate-gradient|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4884" PostTypeId="2" ParentId="4883" CreationDate="2012-12-19T17:39:26.697" Score="10" Body="&lt;p&gt;Computing the determinant of a sparse matrix is typically as expensive as a direct solve, and I am skeptical that CG would be of much help in computing it. It would be possible to run CG for $n$ iterations (where $A$ is $n \times n$) in order to generate information for the entire spectrum of $A$, and to then compute the determinant as the product of the eigenvalues, but this would be both slow and numerically unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be a better idea to compute the sparse-direct Cholesky factorization of your matrix, say $A = L L^H$, where $L$ is lower-triangular.&#xA;Then&#xA;$$&#xA;\text{det}(A) = \text{det}(L) \text{det}(L^H) = |\text{det}(L)|^2,&#xA;$$&#xA;where $\text{det}(L)$ is simply the product of the diagonal entries of the lower-triangular matrix $L$ since the eigenvalues of a triangular matrix lie along its diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a general non-singular matrix, a pivoted LU decomposition should be used, say $PA=LU$, where $P$ is a permutation matrix, so that&#xA;$$&#xA;\text{det}(A) = \text{det}(P^{-1}) \cdot \text{det}(L) \cdot \text{det}(U).&#xA;$$&#xA;Since $P$ is a permutation matrix, $\text{det}(P)=\pm 1$, and, by construction, $L$ will typically have a diagonal of all ones, which implies that $\text{det}(L)=1$. You can thus compute $\text{det}(A)$ as $\pm \text{det}(U)$ and again recognize that the determinant of a triangular matrix is simply the product of its diagonal entries. Thus, the cost of computing the determinant is essentially just that of a factorization.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2012-12-19T19:00:06.523" LastActivityDate="2012-12-19T19:00:06.523" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="4885" PostTypeId="1" CreationDate="2012-12-19T21:19:41.040" Score="10" ViewCount="1835" Body="&lt;p&gt;I am new to the PETSc package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a ~4000x4000 matrix A in matrix-market format and I want to get PETSc to solve this using multiple processors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to solve the system on a single processor, but I don't know how&#xA;to distribute the matrix and vectors among different processors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simple set of instructions to do this?&lt;/p&gt;&#xA;" OwnerUserId="3352" LastActivityDate="2012-12-20T01:24:39.347" Title="Solving a simple Ax=b system in parallel with PETSc" Tags="|petsc|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4886" PostTypeId="2" ParentId="4885" CreationDate="2012-12-20T01:24:39.347" Score="14" Body="&lt;p&gt;Matrix Market is a terrible format for reading in parallel, therefore it is better to preprocess to a better parallel format. Your matrix size is extremely small so performance is not an issue, but the easiest and most general thing is to use Python or Matlab/Octave to write the Matrix Market file in PETSc binary format, which can be read efficiently in parallel using &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatLoad.html&quot;&gt;&lt;code&gt;MatLoad()&lt;/code&gt;&lt;/a&gt;. For example, you can use this Python code for the preprocessing (add &lt;code&gt;$PETSC_DIR/bin/pythonscripts&lt;/code&gt; to your &lt;code&gt;PYTHONPATH&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.io, PetscBinaryIO&#xA;A = scipy.io.mmread('thematrix.mtx')&#xA;PetscBinaryIO.PetscBinaryIO().writeMatSciPy(open('petscmatrix','w'), A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also write a vector to the file at this time. If you just want to read and solve the system, you can use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex10.c.html&quot;&gt;&lt;code&gt;src/ksp/ksp/examples/tutorials/ex10.c&lt;/code&gt;&lt;/a&gt; (with the option &lt;code&gt;-f petscmatrix&lt;/code&gt; to read the binary file you just wrote).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a real application, you should avoid a workflow that involves writing files to disk in any format. It is much better to assemble the matrix in parallel using a domain-decomposed representation of the problem. Most examples in PETSc are written this way.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-12-20T01:24:39.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4888" PostTypeId="1" AcceptedAnswerId="4909" CreationDate="2012-12-20T17:26:21.040" Score="8" ViewCount="568" Body="&lt;p&gt;I would like to know the which is more advantageous when it comes to solving nonlinear hyperbolic equations, Finite Element or Finite difference methods? Which method will be better in capturing shocks? Is it possible to provide a detailed answer/references?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I want to solve problems with non-reflecting boundary conditions in infinite waveguides, can I use Sommerfeld radiation condition in such cases?&lt;/p&gt;&#xA;" OwnerUserId="3467" LastActivityDate="2012-12-25T06:24:49.690" Title="Nonlinear wave equation - Finite element or finite difference" Tags="|finite-element|finite-difference|hyperbolic-pde|nonlinear-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4889" PostTypeId="1" AcceptedAnswerId="4891" CreationDate="2012-12-20T17:34:40.403" Score="8" ViewCount="4040" Body="&lt;p&gt;Assume that we are given a block matrix of the form:&#xA;$$&#xA;M = &#xA;\left[&#xA;\begin{array}{cc}&#xA;A   &amp;amp; b \\&#xA;b^T &amp;amp; c \\&#xA;\end{array}&#xA;\right]&#xA;$$&#xA;where $b$ is a column vector. and $c$ is a scalar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Schur's complement of $A$ in $M$ is given by:&#xA;$$ s = c - b^T A^{-1} b $$&#xA;Assume that we know $A$ is invertible and we want to check if $M$ is invertible. Is it true that $M$ is invertible &lt;em&gt;if, and only if&lt;/em&gt; $s$ is different than zero?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="1932" LastActivityDate="2012-12-20T22:25:59.677" Title="Schur's Complement and Inverse of Block Matrices" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4890" PostTypeId="1" AcceptedAnswerId="24132" CreationDate="2012-12-20T18:12:46.730" Score="3" ViewCount="4068" Body="&lt;p&gt;Does somebody know about any tool for simulation and visualization of magnetic fields either of permanent magnets or solenoids, that can be used on Mac OS X? Maybe there is some software for Linux, but is open-source and can compiled for Mac OS X?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to have something like a canvas at which I can put the magnets, move them, and see how the magnetic field changes around them.&lt;/p&gt;&#xA;" OwnerUserId="3468" LastEditorUserId="9667" LastEditDate="2018-04-25T15:58:50.407" LastActivityDate="2018-04-25T15:58:50.407" Title="Magnetic field simulation and visualization on Mac OS X?" Tags="|simulation|visualization|electromagnetism|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4891" PostTypeId="2" ParentId="4889" CreationDate="2012-12-20T18:15:44.417" Score="13" Body="&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under the assumption that $A^{-1}$ exists, then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\det (M) = \det (A) s.&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\det(A) \neq 0$, $\det (M) \neq 0$ if and only if $s \neq 0$.  Thus $M$ is inveritible if and only if $s \neq 0$.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This theorem on block determinants is standard material that can be found in many linear algebra textbooks.  See for example Carl Meyer's textbook, &quot;Matrix Analysis and Linear Algebra.&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proof is pretty simple- write $M$ as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;M=\left[ \begin{array}{cc}&#xA;I &amp;amp; 0 \\&#xA;b^{T}A^{-1} &amp;amp; 1 \\&#xA;\end{array}&#xA;\right]&#xA;\left[ \begin{array}{cc}&#xA;A &amp;amp; b \\&#xA;0 &amp;amp; c-b^{T}A^{-1}b \\&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then apply the product rule and the rule for determinants of (block) diagonal matrices.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2012-12-20T22:25:59.677" LastActivityDate="2012-12-20T22:25:59.677" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4892" PostTypeId="2" ParentId="4890" CreationDate="2012-12-20T18:29:45.090" Score="1" Body="&lt;p&gt;Have you tried either &lt;a href=&quot;https://wci.llnl.gov/codes/visit/home.html&quot; rel=&quot;nofollow&quot;&gt;VisIt&lt;/a&gt; or &lt;a href=&quot;http://www.paraview.org&quot; rel=&quot;nofollow&quot;&gt;ParaView&lt;/a&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2012-12-20T18:29:45.090" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4893" PostTypeId="1" AcceptedAnswerId="4895" CreationDate="2012-12-20T20:55:25.923" Score="6" ViewCount="838" Body="&lt;p&gt;I've just started a thesis on nonlinear dynamics which entails numerical analysis of the &lt;a href=&quot;http://mathworld.wolfram.com/DuffingDifferentialEquation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Duffing oscillator&lt;/a&gt; (DO). It's basically just a second order ODE, or equivalently a set of ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say, after integration, I have data that looks like this:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/FhM2c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to write an algorithm that 1) analyzes the data, 2) &lt;strong&gt;outputs the period of the DO&lt;/strong&gt; for that particular set of parameters (after a certain but arbitrary(_ily adjusted) relaxation time). In the present case, the period is 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would probably not be hard to write the algorithm if the DO behaved exactly the same way in each period, but it doesn't (necessarily, as I undersand it) - so the algorithm has to be somewhat 'fuzzy'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas, links?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Python/MATLAB, but feel free to give a language-agnostic answer :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers!,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\T&lt;/p&gt;&#xA;" OwnerUserId="742" LastActivityDate="2012-12-21T05:07:47.840" Title="Nonlinear dynamics: algorithm suggest" Tags="|algorithms|matlab|python|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4894" PostTypeId="2" ParentId="4890" CreationDate="2012-12-20T21:24:03.917" Score="0" Body="&lt;p&gt;Try &lt;a href=&quot;http://code.enthought.com/projects/mayavi/&quot; rel=&quot;nofollow&quot;&gt;Mayavi&lt;/a&gt;. Look &lt;a href=&quot;http://mindseye.no/2010/09/25/using-mayavi-to-visualize-electric-fields/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://mindseye.no/2010/10/07/the-beauty-of-mayavi/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for some nice examples of electric field visualizations.&lt;/p&gt;&#xA;" OwnerUserId="1285" LastActivityDate="2012-12-20T21:24:03.917" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4895" PostTypeId="2" ParentId="4893" CreationDate="2012-12-21T05:07:47.840" Score="2" Body="&lt;p&gt;One possible approach could consist of using &lt;a href=&quot;http://en.wikipedia.org/wiki/Poincar%C3%A9_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;Poincaré maps&lt;/a&gt; while keeping in mind that fixed points in the corresponding Poincaré sections may signal the presence of periodic orbits.  You could combine this idea with a clustering algorithm to isolate regions in the section that correspond to &quot;almost&quot; the same point (i.e., trajectory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xwQMn.png&quot; alt=&quot;Duffing Oscillator: time series and Poincaré section for a certain time&quot;&gt; in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the figure above you can see a time series of the Duffing Oscillator and a Poincaré map where the periodic orbit is the fixed point represented by a black dot.  In the figure below, there's the complete phase space and also a plot of the periodic trajectory (after relaxation) that corresponds to the fixed point (black dot) in the Poincaré section.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ofbri.png&quot; alt=&quot;Phase space and periodic orbit&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2012-12-21T05:07:47.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4896" PostTypeId="1" AcceptedAnswerId="4897" CreationDate="2012-12-21T10:04:49.797" Score="3" ViewCount="169" Body="&lt;p&gt;If I have the following problem&#xA;$$-\mu u'' + u' = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions $u(0) = u'(1) = 1$ in the interval $\Omega = (0,1)$. The exact solution is $$u(x) = x + 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will the FEM approximation with piecewise linear functions ($\mathbb{P}^1$) have a very small  (or no) error? If so, how to think about it to have an image in my mind?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2012-12-21T17:24:27.267" Title="Approximation of a linear function with polynomials of degree 1" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4897" PostTypeId="2" ParentId="4896" CreationDate="2012-12-21T11:07:12.513" Score="4" Body="&lt;p&gt;By &lt;a href=&quot;http://en.wikipedia.org/wiki/C%C3%A9a%27s_lemma&quot; rel=&quot;nofollow&quot;&gt;Céa's lemma&lt;/a&gt;, the finite element discretization error is bounded by the best approximation error:&#xA;$$\|u-u_h\|_{H^1(0,1)} \leq C \min_{v_h\in \mathbb{P}^1} \|u-v_h\|_{H^1(0,1)},$$&#xA;where $u\in H^1(0,1)$ is the exact solution, $u_h\in\mathbb{P}^1$ is the finite element solution.&#xA;Since in your case the exact solution $u$ is actually in $\mathbb{P}^1$, the minimum is attained for $v_h = u$, and so the error is zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you are solving the equation by projection onto a finite-dimensional subspace. If your solution actually lies in this subspace, you will find it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The key idea is the following. You are looking for a weak solution, i.e., $u\in V$ such that &#xA;$$a(u,v) = f(v) \quad\text{for all} \quad v\in V.\tag{1}$$ &#xA;(In your case, $a(u,v) = \int_0^1 \mu u'v' + \int_0^1 u' v$ and $f(v) = \int_0^1 v + v(1)$.) The finite element approximation consists in choosing $V_h\subset V$ (in your case, $V_h = \mathbb{P}^1$) and finding $u_h\in V_h$ such that $$a(u_h,v_h) = f(v_h) \quad\text{for all} \quad v_h\in V_h.\tag{2}$$ Setting $v=v_h\in V$ in $(1)$ and subtracting $(2)$ gives&#xA;$$a(u-u_h,v_h) = 0 \quad \text{for all}\quad v_h\in V_h.$$&#xA;This tells you that your error $u-u_h$ is orthogonal to $V_h$ with respect to the &quot;inner product&quot; defined by $a$, or that $u_h$ is the orthogonal projection of $u$ on $V_h$ (again, with respect to $a$). And orthogonal projections on a subspace are characterized by best approximation properties with respect to appropriate norms (in this case, the $H^1$ norm since $a$ happens to be coercive in this norm).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2012-12-21T17:24:27.267" LastActivityDate="2012-12-21T17:24:27.267" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4898" PostTypeId="2" ParentId="3156" CreationDate="2012-12-21T18:52:08.240" Score="5" Body="&lt;p&gt;Unfortunately, in SciPy 0.11 or earlier, the erf function is &lt;a href=&quot;http://projects.scipy.org/scipy/ticket/1207&quot;&gt;rather inaccurate&lt;/a&gt; for complex arguments.  However, SciPy 0.12 will contain accurate complex erf support and includes an &lt;a href=&quot;http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.special.erfi.html&quot;&gt;optimized erfi function&lt;/a&gt;.   Alternatively, you can use the &lt;code&gt;scipy.special.dawsn&lt;/code&gt; function, which computes the Dawson function, a scaled erfi function.  erfi(x) is then given by (2/sqrt(pi)) * exp(x^2) * dawsn(x).&lt;/p&gt;&#xA;" OwnerUserId="3474" LastActivityDate="2012-12-21T18:52:08.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4899" PostTypeId="2" ParentId="4865" CreationDate="2012-12-21T22:39:44.183" Score="1" Body="&lt;p&gt;Consider, in some &quot;high dimensional&quot; Euclidean space $\mathbb{R}^n$, a set $\mathscr{S}$&#xA;of points and all the unordered distinct pairs (edges) $\mathscr{E}$ drawn from that set.&#xA;Denote the usual Euclidean distance for each pair $e \in \mathscr{E}$ by $d_e$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We ask about orthogonally projecting $\mathscr{S}$ onto a lower dimensional subspace, say&#xA;of dimension $k$, so that the (reduced) distances $\delta_e$ between projected pairs $\pi(e)$&#xA;collectively best approximate the original distances $d_e$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two optimality criteria are described in this question, the first being minimization of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (1) \; \sum_{e \in \mathscr{E}} d_e^2 - \delta_e^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the second being a least squares formulation, minimizing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (2) \; \sum_{e \in \mathscr{E}} (d_e - \delta_e)^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question posed is whether a solution optimal for the first criterion is necessarily&#xA;optimal in the second sense.  We propose a simple example showing this is not the case,&#xA;taking the 3-4-5 right triangle in the plane and orthogonally projecting it onto a line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since rigid motions (translation, rotation, reflection) preserve distances, a simple&#xA;setup suffices.  Letting $\mathscr{S} = \{ (0,0),(3,0),(0,4) \}$, we ask what&#xA;(counterclockwise) rotations by angle $\theta$ (about the origin) yield the respective&#xA;minimums of (1) and (2) if rotated points are orthogonally projected onto the $x$-axis.&#xA;Notice that $\pi(0,0) = 0$ regardless of the angle chosen, but $\pi(3,0) = 3\cos(\theta)$&#xA;and $\pi(0,4) = -4\sin(\theta)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a general observation (1) adds certain constants for the original distances $d_e^2$&#xA;and subtracts a sum of projected distances squared $\delta_e^2$ depending on the choice&#xA;of projection.  Thus minimizing (1) amounts to maximizing the latter sum.  Because the&#xA;projected distances are bounded above by the original distances (as alluded to earlier),&#xA;maximizing the sum of squares of projected distances tends to produce reasonable&#xA;approximations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While a general method exists for solving (1) based on the &quot;principal components&quot; &#xA;(eigenvectors) of a symmetric matrix (and we can circle back to that approach), let's&#xA;first write out the subtracted part of (1) as a function of angle $\theta$ and perform&#xA;its necessary maximization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(\theta) = \sum \delta_e^2 = 9\cos^2(\theta) + 16\sin^2(\theta)&#xA;+ (3\cos(\theta) + 4\sin(\theta))^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simplifying with familiar trigonometric identities reveals that the function has&#xA;period $\pi$ (rather than $2\pi$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(\theta) = 25 -7\cos(2\theta) +12\sin(2\theta) = 25 +\sqrt{193}\sin(2\theta -\alpha) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha = \tan^{-1}(\frac{7}{12}) \approx 0.52807445$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By inspection this function $f(\theta)$ has a maximum value $25 +\sqrt{193}$.  The location&#xA;of this (global) maximum (unique within a period $\pi$) is thus:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 2\theta_1 - \alpha = \frac{\pi}{2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \theta_1 = \frac{\pi}{4} + \frac{\alpha}{2} \approx 1.04943539 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus an exact minimum of (1) (resp. maximum of $f(\theta)$) is known, so we next turn&#xA;to its relationship with the minimization of (2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expanding (2) shows that it contains, among other terms, the same constant $\sum d_e^2$ &#xA;that we previously pointed out in (1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum (d_e - \delta_e)^2 = \sum d_e^2 - 2\sum d_e \delta_e + \sum \delta_e^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the last summation is the same as term $f(\theta)$ whose maximum was found above.&#xA;For the minimum of (2) to occur at the same location as (1) would require the middle&#xA;term:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ g(\theta) = -2\sum d_e \delta_e $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to have a &quot;steeper&quot; minimum there which overcomes the maximum of $f(\theta)$.  For &#xA;simplicity we only calculate whether $\theta_1$ is a critical point of $g(\theta)$.&#xA;If not, then a minimum of $g(\theta)$ cannot occur there, and likewise (2) is not&#xA;minimized at the same place as (1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ g(\theta) = -2(9|\cos(\theta)| + 16|\sin(\theta)| + 5|3\cos(\theta) + 4\sin(\theta)|) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a small neighborhood of $\theta_1$ the expressions within absolute values above are&#xA;all positive (it's in the first quadrant).  It follows that in such a neighborhood&#xA;we can differentiate $g(\theta)$ with disregard for those absolute values, so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ g'(\theta_1) = 48\sin(\theta_1) - 72\cos(\theta_1) \approx 5.76245102 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It follows that (2) does not have a critical point at $\theta_1$, so the optimality&#xA;criteria (1) and (2) give different best projections in this case.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2012-12-21T22:39:44.183" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4900" PostTypeId="1" CreationDate="2012-12-23T20:17:26.637" Score="5" ViewCount="206" Body="&lt;p&gt;Computational science remains uncommon in many computer science departments, particularly in universities without an engineering school. Is it not considered part of the standard computer science curriculum?&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2012-12-24T22:48:08.413" Title="Is computational science recommended as part of the typical undergraduate curriculum every computer science department should teach?" Tags="|reference-request|education|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4901" PostTypeId="2" ParentId="4900" CreationDate="2012-12-23T20:17:26.637" Score="3" Body="&lt;p&gt;Yes.  The &lt;a href=&quot;http://ai.stanford.edu/users/sahami/CS2013/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iron Man Draft of the Computer Science&#xA;Curricula 2013 by the ACM/IEEE-CS Joint Task Force&lt;/a&gt;, contains &lt;strong&gt;Computational Science&lt;/strong&gt; and &lt;strong&gt;Parallel and Distributed Computing&lt;/strong&gt; as two of the 18 Knowledge Areas recommended as part of the standard curricula.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, the &lt;a href=&quot;http://ai.stanford.edu/users/sahami/CS2013/ironman-draft/cs2013-ironman-v0.8.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;draft&lt;/a&gt; contains the following specific recommendations:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The “Big Tent” view of CS. As CS expands to include more cross-disciplinary work and new programs of the form “Computational Biology,” “Computational Engineering,” and “Computational X” are developed, it is important to embrace an outward-looking view that sees CS as a discipline actively seeking to work with and integrate into other disciplines.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Computer Science curricula should be designed to prepare graduates for a variety of professions, attracting the full range of talent to the field. Computer Science impacts nearly every modern endeavour. CS2013 takes a broad view of the field that includes topics such as “computational-x” (e.g., computational finance or computational chemistry) and “x-informatics” (e.g., eco-informatics or bio-informatics). Well-rounded CS graduates will have a balance of theory and application, as described in Chapter 3: Characteristics of Graduates.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2012-12-23T20:17:26.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4902" PostTypeId="1" AcceptedAnswerId="4905" CreationDate="2012-12-23T21:28:40.663" Score="15" ViewCount="303" Body="&lt;p&gt;I wrote a program/library which I used to obtain results in an article. (Here it &lt;a href=&quot;https://github.com/certik/hfsolver/&quot;&gt;is&lt;/a&gt;, but my question is general.) I have tests that I run regularly using &lt;code&gt;ctest&lt;/code&gt; (it takes a few minutes to run). In order to reproduce some tables or figures in the article, I have to construct a script or a simple driver program, that runs maybe 10 minutes, sometimes more, so I don't want to have this part of the regular test suite. At the same time, I want to make sure that the results from the article can be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;reproduced later&lt;/li&gt;&#xA;&lt;li&gt;make sure they still give the same/correct results after I keep developing the library&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Currently I try to have a small driver program that I run as part of the regular test suite, and if I want to reproduce results from the article, I uncomment some lines in there. Of course, I never know which exact lines and if I have to tweak some other parameters in order to get precisely the same results as in the article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to have a Python script that calculates the exact figures/tables from the article. Such a script typically stops working after an update to the library, because it is not being run on a regular basis (takes too much time). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best method that occurred to me is to have a Fortran (or C/C++) example, &#xA;that will be regularly compiled (as part of the library), but not run in regular test suite. That way, at least I know that it compiles fine (and thus hopefully also runs). And I'll test some simple (smaller) example as part of a regular test suite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are optimal ways to handle this problem?&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2014-11-13T00:16:38.143" Title="How to to easily reproduce published results in my own articles using my own code" Tags="|reproducibility|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4903" PostTypeId="2" ParentId="4864" CreationDate="2012-12-24T09:51:20.403" Score="4" Body="&lt;p&gt;Your derivation is correct up to minor imprecisions:&#xA;\begin{multline}&#xA;\int_\Omega \bigg((\lambda +2\mu) \nabla^2 \mathbf u \cdot \mathbf v +(\mu+\lambda)\nabla  \times (\nabla \times\mathbf u)\cdot \mathbf v  + \mathbf f \cdot \mathbf v  \bigg) d\Omega&#xA;= \\ =&#xA;-\int_\Omega \bigg((\lambda +2\mu) \nabla \mathbf u :\nabla \mathbf v +(\mu+\lambda)(\nabla  \times \mathbf v)\cdot(\nabla \times\mathbf u) \bigg)d\Omega&#xA;+ \\ + &#xA;\int_{\partial \Omega} \bigg((\lambda +2\mu) (\nabla \mathbf u \,\mathbf n) \cdot &#xA;\mathbf v + (\mu+\lambda) \: \mathbf n \times (\nabla \times \mathbf u) \cdot &#xA;\mathbf v \bigg) d \partial \Omega + &#xA;\int_\Omega \big( \mathbf f \cdot \mathbf v  \big) d\Omega&#xA;\end{multline}&#xA;where $\partial \Omega$ is the boundary and $\mathbf n$ is the unit outward normal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By inspecting the boundary term it is clear that this is &lt;em&gt;not&lt;/em&gt; linear elasticity, unless you have Dirichlet b.c. on the whole boundary $\partial \Omega$. The point here is that if you want to recover the proper Neumann b.c. you are not free to integrate by parts at your will: you have to transfer to the test function the &lt;em&gt;correct&lt;/em&gt; derivatives.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2012-12-24T09:51:20.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4905" PostTypeId="2" ParentId="4902" CreationDate="2012-12-24T21:41:11.140" Score="6" Body="&lt;p&gt;In deal.II, we have a testsuite that is driven by a regular Unix Makefile. It has a default target that runs all the usual tests, and a separate target for expensive tests. Running each test is done using a generic rule but the default target calls the generic rule only for certain tests and the expensive target calls it for the expensive tests. Because it's all done using a generic rule, it &lt;em&gt;must&lt;/em&gt; be up to date at any given time; what may get out of date can only be the lists of names of tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: The text above was correct in 2012. Since 2014, the deal.II testsuite is based on CTest, but the general idea remains valid.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2014-11-13T00:16:38.143" LastActivityDate="2014-11-13T00:16:38.143" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4906" PostTypeId="2" ParentId="4900" CreationDate="2012-12-24T22:48:08.413" Score="3" Body="&lt;p&gt;I like your own answer to your question. I wished this were how it was considered everywhere but I suspect this is not universally the case. In particular, the big-tent-view is not something departments often assume (no department, not CS and not Math). I created a computational science course using FEM software here at Texas A&amp;amp;M (MATH 676) which I'm teaching every other year. It was a bit difficult to get into the university catalog since the CS department did not want me to use the term &quot;Computational Science&quot; (it would step on their turf -- well, maybe so, but there is nothing even closely similar in their department; it's like staking a claim but then not using it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My personal view is that Computational Science should be a set of courses (a couple of basic courses on numerical methods, a basic &quot;software in computational science&quot; course, and then computational-X courses) that are taught in an interdepartmental mode. But that's not what I see is happening today.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-24T22:48:08.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4907" PostTypeId="2" ParentId="4883" CreationDate="2012-12-24T23:08:03.280" Score="6" Body="&lt;p&gt;Others have already noted this but I think it's still worth pointing out that the determinant is not a useful quantity almost always when your matrix is large. The problem is that large matrices are most often approximations to things that are even larger dimensional (statistical samples of large populations, finite dimensional approximations to infinite dimensional things like partial differential equations, etc). Let us say your matrix is $A$ and that it, in some way, approximates some process $B$ where $\textrm{dim}A \ll \textrm{dim} B$ and possibly $\textrm{dim}B=\infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the operator $B$ will also have a spectrum, and the spectrum of $A$ will typically be related in some sense to the spectrum of $B$ -- e.g., the eigenvalues of $A$ approximate (a subset of) those of $B$. But this does not mean that $\textrm{det}A$ approximates $\textrm{det}B$. In fact, if, for example, the eigenvalues of $A$ approximate some of the eigenvalues of $B$, then&#xA;$$&#xA;  \textrm{det}A&#xA;  =&#xA;  \prod_{j=1\ldots \textrm{dim}A} \lambda_i(A)&#xA;  \approx&#xA;  \prod_{j=1\ldots \textrm{dim}A} \lambda_i(B)&#xA;  \prod_{j=\textrm{dim}A+1\ldots \textrm{dim}B} \lambda_i(B)&#xA;$$&#xA;but the second factor in this product is typically either the product of the &lt;em&gt;small&lt;/em&gt; or of the &lt;em&gt;large&lt;/em&gt; eigenvalues of $B$ that aren't resolved by $A$. In other words, this factor is typically either very small (possibly zero, if $\textrm{dim}B=\infty$) or very large (possibly infinite). In other words, in all of these cases, we typically have that $\textrm{det}A$ does not approximate $\textrm{det}B$ and is not usually a useful thing to compute.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-24T23:08:03.280" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4908" PostTypeId="1" CreationDate="2012-12-25T05:36:27.113" Score="11" ViewCount="259" Body="&lt;p&gt;Parallel computation is frequently modeled using a deterministic local rate of computation, latency overhead, and network bandwidth. In reality, these are spatially variable and non-deterministic. Studies such as &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.101.3059&amp;amp;rep=rep1&amp;amp;type=pdf&quot;&gt;Skinner and Kramer (2005)&lt;/a&gt; observe multi-modal distributions, but performance analysis seems to always use either deterministic or Gaussian distributions (not just inaccurate, it's inconsistent due to positive probability of negative latency).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have higher-fidelity statistical models been developed? Do any account for cross-correlation in local compute/memory, latency, and bandwidth variability?&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-04-05T18:33:09.173" Title="Statistical models for local memory/compute, network latency, and bandwidth jitter in HPC" Tags="|performance|hpc|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4909" PostTypeId="2" ParentId="4888" CreationDate="2012-12-25T06:24:49.690" Score="11" Body="&lt;p&gt;If it's shock-capturing that you're interested in, I would suggest you use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_volume_method&quot;&gt;finite volume method&lt;/a&gt; instead of the finite element method. When applied naively, FEM is actually notoriously bad at resolving shocks -- usually there are spurious oscillations or unwanted diffusion. Provided your original PDE is a conservation law, the FVM method will preserve this structure and it can do a pretty decent job resolving shocks. Alternatively, you could look at using the discontinuous Galerkin method, but I find FVM to be much easier to understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good starting place would be looking up &lt;a href=&quot;http://en.wikipedia.org/wiki/Godunov%27s_scheme&quot;&gt;Godunov's scheme&lt;/a&gt;. An excellent reference is Leveque's &lt;em&gt;Finite Volume Methods for Hyperbolic Problems&lt;/em&gt;. His software package CLAWPACK is also a great tool for problems of this type.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2012-12-25T06:24:49.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4910" PostTypeId="1" CreationDate="2012-12-26T18:42:04.420" Score="1" ViewCount="29" Body="&lt;p&gt;Suppose a configuration $X\in\mathbb{R}^{n\times 2}$ is output of PCA on some high-dimensional data $Y\in\mathbb{R}^{n\times h}$. Note that this PCA is performed by $$X=Y\cdot U,$$ where columns of $U$ are eigenvectors of $Y^TY$ corresponding to its 2 dominant eigenvalues. Since $X$ is a result of PCA, then $X^TX=diag(a, b)$, for some $a, b\in\mathbb{R}$. If one now performs PCA on $X$, then the matrix $V$ containing eigenvectors of $X^TX$ would be columns of an identity matrix, and the result would be intact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that a configuration $X$ is normalized (axes of unit length) by $\hat{X}=X\cdot diag^{-1/2}(a, b)$. If one then performs PCA on $\hat{X}$, then $\hat{X}^T\hat{X}$ would be an identity matrix. In other words, by normalizing columns of $X$ one cannot affect the PCA result except for the aspect ratio change (stretching of axes). If fact, any arbitrary stretching of axes of $X$ would have no effect since also then $\hat{X}^T\hat{X}$ would be a diagonal matrix. Am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="1069" LastEditorUserId="1069" LastEditDate="2012-12-26T20:05:09.953" LastActivityDate="2012-12-26T20:05:09.953" Title="PCA performed on a configuration with scaled axes" Tags="|linear-algebra|matrix|high-dimensional|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4911" PostTypeId="2" ParentId="4880" CreationDate="2012-12-26T23:10:04.123" Score="2" Body="&lt;p&gt;While PCA is useful to reduce dimensions, the choice of the number of dimensions should be viewed as a parameter depending on the effectiveness of your classifier on the data generated by your problem.  (It could be three, two, four, ...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then you'll need a classifier, and you might get good mileage from one of the various mathematical segmentation / clustering algorithms.  Two that are popular and fairly easy to implement are K-Means Clustering and Fuzzy C-Means Clustering.  There are lots of existing implementations for these in Matlab, Octave, or R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following links should get you started:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Cluster_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cluster Analysis - in general&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/kmeans.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;K-Means Clustering Tutorial&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/cmeans.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fuzzy C-Means Clustering Tutorial&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/a/2346069/181638&quot;&gt;The Difference between the two methods (good answer on Stack Overflow)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3299" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2012-12-26T23:10:04.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4913" PostTypeId="2" ParentId="3371" CreationDate="2012-12-27T15:03:54.777" Score="5" Body="&lt;p&gt;Since asking this question I have purchased a new machine and have run the same set of code on it.  The new machine has 2 Intel Xeon E5-2640 2.50GHz for a total of 12 cores, and 64 GB of DDR3 RAM on 4 channels at 1333MHz.  Regardless of the number of cores, we saw about a 3x-4x speed up between the two machines on identical problems.  On smaller problems, we also saw slight speed-ups on the new machine on problem sizes where the larger cache was advantageous, but that isn't easy to exploit, since other factors determine our problem size.  I feel fairly confident in saying that the speed-up was almost purely due to the higher memory bandwidth.  Running the same cases on 1 core vs. 12 cores only sees about a 20% speed-up for this code.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2012-12-27T20:05:02.467" LastActivityDate="2012-12-27T20:05:02.467" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4914" PostTypeId="1" AcceptedAnswerId="4929" CreationDate="2012-12-28T02:19:00.787" Score="1" ViewCount="414" Body="&lt;p&gt;I asked this question before but did not have any luck with an answer. It might be a student level question but I need to understand that with possibly some help. I am considering the hyperbolic equation of the form $$u_t+\frac{x}{T-t}u_x=0$$ and some initial data $u(x,0)=u_0(x)$. I would like to claim that this is a well posed problem in $\mathbb{L}^2$ on $[0,T]$. For that existence, uniqueness and stability needs to be established. So I have the following steps to achieve that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 1: there is a change of variables I can employ as $z=x*(T-t)$. Then for $v(z(t,x),t)=u(x,t)$ I can find $$u_t=v_t+v_z(-x),\; u_x=v_z(T-t).$$Plug them into original equation I obtain an equivalent formulation to the original problem(up to a change of variables): $$v_t=0,\; v(z,0)=u_0(z/T,0).$$ The alternative formulation is clearly trivially has a unique solution for all $t\in [0,T]$, thus I claim the original problem does as well. Moreover, $u(x,T)=v(0,T)=u_0(0,0)$, so even though it initially looks like a not defined p.d.e. at that time it does have a solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 2: show stability via energy methods. For an arbitrary hyperbolic equation $w_t-a(y)w_y=0, \; y \in [0,1]$:&#xA;\begin{align*}&#xA;\frac{d||w||^2}{dt}&amp;amp;=(w_t,w)+(w,w_t)=(a(y)w_y,w)+(w,a(y)w_y)\\&#xA;&amp;amp;=(a(y)w_y,w)-(w_y,a(y)w)-(w,a_y(y)w)+a(y)ww|^1_0\\&#xA;&amp;amp;\leq max_{y\in [0,1]}|a_y(y)|||w||^2+a(y)ww|^1_0&#xA;\end{align*}&#xA;Thus, the energy estimate boils down to the bound of $a_y$ provided boundary conditions are bounded. I have to estimate the stability of the original equation for $u(x,t)$ and it is not equivalent to the stability of $v(z,t)$. But from the energy estimate above $a_x=\frac{1}{T-t}$ which is not bounded as $t \to T$. Does that imply that is not well posed or energy methods did not work? What are other approaches to prove stability here as I can explicitly find the solution, so it perfectly stable on $[0,T]$? In this case everything is solvable, but I am considering the general case where the trajectories do have this kind of asymptotic behavior.&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="276" LastEditDate="2016-06-01T21:34:19.017" LastActivityDate="2016-06-01T21:34:19.017" Title="energy norm for transport equation" Tags="|pde|stability|hyperbolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4915" PostTypeId="1" AcceptedAnswerId="4916" CreationDate="2012-12-28T05:07:33.707" Score="4" ViewCount="260" Body="&lt;p&gt;I have a feeling my question is a very basic one, but I am not at all well versed in computational sciences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My equations are of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;y \in \mathbb{R}^3 \\&#xA;\dot{y}(t) = f(y(t)) \\&#xA;y_1(0) = a \\&#xA;y_2(T) = b \\&#xA;y_3(T) = c \\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a known method to numerically solve such a set of equations?&lt;/p&gt;&#xA;" OwnerUserId="3489" LastEditorUserId="276" LastEditDate="2012-12-28T07:15:45.677" LastActivityDate="2012-12-28T07:17:01.700" Title="How do I solve an ODE Two-Point Boundary Value Problem?" Tags="|ode|boundary-conditions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4916" PostTypeId="2" ParentId="4915" CreationDate="2012-12-28T07:03:05.027" Score="2" Body="&lt;p&gt;You should be able to solve this problem using a &lt;a href=&quot;http://en.wikipedia.org/wiki/Direct_multiple_shooting_method&quot; rel=&quot;nofollow&quot;&gt;multiple shooting method&lt;/a&gt;; you need only find initial conditions $y_{2}(0)$ and $y_{3}(0)$ that yield a solution consistent with your stated &quot;final conditions&quot;. These values are typically called &quot;boundary values&quot;; your problem is called a two-point boundary value problem. It is worth noting that multiple shooting methods are more numerically stable than single shooting methods.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2012-12-28T07:17:01.700" LastActivityDate="2012-12-28T07:17:01.700" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4917" PostTypeId="1" AcceptedAnswerId="4918" CreationDate="2012-12-28T16:21:15.300" Score="3" ViewCount="483" Body="&lt;p&gt;The use of DDM in FEA makes parallel solution of the whole analysis e.g. assembly, solver etc possible. DDM splits the model in domains and runs them in parallel. Since there are interconnected nodes between domains, Will DDM give unreliable result? Also, is there any other factor that can affect the correct solution result?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The application is in structural dynamics FEA and the platform is SMP (single machine with 2 processing cores).&lt;/p&gt;&#xA;" OwnerUserId="1327" LastEditorUserId="1327" LastEditDate="2012-12-28T19:55:27.970" LastActivityDate="2012-12-28T19:55:27.970" Title="Limitations of Domain Decomposition Method (DDM) in Finite Element Analysis (FEA)?" Tags="|finite-element|parallel-computing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4918" PostTypeId="2" ParentId="4917" CreationDate="2012-12-28T18:13:11.230" Score="4" Body="&lt;p&gt;Domain decomposition methods do not discard coupling, they just isolate it to enable parallel computation. For &lt;em&gt;explicit&lt;/em&gt; dynamic simulations, this just amounts to exchanging ghost values, and the (distributed) solution is identical to a serial implementation. For &lt;em&gt;implicit&lt;/em&gt; problems, DD is an iterative method and the convergence &lt;em&gt;rate&lt;/em&gt; will depend on the decomposition (number and shapes of subdomains) for any particular method. Methods in which the rate is &lt;em&gt;robust&lt;/em&gt; to the number of subdomains must have more than one level (they need a &quot;coarse grid&quot; to produce global communication on each iteration). Continuity between subdomains will be enforced either by definition of the state space (a non-overlapping vertex partition) or by Lagrange multipliers (the FETI family).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-12-28T18:13:11.230" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4921" PostTypeId="1" AcceptedAnswerId="4922" CreationDate="2012-12-29T23:27:48.547" Score="3" ViewCount="125" Body="&lt;p&gt;Say I have a function &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function u = somefxn(f,n)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;f&lt;/code&gt; is some function, say &lt;code&gt;f = pi^2*sin(pi*x)&lt;/code&gt;, and n is the number of discretizations (let &lt;code&gt;n = 1024&lt;/code&gt;) on the interval $[0,1]$. In my code I've written &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h = 1/n %n = 1024&#xA;x = 0:h:1 %discretizations&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But what if the user uses &lt;code&gt;y&lt;/code&gt; or &lt;code&gt;z&lt;/code&gt; in the function instead of &lt;code&gt;x&lt;/code&gt;? How do I handle that uncertainty? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if this isn't clear. I'm rather new to Matlab.&lt;/p&gt;&#xA;" OwnerUserId="3493" LastEditorUserId="3493" LastEditDate="2013-02-28T19:17:40.490" LastActivityDate="2013-02-28T19:17:40.490" Title="Handling function inputs in Matlab" Tags="|matlab|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4922" PostTypeId="2" ParentId="4921" CreationDate="2012-12-30T00:10:14.967" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The function &lt;code&gt;f&lt;/code&gt; has (should have?) its own local variable, the name of which is independent of the calling scope.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It's much better to use &lt;code&gt;x = linspace(0,1,n+1)&lt;/code&gt; than &lt;code&gt;x = 0:1/n:1&lt;/code&gt; because the latter is susceptible to rounding error.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-12-30T00:10:14.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4923" PostTypeId="1" AcceptedAnswerId="4924" CreationDate="2012-12-30T05:38:33.713" Score="7" ViewCount="168" Body="&lt;p&gt;I have a large number of parallel processes and a large integer &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, and want to randomly partition the integers &lt;span class=&quot;math-container&quot;&gt;$[0,n)$&lt;/span&gt; among the processes with only &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt; communication.&lt;/p&gt;&#xA;&lt;p&gt;One nice way to do this would to generate a pseudorandom permutation &lt;span class=&quot;math-container&quot;&gt;$\pi \in S_n$&lt;/span&gt; represented as a small function, so that only the random key/seed need be exchanged.  Is there a good way to do this?&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2020-07-20T08:59:05.267" LastActivityDate="2020-07-20T08:59:05.267" Title="Random access random permutations" Tags="|partitioning|randomized-algorithms|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4924" PostTypeId="2" ParentId="4923" CreationDate="2012-12-30T05:54:46.867" Score="7" Body="&lt;p&gt;Pick $2^k$ slightly larger than $n$, generate a block cypher $f \in S_{2^k}$ operating on $k$ bit blocks, and construct a permutation on $[0,n)$ by walking along cycles of $f$ until we get back in the desired range.  Specifically, given $x &amp;lt; n$ we set&#xA;$$g(x) = f^p(x) = f(f(f(...x...)))$$&#xA;where $p$ is the least positive integer s.t. $f^p(x) &amp;lt; n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $2^k = O(n)$, and the block cypher is good, the walk takes $O(1)$ expected time.  Note that $p$ is necessarily finite, since eventually we will walk back around the cycle and find $f^p(x) = x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details, see&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Black and Rogaway, Ciphers with Arbitrary Finite Domains, 2001.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://blog.notdot.net/2007/9/Damn-Cool-Algorithms-Part-2-Secure-permutations-with-block-ciphers&quot; rel=&quot;noreferrer&quot;&gt;http://blog.notdot.net/2007/9/Damn-Cool-Algorithms-Part-2-Secure-permutations-with-block-ciphers&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is an example implementation using a truncated TEA block cypher as described in (2):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/otherlab/core/blob/f09fbd19dbaa7b9033eb0888594273a6a3d592a5/random/permute.cpp&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/otherlab/core/blob/f09fbd19dbaa7b9033eb0888594273a6a3d592a5/random/permute.cpp&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2012-12-30T06:29:30.660" LastActivityDate="2012-12-30T06:29:30.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4925" PostTypeId="1" AcceptedAnswerId="4926" CreationDate="2012-12-30T07:26:15.747" Score="7" ViewCount="35765" Body="&lt;p&gt;I have a function with a number of arguments, but some of them are optional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) If when the function is called, some are left empty, how do I code it so that it defaults to a specific value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) What if of 5 arguments, 3 are optional, how can the code recognize which one to set to default? For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [u, v] = somefxn(required, required2, optional, optional2, optional3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and say when I call this function, only &lt;code&gt;optional2&lt;/code&gt; is filled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; Is there something I can use that says&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if optional2 is empty&#xA;    return default_value&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3493" LastEditorUserId="3493" LastEditDate="2012-12-30T20:56:03.497" LastActivityDate="2014-05-26T16:40:15.847" Title="How to code in Matlab: If an argument is empty, then default to &quot;x&quot; value?" Tags="|matlab|" AnswerCount="4" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4926" PostTypeId="2" ParentId="4925" CreationDate="2012-12-30T09:36:58.027" Score="7" Body="&lt;p&gt;Here's one take:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) In Matlab, if you use &lt;code&gt;varargin&lt;/code&gt;, then you must provide all preceding optional arguments (that is, &lt;code&gt;optional&lt;/code&gt; must be given as well as &lt;code&gt;optional2&lt;/code&gt;). In this case (for example, if you call &lt;code&gt;somefxm(myarg, myarg2, myoptarg, myoptarg2)&lt;/code&gt;, then you know based on the length of &lt;code&gt;varargin&lt;/code&gt; (2) that only &lt;code&gt;optional3&lt;/code&gt; needs a default applied. You can have an if statement that says something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if length(varargin) == 2&#xA;    optional3 = 3;  % or whatever&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can have additional conditional statements to handle having only one optional argument (e.g. &lt;code&gt;length(varargin) == 1&lt;/code&gt;) or no optional arguments (e.g. &lt;code&gt;length(varargin) == 0&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &lt;em&gt;Note that with this scheme, Matlab has no clever way of knowing which function argument fits into which option, so you must keep everything in sequential order.&lt;/em&gt; If you need more flexibility, you can do something sort of like keyword arguments by preceding the argument with a descriptive string, which is the way that some common functions like &lt;code&gt;plot&lt;/code&gt; work. Then you either need to write logic to parse &lt;code&gt;varargin&lt;/code&gt; (as in Christian Clason's comment), or you can use the &lt;code&gt;inputParser&lt;/code&gt; object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another (in my opinion, clunkier) way of treating an &quot;empty&quot; &lt;code&gt;optional2&lt;/code&gt; is to test whether the provided argument is simply an empty matrix, with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if isempty(cell2mat(varargin(2))&#xA;    optional2 = 3; % or whatever&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, you could call &lt;code&gt;somefxn&lt;/code&gt; with an &quot;empty&quot; second optional argument as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;somefxn(myarg, myarg2, myoptarg, [], myoptarg3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, the second optional argument will be assigned whatever you've coded into the function. If you do this, you need to make sure that &lt;em&gt;an empty matrix is not an otherwise valid argument to &lt;code&gt;somefxn&lt;/code&gt;&lt;/em&gt;, otherwise you'll have just overloaded the proper behaviour.&lt;/p&gt;&#xA;" OwnerUserId="2015" LastEditorUserId="2015" LastEditDate="2012-12-30T22:31:00.417" LastActivityDate="2012-12-30T22:31:00.417" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4927" PostTypeId="1" CreationDate="2012-12-30T14:50:59.807" Score="12" ViewCount="17666" Body="&lt;p&gt;I am currently trying to solve nonlinear constrained minimization problem as implemented in matlab &quot;fmincon&quot; function. My expectations are, minimize(fun1,x0,uB,lB,fun2) where x0 is initial state, fun1 is function that needs to be minimized, uB are upper bounds, lB are lower bounds and fun2 is function that provides vectors of nonlinear equalities/inequalities as described in &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/fmincon.html&quot;&gt;http://www.mathworks.com/help/optim/ug/fmincon.html&lt;/a&gt; as nonlcon function. These vectors are changing through iterations as well (they are non-linearly dependent on x_n, n-th iteration of solution vector). In matlab implementation they are in a form c(x)&amp;lt;=0. This is the last piece of code that need to be ported from matlab to c++ and I ve been struggling a lot while trying to find appropriate c++ library containing this algorithm. This is why I am seeking help here and I would much appreciate if you could provide your expertise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good example of what I want to do is first one on this page &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/constrained-nonlinear-optimization-examples.html#f10960?s_tid=doc_12b&quot;&gt;http://www.mathworks.com/help/optim/ug/constrained-nonlinear-optimization-examples.html#f10960?s_tid=doc_12b&lt;/a&gt; Only difference is that I need boundaries as well...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Peter&lt;/p&gt;&#xA;" OwnerUserId="3499" LastEditorUserId="1804" LastEditDate="2012-12-30T17:59:16.003" LastActivityDate="2013-12-12T21:53:40.397" Title="C++ library for nonlinear constrained minimization" Tags="|c++|nonlinear-programming|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4928" PostTypeId="1" CreationDate="2012-12-30T19:12:50.543" Score="11" ViewCount="10026" Body="&lt;p&gt;The problem is&#xA;$$\max  f(\mathbf{x}) \text{ subject to } \mathbf{Ax} = \mathbf{b}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f(\mathbf{x}) = \sum_{i=1}^N\sqrt{1+\frac{x_i^4}{(\sum_{i=1}^{N}x_i^2)^2}}$, &lt;br/&gt;&#xA;$\mathbf{x} = [x_1,x_2,...,x_N]^T \in \mathbb{R}^{N\times 1}$, and  &lt;br/&gt;&#xA;$\mathbf{A} \in \mathbb{R}^{M\times N} $ &lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can see that $f(.)$ is in the form of $\sqrt{1+y^2}$ and is a convex function.&#xA;&lt;br/&gt; &#xA;It can be also shown that f(.) is bounded in $[\sqrt{2}, 2]$. &lt;br/&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a convex minimization problem with a linear constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which are the standard algorithms used to solve these kind of problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the specific nature of the problem, is it possible to solve it using any standard optimization software/package?&lt;/p&gt;&#xA;" OwnerUserId="3502" LastActivityDate="2018-02-03T06:51:39.740" Title="Maximizing a convex function (minimizing a concave function) with a linear constraint" Tags="|optimization|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4929" PostTypeId="2" ParentId="4914" CreationDate="2012-12-30T23:03:00.610" Score="3" Body="&lt;p&gt;It is certainly not true that just because one way to prove a result doesn't work, the result is actually false. In other words, just because this way doesn't work doesn't mean that the equation isn't stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you can express the solution $u(x,T)$ using the method of characteristics as $u(x,T)=u_0(y)$ with some $y=y(x)$ (you find $y$ by backtracing the characteristic starting at $x,t=T$). Using this, you will immediately get the estimate $$\|u(\cdot,T)\|_{L^p(0,1)} \le \|u_0(\cdot)\|_{L^\infty(0,1)}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one follows this a bit, one notices that one can rewrite the equation as&#xA;$$&#xA;  \left( 1 \atop \frac{x}{T-t} \right)&#xA;  \cdot &#xA;  \left( \partial_t \atop \partial_x \right)&#xA;  u(x,t)&#xA;  = 0.&#xA;$$&#xA;In other words, the characteristics turn horizontal in the $x-t$ diagram as $t\rightarrow T$; following back all characteristics we find that every characteristic ending in $x,T$ in fact started at $0,0$. In other words, we do indeed find that $u(x,T)=u(0,0)$ as you have already noticed. This yields a trivial stability bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you see is that there are often multiple ways of proving a particular statement. That there are additional ways that do not work does not invalidate the result.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2012-12-30T23:03:00.610" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="4930" PostTypeId="2" ParentId="4928" CreationDate="2012-12-30T23:25:36.417" Score="8" Body="&lt;p&gt;You can take advantage of the structure of the problem, though I know of no prepackaged solver that will do so for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, what you're looking for is minimizing a concave function over a convex polytope (or convex polyhedron). A quick search pulled up a few relevant sources (I vaguely remember one of these being mentioned when I took a class on nonlinear programming over four years ago):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Falk, J. E., and Hoffman, K. L. &lt;a href=&quot;http://seor.gmu.edu/~khoffman/collapsing_polytopes.pdf&quot;&gt;Concave minimization via collapsing polytopes&lt;/a&gt;, Operations Research, 1986, Vol. 34, No. 6, p. 919-929.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hoffman, K. L. &lt;a href=&quot;http://seor.gmu.edu/~khoffman/A_method_for_globally_minimizing_concave_functions_over_convex_sets.pdf&quot;&gt;A method for globally minimizing convex functions over convex sets&lt;/a&gt;, Mathematical Programming, 1981, Vol. 20, p. 22-31.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Benson, H. P. &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/nav.3800320119/abstract&quot;&gt;A finite algorithm for concave minimization over a polyhedron&lt;/a&gt;, Naval Research Logistics, 1985, Vol. 32, No. 1, p. 165-177.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bunch of references on &lt;a href=&quot;http://www.crt.umontreal.ca/~christop/concave_en.html&quot;&gt;Christophe Meyer's web site&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more sources if you Google &quot;minimize concave function over polytope&quot; (or replace &quot;polytope&quot; with &quot;polyhedron&quot;).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-12-30T23:25:36.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4931" PostTypeId="2" ParentId="4927" CreationDate="2012-12-31T01:52:54.000" Score="2" Body="&lt;p&gt;If your function is not differentiable, you should be careful about how you use finite differences. If you want to use derivative information, your best bet is probably some sort of semismooth Newton-type method. A set of notes describing such methods can be found &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.163.7781&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Twelve to thirty variables is probably on the upper end of what's doable with pattern search (also called direct search) methods. A recent review paper by Rios and Sahinidis in Journal of Global Optimization on derivative-free optimization methods (such as pattern search methods) can be found &lt;a href=&quot;http://rd.springer.com/content/pdf/10.1007%2Fs10898-012-9951-y&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, along with a &lt;a href=&quot;http://archimedes.cheme.cmu.edu/?q=dfocomp&quot; rel=&quot;nofollow&quot;&gt;companion web page&lt;/a&gt;. A less recent review paper on these methods by Kolda, Lewis, and Torczon in SIAM Review can be found &lt;a href=&quot;http://www.cs.wm.edu/~va/research/sirev.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. These methods work fairly well with expensive function evaluations, and don't necessarily require differentiability or derivative information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of these methods require some sort of convexity to guarantee convergence to the global optimum, so if you were to solve your problem rigorously, you might need to couple these methods above with a branch-and-bound strategy. However, if you don't care about rigor, an approach like MATLAB's &lt;code&gt;fmincon&lt;/code&gt; might work well enough (there are no guarantees anymore). Finite differences will most likely give you a member of the subdifferential of your nondifferentiable function, which could suffice for your problem instance and particular input data to return a sufficiently accurate result for your purposes. In that case, you should probably look at the libraries mentioned in the answers to the question Christian linked in his comment.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2012-12-31T01:52:54.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4932" PostTypeId="1" AcceptedAnswerId="4935" CreationDate="2012-12-31T04:19:37.153" Score="12" ViewCount="1718" Body="&lt;p&gt;So I understand (or at least I believe I do) how a V-cycle runs. I've written in Matlab the 1-D, recursive version of a V-cycle. However, when I ran my code for FMG, my solution wasn't converging. I believe my trouble lies in my understanding of the actual FMG part. What I currently know is this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Just prior to the FMG Interpolation, I've relaxed my solution $u$&lt;/li&gt;&#xA;&lt;li&gt;Interpolate &lt;em&gt;both&lt;/em&gt; the error &lt;em&gt;and&lt;/em&gt; $u$ (?)&lt;/li&gt;&#xA;&lt;li&gt;Execute a 2-grid v-cycle, passing the &lt;em&gt;error&lt;/em&gt; into the v-cycle (?)&lt;/li&gt;&#xA;&lt;li&gt;Relax the error (on the 2nd coarsest grid)&lt;/li&gt;&#xA;&lt;li&gt;Interpolate $u$ and the error&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Update&lt;/em&gt; $u$ by adding the error to it.&lt;/li&gt;&#xA;&lt;li&gt;Run a v-cycle, then repeat from step 4.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm not sure about the order, but I could also be wrong about what exactly I interpolate and pass into my v-cycle. If I'm missing anything from the algorithm, please let me know.&lt;/p&gt;&#xA;" OwnerUserId="3493" LastActivityDate="2012-12-31T16:52:10.057" Title="How exactly does the *full* multigrid algorithm run?" Tags="|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4934" PostTypeId="2" ParentId="4925" CreationDate="2012-12-31T13:51:00.403" Score="9" Body="&lt;p&gt;You can do this in a way that isn't as prone to breaking when the order of the arguments changes using &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/inputparserclass.html&quot;&gt;&lt;code&gt;inputParser()&lt;/code&gt;&lt;/a&gt;, which also allows you to validate the arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example is &lt;a href=&quot;https://github.com/ketch/RK-opt/blob/master/RK-coeff-opt/rk_opt.m&quot;&gt;here&lt;/a&gt; (just look at the &lt;code&gt;setup_params()&lt;/code&gt; function starting on line 178).  More documentation with examples is &lt;a href=&quot;http://blogs.mathworks.com/community/2012/02/13/parsing-inputs/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2012-12-31T13:51:00.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4935" PostTypeId="2" ParentId="4932" CreationDate="2012-12-31T16:52:10.057" Score="11" Body="&lt;p&gt;Where did you come up with interpolating the &quot;error&quot;? (And how do you measure the error?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the first visit to a finer grid, the entire solution $u$  must be interpolated, ideally using a higher order operator (e.g., postprocessed/reconstructed solution for FEM). This FMG interpolation is $u^h \gets \mathbb{I}_H^h u^H$. (It's okay to use the a normal interpolation $\mathbb{I}_h^H = I_h^H$, but this typically gives up some efficiency, at least for smooth problems.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After FMG interpolation, you just apply one or more V-cycles (or W-cycles, etc). (Make sure to run at least one smoother before restricting.) The most common choices are linear defect correction in which only the residual $r^h = A^h u^h - b^h$ is restricted and the Full Approximation Scheme (FAS) which is a natural for nonlinear problems because it avoids global linearization (e.g, Newton or Picard).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In FAS, the fine grid &lt;em&gt;state&lt;/em&gt; is restricted using the &lt;em&gt;state restriction operator&lt;/em&gt; $\tilde u^H \gets \hat I_h^H \tilde u^h$. State restriction is not required by linear defect correction multigrid (a convenient attribute). The most common state restrictions are nodal injection (for FD and FE) and coarse cell averages (for FV and mixed FE). Now we can write the FAS coarse grid equation (equally valid for nonlinear $A$) as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A^H u^H = \underbrace{I_h^H b^h}_{b^H} + \underbrace{A^H \hat I_h^H \tilde u^h - I_h^H A^h \tilde u^h}_{\tau_h^H}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we have identified the coarse representation of the right hand side, $b^H$, and the additional correction $\tau_h^H$ which represents the influence of the fine grid on the coarse grid equation. Note the property that the restriction of the fine grid solution $u^{h*}$ satisfies the coarse grid equation: $A^H \hat I_h^H u^{h*} = b^H + \tau_h^{H*}$. After solving the coarse grid equation, FAS interpolates the &lt;em&gt;change&lt;/em&gt;, leading to an updated fine solution $u^h \gets \tilde u^h + I_H^h (u^H - \hat I_h^H \tilde u^h)$.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2012-12-31T16:52:10.057" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4936" PostTypeId="1" CreationDate="2013-01-01T01:46:23.513" Score="3" ViewCount="1166" Body="&lt;p&gt;I'm attempting to solve the particle-in-a-box problem using Scipy (with the help of &lt;a href=&quot;http://www.physics.buffalo.edu/phy410-505/2011/topic4/app2/index.html&quot; rel=&quot;nofollow&quot;&gt;http://www.physics.buffalo.edu/phy410-505/2011/topic4/app2/index.html&lt;/a&gt;). At first, I used a 16x16 matrix to model the Hamiltonian, like the link, and my results corresponded to theirs. However, when I used a larger matrix (50x50), I found many extraneous eigenvalues in my results due to the larger matrix size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does a 16x16 matrix produce exactly the correct eigenstates while a larger one produces extraneous ones? When using a larger matrix (which I thought would increase the accuracy of the simulation due to fewer basis elements being omitted), how can I tell which elements correspond to actual eigenstates and which are extraneous?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy import linalg, mat, matrix&#xA;&#xA;def Sfun(m,n):&#xA;    if (m+n)%2==0:&#xA;        v1 = 2/(m+n+5)&#xA;        v2 = 4/(m+n+3)&#xA;        v3 = 2/(m+n+1)&#xA;        return v1 - v2 + v3&#xA;    else:&#xA;        return 0&#xA;&#xA;def Hfun(m,n):&#xA;    if (m+n)%2==0:&#xA;        return -8*(1-m-n-2*m*n)/((m+n+3)*(m+n+1)*(m+n-1))&#xA;    else:&#xA;        return 0&#xA;&#xA;Slist = []&#xA;Hlist = []&#xA;&#xA;for m in range(0,16):&#xA;    tlist = []&#xA;    for n in range(0,16):&#xA;        tlist.append(Sfun(m,n))&#xA;    Slist.append(tlist)&#xA;&#xA;for m in range(0,16):&#xA;    tlist = []&#xA;    for n in range(0,16):&#xA;        tlist.append(Hfun(m,n))&#xA;    Hlist.append(tlist)&#xA;&#xA;Smat = matrix(Slist)&#xA;Hmat = matrix(Hlist)&#xA;&#xA;vals,vecs = linalg.eig(Hmat, Smat)&#xA;&#xA;for i in range(0,16):&#xA;    print('Vector: ', end=&quot;&quot;)&#xA;    print(vecs[i], end=&quot;&quot;)&#xA;    print(&quot;                     Value: &quot;, end=&quot;&quot;)&#xA;    print(vals[i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3505" LastActivityDate="2013-01-01T01:46:23.513" Title="Hamiltonian Matrix Size in Schrodinger Equation" Tags="|quantum-mechanics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4937" PostTypeId="1" CreationDate="2013-01-01T06:54:24.133" Score="0" ViewCount="313" Body="&lt;p&gt;Could someone tell me the computation cost of using Ordinary least squares in estimating a Vector autoregression model?&#xA;I am thinking the cost is O(n) where n is the number of the training instance.&#xA;But I am not sure about this. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="3506" LastActivityDate="2013-01-01T06:54:24.133" Title="computation complexity of OLS in estimating a VAR model" Tags="|least-squares|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4939" PostTypeId="1" AcceptedAnswerId="4942" CreationDate="2013-01-02T00:54:40.877" Score="16" ViewCount="8042" Body="&lt;p&gt;It is probably a student level question but I can't exactly make it cleat to myself. Why is it more accurate to use non-uniform grids in the numerical methods? I am thinking in the context of some finite-difference method for the PDE of the form $u_t(x,t)=u_{xx}(x,t)$. And assume I am interested in a solution at the point $x^{\ast}$. So, I can see that if I approximate the second derivative, for example on a uniform grid using three point approximation, the error is second order $O(h^2)$. Then I can construct non-uniform grid via a mapping and find coefficients for the three points that are used to approximate the derivative. I can do the Taylor expansions and again obtain a bound for the derivative to be a second order $O(h^2)$, where $h$ is the distance on a uniform grid from which I obtained mapping to a non-uniform grid. Both estimates contain derivatives and it is not clear to me why the solution would be more accurate on non-uniform grid as it depends on the magnitude of the corresponding derivatives in error estimates?   &lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2013-01-08T23:25:09.487" Title="uniform vs. non-uniform grid" Tags="|pde|finite-difference|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4940" PostTypeId="2" ParentId="4939" CreationDate="2013-01-02T12:22:55.060" Score="4" Body="&lt;p&gt;The typical reason why a non-uniform can lead to higher accuracy is that the PDE to solve is not of the form $u_t(x,t)=u_{xx}(x,t)$, but of the form $u_t(x,t)=(D(x)u_x(x,t))_x$. If your non-uniform grid allows you to represent the real $D(x)$ more accurately, you will get a more accurate solution. Because $D(x)$ is normally determined by material properties, it is likely constant within each material, so you normally have a piecewise constant function and should really align the grid accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different reason might be that $u(x,0)$ has more variation in certain regions than in others. One can try to compensate this a bit with an adaptively refined non-uniform grid. (However, in my opinion there are other techniques which can handle this situation even better than a non-uniform grid.)&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2013-01-02T12:22:55.060" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4941" PostTypeId="2" ParentId="4928" CreationDate="2013-01-02T12:45:20.107" Score="2" Body="&lt;p&gt;I attended some years ago a lecture on optimisation. Back then we used Matlab in combination with YALMIP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Solvers.Solvers&quot; rel=&quot;nofollow&quot;&gt;The YALMIP Wiki&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3510" LastActivityDate="2013-01-02T12:45:20.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4942" PostTypeId="2" ParentId="4939" CreationDate="2013-01-02T22:58:01.317" Score="20" Body="&lt;p&gt;The rationale for non-uniform meshes goes like this (all equations understood to be qualitative, i.e., in general true but without the pretense to be provably so in all circumstances and for all equations or all possible discretizations):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When solving an equation with, say, linear finite elements, then you typically have an error estimate of the kind&#xA;$$&#xA;  \|u-u_h\|_{L^2(\Omega)} \le C h_{\text{max}}^2 \|\nabla^2 u\|_{L^2(\Omega)},&#xA;$$&#xA;or, equivalently but in a form better suited to the following:&#xA;$$&#xA;  \|u-u_h\|_{L^2(\Omega)}^2 \le C h_{\text{max}}^4 \|\nabla^2 u\|_{L^2(\Omega)}^2.&#xA;$$&#xA;However, this is an overestimate. In fact, one can in many instances show that the error is actually of the form&#xA;$$&#xA;  \|u-u_h\|_{L^2(\Omega)}^2 \le C \sum_{K \in {\mathbb T}} h_K^4 \|\nabla^2 u\|_{L^2(K)}^2.&#xA;$$&#xA;Here, $K$ are the cells of the triangulation $\mathbb T$. This shows that in order to make the error small, it isn't actually necessary to reduce the &lt;em&gt;maximal&lt;/em&gt; mesh size $h_\text{max}$. Rather, the most efficient strategy will be to equilibrate the cellwise error contributions $h_K^4 \|\nabla^2 u\|_{L^2(K)}^2$ -- in other words, you should choose&#xA;$$&#xA;  h_K \propto \|\nabla^2 u\|_{L^2(K)}^{-1/2}.&#xA;$$&#xA;In other words, the local mesh size $h_K$ should be small where the solution is rough (has large derivatives) and large where the solution is smooth, and the formula above provides a quantitative measure for this relationship.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-01-03T16:47:26.323" LastActivityDate="2013-01-03T16:47:26.323" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4943" PostTypeId="1" CreationDate="2013-01-03T11:14:37.657" Score="6" ViewCount="821" Body="&lt;p&gt;I am trying to write a special hexagonal lattice generator, with several kinds of boundary conditions, such as helical BC, periodic BC, and I find it hard to verify whether it works correctly. I tried to draw them using 2-dimensional network drawing (using networkx) and as I expected, it was a total mess. Right now I have to work out the adjacency matrix beforehand and verify the generated network with the calculated adjacency matrix. It's fine with the regular lattice, but if I introduce some random perturbation on the lattice, this approach is very tiresome. It would be a lot easier if I could see the network drawn correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to correctly visualize (probably in 3D?) lattice with various boundary conditions? Or in other software tools? Is there a convenient way to test this kind of network generator?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="3512" LastActivityDate="2013-01-08T21:55:25.610" Title="how to visualize lattice with periodic, helical, etc. boundary conditions?" Tags="|graph-theory|visualization|boundary-conditions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4944" PostTypeId="1" AcceptedAnswerId="4950" CreationDate="2013-01-03T19:26:46.710" Score="2" ViewCount="149" Body="&lt;p&gt;I'm looking for the &lt;a href=&quot;http://en.wikipedia.org/wiki/Extreme_point&quot; rel=&quot;nofollow&quot;&gt;extreme points&lt;/a&gt; of the convex set $S\subset [-1,1]^{n\times 3}$ with $r\in S$ such that&#xA;\begin{equation}&#xA;r_{i} \ge r_{k} \iff i\ge k,&#xA;\end{equation}&#xA;where the first inequality  refers to lexicographical inequality in $\mathbb{R}^3$, and&#xA;\begin{equation}&#xA;\sum_i^n r_i = 0&#xA;\end{equation}&#xA;It was easy to characterize the extreme points satisfying only the first set of constraints. The second set has proven to be more difficult and I don't want to waste my time when there's already a solution out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any program packages or methods out there that would easily solve this problem? If so, what are they?&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2013-01-03T21:04:33.673" LastActivityDate="2013-01-04T05:40:54.353" Title="Extreme points from constraint expression of convex space" Tags="|convex-optimization|convex-hull|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4945" PostTypeId="1" AcceptedAnswerId="4961" CreationDate="2013-01-03T21:25:56.157" Score="8" ViewCount="155" Body="&lt;p&gt;I will be implementing nodal discontinuous Galerkin method soon, and having done this before I know the basic indexing arrays I will need to compute, given a mesh and polynomial data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I ran into in previous code were subtle mistakes I made in computing things like interior/exterior trace indexing. Problems which didn't arise on simpler test cases would arise on larger meshes, and usually this yields an unstable scheme since boundary conditions are not properly imposed (so no chance of just watching the simulation every 10 steps or so and seeing a localized problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm hoping some more experienced folks here know good tests to run on the index arrays to get some confidence that they are right. Quadratures, derivatives and the like are very easy to test, but other things I can't figure out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some tests I have done in the past is adding interior normals to exterior normals, which should yield 0 or +-2. Being able to quickly see the result of some code change is helpful, but I can't think of a meaningful way to do this with indexing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should also mention that these are going in for quads and hexes, with the potential for curvilinear elements. Not much existing code is there to compare against a working library already. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bonus points if there are good unit tests that I can write which wouldn't rely on an existing correct answer to compare against. I'll settle for a lot of good heuristics.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-01-05T23:20:03.667" Title="Testing and visualizing large index arrays" Tags="|software|testing|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4946" PostTypeId="1" CreationDate="2013-01-03T22:37:19.637" Score="4" ViewCount="153" Body="&lt;p&gt;I have a line in the script I submit to torque that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#PBS -o /path/to/output/file/directory/file_name_with_some_underscores.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But when I look for this output file, it's called something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/path/to/output/file/directory/file_name_with_some_underscores.o36091&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where 36091 is the job ID number.  I have multiple jobs running simultaneously with the same output file, but I've done that before and this has never happened previously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is Torque doing this?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2015-07-08T16:27:13.413" Title="Why is Torque changing the name of my output files?" Tags="|pbs|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4947" PostTypeId="5" CreationDate="2013-01-03T23:48:36.243" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-01-03T23:48:36.243" LastActivityDate="2013-01-03T23:48:36.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4948" PostTypeId="4" CreationDate="2013-01-03T23:48:36.243" Score="0" Body="To move in some direction (as a fluid does in a pipe). Often contrasted with diffusion, which is a spreading out without necessarily having any movement of the field as a whole.&#xD;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2013-01-04T00:08:47.550" LastActivityDate="2013-01-04T00:08:47.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4949" PostTypeId="2" ParentId="4939" CreationDate="2013-01-04T00:03:45.803" Score="6" Body="&lt;p&gt;Prove it to yourself with this example.   What is the maximum error when interpolating sqrt(x) on the interval [0,1] with piecewise linear interpolation on a uniform mesh?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the maximum error when interpolating on a mesh in which the ith of n points is given by (i/n)^s, and s is a carefully chosen mesh grading parameter?&lt;/p&gt;&#xA;" OwnerUserId="3340" LastActivityDate="2013-01-04T00:03:45.803" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4950" PostTypeId="2" ParentId="4944" CreationDate="2013-01-04T05:40:54.353" Score="1" Body="&lt;p&gt;The most reputable-looking source I could find was&lt;/p&gt;&#xA;&#xA;&lt;p&gt;M. E. Dyer, L. G. Proll. &lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF01593771&quot; rel=&quot;nofollow&quot;&gt;An algorithm for determining all extreme points of a convex polytope&lt;/a&gt;, &lt;em&gt;Mathematical Programming&lt;/em&gt;, Vol. 12, Issue 1, p. 81-96.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is likely only solvable for the case of convex polyhedra; there is a theorem that states that convex polyhedra can be expressed in terms of a convex combination of a finite number of extreme points and a finite number of extreme rays. No such claim can be made about convex sets in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm runtime will be exponential in the dimensionality of the polyhedron. To enumerate all of the vertices of an $n$-cube, all $2^n$ vertices must be enumerated. The problem is probably NP-hard.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-01-04T05:40:54.353" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4951" PostTypeId="1" CreationDate="2013-01-04T21:00:27.687" Score="1" ViewCount="59" Body="&lt;p&gt;I have a simple system that I want to process with the &lt;code&gt;CellDecomposition&lt;/code&gt; command of Maple. I don't know why Maple is giving an error here! The code is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;restart; &#xA;f1 := 4*b*(5*b*(13*t^2-tau^2)*(t^2+tau^2)^6-3*a*5*t^6-63*t^4*tau^2+35*t^2*tau^4+7*tau^6)); &#xA;f2 := 56*b*tau*(5*b*(t^2+tau^2)^6+a*(9*t^4-30*t^2*tau^2+9*tau^4)); f3 := 1-z*t;&#xA;with(RootFinding[Parametric])&#xA;&#xA;m := CellDecomposition([f1 = 0, f2 = 0, f3 = 0], [t, tau, z])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the error is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error, (in RootFinding:-Parametric:-CellDecomposition) The number of polynomials is smaller than the number of unknowns&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The number of polynomials is obviously the same as the number of variables here!&lt;/p&gt;&#xA;" OwnerUserId="3519" LastEditorUserId="42534" LastEditDate="2022-02-25T20:18:06.457" LastActivityDate="2022-02-25T20:18:06.457" Title="Error in Maple's CellDecomposition Command" Tags="|computational-geometry|rootfinding|" AnswerCount="1" CommentCount="1" ClosedDate="2022-03-07T14:47:40.917" ContentLicense="CC BY-SA 3.0" />
  <row Id="4952" PostTypeId="1" AcceptedAnswerId="4956" CreationDate="2013-01-04T21:08:26.567" Score="4" ViewCount="177" Body="&lt;p&gt;I have a simple (and small) linear homogeneous system $Ax=0$, where the entries of the $N\times M$ matrix $A$ are small integers. I do not need fancy methods which efficiently solve almost singular matrices and treat roundoff errors etc. But I need to know if a system is singular and also the general solution for underdetermined systems. It must be rock-solid. Any reference to such an algorithm is welcome. I am coding in C. Is SVD the way to go?&lt;/p&gt;&#xA;" OwnerUserId="1943" LastEditorUserId="1804" LastEditDate="2013-01-04T22:22:22.953" LastActivityDate="2013-01-05T09:31:23.820" Title="Numeric solution of simple but possibly singular linear system" Tags="|linear-solver|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4954" PostTypeId="2" ParentId="4952" CreationDate="2013-01-04T22:42:11.420" Score="2" Body="&lt;p&gt;The general solution of your underdetermined system is that $x$ is a member of the nullspace of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the nullspace of $A$, the most numerically stable method is to use an SVD. See &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2510/null-space-of-a-rectangular-dense-matrix&quot;&gt;Null-space of a rectangular dense matrix&lt;/a&gt; for further details. &lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-01-04T22:42:11.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4955" PostTypeId="2" ParentId="4952" CreationDate="2013-01-04T22:43:28.107" Score="2" Body="&lt;p&gt;The SVD will both&#xA;(1) tell you what the null space of your matrix is&#xA;(2) give you the pseudo-inverse of the matrix in the event that it is singular.&#xA;The SVD is, generally speaking, a very handy matrix factorization and for your stated purposes should definitely get the job done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're determined to write the code yourself, &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898713617&quot; rel=&quot;nofollow&quot;&gt;Trefethen and Bau's book&lt;/a&gt; has enough details to work from, either implementing the really bad way of finding the eigendecomposition of A*A or the proper way using bidiagonalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you just happen to be working in C and don't feel the need to do it yourself, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow&quot;&gt;Wikipedia page for the SVD&lt;/a&gt; has a long list of libraries that will implement it for you, chief among which is the GNU Scientific Library. Or, a google search of &quot;SVD c source code&quot; turns up a couple results as well.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-01-04T22:43:28.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4956" PostTypeId="2" ParentId="4952" CreationDate="2013-01-04T23:02:31.037" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://cs.uwaterloo.ca/~astorjoh/iml.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Integer Matrix Library&lt;/a&gt; is a C library that claims to be able to compute the null space of an integer matrix. See also &lt;a href=&quot;https://mathoverflow.net/questions/42637/exact-computation-of-the-null-space-basis-of-an-integer-matrix/57582#57582&quot;&gt;this answer on MathOverflow&lt;/a&gt; to the exact same question, which gives a list of libraries (including &lt;a href=&quot;http://pari.math.u-bordeaux.fr&quot; rel=&quot;nofollow noreferrer&quot;&gt;PARI&lt;/a&gt;, which also can be called from C and is still being updated). You could also take a look at &lt;a href=&quot;http://www.linalg.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LinBox&lt;/a&gt;, even though it's written in C++.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2013-01-05T09:31:23.820" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4957" PostTypeId="1" CreationDate="2013-01-04T23:18:58.740" Score="3" ViewCount="985" Body="&lt;p&gt;I've coded full multigrid in Matlab and it doesn't seem to be converging fast enough. When I increase the number of grids or the number of iterations, it converges to the analytical solution. But FMG shouldn't need that many iterations and it should work even for fewer grids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize this is a very vague question, but are there any ideas why this may be?&lt;/p&gt;&#xA;" OwnerUserId="3493" LastActivityDate="2013-01-11T14:41:34.607" Title="Full Multigrid convergence is too slow. What could possibly be causing it?" Tags="|multigrid|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4958" PostTypeId="2" ParentId="4957" CreationDate="2013-01-05T00:00:14.053" Score="7" Body="&lt;p&gt;I assume you are solving a nice elliptic problems like the Laplacian with smooth coefficients. (You should always check convergence in this friendly setting first.)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Confirm that interpolation is as accurate as necessary. There usual rule for vertex-centered MG is that the sum of interpolation and restriction orders should be at least as high as the order of the operator, and preferably higher. This requirement is reduced by one order for cell-centered (aggregation) coarsening. For FMG, you should use an even higher order interpolation the first time you visit a fine grid (when you're transferring the full solution rather than just corrections). You can verify order of accuracy using analytic functions.&lt;/li&gt;&#xA;&lt;li&gt;Check that the smoother is stable and damped correctly (if applicable). On structured grids, you can confirm that your smoother's performance matches the analytic predictions from local Fourier analysis.&lt;/li&gt;&#xA;&lt;li&gt;Check that two-grid convergence factors match the predictions from local Fourier analysis. (Use a problem with an analytic solution, or even solve $A x = 0$ with initial guess $x \ne 0$.)&lt;/li&gt;&#xA;&lt;li&gt;Check that the above is satisfied for V-cycles with more than two grids.&lt;/li&gt;&#xA;&lt;li&gt;Check that you are reaching discretization error in one F-cycle.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Consult Achi Brandt's &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~achi/classics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multigrid Guide&lt;/a&gt; for further details on the issues above and far more depth. You may also find &lt;a href=&quot;https://scicomp.stackexchange.com/questions/513/why-is-my-iterative-linear-solver-not-converging&quot;&gt;this advice&lt;/a&gt; helpful.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-01-11T14:41:34.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4960" PostTypeId="1" CreationDate="2013-01-05T11:41:13.767" Score="5" ViewCount="2232" Body="&lt;p&gt;In finite volume method one is using cell averages. In nonlinear conservation laws discontinuities can be created in the solution process. &#xA;How to compute the flux when the flux contains a gradient and the Taylor approximation is not valid due to discontinuities?&lt;/p&gt;&#xA;" OwnerUserId="3523" LastActivityDate="2015-01-09T18:24:17.387" Title="How to approximate flux (with gradient) when using finite volumes?" Tags="|finite-volume|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4961" PostTypeId="2" ParentId="4945" CreationDate="2013-01-05T12:37:10.503" Score="7" Body="&lt;p&gt;The obvious answer would of course be to not implement it all yourself but to use what others have done before. For example, deal.II (http://www.dealii.org -- disclaimer: I am one of the authors of this library) already has DG elements on quads, hexes, and curved elements, tutorial programs that show how to use them, etc. All this has been tested for a decade already and is tested via 2,700 regression tests after every change to the repository. There is much work to be saved for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to come back to the original question: What I often do for these sort of things when writing test cases for deal.II is to do something for which I know the exact solution and verify that I indeed get it. Example: If you have an interpolating (nodal) element, then on one cell $K$, assign to every degree of freedom the value of a function $f(x)$ at the nodes $x_i$. Do this &lt;em&gt;by hand&lt;/em&gt;, using your knowledge of how degrees of freedom are ordered. This yields a function $u_h(x)=\sum_i U_i \varphi_i(x)$ where $U_i=f(x_i)$. Then use a quadrature formula to compute something like $\|u_h-f\|^2 = \int_K |u_h(x)-f(x)|^2 \; dx$. If $f$ was a polynomial of the same order as your ansatz space, then the result should be zero. If you mixed up indices, either when assigning values to degrees of freedom by hand or in your program when evaluating $u_h$ at quadrature points, you will get something that is nonzero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of all this is that in your test, you use two independent methods to deal with indexing (your knowledge, when assigning values to DoFs by hand, and what your code thinks is correct when evaluating $u_h$) and they need to agree. If they don't, at least one of them is incorrect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, once you have this running, you need to save the testcase and run it periodically to ensure that you are not accidentally breaking this kind of functionality again.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-01-05T23:20:03.667" LastActivityDate="2013-01-05T23:20:03.667" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4962" PostTypeId="2" ParentId="4960" CreationDate="2013-01-05T18:35:54.257" Score="12" Body="&lt;p&gt;Non-degenerate diffusion will prevent discontinuities in the true solution, but those may be poorly resolved. I will assume that your question is about how to evaluate diffusive fluxes of the form $f(u,\nabla u,p) = \kappa(u,\nabla u,p)\nabla u$ where $\kappa$ is a $d\times d$ SPD tensor and $p$ represents some spatially distributed parameters that are cell-centered and discontinuous. There are a few approaches.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Two-point fluxes&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Take the centroids $c_0$ and $c_1$ of the two cells and compute $\nabla u\cdot(c_1 - c_0) = u(c_1) - u(c_0)$. If the mesh is orthogonal (with face quadrature point on line segment connecting cell centroids and normal with aligned normal) and the coefficients are constant and isotropic, this is a consistent second order accurate flux (i.e., superconvergent at centroids and face centers). Unfortunately, it loses accuracy and has stability problems on general meshes and for variable coefficients. Two-point fluxes are often used because they are simple and compact, but it is a fragile methodology since the grids regularity and constant coefficient assumptions are so frequently violated.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Gradient reconstruction&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The underlying principle is to reconstruct a gradient on each cell, typically with a weighted least squares procedure, and use that gradient evaluated at quadrature points (face centers) to define a flux. This procedure works simply for nonlinear fluxes. There are many variants that you can find in the literature, but you may want to look at &lt;a href=&quot;https://engineering.purdue.edu/ME608/webpage/main.pdf&quot;&gt;these notes&lt;/a&gt; for background and the most commonly used variants. The downside of this approach is that the stencil is larger than strictly necessary and the resulting linear systems are generally not symmetric or M-matrices (thus allowing oscillations).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Reinterpretation as mixed finite element methods&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Mixed finite element methods explicitly discretize the flux functions and work with inherently discontinuous (perhaps tensor-valued) coefficients and solution variables. The downside of mixed FEM is that there are more degrees of freedom and the resulting system of equations is a saddle point problem (which makes it more difficult to solve). After a special choice of quadrature, however, the &quot;flux&quot; degrees of freedom can be eliminated resulting in a (usually symmetric) positive definite system for the cell centered variables. An example of this is detailed in &lt;a href=&quot;http://www.math.pitt.edu/~yotov/research/publications/mfmfe.pdf&quot;&gt;Wheeler and Yotov (2006)&lt;/a&gt; which uses an underlying BDM-1 space (linear discontinuous flux functions on edges, constant pressure functions on cells) and a cell quadrature at cell corners which makes the mass matrix block diagonal (with one block per corner). After eliminating the fluxes, the resulting system is SPD and has a minimal stencil (only connecting cells that share a corner). It is second order superconvergent for tensor coefficients and general grids, though the fluxes are only first order accurate for irregular grids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in addition to the relationship between finite volume and mixed finite element, there is a duality between mixed finite element methods and mimetic finite difference methods. In all cases I know of, it is possible to choose basis functions and quadratures to make the methods coincide, though one or the other may be more natural for certain types of analysis or for implementation in a given software package.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-01-05T18:35:54.257" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4964" PostTypeId="1" AcceptedAnswerId="4967" CreationDate="2013-01-07T00:23:47.227" Score="8" ViewCount="3040" Body="&lt;p&gt;I need to implement parallel dense matrix inversion for a language I am using that appears to not have an existing library for this (specifically IDL using IDL Bridge for message passing). I am familiar with parallel programming methods through experience using MPI in C++, though mainly for parallel FFT and N-body methods. I have little experience or knowledge of computational methods, serial or parallel, for linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am looking for is a clear pseudo code description of a robust, stable parallel matrix inversion algorithm that is appropriate for a small number of cores as the machine that runs the (currently serial) IDL task has 8 cores. In practice I will probably use only 4 to keep cores free for other tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd favour simplicity over bleeding edge performance if there are a range of well known algorithms for this task.&lt;/p&gt;&#xA;" OwnerUserId="1314" LastEditorUserId="276" LastEditDate="2013-01-07T01:28:43.037" LastActivityDate="2013-01-07T08:42:51.067" Title="What algorithm to use for parallel dense matrix inversion on at most 8 cores?" Tags="|linear-algebra|algorithms|parallel-computing|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4965" PostTypeId="1" CreationDate="2013-01-07T00:58:15.537" Score="3" ViewCount="257" Body="&lt;p&gt;I am solving a heat equation $u_t=Au$ with Crank-Nicolson finite-difference method and $A$ is a usual discretization matrix for $u_{xx}$ term. I want to tell something about the whole error vector over the entire grid. I can write the discrete  iteration of the the method as $$u^{n+1}=Bu^n=\frac{I+\tau/2A}{I-\tau/2A}u^n$$ Since $B$ is normal I have $\rho(B)=||B||_2$ and I can see that having known eigenvalues of $A$ I can find eigenvalues of $B$ and thus I have $||B||_2\leq 1$ and $||e||_2=O(h^2)$ where $e$ is the vector error over the whole grid in the discrete $L^2$ norm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, my first question is whether I can imply $||e||_{\infty}=O(h^2)$ for Crank-Nicolson as well? For that I need to estimate $||B||_{\infty}=\max_{i,j}b_{i,j}$ and I don't know how to do that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second question, if the question one is a true statement, I have estimates for the error in two discrete norms: $L^2$ and $L^{\infty}$. What is the intuition of having $L^2$ discrete error of the error vector of order two, to me it looks that &quot;on average&quot; the error decreases by two, so if I pick the point on the grid I don't have to have quadratic convergence? Can I imply anything about how that will be converging at a particular point on the grid? The same question about $L^{\infty}$ norm. Which one should I use to measure the error, would one imply the other?   &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="549" LastEditDate="2013-01-07T04:11:16.563" LastActivityDate="2013-01-07T04:11:16.563" Title="intuition behind the different discrete norms for Crank Nicolson" Tags="|pde|stability|parabolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4966" PostTypeId="2" ParentId="4965" CreationDate="2013-01-07T03:28:41.773" Score="5" Body="&lt;p&gt;On your first question: I assume that by &quot;usual discretization matrix&quot; you mean either the 3-point finite difference discretization in 1d, or what you get using linear finite elements. In either case, it's not actually the Crank-Nicolson scheme that determines this. It's true that for the two spatial discretizations mentioned above, the &lt;em&gt;spatial&lt;/em&gt; error is $O(h^2)$ both in the $L^2$ and $L^\infty$ norms and that if you choose $\tau=h$ then the overall error also has this convergence order. But this is only true in 1d.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you were in a 2d, then you'd get $O(h^2)$ in the $L^2$ norm for the error, but $O(h^2 |\log h|)$ in the $L^\infty$ norm. What this shows is that it's not sufficient to simply look at properties of the Crank-Nicolson scheme -- it's in fact also necessary to look at properties of the underlying spatial discretization to prove estimates like the ones you are interested in. Your approach to just look at matrix norms of $B$ can therefore not be sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That also answers part of your second question. In any number of spatial dimensions, you &lt;em&gt;always&lt;/em&gt; have that the $L^2$ error decays like $O(h^2)$, but that isn't true for the $L^\infty$ error. In other words, the first implies that &lt;em&gt;on average&lt;/em&gt;, the spatial error decays quadratically, but that this does not have to be the case pointwise. On the other hand, if it were true that the $L^\infty$ error decayed as in $O(h^2)$, then it is trivial to show that that must also be true for the $L^2$ error.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-07T03:28:41.773" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4967" PostTypeId="2" ParentId="4964" CreationDate="2013-01-07T08:42:51.067" Score="5" Body="&lt;p&gt;Instead of rolling your own, I would suggest you have a look at existing robust and efficient parallel implementations such as &lt;a href=&quot;http://icl.cs.utk.edu/plasma/&quot;&gt;PLASMA&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're only looking for algorithms to implement yourself, most of the routines are documented excessively in the &quot;Documentation&quot; section of that site.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-01-07T08:42:51.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4968" PostTypeId="1" CreationDate="2013-01-08T12:58:01.490" Score="5" ViewCount="133" Body="&lt;p&gt;I need to calculate frequency response, phase response and apply to signals the Butterworth, Chebychev1 and Chebychev2 band-pass filters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm developing in C++ with Qt, and I'm looking for algorithms or a (free) library to do that. I have my plotting solution, I just need the formulas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input values I have are the time/amplitude signal, the order of the filter and the low and high cut-off frequencies.&lt;/p&gt;&#xA;" OwnerUserId="3536" LastEditorUserId="383" LastEditDate="2013-01-08T13:23:22.833" LastActivityDate="2013-01-21T22:02:51.083" Title="Use Butterworth and Chebychev filters" Tags="|reference-request|libraries|c++|signal-processing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4969" PostTypeId="2" ParentId="4943" CreationDate="2013-01-08T21:55:25.610" Score="4" Body="&lt;p&gt;Creating lattice (and other periodic) structures is a major issue in molecular simulations. Therefore, if you can translate your points into something that can be parsed by a viewer such as &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow&quot;&gt;VMD&lt;/a&gt; or &lt;a href=&quot;http://www.pymol.org&quot; rel=&quot;nofollow&quot;&gt;PyMol&lt;/a&gt;, you should be able to generate a view that can tell you if everything is working as expected. (This assumes, of course, that you build more than one periodic &quot;cell&quot; of your lattice.) &lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2013-01-08T21:55:25.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4970" PostTypeId="2" ParentId="4939" CreationDate="2013-01-08T23:25:09.487" Score="4" Body="&lt;p&gt;Kamil,  differential equation solving is global, interpolation is local.   In piecewise polynomial interpolation, the accuracy far from the singularity won't be bothered by the singularity.   Unfortunately, this isn't at all true for solving an elliptic equation, such as a two-point boundary value problem.   The singularity will pollute the approximation globally.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's something to try.   Solve D(sqrt(x) Du) on [0,1] with homogeneous Dirichlet b.c.s.   D is the differentiation operator.  Use finite elements or finite differences on an n-point uniform mesh.   Compare to a mesh in which the ith point is (1/n)^1.5.   Note that the worst error for the uniform mesh is far from the singularity, and much larger than for the graded mesh.&lt;/p&gt;&#xA;" OwnerUserId="3340" LastActivityDate="2013-01-08T23:25:09.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4971" PostTypeId="2" ParentId="173" CreationDate="2013-01-09T02:27:46.137" Score="5" Body="&lt;p&gt;I have little to contribute here other than to point out that whenever numerical methods have trouble with hyperbolic equations (and converge to the wrong solution), it isn't usually because of shocks. Rather, the areas they have difficulty with it are rarefaction waves -- where the solution is smooth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example of something that appears to be difficult is for a simple equation of the kind&#xA;$$&#xA;  u_t + \beta \cdot \nabla F(u) = g&#xA;$$&#xA;you get into trouble in those places where $F'(u)=0$. This is called the &quot;sonic point&quot; of an equation. (Note that the equation can be rewritten as $u_t + \beta F'(u) \cdot \nabla u = g$, which explains why $F'=0$ is a problem). What happens here is that the equation degenerates: it's just an ODE wherever $F'=0$ but the solution is of course affected by surrounding areas where this is not the case. Consequently, if you happen to have $F'=0$ in a whole part of the domain at the initial time, say $\omega\subset\Omega$ where $|\omega|&amp;gt;0$, then this part of the domain may grow or shrink. Because the solution is typically smooth in these areas (in fact, it is typically constant), numerical methods don't add artificial diffusion in these areas and consequently do not converge to a viscosity solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final note: An example for a flux $F(u)$ from multiphase flow in porous media is &#xA;$$&#xA;  F(u) = \frac{u^4}{u^4 + (1-u)^2 \cdot (1-u^2)}&#xA;$$&#xA;where $u$ is the saturation. You have $F'(u)=0$ for $u=0$, i.e., in areas where the saturation is zero.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-09T02:27:46.137" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4972" PostTypeId="1" CreationDate="2012-12-28T20:21:14.727" Score="2" ViewCount="97" Body="&lt;p&gt;I have the following 1st order equations and need to solve them using Maple 12. &#xA;There are unspecified initial conditions and can only be estimated through the Newton raphson method. My problem is how do I implement it so that the equations are numerically solved using rk4-order. Tried out with BVPsolve but doesn't work.&#xA;this is in line with works by Makinde. &#xA;&quot;On MHD boundary-layer flow and mass transfer past a vertical plate in a porous medium with&#xA;constant heat flux.&quot;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; k1 := diff(X[1](t), t) = X[2](t);&#xA;&amp;gt; k2 := diff(X[2](t), t) = M*(X[1](t)-1)-(2*(eta+b))*X[2](t);&#xA;&amp;gt; k3 := diff(X[3](t), t) = X[4](t);&#xA;&amp;gt; k4 := diff(X[4](t), t) = (2*Sc*Du*(eta+b)*X[6](t)-Du*lambda*X[5](t)-2*Pr*(eta+b)*X[4](t)-   Pr*Ec*X[2](t)^2-Pr*Ec*M*(X[1](t)-1)^2)/(1-Du*Sr);&#xA;&amp;gt; k5 := diff(X[5](t), t) = X[6](t);&#xA;&amp;gt; k6 := diff(X[6](t), t) = (lambda*X[3](t)+2*Pr*Sr*(eta+b)*X[4](t)+Pr*Sr*Ec*X[3](t)^2+Pr*Sr*Ec*M*(X[1](t)-1)^2-2*Sc*(eta+b)*X[6](t))/(1-Du*Sr);&#xA;&amp;gt; ICS := X[1](0) = 0, X[2](0) = S[1], X[3](0) = 1, X[4](0) = S[2], X[5](0) = 1, X[6](0) = S[3];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="Saronie" LastEditorUserId="549" LastEditDate="2013-01-10T15:35:01.097" LastActivityDate="2013-01-10T15:35:01.097" Title="System of non-linear ODEs and estimating unspecified initial conditions on Maple 12" Tags="|ode|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4973" PostTypeId="1" AcceptedAnswerId="4976" CreationDate="2013-01-10T13:24:05.040" Score="16" ViewCount="2178" Body="&lt;p&gt;I need simple explanation of the Multigrid Method or some literature about this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiar with iterational methods including BiCGStab,CG,GS,Jacobi and preconditioning, but I am a beginner with multigrid method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain this in detail or at least provide clearly pseudocode or source code, even with good literature for very beginners? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3544" LastActivityDate="2016-06-06T11:42:02.890" Title="multigrid method to solve PDE" Tags="|linear-algebra|pde|multigrid|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4974" PostTypeId="2" ParentId="4973" CreationDate="2013-01-10T14:06:39.153" Score="16" Body="&lt;p&gt;This site is possibly not a good place to ask for a detailed explanation with pseudocode (as stated in the &lt;a href=&quot;https://scicomp.stackexchange.com/faq&quot;&gt;FAQ&lt;/a&gt;, &quot;If you can imagine an entire book that answers your question, you’re asking too much.&quot;), so you might want to start with one of the classical books on this topic (listed below) and come back with specific questions about concrete details you have trouble with.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://computation.llnl.gov/casc/people/henson/mgtut/welcome.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Briggs, &lt;strong&gt;Multigrid Tutorial&lt;/strong&gt;, SIAM, 2000&lt;/a&gt; (You can download slides &lt;a href=&quot;https://computation.llnl.gov/casc/people/henson/mgtut/ps/mgtut.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://computation.llnl.gov/casc/people/henson/mgtut/ps/mgtut_part2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&#xA;This is a casual source, providing a gentle introduction to multigrid principles, mostly for elliptic problems.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.ec-securehost.com/SIAM/CL67.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brandt, &lt;strong&gt;Multigrid Techniques&lt;/strong&gt;, revised edition, SIAM 2011&lt;/a&gt;, (or download the &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~achi/classics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf&lt;/a&gt;).&#xA;This is a great development of multigrid philosophy and multiscale modeling and stands a good chance of profoundly changing your way of thinking about implicit solvers.  &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~achi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Achi Brandt's website&lt;/a&gt; contains many more references, including his &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~achi/review00.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2000 Review of Multiscale Scientific Computation&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/Multigrid.html?id=-og1wD-Nx_wC&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trottenberg, Oosterlee, and Schueller, &lt;strong&gt;Multigrid&lt;/strong&gt;, Academic Press, 2001&lt;/a&gt; This has more worked examples than Brandt, including many experiments and details on specific methods, especially in the context of fluid dynamics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-3-540-12761-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hackbusch, &lt;strong&gt;Multigrid Methods and Applications&lt;/strong&gt;, Springer, 1985&lt;/a&gt;&#xA;This provides a rigorous convergence theory, including &quot;multigrid of the second kind&quot; for Fredholm integral operators.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-01-23T05:35:02.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4975" PostTypeId="2" ParentId="4973" CreationDate="2013-01-10T19:58:41.707" Score="6" Body="&lt;p&gt;Another classic:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Wesseling, An Introduction to Multigrid Methods, John Wiley &amp;amp; Sons, 1992.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Example codes can be found at &lt;a href=&quot;http://www.mgnet.org/&quot;&gt;MGNet&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-01-10T19:58:41.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4976" PostTypeId="2" ParentId="4973" CreationDate="2013-01-10T20:05:22.980" Score="16" Body="&lt;p&gt;The main idea behind multigrid is projection.  I try to think about it as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I want to solve a PDE on with a lot of accuracy, so I proceed to discretize the domain (let's say, using finite difference method) on a very fine grid with lots and lots of points.  In the end, I setup my system of equations and I'm ready to solve it.  I try using my favorite iterative solver (jacobi, gauss seidel, conjugate gradient, etc...).  I proceed to wait more than a day and realize my computer is still trying to compute the answer!!!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why these iterative methods aren't working quickly is because (typically) when you setup a large system of equations like this, the matrix itself has eigenvalues extremely close to 1.  Why does this matter?  Because the rate of convergence of many iterative methods is inversely related to the largest eigenvalue (see Christian Clason's link to Brigg's Multigrid Tutorial Slides, part 1, page 27).  So, the closer the largest eigenvalue is to 1, the slower the iterative method is. (Note: this is oversimplifying things a bit, but it helps motivate the need for multigrid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, it is always faster to solve the problem if there are fewer unknowns (i.e. on a coarse grid with fewer gridpoints).  But more importantly, &lt;strong&gt;the solution (or approximate solution) on a coarser grid is a good starting point to solve the problem on a finer grid.&lt;/strong&gt;  This is the key idea behind most (if not all) multigrid methods.  Why is this the case?  Intuitively, it makes sense, but there is a mathematically rigorous way of justifying this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's look at the fourier modes of the error in an iterative method (for arguments sake, let's say jacobi or gauss seidel) applied to the original fine grid problem.  We would see that within the first few iterations, most of the high frequency (highly oscillatory) errors is removed!  This is great, but there is low frequency (less oscillatory) error that still remains and doesn't go away quickly.  In fact, it is low frequency error that prevents a standard iterative method from converging quickly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;when we solve the problem on a coarser grid (let's say, by an iterative method like jacobi or gauss-seidel), &lt;strong&gt;we are essentially able to remove lower frequency errors much more quickly (i.e. in fewer iterations) than on the fine grid&lt;/strong&gt;.  So, if we solve the problem of a coarse grid, we have a solution whose lower frequency errors have been lessed significantly.  Thus, it would be useful as a starting point for an iterative method on the finer grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are different multigrid methods, most of them operate by some variation of following:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start with the fine grid problem&lt;/li&gt;&#xA;&lt;li&gt;Project onto a coarse grid (also known as &lt;strong&gt;restriction&lt;/strong&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Approximate the solution on the coarse grid (using some other solver)&lt;/li&gt;&#xA;&lt;li&gt;Project the coarse grid solution onto the finer grid (also known as &lt;strong&gt;prolongation&lt;/strong&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Using the projection from 4. as an initial guess, solve the fine grid problem by an iterative method.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For me, the most difficult part of the multigrid method is the projections between grids.  The Briggs tutorials suggested by @ChristianClason handle this subject much better than I can.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-01-10T22:16:00.567" LastActivityDate="2013-01-10T22:16:00.567" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4977" PostTypeId="1" CreationDate="2013-01-11T00:39:27.027" Score="3" ViewCount="114" Body="&lt;p&gt;I do image processing, and right now I need to resample some images taken from slightly different perspectives so I can match up features. The pixel intensities have scientific significance, so I want to do something that will preserve local flux and not &quot;miss&quot; anything. What I want to do is exact area resampling, as discussed here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-261.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-261.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Except the coordinate change I'm doing is not just a translation; at the very least it involves translation, rotation, and perspective change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find anything on file exchange or by googling. Any ideas? &lt;/p&gt;&#xA;" OwnerUserId="3546" LastEditorUserId="1804" LastEditDate="2013-01-11T14:24:33.773" LastActivityDate="2013-01-11T14:24:33.773" Title="exact area resampling" Tags="|algorithms|computational-geometry|" AnswerCount="0" CommentCount="1" ClosedDate="2013-09-05T14:46:43.433" ContentLicense="CC BY-SA 3.0" />
  <row Id="4979" PostTypeId="1" AcceptedAnswerId="4986" CreationDate="2013-01-11T02:53:41.613" Score="0" ViewCount="586" Body="&lt;p&gt;I have see the method PCHIP in matlab that implements the monotone Hermite interpolation method which was originally proposed by Carlson in 1980s. It seem to accomplish the goal of preventing the values go outside of the range. I have not seen the error estimates results but I guess it does as good as cubic polynomial and locally $O(h^4)$. Now, there are more recent ENO/WENO methods and their multiple &quot;kids&quot;. Would like to hear why these methods stand out and why they are &quot;better&quot; or worse compare to monotone Hermite?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2013-01-12T03:12:58.253" Title="ENO/WENO vs monotone Hermite interpolation" Tags="|hyperbolic-pde|interpolation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4982" PostTypeId="1" AcceptedAnswerId="5061" CreationDate="2013-01-11T13:20:17.080" Score="4" ViewCount="6409" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do you know some good free Matlab LBFGS implementations? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The only one I know (and use for the moment) is Liam Stewart's (it can be found at the following link: &lt;a href=&quot;http://www.cs.toronto.edu/~liam/software.shtml&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.toronto.edu/~liam/software.shtml&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My interest is using these algorithms in shape optimization problems, where the sizes of the problems tend to get very large. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do you have any other suggestions of good and fast packages that can be used for optimizatioin problems?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2016-12-25T12:49:16.293" Title="LBFGS or other optimization algorithms - implementations MATLAB" Tags="|optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4983" PostTypeId="2" ParentId="4982" CreationDate="2013-01-11T14:24:01.540" Score="3" Body="&lt;p&gt;A good Matlab implementation of limited-memory BFGS is the one accompanying Tim Kelley's book &lt;a href=&quot;http://www.siam.org/books/kelley/fr18/index.php&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Iterative Methods for Optimization&lt;/em&gt;&lt;/a&gt; (SIAM, 1999; PDF freely downloadable from the publisher's website). You can find his Matlab codes &lt;a href=&quot;http://www4.ncsu.edu/~ctk/matlab_darts.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your second question: I would suggest implementing the algorithm yourself, since it is then much easier to adapt to the special structure of your specific problem (parameter tuning, exploiting derivative information, etc.).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-01-11T14:24:01.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4984" PostTypeId="1" AcceptedAnswerId="4988" CreationDate="2013-01-11T23:45:09.043" Score="4" ViewCount="997" Body="&lt;p&gt;For the optimization problem $\underset{\mathbf{x}\in \mathbb{R}^n}{\operatorname{argmin}} f(\mathbf{x})$, we can use the following standard nonlinear conjugate gradient method to find the solution:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$\mathbf{d}_0 =  -\nabla f(\mathbf{x}_0),$&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{x}_{i+1}=\mathbf{x}_i+\alpha_i\mathbf{d}_i,$ where $\alpha_i$ is found by line search,&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{g}_{i+1} = \nabla f(\mathbf{x}_{i+1})$,&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{d}_{i+1}= -\mathbf{g}_{i+1} + \beta_i\mathbf{d}_i$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I wonder for the problem $\underset{\mathbf{x}\in \mathbb{R}^n, \; \mathbf{y}\in \mathbb{R}^n}{\operatorname{argmin}} f(\mathbf{x},\mathbf{y})$, where $f$ is a function of 2 variables, whether the corresponding nonlinear conjugate gradient method is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$\mathbf{d}_0^x = -\nabla_{\mathbf{x}} f(\mathbf{x}_0, \mathbf{y}_0), \mathbf{d}_0^y = -\nabla_{\mathbf{y}} f(\mathbf{x}_0,  \mathbf{y}_0)$, where $\nabla_{\mathbf{x}}f$ and $\nabla_{\mathbf{y}}f$ are the derivatives of $f$ with respect to $\mathbf{x}$ and $\mathbf{y}$,&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{x}_{i+1}=\mathbf{x}_i+\alpha_i\mathbf{d}_i^x$, $\mathbf{y}_{i+1}=\mathbf{y}_i+\alpha_i\mathbf{d}_i^y$, &lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{g}_{i+1}^x = \nabla_{\mathbf{x}} f(\mathbf{x}_{i+1},\mathbf{y}_{i+1})$, $\mathbf{g}_{i+1}^y = \nabla_{\mathbf{y}} f(\mathbf{x}_{i+1},\mathbf{y}_{i+1})$,&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{d}_{i+1}^x= - \mathbf{g}_{i+1}^x + \beta_i^x\mathbf{d}_i^x$, $\mathbf{d}_{i+1}^y= - \mathbf{g}_{i+1}^y + \beta_i^y\mathbf{d}_i^y$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Also, I am not sure how to compute $\beta_i^x$, $\beta_i^y$. If $\beta_i^x,\beta_i^y$ are computed by the Fletcher–Reeves method, is $\beta_i^x=\beta_i^y=\frac{\|\mathbf{g}_{i+1}\|^2}{\|\mathbf{g}_{i}\|^2}$ (where $\mathbf{g}_{i} = [\mathbf{g}_{i}^x, \mathbf{g}_{i}^y]$), or $\beta_i^x=\frac{\|\mathbf{g}_{i+1}^x\|^2}{\|\mathbf{g}_{i}^x\|^2}$, $\beta_i^y=\frac{\|\mathbf{g}_{i+1}^y\|^2}{\|\mathbf{g}_{i}^y\|^2}$?&lt;/p&gt;&#xA;" OwnerUserId="1614" LastActivityDate="2017-05-25T03:12:01.787" Title="nonlinear conjugate gradient for multivariable functions" Tags="|optimization|nonlinear-programming|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4985" PostTypeId="1" CreationDate="2013-01-11T23:45:43.050" Score="3" ViewCount="364" Body="&lt;p&gt;Do you know any methods/solvers for PDE or eigenvalue problems like&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\begin{cases} \Delta u= 0\ (\text{ or }\lambda u) &amp;amp; \text{ in }\Omega \\ u =0 &amp;amp; \text{ on }\partial \Omega \end{cases}$ (Dirichlet)&lt;/li&gt;&#xA;&lt;li&gt;$\begin{cases} \Delta u= 0\ (\text{ or }\lambda u) &amp;amp; \text{ in }\Omega \\ \frac{\partial u}{\partial n} =0 &amp;amp; \text{ on }\partial \Omega \end{cases}$&#xA;(Neumann)&lt;/li&gt;&#xA;&lt;li&gt;$\begin{cases} \Delta u= 0\ (\text{ or }\lambda u) &amp;amp; \text{ in }\Omega \\ \frac{\partial u}{\partial n}+\beta u =0 &amp;amp; \text{ on }\partial \Omega \end{cases}$&#xA;(Robin)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;in 2D and 3D that do not involve the Finite Element Method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2D there is at least an option (for Dirichlet and Neumann): &lt;a href=&quot;http://code.google.com/p/mpspack/&quot; rel=&quot;nofollow&quot;&gt;mpspack&lt;/a&gt; which uses a basis for the space of harmonic functions and works only with coefficients in this basis. In 3D I do not know anything that doesn't use Finite Element Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in a different method than FEM because in shape optimization domains change at every iteration and so does the mesh, the discretization, etc.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="3548" LastEditDate="2013-01-12T09:32:42.133" LastActivityDate="2013-01-12T09:32:42.133" Title="Solving PDE or eigenvalue problems without FEM" Tags="|pde|eigenvalues|elliptic-pde|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4986" PostTypeId="2" ParentId="4979" CreationDate="2013-01-12T03:12:58.253" Score="4" Body="&lt;p&gt;PCHIP is not a conservative reconstruction, making it inappropriate for conservation laws. Furthermore, hyperbolic problems have &lt;em&gt;discontinuous&lt;/em&gt; solutions so there is generally no benefit to a continuous reconstruction. Conservative monotone spline reconstructions are being investigated by the UK Met Office for use in tracer advection for atmosphere modeling, see papers on the &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2007.01.006&quot; rel=&quot;nofollow&quot;&gt;multi-dimensional&lt;/a&gt; case, &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2009.10.018&quot; rel=&quot;nofollow&quot;&gt;quartic&lt;/a&gt; splines, and &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2006.11.009&quot; rel=&quot;nofollow&quot;&gt;applications&lt;/a&gt;. These methods are relatively new and are not currently popular with many other groups. Some reasons for this include&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;nonlocal reconstruction is inconvenient, especially in parallel&lt;/li&gt;&#xA;&lt;li&gt;these methods are semi-Lagrangian and currently only suitable for advection, especially in multiple dimensions&lt;/li&gt;&#xA;&lt;li&gt;there is no characteristic spline-based reconstruction&lt;/li&gt;&#xA;&lt;li&gt;only structured grids can be used&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-01-12T03:12:58.253" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4987" PostTypeId="2" ParentId="3262" CreationDate="2013-01-12T03:34:20.557" Score="7" Body="&lt;p&gt;The decision tree in Section 4 of the &lt;a href=&quot;http://www.nag.com/numeric/fl/nagdoc_fl23/xhtml/F04/f04intro.xml&quot;&gt;relevant chapter in the NAG Library Manual&lt;/a&gt; answers (in part) some of your questions.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-01-12T03:34:20.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4988" PostTypeId="2" ParentId="4984" CreationDate="2013-01-12T03:46:06.233" Score="8" Body="&lt;p&gt;If you introduce a variable $\mathbf z=(\mathbf x^T, \mathbf y^T)^T \in \mathbb{R}^{2n}$, then you can write $f(\mathbf x, \mathbf y)=f(\mathbf z)$ and it fits the exact format you wrote in your outline of the first method and it will all work as described. You just have to match things like $\nabla_z f(\mathbf z) = (\nabla_x f(\mathbf x,\mathbf y)^T, \nabla_y f(\mathbf x,\mathbf y)^T)^T$ etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="1614" LastEditDate="2013-01-12T04:47:08.837" LastActivityDate="2013-01-12T04:47:08.837" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4989" PostTypeId="1" AcceptedAnswerId="4994" CreationDate="2013-01-12T19:32:14.037" Score="4" ViewCount="432" Body="&lt;p&gt;I have a variable coefficient pde of the form $$u_t=c(t,x)u_{xx}, t\in [0,T], x\in [0,1]$$ with initial data $u_0=u(0,x)$ and $c(x,t)\in C([0,T]\times[0,1])$. I use three point discretization for the diffusion part. I would like to have en error estimate by proving stability and consistency. Doing Taylor expansion, I can show consistency. However, I am confused how to choose the norm? I have seen in the book the author chose discrete $L^2$ norm and by using discrete energy methods the stability followed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First question for the discretized equation: If I pick another discrete norm, which one should I pick (I mean one of the $L^p$ norms), which one is easier or harder? So far, I have obtained the estimate that $||\vec{e}||_2=O(h^2)$, where $\vec{e}:=\vec{u}_h(T)-\vec{u}(T)$. But the other similar estimates would follow in an analogous way once I prove the stability in those norms. Please provide some hints how to choose another norm and what are the &quot;tools&quot; to prove stability for such an equation in other discrete $L^p$ norms.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And second question: is the choice affected by the stability estimate of the original pde? That is, once I establish continuous dependence on initial data, (it follows from maximum principle that I have stability in $L^{\infty}$) and assume I have managed to show it is well-posed in $L^2([0,T]\times [0,1])$(by the same energy methods in continuous setting), does imply it is well posed in $L^1$ or $L^{\infty}$? How is that connected to the fact $L^{\infty}\in L^{2} \in L^1$?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2013-02-04T04:23:48.217" Title="choice of the norm for Crank Nicolson stability estimate" Tags="|stability|parabolic-pde|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4990" PostTypeId="2" ParentId="926" CreationDate="2013-01-12T20:48:47.107" Score="4" Body="&lt;p&gt;Yes you absolutely can, I did. Much of the power out of MATLAB however does come with some of its hugely easy to use toolboxes and builtins which may or may not have equivalents in Octave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also be aware that while the base languages themselves are compatible (except for the newish OO features in MATLAB), Octave has a few 'additions' to the syntax, like being able to use &quot;!=&quot; for NEQ, and it also has some alternate default behaviors like how it parses inline functions. These are minor things that will cause Octave code not to run in MATLAB right off the bat unless you've taken care to avoid relying on that behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also MATLAB is a full computing environment, and not just a language that interfaces with optimized linear algebra routines. So what you learn out of Octave will only go so far. You will be functional in MATLAB if you are proficient with Octave, but you won't be as productive with MATLAB as you could be.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-01-12T20:48:47.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4991" PostTypeId="2" ParentId="480" CreationDate="2013-01-13T02:55:48.240" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://mathformeremortals.wordpress.com/2013/01/12/a-numerical-second-derivative-from-three-points/&quot; rel=&quot;nofollow&quot;&gt;http://mathformeremortals.wordpress.com/2013/01/12/a-numerical-second-derivative-from-three-points/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This addresses your question and shows the formula you are looking for, for the second derivative.  Higher-Order derivatives follow the same pattern.&lt;/p&gt;&#xA;" OwnerUserId="3554" LastActivityDate="2013-01-13T02:55:48.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4992" PostTypeId="1" CreationDate="2013-01-13T07:18:45.367" Score="6" ViewCount="942" Body="&lt;p&gt;I am trying to use LAPACK geqp3 function for QR decomposition with pivoting but result is wrong. I tried almost two days but can't figure out the problem. I compared the result with Matlab and Python CvxOpt both are same and both uses the geqp3  LPACK routines as internally .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is input matrix and code. Any one can tell me what i am doing wrong ? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Blockquote A =&#xA;   -0.9586   -0.8171   -0.6033    1.1895    1.3031   -0.0759&#xA;    1.9877    1.0979    0.7795   -0.7164   -0.0505   -0.2412&#xA;    0.2328    0.2210    0.9401    0.0691    0.1082   -0.8700&#xA;    0.2808    0.9539    0.0198   -0.2278   -1.1498   -0.3127&#xA;    0.1335    1.2238    0.5449    0.2712   -0.7684   -2.5430&#xA;    0.3996   -0.5725    0.7090   -1.2492    0.5508   -1.0513&#xA;   -0.3073    0.2512   -1.2444    0.1169   -1.1271    0.6893&#xA;    0.3238   -0.8187    2.7957    0.3582    1.8493   -1.5166&#xA;    0.6406   -0.4428   -0.1864   -1.0097    0.6713    0.7002&#xA;&#xA;&#xA;void fReadMtrx(string f, double *mtrx, int nrow, int ncol){&#xA;&#xA;    ifstream data(f.c_str(), ios::in);&#xA;    if (!data) {&#xA;        cerr &amp;lt;&amp;lt; &quot;File &quot; &amp;lt;&amp;lt; f &amp;lt;&amp;lt; &quot; could not be opened.&quot; &amp;lt;&amp;lt; endl;&#xA;        exit(1);&#xA;    }&#xA;&#xA;    for(int i = 0; i &amp;lt; nrow; i++) &#xA;        for(int j = 0; j &amp;lt; ncol; j++) &#xA;            data &amp;gt;&amp;gt; mtrx[j*nrow+i]; &#xA;}&#xA;&#xA;void showMtrx(double *mtrx, int nrow, int ncol){&#xA;&#xA;    for (int i = 0; i &amp;lt; nrow; i++) {&#xA;        for (int j = 0; j &amp;lt; ncol; j++) { &#xA;            cout &amp;lt;&amp;lt;  mtrx[j*nrow+i] &amp;lt;&amp;lt; &quot;  &quot;;&#xA;        }&#xA;        cout &amp;lt;&amp;lt; endl;&#xA;    }&#xA;}&#xA;&#xA;&#xA;int main(int argc, char *argv[])&#xA;{&#xA;    int m = 9;&#xA;    int n = 6;&#xA;&#xA;    //allocate and read in a&#xA;    double* a = new double[m*n];&#xA;&#xA;    fReadMtrx(&quot;A.dat&quot;, a, m, n);&#xA;&#xA;    cout &amp;lt;&amp;lt; endl &amp;lt;&amp;lt; &quot;a:&quot; &amp;lt;&amp;lt; endl;&#xA;    showMtrx(a, m, n);&#xA;&#xA;    int ldA = m;&#xA;    double wl;&#xA;    int lwork;&#xA;    int info;&#xA;&#xA;&#xA;    // QR decomposition&#xA;    lwork = -1;&#xA;    dgeqp3_(&amp;amp;m, &amp;amp;n, NULL, &amp;amp;ldA, NULL, NULL, &amp;amp;wl, &amp;amp;lwork, &amp;amp;info);&#xA;&#xA;    lwork = (int)wl;&#xA;    double* work  = new double[lwork];&#xA;    int* jpvt_ptr = new int[n];&#xA;    double* tau = new double[n];&#xA;&#xA;    dgeqp3_(&amp;amp;m, &amp;amp;n, a, &amp;amp;ldA, jpvt_ptr,tau,(double *)work, &amp;amp;lwork, &amp;amp;info);&#xA;&#xA;    cout &amp;lt;&amp;lt; endl &amp;lt;&amp;lt; &quot;QR:&quot; &amp;lt;&amp;lt; endl;&#xA;    showMtrx(a, m, n);&#xA;&#xA;&#xA;   return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When i perform QR decomposition using geqp3 routines it gives me this result.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;QR :&#xA; 2.4063   1.0778   1.6110  -1.5135  -0.0642  -0.7111 &#xA;-0.5907   2.1035  -1.1498   0.4283  -2.6023  -0.6335 &#xA;-0.0692  -0.0423   2.8526   0.7086   0.9759  -2.7723&#xA;-0.0835  -0.3745  -0.0332   1.3431   0.8472   0.5267&#xA;-0.0397  -0.5405  -0.3887  -0.3816   0.7893  -1.2573&#xA;-0.1188   0.3753  -0.1044   0.9602  -0.0370   1.1425&#xA; 0.0913  -0.1996   0.4502  -0.1306   2.1257  -0.1959&#xA;-0.0962   0.4710  -1.1228   0.0682   1.4831  -0.4424&#xA;-0.1904   0.3782   0.4133   0.4090  -1.0372   0.2011&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;While in python and matlab i am getting this result.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;QR :&#xA;   [ 3.47196268, 1.63382425, -2.39788322, -0.26189627,  1.11652819,  -0.1965108 ],&#xA;   [-0.19128521,  2.52976731,  0.93296076,  0.49012156, -0.78217128, -2.06427021],&#xA;   [-0.23067443, -0.0120604 ,  2.26624645, -0.67846333,  0.74839188,  -0.28431637],&#xA;   [-0.00486702,  0.43558037, -0.00634783, -2.00278565,  1.22747977,  -0.01853067],&#xA;   [-0.13370508,  0.3074231 ,  0.76958444, -0.0335343 ,  1.36566733,   1.04995806],&#xA;   [-0.17396713, -0.18659867,  0.30029599, -0.6800568 ,  0.37513732,   0.3212295 ],&#xA;   [ 0.30534072,  0.38815978, -0.10102664,  0.06497129,  0.35993755,   0.72318025],&#xA;   [-0.68601098, -0.61373672,  0.14243916,  0.28777571, -0.68376182,  0.3557272 ],&#xA;   [ 0.04573902, -0.25965053, -0.15498792, -0.52485689, -0.0986465 ,  -0.38370839]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3555" LastEditorUserId="1552" LastEditDate="2013-01-15T15:06:42.283" LastActivityDate="2013-01-16T01:16:50.263" Title="Wrong Result QR decomposition with Pivoting in LAPACK" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4993" PostTypeId="1" CreationDate="2013-01-13T16:57:32.330" Score="7" ViewCount="230" Body="&lt;p&gt;I'm trying to solve the transient behaviour of a compressible flow through a porous medium (porosity is $\epsilon$) where the fraction of B in A ($ \phi$) changes with time. The equations that I found are the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;continuity:&#xA;$\frac{\partial \rho} {\partial t}+\frac{1}{\epsilon}\frac{\partial \rho u} {\partial x}=0 $&lt;/li&gt;&#xA;&lt;li&gt;fraction of component B&#xA;$\frac{\partial \rho \phi} {\partial t}+\frac{1}{\epsilon}\frac{\partial \rho u \phi} {\partial x}=0  $&lt;/li&gt;&#xA;&lt;li&gt;equation of state:&#xA;$P = \rho \cdot R \cdot T \cdot Z(P,\phi)$&lt;/li&gt;&#xA;&lt;li&gt;Darcy:&#xA;$u = -\frac{k}{\mu} \frac{\partial P} {\partial x}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The imposed boundary conditions are:&#xA;$\phi=\phi(t) $,&#xA;$P(x=L)=P_{back}$ and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$P(x=0)=P_{inlet}$ or $u(x=0)=u_{inlet}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I proceed with this problem using a finite differences approach? Filling the third equation in the fourth yields an expression of $u$ as function of $\rho$ which can be used in the first two equations. So one would get:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;continuity:&#xA;$\frac{\partial \rho} {\partial t}+K\frac{\partial \rho \frac{\partial \rho \cdot Z} {\partial x}} {\partial x}=0 $&lt;/li&gt;&#xA;&lt;li&gt;fraction of component B:&#xA;$\frac{\partial \rho \phi} {\partial t}+K\frac{\partial \rho \phi \frac{\partial \rho \cdot Z}{\partial x}} {\partial x}=0  $&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is this a good way in solving the problem? I would proceed with using a discretization scheme for the spatial derivative and using Runge-Kutta to find the time derivative. Then I would get $\rho$ and $\rho \cdot \phi$. But can I do this, because $\rho = f(\phi)$ so there is also a coupling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A last question is, how should I impose the boundary condition for $u$? Because $u$ is not in these last two equations anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3556" LastEditorUserId="24110" LastEditDate="2017-05-13T15:17:50.980" LastActivityDate="2017-09-10T16:41:39.290" Title="Compressible flow through a porous medium with a variable inlet concentration" Tags="|finite-difference|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4994" PostTypeId="2" ParentId="4989" CreationDate="2013-01-13T19:09:21.683" Score="1" Body="&lt;p&gt;For your case, the $L^2$ norm is natural. If you had an equation of the kind&#xA;$$&#xA;  \rho(x) u_t - (a(x) u_x)_x = f&#xA;$$&#xA;then the natural norms would be $\|u\|_{L^2_\text{weighted}} = \|\sqrt{\rho} u\|_{L^2}$ as the analog of the $L^2$ norm, and $|u|_{H^1_\text{weighted}} = \|\sqrt{a} u_x\|_{L^2}$ as the analog of the energy ($H^1$) norm. You get these naturally if you multiply the equation by $u$ or $u_x$, respectively, and integrate over the domain. If the two coefficients are bounded pointwise from below and above, these norms are of course equivalent to the $L^2$ and $H^1$ norms, respectively. Thus, if you have a result in the weighted norms, then you immediately have the same result in the original norms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this technique, show convergence in $L^2_\text{weighted}$, then you immediately have convergence in $L^2$ as well, and consequently in $L^1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't seem to see much sense in showing convergence in any other of the $L^p$ norms for $p$ not either $1,2,\infty$. Nobody seems to care about other values of $p$ and I can not see a reason to care either.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-02-04T04:23:48.217" LastActivityDate="2013-02-04T04:23:48.217" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4995" PostTypeId="1" AcceptedAnswerId="5069" CreationDate="2013-01-14T01:37:40.747" Score="4" ViewCount="2596" Body="&lt;p&gt;What are generally best choices for enumerating all k-cliques (or independent sets of size k)?&#xA;The graphs I am looking at probably won't have more than ~ 100 nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently I code in Python with NetworkX, so if you know of any Python packages that could help me that would be fantastic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your help is much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="1530" LastEditorUserId="1530" LastEditDate="2013-01-30T17:05:44.863" LastActivityDate="2013-01-30T17:05:44.863" Title="Find / Detect / Enumerate all Cliques or Independent Sets" Tags="|python|graph-theory|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4996" PostTypeId="1" AcceptedAnswerId="4997" CreationDate="2013-01-14T08:53:48.363" Score="2" ViewCount="1006" Body="&lt;p&gt;I am new to Ansys Fluent (the Ansys CFD product). Is it a windows-only software? &#xA;My institute owns copies that work on windows only.&#xA;Online tutorials on how to install Fluent on ubuntu aren't clear.&lt;/p&gt;&#xA;" OwnerUserId="3558" LastActivityDate="2013-01-14T13:00:27.000" Title="Where can I find Ansys Fluent for ubuntu?" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4997" PostTypeId="2" ParentId="4996" CreationDate="2013-01-14T13:00:27.000" Score="3" Body="&lt;p&gt;It can be used on Linux systems but it's not free. So if your university only provides Windows versions, you will have to convince them to pay for Linux versions as well or buy it yourself.&lt;/p&gt;&#xA;" OwnerUserId="391" LastActivityDate="2013-01-14T13:00:27.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4998" PostTypeId="1" AcceptedAnswerId="5000" CreationDate="2013-01-14T20:35:17.233" Score="1" ViewCount="1319" Body="&lt;p&gt;I am searching some libraries or toolboxes (preferred MATLAB) for solving acoustic wave equation in heterogeneous media with time varying source term, i.e. $$\nabla^2 \psi(\vec{r},t) - \frac{1}{c(\vec{r})^2} \frac{\partial^2}{\partial t^2}\psi(\vec{r},t) = s(\vec{r},t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually I want to verify the &lt;a href=&quot;http://www.k-wave.org/&quot; rel=&quot;nofollow&quot;&gt;k-Wave&lt;/a&gt; toolbox I modified, which solves another version of the above equation, by using some other softwares, so are there any other recommendations other than k-Wave? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1614" LastActivityDate="2013-01-15T02:33:50.103" Title="software request for solving acoustic wave equation" Tags="|matlab|reference-request|libraries|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4999" PostTypeId="2" ParentId="4989" CreationDate="2013-01-14T22:05:11.177" Score="1" Body="&lt;p&gt;In general, well-posedness (with combines existence, uniqueness, and stability) holds in the space for which you showed it only. There can be no solutions in a smaller space, and multiple solutions in a larger space. The continuous dependence of the solution on the data also depends on the norms (and thus on the normed space) in which you take data and solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Wolfgang Bangerth already pointed out, there is a natural norm (the energy norm) in which stability holds for the continuous problem and thus for any consistent approximation. If(!) your data is sufficiently smooth, you can of course improve this to get stability in higher norms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you focus on a specific discretization, you are correct that equivalence of norms in finite dimensions gives you stability in any norm as long as you have stability in one norm. However (and this is the crucial point), the equivalence constants depends on your discretization, and usually blow up as the discretization parameter $h$ tends to 0. Needless to say, this makes the equivalence argument useless for proving error estimates as $h$ tends to 0...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should also point out that $L^\infty$ in time error estimates are pointwise estimates for $u_h(t)$ at any $t\in[0,T]$, not just for the time steps $u_n = u(t_n)$. This makes these estimates much harder to obtain than (say) $L^2$ estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I think you have the question the wrong way around: It's not&#xA;&quot;Let's show convergence, what norm should I pick?&quot; but &quot;I need convergence in that norm, how do I show it?&quot; (If any norm will do, use the energy norm.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-01-14T22:05:11.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5000" PostTypeId="2" ParentId="4998" CreationDate="2013-01-15T02:33:50.103" Score="4" Body="&lt;p&gt;There is a tutorial program for this in deal.II: step-24, at&#xA;  &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_24.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_24.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I'm one of the authors of this library.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-15T02:33:50.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5002" PostTypeId="2" ParentId="4951" CreationDate="2013-01-15T15:32:13.877" Score="4" Body="&lt;p&gt;It looks like the problem is that there's a factor of &lt;code&gt;b&lt;/code&gt; that is common between &lt;code&gt;f1&lt;/code&gt; and &lt;code&gt;f2&lt;/code&gt;. If I divide both by &lt;code&gt;b&lt;/code&gt; then I get a meaningful answer for this modified system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the &lt;code&gt;CellPlot&lt;/code&gt; of the result, it seems that this decomposition is invariant under multiplying &lt;code&gt;b&lt;/code&gt; by a positive value. (There are different cells for &lt;code&gt;b&lt;/code&gt; positive or negative, separated at &lt;code&gt;b = 0&lt;/code&gt;, but otherwise the only interesting stuff happens in the &lt;code&gt;a&lt;/code&gt; direction.) So that suggests that for nonzero &lt;code&gt;b&lt;/code&gt;, the picture will be the same for the modified system and the original system... or does it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Disclaimer: I work for Maplesoft, but I don't really know anything about the &lt;code&gt;RootFinding:-Parametric&lt;/code&gt; package.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Edit: I talked to someone who &lt;em&gt;does&lt;/em&gt; know about &lt;code&gt;RootFinding:-Parametric&lt;/code&gt;, and he confirmed that this is indeed a bug. Am now submitting into our bug tracking system.&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="49" LastEditorUserId="49" LastEditDate="2013-01-15T18:40:33.787" LastActivityDate="2013-01-15T18:40:33.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5003" PostTypeId="1" AcceptedAnswerId="5025" CreationDate="2013-01-15T17:43:02.800" Score="4" ViewCount="2451" Body="&lt;p&gt;I sincerely apologise if this question is a duplicate. Though it is clearly a question that must have been asked and answered a 1000 times I can't find any reasonable solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I take a simple 2D CAD Drawing (.dwg file) of a 2D boundary and generate a 2D computational mesh suitable for FEM analysis in any of the following file formats:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.xml (DOLFIN XML), .ele/.node (Triangle file format), .mesh (Medit generated by TetGen), .msh/.gmsh (Gmsh), .grid (Diffpack tetrahedral), .inp (Abaqus tetrahedral), .e/.exo/.ncdf (Sandia Exodus II), .vrt/.cell (Star-CD)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing this on my own time so it must be free software. Any assistance would be greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Derek&lt;/p&gt;&#xA;" OwnerUserId="1880" LastActivityDate="2013-08-15T10:13:10.137" Title="Mesh simple 2d CAD boundry drawing" Tags="|finite-element|mesh-generation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5004" PostTypeId="1" CreationDate="2013-01-15T17:54:04.073" Score="2" ViewCount="104" Body="&lt;p&gt;Suppose $g(x_{1:n})$ is the estimate of a gradient, which is calculated at each step of a Stochastic Steepest Descent algorithm. A dataset $x_{1:n}$ is simulated at each step, so if $n$ is small the algorithm is fast but unstable, while if $n$ is large it is slow but stable. So far I have just experimented with many values of n, but maybe somebody knows a better way of determining n.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose $Var(g(x_{1:n})) = f(n)$ is known (i.e. I know how the variance of the gradient varies with the sample size), I was considering:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) To minimize a loss function of the type:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Loss(n)=f(n)+n×c.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) To plot $f(n)$ against $n \times c$ to try determine a good sample size n visually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case $c$ is the time needed to simulate $x_{i}$, for example 0.1sec. I don't think that minimizing a function were different units are summed together makes much sense, so I was wondering whether there is any way to translate CPU times into something more sensible. &lt;/p&gt;&#xA;" OwnerUserId="2140" LastActivityDate="2013-01-22T02:00:27.333" Title="Optimal sample size for Stochastic Steepest Descent" Tags="|optimization|stochastic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5005" PostTypeId="1" AcceptedAnswerId="5037" CreationDate="2013-01-15T21:23:47.900" Score="4" ViewCount="166" Body="&lt;p&gt;I'm pretty ignorant regarding the dark arts of numerical codes and modelling, but i'm interested in trying to pursue it for a particular pet project. It regards modelling of nuclear reactions like those required for NERVA-like rockets or &lt;a href=&quot;http://path-2.narod.ru/design/base_e/nswr.pdf&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any references regarding implementation, models that need to be used (i.e: Navier-Stokes 2D with MonteCarlo neutron diffusion, etc.), or for existing &lt;strong&gt;publicly available&lt;/strong&gt; codes or frameworks that could get me started. I've found that it is very hard to find most of those developed at Livermore because of Export restrictions. I've requested TART and MCNP5 downloads for a couple weeks now and i don't even get a reply&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Are there existing models implemented in publicly available codes for this kind of domain modelling? if not, what is the closest one and what kind of tweaking would be required?&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was able to find this one called &lt;a href=&quot;http://www.talys.eu/more-about-talys/talys-general-features/&quot; rel=&quot;nofollow&quot;&gt;TALYS&lt;/a&gt; but i haven't had a time to look at it deeply. Will try to keep this question updated as i learn more about what i need for my project and if/what portion of it is provided by this library&lt;/p&gt;&#xA;" OwnerUserId="372" LastEditorUserId="372" LastEditDate="2013-01-18T21:49:06.710" LastActivityDate="2013-01-18T21:49:06.710" Title="nuclear reaction fluid modelling" Tags="|fluid-dynamics|monte-carlo|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5006" PostTypeId="2" ParentId="4992" CreationDate="2013-01-16T01:16:50.263" Score="8" Body="&lt;p&gt;&lt;strong&gt;Interpreting the results&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To those of you confused as to the difference, he is reporting the compact representation of the pivoted QR factorization, where the upper triangle gives the factor $R$ from the pivoted QR decomposition and each column of the strictly lower triangle represents a Householder reflection, say&#xA;$$ H_j = I - \tau_j \left[\begin{array}{c}1\\ u_j\end{array}\right] \left[\begin{array}{cc}1 &amp;amp; u_j^H\end{array}\right],$$&#xA;where $u_j$ is the vector stored below the $j$'th diagonal entry of the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend loading his input matrix into MATLAB or Octave in order to follow along at home. You will notice that the first result is essentially the same as an unpivoted QR factorization, e.g., via &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X=qr(A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but with signs of the upper triangle appropriately changed so that the diagonal entries are positive. This is a strong clue as to the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fixing the problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look through the comments at the beginning of the &lt;a href=&quot;http://www.netlib.org/lapack/double/dgeqp3.f&quot;&gt;dgeqp3 source&lt;/a&gt;, you will notice that if an entry of the &lt;em&gt;JPVT&lt;/em&gt; array is nonzero, then that column is not allowed to be a pivot candidate. Notice that your code is simply allocating an integer array, which does not ensure that its contents are zero. If you explicitly zero the &lt;em&gt;JPVT&lt;/em&gt; array before calling dgeqp3 then I suspect that your problem will go away.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2013-01-16T01:16:50.263" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5007" PostTypeId="1" CreationDate="2013-01-16T02:43:49.107" Score="5" ViewCount="3353" Body="&lt;p&gt;I am looking for a very fast polynomial root finder, hopefully with a matlab code. I don't need very accurate results, 2-3 decimal places would be fine. Also the method should be able to optionally take as input an initial guess of the roots. If the initial guess is close, this should speed the process up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do I need this for? I'm working on images and the method I'm implementing requires the roots of a polynomial to be found, at every pixel in the image. Given typical images are from 512x512 and up, the method has to be fast at the expense of accuracy if need be.&lt;/p&gt;&#xA;" OwnerUserId="3565" LastActivityDate="2013-12-15T02:05:48.737" Title="Fastest polynomial root finder for a given accuracy" Tags="|iterative-method|polynomials|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5008" PostTypeId="2" ParentId="5007" CreationDate="2013-01-16T03:14:54.823" Score="4" Body="&lt;p&gt;If you have a reasonable starting guess and the function you're trying to find a root of is not overly complex, then you can't beat Newton's method. It's also easy to implement -- a 10 line exercise in most languages.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-16T03:14:54.823" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5009" PostTypeId="2" ParentId="5005" CreationDate="2013-01-16T03:22:45.140" Score="2" Body="&lt;p&gt;What a fascinating design for a rocket!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having -- unfortunately -- had to observe the struggle of some colleagues with the export control issues, I can certainly tell that you won't be able to find any open source codes for this kind of problem in the United States. Everything that could potentially solve such problems would also be able to model nuclear explosions and would, consequently, be export controlled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how other countries, say in Europe, treat these issues.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-16T03:22:45.140" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5010" PostTypeId="1" CreationDate="2013-01-16T06:46:52.110" Score="4" ViewCount="838" Body="&lt;p&gt;Given a machine precision unit vector $n$, and an arbitrary vector $v$, I want an unconditionally backward stable method to compute&#xA;$$f(v) = \frac{v-nn'v}{\left|v-nn'v\right|}$$&#xA;In other words, project $v$ orthogonal to $n$ and then normalize.  Specifically, the result should satisfy (to machine precision if possible):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$\left|f(v)\right| \approx 1$&lt;/li&gt;&#xA;&lt;li&gt;$n \cdot f(v) \approx 0$&lt;/li&gt;&#xA;&lt;li&gt;The approximation to $f(v)$ is exactly $f(\tilde{v})$ for some $\tilde{v}$ close to v except for normalization error (backward stability).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These should hold even if $n \cdot v = 0$  (in which case $f(v)$ can be any unit vector orthogonal to $n$).  The trick is to find some way of stably projecting and normalizing that doesn't lose accuracy as $n \cdot v$ approaches zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This formula arises in the analytic computation of extrapolated normals to cylinders.  For example, here is one &lt;a href=&quot;https://github.com/otherlab/core/blob/9a7d8bd0b3519dd5b810b7dc56515c5b947f0e47/geometry/Cylinder.cpp#L28&quot; rel=&quot;nofollow&quot;&gt;wrong way to do it&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-01-18T06:31:00.570" Title="Backward stable projection and normalization of a vector" Tags="|computational-geometry|stability|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5011" PostTypeId="2" ParentId="5010" CreationDate="2013-01-16T07:01:10.190" Score="1" Body="&lt;p&gt;In 3D, at least, the most direct way is to compute a orthogonal basis $u_0, u_1$ of vectors orthogonal to $n$, project $v$ into this basis, normalize in 2D, and expand back to 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Jed notes in his answer, the property that this algorithm satisfies is not true backward stability, but it does satisfy the property that I should have asked for, namely &lt;em&gt;stability&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I didn't remember this method until after I wrote the question.)&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2013-01-17T04:45:02.897" LastActivityDate="2013-01-17T04:45:02.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5012" PostTypeId="2" ParentId="5005" CreationDate="2013-01-16T08:09:13.983" Score="2" Body="&lt;p&gt;In Europe, nuclear codes are either proprietary or under the responsibility of the OECD Nuclear Energy Agency in Paris (which acts as an intermediate for some codes with RSICC, the US databank). If you live in an OECD country and you are an OECD citizen, you should be able to ask NEA for the country's &quot;liaison officer&quot;. She/He is the person who can request codes from the databank. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as professor Bangerth mentioned, for anything close to these topics, you will have to have a very good justification (and &quot;pet project&quot; won't fit) and be affiliated to a nuclear research center or university research group to request these codes. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-01-16T08:09:13.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5013" PostTypeId="1" CreationDate="2013-01-16T08:31:59.497" Score="2" ViewCount="441" Body="&lt;p&gt;The Matlab version is 2012b. I am trying using the built-in GA functionality through the Optimization Tool GUI on Matlab. I want to use bit string chromosome with a given length (for example L=24) and a given way to map one chromosome to a real number (suppose the objective function is one dimensional). I have been searching the documentation for some time but haven't found anything useful. Can someone give me a clue how to customize the chromosome length and the chromosome-to-real_number mapping?&lt;/p&gt;&#xA;" OwnerUserId="3512" LastActivityDate="2013-01-16T14:52:20.977" Title="Customizing Genetic Algorithm on Matlab" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5014" PostTypeId="2" ParentId="5007" CreationDate="2013-01-16T08:41:15.233" Score="2" Body="&lt;p&gt;If you don't have initial guesses ready, I would try &lt;a href=&quot;http://en.wikipedia.org/wiki/Bairstow%27s_method&quot; rel=&quot;nofollow&quot;&gt;Bairstow's method&lt;/a&gt;. And Newton refinement if needed (but probably not if you're happy with 2-3 digits).&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-01-16T08:41:15.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5015" PostTypeId="1" CreationDate="2013-01-16T04:32:06.047" Score="4" ViewCount="8679" Body="&lt;p&gt;Suppose, I have a function and want to optimize it. But if I use &lt;code&gt;optim()&lt;/code&gt; which gives &lt;code&gt;warnings()&lt;/code&gt;. How can I avoid these warnings of &lt;code&gt;NaN&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    myfun&amp;lt;-function(par, x){&#xA;&#xA;    f&amp;lt;- sum(x)*length(x)+sum(log(gamma(par))*x)+1&#xA;    return(-f)&#xA;    }&#xA;    optim(0.1, myfun, x=c(1,5,4,7,8,5,6,5,45,8))&#xA;&#xA;$par&#xA;[1] 4.203895e-46&#xA;&#xA;$value&#xA;    [1] -10762.39&#xA;    $counts&#xA;function gradient &#xA;     502       NA &#xA;$convergence&#xA;    [1] 1&#xA;    $message&#xA;NULL&#xA;&#xA;There were 50 or more warnings (use warnings() to see the first 50)&#xA;&#xA;&#xA;warnings()&#xA;&#xA;2: In log(gamma(par)) : NaNs produced&#xA;3: In log(gamma(par)) : NaNs produced&#xA;4: In log(gamma(par)) : NaNs produced&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="Dihan" LastActivityDate="2013-11-05T15:34:19.407" Title="How to avoid NaN in optim?" Tags="|r|optimization|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5016" PostTypeId="2" ParentId="5015" CreationDate="2013-01-16T04:47:35.073" Score="1" Body="&lt;p&gt;The problem is that your optimal solution is a boundary value (at 0, and negative values of &lt;code&gt;par&lt;/code&gt; aren't allowed). You can verify this by plotting &lt;code&gt;myfun&lt;/code&gt; with the given values of $x$. I don't think general optimizers are well configured to handle such issues.&lt;/p&gt;&#xA;" OwnerUserId="3569" OwnerDisplayName="AdamO" LastActivityDate="2013-01-16T04:52:36.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5017" PostTypeId="2" ParentId="5015" CreationDate="2013-01-16T11:14:16.463" Score="3" Body="&lt;p&gt;As AdamO said, you have an issue with non-positive values. Purely for optimization purposes, exponentiating to make sure things are non-negative and adding some salt to avoid zero generally does the trick. So... something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myfun&amp;lt;-function(par, x){&#xA; par &amp;lt;- exp(par) + 10^-10&#xA; f&amp;lt;- sum(x)*length(x)+sum(log(gamma(par))*x)+1&#xA; return(-f)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then none of the two optimizers will give you any trouble.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;optim( log(0.1), myfun, x=c(1,5,4,7,8,5,6,5,45,8), method=&quot;BFGS&quot;)&#xA;optim( log(0.1), myfun, x=c(1,5,4,7,8,5,6,5,45,8), method=&quot;CG&quot;)&#xA;# I logged because I exponentiate in the function.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically you have a constrained optimization problem and you want to express it as an unconstrained one. Therefore you exponentiate your solution space to make sure it is non-negative one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a word of caution: Exponentiation can occasionally lead to problems if you test for very large values, eg. par=113; would lead to evaluating &quot;gamma(exp(113))&quot; which is actually quite a large number. :) &lt;/p&gt;&#xA;" OwnerUserId="3567" LastActivityDate="2013-01-16T11:14:16.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5018" PostTypeId="1" CreationDate="2013-01-16T08:23:31.497" Score="8" ViewCount="1011" Body="&lt;p&gt;I've come across a stiff equation in solving the Circular Restricted Three Body Problem.&#xA;[An object is moving considering the effect of the gravitational forces caused by two gravitational sources fixed in a 2D Space.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations are these:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x''=-\frac{GM_1 (x-x_1)}{\sqrt{(x-x_1)^2+y^2}^3}--\frac{GM_2 (x-x_2)}{\sqrt{(x-x_2)^2+y^2}^3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y''=-\frac{GM_1 y}{\sqrt{(x-x_1)^2+y^2}^3}--\frac{GM_2 y}{\sqrt{(x-x_2)^2+y^2}^3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither Euler Method or Runge Kutta will work as the property near $(x_1, 0)$ or $(x_1, 0)$ is not good. The derivatives change too fast. The simulation can't be solved out right.&#xA;The object is too easy to hit on the gravitational source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I fix this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="3590" OwnerDisplayName="Fin_de_Siecle" LastEditorUserId="127" LastEditDate="2013-05-24T16:07:14.400" LastActivityDate="2013-05-24T17:27:09.177" Title="How to solve the stiff equation in this Restricted Three Body Problem numerically?" Tags="|ode|simulation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5019" PostTypeId="2" ParentId="5007" CreationDate="2013-01-16T12:12:21.023" Score="6" Body="&lt;p&gt;This may seem like a somewhat exotic suggestion, but if your polynomials are of fixed, low degree, why not use the eigenvalues of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Companion_matrix&quot; rel=&quot;nofollow&quot;&gt;companion/colleague-matrix&lt;/a&gt; as an initial value for the roots?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may be expensive, but if you have to repeat the operation for slightly modified polynomials, you could use one or two steps of &lt;a href=&quot;http://en.wikipedia.org/wiki/Inverse_iteration&quot; rel=&quot;nofollow&quot;&gt;inverse iteration&lt;/a&gt; to update the roots. Although each iteration involves a matrix inverse, recall that the colleague/companion matrix, plus the inverse iteration shift, is only a bi-diagonal matrix with an extra column attached, and the system can thus be solved in $\mathcal O(n)$ operations.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2013-01-17T09:38:37.450" LastActivityDate="2013-01-17T09:38:37.450" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5020" PostTypeId="2" ParentId="5013" CreationDate="2013-01-16T14:52:20.977" Score="1" Body="&lt;p&gt;Consider the 24 bits of your bitstring as a 24-bit number $y$, i.e., a number between zero and 16777215. Then you can get a real number $x$ between zero and one by computing $x=y/2^24=y/16777216$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-16T14:52:20.977" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5022" PostTypeId="2" ParentId="5018" CreationDate="2013-01-16T15:50:43.340" Score="7" Body="&lt;p&gt;Conventional integrators do not preserve the &quot;shape&quot; of phase space, leading to systematic energy gain or loss, thus you should consider &quot;sympelectic&quot; integrators. For close encounters, you should consider the method of &lt;a href=&quot;http://dx.doi.org/10.1046/j.1365-8711.1999.02379.x&quot;&gt;Chambers (1999) &lt;em&gt;A hybrid symplectic integrator that permits close encounters between massive bodies&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-01-16T15:50:43.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5023" PostTypeId="1" AcceptedAnswerId="5024" CreationDate="2013-01-16T18:19:56.763" Score="5" ViewCount="326" Body="&lt;p&gt;I have been trying to simulate a simple problem of taking about 100-1000 Ar molecules in a NVE (fixed vol, energy) system with equal speed but randomized velocity, and evolving them to obtain a Maxwellian speed distribution. I am choosing a Lennard Jones pair potential with no cutoff (despite being computationally expensive), periodic boundary conditions, and a varying integration time step to keep numerical error on energy to below 2%. I observe that my system reaches Maxwellian-like speed distribution (visually looks like a good Gaussian) from the initial rather uniform speed distribution. But very soon after some (about 10 in 100) particles accelerate to very large speeds even if my total energy is being held constant (to within 2% of initial energy) skewing the distribution to one side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to vary the no. of particles, increased temperature to allow re-equilibration of what appears like a growing fluctuation but can't make this effect go away. I am puzzled because I am managing to keep total energy with a 2% bound. Has anyone else has observed such long-time behavior in simple MD simulations, and/or is it well-known behavior? I am using the velocity Verlet scheme. I am using a well-scaled system (i.e., not trying to code actual atomic dimensions) and using a reasonable well distributed initial condition.&lt;/p&gt;&#xA;" OwnerUserId="3570" LastActivityDate="2013-01-18T03:54:39.357" Title="NVE MD simulation of inert gas: Problem maintaining equilibrium" Tags="|molecular-dynamics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5024" PostTypeId="2" ParentId="5023" CreationDate="2013-01-16T21:07:08.543" Score="4" Body="&lt;p&gt;Did you check the centre-of-mass velocity of your particles, i.e. did you set it to zero at the beginning of your simulation? The centre-of-mass velocity should be preserved throughout the simulation, and should be zero to get correct temperatures (see &lt;a href=&quot;http://en.wikipedia.org/wiki/Flying_ice_cube&quot; rel=&quot;nofollow&quot;&gt;Flying Ice Cube&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-01-16T21:07:08.543" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5025" PostTypeId="2" ParentId="5003" CreationDate="2013-01-16T22:01:16.633" Score="3" Body="&lt;p&gt;For your setup the most common strategy is to transfer data from the CAD program to the FEM preprocessor via a neutral file format, e.g. &lt;a href=&quot;http://en.wikipedia.org/wiki/IGES&quot; rel=&quot;nofollow&quot;&gt;IGES&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/ISO_10303&quot; rel=&quot;nofollow&quot;&gt;STEP&lt;/a&gt;, which are both supported by Gmsh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I have no direct experience of DraftSight, so please check if it is capable of exporting (saving) models in IGES or STEP format. (Most CAD programs have native support for IGES or STEP, so I would expect that also DraftSight has this capability.)&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-01-16T22:01:16.633" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5026" PostTypeId="7" CreationDate="2013-01-17T00:52:23.040" Score="0" Body="&lt;p&gt;&lt;strong&gt;Computational Science&lt;/strong&gt; is a question and answer site for questions and answers about computational methods used in technical disciplines. It's built and run &lt;em&gt;by you&lt;/em&gt; as part of the &lt;a href=&quot;https://stackexchange.com&quot;&gt;Stack Exchange&lt;/a&gt; network of Q&amp;amp;A sites. With your help, we're working together to build a library of detailed answers to every question about computational science.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="39739" LastEditDate="2021-07-13T11:20:54.693" LastActivityDate="2021-07-13T11:20:54.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5027" PostTypeId="7" CreationDate="2013-01-17T00:58:25.723" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;Software packages or languages used in computational science&lt;/li&gt;&#xA;&lt;li&gt;Algorithms or methods used to solve problems in applied mathematics&lt;/li&gt;&#xA;&lt;li&gt;Appropriate methods for a given application area&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="61" LastEditDate="2013-01-17T01:44:09.487" LastActivityDate="2013-01-17T01:44:09.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5028" PostTypeId="2" ParentId="5010" CreationDate="2013-01-17T01:55:26.590" Score="1" Body="&lt;h3&gt;A backward stable algorithm is not possible&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The reason is discussed in Example 15.2 (Outer Product) of Trefethen and Bau, in which they state&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As a rule, for problems where the dimension of the solution space $Y$ is greater than that of the problem space $X$, backward stability is rare.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Recall the definition: we say that an algorithm $\tilde f$ for computing $f : X \to Y$ is &lt;em&gt;backward stable&lt;/em&gt; if for each $x \in X$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \tilde f(x) = f(\tilde x) \text{ for some $\tilde x$ with } \frac{\lVert \tilde x - x \rVert}{\lVert x \rvert} \in O(\epsilon) . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;expand&quot; step in your proposed algorithm is a function from $R^2$, yielding $\tilde f \in R^3$. Since in general, $\tilde f$ will not be &lt;em&gt;exactly&lt;/em&gt; orthogonal to $n$, there will not exist a vector $\tilde v_2 \in R^2$ such that its exact &quot;expansion&quot; is $\tilde f$. Of course this step lacking backward stability ruins the chances for the overall algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, no algorithm can for your problem can be backward stable because this problem is rank-deficient with an image that does not embed exactly in floating point. Even with an oracle that told you the exact answer, projecting it into a floating point representation would produce an approximation lying outside the image of the exact function. You can, however, have &lt;em&gt;stability&lt;/em&gt; (which relaxes equality in the definition).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-01-17T01:55:26.590" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5029" PostTypeId="1" CreationDate="2013-01-17T14:54:11.430" Score="3" ViewCount="396" Body="&lt;p&gt;I am looking for an easy to implement stabilization scheme that can be used with equal order ($P_1-P_1$ or $Q_1-Q_1$) finite elements for fluid flow. Is there something like this or should I stick to Taylor Hood elements? It appears that there are a number of methods but the ease of implementation is not quite clear.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="119" LastEditDate="2013-01-17T19:42:10.700" LastActivityDate="2013-11-16T00:16:04.533" Title="Finite element stabilization schemes for incompressible flow" Tags="|finite-element|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5030" PostTypeId="2" ParentId="5029" CreationDate="2013-01-17T17:16:06.730" Score="5" Body="&lt;h3&gt;Warning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;First, be warned that stabilization is generally much more fragile than use of stable elements. If you solve a narrow class of problems, you &lt;em&gt;might&lt;/em&gt; be able to choose reasonable parameters and get on with your work, but if you solve a wide range of problems or need to solve demanding under-resolved scenarios, especially with less regular boundaries, coefficient structure, and constitutive relations, stabilization will generally require constant vigilance to detect and quantify the impact of numerical artifacts, and to re-tune parameters for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reasons, I strongly recommend stable elements (especially those with discontinuous pressure space, thus locally conservative) if you want a robust method that does not need to be frequently revisited. In my opinion, reliability and clean analysis is worth the modest increase in data structure complexity.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Stabilization methods&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I'll outline the two most popular choices in the context of a stabilization term $S_h(\mathbf u_h, p_h, \mathbf f; \mathbf v_h, q_h)$ to be added to the weak form.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Pressure-stabilized Petrov-Galerkin (PSPG)&lt;/strong&gt;&#xA;This stabilization has the form&#xA;$$ S_h^{\text{pspg}}(\mathbf u_h,p_h;q_h) = \sum_{T \in \mathcal{T}_h} \alpha_T\Big(-\nabla\cdot(\eta D \mathbf u_h) + \nabla p_h - \mathbf f, \nabla q_h\Big)_T$$&#xA;where the inner product $(\cdot,\cdot)_T$ is evaluated over element $T$. This method was introduced by Hughes, Franca, and Balestra in 1986 and is a residual-based method. It requires second derivatives on elements (if applicable; they are only zero for lowest order spaces with affine elements). When the solution is not very well resolved, the element integral can be very different from the actual cell residual (because the influence from neighbor cells is discarded). The stabilization produces a non-physical boundary condition for the pressure, leading to local inaccuracies impacting functionals of interest (such as drag). There are more robust and accurate edge-stabilized modifications, but these are more complicated and grow the stencil. Residual stabilization adds a large number of couplings for more complicated problems (like reaction and turbulence models) and significant implementation complexity. The formulation is cumbersome for time-dependent problems because the &quot;affine&quot; term enters in an inner product, thus typically precluding a method of lines approach. PSPG also breaks symmetry (relevant for adjoint-consistency) and increases regularity requirements for an adjoint, making it less attractive for optimization and sensitivity analysis.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Local pressure projection&lt;/strong&gt;&#xA;This conceptually simpler method uses a stabilization of the form&#xA;$$ S_h^{\text{lps}} = (\alpha \tilde \pi p_h, \tilde \pi q_h)$$ where $\tilde \pi$ is a local fluctuation operator and $\alpha$ is a scaling factor. Dohrmann and Bochev (2004) studied what is perhaps the simplest choice, $\tilde \pi = 1 - \pi_{k-1}$, where $\pi_{k-1}$ is cell-wise projection into a polynomial space of degree one less than the continuous space. There are many patch-based and edge-stabilized choices, see Braack and Lube's 2009 review. These methods are simpler to implement and preserve symmetry, but are often less robust.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-01-17T17:16:06.730" CommentCount="3" CommunityOwnedDate="2013-01-17T17:16:06.730" ContentLicense="CC BY-SA 3.0" />
  <row Id="5031" PostTypeId="2" ParentId="5010" CreationDate="2013-01-17T21:23:49.423" Score="3" Body="&lt;p&gt;To carry out a projection operation, the best paper out there that I know of discussing the advantages and disadvantages is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stewart, G. W. &lt;a href=&quot;http://epubs.siam.org/action/showAbstract?page=309&amp;amp;volume=32&amp;amp;issue=1&amp;amp;journalCode=sjmael&amp;amp;&quot; rel=&quot;nofollow&quot;&gt;On the Numerical Analysis of Oblique Projectors&lt;/a&gt;, &lt;em&gt;SIMAX&lt;/em&gt; (2011), Vol. 32, Issue 1, p. 309-348.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For orthogonal projectors, naïvely constructing and applying the projection matrix is fine because the condition number of that operation is the norm of the projection matrix (which, for an orthogonal projector, is 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most numerically stable way of constructing such a projector that I know of is to take $n$ and perform an SVD of it. The left singular vectors corresponding to zero singular values (I suppose, technically, there are no singular values corresponding to those vectors, since there will only be one singular value, and it will be nonzero...) will be the basis for the orthogonal complement of the span of $n$; let the matrix of these singular values be $U$. Then the projector you're looking for is $P = UU^{H}$, and you can calculate the vector you want one of two ways:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Let $w = Pv$, and then $f(v) = w / \|w\|$, unless $w = 0$, in which case $f(v) = 0$.&lt;/li&gt;&#xA;&lt;li&gt;Calculate $x = U^{H}v$, then $w = Ux$, and $f(v) = w / \|w\|$, unless $w = 0$, in which case $f(v) = 0$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Based on Stewart's comments in the paper on oblique projectors, method 1 or method 2 should give you roughly the same amount of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also calculate your projector using QR instead of SVD, but SVD is more numerically stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The below method also works:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Calculate $x = U^{H}v$, then set $w = x / \|x\|$ (unless $x = 0$, then $w = 0$). Finally, $f(v) = Uw$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, the argument is that $\|w\|$ and $\|f(v)\|$ are the same because $U$ is a partial isometry. $f(v)$ is guaranteed to be in the span of the columns of $U$ (to within numerical error), and division by $\|x\|$ &lt;em&gt;could&lt;/em&gt; amplify error in this subspace, as opposed to methods 1 and 2, where division by $\|w\|$ &lt;em&gt;could&lt;/em&gt; amplify error in all of $\mathbb{R}^{n}$, including the normal vector, $n$.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-01-18T06:31:00.570" LastActivityDate="2013-01-18T06:31:00.570" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5033" PostTypeId="2" ParentId="5023" CreationDate="2013-01-18T03:54:39.357" Score="0" Body="&lt;p&gt;I think you should capture these few particles behind a door in a separate compartment. There would doubtlessly be some energy that can be gained from them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure Maxwell would be proud of such a demonic scheme if you succeeded!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-18T03:54:39.357" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5034" PostTypeId="2" ParentId="5015" CreationDate="2013-01-18T04:02:30.073" Score="0" Body="&lt;p&gt;Use lgamma(par) instead of log(gamma(par)).&lt;/p&gt;&#xA;" OwnerUserId="3443" LastActivityDate="2013-01-18T04:02:30.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5035" PostTypeId="2" ParentId="5003" CreationDate="2013-01-18T04:36:04.997" Score="3" Body="&lt;p&gt;I would add my two cents about meshing with Gmsh.&#xA;Even though Gmsh has some abilities to define geometry, Gmsh is not CAD package. So for complex models you need to use any third-party CAD software. For example,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;OpenCASCADE (library)&lt;/li&gt;&#xA;&lt;li&gt;Salome&lt;/li&gt;&#xA;&lt;li&gt;FreeCAD&lt;/li&gt;&#xA;&lt;li&gt;BRL-CAD&lt;/li&gt;&#xA;&lt;li&gt;PythonCAD&lt;/li&gt;&#xA;&lt;li&gt;BlenderCAD&lt;/li&gt;&#xA;&lt;li&gt;Archimedes&lt;/li&gt;&#xA;&lt;li&gt;OpenSCAD&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would give you links for all of these packages but StackExchange prohibits me to do this because I'm quite new here. However you can find them &lt;a href=&quot;http://numolab.blogspot.ru/2012/08/gmsh-tasting-2d.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; or just googling by name.&#xA;These packages are all free and most of them are open-source.&#xA;To bind the model from these packages with Gmsh, you need to export geometry in one of OpenCascade formats (BRep, STEP or IGES, but note that IGES format is not recommended), or any other compatible to Gmsh format (STL and many others). If any of presented packages doesn't export a model in suitable format, you can save it in some CAD format and then use &lt;a href=&quot;http://www.cadexchanger.com/&quot; rel=&quot;nofollow&quot;&gt;CAD Exchanger&lt;/a&gt; for converting.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2013-01-18T04:36:04.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5036" PostTypeId="1" AcceptedAnswerId="5065" CreationDate="2013-01-18T05:58:00.987" Score="5" ViewCount="403" Body="&lt;p&gt;I'm using the g2o C++ optimization library to refine a GPS trajectory using accelerometer data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The program uses a Levenberg-Marquardt optimizer over data points representing the position and orientation (vector part of quaternion) of the accelerometer. The only constraints are the accelerations in the local reference frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error is measured as the difference between the accelerometer reading, and the local second derivative of the path traced by the data points — the change in velocity from (a to b) to (b to c), disregarding change in orientation, plus the upward acceleration counteracting gravity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial path is a polygon drawn between GPS readings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimizer runs for five iterations of gradient descent, and seems to make good adjustments. The polygon corners get rounded off a bit. But then it seems to get stuck, and Levenberg's lambda goes up for five iterations before the program terminates.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;BUT&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;It does a lot more refinement if I further constrain it using data from the gyroscope. However, not at as much as I'd expect, and the gyro data needs have artificially high Fisher information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Working on segments of the whole problem doesn't terminate so quickly. It does more refinement using acceleration alone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be going wrong? The inability to refine the initial estimate by acceleration alone seems to indicate a serious problem.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2013-01-21T03:46:49.113" Title="Levenberg optimizer halts quickly when given more variables, or fewer constraints" Tags="|optimization|iterative-method|stability|nonlinear-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5037" PostTypeId="2" ParentId="5005" CreationDate="2013-01-18T07:20:40.617" Score="2" Body="&lt;p&gt;This is a very complex topic, and indeed, for reasons of export control you won't be able to get codes for a 'pet project'. However, there are publicly available documents on results of such computations, and you are advised to look for them in DoE's OSTI and NASA's NTRS systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand what's going on, would suggest reading up from the basics (like Bell and Glasstone, Nuclear Reactor Theory, Van Nostrand, 1970).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are commercially available packages that would do at least part of the work for you (COMSOL Multiphysics, Ansys, and others), if you go the 3-D way. You'd still need to code neutron diffusion and fuel behavior yourself (supposing you get cross-section and fuel data somewhere), and tie it together with thermohydrodynamics of liquid hydrogen or methane or whatever.&lt;/p&gt;&#xA;" OwnerUserId="2155" LastActivityDate="2013-01-18T07:20:40.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5038" PostTypeId="1" AcceptedAnswerId="5041" CreationDate="2013-01-18T10:46:03.090" Score="1" ViewCount="874" Body="&lt;p&gt;I am trying to understand the stability of the forward Euler method. I read there's a model problem to see the stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y'(t) = \lambda y(t) \qquad t \in (0, \infty)$$&#xA;$$y(0) = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then the book shows this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_0 = 1 \qquad \text{does this come from the initial condition?}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{n+1}=u_n(1+\lambda h) = (1+\lambda h)^{n+1}, \qquad n ≥ 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot understand how $(1+\lambda h)^{n+1}$ appeared&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then the analysis says that the only way the $$\lim_{n \rightarrow \infty} u_n = 0$$ is that $$|1+\lambda h| &amp;lt; 1$$ (why does it need to be less than 1?) Which gives (according to the book)&#xA;$$h &amp;lt; \frac{2}{|\lambda|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the inequallity $|1+\lambda h| &amp;lt; 1$ the thing I get is $\frac{-2}{\lambda} &amp;lt; h$ but still don't know how they get the absolute value of $\lambda$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;how does the book arrive to this conclusion? I don't expect a very complex answer (which is appreciated) but a simple way to understand the intermediate steps not shown.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2013-01-18T11:39:21.387" LastActivityDate="2013-01-18T11:44:47.970" Title="Stability of forward euler method" Tags="|stability|explicit-methods|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5039" PostTypeId="1" CreationDate="2013-01-18T10:53:42.977" Score="2" ViewCount="99" Body="&lt;p&gt;How can one solve the following system of linear equations in MAPLE 13?I know how to solve a linear equation with one variable floating around but not this one.&#xA;$$x-2y+3z=10$$&#xA;$$3x-2y+z=2$$&#xA;$$4x+5y+2z=29$$&#xA;Any help would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks In Advance.&lt;/p&gt;&#xA;" OwnerUserId="3579" LastActivityDate="2013-01-18T10:57:33.163" Title="Not Sure How to Solve A System Of Linear Equations In MAPLE13" Tags="|software|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5040" PostTypeId="2" ParentId="5039" CreationDate="2013-01-18T10:57:33.163" Score="2" Body="&lt;p&gt;You make a set of equations and solve for a set of unknowns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eq1 := x-2*y+3*z = 10; eq2 := 3*x-2*y+z=2; eq3 := 4*x+5*y+2*z = 29; solve({eq1,eq2,eq3},{x,y,z});&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-01-18T10:57:33.163" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5041" PostTypeId="2" ParentId="5038" CreationDate="2013-01-18T11:23:56.097" Score="2" Body="&lt;p&gt;Ok. So $u_0 = 1$, $u_1 = (1+\lambda h)$, $u_2 = (1+\lambda h)^2$, $u_3 = (1+\lambda h)^3$, ..., $u_n = (1+\lambda h)^n$, $u_{n+1} = (1+\lambda h)^{n+1}$&#xA;This will answer one part of your question. I don't understand the other part of your question.&#xA;For assessing stability, let's assume $\lambda &amp;lt; 0$. You can think about the other possibilities yourself later. The true solution to the differential equation is $u_0 e^{-\lambda t} $ When t goes to infinity, the solution goes to zero.&#xA;This must be the case for the discrete equation, too.&#xA;The solution of our discrete equation will go to zero, when $(1 + \lambda h) &amp;gt; -1$. So $h &amp;lt; -2/\lambda = 2/|\lambda|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An other hint: When you multiply something smaller than one with itself, you will get something smaller than one. When it's bigger, it will grow.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastEditorUserId="1738" LastEditDate="2013-01-18T11:44:47.970" LastActivityDate="2013-01-18T11:44:47.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5042" PostTypeId="1" AcceptedAnswerId="5046" CreationDate="2013-01-18T12:45:47.677" Score="5" ViewCount="11267" Body="&lt;p&gt;Can you explain me how does the backward Euler method works?&#xA;I have seen the formula and try to understand the method, but what I can't understand is why and how to use the Newton-Rapson method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have a link to a good tutorial? Something that can help me understand it graphically?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to what &lt;strong&gt;Paul&lt;/strong&gt; said, could you please tell me if what follows is correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Cauchy problem is like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(t) = f\big ( u(t), t \big ) \quad \text{where} \quad f\big ( u(t), t \big ) = u'\big ( u(t), t \big )$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then, according to the example given by Paul, I'll have something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u'(t)=\big [u(t) \big]^3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the backward Euler method reads:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{t+1} = u_t + hf\big ( u_{t+1}, t+1 )~~~~~~~~~~\text{not really sure about this}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for, $t = 0$ and $h = 1/2$ I should have the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_1 = u_0 + \frac{1}{2}(u_1)^3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this equation I have to solve by means of Newton-Raphson before going to the next time $t = 1$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_1 - u_0 - \frac{1}{2}(u_1)^3 = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here I don't know what to do, how to use Newton's method??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I finally got it. From backward Euler's method I have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{t+1} = u_t + hf(u_{t+1}, t+1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this can be seen as (just like everybody told me):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(u_{t+1})= u_{t+1} - u_t - hf(u_{t+1}, t+1) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{t+1} = x$ so I have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(x)= x - u_t - hf(x, t+1) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which now I see a familiar way to use Newton's (or whatever other method) to find the value of $x$&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="123" LastEditDate="2016-04-19T08:23:51.870" LastActivityDate="2016-04-19T08:23:51.870" Title="How to implement Newton's method for solving the algebraic equations in the backward Euler method" Tags="|ode|implicit-methods|newton-method|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5043" PostTypeId="2" ParentId="5036" CreationDate="2013-01-18T14:48:34.207" Score="5" Body="&lt;p&gt;It could be that your optimization problem is very badly conditioned when using only the accelerometer data.   In other words, the accelerometer data might not sufficiently constrain the parameters so that many different paths adequately fit the data.  In terms of the minimization problem this means that you'd have a large &quot;flat spot&quot; at the minimum of different paths with nearly the same objective value.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To test for this, look at the condition number of $J^{T}J$ at the optimal trajectory.  If this matrix is ill-conditioned, then you do indeed have a badly conditioned problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would explain the increase in the LM's $\lambda$.  If $J^{T}J$ is approaching singularity, LM will increase $\lambda$ to overcome that singularity.  This results in shorter and shorter steps, and eventually the LM code just gives up.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-01-18T14:48:34.207" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5044" PostTypeId="2" ParentId="206" CreationDate="2013-01-18T15:31:19.263" Score="17" Body="&lt;p&gt;Unit testing for scientific codes is useful for a variety of reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Three in particular are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Unit tests help other people understand the constraints of your code. Basically, unit tests are a form of documentation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Unit tests check to make sure that a single unit of code is returning correct results, and check to make sure that the behavior of a program doesn’t change when the details are modiﬁed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using unit tests makes it easier to modularise your research codes. This can be particularly important if you start trying to target your code at a new platform, for instance you are interested in parallelising it, or running it on a GPGPU machine.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Most of all, unit tests give you confidence that the research results you are producing using your codes are valid and verifiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I note that you mention regression testing in your question. In many cases, regression testing is achieved through the automated, regular execution of unit tests and/or integration tests (which test that pieces of code work correctly when combined; in scientiﬁc computing, this is often done by comparing output to experimental data or the results of earlier programs that are trusted). It sounds like you are already using integration tests or unit test at the level of large complex components successfully.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would say is that as research codes get increasingly complex, and rely on other people's code and libraries, it is important to understand where the error occurs when it does. Unit testing allows the error to pinpointed much more easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may find the description, evidence and references in Section 7 &quot;Plan for mistakes&quot; of the paper I co-authored on &lt;a href=&quot;http://arxiv.org/abs/1210.0530&quot; rel=&quot;nofollow noreferrer&quot;&gt;Best Practices for Scientiﬁc Computing&lt;/a&gt; useful - it also introduces the complementary concept of defensive programming.&lt;/p&gt;&#xA;" OwnerUserId="3582" LastEditorUserId="9667" LastEditDate="2018-10-23T17:14:40.560" LastActivityDate="2018-10-23T17:14:40.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5045" PostTypeId="2" ParentId="5042" CreationDate="2013-01-18T16:23:03.573" Score="4" Body="&lt;p&gt;Backwards Euler, as you state you know, is time advances using the equation: $U_{t+\Delta t} = U_{t}+ U'(t,U_{t+\Delta t}) \Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can not explicitly evaluate $U'_{t+\Delta t}$ since you don't know $U_{t+\Delta t}$.  To make it more clear the equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X = U(t) + U'(t,X)\Delta t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore you must solve the equation for X (which is U at the next time step) using some methodology (this is where you use Newton-Raphson in your case).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as graphical interpretation, think about your standard Euler.  The idea there is that you are taking the current value of $U$ and its derivative at the current time, and assuming that it is nearly linear for some small change in time ($\Delta t$).  So you are drawing a straight line from $U(t)$ at a slope of $U'(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For backwards Euler, all you are doing is using the slope at the end of your line approximation rather than the start of it.  As to why you would want to do this, it is a more complicated answer involving the stability of your solution.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-01-18T19:56:19.843" LastActivityDate="2013-01-18T19:56:19.843" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5046" PostTypeId="2" ParentId="5042" CreationDate="2013-01-18T17:10:02.650" Score="5" Body="&lt;p&gt;You would use backward euler method to solve a differential equation of the form $u_t =f(u,t)$  where $f$  is not necessarily a linear function in u. When f is non-linear, then the backward euler method results in a set of non-linear equations that need to be solved for each time step. Ergo, Newton-raphson can be used to solve it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, take &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{du}{dt}=u^3(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backward euler results in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u_{t+1}-u_{t}}{\Delta t}=u_{t+1}^3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you can't solve for $u_{t+1}$ directly, you have to estimate it by a numerical scheme.  This is where newton-raphson comes into play.  Rearrange this equation so that the RHS=0, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u_{t+1}-u_{t}}{\Delta t}-u_{t+1}^3=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can apply newton's method to find the root of this equation $u_{t+1}$, which in turn becomes the solution to the differential equation at the next timestep.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-01-18T17:10:02.650" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5047" PostTypeId="1" CreationDate="2013-01-18T23:25:43.100" Score="8" ViewCount="1124" Body="&lt;p&gt;Are there any existing programs that are able to compute the (approximate) distribution of eigenvalues for very large (symmetric) sparse matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I do not need the eigenvalues themselves, only their distribution (finding &lt;em&gt;all&lt;/em&gt; the eigenvalues is a more difficult problem), and I am primarily looking for existing software, not only the description of algorithms that I need to implement myself. My matrices are adjacency matrices of undirected graphs (or closely related matrices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;http://www-math.mit.edu/~edelman/homepage/papers/sturm.pdf&quot; rel=&quot;noreferrer&quot;&gt;some papers&lt;/a&gt; discussing how this can be done in practice, so I was hoping that a working implementation exists somewhere.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2013-01-19T02:22:44.027" LastActivityDate="2013-01-22T09:28:37.370" Title="Finding the distribution (histogram) of eigenvalues for large sparse matrices" Tags="|linear-algebra|eigensystem|graph-theory|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5048" PostTypeId="2" ParentId="4883" CreationDate="2013-01-19T02:00:50.167" Score="2" Body="&lt;p&gt;Without getting (again) into why and how determinants are evil, let's assume that your operator is either not easily factorizable or simply not available as a matrix at all and that you &lt;em&gt;really&lt;/em&gt; need to estimate its determinant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to obtain an estimate of the determinant is to use an implementation of CG that is directly based on the Lanczos process. See for instance Algorithm 6.17 (D-Lanczos) in Saad's book &quot;&lt;a href=&quot;http://goo.gl/Fx9nX&quot; rel=&quot;nofollow&quot;&gt;Iterative Methods for Sparse Linear Systems&lt;/a&gt;&quot; (SIAM), page 189. There, the operator $A$ is semi-explicitly factored and you can use the determinant of the factors as an approximation of the determinant of $A$. Let me emphasize that I've never tried to estimate a determinant and I've no idea whether or not what I'm suggesting is a good idea---problably not given all the trouble ghost eigenvalues give us. However, that's what comes to mind when reading your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can probably reverse-engineer how this estimate of the determinant comes about in the standard implementation of CG by following closely Section 6.7.3 of the book.&lt;/p&gt;&#xA;" OwnerUserId="324" LastActivityDate="2013-01-19T02:00:50.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5049" PostTypeId="2" ParentId="5047" CreationDate="2013-01-19T02:52:31.963" Score="1" Body="&lt;p&gt;I would imagine that something of this kind can be inferred from the pseudospectrum function. See the work by Marc Embree, for example:&#xA;  &lt;a href=&quot;http://www.caam.rice.edu/~embree/&quot; rel=&quot;nofollow&quot;&gt;http://www.caam.rice.edu/~embree/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-19T02:52:31.963" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5050" PostTypeId="1" AcceptedAnswerId="5051" CreationDate="2013-01-19T13:40:18.437" Score="12" ViewCount="9055" Body="&lt;p&gt;I have a block matrix (either 2x2 blocks or 3x3 blocks) which is the covariance matrix for a joint space of two or three multivariate normal variables. ie&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C = [Cxx Cxy; Cxy' Cyy];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to compute the cholesky factorisation of this matrix (C), as well as the diagonal blocks (Cxx, Cyy, the covariance matrices of the individual multivariate normals), and I would like to do this as fast as possible. At the moment I am doing three chol decompositions. I was wondering if it would be possible to obtain chol(Cxx) and chol(Cyy) from chol(C) (i.e. from extracting subblocks of the full decomposition) or if there would be any other trick to help do this faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I have looked at QR factorisation instead of explicitly calculating the covariance but for my case it is many times slower)&lt;/p&gt;&#xA;" OwnerUserId="1012" LastActivityDate="2023-06-19T01:19:49.857" Title="cholesky factorization of block matrices" Tags="|linear-algebra|block-decomposition|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5051" PostTypeId="2" ParentId="5050" CreationDate="2013-01-19T16:53:58.947" Score="13" Body="&lt;p&gt;There is some redundant work, but not as much as you were hoping. Recall the &lt;span class=&quot;math-container&quot;&gt;$LDL^T$&lt;/span&gt; factorization&#xA;&lt;span class=&quot;math-container&quot;&gt;$$C = \begin{pmatrix} A &amp;amp; B^T \\ B &amp;amp; D \end{pmatrix}&#xA; =\begin{pmatrix} 1 &amp;amp; 0 \\ B A^{-1} &amp;amp; 1 \end{pmatrix}&#xA;  \begin{pmatrix} A &amp;amp; 0 \\ 0 &amp;amp; S \end{pmatrix}&#xA;  \begin{pmatrix} 1 &amp;amp; A^{-1} B^T \\ 0 &amp;amp; 1 \end{pmatrix}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$S = D - B A^{-1} B^T$&lt;/span&gt; is the Schur complement. Clearly you need &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S^{-1}$&lt;/span&gt; to solve using this factorization, with &lt;span class=&quot;math-container&quot;&gt;$D^{-1}$&lt;/span&gt; providing no value.&lt;/p&gt;&#xA;&lt;p&gt;Substituting &lt;span class=&quot;math-container&quot;&gt;$A = L_A L_A^T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S = L_S L_S^T$&lt;/span&gt; into the factorization above, we obtain the Cholesky factorization [1]:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{aligned}&#xA;C &amp;amp; =\begin{pmatrix}&#xA;A &amp;amp; B^{T}\\&#xA;B &amp;amp; D&#xA;\end{pmatrix}\\&#xA; &amp;amp; =\begin{pmatrix}&#xA;\mathbb{1} &amp;amp; 0\\&#xA;B\left( L_{A} L_{A}^{T}\right)^{-1} &amp;amp; \mathbb{1}&#xA;\end{pmatrix}\begin{pmatrix}&#xA;L_{A} L_{A}^{T} &amp;amp; 0\\&#xA;0 &amp;amp; L_{S} L_{S}^{T}&#xA;\end{pmatrix}\begin{pmatrix}&#xA;\mathbb{1} &amp;amp; \left( L_{A} L_{A}^{T}\right)^{-1} B^{T}\\&#xA;0 &amp;amp; \mathbb{1}&#xA;\end{pmatrix}\\&#xA; &amp;amp; =\left(\begin{pmatrix}&#xA;\mathbb{1} &amp;amp; 0\\&#xA;B\left( L_{A} L_{A}^{T}\right)^{-1} &amp;amp; \mathbb{1}&#xA;\end{pmatrix}\begin{pmatrix}&#xA;L_{A} &amp;amp; 0\\&#xA;0 &amp;amp; L_{S}&#xA;\end{pmatrix}\right)\left(\begin{pmatrix}&#xA;L_{A}^{T} &amp;amp; 0\\&#xA;0 &amp;amp; L_{S}^{T}&#xA;\end{pmatrix}\begin{pmatrix}&#xA;\mathbb{1} &amp;amp; \left( L_{A} L_{A}^{T}\right)^{-1} B^{T}\\&#xA;0 &amp;amp; \mathbb{1}&#xA;\end{pmatrix}\right)\\&#xA; &amp;amp; =\begin{pmatrix}&#xA;L_{A} &amp;amp; 0\\&#xA;BL_{A}^{-T} &amp;amp; L_{S}&#xA;\end{pmatrix}\begin{pmatrix}&#xA;L_{A}^{T} &amp;amp; L_{A}^{-1} B^{T}\\&#xA;0 &amp;amp; L_{S}^{T}&#xA;\end{pmatrix} =L_{C} L_{C}^{T} .&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&#xA;Thus you can simply extract the factors of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; from the factors of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, but will have to factor &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; separately. If the block sizes are equal, &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; takes eight times more work to factor, so this is little savings.&lt;/p&gt;&#xA;&lt;p&gt;If you have a sequence of problems and can partition your system so that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; does not change between subsequent solves, its factorization could be reused, but &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; will change and have to be refactored. Depending on the decomposition and conditioning, iterative methods can be useful to reuse a partial factorization.&lt;/p&gt;&#xA;&lt;p&gt;[1] Note that this &amp;quot;block Cholesky&amp;quot; factorization is also valid with &lt;span class=&quot;math-container&quot;&gt;$L_A$&lt;/span&gt; replaced by a &lt;span class=&quot;math-container&quot;&gt;$A^{1/2}$&lt;/span&gt; or another non-triangular factorization of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="45334" LastEditDate="2023-06-19T01:19:49.857" LastActivityDate="2023-06-19T01:19:49.857" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5052" PostTypeId="1" AcceptedAnswerId="5054" CreationDate="2013-01-19T17:15:43.680" Score="3" ViewCount="162" Body="&lt;p&gt;I am in search of an introduction into numerical linear algebra - or, at least, pure linear algebra - that treats the case when the input data are random variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical application would be to derive an estimate for the expected numerical error when the input variables are subject to random noise. Note that this neither implies that algorithms has access to a perfect coin, nor that the errors in inexact arithmetic obey any probability distribution.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2013-01-19T21:08:30.903" Title="Introduction for (numerical) linear algebra of random variables" Tags="|linear-algebra|stochastic|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5053" PostTypeId="2" ParentId="2987" CreationDate="2013-01-19T21:07:54.407" Score="5" Body="&lt;p&gt;Even though I like Python as a language, distributing Python packages is a mess. I always find people not familiar with Python struggling with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next to the user-local install as outlined by Aron (using --user, or --prefix), another option is EasyBuild (&lt;a href=&quot;http://hpcugent.github.com/easybuild&quot; rel=&quot;noreferrer&quot;&gt;http://hpcugent.github.com/easybuild/&lt;/a&gt;).&#xA;Not only for Python packages, but for any (scientific) software package. Once EasyBuild has support for it, building and installing a software package is basically a single command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a list of software packages currently supported, see &lt;a href=&quot;https://github.com/hpcugent/easybuild/wiki/List-of-supported-software-packages&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/hpcugent/easybuild/wiki/List-of-supported-software-packages&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I am a developer of EasyBuild.&lt;/p&gt;&#xA;" OwnerUserId="3594" LastEditorUserId="276" LastEditDate="2013-01-20T07:37:05.850" LastActivityDate="2013-01-20T07:37:05.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5054" PostTypeId="2" ParentId="5052" CreationDate="2013-01-19T21:08:30.903" Score="1" Body="&lt;p&gt;It isn't clear here whether you're really interested in just the effects of floating point round off error within a computation or the effects of uncertain/noisy inputs to a computation. In most practical situations, assuming that you've been reasonably careful in selecting an appropriate numerical method, and assuming that the problem you're trying to solve isn't inherently ill-conditioned/ill-posed, the effects of uncertain or noisy inputs to the model are much larger than the effects of floating point round off.  There's a burgeoning field of &quot;uncertainty quantification&quot; which attempts to deal with this question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some simple cases are quite easy.  For example, if $x$ has a multivariate normal distribution, a matrix $A$ is known exactly, and $y=Ax$, then $y$ also has a multivariate normal distribution, with $E[y]=AE[x]$, and $\mbox{cov}(y)=A\mbox{cov}(x)A^{T}$.  If $y=f(X)$, where $f$ is nonlinear, then $y$ won't have a multivariate normal distrubiton, but as long as $f$ isn't strongly nonlinear and the uncertainty in $x$ is small, it may be possible to use a linear approximation to $f$ to get an approximate multivariate normal distribution for $y$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very general technique is Monte Carlo simulation.  If $x$ is an uncertain input with known distribution, generate lots of random values of the $x$ vector, compute $y=f(x)$ for each of these random vectors, and then look at the empirical distribution of $y$.  There's a whole literature on methods such as &quot;importance sampling&quot; for making this process more efficient.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently there has been a lot of interest in &quot;polynomial chaos expansions&quot;, a non Monte Carlo approach to analyzing the uncertainty in differential equation models of dynamical systems.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-01-19T21:08:30.903" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5055" PostTypeId="2" ParentId="2469" CreationDate="2013-01-19T21:20:02.187" Score="1" Body="&lt;p&gt;EasyBuild (&lt;a href=&quot;http://hpcugent.github.com/easybuild&quot; rel=&quot;nofollow&quot;&gt;http://hpcugent.github.com/easybuild&lt;/a&gt;) allows you to build and install your own set of GCC compilers easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you give it a simple easyconfig (.eb) file that specifies what you want to build (GCC + version + set of languages you want compilers for + ...), and then building and installing is a single command.&#xA;I'd say it's a bit like Homebrew, but it gives you more freedom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, EasyBuild supports lots of other (scientific) software packages that can be built and installed with a single command, so it's worthwhile to look into.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I'm part of the EasyBuild team. Building GCC (with gfortran included) should work on OS X, but other builds may be troublesome. We're working on it, do let us know if you run into trouble.&lt;/p&gt;&#xA;" OwnerUserId="3594" LastActivityDate="2013-01-19T21:20:02.187" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5059" PostTypeId="2" ParentId="7" CreationDate="2013-01-20T11:15:24.260" Score="4" Body="&lt;p&gt;The answer depends on what you have as the HPC backend. The question refers to existing resources available to NREL users. There are Linux clusters and a Windows cluster based on the Windows Compute Cluster Server.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;To use the linux cluster the author needs to make sure the simulation routine runs on Mono (the implementation of .NET platform for linux) and make sure Mono is installed on the nodes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The next question is whether the users have accounts on the compute clusters and can get the data to the nodes. The simplest case is with a single filesystem accessible from local PCs and from the nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if such a filesystem is available to NREL users, therefore some copying will be involved. This already makes things complicated.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Next the application should be able to compile a job description script, submit it to the respective cluster and query the queueing system for the state of the simulations until the job gets out of the queue.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally the application should be able to retrieve the results.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All This is doable though not trivial and would be highly specific to the setup of this particular lab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Commercial applications from Ansys or MSC let the user save a simulation job description that can be later run in batch mode on a remote cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more general solutions would be to use the Grid infrastructure if available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next one could go for service oriented architecture where a Web-service is set up somewhere to wait for simulation requests and dispatch them onto the cluster thus decoupling the application from the knowledge of the specifics of the available resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is widely used for instance in computational biology, where tools are used to compose simulation workflows consisting entirely of web-service requests for processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to note that offloading computations to remote computing resources is commonly used in applications like Google voice recognition for Android.&lt;/p&gt;&#xA;" OwnerUserId="1361" LastActivityDate="2013-01-20T11:15:24.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5060" PostTypeId="2" ParentId="5047" CreationDate="2013-01-20T16:10:03.573" Score="3" Body="&lt;p&gt;In structural mechanics the number of eigenvalues of a matrix $K$ in a given range $(\alpha,\beta)$ is computed via the &quot;Sturm sequence check&quot;, i. e. computing the $LDL^T$ factorizations of $K-\alpha I$ and $K-\beta I$ and counting the difference in the number of negative pivots. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have reasonably large bins, can be applied to your problem, and should be pretty straightforward to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A search on Lanczos shifted block algorithms should give more info, since this technique is often used in that context to check for missed eigenvalue/eigenvector pairs.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This count is exact, and expensive for large $K$, so your request of an &quot;estimate&quot; or approximate count is still open. Please post any finding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit/update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors of &lt;a href=&quot;http://dx.doi.org/10.1002/nme.2572&quot; rel=&quot;nofollow&quot;&gt;&quot;A Padé-based factorization-free algorithm for identifying the eigenvalues missed by a generalized symmetric eigensolver&quot;&lt;/a&gt; claim&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To the best of the authors’ knowledge, no post-processing technique&#xA;  that does not require the factorization of a matrix related to $K$&#xA;  and/or $M$ is currently available for checking whether an eigensolver&#xA;  applied to the solution of problem (1) has missed some eigenvalues in&#xA;  an arbitrary range of interest $[ \sigma_L , \sigma_R ]$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This means that for obtaining an &lt;em&gt;exact&lt;/em&gt; count of the eigenvalues in a given bin you have to use the classical Sturm sequence method (see also &lt;a href=&quot;http://mathworld.wolfram.com/SylvestersInertiaLaw.html&quot; rel=&quot;nofollow&quot;&gt;Sylvester inertia law&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;General advice for implementing this approach in your case cannot be given, without an analysis of the properties of your adjacency matrices (dimensions, number on non zero entries, fill-ins after reordering, condition number of principal minors...). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless I would suggest starting with a simple no-brainer approach, and see if you experience breakdowns of the implementation (assuming that this computation is &lt;em&gt;not&lt;/em&gt; mission critical). I suggest to use the wonderful &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/SuiteSparse&quot; rel=&quot;nofollow&quot;&gt;SuiteSparse&lt;/a&gt; by Tim Davis.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;reorder your matrix to reduce fill-in, (e.g. calling SYMAMD from &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/colamd/&quot; rel=&quot;nofollow&quot;&gt;COLAMD&lt;/a&gt; on $A - \alpha_0 I$).&lt;/li&gt;&#xA;&lt;li&gt;compute $L_i D_i L_i^T = A - \alpha_i I$, where $\alpha_i$, $i=0\dots m$ are the boundaries of your bins. (Try first a simple implementation like &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/ldl/&quot; rel=&quot;nofollow&quot;&gt;LDL&lt;/a&gt; without pivoting, and go for a pivoting approach only if you experience numerical difficulties.) (Note that without pivoting the symbolic factorization step can be recycled for all $i$.)&lt;/li&gt;&#xA;&lt;li&gt;the count of negative diagonal terms in $D$ gives you the number of eigenvalues $\lambda &amp;lt; \alpha_i$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This approach is effective only if $m$ is small or the factorization time is negligible with respect to eigendecomposition time: you have to perform some experiments to find out. Good luck.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2013-01-22T09:28:37.370" LastActivityDate="2013-01-22T09:28:37.370" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5061" PostTypeId="2" ParentId="4982" CreationDate="2013-01-20T18:26:47.420" Score="6" Body="&lt;p&gt;The 'minFunc' MATLAB package (by Mark Schmidt) is one of the best available:&#xA;&lt;a href=&quot;http://www.di.ens.fr/~mschmidt/Software/minFunc.html&quot;&gt;http://www.di.ens.fr/~mschmidt/Software/minFunc.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This package is fairly mature and has been widely used in the ML community. It contains a stable L-BFGS implementation as well as related Newton and quasi-Newton optimization methods.&lt;/p&gt;&#xA;" OwnerUserId="3599" LastActivityDate="2013-01-20T18:26:47.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5062" PostTypeId="1" CreationDate="2013-01-20T19:04:17.727" Score="2" ViewCount="215" Body="&lt;p&gt;The problem is as follows:&#xA;We are given a graph with each edge length 1 and two pairs of vertices (a,b) and (c,d).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to find shortest paths between from a to b and from c to d, with assumption that there is a way of moving through both paths (step by step) such that in any time our agents (the one moving from a to b and the other moving from c to d) will be away from each other by at least k?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make it more clear:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Agents move separately, so in each step only one agent moves and agent can move few times in row (the other 'waits').&lt;/li&gt;&#xA;&lt;li&gt;We are looking for such paths that the summed length is minimal.&lt;/li&gt;&#xA;&lt;li&gt;Paths are 'valid' if there exists a sequence of agent moves in which at any time agents are at least k distant.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We can assume that the input is nice, so there always exists the pair of paths that meets conditions.&lt;/p&gt;&#xA;" OwnerUserId="3600" LastEditorUserId="3600" LastEditDate="2013-01-22T00:44:22.377" LastActivityDate="2013-03-06T15:33:55.983" Title="Finding two shortest path that are 'distant' in the graph" Tags="|algorithms|graph-theory|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5063" PostTypeId="2" ParentId="1432" CreationDate="2013-01-20T21:52:58.803" Score="1" Body="&lt;p&gt;I wrote an electromagnetic simulation code &lt;a href=&quot;http://www.stanford.edu/group/fan/S4/&quot; rel=&quot;nofollow&quot;&gt;S4&lt;/a&gt; which is limited entirely by the ability to solve a dense nonsymmetric eigenvalue problem (ZGEEV). The code is open source and written in C++.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-01-20T21:52:58.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5064" PostTypeId="2" ParentId="5062" CreationDate="2013-01-20T23:47:25.073" Score="0" Body="&lt;p&gt;I do not know any exact algorithm but I see at least two approaches to approximate an optimal solution:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the shortest path $a-&amp;gt;b$ and $c-&amp;gt;d$ using any of the existing algorithms (e.g., Dijkstra's algorithm). Then simulate the two paths and, if at one step the two agents are too close do one of the following: (i) Detour by one: If agent A is, for example, at vertex $v$ and was at $v'$ before, then replace the step $v'-&amp;gt;v$ by $v'-&amp;gt;v''-&amp;gt;v'$ if you can where $v''$ has maximal distance from the position of agent B at that time but has distance one from both $v$ and $v'$. (ii) If no such $v''$ exists that is neighbor to both $v,v'$, then simply stay at position $v'$ for one more time step.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find a path through the graph that includes $a,c,b,d$ in this order. Then add vertices to this cycle so that the distance between $a$ and $c$ along your cycle is at least the desired separation $k$. The two agents traverse the graph  along this common path.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My best guess is that if the vertices under consideration are &quot;close&quot; to each other (relative to $k$) then the second algorithm will be better. Otherwise, the first algorithm will produce better paths. Both methods find their result in the same complexity as the shortest path finding algorithm.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-20T23:47:25.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5065" PostTypeId="2" ParentId="5036" CreationDate="2013-01-21T03:46:49.113" Score="1" Body="&lt;p&gt;The last vertex (data point) in the initial estimate was not getting initialized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure why this choked the solver. Something to do with the error from one outlier point propagating to the other data points, and pulling everything out of the minimum. It still doesn't make sense that everything would be pulled out after so few iterations, though, and the effect isn't obvious on inspection of the premature solution.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2013-01-21T03:46:49.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5067" PostTypeId="1" CreationDate="2013-01-21T12:28:00.800" Score="6" ViewCount="4056" Body="&lt;p&gt;I would like to write a simple program (in C) using Lanczos algorithm. I came across a Matlab example which helped me to understand a bit further the algorithm, however from this piece of code I can't find the way of getting the eigenvalues and eigenvectors. I can follow the algorithm but I think I must be missing something. Can someone guide me to get the eigenvalues from this example so I can understand the method and then code it in C?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Create a random symmetric matrix &#xA;D=6&#xA;for i=1:D,&#xA;    for j=1:i,&#xA;        A(i,j)=rand; &#xA;        A(j,i)=A(i,j);&#xA;    end &#xA;end&#xA;&#xA;% Iteration with j=0 &#xA;r0 = rand(D,1); &#xA;b0 = sqrt(r0'*r0); &#xA;q1 = r0/b0; &#xA;a1 = q1'*A*q1&#xA;&#xA;%Iteration with j=1&#xA;r1 = A*q1 - a1*q1&#xA;b1 = sqrt(r1'*r1)&#xA;q2 = r1/b1;&#xA;a2 = q2'*A*q2&#xA;&#xA;%Iteration with j=2&#xA;r2 = A*q2 - a2*q2 - b1*q1;&#xA;b2 = sqrt(r2'*r2)&#xA;q3 = r2/b2&#xA;a3 = q3'*A*q3&#xA;&#xA;% Create Matrix Q&#xA;Q = [q1 q2 q3];&#xA;&#xA;%Check orthogonality&#xA;EYE = Q'*Q&#xA;T = Q'*A*Q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3607" LastEditorUserId="3607" LastEditDate="2016-08-15T10:49:56.200" LastActivityDate="2016-08-15T10:49:56.200" Title="Simple Lanczos algorithm code to obtain eigenvalues and eigenvectors of a symmetric matrix" Tags="|linear-algebra|eigensystem|eigenvalues|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5068" PostTypeId="1" AcceptedAnswerId="5077" CreationDate="2013-01-21T13:49:16.610" Score="2" ViewCount="191" Body="&lt;p&gt;I was looking at a book of FEM on problems of Diffusion-Transport.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-div(\mu \nabla u) + b \cdot \nabla u + \gamma u = f \qquad in~\Omega$$&#xA;$$u = 0 \qquad in~\partial\Omega\text{  (in the boundaries)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says that if $\displaystyle \frac{|b|}{\mu} \gg 1$ then the problem is a problem dominated by transport.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking some stuff from the previous chapter, more precisely the Galerking analysis of stability and convergence, and the approximation of the error, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\displaystyle M \cong \mu + |b| \qquad \textrm{continuity constant}$$&#xA;$$\alpha = \mu \qquad \textrm{coercivity constant}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then it has&#xA;$$\displaystyle \frac{M}{\alpha} \cong1+\frac{|b|}{\mu} \gg 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence of that, it concludes that, the estimation of the error is&#xA;$$\displaystyle ||u -u_h|| ≤C\frac{M}{\alpha}h^r|u|_{H^{r+1}(\Omega)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which tell us that the Galerkin's method could give unsatisfactory results if the space step $h$ isn't small enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I Can't understand how can it conclude such thing. Any idea? Please, do not give complex explanations, I'm just trying to understand this FEM stuff, I'm a computer science student more than a math one.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3691" LastEditDate="2013-03-07T03:23:34.213" LastActivityDate="2013-03-07T03:23:34.213" Title="Diffusion-Transport problem FEM" Tags="|finite-element|error-estimation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5069" PostTypeId="2" ParentId="4995" CreationDate="2013-01-21T15:03:12.970" Score="0" Body="&lt;p&gt;Here's my implementation of the algorithm described in the above link.&#xA;I'm giving my implementation for the example described in Fig. 4 in the aforementioned reference but should be adaptable quite easily to whatever you need, in case anyone asks themselves the same question as I did.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the code provided below (hopefully) outputs all cliques (of all sizes k &gt;= 2) in graph G.&#xA;Output the nodes of your graph for k = 1.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import networkx as nx&#xA;from matplotlib import pylab as pl&#xA;&#xA;def greater_neighbors(G, a_node):&#xA;    nodes_sorted = sorted(G.nodes())&#xA;    a_node_index = nodes_sorted.index(a_node)&#xA;&#xA;    neighbors_of_a_node = []&#xA;&#xA;    for another_node_index, another_node in enumerate(nodes_sorted):&#xA;        if another_node_index &amp;gt; a_node_index and another_node in G.neighbors(a_node):&#xA;            neighbors_of_a_node.append(another_node)&#xA;&#xA;    return tuple(neighbors_of_a_node)&#xA;&#xA;G = nx.Graph()&#xA;edges_fig_4 = [('a','b'),('a','c'),('a','d'),('a','e'),&#xA;               ('b','c'),('b','d'),('b','e'),&#xA;               ('c','d'),('c','e'),&#xA;               ('d','e'),&#xA;               ('f','b'),('f','c'),('f','g'),&#xA;               ('g','f'),('g','c'),('g','d'),('g','e')]&#xA;pos = {&#xA;      'b': (0,4), 'f': (0,0),&#xA;      'c': (2,2),&#xA;      'a': (4,6),&#xA;      'd': (6,2),&#xA;      'e': (8,4), 'g': (8,0)&#xA;      } &#xA;&#xA;G.add_edges_from(edges_fig_4)&#xA;&#xA;#pl.figure()&#xA;#nx.draw(G, pos=pos)&#xA;#pl.show()&#xA;&#xA;# sorted list of nodes in graph&#xA;nodes_sorted = sorted(G.nodes())&#xA;&#xA;# starting point: build all 2-clique sublists&#xA;clique_sublists = []&#xA;for a_node_index, a_node in enumerate(nodes_sorted):&#xA;clique_sublist = {}&#xA;# sublist base, sb&#xA;clique_sublist['sb'] = tuple(a_node)&#xA;# common neighbors, cn&#xA;clique_sublist['cn'] = greater_neighbors(G, a_node)&#xA;clique_sublists.append(clique_sublist)&#xA;&#xA;&#xA;while clique_sublists:&#xA;    a_sublist = clique_sublists.pop(0)&#xA;    for node_added in a_sublist['cn']:&#xA;        neighbors_of_node_added = greater_neighbors(G, node_added)&#xA;&#xA;        current_sublist_base = a_sublist['sb']+tuple(node_added)&#xA;        current_sublist_cn = tuple(sorted(set(neighbors_of_node_added).intersection(a_sublist['cn'])))&#xA;&#xA;        print 'clique: '+str(current_sublist_base)&#xA;&#xA;        for node in current_sublist_cn:&#xA;            new_sublist_base = current_sublist_base+tuple(node)&#xA;            new_sublist_cn = tuple(sorted(set(current_sublist_cn).intersection(greater_neighbors(G, node))))&#xA;&#xA;            if len(new_sublist_cn) == 0:&#xA;                print 'clique: '+str(new_sublist_base)&#xA;            elif len(new_sublist_cn) == 1:&#xA;                print 'clique: '+str(new_sublist_base)&#xA;                print 'clique: '+str(new_sublist_base+new_sublist_cn)&#xA;            else:&#xA;                print 'candidate sublist: '+str([new_sublist_base, new_sublist_cn])&#xA;                clique_sublists.append({'sb': new_sublist_base, 'cn': new_sublist_cn})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1530" LastEditorUserId="1530" LastEditDate="2013-01-21T15:22:27.660" LastActivityDate="2013-01-21T15:22:27.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5070" PostTypeId="2" ParentId="5067" CreationDate="2013-01-21T15:05:39.617" Score="8" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Lanczos_algorithm&quot;&gt;Lanczos algorithm&lt;/a&gt; can be used to put the matrix into tridiagonal form, but it doesn't actually find the eigenvalues and eigenvectors of that tridiagonal matrix.  Once you have the matrix in tridiagonal form, the QR algorithm is typically used to find the eigenvalues of the tridiagonal matrix.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="1744" LastEditDate="2013-01-21T15:16:26.993" LastActivityDate="2013-01-21T15:16:26.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5071" PostTypeId="1" AcceptedAnswerId="5078" CreationDate="2013-01-21T16:27:08.637" Score="2" ViewCount="140" Body="&lt;p&gt;I have seen a number of papers that propose a finite-difference method and then show the numerical results for it. Without providing a rigorous analysis(can be some summary or note or whatever, just no proofs involved) one can say method is of &quot;second order&quot;. So to establish convergence the author would pick a point on the grid and measure the ratio of the differences between solutions, that is $$R:=\frac{u_h(h/2)-u_h(h)}{u_h(h/4)-u_h(h/2)}$$&#xA;then, he would say that we observe $2$ or $4$, so it confirms the expectations. It is in essence based on a pointwide convergence, thus if I measure that ratio at every single point on the grid(the initial grid with size $h$, the coarsest one) and observe a certain rate, then it would converge &quot;on average&quot; in the discrete $L^2$-norm, so as in the maximum $L^{\infty}$ error norm with the same rate as well. So what are the disadvantages of checking the convergence this way? I don't have an analytical solution to compare to but I want to verify somehow I have $L^2$ convergence that I have theoretically established. It might be &quot;overkill&quot; to do it this way as I would in fact be checking pointwise convergence, but it would still imply the needed type if convergence if I happen to observe the desired rate. Do I have any other options to establish that or the way I described is valid attempt? What if I don't observe it in some cases, how I can in fact check $L^2$ or $L^{\infty}$ convergence in practice? I have tested the method by using method of manufactured solution but there is an equation that I would like to check the convergence as well and I am not able to construct a solution(it is a weak solution so some of the derivatives don't exist and initial data is only continuous), therefore I need some &quot;brute force&quot; approach.&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2013-01-22T02:44:39.033" LastActivityDate="2013-01-22T09:16:48.557" Title="Pointwise convergence" Tags="|pde|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5072" PostTypeId="1" CreationDate="2013-01-21T19:02:36.633" Score="11" ViewCount="7777" Body="&lt;p&gt;In general, Dirichlet boundary conditions won't be satisfied exactly for FEM for non-homogeneous boundary conditions. The FEM codes I've seen set the degrees of freedom to interpolate the Dirichlet boundary condition but I haven't found any mathematical justification for this. It seems to me that setting essential boundary conditions should probably minimize some functional of the error (e.g. minimize $ ||u -u_h|| $ over the portion of the boundary that the Dirichlet BC is applied ) even though this would be more computationally expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any justification for setting the BC like this and if so, what would the proper norm be?&lt;/p&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2023-08-09T16:56:08.307" Title="How to properly apply non-homogeneous Dirichlet boundary conditions with FEM?" Tags="|pde|finite-element|boundary-conditions|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5073" PostTypeId="2" ParentId="5072" CreationDate="2013-01-21T20:15:56.713" Score="10" Body="&lt;p&gt;There is mathematical justification for setting Dirichlet boundary degrees of freedom to a value. However, you should adjust your variational form accordingly. If you are looking at a general problem, say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find $u\in\mathcal{U}$ such that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a(u,w)=l(w) \ \ \forall w\in\mathcal{V}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal{U}=\{u:\int \nabla u^2 &amp;lt; \infty, u=g\text{ on }\Gamma_D\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal{V}=\{u:\int \nabla u^2 &amp;lt; \infty, u=0\text{ on }\Gamma_D\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead we can write $u = v + g$ where $v\in\mathcal{V}$ and $g$ is the Dirichlet condition. Then the variational form becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a(v+g,w)=l(w)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or by using the linearity of $a(.,.)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a(v,w)=l(w)-a(g,w)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a finite element code, you can form your element stiffness matrix as if there were no boundary conditions. Then you take the column of the local matrix which corresponds to the Dirichlet boundary condition, scale it by the coefficient you want to enforce, and subtract it from the right-hand-side. This is the discrete form of what I wrote above, $-a(g,w)$. Then you zero out that column and the corresponding Dirichlet row, placing a 1 in the diagonal and the coefficient you wish to enforce. This decouples the equation from the system and yet sets the value you wish to enforce. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend &lt;em&gt;The Finite Element Method: Linear Static and Dynamic Finite Element Analysis&lt;/em&gt;, by Tom Hughes. He has an expanded discussion of this issue starting on page 8.&lt;/p&gt;&#xA;" OwnerUserId="1625" LastEditorUserId="1625" LastEditDate="2013-01-21T20:42:54.360" LastActivityDate="2013-01-21T20:42:54.360" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5074" PostTypeId="2" ParentId="4968" CreationDate="2013-01-21T22:02:51.083" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://www.kwon3d.com/theory/filtering/butt.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This site&lt;/a&gt; gives formulas for butterworth and various common filters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here's &lt;a href=&quot;https://stackoverflow.com/a/10498563/181638&quot;&gt;some C code&lt;/a&gt; to try.&lt;/p&gt;&#xA;" OwnerUserId="3299" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-01-21T22:02:51.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5075" PostTypeId="2" ParentId="5004" CreationDate="2013-01-22T02:00:27.333" Score="1" Body="&lt;p&gt;I would expect that you'd want to vary $n$ from step to step - start with a small sample to make large strides quickly, then slowly increase the size to stabilize once you're close to a minimum. And then maybe run this procedure many times if you expect you might get stuck in local minima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sounds a bit like &lt;a href=&quot;http://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow&quot;&gt;simulated annealing&lt;/a&gt; in that you'd use the sample size in a similar way to the temperature in simulated annealing, a large sample corresponding to a low temperature. It is notoriously difficult to design a good cooling schedule for simulated annealing - as far as I know, essentially you need to choose and tweak the parameters by instinct. I expect that similarly you cannot really &lt;em&gt;derive&lt;/em&gt; the correct schedule for changing $n$ - you'll have to guess and experiment...&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2013-01-22T02:00:27.333" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5076" PostTypeId="1" AcceptedAnswerId="5097" CreationDate="2013-01-22T03:54:16.740" Score="5" ViewCount="1483" Body="&lt;p&gt;I know the boundary condition is usually a tricky question. However, I am testing a finite-difference scheme for the equation of the form $$u_t=a(x)u_{xx}$$ that I know the analytical solution of. So if I specify the Dirichlet boundary condition, I can see the perfect convergence with a desired rate, but if I specify the value of the derivative I see the deterioration of accuracy of the numerical solution at the boundary(still have a good convergence in the middle of the grid). Both are exact from the analytical solution but looks like the derivative is not a good choice. What I think is that there should be only one solution that satisfies the initial condition and such a value of the &quot;slope&quot;, or I am wrong? I might have a bug in my code but could not spot it so I wonder if imposing the partial derivative on the boundary instead of the value of the function could destroy uniqueness and this is why I am converging to a slightly different solution?  &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1744" LastEditDate="2013-01-25T17:31:59.240" LastActivityDate="2013-01-25T17:31:59.240" Title="The effect of the boundary condition on the convergence of the fdm scheme" Tags="|pde|parabolic-pde|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5077" PostTypeId="2" ParentId="5068" CreationDate="2013-01-22T06:54:30.277" Score="2" Body="&lt;p&gt;If you follow the analysis until the last equation, then you can see the problem. You write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|| u-u_h|| \le C \frac{M}{\alpha} h^r |u|_{H^{r+1}(\Omega)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which from the expression just above it could also be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|| u-u_h|| \le C \left(1+\frac{|b|}{\mu}\right) h^r |u|_{H^{r+1}(\Omega)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now say you want to solve a problem for a fixed mesh size. In the first problem $|b|=\mu$, then you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|| u-u_h|| \le 2C h^r |u|_{H^{r+1}(\Omega)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you move to the next problem and the transport is strong compared to diffusion, $|b|=10^4 \mu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|| u-u_h|| \le 10^4 C h^r |u|_{H^{r+1}(\Omega)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So on a mesh with fixed size, the second solution is orders of magnitude worse. The only way to improve it is to decrease $h$ to an amount that resolves the transport scale (counteracting the huge constant).&lt;/p&gt;&#xA;" OwnerUserId="1625" LastActivityDate="2013-01-22T06:54:30.277" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5078" PostTypeId="2" ParentId="5071" CreationDate="2013-01-22T09:16:48.557" Score="1" Body="&lt;p&gt;It doesn't have to be $u_h$ at a grid point. You can apply this using any functional of $u_h$ over the domain, including the 2-norm. Assume a second order method, so $\| u_h - u \| \leq Ch^2$. Then $\| u_{2h} - u_h \| \leq \| u_{2h} - u\| + \| u - u_h \| = 5Ch^2$. Similarly $\| u_{4h} - u_{2h} \| \leq 20Ch^2$ and you get the ratio 4 if the code is correct.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-01-22T09:16:48.557" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5079" PostTypeId="1" CreationDate="2013-01-22T10:13:36.020" Score="4" ViewCount="3608" Body="&lt;p&gt;I've been studying this FEM theory and for the parabolic problems, there's the analysis of stability of the $\theta$-method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed the analysis and they get this CFL (Courant-Friedrich-Lewy) condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\lambda_{max} \simeq Ch^{-2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does this CFL condition means? How can I understand this?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2013-01-22T16:29:38.623" Title="Meaning of CFL condition on parabolic problems" Tags="|finite-element|finite-difference|stability|parabolic-pde|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5080" PostTypeId="1" CreationDate="2013-01-22T10:32:44.913" Score="6" ViewCount="1241" Body="&lt;p&gt;I need detailed explanation of the formula below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A2=I1*A1*I2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suppose this formula computes matrix A2 on a coarse grid and here A1 is original matrix on fine grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So A1 is given, but I don't know I1 and I2. That is why I am asking where I can get these matrices I1 and I2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &quot;Briggs, Multigrid Tutorial, SIAM, 2000&quot;  at page 52 &quot;1D Restriction by injection&quot; is explained. There they simply drop every second point in a graph to move into coarse grid and then calculate average of neighbouring points to come back to the fine grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am working with two-dimensional matrix A1. Does that mean that I must drop every second element in one row of matrix or drop every second row to get smaller matrix in coarser grid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I want to solve system of linear algebraic equations using multigrid method.My question is how to relate technique explained at that book(page 52) to the two dimensional matrix A? If formula above is a way, then what is I1 and I2 and how to find them?&lt;/p&gt;&#xA;" OwnerUserId="3544" LastEditorUserId="414" LastEditDate="2013-01-29T07:30:40.827" LastActivityDate="2013-01-29T07:30:40.827" Title="restriction and interpolation in multigrid method" Tags="|linear-algebra|matrix|interpolation|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5081" PostTypeId="2" ParentId="5080" CreationDate="2013-01-22T14:14:04.787" Score="12" Body="&lt;p&gt;Given interpolation $I_H^h$ and restriction $I_h^H$ (where restriction is typically $(I_H^h)^T$ for symmetric problems), with fine grid discretized operator $A^h$, there are two common approaches for constructing the coarse grid operator $A^H$.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;(Petrov-)Galerkin coarse operators&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This explicitly computes the matrix triple product&#xA;$$ A^H = I_h^H A^h I_H^h .$$&#xA;This has well-developed convergence theory for symmetric problems (including those with low regularity) and is a black-box procedure, requiring only the matrices with no information about the underlying problem. When used with finite element methods, the coarse operators can be interpreted as a FEM discretization in some coarse grid with &quot;multiscale&quot; basis functions. For unstructured problems, coarse Galerkin operators often have &quot;stencil growth&quot;, in which the number of nonzeros per row grows &quot;excessively&quot;. Aggressive coarsening helps reduce stencil growth at the expense of less robustness/slower convergence. Computing the triple product is relatively expensive, especially in parallel, usually representing the majority of the setup time for algebraic multigrid.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Rediscretized coarse operators&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If your coarse grids are geometrically meaningful, you can defined $A^H$ by explicitly rediscretizing your equations on the coarse grid. This can be generalized further, e.g., to discretize a different equation set that is more appropriate at that coarse scale. Extreme examples of this include using lower-dimensional integrated models for coarse grids, or even combining fine-grid Monte-Carlo simulations with macro-scale deterministic PDEs. Rediscretization is much more popular than Galerkin coarse operators for steady-state fluids solvers, and fixes the &quot;stencil growth&quot; problem at the expense of requiring more problem knowledge and typically lower robustness for problems with multiscale coefficients. When rediscretization is used, it is usually possible to eliminate all assembled matrices from the algorithm, thus reducing memory requirements.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-01-22T14:14:04.787" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5082" PostTypeId="1" CreationDate="2013-01-22T14:22:28.697" Score="8" ViewCount="3134" Body="&lt;p&gt;I need to learn and utilize a finite volume automated solution package for a project I'm working on and have narrowed it down to these two packages. I was wondering if anybody has experience of both packages and could perhaps comment on a comparison. Considering the similar goal of both packages I am a little surprised that I can't find much in the way of a comparison on-line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My feeling is that FiPy would be easier to learn/implement as it is Python based but that OpenFoam may be more robust and feature rich due to its wide adoption? Both packages have very credible organisations behind them. &lt;/p&gt;&#xA;" OwnerUserId="1880" LastActivityDate="2013-01-23T08:55:30.897" Title="OpenFoam vs FiPy" Tags="|openfoam|finite-volume|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5083" PostTypeId="2" ParentId="5079" CreationDate="2013-01-22T14:34:23.950" Score="14" Body="&lt;p&gt;The CFL condition states that the &quot;mathematical domain of dependence&quot; must be (asymptotically) contained in the numerical domain of dependence. For hyperbolic problems, this provides a bound $\Delta t &amp;lt; C \Delta x$ that is useful at all resolutions. For a parabolic problem, it merely requires that $\Delta t \in o(\Delta x)$ in the limit $\Delta x \to 0$. That is, $\Delta t$ must go to zero &lt;em&gt;strictly faster&lt;/em&gt; than $\Delta x$, which causes information to propagate infinitely fast, thus matching the mathematical (continuum) behavior. You cannot conclude based purely on CFL theory that the time step must indeed go to zero at least as fast as $\Delta x^2$. This result is readily established using von Neumann stability analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/4all.pdf&quot;&gt;Chapter 4&lt;/a&gt; of Trefethen's &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/pdetext.html&quot;&gt;Finite Difference and Spectral Methods for Ordinary and Partial Differential Equations&lt;/a&gt; for further details on this subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Improper usage:&lt;/em&gt; The term &quot;CFL&quot; is sometimes misused to refer to whatever is the appropriate sharp stability requirement for an explicit method applied to the problem being considered. Indeed, CFL analysis is too weak to provide $\Delta t \sim\Delta x^2$ for parabolic problems or exotic dispersive wave problems (such as &lt;a href=&quot;http://dx.doi.org/10.1029/2005JA011357&quot;&gt;VLF Whistler waves&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-01-22T16:29:38.623" LastActivityDate="2013-01-22T16:29:38.623" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5085" PostTypeId="1" CreationDate="2013-01-10T21:15:16.960" Score="5" ViewCount="176" Body="&lt;p&gt;I need some hints on how to solve this diffusion equation ($\alpha, k_1,k_2$ and $k_3$ are constants):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ {\partial P \over \partial y} + k_1 {\partial P \over \partial t} + \alpha P  = {1 \over k_2} {1 \over x}{\partial  \over \partial x} \left(x k_3 {\partial P \over \partial x}\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With boundary conditions:&lt;br&gt;&#xA;$P(x=1,y,t)=0$&lt;br&gt;&#xA;$k_3 {\partial P \over \partial x}|_{x=0}=0$&lt;br&gt;&#xA;$P(x,0,t)=p(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been told that DuFort-Frankel is a proper method for this example, so I tried (not sure if it is correct):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${P_{i,j+1}^n-P_{i,j-1}^n \over 2 \Delta y}+k_1{P_{i,j}^{n+1}-P_{i,j}^{n-1} \over 2 \Delta t}+\alpha P_{i,j}^n = {k_3 \over k_2x}\left({P_{i+1,j}^n-P_{i-1,j}^n \over 2 \Delta x}+x{P_{i+1,j}^n-(P_{i,j}^{n+1}+P_{i,j}^{n-1})+P_{i-1,j}^n \over \Delta x^2}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't know how tie this with boundary conditions so I can solve it using recursive functions... It is supposed to be pretty easy, am I missing something? DuFort-Frankel is not necessary, if You know how to solve it using Taylor, Leapfrog, Richardson or any other method, I would be very grateful for any hints...&lt;/p&gt;&#xA;" OwnerDisplayName="apm" LastActivityDate="2013-01-23T00:06:11.087" Title="Solving diffusion PDE using finite differences" Tags="|pde|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5087" PostTypeId="1" CreationDate="2013-01-09T22:22:29.220" Score="2" ViewCount="157" Body="&lt;p&gt;I want to simulate on my computer the solution of the heat equation in 3 space dimensions with Cauchy initial data, that is &#xA;$$\partial_t u=Tr[A(x)\cdot \Delta u], u(0,x)=u_0(x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_0\in C(\mathbb{R}^3,\mathbb{R})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if $A$ is constant I'm not sure what's numerically the best way to do this but I'm sure this must be in standard toolboxes of Matlab, Scilab etc. however I have quite some trouble with the documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone give me pointers in the right direction?&#xA;More general, what is a good book or lecture notes that deal with numerics for parabolic PDEs (with multidimensions in space)? &lt;/p&gt;&#xA;" OwnerUserId="3690" OwnerDisplayName="john" LastEditorUserId="3690" LastEditDate="2013-02-05T22:16:41.633" LastActivityDate="2013-02-05T22:16:41.633" Title="Numerics for heat equation" Tags="|pde|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5088" PostTypeId="1" CreationDate="2012-12-29T12:57:17.023" Score="8" ViewCount="183" Body="&lt;p&gt;Let $X_t$ be an Ito process&#xA;$$&#xA;   dX_t=a(X_t,t)dt + b(X_t,t)dW_t&#xA;$$&#xA;where $W_t$ is a Wiener process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A numerical approximations of the solution of this equations is proposed by Milstein:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;X_T=X_t+a(X_t,t) \Delta t+ b(X_t,t)\Delta W_t+ \frac{1}{2}b(X_t,t) \frac{\partial{b(X_t,t)} }{\partial{x}} \left( \Delta W_t^2 - \Delta t\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta t = T-t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta W_t = W_T-W_t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According with literature, this can be transformed into a derivative-free scheme via the approximation (known as Platen explicit order 1 strong  scheme):&#xA;$$&#xA;b(X_t,t) \frac{\partial{b(X_t,t)} }{\partial{x}} \approx \frac{b(X_t+a(X_t,t) \Delta t+ b(X_t,t)\sqrt{\Delta t},t)-b(X_t,t)}{\sqrt{\Delta t}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(See: &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.8.7565&quot; rel=&quot;nofollow&quot;&gt;2001, Kloeden, &quot;A brief overview of numerical methods for Stochastic Differential Equations&quot;&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody help understand how this approximation of the partial derivative is obtained?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerDisplayName="Fabio" LastEditorUserId="9918" LastEditDate="2014-08-09T16:59:47.123" LastActivityDate="2016-02-06T06:18:00.140" Title="Approximation of partial derivative of a function of stochastic variable" Tags="|monte-carlo|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5089" PostTypeId="1" CreationDate="2012-12-05T14:54:19.727" Score="9" ViewCount="1928" Body="&lt;p&gt;When it is preferable to use &lt;a href=&quot;http://en.wikipedia.org/wiki/Bernstein_polynomial&quot;&gt;Bernstein polynomials&lt;/a&gt; to approximate a continuous function instead of using the only following preliminary Numerical Analysis methods: &quot;Lagrange Polynomials&quot;, &quot;Simple finite differences operators&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is about compairing these method.&lt;/p&gt;&#xA;" OwnerUserId="3630" OwnerDisplayName="AmirHosein SadeghiManesh" LastEditorUserId="1625" LastEditDate="2013-01-27T06:33:10.530" LastActivityDate="2013-01-27T06:33:10.530" Title="When we use Bernstein polynomials in application" Tags="|finite-element|finite-difference|interpolation|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5091" PostTypeId="2" ParentId="5082" CreationDate="2013-01-23T08:50:13.813" Score="3" Body="&lt;p&gt;I haven't used &lt;a href=&quot;http://www.ctcms.nist.gov/fipy/&quot; rel=&quot;nofollow&quot;&gt;FiPy&lt;/a&gt; but only &lt;a href=&quot;http://openfoam.org&quot; rel=&quot;nofollow&quot;&gt;OpenFOAM&lt;/a&gt;, but I think some points on both homepages suggest it strongly depends on what you want to simulate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenFOAM already has &lt;a href=&quot;http://openfoam.org/docs/user/standard-solvers.php&quot; rel=&quot;nofollow&quot;&gt;some solvers imlemented&lt;/a&gt; (mostly CFD, since your profile states computational plasma physics the &lt;a href=&quot;http://openfoamwiki.net/index.php/MhdFoam&quot; rel=&quot;nofollow&quot; title=&quot;Solver for magnetohydrodynamics (MHD): incompressible, laminar flow of a conducting fluid under the influence of a magnetic field&quot;&gt;&lt;code&gt;mhdFoam&lt;/code&gt;&lt;/a&gt; solver may be what you want or at least serve as basis, &lt;a href=&quot;http://www.extend-project.de/user-groups/23/viewgroup/groups&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;'s its user group), but can be expanded to solve other FVM-feasible PDEs, while in FiPy you'll probably have to implement the equations from scratch anyway - and in the latter case, yes, Python is a lot more comfortable than C++, at least to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some Python codes that make life with OpenFOAM easier: &lt;a href=&quot;http://openfoamwiki.net/index.php/Contrib_PyFoam&quot; rel=&quot;nofollow&quot;&gt;PyFoam&lt;/a&gt; to help setting up cases, and &lt;a href=&quot;http://pythonflu.wikidot.com/&quot; rel=&quot;nofollow&quot;&gt;pythonflu&lt;/a&gt;, a wrapper to write solvers in Python (untested, I don't know how that performs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my experience, setting up a case in OpenFOAM is rather tedious and you will most likely end up writing scripts to automate the mesh setup etc anyway, so I don't know whether FiPy would actually require that much more effort even &lt;em&gt;if&lt;/em&gt; you used one of OpenFOAM's already implemented solvers. And should you choose to use Python for that scripting, I suspect you can write most of the code in a way that the decision on FiPy vs OpenFOAM boils down to a difference of less than 100 lines of code, i.e. it shouldn't be too difficult to change your mind later on, especially if planned beforehand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more important points are of course, how much time does the first time setup take and how well does it perform. I couldn't find any comparisons on either though, so you'd have to compare them yourself...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, had I not already used OpenFOAM before and I would need to simulate a system for which OpenFOAM does not already provide a solver, I would most likely try FiPy first due to my preference of Python over C++. But as mentioned, together with PyFoam one could probably come up with a rather generic framework to compare the two... If a solver of OpenFOAM could be used though, I'd probably use that one.&lt;/p&gt;&#xA;" OwnerUserId="89" LastEditorUserId="89" LastEditDate="2013-01-23T08:55:30.897" LastActivityDate="2013-01-23T08:55:30.897" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5092" PostTypeId="1" CreationDate="2013-01-23T10:04:32.570" Score="3" ViewCount="328" Body="&lt;p&gt;I am using the Fast Marching Method (FMM) to calculate shortest &quot;distance&quot; (traveltime) from some points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way FMM works is: I keep a velocity function in RAM: V(xi,yj,zk). I also keep a priority queue of all points on the front sorted on their V value.&#xA;I repeatedly propagate the first point in this queue one step outwards. As I do this I remove the used point from the queue and insert those &quot;touched&quot; by the front. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current implementation have two problems:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I. I must keep the whole cost (slowness) function in RAM. This limits the size of the cost function I can use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;II. I would like it to be even faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to improve my current implementation?&#xA;For instance would it be possible to implement this on the GPU?&lt;/p&gt;&#xA;" OwnerDisplayName="user3617" LastEditorUserId="3622" LastEditDate="2013-01-24T13:59:12.090" LastActivityDate="2013-01-26T16:12:06.157" Title="A better Fast Marching Method?" Tags="|pde|parallel-computing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5093" PostTypeId="1" AcceptedAnswerId="5098" CreationDate="2013-01-23T11:42:14.127" Score="26" ViewCount="10385" Body="&lt;p&gt;I need to numerically evaluate the integral below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_0^\infty \mathrm{sinc}'(xr) r \sqrt{E(r)} dr$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $E(r) = r^4 (\lambda\sqrt{\kappa^2+r^2})^{-\nu-5/2} K_{-\nu-5/2}(\lambda\sqrt{\kappa^2+r^2})$, $x \in \mathbb{R}_+$ and $\lambda, \kappa, \nu &amp;gt;0$. Here $K$ is the modified Bessel function of the second kind. In my particular case I have $\lambda = 0.00313$, $\kappa = 0.00825$ and $\nu = 0.33$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using MATLAB, and I have tried the built-in functions &lt;code&gt;integral&lt;/code&gt; and &lt;code&gt;quadgk&lt;/code&gt;, which gives me a lot of errors (see below). I have naturally tried numerous other things as well, such as integrating by parts, and summing integrals from $kx\pi$ to $(k+1)x\pi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, do you have any suggestions as to which method I should try next?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE (added questions)&lt;/strong&gt;&lt;br&gt;&#xA;I read the paper @Pedro linked to, and I don't think it was too hard to understand. However, I have a few questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would it be okay to use $x^k$ as the basis-elements $\psi_k$, in the univariate Levin method described?&lt;/li&gt;&#xA;&lt;li&gt;Could I instead just use a Filon method, since the frequency of the oscillations is fixed?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example code&lt;/strong&gt;&lt;br&gt;&#xA;&lt;code&gt;&amp;gt;&amp;gt; integral(@(r) sin(x*r).*sqrt(E(r)),0,Inf)&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;Warning: Reached the limit on the maximum number of intervals in use. Approximate&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;bound on error is   1.6e+07. The integral may not exist, or it may be difficult to&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;approximate numerically to the requested accuracy.&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;&amp;gt; In funfun\private\integralCalc&amp;gt;iterateScalarValued at 372&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;In funfun\private\integralCalc&amp;gt;vadapt at 133&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;In funfun\private\integralCalc at 84&lt;/code&gt;&lt;br&gt;&#xA;&lt;code&gt;In integral at 89&lt;/code&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;ans =&lt;/code&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;3.3197e+06&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3618" LastEditorUserId="127" LastEditDate="2013-05-14T06:05:35.367" LastActivityDate="2017-02-22T01:08:59.570" Title="Method for numerical integration of difficult oscillatory integral" Tags="|matlab|quadrature|special-functions|" AnswerCount="4" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5094" PostTypeId="2" ParentId="5089" CreationDate="2013-01-23T12:05:32.053" Score="7" Body="&lt;p&gt;Bernstein polynomials and Lagrange polynomials both span the same spaces. So in terms of the possible functions one can represent, using one or the other makes no difference. However, if you are thinking of using these as basis functions in either a finite element method or an interpolation problem, the spectral properties of the linear operator you create will depend on the polynomials you choose as a basis. This can cause differences in the convergence of iterative solvers. However in the absence of linear algebra error, you will get the same answer using either basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing this to finite difference operators is a different story. Using polynomials will give you error approximations on a continuous norm. I am not as well versed in finite differences, but my understanding is that you will only get an error estimate at the locations that you choose to discretize. What happens in between these points is not as clear.&lt;/p&gt;&#xA;" OwnerUserId="1625" LastActivityDate="2013-01-23T12:05:32.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5095" PostTypeId="1" AcceptedAnswerId="5130" CreationDate="2013-01-23T12:31:22.560" Score="9" ViewCount="6497" Body="&lt;p&gt;I am reading a book on numerical methods and the square of the discrete $L^2$ norm is defined as $$||x||^2_2=h\sum_1^Nx^2_i$$&#xA;Every point gets a &quot;weight&quot;, which is $h$, thus this is like an average over the squares of the values at all points. This in fact comes from the approximation of a continuous integral. On the other hand, can I define similar norm where the grid is nonuniform with spacing $h_i$ as $$||x||^2_2=\sum_1^Nh_ix^2_i$$&#xA;that seems to me natural as I could approximate a continuous integral this way too but as I don't see that in the books made me suspicious I am missing something! So, if I have non-uniform grid and I want to make some estimates in this norm, how one should define it?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="123" LastEditDate="2013-01-28T11:52:22.240" LastActivityDate="2013-01-29T10:11:09.413" Title="discrete $L^p$ norms for non-uniform grid" Tags="|pde|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5096" PostTypeId="2" ParentId="5089" CreationDate="2013-01-23T12:47:25.933" Score="6" Body="&lt;p&gt;The paper below shows that representing polynomials in Bernstein form leads to numerically stable algorithms in many cases:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;R.T. Farouki, V.T. Rajan, On the numerical condition of polynomials in Bernstein form, &lt;em&gt;Computer Aided Geometric Design&lt;/em&gt;, Volume 4, Issue 3, November 1987, Pages 191-216, DOI: &lt;a href=&quot;http://dx.doi.org/10.1016/0167-8396%2887%2990012-4&quot;&gt;10.1016/0167-8396(87)90012-4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="640" LastActivityDate="2013-01-23T12:47:25.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5097" PostTypeId="2" ParentId="5076" CreationDate="2013-01-23T13:02:28.623" Score="4" Body="&lt;p&gt;As Hui pointed out above, to apply Neumann boundary conditions correctly you should utilize ghost points and extend your $n^{th}$ order discretisation stencil to your domain boundaries. Utilising forward/backward difference or extrapolation at the boundary will degrade your solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming a $2^{nd}$ order central difference scheme the Neuman boundary condition, $u_x(t,x=0) = p_0(t)$, becomes (omitting $(t)$'s for convenience):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_x\rvert_0 = p_0 \approx \frac{u_1 - u_{-1}}{2h}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Rightarrow u_{-1} = u_1 - 2hp_0 &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, substitute for $u_{-1}$ into your discretised equation at the boundary, to obtain a Dirichelt BC $u_0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_0(t) = a(0)\frac{u_1 - 2u_0 + u_{-1}}{h^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Rightarrow u_0 = a(0)\frac{2u_1 - 2hp_0}{h^2 + 2a(0)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying the Neumann boundary condition in this way is consistent and is strictly equivalent to applying the analytic Dirichelt bc in the limit $h\rightarrow 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for which is more accurate, for finite $h$, if you apply the Dirchlet condition at the boundary you impose exact agreement between the analytic and numerical solution at the boundary, but your estimation of the solutions derivative at this point will contain a $2^{nd}$ order error. Applying the Neumann boundary condition simply does the reverse, i.e. yields exact agreement between the analytic and numerical values of the derivative of your solution (estimated using your finite difference stencil) at the boundary, while estimations of $u$ at this point will contain a $2^{nd}$ order error. In the limit $h \rightarrow 0$ both methods should converge to the same unique solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are not utilising ghost points then you are not applying the Neumann conditions in a consistent way and your result will indeed simply be less accurate then when you solve using the Dirichelt boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="1880" LastEditorUserId="1880" LastEditDate="2013-01-23T14:41:15.253" LastActivityDate="2013-01-23T14:41:15.253" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5098" PostTypeId="2" ParentId="5093" CreationDate="2013-01-23T13:14:28.583" Score="12" Body="&lt;p&gt;I've written my own integrator, &lt;a href=&quot;http://www.mathworks.co.uk/matlabcentral/fileexchange/authors/228877&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;quadcc&lt;/code&gt;&lt;/a&gt;, which copes substantially better than the Matlab integrators with singularities, and provides a more reliable error estimate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use it for your problem, I did the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; lambda = 0.00313; kappa = 0.00825; nu = 0.33;&#xA;&amp;gt;&amp;gt; x = 10;&#xA;&amp;gt;&amp;gt; E = @(r) r.^4.*(lambda*sqrt(kappa^2 + r.^2)).^(-nu-5/2) .* besselk(-nu-5/2,lambda*sqrt(kappa^2 + r.^2));&#xA;&amp;gt;&amp;gt; sincp = @(x) cos(x)./x - sin(x)./x.^2;&#xA;&amp;gt;&amp;gt; f = @(r) sincp(x*r) .* r .* sqrt( E(r) );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The function &lt;code&gt;f&lt;/code&gt; is now your integrand. Note that I've just assigned any old value to &lt;code&gt;x&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to integrate on an infinite domain, I apply a substitution of variables:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; g = @(x) f ( tan ( pi / 2 * x ) ) .* ( 1 + tan ( pi * x / 2 ).^2 ) * pi / 2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i.e. integrating &lt;code&gt;g&lt;/code&gt; from 0 to 1 should be the same as integrating &lt;code&gt;f&lt;/code&gt; from 0 to $\infty$. Different transforms may produce different quality results: Mathematically all transforms should give the same result, but different transforms may produce smoother, or more easily integrable &lt;code&gt;g&lt;/code&gt;s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then call my own integrator, &lt;code&gt;quadcc&lt;/code&gt;, which can deal with the &lt;code&gt;NaN&lt;/code&gt;s on both ends:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; [ int , err , npoints ] = quadcc( g , 0 , 1 , 1e-6 )&#xA;int =&#xA;  -1.9552e+06&#xA;err =&#xA;   1.6933e+07&#xA;npoints =&#xA;       20761&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the error estimate is huge, i.e. &lt;code&gt;quadcc&lt;/code&gt; doesn't have much confidence in the results. Looking at the function, though, this is not surprising as it oscillates at values three orders of magnitude above the actual integral. Again, using a different interval transform may produce better results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also want to look at more specific methods such as &lt;a href=&quot;https://people.cs.kuleuven.be/~daan.huybrechs/research/HOQ.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;. It's a bit more involved, but definitely the &lt;em&gt;right method&lt;/em&gt; for this type of problem.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2013-08-20T20:06:13.050" LastActivityDate="2013-08-20T20:06:13.050" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5099" PostTypeId="1" AcceptedAnswerId="5104" CreationDate="2013-01-23T14:47:43.530" Score="8" ViewCount="357" Body="&lt;p&gt;Suppose I have the parabolic system $$u_t=\nabla\cdot(k(x)\nabla u)+f,\quad (x,t)\in\Omega\times I$$ with Dirichlet boundary conditions $$u=g, \quad x\in\partial\Omega$$ and initial condition &#xA;$$u(x,t)= h,\quad t=0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often times in engineering, we are more interested in the asymptotic (steady state) behavior of this PDE rather than the transient behavior.  So, we sometimes neglect the time derivative term and solve the elliptic system &#xA;$$-\nabla\cdot(k(x)\nabla u)=f,\quad (x,t)\in\Omega\times I$$&#xA;instead.  The assumption is that over infinite time, $$\lim_{t\rightarrow \infty}u_{\mathrm{parabolic}}(x,t)= u_{\mathrm{elliptic}}(x,t)$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've observed that when $f\equiv 0$, this limit is true, but I'm not sure if this is the case for arbitrary $f$ or if there are other necessary conditions to guarantee this limit to be true.  Do the boundary conditions have to converge asymptotically to a constant value in order for the parabolic solution to converge to the elliptic solution?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though I'm my question is formulated in the continuous case, I'm also curious if the same conditions are true for the discrete case.  That is, assuming I use a stable and consistent finite difference scheme to approximate $u_{\mathrm{parabolic}}$ and $u_{\mathrm{elliptic}}$, should I expect $$\lim_{t\rightarrow \infty}u_{\mathrm{parabolic}}^\mathrm{fdm}(x,t)= u_{\mathrm{elliptic}}^\mathrm{fdm}(x,t)$$ if $u_{\mathrm{elliptic}}^\mathrm{fdm}$ and $u_{\mathrm{parabolic}}^\mathrm{fdm}$ are discretized on the same spatial grid and $\Delta t\rightarrow\infty$?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="16799" LastEditDate="2015-10-02T23:48:30.123" LastActivityDate="2015-10-02T23:48:30.123" Title="Asymptotic convergence of the solution to a parabolic pde to the solution of an elliptic pde" Tags="|pde|convergence|parabolic-pde|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5100" PostTypeId="1" CreationDate="2012-11-26T19:17:08.357" Score="7" ViewCount="787" Body="&lt;p&gt;I have a $4\times 4$ matrix and I want to use &lt;strong&gt;Jacobi iteration&lt;/strong&gt; on it. Right now the spectral radius is higher than $1$. I know that the method is guaranteed to converge if the matrix is &lt;strong&gt;diagonally dominant&lt;/strong&gt;. I can't do any reordering that would make the matrix diagonally dominant. Is it sensible to try to get as close as possible to diagonally dominant matrix and if the spectral radius is still greater than $1$, I can state that I can not use Jacobi iteration? Or is it possible that there can still exist some reordering of the columns and rows, that would have spectral radius lesser than $1$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;In other words, is there some correlation between some of the matrix parameters and the size of the spectral radius?&lt;/p&gt;&#xA;" OwnerUserId="3627" OwnerDisplayName="Honza Brabec" LastEditorUserId="127" LastEditDate="2013-05-26T16:52:52.783" LastActivityDate="2013-05-26T16:53:09.660" Title="Decreasing spectral radius" Tags="|linear-algebra|iterative-method|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5101" PostTypeId="1" CreationDate="2012-12-26T04:23:13.080" Score="21" ViewCount="3468" Body="&lt;p&gt;$A$ is a $n \times n$ symmetric positive definite (SPD) sparse matrix. $G$ is a sparse diagonal matrix. $n$ is large ($n$ &gt;10000) and the number of nonzeros in the $G$ is usually 100 ~ 1000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A$ has been factorized in the cholesky form as $LDL^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to update the $L$ and $D$ efficiently when $A$ becomes $A+G$?&lt;/p&gt;&#xA;" OwnerDisplayName="luogyong" LastActivityDate="2019-12-04T11:16:00.450" Title="Diagonal update of a symmetric positive definite matrix" Tags="|linear-algebra|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5102" PostTypeId="1" AcceptedAnswerId="5178" CreationDate="2013-01-23T23:11:50.077" Score="15" ViewCount="347" Body="&lt;p&gt;I've heard anecdotally that when one is trying to numerically do an integral of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_0^\infty f(x) J_0(x)\,\mathrm{d}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $f(x)$ smooth and well-behaved (e.g. not itself highly oscillatory, nonsingular, etc.), then it will help accuracy to rewrite it as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{\pi}\int_0^\pi \int_0^\infty f(x) \cos(x\sin\theta) \,\mathrm{d}x\,\mathrm{d}\theta$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and perform the inner integral numerically first. I can't see any reason I should expect this to work, but then again the accuracy of a numerical method is rarely obvious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course I know the best way to actually do it is to use a method optimized for oscillatory integrals like this, but for curiosity's sake, suppose I limit myself to using some quadrature rule. Can anyone confirm or refute that making this transformation tends to improve the accuracy of the integral? And/or point me to a source explaining it?&lt;/p&gt;&#xA;" OwnerUserId="103" LastEditorUserId="127" LastEditDate="2013-05-14T04:28:25.193" LastActivityDate="2013-05-14T04:28:25.193" Title="Does transforming $J_0(x)\to\int\cos(x\sin\theta)$ help with numerical integration?" Tags="|quadrature|special-functions|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5103" PostTypeId="2" ParentId="5089" CreationDate="2013-01-23T23:14:03.097" Score="7" Body="&lt;p&gt;I use Bernstein polynomials in a collocation method to solve boundary value problems for ODEs and PDEs. They are quite interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convergence was exponential for some linear BVPs, but little slower compared to Chebyshev collocation, Legendre Galerkin, and Tau.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the figure comparing convergence rates with some Chebyshev spectral methods. The example problem is linear BVP:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{d^2u}{dx^2}-4\frac{du}{dx}+4u = e^x +C \;,\; x \in [-1,1]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with homogenous Dirichlet BCs, and C is a constant $C=-4e/(1+e)^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/aboeH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also uploaded this figure to &lt;strong&gt;&lt;a href=&quot;http://figshare.com/articles/Convergence_rates_of_different_Spectral_methods_for_linear_boundary_value_problem/154141&quot; rel=&quot;nofollow noreferrer&quot;&gt;figshare&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want, you may check out the code I'm writing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://code.google.com/p/bernstein-poly/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://code.google.com/p/bernstein-poly/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here's the &lt;a href=&quot;http://arxiv.org/abs/1211.3567&quot; rel=&quot;nofollow noreferrer&quot;&gt;arxiv paper I wrote&lt;/a&gt; about solving elliptic BVPs on a square using Bernstein polynomial collocation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last year they celebrated a centennial of Bernstein polynomials - one more interesting fact. &lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="1050" LastEditDate="2013-01-24T09:45:04.997" LastActivityDate="2013-01-24T09:45:04.997" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5104" PostTypeId="2" ParentId="5099" CreationDate="2013-01-24T11:11:58.333" Score="3" Body="&lt;p&gt;Yes, with Dirichlet boundary conditions you always have exponential convergence to the staedy state.  Any PDE book will have a proof.  For a nice explanation from a numerical perspective, see chapter 2 of LeVeque's FDM book.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-01-24T11:11:58.333" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5105" PostTypeId="2" ParentId="5100" CreationDate="2013-01-24T11:15:34.073" Score="4" Body="&lt;p&gt;Reordering will not ever change whether a matrix is diagonally dominant, nor will it change the spectral radius.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="127" LastEditDate="2013-05-26T16:53:09.660" LastActivityDate="2013-05-26T16:53:09.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5106" PostTypeId="2" ParentId="5095" CreationDate="2013-01-24T15:40:21.463" Score="1" Body="&lt;p&gt;Given a function $f(x)$, $x \in (a,b)$, let we define the $L^2$ norm as &#xA;\begin{equation}&#xA;\lVert f \rVert_2^2 = \int_a^b \lvert f(x)\rvert^2 \, dx.&#xA;\end{equation}&#xA;Given a vector $\mathbf f \equiv \{ f_i = f(x_i), \; i=0\dots N\}$, with $x_i = a + i \frac{b-a}{N}$ we define the &lt;em&gt;discrete&lt;/em&gt; $L^2$ norm as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\lVert \mathbf f \rVert_{2,\text{d}}^2 &amp;amp;= h \sum_{i=0}^N \lvert f_i \rvert^2, &#xA;&amp;amp;h = \frac{b-a}{N}&#xA;\end{align}&#xA;In your question you assume that this is done because we want that &#xA;\begin{equation}&#xA;\lim_{h \rightarrow 0} \: \lVert \mathbf f \rVert_{2,\text{d}} = \lVert f \rVert_2&#xA;\end{equation}&#xA;and you are interpreting the &lt;em&gt;discrete&lt;/em&gt; norm as a sort of quadrature rule, and wonder why for non uniform grids a better formula is not used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This interpretation is not wrong, but not the only one possible. As an engineer working with physical quantities, instead of pure numbers, I prefer to think of the discrete norm as of an $\ell^2$-euclidean norm scaled in such a way to make it dimensionally homogeneous to the continuum $L^2$ norm. So if we can prove that $\lVert f - f^h \rVert_2 \rightarrow 0$, we may expect that $\lVert \mathbf f -\mathbf f^h\rVert_{2,\text{d}} \rightarrow 0$. Without the scaling factor this would not be true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I deleted my conclusions here. See the &lt;a href=&quot;https://scicomp.stackexchange.com/a/5130/1744&quot;&gt;answer&lt;/a&gt; by Wolfgang.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;Note that the scaled euclidean norm is easy to compute, while your proposal is a little bit imprecise (may raise some concerns as a quadrature formula) and expensive to compute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottom line: the discrete $L^2$-norm is not (needs not to be) and approximation to the continuous one, but can be simply interpreted as a scaled $\ell^2$-euclidean norm, dimensionally consistent to continuous $L^2$ norm. &#xA;&lt;/strike&gt;&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-01-29T10:11:09.413" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5107" PostTypeId="2" ParentId="5089" CreationDate="2013-01-24T16:35:11.637" Score="2" Body="&lt;p&gt;The control points of a &lt;a href=&quot;http://en.wikipedia.org/wiki/B%C3%A9zier_curve&quot; rel=&quot;nofollow&quot;&gt;Bézier curve&lt;/a&gt; are close to the curve, but not necessarily on the curve. This is exactly the same situation as for the approximation by Bernstein polynomials, and in fact the Bernstein polynomials are the basis for the Bézier curve. You could use a high order Bézier curve to draw a smooth line through a curve given by noisy points, also nobody would do this due to the high computational effort. In fact, high order polynomial interpolation is only rarely used for exactly that reason, only Chebyshev interpolation is occasionally an exception from that rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if we are only talking about low order polynomial interpolation, then the intuitive specification of a Bézier curve via control points is a clear advantage over other methods. However, in this respect NURBS are even better, but at least a Bézier curve is a special case of a NURBS, and Bernstein polynomials are also an important ingredient for NURBS.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2013-01-24T16:35:11.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5108" PostTypeId="1" CreationDate="2013-01-24T20:23:01.480" Score="5" ViewCount="920" Body="&lt;p&gt;I'm working on helping my friend create code to perform the Overlap Dirac Operator and have come across one part that I'm not sure what to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to compute the Eigenvalues and corresponding Eigenvectors for a hermitian matrix (my test is for n=4) so that I can implement a matrix sign function $epsilon(M) = U * epsilon(A) * U^*$ where $U$ is an eigenvector matrix, $U^*$ is the complex transpose of $U$, and $epsilon(A)$ is a matrix with the signs of eigenvalues of $M$ on it's diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any resources or help on existing algorithms/libraries to solve this (or at least make close approximations) would be appreciated. My target programming language is Scala.&lt;/p&gt;&#xA;" OwnerUserId="3623" LastEditorUserId="1744" LastEditDate="2013-01-28T14:56:47.927" LastActivityDate="2013-04-25T01:55:26.970" Title="Eigenvalue Decomposition of Hermitian Matrix in Scala" Tags="|linear-algebra|algorithms|matrix|eigenvalues|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5109" PostTypeId="1" CreationDate="2013-01-25T01:25:20.650" Score="9" ViewCount="429" Body="&lt;p&gt;I am working on improving the optimization process of some demographic modeling software so it can better fit demographic models to data. We'd like to decrease optimization time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time it takes to evaluate our objective function varies a lot, depending on the input values. The relationship between time to evaluate the objective function and the input is known. I am wondering if there are any optimization methods that consider the relative time cost of the objective function when choosing which points to evaluate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Paul requested, here are some salient features of this particular objective function: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The number of parameters is moderate (~12ish)&lt;/li&gt;&#xA;&lt;li&gt;Our problem is non-convex, or at least there are narrow and flat &quot;ridges&quot; in the objective function surface. Right now we're dealing with this using multiple optimizations from different points, but we'd love to do better.&lt;/li&gt;&#xA;&lt;li&gt;The objective function is pretty smooth, although we can only calculate finite-difference approximations to derivatives.&lt;/li&gt;&#xA;&lt;li&gt;The evaluation cost is also a smooth function of the parameter values, and it's quite predictable. roughly speaking, for each parameter the cost to evaluate is high at one end of the range and low at the other end. So we have large regions of expensive-to-evaluate parameter sets, but we know where they are.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1760" LastEditorUserId="1760" LastEditDate="2013-01-25T22:36:16.237" LastActivityDate="2013-01-26T00:45:17.747" Title="Optimization method that considers varying time cost of objective function for different parameters" Tags="|optimization|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5110" PostTypeId="1" CreationDate="2013-01-25T14:53:48.830" Score="16" ViewCount="3835" Body="&lt;p&gt;I'd like to be independent of commercial software for my scientific work. I find a dependence an commercial packages such as Matlab and its toolboxes unsatisfactory, because I do not know if I will have access to Matlab in the future, and because I don't like the language. Therefore, I'm looking for alternatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately, I'm quite fluent in Python (and I love the language), and with NumPy, SciPy, Matplotlib, Basemap, and NetCDF reading and writing routines, it satisfies most of my needs. Most — I still return to Matlab when I need to train satellite retrievals using feed-forward multi-layer perceptrons, e.g. te use Artificial Neural Networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As is not unusual with open-source software, there is more than one package that does neural networks. Considerably more than one:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A while ago I tried &lt;a href=&quot;http://pypi.python.org/pypi/PyBrain/0.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyBrain&lt;/a&gt;, &quot;the swiss army knife for neural networking&quot;, but I didn't succeed in getting any satisfactory results in a short time (both develop-time and run-time). Perhaps I didn't try hard enough, or perhaps it's not really geared toward my exact need.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Just now I discovered  that there is a package called &lt;a href=&quot;https://code.google.com/p/neurolab/&quot; rel=&quot;nofollow noreferrer&quot;&gt;neurolab&lt;/a&gt;, which looks promising: &lt;em&gt;a simple and powerful Neural Network Library for Python&lt;/em&gt;, with an &lt;em&gt;API like Neural Network Toolbox (NNT) from MATLAB&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is &lt;a href=&quot;http://pypi.python.org/pypi/ffnet/0.7.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;FFnet&lt;/a&gt;, &lt;em&gt;a fast and easy-to-use feed-forward neural network training solution for python&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is &lt;a href=&quot;http://pypi.python.org/pypi/simplenn/0.1.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;simplenn&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is &lt;a href=&quot;http://pypi.python.org/pypi/Peach/0.3.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Peach&lt;/a&gt;, a &lt;em&gt;library for computational intelligence and machine learning&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are Python bindings to &lt;a href=&quot;http://leenissen.dk&quot; rel=&quot;nofollow noreferrer&quot;&gt;FANN&lt;/a&gt;, the &lt;em&gt;Fast Artificial Neural Network&lt;/em&gt; library, described as a &lt;a href=&quot;https://stackoverflow.com/a/11477815/974555&quot;&gt;de facto standard in this StackOverflow post&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are probably others.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Has anyone gone through the effort of intercomparing the different options, based on criteria such as easy of use, speed, etc.? My own use case is satellite retrievals, e.g. fitting a strongly non-linear function of many variables. I am very much a &lt;em&gt;user&lt;/em&gt; of neural nets; I am not interested in researching their inner workings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/q/8817/12615&quot;&gt;This question&lt;/a&gt; on Stats.SE is related, but with a different focus.&lt;/p&gt;&#xA;" OwnerUserId="3212" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-08-08T09:26:31.300" Title="Python OSS alternatives for Matlab Neural Network Toolbox. Any intercomparisons?" Tags="|python|machine-learning|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5111" PostTypeId="2" ParentId="5110" CreationDate="2013-01-25T15:19:45.683" Score="1" Body="&lt;p&gt;Did you check out &lt;a href=&quot;http://scikit-learn.org/stable/&quot; rel=&quot;nofollow&quot;&gt;scikit-learn&lt;/a&gt;? It's totally not my domain but I have heard some very positive user experiences...&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-01-25T15:19:45.683" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5112" PostTypeId="2" ParentId="5093" CreationDate="2013-01-25T16:32:39.577" Score="11" Body="&lt;p&gt;As Pedro points out, Levin-type methods are the best established methods for these kinds of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have access to Mathematica? For this problem, Mathematica will detect and use them by default:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[1]:= e[r_] := &#xA; r^4 (l Sqrt[k^2 + r^2])^(-v - 5/2) BesselK[-v - 5/2, l Sqrt[k^2 + r^2]]&#xA;&#xA;In[2]:= {l, k, v} = {0.00313, 0.00825, 0.33};&#xA;&#xA;In[3]:= Block[{x = 10}, &#xA; NIntegrate[Sinc'[x r] r Sqrt[e[r]], {r, 0, \[Infinity]}, &#xA;  PrecisionGoal -&amp;gt; 3]]&#xA;&#xA;Out[3]= -112494.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is a plot over a range of values of x:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[4]:= ListLinePlot[&#xA; Table[NIntegrate[Sinc'[x r] r Sqrt[e[r]], {r, 0, \[Infinity]}, &#xA;   PrecisionGoal -&amp;gt; 3], {x, .5, 10, 0.1}]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2s3x4.png&quot; alt=&quot;Plot from x = 0.5 to x=10&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also manually specify the particular Levin-type method to apply, which in this case can yield a slight performance improvement:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[5]:= method = {&quot;LevinRule&quot;, &quot;Kernel&quot; -&amp;gt; {Cos[r x], Sin[r x]}, &#xA;   &quot;DifferentialMatrix&quot; -&amp;gt; {{0, -x}, {x, 0}}, &#xA;   &quot;Amplitude&quot; -&amp;gt; {(&#xA;     3497.878840962873` Sqrt[(&#xA;      r^4 BesselK[-2.17`, &#xA;        0.00313` Sqrt[&#xA;         0.00006806250000000001` + r^2]])/(0.00006806250000000001` + &#xA;        r^2)^1.415`])/&#xA;     x, -((3497.878840962873` Sqrt[(&#xA;       r^4 BesselK[-2.17`, &#xA;         0.00313` Sqrt[&#xA;          0.00006806250000000001` + r^2]])/(0.00006806250000000001` + &#xA;         r^2)^1.415`])/(r x^2))}, &quot;AdditiveTerm&quot; -&amp;gt; 0};&#xA;&#xA;In[6]:= Block[{x = 10}, &#xA; NIntegrate[Sinc'[x r] r Sqrt[e[r]], {r, 0, \[Infinity]}, &#xA;  PrecisionGoal -&amp;gt; 3, Method -&amp;gt; method]]&#xA;&#xA;Out[6]= -112495.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See the documentation for &lt;a href=&quot;http://reference.wolfram.com/mathematica/tutorial/NIntegrateIntegrationRules.html#32844337&quot; rel=&quot;nofollow noreferrer&quot;&gt;details of Levin-type methods in Mathematica&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="303" LastEditorUserId="276" LastEditDate="2013-08-20T17:43:15.583" LastActivityDate="2013-08-20T17:43:15.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5113" PostTypeId="2" ParentId="5109" CreationDate="2013-01-25T17:22:30.037" Score="4" Body="&lt;p&gt;One common approach to dealing with expensive objective functions is to build (by e.g. regression modeling) a &quot;response surface model&quot; that approximates the original objective function, and then optimize over that response surface rather than working with the original function.  In practice the response surfaces are typically just quadratic models fitting by regression, so finding a minimum of the response surface becomes a very easy optimization problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't said anything about the smoothness or convexity of your objective function.  If the function is nonsmooth or nonconvex, then this obviously becomes much, much harder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also haven't said anything about where the expensive points are in your parameter space.  If they're randomly distributed throughout the parameter space, then you could use design of experiments techniques to build your response surface model while avoiding the expensive points.  If there are larger regions of the parameter space where evaluations are expensive, then you could try to minimize the number of points in those areas that you use in constructing the response surface model.  Of course, if your optimum is located in the middle of such a region, you're going to be doomed to evaluating functions in the expensive region.     &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-01-25T17:22:30.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5114" PostTypeId="2" ParentId="5109" CreationDate="2013-01-26T00:45:17.747" Score="1" Body="&lt;p&gt;I don't know of methods that specifically weigh the relative costs of evaluating the objective at different trial points but if you are able to predict relatively reliably whether a candidate will be expensive to evaluate or not, then I could suggest giving a try to a &lt;em&gt;direct method&lt;/em&gt;. Direct methods fit in the family of derivative-free methods. It's not necessarily a bad thing to use them even if you suspect that your problem is quite smooth because they can provide some level of flexibility that methods for smooth optimization can't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that direct methods define a (iteration-dependent) mesh about the current iterate and a (iteration-dependent) mesh &quot;step&quot;. Based on this mesh step, the method determines the trial points on the mesh that are neighbors of the current iterate (they lie on the mesh and are at a distance defined by the mesh step). It will then proceed to evaluating the objective at the neighbors. As soon as a better candidate is found, it becomes the new current iterate. At your option, you can also evaluate all the neighbors and choose the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, it could be a good idea to order the neighbors based on your estimate of the cost of evaluating the objective there. Evaluate them in this order and pick the first success as next iterate. Intuitively, you're favoring cheap candidates. In direct methods, such orderings fit it the category of &lt;em&gt;surrogate models&lt;/em&gt;, a concept that generalizes that of a &lt;em&gt;response surface model&lt;/em&gt; mentioned by Brian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an excellent implementation of a direct method (in C++): &lt;a href=&quot;http://www.gerad.ca/nomad/Project/Home.html&quot; rel=&quot;nofollow&quot;&gt;http://www.gerad.ca/nomad/Project/Home.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that seems to be giving promising results, please feel free to get back to me and I may be able to suggest other improvements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe NOMAD also has features for global optimization (such as the multi-start you're currently applying) based on the concept of &lt;em&gt;variable neighborhood search&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="324" LastActivityDate="2013-01-26T00:45:17.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5115" PostTypeId="2" ParentId="5092" CreationDate="2013-01-26T16:12:06.157" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My current implementation have two problems:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;I. I must keep the whole cost (slowness) function in RAM. This limits the size of the cost function I can use.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;II. I would like it to be even faster.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The first point is a bit ambiguous. Just computing the cost function and the traveltime for each grid cell and storing them somewhere should not be an issue. The issue is the access pattern of the fast marching method to this data, which is very cache unfriendly and prevents effective parallelization. For a straightforward implementation, the priority queue is the worst offender for the cache. There are cache optimal priority queues, but even then the memory access pattern of the updates is still very irregular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second point is probably most related to the parallelization potential of the algorithm. At least for the second point, one is probably forced to give up on fast marching methods and look at fast sweeping methods instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The observation behind the fast sweeping methods is that for the updates it is only important in which of $2^n$ &quot;general directions&quot; the characteristics point. Especially any region where this &quot;general direction&quot; is constant can be updated by a sweep. Still, there are many different ways to turn this observation into an algorithm, and which strategy will be most efficient depends a bit on whether your velocity function leads to large regions with constant &quot;general direction&quot;, or whether it creates more a sort of labyrinth.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2013-01-26T16:12:06.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5116" PostTypeId="1" CreationDate="2013-01-27T15:33:01.113" Score="1" ViewCount="382" Body="&lt;p&gt;Any source of FDM schemes for the transport equation starts by explaining that explicit central differences for the equation of the type $u_t+au_x=0$ can cause oscillations and unconditionally unstable. However, I am questioning about stability of two other schemes: Crank-Nicolson and Implicit with central differences for the space derivative. I will apply the VN stability analysis and calculate the amplification factor for both schemes. That is, I substitute $u_j^n$ with $e^{nk+ijh\omega}$. $k$ stands for the time step and $h$ for the space step. For the Crank Nicolson with central differences (if I did not make any mistake) I get the following amplification factor:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q=\frac{1-i\frac{ak}{2h}\sin(h\omega)}{1+i\frac{ak}{2h}\sin(h\omega)}$$&#xA;And for the implicit in time central difference&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q=\frac{1}{1+i\frac{ak}{2h}\sin(h\omega)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I calculate the absolute value of both: start with CN:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|Q|=|\frac{(1-i\frac{ak}{2h}\sin(h\omega))(1-i\frac{ak}{2h}\sin(h\omega))}{(1+i\frac{ak}{2h}\sin(h\omega))(1-i\frac{ak}{2h}\sin(h\omega))}|&#xA;$$&#xA;$$&#xA;=1+\left(\frac{ak}{2h}\sin(h\omega)\right)^2&#xA;$$&#xA;and for the implicit Euler:&#xA;$$|Q|=|\frac{(1-i\frac{ak}{2h}\sin(h\omega))}{(1+i\frac{ak}{2h}\sin(h\omega))(1-i\frac{ak}{2h}\sin(h\omega))}|=|\frac{1-i\frac{ak}{2h}\sin(h\omega)}{1+\left(\frac{ak}{2h}\sin(h\omega)\right)^2}|=1&#xA;$$&#xA;From this math, Crank Nicolson has the amplification factor is greater that one and is not $\leq 1+Ck$ for some $C$, does it mean it has some possible instability problems and oscillations might occur? For the implicit, the amplification factor is one in magnitude which makes me think it might be unstable. Please clarify me if I am wrong but I would like to know if these schemes are prone to oscillations at all and when it might happen. &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2013-01-28T04:47:05.770" LastActivityDate="2013-01-28T12:03:08.240" Title="amplification factor of some schemes for the transport equation" Tags="|hyperbolic-pde|" AnswerCount="1" CommentCount="5" ClosedDate="2013-01-28T11:39:07.853" ContentLicense="CC BY-SA 3.0" />
  <row Id="5117" PostTypeId="1" CreationDate="2013-01-27T19:03:44.427" Score="0" ViewCount="3471" Body="&lt;p&gt;Am I right that in FEM we can associate a local coordinate system with every node, not with every element?&lt;/p&gt;&#xA;" OwnerUserId="3636" LastActivityDate="2013-01-28T13:45:05.880" Title="Local and global coordinates in FEM" Tags="|finite-element|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5118" PostTypeId="1" AcceptedAnswerId="5119" CreationDate="2013-01-27T19:35:32.813" Score="4" ViewCount="2007" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using genetic algorithm to solve the multiple-traveling-salesman problem. It work fine when I have only one constraint (distance) for my fitness function: ie: the lower the total distance, the better are the chances of the individual to survive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now I want to add other contraints like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;time-windows: for example: salesman1 (S1) work between 9h and 17h, city1 (C1) is open between 9h and 12h and C2 is open between 11h and 13h.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;capacity: S1 have a vehicle with a capacity of 10 boxes (whatever the unit), C1 wants 2 boxes and C2 will give 3 boxes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;competence: S1 have a refrigerated vehicle, S1 needs perishable goods, and so on.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, I consider a constraint to be soft when an individual can survive a generation (have more chances) even if it the constraint is not fully satisfied (ie: distance, time-window). Where a hard constraint must be satisfied in order for the individual to survive (ie: capacity, competence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am already ok with the genetic representation and the technology I should use to solve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempt&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, this is what I came up with to implement my fitness function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each individual:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(1)&lt;/strong&gt; I will calculate/assign a score for each constraints separately:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;distance: &lt;code&gt;1 / km&lt;/code&gt; where &lt;code&gt;km&lt;/code&gt; is the total distance traveled by each salesman&lt;/li&gt;&#xA;&lt;li&gt;time-window: &lt;code&gt;1 / second&lt;/code&gt; where &lt;code&gt;second&lt;/code&gt; is the difference in second between the actual time (when the salesman vist the city) and the city time-window, for each city&lt;/li&gt;&#xA;&lt;li&gt;capacity: &lt;code&gt;1 / box&lt;/code&gt; where &lt;code&gt;box&lt;/code&gt; is the number of over/under load of the salesman vehicles&lt;/li&gt;&#xA;&lt;li&gt;competence: &lt;code&gt;+1&lt;/code&gt; for each city if the assigned salesman satisfy the required competence&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(2)&lt;/strong&gt; I will multiply each constraint score by a factor (a kind of priority, this is what makes each constraint hard or soft?)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;distance score * 20&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;time-windows score * 20&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;capacity score * 30&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;competence score * 30&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(3)&lt;/strong&gt; Then sum all those results. This sum will be the fitness cost, the higher the beter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;fist, am I in the right direction?&lt;/li&gt;&#xA;&lt;li&gt;is there recommended/general ways to implements a multi-objectives fitness functions?&lt;/li&gt;&#xA;&lt;li&gt;how can I make those constraints hard or soft?&lt;/li&gt;&#xA;&lt;li&gt;how can I prevent premature optimization or local optima (ie: all constraints are satisfied except the distance, but individuals with beter distance optimization keep dying because they perform bad in the fitness function)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Other advices are very welcome!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: sorry about my choice for the tags, I do not have the reputation to create new tags, I would have tag it at least with genetic-algorithm!&lt;/p&gt;&#xA;" OwnerUserId="3635" LastActivityDate="2013-01-27T19:53:06.213" Title="Multi-objectives fitness function with hard and soft constraints" Tags="|optimization|algorithms|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5119" PostTypeId="2" ParentId="5118" CreationDate="2013-01-27T19:53:06.213" Score="2" Body="&lt;p&gt;Using a weighted sum of your different fitness functions is a fairly standard way to handle multiple objectives.  Of course your final answer can depend greatly on the priorities you set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as making constraints hard and soft, it looks as though your current set up has them all soft.  If a certain individual does well enough on one of the objectives, it can fail to meet other requirements, which I don't think is what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would remove the hard constraints from the fitness function, and simply check each individual to make sure they meet all the requirements.  If they don't meet them, don't even evaluate the fitness function, just throw them out.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your soft constraints, one option is to change your multi-objective coefficients as you get into later generations.  For example, let $D$ be the objective value of the distance, with higher $D$ being better.  Let $T$ be the objective value for time, which you mentioned was one of your soft constraints.  You can then formulate your objective function as: $D+f(n) T$ where $n$ is the generation you are in.  If $f(n)$ is an increasing function, you will allow misses on the time in early generations if the individuals perform well enough for your distance metric, however as $n$ increases, $T$ becomes more and more important, until all your individuals meet your time requirements (where $T$ would attain some high, but constant value).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sort of setup would help avoid the local minima you mention in your 4th point.  Of course I will defer to other answerers here since my own experience in optimization of this type is small and there may be more &quot;text-book&quot; ways of handling soft constraints.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-01-27T19:53:06.213" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5121" PostTypeId="2" ParentId="5117" CreationDate="2013-01-27T22:56:55.577" Score="1" Body="&lt;p&gt;I can't think of a use case where a node would need a local coordinate system as it represents a physical location in space and doesn't have any length, area or volume associated with it and thus any coordinate system seems a moot point as there is no distance or direction to measure. The element stiffness matrices are usually computed in a local or parametric coordinate system though and can be very useful.&lt;/p&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2013-01-27T22:56:55.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5123" PostTypeId="2" ParentId="5116" CreationDate="2013-01-28T10:12:51.117" Score="4" Body="&lt;p&gt;This is not an answer.  It is an attempt to point out some of the errors in the question.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Any source of FDM schemes for the transport equation starts by explaining that explicit central differences for the equation of the type $u_t+au_x=0$ can cause oscillations and unconditionally unstable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The explicit centered difference scheme for this equation (in time and space) is often called Leapfrog, and it is one of the most widely used schemes for solving this and other wave equations.  It is stable.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I am questioning about stability of two other schemes: Crank-Nicolson and Implicit with central differences for the space derivative. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&quot;Implicit&quot; is not a scheme.  It is a (huge) set of schemes.  Crank-Nicolson is &lt;em&gt;an&lt;/em&gt; implicit scheme.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I will apply the VN stability analysis and calculate the amplification factor for both schemes. That is, I substitute $u_j^n$ with $e^{nk+ijh\omega}$. $k$ stands for the time step and $h$ for the space step. For the Crank Nicolson with central differences (if I did not make any mistake) I get the following amplification factor:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$Q=\frac{1-i\frac{ak}{2h}\sin(h\omega)}{1+i\frac{ak}{2h}\sin(h\omega)}$$&#xA;  And for the implicit in time central difference&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$Q=\frac{1}{1+i\frac{ak}{2h}\sin(h\omega)}$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Next, I calculate the absolute value of both: start with CN:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$|Q|=|\frac{(1-i\frac{ak}{2h}\sin(h\omega))(1-i\frac{ak}{2h}\sin(h\omega))}{(1+i\frac{ak}{2h}\sin(h\omega))(1-i\frac{ak}{2h}\sin(h\omega))}| $$&#xA;  $$ =1+\left(\frac{ak}{2h}\sin(h\omega)\right)^2 $$&#xA;  and for the implicit Euler:&#xA;  $$|Q|=|\frac{(1-i\frac{ak}{2h}\sin(h\omega))}{(1+i\frac{ak}{2h}\sin(h\omega))(1-i\frac{ak}{2h}\sin(h\omega))}|=|\frac{1-i\frac{ak}{2h}\sin(h\omega)}{1+\left(\frac{ak}{2h}\sin(h\omega)\right)^2}|=1&#xA;$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Much of this is wrong.  Therefore your conclusions below are wrong too.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From this math, Crank Nicolson has the amplification factor is greater that one and is not $\leq 1+Ck$ for some $C$, does it mean it has some possible instability problems and oscillations might occur? For the implicit, the amplification factor is one in magnitude which makes me think it might be unstable. Please clarify me if I am wrong but I would like to know if these schemes are prone to oscillations at all and when it might happen. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2013-01-28T12:03:08.240" LastActivityDate="2013-01-28T12:03:08.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5124" PostTypeId="2" ParentId="5117" CreationDate="2013-01-28T13:45:05.880" Score="3" Body="&lt;p&gt;Let me choose an example problem, in which we have an unknown &lt;em&gt;vector&lt;/em&gt; field, otherwise the question makes no sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering $3$-dimensional &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_elasticity&quot; rel=&quot;nofollow&quot;&gt;linear elasticity&lt;/a&gt;, nodal unknowns are displacement components $u_i$, $i=1,\dots, 3$ which can be expressed either in the global or in a nodal-local system of reference. Local systems of reference are handy when e.g. Dirichlet boundary conditions are given only on the normal component of displacement (roller support) and the domain has a general shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Element quantities, evaluated at Gauss points, are e.g. stress/strain tensor components $\sigma_{ij}$, $\epsilon_{ij}$, $i,j=1,\dots, 3$, which can be computed in an element-local or global system of reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both the nodal-local and element-local reference frames are usually user defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nodal coordinates $x_i$ are typically expressed in the global reference frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the iso-parametric formulation, stiffness matrices are integrated in an element parametric space, that depends on the nodal coordinates and cannot be chosen by the user.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-01-28T13:45:05.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5125" PostTypeId="2" ParentId="5093" CreationDate="2013-01-28T16:31:01.853" Score="5" Body="&lt;p&gt;If you don't have access to Mathematica, you could write a Levin-type (or other specialized oscillatory) method in Matlab as Pedro suggests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you use the &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/23972-chebfun&quot;&gt;chebfun&lt;/a&gt; library for Matlab? I just learned it contains an implementation of a basic Levin-type method &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/23972-chebfun/content/chebfun/examples/quad/LevinIntegrate.m&quot;&gt;here&lt;/a&gt;. The implementation is written by Olver (one of the experts in the oscillatory quadrature field). It doesn't deal with singularities, adaptive subdivision etc, but it may be just what you need to get started.&lt;/p&gt;&#xA;" OwnerUserId="303" LastActivityDate="2013-01-28T16:31:01.853" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5127" PostTypeId="1" CreationDate="2013-01-28T23:51:43.757" Score="2" ViewCount="237" Body="&lt;p&gt;I am looking at building/buying myself a workstation for scientific computing. I will be doing molecular dynamics simulations that are memory intensive (data on a large number of particles and a million time steps or more) and require multiple instances of the same code running (to generate an ensemble). I am new to computing but my guess is I will be using C++ and learn to utilize the best memory handling practices. In terms of ensemble averaging, it appears to me I could either just run the code N times or maybe run N instances of the codes in simultaneously on N processors. At this point I don't have access to HPC resources and I am only trying to get a simple desktop (maybe 2 or 4 cores) but I want to learn to make this code parallel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point I know very little about the code itself to make a more specific assessment of my needs. However, my question is what might be the very essential features I should keep in mind while buying a workstation. For example, things to keep in mind if eventually I want to transition to HPC, things to keep in mind if some amount of parallel processing needs to be done (or just having 2 or more cores in enough) etc.&lt;/p&gt;&#xA;" OwnerUserId="3570" LastEditorUserId="3570" LastEditDate="2013-01-29T18:57:41.163" LastActivityDate="2013-08-27T12:14:12.613" Title="Determining Computing Requirements" Tags="|parallel-computing|gpu|hpc|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5128" PostTypeId="1" AcceptedAnswerId="5132" CreationDate="2013-01-29T02:23:11.827" Score="3" ViewCount="1922" Body="&lt;p&gt;I will try one more time being more detailed and careful. Consider the transport equation of the form $$u_t+au_x=0, t\in[0,T],x\in \mathbb{R}, a&amp;gt;0$$ and initial condition $u(0,x)=u_0(x)$. I would like to establish stability of Crank-Nicolson scheme for that equation by computing the amplification factor. Consider the following iteration of CN:&#xA;$$\frac{u^{n+1}_j-u^{n}_j}{k}+\frac{1}{2}a\frac{u^{n+1}_{j+1}-u^{n+1}_{j-1}}{2h}+\frac{1}{2}a\frac{u^{n}_{j+1}-u^{n}_{j-1}}{2h}=0$$ then I substitute the form $u^n_j=Q^ne^{ijhw}$ in the above equation:&#xA;$$\frac{(Q-1)e^{ijhw}Q^n}{k}+\frac{1}{2}ae^{ijhw}Q^{n+1}\frac{e^{ihw}-e^{-ihw}}{2h}+\frac{1}{2}ae^{ijhw}Q^{n}\frac{e^{ihw}-e^{-ihw}}{2h}=0$$ &#xA;from where with we get:&#xA;$$Q=\frac{1-i\frac{ak}{2h}\sin(hw)}{1+i\frac{ak}{2h}\sin(hw)}$$&#xA;this is a complex number of the form $\frac{1-iC}{1+iC}$, thus I can calculate the absolute value as follows:&#xA;$$|Q|=|\frac{(1-iC)(1-iC)}{(1+iC)(1-iC)}|=|\frac{1-C^2-2iC}{1+C^2}|=\frac{(1-C^2)^2+4C^2}{1+C^2}=\frac{(1+C^2)^2}{(1+C^2)^2}=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from where I deduce that for Crank Nicolson method the amplification factor for that equation has the magnitude of:&#xA;$$|Q|=1$$&#xA;If there is a mistake in the computations please let me know and I proceed from there. But if this is the amplification factor indeed I would like to know if I can deduce stability/conditional stability/instability from it? Is there a possibility of oscillations of any form? It does satisfy the condition less or equal than one, therefore I can use it and will not have problems?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2013-01-29T13:25:22.690" LastActivityDate="2013-01-29T19:14:47.607" Title="amplification factor for the Crank Nicolson scheme for the advection equation" Tags="|hyperbolic-pde|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5130" PostTypeId="2" ParentId="5095" CreationDate="2013-01-29T04:48:08.657" Score="7" Body="&lt;p&gt;You are exactly right: The norm is defined in such a way that the discrete (vector) norm equals (or at least approximates) the continuous norm of a corresponding function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you have non-uniform meshes, the form you give (with the $h_i$ inside the sum) is correct and frequently used in the analysis of non-uniform meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in 2d, the correct formula would contain a factor of $h^2$ and in 3d of $h^3$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-01-29T04:48:08.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5132" PostTypeId="2" ParentId="5128" CreationDate="2013-01-29T19:14:47.607" Score="5" Body="&lt;p&gt;If the amplification factor is $\le 1$, then your scheme is stable; in your case it is exactly one, so, for this PDE, you won't have unbounded growth of errors. The &lt;a href=&quot;http://en.wikipedia.org/wiki/Lax_equivalence_theorem&quot;&gt;Lax theorem&lt;/a&gt; then guarantees convergence of the method. In this narrow sense, the Crank-Nicholson method will work for you. (Have a look at &lt;a href=&quot;http://amzn.com/0898716292&quot;&gt;Leveque's book&lt;/a&gt; if you want a good reference.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, your question of whether or not you will have problems is a deeper one and depends on your objectives. While the Crank Nicholson method does not introduce &lt;em&gt;amplitude&lt;/em&gt; errors, it will incur &lt;em&gt;phase&lt;/em&gt; errors. By this, it's meant that different Fourier modes will have different group velocities -- this contrasts with the true PDE, where all modes travel with velocity $a$. Depending on what your application is, the dispersion associated with CN is unacceptable, convergent or not, and you'd choose a different approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, even for methods that do converge, there is a trade-off between numerical diffusion and numerical dispersion. In some problems, using staggered grids is a great way to wriggle out of this bargain. If you want to know more, I recommend Dale Durran's &lt;a href=&quot;http://amzn.com/1441964118&quot;&gt;Numerical Methods for Fluid Dynamics&lt;/a&gt;; he has a great treatment of this subject.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-01-29T19:14:47.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5133" PostTypeId="1" CreationDate="2013-01-29T20:04:08.437" Score="2" ViewCount="389" Body="&lt;p&gt;Is there a piece of software that can easily take a representation of a biochemical network from a SBML file and translate it into a Petri Net? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I'm looking for something that can give me the Petri Net in the matrix representation (one matrix each for Pre, Post, etc.). Something that translates from SBML to a Petri Net markup language (I think that there's one called PNML) would still leave me with my original problem of having to parse a markup language.&lt;/p&gt;&#xA;" OwnerUserId="34" LastActivityDate="2013-04-17T00:34:58.513" Title="Translating from SBML to Petri Net (matrix representation)" Tags="|stochastic|data-management|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5134" PostTypeId="1" AcceptedAnswerId="5145" CreationDate="2013-01-29T23:14:24.533" Score="6" ViewCount="3722" Body="&lt;p&gt;I'm not sure if I did something wrong or if I just didn't understand the concept of an optimized BLAS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a FEM engineer trying to optimize my setup on a small cluster computer (six nodes). I'm building everything on open source tools. My aim is to use a robust setup which allows me to manage large mechanical simulations. &#xA;After finally compiling my programs using MPI I'm reading the using an optimized BLAS will help me further pushing down calculation time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I installed &lt;strong&gt;OpenBlas&lt;/strong&gt; and built &lt;strong&gt;Lapack 3.4.2&lt;/strong&gt;. The results of the lapack benchmarks were surprising. They are fastest with the system (unoptimzed) BLAS and the run time increases with increasing number of threads specified (&lt;code&gt;export OPENBLAS_NUM_THREADS=i&lt;/code&gt;). The same is true for &lt;em&gt;scalapack&lt;/em&gt;. Furthermore, &lt;code&gt;i&amp;gt;16&lt;/code&gt; doesn't seem to utilize more than &lt;code&gt;16&lt;/code&gt; threads, although I have &lt;code&gt;32&lt;/code&gt; available on this machine. That's ok, because I want some to run a couple of MPI processes (4?) on one node anyways, but I find it suspicious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a beginner and don't know where to start digging from here. Can someone give me a hint? I have a pastebin of the &lt;code&gt;i=32&lt;/code&gt; log &lt;a href=&quot;http://pastebin.com/BQATuymz&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;OpenBLAS build complete.&#xA;OS               ... Linux&#xA;Architecture     ... x86_64 &#xA;BINARY           ... 64bit&#xA;C compiler       ... GCC  (command line : gcc)&#xA;Fortran compiler ... GFORTRAN  (command line : gfortran)&#xA;Library Name     ... libopenblas_sandybridgep-r0.2.5.a (Multi threaded; Max num-threads is 32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Lapack 3.4.2 make.inc (changes from default)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; BLASLIB = /path/to/libopenblas.a -lpthread&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;-- &lt;strong&gt;EDIT:&lt;/strong&gt; (more information because of @Aron Ahmadia's comments)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pastebin.com/ATaUNti4&quot; rel=&quot;nofollow&quot;&gt;system BLAS log&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pastebin.com/fbNYLV7S&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;i=1&lt;/code&gt; log&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pastebin.com/BQATuymz&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;i=32&lt;/code&gt; log&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;CentOS release 6.3 (Final)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;blas-devel-3.2.1-4.el6.x86_64&lt;/code&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;gcc&lt;/code&gt; version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) (same for &lt;code&gt;gfortran&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;MemTotal: 198202172 kB&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep &quot;Xeon(R) CPU E5-2690 0 @ 2.90GHz&quot; /proc/cpuinfo | wc -l &#xA;32. &#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="86" LastEditorUserId="86" LastEditDate="2013-01-30T00:03:23.187" LastActivityDate="2013-01-30T22:52:59.027" Title="Threaded OpenBlas benchmark slower than system blas?" Tags="|parallel-computing|lapack|blas|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="5135" PostTypeId="1" AcceptedAnswerId="5137" CreationDate="2013-01-30T04:05:50.037" Score="10" ViewCount="1435" Body="&lt;p&gt;So I've been debating whether or not I should bother learning Python. From speaking with my professors, Matlab seems to be the common language used in applied mathematics/computational science as far as &lt;em&gt;academia&lt;/em&gt; is concerned; while in &lt;em&gt;industry&lt;/em&gt;, my professors (esp. those who have worked in industry) have said learning c++ is the safest route.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to hear from those of you, in both academia and industry, as to whether I should even bother with Python, or just get really good at what I know (MATLAB and C++) for the time being.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm currently an undergrad in my last year, studying maths with a specialization in computation. I'd like to pursue grad school and stay in research (I've never seen myself enjoying teaching) or work in a lab. Both of those are ideal. As to what areas of research, probably something along the lines of numerical analysis or probability. In case plan A doesn't work out, I would be open to work in industry, as long as preparing myself for industry doesn't take away too much time from school. So, I figured I should learn the languages that are common in industry just as a backup. But this is also why I am conflicted. I can't study every language or prepare myself for every possibility since that would take way too much time.&lt;/p&gt;&#xA;" OwnerUserId="3493" LastEditorUserId="20688" LastEditDate="2022-12-02T23:04:48.863" LastActivityDate="2022-12-02T23:04:48.863" Title="Where does Python fit into the picture within computational science field?" Tags="|python|matlab|c++|career-development|" AnswerCount="6" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5136" PostTypeId="2" ParentId="5135" CreationDate="2013-01-30T05:45:48.007" Score="5" Body="&lt;p&gt;Python may be replacement for both c++ and Matlab. It is well spread in both academia and industry. In industry it is sometimes used as a glue for lower level languages, mostly c/c++.&#xA;Mathematica is completely another story. Its main advantage is where all other mentioned (c/c++; Matlab; Python) are not good: in symbolic computations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, all four are completely different: c++ is old and stable relatively low level programming language; Python is new and evolving high level computer language; Matlab is a numerical computational environment with a strong accent on vector algebra (though it is able to do more or less everything); Mathematica is a computer algebra system with a strong accent on symbolic computations (the same remark as with matlab). Thus, they are not competitors.&lt;/p&gt;&#xA;" OwnerUserId="949" LastActivityDate="2013-01-30T05:45:48.007" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5137" PostTypeId="2" ParentId="5135" CreationDate="2013-01-30T06:35:01.173" Score="15" Body="&lt;p&gt;A difficulty with any of these types of questions is that the answer is highly community-dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer some of your questions in haphazard order:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB is used a lot both in academia and in industry. One of the reasons it's used quite a bit in industry is because it is taught in academia. I know for a fact that MATLAB is used at Lincoln Laboratory and in DuPont's research and development divisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are software packages written in Python that are good at symbolic computation, such as sympy and SAGE. Depending on your particular interests, feature requirements, and personal preferences, Mathematica (or Maple, or other computer algebra systems) may be superior to these packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB has a Symbolic Math Toolbox that can be used for some symbolic computations, but its symbolic manipulation capabilities, in my experience, are weaker than Mathematica and Python. Some symbolic manipulation could theoretically be done in C++, but it is unwieldy. MATLAB is also not a good general purpose language. It does linear algebra and numerical mathematics well, but it does not have good input/output capabilities. It does not have good parallel capabilities (even though there are variants like parallel MATLAB, MATLAB Star-P, and the Parallel Computing Toolbox) compared to C++ or Python. Even its graphics capabilities could use some work. MATLAB is also expensive unless you're affiliated with an institution that has a license. Each toolbox is expensive to purchase, and generally costs on the order of hundreds to thousands of dollars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica does numerical computation in addition to symbolic computation. I haven't seen people use it for numerical computation as much as I have seen people use Python and MATLAB for numerical work. It too has parallel capabilities, but won't scale to large supercomputers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python is a good general purpose language that is regarded as easy to learn and usable. It is used on large supercomputers (see, for instance, PyClaw, petsc4py, mpi4py, and others), and scales well. It also has highly regarded numerical packages (such as NumPy and SciPy); a large, active community; good input/output processing capabilities; and good graphics libraries, along with a large repository of libraries (check out PyPI). It is free, compared to the proprietary packages mentioned above. You can find most of the functionality of MATLAB or Mathematica in freely available Python packages. The main disadvantage of Python is that it tends to be slower than compiled languages like C++, although this disadvantage is diminishing with the continued development of Cython, Numba, and PyPy; it can also be mitigated by replacing slower Python code with C (or C++, or Fortran) code and appropriately written Python wrappers. Being interpreted, many people report higher productivity with Python than compiled languages. It is quite popular, and probably worth learning if you have time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++ is a complicated language, and its use in computational science is controversial. Its large feature set can make it easy to write software that is difficult to maintain and takes forever to compile. However, used judiciously, features such as templating and operator overloading can be employed to great effect, as it has been in projects like deal.II, Blaze, and Elemental (among others). C++ has a steep learning curve when it comes to its advanced features, and I've heard anecdotal reports of people taking years to feel like they've learned the full language. Nevertheless, it is also a popular language, despite the usability concerns and the complicated feature set. It is probably worth learning, if only to make yourself more employable; its main competitors in computational science are Fortran and C, which are also worth learning. C and C++ are separate languages, and I advise you to learn them separately, however often you see people write &quot;C/C++&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever you decide to learn will be based on what you actually need. Sure, it's nice to learn both Python and C++, but given time and resource constraints, you're probably only going to learn what you'll actually need to use, and that depends on the community you work in.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-01-30T06:35:01.173" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5138" PostTypeId="1" CreationDate="2013-01-30T07:28:55.390" Score="3" ViewCount="80" Body="&lt;p&gt;We currently use PORTA software to find the list of facet-defining inequalities (FDI) for polytopes that we work with.  For certain polytopes, PORTA works fine.  But because it is a serial algorithm (uses Fourier-Motzkin) we sometimes encounter cases where the analysis simply takes too long and we have to force the program to stop.  We are wondering if there exists a parallel algorithm that would allow us to complete these computations faster? Ideally we would prefer to have exact solutions like PORTA gives; but in lieu of such software programs we would be open to trying a method that may not yield exact results.  (And if it matters:  our input coordinates are almost always discrete, either 0 or 1.)&lt;/p&gt;&#xA;" OwnerUserId="3661" OwnerDisplayName="cez" LastActivityDate="2013-01-30T08:22:24.440" Title="Parallel algorithm to use in place of PORTA?" Tags="|combinatorics|algorithms|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5139" PostTypeId="2" ParentId="661" CreationDate="2013-01-30T08:51:11.237" Score="2" Body="&lt;p&gt;Take a look at &lt;a href=&quot;http://www.runmycode.org&quot; rel=&quot;nofollow&quot;&gt;http://www.runmycode.org&lt;/a&gt;. They host companion sites for code associated with research papers. If the code is R, Matlab, or a few others, it will actually run the code for you. I haven't tried it yet, but I intend to. I think David Donoho and his collaborators use it. &lt;/p&gt;&#xA;" OwnerUserId="3652" LastActivityDate="2013-01-30T08:51:11.237" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5141" PostTypeId="1" AcceptedAnswerId="5142" CreationDate="2013-01-30T11:34:37.620" Score="4" ViewCount="561" Body="&lt;p&gt;I have a multi-variable convex continuous function which is not differentiable. I am interested to know about different numerical techniques, possibly also references to them, used for this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read the following only if you want to know the function I want to minimize (this is a concave function, I want to maximize it, which is equivalent to minimizing its negative) .&#xA;\begin{align}&#xA;f(t_1,t_2)=\min_{u^Tu=1,u\in \mathbb{R}^{N}}u^T(A_0+t_1A_1+t_2A_2)u&#xA;\end{align} &#xA;$A_0,A_1,A_2$ are all $N \times N$ real symmetric matrices. Actually $f(t_1,t_2)$ is the lowest eigenvalue of matrix $A_0+t_1A_1+t_2A_2$ for a given $t_1,t_2$. I know this can be solved by semi-definite programming.   &lt;/p&gt;&#xA;" OwnerUserId="3255" LastEditorUserId="9" LastEditDate="2013-01-30T13:08:36.183" LastActivityDate="2013-01-30T16:35:27.070" Title="Numerical Methods for minimizing a Non-Differentiable Convex Function of Several Variables" Tags="|linear-algebra|optimization|eigensystem|convex-optimization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5142" PostTypeId="2" ParentId="5141" CreationDate="2013-01-30T13:22:28.223" Score="3" Body="&lt;p&gt;The problem that you've described has very specialized structure.  These kinds of eigenvalue optimization problems have been widely studied, so you'd do well to look at the literature on the subject and go from there.  You don't really want to look at completely general purpose non differentiable convex optimization solvers that won't take advantage of the problem structure you've identified.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't told us anything about how big $N$ is.  This can be very important, since the other size parameter here is the length of the $t$ vector, which is absolutely tiny for your particular eigenvalue problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You've already mentioned the SDP formulation of this problem.  If $N$ is reasonably small (say less than $N=10000$), then off the shelf semidefinite programming tools (SeDuMi, SDPA, CSDP, etc.) using the primal-dual interior point method should be easy to apply to this problem.  In terms of your time, this is probably the quickest way to solve the problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if $N$ is really huge, you'll want to look at methods that are more specialized for eigenvalue optimization problems.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2013-01-30T16:35:27.070" LastActivityDate="2013-01-30T16:35:27.070" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5143" PostTypeId="1" AcceptedAnswerId="5144" CreationDate="2013-01-30T19:36:56.483" Score="5" ViewCount="280" Body="&lt;p&gt;I've got a second order nonlinear ODE (nothing fancy), but the BC are a little odd to me:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$y'(0) = 0$ &lt;/li&gt;&#xA;&lt;li&gt;$y \rightarrow y_a$ as $x \rightarrow \infty$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What's a good numerical method for solving this? Thinking of MATLAB, but I'm open to libraries in C/C++ if there are any.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3658" LastEditorUserId="81" LastEditDate="2013-01-31T00:33:55.550" LastActivityDate="2013-01-31T20:44:13.980" Title="which numerical method for ode with mixed BCs" Tags="|ode|boundary-conditions|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5144" PostTypeId="2" ParentId="5143" CreationDate="2013-01-30T20:39:16.827" Score="4" Body="&lt;p&gt;A common introductory way to solve a BVP like this is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Shooting_method&quot; rel=&quot;nofollow&quot;&gt;Shooting Method&lt;/a&gt;.  As you may know, solving the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y''(t) = f(t,y,y')$ with $y(x_0) = y_0$ and $y'(x_0) = y'_0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;is fairly straight forward using a technique such as Euler's method or any variety of RK methods.  What time-stepping algorithm you choose depends on your problem's stiffness and stability.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't have exactly this problem however.  The shooting method involves solving the above equation over and over with varying $y_0$ until your $y_\infty$ approaches $y_a$.  This can be done by wrapping a root-finding method around your time-stepping algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem also has the added complication of the right BC being defined in the far field, rather than some point.  You obviously can't advance your solution out to infinity, so you must choose some arbitrary right boundary $x_{large}$ and solve it twice with the BC's:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y(x_{large}) = y_a$ and $y(2 x_{large}) = y_a$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your two solutions agree, then you know $x_{large}$ was choosen big enough that it was like having the boundary at $\infty$.  If not, you must resolve it with an even higher boundary.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-01-30T20:39:16.827" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5145" PostTypeId="2" ParentId="5134" CreationDate="2013-01-30T20:53:48.800" Score="5" Body="&lt;p&gt;OpenBLAS is designed as a library to perform effectively for the most common computationally challenging problems that arise in scientific computing, linear algebra problems that take minutes or longer to solve.  As @Stefano and @JedBrown hint in the comments, you are not necessarily likely to depend heavily on the performance of your BLAS library for the performance of your FEM solver, and it is more important to use a freely available robust software framework such as &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt;, (potentially with &lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;nofollow&quot;&gt;deal.II&lt;/a&gt; or &lt;a href=&quot;http://fenicsproject.org/&quot; rel=&quot;nofollow&quot;&gt;FEniCS&lt;/a&gt; as a friendlier FEM interface above), to develop your software.  I can think of very few situations where I would not recommend using a high-level FEM and linear algebra framework to develop and execute your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, you are not seeing a difference in performance because you are using the validation tests for LAPACK, which solves considerably smaller problems.  The run time of launching the processes from the shell and loading them dynamically are confounding your performance measurements.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As suggested in the comments, you will notice a sharp relative improvement in performance by measuring larger problems.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2013-01-30T22:52:59.027" LastActivityDate="2013-01-30T22:52:59.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5146" PostTypeId="2" ParentId="5135" CreationDate="2013-01-30T22:32:12.737" Score="11" Body="&lt;p&gt;As Misha and Geoff Oxberry pointed out, Mathematica really has a different focus (just because you can pound in a nail with a screwdriver doesn't mean you should). So I take your question as being &amp;quot;If I know Matlab, why should I learn Python?&amp;quot;&#xA;[Edit: and so, apparently, did you.]&lt;/p&gt;&#xA;&lt;p&gt;For all intents and purposes, Matlab is the English of scientific computing -- with all the positive and negative connotations this analogy entails. One specific good point is that Matlab code is likely to be useful (i.e., executable as well as  understandable) to more people than code in any other language is. (This is the main reason I provide Matlab codes for all my algorithms.) Also, the Matlab desktop can be really useful while prototyping, especially the ability to run bits of code (cells) directly from the editor, as well as the built-in profiler.&lt;/p&gt;&#xA;&lt;p&gt;That being said, if you do want to learn another high-level language for fun and profit, you could do worse than Python. Some reasons in addition to what Geoff listed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It is much easier to interface with external programs and libraries in Python. No more mex files!&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you're away from your desktop, it's much easier to get Python+NumPy/SciPy up and running than to get access to a Matlab license.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The main reason Matlab is faster than NumPy is that it bundles optimized vendor libraries for linear algebra (MKL, ACML). It is possible (&lt;a href=&quot;https://www.intel.com/content/www/us/en/developer/articles/technical/numpyscipy-with-intel-mkl.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;if a bit tedious&lt;/a&gt;) to build your own NumPy and link it against the same libraries to get nearly the same performance (and multithreading) as Matlab for linear algebra, plus the better performance of Python for everything else. (Although that of course requires a license even for academic use, thus negating the free software bonus, it is still an interesting option to have the same code working on an employer-sponsored accelerated install in the office as well as an off-the shelf install on your home machine or notebook.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;While Matlab's toolboxes are one of its selling points, there are a few areas where Python is way ahead; in particular, &lt;a href=&quot;https://www.sympy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt; and &lt;a href=&quot;https://fenicsproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEniCS&lt;/a&gt; beat the Symbolic and PDE toolboxes by a wide margin.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't forget the fun part (semantic whitespace and &lt;a href=&quot;https://web.archive.org/web/20141221053527/http://python.net:80/%7Emwh/hacks/objectthink.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;name binding&lt;/a&gt; notwithstanding): I've seen several colleagues getting bitten by the Python bug, and there is really something strangely satisfying in writing your algorithm in Python that isn't there in Matlab (although that might just be the joy of learning a foreign language) :)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(If you do start with NumPy, you might find &lt;a href=&quot;https://numpy.org/devdocs/user/numpy-for-matlab-users.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; helpful.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="22284" LastEditDate="2022-12-02T23:01:13.557" LastActivityDate="2022-12-02T23:01:13.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5147" PostTypeId="2" ParentId="5143" CreationDate="2013-01-30T22:53:48.623" Score="3" Body="&lt;p&gt;Godric's solution is a good one, but sometimes there's a better way of doing the shooting method for asymptotic boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically the idea is to approximate the ODE with another ODE which captures the behavior of the original in the large $x$ limit.  For example, terms like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(x-\alpha\right)\frac{d \phi}{dx}$$ or $$\frac{\beta}{x^2}\phi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;would become $$x\frac{d \phi}{dx}$$ and $$0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the resulting differential equation is exactly solvable, you can pick the exact solution that has the asymptotic behavior you require and some $x$ large enough that the asymptotic approximation is good, and then use the shooting method to find the matching point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like this technique also works for oscillatory asymptotic boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you find yourself doing these kinds of problems a lot, you might want to check out &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387989315&quot; rel=&quot;nofollow&quot;&gt;Bender and Orszag's&lt;/a&gt; eldrich tome of asymptotics lore.&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2013-01-31T20:44:13.980" LastActivityDate="2013-01-31T20:44:13.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5148" PostTypeId="1" CreationDate="2013-01-31T00:52:32.903" Score="3" ViewCount="91" Body="&lt;p&gt;I am trying to represent the result of a non-linear function in a small basis, given another small basis that does a good job a representing the argument of the function.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, there is a non-linear map,&#xA;$$ f: \left&amp;lt; r \middle| \rho \middle| r \right&amp;gt; \longrightarrow \left&amp;lt; r \middle| V \middle| r \right&amp;gt; $$&#xA;or alternatively $f[\rho(r)] = V(r)$, where $\rho$ and $V$ are Hermitian and $V$ is diagonal in real-space.  I already have a great (small) basis $u_i$ that represents $\rho$ perfectly:&#xA;$$ \sum_i \left&amp;lt; r \middle| \rho \middle| r \right&amp;gt; \left&amp;lt;r\middle|u_i\right&amp;gt; \left&amp;lt;u_i\middle|r\right&amp;gt; = \left&amp;lt; r \middle| \rho \middle| r \right&amp;gt; $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With access to $f$, is there anything I can do to the basis $u_i$ to make it represent $V$ more completely?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ generally has terms that look like $\rho(r)^{1/3}$, plus some linear terms, but can be more general.  I don't need to evaluate $f$ in this basis; I just want to be able to transform $V$ into it after it is computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scientific back-story: $\rho$ is an electron density and $V$ is the local potential.  Real-space is really big, so I'm trying to get out of it as quickly as possible.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2013-01-31T05:58:23.390" LastActivityDate="2013-01-31T21:24:33.183" Title="How to pick a basis for the result of a non-linear function given a basis for its argument" Tags="|linear-algebra|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5150" PostTypeId="2" ParentId="2267" CreationDate="2013-01-31T06:55:03.093" Score="4" Body="&lt;p&gt;You can get a lot of help from &lt;a href=&quot;http://www.cfd-online.com/&quot; rel=&quot;nofollow&quot;&gt;http://www.cfd-online.com/&lt;/a&gt;. It's a good forum with a lot of active members working on OpenFoam.&lt;/p&gt;&#xA;" OwnerUserId="3662" LastEditorUserId="1804" LastEditDate="2013-01-31T14:42:05.473" LastActivityDate="2013-01-31T14:42:05.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5151" PostTypeId="1" CreationDate="2013-01-31T09:34:47.713" Score="5" ViewCount="235" Body="&lt;p&gt;I don't think the title is very accurate , sorry for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I simulate bodies in space using two timestep:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the TIMESTEP is the &lt;em&gt;Δt&lt;/em&gt; wich I use to make the calculation&#xA;and  XTIME is the number of times I make the calculation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want the each visible step to be 7 days , the Δt will be 86400 (seconds) and XTIME 7 , so it'll be calculate seven days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made that because if  $Δt = 86400*7$ there is a lot of aberration (especially on moon orbit).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I'll use less than 20 planets each time , calculate 7 times Δt is not really a problem , but more could be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is the problem , with $Δt = 86400$ I still have aberation when planet come too close to the massive mass (the sun), the result is the planet going away at the speed of light ! It's because the Δt is too large to calculate the &quot;counter force&quot; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I heard of leapfrog but I must be stupid , I didn't succeed to adapt it to my code (the result was worst than anything, like the moon going away from earth) and i'm don't sure this will solve my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I precise that I want something that look believable but not realistic , so if there is not &quot;not-eating-cpu&quot; method , I'll certainly going to cheat a little with those cases, but I prefer not to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask this question here because i think it's more computing than physics since I already got the physics&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: maybe a lead ? something like if (distanceToOldPosition &gt; FIXED_VALUE) calculate more times&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit2: precision on calculations asked on comment:&#xA;\begin{align}&#xA;F = \frac{GM_{1} M_{2}}{r}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the acceleration &#xA;\begin{align}&#xA;a = \frac{F}{M}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then the linear vector&#xA;\begin{align}&#xA;v_{t+1} = v_{t} + aΔt&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and finally the new position&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;x_{t+1} = x_{t} + v_{t+1}Δt&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerUserId="3651" LastEditorUserId="127" LastEditDate="2013-05-15T15:32:03.070" LastActivityDate="2013-05-15T15:32:03.070" Title="Is there a way to reduce aberration in computations of planets' trajectories?" Tags="|algorithms|ode|simulation|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="5152" PostTypeId="2" ParentId="5148" CreationDate="2013-01-31T10:15:49.643" Score="1" Body="&lt;p&gt;This sounds pretty close to what people doing reduced order modeling for optimization of nonlinear differential equations are working on. They have a reduced basis for expressing (and computing) an approximation $\tilde y$ of the solution $y$, but evaluation of the nonlinearities $f(\tilde y)$ in the equation still have to be performed in the full space. To circumvent this, they construct a reduced model $\widetilde{f(\tilde y)}$ from a basis $\{f(\tilde y_1),f(\tilde y_2),\dots\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, some possible keywords are &quot;nonlinear proper orthogonal decomposition (POD)&quot; and &quot;(discrete) empirical interpolation methods ((D)EIM)&quot; (the latter is the topic of &lt;a href=&quot;http://www.caam.rice.edu/tech_reports/2011/TR11-08.pdf&quot; rel=&quot;nofollow&quot;&gt;Saifon Chaturantabut's thesis&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-01-31T21:24:33.183" LastActivityDate="2013-01-31T21:24:33.183" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5153" PostTypeId="2" ParentId="5143" CreationDate="2013-01-31T13:17:06.093" Score="3" Body="&lt;p&gt;From what little I know about it, the boundary element method (&lt;a href=&quot;http://www.boundary-element-method.com/&quot; rel=&quot;nofollow&quot;&gt;BEM&lt;/a&gt;) can handle asymptotic boundary conditions. It seems to be popular for acoustics and electromagnetics problems due to its simple handling of asymptotic boundary conditions. For a 1D problem it may work out especially well.&lt;/p&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2013-01-31T13:17:06.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5154" PostTypeId="2" ParentId="5135" CreationDate="2013-01-31T15:58:58.493" Score="4" Body="&lt;p&gt;I know your question is about the use of python, but you said that you were interested in &quot;numerical analysis or probability.&quot; I don't know if you have considered &lt;a href=&quot;http://en.wikipedia.org/wiki/R_%28programming_language%29&quot; rel=&quot;nofollow&quot;&gt;R&lt;/a&gt;, but it is designed specifically for this kind of work. R is a very powerful language for probability and statistics, and has a very large and active user base of mathematicians and scientists. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;R is different from Matlab in the sense that it is open-source, has a statistics focus, and makes some very impressive plots (see &lt;a href=&quot;http://docs.ggplot2.org/current/&quot; rel=&quot;nofollow&quot;&gt;ggplot2&lt;/a&gt;). You can do nearly everything in R that you can do in Matlab, but my favorite aspect is the user contribution. Most of the contributed libraries are written by scientists and published in a statistical journal. They also have extremely well written guides (called references and vignettes). My new favorite library gives &lt;a href=&quot;http://brainarray.mbni.med.umich.edu/Brainarray/rgpgpu/&quot; rel=&quot;nofollow&quot;&gt;CUDA support&lt;/a&gt; built on the &lt;a href=&quot;http://www.culatools.com/&quot; rel=&quot;nofollow&quot;&gt;CULA&lt;/a&gt; libraries (free for academic use). There exists a prodigious amount of methods for probability theory as well (see &lt;a href=&quot;http://cran.r-project.org/web/views/Distributions.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, R is definitely designed specifically for your line of work, so check it out and consider adding it to your toolkit :) Remember, your can execute R scripts within Python, and Python scripts within R, and easily capture the outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best of luck!&lt;/p&gt;&#xA;" OwnerUserId="101" LastActivityDate="2013-01-31T15:58:58.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5155" PostTypeId="2" ParentId="5133" CreationDate="2013-01-31T21:16:39.457" Score="2" Body="&lt;p&gt;We are developing a tool, iBioSim, which may have some of the functionality that you are looking for.  We have support for developing Petri net models which are then saved as SBML using our model editor.  Essentially, places are specially annotated parameters while transitions are specially annotated events.  We can also translate these SBML models of Petri nets into a Petri net file format though it is one of our own design and not a standard.  It does not, however, translate an arbitrary SBML file into a Petri net.  We could likely add such support, but I'm not entirely sure what the resulting model could be used for.  Namely, it would encode each species as a place and reaction as a transition, but it would be an infinite state (or very large) system in most practical cases.  We do, however, have a method that translates our genetic circuit models expressed in SBML into a finite state stochastic Petri net through a series of abstractions (see Madsen et al, CIBCB 2012).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I knew more about the purpose of your translation, I could perhaps provide a better answer.&lt;/p&gt;&#xA;" OwnerUserId="3665" LastActivityDate="2013-01-31T21:16:39.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5156" PostTypeId="1" AcceptedAnswerId="5157" CreationDate="2013-01-31T21:38:22.287" Score="5" ViewCount="376" Body="&lt;p&gt;I have not implemented these elements before, but I like their reduced cardinality compared to (e.g.) a tensor product of Lagrange interpolants, which is very &quot;overcomplete&quot; (especially for orders&gt;2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think they are not complete. My test program takes a quad and builds the &lt;span class=&quot;math-container&quot;&gt;$14 \times 14$&lt;/span&gt; mass (projection?) matrix consisting the 8 serendipity functions + the 6 functions that comprise the second order polynomial basis (&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$xx$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$xy$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$yy$&lt;/span&gt;). Matlab reports that matrix has a rank of 9 (not 8), indicating there is a function inside the polynomial basis that the serendipity functions fail to capture. The thing is, the 9'th eigenvalue is like &lt;span class=&quot;math-container&quot;&gt;$10^5$&lt;/span&gt; down from the others, the serendipity basis &quot;almost&quot; captures all the polynomials, just barely doesn't. When I deliberately make the quad rectangular, the rank is reported as 8, the 9th eigenvalue vanishes to machine precision. That 9th eigenvalue grows as the quad is skewed more and more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suspicion has grown further upon reexamination of my reference material more closely (Huebner's &quot;The Finite Element Method for Engineers&quot;, pp 167-171) - the serendipity functions are in the section entitled &quot;Rectangular Elements&quot; (e.g. orthogonal edges), and there's no mention of them in the &quot;Arbitrary Quadrilateral Elements&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been googling and regoogling but thought somebody here might know off the top of their head, being more familiar with the space (seems like its more common in structures - I'm more an electromagnetics guy). &lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="20688" LastEditDate="2019-06-22T19:05:34.777" LastActivityDate="2019-06-22T19:05:34.777" Title="Are serendipity elements still polynomially complete when the quadrilateral is skewed?" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5157" PostTypeId="2" ParentId="5156" CreationDate="2013-01-31T23:04:23.967" Score="7" Body="&lt;p&gt;The serendipity spaces are indeed deficient for non-affine meshes. An excellent reference is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://umn.edu/~arnold/papers/quadapprox.pdf&quot;&gt;Arnold, Boffi, and Falk, &lt;strong&gt;Approximation by Quadrilateral Finite Elements&lt;/strong&gt;, 2002&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm not a fan of serendipity spaces. The number of degrees of freedom saved is not very high (especially in 3D) and their effect is easily removed in the first step of a solver, so they don't actually cost much. For example, a sparse direct solver essentially does static condensation of the internal nodes in its first step. The overall cost does not depend on those degrees of freedom because it does not change the minimal vertex separators. Those rows of the matrix also have fewer degrees of freedom than the interface nodes. Also, leaving the full tensor product enables performance optimizations in local computations.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-01-31T23:04:23.967" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5158" PostTypeId="2" ParentId="5135" CreationDate="2013-02-01T02:08:49.083" Score="3" Body="&lt;p&gt;Others have already commented at length and in more depth than I will attempt here. However, I would like to highlight one point once more: it depends on your community. For example, if you are working on an electrical engineering project, chances are that you'll be using matlab; the same may be true if you are a numerical analyst wanting to verify your convergence rate estimate on a simple model problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you're into scientific computing proper (e.g., simulating flow around an airfoil, simulating nuclear fusion in a plasma, simulating convection in the earth mantle) or into developing numerical methods with such applications in mind, then the lingua franca is C++. Every large scale scientific computing package is today written in C++ (or C), for better or worse, and this is how it's going to stay for a long time to come. To name just a few examples, PETSc and the incredibly large and diverse Trilinos packages are written in C and C++, respectively. Among the large open source finite element libraries I can think of (libMesh, deal.II, oofem, freefem) every single one is written in C++. Among the visualization programs, the two largest ones (Visit and Paraview) are written in C++. I could extend the list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point being that if you are content with small scale simulations, people use one language, but whenever it comes to serious, maybe parallel computations, they all use something else. It's a community thing for sure, but also with being interoperable -- if PETSc and Trilinos are written in C and C++, then I (being the author of deal.II) can't easily choose any other language even if I wanted to because I need to work with PETSc and Trilinos.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-01T02:08:49.083" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5159" PostTypeId="1" AcceptedAnswerId="5170" CreationDate="2013-02-01T05:31:45.297" Score="8" ViewCount="1035" Body="&lt;p&gt;I am a PhD student in Scientific Computing and over the past few months, I spent a good amount of time learning Python and C++ the right way. I feel that I have learnt C++ well and I can use Python to do what I want if I keep a good reference book around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know MATLAB enough to prototype my ideas and get solutions. (If I am too bored to code Python which is my first choice).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read multiple times here that one should club C and C++ into one &quot;C/C++&quot; because they are extremely different languages with different motives and I completely agree with that viewpoint. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I cannot claim to &quot;know&quot; C++ since I am always learning but I think I understand most of how I should use it and how I should use it not. The first language that I learnt was C but it has been very long since I last used it. My question is essentially this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Given that I know MATLAB, C++ and Python; should I invest time in learning C? Will my knowledge of the mentioned 3 languages be enough for me to code?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My research is more on the numerical linear algebra side but I also do some discrete event simulation/stochastic processes consulting. My intention is to work in Industry (My advisor suggested I learn C++ so that I stay employable though he has no personal preferences of languages). &lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="20688" LastEditDate="2019-06-22T20:24:58.367" LastActivityDate="2019-06-22T20:24:58.367" Title="Do I need to learn C?" Tags="|python|c++|education|career-development|" AnswerCount="9" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5160" PostTypeId="2" ParentId="5159" CreationDate="2013-02-01T05:55:28.883" Score="3" Body="&lt;p&gt;It is good to know C, and you can actually implement many of the OO features of C++ that are relevant to scientific computation in C without much trouble (it is very instructive to take a look at the source code for PETSC to see how they do it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, there is no one-size-fits-all answer to this question. There are many factors in choosing a language, runtime being one of them. How long you spend writing,debugging,profiling code is another important factor. In the end you want to be as productive as possible. Knowing C will be great if you intend to get into the high performance side of research, or if you anticipate that your code will take an extremely long time to run (i.e. runtime trumps your coding workflow time). Otherwise if you know C++ and these things aren't an issue, you should be able to understand C code well enough to communicate with those who are using it.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-02-01T05:55:28.883" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5161" PostTypeId="2" ParentId="5159" CreationDate="2013-02-01T06:19:48.697" Score="1" Body="&lt;p&gt;I answered a &lt;a href=&quot;https://stackoverflow.com/questions/1416745/is-c-really-used-for-a-lot-of-scientific-computing/1416788#1416788&quot;&gt;similar question&lt;/a&gt; once on Stack Overflow.  The obvious answer is to learn everything you can, right?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's look at the negative point of view.  You already are more proficient in three languages that I would consider to be more valuable than C for scientific programming.  In my consulting work writing scientific software I've used MATLAB, Python, FORTRAN, and Java.  I've never had a need to use C or C++, but note that all of my work has been on new projects.  I would not advise a client to start a large project in C/C++, but would recommend Java, or even Scala, unless there was some compelling reason to use C/C++.  For a smaller project I would go with Python, probably.  In your case, it looks like you could deal with C on an as-needed basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect that learning more about the domains in which you will work will pay off more than being a language jockey.  In the end, it's the problem solution, not the the tool, that matters.  Knuth's saw about &quot;premature optimization&quot; applies to preparing for a career, as well.&lt;/p&gt;&#xA;" OwnerUserId="3666" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-02-01T06:19:48.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5162" PostTypeId="2" ParentId="1297" CreationDate="2013-02-01T06:41:18.313" Score="1" Body="&lt;p&gt;One may use the optimal damping algorithm &lt;a href=&quot;http://dx.doi.org/10.1002/1097-461X%282000%2979:2%3C82%3a%3aAID-QUA3%3E3.0.CO;2-I&quot; rel=&quot;nofollow&quot;&gt;ODA&lt;/a&gt; in the SCF cycle to obtain a real minimization algorithm. Then it always converges. (Related papers of Eric Cancès are also worth reading.)&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2013-02-01T06:41:18.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5163" PostTypeId="1" CreationDate="2013-02-01T06:55:04.167" Score="3" ViewCount="36" Body="&lt;p&gt;Does anyone know of a variant of the &lt;a href=&quot;http://www.lri.fr/~hansen/cmaesintro.html&quot; rel=&quot;nofollow&quot;&gt;CMA algorithm&lt;/a&gt; that is suitable for an asynchronous parallel implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conventional version of the algorithm allows one to evaluate the fitness function at different points in parallel, but one has to wait until all computations are finished before starting the next iteration. If the computational cost of the fitness function is heterogeneous, the parallel efficiency quickly degrades. Ideally an asynchronous version should be such that parameters are not disadvantaged when the fitness function takes longer to compute.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2013-02-01T06:55:04.167" Title="An asynchronous version of the Covariance Matrix Adaption (CMA)?" Tags="|optimization|parallel-computing|statistics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5164" PostTypeId="2" ParentId="5135" CreationDate="2013-02-01T10:01:10.610" Score="1" Body="&lt;p&gt;To add my two cents, I have had both recent academic experience (a fresh PhD grad :) ) and industry experience (a surveying equipment manufacturer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We do plenty of numeric computations on weak embedded processors (think mobile phone processors). Apart from there being no MATLAB for ARM, C++ is king in this world - many embedded compiler suites don't include FORTAN!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst we do have a limited number of MATLAB licences, the requirement to make a product drawing 2 Watts of power skews our development work in favour of C++, (experimenting aside).&lt;/p&gt;&#xA;" OwnerUserId="761" LastActivityDate="2013-02-01T10:01:10.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5165" PostTypeId="2" ParentId="5159" CreationDate="2013-02-01T11:18:02.803" Score="1" Body="&lt;p&gt;It's probably not worth it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++ and C tend to occupy the same niches in computational science (maybe embedded systems are an exception). It used to be the case that C compilers were better (more feature complete, better at optimizing) than C++ compilers, but from what I understand now, that is no longer the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure, C is a wonderful language (and I prefer it to C++), and if you have the time and desire, I'd recommend learning it, but if you're strapped for time, I see no compelling reason other than &quot;I need to work on a project that is being written in C&quot; (and even then, a great deal of your C++ knowledge will carry over).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-02-01T11:18:02.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5166" PostTypeId="2" ParentId="5159" CreationDate="2013-02-01T13:51:16.480" Score="0" Body="&lt;p&gt;I think some C knowledge is useful, mainly to learn about how to compile things with &quot;extern C&quot; linkage so you can pull legacy C/Fortran libraries (e.g. lapack) into new C++ projects. I wouldn't invest much time into learning the legacy C libraries, that stuff can be picked up/googled as needed, or sometimes replaced with equivalent functionality in the C++ stdlib.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2013-02-01T13:51:16.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5167" PostTypeId="2" ParentId="5159" CreationDate="2013-02-01T14:12:32.390" Score="7" Body="&lt;p&gt;Rather than &quot;learning C&quot;, I'd suggest that you're at a point now where you've done enough programming and worked with enough programming languages that you should focus instead on improving your programming technique and broader knowledge of computer science.  Then learn C if and when you need it for a particular project.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experienced programmers quickly pick up new programming languages as needed for particular projects.  They can do this because they understand important concepts that translate from language to language- the syntax isn't much of a problem if you've mastered those higher level concepts.  When you watch students trying to learn their second programming language, you'll often see them struggle with understanding some new concept that wasn't part of their first language.  For example, students who start with Fortran often struggle with pointers in C.  Students who know C often struggle with the object oriented features of C++, and so on.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-02-01T14:12:32.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5168" PostTypeId="2" ParentId="5159" CreationDate="2013-02-01T14:35:29.000" Score="6" Body="&lt;p&gt;The answer to this question really depends on what you really do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a Computer Scientist who does &lt;strong&gt;high-performance&lt;/strong&gt; computing, I'd say it's essential to know/understand programming in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you're just doing &lt;strong&gt;stuff&lt;/strong&gt; on computers, then your knowledge of C++ should be enough to get you through.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, if you're really interested in computational aspects, i.e. writing algorithms that get the most out of modern computers, you won't be able to avoid having to deal with &lt;strong&gt;low-level aspects&lt;/strong&gt; such as memory management, data layouts, SIMD vectorization, instruction-level parallelism, shared-memory parallelism, or GPU computing (CUDA and OpenCL are based on C).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, most of these things can be taken care of by compilers (e.g. for memory management, data layouts, vectorization) and/or higher-level abstractions (e.g. OpenMP, OpenACC, optimized libraries). But only to a certain extent. Doing things by hand usually involves more work, but the payoff can be an order of magnitude in performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, whenever you want to &lt;strong&gt;control the details yourself&lt;/strong&gt;, you usually end up using C since it is &quot;closer to the metal&quot;. The question is, however, is that what &lt;em&gt;you&lt;/em&gt; really want?&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-02-01T14:35:29.000" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5169" PostTypeId="2" ParentId="5159" CreationDate="2013-02-01T16:00:48.293" Score="2" Body="&lt;p&gt;The value of c, such as it is these days, is that it forces you to become familiar with the functioning of the computer at a low level of abstraction (compared to, say python). Now, c++ has all the low-level facilities of c and could stand in just as well, but most people are encouraged when leaning c++ to &lt;em&gt;avoid&lt;/em&gt; the lowest level abstractions and rely on the (well tested and debugged) high level ones; things like the container library, smart pointers, polymorphism and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It makes sense for most programmers to spend most of their time playing in realms of high abstraction, because they can spend more mental time and energy writing &lt;em&gt;what they mean&lt;/em&gt; than handling the picky little details of how you accomplish it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cost of that attitude is some risk of &lt;a href=&quot;http://www.joelonsoftware.com/articles/LeakyAbstractions.html&quot; rel=&quot;nofollow&quot;&gt;leaky abstractions&lt;/a&gt; biting you &lt;a href=&quot;http://www.codinghorror.com/blog/2009/06/all-abstractions-are-failed-abstractions.html&quot; rel=&quot;nofollow&quot;&gt;when you least expect it&lt;/a&gt;, but it almost certainly adds to overall programming efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you are happy with what you know about the low level functioning of your computer you can safely put c off, and even if you are not, you could use a restricted set of c++ to accomplish the same thing.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="183" LastEditDate="2013-02-02T23:44:17.020" LastActivityDate="2013-02-02T23:44:17.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5170" PostTypeId="2" ParentId="5159" CreationDate="2013-02-01T16:49:59.470" Score="13" Body="&lt;p&gt;I will address only the comparison of C to C++. While it is true that anything written in C can be ported to C++ with a few syntactic touch-ups, the communities have different values.&#xA;The C library community, more than almost any other, values &lt;em&gt;binary&lt;/em&gt; stability.&#xA;Binary stability is critical for low-level libraries to avoid inflicting constant pain on those layers above, especially when used with a binary distribution model and shared libraries.&#xA;C is the overwhelming preference for such libraries that need to just work, with the ability to ship new releases without recompiling layers above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to ship C++ libraries that operate at this level, but you end up needing to &quot;write C in C++&quot;.&#xA;For example, you would never place a structure definition with private or virtual members in a public header, you would never use templates in a public interface, and you would never have an API based on inheritance from classes that have any data members.&#xA;These constraints are necessary to precisely contain dependencies, so that you can modify your implementation without changing the binary interface.&#xA;C tends to be much easier to bind from different languages, due to its simpler object model and well-defined ABI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are writing code at the application level rather than the library level, then your binary interface is unimportant, so many of these concerns go away.&#xA;Use of C++ language features like inheritance and templates still tend to produce more tightly coupled code, leading to more time-consuming recompilation as the project grows.&#xA;In addition to more sprawling compile-time dependencies, simply compiling C code with a C++ compiler significantly increases compilation time (about 2x with most toolchains).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If these concerns interest you, or if you plan to work on lower-level libraries, then spending time on C may be worthwhile.&#xA;If you like using C++ language features and aren't too bothered by binary interfaces and coupling tightness, then it may not be a good use of time.&#xA;But keep C in mind if these things things start bothering you in the future.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-02-01T16:49:59.470" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5171" PostTypeId="2" ParentId="4832" CreationDate="2013-02-01T20:35:27.657" Score="1" Body="&lt;p&gt;Building the Voronoï diagram of $n$ points already takes time $O(n \log n)$. I assume that what you want to compute is the intersection of the Voronoï diagram with the polygon. One way to compute this in time $O((n+p+k) \log(n+p))$ (where $k$ is the number intersection points between the polygon and the Voronoï diagram) is to first compute the Voronoï diagram, and then compute the intersection with the polygon by any sufficiently fast algorithm (for example a sweepline algorithm using Bentley–Ottmann at its core).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a general polygon, we have $k=O(np)$. For a convex polygon, we have $k=O(n)$, and you probably assume $p&amp;lt;n$, so $O((n+p+k) \log(n+p))=O(n\log n)$. This is worse than $O(n\log k)$, but only because we ignored the time $O(n \log n)$ required to build the Voronoï diagram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I know that making Bentley–Ottmann numerical robust is challenging, and most available robust implementation use &quot;exact predicates&quot; based on multi-precision arithmetic. But even if this answer might not be the most practical, it answers your questions with respect to computational complexity of the task at hand.)&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2013-02-01T20:35:27.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5172" PostTypeId="2" ParentId="1296" CreationDate="2013-02-01T21:23:24.850" Score="1" Body="&lt;p&gt;The time-dependent Schrödinger equation is not really a heat equation. Still, the Crank–Nicolson method is well suited for its solution. However, the Crank-Nicolson method is fully implicit, so the statement &quot;doing the implicit part with ADI&quot; sounds a bit suspicious. It probably means that the diffusion like part is done with ADI. I wonder a bit whether that means the potential part is treated by an analytical solution together with another application of an operator-split scheme. (But why would we call this a Crank-Nicolson method?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the heat equation, normally the ADI methods Peaceman–Rachford, Douglas–Rachford and Douglas–Gunn get discussed. I'm not so sure how much this analysis carries over to fake/formal heat equations, but at least Douglas–Rachford is certainly unsuitable for the Schrödinger equation. There certainly are stable ADI schemes that can be used for the Schrödinger equations (probably Douglas–Gunn works), but an arbitrary ADI scheme that works well for the heat equation is not guaranteed to also work well for the Schrödinger equation. But even if it would be unstable, it would probably be only weakly unstable, so that you should still be able to get &quot;some&quot; results. So really &quot;weird&quot; results probably have a different origin than the stability of the ADI scheme.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2013-02-01T21:23:24.850" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5173" PostTypeId="2" ParentId="5133" CreationDate="2013-02-01T23:20:46.900" Score="1" Body="&lt;p&gt;I also would recommend looking at iBioSim (&lt;a href=&quot;http://www.async.ece.utah.edu/iBioSim/&quot; rel=&quot;nofollow&quot;&gt;http://www.async.ece.utah.edu/iBioSim/&lt;/a&gt;), but if that doesn't work for you, and you end up having to roll your own, you can move a step up from parsing the markup language directly by using libsbml (&lt;a href=&quot;http://sbml.org/Software/libSBML&quot; rel=&quot;nofollow&quot;&gt;http://sbml.org/Software/libSBML&lt;/a&gt;).  This will give you semantic access to the events and reactions and other elements in the SBML file.  From there, you could programmatically either write out the matrix representation, or perform your analysis directly.&lt;/p&gt;&#xA;" OwnerUserId="3671" LastActivityDate="2013-02-01T23:20:46.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5174" PostTypeId="2" ParentId="4832" CreationDate="2013-02-02T00:15:36.120" Score="1" Body="&lt;p&gt;I suggest using the method described in a recent paper I published:&#xA;&lt;a href=&quot;http://people.seas.harvard.edu/~nbonneel/vorpaline.pdf&quot; rel=&quot;nofollow&quot;&gt;http://people.seas.harvard.edu/~nbonneel/vorpaline.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is simple, and is as follows :&lt;br&gt;&#xA;You start with your polygon, and the goal is to cut it with the mediator planes defined by each seed using Sutherland-Hodgman polygon clipping algorithm. That can be efficiently done in parallel for each seed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a set of $n$ points $\{P_i\}$, let's say you want to compute the Voronoi cell of the seed $P_k$. First, you sort all the $\{P_i\}$ by increasing distance to $P_k$  (for the current seed, this is $O(\log(n))$ using a kd-tree ). You clip your polygon by the mediator plane of the segment $[P_k, P_i]$ and store the distance $D$ from $P_k$ to the furthest intersection. When for a given $i$, the distance between $P_k$ and $P_i$ is greater than $2\,D$, you can stop and this result in the Voronoi cell restricted to your polygon (further points will not contribute to any intersection).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overall algorithm is $O(n\log(n))$ and can be done in parallel for each seed : $O(n\log(n))$ is required to build the kd-tree, then for each seed there is a $O(\log(n))$ nearest neighbor query, and each query result in $O(1)$ Sutherland-Hodgman polygon clippings.&lt;/p&gt;&#xA;" OwnerUserId="1440" LastActivityDate="2013-02-02T00:15:36.120" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5175" PostTypeId="2" ParentId="5159" CreationDate="2013-02-02T01:15:03.120" Score="4" Body="&lt;p&gt;There is a bit of copy-paste from my answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5135/soft-question-where-does-python-fit-into-the-picture&quot;&gt;this question&lt;/a&gt;, with a few extra details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work in industry (surveying and machine control equipment manufacturer) using weak embedded processors (think mobile phone processors), where about 50% of the computational grunt is spent doing numeric computations - mostly sensor fusion work.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have at one mathematician working for us who used to work in a HPC group, so this is one of odd places where you may eventually find a job!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this environment, where you will almost certainly have an operating system (e.g.  Linux, QNX, WinCE), C++ is king. We use C only for kernel work (i.e. device drivers) and for deeply embedded work without an operating system (8-bit micros).  We don't use C for any numeric work. Indeed, we don't have a FORTRAN compiler for our platform!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;High performance matters to us because we only have a 1 Watt CPU for processing.  Whilst we don't have some of the &quot;glamourous&quot; problems of parallelism, we have to be cache- and memory- aware, and increasingly need to be aware of SIMD (think &lt;a href=&quot;http://en.wikipedia.org/wiki/ARM_architecture#Advanced_SIMD_.28NEON.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEON&lt;/a&gt;). What's more, unlike HPC, we have to acutely aware of latency (this is machine control!) and other operating system aspects like scheduling and context switching.  Memory allocation is especially nasty in this environment as it almost certainly means a context switch ( == latency and, on a 400MHz CPU, expensive in terms of time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These issues are independent of choice of language, so I am going to disagree with @Pedro's answer that C is necessary - after all, compiler intrinsics for SIMD are available when using C++. It does, however, mean that you have to &lt;em&gt;very&lt;/em&gt; careful about which features of C++ you use and what they cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to complete the answer, no, you don't need C. We use MATLAB for analysis work and python for scripting so your two languages other than C++ are good choices - at least in my industry.&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-02-02T01:25:05.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5177" PostTypeId="1" AcceptedAnswerId="5181" CreationDate="2013-02-03T01:18:21.873" Score="4" ViewCount="3454" Body="&lt;p&gt;I am trying to implement the following optimization (from this &lt;a href=&quot;http://ssrn.com/abstract=1483412&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;) in Matlab using fmincon:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_\omega\omega'\Sigma\omega$  subject to $\min_Ur_p \geq r_0$ where $\Sigma$ is a positive definite matrix and $\omega$ is a vector of weights that sum to 1.  Also:  $r_p=\alpha'\omega$ and $U$ can be thought of as the sphere centred at $\alpha$ with radius equal to $|\chi|\alpha$ for $\chi$ between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors show that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_Ur_p=|\alpha||\omega|[cos(\phi)-\chi]$ where $\phi$ is the angle betweent the two vectors $\alpha$ and $\omega$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas how to implement this using fmincon?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those interested, I am trying to implement the technique in this &lt;a href=&quot;http://ssrn.com/abstract=1483412&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;-specifically see pages 6-7.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="3674" LastEditorUserId="3674" LastEditDate="2013-02-03T13:03:48.350" LastActivityDate="2013-02-05T13:00:26.207" Title="Robust optimization using fmincon in Matlab?" Tags="|optimization|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5178" PostTypeId="2" ParentId="5102" CreationDate="2013-02-03T02:04:33.493" Score="3" Body="&lt;p&gt;I don't think it makes any difference. You have to choose high enough quadrature for the integral over $\theta$ so that it is equal to the Bessel function $J_0$. I chose order 20 in the example below, but you always have to do convergence with regards to the exact function and interval that you integrate over. Then I did convergence with $n$, the order of the Gaussian quadrature of the integral over $x$. I chose $f(x) = e^{-x} x^2$ and use domain $[0, x_\text{max}]$, you can change $x_\text{max}$ below. I got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; n      direct         rewritten&#xA; 1  0.770878284949  0.770878284949&#xA; 2  0.304480978430  0.304480978430&#xA; 3  0.356922151260  0.356922151260&#xA; 4  0.362576361509  0.362576361509&#xA; 5  0.362316789057  0.362316789057&#xA; 6  0.362314010897  0.362314010897&#xA; 7  0.362314071949  0.362314071949&#xA; 8  0.362314072182  0.362314072182&#xA; 9  0.362314072179  0.362314072179&#xA;10  0.362314072179  0.362314072179&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, for $n=9$ both integrals are fully converged to 12 significant digits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.integrate import fixed_quad&#xA;from scipy.special import jn&#xA;from numpy import exp, pi, sin, cos, array&#xA;&#xA;def gauss(f, a, b, n):&#xA;    &quot;&quot;&quot;Gauss quadrature&quot;&quot;&quot;&#xA;    return fixed_quad(f, a, b, n=n)[0]&#xA;&#xA;def f(x):&#xA;    &quot;&quot;&quot;Function f(x) to integrate&quot;&quot;&quot;&#xA;    return exp(-x) * x**2&#xA;&#xA;xmax = 3.&#xA;&#xA;print &quot; n      direct         rewritten&quot;&#xA;for n in range(1, 20):&#xA;    def inner(theta_array):&#xA;        return array([gauss(lambda x: f(x) * cos(x*sin(theta)), 0, xmax, n)&#xA;            for theta in theta_array])&#xA;    direct = gauss(lambda x: f(x) * jn(0, x), 0, xmax, n)&#xA;    rewritten = gauss(inner, 0, pi, 20) / pi&#xA;    print &quot;%2d  %.12f  %.12f&quot; % (n, direct, rewritten)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can play with this yourself, just change &lt;code&gt;xmax&lt;/code&gt;, possibly you might need to split the interval $[0, \infty]$ into elements and integrate element by element.&#xA;You can also change the function &lt;code&gt;f(x)&lt;/code&gt;. Make sure that you always converge the integral &lt;code&gt;rewritten = gauss(inner, 0, pi, 20) / pi&lt;/code&gt;, i.e. start with some low order and keep increasing it until the printed results stop changing.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2013-02-03T02:04:33.493" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5179" PostTypeId="1" AcceptedAnswerId="5180" CreationDate="2013-02-03T14:42:42.530" Score="11" ViewCount="12875" Body="&lt;p&gt;I'm entirely new to the notion of computational science, and am looking for a good starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that there's no objectively best language, but I'd like to learn a language that has an unarguably strong and prominent presence in regards to computational science&amp;nbsp;— one considered to have exceptional capability and efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start, I was leaning towards modelling related to atom bonding and interactions, with a requirement for graphical representations/simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do some languages tend to be better for some fields than others (i.e. physics vs. pure math)? Or is choosing a language based on other factors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard the name Fortran being thrown around a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suggestions? &lt;/p&gt;&#xA;" OwnerUserId="3679" LastEditorUserId="6923" LastEditDate="2013-12-28T19:42:36.973" LastActivityDate="2013-12-28T19:42:36.973" Title="Which language should I learn for computational science?" Tags="|languages|" AnswerCount="6" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5180" PostTypeId="2" ParentId="5179" CreationDate="2013-02-03T16:18:33.433" Score="15" Body="&lt;p&gt;Mostly it comes down to the numerical libraries available to you that will help to accomplish your task. C/C++ have a large number of numerical libraries implemented for them, but being low level languages are not the best to prototype something quickly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think to get going quickly towards a solution, I would recommend using something like Matlab or Mathematica. They have large toolset and are very high-level. Most likely, your implementation there will not scale for production use, but it could be a nice playground for trying out different methods. Once you know a path to take, you can always implement something in C/C++ more efficiently. &lt;/p&gt;&#xA;" OwnerUserId="3653" LastActivityDate="2013-02-03T16:18:33.433" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5181" PostTypeId="2" ParentId="5177" CreationDate="2013-02-03T19:52:35.250" Score="4" Body="&lt;p&gt;Based on the discussion in comments on your original question, it appears that your problem can be formulated in the form required by fmincon as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\min \omega^{T} \Sigma \omega&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;-\alpha^{T}\omega + \chi \| \alpha \| \| \omega \| \leq -r_{0}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\sum_{i=1}^{n} \omega_{i} = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha$, $\Sigma$, $\chi$, and $r_{0}$ are all known constants.  You haven't said it, but I wouldn't be surprised if you also wanted to add the constraint $\omega \geq 0$.  Note that I had to multiply the greater than or equal to constraint by negative one to turn it into a less than or equal to constraint.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this problem using fmincon, you need to define a MATLAB function for objective and another MATLAB function for the nonlinear constraint.  One slightly tricky issue here is how to get the parameters $\chi$, $\alpha$, and $\Sigma$ into these functions.  This can be done using global variables (the old fashioned way) or using anonymous functions (the way that the Mathworks now recommends.)  There are instructions on how to do this with an example in &quot;help fmincon&quot;    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-02-03T19:52:35.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5182" PostTypeId="1" AcceptedAnswerId="5186" CreationDate="2013-02-03T23:09:05.427" Score="8" ViewCount="322" Body="&lt;p&gt;I posted this question earlier on stackoverflow, where it was closed as off-topic. I hope it survives here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I our climbing gym, the routes need to be re-set from time to time. The following rules apply:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We have climbing holds with a number of different colors in varying quantities. - When a route is set in a sector, no other route with the same color must be set in that sector or in the nearby sectors to avoid confusion.&lt;/li&gt;&#xA;&lt;li&gt;Some color combinations must be avoided in a sector, like white/gray or red/pink.&lt;/li&gt;&#xA;&lt;li&gt;The goal is to have four routes in each sector, less is ok if four would break the above rules.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have tried two different approaches by now. The first one was &lt;a href=&quot;http://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow&quot;&gt;Simulated Annealing&lt;/a&gt; where I initialized the wall with a random pattern of colors (but with a given color weight) and computed a &lt;em&gt;badness&lt;/em&gt; for each color combination. This badness was also computed for combinations between one sector and its neighbours. In each iteration a randomly chosen route from the worst sector was swapped with a route from a randomly chosen other sector. This showed some sort of convergence, but the result was not usable (i.e. the resulting state contained sectors with double or triple colors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then approached the problem from the opposite side and started with an empty wall. This time, every color had a concentration which decayed from one sector to the adjacent sectors. The concentration of similar colors was also increased, i.e. a red route increased the concentration of orange in a sector and nearby. A weighted random source of colors (the bucket) gave me the next color for the wall, which was placed at the sector with the lowest concentration of this color. If a concentration was above a certain threshold, the color was not added (but put back into the bucket). This was a partial success because the result state did not contain any double colors - but some sectors were empty or contained only one color.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So: What could be an appropriate algorithm for solving this problem, given the above rules? I'll happily add more information when required.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit 1 - More information:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;my test case has 15 sectors,&lt;/li&gt;&#xA;&lt;li&gt;each sector should contain 4 routes&lt;/li&gt;&#xA;&lt;li&gt;the real gym has 3 buildings with an average of 50 sectors each&lt;/li&gt;&#xA;&lt;li&gt;some sectors are arranged around pillars, some are connected by roofs&lt;/li&gt;&#xA;&lt;li&gt;we have about 10 different hold colors&lt;/li&gt;&#xA;&lt;li&gt;the height of the sectors varies between 6 (beginner's section) and 20 meters (13 vertical + 7 roof), so they consume different amounts of holds. However, the average is about 12 and this can be considered constant.&lt;/li&gt;&#xA;&lt;li&gt;there is a limited amount of each color, the amounts are not equal&lt;/li&gt;&#xA;&lt;li&gt;some colors are easier, some more difficult (i. e. we can create a yellow route of any difficulty, whereas creating a very easy orange route for kids will be almost impossible)&lt;/li&gt;&#xA;&lt;li&gt;some sectors are &quot;easier&quot;, so easy colors should go there (this is optional, our route setters can make things harder or easier within a wide range).&lt;/li&gt;&#xA;&lt;li&gt;we can safely say which colors go well together in a sector or in neighbouring sectors and which combinations don't. There are some surprises, such as white and black (bad combo): both turn to gray while rubber (shoes) or chalk (hands) is left on them. &lt;/li&gt;&#xA;&lt;li&gt;some hold colors are combinations like violet/white (in a stripey pattern).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit 2: Some Questions about Genetic Algorithms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now downloaded and compiled ParadisEO and even got my IDE (I'm using Code::Blocks) to compile the QuickStart example. ParadisEO offers genetic algorithms with a single objective as well as multi-objective GA. GertVdE suggested to calculate the fitness of each sector and to maximize the sum of all sectors' fitnesses as a single objective. Could I also maximize the fitness of each sector with a multi-objective GA? That would be some 50 objectives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I'm struggling with the definition of a sensible crossover function. As the maximum amount of each color is fixed, crossing can lead to illegal states. If I allow more than the previously given maximum amount, the overall pattern might converge to a repitition of less &quot;toublesome&quot; combinations where the troublesome colors have been thrown out. On the other hand I can also throw out excess colors until the maximum is reached, making the crossover function non-conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am completely new to genetic algorithms)&lt;/p&gt;&#xA;" OwnerUserId="3680" LastEditorUserId="3680" LastEditDate="2013-02-04T23:22:58.027" LastActivityDate="2013-02-15T16:23:24.993" Title="Looking for an algorithm that allocates climbing hold colors to wall sectors" Tags="|optimization|algorithms|iterative-method|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5183" PostTypeId="2" ParentId="5177" CreationDate="2013-02-03T23:26:18.137" Score="2" Body="&lt;p&gt;I'd recommend using different MATLAB routines instead, if you'd like to take the extra step of making your implementation faster, and are willing to install additional software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem looks like convex program that could be posed as a &lt;a href=&quot;https://cvxr.com/dcp/&quot; rel=&quot;nofollow&quot;&gt;disciplined convex program&lt;/a&gt;, in which case using a solver like &lt;a href=&quot;https://cvxr.com/cvx/&quot; rel=&quot;nofollow&quot;&gt;CVX&lt;/a&gt; has the potential to automatically transform the problem into a form more amenable to efficient solution (a semidefinite program, or a second-order cone program, perhaps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, &lt;code&gt;fmincon&lt;/code&gt; with the formulation suggested by Brian Borchers is probably the simplest and easiest implementation to get started. Your problem looks convex, and so it should not be too sensitive to choices of initial guesses.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-02-03T23:26:18.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5186" PostTypeId="2" ParentId="5182" CreationDate="2013-02-04T11:11:44.337" Score="2" Body="&lt;p&gt;I would solve the problem stated above using a genetic algorithm approach. You code each solution as a vector of integers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assume a maximal amount of routes per sector as M (you choose); assume N sectors&lt;/li&gt;&#xA;&lt;li&gt;Create a coding vector of size M*N where each segment represents a sector and each item in the segment represents a route&lt;/li&gt;&#xA;&lt;li&gt;Assign colors by integer value, the index; use 0 as no route (to allow for less routes than M)&lt;/li&gt;&#xA;&lt;li&gt;For each color index, have the RGB values&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then you define a fitness function as a weighted sum of the minimal &lt;a href=&quot;http://en.wikipedia.org/wiki/Color_difference&quot; rel=&quot;nofollow&quot;&gt;color difference&lt;/a&gt; in each sector and the amount of routes in a sector (the amount of zeros in the vector). You can use the &lt;a href=&quot;http://paradiseo.gforge.inria.fr/&quot; rel=&quot;nofollow&quot;&gt;Paradiseo&lt;/a&gt; framework or &lt;a href=&quot;http://inspyred.github.com/&quot; rel=&quot;nofollow&quot;&gt;Inspyred&lt;/a&gt; for an implementation of Genetic Algorithms.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-02-04T12:05:17.103" LastActivityDate="2013-02-04T12:05:17.103" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5187" PostTypeId="2" ParentId="5062" CreationDate="2013-02-04T14:48:56.943" Score="1" Body="&lt;p&gt;Subject to the OP's clarifications, an exact solution is given by applying Dijkstra's shortest path algorithm to a natural subgraph of a product of two copies of the original graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $V$ be the ordered pairs $(u,v)$ of original nodes which are &quot;away from each other by at least $k$&quot;.  By assumption both endpoints of the desired agents' path $(a,c)$ and $(b,d)$ belong to $V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define edges for $V$ as those where two pairs have exactly one coordinate the same and in the other coordinate are nodes connected by an edge of the original graph.  That is, if $v$ and $w$ are connected nodes in the original graph, for any $u$ not less than $k$ distant from both $v$ and $w$, then an edge of $V$ connects $(u,v)$ to $(u,w)$ and also an edge of $V$ connects $(v,u)$ to $(w,u)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it was clarified one agent &quot;waits&quot; at each step where the other agent moves, it remains only to note that the length of a path from $(a,c)$ to $(b,d)$ in $V$ is simply the sum of path lengths for both agents.  Therefore the shortest path in $V$ corresponds to this quantity which we desired to minimize.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-02-04T14:48:56.943" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5188" PostTypeId="2" ParentId="5179" CreationDate="2013-02-04T15:30:04.883" Score="11" Body="&lt;p&gt;Try Python as described for instance in the book &lt;a href=&quot;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-3-540-73915-9&quot;&gt;Python Scripting for Computational Science&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2013-02-04T15:30:04.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5189" PostTypeId="1" AcceptedAnswerId="5190" CreationDate="2013-02-04T22:11:44.030" Score="4" ViewCount="372" Body="&lt;p&gt;I've implemented fractran and playing with sequences iteratively in C.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://mathworld.wolfram.com/FRACTRAN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mathworld.wolfram.com/FRACTRAN.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sloan: &lt;a href=&quot;https://oeis.org/A007542&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://oeis.org/A007542&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can generate correct results, but I would like to so do in time measured in hours not weeks...  I have a 2 CPU Ultrasparc III machine (zone).  pthreads improves throughput slightly.&lt;/p&gt;&#xA;&lt;p&gt;I have to iterate though each &amp;quot;fraction&amp;quot; to determine the first successful operation.&#xA;code snippet -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for(j=0; j&amp;lt; 14 ;j++)&#xA;{&#xA;  mpz_mul(tmp, elem, num[j]);&#xA;  mpz_tdiv_qr(q, r, tmp, denom[j]);&#xA;  if ( mpz_cmp(r, zero) == 0)&#xA;  {&#xA;         mpz_set(elem, q);&#xA;         break;&#xA;      }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;One gmp multiplication, one division, one compare per &amp;quot;fraction&amp;quot;.  Yes, this is brute force.&lt;/p&gt;&#xA;&lt;p&gt;Suggestions?&lt;/p&gt;&#xA;" OwnerUserId="3685" LastEditorUserId="19029" LastEditDate="2020-10-27T18:14:53.103" LastActivityDate="2020-10-27T18:14:53.103" Title="Conway's FRACTRAN" Tags="|optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5190" PostTypeId="2" ParentId="5189" CreationDate="2013-02-05T00:25:19.087" Score="8" Body="&lt;p&gt;If you've profiled it, you'll find that the problem is division. That's the really slow operation that you'd like to avoid, but avoiding multiplication wouldn't hurt either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Real FRACTRAN programs (yes, I'm going to use the phrase &quot;real FRACTRAN programs&quot; as if there is such a thing) tend to work at a deep level in terms of prime factorization. For example, consider this program (written by me; you're welcome) which implements GCD:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2/7 3/11 13/17 19/23 51/65 1/13 46/95 1/19 5/6 91/2 209/3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You give it 2^p 3^q and it returns 5^gcd(p,q).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This suggests that a better way of implementing FRACTRAN is to work with completely factorized numbers. Rather than 46/95 (an instruction in the above program), you would represent this as an array of integers indexed by prime numbers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#  2,  3,  5,  7, 11, 13, 17, 19, 23&#xA;{  1,  0, -1,  0,  0,  0,  0, -1,  1 }&#xA;# Verify that 2^1 5^-1 19^-1 23^1 = 46/95&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The machine state would similarly be an array indexed by prime numbers. To execute an instruction, you make sure that performing the subtractions would not make the state underflow, and if that checks out, simply add the two arrays together elementwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two outstanding issues to consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, how long should the arrays be? That is, how many primes do you need?  Thankfully that can be determined statically from the instruction stream. If there are primes in the input which are not mentioned in the program, they can just be copied to the output. So pick the array size for the program, and you should be fine there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other problem is how large the capacity for each array entry should be. For the instructions, once again, that can be determined statically. For the machine state, however, it's not clear. Of course, 64 bits ought to be enough for anybody (famous last words), but if you're trying to implement FRACTRAN in FRACTRAN, it may not be. Complicating things is that real FRACTRAN programs tend to use a lot of flags (i.e. primes whose power are always 0 or 1 in the machine state) as pseudo program counters, so using an MPZ for each entry would probably be wasteful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for the machine state, I'd consider a hybrid. If a prime power will fit in a machine word, use a machine word. Otherwise, use a MPZ or (since you only need addition and subtraction) a home-grown equivalent. Bonus points if you can use SIMD instructions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck, and please keep me posted. I'd love to see this on github.&lt;/p&gt;&#xA;" OwnerUserId="3686" LastEditorUserId="3686" LastEditDate="2013-02-05T03:02:55.117" LastActivityDate="2013-02-05T03:02:55.117" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5191" PostTypeId="1" CreationDate="2013-02-05T11:25:56.790" Score="3" ViewCount="58" Body="&lt;p&gt;Can the Lax equivalence theorem (&lt;a href=&quot;http://en.wikipedia.org/wiki/Lax_equivalence_theorem&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Lax_equivalence_theorem&lt;/a&gt;) be applied to the discretization of integro-differential equations, or does a similar theorem exist for them?&lt;/p&gt;&#xA;" OwnerUserId="1943" LastActivityDate="2013-02-05T11:25:56.790" Title="Lax equivalence theorem for integro-differential equation" Tags="|discretization|integral-equations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5192" PostTypeId="1" AcceptedAnswerId="5197" CreationDate="2013-02-05T12:47:21.913" Score="4" ViewCount="519" Body="&lt;p&gt;please can you help me with my code - I use Lapack to solve complex matrix (quite biq) and do it in two steps: I call &lt;code&gt;zgetrf&lt;/code&gt; (LU factorization) and then &lt;code&gt;zgetrs&lt;/code&gt; (solving system).&#xA;But the &lt;code&gt;info&lt;/code&gt; value for &lt;code&gt;zgetrf&lt;/code&gt; is not zero - &lt;code&gt;zgetrf&lt;/code&gt; fails to factorize the matrix. The &lt;code&gt;info&lt;/code&gt; value is 2. In the manual of Intel MKL (I use Lapack from these) they write: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;info&lt;/code&gt;   INTEGER.  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If &lt;code&gt;info=0&lt;/code&gt;, the execution is successful. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If &lt;code&gt;info = -i&lt;/code&gt;,&#xA;  the i-th parameter had an illegal value. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If &lt;code&gt;info = i&lt;/code&gt;, $u_{ii}$ is 0. The&#xA;  factorization has been completed, but $U$ is exactly singular. Division&#xA;  by 0 will occur if you use the factor $U$ for solving a system of linear&#xA;  equations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So I understand my matrix is somehow singular. But what does the number say - if my info=2 then the second line in my matrix is linear dependent? Or something like that?&#xA;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="3688" LastEditorUserId="4575" LastEditDate="2014-01-18T22:30:10.603" LastActivityDate="2014-01-18T22:30:10.603" Title="LAPACK - singular matrices - what does the positive integer info mean?" Tags="|lapack|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5193" PostTypeId="2" ParentId="5177" CreationDate="2013-02-05T13:00:26.207" Score="3" Body="&lt;p&gt;Just adding to the answers above, untangling the notation bit to put it in standard robust optimization format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have $r_p = \alpha + rU $ where the uncertainty $U$ is known to satisfy $U^TU \leq 1$ and your radius happen to be parametrized as $\kappa \left|\alpha\right|$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After performing the minimization over the uncertainty, i.e, eliminating the for-all operator in the uncertain constraint, it boils down to a standard quadratic problem with a convex norm-constraint. Although fmincon most likely will work, it is not really the best tool for the task,  since this is a very particular problem class for which there are dedicated extremly efficient solvers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could also be interesting to know that the problem easily generalizes to other norm-balls on the uncertainty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For fun, here is some experimental code in the MATLAB Toolbox &lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.HomePage&quot; rel=&quot;nofollow&quot;&gt;YALMIP&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To begin with, solve the problem as manually derived above. If you have an efficient second-order cone programming solver installed, it will use that. Otherwise, it will probably use fmincon&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 10;&#xA;alpha = randn(10,1);&#xA;S = randn(10);S = S'*S;&#xA;kappa = 0.01;&#xA;r0 = .01;&#xA;w = sdpvar(n,1);&#xA;Objective = w'*S*w;&#xA;Budget = [w &amp;gt;= 0, sum(w)==1];&#xA;Robust = [w'*alpha - kappa*norm(alpha)*norm(w) &amp;gt;= r0];&#xA;solvesdp([Budget,Robust], Objective)&#xA;double(w)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually, YALMIP has a built-in framework for deriving these kind of robust models automagically. Hence, the following problem derives the result in the paper and solves the problem&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = sdpvar(n,1);&#xA;U = sdpvar(n,1);&#xA;rp = alpha + kappa*norm(alpha)*U;&#xA;Objective = w'*S*w;&#xA;Budget = [w &amp;gt;= 0, sum(w)==1];&#xA;Uncertainty = [rp'*w &amp;gt;= r0, uncertain(U), U'*U &amp;lt;= 1];&#xA;solvesdp([Budget,Uncertainty], Objective)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This allows us to try alternative models, for instance a box-constrained uncertainty (this leads to a simple quadratic program, so fmincon is absolutely not the way to go)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = sdpvar(n,1);&#xA;U = sdpvar(n,1);&#xA;rp = alpha + kappa*norm(alpha)*U;&#xA;Objective = w'*S*w;&#xA;Budget = [w &amp;gt;= 0, sum(w)==1];&#xA;Uncertainty = [rp'*w &amp;gt;= r0, uncertain(U), -1 &amp;lt;= U &amp;lt;= 1];&#xA;solvesdp([Budget,Uncertainty], Objective)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you have an efficient mixed-integer SOCP solver installed (such as gurobi, cplex or mosek), you can for instance add combinatorial structures, such as a cardinality constraint on your positions&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solvesdp([Budget,Uncertainty,nnz(w)&amp;lt;=3], Objective)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, by looking at the problem a bit broader, I hope you realize there is a lot more fun that can be done, than simply throwing a general purpose nonlinear solver at a particular problem definition.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-02-05T13:00:26.207" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5194" PostTypeId="1" CreationDate="2013-02-05T16:43:33.667" Score="2" ViewCount="466" Body="&lt;p&gt;I am looking at ways of finding the global best-fitting model to a set of data. The model is highly non-linear and has anywhere between 5 and 8 parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used particle swarm optimisation with some success, as well as Metropolis-Hastings Markov Chain Monte Carlo. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of any given run, there is still doubt as to whether the parameter space has been completely  - or at least sufficiently - searched. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any algorithms for model fitting which includes some measure of how thoroughly the parameter space has been searched? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, considering PSO, is some way of estimating how &quot;sensitive&quot; any given particle is to its present neighbourhood? Let us consider a PSO particle, passing a local cost minimum in the N-dimensional manifold at a certain velocity and a certain distance from the minimum. Can we quantify the probability that this particle would be attracted to and &quot;investigate&quot; the minimum?&lt;/p&gt;&#xA;" OwnerUserId="3375" LastActivityDate="2014-05-07T14:32:34.670" Title="A cost function minimisation genetic algorithm which includes a measure of search completeness?" Tags="|optimization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5195" PostTypeId="1" CreationDate="2013-02-05T20:20:09.860" Score="-1" ViewCount="65" Body="&lt;p&gt;I have these two sets of number, I also have a result. Is it possible to compute&#xA;the relation between them so that I can figure out how the result is generated ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example I have :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Set A=4 6 12 25 39 41&#xA;Set B=3 7 14 26 47 48&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the result is : 2 24 29 31 32 49&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to figure out a formula for this ? An algorithm converts those two&#xA;sets into one result set as I mentioned above. Basically I wanna break the algorithm.&lt;/p&gt;&#xA;" OwnerUserId="3689" LastActivityDate="2013-02-06T02:45:38.793" Title="Is it possible to find a formula for a set of number from the result?" Tags="|algorithms|data-sets|block-decomposition|" AnswerCount="1" CommentCount="1" ClosedDate="2013-02-06T08:12:35.310" ContentLicense="CC BY-SA 3.0" />
  <row Id="5196" PostTypeId="2" ParentId="5192" CreationDate="2013-02-05T22:33:20.670" Score="3" Body="&lt;p&gt;If you compute the LU factorization of an $n$ by $n$ matrix $A$, then you'll end up with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$PA=LU$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $P$ is a permutation matrix, $L$ is lower triangular and nonsingular, and $U$ is upper triangular.  If $A$ happens to singular then the LAPACK routine will produce a $U$ matrix that is singular.  The only way that an upper triangular matrix can be singular is to have a diagonal entry which is 0 (the determinant of $U$ is the product of its diagonal entries.)  The info value returned by the LAPACK routine tells you the row/column of the first 0 on the diagonal of $U$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use this information (for example) to find a nonzero vector $\hat{x}$ such that $U\hat{x}=0$.  Suppose that info is $k$.  Then let $\hat{x}_{j}=0$ for $j=n, n-1, \ldots, k+1$.  Let $\hat{x}_{k}=1$.  Then use back substitution to find $\hat{x}_{k-1}, \hat{x}_{k-2}, \ldots, \hat{x}_{1}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then $LU\hat{x}=0$, so $\hat{x}$ is in the null space of $PA$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="27" LastEditDate="2013-02-06T00:02:04.100" LastActivityDate="2013-02-06T00:02:04.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5197" PostTypeId="2" ParentId="5192" CreationDate="2013-02-05T22:43:55.280" Score="8" Body="&lt;p&gt;Short answer: nothing more than $U_{ii} = 0$, i.e. that your &lt;em&gt;computed&lt;/em&gt; $U$ factorization is exactly singular. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;xGETRF&lt;/code&gt; is not safe as a rank revealing factorization, so I would not draw any conclusion, apart from the fact that $A$ is ill-conditioned and no solution to $Ax=b$ can be safely computed. Information on rank and null space should be derived (if desired) by other decompositions, eg. $QR$.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-02-05T22:43:55.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5198" PostTypeId="1" CreationDate="2013-02-05T23:05:46.253" Score="10" ViewCount="405" Body="&lt;p&gt;Reading today about the theory of differential forms, I was left impressed how much it reminded me of second order Finite Volume Method (FVM). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm struggling to figure out is thinking this way just trivial or is there some deeper connection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, differential forms serve to generalize some concepts deeply rooted in second order FVM, like flux of fluid trough a surface, and we are all about fluxes in FVM. Then integral theorem (of Stokes) is one of the central objects in theory of differential forms. It's proving involves an integration of differential forms on a manifold-where simplexes (triangles, tetrahedrons,etc.) appear. Manifold is  actually tessellated in a same manner we represent a smooth shape over which fluid passes using straight edged cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are just some of the similar things. The fact is that reading about differential forms made me not being able to stop thinking about FVM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does second order Finite Volume method actually represent Computational manifestation of Differential Forms theory?&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2013-08-14T09:08:07.083" Title="Connections between Differential Forms and the second order Finite Volume Method" Tags="|fluid-dynamics|finite-volume|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5199" PostTypeId="2" ParentId="5195" CreationDate="2013-02-06T02:45:38.793" Score="2" Body="&lt;p&gt;So if I understand you right then what you're asking for is this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given values $a_i,b_i$ as inputs and outputs $r_i=f(a_i,b_i)$, is there a way to identify $f(a,b)$?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The answer is of course &quot;yes&quot;, and it's not even particularly difficult to do. In fact, there are infinitely many solutions. Think, for example if you had only a single input parameters and you were given a sequence of $N$ points $x_i$ and values $y_i=f(x_i)$. Then finding one $f$ can be done for example by looking among the polynomials of degree $N-1$, i.e. you are trying to find a polynomial $f(x)=\alpha_0 + \alpha_1 x + \alpha_2 x^2 + \cdots + \alpha_{N-1}x^{N-1}$ so that $y_i=f(x_i)$ for $i=1\ldots N$. These are $N$ equations for the $N$ alphas an you get them by solving a linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you are looking among the polynomials of degree $N$, then there will be infinitely many polynomials that satisfy these $N$ equations with their $N+1$ coefficients. This shows that the solution is not unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2d (with inputs $a,b$) the situation is not quite that simple because it's not quite clear how many coefficients you need to take in the $a$ and $b$ directions, respectively. But if, for example, you take $N$ coefficients in each direction, i.e. a function of the kind $f(a,b)=\sum_{i=0}^{N-1} \sum_{j=0}^{N-1} \alpha_{ij} a^i b^j$ then you are guaranteed to find coefficients that satisfy all data points. The problem is, of course, underdetermined, so there are infinitely many of these $N^2$ coefficients that yield a solution, but you can for example take the least squares solution of the linear system. Again, it's not particularly difficult to find some kind of solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Polynomial interpolation is not a stable approach if you have a lot of data points. There are better approaches for such cases, but the ideas above show that it's not terribly difficult to find &lt;em&gt;any&lt;/em&gt; solution if you allow enough leeway in the class of functions you are looking in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course it is an entirely different question whether the $f(a,b)$ so identified has anything to do with the function that originally created your data. That is not a question you can answer by just looking at the three sequences of numbers. You would need to have additional information, for example additional data points that you can use to determine whether what your $f(a,b)$ predicts matches the additional output. If it doesn't match then you know that you've got the wrong function and one of the infinitely many other possibilities must be it; if it matches, then all you know is that your function predicted &lt;em&gt;this one point&lt;/em&gt; correctly, but it doesn't tell you anything about other potential points.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-06T02:45:38.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5200" PostTypeId="1" AcceptedAnswerId="5201" CreationDate="2013-02-06T09:49:57.603" Score="4" ViewCount="12154" Body="&lt;p&gt;I want to populate a numpy array with values from the smooth bump function &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(x) = exp ( - 1 / (1 - x^2) )     if |x| &amp;lt; 1,  f(x) =  0 otherwise&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Currently I have something that works (as in gives me the right numbers on my platform)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = linspace(-1.1 , 1.1, 300)   #Sample 300 points between [-1.1,1.1]&#xA;bump = exp( 1 - 1 / (1 - clip(square(x), 0,1)) ) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;when the absolute value of an entry in x is at least 1, its square gets clipped to 1, and we have &quot;&lt;code&gt;1/(1-1) = 1/0 = +inf&lt;/code&gt;&quot; as &quot;expected&quot; on my platform, which then gets set by &quot;&lt;code&gt;exp(1 - inf) = 0&lt;/code&gt;&quot; which is exactly the behaviour I want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I suspect that the above is not the best practice. Am I correct in my suspicions?&lt;/li&gt;&#xA;&lt;li&gt;Are there better ways of handling this division by zero? At the end of the day the array &lt;code&gt;x&lt;/code&gt; may not be just simply a linear list of values. So I want something that can compute &lt;code&gt;f(x)&lt;/code&gt; from &lt;code&gt;x&lt;/code&gt; efficiently. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1481" LastActivityDate="2013-02-06T18:09:40.707" Title="Consistent handling of division by zero in numpy array" Tags="|python|numpy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5201" PostTypeId="2" ParentId="5200" CreationDate="2013-02-06T10:44:45.907" Score="5" Body="&lt;p&gt;Why do you want to generate DivisionByZero exceptions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would use masked arrays:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;x= np.linspace(-1.1,1.1,300)&#xA;masked_idx = (np.abs(x)&amp;gt;1)&#xA;masked_x = np.ma.array(x,mask=idx)&#xA;&#xA;def f(x):&#xA;   return np.exp(-1.0/(1.0-x**2))&#xA;&#xA;masked_f = f(masked_x)&#xA;&#xA;plot(masked_x,masked_f)    # in IPython / pyplot&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want, you can do the masking in your function (by having boundary arguments)&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-02-06T10:44:45.907" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5202" PostTypeId="1" AcceptedAnswerId="5210" CreationDate="2013-02-02T12:53:56.380" Score="4" ViewCount="7613" Body="&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most efficient algorithm for finding a row of a matrix which matches a given row? This is the same as a table lookup based on multiple criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Context&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite Element Matrices are usually large and sparse, so storing the entire matrix in memory is an inefficient use of computer memory. As such, I am generating the matrices as 3-tuples i.e. (row index, column index, matrix value). Due to the unstructured nature of the underlying grid, the same matrix element may need to be incremented a number of times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achieve this, the existing elements must be searched to see if the (row,column) pair already exists. If it does, then the value can be incremented. If it doesn't, a new 3-tuple is created for the new non-zero value of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is: what is an efficient method for checking if the (row,column) pair already exists? And if it does, what the index into the array is?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is essentially the same as finding a row of a matrix which matches a given row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, this is implemented in MATLAB as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% rw is the array of row indices&#xA;% cl is the array of column indices&#xA;% rv is the row being searched for&#xA;% cv is the column being searched for&#xA;% i.e. searching for (rv,cv) in (rw,cl)&#xA;&#xA;possible_row = find(rw == rv);&#xA;column = find(cl(possible_row) == cv);&#xA;&#xA;% check that it was found first&#xA;if isempty(column)&#xA;    % doesn't exist yet&#xA;    element_index = -1; % flag that it needs to be created&#xA;else&#xA;    element_index = possible_row(column);&#xA;end&#xA;&#xA;% return element_index&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Due to the nature of &lt;code&gt;find&lt;/code&gt;, this procedure is much faster than manually looping through the array. I could write a MEX function to do this, but I would prefer not to at this stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1918" OwnerDisplayName="Daryl" LastEditorUserId="1918" LastEditDate="2013-02-07T20:44:18.357" LastActivityDate="2017-03-13T13:47:45.817" Title="Efficient assembly of finite element matrix in MATLAB" Tags="|algorithms|matlab|sparse-matrix|" AnswerCount="4" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="5203" PostTypeId="2" ParentId="5200" CreationDate="2013-02-06T18:09:40.707" Score="3" Body="&lt;p&gt;Another take:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_f = lambda x: np.exp(-1.0/(1.0-x*x))&#xA;f = lambda x: np.piecewise(x, [np.abs(x) &amp;lt; 1, np.abs(x) &amp;gt;= 1], [_f, 0.0])&#xA;&#xA;x = np.linspace(-1.1,1.1,300)&#xA;bump = f(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;my point here is that what you really need for efficiency is a &lt;code&gt;ufunc&lt;/code&gt;, possibly implemented in C. In the absence of a true &lt;code&gt;ufunc&lt;/code&gt; you can go with any numpy trick.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-02-06T18:09:40.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5204" PostTypeId="1" CreationDate="2013-02-06T18:52:20.130" Score="3" ViewCount="170" Body="&lt;p&gt;I have a problem of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[\begin{array}{cc} &#xA;-(\lambda+2\mu)\frac{d^2}{dx^2} &amp;amp; \alpha\frac{d}{dx} \\ &#xA;\frac{\alpha}{\Delta t}\frac{d}{dx} &amp;amp; \frac{c_0}{\Delta t}I-\frac{\kappa}{\mu_f}\frac{d^2}{dx^2}&#xA;\end{array}\right] \left[ \begin{array}{c} u\\ p\end{array}\right]&#xA;=\left[\begin{array}{c} f\\g \end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I apply a finite difference scheme to this system, I obtain a block matrix system of the form  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[ \begin{array}{cc} m_{11} &amp;amp; m_{12}\\ m_{21} &amp;amp; m_{22}\end{array}\right]\left[ \begin{array}{c} \vec{u}\\ \vec{p}\end{array}\right]&#xA;=\left[\begin{array}{c} f\\g \end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $m_{ij}$ are all sparse symmetric matrices and $u_i$, $p_i$ are the discrete values of u &amp;amp; p at the discrete points $x_i$ in the domain.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of solving this system of equations by a fixed point iteration by splitting the block matrices into the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[ \begin{array}{cc} m_{11} &amp;amp; m_{12}\\ 0 &amp;amp; m_{22}\end{array}\right]\left[ \begin{array}{c} \vec{u}\\ \vec{p}\end{array}\right]^{k+1}&#xA;+\left[ \begin{array}{cc} 0 &amp;amp; 0\\ m_{21} &amp;amp; 0\end{array}\right] &#xA;\left[ \begin{array}{c} \vec{u}\\ \vec{p}\end{array}\right]^k&#xA;=\left[\begin{array}{c} f\\g \end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recall from basic numerical analysis that if $A=M+N$ and we iterate $Mx^{k+1}=-Nx^k+b$, then the method is guaranteed to converge from any starting vector $x^0$ if the eigenvalues of $M^{-1}N$ are $&amp;lt;1$ in absolute value.  Is there an easy way to verify whether this is true for this particular matrix splitting?  Can I apply an analogous proof as that of standard gauss-seidel or jacobi methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I know that this system can be solved by other (possibly faster) means in the 1D case, but I'm particularly interested in this form because it allows me to decouple the multiphysics simulation and I can implement fast solvers for each equation in the more complicated multidimensional case.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-02-07T20:06:12.983" LastActivityDate="2013-02-08T00:03:31.580" Title="Iterative Block Matrix Splitting for Multiphysics Simulation" Tags="|pde|operator-splitting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5205" PostTypeId="5" CreationDate="2013-02-06T19:13:12.847" Score="0" Body="&lt;p&gt;E.g. Alternating Direction Implicit Methods, Strang Splitting, Sequential Splitting, Iterative Splitting, etc.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-02-06T19:13:12.847" LastActivityDate="2013-02-06T19:13:12.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5206" PostTypeId="4" CreationDate="2013-02-06T19:13:12.847" Score="0" Body="For questions on methods for solving partial differential equations by decomposition of a continuous or discrete operator into two or more separate operators." OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-02-06T19:13:12.847" LastActivityDate="2013-02-06T19:13:12.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5207" PostTypeId="2" ParentId="5202" CreationDate="2013-02-06T21:55:24.747" Score="2" Body="&lt;p&gt;You are currently using a very inefficient data structure and algorithm.  A simple analysis of your algorithm shows that it runs in $O(n_{nz}^2)$ time in the worst and average case, and since the &lt;code&gt;find&lt;/code&gt; function iterates from the beginning of your array, if you are doing the standard forward loop over your basis elements, you &lt;strong&gt;are&lt;/strong&gt; in the worst case scenario.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most efficient general strategy for what you are trying to do in MATLAB (assemble a matrix from finite element basis) is to preallocate an empty sparse matrix using &lt;code&gt;spalloc&lt;/code&gt; using whatever knowledge of the matrix's structure you have, then loop over the basis column-wise to assemble the matrix and simply use the the native MATLAB matrix syntax to update.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you were working in C, C++, or Fortran, I would have some other advice about how to make your code cache friendly, but that really isn't a detail most MATLAB developers should be paying attention to.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2013-02-06T22:08:38.790" LastActivityDate="2013-02-06T22:08:38.790" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5208" PostTypeId="1" AcceptedAnswerId="5295" CreationDate="2013-02-07T04:24:54.970" Score="1" ViewCount="936" Body="&lt;p&gt;I would like to know what are the necessary and sufficient tests one has to perform in order to show the convergence of the algorithm. I have not found a good reference to state for that as I am interested in the quantity $\|u_h-u\|$ in some discrete norm as $h\to 0$. $u_h$ denotes a numerical solution on a grid with spacing $h$ and $u$ denotes the true solution. Assume there is no exact solution to compare to. I don't want to &quot;manufacture&quot; the solution for the pde as it assumes that there is a strong solution of the pde, which I do not necessarily have. Bottom line, solution is not strong on the entire domain+the boundary. But I have a numerical algorithm that solves that equation and I can observe the results. What are my techniques to judge the accuracy for that equation with the used algorithm? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the ways I propose is ASSUME: $$\| u_h - u \| \leq Ch^2$$. Then $$\| u_{2h} - u_h \| \leq \| u_{2h} - u\| + \| u - u_h \| = 5Ch^2.$$ Similarly $$\| u_{4h} - u_{2h} \| \leq 20Ch^2$$&#xA;Thus, since all these solution live on different finite-dimensional spaces I project the finer solution on a coarser grid, that is $\| u_{2h} - u_h \|$ is measured on the grid space with spacing $2h$ and $\| u_{4h} - u_{2h} \| $ is measured on the grid spacing of $4h$. The ratio of these two quantities would give me $4$ for the second order method. However, I STARTED with an assumption that the method is second order and therefore to see $4$ as a ratio is only necessary, but not sufficient to claim that the algorithm in fact is of second order. Thus, what are my options to show that the rate of the algorithm is in fact second order in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: the equation I am solving is $$u_t=u_{xx}+1_{\{x\geq K\}}u_y,\; x\in [0,2K], y\in [0,2], t\in [0,T]$$ with initial data: $$u(0,x,y)=x-K/2, x\in[K/2,2K], y \in [0,1]$$ and and everywhere else the value is $0$. So the initial data is &quot;rough&quot;, lacking even continuity, the second dimension has no diffusion and the coefficient in pde is discontinuous, as a result, i do not obtain any regularity in $y$ dimension at all. And I don't see how I can manufacture a solution that would exhibit such a shape at $t=0$. For the manufactured solution idea, I have to come up with a function that has $u_t,u_{xx},u_y$ and is &quot;very close&quot; to the initial data, but since it will be smooth apriori in order to take derivative it is still not a good approximation of the real function. &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2013-02-09T19:48:43.220" LastActivityDate="2013-02-16T10:14:29.653" Title="necessary and sufficient tests to show order of convergence for the numerical method" Tags="|pde|convergence|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="5209" PostTypeId="2" ParentId="5204" CreationDate="2013-02-07T06:03:26.833" Score="1" Body="&lt;p&gt;When you write down the product&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M^{-1}N = \begin{pmatrix} m_{11}^{-1} &amp;amp; -m_{11}^{-1} m_{12} m_{22}^{-1} \\ 0 &amp;amp; m_{22}^{-1} \end{pmatrix}&#xA;\begin{pmatrix} 0 &amp;amp; 0 \\ m_{21} &amp;amp; 0 \end{pmatrix} =&#xA;\begin{pmatrix} -m_{11}^{-1} m_{12} m_{22}^{-1} m_{21} &amp;amp; 0 \\ m_{22}^{-1} m_{21} &amp;amp; 0 \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it is clear that the eigenvalues of $M^{-1} N$ depend on $\Delta t$. Note that you can consider the generalized eigenvalue problem $m_{12} m_{22}^{-1} m_{21} x = \lambda m_{11} x$. The stability of this iteration depends on $\Delta t$ and the relative sizes of the two terms in $m_{22}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a robust solver based on splitting, you should look at approximate commutator preconditioners. The idea there is to approximate the Schur complement $m_{11} - m_{12} m_{22}^{-1} m_{21}$ or $m_{22} - m_{21} m_{11}^{-1} m_{12}$ using something that is feasible to apply. These are usually combined with a Krylov method to catch stray eigenvalues arising because the approximate commutator arguments are inexact, especially at boundaries.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-02-08T00:03:31.580" LastActivityDate="2013-02-08T00:03:31.580" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5210" PostTypeId="2" ParentId="5202" CreationDate="2013-02-07T08:30:43.477" Score="7" Body="&lt;p&gt;If you are assembling finite element matrices from arrays of row indices, column indices and values using MATLAB's &lt;code&gt;sparse&lt;/code&gt;, there is no need to implement your own function to take care of repeated indices - &lt;code&gt;sparse&lt;/code&gt; will automatically sum values with identical row and column indices (see the &lt;a href=&quot;http://www.mathworks.de/de/help/matlab/ref/sparse.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt;, especially the third paragraph). Internally, this command creates a sparse compressed column matrix with the requested sparsity pattern and the given entries. For those interested in more details, &lt;a href=&quot;http://www.nada.kth.se/kurser/kth/2D1252/gilbert92sparse.pdf&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt; describes the original implementation of MATLAB's sparse matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is indeed the fastest way of creating a sparse matrix for which you don't know the band structure (you can use &lt;code&gt;spdiags&lt;/code&gt; for those); creating an empty matrix and assigning entries is much slower, since MATLAB needs to recreate the entire data structure after each assignment. Here's a small example to illustrate this (&lt;code&gt;spalloc(n,n,3*n)&lt;/code&gt; allocates an empty sparse matrix with storage for 3n non-zero elements):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 20000;&#xA;%% Allocate, fill row-wise&#xA;tic;&#xA;A = spalloc(n,n,3*n);&#xA;for i = 1:n&#xA;    A(i,1:3) = rand(3,1);&#xA;end&#xA;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Elapsed time is 6.738242 seconds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Allocate, fill column-wise&#xA;tic;&#xA;A = spalloc(n,n,3*n);&#xA;for i = 1:n&#xA;    A(1:3,i) = rand(3,1);&#xA;end&#xA;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Elapsed time is 1.524576 seconds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Using sparse&#xA;tic;&#xA;row = zeros(3*n,1); col = zeros(3*n,1); val = zeros(3*n,1);&#xA;ind = 1;&#xA;for i = 1:n&#xA;    row(ind+(0:2)) = [1,2,3];&#xA;    col(ind+(0:2)) = [i,i,i];&#xA;    val(ind+(0:2)) = rand(3,1);&#xA;    ind = ind+3;&#xA;end&#xA;B = sparse(row,col,val,n,n);&#xA;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Elapsed time is 0.170769 seconds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you repeat that with &lt;code&gt;n=40000&lt;/code&gt;, you'll see that the first two methods scale quadratically while the last method scales linearly.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-02-07T13:41:47.587" LastActivityDate="2013-02-07T13:41:47.587" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5211" PostTypeId="1" CreationDate="2013-02-07T09:49:51.903" Score="1" ViewCount="542" Body="&lt;p&gt;I am new user of OpenFOAM, I have to generate a grid of a human body to calculate the convective heat transfer between the body and the environment.&#xA;Can someone guide me please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&#xA;Moorz&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information:&#xA;My work is to apply CFD tools for the calculation of the heat transfer between the human body and the environment. The convection and the heat transfer from the body are calculated directly without any empiric information.&#xA;These are the Questions that need to be answered:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the slender body model sufficient for calculation of the heat exchange on the human body surface?&lt;/li&gt;&#xA;&lt;li&gt;What is the sensibility of calculations to the grid used?&lt;/li&gt;&#xA;&lt;li&gt;What is the sensibility of calculations to the turbulence model used?&lt;/li&gt;&#xA;&lt;li&gt;What is the difference between RANS and URANS computations?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And i need to find the solution of the Reynolds (or LES) equations, while the temperature transport equation is to be found using the OpenFoam CFD code under the following boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Inlet of the computational domain: The velocity profile is fixed. The pressure is fixed at patm value. The temperature is fixed at the value Tatm.&lt;/li&gt;&#xA;&lt;li&gt;Outlet of the computational domain: The velocity, temperature and the pressure satisfy the zero gradient condition.&lt;/li&gt;&#xA;&lt;li&gt;Sides of the computational domain: The velocity, temperature and the pressure&#xA;satisfy the zero gradient condition.&lt;/li&gt;&#xA;&lt;li&gt;No slip condition on the human body. The temperature fulls the condition T = Tb on the human body. The temperature distribution Tb is given.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As of now i just need to finish the task of generating a grid for human body and use RANS for calculations. If i do that, i will be given more time to finish other tasks.&lt;/p&gt;&#xA;" OwnerUserId="3701" LastEditorUserId="59" LastEditDate="2013-03-10T14:08:40.443" LastActivityDate="2013-03-10T14:08:40.443" Title="OpenFOAM-CFD, human body grid" Tags="|openfoam|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5212" PostTypeId="2" ParentId="5211" CreationDate="2013-02-07T11:42:36.617" Score="1" Body="&lt;p&gt;You can try &lt;a href=&quot;http://www.openfoam.org/docs/user/snappyHexMesh.php&quot; rel=&quot;nofollow&quot;&gt;snappyHexMesh&lt;/a&gt;, which comes with OpenFOAM. It can generate a hex/split-hex mesh from STL surfaces. Your next problem then is to get a suitable STL surface for the human body. You can probably simplify it a lot, depending on your needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is to get a human body mesh for a different toolset and covert it to OpenFOAM. OF has a number of mesh converters listed &lt;a href=&quot;http://www.openfoam.org/docs/user/mesh-conversion.php&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3680" LastActivityDate="2013-02-07T11:42:36.617" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5213" PostTypeId="1" CreationDate="2013-02-07T11:57:46.247" Score="2" ViewCount="89" Body="&lt;p&gt;I have an optimization problem with linear objective function. The constraints are in two different groups. The first set of constraints are linear while the second set is nonlinear. The nonlinear constraints are in the form: $ab-cd \geq 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-The optimization problem should be an instance of a convex optimization problem. right?&#xA;- Is it polynomial solvable?&#xA;- Is it possible to transformed it to a semidefinite optimization problem (SDP)?&lt;/p&gt;&#xA;" OwnerUserId="917" LastActivityDate="2013-04-19T01:21:35.783" Title="Nonlinear bad constraints" Tags="|optimization|nonlinear-programming|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5214" PostTypeId="2" ParentId="5202" CreationDate="2013-02-07T12:41:04.997" Score="2" Body="&lt;p&gt;Third answer, which is only a recap of the other answers (&lt;a href=&quot;https://scicomp.stackexchange.com/a/5207/1744&quot;&gt;this&lt;/a&gt; by Aron and &lt;a href=&quot;https://scicomp.stackexchange.com/a/5210/1744&quot;&gt;this&lt;/a&gt; by Christian) present at this time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a FEM implementation I would suggest the following steps.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Travel element connectivity and compute the sparsity structure of $A$.&lt;/li&gt;&#xA;&lt;li&gt;If a direct solver has to be implemented, renumber d.o.f. to reduce fill-ins, and perform a symbolic factorization of $A$, i.e. compute the sparsity structure of the factors of $A$.&lt;/li&gt;&#xA;&lt;li&gt;Allocate zero sparse matrix in &lt;a href=&quot;http://netlib.org/utk/papers/templates/node93.html#SECTION00931300000000000000&quot; rel=&quot;nofollow noreferrer&quot;&gt;BCRS&lt;/a&gt; format with your target sparsity structure (upper triangular part of $A$ for iterative symmetric solver, upper triangular part of Cholesky factor for sparse Cholesky, etc.)&lt;/li&gt;&#xA;&lt;li&gt;Travel elements, compute elemental stiffness matrix, assemble in global sparse matrix $A$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The important point here is that you have to assemble your matrix only when the sparsity structure is known, so that an efficient storage scheme like BCRS can be used. Of course this is only a hint: you have to choose the sparse matrix storage that better suits your solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia has an &lt;a href=&quot;http://en.wikipedia.org/wiki/Sparse_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;introductory article&lt;/a&gt; on sparse matrices, while in Tim David's wonderful &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;site&lt;/a&gt; you will fond a treasure of software that can help you in implementing some of these concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some notes.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;&#xA;Once the sparsity structure is known you can follow Christian's &lt;a href=&quot;https://scicomp.stackexchange.com/a/5210/1744&quot;&gt;advice&lt;/a&gt; for allocating a sparse matrix of &lt;em&gt;zeros&lt;/em&gt; in Matlab, and then efficiently assemble your stiffness matrix using standard matlab notation.&#xA;&lt;/strike&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question on how to efficiently compute the sparsity structure of $A$ is still open. I'm not so proficient in Matlab, so my advice is to implement it in python or C, applying some graph theoretical concepts, i.e. consider elements in the matrix as edges in a graph and compute the sparsity structure of the adjacency matrix. List of lists or dictionary of keys are common choices.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-02-07T15:23:42.363" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5215" PostTypeId="1" AcceptedAnswerId="5229" CreationDate="2013-02-07T14:22:41.127" Score="2" ViewCount="2463" Body="&lt;p&gt;I have a 2d numpy array, with shape &lt;code&gt;(100,3)&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = np.random((100,3))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I want to call a function &lt;code&gt;fun&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;b = fun(a)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This function is currently defined:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@np.vectorize&#xA;def fun(r):&#xA;  if r &amp;lt;= 0.0: return 0.0&#xA;  else:        return min(2.0/(1.0 + r), 2.0*r/(1.0 + r))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this works fun. But I would like to express this in a more numpythonic (and therefore more efficient) way. I tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fun(r):&#xA;  ans = zeros_like(r)&#xA;  ans[r &amp;gt; 0.0] = min(2.0/(1.0 + r), 2.0*r/(1.0 + r))&#xA;  return ans&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but that don't work as the &lt;code&gt;min&lt;/code&gt; is not doing what I want. I also tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fun(r):&#xA;  return np.piecewise(r&#xA;                     , [r &amp;lt;= 0.0, r &amp;gt; 0.0]&#xA;                     , [0.0, lambda x: min(2.0/(1.0 + x), 2.0*x/(1.0 + x))]&#xA;                     )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but that doesn't work for a similar reason. So how should I do it?&lt;/p&gt;&#xA;" OwnerUserId="3703" LastActivityDate="2013-02-08T14:52:09.153" Title="vectorizing a non-trivial function in numpy" Tags="|python|numpy|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5216" PostTypeId="1" CreationDate="2013-02-07T15:27:58.510" Score="2" ViewCount="53" Body="&lt;p&gt;I am working with a code that solves diffusion/reaction equations on a 2D unstructured mesh.  Due to the stiffness of some of the processes, I start with time steps near 1e-13, and end with a final time in the millisecond range with time steps near 1e-8.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue appears in that the problem is also, what I would like to call geometrically stiff.  During the first ~250 ns of simulation I require a very fine mesh to resolve a geometric characteristic of the forcing function.  After this time, however, the forcing of the system is approximately zero, and I no longer need the spatial resolution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fine mesh takes about 2 weeks to run through that first 250 ns.  The coarse mesh takes 2-4 weeks to run the entire simulation out to 2.5 ms (an order of magnitude speed up).  Running an adaptive mesh is not possible without a serious rewrite of the code (currently beyond my skillset as well), however there are utilities for me to restart a simulation.  What I am considering doing is running the fine mesh out until it is no longer needed, taking the restart dump file, interpolating it onto the coarse mesh, then restarting to let it run out to the final time we want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can estimate the error in my interpolation fairly easily depending on my interpolation method, however I am unsure of how to estimate the error that is propogated once I begin time-stepping again.  I am particularly worried about the smoothing on variables such at temperature will have, since many of my process rates are exponentially dependent on it.  My question is, how do I estimate how some fixed error propogates through my stiff system without running multiple cases (computationally infeasible at the present time).&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-02-07T15:27:58.510" Title="Propogated Error in Mesh Interpolation" Tags="|interpolation|error-estimation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5217" PostTypeId="2" ParentId="3502" CreationDate="2013-02-07T15:45:16.050" Score="2" Body="&lt;p&gt;There is no energy leak associated with &quot;dealiasing&quot; per se, but &lt;strong&gt;there is energy leak&lt;/strong&gt; associated with truncation that is always performed with or without dealiasing. Let me explain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if you have two function &lt;em&gt;f&lt;/em&gt; and &lt;em&gt;g&lt;/em&gt;, both having spectral content up to mode &lt;em&gt;k&lt;/em&gt;, the product &lt;em&gt;f.g&lt;/em&gt; will have spectral content up to mode &lt;em&gt;2k&lt;/em&gt;.&#xA;However, you don't want the spectral content of your representation to double at each time-step. Hence you want to &lt;strong&gt;truncate&lt;/strong&gt; the product &lt;em&gt;f.g&lt;/em&gt; to the first &lt;em&gt;k&lt;/em&gt; modes.&#xA;By doing this, you lose the energy contained in the modes higher than &lt;em&gt;k&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dealiasing (or anti-aliasing) ensures that the modes up to &lt;em&gt;k&lt;/em&gt; are correct in the representation of &lt;em&gt;f.g&lt;/em&gt;, but not the modes larger than &lt;em&gt;k&lt;/em&gt;, because anyway they will be discarded.&lt;/p&gt;&#xA;" OwnerUserId="3260" LastActivityDate="2013-02-07T15:45:16.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5218" PostTypeId="1" CreationDate="2013-02-07T16:31:05.447" Score="3" ViewCount="273" Body="&lt;p&gt;Is there a general survey on the basic algorithms for the compressed sparse rows format (like transposition, multiplication, addition, ...)? While it is not hard to write effective algorithms for that, I am not aware of any comprehensive piece of writing on these algorithmic tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does not defy that many libraries already in existence perform these operations reasonable well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;addendum: The most efficient algorithms that I am aware of generally boil down to assembling (row,column,value) triplets from the input, and building a CSR structure from this.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="238" LastEditDate="2013-02-07T16:37:40.203" LastActivityDate="2013-02-09T12:31:05.723" Title="Algorithms for Compressed Sparse Rows" Tags="|linear-algebra|sparse-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5219" PostTypeId="1" CreationDate="2013-02-07T17:07:11.907" Score="1" ViewCount="101" Body="&lt;p&gt;please how can I write gradient numerically?&#xA;I have an eq.&#xA;$\dfrac{\partial X}{\partial t}=\nabla^2 X + u\nabla X$&#xA;I wonder how write this in 3D.&#xA;Many thanks for any idea... I can find how to solve the &quot;pure&quot; heat eq. but so I don't know.. &lt;/p&gt;&#xA;" OwnerUserId="3688" LastEditorUserId="1804" LastEditDate="2013-06-12T12:29:25.390" LastActivityDate="2013-06-12T12:29:25.390" Title="Modified heat eq. in 3D" Tags="|pde|" AnswerCount="0" CommentCount="6" ClosedDate="2013-06-14T18:45:32.197" ContentLicense="CC BY-SA 3.0" />
  <row Id="5220" PostTypeId="1" AcceptedAnswerId="5225" CreationDate="2013-02-07T21:58:23.750" Score="5" ViewCount="218" Body="&lt;p&gt;I am trying to solve a particular system of non linear equations written as $F(x) = 0$ in an efficient way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, $$F(x) = (I - \gamma A)x - g(x) + C$$ where $\gamma$ is a scalarconstant, $C$ is a vector constant, $A$ is a constant matrix, and $g$ is a non-linear function of $R^n \rightarrow R^n$.&#xA;I am using Newton's method and I am relatively satisfied with it. It involves computing the Jacobian $F'(x) = I - \gamma A - g'(x)$ and solving a linear system at each Newton iteration of the form $ F'(x^k) \Delta x^k = -F(x^k)$ and then set $x^{k+1} = x^k + \Delta x^k $. In this specific case it converges quickly (2-3 iterations at most) and everything works well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as the problem becomes more complex and the size of my system of equations increases, computing the Jacobian and solving that linear system becomes a bottleneck. To overcome this, I use a simplified Newton's method which keeps F' constant over the whole  Newton iteration process, and some parallel computing since the Jacobian can be computed in parallel efficiently in my case. Unfortunately, solving the linear system still remains a bottleneck (despite use of diverse parallel linear algebra techniques).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is then: Is there a way to take advantage of the specific form of $F$ ($i.e$ the fact that it is the sum of a linear operator and a non linear operator) ?&#xA;In particular, I am especially interested in methods which would require me to solve linear systems with either $A$ (or $I - \gamma A$) or $g'$, but not their sum. Indeed, I have specific algorithms to solve linear systems with these matrices, but solving linear system with the sum of  $I - \gamma A$ and $g'$ is very inefficient ($g'$ turns out to be block diagonal so adding $I - \gamma A$ messes it up, and solving $(I - \gamma A)X = B$ by itself is not an issue thanks to the specific structure of A)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3704" LastEditorUserId="123" LastEditDate="2013-02-08T09:40:49.440" LastActivityDate="2013-02-08T09:40:49.440" Title="Solving a nonlinear algebraic system that includes a linear term" Tags="|nonlinear-equations|newton-method|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5221" PostTypeId="2" ParentId="5215" CreationDate="2013-02-07T22:07:23.583" Score="6" Body="&lt;pre&gt;&lt;code&gt;    def f(r):&#xA;        return 2 * np.clip(r, 0, 1) / (r + 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3705" LastActivityDate="2013-02-07T22:07:23.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5222" PostTypeId="1" CreationDate="2013-02-07T22:17:00.797" Score="4" ViewCount="184" Body="&lt;p&gt;I have a numerical problem in which I need to find the values $\lambda$ for which the determinant of a matrix $A_\lambda$ is zero. (The solutions $\lambda$ will give the eigenvalues of an operator...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that I learnt that it is better to look for the singularities of the function $\lambda \mapsto \log(|det(A_\lambda)|)$. An example of the plot of this kind of  function on $[0,10]$ is given in the following figure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Flfcu.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What algorithm is best for finding such a singularity if it is unique in a certain interval $[a,b]$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) How should I approach the problem of finding all singularities in an interval $[a,b]$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[I'm not sure how to choose the right tags for this question, so please feel free to add the right ones.]&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Perhaps an answer for 1) could be the golden ratio search: &lt;a href=&quot;http://en.wikipedia.org/wiki/Golden_section_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Golden_section_search&lt;/a&gt; , for the case when there is a single minimum in the interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, I have no idea how to generalize this to the case where there are multiple intervals to be chosen.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="3548" LastEditDate="2013-02-09T20:08:37.943" LastActivityDate="2014-01-06T11:54:41.220" Title="Algorithm to find singularities of a log function" Tags="|optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5223" PostTypeId="2" ParentId="5222" CreationDate="2013-02-08T00:35:11.877" Score="3" Body="&lt;p&gt;This is an exceptionally bad algorithm to find eigenvalues, at the very least if you're just looking for the eigenvalues of a linear operator or matrix. There are many better algorithms to do what you are trying to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it is true that the mathematical definition of an eigenvalue is that $A_\lambda$ is singular, i.e. that $\det A_\lambda=0$ in the finite dimensional case, this is not a practical definition to find eigenvalues in actual applications.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-08T00:35:11.877" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5224" PostTypeId="2" ParentId="5220" CreationDate="2013-02-08T00:38:28.370" Score="1" Body="&lt;p&gt;Is $g'$ by any change a low-rank correction to the remainder of the matrix? If so, you could think about using the Sherman-Morrison formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Sherman&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Sherman&lt;/a&gt;–Morrison_formula&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-08T00:38:28.370" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5225" PostTypeId="2" ParentId="5220" CreationDate="2013-02-08T03:42:12.427" Score="2" Body="&lt;p&gt;Please state up-front in your question that you are solving a reaction-diffusion problem. You should also be aware that time splitting methods, such as Strang Splitting, are very popular for these problems, though they can have serious accuracy and stability problems (e.g., Ropp, Shadid, and Ober, 2004). That said, you can use a multiplicative iterative method, preferably as a preconditioner to a linear or nonlinear Krylov method. To solve $(A + B) x = f$, compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \tilde x \gets x_0 + \omega_1 A^{-1} \big(f - (A+B) x_0\big) $$&#xA;$$ x_1 \gets \tilde x + \omega_2 B^{-1} \big(f - (A+B) \tilde x \big) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\omega_1,\omega_2$ are weighting parameters. The iteration matrix of this method is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ T = \big(1 - \omega_1 A^{-1}(A+B)\big) \big(1 - \omega_2 B^{-1}(A+B)\big) = (1-\omega_1 - \omega_1 A^{-1} B)(1 - \omega_2 - \omega_2 B^{-1} A). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The map is contractive if you choose $\omega_1,\omega_2$ such that $\lvert T \rvert &amp;lt; 1$, which may not be possible in case of indefinite reaction. One would typically use a Krylov method (and perhaps $\omega_1=\omega_2=1$) instead of tuning $\omega_1$ and $\omega_2$.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-02-08T03:42:12.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5226" PostTypeId="1" AcceptedAnswerId="5234" CreationDate="2013-02-08T08:44:37.900" Score="1" ViewCount="241" Body="&lt;p&gt;please can I ask a bit stupid question? Let say I need to solve an equation in a form&#xA;$\frac{\partial X}{\partial t}=\sum_k M_k * X_{n-k}$&#xA;How can I do the discrete convolution numerically? I will say I will expand X into series (FFT) and then, please?&#xA;I have a layer of free surfaces on the top and bottom and from the sides the material can inflow/outflow.. &#xA;Many thanks &lt;/p&gt;&#xA;" OwnerUserId="3688" LastEditorUserId="3688" LastEditDate="2013-02-08T10:53:32.540" LastActivityDate="2013-02-09T10:49:11.030" Title="Discrete convolution" Tags="|fftw|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5227" PostTypeId="2" ParentId="5151" CreationDate="2013-02-08T09:49:14.397" Score="11" Body="&lt;p&gt;To get something that looks realistic for planetary orbits, you shouldn't use the forward or backward Euler methods.  These will cause your planets to spiral outward or inward.  You should use a symplectic method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also need to adjust the timestep to be smaller when two bodies are very close to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read &lt;a href=&quot;http://dx.doi.org/10.1046/j.1365-8711.1999.02379.x&quot;&gt;Chambers (1999) &lt;em&gt;A hybrid symplectic integrator that permits close encounters between massive bodies&lt;/em&gt;&lt;/a&gt; to get started with such methods.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2013-02-15T13:06:48.950" LastActivityDate="2013-02-15T13:06:48.950" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5228" PostTypeId="1" CreationDate="2013-02-08T10:05:42.383" Score="12" ViewCount="2591" Body="&lt;p&gt;I need to solve the same sparse linear system (300x300 to 1000x1000) with many right hand sides (300 to 1000).&#xA;In addition to this first problem, I would also like to solve different systems, but with the same non-zero elements (just different values), that is many sparse systems with constant sparsity pattern.&#xA;My matrices are indefinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performance of the factorization and initialization is not important, but performance of the solve stage is.&#xA;Currently I'm considering PaStiX or Umfpack, and I will probably play around with Petsc (which supports both solver)&#xA;Are there libraries capable of taking advantage of my specific needs (vectorization, multi-threading) or should I rely on general solvers, and maybe modify them slightly for my needs ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if the sparse matrix is larger, up to $10^6 \times 10^6$ ?&lt;/p&gt;&#xA;" OwnerUserId="3260" LastEditorUserId="4244" LastEditDate="2013-07-01T09:28:28.163" LastActivityDate="2014-02-05T18:01:26.207" Title="Sparse linear solver for many right-hand sides" Tags="|linear-algebra|petsc|linear-solver|sparse-matrix|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5229" PostTypeId="2" ParentId="5215" CreationDate="2013-02-08T14:52:09.153" Score="0" Body="&lt;p&gt;Thanks to both @clipper and @christianclason, I figured out something that works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fun(r):&#xA;  np.clip(r, 0.0, r, out=r)&#xA;  return 2.0 * np.minimum(1.0, r) / (1.0 + r)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and my program gets about a 20% reduction in total run time! Unfortunately, the code now looks 20% more rubbish.&lt;/p&gt;&#xA;" OwnerUserId="3703" LastActivityDate="2013-02-08T14:52:09.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5230" PostTypeId="2" ParentId="5228" CreationDate="2013-02-08T15:08:40.437" Score="9" Body="&lt;p&gt;There is typically a trade-off between the amount of work you put into constructing a good preconditioner for an iterative solver and the work you save by using a good preconditioner when actually solving the linear systems. In your case, the case is pretty clear: put as much work as you can into constructing a good preconditioner because you have to solve so many linear systems. In fact, I think it is appropriate to invest the time to get the perfect preconditioner: an LU decomposition (using UMFPACK, for example, or the Pardiso solver that comes as part of Intel's MKL). Then simply apply this decomposition as many times as necessary. If you have $O(N)$ linear systems to solve, nothing can be expected to beat an exact decomposition.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-08T15:08:40.437" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5231" PostTypeId="2" ParentId="5228" CreationDate="2013-02-08T15:15:45.243" Score="3" Body="&lt;p&gt;You're not quite clear in your statement of the problem when you talk about &quot;the same non-zero elements (just different values)&quot;  Are you saying that the matrix has a constant sparsity pattern but the actual values change?  Or, are you saying that the matrix is in fact constant?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that the sparse matrix is constant and only the right hand side is changing, then you should be looking at methods that use direct factorization (of the form $PA=LU$) of the matrix, and then solve for each right hand side by forward/backward substitution.  Once the factorization is complete, each solution will be extremely fast ($O(n^2)$ time for completely dense factors but for sparse factors this will be proportional to the number of nonzeros in the factors.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For multiple right hand sides and systems of equations of this size, iterative methods are typically not worth while.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the packages you mentioned offer direct factorization methods (although PetSc is mostly known for its iterative solvers.)  However, your systems are so tiny that it is unlikely that you could get substantial parallel speedups, particularly in a distributed memory environment.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest using Umfpack for this job- PaStix and PetSc are overkill.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-02-08T15:15:45.243" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5232" PostTypeId="1" AcceptedAnswerId="5233" CreationDate="2013-02-09T02:32:43.747" Score="5" ViewCount="1256" Body="&lt;p&gt;I would like to find $x$ which minimizes the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{x^HAx}{x^HBx} + \frac{x^HCx}{x^HDx}$ where A, B, C, D are positive-definite. $x$ is not a very large vector (&amp;lt;1000 elements in size.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is easily solvable as a generalized eigenvalue problem when A or C=0, but I am not sure what the best approach is for the general case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="1480" LastEditDate="2013-02-09T04:52:19.233" LastActivityDate="2013-02-09T22:25:41.957" Title="minimization problem: sum of Rayleigh quotients" Tags="|optimization|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5233" PostTypeId="2" ParentId="5232" CreationDate="2013-02-09T07:02:41.657" Score="2" Body="&lt;p&gt;This paper may be useful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.optimization-online.org/DB_FILE/2012/03/3388.pdf&quot; rel=&quot;nofollow&quot;&gt;Lei-Hong Zhang. &quot;On optimizing the sum of the Rayleigh quotient and the generalized Rayleigh quotient on the unit sphere&quot;.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-02-09T07:02:41.657" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5234" PostTypeId="2" ParentId="5226" CreationDate="2013-02-09T10:49:11.030" Score="5" Body="&lt;p&gt;In the continuous case on the whole real line, the convolution of two functions is equal to the inverse Fourier transform of the pointwise product of their Fourier transforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the finite (discrete) case, it depends on the how you define the behavior of the convolution at the end points (as Dirk pointed out), since $n-k$ can be outside the range of indices for your vectors. There are two possibilities to prevent this (but since you didn't specify limits for your sum, I don't know which one you need):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can &quot;wrap around&quot; the indices, i.e., for $n&amp;lt;k$ take $X_{N+n-k}$, where $N$ is the length of $X$ and $M$. This is the &lt;em&gt;cyclic convolution&lt;/em&gt;, defined as&#xA;$$ (M*X)_n = \sum_{k=1}^N M_k X_{(n-k)\ \textrm{mod}\, N},\qquad 1\leq n\leq N.$$&#xA;In this case, you can compute the convolution as in the continuous case:&#xA;$$ M*X = FFT_N^{-1}\left(FFT_N(M)FFT_N(X)\right), $$&#xA;where $FFT_N$ is the (fast) discrete Fourier transform of length $N$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can &quot;truncate&quot; the indices, i.e., take $\max(1,n-k)$ instead of $n-k$. Without going into details, you now need to sum over a larger range; the &lt;em&gt;linear convolution&lt;/em&gt; is defined as&#xA;$$ (M*X)_n = \sum_{k=\max(1,n-N)}^{\min(n,N)} M_k X_{n-k}\qquad 1\leq n\leq 2N-1.$$&#xA;Note that the linear convolution of two vectors of length $N$ has therefore length $2N-1$. To compute this using the FFT, you need to pad the vectors: Let $(X;0_m)$ denote the vector $X$ padded by $m$ zeros. Then,&#xA;$$ (M*X;0_1) = FFT_{2N}^{-1}\left(FFT_{2N}((M;0_N))FFT_{2N}((X;0_N))\right).$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-02-09T10:49:11.030" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5235" PostTypeId="2" ParentId="5228" CreationDate="2013-02-09T11:13:14.263" Score="10" Body="&lt;p&gt;Without taking sides the discussion about whether to use direct or iterative solvers, I just want to add two points:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There exist Krylov methods for systems with multiple right-hand sides (called &lt;em&gt;block Krylov methods&lt;/em&gt;). As an added bonus, these often have faster convergence than standard Krylov methods since the Krylov space is built from a larger collection of vectors. See &lt;a href=&quot;http://www.cs.umd.edu/~oleary/reprints/j06.pdf&quot;&gt;Dianne P. O’Leary, &lt;em&gt;The Block Conjugate Gradient Algorithm and Related Methods&lt;/em&gt;. Linear Algebra and its applications 29 (1980), pages 239-322.&lt;/a&gt; and &lt;a href=&quot;http://www.sam.math.ethz.ch/~mhg/pub/delhipap.pdf&quot;&gt;Martin H. Gutknecht, &lt;em&gt;Block Krylov space methods for linear systems with multiple right-hand sides: An introduction&lt;/em&gt; (2007).&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have different matrices with the same sparsity pattern, you can precompute a symbolic factorization for the first matrix, which can be reused in computing the numerical factorization for this and the subsequent matrices. (In UMFPACK, you can do this using &lt;code&gt;umfpack di symbolic&lt;/code&gt; and passing the result to &lt;code&gt;umfpack_di_numeric&lt;/code&gt;.) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-02-09T11:13:14.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5236" PostTypeId="2" ParentId="5218" CreationDate="2013-02-09T12:31:05.723" Score="4" Body="&lt;p&gt;There are chapters on this in &#xA;&lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898718881&quot; rel=&quot;nofollow&quot;&gt;Tim Davis' book&lt;/a&gt;&#xA;and &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot; rel=&quot;nofollow&quot;&gt;Yousef Saad's book&lt;/a&gt; on linear systems for sparse matrices. Tim Davis also has some &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/sparse11/&quot; rel=&quot;nofollow&quot;&gt;lecture notes&lt;/a&gt; on his website.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-02-09T12:31:05.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5237" PostTypeId="2" ParentId="5232" CreationDate="2013-02-09T22:25:41.957" Score="2" Body="&lt;p&gt;Geoffrey already posted a solution that is adapted to your particular problem. As a general suggestion, here's another approach. Let me define the following function:&#xA;$$&#xA;  J_\alpha(x) = \alpha \frac{x^HAx}{x^HBx} + (1-\alpha) \frac{x^HCx}{x^HDx}.&#xA;$$&#xA;Then we can define $x_\alpha^\ast$ as the maximizer of $J_\alpha$:&#xA;$$&#xA;  x_\alpha^\ast = \max_x J_\alpha(x).&#xA;$$&#xA;The $x$ you are looking for in your original formulation is $x^\ast_{1/2}$. Now, $x^\ast_0$ and $x^\ast_1$ are easy to compute by just finding the eigenvector to the maximal eigenvalues of the individual generalized eigenproblems. Consequently, there are two approaches one could use:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a path-following method where you start at $x^\ast_0$ and you find $x^\ast_{\alpha+\delta\alpha}$ by solving the nonlinear optimization problem with $x^\ast_\alpha$ as the starting point for each of these problems. If you choose $\delta\alpha$ appropriately, this shouldn't take you more than 1 or 2 Newton steps. You simply follow the path to $\alpha=1/2$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the matrices $A,C$ and $B,D$ are somehow related, it may be that the maximal eigenvectors $x^\ast_0, x^\ast_1$ to the respective eigenproblems are not too different. In that case, starting the nonlinear iteration from $\frac 12(x^\ast_0 + x^\ast_1)$ may yield a Newton iteration that converges reasonably quickly to $x^\ast_{1/2}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-09T22:25:41.957" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5238" PostTypeId="1" AcceptedAnswerId="5250" CreationDate="2013-02-10T09:24:07.133" Score="5" ViewCount="4644" Body="&lt;p&gt;I'm a physics undergrad, looking for a good introductory book on computational science, and numerical methods. Mostly I'm looking for applied books. (Simply because... in a theoretical book, if I can't see why I'm studying it, it's easy to lose motivation!) &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I have the following background:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://mathandcode.com/portfolio/&quot; rel=&quot;nofollow&quot;&gt;Strong background&lt;/a&gt; in programming (C++/Java/javascript/BASIC/introductory Mathematica)&lt;/li&gt;&#xA;&lt;li&gt;parallel computing (enough to compile/fiddle with CUDA samples, and to use mutexes/locks semi-competently)&lt;/li&gt;&#xA;&lt;li&gt;Mid-undergraduate physics (so, mechanics, waves, and special relativity, at a decent level, as well as introductory lagrangians)&lt;/li&gt;&#xA;&lt;li&gt;Introductory linear algebra&lt;/li&gt;&#xA;&lt;li&gt;Vector/tensor calculus (I 'learned' this material, but I really haven't applied it much)&lt;/li&gt;&#xA;&lt;li&gt;Proofs/intro real analysis (halfway through baby Rudin, for those familiar, so I'm decent with rigor)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Goals&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm looking for an applied book, because the simulations I write to figure out a problem are sometimes insufficient! Several of these areas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mechanics. &lt;a href=&quot;http://www.khanacademy.org/cs/particle-dynamics/1073930447&quot; rel=&quot;nofollow&quot;&gt;My current method&lt;/a&gt; is to treat everything as a rigid sphere, with a spring force, and some damping on the plane of contact whenever a collision occurs. I don't think I've advanced from this in... maybe, three years? So, it would be very useful to learn a new method. Plus, with my own methods I sometimes &lt;a href=&quot;http://www.youtube.com/watch?v=nWJLG7s3Cfc&quot; rel=&quot;nofollow&quot;&gt;run into bugs&lt;/a&gt;. Finite element analysis looks like a very interesting topic but the only books I've found on it are very rigorous and not very applied, so it's hard to be introduced to the material.&lt;/li&gt;&#xA;&lt;li&gt;Wave Motion. It's pretty straightforward to discretize $\nabla^2\phi=\frac{\partial^2\phi}{\partial t ^2}$ and just to use it, but whenever I run into some instability (large error in energy, or an unstable simulation, or anything else that I don't know how to solve), my only solution is to decrease the timestep/increase the resolution. I only find new things every once in a while (such as a decent method for a nonreflecting boundary condition!). &lt;/li&gt;&#xA;&lt;li&gt;Electromagnetism. I know vector calculus, so, why not?&lt;/li&gt;&#xA;&lt;li&gt;Fluid mechanics? It seems like parts of this aren't that complicated, but I'm not familiar with it.&lt;/li&gt;&#xA;&lt;li&gt;Others? Special relativity seems to necessitate electromagnetism, so that's off limits for now, and I'm not that knowledgeable in quantum mechanics. I'm not opposed to pure math things (attractors, etc) so long as there are interesting things computed, and not just several results proved and unused.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, to clarify, I'm looking for an applied computational, possibly physics-based book, which starts at the introductory level (&lt;em&gt;introductory&lt;/em&gt; not being synonymous with &lt;em&gt;easy&lt;/em&gt;), and ideally provides a broad overview of multiple methods, with lots of applications, notably in the fields above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked quite a bit for books in these areas, but usually they turn out to either be way over my head, or theoretical to the point where I don't know what the purpose of what I'm proving/deriving is!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I understand that this is a very soft question, and I was hesitant posting it because of that, because I'm new to this corner of stackexchange, and because it might be seen as too localized. I don't think it's too localized, since I've found questions like these very useful in getting a grasp on a large field that I'm new in. Usually when I want to learn more about a topic I pick up a book, and to figure out what book to pick up I go to stackexchange, and usually the book recommendations on questions others have asked are great. I didn't find any questions applicable to my level on this site, so I think it's appropriate to ask one. Of course I'll take any objections into consideration.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;One book that was recommended to me (not here) was &quot;A First Course in Computational Physics&quot; by Devries. The practice problems look especially good/interesting.&lt;/p&gt;&#xA;" OwnerDisplayName="user121" LastEditorDisplayName="user121" LastEditDate="2013-02-14T01:46:16.513" LastActivityDate="2014-10-20T10:01:39.463" Title="Recommendation for an introductory level book in computational physics?" Tags="|reference-request|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5239" PostTypeId="2" ParentId="5238" CreationDate="2013-02-10T13:27:08.853" Score="2" Body="&lt;p&gt;Look at the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cambridge.org/gb/knowledge/isbn/item1171557&quot; rel=&quot;nofollow&quot;&gt;Simulating the Physical World: Hierarchical Modeling from Quantum Mechanics to Fluid Dynamics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0306466317&quot; rel=&quot;nofollow&quot;&gt;Computational Physics - An Introduction&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For notes based on the 2nd book by the author of the book himself see &lt;a href=&quot;http://homepage.univie.ac.at/franz.vesely/cp0102/dx/dx.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. For more advanced stuff see&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cambridge.org/gb/knowledge/isbn/item1171410&quot; rel=&quot;nofollow&quot;&gt;Computational Physics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are plenty of other books too.&lt;/p&gt;&#xA;" OwnerUserId="1285" LastActivityDate="2013-02-10T13:27:08.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5240" PostTypeId="1" CreationDate="2013-02-10T16:04:57.557" Score="0" ViewCount="152" Body="&lt;p&gt;please can I ask, how one can make from function sine series (Fourier transform) with MKL? I can do &quot;normal&quot; exponential FFT with MKL (Lapack of course), how can I say that I want only sine series?&#xA;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="3688" LastActivityDate="2013-02-14T20:37:49.653" Title="FFT - function only in sine series? Can be done with MKL / Lapack?" Tags="|lapack|fftw|" AnswerCount="0" CommentCount="3" ClosedDate="2018-03-15T22:35:14.557" ContentLicense="CC BY-SA 3.0" />
  <row Id="5241" PostTypeId="1" CreationDate="2013-02-10T18:20:19.460" Score="0" ViewCount="228" Body="&lt;p&gt;I have a convex optimization problem that is essentially a linear objective function over some linear constraints and also a semidefinite matrix in the following form: $&#xA;         M=&#xA;            \left[ {\begin{array}{cc}&#xA;             a &amp;amp; \sqrt{u} \\&#xA;             \sqrt{u} &amp;amp; b \\&#xA;                \end{array} } \right]&#xA;        \succeq 0$&#xA;Is this problem an instance of a semidefinite programming problem?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="917" LastEditDate="2013-02-10T22:27:32.927" LastActivityDate="2013-03-12T22:38:22.563" Title="Semidefinite programming" Tags="|optimization|linear-programming|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5242" PostTypeId="2" ParentId="5241" CreationDate="2013-02-10T19:46:18.663" Score="3" Body="&lt;p&gt;Your question isn't clearly stated.  It's perfectly possible to have $M$ being a 2 by 2 symmetric and positive semidefinite matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M=\left[&#xA;\begin{array}{cc}&#xA;M_{1,1}  &amp;amp; M_{1,2} \\&#xA;M_{2,1}  &amp;amp; M_{2,2} &#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M \succeq 0$.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presumably you want to write other constraints into your problem that involve the elements of this $M$.  The question is what kinds of constraints do you want to put on the elements of $M$.  For example, if all that you mean by putting $\sqrt{u}$ in the off diagonal elements is that these elements must be nonnegative, that's easy to do.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-02-10T19:46:18.663" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5243" PostTypeId="1" AcceptedAnswerId="5245" CreationDate="2013-02-11T04:41:23.697" Score="6" ViewCount="1882" Body="&lt;p&gt;In a numerical computation, I am required to take a multi-dimensional FFT on a distributed-memory cluster. The data is currently distributed using a distributed array in PETSc (DMDA). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I initial looked at the distributed version of FFTW. However, the layout of my data does not conform to the layout used by FFTW. Therefore, I would need to scatter my data to the layout required by FFTW which would not be ideal. See &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.2/src/mat/examples/tests/ex158.c.html&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt; for an example using PETSc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, the FFT is not the dominant bottleneck in my computation. I just want to minimize the communication overhead required for the FFT. Does anyone have any suggestions for what I should be using? Is FFTW the optimal approach even when my data does not conform to FFTW layout?&lt;/p&gt;&#xA;" OwnerUserId="1031" LastEditorUserId="9" LastEditDate="2013-02-13T22:28:04.297" LastActivityDate="2013-02-13T22:28:04.297" Title="MPI-based Implementations of FFT" Tags="|parallel-computing|petsc|fourier-analysis|fftw|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5244" PostTypeId="1" AcceptedAnswerId="5246" CreationDate="2013-02-11T08:23:11.857" Score="1" ViewCount="81" Body="&lt;p&gt;I solve such a problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets have a function $Y=\sum_{k=-\infty}^\infty i\hat Y e^{ik\pi y}$&#xA;and then I have a function which is defined as $X=\sum_{k=-\infty}^\infty ik^2\hat Y e^{ik\pi y}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the $Y$. The $i$ is imaginary unit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I compute the $X$? I think I do the FFT on $Y$ and obtain thus the $\hat Y$, right? And then I think I will do the backward FFT of function $f=ik^2\hat Y$. But what have I do with the summation index $k$ here in the $f$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is right that $FFT(ik^2\hat Y)=X$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure absolutely what to do with $k$ when the FFT sum is summated per $k$.&lt;/p&gt;&#xA;" OwnerUserId="3688" LastEditorUserId="3688" LastEditDate="2013-02-11T08:53:32.423" LastActivityDate="2013-02-12T03:21:54.623" Title="A function as a sum of serie of modified FFT coeff. of another function - multiplied by sum number" Tags="|fourier-analysis|fftw|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5245" PostTypeId="2" ParentId="5243" CreationDate="2013-02-11T11:04:23.070" Score="7" Body="&lt;p&gt;Michael Pippig at the University of Chemnitz (Germany) has implemented an MPI-parallelized FFT that uses FFTW in the background. This might help you: &lt;a href=&quot;http://www-user.tu-chemnitz.de/~mpip/software.php?lang=en&quot;&gt;http://www-user.tu-chemnitz.de/~mpip/software.php?lang=en&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is using the algorithm proposed by Plimpton from Sandia National Labs as suggested by Eldila's comment.&lt;/p&gt;&#xA;" OwnerUserId="1568" LastActivityDate="2013-02-11T11:04:23.070" CommentCount="0" CommunityOwnedDate="2013-02-11T11:04:23.070" ContentLicense="CC BY-SA 3.0" />
  <row Id="5246" PostTypeId="2" ParentId="5244" CreationDate="2013-02-11T12:46:26.373" Score="2" Body="&lt;p&gt;It appears you are very close to your answer, so I am going to detail the process as a whole to clarify a couple small details.  If this doesn't answer your question, please comment and I will modify my answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You know $Y$, so you evaluate it at a number of points over your interval and plug it into $FFT$.  This produces your $\hat Y$, which is a &lt;em&gt;vector&lt;/em&gt;.  Let's denote each element in this vector as $\hat Y_k$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then have that $\hat X_k = i k^2 \hat Y_k$.  $\hat X$ is a vector the same length as $\hat Y$.  You then plug $\hat X_k$ back into your $FFT^{-1}$ to produce your X.  It appears the only thing you were missing was representing $\hat Y$ as a vector.  Note that the $i k^2$ can be any function of $k$ you want.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-02-11T21:28:39.550" LastActivityDate="2013-02-11T21:28:39.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5247" PostTypeId="1" CreationDate="2013-02-11T14:07:50.193" Score="2" ViewCount="114" Body="&lt;p&gt;In order to compute the sum of a power series, we can use for loop, while loop or the analytic formula. I am wondering what is difference between those algorithms and how to improve the robustness of this question?&lt;/p&gt;&#xA;" OwnerUserId="866" LastEditorUserId="276" LastEditDate="2017-10-10T00:46:34.680" LastActivityDate="2017-10-10T00:46:34.680" Title="How to compute the sum of a power series in a more robust way?" Tags="|numerical-analysis|" AnswerCount="0" CommentCount="3" ClosedDate="2019-06-06T19:33:34.873" ContentLicense="CC BY-SA 3.0" />
  <row Id="5248" PostTypeId="2" ParentId="5238" CreationDate="2013-02-11T14:30:57.853" Score="0" Body="&lt;p&gt;I suggest you check out these books:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A.B. Shiflet and G.W. Shiflet, Introduction to Computational Science, Princeton University Press, 2006&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R.H. Landau, M.J. Paez, and C.C. Bordeianu, A Survey of Computational Physics, Princeton University Press, 2008&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. &lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2013-02-11T14:30:57.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5249" PostTypeId="2" ParentId="5208" CreationDate="2013-02-11T16:01:09.393" Score="2" Body="&lt;p&gt;The usual strategy for problems where you don't have and exact solution is to compute an &quot;overkill&quot; solution on an extremely fine mesh (as fine as you can reasonably run) and then compute a sequence of solutions on successively finer meshes leading up to that mesh. You can then plot the difference between the overkill solution and each of the others and compare to the expected asymptotic rate.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-02-11T16:01:09.393" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5250" PostTypeId="2" ParentId="5238" CreationDate="2013-02-11T17:56:58.417" Score="6" Body="&lt;p&gt;Another suggestion is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An Introduction to Scientific Computing.&#xA;Twelve computational projects solved with MATLAB&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ljll.math.upmc.fr/AI2SC/&quot;&gt;http://www.ljll.math.upmc.fr/AI2SC/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The twelve projects cover many of the areas you mention&#xA;in your post, are each in a separate chapter, and are&#xA;more or less independent of each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bill&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-02-11T17:56:58.417" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5252" PostTypeId="2" ParentId="5244" CreationDate="2013-02-12T03:21:54.623" Score="2" Body="&lt;p&gt;You may not realize this but you have, in fact, that $X(y)=-\frac1{\pi^2}Y''(y)$. If you know $Y(y)$ then it should be simple enough to compute $X(y)$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-12T03:21:54.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5253" PostTypeId="1" AcceptedAnswerId="5254" CreationDate="2013-02-12T14:27:26.560" Score="6" ViewCount="1613" Body="&lt;p&gt;In &lt;a href=&quot;http://en.wikipedia.org/wiki/Floating_point#Addition_and_subtraction&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Floating_point#Addition_and_subtraction&lt;/a&gt;, it gives an example about cancellation problem in float-point numbers, see&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8MMd0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand why it is written : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;The best representation of this difference is e = -1; s=4.877000, which differs&lt;/code&gt;&#xA;&lt;code&gt;more than 20%.....&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example, it uses &lt;code&gt;IEEE 754 decimal32 format&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone explain the idea about cancellation problem in float-point numbers as this example want to illustrate?&lt;/p&gt;&#xA;" OwnerUserId="3717" LastActivityDate="2013-02-12T14:51:58.550" Title="cancellation problem in float-point numbers" Tags="|floating-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5254" PostTypeId="2" ParentId="5253" CreationDate="2013-02-12T14:51:58.550" Score="10" Body="&lt;p&gt;The point they are making is that if you compute the difference&#xA;123457.1467 - 123456.659 = 0.4877&#xA;then you get an answer that seems to be quite different than the difference&#xA;123457.1 - 123456.7 = 0.4&#xA;that you get by subtracting the rounded numbers.  The ~20% figure is because 0.4877 is more than 20% larger than 0.4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This example shows that if you subtract two numbers that are very close to each other in the sense of small relative difference, then if you use floating point arithmetic you might have a bad time.  This means that sometimes it is worthwhile to re-organize your calculations in ways that are mathematically equivalent but which differ in their floating point friendliness.&lt;/p&gt;&#xA;" OwnerUserId="3705" LastActivityDate="2013-02-12T14:51:58.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5255" PostTypeId="1" AcceptedAnswerId="5256" CreationDate="2013-02-12T16:01:14.630" Score="2" ViewCount="836" Body="&lt;p&gt;I am trying to figure out how to write a proper job file for the Sun Grid Engine so I can run a parallel computation using Mathematica &lt;em&gt;on multiple nodes&lt;/em&gt;.  Unfortunately almost all tutorials on the topic assume using MPI, which is not the case here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to get this working, I need to find the answer to a few very specific questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When I request multiple nodes, will the job file be run on all of them, or only on a single node?  I need to run it only on a single node in this case, and start a single process which we can call &lt;em&gt;controlling process&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I access the names / IP addresses of all the other nodes?  I &lt;em&gt;think&lt;/em&gt; I have the answer to this one: the &lt;code&gt;PE_HOSTFILE&lt;/code&gt; variable should point to a file that lists the names of all nodes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I let the controlling process start arbitrary processes at arbitrary times on any node that I can use?  Should a simple &lt;code&gt;ssh nodename command&lt;/code&gt; work for this, or would this require a password in a typical setup?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="605" LastActivityDate="2013-02-12T16:16:12.007" Title="Multi-node jobs with Sun Grid Engine without MPI" Tags="|parallel-computing|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5256" PostTypeId="2" ParentId="5255" CreationDate="2013-02-12T16:16:12.007" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;The job script is run only on a single node by SGE. It is run on the first host listed in &lt;code&gt;PE_HOSTFILE&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;PE_HOSTFILE&lt;/code&gt;can be parsed to get the names of all the nodes&lt;/li&gt;&#xA;&lt;li&gt;Using SSH without a passphrase internal to a cluster is a common setup, but it depends on who setup the machine you'll be using. I would ask them.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In addition, some HPC sites supply tools that might let you launch these tasks in a simpler manner. You should check with the site where you will be running.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-02-12T16:16:12.007" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5257" PostTypeId="1" AcceptedAnswerId="5259" CreationDate="2013-02-12T18:57:32.027" Score="2" ViewCount="415" Body="&lt;p&gt;please can I ask a bit stupid question? I have a complex matrix &lt;em&gt;A&lt;/em&gt; as a set of equations. I wanted to find the solution of &lt;em&gt;Ax=b&lt;/em&gt; where &lt;em&gt;b&lt;/em&gt; is vector of right-hand side. So I have called &lt;em&gt;zgetrf&lt;/em&gt; on &lt;em&gt;A&lt;/em&gt; (does LU factorization) and then &lt;em&gt;zgetrs&lt;/em&gt; on &lt;em&gt;A&lt;/em&gt; and &lt;em&gt;b&lt;/em&gt; (solves linear system of LU matrix) from Lapack/MKL. &#xA;But this ends up with error, &lt;em&gt;zgetrf&lt;/em&gt; can't factorize the matrix &lt;em&gt;A&lt;/em&gt; and says that it is singular. So I have called SVD on it - &lt;em&gt;zgesvd&lt;/em&gt;. And this works. But the eigenvalues of the matrix &lt;em&gt;A&lt;/em&gt; are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2885.24776463951,8.34234423485148)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1.41421356237310,1.41421356237310)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1.41421356237309,0.144427825453540)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1.257628062933306E-014,0.000000000000000E+000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.000000000000000E+000,0.000000000000000E+000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.000000000000000E+000,0.000000000000000E+000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.000000000000000E+000,0.000000000000000E+000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0.000000000000000E+000,0.000000000000000E+000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I find where is the problem for &lt;em&gt;zgetrf&lt;/em&gt; to factorize? I think such eigenvalues say that the matrix A is only of rank 4 in spite of having physical dimension 8 x 8... can I estimate which lines are dependent?&#xA;Thank you very much &lt;/p&gt;&#xA;" OwnerUserId="3688" LastActivityDate="2013-02-12T21:04:59.160" Title="Singular matrix - but SVD works - what does the eigenvalues mean? Find I the &quot;dependent&quot; lines?" Tags="|lapack|eigenvalues|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5258" PostTypeId="1" CreationDate="2013-02-12T21:04:17.153" Score="3" ViewCount="226" Body="&lt;p&gt;Consider the following optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Min \qquad  C^TX$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;S.t.: $\qquad   AX=0;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_ix_j=x_kx_t$$\quad $for some $i\neq j\neq k\neq t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X=(x_1,x_2,...x_n)$&#xA;and $\quad x_j\geq 0\;\;  j=1,2,...,n$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $C=(c_1,c_2,...,c_n), c_i\geq 0$ and $A$ is the adjacency matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Is this problem convex?&#xA;-Can it be solved in polynomial time?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="917" LastEditDate="2013-02-12T21:54:02.740" LastActivityDate="2013-06-21T11:49:34.267" Title="polynomial time solvability" Tags="|optimization|linear-programming|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5259" PostTypeId="2" ParentId="5257" CreationDate="2013-02-12T21:04:59.160" Score="3" Body="&lt;p&gt;I'm going to assume you've given us the singular values from SVD, not the eigenvalues (they are related, but not the same).  You are right: this means your matrix is rank 4 despite being an 8x8 matrix.  The SVD has already given you everything you need: the first four left and right singular vectors are a basis that represents your operator as a 4x4 matrix.  If the matrix A is static, then you &lt;strong&gt;could&lt;/strong&gt; transform your whole calculation in to that those bases and proceed with a non-singular matrix.  A better idea is probably to use the SVD to form a &lt;a href=&quot;http://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse&quot; rel=&quot;nofollow&quot;&gt;pseudo-inverse&lt;/a&gt; of $A$.  Recall the SVD of $A$:&#xA;$$ A = U \Sigma V^* $$&#xA;where $U$, $V$ are unitary and $\Sigma$ is diagonal with non-negative values (but some zeros, as in your case).  The diagonal elements of $\Sigma$ are ordered and called the singular values.  You can construct a pseudo-inverse as:&#xA;$$A^\dagger = V \Sigma^\dagger U^* \quad \Sigma^\dagger_{ii} = \Sigma_{ii}^{-1} \text{ if } \Sigma_{ii} &amp;gt; 0, \text{ else } 0 $$&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2013-02-12T21:04:59.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5261" PostTypeId="2" ParentId="5258" CreationDate="2013-02-13T00:44:17.610" Score="4" Body="&lt;p&gt;Nonlinear equality constraints involving continuous variables are not convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a counterexample for your problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose without loss of generality $x_{1}x_{2} - x_{3}x_{4} = 0$ is your nonlinear equality constraint, and let $S = \{(x_1, x_2, x_3, x_4, \ldots, x_{n}): x_1 x_2 - x_3 x_4 = 0\}$ be its associated feasible set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then $(6, 1, 3, 2, 0, \ldots, 0) \in S$, and $(1, 6, 3, 2, 0, \ldots, 0) \in S$, but $((6, 1, 3, 2, 0, \ldots, 0) + (1, 6, 3, 2, 0, \ldots, 0)) / 2 = (7/2, 7/2, 3, 2, 0, \ldots, 0) \not\in S$, because $(7/2)^2 - 6 = 49/4 - 6 = 25/4 \neq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, nonconvex problems cannot be solved in polynomial time.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-02-13T00:44:17.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5262" PostTypeId="1" AcceptedAnswerId="5264" CreationDate="2013-02-13T01:28:09.797" Score="5" ViewCount="975" Body="&lt;p&gt;I've been working on a finite element code on unstructured methods, which I've parallelized using the Schur complement method. Here's a summary of how I did it:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Assign each triangle of the mesh to a domain&lt;/li&gt;&#xA;&lt;li&gt;For each node, determine which domain it is in or whether it is on the artificial boundary&lt;/li&gt;&#xA;&lt;li&gt;Send each processor its part of the mesh -- the triangles it owns as well as any nodes it owns, even if it shares those nodes with some other domain&lt;/li&gt;&#xA;&lt;li&gt;Build the matrices, in parallel, on each processor&lt;/li&gt;&#xA;&lt;li&gt;Solve some stuff, using some way of exchanging node data between processors&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm solving a quasilinear system of elliptic PDE, so each linear solve is just an iteration of Picard's method. In particular, I need to change the entries of the stiffness matrix at every step, since it depends on the gradient of the solution. (The non-zero structure of the matrix doesn't change.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This all works just fine. However, it disagrees with the usual approach I see taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code divides up the triangles between processes, and some of them have to share nodes. This means a extra storage, but the entries of the stiffness matrix can be filled completely in parallel. On the other hand, libraries like PETSc and METIS divide up all the nodes into &lt;em&gt;disjoint&lt;/em&gt; sets and send them, along with their corresponding rows in the stiffness matrix, to each process. But, you have to communicate to fill the stiffness matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So: why do the big scientific libraries have a marked preference for dividing up the nodes instead of the elements? Am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2017-04-19T15:52:52.087" LastActivityDate="2021-01-13T03:22:27.607" Title="implicit vs. explicit domain decomposition methods" Tags="|linear-algebra|finite-element|partitioning|domain-decomposition|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5263" PostTypeId="2" ParentId="5262" CreationDate="2013-02-13T01:50:34.543" Score="2" Body="&lt;p&gt;Both are valid techniques with their own tradeoffs. PETSc, in particular, has a history as a linear algebra library, and given that, it makes sense for it to assign rows distinctly to processors. For finite element methods, which have element matrix formation, and often some sort of matrix assembly, it often makes sense to assign elements to uniquely to processors. Some codes do element matrix formation this way, then choose an assignment of nodes to processors before calling PETSc during matrix assembly and solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these techniques can be made to work.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-02-13T02:58:44.243" LastActivityDate="2013-02-13T02:58:44.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5264" PostTypeId="2" ParentId="5262" CreationDate="2013-02-13T03:22:12.737" Score="3" Body="&lt;p&gt;Partitioning by elements leads to minor simplifications in assembly at the expense of poor load balancing in the solver, inability to use many popular solution methods, and more communication to apply the operator. The most common reason to use an element partition is simply because the code to handle overlap or to communicate contributions into off-process rows during assembly has not been written. (PETSc will do the communication automatically.) It's rarely faster in the long run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you only assembly locally, you end up with &quot;Neumann problems&quot; on each subdomain. Since these problems have no boundary conditions, they are singular for differential equations with no zeroth order &quot;mass&quot; term. This requires use of much more complicated preconditioners, such as FETI-DP or BDDC, for which the performance is quite sensitive to the partition and that usually require special work if the character of the physical system changes. If you assemble to a vertex-based operator (like PETSc's default matrix formats), you can use many methods that are &quot;easier&quot; to use, such including additive Schwarz and algebraic multigrid. (Broadly speaking, &quot;easier&quot; means that they have a broader performance &quot;sweet spot&quot;, do not finnicky null-space issues in which local mesh problems can cause complete breakdown, have better diagnostics, and require less customization for new problems.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some people, including the 2004 Gordon Bell Prize winner, use a vertex partition everywhere and integrate redundantly in the overlapping elements to assemble the operator without communication. That is usually best for performance, but requires that the mesh be extended to include one cell of overlap, which may or may not be convenient in your code.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-02-13T03:22:12.737" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5265" PostTypeId="1" AcceptedAnswerId="5269" CreationDate="2013-02-13T12:21:30.290" Score="7" ViewCount="345" Body="&lt;p&gt;I am trying to find the required specifications of a RF trap, in which a proton can be confined.(trap dimensions, voltage frequency and amplitude used, etc). I have to solve the equations of motion numerically because the potential doesn't have a closed form for this specific geometry and equations of motion can not be solved analytically (indeed, only a series solution can be obtained for potential using Legendre polynomials and this series is present in the system of equations of motion). This is not the case for hyperbolic Paul trap where the equations are happen to be Mathieu equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this purpose, I solve this system of ODEs numerically (ode45--&gt; 4th and 5th order Runge-Kutta)  and see if the ion leaves the trap.  I can simulate the motion for a &lt;strong&gt;limited time span&lt;/strong&gt; (typically about some milliseconds at most). So if the ion happen to leave the trap after a longer time span, I can't notice that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The question is that how and when I can claim that the ion is confined in the trap? (using numerical methods)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations are: (in 2D polar coordinates with variables $r$ and $\theta$)(put some integer instead of $\infty$ in the series!)($C$ , $U$ , $V$ , $R$ and $\alpha$ are constants)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot{r}-r\dot{\theta}^{2}=C(U+V\cos({\omega}t))\sum_{n=0}^{\infty}n(4n+1)\biggr[\sum_{m=0}^{n} (-1)^{m}\frac{(4n-2m)!(1-(\cos(\alpha))^{2n-2m+1})}{4^{n}m!(2n-m)!(2n-2m+1)!}\biggr]\frac{r^{2n-1}}{R^{2n}}P_{2n}(\cos(\theta))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r\ddot{\theta}+2\dot{r}\dot{\theta}=C(U+V\cos({\omega}t))\sum_{n=0}^{\infty}(4n+1)\biggr[\sum_{m=0}^{n} (-1)^{m}\frac{(4n-2m)!(1-(\cos(\alpha))^{2n-2m+1})}{4^{n}m!(2n-m)!(2n-2m+1)!}\biggr]\frac{r^{2n-1}}{R^{2n}}{d \over d\theta}(P_{2n}(\cos(\theta)))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which with this assignment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X_1=r$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X_2=\theta$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X_3=\dot{r}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X_4=\dot{\theta}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;becomes a simultaneous system of ODEs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*} \scriptsize { \dot{X_1} }  &amp;amp;=  \scriptsize { X_3 }  \\ \scriptsize { \dot{X_2} } &amp;amp;= \scriptsize { X_4 } \\ \scriptsize { \dot{X_3} } &amp;amp; =\scriptsize { X_1X_4^{2}+C(U+V\cos({\omega}t))\sum_{n=0}^{\infty}n(4n+1)\biggr[\sum_{m=0}^{n} (-1)^{m}\frac{(4n-2m)!(1-(\cos(\alpha))^{2n-2m+1})}{4^{n}m!(2n-m)!(2n-2m+1)!}\biggr]\frac{r^{2n-1}}{R^{2n}}P_{2n}(\cos(X_2)) } \\ \scriptsize { \dot{X_4} } &amp;amp;= \scriptsize { \frac{-2X_3X_4}{X_1}+\frac{C(U+V\cos({\omega}t))\sum_{n=0}^{\infty}(4n+1)\biggr[\sum_{m=0}^{n} (-1)^{m} \frac{(4n-2m)!(1-(\cos(\alpha))^{2n-2m+1})}{4^{n}m!(2n-m)!(2n-2m+1)!}\biggr]\frac{r^{2n-1}}{R^{2n}}{d \over dX_2}(P_{2n}(\cos(X_2)))}{X_1} } \end{align*}&lt;/p&gt;&#xA;" OwnerUserId="3725" LastEditorUserId="19378" LastEditDate="2016-02-26T19:34:26.940" LastActivityDate="2016-02-26T19:34:26.940" Title="Numerical investigation of stability of motion (confinement)" Tags="|matlab|ode|stability|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5266" PostTypeId="2" ParentId="5265" CreationDate="2013-02-13T12:41:26.600" Score="2" Body="&lt;p&gt;If you do a finite simulation you can use two stopping criteria:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It leaves the trap (It is obviously not trapped)&lt;/li&gt;&#xA;&lt;li&gt;The status of the system repeats itself (It is trapped)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The second condition requires that randomness does not play a significant role in your simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With sufficient knowledge of the system you can add more stopping criteria:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The status of the system is in a range of predefined 'safe' states&lt;/li&gt;&#xA;&lt;li&gt;The status of the system is 'between' previous states (does that mean it is safe?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course you can add as many as you like, and may want to interpret the result as follows: a particle not leaving during a certain period, and also not meeting the stopping criteria is 'probably' trapped.&lt;/p&gt;&#xA;" OwnerUserId="3315" LastActivityDate="2013-02-13T12:41:26.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5267" PostTypeId="1" AcceptedAnswerId="5268" CreationDate="2013-02-13T14:07:36.603" Score="9" ViewCount="1607" Body="&lt;p&gt;I need to solve Ax=b, but I realize that even if it is sparse, storing the matrix coefficients of my problem will take too much memory.&#xA;So now I'm considering using a matrix-free method, because the same coefficients appear a lot of time in the matrix, so I could use my own private storage scheme (and increase cache efficiency by the way).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking at petsc, which provides interface for such matrix-free linear operators, but what I don't really understand, is how the preconditioner is then computed by petsc ?&#xA;Or should I provide my own preconditioner ? If so, are there tools or recipies available to construct preconditioner from a matrix-free linear operator ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information about my operator: it is unsymmetric, not diagonally dominant, but dominated by a few sidebands (but it is not banded diagonal either)&lt;/p&gt;&#xA;" OwnerUserId="3260" LastEditorUserId="20688" LastEditDate="2019-07-16T20:45:33.897" LastActivityDate="2019-07-16T20:45:33.897" Title="preconditioner for a matrix-free method to solve Ax=b" Tags="|linear-solver|petsc|sparse-operator|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5268" PostTypeId="2" ParentId="5267" CreationDate="2013-02-13T14:15:55.717" Score="8" Body="&lt;p&gt;You'll need to roll your own preconditioner. If you know the matrix, it should not be terribly difficult to implement something like an SSOR preconditioner, for example. If you know something else about the problem, for example that it comes from a PDE whose solution can be well approximated on a coarser mesh, then you can also consider constructing preconditioners by restricting the problem to the coarser mesh, solving there, and extending the solution back out to the original one. Restriction and prolongation can be implemented as matrix-free operations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-13T14:15:55.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5269" PostTypeId="2" ParentId="5265" CreationDate="2013-02-13T14:23:03.560" Score="3" Body="&lt;p&gt;In your case, you are designing a system in which the ion will undergo a periodic motion. It is true that there are systems in which even autonomous ODEs (i.e., ODEs in which the forcing term is not time dependent -- i.e., like in your case where the field is fixed) can give rise to chaotic motion, but I would be very surprised to see that in your system. Rather, you are almost certainly going to see closed orbits of the ion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For such systems, while it's not a guarantee that the orbit is stable, if you can observe a few hundred or thousand orbits and not see the ion leave the trap, then I imagine that that is good enough for practical purposes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-13T14:23:03.560" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5270" PostTypeId="2" ParentId="5262" CreationDate="2013-02-13T14:37:08.747" Score="0" Body="&lt;p&gt;Part of the disconnect you see is historical preference. Domain decomposition methods were popular in the 1990s when there were no linear algebra libraries around that handled the (rather complex) task of communicating matrix entries if you split vertices (or, rather, degrees of freedom) into disjoint sets; processor numbers on parallel machines were, at that time, also in the few dozens. If you had to write algorithms for parallel communication of data yourself on such small (by today's standards) machines, domain decomposition methods made good sense.&lt;/p&gt;&#xA;&lt;p&gt;But times have changed. Libraries such as PETSc handle the communication of matrix elements just fine, without user interaction. And machines today have hundreds, thousands, or more processors, and domain decomposition methods simply do not scale well to these kinds of machines. Consequently, domain decomposition methods that split on cells are not all that popular any more.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2021-01-13T03:22:27.607" LastActivityDate="2021-01-13T03:22:27.607" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5271" PostTypeId="1" CreationDate="2013-02-13T22:49:52.020" Score="5" ViewCount="241" Body="&lt;p&gt;On an uniform mesh of positions in space &lt;span class=&quot;math-container&quot;&gt;$\ (x_i,y_j,z_k)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\ x_i = x_0 + i\Delta x,\quad i=0,\ldots,n_x$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\ y_j = y_0 + j\Delta y,\quad j=0,\ldots,n_y$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\ z_k = z_0 + k\Delta z,\quad k=0,\ldots,n_z$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a velocity function: &lt;span class=&quot;math-container&quot;&gt;$V(x_i,y_j,z_k)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a start position &lt;span class=&quot;math-container&quot;&gt;$(x_s,y_s,z_s)$&lt;/span&gt; I can calculate the time it takes to reach all points in space: &lt;span class=&quot;math-container&quot;&gt;$T(x_i,y_j,z_k)$&lt;/span&gt; by &lt;a href=&quot;http://en.wikipedia.org/wiki/Fast_marching_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Fast Marching Method (FMM)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would like to do this on a mesh larger than can fit in RAM,&#xA;e.g. &lt;span class=&quot;math-container&quot;&gt;$n_x=n_y=n_z=2000$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FMM needs access to ALL velocity data. It is NOT possible to run it on only a part of the mesh and then later combine results somehow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data structure storing the velocity data can, however, be sparse. As long as the FMM can ask for the value at any &lt;span class=&quot;math-container&quot;&gt;$(x_i,y_j,z_k)$&lt;/span&gt; it does not care how it is stored internally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagine setting blocks of values at a time:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;struct SparseArray3D {&#xA;public:&#xA;    SparseArray3D(int nx, int ny, int nz);&#xA;    void setBlock(int i0, int j0, int k0, float block[64][64][64]); // i0 &amp;lt;= i &amp;lt; i0 + 64, j0 &amp;lt;= j &amp;lt; j0 + 64, k0 &amp;lt;= k &amp;lt; k0 + 64  &#xA;    float&amp;amp; operator()(int i, int j, int k); // needed by FMM&#xA;}; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do you have any tips on which data structure I could use?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(As you probably already have guessed) I am programming in C/C++. &lt;/p&gt;&#xA;" OwnerUserId="3622" LastEditorUserId="20688" LastEditDate="2019-06-06T19:32:46.143" LastActivityDate="2019-06-06T19:32:46.143" Title="Adaptive mesh data structure for Fast Marching Method to overcome RAM limit" Tags="|c++|sparse-matrix|adaptive-mesh-refinement|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5273" PostTypeId="1" CreationDate="2013-02-14T10:51:29.450" Score="1" ViewCount="649" Body="&lt;p&gt;Consider the following optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Min\;\;\; CX$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$AX\geq b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_ix_j= x_s x_t\;\;\;  i\neq j \neq s\neq t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_j\geq 0;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $A$ is the adjacency matrix and $C$ is a constant vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can one  check the feasibility of this optimization problem in an efficient way?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="917" LastEditDate="2013-02-14T15:28:45.953" LastActivityDate="2013-02-14T15:28:45.953" Title="Feasibility checking" Tags="|optimization|nonlinear-programming|convex-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5274" PostTypeId="2" ParentId="5273" CreationDate="2013-02-14T11:47:04.697" Score="4" Body="&lt;p&gt;You can't. Optimization and feasibility are equivalent problems, because optimization can be achieved by solving a sequence of feasibility problems.Consequently, both are in the same computational complexity class. &lt;a href=&quot;https://scicomp.stackexchange.com/a/5261/276&quot;&gt;In an answer to a similar question you posed&lt;/a&gt; I showed that your formulation is nonconvex. Determining feasibility such problems is generally an $\mathcal{NP}$-hard problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A deleted answer from that similar question noted that the zero vector is a feasible solution for your problem; if you impose the constraint that the objective must be less than or equal to $-\varepsilon$ for $\varepsilon &amp;gt; 0$, then the feasibility problem is nontrivial.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-02-14T11:47:04.697" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5275" PostTypeId="1" AcceptedAnswerId="5277" CreationDate="2013-02-14T19:54:02.530" Score="5" ViewCount="270" Body="&lt;p&gt;Suppose I'm solving $$\frac{d}{dx}\left(K(x)\frac{du}{dx}\right)=f \text{ in }\Omega,$$ $$u=g \text{ on } \partial\Omega$$where $K(x)$ is smooth  and &#xA;$$&#xA;f(x) = \left\{&#xA;        \begin{array}{ll}&#xA;            0 &amp;amp; \quad x \neq x_0 \\&#xA;            1 &amp;amp; \quad x =x_0&#xA;        \end{array}&#xA;    \right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried solving this problem by a finite difference method with f as given, but I noticed some strange behavior in the solution as the number of discrete nodes gets larger.  I'm pretty sure the strange behavior is due to the discontinuity at $x_0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I propose to replace $f$ by a function $f_\epsilon$ such that $f_\epsilon=0$ everywhere except in the small region $B_\epsilon=\{x\text{ s. t. }|x-x_0|&amp;lt;\epsilon\}$.  In $B_\epsilon$, $f_\epsilon$ is a polynomial such that  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} &#xA;f_\epsilon(x_0)=1 &amp;amp; \quad f_\epsilon'(x_0)=0 \\ &#xA;f_\epsilon(x_0)=0 &amp;amp; \quad f_\epsilon'(x_0-\epsilon)=0 \\&#xA;f_\epsilon(x_0+\epsilon)=0 &amp;amp; \quad f_\epsilon'(x_0+\epsilon)=0&#xA;\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $f_\epsilon=0$ throughout the rest of the domain.&#xA;This gives me a polynomial of degree 5 and ensures $f_\epsilon\in C^1(\Omega)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I suspect that since finite difference methods seek classical solutions, we probably need more differentiability; that is, $f_\epsilon\in C^2(\Omega)$.  It is not immediately obvious to me what other conditions I can/should impose on $f_\epsilon$ to ensure this extra regularity.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I pose the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does $f_\epsilon$ need to be in $C^2(\Omega)$?  &lt;/li&gt;&#xA;&lt;li&gt;What other conditions can I impose on my polynomial to ensure this extra regularity?&lt;/li&gt;&#xA;&lt;li&gt;Are other (non-polynomial) functions better suited to approximate the discontinuous source term?&lt;/li&gt;&#xA;&lt;li&gt;To observe the true behavior of the original discontinuous source term, should I solve a sequence of regularized problems with $f_\epsilon$ such that $\epsilon\rightarrow 0$?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-02-16T12:28:13.933" Title="Regularization of a discontinuous source term in an elliptic pde" Tags="|pde|finite-difference|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5277" PostTypeId="2" ParentId="5275" CreationDate="2013-02-14T21:12:26.340" Score="4" Body="&lt;p&gt;A more typical regularization would be to use&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_\epsilon (x) = e^{1-\frac{1}{1-((x-x_0)/\epsilon)^2}},\, {\rm for }\, |x-x_0|&amp;lt;\epsilon&#xA;$$&#xA;and zero otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This bump function is $C^\infty$, and you can control its width and height arbitrarily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regard to your questions, $f$ would typically need to be less regular than your solution $u$, so that's not your problem. And, yes, you would want to solve a sequence of problems, but once you have no grid points inside $|x-x_0| &amp;lt; \epsilon$ besides the one at $x_0$, there's no point in shrinking $\epsilon$ any more. You'd need to refine the mesh at that point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, it's kind of an odd forcing function. Are you sure you didn't mean to use a delta function? &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-02-14T21:12:26.340" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5278" PostTypeId="2" ParentId="5208" CreationDate="2013-02-15T13:28:24.313" Score="7" Body="&lt;p&gt;Convergence is a statement about the asymptotic limit $h\to0$.  Therefore, it's not possible to &quot;prove&quot; convergence by any finite number of computations.  The best you can do with computations is motivate the belief that an algorithm does converge.  To prove convergence, you need to...write a proof.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-02-15T13:28:24.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5280" PostTypeId="2" ParentId="5182" CreationDate="2013-02-15T16:23:24.993" Score="0" Body="&lt;p&gt;Here is a brief overview of my current implementation, I'll try to stick to the concepts and not go into language details. I used the &lt;a href=&quot;http://paradiseo.gforge.inria.fr/&quot; rel=&quot;nofollow&quot;&gt;ParadisEO Framework&lt;/a&gt; which is C++ template library for genetic algorithms and added some boost here and there.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Climbing hold colors&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The hold colors are stored in an XML file as a pair of color name and amount. The amounts found in a file are added up and normalized. This makes it possible to express an amount as either a total count of routes that can be set with this color or as a percentage. An ID is assigned to each color, starting with 1. Zero is reserved for an &quot;empty&quot; route.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Color Combination Penalties&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Some colors don't go well together in one sector or in adjacent sectors. Every not-so-good color combination is described by the two color names (as in the previous XML file, see above) and an arbitrary &quot;badness&quot;. Internally, the badness values are stored in a matrix which is indexed by color IDs.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The Wall&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The Wall is a class that derives from ParadisEO's genome representation class so that ParadisEO can manipulate and evaluate it, but some more functionality is added. Each gene represents one route color by ID (including zero, or empty). Sectors are represented by a pair of indices to the genes, so that each sector has a beginning and an end. I used iterators to the genes first, but the Wall object has to be copy constructible, which would invalidate iterators without additional work. Currently, all sectors have 4 routes, but that will be configurable in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Wall also has a color bucket. This bucket contains a counter for each color, distributed as described in the color XML file. The color counts add up to the total number of routes in the Wall. A color can be picked from the bucket, decreasing the counter, and it can also be put back, increasing the counter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A color can only be added to a sector if the sector doesn't already contain that color (the sector must remain &quot;legal&quot; when the color is added).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Evaluation Operator&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The evaluation operator sums up all badness values in a sector using the badness matrix described above. Each sector's value is stored in the fitness vector (that's also part of the Wall class), so this is a multi-objective GA. I might change that if it becomes necessary.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Two Point Crossover Operator&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The crossover operator takes two parents, creates a copy of each (the offsprings) and then performs a two-point crossover by recombining &lt;em&gt;whole sectors&lt;/em&gt;. The advantage of this is that sectors remain legal (no double colors). The disadvantage of any crossover operation (for this problem) is that the resulting offsprings can contain excess colors if colors were clustered in the parents. A repair function was added that removes excess routes (color zero) randomly. The offsprings therefore have less routes than the parents of the repair changed the offspring.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Mutation 1: Add route&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The potential decrease in routes in the offsprings is accounted for by an &quot;add route&quot; operator. It takes a color from the Wall's bucket and adds it somewhere. If that's not a legal operation, it does nothing (sometimes there is no legal place for a remaining color).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Mutation 2: Random exchange&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Two random routes of two random sectors are swapped. This pair of swapping is generated until the resulting sectors are legal, then the swap is executed.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Mutation 3: Shift&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A number of sectors is shifted one place to the left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm out of time now, but I might add more information later. Especially the evaluation is quite simple as it is, and the components were programmed as a proof of conecpt. The GA indeed does optimize the Wall to some extent, but the results are not ready for real use. I'm sure this will improve when I've talked to the route setters and more rules for evaluation are available. Pictures will also come when they are available.&lt;/p&gt;&#xA;" OwnerUserId="3680" LastActivityDate="2013-02-15T16:23:24.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5281" PostTypeId="1" CreationDate="2013-02-15T17:21:25.133" Score="0" ViewCount="34" Body="&lt;p&gt;Consider the following optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Min$_{x}$ $\qquad  \sum_{(i,j,t,s)\in I_r}||x_ix_j-x_tx_s||^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;S.t.: $\qquad  x\in \mathcal{C} ;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x=(x_1,x_2,...x_n)$&#xA;and $\quad x_j\geq 0\;\;  j=1,2,...,n$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $\mathcal{C}$ is a convex set and $I_r$ is a polynomial sized index set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this problem be solved in polynomial time?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="276" LastEditDate="2013-02-15T17:41:38.070" LastActivityDate="2013-02-15T17:47:25.653" Title="Polynomial solvability" Tags="|optimization|nonlinear-programming|convex-optimization|" AnswerCount="1" CommentCount="3" ClosedDate="2013-02-15T17:52:22.457" ContentLicense="CC BY-SA 3.0" />
  <row Id="5283" PostTypeId="1" CreationDate="2013-02-15T04:54:31.950" Score="3" ViewCount="159" Body="&lt;p&gt;If I want to do a very accurate simulation of a molecular system (e.g. 2 hydrogen atoms), then I'll want to use something like diffusion Monte Carlo to determine the energies of these atoms in different configurations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I want to create a bonding potential like the harmonic&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E = K(r - r_0)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or Morse potential&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E = D[1-e^{-\alpha(r-r_0)}]^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then I assume what I would do is perform DMC to find the energies for various separation distances, and then perform a regression to fit one of those equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I was recently thinking perhaps this isn't as accurate as it could be, because you're using quantum mechanics to find energies, and then using classical physics (Newton's laws of motion) to evolve the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more accurate alternative, I assume, would be to evolve a system of two hydrogen atoms using the time-dependent Schrödinger equation and then somehow fit this to some easily computable model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm curious though is how the resulting simulations would differ?  I don't know enough about QM to speculate on what kind of effects the classical system evolution would be missing, or exactly how much (numerically speaking), the two approaches would differ.  Does anyone have any idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose I could always try both methods and see, but that's going to take quite a bit of work...&lt;/p&gt;&#xA;" OwnerUserId="1997" OwnerDisplayName="Nick" LastActivityDate="2013-03-22T21:49:04.230" Title="What is the difference in accuracy between fully QM atomic simulations vs QM + classical?" Tags="|quantum-mechanics|molecular-dynamics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5284" PostTypeId="2" ParentId="5283" CreationDate="2013-02-15T05:15:45.983" Score="0" Body="&lt;p&gt;(This is more like a comment but I don't have enough reputation to comment and is kind of long; however this may be useful for you if I am understanding your question). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are only interested in calculating a bonding like potential for a two-electron system such as two hydrogen atoms, that can be done exactly (within the Born-Oppenheimer approx. and for a given basis set, $i.e.$ set of orbitals) using standard quantum chemistry methods. The methods of full configuration interaction (&lt;a href=&quot;http://en.wikipedia.org/wiki/Full_configuration_interaction&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Full_configuration_interaction&lt;/a&gt;), CISD (&lt;a href=&quot;http://www.gaussian.com/g_tech/g_ur/k_cid.htm&quot; rel=&quot;nofollow&quot;&gt;http://www.gaussian.com/g_tech/g_ur/k_cid.htm&lt;/a&gt;) or Coupled Cluster CCSD (&lt;a href=&quot;http://www.gaussian.com/g_tech/g_ur/k_ccd.htm&quot; rel=&quot;nofollow&quot;&gt;http://www.gaussian.com/g_tech/g_ur/k_ccd.htm&lt;/a&gt;) would give all exact answers for two electron systems. You can calculate the energy at different bond lengths with these methods using standard quantum chemistry programs. You can then use a program such a gnuplot to fit the obtained point to your harmonic or Morse potential equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The methods that I am mentioning above are computationally very expensive, but for your small two electron system it is not a problem to do many calculations at different bond lengths with any modern computer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps, cause I am not sure if I understood perfectly your question. Your question talked about mixing QM with classical methods, but I am not mentioning about anything about the latter because you can get the &quot;exact&quot; quantum answer for your small system (and you said you wanted to do a very accurate calculation). &lt;/p&gt;&#xA;" OwnerDisplayName="Goku" LastActivityDate="2013-02-15T05:20:48.147" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5286" PostTypeId="2" ParentId="5281" CreationDate="2013-02-15T17:47:25.653" Score="0" Body="&lt;p&gt;In general, no.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without loss of generality, suppose that $I_r = \{(1, 2, 3, 4)\}$, and let $F(x_{1}, x_{2}, x_{3}, x_{4} = \|x_{1}x_{2} - x_{3}x_{4}\|^2$. Consider the points $(1, 3, 0, 0)$ and $(3, 1, 0, 0)$; their midpoint is $(2, 2, 0, 0)$. If $F$ were convex, then it would be true that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;F(2, 2, 0, 0) \leq \frac{1}{2}F(1, 3, 0, 0) + \frac{1}{2}F(1, 3, 0, 0).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;16 \not\leq \frac{1}{2} \cdot 9 + \frac{1}{2} \cdot 9.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, nonconvex problems are $\mathcal{NP}$-hard.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-02-15T17:47:25.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5287" PostTypeId="1" CreationDate="2013-02-15T19:32:31.590" Score="5" ViewCount="365" Body="&lt;p&gt;I am working with Modified Nodal Admittance representation of circuits. I am doing Model Order Reduction using &lt;a href=&quot;http://pdf.aminer.org/000/282/897/prima_passive_reduced_order_interconnect_macromodeling_algorithm.pdf&quot; rel=&quot;noreferrer&quot;&gt;PRIMA&lt;/a&gt; on MATLAB. I am considering these circuits as &lt;a href=&quot;http://www.weizmann.ac.il/matlab/toolbox/control/manipmod/ltisys12.html&quot; rel=&quot;noreferrer&quot;&gt;Descriptor State-Space&lt;/a&gt; systems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been working with this for quite a while. It is a guaranteed passivity preserving algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for some circuit systems, the implementation I have misbehaves. Although I pass in a &lt;em&gt;stable&lt;/em&gt; model, it returns an &lt;em&gt;unstable&lt;/em&gt; model(!). So I tested the input and output in multiple ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I have checked :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The projection matrix Q : Orthogonal, with the error ~e-15.&lt;/li&gt;&#xA;&lt;li&gt;Checked the poles using MATLAB's &lt;code&gt;pole()&lt;/code&gt; : It showed me exactly one unstable pole pair whose &lt;code&gt;Re&lt;/code&gt; is ~e+11, so I don't see it crossing over the imaginary axis in any way.&lt;/li&gt;&#xA;&lt;li&gt;Checked the poles using &lt;code&gt;eig(-G\C)&lt;/code&gt; &amp;mdash; the poles are actually the inverse of these values, i.e. of those which are non-zero : shows a SINGLE pole (not pair) at ~e-19.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;hsvd()&lt;/code&gt; : shows a couple of unstable modes&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;isstable()&lt;/code&gt; : states that its not stable&lt;/li&gt;&#xA;&lt;li&gt;A custom checking function (from my senior student guide) : 1 unstable pole.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The poles of the original system are also ~e+11. The original is absolutely stable (and passive I think (transmission line), but that's just my speculation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am really confused. What is the difference between &lt;code&gt;pole()&lt;/code&gt; and &lt;code&gt;eig(-G\C)&lt;/code&gt;? And that one pole according to the latter seems quite clearly the real problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E(or C) is symmetric positive definite, and A(or G) satisfies G+G^T to ~e-15. However, B != C^T, the algorithm I'm working on is specifically targeted towards inputs != outputs scenario. I have repeatedly enquired and searched, but haven't seen anything about this condition being necessary &amp;ndash; the authors just assumed simply this condition for simplicity, as far as I know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope someone can give me further suggestions on what to check for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I find that this is happening for only some values of reduced order model. For others, &lt;code&gt;isstable()&lt;/code&gt; reports unstable and &lt;code&gt;eig(-G\C)&lt;/code&gt; says all clear. For yet others, &lt;code&gt;isstable()&lt;/code&gt; reports stable, but &lt;code&gt;hsvd()&lt;/code&gt; still finds unstable modes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't expect people to help me debug this exact issue (that would be asking too much), but I want some help with identifying known gotchas, and discovering unknown ones.&lt;/p&gt;&#xA;" OwnerUserId="3740" LastEditorUserId="3740" LastEditDate="2013-02-16T01:27:37.190" LastActivityDate="2013-04-28T22:37:32.627" Title="PRIMA gives an unstable result?" Tags="|matlab|krylov-method|linear-system|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5288" PostTypeId="2" ParentId="5287" CreationDate="2013-02-15T23:20:48.787" Score="0" Body="&lt;p&gt;The situation is not clear enough. Do you mean the original model is stable? But then it has one unstable pole! If this is the pole of the reduced model, check the eigenvalue of the original model which has smallest real value. It is expected to be close to the imaginary axis. Perhaps the computational error shifts it to the other side of imaginary axis and makes the reduced model unstable.&#xA;Theoretically, a passive system is a stable system. PRIMA preserves the passivity means it will keep the reduced model stable. It might happen that the original is not (numerically) passive (E=E^T&gt;=0, A^T+A &amp;lt;=0, B^T=C). &lt;/p&gt;&#xA;" OwnerUserId="3741" LastActivityDate="2013-02-15T23:20:48.787" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5289" PostTypeId="1" AcceptedAnswerId="5290" CreationDate="2013-02-16T01:27:51.560" Score="12" ViewCount="2593" Body="&lt;p&gt;I am using MATLAB to solve a problem that involves solving $\mathbf{A} \mathbf{x}=\mathbf{b}$ at every timestep, where $\mathbf{b}$ changes with time.  Right now, I am accomplishing this using MATLAB's &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/mldivide.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;mldivide&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = A\b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have the flexibility to make as many precomputations as needed, so I am wondering if there is a faster and/or more accurate method than &lt;code&gt;mldivide&lt;/code&gt;.  What is typically done here?  Thanks all!&lt;/p&gt;&#xA;" OwnerUserId="3742" LastEditorUserId="20417" LastEditDate="2018-03-15T16:49:19.737" LastActivityDate="2018-03-15T16:49:19.737" Title="Repeatedly solving $\mathbf{A} \mathbf{x} = \mathbf{b}$ with same $\mathbf{A}$, different $\mathbf{b}$" Tags="|linear-algebra|matlab|linear-solver|matrix|linear-system|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5290" PostTypeId="2" ParentId="5289" CreationDate="2013-02-16T01:53:31.317" Score="14" Body="&lt;p&gt;The most obvious thing you can do is to precompute &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;[L,U] = lu(A)&lt;/code&gt; ~ O(n^3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you just compute &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;x = U \ (L \ b)&lt;/code&gt; ~ O(2 n^2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would reduce the cost enormously and make it faster. Accuracy would be the same.&lt;/p&gt;&#xA;" OwnerUserId="3740" LastActivityDate="2013-02-16T01:53:31.317" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5291" PostTypeId="2" ParentId="5289" CreationDate="2013-02-16T02:59:44.900" Score="3" Body="&lt;p&gt;Take a look at &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1001/how-does-the-matlab-backslash-operator-solve-ax-b-for-square-matrices&quot;&gt;this question&lt;/a&gt;, the answers show that &lt;code&gt;mldivide&lt;/code&gt; is quite clever, and also gives suggestions as to how to see what Matlab uses to solve &lt;code&gt;A\b&lt;/code&gt;. This may give you a hint regarding optimization options.&lt;/p&gt;&#xA;" OwnerUserId="20" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-02-16T02:59:44.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5292" PostTypeId="2" ParentId="5289" CreationDate="2013-02-16T07:26:11.490" Score="0" Body="&lt;p&gt;Using backslash is more or less equivalent to &lt;code&gt;inv(A)*B&lt;/code&gt;, if you're coding it freely, the latter might be more intuitive. They're about the same (just different in how the computation is carried out), though you should check Matlab documentation for clarification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question, backslash is generally fine, but it does depend on the properties of the mass matrix.&lt;/p&gt;&#xA;" OwnerUserId="3493" LastEditorUserId="276" LastEditDate="2013-02-17T07:55:03.053" LastActivityDate="2013-02-17T07:55:03.053" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5293" PostTypeId="1" AcceptedAnswerId="5296" CreationDate="2013-02-16T08:02:59.863" Score="5" ViewCount="313" Body="&lt;p&gt;I've spent far too much time coding and debugging multigrid. While I clearly can't post all of my code as it would be silly to ask someone to go through all that code, is there anything I should pay special attention to?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I'm posting this because I don't want people to have to waste the time I had. This may not be completely relevant to computational science, but I thought it could be helpful for any beginners (as I certainly still am one).&lt;/p&gt;&#xA;" OwnerUserId="3493" LastEditorUserId="3493" LastEditDate="2013-02-16T08:18:13.910" LastActivityDate="2013-02-16T12:30:18.690" Title="I'm having trouble debugging multigrid. What to do?" Tags="|finite-element|finite-difference|matlab|multigrid|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5295" PostTypeId="2" ParentId="5208" CreationDate="2013-02-16T10:14:29.653" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I STARTED with an assumption that the method is second order&#xA;  and therefore to see 4 as a ratio is only necessary, but not&#xA;  sufficient to claim that the algorithm in fact is of second order.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think the &lt;em&gt;proof&lt;/em&gt; that a given discretization method is second-order accurate takes place on paper (assumptions, theorems and such). Numerical experiments then confirm and illustrate the proof. More importantly, they show whether the implementation of the method in a particular piece of software is correct (so-called &lt;em&gt;Code Verification&lt;/em&gt;).&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-02-16T10:14:29.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5296" PostTypeId="2" ParentId="5293" CreationDate="2013-02-16T10:26:16.957" Score="5" Body="&lt;p&gt;The smoother, restriction and prolongation can be tested independently. In essence, take a periodic domain, put in a particular Fourier mode and see if the code behaves in accordance with Fourier analysis. Also, in Briggs, Henson and McCormick's &lt;em&gt;A Multigrid Tutorial&lt;/em&gt; you will find a section called Diagnostic tools (Chapter 4). That should help you with systematic debugging.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-02-16T10:26:16.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5297" PostTypeId="2" ParentId="5275" CreationDate="2013-02-16T12:28:13.933" Score="2" Body="&lt;p&gt;That can't be the problem you want to solve. Given your right hand side has nonzero values only at $x=x_0$ but is finite there, you have that $\|f\|_{L^2}=0$. On the other hand, if you had zero boundary values, you have the stability estimate $\|\bar u\|_{H^1}\le C\|f\|_{L^2}$ where $\bar u$ is the solution with zero boundary values; in other words, it is zero. The solution to your problem is then the same as to a problem with a zero right hand side (i.e., in 1d, a linear function connecting the boundary values at the left and right).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That you don't converge to this solution is an artifact resulting from the fact that the finite difference method wants to sample point values of the right hand side, but that these are not defined for $L^2$ functions. You can't resolve this by mollification because every appropriately mollified function of zero $L^2$ norm also needs to have zero $L^2$ norm -- i.e., the only useful mollification would be the zero function.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-16T12:28:13.933" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5298" PostTypeId="2" ParentId="5293" CreationDate="2013-02-16T12:30:18.690" Score="0" Body="&lt;p&gt;We have collected a great deal of experience on how to debug finite element codes here:&#xA;  &lt;a href=&quot;https://code.google.com/p/dealii/wiki/FrequentlyAskedQuestions&quot; rel=&quot;nofollow&quot;&gt;https://code.google.com/p/dealii/wiki/FrequentlyAskedQuestions&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-16T12:30:18.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5301" PostTypeId="1" CreationDate="2013-02-16T22:02:54.133" Score="2" ViewCount="76" Body="&lt;p&gt;This question may be too broad. But I really want to know some concrete explanations.  I often find various commutations appear here and there, which concerns the application order of two operators.  For examples, between exterior calculus of the differential forms and discretization,  optimization-then-discretization or discretization-then-optimization, and inner-outer iterations.  Do you have some understanding of concrete examples where the commutation plays an important role? &lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2013-02-17T07:52:49.643" Title="What are Implications of Commutative Diagrams?" Tags="|optimization|iterative-method|discretization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5302" PostTypeId="1" CreationDate="2013-02-17T05:16:47.957" Score="12" ViewCount="1412" Body="&lt;p&gt;I have a collection of computational models that could be described as asynchronous cellular automata. These models resemble the Ising model, but are slightly more complicated. It seems as if such models would benefit from being run on a GPU rather than a CPU. Unfortunately it isn't quite straightforward to parallelise such a model, and it isn't at all clear to me how to go about it. I'm aware that there is literature on the subject, but it all seems to be aimed at hardcore computer scientists who are interested in the details of algorithmic complexity, rather than someone like me who just wants a description of something I can implement, and consequently I find it rather inpenetrable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For clarity, I'm not looking for an optimal algorithm so much as something I can rapidly implement in CUDA that's likely to give a significant speedup over my CPU implementation. Programmer time is much more of a limiting factor than computer time in this project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should also clarify that an &lt;em&gt;asynchronous&lt;/em&gt; cellular automaton is a rather different thing from a synchronous one, and techniques for parallelising synchronous CAs (such as Conway's life) cannot easily be adapted to this problem. The difference is that a synchronous CA updates every cell simultaneously at every time step, whereas an asynchronous one updates a randomly chosen local region at every time step as outlined below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The models I wish to parallelise are implemented on a lattice (usually a hexagonal one) consisting of ~100000 cells (though I'd like to use more), and the non-parallelised algorithm for running them looks like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Choose a neighbouring pair of cells at random&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate an &quot;energy&quot; function $\Delta E$ based on a local neighbourhood surrounding these cells&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With a probability that depends on $e^{-\beta \Delta E}$ (with $\beta$ a parameter), either swap the states of the two cells or do nothing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat the above steps indefinitely.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are also some complications to do with the boundary conditions, but I imagine these won't pose much difficulty for parallelisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth mentioning that I'm interested in the transient dynamics of these systems rather than just the equilibrium state, so I need something that has equivalent dynamics to the above, rather than just something that will approach the same equilibrium distribution. (So variations of the chequerboard algorithm are not what I'm looking for.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difficulty in parallelising the above algorithm is collisions. Because all the calculations depend only on a local region of the lattice, it's possible for many lattice sites to be updated in parallel, as long as their neighbourhoods aren't overlapping. The question is how to avoid such overlaps. I can think of several ways, but I don't know which if any is the best one to implement. These are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the CPU to generate a list of random grid sites and check for collisions. When the number of grid sites equals the number of GPU processors, or if a collision is detected, send each set of coordinates to a GPU unit to update the corresponding grid site. This would be easy to implement but probably wouldn't give much of a speed up, since checking for collisions on the CPU would probably not be all that much cheaper than doing the whole update on the CPU.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Divide the lattice up into regions (one per GPU unit), and have one GPU unit responsible for randomly selecting and updating grid cells within its region. But there are many issues with this idea that I don't know how to solve, the most obvious being what exactly should happen when a unit chooses a neighbourhood overlapping the edge of its region.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Approximate the system as follows: let time proceed in discrete steps. Divide the lattice up into a &lt;em&gt;different&lt;/em&gt; set of regions on every time step according to some pre-defined scheme, and have each GPU unit randomly select and update a pair of grid cells whose neighbourhood does not overlap the region's boundary. Since the boundaries change every time step this constraint might not affect the dynamics too much, as long as the regions are relatively large. This seems easy to implement and likely to be fast, but I don't know how well it will approximate the dynamics, or what is the best scheme for choosing the region boundaries on each time step. I found some references to &quot;block-synchronous cellular automata&quot;, which may or may not be the same as this idea. (I don't know because it seems that all the descriptions of the method are either in Russian or are in sources to which I don't have access.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My specific questions are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Are any of the above algorithms a sensible way to approach GPU parallelisation of an asynchronous CA model?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a better way?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there existing library code for this type of problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Where can I find a clear English-language description of the &quot;block-synchronous&quot; method?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Progress&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe I have come up with a way to parallelise an asynchronous CA that might be suitable. The algorithm outlined below is for a normal asynchronous CA that updates only one cell at a time, rather than a neighbouring pair of cells as mine does. There are some issues with generalising it to my specific case, but I think I have an idea how to resolve them. However, I'm not sure how much of a speed benefit it will give, for reasons discussed below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to replace the asynchronous CA (henceforth ACA) with a stochastic synchronous CA (SCA) that behaves equivalently. To do this we first imagine that the ACA is a Poisson process. That is, time proceeds continuously, and each cell as a constant probability per unit time of performing its update function, independently of the other cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We construct an SCA whose cells each store two things: the &lt;em&gt;state&lt;/em&gt; $X_{ij}$ of the cell (i.e. the data that would be stored in each cell in the sequential implementation), and a floating point number $t_{ij}$ representing the (continuous) &lt;em&gt;time&lt;/em&gt; at which it will next update. This continuous time does not correspond to the update steps of the SCA. I will refer to the latter as &quot;logical time&quot;. The time values are initialised randomly according to an exponential distribution: $t_{ij}(0) \sim \operatorname{Exp}(\lambda)$. (Where $\lambda$ is a parameter whose value can be chosen arbitrarily.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each logical time step, the cells of the SCA are updated as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If, for any $k, l$ in the neighbourhood of $i,j$, the time $t_{kl}&amp;lt;t_{ij}$, do nothing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Otherwise, (1) update the state $X_{ij}$ according to the states $X_{kl}$ of the neighbouring cells, using the same rule as the original ACA; and (2) generate a random value $\Delta t \sim \operatorname{Exp}(\lambda)$ and update $t_{ij}$ to $t_{ij}+\Delta t$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I believe this guarantees that the cells will be updated in an order that can be &quot;decoded&quot; to correspond to the original ACA, while avoiding collisions and allowing some cells to be updated in parallel. However, because of the first bullet point above, it means that most of the GPU processors will be mostly idle on each time step of the SCA, which is less than ideal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to give some more thought to whether the performance of this algorithm can be improved, and how to extend this algorithm to deal with the case where multiple cells are updated simultaneously in the ACA. However, it looks promising so I thought I would describe it here in case anyone (a) knows of anything similar in the literature, or (b) can offer any insight into these remaining issues.&lt;/p&gt;&#xA;" OwnerUserId="3746" LastEditorUserId="3746" LastEditDate="2013-02-25T08:39:40.553" LastActivityDate="2013-03-02T22:57:36.277" Title="Parallel (GPU) algorithms for asynchronous cellular automata" Tags="|parallel-computing|monte-carlo|gpu|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5303" PostTypeId="2" ParentId="5289" CreationDate="2013-02-17T07:48:20.583" Score="5" Body="&lt;p&gt;We did some extensive computer labs in our scientific computing courses about this topic. For the &quot;small&quot; calculations we did there, Matlab's backslash operator was always faster than anything else, even after we had optimized our code as much as possible and re-orded all matrices beforehand (for instance with Reverse Cuthill McKee ordering for sparse matrices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check out one of our &lt;a href=&quot;http://www.csc.kth.se/utbildning/kth/kurser/DN2222/nummet2-12/lab211.pdf&quot;&gt;lab instructions&lt;/a&gt;. The answer to your question is covered (shortly) on page 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good book on the topic is written for instance by &lt;a href=&quot;http://www.amazon.co.uk/Numerical-Mathematics-Computing-ISE-Cheney/dp/0495384720&quot;&gt;Cheney&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2013-02-17T07:48:20.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5305" PostTypeId="1" CreationDate="2013-02-17T16:24:12.790" Score="2" ViewCount="417" Body="&lt;p&gt;I have a 3D point cloud representing ad object. I use a 3D cylinder to fit this object in the point cloud, so I check if each point is inside the cylinder and, if it is, then I assign a weight to that point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, at the end I have an estimate of how many points fall inside the cylinder and what is the weight of the fitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I would like to normalize the fitting dividing for the cylinder volume but this is in decimeter so the proportion I think it is not right. I think I can't count and weight points and divide them for a metric number. I think that I have to transform the metric volume into the &quot;number of points that the cylinder can contain&quot;. This is just an idea of mine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how can I do so? Do you have any other idea?&lt;/p&gt;&#xA;" OwnerUserId="3644" LastEditorUserId="350" LastEditDate="2013-03-20T18:21:03.477" LastActivityDate="2014-02-13T23:28:25.797" Title="how to know cylinder volume in pixels?" Tags="|matlab|data-sets|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5306" PostTypeId="1" AcceptedAnswerId="5307" CreationDate="2013-02-18T06:39:19.660" Score="6" ViewCount="2532" Body="&lt;p&gt;I need to compute the (Moore-Penrose) pseudoinverse of fixed-size 3x3 matrices. I would prefer to have a simple method without bringing in the industrial strength machinery of Lapack. Are there any simple methods to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: I should add two notes: first, I am prioritizing robustness and simplicity over speed. Second, I really only need the action of the pseudo-inverse on a 3x1 vector.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2013-02-18T07:10:45.967" LastActivityDate="2013-02-20T17:11:49.787" Title="Computing the pseudoinverse of a 3x3 matrix" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5307" PostTypeId="2" ParentId="5306" CreationDate="2013-02-18T08:56:50.437" Score="6" Body="&lt;p&gt;The book &lt;em&gt;Generalized Inverses: Theory and Applications&lt;/em&gt;, Second Edition, by Ben-Israel and Greville states the following (Chapter 1, Theorem 5, page 48):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theorem&lt;/strong&gt;: If $A$ is an $m$ by $n$ complex matrix of rank $r &amp;gt; 0$, and has the full rank factorization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A = XY,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then its Moore-Penrose ($\{1,2,3,4\}$-)generalized inverse, $A^{\dagger}$ is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A^{\dagger} = Y^{*}(X^{*}AY^{*})^{-1}X^{*},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the asterisk denotes Hermitian transpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like many theoretical results, this result is amenable to proof, but not immediately useful for computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In numerical analysis, there's no routine for &quot;full rank factorization&quot;. Rather, you get that information using the SVD or a QR factorization. Consequently, you are much better off using LAPACK to calculate one of those two well-known factorizations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that the SVD of $A$ is $A = U\Sigma V^{*}$. Suppose that $\Sigma = \mathrm{diag}(\sigma_{1}, \ldots, \sigma_{n})$. Define $\Sigma_{\varepsilon}^{-} = \mathrm{diag}(\sigma_{1}^{-}, \ldots, \sigma_{n}^{-})$, where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\sigma_{i}^{-} = \left\{\begin{array}{cc}\sigma_{i}^{-1}, &amp;amp; \textrm{if $|\sigma_{i}| \geq \varepsilon$,} \\ 0, &amp;amp; \textrm{if $|\sigma_{i}| &amp;lt; \varepsilon$,}\end{array}\right.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where singular values with magnitude less than $\varepsilon &amp;gt; 0$ are treated as &quot;numerically zero&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A^{\dagger} = V\Sigma_{\varepsilon}^{-}U^{*}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method is not simple, but it will be more robust, and it will be correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Two common formulas for the Moore-Penrose generalized inverse are given on &lt;a href=&quot;https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If $A$ is an $m$ by $n$ matrix with &lt;em&gt;linearly independent columns&lt;/em&gt; (so $m \geq n$), then&#xA;\begin{align}&#xA;A^{\dagger} = (A^{*}A)^{-1}A^{*}.&#xA;\end{align}&lt;/li&gt;&#xA;&lt;li&gt;If $A$ is an $m$ by $n$ matrix with &lt;em&gt;linearly independent rows&lt;/em&gt; (so $m \leq n$), then&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A^{\dagger} = A^{*}(AA^{*})^{-1}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A problem with both of these formula is the requirement of linear independence: in the square case, if all columns are linearly independent, then all rows are also linearly independent, and the matrix is invertible. The inverse of a matrix is also its Moore-Penrose generalized inverse, and if the matrix inverse is truly needed for a computation (often times -- but not always -- it is not, and can be replaced by solving an appropriate linear system) it should be calculated directly using LU, QR, or SVD, rather than using one of the two formulas for Moore-Penrose generalized inverses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem with both of the above formulas in the square case, assuming that the conditions are satisfied, is that they will be less accurate numerically than an SVD- or QR-based approach, because the condition number of $AA^{*}$ or $A^{*}A$ will be the square of the condition number of $A$ (all condition numbers mentioned in this explanation are for linear systems). In the case where $A$ is not square, but one of the two linear independence conditions is satisfied, the product of $A$ with its Hermitian transpose (in the appropriate order) will have a condition number equal to the square of the ratio of the largest singular value of $A$ to the smallest nonzero singular value of $A$. Again, this condition number could be large. Such a matrix product is best avoided unless there are compelling reasons for using it in an application (speed, for instance).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-02-20T17:11:49.787" LastActivityDate="2013-02-20T17:11:49.787" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5308" PostTypeId="2" ParentId="5289" CreationDate="2013-02-18T09:02:08.847" Score="4" Body="&lt;p&gt;Suppose $A$ is a $n\times n$ &lt;em&gt;dense&lt;/em&gt; matrix and you have to solve $Ax_i = b_i$, $i=1\dots m$. If $m$ is &lt;em&gt;big enough&lt;/em&gt; then there is nothing wrong in&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;V = inv(A);&#xA;...&#xA;x = V*b;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Flops are $O(n^3)$ for &lt;code&gt;inv(A)&lt;/code&gt; and $O(n^2)$ for &lt;code&gt;V*b&lt;/code&gt;, therefore in order to determine the break-even value for $m$ some experimentation is needed...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; n = 5000;&#xA;&amp;gt;&amp;gt; A = randn(n,n);&#xA;&amp;gt;&amp;gt; x = randn(n,1);&#xA;&amp;gt;&amp;gt; b = A*x;&#xA;&amp;gt;&amp;gt; rcond(A)&#xA;ans =&#xA;   1.3837e-06&#xA;&amp;gt;&amp;gt; tic, xm = A\b; toc&#xA;Elapsed time is 1.907102 seconds.&#xA;&amp;gt;&amp;gt; tic, [L,U] = lu(A); toc&#xA;Elapsed time is 1.818247 seconds.&#xA;&amp;gt;&amp;gt; tic, xl = U\(L\b); toc&#xA;Elapsed time is 0.399051 seconds.&#xA;&amp;gt;&amp;gt; tic, [L,U,p] = lu(A,'vector'); toc&#xA;Elapsed time is 1.581756 seconds.&#xA;&amp;gt;&amp;gt; tic, xp = U\(L\b(p)); toc&#xA;Elapsed time is 0.060203 seconds.&#xA;&amp;gt;&amp;gt; tic, V=inv(A); toc&#xA;Elapsed time is 7.614582 seconds.&#xA;&amp;gt;&amp;gt; tic, xv = V*b; toc     &#xA;Elapsed time is 0.011499 seconds.&#xA;&amp;gt;&amp;gt; [norm(xm-x), norm(xp-x), norm(xl-x), norm(xv-x)] ./ norm(x)&#xA;ans =&#xA;   1.0e-11 *&#xA;    0.1912    0.1912    0.1912    0.6183&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this trivial example $A^{-1}$ pre-computation is better than $LU$ forward and backward solution for $m&amp;gt;125$.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Some notes&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For stability and error analysis please see the comments to this different &lt;a href=&quot;https://scicomp.stackexchange.com/a/5292/1744&quot;&gt;answer&lt;/a&gt;, especially the one by VictorLiu.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proposed timings are not &quot;scientific&quot; at all, but are meant to show that the approach proposed in the &lt;a href=&quot;https://scicomp.stackexchange.com/a/5290/1744&quot;&gt;answer&lt;/a&gt; by Milind R, while it makes perfect sense if implemented in C or Fortran by calling relevant LAPACK and BLAS subroutines, may prove not so effective in Matlab, even for $m\ll n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Timing were performed with Matlab R2011b on a 12 core computer with a fairly constant UNIX load average of 5; best &lt;code&gt;tic, toc&lt;/code&gt; time of three probes.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-02-18T16:34:03.207" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="5309" PostTypeId="1" CreationDate="2013-02-18T11:40:41.827" Score="4" ViewCount="381" Body="&lt;p&gt;The FEM is usually used with a weak form of PDE.  But for the non-divergence form elliptic operator &#xA;$$&#xA;-a_1(x,y) \frac{\partial^2}{\partial x^2} - a_2(x,y) \frac{\partial^2}{\partial y^2}&#xA;$$&#xA;or another non-divergence form&#xA;$$&#xA;-\frac{\partial^2}{\partial x^2}a_1(x,y)\cdot  - \frac{\partial^2}{\partial y^2} a_2(x,y)\cdot&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is it inevitable to involve the derivatives $\partial_x a_1(x,y), \partial_y a_2(x,y)$?  Is it better to use FD (finite difference) or collocation methods instead of Galerkin FE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Updates&lt;/strong&gt;: Since I just came back from &lt;a href=&quot;http://colloqnum13.epfl.ch/&quot; rel=&quot;nofollow&quot;&gt;Swiss Numerical Colloquium 2013&lt;/a&gt;, there was a fantastic talke given by Endre Suli on the DG approximation of the Hamilton-Jacobi-Bellman equation which is fully non-linear and involves a non-divergence form elliptic equation which has non-smooth coefficients.  He gave very good reviews on the FD methods and proposed the high order DG method.  His paper can be found &lt;a href=&quot;http://ora.ox.ac.uk/search/detailed?q=SULI&amp;amp;ft=metadata&amp;amp;search=SEARCH&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.  I hope interested persons would also enjoy his paper.  Just for fun!&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2013-04-05T22:36:16.013" LastActivityDate="2013-04-05T22:36:16.013" Title="FEM for non-divergence form elliptic equation" Tags="|pde|finite-element|finite-difference|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5310" PostTypeId="2" ParentId="5302" CreationDate="2013-02-18T12:21:42.727" Score="0" Body="&lt;p&gt;I suggest to you that you see this link &lt;a href=&quot;http://www.wolfram.com/training/courses/hpc021.html&quot; rel=&quot;nofollow&quot;&gt;http://www.wolfram.com/training/courses/hpc021.html&lt;/a&gt; approximately 14:15 minutes into the video of course, mathematica training where they make an implementation of a cellular automata using CUDA, from there and you can modify it.&lt;/p&gt;&#xA;" OwnerUserId="3755" LastActivityDate="2013-02-18T12:21:42.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5311" PostTypeId="2" ParentId="5302" CreationDate="2013-02-18T14:41:07.620" Score="4" Body="&lt;p&gt;I would use the first option and would use a synchronous AC run before (using the GPU), to detect collisions, execute a step of a hexagonal AC whose rule is the value of the center cell = Sum (neighbors),&#xA;This CA must have seven states should be initiated with randomly selected cell, and their status verified before running the update rule for each gpu.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample 1. The value of a neighboring cell is shared&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 &lt;strong&gt;1&lt;/strong&gt; 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 0 &lt;strong&gt;1&lt;/strong&gt; 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a step of a CA whose rule is hexagonal central cell = Sum (neighbors)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 1 1 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 1 &lt;strong&gt;1&lt;/strong&gt; 1 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 1 2 1 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 1 &lt;strong&gt;1&lt;/strong&gt; 1 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 1 1 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample 2. The value of a cell to update is taken into account as a neighbor on the other&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 &lt;strong&gt;1&lt;/strong&gt; 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 &lt;strong&gt;1&lt;/strong&gt; 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the iteration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 1 1 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 1 &lt;strong&gt;2&lt;/strong&gt; 2 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 2 &lt;strong&gt;2&lt;/strong&gt; 1 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 1 1 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample 3. There is no relationship&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 &lt;strong&gt;1&lt;/strong&gt; 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 0 &lt;strong&gt;1&lt;/strong&gt; 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the iteration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 1 1 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 1 &lt;strong&gt;1&lt;/strong&gt; 1 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 1 1 0 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 1 1 0 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;  0 0 1 &lt;strong&gt;1&lt;/strong&gt; 1 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0 0 0 1 1 0 0&lt;/p&gt;&#xA;" OwnerUserId="3756" LastActivityDate="2013-02-18T14:41:07.620" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5312" PostTypeId="2" ParentId="5309" CreationDate="2013-02-18T14:43:52.553" Score="6" Body="&lt;p&gt;You can of course re-write the equation in a form that is more amenable to the FEM:&#xA;$$&#xA;  -\nabla \cdot \left(\begin{pmatrix}a_1 &amp;amp; 0 \\ 0 &amp;amp; a_2\end{pmatrix} \nabla u\right) +&#xA;  \begin{pmatrix} \partial_x a_1 \\ \partial_y a_2\end{pmatrix} \cdot \nabla u&#xA;= f.&#xA;$$&#xA;This is an advection-diffusion problem. This form also gives you a better idea of whether the problem is advection dominated or diffusion dominated and consequently whether it will be necessary to stabilize the discretization or not.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-18T14:43:52.553" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5313" PostTypeId="2" ParentId="5305" CreationDate="2013-02-18T15:52:14.577" Score="1" Body="&lt;p&gt;As you mention you cannot devide by a metric, i guess you want a unit-less outcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the volume of a cylinder is always going to be cubic meters (decimeters or any scalefactor can be applied) you will need to devide it by something with the same dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The smallest cylinder that can hold the cloud&lt;/li&gt;&#xA;&lt;li&gt;The smallest cube/sphere that can hold the cloud&lt;/li&gt;&#xA;&lt;li&gt;The smallest cube/sphere that can hold the cloud&lt;/li&gt;&#xA;&lt;li&gt;The size of the range in which the points can occur&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3315" LastActivityDate="2013-02-18T15:52:14.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5314" PostTypeId="1" CreationDate="2013-02-18T17:11:25.357" Score="9" ViewCount="5902" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikibooks.org/wiki/Parallel_Spectral_Numerical_Methods/Finding_Derivatives_using_Fourier_Spectral_Methods&quot; rel=&quot;noreferrer&quot;&gt;This wikibook&lt;/a&gt; states that the output of MATLAB's &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/fft.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;FFT&lt;/code&gt;&lt;/a&gt; corresponds with the wavenumbers ordered as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$k=\left\{0,1,...,\frac{n}{2},-\frac{n}{2}+1,-\frac{n}{2}+2,...,-1\right\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the example codes on the same page, the wavenumbers are coded as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = [0:n/2-1 0 -n/2+1:-1];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is the same as the first, but with the $n/2$-wavenumber (the &quot;maximum wavemnumber&quot;) replaced with $0$.  It seems strange that they would include $0$ twice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems the correct order is necessary for taking derivatives via Fourier transforms, as described in the wikibook.  Which of these is correct, and does MATLAB document this anywhere?&lt;/p&gt;&#xA;" OwnerUserId="3742" LastActivityDate="2019-11-03T22:17:11.093" Title="Order of MATLAB FFT frequencies" Tags="|matlab|fourier-analysis|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5315" PostTypeId="1" CreationDate="2013-02-18T17:35:16.433" Score="5" ViewCount="215" Body="&lt;p&gt;I am struggling with DG methods for 2 phase flow through porous media. I managed to get the global pressure, total flux equations to work with an unconditionally stable mixed FE DG formulation as described here:&#xA;&lt;a href=&quot;http://math.aalto.fi/reports/a531.ps&quot;&gt;http://math.aalto.fi/reports/a531.ps&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but the saturation equation is giving me problems. Basically I tried LDG and IP and both gave pretty weird unphysical results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any good references for DG with flow through porous media? I found at least one good compositional flow paper, but not much about blackoil models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I probably should add that I am writing the code with FEniCS and CBCPDESys. So far it has saved me years in development time.&lt;/p&gt;&#xA;" OwnerUserId="3758" LastEditorUserId="3758" LastEditDate="2013-02-18T17:51:03.077" LastActivityDate="2013-02-18T17:51:03.077" Title="Discontinuous Galerkin for flow through porous media" Tags="|discontinuous-galerkin|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5316" PostTypeId="1" AcceptedAnswerId="5349" CreationDate="2013-02-18T22:49:46.793" Score="3" ViewCount="3017" Body="&lt;p&gt;I'm having trouble understanding how to code 2-D Poisson's Equation with &lt;em&gt;Dirichlet&lt;/em&gt; boundary conditions. What I have thus far is&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Constructed square mesh with triangular elements&lt;/li&gt;&#xA;&lt;li&gt;Assembled stiffness matrix. The function for this also gives the area of each element (all same, since square).&lt;/li&gt;&#xA;&lt;li&gt;Use Gaussian Quadrature to construct the right hand side.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit&lt;/strong&gt;: I may have been a little vague in my question. Hopefully this is a better rephrasing: Separate from FE, my code for gaussian quadrature works just fine. However, I don't understand how gaussian quadrature can even be relevant/involved with obtaining a solution to 2D Poisson; hence, I don't know how to continue coding.&lt;/p&gt;&#xA;" OwnerUserId="3493" LastEditorUserId="3493" LastEditDate="2013-02-20T02:21:51.350" LastActivityDate="2013-06-07T19:18:48.940" Title="Finite Element Method: 2-D Poisson's Equation in Matlab, Gaussian quadrature" Tags="|finite-element|matlab|quadrature|poisson|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5318" PostTypeId="1" AcceptedAnswerId="5320" CreationDate="2013-02-19T06:00:03.993" Score="15" ViewCount="8672" Body="&lt;p&gt;I am porting an existing code from MATLAB to C++ and have a linear system to solve $xA=b$ (rather than the more typical form $Ax=b$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $A$ is dense, and of general form, but is no larger than 1000x1000.&#xA;So in MATLAB, the solution is found by the &lt;code&gt;mrdivide(b,A)&lt;/code&gt; function or the forward-slash notation &lt;code&gt;x = b/A;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I solve this in my C++ code using BLAS and LAPACK routines?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm familiar with the LAPACK routine &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d7/d3b/group__double_g_esolve.html#ga5ee879032a8365897c3ba91e3dc8d512&quot;&gt;&lt;code&gt;DGESV&lt;/code&gt;&lt;/a&gt; which solves $Ax=b$ for $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, one thought I had is to do some manipulations using matrix transpose identities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(xA)^T=b^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A^T x^T = b^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x^T = (A^T)^{-1} b^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then solve the final form using &lt;code&gt;DGESV&lt;/code&gt; operating on the transposed $A^T$. (so cost to transpose $A$ and cost to solve system)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an approach more efficient or otherwise &lt;em&gt;better&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working with matrix and vector classes as well as BLAS implementation from the BOOST uBLAS library as well as bindings to the LAPACK library routines.  I've been using this setup successfully for other operations and am hoping to find a solution limited to these libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I should note I am only performing this type of operation a few times during code setup, so performance is not a critical concern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe this MATLAB &lt;a href=&quot;http://matlab.izmiran.ru/help/techdoc/ref/mldivide.html&quot;&gt;documentation&lt;/a&gt; on &lt;code&gt;mrdivide&lt;/code&gt; is helpful for others.&lt;/p&gt;&#xA;" OwnerUserId="3767" LastEditorUserId="1804" LastEditDate="2016-05-27T15:53:29.630" LastActivityDate="2016-05-27T15:53:29.630" Title="solve $xA=b$ for $x$ using LAPACK and BLAS" Tags="|linear-solver|c++|lapack|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5319" PostTypeId="2" ParentId="5318" CreationDate="2013-02-19T07:38:37.330" Score="1" Body="&lt;p&gt;You can compute the pseudo inverse of $A$, by using say QR decomposition of SVD, which are both included in LAPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;xA=b\\&#xA;xQR=b\\&#xA;x=bR^{-1}Q^T&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will work for any rectangular $A$.&lt;/p&gt;&#xA;" OwnerUserId="3761" LastActivityDate="2013-02-19T07:38:37.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5320" PostTypeId="2" ParentId="5318" CreationDate="2013-02-19T08:08:31.160" Score="10" Body="&lt;p&gt;Trivial answer for &lt;em&gt;square&lt;/em&gt; $A$: use &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d7/d3b/group__double_g_esolve.html#ga9d90ccf6e340cacd08b7bbbb502ceb21&quot;&gt;&lt;code&gt;dgesvx&lt;/code&gt;&lt;/a&gt; which solves also for $A^T x = b$ when &lt;code&gt;TRANS = 'T'&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that with BLAS or LAPACK you hardly have to transpose (swapping elements in memory) a matrix: most of the subroutines have a &lt;code&gt;TRANS&lt;/code&gt; argument to accommodate for operation on the transpose matrix or on a matrix stored with a different memory layout. (Transposition is equivalent to changing Fortran-contiguous memory layout to a C-contiguos one and viceversa.)&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-02-19T08:08:31.160" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5321" PostTypeId="1" AcceptedAnswerId="5323" CreationDate="2013-02-19T11:03:23.757" Score="5" ViewCount="480" Body="&lt;p&gt;I'm trying to understand how to compute the value of Pi by means of the Monte Carlo simulation.&#xA;I have a circle inside a square where the sides of the square are tangent to the circle. As data I have the number of random points and the ratio between the lenght of one side of the square and the diameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\displaystyle \frac{l}{d} = r_{a}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to what I've read I have to count the number of points lying inside the circle and the total of points.&#xA;The area of a circle is $A_{circle} = \pi r^2$ and the area of a square is $A_{square} = l^2$.&#xA;To check if a point is inside the circle I have to check $\sqrt{x^2+y^2} &amp;lt;= r$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to relate the ratio $r_a$ with the radius so to check if the point is inside or outside the circle. Any suggestion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has been a really frustrating stuff and is a really simple example of Monte Carlo approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming $d=1$ then I have that $\displaystyle \frac{l}{d} = l = r_a$&#xA;Therefore, to check if a random point is inside the circle I have to check only that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sqrt{x^2 + y^2} ≤ \frac{1}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or it has to be related to the ratio $r_a$ ??&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2013-02-25T21:50:49.470" LastActivityDate="2015-11-04T10:05:42.677" Title="Monte Carlo approximation of PI" Tags="|monte-carlo|geometry|" AnswerCount="3" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="5322" PostTypeId="1" AcceptedAnswerId="5365" CreationDate="2013-02-19T12:04:13.527" Score="6" ViewCount="1570" Body="&lt;p&gt;I have to estimate a parameter (K), but I don't know how I can do it. I think by a regression model (minimum least square?), but I'm not sure. The system is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dX1/dt = f(t)*X2*X1&#xA;dX2/dt = -K*f(t)*X2*X1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;X1 and X2 are the state variables, f(t) is a time varying function and K is the parameter that I want to estimate. I know the values of X1 and X2 each sample-time Ts=0.5sec. I'm using Matlab.&lt;/p&gt;&#xA;" OwnerUserId="3769" LastEditorUserId="3769" LastEditDate="2013-02-19T12:10:44.900" LastActivityDate="2013-03-19T08:50:44.900" Title="parameters estimation" Tags="|matlab|nonlinear-programming|regression|nonlinear-equations|least-squares|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5323" PostTypeId="2" ParentId="5321" CreationDate="2013-02-19T13:04:42.390" Score="3" Body="&lt;p&gt;Suppose, that your circle have unitary radius: $r=1$, then length of square's side equal to $l = r_a \cdot d = 2\cdot r_a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, area of circle equals to $S_c = \pi$ and area of square equals to $S_s = 4 r_a^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you &quot;throw&quot; $N_t$ random points inside of square, a part of them (say, $N_c$) will fall into the circle. Relation $\frac{N_c}{N_t}$ equals to proportions of their areas, $\frac{S_c}{S_s}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{N_c}{N_t} = \frac{S_c}{S_s}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to calculate $\pi$ you should divide $N_c$ to $N_t$ and multipy by $S_s$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\pi = 4r_a^2\cdot\frac{N_c}{N_t}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of unitary $l$ you should set $r = \frac{1}{2 r_a}$.&lt;/p&gt;&#xA;" OwnerUserId="3768" LastEditorUserId="1552" LastEditDate="2013-02-20T15:38:59.587" LastActivityDate="2013-02-20T15:38:59.587" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5324" PostTypeId="2" ParentId="5321" CreationDate="2013-02-19T13:52:54.000" Score="2" Body="&lt;p&gt;I don't think that you can find the radius $r$ with the informations that you provide. For example, you can imagine having &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a circle with diameter $d=1$ centered in a square with side $l=1$&lt;/li&gt;&#xA;&lt;li&gt;a circle with diameter $d=2$ centered in a square with side $l=2$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both circles will be tangent to their respective squares and both yield $r_a=1$. So, if your data consists only in $r_a$, you cannot distinguish between the two situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, the data $r_a$ seems even useless, since to obtain a circle tangent to the sides of the square, you need $r_a = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to forget about the hypothesis that the circle is tangent to the square, you can still approximate $\pi$ for different values of $r_a$ but you need to modify your formula and this does not fix your problem with finding $r$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One last point is that actually, $r$ does not really matter: if you have a set of random points in the square with size 1, you can scale them to make them appear in a square of size $l$ (whatever is $l$). Applying the formula to the square of size 1 or size $l$ will give you exactly the same approximation of $\pi$.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-02-19T13:52:54.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5325" PostTypeId="1" AcceptedAnswerId="5327" CreationDate="2013-02-19T17:26:04.047" Score="2" ViewCount="66" Body="&lt;p&gt;Say we are given a congruence relation$~\sim$ in a dataset with $n$ elements. I am looking for an algorithm for optimally sorting the $n$ elements into $m$ clusters according to given congruence relations. For instance if the data contains ${a,b,c,d,e,f,g,h}$, and: $$a\sim b,\ d\sim b,\ e \sim h,\ f \sim c$$ The data should be sorted into the following clusters:&#xA;$$\{a,b,d\},\ \{c,f\},\ \{e,h\},\ \{g\}$$&#xA;As said I'm looking for an efficient algorithm to solve this, I am led to believe this can be done in $O(n)$, but I can't seem to work out the details. &lt;/p&gt;&#xA;" OwnerUserId="1699" LastActivityDate="2013-02-19T18:29:21.873" Title="Clustering Algorithm for a congruence relation?" Tags="|algorithms|complexity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5326" PostTypeId="2" ParentId="5325" CreationDate="2013-02-19T17:41:51.120" Score="1" Body="&lt;p&gt;Write your relation as a sparse graph and use a &quot;connected components&quot; function, like &lt;a href=&quot;http://networkx.github.com/documentation/latest/reference/generated/networkx.algorithms.components.connected.connected_components.html#networkx.algorithms.components.connected.connected_components&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3705" LastActivityDate="2013-02-19T17:41:51.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5327" PostTypeId="2" ParentId="5325" CreationDate="2013-02-19T18:29:21.873" Score="2" Body="&lt;p&gt;I've always heard this referred to as a &quot;Union Find&quot;. It's described here, as well as the optimizations you can do to beat the naive implementation: &lt;a href=&quot;http://www.algorithmist.com/index.php/Union_Find&quot; rel=&quot;nofollow&quot;&gt;http://www.algorithmist.com/index.php/Union_Find&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12" LastActivityDate="2013-02-19T18:29:21.873" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5328" PostTypeId="1" CreationDate="2013-02-19T20:16:15.790" Score="4" ViewCount="632" Body="&lt;p&gt;Having a matrix $\mathbf{A} \in \mathcal{C}^{m\times n}$ I solve following least-squares problem $$Re(\mathbf{A}^H \mathbf{A})x=Re(\mathbf{A}^H\mathbf{b}).$$ If the matrix $\mathbf{A}$ was a real matrix, the solution to the equation above could have been written as $$\mathbf{x} = \sum_{i=1}^{rank(\mathbf{A})} \frac{\mathbf{u}_i^T\mathbf{b}}{s_i}\mathbf{v}_i,$$ where $\mathbf{u}_i$ and $\mathbf{v}_i$ are corresponding left and right singular vectors and $s_i$ is an $i$th singular value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is whether the solution to the least-squares problem stated in the first equation can be written in a similar way given the SVD of $\mathbf{A} = \mathbf{U}\mathbf{S}\mathbf{V}^H$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that one could split matrix $\mathbf{\tilde A} = [Re(\mathbf{A});~Im(\mathbf{A})]$ and equivalently solve the real problem, but the condition to stay within the complex SVD of the original matrix is of main concern here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="3771" LastActivityDate="2013-02-20T18:30:10.417" Title="Complex least-squares problem" Tags="|linear-algebra|least-squares|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5329" PostTypeId="1" AcceptedAnswerId="5344" CreationDate="2013-02-19T20:20:58.723" Score="3" ViewCount="167" Body="&lt;p&gt;When using the &lt;a href=&quot;http://www.numeca.com/index.php?id=29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hexpress&lt;/a&gt; grid generator one gets hexahedral cells, possibly with hanging nodes. Because of a smoothing step, the hanging nodes can be shifted: they are not necessarily on the straight line that connects the corners, as shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mIk9P.png&quot; alt=&quot;cell with shifted hanging node&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With FVM one would triangulate the face with 5 nodes in order to approximate its surface, face center and such, see for example the book by Ferziger and Peric, &lt;em&gt;Computational Methods for Fluid Dynamics&lt;/em&gt;, Springer, 2001. How should one treat this situation in FEM? I'm familiar with the hexa 8 mapping to a master element using linear shape functions but that doesn't seem quite enough.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-02-21T00:26:46.677" Title="How to treat hexahedral element with shifted hanging node?" Tags="|finite-element|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5330" PostTypeId="1" AcceptedAnswerId="5333" CreationDate="2013-02-20T04:54:40.070" Score="4" ViewCount="452" Body="&lt;p&gt;I have downloaded and unzipped PETSc from &lt;a href=&quot;http://www.mcs.anl.gov/petsc/download/index.html&quot; rel=&quot;nofollow&quot;&gt;PETSc Website&lt;/a&gt; in my home directory (OS is CentOS 6.3 64bit )  and followed given commands on &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/installation.html&quot; rel=&quot;nofollow&quot;&gt;Installation page&lt;/a&gt;  &quot;./configure --with-cc=gcc --with-fc=gfortran --download-f-blas-lapack --download-mpich&quot; and &quot;make all test&quot;. There was no error throughout the process. Is it installed properly? How should I start using these libraries in C programs? Is it possible to install it in &quot;file system&quot;? so that one can access it like &quot;LAPACK&quot; libraries or &quot;CGNS&quot; libraries.&lt;/p&gt;&#xA;" OwnerUserId="3549" LastActivityDate="2013-02-20T08:36:51.520" Title="About PETSc installation" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5331" PostTypeId="1" AcceptedAnswerId="5335" CreationDate="2013-02-20T06:09:01.983" Score="2" ViewCount="125" Body="&lt;p&gt;I writing some robust geometric algorithms using quantization + integer arithmetic for evaluating exact predicates.  However, since BlueGene's integer support is so terrible, it occurred to me that the use of integer math may (weirdly) kill the portability of the library in terms of performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How worried should I be about this?  Are future HPC systems likely to have similar issues?  I realize that a question about predicting the future may be difficult or impossible to answer, but this seems like an important design decision to get right.  Any thoughts are appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, there's an easy fallback: I can run an initial filtering step using floating point interval arithmetic (assuming the architecture has &quot;round to $-\infty$&quot; support).  This has zero effect on the results since I'd fall back to integers if the filter fails.  Other integer-based algorithms may not be so lucky, however.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-02-20T19:03:23.103" Title="Integer arithmetic support on future HPC systems" Tags="|computer-arithmetic|architecture|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5332" PostTypeId="2" ParentId="5306" CreationDate="2013-02-20T08:18:35.687" Score="1" Body="&lt;p&gt;For 3x3 matrices this should be easy. Here's a least-squares routine in Fortran that forms Moore-Penrose pseudoinverse in the process of solution, it might help:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   function solve_leastsq(A,b,m,n) result(x)&#xA;!&#xA;!  Solve system with m x n system matrix in least square sense (minimizing Euclidean norm).&#xA;!  System is overdetermined so we solve A'A * x = A'b, where A' is transpose of A.&#xA;!  The solution is given by x = (A'A)^(-1)A'*b&#xA;!&#xA;&#xA;!  Result&#xA;   real(dp), dimension(n) :: x&#xA;&#xA;!  Input&#xA;   real(dp), dimension(m,n), intent(in) :: A&#xA;   real(dp), dimension(m), intent(in) :: b&#xA;&#xA;!  Locals&#xA;   real(dp), dimension(n,m) ::  At&#xA;   real(dp), dimension(n,n) ::  AtA, invAtA&#xA;   real(dp), dimension(n) ::  Atb&#xA;&#xA;   At = transpose(A)      ! transpose of A : A'&#xA;   AtA = matmul(At,A)     ! left-hand side multiplication of A by A': A'A&#xA;   Atb = matmul(At,b)     ! A'b&#xA;   invAtA = inv(AtA)      ! an inverse of A'A&#xA;   x = matmul(invAtA,Atb) ! x = (A'A)^(-1) * A'b&#xA;&#xA;   end function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For this to work you need these routines to find an inverse of a 3x3 matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   function det_3x3(a)&#xA;!  Result&#xA;   real(dp) :: det_3x3&#xA;!  Input&#xA;   real(dp), dimension(3,3), intent(in) :: a&#xA;&#xA;   det_3x3 =  a(1,1)*a(2,2)*a(3,3)+a(1,2)*a(2,3)*a(3,1)+a(1,3)*a(2,1)*a(3,2) &amp;amp;&#xA;             -a(1,3)*a(2,2)*a(3,1)-a(1,2)*a(2,1)*a(3,3)-a(1,1)*a(2,3)*a(3,2) &#xA;&#xA;   end function&#xA;!=======================================================================&#xA;   function adj(a)&#xA;!&#xA;!  Adjugate of a 3x3 matrix (the transpose of the cofactor matrix).&#xA;!&#xA;&#xA;!  Result&#xA;   real(dp), dimension(3,3) :: adj&#xA;!  Input&#xA;   real(dp), dimension(3,3), intent(in) :: a&#xA;&#xA;   adj(1,1) = a(2,2)*a(3,3) - a(2,3)*a(3,2)&#xA;   adj(1,2) = a(1,3)*a(3,2) - a(1,2)*a(3,3)&#xA;   adj(1,3) = a(1,2)*a(2,3) - a(1,3)*a(2,2)&#xA;   adj(2,1) = a(2,3)*a(3,1) - a(2,1)*a(3,3)&#xA;   adj(2,2) = a(1,1)*a(3,3) - a(1,3)*a(3,1)&#xA;   adj(2,3) = a(1,3)*a(2,1) - a(1,1)*a(2,3)&#xA;   adj(3,1) = a(2,1)*a(3,2) - a(2,2)*a(3,1)&#xA;   adj(3,2) = a(1,2)*a(3,1) - a(1,1)*a(3,2)&#xA;   adj(3,3) = a(1,1)*a(2,2) - a(1,2)*a(2,1)&#xA;&#xA;   end function&#xA;!=======================================================================&#xA;   function inv(a)&#xA;!&#xA;!  An inverse of a 3x3 matrix.&#xA;!&#xA;&#xA;!  Result&#xA;   real(dp), dimension(3,3) :: inv&#xA;!  Input&#xA;   real(dp), dimension(3,3), intent(in) :: a&#xA;!  Locals&#xA;   real(dp) :: detr&#xA;   detr = 1./det_3x3(a)&#xA;   inv = adj(a)*detr&#xA;   end function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I use double precision for reals:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   integer, parameter :: dp = kind(1.0d0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2013-02-20T08:18:35.687" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5333" PostTypeId="2" ParentId="5330" CreationDate="2013-02-20T08:36:51.520" Score="4" Body="&lt;p&gt;If &quot;make test&quot; reports something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Running test examples to verify correct installation&#xA;Using PETSC_DIR=/home/nate/src/PETSc-3.3 and PETSC_ARCH=arch-linux2-c-opt&#xA;C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process&#xA;C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI processes&#xA;Fortran example src/snes/examples/tutorials/ex5f run successfully with 1 MPI process&#xA;Completed test examples&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then indeed the library should be installed properly. I would recommend that you export the values of PETSC_DIR and PETSC_ARCH (the ones that, for example, are shown when you run the tests) in your .bashrc file such that they are always set. With these environment variables set, you can use PETSc from anywhere if you use makefiles which look something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MyCode: MyCode.o chkopts&#xA;    ${CLINKER} -o $@ $&amp;lt; ${PETSC_LIB}&#xA;&#xA;include ${PETSC_DIR}/conf/variables&#xA;include ${PETSC_DIR}/conf/rules&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you need to make PETSc available for all users on your machine (like in /opt) then just go to the PETSc installation page and look for the section labeled:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Installing PETSc in /usr/local or /opt where sudo or root previledges are required&lt;/p&gt;&#xA;" OwnerUserId="1625" LastActivityDate="2013-02-20T08:36:51.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5334" PostTypeId="2" ParentId="5328" CreationDate="2013-02-20T12:23:53.677" Score="2" Body="&lt;p&gt;I can't see how it can be done, but others might be able to help you further. I get, with $A=USV^H$, $A^H=VS^H U^H$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Re(A^H A)x = Re(A^H b) =&amp;gt; Re(V S^H U^H U S V^H) x = Re(V S^H U^H b) =&amp;gt; Re(V S^H S V^H) x = Re(V S^H U^H b)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, you would now reduce  $V S^H$ away, but since they are inside the real part, and since $Re(V S^H S V^H) \neq Re(V S^H) Re(S V^H)$, I am not sure how to proceed. Can anyone else get an idea?&lt;/p&gt;&#xA;" OwnerUserId="370" LastActivityDate="2013-02-20T12:23:53.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5335" PostTypeId="2" ParentId="5331" CreationDate="2013-02-20T12:52:48.520" Score="2" Body="&lt;p&gt;Different vendors have different paths forward on this, and most of it is under NDA. If you really want to know, ask Intel, AMD, IBM, ARM, and maybe NVIDIA for briefings under NDA. Some of these vendors will have full support for vectorized integer instructions in the future so that integer will keep up with floating-point, and some of them may not. For those that don't, you will most likely see performance at the single add, multiply, or logical operation per clock level with divide and remainder operations taking many cycles.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-02-20T19:03:23.103" LastActivityDate="2013-02-20T19:03:23.103" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5336" PostTypeId="2" ParentId="5331" CreationDate="2013-02-20T13:50:48.207" Score="0" Body="&lt;p&gt;To answer your first question: I wouldn't worry about floating-point being faster than integers. If floating point operations &lt;em&gt;are&lt;/em&gt; faster than regular integer operations, I don't see why you shouldn't use them. If you stay within 23 bits, all single-precision floating-point operations on integer values should be identical to the integer operations on the same values.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-02-20T13:50:48.207" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5338" PostTypeId="2" ParentId="5328" CreationDate="2013-02-20T18:30:10.417" Score="0" Body="&lt;p&gt;Check out the LAPACK routines ZGELLS or ZGELLD. They solve the LS problem using SVD. See &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d6/d10/group__complex16_g_esolve.html#ga61e68db68886c3f80753fac87ca35a6e&quot; rel=&quot;nofollow&quot;&gt;the official LAPACK documentation for the routine&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-02-20T18:30:10.417" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5339" PostTypeId="2" ParentId="5314" CreationDate="2013-02-20T18:37:41.763" Score="4" Body="&lt;p&gt;I want to expand on my comment and rework the example you reference in a way that should be more understandable than the original and to explain why &lt;code&gt;fft&lt;/code&gt; returns the coefficients the way it does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, the fft portion of the example is :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Nx = size(x,2);&#xA;k = 2*pi/(b-a)*[0:Nx/2-1 0 -Nx/2+1:-1];&#xA;dFdx = ifft(1i*k.*fft(f));&#xA;d2Fdx2 = ifft(-k.^2.*fft(f));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I added another section of code directly below it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Nx = size(x,2);&#xA;k = 2*pi/(b-a)*(-Nx/2:Nx/2-1);&#xA;dFdxp = ifft(ifftshift(1i*k.*fftshift(fft(f))));&#xA;d2Fdx2p = ifft(ifftshift(-k.^2.*fftshift(fft(f))));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and wrapped both pieces of code in a &lt;code&gt;tic; toc&lt;/code&gt; for rought timings.  In a more readable format, the second method uses:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ckf = fftshift(fft(f));&#xA;ckdf = 1i*k.*ckf;&#xA;df = ifft(ifftshift(ckdf));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first difference is that the second example has a much more intuitive &lt;code&gt;k&lt;/code&gt;.  This is the main advantage of the second example, since k is now in the form that we think about them.  In the second and third lines I had to add &lt;code&gt;fftshift&lt;/code&gt; around the call to &lt;code&gt;fft&lt;/code&gt;, then a call to &lt;code&gt;ifftshift&lt;/code&gt; directly inside the call to &lt;code&gt;ifft&lt;/code&gt;.  These additional function calls reorder the coefficients from what is required for the computer to work with them to the way humans usually think about them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue with the second example, is that while &lt;code&gt;k&lt;/code&gt; is more intuitive for us, this leaves the internal matrices for solving and inverting &lt;code&gt;fft&lt;/code&gt; in forms that aren't as advantageous.  So either we have to switch the order with calls to &lt;code&gt;fftswitch&lt;/code&gt; and &lt;code&gt;ifftswitch&lt;/code&gt; or it has to be hard coded into the &lt;code&gt;fft&lt;/code&gt; functions.  This is less prone to error from users (assuming they are unfamiliar with the workings of fft, as many people are), but you pay a price in run time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I stated before, I added timing calls around the two blocks for comparison and ran for multiple N.  The timing results were:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N =     1000,  Ex1 = 0.000222 s,   Ex2 = 0.007072 s&#xA;N =    10000,  Ex1 = 0.001576 s,   Ex2 = 0.003506 s&#xA;N =   100000,  Ex1 = 0.023857 s,   Ex2 = 0.034051 s&#xA;N =  1000000,  Ex1 = 0.213816 s,   Ex2 = 0.406250 s&#xA;N = 10000000,  Ex1 = 4.555143 s,   Ex2 = 7.102348 s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the act of switching the values back and forth slows the process considerably, especially at low N (where it 30x slower).  This is just an example, and your computer may show slightly different trends depending on things such as memory speed, processor cores/speed, etc. but it is illustrative of the point.  The reason &lt;code&gt;fft&lt;/code&gt; has confusing output is because it is saving you a nontrivial fraction of your computing time.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-02-20T18:37:41.763" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5340" PostTypeId="2" ParentId="5283" CreationDate="2013-02-20T21:21:57.270" Score="1" Body="&lt;p&gt;It sounds like you are interested in Born-Oppenheimer &lt;a href=&quot;http://en.wikipedia.org/wiki/Molecular_dynamics#Hybrid_QM.2FMM&quot; rel=&quot;nofollow&quot;&gt;molecular dynamics&lt;/a&gt;, i.e. using classical equations of motion for the nuclei while using quantum mechanics for the electrons. This is a fairly common method implemented in a range of &lt;a href=&quot;http://en.wikipedia.org/wiki/Quantum_chemistry_computer_programs&quot; rel=&quot;nofollow&quot;&gt;quantum chemistry packages&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related approximate method is &lt;a href=&quot;http://en.wikipedia.org/wiki/CPMD&quot; rel=&quot;nofollow&quot;&gt;Car-Parinello molecular dynamics&lt;/a&gt;, which may be of use to you. This may be your best bet if your system of study is large.&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2013-02-20T21:21:57.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5341" PostTypeId="5" CreationDate="2013-02-20T21:23:48.173" Score="0" Body="&lt;p&gt;Numerical methods and problems involving the solution of the Schrodinger equation.&lt;/p&gt;&#xA;" OwnerUserId="522" LastEditorUserId="522" LastEditDate="2013-02-21T04:20:21.330" LastActivityDate="2013-02-21T04:20:21.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5342" PostTypeId="4" CreationDate="2013-02-20T21:23:48.173" Score="0" Body="Numerical methods and problems involving the solution of the Schrodinger equation and related subatomic models." OwnerUserId="522" LastEditorUserId="549" LastEditDate="2013-02-20T22:17:30.537" LastActivityDate="2013-02-20T22:17:30.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5343" PostTypeId="1" CreationDate="2013-02-20T21:29:03.727" Score="3" ViewCount="1852" Body="&lt;p&gt;I'm currently running a CFD simulation for an internal flow and need to compare my computed results to experimental data. The experimental data is saved as a comma separated file with columns for x,y,z coordinates and pressure. The computed data is saved in Ensight format on an unstructured 3D mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, I have the following experimental data:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-0.08573,0.00000,0.09154,83600.0&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;-0.08573,0.00000,0.08154,83610.0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to plot the pressure along the y-axis at x=-0.08573 as a function of z, and overlay crosshairs to show the experimental data. I can create the two plots separately, using the &quot;Plot Over Line&quot; filter for the CFD data, and the &quot;Plot Data&quot; filter for the experimental data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried using the &quot;Group Datasets&quot; filter to combine the two sources, but when I try to plot I run into problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use &quot;Plot Over Line&quot;, the experimental data doesn't show in the plot. I think this is due to rounding in the coordinates for the line vs. the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use &quot;Plot Data&quot;, I can get both to show up, but only if I set the X axis data to &quot;Use Array Index From Y Axis Data&quot;. When I do this, the experimental data aren't plotted over the correct part of the CFD data. Switching to &quot;Use Data Array&quot; and Points(2) for the z-coordinates, I get an error &quot;No X column is set (index 0).&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems like it should be a simple thing to accomplish, and a fairly common task, but I've had no luck finding a tutorial or explanation online. I know that I can just export the data and use gnuplot or xmgrace, but I would rather do all the processing in one program if possible.&lt;/p&gt;&#xA;" OwnerUserId="553" LastActivityDate="2013-02-20T21:29:03.727" Title="Create 2D plot from two data sets in ParaView" Tags="|fluid-dynamics|visualization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5344" PostTypeId="2" ParentId="5329" CreationDate="2013-02-21T00:26:46.677" Score="4" Body="&lt;p&gt;The best approach would be to look at adding an isoparametric element for the extra node.  See, for instance, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.edwilson.org/book-wilson/05-iso.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.edwilson.org/book-wilson/05-iso.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Section 5.5 which has a discussion of this for the 2D bilinear element.  The extension to 3D trilinear is rather straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this requires some flexibility in the code in terms of local element spaces -- this element has 9 elements instead of 8.  The best reference for both the math of doing this and the software design of doing this is the book:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0486411818&quot; rel=&quot;nofollow&quot;&gt;http://www.amazon.com/Finite-Element-Method-Mechanical-Engineering/dp/0486411818&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="508" LastActivityDate="2013-02-21T00:26:46.677" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5345" PostTypeId="1" CreationDate="2013-02-21T05:11:12.023" Score="4" ViewCount="877" Body="&lt;p&gt;My question can be rephrased as &quot;FEM library like deal.II but for simplex elements&quot;.&lt;br/&gt;&#xA;Our scientific group works with very complicated 3D geometries, so usually we prefer tetrahedral meshes for our models. And we use Gmsh to build such grids.&lt;br/&gt;&#xA;Programming language that we use is C++. We have never used third-party FEM libraries, we work with our own matrix formats, solvers, etc. But I think that applying FEM libraries (like deal.II, for example) could be increase the productivity of our investigations. deal.II has a lot of advantages, but its main disadvantage (for us) is support of only hexahedral meshes. I read that deal.II can work with meshes from Gmsh through the interface described &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=3&amp;amp;cad=rja&amp;amp;ved=0CD8QFjAC&amp;amp;url=http%3A%2F%2Fwolfgang.math.tamu.edu%2Fsvn%2Fpublic%2Fdeal.II%2Fbranches%2FMATH676%2F2008%2Fdeal.II%2Fexamples%2Fthompson%2Ffinal_presentation.pdf&amp;amp;ei=R5slUYCABLLY4QTByoGoAw&amp;amp;usg=AFQjCNFRAPUSHpxAu2MwzjOJ9xnncX4aGw&amp;amp;sig2=ZHaUe6OfWDXQrA2DSL7zCQ&amp;amp;bvm=bv.42661473,d.bGE&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. Maybe it is a good choice for what I'm looking for. But I'm not sure.&lt;br/&gt;&#xA;Anyway, maybe anybody knows C++ FEM library as well documented, good tested and actively maintained as deal.II, but with support of simplex finite elements.&lt;br/&gt;&#xA;I would like to add a little bit about capabilities that such library can possess. I would very like to have an opportunity to work with a) vector Nedelec and b) discontinuous elements among other standard (scalar, continuous) ones.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastEditorUserId="3576" LastEditDate="2013-02-21T09:47:55.640" LastActivityDate="2017-09-06T02:48:05.987" Title="FEM library with support of simplex elements" Tags="|finite-element|libraries|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5347" PostTypeId="2" ParentId="5345" CreationDate="2013-02-21T08:39:57.637" Score="3" Body="&lt;p&gt;I would suggest &#xA;&lt;strong&gt;getfem&lt;/strong&gt; &lt;a href=&quot;http://download.gna.org/getfem/html/homepage/&quot; rel=&quot;nofollow&quot;&gt;http://download.gna.org/getfem/html/homepage/&lt;/a&gt;&#xA;or &#xA;&lt;strong&gt;dolfin&lt;/strong&gt; &lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/dev/cpp/&quot; rel=&quot;nofollow&quot;&gt;http://fenicsproject.org/documentation/dolfin/dev/cpp/&lt;/a&gt;&#xA;Kind Regards, Tom&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2013-02-21T08:39:57.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5348" PostTypeId="1" AcceptedAnswerId="5351" CreationDate="2013-02-21T09:47:38.687" Score="4" ViewCount="410" Body="&lt;p&gt;I frequently need to solve linear systems with sparse matrices of moderate dimension (say a few thousand). These matrices are composed entirely of small dense blocks (typically 5-10 in dimension), and the overall block structure is sparse (with a pattern like in a PDE spatial discretization). Currently I am just feeding these to UMFPACK, but I am wondering if there are specialized methods that can take advantage of the dense blocks. I anticipate seeing matrices where these dense blocks can be dozens to a few hundred in dimension, with the overall block pattern maintaining sparsity. For larger dense blocks it seems like there would be a much greater advantage to knowing about the block structure.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-02-21T13:33:07.023" Title="Sparse LU for block-sparse matrices" Tags="|linear-algebra|matrix|linear-solver|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5349" PostTypeId="2" ParentId="5316" CreationDate="2013-02-21T13:00:19.607" Score="3" Body="&lt;p&gt;In the finite element method, you need to compute matrix entries (here in the case of the Laplace equation)&#xA;$$&#xA;  A_{ij} = \sum_K \int_K \nabla\varphi_i \cdot \nabla \varphi_j&#xA;$$&#xA;and right hand side entries&#xA;$$&#xA;  F_{i} = \sum_K \int_K \varphi_i(x) f(x)&#xA;$$&#xA;where $K$ are the cells of the mesh. The former can sometimes be computed analytically, but depending on the form of $f(x)$ in the latter, you may not be able to do that. If you can't analytically compute an integral, you use quadrature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if it's a shameless plug, but I've actually recorded a video lecture on this topic -- see lecture 4 here:&#xA;  &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-21T13:00:19.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5350" PostTypeId="2" ParentId="5345" CreationDate="2013-02-21T13:10:25.313" Score="4" Body="&lt;p&gt;I'm going to repeat Nathan's comment as an answer. You probably want &lt;a href=&quot;http://libmesh.sourceforge.net&quot; rel=&quot;nofollow&quot;&gt;LibMesh&lt;/a&gt;. They do have support for some &lt;a href=&quot;http://libmesh.sourceforge.net/doxygen/fe__nedelec__one__shape__2D_8C_source.php&quot; rel=&quot;nofollow&quot;&gt;Nedelec elements&lt;/a&gt;, though my first search found the 2D ones. If you look around or make a request, I'm sure you'll find the 3D ones, too. It has a huge element library.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-02-21T13:10:25.313" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5351" PostTypeId="2" ParentId="5348" CreationDate="2013-02-21T13:33:07.023" Score="4" Body="&lt;p&gt;I believe that UMFPACK will do a good job not just supporting the dense blocks in your original matrix but any dense blocks that appear in the LU factors during decomposition. UMFPACK, like most state-of-the-art sparse solvers, uses what are commonly called &quot;supernodal&quot; techniques to identify and then exploit these dense blocks to significantly improve performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sparse solver, SuperLU, &lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot; rel=&quot;nofollow&quot;&gt;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&lt;/a&gt;&#xA;also uses supernodal techniques (that's what &quot;Super&quot; refers to) so you might consider comparing its performance to UMFPACK for your particular class of problem.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-02-21T13:33:07.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5353" PostTypeId="1" AcceptedAnswerId="5369" CreationDate="2013-02-21T22:39:03.710" Score="4" ViewCount="493" Body="&lt;p&gt;I am going ahead with asking what might appear as a debugging question. But,I am looking for something conceptual that I might have missed, and this question is somewhat related to my previous question: &lt;a href=&quot;https://scicomp.stackexchange.com/q/5023/3570&quot;&gt;NVE MD simulation of inert gas: Problem maintaining equilibrium&lt;/a&gt;, although a different issue now. If the moderators decide to close this, I understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing a constant energy (NVE) simulation of 49 Argon particles in a 100x100 2D box with Lennard Jones potential at temperature 240 K I am using the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Unit system: Natural unit system such that mass $m=1$, LJ parameters, $\sigma=1$, $\epsilon=1$, Boltzmann constant $k=1$ such that temperatureof 120 K is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Initial Conditions: Square lattice position, Maxwell-Boltzmann velocities&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Integration: Velocity-verlet with periodic boundary conditions. Time step of 0.001 (2fs in actual time)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) I am checking for occurrence of a non-zero mean velocity (flying ice cubes): It does not occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unable to keep energy conservation within 1% for more than 2000 time steps (total actual time of 4ps). Most studies, lecture notes, online tutorials,etc., seem to be using these same values for time step, parameters, etc. While these never say how long they are able to conserve energy (and within what accuracy) I find research papers do 10ns on an average. I am failing 3 orders below. Velocity verlet seems like a fairly-accepted algorithm at least for something so simple! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't seem like a basic coding (i.e.,non-conceptual) mistake since the code does well until it gets to around 4ps. Some runs I can go upto 6 or 7ps, just lucky initial conditions I guess. Any helpful suggestions on where I could be going wrong? Thanks.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; 1) The error doesn't seem to scale with time-step $\Delta t$. Most papers seem to use $\Delta t$ =0.001, I have tried the range from 1e-2 to 1e-5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) My integrator code looks like this:&#xA;{ while(t&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  //Energy Calculation&#xA;  Ven(t)=ven;&#xA;  KE(U,V,ken,N);&#xA;  Ken(t)=ken;&#xA;  if(t==0)&#xA;{&#xA;  Hen1=Ken(0)+Ven(0);&#xA;  cout&amp;lt;&amp;lt;&quot;Total energy at t=0  is: &quot;&amp;lt;&amp;lt;Hen1&amp;lt;&amp;lt;endl;&#xA;}&#xA;&#xA;  //Error Checking&#xA;  ErrMeanVel=(U.sum())/N;&#xA;  ErrEConserve=(abs(Ken(t)+Ven(t)-Hen1)*100)/Hen1;&#xA;  if(ErrMeanVel&amp;gt;1e-6)&#xA;{&#xA;  cout&amp;lt;&amp;lt;&quot;Flying ice cube at t=&quot;&amp;lt;&amp;lt;t*Deltat&amp;lt;&amp;lt;&quot;ps. The mean velocity is&quot;&amp;lt;&amp;lt;ErrMeanVel&amp;lt;&amp;lt;endl;&#xA;  break;&#xA;}&#xA;  if(ErrEConserve&amp;gt;ETol)&#xA;    {&#xA;      cout&amp;lt;&amp;lt;&quot;Energy conservation violated at t=&quot;&amp;lt;&amp;lt;t*Deltat&amp;lt;&amp;lt;&quot;ps. The energy at this time is &quot;&amp;lt;&amp;lt;(Ken(t)+Ven(t))&amp;lt;&amp;lt;&quot; and the percentage error is &quot;&amp;lt;&amp;lt;((abs(Ken(t)+Ven(t)-Hen1)*100)/Hen1)&amp;lt;&amp;lt;endl;&#xA;      break;&#xA;    }&#xA;&#xA;  //Velocity Verlet&#xA;  for (i=0;i&amp;lt;N;i++)&#xA;{&#xA;  X(0,i)=X(0,i)+(U(0,i)*Deltat)+0.5*Ax(i)*Deltat*(Deltat/mass);&#xA;  Y(0,i)=Y(0,i)+(V(0,i)*Deltat)+0.5*Ay(i)*Deltat*(Deltat/mass);&#xA;  U(0,i)=U(0,i)+(0.5*Ax(i)*Deltat/mass);&#xA;  V(0,i)=V(0,i)+(0.5*Ay(i)*Deltat/mass);&#xA;}&#xA;  LJ(X,Y,N,ven,Fx,Fy,ctr);&#xA;  Ax=Fx.rowwise().sum();&#xA;  Ay=Fy.rowwise().sum();&#xA;  for (i=0;i&amp;lt;N;i++)&#xA;    {&#xA;      U(0,i)=U(0,i)+(0.5*Ax(i)*Deltat/mass);&#xA;      V(0,i)=V(0,i)+(0.5*Ay(i)*Deltat/mass);&#xA;    }&#xA;  //Boundary Conditions      &#xA;  PeriodicBoundary(X,Y,L,N);&#xA;  t++;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;\Force calculator is: (no cutoff)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; int i=0, j=0;&#xA;  double dx=0, dy=0,r2=0,F6=0,F12=0,F=0,Vcut=0;&#xA;  fx=MatrixXd::Zero(n,n);&#xA;  fy=MatrixXd::Zero(n,n);&#xA;  ven=0;&#xA;  MatrixXd ft(n,n);&#xA;  for (j=0;j&amp;lt;n;j++)&#xA;    {&#xA;      for(i=j+1;i&amp;lt;n;i++)&#xA;    {&#xA;      dx=(a(0,j)-a(0,i));&#xA;      dy=(b(0,j)-b(0,i));&#xA;      r2=dx*dx+dy*dy;&#xA;      if(r2&amp;lt;sigma)&#xA;        {&#xA;          ctr++;&#xA;        }&#xA;      F6=pow((sigma*sigma/r2),3);&#xA;      F12=pow((sigma*sigma/r2),6);&#xA;      F=(24*epsilon/r2)*(F6-2*F12);&#xA;      ven=ven+4*epsilon*(F12-F6);&#xA;      fx(i,j)=F*dx;&#xA;      fy(i,j)=F*dy;&#xA;    }&#xA;    }&#xA;  ft=fx.transpose();&#xA;  fx=fx-ft;&#xA;  ft=fy.transpose();&#xA;  fy=fy-ft;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3570" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2023-05-11T14:24:00.323" Title="Difficulty with possibly the simplest MD simulation" Tags="|molecular-dynamics|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="5354" PostTypeId="2" ParentId="5345" CreationDate="2013-02-22T01:25:33.603" Score="0" Body="&lt;p&gt;I have nothing else to offer but the projects the others have already suggested. I do like that deal.II is apparently considered the &quot;gold standard&quot; ;-) (Disclaimer: I'm one of the principal authors of deal.II :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-02-22T01:25:33.603" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5355" PostTypeId="1" AcceptedAnswerId="5358" CreationDate="2013-02-22T01:30:01.700" Score="19" ViewCount="23405" Body="&lt;p&gt;I am interested in solving the Poisson equation using the finite-difference approach. I would like to better understand how to write the matrix equation with Neumann boundary conditions. Would someone review the following, is it correct?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The finite-difference matrix&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The Poisson equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial^2u(x)}{\partial x^2} = d(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can be approximated by a finite-difference matrix equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{(\Delta x)^2} \textbf{M}\bullet \hat u = \hat d&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\textbf{M}$ is an $n \times n$ matrix and $\hat u$ and $\hat d$ are $1 \times n$ (column) vectors,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/VuQrb.png&quot; alt=&quot;Finite-difference matrix of the Poisson equation&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Adding a Neumann boundary condition&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A Neumann boundary condition enforces a know flux at the boundary (here we apply it at the left-hand side where the boundary is at $x=0$),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u(x=0)}{\partial x} = \sigma&#xA;$$&#xA;writing this boundary condition as a centred finite-difference,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fycvt.png&quot; alt=&quot;Error in equation.&quot;&gt; &lt;strong&gt;NB. I originally made an error here, sign error and didn't divide by 2. The following has been corrected.&lt;/strong&gt;&#xA;$$&#xA;\frac{u_2 - u_0}{2\Delta x} = \sigma&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the introduction of a mesh point outside the original domain ($u_0$). This term can be eliminated by introducing the second equation,&#xA;$$&#xA;\frac{u_0 - 2u_1 + u_2}{(\Delta x)^2} = d_1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation arrises from having more information because of the introduction of the new mesh point. It allows us to write the double derivative of the $u_1$ as the boundary in terms of $u_0$ using a centred finite-difference.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The part I'm not sure about&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Combining these two equations $u_0$ can be eliminated. To show the working, let's first re-arrange for the unknown,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_0 = -2\sigma\Delta x + u_2 \\&#xA;u_0 = (\Delta x)^2 d_1 + 2 u_1 - u_2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next they are set equal and rearranged into the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{u_2 - u_1}{(\Delta x)^2} = \frac{d_1}{2} + \frac{\sigma}{\Delta x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I chose this form because it is the same form as the matrix equation above. Notice that the $u$ terms are divide by $(\Delta x)^2$ both here and in the original equation. Is this the correct approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, using this equation as the first row of the matrix,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/MaCGv.png&quot; alt=&quot;Poisson equation with a Neumann boundary condition on the left-hand side (corrected)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some final thoughts,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this final matrix correct? &lt;/li&gt;&#xA;&lt;li&gt;Could I have used a better approach? &lt;/li&gt;&#xA;&lt;li&gt;Is there a &lt;strong&gt;standard&lt;/strong&gt; way of writing this matrix?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-02-25T02:24:21.810" LastActivityDate="2013-02-25T02:24:21.810" Title="Writing the Poisson equation finite-difference matrix with Neumann boundary conditions" Tags="|finite-difference|boundary-conditions|poisson|banded-matrix|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5356" PostTypeId="2" ParentId="5355" CreationDate="2013-02-22T06:13:49.853" Score="1" Body="&lt;p&gt;Great observation to see that $u_0$ can be eliminated.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step back and think about the problem for a second.  Specifying a Laplace equation fundamentally states that each point is the average of its neighbors.  This is commonly visualized as a rubber sheet, and helps me to think about these things.  (Poisson is similar w/ more or less stretchy points)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you specify the value of the solution surface at the outermost edges you are &quot;pinning&quot; the sheet down in space at those points.  When you specify the sheet by its derivative at the edges, there are any number of solutions that fulfill the equation that are translations the sheet in space whilst maintaining the same actual shape and thus derivatives.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a practical sense however this can be troublesome.  Matrices are ill-conditioned, and solvers act unpredictably.  The most common thing I have seen done is simply &quot;pinning&quot; the solution to a fixed point by specifying $u_0 = 0$ or to a constant relevant to the solution space.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2013-02-22T06:13:49.853" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5357" PostTypeId="2" ParentId="5151" CreationDate="2013-02-22T06:26:12.990" Score="2" Body="&lt;p&gt;In general I would suggest not writing your own ODE solver.  This is a well solved problem, &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_methods_for_ordinary_differential_equations&quot; rel=&quot;nofollow&quot;&gt;read about how popular ones work&lt;/a&gt; and pick one suitable to your problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you let us know what language you are working in I am sure someone can suggest their favorite package. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember, these are never exact solutions, but some are less wrong that others.&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2013-04-16T23:45:38.170" LastActivityDate="2013-04-16T23:45:38.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5358" PostTypeId="2" ParentId="5355" CreationDate="2013-02-22T10:04:35.680" Score="8" Body="&lt;p&gt;I think you are on the right way. If you correct your errors, it will look very similar to &lt;a href=&quot;http://www.math.toronto.edu/mpugh/Teaching/Mat1062/notes2.pdf&quot;&gt;http://www.math.toronto.edu/mpugh/Teaching/Mat1062/notes2.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1738" LastActivityDate="2013-02-22T10:04:35.680" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5359" PostTypeId="2" ParentId="5345" CreationDate="2013-02-22T10:10:41.763" Score="3" Body="&lt;p&gt;DUNE &lt;a href=&quot;http://www.dune-project.org&quot; rel=&quot;nofollow&quot;&gt;http://www.dune-project.org&lt;/a&gt; supports various structured and unstructured grids. If you use the underlying ALUGrid &lt;a href=&quot;http://aam.mathematik.uni-freiburg.de/IAM/Research/alugrid/&quot; rel=&quot;nofollow&quot;&gt;http://aam.mathematik.uni-freiburg.de/IAM/Research/alugrid/&lt;/a&gt; library you get a grid with hexahedral elements via our grid interface. We support gmsh &lt;a href=&quot;http://www.dune-project.org/external_libraries/install_gmsh.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dune-project.org/external_libraries/install_gmsh.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a course about DUNE in march &lt;a href=&quot;http://conan.iwr.uni-heidelberg.de/dune-workshop/index.html&quot; rel=&quot;nofollow&quot;&gt;http://conan.iwr.uni-heidelberg.de/dune-workshop/index.html&lt;/a&gt; (Registration deadline is February 24 already) if you want to learn more about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DISCLAIMER: I am one of the DUNE developers, but tried to be not too subjective in my answer.&lt;/p&gt;&#xA;" OwnerUserId="1605" LastActivityDate="2013-02-22T10:10:41.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5360" PostTypeId="1" AcceptedAnswerId="5363" CreationDate="2013-02-22T21:07:34.510" Score="4" ViewCount="132" Body="&lt;p&gt;Given two square matrices, $A$ and $B$, I need to calculate the product $tr(A^{-1}B)\times detA$. The catch is that $A$ is singular --- more precisely, it depends on some parameter $t$, such that it's non-singular at any finite $t$ and only becomes singular in the limit $t\to 0$ --- which is exactly the limit I need to evaluate the above expression. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems not unreasonable to assume that the product $det(A)\times A^{-1}$  is well-behaved even when both det and inverse are not, since the inverse contains one over the determinant. This way, I can take the limit of  $t\to 0$ analytically, and the original expression reduces to the sum over the products of the elements of $B$ and minors of $A$. Which seems to require $O(N^4)$ operations ($N$ terms, $N^3$ operations for each of the minors), where $N$ is the size of $A$ and $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if I were to just work at finite $t$-s, I'd only need $O(N^3)$ operations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My gut feeling is that there must be a way of doing the $t\to 0$ computation in just $O(N^3)$, but I don't see it. Since I doubt I'm the first person to be thinking about these things, I'd appreciate any hints, tips, pointers to the literature, counterexamples, hints at impossibility --- anything. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it matters, my matrices are dense, and they are not that huge, $N$ is about a hundred, maybe a few hundreds. But I'll need to process quite a number of such pairs, so that overall complexity matters, &lt;/p&gt;&#xA;" OwnerUserId="1004" LastActivityDate="2013-02-22T22:08:13.647" Title="simplifying a product of a determinant and an inverse of a (nearly) singular matrix" Tags="|linear-algebra|complexity|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5361" PostTypeId="2" ParentId="1136" CreationDate="2013-02-22T21:18:04.177" Score="2" Body="&lt;p&gt;The canonical way to freeze cores as described on the Gaussian keyword page is to freeze all non-valence molecular orbitals (MOs) as determined by SCF. This means that for $n$ core electrons the first $n/2$ MOs in ascending MO energy are frozen. This applies only for post-SCF methods. If you want to ensure that CFOUR and Gaussian09 freeze the same MOs you should probably specify explicitly which MOs to freeze as per the ListWindow keyword in Gaussian.&lt;/p&gt;&#xA;" OwnerUserId="522" LastActivityDate="2013-02-22T21:18:04.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5362" PostTypeId="1" AcceptedAnswerId="5364" CreationDate="2013-02-22T21:55:34.240" Score="1" ViewCount="224" Body="&lt;p&gt;I want to solve a nonlinear optimization problem&#xA;$$\underset{\mathbf{x}\in \mathbb{R}^n}{\operatorname{argmin}} f(\mathbf{x})$$&#xA;subject to a support constraint $$\mathbf{x}=[x_1,\cdots,x_n]^T, \quad x_1=x_2=\cdots=x_i=0,\quad x_j=x_{j+1}=\cdots=x_n=0 \quad (1&amp;lt;i&amp;lt;j&amp;lt;n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any algorithm or library able to solve this kind of problem?&lt;/p&gt;&#xA;" OwnerUserId="1614" LastEditorUserId="1804" LastEditDate="2013-06-26T14:59:43.473" LastActivityDate="2013-06-26T15:19:03.867" Title="nonlinear programming with support constraint" Tags="|optimization|nonlinear-programming|constrained-optimization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5363" PostTypeId="2" ParentId="5360" CreationDate="2013-02-22T22:08:13.647" Score="6" Body="&lt;p&gt;I had to do this once, and the trick is to use the singular value decomposition (SVD). The SVD of $A$ is $A=U\Sigma V^H$, so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \operatorname{tr}(A^{-1}B) \cdot \det A = \operatorname{tr}(V\Sigma^{-1}U^H B) \det \Sigma \det (UV^H)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying the cyclic permutation invariance of the trace, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ = \operatorname{tr}(\Sigma^{-1}U^H B V) \det (\Sigma)\det (UV^H) = \operatorname{diag}(\Sigma^{-1})^T\operatorname{diag}(U^H B V) \det(\Sigma)\det (UV^H) $$&#xA;where the first pair of diag's in the right-most expression is an inner product.&#xA;At this point the singularity in $\Sigma^{-1}$ can cancel with the singularity in $\det\Sigma = \prod_j \sigma_j $. This assumes $B$ is square and $A$ is rank deficient by 1.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-02-22T22:08:13.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5364" PostTypeId="2" ParentId="5362" CreationDate="2013-02-23T03:15:26.863" Score="3" Body="&lt;p&gt;If $i$ and $j$ are known, then the resulting constraints on $x_{1}, \ldots, x_{i}$ and $x_{j}, \ldots, x_{n}$ are all linear, and easy to implement in nonlinear programming solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if $i$ and $j$ are actually decision variables, then the problem should be formulated as a mixed-integer nonlinear program (MINLP) and solved using an MINLP solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I'm misunderstanding the question?&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-02-23T03:15:26.863" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5365" PostTypeId="2" ParentId="5322" CreationDate="2013-02-23T03:24:56.057" Score="3" Body="&lt;p&gt;So, there are two ways to solve this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easy, non-rigorous way to solve this problem is to create a function that calls a MATLAB ODE solver using $K$ as a parameter, and returns the solution $(x_{1}(t), x_{2}(t))$ for all times corresponding to measured data points. Then use this function to construct a sum-of-squared error objective function, and minimize using MATLAB's unconstrained optimization solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rigorous way to solve this problem is to look at the literature for solving optimization problems with embedded dynamical systems. As far as I know, there exists no MATLAB solver that will take an optimization problem with an embedded dynamical system as input and return a rigorous global $\varepsilon$-optimum. Such methods normally require a nonconvex global optimization solver (which MATLAB does not have, to my knowledge), interval arithmetic libraries (which do exist for MATLAB; INTLAB is one such library), automatic differentiation tools (which also exist for MATLAB using operator overloading and object-oriented language features), and a lot of mathematical theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since people tend not to want to bother with all of the mathematical theory (there is a lot of it), the first approach tends to be favored by practitioners (it's what I was taught as an undergrad), even though it only yields a local optimum due to the nonconvexity of the problem (in most cases, anyway). Occasionally, the second approach is useful for parameter fitting; both of my advisers used this approach for a chemical kinetics application because the first approach was insufficient. (I'm struggling to remember the name of the paper, though, and being pressed for time, I'll have to come back to this answer later.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-02-23T03:24:56.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5366" PostTypeId="1" CreationDate="2013-02-23T04:53:57.590" Score="4" ViewCount="337" Body="&lt;p&gt;For your information, the original equation comes from &lt;a href=&quot;http://pre.aps.org/abstract/PRE/v47/i6/p4037_1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Note: You DON'T have to read the paper. I will make the question as self-contained as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The central equation to solve is equation (16), which is of the form (my question extends to other system of PDE with similar form):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial f_n(x;\tau)}{\partial \tau}&#xA;=\left[&#xA;[-(n-6)+\dot{\bar{A}}x]\dfrac{\partial}{\partial x}+2\dot{\bar{A}}-(c_++c_-)n&#xA;\right]f_n(x;\tau)&#xA;+c_+(n-1)f_{n-1}(x;\tau)+c_-(n+1)f_{n+1}(x;\tau)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x$ is continuous and positive, $\tau$ is basically time, $n$ is positive integer number and $f_n(x;t)$ goes to zero very quickly as $x\rightarrow\infty$ so that we can have a cutoff $x_{max}$. The boundary conditions used by the paper is called &quot;used open boundary conditions, i.e., probability ($f_n(x;\tau)$ is proportional to probability density) could flow out of the system across the cutoffs, but no probability could flow into the system, since $f_n(x)$vanished beyond the cutoffs.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;additional clarifications for the symbols: $\dot{\bar{A}}=\sum_{k=0}^5(6-k)f_k(0;\tau)$, $c_+=(1/6)f_5(0;\tau)$ and $c_-=(1/6)\sum_{k=0}^5(6-k)f_k(0;\tau)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding of the above description is that the BC looks like some kind of absorbing boundary conditions. The &quot;mass&quot; of $f_n$ can flow outside the interval $[0,x_{max}]$ but nothing outside can flow into the interval, such that the integral of $f_n(x;\tau)$ over the interval will decrease with time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we discretize $x$, it's not very difficult to write a program for explicit scheme according to the above description, but the implicit scheme would be difficult (there are two index $x$ and $n$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: &lt;strong&gt;how to write down the mathematical statement (for continuous $x$) for such boundary conditions?&lt;/strong&gt; The motivation for writing down a mathematical statement instead of directly writing down the BC for discretized $x$ is that I can let software such as &lt;em&gt;Mathematica&lt;/em&gt;. Then why don't I ask the question in Mathematica exchange? Yes, I &lt;a href=&quot;https://mathematica.stackexchange.com/questions/20003/open-boundary-condition-in-ndsolve&quot;&gt;did&lt;/a&gt;. But the folks there seemingly thinks that I should figure out the mathematical statement first.&lt;/p&gt;&#xA;" OwnerUserId="3512" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2013-02-24T09:37:19.197" Title="mathematical statement of &quot;open&quot; boundary condition" Tags="|pde|hyperbolic-pde|boundary-conditions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5367" PostTypeId="2" ParentId="5366" CreationDate="2013-02-23T11:38:56.447" Score="2" Body="&lt;p&gt;I would sum this up as &lt;em&gt;a coupled system of advection-reaction equations&lt;/em&gt;.  You should only need a boundary condition in $x$ at the boundary where characteristics flow into the domain, which depends on the sign of the coefficient of the $x$-derivative term.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-02-23T11:38:56.447" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5368" PostTypeId="1" CreationDate="2013-02-23T23:00:38.037" Score="13" ViewCount="332" Body="&lt;p&gt;I have to solve generalized eigenvalue problems $Ax = \lambda Bx$ where $A$ and $B$ are both tridiagonal, $B$ is symmetric positive definite and real, but $A$ is only complex symmetric (not definite or Hermitian). Furthermore, I need the full eigendecomposition. I am currently just calling Lapack's &lt;code&gt;ZGGEV&lt;/code&gt; generalized eigensolver, but I am wondering if there are better methods for this particular, highly structured problem. In particular, having freely available code (C++) would be the best.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="4405" LastEditDate="2016-01-28T10:36:02.157" LastActivityDate="2016-04-27T11:33:34.253" Title="Specialized methods for complex symmetric tridiagonal generalized eigenvalue problems" Tags="|linear-algebra|eigensystem|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5369" PostTypeId="2" ParentId="5353" CreationDate="2013-02-24T11:53:45.210" Score="8" Body="&lt;p&gt;This is a re-formulation of my comments above as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to enforce the boundary conditions when computing &lt;code&gt;dx&lt;/code&gt; and &lt;code&gt;dy&lt;/code&gt; to account for particles that interact over the periodic boundary. Imagine two particles at opposite ends of the domain: In your current setup they won't interact, but as soon as one of them moves slightly and crosses the boundary, they will all of a sudden see each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most simulations just use what's referred to as the &lt;a href=&quot;http://en.wikipedia.org/wiki/Periodic_boundary_conditions#Practical_implementation%3a_continuity_and_the_minimum_image_convention&quot; rel=&quot;noreferrer&quot;&gt;minimum image convention&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-02-24T11:53:45.210" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5370" PostTypeId="2" ParentId="5302" CreationDate="2013-02-25T14:51:07.980" Score="1" Body="&lt;p&gt;Following your answers to my questions in the comments above, I would suggest you try a lock-based approach in which each thread tries to lock-down the neighbourhood it will be updating before computing the actual update.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this using the atomic operations provided for in CUDA, and an array of &lt;code&gt;int&lt;/code&gt; containing the locks for each cell, e.g. &lt;code&gt;lock&lt;/code&gt;. Each thread then does the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ci, cj = choose a pair at random.&#xA;&#xA;int locked = 0;&#xA;&#xA;/* Try to lock the cell ci. */&#xA;if ( atomicCAS( &amp;amp;lock[ci] , 0 , 1 ) == 0 ) {&#xA;&#xA;    /* Try to lock the cell cj. */&#xA;    if ( atomicCAS( &amp;amp;lock[cj] , 0 , 1 ) == 0 ) {&#xA;&#xA;        /* Now try to lock all the neigbourhood cells. */&#xA;        for ( cn = indices of all neighbours )&#xA;            if ( atomicCAS( &amp;amp;lock[cn] , 0 , 1 ) != 0 )&#xA;                break;&#xA;&#xA;        /* If we hit a break above, we have to unroll all the locks. */&#xA;        if ( cn &amp;lt; number of neighbours ) {&#xA;            lock[ci] = 0;&#xA;            lock[cj] = 0;&#xA;            for ( int i = 0 ; i &amp;lt; cn ; i++ )&#xA;                lock[i] = 0;&#xA;            }&#xA;&#xA;        /* Otherwise, we've successfully locked-down the neighbourhood. */&#xA;        else&#xA;            locked = 1;&#xA;&#xA;        }&#xA;&#xA;    /* Otherwise, back off. */&#xA;    else&#xA;        lock[ci] = 0;&#xA;    }&#xA;&#xA;/* If we got everything locked-down... */&#xA;if ( locked ) {&#xA;&#xA;    do whatever needs to be done...&#xA;&#xA;    /* Release all the locks. */&#xA;    lock[ci] = 0;&#xA;    lock[cj] = 0;&#xA;    for ( int i = 0 ; i &amp;lt; cn ; i++ )&#xA;        lock[i] = 0;&#xA;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this approach is probably not the most optimal, but it could provide an interesting starting point. If there are a lot of collisions between threads, i.e. one or more per 32 threads (as in one collision per warp), then there will be a fair bit of branch diversion. Also, atomic operations can be a bit slow, but since you are only doing compare-and-swap operations, it should scale ok.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The locking overhead may look intimidating, but it's really only a few assignments and branches, not much more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that I'm being fast and loose with notation in the loops of &lt;code&gt;i&lt;/code&gt; over the neighbours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum:&lt;/strong&gt; I was cavalier enough to assume that you could simply back off when to pairs collide. If this is not the case, then you can wrap everything as of the second line in a &lt;code&gt;while&lt;/code&gt;-loop and add a &lt;code&gt;break&lt;/code&gt; at the end of the final &lt;code&gt;if&lt;/code&gt;-statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the threads will then have to wait until the last one is done, but if collisions are rare, you should be able to get away with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum 2:&lt;/strong&gt; Do &lt;em&gt;not&lt;/em&gt; be tempted to add calls to &lt;code&gt;__syncthreads()&lt;/code&gt; anywhere in this code, especially it the looping version described in the previous addendum! Asynchronicity is essential in avoiding repeated collisions in the latter case.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2013-02-25T16:08:39.157" LastActivityDate="2013-02-25T16:08:39.157" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5371" PostTypeId="1" CreationDate="2013-02-25T15:49:48.597" Score="8" ViewCount="1787" Body="&lt;p&gt;Hello if anyone can offer insight on how to solve my problem that would be great!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to calculate the area of intersection between a 3D volume and a 2D plane.&#xA;3D volume: defined by 6 points (will always be a 3D wedge resembling a thick triangle)&#xA;2D plane: defined by 3 points &lt;/p&gt;&#xA;&#xA;&lt;p&gt;any suggestions?&#xA;are there any open source software out there that can perform this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks very much&#xA;Chris&lt;/p&gt;&#xA;" OwnerUserId="3794" LastActivityDate="2013-02-26T08:11:40.883" Title="How to calculate the area of intersection between a 3D volume and a 2D plane" Tags="|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5372" PostTypeId="1" CreationDate="2013-02-24T22:36:49.367" Score="21" ViewCount="4518" Body="&lt;p&gt;I have just installed a Nvidia GT660 graphic card on my desktop&#xA;and, after some struggle, I manage to interface it with R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been playing with several R packages that use GPUs, &#xA;especially gputools, and I was comparing the time taken by my&#xA;GPU and CPU to perform some basic operations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;inverting matrices (CPU faster)&lt;/li&gt;&#xA;&lt;li&gt;qr decomposition (CPU faster)&lt;/li&gt;&#xA;&lt;li&gt;big correlation matrices (CPU faster)&lt;/li&gt;&#xA;&lt;li&gt;matrix multiplication (GPU much faster!)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Notice that I have experimented mainly with gputools so maybe&#xA;other packages perform better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In broad terms my question is: what some routine statistical &#xA;operations that might be worth executing on a GPU rather than a CPU? &lt;/p&gt;&#xA;" OwnerUserId="2140" OwnerDisplayName="Jugurtha" LastActivityDate="2015-05-16T14:52:01.933" Title="For which statistical methods are GPUs faster than CPUs?" Tags="|r|gpu|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5373" PostTypeId="2" ParentId="5372" CreationDate="2013-02-25T07:40:04.733" Score="0" Body="&lt;p&gt;Multiple Imputation methods for Missing Data? Like those in Alice-II (R). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think those tend to be often embarrassingly parallel and hence suitable to a GPU architecture. Never tried it myself though. &lt;/p&gt;&#xA;" OwnerUserId="3795" OwnerDisplayName="curious_cat" LastEditorUserId="3795" LastEditDate="2013-02-25T18:07:28.953" LastActivityDate="2013-02-25T18:07:28.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5374" PostTypeId="2" ParentId="5372" CreationDate="2013-02-25T19:49:33.873" Score="4" Body="&lt;p&gt;For all of the applications you mentioned, GPUs should be more capable (from a hardware perspective) than CPUs for sufficiently large matrices.  I don't know anything about R's implementation, but I've used &lt;a href=&quot;https://developer.nvidia.com/cublas&quot; rel=&quot;nofollow&quot;&gt;cuBLAS&lt;/a&gt; and &lt;a href=&quot;http://icl.cs.utk.edu/magma/index.html&quot; rel=&quot;nofollow&quot;&gt;Magma&lt;/a&gt; with great success for inversions around $n = 2^{10}$ and multiplication/correlation for rectangular matrices with $n,m \approx 2^{10}, k \approx 2^{14}$.  It is an especially big surprise to me that large correlation matrices would be faster on the CPU using R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More broadly, I suspect most statistical operations that spend most of their time in dense linear algebra (BLAS, Lapack functionality) can be efficiently implemented on the GPU.  &lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2013-02-25T19:49:33.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5376" PostTypeId="1" AcceptedAnswerId="5377" CreationDate="2013-02-25T22:31:38.337" Score="4" ViewCount="742" Body="&lt;p&gt;I'm interesting in numerically simulating (with my own code, not an off-the-shelf package) the motion of a single electrically charged particle in a magnetic field. The field will vary in time and space but probably not fast compared to the cyclotron frequency or the Larmor radius.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many well known methods for integrating equations of motion for particles in fields. But the force due to the magnetic field has some special properties such as not changing the kinetic energy of the particle. Are there some numerical procedures that are better adapted to this special case eg. that will give a more accurate simulation and better conserve adiabatic invariants (when they should be conserved)?&lt;/p&gt;&#xA;" OwnerUserId="3797" OwnerDisplayName="sigfpe" LastEditorUserId="3797" LastEditDate="2013-02-26T00:13:42.227" LastActivityDate="2013-04-11T14:54:22.223" Title="Numerical simulation of particle in magnetic field" Tags="|electromagnetism|simulation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5377" PostTypeId="2" ParentId="5376" CreationDate="2013-02-26T05:25:20.683" Score="4" Body="&lt;p&gt;I believe what you are looking for is a sympletic method; see:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Symplectic_integrator&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Symplectic_integrator&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are designed to conserve energy exactly; however by trading up for accuracy one area, your simulation will have greater errors in other quantities (i.e. position as a function of time) compared to your usual Runge-Kutta methods.  &lt;/p&gt;&#xA;" OwnerUserId="3799" LastActivityDate="2013-02-26T05:25:20.683" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5378" PostTypeId="1" AcceptedAnswerId="5393" CreationDate="2013-02-26T05:38:10.837" Score="0" ViewCount="110" Body="&lt;p&gt;Consider $n$ integer-valued and independent random variables $e_1, e_2, \dots, e_n$ with known distribution functions $m_1, m_2, \dots, m_n$. Let's denote with $E^{1..n}$ the random variable given by the sum of $e_1, e_2, \dots e_n$, that is  $E^{1..n} = \displaystyle\sum_i{e_i}$. The PDF of $E^{1..n}$, $m^{1..n}$, is given by the convolution of $m_1, m_2, \dots, m_n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to calculate $P(E^{1..n} = 1 \,| \,e_1 = 1)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intuition tells that that this is equal to $P(E^{2..n} = 0) P(e_1=1)$ but I just can't figure out the math behind it.&lt;/p&gt;&#xA;" OwnerUserId="1922" LastActivityDate="2013-02-27T07:06:29.037" Title="sum of random variables, probability 101" Tags="|probability|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5379" PostTypeId="2" ParentId="5371" CreationDate="2013-02-26T08:00:21.660" Score="5" Body="&lt;p&gt;There is not going to be a magic bullet answer to this question; at some point you will just have to suck it up and consider all the cases. I once had to compute the intersection of a triangle and a circle... it was awful. Since your 3D shape has certain symmetries (like the fact that it is always a triangular prism) helps narrow down the possibilities tremendously.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find which side of the plane each point lies.&lt;/li&gt;&#xA;&lt;li&gt;If all points are on one side, done.&lt;/li&gt;&#xA;&lt;li&gt;If one point is separated from the other 5 by the plane, the intersection is a triangle. Presumably, you have connectivity information to tell you which faces are incident on the vertex, and you can compute the 3 lines due to plane-plane intersections, and then the intersection triangle. Best do all this in a parameterization of the intersecting plane.&lt;/li&gt;&#xA;&lt;li&gt;Two points are separated, either both are on the same triangular face of the prism, or they are two corresponding points on opposite triangular faces. In either case, the intersection is a quadrilateral.&lt;/li&gt;&#xA;&lt;li&gt;For three points, they could all be the same triangular face or only two on it. You have two cases here, for a resulting triangle or hexagon.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Those are really the only high level cases, there are obviously symmetry reductions you need to apply (in case 5, the one point case is equivalent to the 2 point case on the other triangular facet).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My one suggestion to you is to pick a robust representation for your shapes, and to use robust predicates to perform the geometric tests. For the plane, it is best represented as a point on the plane, and a unit normal vector. The prism is represented by defining a orthonormal basis (triad) with one axis aligned with the extrusion direction of the prism. Let one vertex be at the origin, the other two on the triangular face be represented in the uv-coordinates of the triad, and then you only need to store its height, and global offset of its base vertex. Essentially, I'm thinking&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;struct plane{&#xA;    double p[3]; // point on plane&#xA;    double n[3]; // unit normal vector to plane&#xA;};&#xA;struct TriPrism{&#xA;    double basis[9]; // 3x3 orthogonal matrix of local coordinate frame (det = +1)&#xA;    // Stored columnwise, first two vectors are in the plane of the triangular face)&#xA;    // Last vector is parallel to extrusion direction, call the set [u, v, w]&#xA;    double base[3]; // global coordinates of base vertex (where the basis vectors are &quot;rooted&quot;)&#xA;    double buv[2]; // the uv-coordinates of the second point on the triangular face&#xA;    double cuv[2]; // third point on triangular face&#xA;    // Assume that the &quot;bottom&quot; triangular face is formed by vertices (a,b,c)&#xA;    // with base being a, and (b-a) cross (c-a) directed along vector w (instead of against)&#xA;    double h; // height of prism (&quot;top&quot; triangular face is h*w offset from the &quot;bottom&quot; face)&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This representation is robust to moving the prism around and should maintain high relative precision for all but the most pathological cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for robust predicates, I highly recommend &lt;a href=&quot;http://www.cs.cmu.edu/~quake/robust.html&quot;&gt;Shewchuk's robust predicates&lt;/a&gt;&#xA;assuming you use ordinary floating point numbers. You would mainly use &lt;code&gt;orient3d&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of representing the final output polygon, choose a suitable parameterization of the plane. The best choice, in my opinion, is to first choose an orthonormal basis set in the plane, determined by Gram-Schmidt on the plane normal vector (&lt;a href=&quot;https://github.com/victorliu/Cgeom/blob/master/geom_la.c&quot;&gt;see geom_maketriad3d here&lt;/a&gt;). Then, let the origin of that 2D parametric plane be the projection of the prism base point into the plane. This ensures your parameterization is actually rooted near where the resulting polygon will be, to ensure effective use of the bits of the floating point numbers involved. Do all the remaining computations using this parameterization if at all possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, geometric computation is fraught with peril due to these silly numerical considerations (a slight error with a vertex near the plane in case 5 above could drastically change the result from a 4-gon to a 6-gon). I suggest you handle a few cases at a time, and try to visualize the result. I have a rather &lt;a href=&quot;https://github.com/victorliu/PrimView&quot;&gt;simple viewer program here&lt;/a&gt;, with a kind of postscript-like input language capability. You can dump the facets of the prism and draw the plane and also draw the resulting polygons and visually check them to see if they're right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum&lt;/strong&gt;: I forgot that you originally wanted the area of the intersection polygon. It is trivial, but in case you don't know, once you have the collection of lines defining the edges of the polygon in the cut plane, you have to convert it to a vertex representation. Since you constructed the intersections from facetplane-and-plane intersections, you should have been able to keep track of the ordering of the lines, and hence the ordering of the edges going around the polygon. You only need to compute the intersection of successive pairs of these lines to get the vertices. Once you have the vertices, it's a simple matter to get the area (&lt;a href=&quot;https://github.com/victorliu/Cgeom/blob/master/geom_poly.c&quot;&gt;see e.g. geom_polygon_area2d here&lt;/a&gt;). If you worked entirely in uv-coordinates of the cut-plane, then you can feed them directly to such a function to get the area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should add that there is an obvious dumb approach, which is to pick a suitable large region in the cut-plane, and randomly sample points and checking to see if they're in the prism (which is cheap, since it's convex). Then you can calculate the area as a ratio of points inside vs. total points, times the area of the sampling region. If you &lt;em&gt;really&lt;/em&gt; don't care about accuracy, then this will probably be faster, but otherwise, the analytic method should not be much slower, despite its combinatorial complexity.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2013-02-26T08:11:40.883" LastActivityDate="2013-02-26T08:11:40.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5380" PostTypeId="1" AcceptedAnswerId="5382" CreationDate="2013-02-26T09:20:07.083" Score="3" ViewCount="2278" Body="&lt;p&gt;I am currently trying to link a program against the Intel MKL 11.0 library instead of using NetLIB or OpenBLAS. Doing this I recognized the following error which I can not explain to my self at the moment. Consider the following C code example computing a complex scalar product using &lt;code&gt;zdotc&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;complex.h&amp;gt;&#xA;double complex zdotc_(int *n, double complex *X, int *incx, double complex *Y, int *INCY ); &#xA;int main ( ) {&#xA;    int n = 5; &#xA;    int incx = 1, incy = 1; &#xA;    double complex x[5] = {1,I,2,2+I,3}; &#xA;    double complex y[5] = {I,3,I*3, 2+2*I, 9}; &#xA;    double complex ret; &#xA;    ret = zdotc_(&amp;amp;n,x,&amp;amp;incx,y,&amp;amp;incy); &#xA;    printf(&quot;n   = %d\n&quot;, n); &#xA;    printf(&quot;ret = %lg + %lgi\n&quot;, creal(ret), cimag(ret)); &#xA;    return 0; &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I compiled this example using the command line flags given by the &lt;a href=&quot;http://software.intel.com/sites/products/mkl/&quot; rel=&quot;nofollow&quot;&gt;MKL Advisor&lt;/a&gt;. I select &quot;GNU C/C++, 32 Bit Integer, Dynamic Linking, GNU OpenMP&quot;. The resulting command line is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gcc zdotc_test.c -o zdot_mkl_gcc -O2  -L$MKLROOT/lib/intel64 -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -ldl -lpthread -lm  -fopenmp -m64 -I$MKLROOT/include&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output of this program is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n   = 0&#xA;ret = 0 + 1.07933e+21i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is obviously wrong and especially why is &lt;strong&gt;n&lt;/strong&gt; altered?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I select GNU Fortran instead of GNU C/C++, I have to replace &lt;code&gt;-lmkl_intel_lp64&lt;/code&gt; by &lt;code&gt;-lmkl_gf_lp64&lt;/code&gt; and then the correct output &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n   = 5&#xA;ret = 33 + 6i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is produced. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: where are the detailed differences between those to interfaces and why does the first one produced this error?&lt;/p&gt;&#xA;" OwnerUserId="569" LastActivityDate="2013-02-26T10:03:08.307" Title="Intel MKL - Difference between mkl_intel_lp64 and mkl_gf_lp64" Tags="|blas|compiling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5381" PostTypeId="1" CreationDate="2013-02-26T10:02:27.583" Score="5" ViewCount="180" Body="&lt;p&gt;I am looking for a modern version or an alternative to the &lt;a href=&quot;http://www.netlib.org/ode/dresol.f&quot;&gt;DRESOL&lt;/a&gt; package for differential matrix Riccati equations. The main issue that the original package uses single-precision type &lt;code&gt;float&lt;/code&gt; which seems to be not accurate enough for my purposes. Available Matlab alternatives are also appreciated. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2013-06-26T19:42:34.913" Title="Modern alternatives to DRESOL Riccati solver" Tags="|software|reference-request|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5382" PostTypeId="2" ParentId="5380" CreationDate="2013-02-26T10:03:08.307" Score="3" Body="&lt;p&gt;The difference has to do with the calling convention (ABI) differences. &lt;code&gt;ZDOTC&lt;/code&gt; is a problematic function because it returns a &lt;code&gt;double complex&lt;/code&gt;, which is usually considered a struct rather than a &quot;simple&quot; data type. Thus, it can either be returned on the stack as a return value, or by reference as an implicit first argument to the &quot;C&quot; style function. I am guessing that what you are seeing here is that in the &lt;code&gt;_intel_&lt;/code&gt; case, it expects the return value to be an implicit first argument, so it returns the value and ends up modifying n, and the return value is then just completely wrong. In that case the correct declaration is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void zdotc_(double complex *retval, int *n, double complex *X, int *incx, double complex *Y, int *INCY );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's stupid stuff like this that makes me implement everything in C++ if possible. In particular, I tend to implement BLAS level 1 in C++, where performance is limited by memory bandwidth anyways. For BLAS levels 2 and 3, I will call actual BLAS, which does not have functions with this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe I've seen benchmarks before comparing naive dot product implementations with optimized BLAS, and if you use &lt;code&gt;__restrict&lt;/code&gt; type keywords, most optimizing compilers will produce code that's as fast as the optimized BLAS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own version is located &lt;a href=&quot;https://github.com/victorliu/RNP2/blob/master/RNP/BLAS/BLAS_mix.hpp&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, where you would need the corresponding source file to provide forwarding of higher level BLAS, located &lt;a href=&quot;https://github.com/victorliu/RNP2/blob/master/src/BLAS_mix.cpp&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. I would refer you to &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt;, but my code is slightly easier to port to C99 since it's less heavily templated.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-02-26T10:03:08.307" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5383" PostTypeId="1" CreationDate="2013-02-26T12:45:43.817" Score="4" ViewCount="5936" Body="&lt;p&gt;I want to solve a system of differential equations using ode45 , for (many) parameters in MATLAB to investigate the behavior of solutions with coefficients of the equation .I want to find the parameters &lt;strong&gt;for which the solution absolute value doesn't get  larger than some given number&lt;/strong&gt; . The script has a for loop and solves the system for given parameters one after another(in some limited time span (independent variable is time)) . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now,my question is that &quot;how can I set a condition on ode45 to &quot;detect&quot; automatically if the solution for some parameter gets larger than that (given) value&quot;, so I can automatically find those parameters for which the solution gets larger than the given value, by running the for loop once?&lt;/p&gt;&#xA;" OwnerUserId="3725" LastActivityDate="2014-05-22T08:18:17.023" Title="setting a condition on MATLAB ode45 output" Tags="|matlab|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5384" PostTypeId="1" AcceptedAnswerId="5394" CreationDate="2013-02-26T15:00:35.780" Score="3" ViewCount="2451" Body="&lt;p&gt;I have multiple sets of measured data that can easily be visualized using a scatter plot (red and black points in the figure). If my measurements were perfect, the red and black points should lie on a single curve. However they are not, thus I would like to &quot;average&quot;/fit the data points in such a way that I obtain the black line in the figure. The black line could be a line or a set of discrete points with minimum distance to all the measurement data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me get started? Ideally there is already a python library that can handle this for me, but I'm also happy with algorithm descriptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sKkxU.png&quot; alt=&quot;Scattered data and averaged/fitted curve&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="414" LastEditDate="2013-02-27T07:49:18.250" LastActivityDate="2013-02-27T15:09:47.980" Title="Averaging scattered data" Tags="|python|regression|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5385" PostTypeId="2" ParentId="5384" CreationDate="2013-02-26T16:20:40.490" Score="0" Body="&lt;p&gt;A simple idea to obtain a continuous piecewise linear approximation would be to make the average of each pair of points (a red and the corresponding black) and make a &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_interpolation&quot; rel=&quot;nofollow&quot;&gt;piecewise linear interpolation&lt;/a&gt; of these averages. That would produce something close to what you show. (However, it does not make use a regression)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would that fit your needs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the implementation, I think that you can do that manually, the method is very simple.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-02-26T16:20:40.490" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5386" PostTypeId="1" CreationDate="2013-02-26T17:24:27.267" Score="2" ViewCount="599" Body="&lt;p&gt;I wanted to compare the speeds of the GMRES implementations in the &lt;a href=&quot;http://cusplibrary.github.com/&quot; rel=&quot;nofollow&quot;&gt;CUSP&lt;/a&gt; and the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt; libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix (A) used for testing was a &lt;strong&gt;3d Laplacian&lt;/strong&gt; matrix obtained by using the 7 point stencil on a 20x20x20 grid =&gt; the matrix is of size 8000 x 8000. This is a &lt;strong&gt;banded&lt;/strong&gt; matrix with 7 diagonals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The RHS (b) was obtained by multiplying A by a vector of 1's. I wanted to solve Ax=b &#xA;by GMRES &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For both PETSC and CUSP  I timed only the time taken for 200 iterations of the GMRES solver&#xA;(ie time taken for reading in the matrix is not counted) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Petsc the timings were done with PetscGetTime() and on Cusp with Cuda events.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;%--------------------------&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Timings&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Petsc&lt;/strong&gt; (1 cpu) took &lt;strong&gt;0.0558379 sec&lt;/strong&gt;  to do 200 iterations of GMRES.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cusp&lt;/strong&gt;          took &lt;strong&gt;0.1183 sec&lt;/strong&gt;  to do the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;%----------------------------&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see Petsc is 2.14 times faster than the Cusp library even on a single processor. &#xA;The speed gap gets larger (obviously) on using more than 1 processor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This experiment was done on a  &lt;strong&gt;GTX 570&lt;/strong&gt; card on an &lt;strong&gt;Ubuntu 10.10&lt;/strong&gt; running &lt;strong&gt;CUDA 4.0&lt;/strong&gt; having &lt;strong&gt;cusp v 0.3.1&lt;/strong&gt; and &lt;strong&gt;thrust 1.6.0&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read &lt;a href=&quot;http://mgarland.org/files/papers/nvr-2008-004.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; paper by one of the authors of CUSP. &lt;em&gt;However, they have &#xA;not compared the performance of CUSP with Petsc in it.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper also says that the data-structure used for storing the sparse matrix matters. &#xA;Accordingly they have provided several formats like DIA, ELL, HYB, COO etc. &#xA;but even after I tried them all, the Cusp GMRES performance does not change and still takes 0.1183 seconds for the 200 iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://pastebin.com/hxgk0aEB&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is the (quite-short) GPU code on pastebin I used while using CUSP. I am not sure if I am using the Cusp &#xA;library in an optimal manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although it is possible PETSc is still genuinely better than CUSP I want to know if significant speedup is possible with CUSP if used properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If more information is needed to answer this question please let me know. &#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="3352" LastEditorUserId="3352" LastEditDate="2013-02-26T17:30:35.513" LastActivityDate="2013-02-26T17:30:35.513" Title="Cusp Library performance worse than PETSC (GMRES 200 iterations) Why?" Tags="|petsc|performance|cuda|gmres|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5387" PostTypeId="2" ParentId="5381" CreationDate="2013-02-26T18:14:28.637" Score="1" Body="&lt;p&gt;Have you considered &lt;a href=&quot;http://www.slicot.org/index.php&quot; rel=&quot;nofollow&quot;&gt;SLICOT&lt;/a&gt;? I'm not sure they would count as modern for many definitions of that term, but they do support double precision, as &lt;a href=&quot;http://www.slicot.org/shared/doc/SB02PD.html&quot; rel=&quot;nofollow&quot;&gt;this help page&lt;/a&gt; shows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do note that, while the web site claims that the code is available under the GPL, none of the links to the sources seem to work.&lt;/p&gt;&#xA;" OwnerUserId="49" LastActivityDate="2013-02-26T18:14:28.637" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5388" PostTypeId="2" ParentId="4695" CreationDate="2013-02-26T21:29:42.520" Score="2" Body="&lt;p&gt;Speaking from my personal experience, I know quite excellent computational scientists who have little to no actual programming knowledge or experience. On the other hand, most computational scientists I know do know how to program a computer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no final answer to that part of your question. The question you have to ask yourself is: Do I have software available to me that will let me solve my problem, or is part of my job going to be to write the software that can solve the problem first and then use it to solve the actual problem. The answer to this will determine if you need to know programming or not.&lt;/p&gt;&#xA;" OwnerUserId="3806" LastActivityDate="2013-02-26T21:29:42.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5389" PostTypeId="1" CreationDate="2013-02-26T22:22:18.933" Score="9" ViewCount="1290" Body="&lt;p&gt;I am a physicist who is trying to model the current-voltage characteristics of a superconductor-superconductor junction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://i.stack.imgur.com/PDNYl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;equation&lt;/a&gt; for this model is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;I(V) = \frac{1}{eR_{\mathrm{n-n}}}\int_{-\infty}^{\infty}\frac{|E|}{[E^{2} - \Delta_{1}^{2}]^{1/2}}\frac{|E + eV|}{[(E + eV)^{2} - \Delta_{2}^{2}]^{1/2}}[f(E) - f(E + eV)]\,\mathrm{d}E&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current ($I$ or &lt;code&gt;I&lt;/code&gt; in the code) values are calculated by evaluating this integral for given voltages ($V$, or &lt;code&gt;v&lt;/code&gt; in the code).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have attempted this in Python. The code is shown below.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from scipy import integrate&#xA;from numpy import *&#xA;import pylab as pl&#xA;import math&#xA;&#xA;ec = 1.6021764*10**(-19)&#xA;r = 2500&#xA;gap = 200*10**(-6)*ec&#xA;g = (gap)**2&#xA;t = 0.04&#xA;k = 1.3806503*10**(-23)&#xA;kt = k*t&#xA;&#xA;v_values = arange(0,0.001,0.00001)&#xA;&#xA;I=[]&#xA;for v in v_values:&#xA;    result, error = integrate.quad (lambda E:(abs(E)/sqrt((E**2-g)))*(abs(E+ec*v)/(sqrt(((E+ec*v)**2-g))))*(math.exp(-E/kt)*(math.exp(-ec*v/kt)-1)),(-inf),(-gap*0.9-ec*v))&#xA;    I.append(result)&#xA;I = array(I)&#xA;&#xA;I2=[]&#xA;for v in v_values:&#xA;    result2 = integrate.quad(lambda E:(abs(E)/sqrt((E**2-g)))*(abs(E+ec*v)/(sqrt(((E+ec*v)**2-g))))*(math.exp(-E/kt)*(math.exp(-ec*v/kt)-1)),(gap*0.9),(inf))&#xA;    I2.append(result2)&#xA;I2 = array(I2)&#xA;&#xA;pl.plot(v_values,I,'-b',v_values,I2,'-r')&#xA;pl.xlabel(r'Voltage ($V$)')&#xA;pl.ylabel(r'Current ($A$)')&#xA;pl.title('Theoretical I(V) curve')&#xA;pl.grid(True)&#xA;pl.savefig('IVcurve.png')&#xA;pl.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I receive &lt;code&gt;OverflowError: math range error&lt;/code&gt;. Does anybody have any ideas how this can be overcome? Apologies for the &lt;code&gt;10**n&lt;/code&gt; and long integrals. The code runs when the exponentials are removed (returns 0), and herein lies the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas how this can be modelled in Python, or any other language?&lt;/p&gt;&#xA;" OwnerUserId="3807" LastEditorUserId="1804" LastEditDate="2013-04-19T15:56:09.500" LastActivityDate="2013-04-19T15:56:09.500" Title="Numerical integration for modelling curve for superconductors (Python)" Tags="|python|numpy|integral-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5391" PostTypeId="2" ParentId="5287" CreationDate="2013-02-27T01:15:11.823" Score="0" Body="&lt;p&gt;After significant muddling through, I figured one important issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;hsvd()&lt;/code&gt; essentially results in the formation of a balanced system. The process of balancing involves taking the &lt;code&gt;chol()&lt;/code&gt; of the gramians. &lt;code&gt;chol()&lt;/code&gt; requires the input matrix to be positive-definite. Grammians are positive-definite only when the system is controllable(or observable, if you're considering the observability grammian). Being completely controllable and observable means the system is minimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My system was most assuredly NOT minimal. So &lt;strong&gt;&lt;code&gt;hsvd()&lt;/code&gt; was reporting my unobservable/uncontrollable modes&lt;/strong&gt; as &lt;strong&gt;unstable&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like others to clarify if this is really a bug. Also refer to my other &lt;a href=&quot;https://electronics.stackexchange.com/questions/59265/are-descriptor-systems-necessarily-non-minimal&quot;&gt;question&lt;/a&gt;, which is not a full follow-up of this question (i.e. I still want an answer to this).&lt;/p&gt;&#xA;" OwnerUserId="3740" LastEditorUserId="-1" LastEditDate="2017-04-13T12:33:11.577" LastActivityDate="2013-02-27T06:55:55.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5393" PostTypeId="2" ParentId="5378" CreationDate="2013-02-27T07:06:29.037" Score="1" Body="&lt;p&gt;As said in my comments of the question, a better place for asking this kind of question would be on the math part of stackexchange.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, we can find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$P(E^{1..n}=1|e_1=1) = P(E^{2..n}+e_1=1|e_1=1) = P(E^{2..n}=0|e_1=1) = P(E^{2..n}=0)$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the last equality being the result of the independance of the $e_i$.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-02-27T07:06:29.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5394" PostTypeId="2" ParentId="5384" CreationDate="2013-02-27T07:47:19.610" Score="4" Body="&lt;p&gt;Based on your last comment (the fact that you might have multiple measurements and you don't care whether a certain set of measurements is above or below the fit), I think what you are looking for is a spline fit. You can do this using the scipy.interpolate B-spline routines. The script below generates three sets of data based on a function (that you can consider to model your system). For the first set, it adds some random error to the function &quot;above&quot; (red dots), for the second below (blue dots) and for the third just above and below (green dots). The black line is a plot of the exact function (without errors). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then you use the &lt;code&gt;scipy.interpolate.splrep&lt;/code&gt; function to generate a B-spline representation for this data. A B-spline is a piecewise combination of polynomial functions, typically cubic polynomials (cubic B-splines) which have some nice properties. If you want to know more about them, I would strongly suggest to read works by C. de Boor and P. Dierckx. The &lt;code&gt;scipy.interpolate.splrep&lt;/code&gt; function returns a tuple of the knots, the coefficients and the order. Using the &lt;code&gt;scipy.interpolate.splev&lt;/code&gt; function, you can then evaluate the B-spline representation in a number of points in the interval of the fit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script below plots an evaluation of the B-spline in magenta. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import scipy.interpolate as ip&#xA;&#xA;def f(x):&#xA;    return np.sin(np.pi*2*x)*np.exp(-2*x)&#xA;&#xA;N=20&#xA;err=0.15&#xA;&#xA;data = np.zeros([3*N,2])&#xA;&#xA;#red dots above curve&#xA;data[:N,0] = np.sort(np.random.rand(N))&#xA;data[:N,1] = f(data[:N,0])+err*np.random.rand(N)&#xA;&#xA;#blue dots below curve&#xA;data[N:2*N,0] = np.sort(np.random.rand(N))&#xA;data[N:2*N,1] = f(data[N:2*N,0])-err*np.random.rand(N)&#xA;&#xA;#green dots above and below curve&#xA;data[2*N:,0] = np.sort(np.random.rand(N))&#xA;data[2*N:,1] = f(data[2*N:,0])-2*err*(np.random.rand(N)-0.5)&#xA;&#xA;plt.plot(data[:N,0],data[:N,1],'ro')&#xA;plt.plot(data[N:2*N,0],data[N:2*N,1],'bo')&#xA;plt.plot(data[2*N:,0],data[2*N:,1],'go')&#xA;&#xA;data = data[data[:,0].argsort()]&#xA;x = data[:,0]&#xA;y = data[:,1]&#xA;y_exact = f(x)&#xA;&#xA;&#xA;plt.plot(x,y_exact,'k')&#xA;&#xA;w = np.ones([len(x),1])&#xA;spl = ip.splrep(x,y,w)&#xA;&#xA;xn = np.linspace(0,1.0,100)&#xA;sple = ip.splev(xn,spl)&#xA;&#xA;plt.plot(xn,sple,'m')&#xA;&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-02-27T07:47:19.610" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5395" PostTypeId="2" ParentId="4921" CreationDate="2013-02-27T07:54:40.803" Score="1" Body="&lt;p&gt;I think I get your point, which &lt;a href=&quot;https://scicomp.stackexchange.com/a/4922/3740&quot;&gt;this answer&lt;/a&gt; hasn't got.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you mean &lt;em&gt;literally&lt;/em&gt; something like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=1:0.1:10;&#xA;f=(pi^2)*sin(pi*x);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then &lt;code&gt;f&lt;/code&gt; is no longer a function. It is just a variable containing the &lt;strong&gt;result of the function applied to the x&lt;/strong&gt; that you already have. In other words, your function has already been evaluated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the answer to your question is that you don't need to do anything : &lt;code&gt;f&lt;/code&gt; simply does not know about &lt;code&gt;x&lt;/code&gt;, or other variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depends on what's the exact code in question.&lt;/p&gt;&#xA;" OwnerUserId="3740" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-02-27T07:54:40.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5396" PostTypeId="2" ParentId="5389" CreationDate="2013-02-27T11:39:55.103" Score="3" Body="&lt;p&gt;First, it's always good to further debug the issue and seeing exactly where (what term, for what parameters) the overflow is coming from.  This was not clear to me from the question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you know precisely what the problem is, you can better diagnose the issue.  For example, lets take the overflow issue.  This is somewhere above 1e300 if I remember correctly.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You should be asking yourself 'should I really be hitting a range where this variable is getting this high'? &lt;/li&gt;&#xA;&lt;li&gt;If the answer is 'yes, but it is divided out or canceled by another term', you need to rewrite the equations in a way that combines those terms.  That is, if you are calculating $(x +y)/z$ with $x,y,z$ huge numbers try defining $x'=x/z$ and $y'=y/z$ and integrate those variables.  The goals here should be to write your equations so that you are always be comparing terms of similar magnitudes.  &lt;/li&gt;&#xA;&lt;li&gt;If you are convinced you need very large numbers for the problem, try converting your equations to a form in which the fundamental variables are in logspace.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that you're explicitly asking the code to do an improper integral (-inf,inf), so you need to make sure the integrand you're putting into it is well behaved for numerics along that whole domain!  It may help to try truncating the integral at the smallest value you can, lets call it $E_{max}$, where you know physically that you don't expect any contribution to the integral from $E_{max}$ to inf.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="3799" LastActivityDate="2013-02-27T11:39:55.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5397" PostTypeId="2" ParentId="4695" CreationDate="2013-02-27T12:27:11.460" Score="3" Body="&lt;p&gt;Here is a different kind of answer, but in my opinion, quite interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took a look at all the users of this beta with more than 1000 reputation. I think that these people can be considered as doing computational science. For each of them, I looked at the profile and see if they have an account on StackOverflow, which can be seen as a sign of being interested in programming. Here are the results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the &lt;code&gt;25 users having more than 1000 reputation&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;4 &lt;code&gt;(16%)&lt;/code&gt; do not have an account.&lt;/li&gt;&#xA;&lt;li&gt;5 &lt;code&gt;(20%)&lt;/code&gt; just opened the account without gaining reputation.&lt;/li&gt;&#xA;&lt;li&gt;16 &lt;code&gt;(64%)&lt;/code&gt; contributed on StackOverflow.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, at least &lt;code&gt;84%&lt;/code&gt; of them are interested in programming. In my opinion, this shows that programming is an essential part of the job of scientific computing :-) That's just a confirmation of the other answers!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting fact (that I did not quantify) is that all these users have accounts on many topics, not just Computational Science and StackOverflow! I think that this just goes in the direction of the answer of Wolfgang Bangerth.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-02-27T12:27:11.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5398" PostTypeId="2" ParentId="5384" CreationDate="2013-02-27T15:09:47.980" Score="1" Body="&lt;p&gt;To reproduce your particular figure given the red and black labeling of the points, you could use a piece of a voronoi diagram.  In other words, your black line will be the set of points that is equidistant between the nearest red and the nearest black point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But assuming that you want to do something less contrived, you could use some kind of nonlinear svm instead.  For example as shown in figures &lt;a href=&quot;http://sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/ebooks/html/csa/node221.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3705" LastActivityDate="2013-02-27T15:09:47.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5399" PostTypeId="1" CreationDate="2013-02-27T23:58:01.237" Score="1" ViewCount="189" Body="&lt;p&gt;I have a 3D finite-difference formulation (for time-harmonic Maxwell using a standard Yee cell grid.) A &quot;perfect electrical conductor&quot; condition can be implemented by enforcing the desired electric field components to 0. (This can also be approximated by making the effective $\epsilon_r$ have a really large imaginary part but for a PEC it's easiest and most stable to just enforce the 0-field magnitude condition.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I define a single field component (say Ex of a single Yee cell) to be PEC, I am wondering what the effective dimensions\volume of the defined material would be for a grid-spacing of $dx$. Clearly it's not just a infinitesimally thin line of PEC, but at the same time it does not seem like it would be a full grid cell thick (i.e. be a block of PEC with volume $dx^3$) since the field components around it are not PEC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if the question is unclear at all and I'll try to clarify further. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1480" LastActivityDate="2013-02-28T14:57:08.057" Title="Effective &quot;thickness&quot; of finite-difference material elements" Tags="|finite-difference|electromagnetism|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5400" PostTypeId="2" ParentId="2630" CreationDate="2013-02-28T08:34:38.107" Score="3" Body="&lt;p&gt;You could connect to MATLAB using &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/5051&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your matrices aren't too large : hand coding the algorithms shouldn't result in too much time loss, maybe it'll run for 1 hour. It may or may not be too long depending on various factors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though, coding it yourself may not be easy at all. I don't think I can, and I've been dealing with this for the past few months. But the SLICOT algorithm itself is &lt;a href=&quot;http://www.icm.tu-bs.de/NICONET/doc/SB03MD.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3740" LastActivityDate="2013-02-28T08:34:38.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5402" PostTypeId="1" AcceptedAnswerId="5418" CreationDate="2013-02-28T12:30:30.987" Score="30" ViewCount="6004" Body="&lt;p&gt;I am not very familiar with the common discretization schemes for PDEs. I know that Crank-Nicolson is popular scheme for discretizing the diffusion equation. &lt;strong&gt;Is also a good choice for the advection term?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interesting in solving the &lt;a href=&quot;http://en.wikipedia.org/wiki/Reaction%E2%80%93diffusion%E2%80%93advection_equation&quot;&gt;Reaction-Diffusion-Advection&lt;/a&gt; equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} + \nabla \cdot \left( \boldsymbol{v} u - D\nabla u \right) = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ is the diffusion coefficient of substance $u$ and $\boldsymbol{v}$ is the velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my specific application the equation can be written in the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} = \underbrace{D\frac{\partial^2 u}{\partial x^2}}_{\textrm{Diffusion}} + \underbrace{\boldsymbol{v}\frac{\partial u}{\partial x}}_{\textrm{Advection (convection)}} + \underbrace{f(x,t)}_{\textrm{Reaction}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the Crank-Nicolson scheme I have applied,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{u_{j}^{n+1} - u_{j}^{n}}{\Delta t} = D \left[ \frac{1 - \beta}{(\Delta x)^2} \left( u_{j-1}^{n} - 2u_{j}^{n} + u_{j+1}^{n} \right) +  \frac{\beta}{(\Delta x)^2} \left( u_{j-1}^{n+1} - 2u_{j}^{n+1} + u_{j+1}^{n+1} \right) \right] + &#xA;\boldsymbol{v} \left[ \frac{1-\alpha}{2\Delta x} \left( u_{j+1}^{n} - u_{j-1}^{n} \right) + \frac{\alpha}{2\Delta x} \left( u_{j+1}^{n+1} - u_{j-1}^{n+1} \right) \right] + f(x,t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice the $\alpha$ and the $\beta$ terms. This enables scheme to move between:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\beta=\alpha=1/2$ Crank-Niscolson,&lt;/li&gt;&#xA;&lt;li&gt;$\beta=\alpha=1$ it is fully implicit&lt;/li&gt;&#xA;&lt;li&gt;$\beta=\alpha=0$ it is fully explicit&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The values can be different, which allows the diffusion term to be Crank-Nicolson and the advection term to be something else. What is the most stable approach, what would you recommend?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2018-07-08T02:22:13.987" Title="Is Crank-Nicolson a stable discretization scheme for Reaction-Diffusion-Advection (convection) equation?" Tags="|pde|discretization|advection|diffusion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5403" PostTypeId="2" ParentId="5399" CreationDate="2013-02-28T14:57:08.057" Score="1" Body="&lt;p&gt;I think a good way to visualize or extend the behavior of the Yee cell is to recognize that it's a special kind of mixed finite element method, where (i) space has been discretized with axis aligned hexahedra (ii) curl conforming functions (&quot;edge elements&quot;) have been used to model the electric field E (iii) divergence conforming elements (&quot;facet elements&quot;) have been been used to model the magnetic flux B and (iv) all the volume integrals arising from the FEM testing process have been approximated by lumping the integration (each cube has 8 equally weighted quadrature points, one at each node).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PEC boundary conditions get applied to the edges, allowing you to coarsely trace out wires and such by following the edges of the hexahedra (so IMO, your intuition that shorting a single Ex makes a thin PEC line is correct - FDTD won't model this especially accurately because it does not incorporate the field singularities of a thin wire without some modifications, but to first/zeroth order, a wire filament is the effect you'll get). To enforce a sheet of pec, you'd short out both transverse components on whatever collection of facets best approximates your surface (note this will automatically enforce the normal component of B to be zero on that sheet, too)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &quot;A finite element method based on whitney forms to solve Maxwell equations in the time domain&quot; by Man-Fai Wong, Odile Picon and Victor Fouad Hanna[1] has some nice discussion about the similarities between mixed finite elements and Yee's FDTD. You might also find it instructive to read about &quot;finite integration theory&quot;/FIT methods[2] for modeling Maxwell's equations, which are a similar generalization of the Yee method (I think the commercial software &quot;CST microwave studio&quot; is an FIT code.. at least the transient solver part)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=376343&quot; rel=&quot;nofollow&quot;&gt;http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=376343&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://www.jpier.org/PIER/pier32/03.00080103.clemens.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.jpier.org/PIER/pier32/03.00080103.clemens.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2013-02-28T14:57:08.057" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5404" PostTypeId="1" CreationDate="2013-02-28T22:53:49.710" Score="7" ViewCount="262" Body="&lt;p&gt;I have the following problem in $x \in \mathbb C^{205}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\displaystyle\min_{x}x^HAx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to the following constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^HBx = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^HC_ix = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $i \in \{0,1,\dots,203\}$, where $A$ and $B$ are complex $205 \times 205$ matrices and can be assumed to be positive definite. The $C_i$'s are rank-$1$ matrices (each $C_i$ matrix actually only has a single row which is non-zero, namely row $i$) but there are $204$ of them and they are not definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there is likely not a single best algorithm for dealing with this type of problem, but any suggestions for things to try out would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="20417" LastEditDate="2018-06-28T00:31:27.240" LastActivityDate="2018-06-28T00:31:27.240" Title="Algorithm for dealing with medium-size non-convex QCQP" Tags="|optimization|constrained-optimization|nonconvex|qcqp|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5405" PostTypeId="2" ParentId="5404" CreationDate="2013-03-01T00:07:42.363" Score="1" Body="&lt;p&gt;One can approximate the quadratic constraints as a set of linear constraints. Each of those constraints corresponds to the requirement that the solution $x$ lie on an ellipsoid. You can approximate the ellipsoid by a polyhedron (a set of planes). Since you are minimizing a quadratic function, you are looking for solutions which stay outside your unit ellipsoids. So now, you can instead solve several problems subject to a single linear halfspace constraint, and if you get a bounded solution, check it against the other halfspace constraints. This will give you an approximate solution which you can refine with a nonlinear optimization method. The downside is that this scales poorly with the number of facets in your ellipsoid approximation.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-03-01T00:07:42.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5406" PostTypeId="1" AcceptedAnswerId="5407" CreationDate="2013-03-01T02:25:35.940" Score="15" ViewCount="1369" Body="&lt;p&gt;Computational Science people:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;a href=&quot;https://math.stackexchange.com/q/317434/9281&quot;&gt;originally posted this question at Math Stack Exchange&lt;/a&gt; and someone commented that I might get &quot;much better&quot; answers here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a novice at numerical methods and Matlab.  I am attempting to evaluate the following sum of two triple integrals (it can obviously be written more simply, but you still cannot evaluate it symbolically (?)). I am having trouble getting the $\LaTeX$ to work here, so I reluctantly broke it up into pieces here:  I want to find the sum of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{2}{((1/0.3) - 1)^2}\left(\int_1^{1/0.3}\int_1^{r_1}\int_0^{r_1-r_0}F_1(r_0,r_1,t)\exp(-\frac{(0.3)^2 t^2}{4})\,dt\,dr_0\,dr_1 \right),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{2}{((1/0.3) - 1)^2}\left(\int_1^{1/0.3}\int_1^{r_1}\int_{r_1-r_0}^{r_1+r_0} F_2(r_0,r_1,t)\exp(-\frac{(0.3)^2 t^2}{4})\,dt\,dr_0\,dr_1 \right),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F_1(r_0,r_1,t)=\frac{t^2 r_0^3*(0.3)^3}{2r_1^3\sqrt{\pi}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F_2(r_0,r_1,t)=\frac{(0.3)^3\pi^{3/2}(r_0+r_1-t)^4 (t^2+2t(r_0+r_1)-3(r_1-r_0)^2)^2}{288(\frac{4}{3}\pi r_0^3)(\frac{4}{3}\pi r_1^3)}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT (March 2 2013): Someone  responded that they got Mathematica to do the integrals symbolically.  I just attempted to do this (with simplified versions of the integrals) and Mathematica could only do the outer two of the first one, and stalled on the second one.  I would appreciate some help.  Here is  what I did.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attempted to evaluate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_1^2 \int_1^{r_2} \int_0^{r_2-r_1} \frac{r_1^3 t^2 \exp(-t^2)}{r_2^3}\,dt\,dr_1\,dr_2$$&#xA;via &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integrate[r1^3/r2^3*t^2*Exp (-t^2), {t, 0, r2 - r1}, {r1, 1, r2}, {r2, 1, 2}]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and Mathematica returns (I had trouble with the $\LaTeX$ here because the result is long.  I broke it into two equations. if anyone knows a good way to display this please tell me):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_1^2 \frac{1}{64r2^2} e^{-1-r2^2}(2e^{2r2}(25+r2(19+2r2(1+r2)))-$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$e^{1+r2^2}(32r2(2+r2^2)) +\sqrt{\pi}(11+4r2^2(9+r2^2))\operatorname{Erf}[1-r2])\,dr2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I tried to evaluate &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_1^2\int_1^{r_2}\int_{r_2-r_1}^{r_2+r_1} \ldots \qquad \qquad \qquad $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ldots\frac{\exp(-t^2)(r_1+r_2-t)^4(t^2+2t(r_1+r_2)-3(r_2-r_1)^2)^2}{r_1^3 r_2^3}\,dt\,dr_\,dr_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integrate[(r1 + r2 - t)^4*(t^2 + 2*t*(r1 + r2) - 3*(r2 - r1)^2)^2*&#xA; Exp[-t^2]/r1^3/r2^3, {r2, 1, 2}, {r1, 1, r2}, {t, r2-r1, r2 + r1}]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;just now, and Mathematica has not returned an answer after about half an hour (but I am having computer network problems right now, and they may be to blame).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[END OF MARCH 2 EDIT]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used Matlab's &quot;triplequad&quot; command, with no extra options.  I handled the variable limits of integration by means of heaviside functions, because I didn't know any other way to do it.  Matlab gave me $0.007164820144202$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know Matlab is good software, but I have heard that numerical triple integrals are hard to do accurately, and mathematicians are supposed to be skeptical, so I want some way to verify the accuracy of this answer.  The integrals give the expected value of a certain experiment (if anyone wants, I can edit this question to describe the experiment):  I implemented the experiment in Matlab using appropriately randomly generated numbers, a million times, and averaged the results.  I repeated this process four times.  Here are the results (I apologize if I have used the word &quot;trial&quot; improperly):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trial 1: $0.007133292603256$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trial 2: $0.007120455071989$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trial 3: $0.007062595022049$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trial 4: $0.007154940168452$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trial 5: $0.007215000289130$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although each trial used a million samples, the simulation values only agree in the first significant digit.  They are not close enough to each to each other for me to determine whether the numerical triple integral is accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So can anyone tell me whether I can trust the result of &quot;triplequad&quot; here, and under what circumstances one can trust it in general?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One suggestion I got at Math Stack Exchange was to try other software like Mathematica, Octave, Maple, and SciPy.  Is this good advice?  Do people actually do numerical work in Mathematica and Maple?  Octave is kind of a Matlab clone, so can I assume it uses the same integration algorithms?  I haven't even heard of SciPy before and would appreciate any opinions about it.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;UPDATE: &lt;a href=&quot;https://math.stackexchange.com/q/317434/9281&quot;&gt;Someone from Math Stack Exchange&lt;/a&gt; did it in Maple and got $0.007163085468$.  That is agreement to three significant figures.  That is a good sign.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I would appreciate suggestions on how to enter long, multi-line expression in $\LaTeX$ in Stack Exchange.  Can you use the &quot;aligned&quot; environment here?  I tried, and I couldn't get it to work.&lt;/p&gt;&#xA;" OwnerUserId="3821" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-03-02T20:42:05.133" Title="can I trust this numerical triple integral from Matlab?" Tags="|matlab|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5407" PostTypeId="2" ParentId="5406" CreationDate="2013-03-01T08:07:04.833" Score="9" Body="&lt;p&gt;First of all, it is not the software (or at least it shouldn't be) that determines the quality of the solution to a problem, it's the quality and appropriateness of the algorithm that is applied. You should check what algorithm is being used by triplequad in Matlab (I would guess it uses a nested adaptive Gaussian quadrature). And you should check what the requested tolerances are (required absolute and relative tolerance). Chances are that, by default, it only asks for $10^{-8}$ relative precision.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer coming from Maple is probably done by Computer Algebra and maybe it could find a closed solution which was then evaluated using double precision floating point. This has the advantage that you are not approximating the integral by a finite summation (and hence are introducing approximation errors) but the Computer Algebra System will find an expression for the integral which can then be evaluated. Of course, care must be taken when evaluating this expression (for round-off).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would want to do this with SciPy, you would also need to resort to nested adaptive Gaussian quadrature using the underlying Quadpack (Piessens et al.) routines. In Octave, you'll have the same approach. And I wouldn't be too surprised if Matlab also uses Quadpack as quadrature engine (since it is &lt;em&gt;the&lt;/em&gt; reference). &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-03-01T08:07:04.833" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5408" PostTypeId="1" AcceptedAnswerId="5437" CreationDate="2013-03-01T16:19:23.240" Score="10" ViewCount="869" Body="&lt;p&gt;I see very few non-floating point computing libraries/packages around. Given the various inaccuracies of floating point representation, the question arises why there aren't at least &lt;em&gt;some&lt;/em&gt; fields where this increased accuracy might be worth the intricacies of working with fixed-point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any MAJOR difficulties in using, say, a fixed point eigenvalue solver? How slow/fast, inaccurate/accurate would they be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related : &lt;a href=&quot;https://scicomp.stackexchange.com/questions/527/is-there-software-that-can-autogenerate-numerically-accurate-floating-point-c-ro&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://softwareengineering.stackexchange.com/a/101197/22493&quot;&gt;this&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3740" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-03-05T06:16:40.440" Title="Relevance of fixed-point and arbitrary precision computations" Tags="|floating-point|numerics|" AnswerCount="4" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="5409" PostTypeId="2" ParentId="5402" CreationDate="2013-03-01T19:52:20.663" Score="10" Body="&lt;p&gt;Generally speaking, you'll want to use an implicit method for parabolic equations (the diffusion part) -- explicit schemes for parabolic PDE need to have a very short timestep to be stable. Conversely, for the hyperbolic part (advection) you'll want an explicit method as it's cheaper and doesn't disrupt the symmetry of the linear system you have to solve by using an implicit scheme for diffusion. In that case, you want to avoid centered differences like $(u_{j+1}-u_{j-1})/2\Delta t$ and switch to one-sided differences $(u_j-u_{j-1})/\Delta t$ for reasons of stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest you look at &lt;a href=&quot;http://books.google.com/books?id=lZysESfSWwgC&amp;amp;lpg=PR2&amp;amp;dq=leveque%20finite%20difference%20methods&amp;amp;pg=PR2#v=onepage&amp;amp;q=leveque%20finite%20difference%20methods&amp;amp;f=false&quot;&gt;Randy Leveque's book&lt;/a&gt; or &lt;a href=&quot;http://books.google.com/books?isbn=1441964126&quot;&gt;Dale Durran's book&lt;/a&gt; for &quot;von Neumann stability analysis&quot;. It's a general approach to ascertaining the stability of your discretization scheme, provided you have periodic boundary conditions. (There's also a good wiki article &lt;a href=&quot;http://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is to assume that your discrete approximation can be written a sum of plane waves $e^{i(k j\Delta x-\omega n\Delta t)}$, where $k$ is the wave number and $\omega$ the frequency. You cram a plane wave into your approximation to the PDE and pray it doesn't blow up. We can rewrite the plane wave as $\xi^n e^{ikj\Delta x}$ and we want to make sure that $|\xi|\le 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By way of illustration, consider the ordinary diffusion equation with fully implicit differencing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{u^{n+1}_j-u^n_j}{\Delta t} = D\frac{u^{n+1}_{j-1}-2u^{n+1}_j+u^{n+1}_{j+1}}{\Delta x^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we substitute in a plane wave, then divide by $\xi^n$ and $e^{ikj\Delta x}$, we get the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\xi-1}{\Delta t} = D\frac{e^{-ik\Delta x}-2+e^{ik\Delta x}}{\Delta x^2}\xi $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clean this up a little bit now and we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\xi = \frac{1}{1+\frac{2D\Delta t}{\Delta x^2}\left(1-\cos k\Delta x\right)}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is always less than one, so you're in the clear. Try applying this for the explicit, centered scheme for the advection equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{u_j^{n+1}-u_j^n}{\Delta t} = v\frac{u_{j-1}^n-u_{j+1}^n}{2\Delta x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and see what $\xi$ you get. (It'll have an imaginary part this time.) You'll find that $|\xi|^2 &amp;gt; 1$, which is sad times. Hence my admonition that you don't use it. If you can do that, then you shouldn't have much trouble finding a stable scheme for the full advection-diffusion equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I'd use a fully implicit scheme for the diffusion part. Change the differencing in the advection part to $u_j-u_{j-1}$ if $v &amp;gt; 0$ and $u_j-u_{j+1}$ if $v &amp;lt; 0$ and pick a timestep so that $V\Delta t/\Delta x \le 1$. (This is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Courant-Friedrichs-Lewy_condition&quot;&gt;Courant-Friedrichs-Lewy condition&lt;/a&gt;.) It's only first-order accurate, so you may want to look up higher order discretization schemes if that concerns you.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-03-01T19:52:20.663" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5411" PostTypeId="1" CreationDate="2013-03-02T08:19:33.107" Score="3" ViewCount="516" Body="&lt;p&gt;please can you help me with my problem with Stokes flow written using Marker and cell method (MAC)? I need only to solve the eq. of continuity + momentum eq. for a given condition (steady state). I don't understand how to do the things on boundaries (lets say I want to have v=0 on boundaries), how to fill the matrix - I will solve it as Ax=b where A is matrix 3*N*Nx3*N*N and b right hand side vector of dim 3*N*N if my box is NxN and I have 3 unknowns (vx,vy,p) and three eq. (  eq. of continuity + 2 momentum eq.)&#xA;Or is there any code I can have a look at to get a flavor of it?&#xA;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="3688" LastActivityDate="2013-07-20T19:53:11.060" Title="Marker and Cell Method (MAC) - STOKES FLOW - boundaries?" Tags="|boundary-conditions|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5412" PostTypeId="1" CreationDate="2013-03-02T09:12:06.853" Score="2" ViewCount="124" Body="&lt;p&gt;Given a square grid of size $n\times n$ and $m$ symbols (say for example, alphabets A, B, C...), having $N(i)$ number of $i$th symbol; $\sum_{i=1}^{m}N(i) = n\times n$. Is there any computationally tractable algorithm that can be used to distribute the symbols uniformly inside the grid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Uniform distribution here could be &quot;shortest-distance-pairs have similar values&quot;. An example with 3 symbols A,B,C with equal counts of 3 in a $3\times3$ grid, I would expect a distribution of something like $ \left[ \begin{array}{ccc}&#xA;A &amp;amp; C &amp;amp; B \\&#xA;C &amp;amp; B &amp;amp; A \\&#xA;B &amp;amp; A &amp;amp; C \end{array} \right]$. There could be many possible solutions, but the algorithm is expected to give one of them. And one more thing I am looking for is, the algorithm place symbol(s) with count value of 1 at (or near) the center of the grid, rather than at the corners.&lt;/p&gt;&#xA;" OwnerUserId="3826" LastActivityDate="2013-03-02T09:12:06.853" Title="Algorithm to distribute symbols uniformly in a square grid?" Tags="|optimization|algorithms|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5413" PostTypeId="2" ParentId="1681" CreationDate="2013-03-02T15:41:45.977" Score="3" Body="&lt;p&gt;In your original post, you say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;I have also tried using the ARPACK interface in igraph R package. However, it gave an error for one of my matrices.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be interested to know more about the error. If you can make this matrix publicly available somewhere, I'd be interested in trying ARPACK on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on what I've read above, I would expect ARPACK would do a very good job of extracting the largest (or a few of the largest) eigenvalues of a sparse matrix. To be more specific, I would expect Arnoldi methods to work well for this case and that, of course, is what ARPACK is based  on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The slow convergence of the power method when there are closely-spaced eigenvalues in the region of interest was mentioned above. Arnoldi improves this by iterating with several vectors instead of the one in power method.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-03-02T15:41:45.977" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5414" PostTypeId="1" CreationDate="2013-03-02T20:50:32.967" Score="7" ViewCount="206" Body="&lt;p&gt;I solve the Lyapunov equations :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A W_C E^T + E W_C A^T + B B^T = 0 $$&#xA;$$ A^T W_O E^T + E W_O A + C^T C = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to obtain $ W_C $ and $W_O$. My aim is to get the left and right eigenspaces of $W_C W_O$. This is basically for model order reduction through balanced truncation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am explicitly generating $W_C$ and $W_O$, and calculating the &lt;code&gt;schur&lt;/code&gt; decomposition as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;W_c = lyap(A,B*B',[],E);&#xA;W_o = lyap(A',C'*C,[],E);&#xA;W_j = W_c*W_o;&#xA;[Vs,Ts] = schur(W_j);&#xA;Vst = Vs';&#xA;V_lk = Vst(:,1:ordr);&#xA;V_rk = Vs(:,1:ordr);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I optimize the process of calculating the dominant eigenspaces? Links to some theory behind any techniques will be very helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional Information : I am aware of ADI, and read quite a few of the papers based on it. However, there are 2 problems with it being applicable to my case. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;My original system is dense, a product of another model reduction. Isn't ADI specific to Sparse systems? &lt;/li&gt;&#xA;&lt;li&gt;My grammians will NOT be positive definite, because its not fully controllable/observable. They are in-fact indefinite (negative and positive) eigenvalues. Therefore Cholesky factor is not applicable. Is there a different factorization I could use with ADI?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3740" LastEditorUserId="276" LastEditDate="2016-06-01T21:33:41.687" LastActivityDate="2016-06-01T21:33:41.687" Title="Left and right eigenspaces of the product of Gramians" Tags="|linear-algebra|matlab|eigenvalues|matrix-equations|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="5415" PostTypeId="2" ParentId="5302" CreationDate="2013-03-02T22:57:36.277" Score="1" Body="&lt;p&gt;I'm the lead developer of LibGeoDecomp. While I agree with vanCompute that you could emulate your ACA with a CA, you're right that this would not be very efficient, as only few cells in any given step are meant to be updated. This is indeed a very interesting application -- and fun to tinker with!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest you to combine the solutions proposed by jlopez1967 and Pedro: Pedro's algorithm captures the parallelism well, but those atomic locks are awfully slow. The solution of jlopez1967 is elegant when it comes to detecting collisions, but checking all &lt;code&gt;n&lt;/code&gt; cells, when only a smaller subset (I'll from now on assume that there is some parameter &lt;code&gt;k&lt;/code&gt; which denotes the number of cells to be updated simultaneously) are active, is clearly prohibitive.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__global__ void markPoints(Cell *grid, int gridWidth, int *posX, int *posY)&#xA;{&#xA;    int id = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    int x, y;&#xA;    generateRandomCoord(&amp;amp;x, &amp;amp;y);&#xA;    posX[id] = x;&#xA;    posY[id] = y;&#xA;    grid[y * gridWidth + x].flag = 1;&#xA;}&#xA;&#xA;__global__ void checkPoints(Cell *grid, int gridWidth, int *posX, int *posY, bool *active)&#xA;{&#xA;    int id = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    int x = posX[id];&#xA;    int y = posY[id];&#xA;    int markedNeighbors = &#xA;        grid[(y - 1) * gridWidth + x + 0].flag +&#xA;        grid[(y - 1) * gridWidth + x + 1].flag +&#xA;        grid[(y + 0) * gridWidth + x - 1].flag +&#xA;        grid[(y + 0) * gridWidth + x + 1].flag +&#xA;        grid[(y + 1) * gridWidth + x + 0].flag +&#xA;        grid[(y + 1) * gridWidth + x + 1].flag;&#xA;    active[id] = (markedNeighbors &amp;gt; 0);&#xA;}&#xA;&#xA;&#xA;__global__ void update(Cell *grid, int gridWidth, int *posX, int *posY, bool *active)&#xA;{&#xA;    int id = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    int x = posX[id];&#xA;    int y = posY[id];&#xA;    grid[y * gridWidth + x].flag = 0;&#xA;    if (active[id]) {&#xA;        // do your fancy stuff here&#xA;    }&#xA;}&#xA;&#xA;int main() &#xA;{&#xA;  // alloc grid here, update up to k cells simultaneously&#xA;  int n = 1024 * 1024;&#xA;  int k = 1234;&#xA;  for (;;) {&#xA;      markPoints&amp;lt;&amp;lt;&amp;lt;gridDim,blockDim&amp;gt;&amp;gt;&amp;gt;(grid, gridWidth, posX, posY);&#xA;      checkPoints&amp;lt;&amp;lt;&amp;lt;gridDim,blockDim&amp;gt;&amp;gt;&amp;gt;(grid, gridWidth, posX, posY, active);&#xA;      update&amp;lt;&amp;lt;&amp;lt;gridDim,blockDim&amp;gt;&amp;gt;&amp;gt;(grid, gridWidth, posX, posY, active);&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the absence of good global synchronization on the GPU, you need to invoke multiple kernels for the different phases. On Nvidia's Kepler you can move even the main loop to the GPU, but I don't expect that to gain much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithms achieves a (configurable) degree of parallelism. I guess, the interesting question is whether the collisions will affect your random distribution when you increase &lt;code&gt;k&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3829" LastActivityDate="2013-03-02T22:57:36.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5416" PostTypeId="2" ParentId="5408" CreationDate="2013-03-03T03:15:16.657" Score="3" Body="&lt;p&gt;If you look at this library for correct rounding: &lt;a href=&quot;http://lipforge.ens-lyon.fr/www/crlibm/&quot; rel=&quot;nofollow&quot;&gt;CRlibm&lt;/a&gt;, you will see in the documentation that generally, algorithms must be proven accurate (with reasoned proofs).  Why?  The stability and speed of convergence of a result of a function does not have a &quot;one-size-fits-all&quot; answer.  In short, there is &quot;no free lunch&quot; -- you have to work to prove your reasoning is correct.  This is due to the behavior of the functions being modeled, not the underlying hardware (whether you use integer or floating point units, though yes, both have &quot;gotchas&quot;, like overflow/underflow, denormal numbers, etc.)  Even if the result you are looking for converges to an integer, the algorithm used to find the result is not necessarily very stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eigen is a C++ library that has a variety of algorithms for solving matrices, each with different properties.  &lt;a href=&quot;http://eigen.tuxfamily.org/dox-devel/TutorialLinearAlgebra.html&quot; rel=&quot;nofollow&quot;&gt;This page&lt;/a&gt; contains a table which discusses speed vs. accuracy trade-offs for the various algorithms used for solving a matrix.  I suspect the Eigen library can do what you want. :-)&lt;/p&gt;&#xA;" OwnerUserId="3830" LastEditorUserId="3830" LastEditDate="2013-03-03T05:35:06.670" LastActivityDate="2013-03-03T05:35:06.670" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5417" PostTypeId="2" ParentId="527" CreationDate="2013-03-03T05:56:19.803" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://sagemath.org/&quot; rel=&quot;nofollow&quot;&gt;Sage&lt;/a&gt; lets you express formulas in Cython (a variant of python that generates C code); however, in answer to your more general question: no.  Consider &lt;a href=&quot;http://en.wikipedia.org/wiki/Rice%27s_theorem#Proof_sketch&quot; rel=&quot;nofollow&quot;&gt;Rice's Theorem&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3830" LastActivityDate="2013-03-03T05:56:19.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5418" PostTypeId="2" ParentId="5402" CreationDate="2013-03-03T17:48:14.197" Score="15" Body="&lt;p&gt;This is a well-framed question and a very useful thing to understand.  Korrok is correct to refer you to von Neumann analysis and LeVeque's book.  I can add a bit more to that.  I'd like to write a detailed answer, but at the moment I only have time for a short one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $\alpha=\beta=1/2$, you get a method that is absolutely stable for arbitrarily large step sizes, as well as second-order accurate.  However, the method is not &lt;em&gt;L&lt;/em&gt;-stable, so very high frequencies will not be damped, which is unphysical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $\alpha=\beta=1$, you get a method that is also unconditionally stable, but only 1st-order accurate.  This method is very dissipative.  It is &lt;em&gt;L&lt;/em&gt;-stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you take $\alpha\ne\beta$, your method can be understood as applying an &lt;strong&gt;additive Runge-Kutta method&lt;/strong&gt; to the centered-difference semi-discretization.  The stability and accuracy analysis for such methods is considerably more complicated.  A very nice paper on such methods is &lt;a href=&quot;http://www.plouffe.fr/simon/articles/NASA-2001-tm211038.pdf&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which approach to recommend depends strongly on the magnitude of $D$, the kind of initial data you deal with, and the accuracy you seek.  If very low accuracy is acceptable, then $\alpha=\beta=1$ is a very robust approach.  If $D$ is moderate or large, then the problem is diffusion-dominated and very stiff; typically $\alpha=\beta=1/2$ will give good results.  If $D$ is very small, then it may be advantageous to use an explicit method and higher-order upwinding for the convective terms.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-03-03T17:48:14.197" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5419" PostTypeId="2" ParentId="5408" CreationDate="2013-03-03T17:54:03.517" Score="2" Body="&lt;p&gt;For some nice examples of where high-precision arithmetic has been useful in mathematics, take a look at the book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1568814429&quot; rel=&quot;nofollow&quot;&gt;Mathematics by Experiment&lt;/a&gt; by Jonathan Borwein and David Bailey.  There's also &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/156881271X&quot; rel=&quot;nofollow&quot;&gt;this sequel&lt;/a&gt;, which I haven't read.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-03-03T17:54:03.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5420" PostTypeId="1" CreationDate="2013-03-03T18:27:24.513" Score="11" ViewCount="196" Body="&lt;p&gt;Say I have a function $f : \mathbf{R}^3 \to \mathbf{R}$ that I wish to integrate over a tetrahedron $T \subset \mathbf{R}^3$.  If $f$ was arbitrary, Gauss quadrature would be a good solution, but I happen to know that $f$ is harmonic.  How much can Gauss quadrature be accelerated using this information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if $T$ was instead a sphere, evaluating $f$ once at the center of the sphere gives the exact answer by the mean value property.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A search turned up the following paper, which is interesting but generalizes the sphere case in a different direction (to polyharmonic instead of away from spheres):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.105.4125&quot;&gt;Bojanov and Dimitrov, Gaussian extended cubature formulae for polyharmonic functions&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-03-04T15:53:52.803" Title="Integrating a harmonic function over a tetrahedron" Tags="|quadrature|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5421" PostTypeId="1" AcceptedAnswerId="5422" CreationDate="2013-03-03T18:39:40.033" Score="6" ViewCount="365" Body="&lt;p&gt;I know that this question might be very specific and maybe nobody will know the answer, but this is probably the only community where I could find an answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as part of my master's project, I am currently writing a multigrid code for Poisson's equation in order to calculate particle interactions in a molecular dynamcis setup. I'd like to use periodic BCs, but here arises one problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the Multigrid book by Trottenberg, periodic BCs only have a solution if $\sum_{x, y, z} f_h(x,y,z) = 0$, where $f_h$ is the right hand side and the sum is over all gridpoints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm I found in some publications samples the charged particles via a density function to the grid. So in general, even if the total charge of the system is neutral, this condition will not be fulfilled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, all papers I found only deal with this PBC problem on a quick sidenote and I wasn't able to get any valuable information out of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code I wrote so far produced fine results when using different functions which fulfill this condition, but fails when going to the real data, i.e., sampled particle charge densities. My question now: Does anybody know this problem and the solution, or any idea which could help me?&lt;/p&gt;&#xA;" OwnerUserId="3833" LastActivityDate="2013-03-03T20:06:41.640" Title="Periodic BC for Multigrid in MD" Tags="|multigrid|molecular-dynamics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5422" PostTypeId="2" ParentId="5421" CreationDate="2013-03-03T19:39:38.797" Score="6" Body="&lt;p&gt;This isn't a multigrid issue, it's a problem formulation issue. Consider the symmetric system $A x = b$ and suppose that $A e = 0$ for some nonzero vector $e$ (the constant when $A$ is the Laplacian with periodic or Neumann boundary conditions). This system is singular and has no solution if you choose $b$ such that $e^T b \ne 0$. Indeed, the exact periodic problem is ill-posed because the integral of the Coulomb potential with some periodic change density (not averaging to zero) over all space is infinite. That is, if $\sigma(r')$ is periodic with nonzero integral, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Phi (r) = \int_{r'\in R^3} \frac{\sigma(r')}{4 \pi \vert r - r'\rvert} \mathrm{d} r' = \pm \infty .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately, all that matters is the field, not the potential, so you can renormalize. This amounts to projecting out the null space by solving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A x = (I - e e^T) b $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e$ is the normalized null space vector.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-03-03T20:06:41.640" LastActivityDate="2013-03-03T20:06:41.640" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5423" PostTypeId="1" AcceptedAnswerId="5424" CreationDate="2013-03-04T03:40:24.763" Score="13" ViewCount="2289" Body="&lt;p&gt;Question: What methods are available to accurately and efficiently calculate the sparsity structure of a finite element matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Info: I'm working on a Poisson Pressure Equation solver, using Galerkin's method with a quadratic Lagrange basis, written in C, and using PETSc for sparse matrix storage and KSP routines.  To use PETSc efficiently, I need to pre-allocate memory for the global stiffness matrix.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am doing a mock assembly to estimate the number of nonzeros per row&#xA;as follows(pseudocode)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int nnz[global_dim]&#xA;for E=1 to NUM_ELTS&#xA;  for i=1 to 6&#xA;    gi = global index of i &#xA;    if node gi is free&#xA;      for j=1 to 6&#xA;        gj = global index of j&#xA;        if node gj is free &#xA;          nnz[i]++&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This, however overestimates nnz because some node-node interactions can occur in multiple elements.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've considered trying to keep track of which i,j interactions I have found, but I am unsure how to do this without using a lot of memory.&#xA;I could also loop over the nodes, and find the support of the basis function centered at that node, but then I'd have to search through all elements for each node, which seems inefficient.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5202/efficient-assembly-of-finite-element-matrix-in-matlab&quot;&gt;this&lt;/a&gt; recent question, which contained some useful information, especially from Stefano M, who wrote&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;my advice is to implement it in python or C, applying some graph theoretical concepts, i.e. consider elements in the matrix as edges in a graph and compute the sparsity structure of the adjacency matrix. List of lists or dictionary of keys are common choices.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm looking for more details and resources on this.  I admittedly don't know much graph theory, and I'm not familiar with all the CS tricks that might be useful(I'm approaching this from the mathematical side).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3836" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-04-03T14:17:50.293" Title="Calculation of the sparsity structure for finite element matrices" Tags="|matrix|finite-element|petsc|performance|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5424" PostTypeId="2" ParentId="5423" CreationDate="2013-03-04T05:24:55.480" Score="5" Body="&lt;p&gt;Your idea of keeping track of which i,j interactions you have found can work, I think that's the &quot;CS trick&quot; that you and Stefano M are referring to. This amounts to constructing your sparse matrix in &lt;a href=&quot;http://en.wikipedia.org/wiki/Sparse_matrix#List_of_lists_.28LIL.29&quot; rel=&quot;noreferrer&quot;&gt;list of lists format&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not sure how much CS you have so I apologize if this is already known to you: in a &lt;a href=&quot;http://en.wikipedia.org/wiki/Linked_list&quot; rel=&quot;noreferrer&quot;&gt;linked list&lt;/a&gt; data structure, every entry stores a pointer to the entry after it and the entry before. It's cheap to add and delete entries from, but not as simple to find items in it -- you might have to look through all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for each node i, you store a linked list. Then you iterate through all the elements; if you find two nodes i and j are connected, you go look in i's linked list. If j isn't already there, you add it to the list, and likewise add i to j's list. It's easiest if you add them in order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you've populated your list of lists, you now know the number of non-zero entries in each row of the matrix: it's the length of that node's list. This information is exactly what you need to preallocate a sparse matrix in PETSc's matrix data structure. Then you can free your list of lists because you don't need it anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this approach assumes that all you have is the list of which nodes each element contains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some mesh generation packages -- &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;noreferrer&quot;&gt;Triangle&lt;/a&gt; for example -- can output not just a list of elements and which nodes they contain, but also a list of every edge in your triangulation. In that case, you run no risk of overestimating the number of non-zero entries: for piecewise linear elements, each edge gives you exactly 2 stiffness matrix entries. You're using piecewise quadratic, so each edge counts for 4 entries, but you get the idea. In that case, you can find the number of non-zero entries per row with one pass through the edge list using an ordinary array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that approach, you have to read through an extra big file from the hard disk, which could actually be slower than using the element list if your actual computation isn't that big. Nonetheless, I think it's simpler.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-03-04T05:24:55.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5425" PostTypeId="1" AcceptedAnswerId="5428" CreationDate="2013-03-04T12:52:14.127" Score="34" ViewCount="6096" Body="&lt;p&gt;I am trying to solving the advection equation but have a strange oscillation appearing in the solution when the wave reflects from the boundaries. If anybody has seen this artefact before I would be interested to know the cause and how to avoid it!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This is an animated gif, open in separate window to view the animation (it will only play once or not at once it has been cached!)&lt;/strong&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Gkw2W.gif&quot; alt=&quot;Propagation of a Gaussian pulse&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the propagation seems highly stable until the wave begins to reflect from the first boundary. What do you think could be happening here? I have spend a few days double checking my code and cannot find any errors. It is strange because there seems to be two propagating solutions: one positive and one negative; after the reflection from the first boundary. The solutions seems to be travelling along adjacent mesh points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation details follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advection equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} = \boldsymbol{v}\frac{\partial u}{\partial x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\boldsymbol{v}$ is the propagation velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Crank-Nicolson is an &lt;a href=&quot;http://how.gi.alaska.edu/ao/sim/chapters/chap9.pdf&quot; rel=&quot;noreferrer&quot;&gt;unconditionally (pdf link)&lt;/a&gt; &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5402/is-crank-nicolson-a-stable-discretization-scheme-for-reaction-diffusion-advectio&quot;&gt;stable&lt;/a&gt; discretization for the advection equation provided $u(x)$ is slowly varying in space (only contains low frequencies components when Fourier transformed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discretization I have applied is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$    \frac{\phi_{j}^{n+1} - \phi_{j}^{n}}{\Delta t} =&#xA;    \boldsymbol{v} \left[ \frac{1-\beta}{2\Delta x} \left( \phi_{j+1}^{n} - \phi_{j-1}^{n} \right) + \frac{\beta}{2\Delta x} \left( \phi_{j+1}^{n+1} - \phi_{j-1}^{n+1} \right) \right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting the unknowns on the right-hand side enables this to be written in the linear form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\beta r\phi_{j-1}^{n+1} + \phi_{j}^{n+1} -\beta r\phi_{j+1}^{n+1} =  -(1-\beta)r\phi_{j-1}^{n} + \phi_{j}^{n} + (1-\beta)r\phi_{j+1}^{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\beta=0.5$ (to take the time average evenly weighted between the present and future point) and $r=\boldsymbol{v}\frac{\Delta t}{2\Delta x}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These set of equation have the matrix form $A\cdot u^{n+1} = M\cdot u^n$, where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;  \boldsymbol{A} = &#xA;    \left( &#xA;    \begin{matrix}&#xA;      1                 &amp;amp; -\beta r        	&amp;amp;     				&amp;amp;       	&amp;amp;          0		\\&#xA;      \beta r    		&amp;amp; 1         		&amp;amp; -\beta r       	&amp;amp;     	    &amp;amp;		\\&#xA;              			&amp;amp;  \ddots   		&amp;amp;   \ddots  		&amp;amp; \ddots	&amp;amp;            		\\&#xA;						&amp;amp;                   &amp;amp; \beta r  		    &amp;amp; 1         &amp;amp; -\beta r 	        \\&#xA;                 0      &amp;amp;                   &amp;amp;      				&amp;amp; \beta r  	&amp;amp; 1          		\\&#xA;   \end{matrix}&#xA;   \right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;  \boldsymbol{M} = &#xA;    \left( &#xA;    \begin{matrix}&#xA;      1                    &amp;amp; (1 - \beta)r        		&amp;amp;     						&amp;amp;       	          &amp;amp;   0		 	        \\&#xA;      -(1 - \beta)r        &amp;amp; 1 			                &amp;amp; (1 - \beta)r 		        &amp;amp;		     	      &amp;amp;                     \\&#xA;              			   &amp;amp;  \ddots   					&amp;amp;   \ddots  				&amp;amp; \ddots			  &amp;amp;                     \\&#xA;						   &amp;amp;                            &amp;amp; -(1 - \beta)r  		    &amp;amp; 1	                  &amp;amp; (1 - \beta)r	\\&#xA;                 0         &amp;amp;                            &amp;amp;      						&amp;amp;-(1 - \beta)r        &amp;amp; 1	             \\&#xA;   \end{matrix}&#xA;   \right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vectors $u^n$ and $u^{n+1}$ are the known and unknown of the quantity we want to solve for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then apply &lt;strong&gt;closed&lt;/strong&gt; Neumann boundary conditions on the left and right boundaries. By closed boundaries I mean $\frac{\partial u}{\partial x} = 0$ on both interfaces. &lt;strike&gt;For closed boundaries it turns out that (I won't show my working here) we just need to solve the above matrix equation.&lt;/strike&gt; As pointed out by @DavidKetcheson, the above matrix equations actually describe &lt;strong&gt;Dirichlet boundary conditions&lt;/strong&gt;. For Neumann boundary conditions,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;  \boldsymbol{A} = &#xA;    \left( &#xA;    \begin{matrix}&#xA;      1                 &amp;amp; 0        	&amp;amp;     				&amp;amp;       	&amp;amp;          0		\\&#xA;      \beta r    		&amp;amp; 1         		&amp;amp; -\beta r       	&amp;amp;     	    &amp;amp;		\\&#xA;              			&amp;amp;  \ddots   		&amp;amp;   \ddots  		&amp;amp; \ddots	&amp;amp;            		\\&#xA;						&amp;amp;                   &amp;amp; \beta r  		    &amp;amp; 1         &amp;amp; -\beta r 	        \\&#xA;                 0      &amp;amp;                   &amp;amp;      				&amp;amp; 0  	&amp;amp; 1          		\\&#xA;   \end{matrix}&#xA;   \right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The behaviour seems fairly independent of the choice of constants I use, but these are the values for the plot you see above:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\boldsymbol{v}$=2&lt;/li&gt;&#xA;&lt;li&gt;dx=0.2&lt;/li&gt;&#xA;&lt;li&gt;dt=0.005&lt;/li&gt;&#xA;&lt;li&gt;$\sigma$=2 (Gaussian hwhm)&lt;/li&gt;&#xA;&lt;li&gt;$\beta$=0.5&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update II&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simulation with non-zero diffusion coefficient, $D=1$ (see comments below), the oscillation goes away, but the wave no longer reflects!? I don't understand why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/UgGsd.gif&quot; alt=&quot;Diffusion and advection&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2018-10-16T22:47:47.413" Title="Strange oscillation when solving the advection equation by finite-difference with fully closed Neumann boundary conditions (reflection at boundaries)" Tags="|pde|finite-difference|advection|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="5426" PostTypeId="1" CreationDate="2013-03-04T13:57:41.203" Score="1" ViewCount="250" Body="&lt;p&gt;Consider the following robust optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;min c'x&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t.: $Ax\geq b    \;\;\forall (A,b)\in \mathcal{U}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why can the robust counterpart of the problem be written in this form?&#xA;$min_x{\{ max_{(A,b)} Ax\geq b\}}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does it work principally? Does it mean that I have to, for a fixed $x$ value, maximize the constraints $Ax\geq b$ for all realization of the uncertain data, and then minimize the value $x$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please make this point clear to me with a small example.&lt;/p&gt;&#xA;" OwnerUserId="917" LastActivityDate="2013-06-21T11:44:37.090" Title="Robust Counterpart of an uncertain LP" Tags="|optimization|linear-programming|convex-optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5427" PostTypeId="2" ParentId="5420" CreationDate="2013-03-04T15:53:52.803" Score="1" Body="&lt;p&gt;I found something that might be interesting.&#xA;&lt;a href=&quot;http://www.math.kth.se/~gbjorn/exact.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.kth.se/~gbjorn/exact.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps, &#xA;Tom&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2013-03-04T15:53:52.803" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5428" PostTypeId="2" ParentId="5425" CreationDate="2013-03-04T17:33:51.877" Score="30" Body="&lt;p&gt;The equation you're solving does not permit right-going solutions, so there is no such thing as a &lt;em&gt;reflecting boundary condition&lt;/em&gt; for this equation.  If you consider the characteristics, you'll realize that you can only impose a boundary condition at the right boundary.  You are trying to impose a homogeneous Dirichlet boundary condition at the left boundary, which is mathematically invalid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To reiterate: the method of characteristics says that the solution must be constant along any line of the form &lt;span class=&quot;math-container&quot;&gt;$x-\nu t = C$&lt;/span&gt; for any constant &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;.  Thus the solution along the left boundary is determined by the solution at earlier times inside your problem domain; you cannot impose a solution there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike the equation, your numerical scheme &lt;em&gt;does&lt;/em&gt; admit right-going solutions.  The right-going modes are referred to as parasitic modes and involve very high frequencies.  Notice that the right-going wave is a sawtooth wave packet, associated with the highest frequencies that can be represented on your grid.  That wave is purely a numerical artifact, created by your discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For emphasis: &lt;strong&gt;you have not written down the full initial-boundary value problem that you are trying to solve.  If you do, it will be clear that it is not a mathematically well-posed problem.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm glad you posted this here, though, as it's a beautiful illustration of what can happen when you discretize a problem that's not well-posed, and of the phenomenon of parasitic modes.  A big +1 for your question from me.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2018-10-15T15:16:12.880" LastActivityDate="2018-10-15T15:16:12.880" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5430" PostTypeId="1" AcceptedAnswerId="5438" CreationDate="2013-03-04T22:39:15.953" Score="6" ViewCount="337" Body="&lt;p&gt;There is a lot of info about how to use the sparsity pattern of A in order to solve $Ax = b$. However I can't find much about using the sparsity pattern of b. Let me take a concrete example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us assume A is a large $N \times N$ matrix ($N &amp;gt; 10^6$) with a known sparsity pattern. Now let us assume I want to solve repeatedly linear systems of the form $Ax = b$ in which $b$ is a column vector with only one non-zero entry. (this seems to be a similar problem as computing some given columns of the matrix inverse $A^{-1}$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I can either through direct or iterative solvers compute the solution of the linear system and at the same time take advantage of the known sparsity structure of A (which can be precomputed if needed), however what about $b$ ? Doing this feels like I am going to spend as much computing time as solving $Ax=c$ where c is a full vector, but here only one element of b is non-zero, is there a way to take advantage of that specifically ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I can think of a way if using a direct LU decomposition: if $b = e_N$ , the unit column vector with the last element equal to 1, for example (and we can always apply a permutation on A to make it that way I believe), then the forward substitution $L(Ux) = e_N$ consists indeed of only one operation. I assume that there might be other &quot;tricks&quot; like this out there that I am not aware of, hopefully included in some packages (I personnally like using petsc), so any link towards a reference on that topic would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3704" LastActivityDate="2013-03-08T08:25:46.807" Title="Take advantage of the sparsity of b in AX=b" Tags="|linear-solver|sparse-matrix|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5433" PostTypeId="1" AcceptedAnswerId="5445" CreationDate="2013-03-05T00:23:02.617" Score="1" ViewCount="614" Body="&lt;p&gt;CS Beta people:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been doing some multiple integrals with a combination of symbolic and numerical integration (because symbolic answers have not always been possible).  I have been using Mathematica to do the symbolic integration, cut-and-pasting the results into Matlab, and then using quad or dblquad in Matlab.  I could probably do the whole thing in Mathematica, but I would like to develop my Matlab skills.  My question is, is there a convenient way to use Matlab's symbolic capabilities to do this more efficiently, without cut-and-pasting?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only symbolic engine I have access to in Matlab is mupad (I don't have the Maple Toolbox, for example).  I would consult the Mathworks documentation, but I have found it deficient on several other questions, and I would like to hear from someone who has actually done it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT (March 5): Someone asked me to be more specific.  Here is part of what I'm doing.  I want to evaluate the triple integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_1^{RmaxOverRmin} \int_1^{r_2}\int_0^{r_2-r_1}\frac{r_1^3 t^2}{r_2^3}\exp(-\frac{t^2}{4 dOverRmin^2})\,dt\,dr_1\,dr_2.$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica can't do it symbolically, so it is probably impossible.  Mathematica can do the inner double integral symbolically.  So I did it, cut-and-pasted the results into a Matlab function file, and used Matlab's &quot;quad&quot; to integrate the result as $r_2$ went from $1$ to $RmaxOverRmin$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stefan (STack Exchange FAN)&lt;/p&gt;&#xA;" OwnerUserId="3821" LastEditorUserId="3821" LastEditDate="2013-03-05T16:24:48.883" LastActivityDate="2013-03-05T23:01:25.033" Title="How easy is it to combine symbolic and numeric computation in Matlab?" Tags="|matlab|symbolic-computation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5434" PostTypeId="1" AcceptedAnswerId="5440" CreationDate="2013-03-05T03:21:13.613" Score="29" ViewCount="6852" Body="&lt;p&gt;I don't understand the different behaviour of the advection-diffusion equation when I apply different boundary conditions. My motivation is the simulation of a real physical quantity (particle density) under diffusion and advection. Particle density should be conserved in the interior unless it flows out from the edges. By this logic, if I enforce Neumann boundary conditions the ends of the system such as $\frac{\partial \phi}{\partial x}=0$ (on the left and the right sides) then the system should be &lt;em&gt;&quot;closed&quot;&lt;/em&gt; i.e. if the &lt;strong&gt;flux&lt;/strong&gt; at the boundary is zero then no particles can escape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For all the simulations below, I have applied the Crank-Nicolson discretization to the advection-diffusion equation and all simulation have $\frac{\partial \phi}{\partial x}=0$ boundary conditions. However, for the first and last rows of the matrix (the boundary condition rows) I allow $\beta$ to be changed independently of the interior value. This allows the end points to be fully implicit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below I discuss 4 different configurations, only one of them is what I expected. At the end I discuss my implementation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Diffusion only limit&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Here the advection terms are turned off by setting the velocity to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Diffusion only, with $\boldsymbol{\beta}$=0.5 (Crank-Niscolson) at all points&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XGWYQ.gif&quot; alt=&quot;Diffusion only (Neumann boundaries with beta=0.5)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quantity is not conserved as can be seen by the pulse area reducing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Diffusion only, with $\boldsymbol{\beta}$=0.5 (Crank-Niscolson) at interior points, and $\boldsymbol{\beta}$=1 (full implicit) at the boundaries&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/un9MJ.gif&quot; alt=&quot;Diffusion only (Neumann boundaries with beta=0.5 for interior, beta=1 fully implicit) the boundaries&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By using fully implicit equation on the boundaries I achieve what I expect: &lt;strong&gt;no particles escape&lt;/strong&gt;. You can see this by the area being conserved as the particle diffuse. &lt;strong&gt;Why&lt;/strong&gt; should the choice of $\beta$ at the boundary points influence the physics of the situation? Is this a bug or expected?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Diffusion and advection&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When the advection term is included, the value of $\beta$ at the boundaries does not seem to influence the solution. However, for all cases when the boundaries seem to be &lt;em&gt;&quot;open&quot;&lt;/em&gt; i.e. particles can escape the boundaries. Why is this the case? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Advection and Diffusion with $\boldsymbol{\beta}$=0.5 (Crank-Niscolson) at all points&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/1Ck7s.gif&quot; alt=&quot;Advection-Diffusion (Neumann boundaries with beta=0.5)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Advection and Diffusion with $\boldsymbol{\beta}$=0.5 (Crank-Niscolson) at interior points, and $\boldsymbol{\beta}$=1 (full implicit) at the boundaries&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/e1WhI.gif&quot; alt=&quot;Advection and Diffusion (Neumann boundaries with beta=0.5 for interior, beta=1 fully implicit) the boundaries&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Implementation of the advection-diffusion equation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Starting with the advection-diffusion equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial \phi}{\partial t} = D\frac{\partial^2 \phi}{\partial x^2} + \boldsymbol{v}\frac{\partial \phi}{\partial x}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Writing using Crank-Nicolson gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\phi_{j}^{n+1} - \phi_{j}^{n}}{\Delta t} = D \left[ \frac{1 - \beta}{(\Delta x)^2} \left( \phi_{j-1}^{n} - 2\phi_{j}^{n} + \phi_{j+1}^{n} \right) +  \frac{\beta}{(\Delta x)^2} \left( \phi_{j-1}^{n+1} - 2\phi_{j}^{n+1} + \phi_{j+1}^{n+1} \right) \right] + \boldsymbol{v} \left[ \frac{1-\beta}{2\Delta x} \left( \phi_{j+1}^{n} - \phi_{j-1}^{n} \right) + \frac{\beta}{2\Delta x} \left( \phi_{j+1}^{n+1} - \phi_{j-1}^{n+1} \right) \right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $\beta$=0.5 for Crank-Nicolson, $\beta$=1 for fully implicit, and, $\beta$=0 for fully explicit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To simplify the notation let's make the substitution,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;s = D\frac{\Delta t}{(\Delta x)^2} \\&#xA;r = \boldsymbol{v}\frac{\Delta t}{2 \Delta x}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and move the known value $\phi_{j}^{n}$ of the time derivative to the right-hand side,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\phi_{j}^{n+1} = \phi_{j}^{n} + s \left( 1-\beta \right) \left( \phi_{j-1}^{n} - 2\phi_{j}^{n} + \phi_{j+1}^{n} \right) + s \beta \left( \phi_{j-1}^{n+1} - 2\phi_{j}^{n+1} + \phi_{j+1}^{n+1} \right) + &#xA;    r \left( 1 - \beta \right) \left( \phi_{j+1}^{n} - \phi_{j-1}^{n} \right) + r \beta \left( \phi_{j+1}^{n+1} - \phi_{j-1}^{n+1} \right) &#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Factoring the $\phi$ terms gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\underbrace{\beta(r - s)\phi_{j-1}^{n+1} + (1 + 2s\beta)\phi_{j}^{n+1} -\beta(s + r)\phi_{j+1}^{n+1}}_{\boldsymbol{A}\cdot\boldsymbol{\phi^{n+1}}} =  \underbrace{ (1-\beta)(s - r)\phi_{j-1}^{n} + (1-2s[1-\beta])\phi_{j}^{n} + (1-\beta)(s+r)\phi_{j+1}^{n}}_{\boldsymbol{M\cdot}\boldsymbol{\phi^n}}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which we can write in matrix form as $\boldsymbol{A}\cdot\boldsymbol{\phi^{n+1}} = \boldsymbol{M}\cdot\boldsymbol{\phi^{n}}$ where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$  \boldsymbol{A} = &#xA;    \left( &#xA;    \begin{matrix}&#xA;      1+2s\beta         &amp;amp; -\beta(s + r)  	&amp;amp;     				&amp;amp;       	0		\\&#xA;      \beta(r-s)  		&amp;amp; 1+2s\beta 		&amp;amp; -\beta (s + r) 	&amp;amp;     	    		\\&#xA;              			&amp;amp;  \ddots   		&amp;amp;   \ddots  		&amp;amp; \ddots			\\&#xA;						&amp;amp; \beta(r-s)  		&amp;amp; 1+2s\beta			&amp;amp; -\beta (s + r) 	\\&#xA;                 0      &amp;amp;      				&amp;amp; \beta(r-s)  		&amp;amp; 1+2s\beta  		\\&#xA;   \end{matrix}&#xA;   \right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;  \boldsymbol{M} = &#xA;    \left( &#xA;    \begin{matrix}&#xA;      1-2s(1-\beta)        &amp;amp; (1 - \beta)(s + r)  		&amp;amp;     						&amp;amp;       	0		 	\\&#xA;      (1 - \beta)(s - r)   &amp;amp; 1-2s(1-\beta)  			&amp;amp; (1 - \beta)(s + r) 		&amp;amp;		     	    	\\&#xA;              			   &amp;amp;  \ddots   					&amp;amp;   \ddots  				&amp;amp; \ddots				\\&#xA;						   &amp;amp; (1 - \beta)(s - r)  		&amp;amp; 1-2s(1-\beta)				&amp;amp; (1 - \beta)(s + r)	\\&#xA;                 0         &amp;amp;      						&amp;amp; (1 - \beta)(s - r) 		&amp;amp; 1-2s(1-\beta)			\\&#xA;   \end{matrix}&#xA;   \right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Applying Neumann boundary conditions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NB is working through the derivation again I think I have spotted the error. I assumed a fully implicit scheme ($\beta$=1) when writing the finite difference of the boundary condition. If you assume a Crank-Niscolson scheme here the complexity become too great and I could not solve the resulting equations to eliminate the nodes which are outside the domain. However, it would appear possible, there are two equation with two unknowns, but I couldn't manage it. This probably explains the difference between the first and second plots above. I think we can conclude that only the plots with $\beta$=0.5 at the boundary points are valid.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the flux at the left-hand side is known (assuming a fully implicit form),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial\phi_1^{n+1}}{\partial x} = \sigma_L&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Writing this as a centred-difference gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial\phi_1^{n+1}}{\partial x} \approx \frac{\phi_2^{n+1} - \phi_0^{n+1}}{2\Delta x} = \sigma_L&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;therefore,&#xA;$&#xA;\phi_0^{n+1} = \phi_{2}^{n+1} - 2 \Delta x\sigma_L&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this introduces a node $\phi_0^{n+1}$ which is outside the domain of the problem. This node can be eliminated by using a second equation. We can write the $j=1$ node as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\beta(r - s)\phi_0^{n+1} + (1+2s\beta)\phi_1^{n+1} - \beta(s+r)\phi_2^{n+1} = (1-\beta)(s - r)\phi_{j-1}^{n} + (1-2s[1-\beta])\phi_{j}^{n} + (1-\beta)(s+r)\phi_{j+1}^{n}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting in the value of $\phi_0^{n+1}$ found from the boundary condition gives the following result for the $j$=1 row,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;(1+2s\beta)\phi_1^{n+1} - 2s\beta\phi_2^{n+1} =  (1-\beta)(s - r)\phi_{j-1}^{n} + (1-2s[1-\beta])\phi_{j}^{n} + (1-\beta)(s+r)\phi_{j+1}^{n} + 2\beta(r-s)\Delta x\sigma_L&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performing the same procedure for the final row (at $j$=$J$) yields,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;-2s\beta\phi_{J-1}^{n+1} + (1+2s\beta)\phi_J^{n+1} = (1-\beta)(s - r)\phi_{J-1}^{n} + (1 - 2s(1-\beta))\phi_{J}^{n} + 2\beta(s+r)\Delta x\sigma_R&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally making the boundary rows implicit (setting $\beta$=1) gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;(1+2s)\phi_1^{n+1} - 2s\phi_2^{n+1} =  \phi_{j-1}^{n} + 1\phi_{j}^{n} + 2(r-s)\Delta x\sigma_L&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;-2s\phi_{J-1}^{n+1} + (1+2s)\phi_J^{n+1} = \phi_{J}^{n} + 2(s+r)\Delta x\sigma_R&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore with Neumann boundary conditions we can write the matrix equation, $\boldsymbol{A}\cdot\phi^{n+1} = \boldsymbol{M}\cdot\phi^{n} + \boldsymbol{b_N}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;  \boldsymbol{A} = &#xA;    \left( &#xA;    \begin{matrix}&#xA;      1+2s	            &amp;amp; -2s			  	&amp;amp;     				&amp;amp;       	0		\\&#xA;      \beta(r-s)  		&amp;amp; 1+2s\beta 		&amp;amp; -\beta (s + r) 	&amp;amp;     	    		\\&#xA;              			&amp;amp;  \ddots   		&amp;amp;   \ddots  		&amp;amp; \ddots			\\&#xA;						&amp;amp; \beta(r-s)  		&amp;amp; 1+2s\beta			&amp;amp; -\beta (s + r) 	\\&#xA;                 0      &amp;amp;      				&amp;amp; -2s		  		&amp;amp; 1+2s		  		\\&#xA;   \end{matrix}&#xA;   \right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;  \boldsymbol{M} = &#xA;    \left( &#xA;    \begin{matrix}&#xA;      1			           &amp;amp; 0					  		&amp;amp;     						&amp;amp;       	0		 	\\&#xA;      (1 - \beta)(s - r)   &amp;amp; 1-2s(1-\beta)  			&amp;amp; (1 - \beta)(s + r) 		&amp;amp;		     	    	\\&#xA;              			   &amp;amp;  \ddots   					&amp;amp;   \ddots  				&amp;amp; \ddots				\\&#xA;						   &amp;amp; (1 - \beta)(s - r)  		&amp;amp; 1-2s(1-\beta)				&amp;amp; (1 - \beta)(s + r)	\\&#xA;                 0         &amp;amp;      						&amp;amp; 0					 		&amp;amp; 1						\\&#xA;   \end{matrix}&#xA;   \right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;   \boldsymbol{b_N} = \left( &#xA;        \begin{matrix}&#xA;        2 (r - s) \Delta x \sigma_L &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0 &amp;amp; 2 (s + r) \Delta x \sigma_R&#xA;        \end{matrix}&#xA;   \right)^{T}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;My current understanding&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I think the difference between the first and second plots is explained by noting the error outlined above.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;s&gt;Regarding the conservation of the physical quantity. I believe the cause is that, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5425/strange-oscillation-when-solving-the-advection-equation-by-finite-difference-wit&quot;&gt;as pointed out here&lt;/a&gt;, the advection equation in the form I have written it doesn't allow propagation in the reverse direction so the wave just passes through &lt;strong&gt;even with zero-flux boundary conditions&lt;/strong&gt;. My initial intuition regarding conservation only applied when advection term is zero (this is solution in plot number 2 where the area is conserved).&lt;/s&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Even with &lt;strong&gt;Neumann zero-flux&lt;/strong&gt; boundary conditions $\frac{\partial \phi}{\partial x} = 0$ the mass can still leave the system, this is because the correct boundary conditions in this case are &lt;strong&gt;Robin&lt;/strong&gt; boundary conditions in which the &lt;em&gt;total&lt;/em&gt; flux is specified $j = D\frac{\partial \phi}{\partial x} + \boldsymbol{v}\phi = 0$. Moreover the Neunmann condition specifies that mass cannot leave the domain via &lt;strong&gt;diffusion&lt;/strong&gt;, it says nothing about advection. In essence what we have hear are closed boundary conditions to diffusion and open boundary conditions to advection. For more information see the answer here, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10407/implementation-of-gradient-zero-boundary-conditon-in-advection-diffusion-equatio/10576#10576&quot;&gt;Implementation of gradient zero boundary conditon in advection-diffusion equation&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;s&gt;Would you agree?&lt;/s&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://danieljfarrell.github.io/FVM/advection_diffusion_boundary_conditions.html#robin-boundary-conditions-known-flux&quot; rel=&quot;noreferrer&quot;&gt;A tutorial on how to implement Robin boundary conditions.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-06-01T23:32:56.880" Title="Conservation of a physical quantity when using Neumann boundary conditions applied to the advection-diffusion equation" Tags="|pde|boundary-conditions|advection|diffusion|crank-nicolson|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5435" PostTypeId="2" ParentId="5408" CreationDate="2013-03-05T03:36:18.963" Score="4" Body="&lt;p&gt;As an example of why exact arithmetic/fixed point arithmetic is so rarely used, consider this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In the finite element method, as in almost every other method in use in scientific computing, we arrive at linear or nonlinear systems that are only approximations to the real world. For example, in the FEM, the linear system to solve is only an approximation to the original partial differential equation (which may, itself, only be an approximation of the real world). So why put enormous effort into solving something that is only an approximation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Most of the algorithms we use today are iterative in nature: Newton's method, Conjugate Gradients, etc. We terminate these iterations whenever we are satisfied that the accuracy of the iterative approximate to the solution of the problem is sufficient. In other words, we terminate before we have the exact solution. As before, why use exact arithmetic for an iterative scheme when we know that we're only computing approximations?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-05T03:36:18.963" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5437" PostTypeId="2" ParentId="5408" CreationDate="2013-03-05T06:16:40.440" Score="5" Body="&lt;p&gt;The use of fixed point arithmetic can be appropriate under certain circumstances. Generally for scientific computing (at least in the sense that most people think of it) it is not appropriate due to the need for expressing the large dynamic ranges encountered. You mention eigenvalue problems as an example, but very often in science, one is interested in the smallest eigenvalues of a matrix (say, in computing the ground state of a quantum system). The accuracy of small eigenvalues will generally be quite deteriorated relative to large eigenvalues if you use fixed point. If your matrix contains entries which vary by large ratios, the small eigenvalues might be completely unexpressable in the working precision. This is a problem with the representation of numbers; these arguments hold regardless of how you do the intermediate computations. You could possibly work out a scaling to apply to the computed results, but now you've just invented floating point. It is easy to construct matrices whose elements are well behaved, but whose eigenvalues are exceedingly poorly behaved (like &lt;a href=&quot;http://en.wikipedia.org/wiki/Wilkinson_matrix&quot;&gt;Wilkinson matrices&lt;/a&gt;, or even &lt;a href=&quot;http://www.ti3.tu-harburg.de/paper/rump/NiRuOi11.pdf&quot;&gt;matrices with entirely integer entries&lt;/a&gt;). These examples are not as pathological as they might seem, and many problems at the cutting edge of science involve very poorly behaved matrices, so using fixed point in this context is a Bad Idea(TM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might argue that you know the magnitude of the results and you want to not waste bits on the exponent, so let's talk about the intermediates. Using fixed point will generally exacerbate the effects of catastrophic cancellations and roundoff unless you really go through great pains to work in higher precision. The performance penalty would be huge, and I would conjecture that using a floating point representation with the same mantissa bit width would be faster and more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One area where fixed point can shine is in certain areas of geometric computing. Especially if you need exact arithmetic or know the dynamic range of all the numbers beforehand, fixed point lets you take advantage of all of the bits in your representation. For example, suppose you wanted to compute the intersection of two lines, and somehow the endpoints of the two lines are normalized to sit in the unit square. In this case, the intersection point can be represented with more bits of precision than using an equivalent floating point number (which will waste bits on the exponent). Now, it is almost certainly the case that the intermediate numbers required in this calculation need to be computed to higher precision, or at least done very carefully (&lt;a href=&quot;http://msdn.microsoft.com/en-us/library/windows/desktop/aa383718%28v=vs.85%29.aspx&quot;&gt;like when dividing the product of two numbers by another number, you need to be very careful about it&lt;/a&gt;). In this respect, fixed point is advantageous more from the representation standpoint rather than from a computational standpoint, and I would go so far as to say this is generally true when you can establish definite upper &lt;em&gt;and&lt;/em&gt; lower bounds on the dynamic range of your algorithm outputs. This happens rarely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used to think that floating point representations were crude or inaccurate (why waste bits on an exponent?!). But over time I've come to realize that it really is one of the best possible representations for real numbers. Things in nature show up on log scales, so real data ends up spanning a large range of exponents. Also to achieve the highest possible relative accuracy requires working on log scales, making the tracking of an exponent more natural. The only other contender for a &quot;natural&quot; representation is &lt;a href=&quot;http://en.wikipedia.org/wiki/Symmetric_level-index_arithmetic&quot;&gt;symmetric level index&lt;/a&gt;. However, addition and subtraction are much slower in that representation, and it lacks the hardware support of IEEE 754. &lt;a href=&quot;http://www.cs.berkeley.edu/~wkahan/&quot;&gt;A tremendous amount of thought was put into the floating point standards&lt;/a&gt;, by a pillar of numerical linear algebra. I would think he knows what the &quot;right&quot; representation of numbers is.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-03-05T06:16:40.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5438" PostTypeId="2" ParentId="5430" CreationDate="2013-03-05T06:28:15.073" Score="5" Body="&lt;p&gt;First let's rule out iterative methods like Krylov subspace methods. If you form products $Ax$, $A^2x$, etc. and they have a predictable sparsity pattern, then your matrix $A$ has some reducible property to it (it has some block that is decoupled from the rest, or something like that). Generally, these products will not maintain sparsity as the iteration progresses, and computing residuals like $b-Ax$ will destroy the sparsity of intermediates. No luck here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For direct methods involving a matrix factorization, you are presumably already taking advantage of sparsity in the $O(N^3)$ factorization phase. As you mentioned, the sparsity pattern of $b$ can lead to an obvious reduction in the cost of the back-substitution (it is $O(kN)$ where $k$ is the number of nonzeros in $b$. This is already optimal, so it doesn't look like you can do better here either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as you pointed out, you are essentially just interested in a few selected columns of $A^{-1}$. The only algorithms I know of for doing this, like SelInv, require a matrix factorization, so it's no better than the direct method.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-03-05T06:28:15.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5439" PostTypeId="2" ParentId="5423" CreationDate="2013-03-05T08:32:40.693" Score="2" Body="&lt;p&gt;Upvoted&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally dont know of any cheap way of doing this so I simply overestimate the number i.e., use a reasonably large value for all rows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g., for a perfectly structured mesh made of linear 8 node hex elements the nnzs per row in both the diagonal and off diagonal blocks are dof*27. For most fully unstructured automatically generated hex meshes the number rarely exceeds dof*54. For linear tets I have never had the need to go beyond dof*30. For some meshes with very badly shaped/low aspect ratio elements you may have to use slightly larger values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The penalty is that local (on rank) memory consumption is between 2x-5x so you may have to use more compute nodes on your cluster than usual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw I did try using searchable lists but the time taken to determine the sparsity structure was more than the assembly/solve. But my implementation was very simple and did not use information about edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other option is to use routines like DMMeshCreateExodus as shown in &lt;a href=&quot;https://bitbucket.org/petsc/petsc-dev/src/2a552fd584bf855b9dc42efec9e8ab778063a84f/src/dm/impls/mesh/examples/tutorials/ex4f90.F?at=default&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; example.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2013-03-05T09:51:32.750" LastActivityDate="2013-03-05T09:51:32.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5440" PostTypeId="2" ParentId="5434" CreationDate="2013-03-05T11:47:51.013" Score="16" Body="&lt;p&gt;I think that one of your problems is that (as you observed in your comments) &lt;strong&gt;Neumann conditions are not the conditions you are looking for&lt;/strong&gt;, in the sense that they do not imply the conservation of your quantity. To find the correct condition, rewrite your PDE as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial \phi}{\partial t} = \frac{\partial}{\partial x}\left( D\frac{\partial \phi}{\partial x} + v \phi \right) + S(x,t) .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the term that appears in parentheses, $ D\frac{\partial \phi}{\partial x} + v \phi = 0 $ is the &lt;strong&gt;total flux&lt;/strong&gt; and this is the quantity that you must put to zero on the boundaries to conserve $\phi$. (I have added $S(x,t)$ for the sake of generality and for your comments.) The boundary conditions that you have to impose are then (supposing your space domain is $(-10,10)$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D\frac{\partial \phi}{\partial x}(-10) + v \phi(-10) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the left side and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D\frac{\partial \phi}{\partial x}(10) + v \phi(10) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the right side. These are the so-called &lt;a href=&quot;http://en.wikipedia.org/wiki/Robin_boundary_condition&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Robin boundary condition&lt;/strong&gt;&lt;/a&gt; (note that Wikipedia explicitly says these are the insulating conditions for advection-diffusion equations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you set up these boundary conditions, you get the conservation properties that you were looking for. Indeed, integrating over the space domain, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int \frac{\partial \phi}{\partial t} dx = \int \frac{\partial}{\partial x} \left( D \frac{\partial \phi}{\partial x} + v \phi \right) dx + \int S(x,t) dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;a href=&quot;http://en.wikipedia.org/wiki/Integration_by_parts#Higher_dimensions&quot; rel=&quot;noreferrer&quot;&gt;integration by parts&lt;/a&gt; on the right hand side, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int \frac{\partial \phi}{\partial t} dx = \left( D \frac{\partial \phi}{\partial x} + v \phi \right)(10) - \left( D \frac{\partial \phi}{\partial x} + v \phi \right)(-10) + \int S(x,t) dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the two central terms vanish thanks to the boundary conditions. Integrating in time, we obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_0^T \int \frac{\partial \phi}{\partial t} dx dt = \int_0^T \int S(x,t) dx dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;a href=&quot;http://en.wikipedia.org/wiki/Fubini%27s_theorem&quot; rel=&quot;noreferrer&quot;&gt;if we are allowed to switch the first two integrals&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int \phi(x,T) dx - \int \phi(x,0) dx = \int_0^T \int S(x,t) dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shows that the domain is insulated from the exterior. In particular, if $S=0$, we get the conservation of $\phi$.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="21638" LastEditDate="2016-09-09T12:20:09.173" LastActivityDate="2016-09-09T12:20:09.173" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="5441" PostTypeId="2" ParentId="5430" CreationDate="2013-03-05T13:35:08.263" Score="6" Body="&lt;p&gt;It is a fundamental intuition that the inverse of any &quot;interesting&quot; matrix is dense. This means that even if $b$ is sparse, $x = A^{-1} b$ will be dense. Thus in the best case (with a direct solver), you could get a speedup of 2x by using sparsity in the solve, but factorization is no faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With an iterative solver, the preconditioner either does its job to spread information globally as fast as possible (destroying sparsity immediately) or the iteration will converge very slowly with the vector still becoming dense before convergence. Even without considering implementation performance, this would provide very little benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only need a few entries of the inverse, or perhaps the diagonal, algorithms like &lt;a href=&quot;http://dl.acm.org/citation.cfm?doid=1916461.1916464&quot;&gt;SelInv&lt;/a&gt; intelligently carry around the factors from a multifrontal solve instead of computing an explicit dense representation of the inverse. (This idea is old, but it has only recently been restated in common language.) There are also probing methods to estimate properties like the trace of the inverse.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-03-05T13:35:08.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5443" PostTypeId="1" CreationDate="2013-03-05T21:27:34.310" Score="10" ViewCount="434" Body="&lt;p&gt;This might be a silly question, but I know very little about the theoretical properties finite elements, so here goes. Suppose you were to solve the Helmholtz equation (let's say in 2D) with a spatially varying wave speed using finite elements. For reference the equation is&#xA;$$ [\nabla^2 + k^2(\vec{r})]\psi(\vec{r}) = f(\vec{r}) $$&#xA;This being a scattering problem, the source term is limited to, say, Dirichlet values on the boundary of the domain. My question is, when the domain is large (say ten or more of wavelength (I consider this large, it might not be)), how well does energy conservation hold? More precisely, what kind of guarantees can you place on conservation of energy as a function of domain size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume for now linear elements (in practice, we use finite differences, is there a huge difference?) since $k(\vec{r})$ is piecewise constant, so higher orders aren't terribly helpful. I hope I have not revealed too much of my ignorance on this subject!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: I am referring to energy in the &quot;intuitive&quot; sense. I work mostly with Maxwell's equations, which in 2D at constant frequency becomes the Helmholtz equation. I believe the mathematical definition of energy flow at a point is $\Psi\nabla \Psi$, or something to that effect (it should be the Poynting vector). Energy conservation would say that in a source-free region, the integral of the energy in the surface normal direction along the boundary of the region should be identically zero. Intuitively, I should see waves obeying the inverse square law (or whatever its analog is in 2D) from a point source exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I realize there is a difference between finite differences and finite elements. If you could comment on both, that would be even better.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2013-03-06T23:08:30.430" LastActivityDate="2023-05-27T09:55:01.283" Title="Energy conservation in the solution of the Helmholtz equation" Tags="|finite-element|helmholtz-equation|wave-propagation|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5445" PostTypeId="2" ParentId="5433" CreationDate="2013-03-05T23:01:25.033" Score="1" Body="&lt;p&gt;The easiest way would probably be to use &lt;a href=&quot;http://library.wolfram.com/infocenter/Demos/4821/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mEngine&lt;/a&gt;. I have not worked with it, but I am pretty sure it will be a breeze to install and use. Not sure exactly how the expressions will be transferred though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise you could just write the expressions into text files. Then you can run &lt;code&gt;f=sym(somestring);&lt;/code&gt; to have MATLAB parse it into a symbolic object. Then just substitute using &lt;code&gt;subs()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit : Check &lt;a href=&quot;https://stackoverflow.com/questions/4510770/how-to-invoke-a-matlab-function-from-mathematica&quot;&gt;this&lt;/a&gt; answer.&lt;/p&gt;&#xA;" OwnerUserId="3740" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-03-05T23:01:25.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5446" PostTypeId="1" CreationDate="2013-03-05T23:09:52.523" Score="12" ViewCount="2480" Body="&lt;p&gt;I am simulating incompressible flow over a cylinder at Reynold number of 500. I am solving navier stokes equation using pressure correction method. My solution becomes unstable after certain time (approximately 5s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried refining my mesh, stepsize (0.05) (making sure my CFL &amp;lt; 1, even though I am using implicit methods)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My boundary conditions, mesh and unstable results are shown in the attached figures. The domain is about 25 times larger than the cylinder diameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried simulating this problem O grid (which became unstable almost immediately).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following link contains the pictures of the boundary conditions and results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/6pVCw.png&quot; alt=&quot;Boundary Conditions&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/MlKtN.jpg&quot; alt=&quot;Instability&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be grateful if anyone can share their thoughts/experiences on this problem. Many thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;editted:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apologies for the typing mistake:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the following boundary conditions:&#xA;Neumann boundary&#xA;$$\frac{\partial \vec{u}} {\partial n} - \vec{n} p  = 0;  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on Dirichlet Boundary&#xA;$$ \vec{u} = u_x = 1  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;editted:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i have applied velocity boundary conditions on the nodes around the dirichlet boundary. Also, top-right and bottom-right corner node is dirichlet boundary with velocity 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After, I looked more deeply into the simulation results, I notice that instability starts to creep in at inflow/outflow junction.&lt;/p&gt;&#xA;" OwnerDisplayName="illusion" LastEditorUserId="3691" LastEditDate="2014-05-15T00:09:38.610" LastActivityDate="2014-09-12T03:01:41.177" Title="Dirichlet-Neumann boundary condition solution becomes unstable - Pressure Correction Method" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5447" PostTypeId="1" AcceptedAnswerId="5613" CreationDate="2013-03-06T01:58:04.543" Score="1" ViewCount="741" Body="&lt;p&gt;My question is not related to any particular problem, rather, I am looking at the equations of the form $$u_t+c(t,x)u_x=0$$ and attempt to solve it numerically. According to &lt;a href=&quot;http://en.wikipedia.org/wiki/Flux_limiter&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Flux_limiter&lt;/a&gt; I can apply Flux limiter method, say with Van Leer limiter function, and obtain a numerical method for the equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Flux limiter technique is for the discretization in space, I have to make a choice for the time. I compare explicit, implicit, and central difference. I would think explicit would have have some sort of stability/CFL condition which I don't like. Thus, I can use the other two. The latter is preferred due to a bettor error of second order. Thus, my first question is what type of time discretization is used in practice and why? Some basic intuition is sufficient to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, in either one of the implicit/central case, if I write the iteration explicitly for each time step, it requires the performance of Newton's iterations, simply because the equation is not linear anymore. However, in some cases there are only a few iterations are sufficient to obtain a solution provided a good initial guess. Since this is a nonlinear method the complexity is larger relative to a linear method, say implicit Euler in time and one sided difference in space. But, if there are only a few iterations to solve the nonlinear system to be performed, isn't comparable to the linear numerical methods in complexity? Thus, I want to know how complex those limiter methods and can they compete with linear methods in complexity? Or there is something else apart from Newton's iterations that make flux limiter methods more complex? Why one would like to have a linear method to solve an equation compare to non-linear? &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2013-03-13T00:40:17.543" LastActivityDate="2013-03-20T15:59:49.177" Title="complexity of flux limiter techniques" Tags="|hyperbolic-pde|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5449" PostTypeId="1" CreationDate="2013-03-06T06:08:39.247" Score="10" ViewCount="1159" Body="&lt;p&gt;I'm working on a 2D fluid sim using vortex particles/&amp;quot;vortons&amp;quot; as described in &lt;a href=&quot;http://software.intel.com/en-us/articles/fluid-simulation-for-video-games-part-1/&quot; rel=&quot;noreferrer&quot;&gt;Fluid Simulation for Video Games&lt;/a&gt;.  Which I think is the same things as the &amp;quot;discrete vortex method&amp;quot;.  Basically you represent the fluid with a collection of particles with defined vorticity, and calculate the fluid velocity at a point by summing up the induced velocities of all the vortons using Biot–Savart's law (eg: &lt;span class=&quot;math-container&quot;&gt;$v = \frac{\omega \times (p_2-p_1)}{4\pi r^3}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p_2-p_1$&lt;/span&gt; is the difference in position between the sample point and the vorton, &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is the vorticity (a vector quantity in 3D), and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is the Euclidean distance between the vorton and the sample point).&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to introduce a box in to the fluid and have it move back and forth and influence the fluid.  Which means taking in to account the no-slip and no-through boundary conditions for the box.  (That is, having the relative fluid velocity be 0 right at the box's boundary).&lt;/p&gt;&#xA;&lt;p&gt;Right now I'm sampling the relative velocity of the box and the fluid at 80 points around the perimeter of the box.  I also have 80 vortons placed near the perimeter of the box but offset a bit outward.  I form a large matrix and solve for the vorticity values the vortons need to counteract the fluid's velocity at the sample points (using linear least squares).&lt;/p&gt;&#xA;&lt;p&gt;This &lt;em&gt;almost&lt;/em&gt; works, but I've noticed that the solution I get back is highly dependent on the exact layout of sample points and vortons.&lt;/p&gt;&#xA;&lt;p&gt;Sometimes I get alternating patterns of vortons that spin in different directions (blue dots are vortons spinning counter clockwise, and red dots are vortons spinning clockwise):&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wCgX4.png&quot; alt=&quot;Alternating vorton flows&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Other times I get all the vorton's on the left side of the box spinning one way, and all the vortons on the other spinning the other way, like in this pic:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/z4Anq.png&quot; alt=&quot;Consistent flow&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think the second pic is what I actually want.  I also think that the two different solutions imply that the system of equations I'm using is too underconstrained.  Adding additional sample points sometimes helps, sometimes doesn't.&lt;/p&gt;&#xA;&lt;p&gt;Are there any other boundary conditions I could add that would help me get the results I'm after?  Intuitively I'm not doing anything to take in to account how strongly the fluid is disturbed at some given distance from the box.  I'm thinking maybe I could add in conditions for the boundary layer limit.  But I'm not sure exactly what that would look like.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, is there any relevant literature I could look at for how vortex particle based fluid sims handle these sorts of boundary conditions?&lt;/p&gt;&#xA;" OwnerUserId="1365" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-10-27T22:51:04.870" Title="Boundary conditions in fluid simulation" Tags="|fluid-dynamics|simulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5450" PostTypeId="1" CreationDate="2013-03-06T08:20:49.543" Score="3" ViewCount="116" Body="&lt;p&gt;I'm working on an API for simulation of port-Hamiltonian systems. As far as I understand it, a Hamiltonian system is symplectic if it is power conserving, and so including resistive elements would thus destroy the symplectic nature. My question is, if I add an extra state variable, the dissipated heat, along with its adjoint, does this restore its symplecticity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, if I didn't include the extra state variable, would I still benefit from using symplectic integrators? And do systems lying on a contact manifold still benefit from symplectic integrators?&lt;/p&gt;&#xA;" OwnerUserId="3827" LastActivityDate="2013-03-06T08:20:49.543" Title="Dissipation and symplectic manifolds" Tags="|algorithms|computational-geometry|ode|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5451" PostTypeId="2" ParentId="5446" CreationDate="2013-03-06T10:04:32.440" Score="1" Body="&lt;p&gt;I have figured out the problem. I had to increase the size of the domain further to remove boundary effects. Moreover, I had to reduce CFL number to around 0.5-1.0 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think CFL number need to be reduced further for higher reynolds number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially, I thought that I had reduced step size enough, but it was not the case.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2013-03-09T17:39:44.200" LastActivityDate="2013-03-09T17:39:44.200" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5452" PostTypeId="5" CreationDate="2013-03-06T12:19:17.497" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-03-06T12:19:17.497" LastActivityDate="2013-03-06T12:19:17.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5453" PostTypeId="4" CreationDate="2013-03-06T12:19:17.497" Score="0" Body="Often referred to as Algorithmic-Differentiation or AD -- a technique to automatically generate code that evaluates the derivative of a function. AD repeatedly applies the chain rule and classical rules of calculating derivatives. AD usually takes a block of code representing a function and returns a block of code representing that function's derivative." OwnerUserId="414" LastEditorUserId="541" LastEditDate="2013-09-22T15:15:29.037" LastActivityDate="2013-09-22T15:15:29.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5454" PostTypeId="5" CreationDate="2013-03-06T12:21:18.320" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-03-06T12:21:18.320" LastActivityDate="2013-03-06T12:21:18.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5455" PostTypeId="4" CreationDate="2013-03-06T12:21:18.320" Score="0" Body="OpenFOAM is a free, open source CFD software package developed by OpenCFD Ltd at ESI Group and distributed by the OpenFOAM Foundation." OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-03-06T14:14:48.753" LastActivityDate="2013-03-06T14:14:48.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5456" PostTypeId="5" CreationDate="2013-03-06T12:22:51.033" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-03-06T12:22:51.033" LastActivityDate="2013-03-06T12:22:51.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5457" PostTypeId="4" CreationDate="2013-03-06T12:22:51.033" Score="0" Body="NumPy is the fundamental package for scientific computing with Python." OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-03-06T14:14:28.817" LastActivityDate="2013-03-06T14:14:28.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5458" PostTypeId="2" ParentId="5449" CreationDate="2013-03-06T15:05:42.963" Score="1" Body="&lt;p&gt;Is the system you're trying to solve singular? Or at least ill-conditioned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would try adding some regularization to the system, e.g. if you have an energy formulation, add a small multiple of  $\Sigma||v_i||^2$. I think this corresponds to adding a $\lambda I$ term to the matrix you're solving. That should drive the system towards the second configuration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more complicated regularization would be a penalty on the &quot;derivative&quot;, e.g. $\Sigma w_{ij}||v_i - v_j||^2$ where $w_{ij}$ decays with the distance.&lt;/p&gt;&#xA;" OwnerUserId="12" LastActivityDate="2013-03-06T15:05:42.963" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5462" PostTypeId="1" CreationDate="2013-03-07T09:57:07.417" Score="9" ViewCount="174" Body="&lt;p&gt;Is there a reasonably cheap method to solve the large, dense, low rank assignment problem $\max_\pi \sum_i A_{\pi i,i}$, where $\pi$ runs over all permutations.of $1:n$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $A$ is an $n\times n$ matrix of low rank $r$. Typical sizes would be  $n=10000~~$ (possibly much larger), $r=15$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2013-03-08T11:48:15.133" LastActivityDate="2013-06-06T19:12:27.993" Title="large dense low rank assignment problem" Tags="|optimization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5463" PostTypeId="1" CreationDate="2013-03-07T11:06:07.103" Score="8" ViewCount="493" Body="&lt;p&gt;I'm looking at finding a solution to the following problem, but I'm having trouble formulating it sensibly, and then finding an appropriate algorithm to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a list of items placed in a shopping bag: 1, 2, 3, 4...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each item can be part of one or more promotions: A, B, C, D&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We wish to apply the set of promotions such that we maximise the value of the discount, that no item is allowed to participate in more than one promotion, but each promotion could be applied more than once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Promotions can be defined in a number of ways, but for now I'm just considering a type of 'Buy 2 qualifying items, save X', I'm hopeful I'll be able to generalise from here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My example would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Promotion A - Buy 2 Save 8&lt;/li&gt;&#xA;&lt;li&gt;Promotion B - Buy 2 Save 10&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Promotion C - Buy 2 Save 6&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Item 1 - Eligible for A, B&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Item 2 - Eligible for B&lt;/li&gt;&#xA;&lt;li&gt;Item 3 - Eligible for A, C&lt;/li&gt;&#xA;&lt;li&gt;Item 4 - Eligible for B, C&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's quite easy to see here that the correct application of promotions is A to Item 1, 3 and B to 2, 4. This gives a total discount of 18.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a larger case it becomes difficult, hence needing to solve algorithmically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;List all the possible combinations of Promotions we could apply.&lt;/li&gt;&#xA;&lt;li&gt;Discard any that are obviously poor (e.g. a direct copy of a promotion with a higher value).&lt;/li&gt;&#xA;&lt;li&gt;Apply any that have no overlap with other promotions (e.g. If Item 1 and 2 are only valid for promotion A, then apply that promotion).&lt;/li&gt;&#xA;&lt;li&gt;Take the remaining set and attempt a branch-and-bound type search on the results.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, this can take a long time (for large sets with similar discounts).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel this is a type of Knapsack or Assignment problem, but I can't write it sensibly. Without being able to write it sensibly I can't solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a recognised variant of a problem? Any help attacking it would be greatly appreciated, especially with psuedo-code to help solve it&lt;/p&gt;&#xA;" OwnerUserId="3855" LastEditorUserId="348" LastEditDate="2013-06-09T15:06:29.033" LastActivityDate="2013-10-07T16:35:25.133" Title="What is the correct formulation of this &quot;shopping bag&quot; optimization problem, and how can I solve it efficiently?" Tags="|optimization|algorithms|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5464" PostTypeId="1" CreationDate="2013-03-07T11:23:46.603" Score="9" ViewCount="4244" Body="&lt;p&gt;We need to compute covariance matrices with sizes ranging from $10000\times10000$ to $100000\times100000$. We have access to GPUs and clusters, we wonder what is the best parallel approach for speeding up these computations.&lt;/p&gt;&#xA;" OwnerUserId="891" LastEditorUserId="9667" LastEditDate="2015-08-30T20:13:27.287" LastActivityDate="2015-08-30T20:13:27.287" Title="Parallel computation of big covariance matrices" Tags="|matrix|parallel-computing|gpu|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="5465" PostTypeId="2" ParentId="5464" CreationDate="2013-03-07T19:18:12.317" Score="18" Body="&lt;p&gt;The first thing is to recognize that you can do this using BLAS.  If you data matrix is $X = [x_1 x_2 x_3 ...] \in \mathbb{R}^{m\times n}$ (each $x$ is a column vector corresponding to one measurement; rows are trials), then you can write the covariance as:&#xA;$$ C_{ij} = E[x_i,x_j] - E[x_i] E[x_j] = \frac{1}{n} \sum_k x_{ik} x_{jk} - \frac{1}{n^2} \left(\sum_k x_{ik} \right) \left(\sum_k x_{jk}\right) $$&#xA;We can write this as:&#xA;$$C = \frac{1}{n} X^T X - \frac{1}{n^2} (1^T X)^T (1^T X) $$&#xA;where $(1^T)$ is the row-vector with all elements 1 so $(1^T X)$ is a row vector of the column sums of $X$. This can be written completely as BLAS, where $X^T X$ is either a &lt;a href=&quot;http://www.netlib.org/blas/dgemm.f&quot;&gt;GEMM&lt;/a&gt; or, better yet, a &lt;a href=&quot;http://www.netlib.org/blas/dsyrk.f&quot;&gt;SYRK&lt;/a&gt;/&lt;a href=&quot;http://www.netlib.org/blas/zherk.f&quot;&gt;HERK&lt;/a&gt; and you can get the $(1^T X) = b$ with a &lt;a href=&quot;http://www.netlib.org/blas/dgemv.f&quot;&gt;GEMV&lt;/a&gt;, $b^T b$ with GEMM or SYRK/HERK again, and the prefactors with &lt;a href=&quot;http://www.netlib.org/blas/dscal.f&quot;&gt;SCAL&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your data and result matrices can be around 64GB, so you're not going to fit on a single node or a node's worth of GPUs.  For a non-GPU cluster, you might want to look at &lt;a href=&quot;http://www.netlib.org/scalapack/pblas_qref.html&quot;&gt;PBLAS&lt;/a&gt;, which feels like scalapack.  For GPUs, multi-node libraries aren't quite there yet.  &lt;a href=&quot;http://icl.cs.utk.edu/magma/&quot;&gt;Magma&lt;/a&gt; has some sort of underlying parallel BLAS implementation, but it may not be user friendly.  I don't think &lt;a href=&quot;http://www.culatools.com/cula_dense_programmers_guide/&quot;&gt;CULA&lt;/a&gt; does multi-node yet, but its something to keep an eye on.  &lt;a href=&quot;https://developer.nvidia.com/cublas&quot;&gt;CUBLAS&lt;/a&gt; is single-node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd also suggest that you strongly consider implementing the parallelism yourself , especially if you are familiar with MPI and have to hook this into an existing code-base.  That way, you can switch between CPU and GPU BLAS easily and begin and end with the data exactly where you want it.  You shouldn't need more than a few &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Allreduce.html&quot;&gt;MPI_ALLREDUCE&lt;/a&gt; calls.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2014-01-29T20:50:37.950" LastActivityDate="2014-01-29T20:50:37.950" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5468" PostTypeId="1" AcceptedAnswerId="5499" CreationDate="2013-03-07T21:17:57.383" Score="4" ViewCount="684" Body="&lt;p&gt;Let us assume I have an A-stable numerical scheme. I believe that given any linear equation $y' = Ay$, it means that the numerical scheme applied to this equation is stable (and therefore convergent since it is consistent) if the eigenvalues of A have a negative real part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is then, does this result extend to the non-linear case ? I am interested in particular in a system $y' = f(y,t)$ where the Jacobian of $f$ has negative real eigenvalues $\forall y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: Also I guess I could give a particular example here (but my question is not limited to that case): for example let us say the ODE describes a 2nd order chemical reaction network, i.e. $f \in R^n \rightarrow R^n$ is a polynomial function of the $y$ of order 2, where $n$ is the number of species. Considering the network I was able to prove that the eigenvalues of the Jacobian are real negative for any $y$, however I don't know if I can make the further argument that any A-stable method will converge (by the way, what about L-stable methods, B-stable methods, ... ?)&lt;/p&gt;&#xA;" OwnerUserId="3704" LastEditorUserId="3704" LastEditDate="2013-03-08T18:59:05.510" LastActivityDate="2013-03-10T10:04:47.567" Title="Stability of numerical schemes for non-linear equations with a Jacobian with negative eigenvalues" Tags="|stability|nonlinear-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5470" PostTypeId="2" ParentId="5463" CreationDate="2013-03-08T00:10:49.780" Score="1" Body="&lt;p&gt;It would seem like you'd want to take advantage of the structure of the problem that items can only be used toward one promotion. That means that if you've found a solution that uses as many promotions as possible, the only way to do better is to find another solution which takes one or more promotions plus the unassigned items and swaps them with another set of promotions and unused items of higher value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you might start the previous example with (1,2) and (3,4), for 16. You could then check that (1,3) and (2,4) leads to 18, and the swap should be made. &lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2013-03-08T00:10:49.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5471" PostTypeId="1" AcceptedAnswerId="5479" CreationDate="2013-03-08T06:42:34.117" Score="8" ViewCount="3666" Body="&lt;p&gt;Following on from my &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5434/conservation-of-a-physical-quantity-when-using-neumann-boundary-conditions-appli&quot;&gt;previous equation&lt;/a&gt; I'm would like to apply &lt;strong&gt;open&lt;/strong&gt; boundary condition to the advection-diffusion equation (with reaction term),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial \phi}{\partial t} = \frac{\partial}{\partial x}\left( D\frac{\partial \phi}{\partial x} + \boldsymbol{v(x)} \phi \right) + S(x,t) .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I define &quot;open&quot; as meaning a boundary which allows unimpeded transport whether it be by diffusion or drift.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm unsure how to mathematically state this problem. Would I just impose that the open boundary take a Dirichlet boundary condition where the fixed by the initial conditions? Moreover, this would define a node where the value never changes. This doesn't seem right somehow.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-03-09T03:45:19.193" Title="Open boundary conditions with the advection-diffusion equation" Tags="|pde|finite-difference|boundary-conditions|advection|diffusion|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5472" PostTypeId="2" ParentId="5430" CreationDate="2013-03-08T08:25:46.807" Score="2" Body="&lt;p&gt;Decomposing $A$ with $N&amp;gt;10^6$ can be even a problem for sparse direct solvers like UMFPACK. If you want to take an advantage out of the sparsity of $b$ you have to compute $LU=A$ and then compute the reachability set of the one non zero entry of $b$ over the graph defined by $L$. In this way you are able to solve $Ly=b$ and then compute the reachability set of the non zero elements of $y$ over the graph defined by $U$ and solve $Ux=y$ in this way. A easily understandable way computing these sets is described in the book &quot;Direct Solution of Sparse Linear Systems&quot; by Tim Davis. &lt;/p&gt;&#xA;" OwnerUserId="569" LastActivityDate="2013-03-08T08:25:46.807" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5473" PostTypeId="1" CreationDate="2013-03-08T11:19:53.283" Score="14" ViewCount="1656" Body="&lt;p&gt;I wish to implement the following expression in Python:&#xA;$$&#xA;x_i = \sum_{j=1}^{i-1}k_{i-j,j}a_{i-j}a_j,&#xA;$$&#xA;where $x$ and $y$ are numpy arrays of size $n$, and $k$ is a numpy array of size $n\times n$. The size $n$ might be up to about 10000, and the function is part of an inner loop that will be evaluated many times, so speed is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally I'd like to avoid a for loop altogether, though I guess it's not the end of the world if there is one. The problem is that I'm having trouble seeing how to do it without having a couple of nested loops, and that's likely to make it rather slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody see how to express the above equation using numpy in a way that's efficient, and preferably also readable? More generally, what is the best way to approach this sort of thing?&lt;/p&gt;&#xA;" OwnerUserId="3746" LastActivityDate="2016-02-06T12:55:06.897" Title="How to express this complicated expression using numpy slices" Tags="|python|numpy|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5474" PostTypeId="1" AcceptedAnswerId="5475" CreationDate="2013-03-08T02:09:50.370" Score="7" ViewCount="141" Body="&lt;p&gt;I have been reading about convex optimization.  We have:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;minimize $f(x)$ s.t.  $h(x) = 0$, $g(x) \le 0$, $x \in X$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's Lagrangian dual is:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;maximize $\phi(\lambda,\mu)$ s.t. $\mu \ge 0$, where $\phi(\lambda,\mu) = \inf[f(x) + \lambda' h(x) +  \mu 'g(x)]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand why $\mu$ must be greater than zero.  Can anyone please explain?&lt;/p&gt;&#xA;" OwnerUserId="3871" OwnerDisplayName="user34790" LastActivityDate="2013-03-09T15:12:24.293" Title="Confusion related to convex optimization" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5475" PostTypeId="2" ParentId="5474" CreationDate="2013-03-08T05:15:06.717" Score="7" Body="&lt;p&gt;For this post:&#xA;$\mathcal{L} = \mathcal{L}(\lambda, \mu) = f(x) + \lambda h(x) + \mu g(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You want $\mathcal{L}(\lambda,\mu) \le f(x)$ for all valid $x$.&lt;br&gt;&#xA;A valid $x$ is one that satisfies $h(x) = 0$, $g(x) \le 0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $x$ satisfies $h(x) = 0$, then the term $\lambda h(x)$ in $\mathcal{L}$ is zero regardless of $\lambda$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $x$ satisfies $g(x) \le 0$, the the term $\mu g(x)$ in $\mathcal{L}$ is negative or zero as long as $\mu \ge 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is required for $\mathcal{L} = f(x) + \lambda h(x) + \mu g(x) \le f(x)$ for valid $x$. This makes the Lagrangian a lower bound on $f(x)$ for valid $x$.&lt;/p&gt;&#xA;" OwnerUserId="3870" OwnerDisplayName="brent.payne" LastEditorUserId="3870" LastEditDate="2013-03-09T15:12:24.293" LastActivityDate="2013-03-09T15:12:24.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5476" PostTypeId="2" ParentId="5471" CreationDate="2013-03-08T15:43:43.500" Score="4" Body="&lt;p&gt;The best method I know of for this problem is by &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/%28SICI%291097-0363%2819970228%2924:4%3C393%3a%3aAID-FLD505%3E3.0.CO;2-O/abstract&quot; rel=&quot;nofollow&quot;&gt;Griffiths&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-03-08T15:43:43.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5477" PostTypeId="1" CreationDate="2013-03-08T16:03:52.483" Score="6" ViewCount="303" Body="&lt;p&gt;I want to evaluate the sum $$\sum_{k=1}^\infty \left(\frac{i+1}{\sqrt{2}}\right)^k\cdot k^{-\alpha}$$ where $i=\sqrt{-1}$ and $\alpha\in[\frac{3}{4},1]$ with 8 digits accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am willing to expend up to a second of CPU time per calculation, how should I proceed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I want a really fast scheme for the evaluation and I am willing to expend a lot of work. How should I proceed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any leads, clues or suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3868" LastEditorUserId="127" LastEditDate="2013-05-14T18:02:35.097" LastActivityDate="2013-05-19T18:31:37.503" Title="Evaluate the sum" Tags="|performance|fortran|c|special-functions|" AnswerCount="5" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5478" PostTypeId="1" CreationDate="2013-03-08T20:24:06.120" Score="17" ViewCount="22883" Body="&lt;p&gt;I have this confusion about Armijo rule used in line search. I was reading back tracking line search but didn't get what this Armijo rule is all about. Can anyone elaborate what Armijo rule is? The wikipedia doesn't seem to explain well. Thanks&lt;/p&gt;&#xA;" OwnerUserId="3871" OwnerDisplayName="user34790" LastActivityDate="2023-02-27T16:32:51.783" Title="Confusion about Armijo rule" Tags="|optimization|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5479" PostTypeId="2" ParentId="5471" CreationDate="2013-03-08T21:19:46.047" Score="5" Body="&lt;p&gt;That's not a simple question in my opinion. It is like if you were doing an experiment in a box and you just want to simulate the center of it. But obviously, the solution in the center of the box depends on what happens in the rest of the box. That is probably why it is a difficult problem to state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no definite answer to this question, but I still have some clues, hopefully you will find a suitable solution. (I'm not an expert in any of these solutions)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A first idea is to model the remaining part of the &quot;box&quot; with a &lt;strong&gt;reduced order model&lt;/strong&gt;. For example, you could plug an ODE for the two extremities of your domain with are supposed to represent the behaviour of the rest of the box. This is a technique that is used sometimes to &quot;end&quot; vessels in blood flow simulations &lt;code&gt;+&lt;/code&gt; You can easily imagine that the problem is well posed. You can also probably model incoming waves. It applies to FEM, probably to FD as well. &lt;code&gt;-&lt;/code&gt; You have to find out the ODE if not found in the litterature.&lt;/li&gt;&#xA;&lt;li&gt;A second option is to used &lt;strong&gt;absorbing boundary conditions&lt;/strong&gt;. These are tricky conditions that are supposed to absorb the information that is going out of the domain (also used sometimes for blood flows). If you google for &quot;absorbing boundary conditions advection diffusion&quot;, you should find some papers about it &lt;code&gt;+&lt;/code&gt; It (is supposed to) makes the effect you are looking for. &lt;code&gt;-&lt;/code&gt; I think that it was developed for the FEM, so I don't know if it applies to FD.&lt;/li&gt;&#xA;&lt;li&gt;The third option I see is the one proposed by Bill Barth, which is a &lt;strong&gt;&quot;no boundary&quot; condition&lt;/strong&gt;. It consists in prolongating the PDE up to the boundary and do not impose anything. &lt;code&gt;+&lt;/code&gt; In a FEM context, it's easy to implement and works &quot;surprisingly&quot; well (I did try them for fluid flows, in a slightly different context). &lt;code&gt;-&lt;/code&gt; I think that it's difficult to show that the problem is well posed (even if the discrete problem is). It applies to FEM, not sure for FD.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope it helps! (and good luck)&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-03-08T21:19:46.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5480" PostTypeId="2" ParentId="5478" CreationDate="2013-03-08T23:15:35.947" Score="21" Body="&lt;p&gt;Once you obtain a descent direction $p$ for your objective function $f(x)$, you need to pick a &quot;good&quot; step length.  You don't want to take a step that is too large such that the function at your new point is larger than your current point.  At the same time, you don't want to make your step too small such that it takes forever to get to converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Armijo's condition basically suggests that a &quot;good&quot; step length is such that you have &quot;sufficient decrease&quot; in $f$ at your new point.  The condition is mathematically stated as $$f(x_k+\alpha p_k)\leq f(x_k)+\beta\alpha\nabla f(x_k)^Tp_k$$ where $p_k$ is a descent direction at $x_k$ and $\beta\in(0,1)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intuition behind this is that the function value at the new point $f(x_k+\alpha p_k)$ should be under the &lt;em&gt;reduced&lt;/em&gt; &quot;tangent line&quot; at $x_k$ in the direction of $p_k$.  See Nocedal &amp;amp; Wright's book &quot;Numerical Optimization&quot;.  In chapter 3, there's an excellent graphical description of armijo's sufficient decrease condition.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="-1" LastEditDate="2015-05-01T01:12:16.740" LastActivityDate="2015-05-01T01:12:16.740" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5481" PostTypeId="2" ParentId="5473" CreationDate="2013-03-09T00:08:03.133" Score="4" Body="&lt;p&gt;Here's a start. First, my apologies for any mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I experimented with a couple of different approached. I was a bit confused by the limits on the summation - should the upper limit be $i$, rather than $i-1$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: No, the upper limit was correct as provided in the question. I have left it as is here because another answer now uses the same code, but the fix is simple.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First a looped version:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def looped_ver(k, a):&#xA;    x = np.empty_like(a)&#xA;    for i in range(x.size):&#xA;        sm = 0&#xA;        for j in range(0, i+1):&#xA;            sm += k[i-j,j] * a[i-j] * a[j]&#xA;        x[i] = sm&#xA;    return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I made it a single loop with numpy slices:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def vectorized_ver(k, a):&#xA;    ktr = zeros_like(k)&#xA;    ar = zeros_like(k)&#xA;    sz = len(a)&#xA;    for i in range(sz):&#xA;        ktr[i,:i+1] = k[::-1].diagonal(-sz+i+1)&#xA;        a_ = a[:i+1]&#xA;        ar[i,:i+1] = a_[::-1] * a_&#xA;    return np.sum(ktr * ar, 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The numpy version with one explicit loop is about 25x faster on my computer when $n=5000$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I wrote a Cython version of the (more-readable) looped code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import cython&#xA;cimport numpy as np&#xA;&#xA;@cython.boundscheck(False)&#xA;@cython.wraparound(False)&#xA;def cyth_ver(double [:, ::1] k not None,&#xA;              double [:] a not None):&#xA;    cdef double[:] x = np.empty_like(a)&#xA;    cdef double sm&#xA;    cdef int i, j&#xA;&#xA;    for i in range(len(a)):&#xA;        sm = 0.0&#xA;        for j in range(i+1):&#xA;            sm = sm + k[i-j,j] * a[i-j] * a[j]&#xA;        x[i] = sm&#xA;    return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On my laptop, this one is about 200x faster than the looped version (and 8x faster than the 1-loop vectorized version). I'm sure others can do better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I played with a Julia version, and it seemed (if I timed it properly) comparable to the Cython code.&lt;/p&gt;&#xA;" OwnerUserId="2015" LastEditorUserId="2015" LastEditDate="2013-03-11T21:46:11.453" LastActivityDate="2013-03-11T21:46:11.453" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5482" PostTypeId="2" ParentId="5471" CreationDate="2013-03-09T03:45:19.193" Score="1" Body="&lt;p&gt;The exact boundary condition for this case is not local. Rather, it is of the form&#xA;$$&#xA;  u(x,t)|_\Gamma = \int_\Gamma \int_{-\infty}^t Q(x,t,y,s) u(y,s) \; ds \; dy,&#xA;$$&#xA;i.e., it is a (spatially and temporally) nonlocal operator with a kernel $Q$. In some interesting cases, you can write this operator as a pseudo-differential operator and only take the first few terms to get something local. However, these will only be approximations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-09T03:45:19.193" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5483" PostTypeId="2" ParentId="5474" CreationDate="2013-03-09T03:55:44.033" Score="2" Body="&lt;p&gt;Ignore the equality constraint for a moment and only think of the inequality constrained problem (assuming all functions are at least once continuously differentiable). Now think about what it means for a point $x^\ast$ to be a minimizer of your problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If it lies in the interior of the domain described by $g(x)\le 0$ (i.e.: $g(x^\ast)&amp;lt;0$), then $x^\ast$ can only be a minimizer if $\nabla f(x^\ast)=0$ because otherwise going in direction $-\nabla f$ would lower the function value. So, in this case, the optimality condition $\nabla f(x^\ast) + \mu^\ast \nabla g(x^\ast)=0$ is satisfied only if $\mu^\ast=0$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The consider the case where the minimizer $x^\ast$ lies at the boundary of the feasible domain, i.e., $g(x^\ast)=0$. In this case, for $x^\ast$ to be a minimizer, the gradient of $f$ at $x^\ast$ must point perpendicularly to the boundary and &lt;em&gt;into&lt;/em&gt; the feasible set since, otherwise, going into the domain or along some direction along the boundary a little bit would lower the function value. On the other hand, the direction perpendicular to the boundary in the &lt;em&gt;outward&lt;/em&gt; direction is given by $\nabla g(x^\ast)$. Consequently, we can express the optimality condition as saying that $\nabla f(x^\ast)$ must be equal to a &lt;em&gt;negative&lt;/em&gt; multiple of $\nabla g(x^\ast)$. In other words, $\nabla f(x^\ast) + \mu \nabla g(x^\ast) = 0$ with some &lt;em&gt;positive&lt;/em&gt; multiplier $\mu$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Makes sense? It's actually a pretty intuitive argument if you think of it in terms of geometry.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-09T03:55:44.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5485" PostTypeId="2" ParentId="5473" CreationDate="2013-03-09T07:44:32.373" Score="17" Body="&lt;p&gt;Here is the Numba solution.  On my machine the Numba version is &gt;1000x faster than the python version without the decorator (for a 200x200 matrix, 'k' and 200-length vector 'a').   You can also use the @autojit decorator which adds about 10 microseconds per call so that the same code will work with multiple types. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numba import jit, autojit&#xA;&#xA;@jit('f8[:](f8[:,:],f8[:])')&#xA;#@autojit&#xA;def looped_ver(k, a):&#xA;    x = np.empty_like(a)&#xA;    for i in range(x.size):&#xA;        sm = 0.0&#xA;        for j in range(0, i+1):&#xA;            sm += k[i-j,j] * a[i-j] * a[j]&#xA;        x[i] = sm&#xA;    return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I am one of the Numba developers.&lt;/p&gt;&#xA;" OwnerUserId="3869" LastEditorUserId="276" LastEditDate="2013-03-11T16:31:31.280" LastActivityDate="2013-03-11T16:31:31.280" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5486" PostTypeId="1" AcceptedAnswerId="5495" CreationDate="2013-03-09T14:02:55.447" Score="8" ViewCount="14673" Body="&lt;p&gt;Is there an algorithm to rearrange a matrix into block diagonal form, given that the matrix is block diagonal in nature but randomized with an unwise choice of basis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, are there any python modules for this?&lt;/p&gt;&#xA;" OwnerUserId="2102" LastActivityDate="2013-03-09T22:53:51.723" Title="Rearrange an ordinary matrix to block diagonal form" Tags="|matrix|python|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5487" PostTypeId="2" ParentId="5477" CreationDate="2013-03-09T14:10:16.940" Score="6" Body="&lt;p&gt;What have you tried so far? This completely naive implementation manages to compute 7 (maybe 7.5) digits in 2.5 seconds on my laptop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;complex&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;iomanip&amp;gt;&#xA;&#xA;int main ()&#xA;{&#xA;  const double alpha = 1;&#xA;  std::cout.precision(16);&#xA;&#xA;  std::complex&amp;lt;double&amp;gt; sum = 0;&#xA;  for (unsigned int k=1; k&amp;lt;10000000; ++k)&#xA;    {&#xA;      sum += std::pow(std::complex&amp;lt;double&amp;gt;(1,1)/std::sqrt(2.), k)&#xA;         *&#xA;         std::pow(k, -alpha);&#xA;&#xA;      if (k % 1000000 == 0)&#xA;    std::cout &amp;lt;&amp;lt; k &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; sum &amp;lt;&amp;lt; std::endl;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g/x&amp;gt; c++ -O3 x.cc&#xA;g/x&amp;gt; time ./a.out &#xA;1000000 (0.2674004983680959,1.178096037989336)&#xA;2000000 (0.2674002483693692,1.17809664154278)&#xA;3000000 (0.2674001650362904,1.178096842727255)&#xA;4000000 (0.267400123369689,1.178096943319499)&#xA;5000000 (0.2674000983697316,1.178097003674927)&#xA;6000000 (0.2674000817030793,1.178097043911768)&#xA;7000000 (0.2674000697983399,1.178097072652377)&#xA;8000000 (0.2674000608697807,1.17809709420789)&#xA;9000000 (0.2674000539253223,1.178097110973249)&#xA;&#xA;real    0m2.553s&#xA;user    0m2.548s&#xA;sys     0m0.001s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm certain this can be vastly improved, for example by using the fact that $(1+i)^k$ can only attain 8 different values that could be tabulated, that $\sqrt{2}^k$ can be computed very easily, etc. I have no doubt that with 20 minutes of work I could make this run in under one second for 8 digits. It might be harder to do for smaller values of $\alpha$, however, where convergence is slower.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-09T14:10:16.940" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5488" PostTypeId="1" AcceptedAnswerId="7502" CreationDate="2013-03-09T15:17:41.640" Score="4" ViewCount="1775" Body="&lt;p&gt;I am looking for easily to understand example of level set method used to track phases interfaces. I would like to solve it using FEM because my solution is based on the FEM solution of second Fick law. Initialization step is straight forward - it just distance of each node from the interface. But I have problem with second step - solution of the interface velocity equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial \phi }{\partial t}+F\left| \nabla \phi  \right|=0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly I need a weak formulation of that differential equation, where F is linear velocity of the interface. Solution will give me a new position of the interface. Is it correct? &lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2013-06-03T08:46:09.070" Title="Example of level set method" Tags="|finite-element|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5489" PostTypeId="1" CreationDate="2013-03-09T15:54:47.353" Score="6" ViewCount="185" Body="&lt;p&gt;I was reading this paper &lt;a href=&quot;http://www.ist.temple.edu/~vucetic/documents/wang11kdd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ist.temple.edu/~vucetic/documents/wang11kdd.pdf&lt;/a&gt; related to adaptive multi-hyperplane machine for non linear classification&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that paper, they have mentioned about multiclass SVM, with multiple weights for each class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loss for any classification is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$l(x_n,y_n) = max_{i\epsilon y\\\y_n}(0,1 + max g(i,x_n) - g(y_n,x_n))$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $y_n$ is the label for the nth example and $x_n$ is the features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have this confusion when they do the training of this algorithm. They call this SVM MM(Multiple Hyperplane).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They say the convex-approximated problem is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$min_{W}P(W|z) = \frac{\lambda}{2}||W||^2 + \frac{1}{N}\sum_{n=1}^{N}l_{cvx}(W;(x_n,y_n);z_n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where they have the concave term $-g(y_n,z_n)$ replaced with the convex term $-w^T_{y_n,z_n}x_n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if I have described it clearly. But I am going to attach the screenshot of the paper as well. The thing is I didn't get what's the difference between $-g(y_n,z_n)$ and $-w^T_{y_n,z_n}x_n$. They seem the same term to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might be asking a lot. But can anyone provide some info?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/33iB6.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/TNtk0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have marked by the red rectangle the part that I didn't understand. I might be asking a lot. But I didn't get that part. Why is it so?&lt;/p&gt;&#xA;" OwnerUserId="3871" LastActivityDate="2013-09-06T02:21:51.517" Title="Confusion related to convexity and concavity of a function" Tags="|optimization|convex-optimization|support-vector-machines|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5490" PostTypeId="2" ParentId="5486" CreationDate="2013-03-09T18:24:11.930" Score="3" Body="&lt;p&gt;Every matrix is block-diagonal in a wise choice of basis - this is called the &lt;a href=&quot;http://en.wikipedia.org/wiki/Jordan_normal_form&quot; rel=&quot;nofollow&quot;&gt;Jordan normal form&lt;/a&gt;, and the basis is made up of its generalized eigenvectors. If the matrix is symmetric, this basis is made up of eigenvectors, and you can compute it using, e.g., the &lt;a href=&quot;http://en.wikipedia.org/wiki/QR_algorithm&quot; rel=&quot;nofollow&quot;&gt;QR algorithm&lt;/a&gt;. SciPy provides the module &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.qr.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;linalg.qr&lt;/code&gt;&lt;/a&gt; to compute the necessary QR decompositions. Otherwise, you could use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow&quot;&gt;singular value decomposition&lt;/a&gt;, which can be computed using &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.svd.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;linalg.svd&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-03-09T18:24:11.930" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5491" PostTypeId="1" AcceptedAnswerId="5492" CreationDate="2013-03-09T19:25:49.800" Score="2" ViewCount="190" Body="&lt;p&gt;I was going through this lecture related to convex optimization. It was proved that logdet function is concave. However, I didn't get the derivation at a part&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/MY5ri.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't get how the step marked in red in the given picture was derived. Which property was used&lt;/p&gt;&#xA;" OwnerUserId="3871" LastActivityDate="2013-03-09T19:42:39.430" Title="Confusion while proving logdet function to be concave" Tags="|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5492" PostTypeId="2" ParentId="5491" CreationDate="2013-03-09T19:42:39.430" Score="5" Body="&lt;p&gt;There are three properties being used here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the determinant of a product of square matrices of equal size is equal to the product of the determinants of its square, equally sized factors&lt;/li&gt;&#xA;&lt;li&gt;the logarithm of a product is equal to the sum of the logarithms of its factors&lt;/li&gt;&#xA;&lt;li&gt;the determinant of a matrix is equal to the product of its eigenvalues.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Let's start from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;g(t) = \log \det (Z^{1/2}(I + tZ^{-1/2}VZ^{-1/2})Z^{1/2}).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The determinant product property yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;g(t) = \log\Big(\det(Z^{1/2})\det(I + tZ^{-1/2}VZ^{-1/2})\det(Z^{1/2})\Big),&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and by commutativity of multiplication, we can rearrange it so that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;g(t) &amp;amp;= \log\Big(\det(Z^{1/2})\det(Z^{1/2})\det(I + tZ^{-1/2}VZ^{-1/2})\Big), \\&#xA;     &amp;amp;= \log\Big(\det(I + tZ^{-1/2}VZ^{-1/2})\det(Z)\Big).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the product-sum property of logarithms, this expression becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;g(t) &amp;amp;= \log\det(I + tZ^{-1/2}VZ^{-1/2}) + \log\det(Z),&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we're most of the way there with the $\det(Z)$ term already separated out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the eigenvalues of $Z^{-1/2}VZ^{-1/2}$ are $\lambda_{i}$, then:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the eigenvalues of $tZ^{-1/2}VZ^{-1/2}$ are $t\lambda_{i}$, because the eigenvectors of each matrix are the same&lt;/li&gt;&#xA;&lt;li&gt;the eigenvalues of $I + tZ^{-1/2}VZ^{-1/2}$ are $(1 + t\lambda_{i})$ because the eigenvectors of $Z^{-1/2}VZ^{-1/2}$ will also be eigenvectors for $I$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This aside about eigenvalues is important, because to complete the derivation, we need to use the property that the determinant of a matrix is the product of its eigenvalues:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;g(t) &amp;amp;= \log\left(\prod_{i=1}^{n}(1+t\lambda_{i})\right) + \log\det(Z).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To complete the derivation, we use the product-sum property of logarithms (again), and get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;g(t) = \sum_{i=1}^{n} \log(1 + t\lambda_{i}) + \log\det(Z).&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-03-09T19:42:39.430" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5493" PostTypeId="1" CreationDate="2013-03-09T20:18:27.877" Score="0" ViewCount="248" Body="&lt;p&gt;It looks like MATLAB and Mathematica is often used for post-processing and data visualization of 3D vector data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider you are simulating 3D objects and get out gradient fields etc., is there a software or library one can use to change perspective of this 3D visualized vector data like it usual in CAD tools, so you can zoom away via mouse or into that structure, change perspective without having to alter complete post-processing or doing visualizing of 2D slices to get &quot;3D&quot; impression. &lt;/p&gt;&#xA;" OwnerUserId="3873" LastActivityDate="2013-09-01T17:28:39.783" Title="CAD like visualisation of 3D vector data?" Tags="|software|visualization|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5494" PostTypeId="1" CreationDate="2013-03-09T20:58:35.627" Score="1" ViewCount="116" Body="&lt;p&gt;I have this little confusion related to interior point method. In this method we use the log barrier function to approximate the real barrier which is not differential&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now when we find the optimal solution for this objective with log barrier function lets say $x^*(t)$ and its dual $\lambda^*(t)$ and $\gamma^*(t)$ where $\lambda$ is Lagrangian multiplier for the inequality constraints and $\gamma$ for the equality constraints. Now it's proved that the duality gap for $x^*(t)$ and $\gamma^*(t),\lambda^*(t)$ is m/t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe $x^*$ and $\gamma^*(t),\lambda^*(t)$ are for the approximated function with log barrier. However, the real function is still there. They say that the &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x^*(t)) - p^*  \le m/t$ where $p^*$ is the actual optimum of the original function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't get how this came to be true. I mean m/t bound is for the primal and dual optimal of the approximated function. How come it applies for the optimal of the approximated function and optimal of the original function. Suggestions? &lt;/p&gt;&#xA;" OwnerUserId="3871" LastActivityDate="2013-04-18T16:11:38.477" Title="Confusion related to interior point method for optimization" Tags="|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5495" PostTypeId="2" ParentId="5486" CreationDate="2013-03-09T22:53:51.723" Score="7" Body="&lt;p&gt;Is the matrix sparse or dense? Is it symmetric?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm assuming by &quot;rearrange&quot; you mean permute the entries, rather than apply some more general similarity transformation to the matrix. In that case, you can think of an $n\times n$ matrix $A$ as graph; two vertices $i$, $j$ of this graph are connected if $A_{ij} \neq 0$. If the matrix is not symmetric, then the edges are directed, but it's the same idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that your matrix is (up to a reordering) block diagonal means that the graph isn't connected, and finding which vertices should be in a block together amounts to finding the connected components of the graph. You can do this with a &lt;a href=&quot;http://en.wikipedia.org/wiki/Breadth-first_search&quot;&gt;breadth-first search&lt;/a&gt;. Since the &lt;a href=&quot;http://en.wikipedia.org/wiki/Cuthill-McKee_algorithm&quot;&gt;reverse Cuthill-McKee ordering&lt;/a&gt; of a matrix is essentially a breadth-first search, you can probably find someone's Python code for the RCM ordering and use it directly or modify it for your purposes.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-03-09T22:53:51.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5496" PostTypeId="1" CreationDate="2013-03-10T00:16:26.820" Score="1" ViewCount="166" Body="&lt;p&gt;Are there any libraries available for solving problems at the level of college physics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, I'm interested in libraries for general purpose programming languages that can calculate results both numerically &lt;em&gt;and&lt;/em&gt; symbolically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Libraries which could be used for implementing something like this are SymPy (Python), GiNaC (C++), or SymbolicC++ (C++).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking this question because I've started cooking up a &lt;a href=&quot;https://github.com/dharmatech/Symbolism&quot; rel=&quot;nofollow noreferrer&quot;&gt;C# library for symbolic computation&lt;/a&gt; and have used it to solve a few projectile motion problems, but would like to review other work out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/dharmatech/Symbolism/blob/master/PSE%205E%20Example%204.5%20Thats%20Quite%20An%20Arm/Program.cs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's an example&lt;/a&gt; of a program which solves such a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what the program outputs when run:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZX14W.png&quot; alt=&quot;photo&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers or suggestions welcome. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3874" LastActivityDate="2013-03-10T00:16:26.820" Title="What libraries are available for solving problems at the college physics level?" Tags="|symbolic-computation|" AnswerCount="0" CommentCount="4" ClosedDate="2018-11-08T13:37:05.433" ContentLicense="CC BY-SA 3.0" />
  <row Id="5497" PostTypeId="2" ParentId="5489" CreationDate="2013-03-10T00:57:06.163" Score="1" Body="&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; Most of my optimization-related work has been done in nonconvex optimization, and I have no training in Support Vector Machines (SVM) whatsoever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on what I can tell from reading Wikipedia and my optimization background, my guess is that they chose to replace the original nonconvex objective with an upper bound to formulate a convex restriction to the original problem. The &quot;loss term&quot; looks like a penalty term designed to ensure that the normal vectors of the hyperplanes in this particular SVM are all distinct; replacing the nonconvex term with a greater convex term in the objective function of a minimization problem only increases the penalty assigned to normal vectors that are &quot;too similar&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This modification was probably done to make the problem substantially easier to solve; convex optimization problems are solvable in polynomial time (as a function of the number of decision variables, constraints, and the number of bits it takes to store the inputs), whereas nonconvex optimization problems are not solvable in polynomial time, and typically have algorithms that are exponential run times (again, as a function of the number of decision variables, etc.). All of these exponential run time algorithms are sophisticated variants of guess-and-check, because that strategy is the best we can do in the absence of theoretical breakthroughs like $\mathcal{P} = \mathcal{NP}$.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-03-10T00:57:06.163" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5498" PostTypeId="2" ParentId="5493" CreationDate="2013-03-10T05:14:31.523" Score="9" Body="&lt;p&gt;Absolutely. You should check out &lt;a href=&quot;https://wci.llnl.gov/codes/visit/&quot;&gt;VisIt&lt;/a&gt;, &lt;a href=&quot;http://www.paraview.org/&quot;&gt;Paraview&lt;/a&gt;, &lt;a href=&quot;http://www.tecplot.com/&quot;&gt;Tecplot&lt;/a&gt;, &lt;a href=&quot;http://www.ensight.com/&quot;&gt;Ensight&lt;/a&gt;, and similar tools.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-03-10T05:14:31.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5499" PostTypeId="2" ParentId="5468" CreationDate="2013-03-10T10:04:47.567" Score="6" Body="&lt;p&gt;I'm going to answer a more general question than the one you asked: do the eigenvalues of an initial value ODE determine the stability of the solution?  Here I'm referring to mathematical stability, not numerical stability.  Of course, a &quot;yes&quot; to this question is a necessary condition for a &quot;yes&quot; to your question.  And unfortunately, the answer is &quot;no&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, for nonlinear and/or non-autonomous problems, studying the eigenvalues of a &lt;em&gt;frozen linearization&lt;/em&gt;, i.e. the Jacobian with $t$ fixed, gives very useful insight.  However, there are two major caveats:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For non-normal matrices, eigenvalues don't tell the whole story, and one must consider pseudospectra instead.  This caveat applies even to linear, constant-coefficient problems.  See the &lt;a href=&quot;https://pup.princeton.edu/titles/8113.html&quot;&gt;book by Trefethen &amp;amp; Embree&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There exist pathological examples of systems where the solution behavior is completely unrelated to the eigenvalues.  I will give an example that is attributed to Vinograd:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$y'(t) = A(t) y(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A(t) = \begin{pmatrix} -1-9\cos^2(6t)+6\sin(12t) &amp;amp; 12\cos^2(6t) + \frac{9}{2}\sin(12t) \\ -12\sin^2(6t) + \frac{9}{2}\sin(12t) &amp;amp; -1 - 9 \sin^2(6t)-6\sin(12t) \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigenvalues of $A(t)$ are $\lambda=-1, -10$ -- independent of $t$.  But the solution is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y(t) = C_1 \exp(2t) \begin{pmatrix} \cos(6t) + 2\sin(6t) \\ 2\cos(6t)-\sin(6t) \end{pmatrix} + C_2 \exp(-13t) \begin{pmatrix} \sin(6t) - 2\cos(6t) \\ 2\sin(6t)+\cos(6t) \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in this case, the eigenvalues suggest stability, but the solution exhibits unbounded exponential growth.  For a nice discussion of generalizations of this example, see &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0444876340&quot;&gt;this excellent book&lt;/a&gt;.  It is out of print, but used copies can sometimes be found -- for a hefty price -- on Amazon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some further comments: I don't know of a nonlinear, autonomous example exhibiting this kind of behavior, and it may well be that they don't exist.  I also don't know of an example arising in applications.  People do generally use the eigenvalues of the frozen jacobian as an indicator of stability in practice.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-03-10T10:04:47.567" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5500" PostTypeId="2" ParentId="2917" CreationDate="2013-03-10T13:12:38.377" Score="1" Body="&lt;p&gt;Ming Gu and Stanley C. Eisenstat have studied this problem before, see the link: &lt;a href=&quot;http://www.cs.yale.edu/publications/techreports/tr916.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.yale.edu/publications/techreports/tr916.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This paper solves the rank-one permutation problem, which cannot solve the problem here. If anyone meet the rank-one permutation problem, it helps. &lt;/p&gt;&#xA;" OwnerUserId="3877" LastEditorUserId="3877" LastEditDate="2013-05-06T03:27:25.440" LastActivityDate="2013-05-06T03:27:25.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5501" PostTypeId="1" AcceptedAnswerId="5502" CreationDate="2013-03-10T17:54:31.167" Score="3" ViewCount="67" Body="&lt;p&gt;I have a system which I assume is linear. I have a matrix $A$ of which each row is a coefficient of a unknown variables in vector $x$. I have vector $B$ which contains the result of each $Ax$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this system I can obtain the vector x. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: How do I obtain the smallest set of variables in $A$ which result in accurate computation of $b$ ?. i.e., how do I eliminate variables which don't contribute useful information to the solution?&lt;/p&gt;&#xA;" OwnerUserId="1573" LastActivityDate="2013-03-10T19:24:09.157" Title="How to obtain the minimum set of variables required in a model to produce accurate estimation?" Tags="|linear-algebra|statistics|regression|testing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5502" PostTypeId="2" ParentId="5501" CreationDate="2013-03-10T19:24:09.157" Score="2" Body="&lt;p&gt;The problem you are discussing sounds like model order reduction (also called &quot;model reduction&quot;). Principal component analysis (also called Proper Orthogonal Decomposition, Karhunen-Loeve analysis, and other names) is one way of achieving model order reduction by imposing certain assumptions that reduce a problem to a singular value decomposition. There are other methods out there, including but not limited to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;balanced truncation&lt;/li&gt;&#xA;&lt;li&gt;trajectory piecewise linearization (TPWL)&lt;/li&gt;&#xA;&lt;li&gt;discrete empirical interpolation method (DEIM) and its related continuous method, the empirical interpolation method (EIM)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Depending on the assumptions you want to make, you will get different reduced order models, each with different associated approximation errors. If error is important to you, I urge you to do some research to determine if the model order reduction method you use is accurate and appropriate for your linear system. (Error is still an open problem, but for now, you can test a couple different methods and compare to see which ones give you better or worse results on a battery of representative problems for your application.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-03-10T19:24:09.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5503" PostTypeId="1" AcceptedAnswerId="5512" CreationDate="2013-03-10T19:27:37.210" Score="8" ViewCount="4192" Body="&lt;p&gt;In Matlab, I would like to minimize the function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(S)=\mathrm{trace}(S)+\mathrm{trace}(S^{-2})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $S \in \mathcal{M}_{m,m}$ is symmetric and positive definite, which is definitely a convex function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cvx_solver sdpt3&#xA;cvx_begin quiet&#xA;variable S(m,m) symmetric;&#xA;S == semidefinite(m);&#xA;minimize (trace(S)+trace_inv(square(S)));&#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After running this, I got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;??? Error using ==&amp;gt; cvx.trace_inv at 9&#xA;Input must be affine.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually this means that what is written in the code is equivalent to have a quadratic equality in the constraint. That is, if we replace in the code $S^2$ by another $Q$ matrix, we have to add as constraint $Q=S^2$ which is convex equality since it's quadratic whereas in order to have a convex optimization problem we must have as constraints affine equalities and/or a convex inequalities. However, in reality, this is not true since my function is convex and I'm not imposing any constraint but cvx is making its own reformulation of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I even tried to solve the problem in different manner as follow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cvx_solver sdpt3&#xA;cvx_begin quiet&#xA;variable Q(m,m) symmetric;&#xA;Q == semidefinite(m);&#xA;minimize (trace_sqrtm(Q) + trace_inv(Q));&#xA;cvx_end&#xA;S=sqrtm(Q);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;??? Error using ==&amp;gt; cvx.plus at 83&#xA;Disciplined convex programming error:&#xA;   Illegal operation: {concave} + {convex}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which means that the function $\mathrm{trace}(S^{1/2})$ is a concave function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm wondering how can I write my problem in a different manner in order to be accepted by cvx given that my function $f(S)=\mathrm{trace}(S)+\mathrm{trace}(S^{-2})$ is convex.&#xA;If there is another option rather than cvx that would work too for me.&lt;/p&gt;&#xA;" OwnerUserId="3880" LastEditorUserId="20417" LastEditDate="2018-02-05T14:27:53.357" LastActivityDate="2018-05-17T06:15:36.173" Title="Minimizing $\mathrm{trace}(S)+\mathrm{trace}(S^{-2})$ using CVX" Tags="|linear-algebra|matlab|convex-optimization|cvx|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5504" PostTypeId="2" ParentId="5503" CreationDate="2013-03-10T19:38:36.087" Score="3" Body="&lt;p&gt;Nonlinear equality constraints are nonconvex, which is why CVX won't accept your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a scalar example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the equality constraint $x^{2} = y$, and consider its associated feasible set $T = \{(x,y): x^{2} - y = 0\}$. Both $(0,0)$ and $(1,1)$ are in $T$. If $T$ is a convex set, then the line $(z,z)$ for $z \in [0,1]$ should be in $T$. However, $(1/2, 1/2)$ is not in $T$, so $T$ is not convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, the source of your problem could be that $S^{-n}$, for $n$ positive integer, and $S$ positive semidefinite, is not an atomic expression recognized by CVX as convex. If you had to prove it yourself, I doubt it's in there, and I couldn't find it in this &lt;a href=&quot;http://www.stanford.edu/~boyd/papers/pdf/disc_cvx_prog.pdf&quot; rel=&quot;nofollow&quot;&gt;reference&lt;/a&gt;. If you can add $S^{-n}$ as an atomic expression that is convex, you should be okay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use &lt;code&gt;fmincon&lt;/code&gt;, you should theoretically be able to, but I would think that the main problem would be enforcing the constraint that $S$ is positive semidefinite. You would likely be better off using something like YALMIP, which is a frontend for many solvers, including SDP solvers.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-03-10T22:55:51.063" LastActivityDate="2013-03-10T22:55:51.063" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5506" PostTypeId="2" ParentId="5503" CreationDate="2013-03-10T23:04:43.703" Score="1" Body="&lt;p&gt;As Michael Grant (principal author of CVX) responds in his answer &lt;a href=&quot;http://ask.cvxr.com/question/400/trace-of-s-2-in-cvx/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, you're trying to shove a general convex programming problem into CVX, which is designed to handle disciplined convex programming problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why you are not getting anything reasonable out of CVX, you must first try to reformulate your problem into disciplined convex form.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Geoff suggests, you might have luck with more general solvers, particularly if you know your problem is convex and you are not solving a large problem.  &lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2013-03-10T23:04:43.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5508" PostTypeId="1" AcceptedAnswerId="5511" CreationDate="2013-03-11T02:42:12.373" Score="4" ViewCount="2409" Body="&lt;p&gt;I also posted this question on the &lt;a href=&quot;https://math.stackexchange.com/questions/327062/implementing-the-theta-method-for-solving-an-ivp&quot;&gt;Math SE site&lt;/a&gt;, but it was suggested I post here also.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to implement the $\theta$-method to solve an IVP in MATLAB. The $\theta$-method is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y_{j+1} = y_j + h[\theta f(t_j, y_j) + (1 - \theta)f(t_{j+1}, y_{j+1})]$ for $\theta \in [0, 1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use it to solve an IVP of this form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y_1\prime = f_1(y_1, y_2) \;\;\;\;\; y_2\prime = f_2(y_1, y_2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t \in [a, b]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y_1(a) = \alpha \;\;\;\;\; y_2(a) = \alpha \;\;\;\;\;\;$ (The initial conditions)        &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how I would implement this, since I'm used to implementing methods in which there is only one DE, such as $y\prime=f(t, y)$ (like Euler's method or the Runge-Kutta).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I did so far in MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function thetaMeth(a, b, h, f, y)&#xA;&#xA;    j = 1;&#xA;    theta = 0.5&#xA;    for i = a:h:b&#xA;        y(j+1) = y(j) + h*(theta*f(j, y(j)) + (1 - theta)*f(j+1, y(j+1)));&#xA;        fprintf('t = %f; y(%d) = %f\n', i, j, y(j));&#xA;    end&#xA;        j = j + 1;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My parameter &lt;code&gt;f&lt;/code&gt; would be for the $y_1\prime$ and $y_2\prime$ equations (it could possibly be taken in as a cell array of functions). I'm not sure what to do with the &lt;code&gt;y&lt;/code&gt; parameter, because the solution equations $y_1$ and $y_2$ are not given (I'm not really sure if I should even have &lt;code&gt;y&lt;/code&gt; as a parameter). This is an implicit method, and you need to have $y_{j+1}$ calculated beforehand somehow, because it appears in the LHS and RHS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone have any pointers as to how I can implement this? It doesn't have to be in MATLAB, I just need an algorithm that can be translated into code, since I'm not sure how to implement the algorithm myself.&lt;/p&gt;&#xA;" OwnerUserId="3798" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-03-11T13:55:18.117" Title="Implementing the $\theta$-method for solving an IVP" Tags="|algorithms|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5509" PostTypeId="1" CreationDate="2013-03-11T06:48:25.637" Score="5" ViewCount="586" Body="&lt;p&gt;I'm working on a 2D inviscid fluid simulation using a &quot;panel method&quot;, with Potential being used to enforce the no-through boundary condition.  I'm trying to incorporate the &lt;a href=&quot;http://en.wikipedia.org/wiki/Kutta_condition&quot; rel=&quot;nofollow&quot;&gt;Kutta condition&lt;/a&gt;, which says that the pressure above and below an airfoil are equal when the streams meet at the trailing edge, or equivalently that the velocity is smoothly leaving the trailing edge from the top and bottom of the airfoil in the same direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's usually invoked relating to airfoils, and most literature on the subject (eg: &lt;a href=&quot;http://www.ltas-aea.ulg.ac.be/cms/uploads/Aerodynamics04.pdf&quot; rel=&quot;nofollow&quot;&gt;Aerodynamics&lt;/a&gt; or &lt;a href=&quot;http://gallery.baek.eu/albums/userpics/10001/UnsteadyPanelMethod.pdf&quot; rel=&quot;nofollow&quot;&gt;Modelling of Steady &amp;amp; Unsteady Flow Around 2D Airfoils Using Panel Methods&lt;/a&gt;) assumes you know beforehand where your trailing edge is.  However, I want to be able to simulate any arbitrary 2D surface (well, as arbitrary as you can get with simple polygons) in any arbitrary unsteady flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a generalized version of the Kutta condition that I can use?  Something that would work for bluff bodies, airfoils, and any other random shape and still give the proper results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or alternatively, is it actually necessary to enforce it if I'm aiming for a purely inviscid flow?  A couple of different sources have hinted that it's actually a fudge that enables you to take one of the important aspects of viscous flow without all the rest of the overhead.&lt;/p&gt;&#xA;" OwnerUserId="1365" LastEditorDisplayName="user29948" LastEditDate="2019-02-11T10:50:31.400" LastActivityDate="2023-06-21T05:05:40.683" Title="General case Kutta condition" Tags="|fluid-dynamics|computational-physics|simulation|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5510" PostTypeId="2" ParentId="5503" CreationDate="2013-03-11T09:24:36.920" Score="3" Body="&lt;p&gt;You might wish to ask Michael Grant, the author of CVX, to discipline your function in the next release of CVX. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the mean time, try IPOPT form COIN-OR, which is a fairly robust general purpose solver for nonlinear programming problems. To make it work, you need to remove the psd constraint by replacing each occurrence of $C$ by $R^TR$, where $R$ is an upper triangular matrix with nonnegative diagonal entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since IPOPT expects smooth constraints, you also need to replace the nonsmooth L1 constraint $\|S−C\|_1\le \alpha$ (assuming the 1-norm to be the column sum norm) by the smooth constraints&#xA;$$-X\le R^TR-C \le X,~~~ e^TX\le \alpha e,$$&#xA;where $e$ is the all-one vector, and $X$ is a matrix of new variables.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2013-03-12T18:05:21.863" LastActivityDate="2013-03-12T18:05:21.863" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5511" PostTypeId="2" ParentId="5508" CreationDate="2013-03-11T12:28:20.360" Score="4" Body="&lt;p&gt;First of all, the formula that you use for the $\theta$-method is correct, but beware that you mix the indices for the time advancing with the indices of the components. Write for example $y^{(i)}_j$ for the component $j$ of the solution at time $t_i$. This can avoid confusions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the system of ODEs that you want to solve is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{y}\prime = f(t,\mathbf{y}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. written in components&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left\{\begin{array}{c} y_1\prime = f_1(t,\mathbf{y}) \\ y_2\prime = f_2(t,\mathbf{y}) \end{array} \right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\mathbf{y}= \left( \begin{array}{c} y_1 \\ y_2 \end{array} \right)$ and $f(t,y) = \left( \begin{array}{c} f_1(t,\mathbf{y}) \\ f_2(t,\mathbf{y}) \end{array} \right)$. As you mention, the scheme is $$\mathbf{y}^{(j+1)} = \mathbf{y}^{(j)} + h \left( \theta f(t_j, \mathbf{y}^{(j)}) + (1-\theta)f(t_{j+1}, \mathbf{y}^{(j+1)})\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is implicit. The fact that it is implicit scheme implies that (in general) you cannot find directly $\mathbf{y}^{(j+1)}$ from $\mathbf{y}^{(j)}$, but you need to solve non-linear equations. Indeed, you can rewrite the scheme as&#xA;$$G(\mathbf{y}^{(j+1)}) =\mathbf{y}^{(j+1)} - \mathbf{y}^{(j)} - h \left( \theta f(t_j, \mathbf{y}^{(j)}) + (1-\theta)f(t_{j+1}, \mathbf{y}^{(j+1)})\right) = \mathbf{0}$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, at each time step of your algorithm, you need to solve this non-linear equation. This can be done for example with the Matlab function &lt;code&gt;fzero&lt;/code&gt; (applied to each component of the equation). That is what you should do in your &lt;code&gt;for&lt;/code&gt; loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the implementation, you can use an anonymous function for &lt;code&gt;f&lt;/code&gt;, it supports vectorial valued functions. Instead of &lt;code&gt;y&lt;/code&gt;, take &lt;code&gt;alpha&lt;/code&gt; as argument. Then, &lt;code&gt;y&lt;/code&gt; is the output of your function (possible with the times to which the approximations correspond).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="3684" LastEditDate="2013-03-11T13:55:18.117" LastActivityDate="2013-03-11T13:55:18.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5512" PostTypeId="2" ParentId="5503" CreationDate="2013-03-11T12:58:19.823" Score="8" Body="&lt;p&gt;I am pretty sure you can model the trace of the squared inverse by $trace(Z)$ where $Z$ and $X$ satisifes $\begin{bmatrix} X &amp;amp; I\\I &amp;amp; S\end{bmatrix} \succeq 0$ and $\begin{bmatrix}Z&amp;amp; X\\X&amp;amp;I\end{bmatrix} \succeq 0$ (Derived using Schur complements, $X$ is used to model/upper bound the inverse of $S$ and $Z$ is used for the square of $X$). Hence, solvable using standard LMI methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following test computes the value for a fixed matrix $S$ (in YALMIP but the CVX version would be similar)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S = randn(3);S = S*S';&#xA;X = sdpvar(3);&#xA;Z = sdpvar(3);&#xA;BoundInverse = [X eye(3);eye(3) S]&amp;gt;=0;&#xA;BoundSquare = [Z X;X eye(3)] &amp;gt;= 0;&#xA;solvesdp([BoundInverse, BoundSquare],trace(Z))&#xA;&#xA;% Is the objective equal to trace(S^-2)&#xA;trace(inv(S)^2)&#xA;double(trace(Z))&#xA;% Is Z equal to S^-2 at optimality&#xA;norm(double(Z) - S^-2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...and to follow up on the comment, here is a model which solves a problem where $S$ is a decision variable ($S$ should be psd, but also close in 1-norm to a given matrix $C$, and the objective is $trace(S)+trace(S^{-2})$)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S = sdpvar(3);&#xA;C = randn(3);&#xA;X = sdpvar(3);&#xA;Z = sdpvar(3);&#xA;Model = [S &amp;gt;= 0, norm(S-C,1) &amp;lt;= 4];&#xA;BoundInverse = [X eye(3);eye(3) S]&amp;gt;=0;&#xA;BoundSquare = [Z X;X eye(3)] &amp;gt;= 0;&#xA;optimize([Model, BoundInverse, BoundSquare],trace(S) + trace(Z))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2018-05-17T06:15:36.173" LastActivityDate="2018-05-17T06:15:36.173" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="5515" PostTypeId="1" AcceptedAnswerId="5517" CreationDate="2013-03-11T20:22:56.723" Score="14" ViewCount="10227" Body="&lt;p&gt;Theoretically, how does the time to do a density functional theory (DFT) calculation scale with the number of electrons? I'm interested in &quot;typical&quot; DFT implementations such as VASP, ABINIT, etc., not O(N) codes.&lt;/p&gt;&#xA;" OwnerUserId="3887" LastActivityDate="2013-09-30T16:47:53.793" Title="How does density functional theory scale with system size?" Tags="|density-functional-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5516" PostTypeId="2" ParentId="4695" CreationDate="2013-03-11T21:01:45.530" Score="1" Body="&lt;p&gt;I am going to answer the second part. While there can be many overlaps between a general computational scientist and a specialized computational X depending on a person's interest, computational scientist's main concern is introduction of new computational techniques to study a problem in a domain field. New thing in the computational technique can be a new computation which was not possible due to lack of appropriate computational tools or a faster way. Computational X, on the other hand, will mostly be interested in using these new tools to study new problems. Thus, in general, a computational scientist will first of all lay down a problem statement, explain general framework to solve it, impose some limitations on the applicability of tool (speed/error etc) and will solve a problem as a proof of concept. &#xA;As for the first part, in my personal opinion if you want your methods to be used, then you should think about how they will perform on computer and therefore you should know programming. &lt;/p&gt;&#xA;" OwnerUserId="3888" LastActivityDate="2013-03-11T21:01:45.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5517" PostTypeId="2" ParentId="5515" CreationDate="2013-03-12T03:36:33.457" Score="19" Body="&lt;p&gt;The simplest correct answer is that DFT is in $O(N_e^3)$.  This comes from the idea that you are ultimately diagonalizing a Hamiltonian with dimension proportional to the number of elections and diagonalization is technically $O(n^3)$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In reality, DFT is a bunch of steps and different steps are rate-limiting in different context.  If we restrict ourselves to plane-wave (PW) DFT (VASP, ABINIT, QE, and others), we can make some stronger statements.  An important idea to understand for PW DFT codes is that the Hamiltonian is never stored as a big matrix; instead, the action of the Hamiltonian operator is computed and used in what are generally `in house' iterative diagonalizers (conjugate gradient, davidson, etc).  These diagonalizers are formally $O(n_e MV)$, where $MV$ is the cost of computing the action of the Hamiltonian, but given their role in an larger self-consistent algorithm, they tend to perform much faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process of computing the action of the Hamiltonian occurs in a couple steps:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Local potential in real space requires an FFT, $O(n_v \ln n_v)$&lt;/li&gt;&#xA;&lt;li&gt;Projection can occur in real or g-space, $O(n_a n_p)$ or $O(n_a n_p n_v)$, respectively&lt;/li&gt;&#xA;&lt;li&gt;The non-local potential is either diagonal or block diagonal, $O(n_a n_p)$ or $O(n_a n_p^2)$, respectively&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;all of this has to happen once per electron (really, wave-function), so add a factor of $n_e$ to all of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Through some means (Gram-Schmidt, for example) the wave-functions (eigenfunctions of the Hamiltonian) must be kept orthogonal to one another, $O(n_e^2 n_v)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the wave-functions need to be composed into an electron-density.  In PW codes, this is accomplished with one last FFT per wave-function (and a sum), $O(n_e n_v \ln n_v)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I've put in a few different $n$'s: $n_v$ is related to the volume (really, it is the basis size), $n_p$ is the number of projectors per atom, $n_a$ is the number of atoms, and $n_e$ the number of electrons.  Formally $n_v$, $n_a$, and $n_e$ are all linearly related to one another ($n_p$ is a small integer), but you could imagine increasing the volume with fixed number of electrons (adding vacuum in slab/wire geometries) or increasing the number of projectors with fixed number of atoms and electrons (using a more accurate pseudo-potential).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is common that problems are FFT-limited, in which case they are effectively $O(n^2 \ln n)$, which is a somewhat common answer in the literature, if not technically correct.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2013-09-30T16:47:53.793" LastActivityDate="2013-09-30T16:47:53.793" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5518" PostTypeId="1" CreationDate="2013-03-12T05:15:33.283" Score="3" ViewCount="624" Body="&lt;p&gt;I debated between asking this question here and in the Academia stack exchange, but decided Computational Science was the most capable of answering the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a new graduate student (~1 semester now) and would really like to make a push to be the best I can in my field.  My undergraduate was a top 5 engineering school.  However, my graduate school is top 50.  So I'll need some really good research to get noticed.  I have an excellent advisor who is very supportive of whatever I'm working on, so I can somewhat guide the &quot;kind&quot; of research I'll be doing further down the road.  And I also have excellent access to any kind of computational resource I need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I'm doing molecular dynamics simulations and am spending time learning about quantum monte carlo on my own.  I also collaborate with an experimental researcher, thus my work has a tie-in to experimental results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been reading journals in this field, but I'm not sure whether &quot;huh, this is an interesting article&quot; matches up at all with what other researchers consider a top-tier contribution.  Again, I'm still new at this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; So, as of early 2013, what are the cutting-edge, hot topic, highly-publishable research areas in computational science?  I really just need a starting point so I don't go off in a rabbit-hole of research that is interesting to me but that nobody else cares about.&lt;/p&gt;&#xA;" OwnerUserId="1997" LastEditorUserId="6663" LastEditDate="2013-12-26T16:46:14.843" LastActivityDate="2013-12-26T16:46:14.843" Title="What is currently the &quot;top&quot; research in computational science?" Tags="|publications|" AnswerCount="2" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="5519" PostTypeId="1" CreationDate="2013-03-12T08:08:52.333" Score="0" ViewCount="300" Body="&lt;p&gt;How do those matrices work? Do I need to multiple every single pixel? How about the upperleft, upperright, bottomleft and bottomleft pixels where there's no surrounding pixel? And does the matrix work from left to right and from up to bottom or from up to bottom first and then left to right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does this kernel (Edge enhance) : &lt;a href=&quot;https://i.stack.imgur.com/d755G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/d755G.png&lt;/a&gt;&#xA;turns into this image: &lt;a href=&quot;https://i.stack.imgur.com/NRdkK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/NRdkK.jpg&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3890" LastActivityDate="2013-03-13T02:49:18.937" Title="How do convolution matrices work?" Tags="|image-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5520" PostTypeId="1" AcceptedAnswerId="5525" CreationDate="2013-03-12T15:17:10.980" Score="4" ViewCount="3293" Body="&lt;p&gt;I'm trying to test a simple 1D Poisson solver to show that a finite difference method converges with $\mathcal{O}(h^2)$ and that using a deferred correction for the input function yields a convergence with $\mathcal{O}(h^4)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the equation is $ - u'' = f  $ with boundary conditions $u(0) = u(1) = 0$. The method I'm trying to use is using the discretized operator $$ A = \left[ \begin{array}{c} 2&amp;amp;-1&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0 \\-1&amp;amp;2&amp;amp;-1&amp;amp;0&amp;amp;0&amp;amp;0 \\ 0&amp;amp;-1&amp;amp;2&amp;amp;-1&amp;amp;0&amp;amp;0 \\ 0&amp;amp;0&amp;amp;-1&amp;amp;2&amp;amp;-1&amp;amp;0 \\ 0&amp;amp;0&amp;amp;0&amp;amp;-1&amp;amp;2&amp;amp;-1 \\ 0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;-1&amp;amp;2\end{array}\right]  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(the example matrix is for $h = \frac{1}{5}$.) Then solve for $Au=h^2f$. I've shown that theoretically this should converge with $\mathcal{O}(h^2)$, but when I test it on Matlab, I'm getting only $\mathcal{O}(h)$ convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I'm trying what my course instructor called &quot;deferred correction&quot;, and altering $f$ before solving. I concluded that the correction should be $f \mapsto f + \frac{h^2}{12} Af$. I've shown that this should converge with $\mathcal{O}(h^4)$, but in Matlab I still get $\mathcal{O}(h)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the Matlab script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [u err] = threeptsolve(ureal, du2, h)&#xA;&#xA;% INPUT: 'ureal' is the function handle for the real solution.&#xA;%        'du2' is the function handle for the second derivative of 'ureal'&#xA;%        'h' is the step size&#xA;% OUTPUT: 'u' is the approximated solution&#xA;%         'err' is the error at each point&#xA;&#xA;x = [0:h:1]';&#xA;n = length(x);&#xA;f = -du2(x);&#xA;realu = ureal(x);&#xA;&#xA;A = 2 * eye(n);&#xA;A = A + diag(-1*ones(n-1,1), 1) + diag(-1*ones(n-1,1), -1);&#xA;A = (1/h^2) * A;&#xA;&#xA;% uncomment if using deferred correction&#xA;% f = f + h^2/12 * A * f;&#xA;&#xA;u = A\f;&#xA;&#xA;err = (realu - u);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I try this with some sample smooth functions (with appropriate boundary values), and then try again with $h/2$, I get a vector of (approximate) twos when I compute &lt;code&gt;err1 ./ err2(1:2:end)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my math wrong, or is it my code?&lt;/p&gt;&#xA;" OwnerUserId="3892" LastEditorUserId="3892" LastEditDate="2013-03-12T15:24:18.180" LastActivityDate="2016-11-06T16:25:25.030" Title="Testing 1D Poisson Solver" Tags="|matlab|finite-difference|poisson|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5521" PostTypeId="1" AcceptedAnswerId="5523" CreationDate="2013-03-12T15:17:49.997" Score="2" ViewCount="530" Body="&lt;p&gt;So, recently, I have found myself in the position of having to implement a combinations function in MATLAB. What I mean by this is the following: I simply need to list all possible combinations for an ${m \choose n}$ problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More concretely, lets say I am given an $m=4$ length vector, where its indicies increase from $1$ to $m$, (e.g. $[1, 2, 3, 4]$). Let us also say that in this example, $n = 2$. Thus, I want to find all ${4 \choose 2} = 6$ possible combinations, and I would like the routine to therefore spit out: $$\begin{bmatrix}1 &amp;amp; 2 \\1  &amp;amp; 3 \\ 1&amp;amp;4 \\2&amp;amp;3\\2&amp;amp;4\\3&amp;amp;4 \end{bmatrix} $$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The reason I need a list like that is because those numbers are going to be used as indicies of another matrix, but thats not important). Now, Ive googled around, and MATLAB has a function to do this, but I do not have it, as I dont have the stats toolbox. Be that as it may, I suppose I could sook something up, but I do not think that it would be too efficient, and worse still, it might not be generalizable for any $m$ or $n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I therefore would not mind writing my own routine, if it is not too involved, but I would like some help as to how this can be accomplished. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1457" LastActivityDate="2013-03-12T20:01:12.343" Title="Trying to implement a simple/efficient combinations function in MATLAB" Tags="|matrix|matlab|efficiency|combinatorics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5522" PostTypeId="1" AcceptedAnswerId="5524" CreationDate="2013-03-12T15:57:46.140" Score="3" ViewCount="156" Body="&lt;p&gt;I'm building an application where I need to compare found data with the actual data it should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 5 sets of data, each with 3 variables a,b,c.&#xA;Let matrix A be a 3x1 matrix with data a,b,c which represents the actual data.&#xA;Let matrix B be a 3x3 matrix with unknown data that needs to be calculated.&#xA;Let matrix C be a 3x1 matrix with found data a,b,c.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation A = B.C needs to be solved so that it gives the lowest difference for the 5 datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I start with solving this? Do I just have to start with some numbers in B and try new ones each iteration?&lt;/p&gt;&#xA;" OwnerUserId="3893" LastActivityDate="2013-03-12T22:08:06.013" Title="Calculate 3x3 matrix to give lowest difference for data set" Tags="|matrix-equations|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5523" PostTypeId="2" ParentId="5521" CreationDate="2013-03-12T16:09:05.320" Score="2" Body="&lt;p&gt;You can find an implementation in &lt;a href=&quot;http://www.gnu.org/software/octave/&quot; rel=&quot;nofollow&quot;&gt;Octave&lt;/a&gt;. The function is, as in Matlab, called &lt;code&gt;nchoosek&lt;/code&gt;. Documentation is &lt;a href=&quot;http://octave.sourceforge.net/octave/function/nchoosek.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. For the theory behind this type of algorithms, you can consult section 7.2.1 of Volume 4A of The Art Of Computer Programming by Donald Knuth.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-03-12T20:01:12.343" LastActivityDate="2013-03-12T20:01:12.343" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5524" PostTypeId="2" ParentId="5522" CreationDate="2013-03-12T16:26:21.310" Score="6" Body="&lt;p&gt;Let me see if I can restate the problem to answer your question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have 5 sets of data: $y_{i}$, $x_{i}$ such that for each $i = 1, \ldots, 5$, $x_{i}$ and $y_{i}$ are both 3 by 1 vectors. You also have a 3 by 3 matrix $B$ that is supposed to relate $x_{i}$ to $y_{i}$ via:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_{i} = Bx_{i} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this formulation correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, the proper methods for solving this sort of problem use &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_regression&quot; rel=&quot;nofollow&quot;&gt;linear regression&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One linear regression method is &lt;a href=&quot;https://en.wikipedia.org/wiki/Ordinary_least_squares&quot; rel=&quot;nofollow&quot;&gt;ordinary least squares&lt;/a&gt;(OLS). OLS will find a solution to the overdetermined set of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[y_{1}, \ldots, y_{5}] = B \cdot [x_{1}, \ldots, x_{5}]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with minimum residual; that is, it minimizes the sum of squared distances between the data $y_{i}$ and the predictions $Bx_{i}$, hence &quot;least squares&quot;. This sum of squared distances is $\sum_{i=1}^{5}\|y_{i} - Bx_{i}\|^{2}$ (the norm used is a 2-norm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convenience, define the 3 by 5 matrices $X$ and $Y$ by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Y = [y_{1}, \ldots, y_{5}],$$&#xA;$$X = [x_{1}, \ldots, x_{5}],$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so we are finding a solution to the overdetermined system of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Y = BX$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with minimum sum-of-squared distances. This next part is important:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The notation I've used to describe your problem is the transpose of what the Wikipedia article on ordinary least squares uses.&lt;/em&gt; So if you try to compare everything, you're going to have to transpose every matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using some algebra, the value of $B$ that minimizes the sum of squared distances is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B = (XX^{T})^{-1}XY^{T};$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can check the dimensions of each matrix to see that $B$ is 3 by 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do not calculate $B$ this way.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculating matrix inverses directly is usually bad for a variety of reasons; why it is bad is out of the scope of the question. If you need a matrix inverse, it's usually better to solve the system of equations related to that inverse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better way to calculate $B$ would be to solve the related linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$XX^{T}B = XY^{T}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This system of equations is called the &lt;em&gt;normal equations&lt;/em&gt; (plural, because the system of equations is referred to as a group).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do not calculate $B$ using the normal equations either.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the normal equations is not usually a good idea because this system of equations can be ill-conditioned, which will degrade the accuracy of $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An even better way to calculate $B$ would be to use QR decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $X^{T} = [Q_{1}, Q_{2}]([R_{1}, 0])^{T}$, obtained from a QR decomposition. Using the methods outlined &lt;a href=&quot;http://www.cs.cornell.edu/~bindel/class/cs3220-s12/notes/lec10.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B = R_{1}^{-1}Q_{1}^{T}Y^{T},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so I would solve the linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R_{1}B = Q_{1}^{T}Y^{T}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last method should give you what you want (assuming I didn't make any mistakes during my lunch break). &lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-03-12T22:08:06.013" LastActivityDate="2013-03-12T22:08:06.013" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5525" PostTypeId="2" ParentId="5520" CreationDate="2013-03-12T18:09:20.903" Score="5" Body="&lt;p&gt;Your formulation of $A$ assumes that $u_0$ and $u_{n+1}$ are zero, which is correct.  However, you are then including your boundaries at $u_1$ and $u_n$.  Your exact answer satisfies these later BC's but not the imposed BC's in $A$.  These discontinuities floating around cause you to lose your higher order accuracy. To get second order convergence you only need to change one line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = [0:h:1]';&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = [h:h:1-h]';&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I get a column of 4's (for 2nd order conv.) when I execute &lt;code&gt;err1 ./ err2(2:2:end-1)&lt;/code&gt; (Note the shift by 1 index since the solutions now line up at the even indices rather than the odd).  I have not gotten 4th order yet from your &quot;deferred correction&quot;, however this solve a part of your problem.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-03-12T18:09:20.903" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5526" PostTypeId="1" CreationDate="2013-03-12T23:44:06.797" Score="3" ViewCount="144" Body="&lt;p&gt;Are there any open source libraries available to mutate PDB protein residues? If so are do you have any examples of them?&lt;/p&gt;&#xA;" OwnerUserId="3896" LastEditorUserId="20688" LastEditDate="2018-04-06T05:36:04.430" LastActivityDate="2018-07-05T12:43:50.207" Title="C++ library to mutate pdb residues" Tags="|c++|software|computational-chemistry|pdb|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5527" PostTypeId="2" ParentId="5519" CreationDate="2013-03-13T02:49:18.937" Score="3" Body="&lt;p&gt;You might want to read &lt;a href=&quot;http://lodev.org/cgtutor/filtering.html&quot; rel=&quot;nofollow&quot;&gt;this tutorial&lt;/a&gt;, but I'll also try to answer below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do 2D convolutions work?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find it useful to think of 2D convolutions as image filters, as you have.  The idea is that you use the filter to take an old grid of values, the numerical representations of colors, and turn it into a new grid.  This is done by laying the filter on top of the old grid, multiplying the numbers in the filter with the numbers in the grid, summing them, and putting them into the new grid at the center of the filter.  More formally, for a $2n+1$ by $2m+1$ filter $F$:&#xA;$$ A'_{x,y} = \sum_{i=-n}^n ~ \sum_{j=-m}^m (F_{i,j}) (A_{x+i,y+j})$$&#xA;For example, a 3x3 edge-highlighting filter:&#xA;$$ \text{Old} = \begin{bmatrix} 0 &amp;amp; 1 &amp;amp; 2 &amp;amp; 1 &amp;amp; 0 \\ 1 &amp;amp; 2 &amp;amp; 3 &amp;amp; 2 &amp;amp; 1 \\ 2 &amp;amp; 3 &amp;amp; 4 &amp;amp; 3 &amp;amp; 2 \\ 1 &amp;amp; 2 &amp;amp; 3 &amp;amp; 2 &amp;amp; 1 \\ 0 &amp;amp; 1 &amp;amp; 2 &amp;amp; 1 &amp;amp; 0 \end{bmatrix}, \text{Filter} = \begin{bmatrix} 0 &amp;amp; -1 &amp;amp; 0 \\ -1 &amp;amp; 5 &amp;amp; -1 \\ 0 &amp;amp; -1 &amp;amp; 0 \end{bmatrix} \quad \rightarrow \text{New} =  \begin{bmatrix} ? &amp;amp; ? &amp;amp; ? &amp;amp; ? &amp;amp; ? \\ ? &amp;amp; 2 &amp;amp; 5 &amp;amp; 2 &amp;amp; ? \\ ? &amp;amp; 5 &amp;amp; 8 &amp;amp; 5 &amp;amp; ? \\ ? &amp;amp; 2 &amp;amp; 5 &amp;amp; 2 &amp;amp; ? \\ ? &amp;amp; ? &amp;amp; ? &amp;amp; ? &amp;amp; ? \end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do I need to multiple every single pixel?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You always want to apply the filter to the entire inside of the grid.  In order to do the borders, you need some sort of boundary condition.  Common boundary conditions are cyclic, the bottom connects to the top and the left to the right; mirror, add an extra row above that is the same as the top row, below that is the same as the bottom row, etc; or constant, where the border is some fixed pattern, often zero.  For images, the border is generally really small compared to the inside, so you probably don't care if you just neglect it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does the matrix work from left to right and from up to bottom or from up to bottom first and then left to right?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The filter should always be applied to the old grid, which doesn't change.  Therefore, it doesn't matter in what order you apply it; you can start from whichever pixel you'd like and go in any direction.  This is good, because it makes the task embarrassingly parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why does edge enhance work?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The filter:&#xA;$$ F = \begin{bmatrix} 0 &amp;amp; 0 &amp;amp; 0 \\ -1 &amp;amp; 1 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 0 \end{bmatrix} $$&#xA;picks out vertical edges.  It is easiest to see its action on a line:&#xA;$$ \begin{bmatrix} 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \end{bmatrix} \quad \longrightarrow_F \begin{bmatrix} ? &amp;amp; 0 &amp;amp; 1 &amp;amp; -1 &amp;amp; 0 \\ ? &amp;amp; 0 &amp;amp; 1 &amp;amp; -1 &amp;amp; 0 \\ ? &amp;amp; 0 &amp;amp; 1 &amp;amp; -1 &amp;amp; 0 \\ ? &amp;amp; 0 &amp;amp; 1 &amp;amp; -1 &amp;amp; 0 \\ ? &amp;amp; 0 &amp;amp; 1 &amp;amp; -1 &amp;amp; 0 \end{bmatrix}$$&#xA;It is actually computing a directional derivative, $\partial/\partial x$, which picks out slopes along horizontal lines, such as vertical edges.&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2013-03-13T02:49:18.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5531" PostTypeId="1" CreationDate="2013-03-13T04:31:35.233" Score="4" ViewCount="1709" Body="&lt;p&gt;I am trying to fit a data set to an exponential model using scipy. However, the covariance matrix that is returned is 'inf' and I receive the following error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Traceback (most recent call last):&#xA;    File &quot;C:\Users\Christopher\Desktop\advancedLab\03_ML_Mogni\analyzeTHIS.py&quot;, line 21, in &#xA;      print 'A = ', ans[0], '+/-', cov[0][0]&#xA;  TypeError: 'float' object has no attribute '__getitem__'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now, I don't understand why my covariance matrix comes back as infinite... I know that is the source of this error message. My data set has times, which are between 400 and 20000 units, probabilities, which are between 0.005 and 0.3 units, and their respective standard deviations. Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;import scipy.optimize, scipy.stats.stats&#xA;import pylab&#xA;&#xA;use = raw_input('File: ')&#xA;&#xA;time = numpy.loadtxt(use + 'time.txt', unpack = 'True')&#xA;prob = numpy.loadtxt(use + 'prob.txt', unpack = 'True')&#xA;std = numpy.loadtxt(use + 'std.txt', unpack = 'True')&#xA;stdProb = numpy.loadtxt(use + 'stdProb.txt', unpack = 'True')&#xA;&#xA;guess = prob&#xA;guess.tolist()&#xA;&#xA;def muon_model(x,A):&#xA;    return A*numpy.exp(-A*x)&#xA;&#xA;ans, cov = scipy.optimize.curve_fit(muon_model, time, prob, p0 = 2200, sigma = stdProb, maxfev = 10000)&#xA;&#xA;print cov&#xA;print 'A = ', ans[0], '+/-', cov[0][0]&#xA;&#xA;numpy.savetxt(use + '_counts_best_fit.txt', ans)&#xA;numpy.savetxt(use + '_counts_cov.txt', cov)&#xA;&#xA;x = numpy.linspace(0,22000,500)&#xA;y = ans[0]*numpy.exp(-ans[0]*x)&#xA;&#xA;model = muon_model(time,ans[0])&#xA;pylab.plot(x,y,label='Model')&#xA;pylab.errorbar(time, prob, xerr = std, yerr = stdProb, label = 'Data', fmt = 'o')&#xA;pylab.legend()&#xA;pylab.xlabel('Time (nanoseconds)')&#xA;pylab.ylabel('Probability')&#xA;pylab.savefig(use + '_counts_with_fit.png')&#xA;pylab.clf()&#xA;&#xA;mod = []&#xA;ind = 0&#xA;while ind &amp;lt; len(time):&#xA;    mod.append(math.exp(-1000*ans[0]*ind) - math.exp(-1000*ans[0]*(ind + 1)))&#xA;residual = prob - mod&#xA;&#xA;chisq = (residual**2/std**2).sum()&#xA;dof = len(counts)-len(ans)&#xA;print 'Chi-Sq = ', chisq&#xA;print 'dof = ', dof&#xA;print 'Reduced Chi-Sq = ', chisq/dof&#xA;print 'PTE = ', scipy.stats.stats.chisqprob(chisq,dof)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;I changed my guess parameter to 0.5, because I put the wrong guess in. I still ended up with the same answer. Unfortunately, my data points are not equally spaced. What I did was bin my times in 1000 unit intervals, and took my data point in that bin to be the average value of all the values in that bin. I am still getting the same error no matter which guess I use.&lt;/p&gt;&#xA;" OwnerUserId="3899" LastEditorUserId="1552" LastEditDate="2013-03-13T17:35:49.230" LastActivityDate="2013-05-12T18:20:21.877" Title="TypeError from scipy.optimize.curve_fit" Tags="|python|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5532" PostTypeId="2" ParentId="5531" CreationDate="2013-03-13T07:16:10.217" Score="1" Body="&lt;p&gt;welcome to SciComp. First of all, if you need purely exponential fitting and have equidistant data, &lt;a href=&quot;http://www.statsci.org/other/prony.html&quot; rel=&quot;nofollow&quot;&gt;Prony's&lt;/a&gt; method (see &lt;a href=&quot;http://octave.sourceforge.net/optim/function/expfit.html&quot; rel=&quot;nofollow&quot;&gt;expfit&lt;/a&gt; for an implementation in Octave which can easily be ported to Python) is much more appropriate (and numerically stable). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, if I look at your initial guess for the parameter (2200) and you say that you have time spanning from 0 to 22000, your exponential model will give zeros a whole lot of the time (in double precision). And this will lead to numerical trouble. Are you sure about this parameter (no scaling error)? If this is not the issue, can you make (a smaller version of) your data set available somewhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I am the original author of Octave's expfit function (but it has been improved by others later on).&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-03-13T08:45:53.410" LastActivityDate="2013-03-13T08:45:53.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5533" PostTypeId="2" ParentId="5518" CreationDate="2013-03-13T11:14:42.363" Score="7" Body="&lt;p&gt;First, an attempt to just answer the question.  My (personal and subjective) feeling is that many jobs and funding opportunities right now are tied to applications in biology and energy, and that the hot research topics are in &quot;multi-scale&quot; and &quot;multi-physics&quot; phenomena, as well as uncertainty quantification.  However, in your case you need to know what will be important in 5 years or so, which is much more difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, some ideas of where to go to find your own answers to the question.  You could browse some listings at &lt;a href=&quot;http://www.mathjobs.org&quot;&gt;mathjobs.org&lt;/a&gt; for an idea of what kind of specializations employers are looking for, and &lt;a href=&quot;http://www.nsf.gov&quot;&gt;nsf.gov&lt;/a&gt; to get an idea of the grants and funding opportunities that are being offered.  You could also browse some recent issues of &lt;a href=&quot;http://epubs.siam.org/journal/siread&quot;&gt;SIAM Review&lt;/a&gt; to try to see what research is attracting attention.  Big conferences (for example the &lt;a href=&quot;http://meetings.siam.org/program.cfm?CONFCODE=an12&quot;&gt;2012 SIAM annual meeting&lt;/a&gt;) often have two type of invited talks: established researchers surveying a career worth of contributions, and young rising stars talking about their exciting new discoveries.  The latter are working in areas that are &quot;hot&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, some advice not directly answering the question.  We do not know what the hot topics will be in 5 years, and we don't know what the funding landscape and job market will look like, and most people who expect a research-heavy academic job do not get one.  I recommend working hard on some research that is interesting to you, building teaching experience, and at least considering what will happen if you need to look for a job outside academia.  My recent experiences on the job market depended much more on my teaching experience and my programming experience than it did on whether I was in a hot research area.&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2013-03-13T11:14:42.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5534" PostTypeId="1" CreationDate="2013-03-13T11:34:56.190" Score="13" ViewCount="1046" Body="&lt;p&gt;I have a dataset running into millions of data points in 3D. For the calculation I am doing, I need to calculate neighbor (range search) to each data point in a radius, try to fit a function, calculate the error for the fit, repeat this for next datapoint and so on. My code works properly but it is taking very long time to run, around 1 second per datapoint! It is probably because for each point, it has to search in whole dataset. Is there a way I can make the process fast. I have an idea that if I can somehow establish some adjacency relationship between first neighbors, then this can be less slow. If it helps, I am trying to find optimum Parzen window width in 3D.  &lt;/p&gt;&#xA;" OwnerUserId="3888" LastEditorUserId="127" LastEditDate="2013-05-09T02:15:38.830" LastActivityDate="2013-08-29T14:25:48.103" Title="Repeated nearest neighbor calculation for millions of data points too slow" Tags="|algorithms|data-structures|nearest-neighbors|spatial-data|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5535" PostTypeId="2" ParentId="5534" CreationDate="2013-03-13T11:57:19.730" Score="3" Body="&lt;p&gt;You should probably consider building the &lt;a href=&quot;http://en.wikipedia.org/wiki/Delaunay_triangulation&quot; rel=&quot;nofollow&quot;&gt;Delaunay triangulation&lt;/a&gt; (well, its 3D analogue). In 2D, that's a special triangulation of the data points that always contains the nearest neighbor. The same holds in 3D, but with tetrahedra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can build once and for all the triangulation, and then search for the nearest neighbor directly in the triangulation. I think that there are some good algorithms to build the triangulation: in 2D, the construction of the triangulation is in $n \log(n)$ and the later searches for the nearest neighbor is linear in the number of data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="127" LastEditDate="2013-05-09T02:17:06.273" LastActivityDate="2013-05-09T02:17:06.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5536" PostTypeId="2" ParentId="5534" CreationDate="2013-03-13T13:54:34.207" Score="9" Body="&lt;p&gt;I'd suggesting googling for bounding volume hierarchies (BSP tree in particular). Given your point cloud, you can find a plane that splits it into two equal subclouds. Then when you need to find the collection of points that are within some radius R of a test point, you can first compare your test point to that plane, and if it's height above it is more than R, then the entire subcloud below the plane must also be farther away than R too (so you don't need to check any of those points). You can apply this idea recursively too, ultimately yielding n log n type complexities instead of n-squared. (This is BSP / binary space partitioning, but the other kinds of bounding volume hierarchies are conceptually similar - recursively clustering your points in such a way that you can quickly cull away large subcollections with a single test of your geometrical predicate).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2013-03-13T13:54:34.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5537" PostTypeId="1" CreationDate="2013-03-13T14:58:44.910" Score="1" ViewCount="312" Body="&lt;p&gt;I am trying to solve a (large) system of ODEs with GSL solvers. When I use driver method I get an error message of &lt;code&gt;could not allocate space for gsl_interp_accel&lt;/code&gt;, when I define control, error and stepper manually, I get &lt;code&gt;bad_alloc&lt;/code&gt; exception, which is caused, as far as I understand it, by the same thing that causes &lt;code&gt;could not allocate space for gsl_interp_accel&lt;/code&gt; in the other case - the lack of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have consulted with other &lt;code&gt;bad_alloc&lt;/code&gt; queries, such as &lt;a href=&quot;https://stackoverflow.com/questions/8378797/stdbad-alloc-am-i-using-too-much-memory&quot;&gt;this one&lt;/a&gt;, but I haven't found anything useful for my particular case there. Also, I have tried other ODE solvers, but they also end up with memory errors. I have also checked my program with valgrind to make sure there are no memory errors/leaks anywhere else apart from the solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any solver has &quot;limits of integration&quot;, and in my case program works fine for about 10% of the upper limit (which is large comparing to lower limit - I am pretty sure this is the source of errors I get - but I do need to integrate between these particular limits) and then terminates with one of the exceptions I've quoted above. I have tried various (fixed/adaptive) step size, but never reached more than 10% of what I want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The piece of code that gives an exception is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gsl_ode_struct inputstruct;  // Struct that contains parameters for ODEs &#xA;gsl_odeiv2_system sys = {func, NULL, 2*G.getsize(), &amp;amp;inputstruct};&#xA;const gsl_odeiv2_step_type * T = gsl_odeiv2_step_rk8pd;&#xA;gsl_odeiv2_step * stepper = gsl_odeiv2_step_alloc (T, size_of_the_system);&#xA;gsl_odeiv2_control * control = gsl_odeiv2_control_standard_new (1.e-6, 0.0, 1., 0.);&#xA;gsl_odeiv2_evolve * error = gsl_odeiv2_evolve_alloc (size_of_the_system);&#xA;double hparam = 1.e-6; // Initial step size&#xA;double t = xspan[0]; // Initial time&#xA;while(t &amp;lt; final_time){&#xA;    // Here the excpection comes&#xA;    int status = gsl_odeiv2_evolve_apply (error, control, stepper, &amp;amp;sys, &amp;amp;t, final_time, &amp;amp;hparam, array);&#xA;    if(status != GSL_SUCCESS)&#xA;        break;&#xA;    // Do some stuff that includes reading all the intermediate results to a container as I need them later.&#xA;    }&#xA;    gsl_odeiv2_evolve_free (error);&#xA;    gsl_odeiv2_control_free (control);&#xA;    gsl_odeiv2_step_free (stepper);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So if I change &lt;code&gt;final_time&lt;/code&gt; to &lt;code&gt;final_time/10&lt;/code&gt; code executes, but the result then does not make any sense. Even when nothing is done after solver, exception is still thrown, &lt;code&gt;could not allocate space for gsl_interp_accel&lt;/code&gt;, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to split the loop on several (many) loops with erasing memory in between, but this didn't help much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case this is important, I use Ubuntu 12.10, compiled with GNU compiler and Intel C++ Composer. Also tested on Mac (don't know which version of OS) with the same result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is: is there any way to &quot;cheat&quot; on the solver and make the program work properly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.: ODEint solver, that has smarter way of getting intermediate results, also throws an exception.&lt;/p&gt;&#xA;" OwnerUserId="3854" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-04-22T17:21:23.080" Title="Memory allocation error with GSL ODE solver applied to system of 4 ODEs" Tags="|linear-solver|c++|ode|memory-management|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5538" PostTypeId="2" ParentId="5520" CreationDate="2013-03-13T15:40:47.627" Score="1" Body="&lt;p&gt;For defect correction you will need a 4th-order discretization. Let $A_2 u = f_2$ be your second-order discretization and let $A_4 u = f_4 $ be a 4th-order discretization. Defect corrections then become&#xA;\begin{equation}&#xA;A_2 u^{(0)} = f_2 \\&#xA;A_2 u^{(k)} = f_4 - A_4 u^{(k-1)} + A_2 u^{(k-1)} \quad k=1,2,...&#xA;\end{equation}&#xA;As the defect corrections converge $u^{(k)} \approx u^{(k-1)}$ so the $A_2 u$ terms cancel and you have solved the 4th-order discretization without ever &quot;inverting&quot; $A_4$. There is ample theory on how many iterations are needed, see for example Hackbush, Multi-grid methods and applications, Springer 1985.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-03-13T15:40:47.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5541" PostTypeId="1" CreationDate="2013-03-13T19:57:25.860" Score="7" ViewCount="182" Body="&lt;p&gt;What are some of the state of the art models for climate simulation?  I'm looking for downloadable source codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in meteorological research I'm personally familiar with WRF, the Weather Research Forecasting model developed by NCAR (among others).  What is an equivalent code for long range forecasts?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Follow-up question - how much code is shared (if any) between meteorological and climate simulation codes?  I understand that the models must differ for efficiency and conservation issues but certainly the problems must overlap a bit.&lt;/p&gt;&#xA;" OwnerUserId="19" LastActivityDate="2013-08-03T09:28:13.100" Title="State of the art climate models" Tags="|model|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5542" PostTypeId="2" ParentId="5534" CreationDate="2013-03-13T20:49:29.090" Score="7" Body="&lt;p&gt;There are several data structure for storing data that preserves information about position and proximity; there by allowing fast nearest neighbor(s) determination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular &lt;a href=&quot;http://en.wikipedia.org/wiki/R-tree&quot; rel=&quot;nofollow&quot;&gt;R-trees&lt;/a&gt; (and specialized forms like &lt;a href=&quot;http://en.wikipedia.org/wiki/R%2a_tree&quot; rel=&quot;nofollow&quot;&gt;R*-trees&lt;/a&gt;) and &lt;a href=&quot;http://en.wikipedia.org/wiki/X-tree&quot; rel=&quot;nofollow&quot;&gt;X-trees&lt;/a&gt;. Lots of choices that are optimized for slightly different uses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choosing a R*-tree rather than a naive nearest neighbor look-up was a big part of my getting a factor of 10000 speedup out of a particular code. (OK, maybe a few hundred of that was the R*-tree, most of the rest was because the naive look-up had been badly coded so that it smashed the cache. &lt;em&gt;::sigh::&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These structures have typical $O(N \log N)$ ($N$ the number of point stored) insertion performance and storage requirement and $O(\log N)$ look-up performance, so they work well if you will be doing many look ups (say one for each point as in DBSCAN); however some of them have very bad worst-case performance.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="183" LastEditDate="2013-05-08T18:30:07.533" LastActivityDate="2013-05-08T18:30:07.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5544" PostTypeId="2" ParentId="5541" CreationDate="2013-03-13T21:21:58.587" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;http://www.cesm.ucar.edu/&quot; rel=&quot;nofollow&quot;&gt;Community Earth System Model&lt;/a&gt; gets a lot of use.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-03-13T21:21:58.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5545" PostTypeId="1" CreationDate="2013-03-13T22:18:19.320" Score="12" ViewCount="1810" Body="&lt;p&gt;I have an inhomogeneous linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    Ax=b&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is a real $n\times n$ matrix with $n\leq 4$. The nullspace of $A$ is guaranteed to be of zero dimension so the equation has a unique inverse $x=A^{-1} b$. Since the result enters the right hand side of an ODE, which I intend to solve using an adaptive method, it is important that the solution is smooth with respect to small variations of the elements of $A$ and $b$. Because of this requirement and the small dimensionality I thought to implement explicit formulas for $A^{-1} b$. The elements can be exactly zero or take very different values. My question is if this makes sense to you and if there are known stable expressions for this. I am coding in C for x86 systems.                                                                                     &lt;/p&gt;&#xA;" OwnerUserId="1943" LastEditorUserId="1943" LastEditDate="2013-03-14T11:41:14.547" LastActivityDate="2013-03-14T16:32:23.007" Title="Numerically stable explicit solution of small linear system" Tags="|linear-algebra|ode|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5548" PostTypeId="2" ParentId="5545" CreationDate="2013-03-14T01:07:48.120" Score="-2" Body="&lt;p&gt;Not sure that can help but i just think when you talk about stable solution, you're talking about approximation methods. When you compute things explicitle, stability doesn't have sens. That say you have to accept an approximative solution if you want to gain on stabilities.&lt;/p&gt;&#xA;" OwnerUserId="3488" LastActivityDate="2013-03-14T01:07:48.120" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5549" PostTypeId="1" AcceptedAnswerId="11451" CreationDate="2013-03-14T02:11:38.027" Score="21" ViewCount="2719" Body="&lt;p&gt;How do Python/Numpy arrays scale with increasing array dimensions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is based on some behaviour I noticed while benchmarking Python code for this question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5473/how-to-express-this-complicated-expression-using-numpy-slices&quot;&gt;How to express this complicated expression using numpy slices&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem mostly involved indexing to populate an array. I found that the advantages of using (not-very-good) Cython and Numpy versions over a Python loop varied depending on the size of the arrays involved. Both Numpy and Cython experience an increasing performance advantage up to a point (somewhere broadly around $N=500$ for Cython and $N=2000$ for Numpy on my laptop), after which their advantages declined (the Cython function remained the fastest).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this hardware defined? In terms of working with large arrays, what are best practices that one should adhere to for code where performance is appreciated?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/S98G9.png&quot; alt=&quot;Plot of execution time relative to looped code for vectorized and Cython implementations&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/1727/why-isnt-my-matrix-vector-multiplication-scaling/3109#3109&quot;&gt;Why isn&amp;#39;t my Matrix-Vector Multiplication Scaling?&lt;/a&gt;) may be related, but I am interested in knowing more about how different ways of treating arrays in Python scale relative to each other.&lt;/p&gt;&#xA;" OwnerUserId="2015" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-04-23T14:02:28.017" Title="How does the performance of Python/Numpy array operations scale with increasing array dimensions?" Tags="|python|performance|numpy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5550" PostTypeId="2" ParentId="5545" CreationDate="2013-03-14T04:33:47.063" Score="6" Body="&lt;p&gt;The only explicit inverse result I know of is &lt;a href=&quot;https://en.wikipedia.org/wiki/Cramer%27s_rule&quot; rel=&quot;nofollow&quot;&gt;Cramer's Rule&lt;/a&gt;, which has been recently shown to be computable in $\mathcal{O}(n^{3})$ time (like Gaussian elimination; unsure of the constant in front of the leading factor, though).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix inverse of $A$ is a smooth function of $A$ so long as $\det(A) \neq 0$, and the solution $x$ is certainly a smooth function of $b$, so as long as the right-hand side of the ODE is a smooth function of $x$ and you avoid cases where $A$ is rank-deficient, I'd think your right-hand side would be smooth. (Here, I take smooth to mean &quot;at least twice continuously differentiable&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be safe, it's probably best to make sure that $A$ is not numerically rank-deficient either (i.e., does not have small singular values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with Cramer's Rule is that its stability properties are unknown except for $n = 2$ (which is forward stable, but not backward stable). (See &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;, 2nd edition, by N. Higham.) It is not considered a reliable algorithm; Gaussian Elimination with Partial Pivoting (GEPP) is favored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect the problem with using BLAS+LAPACK to carry out GEPP in an ODE solve would be any finite differencing used in an implicit ODE method. I know that people have solved linear programs as part of a right-hand side evaluation, and because they did so naïvely (just plugged the linear program solve into the right hand side, calling a simplex algorithm), they greatly reduced the accuracy of their computed solution &lt;em&gt;and&lt;/em&gt; substantially increased the time it took to solve the problem. A labmate of mine figured out how to solve such problems in a much more efficient, accurate manner; I'll have to look to see if his publication has been released yet. You may have a similar problem regardless of whether you opt to use GEPP or Cramer's Rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there's any way you can calculate an analytical Jacobian matrix for your problem, you may wish to do that to save yourself some numerical headaches. It will be cheaper to evaluate, and probably more accurate, than a finite difference approximation. Expressions for the derivative of the matrix inverse can be found &lt;a href=&quot;https://en.wikipedia.org/wiki/Invertible_matrix#Derivative_of_the_matrix_inverse&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; if you need them. Evaluating the derivative of the matrix inverse looks like it would take at least two or three linear system solves, but they would all be with the same matrix and different right-hand sides, so it would not be considerably more expensive than a single linear system solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if there's any way you can compare your computed solution to a solution with known parameter values, I would do that, so that you can diagnose whether you've encountered any of these numerical pitfalls.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-03-14T16:32:23.007" LastActivityDate="2013-03-14T16:32:23.007" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5551" PostTypeId="1" AcceptedAnswerId="5552" CreationDate="2013-03-14T04:34:47.490" Score="0" ViewCount="57" Body="&lt;p&gt;I was reading this paper related to Multiclass Classiﬁcation with&#xA;Multi-Prototype Support Vector Machines - &lt;a href=&quot;http://www.math.unipd.it/~aiolli/PAPERS/aiolli05a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am having difficulty in understanding why they have mentioned the following problem non convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am really struggling figuring out the convexity/concavity of the functions. Can anyone just skim through the paper to help me out. The paper is about SVM (multi class classification) where each class can have multiple weight vectors/prototypes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TYyNo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3871" LastActivityDate="2013-03-14T07:14:58.020" Title="Confusion related to convexity of a problem" Tags="|machine-learning|support-vector-machines|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5552" PostTypeId="2" ParentId="5551" CreationDate="2013-03-14T07:14:58.020" Score="3" Body="&lt;p&gt;Sounds too simple so maybe I've misunderstood you, but they have binary variables, hence obviously a nonconvex problem (a binary variable is either 0 or 1, which is a nonconvex set)&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-03-14T07:14:58.020" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5553" PostTypeId="2" ParentId="5545" CreationDate="2013-03-14T10:25:18.230" Score="7" Body="&lt;p&gt;Before implementing explicit formulas, I would ask myself the question: &quot;is it worth it?&quot;: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it worth to spend the time writing, debugging and validating these explicit formulas while you could easily link to BLAS+LAPACK that use the classical Gaussian elimination? &lt;/li&gt;&#xA;&lt;li&gt;Do you expect to gain stability? I don't think programming explicit formulas (like Cramer's rule) will give you better stability, on the contrary. &lt;/li&gt;&#xA;&lt;li&gt;Do you expect to gain speed? Did you already profile your whole program? What fraction of time is spent in solving hese 4x4 systems?&lt;/li&gt;&#xA;&lt;li&gt;What is the probability that in one year's time, you improve your model and you need 5 equations instead of 4?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My advice: use the BLAS/LAPACK combination first, see if it works, profile the whole program, ask a student to implement explicit formulas (sorry, being sarcastic here) and make a comparison on speed and robustness.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-03-14T10:25:18.230" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5554" PostTypeId="1" CreationDate="2013-03-14T14:12:08.277" Score="1" ViewCount="2570" Body="&lt;p&gt;Is it possible to calculate the color frequency of a pixel?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, I get a pixel that is red but where this red sits on the spectrum in hertz?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this even possible to do even in a limited way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks.&lt;/p&gt;&#xA;" OwnerUserId="3913" LastActivityDate="2013-03-25T01:50:05.793" Title="Color frequency of a pixel" Tags="|algorithms|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5555" PostTypeId="2" ParentId="5554" CreationDate="2013-03-14T15:12:42.210" Score="1" Body="&lt;p&gt;Each pixel on a typical computer monitor is actually made up of 3 subpixels that are of a single, fixed frequency. The control circuitry of the monitor adjusts the brightness of these three subpixels in order to feed light to your eye that your brain interprets as lying somewhere on the visible light spectrum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answers to nearly the same questions are available &lt;a href=&quot;https://stackoverflow.com/questions/5817474/how-to-get-the-wavelength-of-a-pixel-using-rgb&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-03-14T15:12:42.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5556" PostTypeId="2" ParentId="5518" CreationDate="2013-03-14T17:41:18.167" Score="10" Body="&lt;p&gt;This is not an answer to your question.  But if a student came to me asking this question, I would ask him or her to read these articles:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.cs.utexas.edu/~dahlin/bookshelf/hamming.html&quot;&gt;You and your research (Hamming)&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.weizmann.ac.il/mcb/UriAlon/nurturing/HowToChooseGoodProblem.pdf&quot;&gt;How to choose a good scientific problem (Alon)&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And a warning: jumping into a &quot;hot&quot; subfield as a student is very dangerous if your advisor is not at the front of that subfield.  You're very likely to get scooped or end up doing something irrelevant.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-03-14T17:41:18.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5557" PostTypeId="1" CreationDate="2013-03-14T20:03:16.963" Score="10" ViewCount="2844" Body="&lt;p&gt;Please excuse the longish question, it just needs some explanation to get down to the actual problem. Those familiar with the mentioned algorithms probably could jump directly to the first simplex tablau.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To solve least absolute deviation problems (a.k.a $L_1$-optimization), the Barrodale-Roberts-algorithm is a special purpose simplex method that needs far less storage and computational efforts to find a suitable minimum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My implementation of the algorithm terminates at a simple example before a proper minimum is reached. However, probably let me state the problem in a more elaborated manner first:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given data $(x_i,y_i)$, $L_1$-optimization tries to find $c\in m$ that minimizes&#xA;$$&#xA;\sum_{i=1}^n |y_i-f(x_i)| \quad\text{with}\quad f(x):=A_x\cdot \phi&#xA;$$&#xA;where $A_x$ is a $n\times m$ matrix that depends in some way on $x$. This problem can be stated as a linear program and therefore among others be solved using simplex-like methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Barrodale and Roberts suggested an (apparently widely used) modification of the simplex method that radically simplifies the simplex method using the special structure of $L_1$-problems. Most notably, this is that an optimal solution interpolates at least $\mathop{rank}(A)$ of the given datapoints. Those with Jstor access may find the corresponding article &lt;a href=&quot;http://www.jstor.org/stable/2156318&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lei and Anderson in 2002 proposed a &lt;a href=&quot;http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADP013759&quot;&gt;small modification&lt;/a&gt; that is supposed to increase numerical stability and therefore to overcome known problems with the simplex algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, this algorithm assumes that you start with a given set of points that have to be interpolated, use the given procedures to build a simplex tableau and then use the rules of Barrodale and Roberts to decide on which basis variables to change and therefore modify the set of datapoints that is approximated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Barrodale and Roberts give a small example that I tried to reproduce. It tries to approximate the points $\{(1,1), (2,1), (3,2), (4,3), (5,2)\}$ by a function $a_1+a_2x$. The finish their algorithm with the following condensed simplex tableau:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{l|l|ll}&#xA; \hline&#xA; \text{Basis}&amp;amp;R&amp;amp; u_1 &amp;amp; u_3\\&#xA; \hline&#xA; b_1 &amp;amp; 1/2 &amp;amp; 3/2 &amp;amp; -1/2 \\&#xA; v_2 &amp;amp; 1/2 &amp;amp; 1/2 &amp;amp; 1/2 \\&#xA; b_2 &amp;amp; 1/2 &amp;amp; -1/2 &amp;amp; 1/2 \\&#xA; u_4 &amp;amp; 1/2 &amp;amp; 1/2 &amp;amp; -3/2 \\&#xA; v_5 &amp;amp; 1 &amp;amp; -1 &amp;amp; 2 \\&#xA;\hline&#xA; \text{Marginal cost} &amp;amp;2 &amp;amp; -1 &amp;amp; 0\\&#xA;\hline&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most importantly, the first and third point are interpolated and the overall error is equal to  $2$. They conclude that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since all of the nonbasic vectors have nonpositive marginal cost [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the iteration is finished and and optimum is reached. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use the algorithm of Lei and Anderson, I can reproduce that simplex tableau for the interpolation set {1,3}, as it is expected. However, if I start the algorithm with the set $\{2, 5\}$ (which clearly is not optimal), I get the following simplex tableau:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array}{l|l|ll}&#xA; \hline&#xA; \text{Basis}&amp;amp;R&amp;amp; u_2 &amp;amp; u_5\\&#xA; \hline&#xA; u_1 &amp;amp; 1/3 &amp;amp; -4/3 &amp;amp; 1/3 \\&#xA; b_1 &amp;amp; 1/3 &amp;amp; 5/3 &amp;amp; -2/3 \\&#xA; u_3 &amp;amp; 2/3 &amp;amp; -2/3 &amp;amp; -1/3 \\&#xA; u_4 &amp;amp; 4/3 &amp;amp; -1/3 &amp;amp; -2/3 \\&#xA; b_2 &amp;amp; 1/3 &amp;amp; -1/3 &amp;amp; 1/3 \\&#xA;\hline&#xA; \text{Marginal cost} &amp;amp;7/3 &amp;amp; -10/3 &amp;amp; -5/3\\&#xA;\hline&#xA;\end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This result is puzzling me, though. If I understand the quote above correctly, having no positive marginal cost indicates that the optimum is reached. The function value of about 2.33 certainly is not optimal, though. Exchanging $u_2$ with $u_1$ would yield a result that is on par with the solution of Barrodale and Roberts and therefore optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional info: If I start with the initial tableau given by Barrodale and Roberts, I am also able to reproduce the tableau above by ordinary simplex steps, thus I am fairly confident that the actual numerical values are correct and my interpretation of the pivot selection rule is faulty. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts on this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize that the question itself is quite complicated and probably requires knowledge of at least the Barrodale and Roberts algorithm to be answered sufficiently. The algorithm as a whole is to long to repeat it here in full detail. However, if you have additional questions on steps I took or on missing bits of information, feel free to ask and I will gladly augment the question. &lt;/p&gt;&#xA;" OwnerUserId="3915" LastActivityDate="2013-03-15T17:53:03.957" Title="Least absolute deviations solving using the Barrodale-Roberts-algorithm: Premature termination?" Tags="|optimization|linear-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5560" PostTypeId="1" CreationDate="2013-03-15T09:58:27.960" Score="2" ViewCount="89" Body="&lt;p&gt;I'd like to be able to view in graph form the volume and pressure of steam produced from heating water in a sealed vessel, starting from room temperature water.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Important variables, such as the saturation temperature of steam at different pressures and the energy required to convert water to steam at different pressures are contained in steam tables rather than formulae.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To graph the total volume of steam produced with time, is this the sort of thing these programs like Matlab are good for, automatically interpolating tabular data as appropriate to get a more accurate integration? Do such programs come with many in-built tables like steam tables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was just thinking I'd attempt something in Excel but I'm perplexed over how to handle the circular dependency here, i.e. pressure determining temperature which determines evaporation rate which determines pressure.&lt;/p&gt;&#xA;" OwnerUserId="3921" LastActivityDate="2013-03-15T11:15:13.587" Title="Integrating from tabular data, in particular steam tables" Tags="|software|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5561" PostTypeId="2" ParentId="5560" CreationDate="2013-03-15T11:15:13.587" Score="1" Body="&lt;p&gt;There is already software available that can do this, both commercially and open-source. As an example of the latter, I suggest you take a look at &lt;a href=&quot;http://freesteam.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;Freesteam&lt;/a&gt;. It is based on the  IAPWS-IF97 steam tables.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-03-15T11:15:13.587" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5562" PostTypeId="1" CreationDate="2013-03-15T15:32:32.297" Score="8" ViewCount="671" Body="&lt;p&gt;Suppose we want to solve a hyperbolic conservation law $u_t+f(u)_x=0$. I really like to use Lax-Wendroff, which reads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_j^{n+1} = u_j^n -\frac{\Delta t}{\Delta x}(g(u_{j+1}^n,u_j^n)-g(u_j^n,u_{j-1}^n))$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g(v,w) = \frac12(f(v)+f(w)) - \frac{\Delta t}{2\Delta x}\vert\frac{f(w)-f(v)}{w-v}\vert^2(w-v)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or for $f(u)=au$ (linear advection),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g(v,w)=\frac12a((v+w)-\frac{\Delta t}{\Delta x}a(w-v))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: Is there anything similar that has a higher order? I'm not talking about those fancy high resolution, (W)ENO, MUSCL,... schemes - just a plain third or fourth order, stable scheme that works for arbitrary $f$.&lt;/p&gt;&#xA;" OwnerUserId="3914" LastEditorUserId="3914" LastEditDate="2013-03-18T08:16:34.337" LastActivityDate="2013-04-18T13:02:04.657" Title="Higher order Lax-Wendroff type scheme?" Tags="|fluid-dynamics|hyperbolic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5563" PostTypeId="2" ParentId="5554" CreationDate="2013-03-15T16:42:54.697" Score="12" Body="&lt;p&gt;Yes, but the answer is a little more complicated than you were probably hoping for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short: one color in general corresponds to a distribution of different frequencies, or wavelengths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have three types of cone cells in your eyes [+ one type of rod cell].  Color-blind people have less than three types (or their response curves are closer together).  It is believed a few people may exist, &quot;tetrachromats&quot;, who have four types.  Some species of octopi see in 11 dimensions of color.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z9QOk.png&quot; alt=&quot;Spectral sensitivity curves&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each type of cone cell is sensitive to certain wavelengths.  If you have two lasers, you can draw a vertical line from the frequency of the laser up to each spectral response curve and add the values for each cone type to get the total &quot;stimulus&quot; per type.  It is clear then that you can have a vast combination of frequency pairs that cause the same stimulus in each of the S, M, and L type cells!  This is why a computer monitor can generate the color yellow using red and green subpixels.  This same yellow can occur with photons of one wavelength that give the same total stimulus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the phenomenon of color isn't as simple as matching each one up to a wavelength.  In fact, there are colors that humans can see that correspond to no single wavelength!  No laser can produce the colors pink and purple. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a chromaticity diagram.  I won't go into too much detail about how this is constructed, but it roughly corresponds to all the types of colors you can see.  You can also vary the luminance of the diagram (roughly corresponds to the illy-defined term &quot;brightness&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/kuARE.png&quot; alt=&quot;CIE 1931 Chromaticity Diagram&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, this diagram has a few interesting properties.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;While it represents all of the colors you can see, all of those colors are not actually displayed on your screen.  This is because for multiple light sources, you can only reproduce the colors that are within the vertices of the polygon formed by connecting the corresponding chromaticities of each light source.  Most monitors use three primaries (approximately red, green, and blue) and so these monitors can only reproduce the colors within a triangle.  Note that the &lt;em&gt;area&lt;/em&gt; of the triangle does not correspond to anything particularly meaningful.  The diagram is not designed to give equal space between distinguishable colors.&lt;/li&gt;&#xA;&lt;li&gt;The only way you can actually reproduce all colors that a human can see is with two tunable lasers.  This is possible by splitting light from a blackbody radiation source (e.g. the sun) using a prism, passing the light through a narrow frequency filter, and mixing it with light from another prism of a different wavelength.&lt;/li&gt;&#xA;&lt;li&gt;The edge of the diagram (the smooth curve) corresponds to the wavelength of light.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, with this background, I can answer your question a little more directly.  Since monitors use three light sources per pixel, the nature of each subpixel's light must be characterized with its own spectral power distribution.  For most purposes this can be approximated as a single wavelength.  You then have to determine the three coefficients that weight each subpixel such that the sum of their values times the corresponding intensities on the stimulus response curve equals the same as which produces whatever color it is you're wondering about.  Most color primaries data is freely available online, but I'll leave the math problem for someone else.&lt;/p&gt;&#xA;" OwnerUserId="1997" LastEditorUserId="1997" LastEditDate="2013-03-24T21:14:49.920" LastActivityDate="2013-03-24T21:14:49.920" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5564" PostTypeId="2" ParentId="5557" CreationDate="2013-03-15T17:53:03.957" Score="4" Body="&lt;p&gt;Solved it. Actually, Barrodale and Roberts solved it and I just did not read carefully. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my question I left it to the reader to understand that the variables Barrodale and Roberts labeled $u_i$ stand for the positive residuals the $i$-th datapoint in relation to the current fit. If the residual is negative, $u_i=0$ and $v_i$ takes the corresponding value. As only one of them may be within the basis and the coefficients in the simplex tableau are just the negatives of each other, it is not necessary to explicitly state them in the simplex tableau. Barrodale and Roberts mention in their article:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] and that the sum of the marginal (or reduced) costs of $b_j$ and $c_j$ is  zero and of $u_i$ and $v_i$ is -2.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thus, my simplex tableau above has to be thought to look as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array}{l|l|ll| ll}&#xA; \hline&#xA; \text{Basis}&amp;amp;R&amp;amp; u_2 &amp;amp; u_5 &amp;amp; v_2 &amp;amp; v_5\\&#xA; \hline&#xA; u_1 &amp;amp; 1/3 &amp;amp; -4/3 &amp;amp; 1/3 &amp;amp; 4/3 &amp;amp; -1/3 \\&#xA; b_1 &amp;amp; 1/3 &amp;amp; 5/3 &amp;amp; -2/3 &amp;amp; -5/3 &amp;amp; 2/3 \\&#xA; u_3 &amp;amp; 2/3 &amp;amp; -2/3 &amp;amp; -1/3 &amp;amp; 2/3 &amp;amp; 1/3 \\&#xA; u_4 &amp;amp; 4/3 &amp;amp; -1/3 &amp;amp; -2/3 &amp;amp; 1/3 &amp;amp; 2/3 \\&#xA; b_2 &amp;amp; 1/3 &amp;amp; -1/3 &amp;amp; 1/3 &amp;amp; -1/3 &amp;amp; -1/3 \\&#xA;\hline&#xA; \text{Marginal cost} &amp;amp;7/3 &amp;amp; -10/3 &amp;amp; -5/3 &amp;amp; 4/3 &amp;amp; -1/3\\&#xA;\hline&#xA;\end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we clearly see that $v_2$ can be brought into the basis to archive a better result. Doing this the algorithm terminates while interpolating the first and fifth datapoint with an overall error of 2 - which is the best solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for reading and giving me some place to write my problem down, which usually helps to narrow the solution down significantly. Hopefully, this answer might sometimes be helpful for anybody else trying to implement Barrodale &amp;amp; Roberts. &lt;/p&gt;&#xA;" OwnerUserId="3915" LastActivityDate="2013-03-15T17:53:03.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5565" PostTypeId="1" CreationDate="2013-03-15T18:21:23.577" Score="4" ViewCount="747" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In &lt;a href=&quot;http://pages.cs.wisc.edu/~shuchi/courses/787-F09/scribe-notes/lec13.pdf&quot; rel=&quot;nofollow&quot;&gt;a note:&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;semi-deﬁnite programming is equivalent to vector programming.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;A Vector Program is a Linear Program over dot products.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;In Boyd's Convex Optimization, a vector optimization problem is&#xA;defined to be optimizing a vector-valued function, where the vectors&#xA;are ordered wrt a given cone.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am not sure if the two definitions are equivalent and how they are related? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering what is the general formulation of a vector programming problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is it solved in general?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more references on vector programming?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="3907" LastEditDate="2013-03-24T22:03:18.953" LastActivityDate="2013-03-24T22:03:18.953" Title="What is a vector programming problem?" Tags="|optimization|semidefinite-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5566" PostTypeId="1" AcceptedAnswerId="5570" CreationDate="2013-03-15T21:17:03.967" Score="4" ViewCount="608" Body="&lt;p&gt;From &lt;a href=&quot;http://en.wikipedia.org/wiki/Semidefinite_programming#Equivalent_formulations&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Denote by $\mathbb{S}^n$ the space of all $n \times n$ real symmetric matrices. The space is equipped with the inner product (where ${\rm tr}$ denotes the trace) $$\langle A,B\rangle_{\mathbb{S}^n} = {\rm tr}(A^T B) = \sum_{i=1,j=1}^n A_{ij}B_{ij}.$$&#xA;  We can rewrite the mathematical program given in the previous section equivalently as&#xA;  $$&#xA;    \begin{array}{rl} {\displaystyle\min_{X \in \mathbb{S}^n}} &amp;amp; \langle C, X \rangle_{\mathbb{S}^n} \\ \text{subject to} &amp;amp; \langle A_k, X \rangle_{\mathbb{S}^n} \leq b_k, \quad k = 1,\ldots,m \\ &amp;amp; X \succeq 0 \end{array} &#xA;$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://www.stanford.edu/~boyd/papers/pdf/semidef_prog.pdf&quot; rel=&quot;nofollow&quot;&gt;Boyd's paper&lt;/a&gt;, a semidefinite programming problem is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$$\min_{x \in\mathbb R^m} c^T  x$$&#xA;  subject to &#xA;  $$F_0 + \sum_{i=1}^m x_i F_i  ⪰  0$$&#xA;  where&#xA;  $c \in \mathbb R^m$ and &#xA;  $m + 1$ symmetric matrices $F_0, ..., F_m \in \mathbb R^{n\times n}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was wondering if the two formulations are equivalent? I am not able to see how they are related. Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="258" LastEditDate="2013-03-15T22:35:12.987" LastActivityDate="2013-03-16T16:23:33.853" Title="Are these two formulations of semidefinite programming problems equivalent?" Tags="|optimization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5567" PostTypeId="1" CreationDate="2013-03-15T22:07:48.553" Score="1" ViewCount="64" Body="&lt;p&gt;my question is: can I use a RANS simulation to see the effect of mixed/natural convection?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually I have also a second question: I would like to do this in Comsol multiphysics, but it seems that it has no model that takes into account turbulence (like k epsilon, Reinolds stress, LES, ecc..). &#xA;Ps: this could be because I use a pretty old version of the software, but anyway I haven't found documentation about this topic so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your answers!&lt;/p&gt;&#xA;" OwnerUserId="3860" LastActivityDate="2013-03-15T22:07:48.553" Title="Can I use RANS to see the effect of mixed convection?" Tags="|pde|fluid-dynamics|comsol|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5568" PostTypeId="2" ParentId="5493" CreationDate="2013-03-15T23:33:58.543" Score="3" Body="&lt;p&gt;I would add to that IBM's Data Visualization Explorer, commonly known as &lt;a href=&quot;http://www.opendx.org&quot; rel=&quot;nofollow&quot;&gt;OpenDX&lt;/a&gt; (&lt;a href=&quot;http://en.wikipedia.org/wiki/IBM_OpenDX&quot; rel=&quot;nofollow&quot;&gt;Wikipedia entry&lt;/a&gt;).  It's extremely versatile and open source as well.&lt;/p&gt;&#xA;" OwnerUserId="3923" LastActivityDate="2013-03-15T23:33:58.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5569" PostTypeId="2" ParentId="5566" CreationDate="2013-03-16T03:59:10.880" Score="4" Body="&lt;p&gt;There is a duality theory for SDP that is similar to the duality theory for linear programming.  The SDP duality theory is weaker than LP duality theory in that you need a constraint qualification in order for strong duality to hold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, there is no commonly agreed standard primal and dual problem formulation.  If you replace the less than or equal to constraints in the first formulation with equality constraints and take the dual, and fix the notation a bit, then you'll get Boyd's form of the SDP problem as the dual of the other problem.  As written these aren't quite a primal dual pair, but they are both reasonably normal SDP formulations.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-03-16T03:59:10.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5570" PostTypeId="2" ParentId="5566" CreationDate="2013-03-16T16:23:33.853" Score="3" Body="&lt;p&gt;As Brian says, if you change the inequality in the first model to an equality constraint (which I would say is much more common than the Wikipedia format), you get a model whose dual is given by $\text{maximize} ~b^Ty$ subject to $C - \sum_{i=1}^m A_i y_i \succeq 0$. In that form, the equivalence should be obvious. Hence, they both define the same primal-dual representation of an SDP, module some signs on some matrices. An SDP solver wants the data $(C,A_i,b)$ so which ever form you work with does not make any difference (once you've managed to figure what these matrices are, it is not uniquely determined so you can accidentally define unnecessarily large models)&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-03-16T16:23:33.853" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5571" PostTypeId="2" ParentId="5503" CreationDate="2013-03-16T20:30:56.703" Score="9" Body="&lt;p&gt;Johan has certainly done a more complete job of answering Bel's question here than I did on the CVX Forum. But I do think there is a conceptual difficulty at play here that is worth discussing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my response on the CVX Forum, I said that &quot;you simply cannot expect CVX to accept any expression that you want even if you prove it is convex.&quot; Johan offers a solution that involves rewriting the objective $f(S)=\mathop{\textrm{trace}}(S)+\mathop{\textrm{trace}}(S^{-2})$ considerably. In other words, &lt;em&gt;YALMIP doesn't directly handle the function either&lt;/em&gt;, although he does successfully show that it is possible. To my knowledge, &lt;em&gt;no&lt;/em&gt; modeling framework does, yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order for any modeling framework to &quot;handle&quot; a particular expression, be it $f(S)$ or any other, it needs to be able to construct a &lt;em&gt;computational description&lt;/em&gt; of that expression that is compatible with the underlying solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, many solvers require code to compute the values and derivatives (typically the first two) of the expressions involved. You could supply those derivatives by hand---but for $f(S)$, this is by no means straightforward. In most cases, automatic or symbolic differentiation does the difficult work---but this only works if the expression is composed of the functions and operations that the AD/SD system &lt;em&gt;already knows&lt;/em&gt;. I am unaware of an AD/SD engine that handles matrix expressions such as $f(S)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CVX and YALMIP have a different approach at their disposal: they can construct &lt;em&gt;graph representations&lt;/em&gt; of expressions. Roughly speaking, a graph representation uses equations and inequalities involving &quot;known&quot; expressions to describe the geometry of a &quot;new&quot; inequality. A simple example is a constraint involving the absolute value:&#xA;$$|x| \leq y$$&#xA;Smooth solvers will have difficulty with the non-differentiability at $x=0$. But the solution is simple and well-known: replace the constraint with two linear inequalities:&#xA;$$-y \leq x \leq y$$&#xA;Graph representations let CVX and YALMIP generalize and automate these kinds of transformations. These modeling frameworks provide a variety of functions that are not available in a strict function/derivative approach. But the libraries are still &lt;em&gt;finite&lt;/em&gt;, which means that users must construct their models using only the functions provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What Johan is doing in his post is showing how to construct a graph representation of $f(S)$ by hand, so that it can be solved using YALMIP or CVX. I'm reasonably certain that his derivation is correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modeling frameworks hide and automate the complexity of translating models from their natural form to a machine-solvable form. This is almost always a good thing. But occasionally, it is helpful to understand how the proverbial sausage is made. With a convex optimization framework, it is &lt;em&gt;not enough&lt;/em&gt; simply to prove that your model is convex. One must still consider the problem of translating to solvable form. CVX and YALMIP automate this process &lt;em&gt;if&lt;/em&gt; you are willing to follow the conventions they require.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I wrote &lt;a href=&quot;http://www.stanford.edu/~boyd/papers/graph_dcp.html&quot; rel=&quot;nofollow&quot;&gt;a paper on graph representations&lt;/a&gt; if anyone is interested. &lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-03-18T17:56:39.753" LastActivityDate="2013-03-18T17:56:39.753" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5572" PostTypeId="1" AcceptedAnswerId="5576" CreationDate="2013-03-17T10:04:31.980" Score="10" ViewCount="588" Body="&lt;p&gt;What is a good way to extract the diagonal from a symmetric matrix that is already almost diagonal, but where you don't have the matrix elements (only the ability to apply it to vectors)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further constraints are, (1) applying the n-by-n matrix n-times to explicitly construct the diagonal would be prohibitively costly, and (2) the small elements of the diagonal are important in addition to the large elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example picture of the type of matrix I want to extract the diagonal from (in a small-scale test case):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/0ZULK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="9" LastEditDate="2013-03-17T22:31:44.027" LastActivityDate="2022-04-28T13:22:26.633" Title="Extracting diagonal of an approximately diagonal matrix when we don't know its entries" Tags="|linear-algebra|preconditioning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5573" PostTypeId="1" AcceptedAnswerId="5583" CreationDate="2013-03-17T13:00:36.167" Score="20" ViewCount="2859" Body="&lt;p&gt;A lot of my own work revolves around making algorithms scale better, and one of the preferred ways of showing parallel scaling and/or parallel efficiency is to plot the performance of an algorithm/code over the number of cores, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/KVVLE.png&quot; alt=&quot;artificial parallel scaling plot&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $x$-axis represents the number of cores and the $y$-axis some metric, e.g. work done per unit of time. The different curves show parallel efficiencies of 20%, 40%, 60%, 80%, and 100% at 64 cores respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately though, in many publications, these results are plotted with a &lt;em&gt;log-log&lt;/em&gt; scaling, e.g. the results in &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/early/2013/02/21/bioinformatics.btt055.short&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; or &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/early/2013/02/21/bioinformatics.btt055.short&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; paper. The problem with these log-log plots is that it's incredibly difficult to assess the actual parallel scaling/efficiency, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/731uY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the same plot as above, yet with log-log scaling. Note that now there is no big difference between the results for 60%, 80%, or 100% parallel efficiency. I've written a bit more extensively about this &lt;a href=&quot;http://www.dur.ac.uk/pedro.gonnet/?p=141&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here's my question: &lt;strong&gt;What rationale is there for showing results in log-log scaling?&lt;/strong&gt; I regularly use linear scaling to show my own results, and regularly get hammered by referees saying that my own parallel scaling/efficiency results don't look as good as the (log-log) results of others, but for the life of me I can't see why I should switch plot styles.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2019-01-31T18:07:58.253" Title="Log-log parallel scaling/efficiency plots" Tags="|parallel-computing|hpc|scaling|plotting|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5574" PostTypeId="2" ParentId="5573" CreationDate="2013-03-17T13:48:10.083" Score="17" Body="&lt;p&gt;Georg Hager wrote about this in &lt;a href=&quot;http://blogs.fau.de/hager/2010/07/22/fooling-the-masses-stunt-3-the-log-scale-is-your-friend/&quot;&gt;Fooling the Masses - Stunt 3: The log scale is your friend&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it is true that log-log plots of strong scaling are not very discerning on the high end, they allow for showing scaling across many more orders of magnitude. To see why this is useful, consider a 3D problem with regular refinement. On a linear scale, you can reasonably show performance across about two orders of magnitude, e.g., 1024 cores, 8192 cores, and 65536 cores. It is impossible for the reader to tell from the plot whether you ran anything smaller, and realistically, the plot mostly just compares the largest two runs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now supposing we can fit 1 million grid cells per core in memory, this means that after strong scaling twice by a factor of 8, we can still have 16k cells per core. That is still a sizable subdomain size and we can expect many algorithms to run efficiently there. We've covered the visual spectrum of the chart (1024 to 65536 cores), but haven't even entered the regime where strong scaling becomes difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose instead that we started at 16 cores, also with 1 million grid cells per core. Now if we scale out to 65536 cores, we'll only have 244 cells per core, which is going to be &lt;em&gt;much&lt;/em&gt; more discerning. A log axis is the only way to clearly represent the spectrum from 16 cores to 65536 cores. Of course you can still use a linear axis and have a caption saying &quot;the data points for 16, 128, and 1024 cores overlap in the figure&quot;, but now you're using words instead of the figure itself to show.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A log-log scale also allows your scaling to &quot;recover&quot; from machine attributes like moving beyond a single node or rack. It's up to you whether this is desirable or not.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-03-17T13:48:10.083" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5575" PostTypeId="2" ParentId="5573" CreationDate="2013-03-17T14:27:47.677" Score="2" Body="&lt;p&gt;I agree with everything Jed had to say in his response, but I wanted add the following. I've become a fan of the way Martin Berzins and his colleagues show scaling for their Uintah framework. They plot weak and strong scaling of the code on log-log axes (using the run time per step of the method). I think it shows how the code scales pretty well (though deviation from perfect scaling is a little hard to determine). See page 7 and 8 figures 7 and 8 of &lt;a href=&quot;http://www.sci.utah.edu/publications/luitjens10/Luitjens_ipdps2010.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;* paper for example. They also give a table with the numbers corresponding to each scaling figure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An advantage of this is that once you've provided the numbers, there's not much a reviewer can say (or at least not much that you can't rebut).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*J. Luitjens, M. Berzins. “Improving the Performance of Uintah: A Large-Scale Adaptive Meshing Computational Framework,” In Proceedings of the 24th IEEE International Parallel and Distributed Processing Symposium (IPDPS10), Atlanta, GA, pp. 1--10. 2010. DOI: &lt;a href=&quot;http://dx.doi.org/10.1109/IPDPS.2010.5470437&quot; rel=&quot;nofollow&quot;&gt;10.1109/IPDPS.2010.5470437&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-02-18T18:57:44.617" LastActivityDate="2015-02-18T18:57:44.617" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5576" PostTypeId="2" ParentId="5572" CreationDate="2013-03-17T14:58:58.127" Score="13" Body="&lt;p&gt;I'm going to answer my own question since the following method seems to be very effective. I'm making it an answer so people can upvote or downvote it independently of the question if they think it is good or bad.&lt;/p&gt;&#xA;&lt;p&gt;Answer: use &lt;a href=&quot;http://math.mit.edu/icg/papers/matrix-probing-theory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;randomized matrix probing&lt;/a&gt; applied to the diagonal of the matrix.&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; be the operator we wish to find the diagonal of, and let &lt;span class=&quot;math-container&quot;&gt;$\omega_1,\omega_2,..,\omega_k$&lt;/span&gt; be a small number of gaussian random vectors. Then apply &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to the random vectors to get &lt;span class=&quot;math-container&quot;&gt;$A \omega_1, A \omega_2, ..., A \omega_k$&lt;/span&gt; and solve the following least-squares minimization problem,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\min_{\text{diagonal }D} ||D \omega_1-A\omega_1||^2 + ||D \omega_2-A\omega_2||^2 + ... + ||D \omega_k-A\omega_k||^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The minimum has the exact formula,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$d_i=\frac{\omega_1^i A \omega_1^1 + \omega_2^i A \omega_2^i... + \omega_k^i A \omega_k^i}{(\omega_1^i)^2 + (\omega_2^i)^2 + ... + (\omega_k^i)^2}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Matlab code, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;omegas = randn(16,3); &#xA;dprobe=sum(omegas.*(A*omegas),2)./sum(omegas.^2,2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In my example matrix, with 3 probing vectors, the exact diagonal and probed diagonal compare as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[dprobe, diag(A)]&#xA;&#xA;ans =&#xA;&#xA;1.0e+04 *&#xA;&#xA;2.3297    2.4985&#xA;0.4596    0.4921&#xA;0.1322    0.0897&#xA;0.2838    0.1764&#xA;0.0989    0.0999&#xA;0.0106    0.0071&#xA;0.0068    0.0068&#xA;0.0469    0.0571&#xA;0.0070    0.0070&#xA;0.0355    0.0372&#xA;0.0059    0.0060&#xA;0.0071    0.0064&#xA;0.0067    0.0067&#xA;0.0026    0.0021&#xA;0.0012    0.0012&#xA;0.0015    0.0013&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; I've been experimenting applying these ideas to symmetric block matrices, since a matrix i'm working with is almost block diagonal in a wavelet-like basis. It seems to work pretty well for building preconditioners so long as the matrix is &amp;quot;block-diagonally-dominant&amp;quot; (definition is a little tricky), and so long as you symmetrize the least-squares reconstructed blocks.&lt;/p&gt;&#xA;&lt;p&gt;Recall that a matrix partitioned into blocks &lt;span class=&quot;math-container&quot;&gt;$A_{i,j}$&lt;/span&gt; is &lt;a href=&quot;https://projecteuclid.org/euclid.pjm/1103036125&quot; rel=&quot;nofollow noreferrer&quot;&gt;block-diagonally-dominant&lt;/a&gt; if&#xA;&lt;span class=&quot;math-container&quot;&gt;$$||A_{i,i}^{-1}||^{-1} \ge \sum_j ||A_{i,j}||.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given gaussian random &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt;'s as above, we seek to find the following least-squares block diagonal reconstruction:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min_{\mathrm{block~diagonals~}B} ||B \omega_1-A\omega_1||^2 + ||B \omega_2-A\omega_2||^2 + ... + ||B \omega_k-A\omega_k||^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;After some tensor product manipulations, you can find the exact formula for the &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;'th block &lt;span class=&quot;math-container&quot;&gt;$\tilde{B}^{(l)}$&lt;/span&gt; by solving the local problems:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\tilde{B}^{(l)} = [(A\omega_1)^{(l)}\omega_1^{(l)T} + ... + (A\omega_k)^{(l)}\omega_k^{(l)T}][\omega_1^{(l)}\omega_1^{(l)T} + ... + \omega_k^{(l)}\omega_k^{(l)T}]^{-1},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$(A\omega_i)^{(l)}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\omega_i^{(l)}$&lt;/span&gt; are the portions of &lt;span class=&quot;math-container&quot;&gt;$A\omega_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\omega_i$&lt;/span&gt; corresponding to the indices of the &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;'th block.&lt;/p&gt;&#xA;&lt;p&gt;If I just use these &lt;span class=&quot;math-container&quot;&gt;$\tilde{B}$&lt;/span&gt;'s, the preconditioning seems to be pretty bad, but if I symmetrize as follows,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$B^{(l)} = (\tilde{B}^{(l)} + \tilde{B}^{(l)T})/2,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;in my experiments it becomes almost as good as if I had used the true diagonal blocks (often better!). Here is an example matrix in pictures,&#xA;&lt;img src=&quot;https://i.stack.imgur.com/NAJL5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1845" LastEditDate="2022-04-27T14:13:17.030" LastActivityDate="2022-04-27T14:13:17.030" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5577" PostTypeId="5" CreationDate="2013-03-17T16:33:40.540" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-03-17T16:33:40.540" LastActivityDate="2013-03-17T16:33:40.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5578" PostTypeId="4" CreationDate="2013-03-17T16:33:40.540" Score="0" Body="Questions about analysis, implementation or application of Galerkin methods for partial differential equations using piecewise functions that are not globally continuous (and hence require surface terms on element boundaries in addition to the usual volume terms occurring in finite element methods)." OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-03-17T17:07:11.430" LastActivityDate="2013-03-17T17:07:11.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5579" PostTypeId="1" AcceptedAnswerId="5597" CreationDate="2013-03-17T22:18:43.947" Score="4" ViewCount="1147" Body="&lt;p&gt;In Multigrid, using Poisson's equation, does the equality below &lt;strong&gt;always&lt;/strong&gt; hold regardless of what type of boundary conditions you use?&#xA;$$&#xA;R= c\cdot I^T, \text{ for some constant }c&#xA;$$&#xA;where $R$ and $I$ are the restriction and interpolation operator. Why or why not?&lt;/p&gt;&#xA;" OwnerUserId="3493" LastActivityDate="2013-03-19T02:22:09.473" Title="Prolongation/Restriction Operator in Multigrid" Tags="|interpolation|multigrid|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5581" PostTypeId="2" ParentId="5322" CreationDate="2013-03-18T07:40:31.350" Score="2" Body="&lt;p&gt;Kalman filter (and its variants) can be used to solve this problem. &lt;/p&gt;&#xA;" OwnerUserId="3937" LastActivityDate="2013-03-18T07:40:31.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5582" PostTypeId="2" ParentId="5447" CreationDate="2013-03-18T11:25:00.737" Score="2" Body="&lt;p&gt;As the reviewers already suggested, it would be nice to keep things in a context rather then speaking in general. Still, I believe I could comment on the questions. Even if the implicit solution approach is also not clear (matrix-free, automatic differentiation, lagged Jacobian, or nonlinear-gs with local Newton iterations, Newton-MG, preconditioning?) I would give some insight for flux limiters. They are usually use to track the fields that exhibit abrupt changes like shocks or interfaces in general. Apart selection of the type of the limiters, there are two main steps: calculation of r and calculation of f(r). Starting from the latter, f is usually nonlinear - that has different complexity depending on the selected limiter. Some are simpler when considered in the Jacobian, some not. Even in some cases, max-min combinations should be evaluated (which also requires different measures for CPU and GPU computing). For r, if the mesh is simple - like uniform and structured, it is most of the time straightforward to compute (might need ghost cells in finite volumes to extend the stencils). Yet, for an unstructured mesh - the neighborhood is important. The computations can be handled more efficiently if the grid data is well sorted, still some geometric interpolations will be needed to calculate r (there are papers on that, refer the ones given in &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0131274988&quot; rel=&quot;nofollow&quot;&gt;An Introduction to Computational Fluid Dynamics: The Finite Volume Method&lt;/a&gt; chapters 5 &amp;amp; 11). From my personal experience, I would focus on van Leer and Sweby and condense the study on the complexity.&lt;/p&gt;&#xA;" OwnerUserId="1499" LastActivityDate="2013-03-18T11:25:00.737" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5583" PostTypeId="2" ParentId="5573" CreationDate="2013-03-18T12:17:20.213" Score="18" Body="&lt;p&gt;We are currently writing a paper that contains a number of comparable plots, and we more or less had the same problem. The paper is about comparing the scaling of different algorithms over the number of cores, which ranges between 1 and up to 100k on a BlueGene. The reason for using loglog-plots in this situation is the number of orders of magnitude involved. There is no way one can plot 6 orders of magnitude on a linear scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And indeed, when plotting the time over the number of cores in loglog, the algorithms are not very distinguishable, as you can see in the following plot. &lt;img src=&quot;https://i.stack.imgur.com/PnoVO.png&quot; alt=&quot;Timings of a number of algorithms on loglog-scale. The different algorithms are hard to distinguish.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, we decided to plot the &lt;a href=&quot;http://en.wikipedia.org/wiki/Parallel_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;parallel efficiency&lt;/a&gt; over the number of cores instead of the timing on a semilog-scale. The parallel efficiency is defined as &lt;span class=&quot;math-container&quot;&gt;$E_p=T_1/(p T_p)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$T_1$&lt;/span&gt; is the time when using a single core, &lt;span class=&quot;math-container&quot;&gt;$T_p$&lt;/span&gt; is the time when using &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; cores, and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the number of cores. Basically, &lt;span class=&quot;math-container&quot;&gt;$E_p$&lt;/span&gt; tells you the efficiency of an algorithm at &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, when doing that, absolute differences in the timings do not play a role (i.e. when one algorithm is ten times as slow as another one). To take care of that, we are using what we call the &quot;relative parallel efficiency&quot;, which is defined by &lt;span class=&quot;math-container&quot;&gt;$E_p=T_{ref}/(p T_p)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$T_{ref}$&lt;/span&gt; is the time of the &lt;em&gt;fastest&lt;/em&gt; algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotting the relative parallel efficiency on a semilog scale shows pretty clearly the scaling of an algorithm, and also shows how the algorithms perform relatively to each other. &lt;img src=&quot;https://i.stack.imgur.com/L0g0i.png&quot; alt=&quot;Relative parallel efficiency of a number of algorithms over the number of cores.&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1568" LastEditorUserId="24110" LastEditDate="2019-01-31T18:07:58.253" LastActivityDate="2019-01-31T18:07:58.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5584" PostTypeId="1" AcceptedAnswerId="5588" CreationDate="2013-03-18T13:00:19.927" Score="5" ViewCount="923" Body="&lt;p&gt;I am trying to understand an extra terms that appears when I derive the drift-diffusion equations for semiconductors. The extra term (see below) comes from applying the chain rule to the advection term. The extra term behaves as a source term, however, I can't justify its physical meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the &lt;a href=&quot;http://ecee.colorado.edu/~bart/book/book/chapter2/ch2_10.htm&quot; rel=&quot;nofollow&quot;&gt;semiconductor transport equations&lt;/a&gt; &lt;strong&gt;are&lt;/strong&gt; analogous to solving the an advection-diffusion problem: the semiconductor equations possess a diffusion term and drift (advection) to describe current flow. These equations are combined with the &lt;a href=&quot;http://ecee.colorado.edu/~bart/book/book/chapter2/ch2_9.htm&quot; rel=&quot;nofollow&quot;&gt;continuity equation&lt;/a&gt; so that particle number is conserved. The &lt;a href=&quot;http://en.wikipedia.org/wiki/Reaction%E2%80%93diffusion%E2%80%93advection_equation&quot; rel=&quot;nofollow&quot;&gt;reaction-advection-diffusion&lt;/a&gt; equation has the same properties,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial n}{\partial t} = \nabla\cdot\left(\boldsymbol{v}n + D\nabla n \right) + S$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $n$ is the electron density, $\boldsymbol{v}$ is the drift velocity, $D$ is the diffusion coefficient and $S$ is the source function (generation and recombination of electrons).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In semiconductors the drift velocity is a function of electric field, $\boldsymbol{v}=\mu \boldsymbol{E}$ (where $\mu$ is a material constant - the mobility). When there is no divergence in electric field the drift velocity is zero and the transport is dominated by diffusion. In turn the electric field is found my solving Poisson equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the differental operator acts on $\boldsymbol{v}$ and $n$ which are both function of $x$ therefore we must apply the chain rule to the adevection term,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla\cdot(\boldsymbol{v}n) = \boldsymbol{v}\frac{\partial n}{\partial x} + n \frac{\partial\boldsymbol{v}}{\partial x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting in the expression for drift velocity gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla\cdot(\boldsymbol{v_n}n) = \mu\boldsymbol{E}\frac{\partial n}{\partial x} + \mu n \frac{\partial\boldsymbol{E}}{\partial x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore the final expression is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial n}{\partial t} = \underbrace{D\frac{\partial^2 n}{\partial x^2}}_{\textrm{Diffusion}} + \underbrace{\overbrace{\mu\boldsymbol{E}}^{\boldsymbol{v}}\frac{\partial n}{\partial x}}_{\textrm{Advection}} + \underbrace{\mu n \frac{\partial \boldsymbol{E}}{\partial x}}_{\textrm{?}} + \underbrace{S}_{\textrm{Source}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term marked with the question mark acts like an additional source term which generates/removes (depending on the sign) electrons when the field divergence is non-zero. If you take the example of the  pn-junction, there is always a build-in electric field so how can this ever reach steady-state?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Simulation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have simulated the formation of the pn-junction (neutral p-type on the left, joined to neutral n-type on the right) and I do indeed get generation of carriers in the region where there is no divergence of the electric field (i.e. the depletion region). To solve the equations I am doing a Crank-Nicolson time sweep. At every step I solve the Poisson equation to give the electric field profile, which is used in the subsequent time step to calculate electron (and hole) density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody have an explanation for additional term? Or maybe there is something fundamentally wrong with my approach to solving the equations i.e. is re-calculating the Poisson equation at each time step valid and substituting the electric field into the advection-diffusion equations a valid approach?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-03-21T15:16:17.780" LastActivityDate="2013-03-21T15:16:17.780" Title="Are the drift-diffusion equations from semiconductor physics analogous to solving an advection-diffusion problem?" Tags="|pde|advection|diffusion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5585" PostTypeId="2" ParentId="391" CreationDate="2013-03-18T13:21:01.773" Score="3" Body="&lt;p&gt;I don't know if you're still interested in an answer, but here I go anyway:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You already said you know about shock formation in nonlinear equations. That is exactly why you have to choose your time integrator carefully. It's no use applying a TVD spatial discretization when the time discretization is not - you will see the same oscillations you have probably seen with higher order numerical fluxes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it boils down to is that forward Euler works. You already mentioned SSP (strong stability preserving) in your question. This is a special class of Runge-Kutta methods that makes use of that. Basically, you have to choose the coefficients of the method in such a way that it can be written as a convex combination of Euler steps. That way, properties like TVD and such will be preserved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a very good book on SSP methods by Gottlieb, Ketcheson and Shu called &quot;Strong Stability Preserving Runge-Kutta and Multistep Time Discretizations&quot; &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/9814289264&quot; rel=&quot;nofollow&quot;&gt;amazon link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3914" LastActivityDate="2013-03-18T13:21:01.773" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5586" PostTypeId="1" CreationDate="2013-03-18T13:39:29.347" Score="7" ViewCount="194" Body="&lt;p&gt;I know from &quot;Numerical Methods for Conservation Laws&quot; by Randall J. LeVeque that there is an implication chain of properties of methods for conservation laws:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;monotone $\Rightarrow$ $L^1$-contractive $\Rightarrow$ TVD $\Rightarrow$ monotonicity preserving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the conservation of mass, positivity preservation is most important, but how does it fit into this chain? Does monotonicity preservation imply positivity preservation or vice-versa? A lot of people seem to use both terms as equivalent, but I'm not so sure that's true...&lt;/p&gt;&#xA;" OwnerUserId="3914" LastEditorUserId="123" LastEditDate="2013-03-19T18:37:54.600" LastActivityDate="2015-06-06T02:45:53.400" Title="How does positivity preservation fit into the implication chain from monotone to monotonicity preserving?" Tags="|hyperbolic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5587" PostTypeId="1" CreationDate="2013-03-18T13:59:05.740" Score="6" ViewCount="736" Body="&lt;p&gt;I was looking at this wikipedia page: &#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference_coefficient&quot; rel=&quot;noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Finite_difference_coefficient&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a lists of higher order finite difference approximations, is there any negatives in using these apart from the extra calculations that they require to solve.&lt;/p&gt;&#xA;" OwnerUserId="3939" LastEditorUserId="3939" LastEditDate="2013-03-18T14:06:01.883" LastActivityDate="2013-03-22T20:57:58.120" Title="What are the negatives of using higher order finite diference schemes?" Tags="|finite-difference|stability|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5588" PostTypeId="2" ParentId="5584" CreationDate="2013-03-18T14:23:52.150" Score="0" Body="&lt;p&gt;(I realised the issue, after having a discussion offline).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is yes, and the equations are correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error is in assuming that the electron and hole densities should remain constant when both halves of the pn-junction are brought together. Obviously when the free carrier flow into each other under diffusion they &quot;annihilate&quot; causing a change in carrier number. This will continue until the divergence of the electric field becomes zero at equilibrium. Provided the electric field is calculated self-consistently with the Poisson equation, the resulting transient is physically correct.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-03-19T00:29:02.100" LastActivityDate="2013-03-19T00:29:02.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5589" PostTypeId="2" ParentId="5322" CreationDate="2013-03-18T18:42:20.140" Score="0" Body="&lt;p&gt;You have a so called grey-box system identification problem. If you have the System Identification Toolbox installed, you are basically done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.se/products/sysid/description5.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.se/products/sysid/description5.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.se/help/ident/ug/estimating-nonlinear-grey-box-models.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.se/help/ident/ug/estimating-nonlinear-grey-box-models.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-03-18T18:42:20.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5591" PostTypeId="1" CreationDate="2013-03-18T20:43:20.253" Score="1" ViewCount="144" Body="&lt;p&gt;I have the following algorithm given:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input: Regular Matrix $A \in \mathbb R^{n,n}$&#xA;Output: LU-Decomposition of  A = LU&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;for&lt;/strong&gt; k = 1, . . . , n &lt;strong&gt;do&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;for&lt;/strong&gt; j = k, . . . , n &lt;strong&gt;do&lt;/strong&gt;&#xA;$r_{kj} = a_{kj} − \sum_{i=1}^{k-1} l_{ki}r_{ij}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;end for&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;for&lt;/strong&gt; i = k + 1, . . . , n &lt;strong&gt;do&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$l_{ik} = (a_{ik} − \sum_{j=1}^{k-1} l_{ij}r_{jk})/r_{kk}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;end for&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;end for&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given every elementary operation (+,-,*,/) has the cost 1, how can it be derived that the complexity of this algorithm is &lt;strong&gt;2/3n^3 - 1/2n^2 - 1/6n&lt;/strong&gt;? I am really interested in understanding how such a closed formulae for the complexity is derived.&lt;/p&gt;&#xA;" OwnerUserId="3942" LastActivityDate="2013-03-19T11:59:14.223" Title="How to calculate the complexity of a given Algorithm" Tags="|algorithms|complexity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5592" PostTypeId="2" ParentId="5587" CreationDate="2013-03-19T00:07:47.027" Score="5" Body="&lt;p&gt;The most prominent drawback of high-order methods is that when the solution is not smooth, high order methods suffer from &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge_phenomenon&quot; rel=&quot;noreferrer&quot;&gt;Runge's phenomenon&lt;/a&gt;. These oscillations are often informally referred to as &quot;instability&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementation difficulties, especially with complicated boundaries, is probably the second most important detractor besides efficiency. Many ways to reach high order are computationally impractical, though some classes (e.g., spectral element) can be very efficient. Very high order FD methods are most commonly used for &lt;a href=&quot;http://en.wikipedia.org/wiki/Direct_numerical_simulation&quot; rel=&quot;noreferrer&quot;&gt;DNS&lt;/a&gt; in regular domains.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-03-19T00:07:47.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5593" PostTypeId="1" AcceptedAnswerId="5611" CreationDate="2013-03-19T00:38:19.757" Score="4" ViewCount="496" Body="&lt;p&gt;&lt;em&gt;Hi All, I posted this on the math.se site, but this may be a better location.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a method of finding the maximum of a real valued trigonometric polynomial where I can trade accuracy for speed. The accepted answer to this question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://mathoverflow.net/questions/35538/the-maximum-of-a-real-trigonometric-polynomial&quot;&gt;https://mathoverflow.net/questions/35538/the-maximum-of-a-real-trigonometric-polynomial&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gives a method using semidefinite programming:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let $f(x)=F(e^{ix})$ where $F(z)=\sum_{n=-N}^{N}c_n z^n$, with $c_n=\tfrac{1}{2}(a_n−i b_n)$ and&#xA;  $c_{−n}=\bar{c}_n$. Then $\min_x \,f(x)$ is equal to $c_0$ minus the value of the following&#xA;  semidefinite program: $\min_F tr(F)$ such that $F⪰0$, and $\sum_{p=k}^{N} F_{p,p−k}=c_k$ for&#xA;  $k=1,…,N$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, I don't understand what this means. Would anyone be able to explain in simpler terms and give an idea of how one would code this in MATLAB?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some clarifications:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How does one implement a semidefinite program? Broad question I know! Perhaps better is what MATLAB package is best suited to this kind of semidefinite program problem? $N$ will be  between 3 and 15 so I would like a quick solver for small matrices.&lt;/li&gt;&#xA;&lt;li&gt;In the problem $F(z)$ is a function and $F$ is a matrix (I assume they are different)? When the SDP solves for $F$, how do I get the value of $z$ and $F(z)$ that corresponds to the maximum of $F(z)$?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3565" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2013-04-23T15:16:48.247" Title="How to implement this trigonometric polynomial maximum finding semidefinite program" Tags="|semidefinite-programming|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5594" PostTypeId="2" ParentId="5591" CreationDate="2013-03-19T02:08:44.627" Score="2" Body="&lt;p&gt;In order to compute $r_{kj}$, you need to do $k-1$ multiplications and $k$ additions, for a total of $2k-1$ operations. You need to do this for $j=k...n$, i.e. $n-k$ times for a total of $(n-k)(2k-1)$ operations to compute all of the $r_{kj}$ for a given $k$. Then you need to do this for all $k$ from 1 to $n$, so the cost of this first loop is&#xA;$$&#xA;  \sum_{k=1}^n (n-k)(2k-1)&#xA;= n\sum_{k=1}^n (2k-1) - \sum_{k=1}^n k(2k-1)&#xA;\\&#xA;= 2n\sum_{k=1}^n k - \sum_{k=1}^n 1 - 2 \sum_{k=1}^n k^2 + \sum_{k=1}^n k&#xA;\\&#xA;= 2n\frac 12 n(n+1) - n - \text{something} + \frac 12 n(n+1).&#xA;$$&#xA;You need to look up the formula for the third sum: I forgot its exact value, but it's proportional to $n^3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do the same calculation for the second inner loop of your algorithm.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-03-19T11:59:14.223" LastActivityDate="2013-03-19T11:59:14.223" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5595" PostTypeId="2" ParentId="5587" CreationDate="2013-03-19T02:13:32.390" Score="3" Body="&lt;p&gt;As an extension of Jed's post, higher order finite difference methods often compute the value at one node based not only on the immediate neighbors but also other nodes far away. This is a problem if the current node is close to the boundary (whether the boundary is regular or not): In that case, the nearest neighbor of the current node may be a boundary node, in which case you would know its value if you have Dirichlet boundary conditions, but the farther away nodes the current node depends on may not lie inside the domain any more. What do you do when you need these nodes' values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A separate problem is that the stencils are rather large because they couple so many nodes. This means that the matrices are less sparse than for lower order methods, with the resulting increase in computational effort in solving linear systems.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-19T02:13:32.390" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5596" PostTypeId="2" ParentId="5584" CreationDate="2013-03-19T02:18:47.700" Score="4" Body="&lt;p&gt;The answer is yes. Drift diffusion = advection-diffusion. You will find this rather frequently: different disciplines, trying to describe the phenomena in their field, assign names to the equations they derive that have meaning for this discipline. Other disciplines come up with the same equations, in different contexts, and name them differently. It's still the same equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Another example: Neutron diffusion approximation == Poisson equation.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-19T02:18:47.700" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5597" PostTypeId="2" ParentId="5579" CreationDate="2013-03-19T02:22:09.473" Score="5" Body="&lt;p&gt;It's not a question whether the condition holds or not. You choose $R$ so that it's a multiple of $I^T$ because you &lt;em&gt;want&lt;/em&gt; the condition to hold. You want the condition to hold because that's the only way you can ensure that the multigrid operator is symmetric and amenable to, say, being a preconditioner for CG.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-19T02:22:09.473" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5599" PostTypeId="2" ParentId="5322" CreationDate="2013-03-19T08:29:42.297" Score="0" Body="&lt;p&gt;If the parameter to be found is a single number, then you can do brute-force prediction calculations based on a lot of K's (as Geoff suggests), then plot the difference between computational predictions and measured data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives a &quot;complete picture&quot; of how changing the parameter will influence all the measurements, and is not that costly since solving thousands of ODE's takes only a few minutes on modern computers. You can then visually inspect the results to see what K to choose and how sensitive the results are to changes in K - ie., how certain/uncertain you should be about your result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, you can do some simple least-squares or statistical analysis on the results if you want, but you'll probably get a better idea of what's going on by just looking at it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's some code to do the brute force calculations and plotting! I release this code to the public domain.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Some randomish input data (replace with your times tt and function f)&#xA;tt=0:.01:1;&#xA;Ktrue = 2.31; &#xA;ff =  ifft(fft(exp(randn(1,length(tt)))).* ...&#xA;              fft(exp(-(tt-5).^2/(1.5)^2)))';&#xA;ff = ff/mean(ff);&#xA;f = @(t) interp1(tt,ff,t);&#xA;&#xA;% manufacturing state data X1,X2 (replace with your X1,X2)&#xA;X1initial = 0.93; &#xA;X2initial = 1.87;&#xA;odefuntrue = @(t,Y) [f(t)*Y(1)*Y(2), -Ktrue*f(t)*Y(1)*Y(2)]';&#xA;[T,Ytrue] = ode45(odefuntrue,tt,[X1initial,X2initial]);  &#xA;X1 = Ytrue(:,1); &#xA;X2 = Ytrue(:,2);&#xA;&#xA;% Solve the ODE for a lot of different values of K&#xA;K=0.01:0.01:5;&#xA;Ts = zeros(length(tt),length(K)); &#xA;X1s = Ts; X2s = Ts;&#xA;for jj=1:length(K)&#xA;    odefun = @(t,Y) [f(t)*Y(1)*Y(2), -K(jj)*f(t)*Y(1)*Y(2)]';&#xA;    [T,Y] = ode45(odefun,tt,[X1(1),X2(1)]); &#xA;    X1s(:,jj)=Y(:,1);&#xA;    X2s(:,jj)=Y(:,2);&#xA;end&#xA;&#xA;% Compute the difference between the measured state, (X1,X2), and&#xA;% the predicted state for all different K's, (X1s,X2s)&#xA;errors1 = X1*ones(1,length(K)) - X1s;&#xA;errors2 = X2*ones(1,length(K)) - X2s;&#xA;&#xA;% Plot the full errors at all times for different values of K&#xA;figure;&#xA;subplot(1,2,1); &#xA;imagesc(K,tt,errors1);&#xA;title('error in X1');&#xA;xlabel('K');&#xA;ylabel('t');&#xA;colorbar;&#xA;&#xA;subplot(1,2,2); &#xA;imagesc(K,tt,errors2);&#xA;title('error in X2');&#xA;xlabel('K');&#xA;ylabel('t');&#xA;colorbar;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here's the results. You can see how the error is zero along the vertical line $K=2.31$ corresponding to the true $K$, and the horizontal line $t=0$, and becomes nonzero as $K$ changes away from the true value and time increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rBFHA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/VR9Vv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2013-03-19T08:50:44.900" LastActivityDate="2013-03-19T08:50:44.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5600" PostTypeId="1" CreationDate="2013-03-19T12:29:28.723" Score="10" ViewCount="10801" Body="&lt;p&gt;I am presently working on solving very large symmetric (but not positive definite) systems, generated by some certain algorithms. These matrices have a nice block sparsity which can be used for parallel solving. But I can't decide whether I should be using a direct approach (like Multi-frontal) or iterative one (preconditioned GMRES or MINRES). All my studies show that iterative solvers (even with pretty fast convergence of 7 inner iterations) fail to beat the direct '\' operator in MATLAB. But in theory, direct methods are supposed to be costlier. How is this happening? Is there any up to date document or paper for such case? Can I use block sparsity in parallel systems using direct methods as efficiently as flexible iterative solvers like GMRES?&lt;/p&gt;&#xA;" OwnerUserId="2019" LastEditorUserId="3980" LastEditDate="2013-05-08T18:29:03.157" LastActivityDate="2020-09-03T16:34:56.950" Title="Best choice of solver for a large sparse symmetric (but not positive definite) system" Tags="|parallel-computing|linear-solver|sparse-matrix|linear-system|gmres|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5601" PostTypeId="2" ParentId="5593" CreationDate="2013-03-19T13:50:23.233" Score="1" Body="&lt;p&gt;It's not clear to me what you're asking for here.  Are you looking for a derivation of the semidefinite programming (SDP) formulation of the problem?  Are you looking for an explanation of what SDP is?  Are you looking for information on how to solve the SDP once you've got it?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB doesn't have any built-in functions for solving semidefinite programming problems.  There are a number of open source packages that you can install that will add this capability to MATLAB.  Look at SeDuMi, SDPT3, and CSDP.  A quick google search will lead you to each of these packages.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-03-19T13:50:23.233" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5602" PostTypeId="1" AcceptedAnswerId="5616" CreationDate="2013-03-19T17:12:32.517" Score="3" ViewCount="1281" Body="&lt;p&gt;I'm trying to solve a system non linear-equations:&#xA;$$&#xA;\frac{\partial K(\mathbf{\lambda})}{\partial \lambda_i} - c_i = 0&#xA;$$&#xA;for $i = 1, \dots, 15$, using Newton's method:&#xA;$$&#xA;\lambda^{k + 1} = \lambda^k + p^{k+1}&#xA;$$&#xA;$$&#xA;p^{k+1} = - J(\lambda_k)^{-1} \big (\nabla_\lambda K(\lambda_k) - \mathbf{c}\big ).&#xA;$$&#xA;I know that the Jacobian has to be positive definite in my problem, but in practice it is often quite bad conditioned ( $k(J) \approx 10^{15}$ ). I am perturbing it's eigenvalues but still the algorithm doesn't manage to satisfy the convergence &#xA;criterion:&#xA;$$&#xA;\bigg |\frac{\partial K(\mathbf{\lambda})}{\partial \lambda_i} - c_i \bigg |= 0&#xA;$$&#xA;for some of the equations, while others converge quite rapidly. Increasing the number of iterations doesn't change anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My supervisor suggested me to use a blocked scheme where, if some of the equations have converged you keep iterating on the equations that have not. &#xA;I don't quite understand how to use Newton's algorithm on a subgroup of equations so I would like to ask if somebody knows how this can be done. Alternatively I would appreciate suggestions on how to deal with this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I must add that I'm using line search with objective function:&#xA;$$&#xA;f(\lambda) = K(\mathbf{\lambda}) - \mathbf{\lambda}^T \mathbf{c}&#xA;$$&#xA;which should be fine because it is convex (in theory though because the Hessian is almost singular). I'm checking whether the step is decreasing the objective enough, that is I check:&#xA;$$&#xA;f(\lambda^k + \alpha p^k) &amp;lt; f(\lambda^k) + \gamma \, \alpha \, (\nabla_\lambda K(\lambda_k) - \mathbf{c}\big )^T p^k &#xA;$$&#xA;with $\gamma = 10^{-4}$. If the condition is not true I divide $\alpha$ by 2. Most of the time the condition is true in the first step, so I rarely have to halve the step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to be converging if I use a QR decomposition to solve the linear system, without perturbing the eigenvalues of $J$. Not sure whether I should trust the output though! &lt;/p&gt;&#xA;" OwnerUserId="2140" LastEditorUserId="2140" LastEditDate="2013-03-20T16:28:20.610" LastActivityDate="2013-03-21T03:46:05.487" Title="Non-linear root finding when the Jacobian is almost singular" Tags="|nonlinear-equations|condition-number|newton-method|roots|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5603" PostTypeId="1" AcceptedAnswerId="5615" CreationDate="2013-03-19T19:47:33.677" Score="11" ViewCount="18954" Body="&lt;p&gt;in matlab, both &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/linsolve.html&quot;&gt;linsolve&lt;/a&gt; and &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/mldivide.html&quot;&gt;mldivide&lt;/a&gt; are used for solving a system of linear equations, in all of determined, overdetermined and underdetermined cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading their documents, I was wondering what differences are between them? Are they using almost the same algorithms of matrix factorization and triangularization in the three cases?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If A has the properties in opts, linsolve is faster than mldivide, because linsolve does not perform any tests to verify that A has the specified properties&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does mldivide perform the same tests to verify if A has the special properties? Or does mldivide just treat them as general case without the special properties?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="258" LastEditDate="2013-03-19T19:54:07.057" LastActivityDate="2013-03-20T19:58:09.953" Title="in matlab, what differences are between linsolve and mldivide?" Tags="|matlab|linear-solver|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5604" PostTypeId="1" CreationDate="2013-03-20T14:11:07.750" Score="22" ViewCount="2275" Body="&lt;p&gt;Most numerical methods for quadrature treat the integrand as a black-box function.  What if we have more information?  In particular, what benefit, if any, can we derive from knowing the first few derivatives of the integrand?  What other information might be valuable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For derivatives in particular: error estimates for basic quadrature (rectangle/trapzoid/simpson's rules) are closely related.  Perhaps there is a way to pre-select sampling resolution instead of relying on dynamic adaptivity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in both the univariate and multi-dimensional case.&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2013-03-21T05:02:25.757" LastActivityDate="2016-06-08T15:02:13.433" Title="Numeric Quadrature with Derivatives" Tags="|quadrature|error-estimation|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5605" PostTypeId="2" ParentId="5604" CreationDate="2013-03-20T14:41:48.680" Score="21" Body="&lt;p&gt;I think this is not quite what you had in mind, but for the sake of completeness, let's start with some basics. Most quadrature formulas such as Newton-Cotes and Gauss are based on the idea that in order to evaluate the integral of a function approximately, you can approximate the function by, e.g., a polynomial that you can then integrate exactly:&#xA;$$ \int_a^b f(x) \,dx \approx \int_a^b \sum_j c_j p_j(x)\,dx = \sum_j c_j\int_a^b p_j(x) \,dx. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Newton-Cotes and Gauss are based on &lt;em&gt;Lagrange interpolation&lt;/em&gt;, meaning you interpolate the given function using its values on a set of nodes $x_j$ (which are spaced uniformly for Newton-Cotes and chosen optimally in a certain sense for Gauss). In this case, $c_j = f(x_j)$, and the integrals over the polynomial nodal basis functions $p_j$ are exactly the quadrature weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same approach works with &lt;em&gt;Hermite interpolation&lt;/em&gt;, i.e., interpolation using the values of a function and its derivatives up to a certain order on a set of nodes. In the case of function and first derivative values only, you have&#xA;$$ \int_a^b f(x) \,dx \approx \int_a^b \sum_j f(x_j) p_j(x) + f'(x_j) q_j(x)\,dx = \sum_j f(x_j) w_j+f'(x_j) \bar w_j. $$&#xA;(There is a &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/8836-hermite-quadrature&quot;&gt;Matlab implementation&lt;/a&gt; of this, if you'd like to see how it works.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is related to a variant of Gauss quadrature called Gauss-Legendre quadrature, where the nodes are chosen precisely to make the weights $\bar w_j$ vanish (which is another explanation for the fact that Gauss quadrature with $N$ nodes is exact of order $2N-1$). I think this at least partially answers your question in the second paragraph.&#xA;For this reason, Gauss quadrature is usually used instead of Hermite interpolation, since you get the same order with the same number of points, but do not need derivative information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For multidimensional quadrature, you face the problem that the number of derivatives (including mixed derivatives) you need to evaluate grows very quickly as the order increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming back to your question: A straightforward way of exploiting derivative information would be to use a subdivision of your integration domain and use a separate quadrature for every division. If you know that the derivatives of your function are large in some part of the domain, you'd use either smaller domains (in effect, a summed quadrature formula) or higher quadrature order. This is related to &lt;em&gt;h-&lt;/em&gt; and &lt;em&gt;p-adaptivity&lt;/em&gt;, respectively, in finite element methods.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-03-20T15:05:57.230" LastActivityDate="2013-03-20T15:05:57.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5606" PostTypeId="2" ParentId="5593" CreationDate="2013-03-20T14:58:12.887" Score="3" Body="&lt;p&gt;If you install YALMIP (a modelling layer I develop) and a semidefinite solver (for instance SDPT3), the code would be, if I interpreted it correctly.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F = sdpvar(N+1,N+1,'hermitian','complex')&#xA;Constraints = F &amp;gt;= 0;&#xA;for k = 1:N&#xA;     Constraints = [Constraints, sum(diag(F,k)) == c(k+1)];&#xA;end&#xA;solvesdp(Constraints,trace(F));&#xA;double(F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note the shift in indices in F. It looks like the matrix F uses 0-based indexing which clashes with MATLABs 1-based.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.HomePage&quot; rel=&quot;nofollow&quot;&gt;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.HomePage&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.nus.edu.sg/~mattohkc/sdpt3.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.nus.edu.sg/~mattohkc/sdpt3.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="3565" LastEditDate="2013-04-23T15:16:48.247" LastActivityDate="2013-04-23T15:16:48.247" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="5607" PostTypeId="5" CreationDate="2013-03-20T14:59:13.047" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-03-20T14:59:13.047" LastActivityDate="2013-03-20T14:59:13.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5608" PostTypeId="4" CreationDate="2013-03-20T14:59:13.047" Score="0" Body="a MATLAB-based modeling framework for convex optimization." OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-03-20T17:12:55.280" LastActivityDate="2013-03-20T17:12:55.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5610" PostTypeId="1" CreationDate="2013-03-20T15:40:50.457" Score="3" ViewCount="3670" Body="&lt;p&gt;How would I find the definite integral (between any 2 limits, say a and b) of the absolute value of sin(x)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can calculate for the interval 0 to Pi, and from 0 to 2*Pi, but what if the user enters a value far outside 2*Pi? Could I create a while loop?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Here is a look at my code and its purpose. I have left my analytical function blank for now because I am still working on it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/*This program computes the area between the sine curve and the x-axis&#xA;It estimates the area using the Trapezoidal method and Simpson's 1/3 rule&#xA;and compares all three values */&#xA;&#xA;#include &amp;lt;iostream&amp;gt; //preprocessor directive needed in order to use std::cout&#xA;                    //and std::cin&#xA;#include &amp;lt;iomanip&amp;gt; //preprocessor directive needed in order to use&#xA;                   //a manipulator which uses an argument&#xA;#include &amp;lt;cmath&amp;gt;   //preprocessor directive needed in order to use trig, exp and&#xA;                   //log functions&#xA;&#xA;using namespace std; //avoids having to uses std:: with cout and cin&#xA;&#xA;//declare the variables&#xA;double a, b;&#xA;int n;&#xA;&#xA;//prototypes for f(), analytical(), and integrateTrap()&#xA;double f (double x);&#xA;double analytical (double a, double b);&#xA;double integrateTrap (double a, double b, int n);&#xA;double integrateSimpson (double a, double b, int n);&#xA;&#xA;int main (int argc, char* argv[])&#xA;{  &#xA;    //tell the user what the program does&#xA;    cout &amp;lt;&amp;lt; &quot;The program estimates the integral of sin(x) between\n the limits&quot;&#xA;    &quot;[a,b] using both the Trapezoidal method and Simpson's 1/3 rule&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;    cout &amp;lt;&amp;lt; &quot;\n\nEnter the lower limit a as a floating point number: &quot;;&#xA;    cin &amp;gt;&amp;gt; a;&#xA;    cout &amp;lt;&amp;lt; &quot;\n\nEnter the upper limit b as a floating point number: &quot;;&#xA;    cin &amp;gt;&amp;gt; b;&#xA;    cout &amp;lt;&amp;lt; &quot;\n\nEnter the number of intervals (an integer&amp;gt;0): &quot;;&#xA;    cin &amp;gt;&amp;gt; n;&#xA;&#xA;    //call integrateTrap()&#xA;    cout &amp;lt;&amp;lt; &quot;\n\nThe estimate of the interval between the limits [&quot; &amp;lt;&amp;lt; a &amp;lt;&amp;lt; &quot;, &quot;&#xA;    &amp;lt;&amp;lt; b &amp;lt;&amp;lt; &quot;] using \nthe Trapezoidal rule = &quot; &amp;lt;&amp;lt; integrateTrap(a,b,n) &amp;lt;&amp;lt; endl;&#xA;&#xA;    //call integrateSimpson()&#xA;    cout &amp;lt;&amp;lt; &quot;\n\nThe estimate of the interval between the limits [&quot; &amp;lt;&amp;lt; a &amp;lt;&amp;lt; &quot;, &quot;&#xA;    &amp;lt;&amp;lt;b&amp;lt;&amp;lt; &quot;] using\nSimpson's 1/3 rule = &quot; &amp;lt;&amp;lt; integrateSimpson (a,b,n) &amp;lt;&amp;lt; endl;&#xA;&#xA;    //call analytical()&#xA;    cout &amp;lt;&amp;lt; &quot;\n\nThe analytical answer = &quot; &amp;lt;&amp;lt; analytical(a,b) &amp;lt;&amp;lt; endl;&#xA;&#xA;    //allow the user to see the results before ending the program&#xA;    cout &amp;lt;&amp;lt; &quot;\nPress Enter to end the program&quot;;&#xA;    cin.get();&#xA;    cin.get();&#xA;}&#xA;&#xA;///////////////////////////////////////////////////////////////////////////////&#xA;&#xA;//define sin(x)&#xA;double f (double x)&#xA;{&#xA;       return abs(sin(x));&#xA;}&#xA;&#xA;////////////////////////////////////////////////////////////////////////////////&#xA;&#xA;//define analytical() to be the answer to the definite integral&#xA;double analytical(double a, double b)&#xA;{&#xA;&#xA;}&#xA;&#xA;////////////////////////////////////////////////////////////////////////////////&#xA;&#xA;//define integrateTrap()&#xA;double integrateTrap (double a, double b, int n)&#xA;{&#xA;       //declare and initialise the variables&#xA;       double h = (b-a)/n;&#xA;       double x = a;&#xA;       double sum = 0.0; &#xA;&#xA;       //sum up the area of each interval&#xA;       for (int i = 1; i &amp;lt;= n; i++) &#xA;       {&#xA;           x = a + i*h;&#xA;           sum += f(x);    &#xA;       }&#xA;       return (h/2)*(f(a)+ 2*sum + f(b));     &#xA;}&#xA;&#xA;////////////////////////////////////////////////////////////////////////////////&#xA;&#xA;//define integrateSimpson()&#xA;double integrateSimpson (double a, double b, int n)&#xA;{&#xA;       //declare and initialise the variables&#xA;       double h = (b-a)/n;&#xA;       double even = 0.0, odd=0, x;&#xA;       int i;&#xA;&#xA;       //sum up the area of each interval&#xA;       for (i = 0, x=a+h; i&amp;lt;n; x=x+h, i++) &#xA;       {&#xA;            if(i % 2 == 0)&#xA;            {&#xA;                 even += f(x);&#xA;            }&#xA;            else&#xA;            {&#xA;                 odd += f(x);&#xA;            }&#xA;&#xA;       }  &#xA;       return (h/3)*(f(a)+ 2*even + 4*odd + f(b));&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3954" LastEditorUserId="348" LastEditDate="2013-08-19T13:59:55.643" LastActivityDate="2020-05-23T19:13:37.260" Title="In C++, how do you calculate the analytical value of $\int_a^b \left|\sin x \right|\,dx$?" Tags="|c++|quadrature|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5611" PostTypeId="2" ParentId="5593" CreationDate="2013-03-20T15:44:09.503" Score="4" Body="&lt;p&gt;&lt;em&gt;I would not recommend using SDP to solve this problem.&lt;/em&gt; If &lt;code&gt;roots&lt;/code&gt; is already obtaining the correct answer, I think you should stick with it or find some way to accelerate that approach; because the SDP method will be slower and less accurate. But just in case I am wrong about this, or just in case you insist on trying for other reasons, let me continue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Johan's package, YALMIP/SDPT3 is certainly a good choice for this semidefinite program. So is CVX, which (full-disclosure) I co-authored. I don't think Johan got the constraints quite right, but the error is minor and I'm sure that will get fixed in due course. (EDIT: it has been fixed.) In the meanwhile I will share the CVX model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that you have stored the positive coefficients of $c$ in a length-$N+1$ MATLAB vector &lt;code&gt;c&lt;/code&gt;, so that $c_k$ is in &lt;code&gt;c(k+1)&lt;/code&gt; (because MATLAB indexes from 1, not 0). Then assuming the theorem you've cited is correct, the following CVX model should solve it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cvx_begin sdp&#xA;    variable F(N+1,N+1) Hermitian&#xA;    maximize(c(1)-trace(F))&#xA;    for k = 1 : N,&#xA;        sum(diag(F,k)) == c(k+1)&#xA;    end&#xA;    F &amp;gt;= 0&#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the objective function includes the $c_0$ term, and is subtracting the trace of $F$, so that step is included. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A disclaimer: while I'm sure I'm accurately representing the model you have stated in your question, I'm not 100% convinced that the model itself is correct. It's close, for sure---I'm familiar with the papers in question---but you would want to double-check if you go down this route.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDITED TO ADD: Readers interested in potential applications for this type of model may wish to consult &lt;a href=&quot;http://arxiv.org/abs/1203.5871&quot; rel=&quot;nofollow&quot;&gt;this recent paper&lt;/a&gt;, &quot;Towards a Mathematical Theory of Super-Resolution&quot;, by Candès and Fernandez-Granda. They describe the use of trigonometric polynomial optimization to solve continuous-frequency sparse recovery applications.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-03-20T19:13:56.997" LastActivityDate="2013-03-20T19:13:56.997" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5612" PostTypeId="1" AcceptedAnswerId="5619" CreationDate="2013-03-20T15:53:29.743" Score="3" ViewCount="161" Body="&lt;p&gt;Do &lt;a href=&quot;http://en.wikipedia.org/wiki/Ellipsoid_method&quot; rel=&quot;nofollow&quot;&gt;the ellipsoid methods&lt;/a&gt; belong to &lt;a href=&quot;http://reference.wolfram.com/mathematica/tutorial/UnconstrainedOptimizationTrustRegionMethods.html&quot; rel=&quot;nofollow&quot;&gt;the  trust region methods&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading their descriptions I tend to think the idea of the ellipsoid methods belong to the idea of the trust region methods, but am not sure if I am correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are their relations and differences then?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastActivityDate="2013-03-21T02:44:59.940" Title="Do the ellipsoid methods belong to the trust region methods?" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5613" PostTypeId="2" ParentId="5447" CreationDate="2013-03-20T15:59:49.177" Score="2" Body="&lt;p&gt;If you only consider linear equations, you can use rather simple methods. Lax-Wendroff is a nice example of a second order method, but you could even use an upwind-based method, which is first order and diffusive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with hyperbolic conservation laws comes with nonlinearity. So let's consider&#xA;$u_t + f(u)_x=0$,&#xA;which means $f^\prime=c$ in your example. If $c$ is constant (ie. $f$ is linear), you have linear transport, so your initial data will be transported with velocity $c$. For nonlinear $f$, however, even smooth initial data can lead to discontinuities. Burger's equation is a classic example: $u_t+uu_x=0$ or $u_t+\frac12(u^2)_x=0$. If you look at the characteristics, you see what happens. Suppose we have piecewise constant initial data. If their value on left is greater than on the right, the part on the left travels to the right faster than the rest and this will end up in a shock. If it's the other way around, you have a rarefaction wave where both parts travel away from each other. But shocks are the greater problem as far as high order is concerned. Near shocks or steep gradients, order of accuracy doesn't make sense because it involves derivatives (Taylor series). &#xA;I already mentioned that the upwind method is diffusive, but it can deal with shocks. Higher order ($\ge2$) methods develop spurious oscillations.&#xA;This is why people invented flux limiters and the like. You use your high order method where possible and drop to the safe first order where necessary. There's a lot of theory out there. One property many people want to have is TVD (total variation diminishing). Since you already mentioned the wikipedia site, you have probably seen that. Not all of those flux limiters are TVD, but of those mentioned in that article, superbee is optimal in the sense that it allows the highest possible use of the high order method. Van Leer and Osher are other widely used limiters. If you want to go beyond second order, you should take a look at &lt;a href=&quot;http://en.wikipedia.org/wiki/MUSCL_scheme&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/MUSCL_scheme&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as your question about implicit schemes is concerned, from the theory of nonlinear stability, you usually get a CFL-like condition anyway. The only unconditionally stable (in the strong, nonlinear sense) is backward Euler. There is an excellent book on &quot;Strong Stability Preserving Time Discretizations&quot; by Gottlieb, Ketcheson and Shu. This is the kind of method you want to use for nonlinear problems. The book contains a theorem saying no Runge-Kutta method of order greater than one is unconditionally SSP. Plus, the effective CFL number is probably bounded by two, so you really wouldn't gain anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You see, nonlinear hyperbolic equations are a lot different from what you know about simple ODEs. If you want to learn more, I recommend the book &quot;Finite Volume Methods for Hyperbolic Problems&quot; by Randall J. LeVeque. It covers the theory as well as a wide range of numerical methods (including flux limiters).&lt;/p&gt;&#xA;" OwnerUserId="3914" LastActivityDate="2013-03-20T15:59:49.177" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5614" PostTypeId="1" AcceptedAnswerId="5618" CreationDate="2013-03-20T18:25:12.823" Score="1" ViewCount="863" Body="&lt;p&gt;I'm writing finite-difference method program using C. The stiffness matrix is symmetrical and band. For its storage I'd like to use Sparse Diagonal Storage format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone tell please, what solvers can use diagonal storage format? Intel MKL's Paradiso uses only CRS-format, SparseLib doesn't have, Spooles seems also =(&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="3956" LastActivityDate="2013-03-21T18:09:50.090" Title="What sparse solver supports diagonal storage format" Tags="|sparse-matrix|c|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="5615" PostTypeId="2" ParentId="5603" CreationDate="2013-03-20T19:17:59.643" Score="19" Body="&lt;p&gt;Both of them are direct solver to solve linear systems (opposing to iterative solver). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;mldivide&lt;/code&gt; does perform the tests for $A$ in solving $Ax = b$. Please see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1001/how-does-the-matlab-backslash-operator-solve-ax-b-for-square-matrices&quot;&gt;Allan's answer in this thread&lt;/a&gt; for more information.  Also see MATLAB's help on &lt;a href=&quot;http://www.mathworks.com/help/matlab/math/systems-of-linear-equations.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;mldivide&lt;/code&gt; algorithm&lt;/a&gt; here. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;mldivide&lt;/code&gt; for square matrices: If A is symmetric and has real, positive diagonal elements, MATLAB attempts a Cholesky factorization. If the Cholesky factorization fails, MATLAB performs a symmetric, indefinite factorization. If A is upper Hessenberg, MATLAB uses Gaussian elimination to reduce the system to a triangular matrix. If A is square but is neither permuted triangular, symmetric and positive definite, or Hessenberg, then MATLAB performs a general triangular factorization using LU factorization with partial pivoting&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;linsolve&lt;/code&gt; for square matrices: LU factorization with partial pivoting&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;mldivide&lt;/code&gt; and &lt;code&gt;linsolve&lt;/code&gt; for rectangular matrices: QR factorization&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;linsolve&lt;/code&gt; as the help doc suggests in mathworks website, you could avoid the extra testing process (Allan used the word &quot;overhead&quot; in his answer) by using &lt;code&gt;opts&lt;/code&gt; if and only if you &lt;em&gt;know&lt;/em&gt; what $A$ is like in advance. For large problems, you could save some time. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;opts.POSDEF = true; linsolve(A,b,opts)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;would return $x$ if you know $A$ is positive definite in advance. However, an incorrectly chosen &lt;code&gt;opts&lt;/code&gt; leads to a wrong result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If certain criteria are met, &lt;code&gt;linsolve&lt;/code&gt; and &lt;code&gt;mldivide&lt;/code&gt; do utilize the same factorization process. For example, for a &lt;em&gt;dense&lt;/em&gt; positive definite system satisfying certain properties, or you have an overdetermined system and both perform least square fitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, &lt;code&gt;linsolve&lt;/code&gt; could also perform &lt;a href=&quot;http://www.mathworks.com/help/symbolic/linsolve.html&quot; rel=&quot;noreferrer&quot;&gt;symbolic computation&lt;/a&gt;. This is handy when you have a small underdetermined system which has infinite number of solutions. &lt;code&gt;linsolve&lt;/code&gt; enables you to solve it symbolically, &lt;code&gt;mldivide&lt;/code&gt; can not do that. However, if the variables are not declared symbolically, &lt;code&gt;mldivide&lt;/code&gt; and &lt;code&gt;linsolve&lt;/code&gt; would give you the same warning message &quot;Matrix is singular to working precision.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last but not least, &lt;code&gt;linsolve&lt;/code&gt; does not support &lt;em&gt;sparse&lt;/em&gt; systems like the following matrix (blue dot means non-zero entry). While &lt;code&gt;mldivide&lt;/code&gt; could handle sparse systems robustly when the size is under 200k by 200k.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/OUG5i.png&quot; alt=&quot;sparse example&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-03-20T19:58:09.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5616" PostTypeId="2" ParentId="5602" CreationDate="2013-03-20T20:02:50.380" Score="3" Body="&lt;p&gt;Assuming your jacobian isn't too large, it's a good idea to solve for your newton direction by QR factorization when your jacobian has a large condition number.  It is numerically stable since it breaks the problem into two separate systems with the minimum error amplification factor allowed by the original problem. See &lt;a href=&quot;http://www-old.math.gatech.edu/academic/courses/core/math2601/Web-notes/3num.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This enables you to obtain the &quot;best&quot; possible approximation of the newton direction at each step in your algorithm.  If you are using line-searching with backtracking and Armijo's sufficient decrease condition, this &lt;em&gt;should&lt;/em&gt; guarantee convergence to the true root (assuming your function $f$ is smooth).&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-03-21T03:46:05.487" LastActivityDate="2013-03-21T03:46:05.487" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5617" PostTypeId="1" AcceptedAnswerId="5665" CreationDate="2013-03-20T21:53:58.347" Score="5" ViewCount="1635" Body="&lt;p&gt;I read this post, &quot;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/5434/conservation-of-a-physical-quantity-when-using-neumann-boundary-conditions-appli&quot;&gt;Conservation of a physical quantity when using Neumann boundary conditions applied to the advection-diffusion equation&lt;/a&gt;&quot; and although it is the same type of equation it does not fit my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to solve the following equation with a no flux boundary with the solver d03pf from NAG. Should not be difficult, but i do have troubles with a no flux boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial(u)}{\partial(t)}+\frac{\partial(F(u))}{\partial(x)}=\frac{\partial}{\partial(x)}\left( D(u)\frac{\partial(u)}{\partial(x)}\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have a closed vessel and particles are sedimenting towards the bottom. Before they touch each other it is of hyperbolic type. Otherwise the PDE is parabolic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My flux function is something like $F=u\times\text{velocity}(1-u)^C$ So when i use the standard no flux function for the hyperbolic part, saying the flux at the boundary is zero. The solver sets the mass in the last cell to zero and i am loosing mass.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can say $\text{flux}(\text{end}-1) = -\text{flux}(\text{end})$ and the solver raises the concentration in the last cell greater one (for e.g C=5). But that does not work for even C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F(u)$ as well as $D(u)$ are piecewise differentiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have the space vector, the time at this iteration and the solution of the last three cells given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know another way constructing a no flux boundary ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best wishes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moritz&lt;/p&gt;&#xA;" OwnerUserId="3957" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-10-18T06:17:49.943" Title="No flux boundaries for mixed hyperbolic parabolic PDE" Tags="|pde|boundary-conditions|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="5618" PostTypeId="2" ParentId="5614" CreationDate="2013-03-21T01:45:03.437" Score="4" Body="&lt;p&gt;There are surprisingly few references to &quot;Sparse Diagonal Storage&quot; format on Google. &lt;a href=&quot;https://cug.org/5-publications/proceedings_attendee_lists/2008CD/S08_Proceedings/pages/Authors/01-5Monday/Leback-Monday3A/Leback-Monday3A-slides.PDF&quot; rel=&quot;nofollow&quot;&gt;Here is a PDF&lt;/a&gt; of some slides I found that provide a brief overview of Compressed Row, Jagged Diagonal, and Sparse Diagonal formats. Most of the other links are pointers to a single paper, &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/004579499390080W&quot; rel=&quot;nofollow&quot;&gt;Use of sparse diagonal storage in finite element analysis&lt;/a&gt; by &#xA;K.K. Yalamanchili and S.C. Anand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this isn't good news, really: It suggests that Sparse Diagonal Storage has not been adopted very widely. I'm going to hazard a guess: the format is probably more useful for &lt;em&gt;iterative&lt;/em&gt; methods than for &lt;em&gt;direct&lt;/em&gt; (factorization) methods. Implementing sparse matrix-vector multiplication is relatively straightforward for any of these formats, so for iterative methods you have the freedom to store as you wish (at least until you need an incomplete factorization for a preconditioner!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix has a relatively dense diagonal band, then you may want to consider one of LAPACK's banded solvers. These store the diagonals along the rows of a matrix with B rows, where B is the bandwidth of the matrix. (EDIT: or (B+1)/2 rows if the matrix is symmetric/Hermitian). But if you truly require a sparse direct solver, then it looks like you will either have to implement your own or convert to a more common storage format. (If you decide to implement your own, perhaps you will quickly learn why SDS is not suitable :P)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The abstract for the Yalamanchili-Anand paper suggests that they are focused on vector-matrix multiplication; i.e., iterative methods.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-03-21T13:44:57.147" LastActivityDate="2013-03-21T13:44:57.147" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5619" PostTypeId="2" ParentId="5612" CreationDate="2013-03-21T02:44:59.940" Score="5" Body="&lt;p&gt;No.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In trust region methods you're using a local quadratic (or even linear model) of the objective function to find a minimum of an unconstrained problem.   In each iteration of the algorithm you find the minimum of this quadratic model over a ball or ellipsoid that is the current trust region.  You then move to that point and repeat the process.  The size of the trust region is increased when the quadratic approximation is working well and decreased when it performs poorly.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the ellipsoid method, you start with an ellipsoid containing an optimal solution to a convex minimization problem that can also include convex constraints.  In each iteration you find a hyperplane such that the optimal solution is constrained to be within the intersection of the ellipsoid and a half space defined by the hyperplane.  You construct a new smaller ellipsoid containing an optimal solution that surrounds the points that weren't cut off by your hyperplane.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although both algorithms involve ellipsoids, the underlying ideas are very different.  For example, the ellipsoidal trust region in the trust region method typically doesn't contain an optimal solution to the original problem until the very end of the iterations, while in the ellipsoid method, an optimal solution is contained within the ellipsoid at all times.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way for you to understand is to take the time to study both methods in some depth.  There's a good introduction to the ellipsoid method in Chvatal's book on linear programming.  Trust region methods are described in most textbooks on nonlinear programming.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-03-21T02:44:59.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5620" PostTypeId="2" ParentId="5610" CreationDate="2013-03-21T10:56:59.070" Score="2" Body="&lt;p&gt;EDIT: Here is an updated version of my analytical function. Tested it for a few values and it seems to work. Thanks to Anke for your help&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double analytical(double a, double b)&#xA;{&#xA;  const double PI = 2*acos(0);&#xA;&#xA;  if ( b-a &amp;lt;=PI)&#xA;  {&#xA;       return abs(-cos(b) + cos(a));&#xA;  }&#xA;  else if ((b-a) &amp;gt; PI)&#xA;  {&#xA;       float area=0;&#xA;       double aReduced = fmod(a,PI);&#xA;       double bReduced = fmod(b,PI);&#xA;&#xA;       while (b-PI &amp;gt; a)&#xA;       {&#xA;            b = b - PI;&#xA;            area += 2;&#xA;       }&#xA;&#xA;       area += abs(-cos(PI)+cos(a)) + abs(-cos(b)+cos(PI));&#xA;       return area;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3954" LastEditorUserId="3954" LastEditDate="2013-03-21T14:45:43.243" LastActivityDate="2013-03-21T14:45:43.243" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5621" PostTypeId="1" AcceptedAnswerId="5622" CreationDate="2013-03-21T13:23:55.063" Score="8" ViewCount="90" Body="&lt;p&gt;I was trying to solve a linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{A}\mathbf{x} = \mathbf{y}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but the conditioning number was quite bad (around $10^{17}$).&#xA;I thought that the system was singular, but after scaling the &#xA;system using the diagonal matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{D_{ii}} = 1/\sqrt{\mathbf{A_{ii}}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the fact:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{A}^{-1} = \mathbf{D} (\mathbf{D}\mathbf{A}\mathbf{D})^{-1}\mathbf{D}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conditioning number of $\mathbf{D}\mathbf{A}\mathbf{D}$ is around $10^3$!&#xA;Does this imply that the system was just badly scaled and that there is no&#xA;collinearity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="2140" LastActivityDate="2013-03-21T15:32:18.587" Title="Bad scaling versus collinearity" Tags="|linear-algebra|matrix|linear-solver|scaling|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5622" PostTypeId="2" ParentId="5621" CreationDate="2013-03-21T15:32:18.587" Score="7" Body="&lt;p&gt;Yes, that's what it means.  This is usually illustrated with $A$ a diagonal matrix having both large and small entries.  Clearly such a matrix can be accurately inverted, but a simple measure of condition number being ratio of largest eigenvalue to smallest (in this case, largest diagonal entry to smallest) would easily give a high value.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-03-21T15:32:18.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5623" PostTypeId="2" ParentId="5614" CreationDate="2013-03-21T18:04:22.153" Score="2" Body="&lt;p&gt;Found=), but haven't tried. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;strong&gt;Intel MKL&lt;/strong&gt;  &lt;em&gt;Sparse BLAS Level 2 and Level 3 Routines.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;mkl_?diatrsv&lt;/strong&gt;&#xA;&lt;em&gt;Triangular solvers with simplified interface for a sparse matrix in the &lt;strong&gt;diagonal format&lt;/strong&gt; with one-based indexing.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-9FBF0F88-4731-45DA-BDE9-AF4D7A0A18A4.htm#GUID-9FBF0F88-4731-45DA-BDE9-AF4D7A0A18A4&quot; rel=&quot;nofollow&quot;&gt;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-9FBF0F88-4731-45DA-BDE9-AF4D7A0A18A4.htm#GUID-9FBF0F88-4731-45DA-BDE9-AF4D7A0A18A4&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Description of the used diagonal format is in the middle of the page&#xA;&lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-9FCEB1C4-670D-4738-81D2-F378013412B0.htm&quot; rel=&quot;nofollow&quot;&gt;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-9FCEB1C4-670D-4738-81D2-F378013412B0.htm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3956" LastEditorUserId="3956" LastEditDate="2013-03-21T18:09:50.090" LastActivityDate="2013-03-21T18:09:50.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5624" PostTypeId="1" CreationDate="2013-03-21T20:37:43.477" Score="0" ViewCount="108" Body="&lt;p&gt;I have this confusion related to P and NP problems. Why is P a subset of NP? I didn't get it. P problems can be solved in polynomial time. However, NP problems cannot but only verify if a solution is correct of not in polynomial time. Then how come P is subset of NP?&lt;/p&gt;&#xA;" OwnerUserId="3871" LastActivityDate="2013-03-21T21:39:06.147" Title="Confusion related to P and NP problems" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5625" PostTypeId="2" ParentId="5624" CreationDate="2013-03-21T21:02:13.960" Score="9" Body="&lt;p&gt;The flaw in your thinking is this statement: &quot;NP problems cannot [be solved in polynomial time]&quot;. In other words, you seem to be under the impression that NP is an &lt;em&gt;opposite&lt;/em&gt; of P. This is not correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Class NP contains all problems whose solutions can be &lt;em&gt;verified&lt;/em&gt; or &lt;em&gt;checked&lt;/em&gt; in polynomial time---but it places no conditions on how difficult the problem is to &lt;em&gt;solve&lt;/em&gt;. That is, the standards for Class NP are &lt;em&gt;easier&lt;/em&gt; to meet than for Class P.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Honestly, the &lt;a href=&quot;http://en.wikipedia.org/wiki/P_versus_NP_problem&quot;&gt;Wikipedia page&lt;/a&gt; on the topic provides a relatively decent overview of the matter. The example of a problem in NP that they give is the &quot;subset sum problem&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider the subset sum problem, an example of a problem that is easy&#xA;  to verify, but whose answer may be difficult to compute. Given a set&#xA;  of integers, does some nonempty subset of them sum to 0? For instance,&#xA;  does a subset of the set {−2, −3, 15, 14, 7, −10} add up to 0? The&#xA;  answer &quot;yes, because {−2, −3, −10, 15} add up to zero&quot; can be quickly&#xA;  verified with three additions. However, there is no known algorithm to&#xA;  find such a subset in polynomial time (there is one, however, in&#xA;  exponential time, which consists of $2^n-1$ tries), and indeed such an&#xA;  algorithm can only exist if P = NP; hence this problem is in NP&#xA;  (quickly checkable) but not necessarily in P (quickly solvable).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you are given a very large set of integers, finding a non-empty subset that adds up to zero is extremely expensive; but given a single &lt;em&gt;potential&lt;/em&gt; solution, verifying that it is a valid solution is quite trivial.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-03-21T21:39:06.147" LastActivityDate="2013-03-21T21:39:06.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5626" PostTypeId="1" AcceptedAnswerId="8027" CreationDate="2013-03-21T21:07:09.483" Score="9" ViewCount="460" Body="&lt;p&gt;For certain applications, such as steady state heat transfer and flow in porous media, it is possible to simulate a much larger (infinite) domain by imposing periodic boundary conditions on opposite boundary faces and dirichlet bc's on the remaining boundaries.  For a 2D rectangular domain, the periodic condition can be interpreted as if the domain lies on the surface of a cylinder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious if the same can be said for elasticity problems.  I've noticed that standard linear elasticity problems are limited to finite domains and I've never seen an example where a periodic boundary condition is prescribed or implemented.  I suspect there may be issues with the uniqueness of solutions to this problem due to rigid body motion (translation and/or rotation) induced by periodicity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, let's assume the linear isotropic planar elasticity case on a 2D rectangular domain.  Let's say that I want to model a large (periodic) medium by using fixed displacement (dirichlet) conditions on two opposite boundaries and periodic displacement conditions on the remaining boundaries.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this problem well-posed?&#xA;If not, are there strategies (e.g. additional constraints) can I use to make it  well-posed, knowing that my ultimate goal is to simulate a much larger (infinite) medium with repetitive material properties?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-03-21T21:13:21.553" LastActivityDate="2013-07-21T20:41:42.997" Title="Well-posedness of a linear elasticity problem with periodic boundary conditions" Tags="|well-posedness|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5628" PostTypeId="1" CreationDate="2013-03-22T05:15:00.797" Score="1" ViewCount="1784" Body="&lt;p&gt;I am a bit confused why the 0-1 loss function is not convex. What's wrong with it?&lt;/p&gt;&#xA;" OwnerUserId="3871" LastActivityDate="2013-03-22T05:45:03.050" Title="Confusion related to convexity of 0-1 loss function" Tags="|convex-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5629" PostTypeId="2" ParentId="5628" CreationDate="2013-03-22T05:45:03.050" Score="3" Body="&lt;p&gt;I'm not sure this is what you're looking for, but here goes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A zero-one loss function $L: \mathbb{R} \rightarrow \{0,1\}$ is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;L(x) = \left\{\begin{array}{ll} 0, &amp;amp; \textrm{if $x \geq 0$}, \\ 1, &amp;amp; \textrm{if $x &amp;lt; 0.$}\end{array}\right.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L$ is convex if, for all $x_1, x_2 \in \mathbb{R}$, and all $\lambda \in [0,1]$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;L(\lambda x_1 + (1 - \lambda)x_2) \leq \lambda L(x_1) + (1 - \lambda) L(x_2).&#xA;\end{align} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A counterexample is: $(x_1, x_2, \lambda) = (-1, 1/2, 1/2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;L(\lambda x_1 + (1 - \lambda)x_2) &amp;amp;= L(-1/4) = 1, \\&#xA;L(x_1) &amp;amp;= 1, \\&#xA;L(x_2) &amp;amp;= 0,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $L(\lambda x_1 + (1 - \lambda)x_2) \leq \lambda L(x_1) + (1 - \lambda) L(x_2)$ does not hold, because $1 \leq 1/2$ is not true, so $L$ is not convex.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-03-22T05:45:03.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5630" PostTypeId="1" AcceptedAnswerId="5648" CreationDate="2013-03-22T10:19:55.470" Score="2" ViewCount="279" Body="&lt;p&gt;I have three questions regarding WENO schemes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) How to actually compute the smoothness indicators $\beta_j$ for required order of polynomial? Any reference which explains the algorithm will be helpful.&#xA;The formula for $\beta_j$ is given by equation (2.61) of &lt;a href=&quot;http://ccam.xmu.edu.cn/teacher/jxqiu/undergraduate/icase-1997-65.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) How to obtain modified wave-number plots (as given in &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999183711423#&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; paper) for a WENO scheme? (For finite difference schemes with fixed stencil, the same can be obtained by taking Fourier transform of numerical approximation and equating that with the Fourier transform of exact terms). How to carry out the same when the the contribution of candidate stencils is not constant but is a function of their smoothness? (which is the case for WENO). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) How to find dispersion errors for WENO scheme? How to carry out the analysis? (this is closely related to question # 2) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="1567" LastEditDate="2013-03-22T14:28:04.227" LastActivityDate="2013-03-23T13:19:57.780" Title="Error analysis of WENO scheme" Tags="|finite-difference|fourier-analysis|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5631" PostTypeId="1" CreationDate="2013-03-22T10:54:57.303" Score="1" ViewCount="1095" Body="&lt;p&gt;I'm trying to localize points I see in a camera image in terms of azimuth and elevation and match points between shots.&#xA;Individual shots should differ only in rotation around the camera's center (roughly).&#xA;It's not working (as I'll explain below), and I'm looking for things I might be doing wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried different things, but this is what I'm doing right now:&#xA;Let's say I have tilted my camera (around the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-axis) by some angle &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and rotated it (around the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-axis) by &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;.&#xA;Assuming my lens gives me a perspective projection with a focal length of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;, I can represent pixel &lt;span class=&quot;math-container&quot;&gt;$(i,j)$&lt;/span&gt; as a vector&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \vec{v}=d\cdot\vec{z}' + i\cdot\vec{x}' + j\cdot\vec{y}', $$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\vec{x}', \vec{y}', \vec{z}'$&lt;/span&gt; are the unit vectors of the standard basis rotated by &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; around x-axis and y-axis, resp.&#xA;&lt;span class=&quot;math-container&quot;&gt;$\vec{v}$&lt;/span&gt; then represents the point in 3D space at which the photoreceptor cell in the CMOS was when it perceived the point in real space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say &lt;span class=&quot;math-container&quot;&gt;$\vec{v}=(v_1,v_2,v_3)$&lt;/span&gt;.&#xA;Then I should be able to calculate azimuth &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; and elevation &lt;span class=&quot;math-container&quot;&gt;$\psi$&lt;/span&gt; wrt. an unrotated take as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\phi = \mathrm{arctan}(\frac{v_1}{v_3})\quad\text{and}\quad\psi = \mathrm{arcsin}(\frac{v_2}{|\vec{v}|}).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't work, though.&#xA;That is, without tilting the camera it more or less does.&#xA;But say I have a shot where both &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; are not zero.&#xA;Then, the points which appear in that shot and in others for which &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; are zero don't come out with the same azimuth and elevation.&#xA;It seems that they generally cluster too much around the center of that shot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the problems is that there are a lot of things that could get in my way here, like any combination of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a distorted camera image&lt;/li&gt;&#xA;&lt;li&gt;wrong focal length&lt;/li&gt;&#xA;&lt;li&gt;wrong rotation angles&lt;/li&gt;&#xA;&lt;li&gt;rotation around a point too far from the focal point.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm looking for more potential sources of error and (in)validation for the approach outlined above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BfaFC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BfaFC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Above is a visualization of the outcome of the procedure so far: the white horizontal and vertical axes are azimuth and elevation, resp.&#xA;The distorted rectangles are boundaries of individual shots.&#xA;It's hard to explain what exactly the points are, but in essence points with the same color are identical in the real world and should, therefore, appear at the same absolute azimuth and elevation.&#xA;These points form a distorted shape, not a rectangle.&#xA;You can see that at each level of elevation, the points from each camera image appear more and more contracted around the camera image's center.&lt;/p&gt;&#xA;" OwnerUserId="3966" LastEditorUserId="20688" LastEditDate="2019-08-25T14:36:52.227" LastActivityDate="2019-08-25T14:36:52.227" Title="Pixel-To-Angle Transformation in Camera Image" Tags="|linear-algebra|computer-vision|geometry|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5632" PostTypeId="2" ParentId="2333" CreationDate="2013-03-22T14:04:48.943" Score="2" Body="&lt;p&gt;Another suggestion via a &lt;a href=&quot;http://xcorr.net/2013/03/21/the-secret-ingredient-in-stochastic-gradient-descent/&quot; rel=&quot;nofollow&quot;&gt;blog post I just read&lt;/a&gt; that refers to a talk at &lt;a href=&quot;http://leon.bottou.org/talks/largescale&quot; rel=&quot;nofollow&quot;&gt;NIPS 2007 tutorial&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do a line search once for a subset of samples, and then fix the learning rate to be the effective step size within the line search.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2013-03-22T14:04:48.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5633" PostTypeId="2" ParentId="5617" CreationDate="2013-03-22T14:38:37.833" Score="0" Body="&lt;p&gt;I guess so. At least i get an output. But somehow my mass is not always conserved (depends on proper initial values, which is bad for data fitting)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried today following: I replaced the flux function $F(u) = u*v*(1-u)^C $with&#xA;$ F(u) = u*v*sign(1-u)*|1-u|^C$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so i can set a boundary like &#xA;$v*sign(1-u)*|1-u(end)|^C = -v*sign(1-u)*|1-u(end-1)|^C $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the solver sets the concentration in the last cell &gt; 1 so that everything flowing out is flowing back. That conserves mass but takes longer and sometimes does not converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my boundaries right now are &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if hyperbolic&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;$0=v*(1-u)^C|_{end}$&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;else&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;$0=v*(1-u)^C+D(u)*\frac{\partial(u)}{\partial(x)}|_{end}$&lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2013-03-22T14:38:37.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5635" PostTypeId="1" CreationDate="2013-03-22T16:23:00.090" Score="11" ViewCount="3181" Body="&lt;p&gt;I need to find &lt;strong&gt;all&lt;/strong&gt; the roots of a scalar function in a given interval. The function may have discontinuities. The algorithm can have a precision of ε (e.g. it is ok if the algorithm doesn't find two distinct roots that are closer than ε).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does such algorithm exists? Could you point me papers about that?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Actually, I have a function to find a zero in a given interval using Brent's algorithm, and a function to find a minimum in a given interval. Using those two functions, I built my own algorithm, but I was wondering if a better algorithm exists. My algorithm is like that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start with an interval &lt;code&gt;[a,b]&lt;/code&gt; and a function &lt;code&gt;f&lt;/code&gt;. If &lt;code&gt;sign(f(a+ε)) ≠ sign(f(b-ε))&lt;/code&gt;, I know there is at least one zero between &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;, and I find &lt;code&gt;z = zero(]a,b[)&lt;/code&gt;. I test if &lt;code&gt;z&lt;/code&gt; really is a zero (it could be a discontinuity), by looking a the value of &lt;code&gt;z-ε&lt;/code&gt;and &lt;code&gt;z+ε&lt;/code&gt;. If it is, I add it to the list of found zeros. If &lt;code&gt;f(a+ε)&lt;/code&gt; and &lt;code&gt;f(b-ε)&lt;/code&gt; both are positive, I search &lt;code&gt;m = min(]a, b[)&lt;/code&gt;. If &lt;code&gt;f(m)&lt;/code&gt; still is positive, I search &lt;code&gt;m = max(]a,b[)&lt;/code&gt; because there could be a discontinuity between &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;. I do the opposite if &lt;code&gt;f(a+ε)&lt;/code&gt; and &lt;code&gt;f(b-ε)&lt;/code&gt; were negatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, from the point I found (&lt;code&gt;z&lt;/code&gt; or &lt;code&gt;m&lt;/code&gt;) I build a stack containing the zeros, discontinuities, and inflection points of my function. After the first iteration, the stack now looks like &lt;code&gt;[a, z, b]&lt;/code&gt;. I start again the algorithm from intervals &lt;code&gt;]a,z[&lt;/code&gt; and &lt;code&gt;]z,b[&lt;/code&gt;. When, between two points &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;, the extrema have the same sign than both interval ends, and there is no discontinuities at both extrema, I remove the interval from the stack. The algorithm ends when there is no more intervals.&lt;/p&gt;&#xA;" OwnerUserId="3967" LastActivityDate="2013-03-23T16:31:45.517" Title="Find all the roots of a function in a given interval" Tags="|algorithms|roots|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5636" PostTypeId="2" ParentId="5635" CreationDate="2013-03-22T17:10:35.187" Score="7" Body="&lt;p&gt;If you're using Matlab, you may want to try the &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot;&gt;Chebfun&lt;/a&gt; system (disclaimer: I used to be an active developer of this project). It can find all the roots of a one-dimensional function in a closed or open interval to machine precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main idea behind the Chebfun root-finder is to use a combination of recursive bisection and the Colleague Matrix, an analogue of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Companion_matrix&quot;&gt;Companion Matrix&lt;/a&gt;, on the coefficients of an interpolant of the target function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a simplified version of the code &lt;a href=&quot;http://www.dur.ac.uk/pedro.gonnet/chebroots.m&quot;&gt;here&lt;/a&gt;. The function &lt;code&gt;chebroots&lt;/code&gt; takes an anonymous function as its first input, the finite interval as a second and third argument, and a degree &lt;code&gt;N&lt;/code&gt; as it's fourth and final argument. For reasonable results, you can set &lt;code&gt;N&lt;/code&gt; to &lt;code&gt;100&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-03-22T17:10:35.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5637" PostTypeId="1" AcceptedAnswerId="5659" CreationDate="2013-03-22T18:25:35.073" Score="3" ViewCount="2088" Body="&lt;p&gt;Precisely the title of the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have encountered these terms in two areas: conjugate gradient method, and adaptive finite elements.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2013-03-24T20:33:05.403" Title="What is the meaning of &quot;preasymptotic&quot; and &quot;superconvergent&quot;?" Tags="|finite-element|convergence|terminology|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5638" PostTypeId="2" ParentId="5637" CreationDate="2013-03-22T20:11:42.727" Score="4" Body="&lt;p&gt;&lt;strong&gt;Pre-asymptoticity&lt;/strong&gt; in conjugate gradient method: Asymptotic behavior sometimes assumes certain conditions. For example, given $h$ the meshsize or similar, and $h\to 0$:&#xA;$$&#xA;\|f_h - g_h \| \leq c h.&#xA;$$&#xA;but what if &#xA;$$&#xA;\|f_h - g_h \| \leq c k h^2&#xA;$$&#xA;if we assume $kh = O(1)$ then $f_h$ is asymptotic to $g_h$, this is asympototic analysis. If we do not assume that condition, for example $k$ may be increasing wave number, then this is &lt;em&gt;pre-asymptotic&lt;/em&gt; analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In preconditioned conjugate gradient(PCG) method, the asymptotic convergence rate for large sparse SPD matrices depends on the spectral condition number of the PCG iteration matrix. Normally we just suppose this condition number is fixed, the &lt;em&gt;pre-asymptotic&lt;/em&gt; analysis for conjugate gradient method also takes into account the behavior of this condition number in the analysis of the convergence rate.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Superconvergence&lt;/strong&gt; in adaptive finite elements: suppose $u$ be the true solution to the usual $H^1_0(\Omega)$ variational problem&#xA;$$&#xA;\int_{\Omega} \nabla u \cdot \nabla v = \int_{\Omega} f\,v&#xA;$$&#xA;and $u_h$ be the finite elements solution. &#xA;Adaptive finite elements method(AFEM) essentially want to use an estimator $\eta$ to approximate the error $\|\nabla(u-u_h)\|$ locally and globally, such that we could perform the adaptive mesh refinement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The superconvergence says something like this, for a &lt;em&gt;postprocessed computable&lt;/em&gt; quantity $\sigma$:&#xA;$$&#xA;\|\nabla u - \sigma\| \leq c h^{\alpha} \|\nabla u - \nabla u_h\|.&#xA;$$&#xA;As you can see, when the meshsize is small, $\|\nabla u - \sigma\|$ is much smaller than the true error, convergent at a much faster rate than the approximation error $\|\nabla u - \nabla u_h\|$, and this is what &lt;em&gt;superconvergence&lt;/em&gt; means in AFEM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now simply by triangle inequality:&#xA;$$&#xA;\Big| \|\nabla u - \sigma\| - \|\sigma- \nabla u_h\| \Big|\leq \|\nabla u - \nabla u_h\| \leq \|\nabla u - \sigma\| +  \|\sigma- \nabla u_h\|&#xA;$$&#xA;Since $\|\nabla u - \sigma\|$ is negligible when the meshsize is small, whenever superconvergence is true, we could use the quantity $\|\sigma- \nabla u_h\|$ to be a computable error estimator to estimate the non-computable approximation error $\|\nabla u - \nabla u_h\|$.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2013-03-24T20:33:05.403" LastActivityDate="2013-03-24T20:33:05.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5639" PostTypeId="2" ParentId="5587" CreationDate="2013-03-22T20:57:58.120" Score="0" Body="&lt;p&gt;Depending on how high order you go you run into a lot of frustrating difficulties. The derivatives of order $N$ polynomials grow as $N^2$ (see e.g. Markov's inequality to make this more precise), and these large gradients (mostly near boundaries of your elements) often lead to spuriously large eigenvalues in discretizations, making you have to resolve very fine scale oscillations that probably aren't physically meaningful to begin with. So efficient implementation in the high order regime isn't as straightforward, despite the higher formal order of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, simply calculating the discrete derivative operator is difficult. A lot of literature from the 80s and early 90s put considerable effort into stable calculations of high order differencing matrices. The operator can be highly non-normal, leading to a lot of numerical garbage that is hard to predict.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think a lot of these issues have been well addressed by now, but that just means doing a bit more work to make high order worthwhile.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-03-22T20:57:58.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5641" PostTypeId="1" CreationDate="2013-03-23T00:06:06.833" Score="10" ViewCount="446" Body="&lt;p&gt;I'm interested in suggestions for book references on the subject of numerical PDE and ODE, in particular, a rigorous analysis of such methods in a manner written for professional mathematicians. It does not have to be extremely comprehensive in the sense of listing hundreds or thousands of different methods, but I would be interested in something that at least covers most of the key concepts that guide modern techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it would be proper to draw analogies to textbooks on numerical linear algebra, about which I am more familiar. I'm looking for something that is to stability and truncation errors in numerical differential equations as Higham's &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; is to stability and roundoff errors in numerical linear algebra, and something that discusses modern techniques in ODE and PDE the way that Golub and Van Loan's &lt;em&gt;Matrix Computations&lt;/em&gt; discusses most of the main types of techniques for linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I actually know very little about numerical ODE and PDE. I've been reading through some assortment of online notes, and I have the book &lt;em&gt;Finite Difference Methods for Ordinary and Partial Differential equations&lt;/em&gt; by Randall LeVeque, which is a clear book but not in-depth enough for my purposes. As a more concrete example of the level I'm looking for, I would hope that any section on elliptic and parabolic equations assumes the reader has full familiarity with the theory of Sobolev spaces and their embeddings, and weak solutions for PDE, and uses results from that theory rather freely in deriving error estimates for finite elements, etc. &lt;/p&gt;&#xA;" OwnerUserId="3477" LastActivityDate="2013-03-24T10:08:30.493" Title="Reference request: Rigorous analysis of algorithms for PDE and ODE" Tags="|pde|reference-request|ode|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5642" PostTypeId="2" ParentId="5641" CreationDate="2013-03-23T01:55:19.963" Score="11" Body="&lt;p&gt;You won't find one reference systematically covering the analysis of all the important methods for PDE. The field of discretization techniques for PDE is &lt;em&gt;at least&lt;/em&gt; an order of magnitude larger than either topic you mentioned above. For any methods involving implicit solves, studying discretizations without also considering solution methods (e.g., associated multigrid methods) is a tried and true way to paint yourself into the &quot;hopelessly impractical&quot; corner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presumably you are familiar with &lt;a href=&quot;http://books.google.com/books?id=ci4c_R0WKYYC&quot;&gt;Brenner and Scott, &lt;em&gt;The Mathematical Theory of Finite Element Methods&lt;/em&gt;&lt;/a&gt;. It is a graduate level text, and although it has its share of introductory matter, you can quickly get to the important results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;em&gt;a posteriori&lt;/em&gt; error analysis in FEM, a good source is the review paper, &lt;a href=&quot;http://dx.doi.org/10.1016/S0045-7825%2896%2901107-3&quot;&gt;Ainsworth and Oden, &lt;em&gt;A posteriori error estimation in finite element analysis&lt;/em&gt;, 1997&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For finite volume methods, you might like the Acta Numerica paper &lt;a href=&quot;http://dx.doi.org/10.1017/S0962492906300013&quot;&gt;Morton and Sonar, &lt;em&gt;Finite volume methods for hyperbolic conservation laws&lt;/em&gt;, 2007&lt;/a&gt;. As Acta Numerica papers go, this is not very highly cited. I suspect that is partly because LeVeque's book is very good and because most practitioners that have not used his book are familiar with many of the original sources. Although I am not familiar with it, you might also look at &lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;lr=&amp;amp;id=smIW3R_O654C&quot;&gt;Bouchut, &lt;em&gt;Nonlinear Stability of Finite Volume Methods for Hyperbolic Conservation Laws&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-03-23T01:55:19.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5643" PostTypeId="2" ParentId="1380" CreationDate="2013-03-23T02:57:21.597" Score="0" Body="&lt;p&gt;Are there more inequality constraints not being mentioned? As stated, the problem is to minimize a linear function over a cone, so the optimal value is always either $-\infty$ or $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even with the constraint $U \ge 0$, the problem reduces to a discrete decision problem. Think of the linear function $A$ as corresponding to positive/negative weights of the edges of the complete graph on $n$ vertices. If there is a graph of diameter 2 connecting all the vertices with the sum of its weights strictly negative, then the optimal value is $-\infty$, otherwise the optimal value is $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick sketch on how to prove this. First note that if $a \le b \le c$, then $c \le \max(a,b)$ implies $b=c$. So then the inequalities imply that for every triple $i,j,k$, exactly one of the following must be true:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$U_{ij} &amp;lt; U_{jk} = U_{ik}$&lt;/li&gt;&#xA;&lt;li&gt;$U_{ik} &amp;lt; U_{jk} = U_{ij}$&lt;/li&gt;&#xA;&lt;li&gt;$U_{jk} &amp;lt; U_{ik} = U_{ij}$&lt;/li&gt;&#xA;&lt;li&gt;$U_{ij} = U_{jk} = U_{ik}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So if you fix some threshold $t$ and form a graph $G(t)$ with an edge wherever $U_{ij}=t$, then every 3 vertices must have exactly 0,2, or 3 edges connecting them. So if $U_{ij}=U_{j k}=t$, then for every other vertex $\ell$, we must have either $U_{j \ell}=t$ or $U_{i\ell}=U_{k\ell}=t$. So if the graph $G(t)$ has any edges, it must be of diameter 2.&lt;/p&gt;&#xA;" OwnerUserId="767" LastEditorUserId="767" LastEditDate="2013-03-23T03:09:58.647" LastActivityDate="2013-03-23T03:09:58.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5644" PostTypeId="2" ParentId="5641" CreationDate="2013-03-23T06:16:47.240" Score="4" Body="&lt;p&gt;In addition to Jed's great recommendations (I can personally vouch for Brenner+Scott as a great intro finite elements book), an excellent book for the numerical solution of ODEs is Butcher:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/Numerical_Methods_for_Ordinary_Different.html?id=opd2NkBmMxsC&quot; rel=&quot;nofollow&quot;&gt;http://books.google.com/books/about/Numerical_Methods_for_Ordinary_Different.html?id=opd2NkBmMxsC&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That was my bible for a good while, until my university library had it recalled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also you might find Ern+Guermond to be a valuable book, if you already are comfortable with the delicate mathematics&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/Theory_and_Practice_of_Finite_Elements.html?id=CCjm79FbJbcC&quot; rel=&quot;nofollow&quot;&gt;http://books.google.com/books/about/Theory_and_Practice_of_Finite_Elements.html?id=CCjm79FbJbcC&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having read a few papers from Ern+Guermond I can say they definitely lean towards heavy formalism. The chapters are more or less self contained modulo some notation that you might have to flip around to get the definition of.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-03-23T06:16:47.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5645" PostTypeId="2" ParentId="5637" CreationDate="2013-03-23T06:30:34.443" Score="4" Body="&lt;p&gt;The meanings of those terms depend on context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Superconvergence is usually used to mean you are converging faster than the &quot;optimal&quot; rate, and occasionally this sort of weirdly fast convergence can be proven rigorously. One example in DG is that hyperbolic problems generally have an optimal estimate that decays like $h^{n+1/2}$ but some recent superconvergence papers shows that you can actually recover $h^{2n+1}$ in some limited cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Preasymptotic means: some numerical results you have demonstrated don't fit the analysis for a small parameter choice, but past some cut-off point it does. Sometimes I have seen authors use this to explain why their numerical results don't fit their analysis.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-03-23T06:30:34.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5648" PostTypeId="2" ParentId="5630" CreationDate="2013-03-23T13:19:57.780" Score="3" Body="&lt;p&gt;1) If you want code, use Matthew Emmett's &lt;a href=&quot;http://pyweno.readthedocs.org/en/latest/&quot; rel=&quot;nofollow&quot;&gt;PyWENO&lt;/a&gt; package.  Otherwise, the reference you give is the answer.  You just need to do some simple calculus; you won't find that written down in a paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) That kind of analysis does not apply directly to nonlinear schemes like WENO.  If you can come up with a meaningful generalization of it, you should publish your results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) See #2.  But also see &lt;a href=&quot;http://eprints.maths.ox.ac.uk/1011/1/finalOR42.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; for some insight.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-03-23T13:19:57.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5650" PostTypeId="2" ParentId="5635" CreationDate="2013-03-23T16:31:45.517" Score="0" Body="&lt;p&gt;In general, this is a hopeless quest- without some information about the continuity and/or differentiability of the function, anything could happen.  Consider for example the MATLAB function defined on the interval from 0 to 1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;function y=f(x)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;y=1.0;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if (x==0.01)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;y=0.0;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;end&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if (x==0.013)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;y=0.0;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;end&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if (x==0.753124)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;y=0.0;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;end&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Treating this function as a block box, there's no way to see that it has zeroes at these three points and no other points in the interval from 0 to 1 without checking every floating point number between 0 and 1.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-03-23T16:31:45.517" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5651" PostTypeId="1" AcceptedAnswerId="5653" CreationDate="2013-03-23T16:33:02.480" Score="5" ViewCount="476" Body="&lt;p&gt;What is the current state of research on the existence of incomplete cholesky factorizations (in the context of preconditioning) for symmetric positive definite matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder in particular whether the question of existence is settled for finite element mass matrices.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2013-03-25T08:50:33.850" Title="Existence of incomplete cholesky factorization" Tags="|finite-element|preconditioning|matrix-factorization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5653" PostTypeId="2" ParentId="5651" CreationDate="2013-03-23T19:48:35.673" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://www.ams.org/journals/mcom/1977-31-137/S0025-5718-1977-0438681-4/&quot; rel=&quot;noreferrer&quot;&gt;Meijerink and van der Vorst&lt;/a&gt; showed that incomplete Cholesky dose not break down for $M$-matrices. As for finite element mass matrices, you will have to specify a basis to have any hope of making that claim.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given any basis $\Phi = [\phi_0 | \phi_1 | \phi_2 | \phi_3]$ such that the mass matrix $A = \Phi^T \Phi$ is SPD, we can construct a new basis $\hat \Phi = \Phi B$ such that the new mass matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat\Phi^T \hat \Phi = \begin{pmatrix} 3 &amp;amp; -2 &amp;amp; 0 &amp;amp; 2 \\ -2 &amp;amp; 3 &amp;amp; -2 &amp;amp; 0 \\ 0 &amp;amp; -2 &amp;amp; 3 &amp;amp; -2 \\ 2 &amp;amp; 0 &amp;amp; -2 &amp;amp; 3 \end{pmatrix} =: K$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is the &lt;a href=&quot;http://dx.doi.org/10.1016/0021-9991%2878%2990098-0&quot; rel=&quot;noreferrer&quot;&gt;Kershaw matrix&lt;/a&gt;, an SPD matrix for which incomplete Cholesky produces a negative pivot. The transformation $B$ satisfies $B^T A B = K$, and given the Cholesky factorizations $A = L_AL_A^T$ and $K = L_K L_K^T$, is readily computable as $B = L_A^{-T} L_K^T$.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-03-23T23:01:53.750" LastActivityDate="2013-03-23T23:01:53.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5655" PostTypeId="1" CreationDate="2013-03-23T22:39:12.660" Score="4" ViewCount="1249" Body="&lt;p&gt;I'm learning about numerical methods to obtain the eigenvalues of a system. I have to find the eigenvalues for the time-independent Schrödinger equation but I'm having some difficulties understanding the real problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was told to consider a 2D potential box (what's this? never heard about this before). The domain is $\Omega = (0,0)\times(1,1)$ and inside the domain the potential $V=0$ and outside $V=\infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that in mind the Schrödinger equation looks like $\displaystyle E\Psi = \frac{\hbar}{2m}\Delta\Psi$, is this right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now from my linear algebra class I know that the eigenvalues (I don't know how to correctly express this, so forgive me if I say something incorrectly) are those values such that the following equation holds $Ax = \lambda x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back to the Schrödinger equation, it looks like the previous equation for eigenvalues, where I think $\displaystyle A = \frac{\hbar}{2m}\Delta$ and $\lambda = E$, is this right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if all previous was right, here's what I'm getting confused. If I want to know the function $\Psi$ why do I care to solve for the eigenvalues and find $E$?? I really don't get how the eigenvalues are used to find something else, what's the purpose of this?? I think I'm missing something really important here, if someone can explain me I would be really thankful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, to find the eigenvalues I have to assembly the matrix $A$. To do this I have to replace the derivatives of the laplacian with a finite difference scheme which gives me something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p\Psi_{i-1,j}+p\Psi_{i+1,j}+q\Psi_{i,j-1}+q\Psi_{i,j+1}-4r\Psi_{i,j} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix that represent this will have $N$-rows and $N^2$-columns? Something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A=\pmatrix{&#xA;a_{0,0} &amp;amp; a_{0,1} &amp;amp; a_{0,2} &amp;amp; 0 &amp;amp; 0 &amp;amp; ... &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; a_{1,1} &amp;amp; a_{1,2} &amp;amp; a_{1,3} &amp;amp; 0 &amp;amp; ... &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;...&amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; ... &amp;amp; &amp;amp; ...\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; a_{N-2,N-2} &amp;amp; a_{N-1,N-1} &amp;amp; a_{N,N} \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; a_{N-1,N-1} &amp;amp; a_{N,N} \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; a_{N,N}&#xA;} \cdot &#xA;\pmatrix{&#xA;\Psi_{0,0} \\&#xA;\Psi_{1,0} \\&#xA;\Psi_{2,0} \\&#xA;... \\&#xA;\Psi_{N,0} \\&#xA;\Psi_{0,1} \\&#xA;\Psi_{1,1} \\&#xA;\Psi_{2,1} \\&#xA;... \\&#xA;\Psi_{N,N}&#xA;}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see how to assembly the matrix, what are the values of every entry $a_{i,j}$ how to get them from the finite difference equation? In 1D I can clearly see it but in 2D I don't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a question in SE-Math of the &lt;a href=&quot;https://math.stackexchange.com/questions/23312/what-is-the-importance-of-eigenvalues-eigenvectors&quot;&gt;importance of eigenvalues and eigenvector&lt;/a&gt; to help me understand it.&#xA;The remaining question is how to assembly the matrix from the difference equation.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-08-23T06:15:08.113" Title="2D Schrödinger time-independent finite difference and eigenvalues" Tags="|finite-difference|eigenvalues|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5656" PostTypeId="2" ParentId="5641" CreationDate="2013-03-24T06:26:32.563" Score="7" Body="&lt;p&gt;I second Jed's point about the importance of considering solvers at the same time as discretizations. This is something &quot;purer&quot; mathematicians sometimes fail to do, much to their detriment, as they are &lt;a href=&quot;http://www.cs.utexas.edu/~dahlin/bookshelf/hamming.html&quot;&gt;solving the wrong problem&lt;/a&gt;. Things like the block structure, sparsity pattern, and ability to build preconditioners tends to be much more important than simple things like number of degrees of freedom/mesh size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books/about/Mixed_and_hybrid_finite_elements_methods.html?id=yYwZAQAAIAAJ&quot;&gt;Brezzi &amp;amp; Fortin - &quot;Mixed and Hybrid Finite Element Methods&quot;&lt;/a&gt; covers material complementary to Brenner and Scott. It's out of print though and people really hang onto their copies, so if you don't want to pay several hundred dollars you'd probably have to borrow it from your library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The series of papers by Rannacher et al in the early 2000's such as &lt;a href=&quot;http://numerik.iwr.uni-heidelberg.de/Paper/Preprint2001-03.pdf&quot;&gt;&quot;An Optimal Control Approach to A Posteriori Error Estimation in Finite Element Methods&quot;&lt;/a&gt; provides a deeper and more widely applicable understanding of a posteriori error estimation than what is explained in Ainsworth and Oden's book (in my opinion).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sobolev spaces are not the be-all-end-all function spaces for PDE's, though you might get that impression reading introductory graduate books such as Evans. Besov spaces are more general and quite nice, and force you to think about how and why certain function spaces are constructed by controlling basic building blocks to provide constraints on oscillation, integrability, and multiscale structure. A nice &quot;philosophical&quot; article on the subject of function spaces broadly is &lt;a href=&quot;http://terrytao.wordpress.com/2010/03/11/a-type-diagram-for-function-spaces/&quot;&gt;Terry Tao's post here&lt;/a&gt;. Triebel's book (mainly about Besov spaces), &lt;a href=&quot;http://books.google.com/books/about/Theory_of_Function_Spaces_II.html?id=tLyO_7BdoyYC&quot;&gt;&quot;Theory of Function Spaces II&quot;&lt;/a&gt;, is great! There is a deep connection between Besov spaces and wavelets, so DeVore's &lt;a href=&quot;http://www.math.purdue.edu/~lucier/692/wavelet.pdf&quot;&gt; very readable article&lt;/a&gt; on wavelets is useful.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2013-03-24T06:26:32.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5657" PostTypeId="2" ParentId="5641" CreationDate="2013-03-24T10:08:30.493" Score="3" Body="&lt;p&gt;For PDEs, a book with a similar functional-analytic flavor as Ern and Guermond is &lt;a href=&quot;http://www.cambridge.org/gb/knowledge/isbn/item1164476/?site_locale=en_GB&quot; rel=&quot;nofollow&quot;&gt;D. Braess, &lt;strong&gt;Finite Elements&lt;/strong&gt;, Cambridge University Press, 2007&lt;/a&gt;. Being a textbook rather than a research monograph, it is more accessible, although less comprehensive. On the other hand, it also discusses applications (mainly in elasticity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding ODEs, I believe the bible is still the three-volume work by Hairer and Wanner (&lt;a href=&quot;http://www.springer.com/mathematics/analysis/book/978-3-540-56670-0&quot; rel=&quot;nofollow&quot;&gt;Solving ODEs I&lt;/a&gt;, &lt;a href=&quot;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-3-540-60452-5&quot; rel=&quot;nofollow&quot;&gt;Solving ODEs II&lt;/a&gt; and &lt;a href=&quot;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-3-540-30663-4&quot; rel=&quot;nofollow&quot;&gt;Geometric Numerical Integration&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, don't overlook the many excellent lecture notes available on the internet.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-03-24T10:08:30.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5658" PostTypeId="1" CreationDate="2013-03-24T14:39:03.510" Score="2" ViewCount="157" Body="&lt;p&gt;I have a situation where I am trying to maximize the distance between some point particles.  For example, I have a periodic simulation box that is 100 Å$^3$, and I am putting in 361 particles.  Ideally, the radial distribution function will be zero up until a particular r, at which point it should jump to some particular value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having trouble actually implementing this.  My first step (I'm doing this in LAMMPS) was to randomly place particles with +1 charge in a box and then apply energy minimization.  Unfortunately, this energy minimization only achieves a local minima and the result is horrible.  My next attempt was to use an NVT ensemble to slowly drain energy from the particles as they move, and what I've found is that the slower I drain energy, the better the result is.  The problem is that this takes &lt;em&gt;forever&lt;/em&gt; to get a good result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's got to be a better way to go about this.  Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="1997" LastEditorUserId="1997" LastEditDate="2013-03-24T18:12:57.110" LastActivityDate="2013-03-26T03:11:23.513" Title="Maximizing distance between point particles" Tags="|optimization|molecular-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5659" PostTypeId="2" ParentId="5637" CreationDate="2013-03-24T15:38:08.010" Score="5" Body="&lt;p&gt;&lt;em&gt;Preasymptotic&lt;/em&gt; refers to the following concept: A priori error estimates only say that as the mesh size $h\rightarrow 0$, we have that (for example) $\|e\|\le Ch^2$. But this is an &lt;em&gt;asymptotic&lt;/em&gt; statement: it is not an equality that holds for all $h$ but one typically only sees the quadratic decay &lt;em&gt;whenever $h$ is small enough&lt;/em&gt;. In other words, in numerical experiments, if you plot the error as a function of the mesh size in a log-log plot, you get a nice straight line with slope 2 for small enough mesh sizes. But, for large mesh sizes, the data points are more scattered and do not follow the exact relationship. This, the range of large $h$ where the decay has not yet set in, is called the &lt;em&gt;preasymptotic range&lt;/em&gt; (range as in &quot;a part of the possible range of mesh sizes $h$&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason this preasymptotic range exists is that the exact error relationship is a nonlinear function of the mesh size. For example, if the exact error were of the form $\|e\|=c_1h^2 + c_2h^3 + c_3h^4$, then for small enough $h$, all but the first term are essentially irrelevant. However, for large $h$, the other terms may contribute.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-24T15:38:08.010" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5660" PostTypeId="2" ParentId="5637" CreationDate="2013-03-24T15:48:27.917" Score="2" Body="&lt;p&gt;&lt;em&gt;Superconvergent&lt;/em&gt; refers to the concept that sometimes convergence is faster than one would usually expect. As an example, we know that for the Laplace equation, pointwise convergence happens as $\|u-u_h\|_{L_\infty} \le Ch^2 |\log h|$ when using linear elements. But, on sufficiently regular meshes, it can be shown that at certain points (on quadrilateral cells the $2\times 2$ Gauss integration points) the error is actually of order $h^3$. In other words, at some points of the mesh, the error is significantly smaller than at all the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As hinted at in one of the other answers to this question, this can be exploited. If we know, for example, that the error is smaller at certain points than others, then we should interpolate a higher order polynomial through these points and we can hope to get a more accurate solution than what the finite element method provided us initially. This process is called a &quot;recovery procedure&quot;. It can be used to get a more accurate solution, or a better approximation of the gradient. In either case, the difference between the original finite element solution and the &quot;recovered&quot; one (or their respective gradients) can be used to approximate the error between the original finite element solution and the exact solution.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-24T15:48:27.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5661" PostTypeId="2" ParentId="5565" CreationDate="2013-03-24T18:13:04.907" Score="8" Body="&lt;p&gt;OK, now we're getting somewhere. From &lt;a href=&quot;http://cs.carleton.edu/faculty/dlibenno/old/cs352-spring11/L16-better-max-cut-SDPs.pdf&quot;&gt;these notes&lt;/a&gt; we have:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A vector program consists of: &lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;a collection of vector-valued variables $y_1, ... , y_n\in\mathbb{R}^k$&lt;/li&gt;&#xA;  &lt;li&gt;linear constraints on the dot products of variables&lt;/li&gt;&#xA;  &lt;li&gt;a linear objective function on the dot products&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words, the variables $y_i$ never appear by themselves in the model, they appear &lt;em&gt;only&lt;/em&gt; in dot products with themselves or with other variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here would be a standard form for a vector program:&#xA;$$\begin{array}{ll}&#xA;\text{minimize}   &amp;amp; \sum_{i,j} C_{ij} \langle y_i, y_j \rangle + d \\&#xA;\text{subject to} &amp;amp; \sum_{i,j} A_{ijk} \langle y_i, y_j \rangle = b_k , ~ k=1,2,\dots,K\\&#xA;\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's examine the relationship between a VP and an SDP when $k\geq n$. Note that we can assume that $C_{ij}=C_{ji}$ and $A_{ijk}=A_{jik}$ since $\langle y_i,y_j\rangle = \langle y_j,y_i\rangle$. Collect these constants $C_{ij}$ and $A_{ijk}$ into matrices $C,A_1,A_2,\dots,A_k\in\mathbb{R}^{n\times n}$; these matrices will be symmetric. &#xA;Now consider the following semidefinite program:&#xA;$$\begin{array}{ll}&#xA;\text{minimize}   &amp;amp; \langle C, X \rangle + d \\&#xA;\text{subject to} &amp;amp; \langle A_k, X \rangle = b_k , ~ k=1,2,\dots,K\\&#xA;                  &amp;amp; X \succeq 0&#xA;\end{array}$$&#xA;If $X$ is a feasible point for the SDP, then let $Y\in\mathbb{R}^{k\times n}$ be a square root of $X$; that is, let it satisfy $X=Y^TY$. A square root must exist since $X$ is positive semidefinite. If $k=n$ you can use a Cholesky factor or a symmetric square root; if $k&amp;gt;n$, compute one of those square roots and pad it with $k-n$ zero rows. Then it is not difficult to verify that the columns of $Y$&#xA;$$Y \triangleq \begin{bmatrix} y_1 &amp;amp; y_2 &amp;amp; \dots &amp;amp; y_n \end{bmatrix}$$&#xA;are a feasible point for the VP, and obtain the same objective value. For instance,&#xA;$$\langle C,X \rangle = \langle C,Y^TY \rangle = \sum_{ij} C_{ij} y_i^Ty_j  = \sum_{ij} C_{ij} \langle y_i, y_j \rangle.$$&#xA;Conversely, if $y_1,y_2,\dots,y_k$ are an optimal solution to the VP, then $X=Y^TY$ is feasible for the SDP, and obtains the same objective value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the objective values coincide in both directions, the optimal points are equivalent as well. But note that the mapping from $X$ to $Y$ is not one-to-one: there are an infinite number of possible $Y$ values for each $X$. And what happens when $k&amp;lt;n$? Well, you would have to add a constraint $\mathop{\textrm{rank}}(X) \leq k$ to the SDP to preserve equivalence, and that's a non-convex construct. In fact, from the PDF I quoted above:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Vector programs are p-time solvable, but the solver picks k.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This now makes sense if SDPs are used to solve VPs. If you solve the SDP and the optimal value of $X$ satisfies $\mathop{\textrm{rank}}(X) = k&amp;lt;n$, then you can indeed recover a VP solution for vector size $k$ or larger. The Max Cut problem discussed in those notes needs $k=1$, so the VP and/or SDP models are &lt;em&gt;relaxations&lt;/em&gt; of the true Max Cut problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(It's possible to allow inequality constraints in a VP as well, by the way. This answer is already long enough, but basically you'd add slack variables to convert them to equations, then create one new vector variable for each slack.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now it's clear that the vector program he is referring to in these notes is indeed different than the multiobjective optimization problems discussed in Boyd &amp;amp; Vandenberghe. &lt;em&gt;UPDATE&lt;/em&gt;: A wealth of resources on vector programs and their semidefinite equivalents can be found by searching for &quot;vector program&quot; &quot;semidefinite&quot; in Google. (Quotes included.)&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-03-24T19:50:59.583" LastActivityDate="2013-03-24T19:50:59.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5662" PostTypeId="2" ParentId="5658" CreationDate="2013-03-24T20:29:46.310" Score="2" Body="&lt;p&gt;I'm assuming you want to have the points be &quot;randomly&quot; distributed, such that you don't want much long range order (otherwise put them on some kind of grid). You want what is called a &quot;blue noise point set&quot;, although it's called a number of different things. The electrostatic approach works reasonably well if you know when to stop it before the point pattern gets too regular, but it is generally horribly slow, and if run to convergence, produces something which has large regular patches. I recently needed this in 2D, and the graphics folks have largely solved this problem. I am using &lt;a href=&quot;http://www.geometry.caltech.edu/BlueNoise/&quot; rel=&quot;nofollow&quot;&gt;this code&lt;/a&gt;, and it generalizes quite straightforwardly to 3D (CGAL already has a 3D periodic Delaunay triangulation). That is not to say that it would be easy to rewrite this whole thing in 3D, since the algorithmic details are quite involved. I suggest you check out the references to see what older, perhaps less sophisticated, methods are available.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-03-24T20:29:46.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5663" PostTypeId="2" ParentId="5554" CreationDate="2013-03-25T00:59:26.203" Score="1" Body="&lt;p&gt;What about just taking frequency ranges given on the Wikipedia article on color and pairing them with the hex codes for the background colors they subimpose? This would be a stepped function, but you could always make it piecewise linear, or approximate it with a bezier or some sort of basis function expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, this would certainly have trouble being represented evenly across displays. Perhaps someone who understands color gamuts better could explain exactly what we &lt;em&gt;mean&lt;/em&gt; by &lt;code&gt;FF0000&lt;/code&gt;? The peaks in the plot of responses of different cone cells given by Nick are at 420–440 nm for S, 530–540 nmf for M, and 560–580 nm for L (&lt;a href=&quot;http://en.wikipedia.org/wiki/CIE_1931_color_space&quot; rel=&quot;nofollow&quot;&gt;tristimulus values&lt;/a&gt;), which I think sRGB then somehow links to red, green, and blue values through a 3x3 matrix of some magic numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be able to use those peak center locations to bang out a different first approximation than the one from the Wikipedia list of color frequencies. Then do a weighted average of the two methods! This is real science.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Ok, it looks like I ignored the real problem--we need $f:\mathbb{R}^3 \rightarrow\mathbb{R}^1$ to take us from RGB to frequency space! And apparently taking a linear combination of the three primaries won't cut it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem is that natural light isn't just a single frequency--it's a spectrum (which you could liken to a probability distribution $P(\lambda)$). So, I lied with that earlier function description. Natural light isn't really $\mathbb{R}^1$--it's actually more like $\mathbb{R}^{60}$, for a 5 nm discretization of a 300 nm range. This is converted to $X$, $Y$, and $Z$ by convolving this discretized spectrum of natural light with the three CIE color matching spectra $Q_i(\lambda)$(&lt;a href=&quot;http://www.fourmilab.ch/documents/specrend/&quot; rel=&quot;nofollow&quot;&gt;see equation 1&lt;/a&gt;) (that is, taking the inner product).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Then&lt;/em&gt; you can convert the $X$, $Y$, and $Z$ to $x$, $y$, and $z$ by normalizing them by their sum (equation 2), and &lt;em&gt;then then&lt;/em&gt; convert them to RGB using those sRGB magic numbers (equation 3). Further shenanigans with a white point are required to represent (approximate?) out-of-gamut colors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is great for converting a known spectrum to a particular RGB triad, but I'm not sure how to go the other way. I guess you'd want to create a normal distribution $P(\lambda)$ such that the process I just described gets you the desired RGB triad, and you would find this $P(\lambda)$ through some sort of optimization problem? Then the &quot;single frequency&quot; would be the mean of this distribution? This is quickly getting to be very computationally expensive.&lt;/p&gt;&#xA;" OwnerUserId="3976" LastEditorUserId="3976" LastEditDate="2013-03-25T01:50:05.793" LastActivityDate="2013-03-25T01:50:05.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5664" PostTypeId="2" ParentId="5651" CreationDate="2013-03-25T08:50:33.850" Score="1" Body="&lt;p&gt;I would suggest you the reading of Wang's thesis on the subject.&#xA;&lt;a href=&quot;http://repository.tamu.edu/bitstream/handle/1969.1/ETD-TAMU-3135/WANG-DISSERTATION.pdf?sequence=1&quot; rel=&quot;nofollow&quot;&gt;http://repository.tamu.edu/bitstream/handle/1969.1/ETD-TAMU-3135/WANG-DISSERTATION.pdf?sequence=1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using local element stiffness matrices modification it allows to produce a M-matrix that approximates correctly the original global stiffness matrix (after assembly).&#xA;Hope this helps.&#xA;Tom&lt;/p&gt;&#xA;" OwnerUserId="3977" LastActivityDate="2013-03-25T08:50:33.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5665" PostTypeId="2" ParentId="5617" CreationDate="2013-03-25T14:19:11.373" Score="3" Body="&lt;p&gt;The correct boundary condition that you need to setup is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(u) - D(u) \frac{\partial u}{\partial x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you could have this condition, the mass would be perfectly conserved (see the answer to the question that you refer to for the reasoning). The problem is that what this condition is non-linear, so this is not a Robin condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I would try to overcome this issue: make one part of the condition explicit in time. One option is to use a Neumann condition like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D(u^{(n)}) \frac{\partial u^{(n+1)}}{\partial x} = F(u^{(n)})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u^{(n)}$ is the approximation of $u$ at time $t$. You can also try the following Robin condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D(u^{(n)}) \frac{\partial u^{(n+1)}}{\partial x} = u^{(n+1)} v (1-u^{(n)})^{C}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should give you an approximative conservation of the mass (the conservation should also improve as the time step decreases). If you want to conserve exactly the mass, then you should subiterate at each time step to solve the non-linearity (but then the convergence of the non-linear solver comes into play).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-03-25T14:19:11.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5666" PostTypeId="1" CreationDate="2013-03-25T15:07:02.063" Score="4" ViewCount="2704" Body="&lt;p&gt;On page 3 of &lt;a href=&quot;http://www.mpi-hd.mpg.de/astrophysik/HEA/internal/Numerical_Recipes/f2-7.pdf&quot; rel=&quot;nofollow&quot;&gt;this document&lt;/a&gt;, there are some matrix forms for sparse matrices. I wonder if there are other forms used in computational problems encountered in physics, chemistry, etc., so that reordering the matrix will speedup the computation phase. Preprocessing overhead for matrix reordering should be amortizable via the gain in computation step.&lt;/p&gt;&#xA;" OwnerUserId="3978" LastActivityDate="2013-03-26T03:07:28.127" Title="Reordering sparse matrices in computational science" Tags="|sparse-matrix|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5667" PostTypeId="2" ParentId="5655" CreationDate="2013-03-25T15:22:25.773" Score="1" Body="&lt;p&gt;Finally got it. After establishing the size of the grid (i.e. the internal nodes) in the x and y directions each row of the matrix will correspond to how much each point contributes to the value $\Psi_{i,j}$. Then you will have a matrix of $N^2$ entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I think of a grid with N = 3 (3x3) then matrix A will be something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A=\pmatrix{&#xA;\Psi_{i,j} &amp;amp; \Psi_{i+1,j} &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;\Psi_{i-1,j} &amp;amp; \Psi_{i,j} &amp;amp; \Psi_{i+1,j} &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; \Psi_{i-1,j} &amp;amp; \Psi_{i,j} &amp;amp; 0 &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp;  0 &amp;amp; 0 &amp;amp; 0\\&#xA;\Psi_{i,j+1} &amp;amp; 0 &amp;amp; 0 &amp;amp; \Psi_{i,j} &amp;amp; \Psi_{i+1,j} &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0 &amp;amp; \Psi_{i-1,j} &amp;amp; \Psi_{i,j} &amp;amp; \Psi_{i+1,j} &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0 &amp;amp; \Psi_{i-1,j} &amp;amp; \Psi_{i,j} &amp;amp; 0 &amp;amp; 0 &amp;amp; \Psi_{i,j+1}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0 &amp;amp; 0 &amp;amp; \Psi_{i,j} &amp;amp; \Psi_{i+1,j} &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0 &amp;amp; \Psi_{i-1,j} &amp;amp; \Psi_{i,j} &amp;amp; \Psi_{i+1,j}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \Psi_{i,j+1} &amp;amp; 0 &amp;amp; \Psi_{i-1,j} &amp;amp; \Psi_{i,j}\\&#xA;}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The importance of the eigenvalues and vectors can be seen &lt;a href=&quot;https://math.stackexchange.com/questions/23312/what-is-the-importance-of-eigenvalues-eigenvectors&quot;&gt;in this thread&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-03-25T15:33:24.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5669" PostTypeId="2" ParentId="5666" CreationDate="2013-03-25T15:48:13.967" Score="4" Body="&lt;p&gt;Matrix reordering is not only useful for speedup but often mandatory in order to obtain a code which runs in a reasonable amount of time, especially for sparse direct solvers. These aim to keep the number of extra entries that are filled in during LU-factorization small. For example, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Reverse_Cuthill%E2%80%93McKee_algorithm&quot; rel=&quot;nofollow&quot;&gt;reverse Cuthill-McKee ordering&lt;/a&gt; is a useful heuristic for obtaining a reordering with a much smaller bandwidth, thus giving fewer extra non-zero entries when you factor the matrix. Likewise, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Minimum_degree_algorithm&quot; rel=&quot;nofollow&quot;&gt;approximate minimum degree ordering&lt;/a&gt; aims to reduce the fill-in directly using some ideas from graph theory. Both of these are implemented in Matlab as &lt;code&gt;symrcm&lt;/code&gt; and &lt;code&gt;symamd&lt;/code&gt; respectively if you want to try them yourself. If you want to know more, you should look at Davis's book &lt;a href=&quot;http://books.google.com/books?isbn=0898716136&quot; rel=&quot;nofollow&quot;&gt;Direct Methods for Sparse Linear Systems&lt;/a&gt; in chapter 7.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same reorderings are also helpful if you are using an iterative solver but you're preconditioning with an incomplete LU-factorization. There are other orderings which you might choose instead if you are only using iterative methods. The independent set ordering and multicolor ordering (see &lt;a href=&quot;http://books.google.com/books?isbn=0898715342&quot; rel=&quot;nofollow&quot;&gt;Iterative Methods for Sparse Linear Systems&lt;/a&gt;, chapter 12) are useful for parallelizing Gauss-Seidel iteration and in turn obtaining a parallel implementation of the multigrid algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial cost of reordering your system can be amply paid back in speedup later. For a time-dependent or nonlinear PDE, where you solve not one but several linear systems, this is especially true. Often it is the case for a single linear system too, but you should make that judgment call based on what method you're using, the cost of applying the permutation to the matrix storage format you've chosen, etc. Finally, for discretizing a PDE on an unstructured mesh, you're more likely to wind up with one of the &quot;other&quot; formats, (j) and (k) in the page you linked, than a nice-looking one like banded diagonal.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-03-25T15:48:13.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5670" PostTypeId="2" ParentId="1241" CreationDate="2013-03-25T16:08:06.100" Score="-1" Body="&lt;p&gt;To compute the median one must make a sorted list, count the number of elements in the list, and read either one or two values in the list depending on the number of elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mathworld.wolfram.com/StatisticalMedian.html&quot; rel=&quot;nofollow&quot;&gt;http://mathworld.wolfram.com/StatisticalMedian.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see no reason why this would take substantially longer for any other quantile operation.&lt;/p&gt;&#xA;" OwnerUserId="3979" LastActivityDate="2013-03-25T16:08:06.100" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5671" PostTypeId="1" CreationDate="2013-03-25T20:22:29.297" Score="2" ViewCount="250" Body="&lt;p&gt;I develop a parallel time-domain DAE simulation software using OpenMP and Fortran 2003. The main algorithm is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while t&amp;lt;time_horizon do&#xA;   X_1=X&#xA;   while not converged do&#xA;      !$OMP PARALLEL DO&#xA;         Solve sub-domain DAE using a portion on X and X_1 to get DeltaX&#xA;         Correct X=X+DeltaX&#xA;      !$OMP END PARALLEL DO&#xA;   end&#xA;   t=t+h&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;X is the state variable vector and X_1 is the the vector at the previous time-step used for the discretization algorithm (Trapezoidal method).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sequential implementation profiling shows that the memory copy takes 15% of the computational time, a significant amount...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!$omp parallel workshare&#xA;   X_1=X&#xA;!$omp end parallel worshare&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!$omp parallel&#xA;   i=omp_get_thread_num()&#xA;   X_1(i*length/num_threads+1:(i+1)*length/num_threads)= X(i*length/num_threads+1:(i+1)*length/num_threads)&#xA;!$omp end parallel&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This helped a little bit (dropped to 13%) but not much... Any thoughts? Any other way to keep the history? I keep the X of all subdomains as one vector as it's easier to handle (apply infinite norm, copy, etc.). Should I break it into one for each subdomain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="1602" LastActivityDate="2013-12-21T18:28:57.900" Title="Memory management in time-domain simulation" Tags="|parallel-computing|fortran|memory-management|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5672" PostTypeId="1" CreationDate="2013-03-25T22:02:07.750" Score="0" ViewCount="59" Body="&lt;p&gt;Briggs, &quot;A Multigrid Tutorial&quot; (pg. 35) has the following expressed as 2-D interpolation:&#xA;\begin{align*}&#xA;v^h_{2i,2j} &amp;amp;= v_{i,j}^{2h}\\&#xA;v^h_{2i+1,2j} &amp;amp;= 0.5\cdot(v_{i,j}^{2h} + v_{i+1,j}^{2h})\\&#xA;v^h_{2i,2j+1} &amp;amp;= 0.5\cdot(v_{i,j}^{2h} + v_{i,j+1}^{2h})\\&#xA;v^h_{2i+1,2j+1} &amp;amp;= 0.25\cdot(v_{i,j}^{2h} + v_{i+1,j}^{2h} + v_{i,j+1}^{2h} + v_{i+1,j+1}^{2h}), \quad 0 \leq i,j \leq \frac{n}{2} -1&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe it is number of discretization &lt;em&gt;steps&lt;/em&gt; as otherwise $n/2$ wouldn't be an integer if it were number of points in on axis direction. This means that $i,j$ correspond to coarse gird points. But if that's the case, then the interpolation isn't for the entire grid -- the bound on $i,j$, it only goes up to half the grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain to me why this is or perhaps what I'm misreading? I'd really appreciate a visual display to make sure I know what's going on (if possible).&lt;/p&gt;&#xA;" OwnerUserId="3493" LastEditorUserId="3493" LastEditDate="2013-03-25T22:35:14.673" LastActivityDate="2013-03-25T22:35:14.673" Title="Clarification on interpolation equalities given by Briggs" Tags="|interpolation|multigrid|discretization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5673" PostTypeId="2" ParentId="5672" CreationDate="2013-03-25T22:23:33.387" Score="1" Body="&lt;p&gt;From the formula, $i$ and $j$ are coarse grid indices. The output of interpolation is the fine grid $0 \le i',j' \le n$.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-03-25T22:23:33.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5674" PostTypeId="1" AcceptedAnswerId="5679" CreationDate="2013-03-26T01:26:58.517" Score="5" ViewCount="2074" Body="&lt;p&gt;I'm wondering if anyone can point to a browser-based FEM (or other) 2D PDE solver for simple elliptic problems. It seem like there ought to be a javacript implementation that would allow for the drawing of simple geometries, the creation of simple Dirichlet boundary conditions, and the imposition of simple right-hand-side functions. But, for the life of me, I can't find one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone ever seen such a system?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-03-27T17:19:31.900" Title="Online Poisson Solver" Tags="|finite-element|elliptic-pde|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5676" PostTypeId="2" ParentId="5666" CreationDate="2013-03-26T03:07:28.127" Score="2" Body="&lt;p&gt;In a moment of boredom, I compared a whole lot of reordering strategies with regard to their ability to speed up preconditioners. I put this comparison here (after the heading &lt;em&gt;A comparison of reordering strategies&lt;/em&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/namespaceDoFRenumbering.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/namespaceDoFRenumbering.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-26T03:07:28.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5677" PostTypeId="2" ParentId="5658" CreationDate="2013-03-26T03:11:23.513" Score="1" Body="&lt;p&gt;You are looking for a variant of the &lt;em&gt;sphere packing problem&lt;/em&gt;. In this problem, you ask the question how to pack as many spheres with known radius as possible into a box. In your variant, you ask the question how large the spheres can be so that they still all fit into a box. There is a significant amount of literature on the sphere packing problem as well as, I imagine, software. A literature search for this term may be useful.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-26T03:11:23.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5678" PostTypeId="2" ParentId="5655" CreationDate="2013-03-26T03:14:46.837" Score="1" Body="&lt;p&gt;Many of your questions (including why it's called the &quot;box potential&quot; or &quot;infinite well&quot;) are answered here:&#xA;  &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_36.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_36.html&lt;/a&gt;&#xA;(Some of the answers are given in the &quot;Results&quot; section.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I'm one of the authors of this site and the program described there.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-26T03:14:46.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5679" PostTypeId="2" ParentId="5674" CreationDate="2013-03-26T04:52:51.460" Score="5" Body="&lt;p&gt;See &lt;a href=&quot;http://nclab.com/&quot; rel=&quot;noreferrer&quot;&gt;NCLab&lt;/a&gt;, which has a finite element analysis module using the hp-FEM library &lt;a href=&quot;http://hpfem.org/hermes/&quot; rel=&quot;noreferrer&quot;&gt;Hermes&lt;/a&gt;. For example, see the &lt;a href=&quot;https://nclab.com/elasticity/&quot; rel=&quot;noreferrer&quot;&gt;elasticity module&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-03-26T04:52:51.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5680" PostTypeId="1" AcceptedAnswerId="5687" CreationDate="2013-03-26T05:41:20.613" Score="6" ViewCount="327" Body="&lt;p&gt;I have a 3 dimensional convex polyhedron whose vertex coordinates are rational. For one of these vertices, I would like to find the nearest integer grid point (under the Euclidean metric) that is contained within the polyhedron. The polyhedron can be specified by a collection of inequalities, or as a collection of vertices with appropriate combinatorial information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume I'm looking for some sort of integer program, but I can't quite match up what I want with what I'm reading. How do you solve this problem, and is there efficient code (preferably C/C++) available to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3985" LastEditorUserId="3985" LastEditDate="2013-03-26T07:56:19.753" LastActivityDate="2013-03-29T00:54:49.830" Title="Closest interior point on integer grid to a vertex of a convex polyhedron" Tags="|optimization|computational-geometry|mixed-integer-programming|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5681" PostTypeId="2" ParentId="5680" CreationDate="2013-03-26T07:24:09.790" Score="5" Body="&lt;p&gt;If you simply want a formulation (not necessarily a good way to solve the problem), you can state it as minimizing the distance $||x-z||$ where $z$ is an integer vector, satisfying $Az\leq b$ (inside polyhedron) and $x$ is a binary combination of the vertices $x = \sum_{i=1}^N \delta_i v_i$ where $\sum_{i=1}^N \delta_i = 1$, $\delta$ binary, and $v_i$ are the vertices of the polytope (if you only have the vertices of the polytope, you can replace the constraint $Az\leq b$ by $z = \sum_{i=1}^N \lambda_i v_i$ where $\sum_{i=1}^N \lambda_i = 1$, $\lambda \geq 0$.) Depending on the distance measure you use, both models generate mixed-integer linear or quadratic program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is an implementation in the MATLAB Toolbox YALMIP (developed by me). It assumes you have a reasonably good MIQP solver installed (although the problem seems to be trivially simple for the solvers I tried)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Data&#xA;V = randn(2,50)*2;&#xA;k = convhull(V');&#xA;V = V(:,k);&#xA;clf&#xA;plot(V(1,:),V(2,:),'*');&#xA;N = size(V,2);&#xA;&#xA;delta  = binvar(N,1);&#xA;lambda = sdpvar(N,1);&#xA;z = intvar(2,1);&#xA;x = sdpvar(2,1);&#xA;Inside = [z == V*lambda, lambda &amp;gt;=0, sum(lambda)==1];&#xA;Corner = [x == V*delta, sum(delta)==1];&#xA;solvesdp([Inside,Corner], (x-z)'*(x-z))&#xA;hold on&#xA;grid&#xA;plot(double(x(1)),double(x(2)),'ob')&#xA;plot(double(z(1)),double(z(2)),'pk')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But as I said, I am pretty sure this can be solved more efficiently directly somehow&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2013-03-26T07:37:12.377" LastActivityDate="2013-03-26T07:37:12.377" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5682" PostTypeId="1" AcceptedAnswerId="5685" CreationDate="2013-03-26T07:28:22.187" Score="25" ViewCount="3259" Body="&lt;p&gt;&lt;strong&gt;Experiment description:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In Lagrange interpolation, the exact equation is sampled at &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; points (polynomial order &lt;span class=&quot;math-container&quot;&gt;$N - 1$&lt;/span&gt;) and it is interpolated at 101 points. Here &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is varied from 2 to 64. Each time &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_\infty$&lt;/span&gt; error plots are prepared. It is seen that, when the function is sampled at equi-spaced points, the error drops initially (it happens till &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is less than about 15 or so) and then the error goes up with further increase in &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Whereas, if the initial sampling is done at Legendre-Gauss (LG) points (roots of Legendre polynomials), or Legendre-Gauss-Lobatto (LGL) points (roots of Lobatto polynomials), the error drops to machine level and doesn't increase when &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is further increased.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My questions are,&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;What exactly happens in the case of equi-spaced points?&lt;/p&gt;&#xA;&lt;p&gt;Why does increase in polynomial order cause the error to rise after a certain point?&lt;/p&gt;&#xA;&lt;p&gt;Does this also mean that if I use equi-spaced points for WENO / ENO reconstruction (using Lagrange polynomials), then in the smooth region, I would get errors? (well, these are only hypothetical questions (for my understanding), it is really not reasonable to reconstruct polynomial of the order of 15 or higher for WENO scheme)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Additional details:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Function approximated:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x) =  \cos(\frac{\pi}{2}~x)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x \in [-1, 1]$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; divided into &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; equispaced (and later LG) points. The function is interpolated at 101 points each time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Results:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a) Equi-spaced points (interpolation for &lt;span class=&quot;math-container&quot;&gt;$N = 65$&lt;/span&gt;):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/IduCc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;b) Equi-spaced points (error plot, log scale):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/NlQQq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;a) LG points (Interpolation for &lt;span class=&quot;math-container&quot;&gt;$N = 65$&lt;/span&gt;):&#xA;&lt;img src=&quot;https://i.stack.imgur.com/shD94.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;b) LG points (error plot, log scale):&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EbeJf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-04-18T12:05:26.737" Title="Why do equi-spaced points behave badly?" Tags="|finite-difference|interpolation|error-estimation|polynomials|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5683" PostTypeId="1" AcceptedAnswerId="5695" CreationDate="2013-03-26T07:36:42.420" Score="3" ViewCount="72" Body="&lt;p&gt;I need to solve a &lt;em&gt;very large&lt;/em&gt; &lt;code&gt;O(10^10)&lt;/code&gt; linear selection problem in a distributed memory machine, is there any library that will solve it for me? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In shared memory machines, e.g. &lt;code&gt;std::nth_element&lt;/code&gt; does it in average &lt;code&gt;O(n)&lt;/code&gt; but deterministic &lt;code&gt;O(n)&lt;/code&gt; algorithms also exist. I could live with something worse as long as it's easy to implement. I would also prefer for it to work on input iterator/input ranges than to have linear complexity in order to save memory. I have a dataset that I want to postprocess and then use as input to the linear selection algorithm, it would be better if I could do it &quot;on the fly&quot;. I also only want to solve the linear selection problem for certain values, and wouldn't care to solve it multiple times, once for each value.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2013-03-27T08:01:59.933" LastActivityDate="2013-03-27T14:29:39.330" Title="Library for solving a linear selection problem in a distributed memory machine" Tags="|algorithms|c++|mpi|libraries|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="5684" PostTypeId="2" ParentId="5674" CreationDate="2013-03-26T07:50:56.310" Score="3" Body="&lt;p&gt;NCLab has three (simple) FEM modules - linear elasticity, electrostatics, and general linear second-order PDE with constant coefficients &lt;a href=&quot;https://nclab.com/pde&quot; rel=&quot;nofollow&quot;&gt;https://nclab.com/pde&lt;/a&gt;. Higher-order elements are enabled but not hp-adaptivity yet. You may still encounter bugs.&lt;/p&gt;&#xA;" OwnerUserId="3986" LastActivityDate="2013-03-26T07:50:56.310" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5685" PostTypeId="2" ParentId="5682" CreationDate="2013-03-26T09:56:00.607" Score="26" Body="&lt;p&gt;The problem with equispaced points is that the interpolation error polynomial, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) - P_n(x) = \frac{f^{(n+1)}(\xi)}{(n+1)!} \prod_{i=0}^n (x - x_i),\quad \xi\in[x_0,x_n] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;behaves differently for different sets of nodes $x_i$. In the case of equispaced points, this polynomial blows up at the edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use Gauss-Legendre points, the error polynomial is significantly better behaved, i.e. it doesn't blow up at the edges. If you use &lt;a href=&quot;http://en.wikipedia.org/wiki/Chebyshev_nodes&quot;&gt;Chebyshev nodes&lt;/a&gt;, this polynomial equioscillates and the interpolation error is minimal.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-03-26T09:56:00.607" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5686" PostTypeId="2" ParentId="5671" CreationDate="2013-03-26T14:24:35.250" Score="1" Body="&lt;p&gt;You do not need to copy the state vector. Assuming you are storing two copies X1 and X2, you can have the parallel region first work on X1, and then work on X2 on the next iteration. In Fortran, one easy way to do so is to move the parallel region into a subroutine, and to call it from within the main loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while t&amp;lt;time_horizon do&#xA;   call evolve(X1, X2)&#xA;   t=t+h&#xA;   if (t&amp;gt;=time_horizon) then&#xA;      &quot;swap x1 and x2&quot;&#xA;      exit&#xA;   end if&#xA;   call evolve(X2, X1)&#xA;   t=t+h&#xA;end while&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3987" LastActivityDate="2013-03-26T14:24:35.250" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5687" PostTypeId="2" ParentId="5680" CreationDate="2013-03-26T15:37:24.810" Score="7" Body="&lt;p&gt;In the comments to Johan's post I said it seems a shame to throw a full MIQP solver at this. For a general $n$-dimensional polyhedron, I'd certainly hold to that. But since this is a 3-dimensional problem, it might be competitive to do an intelligent exhaustive search. I suppose it depends on the application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, suppose we have constructed a &lt;a href=&quot;http://en.wikipedia.org/wiki/Generator_%28computer_programming%29&quot; rel=&quot;nofollow&quot;&gt;generator&lt;/a&gt; that enumerates all of the lattice points in (partial) order of increasing Manhattan distance from the target vertex $x_k$. That is, if $k&amp;gt;l$, then $\|x_k-z\|_\infty \geq \|x_l-z\|_\infty$. Mahnattan distance is easier for enumeration purposes; we'll handle the need to pick the best Euclidean distance in the filtering algorithm below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for the filter: let $d_{\text{max}}=\max_i\|v_i-z\|_\infty$, where $v_i$ are the vertices of the polygon, and set $d_{\text{best}}=+\infty$. Then, for $k=1,2,3,...$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If $\|x_k-z\|_\infty &amp;gt; d_{\text{max}}$, terminate.&lt;/li&gt;&#xA;&lt;li&gt;If $x_k$ falls outside of the polygon, repeat steps 1-5 with $k\leftarrow k+1$.&lt;/li&gt;&#xA;&lt;li&gt;If $\|x_k-z\|_\infty &amp;gt; d_{\text{best}}$, terminate.&lt;/li&gt;&#xA;&lt;li&gt;If $\|x_k-z\|_2 &amp;lt; d_{\text{best}}$, set $x_{\text{best}}\leftarrow x_k$ and $d_{\text{best}}=\|x_k-z\|_2$.&lt;/li&gt;&#xA;&lt;li&gt;Repeat steps 1-5 with $k\leftarrow k+1$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This loop will terminate in step 1 if there are no lattice points inside the polygon, and in either step 1 or step 3 when no points closer than $d_{\text{best}}$ remain. If $d_{\text{best}}=+\infty$ on termination, no lattice point was found within the polyhedron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding step 3 is important: since $\|x-z\|_2 \geq \|x-z\|_\infty$, this guarantees that no &lt;em&gt;subsequent&lt;/em&gt; points from the generator will have a lower Euclidean distance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So construct a generator and we're all set, right? Well, guaranteeing the partial ordering $k\geq l ~\rightarrow~ \|x_k-z\|_\infty \geq \|x_l-z\|_\infty$ might be difficult. In particular, it will likely require an internal buffer for sorting, which would be worth avoiding. So let's go simpler: let $\tilde{z}$ be the lattice point obtained by rounding the coordinates of $z$, and consider a a generator that guarantees $\|x_k-\tilde{z}\|_\infty \geq \|x_l-\tilde{z}\|_\infty$ for $k\geq l$. This is much simpler: for a 3-dimensional lattice, this can be programmed with 3 or 4 nested &lt;code&gt;for&lt;/code&gt; loops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge now is that you need to relax Step 3 a bit, because the points coming out of the generator no longer satisfy the original partial ordering. But&#xA;$$&#xA;\|x_k-z\|_\infty = \|x_k-\tilde{z}+(z-\tilde{z})\|_\infty \leq \|x_k-\tilde{z}\|_\infty + \|z-\tilde{z} \|_\infty \leq \|x_k-\tilde{z}\|_\infty + 0.5&#xA;$$&#xA;So I believe that relaxing the test in Step 3 to $\|x_k-\tilde{z}\|_\infty + 0.5 &amp;gt; d_{\text{best}}$ will restore correctness. I'd change $d_{\text{max}}=\max_i\|v_i-\tilde{z}\|_\infty$ and modify Step 1 to test $\|x_k-\tilde{z}\|_\infty$ as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're going to be testing $O(\lceil d_{\text{best}}\rceil^3)$ points with this approach, with a worst case of $O(\lceil d_{\text{max}} \rceil^3)$. But depending upon your application, it might be better than using a general-purpose MIQP solver. And it will be easier than coding one of your own for the same purpose.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-03-26T20:32:51.267" LastActivityDate="2013-03-26T20:32:51.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5689" PostTypeId="1" CreationDate="2013-03-26T20:39:51.640" Score="13" ViewCount="729" Body="&lt;p&gt;Several families of methods exist for numeric quadrature.  If I have a specific class of integrands how do I select the ideal method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the relevant questions to ask both about the integrand (e.g. is it smooth? does it have singularities?) and the computational problem (e.g. error tolerance, computational budget)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do answers to these questions rule out or promote the various families of methods? For simplicity lets consider just single or low-dimensional integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example the &lt;a href=&quot;http://en.wikipedia.org/wiki/QUADPACK&quot;&gt;Wikipedia article on QUADPACK&lt;/a&gt; states that the fairly general &lt;code&gt;QAGS&lt;/code&gt; routine &quot;&lt;em&gt;uses global adaptive quadrature based on 21-point Gauss–Kronrod quadrature within each subinterval, with acceleration by Peter Wynn's epsilon algorithm&lt;/em&gt;&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How was this decision made?  How can one make similar decisions when more is known?&lt;/p&gt;&#xA;" OwnerUserId="19" LastActivityDate="2013-03-27T07:13:27.833" Title="Method selection for numeric quadrature" Tags="|quadrature|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5690" PostTypeId="2" ParentId="5689" CreationDate="2013-03-26T21:27:23.563" Score="12" Body="&lt;p&gt;First of all, you need to ask yourself the question if you need an all-round quadrature routine that should take an integrand as a black box. If so, you cannot but go for adaptive quadrature where you hope that the adaptivity will catch &quot;difficult&quot; spots in the integrand. And that is one of the reasons Piessens et al. chose for a Gauss-Kronrod rule (this type of rule allows you to calculate an approximation of the integral and an estimate of the approximation error using the same function evaluations) of modest order applied in an adaptive scheme (with subdivision of the interval with the highest error) until the required tolerances are reached. The Wynn-epsilon algorithm allows to provide convergence acceleration and typically helps in the cases where there are end-point singularities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you do know the &quot;form&quot; or &quot;type&quot; of your integrand, you can tailor your method to what you need so the computational cost is limited for the accuracy you need. So what you need to look at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integrand:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Smoothness: can it be approximated (well) by a polynomial from an orthogonal polynomial family (if so, Gaussian quadrature will do well)&lt;/li&gt;&#xA;&lt;li&gt;Singularities: can the integral be split in integrals with only end-point-singularities (if so, the IMT-rule or double exponential quadrature will be good on each sub-interval)&lt;/li&gt;&#xA;&lt;li&gt;Computational cost for evaluation? &lt;/li&gt;&#xA;&lt;li&gt;Can the integrand be computed? Or is only limited point-wise data available? &lt;/li&gt;&#xA;&lt;li&gt;Highly oscillatory integrand: look for Levin-type methods. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When dealing with singularities, one typically prefers them to be at the end-point of the integrals (see IMT, double exponential). If this is not the case, one can resort to Clenshaw-Curtis integration where you capture the singularities in the weight-function. One typically defines forms of singularities like $|x-c|^{-\alpha}$ and establishes expressions for the weights of the quadrature as a function of $c$ and $\alpha$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integration interval: finite, semi-infinite or infinite. In case of semi-infinite or infinite intervals, can they be reduced to a finite interval by a variable transformation? If not, Laguerre or Hermite polynomials can be used in the Gaussian quadrature approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have a reference for a real flow sheet for quadrature in general, but the QUADPACK book (not the Netlib manpages, but the real book) has a flow sheet to select the appropriate routine based on the integral you want to evaluate. The book also describes the choices in algorithms made by Piessens et al. for the different routines. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For low-dimensional integrals, one typically goes for nested one-dimensional quadrature. In the special case of two-dimensional integrals (cubature), there exist integration rules for different cases of integration domains. R. Cools has collected a large number of rules in his &lt;a href=&quot;http://nines.cs.kuleuven.be/ecf/&quot; rel=&quot;noreferrer&quot;&gt;Encyclopedia of cubature formulas&lt;/a&gt; and is the main author of the &lt;a href=&quot;http://nines.cs.kuleuven.be/software/CUBPACK/&quot; rel=&quot;noreferrer&quot;&gt;Cubpack&lt;/a&gt; package. For high dimensional integrals, one typically resorts to Monte Carlo type methods. However, one needs typically a very large number of integrand evaluations to get reasonable accuracy. For low-dimensional integrals, approximation methods like quadrature/cubature/nested quadrature often out-perform these stochastic methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;General interesting references:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Quadpack, Piessens, Robert; de Doncker-Kapenga, Elise; Überhuber, Christoph W.; Kahaner, David (1983). QUADPACK: A subroutine package for automatic integration. Springer-Verlag. ISBN 978-3-540-12553-2&lt;/li&gt;&#xA;&lt;li&gt;Methods of Numerical Integration: Second Edition, Ph. Davis and Ph. Rabinowitz, 2007, Dover Books on Mathematics, ISBN 978-0486453392&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-03-27T07:13:27.833" LastActivityDate="2013-03-27T07:13:27.833" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5691" PostTypeId="1" CreationDate="2013-03-27T02:19:02.087" Score="1" ViewCount="367" Body="&lt;p&gt;I managed to display the coordinates of x,y and z into a 3D mesh by using delaunay function. The coordinates are in .obj format actually and i have read it into matrix form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, i would like to extract certain features / regions from the 3D mesh, reconstruct a new mesh and get the coordinates of it (I would like to extract specific ROI or volume of interest (VOI) form the mesh).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 3D facial image mapped into triangular mesh, and i want to extract the nose region. the same method cam be used as template for other data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible? Thank you in advance.&lt;/p&gt;&#xA;" OwnerDisplayName="user3991" LastActivityDate="2013-03-27T02:19:02.087" Title="features recognition &amp; reconstruction of 3d mesh delaunay matlab" Tags="|matlab|mesh-generation|delaunay-triangulation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5692" PostTypeId="1" AcceptedAnswerId="5696" CreationDate="2013-03-27T09:46:54.913" Score="5" ViewCount="331" Body="&lt;p&gt;I need to compute a closest point on a nonconvex polyhedron to a given point in 3D space. I need a simple algorithm or library. I search in &lt;a href=&quot;http://www.cgal.org/&quot; rel=&quot;noreferrer&quot;&gt;CGAL&lt;/a&gt; but did not find a suitable function and the package is very complicated. I also looked on Mauch's &lt;a href=&quot;https://bitbucket.org/seanmauch/stlib/src/7a4d7b836a73/src/cpt?at=default&quot; rel=&quot;noreferrer&quot;&gt;CPT&lt;/a&gt; (it does not compile) and &lt;a href=&quot;http://code.google.com/p/gts-cpt/&quot; rel=&quot;noreferrer&quot;&gt;gts-cpt&lt;/a&gt; (the result is not correct). I do not need any fast algorithm. The calculation is done only once for the grid during initialization. Simple to implement is MUCH more important to me.&lt;/p&gt;&#xA;" OwnerUserId="1244" LastActivityDate="2013-03-27T14:32:07.857" Title="Library for closest point on a polyhedron" Tags="|computational-geometry|closest-point|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5693" PostTypeId="2" ParentId="5692" CreationDate="2013-03-27T12:01:42.287" Score="4" Body="&lt;p&gt;A key phrase that will find you the literature is &lt;em&gt;collision detection&lt;/em&gt; (because one needs to find the closest point to a polyhedral model to detect impending collision).  For example:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Ponamgi, Manocha, Lin.&#xA;  &quot;Incremental algorithms for collision detection between solid models.&quot;&#xA;  (&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=218076&quot; rel=&quot;nofollow&quot;&gt;ACM link&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A software package named SWIFT++ is maintained at the Univ. North Carolina:  &lt;a href=&quot;http://gamma.cs.unc.edu/SWIFT++/&quot; rel=&quot;nofollow&quot;&gt;SWIFT++ link&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;SWIFT++ is a collision detection package capable of detecting intersection, performing tolerance verification, computing approximate and exact distance, or determining the contacts between pairs of objects in a scene composed of general rigid polyhedral models. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Caveat: None of this is simple or straightforward!&lt;/p&gt;&#xA;" OwnerUserId="673" LastActivityDate="2013-03-27T12:01:42.287" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="5694" PostTypeId="1" AcceptedAnswerId="6728" CreationDate="2013-03-27T12:33:37.607" Score="0" ViewCount="196" Body="&lt;p&gt;I'm an engineer and I'm planning to get a bigger toolbox than Excel to solve difficult problems. I started learning Python (as that seems the script language to go for math intense jobs, and runs in the background of many applications) and plan to take a deep look at Sage and Ascend, as those seem to be two good free packages that do different jobs. Ascend especially was written with process engineering in mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I pursue this in my spare time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I look for problems that are not too far from my background  - process &amp;amp; environmental engnieering. Possible tasks (that I think lend them selve to numerical solutions):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;compressible flow through channel&lt;/li&gt;&#xA;&lt;li&gt;flow through networks of pipes (Gas, heating)&lt;/li&gt;&#xA;&lt;li&gt;mass-flow through complex systems with lots of interconnections and chemical/biological reactions&lt;/li&gt;&#xA;&lt;li&gt;thermodynamic cycles&lt;/li&gt;&#xA;&lt;li&gt;sizing heat exchangers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think in most professional environments this would be done in ASPEN or Aquasim, the company I'm at doesn't (need to) work like that. Note that I'm not looking at CFD or Finite Elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of these I could get from my work - but without a correct solution to benchmark my calculations. Ideally, I would have a problem and a correct solution, where the problems need some engineering or physics understanding and the ability to translate this into some code (depending on the platform used to solve) - or a &lt;a href=&quot;http://projecteuler.net/&quot; rel=&quot;nofollow&quot;&gt;Project Euler&lt;/a&gt; for engineers!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find worked examples with results to sharpen my teeth on?&lt;/p&gt;&#xA;" OwnerUserId="3993" LastEditorUserId="276" LastEditDate="2013-04-01T20:14:26.320" LastActivityDate="2013-04-01T20:14:26.320" Title="Where do I find engineering problems to practice solving computationally?" Tags="|reference-request|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="5695" PostTypeId="2" ParentId="5683" CreationDate="2013-03-27T14:29:39.330" Score="2" Body="&lt;p&gt;I think you want to take a look at section 5.1 of this one of my papers:&#xA;  &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications/2010-distributed.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/publications/2010-distributed.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm described there has complexity $O\left(\frac NP \log_2 P\right)$. The code has only some 20 lines.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-27T14:29:39.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5696" PostTypeId="2" ParentId="5692" CreationDate="2013-03-27T14:32:07.857" Score="0" Body="&lt;p&gt;In the end I found that the example &lt;a href=&quot;http://www.cgal.org/Manual/latest/examples/AABB_tree/AABB_polyhedron_facet_distance_example.cpp&quot; rel=&quot;nofollow&quot;&gt;http://www.cgal.org/Manual/latest/examples/AABB_tree/AABB_polyhedron_facet_distance_example.cpp&lt;/a&gt; does exactly what I need. How should a non expert like me know that he should search in some AABB I do not know.&lt;/p&gt;&#xA;" OwnerUserId="1244" LastActivityDate="2013-03-27T14:32:07.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="5697" PostTypeId="1" CreationDate="2013-03-27T14:42:53.010" Score="0" ViewCount="118" Body="&lt;p&gt;I have the following dataset that I need to display on the heat map:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[30, 15, 66, 7, 9999, 78, 42, 132]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if I map the values to the color scale using a linear function I only see the spike while the rest of the values I cannot tell apart because the difference between them is insignificant in the scale of that spike.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried a log function but it didn't get a good looking picture either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't care about accurate correspondence between the value and the color intensity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to map the values to the color scale so that the difference between all values was visible? Can I mitigate that spike somehow? I am rather asking about established approaches or algorithms addressing problems like this.&lt;/p&gt;&#xA;" OwnerUserId="1985" LastActivityDate="2013-03-27T14:59:02.347" Title="mapping data with a spike to a heat map" Tags="|visualization|data-management|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="5698" PostTypeId="2" ParentId="5697" CreationDate="2013-03-27T14:49:45.533" Score="2" Body="&lt;p&gt;There are many different ways of handling outliers, as well as answers to similar questions here already. I would recommend taking a look at&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/9811866/ggplot2-color-scale-over-affected-by-outliers&quot;&gt;https://stackoverflow.com/questions/9811866/ggplot2-color-scale-over-affected-by-outliers&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Outlier#Working_with_outliers&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Outlier#Working_with_outliers&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first deals with this problem in R. Taking the approach of graphing the non-outliers with a heatmap, then overlaying the outliers and annotating the legend to describe that this is what was done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second contains links to general solutions and approaches to outliers.&lt;/p&gt;&#xA;" OwnerUserId="1702" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-03-27T14:59:02.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6697" PostTypeId="1" AcceptedAnswerId="6699" CreationDate="2013-03-27T17:13:10.763" Score="2" ViewCount="145" Body="&lt;p&gt;I am currently learning a code which utilizes Scharfetter-Gummel discretization for unsteady drift-diffusion equations.  For this scheme, a 2D unstructured triangular mesh is used, with the unknowns located at the vertices of the mesh.  Cells are then constructed by perpendicular lines bisecting the chords between vertices and applying the finite volume formulation on the fluxes and internal processes in each cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may simply be my inexperience with unstructured solvers, but I am wondering if there are equations/discretization schemes which would benefit from having the unknowns located at the centroid of each triangle, and the fluxes calculated over the chords.  For instance I know that the current code (and possible future replacements) also solve compressible laminar Navier-Stokes, the energy equation, Poisson's equation, and possible more accurate/coupled drift-diffusion models.  It would also be helpful for understanding if answers included whether it is extendable to 3D (I am assuming it would be).&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-03-27T17:55:19.920" Title="Location of Unknowns in Unstructured Mesh" Tags="|pde|unstructured-mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6698" PostTypeId="2" ParentId="5674" CreationDate="2013-03-27T17:19:31.900" Score="-1" Body="&lt;p&gt;There are a couple of blog posts for solving 2D finite element problems using MATLAB at this scientific computing website.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.particleincell.com/2012/matlab-fem/&quot; rel=&quot;nofollow&quot;&gt;http://www.particleincell.com/2012/matlab-fem/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.particleincell.com/2012/finite-element-examples/&quot; rel=&quot;nofollow&quot;&gt;http://www.particleincell.com/2012/finite-element-examples/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They make use of some MATLAB code from MIT and the distmesh tool for creating a mesh over a 2D domain.&lt;/p&gt;&#xA;" OwnerUserId="3994" LastActivityDate="2013-03-27T17:19:31.900" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6699" PostTypeId="2" ParentId="6697" CreationDate="2013-03-27T17:55:19.920" Score="3" Body="&lt;p&gt;The dual mesh (the one formed by the perpendicular bisectors) is typically used as a FV mesh to allow the two-point flux approximation to be valid.  One requirement of the two-point flux approximation (in which the flux is based only upon the difference of the values at two neighboring cell centers) is that the segment connecting cell centroids is perpendicular to the edge the segment crosses.  If you draw two arbitrary triangles sharing an edge, you can see that this is typically not true for most simplex meshes.  However, if you then form the dual, you're left with a mesh for which this is true (your primal mesh's edges become the segments connecting cell centroids, and by construction are perpendicular to the new dual mesh's edges). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are discretizations that can get around this issue and work on the primal mesh, in which triangles are now the cells.  These include mixed FE discretizations (the simplest is Raviart-Thomas finite elements) and Mimetic Finite Difference methods (nearly mathematically equivalent to methods known as nonlinear Finite Volume methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second set of methods (MFD and variants) don't really &quot;benefit&quot; from working on the primal mesh, but their accuracy is not necessarily worse order, whereas standard FV methods can become 0th order as the above assumption is violated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(All of the above, including forming the dual mesh from tetrahedrons instead of triangles, have natural extensions to 3D.)&lt;/p&gt;&#xA;" OwnerUserId="508" LastActivityDate="2013-03-27T17:55:19.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6701" PostTypeId="1" AcceptedAnswerId="6704" CreationDate="2013-03-28T00:15:18.767" Score="7" ViewCount="12318" Body="&lt;p&gt;Thomas algorithm can be used to solve a tridiagonal matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{bmatrix}&#xA;   {b_ 1} &amp;amp; {c_ 1} &amp;amp; {   } &amp;amp; {   } &amp;amp; { 0 } \\&#xA;   {a_ 2} &amp;amp; {b_ 2} &amp;amp; {c_ 2} &amp;amp; {   } &amp;amp; {   } \\&#xA;   {   } &amp;amp; {a_ 3} &amp;amp; {b_ 3} &amp;amp; \ddots &amp;amp; {   } \\&#xA;   {   } &amp;amp; {   } &amp;amp; \ddots &amp;amp; \ddots &amp;amp; {c_{n-1}}\\&#xA;   { 0 } &amp;amp; {   } &amp;amp; {   } &amp;amp; {a_n} &amp;amp; {b_n}\\&#xA;\end{bmatrix}&#xA;\cdot&#xA;\begin{bmatrix}&#xA;   {x_ 1 }  \\&#xA;   {x_ 2 }  \\&#xA;   {x_ 3 }  \\&#xA;   \vdots   \\&#xA;   {x_n }  \\&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;   {d_ 1 }  \\&#xA;   {d_ 2 }  \\&#xA;   {d_ 3 }  \\&#xA;   \vdots   \\&#xA;   {d_n }  \\&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution can be get by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;c'_i=&#xA;\begin{cases}&#xA;\begin{array}{lcl}&#xA;  \cfrac{c_i}{b_i}                  &amp;amp;&amp;amp;; i = 1 \\&#xA;  \cfrac{c_i}{b_i - c'_{i - 1} a_i} &amp;amp;&amp;amp;; i = 2, 3, \&#xA;\dots, n-1 \\&#xA;\end{array}&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;d'_i=&#xA;\begin{cases}&#xA;\begin{array}{lcl}&#xA;  \cfrac{d_i}{b_i}                  &amp;amp;&amp;amp;; i = 1 \\&#xA;  \cfrac{d_i - d'_{i - 1} a_i}{b_i - c'_{i - 1} a_i} &amp;amp;&amp;amp;; i = 2, 3, \dots, n. \\&#xA;\end{array}&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{lcl}&#xA;x_n&amp;amp;=&amp;amp;d'_n\\&#xA;x_i&amp;amp;=&amp;amp;d'_i-c'_ix_{i+1}&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to generalize this algorithm to block tridiagonal matrix, ie. both a,b,c are small square matrix itself. Do I just replace &quot;times&quot; by &quot;dot&quot; and &quot;divide&quot; by &quot;dot inverse&quot; in the above formula, and everything is OK?&lt;/p&gt;&#xA;" OwnerUserId="3997" LastActivityDate="2018-11-08T00:38:54.787" Title="How to solve block tridiagonal matrix using Thomas algorithm" Tags="|algorithms|linear-solver|sparse-matrix|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6702" PostTypeId="1" CreationDate="2013-03-28T00:15:24.780" Score="6" ViewCount="448" Body="&lt;p&gt;I'm a high school student interested in computational science, and I would like to learn more about it. This year I took AP Computer Science for that reason, but except for some very basic gambling stuff we didn't really do any modeling or simulation. Can one of you kind sirs/madams please point me towards a good introductory text? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's some more info, in case it's relevant:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;This seems to be pretty math heavy. I did Calc BC so that's roughly equivalent to Calc I/II, and I know the very basics of linear algebra as well. I imagine that's not quite enough?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I know Java, and I'm familiar with C. Are they okay, or should I learn something else? Fortran and python both seem to get mentioned a lot here.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I got interested in this after doing some modeling for robotics club, and my interest definitely leans towards computational engineering. I think computational astrophysics is really cool, too.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3998" LastActivityDate="2019-01-04T10:53:24.780" Title="Introduction to computational science?" Tags="|reference-request|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="6703" PostTypeId="2" ParentId="6701" CreationDate="2013-03-28T01:24:56.313" Score="7" Body="&lt;p&gt;Yes. Except it's not &quot;dot&quot; but instead the product of matrices, resulting in another matrix.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-28T01:24:56.313" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6704" PostTypeId="2" ParentId="6701" CreationDate="2013-03-28T09:41:31.593" Score="1" Body="&lt;p&gt;I think it will be too much time consuming, if I try to give you answer and to correct superscripts,subscripts,capital and small letters of your query. Instead of that you can refer a detailed description of Thomas Algorithm from a book &quot;Computational Fluid Dynamics&quot; by John Anderson. You can directly refer page number &quot;534&quot; for it (Publisher: McGraw-Hill Science/Engineering/Math; 1st edition (February 1, 1995)). Its very easy to understand and implement it from there. All the best.&lt;/p&gt;&#xA;" OwnerUserId="3549" LastActivityDate="2013-03-28T09:41:31.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6706" PostTypeId="1" AcceptedAnswerId="6709" CreationDate="2013-03-28T18:06:35.243" Score="7" ViewCount="2814" Body="&lt;p&gt;The motivation for my question is the following: in one of Project Euler questions there is a need to count the spanning trees of a rectangular grid graph of dimension 100x500. By the Matrix-Tree theorem this number is equal to the determinant of the matrix obtained by taking the graph Laplacian and deleting its first row and column. The Laplacian is very sparse and positive definite, but it is a  50000x50000 matrix and it seemed to me not feasible to calculate the determinant with the required precision (5 significant digits in exponential notation: may be I am wrong here?). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found the following interesting reference&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arXiv.org/abs/0712.0681v3&quot;&gt;http://arXiv.org/abs/0712.0681v3&lt;/a&gt;&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which permits to compute the determinant as the determinant of a 100x100 matrix obtained as the left upper corner of the product of 500 200x200 sparse matrices, using the fact that the Laplacian is block tridiagonal (and deriving from the Laplacian a nonsingular 50000x50000 matrix whose determinant is equal to the determinant of the Laplacian with the first row and column deleted), which seemed to be a good thing. &#xA;However the resulting real 100x100 matrix is nonsymmetric, dense, has some very large entries and some very small (e^(- ...)) eigenvalues (I experimented with smaller grids). Because of the very small eigenvalues I am not sure that any of the numerical methods (Gaussian elimination, SVD etc) will work properly, if I don't keep all the digits of the entries of the matrix. But this means to work with arbitrary precision which is probably very slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: does anyone know of a practical method which can be used in such situation to compute the determinant with the required precision (again, 5 significant digits in exponential notation)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the answer to the original Project Euler problem in the form of product of trigonometric terms, so really my interest is not in the solution itself but in these linear algebra questions which come out naturally when I try to apply the Matrix-Tree theorem directly.&lt;/p&gt;&#xA;" OwnerUserId="18877" LastEditorUserId="18877" LastEditDate="2013-03-28T18:17:21.997" LastActivityDate="2015-09-20T06:35:37.430" Title="computing the determinant of a dense nonsymmetric 100x100 matrix having very big and very small eigenvalues" Tags="|linear-algebra|eigensystem|sparse-matrix|precision|dense-matrix|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6707" PostTypeId="1" CreationDate="2013-03-28T13:54:59.403" Score="6" ViewCount="1254" Body="&lt;p&gt;I use OpenFOAM for CFD simulations. A very popular preconditioner is &lt;a href=&quot;http://www.openfoam.com/features/linear-solvers.php&quot;&gt;GAMG&lt;/a&gt; which needs a low number of iterations per a time step in SIMPLE or PISO solvers that are used to simulate the fluid flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to understand how GAMG prepares the internal structures. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it create the internal data structures used in GAMG &lt;strong&gt;only once&lt;/strong&gt; in a single iteration on the basis of the geometry and then the entries in these data structures are only updated? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OR &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it create the interal data structures multiple times in a single iteration? &#xA;I read the documentation &lt;a href=&quot;http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2008/TimBehrens/tibeh-report-fin.pdf&quot;&gt;here&lt;/a&gt; but it does not have the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your answer.&lt;/p&gt;&#xA;" OwnerDisplayName="luksmir" LastActivityDate="2013-03-28T19:33:24.920" Title="How does GAMG in OpenFOAM really work?" Tags="|fluid-dynamics|simulation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6708" PostTypeId="1" AcceptedAnswerId="6725" CreationDate="2013-03-28T20:56:39.583" Score="7" ViewCount="7859" Body="&lt;p&gt;I tried to make the question as detailed as possible. I have an extremely simple solver written for the Schroedinger equation but with imaginary time, which transforms it basically into the diffusion equation (with a potential term). The method is pretty well documented on this page, and I basically followed the steps almost exactly. Here's the link for that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jkwiens.com/2010/01/02/finite-difference-heat-equation-using-numpy/&quot;&gt;http://jkwiens.com/2010/01/02/finite-difference-heat-equation-using-numpy/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the corresponding Python code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jkwiens.com/heat-equation-using-finite-difference/&quot;&gt;http://jkwiens.com/heat-equation-using-finite-difference/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy as sp&#xA;from scipy import integrate, sparse, linalg&#xA;import scipy.sparse.linalg&#xA;import pylab as pl&#xA;&#xA;nx = 8000&#xA;dx = 0.0025&#xA;dt = 0.00002&#xA;niter = 20&#xA;nonlin = 0.0&#xA;gridx = sp.zeros(nx)&#xA;igridx = sp.array(range(nx))&#xA;psi = sp.zeros(nx)&#xA;pot = sp.zeros(nx)&#xA;depth = 0.01&#xA;&#xA;# Set up grid, potential, and initial state&#xA;gridx = dx*(igridx - nx/2)&#xA;pot = depth*gridx*gridx&#xA;psi = sp.pi**(-1/4)*sp.exp(-0.5*gridx*gridx)&#xA;&#xA;# Normalize Psi&#xA;#psi /= sp.integrate.simps(psi*psi, dx=dx)&#xA;&#xA;# Plot parameters&#xA;xlimit = [gridx[0], gridx[-1]]&#xA;ylimit = [0, 2*psi[nx/2]]&#xA;&#xA;# Set up diagonal coefficients&#xA;Adiag = sp.empty(nx)&#xA;Asup = sp.empty(nx)&#xA;Asub = sp.empty(nx)&#xA;bdiag = sp.empty(nx)&#xA;bsup = sp.empty(nx)&#xA;bsub = sp.empty(nx)&#xA;Adiag.fill(1 - dt/dx**2)&#xA;Asup.fill(dt/(2*dx**2))&#xA;Asub.fill(dt/(2*dx**2))&#xA;bdiag.fill(1 + dt/dx**2)&#xA;bsup.fill(-dt/(2*dx**2))&#xA;bsub.fill(-dt/(2*dx**2))&#xA;&#xA;# Construct tridiagonal matrix&#xA;A = sp.sparse.spdiags([Adiag, Asup, Asub], [0, 1, -1], nx, nx)&#xA;b = sp.sparse.spdiags([bdiag, bsup, bsub], [0, 1, -1], nx, nx)&#xA;&#xA;# Loop through time&#xA;for t in range(0, niter) :&#xA;    # Calculate effect of potential and nonlinearity&#xA;    psi *= sp.exp(-dt*(pot + nonlin*psi*psi))&#xA;&#xA;    # Calculate spacial derivatives&#xA;    psi = sp.sparse.linalg.bicg(A, b*psi)[0]&#xA;&#xA;    # Normalize Psi&#xA;    psi /= sp.integrate.simps(psi*psi, dx=dx)&#xA;&#xA;    # Output figures&#xA;    pl.plot(gridx, psi)&#xA;    pl.plot(gridx, psi*psi)&#xA;    pl.plot(gridx, pot)&#xA;    pl.xlim(xlimit)&#xA;    pl.ylim(ylimit)&#xA;    pl.savefig('outputla/fig' + str(t))&#xA;    pl.clf()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;nonlin&lt;/code&gt; variable is just an extra term for nonlinearity (I'll be solving the nonlinear Schroedinger equation later), but if I set it to zero then it is irrelevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem now is that the spatial operator doesn't do anything (it's supposed to diffuse rather quickly), and eventually the entire thing blows up (typical instability.. lines get jagged and eventually the plot just looks crazy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've checked everything I can think of, and I have absolutely no clue why this is happening. I have changed linear solvers, dense matrices instead of sparse, the coefficients for &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; are correct (I think.. they are different from the blog link because Schro. eq. has an &lt;code&gt;i&lt;/code&gt; term, so some negative-positive sign swappings happen.. this is mentioned in the paper referenced below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, here is a paper that describes the calculations (but I don't use the recursion relation, just an operator): &lt;a href=&quot;http://arxiv.org/abs/0904.3131&quot;&gt;http://arxiv.org/abs/0904.3131&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2013-03-28T22:43:31.440" LastActivityDate="2013-03-31T06:57:23.060" Title="Schroedinger/Diffusion equation with Crank-Nicolson in Python/SciPy" Tags="|finite-difference|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6709" PostTypeId="2" ParentId="6706" CreationDate="2013-03-28T22:59:39.727" Score="4" Body="&lt;p&gt;Wikipedia actually has a nice overview of approaches:&#xA;  &lt;a href=&quot;http://en.wikipedia.org/wiki/Determinant#Calculation&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Determinant#Calculation&lt;/a&gt;&#xA;As a general remark, people do not compute determinants of large matrices (large would here be of size &gt;10 or &gt;50) because this is numerically difficult and, likely, not very stable anyway. If you need to do it, I would see if maybe there are randomized algorithms in much the same way as you can use a Monte Carlo approach to approximate the trace.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-28T22:59:39.727" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6710" PostTypeId="1" AcceptedAnswerId="6712" CreationDate="2013-03-28T23:22:19.840" Score="5" ViewCount="1268" Body="&lt;p&gt;Consider a scalar field $u$ on an unstructured triangle mesh which is constant on each face.  Let $A_i$ be the area of triangle $T_i$, $N(i)$ the set of triangles sharing an edge with $T_i$, and $L_{ij}$ the length of the edge between $T_i$ and $T_j$.  I want the simplest possible spatial discretization of the heat equation&#xA;$$u_t = k \Delta u$$&#xA;The natural choice is&#xA;$$A_i \frac{du_i}{dt} = k \sum_{j \in N(i)} L_{ij} \frac{u_j - u_i}{h_{ij}}$$&#xA;where $h_{ij}$ measures &quot;how far apart&quot; the triangles are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The question&lt;/strong&gt;: What are good choices for $h_{ij}$?  In particular, are there choices which behave reasonably in the presence of obtuse/skinny triangles?&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-03-29T11:56:11.107" Title="First order finite volume spatial discretization of the heat equation on an unstructured triangle mesh" Tags="|parabolic-pde|unstructured-mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6711" PostTypeId="2" ParentId="5680" CreationDate="2013-03-29T00:54:49.830" Score="3" Body="&lt;p&gt;Both linear and integer programming can be solved in fast polynomial time if the dimension is fixed.  Here is an example reference for the integer case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.8.3999&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;Eisenbrand, &quot;Fast integer programming in fixed dimension&quot;, 2003.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-03-29T00:54:49.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6712" PostTypeId="2" ParentId="6710" CreationDate="2013-03-29T02:48:42.487" Score="6" Body="&lt;p&gt;A two-point flux like this is not convergent if the mesh is not &quot;orthogonal&quot;, in the sense that the edge/face between two cells is orthogonal to the line segment joining the cell centroids. If your mesh is orthogonal, you would use the distance between centroids for $h_{ij}$ above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like a method to work on more general meshes within the cell-centered finite volume framework, you have to do a bit of extra work. A classical approach is to reconstruct inside cells, as outline in &lt;a href=&quot;https://engineering.purdue.edu/ME608/webpage/main.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these notes&lt;/a&gt;. This is not very robust, especially in the presence of irregular coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative is to consider the mixed finite element method using a BDM-1 space (constant pressure, linear velocity on faces/edges) and choose a reduced quadrature consisting of one point per vertex. That special quadrature causes the velocity mass matrix to be block diagonal, with one block per vertex, so the velocity degrees of freedom can be eliminated, returning you to a cell-centered non-mixed formulation. This is illustrated in Figure 2.3 from &lt;a href=&quot;http://www.math.pitt.edu/~yotov/research/publications/mfmfe.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wheeler and Yotov (2006)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2h9UU.png&quot; alt=&quot;Interaction of degrees of freedom&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After velocity is eliminated, the cell-centered operator is SPD and cells are coupled only to other cells they share a vertex with, which is sparser than the gradient reconstruction method mentioned earlier. With this method, pressure is second order superconvergent and cell centers and velocity is first order convergent, even on irregular meshes. If the mesh is within $O(h^2)$ of affine, velocity is also second order superconvergent at face centers. This method naturally handles general tensor-valued coefficients. This method is a &lt;a href=&quot;http://dx.doi.org/10.1023/A:1021291114475&quot; rel=&quot;nofollow noreferrer&quot;&gt;multipoint flux approximation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is known (see page 10 of &lt;a href=&quot;http://dx.doi.org/10.1006/jcph.2000.6418&quot; rel=&quot;nofollow noreferrer&quot;&gt;Edwards&lt;/a&gt;) that 9-point schemes for regular quadrilateral meshes cannot be monotone for general tensor-valued coefficients. One approach to maintain monotonicity is to use a &lt;a href=&quot;http://cnls.lanl.gov/~shashkov/papers/article_monotoneFV.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;nonlinear discretization&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-03-29T02:58:18.990" LastActivityDate="2013-03-29T02:58:18.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6714" PostTypeId="2" ParentId="6702" CreationDate="2013-03-29T10:44:26.873" Score="8" Body="&lt;p&gt;Kate, you asked for a reference request as a high school student interested in getting started in computational science, which I think I can tackle fairly specifically.  As long as you are comfortable (or getting comfortable) with Calculus, I think there are two great self-interest textbooks for you to go, depending on your interest and access to Python and MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://images.springer.com/cda/content/image/cda_displayimage.jpg?SGWID=0-0-16-1533963-0&quot; alt=&quot;A Primer on Scientific Programming with Python&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first book, &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/3642024742&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Primer on Scientific Programming&lt;/a&gt;, by Hans Petter Langtangen, requires no more background than &quot;high school mathematics&quot; and some Calculus.  It is not free, but there may be a copy of the book at your local college library.  Python and all of the tools associated with the book are freely available and can be downloaded online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/vc9Ll.png&quot; alt=&quot;Experiments with MATLAB&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second book, &lt;a href=&quot;http://www.mathworks.co.uk/moler/exm/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Experiments with MATLAB&lt;/a&gt;, by Cleve Moler, &quot;is an electronic book with chapters that supplement high school and early college courses in mathematics and technical computing, including calculus and matrix theory. The expected background includes algebra, trigonometry, and some familiarity with computers.&quot;  Here, the situation is the opposite.  The book itself is free but a student version of MATLAB (if it is not available in your local high school or college computer laboratory) will cost you $100 for a license.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do not have access to MATLAB, there is also the free (open source) &lt;a href=&quot;http://www.gnu.org/software/octave/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU Octave&lt;/a&gt;. The core numerical commands (especially in the context you are interested in) are completely compatible with MATLAB (i.e., you can use most MATLAB scripts without any modification), although graphical capabilities may differ. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me be the third moderator to welcome you to the community and encourage you to ask &lt;a href=&quot;https://scicomp.stackexchange.com/faq&quot;&gt;good questions&lt;/a&gt; here, as well as helping when you have knowledge to provide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hans Petter and Cleve are two of the world's most recognized experts in computational science.  They both have very clear writing styles, and I strongly recommend either of these books.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="22284" LastEditDate="2019-01-04T10:53:24.780" LastActivityDate="2019-01-04T10:53:24.780" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6715" PostTypeId="2" ParentId="6710" CreationDate="2013-03-29T11:56:11.107" Score="2" Body="&lt;p&gt;If you use the discontinuous Galerkin method with piecewise constant shape functions, you end up with a scheme just like yours. It provides a systematic way of constructing the weights as well as higher order schemes if you want.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-03-29T11:56:11.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6716" PostTypeId="2" ParentId="3229" CreationDate="2013-03-29T13:03:39.843" Score="-2" Body="&lt;p&gt;What about&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my_array[:,[0, 1]] = my_array[:,[1, 0]]&lt;/p&gt;&#xA;" OwnerUserId="4005" LastActivityDate="2013-03-29T13:03:39.843" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6717" PostTypeId="2" ParentId="6706" CreationDate="2013-03-29T13:15:49.113" Score="5" Body="&lt;p&gt;The computation of integer-valued matrix determinants has been a subject of considerable research.  Using exact arithmetic the Smith normal form can be computed, and from this diagonal form the determinant is easily found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Saunders and Wan (2004), &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.144.1674&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Smith Normal Form of Dense Integer Matrices,&#xA;Fast Algorithms into Practice&lt;/a&gt;, say &quot;Over the past thirty years, steady progress has been made in algorithms for the Smith forms.&quot;  They give a reference for an asymptotically fastest known method (but which &quot;may not be pracical&quot;), and then dive into details of improvements on &lt;a href=&quot;http://perso.ens-lyon.fr/gilles.villard/BIBLIOGRAPHIE/PDF/detsmith.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;an algorithm by W. Eberly, M. Giesbrecht, and G. Villard&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially one wants to find the greatest common divisor of matrix entries, forming this by a series of row and column operations and using it to eliminate all other nonzero entries in its column.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://math.stackexchange.com/q/74441/3111&quot;&gt;unanswered Math.SE Question&lt;/a&gt; on &quot;fastest method&quot; for computing integer matrix determinants (in roughly the size you ask about) drew a couple of good Comments, one of which suggests (and reports a successful 100x100 experiment with) &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=LinearAlgebra/Determinant&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maple's Determinant function using method = integer&lt;/a&gt;.  Although their algorithm uses (exact) modular arithmetic, it entails &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=Probabilistic_Algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;a probabilistic element&lt;/a&gt; (a small but positive chance of giving a wrong answer).  The reason for this is that the computation is carried out for a sequence of primes, reducing the precision needed for each particular run, and in the absence of reaching an apriori bound on the determinant (making the answer deterministic), &quot;convergence&quot; may be decided by obtaining the same result after several such runs (the probabilistic case).&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-03-29T18:28:50.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6718" PostTypeId="2" ParentId="6702" CreationDate="2013-03-29T13:54:22.143" Score="6" Body="&lt;p&gt;&lt;img src=&quot;https://images.springer.com/cda/content/image/cda_displayimage.jpg?SGWID=0-0-16-743241-0&quot; alt=&quot;Scientific Computing with MATLAB and Octave&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aron gave some excellent references with a focus on programming in Python and Matlab in the context of computational science. Let me add a reference that goes into a bit more detail on the mathematical side (i.e., how and why these methods work). The book &lt;a href=&quot;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-3-642-12429-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scientific Computing with MATLAB and Octave&lt;/a&gt; by Alfio Quarteroni is accessible with basic knowledge in linear algebra and calculus, and contains a number of worked examples and applications. The emphasis is still on numerical computations and not on proofs, but it is more in-depth than Cleve Moler's book (which is a good place to start).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kwsfl.gif&quot; alt=&quot;Strang&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moving on from there, even more in-depth, but still very readable without a formal mathematical background, is Gil Strang's book on &lt;a href=&quot;http://math.mit.edu/cse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Science and Engineering&lt;/a&gt;. Gil is an outstanding teacher who has a gift for getting the key insights across with a minimum of technicalities. (On the linked page, you can find sample chapters and video lectures to see if this works for you. The text also makes use of MATLAB code, which you can download from there.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="22284" LastEditDate="2019-01-03T09:37:55.677" LastActivityDate="2019-01-03T09:37:55.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6719" PostTypeId="2" ParentId="6706" CreationDate="2013-03-29T14:05:02.487" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://cs.uwaterloo.ca/~astorjoh/iml.html&quot; rel=&quot;nofollow&quot;&gt;Integer Matrix Library&lt;/a&gt; provides a &lt;code&gt;C&lt;/code&gt; function &lt;code&gt;mDeterminant (const FiniteField p, Double *A, const long n)&lt;/code&gt; to compute the determinant &lt;code&gt;det(A) mod p&lt;/code&gt; of an integer matrix &lt;code&gt;A&lt;/code&gt; (using a row echelon transform), which might be of use.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-03-29T14:05:02.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6720" PostTypeId="2" ParentId="3229" CreationDate="2013-03-29T15:37:01.583" Score="1" Body="&lt;p&gt;I don't have enough reputation to comment, but I think the following SO question might be helpful: &lt;a href=&quot;https://stackoverflow.com/questions/4370745/view-onto-a-numpy-array&quot;&gt;https://stackoverflow.com/questions/4370745/view-onto-a-numpy-array&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic points are that you can use &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#basic-slicing&quot; rel=&quot;nofollow noreferrer&quot;&gt;basic slicing&lt;/a&gt; and that will create a view on to the array without copying, but if you do &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#advanced-indexing&quot; rel=&quot;nofollow noreferrer&quot;&gt;advanced slicing/indexing&lt;/a&gt; then it &lt;em&gt;will&lt;/em&gt; create a copy.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-03-29T15:37:01.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6721" PostTypeId="1" CreationDate="2013-03-29T18:39:49.940" Score="2" ViewCount="519" Body="&lt;p&gt;Printing photo matrices (x,y,z) where z is 0:2 colour, I want to see (x in rows, y in columns) 3 times, once for each colour (the way I've been taught by every mathematician). By default, numpy does it last dimension first.&lt;/p&gt;&#xA;" OwnerUserId="4005" LastActivityDate="2013-03-31T15:24:04.080" Title="python numpy print array(x,y,z) as (x by y) by z?" Tags="|python|numpy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6722" PostTypeId="1" AcceptedAnswerId="6942" CreationDate="2013-03-29T23:25:24.040" Score="4" ViewCount="393" Body="&lt;p&gt;I've reading up on the &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_sign_problem&quot; rel=&quot;nofollow&quot;&gt;numerical sign problem&lt;/a&gt;, and how a general solution is NP-Hard. I can't seem to find a proof of this, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know where I can find a proof that the numerical sign problem is NP-Hard?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="447" LastEditDate="2013-04-22T18:38:25.400" LastActivityDate="2013-04-22T18:38:25.400" Title="Where can I find a proof that the numerical sign problem is NP-hard?" Tags="|reference-request|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6723" PostTypeId="1" CreationDate="2013-03-30T18:00:25.940" Score="1" ViewCount="615" Body="&lt;p&gt;I have a problem by solving stokes flow in 2D by finite differences. I am using a marker and cell method, my scheme is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;o-------vx1.1-------o-------vx1.2-------o&#xA;|                           |                          |&#xA;vz1.1   P1.1       vz1.2    P1.2         vz1.3&#xA;|                           |                          |&#xA;o-------vx2.1-------o-------vx2.2-------o&#xA;|                           |                          |&#xA;vz2.1   P2.1       vz2.2    P2.2         vz2.3&#xA;|                           |                          |&#xA;o-------vx3.1-------o-------vx3.2-------o&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I'm solving it with boundary condition &lt;span class=&quot;math-container&quot;&gt;$v=0$&lt;/span&gt; at the boundary. My equations are  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla\cdot v =0$$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$- \nabla P + \eta \nabla^2 v = \text{RHS}$$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My matrix looks like (point . is a zero)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;1.0........  1.0...................&#xA;.  1.0..  1.0........................&#xA;..  1.0..........................&#xA;...  1.0........  1.0................&#xA;....  1.0........................&#xA;.....  1.0.......................&#xA;......  1.0........  1.0.............&#xA;.......  1.0.....................&#xA;........  1.0....................&#xA;.........  1.0...................&#xA;..........  1.0..  1.0...............&#xA;...........  1.0.................&#xA;...  4.5  4.5.. -4.5. -4.5.. -9.0 -4.5  1.5  4.5  4.5 -1.5...  4.5.......&#xA;...  4.5  4.5.. -4.5.... -9.0-13.5.  4.5  4.5 -1.5....  4.5  1.5.....&#xA;.... -1.5.... -1.5..  1.5  1.5...............&#xA;...............  1.0.............&#xA;......  4.5  4.5.. -4.5.... -9.0-13.5.  4.5. -1.5....  4.5  1.5  4.5.&#xA;....... -1.5.... -1.5..  1.5  1.5............&#xA;..................  1.0..........&#xA;...................  1.0..  1.0......&#xA;....................  1.0........&#xA;............  4.5  4.5.. -4.5. -4.5.. -9.0 -4.5  1.5  4.5  4.5 -1.5.  4.5&#xA;......................  1.0......&#xA;............. -1.5.... -1.5..  1.5  1.5......&#xA;........................  1.0....&#xA;.........................  1.0...&#xA;................ -1.5.... -1.5..  1.5  1.5...&#xA;................  1.0..........  1.0.&#xA;.........  1.0..................  1.0&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is for a net  1.1 - 3.2 like I have written it down above but then net is mirrored. The matrix is filled as (columns) vx, vz, P of 1.1, vx,vz,P of 1.2, vx,vz,P of 1.3 (points for vx and P are arbitrary), vx,vz,P for 2.1 and so one.  The matrix seems me to be regular but SVD decomposition says that not, these are the sorted singular values of the matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;   27.5420715513798&#xA;   17.4983644162386&#xA;   13.0585213530020&#xA;   7.91769353816330&#xA;   2.90355494070429&#xA;   2.47099309663979&#xA;   2.01114749046610&#xA;   1.78190740642182&#xA;   1.54014039952304&#xA;   1.48327145289315&#xA;   1.38900389385430&#xA;   1.26550776173684&#xA;   1.12306273303955&#xA;   1.01704742320595&#xA;   1.00000000000000&#xA;   1.00000000000000&#xA;   1.00000000000000&#xA;   1.00000000000000&#xA;   1.00000000000000&#xA;  0.931474730095686&#xA;  0.924742536722870&#xA;  0.799653920064005&#xA;  0.602280940813348&#xA;  0.569579284384847&#xA;  0.488475169437247&#xA;  0.202063898865890&#xA;  0.177767300574206&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and simple Gaussian elimination (&quot;Numerical recipes&quot;) says during dividing the pivot row by a pivot element that pivot element a(24,24)=0 so the matrix is singular. The point 24 is exactly the eq. of continuity for pressure at boundary P 2.2&#xA;Can you please give me some hint on how to find the problem in the formulation or how can I find the mistake in the program?&lt;/p&gt;&#xA;" OwnerUserId="4007" LastEditorUserId="20688" LastEditDate="2019-05-25T11:19:38.777" LastActivityDate="2019-05-25T11:19:38.777" Title="Singular matrix for 2D Stokes flow in finite differences" Tags="|linear-algebra|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6724" PostTypeId="2" ParentId="6723" CreationDate="2013-03-30T23:08:43.563" Score="4" Body="&lt;p&gt;Since you have Dirichlet boundary conditions everywhere, the pressure is only determined up to a constant. If you use a direct solver, the easiest fix is to pin pressure at one point. With iterative solvers, it's usually preferable to work with the original system, but project out the constant mode.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-03-30T23:08:43.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6725" PostTypeId="2" ParentId="6708" CreationDate="2013-03-31T06:57:23.060" Score="6" Body="&lt;p&gt;It &lt;em&gt;is&lt;/em&gt; working, but slowly. Your timestep is 125 times smaller than your grid separation. Given that all the other constants are unity, your cell-crossing time is 125 timesteps. It takes a long time to see the changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you change &lt;code&gt;dt&lt;/code&gt; to &lt;code&gt;0.001&lt;/code&gt;, for instance, there should just be a noticeable difference between figure 1 and figure 19. Of course, only alternating frames can be compared, a bizarre artifact of trying to de-complexify the Schrödinger equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note though that you are going forward in &quot;timesteps&quot; of $-\mathrm{i} \Delta t$ (note the sign!). Compare the tridiagonal matrices you have with the matrix for spatial differencing,&#xA;$$ +\frac{\partial^2}{\partial x^2} \to A \equiv&#xA;\begin{pmatrix}&#xA;  -2 &amp;amp; 1       \\&#xA;  1  &amp;amp; -2 &amp;amp; 1  \\&#xA;     &amp;amp; 1  &amp;amp; -2 \\&#xA;     &amp;amp;    &amp;amp;    &amp;amp; \ddots&#xA;\end{pmatrix}. $$&#xA;Applying this and the typical Crank-Nicolson method to&#xA;$$ \mathrm{i} \frac{\partial}{\partial t} \psi = \left(-\frac{\partial^2}{\partial x^2} + V\right) \psi $$&#xA;results in&#xA;$$ \left(I - \frac{\mathrm{i}\Delta t}{2(\Delta x)^2} A\right) \psi_{n+1} = \left(I + \frac{\mathrm{i}\Delta t}{2(\Delta x)^2} A\right) \psi_n. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either change the sign of &lt;code&gt;dt&lt;/code&gt; in all the entries in matrices &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;, or for a quick fix put a negative sign in your definition of &lt;code&gt;dt&lt;/code&gt; (while making it larger - no need to take such small timesteps). Then you actually do see numerically stable diffusion. As written, your code is running the diffusion equation backward, so blowing up is not unexpected.&lt;/p&gt;&#xA;" OwnerDisplayName="user3224" LastActivityDate="2013-03-31T06:57:23.060" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6726" PostTypeId="2" ParentId="6721" CreationDate="2013-03-31T15:24:04.080" Score="2" Body="&lt;p&gt;You can use array slicing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you create a NumPy array (ndarray, to be precise), it might look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = numpy.arange(27).reshape(3,3,3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And if you do &lt;code&gt;print( a[:,:,:] )&lt;/code&gt; it will print this out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([[[ 0,  1,  2],&#xA;        [ 3,  4,  5],&#xA;        [ 6,  7,  8]],&#xA;&#xA;       [[ 9, 10, 11],&#xA;        [12, 13, 14],&#xA;        [15, 16, 17]],&#xA;&#xA;       [[18, 19, 20],&#xA;        [21, 22, 23],&#xA;        [24, 25, 26]]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And so you can do things like &lt;code&gt;print( a[:,:,0] )&lt;/code&gt; you get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([[ 0,  3,  6],&#xA;       [ 9, 12, 15],&#xA;       [18, 21, 24]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Try experimenting with &lt;code&gt;a[0,:,:]&lt;/code&gt; or &lt;code&gt;a[:,0,:]&lt;/code&gt; to see if that's what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that these are views on to the array, so if you want to store it somehow you should copy it into a new array.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2013-03-31T15:24:04.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6727" PostTypeId="2" ParentId="2051" CreationDate="2013-03-31T17:59:16.643" Score="2" Body="&lt;p&gt;There are basically two approaches to attack inverse problem for PDE systems. Either posed it as an optimization problem or Kalman-filtering (probabilistic approach) techniques. The latter method is useful if your PDE systems contain uncertainties. That may include modelling error and etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your PDE systems are large enough, the optimization approach becomes computationally prohibitive (may equally be expensive as Kalman-filtering technique). Therefore, method like adjoint for computing gradient efficiently is very beneficial. Why is this efficient ? Because adjoint method needs two simulations (one forward simulation and one backward simulation) regardless of the number of decision variable. Contrary to computing gradient using finite-difference methods which require N number of simulations, where N is the number of decision variable. Non-gradient (derivative-free) based optimizations for PDE systems, although they are easy to implement, but the optimization can be costly in term of the runtime.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To model the PDE systems, yes, Finite-element can be used. But it depends on the nature of your PDE systems which perfectly fit to the physics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And don't forget inverse problem for PDE systems is an ill-posed problem. Because you can't get as many as measurement data for the parameter and/or state variables to be estimated. Hence, the solutions are not unique. &lt;/p&gt;&#xA;" OwnerUserId="3940" LastActivityDate="2013-03-31T17:59:16.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6728" PostTypeId="2" ParentId="5694" CreationDate="2013-04-01T01:33:16.583" Score="6" Body="&lt;p&gt;I'm not sure what I have to say here qualifies as an answer, but what I want to say will not fit into a comment, either, so I have decided to take a chance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An autodidact such as yourself will know himself better than I can infer from reading one question. But in my view, casting about for computational &lt;em&gt;problems&lt;/em&gt; to tackle seems bit like putting the cart before the horse. It also seems like a quick way to make the study of computational methods dry and tedious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think a better approach is to search for specific &lt;em&gt;applications&lt;/em&gt; that interest you. At first, don't dwell much on the computational requirements. But once you find an area that strikes your interest, dig a little deeper and see if the extant computational methods look interesting. If so, it might be worth reproducing an algorithm here or there, using &lt;em&gt;existing tools&lt;/em&gt; as your reference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even better---make a judgement about whether or not those existing tools have room for &lt;em&gt;improvement&lt;/em&gt;. Do you think there's a better way? Perhaps an approach drawn from another application that might map over well? Maybe you read something in a paper, but the author seemed better at writing a journal article than publishing useful code. Or perhaps there is a commercial package whose functionality could be duplicated in open source, and you're just the guy to set the algorithm free...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nothing will motivate learning new computational skills---mastering Python or Sage, learning the ins an outs of an open-source library, etc., mastering a new mathematical technique---like the &lt;em&gt;need&lt;/em&gt; to use it to solve a particular problem that's eating at you. &lt;em&gt;That's&lt;/em&gt; where I'd put my stake in the ground and start working on code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure, you may not learn to code as elegantly as someone who sits down and plods through a formal text and works out all of the exercises. It's possible that some of the code you come up with would make a seasoned expert scoff. But there's time to learn the formalities and conventions as you go. You'll be able to tell when it's time to take a break from the creative exploration and sit down with a reference manual for a few hours.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-04-01T05:32:33.487" LastActivityDate="2013-04-01T05:32:33.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6729" PostTypeId="2" ParentId="2051" CreationDate="2013-04-01T08:22:00.283" Score="1" Body="&lt;p&gt;For a Bayesian perspective, Stuart's paper, &lt;a href=&quot;http://journals.cambridge.org/action/displayAbstract?fromPage=online&amp;amp;aid=7701764&quot; rel=&quot;nofollow&quot;&gt;&quot;Inverse problems: A Bayesian perspective&quot;&lt;/a&gt; is really good. It's pretty advanced though. Perhaps one could start reading that paper, then backtrack and learn the necessary background whenever an unfamiliar concept is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an optimization approach, I found the explanation in &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications/2002-phd.pdf&quot; rel=&quot;nofollow&quot;&gt;Bangerth's PhD thesis&lt;/a&gt; was quite good and concise (I actually prefer it to the other paper he posted in this thread). Again, this is the sort of thing where you'd have to backtrack constantly to fill in your knowledge if you're just starting learning.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2013-04-01T08:22:00.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6730" PostTypeId="1" CreationDate="2013-04-01T13:09:09.630" Score="5" ViewCount="213" Body="&lt;p&gt;Currently, I fit a Finite Element solution of a PDE on a spectral basis. The matrices ($R^{25000\times 2000}$) of the corresponding system of linear equations are highly ill-conditioned ($\kappa \approx 10^{15}$). However, I am able to find a solution with Tikhonov-regularization, which has also a small residuum on other points in the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, playing around with a toy system, I came up with the idea to find the quadrature weights on a given subset of all points, such that instead of over-fitting with 25000 points, I have to find 2000 points out of my 100000 FE points such that they have positive quadrature weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What strategy do you suggest for selecting those 2000 points from the 100000?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I do the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pick 2000&lt;/li&gt;&#xA;&lt;li&gt;calculate the weights via qr&lt;/li&gt;&#xA;&lt;li&gt;select all points with negative weights and replace them with new points&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;On my 1D toy system with chebyshev polynoms, I need to draw about 100000 new points for a spectral basis of 25 modes till all my weights are positive. So this approach is not going to work well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions or literature links are highly welcome.&#xA;Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my toy system:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# encoding: utf8                                                           &#xA;import numpy as np                                                         &#xA;import matplotlib.pyplot as pl                                             &#xA;&#xA;def fun(x):                                                                &#xA;    x = np.asarray(x)                                                      &#xA;    y = np.zeros(x.shape)                                                  &#xA;    y[np.abs(x) &amp;gt; 0] = np.exp((-(1. / x[np.abs(x) &amp;gt; 0]) ** 2))             &#xA;    return y                                                               &#xA;&#xA;def gauss_extrema(n):                                                      &#xA;    i = np.arange(n)                                                       &#xA;    return np.cos(((2 * (i + 1)) - 1) / (2.*n) * np.pi)                    &#xA;&#xA;def main():                                                                &#xA;    Nx = 25                                                                &#xA;    nx = Nx                                                                &#xA;    x = 2 * np.random.rand(nx) - 1                                        &#xA;#    x = gauss_extrema(nx)                                                  &#xA;    m = np.arange(Nx)                                                      &#xA;    mm, xx = np.meshgrid(m, x)                                             &#xA;    L = np.cos(mm * np.arccos(xx))                                         &#xA;&#xA;    sp = np.zeros(nx)                                                      &#xA;    sp[0] = np.pi                                                          &#xA;    w = np.empty(nx)                                                       &#xA;    w.fill(-1)                                                             &#xA;    eps = 0.                                                               &#xA;&#xA;    i = 1                                                                  &#xA;    while np.any(w &amp;lt; eps):                                                 &#xA;        u, s, vh = np.linalg.svd(L.T, full_matrices=False)                 &#xA;        if i % 10000 == 0:                                                 &#xA;            print i, s[0] / s[-1], np.sum(w &amp;lt; eps)                         &#xA;        beta = np.dot(u.T, sp)                                             &#xA;        w = np.dot(vh.T, 1. / s * beta)                                    &#xA;        x[w &amp;lt; eps] = 2 * np.random.rand(np.sum(w &amp;lt; eps)) - 1               &#xA;        mm, xx = np.meshgrid(m, x)                                         &#xA;        L = np.cos(mm * np.arccos(xx))                                     &#xA;        i += 1                                                             &#xA;&#xA;    print w                                                                &#xA;    print x                                                                &#xA;    b = fun(x)                # setup right side b                         &#xA;    L = np.multiply(L, w)     # setup weights on all points on L           &#xA;    u = np.linalg.solve(L, b) # solve L u = b for coefficient vector u     &#xA;&#xA;    nx = 100                                                               &#xA;    x = np.linspace(-1, 1, nx)                                             &#xA;    mm, xx = np.meshgrid(m, x)                                             &#xA;&#xA;    L = np.cos(mm * np.arccos(xx))                                         &#xA;    L = np.multiply(L, w)                                                  &#xA;    b = fun(x)                                                             &#xA;&#xA;    fig = pl.figure()                                                      &#xA;    ax = fig.add_subplot(121)                                              &#xA;    ax.plot(x, b)                                                          &#xA;    ax.plot(x, np.dot(L, u))                                               &#xA;    ax = fig.add_subplot(122)                                              &#xA;    ax.plot(x, np.dot(L, u) - b)                                           &#xA;&#xA;    pl.show()                                                              &#xA;&#xA;if __name__ == '__main__':                                                 &#xA;    main()                                                                 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2013-04-01T18:46:56.877" LastActivityDate="2013-04-01T18:46:56.877" Title="Finding quadrature weights for a given set of points? How to select points such that all weights are positive?" Tags="|quadrature|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6731" PostTypeId="1" CreationDate="2013-04-01T14:27:45.153" Score="4" ViewCount="3571" Body="&lt;p&gt;I'm looking for alternatives to Matlab/Simulink and Dymola for simulating a non-linear dynamic system. I know it's possible to implement the time-domain behavior without a lot of code and a good simulator template which would do fixed-step simulation, but I don't want to re-invent the wheel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm basically looking for a template library that allows me to define my system, pass it to a simulator and analyze the results using other C++ libraries. The results would feed a system identifcation code that adjusts the simulated system to data gathered in experiments. Later, I'd like to use the same set of libraries to design a controller for my real system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want to use Matlab/Simulink or Dymola because I don't have access to them at home and I don't want to buy them. Matlab, Simulink, system identification toolbox and all other toolboxes I might need for this project are way too expensive for me as a single person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A google search revealed some candidates, but they don't seem to be very active. The most recent update I found was in 2011. Does anyone here actually use such a library and can recommend one?&lt;/p&gt;&#xA;" OwnerUserId="3680" LastActivityDate="2013-04-02T00:03:31.660" Title="C++ alternatives for simulating dynamic systems" Tags="|simulation|nonlinear-equations|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6732" PostTypeId="2" ParentId="6731" CreationDate="2013-04-01T16:37:52.357" Score="2" Body="&lt;p&gt;The latest boost library has numerical ode solvers -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.boost.org/doc/libs/1_53_0/libs/numeric/odeint/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://www.boost.org/doc/libs/1_53_0/libs/numeric/odeint/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4016" LastActivityDate="2013-04-01T16:37:52.357" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6734" PostTypeId="1" AcceptedAnswerId="7132" CreationDate="2013-04-01T17:06:31.937" Score="4" ViewCount="106" Body="&lt;p&gt;I'm looking for literature on the progression (year on year, or more fine-grained if possible) of Molecular Dynamics simulation sizes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By simulation size I mean number of atoms, time step, total simulation time and the architecture it was executed on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bonus points if anyone knows of a similar database/list for consumer end hardware especially GPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks very much&lt;/p&gt;&#xA;" OwnerUserId="1017" LastActivityDate="2013-05-09T14:06:45.000" Title="Progression of molecular dynamics simulation sizes" Tags="|reference-request|molecular-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6735" PostTypeId="2" ParentId="6731" CreationDate="2013-04-02T00:03:31.660" Score="3" Body="&lt;p&gt;If you're looking for something with a feature set roughly equivalent to that of MATLAB, but templated C++, Boost.odeint is probably the closest you'll get, replacing the NDF/BDF methods in &lt;code&gt;ode15s&lt;/code&gt; with 4th-order Rosenbrock methods and no DAE solution capability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://trilinos.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;Trilinos&lt;/a&gt; is the big templated C++ numerical library out there, written by Sandia National Laboratories. With it, you can probably do almost anything you can think of, including automatic differentiation of your dynamical system to generate Jacobian matrix information, if you so choose. In the design space of C++ templated libraries you could use for nonlinear dynamical systems, it is probably the other extreme in terms of features and learning curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There aren't many established, templated C++ libraries for dynamical systems out there. Most are written in C or Fortran (e.g., &lt;a href=&quot;https://computation.llnl.gov/casc/sundials/main.html&quot; rel=&quot;nofollow&quot;&gt;SUNDIALS&lt;/a&gt;, &lt;a href=&quot;http://www.unige.ch/~hairer/software.html&quot; rel=&quot;nofollow&quot;&gt;Ernst Hairer's collection of codes&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-04-02T00:03:31.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6736" PostTypeId="1" AcceptedAnswerId="6742" CreationDate="2013-04-02T00:10:18.060" Score="7" ViewCount="285" Body="&lt;p&gt;The primal problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{w,b}\frac{1}{2}w^Tw$$&#xA;$$s.t. f_i(w)=1-y_i(w\cdot x_i+b)\leq0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $y_i=\pm1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of using Gradient Descent directly, the dual is usually solved:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{\lambda}\min_{w,b}\frac{1}{2}w^Tw+\sum_{i=1}^M{\lambda_i(1-y_i(w\cdot x_i+b))}$$&#xA;$$s.t. \lambda_i\geq0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why not solve the primal directly?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I was thinking is that it's hard to use gradient descent when $b$ which is part of the constraints is unknown (it can actually be calculated as a function of $w$). But maybe I'm missing something.&lt;/p&gt;&#xA;" OwnerUserId="4019" LastEditorUserId="4019" LastEditDate="2013-04-02T10:06:14.043" LastActivityDate="2013-08-29T18:03:33.197" Title="Why is the Dual problem preferred for Maximal Margin Classification?" Tags="|convex-optimization|support-vector-machines|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6737" PostTypeId="1" CreationDate="2013-04-02T00:52:17.990" Score="2" ViewCount="1592" Body="&lt;p&gt;I'm implementing a very simple Susceptible-Infected-Recovered model with a steady population for an idle side project - normally a pretty trivial task. But I'm running into solver errors using either PysCeS or SciPy, both of which use lsoda as their underlying solver. This only happens for particular values of a parameter, and I'm stumped as to why. The code I'm using is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from pylab import *&#xA;import scipy.integrate as spi&#xA;&#xA;#Parameter Values&#xA;S0 = 99.&#xA;I0 = 1.&#xA;R0 = 0.&#xA;N0 = S0 + I0 + R0&#xA;PopIn= (S0, I0, R0, N0)&#xA;beta= 0.50     &#xA;gamma=1/10.  &#xA;mu = 1/25550.&#xA;t_end = 15000.&#xA;t_start = 1.&#xA;t_step = 1.&#xA;t_interval = np.arange(t_start, t_end, t_step)&#xA;&#xA;#Solving the differential equation. Solves over t for initial conditions PopIn&#xA;def eq_system(PopIn,t):&#xA;    '''Defining SIR System of Equations'''&#xA;    #Creating an array of equations&#xA;    Eqs= np.zeros((3))&#xA;    Eqs[0]= -beta * (PopIn[0]*PopIn[1]/PopIn[3]) - mu*PopIn[0] + mu*PopIn[3]&#xA;    Eqs[1]= (beta * (PopIn[0]*PopIn[1]/PopIn[3]) - gamma*PopIn[1] - mu*PopIn[1])&#xA;    Eqs[2]= gamma*PopIn[1] - mu*PopIn[2]&#xA;    return Eqs&#xA;&#xA;SIR = spi.odeint(eq_system, PopIn, t_interval)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This produces an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; lsoda--  at current t (=r1), mxstep (=i1) steps   &#xA;       taken on this call before reaching tout     &#xA;      In above message,  I1 =       500&#xA;      In above message,  R1 =  0.7732042715460E+04&#xA;Excess work done on this call (perhaps wrong Dfun type).&#xA;Run with full_output = 1 to get quantitative information.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, changing &quot;mu&quot; (which is currently 1/70 years, a pretty common value for life expectancy) to either 26550 or 22550 produces no such error, though some values between 25550 and 22550 seem to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a little puzzled as to why that combination in particular is producing errors. Anyone have some insight?&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2013-04-09T17:33:14.950" Title="Solver error in SciPy/LSODA with a very specific parameter set" Tags="|python|ode|scipy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6738" PostTypeId="1" CreationDate="2013-04-02T01:28:00.217" Score="6" ViewCount="243" Body="&lt;p&gt;When solving $Ax=b$,  prior knowledge about $A$'s structure can help in designing an efficient solver which exploits this information  (e.g conjugate gradient method is to be used when $A$ is symmetric and is preferable to GMRES when we know $A$ is so )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the set of non-linear equations I am working with,  I am using the Newton-Raphson method for its solution. The Jacobian ($A$) of NR for every iteration,  turns out to be sparse and also symmetric in its sparsity pattern for my problem (but &lt;strong&gt;not&lt;/strong&gt; necessarily  in its  value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So $a_{ij} \neq 0 \Leftrightarrow a_{ji} \neq 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a class of methods (iterative or direct) which can solve this class of problems efficiently? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes, then web-links to any implementations would also be very useful.  &lt;/p&gt;&#xA;" OwnerUserId="3352" LastEditorUserId="276" LastEditDate="2013-04-02T21:21:43.810" LastActivityDate="2017-04-20T23:16:25.497" Title="Are there any specialized methods available for solving structurally symmetric sparse linear systems?" Tags="|matrix|linear-solver|sparse-matrix|krylov-method|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6740" PostTypeId="2" ParentId="6737" CreationDate="2013-04-02T12:16:24.550" Score="1" Body="&lt;p&gt;It seems that your system blows up slightly above t=7732. You can check this by setting t_end=7732 and plotting the result using -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;plot( SIR[:,0] ); show()&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your differential equation might have a problem.&lt;/p&gt;&#xA;" OwnerUserId="4016" LastActivityDate="2013-04-02T12:16:24.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6741" PostTypeId="2" ParentId="6738" CreationDate="2013-04-02T13:15:41.523" Score="10" Body="&lt;p&gt;This is called &quot;structurally symmetric&quot;. It simplifies graph traversal, such as occurs when setting up aggregates in algebraic multigrid, but doesn't offer much structure to improve convergence rates. Note that all common PDE discretizations have this property so this is still a huge class of matrices including many instances for which no truly good iterative solver is known.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-04-02T13:15:41.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6742" PostTypeId="2" ParentId="6736" CreationDate="2013-04-02T14:34:07.220" Score="7" Body="&lt;p&gt;You've identiifed the key problem. Certainly the primal &lt;em&gt;can&lt;/em&gt; be solved directly by, say, a quadratic programming solver. But typical QP solvers often don't scale well to large problem sizes. A projected gradient method can often scale to significantly larger problems---but only if the derivatives and projections are simple to compute. As I will show, the dual problem can be solved with inexpensive projected gradient iterations, while the primal cannot be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, let's simplify notation a bit: collect the vectors $y_ix_i$ into the &lt;em&gt;rows&lt;/em&gt; of a matrix $Z$, and the values $y_i$ into the elements of a vector $y$. Then we can write the problem as&#xA;$$\begin{array}{ll} \text{minimize}_{w,b} &amp;amp; f(w,b) \triangleq \tfrac{1}{2}w^T w \\ \text{subject to} &amp;amp; Z w + y b \succeq \vec{1} \end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A projected gradient algorithm will alternate between gradient steps and projections. The gradient is simply $\nabla f(w,b)=(w,0)$, so this is a pretty trivial operation. Let's denote by $(w_+,b_+)$ the result of a single gradient step. We must then project $(w_+,b_+)$ back onto the feasible set: find the nearest point $(w',b')$ that satisfies $Xw+y \succeq \vec{1}$. This means we must solve&#xA;$$\begin{array}{ll}&#xA;\text{minimize}_{w,b} &amp;amp; \|w-w_+\|_2^2+(b_+-b)^2 \\ &#xA;\text{subject to} &amp;amp; Zw+by\succeq \vec{1} \\&#xA;\end{array}$$&#xA;&lt;em&gt;This is virtually the same problem as the original.&lt;/em&gt; In other words, each &lt;em&gt;step&lt;/em&gt; of projected gradients for the primal problem is as expensive as the original problem itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now examine the dual problem. To get a handle on this we need to simplify the dual function&#xA;$$g(\lambda) = \min_{w,b} \tfrac{1}{2} w^T w - \lambda^T ( Z w + b y - \vec{1} )$$&#xA;With a little calculus you can determine that the optimal value of $w=Z^T\lambda$. As for $b$, we observe that if $y^T\lambda\neq 0$, the right-hand expression can be driven to $-\infty$ by growing $b\cdot(y^T\lambda)\rightarrow +\infty$. So&#xA;$$g(\lambda) = \begin{cases} - \vec{1}^T \lambda - \tfrac{1}{2} \lambda^T ZZ^T \lambda &amp;amp; y^T \lambda = 0 \\ -\infty &amp;amp; y^T \lambda \neq 0 \end{cases}$$&#xA;So the effect of $b$ is to introduce an implicit constraint $y^T\lambda =0$, and the dual problem is equivalent to $$\begin{array}{ll} \text{maximize} &amp;amp; \bar{g}(\lambda) \triangleq \vec{1}^T \lambda - \tfrac{1}{2} \lambda^T ZZ^T \lambda \\ \text{subject to} &amp;amp; y^T \lambda = 0 \\ &amp;amp; \lambda \succeq 0 \end{array}$$&#xA;We would not have been able to apply projected gradients to the original dual function, because it wasn't differentiable; but removing the implicit constraint $y^T\lambda=0$ changes that. Now the gradient is $\nabla\bar{g}(\lambda) = \vec{1} - ZZ^T\lambda$---a bit more complex than the primal gradient, but entirely manageable. (Note that we're maximizing now, so gradient steps are taken in the positive direction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's left is to perform the projection. Given the point $\lambda_+$ that comes out of the gradient step, we need to solve&#xA;$$\begin{array}{ll} \text{minimize} &amp;amp; \tfrac{1}{2} \|\lambda-\lambda_+\|_2^2 \\ \text{subject to} &amp;amp; y^T \lambda = 0 \\ &amp;amp; \lambda \geq 0 \end{array}$$This looks much easier than the primal projection, doesn't it? I haven't looked at the literature to see how people solve it now, but this is what I came up with: the value of $\lambda$ is&#xA;$$\lambda_i = \max\{\lambda_{+,i}+sy,0\}, ~i=1,2,\dots, n$$&#xA;where the parameter $s$ is chosen so that $y^T\lambda = 0$. I suspect there is a simple iterative method: start with $s=0$, compute $\lambda$ and $y^T\lambda$, adjust $s$, and repeat. A few $O(n)$ computations get the result. Don't trust me on this, I suspect the extant SVM literature has something solid here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important final step is to recover $(w,b)$ from the optimal solution $\lambda^*$. Our derivations for $g(\lambda)$ showed that $w=Z^T\lambda^*$. Once this has been recovered, $b$ is any value in the following interval:&#xA;$$b \in \left[ 1 - \max_{i:y_i=1} w\cdot x_i, \max_{i:y_i=-1} w \cdot x_i - 1 \right] $$&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-04-05T03:40:49.513" LastActivityDate="2013-04-05T03:40:49.513" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6743" PostTypeId="1" AcceptedAnswerId="6744" CreationDate="2013-04-02T17:55:16.350" Score="3" ViewCount="676" Body="&lt;p&gt;Do CPLEX or Gurobi handle MIQPs with indefinite coefficient matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I am dealing with has quadratic terms in which one variable is binary and the other variable is continuous. The coefficient matrix of the quadratic form is far from positive semi-definite. Its entries are data-dependent. Thus, I have no control over the positive semi-definiteness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can CPLEX or Gurobi handle such models at all? Note that it's a MIQP with binary variables and continuous variables, not a QP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there other software for such problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the state of the art, algorithmically?&lt;/p&gt;&#xA;" OwnerUserId="4026" LastActivityDate="2014-10-08T20:39:34.703" Title="Solver for a MIQP with an indefinite coefficient matrix" Tags="|quadratic-programming|mixed-integer-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6744" PostTypeId="2" ParentId="6743" CreationDate="2013-04-02T18:44:16.780" Score="6" Body="&lt;p&gt;A typical way to deal with this is to replace products $xy$ where $x$ is binary and $y$ continuous with a new variable $w$, and then add a constraint to ensure $w=0$ when $x=0$ and $w=y$ when $x=1$. This can be accomplished with $-M(1-x) \leq w-y \leq M(1-x)$, $-Mx \leq w \leq Mx$ where $M$ is a sufficiently large (but as small as possible) constant to ensure the feasible set does not change, i.e., an upper bound on the absolute value of any optimal $y$.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-04-02T18:44:16.780" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6745" PostTypeId="1" CreationDate="2013-04-02T19:37:12.593" Score="5" ViewCount="4307" Body="&lt;p&gt;I am trying to derive the formula of the 4th order Padé scheme that passes through the points $x_i$, $x_{i-1}$ and $x_{i+1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Big(\frac{\partial\phi}{\partial x} \Big)_i = -\frac{1}{4}\Big(\frac{\partial\phi}{\partial x} \Big)_{i+1}-\frac{1}{4}\Big(\frac{\partial\phi}{\partial x} \Big)_{i-1} + \frac{3}{4}\frac{\phi_{i+1}-\phi_{i-1}}{\Delta x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achieve this I am thinking of solving it using linear algebra, Gauss elimination, to find the coefficients $a_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A=\pmatrix{&#xA;1 &amp;amp; x_{i-1} &amp;amp; x_{i-1}^2 &amp;amp; x_{i-1}^3 &amp;amp;x_{i-1}^4 \\&#xA;1 &amp;amp; x_{i} &amp;amp; x_{i}^2 &amp;amp; x_{i}^3 &amp;amp;x_{i}^4 \\&#xA;1 &amp;amp; x_{i+1} &amp;amp; x_{i+1}^2 &amp;amp; x_{i+1}^3 &amp;amp;x_{i+1}^4 \\&#xA;}&#xA;~~~~~~&#xA;x=\pmatrix{&#xA;a_0\\a_1\\a_2\\a_3\\a_4&#xA;}&#xA;~~~~~~&#xA;y=\pmatrix{&#xA;\phi_{i-1} \\ \phi_{i} \\ \phi_{i+1}&#xA;}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;once I have the coefficients I can use them to find the derivatives in $x_i$, $x_{i-1}$ and $x_{i+1}$ and try to get the formula.&#xA;I think this is a lengthy procedure, is this a good approach to derive the formula?&#xA;Is there a better way (faster) to do it?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2015-09-25T17:39:47.837" Title="4th order Padé scheme formula derivation" Tags="|linear-algebra|pde|finite-difference|fluid-dynamics|ode|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6746" PostTypeId="2" ParentId="2595" CreationDate="2013-04-03T04:01:58.557" Score="1" Body="&lt;p&gt;There is a software called PORTA that can enumerate all the points that are feasible in a set of linear inequalities and equalities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.iwr.uni-heidelberg.de/groups/comopt/software/PORTA/&quot; rel=&quot;nofollow&quot;&gt;http://www.iwr.uni-heidelberg.de/groups/comopt/software/PORTA/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;enjoy it!&lt;/p&gt;&#xA;" OwnerUserId="4028" LastActivityDate="2013-04-03T04:01:58.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6747" PostTypeId="1" AcceptedAnswerId="6748" CreationDate="2013-04-03T09:43:36.953" Score="1" ViewCount="1154" Body="&lt;p&gt;I need to use GMRES solver in MATLAB, and I need to play around with the codes parameters and I had a very simple question about its usage. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation of the solver &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/gmres.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; mentions a parameter &lt;code&gt;tol&lt;/code&gt; which they &#xA;do not seem to have defined except that the defuault value is $10^{-6}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is this number? There don't seem to be any definitions of this anywhere &#xA;on the web. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it the upper-bound on $\frac{||b-Ax_k||}{||b||}$ where we are solving&#xA;$Ax=b$ and $x_k$ is the current GMRES iterate? &lt;/p&gt;&#xA;" OwnerUserId="3352" LastActivityDate="2013-04-03T09:59:51.767" Title="GMRES Matlab 'tol' parameter" Tags="|matlab|gmres|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6748" PostTypeId="2" ParentId="6747" CreationDate="2013-04-03T09:59:51.767" Score="4" Body="&lt;p&gt;It is indeed a tolerance on the relative residual $$ ||b-Ax_k|| / ||b||$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the webpage that you mention, you can find this information (very implicitly)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[x,flag,relres] = gmres(A,b,...) also returns the relative residual&#xA;  norm(b-A*x)/norm(b). If flag is 0, relres &amp;lt;= tol. The third output,&#xA;  relres, is the relative residual of the preconditioned system.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-04-03T09:59:51.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6749" PostTypeId="1" CreationDate="2013-04-03T14:38:40.963" Score="0" ViewCount="211" Body="&lt;p&gt;I am working with an octree mesh where variables are stored in a collocated fashion at octant centers. I want to construct a lookup table for interpolation weights that may occur using only a cell and it's immediately adjacent neighbor leaf cells. There are a specific number of neighbor configurations since each neighbor is only allowed to be 1 level coarser or finer than the cell for which an interpolation is required, and I want to specify a unique table id for each configuration. Figure 4 in the paper below shows pretty clearly what I mean by surrounding cell configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.uh.edu/~molshan/ftp/pub/OctreeMAC.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.uh.edu/~molshan/ftp/pub/OctreeMAC.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm/accepted way to enumerate these cases? &lt;/p&gt;&#xA;" OwnerUserId="1453" LastEditorUserId="1453" LastEditDate="2013-11-11T20:45:25.273" LastActivityDate="2013-11-11T20:45:25.273" Title="Neighbor pattern look-up table enumeration on an octree mesh" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6750" PostTypeId="1" CreationDate="2013-04-03T17:40:22.910" Score="4" ViewCount="247" Body="&lt;p&gt;Consider 2 mathematical problems:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_1(x) = a - x \\&#xA;f_2(x) = e^x -1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The condition number for a function is defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;k(f) = \left| x \cdot \frac{f'}{f} \right|&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets analyze conditioning first:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;k(f_1) = \frac{x}{x - a},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which means that $f_1$ is &lt;strong&gt;ill-conditioned&lt;/strong&gt; near $x = a$;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;k(f_2) = \frac{x \cdot e^x}{e^x - 1},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is undefined near $x = 0$, so lets use L'Hospital:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;k(f_2) = \frac{e^x + x \cdot e^x}{e^x},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which means that $f_2$ is &lt;strong&gt;well-conditioned&lt;/strong&gt; everywhere (including $x = 0$ proximity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now lets analyze stability of these 2 algorithms (if we were to implement them on the computer directly):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{(a - x) \cdot (1 + \epsilon_1) - (a - x)}{a - x} = \epsilon_1,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\epsilon_1 \leq \epsilon_m$, and which means that no (numerical) amplification of errors occurs and the algorithm is &lt;strong&gt;stable&lt;/strong&gt;;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{(e^x \cdot (1 + \epsilon_1) - 1) \cdot (1 + \epsilon_2) - (e^x - 1)}{e^x - 1} \approx \{\epsilon_1 \cdot \epsilon_2 \to 0\} \approx \frac{e^x \cdot \epsilon_1 + (e^x - 1) \cdot \epsilon_2}{e^x - 1} = \epsilon_2 + \epsilon_1 \cdot \frac{e^x}{e^x - 1},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\epsilon_1, \epsilon_2 \leq \epsilon_m$, and which means that the algorithm is &lt;strong&gt;unstable&lt;/strong&gt; near $x = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although everything is allright from the mathematical point of view, i.e. if we obey the formulas and raw theory when obtaining such results, but I begin to doubt in the validity of these results when I try add some logic and reasoning behind it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, as far as I understand, when we study conditioning of the mathematical problem we think of it in &lt;strong&gt;exact arithmetics&lt;/strong&gt; (i.e. we do not think about computers, rounding, floating-point arithmetic, and etc.). Therefore, if I forget for a moment about the result obtained by analyzing $k(f_1)$ and just look on the simple mathematical problem $f_1(x) = a - x$, then I merely don't see how on earth it could be &lt;strong&gt;ill-conditioned&lt;/strong&gt; near $x = a$. What is the physical reasoning behind it? What kind of bad thing can happen in &lt;strong&gt;exact arithmetic&lt;/strong&gt; near $x = a$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My curriculum pointed out &lt;strong&gt;cancellation error&lt;/strong&gt; as an explanation. What kind of &lt;strong&gt;cancellation error&lt;/strong&gt;? From my point of view, there is no such thing as &lt;strong&gt;cancellation error&lt;/strong&gt; in &lt;strong&gt;exact arithmetic&lt;/strong&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my reason against it would be straightforward, since &lt;strong&gt;ill-condition&lt;/strong&gt; implies that &lt;strong&gt;the output changes drastically when the input changes slightly&lt;/strong&gt;, then $f_1$ is clearly linear (moreover with coefficient $1$) and any slight changes of $x$ (regardless of whether near $a$ or not) will always result in the &lt;strong&gt;quantitatively equal&lt;/strong&gt; change of $y = f_1(x)$ (i.e. $\Delta x \equiv \Delta y$). Therefore, I insist that $f_1$ can in no way be &lt;strong&gt;ill-conditioned&lt;/strong&gt; neither at $x = a$ nor anywhere else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any flaws in my reasoning? Please, clarify this for me as I'm actually stuck on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, why it turns out that $f_2$ is &lt;strong&gt;well-conditioned&lt;/strong&gt; while it looks the same as $f_1$? I mean if I follow the same logic as for $f_1$ (i.e. that it's &lt;strong&gt;ill-conditioned&lt;/strong&gt; at $x = a$ because of &lt;strong&gt;cancellation&lt;/strong&gt; as my curriculum states), then I could say the same here - &lt;strong&gt;ill-conditioned&lt;/strong&gt; at $x = 0$, just by looking on the definition of $f_2$! However, mathematics show us that it's not true, but rather that the direct implementation of $f_2$ evaluation on computer would result in &lt;strong&gt;unstable&lt;/strong&gt; algorithm. Due to what? I guess now it's &lt;strong&gt;cancellation error&lt;/strong&gt; because we are in the floating-point world now. But why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And still the question is why these two seemingly similar problems are actually so different? I'd really appreciate an exhaustive breakdown of these problems as I feel that I'm missing something very basic and it might prevent my understanding of more challenging phenomena.&lt;/p&gt;&#xA;" OwnerUserId="4030" LastEditorUserId="4030" LastEditDate="2013-04-03T18:11:58.070" LastActivityDate="2013-04-05T09:27:02.707" Title="Problem Condition and Algorithm Stability" Tags="|algorithms|stability|error-estimation|condition-number|" AnswerCount="2" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="6751" PostTypeId="2" ParentId="6749" CreationDate="2013-04-03T23:27:15.030" Score="2" Body="&lt;p&gt;Your question is too terse to really understand what it is you're looking for. That said, the paper that you should look at is this one by Burstedde et al.:&#xA;  &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/100791634&quot; rel=&quot;nofollow&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/100791634&lt;/a&gt;&#xA;It describes the p4est library and has all sorts of algorithms to traverse octrees.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-03T23:27:15.030" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6753" PostTypeId="2" ParentId="6750" CreationDate="2013-04-04T12:40:25.770" Score="0" Body="&lt;p&gt;As $f_1$, the bad condition number is completely obvious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us assume that $a = 0$. For large $x$, pertubations do not induce large relative error, so $x+\epsilon / x$ is small for $x$ large. If $x$ gets smaller and smaller, then $1 + \epsilon / x$ is fairly large. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not see any problem with that.&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="238" LastEditDate="2013-04-05T09:27:02.707" LastActivityDate="2013-04-05T09:27:02.707" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="6754" PostTypeId="2" ParentId="1909" CreationDate="2013-04-04T15:40:13.843" Score="0" Body="&lt;p&gt;Clearly the problem has an ODD order derivative.  More specifically for larger &lt;a href=&quot;http://en.wikipedia.org/wiki/P%C3%A9clet_number&quot; rel=&quot;nofollow&quot;&gt;Péclet numbers&lt;/a&gt;, the stiffness matrix might not maintain 'fine' shape, which creates zeros during assembly and hence gets singular or sometimes very small determinant that are noticeable by the oscillations in solution plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to these kind of problem is the use of penalty, among other methods. More specifically this is called &lt;a href=&quot;http://en.wikipedia.org/wiki/Petrov%E2%80%93Galerkin_method&quot; rel=&quot;nofollow&quot;&gt;Petrov-Galerkin method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for my bad English comprehension.&lt;/p&gt;&#xA;" OwnerUserId="4035" LastEditorUserId="651" LastEditDate="2013-04-05T16:42:06.180" LastActivityDate="2013-04-05T16:42:06.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6755" PostTypeId="2" ParentId="6750" CreationDate="2013-04-04T17:56:11.473" Score="3" Body="&lt;p&gt;(This just addresses the first part of your question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The subtlety here is in your definition of the condition number -- as defined here, it is a ratio of relative error in the output to relative error in the input (&lt;a href=&quot;http://www.math.utk.edu/~ccollins/M577/Handouts/cond_stab.pdf&quot; rel=&quot;nofollow&quot;&gt;ie these notes&lt;/a&gt;).  As you've discovered, near $x=a$, the relative error of the input is order $\epsilon$, while the relative error of the output is order $\epsilon / \epsilon = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is not the function, but your choice of error norms.  For this problem, the relative norm isn't a good choice near $f = 0$.  In basically any other choice of error norm, things would have been better conditioned.  For the case of a problem we know to be well scaled (i.e. all input and output are expected to approximately order 1), we can guess that the absolute error might have been a better choice.  Then we would define a new condition number, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k_{abs}$ = absolute error output / absolute error input&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k_{abs} = \frac{|f(x_0 + \epsilon) - f(x_0)|}{|(x_0 + \epsilon) - x_0|} \approx |f'(x_0)| = 1 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which suggests (correctly) that $f_1$ is well-conditioned everywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take home message: our condition number is dependent upon the choice of error norms, and care is required in the choice of definition in the general case.  You're right to question these sort of things, and subtleties are common and require more thought than just rote application of formulas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that for most real applications, in selecting a valid solution, the norm used is some combination of absolute and relative error for just this reason.&lt;/p&gt;&#xA;" OwnerUserId="508" LastActivityDate="2013-04-04T17:56:11.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6757" PostTypeId="1" CreationDate="2013-04-04T19:15:54.150" Score="7" ViewCount="8033" Body="&lt;p&gt;This program finds the first root of the function f, defined in the code. There are 5 roots of this function. (x=1,2,3,4,5) I wish to find all of the roots in this program and print them to the screen. In the main function, you can see that I have attempted to write a while loop to do this(commented out in the code below). But this doesn't work, it just keeps printing the same result to the screen infinitely.  Any help would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// This program finds the roots of a polynomial using the half interval search method&#xA;&#xA;#include &amp;lt;iostream&amp;gt; //preprocessor directive needed in order to use std::cout&#xA;                //and std::cin&#xA;#include &amp;lt;iomanip&amp;gt; //preprocessor directive needed in order to use&#xA;               //a manipulator which uses an argument&#xA;&#xA;using namespace std; //avoids having to uses std:: with cout and cin&#xA;&#xA;//declare the variables&#xA;const double a = 0.0, b = 6.0; //limits for f()&#xA;double px = 1e-6;              //define the precision&#xA;double x = a, dx = 1.0;        //start value and initial interval&#xA;double root;&#xA;&#xA;//prototypes for f()&#xA;double f (double x);&#xA;double halfIntervalSearch (double a, double b, double dx, double px); &#xA;&#xA;int main (int argc, char* argv[])&#xA;{&#xA;    //Inform the user what the program does&#xA;    cout &amp;lt;&amp;lt; &quot;This program finds the roots of\n&quot;&#xA;    &quot;f(x) = x^5 - 15x^4 + 85x^3 - 225x^2 + 274x -120 = 0\n&quot;&#xA;    &quot;which is bracketed by [&quot; &amp;lt;&amp;lt; a &amp;lt;&amp;lt; &quot;, &quot; &amp;lt;&amp;lt; b &amp;lt;&amp;lt;&#xA;    &quot;] using the half interval search method.&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;    cout &amp;lt;&amp;lt; &quot;Searching...\n&quot;; //Lets user know what is happening&#xA;&#xA;    root = halfIntervalSearch(a,b,dx,px); //pass the variable&#xA;&#xA;    //print out results&#xA;    //while (x &amp;lt;= b)&#xA;    {&#xA;          //x = root + 2*px; //reset value of x&#xA;          cout &amp;lt;&amp;lt; &quot;\nRoot at x = &quot; &amp;lt;&amp;lt; setw(10) &amp;lt;&amp;lt; setprecision(7) &amp;lt;&amp;lt; fixed &#xA;          &amp;lt;&amp;lt; root &amp;lt;&amp;lt; &quot; of value &quot; &amp;lt;&amp;lt; f(root) &amp;lt;&amp;lt; endl;      &#xA;    }    &#xA;    //allow user to see the results before ending the program&#xA;    cout &amp;lt;&amp;lt; &quot;\n\nPress Enter to end the program&quot;;&#xA;    cin.get();&#xA;}&#xA;&#xA;////////////////////////////////////////////////////////////////////////////////&#xA;//define f()&#xA;double f (double x)&#xA;{&#xA;       return (x*x*x*x*x)-(15*x*x*x*x)+(85*x*x*x)-(225*x*x)+(274*x)-120;  &#xA;}&#xA;///////////////////////////////////////////////////////////////////////////////&#xA;//define halfIntervalSearch()&#xA;double halfIntervalSearch (double a, double b, double dx, double px)&#xA;{&#xA;       while ((x &amp;lt;= b) &amp;amp;&amp;amp; (dx &amp;gt; px))&#xA;       {&#xA;             while ( f(x)* f(x+dx) &amp;gt; 0) //while there are no roots &#xA;             {&#xA;                   //keep track while program finds roots&#xA;                   cout &amp;lt;&amp;lt; &quot;x = &quot; &amp;lt;&amp;lt; setw(18) &amp;lt;&amp;lt; x &amp;lt;&amp;lt; &quot; | dx = &quot; &amp;lt;&amp;lt; setw(15) &#xA;                        &amp;lt;&amp;lt; right &amp;lt;&amp;lt; dx &amp;lt;&amp;lt; endl;&#xA;                   x += dx; //step forward&#xA;             }&#xA;&#xA;             dx = dx/2; //half the interval&#xA;       }&#xA;       return x;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3954" LastEditorUserId="127" LastEditDate="2013-05-11T17:58:48.327" LastActivityDate="2014-12-11T05:53:55.983" Title="How to find more than one root of a polynomial?" Tags="|c++|polynomials|nonlinear-equations|roots|" AnswerCount="4" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="6758" PostTypeId="1" AcceptedAnswerId="6760" CreationDate="2013-04-04T19:42:39.480" Score="10" ViewCount="16315" Body="&lt;p&gt;Like the title says, I want to be able to stop the code at a specific location and have it halt safely. I cannot find a command to do it like for example in FORTRAN there is the stop command.&lt;/p&gt;&#xA;" OwnerUserId="3561" LastActivityDate="2017-09-15T07:32:58.273" Title="Matlab : Is there a way to programatically safely halt code execution (like FORTRAN's stop)?" Tags="|matlab|" AnswerCount="4" CommentCount="0" ClosedDate="2017-09-18T15:35:28.207" ContentLicense="CC BY-SA 3.0" />
  <row Id="6759" PostTypeId="2" ParentId="6745" CreationDate="2013-04-04T19:53:03.887" Score="1" Body="&lt;p&gt;I would think a Taylor table would be the way to go. Moin's book &quot;Fundamentals of Engineering Numerical Analysis&quot; has good easy to follow examples.&lt;/p&gt;&#xA;" OwnerUserId="3561" LastActivityDate="2013-04-04T19:53:03.887" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6760" PostTypeId="2" ParentId="6758" CreationDate="2013-04-04T20:22:16.630" Score="15" Body="&lt;p&gt;The function &lt;code&gt;error('error message');&lt;/code&gt; will exit your program and print the error message to the console.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-04-04T20:22:16.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6762" PostTypeId="2" ParentId="6758" CreationDate="2013-04-04T22:25:13.603" Score="2" Body="&lt;p&gt;There is &lt;code&gt;break&lt;/code&gt; to exit a loop and &lt;code&gt;return&lt;/code&gt; to return to the invoking function or keyboard/promt.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2013-04-04T22:25:13.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6763" PostTypeId="1" AcceptedAnswerId="6830" CreationDate="2013-04-05T00:16:40.530" Score="3" ViewCount="1265" Body="&lt;p&gt;I noticed that adding constraints to an &lt;code&gt;IloModel&lt;/code&gt; one by one can be prohibitively slow. (I am referring to the construction of the model, not the optimization.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have many constraints with the same number of variables, can I add the constraint matrix in one function call?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other ways to speed up the construction of the model?&lt;/p&gt;&#xA;" OwnerUserId="4026" LastActivityDate="2021-03-11T11:05:25.517" Title="Cplex C++ Interface: How to add many constraints quickly?" Tags="|optimization|linear-programming|mixed-integer-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6764" PostTypeId="2" ParentId="6757" CreationDate="2013-04-05T06:13:43.010" Score="7" Body="&lt;p&gt;As pointed out by David Ketcheson, one method is to use the companion matrix and find its eigenvalues (this is what Matlab does for the &lt;code&gt;roots&lt;/code&gt; function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you want to code everything by yourself, you can try to use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Sturm%27s_theorem&quot;&gt;Sturm sequences&lt;/a&gt;, which you use as a first step to find an interval with only one zero. Then, you can apply one standard methods for finding the root within that interval (&lt;a href=&quot;http://en.wikipedia.org/wiki/Newton%27s_method&quot;&gt;Newton&lt;/a&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Bisection_method&quot;&gt;bisection&lt;/a&gt;,...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is easier to code, but probably less robust than the first approach.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-04-05T06:13:43.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6765" PostTypeId="2" ParentId="4908" CreationDate="2013-04-05T11:44:56.653" Score="3" Body="&lt;p&gt;From the Computer Science perspective I do not think that make sense to make an &lt;strong&gt;general statistical model&lt;/strong&gt; for memory access time (latency) and memory bandwidth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does make sense to create an &lt;strong&gt;statistical model for a algorithm&lt;/strong&gt;. That is because each algorithm has a specific memory access pattern, the memory access patterns are relevant to the cache hierarchy, e.g. a algorithm with high data locality will take advantage from low level caches benefiting of really fast memory access times while other algorithms will have to go all the way to the RAM (or even worst the swap memory) and have extremely slow access times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;general purpose values&lt;/strong&gt; are given from the architecture point of view, you can check your architecture and search the access time from a given core to a given memory location (let's say L3 cache). Be aware that the recent architectures are Non Uniform Memory Access &lt;a href=&quot;http://en.wikipedia.org/wiki/Non-Uniform_Memory_Access&quot; rel=&quot;nofollow&quot;&gt;NUMA&lt;/a&gt; which will make your job a bit harder.&lt;/p&gt;&#xA;" OwnerUserId="3580" LastEditorUserId="3580" LastEditDate="2013-04-05T18:33:09.173" LastActivityDate="2013-04-05T18:33:09.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6766" PostTypeId="1" CreationDate="2013-04-05T13:31:15.117" Score="0" ViewCount="98" Body="&lt;p&gt;Given an undirected graph, is it possible to find a criteria that leads to a unique partition of the nodes? The graph is not weighted.&lt;/p&gt;&#xA;" OwnerUserId="917" LastActivityDate="2013-04-14T04:04:34.193" Title="Unique Partition of a Graph" Tags="|optimization|graph-theory|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6767" PostTypeId="2" ParentId="6757" CreationDate="2013-04-05T14:50:33.423" Score="3" Body="&lt;p&gt;Let's lay out some specs for what the &lt;code&gt;halfIntervalSearch&lt;/code&gt; function is supposed to do, and go from there to changes necessary in your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd assume from the name that it is supposed to find (and &lt;code&gt;return&lt;/code&gt;) a root of a (hardcoded) function &lt;code&gt;f&lt;/code&gt; using the method of interval bisection.  The mathematics behind this is that a continuous function which &lt;strong&gt;changes sign&lt;/strong&gt; in an interval must have at least one root there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a typical implementation would be to call the bisection method with such an interval to have the interval refined by successive evaluations at midpoints, with the result that each evaluation results either in finding a (sufficiently approximate) root or in reducing the width of the interval by half (by retaining that half of the earlier interval over which the function is known to change signs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your code for &lt;code&gt;halfIntervalSearch&lt;/code&gt; does something like this, but I suspect it suffers from being overly indirect in its approach.  Of the four arguments passed to this function, the first of them is not used at all.  More to the point, instead of a single looping construct which preserves the change-of-sign property of an interval, your implementation using a nested pair of looping constructs whose inner loop is theoretically capable of &quot;stepping forward&quot; over any particular root (possibly all of them): &lt;code&gt;x += dx&lt;/code&gt;.  I don't think this is a sound approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the search for roots of polynomials is difficult in complete generality, so we should not set the bar too high.  However, going back to the mathematical motivation (Intermediate Value Theorem), we would be better served to implement a routine that iteratively refines an interval with a change-of-signs until a (sufficiently approximate) root is found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The higher level issue is then how to call/make-use-of such a routine.  This is typically done by making a coarsely stepped search of the &quot;big interval&quot; of interest (which in your program is &lt;code&gt;[a,b]&lt;/code&gt;) to detect changes-of-sign.  Once a (modest size) subinterval where the function changes sign is found, the bisection method routine would be called on just that subinterval (guaranteeing us at least to get one root from that subinterval).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a difficult problem to locate all roots of a general polynomial.  If a root has even multiplicity, the polynomial will not change signs (except that it becomes zero exactly at that root).  More generally you may have a pair of roots that are so close together, the coarsely stepped search passes over both of them in one step, thereby missing these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still I recommend you focus on the development of your &lt;code&gt;halfIntervalSearch&lt;/code&gt; function in a way that conserves the interval having a change-of-signs, and once that is in place we can perhaps revisit the somewhat vexing topics of roots with multiplicity &gt; 1, etc.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2013-04-05T15:13:11.387" LastActivityDate="2013-04-05T15:13:11.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6768" PostTypeId="1" CreationDate="2013-04-05T20:45:03.887" Score="1" ViewCount="150" Body="&lt;p&gt;I recently read something that talks about DIST distribution strings.  It appears to be a way to take a long string of previously generated numbers and somehow compress them into a string that can then be used by others.  I believe it's lossy.  I'm having a great deal of trouble finding out anything other than some generalities about it.  It may be proprietary, I don't know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know where I might be able to find something out about the algorithm used to create such a DIST string?  Also if anyone knows anything about this, is this something that's being used anywhere and how useful does it appear to be?&lt;/p&gt;&#xA;" OwnerUserId="4043" LastActivityDate="2013-04-06T00:47:51.507" Title="DIST strings - Monte Carlo Simulation" Tags="|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6769" PostTypeId="2" ParentId="6768" CreationDate="2013-04-06T00:47:51.507" Score="1" Body="&lt;p&gt;Online, there doesn't seem to be much information, although it's supposedly described by its inventor, Sam Savage, in its book &lt;em&gt;The Flaw of Averages&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently, it's a lossy XML standard, according to &lt;a href=&quot;http://c2.com/cgi/wiki?DistributionString&quot; rel=&quot;nofollow&quot;&gt;this website&lt;/a&gt;. It looks like a proprietary standard managed by &lt;a href=&quot;http://vectoreconomics.com/content/content/what-distribution-string-dist&quot; rel=&quot;nofollow&quot;&gt;Vector Economics&lt;/a&gt; and &lt;a href=&quot;http://www.probabilitymanagement.org/&quot; rel=&quot;nofollow&quot;&gt;Probability Management&lt;/a&gt;. If this standard is a key piece of intellectual property for Sam Savage, I doubt the standard is publicized anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears to be used in the financial services industry, but for scientific purposes, the lack of openness may be a big problem when it comes to checking numerical results.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-04-06T00:47:51.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6770" PostTypeId="1" CreationDate="2013-04-06T16:47:27.290" Score="0" ViewCount="779" Body="&lt;p&gt;I have a large system of equations&#xA;$$Ax=b$$ &#xA;and I know matrix $A$ and right-hand side vector $b$. I'm using MKL to solve this system. The matrices are complex. I have used the general solver &lt;code&gt;ZGETRS&lt;/code&gt;&#xA;(&lt;a href=&quot;https://software.intel.com/en-us/node/468712&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel MKL zgetrs&lt;/a&gt;) which needs the matrix input in LU decomposition. So I have used &lt;code&gt;ZGETRF&lt;/code&gt; &lt;a href=&quot;https://software.intel.com/en-us/node/468682&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Intel MKL zgetrf)&lt;/a&gt; that performs the LU decomposition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked the matrix $A$ through SVD as well - the SVD decomposition says no problem, matrix $A$ is regular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I send the $A$ to &lt;code&gt;ZGETRF&lt;/code&gt;, there comes a error code &quot;4&quot; (from manual if &lt;code&gt;info&lt;/code&gt; is positive, &lt;em&gt;&quot;The factorization has been completed, but $U$ is exactly singular. Division by 0 will occur if you use the factor $U$ for solving a system of linear equations.&quot;&lt;/em&gt;)&#xA;But then the program send the LU-factorized matrix $A$ to &lt;code&gt;ZGETRS&lt;/code&gt;, which should solve the $Ax=b$ system. And this one writes me no error (&lt;code&gt;info=0&lt;/code&gt;).&#xA;But the solution it gives me is trivial solution - all unknouwns should be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does it mean - where could be the error in the matrix? When SVD says the matrix is regular but LU decomposer says it's singular?&lt;/p&gt;&#xA;" OwnerUserId="3688" LastEditorUserId="20688" LastEditDate="2017-04-14T17:02:20.420" LastActivityDate="2017-08-12T19:01:13.820" Title="ZGETRF and ZGETRS from MKL - zgetrf fails and still zgetrs works?" Tags="|linear-solver|matrix|svd|intel-mkl|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6772" PostTypeId="1" CreationDate="2013-04-06T19:20:06.300" Score="4" ViewCount="198" Body="&lt;p&gt;How can I represent the calculation in this image mathematically?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/knH9f.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, with the discrete convolution (and Fourier Transform?),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(f * g)[n]\ \stackrel{\mathrm{def}}{=}\ \sum_{m=-\infty}^\infty f[m]\, g[n - m]$$&lt;/p&gt;&#xA;" OwnerUserId="3890" LastEditorUserId="3691" LastEditDate="2013-06-07T14:11:02.900" LastActivityDate="2013-08-06T15:00:45.290" Title="Mathematical way to represent a image kernel?" Tags="|image-processing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6775" PostTypeId="1" AcceptedAnswerId="6777" CreationDate="2013-04-06T23:17:27.407" Score="12" ViewCount="599" Body="&lt;p&gt;I read some references including &lt;a href=&quot;http://en.wikipedia.org/wiki/Compressed_sensing&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am kind of confused what optimization problem compressed sensing builds and tries to solve. Is it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \|x\|_1\\ \text{subject to} &amp;amp; Ax=b\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or/and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \|x\|_0\\ \text{subject to} &amp;amp; Ax=b\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or/and something else?&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="9667" LastEditDate="2018-08-28T21:11:55.940" LastActivityDate="2018-08-28T21:12:17.970" Title="Confusion about compressed sensing problem" Tags="|optimization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6776" PostTypeId="2" ParentId="6775" CreationDate="2013-04-07T00:33:22.670" Score="13" Body="&lt;p&gt;We would love to be able to solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| x \|_{0}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but this problem is an NP-Hard combinatorial optimization problem that is impractical to solve in practice when $A$, $x$, and $b$ are of sizes typical in compressive sensing.  It is possible to efficiently solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| x \|_{1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;both in theory (it can be done in polynomial time) and in computational practice for even fairly large problems that arise in compressive sensing.  We use $\| x \|_{1}$ as a &quot;surrogate&quot; for the $\| x \|_{0}$.  This has some intuitive justification (the one-norm minimization prefers solutions with fewer nonzero entries in $x$), as well as much more sophisticated theoretical justifications (theorems of the form &quot;If $Ax=b$ has a k-sparse solution then minimizing $\| x \|_{1}$ will find that solution with high probability.&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, since data are often noisy, the exact constraint $Ax=b$ is often replaced with a constraint of the form $\| Ax - b \|_{2} \leq \delta$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also quite common to work with a variational form of the constrained problem, where for example we might minimize $\min \| Ax - b\|_{2}^{2} + \lambda \| x \|_{1}$.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-04-07T00:33:22.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6777" PostTypeId="2" ParentId="6775" CreationDate="2013-04-07T01:29:19.770" Score="18" Body="&lt;p&gt;Brian is spot on. But I think it is helpful to add some compressed sensing context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, note that the so-called 0 norm $\|x\|_0$—the cardinality function, or the number of nonzero values in $x$—is &lt;em&gt;not a norm&lt;/em&gt;. It is probably best to write it as something like $\mathop{\textbf{card}}(x)$ in anything but the most casual contexts. Don't get me wrong, you're in good company when you use the $\|x\|_0$ shorthand, but I do think it tends to breed confusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People have known for a long time that minimizing the $\ell_1$ norm $\|x\|_1$ tends to produce sparse solutions. There are some theoretical reasons for this that have to do with linear complementarity. But what was most interesting was not that the solutions were sparse, but that they were often the &lt;em&gt;sparsest possible&lt;/em&gt;. That is, minimizing $\|x\|_1$ &lt;em&gt;really does&lt;/em&gt; give you the minimum-cardinality solution in certain useful cases. (How did they figure this out, when the minimum cardinality problem is NP-hard? By constructing artificial problems with known sparse solutions.) This was not something that the linear complementarity theory could predict.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The field of compressed sensing was born when researchers began to identify conditions on the matrix $A$ that would allow them to &lt;em&gt;guarantee in advance&lt;/em&gt; that the $\ell_1$ solution was also the sparsest. See for example, the earliest papers by &lt;a href=&quot;http://www-stat.stanford.edu/~candes/publications.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Candés, Romberg, and Tao&lt;/a&gt;, and other discussions of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Restricted_isometry_property&quot; rel=&quot;nofollow noreferrer&quot;&gt;Restricted isometry property, or RIP&lt;/a&gt;. Another useful web site if you really want to dive into some theory is &lt;a href=&quot;http://www.math.ucla.edu/~tao/preprints/sparse.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Terence Tao's compressed sensing page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="9667" LastEditDate="2018-08-28T21:12:17.970" LastActivityDate="2018-08-28T21:12:17.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6778" PostTypeId="2" ParentId="6775" CreationDate="2013-04-07T09:01:03.020" Score="8" Body="&lt;p&gt;I have nothing to add to Brians and Michaels explanation about $\ell^1$ vs. $\ell^0$. But since the question seems to be about Compressed Sensing I'd like to add my point of view: Compressed Sensing is &lt;strong&gt;neither&lt;/strong&gt; about solving&#xA;$$\min\|x\|_0\quad\text{s.t}\quad Ax=b$$&#xA;&lt;strong&gt;nor&lt;/strong&gt; about&#xA;$$\min\|x\|_1\quad\text{s.t.}\quad Ax=b.$$&#xA;Compressed Sensing is more a &lt;strong&gt;paradigm&lt;/strong&gt;, which can be stated very roughly as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is possible to identify sparse signals from a few measurements.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Compressed Sensing is really about taking as few &lt;em&gt;measurements&lt;/em&gt; as possible to identify a signal in a certain class of signals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One catchy phrase is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why should your 5 megapixel camera really measure 15 million values (three for each pixel) which cost you 15 megabytes of data when it is only storing about 2 megabytes (after compression)?&lt;br&gt;&#xA;  Could it be possible to measure the 2 megabytes right away?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are quite different frameworks possible:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;linear measurements&lt;/li&gt;&#xA;&lt;li&gt;non-linear ones (e.g. &quot;phaseless&quot; ones)&lt;/li&gt;&#xA;&lt;li&gt;vector data, matrix/tensor data&lt;/li&gt;&#xA;&lt;li&gt;sparsity as just the number of non-zeros&lt;/li&gt;&#xA;&lt;li&gt;sparsity as &quot;low-rank&quot; or even &quot;low complexity&quot;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And there are also more methods to compute sparse solutions like &lt;em&gt;matching pursuits&lt;/em&gt; (variants like orthogonal matching pursuit (OMP), regularized orthogonal matching pursuit (ROMP), CoSaMP) or the more recent methods based on &lt;em&gt;message passing&lt;/em&gt; algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one identifies Compressed Sensing with mere $\ell^1$- or $\ell^0$-minimization, one misses a great deal of flexibility when dealing with practical data acquisition problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one, however, is only interested in obtaining sparse solutions to linear systems, one does something which I would call &lt;strong&gt;sparse reconstruction&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="302" LastEditDate="2016-07-21T06:01:56.563" LastActivityDate="2016-07-21T06:01:56.563" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6780" PostTypeId="1" CreationDate="2013-04-07T16:07:48.003" Score="7" ViewCount="255" Body="&lt;p&gt;Following regularized minimization problem &#xA;$$\min f(x) + \lambda g(x)$$&#xA;where $\lambda&amp;gt;0$,&#xA;and following constrained minimization problem&#xA;$$&#xA;\min f(x)&#xA;$$&#xA;s.t.&#xA;$$&#xA;g(x) \leq \epsilon&#xA;$$&#xA;where $\epsilon&amp;gt;0$, &#xA;can be converted to each other so that they have the same solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples are a $l^1$-regularized linear least square problem and a BPDN problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, which of the two problems is easier to solve? Which one do people prefer for modelling in their applications? Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastActivityDate="2013-04-08T04:13:23.763" Title="Which is easier to solve, regularized minimization, or constrained minimization?" Tags="|optimization|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="6781" PostTypeId="2" ParentId="6780" CreationDate="2013-04-07T18:29:46.990" Score="5" Body="&lt;p&gt;Having dealt with quite a number of nonlinear optimization over the years, my take on this is that the penalized formulation is obviously much simpler to implement. If you're interested in a quick solution for an easy problem, that's the way to go. Historically, this is also the approach that has been taken in the early years of computational optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, the constrained formulation is much more powerful; for example, it can find minima that actually lie at the boundary of the feasible set. The reason why this formulation is not always used is that there is a much higher threshold of entry: it requires a significant effort in implementation and it also requires a significant mathematical machinery to develop efficient numerical methods to solve the linearized problems you have to deal with in every step of, say, a Newton method. In other words, the constrained formulation requires much more work to make it work efficiently; however, it then clearly outperforms penalized methods on all but the simplest problems.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-07T18:29:46.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6782" PostTypeId="2" ParentId="6780" CreationDate="2013-04-07T20:32:15.577" Score="10" Body="&lt;p&gt;Without knowing more about $f$ and $g$, there really isn't going to be a clear winner here. Sometimes, the regularized form is going to be easier; sometimes, the constrained form is going to be easier. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me dispatch some easier cases. First of all, if both $f$ and $g$ are smooth and have readily computable derivatives, then I'll definitely give the edge to the regularized form, because that will give you an unconstrained smooth problem. If you're using an interior-point method to solve either problem, then there should be no consistent advantage for either form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an interesting case. Consider $f$ smooth, with a readily computable gradient. Consider two different possibilities for $g$:&#xA;$$g_1(x)=\|x\|_\infty \quad\text{and}\quad g_2(x)=\|x\|_1$$&#xA;Which will be easier: the regularized form or the constrained form?&#xA;I argue &lt;em&gt;the answer is not the same for both.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A first-order method is going to be a logical choice here, which will involve alternating between standard gradient steps on $f$ and simple nonsmooth minimization steps on $g$. For the regularization form, the $g$ steps will be prox minimizations&#xA;$$&#xA;x = \mathop{\text{arg}\,\text{min}}_z g(x) + \tfrac{1}{2} \|x^{(+)}-z\|_2^2.&#xA;$$&#xA;For the constrained form, the $g$ steps will be projections&#xA;$$&#xA;x = \mathop{\text{arg}\,\text{min}}_{z: g(z)\leq \epsilon} \tfrac{1}{2} \|x^{(+)}-z\|_2^2.&#xA;$$&#xA;In each case, $x^{(+)}$ is the point obtained from taking the gradient step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We implemented both projections and both proximity functions in our software &lt;a href=&quot;http://cvxr.com/tfocs/&quot;&gt;TFOCS&lt;/a&gt;. It turns out that for $g_1(x)=\|x\|_\infty$, the &lt;em&gt;projection&lt;/em&gt; is  the easiest to implement; and for $g_2(x)=\|x\|_1$, the &lt;em&gt;prox minimization&lt;/em&gt; is the easiest. In these two easy cases, the problem decouples---each variable can be determined separately. In the opposite cases---prox minimization for $g_1$, projection for $g_2$---there is coupling across variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe our implementations are $O(n)$ for the easier cases, and $O(n\log n)$ for the harder ones. To be fair, neither cost will be significant if you're solving a problem on a single machine where the gradient steps dominate. But what if you're solving a very large-scale problem on a cluster, using distributed computation to compute the gradient? Then you're going to prefer the &lt;em&gt;decoupling&lt;/em&gt; that the $\|x\|_\infty$ projection and the $\|x\|_1$ prox provide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One case I haven't considered is the one where $f$ is nonsmooth and $g$ is smooth. If the regularized form is simple to compute---because $f(x)$ admits a simple prox or projection---then yes, I'd give the edge to that form. But if you can compute the &lt;em&gt;dual&lt;/em&gt; of the constrained problem, it is likely to take on a &lt;em&gt;regularized form&lt;/em&gt; as well! So there is a real possibility that you could construct an algorithm to solve the constrained form with similar efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in summary, it really depends on the specific characteristics of $f$ and $g$, the kinds of algorithms you're willing to deploy, and how much work you're going to do to &lt;em&gt;extract&lt;/em&gt; efficiency. (EDIT: I'm assuming convex problems throughout here. But if $f$ is non-convex but smooth and $g$ is non-smooth and convex, then most of what I've said above about first-order methods will still apply! Of course you can't guarantee global convergence anymore.)&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-04-08T04:13:23.763" LastActivityDate="2013-04-08T04:13:23.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6784" PostTypeId="1" AcceptedAnswerId="6785" CreationDate="2013-04-08T00:21:48.810" Score="13" ViewCount="7974" Body="&lt;p&gt;Given an unknown function $f:\mathbb R^d \to \mathbb R$, we can evaluate its value at any point in its domain, but we don't have its expression. In other words, $f$ is like a black box to us.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the name for the problem of finding the minimizer of $f$? What are some methods out there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the name for the problem of finding the solution to the equation $f(x)=0$? What are some methods out there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above two problems, is it a good idea to interpolate or fit to some evaluations of f: $(x_i, f(x_i)), i=1, \dots, n$ using a function $g_\theta$ with known form and parameter $\theta$ to be determined, and then minimize $g_\theta$ or find its root?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="258" LastEditDate="2013-04-08T00:34:13.037" LastActivityDate="2016-05-19T19:15:31.620" Title="Optimize an unknown function which can be evaluated only?" Tags="|optimization|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="6785" PostTypeId="2" ParentId="6784" CreationDate="2013-04-08T02:03:16.650" Score="15" Body="&lt;p&gt;The methods you are looking for -- i.e., that only use function evaluations but not derivatives -- are called &lt;em&gt;derivative free optimization methods&lt;/em&gt;. There is a large body of literature on them, and you can find a chapter on such methods in most books on optimization. Typical approaches include&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Approximating the gradient by finite differences if one can reasonably expect the function to be smooth and, possibly, convex;&lt;/li&gt;&#xA;&lt;li&gt;Monte Carlo methods such as Simulated Annealing;&lt;/li&gt;&#xA;&lt;li&gt;Genetic Algorithms.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-08T02:03:16.650" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="6786" PostTypeId="1" CreationDate="2013-04-08T02:35:42.847" Score="1" ViewCount="231" Body="&lt;p&gt;I'm currently learning about using linear and semidefinite programming to find sparse solutions to problems. In particular, finding sparse solutions where the sampling functions are sinusoidal (Dirichlet kernel).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've implemented a linear programming solution with a discrete approximation using &lt;code&gt;l1magic&lt;/code&gt; MATLAB library, and an SDP solution based on the MATLAB example in the paper &lt;a href=&quot;http://arxiv.org/abs/1203.5871&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Towards a mathematical theory of super-resolution&quot;&lt;/a&gt; someone suggested in a &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5593/how-to-implement-this-trigonometric-polynomial-maximum-finding-semidefinite-prog&quot;&gt;previous question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CVX code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cvx_solver sdpt3&#xA;cvx_begin sdp &#xA;    variable X(n+1,n+1) hermitian;&#xA;    X &amp;gt;= 0;&#xA;    X(n+1,n+1) == 1;&#xA;    trace(X) == 2;&#xA;    for j = 1:n-1,&#xA;        sum(diag(X,j)) == X(n+1-j,n+1);&#xA;    end&#xA;    maximize(real(X(1:n,n+1)'*y))&#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the above paper, if the minimum distance between spikes is greater than or equal to $2 \lambda_c$, where $\lambda_c$ is the wavelength of the cutoff frequency, this condition is sufficient to guarantee the solution is unique and can be obtained to infinite precision (page 5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've noticed that if all the spikes are the same sign then shorter distances still give me the correct result. If they are different signs then the above condition seems to apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Question:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to introduce either a&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;cardinality constraint, where the number of spikes is known, or&lt;/li&gt;&#xA;&lt;li&gt;sign constraint, where the signs of the spikes are known&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;that can give a solution where the minimum distance condition is not met?&lt;/p&gt;&#xA;" OwnerUserId="3565" LastEditorUserId="20417" LastEditDate="2018-04-06T14:26:32.530" LastActivityDate="2018-04-06T14:26:32.530" Title="Sign or cardinality constraint when solving for sparse signal" Tags="|matlab|convex-optimization|semidefinite-programming|cvx|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6787" PostTypeId="2" ParentId="6772" CreationDate="2013-04-08T09:33:05.007" Score="1" Body="&lt;p&gt;From the computational point of view, each element of the final matrix (resulting image) is equal to a operation between the around elements in the initial matrix (input image) and the respective elements of the kernel, so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ final_{i,j} = \\&#xA; initial_{r(i-1),r(j-1)} OP kernel_{1,1} + initial_{r(i),r(j-1)} OP kernel_{2,1} + initial_{r(i+1),r(j-1)} OP kernel_{3,1} + \\&#xA; initial_{r(i-1),r(j)} OP kernel_{1,2} + initial_{r(i),r(j)} OP kernel_{2,2} + initial_{r(i+1),r(j)} OP kernel_{3,2} + \\&#xA; initial_{r(i-1),r(j+1)} OP kernel_{1,3} + initial_{r(i),r(j+1)} OP kernel_{2,3} + initial_{r(i+1),r(j+1)} OP kernel_{3,3} \\&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $r(x)$ represents the integer division between the x and the matrix side.&lt;/p&gt;&#xA;" OwnerUserId="3580" LastActivityDate="2013-04-08T09:33:05.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6788" PostTypeId="1" CreationDate="2013-04-08T09:53:28.637" Score="7" ViewCount="9899" Body="&lt;p&gt;I was wondering what differences are between the terminology: &quot;least square (LS)&quot; &quot;mean square (MS)&quot; and &quot;least mean square (LMS)&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get confused when reading in Spall's Introduction to Stochastic Search and Optimization, &lt;a href=&quot;http://books.google.com/books?id=f66OIvvkKnAC&amp;amp;lpg=PP1&amp;amp;pg=PA68#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;section 3.1.2 Mean-Squared and Least-Squares Estimation&lt;/a&gt; and &lt;a href=&quot;http://books.google.com/books?id=f66OIvvkKnAC&amp;amp;lpg=PP1&amp;amp;pg=PA72#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;section 3.2.1 Introduction&lt;/a&gt; and &lt;a href=&quot;http://books.google.com/books?id=f66OIvvkKnAC&amp;amp;lpg=PP1&amp;amp;pg=PA73#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;section 3.2.2 Basic LMS Algorithm&lt;/a&gt;. Is his terminology standard?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks and regards!&lt;/p&gt;&#xA;" OwnerUserId="258" LastActivityDate="2013-04-09T18:36:53.370" Title="Differences between &quot;least square&quot;, &quot;mean square&quot; and &quot;least mean square&quot;?" Tags="|optimization|stochastic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6789" PostTypeId="1" AcceptedAnswerId="6795" CreationDate="2013-04-08T15:01:00.170" Score="3" ViewCount="1162" Body="&lt;p&gt;I apologize if this is a naive question.  I'm trying to create some boostrap data for a system of linear, ordinary differential equations at steady state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the equations represent the concentration of chemical species, X has to be positive.  Further, I'm trying to find the sparsest possible matrix of A, for the sake of keeping the dependency graph of the equations as small as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the cvx toolbox in MATLAB, I put the problem like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;&#xA;% the threshold value below which we consider an element to be zero&#xA;delta = 1e-8;&#xA;&#xA;% problem dimensions (m inequalities in n-dimensional space)&#xA;n = 25;&#xA;&#xA;X  = rand(n, 1)&#xA;b  = zeros(n, 1);&#xA;c = zeros(n, n) + delta^2&#xA;alpha = 0.1&#xA;&#xA;&#xA;% l1-norm heuristic for finding a sparse solution&#xA;fprintf(1, 'Finding a sparse feasible point using l1-norm heuristic ...')&#xA;cvx_begin&#xA;  variable A(n,n) &#xA;  %minimize(nnz(A))&#xA;  minimize(alpha * nnz( A ) + (1-alpha) * norm(A*X - b, 2) )&#xA;cvx_end&#xA;&#xA;% number of nonzero elements in the solution (its cardinality or diversity)&#xA;nonzero = length(find( abs(A) &amp;gt; delta ));&#xA;fprintf(1,['\nFound a feasible A in R^%d that has %d nonzeros ' ...&#xA;           'using the l1-norm heuristic.\n'],n,nonzero);    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This tends to find matrix of A which are filled with incredibly small near zero values.  Does anyone have suggestions for heuristic approaches to find a combination of X and A which satisfy the constraints I described above?  They don't need to be particularly fast or memory efficient, I'm working with relatively small matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be clear - I'm not trying to solve for a particular A or X.  I'm trying to find a combination of A and X that satisfy the constraints I described.&lt;/p&gt;&#xA;" OwnerUserId="4052" LastEditorUserId="4052" LastEditDate="2013-04-08T18:39:31.740" LastActivityDate="2013-04-09T16:25:08.433" Title="Finding A and X such that AX = 0, X is positive non-zero, and A is sparse" Tags="|linear-algebra|matlab|convex-optimization|cvx|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6790" PostTypeId="2" ParentId="6789" CreationDate="2013-04-08T19:36:47.087" Score="3" Body="&lt;p&gt;You talk about l1-heuristics, but in the code it looks more like you are interested in solving the underlying combinatorial problem involving the nnz of A. In a l1-heuristics, you would typically use norm(A(:),1) or something like that. The fact you only get zeros is natural as the problem is homogenous in A when b is 0. You can introduce an arbitrary scale in the problem though, such as sum(sum(A))=1 to ensure the solution is non-zero.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-04-08T19:36:47.087" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6791" PostTypeId="1" AcceptedAnswerId="6792" CreationDate="2013-04-09T00:01:43.580" Score="1" ViewCount="827" Body="&lt;p&gt;Here is some code that hopefully clearly illustrates what I'm doing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; sp.linalg.eig(d)[0].real; sp.linalg.eigh(d)[0]; d; sp.linalg.eig(d)[1]; sp.linalg.eigh(d)[1]&#xA;array([ 1., -1.])&#xA;array([-1.,  1.])&#xA;matrix([[ 0, -1],&#xA;        [-1,  0]])&#xA;array([[ 0.70710678,  0.70710678],&#xA;       [-0.70710678,  0.70710678]])&#xA;array([[-0.70710678, -0.70710678],&#xA;       [-0.70710678,  0.70710678]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To explain the code a little, the first two &lt;code&gt;array()&lt;/code&gt;'s are the eigenvalues from &lt;code&gt;eig()&lt;/code&gt; and &lt;code&gt;eigh()&lt;/code&gt; respectively, where the latter is only for Hermitian matrices. Then I show the matrix &lt;code&gt;d&lt;/code&gt;, and then the eigenvectors are shown as the columns of the following two matrices, in the same order as the eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, what I'm doing is taking a matrix &lt;code&gt;d&lt;/code&gt; which in this case is simply $\sigma_x$, the Pauli x-spin matrix. I'm doing adiabatic quantum simulation, so I'm getting the ground state here of a transverse magnetic field for a 1-qubit case. Higher qubit cases have initial state vector that corresponds to the lowest energy for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D = -\sum_i^N I \otimes I \otimes ...\otimes ~\sigma_x^i \otimes...\otimes ~I$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what I'm doing now is using the general LAPACK eigensolver zgeev with the &lt;code&gt;eig()&lt;/code&gt; call and zheevd with the &lt;code&gt;eigh()&lt;/code&gt; call, which is general and Hermitian solvers, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do I get the ground state eigenvector as one thing from &lt;code&gt;eig()&lt;/code&gt;, but with &lt;code&gt;eigh()&lt;/code&gt; I get the same thing but with an overall negative? And does it matter at all?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth noting that this problem does NOT arise if I say $D$ has positive values instead of negative ones. That bit really confuses me.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2016-03-08T19:34:26.200" Title="Ground state eigenvector different for different eigen solvers (differs by negative sign in the elements). Does it matter?" Tags="|linear-algebra|eigensystem|" AnswerCount="4" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6792" PostTypeId="2" ParentId="6791" CreationDate="2013-04-09T09:59:26.620" Score="5" Body="&lt;p&gt;Eigenvector solutions to the standard eigenvalue problem formulation ($A\mathbf{v}=\lambda\mathbf{v}$) are unique up to a constant multiplier. It is clear that if there exists an eigenvector solution $\mathbf{v_a}$ which satisfies $A\mathbf{v_a}=\lambda\mathbf{v_a}$, then for any real\complex scalar $k$, $\mathbf{v_b}=k\mathbf{v_a}$ is also valid simply because $A\mathbf{v_b}=kA\mathbf{v_a}=k\lambda\mathbf{v_a}=\lambda\mathbf{v_b}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, it is not surprising that two different eigensolvers are returning the same eigenvector scaled differently. You must define your own renormalization procedure if you wish to completely uniquely define the eigenvectors (for instance: always force one of the components of the eigenvector to be 1 by dividing the resulting eigenvector with the value of that component.)&lt;/p&gt;&#xA;" OwnerUserId="1480" LastActivityDate="2013-04-09T09:59:26.620" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6794" PostTypeId="1" AcceptedAnswerId="6803" CreationDate="2013-04-09T16:13:18.413" Score="10" ViewCount="722" Body="&lt;p&gt;C++11 introduces move semantics which can, for example, improve code performance in situations where C++03 would need to perform a copy construction or copy assignment.  This &lt;a href=&quot;http://lemire.me/blog/archives/2012/11/26/why-i-like-the-new-c/&quot;&gt;article&lt;/a&gt; reports that following code experiences a 5x speed up when compiled with C+11:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vector&amp;lt;vector&amp;lt;int&amp;gt; &amp;gt; V;&#xA;for(int k = 0; k &amp;lt; 100000; ++k) {&#xA;    vector&amp;lt;int&amp;gt; x(1000);&#xA;    V.push_back(x);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is the impact of C++11 move semantics in the context of scientific computing?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in this question is general but more specifically I'm also interested in move semantics for Finite Element codes written using boost libraries. I tested some of my own C++03 code using boost version 1.47.0 (since &lt;a href=&quot;http://www.boost.org/users/history/&quot;&gt;boost release notes&lt;/a&gt; mention move semantics are introduced in 1.48.0) and boost version 1.53.0, but I didn't notice much improvement. I guess any savings from not having to do copy-construction for &lt;code&gt;boost::numeric::ublas::vector&lt;/code&gt;/&lt;code&gt;matrix&lt;/code&gt; and &lt;code&gt;boost::function&lt;/code&gt; are not noticeable since solving system matrices constitutes the bulk of the workload.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Actually it looks like move semantics are only implemented for &lt;code&gt;boost::function&lt;/code&gt; (see &lt;a href=&quot;http://www.boost.org/users/history/version_1_52_0.html&quot;&gt;version 1.52.0 release notes&lt;/a&gt;).  There is no mention of move semantics in &lt;code&gt;boost::numeric&lt;/code&gt;, I grepped the sources to make sure and there seem to be no rvalue references.&lt;/p&gt;&#xA;" OwnerUserId="482" LastEditorUserId="482" LastEditDate="2013-04-09T16:19:33.143" LastActivityDate="2014-03-13T01:15:01.123" Title="What is the impact of C++11 move semantics in the context of scientific computing?" Tags="|finite-element|performance|c++|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6795" PostTypeId="2" ParentId="6789" CreationDate="2013-04-09T16:25:08.433" Score="2" Body="&lt;p&gt;Without loss of generality we may treat the case $X$ is all-ones vector $u$, since forming $AD^{-1}X$ is equivalent to $Au$ where $D = \text{diag}(X)$ is the diagonal matrix with entries from $X$, and $A$ has the same sparsity as $AD^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mention is made of &quot;keeping the dependency graph of the equations as small as possible&quot;.  Aside from avoiding the triviality of a matrix of all zeros, one might want to impose a condition that this dependency graph be connected, or more weakly, that no row of $A$ is allowed to be all zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the goal is to minimize the number of nonzero entries in $A$, subject to $Au=0$, then we must have at least two nonzero entries in each nonzero row of $A$.  Moreover this minimum can be attained by placing both a $+1$ and a $-1$ entry in each such row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If connectedness of the dependency graph is sought, it can be achieved without further loss of sparsity.  For example, define $A = I - P$ where $P$ is the permutation matrix s.t. $P_{1,n} = 1$, $P_{i+1,i} = 1$ for $i=1,\ldots,n-1$, and otherwise $P_{i,j} = 0$.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-04-09T16:25:08.433" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6797" PostTypeId="2" ParentId="6737" CreationDate="2013-04-09T17:33:14.950" Score="1" Body="&lt;p&gt;I am guessing that the derivative of one of the functions is already machine precision 0 as the error also indicates. Your entire ODE system evolves within &lt;code&gt;tend &amp;lt; 150&lt;/code&gt; and is in steady state during the next following 14850 steps. At some point the derivative between two consecutive time steps is evaluate to 0.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2013-04-09T17:33:14.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6798" PostTypeId="2" ParentId="6788" CreationDate="2013-04-09T18:36:53.370" Score="6" Body="&lt;p&gt;The term &quot;mean square&quot; is usually used when one wants to minimize a quantity that can be either positive or negative. Consider a series of values $x_i$ for $i = 1, \ldots, N$. If the $x_i$ are all large positive or large negative numbers, then the average value $\left&amp;lt; x \right&amp;gt;$ of the $x_i$ could still be nearly zero, even though none of the individual values are. However, if you want to consider the mean &lt;em&gt;square&lt;/em&gt; of the $x_i$, then you know that $\left&amp;lt; x^2 \right&amp;gt;$ will be small only if the individual $x_i$ are small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &quot;least squares&quot; just refers to the form of regression in which you try to control (minimize) the square of the deviations between the predicted and observed values, while &quot;least mean square&quot; combines these ideas.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2013-04-09T18:36:53.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6799" PostTypeId="2" ParentId="6757" CreationDate="2013-04-09T23:29:46.613" Score="1" Body="&lt;p&gt;Another approach would be the deflation approach. In this approach you apply typically an iterative method like Newton's to the original polynomial to find a first root. Next, you apply again Newton's algorithm but this time to the deflated polynomial, i.e. the original polynomial divided by $ x-x_1 $ where $ x_1$ is the first root. By using Horner's rule intelligently you can stably evaluate the deflated polynomial without calculating the coefficients explicitly. You can repeat this until you have fully deflated the polynomial. Typically one would then do one or two extra Newton steps on the original polynomial using the roots just calculated as initial values. I believe this approach is called Maehly's algorithm.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-04-09T23:29:46.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6800" PostTypeId="1" CreationDate="2013-04-09T23:30:22.360" Score="2" ViewCount="135" Body="&lt;p&gt;For a little project I'm working on, I am trying to generate a wavefunction basis set I can use in Quantum Monte Carlo (DMC to be specific).  Preferably, it would be a linear combination of Slater determinants with a Jastrow term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my problem is that I don't want a pseudo-potential or any sort of electron distribution approximation.  I want to compute the potential energy curve of a Li-C interaction (for different spacings), so it would be 3 + 12 for a total of 15 electrons (fifteen -1 charges) that interact via the BO Schrodinger equation with a positive +3 and a positive +12 charge.  Is there a tool that can do this for me?&lt;/p&gt;&#xA;" OwnerUserId="1997" LastActivityDate="2013-12-01T03:52:38.717" Title="Trying to generate a wave function basis set" Tags="|computational-chemistry|basis-set|quantum-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6801" PostTypeId="1" CreationDate="2013-04-09T23:51:09.790" Score="0" ViewCount="168" Body="&lt;p&gt;I would like to find the analytical form of the point which minimizes the following function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(x_T) = \frac{1}{T} a_1 (x_T-x_0)^2 + a_2 |x_T-x_0| + T a_3 + \sum_{i=1}^M p_i \left[b_{1i} (x_T - l_i)^2 + b_{2i} |x_t - l_i| + b_{3i} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $T,a_1,x_0,a_2,a_3,p_i,b_{1i},l_i,b_{2i},b_{3i}$ are constants, $T,a_1,x_0,a_2,p_i,b_{1i},l_i,b_{2i}$ are greater than zero, and M is a positive index. The function is continuous, convex, and piecewise differentiable. Any ideas on how to find the analytical form of the point, $x^*_T$, which minimizes $f(x_T)$?&lt;/p&gt;&#xA;" OwnerUserId="4060" LastActivityDate="2013-04-10T21:39:40.940" Title="Analytical form of the minimum of a function with absolute values" Tags="|optimization|convex-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6803" PostTypeId="2" ParentId="6794" CreationDate="2013-04-10T03:39:50.073" Score="10" Body="&lt;p&gt;I think in practice the impact is limited and will be limited. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why it &lt;em&gt;is&lt;/em&gt; limited right now is that the big finite element packages are careful to write code that is portable, and so they do not yet use C++11 language constructs in their own codes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, they will benefit from code like the one you show where, even without having to change the source code, you benefit from an improved compiler support library such as libstdc++ for GCC. That said, people in scientific computing are pretty well aware of the expense of copying and so, for objects where it matters, they simply don't copy -- they either generate in-place, or they do shallow copies, or they use shared pointers, or many of the other techniques available to avoid the overhead of copying. In other words, the situation for which move semantics were invented does not actually happen very often in &quot;real&quot; scientific codes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-10T03:39:50.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6804" PostTypeId="2" ParentId="6801" CreationDate="2013-04-10T03:54:56.517" Score="2" Body="&lt;p&gt;I very much doubt that such a closed form solution exists. Certainly, if one can be found, it needs to make use of the fact that what you have is a linear combination of functions of the form $\gamma_1(x_T-y)^2 + \gamma_2 |x_T-y|$ which is certainly a nice function to have. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the solution of your problem is not continuous with respect to the parameters $a_{1/2},b_{i1/2}$. You can already see that if, for a moment, you neglect the quadratic terms (i.e., $a_1=b_{i1}=0$) and consider the case $M=1$. In that case, you add two absolute value functions, i.e., &#xA;$$&#xA;  f(x_T) = a_1 |x_T-x_0| + b_{11} |x_T-l_1|&#xA;$$&#xA;It is easy to see that if $a_1&amp;gt;b_{11}$ then the minimum is at $x_T=x_0$ whereas for $a_1&amp;lt;b_{11}$ the minimum is at $x_T=l_1$. For $a_1=b_{11}$, every point between $x_0$ and $l_1$ is a the minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This example already shows that even a simplified case is not easy to solve. It's only going to get worse if you add more parameters to the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(As an aside: One might speculate that one of the points $x_0, l_i$ is in fact the minimum, given the form of the terms you add up. However, I believe that it would be simple to give a counter-example for which this is not true. But, even if it was, the question which of these finitely many points is the minimum clearly depends on the relative sizes of all of your coefficients and, as the example above shows, it's not going to be a simple decision.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-10T03:54:56.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6805" PostTypeId="1" AcceptedAnswerId="6806" CreationDate="2013-04-10T07:38:25.663" Score="9" ViewCount="9238" Body="&lt;p&gt;How do I approximate the condition number of a large matrix $G$, if $G$ is a combination of Fourier transforms $F$ (non-uniform or uniform), finite differences $R$, and diagonal matrices $S$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrices are very large and not stored in memory and are only available as functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, I have the following matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$G_\mu=S^HF^HFS+\mu R^HR$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to investigate the relation between $\mu$ and the condition number $k(G_\mu)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume one needs some kind of iterative approach?&#xA;Optimally there would be some MATLAB code availabe.&lt;/p&gt;&#xA;" OwnerUserId="2026" LastEditorUserId="127" LastEditDate="2017-05-21T18:51:59.650" LastActivityDate="2017-05-21T18:51:59.650" Title="How to approximate the condition number of a large matrix?" Tags="|linear-algebra|matlab|finite-difference|condition-number|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6806" PostTypeId="2" ParentId="6805" CreationDate="2013-04-10T11:46:48.050" Score="12" Body="&lt;p&gt;MATLAB has a couple of &quot;exact&quot; functions for this, &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/cond.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;cond&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/rcond.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;rcond&lt;/code&gt;&lt;/a&gt;, with the latter returning a reciprocal of the condition number.  Matlab approximate function &lt;code&gt;condest&lt;/code&gt; is more fully described below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often estimates of the condition number are generated as by-products of the solution of a linear system for the matrix, so you might be able to piggyback the condition number estimates on other work you need to do anyway.  See &lt;a href=&quot;http://www.alglib.net/matrixops/rcond.php&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; for a brief description of how estimates are computed.  Also &lt;a href=&quot;http://trilinos.sandia.gov/packages/aztecoo/AztecOOUserGuide.pdf&quot; rel=&quot;noreferrer&quot;&gt;Sandia Labs AztecOO documentation&lt;/a&gt; remarks (see Sec. 3.1) that optional condition number estimates are available from iterative solvers (using the generated tridiagonal Lanczos matrix with Conjugate Gradients or the generated Hessenburg matrix with Restarted GMRES).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your matrices are &quot;very large&quot; and &quot;only available as functions&quot;, the logical approach would be a method that piggybacks on a conjugate gradient solver or variant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A recent arXiv.org paper &lt;a href=&quot;http://arxiv.org/ftp/arxiv/papers/1302/1302.4824.pdf&quot; rel=&quot;noreferrer&quot;&gt;Non-stationary extremal eigenvalue approximations in iterative&#xA;solutions of linear systems and estimators for relative error&lt;/a&gt; proposes such an approach and has a few citations to the earlier literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that I look, this forum has a number of closely related previous Questions (not all with Answers, but check Comments):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/3437/651&quot;&gt;Estimate extreme eigenvalues with CG&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/365/651&quot;&gt;Estimation of condition numbers for very large matrices&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/3534/651&quot;&gt;Fastest algorithm to compute the condition number of a large matrix in Matlab/Octave&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Since availability of MATLAB code was part of the Question, here's some information about &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/condest.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;condest&lt;/code&gt;&lt;/a&gt;, a built-in function that estimates the 1-norm condition number $\|A\|_1 \|A^{-1}\|_1$.  The idea is from Hager(1984), with &lt;a href=&quot;http://www.co.it.pt/~judice/Articles/artigo_TOP_CarmoHager.pdf&quot; rel=&quot;noreferrer&quot;&gt;a 2010 write-up and extensions here&lt;/a&gt;, to explicitly compute $\|A\|_1$ (find maximum 1-norm of a column) and estimate $\|A^{-1}\|_1$ by a gradient method.  See also John Burkardt's &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/m_src/condition/condition.html&quot; rel=&quot;noreferrer&quot;&gt;CONDITION&lt;/a&gt;, a MATLAB library (other languages available) &quot;for computing or estimating the condition number of a matrix.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your matrix is apparently Hermitian and positive definite, perhaps the 2-norm condition number is of greater interest.  The problem then amounts to estimating the ratio of largest to smallest (absolute) eigenvalues.  The challenge is somewhat parallel to the 1-norm case in that generally a good &lt;a href=&quot;http://www.ams.org/journals/mcom/1982-38-157/S0025-5718-1982-0637293-9/S0025-5718-1982-0637293-9.pdf&quot; rel=&quot;noreferrer&quot;&gt;estimate for the largest eigenvalue&lt;/a&gt; can be easily obtained, but estimating the smallest eigenvalue proves more difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although aiming at non-SPD (and even non-square) cases, this recent arXiv.org paper, &lt;a href=&quot;http://arxiv.org/pdf/1301.1107v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;Reliable Iterative Condition-Number Estimation&lt;/a&gt;, gives a good overview of the smallest eigenvalue estimation problem and a promising line of attack by a Krylov-subspace method (LSQR) that amounts to Conjugate Gradients in the SPD case.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-05-14T15:54:08.357" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6808" PostTypeId="1" AcceptedAnswerId="6813" CreationDate="2013-04-10T17:58:52.543" Score="10" ViewCount="289" Body="&lt;p&gt;We're all familiar with the many computational methods to solve the standard linear system &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Ax=b.&#xA;$$However, I'm curious if there are any &quot;standard&quot; computational methods for solving a more general (finite dimensional) linear system of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;LA=B,&#xA;$$ where, say, $A$ is an $m_1\times n_1$ matrix, $B$ is an $m_2\times n_2$ matrix, and $L$ is a linear operator taking $m_1\times n_1$ matrices to $m_2\times n_2$ matrices, which &lt;em&gt;does not involve vectorizing the matrices&lt;/em&gt;, i.e. converting everything to the standard $Ax=b$ form.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I ask is I need to solve the following equation for $u$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;(R^*R+\lambda I)u=f&#xA;$$ where $R$ is the 2d Radon transform, $R^*$ its adjoint, and both $u$ and $f$ are 2d arrays (images).  It is possible to vectorize this equation, but it's a pain, especially if we go to 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, what about $nD$ arrays?  For instance, solving $LA=B$ where $A$ and $B$ are 3D arrays (I'll need to do this with Radon transform at some point as well).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks ahead, and feel free to ship me off to another StackExchange if you feel the need.&lt;/p&gt;&#xA;" OwnerUserId="3303" LastActivityDate="2013-04-12T12:38:28.093" Title="Solving a linear system with matrix arguments" Tags="|linear-algebra|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6809" PostTypeId="2" ParentId="6808" CreationDate="2013-04-10T20:26:50.233" Score="3" Body="&lt;p&gt;As it turns out, because my system is symmetric and positive definite (since my linear operator is written as $R^*R+\lambda I$), &lt;a href=&quot;http://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow&quot;&gt;conjugate gradient&lt;/a&gt; can be adapted to solve this type of equation iteratively.  The only modification comes when computing inner products - i.e. a typical inner product computation in CG looks like $r_k^Tr_k$ or $p_k^TAp_k$.  In the modified version, we use the Frobenius inner product, which can be computed by summing the entries of the Hadamard (pointwise) product.  I.e. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\langle A,B\rangle=\sum_{i,j}A_{ij}B_{ij}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect this will go through just fine when I upgrade to 3D arrays, though I have yet to see the Frobenius inner product defined on 3D arrays (I'll work under the assumption that I can again just sum the pointwise product).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd still be interested in more general methods if anyone knows of any!&lt;/p&gt;&#xA;" OwnerUserId="3303" LastActivityDate="2013-04-10T20:26:50.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6810" PostTypeId="2" ParentId="6801" CreationDate="2013-04-10T21:39:40.940" Score="2" Body="&lt;p&gt;This is not an analytic solution, but rather a strategy to find the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cusp points $x_0,l_i$ partition $\mathbb{R}$ into a series of intervals. The minimum either lies at one of the cusp points, or lies in one of the intervals inbetween. You could do a binary search of these intervals to see which cusp point the derivative changes sign around. Then check the left and right derivatives at that cusp point to see if the subderivative there contains zero. If it does, that's the minimum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if both the left and right derivatives at the cusp point are positive, the solution is in the interval directly to the left. Similarly, if both derivatives are negative, the solution is in the interval directly to the right. In both of these intervals the function is quadratic so there is a formula for the solution.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2013-04-10T21:39:40.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6812" PostTypeId="1" AcceptedAnswerId="6816" CreationDate="2013-04-11T01:58:24.203" Score="15" ViewCount="8334" Body="&lt;p&gt;Sometimes while optimizing code it is required to time certain portions of the code, I have been using the following for years but was wondering if there is a simpler/better way to do it?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;call system_clock(count_rate=clock_rate) !Find the time rate&#xA;call system_clock(count=clock_start)     !Start Timer&#xA;&#xA;call do_something_subroutine             !This is what gets timed&#xA;&#xA;call system_clock(count=clock_stop)      ! Stop Timer&#xA;&#xA;e_time = real(clock_stop-clock_start)/real(clock_rate)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3561" LastEditorUserId="123" LastEditDate="2013-04-15T17:04:05.253" LastActivityDate="2013-04-15T17:04:05.253" Title="Fortran: Best way to time sections of your code?" Tags="|optimization|fortran|profiling|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6813" PostTypeId="2" ParentId="6808" CreationDate="2013-04-11T02:05:55.257" Score="9" Body="&lt;p&gt;Yes, you've got it right, and it will indeed work fine when you upgrade to 3-D. The easiest part, really, is the inner product---just do a standard dot product on the equivalent, unrolled $\mathbb{R}^n$ vectors. Since you'll likely have the data stored contiguously anyway, you can do this in-place. This even works with complex vector spaces---just treat the complex values as pairs of real values. That's because for CG you need the &lt;em&gt;real&lt;/em&gt; inner product $\langle y, x\rangle\triangleq \mathop{\textrm{Re}}(y^Hx)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you have to be careful of when you're implementing CG (or similar iterative approaches) with general linear operators is to implement the &lt;em&gt;adjoint&lt;/em&gt; of your linear operator properly. That is, people often get $y=F(x)$ right, but  make a mistake implementing $z=F^*(y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend implementing a simple test that takes advantage of the following identity: for any conforming $x$ and $y$,&#xA;$$\langle y, F(x) \rangle = \langle F^*(y), x \rangle.$$&#xA;So what you do is generate random values of $x$ and $y$, run them through your forward and adjoint operations, respectively, and compute the two inner products above. Make sure they match to within reasonable precision, and repeat a few times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: what do you do if your linear operator is supposed to be symmetric? Well, you do need to verify that symmetry, too. So use the same test, just noting that $F=F^*$---apply the same operation to $x$ and $y$. Of course, the OP has both an asymmetric operator and a symmetric one to deal with...&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-04-12T12:38:28.093" LastActivityDate="2013-04-12T12:38:28.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6814" PostTypeId="1" AcceptedAnswerId="6835" CreationDate="2013-04-11T02:59:22.890" Score="3" ViewCount="133" Body="&lt;p&gt;I've been reading about the algebraic multigrid algorithm and came across polynomial smoothers in &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999103001943&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. It's my understanding that usually the coarse-level matrices&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_H = I_h^HA_hI_H^h$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are formed explicitly. However, there's still the issue of how to define the coarse space corresponding to a given smoother. The paper suggests using polynomial smoothing as an adjunct to smoothed aggregation, in which case the prolongation operator is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$I_H^h = p(A)\cdot\mathscr{I}_H^h$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $p$ is some shifted Chebyshev polynomial and $\mathscr{I}$ is the aggregation operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is explicitly forming $A_H$ with this approach prohibitively expensive? Moreover, am I even correct in assuming you need to explicitly compute the coarse level matrices?&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-04-13T03:19:35.173" Title="explicitly forming coarse matrices with polynomial smoothing AMG" Tags="|linear-algebra|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6815" PostTypeId="2" ParentId="6812" CreationDate="2013-04-11T05:36:25.457" Score="4" Body="&lt;p&gt;If you use the GNU compiler, check out &lt;a href=&quot;http://www.cs.utah.edu/dept/old/texinfo/as/gprof_toc.html&quot; rel=&quot;nofollow&quot;&gt;gprof&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, you'll add the -g flag to your compiler, like so: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g77 -g -pg -0 myprogram myprogram.F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, run the output, and a file called gmon.out will show up in your directory.  Then, call &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gprof --line myprogram gmon.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will give a line-by-line CPU time profile.&lt;/p&gt;&#xA;" OwnerUserId="3303" LastActivityDate="2013-04-11T05:36:25.457" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6816" PostTypeId="2" ParentId="6812" CreationDate="2013-04-11T06:18:13.333" Score="12" Body="&lt;p&gt;There are a few other ways to do this, with advantages and drawbacks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Wtime.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MPI_WTIME&lt;/a&gt;: This is a high resolution wall-clock.  It is probably the most  `trusted' option; it just works.  The downside is that if your program doesn't already use MPI, you'll have to wrap MPI around it (which isn't hard).&lt;/li&gt;&#xA;&lt;li&gt;Use a fortran intrinsic (as you have): This is probably the easiest, and generally sufficient, but might not work so well on a strange architecture or for parallel jobs.  There is a bit of a &lt;a href=&quot;https://stackoverflow.com/questions/6878246/fortran-intrinsic-timing-routines-which-is-better-cpu-time-or-system-clock&quot;&gt;discussion&lt;/a&gt; on this Stack Overflow&lt;/li&gt;&#xA;&lt;li&gt;Wrap a C call: Fortran and C are object-compatible, so its easy enough to write a wrapper around C calls.  A code I work with uses getrusage, which might be an odd choice.  There are &lt;a href=&quot;https://stackoverflow.com/questions/538609/high-resolution-timer-with-c-and-linux?lq=1&quot;&gt;many&lt;/a&gt; &lt;a href=&quot;https://stackoverflow.com/questions/5644730/c-measuring-computing-time?rq=1&quot;&gt;discussions&lt;/a&gt; of this on Stack Overflow.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My personal recommendation would be MPI_WTIME, as you know it will work well wherever there is MPI.  Here is an &lt;a href=&quot;http://www.hpcx.ac.uk/support/documentation/UserGuide/HPCxuser/Porting_Codes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; from a quick search:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  include 'mpif.h'&#xA;  DOUBLE PRECISION :: start, end&#xA;  start = MPI_Wtime()&#xA;&#xA;  ! code to be timed&#xA;&#xA;  end   = MPI_Wtime()&#xA;  write(*,*) 'That took ',end-start,' seconds'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="521" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-04-12T18:13:02.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6817" PostTypeId="2" ParentId="2328" CreationDate="2013-04-11T06:48:58.037" Score="3" Body="&lt;p&gt;I'm a little late on this one, but you should have a look at version 3 of the parallel linear algebra library &lt;a href=&quot;http://www.ce.uniroma2.it/psblas/&quot; rel=&quot;nofollow&quot;&gt;PSBLAS&lt;/a&gt;, which takes full advantage of the modern Fortran features such as inheritance and polymorphism. There's also a paper &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=2331131&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; where they compared the performance of that version with their older versions, which stuck to F90/F95. Surprisingly enough, the speed difference was negligible overall.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-04-11T06:48:58.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6818" PostTypeId="2" ParentId="6812" CreationDate="2013-04-11T13:45:54.083" Score="2" Body="&lt;p&gt;As mentioned by icurays1 profiling is best. You can also slightly simplify the above ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;use utils&#xA;...&#xA;call tic()&#xA;   ! Section to be timed&#xA;call toc()&#xA;...&#xA;call tic()&#xA;   ! Section to be timed&#xA;call toc()&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the utils module contains ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real(8) :: t1,t2&#xA;...&#xA;subroutine tic()&#xA;  implicit none&#xA;  call cpu_time(t1)&#xA;end subroutine tic&#xA;&#xA;subroutine toc()&#xA;  implicit none&#xA;  call cpu_time(t2)&#xA;  ! if (rank==0) print*,&quot;Time Taken --&amp;gt;&quot;, real(t2-t1)&#xA;  print*,&quot;Time Taken --&amp;gt;&quot;, real(t2-t1)&#xA;end subroutine toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you have many such sections then pass a string, e.g., &quot;section_id&quot; in toc so that it prints the id/name along with the timing.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2013-04-11T13:45:54.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6819" PostTypeId="1" CreationDate="2013-04-11T13:56:36.620" Score="1" ViewCount="93" Body="&lt;p&gt;I'm doing some machine learning where I have lots of data and through optimization I'm trying to learn the weights for the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to check that my learning actually works correctly. For that I can create a set of known weights. My question is from these weights, how can I create synthetic data? On this data I can perform the training and check if the weights in the end are the same as the weights I created.&lt;/p&gt;&#xA;" OwnerUserId="4070" LastActivityDate="2013-04-11T13:56:36.620" Title="How to create synthetic data from known weights" Tags="|machine-learning|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6820" PostTypeId="2" ParentId="5376" CreationDate="2013-04-11T14:54:22.223" Score="3" Body="&lt;p&gt;Further research eventually yielded the &lt;a href=&quot;http://www.particleincell.com/2011/vxb-rotation/&quot; rel=&quot;nofollow&quot;&gt;Boris method&lt;/a&gt;. For a constant B field it produces stable circular orbits.&lt;/p&gt;&#xA;" OwnerUserId="3797" LastActivityDate="2013-04-11T14:54:22.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6822" PostTypeId="1" CreationDate="2013-04-11T20:03:22.060" Score="5" ViewCount="335" Body="&lt;p&gt;I am running my Fortran 95 numerical codes, whose structure is most times easily parallelized by little more than simply adding the corresponding OpenMP instruction before some critical DO sentences, in a dual-core multi-thread CPU Intel CORE i3. Ubuntu &quot;sees&quot; four CPUs (obviously since there are 2 cores, each of them emulating two cores each), and I have read that the &quot;multithread&quot; capability means that, under certain conditions, each of the CPUs may behave close to two separate CPUs itself, but not in general. I have heard somewhere that, in some cases, the four parallel threads may perform worse than two in this kind of machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the kind of coding or tasks in general with which my machine works nearly as a real 4-core, and in which ones it may not be a good idea to emulate that (thus setting for instance the OMP_NUM_THREADS variable to 2 instead of 4)?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: In other words and, according to &lt;a href=&quot;https://stackoverflow.com/questions/8809752/multithreading-on-dual-core-machine/8809791#8809791&quot;&gt;this nice answer in another post&lt;/a&gt;, what is the optimal number of &lt;em&gt;hardware threads&lt;/em&gt; for my machine in relation to the kind of numerical task involved, provided that, although it is a dual-core cpu, it has that &quot;multithread&quot; ability?&lt;/p&gt;&#xA;" OwnerDisplayName="user4015" LastEditorUserId="-1" LastEditorDisplayName="user4015" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-05-25T12:18:38.780" Title="Under which conditions is better OMP_NUM_THREADS = 4 or 2 for multithread two-core CPU?" Tags="|parallel-computing|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6823" PostTypeId="2" ParentId="6822" CreationDate="2013-04-11T20:18:42.593" Score="4" Body="&lt;p&gt;Hyperthreading utilizes the duplication of cpu parts such as registers to allow the cpu to &quot;look&quot; like twice as many cores to the OS and other hardware.  The actual execution units such as the FPU are not duplicated.  This means that if your program is CPU limited, ie, with 2 threads the FPU is already at full capacity, then you won't see any improvement.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, you can only expect either configuration to improve performance by a few tens of percent.  Intel claims an approximately 30% increase in some cases, however even they admit it is application dependent.  The only real way to know how it will affect your code is to profile it.  Take two identical runs of non-trivial size and time them.  I personally work with a code that slight changes in numerical parameters greatly changes the way the code scales with the number of threads.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-04-12T13:02:14.750" LastActivityDate="2013-04-12T13:02:14.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6824" PostTypeId="1" CreationDate="2013-04-11T21:52:12.447" Score="3" ViewCount="754" Body="&lt;p&gt;&lt;strong&gt;Simulation type:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm running a simulation with the OOMMF micromagnetics package&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://math.nist.gov/oommf/&quot; rel=&quot;nofollow&quot;&gt;http://math.nist.gov/oommf/&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where are magnet is represented by a &lt;strong&gt;mesh of 3 million cells&lt;/strong&gt;, it gets excited by a magnetic pulse, the single cells oscillate and data output for every cell is x, y, z direction and absolute magnetic field value of every component over time, so a mini-magnet. Now I want to do a FFT on m_z(t) &lt;strong&gt;for every cell&lt;/strong&gt; and later visualize this magnet over time while cells with m_z(t) showing common oscillation modes/eigenfrequencies get same color. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Data output format:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it is &quot;data binary 4&quot; and as said x, y, z and absolute field value for every component is given for all time steps (below 300-500 time steops)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The manual says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;The binary representations are IEEE &#xA;  floating point in network byte order (MSB)&lt;/strong&gt;. To&#xA;  insure that the byte order is correct, and to provide a partial check that the le hasn't been&#xA;  sent through a non 8-bit clean channel, the first datum is a predened value: 1234567.0&#xA;  (Hex: 49 96 B4 38) for 4-byte mode, and 123456789012345.0 (Hex: 42 DC 12 21 83 77 DE&#xA;  40) for 8-byte mode. The data immediately follow the check value.&#xA;  The structure of the data depends on whether the \meshtype&quot; declared in the header is&#xA;  \irregular&quot; or \rectangular&quot;. &lt;strong&gt;For irregular meshes, each data element is a 6-tuple&lt;/strong&gt;, consisting&#xA;  of the x, y and z components of the node position, followed by the x, y and z components of&#xA;  the field at that position. Ordering among the nodes is not relevant. The number of nodes&#xA;  is specied in the \pointcount&quot; line in the segment header.&#xA;  For &lt;strong&gt;rectangular meshes, data input is field values only, in x, y, z component triples.&lt;/strong&gt;&#xA;  These are ordered with the x index incremented first, then the y index, and the z index last. This is nominally &lt;strong&gt;Fortran order&lt;/strong&gt;, and is adopted here because commonly x will be the longest&#xA;  dimension, and z the shortest, so this order is &lt;strong&gt;more memory-access ecient than the normal&#xA;  C array indexing of z, y, x.&lt;/strong&gt; The size of each dimension is specied in the \xnodes, ynodes,&#xA;  znodes&quot; lines in the segment header.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I use a rectangluar mesh here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FFT can be done using something like MATHEMATICA or MATLAB for a single cell and (m_z/ number, time/number) data list .dat file, but a data binary 4 file for a single time step for 3 million cells is up to 20 MB big, workstation with 30 GB RAM and 12 Cores available. The manual above mentions FORTRAN (to my knowledge not so easy to learn language although very fast). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the way to achieve the results I'm looking for and described above? From my point of view I face the problem to do the FFT in reasonable time with a script process for around 3 million cells and then transforming that data into a spectrogram (to see what different eigenfrequencies and higher harmonics exist, also this can be done by using the average m_z (summed over all single cells) and simply FFT it) but also have a output format of that data binary 4 format after FFT that can be used to feed a visualization software that can read such mesh data and color code it, show single 2D slices of the 3D object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read that MATHEMATICA can deal with binary 4 and offers high performance, but as this is completely new land to me, I first want to ask given above constraints and wishes, what is the best way and software/programming language to achieve this in &lt;strong&gt;reasonable time (few weeks)&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="3873" LastEditorUserId="3873" LastEditDate="2013-04-11T21:57:55.887" LastActivityDate="2013-04-12T00:53:31.613" Title="How to do local FFT on huge 3D vector data cell mesh and visualize it spatially?" Tags="|matlab|visualization|fourier-analysis|data-analysis|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6825" PostTypeId="2" ParentId="1376" CreationDate="2013-04-12T00:10:11.790" Score="5" Body="&lt;p&gt;Another option is to use PiCloud (&lt;a href=&quot;http://www.picloud.com&quot;&gt;http://www.picloud.com&lt;/a&gt;). PiCloud runs on top of Amazon Web Services. You get 20 free (c1) core hours, it's really easy to offload long-running computation, and you don't have to manage any servers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your C# program works with mono, you can use the public mono environment, &quot;/picloud/mono&quot;.&lt;/p&gt;&#xA;" OwnerUserId="4073" LastActivityDate="2013-04-12T00:10:11.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6826" PostTypeId="2" ParentId="6824" CreationDate="2013-04-12T00:53:31.613" Score="2" Body="&lt;p&gt;20 MB data files are really not very large any more, on machines like yours. I would expect Mathematica to be able to read it in under a few seconds, and to be able to do an FFT in a few seconds to maybe a minute. You could try this by simply starting Mathematica, creating a vector with 3 million randomly chosen entries, and asking Mathematica to do the FFT. If this happens within an acceptable time scale, then it's worth writing an interface that can read the data you have. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, what I'm trying to say is that you may be unnecessarily scared by the size of your data. Any reasonable programming environment that offers you to manage data vectors and do an FFT should suffice. This includes Mathematica, Maple, Matlab, or C/C++ programs linked against the FFTW package. I would expect any of them to give you results in a modest amount of runtime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica, Matlab and Maple will have the additional advantage that they have many ways of visualizing data.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-12T00:53:31.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6827" PostTypeId="2" ParentId="6822" CreationDate="2013-04-12T00:57:01.707" Score="5" Body="&lt;p&gt;The community has not really come to much of a conclusion on when this may or may not be worth it. I do compile with 'make -j8' on my 4-core laptop and it is faster than using 'make -j4' but simple multithreaded programs aren't much faster when run with 8 threads than with 4. The only real way to find out is to &lt;em&gt;try&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-12T00:57:01.707" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6828" PostTypeId="1" AcceptedAnswerId="6832" CreationDate="2013-04-12T07:24:23.277" Score="5" ViewCount="691" Body="&lt;p&gt;Consider a continuous Markov chain $X=(X_t)$ on a finite state space and let $Q$ be the (given) transition rate matrix. This matrix is very sparse, with non-zero values on 3 diagonals only (so from each state, there can be transitions to 2 other states only).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $P_t$ be the transition probability matrix, so $P_t(j,k) =$ Prob$(X_t=k|X_0=j)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what is the best way to quickly compute the $j$th row of $P_1$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the Kolmogorov forward equations gives $P_t=e^{Qt}$, so one method is to perform this computation explicitly in matlab: expm(Q). But I'm thinking that there is perhaps a better way, particularly given the structure of $Q$ and since I'm only interested in one row of $P_1$. The actual instance of the problem I'm solving is small (120 states, say), but I would like the computation to be very fast.&lt;/p&gt;&#xA;" OwnerUserId="4074" LastActivityDate="2013-04-13T17:11:26.683" Title="Efficient computation of Markov chain transition probability matrix" Tags="|linear-algebra|matrix|probability|banded-matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6829" PostTypeId="1" CreationDate="2013-04-12T15:33:56.237" Score="1" ViewCount="3587" Body="&lt;p&gt;I've coupled a COMSOL model for fluid dynamics with a very simple pde that model the transport of humidity in air. &#xA;When I solve it for the stationary case, the solution converge easily, but when I try to solve the time dependent problem, the solution can't reach convergence. &#xA;This seems very strange to me, the fluid is laminar and incompressible, the mesh is well designed, the initial values are coherent, all the conditions are smooth. &#xA;Is it even possible that COMSOL isn't able to find a solution for the time dependent problem, while he found very easily the time independent one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="3860" LastActivityDate="2015-10-29T16:30:28.033" Title="Stationary solution converge but time dependent doesn't" Tags="|convergence|comsol|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6830" PostTypeId="2" ParentId="6763" CreationDate="2013-04-12T15:41:27.820" Score="4" Body="&lt;p&gt;I came across this issue recently, and discovered that if you first add the constraints to an IloRangeArray, and then add that object to the model, you'll see a significant speed-up.  Removing constraints is also faster if they are contained in a container object like an IloRangeArray. &lt;/p&gt;&#xA;" OwnerUserId="1635" LastActivityDate="2013-04-12T15:41:27.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6831" PostTypeId="2" ParentId="5477" CreationDate="2013-04-12T16:54:46.923" Score="2" Body="&lt;p&gt;Since the question has a Fortran tag, here is Wolfgang's solution in Fortran:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;implicit none&#xA;integer, parameter :: dp = kind(0.d0)&#xA;complex(dp), parameter :: i_ = (0, 1)&#xA;&#xA;real(dp) :: alpha = 1&#xA;complex(dp) :: s = 0&#xA;integer :: k&#xA;do k = 1, 10000000&#xA;    s = s + ((i_+1)/sqrt(2._dp))**k * k**(-alpha)&#xA;    if (modulo(k, 1000000) == 0) print *, k, s&#xA;end do&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ gfortran -O3 x.f90 &#xA;$ time ./a.out &#xA;     1000000 ( 0.26740049836809593     ,  1.1780960379893362     )&#xA;     2000000 ( 0.26740024836936921     ,  1.1780966415427796     )&#xA;     3000000 ( 0.26740016503629038     ,  1.1780968427272547     )&#xA;     4000000 ( 0.26740012336968905     ,  1.1780969433194985     )&#xA;     5000000 ( 0.26740009836973161     ,  1.1780970036749265     )&#xA;     6000000 ( 0.26740008170307927     ,  1.1780970439117682     )&#xA;     7000000 ( 0.26740006979833991     ,  1.1780970726523770     )&#xA;     8000000 ( 0.26740006086978074     ,  1.1780970942078899     )&#xA;     9000000 ( 0.26740005392532235     ,  1.1780971109732488     )&#xA;    10000000 ( 0.26740004836978204     ,  1.1780971243856078     )&#xA;&#xA;real    0m1.477s&#xA;user    0m1.472s&#xA;sys 0m0.000s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For comparison, on my computer the C++ code takes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ g++ -O3 x.cc&#xA;$ time ./a.out &#xA;1000000 (0.2674004983680959,1.178096037989336)&#xA;2000000 (0.2674002483693692,1.17809664154278)&#xA;3000000 (0.2674001650362904,1.178096842727255)&#xA;4000000 (0.267400123369689,1.178096943319499)&#xA;5000000 (0.2674000983697316,1.178097003674927)&#xA;6000000 (0.2674000817030793,1.178097043911768)&#xA;7000000 (0.2674000697983399,1.178097072652377)&#xA;8000000 (0.2674000608697807,1.17809709420789)&#xA;9000000 (0.2674000539253223,1.178097110973249)&#xA;&#xA;real    0m2.123s&#xA;user    0m2.116s&#xA;sys 0m0.004s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I use gcc 4.6.3.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2013-04-12T16:54:46.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6832" PostTypeId="2" ParentId="6828" CreationDate="2013-04-12T17:58:24.897" Score="3" Body="&lt;p&gt;Computing an approximation of $P_1=e^Q$ is typically done by some kind of polynomial expansion of the matrix exponential. One doesn't take the Taylor expansion (it converges too slowly) but for the sake of the argument, let me assume that you do and that you approximate&#xA;$$&#xA;  P_1 = e^Q \approx \sum_{k=0}^N \frac1{k!} Q^k&#xA;  = I+Q+\frac 1{2!} Q^2 + \frac 1{3!}Q^3 + \cdots&#xA;  = I+Q(I+\frac 12 Q(I+\frac 13 Q(\cdots)))&#xA;$$&#xA;Then remember that you get the $j$th row of $P_1$ by forming the product $e_j^T P_1$ where $e_j$ is the $j$th unit vector $(0, \ldots 0, 1, 0, \ldots, 0)^T$ with the one in $j$th place of the vector. You can use this in the formula above to compute&#xA;$$&#xA;  e_j^T P_1 \approx&#xA;  = e_j^T+(e_j^T Q)(I+\frac 12 Q(I+\frac 13 Q(\cdots)))&#xA;$$&#xA;If you multiply this product out from left to right, you will find that all you have to ever do is form products of a vector (transposed) with a matrix. In other words, you never actually need any matrix-matrix products which would be expensive in your case because your matrix $Q$ is sparse but the powers of $Q$ will become less and less sparse. In the end, using this approach, all you need is $O(N)$ vector-matrix products to take the expansion to $N$th order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I said, in practice, one would not take a Taylor expansion but other polynomial expansions of the matrix exponential. However, the general approach will remain the same: reduce it all to a sequence of vector-matrix products.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-04-12T22:08:22.990" LastActivityDate="2013-04-12T22:08:22.990" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6833" PostTypeId="2" ParentId="5477" CreationDate="2013-04-12T19:51:20.203" Score="1" Body="&lt;p&gt;&lt;strike&gt;The series is divergent for $\alpha &amp;lt; 1$.  Its convergence is suspicious using the ratio test.&lt;/strike&gt;  For $\alpha = 1$, the sum is $-\log(1 - \sqrt{i})$, which is 0.2673999983697852 + 1.1780972450961725$i$, close to what is given in other answers.  Results from Mathematica.  By the way, $\frac{1+i}{\sqrt{2}}$ is $\sqrt{i}$, which also explains that its powers attain only 8 distinct values.&lt;/p&gt;&#xA;" OwnerDisplayName="user335" LastEditorDisplayName="user335" LastEditDate="2013-05-19T18:31:37.503" LastActivityDate="2013-05-19T18:31:37.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6834" PostTypeId="1" CreationDate="2013-04-13T01:25:05.177" Score="1" ViewCount="77" Body="&lt;p&gt;I was reading this&lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;cad=rja&amp;amp;ved=0CDUQFjAA&amp;amp;url=http://jmlr.csail.mit.edu/proceedings/papers/v22/zhang12d/zhang12d.pdf&amp;amp;ei=UpBpUfSnKZfJ4AODr4BA&amp;amp;usg=AFQjCNE4mUPPsSiyQBW5FY_IyeRT7Im9Og&amp;amp;sig2=pY3PIaolEYAfVTmLevmCuA&amp;amp;bvm=bv.45175338,d.dmg&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; related to kernel SVM. It states &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Support Vector Machine (SVM) (Cortes and Vap- nik, 1995) as the state-of-the-art classification algo- rithm has been widely applied in various scientific do- mains. The use of kernels allows the input samples to be mapped to a Reproducing Kernel Hilbert S- pace (RKHS), which is crucial to solving linearly non- separable problems. While kernel SVMs deliver the state-of-the-art results, the need to manipulate the k- ernel matrix imposes significant computational bottle- neck, making it difficult to scale up on large data.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't get what they mean by manipulate the kernel matrix. I mean lets say I am using the RBF kernel. Then my kernel matrix will have elements of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$exp^{\sigma {||x_i-x_j||}_2}{}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which I can calculate once and then have it there. So what is meant by kernel manipulation&lt;/p&gt;&#xA;" OwnerUserId="3871" LastEditorUserId="3871" LastEditDate="2013-04-13T17:06:05.957" LastActivityDate="2013-04-13T17:06:05.957" Title="Confusion related to svm optimization" Tags="|machine-learning|support-vector-machines|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6835" PostTypeId="2" ParentId="6814" CreationDate="2013-04-13T03:19:35.173" Score="2" Body="&lt;h3&gt;Do I have to form $A_H$ explicitly?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It is not strictly necessary, but if you define it implicitly, then the complexity grows from $O(n)$ to $O(n \log n)$, making the algorithm significantly less efficient. It also becomes inconvenient to define properly-scaled pointwise smoothers, which costs another factor.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Is forming $I_H^h = p(A) \mathscr I_H^h$ and ultimately $A_H$ prohibitively expensive?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Forming $I_H^h$ is usually inexpensive because $p(A)$ is almost always chosen to be damped Jacobi (a degree 1 polynomial).  Even so, construction of $A_H$ is usually the most expensvie part of multigrid setup.  If $I_H^h$ were chosen to have more overlap, for example by smoothing the aggregates multiple times, grid complexity would be much higher.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-04-13T03:19:35.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6836" PostTypeId="2" ParentId="6834" CreationDate="2013-04-13T13:36:54.540" Score="2" Body="&lt;p&gt;They are simply referring to the fact that the kernel matrix itself is a central quantity in the algorithm. The problem is that it is $O(n^2)$ in size, where $n$ is the number of points being examined. So the storage and computational requirements surrounding the kernel matrix rapidly become impractical as $n$ gets large.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2013-04-13T13:36:54.540" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6837" PostTypeId="2" ParentId="6828" CreationDate="2013-04-13T17:11:26.683" Score="2" Body="&lt;p&gt;A software package mentioned in Moler and van Loan's &quot;twenty-five years after&quot; portion of their paper (Sec. 13, see link in Christian Clason's comment) is Roger Sidje's &lt;a href=&quot;http://www.maths.uq.edu.au/expokit/&quot; rel=&quot;nofollow&quot;&gt;Expokit&lt;/a&gt;, with &lt;a href=&quot;http://www.maths.uq.edu.au/expokit/paper.pdf&quot; rel=&quot;nofollow&quot;&gt;an accompanying paper&lt;/a&gt; that explains the approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large sparse matrices a Taylor-like polynomial is applied to Krylov subspaces, obtaining the advantages of matrix-vector operations with sharper error bounds than a pure Taylor series approximation.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-04-13T17:11:26.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6838" PostTypeId="2" ParentId="769" CreationDate="2013-04-13T20:12:35.083" Score="2" Body="&lt;p&gt;Resilience will be the limiting factor to exascale.  It is going to be relatively straightforward to build an exascale system for less than $1B that uses less than 50MW if the machine is allowed to crash every 15 minutes or compute the wrong answer most of the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Silent memory errors are already occurring and engineering system software and algorithms to compute accurately in spite of them is going to cost a lot of money and/or power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statement that money and power are the bottlenecks isn't wrong, but those are indirect problems.  We need money and power to make the machine reliable; it's relatively easy to reduce the costs (power and financial) if one reduces the reliability of the system.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-04-13T20:12:35.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6839" PostTypeId="2" ParentId="1" CreationDate="2013-04-13T20:22:34.623" Score="3" Body="&lt;p&gt;There are many ways to answer this, but CCSD is only complete through third-order in many-body perturbation theoretic terms, while CCSD(T) is complete through fourth-order.  It was the desire to have a fourth-order complete method that led Bartlett and coworkers to develop CCSD[T].  The CCSD(T) method includes a fifth-order term (at least in the RHF case), which was later &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0009261497011445&quot; rel=&quot;nofollow&quot;&gt;shown&lt;/a&gt; to be fourth-order in the ROHF case by Stanton, which helps to explain why the inclusion of that term has such an impact on the accuracy of CCSD(T) relative to CCSD[T].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.cfour.de&quot; rel=&quot;nofollow&quot;&gt;CFOUR Bibliography page&lt;/a&gt; has a list of most (all?) the seminal papers on coupled-cluster theory, which might be of interest.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2016-10-06T13:10:39.160" LastActivityDate="2016-10-06T13:10:39.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6840" PostTypeId="2" ParentId="2" CreationDate="2013-04-13T20:32:39.427" Score="1" Body="&lt;p&gt;Hartree Fock will be unreliable for multiconfigurational systems e.g. involving transition metals or when dispersion interactions are significant (as noted by Alex1167623).&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-04-13T20:32:39.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6841" PostTypeId="2" ParentId="5" CreationDate="2013-04-13T20:37:23.630" Score="3" Body="&lt;p&gt;I recommend reading Thom Dunning Jr. “Gaussian basis sets for use in correlated molecular calculations. I. The atoms boron through neon and hydrogen.” &lt;a href=&quot;https://doi.org/10.1063/1.456153&quot; rel=&quot;nofollow noreferrer&quot;&gt;J. Chem. Phys. 90, 1007 (1989)&lt;/a&gt; by for an answer to this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The abstract says the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This leads to the concept of correlation consistent basis sets, i.e., sets which include all functions in a given group as well as all functions in any higher groups.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This refers to the telescoping nature of the angular momentum levels used,  e.g. 5s4p3d2f1g.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2020-04-12T14:10:40.057" LastActivityDate="2020-04-12T14:10:40.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6842" PostTypeId="1" AcceptedAnswerId="6843" CreationDate="2013-04-13T23:38:01.537" Score="4" ViewCount="1056" Body="&lt;p&gt;I'm trying to make highly efficient CFD programming complex for solving combustion problems. I've finished writing core which realises mathematical model, and now I'm concerned about code performance. Combustion problems are very computationally expensive, with no doubt I'll have to make parallelization of my code, but for now I want to make highly efficient serial code, to use it as a base for parallel program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance there is simplified version of what is going on in program:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the main program where time loop is located. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PROGRAM Flame&#xA;USE Geometry&#xA;CALL ALLOCATION  &amp;lt;- Procedure there memory is allocated for X array&#xA;DO while ...&#xA;Time=Time + TH&#xA;CALL CALCULATIONS&#xA;END DO &#xA;END PROGRAM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Modules with all global variables declarations&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MODULE Geometry&#xA;double precision , allocatable :: X(:) &amp;lt;-- one dimensional array for computational grid.&#xA;integer, dimension(NTR) :: KDIF &amp;lt;-- has a mean of pointers on some variable in cell (pressure for example)&#xA;END MODULE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Subroutines for calculations and data processing.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SUBROUTINE CALCULATIONS&#xA;USE Geometry&#xA;IMPLICIT DOUBLE PRECISION (A-H,O-Z)&#xA;IJK = 0&#xA;DO I=1,NX &amp;lt;- number of cells in one dimension (such cycles for all dimensions) &#xA;  DO I1=1,NTR&#xA;    X(IJK+KDIF(I1)) = ... some floating-point operations &amp;lt;-- Possible hot spot&#xA;    I1 = I1+1&#xA;  END DO&#xA;  X(IJK+8)=X(IJK+3)-A/X(IJ+5)-(X(IJ+10)-B)*C/(D*X(IJ+5)) (for example)&#xA;IJK = IJK + NKX  &amp;lt;-- place between two cells in &#xA;END DO&#xA;END SUBROUTINE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is much more subroutines and modules in program, but they all have structure similar to that I gave as an example. I've tried to profile my program with VTune Amplifier, and located some hot spots in places like I pointed in code. In Assembly of hot spots like this most time consuming operations are generally movesd qword; move ebx, dword; etc. I'm not familiar with assembly codes but I think it is related to memory operations. Actually I have no idea how to fix that hot spots.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there are some questions that I need to clarify. Is that style of coding modern? I mean maybe I should use another techniques for such programs, like pointers, FORALL and WHERE operations, which can  be faster and more memory efficient, more appropriate for CFD problems? Is such program structure efficient (modules for data, subroutines, allocatable arrays, one-dimensional array even for multidimansional problems)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for your attention, answers and advices. &lt;/p&gt;&#xA;" OwnerUserId="4084" LastActivityDate="2013-04-15T21:38:28.573" Title="Effecient CFD programming techniques" Tags="|fluid-dynamics|performance|fortran|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6843" PostTypeId="2" ParentId="6842" CreationDate="2013-04-14T02:21:13.633" Score="5" Body="&lt;p&gt;Optimizing code is a very broad problem.  Before you get too far into the details of an implementation, you should make sure your models and algorithms are doing what you want them to as efficiently as possible.  A good algorithm is better than a good implementation and premature optimization is the root of all evil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That much being said, here are some comments on the code you've given:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Don't use implicit typing, ever; it turns typos into bugs.&lt;/li&gt;&#xA;&lt;li&gt;There are mixed opinions on forall, where, etc.  &lt;a href=&quot;https://stackoverflow.com/questions/4122099/do-fortran-95-constructs-such-as-where-forall-and-spread-generally-result-in-fa&quot;&gt;This discussion&lt;/a&gt; is particularly interesting.  Summary: forall, where, etc don't provide much, if any, speed, but they can be nice for representing algorithms.&lt;/li&gt;&#xA;&lt;li&gt;Don't pass array slices as arguments; they can cause unintended temporary allocations and copies.&lt;/li&gt;&#xA;&lt;li&gt;allocatables are fine; only switch to pointers if you need more functionality&lt;/li&gt;&#xA;&lt;li&gt;Modules are fine.  I personally like to reserve modules for parameters or variables that are nearly parameters (and routines), and move data around explicitly.  Fortran is pass by reference, so large array arguments are no more expensive than scalers.  I've seen successful codes do it either way.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="521" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-04-15T21:38:28.573" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="6844" PostTypeId="2" ParentId="6766" CreationDate="2013-04-14T04:04:34.193" Score="2" Body="&lt;p&gt;This is impossible unless the graph is somehow labeled.  For example, consider the complete graph $G = K_{2n}$ consisting of $2n$ vertices and edges between every pair of vertices.  All balanced partitions look exactly the same in terms of this graph, so it is not possible to pick one out uniquely.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-04-14T04:04:34.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6845" PostTypeId="1" AcceptedAnswerId="6850" CreationDate="2013-04-14T09:01:45.360" Score="16" ViewCount="4012" Body="&lt;p&gt;I'm developing some larger code to perform eigenvalue computations of huge sparse matrices, in the context of computational physics. I test my routines against the simple harmonic oscillator in one dimension, since the eigenvalues are well known analytically. Doing so and comparing my own routines to SciPy's inbuilt solvers, I have come across the oddity displayed in the plot below. Here you can see the first 100 numerically computed eigenvalues $\lambda_{num}$ and analytical eigenvalues $\lambda_{ana}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Around eigenvalue number 40, the numercial results start to diverge from the analytical ones. This doesn't surprise me (I won't go into why here, unless it comes up in the discussion). However, what is &lt;strong&gt;surprising&lt;/strong&gt; to me is that &lt;strong&gt;eigsh()&lt;/strong&gt; produces &lt;strong&gt;degenerate eigenvalues&lt;/strong&gt; (around eigenvalue number 80). Why does eigsh() behave like that for even such a small number of eigenvalues?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uxb4i.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;from scipy.sparse.linalg import eigsh&#xA;import myFunctions as myFunc&#xA;import matplotlib.pyplot as plt&#xA;&#xA;#discretize x-axis&#xA;N = 100&#xA;xmin = -10.&#xA;xmax = 10.&#xA;accuracy = 1e-5&#xA;#stepsize&#xA;h = (xmax - xmin) / (N + 1.)&#xA;#exclude first and last points since we force wave function to be zero there&#xA;x = np.linspace(-10. + h,10. - h,N)&#xA;#create potential&#xA;V = x**2&#xA;&#xA;def fivePoint(N,h,V):&#xA;    C0 = (np.ones(N))*30. / (12. * h * h) + V&#xA;    C1 = (np.ones(N)) * (-16.) / (12. * h * h)&#xA;    C2 = (np.ones(N)) / (12. * h * h)&#xA;    H = sp.spdiags([C2, C1, C0, C1, C2],[-2, -1, 0, 1, 2],N,N)&#xA;    return H&#xA;&#xA;H = myFunc.fivePoint(N,h,V)&#xA;eigval,eigvec = eigsh(H, k=N-1, which='SM', tol=accuracy)&#xA;&#xA;#comparison analytical and numerical eigenvalues&#xA;xAxes = np.linspace(0,len(eigval)-1,len(eigval))&#xA;analyticalEigval = 2. * (xAxes + 0.5)&#xA;plt.figure()&#xA;plt.plot(xAxes,eigval, '+', label=r&quot;$\lambda_{num}$&quot;)&#xA;plt.plot(xAxes,analyticalEigval, label=r&quot;$\lambda_{ana}$&quot;)&#xA;plt.xlabel(&quot;Number of Eigenvalue&quot;)&#xA;plt.ylabel(&quot;Eigenvalue&quot;)&#xA;plt.legend(loc=4)&#xA;plt.title(&quot;eigsh()-method: Comparison of $\lambda_{num}$ and $\lambda_{ana}$&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="1804" LastEditDate="2013-04-19T14:50:59.947" LastActivityDate="2013-04-19T14:50:59.947" Title="Why does SciPy eigsh() produce erroneous eigenvalues in case of harmonic oscillator?" Tags="|python|sparse-matrix|numpy|eigenvalues|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6846" PostTypeId="1" AcceptedAnswerId="6848" CreationDate="2013-04-14T16:02:08.107" Score="10" ViewCount="4097" Body="&lt;p&gt;By &quot;the simplest&quot; I mean the simplest to learn and implement from scratch. I hope my question can more or less be answered.&lt;/p&gt;&#xA;" OwnerUserId="3637" LastActivityDate="2014-06-09T09:53:19.463" Title="What is the simplest method for solving incompressible Navier-Stokes equations?" Tags="|fluid-dynamics|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6847" PostTypeId="2" ParentId="2384" CreationDate="2013-04-14T16:04:01.517" Score="6" Body="&lt;p&gt;For the most part I ditto Jed's answer. However, there is a different way out: Given the size of normal floating point numbers, you can store every number in a 4000-or-so bit fixed point number. So if you do a reduction on floating point numbers thus embedded, you get an exact calculation, no matter the associativity. (Sorry, I don't have the reference to who came up with this idea.)&lt;/p&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2013-04-14T16:04:01.517" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6848" PostTypeId="2" ParentId="6846" CreationDate="2013-04-14T16:18:46.140" Score="11" Body="&lt;p&gt;In two dimensions, the velocity-vorticity formulation is the simplest to implement because the variables are collocated, but boundary conditions can be complicated and it's a less direct statement of the problem. For primitive variable formulations, the staggered grid finite difference method of &lt;a href=&quot;http://link.aip.org/link/doi/10.1063/1.1761178&quot;&gt;Harlow and Welch (1965)&lt;/a&gt; is a great place to start.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-04-14T16:18:46.140" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6850" PostTypeId="2" ParentId="6845" CreationDate="2013-04-14T18:27:22.187" Score="13" Body="&lt;p&gt;The degeneracy of some eigenvalues looks to me like the hallmark of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Lanczos_algorithm#Numerical_stability&quot;&gt;breakdown of the Lanczos algorithm&lt;/a&gt;. The Lanczos algorithm is one of the more commonly used methods to approximate the eigenvalues and eigenvectors of Hermitian matrices; it's what scipy.eigsh() uses, through a call to the &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot;&gt;ARPACK library&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In exact arithmetic, the Lanczos algorithm produces a set of orthogonal vectors, but in floating point arithmetic these can fail to be orthogonal and even become linearly dependent. The really annoying thing is that this loss of orthogonality happens precisely when one of the approximate eigenvalues has converged to one of the real eigenvalues -- the algorithm sabotages itself, so to speak. The result is that you'll get some spurious pairs of nearby eigenvalues. There are various fixes for this, for example using Gram-Schmidt to force any converged eigenvectors to be orthogonal at every step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, no method is perfect, &lt;em&gt;especially if you're trying to compute the entire spectrum of your matrix&lt;/em&gt;. So if you're trying to get the 50 smallest eigenvalues, you may be better off approximating the wave function by a vector with 100 elements and only asking &lt;code&gt;eigsh()&lt;/code&gt; for the first 50 energy levels, rather than using a vector with 50 points and asking for all of the eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to read more, look at Yousef Saad's &lt;a href=&quot;http://books.google.com/books?isbn=1611970725&quot;&gt;Numerical Methods for Large Eigenvalue Problems&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-04-14T18:27:22.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6851" PostTypeId="2" ParentId="2214" CreationDate="2013-04-14T20:22:53.407" Score="0" Body="&lt;p&gt;Is this not just the generalization of the packed symmetric matrix indexing problem?  The solution there is offset(i,j) = i*(i+1)/2+j, is it not?  Can't you double down on this and index a doubly-symmetric 4D array?  The implementation requiring branching seems unnecessary.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-04-14T20:22:53.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6853" PostTypeId="1" AcceptedAnswerId="6855" CreationDate="2013-04-14T21:45:31.610" Score="20" ViewCount="13776" Body="&lt;p&gt;I have run the molecular dynamics (MD) code &lt;a href=&quot;http://www.gromacs.org/&quot;&gt;GROMACS&lt;/a&gt; on a Ubuntu Linux cluster consisting of nodes containing 24 Intel Xeon CPUs.  My particular point of interest turns out to be somewhat sensitive to floating point arithmetic precision, so I have had to run GROMACS in double precision rather than single precision -- despite double precision's higher computational cost.  So on the cluster, I have compiled GROMACS in double precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am considering purchasing some GPUs, since there may be a speed-up relative (&quot;GPU acceleration&quot;) to CPUs.  However, I need a GPU that will allow me to do double precision arithmetic.  Do you know if such hardware is commercially available?  A &lt;a href=&quot;http://lists.gromacs.org/pipermail/gmx-users/2013-April/080208.html&quot;&gt;recent post on the GROMACS mailing list&lt;/a&gt; suggests that double precision GPUs are not commercially available:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The hardware does not support it [double precision arithmetic] yet&#xA;  AFAIK.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://en.wikipedia.org/wiki/GPGPU#Programmability&quot;&gt;Wikipedia page&lt;/a&gt; seems to suggest that double precision GPUs are uncommon since they may be inefficient:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The implementations of floating point on Nvidia GPUs are mostly IEEE&#xA;  compliant; however, this is not true across all vendors. This has&#xA;  implications for correctness which are considered important to some&#xA;  scientific applications. While 64-bit floating point values (double&#xA;  precision float) are commonly available on CPUs, these are not&#xA;  universally supported on GPUs; some GPU architectures sacrifice IEEE&#xA;  compliance while others lack double-precision altogether. There have&#xA;  been efforts to emulate double-precision floating point values on&#xA;  GPUs; however, the speed tradeoff negates any benefit to offloading&#xA;  the computation onto the GPU in the first place.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://www.nvidia.com/object/tesla-servers.html&quot;&gt;NVIDIA Tesla page&lt;/a&gt;, in referencing &quot;Peak double precision floating point performance&quot; in the chart, seems to suggest that double precision calculations &lt;strong&gt;can&lt;/strong&gt;, in fact, be done on their GPUs (albeit at higher computational cost).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what should I believe?  Do you have any experience with this issue?&lt;/p&gt;&#xA;" OwnerUserId="1454" LastActivityDate="2014-10-11T22:30:54.213" Title="Do currently available GPUs support double precision floating point arithmetic?" Tags="|hpc|floating-point|molecular-dynamics|gpu|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6854" PostTypeId="1" CreationDate="2013-04-14T22:03:25.193" Score="4" ViewCount="441" Body="&lt;p&gt;I found this method in &lt;a href=&quot;http://books.google.com/books?id=CNi6jpmiwtsC&amp;amp;pg=PA88&amp;amp;lpg=PA88&amp;amp;dq=mehrstellenverfahren%20poisson&amp;amp;source=bl&amp;amp;ots=lZubodtr3y&amp;amp;sig=s2F8uN6J-VbyEv9P3Nc6yOMHQjs&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=xx1rUfaVINK-4AOl94G4Dg&amp;amp;ved=0CEUQ6AEwAw#v=onepage&amp;amp;q=mehrstellenverfahren%20poisson&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;this book&lt;/a&gt; for solving the Poisson equation with error converging with $\mathcal{O}(h^4)$. However, when I try to implement it for the 1D equation $-u''(x)=f(x)$, it only converges with around $\mathcal{O}(h^2)$. The math is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Taylor Series, we get $$\frac{1}{h^2}[-u_{i-1} + 2u_i - u_{i+1}] = -u''_i-\frac{h^2}{12}u^{(4)}_i + \mathcal{O}(h^4).$$&#xA;Substituting in $$-u^{(4)} = \frac{1}{h^2}[-u''_{i-1} + 2u''_i - u''_{i+1}] + \mathcal{O}(h^2)$$&#xA;we get $$\frac{1}{h^2}[-u_{i-1} + 2u_i - u_{i+1}] = f_i+\frac{1}{12}(f_{i-1} -2f_i +f_{1+1})_i + \mathcal{O}(h^4)$$&#xA;where $u_i =u(x_i)$ and $f_i = f(x_i)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that solving the system $$\frac{1}{h^2}Au = f-\frac{1}{12}Af$$&#xA;where $A$ is of the form $$ A = \left[ \begin{array}{c} 2&amp;amp;-1&amp;amp;0&amp;amp;0 \\-1&amp;amp;2&amp;amp;-1&amp;amp;0 \\ 0&amp;amp;-1&amp;amp;2&amp;amp;-1 \\ 0&amp;amp;0&amp;amp;-1&amp;amp;2\end{array}\right]  $$&#xA;should result in an error that converges with $\mathcal{O}(h^4)$. But when I implement this (in Matlab), halving the step-size $h$ results in approximately quartering the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something wrong with my math?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If necessary, I can include the Matlab code here. But as of now I'm thinking the problem lies with the math.)&lt;/p&gt;&#xA;" OwnerUserId="3892" LastActivityDate="2013-04-15T02:49:01.377" Title="Mehrstellenverfahren for Poisson?" Tags="|discretization|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6855" PostTypeId="2" ParentId="6853" CreationDate="2013-04-14T22:12:02.007" Score="19" Body="&lt;p&gt;Double precision is fairly common on newer GPUs.  For instance I own a NVIDIA GTX560 Ti (fairly low end when it comes to computing) that has no issue running ViennaCL in double precision.  From &lt;a href=&quot;http://developer.download.nvidia.com/assets/cuda/files/NVIDIA-CUDA-Floating-Point.pdf&quot;&gt;here (section 4)&lt;/a&gt; it appears all NVIDIA cards from GTX4xx onward support double precision natively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would guess that the GROMACS information is simply outdated.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-04-14T22:12:02.007" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="6856" PostTypeId="1" AcceptedAnswerId="6857" CreationDate="2013-04-14T22:47:43.987" Score="1" ViewCount="777" Body="&lt;p&gt;I came across the following task recently:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Use the von-Neumann stability analysis to investigate the stability of the discrete form of $\frac{\partial c}{\partial x} = \frac{\partial^2 c}{\partial y^2}$. Use the first-order forward finite difference for the first-order derivative and the usual central difference scheme for the second-order derivative. You can use the notation $c_{i, j} = c(ih, jh)$. The corresponding mesh size $h$ is same in both $x$- and $y$-direction. Which restriction arises for mesh size $h$?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Hint:&lt;/strong&gt; the vector $f_k(jh) = \sin(k \pi x)$, $j = 0, \dots, N$, is an eigenvector of finite difference-expression for the second-order derivative. The corresponding eigenvalue is given by $\lambda_k = \frac{2}{h^2}(\cos(\pi kh) - 1)$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm completely confused. I've seen classical example when Von Neumann Stability Analysis is applied to 1D heat equation $\frac{\partial T}{\partial t} = \frac{\partial^2 T}{\partial x^2}$, and it was pretty straightforward. However, this task asks to apply this analysis to stationary problem, therefore I'm not sure how to define amplification factor. Should it be simply $1$? Secondly, now it is 2D, and I don't know how to incorporate this fact into the method. Finally, I'm confused by the hint: I understand neither what it states nor how to utilize it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All I can currently do is discretize it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;2c_{i + 1, j} = c_{i, j - 1} + c_{i, j + 1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how do I perform the analysis properly in this case? Appreciate your help.&lt;/p&gt;&#xA;" OwnerUserId="4030" LastEditorUserId="4030" LastEditDate="2013-04-14T23:10:27.653" LastActivityDate="2013-04-14T23:30:28.730" Title="Von Neumann Stability Analysis" Tags="|pde|finite-difference|stability|discretization|fourier-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6857" PostTypeId="2" ParentId="6856" CreationDate="2013-04-14T23:30:28.730" Score="1" Body="&lt;p&gt;I will just point out that the problem you specify is exactly the same as the transient 1D heat equation only with different letters. In the equation you are analyzing just switch c with T, x with t and y with x and you will get the 1D heat equation.&lt;/p&gt;&#xA;" OwnerUserId="3561" LastActivityDate="2013-04-14T23:30:28.730" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6858" PostTypeId="1" CreationDate="2013-04-14T23:49:19.337" Score="3" ViewCount="310" Body="&lt;p&gt;This question is a follow-up to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6812/fortran-best-way-to-time-sections-of-your-code&quot;&gt;Fortran: Best way to time sections of your code?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to time functions in my code, I know I could use gprof or kcachegrind. I also know that the results from these tools can be skewed (see &lt;a href=&quot;http://www.yosefk.com/blog/how-profilers-lie-the-cases-of-gprof-and-kcachegrind.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.yosefk.com/blog/how-profilers-lie-the-cases-of-gprof-and-kcachegrind.html&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/1777556/alternatives-to-gprof/1779343#1779343&quot;&gt;https://stackoverflow.com/questions/1777556/alternatives-to-gprof/1779343#1779343&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I could add manual timers to each function for which I want data, which can be tedious or impractical for libraries, if I want data for everything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I run into communities that want this timing data to use as evidence in arguing for the performance their methods (to demonstrate improvement in performance, point out spots where performance is bad, for scientific papers, and so on). This seems to be popular with management-types and some academic-types. Is there a better way to get reliably accurate timing data than inserting timers? Should I be using a combination of imperfect tools and sifting through the performance data in some way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;Note:&lt;/strong&gt; This question &lt;em&gt;isn't&lt;/em&gt; about performance tuning, even though it's related. You can do performance tuning without timing things by using random pausing. It also &lt;em&gt;isn't&lt;/em&gt; about whether or not timing is worthwhile, because these communities want timing data, and I don't have the power to change their mind easily. Any comments about these topics are great discussion, but they're not helpful in answering my question, because the reality is that the people I answer to want timing data that somehow reflects performance.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-04-15T17:03:30.313" Title="How do I get reliable timing data for time spent in function calls in my code?" Tags="|performance|profiling|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6859" PostTypeId="2" ParentId="6858" CreationDate="2013-04-15T00:22:25.867" Score="3" Body="&lt;p&gt;You might consider a stack sampling profiler like HPCToolkit or VTune, or the system profiler for Linux, prof. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I don't see what's objectionable about wanting to know how long things take. If you want to demonstrate that your implementation of an algorithm has the asymptotic performance you derived, actually measuring the running time is the best way to do so.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-04-15T00:22:25.867" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="6860" PostTypeId="1" AcceptedAnswerId="6869" CreationDate="2013-04-15T02:00:41.590" Score="2" ViewCount="3943" Body="&lt;p&gt;I have a &lt;span class=&quot;math-container&quot;&gt;$53534\times 3$&lt;/span&gt; matrix with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; coordinates.&#xA;I want to find the element of matrix within ranges as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% coordinate range;&#xA;x1(x&amp;lt;-25|x&amp;gt;0);&#xA;x2(x&amp;lt;0|x&amp;gt;25);&#xA;y1(y&amp;lt;-40|y&amp;gt;0);&#xA;y2(y&amp;lt;0|y&amp;gt;40);&#xA;z1(z&amp;lt;45|z&amp;gt;0);&#xA;z2(z&amp;lt;0|z&amp;gt;82);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and insert them into a new matrix, so that it becomes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;point1=[x1, y1, z1];&#xA;point2=[x2, y2, z2];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to find the distance between the two points.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% define points;&#xA;xd=x2-x1;&#xA;yd=y2-y1;&#xA;zd=z2-z1;&#xA;Distance=sqrt(xd*xd+yd*yd+zd*zd);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is loop preferably efficient?&lt;/p&gt;&#xA;" OwnerDisplayName="user3991" LastEditorUserId="20688" LastEditDate="2019-06-22T19:16:12.920" LastActivityDate="2019-06-22T19:16:12.920" Title="Matlab element within ranges and distance between two points" Tags="|matlab|matrix|performance|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6861" PostTypeId="2" ParentId="6858" CreationDate="2013-04-15T02:44:16.563" Score="3" Body="&lt;p&gt;While it may be true that gprof or valgrind's cachegrind can produce skewed results, they are almost always good enough for what you really want to do -- namely find out which functions are &quot;expensive&quot; and which are &quot;not expensive&quot;. As the article you quote shows, it's possible to generate programs for which profilers do not show the whole story, or in fact even show falsehoods. However, when you apply profiles to &quot;real programs&quot;, they do almost always give you a fairly good picture of the truth, and that is precisely why they are still used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, despite their limitations, I do believe that &lt;em&gt;when applied to real-world programs&lt;/em&gt;, profiles do show very useful data and I would not hesitate to include this data into publications (with a short description of said limitations).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-15T02:44:16.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6862" PostTypeId="2" ParentId="6854" CreationDate="2013-04-15T02:49:01.377" Score="3" Body="&lt;p&gt;I haven't checked the exact coefficients in your formulas, but there are typically two difficulties with this scheme:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is your solution smooth enough to do the Taylor expansion to this order? For example, if your function $f$ is only once continuously differentiable, you can't expect the solution to be four times continuously differentiable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What do you do with boundary conditions? That is, what do you use for $f_{i-1}$ at the left boundary, and similarly $f_{i+1}$ at the right?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-15T02:49:01.377" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6863" PostTypeId="1" CreationDate="2013-04-15T03:32:59.197" Score="7" ViewCount="1793" Body="&lt;p&gt;I am considering two equations $$u_t=a(x)u_{xx}$$ and $$v_t=b(x)v_x$$ as classical representatives of the parabolic and hyperbolic family of equations. If $a(x)=a$ and $b(x)=b$ were constants, to show stability of any finite difference scheme I use for those equations, I could use the Fourier or so called Von Neumann stability analysis, i.e. would have to calculate the magnitude of the amplification factor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of &lt;strong&gt;variable coefficients&lt;/strong&gt; though, where $a=a(x)$ and $b=b(x)$, I am somewhat confused. From the book by Strikwerda (&lt;em&gt;Finite Difference Schemes and PDEs&lt;/em&gt;) on page 59 it says that one can use method of frozen coefficients, i.e. freeze the values at every grid point, use the Fourier analysis and deduce stability for variable coefficients. On the other hand, in the book of Ascher (&lt;em&gt;Numerical methods for Evolutionary Differential Equations&lt;/em&gt;), on page 153 it says that even ff the method is stable with constant coefficients then it doesn't necessary imply stability for variable coefficients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I see a contradiction between two books. My problem is that I see how the von Neumann analysis works and I wonder if I can use that with variable-coefficient PDEs as well. If I can, is that the same for both of the equations I am looking at or there are some differences between the two in terms of the method of frozen coefficients? Basically, where is the limitation of the Von Neumann stability analysis for parabolic and hyperbolic equations?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="123" LastEditDate="2016-03-06T12:18:03.460" LastActivityDate="2016-03-06T12:38:54.110" Title="method of frozen coefficients and its relation to von Neumann stability analysis" Tags="|pde|finite-difference|stability|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6864" PostTypeId="1" CreationDate="2013-04-15T03:50:36.147" Score="5" ViewCount="832" Body="&lt;p&gt;This is a follow up to the question about the method of &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6863/method-of-frozen-coefficients-and-its-relation-to-the-von-neumann-stability-anal&quot;&gt;frozen coefficients&lt;/a&gt;. I think it deserves to be a separate question. The frozen coefficient problems are obtained by fixing the coefficients of the variable coefficient problem. For example, consider pde $$u_t=a(x)u_{xx}, x \in \mathbb{R}, u(0,x)=u_0(x)$$, then at every grid point $x_i$ we can have a frozen problem $$u_t=a(x_i)u_{xx}$$. However, even though it is a frozen problem, in a sense that if I write a finite difference scheme, then at every grid point the diffusion coefficient is constant, but this is still quite different from considering the problem $$u_t=au_{xx}, x \in \mathbb{R}, u(0,x)=u_0(x)$$ where $a$ is just a constant number. So, for the latter case, to compute the amplification factor of a finite-difference scheme, I need to write the equation in terms of Fourier transform only once at some grid point $x_i$, but if I freeze the coefficients, I have to consider that at every single node and recompute the amplification factor at each of them. So, if this is the way, a problem with frozen coefficients and a problem with constant coefficient are not equivalent, the constant coefficient problem is rather a subproblem of the frozen coefficient problem, is that correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And another related question: if I complicate things more and let $a=a(t,x)$, then by the definition I can't use the Fourier analysis, however, if I freeze the coefficients, it is perfectly applicable? Do I have to take care in a special way of the time-dependent coefficients? I am looking for some intuition behind the method of frozen coefficient. It seems to make everything much simpler but I don't want to abuse that and know my limitations.   &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-04-17T11:06:28.780" Title="frozen coefficient vs. constant coefficient" Tags="|pde|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6865" PostTypeId="2" ParentId="6858" CreationDate="2013-04-15T09:50:18.847" Score="3" Body="&lt;p&gt;I've often used &lt;a href=&quot;http://software.intel.com/en-us/intel-vtune-amplifier-xe&quot; rel=&quot;nofollow&quot;&gt;Intel's VTune Amplifier&lt;/a&gt; to get precise timings, which, on the right hardware, will break timings down to the instruction level. The better results come from using on-chip counters, i.e. the Performance Monitoring Unit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The counts are still not exact, but have a much better resolution than what you can get out of software-based collectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your comment on the need for exact numbers, I don't completely agree. I don't consider myself to be a bean counter, but I rely heavily on timer, mostly implemented directly in the code using calls to &lt;a href=&quot;http://www.fftw.org/cycle.h&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;cycle.h&lt;/code&gt; from FFTW&lt;/a&gt; wrapped in macros, for actual research. Specifically, I work on algorithms for task-based parallelism, and need good estimates of the time spent doing actual work vs. overheads in the task allocation. These overheads are usually the sum of many small function calls and can be difficult to assess in a profiler, but are pretty much the only measure of how good a scheduling scheme is. In this case, precise timers are actually a necessary condition to doing good research.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-04-15T09:50:18.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6866" PostTypeId="2" ParentId="3412" CreationDate="2013-04-15T10:25:45.757" Score="1" Body="&lt;p&gt;A deterministic code is not necessarily reversible, so it should not actually add any benefit in terms of statistical sampling, it kinds of produce the same errors but consistently. The consistency is useful for debugging. All codes are numerically integrated, so they will all drift sooner or later from the constant energy. It is important to stay close enough to the energy surface even when sampling in NVT and all codes used by people actually do it using mixed or fixed precision (NAMD, LAMMPS, ACEMD, AMBER, GROMACS, DESMOND, ANTON). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A single gtx680 would produce around 160 ns/day. A single gtx Titan 220 ns/day. So you get 1 microsecond in 4 days with the free version of acemd.&lt;/p&gt;&#xA;" OwnerUserId="4093" LastEditorUserId="4093" LastEditDate="2013-04-15T14:31:09.353" LastActivityDate="2013-04-15T14:31:09.353" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6867" PostTypeId="2" ParentId="6860" CreationDate="2013-04-15T11:41:36.933" Score="3" Body="&lt;p&gt;I made some tests on a slightly different example (you can easily adapt to your need): I generate a 53534x3 random matrix in Matlab (7.9.0)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R = rand(53534,3);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I try to extract those row indices where all the three coordinates are greater than 0.5. I made a first version with a loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I1=[];&#xA;for i=1:53534&#xA;    if ( R(i,1)&amp;gt;0.5 &amp;amp;&amp;amp; R(i,2)&amp;gt;0.5 &amp;amp;&amp;amp; R(i,3)&amp;gt;0.5 )&#xA;       I1 = [I1; i];&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and a second version without loop but with the &lt;code&gt;find&lt;/code&gt; function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I2 = find( (R(:,1) &amp;gt; 0.5) .* (R(:,2)&amp;gt; 0.5) .* (R(:,3)&amp;gt;0.5) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(The element-wise product replaces the AND here). Both codes give exactly the same results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the interesting part is the time required for both codes to run:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The version &lt;strong&gt;with loop&lt;/strong&gt; takes around &lt;strong&gt;0.05 secondes&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;The version &lt;strong&gt;without loop&lt;/strong&gt; takes around &lt;strong&gt;0.004 secondes&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, there is one order of magnitude between them! So, at least in my example, &lt;strong&gt;the loop was not the most efficient implementation&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could argue that you have more flexibility with the loop. For example, if you know that there will be only one index, you can run the loop only until you find it (not until the end). However, this makes you win (in average) only a factor 2, and the &lt;code&gt;find&lt;/code&gt; function has also something similar (check the help).&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-04-15T11:41:36.933" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6868" PostTypeId="2" ParentId="3412" CreationDate="2013-04-15T14:29:35.177" Score="2" Body="&lt;p&gt;For all-atom, explicit solvent, bio-molecular systems of O(100k) atoms you ought now to be using GPU-accelerated codes. Even without knowing exactly the setup of your simulations it is most probable that ACEMD, AMBER, Gromacs, NAMD would all be adequate for your needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally these codes won't scale beyond a single system for your simulation size (unless with network like Infiniband), or even a few GPUs, and strongly favour GPU performance over CPU, so focus on machine configurations with several high-performance GPUs and good PCIe connectivity. Plan for 1-2 core/GPU. With some codes, there's no need to have multi-CPU systems, since the computation is done on the GPU (note that GROMACS will use both CPU and GPU effectively, so their quality should be balanced), nor employ a high performance interconnect, such as Infiniband.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the codes use CUDA so Nvidia GPUs are the way to go. Geforce cards are perfectly adequate (eg the 4GB Geforce GTX680), and substantially more economical than the Teslas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We sell a workstation optimised for ACEMD and other MD codes &lt;a href=&quot;http://www.acellera.com/products/metrocubo/&quot; rel=&quot;nofollow&quot;&gt;Acellera Metrocubo&lt;/a&gt;. Alternatively, register for the &lt;a href=&quot;http://www.nvidia.com/object/gpu-test-drive.html&quot; rel=&quot;nofollow&quot;&gt;NVidia GPU Test Drive&lt;/a&gt; to be put in touch with other suitable hardware resellers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regard to the criticism of hydrogen mass re-partitioning, the theoretical and technical basis was first described in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Improving efficiency of large time-scale molecular dynamics simulations of hydrogen-rich systems&lt;/em&gt; Feenstra et al. JCC 1999&lt;/p&gt;&#xA;&#xA;&lt;p&gt;doi://10.1002/(SICI)1096-987X(199906)20:8&amp;lt;786::AID-JCC5&gt;3.0.CO;2-B&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a widely used method, implemented not only in ACEMD but also Gromacs and, recently, Anton:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Atomic-level description of ubiquitin folding&lt;/em&gt;, Piana et al, PNAS (2013)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;doi://10.1073/pnas.1218321110 &lt;/p&gt;&#xA;" OwnerUserId="4094" LastEditorUserId="3649" LastEditDate="2013-10-10T03:04:48.757" LastActivityDate="2013-10-10T03:04:48.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6869" PostTypeId="2" ParentId="6860" CreationDate="2013-04-15T14:53:17.550" Score="3" Body="&lt;p&gt;I'm suspecting a typo in at least some of the conditionals. For instance, &lt;code&gt;z&amp;lt;45|z&amp;gt;0&lt;/code&gt; is trivially true. But I'm going to pretend its not and write the code out anyway, so that if you correct it (perhaps &lt;code&gt;z&amp;lt;-45|z&amp;gt;0&lt;/code&gt;) you'll still have the right structure.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x  = A(:,1); y = A(:,2); z = A(:,3); &#xA;i1 = find((x&amp;lt;-25|x&amp;gt;0)&amp;amp;(y&amp;lt;-40|y&amp;gt;0)&amp;amp;(z&amp;lt;45|z&amp;gt;0),1,'first'); &#xA;i2 = find((x&amp;lt;0|x&amp;gt;25)&amp;amp;(y&amp;lt;0|y&amp;gt;40)&amp;amp;(z&amp;lt;0|z&amp;gt;82),1,'first'); &#xA;point1 = A(i1,:); &#xA;point2 = A(i2,:); &#xA;Distance = norm(point1-point2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Like Dr. Sam showed, using tight &lt;code&gt;for&lt;/code&gt; loops in MATLAB is truly disastrous for performance. Obviously, as Wolfgang pointed out, MATLAB is using optimized &lt;code&gt;for&lt;/code&gt; loops internally to iterate through the elements of the vectors, but that's compiled code. If you were doing this with compiled C, you'd use &lt;code&gt;for&lt;/code&gt; loops too, and avoid a lot of the temporary generation that MATLAB is doing here.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2013-04-15T14:53:17.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6870" PostTypeId="2" ParentId="6853" CreationDate="2013-04-15T18:11:08.087" Score="8" Body="&lt;p&gt;Every GPU with SM 1.3 (Tesla/GTX2xx) or better has hardware double-precision support.  Starting with the Fermi architecture, Quadro and Tesla variants have better double-precision support than consumer Ge Force models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's weird is that Ge Force Kepler/GTX6xx double-precision support is inferior to GeForce Fermi/GTX5xx support to improve Tesla differentiation in Kepler K20/K20x.  Compounding the weirdness, Tesla K10s have Ge Force-level double-precision support.  And most recently, this got thrown into complete disarray by the introduction of the Ge Force GTX Titan, which has full double-precision support and many CUDA features only present on Tesla models up to now. A GTX Titan costs ~$1,000 - a screaming bargain IMO.&lt;/p&gt;&#xA;" OwnerDisplayName="user4088" LastActivityDate="2013-04-15T18:11:08.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6871" PostTypeId="1" AcceptedAnswerId="6960" CreationDate="2013-04-15T18:19:39.503" Score="2" ViewCount="292" Body="&lt;p&gt;I am using the &lt;a href=&quot;http://dakota.sandia.gov/software.html&quot; rel=&quot;nofollow&quot;&gt;Dakota Optimizer&lt;/a&gt;.&lt;br&gt;&#xA;I have 50+ variables, 50+ constraints and more than one objective function.&lt;br&gt;&#xA;Independent of the actual optimizer used, or the output selected:&lt;br&gt;&#xA;Is there a way to plot the objective functions while the optimizer is running?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dakota offers a graphical output, but this shows a combined objective function - not each objective function individually. Furthermore, with the amount of variables and constraints the standard window/screen is too small for a complete graphical output.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the values for each objective function at each iteration because I provide them to the responses file. However, I would like to avoid generating an additional file that stores these values and call an external function to plot from that file.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a solution which is part of Dakota.&lt;/p&gt;&#xA;" OwnerUserId="4067" LastEditorUserId="4067" LastEditDate="2013-04-23T13:50:17.510" LastActivityDate="2013-04-24T15:03:54.397" Title="Dakota Optimizer - plot objective functions in real-time" Tags="|optimization|software|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6872" PostTypeId="2" ParentId="6846" CreationDate="2013-04-15T19:12:36.013" Score="11" Body="&lt;p&gt;You can find a fully documented implementation of a very simple, yet quite efficient, solution method (Chorin's splitting method) &lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/1.2.0/python/demo/pde/navier-stokes/python/documentation.html&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a selection of other popular methods, take a look at Chapter 21 in &lt;a href=&quot;http://fenicsproject.org/book/&quot;&gt;this book&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I'm the (co)author of both the demo program and the book. The book can be downloaded for free.&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-04-15T19:12:36.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6873" PostTypeId="1" AcceptedAnswerId="6874" CreationDate="2013-04-15T23:17:05.577" Score="1" ViewCount="189" Body="&lt;p&gt;I was assigned a project in my intro to computer programming class. The goal is to implement a numerical model to some engineering or physics/science problem. The course uses Java.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example might be a soccer ball's flight. I'm looking for something a little bit more interesting than just modeling gravity, air drag,.. and solving using Euler's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in numerical optimization and I looked at some algorithms but I could not find any topic to which I could apply them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please shine me with some ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="4099" LastEditorUserId="4244" LastEditDate="2013-07-26T11:23:51.867" LastActivityDate="2013-07-26T11:23:51.867" Title="Topics for project" Tags="|optimization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6874" PostTypeId="2" ParentId="6873" CreationDate="2013-04-16T00:50:33.717" Score="1" Body="&lt;p&gt;There are many interesting ordinary differential equations (ODEs) that make for interesting model cases. For example, predator-prey models like the one by Lotka and Volterra make for interesting cases and can easily be generalized to simulate the dynamics of multiple, interacting species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, modeling the gravitational interaction of multiple bodies makes for nice pictures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feel free to get inspired using the snippets I've got here on an intro math modeling course:&#xA;  &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/teaching.html#2010-fall-442&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/teaching.html#2010-fall-442&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-16T00:50:33.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6875" PostTypeId="1" AcceptedAnswerId="6909" CreationDate="2013-04-16T01:55:54.667" Score="9" ViewCount="1398" Body="&lt;p&gt;I have a Kalman Filter based modelling code that I have developed for a near-real time regional ionospheric mapping application. The code assimilates data from different sensors into a map (described by a set of basis functions) using a Kalman Filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to scale this up to a larger region and more sensors, however the matrix algebra part of the Kalman Filter is becoming very slow, due to the large matrices (thousands of rows/columns) involved. I suspect the best way to attack the runtime issue is to use the fact that these matrices are typically very sparse with 80% or more of the total elements zero. The reason for this is that each sensor has a bias parameter that is jointly estimated with the map coefficients. This shows up as a 1 in the column for that sensor in the Kalman H matrix, with zero in the columns for every other sensor and map co-efficient. There are hundreds of sensors each contributing 8-10 observations at each epoch, hence a lot of zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could look at implementing the components of the Kalman filter using sparse algorithms, specifically multiplication and inversion*, but I wonder if there is an even better approach that re-casts the Kalman filter in a different form more suitable for cases when the matrices are sparse? I know I could use an ensemble Kalman filter or something similar, but if possible I'd like to retain the optimality of the pure linear Kalman filter; the total data volume is not prohibitive, just the large sparse matrices that result from the linear model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of implementation, this is done in IDL, however the core matrix algebra is done via calls to external optimised LA libraries (specifically ATLAS). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;*I know that an optimum Kalman filter implementation avoids inversion and instead uses a UD decomposition. I am considering trying to implement something like this, so that may be the answer, but I'm fishing for whether there is a better solution given the sparseness of the matrices.&lt;/p&gt;&#xA;" OwnerUserId="1314" LastActivityDate="2013-09-05T21:48:10.773" Title="Sparse matrix implementation of the Kalman Filter?" Tags="|linear-algebra|sparse-matrix|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6876" PostTypeId="2" ParentId="6873" CreationDate="2013-04-16T02:10:52.590" Score="0" Body="&lt;p&gt;Expanding Wolfgang's answer, another place you can find ODEs is chemistry.  A 0-D model, assuming Arrhenius rates of the form $A \cdot n_i \cdot n_j$ would lead to a set of non-linear ODEs.  Once you got the solver setup and running, you could easily expand it to include more species and more reactions until you create something interesting enough for your purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One warning though, depending on the system, the system may end up stiff, and therefore require an implicit solver.  I am going to assume that implementing such a solver would be beyond the scope of your class, however, if you are making up fictitious rates, you should be able to avoid this.  If allowed, and you want to, you can download an open source implicit library for java from &lt;a href=&quot;http://www.netlib.org/ode/odeToJava.tgz&quot; rel=&quot;nofollow&quot;&gt;netlib&lt;/a&gt;.  Note: I haven't personally used this library, but just reading the readme, it appears well documented and fairly complete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your question you mention optimization, however that may be slightly more complicated than an ODE solver.  If you do want to go that path however, you could look into something such as finding a local minimum to a scalar function of many variables ($y = f(\vec x)$).  Again, it is hard to know if this would be too much or too trivial for your skill level.  For reference, I would expect this in the second &quot;programming&quot; class for an engineer, likely taken in the 2nd or 3rd year and implemented in matlab.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-04-16T02:10:52.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6878" PostTypeId="1" AcceptedAnswerId="6901" CreationDate="2013-04-16T09:12:01.607" Score="8" ViewCount="6765" Body="&lt;p&gt;I'm not really sure how to explain this problem clearly, so please bear with me.&#xA;I have a basis of 3 orthonormal unit vectors and a position, a standard 4x4 transform matrix in computer graphics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I have several points (offsets) in that space that I transform to world space. &#xA;The points are then perturbed slightly and now I wish to find the new basis that is the closest to representing the perturbed points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not exactly like finding principle components, because I want it to respect the original offsets. If that makes sense. Like springs from each new point to their respective starting positions.&#xA;I think the answer lies in solving a least-squares-problem, but I looked into it an now my head hurts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody explain it simply for me. I would prefer a closed form solution, but an iterative one would be okay too.&#xA;Thanks very much&lt;/p&gt;&#xA;" OwnerUserId="4104" LastEditorUserId="651" LastEditDate="2013-04-17T14:51:29.333" LastActivityDate="2013-04-17T14:51:29.333" Title="Fitting one set of points to another by a rigid motion" Tags="|optimization|regression|basis-set|least-squares|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="6879" PostTypeId="2" ParentId="6846" CreationDate="2013-04-16T09:17:52.963" Score="5" Body="&lt;p&gt;Computer graphics and games has seen a huge explosion of interest in fluid simulation in recent years.&#xA;&lt;a href=&quot;http://www.dgp.toronto.edu/people/stam/reality/Research/pdf/GDC03.pdf&quot; rel=&quot;noreferrer&quot;&gt;Here&lt;/a&gt; is a great paper from Jos Stam which discusses implentation of a solver for realtime applications. It comes with very easy to understand source code.&#xA;I don't know how accurate it is, but it might be what you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="4104" LastActivityDate="2013-04-16T09:17:52.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6880" PostTypeId="1" AcceptedAnswerId="6884" CreationDate="2013-04-16T11:48:01.287" Score="10" ViewCount="5402" Body="&lt;p&gt;As all knows incompressible flow doesn't exist in reality, its an assumption introduced to simplify governing equations. We can not apply this assumption straightforward. Generally Mach number(M&amp;lt;0.3 for incompressible flow), density variation(zero density variation) and divergence of velocity (is equal to zero for incompressible flow) are the common criterion to define flow as incompressible flow. It is observed that in case of heat transfer problem (such as natural convection) density varies, which violates the last two criterion. Is it possible to define incompressible flow assumption which includes heat transfer process also (means density variation)?&lt;/p&gt;&#xA;" OwnerUserId="4107" LastActivityDate="2013-05-11T19:25:23.033" Title="Definition of incompressible flow" Tags="|fluid-dynamics|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="6881" PostTypeId="2" ParentId="6880" CreationDate="2013-04-16T13:40:55.843" Score="10" Body="&lt;p&gt;Incompressibility is ONLY defined as the velocity field being solenoidal.&#xA;Incompressibility DOES NOT mean that density variation must be zero. From the continuity equation, the requirement that the velocity field have zero divergence only requires that the material derivative of density be zero. That is, the density of a material fluid particle must be constant. This is not the same as requiring that the density be spatially constant.&lt;/p&gt;&#xA;" OwnerUserId="1453" LastEditorUserId="1453" LastEditDate="2013-04-16T18:44:53.960" LastActivityDate="2013-04-16T18:44:53.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6882" PostTypeId="2" ParentId="6880" CreationDate="2013-04-16T14:04:53.173" Score="10" Body="&lt;p&gt;To add to John's answer, it is very, very common in low-speed flows with small density variations to use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Boussinesq_approximation_%28buoyancy%29&quot;&gt;Boussinesq Approximation&lt;/a&gt; to approximate the density variation due to temperature or dilute species concentration. This approximates the density variation as a linear function of the temperature and, therefore, removes the variable density from the governing equations.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-04-16T14:04:53.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6883" PostTypeId="2" ParentId="6880" CreationDate="2013-04-16T14:06:10.577" Score="3" Body="&lt;p&gt;Compressibility is defined as &#xA;$$&#xA;\tau =- { dv \over  v dp}&#xA; $$&#xA;So if you substitute $ v = {1 \over \rho}  $ we get $$ d\rho= -\rho \tau dp = - \rho^2 \tau VdV $$ using (Euler equation). Therefore, you can crudely say that effect of compressiblity increases when the velocity, $ V $ is sufficiently high. In regimes when natural convection dominates over forced convection, the velocity or more precisely Richardson number is of the order &amp;lt; O(1). That being the incompressible domain. You may use incompressible NS equations, with Bousinessque approximation for natural convection.&lt;/p&gt;&#xA;" OwnerUserId="4056" LastActivityDate="2013-04-16T14:06:10.577" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6884" PostTypeId="2" ParentId="6880" CreationDate="2013-04-16T15:15:53.377" Score="15" Body="&lt;p&gt;Others have pointed out the Boussinesq approximation (note that it is different from Boussinesq for water waves), but you can also go a step further and allow for large density variation without going to a fully compressible formulation.  This is called an &quot;anelastic&quot; model, and retains essentially the same computational structure as incompressible flow. For a nice introduction, see&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://kiwi.atmos.colostate.edu/group/dave/pdf/anebous.pdf&quot;&gt;David Randall &lt;strong&gt;The Anelastic and Boussinesq Approximations&lt;/strong&gt; 2010&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-04-16T15:15:53.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6885" PostTypeId="1" AcceptedAnswerId="6895" CreationDate="2013-04-16T20:25:02.790" Score="4" ViewCount="126" Body="&lt;p&gt;I know this question is somewhat familiar to my previous question found &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3371/effects-of-memory-speed-architecture-on-pardiso-scaling&quot;&gt;here&lt;/a&gt;, however I am now coming at it from a slightly different and more general angle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have a code whose runtime is in the days to weeks range.  For one reason or another, the code can not be changed to improve performance.  My goal is to purchase a computer to run this specific code as fast as possible.  It is currently parallel is some regards, but must be shared memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I have two computers on which to run it.  They both run the same Linux distribution and both run on Intel-Xeon processors, however the CPU models are different as is the memory speed, and other hardware values (the computers have about 5 years of age difference between them).  On both computers I can easily change the number of cores utilized for running the code, and the newer of the two supports hyperthreading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is there a way to profile runs of the code on one or both computers that would allow me to find where the hardware bottlenecks are, and thereby tune a new computer purchase for optimal performance with this code.  &lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-04-17T02:19:12.963" Title="Determining Hardware Bottlenecks" Tags="|profiling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6886" PostTypeId="1" AcceptedAnswerId="6887" CreationDate="2013-04-16T20:35:34.690" Score="4" ViewCount="256" Body="&lt;p&gt;I'm implementing a Markov Chain Monte Carlo process for a Quantum Monte Carlo routine, in every book and paper I've read so far the success of the routine and quality of the results strongly depends on the random number generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well I've chosen to implement the code in &lt;strong&gt;C++ 2011&lt;/strong&gt;, and I'm presented with a fair wide selection of pseudo random number generator engines and a non deterministic random number generator (based on the hardware random number generator that most of the modern computers have).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the physical or statistical reason to chose a good random number generator?, and which is best for this purpose, a well studied pseudo random number generator engine or the (I think unpredictable) hardware random number generator?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see the engines available &lt;a href=&quot;http://en.cppreference.com/w/cpp/numeric/random&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4113" OwnerDisplayName="Oscar David Arbeláez" LastActivityDate="2013-04-17T02:12:25.597" Title="Hardware random number generator Vs. Pseudo random number generator in the battlefield of Markov Chain Monte Carlo processes" Tags="|statistics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6887" PostTypeId="2" ParentId="6886" CreationDate="2013-04-16T21:18:52.447" Score="4" Body="&lt;p&gt;There is no reason to use hardware random number generation for anything other than full cryptography.  For everything else, including computational physics, pseudorandom generators are fine.  I would suggest using the &lt;a href=&quot;http://www.deshawresearch.com/resources_random123.html&quot; rel=&quot;nofollow&quot;&gt;Random123&lt;/a&gt; library of Salmon et al.: it's fast, trivially parallelizable, and strong (in particular stronger than Mersenne Twister).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to use a high quality generator, though the quality requirements depend very strongly on the application.  An example of what can go wrong: in explicit solvent molecular dynamics, it is common to take time steps on the order of 1 femtosecond ($10^{-15}$ s).  A generator with a period of only $2^{32}$ evaluated once per step would be similar to a radiative input with a frequency of $0.2$ MHz.  If this is a physically interesting frequency for the system in question, the results would be invalid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Happily, since there are numerous random generators which both fast and strong enough for essentially all noncryptographic purposes (including both Random123 and Mersenne Twister), there's no real tradeoff in practice.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2013-04-16T21:53:33.063" LastActivityDate="2013-04-16T21:53:33.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6888" PostTypeId="2" ParentId="6846" CreationDate="2013-04-16T22:34:52.493" Score="6" Body="&lt;p&gt;Simplest is always going to be relative to your particular interests and needs. I agree with Anders that, for incompressible flow on domains with simple geometry, you'd be hard-pressed to beat the projection method (i.e., Chorin's splitting method) if you are prioritizing both ease of use and accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To go into a little more detail, the method is question is introduced in [1]. The more modern, second-order, approximate projection method is explained well in [2]. The motivation is that solving the full incompressible Navier-Stokes equations requires solving for the velocity field and the pressure simultaneously, and the resulting linear system is rather ill-conditioned. The projection method eliminates this problem by splitting each time step into a velocity solve, using the pressure from the previous timestep, followed by a pressure update, which essentially enforces that the velocity field remain incompressible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To implement this, you'll need a few other components, but all can be learned and programmed quite easily.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For the pressure solve, assuming you're interested in systems with constant density, you'll need to solve Poisson's equation. There are, of course, dozens of algorithms to approach this problem, but by far the easiest to implement -- if perhaps not to fully understand -- is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Conjugate_gradient_method&quot;&gt;conjugate gradient&lt;/a&gt; (CG) algorithm. One of the best explanations of CG I've read was written by Jonathan Shewchuk and can be found &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;cad=rja&amp;amp;ved=0CDMQFjAA&amp;amp;url=http%3A%2F%2Fwww.cs.cmu.edu%2F~quake-papers%2Fpainless-conjugate-gradient.pdf&amp;amp;ei=-9BtUY6HAYGuigLL_IGoCw&amp;amp;usg=AFQjCNGRNrF-xuZbtV4TvXb7vlk1Gv45og&amp;amp;sig2=hMByyZ0vse0G1OugeC-iYw&quot;&gt;here&lt;/a&gt;. You certainly need not read the entire paper, however, to be able to simply implement the algorithm.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You'll need another algorithm to handle the advection term in Navier-Stokes. In several dimensions, programming robust implementations of the most flexible methods, e.g. Godunov, can be quite challenging. However, provided you're interested in flows with relatively modest Reynolds number (i.e., with non-negligible viscosity), one of the essentially non-oscillatory (ENO) methods fits the bill nicely in terms of ease of implementation. There is an excellent overview of both the theory and implementation in [3].&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You'll need to handle the viscous term using an implicit method, typically Crank-Nicolson. This is explained in detail in the projection method papers, and you can easily use CG for the matrix solve provided viscosity is constant.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;[1] A. J. Chorin, &lt;em&gt;Numerical solution of the Navier-Stokes equations&lt;/em&gt;, J. Math. Comput., 22 (1968), pp. 745-762&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] A. Almgren, J.B. Bell, and W. Szymczak, &lt;em&gt;A numerical method for the incompressible Navier-Stokes equations based on an approximate projection&lt;/em&gt;, SIAM J. Sci. Comput. 17 (1996), pp. 258-369.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] S. Osher and R. Fedkiw, &lt;em&gt;Level Set Methods and Dynamic Implicit Surfaces&lt;/em&gt;. Springer-Verlag New York,. Applied Mathematical Sciences, 153, 2002&lt;/p&gt;&#xA;" OwnerUserId="1748" LastActivityDate="2013-04-16T22:34:52.493" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6889" PostTypeId="2" ParentId="2384" CreationDate="2013-04-16T23:09:33.687" Score="2" Body="&lt;p&gt;I would like to point out that instead of using higher precision arithmetic for the addition, there's the possibility of using compensated summation (see [1]).  This could increase the accuracy of the summation without the need to resort to larger data types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Higham, N. J. The Accuracy of Floating Point Summation. SIAM Journal on Scientific Computing 14, 783–799 (1993).&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-04-16T23:09:33.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6890" PostTypeId="1" AcceptedAnswerId="6917" CreationDate="2013-04-16T23:13:35.600" Score="2" ViewCount="332" Body="&lt;p&gt;I have a cubic of the form&#xA;$$x^3 - a x - b = 0$$&#xA;where $a, b &amp;gt; 0$.  Thus, I know there is exactly one positive root.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a nice stable formula for this unique positive root?&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-04-18T17:25:27.823" Title="Stable formula for a specific root of a depressed cubic" Tags="|polynomials|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6891" PostTypeId="2" ParentId="5133" CreationDate="2013-04-17T00:34:58.513" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://www-dssz.informatik.tu-cottbus.de/DSSZ/Software/Snoopy&quot; rel=&quot;nofollow&quot;&gt;Snoopy&lt;/a&gt; has the ability to translate between many Petri net formats, SBML, and other formats.&lt;/p&gt;&#xA;" OwnerUserId="283" LastActivityDate="2013-04-17T00:34:58.513" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6892" PostTypeId="2" ParentId="5151" CreationDate="2013-04-17T00:43:45.977" Score="1" Body="&lt;p&gt;If don't need physics, you can just &lt;em&gt;cheat&lt;/em&gt; and make your planets orbit in a perfect circumference or even better in a ellipse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ pos_{planet} = circumference(angle(time),radius,center)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The moon and other satellites would be a bit different:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$pos_{satellite} = pos_{planet} + circumference(angle(time),radius,center)$$&lt;/p&gt;&#xA;" OwnerUserId="3580" LastEditorUserId="3580" LastEditDate="2013-04-17T01:10:43.783" LastActivityDate="2013-04-17T01:10:43.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6893" PostTypeId="1" AcceptedAnswerId="6894" CreationDate="2013-04-17T01:15:05.333" Score="1" ViewCount="843" Body="&lt;p&gt;Let $T$ be a real symmetric tridiagonal matrix. Then the divide-and-conquer eigenvalue algorithm, as detailed in any standard text, goes by subdividing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ T = \begin{bmatrix} T_1 &amp;amp; 0 \\ 0 &amp;amp; T_2 \end{bmatrix} + \rho vv^T,$$&#xA;after which we can solve the eigenvalue problem for $T_1, T_2$, and then solve the full eigenvalue problem by accounting for the rank-$1$ shift $\rho vv^T$ using a root-finding technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, now let $T$ be a complex Hermitian tridiagonal matrix. Then if we attempt to do the same strategy,&#xA;$$ T = \begin{bmatrix} T_1 &amp;amp; 0 \\ 0 &amp;amp; T_2 \end{bmatrix} + S,$$&#xA;where $S$ is Hermitian but is not rank-$1$ (it appears to have rank $2$), so it seems like the same root-finding technique will fail, unless I'm mistaken. How can we adapt the algorithm to work with the complex Hermitian case?&lt;/p&gt;&#xA;" OwnerUserId="3477" LastActivityDate="2013-04-17T01:32:33.343" Title="How does one implement the divide-and-conquer eigenvalue algorithm for a complex Hermitian matrix?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6894" PostTypeId="2" ParentId="6893" CreationDate="2013-04-17T01:32:33.343" Score="5" Body="&lt;p&gt;There is no reason to ever have to give thought to the Hermitian tridiagonal eigenvalue problem, as it is trivial to define a unitary diagonal similarity transformation which reduces it to the real symmetric tridiagonal eigenvalue problem. &lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2013-04-17T01:32:33.343" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6895" PostTypeId="2" ParentId="6885" CreationDate="2013-04-17T02:02:37.407" Score="6" Body="&lt;p&gt;I assume you're talking about server-class parts here. Client parts may be a bit different, but most of the logic holds.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Find a test case that runs with one thread in ~10 minutes that's representative of your longer problems in that it exercises the important parts of your code in similar ratios. You may have to scale down the problem size in order to scale down the initialization time, etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since you only have Intel Xeons, try turning on and off hyperthreading and comparing the run times of a version of the code that uses the maximum thread count in both cases. If your code is modestly faster with hyperthreading turned on, then you should borrow time on an AMD Bulldozer-based machine to test to see if it's shared FPUs are advantageous for your code. It's not exactly the same, but if two hyperthreads can successfully share the Xeon FPU, then you might have a chance at some decent performance out of the shared FPU on an AMD machine. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use Linux or the BIOS's ability to set a sequence of fixed, but reduced CPU clock rates for runs of your code (e.g. 2.7GHz, 2.6GHz, ...). If the performance scales strongly with clock rate, without plateauing at the high end, buy the fastest clock rate CPUs you can afford. If the performance plateaus after some clock speed, then consider not buying the top bin CPU, but backing off and buying more or faster memory. If you at all can, benchmark your code on the actual hardware you want to buy. This may be hard if you only want to buy one node, but ask around, someone you know may have one you can borrow. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider your computational plan. If the cases you need to run are finite and enumerated, calculate the cost of renting some EC2 instances from Amazon instead of buying a machine. If you're only planning on buying one node, you may find that even keeping it fully loaded, your planned runs will take longer than you can afford to wait, and buying 10x as many instances from Amazon and running them there may be more effective. Plus, it's actually hard to keep a single workstation or server node busy all the time. It requires diligence. Amazon nodes you can rent when you need and discard when you don't. You don't have to pay for time you're not using.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Strongly consider applying for supercomputer time with an organization like XSEDE in the US or PRACE in the EU (or your country's local equivalent). We (I work for TACC which participates in XSEDE) literally give away the time, and ~100k core-hours can be had for free in a matter of days or weeks through a start-up allocation. I presume the complementary organizations have similar policies.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-04-17T02:19:12.963" LastActivityDate="2013-04-17T02:19:12.963" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6896" PostTypeId="2" ParentId="6886" CreationDate="2013-04-17T02:12:25.597" Score="4" Body="&lt;p&gt;Geoffrey already answered the question, but I'd like to add another perspective to it. One of the things you will have to do one way or another is to debug code. It may not necessarily be that you have to debug anything that has to do with the random numbers themselves, but, say, a bug in the function evaluation that depends on the randomly selected sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, it is immensely awkward if you use a random number generator that is not &lt;em&gt;deterministic&lt;/em&gt;, i.e., that does not produce the same numbers in every run: because if you do use a non-deterministic random number generator and you see a bug, you will not be able to reproduce it the next time you call your program. As a consequence, I've always told my students to use a deterministic random number generator using a fixed seed. Today's RNGs are good enough that one doesn't really have to be afraid that there is not enough entropy in their number sequences, and their deterministic nature allows you to debug programs.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-17T02:12:25.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6899" PostTypeId="2" ParentId="6890" CreationDate="2013-04-17T08:31:17.983" Score="2" Body="&lt;p&gt;I am not sure there is a stable formula, but given one that is numerically not stable, you could follow it by a single Newton step and improve the accuracy considerably&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-04-17T08:31:17.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6901" PostTypeId="2" ParentId="6878" CreationDate="2013-04-17T11:15:22.107" Score="15" Body="&lt;p&gt;Inge Söderkvist (2009) has a nice write-up of &lt;a href=&quot;http://www.ltu.se/cms_fs/1.51590!/svd-fitting.pdf&quot;&gt;solving the Rigid Body Movement Problem&lt;/a&gt; by singular value decomposition (SVD).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we are given 3D points $\{x_1,\ldots,x_n\}$ that after perturbation take positions $\{y_1,\ldots,y_n\}$ respectively.  We seek a rigid &quot;motion&quot;, i.e. a rotation $R$ and translation $d$ combined, applied to points $x_i$ that minimizes the sum of squares of &quot;errors&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum_{i=1}^n || Rx_i + d - y_i ||^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think of the points $x_i,y_i$ as columns, and of rotation $R$ as a $3\times 3$ orthogonal matrix.  Strictly speaking we require $\det(R)=1$ because as a continuous motion a rotation preserves the &quot;orientation&quot; of the points, i.e. will not involve reflecting them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step is to subtract off the respective means $\overline{x},\overline{y}$ from points $x_i,y_i$, which has the effect of &quot;eliminating&quot; (for now) the unknown translation $d$.  That is, the problem becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \min_{R\in SO(3)} || RA - B ||_F $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A = [x_1 - \overline{x},\ldots,x_n - \overline{x}]$ and $B = [y_1 - \overline{y},\ldots,y_n - \overline{y}]$.  Here $SO(3)$ denotes the &lt;a href=&quot;http://en.wikipedia.org/wiki/Rotation_group_SO%283%29&quot;&gt;special orthogonal group&lt;/a&gt; of rotation matrices in 3D we are allowed to choose $R$ from, and the matrix norm $F$ here denotes the &lt;a href=&quot;http://en.wikipedia.org/wiki/Frobenius_norm#Frobenius_norm&quot;&gt;Frobenius norm&lt;/a&gt;, i.e. the square root of the sum of squares of matrix entries (like a Euclidean norm, but on matrix entries).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now $A,B$ are both $3\times n$ matrices.  The minimization above is &lt;a href=&quot;http://en.wikipedia.org/wiki/Orthogonal_Procrustes_problem&quot;&gt;an orthogonal Procrustes problem&lt;/a&gt; allowing only rotation matrices.  The solution $R$ is given by taking the singular value decomposition of the &quot;covariance&quot; matrix $C = BA^T$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ C = U S V^T $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U,V$ are orthogonal matrices and $S = \text{diag}(\sigma_1,\sigma_2,\sigma_3)$ is the diagonal matrix of singular values $\sigma_1 \ge \sigma_2 \ge \sigma_3 \ge 0$.  Numerical linear algebra packages like Matlab and &lt;a href=&quot;http://www.gnu.org/software/octave/&quot;&gt;Octave&lt;/a&gt; will compute the SVD for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once we have the SVD, define $R = U \text{diag}(1,1,\pm 1) V^T$ where the sign in the middle factor is chosen to make $\det(R) = 1$. Ordinarily a real world application will have $\det(UV^T) = 1$, and thus the sign chosen would be positive.  If not, it means the best orthogonal (distance preserving) fit to the new points involves a reflection, and it suggests checking the data for mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally we define the translation $d = \overline{y} - R\overline{x}$.  Done!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variant of orthogonal Procrustes problems where only rotations are allowed is also the subject of another Wikipedia article on the &lt;a href=&quot;http://en.wikipedia.org/wiki/Kabsch_algorithm&quot;&gt;Kabsch algorithm&lt;/a&gt;.  Notation in the Wikipedia articles differs from ours in multiplying by $R$ on the right, rather than (as here) on the left.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-04-17T11:15:22.107" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6902" PostTypeId="1" AcceptedAnswerId="6916" CreationDate="2013-04-17T12:01:42.333" Score="1" ViewCount="560" Body="&lt;p&gt;There is quasi 1-D C-D nozzle test case for compressible flow codes, like that &quot;is there any 1-D test case for incompressible flow codes?&quot;&lt;/p&gt;&#xA;" OwnerUserId="3549" LastEditorUserId="3549" LastEditDate="2013-04-17T12:21:56.513" LastActivityDate="2013-04-18T16:22:14.783" Title="Is there any 1-D test case for incompressible flow codes?" Tags="|algorithms|fluid-dynamics|testing|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6903" PostTypeId="1" CreationDate="2013-04-17T16:11:28.950" Score="3" ViewCount="2065" Body="&lt;p&gt;I'm calculating the gradient of a function with a symbolic math library called theano. Then I'm using gradient descent to find the minimum of the function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to prove that the minimum is a global minimum. How can I prove that my function is convex?&lt;/p&gt;&#xA;" OwnerUserId="4070" LastActivityDate="2013-04-20T22:57:46.697" Title="How to prove that a function is convex?" Tags="|convex-optimization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6904" PostTypeId="2" ParentId="6903" CreationDate="2013-04-17T16:29:07.910" Score="8" Body="&lt;p&gt;There are many ways of proving that a function is convex:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By definition&lt;/li&gt;&#xA;&lt;li&gt;Construct it from known convex functions using composition rules that preserve convexity&lt;/li&gt;&#xA;&lt;li&gt;Show that the Hessian is positive semi-definite (everywhere that you care about)&lt;/li&gt;&#xA;&lt;li&gt;Show that values of the function always lie above the tangent planes of the function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-04-17T16:29:07.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6905" PostTypeId="2" ParentId="2283" CreationDate="2013-04-17T18:31:32.247" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://fwrap.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;fwrap&lt;/code&gt;&lt;/a&gt; specifically targets Fortran90/95 and uses some of the same software stack as &lt;code&gt;f2py&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="19" LastActivityDate="2013-04-17T18:31:32.247" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6908" PostTypeId="2" ParentId="6903" CreationDate="2013-04-17T23:12:29.717" Score="4" Body="&lt;p&gt;Unless you know something about the properties of the function (e.g., whether it's a quadratic polynomial, monotonic, etc), you can not experimentally determine whether a function is convex. You need to limit your question to a smaller subset of functions.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-17T23:12:29.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6909" PostTypeId="2" ParentId="6875" CreationDate="2013-04-18T03:33:09.990" Score="7" Body="&lt;p&gt;With sparse matrices, it's frequently the case that although a matrix $A$ is sparse, $A^{-1}$ is dense.  In such cases the Cholesky or $LU$ factorization of $A$ is more likely to be sparse (particularly if the rows/columns of $A$ are reordered to improve the sparsity pattern.)  In most cases, if you want to exploit sparsity and aren't interested in using an iterative algorithm for solving systems involving $A$, then you're better off using some factorization of the matrix rather than explicitly computing $A^{-1}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Kalman filtering in particular, rather than computing &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S_{k}=(H_{k}P_{k-1,k}H_{k}^{T}+R_{k})^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you'd typically be better off working working with a factorization of $S_{k}^{-1}$.  Since $S_{k}$ is symmetric and should be positive definite, you can use a Cholesky factorization or $LDL^{T}$ factorization to do this.  You've told us that your $H_{k}$ matrix is sparse, but you haven't told us anything about whether $R_{k}$ is sparse or otherwise structured, and of course $P$ could be quite dense.     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One reason that the Ensemble Kalman Filter (EnKF) and various particle filtering techniques are so popular is that for systems with an extremely large state vector, conventional Kalman filtering becomes very difficult.  EnKF can be efficiently implemented for very large state vectors if $R_{k}$ is diagonal or nearly diagonal.  These questions have been dealt with in great depth by people working in the field of data assimilation, so I'd suggest starting your research by reading about how they've dealt with these issues.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2013-04-18T03:38:29.153" LastActivityDate="2013-04-18T03:38:29.153" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6910" PostTypeId="2" ParentId="6902" CreationDate="2013-04-18T11:01:15.220" Score="1" Body="&lt;p&gt;Assuming that the system you are interested in is not hyperbolic, (shallow water equations model flow of an incompressible fluid and yet are hyperbolic), Following could be taken as test cases for validation of the code. (although none of them is purely 1d)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) As per Bill Barth's suggestion, Couette and Poiseuille flow problems are standard test cases&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Uniform flow over the flat plate, developing laminar boundary layer can be solved. Its behavior is given by the well-established Blasius solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The reason of why there are not a plenty 1d problems available, might be as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In hyperbolic systems, 1d test / benchmark problems are plenty. The reason being that wave formation and transmission are more dominant phenomena in hyperbolic systems. Shocks and contacts can be analysed as a 1d phenomena (In certain 2d problems, these can be converted into 1d and then solved). But in incompressible flow, no purely 1d phenomenon takes place)&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="1567" LastEditDate="2013-04-18T11:36:50.993" LastActivityDate="2013-04-18T11:36:50.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6911" PostTypeId="1" CreationDate="2013-04-18T12:24:50.460" Score="3" ViewCount="731" Body="&lt;p&gt;My dear community, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering why BFGS methods are not so widely used for simulating mechanical problems which heavily still relies on inverting the hessian matrix. I am essentially interested by constrained problems (contact on the interface for instance), and I know that some authors proposed method to deal with Quasi-Newton approximation for Augmented Lagrangian, but it seems to me that computing and inverting the regularized hessian matrix is still the method of choice. My question is: why ? Stability ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kind regards, &#xA;Tom&lt;/p&gt;&#xA;" OwnerUserId="3977" LastActivityDate="2013-04-18T15:02:55.823" Title="BFGS methods for constrained elasticity problems" Tags="|linear-solver|constraints|newton-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6912" PostTypeId="2" ParentId="5562" CreationDate="2013-04-18T13:02:04.657" Score="3" Body="&lt;p&gt;There are plenty of higher order schemes out there. But as per &lt;a href=&quot;http://en.wikipedia.org/wiki/Godunov%27s_theorem&quot; rel=&quot;nofollow&quot;&gt;Godunov's theorem&lt;/a&gt;, only first order scheme can be monotone and hence not create oscillations. &lt;a href=&quot;http://scicomp.math.ntu.edu.tw/wiki/images/6/62/FD.pdf&quot; rel=&quot;nofollow&quot;&gt;This&lt;/a&gt; resource gives a brief idea about the construction and analysis of finite difference schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In REA (Reconstruct, Evolve , Average) algorithm, required order polynomial is reconstructed and corresponding variable values are &lt;a href=&quot;http://en.wikipedia.org/wiki/Polynomial_interpolation&quot; rel=&quot;nofollow&quot;&gt;interpolated&lt;/a&gt; at cell face. This gives the fluxes at the cell faces. (This is same as $g(u_{j+{1}},u_{j})$ for face $j + \frac{1}{2}$ for above mentioned scheme, in which a straight line is reconstructed from cell averages).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then cell values are updated using this fluxes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Leveque's book &quot;Finite Volume Methods for Hyperbolic Problems&quot; gives detailed information on this. Depending on your choice of stencil, you can create an arbitrarily high order scheme. But there will always be oscillations near discontinuity if it is a high order scheme. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other sources of high order schemes are,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;a href=&quot;ftp://ftp.co-ra.com/pub/werne/keith/Tam_Webb_JCP_1993.pdf&quot; rel=&quot;nofollow&quot;&gt;DRP&lt;/a&gt; schemes (this paper also discusses formulation of standard FD schemes of arbitrary high order)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Discontinuous / continuous Galerkin methods (These can have arbitrarily high order of accuracy, but reconstruction unlike FVM, takes place within an element. Cell averaged values are not used for reconstruction)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Spectral methods&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some resources for the numerical shcemes,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &quot;Finite Volume methods for hyperbolic problems&quot;, Randall Leveque&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &quot;Computational Gasdynamics&quot;, Culbert Laney (Discusses ENO, MUSCL nicely too)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) &quot;Riemann Solvers and Numerical Methods for Fluid Dynamics&quot;, E.F.Toro &lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2013-04-18T13:02:04.657" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6913" PostTypeId="2" ParentId="5494" CreationDate="2013-04-18T14:05:28.760" Score="4" Body="&lt;p&gt;Given that you already accept that the duality gap along the central path is $m/t$, then the inequality you're struggling with is really rather simple. Remember, the dual problem provides lower bounds for the optimal value of the primal. So $p^*$ is necessarily in between the objective values of any feasible primal point and any feasible dual point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is: we have&#xA;$$f(x^*(t)) - g(\gamma^*(t),\lambda^*(t)) = m/t$$ &#xA;where $g$ is the dual objective. But we also have&#xA;$$p^* \leq f(x) \quad \forall x\text{ feasible}$$&#xA;and&#xA;$$p^* \geq g(\gamma,\lambda) \quad \forall \gamma,\lambda\text{ feasible}$$&#xA;Thus for &lt;em&gt;any&lt;/em&gt; feasible primal and dual points, we have&#xA;$$f(x)-p^* \leq f(x) - g(\gamma,\lambda) \quad \forall x,\gamma,\lambda\text{ feasible}$$&#xA;And along the central path,&#xA;$$f(x^*(t))-p^* \leq f(x^*(t)) - g(\gamma^*(t),\lambda^*(t))=m/t.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Actually, it looks like you may not accept the premise that the $m/t$ difference applies to the original problem. So let's look at this for an LP. The primal and dual problems are&#xA;$$\begin{array}{llcll}&#xA;\text{minimize} &amp;amp; c^T x &amp;amp; \quad &amp;amp; \text{maximize} &amp;amp; b^T \lambda \\&#xA;\text{subject to} &amp;amp; A x = b &amp;amp;  &amp;amp; \text{subject to} &amp;amp; A^T \lambda + \gamma = c \\&#xA;&amp;amp; x \succeq 0 &amp;amp; &amp;amp; &amp;amp; \gamma \succeq 0&#xA;\end{array}$$&#xA;I prefer $y$ and $z$ above to $\lambda$ and $\gamma$, but for consistency I am keeping your variable names. $\lambda$ is the Lagrange multiplier for the equality constraints, $\gamma$ is the Lagrange multiplier for the inequalities. The barrier problem for the primal is&#xA;$$\begin{array}{ll}&#xA;\text{minimize} &amp;amp; t c^T x - \textstyle \sum_{i=1}^m \log x_i \\ &#xA;\text{subject to} &amp;amp; A x = b&#xA;\end{array}$$&#xA;Note that the domain of this problem is limited to $x\succ 0$ by the barrier term.&#xA;The optimality conditions for a fixed $t&amp;gt;0$ satisfy&#xA;$$t c - \mathop{\textrm{diag}}(x)^{-1} \mathbf{1} - A^Ty = 0 \quad Ax=b \quad x \succ 0$$&#xA;where $y$ is a Lagrange multiplier for the equality constraints. That second term is a bit clumsy: define $z_i\triangleq x_i^{-1}$ for $i=1,2,\dots,m$ so&#xA;$$t c - z - A^Ty =0 \quad Ax=b \quad x,z \succ 0 \quad z_i\triangleq x_i^{-1},~i=1,2,\dots,m$$&#xA;By inspection, we see that if $(x,y,z)$ satisfies these equality constraints, then $(\gamma,\lambda)=(t^{-1}z,t^{-1}y)$ is a feasible dual point &lt;em&gt;for the original problem&lt;/em&gt;. The duality gap for this primal/dual pair $(x,\gamma,\lambda)$ is&#xA;$$c^Tx-b^T\lambda=c^Tx-t^{-1}b^Ty=c^Tx-t^{-1}x^TA^Ty=(c-t^{-1}A^Ty)^Tx=t^{-1}z^Tx=m/t.$$&#xA;So even though we're looking at the modified barrier model, the set of optimal points $x^*(t)$ are all feasible for the original, and they all lead to feasible dual points $(\gamma^*(t),\lambda^*(t))$ for the dual problem as well. The duality gap for the &lt;em&gt;original&lt;/em&gt; model is $m/t$.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-04-18T16:11:38.477" LastActivityDate="2013-04-18T16:11:38.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6914" PostTypeId="2" ParentId="6911" CreationDate="2013-04-18T14:56:27.173" Score="8" Body="&lt;p&gt;For an unconstrained linear SPD problem, BFGS convergence matches unpreconditioned conjugate gradients.  Differential operators appearing in structural mechanics problems are usually ill-conditioned with many large eigenvalues, making unpreconditioned iterative methods hopelessly slow.  To deal with the troublesome spectrum, structural mechanics solvers depends on either sparse direct methods or preconditioned iterative methods (often using algebraic multigrid or domain decomposition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, BFGS was developed in an optimization setting where most problems have a spectrum that looks like an integral operator under grid refinement, e.g., $I + K$ where $K$ is compact.  For linear versions of these problems, Krylov methods would converge in a constant number of iterations, independent of resolution.  The Hessian is often dense and practically inaccessible, so it is applied in unassembled form and only the identity or a diagonal scaling can be used as the &quot;seed&quot; Hessian $H_0$.  BFGS is thus a way to &lt;em&gt;improve&lt;/em&gt; a crude approximation to the inverse Hessian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When solving differential problems, we can use BFGS as a &lt;em&gt;lagging&lt;/em&gt; method, starting with a good approximation that is merely expensive to construct.  This is useful when nonlinearity represents a part of the difficulty in solving a problem, but ill-conditioning is significant enough that preconditioning is needed.  This turns out to work pretty well for some problems like elasticity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Peter Brune and I wrote a short paper on this recently:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://59A2.org/na/BrownBrune-LowRankRobustLagging-2013&quot;&gt;Brown and Brune &lt;strong&gt;Low-rank quasi-Newton updates for robust Jacobian lagging in Newton methods&lt;/strong&gt; 2013&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See Table II on page 9 for a large-deformation elasticity example.  The standard Newton method needs 10 Jacobian evaluations, 11 residual evaluations, and 77 preconditioner applications (a linear V-cycle of algebraic multigrid; note that we can use fewer total preconditioner applications using inexact solves, but this will increase the number of Jacobian evaluations).  LBFGS-6 with the same V-cycle of AMG used for $H_0$ requires 4 Jacobian evaluations, 49 residual evaluations, and 24 preconditioner applications.  LBFGS lets us trade Jacobian work for more residual evaluations while retaining faster convergence than normal Jacobian lagging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can try this out in &lt;a href=&quot;http://mcs.anl.gov/petsc&quot;&gt;PETSc&lt;/a&gt; using only command-line options, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-snes_type qn -snes_qn_restart_type periodic -snes_qn_scale_type jacobian&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The utility of this method for constrained elasticity will depend on how quickly the constraints are changing, but I expect it to be useful.  We are planning to extend PETSc's support for solving constrained systems so that these methods can be used for elastic contact problems.  Please let us know if you're interested in this.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-04-18T15:02:55.823" LastActivityDate="2013-04-18T15:02:55.823" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6915" PostTypeId="1" AcceptedAnswerId="6929" CreationDate="2013-04-18T16:16:52.967" Score="2" ViewCount="2167" Body="&lt;p&gt;Is there a way &lt;strong&gt;in&lt;/strong&gt; Tecplot to change data from face-centred to node-based?&lt;br&gt;&#xA;If I look at the plot, it seems to me that internally Tecplot is changing the data from being at the centre of the face to being at the corner nodes - is this a correct observation and can this data be written to a file?&lt;/p&gt;&#xA;" OwnerUserId="4067" LastActivityDate="2013-04-19T17:11:54.080" Title="tecplot 360: convert face-centered data to node data" Tags="|software|plotting|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6916" PostTypeId="2" ParentId="6902" CreationDate="2013-04-18T16:22:14.783" Score="4" Body="&lt;p&gt;To build on Bill Barth's commment, there really isn't any depth here. The equations you're trying to solve are:&#xA;\begin{align}&#xA;u_t + uu_x &amp;amp;= - \frac{1}{\rho}p_x + \nu u_{xx} \\&#xA;u_x &amp;amp; = 0&#xA;\end{align}&#xA;Now, $u_x = 0$ immediately implies $u$ has no space-dependence, so we can write $u = f(t)$, for some time-dependent function $f: \mathbb{R} \to \mathbb{R}$. The momentum equation then reduces to&#xA;$$ f'(t) = -\frac{1}{\rho}p_x(x,t).$$&#xA;Integrating this, we get&#xA;$$p = -\rho x f'(t) + g(t),$$&#xA;where $g :\mathbb{R} \to \mathbb{R}$ is another function of time. (Note that $g$ is irrelevant to the dynamics of the system, as there is dependence only on the spatial derivatives of $p$.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, the solution space is small and uninteresting, and we haven't even applied boundary conditions. Homogenous Dirichlet boundary conditions on $u$, for example, imply that $u \equiv 0$ is the only solution. Even periodic boundary conditions, by requiring that $p$ be periodic, lead to the condition that $f'(t) \equiv 0$, so $u \equiv C$ for some constant $C$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, what you're looking for simply doesn't exist.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastActivityDate="2013-04-18T16:22:14.783" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6917" PostTypeId="2" ParentId="6890" CreationDate="2013-04-18T17:25:27.823" Score="3" Body="&lt;p&gt;As J.M. pointed out, the standard stable formulas for arbitrary cubics also work for this case, and happily they nicely simplify.  The arbitrary formulas are &lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;id=1aAOdzK3FegC&amp;amp;pg=PA228#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;given here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the special case of a depressed cubic $x^3 - ax - b = 0$ with linear and constant coefficients negative, let $Q = a/3$, $R = -q/2$.  Then $Q &amp;gt; 0$, $R &amp;lt; 0$, and the unique positive root is given by&#xA;$$x = \begin{cases}&#xA;  -2 \sqrt{Q} \cos{\frac{2\pi+\cos^{-1} {R/Q^{3/2}}}{3}} &amp;amp; \mbox{if } R^2 &amp;lt; Q^3 \\&#xA;  A + \frac{Q}{A} &amp;amp; \mbox{if } R^2 \ge Q^3 \\&#xA;\end{cases} $$&#xA;where&#xA;$$A = \left(-R + \sqrt{R^2 - Q^3} \right)^{1/3}$$&#xA;Note that the first case is always positive since $R/Q^{3/2} &amp;lt; 0$ so that&#xA;$$\frac{\pi}{2} &amp;lt; \cos^{-1} \frac{R}{Q^{3/2}} &amp;lt; \pi,$$&#xA;eliminating the need to check through the other real roots (which are guaranteed to be negative).&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-04-18T17:25:27.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6918" PostTypeId="1" CreationDate="2013-04-18T17:27:51.403" Score="2" ViewCount="527" Body="&lt;p&gt;I tried to use PETSC direct solver (SUPER LU) to solve the system of equations.&#xA;After a long time effort, the code is working properly and gives some good results. The only issue that came up recently is a bout memory leakage.&#xA;I found that after iterations RES memory is increasing which leads to the server crash.&#xA;I tried bunch of cures which I found on the web, but none of them worked.&#xA;I am using MatDestroy and VecDestroy at the end of PETSC part in the code to free all the allocated memories. This is not helpful.&#xA;I also tried iterative solvers like Richardson and other direct solver like spooles and the same problem occurred.&#xA;I would appreciate if you could help me in this regard.&lt;/p&gt;&#xA;" OwnerUserId="4122" LastActivityDate="2013-04-18T17:27:51.403" Title="PETSC Memory Leak" Tags="|petsc|" AnswerCount="0" CommentCount="3" ClosedDate="2013-04-22T00:39:24.780" ContentLicense="CC BY-SA 3.0" />
  <row Id="6919" PostTypeId="2" ParentId="5423" CreationDate="2013-04-18T21:06:40.447" Score="5" Body="&lt;p&gt;If you specify your mesh as a DMPlex and your data layout as a PetscSection, then DMCreateMatrix() will give you the correctly preallocated matrix automatically. Here are PETSc examples for the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/src/snes/examples/tutorials/ex12.c.html&quot; rel=&quot;noreferrer&quot;&gt;Poisson Problem&lt;/a&gt; and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/src/snes/examples/tutorials/ex62.c.html&quot; rel=&quot;noreferrer&quot;&gt;Stokes Problem&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2013-04-18T21:06:40.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6920" PostTypeId="2" ParentId="5213" CreationDate="2013-04-19T01:21:35.783" Score="2" Body="&lt;p&gt;No, constraints of the form $ab-cd\geq 0$ are not convex. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can prove this by showing that the set $$\mathcal{C}\triangleq \{(a,b,c,d)\,|\,ab-cd\geq 0\}$$ fails the standard midpoint test for convex sets. That is, given any two points $(a_1,b_1,c_1,d_1)$ and $(a_2,b_2,c_2,d_2)$ in $\mathcal{C}$, the midpoint &#xA;$$(a_3,b_3,c_3,d_3)=(a_1+a_2,b_1+b_2,c_1+c_2,d_1+d_2)/2$$&#xA;must be in $\mathcal{C}$ as well. Let us choose&#xA;$$(a_1,b_1,c_1,d_1)=(2,2,1,1) \quad \text{and} \quad (a_2,b_2,c_2,d_2)=(-2,-2,1,1).$$&#xA;Since $a_ib_i-c_id_i=4-1=3\geq 0$ in both cases, both points are in $\mathcal{C}$. But the midpoint $$(a_3,b_3,c_3,d_3)=(0,0,1,1)$$&#xA;does not, since $a_3b_3-c_3d_3=0-1=-1\not\geq 0$. So the set is not convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus your problem is not a convex optimization problem. This necessarily means that it cannot be represented using semidefinite programming, either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you can somehow create a new, convex model for your application, you will not be able to solve it using convex methods.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2013-04-19T01:21:35.783" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6921" PostTypeId="1" AcceptedAnswerId="7324" CreationDate="2013-04-19T03:00:37.743" Score="1" ViewCount="192" Body="&lt;p&gt;I have a completely beginner question about the setup of artificial neural networks.  Basically it boils down to: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do I put in data and get results?  For example, classification: let say I have 10 input features, all numerical, and 5 output classes. &lt;/li&gt;&#xA;&lt;li&gt;How do I set up my input layer and my output layer?  Would it be good to have 10 input nodes, one for each feature and feed each node a single numerical value during feed forward evaluation?  &lt;/li&gt;&#xA;&lt;li&gt;How about the output layer?  Should I set up 1 node, 5 nodes? &lt;/li&gt;&#xA;&lt;li&gt;What is the classification rule for neural networks?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4123" LastEditorUserId="549" LastEditDate="2013-05-22T16:58:15.470" LastActivityDate="2013-05-22T16:58:15.470" Title="How to setup a Neural Network?" Tags="|machine-learning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6922" PostTypeId="2" ParentId="5423" CreationDate="2013-04-19T11:47:28.723" Score="0" Body="&lt;p&gt;You're looking to enumerate all unique (gi,gj) connections, which suggests placing them all into an (non-duplicating) associative container and then counting its cardinality - in C++ this would be a std::set &amp;lt; std::pair &amp;lt; int,int &gt; &gt;. In your pseudocode, you'd replace &quot;nnz[i]++&quot; with &quot;s.insert[pair(gi,gj)]&quot;, and then the final number of nonzeros is s.size(). It should run in O(n-log-n) time, where n is the number of nonzeros. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you probably already know the range of possible gi's, you can &quot;splay&quot; the table by the gi index to improve performance. This replaces your set with a std::vector &amp;lt; std::set &amp;lt; int &gt; &gt;. You populate that with &quot;v[gi].insert(gj)&quot;, then the total number of nonzeros comes from summing v[gi].size() for all gi's. This should run in O(n-log-k) time, where k is the number of unknowns per element (six for you - essentially a constant for most pde codes, unless you're talking about hp-methods). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note - wanted this to be a comment on the selected answer, but was too long - sorry!)&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2013-04-19T11:47:28.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6924" PostTypeId="1" CreationDate="2013-04-19T13:10:32.013" Score="7" ViewCount="2667" Body="&lt;p&gt;Consider minimization of a biconvex function over a biconvex set. Is the biconvex optimization problems polynomially solvable?&lt;/p&gt;&#xA;" OwnerUserId="917" LastActivityDate="2018-04-07T23:19:35.660" Title="Biconvex optimization problems" Tags="|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6925" PostTypeId="2" ParentId="6924" CreationDate="2013-04-19T13:31:10.173" Score="10" Body="&lt;p&gt;For the general audience: a &lt;em&gt;biconvex optimization problem&lt;/em&gt; is a problem of the form:&#xA;$$\begin{array}{ll} \text{minimize} &amp;amp; f(x,y) \\ \text{subject to} &amp;amp; (X,Y)\in\mathcal{B} \end{array}$$&#xA;which is convex in $X$ for any fixed $Y$, convex in $Y$ for any fixed $X$, but is not convex in both $(X,Y)$ jointly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No, biconvex problems are not polynomially solvable. They may have lots of local minima, so without knowing more about a special case, global optimization is the only alternative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that the standard, obvious heuristic for a biconvex problem is to fix $X$ and minimize over $Y$, then fix $Y$ and minimize over $X$, and repeat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www2.math.uni-wuppertal.de/~klamroth/publications/gopfkl07.pdf&quot;&gt;Here&lt;/a&gt; is a reference you might find useful. I used this reference here. EDIT: the heuristic I describe above is called &quot;Alternate Convex Search&quot; in this reference; Algorithm 4.1.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2013-04-19T13:31:10.173" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6926" PostTypeId="1" AcceptedAnswerId="6928" CreationDate="2013-04-19T14:49:37.987" Score="2" ViewCount="507" Body="&lt;p&gt;With the Sun Grid Engine, how can I get the directory where the job script is located, within the job script?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need this so I can start the script with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd script-location&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to make sure it'll have access to all the necessary files.  So far I have hard-coded the script directory into each script, which makes it much too easy to make mistakes.&lt;/p&gt;&#xA;" OwnerUserId="605" LastActivityDate="2013-04-19T16:19:44.423" Title="Get job script directory with SGE" Tags="|hpc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6928" PostTypeId="2" ParentId="6926" CreationDate="2013-04-19T16:19:44.423" Score="2" Body="&lt;p&gt;The &lt;code&gt;#$ -cwd&lt;/code&gt; option to your job script will guarantee that SGE places you in the submission directory automatically so that you don't have to do &lt;code&gt;cd script-location&lt;/code&gt; manually. The environment variable &lt;code&gt;SGE_O_WORKDIR&lt;/code&gt; is defined and available to your job script as the working directory that you submitted it from as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These and other things are described in the man page for qsub.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-04-19T16:19:44.423" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6929" PostTypeId="2" ParentId="6915" CreationDate="2013-04-19T17:11:54.080" Score="1" Body="&lt;p&gt;The data can be interpolated when importing it into Tecplot.&lt;br&gt;&#xA;If the cell-centred data is imported from a CGNS-file, the &lt;a href=&quot;http://download.tecplot.com/360/current/tpum.pdf&quot; rel=&quot;nofollow&quot;&gt;Tecplot manual&lt;/a&gt; has this to offer:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Load Cell-centered Data Directly: Toggle-on to load cell-centered data directly [default]. When the option is toggled-off, cell-centered data will be averaged to the nodes (using the averaging method specified below).&lt;/li&gt;&#xA;&lt;li&gt;Averaging: This option is available only if “Load Cell-centered Directly” is not selected. When the field variables are stored at cell centers, either Laplacian averaging or arithmetic averaging may be used to average the cell data to the nodes they surround. This can result in a bias at the boundary nodes. Arithmetic averaging is automatically used for ordered/structured zones. When available, Rind data is used in the averaging.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4067" LastActivityDate="2013-04-19T17:11:54.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6930" PostTypeId="1" CreationDate="2013-04-20T00:13:15.800" Score="4" ViewCount="257" Body="&lt;p&gt;First of all, I want to say that I am extremely new to MD simulations, so I apologize for any of these questions if they appear to be something that should be well known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the classical type of picture that MD packages such as Gromacs give us are not really useful for simulating chemical reactions and non local electron phenomenon, so I am interested in pinning down a very precise and accurate simulation method before beginning a lot of work in the MD area.  I also know that many people really like car-parrinello molecular dynamics (CPMD) in order to circumvent the problems classical simulations (like those that can be run with Gromacs) has with only dealing with localized electrons.&lt;br&gt;&#xA;However, &lt;a href=&quot;http://prl.aps.org/abstract/PRL/v55/i22/p2471_1&quot; rel=&quot;nofollow&quot;&gt;CPMD was developed in 1985&lt;/a&gt; So, I was wondering if there are any more recent advancements in doing highly accurate quantum mechanical modeling that will give better results than just CPMD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am interested in exactly what types of properties these simulations will give you.&#xA;I am &lt;em&gt;most&lt;/em&gt; interested in adsorption enthalpies between two materials and optical properties, such as exciton energies and optical oscillator strengths etc.  But it would be nice to know if there are any properties MD &lt;em&gt;can't&lt;/em&gt; give &lt;/p&gt;&#xA;" OwnerUserId="4131" LastActivityDate="2013-04-20T16:45:03.757" Title="What is the most accurate molecular dynamic simulation method" Tags="|molecular-dynamics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6932" PostTypeId="1" AcceptedAnswerId="6935" CreationDate="2013-04-20T05:45:18.810" Score="5" ViewCount="2829" Body="&lt;p&gt;I plan to write a code that solves the flow past a cylinder and try to see the Von-Karman vortex street. I will solve the 2-D viscous, incompressible Navier-Stokes using the projection method. The following figure demonstrates a subset of my intended boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sort of confused about the decoupling of velocity and pressure for the projection method. Since we decouple pressure and velocity how should, for example, the boundary condition for pressure be at the surface of the cylinder? The problem is that we have a whole new set of boundary conditions to set. How are they supposed to be determined? Would I not have to know the physics of the flow beforehand to prescribe these conditions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QGYdm.jpg&quot; rel=&quot;noreferrer&quot;&gt;https://i.stack.imgur.com/QGYdm.jpg&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4134" LastActivityDate="2016-06-20T19:45:29.420" Title="Flow past a cylinder - Projection Method - Boundary Conditions" Tags="|fluid-dynamics|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6933" PostTypeId="1" CreationDate="2013-04-20T16:03:57.093" Score="3" ViewCount="341" Body="&lt;p&gt;In FEM applications, each processor needs only data from neighboring processors, which gives a stencil or star pattern of communication. In MPI_Win for MPI_Get, one needs to setup a group of processors to window on common data.  But what we usually need is specifying a stencil pattern for MPI_Get. And it seems to me MPI_Win is a lot of waste, isn't it?  Maybe there is &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2846/simulating-the-mpi-isend-irecv-wait-model-with-one-sided-communication&quot;&gt;a related question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-07-04T01:54:33.557" Title="Is there MPI_Star instead of MPI_Win (window) for MPI_Get?" Tags="|finite-element|parallel-computing|petsc|mpi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6934" PostTypeId="2" ParentId="6930" CreationDate="2013-04-20T16:45:03.757" Score="5" Body="&lt;p&gt;The area you are asking about is known formally as &lt;em&gt;ab initio molecular dynamics&lt;/em&gt;. Marx and Hütter a few years ago wrote a &lt;a href=&quot;http://www.cambridge.org/gb/knowledge/isbn/item2327682/?site_locale=en_GB&quot;&gt;comprehensive monograph&lt;/a&gt; on recent updates in the method, and can be recommended for providing a useful summary of the (almost) current state-of-the-art in the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one thing that should definitely be considered before doing any serious amount of work in the MD community is understanding what the relevant time and length scales you are working with, and how well those translate into the particular branch of MD you want to run. For instance, you aren't going to be able to simulate 10,000 or 100,000 atoms for 100 nanoseconds using CPMD or any other branch of ab initio molecular dynamics. Similarly, if you want to observe processes occurring at the electronic scale, as you mentioned, classical MD is a poor choice.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2013-04-20T16:45:03.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6935" PostTypeId="2" ParentId="6932" CreationDate="2013-04-20T17:56:10.340" Score="7" Body="&lt;p&gt;The answer depends on the physical boundary condition for the velocity field. Since you are using the incompressible Navier-Stokes equations, in contrast to the Euler equations, the condition $\mathbf{u} \cdot \mathbf{n} = 0$ at the cylinder's surface underdetermines the problem. In contrast, a typical boundary condition for viscous flow past a cylinder is $\mathbf{u} = 0$ at the boundary; this is the &lt;em&gt;no-slip&lt;/em&gt; condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Physically, of course, there is no boundary condition on pressure, but the nature of the projection method requires us to furnish a numerical boundary condition nonetheless. Assuming a no-slip cylinder boundary, the appropriate boundary condition on the pressure update $\phi$ is then&#xA;$$\frac{\partial \phi}{\partial \mathbf{n}} = 0.$$&#xA;In other words, the pressure update $\phi$ should have a Neumann boundary condition across the cylinder boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why this is the case, consider the following. In the projection method, given a value $\mathbf{u}^n$ for the velocity field at time step $n$, we typically solve the Navier-Stokes equations using a lagged pressure term to obtain $\mathbf{u}^*$. We then step forward by projecting out the resulting incompressible part,&#xA;$$\mathbf{u}^{n+1} = \mathcal{P}\mathbf{u}^*.$$&#xA;Using the Helmholtz decomposition to write $\mathbf{u}^* = \nabla \phi + \mathbf{v}$, where $\mathbf{v}$ is divergence-free, we can rewrite the update as&#xA;$$\mathbf{u}^{n+1} = \mathbf{u}^* - \Delta t \nabla \phi^{n+1}.$$&#xA;Taking the divergence of both sides and appealing to the Helmholtz decomposition again, we obtain&#xA;$$\Delta t \Delta \phi^{n+1} = \nabla \cdot \mathbf{u}^*,$$&#xA;which is the equation we solve to determine the pressure correction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a no-slip boundary, we can take the dot product of both sides of the velocity update with the boundary normal vector $\mathbf{n}$ to obtain&#xA;$$\mathbf{u}^{n+1}\cdot\mathbf{n} = \mathbf{u}^*\cdot\mathbf{n} - \Delta t  \mathbf{n}\cdot\nabla \phi.$$&#xA;But since $\mathbf{u}^{n+1}$ necessarily obeys the no-slip boundary condition, we have $\mathbf{u}^{n+1} = 0$ at the boundary. Furthermore, since $\mathbf{u}^*$ was obtained by enforcing the no-slip boundary condition, we have that $\mathbf{u}^* = 0$ at the boundary as well. This leaves us with the constraint that, at the boundary,&#xA;$$\frac{\partial \phi}{\partial \mathbf{n}} = \Delta t  \mathbf{n}\cdot\nabla \phi = 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details, I suggest taking a look at [1]. This paper discusses the projection method in a finite-difference formulation, but the pressure update is handled within a finite element framework, and the analysis should apply to your situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] A. Almgren, J.B. Bell, and W. Szymczak, A numerical method for the incompressible Navier-Stokes equations based on an approximate projection, SIAM J. Sci. Comput. 17 (1996), pp. 258-369.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="14659" LastEditDate="2016-06-20T19:45:29.420" LastActivityDate="2016-06-20T19:45:29.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6936" PostTypeId="2" ParentId="6903" CreationDate="2013-04-20T22:57:46.697" Score="0" Body="&lt;p&gt;You asked how to prove a function is convex---but it looks like your real question is how to prove that your local minimum is global. There are plenty of non-convex functions whose local minima are also global, and sometimes, you can even prove it :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option you might consider is to employ a branch-and-bound global optimization approach. That's a systematic way for exploring the entire region of interest. In theory this could be very expensive, but in practice, you can often prune away large fractions of the region of interest, saving you a lot of computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some &lt;a href=&quot;http://www.stanford.edu/class/ee364b/lectures/bb_slides.pdf&quot; rel=&quot;nofollow&quot;&gt;slides&lt;/a&gt; and &lt;a href=&quot;http://www.stanford.edu/class/ee364b/notes/bb_notes.pdf&quot; rel=&quot;nofollow&quot;&gt;lecture notes&lt;/a&gt; by Stephen Boyd and Jacob Mattingley on the topic. To use this, you not only need to be able to compute local minima of your function (something you currently have) but the ability to compute good &lt;em&gt;lower bounds&lt;/em&gt; for the function over arbitrary regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think this approach may have merit for your application, give us a little more detail about your problem (by editing your question) and perhaps I can fill in some more detail.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2013-04-20T22:57:46.697" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="6941" PostTypeId="2" ParentId="6932" CreationDate="2013-04-21T19:24:23.963" Score="2" Body="&lt;p&gt;The pressure-like variable of the family of projection approaches is actually a mathematical variable for which the relation to physical pressure depends completely on what projection method you choose as well as what time-integration scheme you have picked for the provisional momentum equation. This means that the 'pressure' boundary condition should not be based on physical arguments. It must instead be formulated to be consistent with your numerics, particularly how you treat the velocity boundary conditions on intermediate steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple of good references:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;David Brown , Ricardo Cortez , Michael Minion, Accurate Projection Methods for the Incompressible Navier-Stokes Equations,JCP, 168, 464–499 (2001).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;J.-L. GUERMOND, P. MINEV, J. SHEN, An Overview of Projection methods for incompressible flows, Computer Methods in Applied Mechanics and Engineering, 195 (2006) 6011--6045.&lt;/p&gt;&#xA;" OwnerUserId="1453" LastActivityDate="2013-04-21T19:24:23.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6942" PostTypeId="2" ParentId="6722" CreationDate="2013-04-22T03:13:43.393" Score="2" Body="&lt;p&gt;A proof is purportedly offered in the paper: &quot;Computational complexity and fundamental limitations to fermionic quantum Monte Carlo simulations&quot;. &lt;a href=&quot;http://arxiv.org/abs/cond-mat/0408370&quot; rel=&quot;nofollow noreferrer&quot;&gt;Click here for the arXiv link&lt;/a&gt;. Here is the abstract:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Quantum Monte Carlo simulations, while being efficient for bosons,&#xA;  suffer from the &quot;negative sign problem'' when applied to fermions -&#xA;  causing an exponential increase of the computing time with the number&#xA;  of particles. A polynomial time solution to the sign problem is highly&#xA;  desired since it would provide an unbiased and numerically exact&#xA;  method to simulate correlated quantum systems. Here we show, that such&#xA;  a solution is almost certainly unattainable by proving that the sign&#xA;  problem is NP-hard, implying that a generic solution of the sign&#xA;  problem would also solve all problems in the complexity class NP&#xA;  (nondeterministic polynomial) in polynomial time.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Apparently the folks in statistical mechanics are far more familiar than I am with this. &lt;a href=&quot;https://physics.stackexchange.com/questions/11605/when-is-the-minus-sign-problem-in-quantum-simulations-an-obstacle&quot;&gt;This post on the Physics StackExchange&lt;/a&gt; talks about the sign problem, too---and &lt;a href=&quot;https://physics.stackexchange.com/questions/1453/great-unsolved-physics-problems/1966#1966&quot;&gt;here as well&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2013-04-22T03:13:43.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6943" PostTypeId="1" AcceptedAnswerId="6944" CreationDate="2013-04-22T08:30:53.210" Score="10" ViewCount="1806" Body="&lt;p&gt;In the method of manufactured solutions (MMS) one postulates an exact solution, substitutes it in the equations and calculates the corresponding source term. The solution is then used for code verification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For incompressible Navier-Stokes equations, MMS easily leads to a (non-zero) source term in the continuity equation. But not all codes allow source terms in the continuity equations, so for these codes only manufactured solutions with a divergence-free velocity fields will do. I found this example for a domain $\Omega=[0,1]^2$&#xA;\begin{align}&#xA;u_1 &amp;amp;= -\cos(\pi x) \sin(\pi y) \\&#xA;u_2 &amp;amp;= \sin(\pi x) \cos(\pi y)&#xA;\end{align}&#xA;In general 3D cases, how does one manufacture a divergence-free velocity field? &lt;/p&gt;&#xA;" OwnerUserId="1973" LastEditorUserId="9667" LastEditDate="2015-08-20T20:14:33.420" LastActivityDate="2015-08-20T20:14:33.420" Title="manufactured solutions for incompressible Navier-Stokes -- how to find divergence-free velocity fields?" Tags="|navier-stokes|incompressible|verification|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6944" PostTypeId="2" ParentId="6943" CreationDate="2013-04-22T12:01:17.470" Score="7" Body="&lt;p&gt;Use a vector streamfunction or take the cross product of two gradients. I.e.:&#xA;$$&#xA;\boldsymbol{u}=\nabla\times\boldsymbol{A}&#xA;$$&#xA;where $\boldsymbol{A}$ is a vector field of your choosing, or&#xA;$$&#xA;\boldsymbol{u}=\nabla f\times\nabla g&#xA;$$&#xA;where $f$ and $g$ are two scalar fields of your choosing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's hard both have the velocity be divergence-free and prescribe the boundary conditions, but as long as your code allows you to set arbitrary functions for your boundary conditions, you should be OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ETA: Of course, your momentum equation will have to accept a forcing function, but I've always felt better about forcing the momentum equation than I have adding a right-hand side to the continuity equation.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-04-22T15:00:11.773" LastActivityDate="2013-04-22T15:00:11.773" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6945" PostTypeId="2" ParentId="6943" CreationDate="2013-04-22T14:10:41.963" Score="5" Body="&lt;p&gt;This is not a general answer, but for the Navier-Stokes equations, there are manufactured solutions that describe real flow. For example, the Kovasznay flow field is a popular choice:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/article/10.1007/BF00948290&quot;&gt;http://link.springer.com/article/10.1007/BF00948290&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original reference is: Kovasznay L.I.G., &quot;Laminar flow behind a two-dimensional grid&quot;. Proc. Cambridge Philos. Soc., page 44, 1948.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-22T14:10:41.963" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6946" PostTypeId="1" CreationDate="2013-04-22T19:03:27.363" Score="1" ViewCount="511" Body="&lt;p&gt;I was reading through &lt;a href=&quot;https://developer.nvidia.com/content/how-implement-performance-metrics-cuda-cc&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt; (very good article and excellent blog BTW) to do some measurements in my (very simple) implementation of the Mandelbrot Set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using a Quadro 2000D card which has a theoretical bandwidth of approximately 20 GB/s ( = 1304 MHz * 128 bit-bus).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mandelbrot image has 700x700 pixels (uchar = 1 byte per pixel). Launching the kernel with a grid(44,44) and block(16,16) the time elapsed to compute it is 1.00026 ms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if my calculations are good then I have the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(490000~bytes)(10^{-9}~GB)~~\big/~~(1.00026)(10^{-3~}s)=0.4898~GB/s$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I interpret this result? I am only using approx 2.5% of the bandwidth, which is bad but since the algorithm is doing only a write per pixel there's no way to improve this, is there? Is there something I am not taking into account?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did three types of kernels to see what was going on and reduced the size of the image to 250x250.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First the very naive, 1 thread doing all the work: 125 ms + 3.1% Global efficiency store + 2% occupancy. This is BAD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, one thread per row: 0.20 ms + 97.7% Global store efficiency + 16% occupancy. Much better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, one thread per element: 0.25 ms with 50% Global store efficiency + 55% occupancy. Worst than the previous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, from here what can I do to improve? Is there way to do it?&#xA;The only thing it occurs to me in the third type is to write inside the kernel to shared memory, then synchronize and after that all threads write to global memory so to have coalesced accesses.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2013-04-22T22:13:50.573" LastActivityDate="2013-04-22T22:13:50.573" Title="CUDA Mandelbrot Set effective bandwidth and optimization" Tags="|parallel-computing|gpu|cuda|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6947" PostTypeId="1" AcceptedAnswerId="6948" CreationDate="2013-04-22T20:10:47.610" Score="1" ViewCount="235" Body="&lt;p&gt;I have an FFT code that solves a particular case of the steady Euler equations where a Poisson equation is solved, what is a good way to quantify the error? Is what I am doing ok?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I do not have an analytic solution to compare with  what I have done is I have computed the solution on a very fine grid and then computed the error as such:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $x_f$ be the fine solution and $x_i$ the solution for coarser grids, the error for each grid $i$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$error_i = \frac{\|x_f-x_i\|}{\|x_f\|}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then plot the error in a loglog plot as a function of number of grid points N. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/KdqeI.jpg&quot; alt=&quot;error vs total number of grid points&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The slope ratio is 1 to 1, so would it mean the scheme is 1st order accurate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also have the residual given by $res = \|LHS-RSH\|$ where the terms represent the left hand side minus the right hand side of the equation and have performed a similar plot, I am not sure what is the best way to interpret it though. Any help or input would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FYyvN.jpg&quot; alt=&quot;Norm of Residual vs total number of grid points&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3561" LastEditorUserId="3561" LastEditDate="2013-04-22T20:47:22.983" LastActivityDate="2013-04-22T20:47:22.983" Title="How can I quantify the error of FFT-based poisson solvers?" Tags="|discretization|fourier-analysis|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6948" PostTypeId="2" ParentId="6947" CreationDate="2013-04-22T20:45:27.750" Score="1" Body="&lt;p&gt;Here is a general procedure to do such error plots that I usually follow:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Find such $N$ so that the solution is converged.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Plot the error (for example $\frac{\|x_f-x_i\|}{\|x_f\|}$ as you did). You can use log-log plot (polynomial convergence will be a line) or log-linear plot (exponential convergence will be a line).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Optionally, plot the expected convergence rate --- in your case plot the function $c N^{-1}$, where $c$ is a constant that you adjust so that this function agrees with your convergence graph for the smallest $N$ shown. If the two lines agree on your graph, then you have a first order convergence. It will be apparent from the graph once you plot it. Use $cN^{-2}$ for quadratic convergence and so on.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note about the error formula:&#xA;Sometimes you can plot some value (for example an energy $E$ in the Schroedinger equation) that converges to the exact value as you increase $N$. In that case, you can just plot $E-E_{conv}$, where you determine $E_{conv}$ from the step 1.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2013-04-22T20:45:27.750" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6949" PostTypeId="1" CreationDate="2013-04-23T02:00:54.013" Score="4" ViewCount="7405" Body="&lt;p&gt;When I use the cvx matlab toolbox, I met a puzzled problem. The function of fft (or dct, wavelet, etc.) cannot be recognized by the type of 'cvx'. For the 1-d fft, it can be constructed to an equivalent matrix. But for the 2-d fft, how to transform it to the matrix form ? &lt;/p&gt;&#xA;" OwnerUserId="4148" LastEditorUserId="3907" LastEditDate="2013-04-23T17:21:52.677" LastActivityDate="2018-05-31T13:22:55.943" Title="how can a 2-d fft be constructed to an equivalent matrix?" Tags="|optimization|fourier-analysis|matrix|cvx|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6950" PostTypeId="2" ParentId="6949" CreationDate="2013-04-23T12:53:04.880" Score="6" Body="&lt;p&gt;Yes, it is true, the CVX package (disclosure: I am the author) does not support the use of the &lt;code&gt;fft()&lt;/code&gt; command in constraint and objective expressions. However, it's relatively simple to get the equivalent result, in both the 1D and 2D cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, 1D. Given a vector $x\in\mathbb{R}^n$, the DFT of $x$ is equal to $Wx$, where $W$ is the so-called DFT matrix. Here are a few ways to construct the DFT matrix in MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;W = fft(eye(n))&#xA;W = exp(1j*2*pi*[0:n-1]'*[0:n-1]/n)&#xA;W = exp(bsxfun(@times,1j*2*pi*[0:n-1]',[0:n-1]/8))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now try this with random vectors to verify correctness:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=randn(n,1)+1j*randn(n,1)&#xA;fft(x)&#xA;W*x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What about 2D ffts? Given a square matrix $X\in\mathbb{R}^{n\times n}$, the 2D DFT can be obtained by applying the 1-D DFT to each column of the matrix, and then applying the 1-D DFT to each row of the matrix. Expressed in terms of the DFT matrix $W$, the 2D DFT is simply $WXW^T$. Go ahead, try the following MATLAB expressions and see that they are equivalent:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = randn(n,n)+1j*randn(n,n)&#xA;fft2(X)&#xA;fft(fft(X,[],2),[],1)&#xA;fft(fft(X).').'&#xA;W*X*W.'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note the use of the &lt;em&gt;non-conjugated&lt;/em&gt; transpose &lt;code&gt;W.'&lt;/code&gt; above; that's important. If you use the standard Hermitian transpose instead, &lt;code&gt;'&lt;/code&gt;, you'll get a scrambled result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, given a square matrix variable &lt;code&gt;X&lt;/code&gt;, you can perform a 2D FFT on a square CVX variable as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;W=fft(eye(size(X));&#xA;W*X*W.'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, note that I am using a non-conjugated transpose above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that 2D DFTs get big fast. That is, you have &lt;code&gt;n^2&lt;/code&gt; variables here if the dimensions of your matrix are &lt;code&gt;[n n]&lt;/code&gt;. So even a 512 by 512 image, not that large by matrix standards, is still over 250k variables, and that can be a lot by CVX standards. Once you are satisfied with your model's performance on smaller problems, you may need to develop your own numerical engine to solve larger cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that it is relatively simple to work around CVX's limitation here, it is reasonable to ask: why not just add support for &lt;code&gt;fft()&lt;/code&gt; in CVX, using one of techniques described here? Well, I may do that. However, it's important to remember that FFT stands for &lt;em&gt;fast&lt;/em&gt; Fourier transform. It is a &lt;em&gt;fast algorithm&lt;/em&gt; for generating the discrete Fourier transform (DFT). Specifically, it can compute the DFT in only $O(N\log N)$ operations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix workaround, on the other hand, constructs a full DFT matrix at a cost of $O(N^2)$ operations---so much for &quot;fast&quot;! I do not want to deceive users with a false sense that CVX is really using the &lt;em&gt;fast&lt;/em&gt; Fourier transform. The fact is, it cannot: the underlying solvers do not exploit the $N\log N$ efficiency of the FFT operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://cvxr.com/cvx/examples/&quot; rel=&quot;nofollow&quot;&gt;CVX example library&lt;/a&gt; contains a number of problems that utilize a manually generated Fourier transform. In particular, look at the filter design examples.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-04-23T17:18:07.987" LastActivityDate="2013-04-23T17:18:07.987" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6951" PostTypeId="1" AcceptedAnswerId="7727" CreationDate="2013-04-23T14:30:06.333" Score="1" ViewCount="262" Body="&lt;p&gt;I want to explore the low energy states of an amino acid or a peptide by generating input geometries, defined by multiple dihedral angles of the backbone or the side chain. From these starting conformations I want to do geometry optimization to yield local energy minimum geometries.&lt;br&gt;&#xA;The following approaches failed:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Using the &lt;em&gt;ModRedundant&lt;/em&gt; keyword in the Gaussian09 package to define new values for the dihedral angles. This crashes for larger molecules and major changes in the coordinates.&lt;/li&gt;&#xA;&lt;li&gt;Detecting the substructure of the molecule (e.g. the backbone) by SMILE / SMARTS pattern and using &lt;em&gt;obrotate&lt;/em&gt; from the openbabel package to change the values. Works fine for a single dihedral or maybe two, but afterwards these changes the pattern of the substructure is not necessarily detected anymore and no further dihedral angle can be changed.&lt;/li&gt;&#xA;&lt;li&gt;Usage of a well chosen Z-Matrix: The great disadvantage is, that I don't know how to yield a Z-Matrix with the wanted dihedrals from a given geometry and therefore I can't start from a pre-optimized structure.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Are there any other attempts to this problem?&lt;/p&gt;&#xA;" OwnerUserId="3486" LastActivityDate="2015-04-16T08:04:59.400" Title="How to manipulate dihedral angles in pre-optimized molecule coordinates?" Tags="|computational-chemistry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6952" PostTypeId="1" CreationDate="2013-04-23T15:18:07.123" Score="2" ViewCount="276" Body="&lt;p&gt;I have a Galerkin solution for a heat equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = \Delta u + f&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with Dirichlet conditions&#xA;$$&#xA;u=0, \qquad x \in \partial\Omega&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time discretization is done using a BDF scheme. How can I accurately and efficiently compute the heat flux on the boundary&#xA;$$&#xA;\sigma = \frac{\partial u}{\partial n}, \qquad x \in \partial\Omega&#xA;$$&#xA;from the Galerkin solution $u_h$. &lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2013-04-24T10:47:13.393" Title="Compute heat flux from Galerkin solution" Tags="|finite-element|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6953" PostTypeId="2" ParentId="6949" CreationDate="2013-04-23T17:23:37.903" Score="3" Body="&lt;p&gt;Say A is a M by N matrix, then fft2(A) = dftmtx(M) * A * dftmtx(N). Also, we have $$\text{vec}(ABC)=(C^T \otimes A)\text{vec}(B)$$&#xA;So you can use Kronecker product to construct the 2D DFT matrix. See &lt;a href=&quot;http://www.mathworks.com/matlabcentral/newsreader/view_thread/252244&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="1614" LastEditorUserId="-1" LastEditDate="2018-05-31T13:22:55.943" LastActivityDate="2018-05-31T13:22:55.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6954" PostTypeId="1" AcceptedAnswerId="6958" CreationDate="2013-04-23T21:41:23.953" Score="9" ViewCount="462" Body="&lt;p&gt;In &lt;a href=&quot;http://parlab.eecs.berkeley.edu/sites/all/parlab/files/LU,%20QR%20and%20Cholesky%20factorizations%20using%20vector%20capabilities%20of%20GPUs.pdf&quot; rel=&quot;noreferrer&quot;&gt;&quot;LU, QR and Cholesky Factorizations using Vector Capabilities of GPUs&quot;, by Vasily Volkov and James Demmel&lt;/a&gt;, there is an interesting way to interpret the latencies, line sizes, and page sizes of a cache from a graph like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/1Z0wr.png&quot; alt=&quot;Figure 1 from the paper&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently, the line size is where the the cache begins to plateau (about 32 bytes in this example). Where did that come from?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And how do we know that the page size is where the second plateau begins for global memory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, why do the latencies eventually go down with a large enough stride length? Shouldn't they keep increasing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this actually works, and it seems to, that'd be awesome because we can get a lot of undocumented data about the cache of a GPU.&lt;/p&gt;&#xA;" OwnerUserId="4155" LastEditorUserId="9" LastEditDate="2013-05-14T13:03:55.200" LastActivityDate="2013-05-16T13:13:07.723" Title="How do Volkov and Demmel experimentally determine the latencies, line sizes, and page sizes of a GPU?" Tags="|performance|gpu|benchmarking|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6955" PostTypeId="2" ParentId="6954" CreationDate="2013-04-23T22:03:17.907" Score="0" Body="&lt;p&gt;I'd say that the paragraph on page 3 that begins &quot;A larger latency indicates more cache misses.&quot; explains most of what you're asking pretty well. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-04-23T22:03:17.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6956" PostTypeId="2" ParentId="6952" CreationDate="2013-04-24T10:47:13.393" Score="3" Body="&lt;p&gt;I don't think you can do it to better than first order if you need the heat flux at the boundary. In the interior, you can use &quot;recovery&quot; methods that approximate the gradient of the solution by averaging from the neighboring cells, but at the boundary you do not have enough neighboring cells. I'm afraid the best you can do is actually integrate the normal derivative of the finite element solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, when integrated over a sufficiently long part of the boundary (in 2d), the approximation should be fairly good. I wouldn't be surprised if you can get it to 1% accurate with a not unreasonable number of cells.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-24T10:47:13.393" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6957" PostTypeId="1" AcceptedAnswerId="6959" CreationDate="2013-04-24T11:42:08.550" Score="0" ViewCount="55" Body="&lt;p&gt;There is unstructured grid which contains only quadrangles cells. Each cell has 4 neighbors, and known them (has a pointer to them). I can iterate through all cells in the grid. Some cells are marked as a &quot;A - red&quot; some as a &quot;B - blue&quot; - see attached picture - &lt;a href=&quot;https://i.stack.imgur.com/ZdTDx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cells form blue and red regions. I need to separate them - I need information which cells belong to regions A1, A2, B1, B2... etc. The number of regions of each type is unknown. &lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2013-04-24T13:13:26.107" Title="Cells segregation" Tags="|c++|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6958" PostTypeId="2" ParentId="6954" CreationDate="2013-04-24T13:11:53.730" Score="9" Body="&lt;p&gt;&lt;em&gt;If this actually works, and it seems to, that'd be awesome because we can get a lot of undocumented data about the cache of a GPU.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A frustrating aspect of high performance computing research is digging through all the undocumented instruction sets and architecture features when trying to tune code.  In HPC, the proof is in the benchmark, whether it is High Performance Linpack or STREAMS.  I'm not sure this is &quot;awesome&quot;, but it is definitely how high performance processing units are assessed and evaluated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Apparently, the line size is where the the cache begins to plateau (about 32 bytes in this example). Where did that come from?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seem to be familiar with the performance concepts of the cache hierarchy, described as the &quot;memory mountain&quot; in Bryant and O'Hallaron's &lt;a href=&quot;http://csapp.cs.cmu.edu/public/instructors.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer Systems: A Programmer's Perspective&lt;/a&gt;, but your question indicates a lack of deep understanding of how each level of cache itself works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QZfL5.gif&quot; alt=&quot;memory mountain&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that a cache contains &quot;lines&quot; of data, that is strips of contiguous memory that are harmonized with a memory location somewhere in the main store.  When accessing memory within a line, we have a cache &quot;hit&quot;, and the latency associated with retrieving this memory is called the latency for that particular cache.  For example a 10 cycle L1 cache latency indicates that every time the memory address requested is already in one of the L1 cache lines, we will expect to retrieve it in 10 cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see from the description of the pointer chasing benchmark, it takes strides of a fixed length through memory.  &lt;em&gt;NB: This benchmark would NOT work as expected in the many modern CPUs that feature &quot;stream-detecting&quot; prefetch units that do not behave like simple caches.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It makes sense then, that the first noticeable performance plateau (starting from stride 1), is a stride that gets no reuse from previous cache misses.  How long does a stride have to be before it steps completely past the previously retrieved cache line?  The length of the cache line!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;And how do we know that the page size is where the second plateau begins for global memory?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, the next plateau of stepping beyond reuse will be when the memory page managed by the operating system has to be swapped with each memory access.  The memory page size, unlike cache line size, is normally a parameter that can be set or managed by the programmer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Furthermore, why do the latencies eventually go down with a large enough stride length? Shouldn't they keep increasing&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is definitely the most interesting part of your question, and it explains how the authors postulate the cache structure (associativity and number of sets) based on the information they have previously gleaned (size of cache lines) and the size of the working set of data they are striding over.  Recall from the experimental setup that the strides &quot;wrap around&quot; the array.  For a large enough stride, the total amount of array data that needs to be held in the cache (the so-called working set), is smaller because more of the data in the array is being skipped.  To illustrate this, you can imagine a person continuously running up and down a set of stairs, only taking every 4th step.  This person is touching less total steps than somebody taking every 2 steps.  This idea is somewhat awkwardly stated in the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When the stride is very large, working set &#xA;  decreases until it again fits in the cache, this time producing &#xA;  conflict misses if cache is not fully associative.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As Bill Barth mentions in his answer, the authors state how they make these calculations in the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The data in Fig. 1 suggests a fully associative 16-entry TLB (no TLB overhead &#xA;  for 128MB array, 8MB stride), a 20-way set associative L1 &#xA;  cache (20KB array at 1KB stride fits in L1), and a 24-way set associative L2 cache (back to L2 hit latency for 768KB array, &#xA;  32KB stride). These are the effective numbers and the real &#xA;  implementation might be different. Six 4-way set-associative L2 &#xA;  caches match this data as well.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2013-05-16T13:13:07.723" LastActivityDate="2013-05-16T13:13:07.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6959" PostTypeId="2" ParentId="6957" CreationDate="2013-04-24T13:13:26.107" Score="3" Body="&lt;p&gt;Let's say you have a cell $K$, then here's an algorithm to find all the cells that are connected with $K$ and have the same color (i.e., to find all cells of a domain):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;domain = ()&#xA;new_cells_list = (K)&#xA;while (new_cells_list not empty)&#xA;    // find all neighbors of recently found cells&#xA;    // that have the same color and that we haven't&#xA;    // visited yet&#xA;    next_cells_list = ()&#xA;    for (T in new_cells_list)&#xA;      for (n=1..4)&#xA;        if (T.neighbor(n).color() == K.color()&#xA;            &amp;amp;&amp;amp;&#xA;            T.neighbor(n) not in domain&#xA;            &amp;amp;&amp;amp;&#xA;            T.neighbor(n) not in new_cells_list&#xA;            &amp;amp;&amp;amp;&#xA;            T.neighbor(n) not in next_cells_list)&#xA;          next_cells_list.append(T.neighbor(n))&#xA;    domain.append (new_cells_list);&#xA;    new_cells_list = next_cells_list;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a given cell $K$, this then gives you all other cells of same color that are connected to it. You can now repeat this: Start with one cell, find its domain. Then find the next cell that is not part of the first domain and find its domain. Find the next cell that is not part of the previous two domains and find its domain. Etc. By counting how often this process is repeated, you get how many domains there are.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-24T13:13:26.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6960" PostTypeId="2" ParentId="6871" CreationDate="2013-04-24T15:03:54.397" Score="1" Body="&lt;p&gt;As per the &lt;a href=&quot;http://dakota.sandia.gov/docs/dakota/5.3/Users-5.3.pdf&quot; rel=&quot;nofollow&quot;&gt;Dakota User Manual&lt;/a&gt;, section 14.4 it appears that the quick answer to this question is: &lt;strong&gt;No&lt;/strong&gt;&lt;br&gt;&#xA;Currently that extent of control over the graphical output is not available.&lt;/p&gt;&#xA;" OwnerUserId="4067" LastActivityDate="2013-04-24T15:03:54.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6961" PostTypeId="1" AcceptedAnswerId="6963" CreationDate="2013-04-24T17:50:24.693" Score="21" ViewCount="9481" Body="&lt;p&gt;I would like to use &lt;code&gt;f2py&lt;/code&gt; with modern Fortran.  In particular I'm trying to get the following basic example to work.  This is the smallest useful example I could generate.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;! alloc_test.f90&#xA;subroutine f(x, z)&#xA;  implicit none&#xA;&#xA;! Argument Declarations !&#xA;  real*8, intent(in) ::  x(:)&#xA;  real*8, intent(out) :: z(:)&#xA;&#xA;! Variable Declarations !&#xA;  real*8, allocatable :: y(:)&#xA;  integer :: n&#xA;&#xA;! Variable Initializations !&#xA;  n = size(x)&#xA;  allocate(y(n))&#xA;&#xA;! Statements !&#xA;  y(:) = 1.0&#xA;  z = x + y&#xA;&#xA;  deallocate(y)&#xA;  return&#xA;end subroutine f&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;n&lt;/code&gt; is inferred from the shape of input parameter &lt;code&gt;x&lt;/code&gt;.  Note that &lt;code&gt;y&lt;/code&gt; is allocated and deallocated within the body of the subroutine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I compile this with &lt;code&gt;f2py&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f2py -c alloc_test.f90 -m alloc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then run in Python&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from alloc import f&#xA;from numpy import ones&#xA;x = ones(5)&#xA;print f(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the following error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: failed to create intent(cache|hide)|optional array-- must have defined dimensions but got (-1,)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I go and create and edit the &lt;code&gt;pyf&lt;/code&gt; file manually&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f2py -h alloc_test.pyf -m alloc alloc_test.f90&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Original&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python module alloc ! in &#xA;    interface  ! in :alloc&#xA;        subroutine f(x,z) ! in :alloc:alloc_test.f90&#xA;            real*8 dimension(:),intent(in) :: x&#xA;            real*8 dimension(:),intent(out) :: z&#xA;        end subroutine f&#xA;    end interface &#xA;end python module alloc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Modified&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python module alloc ! in &#xA;    interface  ! in :alloc&#xA;        subroutine f(x,z,n) ! in :alloc:alloc_test.f90&#xA;            integer, intent(in) :: n&#xA;            real*8 dimension(n),intent(in) :: x&#xA;            real*8 dimension(n),intent(out) :: z&#xA;        end subroutine f&#xA;    end interface &#xA;end python module alloc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now it runs but the values of the output &lt;code&gt;z&lt;/code&gt; are always &lt;code&gt;0&lt;/code&gt;.  Some debug printing reveals that &lt;code&gt;n&lt;/code&gt; has the value &lt;code&gt;0&lt;/code&gt; within the subroutine &lt;code&gt;f&lt;/code&gt;.  I assume that I'm missing some &lt;code&gt;f2py&lt;/code&gt; header magic to manage this situation properly.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally what is the best way to link the above subroutine into Python?  I'd strongly prefer not to have to modify the subroutine itself.&lt;/p&gt;&#xA;" OwnerUserId="19" LastEditorUserId="19" LastEditDate="2013-04-24T18:26:05.343" LastActivityDate="2020-06-16T17:37:00.877" Title="F2Py with allocatable and assumed shape arrays" Tags="|python|fortran|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6962" PostTypeId="1" CreationDate="2013-04-24T17:59:48.250" Score="5" ViewCount="5793" Body="&lt;p&gt;SEE: &lt;a href=&quot;http://www.songho.ca/dsp/convolution/convolution2d_example.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.songho.ca/dsp/convolution/convolution2d_example.html&lt;/a&gt;&#xA;&quot;First, flip the kernel, which is the shaded box, in both horizontal and vertical direction&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we need to flip the kernel in 2D convolution in the first place?&#xA;So, why can't we leave it unflipped?&lt;/p&gt;&#xA;" OwnerUserId="3890" LastActivityDate="2013-04-26T23:08:00.577" Title="Flipping the kernel in 2D convolution?" Tags="|image-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6963" PostTypeId="2" ParentId="6961" CreationDate="2013-04-24T20:16:29.290" Score="25" Body="&lt;p&gt;I am not super familiar with f2py internals, but I am very familiar with wrapping Fortran. F2py just automates some or all of the things below.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You first need to export to C using the iso_c_binding module, as described for example here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fortran90.org/src/best-practices.html#interfacing-with-c&quot;&gt;http://fortran90.org/src/best-practices.html#interfacing-with-c&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I am the main author of the fortran90.org pages. This is the only platform and compiler independent way of calling Fortran from C. This is F2003, so these days there is no reason to use any other way. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can only export/call arrays with full length specified (explicit shape), that is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integer(c_int), intent(in) :: N&#xA;real(c_double), intent(out) :: mesh(N)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but not assume shape:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real(c_double), intent(out) :: mesh(:)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That is because the C language does not support such arrays itself. There is talk to include such support in either F2008 or later (I am not sure), and the way it would work is through some supporting C data structures, as you need to carry shape information about the array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Fortran, you should mainly use the assume shape, only in special cases you should use explicit shape, as described here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fortran90.org/src/best-practices.html#arrays&quot;&gt;http://fortran90.org/src/best-practices.html#arrays&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That means, that you need to write a simple wrapper around your assume shape subroutine, that will wrap things into explicit shape arrays, per my first link above.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once you have a C signature, just call it from Python in any way you like, I use Cython, but you can use ctype, or C/API by hand.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;code&gt;deallocate(y)&lt;/code&gt; statement is not needed, Fortran deallocates automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fortran90.org/src/best-practices.html#allocatable-arrays&quot;&gt;http://fortran90.org/src/best-practices.html#allocatable-arrays&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;real*8&lt;/code&gt; should not be used, but rather &lt;code&gt;real(dp)&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fortran90.org/src/best-practices.html#floating-point-numbers&quot;&gt;http://fortran90.org/src/best-practices.html#floating-point-numbers&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The statement &lt;code&gt;y(:) = 1.0&lt;/code&gt; is assigning 1.0 in single precision, so the rest of digits will be random! This is a common pitfall:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fortran90.org/src/gotchas.html#floating-point-numbers&quot;&gt;http://fortran90.org/src/gotchas.html#floating-point-numbers&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to use &lt;code&gt;y(:) = 1.0_dp&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of writing &lt;code&gt;y(:) = 1.0_dp&lt;/code&gt;, you can just write &lt;code&gt;y = 1&lt;/code&gt;, that's it. You can assign integer to a floating point number without losing accuracy, and you don't need to put the redundant &lt;code&gt;(:)&lt;/code&gt; in there. Much simpler.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y = 1&#xA;z = x + y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;just use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z = x + 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and don't bother with the &lt;code&gt;y&lt;/code&gt; array at all.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You don't need the &quot;return&quot; statement at the end of the subroutine.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, you should probably be using modules, and just put &lt;code&gt;implicit none&lt;/code&gt; on the module level and you don't need to repeat it in each subroutine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise it looks good to me. Here is the code following the suggestions 1-10 above::&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;module test&#xA;use iso_c_binding, only: c_double, c_int&#xA;implicit none&#xA;integer, parameter :: dp=kind(0.d0)&#xA;&#xA;contains&#xA;&#xA;subroutine f(x, z)&#xA;real(dp), intent(in) ::  x(:)&#xA;real(dp), intent(out) :: z(:)&#xA;z = x + 1&#xA;end subroutine&#xA;&#xA;subroutine c_f(n, x, z) bind(c)&#xA;integer(c_int), intent(in) :: n&#xA;real(c_double), intent(in) ::  x(n)&#xA;real(c_double), intent(out) :: z(n)&#xA;call f(x, z)&#xA;end subroutine&#xA;&#xA;end module&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It shows the simplified subroutine as well as a C wrapper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as f2py, it probably tries to write this wrapper for you and fails. I am also not sure whether it is using the &lt;code&gt;iso_c_binding&lt;/code&gt; module. So for all these reasons, I prefer to wrap things by hand. Then it's exactly clear what is happening.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="236" LastEditorUserId="19" LastEditDate="2013-04-24T21:39:31.963" LastActivityDate="2013-04-24T21:39:31.963" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6964" PostTypeId="2" ParentId="6962" CreationDate="2013-04-24T21:39:31.077" Score="10" Body="&lt;p&gt;The convolution operation is &lt;em&gt;defined&lt;/em&gt; by this flip, e.g. for 1D&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(f * g )(t)\ \stackrel{\mathrm{def}}{=} \int_{-\infty}^\infty f(\tau)\, g(t-\tau)\, d\tau$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there was no flipping it would be called cross-correlation, in 1D this is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(f \star g)(t)\ \stackrel{\mathrm{def}}{=} \int_{-\infty}^{\infty} f^*(\tau)\ g(t+\tau)\,d\tau$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't have to do the flipping if the resulting kernel is no different, e.g. an isometric filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One reason convolution is used instead of cross-correlation is that convolution is a simple multiplication in the fourier domain, e.g. from wikipedia&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{F}\{f * g\} = k\cdot \mathcal{F}\{f\}\cdot \mathcal{F}\{g\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k$ is a constant depending on the normalisation constant of the Fourier transform used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof can be found on wikipedia here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Convolution_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Convolution_theorem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cross-correlation (no flipping) can also be done in the Fourier domain, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{F}\{f\star g\}=(\mathcal{F}\{f\})^* \cdot \mathcal{F}\{g\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $(\mathcal{F}\{f\})^*$ is the complex conjugate of $\mathcal{F}\{f\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another nice property of convolution is that the operation is commutative, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) * g(x) = g(x) * f(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;whereas for cross-correlation it is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even nicer is that convolution of a signal with a dirac delta returns the original signal, shifted, where as for cross-correlation a flipped, shifted, signal is returned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information on wikipedia here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Cross-correlation&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Cross-correlation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although it is just a simple flip, convolution and cross-correlation can give quite different results, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EQDEh.png&quot; alt=&quot;convolution vs correlation&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(note that change in y axes range). The cross-correlation result has a peak exactly at the centre where the two unflipped functions match. Normalised cross-correlation is often used for template matching. The convolution result is different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dsp.stackexchange.com/questions/2654/what-is-the-difference-between-convolution-and-cross-correlation&quot;&gt;https://dsp.stackexchange.com/questions/2654/what-is-the-difference-between-convolution-and-cross-correlation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematicians take: &lt;a href=&quot;https://math.stackexchange.com/questions/353272/whats-the-difference-between-convolution-and-crosscorrelation&quot;&gt;https://math.stackexchange.com/questions/353272/whats-the-difference-between-convolution-and-crosscorrelation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3565" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2013-04-26T23:08:00.577" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6965" PostTypeId="1" CreationDate="2013-04-25T00:10:34.463" Score="2" ViewCount="293" Body="&lt;p&gt;I know some of the places where SPD linar systems arises such as elliptic PDEs and normal equations. Can I have a more comprehensive list of scientific applications which require solving SPD linear system. I am specially interested in cases where matrix is not sparse and cases where matrix might not be available directly&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2013-04-26T04:18:36.157" Title="Problems where SPD linear system arises" Tags="|linear-solver|nonlinear-equations|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6966" PostTypeId="1" CreationDate="2013-04-25T00:17:08.290" Score="3" ViewCount="1033" Body="&lt;p&gt;I'm working through a problem in a textbook as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Consider the $d \times d$ Toeplitz matrix&#xA;$$ A = \left[ \begin{array}{ccccc} 2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA; -1 &amp;amp;2 &amp;amp;1&amp;amp;\ddots&amp;amp;\vdots\\&#xA;0&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;0 \\&#xA;\vdots &amp;amp;\ddots&amp;amp;-1&amp;amp;2&amp;amp;1\\&#xA;0&amp;amp;\cdots&amp;amp;0&amp;amp;-1&amp;amp;2 \end{array} \right].$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find explicitly all the eigenvalues of the Jacobi iteration matrix $B$. Conclude that this iterative scheme diverges.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deriving the matrix $B$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ B = \left[ \begin{array}{ccccc} 0 &amp;amp; -\frac{1}{2} &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA; \frac{1}{2} &amp;amp;0 &amp;amp;-\frac{1}{2}&amp;amp;\ddots&amp;amp;\vdots\\&#xA;0&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;0 \\&#xA;\vdots &amp;amp;\ddots&amp;amp;\frac{1}{2}&amp;amp;0&amp;amp;-\frac{1}{2}\\&#xA;0&amp;amp;\cdots&amp;amp;0&amp;amp;\frac{1}{2}&amp;amp;0 \end{array} \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and solving for the eigenvalues using the difference equation $q_{k+1} +2 \lambda q_k - q_{k-1} = 0$, I found the set of eigenvalues to be &#xA;$$\lambda_{j} = i \cos(\frac{j\pi}{d+1}),  \quad \quad j = 1,2, \dots, d $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then confirmed this fact using the &lt;code&gt;eig&lt;/code&gt; function in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this leads to the conclusion that $\rho(B) = \max|\lambda_j| = |i\cos(\frac{\pi}{d+1})| &amp;lt; 1$, in which case the Jacobi Iteration should definitely converge (although maybe not very quickly). Why, then, does the textbook think that I should conclude the opposite?&lt;/p&gt;&#xA;" OwnerUserId="3892" LastEditorUserId="3892" LastEditDate="2013-04-25T02:20:13.590" LastActivityDate="2013-04-26T00:04:08.343" Title="Jacobi Iteration diverges?" Tags="|iterative-method|eigenvalues|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6967" PostTypeId="2" ParentId="5108" CreationDate="2013-04-25T01:55:26.970" Score="1" Body="&lt;p&gt;Sorry I delayed so long to answer this question, but I wanted to answer this question in case someone else has a problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a solution to this problem by using the jBLAS library (&lt;a href=&quot;http://mikiobraun.github.io/jblas/&quot; rel=&quot;nofollow&quot;&gt;http://mikiobraun.github.io/jblas/&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the sign function $epsilon(M) = U * epsilon(A) * U^\dagger$, if you have an existing matrix matrixR in jBlas:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;val eigenvecs = Eigen.eigenvectors(matrixR)&#xA;val lambda = new DoubleMatrix(4,4, math.signum(eigenvecs(1).get(0,0).real),0,0,0,&#xA;  0,math.signum(eigenvecs(1).get(1,1).real),0,0,&#xA;  0,0,math.signum(eigenvecs(1).get(2,2).real),0,&#xA;  0,0,0,math.signum(eigenvecs(1).get(3,3).real)) //can be extended to larger matrices&#xA;val epsilon = eigenvecs(0).real.mmul(lambda).mmul(eigenvecs(0).real.transpose) //where epsilon = epsilon(M)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3623" LastActivityDate="2013-04-25T01:55:26.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6968" PostTypeId="1" AcceptedAnswerId="6990" CreationDate="2013-04-25T03:01:16.890" Score="8" ViewCount="4950" Body="&lt;p&gt;Given 3D Poisson equation&#xA;$$&#xA;\nabla^2 \phi(x, y, z) = f(x, y, z)&#xA;$$&#xA;and the right hand side and the domain, am I free to impose any boundary conditions (BC) on the function $\phi$, or do they have to be somehow consistent with the right hand side? In particular, if I impose periodic BC, will there be exactly one solution for any right hand side?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let:&#xA;$$&#xA;f(x, y, z) = - 3 \pi^{2} \sin{\left (\pi x \right )} \sin{\left (\pi y \right )} \sin{\left&#xA; (\pi z \right )}&#xA;$$&#xA;and I solve on a box $(0, 1)\times (0, 1) \times (0, 1)$.&#xA;Now any solution must be a sum of $\phi_0+\phi_1$ where:&#xA;$$&#xA;\phi_0(x, y, z) = \sin{\left (\pi x \right )} \sin{\left (\pi y \right )} \sin{\left (\pi z \right )}\,,&#xA;$$&#xA;because $\nabla^2\phi_0 = f$, and $\phi_1$ is any &lt;a href=&quot;http://en.wikipedia.org/wiki/Harmonic_function&quot;&gt;harmonic function&lt;/a&gt; (i.e. $\nabla^2\phi_1 = 0$). Correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I impose zero Dirichlet BC, then $\phi_0$ is the only solution, because it satisfies the BC, satisfies the equation and the solution must be unique. Correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if I impose periodic BC? Does it mean that there will be some harmonic function $\phi_1$ such that $\phi_0+\phi_1$ satisfies the periodic BC and solves the equation? What is this $\phi_1$ explicitly in this case?&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2018-02-07T22:39:05.507" Title="Role of boundary conditions (e.g. periodic) in Poisson equation" Tags="|pde|boundary-conditions|elliptic-pde|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6969" PostTypeId="1" AcceptedAnswerId="6977" CreationDate="2013-04-25T03:24:56.807" Score="1" ViewCount="2912" Body="&lt;p&gt;I have a few set of data of three dimensional matrices, n x 3 with x, y and z coordinates.&#xA;I viewed the matrices in the figure window.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% read function;&#xA;OBJ1=read_obj('subject1.obj');&#xA;FV.vertices=OBJ1.vertices;&#xA;FV.faces=OBJ1.objects(2).data.vertices;&#xA;figure, patch(FV,'facecolor',[0 0 1]); camlight&#xA;grid on&#xA;axis equal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to read other sets of data by using the same function, but sets the &lt;strong&gt;center of each figure&lt;/strong&gt; (not the center of origin) at [x y z]=[0 0 0].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that when I use the function above, the center of outputs is not always at [0 0 0].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it can be done by using translation function?&lt;/p&gt;&#xA;" OwnerDisplayName="user3991" LastActivityDate="2017-02-06T13:21:15.347" Title="matlab set center of output figure to [0 0 0]" Tags="|matlab|visualization|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6970" PostTypeId="1" CreationDate="2013-04-25T05:30:26.023" Score="2" ViewCount="266" Body="&lt;p&gt;I am currently tasked with calculating the total flux of photons or irradiance from a flat emitter ('pixel'). Previously we measured the Luminance head-on (90 degree from the emitting surface) and calculated the illuminance assuming a lambertian emission. As it turns out the calculation fails for several of our devices because the radiation pattern is very much non-lamberian ($I(\alpha)=I_0*cos(\alpha)$ is not true anymore).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am now trying to overcome this by measuring the Luminance (lm/sr/m^2) for several angles and integrate it over the solid angle to calculate the full illuminance. However I fail when it comes to the proper discretization of the solid angle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attached the equation as I can not write latex here (or can I?) and I am very thankful for any help I can get on this!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish you a nice day!&#xA;Chris&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://imageshack.us/photo/my-images/40/solidanglediscr.png/&quot; rel=&quot;nofollow&quot;&gt;http://imageshack.us/photo/my-images/40/solidanglediscr.png/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(sorry I can not add images due to low reputation!)&lt;/p&gt;&#xA;" OwnerUserId="4160" LastEditorUserId="1552" LastEditDate="2013-04-25T15:55:51.310" LastActivityDate="2013-04-26T02:14:29.070" Title="Flux calculation - discretization of solid angle" Tags="|discretization|integral-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6971" PostTypeId="1" AcceptedAnswerId="7348" CreationDate="2013-04-25T07:19:59.467" Score="5" ViewCount="8937" Body="&lt;p&gt;I have coded a routine for interpolation with B-splines, only to discover later that this functionality is already included in Python's SciPy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I do not understand one parameter in the SciPy module (UnivariateSpline)[&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.UnivariateSpline.html#scipy.interpolate.UnivariateSpline]&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.UnivariateSpline.html#scipy.interpolate.UnivariateSpline]&lt;/a&gt;. The parameter I do not understand is &lt;strong&gt;parameter s&lt;/strong&gt;, which is a &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Positive smoothing factor used to choose the number of knots. If None (default), s=len(w) (w is the array of weights). If s=0, spline will interpolate through all data points.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Below I have included two plots , that interpolate the potential energy curve of the $H^+_2$ molecule, top plot for &lt;strong&gt;s=None&lt;/strong&gt;, bottom plot for &lt;strong&gt;s=0&lt;/strong&gt;. I would have expected it the other way around. This is probably due to my understanding of &lt;strong&gt;knots&lt;/strong&gt; in B-splines. Can anybody explain to me, how the knots are implemented in SciPy's Univariate routine (in a nutshell, I can go through the source myself, but this is not the point).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/H3ZpK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z4iaI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code, you need to download this &lt;a href=&quot;http://www.atom.physto.se/~lindroth/comp08/BO_pot_H2+.dat&quot; rel=&quot;nofollow noreferrer&quot;&gt;data file&lt;/a&gt; and name it *BO_energy.txt&quot; in order to run it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;from scipy.interpolate import UnivariateSpline&#xA;&#xA;krange=[1,5]&#xA;data = np.genfromtxt(&quot;BO_energy.txt&quot;)&#xA;x = data[:,0]&#xA;V = data[:,1]&#xA;mk = [ &quot;-.ks&quot;, &quot;k-.+&quot;, &quot;b--.&quot;, &quot;k-.,&quot;, &quot;b--*&quot;, &quot;k--&quot;, &quot;ro&quot;, &quot;ro&quot;]&#xA;&#xA;#plot for s = None&#xA;plt.figure(1)&#xA;plt.plot(x, V, 'r--o', label=&quot;BO&quot;)&#xA;for i in range(krange[0],krange[1]):&#xA;    spline = UnivariateSpline(x,V,k=i)&#xA;    plt.plot(x,spline(x),mk[i], label=&quot;k=%d&quot;%i)&#xA;plt.legend()&#xA;plt.title(&quot;SciPy: Univariate Splines for various k and s=None&quot;)&#xA;plt.xlabel(&quot;x / $a_0$&quot;)&#xA;plt.ylabel(&quot;V(x) / Hartree&quot;)&#xA;plt.savefig(&quot;../01.png&quot;)&#xA;&#xA;#plot for s = 0&#xA;plt.figure(2)&#xA;plt.plot(x, V, 'r--o', label=&quot;BO&quot;)&#xA;for i in range(krange[0],krange[1]):&#xA;    spline = UnivariateSpline(x,V,k=i,s=0)&#xA;    plt.plot(x,spline(x),mk[i], label=&quot;k=%d&quot;%i)&#xA;plt.legend()&#xA;plt.title(&quot;SciPy: Univariate Splines for various k and s=0&quot;)&#xA;plt.xlabel(&quot;x / $a_0$&quot;)&#xA;plt.ylabel(&quot;V(x) / Hartree&quot;)&#xA;plt.savefig(&quot;../02.png&quot;)&#xA;&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2013-05-24T13:55:12.140" Title="SciPy interpolation with Univariate Splines" Tags="|python|interpolation|scipy|b-spline|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6972" PostTypeId="1" CreationDate="2013-04-25T07:57:06.547" Score="2" ViewCount="248" Body="&lt;p&gt;I am trying to do the speed up analysis of the &lt;a href=&quot;http://www.evl.uic.edu/cavern/rg/20040525_renambot/Viz/Other/filtering_segmentation_cg_viz03.pdf&quot; rel=&quot;nofollow&quot;&gt;rotating mask filter&lt;/a&gt; (section 4.2.3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $N^2$ be the pixels in the image and let $m^2$ be the neighborhood of a given pixel, what I have for my sequential code is basically the following steps&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for every pixel in the image&#xA;   for every rotating mask&#xA;      compute dispersion&#xA;   end&#xA;   compute average brightness with the mask of lowest dispersion&#xA;   update pixel value&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is one formula to compute the idea speedup of an algorithm $\displaystyle S_p = \frac{t_s}{t_p}$ where $S_p$ is the speed up given $t_s$ time to run the sequential algorithm and $t_p$ time to run the parallel algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing I don't understand is that if $t_s$ and $t_p$ are real measured time values or if we can take the complexity based on the input size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the latter is possible, the complexity of this algorithm would be $O(NM)$ where $N$ is the total number of pixels and $M$ is the total number of pixels in the neighborhood. Having this $t_s = NM$ and $t_p = \frac{NM}{p}$ where $p$ is the number of processors, I get the ideal speedup $S_p$ to be linear, is this true?. I feel like I'm missing something, could you enlighten me to understand how to compute the speedup?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2013-04-25T15:52:30.683" LastActivityDate="2013-04-25T15:52:30.683" Title="Speedup prediction of rotating mask filter" Tags="|parallel-computing|performance|gpu|image-processing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6973" PostTypeId="2" ParentId="6972" CreationDate="2013-04-25T09:42:40.463" Score="2" Body="&lt;p&gt;I don't know much about the rotating mask filter, so I'm answering in general.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For the speed up, &lt;strong&gt;you can do both&lt;/strong&gt;. If $t_s$ and $t_p$ are real time measures, you will obtain the actual, real speed up. If $t_s$ and $t_p$ represent theoretical time measures, then you will obtain the theoretical speed up. Of course, you would like that both are the same but this depends on what you take into account for the theoretical time measures.&lt;/li&gt;&#xA;&lt;li&gt;Yes, &lt;strong&gt;it's normal that you obtain a linear speed up&lt;/strong&gt;. Indeed, $t_p = \frac{NM}{p}$ says that you can perfectly split your works among your working units, without any extra cost. In this case, the speed up is &quot;optimal&quot;, i.e. linear.&lt;/li&gt;&#xA;&lt;li&gt;So, in theory, you should obtain a linear speed up. If you measure it but &lt;strong&gt;experimentally it is not linear&lt;/strong&gt;, this means that $t_p \neq \frac{NM}{p}$ and that you are missing something in your theoretical model. This can be a small amount of redundancy in the computation (for some reason, some computations are done twice), a suboptimal load balancing (not every working unit receives the same amount of work), communications (some data must be communicated between working units which takes time), synchronizations (at some point, some working units must wait for other units before going on),...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let's make a small and very simple example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have a list of $N$ numbers $x_i$ and you want to compute for each $i$ the numbers $y_i = x_{i-1}^2 + x_i^2 + x_{i+1}^2$. You could say, I have $N$ numbers $y_i$ to compute, each of the $p$ working units computes $N/p$ numbers, so the speed up is $p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be not that easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(I suppose that you do not share the memory between the working units and for the extremities, we just restart on the other side of the list.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;: You have $20$ numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With one working unit, you can square all the $x_i$ and then all the additions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;=&gt; $20$ square operations and $40$ addition operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First attempt&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;With two working units&lt;/em&gt;, you split the work from $1$ to $10$ and $11$ to $20$. Here is the problem. To compute $y_{10}$ on the first working unit, you need $x_{11}$. The same for the second working unit, which requires $x_{10}$. The same for $y_1$ which requires $x_{20}$ and $y_20$ which requires $x_1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;=&gt; Each working unit makes $12$ square operations and $20$ addition operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that now there are redundant computations (we have $24$ square operations instead of $20$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you generalize this to $N$ numbers and $p$ processes, you obtain that each process needs $N/p+2$ square operations and $2N/p$ additions. So, $t_p$ will be something like $3N/p + 2$ and so the speed up will not be linear any more (even if it will looks like linear if $N&amp;gt;&amp;gt;p$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second attempt&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid the redundant computation, you first compute the squares of all the numbers in parallel and then compute the sums in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;With two working units&lt;/em&gt;, the first working unit computes the squares from $x_1$ to $x_{10}$, with the second the squares from $x_{11}$ to $x_{20}$. Then, you compute the $y_1$ to $y_{10}$ with the first working unit and $y_{11}$ to $y_{20}$ with the second working unit. Now, you have exactly the same number of operations as in serial! Miracle? Nop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will have the square of $x_{11}$ compute on the second unit, but it's required in the first working unit for the computation of $y_{10}$. So you have to communicate the results between the working units, which costs time...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, the speed up is not optimal (looks like the one of the first attempt).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there could be many other things that could go wrong:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do you get the list of numbers? If read from a file with one single working unit, this is a big bottleneck.&lt;/li&gt;&#xA;&lt;li&gt;What if instead of the square operation, we have a more complicated function that is sometimes easy to compute, sometimes hard? It will be very difficult to balance the work between the working units.&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="3684" LastEditDate="2013-04-25T13:33:31.810" LastActivityDate="2013-04-25T13:33:31.810" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6974" PostTypeId="2" ParentId="3055" CreationDate="2013-04-25T09:52:09.810" Score="1" Body="&lt;p&gt;Other options:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use &lt;code&gt;save2pdf.m&lt;/code&gt;  &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/16179-save2pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/16179-save2pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if a vector image is not needed, try &lt;code&gt;myaa.m&lt;/code&gt; &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/20979-myaa-my-anti-alias-for-matlab&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/20979-myaa-my-anti-alias-for-matlab&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3565" LastActivityDate="2013-04-25T09:52:09.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6975" PostTypeId="2" ParentId="6971" CreationDate="2013-04-25T12:19:47.103" Score="4" Body="&lt;p&gt;The B-Spline routines in SciPy are wrappers around the spline package by Paul Dierckx (FORTRAN implementation &lt;a href=&quot;http://www.netlib.org/dierckx/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;), although the docs say FITPACK in the first line (which is in fact another package) but then refer to routines from Dierckx. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When given a task to find a spline fit to a set of data, you have the choice of giving the routine the knots or by asking the routine to find an 'optimal' placement of the knots. In the latter case, the smoothing parameter will help the routine to choose the placement as it allows you to indicate what you find important: interpolation (and possibly an oscillating curve) or a smooth curve (but with possibly larger residuals to your data). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good introduction on splines and approximation using splines is a book by the author of the original FORTRAN code: &quot;Curve and surface fitting with splines&quot;, P. Dierckx, 1995, Oxford University Press. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-04-25T12:19:47.103" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6976" PostTypeId="2" ParentId="6972" CreationDate="2013-04-25T13:03:23.677" Score="0" Body="&lt;p&gt;The typical problem is that there is some portion of your work that doesn't actually parallelize. This is usually captured in the formula for Amdahl's Law. That being said, there must be at least one step in the above pseudocode that actually happens serially. If this is a shared-memory program, I suspect that there is serialization in &quot;update-pixel&quot;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-04-25T13:03:23.677" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6977" PostTypeId="2" ParentId="6969" CreationDate="2013-04-25T14:43:42.073" Score="1" Body="&lt;p&gt;If you want a perfectly square set of axes with the center at $(0,0,0)$, while keeping all of the data on screen, I'd do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;axis auto;&#xA;a = axis;&#xA;amax = max(abs(a));&#xA;axis([-amax,amax,-amax,amax,-amax,amax]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you like your axes limits to be integral, change &lt;code&gt;max(abs(a))&lt;/code&gt; to &lt;code&gt;ceil(max(abs(a)))&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2013-04-25T14:43:42.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6978" PostTypeId="1" AcceptedAnswerId="6981" CreationDate="2013-04-25T15:31:43.470" Score="3" ViewCount="367" Body="&lt;p&gt;we are using a first order implicit finite volume code for simulation of incompressible flows. At its core, the code utilizes a (non-preconditioned) GMRES method for solving linear systems given in &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/m_src/mgmres/mgmres.html&quot; rel=&quot;nofollow&quot;&gt;webpage&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, it is quite difficult to find an &quot;ideal&quot; setup of the GMRES algorithm for a given problem, i.e. especially to find a suitable size of the maximum Krylov space dimension (inner iterations). I wonder if there are any estimates for determining such parameters beforehand, in order to circumvent some of the trial and error procedure (which is really cumbersome and painful for large problems). Maybe someone can point me to literature where such an issue might be discussed....?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any sparse matrix solver better or other than GMRES?&lt;/p&gt;&#xA;" OwnerUserId="4107" LastEditorUserId="4107" LastEditDate="2013-04-25T15:40:08.590" LastActivityDate="2017-04-29T19:45:06.770" Title="Parameter selection difficulty in GMRES" Tags="|linear-algebra|gmres|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6979" PostTypeId="1" AcceptedAnswerId="6986" CreationDate="2013-04-25T16:16:22.553" Score="13" ViewCount="2936" Body="&lt;p&gt;How does MATLAB, for instance, calculate the SVD of a given matrix? I assume the answer probably involves computing the eigenvectors and eigenvalues of &lt;code&gt;A*A'&lt;/code&gt;. If that is the case, I would also like to know how does it compute those?&lt;/p&gt;&#xA;" OwnerUserId="898" LastEditorUserId="127" LastEditDate="2013-05-08T14:44:54.593" LastActivityDate="2013-05-08T14:44:54.593" Title="How is the SVD of a matrix computed in practice" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="6980" PostTypeId="1" AcceptedAnswerId="6984" CreationDate="2013-04-25T17:35:41.163" Score="3" ViewCount="109" Body="&lt;p&gt;Recently when I was solving a system of ODEs using runge-Kutta method , I got much different results when I  transformed the variables from spherical coordinates ($r$ and $\theta$ ) to cylindrical coordinates($\rho$ and $z$) and solved it again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(no need to mention that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$cos(\theta)=\frac{z}{\sqrt{z^2+\rho^2}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r=\sqrt{z^2+\rho^2}$&#xA;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only the results obtained using &lt;strong&gt;second system&lt;/strong&gt; (in cylindrical_ which only contained &lt;em&gt;polynomials&lt;/em&gt; of variables and didn't have fractions of variables_) were in agreement with  published results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I wonder if there is a simple &lt;em&gt;rule&lt;/em&gt; to determine the best form of a system to solve using numerical algorithms such as runge_kutta.(for example , something like :division of variables is better to be avoided) (or maybe this disagreement was a mistake somewhere in my calculations? (i.e. such a difference in results is impossible))&lt;/p&gt;&#xA;" OwnerUserId="3725" LastActivityDate="2013-04-25T19:21:50.223" Title="Best form for a system of ODEs to solve with Runge_kutta" Tags="|ode|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6981" PostTypeId="2" ParentId="6978" CreationDate="2013-04-25T18:18:31.063" Score="6" Body="&lt;p&gt;This is all wasted effort if you are not using a preconditioner because no choice of parameters will lead to a robust or scalable algorithm, and the least-bad choice will be effected by preconditioning.  &lt;em&gt;After&lt;/em&gt; you have found a good preconditioner, you can look at the effect of restarts and decide on how to manage restarts in GMRES, as well as comparing to other methods such as BiCGStab and related.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the Embree reference cited &lt;a href=&quot;https://scicomp.stackexchange.com/a/1057/119&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-04-25T18:18:31.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6982" PostTypeId="1" AcceptedAnswerId="6985" CreationDate="2013-04-25T18:54:33.003" Score="4" ViewCount="179" Body="&lt;p&gt;I have a transient flow and solute transport simulation running using a fortran code. The final solution time is 1 day. I need the output of hydraulic head for the output time of 0.5 day. I want to know what is going to change in my output data (0.5 day) if I run the same simulation for a longer time, e.g., 1000 days?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the system will get closer to steady state, but how will it affect the data for the output time that I want (0.5 day)?&lt;/p&gt;&#xA;" OwnerUserId="1053" LastEditorUserId="651" LastEditDate="2013-04-26T16:14:39.403" LastActivityDate="2013-04-26T16:14:39.403" Title="How does longer simulation time affect simulation results?" Tags="|fortran|simulation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6983" PostTypeId="2" ParentId="6980" CreationDate="2013-04-25T19:03:25.783" Score="0" Body="&lt;p&gt;If you start with one mathematical system, and through coordinate conversion convert it to another system, then solve it, you will get the same result.  If you aren't there are three possibilities (that I can think of):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You made an error in your conversion&lt;/li&gt;&#xA;&lt;li&gt;Some singularity in the conversion causes it to be invalid in the domain you are solving&lt;/li&gt;&#xA;&lt;li&gt;There is more than one solution to the problem, and the conversion simply leads you to pick off a different one (Note if your ODE system has uniqueness, this isn't possible)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When applying a numerical ODE solver to a system you need to have both stability and accuracy.  On top of this, you would like it to run in a reasonably short amount of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both stability and accuracy are achieved by choosing a time step that is small enough for single-step methods.  Some multi-step methods can not achieve stability since their nature introduces additional solutions that may not go to zero.  If you have two formulations of the same problem, that have the same solution, any ODE solver that is both stable and sufficiently accurate will give you the same result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, some formulations may have different stability and accuracy characteristics, requiring the same solver to require different time steps for different formulations of the same problem.  To find out these characteristics, you will have to analyze your problem in both formulations for your solver to know which will theoretically perform better.  Most solvers are analyzed on standard ode's and as such, these analysis results won't giev insight to your question.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-04-25T19:03:25.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6984" PostTypeId="2" ParentId="6980" CreationDate="2013-04-25T19:21:50.223" Score="6" Body="&lt;p&gt;I would identify two main concerns when choosing a basis:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;smooth&quot; physical trajectories should be smooth in the basis (e.g., well-approximated by a polynomial)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the basis should be well-conditioned&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;When using either spherical or cylindrical coordinates, a path going over the pole will experience a jump of $\pi$ in the angle $\phi$.  If such trajectories are possible in your simulation, I would recommend using a normal cartesian basis $(x,y,z)$.  This is also nice for uniqueness.  I assume you don't naively project stages back into $\phi \in [0,2\pi)$, otherwise you would have problems for equatorial orbitals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A constant-velocity trajectory passing through the origin in the $z$ direction is linear in the cylindrical basis, but is discontinuous (jumping from $\theta=\pi$ to $\theta=0$) in the spherical basis.  Perhaps your simulation has trajectories similar to this?&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-04-25T19:21:50.223" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6985" PostTypeId="2" ParentId="6982" CreationDate="2013-04-25T19:32:33.920" Score="3" Body="&lt;p&gt;There is no way to know how far from steady state a nonlinear transient simulation is.  Indeed, many famous ODEs (e.g., stiff Van der Pol or Robertson test problems) have arbitrarily long periods of nearly-stationary behavior before changing rapidly and repeating the process.  If you add some dissipation, the system may or may not undergo the eventual large change, depending on parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an implicit solver, you can solve directly for steady state, if such exists. (See papers on &quot;pseudo-transient continuation&quot; to help globalize and to ensure that you find a physical steady state.)  If the system eventually enters a limit cycle, you can also solve for the limit cycle (though this is more challenging).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-04-25T19:32:33.920" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6986" PostTypeId="2" ParentId="6979" CreationDate="2013-04-25T19:56:49.323" Score="12" Body="&lt;p&gt;This is typically done using the Golub-Reinsch algorithm, and no, it doesn't involve computing eigenvalues and eigenvectors of $AA^{T}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;/p&gt;&#xA;&#xA;&lt;p&gt;G. H. Golub and C. Reinsch.  Singular Value Decomposition and Least Squares Solutions.  Numerische Mathematik 14:403-420, 1970.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This material is discussed in many textbooks on numerical linear algebra.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-04-25T19:56:49.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6987" PostTypeId="2" ParentId="6961" CreationDate="2013-04-25T19:57:09.850" Score="6" Body="&lt;p&gt;All you have to do is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!alloc_test.f90&#xA;subroutine f(x, z, n)&#xA;  implicit none&#xA;&#xA;! Argument Declarations !&#xA;  integer :: n&#xA;  real*8, intent(in) ::  x(n)&#xA;  real*8, intent(out) :: z(n)&#xA;&#xA;! Variable Declarations !&#xA;  real*8, allocatable :: y(:)&#xA;&#xA;! Variable Initializations !&#xA;  allocate(y(n))&#xA;&#xA;! Statements !&#xA;  y(:) = 1.0&#xA;  z = x + y&#xA;&#xA;  deallocate(y)&#xA;  return&#xA;end subroutine f&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Though the size of array x and z is now passed as an explicit argument, f2py makes the argument n optional. Following is the docstring of the function as it appears to python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Type:       fortran&#xA;String Form:&amp;lt;fortran object&amp;gt;&#xA;Docstring:&#xA;f - Function signature:&#xA;  z = f(x,[n])&#xA;Required arguments:&#xA;  x : input rank-1 array('d') with bounds (n)&#xA;Optional arguments:&#xA;  n := len(x) input int&#xA;Return objects:&#xA;  z : rank-1 array('d') with bounds (n)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Importing and calling it from python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from alloc import f&#xA;from numpy import ones&#xA;x = ones(5)&#xA;print f(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ 2.  2.  2.  2.  2.]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="899" LastActivityDate="2013-04-25T19:57:09.850" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6988" PostTypeId="2" ParentId="6982" CreationDate="2013-04-25T21:13:01.353" Score="3" Body="&lt;p&gt;Nothing will change as long as all the other parameters remain the same. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-04-25T21:13:01.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6989" PostTypeId="2" ParentId="6966" CreationDate="2013-04-25T22:24:53.880" Score="5" Body="&lt;p&gt;The book is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another way to see this, using somewhat more heavy machinery. There is a theorem that states that if a matrix $A$ is irreducible and weakly row diagonally dominant, then Jacobi's method converges. Here weakly diagonally row dominant means $|a_{ii}| \geq \sum_{j\neq i} |a_{ij}|$ for all $i$ and irreducible means that there is no permutation matrix $P$ such that&#xA;$$PAP^T = \left[\begin{array}{cc} A_{11} &amp;amp; A_{12} \\ 0 &amp;amp; A_{22}\end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly your matrix is weakly diagonally row dominant. It can be shown that a matrix $A$ is irreducible if and only if its directed graph is strongly connected. Since your matrix's sparsity pattern is identical to that of the one-dimensional Laplacian matrix, and the directed graph of the one-dimensional Laplacian matrix is strongly connected, it follows that the directed graph of your matrix is strongly connected as well and thus your matrix is irreducible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, Jacobi's method applied to your matrix converges. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://books.google.com/books?id=S5F5ccGF-EIC&amp;amp;printsec=frontcover&amp;amp;source=gbs_ge_summary_r&amp;amp;cad=0#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Applied Numerical Linear Algebra&lt;/a&gt; by James Demmel for details. In particular, see Chapter 6 and Theorem 6.2.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-04-26T00:04:08.343" LastActivityDate="2013-04-26T00:04:08.343" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6990" PostTypeId="2" ParentId="6968" CreationDate="2013-04-25T23:15:14.047" Score="10" Body="&lt;p&gt;From a numerical perspective, it's perhaps easiest to discuss the discretizations directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Poisson equation with homogeneous Dirichlet boundary conditions, there is a unique solution for any right-hand side. Once discretized, the equation can be written in the form $Ax = b$, where $A$ is the standard discretization of the 3D Laplacian operator with Dirichlet boundary and $b$ is a standard discretization of $f$. Since $A$ is positive definitive, it is invertible, and the system will have a unique solution for any $b$, and thus any $f$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, of course, the usual issues with undersampling; if two different values of $f$ give rise to the same discretization due to use of a coarse grid or due to discontinuities in $f$, there may be some ambiguity about what system is actually being solved. But provided the discretization of $f$ is well-behaved, a meaningful unique solution will exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation is slightly more complicated in the case of periodic boundary conditions because the standard discretization of the 3D Laplacian operator with periodic boundary is positive &lt;em&gt;semidefinite&lt;/em&gt; and has a one-dimensional kernel $K$ comprising solutions of the form $x \equiv C$ with $C$ constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because $A$ is still symmetric in the periodic case, we have that $\operatorname{Range} A = K^\perp$, and so $Ax = b$ will not have a solution unless $\mathbf{1} \cdot b = 0$, where $\mathbf{1}$ is the vector consisting of all 1s. This provides the consistency condition for the right-hand side in discretized form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, analytically, there is a somewhat simpler way to look at it. Recall that, for $\phi \in C^2(\Omega)$, where $\Omega$ is our domain, we have&#xA;$$\int_\Omega \Delta \phi \, dx = \int_{\partial \Omega} \frac{\partial \phi}{\partial \mathbf{n}}\, dS.$$&#xA;If we stipulate a periodic boundary condition on $\phi$, then the boundary term in the right-hand side disappears, and we're left with&#xA;$$\int_\Omega \Delta\phi\, dx = 0.$$&#xA;So if $\phi$ satisfies $\Delta \phi = f$, it immediately follows that we must have&#xA;$$\int_\Omega f\, dx = 0.$$&#xA;This is the analytical analog to $\mathbf{1} \cdot b = 0$, as both are expressing the fact that the average value of $f$, and thus $b$, over the domain must be zero.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-04-25T23:22:47.653" LastActivityDate="2013-04-25T23:22:47.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6991" PostTypeId="1" AcceptedAnswerId="7308" CreationDate="2013-04-25T23:50:39.993" Score="5" ViewCount="909" Body="&lt;p&gt;I'm looking for the best package for doing the following: building a 3D model (or loading externally) of a non-standard windmill shape, simulating an airflow over the model, and determining the generated torque. Ideally, it would be either open-source, or have a free demo/low-cost version. Nice-to-have, would be one reasonably simple for a beginner to use. (And Python-based scripting, as long as I'm making a wish...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, consider this &lt;a href=&quot;http://www.windturbinesnow.com/cylindrical-conical-wind-turbines.htm&quot; rel=&quot;nofollow&quot;&gt;conical wind turbine&lt;/a&gt;. How exactly could one determine its power output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I've found various formulas for computing the efficiency of wind turbines, they only apply to the standard three-bladed horizontal-axis turbine. And I don't have the mathematical skills to derive a formula for any non-standard shape. So I'm wondering if a) it could be determined by a simulation, and b) if there's a package that I could manage to do it in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Packages I've looked at, include numpy/scipy, Blender 3D, Unity 3D (game engine), and FreeCAD. It's possible one of those would manage it, but my cursory evaluation didn't determine that.&lt;/p&gt;&#xA;" OwnerUserId="4130" LastEditorUserId="1567" LastEditDate="2013-05-21T17:21:30.707" LastActivityDate="2013-05-24T10:39:16.673" Title="Package for determining power output of windmill shapes?" Tags="|python|fluid-dynamics|software|simulation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="6992" PostTypeId="2" ParentId="6970" CreationDate="2013-04-26T02:14:29.070" Score="1" Body="&lt;p&gt;Google knows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://theory.rutgers.edu/~giese/notes/Quadrature.pdf&quot; rel=&quot;nofollow&quot;&gt;http://theory.rutgers.edu/~giese/notes/Quadrature.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/27097-getlebedevsphere&quot; rel=&quot;nofollow&quot;&gt;https://www.mathworks.com/matlabcentral/fileexchange/27097-getlebedevsphere&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's called Lebedev quadrature.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-26T02:14:29.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6994" PostTypeId="2" ParentId="6965" CreationDate="2013-04-26T02:18:40.023" Score="2" Body="&lt;p&gt;Typical (time-)discretizations of parabolic PDEs with a symmetric (self-adjoint) spatial operator also lead to SPD matrices. An example would be the heat equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As do (time-)discretizations of hyperbolic equations with self-adjoint spatial operators such as the wave equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For full matrices, look at integral equations (or the boundary integral re-formulation of partial differential equations) which typically also lead to symmetric and often positive definite matrices. They are a special case of Fredholm integral equations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-04-26T04:18:36.157" LastActivityDate="2013-04-26T04:18:36.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6995" PostTypeId="2" ParentId="6965" CreationDate="2013-04-26T02:36:42.893" Score="2" Body="&lt;p&gt;Statistics. Correlation matrices are positive semidefinite. Specifically, let $x$ be a random variable with instances in $\mathbb{R}^n$. Then $E(xx^T)=\Sigma$, where $\Sigma$ is positive semidefinite. Note that it does not matter what distribution the random variable is drawn from; if the covariance matrix exists, it is positive semidefinite. Of course, there are statistical applications in a wide variety of fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Control theory. The Lyapunov stability theory says that a sufficient condition for the stability of a system described by a vector ODE $\dot{x}=\phi(x)$ is stable if there exists a positive definite matrix $P$ such that $\tfrac{d}{dt} (x^T P x) &amp;lt; 0$ for all $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Semidefinite programming. There are a variety of useful mathematical models that involve finding a positive semidefinite matrix that satisfies certain linear relationships on its elements, and possibly optimizing a linear function of said matrix. These models are called \emph{semidefinite programs}. See &lt;a href=&quot;http://www.stanford.edu/~boyd/papers/sdp.html&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; for a discussion of the mathematics involved, and &lt;a href=&quot;http://www.stanford.edu/~boyd/papers/sdp-apps.html&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; for some applications, including circuit design, structural optimizaton, and others. Search for the word &quot;semidefinite&quot; on &lt;a href=&quot;http://www.stanford.edu/~boyd/papers.html&quot; rel=&quot;nofollow&quot;&gt;this page&lt;/a&gt; for even applications. (Disclosure: this was my research group in college.)&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-04-26T02:42:01.103" LastActivityDate="2013-04-26T02:42:01.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="6996" PostTypeId="2" ParentId="5345" CreationDate="2013-04-26T03:49:06.193" Score="5" Body="&lt;p&gt;Two months after posting my question I realized that I still want to try working with deal.II, and implemented the tet-&gt;hex converter to work with complex geometry. So if someone is interested, welcome to &lt;a href=&quot;http://code.google.com/p/tethex/wiki/Tethex&quot; rel=&quot;noreferrer&quot;&gt;tethex wiki page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2013-04-26T03:49:06.193" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="6998" PostTypeId="1" AcceptedAnswerId="7017" CreationDate="2013-04-26T05:27:23.430" Score="2" ViewCount="401" Body="&lt;p&gt;I am working on implicit schemes for Euler equations.  Implicit methods allow one to use large CFL values, but is there some way to evolve CFL number from a much smaller value than desired value to avoid errors in initial iterations.  There are some strategies available in literature, such as exponential increment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CFL evolution strategy directly affects time for computation, and I want to minimize the time consumption. Is there any standard way to evolve CFL number?  Or is it done by trial and error?&lt;/p&gt;&#xA;" OwnerUserId="3549" LastEditorUserId="3549" LastEditDate="2013-04-26T15:21:11.170" LastActivityDate="2013-04-29T06:22:15.540" Title="CFL evolution techniques for Implicit methods" Tags="|fluid-dynamics|implicit-methods|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="6999" PostTypeId="1" AcceptedAnswerId="7000" CreationDate="2013-04-26T15:28:06.913" Score="8" ViewCount="279" Body="&lt;p&gt;According to Wolfram Alpha and the &lt;a href=&quot;http://www.sagemath.org/&quot;&gt;Sage computer algebra system&lt;/a&gt;, the following identity holds:&#xA;$$&#xA;\cos\left(\arctan\left(\frac{l_1-l_2}{d}\right)\right) = \frac{1}{\sqrt{1 + \frac{(l_1-l_2)^2}{d^2}}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I tried to verify it with an arbitrary example in NumPy, I noticed a rather large difference in the actual values computed by both sides of the idenitity. I have used the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    l1 = 10; l2 = 8; d = 17&#xA;    from numpy import arctan2, cos, sin, sqrt&#xA;&#xA;    alpha = arctan2((l1-l2),d)&#xA;    left = cos(alpha)&#xA;&#xA;    right = sqrt(1 + ((l1-l2)**2)/(d**2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Evaluating the results for &lt;code&gt;left&lt;/code&gt; and &lt;code&gt;right&lt;/code&gt; yielded the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    left = 0.99315060432287616&#xA;    right = 1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is tempting to write this off as simply being a numerical error, but since I have very little experience in how large numerical errors can get, I am not so sure. Is this possible or am I missing something (obvious)?&lt;/p&gt;&#xA;" OwnerUserId="1238" LastActivityDate="2013-04-27T19:32:52.513" Title="Astoundingly large difference when evaulating trigonometric identity with NumPy" Tags="|python|numerics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7000" PostTypeId="2" ParentId="6999" CreationDate="2013-04-26T15:37:26.497" Score="10" Body="&lt;p&gt;I have a suspicion that your Python expression for the right-hand side is doing integer division, not floating-point division. As a result, &lt;code&gt;((l1-l2)**2)/(d**2)&lt;/code&gt; is being evaluated as zero, and the term inside the square root is one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, you forgot the reciprocal in your right-hand expression as well, but that's not the first problem...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; l1 = 10; l2 = 8; d = 17;&#xA;&amp;gt;&amp;gt; alpha = atan2((l1-l2),d)&#xA;alpha =&#xA;    0.1171&#xA;&amp;gt;&amp;gt; left = cos(alpha)&#xA;left =&#xA;    0.9932&#xA;&amp;gt;&amp;gt; right = 1/sqrt(1 + ((l1-l2)^2)/(d^2))&#xA;right =&#xA;    0.9932&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2013-04-26T15:37:26.497" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7001" PostTypeId="1" CreationDate="2013-04-27T02:24:19.760" Score="18" ViewCount="5952" Body="&lt;p&gt;I am trying to solve some unconstrained nonlinear optimization problems on GPU (CUDA).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function is a smooth nonlinear function, and its gradient is relatively cheap to compute analytically, so I don't need to bother with numerical approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve this problem with mostly fp32 maths ops (for various reasons), so which nonlinear optimization method is more robust against round-up errors whilst has good performance? (e.g. conjugate gradient/quasi newton/trust region), have anyone tried BFGS on GPU with good results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, the Hessian, if needed, is relatively small in my case (&amp;lt;64x64 typically), but I need to solve thousands of these small scale optimization problems concurrently.&lt;/p&gt;&#xA;" OwnerUserId="4170" LastEditorUserId="22284" LastEditDate="2017-11-26T21:32:49.700" LastActivityDate="2017-11-26T21:32:49.700" Title="Solving unconstrained nonlinear optimization problems on GPU" Tags="|optimization|cuda|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7002" PostTypeId="1" AcceptedAnswerId="7010" CreationDate="2013-04-27T04:39:30.827" Score="9" ViewCount="5595" Body="&lt;p&gt;I intend to solve Ax = b where A is complex, sparse, unsymmetric and highly ill-conditioned (condition number ~ 1E+20) square or rectangular matrix. I have been able to solve the system with ZGELSS in LAPACK accurately. But as the degrees of freedom in my system grow, it takes a long time to solve the system on a PC with ZGELSS as the sparsity is not exploited. Recently I tried SuperLU (using Harwell-Boeing storage) for the same system but the results were inaccurate for condition number &gt; 1E+12 (I am not sure if this is a numerical issue with the pivoting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am more inclined towards using already developed solvers. Is there a robust solver which can solve the system I mentioned quickly (i.e. exploiting the sparsity) and reliably (in view of the condition numbers)?&lt;/p&gt;&#xA;" OwnerUserId="4171" LastEditorUserId="1748" LastEditDate="2013-04-27T05:11:10.413" LastActivityDate="2013-09-10T12:23:51.253" Title="Solving a sparse and highly ill-conditioned system" Tags="|linear-solver|sparse-matrix|lapack|condition-number|" AnswerCount="3" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="7003" PostTypeId="1" CreationDate="2013-04-27T13:22:58.680" Score="3" ViewCount="314" Body="&lt;p&gt;This question is extended from &lt;a href=&quot;https://math.stackexchange.com/q/374279/21209&quot;&gt;this&lt;/a&gt; question&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original problem is to maximize the Frobenius norm of a matrix product&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$max_{X\in \mathbb{D}}||B^TXA  ||_F$, where $\mathbb{D}$ is the doubly stochastic matrix set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One challenge is that the size of matrix $X$ is big and I would like to reformulate it into a low memory requirement formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to consider the $l_1$ version of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But yet the performance is not too good. (Mentioned in &lt;a href=&quot;https://math.stackexchange.com/q/374279/21209&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any good suggestion?&lt;/p&gt;&#xA;" OwnerUserId="4172" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-04-27T17:33:27.517" Title="Reformulation of a Frobenius norm maximization problem" Tags="|optimization|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7005" PostTypeId="1" AcceptedAnswerId="7006" CreationDate="2013-04-27T15:43:22.473" Score="7" ViewCount="243" Body="&lt;p&gt;I've been having problems designing an effective caching mechanism.&#xA;The mechanism is to be used as part of a project developing scientific software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to save computation time by loading results of &lt;em&gt;identical&lt;/em&gt; computations done in the past.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I've been encountering is how to know if a computation is identical to a previous one. Checking the input parameters can be sufficient in most cases, but sometimes the code changes, and then even though the input parameters can be identical, the results may differ.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that modifying the code now needs to be coupled with clearing the cache, in case that code creates cached results. This is a manual step which is error-prone. Failure can cause the results to stay the same, even after code changes have been made, leading to bugs or erroneous conclusions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is any mechanism for caching which can solve, or somehow help avoid this pitfall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&#xA;Ofer&lt;/p&gt;&#xA;" OwnerUserId="4173" LastActivityDate="2013-04-29T17:21:33.250" Title="Caching mechanism" Tags="|matlab|software|programming-paradigms|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7006" PostTypeId="2" ParentId="7005" CreationDate="2013-04-27T16:01:39.733" Score="5" Body="&lt;p&gt;The answer depends, in part, both on how robust of a solution you're looking for and on what kind of development workflow you use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One particularly simple option is to tag each cache entry with the version number of the software. As long as every change to the code is accompanied by a change in version number, this will give you an easy way to check whether a cache file is still valid. However, version numbering is probably only appropriate if changes to the project are committed all at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A less robust option that is nonetheless more suited for continuous development would be to simply compare the last-modified dates for your [oldest] cache files and your source files. If the source files were last modified before all of the cache files, the latter should still be usable.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;@DeerHunter pointed out a technique that is more robust than either of the above options and should work regardless of your development methodology. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When creating a cache file, include a hash of the current source code. At the start of program execution, hash the current source code and compare the result to the hash contained in each cache file. If they don't match, the file was produced by a different version of the software and should be removed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar procedure can expedite comparisons between input parameters. Store a hash of the input parameters in each cache file, and then compare that hash to a hash of the current input parameters to check for a match. The only downside here is that you can no longer look for cache files with input parameters that are close to those of the current simulation.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-04-27T18:37:49.020" LastActivityDate="2013-04-27T18:37:49.020" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7007" PostTypeId="2" ParentId="1890" CreationDate="2013-04-27T17:26:58.113" Score="5" Body="&lt;p&gt;First of all, I'm going to assume that by $\nabla(\rho_g \bar u T_g)$ you actually mean $\nabla \cdot (\rho_g \bar u T_g)$, the divergence of $\rho_g \bar u T_g$. If this were not the case, you wouldn't be working with a conservation law and your equation wouldn't make much sense on account of adding scalar quantities to vector quantities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, I think there may some confusion about finite volumes and finite differences. The term finite difference is a bit confusing, since it refers to both a type of stencil and a type of method. A finite difference &lt;em&gt;stencil&lt;/em&gt; is an expression, defined on a grid, that approximates a term, or a sum of terms, in a differential equation by using adjacent grid points. In a one-dimensional, unstructured grid setting, a finite difference stencil can be described as an expression of the form&#xA;$$F(u_{i-j},\ldots,u_{i+k})$$&#xA;that approximates a function of $u$ and/or its derivatives at the grid node $u_i$. (Here $j, k \geq 0$ are integers, of course.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A finite difference &lt;em&gt;method&lt;/em&gt;, in contrast, is an approach that attempt to directly discretize function derivative terms onto a mesh using finite difference stencils derived from Taylor series arguments. Such a method does not, in general, preserve aspects of differential equation structure; finite difference methods are not in general conservative. Note that both centered difference methods and basic upwind methods would typically be grouped in with finite difference methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite volume methods, by contrast, are designed with the structure of conservation laws in mind. Recall that general conservation laws have the form&#xA;$$u_t + \nabla \cdot F(u) = 0,$$&#xA;where of course $F$ may depend on other quantities as well. The key idea is that, if this PDE is integrated over a small volume region -- a finite volume -- then we can appeal to the divergence theorem and recast the divergence term as a surface flux. Since surface fluxes between adjacent regions will be equal in magnitude and opposite in direction, the method is conservative. And if we treat the grid values of our functions -- in this case $u$ -- as being cell averages, then we can actually recast a finite volume method into a straightforward finite difference stencil, albeit possibly with the added complexity of flux limiters and upwinding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get back to the main thrust of your question, both centered differences and basic upwind techniques are generally ill-advised if an accurate method is desired that is capable of resolving shocks. Centered differences are particularly problematic, as they do not respect the characteristic direction of information propagation inherent to conservation laws. Instead, there are a number of more modern approaches that do a much better job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the most accurate methods available is &lt;a href=&quot;http://en.wikipedia.org/wiki/Godunov_method&quot;&gt;Godunov's method&lt;/a&gt;. Godunov is a finite volume approach to conservation laws and is able to resolve shocks by exactly solving local Riemann problems on the surfaces between adjacent finite volume cells. Unlike other methods that use approximate Riemann solvers or attempt to avoiding differencing across shocks, Godunov's method introduces very little artificial viscosity, and is thus particularly valuable for resolving convection in inviscid or near-inviscid problems. Of course, there is a price to pay for such high accuracy, and Godunov's method is one of the more difficult methods to program -- particularly for unstructured domains -- and is also quite computationally expensive. For more information on Godunov's method and other exact and approximate Riemann-solving finite volume methods, I highly recommend [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other end of the spectrum are methods that do not resolve shocks exactly, but attempt to approximately &quot;capture&quot; them by appropriate choice of finite difference stencil. These are broadly called &lt;a href=&quot;http://en.wikipedia.org/wiki/Shock_capturing_method&quot;&gt;shock-capturing methods&lt;/a&gt;, and one of the classic examples is the class of essentially non-oscillatory (ENO) methods. ENO methods use standard finite difference stencils derived from Taylor series, but make use of local information, in the form of Newton divided differences, to choose stencils that avoid differencing across shocks. This is advantageous because finite difference stencils are typically only accurate if the underlying function is locally smooth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest [2] for a remarkably clear introduction to the basics of ENO methods, but a Google search will turn up dozens more references. ENO schemes do not resolve shocks nearly as well as Godunov's method, and are not appropriate for inviscid or near-inviscid problems, but they are remarkably easy to program and are quite computationally efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://books.google.com/books?id=3WhqLPcMdPsC&amp;amp;dq=numerical+methods+for+conservation+laws&amp;amp;source=gbs_navlinks_s&quot;&gt;R. LeVeque. Numerical Methods for Conservation Laws. Birkhauser, Zurich. 2005.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://books.google.com/books/about/Level_Set_Methods_and_Dynamic_Implicit_S.html?id=SQQI2vqWR7gC&quot;&gt;S. Osher and R. Fedkiw. Level Set Methods and Dynamic Implicit Surfaces. Springer-Verlag, New York. 2002.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-04-27T17:34:19.503" LastActivityDate="2013-04-27T17:34:19.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7008" PostTypeId="1" CreationDate="2013-04-27T18:59:39.067" Score="10" ViewCount="3160" Body="&lt;p&gt;Is there any preprocessor directives that could be used to use the polylog function? Or is it included in cmath? If so, do you call it by Li or by polylog?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;What I really am trying to do is give an analytical value for the indefinite integral of the function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  \frac{x^3}{e^{x} -1} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which involves polylogarithm functions. But if anyone has a suggestion for another way to integrate this function analytically I'd be welcome to any ideas.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;" OwnerUserId="3954" LastEditorUserId="127" LastEditDate="2013-05-02T05:17:06.313" LastActivityDate="2018-09-27T04:29:57.403" Title="How to use polylogarithm function in c++?" Tags="|c++|special-functions|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7009" PostTypeId="2" ParentId="6999" CreationDate="2013-04-27T19:32:52.513" Score="7" Body="&lt;p&gt;As Michael C. Grant pointed out, the problem is that the division operation is integer division, not floating point division.  In versions of Python before 3, dividing two integers using &lt;code&gt;/&lt;/code&gt; rounds down to an integer.  This behavior can be changed by adding&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;at the top of your script.  Note that this declaration changes the behavior of &lt;code&gt;/&lt;/code&gt; only for the current file.  If you want the original behavior occasionally, use &lt;code&gt;//&lt;/code&gt; (double slash).  This is discussed in more detail here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.python.org/dev/peps/pep-0238&quot;&gt; PEP 238 -- Changing the Division Operator &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that while most languages including C have similar semantics for division, the semantics are more confusing in Python because integers are not usually promoted to floats when passed to functions.  Thus, a function written as if it was operating on floats may in fact be operating on integers.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-04-27T19:32:52.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7010" PostTypeId="2" ParentId="7002" CreationDate="2013-04-27T20:43:41.607" Score="13" Body="&lt;p&gt;When you use ZGELSS to sovle this problem, you're using the truncated singular value decomposition to regularize this extremely ill-conditioned problem.  it's important to understand that this library routine is not attempting to find a least squares solution to $Ax=b$, but rather it is attempting to balance finding a solution that minimizes $\| x \|$ against minimizing $\| Ax-b \|$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the parameter RCOND passed to ZGELSS can be used to specify which singular values should be included and excluded from the computation of the solution.  Any singular value less than RCOND*S(1) (S(1) is the largest singular value) will be ignored.   You haven't told us how you've set the RCOND parameter in ZGELSS, and we no nothing about the noise level of the coefficients in your $A$ matrix or in the right hand side $b$, so it's hard to say whether you've used an appropriate amount of regularization.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You do seem to be happy with the regularized solutions that you're getting with ZGELSS, so it appears that the regularization effected by the truncated SVD method (which finds a minimum $\| x \|$ solution among the least squares solutions that &#xA;minimize $\| Ax-b \|$ over the space of solutions spanned by the singular vectors associated with the singular values larger than RCOND*S(1)) is satisfactory to you.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question could be reformulated as &quot;How can I efficiently obtained regularized least squares solutions to this large, sparse, and very ill-conditioned linear least squares problem?&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation would be to use an iterative method (such as CGLS or LSQR) to minimize the explicitly regularized least squares problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| Ax-b \|^{2} + \alpha^{2} \| x \|^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the regularization parameter $\alpha$ is adjusted so that the damped least squares problem is well conditioned and so that you're happy with the resulting regularized solutions.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-04-27T20:43:41.607" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7011" PostTypeId="1" CreationDate="2013-04-28T02:41:15.473" Score="2" ViewCount="999" Body="&lt;p&gt;I wrote a Python code which solves a second degree nonlinear differential equation using the Newton iteration method. The code converges to a 2-cycle within 50 or so iterations.  The cycle only appears in noticable magnitude in the last half dozen grid points (of ~120).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use Central Differencing for solving a 2 boundary conditions second degree nonlinear equation. The number of gridpoints is 100 and there 3 parameters, $\alpha_i$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u'' + \alpha_1 \cdot u' + {\alpha_2 \cdot u \cdot (u')^{2/3}} + {\alpha_3 \cdot (u')^{2/3}} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$u(x=0) = 0$&lt;/li&gt;&#xA;&lt;li&gt;$du/dx(x=N) = m$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The $\alpha$ parameters and m are all calculated from experimental data, and the $\alpha$'s are functions of x.&lt;br&gt;&#xA;I use the boundary conditions on the Jacobian matrix at the first and the last line.&#xA;I use also these conditions on the calculation of F vector&#xA;if the iteration is:&#xA;$X = X - J^{-1} \cdot F$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that the last value of F array change between + and - constant values.  I have already tuned the initial iterate to be near the expected final solution (from experimental data as well).  What options do I have to avoid a 2-cycle in the iterates of newton's method when it appears near to the solution I wish to find?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, If the F last value (BC2) or one before the end is 0, then I have a convergence, with the expected u values at the last point or last two gridpoints respectively, but still there is a decrement after the #106 point going near to zero.&lt;/p&gt;&#xA;" OwnerUserId="3917" LastEditorUserId="3917" LastEditDate="2013-05-04T23:00:12.237" LastActivityDate="2013-05-04T23:00:12.237" Title="Newton Iteration method convergence" Tags="|python|convergence|boundary-conditions|newton-method|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="7012" PostTypeId="1" CreationDate="2013-04-28T12:24:28.780" Score="0" ViewCount="303" Body="&lt;p&gt;I have some issues solving sparse linear equations Ax = b&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My matrix A is sparse with dimension of 5 million by 5 million. Actually, it is a combination of two matrices. One is tridiagonal and the other matrix has a bigger band. For example the $29450^{th}$  row has non zero diagonal entry. Plus non zero element at $14725^{th}$, $44175^{th}$, $29451^{th}$,  $29449^{th}$,$29605^{th}$,$29295^{th}$,column. So every row has almost 6 non zero elements plus non zero diagonal entry. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use a couple of iterative solvers in matlab, but they don't seem to give good accuracy. Also if I run the iteration for a long time, I will have round off errors and all. What can I do to get the best estimate?&lt;/p&gt;&#xA;" OwnerUserId="3871" LastActivityDate="2013-05-08T20:20:00.560" Title="Issues with solving large sparse linear equations" Tags="|linear-algebra|optimization|sparse-matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7013" PostTypeId="2" ParentId="5604" CreationDate="2013-04-28T13:00:13.213" Score="7" Body="&lt;p&gt;There are a number of &quot;corrected&quot; integration rules which invoke derivatives of the endpoints.  One simple example is the corrected trapezoidal rule.  Suppose we wish to approximate the integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_a^bf(x)\,dx.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $n$ be an integer, and $h=(b-a)/n$.  Then the trapezoidal rule&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;T = \frac{h}{2}\bigl(f(a)+2f(a+h)+2f(a+2h)+\cdots+2f(a+(n-1)h)+f(b)\bigr)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;provides a simple approximation to the integral, with error of order $h^2$.  However, the &quot;corrected&quot; trapezoidal rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;T'=T-\frac{h^2}{12}\bigl(f'(b)-f'(a)\bigr)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;vastly increases the accuracy.  For example, consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I=\int_0^1 e^{-x^2}\,dx&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and choose $n=8$.  The exact value of $I$, to 14 decimal places, is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;0.74682413281243&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the values of $T$ and $T'$ are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;0.7458656148457,\quad 0.74682363422375&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;respectively.  The errors are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;|I-T|=9.5851796673207534 \times 10^{-4}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;|I-T'|=4.9858868145236102 \times 10^{-7}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;showing a remarkable increase in accuracy.  There are further corrections involving higher derivatives, or starting from other Newton-Cotes rules or Gaussian type rules.&lt;/p&gt;&#xA;" OwnerUserId="4176" LastActivityDate="2013-04-28T13:00:13.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7014" PostTypeId="2" ParentId="7002" CreationDate="2013-04-28T15:17:36.167" Score="9" Body="&lt;p&gt;Jed Brown has already pointed this out in the comments to the question, but there is really not very much you can do in usual double precision if your condition number is large: in most cases, you will likely not get a single digit of accuracy in your solution and, worse, you can't even tell because you can't accurately evaluate the residual corresponding to your solution vector. In other words: it's not a question of which linear solver you should choose -- no linear solver can do something useful for such matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These sort of situations typically happen because you choose an unsuitable basis. For example, you get such ill-conditioned matrices if you chose the functions $1,x,x^2,x^3,...$ as the basis of a Galerkin method. (This leads to the Hilbert matrix, which is notoriously badly conditioned.) The solution in such cases is not to ask which solver can solve the linear system, but ask whether there are better bases that can be used. I would encourage you to do the same: think about reformulating your problem so that you don't end up with these kinds of matrices.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-04-28T15:17:36.167" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7015" PostTypeId="2" ParentId="5287" CreationDate="2013-04-28T22:37:32.627" Score="1" Body="&lt;p&gt;I am not able to comment on your answer. However, I ran into similar problem while simulating interconnects in VLSI circuits. That time it turned out I was applying prima incorrectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if system is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E\dot{x}=Ax+Bu$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and Q is projection space, then reduced system is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q^{T}EQ\dot{z}=Q^{T}AQz+Q^{T}Bu$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as oppose to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot{z}=Q^{T}E^{-1}AQz+Q^{T}E^{-1}Bu$&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2013-04-28T22:37:32.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7016" PostTypeId="2" ParentId="7008" CreationDate="2013-04-29T05:18:56.603" Score="8" Body="&lt;p&gt;There's a GPL'd C library, &lt;a href=&quot;https://launchpad.net/anant&quot; rel=&quot;nofollow noreferrer&quot;&gt;ANANT - Algorithms in Analytic Number Theory&lt;/a&gt; by Linas Vepstas, which includes multiprecision implementation of the polylogarithm, building on &lt;a href=&quot;http://gmplib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMP&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From its README file:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This project contains ad-hoc implementations of assorted analytic&#xA;  functions of interest in number theory, including the gamma function,&#xA;  the Riemann zeta function, the polylogarithm, and the Minkowski &#xA;  question mark function. The implementation uses the Gnu&#xA;  Multi-Precision library (GMP) to perform all low-level operations. &#xA;  The code herein is licensed under the terms of the Gnu GPLv3 license.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The GSL (GNU Scientific Library) apparently only has &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Dilogarithm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the dilogarithm function&lt;/a&gt;.  However following a hint from @J.M. one finds the &lt;a href=&quot;http://en.wikipedia.org/wiki/Debye_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Debye function&lt;/a&gt; which gives the ulterior integral (up to a scalar multiple) implemented in double precision (see &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Debye-Functions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSL 7.10 Debye Functions&lt;/a&gt; orders 1 through 6):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ D_n(x) = \frac{n}{x^n} \int_0^x \frac{t^n dt}{e^t - 1} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Symbolic integration software such as Mathematica or Maxima provides:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \int_0^x \frac{t^3 dt}{e^t-1} = 6 \operatorname{Li}_4(e^x) &#xA; - 6x \operatorname{Li}_3(e^x) + 3x^2 \operatorname{Li}_2(e^x)&#xA; + x^3 \log(1-e^x) - \frac{x^4}{4} - \frac{\pi^4}{15} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The left-hand side is obviously a purely real value if &lt;span class=&quot;math-container&quot;&gt;$x \gt 0$&lt;/span&gt;, but the polylogarithms shown will be complex-valued (because &lt;span class=&quot;math-container&quot;&gt;$e^x &amp;gt; 1$&lt;/span&gt;, and so equality hinges on total cancellation of imaginary parts).  We can avoid the need for complex arithmetic in this case by substituting the expression:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \int_0^x \frac{t^3 dt}{e^t-1} = -6 \operatorname{Li}_4(e^{-x}) &#xA;  - 6x \operatorname{Li}_3(e^{-x}) - 3x^2 \operatorname{Li}_2(e^{-x}) &#xA;  - x^3\operatorname{Li}_1(e^{-x}) + \frac{\pi^4}{15} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an improvement because with polylogarithm arguments in &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;, the results are purely real values.  Note the proper result when &lt;span class=&quot;math-container&quot;&gt;$x = 0$&lt;/span&gt; is zero, and this is achieved by cancellation between the leading term and the constant.  Thus relative error could be an issue for small positive values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that our mysterious constant &lt;span class=&quot;math-container&quot;&gt;$\pi^4/15$&lt;/span&gt; is the limiting upper bound on these (monotone increasing) integrals:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \int_0^\infty \frac{t^3 dt}{e^t - 1} = \Gamma(4) \zeta(4)&#xA;  = 6 \cdot \frac{\pi^4}{90} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can now revisit the title question, How to use polylogarithm function in c++?  The point is worth making that there is no standard implementation of polylogarithm functions for C or &lt;a href=&quot;https://en.cppreference.com/w/cpp/numeric/special_math&quot; rel=&quot;nofollow noreferrer&quot;&gt;even C++&lt;/a&gt;.  If the goal is to avoid &lt;em&gt;any additional library&lt;/em&gt; for your implementation, it pretty well sets you out to rolling your own routines, perhaps along the lines suggested by the David C. Wood paper that GertVdE’s Answer links to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides the multiprecision routines suggested in the first part of my Answer, there is a  mature (free) double precision math library in &lt;a href=&quot;http://www.netlib.org/cephes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cephes&lt;/a&gt;  by Stephen L. Moshier which implements both real (&lt;code&gt;polylog&lt;/code&gt;) and complex (&lt;code&gt;cpolylog&lt;/code&gt;) versions of the polylogarithm special functions.  Although their accuracy depends in part on the underlying standard mathematical functions of C, the Cephes &lt;a href=&quot;https://github.com/jeremybarnes/cephes/blob/master/misc/polylog.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;source documentation&lt;/a&gt; reports tests and theoretical peak errors for orders 1 through 4 at about the limits of double precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you may wish to use other software to directly check (not referencing polylogarithms) the quadrature routines you wrote for your integral.  As I sketch out in &lt;a href=&quot;https://math.stackexchange.com/q/377660/3111&quot;&gt;this Math.SE Question&lt;/a&gt;, the power series centered at the origin for the integral has limited convergence, but this can be mitigated by using a continued fraction expansion instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For immediate gratification I recommend the (free) numerical quadrature QUADPACK routines included in &lt;a href=&quot;http://maxima.sourceforge.net/docs/manual/en/maxima_19.html#SEC99&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maxima&lt;/a&gt;, specifically &lt;code&gt;quad_qag&lt;/code&gt;.  For example find the integral over [0,5] with this Maxima command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(%i1) quad_qag(x^3/(%e^x - 1), x, 0, 5, 2);&#xA;(%o1) [4.899892158330582,5.4399730923588665*10^-14,21,0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of the input arguments only the last one bears an explanation.  The fifth argument to &lt;code&gt;quad_qag&lt;/code&gt; specifies what rule to apply in adaptive quadrature.  Possible values are 1 to 6, and give increasing sophistication/accuracy.  The output line gives first the numerical quadrature, followed by an estimate of its absolute error, the number of subintervals/steps used, and a return code (here zero means no error or special conditions found).&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2018-09-27T04:29:57.403" LastActivityDate="2018-09-27T04:29:57.403" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7017" PostTypeId="2" ParentId="6998" CreationDate="2013-04-29T06:22:15.540" Score="3" Body="&lt;p&gt;Your question is correct, but this is still an open problem (see paper &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/fld.1798/citedby&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;). There are many methods to evolve CFL value in case of implicit methods. Every method has its own advantages and disadvantages. Choose the strategy which satisfies your needs or otherwise you can introduce efficient new strategy and publish it. All the best.&lt;/p&gt;&#xA;" OwnerUserId="4107" LastActivityDate="2013-04-29T06:22:15.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7018" PostTypeId="2" ParentId="7008" CreationDate="2013-04-29T07:38:59.623" Score="4" Body="&lt;p&gt;First of all, you should choose based on your application if you need high precision arithmetic (i.e. will you be happy with just IEEE double precision results for the polylog functions or do you need higher precision)? If you do need high precision, you can look in the family of tools around the GMP library. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't, you can use approximations. Some literature research pointed me to &lt;a href=&quot;http://kar.kent.ac.uk/21052/1/ThecompD.pdf&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt;. At the end of the article, is a &quot;selection table&quot;: based on the arguments of the polylogs that you need, you can select an approximation formula. But be careful to check stability and accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't need too many evaluations (not in a nested loop), I would just go for numerical quadrature using the double exponentional method. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-04-29T07:38:59.623" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7019" PostTypeId="2" ParentId="6978" CreationDate="2013-04-29T08:21:53.097" Score="4" Body="&lt;p&gt;In addition to @jed-brown's answer, I can highly recommend the book &lt;a href=&quot;http://ukcatalogue.oup.com/product/9780198528685.do#.UX4si6BQEyo&quot; rel=&quot;nofollow&quot;&gt;Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics&lt;/a&gt; by Elman, Silvester and Wathen. The book covers finite elements, but the general strategies for Krylov subspace methods also hold for finite volumes, of course.&lt;/p&gt;&#xA;" OwnerUserId="532" LastActivityDate="2013-04-29T08:21:53.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7020" PostTypeId="1" CreationDate="2013-04-29T12:56:22.433" Score="3" ViewCount="436" Body="&lt;p&gt;Let's consider the following equation&#xA;$$X=F_{1}XF_{1}^{T}+...+F_{p}XF_{p}^{T}+C$$&#xA;where $p$ is an positive integer and $C$ is a known positive semidefinite matrix. If we augment $F=[F_{1}...F_{p}]$ and $Y=diag (X...X)$, then the equation becomes&#xA;$$FYF^{T}−[I...0]Y[I...0]^{T}+C=0$$&#xA;seems like a generalized Lyapunov equation. However, there is a constraint on $Y$ for its diagonal form. How to compute $X$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I met with this problem for dealing with stability analysis for dynamic systems with multiple multiplicative noise.&lt;/p&gt;&#xA;" OwnerUserId="4185" LastActivityDate="2013-04-29T12:56:22.433" Title="On solution of a class of discrete-time Lyapunov equation for systems with multiplicaitve noise" Tags="|linear-algebra|matrix|matrix-equations|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7021" PostTypeId="2" ParentId="7005" CreationDate="2013-04-29T17:21:33.250" Score="2" Body="&lt;p&gt;I have &lt;a href=&quot;https://github.com/meawoppl/personal-tools/blob/master/do_once.py&quot; rel=&quot;nofollow&quot;&gt;a Python implementation&lt;/a&gt; that satisfies the interest you specify. Take a look at it to see a simple way to do this in languages capable of introspection.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any function that you &lt;a href=&quot;http://www.artima.com/weblogs/viewpost.jsp?thread=240808&quot; rel=&quot;nofollow&quot;&gt;decorate&lt;/a&gt; with it will be run once, and the results cached locally.  Subsequent invocations with:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the same arguments&lt;/li&gt;&#xA;&lt;li&gt;the same function code&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;will be loaded from disk.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It uses a different storage mechanism if the result is an array for faster saves and loads.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be a bunch harder to write in c/c++ and similar.  Though a simple trick might be salting the hash with a &lt;a href=&quot;http://www.cplusplus.com/doc/tutorial/preprocessor/&quot; rel=&quot;nofollow&quot;&gt;pre-processor directive such as &lt;strong&gt;TIME&lt;/strong&gt; or &lt;strong&gt;DATE&lt;/strong&gt;&lt;/a&gt; or some such if you want to err on the side of tossing your cached results after a recompile.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2013-04-29T17:21:33.250" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7023" PostTypeId="1" CreationDate="2013-04-29T23:18:06.980" Score="3" ViewCount="399" Body="&lt;p&gt;I mean, in handling boxed constraints? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of stability, and more importantly, the numerical performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already written some well-optimized and well-tested C/CUDA/C++ codes for several unconstrained optimzation methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I use augmented largrangian to handle constraints, however, it looks like bounded BFGS is quite different than the standard BFGS, I cannot do it over my existing BFGS routine,it require almost a complete re-written, I dont really want to do that unless there is significant gain there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have anyone, with practical experiences, can tell me if the rewritten is worth doing or I can stick to the let Lagrangian to handle it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw, I usually use BFGS to solve MLE type optimzation problems, where the evaluation of the objective function is costly.&lt;/p&gt;&#xA;" OwnerUserId="4170" LastEditorUserId="4170" LastEditDate="2013-04-29T23:31:27.453" LastActivityDate="2013-04-29T23:31:27.453" Title="How much better a bounded BFGS is compared to augmented Lagrangian method with BFGS?" Tags="|optimization|constraints|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7024" PostTypeId="1" CreationDate="2013-04-30T02:46:50.457" Score="3" ViewCount="152" Body="&lt;p&gt;I am trying to understand the LidarBoost algorithm as explained in &lt;a href=&quot;http://www.mpi-inf.mpg.de/~theobalt/cvpr09_superresolution.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; (PDF warning). My apologies if this isn't the appropriate exchange; I've posted in DSP, but I didn't get a response there, and the FAQ said that some image processing algorithms are acceptable here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how they take the original depth-images $Y_k$ and transform them into the up-sampled images $D_k$. I get how optical flow is used to align the $Y_k$ into a chosen reference frame, but is the transform between $Y_k$ and $D_k$ just a standard image up-sampling with nearest neighbors for the &quot;interpolation&quot; step? If that's the case, can someone explain how they get the term $W_k$ in their data term of the energy function? They say this about it, but I don't understand how to construct $W_k$ for each $k$:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$W_k \in \mathbb{R}^{\beta n x \beta m}$ is a banded matrix that&#xA;  encodes the positions of $D_k$ which one samples from during&#xA;  resampling on the high-resolution target grid&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm trying (so far frustratingly ineffectively) to get enough of a sense of the algorithm to try to implement it.&lt;/p&gt;&#xA;" OwnerUserId="4188" LastActivityDate="2013-05-22T23:05:56.550" Title="Explanation of LidarBoost Algorithm?" Tags="|image-processing|computer-vision|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7025" PostTypeId="1" CreationDate="2013-04-30T13:25:24.997" Score="0" ViewCount="12493" Body="&lt;p&gt;I have made a standard curve. The X-axis is logarithmic. The y-axis is linear.&#xA;I have added a logarithmic trendline &#xA;(y = -1.546ln(x) + 39.254; R² = 0.9906).&#xA;How can I re-arrange the equation to calculate an unknown X-axis value?&lt;/p&gt;&#xA;" OwnerUserId="4191" LastActivityDate="2016-07-10T14:14:39.830" Title="How do I extrapolate data from a NON-LINEAR (logarithmic) standard curve in Excel?" Tags="|nonlinear-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7026" PostTypeId="2" ParentId="7025" CreationDate="2013-04-30T13:52:27.900" Score="2" Body="&lt;p&gt;$$y = -1.546 \ln(x) + 39.254$$&#xA;$$\frac{39.254 - y}{1.546} = \ln(x)$$&#xA;$$\exp\left(\frac{39.254 - y}{1.546}\right) = x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In excel this is &lt;code&gt;=EXP((39.254 - Y)/1.546)&lt;/code&gt; where Y is the cell address containing the Y value.&lt;/p&gt;&#xA;" OwnerUserId="3565" LastActivityDate="2013-04-30T13:52:27.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7027" PostTypeId="1" CreationDate="2013-04-30T16:16:56.597" Score="6" ViewCount="204" Body="&lt;p&gt;It is well known that, when a system of linear equations is obtained from discretization of partial differential equation, the solution process can be accelerate significantly by multigrid technique. In the smoothing step of a multigrid cycle, relaxation method, such as Gauss-Seidel or Jacobi, is used. However, for relaxation method, convergence is only guaranteed for a limited class of matrices. For instance, the Gauss-Seidel Method only converges when the coefficient matrix is positive definite of diagonally dominant. My question is, if the coefficient matrix doesn't satisfy convergence requirement of relaxation method, is it still possible to perform multigrid technique? And, how to perform?&lt;/p&gt;&#xA;" OwnerUserId="4192" LastActivityDate="2013-04-30T18:40:38.053" Title="How to perform multigrid technique when relaxation methods don't converge?" Tags="|pde|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7028" PostTypeId="1" CreationDate="2013-04-30T17:26:59.687" Score="-1" ViewCount="876" Body="&lt;p&gt;I am trying to follow the Openfoam user guide but I'm stuck at the first tutorial &lt;a href=&quot;http://www.openfoam.org/docs/user/cavity.php&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt;.&#xA;In fact, when I try to run the &lt;em&gt;blockMesh&lt;/em&gt; command (using the administrator account), the following appears:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Writing polyMesh&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;--&gt; FOAM FATAL ERROR: &#xA;  Failed writing polyMesh.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;From function blockMesh&#xA;     in file blockMeshApp.C at line 325.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;FOAM exiting&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried to follow instructions given in this thread &lt;a href=&quot;http://www.cfd-online.com/Forums/openfoam/87129-polymesh-problem.html&quot; rel=&quot;nofollow&quot;&gt;2&lt;/a&gt; but none of the two here proposed can help me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanking you in advance,&#xA;Filippo&lt;/p&gt;&#xA;" OwnerUserId="4193" LastActivityDate="2013-05-01T05:38:04.927" Title="Open Foam User Guide Tutorial" Tags="|openfoam|" AnswerCount="1" CommentCount="2" ClosedDate="2013-05-12T00:25:59.233" ContentLicense="CC BY-SA 3.0" />
  <row Id="7029" PostTypeId="2" ParentId="7027" CreationDate="2013-04-30T18:40:38.053" Score="3" Body="&lt;p&gt;It is true that if the smoother is not convergent for high frequencies, that multigrid will not converge.  Note that this is weaker than being globally convergent: a relaxation method that is unstable for low frequencies can be used with &quot;compatible relaxation&quot; to yield a stable smoother that is only convergent when combined with a coarse-grid correction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;General convergence statements often use &lt;a href=&quot;http://en.wikipedia.org/wiki/Kaczmarz_method&quot; rel=&quot;nofollow&quot;&gt;Kaczmarz relaxation&lt;/a&gt; because it can be shown to always converge, although it is usually slower when methods such as Jacobi, Gauss-Seidel, or a polynomial smoother are convergent.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-04-30T18:40:38.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7030" PostTypeId="1" AcceptedAnswerId="7035" CreationDate="2013-04-30T19:54:50.387" Score="4" ViewCount="13043" Body="&lt;p&gt;I'm trying to get an animated 2D data surface on MatPlotLib. After a bit of search on the internet, i've found an example that &lt;em&gt;almost&lt;/em&gt; works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from mpl_toolkits.mplot3d import axes3d&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;import time&#xA;&#xA;def generate(X, Y, phi):&#xA;    R = 1 - np.sqrt(X**2 + Y**2)&#xA;    return np.cos(2 * np.pi * X + phi) * R&#xA;&#xA;plt.ion()&#xA;fig = plt.figure()&#xA;ax = axes3d.Axes3D(fig)&#xA;&#xA;xs = np.linspace(-1, 1, 50)&#xA;ys = np.linspace(-1, 1, 50)&#xA;X, Y = np.meshgrid(xs, ys)&#xA;Z = generate(X, Y, 0.0)&#xA;&#xA;wframe = None&#xA;tstart = time.time()&#xA;for phi in np.linspace(0, 360 / 2 / np.pi, 100):&#xA;&#xA;    oldcol = wframe&#xA;&#xA;    Z = generate(X, Y, phi)&#xA;    wframe = ax.plot_wireframe(X, Y, Z, rstride=2, cstride=2)&#xA;&#xA;    # Remove old line collection before drawing&#xA;    if oldcol is not None:&#xA;        ax.collections.remove(oldcol)&#xA;&#xA;    plt.draw()&#xA;&#xA;print 'FPS: %f' % (100 / (time.time() - tstart))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;reference: &lt;a href=&quot;http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14674.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14674.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with this plot is that is not interactive, the window does not rotate.&#xA;It seems the computation and/or the rendering blocks the UI&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is there a better approach for animated 2D data surface plotting?&lt;/p&gt;&#xA;" OwnerUserId="372" LastActivityDate="2017-04-08T21:05:22.487" Title="plotting a 2D animated data surface on MatPlotLib" Tags="|python|visualization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7031" PostTypeId="1" CreationDate="2013-04-30T22:11:19.793" Score="19" ViewCount="5948" Body="&lt;p&gt;I am interested in cases where Conjugate gradient works much better than GMRES method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, CG is preferable choice in many cases of SPD (symmetric-positive-definite) because it requires less storage and theoretical bound on convergence rate for CG is double of that GMRES. &#xA;Are there any problems where such rates are actually observed? Is there any characterization of cases where GMRES performs better or comparable to CG for same number of spmvs (sparse matrix-vector multiplications).&lt;/p&gt;&#xA;" OwnerUserId="3360" LastEditorUserId="336" LastEditDate="2013-05-02T13:21:19.770" LastActivityDate="2013-05-15T09:56:55.047" Title="Problems where Conjugate gradient works much better than GMRES" Tags="|linear-solver|conjugate-gradient|gmres|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7032" PostTypeId="1" CreationDate="2013-05-01T00:27:57.050" Score="1" ViewCount="208" Body="&lt;p&gt;I need to solve the follwing constrained optimization problem. Is there any MATLAB&#xA;toolbox which can  solve this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;minimize $A_1 x$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to $A_2 x =  0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_1$ is an $m\times n$ matrix where $m \lt n$ &#xA;and $A_2$ is a $(n-m) \times n$  matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if $A_1$ were a row vector, this would reduce to the standard linear-programming problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am not sure how to go about solving the above problem.&lt;/p&gt;&#xA;" OwnerUserId="3352" LastEditorUserId="1748" LastEditDate="2013-05-01T03:31:42.710" LastActivityDate="2013-06-21T11:41:19.733" Title="Solving a &quot;generalized&quot; linear programming problem" Tags="|linear-algebra|matlab|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7033" PostTypeId="2" ParentId="7032" CreationDate="2013-05-01T04:12:17.690" Score="3" Body="&lt;p&gt;You haven't explained what you mean by minimizing the $m$ element column vector $A_{1}x$.  There is a large literature devoted to &quot;multicriteria optimization problems&quot; or &quot;vector optimization problems&quot; that addresses this question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are you interested in the partial order that $x \succeq y$ if $x_{i} \geq y_{i}$ for $i=1, 2, \ldots, m$?  For example, in this ordering, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[&#xA;\begin{array}{c}&#xA;1 \\&#xA;1 \\&#xA;\end{array}&#xA;\right]&#xA;\succeq&#xA;\left[&#xA;\begin{array}{c}&#xA;0 \\&#xA;0 \\&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but there is no ordering between &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[&#xA;\begin{array}{c}&#xA;1 \\&#xA;0 \\&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[&#xA;\begin{array}{c}&#xA;0 \\&#xA;1 \\&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this component by component comparison only provides a partial order, the best that you can do is to find a Pareto optimal solution- that is a feasible vector $x^{*}$ such that for every other feasible vector $x^{'}$, either $x^{'} \succeq x^{*}$, or $x^{'}$ and $x^{*}$ can't be compared.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A standard technqique for finding Pareto optimal solutions to such problems is called &quot;scalarization.&quot;  In this technique, you multiply the vector objective function by a weight vector and then do conventional scalar minimization of the weighted objective function.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An introduction to vector optimization problems and the scalarization technique can be found in the convex optimization textbook by Stephen Boyd and Lieven Vandenberghe.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-05-01T04:12:17.690" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7034" PostTypeId="2" ParentId="7028" CreationDate="2013-05-01T05:38:04.927" Score="1" Body="&lt;p&gt;I would check if you are in the proper case directory and not in any other directory, such as &lt;code&gt;$CASE/constant/polyMesh&lt;/code&gt; or something similar. Have you copied the tutorial from &lt;code&gt;$FOAM_TUTORIALS&lt;/code&gt; to your home directory or somewhere else, where you have proper permissions to read and write? I generally suggest not to use the administrator's account for your daily work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps, otherwise please provide more information (OpenFOAM version, etc.)&lt;/p&gt;&#xA;" OwnerUserId="1960" LastActivityDate="2013-05-01T05:38:04.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7035" PostTypeId="2" ParentId="7030" CreationDate="2013-05-01T11:06:51.463" Score="4" Body="&lt;p&gt;Use the &lt;a href=&quot;http://matplotlib.org/api/animation_api.html&quot; rel=&quot;nofollow&quot;&gt;animation&lt;/a&gt; module. Here is a nice &lt;a href=&quot;http://jakevdp.github.io/blog/2012/08/18/matplotlib-animation-tutorial/&quot; rel=&quot;nofollow&quot;&gt;tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tweaking your code a bit,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from mpl_toolkits.mplot3d import axes3d&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import matplotlib.animation as animation&#xA;&#xA;def generate(X, Y, phi):&#xA;    R = 1 - np.sqrt(X**2 + Y**2)&#xA;    return np.cos(2 * np.pi * X + phi) * R&#xA;&#xA;fig = plt.figure()&#xA;ax = axes3d.Axes3D(fig)&#xA;&#xA;xs = np.linspace(-1, 1, 50)&#xA;ys = np.linspace(-1, 1, 50)&#xA;X, Y = np.meshgrid(xs, ys)&#xA;Z = generate(X, Y, 0.0)&#xA;wframe = ax.plot_wireframe(X, Y, Z, rstride=2, cstride=2)&#xA;ax.set_zlim(-1,1)&#xA;&#xA;def update(i, ax, fig):&#xA;    ax.cla()&#xA;    phi = i * 360 / 2 / np.pi / 100&#xA;    Z = generate(X, Y, phi)&#xA;    wframe = ax.plot_wireframe(X, Y, Z, rstride=2, cstride=2)&#xA;    ax.set_zlim(-1,1)&#xA;    return wframe,&#xA;&#xA;ani = animation.FuncAnimation(fig, update, &#xA;        frames=xrange(100), &#xA;        fargs=(ax, fig), interval=100)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1285" LastEditorUserId="1285" LastEditDate="2013-05-01T20:06:19.200" LastActivityDate="2013-05-01T20:06:19.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7036" PostTypeId="2" ParentId="2698" CreationDate="2013-05-01T14:28:53.020" Score="3" Body="&lt;p&gt;Actually there's no restriction that the number of intervals in the fine grid should have to be a power of 2. This number can be either odd or even. No big deal. Take 1-dimension problem as an example, suppose the number of intervals in the fine grid is 7(Thus the number of nodes is 8), then you can use the following restriction operator&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Columns 1 through 7&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      0.5         0.25            0            0            0            0            0&#xA;        0         0.25          0.5         0.25            0            0            0&#xA;        0            0            0         0.25          0.5         0.25            0&#xA;        0            0            0            0            0         0.25          0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Column 8&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        0&#xA;        0&#xA;        0&#xA;     0.25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the number of intervals in the fine grid is 8(Thus the number of nodes is 9), then the restriction operator should be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Columns 1 through 7&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      0.5         0.25            0            0            0            0            0&#xA;        0         0.25          0.5         0.25            0            0            0&#xA;        0            0            0         0.25          0.5         0.25            0&#xA;        0            0            0            0            0         0.25          0.5&#xA;        0            0            0            0            0            0            0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Columns 8 through 9&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        0            0&#xA;        0            0&#xA;        0            0&#xA;     0.25            0&#xA;     0.25          0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you see, whether the number of intervals in the fine grid is odd or even, the multigrid technique would work well.&lt;/p&gt;&#xA;" OwnerUserId="4192" LastActivityDate="2013-05-01T14:28:53.020" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7037" PostTypeId="2" ParentId="7031" CreationDate="2013-05-01T14:37:38.243" Score="9" Body="&lt;p&gt;One thing that CG has in its favor is that it's not minimizing the discrete $l^2$ norm for its residual polynomials (what GMRES does). It's minimizing a matrix-induced norm instead, and very often this matrix-induced norm ends up being very close to the energy norm for discretizations of physical problems, and frequently this is a much more reasonable norm to measure error in because of conservation properties coming from the physics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can actually achieve this sort of effect with GMRES too if performing a Cholesky factorization of a mass matrix isn't too expensive, you can force the inner products to be the energy inner products you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the cases where one should expect CG to perform very differently from GMRES then is when the constants implied in norm equivalence are very different. This can be true for example in a high order spectral-Galerkin method where the discrete $l^2$ norm used in GMRES treats all degrees of freedom as being equal, when in reality polynomial gradients are sharpest near boundaries (hence node clustering), and so the norm equivalence constants between that norm and say the continuous $L^2$ norm given by the mass matrix can be very large.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2013-05-01T18:02:32.237" LastActivityDate="2013-05-01T18:02:32.237" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7038" PostTypeId="1" CreationDate="2013-05-01T17:29:32.020" Score="3" ViewCount="115" Body="&lt;p&gt;I'm studying the usual wave equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial^2 \psi}{\partial x^2} - \frac{1}{c^2} \frac{\partial^2 \psi}{\partial t^2} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with complex field $\psi$, on a n-dimensional cube region of length $L$, which are $n-1$ spatial dimensions of length $L$, and one time dimension of duration $L/c$. Now i proceed to express the field in a Fourier expansion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \psi(x,y,..t) = \sum_{k, \omega} e^{I ( k_x x + k_y y + ... + \omega t)} \psi_{k,\omega}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If i do a cutoff in the expansion so that we only take $2M+1$ cells on each direction in k-space, that is, $- \frac{ \pi M }{L}, - \frac{ \pi (M-1) }{L}, .. 0 .. \frac{\pi M}{L}$ are the allowed values of the $k$ indices (and for $\omega$ too but with a $1/c$ factor somewhere) I am left with $(2M+1)^n$ complex fields to compute, because the field is not real, and the fourier expansion does not satisfy $\psi_{k_x, k_y, ...,\omega} = \psi_{-k_x, k_y, ..., \omega}^{*}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When i apply the Helmoltz condition on the bulk of the cube, i am left with $(2M+1)^n$ (complex) conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(-k_x^2 - k_y^2 - ... + \frac{\omega^2}{c^2} ) \psi_{k_x, k_y,...,\omega} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, it seems that the Helmoltz equation imposes the same number of contraints as there are unknowns. My question: how can boundary conditions be imposed on this problem without producing an over-determined system&lt;/p&gt;&#xA;" OwnerUserId="372" LastActivityDate="2013-05-01T17:29:32.020" Title="Eigenmode constraints on a complex wave field" Tags="|fourier-analysis|helmholtz-equation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7039" PostTypeId="1" CreationDate="2013-05-01T19:04:12.077" Score="2" ViewCount="1665" Body="&lt;p&gt;As stated in the title, I need to make a discrete 3D convolution of two matrix-valued functions. Or, actually, I need to convolve a matrix-valued function with a vector-valued function. That is, I need to compute the discrete version of $(f \ast X)(x)$ for some finite set of $x$'s, where $f:\mathbb{R}^3\to\mathbb{R}^{3 \times 3}$ and $X:\mathbb{R}^3\to\mathbb{R}^3$. Oh, and I'm using MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how do I do this in the fastest way? I am currently using four nested loops, which, needless to say, is quite slow. Is there any other way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thought about doing it as a multiplication in Fourier-space instead. But is there a risk of loosing some information or precision this way, given that $f$ is actually pretty ugly (an odd, slowly decaying function, with a hole cut in the center) and $X$ is random (as in, is actually a stochastic process)?&lt;/p&gt;&#xA;" OwnerUserId="3618" LastActivityDate="2013-05-01T22:12:50.230" Title="Discrete 3D convolution of matrix valued functions" Tags="|matlab|fourier-analysis|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7040" PostTypeId="2" ParentId="7039" CreationDate="2013-05-01T22:12:50.230" Score="3" Body="&lt;p&gt;I'd be pretty surprised if using Fourier techniques didn't help. Moreover, someone has already written a &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/24504&quot; rel=&quot;nofollow&quot;&gt;Matlab code for 3D fast convolution&lt;/a&gt; using FFT; that author tested it and found that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Good usage recommendation: &#xA;    In 1D, this function is faster than CONV for nA, nB &amp;gt; 1000. &#xA;    In 2D, this function is faster than CONV2 for nA, nB &amp;gt; 20. &#xA;    In 3D, this function is faster than CONVN for nA, nB &amp;gt; 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where nA, nB are the lengths of the input vectors A,B. At the very least, you can test that output against your program to compare the two, without worry of wasting time writing a Fourier-based method yourself and having it work poorly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, you may be right to be cautious about using the Fourier transform of your functions. How are these functions given to you? If they're defined at a discrete set of points to begin with, you needn't worry; the discrete Fourier transform is invertible after all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you're given some analytic expression that defines them in all of 3-space -- for example, $X$ is a Brownian sheet -- then you might be in trouble. In that case, sampling them at a finite number of points and using the discrete Fourier transform may either discard high-frequency components of the fields or alias them to low frequencies. Lack of smoothness is what hinders Fourier analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if these concerns do prejudice you against using the FFT, you could try using Matlab's parallel programming toolbox to speed up your nested loop. It's also worth considering the order in which you perform those loops. Matlab stores arrays in column-major order, so it's much faster to access &lt;code&gt;A(:,i)&lt;/code&gt; than it is to access &lt;code&gt;A(i,:)&lt;/code&gt;. Nesting your loops one way as opposed to the other could dramatically slow down a program because it has to take big skips around in memory rather than read entries consecutively.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-05-01T22:12:50.230" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7042" PostTypeId="2" ParentId="7031" CreationDate="2013-05-02T03:04:39.887" Score="3" Body="&lt;p&gt;One thing is that GMRES is never used wherever CG can be applied. I dont think it does make sense to compare these two. For SPD matrices, CG is definitely the winner because of the storage requirements and the reasons you mentioned above. A question that would be interesting is, to find an extension of CG, that is applicable to problems where CG can not be applied. There are methods like BiCG-stab that do not require linearly increasing memory like GMRES, but the convergence is not as good as GMRES (some times even with restarted GMRES).&lt;/p&gt;&#xA;" OwnerUserId="4203" LastActivityDate="2013-05-02T03:04:39.887" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7043" PostTypeId="2" ParentId="5600" CreationDate="2013-05-02T03:39:09.393" Score="2" Body="&lt;p&gt;Any iterative solver can beat direct methods only if the problem is sufficiently large (large, depends on several factors such as storage required, efficiency of implementation). And also any krylov method (for example GMRES) are good only if you use an appropriate preconditioning (in practice). If you are not using any preconditioning, krylov methods are of no use in general. I too work with block sparse matrices (these come from PDE applications) and observed that block preconditioned(overlapping additive schwarz) krylov solver (either restarted GMRES or BiCG-Stab) coupled with a coarse grid correction (or a multigrid) are scalable for these type of matrices.&lt;/p&gt;&#xA;" OwnerUserId="4203" LastActivityDate="2013-05-02T03:39:09.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7044" PostTypeId="2" ParentId="5600" CreationDate="2013-05-02T04:18:02.533" Score="1" Body="&lt;p&gt;If your matrix has dimension in the mid tens of thousands or less, use a direct method, although there aren't many freely available direct methods for indefinite symmetric systems (actually none that I know of are open source). There's &lt;a href=&quot;http://www.hsl.rl.ac.uk/catalogue/ma57.xml&quot; rel=&quot;nofollow&quot;&gt;MA57&lt;/a&gt; from the HSL, but that's only free for academic use. You can certainly ignore symmetry and use &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/umfpack/&quot; rel=&quot;nofollow&quot;&gt;UMFPACK&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At around the low tens of hundreds in dimension, the memory use of a direct method starts to exceed what a typical desktop computer can reasonably handle, so unless you have a beefy shared memory machine, you will need to move to iterative methods. For indefinite problems, you can specialize BiCG (biconjugate gradient) for symmetric systems, although breakdown is possible. There is a recently released &lt;a href=&quot;http://code.google.com/p/minres-qlp/&quot; rel=&quot;nofollow&quot;&gt;MINRES-QLP&lt;/a&gt; for symmetric systems which provides more numerical stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two methods require rather different implementations, since for direct methods you actually need to form the matrix, while in the iterative approach, you usually won't form the matrix explicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of reasons why one approach might be faster than the other, especially as a function of matrix dimension. For highly ill conditioned systems, iterative methods can stagnate pretty badly. For not-so-sparse matrices, direct methods end up creating lots of fill-in, which slows things down a lot. Also, the direct methods in Matlab are highly optimized (it uses UMFPACK or MA57 internally), while iterative methods are usually coded directly in Matlab, and there's fewer opportunities to exploit level 3 BLAS since the bottlenecks are the application of matvecs and dot products.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-05-02T04:18:02.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7045" PostTypeId="2" ParentId="5604" CreationDate="2013-05-02T05:30:25.010" Score="5" Body="&lt;p&gt;Apart from the methods based on Newton-Cotes mentioned in the other answers, there is what is now called &lt;em&gt;Gauss-Turán quadrature&lt;/em&gt; (see e.g &lt;a href=&quot;http://dx.doi.org/10.1137/0702011&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;http://dx.doi.org/10.1016/S0898-1221%2800%2985001-4&quot;&gt;this&lt;/a&gt;, as well as &lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;id=rS35pXlqY9MC&quot;&gt;the venerable reference&lt;/a&gt; by Walter Gautschi). This is a generalization of the usual Gaussian quadrature, where one can now exploit the knowledge of a function's derivatives in finding an optimal set of abscissas and weights that yield a quadrature rule that integrates functions of the form $\text{polynomial}\times\text{weight function}$ exactly. As expected, to use this rule, one is now expected to be able to evaluate your function and a number of its derivatives at arbitrary real points. A search in the usual places should be able to turn up a few more references.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2013-05-02T05:30:25.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7046" PostTypeId="2" ParentId="6979" CreationDate="2013-05-02T06:00:56.577" Score="17" Body="&lt;p&gt;Apart from the (now classical) &lt;a href=&quot;http://dx.doi.org/10.1007/978-3-642-86940-2_10&quot;&gt;Golub-Reinsch paper&lt;/a&gt; Brian notes in his answer (I have linked to the &lt;em&gt;Handbook&lt;/em&gt; version of the paper), as well as the (also now classical) predecessor &lt;a href=&quot;http://dx.doi.org/10.1137/0702016&quot;&gt;paper of Golub-Kahan&lt;/a&gt;, there have been a number of important developments in computing the SVD since then. First, I have to summarize how the usual method works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea in computing the SVD of a matrix is qualitatively similar to the method used for computing the eigendecomposition of a symmetric matrix (and, as noted in the OP, there is an intimate relationship between them). In particular, one proceeds in two stages: the transformation to a &lt;em&gt;bidiagonal matrix&lt;/em&gt;, and then finding the SVD of a bidiagonal matrix. This is completely analogous to the procedure of first reducing a symmetric matrix to tridiagonal form, and then computing the eigendecomposition of the resulting tridiagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For computing the SVD of a bidiagonal matrix, one particularly interesting breakthrough was the &lt;a href=&quot;http://dx.doi.org/10.1137/0911052&quot;&gt;paper by Jim Demmel and Velvel Kahan&lt;/a&gt;, which demonstrated that one can compute even the tiny singular values of a bidiagonal matrix with good accuracy, by suitably modifying the method initially proposed in Golub-Reinsch. This was then followed by the (re?)discovery of &lt;a href=&quot;http://dx.doi.org/10.1007/s002110050024&quot;&gt;the &lt;em&gt;dqd&lt;/em&gt; algorithm&lt;/a&gt;, which is a descendant of the old quotient-difference algorithm of Rutishauser. (Beresford Parlett gives a nice discussion &lt;a href=&quot;http://dx.doi.org/10.1017/S0962492900002580&quot;&gt;here&lt;/a&gt;.) If memory serves, this is now the preferred method used internally by LAPACK. Apart from this, it has always been possible to derive SVD versions of developments in the solution of symmetric eigenproblems; for instance, there is an SVD version of divide-and-conquer, as well as an SVD version of the old Jacobi algorithm (which may be &lt;a href=&quot;http://dx.doi.org/10.1137/0613074&quot;&gt;more accurate&lt;/a&gt; in some circumstances).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for bidiagonalization, one improved method was outlined in &lt;a href=&quot;http://dx.doi.org/10.1137/S0895479898343541&quot;&gt;Barlow's paper&lt;/a&gt;, which requires a bit more work than the original procedure of Golub and Reincsh, but yields more accurate bidiagonal matrices.&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2013-05-02T06:00:56.577" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7047" PostTypeId="1" CreationDate="2013-05-02T11:41:16.533" Score="3" ViewCount="214" Body="&lt;p&gt;Suppose I have some matrix &lt;code&gt;A&lt;/code&gt; whose SVD I know. Now, I am given &lt;code&gt;B&lt;/code&gt; which is &lt;code&gt;A&lt;/code&gt; plus some small perturbation. Is there any algorithm which can use the SVD of &lt;code&gt;A&lt;/code&gt; as an initial guess to gain speedup in the computation of the SVD of &lt;code&gt;B&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="898" LastActivityDate="2013-05-02T11:41:16.533" Title="SVD computation with &quot;initial guess&quot;" Tags="|matrix|svd|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7049" PostTypeId="1" AcceptedAnswerId="7050" CreationDate="2013-05-02T10:51:25.180" Score="5" ViewCount="6982" Body="&lt;p&gt;So the theory is straightforward. We have:&#xA;$$\frac{\partial^2U}{\partial t^2}=c^2 \frac{\partial^2U}{\partial x^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;discretizing it gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{U(i+1,j)- 2U(i,j) + U(i-1,j)}{(\Delta t)^2} = c^2 \frac{U(i,j+1)-2U(i,j) + U(i,j-1)}{(\Delta x)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U(i,j)$ is $U(t_i,x_j)$ with $t_i=i*\Delta t$ and $x_j=j*\Delta x$;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, coding it up in Matlab I have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    function [u_new] = Explicit_W(dx,dt,n,initialize_pos,initialize_vel)&#xA;%Inputs&#xA;X=1;   %X: length of the space domain&#xA;J=X/dx; %J: Total # of length intervals&#xA;f=str2func(initialize_pos); %initial conditions: u(0,x)&#xA;g=str2func(initialize_vel); %initial conditions: du/dt(0,x)&#xA;s2=(dt/dx)^  2;&#xA;c=2;&#xA;%Initializing the system, &#xA;%u_ancient, u_old and u_new are U(i-1,:),U(i-1,:) and U(i+1,:) respectively.&#xA;u_ancient=f(J);&#xA;u_old=    f(J)+ dt*g(J)+ 1/2*c^2*Tridiag(J+1,s2,-2*s2,s2)*f(J);&#xA;u_new=    Tridiag(J+1,c^2*s2,2-2*c^2*s2,c^2*s2)*u_old  - u_ancient;&#xA;&#xA;for i=1:n-2, % iterate n times, 2 of which are done already!&#xA;    %Iterating the system using the Explicit formulation&#xA;    temp=u_new;&#xA;    u_new=Tridiag(J+1,c^2*s2,2-2*c^2*s2,c^2*s2)*u_old - u_ancient;&#xA;    u_ancient=u_old;&#xA;    u_old=temp;&#xA;end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;Tridiag&lt;/code&gt; returns a matrix with entries on the tri-diagonals and its defined to be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [ A ] = Tridiag( N,r1,r2,r3 )&#xA;%This function creates a sparse tridiagonal matrix with fixed values&#xA;%on the Diagonals.&#xA;%   comes in handy when solving PDEs! &#xA;i=[1:N 2:N   1:N-1]; %row indices&#xA;j=[1:N 1:N-1 2:N  ]; %column indices&#xA;s=[(r2)*ones(1,N) r1*ones(1,N-1) r3*ones(1,N-1)]; %nonzero elements&#xA;A=sparse(i,j,s); % create the matrix&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and for &lt;code&gt;f&lt;/code&gt;, &lt;code&gt;g&lt;/code&gt; I use &lt;code&gt;Sin2&lt;/code&gt; and &lt;code&gt;Zero&lt;/code&gt; respectively, which are defined to be: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [ u0 ] = Sin2(L)&#xA;%TENT Summary of this function goes here&#xA;%   Initializes the array to Sin(2*pi*x)&#xA;u0=zeros(floor(L+1),1);&#xA;X=1;&#xA;x=(0:X/L:1)';&#xA;for j=1:(L+1),&#xA;      u0=sin(2*pi*x);&#xA;end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [ u0 ] = Zero( L)&#xA;%Summary of this function goes here&#xA;%   Initializes the array to a zero!&#xA;u0=zeros(floor(L+1),1);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, solving it analytically, if use &lt;code&gt;f=sin2&lt;/code&gt; and &lt;code&gt;g=zeros&lt;/code&gt;, we know the answer should be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x,t)=\frac{\sin(2\pi(x+ct))+\sin(2\pi(x-ct)}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the expected range is $\{-1,1\}$. but when I execute &lt;code&gt;Explicit_W&lt;/code&gt;, say:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Explicit_W(0.05,0.0010,30,'Sin2','Zero')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get funny results, ranging from -124 to 124! Here is a picture of the results:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wON52.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/wON52.png&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it has the form of the solution, but something is scaling it up! I have checked the sub-functions used in this on first order problems so I know there is nothing wrong with them. Whatever is causing the problem is in the main function. Any idea why it's acting the way it does?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it was brought to my attention that I had forgotten to implement the boundary conditions, here $u(0,t)=u(1,t)=0.$ Silly mistake!&#xA;So the modified section of the code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u_ancient=f(J);&#xA;u_old=    f(J)+ dt*g(J)+ 1/2*c^2*Tridiag(J+1,s2,-2*s2,s2)*f(J);&#xA;u_new=    Tridiag(J+1,c^2*s2,2-2*c^2*s2,c^2*s2)*u_old  - u_ancient;&#xA;c1=0;c2=0;%boundary values&#xA;%enforcing the boundary conditions&#xA;u_new(1)=c1; u_new(end)=c2;&#xA;u_old(1)=c1; u_old(end)=c2;&#xA;for i=1:n-2,&#xA;    %Iterating the system using the Explicit formulation&#xA;    temp=u_new;&#xA;    u_new=Tridiag(J+1,c^2*s2,2-2*c^2*s2,c^2*s2)*u_old - u_ancient;&#xA;    u_new(1)=c1;  %boundary conditions&#xA;    u_new(end)=c2;%boundary conditions&#xA;    u_ancient=u_old;&#xA;    u_old=temp;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That obviously fixes the issue with the boundary points, but I am still getting the wrong picture!! -&gt; &lt;a href=&quot;https://i.stack.imgur.com/lrVH7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/lrVH7.png&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;It's Working!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had fixed the issues that was pointed out and the code works fine now! it reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    function [u] = Explicit_W(dx,dt,n,initialize_pos,initialize_vel)&#xA;%Inputs&#xA;%h: length of spatial intervals&#xA;X=1;   %X: length of the space domain&#xA;J=X/dx; %J: Total # of length intervals&#xA;f=str2func(initialize_pos);&#xA;g=str2func(initialize_vel);&#xA;s=(dt/dx);&#xA;c=2;%velocity&#xA;r=(s*c)^2;&#xA;c1=0;c2=0;%boundary values at the two ends&#xA;u=Zero(J+2);u(1)=c1;u(end)=c2;%output solution with its ends fixed&#xA;b=Zero(J);b(1)=c1;b(end)=c2;  %vector b enforces the boundary conditions&#xA;%u_new and u_old are solution vectors with their ends cut off.&#xA;u_old=f(J);%Initializing u(0)&#xA;u_new=f(J)+ dt*g(J)+ 1/2*r*(Tridiag(J-1,1,-2,1)*f(J)+b);&#xA;for i=1:n-2,&#xA;    %Iterating the system using the Explicit formulation&#xA;    temp=u_new; &#xA;    u_new=r*(Tridiag(J-1,1,-2,1)*u_new+b)+2*eye(J-1)*u_new - u_old;&#xA;    u_old=temp;&#xA;end&#xA;u(2:end-1)=u_new;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4205" OwnerDisplayName="Shb" LastEditorUserId="4205" LastEditDate="2013-05-04T00:02:23.583" LastActivityDate="2013-05-04T00:02:23.583" Title="Implementing Explicit formulation of 1D wave equation in Matlab" Tags="|pde|finite-element|matlab|finite-difference|linear-solver|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7050" PostTypeId="2" ParentId="7049" CreationDate="2013-05-02T17:36:52.913" Score="3" Body="&lt;p&gt;There may be several issues at work here, but one problem that stands out is your implementation of boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're working on the interval $[0,1]$, and partitioning it into $J = 1/dx$ subintervals. Including nodes on the boundary, this gives you $J+1$ total points, which we can call $x_1,\ldots, x_{J+1}$, with $x_1 = 0$ and $x_{J+1} = 1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because you are imposing Dirichlet boundary conditions, the value of $U$ at $x_1$ and $x_{J+1}$ is fixed throughout the simulation, and thus these points should not be included in your update matrix. Your update is of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U(i+1,:) = (2I + c^2s_2A)U(i,:) - U(i-1,:)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $s_2 = (dt/dx)^2$ and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = \left(\begin{array}{ccccc} -2 &amp;amp; 1 &amp;amp; &amp;amp; &amp;amp; \\ 1 &amp;amp; -2 &amp;amp; 1 &amp;amp; \\  &amp;amp; 1 &amp;amp; -2 &amp;amp; 1\\ &amp;amp;&amp;amp;&amp;amp;\ddots&amp;amp;&amp;amp;\\&amp;amp;&amp;amp;&amp;amp;1&amp;amp;-2\end{array}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is the standard Laplacian matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But currently, you are applying this update to the vector $U(i,1:J+1)$, when you should be applying it to $U(i,2:J)$. Put differently, you should only be storing in &lt;code&gt;u_ancient&lt;/code&gt;, &lt;code&gt;u_old&lt;/code&gt;, and &lt;code&gt;u_new&lt;/code&gt; the value of $U$ at $x_2,\ldots,x_{J}$. In fact, this formulation of the problem -- and homogenous Dirichlet boundary conditions -- are encoded directly into the matrix $A$ that you are (implicitly) using. Observe that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;AU(i,2:J) &amp;amp;= \left(\begin{array}{c} -2U(i,1) + U(i,2) \\ U(i,1) - 2U(i,2) + U(i,3) \\ \vdots\\U(i,J-1) - 2U(i,J)\end{array}\right)\\&#xA;&amp;amp;=\left(\begin{array}{c} U(i,0) -2U(i,1) + U(i,2) \\ U(i,1) - 2U(i,2) + U(i,3) \\ \vdots\\U(i,J-1) - 2U(i,J) + U(i,J+1)\end{array}\right)&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the first step is just matrix multiplication, but the second step follows precisely because you have assumed homogenous Dirichlet boundary conditions to begin with, i.e. because $U(i,0) = U(i,J+1) = 0$. This hopefully explains why you needn't include the points $x_1$ and $x_{J+1}$ in your system. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: Upon re-reading the question, it occurs to me that the issue outlined above shouldn't have altered the solution on account of the problem's explicit formulation, so I'm afraid this doesn't explain your aberrant results. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;ANOTHER EDIT: I think I found your problem: the way you've named your variables has led you to implement the time-stepping incorrectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that after the initial assignment of &lt;code&gt;u_ancient&lt;/code&gt;, &lt;code&gt;u_old&lt;/code&gt;, and &lt;code&gt;u_new&lt;/code&gt;, it is the variable &lt;code&gt;u_new&lt;/code&gt; that holds the &lt;em&gt;current&lt;/em&gt; value of $U$. In other words &lt;code&gt;u_old&lt;/code&gt; is $U(i-1,:)$ and &lt;code&gt;u_new&lt;/code&gt; is $U(i,:)$. So in the next step, you want to base the &lt;em&gt;new&lt;/em&gt; value, $U(i+1,:)$, off of the &lt;em&gt;current&lt;/em&gt; value, $U(i,:)$, and the value that is one step behind, $U(i-1,:)$. You never actually need the value that is two timesteps behind, which you have named &lt;code&gt;u_ancient&lt;/code&gt;. In contrast, you've been performing the update&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U(i+1,:) = (2I+c^2s_2A)U(i-1,:) - U(i-2,:)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this is inconsistent (in the techinical sense) with the PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fix is very simple. You don't need the variable &lt;code&gt;u_ancient&lt;/code&gt; at all, for one. And your loop should be replaced by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:n-2&#xA;    temp=u_new&#xA;    u_new=Tridiag(J+1,c^2*s2,2-2*c^2*s2,c^2*s2)*u_new - u_old&#xA;    u_old=temp;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-05-02T19:22:47.950" LastActivityDate="2013-05-02T19:22:47.950" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7051" PostTypeId="1" AcceptedAnswerId="7052" CreationDate="2013-05-02T20:48:46.983" Score="4" ViewCount="78" Body="&lt;p&gt;Consider a multivariate polynomial $f(x) = f(x_1, \ldots, x_n)$ with maximum degree $d$.  Following the linear symbolic perturbation scheme described in &lt;a href=&quot;http://link.springer.com/content/pdf/10.1007%2F3-540-57785-8_127.pdf&quot; rel=&quot;nofollow&quot;&gt;Seidel 1998&lt;/a&gt;, I want to evaluate the limit&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\lim_{\epsilon \to 0^+} \textrm{sign}~ f(x + \epsilon y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some $x,y \in \mathbb{R}^n$.  This limit is given by the lowest degree nonzero coefficient of the polynomial&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g(\epsilon) = f(x + \epsilon y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the polynomial is given as an add/multiply expression DAG of size $m$, we can evaluate $f_i(x+ty)$ for each node $f_i$ of the expression DAG, which takes at most $O(m(d+1)^2)$ using naive polynomial multiplication (and probably less in practice since most nodes will have lower degree).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively (as noted in the paper), we could evaluate $g(\epsilon)$ for $\epsilon = 0, \ldots, n-1$ and use polynomial interpolation to recover the coefficients.  This takes time $O(m(d+1)+h(d))$ where $h(d)$ is the time required for polynomial interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Is there a faster method?  In particular, is $O(m(d+1))$ possible?&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-05-02T20:54:07.453" Title="Fastest method for evaluating the limit of the sign of a polynomial" Tags="|computational-geometry|polynomials|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7052" PostTypeId="2" ParentId="7051" CreationDate="2013-05-02T20:54:07.453" Score="4" Body="&lt;p&gt;As I was writing the question, I realized that in fact &lt;em&gt;both&lt;/em&gt; of the given schemes achieve $O(m(d+1))$.  This is because we always have $m \ge d$.  In the expression tree scheme, the cost of quadratic polynomial multiplication at higher nodes in the tree can be amortized to lower nodes.  In the polynomial interpolation scheme, at worst $h(d) = O(d^2)$ by precomputing the necessary matrices, and $O(m(d+1)+d^2) = O(m(d+1))$ since $m \ge d$.  The polynomial interpolation scheme is also nicely modular, since it works even if $f(x)$ is given as a black box.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-05-02T20:54:07.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7053" PostTypeId="1" AcceptedAnswerId="7054" CreationDate="2013-05-03T03:04:03.567" Score="1" ViewCount="1031" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Variants of this question have been crossposted to &lt;a href=&quot;https://stackoverflow.com/questions/16351039/is-there-a-built-in-function-in-matlab-to-determine-if-a-permutation-is-even-or&quot;&gt;Stack Overflow&lt;/a&gt; and &lt;a href=&quot;https://math.stackexchange.com/questions/379809/is-there-a-name-for-this-function-parity-of-a-finite-sequence&quot;&gt;Mathematics Stack Exchange&lt;/a&gt;. Additional answers may be found at these other sites.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Computational Science People:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I originally posed this as part of a question at &lt;a href=&quot;https://math.stackexchange.com/questions/379809/is-there-a-name-for-this-function-parity-of-a-finite-sequence&quot;&gt;https://math.stackexchange.com/questions/379809/is-there-a-name-for-this-function-parity-of-a-finite-sequence&lt;/a&gt; . The title pretty much says it all.  Given a finite sequence of real numbers which I know are distinct, I want an easy way to compute whether it is an even permutation of an increasing sequence or an odd permutation of an increasing sequence.  For example, $(5, 1.3, 8, 6, 10.5)$ is an even permutation of the increasing sequence $(1.3, 5, 6, 8, 10)$, so the answer is &quot;even&quot;.  $(2, 3, 5, 1)$ is an odd permutation of the increasing sequence $(1, 2, 3, 5)$, so the answer is &quot;odd&quot;.  I can get Matlab to give me a permutation &lt;em&gt;vector&lt;/em&gt;, which for my first example would be $[2\ 1\ 4\ 3\ 5]$, but I cannot find a built-in function in Matlab to compute the sign of that permutation vector.  If I could get Matlab to give me the permutation &lt;em&gt;matrix&lt;/em&gt;, I could take the determinant of that matrix.  People have posted quite long Matlab files on the Web to accomplish this, but I'm hoping I can avoid that.  I would also be willing to use Maple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please no nitpicking or unconstructive comments, which are very common at Math Stack Exchange.  I have less experience here, but I'm hoping people are more polite here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stefan (STack Exchange FAN)     &lt;/p&gt;&#xA;" OwnerUserId="3821" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-05-10T00:57:43.583" Title="How can I compute whether a sequence is an even or odd permutation of an increasing sequence?" Tags="|matlab|combinatorics|" AnswerCount="1" CommentCount="0" ClosedDate="2013-05-03T15:46:07.997" ContentLicense="CC BY-SA 3.0" />
  <row Id="7054" PostTypeId="2" ParentId="7053" CreationDate="2013-05-03T03:56:48.427" Score="3" Body="&lt;p&gt;Suppose you have a permutation vector &lt;code&gt;permVec&lt;/code&gt;. Then a permutation matrix should be calculated as follows (modulo some MATLAB syntax bugs, since it's been a while):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;permMat = function permVecToMat(permVec)&#xA;    n = length(permVec);&#xA;    permMat = zeros(n,n);&#xA;    for i = 1:n&#xA;        permMat(i, permVec(i)) = 1&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you just take the determinant, and you're done. Consider the code BSD-licensed, even though the Stack Exchange license is CC-BY-SA 3.0 (and I loathe it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternate way of calculating the sign of a permutation would be to convert it into a product of transpositions. If the number of transpositions in such a decomposition is even, the permutation is even; otherwise, the permutation is odd. However, I don't know an algorithm for decomposing a permutation into a product of transpositions.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-05-04T07:02:21.443" LastActivityDate="2013-05-04T07:02:21.443" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="7055" PostTypeId="1" CreationDate="2013-05-03T03:59:25.247" Score="3" ViewCount="214" Body="&lt;p&gt;Given an energy level $\mu$, I'm looking to calculate the eigenvectors corresponding to the time-independent Schrodinger operator on the torus (that is, periodic boundary conditions)-- $H = -h^2 \Delta + V$ for some nice, bounded potential $V$ -- whose eigenvalues are nearest to $\mu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there are some standard techniques for solving this particular problem, where $h$ can be rather small, say $h = 0.001$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current method is the naive approach (by my reckoning): I use the DFT to convert into phase space, whereby I can compute the map $\hat{u} \mapsto \hat{(H u)} = h^2 |k|^2 \hat{u} + \hat{V} \ast \hat{u}$, and then proceed by using the standard Krylov method for the matrix $(\hat{H} - \mu I)^{-1}$, where the map corresponding to that inverse is computed using biconjugate gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several key problems with this approach. The first is that $h$ is very small, so for certain energy levels I must use the DFT up to a very high frequency in order to get an accurate discretization, but this means that $\hat{H}$ is a both large (larger than $10000\times 10000$) and dense, so Krylov methods are still quite expensive, especially since on each Arnoldi iteration I have to do a full set CG iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other standard methods that I should try? I'd be grateful for any suggestions. Even if it's something obvious, I probably haven't thought of it yet because I'm not very well-versed in numerical eigenvalue problems, especially not in the case of differential operators.&lt;/p&gt;&#xA;" OwnerUserId="3477" LastActivityDate="2013-05-03T16:21:21.023" Title="Solving the elliptic eigenproblem with periodic boundary conditions" Tags="|linear-algebra|eigenvalues|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7056" PostTypeId="1" CreationDate="2013-05-03T08:05:57.113" Score="0" ViewCount="4041" Body="&lt;p&gt;I want to solve job assignment problem using &lt;em&gt;Hungarian algorithm of Kuhn and Munkres&lt;/em&gt; in case when matrix is not square. Namely we have more jobs than workers.&#xA;In this case adding additional row is recommended to make matrix square.&#xA;For example in the following &#xA;&lt;a href=&quot;http://books.google.kz/books?id=yXYzLm1e_IUC&amp;amp;pg=PA94&amp;amp;lpg=PA94&amp;amp;dq=job%20assignment%20problem%20matrix%20is%20non%20square&amp;amp;source=bl&amp;amp;ots=1LU3_P-Ss5&amp;amp;sig=k4hlPV0GLjUDT9PxoLEsomt0K6Q&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=lFODUcW_FcKQOJfUgPgJ&amp;amp;ved=0CFIQ6AEwBQ#v=onepage&amp;amp;q=job%20assignment%20problem%20matrix%20is%20non%20square&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tWzYi.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;And here task IV is assumed to be done.&#xA;But in real we do not have man D. Who will actually do task IV?&#xA;Can someone explain this phenomena?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general I want to complete all tasks by loading workers uniformly and get maximum cost.&#xA;So how to implement this task by using job assignment algorithm above?&lt;/p&gt;&#xA;" OwnerUserId="3544" LastActivityDate="2013-06-21T11:33:06.027" Title="Job assignment problem" Tags="|linear-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7057" PostTypeId="1" CreationDate="2013-05-03T13:48:21.167" Score="2" ViewCount="155" Body="&lt;p&gt;Wikipedia lists a lot of FFT algorithms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cooley–Tukey FFT algorithm, Prime-factor FFT algorithm, Bruun's FFT algorithm, Rader's FFT algorithm, and Bluestein's FFT algorithm&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros and cons of each algorithms?&#xA;And which one do you prefer?&lt;/p&gt;&#xA;" OwnerUserId="4212" LastEditorUserId="127" LastEditDate="2013-05-08T09:14:35.730" LastActivityDate="2013-05-08T09:14:35.730" Title="differences between all the FFTs?" Tags="|fourier-analysis|" AnswerCount="0" CommentCount="5" ClosedDate="2013-05-12T00:27:44.693" ContentLicense="CC BY-SA 3.0" />
  <row Id="7058" PostTypeId="2" ParentId="7056" CreationDate="2013-05-03T13:50:53.143" Score="3" Body="&lt;p&gt;&lt;strong&gt;Purpose of introducing a dummy variable:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your assignment problem is such that, the number of jobs is greater than number of persons available. In every assignment problem, it is assumed that each person will do one job, and a job will be done by only one person. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence assuming that you are a manager, and you have three workers working for you, and you have four jobs, you will accept only three jobs out of four. Now the question is which three you will accept and how would you assign them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you introduce a dummy person. Now do the assignments using standard &lt;a href=&quot;http://en.wikipedia.org/wiki/Hungarian_algorithm&quot; rel=&quot;nofollow&quot;&gt;Hungarian Algorithm&lt;/a&gt;. So three workers get three jobs such that net man-hours are minimum. And outsource the remaining job (in your case, job no. $IV$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solution helps choosing those three jobs, which can be done in the fastest way in your firm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Answers to your questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) No. Nobody in your firm will do job $IV$. The Hungarian algorithm just helped you to identify which three jobs to accept and how to assign them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) What do you mean by 'get maximum cost'? This is a minimization problem and numbers in the table represent man-hours, not the cost. But assuming you want to maximize profit lets say, then you have to convert the maximization problem into minimization problem (as done &lt;a href=&quot;http://e-university.wisdomjobs.com/quantitative-techniques-for-management/chapter-1601-297/assignment-model_maximization-problem_9902.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;) and solve as usual. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more references as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The logic behind the Hungarian algorithm is discussed in &lt;a href=&quot;http://www.youtube.com/watch?v=BUGIhEecipE&quot; rel=&quot;nofollow&quot;&gt;this video&lt;/a&gt; nicely. While the flowchart along with explanation for the algorithm is given in &lt;a href=&quot;http://faculty.ksu.edu.sa/jkhan/Documents/OR/opt-lp5.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; link. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the best.&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="1567" LastEditDate="2013-05-03T17:20:30.673" LastActivityDate="2013-05-03T17:20:30.673" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7059" PostTypeId="2" ParentId="1136" CreationDate="2013-05-03T16:10:30.130" Score="0" Body="&lt;p&gt;CPU cycles are not precious.  In the time you spent posting this question, you could have run e.g. H2O with MP2/cc-pVDZ in both codes and compared the results, which would answer your question sufficiently.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-05-03T16:10:30.130" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7060" PostTypeId="2" ParentId="7055" CreationDate="2013-05-03T16:21:21.023" Score="2" Body="&lt;p&gt;A question first: What program are you using to perform the eigendecomposition of your matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather than look for an answer in phase space, you may be better off considering your problem in physical space instead. In that case, you can use the standard finite difference formula to approximate $\Delta u$ at each point, then form a matrix corresponding to that system, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for this is that the resulting matrix is sparse, so there is less arithmetic to do to compute a matrix-vector product. If you're looking at high frequencies, you'll need a lot of points to resolve them and thus your system will be of high dimension; sparsity really saves you here. Matlab and Scipy both have built-in sparse matrix data types, which you'll have to use in order to take advantage of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also mentioned using the Arnoldi process. Using the Lanczos process will be faster, since your problem is symmetric. If you're using a program smart enough to detect that your matrix is symmetric and act accordingly then you needn't worry about this, but if you explicitly selected Arnoldi then you could be doing unnecessary work solving upper Hessenberg systems when it could be solving tridiagonal systems. Nonetheless, you're still finding the eigenvalues closest to zero of the operator $(H-\mu I)^{-1}$ -- that at least doesn't change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also point you to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6845/why-does-scipy-eigsh-produce-erroneous-eigenvalues-in-case-of-harmonic-oscilla/6850#6850&quot;&gt;this thread&lt;/a&gt; as a caveat: eigensolvers can give erroneous results in a way which, if you're interpreting the results as describing a quantum system, are very unphysical. Long story short, don't look for the whole spectrum of a matrix, just the first few eigenvalues. It sounds like that's what you're doing anyway.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-05-03T16:21:21.023" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7061" PostTypeId="2" ParentId="152" CreationDate="2013-05-03T16:41:52.890" Score="4" Body="&lt;p&gt;The answers previously posted are excellent but have focused mostly on the node architecture, which I think reflects the fact that MPI is generally regarded as sufficient as the internode programming models &lt;em&gt;in most cases&lt;/em&gt; and that it is intranode parallelism where we struggle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are my attempts to answer two questions that are not yet answered or answered in a relatively limited way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What assumptions should I make about process interconnect architecture?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will consider three properties of networks: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;latency,&lt;/li&gt;&#xA;&lt;li&gt;bandwidth, and &lt;/li&gt;&#xA;&lt;li&gt;concurrency.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Latency is inversely proportional to frequency.  We know that frequency scaling has stagnated.  Hence, one can conclude that latency is unlikely to decrease significantly in the future.  The MPI send-recv latency on Blue Gene/Q is around 2 us, which corresponds to 3200 cycles.  More than half of that latency is software, but a good portion of it is required by the MPI standard; extensive tuning might reduce the latency to close to 1 us, particularly if one can assert that MPI wildcards are not going to be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, the hardware latency for packet injection on Blue Gene and Cray systems is around 1 us.  If anything, increasing node-level concurrency makes it difficult to keep this number so low, but I am optimistic that hardware designers will find ways to keep the latency under 5 us for the foreseeable future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Network bandwidth is trivially increased by increasing the number of network links.  This is only part of the story, however.  One put 1000 outbound links on a node and not be able to use them if the processor(s) cannot drive the network at full bandwidth.  For example, some supercomputers bottleneck in the bus (e.g. HyperTransport) rather than the network, in terms of injection bandwidth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are no fundamental limits to network bandwidth, only practical ones.  Bandwidth costs money and power.  System designers will have to factor in the trade-offs between network bandwidth and other parts of the machine when developing future systems.  Many codes are not network-bandwidth limited, so it seems unlikely that we will see machines with dramatically more per-connection bandwidth in the future.  However, the bandwidth per node should increase proportional to the compute power so there need to be multiple connections per node to scale up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The third property of networks that is often overlooked in formal models is how many messages can be sent a single time.  Having a network with 1 ns latency and/or 1 TB/s bandwidth that can only send 1 message at a time would be entirely useless for most usages.  It is important to be able to send lots of messages from lots of threads at the same time and for the network to not collapse under contention.  Both Cray and Blue Gene systems now achieve in excess of 1 MMPS (million messages per second).  I can't remember the exact numbers, but both are able to achieve a significant fraction of peak bandwidth with small messages.  An ideal network might be able to hit peak bandwidth with any size message, but this is impossible in practice due to packet header and related bookkeeping overheads.  However, being able to send millions of 1KB messages at a time and saturate the injection bandwidth with this usage might be more useful than being able to do the same for thousands 1MB messages or a handful of 1GB messages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an incomplete and imperfect answer.  Others are welcome to try to improve it or suggest things I should improve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Will Partitioned Global Address Space languages be available &quot;in production&quot; on petascale machines?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cray XE, XK and XC systems have a production-quality UPC and CAF compiler.  Blue Gene systems can be delivered with XLUPC and XLCAF but no one asks for this so it is not delivered.  PERCS has production-grade XLUPC and XLCAF compilers but no large-scale installations that are accessible to the scientific community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coarrays are part of Fortran 2008, although the implementations in Intel and GNU Fortran are not high-quality yet.  The Intel implementation is reputed to work but also be quite slow (there's a paper at PGAS12 about it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the PGAS programming model (since programming models - not programming languages - are the subject of the original question), the Global Arrays library is a reasonable approximation to production-quality in many cases.  As a runtime, it's not as robust as MPI, but MPI is pretty unique in terms of how quality the implementations are.  The ARMCI-MPI implementation of ARMCI makes Global Arrays much more stable, albeit slower in some cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's relatively easy to implement PGAS-style constructs in a production quality way using MPI-3 RMA.  If someone posts a new question about this, I would be happy to answer it.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-05-03T16:41:52.890" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7062" PostTypeId="1" AcceptedAnswerId="7069" CreationDate="2013-05-03T22:07:10.557" Score="5" ViewCount="457" Body="&lt;p&gt;I was having this discussion with a colleague of mine a week ago and thought I might as well bring it here to a broader audience. That said, I am not certain this is a kind of question that is appropriate for scicomp. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyhow, given their abundance and, relatively low prices, has anyone thought of using smartphones for computational projects? Can their cellular/3G/4G/WiFi/Bluetooth/etc. network capability be used for any interesting collective, maybe (embarrassingly) parallel, computational application? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: Could not really think of a more appropriate tag!&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-08-24T04:04:51.893" Title="Any useful application for smartphones in the world of computational science?" Tags="|parallel-computing|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7063" PostTypeId="1" AcceptedAnswerId="7073" CreationDate="2013-05-04T07:54:14.030" Score="2" ViewCount="1427" Body="&lt;p&gt;From what I understand the GMRES method is (using Arnoldi Iterations/Modified Gram-Schmidt):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first vector of the Krylov subspace span of A is the normalized vector $\frac{\vec b - A\vec x_0} {|| \vec b - A\vec x_0 ||}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each iteration i, calculate a single new orthonormal vector of the existing Krylov subspace. If the norm of that vector is 0 (or close to 0 in floating-precision calculations), then the subspace is &quot;complete&quot;, and we should be able to calculate the converged solution. In theory, this occurs when we have N orthonormal vectors (where N is the size of A).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution can then be computed using $x = x_0 + V y$, where V is the orthonormal vectors computed above, and y minimizes $||H y - \beta e_0||$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\beta = ||\vec b - A\vec x_0||$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e_0 = (1, 0, ..., 0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be solved in a few different ways, (say, the way Saad and Schultz proposed in their original paper using Givens rotations/QR decomposition interleaved with the Arnoldi iteration), and in theory x is converged to the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I tried applying this to a sample test dataset:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{bmatrix}&#xA;1 &amp;amp; -0.25 &amp;amp; 0\\&#xA;0 &amp;amp; 1 &amp;amp; -0.25\\&#xA;0 &amp;amp; 0 &amp;amp; 1&#xA;\end{bmatrix}&#xA;\vec x =&#xA;\begin{bmatrix}&#xA;1\\&#xA;1\\&#xA;1&#xA;\end{bmatrix}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with $\vec x_0$ as a vector of 0's, I calculated 3 valid orthonormal vectors (calculated with double precision floats):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{bmatrix}&#xA;0.57735026918962584\\&#xA;0.57735026918962584\\&#xA;0.57735026918962584&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;-0.40824829046386368\\&#xA;-0.40824829046386368\\&#xA;0.81649658092772548&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;0.70710678118655124\\&#xA;-0.7071067811865438\\&#xA;2.5639502485114192e-15&#xA;\end{bmatrix}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As expected, the 4th vector calculated has a norm close to 0, and is not orthogonal to the other vectors in the Krylov subspace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, after the first &quot;complete&quot; run of the GMRES algorithm (so I have N=3 orthonormal vectors to work with), I calculated x as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{bmatrix}&#xA;1.3075447066949195\\&#xA;1.2455012338896092\\&#xA;1.0068140690151826&#xA;\end{bmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, the &quot;true&quot; solution is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{bmatrix}&#xA;1.3125\\&#xA;1.25\\&#xA;1&#xA;\end{bmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems too large of a difference for me to be able to say that the difference is a result of floating point arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the problem has converged (or stagnated), I would imagine re-running the GMRES would not significantly change the solution. However, running GMRES again would improve the solution significantly up to 3 or 4 times. Repeating the test for similar problems with different dimensions different dimensions and again about re-runs of GMRES improved the solution till it was closer to a small multiple of machine epsilon of the true solution (actual number of times varies slightly, no where near the order of the problem size).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I misunderstanding what convergence means in terms of the GMRES algorithm, or is there something else I'm missing in my implementation? Or is this really a result of imprecise arithmetic and extra iterations of the GMRES algorithm just happen to improve to solution?&lt;/p&gt;&#xA;" OwnerUserId="1295" LastEditorUserId="1295" LastEditDate="2013-05-04T18:50:16.220" LastActivityDate="2013-08-03T04:50:51.503" Title="Convergence of GMRES" Tags="|linear-algebra|algorithms|iterative-method|convergence|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7064" PostTypeId="1" AcceptedAnswerId="7066" CreationDate="2013-05-04T08:45:22.300" Score="1" ViewCount="1361" Body="&lt;p&gt;can you help me? I have a fluid simulation in 2D, where I compute by finite differences on the grid the velocity field and pressure. So this is my output from the program. Now I have to show how the vertical component of traction on the boundary looks like. But I am not sure what the word traction means, what I have to compute. I can compute the stress from the velocity field as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sigma_{xy}=\eta (\frac{dv_x}{dy}+\frac{dv_y}{dx})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sigma_{xx}=2\eta \frac{dv_x}{dx}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sigma_{yy}=2\eta \frac{dv_y}{dy}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this I would know how to compute from my results - by finite differences.&#xA;What does the vertical traction mathematically means, what I have to compute?&lt;br/&gt;&#xA;What I about know is, that the traction in vertical (y-axis) could be:&#xA;$$&#xA;\vec{t_y} = \sigma_{xy}\vec{e_x} + \sigma_{yy}\vec{e_y}&#xA;$$&#xA;Am I right?&#xA;Many thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thsi is about traction:&lt;br/&gt;&#xA;&quot;Note a convention that we have implicitly established: the sense of the&#xA;force per unit area $t$ across the oblique surface is that the fluid penetrated&#xA;by the unit normal $n$ acts on the fluid on the other side of the surface. This&#xA;force per unit area is called the traction across the surface. Note further that&#xA;$t_x$ indicates the traction across the y − z plane, not the x component of a&#xA;traction. Representation of components requires a second subscript: $T_{xy}$ is&#xA;the y component of the traction $t_x$ across the y − z face of the tetrahedron,&#xA;whereas $T_{xy}$ is the x component of the traction ty across the x − z face.&#xA;Thus, the first subscript represents the vector component of the force and&#xA;the second subscript indicates the face on which the force is acting.&#xA;&quot;&#xA;But how to compute it?&lt;/p&gt;&#xA;" OwnerUserId="4007" LastEditorUserId="4007" LastEditDate="2013-05-04T10:19:10.967" LastActivityDate="2013-05-04T13:15:46.710" Title="Traction on the boundary - how to compute from velocity field" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7066" PostTypeId="2" ParentId="7064" CreationDate="2013-05-04T13:15:46.710" Score="4" Body="&lt;p&gt;First, the stress in fluid dynamics must also include the pressure. In fact, it is&#xA;$$&#xA;  \sigma = 2 \eta \varepsilon(\mathbf v) + p I&#xA;$$&#xA;where $\varepsilon(\mathbf v)$ is the symmetric gradient of the velocity $\mathbf v$. Then, the traction is the normal component of the stress at the boundary:&#xA;$$&#xA;  \mathbf t = \sigma \mathbf n&#xA;$$&#xA;If you want you can further decompose the traction into the normal force $f = \mathbf n \cdot \mathbf t = \mathbf n \cdot \sigma \mathbf n=2\eta \mathbf n \cdot \varepsilon(\mathbf v) \mathbf n + p$ and the tangential friction force $\mathbf t - f \mathbf n$ for which it is easy to show that it doesn't depend on the pressure any more.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-04T13:15:46.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7067" PostTypeId="1" CreationDate="2013-05-04T13:55:29.450" Score="3" ViewCount="2208" Body="&lt;p&gt;I have download sparselib++, but it seems that it can't be complied in Windows, only in Unix,  I don't know. So can anyone recommend some which can be used in Visual Studio 2010? I want to do a large sparse matrix operation in VS2010. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4217" LastEditorUserId="372" LastEditDate="2013-05-04T15:50:08.273" LastActivityDate="2013-05-06T14:52:23.533" Title="Can anyone recommend a library in C++ which has the most efficiency in doing sparse matrix operations under Windows system" Tags="|linear-algebra|software|libraries|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7068" PostTypeId="1" CreationDate="2013-05-04T23:34:46.250" Score="1" ViewCount="67" Body="&lt;p&gt;Does anyone know of a java library or algorithm for efficiently implementing the second Chebyshev function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be clear, I'm refering to this expression:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vartheta(x) = \sum_{p \le x}\log p$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(x) = \sum_{m=1}^{\infty}\vartheta\left(\sqrt[m]{x}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\vartheta(x)$ is the first Chebyshev function and $\psi(x)$ is the second Chebyshev function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks very much,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Larry&lt;/p&gt;&#xA;" OwnerUserId="4219" LastActivityDate="2013-05-04T23:34:46.250" Title="Looking for a java library or algorithm for efficiently implementing the second Chebyshev function" Tags="|algorithms|libraries|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7069" PostTypeId="2" ParentId="7062" CreationDate="2013-05-05T02:33:35.053" Score="5" Body="&lt;p&gt;The current generation of ARM processors have terrible floating-point performance. IIRC, the &lt;a href=&quot;http://en.wikipedia.org/wiki/ARM_Cortex-A8&quot; rel=&quot;nofollow&quot;&gt;Cortex-A8&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/ARM_Cortex-A9&quot; rel=&quot;nofollow&quot;&gt;Cortex-A9&lt;/a&gt; are nine cycles for a double.  The latest &lt;a href=&quot;http://en.wikipedia.org/wiki/ARM_Cortex-A15&quot; rel=&quot;nofollow&quot;&gt;Cortex-A15&lt;/a&gt; is finally single-cycle double, but is not yet common in the market. (&lt;strong&gt;EDIT:&lt;/strong&gt; See Addendum below)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly, in the embedded world where I belong, a 400MHz PowerPC - theoretically two doubles per cycle - &lt;a href=&quot;http://en.wikipedia.org/wiki/PowerPC_5000&quot; rel=&quot;nofollow&quot;&gt;(Freescale MPC5121e)&lt;/a&gt; outperforms a 800MHz Cortex-A8 &lt;a href=&quot;http://en.wikipedia.org/wiki/I.MX&quot; rel=&quot;nofollow&quot;&gt;(Freescale i.mx535)&lt;/a&gt; by about 50% &lt;em&gt;total time&lt;/em&gt; (including IO) for a numerically intensive application.  The PowerPC outperforms the ARM by a factor of 2.5x for the actual numeric part!  And the ARM has the benefit of 4+ years of kernel and GCC improvements!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A modest Intel Atom N450 1.6GHz netbook outperforms the 400MHz PowerPC by a factor of 4x, both in total time and the numerically expensive core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So no, I cannot recommend ARM cores for numerical applications, at least not for several generations.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was a little wrong on the Cortex-A9.  Whilst it is indeed a nine-cycle double, unlike the Cortex-A8, they can be pipelined and hence get results close-ish to 1DMFLOPS/MHz.  The Cortex-A8 is barely more than 0.1 DMFLOPS/MHz (though single is a different story).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ATLAS benchmarks for the Cortex-A9 can be found &lt;a href=&quot;http://www.vesperix.com/arm/atlas-arm/bench/gcc-a9-3.10.0-nonieee/index.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.  Benchmarks for the Cortex-A8 can be found &lt;a href=&quot;http://www.vesperix.com/arm/atlas-arm/bench/gcc-a8/index.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="761" LastEditDate="2013-08-24T04:04:51.893" LastActivityDate="2013-08-24T04:04:51.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7070" PostTypeId="1" CreationDate="2013-05-05T02:36:03.680" Score="3" ViewCount="356" Body="&lt;p&gt;Does anyone know where to find an analytic form for the singular values of the finite-difference approximation to the 2D Laplacian, expressed in matrix form for a square grid? This would be for the standard 5-point FD approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found the eigenvalues (they are $\sin^2$ functions), but have not seen expressions for the singular values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&#xA;Tom&lt;/p&gt;&#xA;" OwnerUserId="4220" LastActivityDate="2013-05-05T05:33:53.643" Title="Closed form for singular values of 2D Laplacian?" Tags="|finite-difference|condition-number|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7071" PostTypeId="1" AcceptedAnswerId="7072" CreationDate="2013-05-05T03:06:51.300" Score="5" ViewCount="188" Body="&lt;p&gt;The most computationally intensive part of my application is the triangularisation of a matrix, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{S} = \operatorname{triag}\left(\mathbf{A}\right)$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{S}\mathbf{S}^T = \mathbf{A} \mathbf{A}^T$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For this application &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A} \in \Re^{n \times 2(n+v)}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is generally close to the same value as &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Currently, I solve via a QR decomposition, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}^T = \mathbf{Q} \mathbf{R}$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{A} \mathbf{A}^T = (\mathbf{R}^T \mathbf{Q}^T) (\mathbf{Q} \mathbf{R})$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$   = \mathbf{R}^T \mathbf{Q}^T \mathbf{Q} \mathbf{R}       $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$   = \mathbf{R}^T \mathbf{R}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$   = \mathbf{S} \mathbf{S}^T$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which gives the form that I want.&lt;/p&gt;&#xA;&lt;p&gt;Is there a more efficient way to perform this operation?  Even a modest saving over QR will provide a performance boost as this one operation is profiled as 30% of my total computing time.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\operatorname{triag}()$&lt;/span&gt; function is the heart of the &lt;a href=&quot;https://dl.acm.org/doi/10.1016/S0005-1098(00)00089-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Divided Difference Filter&lt;/a&gt;, which is similar to the more commonly known &lt;a href=&quot;https://en.wikipedia.org/wiki/Kalman_filter#Unscented_Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unscented Kalman Filter&lt;/a&gt;.  The problem is mentioned explicitly in the paper that introduces the &lt;a href=&quot;https://ieeexplore.ieee.org/document/4982682&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cubature Kalman Filter&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In short, it calculates four &amp;quot;divided difference&amp;quot; matrices, based on the non-linear process model and state covariance Cholesky factors.  These four matrices are horizontally concatenated, and then triangularised - this forms the predicted state covariance:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \mathbf{S}(k+1|k) = \operatorname{triag}\left(\begin{bmatrix} &#xA;            \mathbf{S}_{xx}^{(2)} &amp;amp; \mathbf{S}_{xv}^{(1)} &amp;amp; \mathbf{S}_{xx}^{(2)} &amp;amp; \mathbf{S}_{xv}^{(2)} \end{bmatrix} \right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{S}_{pq}^n$&lt;/span&gt; above are the &amp;quot;divided difference&amp;quot; matrices calculated above.  The predicted state covariance Cholesy factor &lt;span class=&quot;math-container&quot;&gt;$\mathbf{S}(k+1|k)$&lt;/span&gt; is then directly used to generate &amp;quot;divided difference&amp;quot; matrices of the predicted measurements.&lt;/p&gt;&#xA;&lt;p&gt;A similar process follows of triangularisation of the predicted measurements, but as my measurement vector is small compared to my state vector, this calculation (and the subsequent Kalman Gain/Update) equations are not computationally intensive (about 4% of the process time).&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="22284" LastEditDate="2022-04-22T16:21:17.477" LastActivityDate="2022-04-22T16:21:17.477" Title="Efficient Triangularisation of $\mathbf{S} = \operatorname{triag}\left(\mathbf{A}\right)$ ; $\mathbf{S}\mathbf{S}^T = \mathbf{A} \mathbf{A}^T$" Tags="|linear-algebra|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7072" PostTypeId="2" ParentId="7071" CreationDate="2013-05-05T04:07:37.667" Score="9" Body="&lt;p&gt;If $AA^{T}$ is well conditioned, then you might be able to get away with using the Cholesky factorization of $AA^{T}$ rather than using the QR factorization.  Depending on the size of $A$ this could be vastly faster than computing the QR factorization of $A$.  If you told us something about the size of $A$ we could address that question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if $AA^{T}$ is badly conditioned, then using the Cholesky factorization could be a bad idea because the Cholesky factorization will be less accurate.  It would help to know something about the condition number of $AA^{T}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't told us why you want $S$ and what you do with it after computing this factorization.  It's quite possible that there's a much more efficient way to solve the higher level problem without ever computing this factorization of $AA^{T}$.  However, we can't make any suggestions unless you provide more background on what you're trying to accomplish.   In other words, a good answer to this question is going to depend on a lot of context that you haven't provided yet.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-05-05T04:07:37.667" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7073" PostTypeId="2" ParentId="7063" CreationDate="2013-05-05T04:18:02.367" Score="1" Body="&lt;p&gt;To summarize Christian's comments and some digging through my code, my understanding of GMRES was correct, I had a code issue involving just the QR decomposition so it was a rather odd situation that just happened to converge with more iterations.&lt;/p&gt;&#xA;" OwnerUserId="1295" LastActivityDate="2013-05-05T04:18:02.367" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7074" PostTypeId="2" ParentId="7070" CreationDate="2013-05-05T05:33:53.643" Score="4" Body="&lt;p&gt;The Laplacian is SPD so the singular values are the same as the eigenvalues (and the eigenvectors are orthogonal).  This article explains the eigenvalues and eigenvectors of the discrete Laplacian obtained using centered differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Kronecker_sum_of_discrete_Laplacians&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Kronecker_sum_of_discrete_Laplacians&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-05-05T05:33:53.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7075" PostTypeId="2" ParentId="5600" CreationDate="2013-05-05T12:24:46.863" Score="9" Body="&lt;p&gt;The MUMPS sparse direct solver can handle symmetric indefinite systems and is freely available (&lt;a href=&quot;http://graal.ens-lyon.fr/MUMPS/&quot;&gt;http://graal.ens-lyon.fr/MUMPS/&lt;/a&gt;).&#xA;Ian Duff was one of the authors of both MUMPS and MA57 so the algorithms have many similarities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MUMPS was designed for distributed-memory parallel computers but it also works well on&#xA;single-processor machines. If you link it with a multi-threaded BLAS library you can&#xA;achieve reasonable speedups on shared memory, multicore processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You didn't say how large &quot;very large&quot; is but MUMPS also has an out-of-core capability to&#xA;handle problems where the factored matrix will not fit in the available memory.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-05-05T12:24:46.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7076" PostTypeId="1" AcceptedAnswerId="7080" CreationDate="2013-05-05T13:36:44.530" Score="3" ViewCount="386" Body="&lt;p&gt;I have a ~4.12 Tb structured relatively-sparse matrix dataset (about 8% of the matrix entries are non-zero) that i want to apply an LU decomposition, however, given the size of it, loading it in memory and using the usual BLAS or Eigen routines is out of the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some unix/linux library that i can use to do LU decomposition on a memory-mapped matrix dataset?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found out that R has a package for memory-mapped matrices (&lt;a href=&quot;http://www.bigmemory.org&quot; rel=&quot;nofollow&quot;&gt;http://www.bigmemory.org&lt;/a&gt;) but i'm not expert in R, and i apologize in advance if this can be answered by installing it and trying it out, i wanted to ask before i embarked in learning a new programming environment if this would really solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="372" LastActivityDate="2013-05-05T18:45:23.970" Title="LU Decomposition with memory-mapped matrices" Tags="|linear-algebra|matrix|linear-solver|r|memory-management|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7078" PostTypeId="1" AcceptedAnswerId="7079" CreationDate="2013-05-05T16:56:11.860" Score="4" ViewCount="741" Body="&lt;p&gt;Suppose I have a fixed point iteration of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_{n+1}=f(x_n).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose further that after some initial testing, I found that it does not converge to an a priori known fixed point $x^*$.  I recall from optimization class that often when the standard &lt;em&gt;newton's method&lt;/em&gt; does not converge, one can implement a global convergence strategy to ensure convergence from any initial point (e.g. line search with backtracking).  Does anything similar exist for general fixed point iterations?  Is there strategy that I can implement to ensure that the sequence of iterations does converge?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-05-05T18:39:06.460" Title="Is it possible to ensure global convergence of a fixed point iteration?" Tags="|iterative-method|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7079" PostTypeId="2" ParentId="7078" CreationDate="2013-05-05T18:39:06.460" Score="5" Body="&lt;p&gt;There is nothing that can &lt;em&gt;guarantee&lt;/em&gt; converge of such a fixed point iteration, but there are criteria that if you can show that they hold, the iteration will converge. In particular, the fixed point iteration will converge if it is a &lt;em&gt;contraction&lt;/em&gt;. I am quite sure, however, that it is not very difficult to find example of functions $f(x)$ where even with damping the iteration will not converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than that, if all you care about is finding a fixed point of $f$, then you can of course reformulate the problem. For example, you can find roots of the function $g(x)=x-f(x)$ (e.g., using Newton's method) or minima of $h(x)=\|x-f(x)\|^2$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-05T18:39:06.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7080" PostTypeId="2" ParentId="7076" CreationDate="2013-05-05T18:45:23.970" Score="7" Body="&lt;p&gt;I don't know of any libraries that are still in use today but the keyword in literature searches you will be looking for is &quot;out of memory solver&quot; or &quot;out of core solver&quot; -- linear solvers (and LU decompositions) that work on matrices stored on disk (or tape, at the time) were quite popular in the 1960s, 70s and 80s when memory was expensive and small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, 4TB is even today an incredibly large size. For sparse matrices, LU decompositions are typically much denser, so you have to expect that any LU decomposition would run into the tens or hundreds of TB -- a size where you will likely find that dealing with it is impractical simply because of the time it takes to do I/O. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your better bet is certainly to employ iterative solvers for linear systems. You can get 4TB into memory on 2-3000 cores on a cluster today. That's a significant, but not an infeasible size. Libraries like PETSc and Trilinos will happily do this for you. Depending on the properties of the matrix and the preconditioner you want to use, you may again need a (small) multiple of the memory and then a corresponding multiple of cores.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-05T18:45:23.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7081" PostTypeId="2" ParentId="112" CreationDate="2013-05-05T19:32:17.863" Score="2" Body="&lt;p&gt;For lazy people:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;qalter -l walltime=&amp;lt;new walltime&amp;gt; &amp;lt;jobid&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="102" LastActivityDate="2013-05-05T19:32:17.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7082" PostTypeId="2" ParentId="7067" CreationDate="2013-05-06T06:25:25.927" Score="0" Body="&lt;p&gt;Assuming that you want a C++ library, &lt;a href=&quot;http://eigen.tuxfamily.org&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt; could be a good candidate, depending on your need. It has a great interface and very good performances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used it with VS2010: as it's a template only library, you do not need to install anything, just add the link to the headers in your project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-05-06T06:25:25.927" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7083" PostTypeId="2" ParentId="7067" CreationDate="2013-05-06T07:30:25.883" Score="0" Body="&lt;p&gt;GMRES is one of the best sparse matrix solvers. I don't know what size of matrix you want to solve, but I will suggest &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/cpp_src/mgmres/mgmres.html&quot; rel=&quot;nofollow&quot;&gt;this site&lt;/a&gt; for GMRES (and most of the variations of GMRES). Here you can find its source and header files which you can add to your program and use it. They have given solved examples of all variations on that site. To know more about GMRES, search questions asked related GMRES on this forum. All the best.&lt;/p&gt;&#xA;" OwnerUserId="3549" LastActivityDate="2013-05-06T07:30:25.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7084" PostTypeId="1" CreationDate="2013-05-06T12:52:33.447" Score="2" ViewCount="72" Body="&lt;p&gt;When solving a FV formulation of a set of equations, a code I am currently working with has user defined normalization factors for scaling equations.  It normalizes time, number densities, potential, and fluxes, each with a different factor.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the impression that these are a legacy attempt at a rough diagonal preconditioner from before the code utilized external solvers and preconditioners.  Besides preconditioning matrices, are there any numerical reasons for normalizing unknowns in a FV problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Bort brings up another reason in his answer that I had forgotten.  Formulating the problems in non-dimensional unknowns and non-dimensional parameters can allow simplification if certain physical quantities are small enough to be neglected.  While this is very useful in the formulation of the problem, the code I am working with does not make any decisions based on non-dimensional parameters.  It simply rescales unknowns by arbitrary, user-provided values while working with them.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-05-07T11:31:04.730" LastActivityDate="2013-05-07T11:31:04.730" Title="Effect of Normalization in Unknowns" Tags="|finite-volume|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7085" PostTypeId="2" ParentId="7067" CreationDate="2013-05-06T14:52:23.533" Score="4" Body="&lt;p&gt;You will likely have trouble building any package that uses the Unix/Linux build system on Windows; instead, look for packages that are built using CMake (Cross-platform Make), which is much better about being able to install packages on either platform. SparseLib++ indeed looks like it was only intended for building on Unix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To that end, I'd recommend &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt; or &lt;a href=&quot;http://trilinos.sandia.gov/index.html&quot; rel=&quot;nofollow&quot;&gt;Trilinos&lt;/a&gt;. Jed Brown, one of the developers of PETSc, posts here often in case you're stuck with anything. There's also &lt;a href=&quot;http://www.simunova.com/home&quot; rel=&quot;nofollow&quot;&gt;MTL4&lt;/a&gt;, which is supposed to work well with Windows and build Visual Studio project files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also lots of sparse matrix data structures and routines in SciPy/NumPy if you're willing to interface to Python instead of using pure C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Here's a &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot; rel=&quot;nofollow&quot;&gt;big list&lt;/a&gt; of free linear algebra software, you can see which ones are in C, C++ or Fortran.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-05-06T14:52:23.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7086" PostTypeId="2" ParentId="7084" CreationDate="2013-05-06T16:36:39.747" Score="3" Body="&lt;p&gt;The main reason to normalize equations is the separation of numerics and physics of the problem. Usually, some characteristic scale can be extracted by &lt;a href=&quot;http://en.wikipedia.org/wiki/Dimensional_analysis&quot; rel=&quot;nofollow&quot;&gt;dimensional analysis&lt;/a&gt; of the problem and the equations reformulated dimensionless. This is not an attempt of preconditioning. Common scale factors like Abbe or Reynolds number are derived this way.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2013-05-06T16:36:39.747" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7087" PostTypeId="1" CreationDate="2013-05-06T18:06:04.667" Score="2" ViewCount="240" Body="&lt;p&gt;I recently started to study non-parametric clustering methods and I come across to CRP. After reading all the material I found on the web there is one thing which is not completely clear to me: which is the aim of this process? You can't cluster according to 'distance' (and in fact it was invented another method called Distance Dependent CRP), so why should someone apply this technique? Being a 'noob' I am sure I am missing something... What?&lt;/p&gt;&#xA;" OwnerUserId="4152" LastActivityDate="2013-07-27T08:45:09.640" Title="Chinese Restaurant Process... Why?" Tags="|probability|hpc|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7088" PostTypeId="1" CreationDate="2013-05-06T21:48:43.420" Score="27" ViewCount="2245" Body="&lt;p&gt;I'm about to start working on a &lt;a href=&quot;https://github.com/vtjnash/ODE.jl&quot;&gt;software library of numerical ODE solvers&lt;/a&gt;, and I'm struggling with how to formulate tests for the solver implementations. My ambition is that the library, eventually, will include solvers for both nonstiff and stiff problems, and at least one implicit solver (more or less on par with the capabilities of &lt;a href=&quot;http://www.mathworks.it/it/help/matlab/ordinary-differential-equations.html&quot;&gt;the &lt;code&gt;ode&lt;/code&gt; routines in Matlab&lt;/a&gt;), so the test methodology needs to reflect the various types of problems and criteria for different solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem now is that I don't know where to begin with this testing. I can think of a few different ways to test the output of an algorithm:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Test for a problem that has an analytical solution, and check that the numerical solution is within tolerance levels for all the returned data points. This requires knowledge of a number of analytical problems which exhibit all the properties that I want the different solvers to work with (stiffness, implicit problems etc), which I don't have, at least not off the top of my head.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method tests the &lt;em&gt;results&lt;/em&gt; of a solver method. Thus, there is no guarantee that the solver actually works, just that it works &lt;em&gt;for the given test problem&lt;/em&gt;. Therefore, I suspect a large number of test problems is needed to confidently verify that the solver works.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Manually calculate the solution for a few time steps using the algorithms I intend to implement, and then do the same with the solvers and check that the results are the same. This requires no knowledge of the &lt;em&gt;true&lt;/em&gt; solution to the problem, but in turn requires quite a lot of hands-on work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method, on the other hand, only tests the &lt;em&gt;algorithm&lt;/em&gt;, which is fine by me - if someone else has proven that 4&lt;sup&gt;th&lt;/sup&gt; order Runge-Kutta works, I don't feel a desperate need to. However, I do worry that it will be very cumbersome to formulate test cases, as I don't know a good method to generate the test data (except maybe by hand, which will be a lot of work...).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both the above methods have serious limitations for me with my current knowledge - I don't know a good set of test problems for the first one, and I don't know a good method of generating test data for the second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there other ways to verify numerical ODE solvers? Are there other criteria on the implementations that should be verified? Are there any good (free) resources on testing ODE solvers out there&lt;sup&gt;1&lt;/sup&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;br&gt;&#xA;Since this question is very broad, I want to clarify a little. The test suite I want to create will fill two main purposes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Verifying that the solvers work as expected, for the problems they're intended to solve. In other words, a solver for non-stiff problems is allowed to go bananas on a stiff problem, but should perform well on non-stiff problems. Also, if there are other solvers in the library that offer higher accuracy, it might not be necessary to enforce very accurate results - just &quot;accurate enough&quot;. Thus, part of my question is what tests should be used for what solvers; or, at least, how one should reason to decide that.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sanity test upon installation of the library. These test need not (should not) be elaborate or time-consuming; just the very basics that can be run in under 5 seconds, but that will alert the user if something is off-the-charts weird. Thus, I also need a way to construct tests that are very simple, but that still tell me something about the state of the library.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;&lt;sup&gt;1&lt;/sup&gt; Yes, I've been Googling my eyes out, but most of what I find is lecture notes with very trivial examples, with the notable exception of &lt;a href=&quot;http://www.dm.uniba.it/~testset/testsetivpsolvers/&quot;&gt;the CWI ODE test set from Bari&lt;/a&gt; which I don't know if, or how, I could use for my purposes, since it treats much more sophisticated solvers than the ones I want to test...&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="3297" OwnerDisplayName="Tomas Lycken" LastEditorUserId="3297" LastEditDate="2013-05-07T05:10:02.487" LastActivityDate="2013-05-14T02:10:09.600" Title="How does one test a numerical ODE solver implementation?" Tags="|ode|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7089" PostTypeId="2" ParentId="7088" CreationDate="2013-05-07T01:06:00.103" Score="12" Body="&lt;p&gt;This is a very broad question and I am going to give you some things to think about (some are already included in your post, but they are repeated here for completeness).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scope of Problems&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You need to define the interface of how to specify problems.&lt;/li&gt;&#xA;&lt;li&gt;Are you going to allow parameters that can be fixed or can vary for solutions?&lt;/li&gt;&#xA;&lt;li&gt;Are you going to allow perturbation parameters to slightly perturb problems and see if they are still solvable (for example, an $\epsilon$ parameter to be defined anywhere) in a specific problem?&lt;/li&gt;&#xA;&lt;li&gt;Are you going to allow infinite precision?&lt;/li&gt;&#xA;&lt;li&gt;Are you going to test for speed and sensitivity to numerical precision?&lt;/li&gt;&#xA;&lt;li&gt;Have you chosen two (maybe more) libraries that already exist to compare results?&lt;/li&gt;&#xA;&lt;li&gt;How will you choose stopping criteria, will you use various methods and let the user select or define their own?&lt;/li&gt;&#xA;&lt;li&gt;Are you going to measure error using various measures and allow the user to turn those on and off?&lt;/li&gt;&#xA;&lt;li&gt;Have you looked at the professional packages like Computer-Algebra-Systems (CAS) and understand all of the options they allow?&lt;/li&gt;&#xA;&lt;li&gt;Are you going to allow displaying of results and/or comparisons and/or plots?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem Recommendations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You need to write a test specification defining the source of problems, the scope of how problems were tested, capturing results and metrics of running the routines.&lt;/li&gt;&#xA;&lt;li&gt;I would certainly look to other libraries already out there for the problems they are using (maybe test files).&lt;/li&gt;&#xA;&lt;li&gt;I would go to college libraries and go through books on ODEs and pull out problems of all types, those with known closed form or numeric only solutions. &lt;/li&gt;&#xA;&lt;li&gt;Case 1: We want as many variations of closed form solution problems as we can get in order to compare exact versus numerical results.&lt;/li&gt;&#xA;&lt;li&gt;Case 2: I would go to every numerical analysis book I can find and capture the worked examples and duplicate them. I would additionally capture the problem sets, particularly the ones that have some pathology that exist in most books (sensitivity to this or that types).&lt;/li&gt;&#xA;&lt;li&gt;Case 3: I would go to different branches of applied math like Physics, Ecology, Biology, Economics, et. al and capture problems from each of those domains to validate that your specification language for problems allows for such examples.&lt;/li&gt;&#xA;&lt;li&gt;Case 4: I would research papers/journals that contain the most useful examples where the particular author had to modify a particular approach to account for some pathology or weirdness or hardness.&lt;/li&gt;&#xA;&lt;li&gt;Case 5: Search the web for additional examples. For &lt;a href=&quot;http://en.wikipedia.org/wiki/Stiff_equation&quot;&gt;&lt;em&gt;stiff&lt;/em&gt;&lt;/a&gt;, see the references here and peruse them ALL to ferret out test problems. Here are some &lt;a href=&quot;http://www.mathworks.com/help/matlab/math/ordinary-differential-equations.html#brfg9jq-1&quot;&gt;&lt;em&gt;MATLAB&lt;/em&gt;&lt;/a&gt; examples to peruse.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is not unique. If you look at the book &quot;Numerical Methods for Unconstrained Optimization and Nonlinear Equations&quot; by Dennis and Schnabel, Appendix B, &quot;Test Problems&quot;, you can see how they did it. After developing one of the most beautiful set of algorithms write ups I have ever seen, they threw a collection of problems at it that made if go nuts. You had to tweak here and there! They included five very different and pathological problems that strained the capabilities of the solvers. This has taught me that we can continue to throw problems at algorithms that they are incapable of handling for a host of reasons. Note, they even borrowed this set of problems from More', Garbow and Hillstrom (you can also look up that reference and perhaps there are others you can use as a guide).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, this is not a trivial task. You need Known-Answer-test cases that always allow you to test the validity of updates and don't break things. That is, a repeatable and extensive set of problems from low to high, from easy to hard, from possible to impossible, ... You also need a collection of problems that your solvers cannot handle in order to truly understand its limitations.&lt;/p&gt;&#xA;" OwnerUserId="14350" OwnerDisplayName="Amzoti" LastActivityDate="2013-05-07T02:22:15.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7091" PostTypeId="2" ParentId="3283" CreationDate="2013-05-07T06:57:48.970" Score="2" Body="&lt;p&gt;As Mike Dunlavey already pointed out in a comment, this is often done using so-called &lt;em&gt;zero-crossing functions&lt;/em&gt;, i.e. functions $g(t, x(t)) \in \mathbb{R}$ that cross from $&amp;gt;0$ to $&amp;lt;0$ (or vice versa) when the RHS has a discontinuity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if you have a moving mass with a block then the distance between the mass and the block can be used as a zero-crossing function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many ODE solvers (e.g. SUNDIALS CVODE) automatically check if any of the zero-crossing functions changed its sign during the last time step. If this is the case then a root finding method is used to determine the exact location of the root. The solver can then be restarted at that particular position. This is either done automatically by the solver itself or manually by the calling code.&lt;/p&gt;&#xA;" OwnerUserId="4230" LastActivityDate="2013-05-07T06:57:48.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7093" PostTypeId="1" AcceptedAnswerId="7094" CreationDate="2013-05-07T13:36:43.000" Score="21" ViewCount="16696" Body="&lt;p&gt;While reading some literature on PDE solvers I came across the term &lt;em&gt;pseudo time-stepping&lt;/em&gt; today. It seems to be a common term, however I failed to find a good definition or an introductionary article for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence: &lt;strong&gt;What is pseudo time-stepping, and how is it usually used?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4230" LastActivityDate="2013-05-07T17:22:26.363" Title="What is pseudo time-stepping?" Tags="|pde|time-integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7094" PostTypeId="2" ParentId="7093" CreationDate="2013-05-07T14:33:11.427" Score="21" Body="&lt;p&gt;Pseudo time-stepping, probably better known as pseudo-transient continuation, is the technique of solving for the steady-state solution of time-evolving partial differential equations by setting an initial guess and using a time-stepper to evolve the solution forward.  It tends to succeed where standard globalization strategies fail by taking advantage of the natural structure of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advisor, David E. Keyes, coauthored a few papers discussing pseudo-transient continuation.  I don't think any of them are purely introductory, but his writing style is very clear.  You might try: &lt;a href=&quot;http://www.cs.odu.edu/~keyes/papers/sinum98.pdf&quot;&gt;Convergence Analysis of Pseudo-Transient Continuation&lt;/a&gt; for a more comprehensive description and discussion.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2013-05-07T14:33:11.427" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7095" PostTypeId="2" ParentId="7093" CreationDate="2013-05-07T17:22:26.363" Score="13" Body="&lt;p&gt;Already one good answer is available here, I just want to highlight some things,&#xA;dual time stepping scheme uses pseudo time in addition to real time (so in your equations two time parameters will come one real and one pseudo). Real time act as a new dimension to the equations and generally discretised implicitly (refer &lt;a href=&quot;http://arc.aiaa.org/doi/abs/10.2514/6.1995-1877&quot;&gt;paper&lt;/a&gt;).&#xA;     Real time step size should be such that, it resolves the flow features satisfactorily and, pseudo time step size is depend on the general stability condition. At every real time step, pseudo time iterations are performed to find out converged solution. This method is used in both incompressible and compressible flow calculations. It is also commonly used for the moving mesh problems (refer &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045793010002045&quot;&gt;paper&lt;/a&gt;). All the best.&lt;/p&gt;&#xA;" OwnerUserId="3549" LastActivityDate="2013-05-07T17:22:26.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7097" PostTypeId="1" AcceptedAnswerId="7169" CreationDate="2013-05-07T20:43:33.307" Score="17" ViewCount="1600" Body="&lt;p&gt;What is the theoretical convergence rate for an FFT Poison solver?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving a Poisson equation:&#xA;$$\nabla^2 V_H(x, y, z) = -4\pi n(x, y, z)$$&#xA;with&#xA;$$n(x, y, z) = {3\over\pi} ((x-1)^2 + (y-1)^2 + (z-1)^2 - 1)$$&#xA;on the domain $[0, 2] \times [0, 2] \times [0, 2]$ with periodic boundary condition. This charge density is net neutral. The solution is given by:&#xA;$$&#xA;V_H({\bf x}) = \int {n({\bf y})\over |{\bf x}-{\bf y}|} d^3 y&#xA;$$&#xA;where ${\bf x}=(x, y, z)$. In reciprocal space&#xA;$$&#xA;V_H({\bf G}) = 4\pi {n({\bf G})\over G^2}&#xA;$$&#xA;where ${\bf G}$ are the reciprocal space vectors. I am interested in the Hartree energy:&#xA;$$&#xA;E_H = {1\over 2} \int {n({\bf x}) n({\bf y})\over |{\bf x}-{\bf y}|} d^3 x d^3 y&#xA;= {1\over 2} \int V_H({\bf x}) n({\bf x}) d^3 x&#xA;$$&#xA;In reciprocal space this becomes (after discretization):&#xA;$$&#xA;E_H = 2\pi \sum_{{\bf G}\ne 0} {|n({\bf G})|^2\over G^2}&#xA;$$&#xA;The ${\bf G}=0$ term is omitted, which effectively makes the charge density net neutral (and since it is already neutral, then everything is consistent).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the test problem above, this can be evaluated analytically and one gets:&#xA;$$&#xA;E_H = {128\over 35\pi} = 1.16410...&#xA;$$&#xA;How fast should this energy converge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a program using NumPy that does the calculation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import empty, pi, meshgrid, linspace, sum&#xA;from numpy.fft import fftn, fftfreq&#xA;E_exact = 128/(35*pi)&#xA;print &quot;Hartree Energy (exact):      %.15f&quot; % E_exact&#xA;f = open(&quot;conv.txt&quot;, &quot;w&quot;)&#xA;for N in range(3, 384, 10):&#xA;    print &quot;N =&quot;, N&#xA;    L = 2.&#xA;    x1d = linspace(0, L, N)&#xA;    x, y, z = meshgrid(x1d, x1d, x1d)&#xA;&#xA;    nr = 3 * ((x-1)**2 + (y-1)**2 + (z-1)**2 - 1) / pi&#xA;    ng = fftn(nr) / N**3&#xA;&#xA;    G1d = N * fftfreq(N) * 2*pi/L&#xA;    kx, ky, kz = meshgrid(G1d, G1d, G1d)&#xA;    G2 = kx**2+ky**2+kz**2&#xA;    G2[0, 0, 0] = 1  # omit the G=0 term&#xA;&#xA;    tmp = 2*pi*abs(ng)**2 / G2&#xA;    tmp[0, 0, 0] = 0  # omit the G=0 term&#xA;    E = sum(tmp) * L**3&#xA;    print &quot;Hartree Energy (calculated): %.15f&quot; % E&#xA;    f.write(&quot;%d %.15f\n&quot; % (N, E))&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here is a convergence graph (just plotting the &lt;code&gt;conv.txt&lt;/code&gt; from the above script, &lt;a href=&quot;http://nbviewer.ipython.org/5537804&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; is a notebook that does it if you want to play with this yourself):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rHunK.png&quot; alt=&quot;FFT convergence graph&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the convergence is linear, which was a surprise for me, I thought that FFT converges much faster than that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution has a cusp at the boundary (I didn't realize this before). In order for FFT to converge fast, the solution must have all derivatives smooth. So I also tried the following right hand side:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nr = 3*pi*sin(pi*x)*sin(pi*y)*sin(pi*z)/4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which you can just put into the script above (updated &lt;a href=&quot;https://gist.github.com/certik/5549650&quot; rel=&quot;noreferrer&quot;&gt;script&lt;/a&gt;). The exact solution is $V_H=\sin(\pi x)\sin(\pi y)\sin(\pi z)$, which should be infinitely differentiable. The exact integral in this case is $E_H = \frac{3\pi}{8}$. Yet the FFT solver still converges only linearly towards this exact solution, as can be checked by running the script above and plotting the convergence (updated &lt;a href=&quot;http://nbviewer.ipython.org/5549641&quot; rel=&quot;noreferrer&quot;&gt;notebook&lt;/a&gt; with plots).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know any benchmark in 3D so that I can see faster convergence than linear? &lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="236" LastEditDate="2013-05-09T18:55:34.077" LastActivityDate="2013-05-12T17:28:03.320" Title="Convergence rate of FFT Poisson solver" Tags="|pde|convergence|fourier-analysis|poisson|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7098" PostTypeId="1" AcceptedAnswerId="7841" CreationDate="2013-05-07T20:40:35.643" Score="5" ViewCount="7967" Body="&lt;p&gt;I am currently working on my bachelor thesis, where I am calculating trajectory of electrons and ions in Hall Effect Ion Thruster. However, to be sure what I am calculating, I need to know how does the electromagnetic field look like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have working code for visualization of electromagnetic field using streamslice, however I would like to take it to the next level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created coordinates $x,$ $y$ and $z$ through meshgrid and computated magnetic field, which is set of not really pretty equations. Electric field points only towards the $z$ direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the snappet of code that is generating picture below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Magnetic field&#xA;plot_field = streamslice(x,y,z,Bx,By,Bz,0.02,0.02,0);&#xA;set(plot_field,'Color','black','LineWidth',2);&#xA;&#xA;% Electric field&#xA;hold on;&#xA;quiver3(x_el,y_el,z_el,Ex,Ey,Ez,1,'r');&#xA;hold off;&#xA;&#xA;% Plot of discharge chamber&#xA;[x_ch, y_ch, z_ch] = cylinder(0.4,40);&#xA;z_ch(1,:) = -0.2;&#xA;z_ch(2,:) = 0.2;&#xA;&#xA;hold on;&#xA;plot_ch = surf(x_ch,y_ch,z_ch);&#xA;hold off;&#xA;set(plot_ch,...&#xA;    'LineWidth',1,...&#xA;    'FaceAlpha',0.5,...&#xA;    'EdgeColor','black',...&#xA;    'EdgeAlpha',0.6,...&#xA;    'DiffuseStrength',1,...&#xA;    'AmbientStrength',1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code generates the following,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PQKVW.jpg&quot; alt=&quot;Plot of magnetic field line slices&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to create streamslice of the magnetic field in various degrees slope (let's say $45$), so I could see how does the field looks like not only when intersecting magnetic dipoles, but also in other locations. That would be great for study of particle motion.&lt;/p&gt;&#xA;" OwnerUserId="4237" OwnerDisplayName="bluecore" LastEditorUserId="3691" LastEditDate="2013-06-29T10:38:41.760" LastActivityDate="2013-06-29T10:38:41.760" Title="How to generate sloped streamslice plot of magnetic field in Matlab" Tags="|visualization|electromagnetism|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7101" PostTypeId="2" ParentId="5179" CreationDate="2013-05-08T07:46:17.223" Score="8" Body="&lt;p&gt;Python can be a great starting point. Following resource is a great starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.openbookproject.net/thinkcs/python/english2e/&quot;&gt;http://www.openbookproject.net/thinkcs/python/english2e/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2013-05-08T07:46:17.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7102" PostTypeId="1" CreationDate="2013-05-08T08:59:55.297" Score="13" ViewCount="452" Body="&lt;p&gt;Recently, I've been pushing my group to include more testing when writing their code. There were several major bugs that took a lot longer to catch than probably speaking was necessary, because we didn't have a good testing regime in place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I suspect that having the appropriate tools to automate (or to help streamline) the process, would certainly be useful. On the other hand, I don't know the various options for C++ testing suites, and how to decide between them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there guidelines for what to look for—and are there any that are specifically geared towards numerical applications?&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2017-08-04T09:52:32.480" Title="Testing suites for numerical applications in C++?" Tags="|testing|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7103" PostTypeId="1" AcceptedAnswerId="7120" CreationDate="2013-05-08T13:15:58.977" Score="9" ViewCount="1966" Body="&lt;p&gt;Is there any application for PDEs with nonlinear Dirichlet boundary conditions? That is, I am looking for an example of a partial differential equation for a state $u$ posed on a domain $\Omega$ with $g(u|_{\Gamma})=0$, where $\Gamma$ is the boundary of $\Omega$ and $g\in L^2(\Gamma)\to L^2(\Gamma)$ is a nonlinear function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I am looking for pure Dirichlet conditions. I am aware of the Stefan-Boltzmann boundary conditions for heat conduction, but they are Robin-type boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @Wolfgang Bangerth has pointed out in his answer, there is no sense in assigning boundary values via a nonlinear $g\in L^2(\Gamma)\to L^2(\Gamma)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if the boundary values are fixed via a function $G \in \bigl ( L^2(\Omega) \to L^2(\Gamma) \bigr)$? Something like $u|_\Gamma(x)=g(x)\|u\|$. Has anyone come across such an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am insisting so much on this point, because I was investigating the incorporation of boundary conditions of a PDE by means of multipliers. When, instead of resolving the BCs in the ansatz space, looking at a PDE of the Form&#xA;$ F(u)=0 \text{ in } (H^1(\Omega))',$&#xA;completed by $G(u)=0$, where $G:H^1(\Omega) \to H^{1/2}(\Gamma)$ assigns the Dirichlet conditions, I found it interesting to think of what may happen with a nonlinear $G$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When considering unsteady processes, there can be a unique solution, even if the boundary data $g(u)=0$ allows for multiple values. An example is the Stefan problem formulated for the enthalpy $u$ as considered e.g. by &lt;a href=&quot;http://www.jstor.org/stable/2938664?origin=JSTOR-pdf&quot; rel=&quot;nofollow&quot;&gt;Nochetto et al.&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There, the boundary condition is given in terms of $\theta(u)=0$ on $\Gamma$, where $\theta$ is a monotone Lipshitz-continuous scalar function, with - in particular - $\theta(s)=0$ for $s\in(0,1)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, there is no way to reduce the nonlinear Dirichlet to a linear assignment, as $\theta$ is not injective in the region of interest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 3:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another source of nonlinear boundary conditions are free surface problems, where the boundary itself depends on the solution. See, for example, this &lt;a href=&quot;http://arxiv.org/abs/1009.1418&quot; rel=&quot;nofollow&quot;&gt;ArXiv preprint&lt;/a&gt; of a paper by Sprittles &amp;amp; Shikhmurzaev. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, this is in line with @Wolfgang Bangerth's answer, as the BCs for free surface problems involve spatial derivatives and, thus, are nonlocal. In his book &lt;em&gt;&lt;a href=&quot;http://www.zentralblatt-math.org/zmath/en/search/?q=an:1087.35002&amp;amp;format=complete&quot; rel=&quot;nofollow&quot;&gt;Nonlinear Partial Differential Equations with Applications&lt;/a&gt;&lt;/em&gt; (p. 155), the author Roubicek provides an example for a wetting problem of flow in porous media. Apart from the formulation as a partial differential variational inequality it includes the boundary condition&#xA;$$&#xA;u\frac{\partial u}{\partial n} = 0,&#xA;$$&#xA;on nonpermeable interfaces.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-07-24T13:50:44.760" LastActivityDate="2013-07-24T13:50:44.760" Title="Example of a PDE model with nonlinear Dirichlet boundary conditions" Tags="|pde|boundary-conditions|parabolic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7104" PostTypeId="2" ParentId="5179" CreationDate="2013-05-08T13:27:15.270" Score="5" Body="&lt;p&gt;Fortran: Matlab like, easy to learn and use and quickly get productive but only good for numerical computing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++: Difficult to master (will take you years) but used a lot outside of numerical computing (job security)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python: Recommended a lot these days but too slow for non-trivial work. You'll have write all your underlying computationally expensive kernels in C and then call them from Python which means that you'll have to learn (at least) two languages&lt;/p&gt;&#xA;" OwnerUserId="4245" LastEditorUserId="4245" LastEditDate="2013-05-08T14:01:44.627" LastActivityDate="2013-05-08T14:01:44.627" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7105" PostTypeId="1" CreationDate="2013-05-08T14:21:28.587" Score="5" ViewCount="188" Body="&lt;p&gt;I am given the advection equation&#xA;$$&#xA;u_t=u_x&#xA;$$&#xA;and then the transformation of coordinates&#xA;$$&#xA;x=x(\xi,\theta), \qquad t=\theta&#xA;$$&#xA;which leads us to the transformed equation&#xA;$$&#xA;x_{\xi} u_{\theta} - u_{\xi} x_{\theta} = u_{\xi} &#xA;$$&#xA;Further, i have to derive the central difference approximation for $u_x$ (at point $x_i$). I derived it via Taylor expansions as &#xA;$$&#xA;u_x|_i \approx \frac{u_{i+1} - u_{i-1}}{x_{i+1}- x_{i-1}} = -\frac{1}{x_{i+1}-x_{i-1}} u_{i-1} - \frac{1}{x_{i+1}-x_{i-1}} u_{i+1}&#xA;$$&#xA;Next, i have to work out local truncation error $\tau$ in terms of the transformation derivatives $x_\xi$,&#xA;$x_{\xi \xi}$,$\dots$ and solution derivatives $u_{xx}$, $u_{xxx}$,$\dots$ and write it in the form&#xA;$$&#xA;\tau = E\cdot H^2 + O(H^3)&#xA;$$&#xA;where $H=\Delta \xi$, the constant stepsize in transformed variable $\xi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get &#xA;$$&#xA;\tau = \frac{1}{x_{i+1}- x_{i-1}} \big( -\frac{(\Delta x_{i-1})^2}{2} + \frac{(\Delta x_{i})^2}{2} \big) u_{xx} + \frac{1}{x_{i+1}- x_{i-1}} \big( \frac{(\Delta x_{i-1})^3}{6} + \frac{(\Delta x_{i})^3}{6} \big) u_{xxx}  + \dots&#xA;$$&#xA;where $\Delta x_i = x_{i+1} - x_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But here my problem arises. I don't know where the $x_\xi,x_{\xi \xi},\dots$ are hidden and where do i find $\Delta \xi$. Any help appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4234" LastEditorUserId="4244" LastEditDate="2013-06-08T11:49:50.877" LastActivityDate="2013-10-06T12:35:24.790" Title="Local truncation error and transformation of coordinates" Tags="|pde|numerical-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7106" PostTypeId="2" ParentId="7102" CreationDate="2013-05-08T14:21:31.520" Score="10" Body="&lt;p&gt;I've recently been using &lt;a href=&quot;https://code.google.com/p/googletest/&quot;&gt;googletest&lt;/a&gt; for testing a couple numerical libraries that I work on, and have been very happy with it.  You can write fairly simple tests very quickly or you can write complicated tests that require data initialization and so on. It also provides (like I'm sure many others do) ways to easily do floating point comparisons rather than bitwise.&lt;/p&gt;&#xA;" OwnerUserId="1439" LastActivityDate="2013-05-08T14:21:31.520" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7107" PostTypeId="2" ParentId="5179" CreationDate="2013-05-08T15:12:56.867" Score="25" Body="&lt;p&gt;&lt;strong&gt;Python!&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start off directly using the &lt;a href=&quot;http://www.numpy.org/&quot;&gt;numpy&lt;/a&gt; library, small scripts, and the &lt;a href=&quot;http://ipython.org/&quot;&gt;ipython&lt;/a&gt; interactive shell.&lt;/li&gt;&#xA;&lt;li&gt;Get more advanced with the help of numerous &lt;a href=&quot;http://nbviewer.ipython.org/urls/raw.github.com/jrjohansson/scientific-python-lectures/master/Lecture-0-Scientific-Computing-with-Python.ipynb&quot;&gt;free books and tutorials&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Get more productive using &lt;a href=&quot;http://www.scipy.org/&quot;&gt;scipy&lt;/a&gt; as a frontend to highly performant numerical routines and &lt;a href=&quot;http://matplotlib.org/&quot;&gt;matplotlib&lt;/a&gt; for visualization&lt;/li&gt;&#xA;&lt;li&gt;Take advantage of well developed and powerful modules for scientific computing as &lt;a href=&quot;https://github.com/andrenarchy/krypy&quot;&gt;Krypy&lt;/a&gt;, &lt;a href=&quot;http://fenicsproject.org/&quot;&gt;FeNiCS&lt;/a&gt; and  &lt;a href=&quot;http://wiki.python.org/moin/NumericAndScientific&quot;&gt;lots of others&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Notice, that the smooth transition between flat and object oriented programming and the inherent modularity of Python make larger projects easy to handle.&lt;/li&gt;&#xA;&lt;li&gt;Make your code as fast as C or Fortran by simply rewriting critical parts in &lt;a href=&quot;http://www.cython.org/&quot;&gt;cython&lt;/a&gt;. You can also easily &lt;a href=&quot;http://docs.python.org/2/extending/extending.html&quot;&gt;include routines written in Fortran or C&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This paraphrases what I think is the best way to approach a problem in scientific computing. Start with getting a hand on the problem by playing around with toy examples in small scripts. Become more systematic and set up a suite of code. Then make your code work!!! Finally, if necessary, do code optimization. Don't reinvent the wheel and don't do premature optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Additional plusses: Python comes for free - no license issues, large community e.g. on stackoverflow, modules for good programming as unit testing  or logging ... )&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-11T15:06:23.043" LastActivityDate="2013-05-11T15:06:23.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7108" PostTypeId="2" ParentId="7012" CreationDate="2013-05-08T17:10:57.763" Score="3" Body="&lt;p&gt;The convergence of iterative methods doesn't depend on the sparsity structure of the matrix, but (for Krylov methods) on the distribution of its eigenvectors. One can even construct diagonal (!) matrices for which GMRES converges prohibitively slowly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix had narrow bandwidth, it would be safe to advice some sort of LU-solver. However, those entries 14725 and 44175 indicate a wide range, and LU might have to use a lot of memory due to the fill-in effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, if you're exploring solution methods for your linear system, 5 million unknowns are way to many to play around with. My first advice would be to adapt your problem such that the number of unknowns are around a few thousand (if your problem stem from a PDE, you could increase the mesh size), and then check out spectral properties and see about a solver.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-05-08T20:20:00.560" LastActivityDate="2013-05-08T20:20:00.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7109" PostTypeId="2" ParentId="2760" CreationDate="2013-05-08T17:15:50.547" Score="1" Body="&lt;p&gt;If you can use finite-elements for space discretization and if your models are given explicitly, I suggest you use &lt;a href=&quot;http://fenicsproject.org/&quot; rel=&quot;nofollow&quot;&gt;FeNiCS&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The FEniCS Project is a collection of free software with an extensive&#xA;  list of features for automated, efficient solution of differential&#xA;  equations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(says the website)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know FeNiCS as a powerful and easy to use but clean and very mathematical software suite. In addition, it is free and open source. Maybe there are already implementations of certain models for non-Newtonian flow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reference for finite-elements for Non-Newtonian flows you can find &lt;a href=&quot;http://archiv.ub.uni-heidelberg.de/volltextserver/13173/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-05-08T17:15:50.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7110" PostTypeId="2" ParentId="5600" CreationDate="2013-05-08T17:24:55.317" Score="7" Body="&lt;p&gt;The general problem that direct solvers are suffering from is the fill-in phenomenon, meaning that the inverse of a sparse matrix may be dense. This leads to huge memory requirements if the structure of the matrix is not &amp;quot;suitable&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;There are attempts to work around these issues, and MATLAB's default &lt;code&gt;lu&lt;/code&gt;-function employs a few of them. See &lt;a href=&quot;http://www.mathworks.de/de/help/matlab/ref/lu.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.de/de/help/matlab/ref/lu.html&lt;/a&gt; for an overview of permutations, diagonal scaling and such.&#xA;The same holds true of course for more advanced packages like MUMPS (&lt;a href=&quot;http://graal.ens-lyon.fr/MUMPS/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://graal.ens-lyon.fr/MUMPS/&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;In general, though, if your problem is large enough, you will hit that memory boundary, and you'll have to use iterative (Krylov) methods.&lt;/p&gt;&#xA;&lt;p&gt;If your problem is symmetric and indefinite, MINRES may be the obvious choice. Note however that it may suffer from loss of orthogonality in the underlying Arnoldi process. GMRES can help here. In exact arithmetic, it does the same thing as MINRES, but tends to cope with orthogonality issues better. This is at the expense of higher memory requirements than MINRS. Some people regard GMRES as &amp;quot;the best implementation of MINRES&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In any case, you'll probably profit from preconditioning your system. If you don't want to spend time on tailoring a preconditioner, incomplete LU-factorization preconditioner (ILU) might already get you somewhere.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2020-09-03T16:34:56.950" LastActivityDate="2020-09-03T16:34:56.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7111" PostTypeId="1" AcceptedAnswerId="7118" CreationDate="2013-05-08T17:25:56.490" Score="3" ViewCount="1269" Body="&lt;p&gt;I'd like to loop over all edges of a simplex (in a mesh). To this end: How can I extract the the points from a cell?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is for the &lt;code&gt;eval&lt;/code&gt; method of a C++ definition of an &lt;code&gt;Expression&lt;/code&gt;, i.e.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void eval(Array&amp;lt;double&amp;gt;&amp;amp; v,                                                     &#xA;          const Array&amp;lt;double&amp;gt;&amp;amp; x,                                               &#xA;          const ufc::cell&amp;amp; c                                                    &#xA;         ) const&#xA;{&#xA;Cell cell(*mesh, c.index);                                                  &#xA;// do something with cell?&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-05-08T18:10:01.200" LastActivityDate="2013-05-08T21:30:34.040" Title="FEniCS: extracting points from a cell" Tags="|c++|fenics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7112" PostTypeId="2" ParentId="7111" CreationDate="2013-05-08T17:42:23.247" Score="3" Body="&lt;p&gt;This should do. Here, I assume that &lt;code&gt;mesh&lt;/code&gt; is your mesh, &lt;code&gt;Q&lt;/code&gt; is the function space and that you use the dolfin interface in python. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i, cell) in enumerate(cells(mesh)):&#xA;    print &quot;Global dofs associated with cell %d: &quot; % i&#xA;    print Q.dofmap().cell_dofs(i)&#xA;    print &quot;The Dof coordinates:&quot;&#xA;    print Q.dofmap().tabulate_coordinates(cell)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-05-08T17:42:23.247" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7113" PostTypeId="1" AcceptedAnswerId="7114" CreationDate="2013-05-08T18:50:48.293" Score="4" ViewCount="1760" Body="&lt;p&gt;I am about to start my journey into the world of CFD and wanted to start with the Fractional-step method for solving the incompressible Navier-Stokes equations. Could you perhaps suggest some articles or books I should look into (for this method specifically)?&lt;/p&gt;&#xA;" OwnerUserId="4249" LastEditorUserId="4249" LastEditDate="2013-06-06T12:58:56.550" LastActivityDate="2013-11-16T17:54:03.707" Title="Fractional-step method" Tags="|incompressible|navier-stokes|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7114" PostTypeId="2" ParentId="7113" CreationDate="2013-05-08T19:14:01.997" Score="6" Body="&lt;p&gt;Check out&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An overview of projection methods for incompressible flows;&lt;br&gt;&#xA;   Guermond, Minev, Shen;&lt;br&gt;&#xA;   Comput. Methods Appl. Mech. Engrg., 195 (2006);&lt;br&gt;&#xA;   &lt;a href=&quot;http://www.math.ust.hk/~mawang/teaching/math532/guermond-shen-2006.pdf&quot;&gt;http://www.math.ust.hk/~mawang/teaching/math532/guermond-shen-2006.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It gives a fairly good overview over a bunch of solver classes, amongst others the fractional-step methods such as pressure-correction (e.g., the Chorin's classical method). It also talks a bit about the difficulty in chosing boundary conditions (especially for the pressure).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more detailed overview, I found the book&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite Element Methods for Flow Problems;&#xA;   Jean Donea, Antonio Huerta;&#xA;&lt;a href=&quot;http://books.google.de/books/about/Finite_Element_Methods_for_Flow_Problems.html?id=S4URqrTtSXoC&amp;amp;redir_esc=y&quot;&gt;http://books.google.de/books/about/Finite_Element_Methods_for_Flow_Problems.html?id=S4URqrTtSXoC&amp;amp;redir_esc=y&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;quite helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the practical implementation, I'd recommend looking into FEniCS. The FEniCS book, &lt;a href=&quot;https://bitbucket.org/fenics-project/fenics-book/downloads/fenics-book-2011-10-27-final.pdf&quot;&gt;https://bitbucket.org/fenics-project/fenics-book/downloads/fenics-book-2011-10-27-final.pdf&lt;/a&gt;, also has a chapter (III-21) devoted to comparing all sorts of different methods for incompressible Navier-Stokes.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-05-08T19:20:54.397" LastActivityDate="2013-05-08T19:20:54.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7115" PostTypeId="5" CreationDate="2013-05-08T19:45:18.523" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://fenicsproject.org/&quot; rel=&quot;nofollow&quot;&gt;FEniCS&lt;/a&gt; is a collection of open source software for the numerical solution of partial differential equations using finite elements. It provides interfaces in C++ and Python (via the package DOLFIN).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions tagged &lt;a href=&quot;/questions/tagged/fenics&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;fenics&amp;#39;&quot; rel=&quot;tag&quot;&gt;fenics&lt;/a&gt; should be concerned with the &lt;strong&gt;use of FEniCS for solving specific problems in computational science&lt;/strong&gt; (e.g., how to implement a specific partial differential equation or boundary condition, or how to use FEniCS in optimization routines).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions about &lt;strong&gt;problems when using FEniCS&lt;/strong&gt; (e.g., error messages when compiling/linking/running C++/Python code, explanation of Form or Expression syntax, clarifications on interfaces) &lt;strong&gt;belong on the &lt;a href=&quot;http://fenicsproject.org/qa&quot; rel=&quot;nofollow&quot;&gt;FEniCS Q&amp;amp;A site&lt;/a&gt;&lt;/strong&gt;. Until May 2013, FEniCS related issues were discussed on &lt;a href=&quot;https://answers.launchpad.net/fenics-project&quot; rel=&quot;nofollow&quot;&gt;launchpad&lt;/a&gt;, and you may also find answers there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before posting a question, please make sure to &lt;strong&gt;check the &lt;a href=&quot;http://fenicsproject.org/documentation/&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt;&lt;/strong&gt; to see if it already addresses your issue. In particular, you might want to consult (in order of thoroughness)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the documented demos in&#xA;&lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/1.4.0/python/demo/index.html&quot; rel=&quot;nofollow&quot;&gt;Python&lt;/a&gt;,&#xA;&lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/1.4.0/cpp/demo/index.html&quot; rel=&quot;nofollow&quot;&gt;C++&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;the &lt;a href=&quot;http://fenicsproject.org/documentation/tutorial/index.html&quot; rel=&quot;nofollow&quot;&gt;tutorial&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;the &lt;a href=&quot;http://launchpad.net/fenics-book/trunk/final/+download/fenics-manual-2011-10-31.pdf&quot; rel=&quot;nofollow&quot;&gt;manual&lt;/a&gt; (in particular Chapter 4.4, which covers the UFL syntax for forms and expressions),&lt;/li&gt;&#xA;&lt;li&gt;the &lt;a href=&quot;http://launchpad.net/fenics-book/trunk/final/+download/fenics-book-2011-10-27-final.pdf&quot; rel=&quot;nofollow&quot;&gt;FEniCS book&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(Note that most of these (except demos) were written for version 1.0.0 -- current as of June 2014 is 1.4.0).&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="4254" LastEditDate="2014-06-25T14:55:44.890" LastActivityDate="2014-06-25T14:55:44.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7116" PostTypeId="4" CreationDate="2013-05-08T19:45:18.523" Score="0" Body="Questions about the finite element method software FEniCS and its subpackages (Dolfin, UFL, FFC)." OwnerUserId="3980" LastEditorUserId="1804" LastEditDate="2013-06-26T14:14:19.300" LastActivityDate="2013-06-26T14:14:19.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7117" PostTypeId="2" ParentId="7102" CreationDate="2013-05-08T20:05:31.130" Score="4" Body="&lt;p&gt;If you're building your code with CMake, then the &lt;a href=&quot;http://cmake.org/Wiki/CMake/Testing_With_CTest&quot; rel=&quot;nofollow&quot;&gt;ctest&lt;/a&gt; mechanism would be the obvious choice. It allows you to test your code manually via the command &lt;code&gt;ctest&lt;/code&gt;, and also supports extensive nightly testing via &lt;a href=&quot;http://www.cdash.org/&quot; rel=&quot;nofollow&quot;&gt;CDash&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-05-08T20:05:31.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7118" PostTypeId="2" ParentId="7111" CreationDate="2013-05-08T20:20:53.280" Score="5" Body="&lt;p&gt;You can always use an iterator to access the entities connected to a cell, like the edges in your case. Just as you suggest, you first need to create a dolfin::Cell from the ufc::cell. Then do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void eval(Array&amp;lt;double&amp;gt;&amp;amp; v,                                                     &#xA;          const Array&amp;lt;double&amp;gt;&amp;amp; x,                                               &#xA;          const ufc::cell&amp;amp; c) const&#xA;{&#xA;  Cell cell(*mesh, c.index);                                                  &#xA;  for (EdgeIterator edge(cell); !edge.end(); ++edge)&#xA;  {&#xA;    // Do something with edge&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For speed, and maybe easier access, you could directly access the edges (entities of dimension 1 as follows):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const unsigned int* edge_indices = cell.entities(1);&#xA;Edge e0(mesh, edge_indices[0]);&#xA;Edge e1(mesh, edge_indices[1]);&#xA;Edge e2(mesh, edge_indices[2]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In a similar way, you can access the vertices of the edges (by calling &lt;code&gt;cell.entities(0)&lt;/code&gt; or &lt;code&gt;ei.entities(0)&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that for this to work, you need to instantiate the edges of the mesh (they will be created automatically by the iterators). You can do this by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh.init(1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="359" LastEditorUserId="3980" LastEditDate="2013-05-08T21:30:34.040" LastActivityDate="2013-05-08T21:30:34.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7119" PostTypeId="2" ParentId="5534" CreationDate="2013-05-08T20:58:46.460" Score="5" Body="&lt;p&gt;This is very similar to one of the biggest challenges in the field of molecular dynamics—computing all of the pairwise interactions between nonbonded particles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There, we use &lt;em&gt;cell lists&lt;/em&gt; (or &lt;em&gt;neighbor lists&lt;/em&gt;) to help us figure out what's nearby; for this application, the cell list is probably the easier algorithm to use:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Divide the box into a series of cells.&lt;/li&gt;&#xA;&lt;li&gt;For each particle, determine to which cell it should be assigned (O(1) per particle).&lt;/li&gt;&#xA;&lt;li&gt;Then, for each particle, check the &quot;own&quot; cell plus the neighbor cells; if any of these are occupied, then no further search is necessary. &lt;/li&gt;&#xA;&lt;li&gt;If all nearest neighbors are empty, then expand to next-nearest neighbors, and so on, until a particle is found.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If your system has a more or less uniform distribution of particles, this will greatly reduce the cost of your algorithm, according to the coarseness of the grid. However, some fine tuning is necessary: too coarse a grid and you won't save much time; too fine, and you'll spend a lot of time cycling over empty grid cells!&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2013-05-08T20:58:46.460" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7120" PostTypeId="2" ParentId="7103" CreationDate="2013-05-08T23:14:33.563" Score="9" Body="&lt;p&gt;This makes no sense. Let us assume that $g(z)=0$ has exactly one solution $z=z^\ast$, then your boundary condition $g(u|_\Gamma)=0$ is equivalent to $u|_\Gamma=z^\ast$, i.e., a linear Dirichlet condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if there are multiple solutions of $g(z)=0$, then you are saying that the value $u|_\Gamma$ could have multiple values, but this does not lead to a unique solution of the equation and so it makes no sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a similar vein, it makes no sense to have nonlinear Neumann boundary conditions. The only thing that does make sense is to have nonlinear Robin-type conditions, $(\partial u/\partial n+g(u))|_\Gamma = 0$. A typical case is the Planck radiation condition $(D\partial u/\partial n+\gamma u^4)|_\Gamma = 0$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-08T23:14:33.563" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7121" PostTypeId="2" ParentId="7102" CreationDate="2013-05-08T23:18:35.580" Score="11" Body="&lt;p&gt;The problem with testing numerical codes is that (i) you may not always know the exact output and you will only be able to save the result of a computation now to compare against later -- i.e., to do regression tests, and (ii) that results may differ by small amounts on different machines due to different round-off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see how deal.II does it, take a look here:&#xA;  &lt;a href=&quot;http://www.dealii.org/developer/development/testsuite.html#regression_tests&quot;&gt;http://www.dealii.org/developer/development/testsuite.html#regression_tests&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-08T23:18:35.580" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7122" PostTypeId="1" AcceptedAnswerId="7131" CreationDate="2013-05-09T02:30:24.273" Score="4" ViewCount="1357" Body="&lt;p&gt;I am using the Chebyshev spectral differentiation technique that is described concisely under &quot;details&quot; &lt;a href=&quot;http://demonstrations.wolfram.com/ChebyshevSpectralDifferentiationViaFastFourierTransform/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.  The idea is to take the initial data $v_0,v_1\,...,v_N$ and store it in union with itself as the vector $$V = [v_0\,v_1\,...\,v_{N-1}\,v_N\,v_{N-1}\,v_{N-2}\,...\,v_1]^\top$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From there, the Fourier transform of this vector $V$ is taken.  However, for the Fourier transform to provide a good interpolation of the data in $V$, $V$ should be smooth and periodic.  Although $V$ is continous and periodic, there is (generally) a discontinuity in its first derivative (around the entries $v_{N-1},v_N,v_{N-1}$).  Why, then, is this method of differentiation still so effective?&lt;/p&gt;&#xA;" OwnerUserId="3742" LastActivityDate="2013-05-09T13:12:47.807" Title="Chebyshev spectral differentiation via FFT" Tags="|fourier-analysis|spectral-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7123" PostTypeId="1" AcceptedAnswerId="7126" CreationDate="2013-05-09T06:18:10.357" Score="7" ViewCount="4444" Body="&lt;p&gt;Given a vector-valued PDE, I'd like to enforce the boundary conditions&#xA;$$&#xA;\vec{n}\cdot u = g\\&#xA;\vec{n}\cdot \nabla (\vec{t}\cdot u) = 0&#xA;$$&#xA;on the solution $\vec{u}$. If the boundary happens to align with one of the coordinate axes, I could use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcs = DirichletBC(V.sub(0), g, 'on_boundary')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as Dirichlet condition. What to do for other geometries though?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2020-11-03T05:46:25.217" Title="FEniCS: separate boundary conditions in normal and tangential direction of mesh boundary" Tags="|boundary-conditions|navier-stokes|fenics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7124" PostTypeId="2" ParentId="7105" CreationDate="2013-05-09T07:13:12.030" Score="1" Body="&lt;p&gt;I have it. I just expand grid function around $x_i$:&#xA;$$&#xA;x_{i+1}(\xi,\theta) = x_i + \Delta \xi x_\xi + \frac{(\Delta \xi)^2}{2}x_{\xi \xi} + \dots &#xA;$$&#xA;and&#xA;$$&#xA;x_{i-1}(\xi,\theta) = x_i - \Delta \xi x_\xi + \frac{(\Delta \xi)^2}{2}x_{\xi \xi} + \dots &#xA;$$&#xA;Then i use&#xA;$$&#xA;(\Delta x_i)^2 - (\Delta x_{i-1})^2 = (\Delta x_{i} - \Delta x_{i-1})(\Delta x_{i} + \Delta x_{i-1})&#xA;$$ &#xA;and &#xA;$$&#xA;(\Delta x_i)^3 + (\Delta x_{i-1})^3 = (\Delta x_{i} + \Delta x_{i-1})((\Delta x_{i})^2 - \Delta x_i \Delta x_{i-1} + (\Delta x_{i-1})^2)&#xA;$$&#xA;and &#xA;$$&#xA;\Delta x_{i} + \Delta x_{i-1} = x_{i+1}-x_{i-1}&#xA;$$&#xA;After calculation you get to the solution.&lt;/p&gt;&#xA;" OwnerUserId="4234" LastActivityDate="2013-05-09T07:13:12.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7125" PostTypeId="1" AcceptedAnswerId="7129" CreationDate="2013-05-09T08:30:00.290" Score="4" ViewCount="1386" Body="&lt;p&gt;I am planning to model the laminar flow reactor shown in the picture below using computational fluid dynamics (CFD). The laminar flow reactor is used to study a multiphase flow: a layer of sheath air that cushions a stream of aerosols.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/kUUTl.png&quot; alt=&quot;laminar flow reactor&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am in the process of narrowing down the software to be used. In my (limited) understanding of CFD, both ANSYS Fluent and Comsol are valid commercial options for modelling this system. However, both come with the troubles of closed source code and licensing cost. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As open source alternatives, I am considering &lt;a href=&quot;http://www.openfoam.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;openFOAM&lt;/a&gt; and &lt;a href=&quot;http://fenicsproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEnics&lt;/a&gt;. If I understand correctly, both have most of what I need (basic solvers, multiphase flow, particle tracking). However, do they have features for &lt;strong&gt;tracking the dynamics of particle size distributions&lt;/strong&gt;? This feature is of importance for modelling aerosols.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, has anybody done modelling of a laminar flow reactor in either openFOAM or FEnics in general? Googeling this topic did not find me any useful information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thankful for any advise. At this point, my CFD skills are limited to single phase incompressible laminar flow. I am planning learn more through this project.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="4244" LastEditDate="2013-05-09T14:14:34.053" LastActivityDate="2013-05-13T19:47:53.440" Title="CFD: multiphase flow modeling of a laminar flow reactor" Tags="|fluid-dynamics|fenics|laminar-flow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7126" PostTypeId="2" ParentId="7123" CreationDate="2013-05-09T08:46:51.027" Score="8" Body="&lt;p&gt;You can use a Nitsche-type method for this. See the following reference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;J. Freund, R. Stenberg. On weakly imposed boundary conditions for second order problems. Proceedings of the Ninth Int. Conf. Finite Elements in Fluids, Venice 1995. M. Morandi Cecchi et al., Eds. pp. 327-336.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have implemented this a while ago in some simple FEniCS-Code to deal with free-slip boundaries in general 2d geometries. Find the demo code &lt;a href=&quot;https://pastebin.com/3zPiJd9m&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and the mesh &lt;a href=&quot;https://pastebin.com/NnePcWV4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (tested with FEniCS 1.3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/5GA88.png&quot; alt=&quot;Cylinder bump: free-slip boundary conditions via Nitsche&amp;#39;s method&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: Meanwhile, others have made similar implementations available. See for instance this &lt;a href=&quot;https://github.com/MiroK/fenics-nitsche&quot; rel=&quot;nofollow noreferrer&quot;&gt;github repository&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2018-04-11T03:18:06.777" LastActivityDate="2018-04-11T03:18:06.777" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7128" PostTypeId="1" AcceptedAnswerId="7130" CreationDate="2013-05-09T10:47:36.403" Score="11" ViewCount="5340" Body="&lt;p&gt;I'm applying finite-difference method to a system of 3 coupled equations. Two of the equations are not coupled, however the third equation couples to both the other two. I noticed that by changing the order of equations, say from $(x, y, z)$ to $(x, z, y)$ that the coefficient matrix becomes symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any advantage to doing this? For example, in terms of stability or efficiency/speed of solution. The matrices are highly sparse, if that is important, the non-zero terms are along central diagonals.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2013-05-09T12:31:23.490" Title="Are there any numerical advantages in solving symmetric matrix compared to matrices without symmetry?" Tags="|finite-difference|symmetry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7129" PostTypeId="2" ParentId="7125" CreationDate="2013-05-09T11:55:42.407" Score="4" Body="&lt;p&gt;I don't know about these things in FEniCS. For openFOAM there are libraries for multiphase flow. The implementation of population balance equations to track particle size distributions in openFOAM has been done and discussed in this &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0098135408000525&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question 'openFOAM or FEniCS' can maybe decided by deciding on whether you want to use finite volumes or finite elements for your discretizations. If your major concern are robust and fast simulations, I suggest you stick to finite volumes and openFOAM. If you have more time and if you want to rely on rather clean and consistent than stable numerical schemes, finite elements and FEniCS might be the better choice.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-05-09T11:55:42.407" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7130" PostTypeId="2" ParentId="7128" CreationDate="2013-05-09T12:25:54.077" Score="12" Body="&lt;p&gt;Absolutely!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First off, some linear algebra systems are smart enough to only store half of the matrix, this could save you a bunch of memory. But even if this isn't the case, various algorithms in numerical linear algebra will exploit the symmetry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given a symmetric matrix, any eigensolver will immediately know that all eigenvalues are real-valued, and the solution method may use that fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical thing that many people will think of are Krylov subspace methods for the solution of equation systems $Ax=b$: If your problem is symmetric, you know that you don't need methods for nonsymmetric problem like GMRES, and can reside to something less memory-intensive like MINRES, or -- if your matrix is also positive-definite -- CG. The convergence behavior of Krylov methods is not influenced by permutations though, so you could even use symmetric methods for your unpermuted system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example is the factorization of your matrix $A=LU$ in to an lower-triangular part $L$ and an upper triangular part $U$. If $A$ is symmetric, then $A=LL^T$, and you only have to store one factor (&lt;a href=&quot;https://en.wikipedia.org/wiki/Cholesky_decomposition&quot;&gt;Cholesky decomposition&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-05-09T12:31:23.490" LastActivityDate="2013-05-09T12:31:23.490" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7131" PostTypeId="2" ParentId="7122" CreationDate="2013-05-09T13:12:47.807" Score="4" Body="&lt;p&gt;Ah, I have realized the answer to my own question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to recognize that the initial data $v_0,...,v_N$ is not stored on a uniform grid, but rather at the Chebyshev points&#xA;$$ x_j = \cos\frac{\pi j}{N},\qquad j=0,...,N.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now as long as the initial data has a decent polynomial interpolation, then&#xA;\begin{align}&#xA;v_j = p(x_j) &amp;amp;= a_0 + a_1x_j + \cdots + a_Nx_j^N \\&#xA;&amp;amp;=a_0 + a_1\cos\frac{\pi j}{N} + \cdots + a_N\cos^N\frac{\pi j}{N} \\&#xA;&amp;amp;=a_0 + a_1\cos\theta_j + \cdots + a_N\cos^N\theta_j = f(\theta_j)&#xA;\end{align}&#xA;where $\theta_j = \pi j/N\in[0,\pi]$ is a &lt;em&gt;uniform&lt;/em&gt; grid.  Therefore, on the new uniform grid the data is an even function (hence the powers of cosine), and in particular $df/d\theta|_{\theta=0} = 0$.  Thus the function can easily be extended to $[-\pi,\pi]$, giving a smooth, even, periodic function with data at uniformly-spaced gridpoints: ripe for the Fourier transform.&lt;/p&gt;&#xA;" OwnerUserId="3742" LastActivityDate="2013-05-09T13:12:47.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7132" PostTypeId="2" ParentId="6734" CreationDate="2013-05-09T14:06:45.000" Score="5" Body="&lt;p&gt;To ask for a progression year-on-year or even more fine-grained is asking for a bit too much detail, I think. Available information is more coarse-grained. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0960982210015289&quot; rel=&quot;noreferrer&quot;&gt;This paper&lt;/a&gt; reports a nine orders of magnitude progress in simulated timescales over three decades of molecular dynamics research on proteins in water. &lt;/p&gt;&#xA;" OwnerUserId="4265" LastActivityDate="2013-05-09T14:06:45.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7133" PostTypeId="2" ParentId="1603" CreationDate="2013-05-09T17:55:38.333" Score="2" Body="&lt;p&gt;Bonmin is an open-source MINLP solver that uses IPOPT to solve the &quot;relaxed&quot; NLPs solutions. Bonmin has the following algorithms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B-BB: NLP-based branch-and-bound algorithm&#xA;B-OA: outer-approximation decomposition algorithm&#xA;B-QG: implementation of Quesada and Grossmann's branch-and-cut algorithm&#xA;B-Hyb: hybrid outer-approximation based branch-and-cut algorithm&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option for MINLP problems is the &lt;a href=&quot;http://apopt.com&quot; rel=&quot;nofollow&quot;&gt;APOPT solver&lt;/a&gt; which I'm currently working on developing. It has an interface to AMPL which it seems like you are currently using.&lt;/p&gt;&#xA;" OwnerUserId="906" LastEditorUserId="906" LastEditDate="2013-05-10T15:46:51.600" LastActivityDate="2013-05-10T15:46:51.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7134" PostTypeId="2" ParentId="2971" CreationDate="2013-05-09T18:11:36.997" Score="0" Body="&lt;p&gt;The &quot;discretize, then optimize&quot; approach that Geoff mentioned is part of the &lt;a href=&quot;http://apmonitor.com&quot; rel=&quot;nofollow&quot;&gt;APM MATLAB or APM Python&lt;/a&gt; packages that solve mixed-integer differential algebraic equation systems. We're developing this software in our research group at Brigham Young University. Below is a link to a presentation that we published recently at the 2012 INFORMS conference that gives more details and a couple example applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hedengren, J.D., Mojica, J.L., Cole, W., Edgar, T.F., &lt;a href=&quot;http://apm.byu.edu/prism/uploads/Members/minlp_apopt_informs2012.pdf&quot; rel=&quot;nofollow&quot;&gt;APOPT: MINLP Solver for Differential Algebraic Systems with Benchmark Testing&lt;/a&gt;, INFORMS National Meeting, Phoenix, AZ, Oct 2012.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of tutorial videos on the APMonitorCom YouTube channel if you are interested in numerical solution to these types of problems.&lt;/p&gt;&#xA;" OwnerUserId="906" LastEditorUserId="906" LastEditDate="2013-05-10T15:45:26.280" LastActivityDate="2013-05-10T15:45:26.280" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7135" PostTypeId="1" AcceptedAnswerId="7138" CreationDate="2013-05-09T18:18:21.130" Score="2" ViewCount="619" Body="&lt;p&gt;Given a Dolfin &lt;code&gt;Function&lt;/code&gt; &lt;code&gt;u&lt;/code&gt; on a mesh, I'd like to check if it is greater than 0 throughout the mesh. I noticed that the expression&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u &amp;gt; 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is an object of the type&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GT(Coefficient(FiniteElement('Lagrange', Domain(Cell('triangle', 2), 'triangle_multiverse', 2, 2), 2, None), 0), Zero((), (), {}))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I evaluate that thing?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-05-09T19:48:43.453" Title="FEniCS: check if function is globally greater than 0" Tags="|fenics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7136" PostTypeId="2" ParentId="541" CreationDate="2013-05-09T18:24:15.837" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/APMonitor&quot; rel=&quot;nofollow&quot;&gt;APM MATLAB and APM Python&lt;/a&gt; packages can solve large-scale (100,000+ variables) of Mixed-Integer Differential Algebraic equation systems. The software is available as a web service for commercial or academic use. If you are solving a PDE system, you could discretize once to get it into DAE or ODE form to put it into APMonitor modeling language. The modeling language uses &lt;a href=&quot;http://apopt.com&quot; rel=&quot;nofollow&quot;&gt;APOPT&lt;/a&gt;, BPOPT, IPOPT, SNOPT, and MINOS solvers.&lt;/p&gt;&#xA;" OwnerUserId="906" LastActivityDate="2013-05-09T18:24:15.837" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7137" PostTypeId="2" ParentId="7135" CreationDate="2013-05-09T19:38:22.320" Score="5" Body="&lt;p&gt;Looks like you're using the Python interface, in which case &lt;code&gt;u&lt;/code&gt; is both a DOLFIN Function object and a UFL object (it inherits from both). The type information you're seeing is related to UFL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can evaluate the function &lt;code&gt;u&lt;/code&gt; at an arbitrary point:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = u((0.5, 0.5, 0.5))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if DOLFIN is configured with CGAL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how you could check that the solution is strictly positive everywhere for arbitrary element type, but you could do a pretty good check integrating a UFL conditional,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c = conditional(lt(u, 0), 1, 0)&#xA;&#xA;# This should be zero if u &amp;gt; 0&#xA;print assemble(c*dx)   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Conditional UFL statements are presented in &lt;a href=&quot;http://arxiv.org/abs/1211.4047&quot;&gt;http://arxiv.org/abs/1211.4047&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3952" LastEditorUserId="3952" LastEditDate="2013-05-09T19:47:42.343" LastActivityDate="2013-05-09T19:47:42.343" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7138" PostTypeId="2" ParentId="7135" CreationDate="2013-05-09T19:48:43.453" Score="3" Body="&lt;p&gt;If the function is piecewise linear, an easy check would be the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print u.vector().min()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This would also work as a good approximation for other spaces.&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-05-09T19:48:43.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7139" PostTypeId="1" CreationDate="2013-05-09T20:39:24.820" Score="4" ViewCount="1972" Body="&lt;p&gt;I am trying to get an interpolation function $f$ (in 3D) at all vertices of cells. I extract all vertices of cells and then I assign the value to each vertex: if it's in a sphere of radius $R$, then I assign the value, say 3.91. If it's outside the sphere, then I assign the value 0. I got it running without the error message, but then when I calculated the function value at points that are not vertices, it does not give me either 3.91 or 0. Am I doing something wrong here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a part of my code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extract vertices of all cells, then I export these points and use another software to       assign value for each point (say 3.91 for points inside sphere, 0 for points outside)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coor = mesh.coordinates()&#xA;numpy.savetxt('meshforE.txt',coor)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the values at all vertices and then interpolate this to function f&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;qvalues2 = numpy.loadtxt('qdata.txt')&#xA;V = FunctionSpace(mesh, &quot;CG&quot;, 1)&#xA;f = Function(V)&#xA;f.vector()[:] = qvalues2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then I read points &lt;code&gt;(xp, yp)&lt;/code&gt; on $z=0$ plane and evaluate the funtion at these points&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with open('xpdata.txt') as g:&#xA;    xp = g.readlines()&#xA;print &quot;xp[1]=&quot;, xp[1]&#xA;with open('ypdata.txt') as h:&#xA;    yp = h.readlines()&#xA;print &quot;yp[2]=&quot;, yp[2]&#xA;&#xA;for i in range(len(xp)):&#xA;    g_in[i] = f(xp[i],yp[i],0.0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now when I plot $g_i$ , it does not look like what it should be, i.e. constant (3.91) inside the circle $R=50$, and 0 outside.&#xA;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4268" LastEditorUserId="1748" LastEditDate="2013-05-10T17:01:51.677" LastActivityDate="2013-05-13T16:09:33.540" Title="FEniCS : How to interpolate data at vertices of (3D) cells?" Tags="|fenics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7140" PostTypeId="1" CreationDate="2013-05-09T21:01:54.563" Score="3" ViewCount="314" Body="&lt;p&gt;I am trying to calculate a 3D Function f on a large set of points, say a uniform cubic grid. Besides, the cubic grid points do not coincide with the mesh vertices where the Function f is defined. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;mesh1 = Mesh( meshfile) # A uniform mesh&#xA;V1 = FunctionSpace(mesh1, 'CG', 1)&#xA;f1 = Function(V1)&#xA;f1 = ... # Construct f1 (e.g. by solving a PDE based on mesh1)&#xA;&#xA;mesh2 = UnitCube(100,100,100) # Geometrically, the unit cube domain is a subset of mesh1's domain&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I want to calculate the values of f1 at all vertices of mesh2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that I can either use interpolate function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;V2 = FunctionSpace(mesh2, ‘CG', 1)&#xA;f2 = interpolate(f1, V2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then f2.vector().array() gives me those values (although I understand that the order is not the same as the mesh vertices for the current version).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or I can loop all vertices of mesh2 explicitly, and at each step invoke f1(x, y, z) to calculate at that vertex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that both these two methods are very very costly, and the interpolate method is even more time consuming than the explicit loop. Does anybody know if there is a more efficient way to do this job?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot,&lt;/p&gt;&#xA;" OwnerUserId="4232" LastActivityDate="2013-05-09T21:15:58.243" Title="Fast function evaluation on a grid" Tags="|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7141" PostTypeId="2" ParentId="7140" CreationDate="2013-05-09T21:15:58.243" Score="4" Body="&lt;p&gt;There will be non-negligible overhead when moving between meshes (some current developments will make this faster than it is now). You could first interpolate, then to evaluate vertex values efficiently use the function &lt;code&gt;Function::compute_vertex_values&lt;/code&gt;. For spaces that are not continuous it can yield some unexpected results.&lt;/p&gt;&#xA;" OwnerUserId="3952" LastActivityDate="2013-05-09T21:15:58.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7142" PostTypeId="1" CreationDate="2013-05-09T21:26:36.257" Score="2" ViewCount="46" Body="&lt;p&gt;I'm currently learning the maximal margin classifier with kernels, and I'm wondering - why do they work? In which cases do they work best?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm especially interested in the RBF and polynomial kernels, but not only.&lt;/p&gt;&#xA;" OwnerUserId="4019" LastActivityDate="2013-08-29T19:14:39.477" Title="Why do some known kernel functions manage to achieve linear separation in feature space?" Tags="|support-vector-machines|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7143" PostTypeId="2" ParentId="7139" CreationDate="2013-05-09T21:27:25.837" Score="0" Body="&lt;p&gt;You are using piecewise linear functions (&lt;code&gt;V = FunctionSpace(mesh, &quot;CG&quot;, 1)&lt;/code&gt;), so if you are evaluating the function on a point which is not a vertex, you will get a linear interpolation. You don't say anything about the mesh you are using, but if it's not curvilinear, I would not assume that points on that circle coincide with points on element boundaries (where the interpolant would be constant).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-05-09T21:27:25.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7144" PostTypeId="2" ParentId="7139" CreationDate="2013-05-09T21:55:03.583" Score="1" Body="&lt;p&gt;I assume that qvalues2 has some computed values at the vertices. You cannot directly assign these to your dof vector as the dofs does not follow vertex numbering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could however try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vertex_to_dof = V.dofmap().vertex_to_dof_map(mesh)&#xA;f.vector()[:] = qvalues2[vertex_to_dof]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4250" LastActivityDate="2013-05-09T21:55:03.583" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7145" PostTypeId="2" ParentId="7139" CreationDate="2013-05-10T11:45:55.890" Score="3" Body="&lt;p&gt;Defining a custom &lt;code&gt;Expression&lt;/code&gt; and then interpolating that into your function space should do the trick.&#xA;Check out&lt;/p&gt;&#xA;&#xA; &lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from dolfin import *                                                        &#xA;&#xA;mesh = UnitSquareMesh(20, 20)                                               &#xA;&#xA;class CharCircle(Expression):                                               &#xA;    def eval(self, value, x):                                               &#xA;        xm0 = x[0] - 0.5                                                    &#xA;        xm1 = x[1] - 0.5                                                    &#xA;        if xm0*xm0 + xm1*xm1 &amp;lt; 0.4**2:                                      &#xA;            value[0] = 1.0                                                  &#xA;        else:                                                               &#xA;            value[0] = 0.0                                                  &#xA;&#xA;V = FunctionSpace(mesh, 'CG', 1)                                            &#xA;u = Function(V)                                                             &#xA;u.interpolate(CharCircle())                                                 &#xA;&#xA;plot(u)                                                                     &#xA;interactive()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-05-10T11:45:55.890" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7146" PostTypeId="1" CreationDate="2013-05-10T15:31:35.373" Score="7" ViewCount="926" Body="&lt;p&gt;I've been trying to find some resources that would help me figure out how to numerically solve a coupled system of ODEs which is also an eigenvalue problem.&lt;/p&gt;&#xA;&lt;p&gt;The system is something like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \tag{1} \kappa h_2(r) +\kappa  r h'_2(r)+  (1+\kappa) W_3(r) + (1+ \kappa)&#xA;   r W'_3(r)+   (1 + \kappa) r^2 W''_3(r) = 0 $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \tag{2} W_3 +r W'_3+ (1+\kappa) h_2 + (1 +\kappa)  r h'(2,r)+ (1+ \kappa)r^2 h''_2  = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm not looking for a solution to these equations per se, these are just the equations I have so I thought I'd put them up as a reference.&lt;/p&gt;&#xA;&lt;p&gt;I need to solve for the functions &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W_3$&lt;/span&gt;, on the domain (&lt;span class=&quot;math-container&quot;&gt;$0,R$&lt;/span&gt;). I have the following boundary conditions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\tag {3} W_3(r \rightarrow 0) = r^3, W_3(r = R) = 0 $&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\tag{4}   h_2(r \rightarrow 0) = r^3, h_2 (r = R) = h_{2_{outside}}(r=R) $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where  &lt;span class=&quot;math-container&quot;&gt;$h_{2_{outside}} $&lt;/span&gt; is a know solution from the domain (&lt;span class=&quot;math-container&quot;&gt;$R,\infty$&lt;/span&gt;). I also have the following condition&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \tag {5}  h_2 (r = R) h_{2_{outside}}'(r=R) - h_2'(r=R)h_{2_{outside}}(r=R) =0  $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am also confused about this system of equations because it would appear that I have one too many boundary conditions. That is unless the conditions on &lt;span class=&quot;math-container&quot;&gt;$W_3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt; going to zero are redundant. That is enforcing one will enforce the other just to keep equations (1) and (2) self-consistent. However, this issue may be digressing from my main question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The main question is: given equations (1) and (2) what numerical method can I use to find &lt;span class=&quot;math-container&quot;&gt;$W_3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$h_2$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I should mention that I've already tried solving these equations with a spectral approach, but have so far been unsuccessful. So I'm mainly looking for new ways to attack the problem. My preference would be finite-difference methods since I know those, but I'm open to learning new methods as well.&lt;/p&gt;&#xA;" OwnerUserId="1970" LastEditorUserId="20688" LastEditDate="2021-12-31T03:25:19.377" LastActivityDate="2021-12-31T03:25:53.377" Title="Solving Coupled ODE eigenvalue problem" Tags="|ode|numerics|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7147" PostTypeId="2" ParentId="7146" CreationDate="2013-05-10T16:37:59.427" Score="9" Body="&lt;p&gt;Reformulating (1) and (2), the system reads&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{pmatrix}&#xA;\mathcal{L}&amp;amp;\\&#xA;&amp;amp;\mathcal{L}&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;W\\&#xA;h_2&#xA;\end{pmatrix}&#xA;=&#xA;\kappa&#xA;\begin{pmatrix}&#xA;-\mathcal{L} &amp;amp;\mathcal{M}\\&#xA;\mathcal{M} &amp;amp; -\mathcal{L}&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;W\\&#xA;h_2&#xA;\end{pmatrix}&#xA;$$&lt;/span&gt;&#xA;where the calligraphic symbols are composed of some linear differential operators. You're right about the fact that this is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigendecomposition_of_a_matrix#Generalized_eigenvalue_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;generalized eigenvalue problem&lt;/a&gt;  (and although technically correct, one would probably not talk about an &amp;quot;ODE&amp;quot; here).&lt;/p&gt;&#xA;&lt;p&gt;On a finite domain like &lt;span class=&quot;math-container&quot;&gt;$(0,R)$&lt;/span&gt;, differential equations without boundary conditions are virtually meaningless. You're right in your assumption that your boundary conditions probably require some more work. For second-order differential equations like (1), (2), the set of conditions&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W(0) = w_0\\&#xA;W(R) = w_R\\&#xA;h_2(0) = h_{20}\\&#xA;h_2(R) = h_{2R}&#xA;$$&lt;/span&gt;&#xA;would constitute a well-defined system. To my knowledge, statements like &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$W(r)=O(r^3)$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$r\to 0$&lt;/span&gt;&amp;quot; cannot be imposed on a differential equation. You may be able to show though that this is a property of any solution of (1), (2).&lt;/p&gt;&#xA;&lt;p&gt;With an appropriate set of boundary conditions, after discretization, you can feed on a huge body of literature on numerical eigenvalue solvers, e.g., &lt;a href=&quot;http://www-users.cs.umn.edu/%7Esaad/eig_book_2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Saad's book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="20688" LastEditDate="2021-12-31T03:25:53.377" LastActivityDate="2021-12-31T03:25:53.377" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7148" PostTypeId="1" AcceptedAnswerId="7151" CreationDate="2013-05-10T19:02:36.133" Score="4" ViewCount="156" Body="&lt;p&gt;Given a vector-valued Dolfin function &lt;code&gt;u&lt;/code&gt; from the function space &lt;code&gt;V*V&lt;/code&gt; with &lt;code&gt;V=FunctionSpace(mesh, 'CG', 2)&lt;/code&gt;, how do I extract&#xA;$$&#xA;\max_{x\in\Omega} \|u(x)\|&#xA;$$&#xA;? An approximation works, too.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-05-10T21:04:15.380" Title="compute max(||u||) for vector field" Tags="|python|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7149" PostTypeId="1" AcceptedAnswerId="18915" CreationDate="2013-05-10T19:22:42.993" Score="8" ViewCount="13798" Body="&lt;p&gt;I have a job already submitted to a TORQUE queue. The system I submitted it on has several queues, and I want to change which queue I'm running it in. I found &lt;a href=&quot;https://scicomp.stackexchange.com/q/112/81&quot;&gt;this question&lt;/a&gt;, which recommends the use of &lt;code&gt;qalter&lt;/code&gt;, but &lt;code&gt;qalter&lt;/code&gt;'s man page doesn't have instructions for changing the queue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&#xA;I tried &lt;code&gt;qalter -q &amp;lt;queuename&amp;gt; &amp;lt;jobid&amp;gt;&lt;/code&gt; and I get an error message &lt;code&gt;qalter: illegally formed job identifier: &amp;lt;queuename&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I change which queue a job is submitted to using a PBS system?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-07-16T09:49:44.580" Title="Changing the queue for an already submitted PBS job" Tags="|pbs|" AnswerCount="3" CommentCount="0" ClosedDate="2017-07-18T20:16:03.260" ContentLicense="CC BY-SA 3.0" />
  <row Id="7150" PostTypeId="1" AcceptedAnswerId="7153" CreationDate="2013-05-10T20:23:38.767" Score="10" ViewCount="1253" Body="&lt;p&gt;I'm using the Crank-Nicolson finite difference scheme to solve a 1D heat equation. I'm wondering if the maximum/minimum principle of the heat equation (i.e. that the maximum/minimum occurs at the initial condition or on the boundaries) also holds for the discretized solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is probably implied by the fact that Crank-Nicolson is a stable and convergent scheme. But it seems that you might be able to prove this directly via a linear algebra argument using the matrices created from the Crank-Nicolson stencil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd appreciate any pointers to literature on this. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4277" LastActivityDate="2013-05-14T15:18:52.527" Title="Is the maximum/minimum principle of the heat equation maintained by the Crank-Nicolson discretization?" Tags="|linear-algebra|pde|finite-difference|crank-nicolson|" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7151" PostTypeId="2" ParentId="7148" CreationDate="2013-05-10T20:45:29.630" Score="2" Body="&lt;pre&gt;&lt;code&gt;u1, u2 = u.split()&#xA;unorm = project((u1**2 + u2**2)**0.5, V, form_compiler_parameters={'quadrature_degree': 4})&#xA;unorm = norm(unorm.vector(), 'linf')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that quadrature degree is set explicitly to twice the degree of V because UFL does not handle well degree estimation of power with non-integer exponent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last line is aproximative unless u is piecewise linear.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-10T21:04:15.380" LastActivityDate="2013-05-10T21:04:15.380" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7152" PostTypeId="1" CreationDate="2013-05-10T21:21:07.080" Score="4" ViewCount="543" Body="&lt;p&gt;I solved the steady state dynamic linear elastic model in a solid. My equation is a function of frequency and the strong form is:&#xA;$$\operatorname{div}(\operatorname{stress}(\vec x, w)) + w^2  \rho u( \vec x,w)=0$$&#xA;where $\rho$ is the density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The BC is&#xA;$$ \sigma(\vec x,w) \cdot n(x)=T(x,w),   \qquad   u(x,w)=U_0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The physical problem is a plate with dimension of $1 \times 1 \times 0.1$ with a harmonic load on the all the top and also the bottom of the plate is fixed.&#xA;I solved this as a test with the fixed frequency $(w=200)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weak form of the this equation is:&#xA;$$\int \sigma(x,w)\cdot \operatorname{grad}(v)\, dv - \rho w^2 \int  v(x)\cdot u(x,w) \, dv - \int v\cdot T(x,w)\, ds(2) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $v$ is test function, $ds(2)$ is the top of the plate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I compared the results with the model with converged mesh from Abaqus, I  checked almost everything in the code the order of the displacement is mostly correct but &lt;strong&gt;the signs of the displacements are wrong and also the error is large&lt;/strong&gt; between the solution of the Abaqus and this code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked the variation form and I can't find any error in it. So the only thing that I suspected to be wrong is the Neumann boundary condition but I can't find any bug in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be very thankful if someone can help me to figure out what is wrong in my code that I get different answers from the code. Or I am not sure if this is a bug and my code is right.&lt;/p&gt;&#xA;" OwnerUserId="4278" LastEditorDisplayName="user10435" LastEditDate="2015-04-08T02:38:18.847" LastActivityDate="2015-04-08T02:38:18.847" Title="Fenics: Result of Steady state dynamic linear elastic doesn't match with actual values" Tags="|pde|fenics|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="7153" PostTypeId="2" ParentId="7150" CreationDate="2013-05-10T21:21:59.590" Score="9" Body="&lt;p&gt;The maximum principle for Crank-Nicolson will hold if&#xA;$$\mu \doteq \frac{k}{h^2} \leq 1$$&#xA;for timestep $k$ and grid spacing $h$. In general, we can consider a $\theta$-scheme of the form&#xA;$$u^{n+1} = u^n + \frac{\mu}{2}\left( (1-\theta)Au^n + \theta Au^{n+1}\right)$$&#xA;where $A$ is the standard Laplacian matrix and $0 \leq \theta \leq 1$. If $\mu(1-2\theta) \leq \frac{1}{2}$, then the scheme is stable. (This can easily be shown by Fourier techniques.) However, the stronger criterion that $\mu(1-\theta) \leq \frac{1}{2}$ is needed for the maximum principle to hold in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a proof, see &lt;a href=&quot;http://books.google.com/books?id=0uf_iQIBFKgC&quot; rel=&quot;nofollow&quot;&gt;Numerical Solutions of Partial Differential Equations by K. W. Morton&lt;/a&gt;. In particular, look at Sections 2.10 and 2.11 and Theorem 2.2.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There's also a nice way to see that the maximum principle will not hold in general for Crank-Nicolson without a constraint on $\mu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the heat equation on $[0,1]$ with a discretization containing 3 points, including the boundary. Let $u_i^k$ denote the discretization at timestep $k$ and grid point $i$. Assume Dirichlet boundary, so that $u^k_0 = u^k_2 = 0$ for all $k$. Then Crank-Nicolson reduces to&#xA;$$\left(1 - \frac{\mu}{2}(-2)\right)u^{n+1}_1 = \left(1 + \frac{\mu}{2}(-2)\right)u^n_1,$$&#xA;which can be further reduced to&#xA;$$u^{n+1}_1 = \left(\frac{1-\mu}{1+\mu}\right)u^n_1.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we consider the initial condition of $u_1^0 = 1$, then we have&#xA;$$u^n_1 = \left(\frac{1-\mu}{1+\mu}\right)^n,$$&#xA;and though it will always be the case that $u^n_1 \leq 1$, we will nonetheless have that $u^n_1 &amp;lt; 0$ for odd $n$ unless $\mu \leq 1$. Thus the maximum/minimum principle is violated unless $\mu \leq 1$. This is particularly noteworthy in light of the fact that Crank-Nicolson is stable for any $\mu$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In response to foobarbaz's request, I've added a sketch of the proof.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key is to write the scheme in the form&#xA;\begin{align*}&#xA;(1+2\theta\mu)u^{n+1}_j &amp;amp;= \theta\mu(u^{n+1}_{j-1} + u^{n+1}_{j+1})\\&#xA;                        &amp;amp;+ (1-\theta)\mu(u^n_{j-1} + u^n_{j+1})\\&#xA;                        &amp;amp;+ [1-2(1-\theta)\mu]u^n_j&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hypothesis that $\mu(1-\theta)\leq \frac{1}{2}$ is exactly equivalent to the fact that all of the above coefficients are nonnegative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose that the maximum is attained at an interior point $u^{n+1}_j$. Note that all of $u^{n+1}_{j-1}$, $u^{n+1}_{j+1}$, $u^n_{j-1}$, $u^n_{j+1}$, $u^n_j$ are less than or equal to $u^{n+1}_j$ by assumption. If any of these is strictly less than $u^{n+1}_j$, then the above equality and the nonnegativity of the coefficients imply that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;(1+2\theta\mu)u^{n+1}_j &amp;amp;&amp;gt; \theta\mu(u^{n+1}_{j-1} + u^{n+1}_{j+1})\\&#xA;                        &amp;amp;+ (1-\theta)\mu(u^n_{j-1} + u^n_{j+1})\\&#xA;                        &amp;amp;+ [1-2(1-\theta)\mu]u^n_j\\&#xA;                        &amp;amp;= (1+2\theta\mu)u^{n+1}_j&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is a contradiction. It follows that the maximum must also be attained at all of the temporal and spatial neighbors of $u^{n+1}_j$, and a connectedness argument then implies that the discretization of $u$ must be constant in space and time, so that the maximum is still attained on the boundary. Note that this connectedness argument mirrors the proof of the analytic (i.e., not discretized) maximum principle.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="4277" LastEditDate="2013-05-14T15:18:52.527" LastActivityDate="2013-05-14T15:18:52.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7154" PostTypeId="1" AcceptedAnswerId="7160" CreationDate="2013-05-10T22:52:18.047" Score="5" ViewCount="519" Body="&lt;p&gt;Given a real-rectangular matrix &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; and inorder to solve this simple quadratic programming problem:&lt;/p&gt;&#xA;&lt;p&gt;Minimize &lt;span class=&quot;math-container&quot;&gt;$w'S'Sw = ||S w||^2$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; subject to&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$e^Tw = 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w \geq 0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;using a solver I want a re-parametrization of the problem to the form:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\min(-d^T b + 1/2 b^T D b)$&lt;/span&gt; with the constraints&#xA;&lt;span class=&quot;math-container&quot;&gt;$A^T b \geq b_0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;so that I can use a general-purpose optimization software for quadratic programming.&lt;/p&gt;&#xA;&lt;p&gt;Question: So now, what would &lt;span class=&quot;math-container&quot;&gt;$d,b,D,A,b_0$&lt;/span&gt; be?&lt;/p&gt;&#xA;&lt;p&gt;Secondly, how is this re-parametrization done (is there a well-known procedural, aspect to this, or is it just algebra)? I ask because I would want to use this general-purpose solver for various quadratic minimization programs.&lt;/p&gt;&#xA;&lt;p&gt;I can see that &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and I am guessing that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is an identity matrix and &lt;span class=&quot;math-container&quot;&gt;$b_0$&lt;/span&gt; is a vector of zeros. What is &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="1934" LastEditorUserId="4405" LastEditDate="2022-01-01T22:30:26.433" LastActivityDate="2022-01-01T22:30:26.433" Title="Reducing a quadratic program to standard form" Tags="|optimization|quadratic-programming|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7156" PostTypeId="1" CreationDate="2013-05-11T04:21:51.710" Score="4" ViewCount="2227" Body="&lt;p&gt;Von Newman stability analysis for acoustic wave equation explicit centered differences: 2nd order time and space (N 2)'th order:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;U_{jk}^{n+1}  =  \left( \frac{\Delta t  V_{jk} }{\Delta s} \right) ^2 \left(  \sum_{a=-N}^N w_a U_{j+a k}^n + \sum_{a=-N}^N w_a U_{j k+a}^n \right) + 2 U_{jk}^{n} - U_{jk}^{n-1}  \nonumber \\&#xA;U_{jk}^{n+1}  =  \left( \frac{\Delta t  V_{jk}}{\Delta s} \right) ^2  \sum_{a=-N}^N  w_a \left( U_{j+a k}^n + U_{j k+a}^n \right) + 2 U_{jk}^{n} - U_{jk}^{n-1} \label{eq:w1}&#xA;\end{eqnarray}(1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For forth order space, we have $N=2$ and $w$ is:&#xA;$$ w = \frac{1}{12} [-1, 16, -30, 16, -1] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can also be simplified to 1st order (N=1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;U_{jk}^{n+1}  =  \left( \frac{\Delta t  V_{jk}}{\Delta s} \right) ^2 \left(  U_{j+1k}^n - 4 U_{jk}^n + U_{jk+1}^n + U_{j-1k}^n + U_{jk-1}^n  \right) + 2 U_{jk}^{n} - U_{jk}^{n-1} \nonumber&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the discrete solution for 2D wave equation, where $ i = \sqrt{-1} $, $ n = n \Delta t $, $ j = j \Delta x $ and $ k = k \Delta z $. Last using $ \Delta x = \Delta z = \Delta s $, follows that the discrete solution can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;U_{jk}^n = e^{i \left( \omega t + px + qz \right)} \nonumber \\&#xA;U_{jk}^n = \epsilon^n e^{i \left( pj\Delta s + qk\Delta s \right)}  \nonumber \\&#xA;U_{jk}^n = \epsilon^n e^{i \Delta s \left( pj + qk \right)}  \label{eq:w2}&#xA;\end{eqnarray}(2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\epsilon $ is the growth factor, and should be $ |\epsilon| \leq 1$ for stability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replacing (2) in (1), using the identities bellow and simplifying dividing both sides by $ U_{jk}^{n+1} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ r = \frac{\Delta t  V_{jk}}{\Delta s} $$&#xA;$$ \phi_{j+l\ k+m} = e^{i \Delta s \left( pl+qm \right)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\Omega = r^2 \sum_{a=-N}^N  w_a \left( \phi_{j+a k} + \phi_{j k+a}  \right) \label{eq:w3}&#xA;\end{equation}(3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;1  =  \left( \Omega + 2 \right) \epsilon^{-1} -\epsilon^{-2} \nonumber \\&#xA;\quad \text{making} \ \ \epsilon^{-1} = \mu \nonumber \\&#xA;\mu^2 - \left( \Omega + 2 \right) \mu + 1 = 0 \nonumber \\&#xA;\mu = \frac{(\Omega+2) \pm \sqrt{\Omega^2 + 4\Omega}}{2} \label{eq:w4}&#xA;\end{eqnarray}(4)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;back to expand $ \Omega $ defined in (3):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\Omega &amp;amp;=&amp;amp; r^2 \sum_{a=-N}^N  w_a \left( \phi_{j+a k} + \phi_{j k+a}  \right) \nonumber \\&#xA;	&amp;amp;=&amp;amp; r^2 \sum_{a=-N}^{N} w_a ( e^{i \Delta s \ p a} + e^{i \Delta s \ q a} )\nonumber &#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;&amp;amp;=&amp;amp; r^2 \begin{pmatrix}&#xA; \cdots &amp;amp; e^{-i \Delta s 2 p} + e^{-i \Delta s 2 q} &amp;amp; e^{-i \Delta s p} + e^{-i \Delta s q} &amp;amp; e^0+e^0 &amp;amp; e^{i \Delta s p} + e^{i \Delta s q} &amp;amp; e^{i \Delta s 2 p} + e^{i \Delta s 2 q} &amp;amp; \cdots \\&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;\cdots \\&#xA;w_{-2} \\&#xA;w_{-1} \\&#xA;w_0 \\&#xA;w_1 \\&#xA;w_2 \\&#xA;\cdots&#xA;\end{pmatrix}&#xA;\nonumber&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $w$ is even $ w_a = w_{-a} $ and $ e^{i\theta} + e^{-i\theta} = 2 \cos{\theta} $ we can rewrite as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;&amp;amp;=&amp;amp; r^2&#xA;\begin{pmatrix}&#xA; \cdots &amp;amp; 2\cos( \Delta s 2 p) + 2\cos(\Delta s 2 q) &amp;amp; 2\cos(\Delta s p) + 2\cos(\Delta s q) &amp;amp; 2 \\&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;\cdots \\&#xA;w_{2} \\&#xA;w_{1} \\&#xA;w_0 \\&#xA;\end{pmatrix}&#xA;\nonumber&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the simplest case 2nd order $ N=1 $ we have $ (w_1, w_0) = (1, -2) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\Omega	&amp;amp;=&amp;amp; r^2 \left( 2\cos(\Delta s p) + 2\cos(\Delta s q) - 4\right) \nonumber \\&#xA;	&amp;amp;=&amp;amp; -4r^2 \left( \sin^2(\frac{\Delta s p}{2}) + \sin^2(\frac{\Delta s q}{2}) \right) \label{eq:w5}&#xA;\end{eqnarray}(5)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note:  $ 2 \cos(\theta) - 2 = -4 \sin ^2 (\theta) $ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can also write (5) using $ \beta = \left( \sin^2(\frac{\Delta s p}{2}) + \sin^2(\frac{\Delta s q}{2}) \right) $ as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Omega = -4r^2\beta $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replacing back to (4) :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\mu &amp;amp;=&amp;amp; \frac{(\Omega+2) \pm \sqrt{\Omega^2 + 4\Omega}}{2} \nonumber \\&#xA;\mu &amp;amp;=&amp;amp; -2r^2\beta+1 \pm 2\sqrt{r^2\beta(r^2\beta-1)} \nonumber&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am a little lost how to find if $ | \mu | &amp;gt;= 1 $ or what limitations I have in $r$ for this requirement, that is the same as needing $ | \epsilon | &amp;lt;= 1  $.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any easier alternative to Von Newman that also could be applied to the general explicit form in (1) ?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4092" LastEditorUserId="4092" LastEditDate="2013-07-22T18:23:48.820" LastActivityDate="2013-08-21T19:13:11.157" Title="Von Newman stability analysis for 2D acoustic wave equation explicit" Tags="|pde|finite-difference|stability|fourier-analysis|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7157" PostTypeId="1" CreationDate="2013-05-11T04:38:18.167" Score="4" ViewCount="221" Body="&lt;p&gt;I have a cost function: $f(x,y,z) \rightarrow \mathbb{R}$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it is very expensive to evaluate&lt;/li&gt;&#xA;&lt;li&gt;$x,y,z \in \mathbb{Z}$&lt;/li&gt;&#xA;&lt;li&gt;0 &amp;lt; x &amp;lt; 10&lt;/li&gt;&#xA;&lt;li&gt;0 &amp;lt; y &amp;lt; 30&lt;/li&gt;&#xA;&lt;li&gt;0 &amp;lt; z &amp;lt; 100&lt;/li&gt;&#xA;&lt;li&gt;I thought it was convex, not sure now based on @Brian's commentary.&lt;/li&gt;&#xA;&lt;li&gt;$f$ can only be evaluated at integers ( I pondered trying to evaluate at real's but it would be fairly insanely difficult to try &amp;amp; the result would be totally bogus).&lt;/li&gt;&#xA;&lt;li&gt;$f$ is smooth but only at the integer values.  So the gradient could only be computed using re-evaluating $f$ at intervals - but again that's really expensive.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm looking for suggestions on algorithms, and libraries ( C++ preferred, C acceptable ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've spent a couple of days trying to brush up on the topics, but its pretty dense.  I've been looking at &lt;a href=&quot;http://www.coin-or.org/&quot; rel=&quot;nofollow&quot;&gt;COIN-OR&lt;/a&gt;, specifically OSI, but I can't seem to figure out how to formulate my problem into the API.  I've also looked at &lt;a href=&quot;http://tabu.diegm.uniud.it/EasyLocal++/&quot; rel=&quot;nofollow&quot;&gt;EasyLocal++&lt;/a&gt;, but haven't really dug in yet.&lt;/p&gt;&#xA;" OwnerUserId="4280" LastEditorUserId="4280" LastEditDate="2013-05-12T03:59:30.750" LastActivityDate="2013-05-12T03:59:30.750" Title="Optimization algorithm selection for 3 variable integer" Tags="|optimization|algorithms|convex-optimization|mixed-integer-programming|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7158" PostTypeId="1" AcceptedAnswerId="7159" CreationDate="2013-05-11T11:01:22.953" Score="8" ViewCount="364" Body="&lt;p&gt;Are stabilization techniques for convection-dominated flows like SUPG+PSPG, interior penalty methods, etc. able to handle turbulent flows without tubulence model being employed, at least up to some Reynolds numbers? Conversely is stabilization needed for turbulent flow being resolved by some RANS model when Reynolds number becomes larger and problem stiffer? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, what is the relationship between stabilization techniques and RANS models? Are they different approaches to similiar problem?&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2015-09-18T16:22:41.920" Title="Stabilization of convection-dominated flow and turbulence modeling" Tags="|finite-element|fluid-dynamics|stability|advection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7159" PostTypeId="2" ParentId="7158" CreationDate="2013-05-11T12:36:37.900" Score="5" Body="&lt;p&gt;Bluntly speaking, SUPG and alike and RANS are different approaches to different problems that, however, have the same name - instability - and the same phenomenology - the failure of numerical routines. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;RANS is used to cope with turbulence as an instability of the equation. If a flow is or becomes turbulent the describing equations are instable, e.g. because of bifurcation. This means that a small disturbance may lead to large changes in the solution. I tend to see RANS as a reformulation of the problem, so that the equations are stable and numerical algorithms are applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SUPG and alike eliminate instabilities of numerical algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your question whether you can use SUPG instead of RANS. I would say no. If your problem is unstable, also stable numerical algorithm are not convergent. Your algorithm may converge to a solution, but you cannot be sure that you are on the right way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The common point I see is that turbulence occurs at high Reynolds numbers, where also the factor space discretization times velocity magnitude $hv$ tends to be large, i.e. where upwind stabilization, as it is mimicked in SUPG, is necessary.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-11T14:29:30.683" LastActivityDate="2013-05-11T14:29:30.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7160" PostTypeId="2" ParentId="7154" CreationDate="2013-05-11T16:12:19.843" Score="5" Body="&lt;p&gt;To transform your original program into the form you specified, use the following mappings:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First, $D = 2S'S$, and $D$ is positive semidefinite.&lt;/li&gt;&#xA;&lt;li&gt;$b = w$, as you pointed out&lt;/li&gt;&#xA;&lt;li&gt;$d = 0$, as Nico notes&lt;/li&gt;&#xA;&lt;li&gt;$A$ takes the following form:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A = \left[\begin{array}{c} I \\ e^{T} \\ -e^{T}\end{array}\right]&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$b_0$ takes the following form:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;b_{0} = \left[ \begin{array}{c} 0 \\ 1 \\ -1\end{array} \right]&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $0$ is a vector of zeros, but $1$ and $-1$ are scalars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values of $A$ and $b_{0}$ are the tricky part. The rows of $A$ and $b_{0}$ can be partitioned into three sets:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the rows of $A$ and $b_{0}$ corresponding to the pair $(I, 0)$ map to $w \geq 0$&lt;/li&gt;&#xA;&lt;li&gt;the row of $A$ and $b_{0}$ corresponding to the pair $(e^{T}, 1)$ map to $e^{T}w \geq 1$&lt;/li&gt;&#xA;&lt;li&gt;the row of $A$ and $b_{0}$ corresponding to the pair $(-e^{T}, -1)$ map to $-e^{T}w \geq -1$, which is equivalent to $e^{T}w \leq 1$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The pair $e^{T}w \geq 1$ and $e^{T}w \leq 1$ is equivalent to $e^{T}w = 1$. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-05-11T16:12:19.843" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7161" PostTypeId="2" ParentId="6880" CreationDate="2013-05-11T17:12:31.973" Score="1" Body="&lt;p&gt;Here&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;K.R. RAJAGOPAL, M. RUZICKA, and A.R. SRINIVASA, Math. Models Methods&#xA;  Appl. Sci. 06, 1157 (1996). &lt;em&gt;ON THE OBERBECK-BOUSSINESQ APPROXIMATION.&lt;/em&gt;&#xA;  &lt;a href=&quot;http://dx.doi.org/10.1142/S0218202596000481&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1142/S0218202596000481&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;you may find Boussinesq approximation derived using perturbation technique. Criterion stating when this approximation is valid is formulated there.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-11T19:25:23.033" LastActivityDate="2013-05-11T19:25:23.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7162" PostTypeId="2" ParentId="6757" CreationDate="2013-05-11T17:57:43.760" Score="9" Body="&lt;p&gt;If you can stand to use complex arithmetic, &lt;em&gt;simultaneous iteration&lt;/em&gt; methods might be preferable for computing all the roots of your polynomial. The simplest simultaneous iteration method, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Durand%E2%80%93Kerner_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Weierstrass-)Durand-Kerner method&lt;/a&gt;, is effectively equivalent to applying Newton-Raphson to the Vieta relations relating the coefficients and roots of a polynomial, treated as simultaneous equations. (See also &lt;a href=&quot;https://math.stackexchange.com/a/84085&quot;&gt;this math.SE answer&lt;/a&gt;.) Thus, this method is also quadratically convergent, if the polynomial has no repeated roots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Customarily, the method is started with initial approximations lying around a circle in the complex plane, with the center and radius determined from the coefficients (which is why I mentioned the need for complex arithmetic). If you know in advance that the roots are real, and you already have good approximations for all the roots, one could also use Durand-Kerner for polishing purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related simultaneous iteration method is &lt;a href=&quot;http://en.wikipedia.org/wiki/Aberth_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ehrlich-Aberth(-Maehly)&lt;/a&gt;, mentioned briefly by Gert in his answer. The method has cubic convergence if the roots of your polynomial are all simple, but the iteration formula is a bit more complicated, involving the evaluation of the polynomial's derivative (which of course can be done with e.g. Horner). Picking between these two methods would require some testing on your part.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-05-11T17:57:43.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7163" PostTypeId="2" ParentId="7157" CreationDate="2013-05-11T18:09:16.640" Score="3" Body="&lt;p&gt;You haven't said whether $f$ can be evaluated at points where $x$, $y$, and $z$ are real numbers within the specified ranges that aren't integers.  If this makes no sense, than the function isn't actually convex.  You also haven't said whether $f$ is smooth (e.g. is it twice continuously differentiable?)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conventional gradient based optimization algorithms will typically require quite a few function evaluations (e.g. for line searches) as well as requiring the gradient of $f$ (which can be approximated by finite differences, but that implies even more function evaluations.)  If the function is convex but not smooth, and you can't describe the non-smoothness in more detail, then these algorithms aren't appropriate anyway.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a low dimensional problem with very expensive (or noisy) function evaluations, its often a good choice to use a &quot;response surface method&quot; to build a regression model of the function, minimize over the simple (e.g. quadratic)function that you've built using regression, and then (as many times as you can afford) narrow your search and build a new regression model.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you afford to do 27 function evaluations at $x=0, 5, 10$, $y=0, 15, 30$, and $z=0, 50, 100$?  That should be enough to construct a first quadratic model of the response surface.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-05-11T18:09:16.640" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7164" PostTypeId="2" ParentId="7157" CreationDate="2013-05-11T18:50:07.487" Score="0" Body="&lt;p&gt;The convexity of $f$ (over the continuous relaxation of the domain) does not matter as far as choosing the outermost solver: you will need a branch-and-bound algorithm if you choose to solve your problem deterministically because you have binary decision variables. I presume $f$ is nonlinear? If that is the case, you will need an MINLP solver like &lt;a href=&quot;https://projects.coin-or.org/Bonmin&quot; rel=&quot;nofollow&quot;&gt;Bonmin&lt;/a&gt;. The convexity of $f$ will help in solving lower bounding problems; if you relax all of the binary variables to continuous variables, you should be able to find a lower bound on your solution with a nonlinear programming solver like &lt;a href=&quot;https://projects.coin-or.org/Ipopt&quot; rel=&quot;nofollow&quot;&gt;IPOPT&lt;/a&gt;. Feasible solutions will give you an upper bound. I'll see if I can dig around to find a good online reference about branch-and-bound algorithms in integer programming. You could also solve the nonlinear problems using derivative-free methods, to save on function evaluations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $f$ is linear, you'll probably want to use something like &lt;a href=&quot;http://www.coin-or.org/projects/Cbc.xml&quot; rel=&quot;nofollow&quot;&gt;CBC&lt;/a&gt;, which is a branch-and-cut framework (similar in spirit to branch-and-bound). However, given that $f$ is expensive to evaluate, I doubt it is linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods don't lend themselves to problems with expensive function evaluations. Are you willing to give up some some accuracy in the solution for a faster algorithm, and if so, how much?&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-05-11T18:50:07.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7165" PostTypeId="1" CreationDate="2013-05-11T20:36:50.507" Score="2" ViewCount="137" Body="&lt;p&gt;I want to save a matlab source code into a string format in matlab. Does anyone know how to do this? For example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;type xxx.m&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can display the source code of xxx.m. Then using what command am I able to save it into a string?&lt;/p&gt;&#xA;" OwnerUserId="346" LastActivityDate="2013-05-11T22:44:45.613" Title="How to save a matlab source code into a string in matlab?" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7166" PostTypeId="2" ParentId="7165" CreationDate="2013-05-11T22:44:45.613" Score="2" Body="&lt;p&gt;Use &lt;code&gt;string =type('filename.m');&lt;/code&gt; as per the MATLAB documentation page for the &lt;a href=&quot;http://www.mathworks.com.au/help/matlab/ref/type.html&quot; rel=&quot;nofollow&quot;&gt;type&lt;/a&gt; command.&lt;/p&gt;&#xA;" OwnerUserId="1918" LastActivityDate="2013-05-11T22:44:45.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7167" PostTypeId="1" AcceptedAnswerId="7181" CreationDate="2013-05-11T23:47:00.080" Score="5" ViewCount="1022" Body="&lt;p&gt;Having&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh = UnitSquareMesh(2, 2)&#xA;Q = FunctionSpace(mesh, 'DG', 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this works as expected&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R = mesh.ufl_cell().circumradius&#xA;R = project(R, Q)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this fails&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h = mesh.ufl_cell().max_facet_edge_length&#xA;h = project(h, Q)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;when compiling generated code with error: 'facet_area' was not declared...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does max_facet_edge_length return &lt;strong&gt;lengths of the longest edge of a cell&lt;/strong&gt; as I expected or does it do something else? If so, is there some implementation of this? Or should I loop over cells and their edges to compute h manually?&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2013-05-22T12:37:18.723" Title="Compute lengths of the longest edge of each cell" Tags="|python|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7169" PostTypeId="2" ParentId="7097" CreationDate="2013-05-12T05:01:39.653" Score="11" Body="&lt;p&gt;Let me first answer all the questions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the theoretical convergence rate for an FFT Poison solver?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The theoretical convergence is exponential as long as the solution is sufficiently smooth.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How fast should this energy converge?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Hartree energy $E_H$ should converge exponentially for a sufficiently smooth solution. If the solution is less smooth, then the convergence is slower.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does anyone know any benchmark in 3D so that I can see faster convergence than linear?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any right hand side that produces a solution that is periodic and infinitely differentiable (including across the periodic boundary) should converge exponentially.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the code above there happens to be a bug, that causes the convergence to be slower than exponential. Using the smooth density code (&lt;a href=&quot;https://gist.github.com/certik/5549650/&quot; rel=&quot;noreferrer&quot;&gt;https://gist.github.com/certik/5549650/&lt;/a&gt;), the following patch fixes the bug:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@@ -6,7 +6,7 @@ f = open(&quot;conv.txt&quot;, &quot;w&quot;)&#xA; for N in range(3, 180, 10):&#xA;     print &quot;N =&quot;, N&#xA;     L = 2.&#xA;-    x1d = linspace(0, L, N)&#xA;+    x1d = linspace(0, L, N+1)[:-1]&#xA;     x, y, z = meshgrid(x1d, x1d, x1d)&#xA;&#xA;     nr = 3*pi*sin(pi*x)*sin(pi*y)*sin(pi*z)/4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem was that the real space sampling &lt;strong&gt;cannot&lt;/strong&gt; repeat the first and last point (which has the same value due to periodic boundary condition). In other words, the problem was in setting up the initial sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this fix, the density converges in one iteration, as Matt said above. So I didn't even plot the convergence graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one can try a more difficult density, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     nr = 3*pi*exp(sin(pi*x)*sin(pi*y)*sin(pi*z))/4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then the convergence is exponential, as expected. Here are the convergence graphs for this density:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/3wwUg.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/OGwjw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="236" LastEditDate="2013-05-12T17:28:03.320" LastActivityDate="2013-05-12T17:28:03.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7170" PostTypeId="1" AcceptedAnswerId="7173" CreationDate="2013-05-12T07:45:09.347" Score="6" ViewCount="2554" Body="&lt;p&gt;I have been given the task of implementing SVD using the Lanczos method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it only helps to find eigenvalues using a sequence of tridiagonal matrices, but i don't know how to apply this to actually computing the SVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to approximate the given matrix with tridiagonal and then compute SVD with different algorithm or something different?&lt;/p&gt;&#xA;" OwnerUserId="4286" LastEditorUserId="127" LastEditDate="2013-05-12T17:15:24.180" LastActivityDate="2013-05-12T17:15:24.180" Title="SVD and Lanczos method" Tags="|linear-algebra|svd|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7171" PostTypeId="1" AcceptedAnswerId="8349" CreationDate="2013-05-12T09:55:44.090" Score="2" ViewCount="3006" Body="&lt;p&gt;We are working on an optimization problem in which we can approximate the eigenvalue calculation by assuming a constant eigenvector, using the formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde{\lambda}=\frac{\{\phi_0\}^T[K]\{\phi_0\}}{\{\phi_0\}^T[K_{G_0}]\{\phi_0\}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\{\phi_0\}$ is the already calculated eigenvector, $[K_{G_0}]$ is the already calculated differential stiffness matrix and $[K]$ is the new stiffness matrix.&#xA;We managed already to print $[K]$ using a couple of DMAP commands (&lt;a href=&quot;https://www.dropbox.com/s/7ms7xj4xp06qa5f/finite-element-nastran-how-to-print-the-differential-stiffness-matrix-in-f06.zip&quot; rel=&quot;nofollow&quot;&gt;see example in this link&lt;/a&gt;), but how to print $[K_{G_0}]$?&lt;/p&gt;&#xA;" OwnerUserId="4287" OwnerDisplayName="sgcastro" LastEditorUserId="4287" LastEditDate="2013-08-28T07:58:03.390" LastActivityDate="2014-01-25T10:32:02.003" Title="Finite Element, NASTRAN, DMAP, how to print the differential stiffness matrix in .f06 output file" Tags="|optimization|finite-element|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7173" PostTypeId="2" ParentId="7170" CreationDate="2013-05-12T17:10:20.390" Score="7" Body="&lt;p&gt;In theory, you can &quot;square up&quot; $A$ as Jan illustrates in his answer, but you really don't want to do that when actually computing the SVD, as it comes at a significant cost of accuracy. A better approach that leverages standard Lanczos tridiagonalization is to perform the tridiagonalization of the matrix $\left[\begin{smallmatrix} 0 &amp;amp; A \\ A^T &amp;amp; 0 \end{smallmatrix}\right]$. This avoids the accuracy loss from squaring up, and you can extract both the singular values and their accompanying left and right vectors from the eigendecomposition of this larger matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally though what you're looking for is &lt;em&gt;Lanczos bidiagonalization&lt;/em&gt;. Lanczos bidiagonlization is intimately related to the tridiagonalization of the cyclic matrix, but can also be derived on its own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://soi.stanford.edu/~rmunk/PROPACK/&quot;&gt;PROPACK&lt;/a&gt; is a good example of an existing software package that implements Lanczos bidiagonalization for SVD. It sounds like you're required to implement it yourself, but the page I'm linking to here provides a number of papers that provide an introduction to the algorithmic approach. You can likely ignore the &quot;partial reorthogonalization&quot; that is a key part of the PROPACK package and focus on the bidiagonalization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if this isn't a homework assignment or educational exercise, then I would suggest considering &lt;em&gt;using&lt;/em&gt; PROPACK or related packages instead of implementing it yourself! There's no need to reinvent the wheel. &lt;a href=&quot;http://www.grycap.upv.es/slepc/documentation/manual.htm&quot;&gt;SLEPc&lt;/a&gt; also includes a Lanczos SVD implementation. The SLEPc documentation provides a good introduction to Lanczos bidiagonalization as well.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-05-12T17:15:22.267" LastActivityDate="2013-05-12T17:15:22.267" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7174" PostTypeId="2" ParentId="7154" CreationDate="2013-05-12T17:23:48.360" Score="3" Body="&lt;p&gt;Here's another approach: subdivide $w$ and $S$ as follows:&#xA;$$w=\begin{bmatrix}\bar{w} \\ w_n \end{bmatrix} \quad&#xA;  S=\begin{bmatrix}\bar{S} &amp;amp; s \end{bmatrix}$$&#xA;Specifically, $w_n$ is the very last element of $w\in\mathbb{R}^n$, and $\bar{w}\in\mathbb{R}^{n-1}$ is a vector containing every element but that last one.&#xA;Now we have&#xA;$$e^Tw = e^T\bar{w} + w_n = 1 \quad\Longrightarrow\quad w_n=1-e^T\bar{w}$$&#xA;and &#xA;$$Sw=\bar{S}\bar{w} + s w_n = \bar{S} \bar{w} + s ( 1 - e^T\bar{w} ) = (\bar{S}-se^T)\bar{w}+s$$&#xA;and the problem becomes&#xA;$$&#xA;\begin{array}{ll}&#xA;\text{minimize}   &amp;amp; \| (\bar{S}-se^T) \bar{w} + s \|^2 \\&#xA;\text{subject to} &amp;amp; \bar{w} \geq 0 \\ &#xA;                  &amp;amp; e^T \bar{w} \leq 1&#xA;\end{array}&#xA;$$&#xA;Now you've eliminated the equality constraint and constructing the standard form should be straightforward.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-05-12T18:01:40.783" LastActivityDate="2013-05-12T18:01:40.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7175" PostTypeId="1" AcceptedAnswerId="7177" CreationDate="2013-05-13T02:28:35.193" Score="6" ViewCount="2601" Body="&lt;p&gt;Neumann boundary conditions are implemented by introducing ghost points outside the domain and then using the boundary conditions to eliminate the ghost points. For example, see this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5434/conservation-of-a-physical-quantity-when-using-neumann-boundary-conditions-appli&quot;&gt;question&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a set of three equations in variables $(u,v,w)$ which are one dimensional functions of $x$, and I cannot see how to &lt;strong&gt;eliminate&lt;/strong&gt; the ghost points for the equation for the $w$ variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is something like this,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial u}{\partial t} = a_u\frac{\partial^2 u}{\partial x^2} + b_u\frac{\partial u}{\partial x} + f_u(x,u,v,w) \\&#xA;\frac{\partial v}{\partial t} = a_v\frac{\partial^2 v}{\partial x^2} + b_v\frac{\partial v}{\partial x} + f_v(x,u,v,w) \\&#xA;\frac{\partial w}{\partial t} = a_u\frac{\partial u}{\partial x} +a_v\frac{\partial v}{\partial x} + f_w(x,u,v,w) \\&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The set of equations describe a two species advection-diffusion problem where the  third equation couples to the other two. I am having problems applying Neumann boundary conditions to the third equation. Notice that the third equation does not contain a differential of $w$, but it does contain differentials of both $u$ and $v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In discretizated form (only showing the $w$ equation, where $\Delta t$ is the time step and $s_u$ and $s_v$ are the coefficients of the equations after discretization),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;w_j^{n+1} - s_u\beta(u_{j+1}^{n+1} - u_{j-1}^{n+1}) - s_v\beta(v_{j+1}^{n+1} - v_{j-1}^{n+1}) = s_u(1-\beta)(u_{j+1}^{n} - u_{j-1}^{n}) + s_v(1-\beta)(v_{j+1}^{n} - v_{j-1}^{n}) + w_j^n + \Delta tf_{j}^n&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above is a Crank-Nicolson discretization, but for the boundary points let's use an implicit scheme (setting $\beta=1$),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;w_1^{n+1} - s_u(u_{2}^{n+1} - u_{0}^{n+1}) - s_v(v_{2}^{n+1} - v_{0}^{n+1}) = w_1^n + \Delta tf_{1}^n&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ghost points, $u_0^{n+1}$ and $v_0^{n+1}$ can be eliminated, by using the boundary conditions for the $u$ and $v$ variables. However, applying Neumann boundary conditions to the $w$ variable gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial w}{\partial x} = \sigma |_{j=1} \\ \frac{1}{2 \Delta x} (w_{2}^{n+1} - w_{0}^{n+1} ) =  \sigma&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem is that the boundary equation cannot be substituted into the equation for $j=1$ because it does not contain at term for $w_{0}^{n+1}$.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any suggestions on how I can apply Neumann boundary conditions to this equation?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-05-13T16:08:17.340" Title="Trouble implementing Neumann boundary conditions because the ghost points cannot be eliminated" Tags="|finite-difference|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7177" PostTypeId="2" ParentId="7175" CreationDate="2013-05-13T03:34:30.933" Score="5" Body="&lt;p&gt;As there are no spatial derivatives of $w$ in the PDE system, asserting Neumann boundary conditions on $w$ overdetermines the problem. Provided you have an appropriate initial condition for $w$, and thus for $w^0_j$, the discretized evolution equation for $w^{n+1}_j$ (including for $w^{n+1}_1$) you derived above fully determines $w$ for all time.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To put the above in slightly different terms, consider that, in your discretization, no boundary condition on $w$ is needed to evolve the solution forward in time. Given an initial condition $w^0_j$ defined on the boundary and interior, the evolution equations for $u$, $v$, and $w$ can all be solved without reference to values of $w$ across the boundary (i.e., ghost cells). The key is that, because there are no spatial derivatives of $w$, the ghost term $w^n_0$ does not appear in the discretized evolution equations for $u$, $v$, or $w$.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-05-13T16:08:17.340" LastActivityDate="2013-05-13T16:08:17.340" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7179" PostTypeId="2" ParentId="7150" CreationDate="2013-05-13T11:21:45.457" Score="0" Body="&lt;p&gt;Stability means that a perturbation remains bounded in time. It doesn't mean that the maximum principle is satisfied on the discrete level, that's a different issue. Satisfying the discrete maximum principle is sufficient but not necessary for stability.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-05-13T11:21:45.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7180" PostTypeId="2" ParentId="7139" CreationDate="2013-05-13T16:09:33.540" Score="0" Body="&lt;p&gt;If you have the information of your function on all vertices, a representation with Functionspace of continous galerkin degree 1 is only exact at those vertices. Evaluating it at any other point gives the linear interpolation between the points. That's why you observe different values to 3.91 in between values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want that any evaluation within the sphere to be exactly equal to the assigned constant, you need the information of the function on each cell. Maybe by counting a cell to be inside the sphere, when all vertices are inside the sphere. Then you can represent the function with discontinous galerking degree 0. Any evaluation at any point gives than the expected value. &lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2013-05-13T16:09:33.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7181" PostTypeId="2" ParentId="7167" CreationDate="2013-05-13T16:32:56.377" Score="2" Body="&lt;p&gt;Well, &lt;strong&gt;lengths of the longest edge of a cell&lt;/strong&gt; &lt;code&gt;h&lt;/code&gt; can be computed pythonically - i.e. not efficiently&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h = Function(Q)&#xA;for c in cells(mesh):&#xA;    h.vector()[c.index()] = max(e.length() for e in edges(c))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For the clarification&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh.ufl_cell().max_facet_edge_length&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;defines facet diameter (length of the longest edge) which can be evaluated on facets.&#xA;For example interior penalty stabilization can be implemented this way&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h_F = mesh.ufl_cell().max_facet_edge_length&#xA;n = FacetNormal(mesh)&#xA;alpha = Constant(0.01)&#xA;F += alpha('+')*h_F**2*inner(jump(grad(u), n), jump(grad(v), n))*dS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;u&lt;/code&gt; being stabilize quantity, &lt;code&gt;v&lt;/code&gt; its test function and &lt;code&gt;alpha&lt;/code&gt; stabilization parameter here taken as constant. But may be for example scaled with convecting velocity for convection-dominated problems. See &lt;a href=&quot;http://dx.doi.org/10.1090/S0025-5718-07-01951-5&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1090/S0025-5718-07-01951-5&lt;/a&gt; and &lt;a href=&quot;http://dx.doi.org/10.1007/s00211-007-0070-5&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1007/s00211-007-0070-5&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-22T12:37:18.723" LastActivityDate="2013-05-22T12:37:18.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7182" PostTypeId="1" AcceptedAnswerId="7183" CreationDate="2013-05-13T17:26:03.613" Score="4" ViewCount="226" Body="&lt;p&gt;My heat equation is &#xA;$$&#xA;\frac{\partial u}{\partial t} = \frac{\partial^2 u}{\partial x^2}, \quad x \in [0,1], \quad t \in (0,0.1]&#xA;$$&#xA;with initial condition $u(x,0)=\sin(\pi x)$ and homogeneous Dirichlet boundary conditions $u(0,t) = u(1,t) = 0$. The exact solution is $u(x,t) = e^{-\pi^2t}\sin(\pi x)$. I approximate the equation with forward in time numerical scheme&#xA;$$&#xA;\frac{u_i^{n+1} - u_i^n}{\Delta t} = \frac{u_{i+1}^n - 2 u_i^n + u_{i-1}^n}{(\Delta x)^2}&#xA;$$&#xA;I would like to use boundary value technique to numerically solve this equation, i.e. i would like to write the following system of $MN$ unknowns in matrix form and solve it in Matlab. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I partition time as $0 = t_0 &amp;lt; t_1 &amp;lt; \dots &amp;lt; t_{N+1} = 0.1$. Step size is of length $h=\Delta t =\frac{0.1}{N+1}$, i.e. $t_n = t_0 + nh$, $n=1,2,\dots,N+1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I partition space as $0 = x_0 &amp;lt; x_1 &amp;lt; \dots &amp;lt; x_{M+1} = 1$. Step size is of length $h'=\Delta x =\frac{1}{M+1}$, i.e. $x_i = x_0 + ih'$, $i=1,2,\dots,M+1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system to solve is then&#xA;\begin{align*}&#xA;\frac{u_i^{n+1} - u_i^n}{\Delta t} &amp;amp;= \frac{u_{i+1}^n - 2 u_i^n + u_{i-1}^n}{(\Delta x)^2}, \quad i=1,2,\dots,M, \quad n=0,1,2,\dots,N \\&#xA; u_0^n &amp;amp;= u_M^n = 0, \quad n=0,1,\dots,N+1 \\&#xA; u_i^0 &amp;amp;= \sin((i\Delta x) x), \quad i=0,1,\dots,M+1&#xA;\end{align*}&#xA;I have big problems how to write this in matrix form. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="4234" LastActivityDate="2013-05-13T20:14:29.033" Title="Boundary value technique for heat equation" Tags="|pde|parabolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7183" PostTypeId="2" ParentId="7182" CreationDate="2013-05-13T18:16:25.847" Score="4" Body="&lt;p&gt;Let &#xA;$$A = \left(\begin{array}{ccccc} -2 &amp;amp; 1 \\ 1 &amp;amp; -2 &amp;amp; 1 \\ &amp;amp;&amp;amp;\ddots\\ &amp;amp;&amp;amp;1 &amp;amp; -2 &amp;amp; 1\\&amp;amp;&amp;amp;&amp;amp;1&amp;amp;-2\end{array}\right)$$&#xA;be the standard Laplacian matrix. Your system can be written in traditional time-stepping form as&#xA;$$u^{n+1} = \left(I + \frac{h}{(h')^2}A\right)u^n \doteq Mu^n,$$&#xA;where &#xA;$$u^n = \left(\begin{array}{c} u^n_1 \\ \vdots \\u^n_M\end{array}\right).$$&#xA;Now define&#xA;$$u = \left(\begin{array}{c} u^1 \\ \vdots \\ u^{N}\end{array}\right),$$&#xA;and&#xA;$$B = \left(\begin{array}{ccccc} I \\ -M &amp;amp; I \\ &amp;amp;-M &amp;amp;I\\&amp;amp;&amp;amp;&amp;amp;\ddots\\&amp;amp;&amp;amp;&amp;amp;-M &amp;amp;I\end{array}\right)$$&#xA;and&#xA;$$b = \left(\begin{array}{c} Mu^0 \\ 0 \\\vdots\\0\end{array}\right).$$&#xA;The solution for all time $u$ is then determined by the solution to the linear system&#xA;$$Bu = b.$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It's worth noting that this is a highly inefficient way to solve the heat equation. A naive approach to solving this system would require $\mathcal{O}(N^3M^3)$ computations. In fact, any method short of multigrid would require significantly more than $\mathcal{O}(NM)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, formulating the system with time-stepping results in $\mathcal{O}(NM)$ computations using the relatively naive approach of applying the stencil pointwise at each time step.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-05-13T20:14:29.033" LastActivityDate="2013-05-13T20:14:29.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7184" PostTypeId="2" ParentId="7125" CreationDate="2013-05-13T19:37:50.110" Score="-1" Body="&lt;p&gt;Given the complex multi-phase nature of your problem, and the need for particle tracking, you may consider a solver based on &lt;a href=&quot;http://www.palabos.org/software/lattice-boltzmann-method&quot; rel=&quot;nofollow&quot;&gt;the lattice Boltzmann method&lt;/a&gt;, which excels in both fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your list of open-source tools, you can add the lattice Boltzmann solver &lt;a href=&quot;http://www.palabos.org&quot; rel=&quot;nofollow&quot;&gt;Palabos&lt;/a&gt;. As you can see from the examples on the web page, it provides the models you are looking for.&lt;/p&gt;&#xA;" OwnerUserId="4294" LastEditorUserId="4294" LastEditDate="2013-05-13T19:47:53.440" LastActivityDate="2013-05-13T19:47:53.440" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7185" PostTypeId="2" ParentId="7088" CreationDate="2013-05-13T20:09:27.623" Score="1" Body="&lt;p&gt;You might try investigating the &quot;Method of Manufactured Solutions&quot;, which is a general methodology used to test implementation of codes which solve PDEs (it can be used to find both mathematical and coding errors). I imagine it could be adapted to work for solving ODEs, if your solution methodology is general enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://prod.sandia.gov/techlib/access-control.cgi/2000/001444.pdf&quot; rel=&quot;nofollow&quot;&gt;http://prod.sandia.gov/techlib/access-control.cgi/2000/001444.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7477" LastActivityDate="2013-05-13T20:09:27.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7186" PostTypeId="1" AcceptedAnswerId="7197" CreationDate="2013-05-13T20:57:23.853" Score="1" ViewCount="347" Body="&lt;p&gt;I am trying to model a flow past a cylinder using the projection method. I am a bit confused on how to define the circular cylinder in my computational domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the cylinder has the coordinates $(x-a)^2 + (y-b)^2 = r^2$&#xA;but what is the best way to define that in terms of x(i,j), y(i,j) coordinates such that I will not have any points inside the cylinder and so that I can easily apply boundary conditions to the cylinder in my computational domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to avoid doing a coordinates transformation. It seems that the NS eqtns seem to become very complicated.&lt;/p&gt;&#xA;" OwnerUserId="4134" LastActivityDate="2013-05-14T17:08:55.790" Title="How to define a cylinder surrounded by a clustered cartesian mesh" Tags="|fluid-dynamics|mesh|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7187" PostTypeId="2" ParentId="7186" CreationDate="2013-05-13T21:13:24.457" Score="2" Body="&lt;p&gt;It's really no different than if you wanted to solve the Laplace equation on such a domain -- you simply can't use points $x_i=i \; \Delta x, y_j=j \; \Delta y$. Instead, you need to have some cells in the vicinity of the cylinder that are distorted, and you need to treat those properly: either via a transformation when using the finite element method, or by changing your finite difference stencil.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-13T21:13:24.457" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7188" PostTypeId="1" CreationDate="2013-05-13T23:32:51.730" Score="4" ViewCount="1992" Body="&lt;p&gt;With the move away from launchpad I hope this is the right place for this question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to initialize the DOFs for a function? generated by &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;u  = Function(V)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When I run a code which I will put in at the end I get the fenics error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    *** Error:   Unable to initialize vector of degrees of freedom for function.&#xA;***Reason:  Cannot be created from subspace. Consider collapsing the function space. &#xA;****Where:   This error was encountered inside Function.cpp. &#xA;****Process: 0** &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is interesting about this error is that it does not occur when the mesh is generated by UnitCubeMesh() but when the same mesh is generated manually as bellow then it occurs (This is a example so that I didnt have to post my large main code, where I need to use mesh editor). It also runs fine on a single core but when using more than one the error occurs. Eg by &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;mpirun -n 4 python demo_hyperelasticity.py&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    from dolfin import *&#xA;&#xA;&#xA;mesh = Mesh();&#xA;editor = MeshEditor();&#xA;editor.open(mesh, 'tetrahedron', 3, 3)&#xA;editor.init_vertices(8)&#xA;editor.init_cells(6)&#xA;&#xA;editor.add_vertex(0, 0, 0, 0)&#xA;editor.add_vertex(1, 1, 0, 0)&#xA;editor.add_vertex(2, 0, 1, 0)&#xA;editor.add_vertex(3, 1, 1, 0)&#xA;editor.add_vertex(4, 0, 0, 1)&#xA;editor.add_vertex(5, 1, 0, 1)&#xA;editor.add_vertex(6, 0, 1, 1)&#xA;editor.add_vertex(7, 1, 1, 1)&#xA;&#xA;&#xA;editor.add_cell(0, 0, 1, 2, 4)&#xA;editor.add_cell(1, 1, 2, 3, 5)&#xA;editor.add_cell(2, 1, 2, 4, 5)&#xA;&#xA;editor.add_cell(3, 4, 5, 6, 2)&#xA;editor.add_cell(4, 5, 6, 7, 3)&#xA;editor.add_cell(5, 5, 6, 3, 2)&#xA;&#xA;&#xA;editor.close()&#xA;&#xA;#mesh = UnitCubeMesh(1, 1, 1)&#xA;&#xA;#plot(mesh1, interactive=True)&#xA;# Optimization options for the form compiler&#xA;parameters[&quot;form_compiler&quot;][&quot;cpp_optimize&quot;] = True&#xA;ffc_options = {&quot;optimize&quot;: True, \&#xA;               &quot;eliminate_zeros&quot;: True, \&#xA;               &quot;precompute_basis_const&quot;: True, \&#xA;               &quot;precompute_ip_const&quot;: True}&#xA;&#xA;# Create mesh and define function space&#xA;V = VectorFunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;&#xA;&#xA;u  = Function(V) ###### ERROR HERE#####&#xA;&#xA;def boundary_Z0(x):&#xA;    tol = 1E-15&#xA;    return x[2] &amp;lt; tol&#xA;u0 = Expression(('0','0','0'))&#xA;bc = DirichletBC(V, u0, boundary_Z0)&#xA;# Define functions&#xA;du = TrialFunction(V)            # Incremental displacement&#xA;v  = TestFunction(V)             # Test function&#xA;&#xA;B  = Constant((0.0, -0.5, 0.0))  # Body force per unit volume&#xA;T  = Constant((0.0,  0.0, 0.0))  # Traction force on the boundary&#xA;&#xA;# Kinematics&#xA;I = Identity(V.cell().d)    # Identity tensor&#xA;F = I + grad(u)             # Deformation gradient&#xA;C = F.T*F                   # Right Cauchy-Green tensor&#xA;&#xA;# Invariants of deformation tensors&#xA;Ic = tr(C)&#xA;J  = det(F)&#xA;&#xA;# Elasticity parameters&#xA;E, nu = 10.0, 0.3&#xA;mu, lmbda = Constant(E/(2*(1 + nu))), Constant(E*nu/((1 + nu)*(1 - 2*nu)))&#xA;&#xA;# Stored strain energy density (compressible neo-Hookean model)&#xA;psi = (mu/2)*(Ic - 3) - mu*ln(J) + (lmbda/2)*(ln(J))**2&#xA;&#xA;# Total potential energy&#xA;Pi = psi*dx - dot(B, u)*dx - dot(T, u)*ds&#xA;&#xA;# Compute first variation of Pi (directional derivative about u in the direction of v)&#xA;F = derivative(Pi, u, v)&#xA;&#xA;# Compute Jacobian of F&#xA;J = derivative(F, u, du)&#xA;&#xA;# Solve variational problem&#xA;solve(F == 0, u, bc, J=J,&#xA;      form_compiler_parameters=ffc_options)&#xA;&#xA;# Plot and hold solution&#xA;plot(u, mode = &quot;displacement&quot;, interactive = True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any ideas as to how to get this to work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks For any help&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&#xA;Thanks for your help Garth&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that sorted today I have been trying to get the next bit of my code to run in parallel. What this block does is takes coordinates from the mesh, deforms them by the u vector then recombines them into a new mesh (it adds some more elements as well). It works fine in serial but I cant work out how to link the thread specific dofs and interpolated coordinates back into a single vector to manually create the new mesh. Is this even doable, or is there a much better way of updating mesh coordinates in parallel?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coor_int = interpolate(Expression((&quot;x[0]&quot;, &quot;x[1]&quot;, &quot;x[2]&quot;), value_shape=3), V).vector().array()&#xA;    #turing displacement into numpy array&#xA;u_vec = u.vector().array()&#xA;    #calc new coordinates by subtaracting the displacement at each interploated&#xA;    #node from the oringonal mesh&#xA;new_coor = coor_int+u_vec&#xA;&#xA;    #initalising matries&#xA;dofs_to_vert = np.zeros(num_vert, dtype=np.uintp)&#xA;vectordofs_to_vert = np.zeros((num_vert*3), dtype=np.uintp)&#xA;vectordofs_to_subvert = np.zeros((num_vert*3),dtype=np.uintp)&#xA;cellinds = np.zeros((num_cells,4), dtype=np.uintp)&#xA;    #creating degrees of freedom maps&#xA;dm = VV.dofmap()&#xA;dms = [V.sub(i).dofmap() for i in range(3)]&#xA;&#xA;for cell in cells(mesh):&#xA;    cell_ind = cell.index()&#xA;    vert_inds = cell.entities(0)&#xA;    cellinds[cell_ind,:] = vert_inds #retuns the local indacies for each cell&#xA;    dofs_to_vert[dm.cell_dofs(cell_ind)] = vert_inds #dofs to vertcies map&#xA;    for i, (dms_i, dmcs_i) in enumerate(zip(dms, dms)):&#xA;        vectordofs_to_vert[dms_i.cell_dofs(cell_ind)] = vert_inds #gives map for coords to cells, not sencitive to xyz position&#xA;        vectordofs_to_subvert[dms_i.cell_dofs(cell_ind)] = i #gives map for each x,y,z to to particular cells for map above&#xA;#            #initalise more arrays       &#xA;map_mat = np.zeros((3*num_vert,3), dtype=float)&#xA;coorxyz = np.zeros((num_vert,3), dtype=float)&#xA;#    #make matrix of maps above&#xA;map_mat[:,0] = vectordofs_to_vert&#xA;map_mat[:,1] = vectordofs_to_subvert&#xA;map_mat[:,2] = new_coor&#xA;&#xA;    #sort entries using maps matrix above&#xA;for ij in range(0,len(map_mat)):&#xA;            if map_mat[ij,1] == 0:&#xA;                coorxyz[map_mat[ij,0],0] = map_mat[ij,2]&#xA;            if map_mat[ij,1] == 1:&#xA;                coorxyz[map_mat[ij,0],1] = map_mat[ij,2]&#xA;            if map_mat[ij,1] == 2:&#xA;                coorxyz[map_mat[ij,0],2] = map_mat[ij,2] &#xA;&#xA;&#xA;mesh1 = Mesh()&#xA;&#xA;me = MeshEditor()&#xA;me.open(mesh1,'tetrahedron', 3, 3)&#xA;me.init_vertices(10)&#xA;me.init_cells(9)&#xA;&#xA;for ii in range(0, len(coorxyz)):&#xA;    me.add_vertex(ii, coorxyz[ii,:])&#xA;for jj in range(0, len(cellinds)):&#xA;    me.add_cell(jj, cellinds[jj,:])&#xA;&#xA;me.add_vertex(8, 2, 1, 0)&#xA;me.add_vertex(9, 2, 0, 0)&#xA;&#xA;me.add_cell(6, 1, 3, 8, 5)&#xA;me.add_cell(7, 3, 8, 7, 5)&#xA;me.add_cell(8, 1, 5, 9, 8)&#xA;&#xA;me.close()&#xA;&#xA;plot(mesh1, interactive=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again thanks for your help, hopefully this is trivial as well. Sorry about the questions, this is the first time I have ever done anything in parallel. &lt;/p&gt;&#xA;" OwnerUserId="4296" LastEditorUserId="4296" LastEditDate="2013-05-15T05:25:52.893" LastActivityDate="2014-10-27T17:32:30.490" Title="Fenics, initialize vector of degrees of freedom for function" Tags="|python|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7189" PostTypeId="1" CreationDate="2013-05-14T01:44:04.397" Score="10" ViewCount="894" Body="&lt;p&gt;I am looking to port some code that resolves a set of partial differential equations (PDE) by the finite volume method in IMPLICIT form (for the time discretization).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As result there is a tridiagonal system of equations in x,y,z directions which is handled by the ADI/TDMA scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can not seem to find anything regarding implicit solution of PDEs with CUDA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the ADI/TDMA scheme possible to implement in CUDA?? Is there an example like 2D heat diffusion equation available somewhere??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All I could find is a CUDA sample code for 2D heat diffusion equation in finite differences but in EXPLICIT form (University of Cambridge).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hint/reference would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4298" LastActivityDate="2016-02-15T23:33:12.187" Title="cuda and numerical methods with implicit time discretization" Tags="|parallel-computing|implicit-methods|cuda|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7190" PostTypeId="2" ParentId="7088" CreationDate="2013-05-14T02:10:09.600" Score="2" Body="&lt;p&gt;One sanity check I run against my ODE solvers is to simply check it on smaller linear systems via exactly computing the exponential of the system's matrix. i.e. given&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d\mathbf{u}}{dt} = \mathbf{Au} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;check the error in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\exp{(t\mathbf{A})}\mathbf{u}_0 - \hat{\mathbf{u}}(t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\hat{\mathbf{u}}(t)$ is your calculated solution. This way you don't need to know any real analytic solution (assuming you have some equivalent of 'expm'), but you can still get an idea of whether your solver is doing what it should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just don't calculate the exponential with one of your time-steppers (i.e. dubious method number 6 :) &lt;a href=&quot;http://www.cs.cornell.edu/cv/researchpdf/19ways+.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.cornell.edu/cv/researchpdf/19ways+.pdf&lt;/a&gt; )&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-05-14T02:10:09.600" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7191" PostTypeId="1" AcceptedAnswerId="43696" CreationDate="2013-05-14T03:40:48.280" Score="5" ViewCount="578" Body="&lt;p&gt;I would like to compute the decomposition of a real symmetric positive definite matrix $\mathbf{A} = \mathbf{UDU}^\top$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LINPACK seems to have it as &lt;a href=&quot;http://www.netlib.org/linpack/dsifa.f&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;DSIFA&lt;/code&gt;&lt;/a&gt;, but I cannot find an equivalent routine in LAPACK.  It also doesn't appear to implemented in Eigen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other packages and routines support this decomposition?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my field, the most common of Kalman Filter implemented in the &quot;real world&quot; is via &lt;a href=&quot;http://en.wikipedia.org/wiki/Kalman_filter#Square_root_form&quot; rel=&quot;nofollow&quot;&gt;UDU&lt;/a&gt; as evidenced in &lt;a href=&quot;http://books.google.com.au/books/about/Global_Positioning_Systems_Inertial_Navi.html&quot; rel=&quot;nofollow&quot;&gt;multiple&lt;/a&gt; &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0071493298&quot; rel=&quot;nofollow&quot;&gt;books&lt;/a&gt; &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1563479273&quot; rel=&quot;nofollow&quot;&gt;on the subject.&lt;/a&gt;   I understand that there is an equivalent $\mathbf{LDL}^\top$ form, but I am trying to stick with the convention in my field.&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="127" LastEditDate="2013-05-14T04:07:31.497" LastActivityDate="2024-01-13T16:12:08.507" Title="$\mathbf{UDU}^\top$ decomposition routines in LAPACK/Eigen?" Tags="|linear-algebra|matrix|lapack|eigen|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7193" PostTypeId="2" ParentId="4757" CreationDate="2013-05-14T05:47:27.963" Score="2" Body="&lt;p&gt;The procedure I prefer, which I have also seen used by some FFT implementations, also involves a three-term recurrence, but is a bit more stable than the computation of $\cos\,kx$ and $\sin\,kx$ from previous predecessors. (There is a nice analysis of these recurrences in &lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;id=1oDXWLb9qEkC&amp;amp;pg=PA22&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bulirsch/Stoer&lt;/a&gt;; see examples 2 and 3. A geometric way of looking at the instability is that although you trace out an approximate circle at first, in the long run you literally spiral out of control.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recurrence (effectively the same as example 4 in Bulirsch/Stoer, see that for a detailed analysis) proceeds like so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align*}&#xA;\cos(\theta+\varepsilon)&amp;amp;=\cos\,\theta-(p\cos\,\theta+q\sin\,\theta)\\&#xA;\sin(\theta+\varepsilon)&amp;amp;=\sin\,\theta-(p\sin\,\theta-q\cos\,\theta)&#xA;\end{align*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we have the precomputed constants&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p=2\sin^2\frac{\varepsilon}{2},\qquad q=\sin\,\varepsilon$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As noted, they perform exceptionally well, especially in the case where $\varepsilon$ is tiny.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="127" LastEditDate="2017-09-01T15:17:22.803" LastActivityDate="2017-09-01T15:17:22.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7195" PostTypeId="2" ParentId="7188" CreationDate="2013-05-14T08:06:21.580" Score="6" Body="&lt;p&gt;Your sample code will not work in parallel because you are creating the same vertices and cells on all processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you add mesh entities only on process 0 and then call &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MeshPartitioning.build_distributed_mesh(mesh)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;on all processes, your sample code runs fine.&lt;/p&gt;&#xA;" OwnerUserId="3952" LastActivityDate="2013-05-14T08:06:21.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7196" PostTypeId="1" AcceptedAnswerId="7198" CreationDate="2013-05-14T08:31:15.563" Score="6" ViewCount="1858" Body="&lt;p&gt;I need to solve a real generalized eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax= \lambda Bx(*)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A and B are calculated from equations below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A=\sum_{i,j=1}^{N}W_{ij}(K_{i}-K_{j})\beta\beta^{T}(K_{i}-K_{j})^{T}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B=\sum_{i=1}^{N}D_{ii}K_{i}\beta\beta^{T}K_{i}^{T}$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $W$ is a real symmetric $N*N$ matrix with diagonal entries being $0$ and off-diagonal entries between $(0,1)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D$ is an $N*N$ diagonal matrix with $D_{ii}=\sum_{j=1}^NW_{ij}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K_i$ is an $N*M$ matrix with all entries positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\beta&amp;gt;0$ is an $M$ dimensional column vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From above equations, A and B should be symmetric semi-definite and B should be positive definite(I did some proof myself).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe because some numerical losses( I are not sure :( ), $B$ appears to have small negative eigenvalues( I do the eigenvalue decomposition using LAPACK routine &lt;a href=&quot;http://www.netlib.org/lapack/double/dsyev.f&quot;&gt;dsyev()&lt;/a&gt; ) and $(*)$ gives complex eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to select P smallest eigenvalues of this generalized eigenvalue problem, so complex values here are really a problem. Is there any way to avoid complex eigenvalues in such a case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way I used armadillo as linear algebra library and solve $(*)$ directly using LAPACK routine &lt;a href=&quot;http://www.netlib.org/lapack/double/dggev.f&quot;&gt;dggev()&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2013-05-15T09:01:19.590" Title="generalized eigenvalue problem" Tags="|linear-algebra|eigensystem|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7197" PostTypeId="2" ParentId="7186" CreationDate="2013-05-14T09:23:57.550" Score="5" Body="&lt;p&gt;There is no continuous mapping from a domain with a hole (the tube with the cylinder in it) to a rectangle as in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ucCZu.png&quot; alt=&quot;continuous mapping between rectangle and distorted rectangle&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even for much simpler domains (without holes), like unions of rectangles, one typically would not try and map the domain to a rectangle to then employ a finite-difference approach. This only moves the complication from finding a proper grid to finding a proper mapping, and those mappings will typically massively distort your PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical approach for nonrectangular domains would be to use a discretization that is more flexible when it comes to arbitrarily shaped domains; the most popular options are certainly the &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_element_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite element method&lt;/a&gt; and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_volume_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite volume method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the most default form of the 2D FEM, you would create a triangular mesh of the domain and then approximate the solution by a function that is piecewise linear on those triangles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FQY5N.png&quot; alt=&quot;FEM discretization of the Karman vortex street&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-05-14T17:08:55.790" LastActivityDate="2013-05-14T17:08:55.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7198" PostTypeId="2" ParentId="7196" CreationDate="2013-05-14T11:32:29.810" Score="5" Body="&lt;p&gt;If, as you say, you are sure that you have a symmetric-definite pencil (that is, $\mathbf A$ is symmetric, and $\mathbf B$ is symmetric positive-definite), then LAPACK already has something for directly handling your problem: &lt;a href=&quot;http://www.netlib.org/lapack/double/dsygv.f&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;dsygv()&lt;/code&gt;&lt;/a&gt;. What it does is to perform a Cholesky decomposition of $\mathbf B$ (if in fact your $\mathbf B$ is not symmetric positive-definite, then you should see a warning), after which the Cholesky triangle thus produced is used to convert your generalized eigenproblem into a regular symmetric eigenproblem that can be solved with all the usual methods. Since this method requires the inversion of the Cholesky triangle of $\mathbf B$, you'll probably want to check if $\mathbf B$ is well-conditioned; you can use &lt;a href=&quot;http://www.netlib.org/lapack/double/dsycon.f&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;dsycon()&lt;/code&gt;&lt;/a&gt; for the purpose.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There is an alternative method based on the eigendecomposition of $\mathbf B$ if the Cholesky route fails, also discussed in Golub and Van Loan. Briefly, the procedure proceeds like so: give the eigendecomposition $\mathbf B=\mathbf V\mathbf D\mathbf V^\top$, form the matrix $\mathbf W=\mathbf V\sqrt{\mathbf D}$, where $\sqrt{\mathbf D}$ is done by taking the square roots of the diagonal elements. Having formed $\mathbf W$, form $\mathbf C=\mathbf W^{-1}\mathbf A\mathbf W^{-\top}$, which has the same eigenvalues as the pencil $(\mathbf A,\mathbf B)$. (I'll leave the procedure of how to form the eigenvectors as an exercise.)&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="127" LastEditDate="2013-05-15T09:01:19.590" LastActivityDate="2013-05-15T09:01:19.590" CommentCount="26" ContentLicense="CC BY-SA 3.0" />
  <row Id="7199" PostTypeId="1" CreationDate="2013-05-14T12:52:07.507" Score="5" ViewCount="282" Body="&lt;p&gt;I'm using Eigen (a C++ library for numerical linear algebra) to solve a linear equation with the the bi-conjugate gradient BICGSTAB algorithm with Incomplete LU preconditioner. However, the result &lt;code&gt;bg.error()(&quot;estimated error&quot;)&lt;/code&gt; becomes &lt;code&gt;-1.#IND&lt;/code&gt; after 1 iteration. The size of the linear system is $N=20000$. Could anyone help me about this problem? I also tried smaller system, e.g, $N=200$, and that worked OK.&lt;/p&gt;&#xA;" OwnerUserId="5069" LastEditorUserId="1748" LastEditDate="2013-05-14T15:23:33.420" LastActivityDate="2018-03-20T17:11:24.850" Title="Why do I get “estimated error” -1.#IND when doing BICGSTAB linear solver using ILUT perconditioner in eigen" Tags="|linear-solver|numerics|eigen|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7200" PostTypeId="1" AcceptedAnswerId="7205" CreationDate="2013-05-14T13:52:15.183" Score="3" ViewCount="1161" Body="&lt;p&gt;I am trying to implementing a nodal discontinuous Galerkin spectral element method for linear and non-linear systems of equations. The solution at each time step is given at &lt;code&gt;N&lt;/code&gt; nodes, which are located at the integration points for Legendre-Gauss/Legendre-Gauss-Lobatto quadrature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For visualization purposes I would like to get the solution at &lt;code&gt;M&lt;/code&gt; equidistant visualization nodes, where typically &lt;code&gt;M &amp;gt; N&lt;/code&gt;. As far as I know, generally L2 projection is preferred to polynomial interpolation (albeit being more expensive), especially when going to higher degrees of freedom, since the projection gives the best approximation w.r.t. the L2 norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand it, this is a rough outline of the steps I have to take:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Assemble the Vandermonde matrix &lt;code&gt;V&lt;/code&gt; for the old nodes.&lt;/li&gt;&#xA;&lt;li&gt;Invert &lt;code&gt;V&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;code&gt;V&lt;/code&gt; and the nodal values to get modal coefficients.&lt;/li&gt;&#xA;&lt;li&gt;Project to new nodes.&lt;/li&gt;&#xA;&lt;li&gt;Assemble Vandermonde matrix &lt;code&gt;V'&lt;/code&gt; for the new nodes.&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;code&gt;V'&lt;/code&gt; and the modal coefficients to get the nodal values.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Finally, my question(s):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a) Is this approach generally correct?&lt;/li&gt;&#xA;&lt;li&gt;b) Are there algorithmic simplifications for my specific problem, e.g. &#xA;&lt;ul&gt;&#xA;&lt;li&gt;the inversion of &lt;code&gt;V&lt;/code&gt; is trivial because I can use some properties of the used polynomials (Legendre) or nodes (L-Gauss/L-Gauss-Lobatto)?&lt;/li&gt;&#xA;&lt;li&gt;the whole algorithm can be simplified because I already start out with a polynomial approximation and not an arbitrary exact function?&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any suggestions or pointers to further reading material are highly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Lagrangian basis functions for my computations, the GL/LGL nodes only come in for the purpose of numerical integration. So at the point where I want to get the solution at the new nodes, I do not have any information on the solution but the values at the old nodes.&lt;/p&gt;&#xA;" OwnerUserId="1724" LastEditorUserId="1724" LastEditDate="2013-05-14T18:39:17.953" LastActivityDate="2013-05-14T20:21:10.603" Title="How to properly use polynomial projection to get values at visualization nodes?" Tags="|interpolation|polynomials|discontinuous-galerkin|projection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7202" PostTypeId="1" CreationDate="2013-05-14T13:18:08.397" Score="5" ViewCount="352" Body="&lt;p&gt;I'm working on a time integration scheme for my research. As a result, I have come across an interesting phenomenon. Somehow, the total energy of the scheme oscillates. At any given time the total energy of the system could be increasing or decreasing but does so in a oscillatory fashion. Interestingly enough, for what I'm modeling, this integration scheme works great. I'm curious if there is a way to prove whether the energy of this scheme is stable throughout the time history.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYI, I'm a Ph.D. student in structural engineering. So, this topic is a little out of my field. I'm just curious if the scheme is stable or not, and if it can be proven. Are their other time integration schemes that don't satisfy the conservation of energy but remain stable? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="15207" OwnerDisplayName="Grady F. Mathews Iv" LastEditorUserId="127" LastEditDate="2013-05-14T15:50:45.903" LastActivityDate="2013-05-14T15:50:45.903" Title="Energy Conservation" Tags="|pde|time-integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7203" PostTypeId="2" ParentId="7202" CreationDate="2013-05-14T13:25:49.533" Score="4" Body="&lt;p&gt;Integration schemes that conserve energy are a research field of their own. They are called &lt;em&gt;symplectic&lt;/em&gt; or &lt;em&gt;symplectic integrator&lt;/em&gt;. When you google &lt;em&gt;symplectic ingegrator&lt;/em&gt;, you will find a lot of search hits. Unfortunately many of them are quite mathematical. Others are not mathematical at all, but may lack the necessary depth to help you with what you are doing. I think the one in &lt;a href=&quot;http://www.unige.ch/~hairer/poly_geoint/week2.pdf&quot; rel=&quot;nofollow&quot;&gt;this link&lt;/a&gt; is rather ok. There is also a &lt;a href=&quot;http://en.wikipedia.org/wiki/Symplectic_integrator&quot; rel=&quot;nofollow&quot;&gt;Wikipedia entry on symplectic integrators&lt;/a&gt; that links to further references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in order to actually help you, you would have to describe what you are doing and also describe the algorithm you are using. I think this question would be a better match for scientific computing stack exchange. I'll send a flag to a moderator.&lt;/p&gt;&#xA;" OwnerUserId="3748" OwnerDisplayName="sebastian" LastActivityDate="2013-05-14T13:25:49.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7204" PostTypeId="1" AcceptedAnswerId="7220" CreationDate="2013-05-14T16:25:59.833" Score="11" ViewCount="3811" Body="&lt;p&gt;I have a physical problem governed by the Poisson equation in two dimensions&#xA;$$&#xA; -\nabla^2 u = f(x,y), \; in \; \Omega&#xA;$$&#xA;I have measurements of the two gradient components $\partial{u}/\partial{x}$ and $\partial{u}/\partial{y}$ along some part of the boundary, $\Gamma_m$, so would like to impose&#xA;$$&#xA;\frac{\partial{u}}{\partial{x_i}}_0 = g_m, \; on \; \Gamma_m&#xA;$$&#xA;and propagate into the far field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tangential gradient component, $\frac{\partial{u}}{\partial{x}}_{(t,0)}$, I can just integrate and then enforce through a Dirichlet condition, such that&#xA;$$&#xA;\int_{\Gamma_m}\frac{\partial{u}}{\partial{x}}_{(t,0)} \, ds = u_0&#xA;$$&#xA;In order to simultaneously impose the normal component, $\frac{\partial{u}}{\partial{x}}_{(n,0)}$, I gathered I would have to go via Lagrange multipliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think the variational form is then&#xA;$$&#xA;\int_\Omega \nabla{u} \cdot\nabla{v}\, dx - \lambda \int_{\Gamma_m} ( \frac{\partial{u}}{\partial{x}}_{(n,0)}-g_m ) v \,ds = \int_\Omega f\, v\, dx&#xA;$$&#xA;I spent a long time trying to piece it together from the information on related problems such as&#xA;&lt;a href=&quot;https://answers.launchpad.net/fenics/+question/212434&quot;&gt;https://answers.launchpad.net/fenics/+question/212434&lt;/a&gt;&#xA;&lt;a href=&quot;https://answers.launchpad.net/fenics/+question/216323&quot;&gt;https://answers.launchpad.net/fenics/+question/216323&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but still cannot see where I am going wrong. My solution attempt so far is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;&#xA;# Create mesh and define function space&#xA;mesh = UnitSquareMesh(64, 64)&#xA;V = FunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;R = FunctionSpace(mesh, &quot;R&quot;, 0)&#xA;W = V * R&#xA;&#xA;# Create mesh function over cell facets&#xA;boundary_parts = MeshFunction(&quot;uint&quot;, mesh, mesh.topology().dim()-1)&#xA;&#xA;# Mark left boundary facets as subdomain 0&#xA;class LeftBoundary(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        return on_boundary and x[0] &amp;lt; DOLFIN_EPS&#xA;&#xA;Gamma_Left = LeftBoundary()&#xA;Gamma_Left.mark(boundary_parts, 0)&#xA;&#xA;class FarField(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        return on_boundary and ( (x[0] &amp;gt; 1.0-DOLFIN_EPS) \&#xA;               or (x[1]&amp;lt;DOLFIN_EPS) or (x[1]&amp;gt; 1.0-DOLFIN_EPS) )&#xA;&#xA;Gamma_FF = FarField()&#xA;Gamma_FF.mark(boundary_parts, 1)&#xA;&#xA;# Define boundary condition&#xA;u0 = Expression(&quot;sin(x[1]*pi)&quot;)&#xA;bcs = [DirichletBC(V, u0, Gamma_Left)]&#xA;&#xA;# Define variational problem&#xA;(u, lmbd) = TrialFunctions(W)&#xA;(v, d) = TestFunctions(W)&#xA;&#xA;f = Expression(&quot;10*exp(-(pow(x[0] - 0.5, 2) + pow(x[1] - 0.5, 2)) / 0.02)&quot;)&#xA;g = Constant(0.0)&#xA;h = Constant(-4.0)&#xA;n = FacetNormal(mesh)&#xA;&#xA;F = inner(grad(u), grad(v))*dx + d*dot(grad(u),n)*ds(0) + lmbd*dot(grad(v),n)*ds(0)-\&#xA;   (f*v*dx + g*v*ds(1) + h*d*ds(0) + lmbd*h*ds(0))&#xA;&#xA;a = lhs(F)&#xA;L = rhs(F)&#xA;&#xA;# Compute solution&#xA;A = assemble(a, exterior_facet_domains=boundary_parts)&#xA;b = assemble(L, exterior_facet_domains=boundary_parts)&#xA;for bc in bcs: bc.apply(A, b)&#xA;&#xA;w = Function(W)&#xA;solve(A, w.vector(), b, 'lu')&#xA;(u,lmbd) = w.split()&#xA;&#xA;# Plot solution&#xA;plot(u, interactive=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which runs but gives a noisy result not at all resembling a solution to a Poisson equation.&#xA;It seems to have something to do with the combined function spaces, but I cannot find the mistake.&lt;br&gt;&#xA;I would appreciate any help or pointers in the right direction - many thanks already!&lt;br&gt;&#xA;Cheers&lt;br&gt;&#xA;Markus&lt;/p&gt;&#xA;" OwnerUserId="4306" LastActivityDate="2014-04-23T11:42:54.710" Title="Poisson equation: Impose full gradient as boundary condition via Lagrange multipliers" Tags="|finite-element|python|fenics|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7205" PostTypeId="2" ParentId="7200" CreationDate="2013-05-14T17:00:33.020" Score="2" Body="&lt;p&gt;You want to interpolate here, not perform a $L^2$ projection. You want the value of the &lt;em&gt;DG solution&lt;/em&gt; at equally spaced nodes. In any case a $L^2$ projection will just feed you back the same polynomial you started with (remember, it minimizes $L^2$ error.. if you're already a polynomial, then you can't do any better).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the matrices you'll need to compute&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Vandermonde for LGL nodes: $\mathbf{V}$&lt;/li&gt;&#xA;&lt;li&gt;Vandermonde for desired equally spaced noes: $\mathbf{V}_{eq}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then given a solution vector $\mathbf{u}$ the interpolation will look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{u}_{eq} = \mathbf{V}_{eq} \mathbf{V}^{-1} \mathbf{u} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this assumes that you are not directly using the Lagrange polynomial basis for computations, but instead an orthogonal basis like Legendre polynomials, so your Vandermondes will be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{V}_{ij} = P_j(x_i)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in whatever basis you're using for the computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Reading that over again It isn't clear whether you're trying to visualize a computed DG solution or the analytic solution (or initial condition). In the latter case, still use interpolation. It's much easier, and you won't notice the accuracy difference in a visualization. I'd keep it to the LGL nodes though and then sample the interpolating polynomial for visualization purposes.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2013-05-14T17:07:02.880" LastActivityDate="2013-05-14T17:07:02.880" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7206" PostTypeId="1" AcceptedAnswerId="7212" CreationDate="2013-05-14T17:05:27.493" Score="8" ViewCount="146" Body="&lt;p&gt;In my courses on numerical analysis, I have been taught that the main and principal motivation for preconditioning linear systems of equations is to increase the convergence rate of iterative solvers for that LSE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, is there any effect on the precision of the computed solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can remember a result on the precision of computed solution of Gaussian elimination, which can be found in &lt;em&gt;Matrix Computations&lt;/em&gt; by Golub and Van Loan (p. 122). The condition number (with respect to some particular norm) does indeed affect the precision of the numerical solution computed by that algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might expect that something similar holds for solutions obtained by, e.g., Conjugate Gradients. I think that I have observed this in a computational experiment. When I had the Conjugate gradient method run on an unpreconditioned system for a (long) time until some stopping criterion was met, the computed solution still displayed a high residual. &lt;strong&gt;So I wonder whether lower condition numbers not only lead to lower run-times, but also to a lower residual (or error) in the computed solution.&lt;/strong&gt; Note that this is necessarily a question of numerical stability, which requires that we work in inprecise arithmetics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I have asked the same question on math.SE, but I think that this site might be more appropriate.)&lt;/p&gt;&#xA;" OwnerUserId="238" LastEditorUserId="238" LastEditDate="2013-05-14T19:28:26.337" LastActivityDate="2013-05-15T01:10:47.867" Title="Preconditioning and effects on precision of solution of LSE" Tags="|linear-algebra|condition-number|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7207" PostTypeId="2" ParentId="7204" CreationDate="2013-05-14T17:24:13.977" Score="4" Body="&lt;p&gt;&lt;strong&gt;You can't expect that solution to your altered problem would be a solution to Poisson problem&lt;/strong&gt; because you need to change the problem somehow to make it well-posed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could guess that possible problem formulation is to minimize &#xA;$$F(u, \lambda) = \int_\Omega \frac{1}{2}|\nabla u|^2 \;\mathrm{d}x - \int_\Omega f u \;\mathrm{d}x - \int_{\Gamma_\mathrm{N}} g u \;\mathrm{d}S + \int_{\Gamma_\mathrm{N}} \lambda (u-u_\mathrm{D})\;\mathrm{d}S$$&#xA;over $(u,\lambda) \in V\times L^2(\Gamma_\mathrm{N})$ where $V = \{v\in H^1;v|_{\Gamma_\mathrm{D}}=0\}$ is space with zero trace on Dirichlet boundary $\Gamma_\mathrm{D}$ and $u_\mathrm{D}$ is Dirichlet value you would like to enforce additionaly on Neumann boundary $\Gamma_\mathrm{N}$. &#xA;Without the last term minimization of $F(u)$ yields&#xA;$$ 0 = \mathrm{D}F(u)[v] = \int_\Omega \nabla u\cdot\nabla v\;\mathrm{d}x - \int_\Omega fv\;\mathrm{d}x - \int_{\Gamma_\mathrm{N}} gv\;\mathrm{d}S \quad \forall v\in V,$$&#xA;which is exactly Poisson problem with Neumann condition on ${\Gamma_\mathrm{N}}$ and zero (it is simply adjusted to non-zero) Dirichlet condition on ${\Gamma_\mathrm{D}}$. With additional Lagrange multiplier term necessary condition of minimality reads&#xA;$$ 0 = \mathrm{D}F(u,\lambda)[v,\mu] = \int_\Omega \nabla u\cdot\nabla v\;\mathrm{d}x - \int_\Omega fv\;\mathrm{d}x - \int_{\Gamma_\mathrm{N}} gv\;\mathrm{d}S \quad  + \int_{\Gamma_\mathrm{N}} \lambda v\;\mathrm{d}S + \int_{\Gamma_\mathrm{N}} \mu (u-u_\mathrm{D})\;\mathrm{d}S \quad \forall (v,\mu)\in V\times L^2(\Gamma_\mathrm{N}),$$&#xA;so we see that this modified problem admits Poisson problem $-\Delta u=f$ but with modified Neumann condition $\frac{\partial u}{\partial\mathbf{n}} = g-\lambda$ on $\Gamma_\mathrm{N}$. Simply said - requiring additional Dirichlet condition on $\Gamma_\mathrm{N}$ changed original Neumann condition required there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you solve this and $\lambda\ll|g|$ holds then maybe your over-specified conditions are consistent with Poisson problem. I say 'maybe' because of stability issue @ChristianClason mentioned in his answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely we could try to enforce additional Neumann condition as a constraint (via Lagrange multiplier) on Dirichlet boundary $\Gamma_\mathrm{D}$. But that would not work either as test function $v\in V$ is zero on $\Gamma_\mathrm{D}$ which rules out constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion is you that you can't expect that second order PDE will admit two independent boundary conditions.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As this question relates to FEniCS, note that it easy in FEniCS to specify $F(u,\lambda)$ and let UFL calculate its Gateaux derivative $\mathrm{D}F(u,\lambda)[v,\mu]$ by &lt;code&gt;derivative()&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately $F(u,\lambda)$ can't be easily defined in FEniCS yet, as library does not support function $\lambda\in L^2(\Gamma_\mathrm{N})$ restricted to boundary. But another minimizations problems with Lagrange multipliers $\lambda\in L^2(\Omega)$ or $\lambda\in\mathbb{R}$ can be easily implemented this way.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-15T22:29:32.997" LastActivityDate="2013-05-15T22:29:32.997" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7208" PostTypeId="2" ParentId="5682" CreationDate="2013-05-14T17:24:22.157" Score="22" Body="&lt;p&gt;This is a really interesting question, and there are a lot of possible explanations.  If we are attempting to use a polynomial interpolation, then note that polynomial satisfy the following &lt;em&gt;annoying&lt;/em&gt; inequality&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a polynomial $P$ of degree not exceeding $N$ we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ |P^{\prime}(x)| \leq \frac{N}{\sqrt{1-x^2}}\max _x |P(x) | $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for every $x \in (-1,1)$. This is known as &lt;em&gt;Bernstein's inequality&lt;/em&gt;, note the singularity in this inequality. This can be bounded by the &lt;em&gt;Markov inequality&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max _x |P^{\prime} (x) | \leq N^2 \max _x |P(x) | $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and note that this is &lt;em&gt;sharp&lt;/em&gt; in the sense that Chebysehv polynomials make this an equation. So in other words we have the following combined bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ |P^{\prime}(x)| \leq \min \left(\frac{N}{\sqrt{1-x^2}},N^2\right)\max _x |P(x) |$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this means: Gradients of polynomials grow linearly in their order everywhere except in small neighborhoods of the interval boundaries. At the boundaries they grow more like $N^2$. It is no accident that stable interpolation nodes all have a $1/{N^2}$ clustering near boundaries. The clustering is necessary to control the gradients of the basis, whereas near the midpoint one can be a bit more relaxed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turns out however that this is not necessarily a polynomial phenomena, I suggest the following paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://math.la.asu.edu/~platte/pub/prevised.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://math.la.asu.edu/~platte/pub/prevised.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says loosely: If you have the same approximation power of the polynomial basis, then you can't use equally spaced points in a stable way.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2015-10-22T21:56:22.193" LastActivityDate="2015-10-22T21:56:22.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7209" PostTypeId="2" ParentId="7204" CreationDate="2013-05-14T17:35:12.733" Score="3" Body="&lt;p&gt;Your approach cannot work, definitely because of the implementation and probably because of your formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imposing Dirichlet conditions in dolfin, eventually sets the corresponding DOFs of your testspace to zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an excerpt from the &lt;a href=&quot;http://fenicsproject.org/documentation/&quot; rel=&quot;nofollow&quot;&gt;fenics-manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Chapter 3.3.9 (end): The application of a Dirichlet boundary condition&#xA;  to a linear system will identify all degrees of freedom that should be&#xA;  set to the given value and modify the linear system such that its&#xA;  solution respects the boundary condition. This is accomplished by&#xA;  zeroing and inserting 1 on the diagonal of the rows of the matrix&#xA;  corresponding to Dirichlet values, and inserting the Dirichlet value&#xA;  in the corresponding entry of the right-hand side vector [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thus, your $v$ will be set to zero on $\Gamma_m$, what makes also your term with the multiplier disappear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, using the default routine in dolfin you cannot apply both Dirichlet and other conditions on the same boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, before you try to fix this in your implementation, go find the right test spaces for your mathematical formulation (as @Jan Blechta just has mentioned.)&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-15T14:19:21.150" LastActivityDate="2013-05-15T14:19:21.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7210" PostTypeId="2" ParentId="7200" CreationDate="2013-05-14T20:21:10.603" Score="1" Body="&lt;p&gt;In my codes, when I get the FE solution $f(x)$ as a linear combination of the basis functions $\phi_i(x)$:&#xA;$$&#xA;f(x) = \sum_i q_i \phi_i(x)&#xA;$$&#xA;I can easily evaluate it at &lt;em&gt;any&lt;/em&gt; point $x_0$ by evaluating the basis functions at this point ($\phi_i(x_0)$) and doing the linear combination:&#xA;$$&#xA;f(x_0) = \sum_i q_i \phi_i(x_0)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also use this approach to transfer the solution to quadrature points (in case they are not the same as the nodes). Obviously one of the advantages of spectral elements is to use the same points as nodes and quadrature points, then one doesn't have to do that --- but you still need this for plotting (as you asked), though again, many times I save myself the trouble and just plot the values at the nodes, that usually gives me a pretty good idea how the solution looks like.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2013-05-14T20:21:10.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7211" PostTypeId="1" CreationDate="2013-05-14T21:18:42.857" Score="9" ViewCount="432" Body="&lt;p&gt;Which turbulence model is suitable for resolving incompressible buoyancy-driven flow of a fluid within an cylindrical ampoule?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I prefer turbulence model which is sufficiently simple so that fully coupled (UFL) variational form of Navier-Stokes-Fourier is changed in way that each added term&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;is treated by Newton solver, or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;is precomputed prior to assembling tensors at every Newton iteration&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;so that no additional fixed-point iteration scheme is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can answer immediately or read description of problem below if useful.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y3yr0.png&quot; alt=&quot;Timedependent solution&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is coupled problem of flow of melt in the ampoule, flow of air in the furnace (outside of the ampoule) and heat conduction in rest parts of the system (including crystal on bottom of the ampoule).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For purposes of this question is relevant flow of the melt in the ampoule which is similiar to buoyancy-driven cavity (especially in top part) but&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;problem is cylindrically symmetric, symmetry axis is on the left&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;boundary conditions for temperature are more complicated than in classical benchmark; furnace wall (right boundary) has non-monotonic temperature profile (see image below - note that interior of the ampoule occupies cca $0.3&amp;lt;x&amp;lt;0.4$, $x$ being vertical coordinate) - increasing with altitude on most of the wall but decreasing near top; top and bottom boundaries of fluid have temperature continuous with outter wall (it is coupled to the rest of the system; bottom boundary is not planar)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OhCJ7.png&quot; alt=&quot;Temperature profile&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;almost all material coefficients are temperature dependent&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Flow in the ampoule resembles buoyancy-driven cavity in top part where temperature profile induces unstable stratification of fluid. This cavity-like flow is bounded by stably-stratified fluid from below. This was achieved with two orders of magnitude higher viscosity than target one and is non-stationary solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was tested that &lt;strong&gt;stationary solution&lt;/strong&gt; with implicit Euler in time and Newton solver accounting for non-linearity &lt;strong&gt;can be computed with three orders higher viscosity than target one&lt;/strong&gt;. For only two orders higher viscosity stationary solution seems to not beiing stable but time-marching with Crank-Nicolson scheme with small timesteps is possible. I'm currently going to switch to implicit Euler timesteping, lower timestep more, try adding SUPG/PSPG stabilization and check how small viscosity is manageable. But I guess that three orders lower viscosity than that enabling stationary solution will induce turbulence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the target viscosity Grashof anfd Rayleigh numbers are&#xA;$$ \mathrm{Gr = 3.6E8},$$&#xA;$$ \mathrm{Ra = 1.1E9},$$&#xA;taking whole length of ampoule as chracteristic length and $850-775 = 75\;\mathrm{K}$&#xA;as temperature difference scale; $775\;\mathrm{K}$ is temperature at bottom of the ampoule and $850\;\mathrm{K}$ is peak temperature of temperature profile. Note that this numbers could few orders of magnitude lower taking into account that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;one could take only depth of unstably-stratified region as a characteristic lenght&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;one could take smaller temperature difference corresponding to unstably-stratified region&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;peak furnace temperature $850\;\mathrm{K}$ is not reached inside the ampoule&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-15T22:34:59.700" LastActivityDate="2013-05-22T19:46:23.030" Title="Simple turbulence model appropriate for buoyancy-driven cavity like problem" Tags="|finite-element|fluid-dynamics|fenics|" AnswerCount="0" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="7212" PostTypeId="2" ParentId="7206" CreationDate="2013-05-15T01:10:47.867" Score="5" Body="&lt;p&gt;The accuracy of Gaussian elimination is &lt;em&gt;not&lt;/em&gt; bounded in terms of condition number.  There is an example of a well-conditioned matrix in Trefethen and Bau (and probably elsewhere) for which Gaussian Elimination is exponentially unstable with respect to matrix size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for iterative methods: CG finds the best approximation over the subspace in the energy norm of the preconditioned operator.  If the preconditioner is effective, then you can &lt;em&gt;usually&lt;/em&gt; evaluate this norm more accurately.  This is not guaranteed: the preconditioner is also ill-conditioned and in general, we can only evaluate $ A B x$ with accuracy proportional to $\kappa(A) \kappa(B)$, not $\kappa(A B)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, preconditioning does two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It makes the solution well-represented in a small number of vectors that appear early in the iteration.&lt;/li&gt;&#xA;&lt;li&gt;It makes the norm in which we evaluate optimality over a subspace closer to the norm of the error than the norm of the residual.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It is up to you whether you want the error to be small or the residual to be small.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-05-15T01:10:47.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7213" PostTypeId="1" CreationDate="2013-05-15T05:11:11.717" Score="5" ViewCount="150" Body="&lt;p&gt;I need a little help with Matlab code for the method mentioned in &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0912058&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for computing the inverse of the matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper, the author presented scaled Newton iteration given by &lt;span class=&quot;math-container&quot;&gt;$X_{k+1} = \alpha_{k+1}(2I - X_{k}A)X_{k}, k = 0, 1, 2, \ldots$&lt;/span&gt; for a given matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initial approximation &lt;span class=&quot;math-container&quot;&gt;$X_{0} = \alpha_{0}A^t$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$\alpha_{0} = \frac{2}{\sigma_{\min}+\sigma_{\max}}$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$\sigma_{\min}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\sigma_{\max}$&lt;/span&gt; are the the minimum and maximum singular values of the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha_{k+1}  = \frac{2}{1+(2 - \rho_{k} )\rho_{k}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\rho_{0} = \alpha_{0} \sigma_{\min}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\rho_{k+1} = \alpha_{k+1} (2 - \rho _{k} )\rho_{k} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the algorithm that I want to implement in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find singular values &lt;span class=&quot;math-container&quot;&gt;$\sigma_1 \geq \sigma_2\ldots \geq\sigma_{r}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha_{0} = \frac{2}{\sigma_{r}^{2}+\sigma_{1}^2}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\rho_{0} = \alpha_{0} \sigma_{r}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$X_{0} = \alpha_{0}A^t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;for &lt;span class=&quot;math-container&quot;&gt;$k = 1, 2\ldots\quad$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\alpha_k = \frac{2}{[1 + (2 - \rho_{k-1})\rho_{k-1}]}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$X_{k} = \alpha_{k}(2I - X_{k-1}A)X_{k-1}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\rho_{k} = \alpha_{k} (2 - \rho_{k-1})\rho_{k-1}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is the matlab code that I have made&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = rand(5);   % given matrix &#xA;  svds = svd(A)  % singular value of the matrix A&#xA;   s = min(svds)  %smallest singular value&#xA;   s1  = max(svds) %lsrgest singular value&#xA;   s2 = s^2 + s1^2  &#xA;   alpha0 = 2/s2  %alphazero&#xA;  rho0 = alpha0 *s&#xA;  x0 = alpha0*A';  %initial approximation&#xA;&#xA;  I = eye(5)&#xA;&#xA;  iter = 0&#xA;   for i = 1:20&#xA;       a1 = 2/(1 + (2 - rho0)*rho0)&#xA;       g1 = x0+x0*(I - A*x0);&#xA;       x1 = a1*(g1);     % approximation of inverse&#xA;       rho1 = a1*(2*rh0 -rh0*rh0 )&#xA;&#xA;       x0 = g1;&#xA;       rho0 = rho1&#xA;&#xA;       iter = iter +1&#xA;   end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code gives the correct result. But after few iterations, the values of &lt;code&gt;rho1&lt;/code&gt; and &lt;code&gt;a1&lt;/code&gt; coincide and become &lt;code&gt;1&lt;/code&gt; and, hence, it is taking an equal number of iterations compared to the classical Newton iteration. While it should take less number of iterations as mentioned in the paper. I want to know where I am wrong.&lt;/p&gt;&#xA;" OwnerUserId="4313" LastEditorUserId="20688" LastEditDate="2019-01-06T23:44:06.793" LastActivityDate="2019-01-06T23:44:06.793" Title="Matlab help related with the scaled Newton's iteration method" Tags="|linear-algebra|matlab|newton-method|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7214" PostTypeId="2" ParentId="7031" CreationDate="2013-05-15T09:56:55.047" Score="4" Body="&lt;p&gt;I suspect there is in general not much difference between GMRES and CG for an SPD matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we are solving $ Ax = b $ with $ A $ symmetric positive definite and the starting guess $ x_0 = 0 $ and generating iterates with CG and GMRES, call them $ x_k^c $ and $ x_k^g $.  Both iterative methods will be building $ x_k $ from the same Krylov space $ K_k = \{ b, Ab, A^2b, \ldots \} $.  They will do so in slightly different ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CG is characterized by minimizing the error $ e_k^c = x - x_k^c $ in the energy norm induced by $ A $, so that&#xA;\begin{equation}&#xA;  (A e_k^c, e_k^c) = (A (x - x_k^c), x - x_k^c) = \min_{y \in K} (A (x-y), x-y).&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GMRES minimizes instead the residual $ r_k = b - A x^g_k $, and does so in the discrete $ \ell^2 $ norm, so that&#xA;\begin{equation}&#xA;  (r_k, r_k) = (b - A x_k^g, b - A x_k^g) = \min_{y \in K} (b - Ay, b - Ay).&#xA;\end{equation}&#xA;Now using the error equation $ A e_k = r_k $ we can also write GMRES as minimizing&#xA;\begin{equation}&#xA;  (r_k, r_k) = (A e_k^g, A e_k^g) = (A^2 e_k^g, e_k^g)&#xA;\end{equation}&#xA;where I want to emphasize that this only holds for an SPD matrix $ A $.  Then we have CG minimizing the error with respect to the $ A $ norm and GMRES minimizing the error with respect to the $ A^2 $ norm.  If we want them to behave very differently, intuitively we would need an $ A $ such that these two norms are very different.  But for SPD $ A $ these norms will behave quite similarly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get even more specific, in the first iteration with the Krylov space $ K_1 = \{ b \} $, both CG and GMRES will construct an approximation of the form $ x_1 = \alpha b $.  CG will choose&#xA;\begin{equation}&#xA;  \alpha = \frac{ (b,b) }{ (Ab,b) }&#xA;\end{equation}&#xA;and GMRES will choose&#xA;\begin{equation}&#xA;  \alpha = \frac{ (Ab,b) }{ (A^2b,b) }.&#xA;\end{equation}&#xA;If $ A $ is diagonal with entries $ (\epsilon,1,1,1,\ldots) $ and $ b = (1,1,0,0,0,\ldots) $ then as $ \epsilon \rightarrow 0 $ the first CG step becomes twice as large as the first GMRES step.  Probably you can construct $ A $ and $ b $ so that this factor of two difference continues throughout the iteration, but I doubt it gets any worse than that.&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2013-05-15T09:56:55.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7215" PostTypeId="2" ParentId="7213" CreationDate="2013-05-15T11:18:58.607" Score="3" Body="&lt;p&gt;I can't access the article, but from what you describe, it looks quite normal that $\alpha$ and $\rho$ become close to $1$: if the method converges, you will have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ X = \alpha(2I - XA)X $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is equivalent to (supposing $X$ invertible)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ I = \alpha(2I - XA) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ XA = (2-\frac{1}{\alpha}) I$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and (supposing $A$ is invertible)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ X = (2-\frac{1}{\alpha})A^{-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want $X$ to converge to $A^{-1}$, you need $\alpha = 1$ and since $\alpha = \frac{2}{1+(2-\rho)\rho}$, you need $\rho$ to go to $1$ as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, it looks like the method is just working as it should. The reduced number of iterations might come from the &quot;warm-up&quot; phase at the beginning (that's just a guess).&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-05-15T11:18:58.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7216" PostTypeId="1" AcceptedAnswerId="7226" CreationDate="2013-05-15T14:29:32.427" Score="4" ViewCount="167" Body="&lt;p&gt;Long time ago I came across with a paper that covered early theoretical work (first half of 20th century) in the field of inverse problems. I remember there was a reference to a paper which proved that inverse problem can be solved uniquely for a spherical domain with infinitely many sources and very precise receivers distributed continuously over its surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I cannot find neither of these papers. Does anyone know this work or anything similar?&lt;/p&gt;&#xA;" OwnerUserId="4315" LastEditorUserId="127" LastEditDate="2013-05-15T14:31:28.487" LastActivityDate="2013-05-16T01:28:56.257" Title="Early work on inverse problems" Tags="|reference-request|inverse-problem|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7217" PostTypeId="1" CreationDate="2013-05-15T15:20:05.737" Score="3" ViewCount="229" Body="&lt;p&gt;Edit: I made a test model with 2 bodies. The sun and earth or w/e.&#xA;But I now have the problem of zero divisions every time that one of the coordinates is zero.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from numpy import exp, sqrt, pi, linspace&#xA;from time import clock,time&#xA;import pylab as pylab&#xA;&#xA;S1 = 1E3                   #Mass Star&#xA;M1 = 1                     #Mass Planet&#xA;&#xA;&#xA;Rs = np.array([0, 0])      #position&#xA;R1 = np.array([0,200])&#xA;&#xA;Vs = np.array([0,0])       #velocity&#xA;V1 = np.array([10,0])&#xA;&#xA;As = np.array([0,0])       #acceleration&#xA;A1 = np.array([0,0])&#xA;&#xA;&#xA;dt = 0.1                   #timestep&#xA;N = 1000                   #steps&#xA;G = 1                       #G const&#xA;&#xA;for i in range(1,N):&#xA;&#xA;    deltaR = Rs - R1&#xA;    Rsq = deltaR**2&#xA;    F12 = -G*M1*S1/(Rsq) &#xA;    F21 = -F12&#xA;&#xA;    As = F21/S1&#xA;    A1 = F12/M1&#xA;&#xA;    Vs += As*dt&#xA;    V1 += A1*dt&#xA;&#xA;    Rs += Vs*dt + 0.5*As*dt**2&#xA;    R1 += V1*dt + 0.5*A1*dt**2&#xA;&#xA;    x = [ Rs[0], R1[0]]&#xA;    y = [ Rs[0], R1[0]]&#xA;    pylab.ion()&#xA;    pylab.scatter( x, y )&#xA;    pylab.axis([-200, 200, -200, 200])&#xA;    pylab.plot&#xA;    pylab.clf()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any tips?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I want to simulate the solar system but want to start simple with one orbiting body. However, I never did anything like this before and was wondering if anyone here could give me some hints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly the first thing we need to use is the law of gravitation and Kepler. Then I'd like to throw everything into an array and just calculate the timesteps. However, I am not sure on how to get the position of the planet/moon as a function of time. (Can I just integrate the acceleration via the law of gravitation?)&lt;/p&gt;&#xA;" OwnerUserId="4316" LastEditorUserId="4316" LastEditDate="2013-05-24T17:53:39.977" LastActivityDate="2013-05-24T17:53:39.977" Title="Earth and Moon computer simulation" Tags="|simulation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7218" PostTypeId="2" ParentId="5477" CreationDate="2013-05-15T16:31:41.237" Score="1" Body="&lt;p&gt;mpmath has this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;&gt;&gt;&gt; import mpmath&#xA;&gt;&gt;&gt; mpmath.polylog(1, (1+1j)/mpmath.sqrt(2))&#xA;mpc(real='0.26739999836978523', imag='1.1780972450961724')&#xA;&gt;&gt;&gt; import numpy&#xA;&gt;&gt;&gt; for x in numpy.linspace(0.75, 1.0, 11): print mpmath.polylog(x, mpmath.sqrt(1j))&#xA;... &#xA;(0.13205298513153 + 1.22168160828646j)&#xA;(0.147070653672424 + 1.21783198879211j)&#xA;(0.161744021745461 + 1.21384547018142j)&#xA;(0.17607856182228 + 1.20973163332567j)&#xA;(0.190079791459314 + 1.20549967730407j)&#xA;(0.2037532636373 + 1.20115843051452j)&#xA;(0.217104557681381 + 1.19671636161604j)&#xA;(0.230139270736483 + 1.19218159029609j)&#xA;(0.242863009773464 + 1.18756189785626j)&#xA;(0.25528138410235 + 1.18286473761121j)&#xA;(0.267399998369785 + 1.17809724509617j)&#xA;&gt;&gt;&gt; &#xA;&lt;/pre&gt;&#xA;" OwnerUserId="3705" LastEditorUserId="3705" LastEditDate="2013-05-15T16:39:56.560" LastActivityDate="2013-05-15T16:39:56.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7219" PostTypeId="2" ParentId="7217" CreationDate="2013-05-15T16:59:33.753" Score="1" Body="&lt;p&gt;If you want to start simple, you can just use simple kinematics:&#xA;\begin{align*}&#xA;a(t) &amp;amp;= \text{Gravity}[x(t)] \\&#xA;v(t+ \Delta t) &amp;amp;= v(t) + a(t) \Delta t \\&#xA;x(t+\Delta t) &amp;amp;= x(t) + v(t) \Delta t + a(t) (\Delta t)^2/2 \\&#xA;\end{align*}&#xA;That should give you something that &lt;em&gt;looks&lt;/em&gt; OK for small time-steps, $\Delta t$, and short duration.  I also think it helps demonstrate the connection between continuous physics and &lt;a href=&quot;http://en.wikipedia.org/wiki/Explicit_and_implicit_methods&quot; rel=&quot;nofollow&quot;&gt;explicit methods&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For all kinds of reasons, simple methods like these don't work for long durations.  For a quantitatively valid scheme, check out the question's comments or a different answer.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2013-05-16T15:44:49.203" LastActivityDate="2013-05-16T15:44:49.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7220" PostTypeId="2" ParentId="7204" CreationDate="2013-05-15T18:00:09.430" Score="8" Body="&lt;p&gt;First, a general point: you cannot prescribe arbitrary boundary conditions for a partial differential operator and expect that the partial differential equation (which always includes both operator and boundary conditions) is well-posed, i.e., admits a unique solution that depends continuously on the data -- all of which is a necessary condition for actually trying to compute something. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the operator, there are often quite a number of valid conditions you can impose (to get a taste, see the three-volume monograph by Lions and Magenes). However, what you are trying to do (specify the full gradient, which is equivalent to both Dirichlet and Neumann conditions on the same (part of the) boundary for a second-order elliptic PDE) is not among them - this is known as a &lt;em&gt;lateral Cauchy problem&lt;/em&gt;, and is ill-posed: there is no guarantee that a given pair of boundary data admits a solution, and even if one exists, there is no stability with respect to small perturbations in the data. (In fact, this is the original ill-posed problem in the sense of Hadamard, and the classical example why you cannot ignore boundary conditions when discussing well-posedness. You can find an explicit example in his &lt;a href=&quot;http://archive.org/stream/lecturesoncauchy00hadauoft#page/33/mode/2up&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Lectures on Cauchy's problem in linear partial differential equations&lt;/em&gt;&lt;/a&gt; from the 1920s.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to your specific problem (which might be a textbook example of an &lt;a href=&quot;https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem&quot;&gt;XY problem&lt;/a&gt;).&#xA;If I understand correctly, you wish to solve an exterior problem for the Poisson equation, and you take a two-dimensional computional domain $(r,R)\times(a,b)$. You have full data (Dirichlet - using the tangential derivative trick - and Neumann) on (say) $x=r$. If $R$ is sufficiently large, you can justify a &lt;em&gt;radiation condition&lt;/em&gt; (which prescribes the asymptotic behavior of your solution a t $x\to \infty$); see, e.g., the book &lt;a href=&quot;http://books.google.com/books?id=Anx6zob_lYAC&amp;amp;pg=PA12&amp;amp;lpg=PA12&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical methods for exterior problems&lt;/a&gt;&#xA; by Long'an Ying.&#xA; The question is what you know about the remaining boundary parts, $y=a$ and $y=b$.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you can impose boundary conditions (Neumann, Robin, Dirichlet - which you would need to fix the constant in the integration of the tangential derivative, by the way), then it is enough to use either the normal components of your gradient as Neumann condition (if you can fix the constant mode) or integrate the tangential components as a Dirichlet condition. Since both conditions presumably correspond to the same function, you get the same solution either way. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you don't know the behavior at $y=a$ and $y=b$, you really have a lateral Cauchy problem, and you cannot compute a solution using standard finite element methods. The standard way of dealing with this is the &lt;em&gt;method of quasi-reversibility&lt;/em&gt; (introduced by Lattés and Lions in the 1960s; see their &lt;a href=&quot;http://books.google.com/books?id=hCmKQgAACAAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt;), which consists in approximating the second-order problem by a fourth-order problem (where you can - and need to - prescribe two boundary conditions). In your case, this would amount to replacing $-\Delta u=f$ by $-\Delta u  + \varepsilon \Delta^2 u=f$ for some small $\varepsilon&amp;gt;0$. (This can also be interpreted as minimizing the residual in a suitable norm and adding an $H^2$ regularization term; related to your comment on treating the problem as an inverse problem.)&#xA;You can show that for compatible data (i.e., a pair of boundary conditions that actually correspond to a solution $u$ of the Poisson equation), the quasi-reversibility solutions $u_\varepsilon$ converge to $u$ as $\varepsilon\to0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this is now a fourth-order problem with solutions in $H^2$, the best way of solving it numerically is using a mixed finite element formulation such as the one described in &lt;a href=&quot;http://www.jeremidarde.toile-libre.org/Articles_light/qr_mixed_light.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt; by Dardé, Hannukainen and Hyvönen. (There are also some &lt;a href=&quot;http://www.ljll.math.upmc.fr/Journees-Lions-Magenes/pdf/Darde_Lions-Magenes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slides&lt;/a&gt; online.) It should not be too difficult to implement this approach using FEniCS.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-03-20T10:29:50.967" LastActivityDate="2014-04-23T11:42:54.710" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7221" PostTypeId="1" AcceptedAnswerId="7222" CreationDate="2013-05-15T19:26:09.413" Score="5" ViewCount="1259" Body="&lt;p&gt;The integral I need to evaluate is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_x^{\infty} \frac{t^n}{e^{t} -1} dt $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some research I found a &lt;a href=&quot;http://dx.doi.org/10.1016/0927-0248%2895%2980004-2&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; saying,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The numerical values of the two integrals [...] are easily calculated either directly or through series expansion.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's easy to evaluate, right? So I try plugging this into my 32bit 7.7.0.471 (R2008b) MATLAB program and it cannot integrate it. Fortunately they actually cited this and I was lead to a mathematics table which stated,&#xA;$$\int_x^{\infty} \frac{t^n dt}{e^t -1} &#xA;= \sum_{k=1}^{\infty} \exp^{-kx} ( \frac{x^n}{k} + \frac{n x^{n-1}}{k^2} + \frac{ (n)(n-1) x^{n-2}}{k^3} + \cdots +\frac{n!}{k^{n+1}} )$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using $n=2$ so my actual expression is&#xA;$$\int_x^{\infty} \frac{t^2 dt}{e^t -1} &#xA;= \sum_{k=1}^{\infty} \exp(-kx) ( \frac{x^2}{k} + \frac{2 x^{1}}{k^2} + \frac{ (2)(1) x^{0}}{k^3})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, however, that my $x$ is very large for the environment, for example $10^{19}$, and the precision for $\exp$ causes it to be evaluated as 0. If I try to scale this on a $\log$ basis then I can't seem to get it converge even after several thousand summations. My software is also too old to be compatible with newer contributions which would solve my problem by allowing me to arbitrarily increase the precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This paper is from the mid 90's, why did they say it is so easy? Is there a better way, mathematically, to attack this that a novice like myself can understand and implement? How does one even find this series? It seems crazy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did find &lt;a href=&quot;http://dx.doi.org/10.1016/j.sse.2006.06.017&quot; rel=&quot;nofollow&quot;&gt;a paper&lt;/a&gt; that seems to be doing what I want, although I am not sure how to implement it.&lt;/p&gt;&#xA;" OwnerUserId="4318" OwnerDisplayName="sciencenewbie" LastEditorUserId="4244" LastEditDate="2013-06-10T14:48:49.840" LastActivityDate="2013-06-10T14:48:49.840" Title="Integration of an indefinite integral: matlab precision problem" Tags="|matlab|numerical-analysis|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7222" PostTypeId="2" ParentId="7221" CreationDate="2013-05-15T20:17:18.610" Score="6" Body="&lt;p&gt;For $n&amp;gt;0, x=0$, this evaluates to $\Gamma(n+1)\ \mathrm{Li}_{n+1}(1)$, and for $n=2$ this is $2\ \zeta (3)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first half, maybe &lt;a href=&quot;http://en.wikipedia.org/wiki/Debye_function&quot;&gt;this&lt;/a&gt; helps regarding more references. &lt;a href=&quot;http://mathworld.wolfram.com/DebyeFunctions.html&quot;&gt;Here&lt;/a&gt;, is an expression for the other half and even a closed (well, in terms of polylogs) for for $n=2$.&lt;/p&gt;&#xA;" OwnerUserId="440" LastActivityDate="2013-05-15T20:17:18.610" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7223" PostTypeId="2" ParentId="5477" CreationDate="2013-05-15T22:30:45.777" Score="1" Body="&lt;p&gt;If you plot the partial sums for the series, you get the following result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bBJ2N.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the real part for $\alpha=0.91$ for partial sums of 10 to 100 terms. Clearly, there is a pattern here, and the imaginary part follows a similar pattern. It seems that you can probably take 8 successive partial sums and average them to get a much better approximation. Or at least you can get the asymptotic numeric behavior from these partial sums.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More analytically, you can break the sum into 8 separate sums over the different roots of unity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum_{j=1}^8 \left( \frac{i+1}{\sqrt{2}} \right)^j \sum_{k=0}^\infty (8k+j)^{-\alpha} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of the inner sums is a Hurwitz zeta function, for which there are special evaluation routines (like in &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Hurwitz-Zeta-Function.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSL&lt;/a&gt;). Plotting out the sum in this way as a function of $\alpha$ gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sD9RU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The blue line is the real part, red is the imaginary part. This seems pretty well behaved.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-05-15T22:30:45.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7224" PostTypeId="1" AcceptedAnswerId="7225" CreationDate="2013-05-15T22:41:33.920" Score="9" ViewCount="4263" Body="&lt;p&gt;I read in a paper and also at &lt;a href=&quot;http://en.wikipedia.org/wiki/Circulant_matrix#In_linear_equations&quot;&gt;wiki&lt;/a&gt; that we can solve the system &#xA;$$Ax=B$$&#xA;by Fast Fourier Transform, where $A$ is a circulant matrix. The solution is&#xA;$$x=\mathtt{ifft}(\mathtt{fft}(B)/\mathtt{fft}(a))$$&#xA;where $a$ is first column of $A$, &lt;code&gt;ifft&lt;/code&gt; is the inverse of &lt;code&gt;fft&lt;/code&gt; and $/$ denotes component-wise division. For example, the solution of the following system&#xA;$$\begin{pmatrix} 2 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\ 1 &amp;amp; 2 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0\\ 0 &amp;amp; 1 &amp;amp; 2 &amp;amp; -1 &amp;amp; 0\\ 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 2 &amp;amp; -1\\ 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 2 \end{pmatrix}x=\begin{pmatrix} 2\\ 2\\ -4\\ 7\\ -6 \end{pmatrix}$$&#xA;is&#xA;$$x=\begin{pmatrix} 1\\ 0\\ -1\\ 2\\ -4 \end{pmatrix}$$&#xA;But when I implement $$x=\mathtt{ifft}(\mathtt{fft}(B)/\mathtt{fft}(a))$$, I get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x=\begin{pmatrix} \frac{118}{33}\\ - \frac{26}{33}\\ - \frac{53}{33}\\ \frac{142}{33}\\ - \frac{170}{33} \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is my fault?&lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="127" LastEditDate="2013-05-16T03:52:04.480" LastActivityDate="2020-01-13T01:50:19.133" Title="Solving linear systems by fft" Tags="|linear-solver|fourier-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7225" PostTypeId="2" ParentId="7224" CreationDate="2013-05-15T23:02:57.137" Score="14" Body="&lt;p&gt;Your matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; isn't a circulant matrix- it's just Toeplitz.  The method that you're trying to use fundamentally only works for circulant systems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, your &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; vector doesn't have the &quot;-1&quot; in it anywhere, so you clearly don't have sufficient information.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A method that involves embedding the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; Toeplitz matrix in a double-sized circulant matrix and doing an FFT on a vector of length &lt;span class=&quot;math-container&quot;&gt;$2n$&lt;/span&gt; is described in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;R. Kumar.  A Fast Algorithm for Solving a Toeplitz System of Equations. &lt;em&gt;IEEE Transactions on Acoustics, Speech, and Signal Processing&lt;/em&gt; 33(1): 1985. &lt;a href=&quot;https://doi.org/10.1109/TASSP.1985.1164492&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1109/TASSP.1985.1164492&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2020-01-13T01:50:19.133" LastActivityDate="2020-01-13T01:50:19.133" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7226" PostTypeId="2" ParentId="7216" CreationDate="2013-05-16T01:28:56.257" Score="5" Body="&lt;p&gt;I think you're looking for the paper by Calderon on electrical impedance tomography:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@InCollection{Cal80,&#xA;  author =   {A. Calder{\'o}n},&#xA;  title =    {On an inverse boundary value problem},&#xA;  booktitle =    {Seminar on Numerical Analysis and its Applications to&#xA;                  Continuum Physics},&#xA;  pages =    {65--73},&#xA;  publisher =    {Soc. Brasileira de Matematica, Rio de Janeiro},&#xA;  year =     1980,&#xA;  editor =   {W. H. Meyer and M. A. Raupp}&#xA;}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a long sequence of papers that came later (by Uhlmann, Paivarinta and others) that showed unique solvability for larger and larger classes of parameters.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-16T01:28:56.257" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7227" PostTypeId="2" ParentId="7217" CreationDate="2013-05-16T10:56:46.397" Score="3" Body="&lt;p&gt;For this type of problem, the leapfrog time-integration method is a good choice, see details &lt;a href=&quot;http://en.wikipedia.org/wiki/Leapfrog_integration&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4325" LastEditorUserId="127" LastEditDate="2013-05-16T15:42:08.690" LastActivityDate="2013-05-16T15:42:08.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7228" PostTypeId="1" AcceptedAnswerId="7229" CreationDate="2013-05-16T11:17:15.793" Score="8" ViewCount="209" Body="&lt;p&gt;I'm relatively new in the CFD modelling and I'm making a VOF model of a rectangular box (&lt;span class=&quot;math-container&quot;&gt;$l \times w \times h = 120\times 80 \times8$&lt;/span&gt; m) with an inlet (&lt;span class=&quot;math-container&quot;&gt;$w\times h = 10\times 8$&lt;/span&gt;) at the long side. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pressure outlet is the upper face of the box (&lt;span class=&quot;math-container&quot;&gt;$l\times w=120\times 80$&lt;/span&gt;m). The box is initially filled with &lt;span class=&quot;math-container&quot;&gt;$5$&lt;/span&gt;m water. Water is flowing in through the inlet, velocities at inlet: around &lt;span class=&quot;math-container&quot;&gt;$0.1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$0.01$&lt;/span&gt; m/s&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many cells/elements do I need to get a good indication for the flow pattern in the box? &lt;/p&gt;&#xA;" OwnerUserId="4326" LastEditorUserId="20688" LastEditDate="2019-06-22T19:11:19.193" LastActivityDate="2019-06-22T19:11:19.193" Title="How many cells/elements do I need?" Tags="|fluid-dynamics|mesh-generation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7229" PostTypeId="2" ParentId="7228" CreationDate="2013-05-16T11:54:13.867" Score="6" Body="&lt;p&gt;The answer to your question can depend on quite a few things, such as whether you need a turbulence model, and which one you choose if so, and how you are handling the &quot;top&quot; of your fluid since moving interfaces are nontrivial.  You are also imposing discontinuities on the boundary at the edges of your inlet so a nonuniform mesh would be advantageous for you.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While someone may be able to give you a real answer, the best way is to find it yourself.  Mesh your problem with what you deem a &quot;reasonable&quot; mesh, solve it, mesh it again with twice as many points, and see if your answers changed.  If they didn't you are done, and you have your answer.  If they did, either double your number of points again if you think you are close, or start over with the meshing based on where you biggest errors are.  Not only will this guarentee a good answer for you problem, it will also start to build your intuition for what types of features need more mesh points, so the next time you will start with a much better mesh.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-05-16T16:44:07.683" LastActivityDate="2013-05-16T16:44:07.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7230" PostTypeId="2" ParentId="7228" CreationDate="2013-05-16T12:12:24.317" Score="4" Body="&lt;p&gt;Concerning the flow simulation, an upper bound is given by the relation $N\geq Re^{9/4}$, where $N$ is the number of mesh nodes and $Re$ is the Reynolds number, derived for &lt;a href=&quot;http://en.wikipedia.org/wiki/Direct_numerical_simulation&quot; rel=&quot;nofollow&quot;&gt;direct numerical simulation&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case with $L=120m$, $u=0.1m/s$ and $\nu = 10^{-6}m^2/s$, you have $Re \approx 1.2 \cdot 10^5$ what - in theory - means that you will need about $2.7\cdot 10^{11}$ nodes in your mesh to resolve all scales of the flow dynamics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practise, however, this number is drastically decreased by using turbulence models or by just not caring about the smallest scales and doing a quasi DNS with adaptive mesh spacing (as @Godric Seer has pointed out).&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-16T13:21:24.693" LastActivityDate="2013-05-16T13:21:24.693" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7232" PostTypeId="1" AcceptedAnswerId="7241" CreationDate="2013-05-16T13:21:45.653" Score="4" ViewCount="518" Body="&lt;p&gt;I have probably a very stupid problem. I can't solve a simple Poiseuille flow in a straight 2-D channel driven by a pressure drop. Results are complete nonsense. (Setting zero pressure on the outflow gives trivial solution/no flow. Taking pressure on the outflow non-zero (but still having some pressure drop) gives a solution flowing in to the channel from both ends). Also satisfying of the incompressibility is very vague. Could you, please, look at my code? I have no idea what I am doing wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Marek&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;mesh = Mesh(&quot;P.xml&quot;)&#xA;&#xA;V = VectorFunctionSpace(mesh,&quot;CG&quot;,2)&#xA;Q = FunctionSpace(mesh,&quot;CG&quot;,1)&#xA;W = V*Q&#xA;&#xA;w = Function(W)&#xA;&#xA;nu = 100&#xA;L = 20&#xA;&#xA;noslip = DirichletBC(W.sub(0),(0,0),&quot;on_boundary &amp;amp;&amp;amp; (x[1] &amp;gt; 4.0 - DOLFIN_EPS | x[1] &amp;lt; DOLFIN_EPS)&quot;)&#xA;inflow = DirichletBC(W.sub(1),10,&quot;on_boundary &amp;amp;&amp;amp; x[0] &amp;lt; DOLFIN_EPS&quot;)&#xA;outflow = DirichletBC(W.sub(1),2,&quot;on_boundary &amp;amp;&amp;amp; x[0] &amp;gt; L - DOLFIN_EPS&quot;)&#xA;&#xA;bcs = [noslip,inflow,outflow]&#xA;&#xA;(u,p) = TrialFunctions(W)&#xA;(v,q) = TestFunctions(W)&#xA;&#xA;f = Constant((0,0))&#xA;a = nu*inner(grad(u),grad(v))*dx + p*div(v)*dx + div(u)*q*dx&#xA;L = inner(f,v)*dx&#xA;&#xA;problem = LinearVariationalProblem(a,L,w,bcs=bcs)&#xA;solver = LinearVariationalSolver(problem)&#xA;solver.solve()&#xA;(u,p) = split(w)&#xA;&#xA;plot(p)&#xA;plot(div(u))&#xA;plot(u)&#xA;interactive()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4329" LastEditorUserId="276" LastEditDate="2013-05-16T16:21:59.923" LastActivityDate="2013-05-16T19:45:22.387" Title="Poiseuille flow" Tags="|fluid-dynamics|fenics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7233" PostTypeId="2" ParentId="7232" CreationDate="2013-05-16T13:36:08.950" Score="7" Body="&lt;p&gt;Part of your problem is your choice of boundary conditions. &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/fld.1427/abstract&quot;&gt;This paper&lt;/a&gt; (which I wrote) develops the criteria for imposing a pressure boundary condition on the incompressible Navier-Stokes equations. The short version is that you need to impose the normal component of the normal traction on the inlet and outlet to be the pressures you want on each surface, and also force the flow to be normal to those surfaces. The first condition is achieved by through a natural boundary condition, and the latter can be done through a Dirichlet condition &lt;em&gt;if&lt;/em&gt; the surfaces are axis-aligned and flat (otherwise you'll have to use a Largrange multiplier, penalty, or constraint).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a typical Taylor-Hood FEM formulation you should not impose Dirichlet conditions on the pressure. It looks to me like you're trying to use these elements (piecewise quadratic velocities and piecewise linear pressures), but I'm no expert in what I'm guessing is FENICs (and so I can't help you with the code itself). &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-05-16T13:36:08.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7234" PostTypeId="1" AcceptedAnswerId="7239" CreationDate="2013-05-16T15:11:24.343" Score="1" ViewCount="1521" Body="&lt;p&gt;I am getting into computational fluid dynamics (CFD). One of my professors mentioned that a &lt;strong&gt;cylindrical wake&lt;/strong&gt; would be a good starting point to learn about turbulence modelling when using CFD software. I'm not a native speaker, so I googled &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;cylindrical wake CFD&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;but the results confuse me (also the translations of my dictionary). How is &lt;em&gt;wake&lt;/em&gt; defined in the context of CFD and what is a good resource to read up on it?&lt;/p&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2013-05-16T18:09:21.607" Title="What is a &quot;wake&quot; in the context of CFD?" Tags="|fluid-dynamics|reference-request|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7235" PostTypeId="1" CreationDate="2013-05-16T15:36:32.290" Score="5" ViewCount="1492" Body="&lt;p&gt;My question is basically the best way to compute the matrix exponential in parallel. I've got a sequential code written in Python (&lt;a href=&quot;https://github.com/hadsed/AdiaQC&quot; rel=&quot;nofollow&quot;&gt;https://github.com/hadsed/AdiaQC&lt;/a&gt;) and I need to extend it to run in a high performance environment (cluster, supercomputer, maybe GPU).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code simulates adiabatic quantum computation, and the main things to worry about is a complex-valued vector representing the quantum state, and a Hamiltonian matrix which I do an eigendecomposition on, as well as an exponent for the matrix exponential operator that I apply to the state vector. Hamiltonian and exponent are dense, and it grows as $2^n\times 2^n$, and state vector of course is $2^n$ as well. Here $n$ is the number of qubits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to be some development in SLEPc for a matrix exponential module, but I don't think I can wait for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking that the eigendecomposition method for the matrix exponential probably isn't the most efficient, but I'm not exactly sure how to efficiently do the other methods (and if they really are better). Any suggestions? I have gotten petsc4py and slepc4py working finally, so I'd like to exploit that as much as I can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if I need to be more specific about the actual problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Thought it might be helpful to explain more about Hamiltonian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my Ising Hamiltonian: $H = \sum\limits_{ij} \beta_{ij} z_i z_j + \sum\limits_{i} \alpha_i z_i + \sum\limits_i \delta_i x_i$ where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;z_i = I \otimes ... \otimes ~ \sigma^z_i ~ \otimes ... \otimes I&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sigma^z$ is the Pauli-z matrix, 2x2, and basically the identity but the lower diagonal element is -1. $I$ is just the identity, 2x2. So $\sigma^z_i$ is at the $i$th position of this bunch of Kronecker products. We cycle through i := {0, 1, ... , n} where n is the number of qubits. You do the same thing for $x_i$ (this time with the Pauli-x matrix, which is a 2x2 matrix, ones on the antidiagonal. With the two $z$s you just shift the position of two Pauli-z matrices instead of one in this big Kronecker product thing and sum those. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this Hamiltonian is time-dependent. $\alpha$, $\beta$, and $\delta$ are the only time-dependent components. Actually, they are coefficient matrices/vectors, but they are multiplied by something that makes it time-dependent. So,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;H_{exponent} = \frac{t}{T} H_{\alpha \beta} + (1 - \frac{t}{T}) H_{\delta}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H_{\alpha \beta}$ is just the first two sums in the Ising Hamiltonian, and same for the $H_{\delta}$. This is what I take the exponential of to apply to my state vector $\psi$ (I'm slightly lying here to simplify, it's not as simple as $t/T$ but it doesn't matter, they're just numbers that change with time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something to notice is that $H_{\alpha \beta}$ is completely diagonal. Really convenient, but unfortunately $H_{\delta}$ is completely the opposite of diagonal (read: dense as hell). In the beginning, $H_{exponent} = H_{\delta}$, and at the end it's $H_{\alpha \beta}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, explaining all this in the hopes that maybe someone can spot some exploitable structure.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2013-05-17T01:18:19.750" LastActivityDate="2013-05-17T01:18:19.750" Title="Computing matrix exponential with PETSc/SLEPc" Tags="|linear-algebra|parallel-computing|petsc|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7236" PostTypeId="1" CreationDate="2013-05-16T16:32:28.903" Score="3" ViewCount="305" Body="&lt;p&gt;I'm still pretty new in the CFD modelling world. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone advise me how to define a inlet for a tidal basin in Fluent? &#xA;The water level and the velocity at the inlet vary in time due to the tide cycle. &lt;/p&gt;&#xA;" OwnerUserId="4326" LastActivityDate="2014-12-08T02:42:18.147" Title="CFD (Fluent) define a inlet for a tidal basin" Tags="|fluid-dynamics|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7237" PostTypeId="2" ParentId="7235" CreationDate="2013-05-16T17:43:37.770" Score="6" Body="&lt;p&gt;Does your dense Hamiltonian have structure (e.g., sparse plus low-rank) or is it unstructured dense (no compressed representation available)?  Is it well-conditioned?  You can use &lt;a href=&quot;https://code.google.com/p/elemental/&quot;&gt;Elemental&lt;/a&gt;, either on its own or through PETSc's interface, for the parallel dense linear algebra.  I'm not aware of a suitable library implementation of the exponential, but you can build one out of the components provided in these libraries.  If you end up developing a general-purpose parallel implementation, please consider contributing it to PETSc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also these recent papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/100788860&quot;&gt;Al-Mohy and Higham, &lt;strong&gt;Computing the Action of the Matrix Exponential, with an Application to Exponential Integrators&lt;/strong&gt;, 2011&lt;/a&gt; &lt;a href=&quot;http://eprints.ma.man.ac.uk/1591/01/covered/MIMS_ep2010_30.pdf&quot;&gt;(author copy)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/040605461&quot;&gt;van den Eshof and Hochbruck, &lt;strong&gt;Preconditioning Lanczos Approximations to the Matrix Exponential&lt;/strong&gt;, 2006&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-05-16T17:50:22.257" LastActivityDate="2013-05-16T17:50:22.257" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="7238" PostTypeId="2" ParentId="7236" CreationDate="2013-05-16T18:07:33.960" Score="1" Body="&lt;p&gt;Depending on what you are trying to accomplish, you might want to look at some of the ocean modeling codes that exist to model a tidel inlet.  The Princeton Ocean Model (POM) is one of the most well-known of these and is relatively easy to setup.  Other examples include &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;FVCOM (&lt;a href=&quot;http://fvcom.smast.umassd.edu/FVCOM/&quot; rel=&quot;nofollow&quot;&gt;http://fvcom.smast.umassd.edu/FVCOM/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;ROMS (&lt;a href=&quot;http://www.myroms.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.myroms.org/&lt;/a&gt;) and &lt;/li&gt;&#xA;&lt;li&gt;SELFE (&lt;a href=&quot;http://www.stccmop.org/knowledge_transfer/software/selfe&quot; rel=&quot;nofollow&quot;&gt;http://www.stccmop.org/knowledge_transfer/software/selfe&lt;/a&gt;), &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;all of which have an active user community with examples.&lt;/p&gt;&#xA;" OwnerUserId="567" LastActivityDate="2013-05-16T18:07:33.960" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7239" PostTypeId="2" ParentId="7234" CreationDate="2013-05-16T18:09:21.607" Score="3" Body="&lt;p&gt;The two comments above are good for the definition of a wake, but in the case of the flow around the cylinder in 2D, the &lt;a href=&quot;http://en.wikipedia.org/wiki/K%C3%A1rm%C3%A1n_vortex_street&quot; rel=&quot;nofollow&quot;&gt;von Karman vortex street&lt;/a&gt; is the canonical problem that your professor was talking about.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-05-16T18:09:21.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7240" PostTypeId="1" AcceptedAnswerId="7395" CreationDate="2013-05-16T19:37:15.840" Score="3" ViewCount="8016" Body="&lt;p&gt;I'm trying to analyze the complexity of MATLAB code I wrote.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to figure out how much (in terms of $O$ or $\Theta$) to give functions like &lt;code&gt;find&lt;/code&gt;, matrix &lt;code&gt;*&lt;/code&gt; operator, matrix &lt;code&gt;.*&lt;/code&gt; operator, and others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to calculate this? Or a reference?&lt;/p&gt;&#xA;" OwnerUserId="4335" LastActivityDate="2013-05-25T21:48:48.147" Title="What is the Complexity of MATLAB operations" Tags="|matlab|complexity|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7241" PostTypeId="2" ParentId="7232" CreationDate="2013-05-16T19:45:22.387" Score="3" Body="&lt;p&gt;This actually works out fine if you don't integrate the pressure term by parts and write your variational problem as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = nu*inner(grad(u),grad(v))*dx + dot(grad(p), v)*dx + div(u)*q*dx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You also also need write the actual numerical value (not L) in your definition of the outflow boundary. Your variable L will otherwise not be used. I'm surprised you didn't get an error message that L is undefined.&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-05-16T19:45:22.387" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7242" PostTypeId="1" AcceptedAnswerId="7246" CreationDate="2013-05-16T19:52:53.643" Score="12" ViewCount="771" Body="&lt;p&gt;Suppose I had a boundary value problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2u}{dx^2} + \frac{dv}{dx}=f \text{ in } \Omega$$&#xA;$$\frac{du}{dx} +\frac{d^2v}{dx^2}  =g \text{ in } \Omega$$&#xA;$$u=h \text{ in } \partial\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to decompose the solution of this coupled problem into a sequence of uncoupled PDE's.  To decouple the system, I'm applying a fixed point iteration over a sequence of approximations $(u^k,v^k)$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2u^k}{dx^2} + \frac{dv^{k-1}}{dx}=f $$&#xA;$$\frac{du^{k-1}}{dx} +\frac{d^2v^k}{dx^2}  =g $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically, this would allow me to solve both equations as a purely elliptic PDE's.  However, I have never seen fixed point iterations applied to PDE's in this way.  I've seen fixed point iterations applied to the numerically discretized equations (finite difference method, finite element method, etc.), but never to the continuous equations directly.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I violating any blatant mathematical principle by doing this?  Is this mathematically valid?  Could I solve the coupled PDE as a sequence of uncoupled PDE's by using fixed point iteration applied to the CONTINUOUS variable problem, rather than the DISCRETE variable problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, I'm not really concerned with whether it is practical to use this method, but rather whether it is theoretically plausible.  Any feedback would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-05-16T20:39:17.923" LastActivityDate="2014-01-14T09:22:03.643" Title="Using fixed point iteration to decouple a system of pde's" Tags="|pde|iterative-method|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7244" PostTypeId="2" ParentId="7235" CreationDate="2013-05-16T20:38:28.247" Score="5" Body="&lt;p&gt;The development version of scipy has recently added expm_multiply which computes the action of the matrix exponential without explicitly computing the matrix exponential itself.  This uses the algorithm in &quot;Computing the Action of the Matrix Exponential...&quot;&#xA;&lt;br&gt;&#xA;&lt;a href=&quot;http://eprints.ma.man.ac.uk/1591/&quot; rel=&quot;nofollow&quot;&gt;http://eprints.ma.man.ac.uk/1591/&lt;/a&gt;&#xA;&lt;br&gt;&#xA;&lt;a href=&quot;https://github.com/scipy/scipy/pull/2456&quot; rel=&quot;nofollow&quot;&gt;https://github.com/scipy/scipy/pull/2456&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation has been recently patched to work with complex numbers, but it is less tested for this case.&#xA;&lt;br&gt;&#xA;&lt;a href=&quot;https://github.com/scipy/scipy/pull/2495&quot; rel=&quot;nofollow&quot;&gt;https://github.com/scipy/scipy/pull/2495&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its application to your problem would look something like this.&#xA;&lt;br&gt;&#xA;&lt;a href=&quot;https://github.com/hadsed/AdiaQC/pull/2&quot; rel=&quot;nofollow&quot;&gt;https://github.com/hadsed/AdiaQC/pull/2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;&lt;br&gt;&#xA;Even if the matrix exponent is not sparse, the expm_multiply function can still be accelerated if you have a clever way of quickly computing exponent-vector products, for example if the exponent has been factored or decomposed in certain ways.  I don't know enough quantum mechanics to know whether or not this would apply to your situation.&lt;/p&gt;&#xA;" OwnerUserId="3705" LastEditorUserId="3705" LastEditDate="2013-05-16T21:26:42.903" LastActivityDate="2013-05-16T21:26:42.903" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7245" PostTypeId="1" AcceptedAnswerId="7259" CreationDate="2013-05-16T21:27:42.050" Score="13" ViewCount="443" Body="&lt;p&gt;When FEM-discretizing and solving a reaction-diffusion problem, e.g.,&#xA;$$&#xA;- \varepsilon \Delta u + u = 1 \text{ on } \Omega\\&#xA;u = 0 \text{ on } \partial\Omega&#xA;$$&#xA;with $0 &amp;lt; \varepsilon \ll 1$ (singular perturbation), the solution of the discrete problem will typically exhibit oscillatory layers close to the boundary. With $\Omega = (0,1)$, $\varepsilon=10^{-5}$ and linear finite elements, the solution $u_h$ looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/D4AMu.png&quot; alt=&quot;solution of singularly perturbed problem&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see there's a lot of literature out there for such unwanted effects when they are caused by convection (e.g., upwind discretizations), but when it comes to  reaction, people seem to focus on refined meshes (Shishkin, Bakhvalov).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there discretizatons that avoid such oscillations, i.e., that preserve monotonicity? What else may be useful in this context?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="4244" LastEditDate="2013-06-05T10:33:38.647" LastActivityDate="2013-06-05T10:33:38.647" Title="Oscillations in singularly perturbed reaction-diffusion problems with finite elements" Tags="|finite-element|discretization|numerical-analysis|singular-perturbation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7246" PostTypeId="2" ParentId="7242" CreationDate="2013-05-16T21:40:37.157" Score="11" Body="&lt;p&gt;You define a sequence in, say, $C^{\infty}(\Omega)\times C^{\infty}(\Omega)$ by&#xA;$$&#xA;\frac{\text{d}^2u^k}{\text{d}x^2} + \frac{\text{d} v^{k-1}}{\text{d} x} =f\\&#xA;\frac{\text{d}^2v^k}{\text{d}x^2} + \frac{\text{d} u^{k-1}}{\text{d} x} =g\\&#xA;$$&#xA;(plus boundary conditions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear that &lt;em&gt;if&lt;/em&gt; this sequence converges, it will be a solution of your original set of PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for proving whether the sequence converges in the Banach space, &lt;a href=&quot;https://en.wikipedia.org/wiki/Banach_fixed-point_theorem&quot; rel=&quot;nofollow&quot;&gt;Banach's fixed point theorem&lt;/a&gt; may come in handy. It says that if your mapping $x^k\to x^{k+1}$ is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Contraction_mapping&quot; rel=&quot;nofollow&quot;&gt;contraction&lt;/a&gt;, you can be sure to approach the solution given any initial guess $u^0$, $v^0$. Hence, what you'd have to check is if&#xA;$$&#xA;\left\|&#xA;\begin{pmatrix}&#xA;u^k\\&#xA;v^k&#xA;\end{pmatrix}&#xA;-&#xA;\begin{pmatrix}&#xA;\hat{u}^k\\&#xA;\hat{v}^k&#xA;\end{pmatrix}&#xA;\right\|&#xA;\le&#xA;q&#xA;\left\|&#xA;\begin{pmatrix}&#xA;u^{k-1}\\&#xA;v^{k-1}&#xA;\end{pmatrix}&#xA;-&#xA;\begin{pmatrix}&#xA;\hat{u}^{k-1}\\&#xA;\hat{v}^{k-1}&#xA;\end{pmatrix}&#xA;\right\|&#xA;$$&#xA;with a fixed $|q|&amp;lt;1$ for any given $(u^{k-1}, v^{k-1})$, $(\hat{u}^{k-1}, \hat{v}^{k-1})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This logic works both in the continuous and the discrete space.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2014-01-14T09:22:03.643" LastActivityDate="2014-01-14T09:22:03.643" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7247" PostTypeId="2" ParentId="7240" CreationDate="2013-05-16T22:24:18.063" Score="11" Body="&lt;p&gt;The cost of standard operations like find-and-replace, maxtrix-matrix multiplications and elementwise matrix-matrix multiplications is well-known: Check out &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations#Matrix_algebra&quot;&gt;the corresponding wiki site&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd really like to know what your MATLAB installation does, you can always go ahead and measure the time, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t = zeros(1,100);&#xA;for n = 1:100&#xA;    A = rand(n,n);&#xA;    B = rand(n,n);&#xA;    tic;&#xA;    C = A*B;&#xA;    t(n) = toc;&#xA;end&#xA;plot(t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This may also give you some insight in cache behavior and such.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-05-16T22:24:18.063" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7248" PostTypeId="1" CreationDate="2013-05-17T01:34:02.963" Score="5" ViewCount="986" Body="&lt;p&gt;I'm interested in finite-difference approaches to the incompressible Navier-Stokes equations that can handle complex geometry without the use of an unstructured mesh or a non-Cartesian grid. To be clear, I'm aware of standard approaches, e.g. &lt;a href=&quot;http://en.wikipedia.org/wiki/Projection_method_%28fluid_dynamics%29&quot;&gt;Chorin's projection method&lt;/a&gt;, to solving the Navier-Stokes equations on a rectangular domain, but I'd I'd like to know more about what methodologies exist to extend these techniques to more sophisticated geometries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify my intent, one particularly notable example of what I'm looking for would be &lt;a href=&quot;http://en.wikipedia.org/wiki/Immersed_boundary_method&quot;&gt;Peskin's Immersed Boundary Method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See below for a more precise statement of the particular problem I'm interested in.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Consider solving the incompressible Navier-Stokes equations&#xA;\begin{align*}&#xA;\rho\left(\mathbf{u}_t + (\mathbf{u}\cdot\nabla)\mathbf{u}\right) &amp;amp;= - \nabla p + \mu\Delta\mathbf{u} + \mathbf{f}\\&#xA;\nabla\cdot\mathbf{u} &amp;amp;= 0&#xA;\end{align*}&#xA;with&#xA;$$\mathbf{f} = (f_0,0,0)$$&#xA;on the domain $\Omega = [-1,1]^d \setminus C$ where&#xA;$$C = \left\{\mathbf{x} \in [-1,1]^d : |\mathbf{x}| &amp;lt; \frac{1}{2}\right\}.$$&#xA;The boundary conditions are no-slip (i.e., $\mathbf{u} = 0$) except at $\{x=-1\}$ and $\{x=1\}$, where we enforce a periodic boundary condition. In other words, this is periodic Poiseuille flow around a cylinder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge here lies entirely in enforcing the no-slip condition on $\partial C$, the boundary of the cylinder. A naive -- and inaccurate -- approach is to simply set $\mathbf{u} = 0$ at grid points inside the cylinder every time step. The Immersed Boundary Method is another option. Simply put, what other techniques are out there?&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-05-17T01:39:45.907" LastActivityDate="2015-10-14T13:01:26.210" Title="What methods exist to solve for the fluid flow past a cylinder using finite differences on a Cartesian grid?" Tags="|finite-difference|fluid-dynamics|reference-request|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7250" PostTypeId="2" ParentId="7191" CreationDate="2013-05-17T07:21:35.417" Score="7" Body="&lt;p&gt;Eigen's LDLT class actually performs a U^T.D.U factorization if the input matrix is column-major with the symmetric entries stored in the upper triangular part:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MatrixXd A;&#xA;// fill at least the upper triangular part of A&#xA;LDLT&amp;lt;MatrixXd,Upper&amp;gt; udu(A); // Only the upper part of A is read to form U&#xA;udu.matrixU(); // returns an expression of the triangular matrix U (column major)&#xA;udu.matrixL(); // returns an expression of the triangular matrix U^T (row major)&#xA;udu.vectorD(); // expression of the diagonal coefficients of the matrix D as a vector&#xA;udu.vectorD().asDiagonal(); // an expression of the diagonal matrix D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4337" LastActivityDate="2013-05-17T07:21:35.417" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7251" PostTypeId="2" ParentId="7240" CreationDate="2013-05-17T10:05:04.453" Score="2" Body="&lt;p&gt;To check your implementation as a whole, you can use MATLAB's &lt;a href=&quot;http://www.mathworks.de/de/help/matlab/matlab_prog/profiling-for-improving-performance.html#f9-17315&quot; rel=&quot;nofollow&quot;&gt;profiler&lt;/a&gt;. It is a built-in functionality that analyses your code in terms of execution time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It provides an interface to all kind of measurements and you can dig into the results down to execution time of particular code lines.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-05-17T10:05:04.453" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7252" PostTypeId="1" AcceptedAnswerId="7253" CreationDate="2013-05-17T12:56:09.437" Score="5" ViewCount="1415" Body="&lt;p&gt;For the numerical integration of reaction terms in my PDE on a 2D triangular mesh, I would like to use the scheme&#xA;$$&#xA;\int_T \Phi(x) \approx \frac{|T|}{3} (\Phi(x_0) + \Phi(x_1) + \Phi(x_2))&#xA;$$&#xA;where the $x_i$ are the triangle nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to specify this when &lt;code&gt;assemble&lt;/code&gt;ing the system?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-05-17T13:53:51.300" Title="FEniCS: custom quadrature rule" Tags="|quadrature|fenics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7253" PostTypeId="2" ParentId="7252" CreationDate="2013-05-17T13:23:07.567" Score="3" Body="&lt;p&gt;Actually you don't specify quadrature rule to assembler but to form compiler. You can tweak &lt;code&gt;ffc/quadrature_schemes.py&lt;/code&gt; for your needs or you can adjust generated code. In both cases you just supply your quadrature points and weights.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-17T13:32:39.647" LastActivityDate="2013-05-17T13:32:39.647" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7254" PostTypeId="2" ParentId="7252" CreationDate="2013-05-17T13:53:51.300" Score="2" Body="&lt;p&gt;You can adjust the quadrature degree by setting&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;parameters[&quot;form_compiler_parameters&quot;][&quot;quadrature_degree&quot;] = q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will change the quadrature degree such that polynomials of degree q are integrated exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FEniCS will try to autodetect the necessary quadrature degree so this should normally not be needed. However, in some cases you might want to underintegrate by setting a lower value of q.&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-05-17T13:53:51.300" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7255" PostTypeId="1" CreationDate="2013-05-17T17:49:06.143" Score="1" ViewCount="42" Body="&lt;p&gt;I am trying to simulate a metal-organic framework in LAMMPS using the UFF potential.  It's working quite well for some structures where all molecules have an occupancy of 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I have a solvent molecule that has an occupancy of 0.5, what I do is randomly select one of the two positions for each unit cell.  The structure collapses upon simulation.  I'm not sure how to fix this.  My MOF is about 6x3x3 unit cells big; would there be any beneficial reason to try a huge system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried simulating at low temperatures, NPT, NVT; it falls apart everytime.  You're not really supposed to use partial charges with UFF, but I tried it with and without.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's just amazing to me that one structure is so incredibly stable and then a slightly different structure that has 0.5 occupancies falls apart.  Does anyone have any experience with this?&lt;/p&gt;&#xA;" OwnerUserId="1997" LastActivityDate="2013-05-17T17:49:06.143" Title="Simulation of asymmetric structures (occupancy = 0.5) unstable" Tags="|stability|molecular-dynamics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7256" PostTypeId="1" AcceptedAnswerId="8011" CreationDate="2013-05-17T18:11:21.957" Score="2" ViewCount="312" Body="&lt;p&gt;The explicit 4th order discretization for the 2D scalar wave equation is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;U_{jk}^{n+1}  =  \left( \frac{\Delta t  V_{jk} }{\Delta s} \right) ^2 \left(  \sum_{a=-N}^N w_a U_{j+a k}^n + \sum_{a=-N}^N w_a U_{j k+a}^n \right) + 2 U_{jk}^{n} - U_{jk}^{n-1}  \nonumber \\&#xA;U_{jk}^{n+1}  =  \left( \frac{\Delta t  V_{jk}}{\Delta s} \right) ^2  \sum_{a=-N}^N  w_a \left( U_{j+a k}^n + U_{j k+a}^n \right) + 2 U_{jk}^{n} - U_{jk}^{n-1} \label{1}&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $ \Delta s = \Delta x = \Delta z $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Jing-Bo Cheen&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2360/how-can-i-derive-a-bound-on-the-spurious-oscillations-in-the-numerical-solution&quot;&gt;1&lt;/a&gt; the stability limit is given by, where $V$ is $\max(V_{jk})$ for $N = 2$ forth order for space derivatives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Delta t \leq \frac{2 \Delta s}{ V \sqrt{\sum_{a=-N}^{N} (|w_a^1| + |w_a^2|)}} $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $w_a$ are the centered differences weights, In the code bellow it's defined be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ w_a = [-1.0/12, 4.0/3, -5.0/2, 4.0/3, -1.0/12] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the code bellow, I wrote using numpy, I had the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Delta s = 10 $$&#xA;$$ V = 2000.0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grid dimensions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Nz = Nx = 20 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Triangular Wavelet placed at (10, 10) with 23 samples or (23 ms).&#xA;Amplitude of peak 1.0 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number of iterations (enough to see some propagation):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ numberiter = 200 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I would expect that :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Delta t \leq 0.0030618621784789723 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This case, should be ok no?&lt;/strong&gt; since it's :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Delta t = 0.001 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Updated:.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading some &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2360/how-can-i-derive-a-bound-on-the-spurious-oscillations-in-the-numerical-solution&quot;&gt;posts&lt;/a&gt; I guess is something related to not having an smooth sorce or using spatial derivatives high order. Still I don't have all the skills to make this here work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I am seing is a lot of dispersion : just in 6 iterations my maximum values on my grid (non zero) are on E-14 order.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Also looking on a specific point of my grid, close to the source (13,13) I get the following picture as time evolves (first 50 iterations)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rVLFw.png&quot; alt=&quot;oscilation 50th iterations&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;What am I doing wrong? Suggetions to make it stable will be very appreaciated&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import sys&#xA;&#xA;def Triangle(fc, dt, n=None):&#xA;    r&quot;&quot;&quot;&#xA;    Triangle Wave one Period.&#xA;    Defined by frequency and sample rate or by size&#xA;&#xA;    * fc        : maximum desired frequency&#xA;    * dt        : sample rate&#xA;    * n         : half length of triangle    &#xA;    &quot;&quot;&quot;&#xA;    if(n==None):&#xA;        n=int(1/float(fc*dt))&#xA;&#xA;    t = np.arange(0+1.0/n, 1, 1.0/n)&#xA;    y = 1-t&#xA;    y = np.append(y, 0.0)&#xA;    y_ = 1-t[::-1]&#xA;    y_ = np.insert(y_, 0, 0.0)&#xA;&#xA;    return np.append(y_, np.append(1, y))&#xA;&#xA;Nz = Nx = 20&#xA;Dt = 0.001&#xA;Ds = 10&#xA;numberiter = 200&#xA;&#xA;Source = Triangle(90, 0.001)&#xA;Uprevious = np.zeros([Nz, Nx]) # previous time&#xA;Ucurrent = np.zeros([Nz, Nx]) # current time&#xA;Ufuture = np.zeros([Nz, Nx]) # future time&#xA;V = np.zeros([Nz, Nx]) &#xA;V[:][:] = 2000.0&#xA;&#xA;# additional not needed &#xA;Simulation  = np.zeros([numberiter, Nz, Nx])&#xA;&#xA;# source activation, center of grid&#xA;Uprevious[10][10] = Source[0]&#xA;&#xA;for i in range(1, numberiter+1):&#xA;&#xA;    # tringular source position center grid&#xA;    if(i &amp;lt; np.size(Source)):&#xA;        Ucurrent[10][10] = Source[i]&#xA;&#xA;    for k in range(Nz):&#xA;        for j in range(Nx):&#xA;            # u0k u1k*ujk*u3k u4k     &#xA;            # Boundary fixed 0 outside        &#xA;            u0k=u1k=u3k=u4k=0.0&#xA;            uj0=uj1=uj3=uj4=0.0&#xA;            ujk = Ucurrent[k][j]      &#xA;&#xA;            if(j-2 &amp;gt; -1):&#xA;                u0k = Ucurrent[k][j-2]  &#xA;            if(j-1 &amp;gt; -1):&#xA;                u1k = Ucurrent[k][j-1]&#xA;            if(j+1 &amp;lt; Nx):&#xA;                u3k = Ucurrent[k][j+1]            &#xA;            if(j+2 &amp;lt; Nx):&#xA;                u4k = Ucurrent[k][j+2]&#xA;            if(k-2 &amp;gt; -1):&#xA;                uj0 = Ucurrent[k-2][j]&#xA;            if(k-1 &amp;gt; -1):&#xA;                uj1 = Ucurrent[k-1][j]&#xA;            if(k+1 &amp;lt; Nz):&#xA;                uj3 = Ucurrent[k+1][j]            &#xA;            if(k+2 &amp;lt; Nz):&#xA;                uj4 = Ucurrent[k+2][j]&#xA;&#xA;            d2u_dx2 = (-u0k+16*u1k-30*ujk+16*u3k-u4k)/12.0&#xA;            d2u_dz2 = (-uj0+16*uj1-30*ujk+16*uj3-uj4)/12.0&#xA;            Ufuture[k][j] = (d2u_dx2+d2u_dz2)*(Dt*V[k][j]/Ds)**2&#xA;            Ufuture[k][j] += 2*Ucurrent[k][j]-Uprevious[k][j]&#xA;&#xA;    # make the update in the time stack&#xA;    Uprevious = Ucurrent&#xA;    Ucurrent = Ufuture&#xA;    Simulation[i-1] = Ucurrent&#xA;&#xA;    sys.stdout.write(&quot;\r %d&quot; %(i) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2360/how-can-i-derive-a-bound-on-the-spurious-oscillations-in-the-numerical-solution&quot;&gt;1&lt;/a&gt; A stability formula for Lax-Wendroff methods with fourth-order in time and general-order in space for the scalar wave equation - Geophysics Vol. 76 No. 2 2011&lt;/p&gt;&#xA;" OwnerUserId="4092" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-07-19T03:09:43.090" Title="Explicit 4th order space wave equation not stable implementation?" Tags="|pde|finite-difference|python|discretization|numpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7257" PostTypeId="2" ParentId="4822" CreationDate="2013-05-18T02:14:59.537" Score="1" Body="&lt;p&gt;There's a lot of information online that search engines are capable of finding.  Here's three hits to get the ball rolling:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1109/TC.2010.121&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1109/TC.2010.121&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://smartech.gatech.edu/bitstream/handle/1853/45737/GT-CSE-2012-01.pdf&quot; rel=&quot;nofollow&quot;&gt;http://smartech.gatech.edu/bitstream/handle/1853/45737/GT-CSE-2012-01.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dl.acm.org/citation.cfm?doid=2425676.2425691&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?doid=2425676.2425691&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2013-06-20T16:53:55.353" LastActivityDate="2013-06-20T16:53:55.353" CommentCount="2" CommunityOwnedDate="2013-05-18T02:14:59.537" ContentLicense="CC BY-SA 3.0" />
  <row Id="7258" PostTypeId="1" CreationDate="2013-05-18T11:01:16.893" Score="6" ViewCount="381" Body="&lt;p&gt;&lt;strong&gt;Motivation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In isogeometric analysis, state variables(e.g. displacement) are defined in the parametric domain, which can be mapped to the physical domain by $\boldsymbol{\xi}\mapsto \boldsymbol{x}$ as shown beneath. However the quantity related to displacment such as stress, strain are spacial derivatives of displacement. The following procedure is commonly used for solving those derivatives. $\blacksquare$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $u$ be one component of displacement vector $\boldsymbol{u}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u(\xi,\eta) = \sum_{i} c_iN^i(\xi,\eta) &#xA;\end{equation}&#xA;with geometric mapping from the parametric domain to the physical domain&#xA;$$x(\xi,\eta) = \sum_{i} x_i N^i(\xi,\eta), \quad y(\xi,\eta) = \sum_{i} y_i N^i(\xi,\eta),$$&#xA;where $c_i,x_i,y_i$ are constants, with assumption that $(\xi,\eta)\mapsto(x,y)$ is bijective, i.e. inverse exists, $$J :=[\frac{\partial x_i}{\partial \xi_j}],\: |J| \neq 0\quad (\text{where }x_2 = y,\,\xi_2 = \eta).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By chain rule, &#xA;$$\frac{\partial u}{\partial \xi_j} = \frac{\partial u}{\partial x_i}\frac{\partial x_i}{\partial \xi_j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{bmatrix}&#xA;\frac{\partial u}{\partial \xi}\\&#xA;\frac{\partial u}{\partial \eta}&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial x}{\partial \xi} &amp;amp;&#xA;\frac{\partial y}{\partial \xi}\\&#xA;\frac{\partial x}{\partial \eta} &amp;amp;&#xA;\frac{\partial y}{\partial \eta}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\frac{\partial u}{\partial x}\\&#xA;\frac{\partial u}{\partial y}&#xA;\end{bmatrix}&#xA; = J^T&#xA;\begin{bmatrix}&#xA;\frac{\partial u}{\partial x}\\&#xA;\frac{\partial u}{\partial y}&#xA;\end{bmatrix}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence,&#xA;$$&#xA;\begin{bmatrix}&#xA;\frac{\partial u}{\partial x}\\&#xA;\frac{\partial u}{\partial y}&#xA;\end{bmatrix}&#xA;=&#xA;(J^T)^{-1}&#xA;\begin{bmatrix}&#xA;\frac{\partial u}{\partial \xi}\\&#xA;\frac{\partial u}{\partial \eta}&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the common procedure used in isogemetric analysis for computing stiffness matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when working on sensitivity analysis, I need to compute the second derivatives $$\frac{\partial^2 u}{\partial x_i\partial x_j},\quad i \text{ and }j \in \{1,2\}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One may think displacement $\boldsymbol{u} = [u, v]^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having searched on Scirus site, I failed to find any useful papers. But I believe it has to be done by some groups already. And reference would be appreciated. I have many accesses to scientific database, so only a link to the paper would be sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A little bit explanation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have encountered this problem when working on sensitivity analysis, especially adjoint method with the approach of material derivatives.&#xA;Let assume we have an objective functional&#xA;$$\phi = \int_{\Omega}f(\sigma,\epsilon,p)\,\mathrm{d}\Omega$$&#xA;where $p$ is a design parameter(e.g. coordinate of one control point).&#xA;With linear elastostatics, $\sigma,\,\epsilon$ are functions of $\nabla \boldsymbol{u}$, so we can write $f = f(\nabla u)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the material derivative of the domain functional is&#xA;$$\dot{\phi} = \frac{\mathrm{d}}{\mathrm{d} p}\phi &#xA;= \int_{\Omega}\frac{\partial}{\partial \nabla \boldsymbol{u}}f:\nabla\dot{\boldsymbol{u}}-\frac{\partial}{\partial \nabla \boldsymbol{u}}f:\nabla((\nabla\boldsymbol{u})\boldsymbol{v})\,\mathrm{d}\Omega + &#xA;\int_{\partial\Omega}f(\boldsymbol{v}\cdot\boldsymbol{n})\,\mathrm{d}\Gamma.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first term in the domain integral is taken care of by adjoint formulation. From the second term in the domain integral, the second derivative is required. &lt;/p&gt;&#xA;" OwnerUserId="1152" LastEditorUserId="1152" LastEditDate="2013-05-20T17:06:04.937" LastActivityDate="2018-02-12T07:48:20.330" Title="Compute spatial second derivatives in Isogeometric analysis" Tags="|optimization|finite-element|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7259" PostTypeId="2" ParentId="7245" CreationDate="2013-05-18T11:51:44.820" Score="6" Body="&lt;p&gt;In the case you show, the solution has a boundary layer. If you can't resolve it because your mesh is too coarse, then for all practical matters the solution is discontinuous to the numerical scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you just apply a standard discretization to this problem, the discrete solution is the result of applying a linear projection operator to the exact solution, projecting onto a finite dimensional space. This is really no different than, for example, taking the first $N$ terms of a Fourier series. But there you know what's going to happen when you apply it to a discontinuous function: you get Gibbs phenomenon with over- and undershots. The situation here is really no different and it will happen with any linear scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are essentially two approaches you can follow to working around this problem: (i) You add &lt;em&gt;artificial diffusion&lt;/em&gt;, i.e., you add a diffusion term that is proportional to the mesh size to some power; this increases the size of the boundary layer to a width that can be represented by the mesh, but it goes back to $\varepsilon$ as $h\rightarrow 0$. (ii) You use a nonlinear projection scheme; the hyperbolic literature has any number of such schemes, e.g., shock capturing, slope limiting, etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-18T11:51:44.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7260" PostTypeId="2" ParentId="1449" CreationDate="2013-05-18T14:56:24.690" Score="3" Body="&lt;p&gt;To add to the available great answers, there is a tool developed at Rice which automates stack sampling and therefore has very little overhead:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hpctoolkit.org/&quot; rel=&quot;nofollow&quot;&gt;http://hpctoolkit.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-05-18T14:56:24.690" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7261" PostTypeId="2" ParentId="5462" CreationDate="2013-05-18T15:41:03.260" Score="3" Body="&lt;p&gt;Since $A=R_1R_2^T$ with $R_1, R_2\in \mathbb{R}^{n \times r}$, we have&#xA;$$&#xA;\sum_i A_{\pi i, i} = \sum_i (P_{\pi} A)_{i, i} = \text{trace}(P_{\pi}R_1R_2^T)&#xA;$$&#xA;where $P_{\pi}$ is the permutation matrix corresponding to $\pi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any $\pi$, the trace can be computed as&#xA;$$&#xA;\text{trace}(P_{\pi}R_1R_2^T) = \sum_{i} \sum_{k} (P_{\pi}R_1)_{i,k} (R_2^T)_{k,i}&#xA;= \sum_{i,k} ((P_{\pi}R_1)\circ R_2)_{i,k}.&#xA;$$&#xA;(This quantity is also known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_multiplication#Frobenius_product&quot; rel=&quot;nofollow&quot;&gt;Frobenius product&lt;/a&gt;, $P_{\pi}R_1:R_2$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea doesn't take away the burden of having to go through all permutations and brute-force search for the maximum of all Frobenius products, and in fact is has the same arithmetic complexity as explicitly computing $A=R_1R_2^T$. However, it has much lower memory requirements since you never have to actually form $A$.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-06-06T19:12:27.993" LastActivityDate="2013-06-06T19:12:27.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7262" PostTypeId="1" CreationDate="2013-05-18T22:03:50.593" Score="1" ViewCount="274" Body="&lt;p&gt;I am still a fairly new graduate student, and I am having trouble simulating a metal-organic framework.  I am currently using UFF (I implemented it for LAMMPS), and it works well for some MOFs, but does not give empirically accurate results for others.  What I mean by this is that I have a CIF file for the experimentally obtained structure, and when simulate it, the structure changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UFF is fairly old (1992?); there's got to be something better that will allow this to match the experimental structure.  Any ideas?  Currently, I'm obtaining partial charges for the solvent molecules, copper ions, and chlorine from various papers, but I am considering trying to use Gaussian even though I have never done that before.  UFF is generally used without partial charges, but I think the strong electrostatics in my MOF necessitates their use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MOF contains carbon, nitrogen, hydrogen, chlorine, oxygen, and copper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm still working with the possibility that I may have screwed something up somewhere in the simulation process, but I would still think there's something newer/more accurate than UFF).&lt;/p&gt;&#xA;" OwnerUserId="1997" LastActivityDate="2013-05-18T22:03:50.593" Title="Accurate force-field for MD" Tags="|molecular-dynamics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7263" PostTypeId="1" CreationDate="2013-05-19T02:20:51.153" Score="11" ViewCount="161" Body="&lt;p&gt;At work, I essentially function as an independent consultant. For management and customers, I need to estimate the amount of time it will take to develop software as part of my computational science research. However, my time estimates are usually off. I know that there are methods to estimate the time it takes to develop software. Will these methods give me accurate estimates for research tasks? Are they even useful in a research setting? If they aren't, is there a better method than &quot;keep records and revise estimates accordingly&quot;?&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-05-20T13:59:13.283" Title="Does anyone use software estimation methods in their computational science research?" Tags="|software|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7264" PostTypeId="1" CreationDate="2013-05-19T02:56:29.713" Score="2" ViewCount="979" Body="&lt;p&gt;In one of my PETSc programs, I need a processor to access values from another processor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the PETSc FAQ's this issue has been briefly addressed &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/faq.html#mpi-vec-access&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. &#xA;To create a local vector, I tried this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Vec local;&#xA;ierr = VecCreateSeq(PETSC_COMM_WORLD,10,&amp;amp;local);CHKERRQ(ierr);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where 10 is the size of the local vector I wish to create. &#xA;But running with 2 processors gives me an error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0]PETSC ERROR: --------------------- Error Message ------------------------------------&#xA;[0]PETSC ERROR: Invalid argument!&#xA;[0]PETSC ERROR: Cannot create VECSEQ on more than one process!&#xA;[0]PETSC ERROR: ------------------------------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So to be more precise,&#xA;I would like to write a toy code where I want to add all elements from a big vector &lt;code&gt;v&lt;/code&gt; which is spread across n processors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to do this &#xA;by gathering all elements from &lt;code&gt;v&lt;/code&gt; onto a vector local to Process 0 &#xA;and add them all up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also more generally how would Process 0 get elements from any specific processor&#xA;say processor &lt;code&gt;i&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3352" LastActivityDate="2013-05-19T04:02:24.503" Title="Creating a local vector in a parallel PETSC program" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7265" PostTypeId="2" ParentId="7264" CreationDate="2013-05-19T04:02:24.503" Score="2" Body="&lt;p&gt;&lt;code&gt;PETSC_COMM_WORLD&lt;/code&gt; is a &lt;em&gt;global&lt;/em&gt; communicator.  Use &lt;code&gt;PETSC_COMM_SELF&lt;/code&gt; to create a sequential object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatterCreateToZero.html&quot; rel=&quot;nofollow&quot;&gt;VecScatterCreateToZero&lt;/a&gt; for gathering to rank 0. (Obviously this is not memory scalable, so try not to do this in a real application.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatter.html&quot; rel=&quot;nofollow&quot;&gt;VecScatter&lt;/a&gt; for arbitrary (often sparse) collective vector distributions.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-05-19T04:02:24.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7266" PostTypeId="2" ParentId="7245" CreationDate="2013-05-19T09:38:51.943" Score="4" Body="&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; Your options are limited 1) go brute force adaptive for accurate and expensive solution 2) use numerical diffusion for a less accurate but stable solution or (my favorite) 3) leverage the fact that this is a singular perturbation problem and solve two inexpensive inner/outer problems and let matched asymptotics do its magic!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you really must obtain a uniform numerical solution to the problem, there is really not much you could do beyond adaptive mesh refinement. You are facing a singular perturbation problem that develops a boundary layer of thickness $\delta = \mathcal{O}(\sqrt{\epsilon})$ near the boundary. This boundary layer separates the inner and outer solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more precise, consider the left boundary, where $x = \mathcal{O}(\delta)$. In this region, it is possible to rescale the equation, by using &lt;em&gt;stretched&lt;/em&gt; coordinate $\eta = x/\delta$, for the inner solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-\Delta u_i + u_i = 1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subjected to the boundary conditions $u(0) = 0$ and $u_i(\eta\rightarrow\infty) = u_o(x\rightarrow0)$. For the outer solution, $u_o$, one assumes that $x = \mathcal{O}(1)$ and thus the &lt;em&gt;dominant balance&lt;/em&gt; is between $u$ and $1$ which simply gives $u_0 = 1$. With the outer solution at hand, you can now solve the &lt;em&gt;regularized&lt;/em&gt; inner solution with ease -- in this case even analytically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is in fact the technique that was (and still is) very popular for solving laminar boundary layer problems in fluid mechanics back in the day. In fact if you look at the Navier-Stokes equations, at high Reynolds numbers, you are effectively facing a singular perturbation problem, which like the one you mentioned here, develops a boundary layer (fun fact: the terms &quot;boundary layer&quot; in perturbation analysis actually comes from the fluid boundary layer problem I just described).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, back in the day where computational resources were very limited, the typical thing to do was to separate the outer and inner problems. For the outer, one would first solve the potential flow (analogous to $u_0 = 1$ here) and for the inner, well the boundary layer equations! All this without the needing fancy adaptive methods if you were to tackle the whole problem at once and in a single computational domain.&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="485" LastEditDate="2013-05-19T21:32:45.730" LastActivityDate="2013-05-19T21:32:45.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7267" PostTypeId="1" AcceptedAnswerId="7269" CreationDate="2013-05-19T12:21:17.210" Score="14" ViewCount="1792" Body="&lt;p&gt;I am trying to understand the advection equation with variable velocity coefficient a bit better. In particular I don't understand how the equation can be conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Advection&quot; rel=&quot;nofollow noreferrer&quot;&gt;advection equation&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} + \frac{\partial}{\partial x}(\boldsymbol{v}u) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's interpret $u(x,t)$ as being the concentration of some physical species ($cm^{-3}$) or some other physical quantity which cannot be created or destroyed. If we integrate $u(x,t)$ over our domain then we should get constant,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{x_{\text{min}}}^{x_{\text{max}}} u(x,t) dx = \text{constant}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is what I mean by being conservative.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we now let the velocity be a function of space (and time), $\boldsymbol{v}(x,t)$, then the chain rule must be applied to give,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} + \boldsymbol{v}\frac{\partial u}{\partial x} + \underbrace{u\frac{\partial \boldsymbol{v}}{\partial x}}_{\text{?}}  = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final term &quot;looks&quot; like a &lt;strong&gt;source term&lt;/strong&gt; and this is what I find confusing. It will increase or decrease the quantity $u$ depending on the divergence of the velocity field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5434/conservation-of-a-physical-quantity-when-using-neumann-boundary-conditions-appli&quot;&gt;question&lt;/a&gt;, I understand how to impose conservation boundary conditions. However, for the variable velocity advection equation &lt;strong&gt;I don't understand&lt;/strong&gt; how conservation boundary conditions can be derived because of the additional &quot;source term&quot; that is introduced by applying the chain rule. Can this equation be conservative? If so, how can correct boundary conditions be applied?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-05-20T03:10:17.030" Title="Can the advection equation with variable velocity be conservative?" Tags="|advection|conservation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7268" PostTypeId="1" CreationDate="2013-05-19T12:55:19.233" Score="5" ViewCount="2139" Body="&lt;p&gt;Note: this question was also posted in StackOverflow and math.stackexchange.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a set of 3D points defining a 3D contour, as shown below. The points in this contour lie in their best-fit plane and I want to obtain a 3D triangular mesh representation of the surface inside this contour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TfFOY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing some research I found that this is basically a minimal surface problem and its solution is related with the &lt;a href=&quot;http://mathworld.wolfram.com/BiharmonicEquation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biharmonic Equation&lt;/a&gt;. I also found that the &lt;a href=&quot;http://mathworld.wolfram.com/ThinPlateSpline.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thin-plate spline&lt;/a&gt; is the fundamental solution to this equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think the approach would be to try to fit this sparse representation of the surface (given by the 3D contour of points) using thin-plate splines. I found &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.9.0/reference/tutorial/interpolate.html#id1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example in scipy.interpolate&lt;/a&gt; where scattered data (x,y,z format) is interpolated using thin-plate splines to obtain the ZI coordinates on a uniform grid (XI,YI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two questions arise:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Would thin-plate spline interpolation be the correct approach for the problem of computing the surface from the set of 3D contour points?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If so, how to perform thin-plate interpolation on scipy with a non-uniform grid?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4345" LastEditorUserId="9667" LastEditDate="2019-12-01T15:28:45.287" LastActivityDate="2019-12-19T13:33:02.320" Title="Minimal surface solution in Python" Tags="|computational-geometry|interpolation|mesh-generation|geometry|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7269" PostTypeId="2" ParentId="7267" CreationDate="2013-05-19T13:45:14.427" Score="16" Body="&lt;p&gt;The fundamental quantity in transport is the &lt;em&gt;flux&lt;/em&gt;, $\mathbf v u$ for advection.  The divergence theorem states that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega \nabla\cdot (\mathbf v u) = \int_{\partial \Omega} (\mathbf v u) \cdot \mathbf n . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An equation is conservative when it is preserves this equality.  Dropping to 1D with $\Omega = (a,b)$ and using the equation $u_t + (\mathbf v u)_x = 0$, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Big(\int_a^b u\Big)_t = \int_a^b u_t = -\int_a^b (\mathbf v u)_x  = -\mathbf v u |_a^b $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the term on the right is just the difference in flux between the left and right boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your second observation, the non-conservative (non-divergence) form is misleading (and only justified for smooth solutions). The product $\mathbf v \cdot \nabla u$ is &lt;em&gt;not&lt;/em&gt; conservative transport if $\mathbf v$ is not divergence-free (i.e., constant in 1D). You should stick with the conservative form and resist the urge to apply the chain rule when evaluating conservation properties.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-05-19T13:45:14.427" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7270" PostTypeId="1" AcceptedAnswerId="7286" CreationDate="2013-05-19T14:45:25.867" Score="3" ViewCount="447" Body="&lt;p&gt;I'd like to program a trajectory planner, let's say for a robot, and I can pass acceleration commands to the robot, the robot can move in one dimension. The outcome of the planner is now a vector with accelerations, for example 10 accelerations, and since I set the accelerations with a frequency of 2, this will be sufficient to steer the robot for the next 5 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trajectory should be planned in a way that (a) minimizes jerk and (b) is as near at the desired velocity as possible. So I know what makes up my fitness function: sum over squared jerks and squared differences v, v_desired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I want to minimize jerk. I thought I search for a jerk profile (that is, jerk values at 10 discrete times, like 0.5, 1, ..., 5, to be consistent with the example above), integrate this twice so that I also have the speed profile, and apply my fitness function. The jerk profile is searched with a particle swarm optimizer. The nice thing is, 0 is already a good initialization for the profile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I thought about approximating the jerk profile with a polynomial, and not to search for the profile directly, but for the polynomial parameters. This way, integration would be trivial, everything would be steady and smooth, and I could sample points at whatever times I'd like. At least I thought this was a good idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: Are polynomials a good tool for this? I can't believe that estimating e.g. 5 parameters (and evaluating the resulting polynomial at e.g. 30 positions) will yield better results than searching for 30 points (the jerk) directly. My experiments from the last hours shows that the curves very quickly start looking like waves, and that's not what you want for a trajectory, especially not for the velocity profile ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my approach with &quot;estimate function parameters rather than sample points&quot; bad? Is a polynomial bad? Are there better functions, easily to extend in number of parameters and trivially to integrate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope my question is understandable, if not, please complain and I will try being more specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The picture below shows the result of my first approach, with &quot;optimizing&quot; parameters for a 4th degree polynomial for jerk. This was integrated with the correct initial values for a, v and s in my one-dimensional world. The initial v was set to 5.0, v desired is 2.0, and initial acceleration was 2.0. I would expect the velocity to first increase (caused by the initial acceleration), and then decrease slowly to 2. At point number 10, you can see the &quot;overshoot&quot; of v and the wiggly behaviour mentioned in the first comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/r8MH3.png&quot; alt=&quot;attempt at a trajectory&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4344" LastEditorUserId="127" LastEditDate="2013-05-19T15:19:42.663" LastActivityDate="2013-05-22T08:54:46.067" Title="Trajectory Planning: Polynomials as universal, integrable approximation functions?" Tags="|optimization|polynomials|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7271" PostTypeId="2" ParentId="7268" CreationDate="2013-05-19T15:13:47.327" Score="3" Body="&lt;p&gt;I don't think you can just interpolate. You may know the Fundamental Solution to the governing equations, but you have no data about the interior points of the problem, and I suspect that the solution will be poor far from the boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it's been awhile, but I usually associate the following equation with minimal surfaces (for small displacements, at least):&#xA;$$&#xA;(1+u_x^2)u_{yy} - 2u_x u_y u_{xy} + (1+u_y^2)u_{xx} = 0&#xA;$$&#xA;where $u(x,y)$ is the small displacement of the surface in question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could solve this by projecting your boundary points down into the $(x,y)$-plane and using their $z$ value as $u(x,y)$ on the boundary. Using the $(x,y)$ coordinates of the boundary points as the boundary of your domain, you could create a triangular mesh on the interior and then solve the above equation with above boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_15.html&quot; rel=&quot;nofollow&quot;&gt;deal.II's example #15&lt;/a&gt; does this, though I think you'd need a quadrilateral mesh rather than triangles. I can't tell from the figure, but if your boundary is a nominally a circle, then generating a quad mesh for it should be straightforward. If it's arbitrary, then you'll have to do some more work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There might also be a &lt;a href=&quot;http://fenicsproject.org/&quot; rel=&quot;nofollow&quot;&gt;FEniCS&lt;/a&gt; example for this if you prefer Python to C++, but I couldn't tell from a quick search around the web. There's also a quick &lt;a href=&quot;http://www.mathworks.com/help/pde/examples/minimal-surface-problem-on-the-unit-disk.html&quot; rel=&quot;nofollow&quot;&gt;MATLAB example&lt;/a&gt; using the PDE Toolbox if you have access to that. It will even do the mesh generation for you.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-05-19T15:13:47.327" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7272" PostTypeId="1" CreationDate="2013-05-19T17:44:06.137" Score="1" ViewCount="33" Body="&lt;p&gt;This title certainly needs some clarification: I need to compute parameters $a_i$ for a helper function $f(\vec{a};k)$ (for grid interpolation) which is fitted to a number of values $y_k$ which are subject to evolution dictated by a system of ODE's $d y_k/d t = F_k(t,\vec{y})$. The Levenberg-Marquardt fit of the values $y_k$ to the function $f(\vec{a};k)$ is repeated in each time step $t_j$. The precise values found for the parameters $a_{i,j}$, which are varied in this fit, does not matter, but I found that in the presence of adaptive step-size control for the ODE this simple algorithm can lead to instabilities as it influences the value of the rhs-functions $F_k(t,\vec{y})$ at higher order. For the present purpose I can assume that the evolution of the parameters is smooth plus some noise $a_{i,j} = A_i(t_j)+\delta_{i,j}$ and the model for the interpolating functions $f(\vec{a};k)$ does not matter. My idea to fix the stability issues is therefore to make the parameters $a_i$ subject to some smoothening procedure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concept for this procedure in turn is to estimate the new values $a_{i,J}$ based on the previous ones $a_{i,j}$, $j&amp;lt;J$ and the new $\chi^2$ fit by adding to the function to be minimized in the fit a contribution proportional to $\sum_l \sigma_l (A_l(t_J)-a_{l,J})^2$. Here $\sigma_l$ are positive constant coefficients and $A_l(t_J)$ are the values suggested by the trend of the previous ones (only loosely related to $A_l(t_J)$ above). The obtained $a_{l,J}$ would automatically be consistent with constraints in the fit. (Note that this could be different if I first fit and later smooth) The question is which method to use to predict (to extrapolate) the values $A_l(t_J)$. I wanted to avoid using a simple polynomial or spline fit, because I definitely need to avoid overshooting. On the other hand a second $\chi^2$ fit of a model function seems to overdo it. Also due to the fact that the previous values, $a_{i,j}$, $j&amp;lt;J$, where themselves found based on the combined fit and I am not sure that this does not introduce any bias. What do you think? I also wondered if I could miss use a Sawatzki-Golay smoothing filter (assuming as an approximation that $t_{i+1}-t_i=const$ to avoid the overhead of a full $\chi^2$ fit. But, can it be used for extrapolation?&lt;/p&gt;&#xA;" OwnerUserId="1943" LastActivityDate="2013-05-19T17:44:06.137" Title="Combining trend estimation and constrained Marquart fit" Tags="|ode|interpolation|smoothing|extrapolation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7273" PostTypeId="1" AcceptedAnswerId="7276" CreationDate="2013-05-19T19:15:47.933" Score="4" ViewCount="118" Body="&lt;p&gt;When SUPG/PSPG stabilization is added to momentum equation of flow problem, is needed stabilization for energy equation also? I would guess that when stabilization for velocity works fine so one gets velocity without spurious wiggles and assuming that thermal diffusivity is large enough then answer is no. How are these two conditions quantified in the language of dimensionless quantites?&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2013-05-19T21:14:27.033" Title="Is stabilization of energy equation needed when momentum equation needs it?" Tags="|fluid-dynamics|stability|advection|navier-stokes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7274" PostTypeId="1" AcceptedAnswerId="7275" CreationDate="2013-05-19T19:50:12.307" Score="0" ViewCount="152" Body="&lt;p&gt;I'm trying to model a spring damper system from a tutorial that I've found &lt;a href=&quot;http://gamedev.tutsplus.com/tutorials/implementation/make-a-splash-with-2d-water-effects/&quot; rel=&quot;nofollow&quot;&gt;on this site&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use the exact same parameters as the ones in the tutorial the system is not stable. I've downloaded the code and verified that the parameters were the same as mine.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = (k * u) - (d * v);&#xA;v = v + a;&#xA;s = s + v;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jsfiddle.net/bM6Eh/2/&quot; rel=&quot;nofollow&quot;&gt;jsfiddle available here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can make it stable by adding a factor 0.3, like so&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s = 0.3 * s + v&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I would like to know how to make it stable without hacking it. I'm aware that the integrator is not good, but it should still work for this case, because it does so in the demo.&lt;/p&gt;&#xA;" OwnerUserId="4347" OwnerDisplayName="stackoverflownewbie" LastEditorUserId="127" LastEditDate="2013-05-19T20:25:07.260" LastActivityDate="2013-05-19T20:42:08.570" Title="Spring damper model does not work very well" Tags="|ode|simulation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7275" PostTypeId="2" ParentId="7274" CreationDate="2013-05-19T20:13:12.393" Score="1" Body="&lt;p&gt;Your spring model is fine. It is the numerical integration that needs attention. What you have above is &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler%27s_method&quot; rel=&quot;nofollow&quot;&gt;Euler's&lt;/a&gt; method for numerical integration with $\Delta t=1$. This method is known to be unstable. You made it stable with artificial (numerical) damping. For a simple spring damper system try to use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Midpoint_method&quot; rel=&quot;nofollow&quot;&gt;mid-point&lt;/a&gt; method or better yet a four step &lt;a href=&quot;http://www.ctcms.nist.gov/~langer/oof2man/RegisteredClass-RK4.html&quot; rel=&quot;nofollow&quot;&gt;Runge-Kutta&lt;/a&gt; method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information &lt;a href=&quot;http://www.physics.drexel.edu/students/courses/Comp_Phys/Integrators/simple.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://www.physics.drexel.edu/students/courses/Comp_Phys/Integrators/rk4.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your code would like something like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// time step h=1 or less&#xA;&#xA;a1 = (k*u) - (d*v)&#xA;u2 = u + (h/2)*v&#xA;v2 = v + (h/2)*a1&#xA;a2 = (k*u2) - (d*v2)&#xA;&#xA;u = u + h*v2&#xA;v = v + h*a2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1030" OwnerDisplayName="ja72" LastEditorUserId="1030" LastEditDate="2013-05-19T20:42:08.570" LastActivityDate="2013-05-19T20:42:08.570" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7276" PostTypeId="2" ParentId="7273" CreationDate="2013-05-19T21:14:27.033" Score="5" Body="&lt;p&gt;Look at the cell Péclet number&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{Pe}_h = h v / K$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h$ is mesh size, $v$ is the magnitude of velocity, and $K$ is diffusivity.  It is analogous to cell Reynolds number for the momentum equation and is small when &quot;thermal diffusivity is large compared to advection&quot;.  It is common common in macro-scale fluid dynamics that thermal diffusivity $K$ is very small.  If the cell Péclet is much larger than 1, you need stabilization.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-05-19T21:14:27.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7277" PostTypeId="2" ParentId="7258" CreationDate="2013-05-20T03:06:01.657" Score="1" Body="&lt;p&gt;You're on the right track already. To calculate the first derivatives you wanted using the results you calculated, you used the chain rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the second derivatives you need using the information you have, you're essentially using the chain rule twice. &lt;a href=&quot;http://en.wikipedia.org/wiki/Fa%C3%A0_di_Bruno%27s_formula&quot; rel=&quot;nofollow&quot;&gt;Faà di Bruno's formula&lt;/a&gt; is a formula for the chain rule that you can use in this situation.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-05-20T03:06:01.657" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7278" PostTypeId="1" AcceptedAnswerId="7279" CreationDate="2013-05-20T03:32:35.283" Score="8" ViewCount="878" Body="&lt;p&gt;Following on from the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7267/can-the-advection-equation-with-variable-velocity-be-conservative&quot;&gt;earlier question&lt;/a&gt; I am trying to derive a finite-difference scheme for the advection equation which is conservative. It was suggested that for advection equation with variable velocity,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial x} + \frac{\partial}{\partial x}\left( \boldsymbol{v}(x)u\right) = 0&#xA;$$&#xA;that the chain rule should &lt;em&gt;not&lt;/em&gt; be applied if conservation is required because the flux of $u$ is the fundamental quantity.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The product $\boldsymbol{v}(x)⋅∇u$ is not conservative transport if $\boldsymbol{v}(x)$ is not divergence-free (i.e., constant in 1D). You should stick with the conservative form and resist the urge to apply the chain rule when evaluating conservation properties.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the following a reasonable approach for discretization?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have kept the flux together (i.e the product $\boldsymbol{v}_ju_j$) while applying the discretization. &lt;/li&gt;&#xA;&lt;li&gt;However to write as a matrix equation where $u_j$ is the unknown they eventually must be separated.&lt;/li&gt;&#xA;&lt;li&gt;I have a slight concern that this is incorrect because the only difference between the constant velocity case and the varying velocity case is that that $\boldsymbol{v}$ becomes a vector.&lt;/li&gt;&#xA;&lt;li&gt;If we applied the chain rule we could get exactly the same equation but with the additional $u \frac{\partial\boldsymbol{v(x)}}{\partial x}$ term which is treated as a source term.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, if we apply Crank-Nicolson scheme to the advection equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{u_{j}^{n+1} - u_{j}^{n}}{\Delta t} + \left[ \frac{1-\beta}{2\Delta x} \left( \boldsymbol{v}_{j+1}^{n}  u_{j+1}^{n} - \boldsymbol{v}_{j-1}^{n} u_{j-1}^{n} \right) + \frac{\beta}{2\Delta x} \left( \boldsymbol{v}_{j+1}^{n+1} u_{j+1}^{n+1} - \boldsymbol{v}_{j-1}^{n+1} u_{j-1}^{n+1} \right) \right] = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the following substitution,&#xA;$$ \boldsymbol{r}_j^n = \frac{\boldsymbol{v}_j^n}{2}\frac{\Delta t}{\Delta x}&#xA;$$&#xA;and move the $n+1$ terms to the l.h.s gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_{j}^{n+1} + \beta\boldsymbol{r}_{j+1}^{n+1} u_{j+1}^{n+1} - \beta\boldsymbol{r}_{j-1}^{n+1} u_{j-1}^{n+1}  = u_{j}^{n} - (1-&#xA;\beta)\boldsymbol{r}_{j+1}^{n}  u_{j+1}^{n} + (1-&#xA;\beta)\boldsymbol{r}_{j-1}^{n} u_{j-1}^{n} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can write this as the matrix equation,&#xA;$$&#xA;    \begin{pmatrix}&#xA;      1      &amp;amp; \beta r_2^{n+1}		&amp;amp;     				&amp;amp;       	0		\\&#xA;      -\beta r_1^{n+1}	&amp;amp; 1 		&amp;amp; \beta r_3^{n+1}	&amp;amp;     	    		\\&#xA;              			&amp;amp;  \ddots   		&amp;amp;   \ddots  		&amp;amp; \ddots			\\&#xA;						&amp;amp; -\beta r_{J-2}^{n+1}	&amp;amp; 1		&amp;amp; \beta r_J^{n+1}	\\&#xA;                 0      &amp;amp;      				&amp;amp; -\beta r_{J-1}^{n+1}	&amp;amp; 1  		\\&#xA;   \end{pmatrix}&#xA;   \begin{pmatrix}&#xA;   	u_{1}^{n+1} \\&#xA;	u_{2}^{n+1} \\&#xA;	\vdots \\&#xA;	u_{J-1}^{n+1} \\&#xA;	u_{J}^{n+1}&#xA;   \end{pmatrix} =&#xA;    \begin{pmatrix}&#xA;      1		&amp;amp; -(1 - \beta)r_2^n	  	&amp;amp;     						&amp;amp;       	0		 	\\&#xA;      (1 - \beta) r_1^n	&amp;amp; 1 			&amp;amp; -(1 - \beta)r_3^n	  	&amp;amp;		     	    	\\&#xA;              			   	&amp;amp;  \ddots   				&amp;amp;   \ddots  				&amp;amp; \ddots				\\&#xA;						   	&amp;amp; (1 - \beta) r_{J-2}^n		&amp;amp; 1			&amp;amp; -(1 - \beta)r_J^n	\\&#xA;                 0         	&amp;amp;      						&amp;amp; (1 - \beta) r_{J-1}^n		&amp;amp; 1		\\&#xA;   \end{pmatrix}&#xA;   \begin{pmatrix}&#xA;   	u_{1}^{n} \\&#xA;	u_{2}^{n} \\&#xA;	\vdots \\&#xA;	u_{J-1}^{n} \\&#xA;	u_{J}^{n}&#xA;   \end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update I&lt;/strong&gt; Corrected a sign error suggested by @Geoff-Oxberry.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-05-21T14:53:21.150" Title="Conservative finite-difference expression for the advection equation" Tags="|finite-difference|hyperbolic-pde|advection|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7279" PostTypeId="2" ParentId="7278" CreationDate="2013-05-20T04:14:22.063" Score="4" Body="&lt;p&gt;In 1-D, the first discretization you've presented is correct. The matrix equation does not look right, though. For starters, it's not clear what your boundary conditions would be or how you would incorporate them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In $N$ dimensions, the advection equation looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial{u}}{\partial{t}} + \sum_{i=1}^{N}\frac{\partial}{\partial{x^{i}}}(v^{i} u) &amp;amp;= 0,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;adopting the convention that superscripts refer to components of vectors; you would discretize this equation in similar fashion to the 1-D case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $v$ (or in $N$-D, $\mathbf{v}$) is a known function of $x$ (respectively, $\mathbf{x}$), then the values $v_{j}$ (resp., $\mathbf{v}_{j}$) become coefficients for the various $u_{j}$ terms in the discrete equation.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-05-20T04:44:24.490" LastActivityDate="2013-05-20T04:44:24.490" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7280" PostTypeId="1" CreationDate="2013-05-20T05:13:28.853" Score="4" ViewCount="395" Body="&lt;p&gt;I have obtained some data from neutron diffraction for some material samples.  The &quot;rawest&quot; form of the data is the structure function $S(Q)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can choose a variety of different Q-maxes when Fourier transforming the data into $g(r)$, along with applying something like a Lorch function to smooth it out.  Right now the whole process is done by eyeballing it and using whichever curve ends up looking the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, it turns out there may be some long-range order at high values of $r$, which — if this order really exists — will lead to a very exciting paper.  However, I want to be absolutely certain that what we're seeing is not some Fourier transform artifact or some kind of systematic artifact.  However, I'm a computational modeler, not an experimentalist, so I don't really know how to verify this mathematically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would like the $g(r)$ curve to have an envelope around it, the &quot;error envelope&quot; that represents some confidence interval.  But I don't know the process of getting it, and in fact, a literature search reveals no one may have even done this before (though you'd think all experimentally obtained data would have error bars nowadays).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?  Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="4362" OwnerDisplayName="Nick" LastEditorUserId="6663" LastEditDate="2013-12-26T16:46:27.047" LastActivityDate="2013-12-28T21:50:45.353" Title="Error bars for pair-correlation function" Tags="|fourier-analysis|error-estimation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7281" PostTypeId="1" AcceptedAnswerId="7282" CreationDate="2013-05-20T07:18:34.533" Score="7" ViewCount="5671" Body="&lt;p&gt;I am continuing my dive into computational fluid dynamics. I would like to build a simple test case for modelling the flow around a cylinder in FEniCS and then continue to model turbulence in order to observe the &lt;a href=&quot;http://en.wikipedia.org/wiki/K%C3%A1rm%C3%A1n_vortex_street&quot;&gt;Karman vortex street&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a newby in both FEniCS and CFD. I have started to go through the FEniCS book and went through the first simple scripts in the tutorial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this is a standard problem and I do not like to re-invent the wheel, my question is: is there some tutorial available on how to do this? Even a not-well commented code / FEniCS script would do as a reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have googled this and did not get much out of it, which is why I ask here.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="3748" LastEditDate="2013-05-20T10:12:26.790" LastActivityDate="2017-07-19T00:11:43.107" Title="Tutorial for flow around a cylinder in FEniCS" Tags="|fluid-dynamics|fenics|vorticity|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7282" PostTypeId="2" ParentId="7281" CreationDate="2013-05-20T07:39:38.270" Score="11" Body="&lt;p&gt;I suggest you start by looking at the FEniCS Navier-Stokes demo which is documented here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/1.2.0/python/demo/pde/navier-stokes/python/documentation.html&quot;&gt;http://fenicsproject.org/documentation/dolfin/1.2.0/python/demo/pde/navier-stokes/python/documentation.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your specific test case, you might want to look at the NSBENCH set of test problem (which are described in Chapter 21 of the FEniCS Book). The code for that book chapter is available at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://launchpad.net/nsbench/&quot;&gt;http://launchpad.net/nsbench/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code for your test case (flow around a cylinder) can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://bazaar.launchpad.net/~nsbench/nsbench/main/view/head:/problems/cylinder.py&quot;&gt;http://bazaar.launchpad.net/~nsbench/nsbench/main/view/head:/problems/cylinder.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this code depends on the interface defined by the solvers in NSBENCH, but it might still be useful to look at for how to set the boundary conditions, and how to design an interface for your own solver.&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-05-20T07:39:38.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7284" PostTypeId="2" ParentId="4833" CreationDate="2013-05-20T11:36:59.997" Score="3" Body="&lt;p&gt;As far as I can tell, indexing a simple cubic lattice in the way you require should be easy. If we let $i, j, k$ be the indices along the $x, y, z$ directions, we can easily map these to a linear index $n$ like so:&#xA;$$ n = ((k  N_y) + j)  N_x + i$$&#xA;Next, we need to accomodate 8 atoms per unit cell. The easiest way to do this should be to add a stride of 8 to the equation above and let $l = \{0, 1, ..., 7\}$ be the index describing which atom within the unit cell we are referring to,&#xA;$$ n = (((k  N_y) + j)  N_x + i) \times 8 + l.$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To find the $z$ coordinate is relatively easy: First find the $z$ coordinate of the unit cell in question, which is $\lfloor n / (8 N_x N_y) \rfloor$ times the lattice constant, then check $l = n \; \text{mod} \; (8 )$ to find out which atom within the unit cell we are dealing with and adjust $z$ accordingly.&lt;/li&gt;&#xA;&lt;li&gt;Finding the nearest neighbours is somewhat more painful, but on the upside, it should be an efficient computation. Unless you can find a clever way, you may have to code something specifically for each value of $l$...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4349" LastEditorUserId="-1" LastEditDate="2013-07-19T22:26:33.937" LastActivityDate="2013-07-19T22:26:33.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7285" PostTypeId="1" AcceptedAnswerId="7292" CreationDate="2013-05-20T11:50:13.337" Score="1" ViewCount="351" Body="&lt;p&gt;while executing my script i get this error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Traceback (most recent call last):&#xA;  File &quot;/media/oss/Fenics/demo/box_3delasticity.py&quot;, line 33, in &#xA;    mesh = Box (0, 0, 0, 1, 1, 1, 1, 1, 1)&#xA;  File &quot;/usr/lib/python2.7/dist-packages/dolfin/cpp/mesh.py&quot;, line 8698, in &lt;strong&gt;init&lt;/strong&gt;&#xA;    _mesh.Box_swiginit(self,_mesh.new_Box(*args)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know exactly how to fix this bug??&lt;/p&gt;&#xA;" OwnerUserId="4350" LastActivityDate="2013-05-20T14:08:22.123" Title="Fenics: problem with the mesh generation" Tags="|fenics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7286" PostTypeId="2" ParentId="7270" CreationDate="2013-05-20T12:10:45.800" Score="2" Body="&lt;p&gt;Ok, I experimented a lot today and using splines definitely looks promising. Although a little bit more computational effort to setup the splines (inversion of a matrix and so on, but I used Eigen3 for that), integration is rather trivial. With the polynomial representation, I can do a lot analytically, for example, the costs for deviation in velocity will become a difference of polynomials (one of a piecewise aka. spline and one of zeroth degree, constant), and it's exactly one step to get the costs, no need to draw samples at discrete points and sum them up. Don't know if I could need that, but it has been a good training at least.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this is the first result my optimizer came up with, I optimized the support points for the splines (5 in this example, just conincidence that it's also the chosen time horizon on the horizontal axis).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/o9CEG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compared with the first version with the high degree polynomial, this looks promising!&lt;/p&gt;&#xA;" OwnerUserId="4344" LastActivityDate="2013-05-20T12:10:45.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7287" PostTypeId="1" CreationDate="2013-05-20T12:20:05.747" Score="2" ViewCount="475" Body="&lt;p&gt;I am doing a small project at school. I have done my code implementations in CUDA and did some performance measurements with real values, i.e. running the program with different number of threads, changing the size of the problem, changing both, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results of (time, speedup) look really nice. But then I was asked to do some theoretical analysis of the algorithms, specifically do a prediction of the time it will take my implementation to run as the number of threads increases. The problem was that I received some documents where they use &quot;Timing diagrams&quot; to try to model this, but this approach was for problems solved using MPI and I'm being asked to use &lt;strong&gt;exactly&lt;/strong&gt; the same stuff to create a prediction model for CUDA. Is it possible to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do a prediction of how a given CUDA program (kernel) will run as I increase the number of threads used for computation? Say that my GPU has 240 cuda cores, now for sake of simplicity I am taking that if I launch a kernel with one thread that kernel will use only 1 core, if I launch 4 threads it'll use 4 cores up to a limit of 240 in that case when I launch 500 threads there won't be no more than 240 cores in use. It is highly likely that this is somehow wrong but It's something to start with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please give me some ideas to sketch this prediction model? I am really cracking my head because of this, I cannot came to a simple prediction model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The very first problem was how to simulate a sequential running on the GPU in CUDA. As mentioned before what I did was launch a kernel with only one block and one thread, the result was what I expected as it turned out that the kernel ran slower. Then increasing the number of threads launched reduced the running time so I was getting benefit from the parallel architecture of the GPU. Now, the key question is &lt;strong&gt;launching K threads in CUDA, can I be sure that the number of cores used in the GPU equals the number of K threads?&lt;/strong&gt; Somehow this is true because of what I said before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now moving forward I have to make a prediction model, based on timing diagrams. So first the sketch of the timing diagram is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/YkedO.jpg&quot; alt=&quot;Timing diagram for computing Mandelbrot Set in GPU&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What my diagram shows (or at least tries to show) is that the critical path for the application to run depends on&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Sending data from CPU to GPU&lt;/li&gt;&#xA;&lt;li&gt;CUDA initialization and kernel launch configuration&lt;/li&gt;&#xA;&lt;li&gt;The time it takes the last warp to execute&lt;/li&gt;&#xA;&lt;li&gt;Sending data from GPU to CPU&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The time of (1) and (4) can be straight forward computed: I know the GPU is connected to the CPU via PCI-Express 2nd gen and knowing the size of the problem I can know the actual time of transfers. Good :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time of (2) is hardware dependent, but for what I've been reading it takes around 60 to 65 ms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally (3) the hardest part to predict is the actual running time of the computation (kernel runtime for a given number of threads). The way I modeled this was &lt;strong&gt;Tprocessing = Tcomputation + Tmemory&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tcomputation : number of cycles one thread does to compute 1 pixel of the set. The key question here is &lt;strong&gt;How to count the number of cycles?&lt;/strong&gt; I naively counted the number of sums and multiplications and then multiplied it by the cycles each of them takes according to the architecture of the GPU. I am completely uncertain about this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tmemory : number of cycles one thread takes to go to global memory to read/write. From various references (including Udacity course on GPU) is no average 500 cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we know the frequency at which the GPU clock works and summing all I would have an upper bound of the time it will take my program to run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically everything is based on the idea that if I run one thread only one CUDA core is used, if I run ten threads then ten CUDA cores are used. Then I can predict what will is likely to happen if we have a GPU with 2048 CUDA cores as the GTX Titan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I test my model with the Mandelbrot Set and well my timing was much bigger than the actual one, so something is missing.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2013-05-20T14:30:02.243" LastActivityDate="2013-05-20T14:30:02.243" Title="Prediction model on GPU" Tags="|parallel-computing|performance|gpu|cuda|" AnswerCount="0" CommentCount="7" ClosedDate="2013-05-23T22:39:55.697" ContentLicense="CC BY-SA 3.0" />
  <row Id="7288" PostTypeId="1" AcceptedAnswerId="7291" CreationDate="2013-05-20T13:09:36.880" Score="13" ViewCount="2175" Body="&lt;p&gt;My system is a symmetric FE problem with lagrange multipliers (e.g. incompressible Stokes' flow):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{pmatrix}A &amp;amp; B^T \\ B &amp;amp; C\end{pmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C = 0$ is the typical case (I have even made sure that the equations are numbered so that the Lagrange multipliers appear last ). The system is quite large (+100k lines).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having read the answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/619/iterative-methods-for-indefinite-systems-without-block-structure&quot;&gt;this question&lt;/a&gt;, I was given the impression that there are suitable preconditioners that can be used for mixed FE-problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using PETSc, I've managed to solve the system with MINRES (&lt;code&gt;-ksp_type minres -pc_type none -mat_type sbaij&lt;/code&gt; ), although the precision isn't great (causing several Newton-iterations for a linear problem). No other combination of preconditioner and ksp-solver seems to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is there any combination of flags for PETSc that will solve this system faster than with just MINRES?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4308" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2022-05-03T23:05:31.923" Title="Which preconditioners (and solver) in PETSc for indefinite symmetric systems should I use?" Tags="|finite-element|petsc|krylov-method|preconditioning|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7289" PostTypeId="2" ParentId="7288" CreationDate="2013-05-20T13:36:46.337" Score="3" Body="&lt;p&gt;You do want a preconditioner, and the construction of one is discussed here:&#xA;&lt;a href=&quot;http://www.math.colostate.edu/%7Ebangerth/videos.676.38.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.676.38.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2022-05-03T23:05:31.923" LastActivityDate="2022-05-03T23:05:31.923" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7290" PostTypeId="2" ParentId="7263" CreationDate="2013-05-20T13:59:13.283" Score="1" Body="&lt;p&gt;In most research settings, you do not have access to professional programmers with relatively uniform skill level whose time you can dictate. Rather, you have a mix of experienced programmers (maybe postdocs, maybe faculty) and grad students with little programming experience and no experience at all regarding large-scale projects. Furthermore, all of these actors have numerous other duties (teaching, TAing, reviewing papers, sitting on committees) that vary greatly throughout time. As a consequence, I think it's going to be very difficult to have accurate estimates. Certainly, my experience is that &quot;it's going to take longer than I thought&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-20T13:59:13.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7291" PostTypeId="2" ParentId="7288" CreationDate="2013-05-20T14:03:35.777" Score="14" Body="&lt;h3&gt;Warning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Solving saddle point problems involves a lot more choices than definite problems, and there are a lot more things that can go wrong.  Use monitors for all levels to debug convergence, to sure that null spaces are handled correctly when auxiliary operators are singular (usually just a constant null space), and to ensure that preconditioners are stable.  These techniques are discussed in &lt;a href=&quot;https://scicomp.stackexchange.com/q/513&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Solving saddle point problems&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You can start with the section on solving block matrices in the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;User's Manual&lt;/a&gt;.  For Stokes-like problems, you can build a Schur complement preconditioner using &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/PC/PCFIELDSPLIT.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCFIELDSPLIT&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_detect_saddle_point&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This may be combined with a &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/PC/PCLSC.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Least Squares Commutator&lt;/a&gt; preconditioner for the Schur complement (&lt;code&gt;-fieldsplit_1_pc_type lsc&lt;/code&gt;).  It is common to &lt;code&gt;-fieldsplit_1_ksp_type preonly&lt;/code&gt; and let the outer iteration do most of the work.  The block-triangular variants are popular when used in this way, e.g., &lt;code&gt;-pc_fieldsplit_schur_fact_type upper&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find further details on solver composition using options options in &lt;a href=&quot;http://www.computer.org/csdl/proceedings/ispdc/2012/4805/00/4805a055-abs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;our paper&lt;/a&gt; &lt;a href=&quot;http://www.mcs.anl.gov/uploads/cels/papers/P2017-0112.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(preprint)&lt;/a&gt;, which also discusses commuting multigrid with the block decomposition (putting the fieldsplit inside multigrid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For many problems, you'll want to customize the field decomposition and Schur complement preconditioners, many of which involve auxiliary operators.  For example, the &quot;pressure convection-diffusion&quot; (PCD) method from Elman et al requires discretization of an auxiliary operator in the pressure space.  See examples of &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/PC/PCSHELL.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCSHELL&lt;/a&gt; and the user's manual section for this purpose.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-05-20T14:03:35.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7292" PostTypeId="2" ParentId="7285" CreationDate="2013-05-20T14:08:22.123" Score="1" Body="&lt;p&gt;Try this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh = BoxMesh(0, 0, 0, 1, 1, 1, 1, 1, 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The name of the class has been changed from Box to BoxMesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The class Box is now something different: a geometric primitive which can be used for defining and geometries used as input for the DOLFIN mesh generator.&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-05-20T14:08:22.123" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7293" PostTypeId="2" ParentId="7281" CreationDate="2013-05-20T17:49:14.393" Score="5" Body="&lt;p&gt;This is I think rotating cylinder and Karman vortex street behind it. You can also uncomment SUPG/PSPG stabilization. But I suspect that mesh size &lt;code&gt;h&lt;/code&gt; computed as 2*circumradius is not good quantity and prefer cell diameter computed this way &lt;a href=&quot;https://scicomp.stackexchange.com/a/7181/4254&quot;&gt;https://scicomp.stackexchange.com/a/7181/4254&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;import time&#xA;&#xA;#set_log_level(PROGRESS)&#xA;parameters['form_compiler']['representation'] = 'quadrature'&#xA;parameters['form_compiler']['optimize'] = True&#xA;parameters['form_compiler']['cpp_optimize'] = True&#xA;&#xA;center = Point(0.2, 0.2)&#xA;radius = 0.05&#xA;&#xA;class Cylinder(SubDomain):&#xA;    def inside(self,x,on_boundary):&#xA;        r = sqrt((x[0] - center[0])**2 + (x[1] - center[1])**2)&#xA;        return(on_boundary and (r &amp;lt; 2*radius + sqrt(DOLFIN_EPS)))&#xA;    def snap(self, x):&#xA;        r = sqrt((x[0] - center[0])**2 + (x[1] - center[1])**2)&#xA;        if r &amp;lt;= radius:&#xA;            x[0] = center[0] + (radius / r)*(x[0] - center[0])&#xA;            x[1] = center[1] + (radius / r)*(x[1] - center[1])&#xA;&#xA;class Walls(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        return (on_boundary and ((x[1] &amp;lt; DOLFIN_EPS) or (x[1] &amp;gt; (0.41 - DOLFIN_EPS))))&#xA;&#xA;class Inflow(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        return (on_boundary and (x[0] &amp;lt; DOLFIN_EPS))&#xA;&#xA;class Outflow(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        return (on_boundary and (x[0] &amp;gt; (2.2 - DOLFIN_EPS)))&#xA;&#xA;# Parameters&#xA;nu = Constant(0.001)&#xA;dt = 0.1&#xA;idt = Constant(1./dt)&#xA;t_end = 10.0&#xA;theta=0.5   # Crank-Nicholson timestepping&#xA;&#xA;# Mesh&#xA;mesh = Mesh(&quot;bench_L1.xml&quot;)&#xA;boundary_parts = MeshFunction(&quot;size_t&quot;, mesh, mesh.topology().dim()-1)&#xA;boundary_parts.set_all(0)&#xA;Gamma = Cylinder()&#xA;Gamma.mark(boundary_parts, 1)&#xA;&#xA;#refine the mesh and snap boundary&#xA;mesh = refine(mesh)&#xA;mesh.snap_boundary(Gamma, False)&#xA;&#xA;# Define function spaces (Taylor-Hood)&#xA;V = VectorFunctionSpace(mesh, &quot;CG&quot;, 2)&#xA;P = FunctionSpace(mesh, &quot;CG&quot;, 1)&#xA;W = MixedFunctionSpace([V, P])&#xA;&#xA;# No-slip boundary condition for velocity&#xA;noslip = Constant((0, 0))&#xA;bc0 = DirichletBC(W.sub(0), noslip, Walls())&#xA;&#xA;vc = Expression((&quot;-cos(atan2(x[0]-0.2,x[1]-0.2))&quot;,&quot;sin(atan2(x[0]-0.2,x[1]-0.2))&quot;))&#xA;bc_cylinder = DirichletBC(W.sub(0), vc, Cylinder())&#xA;&#xA;# Inflow boundary condition for velocity and temperature&#xA;v_in = Expression((&quot;1.5 * 4.0 * x[1] * (0.41 - x[1]) / ( 0.41 * 0.41 )&quot;,&quot;0.0&quot;))&#xA;bc1 = DirichletBC(W.sub(0), v_in, Inflow())&#xA;&#xA;# Collect boundary conditions&#xA;bcs = [bc_cylinder, bc0, bc1]&#xA;&#xA;# Define unknown and test function(s)&#xA;(u, q) = TestFunctions(W)&#xA;&#xA;n = FacetNormal(mesh)&#xA;I = Identity(V.cell().d)    # Identity tensor&#xA;&#xA;# current time step&#xA;w = Function(W)&#xA;(v, p) = (as_vector((w[0], w[1])), w[2])&#xA;D = 0.5*(grad(v)+grad(v).T)&#xA;T = -p*I + 2.0*nu*D&#xA;&#xA;# previous time step&#xA;w0 = Function(W)&#xA;(v0, p0) = (as_vector((w0[0], w0[1])), w0[2])&#xA;D0 = 0.5*(grad(v0)+grad(v0).T)&#xA;T0 = -p*I + 2.0*nu*D0&#xA;&#xA;# Define variational forms without time derivative in previous time&#xA;F0_eq1 = (inner(T0, grad(u)))*dx + inner(grad(v0)*v0, u)*dx&#xA;F0_eq2 = 0*q*dx&#xA;F0 = F0_eq1 + F0_eq2&#xA;&#xA;# variational form without time derivative in current time&#xA;F1_eq1 = (inner(T, grad(u)) + inner(grad(v)*v, u))*dx&#xA;F1_eq2 = q*div(v)*dx&#xA;F1 = F1_eq1 + F1_eq2&#xA;&#xA;# combine variational forms with time derivative&#xA;#&#xA;#  dw/dt + F(t) = 0 is approximated as&#xA;#  (w-w0)/dt + (1-theta)*F(t0) + theta*F(t) = 0&#xA;#&#xA;F = idt*inner((v-v0),u)*dx + (1.0-theta)*F0 + theta*F1&#xA;&#xA;# residual of strong Navier-Stokes&#xA;r = idt*(v-v0) + theta*grad(v)*v + (1.0-theta)*grad(v0)*v0 \&#xA;   - theta*div(T) - (1.0-theta)*div(T0)&#xA;&#xA;# stabilization parameter&#xA;h = CellSize(mesh)&#xA;velocity = v0&#xA;vnorm = sqrt(dot(velocity, velocity))&#xA;tau = ( (2.0*idt)**2 + (2.0*vnorm/h)**2 + (4.0*nu/h**2)**2 )**(-0.5)&#xA;&#xA;# add SUPG stabilization&#xA;#F += tau*inner(grad(u)*v, r)*dx&#xA;&#xA;# add PSPG stabilization&#xA;#F += tau*inner(grad(q), r)*dx&#xA;&#xA;# define Jacobian&#xA;J = derivative(F, w)&#xA;&#xA;# Create files for storing solution&#xA;ufile = File(&quot;results/velocity.pvd&quot;)&#xA;pfile = File(&quot;results/pressure.pvd&quot;)&#xA;&#xA;# create variational problem and solver&#xA;problem = NonlinearVariationalProblem(F, w, bcs, J)&#xA;solver  = NonlinearVariationalSolver(problem)&#xA;solver.parameters['newton_solver']['maximum_iterations'] = 20&#xA;&#xA;# Time-stepping&#xA;t = dt&#xA;while t &amp;lt; t_end:&#xA;&#xA;    print &quot;t =&quot;, t&#xA;&#xA;    # Compute&#xA;    begin(&quot;Solving ....&quot;)&#xA;    solver.solve()&#xA;    end()&#xA;&#xA;    # Extract solutions:&#xA;    (v, p) = w.split()&#xA;&#xA;    # Plot&#xA;    #plot(v)&#xA;&#xA;    # Save to file&#xA;    ufile &amp;lt;&amp;lt; v&#xA;    pfile &amp;lt;&amp;lt; p&#xA;&#xA;    # Move to next time step&#xA;    w0.assign(w)&#xA;    t += dt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and mesh file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;?xml version=&quot;1.0&quot;?&amp;gt;&#xA;&amp;lt;dolfin xmlns:dolfin=&quot;http://fenicsproject.org&quot;&amp;gt;&#xA;  &amp;lt;mesh celltype=&quot;triangle&quot; dim=&quot;2&quot;&amp;gt;&#xA;    &amp;lt;vertices size=&quot;147&quot;&amp;gt;&#xA;      &amp;lt;vertex index=&quot;0&quot; x=&quot;0&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;1&quot; x=&quot;2.2&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;2&quot; x=&quot;2.2&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;3&quot; x=&quot;0&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;4&quot; x=&quot;0.15&quot; y=&quot;0.2&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;5&quot; x=&quot;0.25&quot; y=&quot;0.2&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;6&quot; x=&quot;0.2&quot; y=&quot;0.15&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;7&quot; x=&quot;0.2&quot; y=&quot;0.25&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;8&quot; x=&quot;0.104762&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;9&quot; x=&quot;0.209524&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;10&quot; x=&quot;0.314286&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;11&quot; x=&quot;0.419048&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;12&quot; x=&quot;0.52381&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;13&quot; x=&quot;0.628571&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;14&quot; x=&quot;0.733333&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;15&quot; x=&quot;0.838095&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;16&quot; x=&quot;0.942857&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;17&quot; x=&quot;1.04762&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;18&quot; x=&quot;1.15238&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;19&quot; x=&quot;1.25714&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;20&quot; x=&quot;1.3619&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;21&quot; x=&quot;1.46667&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;22&quot; x=&quot;1.57143&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;23&quot; x=&quot;1.67619&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;24&quot; x=&quot;1.78095&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;25&quot; x=&quot;1.88571&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;26&quot; x=&quot;1.99048&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;27&quot; x=&quot;2.09524&quot; y=&quot;0&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;28&quot; x=&quot;2.2&quot; y=&quot;0.1025&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;29&quot; x=&quot;2.2&quot; y=&quot;0.205&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;30&quot; x=&quot;2.2&quot; y=&quot;0.3075&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;31&quot; x=&quot;2.1&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;32&quot; x=&quot;2&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;33&quot; x=&quot;1.9&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;34&quot; x=&quot;1.8&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;35&quot; x=&quot;1.7&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;36&quot; x=&quot;1.6&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;37&quot; x=&quot;1.5&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;38&quot; x=&quot;1.4&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;39&quot; x=&quot;1.3&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;40&quot; x=&quot;1.2&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;41&quot; x=&quot;1.1&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;42&quot; x=&quot;1&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;43&quot; x=&quot;0.9&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;44&quot; x=&quot;0.8&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;45&quot; x=&quot;0.7&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;46&quot; x=&quot;0.6&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;47&quot; x=&quot;0.5&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;48&quot; x=&quot;0.4&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;49&quot; x=&quot;0.3&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;50&quot; x=&quot;0.2&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;51&quot; x=&quot;0.1&quot; y=&quot;0.41&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;52&quot; x=&quot;0&quot; y=&quot;0.3075&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;53&quot; x=&quot;0&quot; y=&quot;0.205&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;54&quot; x=&quot;0&quot; y=&quot;0.1025&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;55&quot; x=&quot;0.156699&quot; y=&quot;0.225&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;56&quot; x=&quot;0.175&quot; y=&quot;0.243301&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;57&quot; x=&quot;0.225&quot; y=&quot;0.243301&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;58&quot; x=&quot;0.243301&quot; y=&quot;0.225&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;59&quot; x=&quot;0.243301&quot; y=&quot;0.175&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;60&quot; x=&quot;0.225&quot; y=&quot;0.156699&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;61&quot; x=&quot;0.175&quot; y=&quot;0.156699&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;62&quot; x=&quot;0.156699&quot; y=&quot;0.175&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;63&quot; x=&quot;0.352033&quot; y=&quot;0.262216&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;64&quot; x=&quot;1.04579&quot; y=&quot;0.252253&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;65&quot; x=&quot;1.24065&quot; y=&quot;0.243419&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;66&quot; x=&quot;0.985775&quot; y=&quot;0.171509&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;67&quot; x=&quot;1.30324&quot; y=&quot;0.187395&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;68&quot; x=&quot;0.890988&quot; y=&quot;0.190768&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;69&quot; x=&quot;1.39085&quot; y=&quot;0.202767&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;70&quot; x=&quot;0.801464&quot; y=&quot;0.204685&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;71&quot; x=&quot;1.49904&quot; y=&quot;0.204666&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;72&quot; x=&quot;0.704267&quot; y=&quot;0.202289&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;73&quot; x=&quot;1.60238&quot; y=&quot;0.204951&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;74&quot; x=&quot;0.487789&quot; y=&quot;0.19191&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;75&quot; x=&quot;1.72935&quot; y=&quot;0.219637&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;76&quot; x=&quot;1.83335&quot; y=&quot;0.187427&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;77&quot; x=&quot;2.03359&quot; y=&quot;0.193642&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;78&quot; x=&quot;0.314663&quot; y=&quot;0.307348&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;79&quot; x=&quot;0.249039&quot; y=&quot;0.333583&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;80&quot; x=&quot;2.12147&quot; y=&quot;0.253405&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;81&quot; x=&quot;2.06373&quot; y=&quot;0.0836837&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;82&quot; x=&quot;0.0734726&quot; y=&quot;0.267055&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;83&quot; x=&quot;0.143989&quot; y=&quot;0.0677174&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;84&quot; x=&quot;0.0613613&quot; y=&quot;0.33781&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;85&quot; x=&quot;0.299628&quot; y=&quot;0.236829&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;86&quot; x=&quot;0.527114&quot; y=&quot;0.295386&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;87&quot; x=&quot;0.262384&quot; y=&quot;0.26549&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;88&quot; x=&quot;0.275777&quot; y=&quot;0.18341&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;89&quot; x=&quot;1.74578&quot; y=&quot;0.330928&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;90&quot; x=&quot;1.84001&quot; y=&quot;0.302448&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;91&quot; x=&quot;1.64998&quot; y=&quot;0.313838&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;92&quot; x=&quot;0.648421&quot; y=&quot;0.302367&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;93&quot; x=&quot;1.54544&quot; y=&quot;0.308508&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;94&quot; x=&quot;0.755686&quot; y=&quot;0.306156&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;95&quot; x=&quot;1.42668&quot; y=&quot;0.314809&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;96&quot; x=&quot;0.870524&quot; y=&quot;0.298596&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;97&quot; x=&quot;1.22553&quot; y=&quot;0.112785&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;98&quot; x=&quot;1.05603&quot; y=&quot;0.147372&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;99&quot; x=&quot;0.969346&quot; y=&quot;0.329867&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;100&quot; x=&quot;1.32527&quot; y=&quot;0.276284&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;101&quot; x=&quot;2.02914&quot; y=&quot;0.29511&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;102&quot; x=&quot;1.88168&quot; y=&quot;0.119486&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;103&quot; x=&quot;1.79545&quot; y=&quot;0.104842&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;104&quot; x=&quot;1.35688&quot; y=&quot;0.0892539&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;105&quot; x=&quot;1.69041&quot; y=&quot;0.105324&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;106&quot; x=&quot;0.914199&quot; y=&quot;0.0955566&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;107&quot; x=&quot;1.58234&quot; y=&quot;0.102923&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;108&quot; x=&quot;0.61843&quot; y=&quot;0.10815&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;109&quot; x=&quot;1.47999&quot; y=&quot;0.101161&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;110&quot; x=&quot;0.725217&quot; y=&quot;0.104273&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;111&quot; x=&quot;0.821727&quot; y=&quot;0.102306&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;112&quot; x=&quot;1.96725&quot; y=&quot;0.105778&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;113&quot; x=&quot;0.418624&quot; y=&quot;0.241547&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;114&quot; x=&quot;0.135226&quot; y=&quot;0.239087&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;115&quot; x=&quot;1.14921&quot; y=&quot;0.314841&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;116&quot; x=&quot;0.404508&quot; y=&quot;0.333259&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;117&quot; x=&quot;0.590155&quot; y=&quot;0.206269&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;118&quot; x=&quot;2.12554&quot; y=&quot;0.150731&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;119&quot; x=&quot;1.93166&quot; y=&quot;0.22436&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;120&quot; x=&quot;0.104689&quot; y=&quot;0.190046&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;121&quot; x=&quot;0.267856&quot; y=&quot;0.0744753&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;122&quot; x=&quot;0.269368&quot; y=&quot;0.221581&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;123&quot; x=&quot;0.434446&quot; y=&quot;0.114927&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;124&quot; x=&quot;1.0089&quot; y=&quot;0.0793381&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;125&quot; x=&quot;0.953023&quot; y=&quot;0.246698&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;126&quot; x=&quot;1.25853&quot; y=&quot;0.33355&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;127&quot; x=&quot;1.05268&quot; y=&quot;0.340828&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;128&quot; x=&quot;2.1369&quot; y=&quot;0.067383&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;129&quot; x=&quot;0.522398&quot; y=&quot;0.112537&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;130&quot; x=&quot;1.3421&quot; y=&quot;0.348929&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;131&quot; x=&quot;0.215186&quot; y=&quot;0.289473&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;132&quot; x=&quot;0.203197&quot; y=&quot;0.335348&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;133&quot; x=&quot;1.13984&quot; y=&quot;0.190728&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;134&quot; x=&quot;1.93813&quot; y=&quot;0.330504&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;135&quot; x=&quot;1.10505&quot; y=&quot;0.0883705&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;136&quot; x=&quot;0.341988&quot; y=&quot;0.0824728&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;137&quot; x=&quot;0.253113&quot; y=&quot;0.138035&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;138&quot; x=&quot;0.291932&quot; y=&quot;0.131907&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;139&quot; x=&quot;0.203713&quot; y=&quot;0.102439&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;140&quot; x=&quot;0.150375&quot; y=&quot;0.134668&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;141&quot; x=&quot;0.151494&quot; y=&quot;0.309119&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;142&quot; x=&quot;0.354384&quot; y=&quot;0.17896&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;143&quot; x=&quot;0.0685367&quot; y=&quot;0.100776&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;144&quot; x=&quot;0.424128&quot; y=&quot;0.180088&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;145&quot; x=&quot;0.474925&quot; y=&quot;0.0568661&quot; /&amp;gt;&#xA;      &amp;lt;vertex index=&quot;146&quot; x=&quot;2.10843&quot; y=&quot;0.347669&quot; /&amp;gt;&#xA;    &amp;lt;/vertices&amp;gt;&#xA;    &amp;lt;cells size=&quot;231&quot;&amp;gt;&#xA;      &amp;lt;triangle index=&quot;0&quot; v0=&quot;29&quot; v1=&quot;30&quot; v2=&quot;80&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;1&quot; v0=&quot;26&quot; v1=&quot;27&quot; v2=&quot;81&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;2&quot; v0=&quot;8&quot; v1=&quot;9&quot; v2=&quot;83&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;3&quot; v0=&quot;3&quot; v1=&quot;51&quot; v2=&quot;84&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;4&quot; v0=&quot;3&quot; v1=&quot;52&quot; v2=&quot;84&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;5&quot; v0=&quot;49&quot; v1=&quot;78&quot; v2=&quot;79&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;6&quot; v0=&quot;52&quot; v1=&quot;82&quot; v2=&quot;84&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;7&quot; v0=&quot;63&quot; v1=&quot;78&quot; v2=&quot;85&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;8&quot; v0=&quot;78&quot; v1=&quot;79&quot; v2=&quot;87&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;9&quot; v0=&quot;77&quot; v1=&quot;80&quot; v2=&quot;101&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;10&quot; v0=&quot;26&quot; v1=&quot;81&quot; v2=&quot;112&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;11&quot; v0=&quot;77&quot; v1=&quot;81&quot; v2=&quot;112&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;12&quot; v0=&quot;74&quot; v1=&quot;86&quot; v2=&quot;113&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;13&quot; v0=&quot;28&quot; v1=&quot;29&quot; v2=&quot;118&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;14&quot; v0=&quot;76&quot; v1=&quot;102&quot; v2=&quot;119&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;15&quot; v0=&quot;76&quot; v1=&quot;90&quot; v2=&quot;119&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;16&quot; v0=&quot;72&quot; v1=&quot;108&quot; v2=&quot;117&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;17&quot; v0=&quot;63&quot; v1=&quot;78&quot; v2=&quot;116&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;18&quot; v0=&quot;55&quot; v1=&quot;56&quot; v2=&quot;114&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;19&quot; v0=&quot;5&quot; v1=&quot;59&quot; v2=&quot;88&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;20&quot; v0=&quot;78&quot; v1=&quot;85&quot; v2=&quot;87&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;21&quot; v0=&quot;64&quot; v1=&quot;66&quot; v2=&quot;98&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;22&quot; v0=&quot;65&quot; v1=&quot;67&quot; v2=&quot;97&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;23&quot; v0=&quot;67&quot; v1=&quot;97&quot; v2=&quot;104&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;24&quot; v0=&quot;15&quot; v1=&quot;16&quot; v2=&quot;106&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;25&quot; v0=&quot;66&quot; v1=&quot;68&quot; v2=&quot;106&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;26&quot; v0=&quot;68&quot; v1=&quot;106&quot; v2=&quot;111&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;27&quot; v0=&quot;15&quot; v1=&quot;106&quot; v2=&quot;111&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;28&quot; v0=&quot;20&quot; v1=&quot;21&quot; v2=&quot;104&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;29&quot; v0=&quot;67&quot; v1=&quot;69&quot; v2=&quot;104&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;30&quot; v0=&quot;69&quot; v1=&quot;104&quot; v2=&quot;109&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;31&quot; v0=&quot;21&quot; v1=&quot;104&quot; v2=&quot;109&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;32&quot; v0=&quot;14&quot; v1=&quot;15&quot; v2=&quot;111&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;33&quot; v0=&quot;68&quot; v1=&quot;70&quot; v2=&quot;111&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;34&quot; v0=&quot;70&quot; v1=&quot;110&quot; v2=&quot;111&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;35&quot; v0=&quot;14&quot; v1=&quot;110&quot; v2=&quot;111&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;36&quot; v0=&quot;21&quot; v1=&quot;22&quot; v2=&quot;109&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;37&quot; v0=&quot;69&quot; v1=&quot;71&quot; v2=&quot;109&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;38&quot; v0=&quot;71&quot; v1=&quot;107&quot; v2=&quot;109&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;39&quot; v0=&quot;22&quot; v1=&quot;107&quot; v2=&quot;109&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;40&quot; v0=&quot;77&quot; v1=&quot;101&quot; v2=&quot;119&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;41&quot; v0=&quot;33&quot; v1=&quot;34&quot; v2=&quot;90&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;42&quot; v0=&quot;75&quot; v1=&quot;76&quot; v2=&quot;90&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;43&quot; v0=&quot;34&quot; v1=&quot;35&quot; v2=&quot;89&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;44&quot; v0=&quot;34&quot; v1=&quot;89&quot; v2=&quot;90&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;45&quot; v0=&quot;75&quot; v1=&quot;89&quot; v2=&quot;90&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;46&quot; v0=&quot;35&quot; v1=&quot;36&quot; v2=&quot;91&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;47&quot; v0=&quot;35&quot; v1=&quot;89&quot; v2=&quot;91&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;48&quot; v0=&quot;73&quot; v1=&quot;75&quot; v2=&quot;91&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;49&quot; v0=&quot;75&quot; v1=&quot;89&quot; v2=&quot;91&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;50&quot; v0=&quot;45&quot; v1=&quot;46&quot; v2=&quot;92&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;51&quot; v0=&quot;72&quot; v1=&quot;92&quot; v2=&quot;117&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;52&quot; v0=&quot;36&quot; v1=&quot;37&quot; v2=&quot;93&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;53&quot; v0=&quot;36&quot; v1=&quot;91&quot; v2=&quot;93&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;54&quot; v0=&quot;71&quot; v1=&quot;73&quot; v2=&quot;93&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;55&quot; v0=&quot;73&quot; v1=&quot;91&quot; v2=&quot;93&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;56&quot; v0=&quot;44&quot; v1=&quot;45&quot; v2=&quot;94&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;57&quot; v0=&quot;45&quot; v1=&quot;92&quot; v2=&quot;94&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;58&quot; v0=&quot;70&quot; v1=&quot;72&quot; v2=&quot;94&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;59&quot; v0=&quot;72&quot; v1=&quot;92&quot; v2=&quot;94&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;60&quot; v0=&quot;37&quot; v1=&quot;38&quot; v2=&quot;95&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;61&quot; v0=&quot;37&quot; v1=&quot;93&quot; v2=&quot;95&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;62&quot; v0=&quot;69&quot; v1=&quot;71&quot; v2=&quot;95&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;63&quot; v0=&quot;71&quot; v1=&quot;93&quot; v2=&quot;95&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;64&quot; v0=&quot;43&quot; v1=&quot;44&quot; v2=&quot;96&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;65&quot; v0=&quot;44&quot; v1=&quot;94&quot; v2=&quot;96&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;66&quot; v0=&quot;68&quot; v1=&quot;70&quot; v2=&quot;96&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;67&quot; v0=&quot;70&quot; v1=&quot;94&quot; v2=&quot;96&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;68&quot; v0=&quot;42&quot; v1=&quot;43&quot; v2=&quot;99&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;69&quot; v0=&quot;43&quot; v1=&quot;96&quot; v2=&quot;99&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;70&quot; v0=&quot;13&quot; v1=&quot;14&quot; v2=&quot;110&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;71&quot; v0=&quot;70&quot; v1=&quot;72&quot; v2=&quot;110&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;72&quot; v0=&quot;72&quot; v1=&quot;108&quot; v2=&quot;110&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;73&quot; v0=&quot;13&quot; v1=&quot;108&quot; v2=&quot;110&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;74&quot; v0=&quot;22&quot; v1=&quot;23&quot; v2=&quot;107&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;75&quot; v0=&quot;71&quot; v1=&quot;73&quot; v2=&quot;107&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;76&quot; v0=&quot;73&quot; v1=&quot;105&quot; v2=&quot;107&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;77&quot; v0=&quot;23&quot; v1=&quot;105&quot; v2=&quot;107&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;78&quot; v0=&quot;23&quot; v1=&quot;24&quot; v2=&quot;105&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;79&quot; v0=&quot;73&quot; v1=&quot;75&quot; v2=&quot;105&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;80&quot; v0=&quot;75&quot; v1=&quot;103&quot; v2=&quot;105&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;81&quot; v0=&quot;24&quot; v1=&quot;103&quot; v2=&quot;105&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;82&quot; v0=&quot;24&quot; v1=&quot;25&quot; v2=&quot;103&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;83&quot; v0=&quot;75&quot; v1=&quot;76&quot; v2=&quot;103&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;84&quot; v0=&quot;76&quot; v1=&quot;102&quot; v2=&quot;103&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;85&quot; v0=&quot;25&quot; v1=&quot;102&quot; v2=&quot;103&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;86&quot; v0=&quot;40&quot; v1=&quot;41&quot; v2=&quot;115&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;87&quot; v0=&quot;65&quot; v1=&quot;67&quot; v2=&quot;100&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;88&quot; v0=&quot;67&quot; v1=&quot;69&quot; v2=&quot;100&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;89&quot; v0=&quot;69&quot; v1=&quot;95&quot; v2=&quot;100&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;90&quot; v0=&quot;25&quot; v1=&quot;26&quot; v2=&quot;112&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;91&quot; v0=&quot;25&quot; v1=&quot;102&quot; v2=&quot;112&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;92&quot; v0=&quot;77&quot; v1=&quot;112&quot; v2=&quot;119&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;93&quot; v0=&quot;102&quot; v1=&quot;112&quot; v2=&quot;119&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;94&quot; v0=&quot;48&quot; v1=&quot;49&quot; v2=&quot;116&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;95&quot; v0=&quot;49&quot; v1=&quot;78&quot; v2=&quot;116&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;96&quot; v0=&quot;46&quot; v1=&quot;47&quot; v2=&quot;86&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;97&quot; v0=&quot;46&quot; v1=&quot;86&quot; v2=&quot;92&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;98&quot; v0=&quot;74&quot; v1=&quot;86&quot; v2=&quot;117&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;99&quot; v0=&quot;86&quot; v1=&quot;92&quot; v2=&quot;117&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;100&quot; v0=&quot;47&quot; v1=&quot;48&quot; v2=&quot;116&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;101&quot; v0=&quot;47&quot; v1=&quot;86&quot; v2=&quot;116&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;102&quot; v0=&quot;63&quot; v1=&quot;113&quot; v2=&quot;116&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;103&quot; v0=&quot;86&quot; v1=&quot;113&quot; v2=&quot;116&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;104&quot; v0=&quot;82&quot; v1=&quot;114&quot; v2=&quot;120&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;105&quot; v0=&quot;49&quot; v1=&quot;50&quot; v2=&quot;79&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;106&quot; v0=&quot;9&quot; v1=&quot;10&quot; v2=&quot;121&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;107&quot; v0=&quot;58&quot; v1=&quot;87&quot; v2=&quot;122&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;108&quot; v0=&quot;85&quot; v1=&quot;87&quot; v2=&quot;122&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;109&quot; v0=&quot;66&quot; v1=&quot;98&quot; v2=&quot;124&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;110&quot; v0=&quot;66&quot; v1=&quot;106&quot; v2=&quot;124&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;111&quot; v0=&quot;66&quot; v1=&quot;68&quot; v2=&quot;125&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;112&quot; v0=&quot;96&quot; v1=&quot;99&quot; v2=&quot;125&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;113&quot; v0=&quot;68&quot; v1=&quot;96&quot; v2=&quot;125&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;114&quot; v0=&quot;39&quot; v1=&quot;40&quot; v2=&quot;126&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;115&quot; v0=&quot;41&quot; v1=&quot;42&quot; v2=&quot;127&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;116&quot; v0=&quot;42&quot; v1=&quot;99&quot; v2=&quot;127&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;117&quot; v0=&quot;1&quot; v1=&quot;27&quot; v2=&quot;128&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;118&quot; v0=&quot;1&quot; v1=&quot;28&quot; v2=&quot;128&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;119&quot; v0=&quot;27&quot; v1=&quot;81&quot; v2=&quot;128&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;120&quot; v0=&quot;108&quot; v1=&quot;117&quot; v2=&quot;129&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;121&quot; v0=&quot;74&quot; v1=&quot;117&quot; v2=&quot;129&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;122&quot; v0=&quot;38&quot; v1=&quot;39&quot; v2=&quot;130&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;123&quot; v0=&quot;95&quot; v1=&quot;100&quot; v2=&quot;130&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;124&quot; v0=&quot;38&quot; v1=&quot;95&quot; v2=&quot;130&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;125&quot; v0=&quot;64&quot; v1=&quot;66&quot; v2=&quot;125&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;126&quot; v0=&quot;64&quot; v1=&quot;99&quot; v2=&quot;125&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;127&quot; v0=&quot;39&quot; v1=&quot;126&quot; v2=&quot;130&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;128&quot; v0=&quot;100&quot; v1=&quot;126&quot; v2=&quot;130&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;129&quot; v0=&quot;28&quot; v1=&quot;118&quot; v2=&quot;128&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;130&quot; v0=&quot;81&quot; v1=&quot;118&quot; v2=&quot;128&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;131&quot; v0=&quot;5&quot; v1=&quot;88&quot; v2=&quot;122&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;132&quot; v0=&quot;5&quot; v1=&quot;58&quot; v2=&quot;122&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;133&quot; v0=&quot;85&quot; v1=&quot;88&quot; v2=&quot;122&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;134&quot; v0=&quot;29&quot; v1=&quot;80&quot; v2=&quot;118&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;135&quot; v0=&quot;77&quot; v1=&quot;81&quot; v2=&quot;118&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;136&quot; v0=&quot;77&quot; v1=&quot;80&quot; v2=&quot;118&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;137&quot; v0=&quot;16&quot; v1=&quot;17&quot; v2=&quot;124&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;138&quot; v0=&quot;16&quot; v1=&quot;106&quot; v2=&quot;124&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;139&quot; v0=&quot;64&quot; v1=&quot;99&quot; v2=&quot;127&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;140&quot; v0=&quot;41&quot; v1=&quot;115&quot; v2=&quot;127&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;141&quot; v0=&quot;64&quot; v1=&quot;115&quot; v2=&quot;127&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;142&quot; v0=&quot;12&quot; v1=&quot;13&quot; v2=&quot;108&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;143&quot; v0=&quot;12&quot; v1=&quot;108&quot; v2=&quot;129&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;144&quot; v0=&quot;65&quot; v1=&quot;100&quot; v2=&quot;126&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;145&quot; v0=&quot;65&quot; v1=&quot;115&quot; v2=&quot;126&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;146&quot; v0=&quot;40&quot; v1=&quot;115&quot; v2=&quot;126&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;147&quot; v0=&quot;79&quot; v1=&quot;87&quot; v2=&quot;131&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;148&quot; v0=&quot;50&quot; v1=&quot;79&quot; v2=&quot;132&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;149&quot; v0=&quot;65&quot; v1=&quot;115&quot; v2=&quot;133&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;150&quot; v0=&quot;64&quot; v1=&quot;115&quot; v2=&quot;133&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;151&quot; v0=&quot;79&quot; v1=&quot;131&quot; v2=&quot;132&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;152&quot; v0=&quot;57&quot; v1=&quot;87&quot; v2=&quot;131&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;153&quot; v0=&quot;57&quot; v1=&quot;58&quot; v2=&quot;87&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;154&quot; v0=&quot;74&quot; v1=&quot;123&quot; v2=&quot;129&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;155&quot; v0=&quot;32&quot; v1=&quot;33&quot; v2=&quot;134&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;156&quot; v0=&quot;32&quot; v1=&quot;101&quot; v2=&quot;134&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;157&quot; v0=&quot;33&quot; v1=&quot;90&quot; v2=&quot;134&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;158&quot; v0=&quot;90&quot; v1=&quot;119&quot; v2=&quot;134&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;159&quot; v0=&quot;101&quot; v1=&quot;119&quot; v2=&quot;134&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;160&quot; v0=&quot;65&quot; v1=&quot;97&quot; v2=&quot;133&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;161&quot; v0=&quot;64&quot; v1=&quot;98&quot; v2=&quot;133&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;162&quot; v0=&quot;17&quot; v1=&quot;18&quot; v2=&quot;135&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;163&quot; v0=&quot;17&quot; v1=&quot;124&quot; v2=&quot;135&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;164&quot; v0=&quot;98&quot; v1=&quot;124&quot; v2=&quot;135&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;165&quot; v0=&quot;97&quot; v1=&quot;133&quot; v2=&quot;135&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;166&quot; v0=&quot;98&quot; v1=&quot;133&quot; v2=&quot;135&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;167&quot; v0=&quot;10&quot; v1=&quot;121&quot; v2=&quot;136&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;168&quot; v0=&quot;10&quot; v1=&quot;11&quot; v2=&quot;136&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;169&quot; v0=&quot;11&quot; v1=&quot;123&quot; v2=&quot;136&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;170&quot; v0=&quot;121&quot; v1=&quot;136&quot; v2=&quot;138&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;171&quot; v0=&quot;121&quot; v1=&quot;137&quot; v2=&quot;138&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;172&quot; v0=&quot;60&quot; v1=&quot;137&quot; v2=&quot;139&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;173&quot; v0=&quot;19&quot; v1=&quot;20&quot; v2=&quot;104&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;174&quot; v0=&quot;19&quot; v1=&quot;97&quot; v2=&quot;104&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;175&quot; v0=&quot;121&quot; v1=&quot;137&quot; v2=&quot;139&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;176&quot; v0=&quot;18&quot; v1=&quot;19&quot; v2=&quot;97&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;177&quot; v0=&quot;18&quot; v1=&quot;97&quot; v2=&quot;135&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;178&quot; v0=&quot;9&quot; v1=&quot;121&quot; v2=&quot;139&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;179&quot; v0=&quot;9&quot; v1=&quot;83&quot; v2=&quot;139&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;180&quot; v0=&quot;52&quot; v1=&quot;53&quot; v2=&quot;82&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;181&quot; v0=&quot;59&quot; v1=&quot;88&quot; v2=&quot;137&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;182&quot; v0=&quot;59&quot; v1=&quot;60&quot; v2=&quot;137&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;183&quot; v0=&quot;88&quot; v1=&quot;137&quot; v2=&quot;138&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;184&quot; v0=&quot;4&quot; v1=&quot;62&quot; v2=&quot;120&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;185&quot; v0=&quot;62&quot; v1=&quot;120&quot; v2=&quot;140&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;186&quot; v0=&quot;53&quot; v1=&quot;82&quot; v2=&quot;120&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;187&quot; v0=&quot;82&quot; v1=&quot;114&quot; v2=&quot;141&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;188&quot; v0=&quot;85&quot; v1=&quot;88&quot; v2=&quot;142&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;189&quot; v0=&quot;88&quot; v1=&quot;138&quot; v2=&quot;142&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;190&quot; v0=&quot;0&quot; v1=&quot;8&quot; v2=&quot;143&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;191&quot; v0=&quot;0&quot; v1=&quot;54&quot; v2=&quot;143&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;192&quot; v0=&quot;74&quot; v1=&quot;113&quot; v2=&quot;144&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;193&quot; v0=&quot;74&quot; v1=&quot;123&quot; v2=&quot;144&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;194&quot; v0=&quot;82&quot; v1=&quot;84&quot; v2=&quot;141&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;195&quot; v0=&quot;63&quot; v1=&quot;85&quot; v2=&quot;142&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;196&quot; v0=&quot;8&quot; v1=&quot;83&quot; v2=&quot;143&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;197&quot; v0=&quot;53&quot; v1=&quot;54&quot; v2=&quot;143&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;198&quot; v0=&quot;83&quot; v1=&quot;140&quot; v2=&quot;143&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;199&quot; v0=&quot;120&quot; v1=&quot;140&quot; v2=&quot;143&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;200&quot; v0=&quot;53&quot; v1=&quot;120&quot; v2=&quot;143&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;201&quot; v0=&quot;61&quot; v1=&quot;62&quot; v2=&quot;140&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;202&quot; v0=&quot;136&quot; v1=&quot;138&quot; v2=&quot;142&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;203&quot; v0=&quot;63&quot; v1=&quot;113&quot; v2=&quot;142&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;204&quot; v0=&quot;113&quot; v1=&quot;142&quot; v2=&quot;144&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;205&quot; v0=&quot;123&quot; v1=&quot;136&quot; v2=&quot;142&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;206&quot; v0=&quot;123&quot; v1=&quot;142&quot; v2=&quot;144&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;207&quot; v0=&quot;50&quot; v1=&quot;51&quot; v2=&quot;141&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;208&quot; v0=&quot;51&quot; v1=&quot;84&quot; v2=&quot;141&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;209&quot; v0=&quot;50&quot; v1=&quot;132&quot; v2=&quot;141&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;210&quot; v0=&quot;4&quot; v1=&quot;55&quot; v2=&quot;114&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;211&quot; v0=&quot;4&quot; v1=&quot;114&quot; v2=&quot;120&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;212&quot; v0=&quot;11&quot; v1=&quot;12&quot; v2=&quot;145&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;213&quot; v0=&quot;123&quot; v1=&quot;129&quot; v2=&quot;145&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;214&quot; v0=&quot;11&quot; v1=&quot;123&quot; v2=&quot;145&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;215&quot; v0=&quot;12&quot; v1=&quot;129&quot; v2=&quot;145&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;216&quot; v0=&quot;6&quot; v1=&quot;60&quot; v2=&quot;139&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;217&quot; v0=&quot;131&quot; v1=&quot;132&quot; v2=&quot;141&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;218&quot; v0=&quot;6&quot; v1=&quot;61&quot; v2=&quot;140&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;219&quot; v0=&quot;83&quot; v1=&quot;139&quot; v2=&quot;140&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;220&quot; v0=&quot;6&quot; v1=&quot;139&quot; v2=&quot;140&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;221&quot; v0=&quot;2&quot; v1=&quot;30&quot; v2=&quot;146&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;222&quot; v0=&quot;2&quot; v1=&quot;31&quot; v2=&quot;146&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;223&quot; v0=&quot;31&quot; v1=&quot;32&quot; v2=&quot;146&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;224&quot; v0=&quot;80&quot; v1=&quot;101&quot; v2=&quot;146&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;225&quot; v0=&quot;30&quot; v1=&quot;80&quot; v2=&quot;146&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;226&quot; v0=&quot;32&quot; v1=&quot;101&quot; v2=&quot;146&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;227&quot; v0=&quot;7&quot; v1=&quot;57&quot; v2=&quot;131&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;228&quot; v0=&quot;56&quot; v1=&quot;114&quot; v2=&quot;141&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;229&quot; v0=&quot;7&quot; v1=&quot;56&quot; v2=&quot;141&quot; /&amp;gt;&#xA;      &amp;lt;triangle index=&quot;230&quot; v0=&quot;7&quot; v1=&quot;131&quot; v2=&quot;141&quot; /&amp;gt;&#xA;    &amp;lt;/cells&amp;gt;&#xA;  &amp;lt;/mesh&amp;gt;&#xA;&amp;lt;/dolfin&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-05-20T17:49:14.393" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7294" PostTypeId="1" AcceptedAnswerId="7295" CreationDate="2013-05-20T18:15:43.080" Score="10" ViewCount="1302" Body="&lt;p&gt;It seems that people usually use the Single Active Electron (SAE) approximation to deal with a multi-electron system, transforming the problem into a single electron problem. For example, in numerically solving the problem of a helium atom interact with laser fields, people usually approximate include the electron-electron effect by a pseudo-potential and essentially solve the one electron problem.&#xA;So why it is difficult to even numerically solve the time-dependent multi-electron Schrödinger's equation? Is it much difficult than the classical n-body problem? I've seen there are a lot a huge classical $n$-body problem solved numerically in astronomy even in real time, for example &lt;a href=&quot;http://www.youtube.com/watch?v=aByz-mxOXJM&quot;&gt;here&lt;/a&gt; simulate in real time a collision of two galaxies involving 280000 particles interaction. &lt;/p&gt;&#xA;" OwnerUserId="3997" OwnerDisplayName="xslittlegrass" LastEditorUserId="127" LastEditDate="2013-05-20T23:46:54.207" LastActivityDate="2013-05-20T23:46:54.207" Title="Why is it difficult to numerically solve multi-electron time-dependent Schrödinger's equation" Tags="|pde|quantum-mechanics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7295" PostTypeId="2" ParentId="7294" CreationDate="2013-05-20T18:36:23.763" Score="18" Body="&lt;p&gt;Yes, it is &lt;em&gt;much&lt;/em&gt; more difficult to do so. For the $N$ body problem, all you need to compute are the trajectories $\mathbf x_i(t), i=1\ldots N$ which are just $N$ functions of a single variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, even for a single electron, the solution of the Schroedinger equation is a function $\Psi(x,y,z,t)$, i.e., a function of four variables. For two electrons, you are looking for a function $\Psi(x_1,y_1,z_1,x_2,y_2,z_3,t)$ describing the wave function as a function of the locations of the two electrons plus time. That's seven variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you remember how to solve ordinary differential equations such Newton's equations for the $N$ body problem, then you need to move every equation forward by time stepping from time $t$ to $t+\Delta t$ and compute the solution there. So, if you divide your time interval $[0,T]$ into $M$ intervals of length $\Delta t=T/M$ then the effort for every time step will be $N^2M$ using a naive implementation of the interactions of the $N$ bodies (you can use methods to achieve $N (\log N) M$ effort, but that's besides the point).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, to find a function of 7 variables, assume that you subdivide the time interval into $M$ subintervals as above, but that you also do the same for the 6 spatial coordinates. Then there are a total of $M^7$ grid points to consider. And in general, for an $N$ body quantum system, you have $M^{3N+1}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is now easy to verify that even for rather small numbers $N,M$, the effort $M^{3N+1}$ is vastly larger than $N^2M$, which explains why multibody quantum computations are so extremely much more expensive to do than $N$ body classical mechanics.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-20T18:36:23.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7296" PostTypeId="1" AcceptedAnswerId="7299" CreationDate="2013-05-20T18:48:12.967" Score="0" ViewCount="574" Body="&lt;p&gt;I just went through the Fenics book to get the nodal displacements and tried this code ....&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u_nodal_values = U.vector()&#xA;u_array = u_nodal_values.array()&#xA;coor = mesh.coordinates()&#xA;center = (0.5,0.5,0.5)&#xA;print U(center)&#xA;if mesh.num_vertices()==len(u_array):&#xA;   for i in range(mesh.num_vertices()): &#xA;       print ’U(%8g,%8g,%8g) = %g’% (coor[i][0], coor[i][1], u_array[i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but i don't get the expected result..&#xA;can anyone suggest me the modification ??&lt;/p&gt;&#xA;" OwnerUserId="4350" LastEditorUserId="1804" LastEditDate="2013-05-20T19:49:54.420" LastActivityDate="2013-05-20T21:38:46.327" Title="Fenics : looking forward to compute displacements at nodes" Tags="|python|fenics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7299" PostTypeId="2" ParentId="7296" CreationDate="2013-05-20T21:31:48.173" Score="2" Body="&lt;p&gt;I think the problem is that you are relying on a specific DOF ordering, which was changed in release 1.1. However, the vertex to DOF map which you need to solve your problem can be obtained by the following code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;map = V.dofmap().vertex_to_dof_map(mesh)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2013-05-20T21:38:46.327" LastActivityDate="2013-05-20T21:38:46.327" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7300" PostTypeId="1" CreationDate="2013-05-20T21:36:27.787" Score="6" ViewCount="106" Body="&lt;p&gt;I am about to test my algorithms for solving optimal control problems of type:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find an input $u$, such that for a time interval $(0,T]$ the cost functional &#xA;$$J(v,u) = \mathcal M(v(T)) + \int_0^T\mathcal K(v(t),u(t)) dt, $$&#xA;becomes minimal, where $v$ is the velocity of flow modelled by the incompressible Navier-Stokes equations. The functionals $\mathcal M$ and $\mathcal K$ measure the state trajectory, e.g. in terms of how well a given target flow is approximated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 2D, typical test problems I have seen comprise the backward facing step, driven cavity (as in the pics below) or the cylinder wake. However, I haven't found discussions on &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is a good configuration, or even a benchmark, for testing numerical solution approaches to optimal flow control?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to do my first tests on a very basic level. That's why I think the setup should be&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;reasonably sized (~$10^5$ DOFs), &lt;/li&gt;&#xA;&lt;li&gt;laminar,&lt;/li&gt;&#xA;&lt;li&gt;and for &lt;em&gt;distributed&lt;/em&gt; control.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here, distributed means that the control appears in the right hand side of the Navier-Stokes equations and not in the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EMHA3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-26T12:39:28.377" LastActivityDate="2013-05-26T12:39:28.377" Title="Benchmarks or generic configurations for optimal flow control" Tags="|optimization|navier-stokes|laminar-flow|optimal-control|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7301" PostTypeId="5" CreationDate="2013-05-20T21:46:33.517" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Optimal_control&quot; rel=&quot;nofollow&quot;&gt;Optimal control&lt;/a&gt; is a special case of constrained optimization applied to physical, e.g. mechanical systems. &lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-20T23:33:11.477" LastActivityDate="2013-05-20T23:33:11.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7302" PostTypeId="4" CreationDate="2013-05-20T21:46:33.517" Score="0" Body="A tag for questions that relate to numerical approaches to optimal control of systems. " OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-20T23:33:21.460" LastActivityDate="2013-05-20T23:33:21.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7305" PostTypeId="1" AcceptedAnswerId="7313" CreationDate="2013-05-21T04:23:13.497" Score="8" ViewCount="2668" Body="&lt;p&gt;This is not a technical question, and is asked just out of curiosity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are computational fluid dynamics (CFD) simulations used in animation movies? (something like shallow water equations, explosion problem etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, can computational simulations of real world problems be used in art? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I have tagged the question with simulation, but please feel free to retag it as I can't find appropriate tags)&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="11781" LastEditDate="2015-01-26T19:36:23.790" LastActivityDate="2015-01-26T19:36:23.790" Title="Is CFD used in animations / movies / art?" Tags="|fluid-dynamics|simulation|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7306" PostTypeId="2" ParentId="7305" CreationDate="2013-05-21T04:40:17.773" Score="3" Body="&lt;p&gt;After a bit of search, I discovered that CFD is heavily used in films, games and special effects in media. &lt;a href=&quot;http://en.wikipedia.org/wiki/Fluid_simulation&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is the wiki page of the relevant topic, references therein are helpful as well.&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2013-05-21T04:40:17.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7307" PostTypeId="2" ParentId="7305" CreationDate="2013-05-21T04:46:44.470" Score="16" Body="&lt;p&gt;One of the leaders in the field of using CFD for animation, Ron Fedkiw, had a &lt;a href=&quot;http://physbam.stanford.edu/~fedkiw/&quot;&gt;web page&lt;/a&gt; with some fantastic examples, including references to the relevant publications.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastActivityDate="2013-05-21T04:46:44.470" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7308" PostTypeId="2" ParentId="6991" CreationDate="2013-05-21T06:39:09.950" Score="6" Body="&lt;p&gt;(actually the comments provide an answer to your question, but this is additional information).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;a href=&quot;http://qblade.de.to/&quot; rel=&quot;nofollow&quot;&gt;http://qblade.de.to/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a link to a free software called as QBlade. It is a wind turbine design and optimization software with emphasis on blade design (using &lt;a href=&quot;http://web.mit.edu/drela/Public/web/xfoil/&quot; rel=&quot;nofollow&quot;&gt;XFOIL&lt;/a&gt;). This might come useful in designing/ optimization the rotor blades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What can it be used for? :-&lt;/strong&gt;&#xA;In designing and optimizing horizontal, vertical axis turbines. Design of turbine blades. (mainly aerodynamics)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &lt;a href=&quot;http://www.fm.mek.dtu.dk/upload/institutter/mek/fm/eksamensprojekter/andreespaze2007.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.fm.mek.dtu.dk/upload/institutter/mek/fm/eksamensprojekter/andreespaze2007.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a link to a report titled 'free software on wind turbine modelling'. This is written in python, and mainly combines various free codes/softwares. It uses &lt;a href=&quot;http://www.aero.polimi.it/mbdyn/&quot; rel=&quot;nofollow&quot;&gt;MBDyn&lt;/a&gt;  for simulation of structural mechanics of turbine. It uses unsteady BEM for aerodynamic loads simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why should it be useful? :-&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This report discusses about python scripting and python integration with other free softwares, which as per the question is a desirable feature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) &lt;a href=&quot;https://code.google.com/p/windenergytk/&quot; rel=&quot;nofollow&quot;&gt;https://code.google.com/p/windenergytk/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a link to &lt;a href=&quot;https://code.google.com/p/windenergytk/&quot; rel=&quot;nofollow&quot;&gt;windenergytk&lt;/a&gt;, a compilation of software tools aimed towards full design and development of entire wind-turbine project. The modules include analysis, synthesis, aerodynamics, vibrations, electrodynamics and system performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why have I suggested this? :-&lt;/strong&gt; Although this code is aimed towards designing of the conventional turbines, it has different modules covering different aspects, So you can (I think) use one / several of them for specialized tasks, and integrate them with your code. In short, you might derive chunks of code from this which can then be used to do some of your tasks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) &lt;a href=&quot;https://kb.osu.edu/dspace/bitstream/handle/1811/44504/richard-ves?sequence=1&quot; rel=&quot;nofollow&quot;&gt;https://kb.osu.edu/dspace/bitstream/handle/1811/44504/richard-ves?sequence=1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a link to a report titled 'OPTIMIZATION OF A WIND TURBINE ROTOR WITH VARIABLE AIRFOIL SHAPE VIA A GENETIC ALGORITHM'. And discusses the optimization procedure and consequences of the blade shape of a rotor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Purpose of including this link:-&lt;/strong&gt; In order to develop unconventional rotors, you might have to do lot of optimization in shape of rotor blades, so this reference can be referred to see how it is done in case of conventional turbines, and may possibly be extended to unconventional designs. (I have not done this work, but I thought somebody might find this useful). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) &lt;a href=&quot;http://mbdynsimsuite.sourceforge.net/windSimSuite/index.html&quot; rel=&quot;nofollow&quot;&gt;http://mbdynsimsuite.sourceforge.net/windSimSuite/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a link to the software called &lt;a href=&quot;http://mbdynsimsuite.sourceforge.net/windSimSuite/index.html&quot; rel=&quot;nofollow&quot;&gt;Wind sim suite&lt;/a&gt;. It also uses MBDyn. Two models are included for simulations, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) 2 bladed Control Advanced Research Turbine (CART)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) 3 Blades 5 MW  National Renewable Energy Laboratory (NREL) turbine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why this link? :-&lt;/strong&gt; There is extensive documentation and case studies available. The software is written in python.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following link is not really a software, but shows some unconventional wind turbine designs,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.greendiary.com/innovative-wind-turbine-designs-of-the-year.html&quot; rel=&quot;nofollow&quot;&gt;http://www.greendiary.com/innovative-wind-turbine-designs-of-the-year.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMO, design and development of a novel design is an iterative process, and one might have to use conventional CFD / FEA tools to do so (packages like OpenFAOM, or commercial softwares like Fluent, ANSYS etc). If it is a very new design / concept, then only simulations are not sufficient and one might have to carry out some experimentation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;all the best&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="1567" LastEditDate="2013-05-24T10:39:16.673" LastActivityDate="2013-05-24T10:39:16.673" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7309" PostTypeId="1" AcceptedAnswerId="7318" CreationDate="2013-05-21T07:38:05.123" Score="0" ViewCount="433" Body="&lt;p&gt;Sunny day today, isn't it? Please, I need help with my problem. I have written a program to do 2D ray tomography, according to &lt;a href=&quot;http://www.ipgp.fr/~tarantola/exercices/chapter_04/RayTomography.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the result, I use formula (4.15) from the paper. Now I need to do SVD regularization on the result, in order to get good results for lower count of rays (lets say I will have $N$ boxes and $N$ rays). Now it is purely impossible with the formula (4.15)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to understand the regularization - I think it should be only to add something to the diagonal, right? Please can you help me with it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found the paper &quot;Regularized matrix computation&quot; from AE Yagle, but I'm not sure how to implement it in my case. Many thanks.&lt;/p&gt;&#xA;" OwnerUserId="4007" LastEditorUserId="1748" LastEditDate="2013-05-21T15:55:05.980" LastActivityDate="2013-05-21T23:56:22.720" Title="SVD regularization - ray 2D tomography" Tags="|least-squares|svd|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7310" PostTypeId="1" AcceptedAnswerId="7311" CreationDate="2013-05-21T09:45:55.710" Score="2" ViewCount="938" Body="&lt;p&gt;In a elastodynamics problem, I define the stress as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def sigma(v):&#xA;    return 2.0*mu*sym(grad(v)) + lmbda*tr(sym(grad(v)))*Identity(v.cell().d) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have computed the displacements at the nodes. Now I want to compute the stresses elementwise. Can anyone help?&lt;/p&gt;&#xA;" OwnerUserId="4350" LastEditorUserId="348" LastEditDate="2013-05-21T10:56:16.247" LastActivityDate="2013-05-21T11:59:38.217" Title="Fenics : Elementwise stress computation" Tags="|python|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7311" PostTypeId="2" ParentId="7310" CreationDate="2013-05-21T09:59:15.350" Score="4" Body="&lt;p&gt;The following compiles:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;from random import random&#xA;&#xA;lmbda = Constant(1.0)&#xA;mu = Constant(1.0)&#xA;&#xA;def sigma(v):&#xA;    return 2.0*mu*sym(grad(v)) + lmbda*tr(sym(grad(v)))*Identity(v.cell().d) &#xA;&#xA;mesh = UnitCubeMesh(8,8,8)&#xA;V = VectorFunctionSpace(mesh, &quot;CG&quot;, 1)&#xA;&#xA;# Create random displacement vector&#xA;displacement = Function(V)&#xA;for i in xrange(V.dim()):&#xA;    displacement.vector()[i] = random()&#xA;&#xA;T = TensorFunctionSpace(mesh, &quot;DG&quot;, 0)&#xA;form = inner(sigma(displacement),TestFunction(T))*dx&#xA;scaling = 1.0/CellSize(mesh)&#xA;stress = Function(T)&#xA;assemble(scaling*form, tensor=stress.vector())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4292" LastEditorUserId="4292" LastEditDate="2013-05-21T11:59:38.217" LastActivityDate="2013-05-21T11:59:38.217" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7313" PostTypeId="2" ParentId="7305" CreationDate="2013-05-21T13:08:58.317" Score="9" Body="&lt;p&gt;There was a paper in &lt;em&gt;Notices of the American Mathematical Society&lt;/em&gt; on this subject: &lt;a href=&quot;http://www.ams.org/notices/201005/rtx100500614p.pdf&quot;&gt;Crashing Waves, Awesome Explosions, Turbulent Smoke and Beyond: Applied Mathematics and Scientific Computing in the Visual Effects Industry&lt;/a&gt;. In particular, &lt;a href=&quot;http://www.nextlimit.com/products.php&quot;&gt;these commercial packages&lt;/a&gt; constitute examples of simulation software used in the film industry.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-05-21T13:08:58.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7314" PostTypeId="1" AcceptedAnswerId="7317" CreationDate="2013-05-21T13:27:44.460" Score="15" ViewCount="1456" Body="&lt;p&gt;In practice, the runtime of numerically solving an IVP&#xA;$$&#xA;\dot{x}(t) = f(t, x(t)) \quad \text{ for } t \in [t_0, t_1]&#xA;$$&#xA;$$&#xA;x(t_0) = x_0&#xA;$$&#xA;is often dominated by the duration of evaluating the right-hand side (RHS) $f$. Let us therefore assume that all other operations are instant (i.e. without computational cost). If the overall runtime for solving the IVP is limited then this is equivalent to limiting the number of evaluations of $f$ to some $N \in \mathbb{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are only interested in the final value $x(t_1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for theoretical and practical results that help me choose the best ODE method in such a setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, for example, $N = 2$ then we could solve the IVP using two explicit Euler steps of width $(t_1 - t_0)/2$ or one step of width $t_1 - t_0$ using the midpoint method. It is not immediately clear to me which one is preferrable. For larger $N$, one can of course also think about multi-step methods, iterated Runge-Kutta schemes, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm looking for are results similar to the ones that exist, for example, for quadrature rules: We can pick $n$ weights $\{w_i\}$ and associated points $\{x_i\}$ such that the quadrature rule $\sum_{i = 1}^n w_i g(x_i)$ is exact for all polynomials $g$ such that $deg(g) \le 2n - 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence I'm looking for upper or lower bounds on the global accuracy of ODE methods, given a limited number of allowed evaluations of the RHS $f$. It's OK if the bounds only hold for some classes of RHS or pose additional constraints on the solution $x$ (just like the result for the quadrature rule which only holds for polynomials up to a certain degree).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Some background information: This is for hard real-time applications, i.e. the result $x(t_1)$ must be available before a known deadline. Hence the limit on the number of RHS evaluations $N$ as the dominating cost factor. Typically our problems are stiff and comparatively small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2: Unfortunately I don't have the precise timing requirements, but it is safe to assume that $N$ will be rather small (definitely &amp;lt;100, propably closer to 10). Given the real-time requirement we have to find a tradeoff between the accuracy of the models (with better models leading to longer execution times of the RHS and hence to a lower $N$) and the accuracy of the ODE method (with better methods requiring higher values of $N$).&lt;/p&gt;&#xA;" OwnerUserId="4230" LastEditorUserId="4230" LastEditDate="2013-07-02T10:27:59.750" LastActivityDate="2013-07-02T10:27:59.750" Title="Optimal ODE method for fixed number of RHS evaluations" Tags="|ode|numerical-analysis|" AnswerCount="4" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="7315" PostTypeId="1" AcceptedAnswerId="7323" CreationDate="2013-05-21T13:55:56.127" Score="-3" ViewCount="690" Body="&lt;p&gt;`I am working on a problem found here &quot; &lt;a href=&quot;http://www.caesarsystems.co.uk/NAFEMS_benchmarks/le10.html&quot; rel=&quot;nofollow&quot;&gt;http://www.caesarsystems.co.uk/NAFEMS_benchmarks/le10.html&lt;/a&gt;&quot; ..I am really confused about the boundary conditions and tried my level best to compute it but result is totally different&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is my bcs`&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Left(SubDomain): def inside(self, x, on_boundary):&#xA;    return (between (x[1], (1.0,2.75)) and between (x[2], (0.0,0.600000000))&#xA;&#xA;class Right(SubDomain):&#xA;def inside(self, x, on_boundary):&#xA;    return (between(x[0], (2.0,3.25)) and between (x[2], (0.0,0.600000000)))  &#xA;&#xA;class Outerall(SubDomain):&#xA;def inside(self, x, on_boundary):&#xA;    return (((x[0]/3.25)**2 + (x[1]/2.75)**2 ) -1.0 &amp;lt; tol) and abs(x[2]-0.600000000)&#xA;&#xA;&#xA;class Outermid(SubDomain):&#xA;def inside(self, x, on_boundary):&#xA;    return  (((x[0]/3.25)**2 + (x[1]/2.75)**2 ) -1.0 &amp;lt; tol) and abs(x[2]-0.300000000) &amp;lt; tol&#xA;&#xA;class Top(SubDomain):&#xA;def inside(self, x, on_boundary):&#xA;    return abs(x[2]-0.600000000) &amp;lt; tol&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4350" LastEditorUserId="4350" LastEditDate="2013-05-21T14:16:42.103" LastActivityDate="2013-05-21T19:35:39.327" Title="Fenics : boundary condition" Tags="|python|fenics|" AnswerCount="1" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7316" PostTypeId="2" ParentId="7314" CreationDate="2013-05-21T14:31:33.337" Score="6" Body="&lt;p&gt;There aren't many results in this direction because it is more difficult than just fixing accuracy, since stability considerations can often require you to pick time-steps that are smaller than you would need for the accuracy you want. So results are split between the stiff and non-stiff cases. The former case the time-steps and RHS evaluations requirements are generally not governed by accuracy, and in the latter case they are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to focus on explicit methods, since the implicit case is far less obvious how many RHS evaluations you will need to use.. that entirely depends on how you decide to solve the resulting system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For non-stiff systems:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are stage limitations for explicit Runge-Kutta methods for which say how many stages (RHS evaluations) are needed to achieve a certain order of accuracy. After fourth order the number of stages exceeds the order of accuracy, and the disparity continues to grow. Butcher's big ODE book:&#xA;&lt;a href=&quot;http://books.google.com/books/about/Numerical_Methods_for_Ordinary_Different.html?id=opd2NkBmMxsC&quot; rel=&quot;nofollow&quot;&gt;http://books.google.com/books/about/Numerical_Methods_for_Ordinary_Different.html?id=opd2NkBmMxsC&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;does a good job explaining some of these 'non-existence' proofs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your quadrature rule example leads either to a multistep type method such as Adams-bashforth, or to what are now called spectral-deferred-correction methods. For adams-bashforth you only need one RHS evaluation per step, but since the stability regions are so small in general for these methods you generally end up doing the same amount of work in terms of RHS evaluations as a Runge-Kutta method with the same order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a paper on spectral deferred correction:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.google.com/search?q=spectral+deferred+correction&amp;amp;aq=f&amp;amp;oq=spectral+deferred+correction&amp;amp;aqs=chrome.0.57j0l2j62.3336j0&amp;amp;sourceid=chrome&amp;amp;ie=UTF-8&quot; rel=&quot;nofollow&quot;&gt;https://www.google.com/search?q=spectral+deferred+correction&amp;amp;aq=f&amp;amp;oq=spectral+deferred+correction&amp;amp;aqs=chrome.0.57j0l2j62.3336j0&amp;amp;sourceid=chrome&amp;amp;ie=UTF-8&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unsure how these integration methods play out against standard explicit methods, they often require a lot more memory to save solution states at quadrature nodes and so I have never used them myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For stiff systems:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;there are 'optimized' time-steppers, but precise theoretical results concerning how good these can get are unfortunately limited to some simple cases (and even those turned out to not be a trivial bit of work). The three standard results say that for Runge-Kutta methods with $S$ stages: The most negative real axis it can include in its stability region is an interval of length $2S^2$, the most imaginary axis it can contain is an interval of length $S-1$, and the largest circle tangent to the imaginary axis it can contain has radius $S$ (all of these are mutually exclusive as well).&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2013-05-21T16:36:47.360" LastActivityDate="2013-05-21T16:36:47.360" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7317" PostTypeId="2" ParentId="7314" CreationDate="2013-05-21T14:49:01.887" Score="7" Body="&lt;p&gt;I think a key reference to answer your question is &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0898122194001510&quot;&gt;this paper by Hosea and Shampine&lt;/a&gt;.  Now I will give some background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the step size that you may use when numerically integrating an IVP can be restricted by stability or accuracy.  If you want to choose the best solver in terms of stability, you need to consider the &lt;em&gt;region of absolute stability&lt;/em&gt;.  For a one-step method this is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S = \{ z \in {\mathbb C} : |P(z)|\le 1\}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $P(z)$ is the stability function of the method; see e.g. the text of Hairer et. al.  A necessary condition for stability is that $\lambda h \in S$ where $\lambda$ ranges over the eigenvalues of the jacobian of $f$ and $h$ is the step size.  This is not always a sufficient condition for nonlinear problems but it is usually a good rule of thumb and is used in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an extensive treatment of the problem of finding (explicit) methods that allow large stable step sizes, see &lt;a href=&quot;http://numerics.kaust.edu.sa/papers/stability_polynomials/camcos-v7-n2-p04-s.pdf&quot;&gt;this paper of mine on stability polynomials&lt;/a&gt; and &lt;a href=&quot;http://arxiv.org/pdf/1207.5830v2.pdf&quot;&gt;this one on optimization of Runge-Kutta methods for compressible fluid simulations&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stability is the relevant concern if you find that the largest stable step size already gives you enough accuracy.  On the other hand, the step size may instead be restricted by your accuracy requirements.  What is typically done is local error control.  The solution is computed using two methods, and their difference is used as an estimate of the error in the less accurate one.  The step size is chosen adaptively to achieve as closely as possible the prescribed tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two theoretical measures are key to predicting accuracy efficiency.  The first is the &lt;em&gt;order of accuracy&lt;/em&gt; of the method, which describes the rate at which the error approaches zero when the step size is decreased.  The second is the &lt;em&gt;accuracy efficiency index&lt;/em&gt; (see the paper of Hosea and Shampine linked in the first sentence above) which takes into account the constants appearing in the error terms and allows comparisons between methods of the same order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accuracy and stability efficiency of a wide range of methods can be computed in a simple and automated way using &lt;a href=&quot;https://github.com/ketch/nodepy&quot;&gt;NodePy&lt;/a&gt; (disclaimer: NodePy is developed by me).&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-05-21T14:49:01.887" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7318" PostTypeId="2" ParentId="7309" CreationDate="2013-05-21T14:56:34.257" Score="4" Body="&lt;p&gt;First, never calculate an explicit inverse, allthough sometimes you cannot avoid it. Most of the time however, the solution can be restated as multiplication of a vector with an inverse matrix. In that case several methods exist, which don't exlpicitely calculate the inverse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, common regularization techniques are &lt;a href=&quot;http://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow&quot;&gt;truncated singular value decomposition&lt;/a&gt;(see truncated svd) or &lt;a href=&quot;http://en.wikipedia.org/wiki/Tikhonov_regularization&quot; rel=&quot;nofollow&quot;&gt;tikhonov regularization&lt;/a&gt;. Both techniques essentially filter small singular values. &#xA;This is necessary because the error introduced by small singular values to the solution vector can become dominant to that extent, that all digits of the solution vector are off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now having two different techniques at hand, the next question is, how to choose the regularization parameter? The idea is to balance the error in the solution vector with the error of the residuum. &#xA;For my problems, the &lt;a href=&quot;http://www.jstor.org/discover/10.2307/2132628?uid=3738992&amp;amp;uid=2&amp;amp;uid=4&amp;amp;sid=21102245387641&quot; rel=&quot;nofollow&quot;&gt;l-curve criterion&lt;/a&gt; worked usually fine.&#xA;I can highly recommend that paper. It describes all steps for a calculation of the l-curve criterion starting from the svd of the data matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also did a quick comparision between the truncated svd and tikhonov regularization. Both give very similiar results if the cut off / regularization parameters are choosen according to the l-curve criterion. I will probably update my post later with a short python program, demonstrating both techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As promised a code snippet calculating tikhonov parameters of an ill posed problem. The &lt;code&gt;tikhonov_lcurve&lt;/code&gt; routine uses the same notation as in the paper.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# -*- coding: utf8 -*-&#xA;'''&#xA;Created on Mar 10, 2013&#xA;&#xA;'''&#xA;import numpy as np&#xA;import matplotlib.pyplot as pl&#xA;import os&#xA;import sys&#xA;&#xA;def fun(x):&#xA;    # witch of agnesi funcion&#xA;    # because of the two complex poles near the x-axes at z = +/- I epsilon&#xA;    # many chebyshev modes are necessary for approximation&#xA;    eps = 0.03&#xA;    return eps ** 2 / (x ** 2 + eps ** 2) &#xA;&#xA;def main():&#xA;    # setting up an ill-posed problem&#xA;    # approximate the witch of agnesi function&#xA;    # up to order 100 with 100 random points&#xA;    np.random.seed(1)&#xA;    N = 100           # number of chebyshev modes&#xA;    NN = 100          # number of data points which are not gauss quadrature points&#xA;&#xA;    x = 2 * np.random.rand(NN) - 1 # 100 random points in the interval -1,1&#xA;&#xA;    n = np.arange(N)&#xA;    nn, xx = np.meshgrid(n, x)&#xA;    L = np.cos(nn * np.arccos(xx))&#xA;    b = fun(x)&#xA;&#xA;    # ill posed problem stated: L c = b&#xA;    # L in this case is my data matrix&#xA;&#xA;    U, sigma, Vh = np.linalg.svd(L, full_matrices=False) # do the svd  &#xA;    print 'condition of L:', sigma[0] / sigma[-1]&#xA;&#xA;    beta = np.dot(U.T, b) # setup right side of the ill-posed problem&#xA;    x1, y1 = truncation_lcurve(U.copy(), sigma.copy(), Vh.copy(), beta.copy())&#xA;    x2, y2, xb, yb = tikhonov_lcurve(U.copy(), sigma.copy(), Vh.copy(), beta.copy())&#xA;&#xA;    # plot the balance between norm of solution vector and norm of residuum vector&#xA;    fig = pl.figure()&#xA;    ax = fig.add_subplot(111)&#xA;    ax.loglog(x2, y2, '.-',label='Tikhonov')&#xA;    ax.loglog(x1, y1, '.-',label='Truncated SVD')&#xA;    ax.loglog(xb, yb, 'o',label='Best Tikhonov Parameter')&#xA;    ax.set_xlim(1e-8, 1)&#xA;    ax.set_ylim(0.01, 1e10)&#xA;    ax.set_xlabel('|Ax-b|')&#xA;    ax.set_ylabel('|x|')&#xA;    pl.legend()&#xA;    pl.show()&#xA;&#xA;def truncation_lcurve(U, sigma, Vh, beta):&#xA;    l = np.arange(90) + 10  # truncate singular values from 100 down to 10 and check the effect on the lcurve&#xA;&#xA;    ll, ss = np.meshgrid(l, sigma)&#xA;    f = (0 * ss + 1)&#xA;    for i, v in enumerate(l):&#xA;        s = np.ones(sigma.shape)&#xA;        s[v:] = 0.                # set singular values after cut off to zero == truncation == singular value filter&#xA;        f[:, i] = s&#xA;&#xA;    _, bbeta = np.meshgrid(l, beta)&#xA;&#xA;    # xl contains the regularized solution for all l trunctions&#xA;    xl = np.dot(Vh.T, f * bbeta / ss)&#xA;&#xA;&#xA;    # the following calculations are necessary for the l-curve criterion, see publication&#xA;    nxl = np.sum(np.abs(xl) ** 2, axis=0) ** (1. / 2)# norm of the solution vector&#xA;    axmb = np.sum(np.abs((1 - f) * bbeta) ** 2, axis=0) ** (1. / 2)# norm of the resdiuum vector&#xA;&#xA;    return axmb, nxl&#xA;&#xA;&#xA;def tikhonov_lcurve(U, sigma, Vh, beta):&#xA;    # as shown in the publication, the kink is between the lowest and highest singular value&#xA;    # let's try 1000 regularization parameters in the interval of the singular values&#xA;    l = np.logspace(np.log10(sigma[-1]) - 1, np.log10(sigma[0]) + 1, 1000) &#xA;&#xA;    ll, ss = np.meshgrid(l, sigma)&#xA;    # regularize the singular value with ll**2, note if ss &amp;gt;&amp;gt; ll it has no effect, &#xA;    # if ss is of order of ll however, damping kicks in&#xA;    # that's why one calls both regularization technique&#xA;    # a filter for singular values&#xA;    f = ss ** 2 / (ss ** 2 + ll ** 2)&#xA;    _, bbeta = np.meshgrid(l, beta)&#xA;&#xA;    # xl contains the regularized solution for all l parameters&#xA;    xl = np.dot(Vh.T, f * bbeta / ss)&#xA;&#xA;    # the following calculations are necessary for the l-curve criterion, see publication&#xA;    # e.g. kappa is the curvate of the l-curve on the double log scale&#xA;    nxl = np.sum(np.abs(xl) ** 2, axis=0) ** (1. / 2)  # norm of the solution vector&#xA;    axmb = np.sum(np.abs((1 - f) * bbeta) ** 2, axis=0) ** (1. / 2) # norm of the resdiuum vector&#xA;&#xA;    eta = nxl ** 2&#xA;    rho = axmb ** 2&#xA;&#xA;    eta_p = -4 / l * np.sum((1 - f) * f ** 2 * bbeta ** 2 / ss ** 2, axis=0)&#xA;    kappa = 2 * eta * rho / eta_p * (l ** 2 * eta_p * rho + 2 * l * eta * rho + l ** 4 * eta * eta_p) / (l ** 2 * eta ** 2 + rho ** 2) ** (3. / 2)&#xA;    bi = kappa.argmin()&#xA;    lb = l[bi]&#xA;    print 'Best Tikhonov Parameter ', lb&#xA;&#xA;    # xbl contains the solution where the tikhonov regularization&#xA;    # is near the kink of the l-curve&#xA;    xbl = np.dot(Vh.T, sigma / (sigma ** 2 + lb ** 2) * beta)&#xA;    nxbl = np.linalg.norm(xbl)&#xA;    axmb_xbl = np.linalg.norm((1 - sigma ** 2 / (sigma ** 2 + lb ** 2)) * beta)&#xA;&#xA;    return axmb, nxl, axmb_xbl, nxbl&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2013-05-21T23:56:22.720" LastActivityDate="2013-05-21T23:56:22.720" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7319" PostTypeId="1" CreationDate="2013-05-21T15:09:27.890" Score="2" ViewCount="836" Body="&lt;p&gt;I'm trying to use periodic boundary conditions within a Raviart-Thomas finite element space in Fenics (dolfin 1.2.0) in a Ubuntu 12.04 (amd64) machine (with python 2.7). If other FE space is used, everything works fine, but if a Raviart-Thomas is considered, python crashes (due to libdolfin errors). In what follows, it is attached a test code to reproduce the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;&#xA;# Periodic boundary condition class&#xA;class PeriodicWall(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        return bool(on_boundary and near(x[0], 0.5))  &#xA;    def map(self, x, y):&#xA;        y[0] = x[0] &#xA;        y[1] = x[1] + 1.0&#xA;&#xA;# Mesh and periodic boundary conditions&#xA;mesh = RectangleMesh(-0.5,-0.5,0.5,0.5,5,5,&quot;left&quot;)&#xA;periodicwall = PeriodicWall()&#xA;&#xA;# Raviart-Thomas space with periodic boundaries&#xA;V = FunctionSpace(mesh, &quot;RT&quot;, 1, constrained_domain=periodicwall)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any kind of help would be helpful: are there any errors in the test code? or is it a known bug in Dolfin?&lt;/p&gt;&#xA;" OwnerUserId="4359" LastEditorUserId="4359" LastEditDate="2013-05-21T15:30:48.803" LastActivityDate="2013-05-21T15:30:48.803" Title="How to apply periodic boundary conditions to a Raviart-Thomas finite element space in Fenics?" Tags="|finite-element|fenics|constrained-optimization|" AnswerCount="0" CommentCount="5" ClosedDate="2013-06-14T18:45:52.887" ContentLicense="CC BY-SA 3.0" />
  <row Id="7321" PostTypeId="1" AcceptedAnswerId="7322" CreationDate="2013-05-20T14:56:52.860" Score="5" ViewCount="142" Body="&lt;p&gt;I am just wondering where the term &lt;a href=&quot;http://en.wikipedia.org/wiki/Umbrella_sampling&quot;&gt;Umbrella Sampling&lt;/a&gt; came from. Is there another meaning of literal &quot;umbrella&quot; in physics or mathematics?&lt;/p&gt;&#xA;" OwnerUserId="4373" OwnerDisplayName="pyrami" LastActivityDate="2013-05-21T19:14:31.480" Title="The etymology of &quot;Umbrella Sampling&quot;" Tags="|terminology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7322" PostTypeId="2" ParentId="7321" CreationDate="2013-05-20T19:06:43.740" Score="5" Body="&lt;p&gt;The idea is to favor parts of phase space that would not be accessible within a normal Monte Carlo run. To do so, one uses a biasing potential around each of the unfavored regions. Basically, the biasing potential has a &quot;U&quot; shape in the region of interest which yields an &quot;umbrella&quot; shape for the biased probability to be there. When the region of interest is transition path, the idea is to subdivide the path in small regions each covered by an umbrella. The results once unbiased for each umbrella are then &quot;stiched&quot; together&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's the origin of the name I would say&lt;/p&gt;&#xA;" OwnerDisplayName="gatsu" LastActivityDate="2013-05-20T19:06:43.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7323" PostTypeId="2" ParentId="7315" CreationDate="2013-05-21T19:35:39.327" Score="0" Body="&lt;p&gt;The boundary conditions you posted look fine bar the indents which I assume is just because its on here. I would have a look at this &lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/1.0.0/python/demo/pde/subdomains-poisson/python/documentation.html&quot; rel=&quot;nofollow&quot;&gt;http://fenicsproject.org/documentation/dolfin/1.0.0/python/demo/pde/subdomains-poisson/python/documentation.html&lt;/a&gt; and check that you are implementing the boundary conditions correctly later on. For example for your left boundary condition you will still need:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;left = Left()&#xA;boundaries = FacetFunction(&quot;uint&quot;, mesh) ### size_t replaces uint from 1.1 on&#xA;boundaries.set_all(0)&#xA;left.mark(boundaries, 1)&#xA;bcs = [DirichletBC(V, 5.0, boundaries, 1)&#xA;ds = Measure(&quot;ds&quot;)[boundaries]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4296" LastActivityDate="2013-05-21T19:35:39.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7324" PostTypeId="2" ParentId="6921" CreationDate="2013-05-21T20:14:58.917" Score="1" Body="&lt;p&gt;From the top of my mind, or the rest of knowlegd about back-propagation neural networks, I would start with two layers with 10 perceptron in the first and 5 in the last using a the sigmoid logistic activation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 1/(1+exp(-\beta x)) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Entering dimension 10, output dimension 5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Better take a look at &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0132733501&quot; rel=&quot;nofollow&quot;&gt;Haykin, S. (1999). Neural Networks: A comprehensive foundation (2nd ed). Prentice Hall International.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chapter 4 Multlayer Perceptrons. If back-propagation NN most commom feed forward NN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if you did not take a look, check &lt;a href=&quot;http://en.wikipedia.org/wiki/Backpropagation&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you give more details about your problem we might be able to give a better asnwer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you mean by classification rule? &lt;/p&gt;&#xA;" OwnerUserId="4092" LastActivityDate="2013-05-21T20:14:58.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7325" PostTypeId="1" CreationDate="2013-05-21T21:28:55.147" Score="8" ViewCount="3070" Body="&lt;p&gt;I am trying to optimize some force-field parameters in a molecular framework so that the result of simulation comes as close as it can to the experimental structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the past, I have written a genetic algorithm where I essentially randomly sample the parameter space, select the combination that works best, create sets of mutated parameters, and repeat the process until I get the best parameters for some objective function.  I also perform some optimization of the algorithm itself, where the distribution of the mutated values is optimized to favor  faster convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advisor has not heard of genetic algorithms, and I have never heard of the methods he recommended: conjugate gradient method and the simplex algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my situation, the objective function is a function of every atom's deviation from its experimental location (so its a structural optimization).  The system is 4-10K atoms.  Is it worth it to invest some time into learning CGM or the simplex algorithm?  Out of all three, which is the best for this situation?&lt;/p&gt;&#xA;" OwnerUserId="4362" LastActivityDate="2013-05-22T09:09:28.630" Title="Genetic algorithm vs conjugate gradient method" Tags="|optimization|molecular-dynamics|conjugate-gradient|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7326" PostTypeId="2" ParentId="7325" CreationDate="2013-05-22T02:39:58.543" Score="12" Body="&lt;p&gt;The conjugate gradient method is good for finding the minimum of a strictly convex functional. This is typical when you reformulate a nonlinear elliptic PDE as an optimization problem. If you want to learn about it, I recommend you read about the CG method for linear systems first, for which &lt;a href=&quot;http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf&quot;&gt;An Introduction to the Conjugate Gradient Method Without the Agonizing Pain&lt;/a&gt; is a good reference. Unfortunately, many books treat it by pulling the algorithm out of a hat without the least concern for whether you understand it at an intuitive level, hence the title. From there, it's not a great leap to seeing how one could devise a nonlinear conjugate gradient method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I know about genetic algorithms, they would be better suited for finding the global minimum of an objective functional which has many local minima, which can be the case for molecular systems that have lots of meta-stable equilibria. In that case, the objective function isn't convex everywhere, which precludes the use of CG. The cost is the slower convergence of randomized algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether it's worth it for you to learn about CG depends on:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;whether or not CG is even applicable&lt;/li&gt;&#xA;&lt;li&gt;how fast your approach is already&lt;/li&gt;&#xA;&lt;li&gt;how much faster CG might be (my guess: a lot, if you can use it)&lt;/li&gt;&#xA;&lt;li&gt;how much time you will spend learning/coding CG&lt;/li&gt;&#xA;&lt;li&gt;how badly you want to not appear ignorant to your adviser&lt;/li&gt;&#xA;&lt;li&gt;how big you expect your problem to get later.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My two cents: it's a pretty neat tool, but I use it all the time so I'm biased.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2013-05-22T02:51:08.693" LastActivityDate="2013-05-22T02:51:08.693" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7327" PostTypeId="2" ParentId="7314" CreationDate="2013-05-22T05:19:03.677" Score="4" Body="&lt;p&gt;Reid and David have already answered the technical questions, but I'd like to provide some context to it anyway. Today, for most ODEs you can think of, you can achieve essentially perfect accuracy (say, $10^{-14}$) with most of the good ODE solver packages in essentially no time if evaluating $f(x)$ is free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are of course exceptions (very large systems, very stiff systems) but a common sentiment in the community is that the question of designing ODE solvers for &quot;standard&quot; systems is a solved one. Consequently, I think that the question you pose is not a very interesting one -- it addresses a component of ODE solver design that is no longer of importance. This may also explain the lack of literature on the subject.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-22T05:19:03.677" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7328" PostTypeId="2" ParentId="7270" CreationDate="2013-05-22T08:54:46.067" Score="2" Body="&lt;p&gt;If you approximate a function on an interval by polynoms and you have the freedom to choose &lt;code&gt;x&lt;/code&gt;, the approximation points, the recommend method is to use gauss quadrature points and the corresponding polynom basis. Once the approximation is obtained you are free to evaluate it of course anywhere in the interval.&#xA;So use &lt;a href=&quot;http://en.wikipedia.org/wiki/Chebyshev_polynomials&quot; rel=&quot;nofollow&quot;&gt;chebyshev polynoms&lt;/a&gt; together with &lt;a href=&quot;http://en.wikipedia.org/wiki/Chebyshev_nodes&quot; rel=&quot;nofollow&quot;&gt;chebyshev nodes&lt;/a&gt;. This way you won't see any oscillations and you can easily integrate / differentiate the solution on the entire domain.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2013-05-22T08:54:46.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7329" PostTypeId="2" ParentId="7325" CreationDate="2013-05-22T09:09:28.630" Score="3" Body="&lt;p&gt;You can also look at &lt;a href=&quot;http://en.wikipedia.org/wiki/CMA-ES&quot; rel=&quot;nofollow&quot;&gt;CMAES&lt;/a&gt;. It essentially boils down to CG for convex functions, yet represents global and robust optimizer for other types of functions (including non-convex functions with multiple minima). I have not seen, however, its application to anything larger than a couple of hundreds unknowns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note also that CG can be applied in combination with Tikhonov regularization which, in a way, makes your function more convex and easier to minimize, although at the cost of some bias in the solution, which is often acceptable trade off.&lt;/p&gt;&#xA;" OwnerUserId="1089" LastActivityDate="2013-05-22T09:09:28.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7330" PostTypeId="1" AcceptedAnswerId="7332" CreationDate="2013-05-22T09:32:26.707" Score="4" ViewCount="4029" Body="&lt;p&gt;I am confused most of the time with the application of boundary conditions in Fenics. Suppose I have an xml mesh for a rectangular beam and want to apply Dirichlet BC ($u =0$) on the left side of the beam in my mesh file. Can anyone tell me how can I achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know another way of doing it by defining subdomains like here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Left(SubDomain):&#xA;def inside(self, x, on_boundary):&#xA;    return near(x[0], 0.0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But for complicated geometries, I am sure this method is not useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4350" LastEditorUserId="1748" LastEditDate="2013-05-22T16:57:33.830" LastActivityDate="2013-05-23T12:42:56.287" Title="Fenics: Meshfunction usage" Tags="|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7331" PostTypeId="1" CreationDate="2013-05-22T10:34:23.777" Score="3" ViewCount="591" Body="&lt;p&gt;Is there a code for the equation&#xA;$$&#xA;\frac{\partial^2u}{\partial x^2}+\frac{\partial^2u}{\partial y^2} = \sin(u)&#xA;$$&#xA;or for the sine gordon equation in two dimensions because I want to change some boundary values to see the results?&lt;/p&gt;&#xA;" OwnerUserId="4366" LastEditorUserId="1804" LastEditDate="2013-05-22T14:25:01.803" LastActivityDate="2013-05-22T15:55:10.827" Title="Fenics: time-independent Sine-Gordon equation" Tags="|finite-element|fenics|nonlinear-equations|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7332" PostTypeId="2" ParentId="7330" CreationDate="2013-05-22T11:11:59.370" Score="8" Body="&lt;p&gt;That depends how did you generated the file. For example your mesh generator can produce some cell or facet markers according to its input. Then &lt;code&gt;dolfin-convert&lt;/code&gt; script may or may not succeed converting all these markers to XML. They're then stored within XML mesh file or in separate XML. You can then use them as &lt;code&gt;FacetFunction&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# if stored within mesh&#xA;facet_domains = mesh.domains().facet_domains(mesh) # older versions of DOLFIN&#xA;facet_domains = mesh.domains().facet_domains() # newer versions of DOLFIN&#xA;&#xA;# if stored separately&#xA;facet_domains = MeshFunction('uint', mesh, file) # older versions of DOLFIN&#xA;facet_domains = MeshFunction('size_t', mesh, file) # newer versions of DOLFIN&#xA;&#xA;# now define homogenous Dirichlet BC where facet_domains is equal to 42&#xA;bc42 = DirichletBC(V, 0.0, facet_domains, 42)&#xA;&#xA;# now integrate over interior facets where facet_domains equals 43&#xA;dS = Measure('dS')[facet_domains]&#xA;dS43 = dS(43)&#xA;integral43 = foo*dS43&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If your mesh generator does not supply you with markers you need, you can produce first these markers using &lt;code&gt;EntityIterators&lt;/code&gt;, for example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;facet_domains = FacetFunction('size_t', mesh, 0)&#xA;for c in cells(mesh):&#xA;    # here some conditions on c&#xA;    for f in facets(c):&#xA;        # here some conditions on f&#xA;        facet_domains[f] = some_computed_value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can use these &lt;code&gt;facet_domains&lt;/code&gt; as suggested above.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Application of &lt;code&gt;DirichletBC&lt;/code&gt; at one particular vertex&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining &lt;code&gt;DirichletBC&lt;/code&gt; in one vertex is not possible through &lt;code&gt;MeshFunction&lt;/code&gt;s as &lt;code&gt;DirichletBC&lt;/code&gt; only supports &lt;code&gt;FacetFunction&lt;/code&gt;s. But you can take advantage of &lt;code&gt;DirichletBC(..., method='pointwise')&lt;/code&gt; and do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;class Pinpoint(SubDomain):&#xA;    TOL = 1e-3&#xA;    def __init__(self, coords):&#xA;        self.coords = np.array(coords)&#xA;        SubDomain.__init__(self)&#xA;    def move(self, coords):&#xA;        self.coords[:] = np.array(coords)&#xA;    def inside(self, x, on_boundary):&#xA;        return np.linalg.norm(x-self.coords) &amp;lt; TOL&#xA;&#xA;pinpoint = Pinpoint([0.,0.,0.])&#xA;&#xA;V = VectorFunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;bc = DirichletBC(V.sub(0), 0.0, pinpoint, 'pointwise')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this has only meaning for &lt;code&gt;V&lt;/code&gt; being continuous piecewise linear functions. Constructor &lt;code&gt;Pinpoint.__init__&lt;/code&gt; takes an &lt;code&gt;coords&lt;/code&gt; argument (of type &lt;code&gt;tuple&lt;/code&gt;, &lt;code&gt;list&lt;/code&gt; or &lt;code&gt;np.array&lt;/code&gt;) allowing of specification of coordinates where you want to apply BC. Also method &lt;code&gt;Pinpoint.move&lt;/code&gt; can be useful to change this point without a need of creating new &lt;code&gt;DirichletBC&lt;/code&gt; instance - especially when &lt;code&gt;mesh&lt;/code&gt; moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is to directly fiddle with &lt;code&gt;Tensor&lt;/code&gt;s' entries.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-23T12:42:56.287" LastActivityDate="2013-05-23T12:42:56.287" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7333" PostTypeId="1" AcceptedAnswerId="7334" CreationDate="2013-05-22T12:58:14.373" Score="6" ViewCount="548" Body="&lt;p&gt;I'm solving the &lt;a href=&quot;http://en.wikipedia.org/wiki/Helmholtz_equation&quot; rel=&quot;nofollow&quot;&gt;Helmholtz equation&lt;/a&gt; using PETSc.  I found with the PETSc configure option &lt;code&gt;--download-f-blas-lapack&lt;/code&gt; my program runs twice as fast over running it with MKL. Is this a common trend or are there other factors at play? I'm using gcc and Open MPI with PETSc.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="9" LastEditDate="2013-05-23T09:35:20.637" LastActivityDate="2013-05-23T09:35:20.637" Title="Reference BLAS/LAPACK from NETLIB is twice as fast as MKL for complex numbers" Tags="|petsc|performance|blas|complex-arithmetic|intel-mkl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7334" PostTypeId="2" ParentId="7333" CreationDate="2013-05-22T13:45:17.913" Score="9" Body="&lt;p&gt;This is usually caused by trying to use a threaded MKL combined with MPI, resulting in over-subscription.  Either explicitly configure PETSc to use non-threaded MKL or add &lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/mkl/lin/MKL_UG_managing_performance/Using_Additional_Threading_Control.htm&quot;&gt;&lt;code&gt;MKL_NUM_THREADS=1&lt;/code&gt;&lt;/a&gt; to your environment.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-05-22T13:45:17.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7335" PostTypeId="2" ParentId="7331" CreationDate="2013-05-22T14:16:29.753" Score="5" Body="&lt;p&gt;That's actually very easy to do in Dolfin:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from dolfin import *&#xA;# define mesh, function space (piecewise linear)&#xA;mesh = UnitSquareMesh(64,64)&#xA;V = FunctionSpace(mesh,'CG',1)&#xA;# inhomogeneous boundary conditions (otherwise the solution is trivial)&#xA;bc = DirichletBC(V, Constant(1.0), lambda x,on_boundary: on_boundary)&#xA;# define bi(non)linear form&#xA;# note that nonlinear problems need u to be a Function, not TestFunction&#xA;u = Function(V)&#xA;v = TestFunction(V)&#xA;F = inner(grad(u),grad(v))*dx + sin(u)*v*dx&#xA;# solve using Newton method (Jacobian is computed automatically)&#xA;solve(F==0,u,bc)&#xA;plot(u,interactive=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is also a &lt;code&gt;nonlinear-poisson&lt;/code&gt; demo in the &lt;code&gt;demo/undocumented&lt;/code&gt; directory, which also shows how to enter define more complicated nonlinear forms and thow to control the solver parameters.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-05-22T15:55:10.827" LastActivityDate="2013-05-22T15:55:10.827" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="7336" PostTypeId="1" CreationDate="2013-05-22T14:50:51.650" Score="2" ViewCount="455" Body="&lt;p&gt;My program (in Python, using FEniCS):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh=UnitSquareMesh(16,16)&#xA;U_h = VectorFunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;B = VectorFunctionSpace(mesh, &quot;Bubble&quot;, degree=3,dim=2)&#xA;Mini_h = U_h + B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Boundary definition (for example, &lt;code&gt;u0_boundary&lt;/code&gt; and &lt;code&gt;u1_boundary&lt;/code&gt; are part of square, respectively):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w0 = Constant(&quot;0.0&quot;)&#xA;bc0 = DirichletBC(Mini_h.sub(0), w0, u0_boundary) #error&#xA;bc1 = DirichletBC(Mini_h.sub(1), w0, u1_boundary)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get: &lt;code&gt;ValueError: Can only extract SubSpaces with i = 0 ... -1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Who can tell me why? &lt;code&gt;Mini_h.sub(0)&lt;/code&gt; is the first component of &lt;code&gt;Mini_h&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, the problem is only for vector bubble element, everything is OK. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;B = VectorFunctionSpace(mesh, &quot;Bubble&quot;, degree=3,dim=2)&#xA;bc0 = DirichletBC(B.sub(0), w0, u0_boundary)&#xA;bc1 = DirichletBC(B.sub(1), w0, u1_boundary)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In addition, I want to know why can not directly added to a mini finite element, rather than CG + Bubble.&lt;/p&gt;&#xA;" OwnerUserId="4369" LastEditorUserId="1748" LastEditDate="2013-05-22T16:57:49.630" LastActivityDate="2013-05-27T13:13:41.830" Title="2d-mini element-can't use sub(0),but bubble element is ok" Tags="|python|fenics|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7338" PostTypeId="1" AcceptedAnswerId="7339" CreationDate="2013-05-22T16:17:38.353" Score="4" ViewCount="7914" Body="&lt;p&gt;I generated a mesh file in gambit and wanted to convert it to xml format. I tried the code below, but there is no output.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *    &#xA;from dolfin_utils import meshconvert&#xA;&#xA;dolfin-convert cylinder6.msh out.xml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;can anyone tell me how to convert using dolfin??&lt;/p&gt;&#xA;" OwnerUserId="4350" LastEditorUserId="16947" LastEditDate="2016-10-25T22:52:03.360" LastActivityDate="2018-04-26T09:03:57.800" Title="Fenics: msh to xml conversion" Tags="|fenics|mesh|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7339" PostTypeId="2" ParentId="7338" CreationDate="2013-05-22T16:19:17.443" Score="7" Body="&lt;p&gt;Use dolfin-convert from a terminal window.&lt;/p&gt;&#xA;" OwnerUserId="4292" LastActivityDate="2013-05-22T16:19:17.443" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7340" PostTypeId="2" ParentId="7336" CreationDate="2013-05-22T18:40:27.850" Score="2" Body="&lt;p&gt;The vector part of the MINI element is not a mixed function space, which is why you cannot extract a subspace from it. The bubble is an enrichment (which is indicated by the '&lt;code&gt;+&lt;/code&gt;' in your construction of &lt;code&gt;Mini_h&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distinction between mixed and enriched spaces is discussed in Section 2.3 of the UFL paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/1211.4047&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/abs/1211.4047&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="3952" LastActivityDate="2013-05-22T18:40:27.850" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7341" PostTypeId="2" ParentId="7336" CreationDate="2013-05-22T19:24:51.377" Score="0" Body="&lt;p&gt;You have two options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;enforce Dirichlet condition for all components &lt;code&gt;DirichletBC(Mini_h, u_D, domain)&lt;/code&gt; where &lt;code&gt;u_D&lt;/code&gt; is some &lt;code&gt;Function&lt;/code&gt;, &lt;code&gt;Expression&lt;/code&gt; or &lt;code&gt;Constant&lt;/code&gt; of appropriate dimension, or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;use coordinate-free approach, for example &lt;a href=&quot;https://scicomp.stackexchange.com/q/7123/4254&quot;&gt;FEniCS: separate boundary conditions in normal and tangential direction of mesh boundary&lt;/a&gt;, but I'm not sure if it can be succesfully implemented in FEniCS.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-05-22T19:24:51.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7342" PostTypeId="1" AcceptedAnswerId="7353" CreationDate="2013-05-22T21:15:23.123" Score="2" ViewCount="199" Body="&lt;p&gt;I'm looking for sparse SPD matrices with right hand side?  There is UF collection of sparse matrices, however, I'm not sure how do I search of the matrices of these kind efficiently (I'm doing a naive search which hasn't given me any results so far and it takes arbitrary long for some of the matrices). &lt;/p&gt;&#xA;" OwnerUserId="3360" LastEditorUserId="1804" LastEditDate="2016-05-27T20:43:41.420" LastActivityDate="2016-05-27T20:43:41.420" Title="SPD matrices with right hand sides" Tags="|sparse-matrix|matrix|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7344" PostTypeId="2" ParentId="3400" CreationDate="2013-05-23T06:34:09.660" Score="4" Body="&lt;p&gt;I am not familiar with the equations given here, but I remember learning another method for checking the stability of a numerical scheme in my coursework. It is known as Modified Equation analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a good reference for that,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;http://193.146.160.29/gtb/sod/usu/$UBUG/repositorio/10291890_Warming.pdf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the above reference, the connection between stability theory based on Modified Equation analysis and Von Neumann stability analysis is established. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a bit of online search, I came across following references, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1029/2006JB004366/full&quot; rel=&quot;nofollow&quot;&gt;This&lt;/a&gt; paper discusses Finite difference modeling of Biot's poroelastic equations at seismic frequencies. It has a section on stability of numerical scheme as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://www.simtech.uni-stuttgart.de/publikationen/prints.php?ID=683&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; presents a solution strategy of decoupling the coupled system, and checking the stability of numerical scheme.  &lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="1567" LastEditDate="2013-05-23T06:41:53.880" LastActivityDate="2013-05-23T06:41:53.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7345" PostTypeId="2" ParentId="3400" CreationDate="2013-05-23T12:29:43.947" Score="8" Body="&lt;p&gt;If you substitute, at least for your analysis, $\frac{\partial u}{\partial x}$ by $u_x$, you can write your system as&#xA;$$ \begin{bmatrix} 0 &amp;amp; 0 \\ I &amp;amp; I \end{bmatrix} \frac{d}{dt} \begin{bmatrix} p_h(t) \\ u_{x,h}(t) \end{bmatrix}+\begin{bmatrix}  -\partial_h &amp;amp; \partial_h \\ -\Delta_h &amp;amp; 0 \end{bmatrix}\begin{bmatrix} p_h(t) \\ u_{x,h}(t) \end{bmatrix}= \begin{bmatrix} q_h(t) \\ 0 \end{bmatrix} \quad (*)$$&#xA;where all constants are set to $1$ and where the subscript ${}_h$ refers to the space discretization both of the variables and the differential operators. Your scheme is then obtained by approximating $\frac{d}{dt} $ via implicit Euler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the differential-algebraic (DAE) structure is evident. For the variables there are both differential (in time) and algebraic equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can show that  $ \begin{bmatrix} -\partial_h &amp;amp; \partial_h \\ I &amp;amp; I \end{bmatrix}$ is invertible, cf. this &lt;a href=&quot;http://www3.math.tu-berlin.de/preprints/abstracts/Report-16-2011.rdf.html&quot; rel=&quot;noreferrer&quot;&gt;preprint [p. 3]&lt;/a&gt; and the edit below, than the DAE is of &lt;em&gt;index 1&lt;/em&gt; or &lt;em&gt;strangeness-free&lt;/em&gt; and implicit Euler is known to be convergent, see Theorem 5.12 in this &lt;a href=&quot;http://www.ems-ph.org/books/book.php?proj_nr=36&amp;amp;srch=browse_authors%7CMehrmann,%20Volker&quot; rel=&quot;noreferrer&quot;&gt;book&lt;/a&gt;. (Disclaimer: This book is not freely available and written by my PhD supervisor) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this approach, you maybe get around the stability analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a direct proof of $L^2$ stability, I would try to use Equation $(*)$ to apply von Neumann stability analysis using the eigenfunctions of $\Delta_h$ and investigating the effect of $\partial_h$ onto the eigenfunctions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt;, if stability cannot be established for $(*)$, it doesn't mean your scheme is not convergent -- because of the substitution of $u\leftarrow u_x $. Generally speaking, one can expect stability for schemes approximating the actual variables, rather than for schemes approximating their derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;APPENDIX:&lt;/strong&gt;&#xA;A DAE is said to be index 1, if it can be transformed into an ODE without differentiating the equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say, the DAE is of the form&#xA;$$ \begin{bmatrix}&#xA;	E_1 \\ 0&#xA;\end{bmatrix}\dot y + &#xA;\begin{bmatrix}&#xA;	A_1 \\ A_2&#xA;\end{bmatrix} y  = f. $$&#xA;Then invertibility of &#xA;$ \begin{bmatrix}&#xA;	E_1 \\ A_2&#xA;\end{bmatrix}$ implies, that there is a variable transform $\tilde y \to y$ that eventually swaps the columns of the coefficients so that $ \begin{bmatrix}&#xA;	E_1 \\ A_2&#xA;\end{bmatrix} \to &#xA;\begin{bmatrix}&#xA;	\tilde E_{11} &amp;amp; \tilde E_{12} \\ \tilde A_{21} &amp;amp; \tilde A_{22} &#xA;\end{bmatrix}$&#xA;with $\tilde A_{22}$ invertible (full rank property of $A_2$) and $\tilde A_{11} - \tilde E_{12} \tilde A_{22}^{-1} \tilde A_{21}$ invertible (the Schur complement). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the system $(*)$ this means that the algebraic part defined with $A_2:=[-\partial_h ~ \partial_h]$ can be used to solve for a part $\tilde y_2$ of $(p_h,u_{x,h})$. Then, one can eliminate $\frac{d}{dt}\tilde y_2$ from the differential part (the second block line in $(*)$), to obtain an ODE for the remaining variables.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-25T12:56:15.583" LastActivityDate="2013-05-25T12:56:15.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7346" PostTypeId="1" CreationDate="2013-05-23T13:38:02.837" Score="5" ViewCount="653" Body="&lt;p&gt;I plans to use &lt;a href=&quot;https://projects.coin-or.org/Csdp/&quot; rel=&quot;nofollow&quot;&gt;CSDP&lt;/a&gt; to solve the following semi-definite problem:&#xA;$$\min_{B, \beta}\operatorname{trace}(CB) \\&#xA;\text{s.t.} \ \operatorname{trace}(AB)=1 \\&#xA;\beta\geqslant 0 \\&#xA;\begin{bmatrix}&#xA;1 &amp;amp; \beta^{T}\\ &#xA; \beta&amp;amp; B&#xA;\end{bmatrix}\succeq 0$$&#xA;where $A$, $B$, $C$ are $M\times M$ matrices,and $\beta$ is an $M$-dimensional column vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I have some difficulty in transforming it to the standard form:&#xA;$$\max_{X}\operatorname{trace}(CX) \\&#xA;\text{s.t.} \ \operatorname{trace}(A_{i}X)=a \\&#xA;X\succeq 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="1748" LastEditDate="2013-05-23T16:15:45.333" LastActivityDate="2013-05-23T17:20:45.913" Title="How to transform such an SDP to standard form" Tags="|convex-optimization|semidefinite-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7347" PostTypeId="2" ParentId="2369" CreationDate="2013-05-23T13:41:34.943" Score="1" Body="&lt;p&gt;You can easily create a rather good preconditioner for elasticity if your material are not too incompressible (Poisson ration $&amp;lt; 0.4$) by using &lt;a href=&quot;http://www.math.kun.nl/onderzoek/reports/rep2003/rep03_16.ps.gz&quot; rel=&quot;nofollow&quot;&gt;separate displacement components preconditioner&lt;/a&gt;. The idea is simply to have a block diagonal matrix with 3 blocks, each of which represents the couplings along the same physical dimension $(x,y,z)$ ie: $K_{xx}$, $K_{yy}$, $K_{zz}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case you can use less advanced AMG methods to compute an approximation of each block inverse and get a pretty good preconditioner.&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="1748" LastEditDate="2013-05-23T16:15:34.333" LastActivityDate="2013-05-23T16:15:34.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7348" PostTypeId="2" ParentId="6971" CreationDate="2013-05-23T14:41:29.360" Score="3" Body="&lt;p&gt;&lt;code&gt;s&lt;/code&gt; sets a target residual value; &lt;code&gt;s=0&lt;/code&gt; interpolates the data.&#xA;Use &lt;code&gt;get_residual()&lt;/code&gt; to look at the &lt;code&gt;s&lt;/code&gt; achieved, and &lt;code&gt;get_knots()&lt;/code&gt; to look at the knots,&#xA;as in the little test below.&#xA;You see that  &lt;code&gt;s=0&lt;/code&gt; has about &lt;code&gt;N = len(x)&lt;/code&gt; knots,&#xA;and increasing &lt;code&gt;s&lt;/code&gt; =&gt; fewer knots (polynomial pieces).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;s&lt;/code&gt; is the sum of residuals^2, |y - yinterpolated|^2, at the input data points;&#xA;this scales with &lt;code&gt;y&lt;/code&gt; and &lt;code&gt;N&lt;/code&gt;, making it difficult to use.&#xA;To set a target &lt;code&gt;rmserror&lt;/code&gt; like 1 % of max |y|, set &lt;code&gt;s&lt;/code&gt; like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s = N * (rmserror * np.fabs(y).max())**2&#xA;UnivariateSpline( x, y, s=s )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;s=None&lt;/code&gt; uses a not-very-useful value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/t6OVt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;&quot;&quot; scipy UnivariateSpline sensitivity to smoothing parameter s&#xA;    interpolate N -&amp;gt; N*H points with various s&#xA;&quot;&quot;&quot;&#xA;# 2013-01-27 Jan denis&#xA;&#xA;from __future__ import division&#xA;import sys&#xA;import numpy as np&#xA;from scipy.interpolate import UnivariateSpline  # $scipy/interpolate/fitpack2.py&#xA;    # http://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html&#xA;    # InterpolatedUnivariateSpline : Subclass with smoothing forced to 0&#xA;&#xA;#...............................................................................&#xA;N = 10&#xA;H = 5  # xfine ~ N * H&#xA;freq = .2&#xA;noise = 0&#xA;s = [0, 1, 2, None]  # smoothing parameter, scales with y; s=0 interpolates&#xA;plot = 0&#xA;seed = 0&#xA;&#xA;exec( &quot;\n&quot;.join( sys.argv[1:] ))  # run this.py N= ...  from sh or ipython&#xA;np.set_printoptions( 1, threshold=100, edgeitems=5, suppress=True )&#xA;np.random.seed(seed)&#xA;if plot:&#xA;    import pylab as pl&#xA;    pl.figure( figsize=[10,4] )&#xA;&#xA;#...............................................................................&#xA;x = np.arange( N )&#xA;y = np.sin( 2*np.pi * freq * x )  # &amp;lt;-- your function here&#xA;xfine = np.arange( (N - 1) * H + 1. ) / H  # |...|...|...|&#xA;if noise &amp;gt; 0:&#xA;    y += np.random.normal( 0, noise, y.shape )&#xA;&#xA;#...............................................................................&#xA;title = &quot;UnivariateSpline N=%d H=%d s=%s&quot; % (N, H, s)&#xA;print 80 * &quot;-&quot;&#xA;print title&#xA;for s in s:&#xA;    uspline = UnivariateSpline( x, y, s=s )  # s=0 interpolates&#xA;    yfine = uspline( xfine )  # y interpolated n a fine grid&#xA;&#xA;    res = uspline.get_residual()&#xA;    sumres2 = np.sum( (y - yfine[::H]) ** 2 )  # == res == s&#xA;    rms = np.sqrt( sumres2 / N )&#xA;    knots = uspline.get_knots()&#xA;    label = &quot;s %s  res %.2g  rms %.1g  knots %d &quot; % (s, res, rms, len(knots))&#xA;    knots = uspline.get_knots()&#xA;    label = &quot;s %s  res %.2g  knots %d &quot; % (s, res, len(knots))&#xA;    print label, knots&#xA;    print &quot;&quot;&#xA;    if plot:&#xA;        pl.plot( xfine, yfine, label=label )&#xA;&#xA;if plot:&#xA;    pl.title(title)&#xA;    pl.legend()&#xA;    pl.savefig( &quot;tmp.png&quot; )&#xA;    pl.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2013-05-24T13:55:12.140" LastActivityDate="2013-05-24T13:55:12.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7350" PostTypeId="2" ParentId="2541" CreationDate="2013-05-23T14:57:59.233" Score="1" Body="&lt;p&gt;For fast easy spline interpolation on a uniform grid in 1d 2d 3d and up,&#xA;I recommend &lt;a href=&quot;http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.ndimage.interpolation.map_coordinates.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.ndimage.map_coordinates&lt;/a&gt;;&#xA;see the plot and example code under&lt;br&gt;&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/6238250/multivariate-spline-interpolation-in-python-scipy&quot;&gt;multivariate-spline-interpolation-in-python-scipy&lt;/a&gt; on SO.&lt;br&gt;&#xA;For smoothly-varying nonuniform grids, there's a helper class&#xA;&lt;a href=&quot;http://denis-bz.github.com/docs/intergrid.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intergrid&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-05-23T14:57:59.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7351" PostTypeId="2" ParentId="7346" CreationDate="2013-05-23T17:15:35.817" Score="3" Body="&lt;p&gt;[NOTE: I'm using $n$ instead of $M$ as the key dimension here, sorry... it's too much trouble to go back and change everything...]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, $X$ is actually going to be an $2n+1\times 2n+1$ matrix with this block structure when you are done:&#xA;$$\begin{bmatrix} 1 &amp;amp; \beta^T \\ \beta &amp;amp; B \\ &amp;amp; &amp;amp; \beta_1 \\ &amp;amp; &amp;amp; &amp;amp; \beta_2 \\ &amp;amp;  &amp;amp; &amp;amp; &amp;amp; \ddots \\ &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; \beta_n \end{bmatrix}$$&#xA;So in addition to your single equality constraint you need to make sure that $X_{1,1}=1$ and $X_{k+1,1}=X_{n+k+1,n+k+1}$ for $k=1,2,\dots,n+1$. So you have $n+2$ equality constraints to build. So I think it looks something like this:&#xA;$$\bar{C} = \begin{bmatrix} 0 &amp;amp; 0_{1\times n} \\ 0_{n\times 1} &amp;amp; C \\ &amp;amp; &amp;amp; 0\\ &amp;amp; &amp;amp; &amp;amp; 0\\ &amp;amp;  &amp;amp; &amp;amp; &amp;amp; \ddots \\ &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; 0 \end{bmatrix}$$&#xA;$$\bar{A}_1 = \begin{bmatrix} 0 &amp;amp; 0_{1\times n} \\ 0_{n\times 1} &amp;amp; A \\ &amp;amp; &amp;amp; 0\\ &amp;amp; &amp;amp; &amp;amp; 0\\ &amp;amp;  &amp;amp; &amp;amp; &amp;amp; \ddots \\ &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; 0 \end{bmatrix}, \quad a_1 = 1$$&#xA;$$\bar{A}_2 = \begin{bmatrix} 1 &amp;amp; 0_{1\times n} \\ 0_{n\times 1} &amp;amp; 0_{n\times n} \\ &amp;amp; &amp;amp; 0\\ &amp;amp; &amp;amp; &amp;amp; 0\\ &amp;amp;  &amp;amp; &amp;amp; &amp;amp; \ddots \\ &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; 0 \end{bmatrix}, \quad a_2 = 1$$&#xA;$$\bar{A}_{k+2} = \begin{bmatrix} 0 &amp;amp; \tfrac{1}{2}e_k^T \\ \tfrac{1}{2}e_k &amp;amp; 0_{n\times n} \\ &amp;amp; &amp;amp; -\mathop{\textrm{diag}}(e_k) \end{bmatrix}, \quad a_{2+k} = 0, \quad k=1,2,\dots, n$$&#xA;where $e_k$ is the $n$-vector with $1$ in the $k$th position and zeros elsewhere. See the section &quot;Using the subroutine interface to CSDP&quot; in the CSDP user guide for information about how to translate this block structure into CSDP's preferred format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, since I have you here: it would be much easier for you if you did not have to plug this into CSDP. If running in MATLAB is acceptable to you, you could use my toolbox &lt;a href=&quot;http://cvxr.com/cvx&quot; rel=&quot;nofollow&quot;&gt;CVX&lt;/a&gt; to solve this, and the model would look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cvx_begin sdp&#xA;    variables B(n,n) b(n)&#xA;    minimize(trace(C*B))&#xA;    subject to&#xA;        trace(A*B) == 1;&#xA;        [ 1, b' ; b, B ] &amp;gt;= 0;&#xA;        b &amp;gt;= 0; &#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course, if you're stuck in C, I understand! CSDP is a very nice solver particularly for standalone applications. I hope I have been helpful either way.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2013-05-23T17:20:45.913" LastActivityDate="2013-05-23T17:20:45.913" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7352" PostTypeId="2" ParentId="7314" CreationDate="2013-05-23T17:51:25.430" Score="1" Body="&lt;p&gt;My reasoning is the following. Since your problems are stiff and not big then most probably you should use some implicit method. If you use implicit method then you solve at least one linear system on each step. It means that if the Jacobian is not structured then each step will take $O(\mathit{dim}^3)$ or $O(\mathit{dim}^2)$ flops on each step (depending on how frequently you LU-decompose the Jacobian), &lt;em&gt;aside from&lt;/em&gt; evaluating the RHS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the first point is to make sure if your RHS is really more expensive than the underlying linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second point: it is known from literature that solvers based on &quot;expensive&quot; methods (i. e. explicit RK methods) sometimes perform faster than &quot;cheaper&quot; ones (explicit multistep methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summing up, I think that you should not only consider the RHS evaluations count.&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="127" LastEditDate="2013-05-23T17:55:12.083" LastActivityDate="2013-05-23T17:55:12.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7353" PostTypeId="2" ParentId="7342" CreationDate="2013-05-23T18:06:49.850" Score="8" Body="&lt;p&gt;&lt;em&gt;[The comment thread is long enough to convert to an answer.]&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We acknowledge that most matrix collections don't include physically reasonable right hand sides.  If we want to test solver performance, we have to generate problems to solve --- either the right-hand sides $b$ or solutions $x$ from which we compute $b \gets A x$ and then solve $A y = b$ to some precision.  The latter is convenient because it lets us check &lt;em&gt;accuracy&lt;/em&gt;.  Solvers like unpreconditioned GMRES are optimal (over a subspace) in the 2-norm of the residual, thus the $A^* A$-norm of the error.  This is a significantly weaker norm than the 2-norm of the error, for example, thus it really is useful to compare solvers in the error norm.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Dangers of choosing a random vector&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;A common method of generating $x$ or $b$ is to draw from an independent random distribution.  This is flawed because these random vectors will be nearly orthogonal to the eigenvectors associated with the smallest eigenvalues -- exactly the modes that slow solver convergence.  For example, if you use a Gaussian distribution with mean zero for a Neumann Laplacian (has a constant null space), then the average over any significant portion of the domain will be nearly zero, meaning that no long-range interactions take place.  In this case, a coarse grid becomes unnecessary since you only have to move information far enough for the random process to homogenize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your distribution is orthogonal to that lowest eigenvector, every random vector ($x$ or $b$ doesn't matter when comparing to an eigenvector $z$, though if $z^T x$ is close to zero, $z^T b = z^T A x$ will be even smaller) that you draw will have this property of being nearly orthogonal to the &quot;problematic modes&quot;, thus converging artificially fast.  An independent random distribution will be orthogonal to most or all these &quot;problematic modes&quot; (e.g., translations and rotations for elasticity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This mistake gets published from time to time, so it's worth increasing awareness.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;A Fix&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you don't know anything about the problem, it's inconvenient to tune the random distribution so that its expected value has nonzero component in the direction of all the low-energy eigenvectors---you would have to solve an expensive eigenproblem to find those &quot;bad&quot; vectors.  Instead, I would recommend the following ad-hoc procedure:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Recursively bisect the graph.&lt;/li&gt;&#xA;&lt;li&gt;At each level of the bisection tree, choose a random value from a random distribution with nonzero mean.&lt;/li&gt;&#xA;&lt;li&gt;Define the vector $x$ to be the sum of the random value assigned to all the parts that contain it (i.e., the sum of the path from the root of the tree).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is much like random sampling in a wavelet basis and provides correlation at all scales.  It doesn't require any special knowledge about the SPD matrix other than its sparsity pattern.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Extensions and alternatives&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[Collecting comments from offline discussion.]&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Michael Grant asks &lt;strong&gt;&quot;What about dense systems?&quot;&lt;/strong&gt;&#xA;My reply: For an unstructured dense matrix, I would threshold since that should be some measure of locality.  For a structured dense matrix (e.g., an $H$-matrix), you already have a hierarchy.  The details of the bisection/partitioning isn't that important. All it needs to do is provide correlation at different scales so that the distribution isn't orthogonal to the problematic eigenvectors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Jack Poulson suggests &lt;strong&gt;using a combination of &quot;point-sources, plane waves, wave packets, and Gaussian random vectors.&quot;&lt;/strong&gt;&#xA;My reply: Plane waves and wave packets require some additional knowledge beyond the matrix itself.  Point sources aren't great for heterogeneous media problems because the Green's functions might be local almost everywhere, but global in very select places (e.g., faults, wires). The chance of a random point source activating that long-range coupling is low, but it's important for the real system. The bulk of the answer above was explaining why vectors drawn from an independent random distribution are inadequate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-05-24T17:08:24.150" LastActivityDate="2013-05-24T17:08:24.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7354" PostTypeId="1" AcceptedAnswerId="7355" CreationDate="2013-05-23T23:41:15.700" Score="6" ViewCount="951" Body="&lt;p&gt;I need to solve the following equation in FEniCS:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\boldsymbol{\nabla} \cdot \begin{pmatrix}&#xA;  f(y)\frac{\partial u}{\partial x} - g(x,y)\frac{\partial u}{\partial y} \\&#xA;  - g(x,y)\frac{\partial u}{\partial x}+f(x)\frac{\partial u}{\partial y}&#xA; \end{pmatrix}=0.&#xA;$$&lt;/span&gt;&#xA;As far as I understand, the weak form will look like:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int_{\Omega} \begin{pmatrix}&#xA;  f(y)\frac{\partial u}{\partial x} - g(x,y)\frac{\partial u}{\partial y} \\&#xA;  - g(x,y)\frac{\partial u}{\partial x}+f(x)\frac{\partial u}{\partial y}&#xA; \end{pmatrix} \cdot \nabla v \, \mathrm{d}x =&#xA;       0 \quad \forall v \in V.&#xA;$$&lt;/span&gt;&#xA;I have problems expressing this equation in FEniCS. I figured I need something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;u = TrialFunction(V)&#xA;v = TestFunction(V)&#xA;g = as_vector((u.dx(0), u.dx(1))) # &amp;lt;-- modify here?&#xA;a = dot(g, grad(v))*dx&#xA;u = Function(V)&#xA;solve(a == Constant(0) * v * dx, u, bc)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to add something that depends on coordinates &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; to the variable &lt;code&gt;g&lt;/code&gt; above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also do I have to write &lt;code&gt;Constant(0) * v * dx&lt;/code&gt; as the right part? Can it be just 0?&lt;/p&gt;&#xA;" OwnerUserId="4379" LastEditorUserId="4379" LastEditDate="2018-11-01T17:51:48.827" LastActivityDate="2018-11-01T17:51:48.827" Title="FEniCS: how to access coordinates when writing an equation for a trial function" Tags="|finite-element|python|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7355" PostTypeId="2" ParentId="7354" CreationDate="2013-05-24T00:31:22.920" Score="8" Body="&lt;pre&gt;&lt;code&gt;x = V.cell().x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then use &lt;code&gt;x[0]&lt;/code&gt; and &lt;code&gt;x[1]&lt;/code&gt; as $x$ and $y$ respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right-hand side of &lt;code&gt;LinearVariationalProblem&lt;/code&gt; needs to be rank 1 form - this is expressed by dependendnce on &lt;code&gt;TestFunction&lt;/code&gt; and independence on &lt;code&gt;TrialFunction&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2013-05-24T00:31:22.920" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7356" PostTypeId="1" CreationDate="2013-05-24T02:21:08.233" Score="3" ViewCount="163" Body="&lt;p&gt;Eventually, I would like to numerically simulate the transient compressible flow in an axial compressor during start-up.&lt;br&gt;&#xA;However, I know that this is a very challenging undertaking (to say the least).&lt;br&gt;&#xA;Nonetheless I would like to ask if there are recommendations for background literature, simple models, examples, frankly anything that can get me started.&lt;br&gt;&#xA;I recognize that is a potentially board question, but my &quot;google-mojo&quot; has left me on this one and the &lt;a href=&quot;http://en.wikipedia.org/wiki/Transient_flow&quot; rel=&quot;nofollow&quot;&gt;related Wikipedia entry&lt;/a&gt; is very brief.&lt;/p&gt;&#xA;" OwnerUserId="4067" LastEditorUserId="4244" LastEditDate="2013-09-05T16:33:52.017" LastActivityDate="2013-09-05T16:33:52.017" Title="Transient Fluid Dynamics" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7357" PostTypeId="1" CreationDate="2013-05-24T03:52:27.637" Score="3" ViewCount="361" Body="&lt;p&gt;Full configuration interaction quantum Monte Carlo seems like it is poised to overtake DFT in some applications pretty soon.  I am curious if there is any freely available implementation of the method, as I cannot seem to find one.&lt;/p&gt;&#xA;" OwnerUserId="4362" LastActivityDate="2017-03-15T18:23:21.793" Title="What FCIQMC codes are out there?" Tags="|monte-carlo|quantum-mechanics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7358" PostTypeId="2" ParentId="7336" CreationDate="2013-05-24T06:23:06.363" Score="4" Body="&lt;p&gt;You can define the Mini element in this alternative manner if you want a mixed enriched element rather than an enriched mixed element: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;U = FunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;B = FunctionSpace(mesh, &quot;Bubble&quot;, 3)&#xA;M = U + B&#xA;Mini_h = MixedFunctionSpace([M, M])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4383" LastActivityDate="2013-05-24T06:23:06.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7359" PostTypeId="1" CreationDate="2013-05-24T08:24:25.593" Score="4" ViewCount="221" Body="&lt;p&gt;I'm conducting a molecular dynamics simulation for silica. Some time ago I turned to the fluctuating dipole model, and after much effort I'm still having problems implementing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, all oxygen atoms in the system are polarizable, and their dipole moments depend on their position with respect to all the other atoms in the system. More particularly, I use TS potential (&lt;a href=&quot;http://digitallibrary.sissa.it/bitstream/handle/1963/2874/tangney.pdf?sequence=2&quot; rel=&quot;nofollow&quot;&gt;http://digitallibrary.sissa.it/bitstream/handle/1963/2874/tangney.pdf?sequence=2&lt;/a&gt;), where dipoles are found iteratively at each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that when evaluating forces acting on atoms, I have to take into account this potential energy dependency on coordinates. Before, I was using simple pairwise potential models, and so I would set my program to compute forces using analytic formulas obtained by differentiating potential energy expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm at a loss: how to implement this new potential? In all the articles that I've found they only give you formulas, but not the algorithm. As I see it, when I compute forces, acting on a certain atom, I have to take into account the change of the dipole of this atom, the change of dipoles of all the neighboring atoms, then tge change of dipoles of still more atoms, and so on, as they depend on each other. After all, it is because of this interdependency that the dipoles are found iteratively at each time step. Clearly, I can't compute forces iteratively for each atom, because the computational complexity of the algorithm would be way too high. Should I use some simple functions to account for the change of dipoles? This doesn't look like a good idea either, cause dipoles are calculated iteratively, with high precision, and then, where it actually matters (computing forces), we would use crude functions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how do I implement this model? Also, is it possible to compute forces analytically, as I did before, or is it necessary to compute them using finite difference formula for derivative?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't found the answer for my question in literature, but if you know some article, or site, or book, where this material is highlighted, please, direct me to that source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your answer. &#xA;Unfortunately, this was not my question. I didn't ask how to compute dipoles, but how to compute forces given that those dipoles vary with movement considerably. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The straightforward way to compute them doesn't account for that dependence. In fact, I tried to compute forces in straightforward manner, but the results I got were not physically correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To analyze the situation, I set up a simulation of a system consisting of just two atoms: Si and O. They have opposite charges, and so they oscillate. And the energy time dependence graphic looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://yadi.sk/d/9IfGEouw58I4c&quot; rel=&quot;nofollow&quot;&gt;http://yadi.sk/d/9IfGEouw58I4c&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(sorry for the link, it appears that I don't have enough points to post pictures yet)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The curve on the top represents kinetic energy, the one in the middle represents potential energy without taking dipole interaction into account, and the one at the bottom represents potential energy of the system, where dipole interaction was taken into account. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can clearly see from this graphic, that the system is doing what it shouldn't do: climbing up the potential slope. So I decided this is due to the fact that I didn't take dipole moment coordinate dependence into account. For instance, at a given time point, we compute the forces, and they are directed so as to move both atoms toward each other. But when we do move them towards each other (even slightly), the dipole moment changes, and we find out that we actually ended up with higher potential energy than before! During the next time step the situation is the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is, how to take this effect into account, cause what few ways I can think of are either way too computationally intense, or way too crude. &lt;/p&gt;&#xA;" OwnerUserId="4386" LastEditorUserId="4386" LastEditDate="2013-05-26T17:58:28.600" LastActivityDate="2013-05-26T17:58:28.600" Title="Molecular dynamics simulation: fluctuating dipole model implementation" Tags="|algorithms|molecular-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7360" PostTypeId="1" CreationDate="2013-05-24T08:31:52.540" Score="6" ViewCount="1623" Body="&lt;p&gt;I am learning OpenFOAM as a hobby and using my Vespa racing as the topic to apply it to. The objective is to produce modifications that improve the top speed (as well as getting some values such as lift). I am basing my simulation on the motorbike tutorial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The motorbike tutorial uses the kOmegaSST turbulence model but someone had suggested to me in the past that SpalartAllmaras might be a better choice. I have access to a super computer so saving CPU cycles is not much of an issue for me. Below is a picture of my model (which I am cleaning up). Is kOmegaSST a good enough choice or is there another worth focusing on?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;http://www.openfoam.com/features/RAS.php&quot; rel=&quot;noreferrer&quot;&gt;http://www.openfoam.com/features/RAS.php&lt;/a&gt;)!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/qUrsN.png&quot; alt=&quot;Vespa 3D Scan&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2013-05-24T08:31:52.540" Title="What is the best OpenFOAM RAS turbulence model for a motorbike/vespa problem?" Tags="|fluid-dynamics|openfoam|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7362" PostTypeId="1" AcceptedAnswerId="7363" CreationDate="2013-05-24T10:23:00.920" Score="7" ViewCount="3145" Body="&lt;p&gt;I have a numerical ODE simulation that I computed at fixed time step $h$ using a 4-th order Runge-Kutta method (&lt;code&gt;RK4&lt;/code&gt;), producing a series of results $(x_1,y_1), (x_2,y_2), (x_3,y_3) \dots (x_N,y_N)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to find an approximate solution $y$ at a location $x$ in between my intervals, I could use &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;linear interpolation (just kidding, I wouldn't use this) &lt;/li&gt;&#xA;&lt;li&gt;cubic splines (current solution, whats my error estimate ?)&lt;/li&gt;&#xA;&lt;li&gt;a new &lt;code&gt;RK4&lt;/code&gt; step with $h=x-x_i$ (is error still $O(h^4)$ ?)&lt;/li&gt;&#xA;&lt;li&gt;some other appropriate method&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What are the recommended methods for interpolating Runge-Kutta results and what is their error order?&lt;/p&gt;&#xA;" OwnerUserId="1030" LastEditorUserId="127" LastEditDate="2013-05-24T12:48:50.253" LastActivityDate="2013-05-24T12:48:50.253" Title="Intermediate values (interpolation) after Runge-Kutta calculation" Tags="|ode|interpolation|runge-kutta|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7363" PostTypeId="2" ParentId="7362" CreationDate="2013-05-24T10:38:29.700" Score="9" Body="&lt;p&gt;You're asking how to produce &lt;em&gt;dense output&lt;/em&gt; from your Runge-Kutta method. There are a number of ways to do this (see e.g. &lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;id=F93u7VcSRyYC&amp;amp;pg=PA188&quot;&gt;Hairer/Nørsett/Wanner&lt;/a&gt;). As noted in that reference, if you don't want to do more function evaluations aside from those already done by your fourth-order Runge-Kutta method, the best you can hope for is a third-order interpolant. This is fine, since it can also be shown that for a $p$-th order Runge-Kutta method, you can get by with dense output of order $p-1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest third-order dense output you can construct is of course the cubic Hermite interpolant. Recall that given two function values and two derivative values, you can always build a unique cubic: the Hermite interpolant. Thus, you are guaranteed a $C^1$ interpolating function.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="127" LastEditDate="2013-05-24T10:44:46.917" LastActivityDate="2013-05-24T10:44:46.917" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7364" PostTypeId="2" ParentId="7362" CreationDate="2013-05-24T10:43:27.827" Score="7" Body="&lt;p&gt;The RK4 method implicitly constructs a degree 3 polynomial interpolant, using the data $f(x_i)$, $f(x_{i+1})$, $f'(x_i)$, and $f'(x_{i+1})$ in each interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This interpolant can be constructed rather easily and efficiently using a linear combination of shifted &lt;a href=&quot;http://en.wikipedia.org/wiki/Cubic_Hermite_spline&quot;&gt;Hermite basis functions&lt;/a&gt; in each interval.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-05-24T10:43:27.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7366" PostTypeId="1" AcceptedAnswerId="7700" CreationDate="2013-05-24T13:01:58.780" Score="12" ViewCount="3939" Body="&lt;p&gt;&lt;strong&gt;Hey there&lt;/strong&gt; and thanks for giving time to look at my question. This is a updated version of my question which I posted earlier in physics.stackexchange.com &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently studying a 2D exciton spinor Bose-Einstein Condensate and am curious about the ground state of this system. The mathematical method of getting to the ground state is called &lt;strong&gt;the imaginary time method&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method is very simple where time in quantum mechanics is replaced by imaginary one&#xA;$$ t = -i \tau $$&#xA;This substitution causes the high energy particles in my system to decay faster than the low energy ones. Re-normalizing the number of particles in every step of the calculation we end up with a system of lowest energy particles, aka. the ground state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation(s) in question is nonlinear, called the &lt;strong&gt;nonlinear Schrödinger equation&lt;/strong&gt;, sometimes the &lt;strong&gt;Gross-Pitaevskii equation&lt;/strong&gt;. To solve the problem I'm using Matlabs ode45 which evolves the system forward in time and eventually reaches the ground state.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Note!&lt;/strong&gt; The nonlinear Schrödinger equation includes the laplacian and some other differential terms in space. These are all solved using fast Fourier transform. In the end we have only a time ODE. *&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My problem and question:&lt;/strong&gt; The calculations go from $t_0$ to $t_f$. The ode45 is put in a &lt;em&gt;for&lt;/em&gt; loop so it doesn't calculate a giant vector $[t_0,\dots,t_f]$ at the same time. The first round would start with  ode45(odefun,$[t_0, t_0+\Delta/2, t_0 + \Delta],y,\dots$ ) and then go on next time from $t_0 + \Delta$. Here the time step $\Delta$ is my problem.  Different choices in time steps gives me different ground state solutions and I have no idea how to determine which time step gives me the &quot;most&quot; correct ground state! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempt:&lt;/strong&gt; I realize that in this scheme large time steps will cause large number of particles to decay before being re-normalized to the original number of particles whereas small time steps will cause smaller amount of particles to decay before getting re-normalized. My initial thought is that small time steps should give a more accurate solution but it seems to be the opposite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not a numerical expert so the choice of ode45 was simply arbitrary. ode113 gives me the same thing. :(&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any thoughts on this matter. Let me know if any additional detail is needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 1:&lt;/strong&gt;&#xA;I've been researching the imaginary time method and the ODEs. It would seem that if the time step isn't small enough the whole thing becomes unstable. This makes me wonder if my &lt;a href=&quot;https://i.stack.imgur.com/UQupG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;nonlinear equations&lt;/a&gt; are stiff which makes things a lot more difficult from what I understand. I'll keep you updated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 2:&lt;/strong&gt;&#xA;FIXED: The problem was indeed having the normalization outside of the ODE. If the normalization is kept inside odefun then the ODE returns the same outcome for different choices of &quot;outside&quot; time steps. My colleague showed me older codes and I simply added one line in my odefun.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function y_out = odefun(t,y_in,...variables...) &#xA;&#xA;    ...&#xA;    [ Nonlinear equations evaluated ]  &#xA;    ...&#xA;&#xA;&#xA;    y_out = y_out + 0.1*y_in*(N0-Ntemp) ;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last line calculates the difference in current number of particles (Ntemp) and the number of particles which the system should hold (N0). It adds a portion of the particles back to the output and thus creates a total particle number stability in the system instead of having them all decay away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will pose also a new question regarding the dimensionality of the problem and some differences in working with either picoseconds or nanoseconds as time steps in the ODE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you all. :)&lt;/p&gt;&#xA;" OwnerDisplayName="user4388" LastEditorDisplayName="user4388" LastEditDate="2013-06-18T18:11:37.750" LastActivityDate="2013-06-18T18:14:41.637" Title="Choice of step size using ODEs in matlab" Tags="|matlab|ode|quantum-mechanics|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7367" PostTypeId="2" ParentId="7359" CreationDate="2013-05-24T13:09:57.843" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://e-collection.library.ethz.ch/view/eth:29128&quot; rel=&quot;nofollow&quot;&gt;This link&lt;/a&gt; points to a thesis that contains an otherwise unpublished chapter (chapter 6) on the simulation of a droplet of water on graphite using a force field that includes a linear response model for the dipole moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of the approach is based on the following publication:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Abdolnour Toukmaji, Celeste Sagui, John Board, and Tom Darden.&#xA;  Efficient particle-mesh Ewald based approach to fixed and induced&#xA;  dipolar interactions. J. Chem. Phys., 113(24):10913–10927, 2000.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The approach is very crude / brute force:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Perform one step of MD  &lt;/li&gt;&#xA;&lt;li&gt;Calculate electrical field ($E_0$) using charges and old dipoles ($d_0$)  &lt;/li&gt;&#xA;&lt;li&gt;From electrical field ($E_0$) calculate new dipoles ($d_1$)  &lt;/li&gt;&#xA;&lt;li&gt;Check on convergence by comparing change in dipole moments ($d_1$ vs $d_0$)  &lt;/li&gt;&#xA;&lt;li&gt;Calculate electrical field ($E_1$) using charges and new dipoles ($d_1$)  &lt;/li&gt;&#xA;&lt;li&gt;From electrical field ($E_1$) calculate new dipoles ($d_2$)  &lt;/li&gt;&#xA;&lt;li&gt;Check on convergence ($d_2$ vs. $d_1$)  &lt;/li&gt;&#xA;&lt;li&gt;Repeat until convergence is reached&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Convergence was not great and if I recall correctly it took about 5-10 iterations until the dipole moments had converged ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have the electrical field, the forces are straight forward..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps, &#xA;best regards&#xA;urs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed I did not understand your question correctly. Can you give me some more information:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Can you provide me with information about the distance between the two ions at $t=0$, $t=330$ and $t=660$ (at the start and at the first two minima of kinetic energy)? The following comments are based on the assumption that the distance at $t=0$ is larger than at $t=330$, i.e. the ions start some distance apart and first move closer together. Can you confirm this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You state the bottom line is &lt;em&gt;potential energy with dipole interaction&lt;/em&gt;. However, you make no reference to &lt;em&gt;dipole-charge interaction&lt;/em&gt;? Did you include it in your potential energy calculation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You do not make any reference to &lt;em&gt;polarisation energy&lt;/em&gt;? Did you include that in your potential energy computation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4389" LastEditorUserId="4389" LastEditDate="2013-05-24T19:49:59.800" LastActivityDate="2013-05-24T19:49:59.800" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7368" PostTypeId="1" CreationDate="2013-05-23T14:16:44.867" Score="-4" ViewCount="71" Body="&lt;p&gt;I want to implement differential equations in C# from Open Cell cellml file. The mathematical model is described in &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1367306/&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt;. I have parsed xml, and calculate operations, but can't correctly implement the equations. I can not understand how to evaluate differential equations. The format of operations is content mathml. Here is snippet:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt; &amp;lt;apply&amp;gt;&#xA;    &amp;lt;eq/&amp;gt;&#xA;    &amp;lt;apply&amp;gt;&#xA;       &amp;lt;diff/&amp;gt;&#xA;       &amp;lt;bvar&amp;gt;&#xA;          &amp;lt;ci&amp;gt;time&amp;lt;/ci&amp;gt;&#xA;       &amp;lt;/bvar&amp;gt;&#xA;       &amp;lt;ci&amp;gt;Casp8&amp;lt;/ci&amp;gt;&#xA;    &amp;lt;/apply&amp;gt;&#xA;    &amp;lt;apply&amp;gt;&#xA;       &amp;lt;plus/&amp;gt;&#xA;       &amp;lt;apply&amp;gt;&#xA;          &amp;lt;minus/&amp;gt;&#xA;          &amp;lt;ci&amp;gt;J_0&amp;lt;/ci&amp;gt;&#xA;       &amp;lt;/apply&amp;gt;&#xA;       &amp;lt;ci&amp;gt;J_f0&amp;lt;/ci&amp;gt;&#xA;       &amp;lt;ci&amp;gt;J_Casp8&amp;lt;/ci&amp;gt;&#xA;    &amp;lt;/apply&amp;gt;&#xA; &amp;lt;/apply&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerDisplayName="Galym" LastEditorUserId="4397" LastEditDate="2013-05-24T21:11:29.413" LastActivityDate="2013-05-26T00:22:39.690" Title="How to evaluate 1st and 2nd order reactions, with rate constants and fluxes?" Tags="|simulation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7369" PostTypeId="1" AcceptedAnswerId="7466" CreationDate="2013-05-24T14:28:50.283" Score="16" ViewCount="16409" Body="&lt;p&gt;I'm trying to figure out if there is a faster way to compute all the eigenvalues and eigenvectors of a very big and sparse adjacency matrix than using scipy.sparse.linalg.eigsh As far as I know, this methods only uses the sparseness and symmetry attributes of the matrix. An adjacency matrix is also binary, what makes me think there is a faster way to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created a random 1000x1000 sparse adjacency matrix, and compared between several methods on my x230 ubuntu 13.04 laptop:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;scipy.sparse.linalg.eigs: 0.65 seconds&lt;/li&gt;&#xA;&lt;li&gt;scipy.sparse.linalg.eigsh: 0.44 seconds&lt;/li&gt;&#xA;&lt;li&gt;scipy.linalg.eig: 6.09 seconds&lt;/li&gt;&#xA;&lt;li&gt;scipy.linalg.eigh: 1.60 seconds&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With the sparse eigs and eigsh, I set k, the number of the desired eigenvalues and eigenvectors, to be the rank of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem starts with bigger matrices - on a 9000x9000 matrix, it took scipy.sparse.linalg.eigsh 45 minutes!&lt;/p&gt;&#xA;" OwnerUserId="4391" LastActivityDate="2023-09-22T18:33:32.390" Title="What is the fastest way to compute all eigenvalues of a very big and sparse adjacency matrix in python?" Tags="|linear-algebra|python|performance|eigensystem|scipy|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7370" PostTypeId="1" AcceptedAnswerId="7371" CreationDate="2013-05-24T16:00:44.757" Score="7" ViewCount="770" Body="&lt;p&gt;Let's say you have a general matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, with diagonal entries &lt;span class=&quot;math-container&quot;&gt;$a_{ii} = d&amp;gt;0$&lt;/span&gt;. (No assumptions are made about the off-diagonal elements.) Then Jacobi preconditioning doesn't improve condition number, because&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\kappa(A)= \left|\frac{\lambda_{\rm{max}}(A)}{\lambda_{\rm{min}}(A)}  \right|&#xA;$$&lt;/span&gt;&#xA;becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\kappa(M^{-1}A)= \left|\frac{\lambda_{\rm{max}}(A)/d}{\lambda_{\rm{min}}(A)/d}  \right| = \kappa(A).&#xA;$$&lt;/span&gt;&#xA;In this case, could the use of a Jacobi preconditioner do &lt;em&gt;anything&lt;/em&gt; useful? &lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="20688" LastEditDate="2020-03-29T07:46:59.233" LastActivityDate="2020-03-29T07:46:59.233" Title="Jacobi preconditioner not reducing condition number?" Tags="|linear-algebra|preconditioning|condition-number|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7371" PostTypeId="2" ParentId="7370" CreationDate="2013-05-24T16:52:07.507" Score="7" Body="&lt;p&gt;No, Jacobi only ever corrects relative scales. It does nothing for &quot;smooth&quot; ill-conditioning, such as the $\kappa(A) \in O(h^{-2})$ asymptotics for second order elliptic problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using a Krylov method, the global scale is automatically corrected, but with a stationary iteration, the (constant) scaling is needed somehow (could just be in the damping parameter for the stationary method).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-05-24T16:52:07.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7372" PostTypeId="1" CreationDate="2013-05-24T17:02:49.777" Score="1" ViewCount="58" Body="&lt;p&gt;I have such a problem - I would need to expand a discrete function in a sine fourier series but I would like to use exponential based library for FFT (I will use CUDA to compute it).&#xA;What have I to do with the signal I'm sending to the procedure so this is treated as sine fourier? I think I should add zeros but I am not sure where they have to be.&#xA;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="4007" LastActivityDate="2013-05-24T17:02:49.777" Title="Sine series using exponential based FFT" Tags="|fourier-analysis|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7373" PostTypeId="2" ParentId="37" CreationDate="2013-05-24T17:07:36.323" Score="2" Body="&lt;p&gt;Always implement at least two distinct orders. On a representative problem, solve once using each order. Compare the two on a grid fine enough to be converged at the lower order. Ensure your two answers are reasonably close which gives some indication that the numerical behavior of the lower order scheme hasn't overwhelmingly damaged the solution. If it has, toss the lower order scheme and start over.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you didn't have to start over, coarsen the grid for the higher order as much as possible while still maintaining a reasonably accurate solution as measured by the specific quantity of interest you want. Compare the computational cost for the lower order on the finer grid to that of the higher order on the coarser grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choose whichever is more operationally advantageous. Document the process for naysayers and so that you can repeat it when the representative problem or quantity of interest changes.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2013-05-24T17:07:36.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7374" PostTypeId="2" ParentId="5018" CreationDate="2013-05-24T17:27:09.177" Score="0" Body="&lt;p&gt;You can find a very readable introduction to modern ODE solvers in chapter 7 of Cleve Moler's book, Numerical Computing With MATLAB, available online here:&#xA;&lt;a href=&quot;http://www.mathworks.com/moler/odes.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/moler/odes.pdf&lt;/a&gt;&#xA;Among the topics he discusses are stability, how to obtain a prescribed accuracy with variable time step algorithms, and application to the two-body problem.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-05-24T17:27:09.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7375" PostTypeId="1" AcceptedAnswerId="32653" CreationDate="2013-05-24T17:40:28.653" Score="2" ViewCount="170" Body="&lt;p&gt;How can I convert matrices &lt;code&gt;L&lt;/code&gt; and &lt;code&gt;U&lt;/code&gt; output by &lt;code&gt;dgssvx()&lt;/code&gt; of SuperLU to triples format (to matrix market format)? Also how can I convert input matrix &lt;code&gt;A&lt;/code&gt; in triples format (in matrix market format) to the format required by SuperLU? What is the easiest way to perform these format conversions?&lt;/p&gt;&#xA;" OwnerUserId="3978" LastEditorUserId="20688" LastEditDate="2019-05-14T21:18:36.557" LastActivityDate="2019-05-14T21:18:36.557" Title="Converting matrices L and U output by dgssv() of SuperLU to triples format" Tags="|matrix|scipy|matrix-factorization|io|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7376" PostTypeId="1" AcceptedAnswerId="7656" CreationDate="2013-05-24T19:17:54.067" Score="10" ViewCount="1388" Body="&lt;p&gt;I am new to scientific computing. I am looking for a Fortran ( preferably f90) implementation of an Octree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem requires an Octree which divides my domain until there aren't more than some N particles(or sources where I know the density value which can be plugged in an integral equation method to solve an equation like the Laplace equation) in any box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All I could find was &lt;a href=&quot;http://nomis80.org/code/octree.html&quot; rel=&quot;nofollow&quot;&gt;this C++ implementation&lt;/a&gt; . I want to know if there are already existing Fortran libraries that I can use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, good paper recommendations on Octree implementations in Fortran that can be used so that applying fast integral equation methods on the computational box becomes easy, will be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="3558" LastEditorUserId="3558" LastEditDate="2014-01-28T05:07:31.507" LastActivityDate="2014-01-28T05:07:31.507" Title="An Octree Code in Fortran" Tags="|fortran|integral-equations|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7378" PostTypeId="1" CreationDate="2013-05-24T20:30:52.663" Score="7" ViewCount="6455" Body="&lt;p&gt;I would like to numerically find a mutual capacitance of two stripes of metal on the opposites sides of a cylinder. The problem is obviously a 2D Laplace equation. I would like to find the potential outside the cylinder as well. Therefore I have something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;mesh = UnitCircleMesh(50)&#xA;V = FunctionSpace(mesh, 'Lagrange', 1)&#xA;&#xA;u_L = Constant(-1)&#xA;def left_boundary(x, on_boundary):&#xA;    r = math.sqrt(x[0] * x[0] + x[1] * x[1])&#xA;    return near(r, 0.7) and x[0] &amp;lt; 0 and between(x[1], (-0.5, 0.5))&#xA;&#xA;u_R = Constant(1)&#xA;def right_boundary(x, on_boundary):&#xA;    r = math.sqrt(x[0] * x[0] + x[1] * x[1])&#xA;    return near(r, 0.7) and x[0] &amp;gt; 0 and between(x[1], (-0.5, 0.5))&#xA;&#xA;leftPlate = DirichletBC(V, u_L, left_boundary)&#xA;rightPlate = DirichletBC(V, u_R, right_boundary)&#xA;bcs = [leftPlate, rightPlate]&#xA;&#xA;u = TrialFunction(V)&#xA;v = TestFunction(V)&#xA;a = dot(grad(u), grad(v)) * dx&#xA;&#xA;u = Function(V)&#xA;solve(a == Constant(0) * v * dx, u, bcs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run it, I receive &lt;code&gt;*** Warning: Found no facets matching domain for boundary condition.&lt;/code&gt;. And the found solution is 0. What is wrong?&lt;/p&gt;&#xA;" OwnerUserId="4379" LastActivityDate="2021-03-03T14:40:42.557" Title="FEniCS: how to specify boundary conditions on a circle inside 2D mesh" Tags="|python|fenics|boundary-conditions|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7379" PostTypeId="2" ParentId="7376" CreationDate="2013-05-24T21:36:32.603" Score="6" Body="&lt;p&gt;You seem to be very set on using Fortran. Octrees, when implemented efficiently, are rather complex data structures and, as such, better suited to programming languages that have more support for this, such as C/C++. There are a number of very high quality implementations in C/C++ that you could use.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-24T21:36:32.603" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7380" PostTypeId="2" ParentId="7366" CreationDate="2013-05-24T21:48:28.327" Score="4" Body="&lt;p&gt;Since you didn't post your MATLAB code, I'm not sure how you're calling ode45. I'm guessing you are changing the tspan vector (second argument) on each call to ode45. The first thing to understand is that the tspan vector has (almost) no effect on the time step used by ode45. The tspan vector simply allows you to pass to ode45 the time span of the integration and at what times you want output. The time step used by the Runga-Kutta algorithm in ode45 is adjusted internally to achieve a prescribed accuracy. The two parameters that control this accuracy are RelTol and AbsTol in the options structure passed to ode45. They have reasonable defaults and since you didn't mention these, I'm assuming you didn't change them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I said &quot;almost&quot; no effect on the normal ode45 time step. If you are requesting output at time intervals very small relative to the time step ode45 would otherwise take, then it will have to reduce the time step to satisfy your output request. I believe this is what J.M. is assuming is going on. Why do you need the solution at so many output times? Usually it is sufficient to just request output at enough times to generate a smooth plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to the change in solution you are seeing, maybe the default values of RelTol and AbsTol are not appropriate for your problem. I suggest replacing your loop on ode45 with a single call, request output at some reasonable number of times, and experiment with smaller values of RelTol and AbsTol until you get a converged solution.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-05-24T21:48:28.327" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7381" PostTypeId="2" ParentId="7378" CreationDate="2013-05-24T21:56:37.120" Score="5" Body="&lt;p&gt;As the error message suggest the DirichletBC does not hit any mesh entities with corresponding dofs. You need to examine your subdomains. One way to debug these are to apply the DirichletBC to a Function and plot the result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def left_boundary(x, on_boundary):&#xA;    r = math.sqrt(x[0] * x[0] + x[1] * x[1])&#xA;    return r &amp;lt; 0.5&#xA;&#xA;leftPlate = DirichletBC(V, u_L, left_boundary)&#xA;u = Function(V)&#xA;u.vector()[:] = 0&#xA;leftPlate.apply(u.vector())&#xA;plot(u, interactive=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will plot a nice hat. Now you can start tweak your left_boundary function until you reach the result you want.&lt;/p&gt;&#xA;" OwnerUserId="4250" LastEditorUserId="4250" LastEditDate="2013-05-24T22:13:03.507" LastActivityDate="2013-05-24T22:13:03.507" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7383" PostTypeId="2" ParentId="7366" CreationDate="2013-05-24T22:30:10.783" Score="2" Body="&lt;p&gt;Since the nonlinear Schrödinger equation is, well, nonlinear, it can have a large number of stationary states, some of which may be stable. In the physical reality, starting from one particular state, the system will deterministically evolve into one final state. If the numerical scheme gives you different results for different discretizations (time steps), then this is fundamental flaw of your discretization. Check your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you end up with a state $\psi_0$, then it is easy to verify if it is really a stationary state: If the time evolution is given by&#xA;$$&#xA;\frac{\text{d}\psi}{\text{d}t} = F(\psi),&#xA;$$&#xA;then &#xA;$$&#xA;F(\psi_0) = 0.&#xA;$$&#xA;Should you indeed end up with different stationary states, you can compare their Gibbs energies&#xA;$$&#xA;G(\psi) = \int_\Omega E(\psi)&#xA;$$&#xA;where $E(\cdot)$ is the energy density. When $F(\psi)=0$, $E(\psi)$ often looks quite simple, e.g., for Ginzburg-Landau equations, &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/100807132&quot; rel=&quot;nofollow&quot;&gt;$E(\psi) = -|\psi|^4$&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-05-24T22:30:10.783" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7386" PostTypeId="1" CreationDate="2013-05-25T04:19:01.057" Score="14" ViewCount="13569" Body="&lt;p&gt;I have a dataset of 3-dimensional points for which I'd like to construct a mesh, using python. All the software I've seen requires that you provide the edges. Is there a program in python which takes as the input a set of points in 3D and output a mesh? If possible, I'd like the meshing to be uniform.&lt;/p&gt;&#xA;" OwnerUserId="4403" LastActivityDate="2022-04-24T05:09:47.567" Title="Meshing 3D surface data in python" Tags="|python|mesh-generation|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7387" PostTypeId="2" ParentId="7386" CreationDate="2013-05-25T07:19:52.767" Score="0" Body="&lt;p&gt;You could try &lt;a href=&quot;http://www.vtk.org/&quot; rel=&quot;nofollow&quot;&gt;VTK&lt;/a&gt; which has a Python API. I would first try ParaView and bring your points into that and then try some of the filters (e.g. Delaunay). If the filters work in &lt;a href=&quot;http://www.paraview.org/&quot; rel=&quot;nofollow&quot;&gt;ParaView&lt;/a&gt; with your data (which is VTK based) then you can use VTK to do the job. How successful you will be will depend on what the points look like and how nicely they suit the filters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If VTK looks like it will work, have a look at the Kitware tutorials on VTK as I know there is one that walks through using Python/VTK.&lt;/p&gt;&#xA;" OwnerUserId="564" LastEditorUserId="1567" LastEditDate="2013-05-25T11:22:11.320" LastActivityDate="2013-05-25T11:22:11.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7388" PostTypeId="2" ParentId="7386" CreationDate="2013-05-25T08:54:00.353" Score="4" Body="&lt;p&gt;CGAL (&lt;a href=&quot;http://www.cgal.org&quot; rel=&quot;nofollow&quot;&gt;http://www.cgal.org&lt;/a&gt;) has a number of modules for triangulating points in 3D (surface meshes from points, triangulations of points in 3D, etc). Python wrappers for a subset of CGAL modules are available (&lt;a href=&quot;https://code.google.com/p/cgal-bindings&quot; rel=&quot;nofollow&quot;&gt;https://code.google.com/p/cgal-bindings&lt;/a&gt;), including for 3D triangulations. I've used the CGAL C++ interface for triangulating points in 3D, but I have no experience with the Python interface.&lt;/p&gt;&#xA;" OwnerUserId="3952" LastActivityDate="2013-05-25T08:54:00.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7390" PostTypeId="1" AcceptedAnswerId="7393" CreationDate="2013-05-25T16:21:59.290" Score="16" ViewCount="1514" Body="&lt;p&gt;In semiconductor simulation, it is common that the equations are scaled so they have normalised values. For example, in extreme cases electron density in semiconductors can vary over 18 order of magnitude, and electric field can change shapely, over 6 (or more) orders of magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the papers never really give a reason for doing this. Personally I am happy dealing with equations in real units, is there any numerical advantage to do this, is it impossible otherwise? I thought with double precision there would be enough digits to cope with these fluctuations.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Both answers are very useful, thanks very much!&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-05-26T09:30:47.597" LastActivityDate="2013-08-28T00:24:14.950" Title="Is variable scaling essential when solving some PDE problems numerically?" Tags="|pde|condition-number|scaling|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7391" PostTypeId="2" ParentId="7390" CreationDate="2013-05-25T16:58:51.597" Score="2" Body="&lt;p&gt;Dealing with floating point numbers can be trick with regards to subtraction of small numbers from larger numbers, as well as with many other aspects. I would recommend reading John D. Cooks blog posts on them, such as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.johndcook.com/blog/2009/04/06/anatomy-of-a-floating-point-number/&quot; rel=&quot;nofollow&quot;&gt;Anatomy of a Floating Point Number&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;as well as Oracle's &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html&quot; rel=&quot;nofollow&quot;&gt;What Every Computer Scientist Should Know About Floating-Point Arithematic&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also certain numerical algorithms for minimization, or maximization require normalization for numerical stability.&lt;/p&gt;&#xA;" OwnerUserId="1702" LastActivityDate="2013-05-25T16:58:51.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7392" PostTypeId="2" ParentId="7390" CreationDate="2013-05-25T17:12:02.110" Score="9" Body="&lt;p&gt;PDEs of which the solutions have sharp boundaries pose problems that go beyond being able to represent the solution in floating point. This is especially true when solutions have a certain physical meaning, e.g., a density (that per se cannnot be smaller than 0). Consider, for example,&#xA;$$&#xA;-\varepsilon \Delta u + u = 0 \text{ on } \Omega,\\&#xA;u = 1 \text{ on } \partial\Omega.&#xA;$$&#xA;The exact solution is pointwise positive, but when discretizing with Finite Elements, the may no longer be true. A &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7245/oscillations-in-singularly-perturbed-reaction-diffusion-problems-with-finite-ele&quot;&gt;similar question&lt;/a&gt; came up recently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, there is no scaling of variables or domains that removes this difficulty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One sensible reason for scaling out physical quantities is to have the actual equation as simple as possible, i.e., as few parameters as possible involved (that you'd have to play around with). For example, instead of looking for the solution $u_{\alpha}$ of&#xA;$$&#xA;-\alpha^2\Delta u = f_\alpha \text{ on } \alpha\Omega&#xA;$$&#xA;(with some boundary conditions) for many different $\alpha$, you could just go ahead and find the solution $u_1$ of&#xA;$$&#xA;-\Delta u = f \text{ on } \Omega.&#xA;$$&#xA;You deduce immediately that $u_{\alpha}(\mathbf{x}):=u_1(\mathbf{x}/\alpha)$ for any $\alpha$. This shows that scaling out the parameter $\alpha$ doesn't really influence the behavior of the solution at all -- a fact that is quite obvious here, but may be quite difficult to see in other cases. You can construct quite complex examples with more parameters in the same way, e.g., &lt;a href=&quot;https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Navier-Stokes&lt;/a&gt;  and its &lt;a href=&quot;https://en.wikipedia.org/wiki/Nondimensionalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;nondimensionalization&lt;/a&gt; into the &lt;a href=&quot;https://en.wikipedia.org/wiki/Reynolds_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reynolds-number&lt;/a&gt;-formulation.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-08-28T00:24:14.950" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7393" PostTypeId="2" ParentId="7390" CreationDate="2013-05-25T17:24:51.233" Score="9" Body="&lt;p&gt;Solving a (linear) PDE consists in discretizing the equation to yield a linear system, which is then solved by a linear solver whose convergence (rate) depends on the condition number of the matrix. Scaling the variables often reduces this condition number, thus improving convergence. (This basically amounts to applying a diagonal preconditioner, see Nicholas Higham's &lt;a href=&quot;http://books.google.com/books?id=5tv3HdF-0N8C&amp;amp;pg=PA125&quot;&gt;&lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving nonlinear PDEs in addition requires a method for solving nonlinear equations such as Newton's method, where the scaling can also influence convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since normalizing everything usually takes very little effort, it is almost always a good idea.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-05-25T17:24:51.233" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7394" PostTypeId="2" ParentId="7386" CreationDate="2013-05-25T18:32:42.020" Score="7" Body="&lt;p&gt;If an unrestricted triangulation is OK, you can do it with &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.Delaunay.html#scipy.spatial.Delaunay&quot; rel=&quot;noreferrer&quot;&gt;scipy.spatial.Delaunay&lt;/a&gt; which uses Qhull.&lt;/p&gt;&#xA;" OwnerUserId="1645" LastActivityDate="2013-05-25T18:32:42.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7395" PostTypeId="2" ParentId="7240" CreationDate="2013-05-25T19:33:00.203" Score="8" Body="&lt;p&gt;&lt;strong&gt;Matlab 6 used to have a function&lt;/strong&gt; &lt;code&gt;flop&lt;/code&gt; to count them, but &lt;a href=&quot;http://www.mathworks.com/matlabcentral/newsreader/view_thread/133931&quot; rel=&quot;nofollow noreferrer&quot;&gt;it was removed&lt;/a&gt; in later versions. The main reason was technical (they switched to LAPACK as the linear algebra core, and it did not return a flop count). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today this option isn't available anymore; there is no easy equivalent, so if you wish to have one &lt;strong&gt;you'll have to count them yourself&lt;/strong&gt; using the definition and your best guess on how these operations are implemented. It should be easy to come up with big-O estimates; not always so to pin down the exact constant. In the examples you ask for it is easy to guess what Matlab is doing and counting the operations by hand, but it isn't always so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, such as the QR/Francis algorithm for computing eigenvalues (&lt;code&gt;eig&lt;/code&gt;), this &quot;constant&quot; will depend on the specific matrix. &lt;a href=&quot;http://books.google.it/books?id=S6gpNn1JmbgC&amp;amp;lpg=PR2&amp;amp;dq=functions%20of%20matrices&amp;amp;pg=PA335#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Appendix C&lt;/a&gt; of Higham's &lt;em&gt;Function of Matrices&lt;/em&gt; contains good reference values for some dense linear algebra operations. However, &lt;strong&gt;the real answer depends on the details of Matlab and LAPACK implementation&lt;/strong&gt;, which is ultimately unknown since the former is closed source, and it will often change from version to version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, let me point out that &lt;strong&gt;the answer depends on how you define $O$ / $\Theta$ complexity&lt;/strong&gt;. A common choice in most theoretical papers is &quot;arithmetic operations such as sums and products count 1, everything else is free&quot; (sometimes called &lt;em&gt;flop count&lt;/em&gt;). With this definition, however, &lt;code&gt;find&lt;/code&gt; costs zero, since it makes a bunch of comparisons but no true arithmetic operation. To play devil's advocate, you can even replace a floating point operation with a bunch of comparisons and assignments. So there are plenty of reasons to think that it is not a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a less contrived example, consider Gaussian elimination with complete pivoting: it has the same flop-count cost as partial pivoting, but it involves $O(n^3)$ comparisons rather than $O(n^2)$, and these comparisons increase its actual cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, &lt;strong&gt;the execution time on a modern computer does not depend (only) on flops anymore&lt;/strong&gt;. Details such as memory layout, cache misses, branch prediction have a large impact. There is no easy way to tell how long an operation will take, apart from running it and measuring the time. Check out &lt;a href=&quot;https://stackoverflow.com/questions/11227809/why-is-processing-a-sorted-array-faster-than-an-unsorted-array&quot;&gt;this thread&lt;/a&gt; for an illuminating example. There is a whole research area on how to reduce  &quot;hidden costs&quot; such as memory access in linear algebra operations; check for instance the section &quot;communication-avoiding algorithms&quot; on &lt;a href=&quot;http://www.cs.berkeley.edu/~demmel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jim Demmel's home page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The performance of Matlab itself is another can of worms.&lt;/strong&gt; Matlab is an interpreted language, and it does lots of additional work and bookkeeping between the lines. If you use Matlab profiler to check which instructions take the most time, you'll often find that the culprits are surprisingly unusual lines such as memory allocations, comparisons, or calls to functions which do almost nothing. Merely putting a function in a namespace or a class can have a huge impact on its &lt;a href=&quot;https://stackoverflow.com/questions/1693429/is-matlab-oop-slow-or-am-i-doing-something-wrong/1745686#1745686&quot;&gt;execution time&lt;/a&gt;.&#xA;A JIT compiler (Matlab accelerator) optimizes some tight cycles, but since it is itself closed-source and continuously evolving, it's impossible to predict if and when it will kick in. The Mathworks explicitly refuse to provide an interface to tell if a function has been JIT-compiled or not. &lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-05-25T21:48:48.147" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7396" PostTypeId="2" ParentId="7368" CreationDate="2013-05-26T00:22:39.690" Score="2" Body="&lt;p&gt;Ok, this is specifying the equation in a way that is going to be a pain to work work with. It look like it is designed to be read by a program capable of symbolic calculation, implementation of which is a pain. It is specified in functional form designed to be easy to convert into a symbolic equation. To interpret from scratch it you will need to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Create a representation of variables&lt;/li&gt;&#xA;&lt;li&gt;Create representions of + - * / ^ etc&lt;/li&gt;&#xA;&lt;li&gt;Create representions of the differential operator&lt;/li&gt;&#xA;&lt;li&gt;Implement substitution/evaluations&lt;/li&gt;&#xA;&lt;li&gt;Decide how to convert differential equations into a machine integrable form&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A class (or function) structure like this is pretty standard:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+ Expression&#xA;   + Variable(String name)&#xA;   + Operation(Expression ex1, Expression ex2 ...)&#xA;   + Differentiate(Expression ex, Variable v)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, you don't want to do that, unless you have loads of time. Trust me, I have done it myself, its a pain. Best to find a symbolic algebra package. See &lt;a href=&quot;https://stackoverflow.com/questions/6721736/c-sharp-library-for-algebra-simplification-and-solving&quot;&gt;this SE question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Once you've done this you can easily write a function that iterates a differential equation. In sudo code,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# For an equation of the form d/dt X = Y(x,t)&#xA;&#xA;function step(equation, x, t, dt)&#xA;    lhs = equation[0]&#xA;    rhs = equation[1]&#xA;&#xA;    return x + dt * rhs.evaluate(x,t)&#xA;&#xA;function euler_integrate(equation, x0, t0, dt,iters)&#xA;    output = empty_list&#xA;&#xA;    x = x0&#xA;    t = t0&#xA;&#xA;    for i in 1...iters&#xA;    do&#xA;        x = step(equation, x, t, dt)&#xA;        t = t + dt&#xA;&#xA;        output.append(x)&#xA;    done&#xA;&#xA;    return output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which generalises pretty easily.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As for the equation I understand that it says:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial}{\partial t} [\text{Casp8}] = [\text{f0}]_J + [\text{Casp8}]_J - [\text{leak of some kind}]_J$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $[X]$ is a concentration of and $[X]_J$ is a flux (and, therefore, itself a temporal derivative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve the equation how you like, using mass action kinetics, or the Gillespie algorithm, or whatever..&lt;/p&gt;&#xA;" OwnerUserId="4407" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-05-26T00:22:39.690" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7397" PostTypeId="1" AcceptedAnswerId="7400" CreationDate="2013-05-26T05:53:56.590" Score="7" ViewCount="2492" Body="&lt;p&gt;I carefully read all circa 70 pages of FEniCS tutorial and I still do not understand how to solve electrostatic problems when I have materials with different dielectric constant. The self contained system of equations in electrostatic&#xA;$$&#xA;\mathrm{div}\boldsymbol{\mathcal{D}}=4\pi\rho,\\&#xA;\boldsymbol{\mathcal{D}}=\varepsilon\mathbf{E},\\&#xA;\mathbf{E}=-\nabla\varphi.&#xA;$$&#xA;works only in regions where the functions are differentiable and smooth. At the interface between two regions with different materials these equations have to be augmented with the following boundary conditions:&#xA;$$&#xA;\varphi_1=\varphi_2,\\&#xA;\varepsilon_1\frac{\partial\varphi_1}{\partial n}=\varepsilon_2\frac{\partial\varphi_2}{\partial n}.&#xA;$$&#xA;How can I specify the second boundary condition? The example in the tutorial considers only a simple case of&#xA;$$&#xA;\frac{\partial\varphi}{\partial n}=0&#xA;$$ between two regions.&lt;/p&gt;&#xA;" OwnerUserId="4379" LastEditorUserId="4379" LastEditDate="2013-05-26T11:17:15.857" LastActivityDate="2013-05-26T13:37:32.030" Title="FEniCS: boundary conditions for electrostatic problems with dielectrics" Tags="|fenics|boundary-conditions|electromagnetism|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7398" PostTypeId="2" ParentId="7397" CreationDate="2013-05-26T10:49:16.290" Score="5" Body="&lt;p&gt;Since the problem is treated in &lt;code&gt;subdomains-poisson&lt;/code&gt; for the case, where the mesh is aligned with the interface, I assume in the following that in your case the jump can occur anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the first step here is not to think about implementation FEniCS, but rather to find an adequate variational form for this problem, where the interface conditions are properly incorporated. The problem looks like it can be treated with a variant of the following method:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Hansbo, A. and Hansbo, P. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782502005248&quot; rel=&quot;nofollow&quot;&gt;An unfitted finite element method, based on Nitsche’s method, for elliptic interface problems&lt;/a&gt; (Comp. Meth. in Appl. Mech. and Eng., Volume 191, Issues 47–48, 22 November 2002, Pages 5537–5552)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer:&lt;/em&gt; There may be others and maybe also better alternatives for your specific case, but I am mostly familiar with Nitsche-type techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have an idea about how to discretize your problem, you can think about implementation in FEniCS. As far as I know, it is not possible to implement this method straightforwardly in vanilla FEniCS. &lt;a href=&quot;https://launchpad.net/dolfin-olm&quot; rel=&quot;nofollow&quot;&gt;dolfin-olm&lt;/a&gt; may be used to implement this method. I have not used it so far, I don't know if it works with the current version of FEniCS, and I don't know if they merged (parts of) it into dolfin in the meantime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, note that in standard finite element methods the interface condition can be taken into account by aligning your mesh with the interface. If that is an option, you should go for that.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2013-05-26T12:55:48.577" LastActivityDate="2013-05-26T12:55:48.577" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7399" PostTypeId="1" AcceptedAnswerId="7423" CreationDate="2013-05-26T11:06:09.873" Score="8" ViewCount="15642" Body="&lt;p&gt;The advection equation needs to be discretized in order to be used for the Crank-Nicolson method. Can someone show me how to do that?&lt;/p&gt;&#xA;" OwnerUserId="4409" LastEditorUserId="3691" LastEditDate="2013-05-28T13:48:45.783" LastActivityDate="2021-05-28T22:14:42.293" Title="How to discretize the advection equation using the Crank-Nicolson method?" Tags="|pde|advection|crank-nicolson|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7400" PostTypeId="2" ParentId="7397" CreationDate="2013-05-26T12:06:13.943" Score="4" Body="&lt;p&gt;&lt;strong&gt;Vatiational formulation of Poisson problem with $\varepsilon \in L^\infty$&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both interfacial conditions are incorporated in following variational problem.&#xA;Given &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\Omega$ Lipschitz domain&lt;/li&gt;&#xA;&lt;li&gt;$\partial\Omega = \Gamma_\mathrm{D} \cup \Gamma_\mathrm{N}$&lt;/li&gt;&#xA;&lt;li&gt;$V=\{v\in H^1(\Omega); v|_{\Gamma_\mathrm{D}}=0 \} $&lt;/li&gt;&#xA;&lt;li&gt;$\varepsilon\in L^\infty(\Omega)$&lt;/li&gt;&#xA;&lt;li&gt;$g \in L^2(\Gamma_\mathrm{N}) $&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;find $\varphi\in V$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \varepsilon \nabla \varphi \cdot \nabla \psi \;\mathrm{d}x = \int_\Omega \rho \psi \;\mathrm{d}x + \int_{\Gamma_\mathrm{N}} g \psi \;\mathrm{d}S \quad \forall \psi\in V$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $g$ represents Neumann condition on $\varepsilon\frac{\partial \varphi}{\partial \mathbf{n}} = g$ on $\Gamma_N$. Both your interfacial conditions are handled automatically by this formulation. You check it by dividing $\Omega$ into subdomains where $\varepsilon$ takes two different values and testing by something close to characteristic functions of these subdomains. Moreover $\varepsilon$ can be much more arbitrary - essential boundedness suffices. Note that I threw away $4\pi$ which corresponds to SI units. If you would like to derive this formulation from physicist point of view, just take Gateaux derivative of field energy minus charge energy in appropriate (accounting to Dirichlet condition) space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For implementation in FEniCS just pick conforming FE space, i.e. arbitrary order Lagrange/CG elements. Problem is implemented in &lt;code&gt;subdomains-poisson&lt;/code&gt; demo assuming discontinuity in $\epsilon$ matches mesh facets.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discontinuity in $\varepsilon$ non-matching with mesh&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need discontinuity non-matching mesh, you could try &lt;code&gt;DOLFIN-PUM&lt;/code&gt; library which implemets XFEM/PUM method. It is not maintained now and Garth said that there are plans for better implementation directly into FEniCS. Library is able to integrate exactly this type of discontinuity in $\epsilon$. But there is additional problem that XFEM/PUM method introduces dicontinuity also to your solution and you need to enforce somehow continuity. It can be solved with Lagrange multiplier, penalty, etc. but I can't rememeber if I succedeed doing this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also solve problem with non-matching mesh using standard formulation and increase quadrature degree or interpolate $\varepsilon$ to some higher-order space prior to computation. I also use $C^k$ approximation to Heaviside function with $k=0,1,2$ to define material discontinuities. You can implement this using &lt;code&gt;conditional&lt;/code&gt;. None of these approaches can be integrate $\varepsilon$ exactly but it could enhance solution somehow.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-26T13:37:32.030" LastActivityDate="2013-05-26T13:37:32.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7403" PostTypeId="1" AcceptedAnswerId="7404" CreationDate="2013-05-27T01:11:44.710" Score="13" ViewCount="2169" Body="&lt;p&gt;The title is the question.  This technique involves using the &quot;matrix of cofactors&quot;, or &quot;adjugate matrix&quot;, and gives explicit formulae for the components of the inverse of a square matrix.  It is not easy to do by hand for a matrix bigger than, say, $3\times 3$.  For an $n\times n$ matrix, it requires computing the determinant of the matrix itself and computing $n^2$ determinants of $(n-1)\times(n-1)$ matrices.  So I'm guessing it is not that useful for applications.  But I would like confirmation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not asking about the theoretical significance of the technique in proving theorems about matrices.&lt;/p&gt;&#xA;" OwnerUserId="3821" LastEditorUserId="3980" LastEditDate="2013-05-27T11:44:39.467" LastActivityDate="2015-11-06T20:34:53.877" Title="Does the &quot;cofactor technique&quot; for inverting a matrix have any practical significance?" Tags="|linear-algebra|matrix|complexity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7404" PostTypeId="2" ParentId="7403" CreationDate="2013-05-27T02:25:35.027" Score="12" Body="&lt;p&gt;You're right -- it has absolutely no practical relevance for computing. Even if computing the determinant was an $O(n)$ operation, the complexity of the method would be at least $O(n^3)$ and, consequently, of the same complexity as Gaussian elimination. In practice, computing the determinant of a matrix is actually of exponential complexity, making this method completely unusable.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-27T02:25:35.027" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="7405" PostTypeId="2" ParentId="7149" CreationDate="2013-05-27T08:45:31.257" Score="7" Body="&lt;p&gt;The command &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;qalter -q &amp;lt;new queue name&amp;gt; &amp;lt;jobid&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;should work for already submitted jobs. For new jobs use the same syntax with qsub&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;qsub -q &amp;lt;new queue name&amp;gt; &amp;lt;scriptname&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that if your job is already running, the qalter will not make it switch queues. Only if the running job gets interrupted and restarted, it will switch queue. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-05-31T06:11:41.887" LastActivityDate="2013-05-31T06:11:41.887" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7406" PostTypeId="1" AcceptedAnswerId="7444" CreationDate="2013-05-27T09:35:03.167" Score="4" ViewCount="2200" Body="&lt;p&gt;I'm making a model of a square box where water comes in and the water level rises. I want it to be a transient, turbulent, VOF-model. The velocity of water entering changes in time ($-0.2$ to $0.2$ m/s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now I use a steady, laminar inflow of $0.2$ m/s, a grid of $10,000$ cells (the geometry is roughly $100\times 100\times 6$ m) and solution method PISO. &#xA;To run the model I've used time step size $0.1$ s and $3600$ time steps ($10$ minutes); bigger time steps led to divergence. It took $2.5$ hours to calculate this and it wasn't even turbulent yet. In the end I've to model a period of $12$ hours, with this speed of calculation that will take more than $5$ days. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are things I can adapt in Fluent which will (extremely) reduce this calculation time? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="4326" LastEditorUserId="1748" LastEditDate="2013-06-07T21:39:16.113" LastActivityDate="2014-08-22T14:16:50.113" Title="calculation time in Fluent" Tags="|fluid-dynamics|efficiency|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7407" PostTypeId="2" ParentId="7336" CreationDate="2013-05-27T13:13:41.830" Score="0" Body="&lt;p&gt;This problem have be solved,thanks everyone!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;U = FunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;B = FunctionSpace(mesh, &quot;Bubble&quot;, 3)&#xA;Q = FunctionSpace(mesh, &quot;DG&quot;, 1)&#xA;M = (U + B)*(U + B)&#xA;Mixed_h = MixedFunctionSpace([M,Q])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This time we can use Mixed_h.sub(0).sub(0).&lt;/p&gt;&#xA;" OwnerUserId="4369" LastActivityDate="2013-05-27T13:13:41.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7409" PostTypeId="1" AcceptedAnswerId="7419" CreationDate="2013-05-27T16:33:54.393" Score="2" ViewCount="2343" Body="&lt;p&gt;This &lt;a href=&quot;http://fenicsproject.org/pub/workshops/fenics12/slides/Ham.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;presentation by the Imperial College&lt;/strong&gt;&lt;/a&gt; in London has a nice example in it, on page 8, Burgers Equations. The first part of their code reads like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import*&#xA;&#xA;n = 30&#xA;mesh = UnitInterval(n)&#xA;V = FunctionSpace(mesh, &amp;quot;CG&amp;quot;,2)&#xA;ic = project(Expression(&amp;quot;sin(2*pi*x[0])&amp;quot;), V)&#xA;u = Function(ic)&#xA;u_next = Function(V)&#xA;v= TestFunction(V)&#xA;nu = Constant(0.0001)&#xA;timestep = Constant(1.0/n)&#xA;&#xA;F1 = ((u_next - u)/timestep*v)*dx&#xA;F2 = (u_next*grad(u_next)*v)*dx&#xA;F3 = (nu*grad(v)*grad(u_next))*dx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;On my system, this code fails, giving shape mismatch errors for the sums &lt;code&gt;F2&lt;/code&gt; and &lt;code&gt;F3&lt;/code&gt;. I don't understand why.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="20688" LastEditDate="2021-12-21T02:19:55.347" LastActivityDate="2021-12-21T02:19:55.347" Title="Simple FEniCS problem shape mismatch" Tags="|pde|fenics|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7410" PostTypeId="2" ParentId="7409" CreationDate="2013-05-27T17:08:28.937" Score="2" Body="&lt;p&gt;The error is in the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = inner(inner(nabla_grad(u),u),v)*dx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here you take the inner product between the vector nabla_grad(u) and the scalar u. That does not make sense.&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-05-27T17:08:28.937" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7411" PostTypeId="1" CreationDate="2013-05-27T17:29:37.307" Score="1" ViewCount="99" Body="&lt;p&gt;I have an ODE without a boundary condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\small\left(\frac{d}{dr}C(r)\right)^4-2C(r)\left(\frac{d^2}{dr^2}C(r)\right)\left(\frac{d}{dr}C(r)\right)^2+2\left(\frac{d}{dr}C(r)\right)C(r)^2\left(\frac{d^3}{dr^3}C(r)\right)-4\left(\frac{d^2}{dr^2}C(r)\right)^2C(r)^2+12C(r)^2\left(\frac{d^2}{dr^2}C(r)\right)-8C(r)^2=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only know the behavior of the function at infinity. Would you please help me to solve the ODE? I can send it to everybody who like help me.&#xA;Thanks a lot,&#xA;M.KH.&lt;/p&gt;&#xA;" OwnerUserId="4416" LastEditorUserId="127" LastEditDate="2013-05-27T18:01:04.720" LastActivityDate="2013-05-28T03:03:24.463" Title="Solving an ODE without a boundary condition" Tags="|ode|" AnswerCount="1" CommentCount="15" ClosedDate="2013-05-28T15:31:12.470" ContentLicense="CC BY-SA 3.0" />
  <row Id="7412" PostTypeId="2" ParentId="7409" CreationDate="2013-05-27T17:54:55.583" Score="3" Body="&lt;p&gt;If your domain is one dimensional you should use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh = UnitIntervalMesh(10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For partial derivative in x direction use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u.dx(0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4250" LastActivityDate="2013-05-27T17:54:55.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7413" PostTypeId="2" ParentId="943" CreationDate="2013-05-27T23:01:03.397" Score="4" Body="&lt;p&gt;One point not made so far is that the current generation of GPU's don't do as well at double precision floating point computations as with single precision computations.  If your computations have to be done in double precision, then you can expect the run time to increase by a factor of 10 or so over single precision.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-05-27T23:01:03.397" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7414" PostTypeId="1" AcceptedAnswerId="7416" CreationDate="2013-05-28T01:33:50.697" Score="8" ViewCount="4018" Body="&lt;p&gt;I was wondering if there exist any good (accurate/fast/easy-to-use) open-source FEM solvers for 3D time-harmonic Maxwell's equations. I am looking to simulate systems a few wavelengths large in the X/Y dimensions and tens of wavelengths in the Z dimension, so relatively small-size problems. All my dielectrics are homogeneous and perfectly rectangular (boxes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some searching online and found various mature open-source FEM packages, but they do not seem to support Maxwell. Any suggestions would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="20688" LastEditDate="2019-07-01T08:17:06.167" LastActivityDate="2019-07-01T08:17:06.167" Title="Open-source 3D FEM Solver for Electromagnetics (Time-Harmonic Maxwell)" Tags="|finite-element|software|electromagnetism|open-source|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7415" PostTypeId="2" ParentId="7411" CreationDate="2013-05-28T03:03:24.463" Score="5" Body="&lt;p&gt;Since your ODE is already nonlinear, you can transform the coordinate variable, for example using&#xA;$$&#xA;  r = 1/s&#xA;$$&#xA;and defining &#xA;$$&#xA;  C(r) = C(r(s)) = \tilde C(s).&#xA;$$&#xA;Then replace every occurrence of $d/dr$ by $ds/dr d/ds=-1/s^2 d/ds$ to obtain an ODE for $\tilde C$ only in $s$ instead of $r$. Now note that the boundary condition $C(\infty)=1$ corresponds to $\tilde C(0)=1$, so you can now forward integrate $\tilde C$ from zero to whatever value you care about, and then compute $C(r) = \tilde C(1/r)$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-28T03:03:24.463" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7416" PostTypeId="2" ParentId="7414" CreationDate="2013-05-28T03:05:59.973" Score="6" Body="&lt;p&gt;deal.II (see &lt;a href=&quot;http://www.dealii.org/&quot;&gt;http://www.dealii.org/&lt;/a&gt;) does support Nedelec elements and, as a consequence, can solve the problems you're interested in. (Full disclaimer: I'm one of the principal developers of deal.II.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-28T03:05:59.973" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7418" PostTypeId="2" ParentId="7414" CreationDate="2013-05-28T04:08:46.607" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://computation.llnl.gov/casc/hypre/software.html&quot; rel=&quot;nofollow&quot;&gt;Hypre&lt;/a&gt; has several built-in preconditioners for solving the Maxwell equations. There are several packages that interface to it (you can use hypre from PETSc) as a solver for linear algebraic systems, but it also has a structured grid and finite element interface too.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-05-28T04:08:46.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7419" PostTypeId="2" ParentId="7409" CreationDate="2013-05-28T06:50:16.117" Score="4" Body="&lt;p&gt;So the problem is that FEniCS doesn't recognize that in 1D, &lt;code&gt;u&lt;/code&gt; and &lt;code&gt;grad(u)&lt;/code&gt; have the same shape. (You could file an issue about this on the &lt;a href=&quot;https://bitbucket.org/fenics-project/dolfin/issues&quot; rel=&quot;nofollow&quot;&gt;Dolfin issue tracker&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you replace &lt;code&gt;grad&lt;/code&gt; by &lt;code&gt;div&lt;/code&gt;, your code works. (The general form of the inviscid Burgers equation is $\mathop{\mathrm{div}}(u^2) = f$.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-05-28T06:55:46.517" LastActivityDate="2013-05-28T06:55:46.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7420" PostTypeId="1" CreationDate="2013-05-28T09:07:54.837" Score="11" ViewCount="3615" Body="&lt;p&gt;I am trying to solve the 2D Poisson equation using&#xA;the Discontinuous Galerkin method (DG) and the following&#xA;discretization (I have a png file but I am not allowed&#xA;to upload it, sorry):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equation :&#xA;$$\nabla \cdot( \kappa \nabla T) + f = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;New equations :&#xA;$$q = \kappa \nabla T\\\nabla \cdot q = -f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Weak form with numerical fluxes $\hat{T}$ and $\hat{q}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int q \cdot w dV = - \int T \nabla \cdot (\kappa w) dV +  \int \kappa \hat{T} n \cdot w dS\\&#xA;\int q \cdot \nabla v dV =  \int v f dV +  \int \hat{q} \cdot n v dS$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical fluxes (IP method):&#xA;$$\hat{q} = \{\nabla T\} – C_{11} [T]\\&#xA;\hat{T} = \{T\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &#xA;$$&#xA;\{T\} = 0.5 (T^+ + T^-)\\&#xA;[T] = T^+ n^+ + T^- n^-&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a simple fenics python script to solve the&#xA;equation. The solution I get is not good. I would &#xA;really appreciate if somebody familiar with the DG method&#xA;could have have a quick look at the script below and tell &#xA;me what I am doing wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The continuous galerkin formulation that I added in the&#xA;script gives a nice solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot in advance.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;&#xA;method = &quot;DG&quot; # CG / DG&#xA;&#xA;# Create mesh and define function space&#xA;mesh = UnitSquare(32, 32)&#xA;V_q = VectorFunctionSpace(mesh, method, 2)&#xA;V_T = FunctionSpace (mesh, method, 1)&#xA;W = V_q * V_T&#xA;&#xA;# Define test and trial functions&#xA;(q, T) = TrialFunctions(W)&#xA;(w, v) = TestFunctions(W)&#xA;&#xA;# Define mehs quantities: normal component, mesh size&#xA;n = FacetNormal(mesh)&#xA;&#xA;# define right-hand side&#xA;f = Expression(&quot;500.0*exp(-(pow(x[0] - 0.5, 2) + pow(x[1] - 0.5, 2)) / 0.02)&quot;)&#xA;&#xA;# Define parameters&#xA;kappa = 1.0&#xA;&#xA;# Define variational problem&#xA;if method == 'CG':&#xA;  a = dot(q,w)*dx \&#xA;       + T*div(kappa*w)*dx \&#xA;       + div(q)*v*dx&#xA;&#xA;elif method == 'DG':&#xA;  #modele = &quot;IP&quot;&#xA;  C11 = 1.&#xA;&#xA;  a = dot(q,w)*dx + T*div(kappa*w)*dx \&#xA;      - kappa*avg(T)*dot(n('-'),w('-'))*dS \&#xA;                                           \&#xA;      + dot(q,grad(v))*dx \&#xA;      - dot( avg(grad(T)) - C11 * jump(T,n) ,n('-'))*v('-')*dS&#xA;&#xA;L = -v*f*dx&#xA;&#xA;# Compute solution&#xA;qT = Function(W)&#xA;solve(a == L, qT)&#xA;&#xA;# Project solution to piecewise linears&#xA;(q , T) = qT.split()&#xA;&#xA;# Save solution to file&#xA;file = File(&quot;poisson.pvd&quot;)&#xA;file &amp;lt;&amp;lt; T&#xA;&#xA;# Plot solution&#xA;plot(T); plot(q)&#xA;interactive()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4422" LastEditorUserId="4260" LastEditDate="2013-05-28T10:36:57.553" LastActivityDate="2014-02-08T10:02:26.303" Title="Discontinuous Galerkin / Poisson / Fenics" Tags="|pde|finite-element|fenics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7421" PostTypeId="2" ParentId="7420" CreationDate="2013-05-28T10:21:52.480" Score="11" Body="&lt;p&gt;To implement your problem in FEniCS, you have to replace the integrals in terms of boundaries by integrals in terms of edges. This introduces jumps/averages in the test functions, which you entirely miss in your implementation. Hence, the system is not invertible and your solution does not look right. Equation (3.3) in Arnold et. al. 2002 gives you a tool to rewrite the weak form:&#xA;$$&#xA;\sum_{K\in\mathcal{T}_h}\int_{\partial K} q_K \cdot n_K \phi_K\,ds=\int_\Gamma [q] \cdot \{\phi\}\,ds + \int_{\Gamma^0} \{q\} \cdot [\phi]\,ds&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $\Gamma$ is the union of your edges and $\Gamma^0$ the same without boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now your fluxes are single-valued, which means that you can drop the jumps of your fluxes. Hence&#xA;$$&#xA;\sum_{K\in\mathcal{T}_h}\int_{\partial K} \hat{q}\cdot n_K v_K\,ds=\int_{\Gamma^0} \hat{q} \cdot [v]\,ds + \int_{\partial\Omega} \hat{q} \cdot n v\,ds\\&#xA;\sum_{K\in\mathcal{T}_h}\int_{\partial K} w\cdot n_K \kappa\hat{T}\,ds=\int_{\Gamma} [w] \cdot \kappa\hat{T}\,ds&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads us to the following modification of your code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C11 = 1.&#xA;qhat = avg(grad(T)) - C11 * kappa*jump(T,n)&#xA;qhatbnd = grad(T) - C11 * kappa*T*n&#xA;&#xA;a = dot(q,w)*dx + T*div(kappa*w)*dx \&#xA;  - kappa*avg(T)*jump(w,n)*dS \&#xA;  - kappa*T*dot(w,n)*ds \&#xA;  - dot(q,grad(v))*dx \&#xA;  + dot( qhat, jump(v,n))*dS \&#xA;  + dot( qhatbnd, v*n)*ds&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I did not have the time yet to actually try this, so be aware of possible sign-errors etc. But I hope this helps anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References:&lt;/strong&gt;&#xA;D.N. Arnold, F. Brezzi, B. Cockburn, L.D. Marini: &lt;a href=&quot;http://www.ima.umn.edu/~arnold//papers/dgerr.pdf&quot; rel=&quot;nofollow&quot;&gt;Unified Analysis of Discontinuous Galerkin Methods for Elliptic Problems&lt;/a&gt; SIAM J. Num. Anal, 39  (2002), 1749-1779&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2014-02-08T10:02:26.303" LastActivityDate="2014-02-08T10:02:26.303" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7423" PostTypeId="2" ParentId="7399" CreationDate="2013-05-28T12:23:39.467" Score="20" Body="&lt;p&gt;Starting with the advection equation is conservative form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} = -\frac{\partial (\boldsymbol{v} u)}{\partial x} + s(x,t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Crank-Nicolson method consists of a time averaged centered difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u_{j}^{n+1} - u_{j}^{n}}{\Delta t} = -\boldsymbol{v} \left[ \frac{1-\beta}{2\Delta x} \left( u_{j+1}^{n} - u_{j-1}^{n} \right) + \frac{\beta}{2\Delta x} \left( u_{j+1}^{n+1} - u_{j-1}^{n+1} \right) \right] + s(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the notation, &lt;em&gt;subscripts&lt;/em&gt; are for points in space, and the &lt;em&gt;superscripts&lt;/em&gt; are for points in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The points at $n+1$ are in the future: they are unknowns. We now need to rearrange the above equation so that all &lt;em&gt;knowns&lt;/em&gt; are on the r.h.s and &lt;em&gt;unknowns&lt;/em&gt; are on the l.h.s..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Making the substitution,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r = \frac{\boldsymbol{v}}{2}\frac{\Delta t}{\Delta x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\beta r\phi_{j-1}^{n+1} + \phi_{j}^{n+1} + \beta r\phi_{j+1}^{n+1} =  (1-\beta)r\phi_{j-1}^{n} + \phi_{j}^{n} - (1-\beta)r\phi_{j+1}^{n}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the advection equation discretised using the Crank-Nicolson method. You can write it as a matrix equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    \begin{pmatrix}&#xA;      1                 &amp;amp; \beta r        	&amp;amp;     				&amp;amp;       	&amp;amp;          0		\\&#xA;      -\beta r    		&amp;amp; 1         		&amp;amp; \beta r       	&amp;amp;     	    &amp;amp;		\\&#xA;              			&amp;amp;  \ddots   		&amp;amp;   \ddots  		&amp;amp; \ddots	&amp;amp;            		\\&#xA;						&amp;amp;                   &amp;amp; -\beta r  		    &amp;amp; 1         &amp;amp; \beta r 	        \\&#xA;                 0      &amp;amp;                   &amp;amp;      				&amp;amp; -\beta r  	&amp;amp; 1          		\\&#xA;   \end{pmatrix}&#xA;   \begin{pmatrix}&#xA;u_1^{n+1} \\&#xA;u_2^{n+1} \\&#xA;\vdots \\&#xA;u_{J-1}^{n+1} \\&#xA;u_{J}^{n+1} \\&#xA;   \end{pmatrix} = &#xA;    \begin{pmatrix}&#xA;      1                    &amp;amp; -(1 - \beta)r        		&amp;amp;     						&amp;amp;       	          &amp;amp;   0		 	        \\&#xA;      (1 - \beta)r        &amp;amp; 1 			                &amp;amp; -(1 - \beta)r 		        &amp;amp;		     	      &amp;amp;                     \\&#xA;              			   &amp;amp;  \ddots   					&amp;amp;   \ddots  				&amp;amp; \ddots			  &amp;amp;                     \\&#xA;						   &amp;amp;                            &amp;amp; (1 - \beta)r  		    &amp;amp; 1	                  &amp;amp; -(1 - \beta)r	\\&#xA;                 0         &amp;amp;                            &amp;amp;      						&amp;amp;(1 - \beta)r        &amp;amp; 1	             \\&#xA;   \end{pmatrix}   \begin{pmatrix}&#xA;u_1^{n} \\&#xA;u_2^{n} \\&#xA;\vdots \\&#xA;u_{J-1}^{n} \\&#xA;u_{J}^{n} \\&#xA;   \end{pmatrix} &#xA;$$&#xA;Setting $\beta=1/2$ will give you trapezoidal integration in time, so for Crank-Nicolson this is what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few words of warning. This is basic solution you wanted, but you will need to include some sort of &lt;strong&gt;boundary condition&lt;/strong&gt; for a well-posed problem. Also,&#xA;Crank-Nicolson is not necessarily the best method for the advection equation. It is second order accurate and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5402/is-crank-nicolson-a-stable-discretization-scheme-for-reaction-diffusion-advectio&quot;&gt;unconditionally stable&lt;/a&gt;, which is fantastic. However it will generate (as with all centered difference stencils) spurious oscillation if you have very sharp peaked solutions or initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote the following code for you in Python, it should get you started. The code solves the advection equation for an initial Gaussian curve moving to the right with constant velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zFaVE.gif&quot; alt=&quot;Gaussian curve moving to the right with constant velocity&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;from scipy.sparse import spdiags&#xA;from scipy.sparse.linalg import spsolve&#xA;import numpy as np&#xA;import pylab&#xA;&#xA;def make_advection_matrices(z, r):&#xA;    &quot;&quot;&quot;Return matrices A and M for advection equations&quot;&quot;&quot;&#xA;    ones = np.ones(len(z))&#xA;    A = spdiags( [-beta*r, ones, beta*r], (-1,0,1), len(z), len(z) )&#xA;    M = spdiags( [(1-beta) * r, ones, -(1-beta) * r], (-1,0,1), len(z), len(z) )&#xA;    return A.tocsr(), M.tocsr()&#xA;&#xA;def plot_iteration(z, u, iteration):&#xA;    &quot;&quot;&quot;Plot the solver progress&quot;&quot;&quot;&#xA;    pylab.plot(z, u, label=&quot;Iteration %d&quot; % iteration)&#xA;&#xA;# Set up basic constants&#xA;beta = 0.5&#xA;J = 200 # total number of mesh points&#xA;z = np.linspace(-10,10,J) # vertices&#xA;dz = abs(z[1]-z[0]) # space step&#xA;dt = 0.2    # time step&#xA;v = 2 * np.ones(len(z)) # velocity field (constant)&#xA;r = v / 2 * dt / dz&#xA;&#xA;# Initial conditions (peak function)&#xA;gaussian = lambda z, height, position, hwhm: height * np.exp(-np.log(2) * ((z - position)/hwhm)**2)&#xA;u_init = gaussian(z, 1, -3, 2)&#xA;&#xA;A, M = make_advection_matrices(z, r)&#xA;u = u_init&#xA;for i in range(10):&#xA;    u = spsolve(A, M * u)&#xA;    plot_iteration(z, u, i)&#xA;&#xA;pylab.legend()&#xA;pylab.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-05-29T22:49:26.843" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7424" PostTypeId="1" CreationDate="2013-05-28T13:51:56.480" Score="7" ViewCount="417" Body="&lt;p&gt;I've seen that there is a way to use the finite differences method, on a cartesian orthogonal grid, to perform calculations on potential flow about an obstacle without using the Neumann conditions, but only Dirichlet conditions. It is done using the stream function, i tested it and it seem to work. But this method has a big drawback: it can be used only with obstacles that are symmetric along an horizontal axis (the direction of the undisturbed stream). The method is based on the fact that there is a horizontal streamline that ideally follows all the boundary of the obstacle (so you can put a constant value of the stream function on it). Do you now any workaround or method to use only Dirichlet conditions, but on an arbitrary shaped obstacle (a wing, for example) ? Thank you in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before, someone advised me to use panel methods, coordinate transformations, etc. It would be good, but i'm looking for something simpler, if possible. It's just for demonstration of the finite differences method, it doesn't have to be particularly efficient of fast. Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As GradGuy indicated, the stream function is constant on the boundary also when the obstacle is asymmetric; but I didn't say it was not.  The method I used was this: I took a rectangular box, then I put two arbitrary values of the stream functions (but the same in all the side) in the upper and lower side; then I linearly varied it along the lateral sides (so there is continuity with the potential on the upper and lower sides). In this way I define the undisturbed flow far from the obstacle. Then I placed the symmetric obstacle right in the middle of the box, so I knew for sure there was horizontal streamline that divided the box in two and followed the obstacle boundary, so its potential was exactly the average value between the upper and lower side. I don't' know how to do same thing with an asymmetric obstacle... Any help would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hello, I tried to post a new question in which I posted the same requests as this question, but in a more clear way. It seems that question has been closed because it's a duplicate of this one. They complained also because I opened many accounts, I'm sorry for that but there was a problem with my browser that didn't allow me to do a password protected account (it crashed on the account creation page), so I can only open guest accounts.&#xA;They advised me to edit that question instead. &#xA;So, I appreciate your efforts but the answers I received till now didn't solve my problem. This question is to be considered still open... It may also be that what I'm asking is impossible...&lt;/p&gt;&#xA;" OwnerUserId="4425" LastEditorUserId="-1" LastEditDate="2013-06-02T13:31:18.133" LastActivityDate="2018-01-28T19:28:38.630" Title="Potential flow around a non-symmetric obstacle using stream functions" Tags="|fluid-dynamics|boundary-conditions|laminar-flow|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7427" PostTypeId="2" ParentId="7420" CreationDate="2013-05-28T16:05:47.237" Score="-2" Body="&lt;p&gt;Yes I was really missing something! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is working fine now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very very much for your help! &lt;/p&gt;&#xA;" OwnerUserId="4422" LastActivityDate="2013-05-28T16:05:47.237" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7428" PostTypeId="1" CreationDate="2013-05-28T16:34:18.803" Score="6" ViewCount="361" Body="&lt;p&gt;I have this funny equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial^2 u}{\partial t^2} = \frac{\partial^3 u}{\partial x^3}, \qquad x \in [0,1], \qquad t \in (0,T]&#xA;$$&#xA;with initial conditions $u(x,0) = \sin(2\pi x)$, $\frac{\partial u}{\partial t}(x,0) = 0$ and periodic boundary conditions: $u(0,t) = u(1,t)$, $\frac{\partial u}{\partial x}(0,t) = \frac{\partial u}{\partial x}(1,t)$ and $\frac{\partial^2 u}{\partial x^2}(0,t) = \frac{\partial^2 u}{\partial x^2}(1,t)$. The exact solution of this model is &#xA;$$&#xA;u(x,t) = \frac{1}{2}[e^{2 \pi^{\frac32}t} \sin (2\pi x - 2\pi^{\frac32} t) + e^{-2\pi^{\frac32}t} \sin (2\pi x + 2 \pi^{\frac32}t)]&#xA;$$&#xA;I would like to use boundary value technique to numerically solve this equation, i.e. i would like to write the following system of $2MN$ unknowns in matrix form and solve it in Matlab. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I partition time as $0 = t_0 &amp;lt; t_1 &amp;lt; \dots &amp;lt; t_{N} = 0.1$. Step size is of length $h=\Delta t =\frac{0.1}{N}$, i.e. $t_n = t_0 + nh$, $n=1,2,\dots,N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I partition space as $0 = x_0 &amp;lt; x_1 &amp;lt; \dots &amp;lt; x_{M} = 1$. Step size is of length $h'=\Delta x =\frac{1}{M}$, i.e. $x_i = x_0 + ih'$, $i=1,2,\dots,M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I rewrite the equation as system,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;u_t &amp;amp;= v, \\&#xA;v_t &amp;amp;= u_{xxx}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and using central approximation for time derivative and second order scheme for space derivative&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\frac{u_i^{n+1} - u_i^{n-1}}{2\Delta t} &amp;amp;= v_i^n \\&#xA;\frac{v_i^{n+1} - v_i^{n-1}}{2\Delta t} &amp;amp;= \frac{u_{i+2}^n - 2 u_{i+1}^n +2u_{i-1}^n -u_{i-2}^n}{2\Delta x^3} &#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To add an interesting fact: this can't be solved via step by step approach. At least as far as i know. BTCS and FTCS methods do not work. Also other schemes are not stable (eigenvalues of step by step matrix form a cross in complex plane and are outside stability regions of the schemes for any $\Delta t$ and $\Delta x$). This is why i would like to use the approach of solving system of equations to get values at grid points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my try deriving system of equation when i introduce vector $v$. I define $u^{n}:=[u_1^{n},u_2^{n},\dots,u_M^{n}]^T$ and $v^{n}:=[v_1^{n},v_2^{n},\dots,v_M^{n}]^T$ and then $w^n= [(u^n)^T,(v^n)^T]^T$ and then discretizing the space yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{pmatrix}&#xA;u^n \\&#xA;v^n&#xA;\end{pmatrix}_t &#xA;= \begin{pmatrix}&#xA;0 &amp;amp; I \\&#xA;A &amp;amp; 0&#xA;\end{pmatrix}w^n =: D w^n&#xA;$$&#xA;where &#xA;$$&#xA;A=\frac{1}{2 \Delta x^3}&#xA;\begin{pmatrix}&#xA;0 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1 &amp;amp; 2 \\ &#xA;2 &amp;amp; 0 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1\\&#xA;-1 &amp;amp; 2 &amp;amp; 0 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0  \\&#xA;0 &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;\vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; 0 \\&#xA;0 &amp;amp; \cdots &amp;amp; 0 &amp;amp;  -1 &amp;amp; 2 &amp;amp; 0 &amp;amp; -2 &amp;amp; 1 \\&#xA;1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1 &amp;amp; 2 &amp;amp; 0 &amp;amp; -2 \\&#xA;-2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1 &amp;amp; 2 &amp;amp; 0 &#xA;\end{pmatrix}&#xA;$$&#xA;using periodic boundary conditions. Central discretization in time yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w^{n+1} - w^{n-1} - 2\Delta tD w^n = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the last step i use backward in time derivative approximation to get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-w^{N-1} - (\Delta tD - I) w^{N} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, defining $w=[(w^0)^T,(w^1)T,\dots,(w^N)^T]^T$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;C &amp;amp;=\begin{pmatrix}&#xA;    I      &amp;amp; 0      &amp;amp; 0      &amp;amp; 0      &amp;amp; \dots  &amp;amp; 0      &amp;amp; 0      &amp;amp; 0      \\&#xA;    -I     &amp;amp; -2\Delta tD      &amp;amp; I      &amp;amp; 0      &amp;amp; \dots  &amp;amp; 0      &amp;amp; 0      &amp;amp; 0      \\    &#xA;    0      &amp;amp; -I     &amp;amp; -2\Delta tD      &amp;amp; I      &amp;amp; \ddots  &amp;amp; 0      &amp;amp; 0      &amp;amp; 0      \\&#xA;    0      &amp;amp; 0      &amp;amp; -I     &amp;amp; -2\Delta tD      &amp;amp; \ddots  &amp;amp; \ddots     &amp;amp; 0      &amp;amp; 0      \\&#xA;    \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;    0      &amp;amp; 0      &amp;amp; 0      &amp;amp; \ddots      &amp;amp; \ddots  &amp;amp; -2\Delta tD      &amp;amp; I      &amp;amp; 0      \\&#xA;    0      &amp;amp; 0      &amp;amp; 0      &amp;amp; 0      &amp;amp; \ddots  &amp;amp; -I     &amp;amp; -2\Delta tD        &amp;amp; I      \\&#xA;    0      &amp;amp; 0      &amp;amp; 0      &amp;amp; 0      &amp;amp; \dots  &amp;amp; 0      &amp;amp; -I     &amp;amp; I-\Delta tD   \\&#xA;\end{pmatrix}\\&#xA;\end{align*}&#xA;and $c = ((w^0)^T,0...,0)^T$&#xA;i have to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C w = c$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: It works in Matlab now and the results are nice. Just matrix inversion is not fast. &lt;/p&gt;&#xA;" OwnerUserId="4234" LastEditorUserId="4234" LastEditDate="2013-05-29T08:56:52.963" LastActivityDate="2013-05-29T08:56:52.963" Title="Boundary value method for equation $u_{tt} = u_{xxx}$" Tags="|pde|discretization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7431" PostTypeId="2" ParentId="7428" CreationDate="2013-05-29T00:36:55.233" Score="2" Body="&lt;p&gt;The periodic boundary condition is really &lt;em&gt;not&lt;/em&gt; a boundary condition; it's merely a statement that the domain is &lt;em&gt;infinite&lt;/em&gt;. What that means is that you need to  discretize your PDE is space at each and every point, including $i=0$ and $i=M$ and simply use &lt;em&gt;periodic&lt;/em&gt; indecies, i.e. $i&amp;gt;M: i\leftarrow i-M$ and $i&amp;lt;0: i\leftarrow i+M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the time discretization, how about the following? &#xA;$$&#xA;\frac{u_i^{n+1}-2u_i^n+u^{n-1}_i}{\Delta t^2} = \frac{u_{i+2}^{n+1}-2u_{i+1}^{n+1} + 2u_{i-1}^{n+1} - u_{i-2}^{n+1}}{2\Delta x^3}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be warned though, this matrix has constant null space that you need to remove before solving. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What sort of problems are you running into? Maybe if you post them here, we could be of more help.&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="485" LastEditDate="2013-05-29T01:21:41.830" LastActivityDate="2013-05-29T01:21:41.830" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7432" PostTypeId="2" ParentId="7424" CreationDate="2013-05-29T01:08:23.360" Score="3" Body="&lt;p&gt;What you mention about the value of the stream function not being constant for non-symmetric boundaries is not correct. Indeed for &lt;em&gt;any&lt;/em&gt; boundary for which the normal component of velocity is zero, the stream function should be constant on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can formally show this property as follows. In 2D, a stream function $\psi$ is defined such that the velocity vector can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\vec{v} = \hat{\mathbf{k}} \times \nabla \psi&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\hat{\mathbf{k}}$ is the unit vector in the $z$ direction. Thus the normal component of velocity is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;v_n =  \hat{\mathbf{n}} \cdot \left(\hat{\mathbf{k}} \times \nabla \psi\right) = -\hat{\mathbf{k}} \cdot \left(\hat{\mathbf{n}} \times \nabla \psi\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\hat{\mathbf{n}}$ is the unit vector normal to the boundary. For $v_n$ to vanish on the boundary, it is necessary to have $\hat{\mathbf{n}} \times \nabla \psi = 0$ on the boundary. This last requirement simply states that $\nabla \psi$ is parallel to the normal unit vector on the boundary. In other words, one must have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla_t \psi = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\hat{\mathbf{t}}$ denotes the tangential unit vector on the boundary. The only way this can be achieved for every point on the boundary is to have $\psi = \text{const.}$ on the boundary.&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-05-29T01:08:23.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7433" PostTypeId="1" CreationDate="2013-05-29T01:12:01.150" Score="1" ViewCount="1169" Body="&lt;p&gt;This is my first time using FEniCS. I am trying to solve an elliptic PDE, fairly similar to the Laplace equation, on a rectangle with two holes in it. The holes are level sets of an energy function, so I am not completely in control over their geometry. For a first approximation I can use ellipses, but I would like to still use an extensible technique so that the potential can be varied flexibly. The tutorial mentions that mesh preprocessors are usually required to initialize problems with complex geometry, but I was unable to find an actual example of how to do this. Can anyone suggest a method for building the mesh?&lt;/p&gt;&#xA;" OwnerUserId="4438" LastEditorUserId="4438" LastEditDate="2013-05-29T01:53:07.280" LastActivityDate="2013-05-29T10:19:26.880" Title="How can I build a mesh with holes for use in FEniCS?" Tags="|pde|finite-element|fenics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7435" PostTypeId="2" ParentId="7409" CreationDate="2013-05-29T04:14:24.967" Score="0" Body="&lt;p&gt;Thanks for all the tips and hints. So after some dist-upgrading, debugging, manual reading, I finally got this script running. I'm including my solution here, just for the sake of completeness. Note that this code models the &lt;a href=&quot;http://en.wikipedia.org/wiki/Burgers%27_equation&quot; rel=&quot;nofollow&quot;&gt;inviscid Burgers equations&lt;/a&gt; in one dimension.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import*&#xA;&#xA;n = 100&#xA;mesh = UnitIntervalMesh(n)&#xA;#mesh = IntervalMesh(500,-1.,10.)&#xA;V = FunctionSpace(mesh,'CG',1)&#xA;u = Function(V)&#xA;u0 = Expression(&quot;1./(sqrt(2*pi)*0.1) * exp(-0.5*pow((x[0]-0.5),2)/0.01)&quot;)&#xA;u.interpolate(u0)&#xA;&#xA;u_next = Function(V)&#xA;v = TestFunction(V)&#xA;&#xA;nu = Constant(0.01)&#xA;timestep = Constant(0.01)&#xA;&#xA;F = ((u_next-u)/timestep*v&#xA;     + inner(u_next.dx(0)*u_next, v))*dx&#xA;bc = DirichletBC(V, 0.0, &quot;on_boundary&quot;)&#xA;&#xA;t = 0.0&#xA;end =10&#xA;while(t &amp;lt;= end):&#xA;    solve(F==0, u_next, bc)&#xA;    u.assign(u_next)&#xA;    t+=float(timestep)&#xA;    plot(u)&#xA;interactive()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2013-05-29T04:14:24.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7436" PostTypeId="1" AcceptedAnswerId="7449" CreationDate="2013-05-29T04:42:49.983" Score="3" ViewCount="2080" Body="&lt;p&gt;I want to calculate the &lt;em&gt;unknown&lt;/em&gt; electrical potential &lt;strong&gt;distribution&lt;/strong&gt; $\phi(x)$ (notice this is a function of $x$) from a &lt;em&gt;known&lt;/em&gt; electric field distribution $\boldsymbol{E}(x)$ using the Poisson equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-\frac{\partial^2 \phi}{\partial x^2} = \frac{\partial\boldsymbol{E}}{\partial x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to do this &lt;strong&gt;numerically&lt;/strong&gt;, but I am not sure about the procedure I need to follow to get $\phi(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I have numerical data for $\boldsymbol{E}(x)$ I first integrated the Poisson equation once,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int \frac{\partial^2 \phi(x)}{\partial x^2} dx = \int \frac{\partial\boldsymbol{E}(x)}{\partial x} dx&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to get,&#xA;$$&#xA;\frac{\partial \phi(x)}{\partial x} = -\boldsymbol{E}(x) + C&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would imply that all I need to do to find $\phi(x)$ is to integrate the numerical data I have for $\boldsymbol{E}(x)$. This is easily done using the trapezium rule but it will give a &lt;strong&gt;constant&lt;/strong&gt; value, not a distribution! Basically I don't understand how to convert $\boldsymbol{E}(x)$ to $\phi(x)$ numerically, where I have numerical data for $\boldsymbol{E}(x)$. Do you have suggestions on how I can do this?&lt;/p&gt;&#xA;" OwnerUserId="4440" OwnerDisplayName="semiphys" LastEditorUserId="3691" LastEditDate="2013-06-29T00:36:14.167" LastActivityDate="2016-05-05T23:18:29.087" Title="How can I calculate numerically an electrical potential distribution from an electric field distribution?" Tags="|poisson|electromagnetism|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7437" PostTypeId="2" ParentId="7433" CreationDate="2013-05-29T09:09:39.420" Score="5" Body="&lt;p&gt;Take a look at the CSG (Constructive Solid Geometry) demos in DOLFIN. You should be able to generate a rectangle with two holes with something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh = Mesh(Rectangle(...) - Ellipse(...) - Ellipse(...), resolution)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4243" LastActivityDate="2013-05-29T09:09:39.420" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7438" PostTypeId="2" ParentId="2630" CreationDate="2013-05-29T09:51:10.627" Score="6" Body="&lt;p&gt;SLICOT is the tool to use for dense problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large but sparse system, there is the &lt;a href=&quot;http://www.tu-chemnitz.de/sfb393/lyapack/&quot; rel=&quot;noreferrer&quot;&gt;lyapack&lt;/a&gt; toolbox for MATLAB.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithms in lyapack base on computing iteratively low-rank factors $Z_n$, so that $Z_n^HZ_n$ approaches $\Sigma$, where $\Sigma$ is the symmetric (positive or negative) definite solution of the Lyapunov equation. Computing only the factors in combination with sparsity of the coefficients makes this approach feasible for large scale equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is vivid ongoing research at the Max-Planck Institute in Magdeburg, Germany, on sparse Lyapunov equations. However, the announcement of the upcoming release of the sucessor of lyapack - MESS - is quite a few years old. Nevertheless, it is worth checking the MESS's &lt;a href=&quot;http://www.mpi-magdeburg.mpg.de/mpcsc/mitarbeiter/saak/Software/mess.php?lang=en&quot; rel=&quot;noreferrer&quot;&gt;webpage&lt;/a&gt; and the publications of the contributing authors from time to time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: My thesis supervisor is a major contributor both to SLICOT and lyapack and I am in regular contact with the developers of MESS.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-29T15:24:51.723" LastActivityDate="2013-05-29T15:24:51.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7439" PostTypeId="2" ParentId="7433" CreationDate="2013-05-29T10:19:26.880" Score="2" Body="&lt;p&gt;If your mesh depends on solution you can do fixed-point iteration:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;compute solution,&lt;/li&gt;&#xA;&lt;li&gt;move mesh,&lt;/li&gt;&#xA;&lt;li&gt;if mesh is deformed to much, create new one and project solution on it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you use CSG in DOLFIN, then you can handle all three steps within DOLFIN. If using external mesh generator third step requires instructing mesh generator by DOLFIN. One way to do it is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;prepare input for mesh generator parametrized by few &lt;code&gt;float&lt;/code&gt;s,&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;subprocess.call&lt;/code&gt; mesh generator, &lt;code&gt;meshconvert&lt;/code&gt;,&lt;/li&gt;&#xA;&lt;li&gt;read in new mesh&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow&quot;&gt;Triangle&lt;/a&gt; can generate triangulations of domain with holes in 2D. &lt;code&gt;meshconvert&lt;/code&gt; handles also facet and cell markers produced Triangle.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2013-05-29T10:19:26.880" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7440" PostTypeId="1" AcceptedAnswerId="7441" CreationDate="2013-05-29T12:41:54.950" Score="0" ViewCount="181" Body="&lt;p&gt;today I tried some evaluation on a DG function space (Fenics 1.2.0).&#xA;In my calculations I need to use meshes with a large number of cells (~10mio).&#xA;At the moment it takes a very long time to create a DG function space (in my distinct case a vector function space) for meshes which contains many cells.&#xA;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;set_log_level(DEBUG)&#xA;mesh = UnitCubeMesh(100,100,100)&#xA;H13 = VectorFunctionSpace(mesh, &quot;DG&quot;, 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A CG function space takes around about 20 seconds for creation. For DG I aborted the job after 10 minutes. For another mesh I waited more than one hour without termination.&#xA;Is there a way to speed up the DG function space creation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot in advance&lt;/p&gt;&#xA;" OwnerUserId="4441" LastActivityDate="2013-05-29T14:58:27.527" Title="Speed up DG FunctionSpace evalutation" Tags="|fenics|" AnswerCount="2" CommentCount="3" ClosedDate="2013-05-29T16:49:45.717" ContentLicense="CC BY-SA 3.0" />
  <row Id="7441" PostTypeId="2" ParentId="7440" CreationDate="2013-05-29T14:48:35.867" Score="0" Body="&lt;p&gt;Some code in &lt;code&gt;DofMapBuilder::reorder_local&lt;/code&gt; may scale poorly (see Anders' answer). As a workaround you may disable reordering dofs&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;parameters['reorder_dofs_serial'] = False&#xA;H13 = VectorFunctionSpace(mesh, &quot;DG&quot;, 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-05-29T14:58:27.527" LastActivityDate="2013-05-29T14:58:27.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7442" PostTypeId="2" ParentId="7440" CreationDate="2013-05-29T14:56:04.220" Score="1" Body="&lt;p&gt;The DG space in this case has 72,000,000 dofs while the dimension of the CG space is only 3,090,903. So expect it to take quite some time longer to create the DG space, especially if the dofs are reordered (which is the default behavior).&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-05-29T14:56:04.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7444" PostTypeId="2" ParentId="7406" CreationDate="2013-05-29T15:53:08.307" Score="2" Body="&lt;p&gt;The reduction of calculation time in CFD is a general problem. Although I have never worked with FLuent, I don't think that there are features that magically speed up your simulation. That's why I can only give some VERY general hints. For further insight, I suggest you browse the internet, e.g. &lt;a href=&quot;http://www.cfd-online.com/&quot; rel=&quot;nofollow&quot;&gt;cfd-online&lt;/a&gt; for the highlighted &lt;code&gt;keywords&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use &lt;code&gt;mesh adaptation&lt;/code&gt; to reduce the scale of your problem. In the regions away from the walls, you maybe can reduce the number of cells. For the near-wall regions you have to take care of the boundary layer. Monitoring the &lt;code&gt;y+&lt;/code&gt; value helps you estimate how coarse you can go. Also the use of &lt;code&gt;wall functions&lt;/code&gt; for your turbulence model can be considered.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do some try and error with the &lt;code&gt;relaxation parameters&lt;/code&gt;. This is eventually balancing stability and convergence speed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And, as Paul suggested, you can simply use more computation power.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-06-07T09:58:34.667" LastActivityDate="2013-06-07T09:58:34.667" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7446" PostTypeId="1" CreationDate="2013-05-29T19:13:20.113" Score="6" ViewCount="701" Body="&lt;p&gt;I want to know if there is any way to define the test and trial function in the way that I want instead of using the default functions. So if I want define the polynomial and basis and coefficient, how can I do it in Fenics? Here is the concept in general:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve stochastic partial differential equation which is like&#xA;$$\Pi(x,t,w;u)=f(x,t;w)$$   $$x \in \Re^d  \qquad t \in [0,T] \qquad w \in Ω $$&#xA;where is the solution is $U:=U(x,t;w)$ and $f(x,t;w)$ is the source term.&#xA;We can discretize the stochastic space using a finite number of independent random variables.&#xA;$$\xi(w)=\{\xi(w_1),\ldots,\xi(w_n)\}$$&#xA;This allows us to rewrite the governing equation as a dimensional &#xA;differential equation in the strong form:&#xA;$$\Pi(x,t,\xi;u)=f(x,t;w)\qquad (\xi,x)\in \Gamma \times D$$&#xA;Thus the solution which is regarded as a stochastic process, can &#xA;be expanded by the generalized polynomial as:&#xA;$$u(x,t;w)=\sum_{i=0}^P u_i(x,t) \phi_i (\xi)$$&#xA;where $P$ is the number of GPCE terms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we substitute $U$ into the weak form.&#xA;Choosing the test function the same as the generalized polynomial &#xA;in order to let the residual to be orthogonal to the subspace, thus &#xA;minimize the error, we obtain:&#xA;$$  \left\langle \Pi\left(x,t,\xi;\sum_{i=0}^P u_i(x,t) \phi_i (\xi)\right),\phi_k (\xi)\right\rangle = \left\langle  f(x,t;\xi),\phi_k (\xi)  \right\rangle   $$   $K=0,1,.. P$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Results in a set of coupled deterministic equations. One can &#xA;employ e.g. classical finite element methods to seek the &#xA;deterministic coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make the problem more clear there is well defined problem related to 2D Poisson Equation: &lt;a href=&quot;http://www.stanford.edu/group/uq/docs/sgs_primer.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here on page 7&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we only have one random variable, the number of random dimensions is 1. Polynomial chaos expansion value for my problem with polynomial of order of 3 will be a PCE=3*3 by matrix.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Problem&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I need to modify the &lt;code&gt;V = VectorFunctionSpace(mesh, &quot;Lagrange&quot;, 1)&lt;/code&gt;so in this problem in order that V will be the Kronecker tensor product of V and PCE which will lead to a 10*3 ,10*3, 10*3 and based on that matrix I can modify my equation to calculate the statistics of the solution. Is there any way that I can calculate the Kronecker tensor product of V and another matrix and replace it with my V? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically  I am trying to solving this equation with uncertain location load with this method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the my deterministic code,&lt;a href=&quot;https://scicomp.stackexchange.com/questions/7152/fenicsresult-of-steady-state-dynamic-linear-elastic-doesnt-match-with-actual-va&quot;&gt;Fenics: Result of Steady state dynamic linear elastic doesn&amp;#39;t match with actual values&lt;/a&gt;  how I can make this deterministic code to stochastic with modifying test and trial function in Fenics.&lt;/p&gt;&#xA;" OwnerUserId="4278" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-04-08T02:38:27.327" Title="Stochastic Galerkin projection approach for using generalized polynomial chaos expansion (GPCE) in solving PDE" Tags="|fenics|polynomials|stochastic|spectral-method|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7447" PostTypeId="2" ParentId="2515" CreationDate="2013-05-29T19:48:26.100" Score="1" Body="&lt;p&gt;check this : &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/m_src/sandia_sparse/sandia_sparse.html&quot; rel=&quot;nofollow&quot;&gt;http://people.sc.fsu.edu/~jburkardt/m_src/sandia_sparse/sandia_sparse.html&lt;/a&gt;&#xA;this may help you. this is for just points.&#xA;also this is very nice tool box &lt;a href=&quot;http://www.ians.uni-stuttgart.de/spinterp/&quot; rel=&quot;nofollow&quot;&gt;http://www.ians.uni-stuttgart.de/spinterp/&lt;/a&gt; you can generate the points in space and then use deterministic finite element solver to evaluate those points and then calculate the statistic of the responses.&lt;/p&gt;&#xA;" OwnerUserId="4278" LastActivityDate="2013-05-29T19:48:26.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7448" PostTypeId="1" CreationDate="2013-05-29T17:43:52.330" Score="4" ViewCount="447" Body="&lt;p&gt;I found the following simulation of an orbit given the semi-major axis of the ellipse, eccentricity, initial velocity, the mass of the object being orbited, and initial position of the orbiting satellite.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The program basically starts off with the following initial conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;h \to stepSize\\&#xA;a \to semiMajorAxis\\&#xA;e \to eccentricity\\&#xA;g \to GM/a\\&#xA;\vec{X_0} = (a(1+e), 0, 0)\\&#xA;\vec{V_o} = (0, \sqrt{\frac{g(1-e)}{(1+e)}},0)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given these initial conditions, the next state is calculated as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\vec{X_{temp}} = \vec{X_0} +  (\frac h2)\vec{V_0}\\&#xA;r^3 \to (\sqrt{x_0^2 + y_o^2})^3\\&#xA;\vec{Acc_{temp}} = \frac{-GM}{r^3}(\vec{X_{temp}})\\&#xA;\vec{V_1} = \vec{V_0} + h*\vec{Acc_{temp}}\\&#xA;\vec{X_1} = \vec{X_{temp}} + \frac h2 \vec{V_1}\\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approximations continue accordingly with accuracy increasing for smaller stepSize values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I presented this in a way that makes sense, my question is in regards to my result.  Look at my simulation here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://zixg.com/orbits/&quot; rel=&quot;nofollow&quot;&gt;http://zixg.com/orbits/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take note of the two stationary blue dots.  These represent the apogee and perigee of an ellipse with semi-major axis 'a' and eccentricity 'e' (Hence, the sun-like object is centered at a corresponding focal point).  So it seems that this method doesn't approximate the initial ellipse at all, but rather results in a orbital trajectory that is much larger than the initially assumed ellipse. I'm not sure what to make of this?&lt;/p&gt;&#xA;" OwnerDisplayName="Tyrick" LastEditorUserId="521" LastEditDate="2013-05-30T04:23:36.873" LastActivityDate="2013-05-30T04:23:36.873" Title="Verlet Leap-frog Method of approximating orbits" Tags="|simulation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7449" PostTypeId="2" ParentId="7436" CreationDate="2013-05-29T21:29:05.337" Score="3" Body="&lt;p&gt;The basic equation of electrostatic is&#xA;$$&#xA;\mathbf{E}=-\nabla\varphi&#xA;$$&#xA;not&#xA;$$&#xA;\frac{\partial\varphi}{\partial x}=-\mathbf{E}+C&#xA;$$&#xA;It follows from a simple principle that the work electric field does when moving a charge from point A to point B &lt;em&gt;does not&lt;/em&gt; depend on the path between A and B, otherwise, we would have a perpetuum mobile. &lt;em&gt;Potential&lt;/em&gt; is by definition is amount of this work. Therefore in order to get potential you have to integrate the force electric field creates at a test charge on any path that is &lt;em&gt;convenient&lt;/em&gt; for you when you move this charge from $\infty$  to the point of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's say you have a path that is described by a function $\mathbf{x}\left(t\right)$ such as&#xA;$$&#xA;\left\|\frac{\mathrm{d}\mathbf{x}}{\mathrm{d} t}\right\|=1&#xA;$$&#xA;In this case&#xA;$$&#xA;\varphi\left(\mathbf{x}_0\right)=-\int^{\mathbf{x}_0}_\infty\mathbf{E}\cdot \frac{\mathrm{d}\mathbf{x}}{\mathrm{d} t}\, \mathrm{d}t&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="4379" LastEditorUserId="4379" LastEditDate="2016-05-05T23:18:29.087" LastActivityDate="2016-05-05T23:18:29.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7450" PostTypeId="1" AcceptedAnswerId="7451" CreationDate="2013-05-29T23:38:07.303" Score="5" ViewCount="243" Body="&lt;p&gt;This might be a naive question, but when applying a implicit discretization to a PDE with a source term, should the source be averaged in time? For example if we take the diffusion equation with a non-linear source term,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = u_{xx}+s(x,t,u)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can apply the following central difference implicit scheme to the differential term,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{u_j^{n+1} - u_j^n}{\Delta t} = \left[ (1-\theta) (u_{j-1} - 2u_{j} + u_{j+1}) + \theta (u_{j-1} - 2u_{j} + u_{j+1})\right] + s(x,t,u)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but how should $s(x,t,u)$ be treated? Should we simply take the value the $n$ time point (this is what I have always done in the past),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;s(x,t) = s_j^n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or averaged over time,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;s(x,t) = (1-\theta)s_j^{n+1} + \theta s_j^{n}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure it is possible to implement a time average in this way because, in general, the $n+1$ points in time are &lt;strong&gt;unknowns&lt;/strong&gt;! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a silly question? Or is there some way of improving the time integration of the above equation by taking averages in time?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-05-30T02:28:49.517" LastActivityDate="2013-06-01T05:00:14.190" Title="Improving the time integration of implicit discretized PDE with a non-linear source term" Tags="|pde|discretization|time-integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7451" PostTypeId="2" ParentId="7450" CreationDate="2013-05-30T00:30:46.497" Score="6" Body="&lt;p&gt;If the right hand side were independent of $u$ then one would generally use the averaged form&#xA;$$&#xA;  (1-\theta)s_(x,t^{n+1}) + \theta s(x,t^n).&#xA;$$&#xA;In the nonlinear case you can't do that easily, as you note, but you can at least use some kind of extrapolation, for example approximate&#xA;$$&#xA;  (1-\theta)s(x,t^{n+1},u^{n+1}) + \theta s(x,t^n,u^n)&#xA;  \approx&#xA;  (1-\theta)s\left(x,t^{n+1},u^{n}+\Delta t\frac{u^n-u^{n-1}}{\Delta t}\right) + \theta s(x,t^n,u^n)&#xA;  .  &#xA;$$&#xA;You will find more tricks like this in the ODE literature -- read up on semi-implicit methods for ODEs of the kind $\dot x(t) = A x(t) + f(x(t),t)$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-30T00:30:46.497" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7452" PostTypeId="1" CreationDate="2013-05-30T03:03:49.123" Score="3" ViewCount="919" Body="&lt;p&gt;I just started working in a lab that is trying to design microfluidics devices to trap and manipulate colonies of E. coli cells. Given a 3D model of such a device created in Autodesk Inventor (or a similar program), and leaving out the cells, what would be the best way to simulate fluid flow through the channels of the device? The flows are completely laminar, so it seems like a computationally tractable-ish problem.&lt;/p&gt;&#xA;" OwnerUserId="34" LastActivityDate="2015-06-09T18:49:50.400" Title="Simulating laminar fluid flow in microfluidics device" Tags="|simulation|microfluidics|fluid-dynamics|3d|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7453" PostTypeId="2" ParentId="7452" CreationDate="2013-05-30T03:11:17.890" Score="4" Body="&lt;p&gt;Given the small dimensions in microfluidics, flow is well described by the Stokes equations. There are a number of commercial packages that I'm sure can simulate this kind of flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most available open source finite element packages can also do this, though likely with less ease. For example, in deal.II (that's a package of which I'm one of the principal developers), there's a tutorial program that shows how to do that:&#xA;  &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_22.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_22.html&lt;/a&gt;&#xA;.&#xA;I'm sure FEniCS and libMesh have similar capabilities to solve the Stokes equations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-30T03:11:17.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7454" PostTypeId="2" ParentId="7448" CreationDate="2013-05-30T04:16:07.107" Score="3" Body="&lt;p&gt;Looking at your code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var v = new Array(0, Math.sqrt((g*(1-e))/1+e), 0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You seem to be missing a set of parenthesis around (1+e).  Should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var v = new Array(0, Math.sqrt((g*(1-e))/(1+e)), 0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="521" LastActivityDate="2013-05-30T04:16:07.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7456" PostTypeId="1" AcceptedAnswerId="7462" CreationDate="2013-05-30T08:04:40.240" Score="11" ViewCount="2221" Body="&lt;p&gt;Starting with the advection equation in conservation form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = (a(x)u)_x&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a(x)$ is a velocity which depend on space, and $u$ is a concentration of a species which is conserved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discretising the flux (where the flux $f=a(x)u$, is defined on the edges of the cells between mesh points) gives,&#xA;$$&#xA;u_t = \frac{1}{h}\left( f_{j-{\frac{1}{2}}} - f_{j+{\frac{1}{2}}} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a first order upwind we approximate the fluxes as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_{j-{\frac{1}{2}}} = a(x_{j-\frac{1}{2}})u_{j-1} \\&#xA;f_{j+{\frac{1}{2}}} = a(x_{j+\frac{1}{2}})u_{j}&#xA;$$&#xA;Which gives,&#xA;$$&#xA;u_t = \frac{1}{h} \left( a(x_{j-\frac{1}{2}})u_{j-1} - a(x_{j+\frac{1}{2}})u_{j} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $a(x)$ was constant then this will reduce to the familiar upwind scheme i.e.,&#xA;$&#xA;u_t = \frac{a}{h} \left( u_{j-1} - u_{j} \right)&#xA;$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how can we treat the &lt;strong&gt;non-constant&lt;/strong&gt; coefficients of the advection equation? The velocity is defined at the cell centers, so a simple approach would be the following,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;a(x_{j-\frac{1}{2}}) \rightarrow a(x_{j-1}) \\&#xA;a(x_{j+\frac{1}{2}}) \rightarrow a(x_{j})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my preferred approach because it is very simple to implement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, we could also use (I am guessing) an averaging scheme to define the velocity at the cell edges,&#xA;$$&#xA;a(x_{j-\frac{1}{2}}) \rightarrow \frac{1}{2}a(x_{j-1} ) + \frac{1}{2}a(x_{j} ) \\&#xA;a(x_{j+\frac{1}{2}}) \rightarrow \frac{1}{2}a(x_{j} ) + \frac{1}{2}a(x_{j+1} ) \\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521009243&quot; rel=&quot;nofollow&quot;&gt;LeVeque's book&lt;/a&gt; he says,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So far we have assumed that the variable velocity $a(x)$ is specified by&#xA;  a constant value $a_j$ within the j-th grid cell. In some cases it is more&#xA;  natural to instead assume that a velocity $a_{j−{\frac{1}{2}}}$ is specified at each cell interface.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But he doesn't really elaborate too much after that. What is a common approach? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving a conservation problem (I am using the advection equation as a continuity equation) so I want to make sure that after applying discretisation that the conservation property is preserved. I would like to avoid any hidden surprises regarding these variable coefficients! Does anybody have some general comments and guidance?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; There are two really good answers below and I could only pick one :(&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-05-31T00:00:52.860" LastActivityDate="2013-05-31T01:47:22.370" Title="How should non-constant coefficients be treated with finite-volume first order upwind scheme?" Tags="|discretization|finite-volume|numerical-analysis|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7457" PostTypeId="1" CreationDate="2013-05-30T12:52:07.400" Score="0" ViewCount="34" Body="&lt;p&gt;I know I already posted this question and I thank you for your answers, but unfortunately I didn't find what I was looking for among them. Anyway now I'll rewrite the question more clearly, because before perhaps I've not been specific enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen that there is a way to use the finite differences method, on a cartesian orthogonal grid, to perform calculations on potential flow about an obstacle without using the Neumann conditions, but only Dirichlet conditions. It is done using the stream function, i tested it and it seem to work.&#xA;This method works this way:&#xA;You take a rectangular box, then you put two arbitrary values of the stream function (but the same in all the side) in the upper and lower side; then you linearly vary it along the lateral sides (so there is continuity with the potential on the upper and lower sides). In this way you define the undisturbed flow far from the obstacle. Then you place the symmetric obstacle right in the middle of the box, so you know for sure there is an horizontal streamline that divides the box in two and follows the obstacle boundary, so its potential is exactly the average value between the upper and lower side. So you have defined the Dirichlet conditions in all the box boundaries and in the obstacle boundary, and the problem is well defined and can be easily solved by using the finite difference scheme: you solve the Laplace equation for the stream function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this method I used has a big drawback: it can be used only with obstacles that are symmetric along an horizontal axis (the direction of the undisturbed stream). If the obstacle is asymmetric, the streamline that follows the obstacle is no longer horizontal outside the obstacle, and I don't know where it hits the box wall; then I don't know what is the stream function value that I have to put in the obstacle boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you now any workaround or method to use only Dirichlet conditions, but on an arbitrary shaped obstacle (a wing, for example) ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before, someone advised me to use panel methods, coordinate transformations, etc. It would be good, but I'm looking for something simpler, if possible. It's just for demonstration of the finite differences method, it doesn't have to be particularly efficient of fast; I'd prefer just the simplest solution you can think. Something I can possibly add to my program editing it the less possible. If instead you know a simple way to implement the Neumann conditions, it would also be good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance, your help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4448" LastActivityDate="2013-05-30T12:52:07.400" Title="Computation of potential flow using dirichlet conditions" Tags="|finite-difference|boundary-conditions|laminar-flow|" AnswerCount="0" CommentCount="3" ClosedDate="2013-05-30T14:36:17.843" ContentLicense="CC BY-SA 3.0" />
  <row Id="7458" PostTypeId="5" CreationDate="2013-05-30T13:03:16.730" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_analysis&quot; rel=&quot;nofollow&quot;&gt;Wikipedia article on numerical analysis&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-30T13:29:24.523" LastActivityDate="2013-05-30T13:29:24.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7459" PostTypeId="4" CreationDate="2013-05-30T13:03:16.730" Score="0" Body="Construction and analysis of algorithms to compute approximate discrete solutions of continuous problems. A canonical example is the approximation of derivatives via difference quotients. " OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-05-30T13:29:44.510" LastActivityDate="2013-05-30T13:29:44.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7461" PostTypeId="1" CreationDate="2013-05-30T16:45:58.933" Score="2" ViewCount="417" Body="&lt;p&gt;Looking at the stokes-iterative demo of FEniCS, after solving the linear system using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;U = Function(W)&#xA;solver.solve(U.vector(), bb)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the solution &lt;code&gt;U&lt;/code&gt; is split:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u, p = U.split()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a function with which I can get the indices of &lt;code&gt;U&lt;/code&gt;which contain the &lt;code&gt;u&lt;/code&gt; solution and the &lt;code&gt;p&lt;/code&gt; solution?&#xA;For example, if &lt;code&gt;U =[u1 p1 u2 p2,...]&lt;/code&gt; then I would like to have the arrays&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;indices_u = [0,2,4,...]&#xA;indices_p = [1,3,5,...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm guessing it's in FEniCS somewhere since it should be used to split the solution? I'd like to know how to do this in both C++ and python.&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4451" LastEditorUserId="1748" LastEditDate="2013-05-30T17:20:32.090" LastActivityDate="2013-05-30T20:56:55.503" Title="Mixed FEM vector indices of pressure and velocity in FEniCS" Tags="|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7462" PostTypeId="2" ParentId="7456" CreationDate="2013-05-30T17:01:51.413" Score="4" Body="&lt;p&gt;Depending on what kind of system you're looking at, it may be more convenient to consider the velocity $a$ as piecewise-constant within each cell, or that it's defined at the cell interfaces. For example, in meteorology, &lt;a href=&quot;http://en.wikipedia.org/wiki/Arakawa_grids&quot; rel=&quot;nofollow&quot;&gt;staggered grids&lt;/a&gt; are quite common, where pressure might be defined inside cells and velocity at cell interfaces. You could just as easily think of the velocity as defined within the cells. All told: the choice of representation should not affect the convergence of your method*, provided your discretization is stable and consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is most important (and you've already touched on this in your question) is that the discretized system is still conservative. Provided that your scheme can be written in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial u_j}{\partial t} = F_{j-\frac{1}{2}}(u_{j-1},u_j) - F_{j+\frac{1}{2}}(u_j,u_{j+1}) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then it should be conservative, since&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial}{\partial t}\int u dx = \sum_j\frac{\partial u_j}{\partial t}\delta x = \sum_j( F_{j-\frac{1}{2}}- F_{j+\frac{1}{2}} )\delta x = (F_{-\frac{1}{2}}-F_{N+\frac{1}{2}})\delta x $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your simple approach should work fine, as will averaging the velocity between cells to define it on cell interfaces, provided the velocity is always positive. Moreover, I don't think that averaging will net you any higher accuracy, so you are right to prefer the simple way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are also solving for the velocity and you have a system of equations, you may well need to be more careful. Likewise, if you're solving a nonlinear hyperbolic PDE and using flux limiters, you have to be yet more cautious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*However, for a system of hyperbolic PDEs, using staggered grids can &lt;em&gt;substantially&lt;/em&gt; ameliorate artificial dispersion/diffusion. If you want to know more, look up Arakawa C-grids or check out chapter 4 of &lt;a href=&quot;http://books.google.com/books?isbn=1441964126&quot; rel=&quot;nofollow&quot;&gt;this book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-05-30T17:01:51.413" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7463" PostTypeId="2" ParentId="7456" CreationDate="2013-05-30T17:09:09.643" Score="4" Body="&lt;p&gt;You can use any type of interpolation to determine $a(x_{j-\frac{1}{2}})$, and the method will remain conservative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why this is so, consider that the analytic definition of conservative is that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial}{\partial t}\int_D u(x)\, dx = \int_{\partial D} a(x)u(x) dS,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ is the problem domain. This says that the change in the conserved quantity is equal to the flux at the boundary, and follows immediately from integrating the conservation law. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If our discretization is of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t(x_j) = \frac{1}{h}\left(a(x_{j-\frac{1}{2}})u_{j-\frac{1}{2}} - a(x_{j+\frac{1}{2}})u_{j+\frac{1}{2}}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x_1,\ldots, x_n$ are our grid points, $D = [c,d]$, and $c = x_{\frac{1}{2}}$, $d = x_{n+\frac{1}{2}}$, then the equivalent discrete statement of conservation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{h}\sum_{j=1}^n\left(a(x_{j-\frac{1}{2}})u_{j-\frac{1}{2}} - a(x_{j+\frac{1}{2}})u_{j+\frac{1}{2}}\right) = a(x_{\frac{1}{2}})u_{\frac{1}{2}} - a(x_{n+\frac{1}{2}})u_{n+\frac{1}{2}},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this can easily be observed to hold by expanding the sum on the left-hand side. Note that, in your case with upwinding, $u_{j-\frac{1}{2}} = u_{j-1}$ and $u_{j+\frac{1}{2}} = u_j$, though it should be noted that this scheme is only an appropriate upwind scheme if $a(x)u$ is always positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For higher-order methods, provided that $a(x)$ is smooth, one can simply fit a polynomial to the points $a(x_{j-r}), \ldots, a(x_{j+s})$, and evaluate the polynomial at $a(x_{j-\frac{1}{2}})$.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-05-30T17:23:15.943" LastActivityDate="2013-05-30T17:23:15.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7464" PostTypeId="2" ParentId="7461" CreationDate="2013-05-30T18:30:47.157" Score="7" Body="&lt;p&gt;Doing &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u, p = U.split()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives you a view into &lt;code&gt;U&lt;/code&gt;, i.e. &lt;code&gt;u&lt;/code&gt;, &lt;code&gt;p&lt;/code&gt; and &lt;code&gt;U&lt;/code&gt; share degree of freedom data. Doing  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u, p = U.split(deepcopy=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives a &lt;code&gt;u&lt;/code&gt; and a &lt;code&gt;p&lt;/code&gt; that have their own degree-of-freedom vectors. Use of shallow and deep sub-functions is demonstrated in the DOLFIN demo &lt;code&gt;demo/undocumented/stokes-taylor-hood&lt;/code&gt; (from both C++ and Python). The syntax is a bit different from the above in C++, but is shown in the demo.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are no guarantees on the ordering of the degrees of freedom in the underlying vector. They should, in general, be accessed via function interfaces.&lt;/p&gt;&#xA;" OwnerUserId="3952" LastEditorUserId="3952" LastEditDate="2013-05-30T20:56:55.503" LastActivityDate="2013-05-30T20:56:55.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7465" PostTypeId="2" ParentId="1202" CreationDate="2013-05-30T18:51:37.723" Score="1" Body="&lt;p&gt;Have you looked at Yousef Saad's package of iterative solvers &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/software/ITSOL/index.html&quot; rel=&quot;nofollow&quot;&gt;ITSOL&lt;/a&gt;? It includes another library called the Algebraic Recursive Multilevel Solver &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/software/pARMS/index.html&quot; rel=&quot;nofollow&quot;&gt;pARMS&lt;/a&gt;, which I'm pretty sure is what you're talking about; Saad is the author of the paper you linked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also &lt;a href=&quot;http://www.cs.odu.edu/~pothen/Software/Euclid/&quot; rel=&quot;nofollow&quot;&gt;Euclid&lt;/a&gt;, which PETSc can interface to, but I don't think Euclid works in a multilevel fashion like pARMS.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-05-30T18:51:37.723" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7466" PostTypeId="2" ParentId="7369" CreationDate="2013-05-30T19:28:02.227" Score="10" Body="&lt;p&gt;&lt;a href=&quot;http://www-users.cs.umn.edu/~saad/software/filtlan/index.html&quot;&gt;FILTLAN&lt;/a&gt; is a C++ library for computing interior eigenvalues of sparse symmetric matrices. The fact that there is a whole package devoted to just this should tell you that it's a pretty hard problem. Finding the largest or smallest few eigenvalues of a symmetric matrix can be done by shifting/inverting and using the Lanczos algorithm, but the middle of the spectrum is another matter. If you do want to use this, you can use SWIG to call a C++ program from python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your end goal is to compute large powers of the matrix, you could just compute eigenvectors corresponding to the largest eigenvalues, content in the knowledge that the smaller modes will be less important as you take large powers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, you may indeed be better off computing the powers directly. They'll grow less and less sparse as you compute higher powers, which will mean taking up more memory; depending on how high $k$ is, you may eventually want to switch to a dense matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forgive me if these are already obvious to you: you can exploit the binary nature of the matrix by telling numpy that it consists of integers instead of floats, say by using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = np.zeros(100,dtype=np.uint)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will (hopefully) save some space. You can save time (but not memory) by blocking the matrix multiplications. Say you want to compute $A^{16}$; you compute $A^2$, then square this to get $A^4$, square this to get $A^8$, and so on. That way, you do $\approx\log_2 k$ matrix multiplications instead of $k$ multiplications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also explore calling a parallel sparse linear algebra library like CUSP or cuSPARSE from Python if speed is your concern and you have an NVIDIA GPU.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-05-30T19:28:02.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7467" PostTypeId="1" AcceptedAnswerId="7477" CreationDate="2013-05-29T08:17:29.617" Score="6" ViewCount="461" Body="&lt;p&gt;I want to determine the big and small frequency seperation from timeseries data for the sun. An excerpt of the data (timeseries and power series) is plotted below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The power series is calculated in MATLAB like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = length(t_obs);&#xA;dt = diff(t_obs(1:2));&#xA;y = fft(d_obs, n);&#xA;P = y .* conj(y)/n;&#xA;f = (0:n/2)/(n*dt);&#xA;&#xA;f = f(1:(n/2));&#xA;P = P(1:(n/2));&#xA;plot(f, P);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I can't understand is this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can I get the big frequency separation $\Delta \nu$ and the small separation $\delta \nu$ for $l=0$ and $l=1$ from the powerseries without having to read it manually from the plot (there are many datasets). If you should want to show an example of implementing this (although an explanation or a hint will suffice), I'm fluent in Python as well as MATLAB.&lt;/li&gt;&#xA;&lt;li&gt;What is the unit of the y-axis on the powerseries?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Timeseries plot&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/BQ9Qe.png&quot; alt=&quot;https://i.stack.imgur.com/wD9fR.png&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Power series plot&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/BQA3X.png&quot; alt=&quot;https://i.stack.imgur.com/M4m2U.png&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference:&#xA;$\delta\nu_l = \nu_{nl}-\nu_{n-1l+2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope you can help.&lt;/p&gt;&#xA;" OwnerDisplayName="user4452" LastEditorDisplayName="user4452" LastEditDate="2013-05-30T20:48:17.177" LastActivityDate="2013-05-31T07:46:56.280" Title="Calculate large and small frequency separation for the Sun" Tags="|matlab|python|numpy|signal-processing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7468" PostTypeId="1" AcceptedAnswerId="7469" CreationDate="2013-05-30T21:24:32.033" Score="15" ViewCount="2705" Body="&lt;p&gt;I've just started messing around with FEniCS. I am solving Poisson with 3rd order elements and would like to visualize the results. However, when I use plot(u), the visualization is just a linear interpolation of the results. I get the same thing when I output to VTK. In another code I am working with, I wrote a VTK outputter that would upsample higher order elements so that they actually looked higher order in Paraview. Is there anything like this (or better) in FEniCS?&lt;/p&gt;&#xA;" OwnerUserId="3309" LastActivityDate="2013-06-09T21:47:17.153" Title="FeniCS: Visualizing high order elements" Tags="|finite-element|fenics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7469" PostTypeId="2" ParentId="7468" CreationDate="2013-05-30T21:44:19.527" Score="12" Body="&lt;p&gt;You can interpolate the solution onto a finer mesh and then plot it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;&#xA;coarse_mesh = UnitSquareMesh(2, 2)&#xA;fine_mesh = refine(refine(refine(coarse_mesh)))&#xA;&#xA;P2_coarse = FunctionSpace(coarse_mesh, &quot;CG&quot;, 2)&#xA;P1_fine = FunctionSpace(fine_mesh, &quot;CG&quot;, 1)&#xA;&#xA;f = interpolate(Expression(&quot;sin(pi*x[0])*sin(pi*x[1])&quot;), P2_coarse)&#xA;g = interpolate(f, P1_fine)&#xA;&#xA;plot(f, title=&quot;Bad plot&quot;)&#xA;plot(g, title=&quot;Good plot&quot;)&#xA;&#xA;interactive()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice how you can see the outline of the coarse P2 triangles in the plot on the finer mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PnfdL.png&quot; alt=&quot;Plot of P2 function on coarse mesh&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/muiyt.png&quot; alt=&quot;Plot of P2 function interpolated to a P1 function on a fine mesh&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="359" LastEditorUserId="359" LastEditDate="2013-05-30T21:57:03.567" LastActivityDate="2013-05-30T21:57:03.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7471" PostTypeId="2" ParentId="7456" CreationDate="2013-05-31T01:47:22.370" Score="5" Body="&lt;p&gt;Others have said it all, but I just wanted to add a simple, yet sometimes subtle, point. Your upwind discretization remains conservative as long as you use a &lt;strong&gt;consistent&lt;/strong&gt; interpolation of $a(x)$ on the cell boundaries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by consistent is that the &lt;em&gt;only&lt;/em&gt; condition that the interpolation needs to satisfy is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;a_{i+1/2}^+ = a_{i+1/2}^-&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, as long as your interpolation method is &lt;strong&gt;continuous&lt;/strong&gt; across cell boundaries, your discretization is guaranteed to remain conservative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may not seem like a big issue here in 1D (and it should not) but can cause issues at the coarse-fine interfaces on multi-level AMR grids.&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-05-31T01:47:22.370" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7474" PostTypeId="1" AcceptedAnswerId="7476" CreationDate="2013-05-31T04:34:03.670" Score="16" ViewCount="6114" Body="&lt;p&gt;In the incompressible Navier-Stokes equations,&#xA;\begin{align*}&#xA;\rho\left(\mathbf{u}_t + (\mathbf{u} \cdot \nabla)\mathbf{u}\right) &amp;amp;= - \nabla p + \mu\Delta\mathbf{u} + \mathbf{f}\\&#xA;\nabla\cdot\mathbf{u} &amp;amp;= 0&#xA;\end{align*}&#xA;the pressure term is often referred to as a Lagrange multiplier enforcing the incompressibility condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In what sense is this true? Is there a formulation of the incompressible Navier-Stokes equations as an optimization problem subject to the incompressiblity constraint? If so, is there a numerical analog in which the equations of incompressible fluid flow are solved within an optimization framework?&lt;/p&gt;&#xA;" OwnerUserId="1748" LastActivityDate="2013-05-31T05:09:36.523" Title="Pressure as a Lagrange Multiplier" Tags="|optimization|fluid-dynamics|navier-stokes|incompressible|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7475" PostTypeId="1" CreationDate="2013-05-21T17:30:41.717" Score="7" ViewCount="690" Body="&lt;p&gt;Consider the semi-classical laser driving two-level atom, where the laser is treated classically and the atom is treated quantum mechanically. The effect of laser on the atom is a dipole coupling:&#xA;$$&#xA;i\,\dot\Psi(t)=[-\frac{\omega_0}{2}\sigma_z+\Omega(t)\cos(\omega t)\sigma_x]\Psi(t)&#xA;$$&#xA;where $\Psi(t)$ is the wavefunction, $\omega_0$ is the two level frequency, $\omega$ is the driving frequency, $\Omega(t)$ is the coupling strength. $\sigma_x,\sigma_z$ are the Pauli matrixes.&#xA;In matrix form this can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;i \left(&#xA;\begin{array}{c}&#xA; \dot C_1(t) \\&#xA; \dot C_2(t) \\&#xA;\end{array}&#xA;\right)=\left(&#xA;\begin{array}{cc}&#xA; -\frac{\omega _0}{2} &amp;amp; \Omega(t)\cos(\omega t)\\&#xA; \Omega(t)\cos(\omega t) &amp;amp; \frac{\omega _0}{2} \\&#xA;\end{array}&#xA;\right)\cdot&#xA;\left(&#xA;\begin{array}{c}&#xA; C_1(t) \\&#xA; C_2(t) \\&#xA;\end{array}&#xA;\right)&#xA;$$&#xA;where $C_1(t)$ and $C_2(t)$ are the state amplitude for the two states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the coupling $\Omega(t)$ is weak and $\omega_0\approx\omega$ then this can be solved using the Rotating-Wave Approximation.&#xA;I'm intrested in the region where $\Omega$ not small compared to $\omega_0$ or $\omega$, and $\Omega(t)$ has a shape &#xA;$$&#xA;\Omega(t)= \left\{&#xA;        \begin{array}{ll}&#xA;             \Omega_0\sin^2(\pi t/\tau) &amp;amp; 0\leq t&amp;lt;\tau\\[1em]&#xA;             0 &amp;amp; \tau \leq t&#xA;        \end{array}&#xA;    \right.&#xA;$$&#xA;where $\tau&amp;gt;&amp;gt;\frac{2\pi}{\omega}$.&#xA;If the coupling $\Omega(t)$ is time independent, then we can use the standard &lt;a href=&quot;http://en.wikipedia.org/wiki/Floquet_theory&quot;&gt;Floquet&lt;/a&gt; theory to solve the equation. But if $\Omega(t)$ is time-dependent, how do I use the Floquet formalism to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="3997" OwnerDisplayName="xslittlegrass" LastActivityDate="2014-04-28T18:32:24.773" Title="How to numerically solve a laser driving semi-classical two-level system using Floquet formalism?" Tags="|quantum-mechanics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7476" PostTypeId="2" ParentId="7474" CreationDate="2013-05-31T05:09:36.523" Score="21" Body="&lt;p&gt;This is most easily seen by considering the stationary Stokes equations&#xA;$$&#xA;  -\mu \Delta u + \nabla p = f \\&#xA;  \nabla \cdot u = 0&#xA;$$&#xA;which is equivalent to the problem&#xA;$$&#xA;  \min_u \frac\mu 2 \|\nabla u\|^2 - (f,u) \\&#xA;  \text{so that} \; \nabla\cdot u = 0.&#xA;$$&#xA;If you write down the Lagrangian and then the optimality conditions of this optimization problems, you will find that indeed the pressure is the Lagrange multiplier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equivalence between problems is not exploited in any numerical scheme (that I know of) but it is an important tool in the analysis because it shows that the Stokes equations are essentially the Poisson equation on a linear subspace. The same holds true for the time-dependent Stokes equations (which corresponds to the heat equation on the subspace) and it can be extended to the Navier-Stokes equations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-05-31T05:09:36.523" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7477" PostTypeId="2" ParentId="7467" CreationDate="2013-05-31T07:46:56.280" Score="4" Body="&lt;p&gt;&lt;strong&gt;Frequency separations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest way to estimate the large separation (without fitting the individual frequencies) is to take the autocorrelation of the power spectrum and find the maximum. That's a start. To find the small separation, you'd be looking for a second peak. Anyway, have a look at the autocorrelation of the pwer spectrum and see if you can see the separations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try to find the maxima of the autocorrelation, but I suspect the numerical derivatives will make it hard to find the correct zero-point. And, if you actually fit the frequencies, then you can take the average of the differences directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Units of the power spectrum&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can follow the units through the calculation of the objects you're working with. Suppose your time-series has units $x$. The Fourier transform is a sum of products of the time-series with something like $e^{i\omega}$, which is dimensionless. So the elements of the Fourier transform have units $x$ too. Finally, the power series is like the transform squared, so it has units $x^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you end up with weird units this way, note that in asteroseismology, people usually divide their signal my the mean (or median) and subtract one, so that have a dimensionless number. They usually express the power series in units of parts per million (ppm).&lt;/p&gt;&#xA;" OwnerUserId="2103" LastActivityDate="2013-05-31T07:46:56.280" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7478" PostTypeId="1" AcceptedAnswerId="7485" CreationDate="2013-05-31T09:22:47.313" Score="0" ViewCount="64" Body="&lt;p&gt;kernel LPP is of form:&#xA;$$\min_{\alpha} \  \alpha^{T}KLK\alpha&#xA;\\&#xA;s.t. \  \alpha^{T}KDK\alpha = 1$$&#xA;and it eventually results in solving generalized eigenvalue problem below:&#xA;$$KLK \alpha= \lambda KDK \alpha$$&#xA;It has same eigenvalues as:&#xA;$$Ly=\lambda Dy$$&#xA;so here I solve the above eigen system knowing that:&#xA;$$y=K \alpha$$&#xA;Notice the objective value of original kernel LPP is:&#xA;$$\alpha^{T}KLK\alpha=\lambda\alpha^{T}KDK\alpha=\lambda y^{T}Dy$$&#xA;Does it mean that changing K won't affect the objective value of KLPP? Because here I get eigenvalues $\lambda$ and eigenvectors $y$ without using K.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any answers will be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="4302" LastActivityDate="2013-06-01T03:29:16.357" Title="a question about kernelized locality preserving projections" Tags="|machine-learning|spectral-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7479" PostTypeId="1" AcceptedAnswerId="7921" CreationDate="2013-05-30T15:24:55.703" Score="6" ViewCount="138" Body="&lt;p&gt;Suppose I have a function which divides an $m$-bit unsigned integer $a$ by an $n$-bit unsigned integer $b$ and returns the quotient as a fixed-point number with $t$ fractional bits, truncating towards zero. So I have $f(a,b)= 2^{-t}\lfloor 2^t\cdot(a/b)\rfloor$, with $0 \leq a &amp;lt; 2^m$ and $0 &amp;lt; b &amp;lt; 2^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the smallest $t$ which will guarantee that $f(a_1,b_1)=f(a_2,b_2)$ only if $a_1/b_1=a_2/b_2$? That is, how many fractional bits do I need to preserve the ordering?&lt;/p&gt;&#xA;" OwnerUserId="4459" OwnerDisplayName="Ben" LastEditorUserId="127" LastEditDate="2013-05-31T10:18:00.860" LastActivityDate="2013-07-08T22:52:59.637" Title="How many bits to unambiguously represent fixed-point division?" Tags="|computer-arithmetic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7480" PostTypeId="1" AcceptedAnswerId="7482" CreationDate="2013-05-31T11:53:47.250" Score="13" ViewCount="398" Body="&lt;p&gt;I am trying to solve Ideal MHD equation using semi discrete methods, ENO spatial reconstructions and TVD RK time stepping. I am getting different steady state solutions with different temporal order. Is it correct?&lt;/p&gt;&#xA;" OwnerUserId="3640" LastActivityDate="2013-05-31T12:48:45.303" Title="CFD: Does order of time stepping scheme affect steady state solution? If yes why?" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7481" PostTypeId="2" ParentId="7475" CreationDate="2013-05-31T12:17:14.323" Score="0" Body="&lt;p&gt;You have two coupled differential equations with complex variables. You can use differential equation solvers here. There are two ways to handle such problems numerically:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Define suitable function with two inputs- a complex array of size 2 and time; and return the RHS of the ODE system. Pass this function to a suitable ODE solver that can handle complex numbers. For an example in Python using scipy take a look at the code at the bottom of this page: &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html&quot; rel=&quot;nofollow&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of complex array of size 2 use real array of size 4 and explicitly calculate the RHS of the ODE as 4 real equations. Then pass the function to the solver. Again an example in Python is at: &lt;a href=&quot;http://www.scipy.org/Cookbook/Zombie_Apocalypse_ODEINT&quot; rel=&quot;nofollow&quot;&gt;http://www.scipy.org/Cookbook/Zombie_Apocalypse_ODEINT&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that in both cases the RHS can explicitly depend on time. You can use matlab or any other ODE solver for this problem.&lt;/p&gt;&#xA;" OwnerUserId="4016" LastActivityDate="2013-05-31T12:17:14.323" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7482" PostTypeId="2" ParentId="7480" CreationDate="2013-05-31T12:48:45.303" Score="13" Body="&lt;p&gt;If the semi-discrete model has a unique steady state and no limit cycles, then all convergent methods should converge to the same steady state. If the problem has multiple steady states (hysteresis) or stable limit cycles, then &lt;em&gt;any&lt;/em&gt; change to the method (RK method, initial time step, and even non-associativity of floating point when summing stages) can change the computed steady state.  Since any convergent method approximates the constant solution correctly, the differences are in how you got there.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-05-31T12:48:45.303" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7483" PostTypeId="1" AcceptedAnswerId="7484" CreationDate="2013-05-31T18:17:03.460" Score="3" ViewCount="473" Body="&lt;p&gt;I have a gas (assuming air) at $T$ = 500 K that enters a cylindrical pipe. The outlet target temperature is 330 K.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will be heat transfer via: Forced convection from the gas to the inside of the pipe, conduction through the pipe thickness, convection to the ambient air environment ($T_\infty=300$ K) and radiation from the pipe surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think I can assume a constant surface temperature to get a relatively accurate answer. The way I was thinking of solving this was to assume an inner and outer radius, split the pipe into segments, first segment from 500 K to 470 K, compute all the relative non-dimensional parameters and solve for heat loss per unit length in that particular section. Then I would advance to the next segment, from 470 K to 440 K, solve for the non-dimensional parameters and find $q$ per unit length for that segment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure, however, how to determine the total length of the pipe in the end, given the heat loss per unit length in each section, since the heat loss is not really linear.&lt;/p&gt;&#xA;" OwnerUserId="4134" LastEditorUserId="276" LastEditDate="2013-06-04T06:33:27.780" LastActivityDate="2013-06-04T06:33:27.780" Title="Heat transfer in pipe" Tags="|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7484" PostTypeId="2" ParentId="7483" CreationDate="2013-06-01T03:27:21.927" Score="6" Body="&lt;p&gt;Here's what I would do to solve this problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Set up a differential equation that governs the temperature in the pipe. Judging from what you've said, it seems reasonable to assume that for any cross-section of the pipe, the temperature will be assumed homogeneous, so your problem has one spatial dimension.&lt;/li&gt;&#xA;&lt;li&gt;Unless you need to model the temperature of the pipe surface separately (which requires another separate -- probably differential -- equation), I'd just treat both the conduction and convection terms using a source term governing heat loss, like $h(T - T_{\infty})$, where $h$ is a heat transfer coefficient. You can model thermal radiation using a Stefan-Boltzmann-law-like relationship (that is, $\sigma A (T^{4} - T_{\infty}^{4})$, where $A$ will be the cross sectional surface area). However, if your inlet temperature is 500 K, and your ambient temperature is 300 K, radiative heat loss is probably negligible over the length and time scales you're considering, compared to conductive and convective heat losses (not to mention fluid advection within the pipe).&lt;/li&gt;&#xA;&lt;li&gt;Your differential equation will be in terms of one spatial dimension and have one initial condition (ignore the boundary condition at the end of the pipe for now). So you have an initial value problem. Solve it using an ODE solver, and pick an arbitrary end of the domain -- you want it to be longer than the end of the pipe. A good ODE solver will take care of all of the discretization for you, so you won't need to worry about splitting the pipe into segments.&lt;/li&gt;&#xA;&lt;li&gt;Plot the solution you got from the previous step. The temperature in the pipe should decrease monotonically going along the pipe away from its inlet. The temperature should never go below 300 K; wherever you find the temperature is at 330 K, that should be the length of your pipe. If the temperature does not fall below 330 K, then solve the differential equations over a larger domain (that is, increase the right end point of your integration interval) until there is a point where the temperature is below 330 K.&lt;/li&gt;&#xA;&lt;li&gt;If you really need the exact length of the pipe, use an ODE solver with event location, and set the event function such that it triggers the ODE solver to stop when the temperature reaches 330 K.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-06-01T06:25:29.897" LastActivityDate="2013-06-01T06:25:29.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7485" PostTypeId="2" ParentId="7478" CreationDate="2013-06-01T03:29:16.357" Score="1" Body="&lt;p&gt;Yes, you are correct. The minimum of the objective function is the minimal eigenvalue of the generalized eigenproblem $Ly = \lambda Dy$, regardless of $K$, at least as long as $K$ has full column rank.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-01T03:29:16.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7486" PostTypeId="2" ParentId="7450" CreationDate="2013-06-01T05:00:14.190" Score="1" Body="&lt;p&gt;The most robust way is solving it implicitly, otherwise for stiff nonlinearity in the source function $s(u)$ you will have to use very small time step to maintain numerical stability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_t = s(t,u)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u^{n+1}  = u^n + dt \; s(t+dt, u^{n+1})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here one can use average between n and n+1 in the RHS to increase the order, or use more temporal points for even higher order in time. But to make it simple let's assume first order implicit time-stepping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the equation can be cast in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${F(u^{n+1})}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be solved by the Newton method, and  for the discretized system &#xA;doing linear solves with the Krylov subspace is the best known approach, so this is the Newton-Krylov method.&lt;/p&gt;&#xA;" OwnerUserId="4325" LastActivityDate="2013-06-01T05:00:14.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7487" PostTypeId="2" ParentId="1681" CreationDate="2013-06-01T06:20:55.677" Score="8" Body="&lt;p&gt;There has been some good research on this recently. The new approaches use &quot;randomized algorithms&quot; which only require a few reads of your matrix to get good accuracy on the largest eigenvalues. This is in contrast to power iterations which require several matrix-vector multiplications to reach high accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more about the new research here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://math.berkeley.edu/~strain/273.F10/martinsson.tygert.rokhlin.randomized.decomposition.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://math.berkeley.edu/~strain/273.F10/martinsson.tygert.rokhlin.randomized.decomposition.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/0909.4061&quot; rel=&quot;noreferrer&quot;&gt;http://arxiv.org/abs/0909.4061&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code will do it for you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cims.nyu.edu/~tygert/software.html&quot; rel=&quot;noreferrer&quot;&gt;http://cims.nyu.edu/~tygert/software.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bitbucket.org/rcompton/pca_hgdp/raw/be45a1d9a7077b60219f7017af0130c7f43d7b52/pca.m&quot; rel=&quot;noreferrer&quot;&gt;https://bitbucket.org/rcompton/pca_hgdp/raw/be45a1d9a7077b60219f7017af0130c7f43d7b52/pca.m&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://code.google.com/p/redsvd/&quot; rel=&quot;noreferrer&quot;&gt;http://code.google.com/p/redsvd/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cwiki.apache.org/MAHOUT/stochastic-singular-value-decomposition.html&quot; rel=&quot;noreferrer&quot;&gt;https://cwiki.apache.org/MAHOUT/stochastic-singular-value-decomposition.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your language of choice isn't in there you can roll your own randomized SVD pretty easily; it only requires a matrix vector multiplication followed by a call to an off-the-shelf SVD.&lt;/p&gt;&#xA;" OwnerUserId="979" LastActivityDate="2013-06-01T06:20:55.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7488" PostTypeId="1" AcceptedAnswerId="7489" CreationDate="2013-06-01T19:19:45.453" Score="4" ViewCount="1831" Body="&lt;p&gt;Let $A$ be a general symmetric matrix operator and $P$ be the unique orthogonal projection onto $\operatorname{Range}(A) = \operatorname{Null}(A)^\perp$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analytically, the system &#xA;$$Ax = Pb$$&#xA;should have a solution for any vector $b$. If we further stipulate that this solution lie in $\operatorname{Range}(A)$, the solution should be unique. (And in the positive definite case, this is precisely the solution that the conjugate gradient algorithm will find in exact arithmetic.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above is a common tactic when solving the Poisson equation with either Neumann or periodic boundary conditions; if $A$ is taken to be the Neumann or periodic Laplacian, then &#xA;$$ P = I - \frac{1}{N}(1_N)(1_N)^T,$$&#xA;where $1_N = [1, \cdots, 1]^T$, and we can solve $Ax = Pb$ using a conjugate gradient approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if the projection operator $P$ doesn't have a simple analytic expression? Can it be calculated numerically from $A$? Is there a way to solve this type of singular problem in the case that $A$ is a black-box symmetric operator? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize one approach would be to solve the normal equations $A^2x = Ab$, which avoids using a projection operator entirely, but this also severely worsens the conditioning of the system, so I'm interested to know if there's another way.&lt;/p&gt;&#xA;" OwnerUserId="1748" LastEditorUserId="1748" LastEditDate="2013-06-01T19:36:21.380" LastActivityDate="2013-06-02T05:42:06.377" Title="Null Space Projection for Singular Systems" Tags="|linear-algebra|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7489" PostTypeId="2" ParentId="7488" CreationDate="2013-06-01T19:50:08.750" Score="3" Body="&lt;p&gt;Computing the null space without any further knowledge is expensive in general.  There are some iterative methods that converge to minimum-norm solutions even when presented with inconsistent right hand sides.  &lt;a href=&quot;http://www.stanford.edu/group/SOL/software/minresqlp/MINRESQLP-SISC-2011.pdf&quot; rel=&quot;nofollow&quot;&gt;Choi, Paige, and Saunders' MINRES-QLP&lt;/a&gt; is a nice example of such a method.  For non-symmetric problems, see &lt;a href=&quot;http://www.math.kent.edu/~reichel/publications/bfgmres.pdf&quot; rel=&quot;nofollow&quot;&gt;Reichel and Ye's Breakdown-free GMRES&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, usually some characterization of the null space is important for effective preconditioning.  Since most practical problems require preconditioning, the purely iterative methods have seen limited adoption.  Note that in case of very large null space, preconditioners will often be used in an auxiliary space where the null space has been removed.  See the &quot;auxiliary-space Maxwell&quot; methods for more details.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-06-01T19:50:08.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7490" PostTypeId="2" ParentId="7488" CreationDate="2013-06-02T05:42:06.377" Score="2" Body="&lt;p&gt;Jed has already answered your question, implicitly assuming that your matrix is large and sparse. If this should not be the case, you can do an eigenvalue decomposition and write $A=X\Lambda X^T$ where the columns of $X$ correspond to the eigenvectors and the entries of the diagonal matrix $\Lambda$ are the eigenvalues. (Note that this decomposition is not easily computable if $A$ is large, as Jed mentioned.) Then, the null space consists of exactly those vectors spanned by the eigenvectors corresponding to zero (or very small) eigenvalues, i.e., the projector onto the null space can be written as&#xA;$$&#xA;  1_N = X \tilde\Lambda X^T&#xA;$$&#xA;where $(\tilde\Lambda)_{ii} = 0$ if $|\Lambda_{ii}|\ge \varepsilon$ and $(\tilde\Lambda)_{ii} = \Lambda_{ii}$ if $|\Lambda_{ii}|&amp;lt;\varepsilon$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-02T05:42:06.377" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7491" PostTypeId="1" CreationDate="2013-06-02T09:40:32.007" Score="3" ViewCount="994" Body="&lt;p&gt;I have the following semi-definite programming problem that I want to put in a standard form in order to estimate its order of complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\max_{x_{i,j}}\sum_{i \in \mathbf{F} }^{ }\sum_{j \in \mathbf{F} }^{ } w_{i,j}^{+}x_{i,j}+w^{-}\left(1- x_{i,j}\right )&#xA;$$&#xA;subject to the constraints&#xA;$$&#xA;C1: x_{i,i}=1, \forall i\in \mathbf{F} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C2: x_{i,j}+x_{j,k}-x_{i,k} \leq 1, \forall i,j,k \in \mathbf{F}:k&amp;gt;i, j \neq i,k &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C3: \sum_{j \in F}^{ }x_{i,j} \leq M, \forall i \in \mathbf{F}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C4:x_{i,j} \geq 0, \forall i,j \in \mathbf{F}  &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C5: X=\left(x_{i,j} \right )\succeq 0 &#xA;$$&#xA;where, $M \in R^+$ and $X$ is a PSD $(F \times F)$ symmetric matrix.&#xA;I want to put it in a standard equality form to be able to know number of equality constraints $m$ and the final matrix dimensions of $X$ which is $n$ to estimate the complexity of solving the problem using $O((mn^3+m^2n^2+m^3)\sqrt{n}\log\frac{1}{\epsilon})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you and Best Regards,&lt;/p&gt;&#xA;" OwnerUserId="4467" LastEditorUserId="3481" LastEditDate="2013-06-02T21:03:15.173" LastActivityDate="2013-06-02T21:03:15.173" Title="How to put following SDP probllem into an equality standard form" Tags="|semidefinite-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7493" PostTypeId="2" ParentId="7491" CreationDate="2013-06-02T16:32:07.720" Score="4" Body="&lt;p&gt;The standard approach to handling linear inequality constraints is add nonnegative slack variables to each of the inequality constraints to make them equality constraints and expand your $X$ matrix to a block diagonal matrix in which the nonnegative slack variables appear as 1x1 blocks on the diagonal.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In computational practice it's important to take advantage of the ability of SDP solvers to exploit this block diagonal structure rather than simply making $X$ larger. Assuming that you just want to show that the SDP can be solved in polynomial time, you should be content to just expand the size of $X$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, all that you need to do here is count the number of linear inequality constraints to get the number of slack variables (that will increase the size of $X$), and count the number of resulting equality constraints (to get $m$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C1: $F$ equality constraints, no added slack variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C2: $C(F,2)(F-2)$ equality constraints with $C(F,2)(F-2)$ slack variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C3: $F$ equality constraints, with $F$ slack variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C4: $F^{2}$ equality constraints, with $F^{2}$ slack variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C5: $X \succeq 0$ is already in standard form.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding this up, you end up with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$m=F+C(F,2)(F-2)+F+F^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n=F+C(F,2)(F-2)+F+F^{2}$&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-06-02T16:32:07.720" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7494" PostTypeId="1" AcceptedAnswerId="7496" CreationDate="2013-06-02T16:57:33.707" Score="8" ViewCount="3337" Body="&lt;p&gt;I am running a mesh independence study. I start with &lt;b&gt;Mesh 1&lt;/b&gt; and proceed up to &lt;b&gt;Mesh 4&lt;/b&gt;, each time doubling the number cells in the mesh. In parallel, I am comparing my computational results to experimental data. &lt;b&gt;M. 1&lt;/b&gt; shows poor results. &lt;b&gt;M. 2&lt;/b&gt; shows a significant improvement and a good match to experimetnal results. &lt;b&gt;M. 3&lt;/b&gt; and &lt;b&gt;M. 4&lt;/b&gt; produce identical results, that are only slightly different from &lt;b&gt;M. 2&lt;/b&gt;. It would then seem sensible to pick &lt;b&gt; M. 3&lt;/b&gt; as my final mesh. &lt;i&gt; But&lt;/i&gt; it seems that the results are too smooth, loosing some of the fine details, produced by &lt;b&gt;M. 2&lt;/b&gt; (and observed in the experiments).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can there actually be some sort of &lt;i&gt; overcorrection&lt;/i&gt;? Could it be that the mesh independent solution is not necesarily the best one?&lt;/p&gt;&#xA;" OwnerUserId="4469" LastActivityDate="2013-06-02T18:47:43.323" Title="Mesh Independence Study: Is the Independent Mesh Really the Best One?" Tags="|fluid-dynamics|mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7495" PostTypeId="2" ParentId="7491" CreationDate="2013-06-02T18:16:45.023" Score="2" Body="&lt;p&gt;An SDP in standard form is defined by data the matrices $C$ and $A_i$ and scalars $b_i$  ($i=1\ldots m$), parameterizing the primal problem $minimize~C\bullet Z$ subject to $A_i\bullet Z = b_i, Z\succeq 0$ and the dual problem $maximize~b^T$ subject to $C-\sum A_i y_i \succeq 0$. Most solvers today are primal-dual, i.e., they compute $Z$ and $y$ simultaneously. Hence, when facing a problem, it is important to understand that you have an option in how you interpret your model. I have chosen to not call any of the variables $X$ or $x$ to emphasize this freedom you have, don't let the name fool you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, trying to extract data from the description to fit the primal form (standard equality as you call it) is not necessarily the best choice if you target a standard solver. Instead, when you have many ($O(F^2)$ or more) linear inequalities, it could very well be that you should try to look at your model as something describing the dual. Indeed, $X\succeq 0$ can be seen as a linear matrix inequality $C-\sum A_i y_i$ defined using $F(F+1)/2$ variables, i.e, $y$ corresponds to the elements defining a symmetric matrix. That means your model has complexity $m=F(F+1)/2$, which Brian calls $C(F,2)$. Notably smaller than what you would have obtained trying to fit the model to a standard equality form (primal form). These variables are used in a model with one LMI constraint of size $F$, and a bunch of linear inequalities (total number given by the same analysis as Brian counts when he sums up the number of slacks required, plus the fact that your $F$ equalities have to be written using $2F$ linear constraints).&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-06-02T18:16:45.023" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7496" PostTypeId="2" ParentId="7494" CreationDate="2013-06-02T18:30:32.623" Score="10" Body="&lt;p&gt;No. As the mesh size $h\rightarrow 0$, the solution on a given mesh converges to the solution of the differential equation (assuming a well-posed PDE and a suitable discretization). Consequently, if your discrete solution on M2 is closer to experimental results than that on M3 or M4, then there are only two possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The differential equation does not appropriately describe the real world because its solution is not close to what you measured.&lt;/li&gt;&#xA;&lt;li&gt;Your measured data are not correct.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Essentially, what you are seeing is that the exact solution of the differential equation (apparently well approximated on meshes M3 and M4) does not match your measurements. Which one is wrong is for you to find out now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(All this was written under the assumption that your numerical scheme is indeed correct and, thus, that your numerical solution on fine meshes is a good approximation to the exact solution.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-02T18:30:32.623" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7497" PostTypeId="2" ParentId="7494" CreationDate="2013-06-02T18:47:43.323" Score="7" Body="&lt;p&gt;I claim that the independent mesh &lt;em&gt;is&lt;/em&gt; the best one. Say the actual solution is $U$ and your solver delivers an $u_h$ depending on a mesh parameter $h$. Then you can do the estimate for the distance of $u_h$ to the actual solution&#xA;$$\|U-u_h \| \leq \|U-u_m \| + \|u_m - u_h\|,$$&#xA;where $u_m$ is the solution of the model used to describe the problem mathematically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume the modelling error $\| U - u_m\|$, that is independent of the mesh, is of constant size $C_m$, while the numerical error satisfies an estimate of type $\|u_m - u_h\| &amp;lt; C_h h^p$, with $p\in \mathbb N$. Thus, the error measured can be expressed as &#xA;$$\|U-u_h\| \approx C_m + C_hh^p.$$&#xA;Reducing the mesh size, i.e. $h\to 0$, there will be a point, when the numerical error goes below the modelling error, and the differences to the actual solution does not change anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point your solution is called mesh independent, since the modelling error dominates over the numerical error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, your observation is probably due to a lucky cancellation of errors. But I would not call on this, since I don't see a mathematical justification for using coarser discretizations, unless one faces instabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would rather consider a remodelling...&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-06-02T18:47:43.323" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7498" PostTypeId="1" AcceptedAnswerId="7500" CreationDate="2013-06-02T22:27:42.883" Score="3" ViewCount="4810" Body="&lt;p&gt;I have a big file with 3 columns: density, dimension, value.&lt;br&gt;&#xA;example:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;10 0.3 200  &#xA;10 0.4 300  &#xA;20 0.3 250  &#xA;20 0.4 320  &#xA;...  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to draw a 3d plot - mesh with &lt;code&gt;mesh()&lt;/code&gt; function in octave, like this:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data = load (&quot;file.txt&quot;);  &#xA;mesh(data(:,1), data (:,2), data (:,3));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Problem I have is , I always get error:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;rows (z) must be the same as length (y), columns (z) must be the same as length (x).&lt;/code&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It worked with function &lt;code&gt;plot3()&lt;/code&gt;, but I would like a mesh kind of plot.  &lt;/p&gt;&#xA;" OwnerUserId="4470" LastEditorUserId="10134" LastEditDate="2014-11-02T21:11:10.663" LastActivityDate="2014-11-02T21:11:10.663" Title="Octave 3D mesh, data from file" Tags="|matlab|octave|mesh|3d|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7500" PostTypeId="2" ParentId="7498" CreationDate="2013-06-03T01:41:44.547" Score="2" Body="&lt;p&gt;The &lt;code&gt;mesh&lt;/code&gt; function plots functions &lt;code&gt;z=f(x,y)&lt;/code&gt;. So to call the &lt;code&gt;mesh()&lt;/code&gt; function, you must have 2D data. You can give vectors for &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt;, but &lt;code&gt;z&lt;/code&gt; must be an array with &lt;code&gt;length(x)&lt;/code&gt; rows and &lt;code&gt;length(y)&lt;/code&gt; columns, or &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; and &lt;code&gt;z&lt;/code&gt; must be all be 2D arrays of the same size. Your data has been pulled out into a single, long vector which you need to two-dimensionalize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the &lt;code&gt;reshape&lt;/code&gt; function will do what you need. E.g. something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;x=reshape(data(:,1),m,n);&#xA;y=reshape(data(:,2),m,n);&#xA;z=reshape(data(:,3),m,n);&#xA;mesh(x,y,z);&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;m&lt;/code&gt; and &lt;code&gt;n&lt;/code&gt; are the appropriate dimensions in &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; of your data.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-03T01:41:44.547" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7501" PostTypeId="2" ParentId="5488" CreationDate="2013-06-03T04:11:30.030" Score="5" Body="&lt;p&gt;For the level set equation, $\phi=0$ denotes your interface and which you initialize suitably. Then based on your convection equation, you move $\phi$ (depends on how you want to solve it, if using FEM, then you need the weak formulation etc.) and again find  $\phi=0$ which would give you the location of your convected interface. However, the level set function gets smeared out after some iterations. Hence, you may have to re-initialize it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find this tutorial to be relatively easier to understand: &lt;a href=&quot;http://persson.berkeley.edu/pub/persson05levelset.pdf&quot; rel=&quot;nofollow&quot;&gt;http://persson.berkeley.edu/pub/persson05levelset.pdf&lt;/a&gt; Hope it helps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the FEM framework, Chapter 5 of this &lt;a href=&quot;http://www.nada.kth.se/utbildning/grukth/exjobb/rapportlistor/2006/rapporter06/lindbo_dag_06153.pdf&quot; rel=&quot;nofollow&quot;&gt;thesis&lt;/a&gt; should be helpful. &lt;/p&gt;&#xA;" OwnerUserId="4364" LastEditorUserId="4364" LastEditDate="2013-06-03T08:46:09.070" LastActivityDate="2013-06-03T08:46:09.070" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7502" PostTypeId="2" ParentId="5488" CreationDate="2013-06-03T06:50:48.267" Score="6" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: Yes, $\phi=0$ always locates the interface even at later times. See below for more details:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A level set function, $\phi(x,y,t)$, in two dimensions, describes a function whose zero-level set describes your interface. To be precise, a level-set function $\phi$ divides your domain $\Omega$ into three disjoint subdomains:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\{&#xA;\begin{array}{lcr}&#xA;\Omega^- &amp;amp;:(x,y,t) \:&amp;amp;|\: &amp;amp; \phi(x,y,t) &amp;lt; 0 \\&#xA;\Gamma   &amp;amp;:(x,y,t) \:&amp;amp;|\: &amp;amp; \phi(x,y,t) = 0 \\&#xA;\Omega^+ &amp;amp;:(x,y,t) \:&amp;amp;|\: &amp;amp; \phi(x,y,t) &amp;gt; 0 \\&#xA;\end{array}&#xA;\right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that $\phi(x,y,t) = 0$ &lt;strong&gt;always&lt;/strong&gt; denotes the interface $\Gamma$, it is clear that the the interface must satisfy:&#xA;$$&#xA;\frac{D \phi}{D t} = \frac{\partial \phi}{\partial t} + \mathbf{V} \cdot \nabla \phi = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, since $\phi(x,y,t) = 0$ on $ \Gamma$, this equation may be written to only involve the velocity in the &lt;strong&gt;normal&lt;/strong&gt; direction &#xA;$$&#xA;\frac{\partial \phi}{\partial t} + v_n \frac{\partial \phi}{\partial n} = 0&#xA;$$&#xA;However, since $\Gamma$ is a level-set, we have the following relation for the unit normal:&#xA;$$&#xA;\hat{\mathbf{n}} = \frac{\nabla \phi}{\left|\nabla \phi\right|}&#xA;$$&#xA;Thus, the evolution equation simplifies to:&#xA;$$&#xA;\frac{\partial \phi}{\partial t} + v_n \left|\nabla \phi\right| = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is your version of the equation with $F = v_n$. It is sometimes referred to as &lt;strong&gt;motion in the normal direction&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize, the zero level-set of $\phi(x,y,t)$ always locates the interface when you solve the level-set equation (either in its original form or normal motion form). I suggest you consult the following two standard texts for more details and actual implementation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521645573&quot; rel=&quot;noreferrer&quot;&gt;Sethian's book&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387954821&quot; rel=&quot;noreferrer&quot;&gt;Osher &amp;amp; Fedkiw's book&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one includes a chapter on weak formulation and FEM methods for the solution whereas the second is mainly involved with FDM methods.&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="485" LastEditDate="2013-06-03T06:58:15.437" LastActivityDate="2013-06-03T06:58:15.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7504" PostTypeId="1" AcceptedAnswerId="7511" CreationDate="2013-06-03T09:27:15.990" Score="6" ViewCount="460" Body="&lt;p&gt;Is there an rigorous proof of ENO-WENO schemes being non oscillatory?&lt;/p&gt;&#xA;" OwnerUserId="3640" LastActivityDate="2013-06-06T19:31:30.543" Title="ENO-WENO Schemes: Are ENO-WENO schemes non oscillatory for all kinds (linear/non linear) of problems?" Tags="|fluid-dynamics|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7505" PostTypeId="1" CreationDate="2013-06-03T10:01:19.527" Score="3" ViewCount="468" Body="&lt;p&gt;I am looking for a simple method to compute potential flow (then non-viscous) around an obstacle. The method I am looking for must NOT use coordinate transformation, panel methods, finite element methods; it should work just on a standard cartesian grid and it should preferably use the finite differences method. There is no need for it to be efficient or fast, it should only be SIMPLE. It is only for demonstration purposes.&#xA;Thank you.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked at the Shortley-Weller algorithm, but I didn't understand what should I do exactly... can it be used to impose Neumann conditions on the obstacle boundary (even an asymmetric arbitrary boundary)? If yes, how can it be done? &lt;/p&gt;&#xA;" OwnerUserId="4471" LastEditorUserId="-1" LastEditDate="2013-06-04T09:15:28.270" LastActivityDate="2013-06-04T09:15:28.270" Title="Simple CFD method" Tags="|finite-difference|fluid-dynamics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7506" PostTypeId="2" ParentId="7505" CreationDate="2013-06-03T12:00:35.793" Score="3" Body="&lt;p&gt;I think the easiest way would be to use a Shortley-Weller finite difference scheme. In particular if you know a simple representation of your obstacle (e.g., a circle), it should be easy to identify the nodes near and at the boundary. You then only need to modify the stencils near the boundaries and use normal finite differences in the interior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a detailed description, you can for instance refer to &lt;a href=&quot;http://books.google.de/books?id=vPjn5hImtEYC&quot; rel=&quot;nofollow&quot;&gt;Hackbusch's book on Elliptic Differential Equations&lt;/a&gt; (Chapter 4.8)&lt;/p&gt;&#xA;" OwnerUserId="4260" LastActivityDate="2013-06-03T12:00:35.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7508" PostTypeId="1" CreationDate="2013-06-03T14:07:32.197" Score="9" ViewCount="1847" Body="&lt;p&gt;can you give me a link for a good and simple explanation of the Shortley-Weller finite-difference scheme? I tried to google it but all I get is (inaccessible) academic publications. I also tried reading the dedicated chapter (4.8) in the Wolfgang Hackbusch's book  &quot;Elliptic Differential Equations&quot; but I found it rather difficult. Thank you&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To Christian Clason: thank you for you answer , but there is a thing I still don't understand: what do I have to do to apply this method be to arbitrary boundaries, for example an asymmetric airfoil in a flow? &lt;/p&gt;&#xA;" OwnerUserId="4472" LastEditorUserId="-1" LastEditDate="2013-06-04T16:48:21.557" LastActivityDate="2013-06-04T16:48:21.557" Title="Shortley-Weller finite difference method" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7509" PostTypeId="2" ParentId="7508" CreationDate="2013-06-03T14:40:41.490" Score="9" Body="&lt;p&gt;As far as I can tell, this scheme just consists in replacing the uniform finite difference stencil near the boundary with a non-uniform stencil (with at least one point shifted to lie on the boundary). Basically, you take your arbitrarily shaped domain, put it in a box, discretize the box with a uniform grid, throw away all grid points that do not have at least one neighbor inside the domain, and shift the remaining grid points outside the domain either horizontally or vertically (whichever is shortest) so that they lie on the boundary. (The actual implementation is much more tedious, of course.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain the non-uniform stencil at one of the nodes next to a boundary node, one proceeds similarly to (one of) the derivations of the uniform stencil: Interpolate the (unknown) function by a quadratic polynomial in the nodes and take the second derivative. It suffices to consider the one-dimensional case with the nodes $x_1=x-h_1,x_2=x,x_3=x-h_2$. Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D^2_h u(x) \approx u(x-h_1)\ell''_1(x) + u(x) \ell_2''(x) + u(x+h_2)\ell''_3(x),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\ell_j=\Pi_{i\neq j}(x-x_i)/(x_j-x_i)$ are the Lagrange polynomials corresponding to the nodes. Computing the derivatives yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D^2_h u(x) = \frac{2}{h_1(h_1+h_2)} u(x-h_1) - \frac{2}{h_1h_2} u(x) + \frac{2}{h_2(h_1+h_2)} u(x+h_2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as claimed. (You can also use the Newton form of the interpolating polynomial, which simplifies computing the derivatives, especially for higher orders.) Doing the same in $y$ and summing the stencils gives equation (4.8.7).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find more detailed examples in Randy LeVeque's &lt;a href=&quot;http://books.google.com/books?id=qsvmsXe8Ug4C&amp;amp;pg=PA9&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Finite Difference Methods for Ordinary and Partial Differential Equations&lt;/em&gt; (e.g., page 9)&lt;/a&gt;, or on &lt;a href=&quot;http://www.scientificpython.net/1/post/2012/04/non-uniform-first-order-finite-differences.html&quot; rel=&quot;noreferrer&quot;&gt;this blog post&lt;/a&gt; (which also contains NumPy code for computing the coefficients given arbitrary $h_1$ and $h_2$). This is also treated in detail in Morton and Mayers, &lt;a href=&quot;http://books.google.com/books?id=0uf_iQIBFKgC&amp;amp;lpg=PP1&amp;amp;vq=page%2071&amp;amp;hl=de&amp;amp;pg=PA71&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Numerical solution of Partial Differential Equations&lt;/em&gt;&lt;/a&gt;, section 3.4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How you treat the boundary nodes depends on your boundary conditions. For Dirichlet conditions, you proceed as you would for a uniform mesh. For Neumann conditions, you use the above approach (non-uniform interpolation -- now simultaneously in $x$ and $y$ -- and differentiation) to approximate the normal derivative at the boundary node to get a local stencil; see Morton and Mayers, page 75ff.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-06-04T15:32:58.507" LastActivityDate="2013-06-04T15:32:58.507" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7510" PostTypeId="1" CreationDate="2013-06-03T18:10:49.037" Score="2" ViewCount="161" Body="&lt;p&gt;For the ODE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\dot{x} = f(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have the 2-stage, second order SSP RK scheme (Shu, Osher, Gottlieb)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x^{(0)} = x^n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x^{(1)} = x^{(0)} + \Delta t f(x^{(0)})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x^{(2)} = \frac{1}{2} x^n + \frac{1}{2}[ x^{(1)} + \Delta t f(x^{(1)})]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x^{n+1} = x^{(2)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the scheme in case of the following ODE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\dot{x} = f(x,t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular I am looking for papers which deal with this case.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2013-06-03T18:10:49.037" Title="Strong stability preserving RK scheme" Tags="|time-integration|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7511" PostTypeId="2" ParentId="7504" CreationDate="2013-06-03T18:28:52.980" Score="8" Body="&lt;p&gt;Depends on what you mean by 'non-oscillatory'. In their original paper, &lt;a href=&quot;http://www.jstor.org/stable/2157557&quot; rel=&quot;nofollow&quot;&gt;Harten &amp;amp; Osher (1987)&lt;/a&gt;, design the ENO schemes with the 'non-oscillatory' property defined as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this paper we construct a uniformly second-order approximation, which is &lt;strong&gt;nonoscillatory in the sense that the number of extrema of the discrete solution is not increasing in time&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They also provide proof in the paper (pp. 281). In this sense, and formally, when an ENO scheme is used for interpolations to compute numerical fluxes in solving Hyperbolic Conservation Laws of the form:&#xA;$$&#xA;\frac{\partial u}{\partial t} + \frac{\partial}{\partial x} F(u) = 0,&#xA;$$&#xA;it is guaranteed that the final numerical solution will have equal or less number of extrema when compared to the exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, this is achieved by using an adaptive stencil that is constructed in a hierarchical fashion to avoid interpolating across non-smooth regions (e.g. large gradients, shocks, etc.). Note that ENO schemes are just one of the ingredients needed for a numerical solution to converge to the viscosity solution of nonlinear PDEs such as Hamilton-Jacobi or Hyperbolic Conservation Laws. Other ingredients include&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use of monotone and consistent numerical flux (or Hamiltonian)&lt;/li&gt;&#xA;&lt;li&gt;A TVD integrator in time&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For more details please consult the aforementioned paper. I could add more references including those that discuss implementations if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why TVD integrators? Note that ENO (or WENO) only quarantines that you do not interpolate across discontinuities. It assumes nothing about what happens to the solution in time. To ensure that the solution remains non-oscillatory, time integration should have the TVD property. TV of a discrete data-set is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;TV(u^n) = \sum_{i=0}^N \left|u^n_i - u^n_{i-1}\right|&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A time integration scheme is said to be TVD if satisfies the following property:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;TV(u^{n+1}) \le TV(u^n)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that for a time integration scheme, such as RK methods, to be TVD, they need to satisfy certain bounds on the CFL number. This is a bg topic in itself and I suggest you consult the following papers for accurate discussions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://linkinghub.elsevier.com/retrieve/pii/0021999189902222&quot; rel=&quot;nofollow&quot;&gt;Efficient implementation of essentially non-oscillatory shock-capturing schemes (Shu, C.W. and&#xA;Oshert, S)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.aip.org/link/SJOCE3/v9/i6/p1073/s1&amp;amp;Agg=doi&quot; rel=&quot;nofollow&quot;&gt;Total-Variation-Diminishing Time Discretizations (C. W. Shu)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ams.org/journal-getitem?pii=S0025-5718-98-00913-2&quot; rel=&quot;nofollow&quot;&gt;Total variation diminishing Runge-Kutta schemes (S. Gottlieb and&#xA;C. W. Shu)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="485" LastEditorUserId="485" LastEditDate="2013-06-06T19:31:30.543" LastActivityDate="2013-06-06T19:31:30.543" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7512" PostTypeId="1" AcceptedAnswerId="7514" CreationDate="2013-06-03T19:16:09.687" Score="4" ViewCount="505" Body="&lt;p&gt;I am trying to gather literature material to study how people propose to implement generic frameworks to solve partial differential equations in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite my effort to search the web, the only thing I found so far is the &lt;a href=&quot;http://www.foamcfd.org/Nabla/guides/ProgrammersGuide.html&quot; rel=&quot;nofollow&quot;&gt;Programmer's Guide&lt;/a&gt; of OpenFOAM, which in chapter 1 and 2 gives an idea of the mapping between mathematical entities and C++ classes in order to create a domain specific language for the solution of partial differential equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to have a broader view of what has been proposed so far, I would like to ask the experts out there:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are the current state of the art references in this field?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion ranging from a journal paper to an actual library documentation will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="305" LastActivityDate="2013-06-19T01:01:56.460" Title="Literature material discussing generic frameworks to solve PDEs" Tags="|pde|c++|libraries|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7513" PostTypeId="2" ParentId="7512" CreationDate="2013-06-03T19:27:04.270" Score="5" Body="&lt;p&gt;I recommend looking at different library docs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://fenicsproject.org/book/&quot;&gt;http://fenicsproject.org/book/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.dealii.org/developer/&quot;&gt;http://www.dealii.org/developer/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30" LastActivityDate="2013-06-03T19:27:04.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7514" PostTypeId="2" ParentId="7512" CreationDate="2013-06-03T19:27:57.813" Score="7" Body="&lt;p&gt;While there are some books on the implementation of finite element methods (e.g., Mark Gockenbach's &lt;a href=&quot;http://www.ec-securehost.com/SIAM/OT97.html&quot;&gt;&lt;em&gt;Understanding and Implementing the Finite Element Method&lt;/em&gt;&lt;/a&gt; and Yair Shapiro's &lt;a href=&quot;http://www.ec-securehost.com/SIAM/CS09.html&quot;&gt;&lt;em&gt;Solving PDEs in C++: Numerical Methods in a Unified Object-Oriented Approach&lt;/em&gt;&lt;/a&gt;), I think it would indeed be most useful to look at (and compare!) actual large-scale libraries:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There are a few &lt;a href=&quot;http://www.dealii.org/developer/publications/index.html#details&quot;&gt;publications on the details&lt;/a&gt; of &lt;code&gt;deal.II&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a &lt;a href=&quot;http://fenicsproject.org/book/&quot;&gt;book&lt;/a&gt; on &lt;code&gt;FEniCS&lt;/code&gt; including chapters about the design and implementation of the components, which are also described in various &lt;a href=&quot;http://fenicsproject.org/citing/&quot;&gt;papers&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;References on the design of &lt;code&gt;DUNE&lt;/code&gt; can be found &lt;a href=&quot;http://www.dune-project.org/publications.html&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(The main developers of the first two are on this list, and are sure to expand on this.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-06-03T19:41:01.890" LastActivityDate="2013-06-03T19:41:01.890" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7515" PostTypeId="1" AcceptedAnswerId="7517" CreationDate="2013-06-03T19:50:24.090" Score="3" ViewCount="1084" Body="&lt;p&gt;I am solving an elliptic boundary value problem on a subset of the rectangle [-1,1]x[-1,1]. The domain contains the line segment x=0, however this does not need to be a part of the boundary, so it is not aligned with the facets by default. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to compute the flux of a solution function across the line segment x=0. I know from the tutorial how to compute fluxes over boundary elements. However, I am not sure whether making the line a boundary element is necessary in this case. Nor am I sure how to insert it, since simply inserting the line into the mesh will cause degrees of freedom to conflict, as would naively gluing rectangles together. Can you offer any suggestions on how to compute this flux?&lt;/p&gt;&#xA;" OwnerUserId="4438" LastEditorUserId="4438" LastEditDate="2013-06-03T21:20:58.877" LastActivityDate="2013-06-06T02:34:46.537" Title="Flux across a non-boundary line segment in FEniCS" Tags="|pde|finite-element|fenics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7516" PostTypeId="2" ParentId="7505" CreationDate="2013-06-03T19:55:02.987" Score="1" Body="&lt;p&gt;I recommend the book&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical Simulation in Fluid Dynamics: A Practical Introduction&#xA;Volume 3 of Monographs on Mathematical Modeling and Computation&#xA;Authors: Michael Griebel, Thomas Dornsheifer, Tilman Neunhoeffer&lt;/p&gt;&#xA;" OwnerUserId="30" LastActivityDate="2013-06-03T19:55:02.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7517" PostTypeId="2" ParentId="7515" CreationDate="2013-06-03T21:50:30.113" Score="5" Body="&lt;p&gt;Make sure that the mesh is aligned with x = 0. Then mark the facets along that line and compute the flux using an &quot;interior facet integral&quot;. You can do this by supplying a facet function &lt;code&gt;markers&lt;/code&gt; that (e.g.) marks those facets by &lt;code&gt;1&lt;/code&gt; and the remaining facets by &lt;code&gt;0&lt;/code&gt;. Then compute the flux by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;line_segment = AutoSubDomain(lambda x: near(x[0], 0))&#xA;markers = FacetFunction(&quot;size_t&quot;, mesh)&#xA;markers.set_all(0)&#xA;line_segment.mark(markers, 1)&#xA;dS = dS[markers]&#xA;flux = assemble(&amp;lt;expression for flux&amp;gt;*dS(1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you use a built-in mesh like &lt;code&gt;RectangleMesh&lt;/code&gt;, it is easy to make sure that the mesh is aligned with the line segment. If you use an external mesh generator, it should have support for respecting your line segment when meshing.&lt;/p&gt;&#xA;" OwnerUserId="359" LastActivityDate="2013-06-03T21:50:30.113" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7518" PostTypeId="1" AcceptedAnswerId="7519" CreationDate="2013-06-03T22:43:53.610" Score="3" ViewCount="311" Body="&lt;p&gt;I'm trying to solve for electron and hole distribution function using Boltzmann equation with various scattering mechanisms. Since I land up with an integro-differential equation, analytical solution is very difficult. So, I am looking for general advice on where to start, and on how this problem could be solved using numerical methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you suggest some references with illustrative examples that would be really useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4477" LastEditorUserId="3691" LastEditDate="2013-06-03T23:37:26.783" LastActivityDate="2013-06-03T23:46:25.757" Title="Is using Monte Carlo method a good approach for solving Boltzmann equation?" Tags="|simulation|monte-carlo|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7519" PostTypeId="2" ParentId="7518" CreationDate="2013-06-03T23:46:25.757" Score="6" Body="&lt;p&gt;A fantastic introductory book (final year undergraduate to graduate level) for simulation of electron and holes in semiconductor is, &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521637244&quot; rel=&quot;noreferrer&quot;&gt;Fundamentals of Carrier Transport by M. Lundstrom&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XziUj.jpg&quot; alt=&quot;Fundamentals of Carrier Transport front cover&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book covers difference techniques of solving the Boltzmann transport equation and it has a chapter on Monte-Carlo. It should provide you with an overview of the relevant techniques, hopefully this will get you started or provide you with references where you need more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prof. Lundstrom's lectures are also published on &lt;a href=&quot;http://nanohub.org/members/2862/contributions&quot; rel=&quot;noreferrer&quot;&gt;nanohub.org&lt;/a&gt;, I would recommend viewing the &lt;a href=&quot;http://nanohub.org/search?terms=ECE%20656&quot; rel=&quot;noreferrer&quot;&gt;ECE 656 lecture series&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2013-06-03T23:46:25.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7523" PostTypeId="1" CreationDate="2013-06-04T13:23:40.180" Score="2" ViewCount="173" Body="&lt;p&gt;Well, in a numerical technique test we were given the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A physical phenomenon is modeled such that,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F(f,d) = A(f)/d^2 + L$;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where, $F$ is a function of frequency $f$ and distance $d$, and $L$ is random error.&#xA;Function $A(f)$ is unknown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A function is given, to produce data set $(F,f,d)$. The function is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function F = get_data(d,f)&#xA;    for i= 1:length(f)&#xA;        for j = 1:length(d)&#xA;            F(i,j) = 2*f(i)^2 / d(j)^2 + 0.3*rand()*(1-rand());&#xA;        end&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;here d, f are arrays. F is a 2D array. rand is a function that gives some random numbers less than 1.&#xA;It is assumed that, we cannot poke into this function for an answer. We can't just look at the function and say A(f) = 2*f^2. We have to find this or a very close approximation to this by some numerical method....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for&#xA;d = 1:1:10;&#xA;f = 30:30:300;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have to produce a data set for $F$.&#xA;From, this data set, we have to calculate,&#xA;(1) mean of $L$.&#xA;(2) an expression for $A(f)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what I understand is that, 2nd if we have $A(f)$ then we can produce the values of $L$, and can easily evaluate mean of $L$. And, for $A(f)$ we could assume $L$ to be too low that we can ignore......&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, how could I actually do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll be very happy if someone helped me.&lt;/p&gt;&#xA;" OwnerUserId="4483" LastEditorUserId="4244" LastEditDate="2013-06-04T14:11:30.383" LastActivityDate="2013-06-08T12:32:39.557" Title="How to solve this numerical technique problem?" Tags="|matlab|stochastic|inverse-problem|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7524" PostTypeId="2" ParentId="7338" CreationDate="2013-06-04T17:46:44.110" Score="0" Body="&lt;p&gt;To run a shell script (such as &lt;code&gt;dolfin-convert&lt;/code&gt;) from python, you have to pass the necessary command line to the shell via &lt;code&gt;os.system()&lt;/code&gt;, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;os.system('dolfin-convert carbon_f.msh mesh.xml')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4486" LastEditorUserId="1804" LastEditDate="2013-06-08T08:46:40.427" LastActivityDate="2013-06-08T08:46:40.427" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7525" PostTypeId="1" AcceptedAnswerId="29271" CreationDate="2013-06-04T18:06:13.167" Score="16" ViewCount="1370" Body="&lt;p&gt;I'm looking for a mesh generation software that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is free and open source,&lt;/li&gt;&#xA;&lt;li&gt;provides a sane scripting interface for domain specification,&lt;/li&gt;&#xA;&lt;li&gt;works for complex geometries,&lt;/li&gt;&#xA;&lt;li&gt;can generate 2D and 3D meshes,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What options do I have?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="1552" LastEditDate="2013-06-05T13:15:48.140" LastActivityDate="2018-08-13T16:15:09.397" Title="Scripted Mesh Generation Software" Tags="|mesh-generation|" AnswerCount="7" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7526" PostTypeId="2" ParentId="7525" CreationDate="2013-06-04T18:10:19.893" Score="10" Body="&lt;p&gt;I would recommend you look at &lt;a href=&quot;http://geuz.org/gmsh/&quot;&gt;gmsh&lt;/a&gt;.  It has both text and CAD-like input, capable of 2D and 3D, higher order meshes.  It is licensed under the GPL, so there are some restrictions on integrating it into closed source software, but is otherwise completely free/open-source.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-06-04T18:10:19.893" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7527" PostTypeId="2" ParentId="7338" CreationDate="2013-06-04T22:01:11.813" Score="2" Body="&lt;p&gt;To see which mesh types dolfin-convert can handle see the command line help:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ dolfin-convert &#xA;Usage: dolfin-convert [OPTIONS] ... input.x output.y&#xA;&#xA;Options:&#xA;&#xA;  -h         display this help text and exit&#xA;  -i format  specify input format&#xA;  -o format  specify output format&#xA;&#xA;Alternatively, the following long options may be used:&#xA;&#xA;  --help     same as -h&#xA;  --input    same as -i&#xA;  --output   same as -o&#xA;&#xA;Supported formats:&#xA;&#xA;  xml      - DOLFIN XML mesh format (current)&#xA;  xml-old  - DOLFIN XML mesh format (DOLFIN 0.6.2 and earlier)&#xA;  mesh     - Medit, generated by tetgen with option -g&#xA;  Triangle - Triangle file format (input prefix of .ele and .node files)&#xA;  gmsh     - Gmsh, version 2.0 file format&#xA;  metis    - Metis graph file format&#xA;  scotch   - Scotch graph file format&#xA;  diffpack - Diffpack tetrahedral grid format&#xA;  abaqus   - Abaqus tetrahedral grid format&#xA;  ExodusII - Sandia Format (requires ncdump utility from NetCDF)&#xA;  Star-CD  - Star-CD tetrahedral grid format&#xA;&#xA;If --input or --output are not specified, the format will&#xA;be deduced from the suffix:&#xA;&#xA;  .xml  - xml&#xA;  .mesh - mesh&#xA;  .gmsh - gmsh&#xA;  .msh  - gmsh&#xA;  .gra  - metis&#xA;  .grf  - scotch&#xA;  .grid - diffpack&#xA;  .inp  - abaqus&#xA;  .e    - Exodus II&#xA;  .exo  - Exodus II&#xA;  .ncdf - ncdump'ed Exodus II&#xA;  .vrt and .cell  - starcd&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To call dolfin mesh conversion utilities from python, try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin_utils import meshconvert&#xA;&#xA;# Convert to XML&#xA;meshconvert.convert2xml(ifilename, ofilename, iformat=iformat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but then you have to order the mesh yourself (or call the command dolfin-order)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Order mesh&#xA;os.system(&quot;dolfin-order %s&quot; % ofilename)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30" LastActivityDate="2013-06-04T22:01:11.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7528" PostTypeId="2" ParentId="7525" CreationDate="2013-06-04T22:07:01.780" Score="4" Body="&lt;p&gt;I usually use &lt;a href=&quot;http://tetgen.berlios.de/&quot; rel=&quot;nofollow&quot;&gt;tetgen&lt;/a&gt; for 3D (MIT license for research/non-commercial) and &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow&quot;&gt;triangle&lt;/a&gt; for 2D (Custom license free for non-commercial).  To script them, you write a input file and call the command line.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="30" LastEditDate="2013-06-05T13:44:08.293" LastActivityDate="2013-06-05T13:44:08.293" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7529" PostTypeId="1" CreationDate="2013-06-05T06:33:34.263" Score="0" ViewCount="35" Body="&lt;p&gt;I plan to implement some optimization algorithms in Python, say to follow the sequence &lt;a href=&quot;http://www.scientificpython.net/uploads/1/1/5/9/11598566/lecturenotes1.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387303030&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be a systematic approach to testing the algorithms I write, using either well-known test datasets or creating my own?&lt;/p&gt;&#xA;" OwnerUserId="1969" LastActivityDate="2013-06-05T06:33:34.263" Title="Data to test optimization algorithms" Tags="|optimization|python|" AnswerCount="0" CommentCount="3" ClosedDate="2013-06-05T08:12:30.680" ContentLicense="CC BY-SA 3.0" />
  <row Id="7530" PostTypeId="1" AcceptedAnswerId="7533" CreationDate="2013-06-05T08:01:09.927" Score="14" ViewCount="35480" Body="&lt;p&gt;I'm looking into moving some computations of mine to a data center to get more computation power. In the context of this process, I am getting confused by the differentiation of a &lt;strong&gt;computation node&lt;/strong&gt; and a &lt;strong&gt;CPU&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently some commercial software packages differentiate between the two. Is it correct to imagine each &lt;strong&gt;computation&lt;/strong&gt; node as a different physical computer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A link to a tutorial on this &lt;em&gt;for dummies&lt;/em&gt; would be highly welcome. Evidently, googling this topic provides me with a lot of tech lingo that is cumbersome to decrypt.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2021-12-05T22:08:46.720" Title="Difference between Nodes and CPUs when running software on a cluster?" Tags="|reference-request|hpc|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7531" PostTypeId="2" ParentId="7530" CreationDate="2013-06-05T10:33:05.080" Score="7" Body="&lt;p&gt;Many modern computers have multiple CPUs (chips) and each chip may have multiple cores. Each core can execute one (or sometimes multiple) stream of instructions.&lt;/p&gt;&#xA;&lt;p&gt;I've recorded an introduction to these sorts of issues here:&#xA;&lt;a href=&quot;https://www.math.colostate.edu/%7Ebangerth/videos.676.39.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.math.colostate.edu/~bangerth/videos.676.39.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="23791" LastEditDate="2021-12-05T22:08:46.720" LastActivityDate="2021-12-05T22:08:46.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7532" PostTypeId="2" ParentId="7479" CreationDate="2013-06-05T14:42:50.970" Score="1" Body="&lt;p&gt;I don't have a theoretical backing for this, I'm afraid, but based on brute-force testing I've found that $t = 2n$ never produces collisions (for $n$ up to $20$), whereas $t = 2n-1$ always does.&lt;/p&gt;&#xA;" OwnerUserId="4459" LastActivityDate="2013-06-05T14:42:50.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7533" PostTypeId="2" ParentId="7530" CreationDate="2013-06-05T14:51:28.570" Score="19" Body="&lt;p&gt;This issue has become much more nuanced as the changes in architectures has shifted the HPC landscape. As Wolfgang Bangerth mentions one current longstanding view, I'll split my answer into basic definitiions and further details.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Basic Definition&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A &lt;strong&gt;node&lt;/strong&gt; refers to the physical box, i.e. &lt;strong&gt;cpu sockets&lt;/strong&gt; with north/south switches connecting memory systems and extension cards, e.g. disks, nics, and accelerators&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A &lt;strong&gt;cpu socket&lt;/strong&gt; is the connector to these systems and the &lt;strong&gt;cpu cores&lt;/strong&gt;, you plug in chips with multiple cpu cores.  This creates a split in the cache memory space, hence the need for NUMA aware code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A &lt;strong&gt;cpu core&lt;/strong&gt; is an independent computing with its own computing pipeline, logical units, and memory controller. Each cpu core will be able to service a number of &lt;strong&gt;cpu threads&lt;/strong&gt;, each having an independent instruction stream but sharing the cores memory controller and other logical units. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This notion of &lt;strong&gt;node&lt;/strong&gt; and &lt;strong&gt;cpu core&lt;/strong&gt; gets you through most HPC queuing systems, but note many HPC centers will charge you &quot;Service Units&quot; which is a variable rate dependent on aspects of the node.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Going further&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you are interested in actually giving some performance details of a distributed code, this story is a bit more troublesome.  Let me put it in terms of questions this model doesn't answer:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How many &lt;em&gt;cores&lt;/em&gt; is a GPU accelerator?  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The GPU has very small processors with few logical units, so comparing them to an x86 cpu is not fair.  Nonetheless marketers will tell you that GPUs have 1000s of cpus.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cloud architectures are putting many traditional &lt;em&gt;nodes&lt;/em&gt; on a single physical server with integrated networking between them.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Companies like &lt;a href=&quot;http://www.calxeda.com/&quot;&gt;Calxeda&lt;/a&gt; are going around many of the inefficiencies in the current node configurations and your traditional node configuration is sharing many more systems.  This idea of node is becoming vague.&lt;/p&gt;&#xA;" OwnerUserId="30" LastEditorUserId="30" LastEditDate="2013-06-05T15:23:45.867" LastActivityDate="2013-06-05T15:23:45.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7534" PostTypeId="2" ParentId="6875" CreationDate="2013-06-05T15:14:58.930" Score="3" Body="&lt;p&gt;We have a robust algorithm for the Ensemble Kalman (and regular Kalman) filter.&#xA;It is well suited to sparse matrices and parallel computation because it is&#xA;based on orthogonal matrices and it related to the square root or UD algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would be glad to send the paper&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thomas, S. J., J. Hacker and J. Anderson, (2009):&#xA;&lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/qj.372/abstract&quot; rel=&quot;nofollow&quot;&gt;A robust formulation of the ensemble Kalman filter&lt;/a&gt;&#xA;Quart J. Royal Met. Soc, vol 135, 507-521,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(PDF from the publisher is &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/qj.372/pdf&quot; rel=&quot;nofollow&quot;&gt;free&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="4490" LastEditorUserId="761" LastEditDate="2013-06-06T00:50:23.550" LastActivityDate="2013-06-06T00:50:23.550" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7535" PostTypeId="1" CreationDate="2013-06-03T09:05:10.143" Score="8" ViewCount="191" Body="&lt;p&gt;The result I'm interested in is found within &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/052153352X&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Synchronization: A Universal Concept in Nonlinear Sciences&quot;&lt;/a&gt; page $333$ figure $14.3$. The peculiar fragment is also provided at the end of this post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically there is this dissipative coupling applied to one-dimensional array of initial conditions (horizontal axis) which evolves over time (vertical axis). I realize I won't be able to produce identical result since I don't know the exact initial conditions but that's not the point of this post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual problem is I am not sure how to apply the rule of evolution. If I have some initial conditions which undergo a single iteration the result is essentially a function of those initial conditions... What is this function (procedure) in question?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd love to be able calculate it in MatLab. There surely has to be somewhat related standard functions in there...&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fZbYD.png&quot; alt=&quot;Coupling&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4491" OwnerDisplayName="Pranasas" LastEditorUserId="4491" LastEditDate="2013-06-05T18:32:00.977" LastActivityDate="2014-03-26T18:47:28.783" Title="How to recreate this result (from a book)?" Tags="|matlab|parabolic-pde|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7536" PostTypeId="1" AcceptedAnswerId="7540" CreationDate="2013-06-05T18:00:07.100" Score="2" ViewCount="452" Body="&lt;h3&gt;Problem&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I'm building a simple MPI Send/Recv hello world in Fortran and am running into an odd error.  I want to make sure I'm not missing anything trivial before blaming OpenMPI or gfortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code below I send a simple array between two MPI processes.  The first few entries of the data aren't sent correctly.  The tail is fine.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sent      1   2   3   4   5   6   7   8&#xA;Received  0   0  32   0   5   6   7   8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I expect &lt;code&gt;sent&lt;/code&gt; to equal &lt;code&gt;received&lt;/code&gt; and it does so after the first few entries.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Environment&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;These results come from &lt;code&gt;gfortran-4.6.3&lt;/code&gt; and &lt;code&gt;openmpi-1.6.4&lt;/code&gt;.  A colleague ran this using &lt;code&gt;ifort 13.0.1&lt;/code&gt; and obtained correct results.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Code&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program main&#xA;    implicit none&#xA;    include 'mpif.h'&#xA;&#xA;    integer rank, size, ierr&#xA;&#xA;    call MPI_INIT( ierr )&#xA;    call MPI_COMM_RANK( MPI_COMM_WORLD, rank, ierr )&#xA;    call MPI_COMM_SIZE( MPI_COMM_WORLD, size, ierr )&#xA;    ! print *, 'Process ', rank, ' of ', size, ' is alive' &#xA;&#xA;    if (rank .eq. 0) then&#xA;        call sender()&#xA;    end if&#xA;    if (rank .eq. 1) then&#xA;        call recver()&#xA;    end if&#xA;    call MPI_FINALIZE( ierr )&#xA;end program&#xA;&#xA;subroutine sender()&#xA;    implicit none&#xA;    include 'mpif.h'&#xA;&#xA;    integer i, ierr&#xA;    integer dat(8)&#xA;    do i=1, 8&#xA;        dat(i) = i&#xA;    enddo&#xA;    call MPI_SEND( dat, 8, MPI_INTEGER, 1, 2001, MPI_COMM_WORLD, ierr )&#xA;    print *, &quot;Sent     &quot;, dat&#xA;&#xA;end subroutine&#xA;&#xA;subroutine recver()&#xA;    implicit none&#xA;    include 'mpif.h'&#xA;&#xA;    integer i, ierr&#xA;    integer stat&#xA;    integer dat(8)&#xA;&#xA;    dat = -1&#xA;    print *, &quot;Empty    &quot;, dat&#xA;    call MPI_RECV( dat, 8, MPI_INTEGER, 0, 2001, MPI_COMM_WORLD, stat, ierr )&#xA;    print *, &quot;Received &quot;, dat&#xA;&#xA;end subroutine&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19" LastActivityDate="2013-06-05T19:13:17.327" Title="MPI_Send/Recv not transmitting initial entries" Tags="|fortran|mpi|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7538" PostTypeId="2" ParentId="7536" CreationDate="2013-06-05T18:26:35.133" Score="2" Body="&lt;p&gt;You probably have a mismatch between your &lt;code&gt;dat&lt;/code&gt; integer's size and &lt;code&gt;MPI_INTEGER&lt;/code&gt;. I would be explicit about both declaring &lt;code&gt;dat&lt;/code&gt; with an size specification (4 bytes or 8 bytes) &lt;strong&gt;&lt;em&gt;and&lt;/em&gt;&lt;/strong&gt; which MPI data type you are using. E.g. something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;...&#xA;integer(kind=4) dat(8) &#xA;...&#xA;call MPI_RECV( dat, 8, MPI_INTEGER4, 0, 2001, MPI_COMM_WORLD, stat, ierr )&#xA;...&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or &lt;code&gt;kind=8&lt;/code&gt; and &lt;code&gt;MPI_INTEGER8&lt;/code&gt; if you need 64-bit types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I couldn't reproduce your error exactly as written because Fortran &lt;code&gt;integer&lt;/code&gt; and &lt;code&gt;MPI_INTEGER&lt;/code&gt; are the same size on my architecture, but by explicitly mismatching them, I was able to get spurious content at the &lt;strong&gt;&lt;em&gt;end&lt;/em&gt;&lt;/strong&gt; of the received data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that you have similar issue on your architecture.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-05T18:26:35.133" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7539" PostTypeId="2" ParentId="6875" CreationDate="2013-06-05T18:53:36.787" Score="0" Body="&lt;p&gt;Long time back I had a chance to work on dimensionality reduction which deals with processing data that falls in large sets. The basic idea behind it is that it processes the data through a few steps to orient it in the way that most information can be calculated from it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works pretty fine for matrices as well and is used largely. The best part is that you need not even program it as there are standard libraries available for it already. Major mathematical tools like Matlab and Mathematica also support this &lt;a href=&quot;http://reference.wolfram.com/mathematica/ref/PrincipalComponents.html&quot; rel=&quot;nofollow&quot;&gt;functionality&lt;/a&gt; straightforwardly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two main algorithms that achieve this - Principal Component Analysis and Singular Value Decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What these algorithms actually achieve is finding that data that actually affects your reading by a significant margin. The internet is full of information on these algorithms. &lt;a href=&quot;https://cwiki.apache.org/MAHOUT/dimensional-reduction.html&quot; rel=&quot;nofollow&quot;&gt;This&lt;/a&gt; will show you how Apache is doing it.&lt;/p&gt;&#xA;" OwnerUserId="3243" LastEditorUserId="1748" LastEditDate="2013-06-06T02:34:55.303" LastActivityDate="2013-06-06T02:34:55.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7540" PostTypeId="2" ParentId="7536" CreationDate="2013-06-05T19:13:17.327" Score="6" Body="&lt;p&gt;You seem to have the wrong declaration for stat. It must be declared as an array of size MPI_STATUS_SIZE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;integer stat(MPI_STATUS_SIZE)&lt;/p&gt;&#xA;" OwnerUserId="1453" LastActivityDate="2013-06-05T19:13:17.327" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7541" PostTypeId="1" AcceptedAnswerId="7559" CreationDate="2013-06-05T19:26:55.733" Score="4" ViewCount="150" Body="&lt;p&gt;Assume we have few terms contributing to element tensor and each requires different quadrature degree to be integrated exactly. Does it generally worth using different quadrature degrees for each term or should one use the single one with highest degree for all terms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would guess that for sufficiently large $N$ (number degrees of freedom) it does not matter as solver cost being $O(N^\alpha)$ with $\alpha&amp;gt;1$ beats assembler cost $O(N)$.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2013-06-07T02:24:14.100" Title="Assembling FE tensors using different quadrature degrees" Tags="|finite-element|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7542" PostTypeId="2" ParentId="5383" CreationDate="2013-06-05T20:32:39.963" Score="0" Body="&lt;p&gt;I would put a switch in the target function, and allow it to write to a file the parameters of interest then switch the output falsely such that the ode45 truncates the run, possibly with a NaN or other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you need some pseudocode?  If so could you provide pseudocode that has some of your info structure and a derivative function that you find useful?&lt;/p&gt;&#xA;" OwnerUserId="3979" LastActivityDate="2013-06-05T20:32:39.963" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7543" PostTypeId="2" ParentId="7515" CreationDate="2013-06-05T21:52:44.757" Score="1" Body="&lt;p&gt;As Anders said, the CSG functionality in Dolfin will be able to handle this  soon. If you want to try it out now, get the code here: &lt;a href=&quot;https://bitbucket.org/benjamik/dolfin&quot; rel=&quot;nofollow&quot;&gt;https://bitbucket.org/benjamik/dolfin&lt;/a&gt; and check out the branch called &lt;code&gt;benjamik/csg-2d&lt;/code&gt;. Take a look at &lt;code&gt;demo/undocumented/csg/2d/python/demo_csg_2d.py&lt;/code&gt;. The functionality is not documented yet except from this demo.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of remaining issues that need to be sorted out before merging this into the official dolfin repo, but none of them should affect your case.&lt;/p&gt;&#xA;" OwnerUserId="4480" LastEditorUserId="1748" LastEditDate="2013-06-06T02:34:46.537" LastActivityDate="2013-06-06T02:34:46.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7544" PostTypeId="2" ParentId="7530" CreationDate="2013-06-05T23:02:58.137" Score="3" Body="&lt;p&gt;Answering from a cluster perspective... You need to be careful as the definition can be dependent on the system configuration. Our system uses PBS and a &quot;node&quot; can either be a physical box or a single core &quot;virtual node&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On our system, if you request nodes=1 you get one core (which is one of 6 cores on a physical CPU (which is one of two CPUs on a physical box)). In PBS these are called &quot;virtual nodes&quot; and I think these may be the same as your &quot;computation node&quot;. If you requested nodes=12, then the cores &quot;could&quot; be allocated anywhere in the cluster (though the queuing system tries to put them on the same physical box).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you request nodes=1:ppn=2 you get two cores on a one physical node. The ppn refers to processors per node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some info I have developed here:&#xA;&lt;a href=&quot;https://www.massive.org.au/userguide/cluster-instructions/running-pbs-script-jobs&quot; rel=&quot;nofollow&quot;&gt;https://www.massive.org.au/userguide/cluster-instructions/running-pbs-script-jobs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2013-06-05T23:02:58.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7545" PostTypeId="1" CreationDate="2013-06-06T02:20:36.537" Score="13" ViewCount="1176" Body="&lt;p&gt;I am interested in implementing an moving mesh for an advection-diffusion problem. &lt;a href=&quot;http://www.springer.com/mathematics/computational+science+&amp;amp;+engineering/book/978-1-4419-7915-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adaptive Moving Mesh Methods&lt;/a&gt; gives a good example of how to do this for Burger's equation in 1D using finite-difference. Would someone be able to offer a worked example on solving the 1D advection-diffusion equation using finite-difference with a moving mesh?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in conservative form the equation is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = (a(x)u + du_x)_x&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a(x)$ is the velocity (a function of space). The initial conditions $u(0,x)$ could specify (for example) a flow species moving from left to right (e.g. along a pipe) where the initial condition has a sharp gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should the equidistribution problem for the moving mesh be solved (possibly with De Boor's algorithm or other approach)? I wish to implement this myself in Python so if your answer can be readily translated into code all the better!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Old question prior to bounty&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What are the basic approaches for generating an adaptive mesh based on properties of the system? Should I use flux as a measure of where the gradients are large?&lt;/li&gt;&#xA;&lt;li&gt;Because I seek an iterative (time sweep) solution. I imagine it is important to interpolate from the old grid to the new grid, what is the usual approach?&lt;/li&gt;&#xA;&lt;li&gt;I would be really interested to see a worked example for a simple problem (like the advection equation).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A bit of background about the specifics of the problem. I am simulating a 1D coupled system of equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial u}{\partial t} = a_u\frac{\partial^2 u}{\partial x^2} + b_u\frac{\partial u}{\partial x} + f_u(x,u,v,w) \\&#xA;\frac{\partial v}{\partial t} = a_v\frac{\partial^2 v}{\partial x^2} + b_v\frac{\partial v}{\partial x} + f_v(x,u,v,w) \\&#xA;\frac{\partial w}{\partial t} = a_u\frac{\partial u}{\partial x} +a_v\frac{\partial v}{\partial x} + f_w(x,u,v,w) \\&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The set of equations describe a two species advection-diffusion problem where the  third equation couples to the other two. The solution change rapidly near the centre of my grid, see below (these are illustration not calculations),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/iabMQ.png&quot; alt=&quot;Example solutions&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that log scale on the lower graph, the solutions for $u$ and $v$ vary over orders of magnitude. On the upper graph ($w$) there is a discontinuity at the centre. I am solving the above system with an adaptive upwind where the discretization can adapt from central to upwind dominated depending on local value of &lt;a href=&quot;http://en.wikipedia.org/wiki/P%C3%A9clet_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;Péclet number&lt;/a&gt;. I am solving the system implicitly with trapezoidal integration in time (&quot;Crank-Nicolson&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in applying an adaptive grid to this problem. I think it is important because otherwise details of the shape peak (the $w$) parameter could be lost. Unlike this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/861/is-there-a-general-purpose-library-for-structured-grid-adaptive-mesh-refinement&quot;&gt;question&lt;/a&gt;, I would like to apply, a hopefully simply, algorithm for mesh generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As this is an advection-diffusion problem, one could imagine an adaptive mesh scheme based on the &lt;strong&gt;fluxes&lt;/strong&gt; of $u$ and $v$ at the cell boundaries. As this would indicate where the value is changing rapidly. The peak of the $w$ also corresponds to the where the flux are the largest.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-06-08T13:43:04.050" Title="What are the basic principles behind generating a moving mesh?" Tags="|finite-difference|adaptive-mesh-refinement|advection-diffusion|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7546" PostTypeId="1" AcceptedAnswerId="7548" CreationDate="2013-06-06T08:04:06.553" Score="8" ViewCount="1830" Body="&lt;p&gt;I have COMSOL and the CFD package therein available, which provides me with an all-in-one solution for solving CFD problems. I have modeled and solved the geometry shown in the pictures below, which took me roughly one hour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now (just for kicks) I want to do the same in FEniCS and see how that goes. At this point, I am trying to establish an optimal work flow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach at the moment is this: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;create the geometry in &lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;noreferrer&quot;&gt;gmsh&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;create the mesh in gmsh&lt;/li&gt;&#xA;&lt;li&gt;import the mesh to dolfin using &lt;em&gt;dolfin-convert mesh_file.msh mesh_file.xml&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;set-up boundary conditions in dolfin&lt;/li&gt;&#xA;&lt;li&gt;solve navier-stokes equations&lt;/li&gt;&#xA;&lt;li&gt;post-processing in ParaView&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Before&lt;/strong&gt; I start doing that, I would like to know if this work-flow is a good idea?! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possible issues I see are setting up the boundary conditions from the imported gmsh mesh in dolfin, based on this &lt;a href=&quot;https://answers.launchpad.net/dolfin/+question/161172&quot; rel=&quot;noreferrer&quot;&gt;launchpad post&lt;/a&gt;. I.e. is it better to define the boundary conditions in gmsh or in dolfin?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Szwpo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q29lP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2013-06-06T08:25:03.843" Title="FEniCS CFD work-flow for complex geometries" Tags="|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7547" PostTypeId="1" CreationDate="2013-06-06T08:08:04.713" Score="4" ViewCount="2369" Body="&lt;p&gt;Which is the preferred way to import into a FEniCS (python) program a mesh which was generated in an external generator which provides sub-domain and boundary markers? I was using MeshData-&gt;MeshFunctions for that purpose, but switching to parallel mpi program I've found that those MeshFunctions are not transferred to  partial meshes used by mpi processes. MeshFunctions seem to be no longer supported - what is the alternative? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using geometry information to recover sub-domains is extremely inconvenient for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using my own (diffpack)grid to xml converter which is currently creating two MeshFunctions with subdomain and boundary information. Reading such xml Mesh file from a single-process code I'am able to get and use those MeshFunctions. However, when reading the same mesh from mpi code it is divided into sub-meshes and MeshFunctions are lost at this stage. My question is if may I create an xml mesh file with data which will not get lost during mesh sub-division.&lt;/p&gt;&#xA;" OwnerUserId="4496" LastEditorUserId="4244" LastEditDate="2013-06-06T14:28:34.387" LastActivityDate="2013-06-06T14:28:34.387" Title="FEniCS: importing mesh along with material and boundary data" Tags="|finite-element|fenics|mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7548" PostTypeId="2" ParentId="7546" CreationDate="2013-06-06T08:12:36.390" Score="7" Body="&lt;p&gt;&lt;a href=&quot;https://plus.google.com/110475963061639463862/posts/5NWw8dzrtst&quot;&gt;I've used&lt;/a&gt; this exact workflow for solving the Navier-Stokes equations in FEniCS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Almost certainly the hardest part in the process will be implementing the numerical solution method; there is no out-of-the-box support for that in FEniCS (although of course all differential operators you need are available).&#xA;For reference, check out &lt;a href=&quot;https://launchpad.net/nsbench&quot;&gt;nsbench&lt;/a&gt;, a Navier-Stokes benchmark suite written for FEniCS. It features a collection of common numerical solution methods and tests them against a number of example problems. Also check out &lt;a href=&quot;https://bitbucket.org/fenics-project/fenics-book&quot;&gt;the FEniCS book&lt;/a&gt;, chapter 21. Depending on the Reynolds number of the flow you intent to simulate, you may need to stabilize the numerical solution with, e.g., &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/0045782582900718&quot;&gt;SUPG&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing to COMSOL, using FEniCS for the task will definitely require a better understanding of the Navier-Stokes equations, numerical solution methods, and appropriate boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the specification of boundary conditions, I think &lt;code&gt;dolfin-convert&lt;/code&gt; still doesn't carry over the tag information from Gmsh meshes to the dolfin format. If your domain isn't too complicated, manually re-tagging the boundaries in FEniCS  should be doable though.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-06-06T08:25:03.843" LastActivityDate="2013-06-06T08:25:03.843" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7549" PostTypeId="2" ParentId="7547" CreationDate="2013-06-06T09:56:25.277" Score="3" Body="&lt;p&gt;The questions depends on what program you have used to generate the external meshes. Dolfin provides the script &lt;code&gt;dolfin-convert&lt;/code&gt;, which convert external meshes to dolfin meshes. Depending on what format you convert from you get different representations of the boundary condition within the mesh, because not all formats have been updated to the latest storage formats. As you point out &lt;code&gt;MeshFunctions&lt;/code&gt; stored in Mesh.data is no longer the preferred storage format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are able to save your mesh in &lt;code&gt;diffpack&lt;/code&gt; (.grid) format, I know that pipeline should be pretty updated. You should then be able to access the domains through:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh.domains().facet_domains()&#xA;mesh.domains().cell_domains()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;however this syntax has changed in the development version, where you need to explicitly create the &lt;code&gt;MeshFunction&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dim = mesh.topology().dim()&#xA;facet_domains = MeshFunction(&quot;size_t&quot;, mesh, dim-1, mesh.domains())&#xA;cell_domains = MeshFunction(&quot;size_t&quot;, mesh, dim, mesh.domains())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that mesh entities which has no marker will get the maximal number of the data type. &lt;/p&gt;&#xA;" OwnerUserId="4250" LastActivityDate="2013-06-06T09:56:25.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7550" PostTypeId="2" ParentId="7547" CreationDate="2013-06-06T12:56:47.413" Score="0" Body="&lt;p&gt;I've found that the MeshFunctions need to be stored in separate files, but now I'm getting &quot;Reading old-style XML MeshFunctions is not supported in parallel. Consider using the new format&quot; - will search for the specification of this new format. Programming FEniCS is definitively exciting, but some stability of the code would probably be nice ;-)&lt;/p&gt;&#xA;" OwnerUserId="4498" LastEditorUserId="4498" LastEditDate="2013-06-06T13:10:41.290" LastActivityDate="2013-06-06T13:10:41.290" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7552" PostTypeId="1" AcceptedAnswerId="7554" CreationDate="2013-06-06T16:08:29.657" Score="6" ViewCount="6856" Body="&lt;p&gt;I have a 3x3 covariance matrix (so, real, symmetric, dense, 3x3), I would like it's principal eigenvector, and speed is a concern. Is there a fast algorithm for this specific problem? I've seen algorithms for calculating all the eigenvectors of a real symmetric matrix, but those routines seem to be optimized for large matrices, and I don't care about the non-principal eigenvectors anyway. I've also seen the power method for calculating the principal eigenvector of a real matrix, but again, that method seems to me to be inefficient for small matrices. Maybe I'm over-stating the gains that I could get from knowing that my matrix is small? &lt;/p&gt;&#xA;" OwnerUserId="4188" LastEditorUserId="4188" LastEditDate="2013-06-06T16:13:40.957" LastActivityDate="2013-06-06T19:58:25.733" Title="Algorithm for Principal Eigenvector of a Real Symmetric 3x3 Matrix" Tags="|linear-algebra|algorithms|eigensystem|eigenvalues|dense-matrix|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7554" PostTypeId="2" ParentId="7552" CreationDate="2013-06-06T19:03:15.023" Score="3" Body="&lt;p&gt;There's this paper specialized for 3x3 matrices that gets very technical:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://pages.cs.wisc.edu/~sifakis/project_pages/svd.html&quot; rel=&quot;nofollow&quot;&gt;http://pages.cs.wisc.edu/~sifakis/project_pages/svd.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is code for it, but it's not simple at all. I would just stick to the power method, and write a specialized loop-unrolled matrix-vector multiplication routine, and call it a fixed number of times. If you can bound the maximum eigenvalue, you can probably just do re-normalization every $n$ iterations, instead of every iteration, which should save a lot of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can try the Jacobi eigenvalue algorithm, which is extremely simple, and you have only 3 off-diagonal elements to annihilate, so it should not require many iterations to converge. The problem is that this won't get you the eigenvector, so you'd have to do a separate post-processing step for that, like with a rank-revealing QR factorization, which is somewhat complex.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-06-06T19:03:15.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7555" PostTypeId="2" ParentId="7552" CreationDate="2013-06-06T19:13:15.820" Score="5" Body="&lt;p&gt;A Google search returns many relevant links for this problem since it is a common problems in mechanics. A simple algorithm for symmetric and real 3x3 matrices is presented on Wikipedia: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Eigenvalue_algorithm#3.C3.973_matrices&quot; rel=&quot;noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Eigenvalue_algorithm#3.C3.973_matrices&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3952" LastActivityDate="2013-06-06T19:13:15.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7556" PostTypeId="2" ParentId="7552" CreationDate="2013-06-06T19:58:25.733" Score="5" Body="&lt;p&gt;From geometrical tools (www.geometrictools.com)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.geometrictools.com/Documentation/EigenSymmetric3x3.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.geometrictools.com/Documentation/EigenSymmetric3x3.pdf&lt;/a&gt;&#xA;I hope this helps&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2013-06-06T19:58:25.733" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7558" PostTypeId="2" ParentId="7545" CreationDate="2013-06-07T02:09:48.967" Score="2" Body="&lt;p&gt;If it is indeed 1D then you probably won't need any adaptive mesh here, for such a simple problem you can probably resolve all you need with a static grid, with a computing power of a modern workstation. But it is a perfectly reasonable strategy, in the process of time-integration, to identify periodically areas where the numerical resolution is stressed, add grid points there (and remove grid points from over-resolved areas), and interpolate to the new grid. But this should not be done too frequently because interpolation can be costly, and it would add numerical error in the overall calculation.&lt;/p&gt;&#xA;" OwnerUserId="4325" LastActivityDate="2013-06-07T02:09:48.967" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7559" PostTypeId="2" ParentId="7541" CreationDate="2013-06-07T02:24:14.100" Score="4" Body="&lt;p&gt;Use the highest order you need for all terms. The major expense is evaluating the values and gradients of the shape functions at the quadrature points, which you do once and then reuse as often as is necessary. If you use multiple different quadrature formulas, you end up with more quadrature points at which you need to evaluate shape functions, for a larger overall expense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are exceptions to this rule (sometimes you do want to purposefully under-integrate certain terms) but I think it's still generally true.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-07T02:24:14.100" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7560" PostTypeId="2" ParentId="7545" CreationDate="2013-06-07T02:37:45.853" Score="3" Body="&lt;p&gt;I was (still am) looking for good answers for this. I work with multi-level adaptive grids where I use some sort of criterion for refinement. Folks doing FEM enjoy, rather cheap (computationally), rigorous error estimates that they use as refinement criterion. For us doing FDM/FVM, I have not had luck finding any such estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, if you want to be rigorous about refinement, i.e refine based on &lt;em&gt;some&lt;/em&gt; estimation of the actual error, your (almost) only choice is Richardson Extrapolation. This is what was, for instance, used by &lt;a href=&quot;http://linkinghub.elsevier.com/retrieve/pii/0021999184900731&quot; rel=&quot;nofollow&quot;&gt;Berger and Oliger (1984)&lt;/a&gt; for their block-structured, AMR hyperbolic solver. The methodology is general in the sense that you can use Richardson Extrapolation for virtually any problem. The only issue with it is that it is expensive, especially for transient problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than Richardson Extrapolation, all other criterion (in my humble opinion) are just ad hoc. Yes you can set a certain threshold on a &quot;quantity of interest&quot; and refine based on that. You could use fluxes or derivatives of some quantity to alert some large gradient and use that. Or if you are tracking an interface, you could refine based on how close you are to the interface. All of these are very cheap, of course, but there is nothing rigorous about them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for interpolation between grids, you generally need something that is at least as accurate as you solver. Sometimes it is possible to build interpolations that satisfy certain properties, e.g. conserve mass or are convex thus do not introduce new extrema. I have noted that this last property is sometimes very important to the stability of the overall scheme.&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-06-07T02:37:45.853" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7561" PostTypeId="1" AcceptedAnswerId="7563" CreationDate="2013-06-07T06:44:11.457" Score="10" ViewCount="3298" Body="&lt;p&gt;I use RBF kernel function to implement one kernel based machine learning algorithm(KLPP),&#xA;the resulting kernel matrix $K$ $$K(i,j)= \exp\left({\frac{-(x_{i}-x_{j})^2}{ \sigma_{m}^2}}\right)$$ &#xA;is shown to be extremely ill-conditioned.The condition number of L2-norm comes $10^{17}-10^{64}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to make it well-conditioned? I guess parameter $ \sigma$ needs to be tuned, but I don't know how exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4302" LastEditorUserId="4302" LastEditDate="2013-06-08T01:53:30.743" LastActivityDate="2014-02-16T16:12:18.510" Title="Do RBF kernel matrices tend to be ill-conditioned?" Tags="|linear-algebra|machine-learning|interpolation|support-vector-machines|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7563" PostTypeId="2" ParentId="7561" CreationDate="2013-06-07T09:07:32.333" Score="12" Body="&lt;p&gt;Reducing the kernel width $\sigma_m$ will usually reduce the condition number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, kernel matrices can become singular, or close to singular, for any basis function or point distribution, provided the basis functions overlap. The reason for this is actually quite simple:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The kernel matrix $K$ is singular when its determinant $\det(K)$ is zero.&lt;/li&gt;&#xA;&lt;li&gt;Swapping two points $x_i$ and $x_j$ in your interpolation is equivalent to exchanging two rows in $K$, assuming your trial points remain constant.&lt;/li&gt;&#xA;&lt;li&gt;Swapping two rows in a matrix switches the sign of its determinant.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now imagine picking two points $x_i$ and $x_j$ and slowly rotating them so that they switch places. While doing this, the determinant of $K$ will switch sign, becoming zero at some point in between. At this point, $K$ is, by definition, singular.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-06-07T09:07:32.333" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7565" PostTypeId="2" ParentId="7545" CreationDate="2013-06-07T10:39:32.427" Score="4" Body="&lt;p&gt;An adaptive grid is a grid network that automatically clusters grid points in regions of high flow field gradients; it uses the solution of the flow field properties to locate the grid points in the physical plane. The adaptive grid evolves in steps of time in conjunction with a time-dependent solution of the governing flow field equations, which computes the flow field variables in steps of time. During the course of the solution, the grid points in the physical plane move in such a fashion to ‘adapt’ for regions of large flow field gradients. Hence, the actual grid points in the physical plane are constantly in motion during the solution of the flow field, and become stationary only when the flow solution approaches a steady state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grid adaptation is used for both steady and unsteady type of problems. In case of steady flow problems grid is adapted after predetermined number of iterations and grid adaptation will stop at the point when solution is converged. In case of time accurate solutions the grid point motion and refinement are performed in conjunction with time accurate solution of the physical problem. This requires time accurate coupling of PDEs of the physical problem and those describing the grid movement or grid adaptation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the computations of newer configurations dependence on best practice guidelines for mesh generation and previous experience leaves the door open to large amounts of numerical error. Grid adaptation methods can produce substantial improvements in solution quality and promises better results because no limitations exist that define the limit on grid resolution that can be attained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three basic types of grid adaptation techniques, that are $h$-method, $r$-method and $p$-method. There are some some mixed types of approaches one can find, such as $rp$-adaptation or $hp$-adaptation. Out of this $r$ and $h$ type of grid adaptation techniques are more popular in finite volume and finite difference schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$h$ type:-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $h$ method involves automatic refinement or coarsening of the spatial mesh based on a posteriori error estimates or error indicators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r$ type:-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of making local topological changes to the mesh and its connectivity, r-adaptive methods make local changes to the resolution by moving the locations of a fixed total number of mesh points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p$ type:-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very much popular method of grid adaptation in finite element approach rather than finite volume or finite element method. It reduces the error in the solution by enrichment of the polynomial of interpolating functions with the same geometric element  order.Here no new mesh, geometry to be calculated and another advantage of this method is that it can better approximate irregular or curved boundaries with less sensitivity to aspect ratio and skew. Because of the this its very famous in structural application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Driving-sources-of-grid-adaptation$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$1. Feature-based-adaptation$&#xA;Feature based approximately largely used approach of grid adaptation employs feature of solution as driving force for grid adaptation. These often uses features of the solution such as solution gradients and solution curvature. Flow regions that have large solution gradients are resolved with more points and regions of minimal significance are coarsened. This leads to refinement of the region which is physically specific such as boundary layer, shocks,separation lines, stagnation points, etc. In some cases, gradient-based refinement can actually increase the solution error so there  are some issues regarding feature based adaptation like robustness and others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$2.Truncation-error-based-adaption$&#xA;Truncation error is the difference between partial differential equation and its discretized equation. Truncation error is more suitable approach to find where adaptation should occur. General concept behind truncation error based adaptation is to equidistribute the error over the domain of simulation to reduce total discretization error. For simple equations evaluation of truncation error is easiest job but for complex schemes its difficult so different approach is needed for that purpose. For simple discretization schemes, the truncation error can be computed directly. For more complex schemes where direct evaluation of the truncation is difficult, an approach for estimating the truncation error is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$3.Adjoint-based-adaptation$&#xA;Next promising approach is the adjoint approach. Its very good in estimating the local contribution of each cell or element to the discretization error  in any solution functionals of interest such as lift, drag, and moments. So it is useful in targeted grid adaptation as per requirement of the solution so thats its also called as goal oriented adaptation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the best!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$References:-$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Fidkowski Krzysztof J. and Darmofal David L. Review of output-based error es-timation and mesh adaptation in computational ﬂuid dynamics. AIAA Journal,&#xA;49:673–694, 2011.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] John Tannehill Richard Pletcher and Dale Anderson. Computational ﬂuid mechanics and heat transfer. Taylor &amp;amp; Francis, 1997.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] J. D. Jr. Anderson. Computational ﬂuid dyanamics: The basics with applications.McGraw Hill Inc., 1995.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] Roy Christopher J. Strategies for driving mesh adaptation in cfd. In 47th AIAA&#xA;Aerospace Sciences Meeting Including The New Horizons Forum and Aerospace Ex-&#xA;position, 2009.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] McRae Scott D. r-reﬁnement grid adaptation algorithms and issues. Computatioanal&#xA;methods in applied mechanics and engineering, 189:1161–1182, 2000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] Ivanenko Sergey A. Azarenok Boris N. and Tang Tao. Adapative mesh redistribution&#xA;method based on godunovs scheme. Comm. math. sci., 1:152–179.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[7] Ahmadi Majid and Ghaly Wahid S. Simulation of inviscid ﬂow in cascades using a&#xA;ﬁnite volume method with solution adaptation. In CASI 6th Aerodynamics Sympo-&#xA;sium, 1997.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[8] Jasak H. and Gosman A. D. Automatic resolution control for the ﬁnite-volum e&#xA;m ethod, part 1: a-posteriori error estimates. Numerical Heat Transfer,Taylor &amp;amp;&#xA;Francis, 38:237–256, 2000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[9] Jasak H. and Gosman A. D. Automatic resolution control for the ﬁnite-volum e m&#xA;ethod, part 2: Adaptive mesh reﬁnement and coarsening. Numerical Heat Trans-&#xA;fer,Taylor &amp;amp; Francis, 38:257–271, 2000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[10] Thompson David S. Soni Bharat K., Koomullil Roy and Thornburg Hugh. Solution&#xA;adaptive grid strategies based on point redistribution. Computatioanal methods in&#xA;applied mechanics and engineering, 189:1183–1204, 2000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[11] Venditti David A. and Darmofal David L. Adjoint error estimation and grid adap-&#xA;tation for functional outputs: Application to quasi-one-dimensional ﬂow. Journal of&#xA;Computational Physics, 164:204–227, 2000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[12] Balasubramanian R. and Newman J. C. Comparison of adjoint-based and feature-&#xA;based grid adaptation for functional outputs. International journal for numerical&#xA;methods in ﬂuids, 53:1541–1569, 2007.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[13] Hartmann Ralf. Error estimation and adjoint-based adaptation in aerodynamics. In&#xA;European Conference on Computational Fluid Dynamics, 2006.&lt;/p&gt;&#xA;" OwnerUserId="3549" LastEditorUserId="3549" LastEditDate="2013-06-08T07:35:08.403" LastActivityDate="2013-06-08T07:35:08.403" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7566" PostTypeId="1" AcceptedAnswerId="7567" CreationDate="2013-06-07T12:23:10.937" Score="11" ViewCount="5379" Body="&lt;p&gt;For model reduction, I want to compute the left singular vectors associated to the - say 20 - largest singular values of a matrix $A \in \mathbb R^{N,k}$, where $N\approx 10^6$ and $k\approx 10^3$. Unfortunately, my matrix $A$ will be dense without any structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I just call the &lt;code&gt;svd&lt;/code&gt; routine from the &lt;code&gt;numpy.linalg&lt;/code&gt; module in Python for a random matrix of this size, I run into a memory error. This is due to the allocation of $V\in \mathbb R^{N,N}$ for the decomposition $A = VSU$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there algorithms around, that avoid this pitfall? E.g. by setting up only the singular vectors assiociated with nonzero singular values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am ready to trade in computation time and accuracy.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="27" LastEditDate="2013-06-07T14:15:47.597" LastActivityDate="2023-07-12T12:00:15.137" Title="Memory efficient implementations of partial Singular Value Decompositions (SVD)" Tags="|linear-algebra|python|reference-request|numpy|svd|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7567" PostTypeId="2" ParentId="7566" CreationDate="2013-06-07T14:38:41.403" Score="7" Body="&lt;p&gt;If you only want a few singular values/vectors, &lt;a href=&quot;http://en.wikipedia.org/wiki/ARPACK&quot;&gt;ARPACK&lt;/a&gt; should do the trick.  The &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/UG/node130.html&quot;&gt;SVD docs&lt;/a&gt; aren't great, and &lt;a href=&quot;http://forge.scilab.org/index.php/p/arpack-ng/downloads/&quot;&gt;this distribution&lt;/a&gt; is more up to date.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: If you want to do this in python, SciPy has a &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.svds.html&quot;&gt;wrapper&lt;/a&gt;.  Since your matrix is dense, you could try the &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.bsr_matrix.html&quot;&gt;block sparse row&lt;/a&gt; (BSR) format.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2013-06-10T18:51:29.183" LastActivityDate="2013-06-10T18:51:29.183" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7568" PostTypeId="1" CreationDate="2013-06-07T15:44:18.953" Score="4" ViewCount="192" Body="&lt;p&gt;What are the strategies one can use to keep maximum number of non attacking pieces (all pieces other than pawn) on an $n\times n$ board? It is like an $n$-queen problem but here instead of only queen other pieces also come into picture. Like if I have a knight, queen, king, bishop, knight what are the best strategies to keep them on a $n\times n$ board such that non of the pieces attack each other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to add to it, I will be getting pieces one after another in a random manner, so basically I don't have any control over the sequence and count of pieces that come along. Like 1. Queen 2. Bishop 3. Queen 4. Knight and I have to keep them on an $n\times n$ board such that no pieces attack each other.&lt;/p&gt;&#xA;" OwnerUserId="4505" LastEditorUserId="1552" LastEditDate="2013-08-25T17:00:37.217" LastActivityDate="2013-08-25T17:00:37.217" Title="What strategies one can use to keep maximum number of non attacking pieces on an $n \times n$ chess board?" Tags="|combinatorics|heuristics|" AnswerCount="1" CommentCount="3" ClosedDate="2013-08-25T01:13:08.550" ContentLicense="CC BY-SA 3.0" />
  <row Id="7569" PostTypeId="2" ParentId="7568" CreationDate="2013-06-07T15:58:37.123" Score="1" Body="&lt;p&gt;While this isn't a full algorithm, I think I can point you to a goal.  Every piece has a specific number of spaces it can move in the greater 15x15 grid with itself at the center.  Whenever you place a piece on the board, the spaces that it can move to become &quot;unsafe&quot;.  Your overall goal is to retain safe spaces for as long as possible so that you can continue to place more pieces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you don't know the order and type of pieces beforehand, all I can suggest is the greedy algorithm.  When you get the next piece, cycle through all the &quot;safe&quot; spaces and see if any of them are invalid due to the new piece being able to attack another from that position.  Once you have a set of valid &quot;safe&quot; spaces, choose the one that minimizes the number of new &quot;unsafe&quot; spaces.  In other words you are trying to put as many of the new piece's moveable spaces either off the edges of the board, or on top of pre-existing &quot;unsafe&quot; spaces.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you knew beforehand the pieces you would be getting, you could come up with something more sophisticated.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-06-07T15:58:37.123" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7572" PostTypeId="1" AcceptedAnswerId="7578" CreationDate="2013-06-07T21:05:17.760" Score="7" ViewCount="4134" Body="&lt;p&gt;Currently the way I compute histograms for data is by generating grid in $N$ dimensions (where $N$ is the dimension of the data) and searching through the $M$ data points in each dimension to see in which bin each one of them fits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the same as an exhaustive search and takes very long if $N$ and $M$ are large (in my case $N=6$, $M=125,000$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: &lt;strong&gt;is there a faster way to accurately compute (or possibly estimate) the number of samples in each bin given a grid and data?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if there is no way to estimate/compute a histogram, is there an alternative method for construct distributions that is fast for high-dimensional large datasets? I've used kernel density estimators, but they are sensitive to which type of kernel you use and depend on prior knowledge of the nature of the data to get good results. (I have no prior knowledge of the nature of the data.)&lt;/p&gt;&#xA;" OwnerUserId="4509" LastEditorUserId="1748" LastEditDate="2013-06-08T01:03:48.727" LastActivityDate="2015-07-31T03:21:44.833" Title="Is there a fast way to compute histograms for high-dimensional large datasets?" Tags="|algorithms|statistics|visualization|data-management|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7573" PostTypeId="1" CreationDate="2013-06-07T22:54:15.587" Score="5" ViewCount="88" Body="&lt;p&gt;I am attempting to calculate temperature of section of rock in the earth as a function of vertical position in the rock and time.  Along with it I am calculating the heat flow through the rock as a function of vertical position in the rock and time.  Note that this is a one-dimensional examination of the problem- imagining that the rock is uniform in every direction but up-down.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Please be patient with me: I currently do not have the vocabulary nor the background to talk at the level of familiarity with FEM and differential equations I see in other questions on this site.  I am a software developer with some physics background, which has allowed me to get this far.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Description of methodology:&lt;/strong&gt; I am currently splitting up the section of rock into horizontal slices (depth steps), and then calculating the temperature and heat flow of each depth step at specific time-steps.  I am assuming that heat only flows upward from below the section of rock.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, the temperature difference between the top and bottom of a depth step would be expressed as:&#xA;$$&#xA;\Delta T = \frac{Q\Delta Z}{\lambda}&#xA;$$&#xA;Where $Q$ is the heat flow at the middle of the depth step in $mW/m^2$, $\Delta Z$ is the thickness of a depth step at this time, and $\lambda$ is the thermal conductivity of the rock (constant).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, the change in the heat flow between the bottom and top of a depth step could be expressed as:&#xA;$$&#xA;\Delta Q = - C_p \rho \Delta Z \frac{\Delta T}{\Delta t}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C_p$ is the heat capacity of the rock (constant), $\rho$ is the density (constant), and $\frac{\Delta T}{\Delta t}$ would be the change in temperature of the depth step over the course of this time-step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a starting point for the calculations, I know the temperature at the top of the rock layer as a whole (but &lt;strong&gt;not&lt;/strong&gt; the temperature at the bottom of the rock layer). I also know the heat flow value just below the rock layer (but &lt;strong&gt;not&lt;/strong&gt; the heat flow value at the top).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key piece of complexity to this is that the rock layer is increasing in thickness through time (aka deposition), so that I am adding to the $\Delta Z$ of the top-most depth step and then adding more depth steps to the top of the section of rock as time progresses.  As a consequence, individual depth steps get more and more buried, and thus hotter through time, which thus causes the heat flow to decrease as you move upwards in the rock layer (as per equation 2 above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My method of calculating the temperatures and heat flows for each time-step is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Increase the thickness of the rock layer, adding depth steps to the top as necessary.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the temperatures at the top and bottom of each depth step going &lt;strong&gt;down&lt;/strong&gt; the layer (since I know the surface temperature, and thus the starting point), using the heat flow values from the previous time-step.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the heat flow values at the bottom and top of each depth step going &lt;strong&gt;up&lt;/strong&gt; the layer (since I only know the heat flow at the very bottom of the rock layer), using the temperatures of each depth step calculated in step #2, along with the temperature in the previous time-step. &lt;/li&gt;&#xA;&lt;li&gt;Repeat steps 2-3 until the solution converges. Note that in step #2 I can now use the heat flows calculated in step #3.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Problem:&lt;/strong&gt;  This solution does not always converge.  As the rock layer gets thicker and thicker through time, each time-step takes longer and longer to converge, until it reaches a tipping point and starts to diverge.  &lt;strong&gt;What can I do to force this to converge?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am suspicious that the reason for divergence is because I'm forced to calculate temperature down the depth steps and heat flows up the depth steps.  This means that the bottom-most depth step's temperature is sensitive to ALL the changes in temperature above it as per the first equation above.  But this then causes the heat flow at the bottom to change from the previous iteration by a fair amount, which then propagates all the way to the top of the rock layer via the second equation above, which then affects the temperature at the top strongly, etc.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Warning:&lt;/strong&gt; The above problem is actually a major simplification of a larger problem I'm trying to solve- the constants in the equation above are actually temperature dependent &lt;strong&gt;and&lt;/strong&gt; depth dependent in a complicated, non-continuous way.  However, the problem I am facing still exists in the simplification as described above.&lt;/p&gt;&#xA;" OwnerUserId="4492" LastActivityDate="2013-06-07T22:54:15.587" Title="Non-convergance when calculating temperature/heat flows through a section of rock" Tags="|convergence|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7574" PostTypeId="1" CreationDate="2013-06-08T00:08:58.807" Score="3" ViewCount="168" Body="&lt;p&gt;I need to assemble a matrix (in dense form, of moderate size, say dimension 1000) which is most easily expressed as the product of several (4) sparse matrices. These matrices are most easily expressed by their action (a matrix-vector multiplication routine), although I suppose it's not too hard to write them out explicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what is the best way for computing the product, keeping in mind that the result will wind up being stored as a dense matrix anyways. Clearly, expressing everything as a dense matrix and computing the products seems ridiculously inefficient, since the intermediates are still quite sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More details: The actual form of the matrix is like&#xA;$$ A = DBD^HC $$&#xA;where $D$ is a mesh adjacency matrix (only two nonzeros per row), and $B$ and $C$ are banded matrices with bandwidth under 10.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2018-03-11T08:34:17.413" Title="Computing sparse matrix products into a dense result" Tags="|matrix|sparse-matrix|dense-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7575" PostTypeId="2" ParentId="7574" CreationDate="2013-06-08T01:20:09.970" Score="2" Body="&lt;p&gt;If you really want all of the entries of the dense matrix product, you could compute it column by column, by computing the action on columns of the identity matrix.&lt;/p&gt;&#xA;" OwnerUserId="3705" LastActivityDate="2013-06-08T01:20:09.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7576" PostTypeId="1" AcceptedAnswerId="7585" CreationDate="2013-06-08T01:57:03.587" Score="1" ViewCount="124" Body="&lt;p&gt;I'm using Trilinos for a problem and I have an &lt;code&gt;Epetra_MultiVector&lt;/code&gt; object with 1 vector of length &lt;code&gt;m*n*nFrames&lt;/code&gt;. I need to turn it into an &lt;code&gt;Epetra_MultiVector&lt;/code&gt; object with &lt;code&gt;nFrames&lt;/code&gt; of length &lt;code&gt;m*n&lt;/code&gt;. The first &lt;code&gt;m*n&lt;/code&gt; entries need to go into the first vector, the second into the second vector and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have this working fine with just one process. What I'm doing is extracting all the data to a standard C++ array on process 0, reshaping that into a 2D standard C++ array (an array of pointers actually), and then creating a new &lt;code&gt;Epetra_MultiVector&lt;/code&gt; with my 2D array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With more than one process, however, because everything is on process 0, when I create the new &lt;code&gt;Epetra_MultiVector&lt;/code&gt;, all the data for the other process is nonsense (accessing undefined memory, I guess).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question ultimately is: how do I reshape a Trilinos &lt;code&gt;Epetra_MultiVector&lt;/code&gt; correctly with more than one process?&lt;/p&gt;&#xA;" OwnerUserId="4420" LastEditorUserId="1748" LastEditDate="2013-06-08T03:49:56.670" LastActivityDate="2013-06-10T18:33:18.913" Title="Reshaping a MultiVector in Trilinos" Tags="|parallel-computing|mpi|trilinos|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7578" PostTypeId="2" ParentId="7572" CreationDate="2013-06-08T08:03:32.447" Score="6" Body="&lt;p&gt;Histograms are not useful for high dimensional data. &lt;a href=&quot;http://en.wikipedia.org/wiki/Curse_of_dimensionality&quot; rel=&quot;nofollow&quot;&gt;The curse of dimensionality&lt;/a&gt; affects one quite fast. As in your case if the grid is of size 7**6, you have on average one point in one bin. Kernel density estimator are better suited as long as you keep the kernel bandwidth large enough. In my experience the top hat kernel as k-nearest neighbor yields reasonable results up to D=10, if sampling is sufficient. &#xA;There is also a quite efficient algorithm for calculating &lt;a href=&quot;http://www.cs.umd.edu/~mount/ANN/&quot; rel=&quot;nofollow&quot;&gt;k-nearest neighbors&lt;/a&gt; in higher dimensions, which I can recommend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the kernel shape doesn't really matter so much, because you need to keep the bandwidth large enough due to lack of data. If you see a dependency on the kernel shape your bandwidth is likely too small.&#xA;There are a couple of &lt;a href=&quot;http://en.wikipedia.org/wiki/Kernel_density_estimation&quot; rel=&quot;nofollow&quot;&gt;rule of thumbs&lt;/a&gt; how to select the bandwidth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you calculate some other property from the probability density, in nearly all cases you are better off not computing the density at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit to properly comment on the comment&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am afraid you cannot capture nuanced differences in high dimensional data with histograms if you check the statistical error for each bin. Go ahead and do some simple random number experiment and check the fluctuation in each bin with your sample size. Unless you use a really small grid size like 2**6, which is pointless to begin with, you will only see noise as nuanced differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For calculating entropies == Jensen Shannon divergences I recommend following papers which I used in my phd thesis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Article (Hnizdo2007) Hnizdo, V.; Darian, E.; Fedorowicz, A.; Demchuk, E.; Li, S. &amp;amp; Singh, H. Nearest-neighbor nonparametric method for estimating the configurational entropy of complex molecules. Journal of computational chemistry, J Comput Chem, 2007, 28, 655&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Article (Hnizdo2008) Hnizdo, V.; Tan, J.; Killian, B. &amp;amp; Gilson, M. Efficient calculation of configurational entropy from molecular simulations by combining the mutual-information expansion and nearest-neighbor methods Journal of computational chemistry, NIH Public Access, 2008, 29, 1605&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no idea for the earth mover distance and have never used that before though. It kinda looks like that you need the work for a phase space transformation bringing two distributions together. It seems to me that is similar to a free energy difference between the two systems given by the distributions.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2013-06-09T09:51:07.297" LastActivityDate="2013-06-09T09:51:07.297" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7579" PostTypeId="1" AcceptedAnswerId="7582" CreationDate="2013-06-08T08:45:00.000" Score="5" ViewCount="482" Body="&lt;p&gt;I am following the approach of Hundsdorfer from &lt;a href=&quot;http://books.google.com/books?id=WwB1hiqj8OkC&amp;amp;pg=PA254&amp;amp;dq=Example%203.3%20Adaptive%20upwinding&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=TuayUfmtEYfkkgXMyYHoAg&amp;amp;redir_esc=y#v=onepage&amp;amp;q=Example%203.3%20Adaptive%20upwinding&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Numerical solution of time-dependent advection-diffusion-reaction equations&lt;/a&gt; in which they introduce adaptive upwinding. The method can adapt from pure upwind ($\kappa=1$) to pure central ($\kappa=0$) discretisation for the advection term (&lt;strong&gt;N.B.&lt;/strong&gt; the diffusion term is always based on a central difference),&#xA;$$&#xA;u_j^{\prime} = \frac{1}{2h}a \left( u_{j-1} - u_{j+1} \right) + \left( \frac{d}{h^2} + \kappa\frac{a}{2h} \right) \left( u_{j-1} - 2u_{j} + u_{j+1} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_j^{\prime}$ is shorthand for the time derivative, $a$ is the velocity ($a&amp;gt;0$), $d$ is the diffusion coefficient and $h$ is the step size between &lt;strong&gt;uniform&lt;/strong&gt; mesh points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to extend this to the case of a &lt;strong&gt;non-uniform&lt;/strong&gt; mesh, however I run into a problem because I get a factor of $4$ appearing when I do so, which seems inconsistent. For example, modify the above equation to include non-uniform steps,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_j^{\prime} = \frac{1}{2h_j^{+}}a \left( u_{j-1} - u_{j+1} \right) + \left( \frac{d}{(h_j^{+} + h_j^{-})^2} + \kappa\frac{a}{2h_j^{+}} \right) \left( u_{j-1} - 2u_{j} + u_{j+1} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h_j^{+}=x_{j+1} - x_j$ and $h_j^{-}=x_{j} - x_{j-1}$. We can check the consistence by putting this equation back on to a uniform mesh. Substituting $h_j^{+}\rightarrow h$ and $h_j^{-}\rightarrow h$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_j^{\prime} = \frac{1}{2h}a \left( u_{j-1} - u_{j+1} \right) + \left( \frac{d}{4h^2} + \kappa\frac{a}{2h} \right) \left( u_{j-1} - 2u_{j} + u_{j+1} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is identical appart form the factor of $4$ appearing in the denominator!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't much experience with these adaptive upwinds, can you see an issue with using a modified version of the original equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_j^{\prime} = \frac{1}{2h}a \left( u_{j-1} - u_{j+1} \right) + \left( \frac{4d}{(2h)^2} + \kappa\frac{a}{2h} \right) \left( u_{j-1} - 2u_{j} + u_{j+1} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The additional constants cancel on both a uniform and non-uniform grid, giving the correct central difference diffusion term in all cases,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;d\frac{\left( u_{j-1} - 2u_{j} + u_{j+1} \right) }{h^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a Taylor series to calculate the second derivative (the diffusion term) with non-uniform step gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;2d \frac{h^{+} \left(- u_{j} + u_{{j-1}}\right) + h^{-} \left(- u_{j} + u_{{j+1}}\right)}{h^{+} h^{-} \left(h^{+} + h^{-}\right)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This reduces to the correct expression when the grid is uniform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this makes the equation hard to write in a compact form because the $h^{-}$ and $h^{+}$ terms appear in the numerator. Does anybody you have any comments on whether this can be simplified further, i.e. so the $h$ terms don't appear in the numerator? This will allow the write the equation in the original form.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-06-09T12:57:34.897" LastActivityDate="2013-06-09T12:57:34.897" Title="Implementing an adaptive discretisation (upwind/central hybrid) for the advection-diffusion equation on a non-uniform mesh" Tags="|discretization|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7580" PostTypeId="2" ParentId="7523" CreationDate="2013-06-08T12:25:43.493" Score="1" Body="&lt;p&gt;I am not giving the full answer here but as an idea you could think about your data as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have &lt;a href=&quot;http://en.wikipedia.org/wiki/Homoscedasticity&quot; rel=&quot;nofollow&quot;&gt;homoscedastic&lt;/a&gt; data, which is good. But a non-normal noise term.&#xA;Now there a several techniques for defining a statistical model on these data. One could do a B-spline regression on the model or one could choose an appropriate basis on the domain of $f$ for an approximation on $A(f)\approx \hat{A}(f) = \sum_i c_i b_i(f)$, with basis functions $b_i(f)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either way, your fit model needs to find some coefficients and reads: $F(f,d) = \hat{A}(f)/d^2 +L$ and $\hat{A}(f)$ is the respective approximation to $A(f)$. Once you have obtained the fit, the residuals represent $L$. A hypothesis test should reveal if $E(L)=0$ in your model, so you catch the features of $A(f)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likely, the fit matrix will be ill-conditioned, so when performing the fit you need to apply some regularization technique, e.g. truncated svd.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2013-06-08T12:32:39.557" LastActivityDate="2013-06-08T12:32:39.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7581" PostTypeId="1" CreationDate="2013-06-08T18:20:08.800" Score="1" ViewCount="242" Body="&lt;p&gt;I am trying to implement a program the numerical method to solve ODE called &lt;strong&gt;Block BDF&lt;/strong&gt; as explained in this article: &lt;a href=&quot;https://waset.org/journals/waset/v38/v38-49.pdf&quot; rel=&quot;nofollow&quot;&gt;https://waset.org/journals/waset/v38/v38-49.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it is variable step-size, I need to compute the LTE (local truncation), that is $LTE=y_{n+2}^{(5)}-y_{n+2}^{(4)}$. There is a formula in the article for $y_{n+2}^{(5)}$ but I cannot find one for $y_{n+2}^{(4)}$; where $y_{(n+2)}^{(k)}$ means the $k-$order approximation to $y_{(n+2)}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please help me finding the formula for fourth order approximation? (As I'm only trying to implement, information only about the formula can be helpful, but explanation is encouraged).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. The more general method is BDF: &lt;a href=&quot;http://en.wikipedia.org/wiki/Backward_differentiation_formula&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Backward_differentiation_formula&lt;/a&gt; but this method works in block of 2 (works for $y(n+1)$ and $y(n+2)$ at the same step).&lt;/p&gt;&#xA;" OwnerUserId="4513" LastEditorUserId="4244" LastEditDate="2013-06-08T19:36:30.990" LastActivityDate="2013-06-08T19:36:30.990" Title="Block Backward Differentiation Formula (BBDF), on order 4 formula" Tags="|ode|numerical-analysis|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7582" PostTypeId="2" ParentId="7579" CreationDate="2013-06-08T18:57:58.760" Score="3" Body="&lt;p&gt;You need to rederive the term&#xA;$$&#xA;\frac{d}{(h_j^+ + h_j^-)^2}&#xA;$$&#xA;because this appears to be wrong. It &lt;strong&gt;&lt;em&gt;must&lt;/em&gt;&lt;/strong&gt; lead to factor of 4 when switching to a uniform grid because you've taken the sum of two things that are $O(h)$ and squared it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps if you spell out the derivation, we can help you find the error. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-08T18:57:58.760" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7583" PostTypeId="1" CreationDate="2013-06-08T20:22:15.627" Score="5" ViewCount="318" Body="&lt;p&gt;For a relatively simple Markov chain Monte Carlo process, such as using Metropolis to find calculate thermal averages for an Ising model, how is it possible to determine whether quantities have converged?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one knows the autocorrelation time, this seems relatively simple. You just run it for a sufficiently high multiple of this time and Bob's your uncle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't know the autocorrelation time, it would seem more complex. No matter how hard you try to ensure that your averages have converged, there's always the possibility that it is stuck in a local minima. This is especially troublesome if the autocorrelation time scales polynomially or even exponentially with the system size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how can you ensure that the quantities really have converged? What convergence tests are not fooled by local minima? I read &lt;a href=&quot;http://arxiv.org/abs/1306.0540&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; recently about the method of logarithmic binning. Could that do the job?&lt;/p&gt;&#xA;" OwnerDisplayName="Matthew Matic" LastEditorUserId="127" LastEditDate="2013-06-09T05:06:48.787" LastActivityDate="2013-06-10T08:57:46.390" Title="Convergence tests in Markov Chain Monte Carlo" Tags="|monte-carlo|stochastic|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7584" PostTypeId="1" AcceptedAnswerId="7588" CreationDate="2013-06-08T22:30:12.917" Score="3" ViewCount="118" Body="&lt;p&gt;I have to present a working 3D simulation of an arbitrary comet in a close proximity to the Sun or to the Jupiter. In specific, I want to simulate comets D/1993 F2 and C/2011 W3, using the theoretical model I have proposed in my recent reseach. My deadline is 1st April 2014.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I see it now - I make my comet of tons of small particles, define the physics of these particles, define the external physics and just run the simulation for some period of time. I need to obtain both select data in numerical form, and a 3D visualisation of the whole process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now I do not have to do that in Minkowski space, just regular 3D space, albeit in spherical coordinates. Physics wise - thermodynamics, gravity, mechanical properties of materials, solar wind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've got:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Chunk of free time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A Windows/Linux laptop. Might have an access to OS X workstation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Solid knowledge of mathematics behind these physical phenomena (PDEs, integrals, series, complex functions).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A little bit of experience in coding - Turbo Pascal 7, C++, Delphi (order from most to less experience).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Budget of roughly 500 USD - for software, literature etc. I am qualified for student discounts.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I can not use existing solutions, as I have to define the mathematical model for physics according to my research.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I do not know where to start, and from what do I begin.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit according to the input from Ben.&#xA;I have a developed mathematical model of a system comet-star, or comet-planet (it is simplified comet-sun case). I am looking on how I can make a 3D simulation of this model, with gathering specific data and the visual representation of the processes happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps. If it is still to broad to be answered, feel free to specify more details.&lt;/p&gt;&#xA;" OwnerDisplayName="user4515" LastEditorDisplayName="user4515" LastEditDate="2013-06-08T23:16:20.973" LastActivityDate="2013-06-09T18:01:15.847" Title="Simulating evolution of comet" Tags="|simulation|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7585" PostTypeId="2" ParentId="7576" CreationDate="2013-06-09T05:50:09.663" Score="1" Body="&lt;p&gt;I've written a function that does it. It does not do it in place, and is not particularly well written, but for posterity I'll add my solution. I realize that I really don't need to reshape the standard C++ array, but I'll leave it there until it becomes a problem for me.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int ReshapeMultiVector(Epetra_MultiVector *&amp;amp;reshapedMultiVec, const int length, const int numVecs, const Epetra_MultiVector &amp;amp;originalMultiVec, Epetra_MpiComm &amp;amp;comm){&#xA;&#xA;    /* Function to reshape a multivector. Does it by copying the values, so additional memory will be required.&#xA;     * Note that this function does NOT delete the original vector.&#xA;     * INPUTS&#xA;     *  reshapedMultiVec - This is a reference to a null pointer that will point to the&#xA;     *             new (reshaped) multivector.&#xA;     *  length       - Global length of the new multivector&#xA;     *  numVecs      - The number of vectors in the new multivector.&#xA;     *  originalMultiVec - The original multivector that is being reshaped.&#xA;     *  comm         - MPI communicator&#xA;     * OUTPUT&#xA;     *  int      - Error flag&#xA;     */&#xA;&#xA;    int err;&#xA;&#xA;    int numVecsOrig = originalMultiVec.NumVectors();&#xA;    int lengthOrig = originalMultiVec.GlobalLength();&#xA;&#xA;    //extract phase_gradx and phase_grady as standard arrays so that we can reshape them.&#xA;    double** originalMultiVec_std = new double*[numVecsOrig];&#xA;    for(int i=0;i&amp;lt;numVecsOrig;++i){&#xA;        originalMultiVec_std[i] = new double[lengthOrig];&#xA;    }&#xA;    err = MultiVectorToArray(originalMultiVec_std, originalMultiVec, false);&#xA;    if(err) return err;&#xA;&#xA;    // Copy the values to all the processors after extracting them.&#xA;    for(int i=0;i&amp;lt;numVecsOrig;++i){&#xA;        comm.Broadcast(originalMultiVec_std[i], lengthOrig, 0);&#xA;    }&#xA;&#xA;&#xA;    double** reshaped_std = new double*[numVecs];&#xA;    for(int i=0;i&amp;lt;numVecs;++i){&#xA;        reshaped_std[i] = new double[length];&#xA;    }&#xA;    err = Reshape2DArray(originalMultiVec_std, lengthOrig, numVecsOrig, reshaped_std, length, numVecs);&#xA;    if(err) return err;&#xA;&#xA;&#xA;    Epetra_Map reshapedMap(length,0,comm);&#xA;&#xA;    //Initialize the vector&#xA;    int numMyElements = reshapedMap.NumMyElements();&#xA;    int* myGlobalElements = reshapedMap.MyGlobalElements(); &#xA;&#xA;&#xA;    reshapedMultiVec = new Epetra_MultiVector(reshapedMap,numVecs);&#xA;    double ** Ap = reshapedMultiVec-&amp;gt;Pointers();&#xA;&#xA;    for (int j=0; j&amp;lt;numVecs; ++j)&#xA;    {&#xA;        double * v = Ap[j];&#xA;&#xA;        // Fill it&#xA;        for (int i=0; i&amp;lt;numMyElements; ++i)&#xA;        {&#xA;            v[i] = reshaped_std[j][myGlobalElements[i]];&#xA;        }&#xA;    }&#xA;&#xA;&#xA;    // free memory&#xA;    for(int i=0;i&amp;lt;numVecsOrig;++i){&#xA;        delete[] originalMultiVec_std[i];&#xA;    }&#xA;    delete[] originalMultiVec_std;&#xA;    for(int i=0;i&amp;lt;numVecs;++i){&#xA;        delete[] reshaped_std[i];&#xA;    }&#xA;    delete[] reshaped_std;&#xA;&#xA;&#xA;    return(EXIT_SUCCESS);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4420" LastActivityDate="2013-06-09T05:50:09.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7586" PostTypeId="2" ParentId="7583" CreationDate="2013-06-09T14:40:59.333" Score="1" Body="&lt;p&gt;It's very hard to &lt;em&gt;prove&lt;/em&gt; convergence for an ensemble in statistical mechanics, for exactly the reasons you've suggested—you may be stuck moving around a local minimum, instead of sampling the complete phase space. About the best that you can do is to demonstrate &lt;em&gt;likely&lt;/em&gt; convergence of your simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can do calculations to demonstrate autocorrelation of various properties. For instance, you can measure the autocorrelation of the energy function: $\left&amp;lt;E(t)E(t + \delta t) \right&amp;gt;$. Alternatively, you can use the &lt;a href=&quot;http://link.aip.org/link/doi/10.1063/1.457480&quot; rel=&quot;nofollow&quot;&gt;multiscale error correlation method of Flyvbjerg and Petersen&lt;/a&gt; to determine when your data samples are effectively decorrelated from one another, and use that as a means of determining whether you have enough data (and to estimate the statistical error in the mean of your data samples).&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2013-06-09T14:40:59.333" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7587" PostTypeId="2" ParentId="7572" CreationDate="2013-06-09T17:43:45.020" Score="1" Body="&lt;p&gt;I don't have a rigorous proof for this, but we have solved this problem with a random sampling based approach.  Or data is something like $4\,096 \times 12\,000 \times 1\,000 \times 6 \times 3 \times 2 \times 4 $.  Hint: (X, Y, Z, stack#, RGB, state, samples)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, we figure out what two categories we want to do differential histograms for, then pick random points using uniform sampling over the remaining axes.  An exact answer obviously requires consideration of all the points, but for our data-set consideration of 1-3% gets an answer who's PDF is &gt;0.95 correct (1 - RMSE) of the full data-set.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: This answer is also clearly contingent on the underlying distribution.  If, for example, each sample were uniformly spaced over all of IEEE 754 float 64 space, then this approach will fall on its face.  So, the accuracy of this approach actually depends on the underlying PDF.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in the above answer by @bort, the Entropy of you data-set will directly be a measure of the accuracy of a histogram which selectively samples, as it is after all direct measure of both the total informational content or capacity of a given sample to represent the whole.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2013-06-09T17:43:45.020" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7588" PostTypeId="2" ParentId="7584" CreationDate="2013-06-09T18:01:15.847" Score="2" Body="&lt;p&gt;I would check out the field of &quot;&lt;a href=&quot;http://en.wikipedia.org/wiki/Smoothed-particle_hydrodynamics&quot; rel=&quot;nofollow&quot; title=&quot;Smoothed-particle hydrodynamics&quot;&gt;Smoothed-particle hydrodynamics&lt;/a&gt;&quot;, and its sister discipline, smoothed-particle mechanics.  These approaches lend well to variable boundary conditions and integration of numerous mathematical models/forms.  I imagine there are heat and mass flows within the system as well as external (possibly time varying) conditions imposed on them.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The chief downside is that they are computationally expensive, and depending on the phenomenon integrated, your time-scales may get painfully small.  For example, incorporating surface tension on a fluid model we worked on shrunk time scales into the microseconds per step, this was ok for us because our particle count was in the hundreds, and time only needed to span a few seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of packages out there that do these physics, and I can point you at one or more given some more specific information.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2013-06-09T18:01:15.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7589" PostTypeId="1" AcceptedAnswerId="7591" CreationDate="2013-06-09T21:30:14.820" Score="0" ViewCount="458" Body="&lt;p&gt;I am a new Matlab user and wondering if there is any built-in operator or function which could convert the hour of the year into calendar date and time for that year. For example, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hour 0 in year 2000 is 2000-01-01 00:00:00&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hour 1 is 2000-01-01 01:00:00&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and so on.&lt;/p&gt;&#xA;" OwnerUserId="2091" LastEditorUserId="127" LastEditDate="2013-06-10T03:13:21.760" LastActivityDate="2013-06-10T03:13:21.760" Title="Convert hour of the year into date in Matlab" Tags="|matlab|" AnswerCount="1" CommentCount="0" ClosedDate="2013-06-10T23:49:58.183" ContentLicense="CC BY-SA 3.0" />
  <row Id="7590" PostTypeId="2" ParentId="7468" CreationDate="2013-06-09T21:47:17.153" Score="8" Body="&lt;p&gt;I've been working a bit on adaptive refinement to do the job (see the code below). The scaling of the error indicator with total mesh size and total variation of the mesh function is not perfect, but you could fit that to your needs. The images below are for testcase #4. The number of cells increases from 200 to about 24,000, which may be a bit over the top, but the result is quite nice. The mesh shows that only the relevant portions have been refined. The artefacts you can still see, are what the third order elements themselves couldn't represent sufficiently accurate.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;from numpy import abs&#xA;&#xA;&#xA;def compute_error(expr, mesh):&#xA;    DG = FunctionSpace(mesh, &quot;DG&quot;, 0)&#xA;    e = project(expr, DG)&#xA;    err = abs(e.vector().array())&#xA;    dofmap = DG.dofmap()&#xA;    return err, dofmap&#xA;&#xA;&#xA;def refine_by_bool_array(mesh, to_mark, dofmap):&#xA;    cell_markers = CellFunction(&quot;bool&quot;, mesh)&#xA;    cell_markers.set_all(False)&#xA;    n = 0&#xA;    for cell in cells(mesh):&#xA;        index = dofmap.cell_dofs(cell.index())[0]&#xA;        if to_mark[index]:&#xA;            cell_markers[cell] = True&#xA;            n += 1&#xA;    mesh = refine(mesh, cell_markers)&#xA;    return mesh, n&#xA;&#xA;&#xA;def adapt_mesh(f, mesh, max_err=0.001, exp=0):&#xA;    V = FunctionSpace(mesh, &quot;CG&quot;, 1)&#xA;    while True:&#xA;        fi = interpolate(f, V)&#xA;        v = CellVolume(mesh)&#xA;        expr = v**exp * abs(f-fi)&#xA;        err, dofmap = compute_error(expr, mesh)&#xA;&#xA;        to_mark = (err&amp;gt;max_err)&#xA;        mesh, n = refine_by_bool_array(mesh, to_mark, dofmap)&#xA;        if not n:&#xA;            break&#xA;&#xA;        V = FunctionSpace(mesh, &quot;CG&quot;, 1)&#xA;    return fi, mesh&#xA;&#xA;&#xA;def show_testcase(i, p, N, fac, title1=&quot;&quot;, title2=&quot;&quot;):&#xA;    funcs = [&quot;sin(60*(x[0]-0.5)*(x[1]-0.5))&quot;,&#xA;             &quot;sin(10*(x[0]-0.5)*(x[1]-0.5))&quot;,&#xA;             &quot;sin(10*(x[0]-0.5))*sin(pow(3*(x[1]-0.05),2))&quot;]&#xA;&#xA;    mesh = UnitSquareMesh(N, N)&#xA;    U = FunctionSpace(mesh, &quot;CG&quot;, p)&#xA;    f = interpolate(Expression(funcs[i]), U)&#xA;&#xA;    v0 = (1.0/N) ** 2;&#xA;    exp = 1&#xA;    #exp = 0&#xA;    fac2 = (v0/100)**exp&#xA;    max_err = fac * fac2&#xA;    #print v0, fac, exp, fac2, max_err&#xA;    g, mesh2 = adapt_mesh(f, mesh, max_err=max_err, exp=exp)&#xA;&#xA;    plot(mesh, title=title1 + &quot; (mesh)&quot;)&#xA;    plot(f, title=title1)&#xA;    plot(mesh2, title=title2 + &quot; (mesh)&quot;)&#xA;    plot(g, title=title2)&#xA;    interactive()&#xA;&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    N = 10&#xA;    fac = 0.01&#xA;    show_testcase(0, 1, 10, fac, &quot;degree 1 - orig&quot;, &quot;degree 1 - refined (no change)&quot;)&#xA;    show_testcase(0, 2, 10, fac, &quot;degree 2 - orig&quot;, &quot;degree 2 - refined&quot;)&#xA;    show_testcase(0, 3, 10, fac, &quot;degree 3 - orig&quot;, &quot;degree 3 - refined&quot;)&#xA;    show_testcase(0, 3, 10, 0.2*fac, &quot;degree 3 - orig&quot;, &quot;degree 3 - more refined&quot;)&#xA;    show_testcase(1, 2, 10, fac, &quot;smooth: degree 2 - orig&quot;, &quot;smooth: degree 2 - refined&quot;)&#xA;    show_testcase(1, 3, 10, fac, &quot;smooth: degree 3 - orig&quot;, &quot;smooth: degree 3 - refined&quot;)&#xA;    show_testcase(2, 2, 10, fac, &quot;bumps: degree 2 - orig&quot;, &quot;bumps: degree 2 - refined&quot;)&#xA;    show_testcase(2, 3, 10, fac, &quot;bumps: degree 3 - orig&quot;, &quot;bumps: degree 3 - refined&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/n3NRJ.png&quot; alt=&quot;Plot on unrefined mesh&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/2ZPnS.png&quot; alt=&quot;Unrefined mesh&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/HGzIQ.png&quot; alt=&quot;Plot on refined mesh&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/QLN8Q.png&quot; alt=&quot;Adaptively refined mesh&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4518" LastActivityDate="2013-06-09T21:47:17.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7591" PostTypeId="2" ParentId="7589" CreationDate="2013-06-09T21:58:00.183" Score="3" Body="&lt;p&gt;I think I solved it. I used the addtodate function documented &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/addtodate.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how I am doing it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;start=datenum(2000, 1, 1, 0, 0, 0);&#xA;curr=addtodate(start, hour_of_yr, 'hour');&#xA;disp (datestr (curr));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where hour_of_yr is passed to matlab function from commandline.&lt;/p&gt;&#xA;" OwnerUserId="2091" LastActivityDate="2013-06-09T21:58:00.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7592" PostTypeId="1" CreationDate="2013-06-09T22:30:17.823" Score="3" ViewCount="605" Body="&lt;p&gt;Is there any way to implement an element wise contraction force (i.e., a force which causes the FEs themselves to contract onto themselves)? For example this would happen when something dehydrates. Preferably with first order tetrahedral elements, but if the only way to do it is have a vertex in the centre of the elements then higher order will have to do (it just opens up other issues to be solved) also I'm only interested in the x[2] direction. This would be something like a body force such as &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;dot(body force, displacement)*dx(subdomain)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;but instead of the body force being defined in the global sense, and acting in a global direction from a reference point (for example gravity acting downward over the whole domain constrained by a boundary condition such as the ground stopping the object falling) it would be within the local element with the reference frame being dependent on the element. I have tried implementing this with multiple body forces on multiple subdomains acting in opposite directions but this seems to cause the forces to effectively cancel each other out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple physical example would be a wet sponge sitting on a table drying out. It has two forces acting on it. The first simple one being gravity in the vertical direction causing it to stay sitting on the table, with the boundary condition of the table not letting it fall. The second is the contraction force caused when the water evaporates and the sponge gets smaller. If we were to model this using say 1 element, each of the vertices defining the cell would need to contract to the cells local centre. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is implemented in Fenics. &lt;/p&gt;&#xA;" OwnerUserId="4296" LastEditorDisplayName="user10435" LastEditDate="2015-03-13T23:35:56.070" LastActivityDate="2015-03-13T23:35:56.070" Title="Implementation of a contraction force in Fenics" Tags="|finite-element|fenics|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7593" PostTypeId="1" CreationDate="2013-06-07T11:53:17.747" Score="5" ViewCount="176" Body="&lt;p&gt;I  have a two  iterative method for approximating the inverse of given square matrix $A$ whose error terms are given as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Error estimate of method $1$:&lt;/strong&gt;&#xA;$\lVert A^{-1} - X_{k}\rVert \leq q^{2^{k+1}-3}(1-q^{2^{k+1}})^{-1}\|R_{0}\|$, $0&amp;lt;q&amp;lt;1$, $k = 0, 1,\dots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R_{0} = I - AX_{0}$ , $X_{k}$ is the sequence of approximations for $A^{-1}$ and $X_{0}$ is the initial approximation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Error estimate of method $2$:&lt;/strong&gt;&#xA;$\lVert A^{-1} - X_{k}\rVert \leq q^{2^{k}}(1-q^{2^k})^{-1}\|R_{0}\|$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly method $1$ has correct order $q^{2^{k+2}-3}$ , while second one has order $q^{2^{k}}$. Both the methods are of second order. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is  could we compare  the speed of convergence of two different iteration methods of same order looking at their error estimates? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I beg pardon if this question is stupid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="1580" OwnerDisplayName="srijan" LastEditorUserId="6663" LastEditDate="2013-12-26T16:51:15.807" LastActivityDate="2013-12-26T16:51:15.807" Title="Can we compare the speed of convergence of two different iteration methods of same order looking at their error estimates?" Tags="|iterative-method|error-estimation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7594" PostTypeId="2" ParentId="7593" CreationDate="2013-06-07T13:07:26.770" Score="2" Body="&lt;p&gt;Normally when one makes error estimates they are not particularly tight bounds, i.e. there could easily be a factor of a constant off.  Your two methods differ by approximately a constant, namely $q^{-3}$, so you do not have enough data to conclude that Method 2 has smaller error than Method 1.  To conclude this you would need the difference to approach 0 as $k\rightarrow \infty$, and even then you would only know that Method 2 has smaller error for sufficiently large $k$.&lt;/p&gt;&#xA;" OwnerDisplayName="vadim123" LastActivityDate="2013-06-07T13:12:46.807" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7595" PostTypeId="1" CreationDate="2013-06-10T08:28:10.297" Score="6" ViewCount="964" Body="&lt;p&gt;I'm a bit confused by the vast amount of literature on solving eigenvalue problems. I have a sparse (large) matrix which I have already factored (by Cholesky or LDU). I would like to compute few eigenvectors of this matrix associated to the smallest eigenvalues. Is there any method that is :&#xA;1) efficient in terms of computation time&#xA;2) robust&#xA;3) could re-use the factorization of the original matrix ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you !&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="127" LastEditDate="2013-06-10T15:26:13.427" LastActivityDate="2013-06-10T15:26:13.427" Title="Computing smallest eigenvectors of a sparse matrix, having its inverse" Tags="|linear-algebra|sparse-matrix|eigenvalues|matrix-factorization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7596" PostTypeId="2" ParentId="7583" CreationDate="2013-06-10T08:57:46.390" Score="1" Body="&lt;p&gt;I can't comment yet, so I would be grateful if someone could turn this into a comment to aeismail's answer. Although it's quite long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have access to the above mentioned Flyvbjerg and Petersen paper and thus can't really judge on what follows, but in the really nice paper &lt;a href=&quot;http://membrane.urmc.rochester.edu/sites/default/files/papers/ARCC_2009.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;Quantifying uncertainty and sampling quality in biomolecular simulations&quot;&lt;/a&gt; by Grossfield and Zuckerman, the authors comment: &quot;The approach can be described simply (although it is not easily understood from the original reference).&quot;&#xA;In my opinion, Grossfield and Zuckerman give a really nice and simple explanation of this procedure which they call &quot;block averaging&quot; and how to obtain estimates of correlation times with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another paper which might be interesting is &lt;a href=&quot;http://pubs.acs.org/doi/abs/10.1021/ct1002384&quot; rel=&quot;nofollow&quot;&gt;Automated Sampling Assessment for Molecular Simulations Using the Effective Sample Size&lt;/a&gt; by Zhang, Bhatt and Zuckerman. In this article, they assess sampling quality and convergence via an Effective Sample Size (ESS) estimated from the variances in the populations of physical states. They argue that state populations are the most critical slow observables and thus are a good observable to estimate correlation times from.&#xA;But they also put the fundamental problem in a kind of disclaimer, stating that they answer the question: &quot;What is the statistical quality of the sampling based on the configurational states visited in a given set of simulations?&quot; and they of course are aware that &quot;without prior knowledge or assumptions about a landscape, it would appear impossible to know whether every important state has been visited in a given simulation.&quot;&lt;/p&gt;&#xA;" OwnerUserId="4520" LastActivityDate="2013-06-10T08:57:46.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7597" PostTypeId="1" CreationDate="2013-06-10T10:03:22.220" Score="2" ViewCount="1533" Body="&lt;p&gt;I'm trying to understand and implement panel methods for a two-dimensional airfoil.&#xA;I haven't found yet a very detailed explanation on how to implement it, and there are  some things I don't' still understand.&#xA;Here it is how I would implement it, I know it is only one of many possible implementations, please let me know if it's correct and if it would work.&#xA;I need just to compute the velocity field data of an airfoil in a flow, so I think I can use only vortices, without the need of sources, sinks and doublets.&#xA;I describe the airfoil as a set of panels; then I place a vortex of unknown strength in the center of each panel. Then I create a system of equations to obtain the unknown strengths. I have to impose that the velocity field created by every vortex plus the flow has a zero component perpendicular to every panel, calculated at the center of the panel. Now I still need an extra condition, the Kutta condition, that states that the velocities above and below the trailing edge must be the same. So i obtain the unknown strengths by solving the system. But I have some doubts on the Kutta condition. Can I implement this condition just by imposing that the velocities are the same in the two control points that are the closest to the trailing edge?&lt;br&gt;&#xA;And is it necessary to impose that the vortices in the lower airfoil edge rotate counterclockwise and those on the upper edge rotate clockwise, or it just results automatically from the solution of the system? Thank you.       &lt;/p&gt;&#xA;" OwnerUserId="4521" LastEditorUserId="-1" LastEditDate="2013-06-10T14:12:01.967" LastActivityDate="2015-06-08T23:55:45.750" Title="Vortex Panel Method implementation" Tags="|fluid-dynamics|linear-system|laminar-flow|vorticity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7598" PostTypeId="2" ParentId="7595" CreationDate="2013-06-10T12:27:26.550" Score="2" Body="&lt;p&gt;A rather straight forward method would be to use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Inverse_iteration&quot; rel=&quot;nofollow&quot;&gt;Inverse Power Method&lt;/a&gt; with $\mu = 0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method will converge to the eigenvalue of A with the smallest magnitude.  One thing you want to be careful of is if the smallest eigenvalue isn't very close to 0 in magnitude, or you have several with the same magnitude, all of which are the minimum, you could end up with slow convergence or oscillations.  Usually these shortcomings could be dealt with by changing $\mu$ every few iterations, however this would require refactoring the matrix.  If its just slow convergence, and you don't want to refactor the matrix, you can just throw more iterations at it.  If there are multiple eigenvalues with the same magnitude, you will have to change $\mu$ and refactor atleast once, which may or may not be worth it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Luckily, trying this, even with your own implementation is a matter of a small handful of lines of code, so you can try it, see how it converges, and see if it will work in your case.  &lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-06-10T12:27:26.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7599" PostTypeId="2" ParentId="7595" CreationDate="2013-06-10T12:46:47.860" Score="3" Body="&lt;p&gt;Use Arnoldi iteration on inv(A) and it should converge quickly to the smallest eigenpairs of A. ARPACK (and its reverse communication interface) would probably be the easiest route to flesh out working code quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Just to clarify, you don't actually compute inv(A), you just apply inv(A) to a vector x whenever ARPACK asks you to do a matvec (by applying L\D\U\x, for instance).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2013-06-10T12:46:47.860" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7600" PostTypeId="1" CreationDate="2013-06-10T13:55:24.607" Score="2" ViewCount="481" Body="&lt;p&gt;I'm modelling a tidal harbour in Ansys Fluent and want to include a turbulence model in my simulation. I want to predict the water movements around a pontoon which is situated in the harbour. Which turbulence model should I use, a $k$-$\varepsilon$ model or a $k$-$\omega$ model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The harbour is modelled as a rectangular ($120\times 80\times 6$ m LxWxH) box and the water flows in through an entrance channel ($40 \times 10 \times 6$ m). To simulation flow around a pontoon, a rectangular box ($40\times 3\times 5$ m) is placed at the long side of the harbour and water can flow around an underneath the pontoon. Velocities are 0.3 m/s and lower.&lt;/p&gt;&#xA;" OwnerUserId="4326" LastEditorUserId="348" LastEditDate="2013-06-12T16:06:54.583" LastActivityDate="2013-06-12T16:06:54.583" Title="Which turbulence model: k-epsilon or k-omega?" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7601" PostTypeId="1" AcceptedAnswerId="7604" CreationDate="2013-06-10T14:27:43.543" Score="13" ViewCount="1444" Body="&lt;p&gt;I know that ATLAS is able to optimize itself for the machine it is compiled on and thus maximum benefits are found by compiling from source. Is there any benefit to compiling LAPACK from source? It would be much easier to just install the prebuilt package.&lt;/p&gt;&#xA;" OwnerUserId="4524" LastEditorUserId="9" LastEditDate="2013-06-10T22:26:55.343" LastActivityDate="2013-06-11T10:15:29.150" Title="Is there any benefit to compiling LAPACK from source versus installing the prebuilt package from Ubuntu?" Tags="|performance|lapack|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7602" PostTypeId="2" ParentId="7601" CreationDate="2013-06-10T14:36:58.780" Score="6" Body="&lt;p&gt;In my experience, the best way to use blas/lapack on recent versions of ubuntu is to use the packaged openblas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, I mostly use blas/lapack through python numpy/scipy, and using openblas speeds up some of the linear algebra by like 200x vs. the default.  I've tried using custom ATLAS, but it was a huge pain and didn't give much if any speedup vs. openblas, but I might have been doing it wrong.&lt;/p&gt;&#xA;" OwnerUserId="3705" LastEditorUserId="3705" LastEditDate="2013-06-10T17:54:48.123" LastActivityDate="2013-06-10T17:54:48.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7603" PostTypeId="2" ParentId="7592" CreationDate="2013-06-10T16:37:34.030" Score="1" Body="&lt;p&gt;It simply seems that you want to calculate elasticity problem (with large displacement). I don't see a problem in this. Just check &lt;code&gt;hyperelasticity&lt;/code&gt; demo. But be careful - if you take an initial state as reference state notice that your reference state is not stress-free which is quite usual assumption in deriving constitutive relation. After all you would calculate nothing if starting from stress-free state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need (for some purpose) to deform mesh according to calculated displacement there is facility for doing this in FEniCS - check &lt;code&gt;void Mesh::move(const GenericFunction&amp;amp; displacement)&lt;/code&gt;. Then it is natural to use continuous piece-wise linear elements (CG1) for displacement which just defines displacement in all mesh vertices. You just need to reset your reference configuration after each time step.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2013-06-10T16:37:34.030" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7604" PostTypeId="2" ParentId="7601" CreationDate="2013-06-10T17:10:16.473" Score="9" Body="&lt;p&gt;OpenBlas is quite fast, so you can link it to LAPACK. Have you tried precompiled version of LAPACK/BLAS from your CPU vendor? For example AMD ACML (free) or Intel MKL (free on linux for non-commercial and non-academic use)? You simply need to unpack and run install file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion the only advantage of using ATLAS is then when you use some unusual CPU. Otherwise use the one from CPU vendor. Also there are nVIDIA CUDA and AMD OpenCL versions available which really rock.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: remember that you can always build an Ubuntu DEB package from source which is usually much easier than compiling software from source.&lt;/p&gt;&#xA;" OwnerUserId="4334" LastEditorUserId="4334" LastEditDate="2013-06-11T10:15:29.150" LastActivityDate="2013-06-11T10:15:29.150" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7605" PostTypeId="2" ParentId="7576" CreationDate="2013-06-10T18:33:18.913" Score="2" Body="&lt;p&gt;Reshaping arrays is something that's a bit Matlab-specific -- most other environments (including Trilinos, PETSc, ...) allow you to create a vector or matrix but not let you re-interpret it as something else. You need to copy the elements by hand into a matrix of different size.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-10T18:33:18.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7606" PostTypeId="2" ParentId="7601" CreationDate="2013-06-10T20:02:03.447" Score="7" Body="&lt;p&gt;The repository package is not safe to use with threading due to the way it was compiled. I reported the bug on the &lt;a href=&quot;http://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=2&amp;amp;t=1930&quot;&gt;Lapack forum&lt;/a&gt;, but it will take a long time for workarounds or solutions to trickle down into the repository. If you compile it yourself, be sure to add the &quot;-frecursive&quot; to gfortran.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-06-10T20:02:03.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7607" PostTypeId="2" ParentId="7597" CreationDate="2013-06-11T03:03:17.903" Score="2" Body="&lt;p&gt;I had worked on this problem and prepared a short report which might answer some of your questions. In chapter 1, both lifting and non-lifting configurations are described in the context of cylinder and an airfoil.&#xA;Here is the link to the report:&#xA;&lt;a href=&quot;https://www.dropbox.com/s/zqiawtqrfapgbmf/report.pdf&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/s/zqiawtqrfapgbmf/report.pdf&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eq. (1.26) needs to be implemented to ensure the boundary condition for no-penetration condition and for the Kutta condition, based on Eq. (1.27), you need to add another equation to enforce it. Hope you find it useful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For implementing the Kutta condition, you may also refer slide 25-29 from this presentation: &lt;a href=&quot;http://www.ltas-aea.ulg.ac.be/cms/uploads/Aerodynamics04.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.ltas-aea.ulg.ac.be/cms/uploads/Aerodynamics04.pdf&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="4364" LastActivityDate="2013-06-11T03:03:17.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7608" PostTypeId="1" CreationDate="2013-06-11T03:16:51.903" Score="8" ViewCount="702" Body="&lt;p&gt;Suppose a function $f$ can be computed such that the bound on the relative error is $R$ i.e. $f^-(x) = f(x)(1+r)$ where $f^-$ and $f$  are respectively the &#xA;computed and exact value $f$ and $|r| \leq R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to bound the relative error of the following derivative approximations&#xA;in terms of $h$ and $R$ for a general $f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;    f^{'}(x) &amp;amp;\approx \frac{f(x+h)-f(x-h)}{2h}\\&#xA;    f^{''}(x)&amp;amp;\approx \frac{f(x+h)-2f(x)+f(x-h)}{h^2}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Ralston and Rabinowitz the bounds are given to be $\frac{R}{h}$ and  $\frac{4R}{h^2}$ respectively. But this has not been proven and has been mentioned in passing as part of an explanation about Richardson Extrapolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on its proof?&lt;/p&gt;&#xA;" OwnerUserId="3352" LastActivityDate="2014-11-30T20:10:44.863" Title="Bounding the relative error of derivative given relative error of the function" Tags="|error-estimation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7609" PostTypeId="1" AcceptedAnswerId="7613" CreationDate="2013-06-11T03:30:46.983" Score="2" ViewCount="1230" Body="&lt;p&gt;I want to determine the spectral radius of a large non-symmetric matrix $A$ whose dominant eigenvalues are a pair of complex conjugates.  My first instinct was to use a power iteration with a starting vector $x$ in the complex plane.  I do recall that this method tends to have trouble when the dominant eigenvalue is complex and indeed my numerical experiments seem agree with this assessment.  Is there a way I can modify the power iteration to converge to one of the two dominant eigenvalues in the complex plane?  Is it more advisable to use a arnoldi algorithm to determine the spectral radius?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-06-11T18:29:57.573" Title="Estimating the spectral radius when the dominant eigenvalues are complex conjugates" Tags="|eigenvalues|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7610" PostTypeId="1" CreationDate="2013-06-11T08:45:15.870" Score="2" ViewCount="105" Body="&lt;p&gt;I have been reading about Tetrahedral Mesh Refinement algorithms, but the literature covering this is very wide. My work involves implementation of different 3D computational geometry algorithms, and I have often experienced that an algorithm with very good theoretical complexity may be an implementational nightmare. Apart from the most cited articles (e.g. &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=276894&quot; rel=&quot;nofollow&quot;&gt;Shewchuk et al&lt;/a&gt;, and &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/nme.2318/abstract&quot; rel=&quot;nofollow&quot;&gt;Si&lt;/a&gt;), what algorithms (with references please) are implemented in the open source mesh refinement libraries? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, the dynamical refinement algorithm should be Delaunay, because this allows tetrahedral cell agglomeration into polyhedral cells which then remain convex. &lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2013-06-11T21:51:44.970" Title="Scalable, effective and mesh quality assuring local dynamic tetrahedral mesh refinement algorithm" Tags="|reference-request|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7611" PostTypeId="1" AcceptedAnswerId="7615" CreationDate="2013-06-11T11:09:21.557" Score="3" ViewCount="525" Body="&lt;p&gt;For an implicit scheme I want to solve system $Ax=B$, where $A$ is a non-symmetric square matrix. I want source codes of large sparse matrix solvers (e.g. LU-SGS) to use in my code which is in C language. I have tried preconditioned GMRES code from &lt;a href=&quot;http://www.google.com/url?q=http://people.sc.fsu.edu/~jburkardt/cpp_src/mgmres/mgmres.html&amp;amp;sa=D&amp;amp;sntz=1&amp;amp;usg=AFQjCNFIdnCp_e8EVq5oZW-RGoD1jEfBBA&quot; rel=&quot;nofollow&quot;&gt;this site&lt;/a&gt; but its getting slow and takes huge amount of memory(sometimes program is getting &quot;killed&quot; automatically).&lt;/p&gt;&#xA;" OwnerUserId="3549" LastEditorUserId="4244" LastEditDate="2013-06-11T11:36:30.217" LastActivityDate="2014-12-01T07:01:23.253" Title="Sources to get source codes for sparse matrix solvers (non-symmetric matrix)" Tags="|linear-solver|sparse-matrix|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7613" PostTypeId="2" ParentId="7609" CreationDate="2013-06-11T11:13:00.647" Score="4" Body="&lt;p&gt;The power method indeed does not converge in the presence of multiple dominant eigenvalues of the same magnitude. (If you follow the proof, you can see that the iterates will get closer and closer to the subspace spanned by the dominant eigenvectors, but not to any particular vector in this eigenspace).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A restarted Arnoldi method is in fact the way to go, but it can be quite a bit simplified since you are only interested in the dominant eigenvalue. Recall that the Arnoldi method consists in computing a unitary matrix $Q\in\mathrm{C}^{n\times k}$ (consisting of basis vectors of the Krylov space spanned by $v,Av,A^2v,\dots A^{k-1}v$ for some given vector $v$) and then solving the smaller eigenvalue problem for $Q^*AQ$ (using the fact that it is in upper Hessenberg form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is now to restart every second step, i.e., given an iterate $x^k$, compute an orthonormal basis of the span of $\{x^k, Ax^k\}$ (since $x^k$ is assumed to be normalized, you only need to orthogonalize $Ax^k$ against $x^k$). Since projection methods best approximate the extremal eigenvalues, the eigenvalues of the projected matrix $Q^*AQ$ will converge to the complex conjugate pair of dominant eigenvalues. In fact, since you are only interested in one of the two eigenvalues, you can just take the Rayleigh quotient for $x^k$; the full iteration (started from a normalized complex vector &lt;code&gt;x&lt;/code&gt;) is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = A*x;&#xA;l = w'*x;&#xA;w = w-l*x;&#xA;x = w/norm(w);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;l&lt;/code&gt; converges to one of the dominant eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is Problem P-4.2 in Saad's book &lt;em&gt;Numerical Methods for Large Eigenvalue Problems&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Paul explicitly asked about power methods, but for the sake of later readers I should caution that the above works only if $A$ is real and in fact has a pair of complex conjugate dominant eigenvalues. If this is not known a priori, it's better to do several power iterations (with a real starting vector), orthonormalize the last two iterates, compute the eigenvalues of the projected $2\times 2$ matrix, and repeat if necessary:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k = 1:kmax&#xA;    x = w/norm(w);&#xA;    w = A*x;&#xA;end&#xA;w = w-(x'*w)*x;&#xA;w = w/norm(w);&#xA;Q = [x,w];&#xA;l = max(eig(Q'*A*Q));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Depending on the available routines for dense linear algebra, that might even be faster for the original question as well due to the better convergence of Krylov methods. In fact, if you have access to ARPACK or something equivalent, &lt;code&gt;eigs(A,1)&lt;/code&gt; (or its equivalent) will be pretty hard to beat, especially if you need high accuracy.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-06-11T18:29:57.573" LastActivityDate="2013-06-11T18:29:57.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7614" PostTypeId="2" ParentId="7611" CreationDate="2013-06-11T11:57:34.617" Score="3" Body="&lt;p&gt;You didn't say how big your system of equations is. But if you are interested in trying direct sparse solvers, here are links to two for non-symmetric systems that are known to be of high quality and are written in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/&quot; rel=&quot;nofollow&quot;&gt;SuiteSparse&lt;/a&gt;&#xA;&lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot; rel=&quot;nofollow&quot;&gt;SuperLU&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-06-11T11:57:34.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7615" PostTypeId="2" ParentId="7611" CreationDate="2013-06-11T11:58:40.177" Score="5" Body="&lt;p&gt;I would look at &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot;&gt;PETSc&lt;/a&gt; and &lt;a href=&quot;http://trilinos.sandia.gov/&quot;&gt;Trilinos&lt;/a&gt;. They have tons of different sparse solvers available.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-11T11:58:40.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7616" PostTypeId="1" AcceptedAnswerId="7620" CreationDate="2013-06-11T10:47:29.487" Score="9" ViewCount="7549" Body="&lt;p&gt;I have one algorithm that generates a feasible solution to a linear programming problem. However, it is very likely that this is not a corner point. This makes it not suitable for direct use as an initial feasible solution for a bounded Simplex solver. How can I efficiently find a corner point from this solution that I can use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, how can I start the Simplex method from a feasible internal point?&lt;/p&gt;&#xA;" OwnerUserId="4530" OwnerDisplayName="Dylan" LastEditorUserId="127" LastEditDate="2013-06-20T12:55:38.540" LastActivityDate="2014-11-23T17:12:01.850" Title="How to start the Simplex method from a feasible internal point?" Tags="|optimization|linear-programming|" AnswerCount="4" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7617" PostTypeId="1" AcceptedAnswerId="7618" CreationDate="2013-06-11T15:23:04.057" Score="3" ViewCount="1163" Body="&lt;p&gt;I use matlab/fortran to deal with some large eigenvalue problem (FEM type).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I use spalloc and eigs to declare and solve the problem in Matlab,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;could anyone give some advice on how can I do it in Fortran? any routine like spalloc/eigs?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="1552" LastEditDate="2013-06-11T17:51:43.540" LastActivityDate="2013-06-11T18:03:20.720" Title="Programming with Fortran, large eigen value problem" Tags="|fortran|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7618" PostTypeId="2" ParentId="7617" CreationDate="2013-06-11T16:16:44.377" Score="4" Body="&lt;p&gt;If your matrices are dense, then &lt;a href=&quot;http://www.netlib.org/lapack/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAPACK&lt;/a&gt; is the way to go.  If they are sparse, then &lt;a href=&quot;http://forge.scilab.org/index.php/p/arpack-ng/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARPACK&lt;/a&gt; is often better suited.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no standard equivalent of spalloc: LAPACK only supports dense matrices and ARPACK lets you store the data however you like, you just compute the action $A x$ for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also want to check out this stackoverflow &lt;a href=&quot;https://stackoverflow.com/questions/3293282/can-i-use-lapack-for-calculating-the-eigenvalues-and-eigenvectors-of-large-spars&quot;&gt;question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-06-11T16:16:44.377" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7619" PostTypeId="2" ParentId="7617" CreationDate="2013-06-11T18:03:20.720" Score="3" Body="&lt;p&gt;If you don't need (distributed memory) parallelization, LAPACK or ARPACK depending on the sparsity of your problems are good choices, as already answered by Max Hutchinson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Large parallel eigenvalue solvers are a topic for ongoing research. For sparse problems, check out e.g. &lt;a href=&quot;http://www.grycap.upv.es/slepc/&quot; rel=&quot;nofollow&quot;&gt;SLEPc&lt;/a&gt;, for dense problems &lt;a href=&quot;http://elpa.rzg.mpg.de/&quot; rel=&quot;nofollow&quot;&gt;ELPA&lt;/a&gt; could be interesting for you if the old workhorse &lt;a href=&quot;http://www.netlib.org/scalapack/&quot; rel=&quot;nofollow&quot;&gt;ScaLAPACK&lt;/a&gt; doesn't cut it. The &lt;a href=&quot;http://elpa-lib.fhi-berlin.mpg.de/wiki/index.php/Main_Page&quot; rel=&quot;nofollow&quot;&gt;ELPA wiki&lt;/a&gt; contains links to a few other parallel eigensolvers as well.&lt;/p&gt;&#xA;" OwnerUserId="1077" LastActivityDate="2013-06-11T18:03:20.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7620" PostTypeId="2" ParentId="7616" CreationDate="2013-06-11T21:47:48.230" Score="6" Body="&lt;p&gt;Every book on linear optimization explains the simplex method as a two-stage algorithm: the first for finding a feasible corner as a starting point, and the second for finding the optimum. The first uses a dual problem. Take a look at &lt;em&gt;D. Bertsimas and J. N. Tsitsiklis: &quot;Introduction to linear optimization&quot;&lt;/em&gt;, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason one needs the two-phase approach is that it is typically difficult to find &lt;em&gt;any&lt;/em&gt; feasible point -- in high dimensional spaces, the feasible set is very small compared to, say, the unit box. From your question, it sounds as if you have a different way to find at least one feasible point, and in that case it may be possible to generate a vertex of the feasible polyhedron from this point. One idea would be to use the following approach: each inequality constraint represents a half space separated by a hyperplane. Given a feasible point $x^\ast$, find the $n+1$ hyperplanes that are closest to $x^\ast$ and take their intersection. Intuitively, this vertex should be feasible, though I will admit that one would need to think about this a bit more.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-06-13T20:07:39.947" LastActivityDate="2013-06-13T20:07:39.947" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7621" PostTypeId="2" ParentId="7610" CreationDate="2013-06-11T21:51:44.970" Score="1" Body="&lt;p&gt;In the finite element community, many of us have come to the conclusion that it is easiest to just refine cells by bisection (which in the context of tetrahedra would mean subdividing each tetrahedron into 5 congruent ones) and then deal with the fact that there are vertices of the child cells that are not vertices of the neighboring cells if they happen not to be refined as well (these vertices are called &quot;hanging nodes&quot;). I don't know if that could apply to your problem as well, but it's proven to be simpler than the myriad ways of doing local mesh refinement while trying to guarantee shape regularity of cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in 2d, the algorithm to choose is red-green refinement. I imagine that there is an analogue for tetrahedra as well, but I don't know for sure.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-11T21:51:44.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7622" PostTypeId="1" CreationDate="2013-06-11T23:02:29.833" Score="3" ViewCount="2759" Body="&lt;p&gt;Is there a fast nearest neighbor search algorithm that generates the nearest neighbors, not based on Euclidean distances but based on geographic distances over a set of latitudes/longitudes. The fast nearest neighbor search based on Euclidean distances though is based on spatial indexing using R-Trees, K-D Trees etc. What about for Lat/Long distances?&lt;/p&gt;&#xA;" OwnerUserId="1934" LastActivityDate="2013-06-12T08:56:42.467" Title="Fast nearest neighbor search, Latitude Longitude" Tags="|algorithms|computational-geometry|complexity|nearest-neighbors|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7623" PostTypeId="1" AcceptedAnswerId="7629" CreationDate="2013-06-12T01:37:25.473" Score="8" ViewCount="1157" Body="&lt;p&gt;Following &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3540034404&quot; rel=&quot;noreferrer&quot;&gt;Hundsdorfer approach&lt;/a&gt; the finite volume discretisation of the advection-diffusion equation (conservative form) on &lt;em&gt;non-uniform&lt;/em&gt; cell centered grid can be written as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;w_j^{\prime} = \frac{w_{j-1}}{h_j}\left( \frac{ah_j}{2h_{-}} + \frac{d}{h_{-}} \right) - \frac{w_j}{h_j}\left( \frac{a}{2}\left[ \frac{h_{j-1}}{h_{-}} - \frac{h_{j+1}}{h_{+}} \right] + d\left[-\frac{1}{h_{-}} - \frac{1}{h_{+}} \right]\right) +  \frac{w_{j+1}}{h_j}\left(- \frac{ah_j}{2h_{+}} + \frac{d}{h_{+}} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having trouble implementing this stencil because the edge terms have a dependence on the &lt;strong&gt;position&lt;/strong&gt; of ghost cells which are outside of the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we write the equation at the left hand side boundary (this corresponds to cell centre $x=x_1$),&#xA;$$&#xA;w_1^{\prime} = - \frac{w_1}{h_1}\left( \frac{a}{2}\left[ \frac{h_{0}}{h_{-}} - \frac{h_{2}}{h_{+}} \right] + d\left[-\frac{1}{h_{-}} - \frac{1}{h_{+}} \right]\right) +  \frac{w_{2}}{h_1}\left(- \frac{ah_1}{2h_{+}} + \frac{d}{h_{+}} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;N.B.&lt;/strong&gt; The terms in the first bracket depend on the position of the ghost point. To make this clear see the diagram below and notice that the $h_0$ and $h_{-}$ term stray outside of the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/95PqK.png&quot; alt=&quot;Cell centered grid at the left hand side boundary.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I free to choose the position and size of the ghost cell? For example,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can I set $h_{-}\equiv h_{+}$? &lt;/li&gt;&#xA;&lt;li&gt;Or, can I set $h_{-}=0$? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Regarding the latter, does it make sense to have a cell with zero volume? This would mean that the cell &quot;center&quot; and vertex coincide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of the variable at the ghost point ($w_0$) is never used in numerical methods (this is clear!). However, the stencil above would seem to imply that we &lt;strong&gt;must&lt;/strong&gt; at least choose a location and size for the ghost cell and this &lt;strong&gt;must&lt;/strong&gt; be included in the discretisation. Is that assessment correct?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-06-12T01:51:06.027" LastActivityDate="2013-06-12T08:08:55.527" Title="Are we free to choose the position of ghost cells on a non-uniform finite-volume mesh?" Tags="|boundary-conditions|finite-volume|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7624" PostTypeId="1" CreationDate="2013-06-12T01:49:39.770" Score="14" ViewCount="942" Body="&lt;p&gt;Has anyone here used double precision scientific computing with new generation (e.g. K20) GPUs through Python?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that this technology is rapidly evolving, but what is the best way to do this currently?  GPU is out of scope for the popular scientific Python libraries numpy and scipy, and I had wanted to use theano but it seems to use only float32 precision for GPU.  I am aware that google can provide search results for python gpu, but I am hoping for more insight than a list of projects which may or may not be on their way to meet their maker.&lt;/p&gt;&#xA;" OwnerUserId="3705" LastActivityDate="2013-12-05T23:40:19.607" Title="Scientific computing with Python with modern GPUs with double precision" Tags="|python|gpu|precision|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7625" PostTypeId="1" CreationDate="2013-06-12T03:17:28.813" Score="3" ViewCount="302" Body="&lt;p&gt;I am not trying to start a Language flame war here. I merely want to ask a simple question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a good resource for comparisons for languages for scientific computing?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By comparisons, I mean on the grounds of:&#xA;Number of stable libraries which can be used.&#xA;Programming Effort&#xA;Speed&#xA;etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a couple of comparisons online but all seemed to be relevant to general computer programmers (not for Numerical Computing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am asking the question for general scientific computing, not necessarily for matrix algebra. For instance, this also includes statistics, engineering etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not trying to ask a &quot;which language should I learn&quot; type of question. I am asking for good comparisons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am especially interested in C,Java,C++,C# and Python. The goal is to build usable code in variety of scientific applications with different hardwares (including Android) using as many stable libraries along the way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2486/parallel-scientific-computation-software-development-language&quot;&gt;Parallel Scientific Computation Software Development Language?&lt;/a&gt; but it is too focussed on Matrix algebra. &lt;/p&gt;&#xA;" OwnerUserId="4534" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-06-12T16:19:57.210" Title="A Comparison of Languages" Tags="|languages|" AnswerCount="3" CommentCount="5" ClosedDate="2013-06-13T05:35:32.193" ContentLicense="CC BY-SA 3.0" />
  <row Id="7626" PostTypeId="1" CreationDate="2013-06-12T05:22:18.540" Score="11" ViewCount="1356" Body="&lt;p&gt;I am currently developing a domain decomposition method for the solution of the scattering problem. Basically I am solving a system of Helmholtz BVPs iteratively. I discretize the equations using finite element method over triangular or tetrahedral meshes. I'm developing the code towards my Phd thesis. I am aware of some of the existing finite element libraries out there such as deal.ii or DUNE and though I think they are great, with inspirational design and API, for learning purposes I wanted to develop my own little application from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am at a point where I have my serial versions running and now I want to parallelize them. After all, it is one of the strengths of the domain decomposition framework to formulate algorithms that are easy to parallelize, at least in principle. In practice however, there are many details one must consider. Mesh management is one of them. If the applications is to achieve high resolution while scaling well to many CPUs the replication of an entire mesh on every CPU is inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to ask those developers who work on similar applications in high performance computing environments how they deal with this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is p4est library for distributed mesh management. I do not need AMR so it might be an overkill since I'm only interested in using uniform meshes and I'm not sure if it can refine triangular meshes. I could also simply create a uniform mesh then feed it into one of the mesh partitioners and do some post processing of the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest approach seems to create a separate file for each partition containing mesh information relevant only to that particular partition. This file would be read by a single CPU which would be responsible for assembly of the discrete system on that portion of the mesh. Of course, some global partition connectivity/neighborhood information would also need to be stored in a file read by all CPUs for inter process communication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other approaches are out there? If some of you could share, what are some of the commonly used methodologies in the industry, or government research institutions related to handling this issue? I am quite new to programming a parallel finite element solver and I wanted to get a feel for whether or not I'm thinking about this problem correctly and how others are approaching it. Any advice or pointers to relevant research articles would be greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="4535" LastEditorUserId="3481" LastEditDate="2013-06-12T15:15:14.750" LastActivityDate="2014-03-14T22:10:59.443" Title="Best Methodologies for Managing a Mesh in Parallel Finite Element Computation?" Tags="|finite-element|parallel-computing|mesh-generation|domain-decomposition|mesh|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7627" PostTypeId="1" AcceptedAnswerId="7647" CreationDate="2013-06-12T05:43:42.790" Score="4" ViewCount="128" Body="&lt;p&gt;I want to incorporate matrix solver into my program. Is it possible to separate out any library of matrix solver from PETSc? I am working in C language. It will be very helpful if I get any library without installing PETSc. If I get it, then I will just add those source codes and will use it.&lt;/p&gt;&#xA;" OwnerUserId="4107" LastEditorUserId="1552" LastEditDate="2013-06-12T17:22:53.423" LastActivityDate="2013-06-12T22:30:43.760" Title="Installing only a portion of PETSc" Tags="|petsc|libraries|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7628" PostTypeId="1" AcceptedAnswerId="7667" CreationDate="2013-06-12T08:02:41.147" Score="10" ViewCount="8111" Body="&lt;p&gt;What is the difference between explicit FEM and implicit FEM exactly? According to the post &lt;a href=&quot;http://www.imechanica.org/node/5396&quot;&gt;here&lt;/a&gt;, it seems that the only difference is whether implicit or explicit time integration is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I remember from one book that I read, implicit FEM is where the mass is not lumped to the nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the exact definitions of explicit and implicit FEM?&lt;/p&gt;&#xA;" OwnerUserId="1976" LastEditorUserId="1748" LastEditDate="2013-06-12T18:12:08.733" LastActivityDate="2013-06-14T21:30:40.357" Title="What is the difference between implicit FEM and explicit FEM?" Tags="|finite-element|implicit-methods|explicit-methods|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7629" PostTypeId="2" ParentId="7623" CreationDate="2013-06-12T08:08:55.527" Score="4" Body="&lt;p&gt;Including an actual ghost cell for FVM discretization is more or less a matter of convenience. For instance, if the boundary condition on the wall is a Neumann type, you actually do not need to discretize in the $x^-$ direction since the flux is given by the boundary condition and is known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the boundary condition is of Dirichlet type, what you need is to approximate the flux using the interior point(s) and the value on the boundary. In the simplest case, this would result from a simple linear extrapolation using $x_1$ and the wall point $x_w$. This effectively means that the value of the flux will be independent of the location of the ghost cell (slope of the extrapolating function for a simple linear method).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this observation, I'd say anything but $h^- = 0$ would be good. I always personally choose $h^- = h^+$ as that makes life easier!&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-06-12T08:08:55.527" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7630" PostTypeId="2" ParentId="7622" CreationDate="2013-06-12T08:56:42.467" Score="3" Body="&lt;p&gt;On a sphere, you can use the nearest neighbor list from the Euclidean distance to get the correct points ordered by distance, because the Euclidean distance is less or equal to the geodesic distance. Once you have that list of points you can simply calculate the geodesic distance for the point of interest. So the fast k-nearest neighbor algorithms based on the Euclidean metric can be used.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2013-06-12T08:56:42.467" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7631" PostTypeId="2" ParentId="7625" CreationDate="2013-06-12T10:20:39.363" Score="2" Body="&lt;p&gt;It is possible to do any feasible computations in any Turing-complete language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the comparison is not between languages but between compilers, runtimes, and library sets. This assumes that those who devise and implement the algorithm are competent enough in the chosen language. The runtimes and libraries (as-shipped) should be using the lowest complexity algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://benchmarksgame.alioth.debian.org/&quot; rel=&quot;nofollow&quot;&gt;The Benchmarks' Game&lt;/a&gt; is a fantastic resource that could be used for educational purposes. Really an eye-opener for performance-obsessed types (and if you are dealing with realistically-sized problems, you &lt;strong&gt;are&lt;/strong&gt; one of them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However speed is not everything - the real devil lies in small details (&lt;strong&gt;correctness&lt;/strong&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How are floating-point numbers parsed, rounded, and output?&lt;/li&gt;&#xA;&lt;li&gt;Is there any library function to force a specific rounding mode?&lt;/li&gt;&#xA;&lt;li&gt;Are math functions working for corner cases (plus and minus zero, etc.)?&lt;/li&gt;&#xA;&lt;li&gt;Is there any handling for underflows?&lt;/li&gt;&#xA;&lt;li&gt;Do built-in facilities for handling complex numbers conform to accepted practices and standards?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Must say your question is misleading: &lt;strong&gt;there are precious few things in statistics and engineering and whatever other disciplines that can be accomplished without matrix computations&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2155" LastActivityDate="2013-06-12T10:20:39.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7632" PostTypeId="2" ParentId="7628" CreationDate="2013-06-12T11:31:36.023" Score="7" Body="&lt;p&gt;Yes it is the time integration but it also means that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You have to solve a linear system of the type Ax=b in the implicit scheme where as in the explicit scheme you do not, as the lumped mass matrix only has diagonal entries so inv(M) is trivial.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your time step in the explicit scheme is limited by the CFL criteria for stability. Implicit schemes are unconditionally stable (though in practice you still need a reasonable time step for accurary)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Typically problems where inertial effects are important (e.g., wave propagation) are solved by explicit schemes where as quasi-static problems usually use an implicit scheme. However there are exceptions.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2013-06-12T11:31:36.023" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7633" PostTypeId="1" AcceptedAnswerId="7636" CreationDate="2013-06-12T11:47:07.823" Score="0" ViewCount="50" Body="&lt;p&gt;I have a set of users who have won a game&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;('jim', 12), ('james', 54), ('john', 76), ('dave', 22), ('garry', 34), ('stuart', 16)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to award them a share of points based on their position in the game on a sliding scale. The 'global pot' is $100 and I would like the winner to get the most with other users getting less based on a sliding scale to their position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anyone know of a 'scoring' algorithm to do this...? Similar to ones used in poker but with a fixed pot.&lt;/p&gt;&#xA;" OwnerUserId="4537" LastEditorUserId="3481" LastEditDate="2013-06-12T15:16:58.267" LastActivityDate="2013-06-12T15:16:58.267" Title="algorithm to assign points to winning users" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7634" PostTypeId="2" ParentId="7626" CreationDate="2013-06-12T11:53:50.750" Score="7" Body="&lt;p&gt;If you are not using AMR and do not want to scale beyond 1K-4K cores then simply do this.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Rank 0 reads the entire mesh and partitions it using METIS/Scotch etc. (Note: This is a serial operation).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rank 0 broadcasts the element/node partitioning info to all other ranks and frees the memory (used to store the mesh)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;All ranks read the nodes/elements they own (including ghost nodes) from the same input file (Note: 2000 ranks accessing the same input file might sound slow but is not in practice, though it may be bad for the file system but then we are doing it only once).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;All ranks need to create the local to global node/element/dof mappings for application of BCs and assembling of matrices and renumber the nodes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;After everything is said and done all data on a rank will be local so you should be able to scale well (memory wise). I do all this in about 100 lines (see lines 35-132 &lt;a href=&quot;http://code.google.com/p/defmod/source/browse/src/main.F90&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) in a small code of mine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if your mesh is too large (e.g., &gt;100-250 million elements) that you cannot partition it using METIS on a single node and need ParMETIS/PT-Scotch then you have the additional work of partitioning it in parallel before all cores/ranks can read it. In such a scenario it might be easier to keep the partitioning phase separate from the main code for logistical reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw AMR libs usually dont do tets. Also PETSc is good choice for parallelization of your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Also see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10981/meshes-in-codes-that-do-amr&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10943/which-novel-data-structures-are-used-in-adaptive-fem&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-03-13T13:16:06.867" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7635" PostTypeId="2" ParentId="7625" CreationDate="2013-06-12T11:55:42.507" Score="0" Body="&lt;p&gt;I would reccomend such an approach: you should learn one general purpose (generating fast code - compiled) language like C/C++ or Fortran. My favourite is C++. On each of those you can use easily dll's, you have BLAS/LAPACK libs, also there are CUDA (Fortran and C) or OpenCL (looks like C) extensions for computing on GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand it is convinient to use a so-called script language that is oriented for scientific computing, so you could easily check if something will work at all, to make nice plots easily and so on. I have used Scilab, and now I am trying to learn Mathematica for this purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My choice is: C++ (a lot of available numerical (BLAS/LAPACK, GNU math library, CGAL, ...) and visualization (VTK) libs. For high efficiency work (not HPC computing) Mathematica or Maple is quite useful but is sometimes annoying, but really powerful.&lt;/p&gt;&#xA;" OwnerUserId="4334" LastActivityDate="2013-06-12T11:55:42.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7636" PostTypeId="2" ParentId="7633" CreationDate="2013-06-12T12:02:14.657" Score="0" Body="&lt;p&gt;You could easily use their fraction of the total points as the fraction of the pot to win. I.e., divide each score by the total of all the scores then multiply that by $100.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-12T12:02:14.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7638" PostTypeId="1" AcceptedAnswerId="7639" CreationDate="2013-06-12T15:09:51.123" Score="14" ViewCount="1606" Body="&lt;p&gt;I am comparing two iterative methods for inverting random square matrices.  Since the matrices are random, every test case takes both different amounts of iterations and different elapsed times.  My question is, on top of mean CPU time, is the mean value of iterations taken by both the methods useful information for comparing the methods.&lt;/p&gt;&#xA;" OwnerUserId="1580" LastEditorUserId="1552" LastEditDate="2013-06-12T15:26:47.613" LastActivityDate="2013-06-21T15:32:00.313" Title="Comparison of iteration methods: number of iterations vs. cpu time" Tags="|performance|benchmarking|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7639" PostTypeId="2" ParentId="7638" CreationDate="2013-06-12T15:32:55.630" Score="13" Body="&lt;p&gt;In general, both methods of performance comparisons have their place. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Comparing &lt;strong&gt;cpu time&lt;/strong&gt; is in a sense the most interesting metric, because at the end of the day you are really interested in which of the methods is faster. (But make sure that the termination criteria are comparable; e.g., that both methods yield an approximation with the same accuracy). The drawback is that this only tells you which method (and more importantly, which &lt;em&gt;implementation&lt;/em&gt;) is faster on the machine you performed the tests on. There is no guarantee that a different machine with different architecture or software would pick the same winner.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Comparing &lt;strong&gt;iteration numbers&lt;/strong&gt;, on the other hand, is machine independent, but potentially misleading if the two methods have very different iterations - in this case the method with fewer but more expensive iterations might not be preferable (e.g., Newton vs. gradient methods for optimization if you only need very low accuracy).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, yes, it makes sense to give both numbers [1], and I've frequently seen it done in publications. There is also a third option:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Comparing &lt;strong&gt;numbers of elementary operations&lt;/strong&gt;. If both iterations consist of the same kind of suitably expensive operation, but require a different number (possibly not even the same number in each iteration), it makes sense to count the &lt;em&gt;total&lt;/em&gt; number of these operations. In your case, a likely candidate would be matrix-vector or matrix-matrix multiplications.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;[1] Definitely present statistics over multiple runs; if you show means, don't forget to include standard deviations as well.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-06-12T17:35:20.737" LastActivityDate="2013-06-12T17:35:20.737" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7642" PostTypeId="2" ParentId="7638" CreationDate="2013-06-12T16:06:28.983" Score="4" Body="&lt;p&gt;I find the number of iterations to be a misleading metric because it suggests &quot;speed&quot; when it is not. For a simple example of comparing a few different preconditioners that shows this difference, see here: &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_6.html#Possibilitiesforextensions&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_6.html#Possibilitiesforextensions&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-12T16:06:28.983" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7643" PostTypeId="2" ParentId="7628" CreationDate="2013-06-12T16:09:21.383" Score="5" Body="&lt;p&gt;The terms &quot;explicit&quot; and &quot;implicit&quot; arise in the time discretization, and these terms are already used in the literature on ordinary differential equations (i.e., they are not specific to the finite element method). It would be worth taking a look at a book discussing the numerical solution of ODEs, e.g. Hairer &amp;amp; Wanner.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-12T16:09:21.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7644" PostTypeId="2" ParentId="7627" CreationDate="2013-06-12T16:11:13.093" Score="4" Body="&lt;p&gt;No. Like many other libraries, you will spend far more time trying to isolate a piece than to simply use the whole thing. Why not just install PETSc somewhere and use it as-is?&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-12T16:11:13.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7645" PostTypeId="2" ParentId="7625" CreationDate="2013-06-12T16:19:57.210" Score="3" Body="&lt;p&gt;My advise to students is always this: The most precious resource of all is your own time. If there is a programming language &lt;em&gt;A&lt;/em&gt; that allows you to write a program in half the time it would take you with language &lt;em&gt;B&lt;/em&gt;, but the program is 20% slower, then this is still almost always a win, unless you need to run the program for months.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the reason why most modern scientific computing libraries are written in C++ and no longer in Fortran or plain C: because it is so much faster to write and debug code in C++ than in Fortran, and because of the huge number of other libraries you can then use (including, in particular, the C++ standard library) and that make programming so much faster.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-12T16:19:57.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7646" PostTypeId="1" AcceptedAnswerId="7651" CreationDate="2013-06-12T16:54:23.760" Score="4" ViewCount="1682" Body="&lt;p&gt;I just started working on some Biophysics research, and I was looking around for a good MD library to use. There is one right now at the university set up in C, with openMC parallelization, but the code is so hacked together at this point that it is hard to tell what is going on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have access to a computing cluster, and several nodes do have Nvidia Tesla GPUs attached. So something that could distribute the load and use CUDA would be a huge help. The actual simulations that I am going to try to run are pretty simple - diffusion of some set of particles, with the twist that there are some other objects that can act between the particles, causing extra non-Brownian forces. I know that this can cause problems in some MD libraries. Right now, a single simulation is run on a node at a time, and then we run the N simulations across all available nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A visualization to go with the simulation would also be incredibly helpful as well, since making movies of the simulations in some cases would be useful in talks, etc.&lt;/p&gt;&#xA;" OwnerUserId="4545" LastEditorUserId="276" LastEditDate="2013-06-13T03:56:32.407" LastActivityDate="2015-01-24T20:40:16.980" Title="Finding the correct Molecular Dynamics library" Tags="|c++|hpc|simulation|molecular-dynamics|computational-biology|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7647" PostTypeId="2" ParentId="7627" CreationDate="2013-06-12T22:30:43.760" Score="6" Body="&lt;p&gt;You can configure with separate libraries (&lt;code&gt;--with-single-library=0&lt;/code&gt;) and link only the ones you need (e.g., &lt;code&gt;-lpetscmat -lpetscvec -lpetscsys&lt;/code&gt;), but this is generally a waste of effort. If you use static libraries, then only the parts you reference go into the binary (if you're trying to squeeze the last megabyte out of a memory-constrained environment). PETSc compiles in about one minute, so build time is not a reason. We don't distribute stand-alone sub-components only because it's simpler to build everything and configure is really the hard part of distribution.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-06-12T22:30:43.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7648" PostTypeId="2" ParentId="7646" CreationDate="2013-06-13T00:57:17.770" Score="3" Body="&lt;p&gt;Since you are specifically looking for a &lt;strong&gt;library&lt;/strong&gt;, you might be interested in &lt;a href=&quot;https://simtk.org/home/openmm&quot; rel=&quot;nofollow&quot;&gt;OpenMM&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two popular visualization packages are &lt;a href=&quot;http://www.pymol.org/&quot; rel=&quot;nofollow&quot;&gt;PyMol&lt;/a&gt; and &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow&quot;&gt;VMD&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-06-13T00:57:17.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7649" PostTypeId="2" ParentId="7608" CreationDate="2013-06-13T05:56:20.313" Score="-1" Body="&lt;p&gt;To answer your direct question (and not cater for Brian Borchers comment re truncation error):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the definition you have for $f^-$, its relative error $|(f^--f)/f|\leq R$, and your definition doesn't say it explicitly, but $r$ isn't constant, so the relative error in $|f^-(x+h)-f^-(x-h)| $ is $\leq 2R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads directly to the relative errors for $f^{'-}$ to be $R/h$ and similarly for $f^{''-}$ to be $4R/h^2$.&lt;/p&gt;&#xA;" OwnerUserId="1523" LastEditorUserId="1523" LastEditDate="2013-06-13T06:01:27.667" LastActivityDate="2013-06-13T06:01:27.667" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7650" PostTypeId="1" AcceptedAnswerId="7663" CreationDate="2013-06-13T08:23:28.540" Score="17" ViewCount="17212" Body="&lt;p&gt;Following from my &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7623/are-we-free-to-choose-the-position-of-ghost-cells-on-a-non-uniform-finite-volume&quot;&gt;previous question&lt;/a&gt; I am trying to apply boundary conditions to this non-uniform finite volume mesh,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tPF6Y.png&quot; alt=&quot;Left hand side boundary include ghost cell.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to apply a Robin type boundary condition to the l.h.s. of the domain ($x=x_L)$, such that,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sigma_L = \left( d u_x + a u \right) \bigg|_{x=x_L}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\sigma_L$ is the boundary value; $a, d$ are coefficients defined on the boundary, advection and diffusion respectively; $u_x = \frac{\partial u}{\partial x}$, is the derivative of $u$ evaluated at the boundary and $u$ is the variable for which we are solving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Possible approaches&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of two ways to implement this boundary condition on the above finite volume mesh:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A ghost cell approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Write $u_x$ as a finite difference including a ghost cell.$$&#xA;\sigma_L = d \frac{u_1 - u_0}{h_{-}} + a u(x_L)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Then use linear &lt;em&gt;interpolation&lt;/em&gt; with points $x_0$ and $x_1$ to find the intermediate value, $u(x_L)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B. Alternatively find $u(x_L)$ by averaging over the cells, $u(x_L) = \frac{1}{2}(u_0 + u_1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either case, the dependence on ghost cell can be eliminated in the usual way (via substitution into the finite volume equation).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An extrapolation approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fit a linear (or quadratic) function to $u(x)$ by using the values at points $x_1, x_2$ ($x_3$). This will provide the value at $u(x_L)$. The linear (or quadratic) function can then be differentiated to find an expression for the value of the derivative, $u_x(x_L)$, at the boundary. This approach does &lt;em&gt;not&lt;/em&gt; use a ghost cell.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Which approach of the three, (1A, 1B or 2) is &quot;standard&quot; or you would recommend? &lt;/li&gt;&#xA;&lt;li&gt;Which approach introduces the smallest error or is the most stable? &lt;/li&gt;&#xA;&lt;li&gt;I think I can implement the ghost cell approach myself, however, how can the extrapolation approach be implemented, does this approach have a name?&lt;/li&gt;&#xA;&lt;li&gt;Are there any stability difference between fitting a linear function or a quadratic equation?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Specific equation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to apply this boundary to the advection-diffusion equation (in conservation form) with non-linear source term,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = -au_x + du_{xx} + s(x,u,t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discretising this equation on the above mesh using the $\theta$-method gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;w_{j}^{n+1} - \theta r_a w_{j-1}^{n+1} - \theta r_b w_{j}^{n+1} - \theta r_c w_{j+1}^{n+1} = w_j^n + (1-\theta) r_a w_{j-1}^n + (1-\theta) r_b w_j^n + (1-\theta) r_c w_{j+1}^n + s(x_j,t_n)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However for the boundary point ($j=1$) I prefer to use a fully implicit scheme ($\theta=1$) to reduce the complexity,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;w_{1}^{n+1} - r_a w_{0}^{n+1} - r_b w_{1}^{n+1} - r_c w_{2}^{n+1} = w_1^n + s_1^n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice the ghost point $w_0^{n+1}$, this will be removed by applying the boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coefficients have the definitions,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r_a = \frac{\Delta t}{h_j}\left( \frac{ah_j}{2h_{-}} + \frac{d}{h_{-}} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r_b = - \frac{\Delta t}{h_j}\left( \frac{a}{2}\left[ \frac{h_{j-1}}{h_{-}} - \frac{h_{j+1}}{h_{+}} \right] + d\left[-\frac{1}{h_{-}} - \frac{1}{h_{+}} \right]\right) &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r_c = \frac{\Delta t}{h_j}\left(- \frac{ah_j}{2h_{+}} + \frac{d}{h_{+}} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the &quot;$h$&quot; variables are defined as in the above diagram. Finally, $\Delta t$ which is the time step (&lt;strong&gt;N.B.&lt;/strong&gt; this is a &lt;em&gt;simplified&lt;/em&gt; case with constant $a$ and $d$ coefficients, in practice the &quot;$r$&quot; coefficients are slightly more complicated for this reason).&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-10-21T05:44:45.800" Title="How should boundary conditions be applied when using finite-volume method?" Tags="|boundary-conditions|finite-volume|numerical-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7651" PostTypeId="2" ParentId="7646" CreationDate="2013-06-13T09:03:15.583" Score="2" Body="&lt;p&gt;Allow me to plug my own library, &lt;a href=&quot;http://mdcore.sourceforge.net&quot; rel=&quot;nofollow&quot;&gt;mdcore&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mdcore is an Open-Source, platform-independent library for Molecular Dynamics simulations written in C. It runs on clusters, multi-cores, and GPUs and can handle any type of inter-particle interaction potential. The code is mainly a test-bed for new algorithms, but it is extremely fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The library is relatively straight-forward to use: You initialize a simulation &lt;code&gt;engine&lt;/code&gt;, insert particles, declare the potentials, and the call &lt;code&gt;engine_step&lt;/code&gt; to compute the forces and trajectories for each time step. In between time steps you can add whatever forces and/or interactions you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another similar tool, with a nice Python interface, is &lt;a href=&quot;http://dirac.cnrs-orleans.fr/MMTK/&quot; rel=&quot;nofollow&quot;&gt;MMTK&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-06-13T09:03:15.583" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7652" PostTypeId="1" CreationDate="2013-06-13T11:08:35.730" Score="2" ViewCount="817" Body="&lt;p&gt;I'm trying to solve this problem using genetic algorithms and am having difficulty choosing the fitness function. My problem is a little different than the original Traveling Salesman Problem, since the population and maybe also the win unit do not necessarily contain all the cities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I have 2 values for each unit: the amount of cities he visits, the total time and the order he visits the cities. I tried 2-3 fitness function but they don't give good solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need an idea of a good fitness function which takes into account the amount of cities he visited and also the total time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: more accurate description&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective of the PTSP is to visit the maximum number of waypoints of the map in the minimum number of time steps. The map takes the form of a two-dimensional board, where ten waypoints are scattered around and multiple obstacles are present. The following image is an example of a map with obstacles and waypoints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4550" LastEditorUserId="4550" LastEditDate="2013-06-13T18:49:15.613" LastActivityDate="2013-06-14T08:55:03.080" Title="Evolutionary algorithm - Traveling Salesman -fitness function" Tags="|algorithms|graph-theory|combinatorics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7653" PostTypeId="1" CreationDate="2013-06-13T14:20:07.440" Score="2" ViewCount="136" Body="&lt;p&gt;I have an issue of finding the cut sets of large voxel sets. The voxels are assumed connected if they touch by face/edge/vertex (can vary), and ideally what I want is given any two members of the set to find a minimal cut set of voxels that separates them into two components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sets are 'large', 10^7 upwards, so I'm maybe looking at some kind of multi-res method? Any pointers be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify, voxels are assumed to exist on a 3D integral lattice, and we call them 'adjacent' if the coordinates vary by 1 in at most 1 (2, or 3) dimensions (for face/edge/vertex connectedness respectively). Voxels v0 and vN are connected if there is a sequence v0,..vi,vi+1,...vN where the vi,vi+1 are adjacent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A component is a maximal set of connected voxels, and a cut set is any subset of a component whose removal will separate the component into 2 or more components. &lt;/p&gt;&#xA;" OwnerUserId="4551" LastEditorUserId="4551" LastEditDate="2013-06-20T12:00:17.080" LastActivityDate="2013-06-20T12:00:17.080" Title="Finding cut sets of very large voxel set" Tags="|algorithms|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="7654" PostTypeId="1" CreationDate="2013-06-13T14:41:52.347" Score="14" ViewCount="447" Body="&lt;p&gt;After doing some research, I've found that most of the literature on operator splitting methods (e.g. Strang Splitting, Fractional Step, etc.) are specifically designed for a standard problem type of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u'=(A+B)u.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my application, I have a slightly different form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Mu'=(A+B)u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $M$ is not the identity operator.  Can the same methodologies be applied when an operator is applied to the $u'$ term?  Are there any other numerical methods to handle this case specific case there $M$ is not the identity operator?  In particular, I'm looking at a case where $M$ is singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Update&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put my particular problem into greater context, my operator is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M=\left[\begin{array}{cc} 0 &amp;amp; 0 \\ I &amp;amp; I\end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u=\left[\begin{array}{c} u_1 \\ u_2 \end{array}\right]$$&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="123" LastEditDate="2015-06-05T18:12:08.500" LastActivityDate="2015-06-05T18:12:08.500" Title="Operator Splitting methods for DAEs" Tags="|operator-splitting|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7655" PostTypeId="2" ParentId="7561" CreationDate="2013-06-13T15:33:11.493" Score="4" Body="&lt;p&gt;A couple of suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Choose $\sigma \sim$ the average distance | random $x$ - nearest $x_i$.&#xA;(A cheap approximation for $N$ points uniformly distributed in the unit cube in $\mathbb{R}^d, d\  2 .. 5$, is&#xA;0.5 / $N^{1/d}$.)&lt;br&gt;&#xA;We want $\phi( |x - x_i| )$ to be large for $x_i$ near $x$,&#xA;small for background noise; plot that for a few random $x$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Shift $K$ away from 0, $K \to K + \lambda I$, $\lambda \sim 10^{-6}$ or so;&#xA;that is, regularize.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Look at the weights from solving $(K + \lambda I) w = f$.&#xA;If some are &lt;em&gt;still&lt;/em&gt; huge (regardless of condition number), that would tend to confirm Boyd (below)&#xA;that Gaussian RBF is fundamentally weak.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(One alternative to RBF is Inverse-distance weighting, IDW. It has the advantage&#xA;of auto-scaling, the same for nearest distances 1 2 3 $\dots$&#xA;as for 100 200 300 $\dots$ &#xA;Also I find explicit user choice of $Nnear$, the number of near neighbors to consider,&#xA;clearer than grid search on $\sigma, \lambda$ .)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;John P. Boyd, &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999109005841&quot; rel=&quot;nofollow&quot;&gt;The uselessness of the Fast Gauss Transform for summing Gaussian radial basis function series&lt;/a&gt;, says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the Gaussian RBF interpolant is ill-conditioned for most series in the sense that&#xA;  the interpolant is the small difference of terms with exponentially large coefficients.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Hope this helps; please share your experience.&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2014-02-16T16:12:18.510" LastActivityDate="2014-02-16T16:12:18.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7656" PostTypeId="2" ParentId="7376" CreationDate="2013-06-13T19:37:05.573" Score="9" Body="&lt;p&gt;I don't have experience with Octrees, but whenever there is some nice C++ library that I want to use in Fortran, I simply write a simple C driver --- typically a few C functions that do exactly what I need. Then I call them from Fortran using the iso_c_binding module. This has the great advantage that you reuse a well tested library with a community around it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For information how to use iso_c_binding, I wrote up some information here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fortran90.org/src/best-practices.html#interfacing-with-c&quot;&gt;http://fortran90.org/src/best-practices.html#interfacing-with-c&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this particular info is for calling Fortran from C. I can see that I should update it. Here is my code where I call C++ library from Fortran, the first link is the simple C driver, the second link is a Fortran interface, and then I just call it like any other Fortran subroutine:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/certik/hfsolver/blob/master/src/qc_libint.cpp&quot;&gt;https://github.com/certik/hfsolver/blob/master/src/qc_libint.cpp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/certik/hfsolver/blob/master/src/qc.f90&quot;&gt;https://github.com/certik/hfsolver/blob/master/src/qc.f90&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2013-06-13T19:37:05.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7659" PostTypeId="2" ParentId="7652" CreationDate="2013-06-14T08:55:03.080" Score="1" Body="&lt;p&gt;You have a multi-objective optimization problem: you want to maximize the number of cities but also minimize the amount of time. This is, by itself, not a well posed problem since you need to say which of two solutions $(N_1,T_1)$ and $(N_2,T_2)$ is better if two paths visit $N_i$ cities and take $T_i$ time units. What you need to do, then, is to first define a partial ordering of such tuples that describes which of the two you consider better. Once you have that, coming up with an objective function will become mostly obvious.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-14T08:55:03.080" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7663" PostTypeId="2" ParentId="7650" CreationDate="2013-06-14T14:16:06.613" Score="11" Body="&lt;p&gt;This is rather a general remark on FVM than an answer to the concrete questions. And the &lt;em&gt;message&lt;/em&gt; is that there shouldn't be the need for such an adhoc discretization of the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike in FE- or FD-methods, where the starting point is a discrete ansatz for the solution, the FVM approach leaves the solution untouched (at first) but averages on a segmentation of the domain. The discretization of the solution comes into play only when the obtained system of balance equations is turned into an algebraic equation system by approximating the fluxes across the interfaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this sense, in view of the boundary conditions, I advise to stick to the continuous form of the solution as long as possible and to introduce the discrete approximations only at the very end. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say, the equation &#xA;$$&#xA;u_t = -au_x + du_{xx} + s(x,u,t)&#xA;$$&#xA;holds on the entire domain. Then it holds on the subdomain $[0,h_1)$, and an integration in space gives&#xA;\begin{align}&#xA;\int_0^{h_1}u_t \text{d}x &amp;amp;=&amp;amp; \int_0^{h_1} \partial_x(-au + du_{x})\text{d}x &amp;amp;+&amp;amp; \int_0^{h_1} s(x,u,t)\text{d}x \\ &amp;amp;=&amp;amp; (-au + du_{x})|_{x=h_1}-(-au + du_{x})|_{x=0}&amp;amp;+&amp;amp;\int_0^{h_1} s(x,u,t)\text{d}x,&#xA;\end{align}&#xA;which is the contribution of the first cell to the equation system. Note that, apart from taking only averages, there has been no discretization of $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now, to turn this into an algebraic equation, one typically assumes that on cell $C_i$ the function $u$ is constant in space, i.e. $u(t,x)|_{C_i} = u_i(t)$. Thus, having associated $u(x_i)\approx u_i$, one can express $u_x|_{h_i}$ at the cell boarders via the difference quotient in $u_{i}$ and $u_{i+1}$. To express $u$ at the cell boarders one can use interpolation (i.e. central differences or upwind schemes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What to do at the boundary? In the example, it is all about approximating $(-au + du_{x})|_{x=0}$, no matter what has been done to $u$ so far.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Given $u|_{x=0}=g_D$ one can introduce a ghost cell and the condition that an interpolant between $u_0$ and $u_1$ is equal to $g_D$ at the boarder.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Given ${u_x|}_{x=0} = g_N$ one can introduce a ghost cell and the condition that an approximation to the derivative between $u_0$ and $u_1$ matches $g_N$ at the boarder&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the flux itself is prescribed: $(-a u + du_x )|_{x=0}= g_R$, there is no need for a discretization.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, I am not sure, what to do in the case that there are Robin type bc's that do not match the flux directly. This, will need some regularization because of the discontinuity of the advection and diffusion parameters. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;===&gt; Some personal thoughts on FVM &amp;lt;===&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;FVM is &lt;strong&gt;not&lt;/strong&gt; a scaled FDM, as examples of 1D Poisson's equations on a regular grid often suggest&lt;/li&gt;&#xA;&lt;li&gt;There shouldn't be a grid in FVM, there should be cells with interfaces and, if necessary, centers&lt;/li&gt;&#xA;&lt;li&gt;That's why I think that a stencil formulation of the discretization is not suitable &lt;/li&gt;&#xA;&lt;li&gt;Assembling of the equation system should be done according to the discretization approach, i.e. by iterating over the cells rather than defining an equation for every unknown. I mean to think of the $i$-th row of the coefficient matrix as the part of the problem posed on cell $\Omega_i$, rather than of the equation that is associated with $u_i$.&lt;/li&gt;&#xA;&lt;li&gt;This is particularly important for 2D or 3D problems but may also help to have a clear notation in 1D: Make a difference between the volume (in 1D: length) of the cell, here $h_i$, and the distance between the centers, maybe in 1D: $d_i:=d_{i,i+1}=|x_i-x_{i+1}|$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4868" LastEditDate="2013-10-21T05:44:45.800" LastActivityDate="2013-10-21T05:44:45.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7664" PostTypeId="1" CreationDate="2013-06-14T15:43:00.973" Score="8" ViewCount="3621" Body="&lt;p&gt;I have worked out some poor code to achieve the goal of 3D Delauney triangulation(random points in E3), but the time consuming is huge, and when five points are exactly (or nearly due to the round-off error) on one sphere, my code can not handle this situation properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the basic data-structure which is a list of tetrahedrons and a list of points and a list of relationship of tetrahedrons with their neighborhood.&#xA;   The algorithm is incremental insertion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody tell me which kinds of data-structures and algorithm should i prefer to? Can quad-edge data-structure be used in the situation ? When I read papers about this topic,I find that maybe this data-structure is not suitable for 3D application(strictly speaking, not suitable for 3D manifold application?I just know what is manifold yesterday, Please help me...).  Is divide-conquer a better algorithm?&#xA;   Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4557" LastEditorUserId="4244" LastEditDate="2013-06-14T18:46:27.247" LastActivityDate="2013-06-14T21:03:08.137" Title="Proper data-structure and algorithm for 3-D Delaunay triangulation" Tags="|algorithms|delaunay-triangulation|data-structures|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7665" PostTypeId="2" ParentId="7664" CreationDate="2013-06-14T16:37:27.877" Score="4" Body="&lt;p&gt;This is implemented in qhull which is available from scipy (python).  If you cannot use these implementations directly for some reason, the explanations of the data structures in the docs might be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.qhull.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.qhull.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.Delaunay.html#scipy.spatial.Delaunay&quot; rel=&quot;nofollow&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.Delaunay.html#scipy.spatial.Delaunay&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3705" LastActivityDate="2013-06-14T16:37:27.877" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7666" PostTypeId="2" ParentId="7664" CreationDate="2013-06-14T21:03:08.137" Score="0" Body="&lt;p&gt;The data structure in 3D is pure algebraic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need to have is the following arrays:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;Vertex&lt;/code&gt; &lt;code&gt;V&lt;/code&gt;: the vertices' coordinates for the mesh, a $(\# \text{ of vertices}) \times 3$ array, with each column being $x$-, $y$- and $z$-coordinates, and the row index corresponds to the index of that vertex.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;Element to Vertex&lt;/code&gt; &lt;code&gt;E2V&lt;/code&gt;: the tetrahedral element to the vertex numbering, a $(\# \text{ of tetrahedra}) \times 4$ array. Each row represents a tetrahedron, and each column on that row is the index vertex(row number of this specific index in &lt;code&gt;V&lt;/code&gt;) of that tetrahedron.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;Face to Vertex&lt;/code&gt; &lt;code&gt;F2V&lt;/code&gt;: triangle faces to the vertex numbering, with a $(\# \text{ of faces}) \times 3$ array. Each row represents a face, and each column on that row is the index vertex(row number of this specific index in &lt;code&gt;V&lt;/code&gt;) of that face.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;Edge to Vertex&lt;/code&gt; &lt;code&gt;F2V&lt;/code&gt;: edge to the vertex numbering, with a $(\# \text{ of edges}) \times 2$ array. Each row represents an edge, and each column on that row is the index vertex(row number of this specific index in &lt;code&gt;V&lt;/code&gt;) of that edge.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;First two are necessary data structure&lt;/em&gt;, all other array can be generated from the first two using algebraic operations. Other notable arrays are &lt;code&gt;Element to Edge&lt;/code&gt;, &lt;code&gt;Face to Edge&lt;/code&gt;, &lt;code&gt;Vertex to Element&lt;/code&gt;(the elements sharing a vertex), &lt;code&gt;Face to Element&lt;/code&gt;(the elements sharing a face),  &lt;code&gt;Edge to Face&lt;/code&gt;(the faces sharing an edge), etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation of 3D Delaunay triangulation does not sound as trivial as the other answer said. Depends on your software of interest, I may update my answer more.&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-06-14T21:03:08.137" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7667" PostTypeId="2" ParentId="7628" CreationDate="2013-06-14T21:30:40.357" Score="7" Body="&lt;p&gt;The FEM method for transient problems typically uses the method of lines, i.e. the spatial discretization is decoupled from the time discretization:&#xA;\begin{equation}&#xA;u^h(x,t) = \mathbf{\Phi}(x)^T \, \mathbf{U}(t)&#xA;\end{equation}&#xA;where $\mathbf{U}(t)$ is the vector of nodal quantities, assumed as unknown functions of time. Under this assumption the space-time PDE's in $(x,t)$ are reduced (discretized) to ODE's in $t$ alone using the usual FEM machinery for static problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already pointed out by other answers, we speak of explicit or implicit FEM with reference to the time integration scheme of these ODE's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With reference to continuum mechanics problems (without damping), we end up with a system of ODE's like&#xA;\begin{equation}&#xA;\mathbf{M} \ddot{\mathbf{U}}(t) + \mathbf{F}_\text{i}(\mathbf{U}(t)) = \mathbf{F}_\text{e}(t)&#xA;\end{equation}&#xA;where $\mathbf{F}_\text{i}$ and $\mathbf{F}_\text{e}$ are the internal and external nodal equivalent forces. For linear problems $\mathbf{F}_\text{i}(t) = \mathbf{K}\, \mathbf{U}(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the risk of over-simplifing let us assume that in an explicit scheme you just need to solve for $\ddot{\mathbf{U}}(t)$&#xA;\begin{equation}&#xA;\mathbf{M} \ddot{\mathbf{U}}(t) = -\mathbf{F}_\text{i}(\mathbf{U}(t)) + \mathbf{F}_\text{e}(t)&#xA;\end{equation}&#xA;which is trivial if the mass matrix is lumped. On the contrary in implicit methods you need to solve the (non)-linear equations $\mathbf{F}_\text{i}(\mathbf{U}(t)) =  \mathbf{b}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fully answer your question: explicit/implicit refers to the solution of the ODE system and not the nature of the mass matrix. Of course every reasonable implementation of an explicit scheme requires the mass matrix to be lumped, otherwise the advantages of the method are lost in the solution for $\ddot{\mathbf{U}}(t)$. On the contrary for implicit schemes you can have both lumped and consistent mass matrices.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-06-14T21:30:40.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7668" PostTypeId="2" ParentId="7305" CreationDate="2013-06-15T05:05:39.890" Score="2" Body="&lt;p&gt;Yes a lot. See for example &lt;a href=&quot;http://www.afterworks.com/fumefx/overview.asp&quot; rel=&quot;nofollow&quot;&gt;FumeFx&lt;/a&gt; (3ds max plugin, grid based simulation) or &lt;a href=&quot;http://www.realflow.com/&quot; rel=&quot;nofollow&quot;&gt;RealFlow&lt;/a&gt; (SHP simulation) and &lt;a href=&quot;http://en.wikipedia.org/wiki/Fluid_simulation&quot; rel=&quot;nofollow&quot;&gt;more&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And there is still research going on in this field. See Siggraph &lt;a href=&quot;http://kesen.realtimerendering.com/sig2013.html&quot; rel=&quot;nofollow&quot;&gt;papers&lt;/a&gt;. My favorite this year is &lt;a href=&quot;http://www.youtube.com/watch?feature=player_detailpage&amp;amp;v=JAFhkdGtHck#t=169s&quot; rel=&quot;nofollow&quot;&gt;snow simulation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4561" LastActivityDate="2013-06-15T05:05:39.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7669" PostTypeId="1" CreationDate="2013-06-15T10:31:54.043" Score="4" ViewCount="431" Body="&lt;p&gt;I want to approximate the integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I = \int f(\boldsymbol{x})d\boldsymbol{x} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\boldsymbol{x}$ is $d$-dimensional. I have a set of non-equally spaced points $\boldsymbol{x}_1, \dots, \boldsymbol{x_n}$, and the corresponding values $f(\boldsymbol{x}_1), \dots, f(\boldsymbol{x_n}$), and I would like to use them to approximate the integral $I$. Actually the integral is over $R^d$, but I know that $ f(\boldsymbol{x})$ goes to zero quite rapidly outside my sample $\boldsymbol{x}_1, \dots, \boldsymbol{x_n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is the following: are there rules to approximate the integral using the non-equally spaced points? I've found simple rules for the univariate case, but for multiple dimensions.&#xA;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: the points $\boldsymbol{x}_1, \dots, \boldsymbol{x_n}$ are given and I can't choose them.&lt;/p&gt;&#xA;" OwnerUserId="2140" LastActivityDate="2013-06-16T10:11:49.630" Title="Multivariate numerical integration with a non-uniform grid" Tags="|high-dimensional|numerical-analysis|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7670" PostTypeId="2" ParentId="7653" CreationDate="2013-06-15T16:48:12.510" Score="1" Body="&lt;p&gt;It looks like you're trying to solve a &lt;a href=&quot;http://en.wikipedia.org/wiki/Graph_partitioning&quot; rel=&quot;nofollow&quot;&gt;graph partitioning&lt;/a&gt; problem, with the constraint that two specific points be in different partitions. The library &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/metis/metis/overview&quot; rel=&quot;nofollow&quot;&gt;METIS&lt;/a&gt; is pretty popular for this application and can be run in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;METIS might be overkill; it partitions the whole graph and thus looks at every point, whereas you just want to find a cut set and don't care about which side of the cut any of the other points end up on. Nonetheless, you can always read about how their algorithm works for inspiration. They use a multi-level approach, as you pointed out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, your graph is highly structured. Any utility for partitioning general unstructured graphs probably won't take advantage of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reasons, you may be better off writing your own program if you need really fast performance and you're computing cut sets of lots of point pairs. Whether that's worth it is a judgment call you'll have to make.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2013-06-15T19:04:18.983" LastActivityDate="2013-06-15T19:04:18.983" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7672" PostTypeId="2" ParentId="5414" CreationDate="2013-06-15T22:09:48.130" Score="5" Body="&lt;p&gt;One of the best-performing methods for solving large-scale Lyapunov equations is ADI. It is an iterative algorithm that returns an approximate low-rank decomposition $X \approx VV^T$ of the solution $X$. In this case, you can work with this decomposition of both Gramians to reduce the eigenproblem to a smaller one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you to start approaching this algorithm by reading the paper &lt;em&gt;Numerical solution of large-scale Lyapunov equations, Riccati equations, and linear-quadratic optimal control problems&lt;/em&gt;, by Benner, Li and Penzl, the user manual for the Matlab library &lt;a href=&quot;http://www.tu-chemnitz.de/sfb393/lyapack/&quot;&gt;Lyapack&lt;/a&gt;, or one of the many talks on the webpage of &lt;a href=&quot;http://www.mpi-magdeburg.mpg.de/mpcsc/benner/talks.php?lang=en&quot;&gt;Peter Benner&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2013-06-15T22:09:48.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7674" PostTypeId="1" CreationDate="2013-06-16T04:12:08.197" Score="7" ViewCount="423" Body="&lt;p&gt;For 1D derivative we have&#xA;   \begin{equation}&#xA;    F(x) = \frac{\partial f(x)}{\partial x}&#xA;   \end{equation}&#xA;   \begin{equation}&#xA;    f(x)=\sum_{i}f_ie_i(x)&#xA;   \end{equation}&#xA;   \begin{equation}&#xA;    F(x)=\sum_{i}F_ie_i(x)&#xA;   \end{equation}&#xA;where $e_i(x)$ are the FEM basis functions. We can then apply the Galerkin procedure to the derivative,and then get the matrix form&#xA;\begin{equation}&#xA;A\widehat{F}=B\widehat{f}&#xA;\end{equation}&#xA;\begin{equation}&#xA;\widehat{F}=A^{-1}B\widehat{f}&#xA;\end{equation}&#xA;$\widehat{F}$and$\widehat{f}$are the vector form of $F_i$and $f_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the derivative of the product of two functions $f(x)$and $g(x)$,applying product rule we get&#xA;\begin{equation}&#xA; \frac{\partial (f(x).g(x))}{\partial x}=f(x). \frac{\partial g(x)}{\partial x}+g(x). \frac{\partial f(x)}{\partial x}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in FEM I want the derivative matrix $C=A^{-1}B$ also to fulfill the similar constraint&#xA;\begin{equation}&#xA;C \widehat{fg} = [f]_{dig}.C\widehat{g} + [g]_{dig}.C\widehat{f} &#xA;\end{equation}&#xA;where $[f]_{dig}$ means diagonal matrix with the values $f_i$ in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does any body know what kinds of basis function $e_i(x)$  I can chose?&lt;/p&gt;&#xA;" OwnerUserId="4564" LastActivityDate="2020-04-15T19:40:37.937" Title="A simple question about 1D finite element derivatives" Tags="|finite-element|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7675" PostTypeId="2" ParentId="7669" CreationDate="2013-06-16T10:11:49.630" Score="4" Body="&lt;p&gt;This depends on how many dimensions you have an how many points. Also on how the points are structured.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the number of dimensions are high and the points are randomly chosen, you essentially have a Monte Carlo integration procedure. If the points are randomly chosen from a distribution, then you have a kind of importance sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the number of points is small and you are in a low-dimensional situation, then you can think of this as trying to find a polynomial that interpolates $f(x)$ and the weights are then computed so that the integral over $f(x)$ is chosen as the integral over the approximating polynomial.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-16T10:11:49.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7676" PostTypeId="1" AcceptedAnswerId="7681" CreationDate="2013-06-16T14:29:01.237" Score="2" ViewCount="352" Body="&lt;p&gt;I'm testing Quicksort based on Niklaus Wirth algorithm. I'm checking how much time I need to sort an array. There is interesting thing that for an array consisting elements in these sequence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1    3    5    7    9    10    8    6    4    2 (example how an array looks like for bigger instance)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.14 seconds - array with 99999 elements&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5.91 seconds - array with 100000 elements&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.11 seconds - array with 100001 elements&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It last quite long for 100000 elements. Do you have any idea why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Windows.&#xA;Code with Quicksort in C++:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    void quicksort (int l,int p)&#xA;    {&#xA;        int i, j, x, w;&#xA;        i=l;&#xA;        j=p;&#xA;        x=tab[(l+p)/2];&#xA;&#xA;    do&#xA;        {&#xA;            while (tab[i]&amp;lt;x) i=i+1;&#xA;            while (x&amp;lt;tab[j]) j=j-1;&#xA;&#xA;            if (i&amp;lt;=j)&#xA;            {&#xA;                w=tab[i];&#xA;                tab[i]=tab[j];&#xA;                tab[j]=w;&#xA;                i=i+1;&#xA;                j=j-1;&#xA;            }&#xA;&#xA;        }while(!(i&amp;gt;j));&#xA;&#xA;        if(l&amp;lt;j)quicksort(l,j);&#xA;        if(i&amp;lt;p)quicksort(i,p);&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4566" LastEditorUserId="4566" LastEditDate="2013-06-16T15:02:32.247" LastActivityDate="2013-06-17T02:27:46.260" Title="Quicksort - difference in time" Tags="|algorithms|sorting|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7677" PostTypeId="2" ParentId="7676" CreationDate="2013-06-16T20:58:58.027" Score="2" Body="&lt;p&gt;I'm guessing you're seeing a large number of non-compulsory cache misses due to address conflicts, but it's hard to verify this without actually measuring the number of cache misses.  That said, 50x is a rather large performance difference.  This sort of large inconsistency should go away if you adjust your pivoting strategy (as J.M. suggests in the comments, you really should be using a random pivot).&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2013-06-16T20:58:58.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7678" PostTypeId="1" CreationDate="2013-06-16T21:49:18.463" Score="3" ViewCount="110" Body="&lt;p&gt;Suppose I have a time-dependent PDE discretized by the Rothe method and FEM, like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \int_{\Omega} k^{n+1/2}(u^{n+1}-u^{n}) v \;\mathrm{d}x = F^{n+1/2}(u^{n+1},u^n)[v] \quad \forall v\in V_h^n.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a good way of quantifying the difference between a transient solution $u^{n} \in V_{h}^{n}$ and a stationary solution $u^{n}_{\mathrm{S}}$ (satisfying the given equation with $k^{n+1/2} = 0$), without actually computing $u_{\mathrm{S}}$ and $\|u^{n} - u^{n}_{\mathrm{S}}\|$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $F^n$, $k^{n+1/2}$, and $V^n_h$ all depend on $n$ so that stationary solutions $u^n_\mathrm{S}$ are different at each time level $t_n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm targeting the computation of norms of some element tensors etc.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="6775" LastEditDate="2018-11-16T14:22:17.120" LastActivityDate="2018-11-16T14:22:17.120" Title="Quantification of non-stationarity of PDE solution" Tags="|finite-element|pde|fenics|rothe-method|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7679" PostTypeId="2" ParentId="7678" CreationDate="2013-06-16T22:16:44.527" Score="3" Body="&lt;p&gt;You could look at the residual: $\|F(u^n)\|$. If $u^n=u_S$ then $F(u_S)=0$, but this isn't true for a transient solution.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-16T22:16:44.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7680" PostTypeId="1" CreationDate="2013-06-17T00:17:45.930" Score="1" ViewCount="95" Body="&lt;p&gt;I was attempting to do a discrete Fourier transform through a computer program on a list of numbers. Before doing that I decided to test it by running through a list of 1000 numbers which I created by summing 20 sine waves which were created by the formula sin(2*pi*integer between 1 and 1000/ desired frequency). Based upon my understanding of the DFT I should have got a 0 for all values except for my twenty desired frequencies of 8, 13, 13, 14, 18, 23, 25, 32, 33, 38, 41, 48, 51, 64, 73, 79, 82, 85, 91, and 100. Some of my first outputs were:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 : (-140.00101976200014-0j)&#xA;1 : (-140.50548903670193-3.1117640585553277j)&#xA;2 : (-142.0510770346883-6.33051372898894j)&#xA;3 : (-144.74035295772114-9.776661681575883j)&#xA;4 : (-148.76670548437778-13.601419744876273j)&#xA;5 : (-154.45783389980753-18.014310511371548j)&#xA;6 : (-162.3639888189583-23.333771555566614j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From this program in python 3.2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;import cmath&lt;/p&gt;&#xA;&#xA;&lt;p&gt;def compute_dft(input):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = len(input)&#xA;output = [0] * n&#xA;for k in range(n):  &#xA;    s = 0&#xA;    for t in range(n):  &#xA;        s += input[t] * cmath.exp(-1j * 2 * cmath.pi * t * k / n)&#xA;    output[k] = -s&#xA;return output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;answer = compute_dft(unsorted_v)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for item in answer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(answer.index(item), &quot;:&quot;, item)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then went and checked the program on some examples of a DFT online and found that the answers it spit out were the same as the examples answers, which led me to wonder if my expectations of the output was wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice on resolving my problem would be much appreciated.&lt;/p&gt;&#xA;" OwnerDisplayName="Patrick" LastEditorUserId="-1" LastEditDate="2013-06-17T02:46:38.437" LastActivityDate="2013-06-17T08:36:29.087" Title="What to look for in a discrete fourier transform" Tags="|fourier-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7681" PostTypeId="2" ParentId="7676" CreationDate="2013-06-17T01:19:31.383" Score="4" Body="&lt;p&gt;Quicksort exhibits suboptimal time performance if it consistently selects a pivot that results in a uneven partition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual speed $O(n \log n)$ in time depends of getting a division of the input into two equal (or at least nearly equal) parts at each pass. If you get a highly unbalanced division regularly then you get a very different time behavior. In the worst case---in which only one element is removed in each pass---quicksort ends up with a time behavior of $O(n^2)$. This is the case when you select either the first of the last element as the pivot and the input is sorted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your pivot selection---from the middle---requires a slightly more complicated input to get pathological behavior, but depending on the details of your input and on the exact length (because you pick the &quot;middle&quot; element) you can have one or more &quot;bad&quot; passes. It looks like you get a great many bad passes at that one length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choosing the pivot element randomly is generally considered sufficient to reduce the odds of pathological behavior to a satisfactory level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately there are other algorithms (like mergesort) whose division does not depend on picking a value from among the inputs which don't have the failure mode.&lt;/p&gt;&#xA;" OwnerUserId="183" LastEditorUserId="183" LastEditDate="2013-06-17T02:27:46.260" LastActivityDate="2013-06-17T02:27:46.260" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7684" PostTypeId="2" ParentId="7680" CreationDate="2013-06-17T08:36:29.087" Score="3" Body="&lt;p&gt;Your input is not the sum of sine functions. I think if you print out the values of &lt;code&gt;s&lt;/code&gt;, you'll find that it has complex values. Replace &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;cmath.exp(-1j * 2 * cmath.pi * t * k / n)&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;cmath.sin(2 * cmath.pi * t * k / n)&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and you should be set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, due to finite precision arithmetic, you may see some tiny terms outside the 20 frequencies you expect to see. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-17T08:36:29.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7685" PostTypeId="1" AcceptedAnswerId="7686" CreationDate="2013-06-17T15:07:46.983" Score="11" ViewCount="1490" Body="&lt;p&gt;I have a set of known points/nodes irregularly spaced in N-Dimensional space (N&gt;=2), and I would like a way to generate the Delaunay triangulation of these points, and return the corresponding elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any existing meshing libraries that will do an N-D Delaunay triangulation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am doing this because I want to use the meshed elements as a basis for linear interpolation at any point in space. My dimension is currently handled by a C++ class templated over dimension if that makes any difference to suggestions...)&lt;/p&gt;&#xA;" OwnerUserId="4575" LastEditorUserId="549" LastEditDate="2013-06-17T17:33:28.330" LastActivityDate="2014-01-08T09:18:10.243" Title="N-dimensional Delaunay Tesselation Software Libraries" Tags="|computational-geometry|high-dimensional|mesh-generation|delaunay-triangulation|unstructured-mesh|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7686" PostTypeId="2" ParentId="7685" CreationDate="2013-06-17T16:20:49.770" Score="4" Body="&lt;p&gt;I think you can do this using convex hull software (e.g. QHull) via the lifting algorithm. At least, the documentation of matlab's &quot;delaunayn&quot; command seems to indicate as much.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2013-06-17T16:20:49.770" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7687" PostTypeId="2" ParentId="7685" CreationDate="2013-06-17T16:52:18.273" Score="5" Body="&lt;p&gt;This feature seems to be &lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Convex_hull_d_ref/Class_Delaunay_d.html&quot; rel=&quot;noreferrer&quot;&gt;available in CGAL&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-06-17T16:52:18.273" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7688" PostTypeId="1" CreationDate="2013-06-17T19:03:30.307" Score="5" ViewCount="2911" Body="&lt;p&gt;&lt;em&gt;Disclaimer&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the process of typing up this question, I determine its solution.  Since I went through the trouble of typing up the question in its entirety, I will post its answer as well.  It may help out others who find themselves in the same predicament.  Think of this as a sort of blog-post, if you will.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Goal&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the mixed boundary value problem&#xA;\begin{align}&#xA;\frac{d}{dx}\left(k(x)\frac{du}{dx}\right)=f \text{ in }\Omega\\&#xA;u=P \text{ at } x=0\\&#xA;\frac{du}{dx}=T \text{ at } x=1&#xA;\end{align}&#xA;where $P$ and $T$ are constants and $f$ is a source term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using finite differences and my goal is to impose the boundary condition at $x=1$ in such a way as to achieve second order accuracy.  Assume that the grid has $N+1$ equispaced points (including the boundary points) given as $x_0,x_1,...,x_N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the right boundary, use a 2nd order centered difference for the boundary condition&#xA;\begin{equation}&#xA;\frac{u_{N+1}-u_{N-1}}{\Delta x}=T\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the 2nd derivative operator as:&#xA;\begin{align}&#xA;\frac{k_{N+\frac{1}{2}}\frac{u_{N+1}-u_{N}}{\Delta x} -k_{N-\frac{1}{2}}\frac{u_{N}-u_{N-1}}{\Delta x}}{\Delta x}=f_i\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can solve for the ghost point $U_{N+1}$ in the 1st equation, substitute it into the 2nd equation and simplify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this discretization requires the evaluation of $k_{N+\frac{1}{2}}=k(x_N+\frac{\Delta x}{2})$, which is outside of the domain!  In general, $k(x)$ is only defined within the domain and I can't/shouldn't use values outside of it.  Thus, I don't think this is the right approach to achieve 2nd order accuracy.  What else can I do in this case?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-06-17T19:27:43.213" Title="Finite Difference Method Neumann Boundary Condition with Variable Coefficients" Tags="|finite-difference|boundary-conditions|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7689" PostTypeId="2" ParentId="7688" CreationDate="2013-06-17T19:03:55.637" Score="2" Body="&lt;p&gt;Forget about discretizing the 2nd derivative operator since it requires evaluating at points outside the domain.  Instead, just adjust the stencil for the boundary condition to use points $u_{N},u_{N-1},$ and $u_{N-2}$.  By using standard &lt;em&gt;method of undetermined coefficients&lt;/em&gt;, you can produce the coefficients necessary to achieve 2nd order accuracy.  If i'm not mistaken, the discretized boundary condition in this case turn out to produce &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{du}{dx}\approx\frac{-\frac{1}{2}u_{N-2} +2u_{N-1} -\frac{3}{2}u_N}{\Delta x}=T$$&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-06-17T19:27:43.213" LastActivityDate="2013-06-17T19:27:43.213" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7690" PostTypeId="2" ParentId="7678" CreationDate="2013-06-17T19:24:42.390" Score="0" Body="&lt;p&gt;Let's assume that stationary solutions $u^n_\mathrm{S}$ are given by&#xA;$$ &#xA;0 = F^{n+1/2}(u^{n+1}_\mathrm{S}, u^n_\mathrm{S}) [v] .&#xA;$$&#xA;We can do that - imagine that $F^{n+1/2}$ is Crank-Nicolson discretization; then this means $$&#xA;0=F^{n+1/2}(u^{n+1/2}_\mathrm{S}) [v] .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's define&#xA;$$&#xA;r^n := u^n - u^n_\mathrm{S} .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we use Taylor theorem on the RHS of the first equation (&lt;strong&gt;assuming $r^n$, $r^{n+1}$ are small&lt;/strong&gt;) and we have up to the first order&#xA;$$&#xA;0 \approx F^{n+1/2}(u^{n+1}, u^n) [v] &#xA;        + D_{u^{n+1}}F^{n+1/2}(u^{n+1},u^n) [v][-r^{n+1}]&#xA;        + D_{u^{n  }}F^{n+1/2}(u^{n+1},u^n) [v][-r^{n  }] .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now define notation &#xA;$R^{n+1/2} := F^{n+1/2}(u^{n+1}, u^n)$,&#xA;$J^{n+1/2} := D_{u^{n+1}}F^{n+1/2}(u^{n+1},u^n)$ &#xA;and &#xA;$\widetilde{J}^{n+1/2} := D_{u^{n  }}F^{n+1/2}(u^{n+1},u^n)$&#xA;so that last approximation reads&#xA;$$&#xA;0 = R^{n+1/2} [v] - J^{n+1/2} [v][r^{n+1}] - \widetilde{J}^{n+1/2} [v][r^{n  }] .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Picking $v:=r^{n+1}$ and rearranging last equation we have&#xA;$$&#xA;J^{n+1/2}[r^{n+1}][r^{n+1}] = R^{n+1/2} [r^{n+1}] - \widetilde{J}^{n+1/2} [r^{n+1}][r^{n  }] .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $J^{n+1/2}$ is elliptic $J^{n+1/2}[v][v]\geq\alpha||v||^2$&#xA;$$&#xA;||r^{n+1}|| \leq \frac{1}{\alpha}\left|\left|R^{n+1/2} - \widetilde{J}^{n+1/2} [\cdot][r^{n  }]\right|\right| ,&#xA;$$&#xA;which could be useful as reccursion relation for $||r^n||$. In a special case where $F^{n+1/2}$ is backward Euler discretization so that $\widetilde{J}^{n+1/2} = 0$ one have simply&#xA;$$&#xA;||r^{n+1}|| \leq \frac{1}{\alpha}\left|\left|R^{n+1/2}\right|\right| .&#xA;$$&#xA;LHS of the last equation is target quantity and norm on the RHS can be calculated cheaply from already computed transient solutions. The biggest trouble is $\alpha$, the ellipticity constant of the Jacobian $J^{n+1/2}$&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2013-06-17T19:24:42.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7691" PostTypeId="2" ParentId="7646" CreationDate="2013-06-17T19:30:48.473" Score="2" Body="&lt;p&gt;You might want to look at ProtoMol. It is not a library but an application but it is designed to be easily extendable. Thus it might be easier to add your custom potential than it would be with e.g. LAMPPS or Gromacs. The later two would be better options if you want to have good parallel performance. &lt;/p&gt;&#xA;" OwnerUserId="1442" LastActivityDate="2013-06-17T19:30:48.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7693" PostTypeId="1" AcceptedAnswerId="7698" CreationDate="2013-06-17T21:58:19.370" Score="2" ViewCount="377" Body="&lt;p&gt;I want to develop a visual simulation of a propagating 2D wavefunction with an added attractive potential. Basically I have to numerically solve the time-dependent Schrodinger equation (PDE with x,y,t as variables) but, instead of boundary conditions, I want to have an incoming wavefunction of my choice as the initial wavefunction profile. What software would give me a good environment for such a project (I'm thinking Mathematica, Comsol, Matlab as a starting point)?&lt;/p&gt;&#xA;" OwnerUserId="4578" LastActivityDate="2013-06-18T12:02:35.370" Title="Time-dependent wavefunction numerical simulator" Tags="|quantum-mechanics|wave-propagation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7694" PostTypeId="1" AcceptedAnswerId="7701" CreationDate="2013-06-17T23:09:44.140" Score="4" ViewCount="1518" Body="&lt;p&gt;How to do transpose for trtrs (or tptrs) in blas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;XA = B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it seems that trtrs only lets me solve:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AX = B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, using the 'transpose' flag, in trtrs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A'X = B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which, rearranging is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(A'X)' = B'&#xA;X'A = B'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I can use it to solve XA = B, but I have to first transpose B manually myself, and then, again, transpose the answer.  Am I missing some trick to avoid having to do the two manual transposes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I've read through &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5318/solve-xa-b-for-x-using-lapack-and-blas&quot;&gt;solve $xA=b$ for $x$ using LAPACK and BLAS&lt;/a&gt; , but it doesn't seem to explain how to avoid the two manual transposes, as far as I can tell?  Or maybe I'm missing something somehow?&lt;/p&gt;&#xA;" OwnerUserId="3613" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-06-18T19:23:01.337" Title="How to do transpose for trtrs (or tptrs) in blas?" Tags="|linear-algebra|linear-solver|blas|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7695" PostTypeId="1" AcceptedAnswerId="7697" CreationDate="2013-06-17T23:15:23.347" Score="1" ViewCount="378" Body="&lt;p&gt;I'm compiling a blas to use with scalapack.  Some blases, such as openblas, are naturally multithreaded, but scalapack uses mpi, so maybe needs a single-threaded blas?  My questions then are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;does scalapack need a multi-threaded blas? a single-threaded blas?  or it will work just fine on either because, for example, it's somehow forcing the number of threads to 1 at runtime?&lt;/li&gt;&#xA;&lt;li&gt;in the case that scalapack forces the number of threads to 1 at runtime, how does it do this?  does this work with openblas?  It seems that the api for this in openblas is &lt;code&gt;openblas_set_num_threads(1)&lt;/code&gt;, which doesn't sound like a particularly portable command, and sounds like such configuration is blas-specific?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit: I suppose I could just use a multithreaded OpenBLAS, and call &lt;code&gt;openblas_set_num_threads(1)&lt;/code&gt; myself?&lt;/p&gt;&#xA;" OwnerUserId="3613" LastEditorUserId="3613" LastEditDate="2013-06-18T04:39:30.847" LastActivityDate="2013-06-18T11:40:37.507" Title="Does scalapack need threaded or unthreaded blas?" Tags="|linear-algebra|parallel-computing|blas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7696" PostTypeId="1" CreationDate="2013-06-18T07:16:02.187" Score="11" ViewCount="1058" Body="&lt;p&gt;I am given a $12 \times 12$ matrix $Q$ that is symmetric, invertible, positive definite and dense. I need to test if $$\det(Q) = \det(12I-Q-J) \; \; (1)$$ where $J$ is the all ones matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently doing this with the &lt;a href=&quot;http://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;cad=rja&amp;amp;ved=0CC0QFjAA&amp;amp;url=http://arma.sourceforge.net/&amp;amp;ei=iQjAUZiBK8nQtAaV0IDgDQ&amp;amp;usg=AFQjCNGQHR_1xEYlKZKJmnpyvZMTIFkgBQ&amp;amp;sig2=AlnDHWBzKnHy8ch494UQqQ&amp;amp;bvm=bv.47883778,d.Yms&quot; rel=&quot;nofollow noreferrer&quot;&gt;armadillo&lt;/a&gt; library but it turns out to be too slow. The thing is that I need to do this for a trillion of matrices and it turns out that computing the two determinants is the bottleneck of my program. Hence I have two questions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any trick I could use to compute the determinant faster given that I known their size? Is perhaps a messy expansion for $12 \times12$ matrices that could work in this case?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any other efficient way to test the equality $(1)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit.&lt;/strong&gt; To answer the comments. I need to compute all connected  non self-complementary graphs $G$ of order $13$ such that $G$ and $\overline{G}$ have the same number of spanning trees. The motivation for this can be found in this &lt;a href=&quot;https://mathoverflow.net/questions/132728/tutte-polynomials-graph-complements-and-degree-sequences&quot;&gt;mathoverflow&lt;/a&gt; post. As for the machine I am running this on a 8 core 3.4GHh machine in parallel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit.&lt;/strong&gt; I was able to reduce the expected running time by 50% by making a C program for specifically computing the determinant of a $12 \times 12$ matrix. Suggestions are still welcome. &lt;/p&gt;&#xA;" OwnerUserId="4582" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2019-05-22T08:02:19.683" Title="Testing if two 12x12 matrices have the same determinant" Tags="|linear-algebra|matrix|c++|graph-theory|c|" AnswerCount="3" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="7697" PostTypeId="2" ParentId="7695" CreationDate="2013-06-18T11:40:37.507" Score="2" Body="&lt;p&gt;You have to choose. It may be that running a threaded BLAS with a reduced MPI rank count gives better performance than running lots of ranks with one thread per rank. If you have $p$ processor cores, then any pair of integers $t$ and $r$ with $tr \le p$ is a viable possibility for combinations of threads ($t$) and ranks ($r$). The fastest of which will depend on your machine and your problem. Typical choices would be $r=p$ with $t=1$ and $r=s$ with $t=p/s$ where $s$ is the number of CPU sockets in the node. At the very least you should test these two. Don't forget to choose an appropriate binding of ranks and threads to sockets, or your performance may suffer.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-18T11:40:37.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7698" PostTypeId="2" ParentId="7693" CreationDate="2013-06-18T12:02:35.370" Score="0" Body="&lt;p&gt;You could start from the step-23 or step-25 tutorial programs of deal.II, see for example &lt;a href=&quot;http://www.dealii.org/7.2.0/doxygen/deal.II/step_23.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/7.2.0/doxygen/deal.II/step_23.html&lt;/a&gt; . (Disclaimer: I'm one of the authors of this library.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-18T12:02:35.370" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7699" PostTypeId="2" ParentId="7616" CreationDate="2013-06-18T16:00:55.730" Score="-3" Body="&lt;p&gt;I was searching for the solution for similar question: for a very large sparse linear program, only simplex method tested work, but only when the default 0 solution is feasible. Seems the phase one algorithm (like in linprog of Matlab) is bad. And the phase one source code is so complex to substitute it by other algorithm like genetic algorithm, so a work around method is to make linear transformation of the original problem, so that in the new varibles the provided initial feasible solution is 0, and this 0 will be used by phase one without using its own method to find a different starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In testing this method, by step through linprog.m, simplex.m, simplexpresolve.m, and simplexphaseone.m, in cases where only inequality constraints are used, it is confirmed that the default 0 will be used for the original variables, where the slack variables will take the differences. So the linear transformation can sneak x0 into simplex which intentionally prevent user provided x0.You can then see the message of &lt;em&gt;&quot;The default starting point is feasible, skipping Phase 1.&quot;&lt;/em&gt; On the other hand, usually GA can find solution close to linear program to 0.01 percent by using double or triple times, so it may not merit the efforts of the linear transformation of those constraints, objective and bounds, especially when the constraints are artificially created.&lt;/p&gt;&#xA;" OwnerUserId="4584" LastEditorUserId="4584" LastEditDate="2013-06-20T15:18:17.880" LastActivityDate="2013-06-20T15:18:17.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7700" PostTypeId="2" ParentId="7366" CreationDate="2013-06-18T18:14:41.637" Score="1" Body="&lt;p&gt;&lt;strong&gt;Problem solved:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normalization needs to be a part of the function evaluated in the ODE. Breaking up the ODE in many steps and normalizing between them causes seemingly numerical instability and produces different results depending on the time intervals the ODE is broken into. (See edit 2 in question for more details.)&lt;/p&gt;&#xA;" OwnerDisplayName="user4388" LastActivityDate="2013-06-18T18:14:41.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7701" PostTypeId="2" ParentId="7694" CreationDate="2013-06-18T19:23:01.337" Score="4" Body="&lt;p&gt;Since basically all that xTRTRS does is call xTRSM (from BLAS), why not just call that directly? xTRSM contains a SIDE parameter which lets you specify exactly what you want.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-06-18T19:23:01.337" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7702" PostTypeId="1" AcceptedAnswerId="7710" CreationDate="2013-06-18T21:08:41.107" Score="8" ViewCount="11152" Body="&lt;p&gt;&lt;strong&gt;Overview&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that one should use a time step $\Delta t &amp;lt; \frac{h}{v}$ (where h - smallest mesh element, v - velocity) to get an accurate result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how important is this really for the accuracy of the simulation?&#xA;Is it as important as having an independent mesh?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there even such a thing as a time step independent solution?&#xA;Can a very small $\Delta t$ actually be bad for the accuracy of the solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am running computational optimisation, where speed is important.&#xA;Just how much am I justified to use $\Delta t &amp;gt; \frac{h}{v}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am running a transient simulation, where $v$ changes from zero to 60 m/s. Should I just set it to the smallest $\Delta t \approx 0.0007$ s (I can't dynamically change $\Delta t$)?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem Details&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using an Euler-Euler model (in Fluent™) to simulate particle-air interaction in a fluidised bed.&lt;/p&gt;&#xA;" OwnerUserId="4469" LastEditorUserId="4469" LastEditDate="2013-07-26T09:14:35.327" LastActivityDate="2017-03-30T16:25:37.643" Title="Importance of the Time Step Value for the Accuracy of a Transient CFD Simulation" Tags="|fluid-dynamics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7703" PostTypeId="1" AcceptedAnswerId="7732" CreationDate="2013-06-19T00:57:50.913" Score="3" ViewCount="373" Body="&lt;p&gt;Hello there and thanks for taking a look at this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is related to my previous question and I will therefore use a similar introduction from, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7366/choice-of-step-size-using-odes-in-matlab&quot;&gt;Choice of step size using ODEs in matlab&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Introduction:&lt;/strong&gt;&#xA;I'm currently studying a 2D exciton spinor Bose-Einstein Condensate and am curious about the ground state of this system. The mathematical method of getting to the ground state is called the imaginary time method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method is very simple. Normal real time in quantum mechanics is replaced by imaginary one&#xA;$$ t = -i \tau$$&#xA;This substitution causes the high energy particles in the system to decay faster than the low energy ones. Re-normalizing the number of particles in every step of the calculation we end up with a system of lowest energy particles, aka. the ground state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation(s) in question is nonlinear, called the nonlinear Schrödinger equation, sometimes the Gross-Pitaevskii equation. To solve the problem I'm using Matlabs ode45 which evolves the system forward in time and eventually reaches the ground state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem and the question:&lt;/strong&gt;&#xA;There are a lot of parameters controlling the physics here. It is very appropriate to work with constants and variables of comfortable dimensionality such as, micrometers, picoseconds, millielectronvolts and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, working with picoseconds and millielectronvolts seems to produce drastically different results then working with nanoseconds and microelectronvolts.&#xA;$$ (\text{ns},\mu \text{eV}) \quad  ^?\leftrightarrow \ ^? \quad (\text{ps}, \text{meV})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: If all parameters are set so that these two representations correspond correctly, i.e. are effectively the same thing, why would the results be completely different for say...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...in the representation of ps and meV&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[~,y_out] = ode113(odefun,[0:100:10000],y_in, ...variables_in_ps_and_meV)...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and otherwise in the representation of ns and ueV?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[~,y_out] = ode113(odefun,[0:0.1:10],y_in, ...variables_in_ns_and_ueV...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the time step 100ps corresponds to 0.1ns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;&#xA;In the case of using nanoseconds, i.e. time steps 0.1, the ODE calculates much, much faster. This has me inclined to believe the method of using nanoseconds is somehow rushed and not accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any thoughts!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 1:&lt;/strong&gt;&#xA;Here are two &lt;a href=&quot;https://i.stack.imgur.com/4CfmD.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;pictures&lt;/a&gt; showing the completely different results&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 2:&lt;/strong&gt;&#xA;I just found out that it doesn't matter at all whether I work with $\mu$eV or meV. The same results are displayed! This narrows the problem down to only the choice of appropriate time dimensionality for the ode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 3:&lt;/strong&gt;&#xA;Major breakthrough. I decided to write a simple 1D Schrödinger equation in a harmonic potential and tried out nanoseconds vs. picoseconds. The results can be found &lt;a href=&quot;http://www.youtube.com/watch?v=yAXuXCTQuAY&amp;amp;feature=youtu.be&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, there is no difference when using either nanoseconds or picoseconds (not very surprising). But now just remains to figure out what in the devil is causing this strange behavior in my 2D nonlinear Schrödinger equations. &lt;/p&gt;&#xA;" OwnerDisplayName="user4388" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-06-20T19:33:45.483" Title="Nanoseconds vs. picoseconds in numerical quantum problems with Matlab ODEs" Tags="|matlab|ode|nonlinear-equations|quantum-mechanics|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7704" PostTypeId="2" ParentId="7512" CreationDate="2013-06-19T01:01:56.460" Score="1" Body="&lt;p&gt;By a chance I have spotted your question.&#xA;I would suggest to look at these:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;http://www.iue.tuwien.ac.at/phd/heinzl/  Rene Heinzl&#xA;dissertation,    Concepts for Scientific Computing, &#xA;&#xA;http://www.math.tu-cottbus.de/~berti/diss/, Guntram Berti&#xA;dissertation,    Generic Software Components for Scientific&#xA;Computing. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Documentation for GetFEM++ package    &lt;/p&gt;&#xA;" OwnerUserId="4587" LastActivityDate="2013-06-19T01:01:56.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7705" PostTypeId="1" AcceptedAnswerId="7721" CreationDate="2013-06-19T02:14:12.650" Score="4" ViewCount="790" Body="&lt;p&gt;What libraries can compute dense Cholesky factorizations in distributed-memory environments (preferably using &lt;a href=&quot;https://en.wikipedia.org/wiki/Message_Passing_Interface&quot; rel=&quot;nofollow&quot;&gt;MPI&lt;/a&gt;). I'm using C/C++, and have access to a cluster of between 1 and 40 nodes, where each node has 12 cores. I've heard of &lt;a href=&quot;http://netlib.org/scalapack&quot; rel=&quot;nofollow&quot;&gt;ScaLAPACK&lt;/a&gt; but am interested in alternatives.&lt;/p&gt;&#xA;" OwnerUserId="3613" LastEditorUserId="3613" LastEditDate="2013-06-20T06:32:20.853" LastActivityDate="2013-06-20T06:32:20.853" Title="Libraries for distributed-memory Cholesky factorization?" Tags="|linear-algebra|matrix|parallel-computing|c|mpi|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7706" PostTypeId="1" CreationDate="2013-06-19T02:38:16.887" Score="4" ViewCount="821" Body="&lt;p&gt;I would like to use a high-quality renderer (physically based path-tracing or MLT) to generate some images of molecular systems I have.  Right now I'm just using some of the simple renderers in VMD, but I am curious if anyone has developed a plugin for something like Indigo Renderer or Luxrender?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially it would convert an XYZ/CIF/PDB file into an input file for one of these programs, generating appropriate shapes and BRDF's for each type of atom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could write some code myself but I'd rather not reinvent the wheel, so if anyone knows of something out there, that'd be great!  Thanks&lt;/p&gt;&#xA;" OwnerUserId="4362" LastActivityDate="2014-04-08T14:47:20.670" Title="High quality molecular renderings" Tags="|visualization|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7707" PostTypeId="1" CreationDate="2013-06-19T08:30:40.650" Score="3" ViewCount="299" Body="&lt;p&gt;This is maybe a trivial question, but I am stuck with the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have a general graph:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$G=(V,E)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each edge represents a task, each vertex represents a data for the task (hence each task works with two vertices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When two tasks with common vertex run in parallel, some computation time is lost because two tasks cannot use the same vertex at the same time (one have to wait).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking about the following approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Create a table of locks telling whether the vertex is free to use or locked. The run all tasks and each taks will wait until the required data are free (maybe setting priority of the waiting tasks to low would suffice).&lt;/li&gt;&#xA;&lt;li&gt;Pick maximum number of edges that have no common vertices (better way then greedy algorithm?) and then run all corresponding tasks without locking. Repeat until there are remaining edges.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am not sure which approach is more efficient. Any ideas/insights? I am totally new to parallel programming...&lt;/p&gt;&#xA;" OwnerUserId="4591" LastActivityDate="2013-06-19T09:13:10.850" Title="Optimal Scheduling of Parallel Tasks with Known Dependencies" Tags="|optimization|parallel-computing|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7708" PostTypeId="2" ParentId="7707" CreationDate="2013-06-19T09:13:10.850" Score="1" Body="&lt;p&gt;I work on similar problems, i.e. &lt;a href=&quot;http://www.dur.ac.uk/pedro.gonnet/?page_id=6&quot; rel=&quot;nofollow&quot;&gt;task-based parallelism&lt;/a&gt;, and use something similar to your first approach. This is usually referred to as &quot;dynamic scheduling&quot; or &quot;data-dependent execution&quot; vs. &quot;static scheduling&quot;, i.e. your second approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The efficiency of this approach is highly dependent on how you implement your locks, and what the threads do if a task cannot be computed. I use atomic &lt;a href=&quot;http://en.wikipedia.org/wiki/Compare-and-swap&quot; rel=&quot;nofollow&quot;&gt;compare-and-swap&lt;/a&gt; operations to implement the locks, and fast task queues to find tasks that have free locks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also worthwhile for each thread to look for tasks that require resources (vertices) that it has previously owned, as this can improve caching behaviour.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-06-19T09:13:10.850" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7709" PostTypeId="2" ParentId="5414" CreationDate="2013-06-19T10:58:33.037" Score="0" Body="&lt;p&gt;In a 2012 &lt;a href=&quot;https://www.math.temple.edu/~szyld/reports/report_Riccati.rev.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;, Simoncini, Szyld and Monsalve review several techniques for solving large scale Riccati equations but mainly focus on a Galerkin projection approach to Lyapunov equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as a basic assumption they have $A$ always Hurwitz, what would render the Gramians definite (if E is positive definite). &lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-06-19T10:58:33.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7710" PostTypeId="2" ParentId="7702" CreationDate="2013-06-19T13:35:09.487" Score="12" Body="&lt;p&gt;It depends on your problem and your ODE solver/time discretization. If you have a hyperbolic PDE and want to solve it with an explicit method, then you need the time step restriction (called the Courant-Friedrichs-Lewy/CFL condition) or your numerical solution will typically oscillate and may grow to $\pm\infty$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you have a parabolic problem and an implicit time discretization, then you don't &lt;em&gt;need&lt;/em&gt; the restriction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will have to tell more about your problem for us to be able to give a more detailed answer.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-06-19T20:52:54.340" LastActivityDate="2013-06-19T20:52:54.340" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7711" PostTypeId="1" CreationDate="2013-06-19T14:40:14.323" Score="4" ViewCount="3359" Body="&lt;p&gt;I have a $50000\times 50000$ matrix $A$ sparse matrix containing only 5 non-zero elements in each row. Now the problem is that the diagonal elements and the constants (in $B$ matrix such that $AX=B$) get updated after every iteration. If I go by the usual Matlab function &lt;code&gt;inv&lt;/code&gt; or via Gauss Elimination, it takes around 130 seconds for the solution to be computed for a single iteration and the number of iterations required by the problem is somewhat of the order 100-500 to compute the final solution $X$. Need some suggestions on this one if I am not wanting to use parallel computation.&lt;/p&gt;&#xA;" OwnerUserId="4593" LastEditorUserId="386" LastEditDate="2013-07-03T16:01:20.580" LastActivityDate="2015-11-04T10:11:17.790" Title="Algorithm for Sparse-Matrix Inverse" Tags="|linear-algebra|matrix|sparse-matrix|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="7712" PostTypeId="2" ParentId="7711" CreationDate="2013-06-19T15:01:37.123" Score="3" Body="&lt;p&gt;The first idea that comes to my mind is to use an iterative method for solving the linear system. This might speed-up the computation of a single iteration (depending on the method (GMRES, conjugate gradient, BiCGStab,... and the size of B) and it will take advantage of the sparsity of the matrix A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, iterative methods let you define a preconditioner. If the diagonal coefficient does not change too much between the iterations, it might be worth spending some time at the begining to build a good preconditioner (e.g. ILU) and reuse it for all the iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The answers to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5228/sparse-linear-solver-for-many-right-hand-sides&quot;&gt;this question&lt;/a&gt; might be useful.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-06-19T15:01:37.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7714" PostTypeId="1" AcceptedAnswerId="7791" CreationDate="2013-06-19T16:27:30.213" Score="41" ViewCount="5427" Body="&lt;p&gt;I'm currently looking into parallel methods for ODE integration. There is a lot of new and old literature out there describing a wide range of approaches, but I haven't found any recent surveys or overview articles describing the topic in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's the book by Burrage [1], but it's almost 20 years old and hence does not cover many of the more modern ideas like the parareal algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] K. Burrage, Parallel and Sequential Methods for Ordinary Differential Equations, Clarendon Press, Oxford, 1995&lt;/p&gt;&#xA;" OwnerUserId="4230" LastActivityDate="2020-11-30T07:40:58.633" Title="What's the state of the art in parallel ODE methods?" Tags="|parallel-computing|ode|numerical-analysis|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7715" PostTypeId="1" CreationDate="2013-06-19T19:58:46.780" Score="3" ViewCount="1628" Body="&lt;p&gt;I'd like to compute $u(y) = \int f(x,y) dx$, but don't see a really efficient way to do this in FEniCS.  My first thought was to just assemble(f*v*dx) for two different meshes. (try1, below), but it seems I can't combine 1D and 2D meshes, and that the DG meshes have to have at least 2 basis functions in each direction, and I'm not sure how the mesh topology -&gt; basis coefficient layout is controlled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My next thought was to solve du/dx = f with a zero-BC on one side, to accumulate the integral on x = 1.  That one gets all the way through assembling the problem, then fails with a singular matrix exception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to do this in FEniCS?  To solve this problem in general, I would suggest adding general convolutions to FEniCS, so that $u = \int h(\vec x-\vec y) v(\vec y) dy$ could be done by an appropriate matrix-multiply...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;&#xA;m = UnitSquareMesh(12, 12)&#xA;V = FunctionSpace(m, &quot;Lagrange&quot;, 1)&#xA;&#xA;f = Function(V)&#xA;f.interpolate(Expression(&quot;x[1]-x[0]&quot;)) # int_0^1 f dx = x[1] - 0.5&#xA;&#xA;def try1():&#xA;    m2 = UnitSquareMesh(1, 12)&#xA;    V2 = FunctionSpace(m2, &quot;DG&quot;, 0)&#xA;    u = Function(V2)&#xA;    print u.vector().array().shape&#xA;    v = TestFunction(V2)&#xA;    A = assemble(f*v*dx)&#xA;    print A.array()*12.0&#xA;&#xA;def try2():&#xA;    du = TrialFunction(V)&#xA;    v = TestFunction(VectorFunctionSpace(m, &quot;Lagrange&quot;, 1))&#xA;&#xA;    bc = DirichletBC(V, Constant(0.0), lambda x,b: b and x[0] &amp;lt; 1e-8)&#xA;    #bc2 = DirichletBC(V, Constant(0.0), lambda x,b: b and x[1] &amp;lt; 1e-8)&#xA;&#xA;    a = inner(grad(du), v)*dx&#xA;    L = f*v[0]*dx&#xA;    #L2 = f*v[1]*dx&#xA;&#xA;    u = Function(V)&#xA;&#xA;    solve(a == L, u, bcs=bc)&#xA;&#xA;    print u.vector().array()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4594" LastActivityDate="2013-06-20T15:14:09.260" Title="Subdomain Integrals in Fenics" Tags="|fenics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7716" PostTypeId="1" CreationDate="2013-06-19T20:34:24.897" Score="1" ViewCount="132" Body="&lt;p&gt;I'm new here so please forgive me if I lack proper stack exchange etiquette. &#xA;So, I was wondering if anyone here could provide insight on a problem that I am running into with with a Hartree-Fock calculation. I wrote a code from scratch that attempts to calculate the HF energy and basis for a 1D potential that is supposed to model the nucleon-nucleon two body interaction: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;V(x)= 60 e^{-25x^2} - 15 e^{-1.56x^2} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I run the code for so called &quot;spinless&quot; fermions, it converges and gives results similar to those given in literature on this potential. However, if I add any spin degrees of freedom to the problem, the SCF procedure does not converge. Instead it arrives at a group of two or more sets of eigenvectors which when used to calculate the new fock-matrix will eventually reproduce one another.     I want to believe that this is because I am trapped in a local minimum in the HF functional.   So my main question is: Have any of you ever encountered such a problem with a basic HF code? Considering that in some cases I get good results and in other cases no results, I feel that the problem lies in the physics of the potential, rather than in my code.   Is this a reasonable statement in your eyes? &lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2013-06-19T20:34:24.897" Title="Hatree-Fock, reasons for convergence/ non-convergence" Tags="|convergence|hartree-fock|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7717" PostTypeId="2" ParentId="7715" CreationDate="2013-06-19T20:54:22.720" Score="1" Body="&lt;p&gt;I don't think there is a generic way to do this, since the (discrete) representation of $u(y)$ has to be defined beforehand. If you want it expressed in a 1D finite element basis then you can do the following&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;define the 1D FEM basis $\{\phi_i(y)\}$&lt;/li&gt;&#xA;&lt;li&gt;for every $\{\phi_i(y)\}$ define a 2D expression $b_i(x,y) = \phi_i(y)$&lt;/li&gt;&#xA;&lt;li&gt;test every $b_i$ against your $f$ to get&#xA;$$ \int \int b_i(x,y)f(x,y)dxdy=\int\phi_i(y)u(y)dy =: \tilde u_i&#xA;$$&lt;/li&gt;&#xA;&lt;li&gt;compute the coefficients of the expansion of $u$ in the FEM basis - $u(y) \approx \sum_i u_i \phi_i(y)$ - via&#xA;$$ \begin{bmatrix} \vdots \\ u_i \\ \vdots \end{bmatrix} = \mathcal M_{\phi}^{-1} \begin{bmatrix} \vdots \\ \tilde u_i \\ \vdots \end{bmatrix},&#xA;$$&#xA;where $M_\phi$ is the FEM mass matrix.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you have the expression &lt;code&gt;b_i&lt;/code&gt; in fenics and function &lt;code&gt;f&lt;/code&gt;, then you find&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tu_i = assemble(b_i*f*dx)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-06-19T20:54:22.720" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7718" PostTypeId="2" ParentId="7696" CreationDate="2013-06-19T21:42:41.227" Score="5" Body="&lt;p&gt;Without some information about the construction of these $12\times 12$ positive definite real symmetric matrices, the suggestions to be made are of necessity fairly limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I downloaded the Armadillo package from Sourceforge and took a look at the documentation.  Try to improve performance of separately computing $\det(Q)$ and $\det(12I - Q - J)$, where $J$ is the rank one matrix of all ones, by setting e.g &lt;code&gt;det(Q,slow=false)&lt;/code&gt;.  The documentation notes that this is the default for matrices up to size $4\times 4$, so by omission I assume the &lt;code&gt;slow=true&lt;/code&gt; option is a default for the $12\times 12$ case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What &lt;code&gt;slow=true&lt;/code&gt; &lt;em&gt;presumably&lt;/em&gt; does is partial or full pivoting in getting a row echelon form, from which the determinant is easily found.  However you know in advance the matrix $Q$ is positive definite, so pivoting is unnecessary for stability (at least presumptively for the bulk of your computations.  It's unclear if the Armadillo package throws an exception if the pivots become unduly small, but this should be a feature of a reasonable numerical linear algebra package. &lt;strong&gt;EDIT:&lt;/strong&gt; I found the Armadillo code that implements &lt;code&gt;det&lt;/code&gt; in header file &lt;code&gt;include\armadillo_bits\auxlib_meat.hpp&lt;/code&gt;, using C++ templates for substantial functionality.  The setting &lt;code&gt;slow=false&lt;/code&gt; doesn't appear to affect how a $12\times 12$ determinant will be done because the computation gets &quot;thrown over a wall&quot; to LAPACK (or ATLAS) at that point with no indication that pivoting is not required; see &lt;code&gt;det_lapack&lt;/code&gt; and its invocations in that file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other point would be to follow their recommendation of building the Armadillo package linking to high speed replacements for BLAS and LAPACK, if you are indeed using those; see Sec. 5 of the Armadillo README.TXT file for details.  [The use of a dedicated 64-bit version of BLAS or LAPACK is also recommended for speed on current 64-bit machines.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Row reduction to echelon form is essentially Gaussian elimination, and has arithmetic complexity $\frac{2}{3} n^3 + O(n^2)$.  For both matrices this then amounts to twice that work, or $\frac{4}{3} n^3 + O(n^2)$.  These operations may well be the &quot;bottleneck&quot; in your processing, but there's little hope that without special structure in $Q$ (or some known relationships among the trillion test cases allowing amortization) the work could be reduced to $O(n^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For comparison, expansion by cofactors of a general $n\times n$ matrix involves $n!$ multiplication operations (and roughly as many additions/subtractions), so for $n=12$ the comparison ($12! = 479001600$ vs. $\frac{2}{3} n^3 = 1152$) clearly favors elimination over cofactors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach requiring $\frac{4}{3} n^3 + O(n^2)$ work would be reducing $Q$ to tridiagonal form with Householder transformations, which also puts $12I - Q$ into tridiagonal form.  Computing $\det(Q)$ and $\det(12I - Q - J)$ can thereafter be done in $O(n)$ operations.  [The effect of the rank one update $-J$ in the second determinant can be expressed as a scalar factor given by solving one tridiagonal system.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementing such an independent computation might be worthwhile as a check on the results of successful (or failed) calls to Armadillo's &lt;code&gt;det&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Special Case:&lt;/strong&gt;  As suggested by a Comment of Jernej, suppose that $Q = D - J$ where $J$ as before is the (rank 1) matrix of all ones and $D=\text{diag}(d_1,\ldots,d_n)$ is a nonsingular (positive) diagonal matrix.  Indeed for the proposed application in graph theory these would be integer matrices.  Then an explicit formula for $\det(Q)$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \det(Q) = \left(\prod_{i=1}^n d_i \right)\left(1 - \sum_{i=1}^n d_i^{-1} \right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sketch of its proof affords an opportunity to illustrate wider applicability, i.e. whenever $D$ has a known determinant and the system $Dv = (1\ldots 1)^T$ is quickly solved.  Begin by factoring out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \det(D - J) = \det(D) \cdot \det(I - D^{-1}J) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now $D^{-1}J$ is again rank 1, namely $(d_1^{-1}\ldots d_n^{-1})^T(1\ldots 1)$.  Note that the second determinant is simply:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(1) = \det(I- D^{-1}J) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f(x)$ is the characteristic polynomial of $D^{-1}J$.  As a rank 1 matrix, $f(x)$ must have (at least) $n-1$ factors of $x$ to account for its nullspace.  The &quot;missing&quot; eigenvalue is $\sum d_i^{-1}$, as may be seen from the computation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D^{-1}J\; (d_1^{-1}\ldots d_n^{-1})^T = &#xA;\left(\sum d_i^{-1} \right) (d_1^{-1}\ldots d_n^{-1})^T $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It follows that the characteristic polynomial $f(x) = x^{n-1} (x - \sum d_i^{-1})$, and $f(1)$ is as shown above for $\det(I - D^{-1}J)$, $1 - \sum d_i^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that if $Q = D-J$, then $12I - Q - J = 12I - D + J - J = 12I - D$, a diagonal matrix whose determinant is simply the product of its diagonal entries.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2013-06-20T19:48:56.100" LastActivityDate="2013-06-20T19:48:56.100" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7719" PostTypeId="2" ParentId="7696" CreationDate="2013-06-20T01:42:24.153" Score="8" Body="&lt;p&gt;Since you're already using C++ and your matrices are symmetric positive definite, I would perform an unpivoted $LDL^T$ factorization of $Q$ and also of $12I-Q-J$. Here I'm assuming that $12I-Q-J$ is also positive definite, otherwise the $LDL^T$ will require pivoting for numerical stability (it's also possible that even though it's not positive definite, pivoting is not needed, but you have to try it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is faster than an LU factorization, and also faster than Cholesky because square roots are avoided. The determinant is simply the product of the elements of the diagonal $D$ matrix. The code to perform an LDL factorization is so simple you can write it in less than 50 lines of C. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Cholesky_decomposition#LDL_decomposition_2&quot;&gt;Wikipedia page&lt;/a&gt; on it describes the algorithm, and I have some simple templated code to do Cholesky &lt;a href=&quot;https://github.com/victorliu/RNP2/blob/master/RNP/LA/Cholesky.hpp&quot;&gt;here&lt;/a&gt;. You can vastly simplify that and modify it to avoid the square root to implement the $LDL^T$ factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you can also control the storage format, you can further optimize the routine to store only half the matrix and pack it in a linear array to maximize memory locality. I would also write simple custom dot product and rank-1 update routines since the problem sizes are so small, you should let the compiler inline the routines to reduce call overhead. Since it's a fixed size loop, the compiler should be able to automatically inline and unroll things when appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would avoid trying to play tricks to take advantage of the fact that $12I-Q-J$ contains $Q$ inside the expression. It is likely that for such small problem sizes, these tricks end up being slower than simply performing two separate determinant computations. Of course, the only way to verify these claims is to try it.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-06-20T01:42:24.153" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7720" PostTypeId="1" AcceptedAnswerId="7762" CreationDate="2013-06-20T03:13:05.287" Score="4" ViewCount="758" Body="&lt;p&gt;I want to use the PETSc library to do some numerical work on finite element and parallel computing, but I wonder what I should know first to use these libraries. Could you give me some guiding information? Any answer of comment will be appreciated. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="4562" LastActivityDate="2013-06-23T17:28:38.463" Title="What is the basic requirement to understand the PETSc library?" Tags="|finite-element|finite-difference|petsc|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7721" PostTypeId="2" ParentId="7705" CreationDate="2013-06-20T04:11:04.603" Score="7" Body="&lt;p&gt;In alphabetical order (&lt;strong&gt;disclaimer&lt;/strong&gt;: I am the main author of &lt;a href=&quot;http://code.google.com/p/elemental&quot;&gt;Elemental&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;DPLASMA&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://icl.eecs.utk.edu/dplasma/&quot;&gt;Distributed Parallel Linear Algebra Software for Multicore Architectures (DPLASMA)&lt;/a&gt; is a relatively recent and ongoing effort by &lt;a href=&quot;http://icl.cs.utk.edu/~bosilca/&quot;&gt;Bosilca&lt;/a&gt; et al. to extend &lt;a href=&quot;http://icl.cs.utk.edu/plasma/index.html&quot;&gt;PLASMA&lt;/a&gt; to distributed-memory machines. &lt;a href=&quot;http://icl.cs.utk.edu/parsec/custom/index.html?lid=145&amp;amp;slid=256&quot;&gt;Version 1.0.0&lt;/a&gt; supports distributed Cholesky factorizations, among many other operations. DPLASMA is written in C and uses MPI. Their website links to the following &lt;a href=&quot;http://icl.cs.utk.edu/graphics/posters/files/SC12-DPLASMA-PaRSEC.pdf&quot;&gt;handout from SC12&lt;/a&gt;, though there has been significant progress since then. This is the only software in the list which currently makes use of &lt;a href=&quot;https://en.wikipedia.org/wiki/Directed_acyclic_graph&quot;&gt;DAG&lt;/a&gt;-scheduling, and it appears to have begun development around 2010.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Elemental&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://code.google.com/p/elemental&quot;&gt;Elemental&lt;/a&gt; is a modern retake and extension of many of the ideas behind &lt;a href=&quot;http://www.cs.utexas.edu/~plapack&quot;&gt;PLAPACK&lt;/a&gt; and &lt;a href=&quot;http://www.netlib.org/scalapack&quot;&gt;ScaLAPACK&lt;/a&gt; (described below). It is written in C++ and currently sits on top of &lt;a href=&quot;http://www.netlib.org/blas&quot;&gt;BLAS&lt;/a&gt;, &lt;a href=&quot;http://netlib.org/lapack&quot;&gt;LAPACK&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Message_Passing_Interface&quot;&gt;MPI&lt;/a&gt;. Please see &lt;a href=&quot;https://elemental.googlecode.com/files/using-elem.pdf&quot;&gt;this recent presentation&lt;/a&gt; for a more detailed overview and some short example programs. Development began around 2009 and is still picking up steam. Contrary to PLAPACK and ScaLAPACK, its characteristic communication pattern is &lt;a href=&quot;https://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Allgather.html&quot;&gt;allgather&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;PLAPACK&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.utexas.edu/~plapack&quot;&gt;Parallel Linear Algebra PACKage (PLAPACK)&lt;/a&gt; was developed from around 1997-2000 and introduced the idea of providing an easy means (in the sense of just calling a routine named &lt;em&gt;Copy&lt;/em&gt;) of redistributing (sub)matrices from one distribution to another. &lt;a href=&quot;http://www.cs.utexas.edu/~rvdg&quot;&gt;The main author of PLAPACK&lt;/a&gt; has actively contributed to Elemental's development. PLAPACK was written with C and MPI, it sits on top of BLAS and LAPACK, and its characteristic communication pattern is &lt;a href=&quot;https://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Bcast.html&quot;&gt;broadcast&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;ScaLAPACK&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://netlib.org/scalapack&quot;&gt;Scalable Linear Algebra PACKage (ScaLAPACK)&lt;/a&gt; has been under development since about 1995 and is thus the earliest (and also most commonly used) of the bunch. ScaLAPACK is primarily written in Fortran, but several of its routines are written in C, and most of its communication is performed in terms of the &lt;a href=&quot;http://netlib.org/blacs&quot;&gt;Basic Linear Algebra Communication Subprograms&lt;/a&gt;, which is (now) typically implemented using MPI. It was purposely designed so that its high-level code resembles that of LAPACK, and the local computations that occur are typically calls to BLAS and LAPACK. The characteristic communication pattern in ScaLAPACK is &lt;a href=&quot;https://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Bcast.html&quot;&gt;broadcast&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2013-06-20T04:16:26.097" LastActivityDate="2013-06-20T04:16:26.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7722" PostTypeId="1" AcceptedAnswerId="7725" CreationDate="2013-06-20T06:53:46.960" Score="1" ViewCount="156" Body="&lt;p&gt;I have a &lt;code&gt;Matrix&amp;lt;double&amp;gt;&lt;/code&gt; on a single core, the result of doing an &lt;code&gt;MPI_Reduce&lt;/code&gt;. I want to do the Cholesky, so I need to distribute over multiple cores.  What's the easiest/most standard way of doing this in Elemental?  Do I need to &lt;code&gt;MPI_Bcast&lt;/code&gt; it out to all nodes first, and then move it into a &lt;code&gt;DistMatrix&lt;/code&gt;, or is there some way of doing something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Matrix&amp;lt;double&amp;gt; A = doSomeStuff();&#xA;DistMatrix&amp;lt;double, single-node&amp;gt; AdistSingle( A );&#xA;DistMatrix&amp;lt;double&amp;gt; Adist(AdistSingle );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or perhaps:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Matrix&amp;lt;double&amp;gt; A = doSomeStuff();&#xA;DistMatrix&amp;lt;double&amp;gt; Adist(A );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3613" LastActivityDate="2013-06-20T08:19:07.010" Title="How to convert a Matrix to a DistMatrix in Elemental?" Tags="|linear-algebra|parallel-computing|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7723" PostTypeId="1" AcceptedAnswerId="7724" CreationDate="2013-06-20T07:14:57.317" Score="3" ViewCount="194" Body="&lt;p&gt;TVD (total variation diminishing) finite difference methods that produce non-oscillatory solutions are based on the total variation. In &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521009243&quot; rel=&quot;nofollow&quot;&gt;LeVeque's book&lt;/a&gt; the total variation of a function $q(x)$ is defined as (p. 109)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$TV(q)=\sup\sum_{j=1}^N\left|q(\xi_i)-q(\xi_{i-1})\right|$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &quot;the supremum is taken over all subdivisions of the real line $-\infty=\xi_0&amp;lt;\xi_1&amp;lt;\ldots &amp;lt; \xi_N=\infty$&quot;. I do not understand that last part. Assuming a specific value for $N$ and a function $q(x)$, we can carry out the summation. I don't understand where the supremum comes into it or why it is necessary.  &lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2013-06-20T07:25:51.867" Title="Definition of TV in TVD finite difference methods" Tags="|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7724" PostTypeId="2" ParentId="7723" CreationDate="2013-06-20T07:25:51.867" Score="4" Body="&lt;p&gt;The supremum in the definition is necessary to capture the maximum variation in the function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example to illustrate what happens if we ignore the supremum and take any fixed partition of the real line:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the function $\sin(x)$, and let the subdivision of the real line be the zeros of $\sin(x)$. Then its variation over that subdivision is zero, even though its total variation is infinite because it does not have compact support. We could choose a similar example with compact support by replacing $\sin(x)$ with the zero function sufficiently far away from the origin and the point would be the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea of total variation is to capture the unsigned, total vertical distance traveled by the function. Oscillations like the sine function will travel a nonzero vertical distance, and will travel a greater vertical distance over a fixed interval as their frequency increases. To capture that vertical distance accurately for &lt;em&gt;all&lt;/em&gt; functions, you need the supremum to be able to adapt the subdivision to the function. Then you can detect the degree to which your solution &quot;oscillates&quot;, which makes it a useful property to track for numerical methods for solving hyperbolic partial differential equations.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-06-20T07:25:51.867" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7725" PostTypeId="2" ParentId="7722" CreationDate="2013-06-20T08:19:07.010" Score="3" Body="&lt;p&gt;&lt;code&gt;DistMatrix&amp;lt;T,CIRC,CIRC&amp;gt;&lt;/code&gt; was recently created for exactly this situation and stores a fully copy of the matrix on a single process. If you have an $n \times n$ &lt;code&gt;Matrix&amp;lt;double&amp;gt;&lt;/code&gt; stored on the root process, you can distribute it as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Construct an n x n matrix owned by process 0&#xA;DistMatrix&amp;lt;double,CIRC,CIRC&amp;gt; ARoot( n, n );&#xA;// Have the root process fill the local matrix of ARoot&#xA;if( commRank == 0 )&#xA;    ARoot.Matrix() = A;&#xA;// Redistribute ARoot into the standard matrix distribution&#xA;// (via a scatter)&#xA;DistMatrix&amp;lt;double&amp;gt; ADist( ARoot );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the &lt;code&gt;DistMatrix&amp;lt;T,CIRC,CIRC&amp;gt;&lt;/code&gt; class is very new and that the syntax might be simplified in the near future.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2013-06-20T08:19:07.010" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7726" PostTypeId="1" AcceptedAnswerId="7737" CreationDate="2013-06-20T08:24:04.317" Score="3" ViewCount="409" Body="&lt;p&gt;What implementation details need to change if I use a cell average approach rather than a cell total approach for the finite-volume method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the conservation law,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t + \mathcal{F}_x = s(x,t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;cell average&lt;/strong&gt; approach yields,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial}{\partial t}\int_{x_{j-1/2}}^{x_{j+1/2}} u(x,t)~dx = -\int_{x_{j-1/2}}^{x_{j+1/2}} F_x~dx + \int_{x_{j-1/2}}^{x_{j+1/2}}s(x,t)~dx \\&#xA;\frac{\partial}{\partial t} \tilde{u}(x,t) = \frac{1}{x_{j+1/2} - x_{j-1/2}}\left(\mathcal{F}_{j-1/2} - \mathcal{F}_{j+1/2}  \right) + \tilde{s}(x,t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;cell total&lt;/strong&gt; is the same but without dividing the flux term by the cell length $(x_{j+1/2} - x_{j-1/2})$ (I am considering 1D only).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When solving the equation numerically, do I need to define $\tilde{u}$ and $\tilde{s}$ differently in these two cases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A little confused about this subtlety. My first impression what that nothing needs to change other than the dividing the flux term by $(x_{j+1/2} - x_{j-1/2})$ when using a cell averaged approach. Is that correct?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-06-21T06:12:21.453" LastActivityDate="2013-06-21T07:39:21.120" Title="Finite volume with cell averages vs cell totals for conservation equations" Tags="|finite-volume|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7727" PostTypeId="2" ParentId="6951" CreationDate="2013-06-20T09:21:42.593" Score="-2" Body="&lt;p&gt;I did this stuff by a self-made script for a long time (but had to prepare the input manually for every molecule).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, I came along &lt;a href=&quot;https://code.google.com/p/confab/&quot; rel=&quot;nofollow&quot;&gt;confab&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Confab is an open source conformation generator whose goal is the systematic coverage of conformational space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A detailed description how it works can be found in the authors &lt;a href=&quot;http://dx.doi.org/10.1186/1758-2946-3-8&quot; rel=&quot;nofollow&quot;&gt;publication&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3486" LastEditorUserId="3486" LastEditDate="2015-04-16T08:04:59.400" LastActivityDate="2015-04-16T08:04:59.400" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7728" PostTypeId="1" AcceptedAnswerId="8042" CreationDate="2013-06-20T09:30:02.347" Score="0" ViewCount="367" Body="&lt;p&gt;Is there any free / open source solution to search for the minimum energy path with the nudged elastic band method in combination with the Turbomole Package?&lt;/p&gt;&#xA;" OwnerUserId="3486" LastActivityDate="2013-07-23T04:04:48.363" Title="nudged elastic band with Turbomole" Tags="|optimization|computational-chemistry|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7729" PostTypeId="1" CreationDate="2013-06-20T09:56:05.753" Score="3" ViewCount="673" Body="&lt;p&gt;How to make Elemental Gemm run quickly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;elemental.hpp&quot;&#xA;using namespace std;&#xA;using namespace elem;&#xA;&#xA;extern &quot;C&quot; {&#xA;    void openblas_set_num_threads(int num_threads);&#xA;}&#xA;&#xA;int main( int argc, char *argv[] ) {&#xA;    Initialize( argc, argv );&#xA;    const mpi::Comm comm = mpi::COMM_WORLD;&#xA;    const int commRank = mpi::CommRank( comm );&#xA;    const int commSize = mpi::CommSize( comm );&#xA;&#xA;    try {&#xA;        const int n = Input(&quot;--n&quot;,&quot;size of matrices&quot;,1000);&#xA;        const int nb = Input(&quot;--nb&quot;,&quot;algorithmic blocksize&quot;,128);&#xA;        int r = Input(&quot;--r&quot;,&quot;process grid height&quot;,0);&#xA;        ProcessInput();&#xA;&#xA;        SetBlocksize( nb );&#xA;        // If no process grid height was specified, try for a square&#xA;        if( r == 0 )&#xA;            r = Grid::FindFactor( commSize );&#xA;        Grid g( comm, r );&#xA;&#xA;        Matrix&amp;lt;double&amp;gt; A, B, C;&#xA;        Uniform( A, n, n );&#xA;        Uniform( B, n, n );&#xA;        Uniform( C, n, n );&#xA;        mpi::Barrier( comm );&#xA;        Timer timer;&#xA;        if( commRank == 0 ) {&#xA;            timer.Start();&#xA;            Gemm( NORMAL, NORMAL, 1., A, B, 0., C );&#xA;            std::cout &amp;lt;&amp;lt; &quot;Multithreaded time: &quot; &amp;lt;&amp;lt; timer.Stop() &amp;lt;&amp;lt; &quot; secs&quot;&#xA;                      &amp;lt;&amp;lt; std::endl;  &#xA;&#xA;            openblas_set_num_threads(1);&#xA;            timer.Start();&#xA;            Gemm( NORMAL, NORMAL, 1., A, B, 0., C );&#xA;            std::cout &amp;lt;&amp;lt; &quot;Sequential time: &quot; &amp;lt;&amp;lt; timer.Stop() &amp;lt;&amp;lt; &quot; secs&quot;&#xA;                      &amp;lt;&amp;lt; std::endl;  &#xA;        }&#xA;        openblas_set_num_threads(1);&#xA;&#xA;        if( commRank == 0 )&#xA;            timer.Start();&#xA;        DistMatrix&amp;lt;double,CIRC,CIRC&amp;gt; ARoot(n,n,g), BRoot(n,n,g); &#xA;        if( commRank == 0 ) {&#xA;            ARoot.Matrix() = A;&#xA;            BRoot.Matrix() = B;&#xA;            std::cout &amp;lt;&amp;lt; &quot;Population time: &quot; &amp;lt;&amp;lt; timer.Stop() &amp;lt;&amp;lt; &quot; secs&quot;&#xA;                      &amp;lt;&amp;lt; std::endl;&#xA;        }&#xA;&#xA;        if( commRank == 0 )&#xA;            timer.Start();&#xA;        DistMatrix&amp;lt;double&amp;gt; ADist( ARoot ), BDist( BRoot ), CDist(g);&#xA;        Zeros( CDist, n, n );&#xA;        mpi::Barrier( comm );&#xA;        if( commRank == 0 ) &#xA;            std::cout &amp;lt;&amp;lt; &quot;Scatter from root: &quot; &amp;lt;&amp;lt; timer.Stop() &amp;lt;&amp;lt; &quot; secs&quot;&#xA;                      &amp;lt;&amp;lt; std::endl;&#xA;&#xA;        if( commRank == 0 )&#xA;            timer.Start();&#xA;        Gemm( NORMAL, NORMAL, 1., ADist, BDist, 0., CDist ); &#xA;        mpi::Barrier( comm );   &#xA;        if( commRank == 0 ) &#xA;            std::cout &amp;lt;&amp;lt; &quot;Distributed: &quot; &amp;lt;&amp;lt; timer.Stop() &amp;lt;&amp;lt; &quot; secs&quot;&#xA;                      &amp;lt;&amp;lt; std::endl;&#xA;&#xA;        DistMatrix&amp;lt;double,CIRC,CIRC&amp;gt; CRoot( CDist );&#xA;        mpi::Barrier( comm );&#xA;        if( commRank == 0 ) &#xA;            std::cout &amp;lt;&amp;lt; &quot;Gather to root: &quot; &amp;lt;&amp;lt; timer.Stop() &amp;lt;&amp;lt; &quot; secs&quot;&#xA;                      &amp;lt;&amp;lt; std::endl;&#xA;    } catch( std::exception&amp;amp; e ) { ReportException(e); }&#xA;    Finalize();&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm running on 4 nodes, each node with 12 cores.  The blas is OpenBLAS, compiled from source, so it supports 12 threads, and affinity is turned off.  I get the following results for gemm, with n=4000:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;multithreaded blas, via Elemental Matrix: &lt;strong&gt;1.6s&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;singlethreaded blas. via Elemental Matrix: 11.9s&lt;/li&gt;&#xA;&lt;li&gt;DistMatrix, 1 mpi processes, singlethreaded blas: 13.3 seconds&lt;/li&gt;&#xA;&lt;li&gt;DistMatrix, 4 mpi processes, singlethreaded blas: 6.6 seconds&lt;/li&gt;&#xA;&lt;li&gt;DistMatrix, 48 mpi processes, singlethreaded blas: 30.2  seconds&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I also tried using multithreaded blas with DistMatrix, by commenting out the lines &lt;code&gt;openblas_set_num_threads(1)&lt;/code&gt; in the above code, and got the following results:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;DistMatrix, 4 mpi processes, multithreaded blas: 4.5 seconds&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Why am I getting times for the DistMatrix on 4 nodes which are not competitive with the multithreaded blas time on a single compute node?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I also tried setting A to MC,STAR; and B to STAR,MR, following page 27 of &quot;Rethinking distributed dense linear algebra&quot;, by Jack Poulson 2012, but the results were no better for me:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;DistMatrix, mpi 4 processes, multithreaded blas: 6.2 seconds&lt;/li&gt;&#xA;&lt;li&gt;DistMatrix, mpi 48 processes, singlethreaded blas: 35.3 seconds&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit 2: added code to prevent any code being optimized out, ie the functions &lt;code&gt;readMatrix&lt;/code&gt;, and the &lt;code&gt;cout &amp;lt;&amp;lt; sum &amp;lt;&amp;lt; endl&lt;/code&gt; at the end, but no change in timings:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;blas single-threaded: 12.2 seconds&lt;/li&gt;&#xA;&lt;li&gt;blas multi-threaded: &lt;strong&gt;1.6&lt;/strong&gt; seconds&lt;/li&gt;&#xA;&lt;li&gt;DistMatrix 4 nodes, 4 processes, multithreaded blas: 4.5 seconds&lt;/li&gt;&#xA;&lt;li&gt;DistMatrix 4 nodes, 48 processes, singlethreaded blas: &lt;strong&gt;29.8&lt;/strong&gt; seconds&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit 3: note that scalapack sdgemm also seems to run more slowly than multithreaded blas for me, but faster than Elemental, for me:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;scalapack sdgemm, 4 nodes, 4 processes, multithreaded blas: 2.11 seconds&#xA;scalapack sdgemm, 4 nodes, 48 processes, singlethreaded blas: 29.1 seconds&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that for both scalapack and elemental, it seems that it is faster to use one mpi process one node, and turn on OpenBLAS multithreading, than to use one mpi process per core, and turn off OpenBLAS multithreading.  Which makes sense, since one can then take advantage of shared memory, reducing the amount of communications required?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;code for scalapack tests:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma once&#xA;&#xA;extern &quot;C&quot; {&#xA;    struct DESC{&#xA;        int DTYPE_;&#xA;        int CTXT_;&#xA;        int M_;&#xA;        int N_;&#xA;        int MB_;&#xA;        int NB_;&#xA;        int RSRC_;&#xA;        int CSRC_;&#xA;        int LLD_;&#xA;    } ;&#xA;&#xA;    void blacs_pinfo_( int *iam, int *nprocs );&#xA;    void blacs_get_( int *icontxt, int *what, int *val );&#xA;    void blacs_gridinit_( int *icontxt, char *order, int *nprow, int *npcol );&#xA;    void blacs_gridinfo_( int *context, int *nprow, int *npcol, int *myrow, int *mycol );&#xA;    void blacs_gridexit_( int *context );&#xA;    void blacs_exit_( int *code );&#xA;&#xA;    int numroc_( int *n, int *nb, int *iproc, int *isrcproc, int *nprocs );&#xA;    void descinit_( struct DESC *desc, int *m, int *n, int *mb, int *nb, int *irsrc, int *icsrc, int *ictxt, int *lld, int *info );&#xA;    void pdlaprnt_( int *m, int *n, double *a, int *ia, int *ja, struct DESC *desca, int *irprnt,&#xA;        int *icprnt, const char *cmatnm, int *nout, double *work, int cmtnmlen );&#xA;    void pdgemm_( char *transa, char *transb, int *m, int *n, int *k, double *alpha,&#xA;         double *a, int *ia, int *ja, struct DESC *desca, double *b, int *ib, int *jb,&#xA;        struct DESC *descb, double *beta, double *c, int *ic, int *jc, struct DESC *descc );&#xA;}&#xA;&#xA;void blacs_pinfo( int *p, int *P ) {&#xA;    blacs_pinfo_( p, P );&#xA;}&#xA;&#xA;int blacs_get( int icontxt, int what ) {&#xA;    int val;&#xA;    blacs_get_( &amp;amp;icontxt, &amp;amp;what, &amp;amp;val );&#xA;    return val;&#xA;}&#xA;&#xA;int blacs_gridinit( int icontxt, bool isColumnMajor, int nprow, int npcol ) {&#xA;    int newcontext = icontxt;&#xA;    char order = isColumnMajor ? 'C' : 'R';&#xA;    blacs_gridinit_( &amp;amp;newcontext, &amp;amp;order, &amp;amp;nprow, &amp;amp;npcol );&#xA;    return newcontext;&#xA;}&#xA;&#xA;void blacs_gridinfo( int context, int nprow, int npcol, int *myrow, int *mycol ) {&#xA;    blacs_gridinfo_( &amp;amp;context, &amp;amp;nprow, &amp;amp;npcol, myrow, mycol );&#xA;}&#xA;&#xA;void blacs_gridexit( int context ) {&#xA;    blacs_gridexit_( &amp;amp;context );&#xA;}&#xA;&#xA;void blacs_exit( int code ) {&#xA;    blacs_exit_( &amp;amp;code );&#xA;}&#xA;&#xA;int numroc( int n, int nb, int iproc, int isrcproc, int nprocs ) {&#xA;    return numroc_( &amp;amp;n, &amp;amp;nb, &amp;amp;iproc, &amp;amp;isrcproc, &amp;amp;nprocs );&#xA;}&#xA;&#xA;void descinit( struct DESC *desc, int m, int n, int mb, int nb, int irsrc, int icsrc, int ictxt, int lld ) {&#xA;    int info;&#xA;    descinit_( desc, &amp;amp;m, &amp;amp;n, &amp;amp;mb, &amp;amp;nb, &amp;amp;irsrc, &amp;amp;icsrc, &amp;amp;ictxt, &amp;amp;lld, &amp;amp;info );&#xA;    if( info != 0 ) {&#xA;        throw runtime_error( &quot;non zero info: &quot; + toString( info ) );&#xA;    }&#xA;//    return info;&#xA;}&#xA;&#xA;void pdlaprnt( int m, int n, double *A, int ia, int ja, struct DESC *desc, int irprnt,&#xA;    int icprnt, const char *cmatnm, int nout, double *work ) {&#xA;    int cmatnmlen = strlen(cmatnm);&#xA;    pdlaprnt_( &amp;amp;m, &amp;amp;n, A, &amp;amp;ia, &amp;amp;ja, desc, &amp;amp;irprnt, &amp;amp;icprnt, cmatnm, &amp;amp;nout, work, cmatnmlen );&#xA;}&#xA;&#xA;void pdgemm( bool isTransA, bool isTransB, int m, int n, int k, double alpha,&#xA;     double *a, int ia, int ja, struct DESC *desca, double *b, int ib, int jb,&#xA;    struct DESC *descb, double beta, double *c, int ic, int jc, struct DESC *descc ) {&#xA;    char transa = isTransA ? 'T' : 'N';&#xA;    char transb = isTransB ? 'T' : 'N';&#xA;    pdgemm_( &amp;amp;transa, &amp;amp;transb, &amp;amp;m, &amp;amp;n, &amp;amp;k, &amp;amp;alpha, a, &amp;amp;ia, &amp;amp;ja, desca, b, &amp;amp;ib, &amp;amp;jb,&#xA;        descb, &amp;amp;beta, c, &amp;amp;ic, &amp;amp;jc, descc );&#xA;}&#xA;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;using namespace std;&#xA;&#xA;#include &quot;cpputils.h&quot;&#xA;&#xA;#include &quot;args.h&quot;&#xA;#include &quot;scalapack.h&quot;&#xA;&#xA;extern &quot;C&quot; {&#xA;    void openblas_set_num_threads(int num_threads);&#xA;}&#xA;&#xA;int getRootFactor( int n ) {&#xA;    for( int t = sqrt(n); t &amp;gt; 0; t-- ) {&#xA;        if( n % t == 0 ) {&#xA;            return t;&#xA;        }&#xA;    }&#xA;    return 1;&#xA;}&#xA;&#xA;// conventions:&#xA;// M_ by N_ matrix block-partitioned into MB_ by NB_ blocks, then&#xA;// distributed according to 2d block-cyclic scheme&#xA;&#xA;// based on http://acts.nersc.gov/scalapack/hands-on/exercise3/pspblasdriver.f.html&#xA;&#xA;int main( int argc, char *argv[] ) {&#xA;    int p, P;&#xA;    blacs_pinfo( &amp;amp;p, &amp;amp;P );&#xA;    mpi_print( toString(p) + &quot; / &quot; + toString(P) );&#xA;&#xA;    int n;&#xA;    int numthreads;&#xA;    Args( argc, argv ).arg(&quot;N&quot;, &amp;amp;n ).arg(&quot;numthreads&quot;, &amp;amp;numthreads ).go();&#xA;    openblas_set_num_threads( numthreads );&#xA;&#xA;    int nprows = getRootFactor(P);&#xA;    int npcols = P / nprows;&#xA;    if( p == 0 ) cout &amp;lt;&amp;lt; &quot;grid: &quot; &amp;lt;&amp;lt; nprows &amp;lt;&amp;lt; &quot; x &quot; &amp;lt;&amp;lt; npcols &amp;lt;&amp;lt; endl;&#xA;&#xA;    int system = blacs_get( -1, 0 );&#xA;    int grid = blacs_gridinit( system, true, nprows, npcols );&#xA;    if( p == 0 ) cout &amp;lt;&amp;lt; &quot;system context &quot; &amp;lt;&amp;lt; system &amp;lt;&amp;lt; &quot; grid context: &quot; &amp;lt;&amp;lt; grid &amp;lt;&amp;lt; endl;&#xA;&#xA;    int myrow, mycol;&#xA;    blacs_gridinfo( grid, nprows, npcols, &amp;amp;myrow, &amp;amp;mycol );&#xA;    mpi_print(&quot;grid, me: &quot; + toString(myrow) + &quot;, &quot; + toString(mycol) );&#xA;&#xA;    if( myrow &amp;gt;= nprows || mycol &amp;gt;= npcols ) {&#xA;        mpi_print(&quot;not needed, exiting&quot;);&#xA;        blacs_gridexit( grid );&#xA;        blacs_exit(0);&#xA;        exit(0);&#xA;    }&#xA;&#xA;    // A     B       C&#xA;    // m x k k x n = m x n&#xA;    // nb: blocksize&#xA;&#xA;    // nprows: process grid, number rows&#xA;    // npcols: process grid, number cols&#xA;    // myrow: process grid, our row&#xA;    // mycol: process grid, our col&#xA;    int m = n;&#xA;    int k = n;&#xA;//    int nb = min(n,128); // nb is column block size for A, and row blocks size for B&#xA;    int nb=min(n/P,128);&#xA;&#xA;    int mp = numroc( m, nb, myrow, 0, nprows ); // mp number rows A owned by this process&#xA;    int kp = numroc( k, nb, myrow, 0, nprows ); // kp number rows B owned by this process&#xA;    int kq = numroc( k, nb, mycol, 0, npcols ); // kq number cols A owned by this process&#xA;    int nq = numroc( n, nb, mycol, 0, npcols ); // nq number cols B owned by this process&#xA;    mpi_print( &quot;mp &quot; + toString(mp) + &quot; kp &quot; + toString(kp) + &quot; kq &quot; + toString(kq) + &quot; nq &quot; + toString(nq) );&#xA;&#xA;    struct DESC desca, descb, descc;&#xA;    descinit( (&amp;amp;desca), m, k, nb, nb, 0, 0, grid, max(1, mp) );&#xA;    descinit( (&amp;amp;descb), k, n, nb, nb, 0, 0, grid, max(1, kp) );&#xA;    descinit( (&amp;amp;descc), m, n, nb, nb, 0, 0, grid, max(1, mp) );&#xA;    mpi_print( &quot;desca.LLD_ &quot; + toString(desca.LLD_) + &quot; kq &quot; + toString(kq) );&#xA;    double *ipa = new double[desca.LLD_ * kq];&#xA;    double *ipb = new double[descb.LLD_ * nq];&#xA;    double *ipc = new double[descc.LLD_ * nq];&#xA;&#xA;    for( int i = 0; i &amp;lt; desca.LLD_ * kq; i++ ) {&#xA;        ipa[i] = p;&#xA;    }&#xA;    for( int i = 0; i &amp;lt; descb.LLD_ * nq; i++ ) {&#xA;        ipb[i] = p;&#xA;    }&#xA;&#xA;    if( p == 0 ) cout &amp;lt;&amp;lt; &quot;created matrices&quot; &amp;lt;&amp;lt; endl;&#xA;    double *work = new double[nb];&#xA;    if( n &amp;lt;=5 ) {&#xA;        pdlaprnt( n, n, ipa, 1, 1, &amp;amp;desca, 0, 0, &quot;A&quot;, 6, work );&#xA;        pdlaprnt( n, n, ipb, 1, 1, &amp;amp;descb, 0, 0, &quot;B&quot;, 6, work );&#xA;    }&#xA;&#xA;    NanoTimer timer;&#xA;    pdgemm( false, false, m, n, k, 1,&#xA;                  ipa, 1, 1, &amp;amp;desca, ipb, 1, 1, &amp;amp;descb,&#xA;                  1, ipc, 1, 1, &amp;amp;descc );&#xA;    MPI_Barrier( MPI_COMM_WORLD );&#xA;    if( p == 0 ) timer.toc(&quot;pdgemm&quot;);&#xA;&#xA;    blacs_gridexit( grid );&#xA;    blacs_exit(0);&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the output from Jack Poulson's elemental/examples/blas-like/Gemm.cpp program, run with &lt;code&gt;OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 mpirun.mpich2 -hosts host3,host1,host2,host4 48 ./ElementalExampleGemm --m 2000 --n 2000 --k 2000 --nb 128&lt;/code&gt;, ie 48 processes, 1 thread per process:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g: 6 x 8&#xA;Sequential: 1.74586 secs and 9.16452 GFLops&#xA;Populate root node: 0.0340741 secs&#xA;Spread from root: 0.443471 secs&#xA;[MC,* ] AllGather: 0.0068028 secs, 50.2758 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.224466 secs, 1.14048 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00301719 secs and 7.08474 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00582409 secs, 58.7244 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.221785 secs, 1.15427 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00290704 secs and 7.35319 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00559711 secs, 61.1058 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.258585 secs, 0.990002 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00293088 secs and 7.29337 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00562692 secs, 60.7821 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.266162 secs, 0.961821 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00652504 secs and 3.276 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00574803 secs, 59.5014 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.253986 secs, 1.00793 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00300407 secs and 7.11567 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00567889 secs, 60.2258 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.263011 secs, 0.973343 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00289297 secs and 7.38894 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00561213 secs, 60.9422 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.0310259 secs, 8.25117 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00288296 secs and 7.41461 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00552988 secs, 61.8487 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.229407 secs, 1.11592 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00290298 secs and 7.36346 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00556993 secs, 61.4039 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.259156 secs, 0.987822 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00277686 secs and 7.6979 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00564504 secs, 60.587 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.260839 secs, 0.981448 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00277185 secs and 7.7118 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00549412 secs, 62.2513 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.224814 secs, 1.13872 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00276208 secs and 7.7391 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00556684 secs, 61.4381 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.216236 secs, 1.18389 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00276899 secs and 7.71977 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00551414 secs, 62.0252 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.22506 secs, 1.13747 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00276208 secs and 7.7391 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.005409 secs, 63.2309 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.255941 secs, 1.00023 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00276995 secs and 7.71712 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00536704 secs, 63.7252 MB/s for 334 x 128 local matrix&#xA;[* ,MR] AllGather: 0.225583 secs, 1.13484 MB/s for 128 x 250 local matrix&#xA;Local gemm: 0.00295305 secs and 7.23861 GFlops for 334 x 250 x 128 product&#xA;[MC,* ] AllGather: 0.00358391 secs, 59.6444 MB/s for 334 x 80 local matrix&#xA;[* ,MR] AllGather: 0.251425 secs, 0.636373 MB/s for 80 x 250 local matrix&#xA;Local gemm: 0.00167489 secs and 7.97664 GFlops for 334 x 250 x 80 product&#xA;Distributed Gemm: 3.80641 secs&#xA;Gathered to root: 0.399101 secs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: and results for 4 mpi processes (on 4 12-core nodes), run with &lt;code&gt;mpirun.mpich2 -hosts host3,host1,host2,host4 -np 4 ./ElementalExampleGemm --m 2000 --n 2000 --k 2000 --nb 128&lt;/code&gt;, ie with OpenBLAS multithreading activated:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g: 2 x 2&#xA;Sequential: 0.219173 secs and 73.0017 GFLops&#xA;Populate root node: 0.035708 secs&#xA;Spread from root: 0.482837 secs&#xA;[MC,* ] AllGather: 0.0147331 secs, 69.5035 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.010592 secs, 96.6769 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00869703 secs and 29.4353 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00796413 secs, 128.576 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00883698 secs, 115.877 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00752282 secs and 34.0298 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00717402 secs, 142.737 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.0083642 secs, 122.427 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00796413 secs and 32.1441 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00718212 secs, 142.576 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.0080409 secs, 127.349 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00650787 secs and 39.337 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00641584 secs, 159.605 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00688195 secs, 148.795 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00576997 secs and 44.3677 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00597095 secs, 171.497 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00652695 secs, 156.888 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00652885 secs and 39.2106 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00575399 secs, 177.963 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00634193 secs, 161.465 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00574183 secs and 44.5851 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00563598 secs, 181.69 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00627708 secs, 163.133 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00576282 secs and 44.4227 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.005867 secs, 174.535 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00619698 secs, 165.242 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.01108 secs and 23.1046 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00589108 secs, 173.822 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00623584 secs, 164.212 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00559402 secs and 45.7632 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00580001 secs, 176.551 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00648689 secs, 157.857 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00570703 secs and 44.857 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00566101 secs, 180.886 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00638199 secs, 160.452 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00575209 secs and 44.5056 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00572801 secs, 178.771 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00630784 secs, 162.338 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.005795 secs and 44.176 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00581098 secs, 176.218 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00644612 secs, 158.855 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00571203 secs and 44.8177 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00570321 secs, 179.548 MB/s for 1000 x 128 local matrix&#xA;[* ,MR] AllGather: 0.00653315 secs, 156.739 MB/s for 128 x 1000 local matrix&#xA;Local gemm: 0.00570703 secs and 44.857 GFlops for 1000 x 1000 x 128 product&#xA;[MC,* ] AllGather: 0.00371313 secs, 172.361 MB/s for 1000 x 80 local matrix&#xA;[* ,MR] AllGather: 0.00396085 secs, 161.582 MB/s for 80 x 1000 local matrix&#xA;Local gemm: 0.00396299 secs and 40.3735 GFlops for 1000 x 1000 x 80 product&#xA;Distributed Gemm: 0.327859 secs&#xA;Gathered to root: 0.237197 secs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3613" LastEditorUserId="3613" LastEditDate="2013-06-22T04:54:28.817" LastActivityDate="2013-06-22T23:27:51.540" Title="How to make Elemental Gemm run quickly?" Tags="|linear-algebra|matrix|parallel-computing|mpi|blas|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7730" PostTypeId="2" ParentId="7715" CreationDate="2013-06-20T15:14:09.260" Score="0" Body="&lt;p&gt;Another way would be to solve the Poisson equation using $f$ as a source, then use Gauss' law to compute the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7515/flux-across-a-non-boundary-line-segment-in-fenics&quot;&gt;flux across a line segment&lt;/a&gt; for each dividing plane.  That method still wouldn't be vectorized, but at least only requires 1 mesh function (for marking domains).  If I make that compromise, I may as well just iterate in python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def slice(mesh, n, L):&#xA;    subdomains = MeshFunction(&quot;size_t&quot;, mesh, mesh.topology().dim())&#xA;    subdomains.set_all(0)&#xA;    dx = L/float(n)&#xA;    for i in range(1, n):&#xA;            b = AutoSubDomain(lambda x: x[1] &amp;gt; i*dx-1e-8)&#xA;            b.mark(subdomains, i)&#xA;    return subdomains&#xA;&#xA;def try3():&#xA;    markers = slice(m, 12, 1.0)&#xA;    dX = dx[markers]&#xA;    v = [assemble(f*dX(i))*12.0 for i in range(12)]&#xA;    print v&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4594" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-06-20T15:14:09.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7731" PostTypeId="2" ParentId="7703" CreationDate="2013-06-20T16:00:39.103" Score="4" Body="&lt;p&gt;You should rewrite the equation dimensionless with &lt;a href=&quot;http://en.wikipedia.org/wiki/Dimensional_analysis&quot; rel=&quot;nofollow&quot;&gt;dimensional analysis&lt;/a&gt;. Proper step size should be evident then and can be compared more easily.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2013-06-20T16:00:39.103" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7732" PostTypeId="2" ParentId="7703" CreationDate="2013-06-20T19:33:45.483" Score="2" Body="&lt;p&gt;BOOM SOLVED!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry, I'm just super happy about finding the one line who that the honor of making me question reality for the past five days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inside the odefun there is a line which was suggested by a friend of mine, who has great experience in these things (he's in India doing shenanigans, otherwise I would have contacted him right away). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This line restores a portion of the particles back to the system which is extremely important in the imaginary time method. Otherwise the particles decay all away leaving an empty system. This line was the solution to my previous problem regarding choice of time step size and normalization.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function y_out = odefun(t,y_in,...variables...) &#xA;&#xA;    ...&#xA;    [ Nonlinear equations evaluated ]  &#xA;    ...&#xA;&#xA;&#xA;    y_out = y_out + 0.1*y_in*(N0-Ntemp) ;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I knew that the choice of 0.1 in the last line was never perfectly arbitrary but I had no idea it was so explicitly connected to the type of time steps used in the ODE. The results for nanoseconds showed that a great portion of the particles seem to have decayed away, so I changed 0.1 to 100 since that is the order between picoseconds and nanoseconds and voila! Exactly same results as for the case where I used picoseconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for all who gave this problem a thought. I'm just happy it's solved!&lt;/p&gt;&#xA;" OwnerDisplayName="user4388" LastActivityDate="2013-06-20T19:33:45.483" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7733" PostTypeId="2" ParentId="7102" CreationDate="2013-06-20T22:38:56.720" Score="1" Body="&lt;p&gt;For our computational biology C++ library (&lt;a href=&quot;http://www.cs.ox.ac.uk/chaste&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chaste&lt;/a&gt;) we use &lt;a href=&quot;http://cxxtest.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cxxtest.com/&lt;/a&gt;. This is fairly simple to use, works well, it provides a few macros for testing with &lt;code&gt;assert()&lt;/code&gt; style statements. For scientific computing these are generally simple direct comparisons with &lt;code&gt;TS_ASSERT_EQUALS(a,b)&lt;/code&gt; or numerical comparisons with &lt;code&gt;TS_ASSERT_DELTA(a,b,tolerance)&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extra macros can easily be written using these basic ones to compare your own vectors/matrices of choice too. Usefully, you can also check that your code throws appropriate warnings and error messages in given situations. You can browse some examples in the &lt;code&gt;test&lt;/code&gt; folders of our source code here: &lt;a href=&quot;https://chaste.cs.ox.ac.uk/trac/browser/trunk&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://chaste.cs.ox.ac.uk/trac/browser/trunk&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4575" LastEditorUserId="4575" LastEditDate="2017-08-04T09:52:32.480" LastActivityDate="2017-08-04T09:52:32.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7736" PostTypeId="1" AcceptedAnswerId="7760" CreationDate="2013-06-21T06:23:01.910" Score="17" ViewCount="1172" Body="&lt;p&gt;Quiet a lot of insight can be gained form experience, I was just wondering if anybody has seen something similar to this before. The plot shows the initial condition (green) for the advection-diffusion equation, then the solution at iteration 200 (blue) and then again at iteration 400 (red).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OLjBk.png&quot; alt=&quot;Oscillatory error&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution of the advection-diffusion equation blows up after a few iterations. The Péclet number $\mu\approx0.07$, and the CFL condition is satisfied, $C\approx 0.0015$, so the equations &lt;em&gt;should&lt;/em&gt; be stable. I anticipate I have a bug in the numerical code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background.&lt;/strong&gt; The discretisation is central difference for both advection and diffusion terms. I believe this is first order of advection and second order for diffusion. I have implemented this using a finite-volume approach (for the first time) in which the coefficients (velocity and diffusion coefficient) values at the cell faces is found by linear interpolation from the cell averages. I apply Robin boundary condition on the left and right surfaces and set the flux at the boundaries to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you debug your numerical code? Has anybody scene something like this before, where would be a good place to starting looking?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Here is my personal &quot;lab book&quot; style notes on implementing a finite volume method for the advection-diffusion equation, &lt;a href=&quot;http://danieljfarrell.github.io/FVM/&quot; rel=&quot;noreferrer&quot;&gt;http://danieljfarrell.github.io/FVM/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;The Python source code is available here, &lt;a href=&quot;http://github.com/danieljfarrell/FVM.git&quot; rel=&quot;noreferrer&quot;&gt;http://github.com/danieljfarrell/FVM.git&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution couldn't be more simple! I just made a sign error on the diffusion term. It's strange, I'm sure it I had not of posted this I would not have found the error! If someone wants to share tips on how they debug their numerical code I am still interested. I don't have a method, it's a bit hit and miss, I keep on trying stuff to get clues, but this process can take weeks (sometimes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof it works (&lt;strong&gt;N.B.&lt;/strong&gt; that with the finite-volume method all you need to do to calculate the area is a summation of width $\times$ height for all the cells, if you use an integration method such as numpy.trapz your results include the numerical error of the trapezium method). What is happening here? There is a constant velocity and diffusion coefficients &lt;em&gt;but&lt;/em&gt; with &lt;strong&gt;closed&lt;/strong&gt; boundary conditions. Therefore at the boundary we see the equilibrium between the velocity field pushing to the right and the diffusion push to the left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ouPGs.gif&quot; alt=&quot;Advection-diffusion equation with closed boundary conditions via finite-volume method.&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-08-07T12:46:44.387" LastActivityDate="2018-03-17T14:34:23.553" Title="How do you debug numerical code, what could be source of this oscillatory error?" Tags="|numerical-analysis|advection-diffusion|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7737" PostTypeId="2" ParentId="7726" CreationDate="2013-06-21T07:33:33.033" Score="1" Body="&lt;p&gt;Before answering, note that there are a couple of mistakes in the formulation of your question. First, your equation is not a conservation law due to the presence of the source term; it is somewhat common to refer to an equation like this as a balance law instead. Second, by definition the cell average is only a function of time, so you should write &#xA;$\tilde{u}(t)$ (the same goes for $\tilde{s}(t)$). Third, following from the second observation, the time derivative that operates on the cell average should be an ordinary time derivative.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now turning to your question as such. The definitions of $\tilde{u}$ and $\tilde{s}$ do not depend on what form you write your equation in. The only thing you need to be careful about is if the grid is a function of time (moving grid). If it moves, note that $x_{i+1/2}-x_{i-1/2}$ would remain inside the time derivative. In that case, you would also need to obey the so-called geometric conservation law, which relates the normal velocity at which the cell faces move to the rate of change in time of the cell volume (in this case the length).  &lt;/p&gt;&#xA;" OwnerUserId="1792" LastEditorUserId="1792" LastEditDate="2013-06-21T07:39:21.120" LastActivityDate="2013-06-21T07:39:21.120" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7738" PostTypeId="1" AcceptedAnswerId="7742" CreationDate="2013-06-21T09:19:55.573" Score="6" ViewCount="5654" Body="&lt;p&gt;I have two FMU developed with different simulation tools. I would like to connect them through some variables and run a co-simulation with an open source tool as OpenModelica. This co-simulation has to be rather tight as both FMU have discontinuities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that possible?. Could you provide a simple sample?&lt;/p&gt;&#xA;" OwnerUserId="4607" LastEditorUserId="4230" LastEditDate="2013-06-21T09:32:06.300" LastActivityDate="2013-06-25T12:36:22.440" Title="How to do co-simulation with two FMU models" Tags="|simulation|fmi|openmodelica|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7739" PostTypeId="5" CreationDate="2013-06-21T09:33:59.527" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-06-21T09:33:59.527" LastActivityDate="2013-06-21T09:33:59.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7740" PostTypeId="4" CreationDate="2013-06-21T09:33:59.527" Score="0" Body="FMI (Functional Mock-up Interface) is a set of standards for exchanging simulation models (&quot;FMI for Model-Exchange&quot;) and performing co-simulation (&quot;FMI for Co-Simulation&quot;). Exported models are called FMUs (&quot;Functional Mock-up Units&quot;)." OwnerUserId="4230" LastEditorUserId="4230" LastEditDate="2013-06-21T14:11:16.690" LastActivityDate="2013-06-21T14:11:16.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7742" PostTypeId="2" ParentId="7738" CreationDate="2013-06-21T09:42:45.613" Score="7" Body="&lt;h1&gt;Model Exchange vs. Co-Simulation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;This depends on how you export your FMUs: You can either use FMI for model-exchange or FMI for co-simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the model-exchange scenario, the FMU contains only the model and no solver. Therefore the solver of the importing simulator is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the co-simulation scenario, the FMU contains both the model and a solver. Here the importing simulator acts as the master of the co-simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases you will need a simulator that supports the import of the respective FMU type. OpenModelica has support for import of model-exchange FMUs since version 1.8.0. Hence, if you have two FMUs that you have exported using FMI for model-exchange, you can import them both into OpenModelica and simulate them using the OpenModelica solver. Note that strictly speaking, this isn't a real co-simulation since only one solver is involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use a separate solver for each FMU (i.e. perform a real co-simulation) then you should only export one of the models as an FMU using FMI for co-simulation. Import that FMU into the simulator of the other model. Note that OpenModelica 1.8.0 doesn't support FMI for co-simulation.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Model Exchange Example&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Here's a working example (tested in Dymola, I don't have OpenModelica installed here right now):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;File &lt;code&gt;TestFMU1.mo&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model TestFMU1&#xA;  parameter Real p = 1.0;&#xA;&#xA;  connector TestOutputConnector&#xA;    output Real value;&#xA;  end TestOutputConnector;&#xA;  TestOutputConnector c;&#xA;&#xA;equation &#xA;  c.value = p;&#xA;&#xA;end TestFMU1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;File &lt;code&gt;TestFMU2.mo&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model TestFMU2&#xA;  Real result;&#xA;&#xA;  connector TestInputConnector&#xA;    input Real value;&#xA;  end TestInputConnector;&#xA;  TestInputConnector c;&#xA;&#xA;equation &#xA;  result = c.value;&#xA;&#xA;end TestFMU2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Export both of these as FMUs. Then import them and combine them as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model TestConnection&#xA;  TestFMU1_fmu OutputFMU;&#xA;  TestFMU2_fmu InputFMU;&#xA;equation &#xA;  connect(OutputFMU.c_value, InputFMU.c_value);&#xA;end TestConnection;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;OpenModelica&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;It seems that OpenModelica currently does not support simulating two FMUs simultaneously. This is a &lt;a href=&quot;https://trac.openmodelica.org/OpenModelica/ticket/2131&quot;&gt;known bug&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4230" LastEditorUserId="4230" LastEditDate="2013-06-25T12:36:22.440" LastActivityDate="2013-06-25T12:36:22.440" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7743" PostTypeId="5" CreationDate="2013-06-21T09:43:36.417" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-06-21T09:43:36.417" LastActivityDate="2013-06-21T09:43:36.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7744" PostTypeId="4" CreationDate="2013-06-21T09:43:36.417" Score="0" Body="OpenModelica is an open-source implementation for the Modelica modeling language." OwnerUserId="4230" LastEditorUserId="22193" LastEditDate="2017-11-26T21:11:43.373" LastActivityDate="2017-11-26T21:11:43.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7745" PostTypeId="2" ParentId="5362" CreationDate="2013-06-21T11:09:04.610" Score="-1" Body="&lt;p&gt;Even if $i$ and $j$ are known, this is hard if the object function is nonconvex.&#xA;Therefore, if only the object function is convex or concave, you find exact solution of this problem.&#xA;if the object function is nonconvex, you will find approximate solution of this problem by DC Programming.&lt;/p&gt;&#xA;" OwnerUserId="4608" LastEditorUserId="4230" LastEditDate="2013-06-21T14:11:12.910" LastActivityDate="2013-06-21T14:11:12.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7747" PostTypeId="2" ParentId="7056" CreationDate="2013-06-21T11:33:06.027" Score="-2" Body="&lt;p&gt;Job Assignment Problem is a kind of Pure Binary Linear Problem.&#xA;Therefore, you can solve this problem by using BLP.&lt;/p&gt;&#xA;" OwnerUserId="4608" LastActivityDate="2013-06-21T11:33:06.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7751" PostTypeId="1" CreationDate="2013-06-21T13:07:39.910" Score="-2" ViewCount="711" Body="&lt;p&gt;In some linear program that is hard to find an initial feasible point, simplex method will fail in its phase one algorithm. But we can find such a feasible point through other method such as genetic algorithm: the problem is such solution may have many &quot;noise&quot; of small numbers for some coordinate. We want to use simplex to finish up such a solution that will eliminate those noise. But simplex method will not use the user provided x0. There is a proposed linear thansformation method to change the original problem so that 0 will be the original x0 and circumvent that restriction of simplex: &lt;a href=&quot;https://scicomp.stackexchange.com/a/7699/4584&quot;&gt;https://scicomp.stackexchange.com/a/7699/4584&lt;/a&gt;. This method need additional work on reformate the problem. Is there more easy solution, such as modify a few lines of the code of simplexphaseone so that x0 will be used directly?&lt;/p&gt;&#xA;" OwnerUserId="4584" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-07-05T14:10:53.340" Title="how to make linear program simplex method use user provided feasible initial point" Tags="|linear-programming|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7752" PostTypeId="1" CreationDate="2013-06-21T13:35:40.123" Score="2" ViewCount="689" Body="&lt;p&gt;I have written a short example to run the simplex algorithm with coin-or Clp, something quite simple like this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c++ prettyprint-override&quot;&gt;&lt;code&gt;ClpSimplex solver;&#xA;solver.readLp(file_in);        &#xA;solver.initialSolve();&#xA;if(solver.isProvenOptimal()) {&#xA;  int ncol = solver.getNumCols();&#xA;  const double *solution = solver.getColSolution();&#xA;  for(int i=0; i&amp;lt;ncol; i++) {&#xA;    printf(&quot;%f &quot;, solution[i]);&#xA;  }&#xA;  printf(&quot;\n&quot;);&#xA;}&#xA;if(solver.isAbandoned())&#xA;  printf(&quot;Numerical problems found\n&quot;);&#xA;if(solver.isProvenPrimalInfeasible())&#xA;  printf(&quot;Primal Infeasible\n&quot;);&#xA;if(solver.isProvenDualInfeasible())&#xA;  printf(&quot;Dual Infeasible\n&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now that I have the solution, I would like to do sensitivity analysis that is to say, change the objective function and/or the second member and then check whether the same basis is still optimum or not (of course not restarting from scratch!!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is coin-or Clp able to do that and how ?&#xA;If not, do you know another free LP solver which can make it ?&lt;/p&gt;&#xA;" OwnerUserId="4609" LastEditorUserId="1804" LastEditDate="2013-06-21T14:38:52.660" LastActivityDate="2013-06-22T07:30:32.923" Title="Sensitivity analysis of linear program with coin-or clp" Tags="|optimization|linear-solver|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7753" PostTypeId="2" ParentId="7638" CreationDate="2013-06-21T15:15:48.893" Score="2" Body="&lt;p&gt;In case it's not clear in the other answers, what number-of-iterations is good for is big-O arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not good for absolute speed, because that depends on the average-time-per-iteration, which may differ between methods by a large factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, there is a tendency to ignore the cost of calculating array indices, and that may well account for a large fraction of the CPU time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADDED: Also, as I've pointed out elsewhere, for every invocation of the method there is typically a setup cost.&#xA;Then if the matrices are typically not very large, that setup cost can itself account for a large fraction of CPU time (such that removing it would make a large difference in speed).&lt;/p&gt;&#xA;" OwnerUserId="1262" LastEditorUserId="1262" LastEditDate="2013-06-21T15:32:00.313" LastActivityDate="2013-06-21T15:32:00.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7754" PostTypeId="1" CreationDate="2013-06-21T16:18:59.107" Score="3" ViewCount="10004" Body="&lt;p&gt;I would like to know which is the algorithm Comsol uses in order to correct the time step it uses.&#xA;For example, when you try to solve an equation you've written in the PDE coefficient interface, and you set free time-step, Comsol automatically choose the time step in order to obtain the specified absolute error. Anyway, how does Comsol choose the correct time step?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In Navier Stokes equations the parameter to look to, is the CFL number, that has to be smaller than one. &lt;/li&gt;&#xA;&lt;li&gt;For generic transport problem the parameter is the Peclét number. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But what about all the other problems? There is a general way to find a toughtful maximum time step? &lt;/p&gt;&#xA;" OwnerUserId="3860" LastEditorUserId="4964" LastEditDate="2014-08-21T12:04:37.013" LastActivityDate="2014-08-21T12:04:37.013" Title="Time stepping in comsol multiphysics" Tags="|pde|discretization|comsol|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7755" PostTypeId="2" ParentId="7729" CreationDate="2013-06-21T19:36:11.397" Score="3" Body="&lt;p&gt;If you are interested in making more precise statements about where the time is being spent in Gemm, you might want to take a look at &lt;a href=&quot;https://github.com/poulson/Elemental/blob/master/examples/blas-like/Gemm.cpp&quot; rel=&quot;nofollow&quot;&gt;the following example driver which I just created&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance,&#xA;&lt;code&gt;mpirun -np 48 ./Gemm --m 2000 --n 2000 --k 2000 --nb 128&lt;/code&gt;&#xA;will run an instrumented version of your example code which reports the local MB/s and GFlop/s achieved in the &lt;code&gt;MPI_Allgather&lt;/code&gt; and &lt;code&gt;dgemm&lt;/code&gt; calls which take place at each stage of the algorithm. If the achieved local GFlops within the distributed Gemm are sufficiently lower than what that of the sequential Gemm, then you may want to look into your BLAS stack. If the AllGathers are performing terribly, then you may want to look into your MPI configuration. Roughly speaking, ScaLAPACK's &lt;code&gt;pdgemm&lt;/code&gt; works similarly to Elemental's, but it uses the equivalent of &lt;code&gt;MPI_Bcast&lt;/code&gt; where Elemental uses &lt;code&gt;MPI_Allgather&lt;/code&gt;. You might want to measure the achieved MB/s in &lt;code&gt;MPI_Bcast&lt;/code&gt; versus &lt;code&gt;MPI_Allgather&lt;/code&gt; on your machine for the same result size.&lt;/p&gt;&#xA;" OwnerUserId="67" LastActivityDate="2013-06-21T19:36:11.397" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="7756" PostTypeId="2" ParentId="7752" CreationDate="2013-06-22T07:30:32.923" Score="1" Body="&lt;p&gt;Any linear programming solver should be able to give you sensitivity information. The information you'll need for assessing the sensitivity to changes in the cost vector is the reduced cost vector and the simplex tableau. You should be able to get the reduced cost information from a call to &lt;code&gt;solver.getReducedCost()&lt;/code&gt; (for the reduced cost; uses nomenclature from your example program).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the simplex tableau should be available directly, it is not. You will need to use calls to &lt;code&gt;solver.getColumnStatus(j)&lt;/code&gt; (to determine whether or not $x_{j}$ is a basic variable). Once you know the basic variables -- they will have values of &lt;code&gt;ClpSimplex::basic&lt;/code&gt; -- you can calculate the simplex tableau explicitly from this information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should be able to get the call signatures from the &lt;a href=&quot;http://www.coin-or.org/Clp/userguide/clpuserguide.html&quot; rel=&quot;nofollow&quot;&gt;CLP user guide&lt;/a&gt;, and then you can use a linear programming textbook to calculate the sensitivity information explicitly in your code.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-06-22T07:30:32.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7757" PostTypeId="1" CreationDate="2013-06-22T13:46:40.113" Score="1" ViewCount="618" Body="&lt;p&gt;CHOLMOD is very fast, but I am just wondering what kinds of size A such that it can solve Ax=b. I have a A of 200,000 * 200,000, but it outputs errors&quot; problem too large&quot;. I am very appreciated if anyone can tell me which solver/software for numerical computing can solve such large size problem. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="5069" LastEditorUserId="4244" LastEditDate="2013-06-22T14:37:40.237" LastActivityDate="2013-06-22T20:12:20.100" Title="What kinds of size of matrix A that CHOLMOD can solve Ax=b" Tags="|parallel-computing|software|linear-solver|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7758" PostTypeId="2" ParentId="7757" CreationDate="2013-06-22T20:12:20.100" Score="1" Body="&lt;p&gt;The storage required by a sparse matrix depends (roughly linearly) on the number of nonzero elements in the matrix.  When you then attempt to compute a Cholesky factorization (or more generally an LU factorization) of your sparse matrix, the factors typically have substantially more nonzero elements than the original matrix had- this is referred to as &quot;fill-in.&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The amount of fill-in that occurs will depend on the number and placement of the nonzeros in the original matrix.  In many cases you can reduce fill-in by carefully ordering the rows/columns of your matrix before factoring it.  There are many algorithms for reducing the fill-in, but computing an optimal ordering is an NP-hard problem, so in practice these algorithms use heuristic approaches to get a good fill-in reducing ordering.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears that CHOLMOD couldn't factor your matrix because there wasn't sufficient storage available to handle the fill-in that occurred during the factorization process.  Did you reorder the rows/columns of your matrix before factoring it to reduce fill-in?  If not, then using a good ordering might be enough to make this problem solvable within the memory that you have available.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you try to solve your problem on a computer with more memory?  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-06-22T20:12:20.100" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7759" PostTypeId="2" ParentId="7751" CreationDate="2013-06-22T22:22:01.510" Score="1" Body="&lt;p&gt;I don't understand how you got to this question because the phase 1 of the simplex algorithm is designed specifically to find a feasible point of the original problem from which phase 2 can start. There is no need to use a genetic algorithm or similar to find a better starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But other than that, your question is very much specific to one particular implementation you appear to be using, but about which you tell us nothing. Clearly, if you implement the simplex method yourself, it is easy to just skip phase 1 and start phase 2 from somewhere. It's your code, you can implement it however you want. But if it's not a code you have written yourself you oftentimes do not have the luxury to just start it with phase 2. In that case, the easiest choice of course is to use it as is, i.e., including phase 1, since phase 1 is guaranteed to give you a starting point. If phase 1 fails, you need to explain to us&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what implementation you are using&lt;/li&gt;&#xA;&lt;li&gt;how phase 1 fails&lt;/li&gt;&#xA;&lt;li&gt;what the error message means that you get from your code.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It may be that your phase 1 fails because there is &lt;em&gt;no&lt;/em&gt; feasible point in your linear program and, consequently, no starting point for phase 2 can be found. Of course, in that case, that's a situation that you won't be able to correct by using a different method to choose a starting point for phase 2.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-22T22:22:01.510" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7760" PostTypeId="2" ParentId="7736" CreationDate="2013-06-22T23:21:44.683" Score="10" Body="&lt;p&gt;I have collected some of my experience on debugging numerical codes here: &lt;a href=&quot;https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions#debugging-dealii-applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;deal.II FAQ: debugging&lt;/a&gt;. I don't know if that would have helped you in this particular case, but it may in others.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="20688" LastEditDate="2018-03-17T14:34:23.553" LastActivityDate="2018-03-17T14:34:23.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7761" PostTypeId="2" ParentId="7729" CreationDate="2013-06-22T23:27:51.540" Score="2" Body="&lt;p&gt;My best take is that your problem is simply too small. If you use MPI, it's true that you get more processors working on your problem, but they have to exchange data rather slowly using the external network. With such a small matrix, you may well get the execution time down to very small numbers by using enough processors via MPI, but the network communication time will then dominate and be the time reported as the overall run time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find out more about what it really is, you need to do parameter studies: run your tests with a variety of matrices of size 100, 500, 1000, 2000, 4000, 1e4, ... and plot graphs of the run time. You will see that different implementations will behave differently as the problems become larger.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-22T23:27:51.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7762" PostTypeId="2" ParentId="7720" CreationDate="2013-06-22T23:32:07.150" Score="4" Body="&lt;p&gt;PETSc is an outstanding linear algebra library. It also has modules for nonlinear solvers and, to some degree, for meshes and finite elements, but the latter aren't frequently used as there are much more comprehensive finite element packages out there. So, if you're interested in FEM calculations, my recommendation would be to go with one of the other packages (which oftentimes use PETSc internally to do their linear algebra computations). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your goal is to do linear algebra, however, then PETSc is the right tool. As has been mentioned, the example programs help, but the manual is pretty decent to get an overview of what is available. You won't have to read the whole thing -- just stick with those parts you're particularly interested in.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-22T23:32:07.150" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7763" PostTypeId="2" ParentId="7751" CreationDate="2013-06-23T01:20:52.987" Score="1" Body="&lt;p&gt;If a user provides a basic feasible solution to simplex, I don't see why one couldn't start directly with Phase 2 simplex. (In fact, solvers that combine interior-point methods and simplex methods transition from the interior-point method to the simplex method by using a &quot;cross-over&quot; to find a basic feasible solution from the current interior-point iterate.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, Wolfgang is absolutely right: Phase I either finds a basic feasible solution to the problem, or it fails, in which case your linear program is infeasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could be that you mean that Phase I takes too long, in which case I'd recommend using an interior point method. In fact, if you find &lt;em&gt;any&lt;/em&gt; feasible solution, you can use it as a starting point to an interior point method, whereas simplex can &lt;em&gt;only&lt;/em&gt; use basic feasible solutions. I would not use a genetic algorithm to generate basic feasible solutions without using something like a cross-over procedure. I'm not even sure I would use a genetic algorithm to generate feasible solutions unless I had to; the technology for solving linear programs is so mature, even for problems containing millions to tens of millions of decision variables, that unless your problem were significantly larger than that, it would be difficult to see why you would want to use alternate methods to find feasible solutions because again: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the simplex method and interior-point methods each have robust initialization procedures that calculate valid initial points&lt;/li&gt;&#xA;&lt;li&gt;the simplex method requires a basic feasible solution&lt;/li&gt;&#xA;&lt;li&gt;interior-point methods merely require a feasible solution&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-06-23T01:20:52.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7765" PostTypeId="2" ParentId="7754" CreationDate="2013-06-23T06:05:38.350" Score="1" Body="&lt;p&gt;It's not possible to answer your question in general, since comsol likely uses different methods depending on the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general answer is that a partial differential equation is discretized in both space and time.  Just like you have to set up a volume grid, you have to break time into chunks.  The solution accuracy is a function of the time-step &quot;grid&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What variable time-step algorithms do is try to dynamically pick the time-step, to make it smaller when the solution is changing rapidly.  This can involve testing a bunch of different time-steps, and picking one that gives acceptable error with respect to the finest grid it's checked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The maximum time-step then controls how quickly the solution can be found when the solution looks linear in time.  It should be on order of the relevant time-scale of your system, since if your estimate is way off, the integrator won't be able to draw a smooth line between its attempted time-steps.  This may happen, in particular, if the thing oscillates - in which case you'd need less than the oscillation period.&lt;/p&gt;&#xA;" OwnerUserId="4594" LastEditorUserId="4594" LastEditDate="2013-06-23T06:13:26.140" LastActivityDate="2013-06-23T06:13:26.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7766" PostTypeId="1" AcceptedAnswerId="7769" CreationDate="2013-06-23T07:48:25.453" Score="5" ViewCount="1281" Body="&lt;p&gt;I'm comparing the performance of distributed gemm, using Scalapack over OpenBLAS, with threaded gemm, using OpenBLAS.  It seems quite hard for me to get scalapack to give better results than multithreaded BLAS.  Is there some tweaking/optimization/configuration I need to do, or something I need to do differently somehow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My hardware:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;4 blades in a chassis, which I will call 'nodes' henceforth&lt;/li&gt;&#xA;&lt;li&gt;connected by gigabit ethernet, not the fastest I know, maybe the cause of the poor results here?&lt;/li&gt;&#xA;&lt;li&gt;each node has 12 physical cores&lt;/li&gt;&#xA;&lt;li&gt;software: mpich2 2.3, Ubuntu 12.04 64-bit, gfortran&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm comparing three software configurations, for a GEMM of two square matrices, of dimension &lt;code&gt;n&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;multi-threaded BLAS, using OpenBLAS, with 12 threads.  1 single process, on 1 single node&lt;/li&gt;&#xA;&lt;li&gt;scalapack over OpenBLAS, 48 mpi processes, 1 thread per mpi process, on 4 nodes&lt;/li&gt;&#xA;&lt;li&gt;scalapack over OpenBLAS, 4 mpi processes, 12 threads per mpi process, on 4 nodes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is a graph of some results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QIpeh.png&quot; alt=&quot;Scalapack versus multithreaded BLAS&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This graph seems to show:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;using pure undistributed BLAS is actually super fast&lt;/li&gt;&#xA;&lt;li&gt;using distributed scalapack, with 1 mpi process per node gives the best performance, but is not dramatically faster than pure undistributed BLAS, even for really large matrices (the largest n in the graph is 30,000)&lt;/li&gt;&#xA;&lt;li&gt;using distributed scalapack with 1 thread per mpi process gives the worst results, always worse than simple undistributed BLAS&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how typical are these results?&lt;/li&gt;&#xA;&lt;li&gt;is there something I'm doing terribly wrong, or could do better, that might improve the results of the distributed scalapack results?&lt;/li&gt;&#xA;&lt;li&gt;is it normal that the 1 thread per mpi process configuration gives worse results than the other configurations tested?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is the test code used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For GEMM:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;cstdio&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;using namespace std;&#xA;&#xA;#include &quot;mycblas.h&quot;&#xA;#include &quot;utils/NanoTimer.h&quot;&#xA;#include &quot;utils/stringhelper.h&quot;&#xA;#include &quot;args.h&quot;&#xA;&#xA;extern &quot;C&quot; {&#xA;    void openblas_set_num_threads(int num_threads);&#xA;}&#xA;&#xA;int main( int argc, char *argv[] ) {&#xA;    int N, its, threads;&#xA;    Args( argc, argv ).arg(&quot;N&quot;, &amp;amp;N).arg(&quot;its&quot;, &amp;amp;its).arg(&quot;threads&quot;,&amp;amp;threads).go();&#xA;&#xA;    openblas_set_num_threads( threads );    &#xA;&#xA;    NanoTimer timer;&#xA;    double *A = (double*)malloc(sizeof(double)*N*N);&#xA;    double *B = (double*)malloc(sizeof(double)*N*N);&#xA;    int linsize = N * N;&#xA;    for( int i = 0; i &amp;lt; linsize; i++ ) {&#xA;        A[i] = i + 3;&#xA;        B[i] = i * 2;&#xA;    }&#xA;    int m = N;&#xA;    int n = N;&#xA;    int k = N;&#xA;    double alpha = 1;&#xA;    double beta = 0;&#xA;    double *C = (double*)malloc(sizeof(double)*N*N);&#xA;    timer.toc(&quot;setup input matrices&quot;);&#xA;    for( int it = 0; it &amp;lt; its; it++ ) {&#xA;        dgemm(false,false,N,N,N, 1, A, N, B, N, 0, C, N );&#xA;        timer.toc(&quot;it &quot; + toString(it) );&#xA;    }&#xA;    int sum = 0;&#xA;    for( int mult = 0; mult &amp;lt; log(N)/log(10); mult++ ) {&#xA;        int offset = pow(10,mult);&#xA;        sum += C[offset];&#xA;    }&#xA;    cout &amp;lt;&amp;lt; &quot;sum, to prevent short-cut optimization &quot; &amp;lt;&amp;lt; sum &amp;lt;&amp;lt; endl;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For scalapack:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;stdexcept&amp;gt;&#xA;#include &amp;lt;cstring&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;using namespace std;&#xA;&#xA;#include &quot;mpi.h&quot;&#xA;&#xA;#include &quot;utils/NanoTimer.h&quot;&#xA;#include &quot;utils/stringhelper.h&quot;&#xA;#include &quot;args.h&quot;&#xA;#include &quot;scalapack.h&quot;&#xA;&#xA;extern &quot;C&quot; {&#xA;    void openblas_set_num_threads(int num_threads);&#xA;}&#xA;&#xA;int getRootFactor( int n ) {&#xA;    for( int t = sqrt(n); t &amp;gt; 0; t-- ) {&#xA;        if( n % t == 0 ) {&#xA;            return t;&#xA;        }&#xA;    }&#xA;    return 1;&#xA;}&#xA;&#xA;// conventions:&#xA;// M_ by N_ matrix block-partitioned into MB_ by NB_ blocks, then&#xA;// distributed according to 2d block-cyclic scheme&#xA;&#xA;// based on http://acts.nersc.gov/scalapack/hands-on/exercise3/pspblasdriver.f.html&#xA;&#xA;int main( int argc, char *argv[] ) {&#xA;    int p, P;&#xA;    blacs_pinfo( &amp;amp;p, &amp;amp;P );&#xA;//    mpi_print( toString(p) + &quot; / &quot; + toString(P) );&#xA;&#xA;    int n;&#xA;    int numthreads;&#xA;    int its;&#xA;    int blocksize;&#xA;    Args( argc, argv ).arg(&quot;N&quot;, &amp;amp;n ).arg(&quot;num iterations&quot;, &amp;amp;its ).arg(&quot;numthreads&quot;, &amp;amp;numthreads ).arg(&quot;blocksize&quot;, &amp;amp;blocksize).go();&#xA;    openblas_set_num_threads( numthreads );&#xA;&#xA;    int nprows = getRootFactor(P);&#xA;    int npcols = P / nprows;&#xA;    if( p == 0 ) cout &amp;lt;&amp;lt; &quot;grid: &quot; &amp;lt;&amp;lt; nprows &amp;lt;&amp;lt; &quot; x &quot; &amp;lt;&amp;lt; npcols &amp;lt;&amp;lt; endl;&#xA;&#xA;    int system = blacs_get( -1, 0 );&#xA;    int grid = blacs_gridinit( system, true, nprows, npcols );&#xA;    if( p == 0 ) cout &amp;lt;&amp;lt; &quot;system context &quot; &amp;lt;&amp;lt; system &amp;lt;&amp;lt; &quot; grid context: &quot; &amp;lt;&amp;lt; grid &amp;lt;&amp;lt; endl;&#xA;&#xA;    int myrow, mycol;&#xA;    blacs_gridinfo( grid, nprows, npcols, &amp;amp;myrow, &amp;amp;mycol );&#xA;//    mpi_print(&quot;grid, me: &quot; + toString(myrow) + &quot;, &quot; + toString(mycol) );&#xA;&#xA;    if( myrow &amp;gt;= nprows || mycol &amp;gt;= npcols ) {&#xA;//        mpi_print(&quot;not needed, exiting&quot;);&#xA;        blacs_gridexit( grid );&#xA;        blacs_exit(0);&#xA;        exit(0);&#xA;    }&#xA;&#xA;    // A     B       C&#xA;    // m x k k x n = m x n&#xA;    // nb: blocksize&#xA;&#xA;    // nprows: process grid, number rows&#xA;    // npcols: process grid, number cols&#xA;    // myrow: process grid, our row&#xA;    // mycol: process grid, our col&#xA;    int m = n;&#xA;    int k = n;&#xA;//    int nb = min(n,128); // nb is column block size for A, and row blocks size for B&#xA;    int nb=min(n/P,128);&#xA;&#xA;    int mp = numroc( m, nb, myrow, 0, nprows ); // mp number rows A owned by this process&#xA;    int kp = numroc( k, nb, myrow, 0, nprows ); // kp number rows B owned by this process&#xA;    int kq = numroc( k, nb, mycol, 0, npcols ); // kq number cols A owned by this process&#xA;    int nq = numroc( n, nb, mycol, 0, npcols ); // nq number cols B owned by this process&#xA;//    mpi_print( &quot;mp &quot; + toString(mp) + &quot; kp &quot; + toString(kp) + &quot; kq &quot; + toString(kq) + &quot; nq &quot; + toString(nq) );&#xA;&#xA;    struct DESC desca, descb, descc;&#xA;    descinit( (&amp;amp;desca), m, k, nb, nb, 0, 0, grid, max(1, mp) );&#xA;    descinit( (&amp;amp;descb), k, n, nb, nb, 0, 0, grid, max(1, kp) );&#xA;    descinit( (&amp;amp;descc), m, n, nb, nb, 0, 0, grid, max(1, mp) );&#xA;//    mpi_print( &quot;desca.LLD_ &quot; + toString(desca.LLD_) + &quot; kq &quot; + toString(kq) );&#xA;    double *ipa = new double[desca.LLD_ * kq];&#xA;    double *ipb = new double[descb.LLD_ * nq];&#xA;    double *ipc = new double[descc.LLD_ * nq];&#xA;&#xA;    for( int i = 0; i &amp;lt; desca.LLD_ * kq; i++ ) {&#xA;        ipa[i] = p;&#xA;    }&#xA;    for( int i = 0; i &amp;lt; descb.LLD_ * nq; i++ ) {&#xA;        ipb[i] = p;&#xA;    }&#xA;&#xA;    if( p == 0 ) cout &amp;lt;&amp;lt; &quot;created matrices&quot; &amp;lt;&amp;lt; endl;&#xA;    double *work = new double[nb];&#xA;    if( n &amp;lt;=5 ) {&#xA;        pdlaprnt( n, n, ipa, 1, 1, &amp;amp;desca, 0, 0, &quot;A&quot;, 6, work );&#xA;        pdlaprnt( n, n, ipb, 1, 1, &amp;amp;descb, 0, 0, &quot;B&quot;, 6, work );&#xA;    }&#xA;&#xA;    NanoTimer timer;&#xA;    for( int it = 0; it &amp;lt; its; it++ ) {&#xA;        pdgemm( false, false, m, n, k, 1,&#xA;                      ipa, 1, 1, &amp;amp;desca, ipb, 1, 1, &amp;amp;descb,&#xA;                      1, ipc, 1, 1, &amp;amp;descc );&#xA;        MPI_Barrier( MPI_COMM_WORLD );&#xA;        if( p == 0 ) timer.toc(&quot;it &quot; + toString(it) + &quot; pdgemm&quot;);&#xA;    }&#xA;&#xA;    blacs_gridexit( grid );&#xA;    blacs_exit(0);&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Other files:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mycblas.h:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;extern &quot;C&quot; {&#xA;#define ADD_&#xA;    // blas:&#xA;    #include &amp;lt;cblas_f77.h&amp;gt;&#xA;&#xA;    // lapack:&#xA;    void dpotrf_( char *uplo, int *n, double *A, int *lda, int *info );&#xA;    void dtrtrs_( char *uplo, char *trans, char *diag, int *n, int *nrhs, double *A, int *lda,&#xA;         double *B, int *ldb, int *info );&#xA;    void dtrsm_( char *side, char *uplo, char *transA, char *diag, const int *m, const int *n, &#xA;                 const double *alpha, const double *A, const int *lda, double *B, const int *ldb );&#xA;}&#xA;&#xA;char boolToChar( bool value ) {&#xA;    return value ? 't' : 'n';&#xA;}&#xA;&#xA;// double, general matrix multiply&#xA;void dgemm( bool transa, bool transb, int m, int n, int k, double alpha, double *A,&#xA;    int lda, double *B, int ldb, double beta, double *C, int ldc ) {&#xA;    char transachar = boolToChar( transa );&#xA;    char transbchar = boolToChar( transb );&#xA;    dgemm_(&amp;amp;transachar,&amp;amp;transbchar,&amp;amp;m,&amp;amp;n,&amp;amp;k, &amp;amp;alpha, A, &amp;amp;lda, B, &amp;amp;ldb, &amp;amp;beta, C, &amp;amp;ldc );    &#xA;}&#xA;&#xA;// double, triangular, solve matrix&#xA;void dtrsm( bool XA, bool isUpper, bool transA, bool isUnitTriangular, int m, int n,&#xA;       double alpha, double *A, int lda, double *B, int ldb ) {&#xA;    char sideChar = XA ? 'R' : 'L';&#xA;    char isUpperChar = isUpper ? 'U' : 'L';&#xA;    char transAChar = transA ? 'T' : 'N';&#xA;    char isUnitTriangularChar = isUnitTriangular ? 'U' : 'N';&#xA;    dtrsm_( &amp;amp;sideChar, &amp;amp;isUpperChar, &amp;amp;transAChar, &amp;amp;isUnitTriangularChar, &amp;amp;m, &amp;amp;n,&#xA;        &amp;amp;alpha, A, &amp;amp;lda, B, &amp;amp;ldb );&#xA;}&#xA;&#xA;// double, symmetric positive definite, triangular factorization (=cholesky)&#xA;int dpotrf( bool isUpper, int N, double *A, int lda ) {&#xA;    int info;&#xA;    char uplo = isUpper ? 'U' : 'L';&#xA;    dpotrf_( &amp;amp;uplo, &amp;amp;N, A, &amp;amp;lda, &amp;amp;info );&#xA;    return info;&#xA;}&#xA;&#xA;// double, triangular, triangular solve&#xA;int dtrtrs( bool isUpper, bool transA, bool isUnitTriangular, int n, int nrhs, double *A, int lda,&#xA;             double *B, int ldb ) {&#xA;    int info;&#xA;    char isUpperChar = isUpper ? 'U' : 'L';&#xA;    char transChar = transA ? 'T' : 'N';&#xA;    char isUnitTriangularChar = isUnitTriangular ? 'U' : 'N';&#xA;    dtrtrs_( &amp;amp;isUpperChar, &amp;amp;transChar, &amp;amp;isUnitTriangularChar, &amp;amp;n, &amp;amp;nrhs, A, &amp;amp;lda, B, &amp;amp;ldb, &amp;amp;info );&#xA;    return info;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;scalapack.h:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma once&#xA;&#xA;extern &quot;C&quot; {&#xA;    struct DESC{&#xA;        int DTYPE_;&#xA;        int CTXT_;&#xA;        int M_;&#xA;        int N_;&#xA;        int MB_;&#xA;        int NB_;&#xA;        int RSRC_;&#xA;        int CSRC_;&#xA;        int LLD_;&#xA;    } ;&#xA;&#xA;    void blacs_pinfo_( int *iam, int *nprocs );&#xA;    void blacs_get_( int *icontxt, int *what, int *val );&#xA;    void blacs_gridinit_( int *icontxt, char *order, int *nprow, int *npcol );&#xA;    void blacs_gridinfo_( int *context, int *nprow, int *npcol, int *myrow, int *mycol );&#xA;    void blacs_gridexit_( int *context );&#xA;    void blacs_exit_( int *code );&#xA;&#xA;    int numroc_( int *n, int *nb, int *iproc, int *isrcproc, int *nprocs );&#xA;    void descinit_( struct DESC *desc, int *m, int *n, int *mb, int *nb, int *irsrc, int *icsrc, int *ictxt, int *lld, int *info );&#xA;    void pdlaprnt_( int *m, int *n, double *a, int *ia, int *ja, struct DESC *desca, int *irprnt,&#xA;        int *icprnt, const char *cmatnm, int *nout, double *work, int cmtnmlen );&#xA;    void pdgemm_( char *transa, char *transb, int *m, int *n, int *k, double *alpha,&#xA;         double *a, int *ia, int *ja, struct DESC *desca, double *b, int *ib, int *jb,&#xA;        struct DESC *descb, double *beta, double *c, int *ic, int *jc, struct DESC *descc );&#xA;}&#xA;&#xA;void blacs_pinfo( int *p, int *P ) {&#xA;    blacs_pinfo_( p, P );&#xA;}&#xA;&#xA;int blacs_get( int icontxt, int what ) {&#xA;    int val;&#xA;    blacs_get_( &amp;amp;icontxt, &amp;amp;what, &amp;amp;val );&#xA;    return val;&#xA;}&#xA;&#xA;int blacs_gridinit( int icontxt, bool isColumnMajor, int nprow, int npcol ) {&#xA;    int newcontext = icontxt;&#xA;    char order = isColumnMajor ? 'C' : 'R';&#xA;    blacs_gridinit_( &amp;amp;newcontext, &amp;amp;order, &amp;amp;nprow, &amp;amp;npcol );&#xA;    return newcontext;&#xA;}&#xA;&#xA;void blacs_gridinfo( int context, int nprow, int npcol, int *myrow, int *mycol ) {&#xA;    blacs_gridinfo_( &amp;amp;context, &amp;amp;nprow, &amp;amp;npcol, myrow, mycol );&#xA;}&#xA;&#xA;void blacs_gridexit( int context ) {&#xA;    blacs_gridexit_( &amp;amp;context );&#xA;}&#xA;&#xA;void blacs_exit( int code ) {&#xA;    blacs_exit_( &amp;amp;code );&#xA;}&#xA;&#xA;int numroc( int n, int nb, int iproc, int isrcproc, int nprocs ) {&#xA;    return numroc_( &amp;amp;n, &amp;amp;nb, &amp;amp;iproc, &amp;amp;isrcproc, &amp;amp;nprocs );&#xA;}&#xA;&#xA;void descinit( struct DESC *desc, int m, int n, int mb, int nb, int irsrc, int icsrc, int ictxt, int lld ) {&#xA;    int info;&#xA;    descinit_( desc, &amp;amp;m, &amp;amp;n, &amp;amp;mb, &amp;amp;nb, &amp;amp;irsrc, &amp;amp;icsrc, &amp;amp;ictxt, &amp;amp;lld, &amp;amp;info );&#xA;    if( info != 0 ) {&#xA;        throw runtime_error( &quot;non zero info: &quot; + toString( info ) );&#xA;    }&#xA;//    return info;&#xA;}&#xA;&#xA;void pdlaprnt( int m, int n, double *A, int ia, int ja, struct DESC *desc, int irprnt,&#xA;    int icprnt, const char *cmatnm, int nout, double *work ) {&#xA;    int cmatnmlen = strlen(cmatnm);&#xA;    pdlaprnt_( &amp;amp;m, &amp;amp;n, A, &amp;amp;ia, &amp;amp;ja, desc, &amp;amp;irprnt, &amp;amp;icprnt, cmatnm, &amp;amp;nout, work, cmatnmlen );&#xA;}&#xA;&#xA;void pdgemm( bool isTransA, bool isTransB, int m, int n, int k, double alpha,&#xA;     double *a, int ia, int ja, struct DESC *desca, double *b, int ib, int jb,&#xA;    struct DESC *descb, double beta, double *c, int ic, int jc, struct DESC *descc ) {&#xA;    char transa = isTransA ? 'T' : 'N';&#xA;    char transb = isTransB ? 'T' : 'N';&#xA;    pdgemm_( &amp;amp;transa, &amp;amp;transb, &amp;amp;m, &amp;amp;n, &amp;amp;k, &amp;amp;alpha, a, &amp;amp;ia, &amp;amp;ja, desca, b, &amp;amp;ib, &amp;amp;jb,&#xA;        descb, &amp;amp;beta, c, &amp;amp;ic, &amp;amp;jc, descc );&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;utils/NanoTimer.h:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma once&#xA;#include &amp;lt;cstdio&amp;gt;&#xA;#include &amp;lt;cstdlib&amp;gt;&#xA;#include &amp;lt;ctime&amp;gt;&#xA;#include &amp;lt;sys/time.h&amp;gt;&#xA;&#xA;class NanoTimer {&#xA;public:&#xA;   struct timespec start;&#xA;&#xA;   NanoTimer() {&#xA;      clock_gettime(CLOCK_MONOTONIC,  &amp;amp;start);&#xA;&#xA;   }&#xA;   double elapsedSeconds() {&#xA;      struct timespec now;&#xA;      clock_gettime(CLOCK_MONOTONIC,  &amp;amp;now);&#xA;      double time = (now.tv_sec - start.tv_sec) + (double) (now.tv_nsec - start.tv_nsec) * 1e-9;&#xA;      start = now;&#xA;      return time;&#xA;   }&#xA;    void toc(string label) {&#xA;        double elapsed = elapsedSeconds();&#xA;        cout &amp;lt;&amp;lt; label &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; elapsed &amp;lt;&amp;lt; &quot;s&quot; &amp;lt;&amp;lt; endl;        &#xA;    }&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;utils/stringhelper.h:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma once&#xA;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;#include &amp;lt;sstream&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cstdlib&amp;gt;&#xA;&#xA;template&amp;lt;typename T&amp;gt;&#xA;std::string toString(T val ) { // not terribly efficient, but works...&#xA;   std::ostringstream myostringstream;&#xA;   myostringstream &amp;lt;&amp;lt; val;&#xA;   return myostringstream.str();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;args.h:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma once&#xA;&#xA;// usage:&#xA;// int N, its;&#xA;// arg( &quot;N&quot;, &amp;amp;N );&#xA;// arg( &quot;its&quot;, &amp;amp;its );&#xA;// args( argc, argv );&#xA;&#xA;class Arg {&#xA;public:&#xA;    virtual void assign( const char *argvalue ) = 0;&#xA;    virtual void print( ostream &amp;amp;os ) const = 0;&#xA;};&#xA;ostream &amp;amp;operator&amp;lt;&amp;lt;( ostream &amp;amp;os, const Arg &amp;amp;arg ) {&#xA;    arg.print( os );&#xA;    return os;&#xA;}&#xA;&#xA;class IntArg : public Arg {&#xA;public:&#xA;    int *argptr;&#xA;    IntArg( int *_argptr ) : argptr(_argptr ) {&#xA;    }&#xA;    void assign( const char *argvalue ) {&#xA;        *argptr = atoi( argvalue );&#xA;    }&#xA;    void print( ostream &amp;amp;os ) const {&#xA;        os &amp;lt;&amp;lt; (*argptr );&#xA;    }&#xA;};&#xA;&#xA;vector&amp;lt;string&amp;gt; argnames;&#xA;vector&amp;lt;Arg *&amp;gt; argptrs;&#xA;&#xA;void arg_usage(string cmd) {&#xA;    cout &amp;lt;&amp;lt; &quot;Usage: &quot; &amp;lt;&amp;lt; cmd;&#xA;    for( int i = 0; i &amp;lt; argnames.size(); i++ ) {&#xA;        cout &amp;lt;&amp;lt; &quot; [&quot; &amp;lt;&amp;lt; argnames[i] &amp;lt;&amp;lt; &quot;]&quot;;&#xA;    }&#xA;    cout &amp;lt;&amp;lt; endl;&#xA;    exit(1);&#xA;}&#xA;&#xA;void arg( string name, int *p_value ) {&#xA;    argnames.push_back(name);&#xA;    argptrs.push_back( new IntArg( p_value ) );&#xA;}&#xA;&#xA;void args( int argc, char *argv[] ) {&#xA;    if( argc - 1 != argnames.size() ) {&#xA;        arg_usage(argv[0]);&#xA;    }&#xA;    for( int i = 0; i &amp;lt; argnames.size(); i++ ) {&#xA;        argptrs[i]-&amp;gt;assign( argv[i+1] );&#xA;        cout &amp;lt;&amp;lt; argnames[i] &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; (*argptrs[i]) &amp;lt;&amp;lt; endl;&#xA;    }&#xA;}&#xA;&#xA;class Args {&#xA;public:&#xA;    int argc;&#xA;    char **argv;&#xA;    Args( int _argc, char *_argv[] ) : argc(_argc), argv(_argv) {&#xA;    }&#xA;    void go() {&#xA;        args( argc, argv );&#xA;    }&#xA;    Args &amp;amp;_( string name, int *pvalue ) {&#xA;        ::arg( name, pvalue );&#xA;        return *this;&#xA;    }&#xA;    Args &amp;amp;arg( string name, int *pvalue ) {&#xA;        ::arg( name, pvalue );&#xA;        return *this;&#xA;    }&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: in answer to Jeff's question about my Elemental code, here is my Elemental code for GEMM:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;mpi.h&quot;&#xA;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cstdio&amp;gt;&#xA;#include &amp;lt;cstdlib&amp;gt;&#xA;#include &amp;lt;ctime&amp;gt;&#xA;#include &amp;lt;sys/time.h&amp;gt;&#xA;using namespace std;&#xA;&#xA;#include &quot;elemental.hpp&quot;&#xA;using namespace elem;&#xA;&#xA;extern &quot;C&quot; {&#xA;    void openblas_set_num_threads(int num_threads);&#xA;}&#xA;&#xA;class NanoTimer {&#xA;public:&#xA;   struct timespec start;&#xA;&#xA;   NanoTimer() {&#xA;      clock_gettime(CLOCK_MONOTONIC,  &amp;amp;start);&#xA;&#xA;   }&#xA;   double elapsedSeconds() {&#xA;      struct timespec now;&#xA;      clock_gettime(CLOCK_MONOTONIC,  &amp;amp;now);&#xA;      double time = (now.tv_sec - start.tv_sec) + (double) (now.tv_nsec - start.tv_nsec) * 1e-9;&#xA;      start = now;&#xA;      return time;&#xA;   }&#xA;    void toc(string label) {&#xA;        double elapsed = elapsedSeconds();&#xA;        cout &amp;lt;&amp;lt; label &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; elapsed &amp;lt;&amp;lt; endl;        &#xA;    }&#xA;};&#xA;&#xA;int sum = 0;&#xA;void readMatrix( Matrix&amp;lt;double&amp;gt; &amp;amp;A ) {&#xA;    for( int i = 1; i &amp;lt; A.Width(); i *= 10 ) {&#xA;        sum += A.Get(i,i);&#xA;    }&#xA;}&#xA;void readMatrix( DistMatrix&amp;lt;double,CIRC,CIRC&amp;gt; &amp;amp;A ) {&#xA;    for( int i = 1; i &amp;lt; A.Width(); i *= 10 ) {&#xA;        sum += A.Get(i,i);&#xA;    }&#xA;}&#xA;&#xA;int main( int argc, char *argv[] ) {&#xA;    elem::Initialize( argc, argv );&#xA;    int p = mpi::CommRank(mpi::COMM_WORLD);&#xA;    int P = mpi::CommSize(mpi::COMM_WORLD);&#xA;&#xA;    if( argc &amp;lt; 3 ) {&#xA;        if( p == 0 ) cout &amp;lt;&amp;lt; &quot;Usage: &quot; &amp;lt;&amp;lt; argv[0] &amp;lt;&amp;lt; &quot; [N] [multithreaded: 1|0]&quot; &amp;lt;&amp;lt; endl;&#xA;        return -1;&#xA;    }&#xA;    int n = atoi( argv[1] );&#xA;    int multithreaded = atoi( argv[2] );&#xA;&#xA;    Matrix&amp;lt;double&amp;gt; A(n,n);&#xA;    Matrix&amp;lt;double&amp;gt; B(n,n);&#xA;    for( int i = 0; i &amp;lt; n; i++ ) {&#xA;        for( int j = 0; j &amp;lt; n; j++ ) {&#xA;            A.Set(i,j, i*j + 2 );&#xA;            B.Set(i,j, i*j + 4 );&#xA;        }&#xA;    }&#xA;&#xA;    Matrix&amp;lt;double&amp;gt; C(n,n);&#xA;    MPI_Barrier(MPI_COMM_WORLD);&#xA;    NanoTimer timer;&#xA;    if( p == 0 ) {&#xA;        Gemm&amp;lt;double&amp;gt;(NORMAL, NORMAL, 1, A, B, 0, C );  &#xA;        // read some values, to prevent being optimized out :-P&#xA;        timer.toc(&quot;blas multithreaded&quot;);&#xA;        readMatrix(C);&#xA;&#xA;        if( !multithreaded ) {&#xA;            openblas_set_num_threads(1);&#xA;            Gemm&amp;lt;double&amp;gt;(NORMAL, NORMAL, 1, A, B, 0, C );    &#xA;            timer.toc(&quot;blas singlethreaded&quot;);&#xA;            readMatrix(C);&#xA;        }&#xA;    }&#xA;    if( !multithreaded ) {&#xA;        openblas_set_num_threads(1);&#xA;    }&#xA;&#xA;    Grid g;&#xA;    DistMatrix&amp;lt;double,CIRC,CIRC&amp;gt; Aroot(n,n,g);&#xA;    DistMatrix&amp;lt;double,CIRC,CIRC&amp;gt; Broot(n,n,g);&#xA;    DistMatrix&amp;lt;double,CIRC,CIRC&amp;gt; Croot(n,n,g);&#xA;    Aroot.SetRoot(0);&#xA;    Broot.SetRoot(0);&#xA;    Croot.SetRoot(0);&#xA;&#xA;    if( p == 0 ) {&#xA;        for( int i = 0; i &amp;lt; n; i++ ) {&#xA;            for( int j = 0; j &amp;lt; n; j++ ) {&#xA;                Aroot.Set(i,j,i*j+2);&#xA;                Broot.Set(i,j,i*j+2);&#xA;            }&#xA;        }&#xA;    }&#xA;    if( p == 0 ) timer.toc(&quot;populate root node&quot;);&#xA;&#xA;//    DistMatrix&amp;lt;double,MC,STAR&amp;gt; Adist( Aroot );&#xA;//    DistMatrix&amp;lt;double,STAR,MR&amp;gt; Bdist( Broot );&#xA;//    DistMatrix&amp;lt;double,MC,MR&amp;gt; Cdist(n,n,g);&#xA;    DistMatrix&amp;lt;double&amp;gt; Adist( Aroot );&#xA;    DistMatrix&amp;lt;double&amp;gt; Bdist( Broot );&#xA;    DistMatrix&amp;lt;double&amp;gt; Cdist(n,n,g);&#xA;    MPI_Barrier(MPI_COMM_WORLD);&#xA;    if( p == 0 ) timer.toc(&quot;distributed to slaves&quot;);&#xA;&#xA;    Gemm&amp;lt;double&amp;gt;(NORMAL, NORMAL, 1, Adist, Bdist, 0, Cdist );    &#xA;    MPI_Barrier(MPI_COMM_WORLD);&#xA;    if( p == 0 ) timer.toc(&quot;distmatrix gemm&quot;);&#xA;&#xA;    Croot = Cdist;&#xA;    MPI_Barrier(MPI_COMM_WORLD);&#xA;    if( p == 0 ) timer.toc(&quot;gathered to master&quot;);&#xA;    readMatrix(Croot);&#xA;&#xA;    if( p == 0 ) cout &amp;lt;&amp;lt; &quot;sum, to prevent optimization out: &quot; &amp;lt;&amp;lt; sum &amp;lt;&amp;lt; endl;&#xA;&#xA;    elem::Finalize();&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3613" LastEditorUserId="3613" LastEditDate="2013-07-07T02:10:38.620" LastActivityDate="2013-07-07T02:10:38.620" Title="Performance optimization or tuning possible for Scalapack Gemm?" Tags="|linear-algebra|matrix|parallel-computing|mpi|blas|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7767" PostTypeId="2" ParentId="7766" CreationDate="2013-06-23T12:05:13.793" Score="2" Body="&lt;p&gt;These results seem normal. Your network is terrible for distributed BLAS. I can't say about the mpich2 shared memory implementation, but it might also not be up to snuff and limiting your 1 thread per process/12 processes per node results. You might try OpenMPI or MVAPICH2 (though I've never run the latter with just Ethernet). &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-23T12:05:13.793" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7768" PostTypeId="1" CreationDate="2013-06-23T12:05:45.070" Score="2" ViewCount="185" Body="&lt;p&gt;I want to implement thin spline interpolation of scattered elevation data $ \{z_i(x_i,y_i)\}_{i=1..n} $ in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems fairly simple using Radial Basis Functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ z(x,y) = p(x,y) + \sum_i l_i\phi(r)$$&#xA;where p(x,y) is a polynomial and $\ \phi= r^2\ln(r) $ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Requiring the interpolated surface to go exactly trough the n datapoints + ortogonality of polynomials leads to a matrix equation. This is then solved to find the coefficients $l_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I want the implementation to take into account &quot;barriers&quot;:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Mbr5S.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see in this figure: points on one side of the &quot;barrier&quot; should not be influenced by points on the other side of the &quot;barrier&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example consider this simple form of interpolation:&#xA;the elevation in a grid node is found as an average over the elevation of points within a circle of radius R. Now consider a grid node very close to the &quot;barrier&quot;. This would be influenced by points on both side of the &quot;barrier&quot; and as a result the interpolated elevation would be somewhere in between. The resulting surface would not show a discontinuity (&quot;cliff&quot;) instead it would show a &quot;hill&quot;. In this case things could be remedied by only using points from the same side of the &quot;barrier&quot; as the grid node when doing the averaging. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB! The &quot;barriers&quot; do not necessarily split the data into two sets like in the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I do this?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3622" LastEditorUserId="3622" LastEditDate="2013-06-24T07:38:36.307" LastActivityDate="2013-06-24T07:38:36.307" Title="How do I implement thin plate splines with barriers?" Tags="|linear-algebra|interpolation|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7769" PostTypeId="2" ParentId="7766" CreationDate="2013-06-23T12:46:55.413" Score="5" Body="&lt;p&gt;There's nothing surprising about these results.  Matrix multiplication is well known to be communication intensive, and you've got a relatively slow communications network between your four nodes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using MPI between two processes on the same node is certainly faster than using MPI between processes on different nodes because you don't have the bandwidth limitations of the ethernet.  However, you still have to pay a price in calling the MPI library.  In comparison, the multithreaded BLAS gets very fast communication between threads because the memory is fast and because you don't pay the overhead of calling the MPI library routines and using the OS to pass messages between threads as you would with MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, for large $n$, you should see the 4 nodes, 1 process per node, 12 threads per process method coming in at four times as fast as 12 threads on a single node.  The two curves are diverging towards that ratio of 4, but not very fast (you're 3 times as fast at n=30000)- this is an indication that the bandwidth of the gigabit ethernet is really holding you back.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-06-23T12:46:55.413" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7770" PostTypeId="1" CreationDate="2013-06-23T14:18:25.313" Score="0" ViewCount="328" Body="&lt;p&gt;Changing &lt;code&gt;cholmod_*&lt;/code&gt; to &lt;code&gt;cholmod_l_*&lt;/code&gt; results in the following error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;sparse:error: integer and real must match the routines&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does this error message mean? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This message shows after&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cholmod_l_start(&amp;amp;c); &#xA;cholmod_l_print_sparse(A,&quot;A&quot;,&amp;amp;c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then the following steps &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;L = cholmod_l_analyze(A,&amp;amp;c); &#xA;cholmod_l_factorzie(A,L,&amp;amp;c); &#xA;x=cholmod_l_solve(L,&amp;amp;c); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;cannot be executed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Motivation: the size of my matrix is large, so using &lt;code&gt;cholmod_*&lt;/code&gt; will result in an error: &quot;&lt;em&gt;problem too large&lt;/em&gt;&quot;&lt;/p&gt;&#xA;" OwnerUserId="5069" LastEditorUserId="20688" LastEditDate="2021-12-25T08:41:24.893" LastActivityDate="2021-12-25T08:41:24.893" Title="A question on CHOLMOD: long int vs int, still failing after change to long int" Tags="|linear-solver|matrix-factorization|" AnswerCount="1" CommentCount="3" ClosedDate="2018-04-15T01:10:43.270" ContentLicense="CC BY-SA 3.0" />
  <row Id="7771" PostTypeId="1" CreationDate="2013-06-23T16:14:19.433" Score="5" ViewCount="183" Body="&lt;p&gt;I am trying to solve numerically next PDE system:&#xA;$$\frac{\partial c}{\partial t}=\epsilon\frac{\partial}{\partial x}(\frac{\partial c}{\partial x}+\rho\frac{\partial \varphi}{\partial x}+\frac{vc}{1-vc}\frac{\partial c}{\partial x})$$&#xA;$$\frac{\partial \rho}{\partial t}=\epsilon\frac{\partial}{\partial x}(\frac{\partial \rho}{\partial x}+c\frac{\partial \varphi}{\partial x}+\frac{v\rho}{1-vc}\frac{\partial c}{\partial x})$$&#xA;$$-\epsilon^2\frac{\partial \varphi^2}{\partial x^2}=\rho$$&#xA;with completely blocking boundary conditions at x= ±1;&#xA;$v$ and $\epsilon$ are constants.&#xA; However, at some point of calculation the oscillations near the boundaries occur due to the too big gradients near the boundaries. I have found some information that there are several techniques for handling numerical instabilities. (Petrov-Galerkin (SUPG) or  Galerkin least-squares (GLS) method). Unfortunately all of them usually described for the case of simple Convection-Diusion equation. Can anybody help me how to adjust some of those methods (or maybe another method) for my case? Also I need to add, that problem is not in the fact that $(1-vc)$ may tended to zero in  some points.&lt;/p&gt;&#xA;" OwnerUserId="4619" LastEditorUserId="4244" LastEditDate="2013-06-24T10:41:13.130" LastActivityDate="2013-07-24T11:38:51.200" Title="Stabilization of solution to one-dimensional system of PDE" Tags="|simulation|stability|parabolic-pde|numerical-analysis|multiphysics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7772" PostTypeId="2" ParentId="7720" CreationDate="2013-06-23T17:28:38.463" Score="1" Body="&lt;p&gt;Start with src/ksp/ksp/examples/tutorials/ex2.c or ex2f.F and work your way upwards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc is very easy to use. You just have to worry about local element level calculations (matrices, rhs etc.) and PETSc takes care of everything else (e.g. parallel assembly and solving).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Developing a parallel high performance FE code quickly is very easy with PETSc even if you don't have a strong background in scientific computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a slight learning curve and a PETSc way of doing things but you should be productive fairly quickly.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2013-06-23T17:28:38.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7773" PostTypeId="2" ParentId="7771" CreationDate="2013-06-23T20:18:21.840" Score="2" Body="&lt;p&gt;If you are using a finite element method to solve this problem, why not put a few grid points in the boundary layer? 1-D implementations are usually pretty trivial to create that support meshes of varying cell size.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-06-23T20:18:21.840" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7774" PostTypeId="5" CreationDate="2013-06-24T04:29:48.423" Score="0" Body="&lt;p&gt;$$&#xA;u_t = (-au + du_x)_x + s&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-06-24T04:37:40.463" LastActivityDate="2013-06-24T04:37:40.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7775" PostTypeId="4" CreationDate="2013-06-24T04:29:48.423" Score="0" Body="Questions related to solving the advection-diffusion equation using numerical methods, including derivation and implementation of boundary conditions." OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-06-24T05:33:46.290" LastActivityDate="2013-06-24T05:33:46.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7776" PostTypeId="1" CreationDate="2013-06-24T05:14:42.140" Score="2" ViewCount="2356" Body="&lt;p&gt;Given a real symmetric matrix $M$, ostensibly of &quot;low rank&quot;, efficiently find an expression $M = \sum \alpha_i u_i u_i^T$ using the number of terms rank($M$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 2011 StackOverflow Question &lt;a href=&quot;https://stackoverflow.com/questions/8636518/dense-cholesky-update-in-python&quot;&gt;Dense Cholesky Update in Python&lt;/a&gt; asked about doing low rank updates to Cholesky decompositions, but was answered largely with ways to do rank 1 updates in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naturally a rank one symmetric term would have the form $\alpha u u^T$, and if we have a rank two symmetric update of the form $uv^T + vu^T$, then we can rewrite it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ uv^T + vu^T = \frac{1}{2} ((u+v)(u+v)^T - (u-v)(u-v)^T) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but this does not seem to lead to an efficient general method for similarly expanding an arbitrary symmetric real matrix into as many such terms as the rank requires.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Existence of such an expresssion is guaranteed by eigenbasis expansion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ M = \sum \alpha_i u_i u_i^T $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $u_i$ are an orthonormal basis of eigenvectors with corresponding eigenvalues $\alpha_i$ (and the limitation to as many terms as the rank requires is met by ignoring zero eigenvalues).  But a full eigenbasis expansion would call for a disproportionate amount of work relative to the Cholesky updates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could easily expand a rank $k$ matrix as a sum of $k$ not necessarily symmetric rank one terms and use the symmetry of the sum to rewrite it as $2k$ symmetric rank one terms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum u_i v_i^T = \frac{1}{2} \sum (u_i v_i^T + v_i u_i^T) = \frac{1}{4} \sum ((u_i+v_i)(u_i+v_i)^T - (u_i-v_i)(u_i-v_i)^T) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this uses twice the number of rank one updates as ought to be necessary.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-06-28T00:20:21.220" Title="Writing Real Symmetric Matrices as Linear Combination of Rank One Symmetric Terms $uu^T$" Tags="|matrix|rank|update|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7777" PostTypeId="1" AcceptedAnswerId="7856" CreationDate="2013-06-24T10:06:28.100" Score="6" ViewCount="2230" Body="&lt;p&gt;I would like to apply Dirichlet conditions to the advection-diffusion equation using the &lt;em&gt;finite-volume method&lt;/em&gt;. This answer, &quot;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/7650/how-should-boundary-conditions-be-applied-when-using-finite-volume-method&quot;&gt;How should boundary conditions be applied when using finite-volume method?&lt;/a&gt;&quot;&#xA;emphases the benefit of staying with integral form of the equations for as long as possible. This works fantastically for &lt;em&gt;Robin&lt;/em&gt; boundary conditions because ghost cells nor interpolation is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can Dirichlet conditions be applied to the integral form, or only the discretised equation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the advection-diffusion equation in flux form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = -(\cal{F})_x + s&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{F}=au - du_x&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After applying the finite volume method this becomes (in semi-discrete form),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;w_1^{\prime} = -\frac{\mathcal{F_{3/2}}}{h_1} + \frac{\mathcal{F_{1/2}}}{h_1} + \bar{s}_1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can Dirchlet conditions be applied to the &lt;strong&gt;flux&lt;/strong&gt; term at the left hand side interface, $\mathcal{F_{1/2}}$, this would imply that we only know the advection part of the flux, i.e. the $au$ part where $u$ is known at the boundary.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-07-01T05:33:44.327" Title="Finite-volume method: can Dirichlet boundary conditions be applied to the integral form?" Tags="|boundary-conditions|finite-volume|advection-diffusion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7778" PostTypeId="1" AcceptedAnswerId="7787" CreationDate="2013-06-24T10:39:04.823" Score="11" ViewCount="1519" Body="&lt;p&gt;I have to write a finite volume code for Magnetohydrodynamics (MHD). I have written numerical code before but not at this scale. I just wanted to ask which will be a good choice, using a data structure (object orientated approach) with classes or just using multiple arrays for different properties, in terms of speed, scalability etc. I plan to write the code in python, and use fortran for numerically intensive part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example for class in python would be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Cell:&#xA;   def __init__(self, x, y, z, U):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Arrays can be simply defined as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x[nx][ny][nz]&#xA;y[nx][ny][nz]&#xA;z[nx][ny][nz]&#xA;U[nx][ny][nz]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;" OwnerUserId="3640" LastEditorUserId="1804" LastEditDate="2013-06-24T11:08:24.777" LastActivityDate="2013-06-24T20:08:56.757" Title="Data structures for finite volume code: Arrays vs Classes" Tags="|python|finite-volume|data-structures|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7779" PostTypeId="2" ParentId="7778" CreationDate="2013-06-24T10:59:23.607" Score="10" Body="&lt;p&gt;I was pondering this a few days ago (also in Python). Personally I don't think that object oriented programming is always a good fit for numerical programming. You can get distracted with designing the classes rather than just solving the equations. I prefer to stay with simple functions, and with &lt;a href=&quot;http://www.numpy.org/&quot;&gt;numpy&lt;/a&gt; you can have your equations vectorised so the number of lines you need is very few. Numpy is pretty fast because the actual computations are done with a C (or FORTRAN?) back end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would recommend you do,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write a Python script that solves the &lt;strong&gt;simplest possible&lt;/strong&gt; version of your problem using a functional approach with numpy. For example, have everything in arbitrary unit and try 1D (or 2D) only. It is perfectly fine at this stage if the code is messy. The important thing is that you are moving forward with your project.&lt;/li&gt;&#xA;&lt;li&gt;Once you have something that works. Identify where the code is verbose and refractor. At this stage you can play around with different ideas on how to simplify your code. Maybe introduce functions where you notice you are repeating yourself. You can compare with the original version so you know that you are not introducing bugs.&lt;/li&gt;&#xA;&lt;li&gt;Decide if object orientated approach will reduce the complexity of the code further.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The main message is don't start writing classes until you have already solved the problem in the simplest possible way. It is only through gaining the experience of solving a problem you will know how to define your object orientated interface. If you do this before hand it is likely to just get in the way.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-06-24T11:16:57.253" LastActivityDate="2013-06-24T11:16:57.253" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7780" PostTypeId="1" CreationDate="2013-06-24T12:32:03.590" Score="5" ViewCount="2579" Body="&lt;p&gt;I want to numerically solve the non-linear diffusion equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial}{\partial t} T(x,t)= \frac{\partial}{\partial x}\left(T^{5/2} \frac{\partial T}{\partial x} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use finite difference approach to solve it via Crank-Nicolson method. But I don't understand how to treat the non-linear coefficient when applying the numerical method.&lt;/p&gt;&#xA;" OwnerUserId="4624" LastEditorUserId="3691" LastEditDate="2013-06-24T13:01:52.507" LastActivityDate="2013-10-05T14:05:43.507" Title="Numerical solution of non-linear diffusion equation via finite-difference with the Crank-Nicolson method" Tags="|finite-difference|parabolic-pde|nonlinear-equations|diffusion|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7781" PostTypeId="2" ParentId="7754" CreationDate="2013-06-24T13:16:58.907" Score="3" Body="&lt;p&gt;Given a PDE choosing the correct numerical solving strategy requires some knowledge/expertise. (Computational &lt;em&gt;Science&lt;/em&gt; is indeed a &quot;&lt;em&gt;Science&lt;/em&gt;&quot; and one has to learn it.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In application specific software (e.g commercial FEM solvers for engineering problems like &lt;a href=&quot;http://en.wikipedia.org/wiki/Crash_simulation&quot; rel=&quot;nofollow&quot;&gt;crash problems&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Forming_%28metalworking%29&quot; rel=&quot;nofollow&quot;&gt;metal forming&lt;/a&gt;) this knowledge is somehow &quot;crystalized&quot; and embedded, so that most application engineers should be able to numerically solve physical problems without too much trouble. Analysts can concentrate on the physics of the problem, and not too much on the numerics. This is an optimistic view: really tough problems require multidisciplinary expertise (num. analysis, computer science, engineering, physics) even when application specific commercial software is used; but at least, in a day by day use, big errors do not occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the contrary your question is very general and broad, asking for guidelines applicable to a generic PDE and solving strategy, in a non-application specific environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that without any little hint to the type of problem you are trying to solve it is impossible to give you a sensible answer, as it is impossible to condense a sizable chunk of Computational Science in a few paragraphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless I think that some useful advice could be given.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Physical quantities have dimensions: saying that $10^{-12}$ is a very small absolute error is meaningless. Check your equation for dimensional consistency; if possible try to write them in a non-dimensional well-scaled form.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For PDE's the &quot;correct&quot; time step may depend not only on how the equations are scaled (choice of physical units of measure) but also on the spatial discretization (finer mesh, smaller time step.) If you do not have theoretical results at hand, compare results varying both time-step and mesh-size. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Always try to compute a global quantity (e.g. kinetic energy) and plot its evolution over time at variable time steps: you will gain insight on what's going on.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is nothing wrong in a trial-an-error procedure, provided you are able to asses if a given solution is physically meaningful.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Not much help, I know, but I hope this will be a good starting point.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2013-06-24T17:26:42.140" LastActivityDate="2013-06-24T17:26:42.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7782" PostTypeId="1" AcceptedAnswerId="7783" CreationDate="2013-06-24T16:12:52.023" Score="5" ViewCount="718" Body="&lt;p&gt;my question is between mathematics, physics and informatics. Suppose i have an Hamiltonian (hermitian matrix) that i can diagonalize. The matrix that allows this transformation is a unitary matrix build with the eigenvectors of my Hamiltonian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose I select only the $n$ first eigenvectors of length $m$. The matrix $A$ can be build with those eigenvectors, it's size is $m\times n$ with $m&amp;gt;n$. The eigenvectors are the column of the $A$ matrix. The $n$ eigenvectors are of course linearly independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I select $n$ rows of my $A$ matrix and build $B$, a new $n\times n$ matrix. It is logic that $B$ can be such that $\det(B)=0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my question : Is there a way (using Lapack if possible) to select randomly the rows in order to be sure that $\det(B)\neq0$ ?&lt;/p&gt;&#xA;" OwnerUserId="4626" LastEditorUserId="4626" LastEditDate="2013-06-26T06:51:36.983" LastActivityDate="2013-06-26T06:51:36.983" Title="find a set of linearly independent columns in a $m\times n$ matrix" Tags="|eigensystem|matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7783" PostTypeId="2" ParentId="7782" CreationDate="2013-06-24T16:48:11.937" Score="5" Body="&lt;p&gt;Yes. Simply run $n$ steps of column-pivoted QR on the adjoint of your matrix. You may want to look into &lt;a href=&quot;http://www.netlib.org/lapack/complex16/zgeqp3.f&quot; rel=&quot;nofollow&quot;&gt;zgeqp3&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you express the result of the pivoted $QR$ decomposition as:&#xA;$$ A^H \Omega = Q R, $$&#xA;your $B$ matrix will be the adjoint of the left-most $n$ columns of $A^H \Omega$, as $\Omega$ is a permutation matrix.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2013-06-25T15:28:30.070" LastActivityDate="2013-06-25T15:28:30.070" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7784" PostTypeId="1" CreationDate="2013-06-24T17:04:09.777" Score="0" ViewCount="1233" Body="&lt;p&gt;I'd very much appreciate if you could help me out with this as I think I'm missing something simple out or should be using alternate syntax:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am running the following Python code in FEniCS, where I solve for a variable - the solution looking as I expect it to, so no problems there, now I wish to solve a next equation and I need to define the subdomains according to the obtained solution, but the resultant plot of subdomains for the new problem (in the same code) is erroneous. It looks as if it cannot take the solution variable as a legitimate parameter appearing in the subdomain definitions; perhaps I need to change/avoid the inside method?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solve(a == L, phi1)&#xA;&#xA;plot(phi1, title=&quot;phi1_t01&quot;)&#xA;&#xA;# S at t1&#xA;# define mesh&#xA;htopv = 0.5&#xA;&#xA;mesh = Rectangle(0, 0, 1, htopv, 250, 250)&#xA;&#xA;# define a meshfunction for numbering subdomains&#xA;subdomains = MeshFunction(&quot;uint&quot;, mesh, 2)&#xA;&#xA;# define the subdomains&#xA;class Biomass(SubDomain):&#xA;    def inside(self, x):&#xA;        return True if phi1 &amp;lt;= 0 else False&#xA;class Interface(SubDomain):&#xA;    def inside(self, x):&#xA;        return True if phi1 &amp;gt;= 0 and phi1 &amp;lt;= 0.1 else False&#xA;class Bulk(SubDomain):&#xA;    def inside(self, x):&#xA;        return True if phi1 &amp;gt;= 0.1 else False&#xA;# note: it is essential to use &amp;lt;= and &amp;gt;= in the comparisons&#xA;&#xA;# mark the subdomains with numbers&#xA;subdomain0 = Biomass()&#xA;subdomain0.mark(subdomains, 0)&#xA;subdomain1 = Interface()&#xA;subdomain1.mark(subdomains, 1)&#xA;subdomain2 = Bulk()&#xA;subdomain2.mark(subdomains, 2)&#xA;&#xA;# plot the subdomains&#xA;plot(subdomains, title=&quot;S_t1 subdomains&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4628" LastEditorUserId="1804" LastEditDate="2013-06-24T17:37:11.637" LastActivityDate="2013-06-24T17:43:30.867" Title="defining subdomains in FEniCS (Python) using obtained (solved) variable" Tags="|finite-element|python|fenics|" AnswerCount="1" CommentCount="3" ClosedDate="2013-06-25T23:44:52.553" ContentLicense="CC BY-SA 3.0" />
  <row Id="7785" PostTypeId="2" ParentId="7784" CreationDate="2013-06-24T17:43:30.867" Score="5" Body="&lt;p&gt;You need to evaluate the Function at the given point. Assuming &lt;code&gt;phi1&lt;/code&gt; is a scalar &lt;code&gt;Function&lt;/code&gt;, try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Bulk(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        return phi1(x) &amp;gt;= 0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and similar for the other &lt;code&gt;SubDomains&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="4250" LastActivityDate="2013-06-24T17:43:30.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7787" PostTypeId="2" ParentId="7778" CreationDate="2013-06-24T20:08:56.757" Score="9" Body="&lt;p&gt;Simple answer: in modern python every data type is a class, so formally there is no difference between the two solutions you proposed. (Please remember to use new-style classes: classic classes are obsolete! See &lt;a href=&quot;http://docs.python.org/2/reference/datamodel.html#new-style-and-classic-classes&quot;&gt;http://docs.python.org/2/reference/datamodel.html#new-style-and-classic-classes&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question should be: how do I organize an efficient data structure in python? There is no doubt that the very idea of organizing the cells as an array of &lt;code&gt;class Cell&lt;/code&gt; instances is &lt;em&gt;way too&lt;/em&gt; inefficient. You will end up with a mess of pointers and non-contigous data organized like a complicated linked list. You have of course the ability of easily insert new cells in your list: but do you need this feature? On the contrary you will have non contiguous data storage, and you have to access every cell by different levels of indirection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you organize your data as a &lt;code&gt;numpy.ndarray&lt;/code&gt; then data is memory-contiguous, and accessing different cells is simply done striding through your memory block: space efficient (no memory wasted for pointers) and &lt;em&gt;fast&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by Ethan, OO concepts should be used, but at higher level, once an efficient low level data structure has been implemented, usually through &lt;code&gt;numpy.ndarray&lt;/code&gt;'s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OO programming means binding data to the methods that operate on the data itself at higher level of abstraction. (An example: I implemented a FEM code in which the stiffness matrix was defined as a class with a method for sparse super-nodal cholesky factorization. The first implementation was in-core: when an out-of-core implementation was needed, this was obtained via inheritance and &lt;em&gt;minimal&lt;/em&gt; adjustments to the underlining data storage. Almost 100% of the super-nodal cholesky code was reused.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A last comment, but crucial: an efficient numerical procedure is the result of a smart mapping of an algorithm and a data structure to your target computing architecture. If you start with the wrong data structure, there is &lt;em&gt;no way&lt;/em&gt; of recovering efficiency, without a complete rewrite.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-06-24T20:08:56.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7788" PostTypeId="2" ParentId="7782" CreationDate="2013-06-24T22:26:37.523" Score="2" Body="&lt;p&gt;I think if I understand your (long) question correctly, then here is a summary of it: You have a matrix $A\in {\mathbb R}^{n\times m}, n&amp;gt;m$ with full column rank. Can I find a subset ${\cal I}\subset [1,n], |{\cal I}|=m$ of rows of $A$ that are linearly independent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, then here's an algorithm: Start with ${\cal I}=\{\}$ and repeat:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;add a new row if it is linearly independent of the previous rows;&lt;/li&gt;&#xA;&lt;li&gt;if $|{\cal I}|=m$ then you're done&lt;/li&gt;&#xA;&lt;li&gt;otherwise, if no new vector could be found that is linearly independent of the previous ones, drop the element last added to the index set and add a different vector&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is essentially a depth first search through a tree whose terminal nodes represent all $m$-tuples of rows and tries to find the first one that contains $m$ linearly independent trees, pruning those sub-trees where less than $m$ vectors are already linearly dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may not be an efficient algorithm, but it should work just fine for small $m$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-24T22:26:37.523" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7789" PostTypeId="2" ParentId="7776" CreationDate="2013-06-24T23:29:31.677" Score="3" Body="&lt;p&gt;Performing $k$ steps of a pivoted (rank-revealing) LDL^T factorization should give you such an approximation. This is more obvious when you recognize that an $LDL^T$ factorization is in the form of your symmetric expansion, where each column of $L$ is one of your basis vectors, say $u_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pivoted $LDL^T$ factorizations yield:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ P A P^T = L D L^T, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = (P^T L) D (P^T L)^T $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;provides a symmetric decomposition. You can stop the factorization process once the remaining portion of the matrix is sufficiently small in your favorite norm. The total amount of work for a rank-$k$ approximation should be $O(k n^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Gu and Miranian's paper &lt;a href=&quot;http://libra.msra.cn/Publication/4521400/strong-rank-revealing-cholesky-factorization&quot; rel=&quot;nofollow&quot;&gt;Strong rank-revealing Cholesky factorization&lt;/a&gt; for a discussion of this issue in the SPD regime.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2013-06-24T23:38:31.693" LastActivityDate="2013-06-24T23:38:31.693" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7790" PostTypeId="2" ParentId="7780" CreationDate="2013-06-25T00:19:15.293" Score="4" Body="&lt;p&gt;The Crank-Nicolson discretization of this equation will read&#xA;$$&#xA;  \frac{T^n-T^{n-1}}{\Delta t} = \frac 12 \left[&#xA;    \partial_x \left((T^n)^{5/2} \partial_x T^n\right)&#xA;    +&#xA;    \partial_x \left((T^{n-1})^{5/2} \partial_x T^{n-1}\right)&#xA;\right]&#xA;$$&#xA;which is a nonlinear, time-independent, elliptic partial differential equation in $T^n$. The way to solve such equations is summarized in step-15 of the deal.II tutorial, see &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_15.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_15.html&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I'm one of the authors of deal.II, and I supervised the student who wrote step-15.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-06-25T13:48:02.540" LastActivityDate="2013-06-25T13:48:02.540" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7791" PostTypeId="2" ParentId="7714" CreationDate="2013-06-25T02:47:00.850" Score="36" Body="&lt;p&gt;I'm not aware of any recent overview articles, but I am actively involved in the development of the PFASST algorithm so can share some thoughts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are three broad classes of time-parallel techniques that I am aware of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;across the method — independent stages of RK or extrapolation integrators can be evaluated in parallel; see also the RIDC (revisionist integral deferred correction algorithm)&lt;/li&gt;&#xA;&lt;li&gt;across the problem — waveform relaxation&lt;/li&gt;&#xA;&lt;li&gt;across the time-domain — Parareal; PITA (parallel in time algorithm); and PFASST (parallel full approximation scheme in space and time).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Methods that parallelize across the method usually perform very close to spec but don't scale beyond a handful of (time) processors.  Typically they are relatively easier to implement than other methods and are a good if you have a few extra cores lying around and are looking for predictable and modest speedups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Methods that parallelize across the time domain include Parareal, PITA, PFASST.  These methods are all iterative and are comprised of inexpensive (but inaccurate) &quot;coarse&quot; propagators and expensive (but accurate) &quot;fine&quot; propagators.  They achieve parallel efficiency by iteratively evaluating the fine propagator in parallel to improve a serial solution obtained using the coarse propagator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Parareal and PITA algorithms suffer from a rather unfortunate upper bound on their parallel efficiency $E$: $E &amp;lt; 1/K$ where $K$ is the number of iterations required to obtain convergence throughout the domain.  For example, if your Parareal implementation required 10 iterations to converge and you are using 100 (time) processors, the largest speedup you could hope for would be 10x.  The PFASST algorithm relaxes this upper bound by hybridizing the time-parallel iterations with the iterations of the Spectral Deferred Correction time-stepping method and incorporating Full Approximation Scheme corrections to a hierarchy of space/time discretizations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lots of games can be played with all of these methods to try and speed them up, and it seems as though the performance of these across-the-domain techniques depends on what problem you are solving and which techniques are available for speeding up the coarse propagator (coarsened grids, coarsened operators, coarsened physics etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some references (see also references listed in the papers):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;This paper demonstrates how various methods can be parallelised across the method:  &lt;a href=&quot;http://arxiv.org/abs/1305.6165&quot;&gt;A theoretical comparison of high order explicit Runge-Kutta, extrapolation, and deferred correction methods&lt;/a&gt;; Ketcheson and Waheed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This paper also shows a nice way of parallelizing across the method, and introduces the RIDC algorithm: &lt;a href=&quot;http://people.maths.ox.ac.uk/~macdonald/ridc.pdf&quot;&gt;Parallel high-order integrators&lt;/a&gt;; Christlieb, MacDonald, Ong.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This paper introduces the PITA algorithm: &lt;a href=&quot;http://www.stanford.edu/group/frg/publications/recent/pap.pdf&quot;&gt;A Time-Parallel Implicit Method for Accelerating the Solution of Nonlinear Structural Dynamics Problems&lt;/a&gt;; Cortial and Farhat.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are lots of papers on Parareal (just Google it).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Here is a paper on the Nievergelt method: &lt;a href=&quot;http://arxiv.org/abs/1304.6514&quot;&gt;A minimal communication approach to parallel time integration&lt;/a&gt;; Barker.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This paper introduces PFASST: &lt;a href=&quot;http://msp.org/camcos/2012/7-1/p04.xhtml&quot;&gt;Toward an efficient parallel in time method for partial differential equations&lt;/a&gt;; Emmett and Minion; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This papers describes a neat application of PFASST: &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=2389121&quot;&gt;A massively space-time parallel N-body solver&lt;/a&gt;; Speck, Ruprecht, Krause, Emmett, Minion, Windel, Gibbon.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have written two implementations of PFASST that are available on the 'net: &lt;a href=&quot;http://pypfasst.readthedocs.org/en/latest/&quot;&gt;PyPFASST&lt;/a&gt; and &lt;a href=&quot;https://libpfasst.readthedocs.org/en/latest/&quot;&gt;libpfasst&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="6923" LastEditDate="2013-12-28T20:22:09.873" LastActivityDate="2013-12-28T20:22:09.873" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7794" PostTypeId="2" ParentId="7714" CreationDate="2013-06-25T07:45:41.457" Score="3" Body="&lt;p&gt;Here is a short introduction to &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/EncyclopediaSpringer.pdf&quot; rel=&quot;nofollow&quot;&gt;waveform relaxation&lt;/a&gt;.  When talking about the time-parallel method like parareal or PITA or other methods, one should distinguish between the dissipative and the conservative (Hamiltonian) ODE systems.  The latter seems more difficult to parallelize in time dimension by partitioning to time sub-intervals. Here is an analysis of parareal for &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/parareal-gh.pdf&quot; rel=&quot;nofollow&quot;&gt;Hamiltonian systems&lt;/a&gt;. The dissipative system is easier because the error caused at initial time $u_0$ tends to disappear due to the dissipation $u(t)=\exp(-\lambda t)u_0,$ $\mathrm{Re}\,\lambda&amp;gt;0.$&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2013-06-25T09:36:12.003" LastActivityDate="2013-06-25T09:36:12.003" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7795" PostTypeId="1" AcceptedAnswerId="7797" CreationDate="2013-06-25T16:14:20.273" Score="4" ViewCount="167" Body="&lt;p&gt;I am using an &lt;em&gt;Euler-Euler&lt;/em&gt; method to model two phases - both are treated as a continuum using modified &lt;em&gt;Navier-Stokes&lt;/em&gt; equations. One phase is air and the other is particles, that are being entrained by the air.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original geometry is very small, the airpath being 2 x 20 mm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the mesh is scaled up 10 times (grid fineness is &lt;strong&gt;not&lt;/strong&gt; changed, the &lt;strong&gt;dimensions&lt;/strong&gt; are, i.e. become 20 x 200 mm), the results match experimental ones pretty well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grid independence has already been carried out. Irrespective, the scaling problem is seen in both coarse and fine grids. I am thinking along the lines - discrete phase is more accurately modelled with bigger dimenions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a transient simulation, with atmospheric pressure at inlet and a pressure gradient at outlet. The computational domain resembles a 'T' shape, with powder at the bottom and airflow at the top.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Errors in experimental data are unlikely, as it is an established powder entrainment pattern.&lt;/p&gt;&#xA;" OwnerUserId="4469" LastEditorUserId="4469" LastEditDate="2013-06-26T09:00:13.270" LastActivityDate="2013-06-26T09:00:13.270" Title="Why does the wrong mesh scale produce a more accurate result?" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7797" PostTypeId="2" ParentId="7795" CreationDate="2013-06-25T16:53:19.243" Score="6" Body="&lt;p&gt;Without knowing anything about your method, this is most common when one of two things are occurring:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a boundary layer effect happening in your experiment that cannot be represented in your numerical model. (bigger domain works is a feature!)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is an inconsistency in how you're representing a boundary condition as compared to your physical system. (bigger domain works implies bug!)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;By putting your boundary conditions, which are apparently not quite physically correct, far from the area of interest, boundary layers stop affecting your numerical observations (as much as they do in a smaller domain).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I don't know much about two-phase NS, but for single-phase NS it seems possible that your BCs are much smoother than is physically realistic.  To experimentally ensure a constant pressure and flux on the inflow/outflow would be difficult.  Putting the boundaries further away allows flow to establish itself before entering the &quot;region of interest&quot; where your samples are taken.  For instance, in pipe-flow, using a constant BC when the &quot;established&quot; flow would be Poiseuille flow would result in more accurate observations far from the BCs.  (I'm sure this is much simpler than your case, but hopefully gets you thinking in the right direction!)&lt;/p&gt;&#xA;" OwnerUserId="508" LastEditorUserId="508" LastEditDate="2013-06-26T00:03:10.620" LastActivityDate="2013-06-26T00:03:10.620" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7798" PostTypeId="2" ParentId="7795" CreationDate="2013-06-25T17:03:17.090" Score="1" Body="&lt;p&gt;I'm not sure what exactly you mean by saying scaling up the grid but keeping the fineness. Surely the solution domain remains constant. You are also leaving out many other potentially important details, such as the numerical method (space and time, steady/unsteady, boundary conditions, etc). I will proceed assuming that you mean coarsening the grid by a factor of 10. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you see - better agreement between CFD and experimental data with coarse grids - is not unheard of. But it nevertheless is probably spurious in the sense that it does not accurately reflect what is the &quot;true&quot; numerical solution of your model. To obtain this &quot;true&quot; solution, you need to systematically refine your spatial and temporal resolution, i.e. you need to carry out a grid and time-step refinement study. When the numerical results become independent (or close to independent) of the resolution, you have obtained the &quot;true&quot; numerical solution of your model. Any deviations that remain between this solution and the experimental data must be due to the model you are solving. (In your particular case the modeling of the solid phase and the coupling of the phases.) Another source for deviations may be measurement errors in the experiment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Refinement studies are usually carried out by halving the grid spacing in each coordinate direction for each refinement. This means that you will be carrying out simulations with your original grid of N points (or cells), then with a grid with 4N, 16N, etc. The same is done with the time step. If you solve the unsteady form of the equations, you want to keep the CFL number constant (and small) as you refine the grid. It is not difficult to find papers that use grids that are much too coarse. I remember recent articles on fluidized bed simulations in which it was shown that what was widely thought to be a model deficiency was in fact nothing but a lack of resolution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many multiphase flows are inherently unsteady, in which case you need to average your solution in some form. Then you need to be careful about your averaging duration and sampling frequency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I base all of the above on the assumption that your code has undergone code verification. This is a prerequisite for any validation (comparison of simulation with experimental data). What I describe above is solution verification. If you are not familiar with these terms, I suggest you look at the book by Oberkampf and Roy. &lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2013-06-25T17:03:17.090" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7799" PostTypeId="1" AcceptedAnswerId="7811" CreationDate="2013-06-26T05:38:17.947" Score="2" ViewCount="1166" Body="&lt;p&gt;Monte Carlo simulations for MATLAB sometimes require a huge usage of CPU and in some cases may take a long time.&#xA; MATLAB has provided some structures in order to enable the parallel processing. But I have been intrested in using the processors of the graphical card (GPU)... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any tutorial for using GPU in order to perform a parallel processing in MATLAB ?&lt;/p&gt;&#xA;" OwnerUserId="4651" OwnerDisplayName="Mamal" LastEditorUserId="9" LastEditDate="2013-06-26T17:51:28.903" LastActivityDate="2013-06-29T08:10:38.930" Title="Introductory tutorial for Monte Carlo computations on the GPU?" Tags="|matlab|gpu|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7800" PostTypeId="1" AcceptedAnswerId="7801" CreationDate="2013-06-26T07:29:54.340" Score="1" ViewCount="2660" Body="&lt;p&gt;I have X, Y and Z variables in matrix form, each of size n x 1. &#xA;Eg.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = [-38.0400, -38.6700, -38.9300, -39.4500...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Whenever I run the code below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% find center of X&#xA;minx = find(X==min(min(X)));&#xA;maxx = find(X==max(max(X)));&#xA;centx = (minx + maxx) / 2;&#xA;xy = Y(centx);&#xA;xz = Z(centx);&#xA;centx = [centx xy xz];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'll get the initial answer as, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;centx = 6.6034e+04&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but then an error will be displayed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Subscript indices must either be real positive integers or logicals.&#xA;Error in max_point(line 5)&#xA;xy = Y(centx);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My questions are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the answer given in scientific notation (exponential form) because of the trailing zeros in my original entries?&lt;/li&gt;&#xA;&lt;li&gt;How can I avoid getting the answer in scientific notation (exponential form) (because I think that is the cause for the error)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerDisplayName="user3991" LastEditorUserId="1642" LastEditDate="2013-06-29T00:44:36.047" LastActivityDate="2013-06-29T00:44:36.047" Title="Matlab output in scientific notation [ERROR: Subscript indices must either be real positive integers or logicals.]" Tags="|matrix|matlab|precision|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7801" PostTypeId="2" ParentId="7800" CreationDate="2013-06-26T07:38:05.257" Score="3" Body="&lt;p&gt;The problem is here,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;centx = (minx + maxx) / 2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You are using a floating point number as the index of an array. The error is simply saying that an index must be an integer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You want to know which index is the center of the x range? You will need to do that in a different way. How about this,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% find center of X&#xA;minx = find(X==min(min(X)));&#xA;maxx = find(X==max(max(X)));&#xA;centx = uint32(round((minx + maxx) / 2)); % notice I used the round function here and the cast to make it an integer!&#xA;xy = Y(centx);&#xA;xz = Z(centx);&#xA;centx = [centx xy xz];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This uses the round function,&#xA;&lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/round.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/matlab/ref/round.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;round&lt;/code&gt; function automatically use the most appropriate integer rounding function to get you to the &lt;strong&gt;closest value&lt;/strong&gt;, which I think it what you want in this case. You still need to cast to an integer as pointed out below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Internally the &lt;code&gt;round&lt;/code&gt; function is applying &lt;code&gt;ceil&lt;/code&gt; if the fractional part is $&amp;gt;=\frac{1}{2}$, or the &lt;code&gt;floor&lt;/code&gt; function when the fractional part $&amp;lt;\frac{1}{2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exponential answer is just because that is the way MATLAB prints floats.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-06-27T08:23:01.137" LastActivityDate="2013-06-27T08:23:01.137" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7802" PostTypeId="2" ParentId="7800" CreationDate="2013-06-26T07:52:27.020" Score="1" Body="&lt;p&gt;&lt;code&gt;centx = (minx + maxx) / 2;&lt;/code&gt; &#xA;will result in non-integer array index as @boyfarrell pointed out. You could use function &lt;code&gt;ceil&lt;/code&gt; or &lt;code&gt;floor&lt;/code&gt; to get the nearest integer index if that is what you are looking for. &lt;/p&gt;&#xA;" OwnerUserId="4364" LastActivityDate="2013-06-26T07:52:27.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7804" PostTypeId="1" AcceptedAnswerId="7805" CreationDate="2013-06-26T10:42:15.847" Score="1" ViewCount="1768" Body="&lt;p&gt;I'm trying the Torque scheduler.  I have previous experience with the LSF scheduler, and I quite like being able to use 'bsub -I make -j 12' to compile a program really quickly, i.e. running an 'interactive' job, and seeing any errors as they happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there is an interactive option in Torque, but it's really, really interactive: you actually have to type the commands by hand, all the time tieing up the machine, and being billed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I'm trying to avoid using Torque's 'interactive' facility, but I'd still like to be able to see the output of a job in real time, eg by doing 'tail -f' on the output file.  As far as I can tell, there doesn't appear to be any way of causing the output file to be flushed in real-time, as things happen, in Torque?  Or is there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary of goal:&#xA;- be able to simulate the '-I' option from LSF in Torque&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary of sub-goal:&#xA;- be able to request the output file to be flushed to disk continuously, whilst a job is running, in Torque&lt;/p&gt;&#xA;" OwnerUserId="3613" LastActivityDate="2020-03-10T20:51:42.120" Title="Flush output in Torque scheduler?" Tags="|hpc|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7805" PostTypeId="2" ParentId="7804" CreationDate="2013-06-26T12:34:22.037" Score="2" Body="&lt;p&gt;From the Torque website:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Spooling can be disabled by using the qsub '-k' option. With this option, job output and error streams can be sent directly to a file in the job's working directory bypassing the intermediate spooling step. This is useful if the job is submitted from within a parallel filesystem or if the output is small and the user would like to view it in real-time. If the output is large and the remote working directory is not available via a high performance network, excessive use of this option may result in reduced cluster performance. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-06-26T12:34:22.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7806" PostTypeId="5" CreationDate="2013-06-26T13:55:55.597" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-06-26T13:55:55.597" LastActivityDate="2013-06-26T13:55:55.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7807" PostTypeId="4" CreationDate="2013-06-26T13:55:55.597" Score="0" Body="Questions about optimization problems subject to additional constraints." OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-06-26T14:13:42.937" LastActivityDate="2013-06-26T14:13:42.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7808" PostTypeId="2" ParentId="5362" CreationDate="2013-06-26T14:59:32.253" Score="3" Body="&lt;p&gt;As Geoff Oxberry points out in his comment, if you know a priori that $x_k=0$ for $k\leq i$ and $k\geq j$, then there's nothing to optimize with respect to these variables and you can just solve the (smaller) reduced unconstrained problem $\min_{x\in\mathbb{R}^{j-i-1}}\hat f(x)$, where $\hat f$ takes only the nonzero variables as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not feasible for some reason, you might want to update your question to include this in order to get concrete advice. But to address the question in your comment: One possible approach to solve a constrained optimization problem of the form $\min_{x\in C} f(x)$ for some (convex) set $C\subset\mathbb{R}^n$ (in your case, $C=\{x\in\mathbb{R}^n: x_k=0,\ k\leq i \text{ or }k\geq j\}$) is &lt;em&gt;projected gradient descent&lt;/em&gt; (which is a special instance of proximal gradient methods), defined by the iteration&#xA;$$x^{k+1} = P_C \left(x^k - \alpha_k f'(x^k)\right)$$&#xA;where $P_C$ is the projection onto the convex set $C$ (in your case, simply setting the corresponding $x_k$ to zero) and $\alpha_k$ is a suitable step size (e.g., of Armijo type). If $f$ is smooth, this iteration converges to a stationary point (which is a minimizer if $f$ is convex); see, e.g., &lt;a href=&quot;http://research.microsoft.com/en-us/um/people/manik/projects/trade-off/papers/calamaimathprog87.pdf&quot; rel=&quot;nofollow&quot;&gt;PH Calamai, JJ Moré: Projected gradient methods for linearly constrained problems, Mathematical Programming 39, 93-116&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, this is what you are doing, except you are using the conjugate gradient as a search direction instead of the gradient. I haven't checked the proofs, but I'd assume that your version should still work as long as you have a descent direction. Whether it is reasonable, though, depends on your definition (it is a bit of a sledgehammer approach...)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2013-06-26T15:19:03.867" LastActivityDate="2013-06-26T15:19:03.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7809" PostTypeId="2" ParentId="7804" CreationDate="2013-06-26T15:31:09.120" Score="1" Body="&lt;p&gt;I've marked Gert's answer as the correct answer given the title of my question.  However, for completeness, note that what I did in the end was add an option &lt;code&gt;-x&lt;/code&gt;.  It looks like &lt;code&gt;-I -x&lt;/code&gt; in Torque is approximately equivalent to &lt;code&gt;-I&lt;/code&gt; in LSF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference to the -x option: &lt;a href=&quot;http://docs.adaptivecomputing.com/torque/4-1-3/help.htm#topics/commands/qsub.htm&quot; rel=&quot;nofollow&quot;&gt;http://docs.adaptivecomputing.com/torque/4-1-3/help.htm#topics/commands/qsub.htm&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;By default, if you submit an interactive job with a script, the script will be parsed for PBS directives but the rest of the script will be ignored since it's an interactive job. The -x option allows the script to be executed in the interactive job and then the job completes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;script.sh &#xA;#!/bin/bash &#xA;ls &#xA;---end script--- &#xA;qsub -I script.sh &#xA;qsub: waiting for job 5.napali to start &#xA;dbeer@napali:# &#xA;&amp;lt;displays the contents of the directory, because of the ls command&amp;gt; &#xA;qsub: job 5.napali completed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&quot;&lt;/p&gt;&#xA;" OwnerUserId="3613" LastEditorUserId="3613" LastEditDate="2013-06-27T00:49:30.027" LastActivityDate="2013-06-27T00:49:30.027" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7810" PostTypeId="2" ParentId="7800" CreationDate="2013-06-26T15:40:42.277" Score="0" Body="&lt;p&gt;In matlab by default everything is a matrix of double precision floating point numbers. If you like to do index arithmetic using integer variables you can perform an explicit type cast using the conversion functions listed in &lt;a href=&quot;http://www.mathworks.it/it/help/matlab/numeric-types.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.it/it/help/matlab/numeric-types.html&lt;/a&gt;. E.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minx = uint32(find(X==min(min(X))));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(here I'm using 32 bit unsigned integers, under the assumption that you do not need 64 bit pointers.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; a = 3/2&#xA;a =&#xA;    1.5000&#xA;&amp;gt;&amp;gt; b = uint32(3)/2 &#xA;b =&#xA;           2&#xA;&amp;gt;&amp;gt; whos&#xA;  Name      Size            Bytes  Class     Attributes&#xA;&#xA;  a         1x1                 8  double              &#xA;  b         1x1                 4  uint32              &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively you can do index arithmetic in double and then use the &lt;code&gt;ceil&lt;/code&gt; and &lt;code&gt;floor&lt;/code&gt; function to round the floating point number, as suggested in other answers. (Please note that &lt;code&gt;ceil&lt;/code&gt; and &lt;code&gt;floor&lt;/code&gt; return a double, they do not cast to an integer type.) &lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-06-26T15:40:42.277" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7811" PostTypeId="2" ParentId="7799" CreationDate="2013-06-26T17:50:24.650" Score="4" Body="&lt;p&gt;See the MathWorks example &lt;a href=&quot;http://www.mathworks.com/help/distcomp/examples/calculating-the-mandelbrot-set-on-a-gpu.html&quot; rel=&quot;nofollow&quot;&gt;Calculating the Mandelbrot Set on a GPU&lt;/a&gt;, which explains how you can use the Parallel Computing Toolbox to run simple &quot;embarassingly parallel&quot; jobs on GPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you may be interested in the &lt;a href=&quot;http://mathema.tician.de/software/pycuda&quot; rel=&quot;nofollow&quot;&gt;PyCUDA&lt;/a&gt; and &lt;a href=&quot;http://mathema.tician.de/software/pyopencl&quot; rel=&quot;nofollow&quot;&gt;PyOpenCL&lt;/a&gt; projects, which provide a similar capability to interactively launch GPU kernels.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2013-06-26T17:50:24.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7812" PostTypeId="1" CreationDate="2013-06-26T20:51:59.073" Score="4" ViewCount="421" Body="&lt;p&gt;I have some large (n~100) square matrices with entries two variable polynomials of bounded degree (roughly &amp;lt;20, but many entries are smaller) and integer coefficients, and I'd like to be able to compute their determinants (exactly, as two variable polynomials with integer coefficients).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What method should I use? Is there an efficient software implementation already?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In smaller instances of the problem, I've just been using &lt;code&gt;Mathematica&lt;/code&gt;'s builtin &lt;code&gt;Det&lt;/code&gt; function and hoping for the best, but I suspect it's just doing row reduction with two variable rational functions, which quickly becomes inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried (thanks Emily!) substituting integer values, calculating the determinant using row reduction with (much faster) arbitrary precision rational arithmetic, and then Newton interpolation. Sadly my implementation seems even slower than &lt;code&gt;Mathematica&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point I'm happy with 'good guesses', where if I check a few new values and consistently get the right answers I'm satisfied --- if proving the answer is really correct is much slower that's okay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, in case this suggests a good trick, there appears to some structure to the answers for my problem instances: the determinant factors, with a large fraction of the total degree coming from high powers of a few low degree polynomials, and then a handful of large factors without multiplicity. I can even predict some of the small factors and (approximately) their multiplicity.&lt;/p&gt;&#xA;" OwnerUserId="4645" LastActivityDate="2013-12-16T00:07:45.763" Title="Evaluating large determinants with multivariate polynomial entries" Tags="|matrix|polynomials|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7813" PostTypeId="2" ParentId="7812" CreationDate="2013-06-26T23:48:14.893" Score="3" Body="&lt;p&gt;Computing the determinant of matrices of size 100 is an expensive problem to begin with, even with just numbers. I would say it's still within the feasible range, but it's definitely at the limit of what can conveniently be done in a stable way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's even more expensive if you want to use symbolic computations for polynomials. The question, of course, is what happens even if you managed to do it: you'd get a polynomial of degree ~2,000 that you will find is impossible to evaluate stably unless it has a very particular structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation is not to wonder how it can be done, but whether you really need to do it and what you want to do with the result. As I say, whatever you get will be rather unreliable and you should try to come up with a method for whatever you want to do that does not involve computing the determinant of these matrices at all.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-26T23:48:14.893" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7814" PostTypeId="2" ParentId="7799" CreationDate="2013-06-26T23:53:34.210" Score="1" Body="&lt;p&gt;GPUs are the wrong approach for Monte Carlo methods if you want to use the standard Markov Chain approach. The reason is that Markov Chains are inherently sequential, whereas GPUs are only appropriate if you can parallelize an algorithm. It is &lt;em&gt;very&lt;/em&gt; difficult to design parallel Markov Chain Monte Carlo (MCMC) methods.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-26T23:53:34.210" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7815" PostTypeId="2" ParentId="7804" CreationDate="2013-06-27T02:28:51.617" Score="1" Body="&lt;p&gt;If auto connect to nodes is enabled you should be able to do this for batch jobs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;qpeek &amp;lt;jobid&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will dump the output log locally.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Usage:  qpeek [options] JOBID&#xA;&#xA;Options:&#xA;  -c      Show all of the output file (&quot;cat&quot;, default)&#xA;  -h      Show only the beginning of the output file (&quot;head&quot;)&#xA;  -t      Show only the end of the output file (&quot;tail&quot;)&#xA;  -f      Show only the end of the file and keep listening (&quot;tail -f&quot;)&#xA;  -e      Show the stderr file of the job&#xA;  -o      Show the stdout file of the job&#xA;&#xA;  -ssh               Use the ssh command rather than rsh to remote access the mother superior node&#xA;  -spool=&amp;lt;spool_loc&amp;gt; Specifiy the location of the spool directory, defaults to /var/spool/torque/spool&#xA;  -host=&amp;lt;host&amp;gt;       The name of the host to use in the filename for the jobs stdout or stderr&#xA;&#xA;  -help|? Display help message&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="564" LastEditorUserId="564" LastEditDate="2013-06-27T02:44:05.843" LastActivityDate="2013-06-27T02:44:05.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7816" PostTypeId="1" CreationDate="2013-06-27T07:12:41.640" Score="1" ViewCount="301" Body="&lt;p&gt;I have a certain ODE problem that needs to be solved using Scilab.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dx(1)/dt=k*x(1)-x(5)&#xA;dx(2)/dt=k2*x(2)-k1*x(1)&#xA;dx(3)/dt=k1*[x(2)-x(3)]&#xA;dx(4)/dt=k1*[x(3)-x(4)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;x(5)&lt;/code&gt; takes value &lt;code&gt;0&lt;/code&gt; till time &lt;code&gt;t0&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me with coding this problem? Any suggestions would be of immense help.&lt;/p&gt;&#xA;" OwnerUserId="4649" LastEditorUserId="20688" LastEditDate="2019-11-08T22:17:07.890" LastActivityDate="2019-11-08T22:17:07.890" Title="ODE Solving in Scilab" Tags="|ode|scilab|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7817" PostTypeId="2" ParentId="7087" CreationDate="2013-06-27T07:56:00.167" Score="3" Body="&lt;p&gt;The Chinese Restaurant Process is a way of looking at a Dirichlet process.  It is a distribution over distributions.  There are various ways of thinking about either.  One way of looking at it is that as you draw samples, for each new sample:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;there is a finite probability that the new sample is assigned to an existing cluster&lt;/li&gt;&#xA;&lt;li&gt;and otherwise it becomes the first sample in a new cluster&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, you don't have to specify how many clusters there are, which is why it's 'non-parametric'.  You do have to specify the probability of each new sample being assigned to a new cluster however, so there is a parameter for that, eg &lt;code&gt;alpha&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you use the DP (&quot;Dirichlet Process&quot;) as part of a model, feed it data, and use some way to solve the model.  Typically, the model is analytically non-tractable, so one can use Markov Chain Monte Carlo or Variational Approximation to solve it, and get some estimation of the number of clusters, given the data, and the &lt;code&gt;alpha&lt;/code&gt; parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I've simplified a little: the DP is a distribution over distributions.  The samples drawn above correspond to samples from a single draw from the distribution over distributions.  One can repeat the process of drawing samples, in order to draw samples from a new draw from the distribution over distributions...&lt;/p&gt;&#xA;" OwnerUserId="3613" LastActivityDate="2013-06-27T07:56:00.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7818" PostTypeId="2" ParentId="5127" CreationDate="2013-06-27T08:06:35.367" Score="0" Body="&lt;p&gt;In the simplest case, you can simulate an HPC environment using a computer with 2 or more cores.  You can run both OpenMP and/or MPI in such an environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, if you're doing large calculations on an HPC, you might use more than one compute node.  This means there are two 'computers', communicating across a network.  This imposes some constraints compared to a single multithreaded compute node:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;OpenMP is no longer sufficient for parallelization, at least on its own.  MPI becomes the de facto standard&lt;/li&gt;&#xA;&lt;li&gt;the network speed becomes a huge bottleneck for computation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now this doesn't mean that you can't run HPC code if you only have a single 2-core PC.  You can.  And the easiest way is to use MPI for parallelizing your code, rather than OpenMP, or pthreads and so on. However, if you can get access to two or more multicore PCs, with a fast network between them (at least Gigabit), then this will give you a better feel for what is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So much for multicore processing using MPI. Another thing to think about is GPGPU.  Clusters nowadays often make GPUs available to you, which you can use for linear algebra, using libraries such as gnumpy in python, or similar.  If you want to be able to experiment with GPGPU, you will probably want to either:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;get a computer with a GPU that can run CUDA code, ie an nVidia GPU&lt;/li&gt;&#xA;&lt;li&gt;get a computer that can run OpenCL.  In fact, Intel processors since last year (Ivy bridge onwards) contain an OpenCL-compliant GPU.  AMD processors also contain OpenCL-compliant GPUs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, in summary:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the simplest system to experiment with MPI would be a dual-core CPU&lt;/li&gt;&#xA;&lt;li&gt;for more realistic experimentation, you want two networked multi-core PCs&lt;/li&gt;&#xA;&lt;li&gt;you might also consider obtaining either a CUDA or OpenCL compliant GPU, or making sure that your CPU contains an OpenCL-compliant GPU.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3613" LastActivityDate="2013-06-27T08:06:35.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7819" PostTypeId="1" AcceptedAnswerId="7839" CreationDate="2013-06-27T10:10:33.840" Score="12" ViewCount="15252" Body="&lt;p&gt;For a project, I have to implement these two methods and compare how they perform on different functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like the &lt;em&gt;conjugate gradient method&lt;/em&gt; is meant to solve systems of linear equations of the for &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A\mathbf{x} = \mathbf{b}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $A$ is an n-by-n matrix that is symmetric, positive-definite and real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, when I read about &lt;em&gt;gradient descent&lt;/em&gt; I see the example of the &lt;em&gt;Rosenbrock function&lt;/em&gt;, which is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(x_1,x_2) = (1-x_1)^2+100(x_2-x_1^2)^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I see it, I can't solve this with a conjugate gradient method. Or do I miss something?&lt;/p&gt;&#xA;" OwnerUserId="4653" LastEditorDisplayName="user9925" LastEditDate="2017-10-14T19:54:55.383" LastActivityDate="2017-10-14T19:54:55.383" Title="Gradient descent and conjugate gradient descent" Tags="|optimization|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7820" PostTypeId="1" CreationDate="2013-06-27T10:22:34.347" Score="7" ViewCount="768" Body="&lt;p&gt;My code solves the incompressible Navier-Stokes equation in a conducting fluid, together with the induction equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \partial_t u + u \nabla u + 2\Omega \times u = -\nabla p + \nu \Delta u + (\nabla \times b) \times b \\&#xA;\partial_t b = \nabla \times (u \times b) + \eta \Delta b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ and $b$ are the velocity and magnetic fields, $\eta$ and $\nu$ are diffusion constants, $p$ is the pressure, $\Omega$ the rotation rate of the reference frame.&#xA;I solve this PDE system in spherical geometry using finite differences in the radial direction and spherical harmonic expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's focus on the Navier-Stokes equation which contains the Lorentz force $(\nabla \times b) \times b$.&#xA;I use a Crank-Nicholson semi-implicit scheme for the diffusive terms, while the non-linear terms $u\nabla u$ and $(\nabla \times b)\times b$ are treated explicitely:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Au_{n+1} = Bu_{n} + NL(u_n,b_n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_n$ and $b_n$ are respectively my velocity and magnetic field at time-step $n$, and $A$ and $B$ are matrices containing the time-step. (Note that $NL(u_n,b_n)$ actually also includes $2\Omega\times u$ for convenience, and that $\nabla p$ is eliminated by taking the curl)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to adjust dynamically my time-step. For Navier-Stokes, the usual CFL criterion can be used, but what about the Lorentz force ? I can proably come up with some specific criterion, but here is my question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a way to adjust the time-step $\Delta t$ knowing only the non-linear term $NL(u_n,b_n)$ and ignoring its analytical form or physical meaning ?&lt;/strong&gt;&#xA;Some sort of generalised CFL criterion ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've thought about something like $NL \Delta t &amp;lt; c\, U$ (where $c$ is a constant).&#xA;When $NL = u\nabla u \sim \frac{U^2}{\Delta x}$, it reduces to the CFL $\frac{U\Delta t}{\Delta x} &amp;lt; c$.&#xA;Doest it makes sense ? Is there a better way to think about this ? Are there references on the subject ?&lt;/p&gt;&#xA;" OwnerUserId="3260" LastEditorUserId="3260" LastEditDate="2013-07-02T17:40:46.030" LastActivityDate="2013-07-02T17:40:46.030" Title="Navier-Stokes solver: How to adjust the time step based on non-linear terms?" Tags="|pde|fluid-dynamics|time-integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7821" PostTypeId="2" ParentId="7819" CreationDate="2013-06-27T10:59:06.590" Score="12" Body="&lt;p&gt;In this context, both methods can be thought of as minimization problems of the function:&#xA;$$&#xA;\phi(\boldsymbol{x}) = \frac{1}{2}\boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x} - \boldsymbol{x}^T\boldsymbol{b} &#xA;$$&#xA;When $\boldsymbol{A}$ is symmetric, then $\phi$ is minimized when $\boldsymbol{A}\boldsymbol{x} = \boldsymbol{b}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gradient descent is the method that iteratively searches for a minimizer by looking in the gradient direction. Conjugate gradient is similar, but the search directions are also required to be orthogonal to each other in the sense that $\boldsymbol{p}_i^T\boldsymbol{A}\boldsymbol{p_j} = 0 \; \; \forall i,j$.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-06-28T02:54:24.830" LastActivityDate="2013-06-28T02:54:24.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7822" PostTypeId="1" AcceptedAnswerId="7831" CreationDate="2013-06-27T12:39:49.727" Score="2" ViewCount="99" Body="&lt;p&gt;I have just read &quot;Accelerating the convergence of spectral deferred correction methods&quot;. The link is here: &lt;a href=&quot;http://www.unc.edu/~junjia/papers/sdcgmres.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.unc.edu/~junjia/papers/sdcgmres.pdf&lt;/a&gt;‎.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I wonder how to understand deferred correction method. Can you give me a description of this method? Or any other papers or books to read to understand this method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I may want to add a parallel-computing tag because I think PFASST has something to do with this.&lt;/p&gt;&#xA;" OwnerUserId="4562" LastActivityDate="2013-06-27T21:42:30.107" Title="Can you give me a detailed description of (spetral) deferred correction method?" Tags="|parallel-computing|ode|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7823" PostTypeId="1" AcceptedAnswerId="7834" CreationDate="2013-06-27T12:43:34.223" Score="2" ViewCount="1924" Body="&lt;p&gt;I am interesting in solving the following nonlinear, time-dependent pde in 2 spatial dimensions (complex Gross-Pitaevskii eq):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$i \frac{\partial \psi}{\partial t} = \left[ -\nabla^2 + (1-i \sigma)(|\psi|^2-1) \right] \psi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to find steady state solutions for the function $\psi(x,y,t)$, for different parameters $\sigma$. Here $\sigma$ is a positive real-valued parameter. The boundary conditions can initially be of the Dirichlet type, setting the function $\psi$ to zero on the contour of a square, but later on I plan to implement some absorbing BC (perhaps using the perfectly matched layer method?). I am also considering solving the equation in a different geometry later on, on a disk using polar coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the time-dependence, I plan to use the Gryphon module of Erik Skare (&lt;a href=&quot;https://launchpad.net/gryphonproject&quot; rel=&quot;nofollow&quot;&gt;https://launchpad.net/gryphonproject&lt;/a&gt;), which is basically a Runge-Kutta solver and for the spatial part Fenics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is, do you think this is feasible to do with Fenics, and if so, how would one proceed?&lt;/p&gt;&#xA;" OwnerUserId="4656" LastActivityDate="2013-06-28T02:28:47.557" Title="time-dependent nonlinear pde in fenics" Tags="|pde|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7824" PostTypeId="1" AcceptedAnswerId="7830" CreationDate="2013-06-27T12:55:37.020" Score="10" ViewCount="856" Body="&lt;p&gt;The Parareal, PITA, and PFASST algorithms are all &lt;em&gt;across-the-domain&lt;/em&gt; techniques for parallelizing the solution of time-dependent problems in time. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the guiding principles behind these methods?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the main differences between them?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I say that one is based on another? How?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What about their applications?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I know there will be no answer to the question &quot;which is better?&quot;, but a good understanding of their application areas and validation conditions is helpful to me.&lt;/p&gt;&#xA;" OwnerUserId="4562" LastEditorUserId="486" LastEditDate="2013-06-27T16:52:19.720" LastActivityDate="2013-06-27T21:31:42.710" Title="What are the differences between Parareal, PITA, and PFASST?" Tags="|parallel-computing|time-integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7825" PostTypeId="1" AcceptedAnswerId="7829" CreationDate="2013-06-27T13:15:31.403" Score="2" ViewCount="2232" Body="&lt;p&gt;I have got an input series of matrices&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_1, A_2, A_3, \dots $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the difference between $A_i$ and $A_{i+1}$ is the replacement of one single column. Before i get to know $A_{i+1}$, I have to compute $A_i^{-1} b$ for some fixed vector $b$, so I also have a series of vectors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_1^{-1}b, A_2^{-1}b, A_3^{-1}b, \dots $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to compute. For your interest, this appears in implementations of the simplex method, but I prefer to approach with the point of view of numerical linear algebra. Questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Where can I find a reference on updating the QR decomposition after replacing one column?&lt;/li&gt;&#xA;&lt;li&gt;Since I am only interested in the QR decomposition in order to solve a sequence of linear problems, can this be exploited in the algorithms,too?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I use the QR decomposition only because this is a standard method - in the case that you have a better recommendation for the above setting, feel free to provide your advice.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2013-06-27T15:49:07.483" Title="Update QR decomposition when one column is exchanged" Tags="|linear-algebra|linear-solver|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7826" PostTypeId="2" ParentId="7816" CreationDate="2013-06-27T13:50:53.067" Score="2" Body="&lt;p&gt;As suggested by GertVdE, you can use &lt;a href=&quot;http://help.scilab.org/docs/5.3.2/en_US/ode.html&quot; rel=&quot;nofollow&quot;&gt;ODE&lt;/a&gt; function in scilab, &#xA;or refer the following document,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.infoclearinghouse.com/files/scilab/scilab09.pdf&quot; rel=&quot;nofollow&quot;&gt;Ordinary Differential Equations with SCILAB&lt;/a&gt;, written by Gilberto E. Urroz discusses various ODE's, their types and their solution with SCILAB,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on page 45, you might an answer to your question. &lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2013-06-27T13:50:53.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7827" PostTypeId="2" ParentId="7525" CreationDate="2013-06-27T14:11:15.333" Score="1" Body="&lt;p&gt;I have found &lt;a href=&quot;http://www.salome-platform.org/&quot; rel=&quot;nofollow&quot;&gt;Salome&lt;/a&gt; to be very flexible, with a much better environment than gmsh. It has a much more professional feel IMHO. Moreover, it can generate also hexahedral meshes. It is really worth a try!&lt;/p&gt;&#xA;" OwnerUserId="4657" LastActivityDate="2013-06-27T14:11:15.333" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7828" PostTypeId="2" ParentId="7525" CreationDate="2013-06-27T14:33:39.867" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://mathema.tician.de/software/meshpy&quot; rel=&quot;nofollow&quot;&gt;MeshPy&lt;/a&gt; can be used for obtaining triangular and tetrahedral meshes. It provides an easy means of scripted mesh generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MeshPy provides Python interfaces to &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow&quot;&gt;Triangle&lt;/a&gt; and &lt;a href=&quot;http://tetgen.berlios.de/&quot; rel=&quot;nofollow&quot;&gt;TetGen&lt;/a&gt; mentioned by aterrel. &lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2013-06-27T14:33:39.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7829" PostTypeId="2" ParentId="7825" CreationDate="2013-06-27T15:49:07.483" Score="8" Body="&lt;p&gt;There is an extensive literature on numerical aspects of the implementation of the simplex method including methods for updating the factorization of the basis as well as techniques that maintain the inverse of the basis in product form.  Replacing a column of the basis matrix is a &quot;rank-one update&quot; since it can be written in terms of adding a matrix to the basis that has only one nonzero column and is thus of rank one.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The QR factorization is not typically used in implementations of the simplex method, since it is very slow in comparison with alternative approaches.  In practice an initial LU factorization of the basis is typically combined with rank one updates to the factorization for each iteration.  Because too many rank one updates can lead to numerical problems, the basis is typically refactored every so often (e.g. once every 50 iterations.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are efficient algorithms for rank one updates to the QR factorization.  A classic reference is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ams.org/journals/mcom/1976-30-136/S0025-5718-1976-0431641-8/S0025-5718-1976-0431641-8.pdf&quot;&gt;J. W. Daniel, W. B. Gragg, L. Kaufman, and G. Stewart, Reorthogonalization and Stable Algorithms for Updating the Gram-Schmidt QR Factorization.  Mathematics of Computation 30:&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-06-27T15:49:07.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7830" PostTypeId="2" ParentId="7824" CreationDate="2013-06-27T21:31:42.710" Score="6" Body="&lt;p&gt;These methods can&#xA;be roughly described in terms of two time-stepping methods,&#xA;denoted here by $G$ and $F$.  Both&#xA;$G$ and $F$ propagate an initial value&#xA;$U_n \approx u(t_n)$ by approximating the solution to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u(t) = u_0 + \int_0^t f(\tau,u(\tau)) \,d\tau&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from $t_n$ to $t_{n+1}$ (that is, $\dot{u} =&#xA;f(u,t)$).  For the methods to be efficient, it must be the case that&#xA;the $G$ propagator is computationally less expensive&#xA;than the $F$ propagator, and hence&#xA;$G$ is typically a low-order method.  Since the&#xA;overall accuracy of the methods is limited by the accuracy of the&#xA;$F$ propagator, $F$ is typically&#xA;higher-order and in addition may use a smaller time step than&#xA;$G$.  For these reasons, $G$ is&#xA;referred to as the coarse propagator and $F$ the fine&#xA;propagator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Parareal method begins by computing a first approximation&#xA;$U_{n+1}^0$ for $n = 0 \ldots N-1$ where $N$ is the&#xA;number of time steps, using the coarse propagator.  The Parareal method then proceeds iteratively, alternating between the&#xA;parallel computation of $F(t_{n+1},t_n,U_n^k)$ and an&#xA;update of the initial conditions at each processor of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;U_{n+1}^{k+1} = G(t_{n+1}, t_n, U_n^{k+1})&#xA;                   + F(t_{n+1}, t_n, U_n^k)&#xA;                   - G(t_{n+1}, t_n, U_n^{k})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $n = 0 \ldots N-1$.  That is, the fine propagator is used&#xA;to refine the solution in each time slice in parallel, while the&#xA;coarse propagator is used to propagate the refinements performed by&#xA;the fine propagator through time to later processors.  Note that at this point we haven't specified what the $G$ and $F$ propagators are: they could be, for example, Runge-Kutta schemes of varying order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PITA method is very similar to Parareal, but it keeps track of previous updates and only updates the initial condition on each processor in a manner reminiscent of Krylov subspace methods.  This allows PITA to solve linear second-order equations which Parareal cannot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PFASST method differs from the Parareal and PITA methods in two fundamental ways: first, it relies on the iterative Spectral Deferred Correction (SDC) time-stepping scheme, and second it incorporates Full Approximation Scheme corrections to the coarse propagator, and in fact PFASST can use a hierarchy of propagators (instead of just two).  Using SDC allows the time-parallel and SDC iterations are hybridized which relaxes the efficiency constraints of Parareal and PITA.  Using FAS corrections enables a lot of flexibility when constructing the coarse propagators of PFASST (making the coarse propagators as cheap as possible helps increase parallel efficiency).  Coarsening strategies include: time-coarsening (fewer SDC nodes), space-coarsening (for grid based PDEs), operator coarsening, and reduced physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this outlines the fundamentals, differences, and similarities between the algorithms.  Please see the references in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7714/whats-the-state-of-the-art-in-parallel-ode-methods&quot;&gt;this post&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding applications, the methods have been applied to a wide variety of equations (planetary orbits, Navier-Stokes, particle systems, chaotic systems, structural dynamics, atmospheric flows etc etc).  When applying time-parallelization to a given problem you should certainly validate the method in a manner appropriate for the problem being solved.&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-06-27T21:31:42.710" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7831" PostTypeId="2" ParentId="7822" CreationDate="2013-06-27T21:42:30.107" Score="4" Body="&lt;p&gt;The original paper on SDC is &quot;Spectral deferred correction methods for ordinary differential equations&quot; by Dutt, Greengard, and Rokhlin.  There are few papers by M Minion (see &lt;a href=&quot;http://amath.unc.edu/Minion/Publications&quot; rel=&quot;nofollow&quot;&gt;his old research page&lt;/a&gt;) that explain SDC as well.  Note that these methods belong to a larger class of defect/deferred correction strategies -- lots of literature out there, please see references in the above.  Finally, SDC methods stand on their own apart from PFASST, although they are a fundamental piece of the PFASST algorithm.&lt;/p&gt;&#xA;" OwnerUserId="486" LastActivityDate="2013-06-27T21:42:30.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7832" PostTypeId="2" ParentId="7776" CreationDate="2013-06-28T00:20:21.220" Score="0" Body="&lt;p&gt;Even for an indefinite real symmetric matrix, an exact symmetric expansion can be found in $O(kn^2)$ operations, where $n\times n$ matrix $M$ has rank $k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof is by induction and constructive.  Because our steps reduce $n$ either by 1 or 2, base cases $n=0$ (trivial sum) and $n=1$ (scalar) are adduced by inspection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 0:&lt;/strong&gt;  The simplest case is where a row and column &quot;pair&quot; (same index) are all zero.  After any needed row and column swaps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ M = \begin{bmatrix} 0 &amp;amp; 0 \\ 0 &amp;amp; C \end{bmatrix}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is evident that applying the induction hypothesis to $(n-1)\times (n-1)$ symmetric $C$, modified only by tacking a zero entry to the front of each vector used, produces the required expansion using precisely rank($M$) = rank($C$) terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 1:&lt;/strong&gt;  Otherwise, if some diagonal entry of $M$ is nonzero, then a Cholesky-like step as in Jack Poulson's Answer reduces the matrix size by 1.  After distributing out said nonzero diagonal entry, which may be positioned first by suitable row/column exchanges, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ M = \begin{bmatrix} 1 &amp;amp; u^T \\ u &amp;amp; B \end{bmatrix} = &#xA;\begin{bmatrix} 1 \\ u \end{bmatrix} \begin{bmatrix} 1 &amp;amp; u^T \end{bmatrix}&#xA;+ \begin{bmatrix} 0 &amp;amp; 0 \\ 0 &amp;amp; C \end{bmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C = B - uu^T$.  After applying Case 0 for the last term (which tacks zero on as leading entries), the nonzero leading entry in the first term makes it linearly independent from the terms in the symmetric expansion of the latter.  Thus rank($M$) is one more than rank($C$), and the combined expansion has the right number of terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 2:&lt;/strong&gt; Finally suppose all rows and columns contain a nonzero entry but have zero on the diagonal.  By distributing out the scalar and swapping rows and columns suitably, we can arrange:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ M = \begin{bmatrix} 0 &amp;amp; u^T &amp;amp; 1 \\ u &amp;amp; B &amp;amp; v \\ 1 &amp;amp; v^T &amp;amp; 0 \end{bmatrix} =&#xA;\begin{bmatrix} 0 \\ u \\ 1 \end{bmatrix} \begin{bmatrix} 1 &amp;amp; v^T &amp;amp; 0 \end{bmatrix}&#xA;+ \begin{bmatrix} 1 \\ v \\ 0 \end{bmatrix} \begin{bmatrix} 0 &amp;amp; u^T &amp;amp; 1 \end{bmatrix}&#xA;+ \begin{bmatrix} 0 &amp;amp; 0 &amp;amp; 0 \\ 0 &amp;amp; C &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 0 \end{bmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C = B - (uv^T + vu^T)$.  Since as noted in the Question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ uv^T + vu^T = \frac{1}{2} ((u+v)(u+v)^T - (u-v)(u-v)^T) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;applying the induction hypothesis to $C$ and tacking on first and last zero coordinates to each vector in its symmetric expansion gives terms that may be summed with the two above.  Since $uv^T$ has first row zero and last row not, and conversely for $vu^T$, these two terms are independent in combination with the terms from $C$ (which have both first and last rows zero).  Thus rank($M$) is two more than rank($C$), and again the combined expansion has the right number of terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Work in each reduction step is $O(n^2)$, so provided we halt (in exact arithmetic) when the reduced block $C$ is recognized to be zero, $O(kn^2)$ is the total complexity.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-06-28T00:20:21.220" CommentCount="0" CommunityOwnedDate="2013-06-28T00:20:21.220" ContentLicense="CC BY-SA 3.0" />
  <row Id="7833" PostTypeId="1" CreationDate="2013-06-28T00:26:21.103" Score="5" ViewCount="269" Body="&lt;p&gt;I need to evaluate the following derivative:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{\prod_i \xi_i!}\frac{1}{\prod_j \eta_j!}&#xA;\left.\frac{\partial^{\xi_1 + \cdots + \xi_m}}{\partial\alpha_1^{\xi_1}\ldots\partial\alpha_m^{\xi_m}}&#xA;\frac{\partial^{\eta_1 + \cdots + \eta_n}}{\partial\beta_1^{\eta_1}\ldots\partial\beta_n^{\eta_n}}&#xA;\exp\left(\sum_{ij} a_{ij} \alpha_i \beta_j\right)&#xA;\right|_{\alpha_1 = \cdots = \alpha_m = \beta_1 = \cdots = \beta_n = 0}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $\xi_i$ and $\eta_j$ are non-negative integers, with $i = 1...m$ and $j = 1...n$, and the $a_{ij}$ are non-negative real numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good numerical algorithm to do this? Is it efficient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(See also: &lt;a href=&quot;https://math.stackexchange.com/a/430925/10063&quot;&gt;https://math.stackexchange.com/a/430925/10063&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-07-01T21:19:57.470" Title="Numerical evaluation of partial derivatives" Tags="|algorithms|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7834" PostTypeId="2" ParentId="7823" CreationDate="2013-06-28T02:28:47.557" Score="4" Body="&lt;p&gt;I would try pushing this form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;V = VectorFucntioSpace(mesh, 'CG', 1, dim=2)&#xA;u = Function(V)&#xA;v = TestFunction(V)&#xA;sigma = Constant(0.1)&#xA;F = inner(grad(u), grad(v))*dx \&#xA;  + (inner(u, u)-1.0)*((u[0]+sigma*u[1])*v[0] + (u[1]-sigma*u[0])*v[1])*dx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;into &lt;code&gt;nonlinear-poisson&lt;/code&gt; demo. But note that this would yield trivial solution if you start from zero field with zero Dirichlet.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2013-06-28T02:28:47.557" CommentCount="18" ContentLicense="CC BY-SA 3.0" />
  <row Id="7835" PostTypeId="2" ParentId="7833" CreationDate="2013-06-28T03:37:35.900" Score="8" Body="&lt;p&gt;Do it by hand or, if you're lazy, do it symbolically with Maple or Mathematica. This is certainly going to be more accurate than trying to do high-order derivatives numerically.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-06-28T03:37:35.900" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7836" PostTypeId="1" CreationDate="2013-06-28T08:21:51.247" Score="3" ViewCount="1315" Body="&lt;p&gt;I'm trying to train an L2-regularized L1-hinge loss SVM using vowpal-wabbit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the following commands to train and test on the &lt;code&gt;splice&lt;/code&gt; dataset:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time vw --passes 10 -c --loss_function hinge -f model.txt -d train_vw.txt --l1 0 --l2 1&#xA;time vw -t -i model.txt -p out.txt -d test_vw.txt&#xA;perf -ACC -files test.labels out.txt -t 0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result is an accuracy of 48%, which is terrible! On splice, other SVMs, eg liblinear, give about 84% accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I remove the &lt;code&gt;--l2 1&lt;/code&gt; bit, or reduce it to &lt;code&gt;--l2 0.001&lt;/code&gt; or so, then I get ok-ish accuracy, at least 81% or so.  I'm expecting that the --l2 parameter is equivalent to the &lt;code&gt;C&lt;/code&gt; parameter of other SVMs.  Am I wrong?  If so, what is the relationship between the &lt;code&gt;--l2&lt;/code&gt; parameter and the standard &lt;code&gt;C&lt;/code&gt; SVM parameter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Full details of code used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat train_svmlight.txt | sed -e &quot;s/^+1 /1 |f /&quot; | sed -e &quot;s/^-1 /-1 |f /&quot; &amp;gt; train_vw.txt&#xA;cat test_svmlight.txt | sed -e &quot;s/^+1 /1 |f /&quot; | sed -e &quot;s/^-1 /-1 |f /&quot; &amp;gt; test_vw.txt&#xA;time vw --passes 10 -c --loss_function hinge -f model.txt -d train_vw.txt --l1 0 --l2 1&#xA;time vw -t -i model.txt -p out.txt -d test_vw.txt&#xA;cat test_vw.txt | cut -d ' ' -f 1 | sed -e 's/^-1/0/' &amp;gt; test.labels&#xA;perf -ACC -files test.labels out.txt -t 0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3613" LastActivityDate="2014-03-14T16:22:55.963" Title="How to train an L2-regularized L1 Hinge Loss SVM using vowpal wabbit?" Tags="|optimization|machine-learning|support-vector-machines|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7837" PostTypeId="1" CreationDate="2013-06-28T12:11:36.617" Score="7" ViewCount="2835" Body="&lt;p&gt;I'm looking for an efficient, multicore, library to do incomplete cholesky (possibly modified). Many ILU code exists, but I can't find much about IC except in PETSC or Pastix. Could some of you drop me any library name ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you !&#xA;Tom&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="3481" LastEditDate="2013-06-29T14:15:02.597" LastActivityDate="2014-12-22T05:14:52.143" Title="Sparse Incomplete Cholesky" Tags="|linear-algebra|sparse-matrix|matrix-factorization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7838" PostTypeId="2" ParentId="7777" CreationDate="2013-06-28T16:35:05.870" Score="4" Body="&lt;p&gt;The answer is YES, you can impose the Dirichlet boundary condition weakly on the inflow boundary $\partial \Omega^-$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say for the following advective equation:&#xA;$$\begin{cases}&#xA;u_t + \nabla \cdot (\mathbf{v}  u) = 0 \quad \text{in }\Omega,&#xA;\\&#xA;u= g \quad \text{on }\partial \Omega^-,&#xA;\end{cases}&#xA;$$&#xA;where $\partial \Omega^-$ is the inflow boundary: $\mathbf{v}\cdot \mathbf{n}&amp;lt; 0$ for outward normal $\mathbf{n}$, this means the flow field $\mathbf{v}$ is flowing into the domain here on this inflow boundary. We can assume the flow is incompressible as well ($\nabla \cdot \mathbf{v} = 0$). Now common FV reads:&#xA;$$&#xA;\int_K u_t(x,t)\,dx + \int_{\partial K} \mathbf{v}(x)\cdot \mathbf{n}(x) u(x,t)\,dS = 0 \quad \text{for any } t,&#xA;$$ &#xA;where $K$ is the control volume (interval in your case). Now decomposing each control volume's boundary into two parts:&#xA;$$&#xA;\int_K u_t \,dx + \int_{\partial K\backslash \partial \Omega^-} \mathbf{v} \cdot \mathbf{n}  \,u \,dS + \color{blue}{\int_{\partial K\cap\partial \Omega^-} \mathbf{v} \cdot \mathbf{n}\,  u \,dS} = 0&#xA;$$&#xA;(Thanks to Jan) Just moving the blue term to the right and replacing $u$ with its boundary data will do:&#xA;$$&#xA;\int_K u_t \,dx + \int_{\partial K\backslash \partial \Omega^-} \mathbf{v} \cdot \mathbf{n}  \,u \,dS  = -\color{blue}{\int_{\partial K\cap\partial \Omega^-} \mathbf{v} \cdot \mathbf{n}\,  g\,dS} &#xA;$$&#xA;&lt;s&gt;Blue terms are equal to each other&lt;/s&gt;, but after you solved the equation you will find that the Dirichlet boundary condition is not precisely satisfied even when the data $g$ is a piecewise constant, for we only impose it weakly and treat the inflow Dirichlet boundary as if it is Neumann. I saw this approach a long time ago for Discontinuous Galerkin formulation (essentially higher order FVM) for convection-diffusion equation in Ern and Guermond's book.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2013-06-30T15:38:30.227" LastActivityDate="2013-06-30T15:38:30.227" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7839" PostTypeId="2" ParentId="7819" CreationDate="2013-06-28T19:04:08.927" Score="18" Body="&lt;p&gt;Gradiant descent and the conjugate gradient method are both algorithms for minimizing nonlinear functions, that is, functions like the Rosenbrock function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;f(x_1,x_2) = (1-x_1)^2 + 100(x_2 - x_1^2)^2&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or a multivariate quadratic function (in this case with a symmetric quadratic term)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;f(x) = \frac{1}{2} x^T A^T A x - b^T A x.&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both algorithms are also iterative and search-direction based. For the rest of this post, $x$, and $d$ will be vectors of length $n$; $f(x)$ and $\alpha$ are scalars, and superscripts denote iteration index. Gradient descent and the conjugate gradient method can be used to find the value $x^*$ that solves&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\min f(x)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both methods start from an initial guess, $x^0$, and then compute the next iterate using a function of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;x^{i+1} = x^i + \alpha^i d^i.&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In words, the next value of $x$ is found by starting at the current location $x^i$, and moving in the search direction $d^i$ for some distance $\alpha^i$. In both methods, the distance to move may be found by a line search (minimize $f(x^i + \alpha^i d^i)$ over $\alpha_i$). Other criteria may also be applied. Where the two methods differ is in their choice of $d^i$. For the gradient method, $d^i = -\nabla f(x^i)$. For the conjugate gradient method, the Grahm-Schmidt procedure is used to orthogonalize the gradient vectors. In particular, $d^0 = -\nabla f(x^0)$, but then $d^1$ is equal $-\nabla f(x^1)$ minus that vector's projection onto $d^0$ such that $(d^1)^Td^0 = 0$. Each subsequent gradient vector is orthogonalized against all the previous ones, which leads to very nice properties for the quadratic function above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above quadratic function (and related formulations) is also where the discussion of solving $Ax = b$ using the conjugate gradient method comes from, since the minimum of that $f(x)$ is achieved at the point $x$ where $Ax = b$. &lt;/p&gt;&#xA;" OwnerUserId="1642" LastActivityDate="2013-06-28T19:04:08.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7840" PostTypeId="2" ParentId="7436" CreationDate="2013-06-29T01:15:10.943" Score="2" Body="&lt;p&gt;If $\mathbf{E}$ is know, we want to find $-\nabla\phi = \mathbf{E}$, then like you wrote in the first equation, but rather we have all $x$, $y$, $z$ second partial derivatives. The following expression is obtained by taking divergence on both sides:&#xA;$$&#xA;-\Delta \phi = -\left(\frac{\partial^2 \phi}{\partial x^2} + \frac{\partial^2 \phi}{\partial y^2} + \frac{\partial^2 \phi}{\partial z^2}\right) = \nabla \cdot \mathbf{E}.\tag{1}&#xA;$$&#xA;The equation you wrote pre-suppose that $\phi$ and $\mathbf{E}$ are both separable, which is not true for general domain (please see &lt;a href=&quot;https://math.stackexchange.com/questions/420868/what-kinds-of-pde-cant-be-solved-by-separation-of-variables/421568&quot;&gt;my answer here&lt;/a&gt; to see why the shape of the domain is related to the separability).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now equation (1) is Poisson equation, for a bounded domain of interest $\Omega$, we can formulate a Neumann boundary value problem:&#xA;$$&#xA;\left\{\begin{aligned}&#xA;-\Delta \phi &amp;amp;= \nabla \cdot \mathbf{E} \quad \text{in }\Omega,&#xA;\\&#xA;\nabla \phi\cdot \mathbf{n}&amp;amp;= -\mathbf{E} \cdot \mathbf{n}\quad \text{on }\partial \Omega.&#xA;\end{aligned}\tag{2}\right.&#xA;$$&#xA;This equation (2) is well-posed when certain compatibility condition is assumed, then you can use whichever numerical method that suits your needs (finite element, finite differences, finite volume, Fourier, layer potential, etc).&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-06-29T01:15:10.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7841" PostTypeId="2" ParentId="7098" CreationDate="2013-06-29T01:36:47.163" Score="1" Body="&lt;p&gt;It is possible to draw a &lt;code&gt;streamslice&lt;/code&gt; slicing using non-planar or at arbitrary angle. You can use &lt;code&gt;rotate&lt;/code&gt; command to rotate your slicing plane(s). Your &lt;code&gt;plot_field&lt;/code&gt; object can be modified by adding several lines&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;slicing = surf(linspace(-1,1,10),linspace(-1,1,10),zeros(10));&#xA;% third input: rotating angle&#xA;% second input: the vector to which you are rotating your data with respect&#xA;rotate(slicing,[0,0,0],45);&#xA;xs = get(slicing,'XData');&#xA;ys = get(slicing,'YData');&#xA;zs = get(slicing,'ZData');&#xA;plot_field = streamslice(x,y,z,Bx,By,Bz,xs,ys,zs);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also &lt;code&gt;contourslice&lt;/code&gt; is another my own favorite to visualize electromagnetic field: &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/contourslice.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.com/help/matlab/ref/contourslice.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you know how to use VTK, then you can export your 3D vector data to a vtk file using: &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/34310-matlab-3d-vectors-to-paraview-via-binary-vtk-file&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/34310-matlab-3d-vectors-to-paraview-via-binary-vtk-file&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you could use software like Paraview to visualize vector field as you like, for example, slicing at $45^{\circ}$ degree of the box like you said (I used the data of my own from a dielectric with a cavity Maxwell pde):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ibPc8.png&quot; alt=&quot;slice&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-06-29T01:36:47.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7842" PostTypeId="2" ParentId="7799" CreationDate="2013-06-29T02:45:10.907" Score="2" Body="&lt;p&gt;Nvidia Cuda and AMD APP SDK's contain sample code which may help you get started. I've successfully carried out some work on this in the past, and the performance benefits were about 32x for a simple implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the references listed &lt;a href=&quot;http://www.scribd.com/doc/37248935/Thesis&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; may be beneficial, although this was written when GPGPU computing with OpenCL was still in its infancy. There are also some performance metrics compared to CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sheer speed in pseudo-random number generation you can take a look at the &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MTGP/&quot; rel=&quot;nofollow&quot;&gt;Mersenne Twister for GPU's (MTGP)&lt;/a&gt;. This implementation exists solely in CUDA, but it may be an easier foray into the use of GPU's than via OpenCL. AMD's SDK Monte Carlo sample contains a variant of the algorithm, so you are not tied to using solely Cuda. It would be best to decide on your requirements first, then choose the most appropriate language to carry out your task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could consider using these in their own right, or connecting with MATLAB via the MEX file interface.&lt;/p&gt;&#xA;" OwnerUserId="4664" LastEditorUserId="4664" LastEditDate="2013-06-29T08:10:38.930" LastActivityDate="2013-06-29T08:10:38.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7844" PostTypeId="2" ParentId="7837" CreationDate="2013-06-29T14:14:13.717" Score="3" Body="&lt;p&gt;First of all, if you have an incomplete LU factorization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A \approx LU$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can write the upper triangular factor $U$ as $U = DR$ where $D$ is diagonal and $R$ is unit right-triangular. If the matrix $A$ is symmetric positive definite, then $R = L^\top$. You can then easily modify the incomplete LU factorization to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A = LDL^\top$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or to an honest incomplete Cholesky factorization. The matrices you do find should be transposes of each other to within machine precision. While this doesn't answer your question of finding a library that specifically does parallel incomplete Cholesky, it will, with minor modifications, get you what you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.cs.odu.edu/~pothen/Software/Euclid/&quot; rel=&quot;nofollow&quot;&gt;Euclid library&lt;/a&gt; is pretty popular for parallel ILU; PETSc interfaces to it. However, near as I can tell, it only does ILU and not IC, hence my digression above. I was going to recommend that you look at &lt;a href=&quot;http://acts.nersc.gov/hypre&quot; rel=&quot;nofollow&quot;&gt;Hypre&lt;/a&gt;, but upon looking through their user manual, they tell you to just use Euclid instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other one that comes to mind is &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/software/pARMS/index.html&quot; rel=&quot;nofollow&quot;&gt;pARMS&lt;/a&gt; (parallel algebraic recursive multilevel solver). pARMS isn't exactly for pure ILU decompositions; it works in a multi-level framework not unlike algebraic multigrid. (There's a good explanation in &lt;a href=&quot;http://books.google.com/books?isbn=0898718007&quot; rel=&quot;nofollow&quot;&gt;Yousef Saad's book&lt;/a&gt;, which is also a good reference for how parallel ILU works.) Nonetheless, it's parallel and you may be able to set the number of levels to 0 to recover the usual ILU factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, &lt;a href=&quot;http://trilinos.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;Trilinos&lt;/a&gt; has a parallel ILU preconditioner. However, it only computes ILU factorizations locally to each processor and uses some overlap to guarantee that the method is scalable. From what I gather, this isn't exactly what you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-06-29T14:14:13.717" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7845" PostTypeId="1" CreationDate="2013-06-29T17:53:51.087" Score="29" ViewCount="5967" Body="&lt;p&gt;When going from the strong form of a PDE to the FEM form it seems one should always do this by first stating the variational form. To do this you multiply the strong form by an element in some (Sobolev) space and integrate over your region. This I can accept. What I don't understand is why one also has to use Green's formula (one or several times).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've mostly been working with Poisson's equation, so if we take that (with homogenous Dirichlet boundary conditions) as an example, i e&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;-\nabla^2u &amp;amp;= f,\quad u\in\Omega \\&#xA;u &amp;amp;= 0, \quad u\in\partial\Omega&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then it is claimed that the correct way to form the variational form is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;\int_\Omega fv\,\mathrm{d}\vec{x} &amp;amp;= -\int_\Omega\nabla^2 uv\,\mathrm{d}\vec{x} \\&#xA;&amp;amp;=\int_\Omega\nabla u\cdot\nabla v\,\mathrm{d}\vec{x} - \int_{\partial\Omega}\vec{n}\cdot\nabla u v\,\mathrm{d}\vec{s} \\&#xA;&amp;amp;=\int_\Omega\nabla u\cdot\nabla v\,\mathrm{d}\vec{x}.&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what stops me from using the expression on the first line, isn't that also a variational form that can be used to get a FEM form? Isn't it corresponding to the bilinear and linear forms $b(u,v)=(\nabla^2 u, v)$ and $l(v)=(f, v)$? Is the problem here that if I use linear basis functions (shape functions) then I'll be in trouble because my stiffness matrix will be the null matrix (not invertible)? But what if I use non-linear shape functions? Do I still have to use Green's formula? If I don't have to: is it advisable? If I don't, do I then have a variational-but-not-weak formulation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's say that I have a PDE with higher order derivatives, does that mean that there are many possible variational forms, depending on how I use Green's formula? And they all lead to (different) FEM approximations?&lt;/p&gt;&#xA;" OwnerUserId="4669" LastEditorUserId="123" LastEditDate="2013-06-30T16:34:42.407" LastActivityDate="2013-07-02T07:58:43.277" Title="What is the purpose of using integration by parts in deriving a weak form for FEM discretization?" Tags="|pde|finite-element|elliptic-pde|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7846" PostTypeId="1" AcceptedAnswerId="8454" CreationDate="2013-06-29T18:57:14.743" Score="6" ViewCount="889" Body="&lt;p&gt;Can you all give me (at least) one example about &lt;a href=&quot;https://math.stackexchange.com/questions/423985/what-is-convection-dominated-pde-problems&quot; title=&quot;a question about this problem has been solved in MathExchange&quot;&gt;convection-dominated problem&lt;/a&gt; in order that I can test it (them) on FreeFEM++. If possible, please give me specific examples (it/they contain(s) full equations, boundary condition, initial condition, value of parameters, code in FreeFEM++,...)&lt;/p&gt;&#xA;" OwnerUserId="4670" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-11-27T13:55:54.767" Title="Need an example of convection-dominated problem to test on FreeFEM++" Tags="|finite-element|numerical-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7847" PostTypeId="2" ParentId="7845" CreationDate="2013-06-29T19:20:47.960" Score="18" Body="&lt;p&gt;Nothing stops you from doing that technically, but when you integrate by parts you get more flexibility with the solution space in that they need not have $H^2$ regularity (required for the non I.B.P formulation). The linear elements you suggest generally have enforced continuity between elements, and so could not be in $H^2$. The I.B.P formulation furthermore is symmetric, which has some of its own advantages as well.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2013-06-29T19:36:26.843" LastActivityDate="2013-06-29T19:36:26.843" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7848" PostTypeId="1" CreationDate="2013-06-29T19:53:44.297" Score="6" ViewCount="980" Body="&lt;p&gt;I need to solve the following system of differential equations. &#xA;&lt;img src=&quot;https://i.stack.imgur.com/VhwSB.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I have the solutions for $n_f$ and $v$, I need to find and plot $J=-e_\cdot n_{f} \cdot v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a code in matlab with all ODEs like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      function systemSolve&#xA;      clc&#xA;&#xA;&#xA;      tr=50e-12;        % Recombination lifetime&#xA;      n0=1e11;         % Density of free carriers (Recordar que es 1e17 cm-3)&#xA;      tc=2e-12;         % Trapping time&#xA;      ts=30e-15;        % Carrier scattering time&#xA;      m=0.067*9.11e-31; % effective mass GaAs&#xA;      ev=8.854e-12;     % permitivity  &#xA;      n=900;            % factor geometry&#xA;      q=1.6e-19         % electron charge&#xA;&#xA;      de=50e-15         %  delta t&#xA;&#xA;&#xA;      timeRange=[0 0.1e-12]; &#xA;      initialConditionVector=[0;1e-15;1e-15;1e-15];&#xA;      [t,x]=ode45(@xprime,timeRange,initialConditionVector);&#xA;      figure(1),plot(t,x(:,1))&#xA;&#xA;      J=q*x(:,3).*x(:,1);&#xA;      figure(2),plot(t,J)&#xA;&#xA;      function f=xprime(t,x)&#xA;      f=[x(4); ...&#xA;        -(x(2)/tr)+(x(3)*q*x(1)); ...&#xA;        -(x(3)/tc)+(n0*(exp((t/de)^2))); ...&#xA;        -((1/ts)*x(4))-(((x(3)*q^2)/m*ev)*x(1)/n)+(q*x(2)/(m*n*ev*tr))];&#xA;      end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suppose that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_1=v$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_2=P_{sc}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_3=n_f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_4=x'_1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect to find a current pulse like figure 1 but I get a exponential solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/P9Pnd.jpg&quot; alt=&quot;Figure 1&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is wrong in this code and can you suggest me how i can solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="4671" LastEditorUserId="4671" LastEditDate="2013-07-01T16:49:17.347" LastActivityDate="2013-07-01T16:49:17.347" Title="Code to numerically integrate a system of first-order ODEs" Tags="|matlab|ode|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7849" PostTypeId="1" CreationDate="2013-06-29T20:10:08.620" Score="2" ViewCount="165" Body="&lt;p&gt;There are several variants of the simplex method known, which differ by the choice of entering and leaving variables. But neither have I found a reference, which variants are used in which applications, nor have I found an attempt to compare the run-time of those methods. I suspect that a fair share of these variants have remained academic during the past years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody provide a reference or an experimental comparison of these methods?&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2013-06-29T20:43:42.153" Title="Which variants of the simplex method are actually used in applications" Tags="|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7850" PostTypeId="2" ParentId="7848" CreationDate="2013-06-29T20:40:38.407" Score="5" Body="&lt;p&gt;You can use MATLAB's &lt;code&gt;ode45&lt;/code&gt;, please refer to the first example here: &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ode23.html&quot;&gt;Solve nonstiff differential equations; low order method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I use $y$ instead of $x$. If you don't know how to write a $\mathtt{func.m}$ script and save it in your path, you can use function handle as well, in your case:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% output of function handle in vector form&#xA;rigid = @(t,y)[x(4); -x(2)+x(3);  ??? ; ???];&#xA;options = odeset('RelTol',1e-6,'AbsTol',[1e-6 1e-6 1e-6 1e-6]);&#xA;[t,y] = ode45(rigid,[0 T],[y10 y20 y30 y40],options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;???&lt;/code&gt; is left for you to fill up, you can tweak the &lt;code&gt;options&lt;/code&gt; if you read the document of the link above, &lt;code&gt;y10&lt;/code&gt; through &lt;code&gt;y40&lt;/code&gt; are the initial values. Lastly &lt;code&gt;J = y(:,1).*y(:,3)&lt;/code&gt; then &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/plot.html&quot;&gt;plot&lt;/a&gt; $J$ against $t$.&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-06-29T20:40:38.407" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="7851" PostTypeId="2" ParentId="7849" CreationDate="2013-06-29T20:43:42.153" Score="5" Body="&lt;p&gt;Unfortunately, the vendors of linear programming software have often been unwilling to tell us about the particular tricks that they use in implementing the simplex method, so there aren't many published articles that will give you these kinds of details. Robert Bixby's 2002 paper, &lt;a href=&quot;http://or.journal.informs.org/content/50/1/3.short&quot;&gt;Solving Real-World Linear Programs: A Decade and More of Progress&lt;/a&gt; is a good place to start your research on this subject.  He has given more recent presentations on this topic, but I haven't seen that he's published a follow up paper.      &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-06-29T20:43:42.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7852" PostTypeId="2" ParentId="7845" CreationDate="2013-06-30T04:39:56.913" Score="19" Body="&lt;h2&gt;Short answer:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;No, you don't have to do integration for certain FEMs. But in your case, you have to do that.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Long answer:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Let's say $u_h$ is the finite element solution. If you choose piecewise linear polynomial as your basis, then taking $\Delta$ on it will give you an order 1 distribution (think taking derivative on a Heaviside step function), and the integration of $-\Delta u_h\in H^{-1}$ multiplying with $v$ will only make sense when you take it as a duality pair rather than an $L^2$-inner product. You will neither get a null matrix, the Riesz representation theorem says that there is an element in $\varphi_{-\Delta u_h} \in H^1_0$ can characterize the duality pair by the inner product in $H^1$:&#xA;$$&#xA;\langle-\Delta u_h ,v \rangle_{H^{-1},H^1_0} = \underbrace{\int_{\Omega}\nabla  \varphi_{-\Delta u_h} \cdot \nabla v}_{\text{inner product in }H^1}.&#xA;$$&#xA;Integrating by parts element by element for $u_h$ will shed a light on this duality pair: for $T$ an element in this triangulation&#xA;$$&#xA;\int_{\Omega}\nabla u_h \cdot \nabla v = -\sum_{T}\left(\int_{T} \Delta u_h\,v + \int_{\partial T}\frac{\partial u_h}{\partial n}v\,dS\right),&#xA;$$&#xA;this tells you that $-\Delta u_h$ should include inter-element flux jump in its duality pair representation, notice the integration on the boundary of each element is also a duality pair between $H^{1/2}$ and $H^{-1/2}$. Even if you use quadratic basis, which has a non-vanishing $\Delta$ on each element, you still can't write $(\Delta u, v)$ as an inner product, because of this inter-element flux jump's presence.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Integration by parts can be traced back to the Sobolev theory for elliptic pde using smooth function, where the $W^{k,p}$-spaces are all closure of smooth functions under the $W^{k,p}$ type of integral norm. Then people say what is the minimum regularity here that we can perform inner product.  Also bearing in mind that an $H^1$-regular weak solution under certain condition &lt;strong&gt;&lt;em&gt;is&lt;/em&gt;&lt;/strong&gt;  the $H^2$-strong solution (elliptic regularity). But piecewise continuous linear polynomial is not $H^2$, from this point of view, it doesn't make any sense to take inner product using $\Delta u_h$ either.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For certain FEMs, you don't have to do integration by parts. For example, Least-square finite element. Write the second order pde as a first order system:&#xA;$$&#xA;\begin{cases}&#xA;\boldsymbol{\sigma} = -\nabla u,&#xA;\\&#xA;\nabla \cdot \boldsymbol{\sigma} = f.&#xA;\end{cases}&#xA;$$&#xA;Then you wanna minimize the least-square functional:&#xA;$$&#xA;\mathcal{J}(v) = \|\boldsymbol{\sigma}  + \nabla u\|_{L^2{\Omega}}^2 +  \|\nabla \cdot \boldsymbol{\sigma} - f\|_{L^2{\Omega}}^2,&#xA;$$&#xA;bearing the same spirit with Ritz-Galerkin functional, the finite element formulation of minimizing above functional in a finite element space does not require integration by parts.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-06-30T04:39:56.913" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7855" PostTypeId="1" CreationDate="2013-06-30T12:20:18.020" Score="1" ViewCount="2068" Body="&lt;p&gt;I need to build a system in Simulink that solves a PDE, but I can't find any literature or books where it is described how to do it (especially any stuff according to modeling PDE in Simulink).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please give some advice where I can find such literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks beforehand!&lt;/p&gt;&#xA;" OwnerUserId="4676" LastEditorUserId="4428" LastEditDate="2013-07-04T17:36:43.123" LastActivityDate="2013-07-04T17:36:43.123" Title="Simulink PDE (MATLAB)" Tags="|pde|matlab|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7856" PostTypeId="2" ParentId="7777" CreationDate="2013-06-30T15:44:39.250" Score="2" Body="&lt;p&gt;In general this is not possible. For pure advection it is - as Shuhao Cao has pointed out - for problems with diffusion it is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing the integration on a control volume and applying the &lt;a href=&quot;http://en.wikipedia.org/wiki/Divergence_theorem#Multiple_dimensions&quot; rel=&quot;nofollow&quot;&gt;divergence theorem&lt;/a&gt;, the Laplacian on the interior is turned into the normal derivative taken along the boundary:&#xA;$$&#xA;   \int_\Omega \nabla \cdot \nabla u dx = \oint _{\partial \Omega} \nabla u \cdot n ds&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no possibility to incorporate Dirichlet data here, since you cannot get from $u=g$ on $\partial \Omega_{-}$ to the normal derivative $\frac{\partial u}{\partial n}$ on $\partial \Omega_{-}$. (In 1D this would mean to take a derivative of a scalar...)&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-07-01T05:33:44.327" LastActivityDate="2013-07-01T05:33:44.327" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7857" PostTypeId="1" CreationDate="2013-06-30T15:46:22.453" Score="2" ViewCount="68" Body="&lt;p&gt;In discussing smoothing filters,&#xA;&lt;a href=&quot;http://apps.nrbook.com/empanel/index.html?pg=772&quot; rel=&quot;nofollow&quot;&gt;Numerical Recipes p. 772&lt;/a&gt; says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... irregularly sampled data, where the values $f_i$ are not uniformly spaced ...&lt;br&gt;&#xA;  one can simply pretend that the data points &lt;em&gt;are&lt;/em&gt; equally spaced ...&#xA;  a rough criterion:&lt;br&gt;&#xA;  If the change in $f$ across the full width of the $N$ point window is&#xA;  less than $\sqrt{N/2}$ times the measurement noise on a single point,&#xA;  then the cheap method can be used.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Where does this come from ?&#xA;It looks odd:&#xA;for $f(x) = \text{sin} ( 2 \pi f x )$ on [$-N \dots N]$,&lt;br&gt;&#xA;change in $f = 2 &amp;lt; \sqrt{N/2}\ \sigma$ would require bigger $N$ for smaller $\sigma$ . Or have I misunderstood ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, does this work for smoothing data on non-uniformly spaced 2d squares, 3d cubes ... ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added: &lt;a href=&quot;https://gist.github.com/denis-bz/5957279&quot; rel=&quot;nofollow&quot;&gt;https://gist.github.com/denis-bz/5957279&lt;/a&gt; is a short Python program&#xA;that tries to elucidate this, for quantization noise&#xA;(sample at integers +- random-uniform( -1/2, 1/2 )).&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2013-07-09T13:26:12.300" LastActivityDate="2013-07-09T13:26:12.300" Title="Error in treating non-uniformly spaced data as uniform" Tags="|data-analysis|signal-processing|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7858" PostTypeId="2" ParentId="7820" CreationDate="2013-06-30T16:52:49.240" Score="4" Body="&lt;p&gt;It would be helpful if you would write down the full system of PDEs you are considering, since (to me, at least) there is still some ambiguity.  It would also be helpful if you would describe the discretization more precisely.  But I will provide a rough answer based on the information given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First we should distinguish between two concepts: (1) the strict CFL condition that is a necessary condition for convergence; and (2) the restriction on the CFL number that is both necessary and sufficient for stability.  I think you are interested in the latter, which is the one with more practical significance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your thinking is correct -- for a typical explicit discretization of any kind of transport PDE, the restriction on the CFL number that ensures stability will  have the form&#xA;$$\Delta t \le c \frac{\Delta x}{U_{max}}$$&#xA;where $U_{max}$ is the largest characteristic speed and $c$ is a constant depending on the numerical discretization.  The reason the stability condition scales this way is that the eigenvalues of the spatial discretization scale this way.  For the convective term you mention, $U_{max}=2\max(U)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, determining $U_{max}$ can be complicated if the solution is not smooth or the flux is non-convex.  But for smooth solutions and convex flux, $U_{max}$ is simply the largest eigenvalue of the jacobian of the flux function.  To learn more, get a good book on hyperbolic PDEs. I would recommend either the second or the third book on &lt;a href=&quot;http://faculty.washington.edu/rjl/booksnotes.html&quot; rel=&quot;nofollow&quot;&gt;this page&lt;/a&gt;.  For a very simple introduction, look at Chapter 10 of the first book on that page.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-06-30T16:52:49.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7859" PostTypeId="1" AcceptedAnswerId="7860" CreationDate="2013-06-30T18:09:23.640" Score="3" ViewCount="111" Body="&lt;p&gt;For a given 3D domain, two unstructured tetrahedral meshes M_1 and M_2 are generated. The average edge length in M_1 is double the average edge length of M_2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there estimates on the ratio of the number of tetrahedra in both meshes?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-06-30T23:30:39.860" Title="number of tetrahedra in unstructured mesh when edge length is halved" Tags="|unstructured-mesh|mesh|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7860" PostTypeId="2" ParentId="7859" CreationDate="2013-06-30T23:30:39.860" Score="3" Body="&lt;p&gt;If the edge lengths decrease by a factor $k$, the number of tetrahedra will typically increase by between $k^2$ and $k^3$ depending on the adaptivity of the mesh.  A purely uniform mesh will see $k^3$, just as a uniform grid of length $1/n$ has $O(n^3)$ cells.  A highly adaptive mesh with full resolution near a surface will increase by roughly $k^2$, since most of the elements will occur in a band near the surface.  It is also possible to see only a $k$ factor increase if the refinement is near a curve, but this is less common.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-06-30T23:30:39.860" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7861" PostTypeId="1" AcceptedAnswerId="7870" CreationDate="2013-07-01T07:43:10.943" Score="1" ViewCount="170" Body="&lt;p&gt;I'm writing some image processing software to detect bubbles in oil at work. One of the image filters I need (Perona Malik) seems like it will take a long time to compute. (It involves solving some partial differential equations numerically). One possible solution I'm thinking of investigating is using a super computer cluster. I've never had any experience with Beowulf clusters or the like ... Can anyone suggest some technologies I should consider. Any pros and cons of the different options would be useful to know as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chris&lt;/p&gt;&#xA;" OwnerUserId="4680" LastEditorUserId="3725" LastEditDate="2014-07-05T16:22:50.733" LastActivityDate="2014-07-05T16:22:50.733" Title="Looking for super computer technology suggestions" Tags="|mpi|cuda|opencl|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="7862" PostTypeId="2" ParentId="7833" CreationDate="2013-07-01T21:19:57.470" Score="3" Body="&lt;p&gt;Here's the easy part.  Define&#xA;$$A = \prod_i A_i = \prod_i \frac{1}{\xi_i!} \left(\frac{\partial}{\partial \alpha_i} \right)^{\xi_i}$$&#xA;$$B = \prod_j B_j = \prod_j \frac{1}{\eta_j!} \left(\frac{\partial}{\partial \beta_j} \right)^{\eta_j}$$&#xA;$$f = \exp \sum_{ij} a_{ij} \alpha_i \beta_j = \prod_{ij} f_{ij} = \prod_{ij} e^{a_{ij} \alpha_i \beta_j}$$&#xA;We have&#xA;$$ \frac{\partial}{\partial \alpha_k} f_{ij} = \begin{cases} a_{ij} \beta_j f_{ij} &amp;amp; i=k \\ 0 &amp;amp; i \ne k \end{cases}$$&#xA;$$ A_i f = \frac{f}{\xi_i!}\left(\sum_j a_{ij} \beta_j \right)^{\xi_i}$$&#xA;$$ A f = f \prod_i \frac{\left(\sum_j a_{ij} \beta_j \right)^{\xi_i}}{\xi_i!}$$&#xA;Unfortunately, the next step is harder. :)&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-07-01T21:19:57.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7863" PostTypeId="2" ParentId="7845" CreationDate="2013-07-02T07:58:43.277" Score="15" Body="&lt;p&gt;Excellent answers already on this page, but there is still a (small) missing point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The OP asked: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Now, let's say that I have a PDE with higher order derivatives, does&#xA;  that mean that there are many possible variational forms, depending on&#xA;  how I use Green's formula? And they all lead to (different) FEM&#xA;  approximations?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Integrating by parts (in the &lt;em&gt;correct&lt;/em&gt; way) is important when you have Neumann type of boundary conditions. In fact it is by the i.b.p that you take into account the Neumann b.c in your variational formulation. The form of the Neumann b.c depends on how you integrate by parts, cf. this &lt;a href=&quot;https://scicomp.stackexchange.com/a/4903/1744&quot;&gt;answer&lt;/a&gt; on integration by parts in linear elasticity. So even for second order elliptic PDE's, integration by parts has to be performed in a given way, in order to recover a variational formulation valid for Neumann or mixed boundary conditions. (And this of course irrespective of the fact that you discretize by FEM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In mathematical physics, where the Neumann b.c. have a well defined meaning (heat flux, stress...), integration by parts is important in order to maintain the correct interpretation of the results. Even for homogeneous Dirichlet conditions and FEM this is true, since if we use a Lagrange multiplier method to impose the b.c.'s, the multipliers become physical quantities, like concentrated fluxes or forces.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-07-02T07:58:43.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7864" PostTypeId="2" ParentId="6933" CreationDate="2013-07-02T15:58:24.927" Score="5" Body="&lt;p&gt;I'm not completely clear on what you're asking here, but it looks like a solution to your problem might be the new Neighborhood Collective in MPI-3.0 Chapter 7.6 (you can find the PDF &lt;a href=&quot;http://www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm far from an expert in these things, but the basic idea is that you attach a topology to your MPI_Communicator and get a new communicator where you can use sparse collective communications that don't have to involve all processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use this, you'll have to make sure you have an implementation that supports the new features of MPI. I know that the latest version of &lt;a href=&quot;http://www.mpich.org&quot; rel=&quot;noreferrer&quot;&gt;MPICH&lt;/a&gt; (3.0.4) supports them. Last I heard, &lt;a href=&quot;http://www.open-mpi.org&quot; rel=&quot;noreferrer&quot;&gt;Open MPI&lt;/a&gt; hadn't finished their implementation yet, but I'm sure it's coming.&lt;/p&gt;&#xA;" OwnerUserId="4658" LastActivityDate="2013-07-02T15:58:24.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7865" PostTypeId="1" CreationDate="2013-07-02T17:15:50.903" Score="2" ViewCount="2413" Body="&lt;p&gt;How to divide a surface to subdomains, which can be recognized in msh file? Avoiding duplicated nodes is necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general I have &lt;a href=&quot;http://pastebin.com/8D6LGiaM&quot; rel=&quot;nofollow&quot;&gt;geo file&lt;/a&gt; which contains definition of smaller and bigger region, now I would like to assign different physical surfaces (tags) to both - but right now is impossible due to only one plane surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to assign tag to cells directly in mesh module (but not manually in file)?&lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2013-07-03T09:30:18.510" Title="Gmsh - dividing a plane surface" Tags="|mesh|gmsh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7866" PostTypeId="1" CreationDate="2013-07-02T17:43:42.293" Score="3" ViewCount="3190" Body="&lt;p&gt;Based on the variational principle, one might expect that the ground state energy of a density functional theory (DFT) calculation will decrease as the basis set size increases. (As I understand it, this is the case in Hartree-Fock.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically speaking, is it in fact true that the DFT energy decreases with basis set size? If so how would you prove it? It seems like the variational principle might not directly apply since DFT, unlike Hartree-Fock, does not explicitly construct a many-body wavefunction.&lt;/p&gt;&#xA;" OwnerUserId="3887" LastEditorUserId="3887" LastEditDate="2013-07-03T14:31:52.423" LastActivityDate="2014-05-26T14:54:14.773" Title="Does energy decrease with basis set size in density functional theory?" Tags="|computational-chemistry|convergence|basis-set|density-functional-theory|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7868" PostTypeId="1" AcceptedAnswerId="7869" CreationDate="2013-07-02T19:07:12.577" Score="4" ViewCount="3791" Body="&lt;p&gt;I would like to find the optimal set $ \{ x_i \} $ given $ L $ and $ \{ a_i \} $ that minimizes the problem below.  My first thought was to use linear programming.  Is there a transformation that makes it possible, or do I need a more general optimization technique?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \min_{x_i} \left[ 2\sum_i | x_i |  + \max_i | x_i + a_i | \right]$$&#xA;$$ \mathrm{s.t.} \sum_i (x_i + a_i) \le L$$&lt;/p&gt;&#xA;" OwnerUserId="4685" LastActivityDate="2013-07-02T19:45:24.717" Title="Constrained optimization with max and absolute values in objective function" Tags="|optimization|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7869" PostTypeId="2" ParentId="7868" CreationDate="2013-07-02T19:45:24.717" Score="7" Body="&lt;p&gt;You haven't told us what set $i$ ranges over, so I'll just assume $i=1, 2, \ldots, n$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A standard trick in LP formulation of problems with absolute values is to introduce auxiliary variables and constraints with the basic idea that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min | x | $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is equivalent to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min t $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t \geq x $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t \geq -x $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying that idea to your problem, introduce auxiliary variables $t_{i}$, $i=1, 2, \ldots n$, and $s$.  Then formulate the problem as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min 2 \sum_{i=1}^{n} t_{i}    +  s $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_{i} \geq x_{i}$, $i=1, 2, \ldots n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_{i} \geq -x_{i}$, $i=1, 2, \ldots n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s \geq x_{i}+a_{i} $, $i=1, 2, \ldots, n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s \geq -(x_{i}+a_{i}) $, $i=1, 2, \ldots, n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i=1}^{n} (x_{i}+a_{i}) \leq L$&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-07-02T19:45:24.717" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7870" PostTypeId="2" ParentId="7861" CreationDate="2013-07-03T07:21:24.290" Score="2" Body="&lt;p&gt;Before you invest in either a cluster or a supercomputer (I think they're different beasts, others may disagree) I suggest you invest in a beefy desktop workstation, maybe 2 x 4- or 6-core processors.  You can use this to try out all your parallelisation options -- MPI, OpenMP,  CUDA, OpenCL (these two will need a GPU), probably others.  Your experience on the workstation will provide useful input to your decision on further hardware purchases.&lt;/p&gt;&#xA;" OwnerUserId="1003" LastActivityDate="2013-07-03T07:21:24.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7871" PostTypeId="2" ParentId="7865" CreationDate="2013-07-03T09:30:18.510" Score="3" Body="&lt;p&gt;is there a reason why you are trying to avoid creating &lt;a href=&quot;http://pastebin.com/neRBym8x&quot; rel=&quot;nofollow&quot; title=&quot;two surfaces&quot;&gt;two surfaces&lt;/a&gt;? The points on lines shared by the two surfaces should not be duplicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gmsh GUI enables the user to define physical surfaces (see Geometry-&gt;Physical Groups-&gt;Add-&gt;Surface), but you can only mark existing (elementary) surfaces with a physical tag. You cannot select some cells of existing surface and tell gmsh that they should belong to a new physical surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe it would be better to ask a question like this on Gmsh mailing list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Martin&lt;/p&gt;&#xA;" OwnerUserId="4688" LastActivityDate="2013-07-03T09:30:18.510" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7872" PostTypeId="2" ParentId="7706" CreationDate="2013-07-03T10:13:39.573" Score="1" Body="&lt;p&gt;So, I followed Bort's advice.  It turns out VMD has a lot more features than I thought.  To export a scene using VMD:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;From the menu, go to &quot;Extensions&quot; -&gt; &quot;Tk Console&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Type &quot;render Wavefront system.obj&quot;.  This will create system.obj and system.mtl in the current directory (you can navigate to a different one in Tk Console).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You're not done, because there's a bug as of VMD 1.9.1.  Open the system.mtl file (preferably in something like Vim) and remove all of the underscores in the lines containing &quot;illum_&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So: &lt;code&gt;:%s/illum_/illum /g&lt;/code&gt; in Vim.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Download Blender: &lt;a href=&quot;http://www.blender.org/download&quot; rel=&quot;nofollow&quot;&gt;http://www.blender.org/download&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Download Luxrender (a physically based state-of-the-art light transport solver): &lt;a href=&quot;http://www.luxrender.net/en_GB/download&quot; rel=&quot;nofollow&quot;&gt;http://www.luxrender.net/en_GB/download&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;From here, follow the instructions included with Luxrender to get it working with Blender.  Then you can import your system.obj.  You can then modify the materials to be whatever you like for Luxrender, along with the camera settings and a whole bunch of other stuff.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'll add a sample rendering of mine here in a little while.&lt;/p&gt;&#xA;" OwnerUserId="4362" LastActivityDate="2013-07-03T10:13:39.573" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7873" PostTypeId="1" AcceptedAnswerId="7889" CreationDate="2013-07-03T13:33:29.073" Score="3" ViewCount="1797" Body="&lt;p&gt;The advection-diffusion problem, where $0&amp;lt;x&amp;lt;1$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t = (-au + du_x)_x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with Dirichlet boundary conditions,&#xA;$&#xA;u(0)=1,~u(1)=0 &#xA;$&#xA;, has the &lt;strong&gt;steady-state&lt;/strong&gt; solution,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u(x) = \frac{e^{\lambda} - e^{x\lambda} }{e^{\lambda} - 1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This of course assumes both $a$ and $d$ are constants and $\lambda=a/d$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q.&lt;/strong&gt; Does this problem also have time-dependent analytical solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below I have written some tests for numerical code on various grids and I would like to compare with the analytical solution (the black line) as a function of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://vimeo.com/69527955&quot; rel=&quot;nofollow&quot;&gt;http://vimeo.com/69527955&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-07-03T23:09:07.377" LastActivityDate="2013-07-04T05:31:00.020" Title="Does the time-dependent advection-diffusion equation have an analytical solution?" Tags="|advection-diffusion|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7874" PostTypeId="1" AcceptedAnswerId="7878" CreationDate="2013-07-03T14:05:38.103" Score="3" ViewCount="137" Body="&lt;p&gt;I have a set of column vectors $[v_1,v_2,v_3,...,v_k]$ and I want to get the orthonormal basis $P$ for the range of those vectors. I use modified Gram-Schmidt to do orthogonalization. If $v_3$ is a linear combination of $v_1$ and $v_2$, when we orthogonalize $v_3$, it will return a $0$-vector. My problem is when a vector should be regarded as numerically zero? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to calculate the norm of the residual after subtracting the projections onto previous basis and compare it with the norm of $v_3$, if this ratio is less than some number (like 1%), then I can regard it as a $0$-vector. But this method doesn't work well. Is there anybody can help me with this? I prefer not to use SVD because it's too expensive and I need good accuracy as well as efficiency in this case. Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="4689" LastEditorUserId="414" LastEditDate="2013-07-03T15:11:26.383" LastActivityDate="2013-07-03T15:58:45.620" Title="How to use orthogonalization methods to identify linearly dependent vectors numerically and efficiently?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7875" PostTypeId="2" ParentId="135" CreationDate="2013-07-03T14:24:25.047" Score="0" Body="&lt;p&gt;There are some software packages, such as Materials Studio, that can automatically identify the point group of a molecule for you. However if you want to figure it out yourself, there is a nice &lt;a href=&quot;http://symmetry.otterbein.edu/common/images/flowchart.pdf&quot; rel=&quot;nofollow&quot;&gt;flowchart&lt;/a&gt; that will will walk you through the process. You can also look at the &lt;a href=&quot;http://symmetry.otterbein.edu/&quot; rel=&quot;nofollow&quot;&gt;Otterbein symmetry website&lt;/a&gt; for some tutorials and and interactive demos.&lt;/p&gt;&#xA;" OwnerUserId="3887" LastActivityDate="2013-07-03T14:24:25.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7876" PostTypeId="5" CreationDate="2013-07-03T15:51:32.200" Score="0" Body="&lt;p&gt;From the Intel-MKL Product Brief:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Intel® MKL includes highly vectorized and threaded Linear Algebra, Fast Fourier Transforms &#xA;  (FFT), Vector Math and Statistics functions. Through a single C or Fortran API call, these &#xA;  functions automatically scale across previous, current and future processor architectures by &#xA;  selecting the best code path for each.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The linear algebra part contains BLAS levels 1, 2 and 3 routine for single/double precision and complex types. It also includes LAPACK routines for solving linear algebra problems like LU, Cholesky, QR factorisation. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-07-03T16:44:28.470" LastActivityDate="2013-07-03T16:44:28.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7877" PostTypeId="4" CreationDate="2013-07-03T15:51:32.200" Score="0" Body="The Intel Math Kernel Library is a software library for Intel and compatible processors. It provides vectorized and threaded routines for optimal performance and is compatible with all C, C++ and FORTRAN compilers." OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-07-03T16:44:36.067" LastActivityDate="2013-07-03T16:44:36.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7878" PostTypeId="2" ParentId="7874" CreationDate="2013-07-03T15:58:45.620" Score="2" Body="&lt;p&gt;What you are trying to do is define a (relative) threshold $\epsilon$ to determine when a vector is zero:&#xA;$$&#xA; \| v_3^\perp \| \le \epsilon \|v_3\|.&#xA;$$&#xA;You've chosen $\epsilon=0.01$ but this is of course as arbitrary as any other number you may have chosen. The problem is that there is no such universal constant that can be used. Among the factors you should take into account are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The precision $\tau$ with which you do computations (e.g., float vs. double)&lt;/li&gt;&#xA;&lt;li&gt;The sizes of the vectors&lt;/li&gt;&#xA;&lt;li&gt;Whether the entries of the vectors are exact or subject to measurement noise&lt;/li&gt;&#xA;&lt;li&gt;etc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, if the elements are exact but you do computations in double precision $\tau\approx 10^{-16}$, then a vector for which&#xA;$$&#xA; \| v_3^\perp \| \le \tau\sqrt{N} \|v_3\|&#xA;$$&#xA;must certainly be considered to be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, you will probably want to choose $\epsilon$ significantly larger than $\tau\sqrt{N}$, but 0.01 seems rather large unless the entries of your vectors are only inexactly known.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-03T15:58:45.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7879" PostTypeId="2" ParentId="7873" CreationDate="2013-07-03T19:49:00.117" Score="6" Body="&lt;p&gt;Add a right-hand-side/forcing function, and use the Method of Manufactured Solutions. Then you can have any solution you want. Also, Separation of Variables works on this PDE just fine, so it also has an analytical solution.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-07-03T19:49:00.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7880" PostTypeId="1" AcceptedAnswerId="7882" CreationDate="2013-07-03T23:39:51.357" Score="8" ViewCount="7894" Body="&lt;p&gt;In &lt;code&gt;MPI&lt;/code&gt;, is there any built-in mechanism to notify a group of processes that they need to receive messages from other processes? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my application every process needs to send data to a group of processes with known rank IDs (which potentially changes at each iteration), but they do not know the rank of the processes that will be receiving messages from. Is there a portable, preferably built-in, way of achieving this without querying each and every process?&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2018-01-30T06:14:27.670" Title="What is an efficient way of notifying MPI processes of receiving messages?" Tags="|mpi|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7881" PostTypeId="2" ParentId="7880" CreationDate="2013-07-03T23:53:53.963" Score="1" Body="&lt;p&gt;If the senders know to whom they are sending but the receivers don't know from whom they are receiving, just post your receives from MPI_ANY_SOURCE. That will match the receive with any process that sends a message to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the receivers might not actually receive anything at all, they can post an iRecv. Whenever a process is done sending its messages, it can post am Ibarrier. When the Ibarrier completes because all processes have entered it (and called MPI_WAIT on the request object), everyone knows that the communication is done and they can cancel their previous Irecv. &lt;/p&gt;&#xA;" OwnerUserId="4658" LastActivityDate="2013-07-03T23:53:53.963" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7882" PostTypeId="2" ParentId="7880" CreationDate="2013-07-04T01:44:03.863" Score="7" Body="&lt;p&gt;You could try having all processors use &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Iprobe.html&quot;&gt;MPI_IProbe&lt;/a&gt; or &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Probe.html&quot;&gt;MPI_Probe&lt;/a&gt; with MPI_ANY_SOURCE to check if there are any receivable messages with a given tag.  If there are matching messages, you can extract the senders rank from the returned status and call MPI_Recv immediately.&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="486" LastEditDate="2013-07-04T01:49:40.867" LastActivityDate="2013-07-04T01:49:40.867" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7883" PostTypeId="2" ParentId="7880" CreationDate="2013-07-04T01:51:37.827" Score="7" Body="&lt;p&gt;&lt;a href=&quot;http://mpitutorial.com/tutorials/dynamic-receiving-with-mpi-probe-and-mpi-status/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mpitutorial.com/tutorials/dynamic-receiving-with-mpi-probe-and-mpi-status/&lt;/a&gt; has a tutorial describing the use of MPI_Probe that might be useful to you.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2018-01-30T06:14:27.670" LastActivityDate="2018-01-30T06:14:27.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7884" PostTypeId="2" ParentId="6933" CreationDate="2013-07-04T01:54:33.557" Score="1" Body="&lt;p&gt;Wes is exactly right about neighborhood collectives, which had your usage (to the extent I understand it) as one of its motivators.  However, like MPI RMA (i.e. MPI_Win), this is a static model that requires collective initialization.  You could also use MPI_Probe and dynamically communicator between your neighbors.  This may not allow for the same optimizations as neighborhood collectives or one-sided but it is dynamic and should work pretty well in any implementation of MPI-1.x forward (I do not recall when Probe was added).&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-04T01:54:33.557" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7885" PostTypeId="2" ParentId="7766" CreationDate="2013-07-04T02:02:10.660" Score="2" Body="&lt;p&gt;The algorithm in ScaLAPACK is not very good.  Look at SUMMA (&lt;a href=&quot;http://www.cs.utexas.edu/ftp/techreports/tr95-13.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.utexas.edu/ftp/techreports/tr95-13.pdf&lt;/a&gt;) and its implementation in Elemental  (code.google.com/p/elemental/) or CTF (&lt;a href=&quot;http://ctf.eecs.berkeley.edu/&quot; rel=&quot;nofollow&quot;&gt;http://ctf.eecs.berkeley.edu/&lt;/a&gt;) instead.  Papers by Solomonik and coworkers show substantial performance improvements over ScaLAPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A colleague of mine - a physicist, not a computer scientist - implemented SUMMA himself and found it was faster than ScaLAPACK, so it's not &lt;em&gt;that&lt;/em&gt; difficult to beat the latter.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-04T02:02:10.660" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7886" PostTypeId="2" ParentId="7866" CreationDate="2013-07-04T02:11:05.660" Score="3" Body="&lt;p&gt;In the standard form, KS-DFT is solved variationally, which means that additional degrees of freedom in the basis set must lead to a lower (or equal) energy.  This is a very basic property of the variational method and the math is almost certainly explained on Wikipedia or equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming the same functional is used.  Each DFT functional provides an effective one-body Schrodinger equation that is solved exactly using the SCF variational approach (assuming no spurious solutions are found).  Hence, DFT is an approximate theory with an exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hartree-Fock (HF), on the other hand, is the variational solution of the &lt;em&gt;exact&lt;/em&gt; non-relativistic Born-Oppenheimer Schrodinger equation within the given basis set with the caveat that electron-electron interactions are approximated using a mean-field.  In contrast to DFT, HF is an approximation solution to an exact^1 theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;^1 For some definition of exact, i.e. the one where full configuration interface (FCI) in a finite basis set is considered the exact solution to a particular family of Hamiltonians.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-04T02:11:05.660" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7887" PostTypeId="2" ParentId="6800" CreationDate="2013-07-04T02:12:59.103" Score="1" Body="&lt;p&gt;Not using pseudo-potentials is a great way to drive up the computational cost of a QMC enormously while not adding to the accuracy much at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a very recent paper about PPs for QMC: &lt;a href=&quot;http://jcp.aip.org/resource/1/jcpsa6/v139/i1/p014101_s1&quot; rel=&quot;nofollow&quot;&gt;http://jcp.aip.org/resource/1/jcpsa6/v139/i1/p014101_s1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-04T02:12:59.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7888" PostTypeId="2" ParentId="2026" CreationDate="2013-07-04T03:27:30.267" Score="12" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/ashwin/gstar4d&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;gStar4D&lt;/strong&gt;&lt;/a&gt; is a fast and robust 3D Delaunay algorithm for the GPU. It is implemented using CUDA and works on NVIDIA GPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar to &lt;a href=&quot;http://www.comp.nus.edu.sg/~tants/delaunay2DDownload.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;GPU-DT&lt;/strong&gt;&lt;/a&gt;, this algorithm constructs the 3D digital Voronoi diagram first. However, in 3D this cannot be dualized to a triangulation due to topological and geometrical problems. Instead, gStar4D uses the neighborhood information from this diagram to create stars lifted to 4D and performs star splaying on them efficiently on the GPU. By extracting the lower hull from this, the 3D Delaunay triangulation is obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fastest 3D Delaunay implementation is &lt;a href=&quot;https://github.com/ashwin/gDel3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;gDel3D&lt;/strong&gt;&lt;/a&gt;, which is a hybrid GPU-CPU algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It performs parallel insertion and flipping on the GPU. The result is close to Delaunay. It then fixes this result using a conservative star splaying method on the CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both these methods are robust, so they can handle any kind of degenerate input. They can handle millions of points, if you have GPU memory large enough to hold the intermediate data structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I am the author of these algorithms and implementations :)&lt;/p&gt;&#xA;" OwnerUserId="4694" LastEditorUserId="4287" LastEditDate="2017-02-04T19:51:25.233" LastActivityDate="2017-02-04T19:51:25.233" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7889" PostTypeId="2" ParentId="7873" CreationDate="2013-07-04T05:31:00.020" Score="6" Body="&lt;p&gt;Because the advection-diffusion equation is linear, there are many exact solutions. One reference with many exact solutions (including source terms) is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;M. Th. van Genuchten and W. J. Alves&#xA;Analytical Solutions of the One-Dimensional Convective-Dispersive Solute Transport Equation&#xA;U. S. Department of Agriculture, Agricultural Research Service&#xA;Technical Bulletin 1661, 1982&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get a pdf of this report &lt;a href=&quot;http://naldc.nal.usda.gov/download/CAT82780278/PDF&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you search for papers that cite this report, you will turn up many additional problems with exact solutions. &lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2013-07-04T05:31:00.020" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7890" PostTypeId="1" AcceptedAnswerId="7917" CreationDate="2013-07-04T08:13:51.183" Score="8" ViewCount="5776" Body="&lt;p&gt;I would like some &lt;strong&gt;super simple computational code for solving magnetohydrodynamics problems&lt;/strong&gt;. High accuracy nor performance is not my concern. I wan't it just to visually explore qualitative behavior of plasma. Important for me is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Generality&lt;/strong&gt; - it should not be specialised on any particular subproblem&#xA;of MHD &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Easy to get it work&lt;/strong&gt; - does not require any libraries difficult to instal. Input is strightforward and well documented by some examples&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Easy to read and modify&lt;/strong&gt; - you can read the code and learn about MHD solution from it. You can use this code as starting point for your own MHD code development &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Something as simple as this &lt;a href=&quot;http://www.openprocessing.org/sketch/10419&quot;&gt;Navier stokes solver&lt;/a&gt; would be best&lt;/p&gt;&#xA;&#xA;&lt;p&gt;best would be something in python-numpy or java, but C++ or Fortran is not a big  problem for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Up to now I found:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;some libraries ( &lt;em&gt;MHD-Hermes&lt;/em&gt; and &lt;em&gt;python-mhd&lt;/em&gt; ) which are in python, but doesn't seem to have any documantation or examples how to use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this &#xA;&lt;a href=&quot;http://www.cita.utoronto.ca/~pen/MHD/&quot;&gt;&lt;em&gt;A Free, Fast, Simple and Efficient TVD MHD Code&lt;/em&gt;&lt;/a&gt; which is in frotran but has just 400 lines&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about writing something by myself but I want to be sure there is nothing similar already avaible. I think it could be quite easy, just take the Navier-Stokes solver from the link above and introduce electric and magnetic potential by FFT poisson solver ( or Particle Mesh Ewald ).  &lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="4696" LastEditDate="2013-07-05T06:51:02.853" LastActivityDate="2013-07-08T07:06:50.070" Title="simple MHD simulation code for (self) education and play with" Tags="|fluid-dynamics|education|electromagnetism|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7891" PostTypeId="2" ParentId="7890" CreationDate="2013-07-04T14:24:15.360" Score="1" Body="&lt;p&gt;Give a try to PLUTO&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://plutocode.ph.unito.it/&quot; rel=&quot;nofollow&quot;&gt;http://plutocode.ph.unito.it/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps !&#xA;Cheers, &#xA;Tom&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2013-07-04T14:24:15.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7892" PostTypeId="1" AcceptedAnswerId="7897" CreationDate="2013-07-04T14:27:36.797" Score="1" ViewCount="1071" Body="&lt;p&gt;I'd like to do a $A=QR$ decomposition of a matrix $A$, with $m\times n$. I use &lt;code&gt;dgeqrf_&lt;/code&gt; (or &lt;code&gt;dgeqp3_&lt;/code&gt;) to proceed to the first part of the decomposition. Then, I can easily compute the matrix $R$ by taking the upper diagonal part of the output matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is such that $m &amp;gt; n$, the Lapack subroutine &lt;code&gt;dorgqr_&lt;/code&gt; create the correct $Q$ matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if $A$ is such that $m &amp;lt; n$, the same subroutine gives me an error :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;** On entry to DORGQR parameter number  2 had an illegal value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I guess it's because $m&amp;lt;n$... should I use another routine for this case?&lt;/p&gt;&#xA;" OwnerUserId="4626" LastActivityDate="2013-07-04T22:40:07.000" Title="lapack dorgqr qr decomposition matrix $m\times n$ with $m&lt;n$" Tags="|lapack|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7894" PostTypeId="1" CreationDate="2013-07-04T15:28:46.687" Score="5" ViewCount="482" Body="&lt;p&gt;I need to examine the (static) electric field distribution around various electrode configurations in the presence of a dielectric, and have been using a finite difference approach to the 2D generalised Poisson equation do this. James Nagel's nice paper at &#xA;&lt;a href=&quot;http://www.drjamesnagel.com/notes/Nagel%20-%20Numerical%20Poisson.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.drjamesnagel.com/notes/Nagel%20-%20Numerical%20Poisson.pdf&lt;/a&gt; &#xA;has been my guide and suggests SOR on a discretized integral form of Poisson's equation. But Nagel's paper also notes that for large dielectric constants - which I have - SOR converges slowly or fails: then he uses the Matlab/Octave &quot;\&quot; as a blackbox operation on the sparse system matrix. This works fine for me on problems up to 300x300 grid or thereabouts - but it is slow and I really want to use C or something similar to integrate with other routines.&#xA;So I am seeking advice on a suitable algorithm to apply to this problem. I think ADI may be what I need - but before learning how to code and use it, I'd value guidance and perhaps alternative recommendations. Perhaps I just need to call the LAPACK version of &quot;\&quot;? Of course, I want unconditional stability, high accuracy and instant convergence! Thanks for reading!&lt;/p&gt;&#xA;" OwnerUserId="4700" LastActivityDate="2016-09-18T07:13:16.920" Title="Request algorithm recommomendation for 2D generalised Poisson solution" Tags="|linear-algebra|pde|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7895" PostTypeId="1" CreationDate="2013-07-04T15:51:41.800" Score="1" ViewCount="72" Body="&lt;p&gt;The discrete analogue of the $L_p$ norm for the mesh function $V$ is $$\|V\|_{l^p(\bar{\Omega}^N)}=\left(\sum_{i=0}^NV_i^p\bar{h}_i\right)^{1/p}$$ &#xA;where $\bar{\Omega}^N$ is an arbitrary mesh, $\bar{h}_i=(h_{i+1}+h_i)/2$ and $h_i=x_i-x_{i-1}$. The error $e$ of a certain approximation is such that $e_i=-e^{-x_i/\epsilon}$ for $1\leq i\leq N-1$ and 0 for $i=0$ or $i=N$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any $p$, $1\leq p\leq \infty$ the $l^p(\bar{\Omega}^N)$ norm of the error on a general non-uniform mesh is &#xA;$$\|e\|_{l^p(\bar{\Omega}^N)}=\left(\sum_{i=1}^{N-1}e^{-px_i/\epsilon}\bar{h}_i\right)^{1/p}$$&#xA;My first question is why is this expression $O(N^{-1/P})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a uniform mesh, why is $$\left\vert {e_i}^p\right\vert \leq N\sum_{j=1}^{N-1}\left\vert e_j\right\vert^ph$$ How does this imply $$\|e\|_{\bar{\Omega}^N}\leq N^{1/P}\|e\|_{l^p(\bar{\Omega}^N)}$$&lt;/p&gt;&#xA;" OwnerUserId="4698" LastEditorUserId="27" LastEditDate="2013-07-04T22:23:01.607" LastActivityDate="2013-07-04T22:23:01.607" Title="discrete versions of Lp norm" Tags="|singular-perturbation|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7896" PostTypeId="2" ParentId="7892" CreationDate="2013-07-04T18:23:34.260" Score="2" Body="&lt;p&gt;Usually people use the QR decomposition for tall/skinny matrices, as Jan pointed out. For short/fat matrices, the LQ decomposition is more appropriate. It is basically a QR on the transpose of the matrix, and you use routine &lt;code&gt;dgelqf&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you state what you intend to do with the factorization, we might be able to point out better and more specific solutions.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-07-04T18:23:34.260" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7897" PostTypeId="2" ParentId="7892" CreationDate="2013-07-04T22:31:03.783" Score="3" Body="&lt;p&gt;You seem to be confusing the input arguments for &lt;a href=&quot;http://www.netlib.org/lapack/double/dorgqr.f&quot; rel=&quot;nofollow&quot;&gt;dorgqr&lt;/a&gt;. The second argument is the width of the orthogonal/unitary matrix which you would like to generate as the product of the Householder reflectors generated by &lt;a href=&quot;http://www.netlib.org/lapack/double/dgeqrf.f&quot; rel=&quot;nofollow&quot;&gt;dgeqrf&lt;/a&gt; or &lt;a href=&quot;http://www.netlib.org/lapack/double/dgeqp3.f&quot; rel=&quot;nofollow&quot;&gt;dgeqp3&lt;/a&gt;. While both of these routines support the $m \le n$ case, you should recognize that, when $m \le n$, the QR decomposition implies a $Q$ which is $m \times m$ and an $R$ which is $m \times n$, and $Q$ is implicitly defined as the product of the $m$ Householder transformations implied by each of the first $m$ columns of the output of &lt;a href=&quot;http://www.netlib.org/lapack/double/dgeqrf.f&quot; rel=&quot;nofollow&quot;&gt;dgeqrf&lt;/a&gt;/&lt;a href=&quot;http://www.netlib.org/lapack/double/dgeqp3.f&quot; rel=&quot;nofollow&quot;&gt;dgeqp3&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should thus have the first, second, and third argument of &lt;a href=&quot;http://www.netlib.org/lapack/double/dorgqr.f&quot; rel=&quot;nofollow&quot;&gt;dorgqr&lt;/a&gt; each equal to $m$. My guess is that you used the width of $A$, $n$, for the second argument, which would be illegal.&lt;/p&gt;&#xA;" OwnerUserId="67" LastEditorUserId="67" LastEditDate="2013-07-04T22:40:07.000" LastActivityDate="2013-07-04T22:40:07.000" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7898" PostTypeId="2" ParentId="7861" CreationDate="2013-07-05T03:43:16.223" Score="0" Body="&lt;p&gt;Ok, so if I want to go down the GPU route I have a number of options:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Shader languages&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;HLSL &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use with xna&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;GLSL&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use with OpenTk&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cg&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use with Unity&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm thinking of using OpenTk because it lets me use the GPU and integrates with monodevelop and visual studio which I use at work for other projects. It also alows me to use OpenCL if I should need it.&lt;/p&gt;&#xA;" OwnerUserId="4680" LastActivityDate="2013-07-05T03:43:16.223" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7899" PostTypeId="2" ParentId="7894" CreationDate="2013-07-05T06:43:43.283" Score="3" Body="&lt;p&gt;A good way to approach this is by using existing powerful numerical packages such as PETSc. Look for PETSc examples of solving the Poisson equation, and that will be a good starting point.&lt;/p&gt;&#xA;" OwnerUserId="4325" LastEditorUserId="4325" LastEditDate="2013-07-06T16:02:30.033" LastActivityDate="2013-07-06T16:02:30.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7900" PostTypeId="1" AcceptedAnswerId="7902" CreationDate="2013-07-05T07:20:54.807" Score="3" ViewCount="2259" Body="&lt;p&gt;I would like to gain knowledge on available free or commercial tools to generate  automatically structured mesh from geometry, particularly quadrilateral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which options do I have ?&lt;/p&gt;&#xA;" OwnerUserId="4707" LastActivityDate="2013-07-06T11:46:06.823" Title="Survey on automatic structured mesh generation tools" Tags="|software|mesh-generation|mesh|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7901" PostTypeId="1" CreationDate="2013-07-05T12:08:27.320" Score="1" ViewCount="1060" Body="&lt;p&gt;I'd like to find a condition that allows me to determine if a matrix is invertible or not. naively, I computed the determinant to see if it was zero. but then I realized that even for very small values (1e-100), lapack is still able to find an invert matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5192/lapack-singular-matrixes-what-does-the-positive-integer-info-mean&quot;&gt;here&lt;/a&gt; that a $QR$ factorization can be used to compute the rank of a matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is a $QR$ factorization the best way to check if a matrix is invertible? will it take into account the numerical errors ? is there a more efficient way ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found this &lt;a href=&quot;https://stackoverflow.com/questions/4974349/computing-the-reciprocal-condition-number-with-lapack-i-e-rcondx&quot;&gt;link&lt;/a&gt; that gives a reciprocal condition number $\kappa$. What does this number mean ? I know that if $\kappa=\infty$ then the matrix is singular ! But what is the cutoff, what is $\infty$ for a computer ?&lt;/p&gt;&#xA;" OwnerUserId="4626" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-01-08T15:55:04.327" Title="lapack singular matrix" Tags="|lapack|matrix|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="7902" PostTypeId="2" ParentId="7900" CreationDate="2013-07-05T13:36:23.930" Score="5" Body="&lt;p&gt;Permit me to clarify - you ask about &quot;structured mesh&quot; that's &quot;quadrilateral.&quot; By definition, a structured mesh (or grid) consists of quads (2D) and hexes (3D). So I want to clarify that you're inquiring about automatic structured quad/hex grid generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, you're seeking the holy grail. While there are many software tools for generating structured grids the degree of &quot;automation&quot; varies greatly. I put automation in quotes because exactly what is meant by automation depends on who's asking. (To not digress too much, Fred Brooks in his book The Mythical Man Month wrote that automatic has always been a synonym for &quot;better than we're currently doing.&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I think of automatic grid generation I do so in the literal sense. You have software into which you import a CAD geometry, maybe you specify a handful of meshing parameters, you press a button, magic happens, and a grid pops out the other end. (The other problem with automation is people mean &quot;make a grid automatically and &lt;em&gt;exactly the way I would've made it manually&lt;/em&gt;.&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cubit.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;CUBIT&lt;/a&gt; is always the first thing that comes to mind when thinking about automatic hex generation. However, it doesn't always generate a structured hex grid - it uses many techniques that generate unstructured hexes. As you'll learn when you Google it, CUBIT is a U.S. Dept. of Energy product that has been commercialized as &lt;a href=&quot;http://www.csimsoft.com/trelis.jsp&quot; rel=&quot;nofollow&quot;&gt;Trelis&lt;/a&gt;. Also, I was told of a plan to release CUBIT as open source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.gridpro.com/&quot; rel=&quot;nofollow&quot;&gt;GridPro&lt;/a&gt; is a commercial tool for structured grid generation that has cool technology for doing what many consider to be the hard part of structured hex grid generation - the decomposition of the 3D domain into zones or blocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.pointwise.com/&quot; rel=&quot;nofollow&quot;&gt;Pointwise&lt;/a&gt; and its predecessor &lt;a href=&quot;http://www.pointwise.com/gridgen/&quot; rel=&quot;nofollow&quot;&gt;Gridgen&lt;/a&gt;, have been generating structured grids since 1984. This is a commercial product. (full disclosure - I work for Pointwise.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't specify your intended area of application. CUBIT, for example, seems to be more for structural mechanics. The other two packages I mention are more for CFD. The requirements of the analysis will dictate which features are important to have in an automatic gridder. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, there are too many structured grid generators of all sorts (commercial, government, academic) to list here. Google is your friend.&lt;/p&gt;&#xA;" OwnerUserId="4711" LastEditorUserId="1567" LastEditDate="2013-07-06T11:46:06.823" LastActivityDate="2013-07-06T11:46:06.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7903" PostTypeId="2" ParentId="7751" CreationDate="2013-07-05T14:10:53.340" Score="-1" Body="&lt;p&gt;Instead of modify simplexphaseone, as an alternative solution, we revised a few places of the repair operator of GA-MPC, so that it can finish up the solution by itself and get ride of those &quot;noises&quot;. In our test case of TVaR constraint optimization problem, it can get solution as accurate as the linearized solution by linprog , but with running time 60% less than that of linprog.&lt;/p&gt;&#xA;" OwnerUserId="4584" LastActivityDate="2013-07-05T14:10:53.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7904" PostTypeId="2" ParentId="7894" CreationDate="2013-07-05T15:13:52.103" Score="3" Body="&lt;p&gt;I suggest using the &quot;Run and Time&quot; option in MATLAB to see exactly where the time is being spent in your 300x300 grid example. I am almost certain it is not in the \ operator. Generally, MATLAB has very fast, state-of-the-art implementations for sparse matrix factorizations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took at look at James Nagel's MATLAB code at the site referred to in your link. I didn't try to run it since I couldn't find an example case. But I did notice that there is a loop over all points in the grid (e.g. 300x300) to fill the sparse matrix prior to calling . I strongly suspect this is where most of the time is being spent. It is generally more efficient in MATLAB to write the code so that operations are performed on long vectors rather than scalars. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find many references on this topic if you search for &quot;vectorized MATLAB code&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should add that if you are interested in a fast, basic finite element code for the Poisson equation written in MATLAB, you can find one at the following link&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.isima.fr/~jkoko/codes.html&quot; rel=&quot;nofollow&quot;&gt;http://www.isima.fr/~jkoko/codes.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;along with a discussion of how it was designed.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="19791" LastEditDate="2016-09-18T07:13:16.920" LastActivityDate="2016-09-18T07:13:16.920" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7905" PostTypeId="2" ParentId="7900" CreationDate="2013-07-05T16:11:41.287" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://geuz.org/gmsh/&quot;&gt;gmsh&lt;/a&gt; is a viable way to generate quadrilateral meshes in 2d. It's also open source.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="27" LastEditDate="2013-07-05T18:50:41.937" LastActivityDate="2013-07-05T18:50:41.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7906" PostTypeId="2" ParentId="7894" CreationDate="2013-07-05T16:18:07.547" Score="5" Body="&lt;p&gt;There are many excellent codes out there that solve the Poisson equation. I may point you at step-6 of the deal.II tutorial at&#xA;  &lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/step_6.html&quot; rel=&quot;noreferrer&quot;&gt;http://dealii.org/developer/doxygen/deal.II/step_6.html&lt;/a&gt;&#xA;as just one example. (Disclaimer: That's my own code.) Most other finite element libraries will have similar example codes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-05T16:18:07.547" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7907" PostTypeId="2" ParentId="7880" CreationDate="2013-07-05T16:21:30.453" Score="12" Body="&lt;p&gt;Others have already proposed the various MPI_Probe variants but I'd like to point out one thing: MPI is not a remote procedure call, i.e., there are no ways to &lt;em&gt;notify&lt;/em&gt; a process that some message has come in (e.g., by raising a signal). Messages are sent but if the receiving process doesn't actually go look for them, then nothing will happen. As such, the answer to your question is &quot;no&quot;, but a process can actively figure out whether a message has been sent to it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-05T16:21:30.453" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7908" PostTypeId="1" AcceptedAnswerId="7910" CreationDate="2013-07-05T17:13:15.900" Score="2" ViewCount="965" Body="&lt;p&gt;After exporting the solution (u) and the mesh (p, e, t) from pdetool to workspace, I need to compute the solution at a set of predefined points. These predefined points where the solution is sought are not uniformly spaced and are scattered in the xy space. I know that if one has the solution (u) from pde toolbox, it can be exported to a uniform grid using either &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/8998-surface-fitting-using-gridfit,&quot; rel=&quot;nofollow&quot;&gt;griddata &lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[u1,x1,y1] = gridfit(x,y,u,xnodes,ynodes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or the function &lt;a href=&quot;http://www.mathworks.co.uk/help/pde/ug/tri2grid.html&quot; rel=&quot;nofollow&quot;&gt;tri2grid &lt;/a&gt;. I believe both gridfit and tri2grid need the output sampling points to be monotonically increasing which unfortunately is not my case. How do I export/interpolate the solution to a non-uniform/scattered grid?&lt;/p&gt;&#xA;" OwnerUserId="4171" LastActivityDate="2013-07-05T20:59:11.140" Title="export pde solution to non-uniform grid" Tags="|pde|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7910" PostTypeId="2" ParentId="7908" CreationDate="2013-07-05T20:59:11.140" Score="3" Body="&lt;p&gt;To interpolate/extrapolate solution on a uniform grid to a non-uniform grid, or vice versa, using the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/triscatteredinterpclass.html&quot; rel=&quot;nofollow&quot;&gt;$\mathtt{TriScatteredInterp}$ class&lt;/a&gt; is the way to go. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let say your &lt;code&gt;u&lt;/code&gt; is associated with nodes: &lt;code&gt;p&lt;/code&gt;, an array size $(\text{ # of nodes})\times (\text{Dimension})$, is originally where your solution &lt;code&gt;u&lt;/code&gt; is defined on, so that &lt;code&gt;size(u,1) = size(p,1)&lt;/code&gt;. The the following commmand let you create an interpolant class (abstractly):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;uI = triScatteredInterp(p(:,1),p(:,2),u,'natural'); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for two dimensional data, or &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;uI = triScatteredInterp(p(:,1),p(:,2),p(:,3),u,'natural'); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for 3D data. I personally prefer &lt;code&gt;'natural'&lt;/code&gt; as method here, &lt;code&gt;'nearest'&lt;/code&gt; is not that nice for PDE solution, and &lt;code&gt;'linear'&lt;/code&gt; will result in loss of interpolation near boundary where two sides data are needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to get the values on the non-uniform grid, what is best about &lt;code&gt;triScatteredInterp&lt;/code&gt; class is that it works like a function, you can use &lt;code&gt;feval&lt;/code&gt; to evaluate like a function handle: say your non-uniform grid points are array &lt;code&gt;q&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; uInterp_at_q = uI(q(:,1),q(:,2));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in 2D or &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; uInterp_at_q = uI(q(:,1),q(:,2),q(:,3));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in 3D. And &lt;code&gt;uInterp_at_q(i,1)&lt;/code&gt; is the value of the interpolant associated with the $i$-th node &lt;code&gt;q(i,:)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Notice mathworks' doc says that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;TriScatteredInterp&lt;/code&gt; will be removed in a future release. Use &lt;code&gt;scatteredInterpolant&lt;/code&gt; instead.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The usage will almost the same, just &lt;code&gt;scatteredInterpolant&lt;/code&gt; has an extra extrapolation method switch.&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-07-05T20:59:11.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7912" PostTypeId="1" CreationDate="2013-07-06T17:37:59.490" Score="3" ViewCount="1536" Body="&lt;p&gt;I am trying to implement the following system of time-dependent, coupled nonlinear pdes in FEniCS:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_{t}\rho+2\left(\nabla\rho\nabla\phi+\rho\nabla^{2}\phi\right)+2\sigma\rho\left(\rho-1\right)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_{t}\phi+\left(\nabla\phi\right)^{2}-\frac{1}{2}\left[\frac{\nabla^{2}\rho}{\rho}-\frac{1}{2}\frac{\left(\nabla\rho\right)^{2}}{\rho^{2}}\right]+\rho-1=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the unknown scalar functions $\rho(\mathbf{r},t)$ and $\phi(\mathbf{r},t)$ are real-valued (2D). Boundary conditions are homogeneous Neumann for both functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand from the documentation of Fenics, the first step is to write the &lt;strong&gt;weak formulation&lt;/strong&gt; of the problem, but the examples in the tutorial are all very basic and I would really appreciate if someone showed me how to implement the system above.&lt;/p&gt;&#xA;" OwnerUserId="4656" LastEditorUserId="4656" LastEditDate="2013-07-06T18:27:17.100" LastActivityDate="2014-03-24T00:40:55.873" Title="weak formulation of coupled pdes for fenics" Tags="|pde|finite-element|fenics|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="7913" PostTypeId="2" ParentId="46" CreationDate="2013-07-06T19:11:59.247" Score="8" Body="&lt;p&gt;In computational electromagnetism, there is a famous (or infamous because of the difficulties in some) set of test problems: &lt;a href=&quot;http://www.compumag.org/jsite/team.html&quot; rel=&quot;noreferrer&quot;&gt;Testing Electromagnetic Analysis Methods (T.E.A.M.)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of them really need seriously state-of-art numerical techniques to get the correct simulation results aligned with the experimental data. For example, &lt;a href=&quot;http://www.cc.ac.cn/06research_report/0602.pdf&quot; rel=&quot;noreferrer&quot;&gt;the conductor-coil problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another set of testing problems for Maxwell equations are compiled by Dauge: &lt;a href=&quot;http://perso.univ-rennes1.fr/monique.dauge/benchmax.html&quot; rel=&quot;noreferrer&quot;&gt;Benchmark computations for Maxwell equations &#xA;for the approximation of highly singular solutions&lt;/a&gt;. The one in the famous (or infamous) Fichera cube:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/D1KSd.png&quot; alt=&quot;fichera&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;any $\phi\in H^{1+\epsilon}$ and $\mathbf{E} = -\nabla \phi$ living on this cube will be a challenge to your numerical PDE codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly numerical PDE, there is hpFEM's &lt;a href=&quot;http://www.hpfem.org/hermes/hermes-examples/doc/_build/html/src/hermes2d/benchmarks-general.html&quot; rel=&quot;noreferrer&quot;&gt;Benchmarks in 2D (Problems with Known Exact Solutions)&lt;/a&gt;, I have been using the test problems in it for a long time to test my finite element codes. For example, the famous &lt;a href=&quot;http://www.hpfem.org/hermes/hermes-examples/doc/_build/html/src/hermes2d/benchmarks-general/lshape.html&quot; rel=&quot;noreferrer&quot;&gt;non-smooth near the origin of the L-shaped domain example&lt;/a&gt;&#xA;$$&#xA;\Delta u= 0,\quad \text{where }\;u = r^{\alpha}\sin(\alpha \theta).&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-07-06T19:11:59.247" CommentCount="0" CommunityOwnedDate="2013-07-06T19:11:59.247" ContentLicense="CC BY-SA 3.0" />
  <row Id="7915" PostTypeId="1" CreationDate="2013-07-07T11:53:44.360" Score="3" ViewCount="97" Body="&lt;p&gt;On page 18 of SuperLU's &lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/superlu_ug.pdf&quot; rel=&quot;nofollow&quot;&gt;user guide&lt;/a&gt;, the following sample input matrix &lt;code&gt;A&lt;/code&gt; is given:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;19,0,21,21,0&#xA;12,21,0,0,0&#xA;0,12,16,0,0&#xA;0,0,0,5,21&#xA;12,12,0,0,18&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;U&lt;/code&gt; matrix is given as follow in the user guide:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;19,0,21,21,0&#xA;0,21,-13.26,-13.26,0&#xA;0,0,23.58,7,58,0&#xA;0,0,0,5,21&#xA;0,0,0,0,34.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But &lt;code&gt;dgssv()&lt;/code&gt; produces the following matrix, which is printed via &lt;code&gt;dPrint_CompCol_Matrix(&quot;U&quot;, U)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CompCol matrix U:&#xA;Stype 0, Dtype 1, Mtype 4&#xA;nrow 5, ncol 5, nnz 11&#xA;nzval: 21.000000  -13.263158  7.578947  21.000000  &#xA;rowind: 0  1  2  0  &#xA;colptr: 0  0  0  1  4  4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;U&lt;/code&gt; matrix given in the user guide is correct (you can verify it at &lt;a href=&quot;http://www.bluebit.gr/matrix-calculator/&quot; rel=&quot;nofollow&quot;&gt;this link&lt;/a&gt;). I wonder why SuperLU produces such a matrix. How can I generate correct &lt;code&gt;U&lt;/code&gt; matrix from the SuperLU output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. For ordering columns of &lt;code&gt;A&lt;/code&gt;, I have used my permutation vector, which is 1,2,3,4,5. In other words, there is no reordering for columns. Rows of &lt;code&gt;A&lt;/code&gt; are not reordered by SuperLU.&lt;/p&gt;&#xA;" OwnerUserId="3978" LastEditorUserId="3978" LastEditDate="2013-07-07T12:14:44.343" LastActivityDate="2013-07-08T08:57:07.760" Title="Missing nonzeros in U matrix computed by SuperLU" Tags="|linear-algebra|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7916" PostTypeId="2" ParentId="7915" CreationDate="2013-07-07T12:27:41.823" Score="3" Body="&lt;p&gt;I have found out that the missing nonzeros are stored in &lt;code&gt;L&lt;/code&gt; matrix. Storage type of &lt;code&gt;L&lt;/code&gt; is supernodel to speedup the factorization process via utilizing dense blocks. So some nonzeros of &lt;code&gt;U&lt;/code&gt; matrix is stored in &lt;code&gt;L&lt;/code&gt; matrix.&lt;/p&gt;&#xA;" OwnerUserId="3978" LastEditorUserId="3978" LastEditDate="2013-07-08T08:57:07.760" LastActivityDate="2013-07-08T08:57:07.760" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7917" PostTypeId="2" ParentId="7890" CreationDate="2013-07-08T07:06:50.070" Score="3" Body="&lt;p&gt;Have a look on these three:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://piernik.astri.umk.pl/doku.php&quot; rel=&quot;nofollow&quot;&gt;Piernik MHD&lt;/a&gt; it is a code which evolved from the Pen &amp;amp; Trac MHD you mentioned in your question. Now its quite mature and the development is still active. Written in modern Fortran 95/2003.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.amuncode.org/?page_id=17&quot; rel=&quot;nofollow&quot;&gt;Godunov MHD&lt;/a&gt; an MHD code designed especially for simulating the reconnection events, but can be easily modified and applied to other problems. It can be parallelized from single multicore machine (OpenMP) up to clusters with many nodes (MPI). Written in Fortran 90/95.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://code.google.com/p/python-mhd/&quot; rel=&quot;nofollow&quot;&gt;python-mhd&lt;/a&gt; a small and simple MHD code, easy to play with it. Written in Python and C.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="829" LastActivityDate="2013-07-08T07:06:50.070" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7918" PostTypeId="2" ParentId="46" CreationDate="2013-07-08T12:28:08.313" Score="3" Body="&lt;p&gt;There is a collection of reference PDE-constrained optimization problems maintained by Roland Herzog at TU-Chemnitz &lt;a href=&quot;http://www.optpde.net&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2013-07-08T12:28:08.313" CommentCount="0" CommunityOwnedDate="2013-07-08T12:28:08.313" ContentLicense="CC BY-SA 3.0" />
  <row Id="7919" PostTypeId="1" AcceptedAnswerId="7933" CreationDate="2013-07-08T14:00:00.357" Score="8" ViewCount="526" Body="&lt;p&gt;I want to solve this PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/smDXB.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I have some code that will automatically generate pde solutions for a very similar pde that includes a time derivative (partial d/ partial t) using an ADI method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if there is a way to approximate the attached pde with a pde that includes a time derivative? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there are approaches for one dimensional pdes. For example in the attached pde if you remove all the Y derivatives, then the pde can be approximated by adding the time derivative and multiplying the diffusion term by a large number, using implicit stepping and using 1 step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated,&#xA;Thanks,&#xA;Rob&lt;/p&gt;&#xA;" OwnerUserId="777" LastEditorUserId="123" LastEditDate="2013-07-08T17:25:28.133" LastActivityDate="2013-07-09T13:43:29.097" Title="Can I solve this time-independent PDE by adding a time derivative and marching in time?" Tags="|pde|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7920" PostTypeId="1" CreationDate="2013-07-08T21:28:58.577" Score="3" ViewCount="99" Body="&lt;p&gt;I need to generate an output file from my molecular data that can be loaded into VMD to visualize the bond connectivity.  I'm not sure what the best choice is.  I know the PDB format has a HETATOM record with a CONECT field, but I believe this is supposed to be used only for unusual connections or very small molecules.  Also, this field is limited to 5 digits, and some of my systems have &gt; 1 million atoms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other option is to write two files, a PDB file with coordinate information and a PSF file with connectivity.  But then I have to lug around two files everywhere, which seems unnecessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better format I'm missing?&lt;/p&gt;&#xA;" OwnerUserId="4362" LastActivityDate="2013-07-09T11:21:39.107" Title="Best file format for bond connectivity?" Tags="|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7921" PostTypeId="2" ParentId="7479" CreationDate="2013-07-08T22:40:36.797" Score="3" Body="&lt;p&gt;Assume $m \ge n$.  Consider the function&#xA;$$&#xA;\begin{aligned}g(k) &amp;amp;= \frac{2^n-k-1}{2^n-k} \\&#xA;&amp;amp;= \frac{1-(k+1)2^{-n}}{1-k2^{-n}}  \\&#xA;&amp;amp;= \left(1-(k+1)2^{-n} \right)\left(1 + k2^{-n} + k^2 2^{-2n} + O(2^{-3n})\right) \\&#xA;&amp;amp;= 1-2^{-n}-k 2^{-2n} + O(2^{-3n})&#xA;\end{aligned}$$&#xA;For large enough $n$, $g(1)$ and $g(2)$ differ by essentially $2^{-2n}$, and therefore require $2n$ bits to distinguish.  Conversely, we have&#xA;$$\left|\frac{a_1}{b_1}-\frac{a_2}{b_2}\right| = \left|\frac{a_1 b_2 - a_2 b_1}{b_1 b_2} \right| \ge \frac{1}{(2^n-1)(2^n-1)} &amp;gt; 2^{-2n}$$&#xA;for all distinct fractions, so $2n$ bits is also sufficient.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2013-07-08T22:52:59.637" LastActivityDate="2013-07-08T22:52:59.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7922" PostTypeId="1" AcceptedAnswerId="7934" CreationDate="2013-07-09T03:19:56.057" Score="6" ViewCount="225" Body="&lt;p&gt;I am taking a shot at solving a coupled physics problem. I have this matrix formed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{J}=\begin{bmatrix}&#xA;\mathbf{A} &amp;amp; \mathbf{B}\\ &#xA;\mathbf{C} &amp;amp; \mathbf{D}&#xA;\end{bmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{A}$ and $\mathbf{D}$ represent two different physics, and $\mathbf{C}=-\mathbf{B}^{T}$.  Originally, I wanted to use the Schur complement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{S}_{D}=\mathbf{D}-\mathbf{C}\mathbf{A}^{-1}\mathbf{B}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this won't work as shown because $\mathbf{A}$ is non-invertible. It's guaranteed at least one row in $\mathbf{A}$ will be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what other methods could I apply to solve this problem? Is there any way to take advantage of the fact that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{S}_{A}=\mathbf{A}-\mathbf{C}\mathbf{D}^{-1}\mathbf{B}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;exists since $\mathbf{D}$ is nonsingular?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Follow-up&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The size of $\mathbf{J}$ shouldn't exceed 1000x1000. I'm solving a statics problem for a geometrically nonlinear structure with no PDEs involved. In the past, this problem was solved using a monolithic algorithm, where the $\mathbf{A}$/$\mathbf{D}$ physics are solved independently of one another. I want to try a new approach since a rudimentary algorithm  already developed shows significant speed-up over traditional methods. As Jan pointed out below, using $S_{A}$ is an option. Is there a PETSc run-time option to alternate from $S_{D}$ to $S_{A}$ (for background, page 87 in the PETSc manual discusses Schur factorization)? Or do I have to reorganize the $\mathbf{J}$ matrix structure, essentially swapping the $\mathbf{A}$ and $\mathbf{D}$ blocks?&lt;/p&gt;&#xA;" OwnerUserId="4724" LastEditorUserId="4724" LastEditDate="2013-07-09T07:02:55.550" LastActivityDate="2013-07-09T13:50:37.190" Title="Advice on solving a coupled physics problem" Tags="|linear-algebra|petsc|linear-solver|multiphysics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7923" PostTypeId="5" CreationDate="2013-07-09T07:04:15.587" Score="0" Body="&lt;p&gt;Sorting algorithms arrange similar items (names, int's, tuples, ...) according to a predefined order (increasing int's, alphabetical, ...). In the case of sorting tuples (or records), the sorting key defines the element on which the ordering should take place. If needed, more keys (and their hierarchy) can be defined. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typical sorting algorithms are bubble sort, selection sort, insertion sort and quick sort. The latter is one of the most popular sorting algorithms and has an average computational complexity in $\mathcal O(n\log(n))$ (although the worst case is in $\mathcal O(n^2)$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most famous reference book on sorting algorithms is &quot;The Art of Computer Programming, Volume 3, Searching and Sorting&quot;, 2nd edition (1998) by Donald E. Knuth. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="541" LastEditDate="2013-07-09T07:52:30.943" LastActivityDate="2013-07-09T07:52:30.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7924" PostTypeId="4" CreationDate="2013-07-09T07:04:15.587" Score="0" Body="Sorting is the process of arranging similar items according to a predefined order. " OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-07-09T09:14:15.430" LastActivityDate="2013-07-09T09:14:15.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7925" PostTypeId="5" CreationDate="2013-07-09T07:13:35.810" Score="0" Body="&lt;p&gt;Simulation is the process of mimicking the behaviour of a real physical process or system using a representative model. It assumes that in a first step, a model is built that captures the key characteristics of the real world system. As real world systems tend to be very complex, the modeler has to make assumptions and simplifications. Therefore, it is important that the modeler verifies and validates her model to assure that indeed the right &quot;physics&quot; of the real world is represented in the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a next step, a computer program or algorithm is used to evaluate the equations that result from the modeling step. Because of the possible complexity of the model, this can require again approximations like model simplifications and/or discretisation of continuum equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes the simulation can be run using Monte Carlo simulation: a process where by using (pseudo-)random variables, the model can be sampled and an average behaviour can be predicted.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-07-09T09:17:29.490" LastActivityDate="2013-07-09T09:17:29.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7926" PostTypeId="4" CreationDate="2013-07-09T07:13:35.810" Score="0" Body="Simulation is the process of mimicking the behaviour of a real physical process or system using a representative model. " OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-07-09T09:14:06.117" LastActivityDate="2013-07-09T09:14:06.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7927" PostTypeId="5" CreationDate="2013-07-09T07:21:54.717" Score="0" Body="&lt;p&gt;A polynomial is an expression constructed by a finite linear combination of positive integer powers of a number of variables. If there is only one variable, the polynomial is called univariate, otherwise multivariate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Polynomials appear in many fields of mathematics and physics and many have been named after mathematicians who have discovered some of their properties (Legendre, Chebychev, Laguerre, Jacobi, ...). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the roots of a polynomial (i.e. the values of the variables for which the polynomial expression evaluates to exactly zero) is an ever-recurring problem in mathematics and physics. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="549" LastEditDate="2013-07-09T09:17:19.317" LastActivityDate="2013-07-09T09:17:19.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7928" PostTypeId="4" CreationDate="2013-07-09T07:21:54.717" Score="0" Body="For questions about solving and representing polynomials computationally." OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-15T16:13:17.980" LastActivityDate="2021-11-15T16:13:17.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7930" PostTypeId="2" ParentId="7920" CreationDate="2013-07-09T11:21:39.107" Score="3" Body="&lt;p&gt;As you say, you can't use the CONECT records in a PDB file because it's limited to 99,999 atoms (rather, I think the first digit is treated in base 36, so 360,000 atoms).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VMD supports other file formats, and has a plugin system for you to &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/plugins/molfile/&quot; rel=&quot;nofollow&quot;&gt;define your own format&lt;/a&gt;. Of the ones listed there, your best bet is likely the Tripos mol2 file.&lt;/p&gt;&#xA;" OwnerUserId="1735" LastActivityDate="2013-07-09T11:21:39.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7931" PostTypeId="1" CreationDate="2013-07-09T11:29:33.857" Score="3" ViewCount="753" Body="&lt;p&gt;Consider an instance of non-convexoptimization problem:&#xA; It seems that this problem is NP-complete. How can I find a suitable reduction for this? &lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="917" LastEditDate="2014-07-29T13:54:59.023" LastActivityDate="2014-07-29T13:54:59.023" Title="NP-Completeness" Tags="|convex-optimization|nonlinear-programming|nonconvex|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7932" PostTypeId="1" CreationDate="2013-07-09T11:32:57.637" Score="3" ViewCount="82" Body="&lt;p&gt;I have a very sparse non-linear system $N(u) = 0$ that can be solved as a time-dependent ODE, $\frac{du}{dt} = N(u)$, and explicitly integrated until $\frac{du}{dt} = N(u) = 0$, e.g. by forward euler, $u^{n+1} = u^n+\Delta t N(u^n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the process of this numerical integration, is there any way to recover the eigenvalues of the system by examining something about the growth/response in $u$ and it's dependence on the timestep? Gershgorin's theorem provides a bound on the spectrum, but in practice this bound's range is too large to be useful. I don't need exact eigenvalues, but relatively tight approximations would be great. I suspect there's something about this in the literature of linear time-invariant (LTI) theory, but I don't know where to start looking.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2013-07-09T12:39:44.377" LastActivityDate="2013-07-09T12:39:44.377" Title="Estimating eigenvalues from time-dependent non-linear operator" Tags="|linear-algebra|eigenvalues|nonlinear-equations|runge-kutta|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7933" PostTypeId="2" ParentId="7919" CreationDate="2013-07-09T13:43:29.097" Score="7" Body="&lt;p&gt;The equation is too complex for me to tell whether the answer is yes or no. But in general, here is what you need to consider: Assume you have &lt;em&gt;any&lt;/em&gt; equation (algebraic, differential, or partial differential) $f(u)=0$ and you wonder whether you can find $u$ by considering the problem $\frac{dv(t)}{dt}+f(v(t))=0$ in hopes that $u = \lim_{t\rightarrow \infty} v(t)$. Then, the following is clear:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It is certainly true that $v_1(t)=u$ is a stationary solution of the time dependent equation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;But it is not clear whether it is a &lt;em&gt;stable&lt;/em&gt; solution. This is important because if you start with a $v_2(t) \neq u$, then $v_2(t) \rightarrow v_1(t)=u$ only if $v_1(t)$ is a stable solution of the equation. In other words, without stability of $v_1(t)$, you can't expect the pseudo-time dependent problem to converge to the solution of the original, time independent problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;That stability is not automatically guaranteed is easy to see. Consider, for example, the equation $f(u)=-\Delta u-h=0$. You can find the solution to this by solving the heat equation, $\frac{dv(t)}{dt}-\Delta v(t)=h$. But if you had started with $\tilde f(\tilde u)=\Delta \tilde u+h=0$ (which, of course, has exactly the same solution), you would not have been able to find the solution by solving $\frac{d\tilde v(t)}{dt}+\Delta \tilde v(t)=-h$ since this equation does not in general have a solution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-09T13:43:29.097" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7934" PostTypeId="2" ParentId="7922" CreationDate="2013-07-09T13:50:37.190" Score="3" Body="&lt;p&gt;For linear problems of size ~1000, you can't beat a direct solver by trying to use an iterative solver. For problems of this size, it also doesn't really matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem was any bigger, I would recommend you watch/listen lectures 34 and 38 of the course I recorded this spring: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-09T13:50:37.190" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7935" PostTypeId="1" AcceptedAnswerId="7937" CreationDate="2013-07-09T13:52:02.023" Score="0" ViewCount="235" Body="&lt;p&gt;It seems to me many CFD codes in coding the turbulence models, uses the relation $\tau_{ij}{\partial _j}{U_i}=\tau_{ij}W_{ij}$ instead of using strain-rate tensor, i.e., $\tau_{ij}{\partial _j}{U_i}=\tau_{ij}S_{ij}$ for the sake of numerical friendliness, to estimate the production term $P_k=\tau_{ij}{\partial _j}{U_i}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\tau_{ij}$ is the specific Reynolds Stress Tensor, it is symmetric. And $S_{ij}=\frac{1}{2}({\partial _j}{U_i}+{\partial _i}{U_j})$ and $W_{ij}=\frac{1}{2}({\partial _j}{U_i}-{\partial _i}{U_j})$ are Rate-of-Strain Tensor and Rotation-Rate Tensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fully understand why $\tau_{ij}{\partial _j}{U_i}=\tau_{ij}S_{ij}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it seems to me that $\tau_{ij}W_{ij}$ should be &lt;strong&gt;ZERO&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tau_{ij}W_{ij}=0$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because $W_{12}=-W_{21}, W_{13}=-W_{31}, W_{23}=-W_{32}$ and $W_{11}=W_{22}=W_{33}=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="1229" LastEditorUserId="1229" LastEditDate="2013-07-09T15:38:03.247" LastActivityDate="2013-07-09T21:00:49.417" Title="Does it make any sense to use rotation rate tensor for production estimation?" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7937" PostTypeId="2" ParentId="7935" CreationDate="2013-07-09T21:00:49.417" Score="1" Body="&lt;p&gt;In Reynolds Stress Models (RSM) and Explicit Algebraic RSM you calculate exact production using Reynolds stress tensor and velocity gradient tensor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in RANS models we use Boussinesq hypothesis and use strain rate magnitude $S$ or vorticity magnitude $\Omega$, a scalar quantities to model production e.g. $P_k= \mu_t S^2$. Kato-Launder modification replaces one $S$ with $\Omega$ in an attempt to remedy large production in stagnation regions, so we have $P_k=\mu_t S \Omega$. Menter's SST model used $\Omega$ until model revision in 2003.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think modelers used a lot of intuition in deciding which one to use, based on behavior of these quantities in certain regions. Therefore you may find $ \Omega$ more often in models tuned for attached boundary layers.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2013-07-09T21:00:49.417" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7938" PostTypeId="1" CreationDate="2013-07-09T22:54:50.953" Score="5" ViewCount="2643" Body="&lt;p&gt;I have a nonlinear inequality constrained optimization problem of the form&#xA;$$\begin{array}{cc} \min &amp;amp; f(x) \\ \textrm{s.t.} &amp;amp; g(x) \ge 0 \end{array}$$&#xA;where $x \in \mathbb{R}^n$, $f : \mathbb{R}^n \to \mathbb{R}$, $g : \mathbb{R}^n \to \mathbb{R}^m$.  I am currently solving this system with pseudotransient continuation (3) applied to a semismooth Newton's method (1,2).  The semismooth Newton's method encodes the KKT conditions for a local solution to the constrained optimization into a single semismooth equation.  Pseudotransient continuation is an unnecessarily fancy name for adding a diagonal term to the gradient of this equation (which includes the Hessian of the energy $f$) and running Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, pseudotransient continuation globally convergent only for sufficiently large diagonal adjustments, and my current problem is converging only down to a nasty period two oscillation between two states.  Without the constraints, global convergence could be enforced using line search using the original energy function $f$.  However, the KKT conditions depend only on the gradient of $f$, not its value, and the original energy is somewhat obscured in the passage to the semismooth equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Are there natural ways to perform line search in the context of constrained optimization?  Note that it is not sufficient to apply line search to the residual norm, &lt;a href=&quot;http://lists.mcs.anl.gov/pipermail/petsc-users/2012-May/013494.html&quot;&gt;as described by Jed Brown here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.38.8330&quot;&gt;T. De Luca, F. Facchinei, and C. Kanzow (1996), A semismooth equation approach to the solution of nonlinear complementarity problems, Mathematical programming, 75(3), 407-439.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.8613&quot;&gt;F. Facchinei, A. Fischer, and C. Kanzow, A semismooth Newton method for variational inequalities: Theoretical results and preliminary numerical experience.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.113.6788&quot;&gt;Kelley, C., Keyes, D. E. (1998). Convergence analysis of pseudo-transient continuation. SIAM Journal on Numerical Analysis, 508–523.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-07-10T03:15:08.037" Title="Line search for constrained optimization" Tags="|constrained-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7939" PostTypeId="1" CreationDate="2013-07-10T02:04:54.657" Score="5" ViewCount="708" Body="&lt;p&gt;I am performing series of random projections i.e. projecting the input matrix onto randomly generated orthonormal bases (of much lower dimensionality). The projection is just a matrix multiplication like $A \cdot B = P$, where $B$ is the basis composed of several mutually orthogonal column vectors of unit length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have come to conclusion that there is no need to generate new basis before each projection because simple permutation of its rows should be sufficient to keep the randomness of the whole process. If this is true, doing so would be very profitable in terms of computational complexity, since row-permutation also keeps orthonormality of the basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I am a programmer and no matematician, so I'm not sure if indeed projections over such bases are random and thus independent of each other, or they are somehow similar and thus limited. Is my idea right or wrong, and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edited&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $A$ consists of $n$ row vectors of binary values ($0$ or $1$), where the vector dimension is close to $n$ which is between $10^5$ and $10^7$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process of repeating random projections aims to find well-clusterable projection, or at least avoid worst cases. Currently the clusterability is defined as variance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to minimize complexity of row permutation I'm just rotating row indices, so the original basis is not altered at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final goal is to reduce dimensionality of the input vectors and then cluster them in this low-dimensional space with some distance or density based algorithm.&lt;/p&gt;&#xA;&#xA;&lt;hr/&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;B = \begin{pmatrix}&#xA;1 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 \\&#xA;0 &amp;amp; 0 \\&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above counterexample clearly shows potential risk of generating a basis which is insensitive to certain permutation - exchange rows 1 and 2, and the result is the same as changing the order of basis vectors, which does not change the projection at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;: What is the &lt;em&gt;probability&lt;/em&gt; of generating a random basis with such an inherent flaw.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that row permutation somehow reflects the basis, but I don't know which bases do not reflect well. I do not expect exact numbers, since it would require precise definition of words &quot;have&quot;, &quot;flaw&quot; and &quot;well&quot;. Any help in expressing the problem more profesionally is greatly welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;: Is it the only sort of flaw that can reduce randomness of the entire process?&lt;/p&gt;&#xA;" OwnerUserId="4745" LastEditorUserId="4745" LastEditDate="2013-09-07T12:02:47.297" LastActivityDate="2013-09-07T12:02:47.297" Title="Generating pseudo-random orthonormal bases for random projection" Tags="|matrix|random-number-generation|projection|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7940" PostTypeId="1" AcceptedAnswerId="7943" CreationDate="2013-07-10T02:27:57.143" Score="6" ViewCount="161" Body="&lt;p&gt;Are there high order energy-conserving or symplectic methods for solving $y'=f(y)$?&lt;/p&gt;&#xA;" OwnerUserId="4746" LastActivityDate="2013-07-10T06:51:33.280" Title="Are there high order symplectic methods for $y'=f(y)$?" Tags="|ode|conservation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7941" PostTypeId="2" ParentId="7938" CreationDate="2013-07-10T03:15:08.037" Score="3" Body="&lt;p&gt;Yes. Constrained optimization algorithms update their iterates in the same basic fashion as unconstrained optimization algorithms, using updates of the form $x_{k + 1} = x_{k} + \alpha_{k}d_{k} \in \mathbb{R}^{n}$, where $d_{k} \in \mathbb{R}^{n}$ is the direction of the update, $x_{k} + d_{k}$ is feasible, and $\alpha_{k} \in [0, 1]$ is chosen such that $f(x_{k} + \alpha_{k}d_{k}) &amp;lt; f(x_{k})$ using a line search algorithm. (Some variants will set $\alpha_{k} = 1$, and force the direction-finding algorithm to choose and scale $d_{k}$ appropriately in order to achieve convergence.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of algorithms of this form include feasible direction, conditional gradient, and gradient projection algorithms.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-07-10T03:15:08.037" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7942" PostTypeId="1" AcceptedAnswerId="7945" CreationDate="2013-07-10T05:19:05.433" Score="2" ViewCount="433" Body="&lt;p&gt;I wish to solve a coupled system of non-linear equation of this form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = -(\mathcal{F})_x + f(x,u,w) \\&#xA;w_t = \mathcal{F} + g(x,u,w)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by stepping the equations forward in time.  The first equation is the &lt;em&gt;advection-diffusion equation&lt;/em&gt; with non-linear reaction term, and where the &lt;em&gt;flux&lt;/em&gt; has the definition,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{F}=au-du_x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First equation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is &lt;a href=&quot;http://danieljfarrell.github.io/FVM/finite_volume_method.html&quot; rel=&quot;nofollow&quot;&gt;straight forward to write this using the finite volume method&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{\Omega}u_t~dx = -\int_{\Omega}(\mathcal{F})_x~dx + \int_{\Omega}f(x,u,w)~dx \\&#xA;\tilde{u}_t = \frac{1}{h_j}\left( -\mathcal{F}_{j+\frac{1}{2}} +  \mathcal{F}_{j-\frac{1}{2}} \right) + \tilde{f}(x_j,u_j,w_j)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h_j$ is the width (not volume because I am considering 1D only) of the $j$-th cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second equation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, how should the second equation be written, can I do the following?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{\Omega}w_t~dx = \int_{\Omega}\mathcal{F}~dx + \int_{\Omega}g(x,u,w)~dx \\&#xA;\tilde{w}_t = \frac{1}{h_j}\mathcal{F}_jh_j + \tilde{g}(x_j,u_j,w_j)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, if we assume that the flux in the $j$-th cell is constant across the cell then the  integral becomes,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{\Omega}\mathcal{F}~dx = \mathcal{F}_jh_j&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is simply base $\times$ height of the cell, but because I want to know the cell average values I divide again by the cell width. Finally I get the following for the second equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde{w}_t = \mathcal{F}_j + \tilde{g}(x_j,u_j,w_j)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q1.&lt;/strong&gt; Is this correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of implementation I know 3 of the terms in $\mathcal{F}_j$, they are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$a_j$, the cell advection velocity,&lt;/li&gt;&#xA;&lt;li&gt;$d_j$, the cell diffusion coefficient,&lt;/li&gt;&#xA;&lt;li&gt;$u_j$, the cell mass concentration,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, $u_x$ is not known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q2.&lt;/strong&gt; Can I simply write $u_x$ using the values of concentration interpolated to the cell faces? For example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_x = \frac{1}{h_j}\left( u_{j+\frac{1}{2}} - u_{j-\frac{1}{2}} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't seem quite right because I was taught that in the finite-volume method, concentrations are defined in the cells and fluxes are defined on the faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding definition of fluxes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;strong&gt;first equation&lt;/strong&gt; I define the fluxes as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{F}_{j+\frac{1}{2}} = a_{j+\frac{1}{2}}\left( \frac{h_{j+1}}{2h_{+}} u_j + \frac{h_j}{2h_{+}} u_{j+1} \right) - d_{j+\frac{1}{2}} \frac{u_{j+1}-u_j}{h_{+}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{F}_{j-\frac{1}{2}} = a_{j-\frac{1}{2}}\left( \frac{h_{j}}{2h_{-}} u_{j-1} + \frac{h_{j-1}}{2h_{-}} u_{j} \right) - d_{j-\frac{1}{2}} \frac{u_{j}-u_{j-1}}{h_{-}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h_j$ is the cell width, and $h_{\pm}$ are the centroid distances in the forward and backward direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here linear interpolation is used to define the variable at the cell faces $u(x_{j-1/2})$ and $u(x_{j+1/2})$, and forward difference and backward difference  is used to define $u_x(x_{j+1/2})$ and $u_x(x_{j-1/2})$. This gives a central discretisation for both the advection and diffusion terms (however, &lt;a href=&quot;http://danieljfarrell.github.io/FVM/finite_volume_method.html&quot; rel=&quot;nofollow&quot;&gt;upwinding can still be introduced easily&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However for the &lt;strong&gt;second equation&lt;/strong&gt; the flux is &lt;em&gt;not&lt;/em&gt; defined on the cell's left and right face, &lt;strong&gt;it is defined as the cell average&lt;/strong&gt;, so I need to use a different definition for,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{F}_j = a(x_j)u(x_j) - d(x_j)u_x(x_j)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, $a(x_j)$, $u(x_j)$, and $d(x_j)$ are already known. The only term than needs defining for the second equation is $u_x(x_j)$. So maybe I could use,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_x(x_j) = \frac{1}{h_j}\left( u_{j+1/2} - u_{j-1/2} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with linear interpolation, which as you pointed out would give a second order accurate central difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could I also use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_x(x_j) = \frac{1}{h_{+}}\left( u_{j+1} - u_{j} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my comment below, this is what I was getting at. How do define the $u_x(x_j)$ for the second equation, what are the pros. and cons. of using a central or upwinding scheme for this term.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="123" LastEditDate="2013-07-14T14:19:16.823" LastActivityDate="2013-07-14T14:19:16.823" Title="How to write this non-linear PDE with the finite volume method?" Tags="|pde|finite-volume|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7943" PostTypeId="2" ParentId="7940" CreationDate="2013-07-10T06:51:33.280" Score="9" Body="&lt;p&gt;There are the geometric integrations written by Ernst Hairer &amp;amp; co:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;E. Hairer, C. Lubich and G. Wanner (2002): Geometric Numerical Integration. Structure-Preserving Algorithms for Ordinary Differential equations. Springer Series in Comput. Math., vol. 31&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;E. Hairer and M. Hairer (2002): GniCodes - Matlab programs for geometric numerical integration&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;for which you can download the routines in FORTRAN77, Matlab or C++ from &lt;a href=&quot;http://www.unige.ch/~hairer/software.html&quot;&gt;Professor Hairer's website&lt;/a&gt;. The Implicit Runge-Kutta (IRK) scheme he has implemented has order 8. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-07-10T06:51:33.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7945" PostTypeId="2" ParentId="7942" CreationDate="2013-07-10T15:19:32.243" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Second equation&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;However, how should the second equation be written, can I do the&#xA;  following?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$ \int_{\Omega}w_t~dx = \int_{\Omega}\mathcal{F}~dx + \int_{\Omega}g(x,u,w)~dx \\ \tilde{w}_t =&#xA; \frac{1}{h_j}\mathcal{F}_jh_j + \tilde{g}(x_j,u_j,w_j) $$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Moreover, if we assume that the flux in the $j$-th cell is constant&#xA;  across the cell then the  integral becomes,&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$ \int_{\Omega}\mathcal{F}~dx = \mathcal{F}_jh_j $$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is also true if the flux is &lt;em&gt;linear&lt;/em&gt; in the cell, not just constant.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Q1.&lt;/strong&gt; Is this correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, $u_x$ is not known.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Q2.&lt;/strong&gt; Can I simply write $u_x$ using the values of concentration interpolated to the cell faces? For example,&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$ u_x = \frac{1}{h_j}\left( u_{j+\frac{1}{2}} - u_{j-\frac{1}{2}} \right) $$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, in finite difference/volume you generally have to compute the derivatives from the conserved variables. Note that to find the derivative $u_x$ at $j$ you can use a central difference (which may be unstable depending on the form of the diffusion and you might need to apply upwinding), which would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_{x,j} = \frac{1}{2h} (u_{j+1} - u_{j-1}) + O(h^2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;assuming constant $h$ cell size. If you were using a linear interpolation to find $u_{j-1/2} = 1/2(u_{j-1}+u_{j})$ you can see that the above expression for central differencing using $j+1$ and $j-1$ is exactly the same as your $ u_x = \frac{1}{h_j}\left( u_{j+\frac{1}{2}} - u_{j-\frac{1}{2}} \right) $&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-07-10T15:19:32.243" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7947" PostTypeId="1" AcceptedAnswerId="7966" CreationDate="2013-07-10T16:23:20.300" Score="10" ViewCount="531" Body="&lt;p&gt;I have implemented a V-Cycle multigrid solver using both a linear defect correction (LDC) and full approximation scheme (FAS).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is the following: Using LDC the residual is reduced by a factor of ~0.03 per cycle. The FAS implementation does converge with a linear factor too, but the factor is only ~0.58. Thus FAS needs about 20 times the number of cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the code is shared, the only difference are the down/up calculations, LDC uses&lt;/p&gt;&#xA;&#xA;&lt;p&gt;down: $u_H:=0,\quad b_H:=I_h^H(b_h-L_hu_h)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;up: $u_h:=u_h+I_H^hu_H$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and FAS uses&lt;/p&gt;&#xA;&#xA;&lt;p&gt;down: $u_H:=I_h^Hu_h,\quad b_H:=I_h^Hb_h+L_HI_h^Hu_h-I_h^HL_hu_h$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;up: $u_h:=u_h+I_H^h(u_H-I_h^Hu_h)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My test setting is from Brigg's &quot;A Multigrid Tutorial, Second Edition&quot;, p. 64, has the analytical solution &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x,y)=(x^2-x^4)(y^4-y^2) \quad$ with $x,y\in [0,1]^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the equation is $Lv=\Delta u=:b$ using the typical linear 5-point stencil as the Laplace-Operator $L$. The initial guess is $v=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing the test setting, e.g. to the trivial $u(x,y)=0$ using an initial guess of $v=1$ results in nearly the same convergence factors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since only the down/up code differs, the LDC results comply with the book and the FAS at least seems to work too, I have no Idea why it is so much slower in the same linear setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one odd behavior in both LDC and FAS that I cannot explain yet that only happens if the initial guess is bad (e.g. $=0$ but also in my full multigrid experiments where the interpolation to the new fine grid increases the residual from $10^{-15}$ to $10^{-1}$): If I increase to number of post correction relaxations to a very high number such that the solution is solved to machine precision on the coarse grid, it looses nearly all digits when going one step up to the next fine grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since a picture says more than words:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// first cycle, levels 0-4&#xA;// DOWN&#xA;VCycle top 4, start               res_norm 3.676520e+02 // initial residual&#xA;VCycle top 4, cycle 0, current 4, res_norm 3.676520e+02&#xA;VCycle top 4, cycle 0, current 4, res_norm 1.520312e+02 // relaxed (2 iterations)&#xA;VCycle tau_norm 2.148001e+01 (DEBUG calculation)&#xA;VCycle top 4, cycle 0, current 3, res_norm 1.049619e+02 // restricted&#xA;VCycle top 4, cycle 0, current 3, res_norm 5.050392e+01 // relaxed (2 iterations)&#xA;VCycle top 4, cycle 0, current 2, res_norm 3.518764e+01 // restricted&#xA;VCycle top 4, cycle 0, current 2, res_norm 1.759372e+01 // relaxed (2 iterations)&#xA;VCycle top 4, cycle 0, current 1, res_norm 1.234398e+01 // restricted&#xA;VCycle top 4, cycle 0, current 1, res_norm 4.728777e+00 // relaxed (2 iterations)&#xA;VCycle top 4, cycle 0, current 0, res_norm 3.343750e+00 // restricted&#xA;// coarsest grid&#xA;VCycle top 4, cycle 0, current 0, res_norm 0.000000e+00 // solved&#xA;// UP&#xA;VCycle top 4, cycle 0, current 1, res_norm 3.738426e+00 // prolonged&#xA;VCycle top 4, cycle 0, current 1, res_norm 0.000000e+00 // relaxed (many iterations)&#xA;VCycle top 4, cycle 0, current 2, res_norm 1.509429e+01 // prolonged (loosing digits)&#xA;VCycle top 4, cycle 0, current 2, res_norm 2.512148e-15 // relaxed (many iterations)&#xA;VCycle top 4, cycle 0, current 3, res_norm 4.695979e+01 // prolonged (loosing digits)&#xA;VCycle top 4, cycle 0, current 3, res_norm 0.000000e+00 // relaxed (many iterations)&#xA;VCycle top 4, cycle 0, current 4, res_norm 1.469312e+02 // prolonged (loosing digits)&#xA;VCycle top 4, cycle 0, current 4, res_norm 9.172812e-24 // relaxed (many iterations)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if there can be only a few digits gained per cycle or if this indicates an error during the interpolation to the fine grid. If it is the latter case, how can the LDC achieve the by-the-book residual ratios of ~0.03 when using always 2 relaxations?&lt;/p&gt;&#xA;" OwnerUserId="4742" LastActivityDate="2013-07-12T18:40:03.873" Title="FAS-multigrid slower than linear defect correction?" Tags="|multigrid|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7948" PostTypeId="2" ParentId="7947" CreationDate="2013-07-10T17:26:42.430" Score="7" Body="&lt;p&gt;I don't directly know your answer as I mainly use FAS instead of correction since I do multigrid for nonlinear problems, but some thoughts you can look into:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You're applying a linear correction scheme to a linear problem, so it's not shocking that it does very well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider your boundary conditions: make sure you're doing them correctly, and also note that complicated BCs could look completely different on the coarse grid, making corrections there not so useful.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Double-check your treatment of the source term; I recall screwing up something in the prolongation stage related to that term when I wrote it for poisson.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I've never seen the need to iterate to convergence on the coarsest grid. A solution there depends on the fine grid residual being correct, which it isn't. You're trying to push those errors out of the domain / smooth them out. If you're fully converged on the coarsest grid in an early iteration, your solution is naturally pretty far from the correct fine grid solution because your residuals there aren't up to date. This is almost certainly the reason for why you're seeing residuals jump up in the prolongation stage.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, try a relaxation factor for both the restriction and prolongation operators, say 0.75. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If it helps, this is what my FAS residual history looked like for a poisson problem using single grid through full 7V cycles. I believe the relaxation factor was 0.75, and I was using a 3-stage RK scheme as a smoother with a single iteration at each grid level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/m2FXk.png&quot; alt=&quot;res history&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2013-07-10T17:47:15.040" LastActivityDate="2013-07-10T17:47:15.040" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7950" PostTypeId="2" ParentId="2824" CreationDate="2013-07-10T20:46:12.237" Score="5" Body="&lt;p&gt;Let's gather all the answers here. I suppose your matrix is symmetric, since you say that the eigenvectors are orthogonal and try to order the eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;No, the eigenvalues could come in any order; there is no guarantee that they are ordered. &lt;br/&gt;1bis. There are some classes of matrices (such as &lt;a href=&quot;http://en.wikipedia.org/wiki/Z-matrix_%28mathematics%29&quot; rel=&quot;noreferrer&quot;&gt;Z-matrices&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Nonnegative_matrix&quot; rel=&quot;noreferrer&quot;&gt;nonnegative matrices&lt;/a&gt;) for which it is known that the largest or smallest eigenvector is nonnegative. That must be your case.&#xA;&lt;br/&gt;1ter. If another eigenvector were to be nonnegative, then the scalar product with the dominant eigenvector $u^{(1)}$ would be positive, as it's the sum of nonnegative terms, one of them at least positive.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;em&gt;columns&lt;/em&gt; of the returned $V$ are the eigenvectors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, they come in pair, in the sense that $D(i,i)$ is the eigenvalue relative to $V(:,i)$. This follows from $X*V=V*D$, if you think about it.&lt;br/&gt;&#xA;3bis. As written above, the eigenvalues are not returned in a specific order. If you wish you can sort $D$ and $V$ conformably using these commands:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;[uselessVariable,permutation]=sort(diag(D));&lt;/p&gt;&#xA;&#xA;&lt;p&gt;D=D(permutation,permutation);V=V(:,permutation);&#xA;&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I guess that &quot;most significant&quot; means larger. This means the last two columns of $V$, after you reorder as above.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Check &lt;a href=&quot;http://en.wikipedia.org/wiki/Cross-correlation&quot; rel=&quot;noreferrer&quot;&gt;cross-correlation&lt;/a&gt; on Wikipedia. This statement probably means that the sample covariance is zero, i.e., $\frac{1}{n}\sum_{t=1}^n (\phi_t-\mu) (\phi'_t-\mu)'^T=0$, where $\mu$ and $\mu'$ are the means of the two time series. If you wish to verify this experimentally, I guess you'll have a hard time getting an exact zero out of Matlab, since this sum converges quite slowly to its asymptotical value usually. Consider yourself lucky if you have 2 significative digits.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2013-07-10T20:46:12.237" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7951" PostTypeId="2" ParentId="7947" CreationDate="2013-07-11T02:53:11.693" Score="6" Body="&lt;p&gt;If you are using a vertex-centered discretization, then state restriction should be injection rather than the full-weighted residual restriction that it appears you use.  That is, replace $I_h^H$ with $\hat I_h^H$ when restricting the state.  Using full-weighted restriction for state produces aliasing of high-frequency components of the state which after applying $u^h \gets u^h (u^H - I_h^H u^h)$ results in new noise on the same scale as before the coarse correction (boundary conditions are especially likely culprits for this effect).  Use injection, $\hat I_h^H u^h$, and this problem should go away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Spectrally, state restriction only needs high secondary order (accurate preservation of low frequencies), but the primary order (aliasing of high frequencies) does not matter. Injection has primary order 0 and infinite secondary order.  Meanwhile, residual restriction needs both primary and secondary order to be positive (at least).  See section 4.3 of &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~achi/classics.pdf&quot;&gt;Achi Brandt's Multigrid Guide&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When designing MG methods, it's also good to look at error rather than residuals and to make sure you weight the norm appropriately.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-07-11T02:53:11.693" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7953" PostTypeId="1" AcceptedAnswerId="8022" CreationDate="2013-07-11T10:22:08.207" Score="8" ViewCount="740" Body="&lt;p&gt;I'm working on a parameter study of &lt;a href=&quot;http://en.wikipedia.org/wiki/Duffing_equation&quot;&gt;Duffing's equation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\ddot x + \delta \dot x + \alpha x + \beta x^3 = \gamma \cos{\omega t},$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\delta, \alpha, \beta, \gamma$ and $\omega$ are real parameters and $\dot x := dx/dt$. Substituting $x \rightarrow u, \dot x\rightarrow v$ in the usual manner, the idea is to integrate the system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot u = v$,&lt;br&gt;&#xA;$\dot v = \gamma \cos{\omega t}  - \delta v - \alpha u - \beta u^3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the fourth order &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods&quot;&gt;Runge-Kutta method&lt;/a&gt;. I've chosen this pretty much because it's standard. My questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) How do I determine whether this method is at all applicable to the problem? Is the problem stiff, and should I choose a different method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This of course is complicated by the fact that Duffing's equation doesn't have an analytical solution for most parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) How do I determine what step size to use - ie. how is the error resulting from a given step size estimated? (I would very much like to avoid implementing an adaptive step size.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being pointed towrads good references will be much appreciated! Best regards, \T&lt;/p&gt;&#xA;" OwnerUserId="742" LastActivityDate="2013-07-20T19:44:40.620" Title="How to decide stability of Runge-Kutta method for non-linear ODE?" Tags="|ode|stability|runge-kutta|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7954" PostTypeId="1" CreationDate="2013-07-11T13:28:27.023" Score="5" ViewCount="1910" Body="&lt;p&gt;Suppose I am minimizing the following function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g(\alpha)=\sum_{i=1}^n(y_i-f(x_i,\alpha))^2,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $y_i$ and $x_i$ are data, $f$ is a known non-linear function and $\alpha$ parameter (of dimension larger than 1) of interest. Is it better to minimise &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{g(\alpha)}{n}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;instead in order to guard against round-off errors, etc? In my case I get conflicting results, for some problems normalisation leads to a vast improvement in proportion of achieved convergence (I do MC simulations so I know the true value $\alpha$) in other cases not so much. Maybe there is some kind of an algorithm or a general advice when to choose the scaling and when not to?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use R's &lt;code&gt;optim&lt;/code&gt; function for optimisation. I tried &lt;code&gt;method=Nelder-Mead&lt;/code&gt;, &lt;code&gt;&quot;CG&quot;&lt;/code&gt; and &lt;code&gt;&quot;BFGS&quot;&lt;/code&gt;. Results are different for &lt;code&gt;CG&lt;/code&gt; and &lt;code&gt;BFGS&lt;/code&gt; and the same for &lt;code&gt;Nelder-Mead&lt;/code&gt;. The differences are slight, but measurable especially for &lt;code&gt;BFGS&lt;/code&gt;. Naturally I use the same starting values and the same data for each optimisation run.  &lt;/p&gt;&#xA;" OwnerUserId="128" LastEditorUserId="128" LastEditDate="2013-07-12T08:40:25.607" LastActivityDate="2018-07-13T07:13:59.480" Title="Scaling of optimisation function in non-linear least squares problem" Tags="|optimization|nonlinear-programming|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="7956" PostTypeId="1" AcceptedAnswerId="7957" CreationDate="2013-07-11T20:01:07.060" Score="1" ViewCount="1969" Body="&lt;p&gt;Computational Science People:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The title is the question.  I am trying to numerically compute a certain integral over a square in Octave (an open-source Matlab clone).  I'm getting the wrong answer.  To show that something really is going wrong, I've created the following toy problem:  Let &#xA;$f = \chi_{(-\infty, 1)}$ ($f(t) = 1$ if $t &amp;lt; 1$, otherwise $0$).  Then $\int_0^3 \int_0^3 f(t) f(t')\,dt'\,dt = 1$.  I wrote an Octave script to compute this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    junk=10;  %this is here just so Octave doesn't complain about the file name&#xA;    function out =R1(t)   &#xA;    if t &amp;lt; 1     &#xA;       out = 1; &#xA;    else  &#xA;       out =0;    &#xA;    end%if                               &#xA;    end%function&#xA;&#xA;    dIF = @(t1,t2) ones(size(t1))* R1(t1(1))*R1(t2);&#xA;&#xA;    dblquad(dIF, 0, 3, 0, 3)  %should be 1.  Octave incorrectly gives 3.&#xA;&#xA;    tx=0:0.1:3;&#xA;    ty=0:0.1:3;&#xA;    [xx, yy] = meshgrid (tx, ty);&#xA;    z=zeros(31,31);&#xA;&#xA;    for i=1:31&#xA;      for j=1:31 &#xA;        z(i,j)=dIF(xx(i,j),yy(i,j)); &#xA;      end%for j&#xA;    end%for i   &#xA;    mesh(xx,yy,z) %graph is correct  &#xA;&#xA; %end script&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Octave returns $3$ for the value of the double integral, and it should be $1$.  As you can see, the script plots the integrand, which is, as intended, $1$ if $x$ and $y$ are both between $0$ and $1$, and $0$ otherwise.  I defined the function dIF the way I did because in my actual problem, I need to pass additional parameters to the function I'm integrating, and this is the only way I know how to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my actual problem, I'll probably just cut my double integral into nine pieces, not requiring piecewise functions, and add them.  But I'd like to know if there's a better way to do it, in case I face a similar problem in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not attempted this in Matlab, only Octave.  I tested it just now with a copy of Octave I downloaded today from what I am quite sure is a reputable source. &lt;/p&gt;&#xA;" OwnerUserId="3821" LastEditorUserId="4244" LastEditDate="2013-07-12T09:28:18.287" LastActivityDate="2013-07-12T09:28:18.287" Title="How can I get Octave to correctly compute a double integral of a piecewise function?" Tags="|matlab|quadrature|octave|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7957" PostTypeId="2" ParentId="7956" CreationDate="2013-07-11T22:45:21.113" Score="2" Body="&lt;p&gt;Your R1 function should be vector valued (it appears as though it is scalar valued).  For example, this appears to work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function out = R1(t)   &#xA;  out = t &amp;lt; 1;&#xA;end&#xA;&#xA;dIF = @(x,y) R1(x) .* R1(y);&#xA;&#xA;dblquad(dIF, 0, 3, 0, 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="486" LastActivityDate="2013-07-11T22:45:21.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7958" PostTypeId="2" ParentId="7953" CreationDate="2013-07-12T00:31:01.673" Score="9" Body="&lt;p&gt;These are all standard questions discussed in most books on ODE solvers. I would recommend Hairer &amp;amp; Wanner.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-12T00:31:01.673" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7959" PostTypeId="2" ParentId="7954" CreationDate="2013-07-12T00:37:55.783" Score="4" Body="&lt;p&gt;The difficulty in optimization is finding the &lt;em&gt;location&lt;/em&gt; of the minimum, not the &lt;em&gt;value&lt;/em&gt; at this point. This is why your scaling makes no difference: the location is exactly the same. Furthermore, every reasonable optimization algorithm should produce exactly the same sequence of intermediate points (iterates) whether you scale the objective function or not. This is, for example, easy to see for Newton's method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you are using your own minimizer and it doesn't possess the property that the sequence of iterates is the same whether the function is scaled or not, then you are in trouble. However, this mostly just indicates a problem with the design of the algorithm.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-12T00:37:55.783" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7960" PostTypeId="1" CreationDate="2013-07-12T10:11:19.573" Score="1" ViewCount="578" Body="&lt;p&gt;I am writing a program in C++ in which I am trying to reduce the run-time by computing eigen-decomposition of several matrices in parallel. This might be a programming question but since many Physics people might use it, I am asking the question here. This might be a simple example of how to perform computational tasks in parallel. Can anybody tell through a simple example, how to do this ? (Assume I have a multi-core processor )&lt;/p&gt;&#xA;" OwnerUserId="4756" OwnerDisplayName="cleanplay" LastActivityDate="2013-07-12T17:44:32.530" Title="Computing eigen-decomposition of several matrices in parallel in C++" Tags="|quantum-mechanics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7961" PostTypeId="1" CreationDate="2013-07-12T13:53:25.437" Score="5" ViewCount="2304" Body="&lt;p&gt;I'm converting a piece of matlab code to python but I am having difficulty recreating the solution it finds for an overdetermined sparse matrix. The original code used the overloaded left division operator but I have recreated the results using linsolve(). I have compared the result with several scikit solvers (OLS, ridge regression, lasso, etc) but have not managed to match it. Lasso came closest as the matlab coefficients are sparse so I think it must be something like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have debugged it and used type to try and access the source code, with no success. Does anyone know what solver it uses or how I could get a peak &quot;under the hood&quot;?&lt;/p&gt;&#xA;" OwnerUserId="4758" LastActivityDate="2013-07-13T05:05:35.990" Title="What is linsolve() doing?" Tags="|linear-solver|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7963" PostTypeId="2" ParentId="7960" CreationDate="2013-07-12T14:24:57.900" Score="0" Body="&lt;p&gt;Parallel solution for eigenvalues and eigenvectors of a given matrix can be performed with the package SLEPc available online (which is built on top of a large package PETSc). There are usage examples in the SLEPc distribution. &lt;/p&gt;&#xA;" OwnerUserId="4325" LastEditorUserId="4325" LastEditDate="2013-07-12T17:22:40.480" LastActivityDate="2013-07-12T17:22:40.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7965" PostTypeId="2" ParentId="7960" CreationDate="2013-07-12T17:44:32.530" Score="3" Body="&lt;p&gt;You would use something like OpenMP to exploit thread level parallelism. An example in C++ would be like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma omp parallel for&#xA;for(int i = 0; i &amp;lt; num_matrices; i++){&#xA;    DoEigendecomposition(matrix[i]);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The pragma will automatically cause each loop iteration to be performed in parallel (assuming you compile with the right flags; e.g. &lt;code&gt;-fopenmp&lt;/code&gt; for GCC).&#xA;Within DoEigendecomposition, you would allocate the necessary memory for temporary work arrays (must be distinct for each thread, so don't make them static or shared), and call the proper routine to do the eigen-decomposition as you would in the single-core case (most likely in Lapack for dense matrices, or some Krylov-like method for sparse). You can also use pthreads, but that is much lower level, and you have to manually create, synchronize, and destroy the threads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: You need to make sure that whatever code does the eigendecomposition is thread-safe (does not use &lt;code&gt;static&lt;/code&gt; variables in C, no &lt;code&gt;COMMON&lt;/code&gt; or &lt;code&gt;SAVE&lt;/code&gt; blocks in Fortran). Lapack's non-symmetric eigensolvers (double and zomplex precision) typically are not thread-safe due to the way the library is usually compiled. For details, see &lt;a href=&quot;http://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=2&amp;amp;t=1930&quot; rel=&quot;nofollow&quot;&gt;this forum post&lt;/a&gt;. Disclaimer: I am the original bug reporter.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-07-12T17:44:32.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7966" PostTypeId="2" ParentId="7947" CreationDate="2013-07-12T18:40:03.873" Score="1" Body="&lt;p&gt;I solved by problem now. I stored $u_{old}^H=I_h^Hu^h$ when going down during the V-cycle and reused it later in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u^h\leftarrow u^h+I_H^h(u^H-I_h^Hu^h)=u^h+I_H^h(u^H-u_{old}^H)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem was before going down again from $H$ to $2H$, $u_{old}^H$ was relaxed &lt;strong&gt;in place&lt;/strong&gt;. Storing a copy before the relaxation steps helped. Since $u_{old}^H$ was only required in FAS, it didn't show up in linear calculations.&lt;/p&gt;&#xA;" OwnerUserId="4742" LastActivityDate="2013-07-12T18:40:03.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7967" PostTypeId="1" CreationDate="2013-07-12T22:28:47.500" Score="1" ViewCount="76" Body="&lt;p&gt;I don't know if I have formulated this problem right:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tons of items and the distances between each pair of them. Feeding this data into some visualization tool, I am able to create a nice image of network with arbitrarily defined center of the coordinate. (Since I only have distances, which do not imply any coordinate) I can eye-ball some clusters in the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to show with high confidence that there exists some valid classification of the items.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's not ask from where the number of clusters comes. The worst case is I have to search for it exhaustively within a reasonable interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the distances in a matrix form, where the (row, column) = (i, j) entry is the distance from point i to point j. Distances are normalized by the largest, therefore they range from 0 to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good way to extract the clusters out of this?&lt;/p&gt;&#xA;" OwnerUserId="4761" LastActivityDate="2013-10-11T19:38:00.350" Title="Extract clusters from a graph of absolute-distance edge" Tags="|machine-learning|statistics|matrix|data-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7968" PostTypeId="2" ParentId="7961" CreationDate="2013-07-13T01:31:49.823" Score="5" Body="&lt;p&gt;MATLAB's linsolve() function uses QR factorization with column pivoting to find a least squares solution to your over determined problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[x,r]=linsolve(A,b)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to get an estimate of the rank of $A$.  I believe that if $A$ is rank deficient, then MATLAB will set all of the free variables to $0$ in computing the least squares solution.  If this is happening, it could explain why MATLAB's least squares solution is sparse.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix is of full column rank, then the least squares solution should be unique and your Python code should be finding essentially the same solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix not of full column rank, then there will be infinitely many least squares solutions, and it would be no surprise that your python code found a very different solution.  As long as $\| Ax-b \|_{2}$ is the same for the solution obtained by your code and by MATLAB, you've got no real reason to complain.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most complicated situation occurs when your matrix is badly conditioned so that it is on the very edge of not having full column rank.  If you're in this situation then there may be many solutions that have very nearly equal values of $\| Ax -b \|_{2}$ and very different values of $x$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest three things to look at:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Compare the values of $\| Ax-b \|_{2}$ for MATLAB's solution and the solutions produced by your Python code.  They should be essentially equal.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use [x,r]=linsolve(A,b) in MATLAB to get a least squares solution together with an estimate of the rank of $A$.  This will give you some indication of whether $A$ is rank deficient.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Take a reasonably small test problem and compute the condition number of your $A$ matrix (in MATLAB, cond(A) will do this.)  This should tell you whether you have a matrix that is effectively of full column rank or not.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The answers to these questions might help us in directing you further.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-07-13T01:31:49.823" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7969" PostTypeId="2" ParentId="7961" CreationDate="2013-07-13T05:05:35.990" Score="3" Body="&lt;p&gt;The answer &lt;a href=&quot;https://scicomp.stackexchange.com/a/1459/276&quot;&gt;https://scicomp.stackexchange.com/a/1459/276&lt;/a&gt; provides some useful advice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you insert the line &lt;code&gt;spparms('spumoni',1);&lt;/code&gt; before your sparse solve, it will output the decisions made by the algorithm. Chances are, it uses a sparse QR factorization.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-07-13T05:05:35.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7971" PostTypeId="2" ParentId="7967" CreationDate="2013-07-13T18:34:05.673" Score="1" Body="&lt;p&gt;You could start by using a simple &lt;a href=&quot;http://en.wikipedia.org/wiki/K-means_clustering&quot; rel=&quot;nofollow&quot;&gt;k-means clustering algorithm&lt;/a&gt;. In this approach, you aim to pick a set of $k$ means $m_i$, and assign each point $x$ in your data to a certain set $S_i$; within each set, you want to minimize the average distance:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i=1}^k\sum_{x\in S_i}|x-m_i|^2 = $ minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given some guess for the set of means $\{m_i\}_{i=1}^k$, the data points closest to each mean can be found by computing the &lt;a href=&quot;http://en.wikipedia.org/wiki/Voronoi_diagram&quot; rel=&quot;nofollow&quot;&gt;Voronoi diagram&lt;/a&gt; of the means. The means are then updated iteratively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;k-means may be good enough for your purposes, but if you try it and find it insufficient, there is a whole host of &lt;a href=&quot;http://en.wikipedia.org/wiki/Category%3aData_clustering_algorithms&quot; rel=&quot;nofollow&quot;&gt;other algorithms&lt;/a&gt; for this problem. These will all be heuristics rather than give you exact solutions, since clustering is usually NP-complete for any realistic criterion (I think).&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-07-13T18:34:05.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7972" PostTypeId="1" CreationDate="2013-07-13T23:35:17.417" Score="0" ViewCount="116" Body="&lt;p&gt;Sorry for the noob question.  I'm using Matlab version R2012b on a computer that's running 64 bit Linux.  It seems that no version of Matlab's parallel computing toolbox will work for me.  Can anyone please tell me if I'm wrong, or else confirm that I'm correct?  Thanks.&lt;/p&gt;&#xA;" OwnerUserId="3274" LastActivityDate="2013-07-15T10:51:12.990" Title="Can I use the Matlab Parallel Computing Toolbox with 64 bit linux (using Matlab R2012b)?" Tags="|matlab|" AnswerCount="1" CommentCount="3" ClosedDate="2013-07-17T05:24:28.757" ContentLicense="CC BY-SA 3.0" />
  <row Id="7973" PostTypeId="1" CreationDate="2013-07-14T02:29:14.843" Score="3" ViewCount="1016" Body="&lt;p&gt;I'm using umfpack for solving large systems of equation. However, I'm constantly getting out of memory issues for even modest size problems as pre2, torso3, ohne2 Hamrle3 (all from Tim Devis's collection). The useable memory is 44GB in machine where I'm trying to run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These problems are not too big that index would be out of 2GB (I'm using 32 bit integers).  One way would be try out 64 bit integer version. However, I might be making a mistake in choosing the parameters itself for the symbolic factorization, which right now is all set to default.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2013-07-14T15:29:15.837" Title="Memory management for solving large sparse systems with UMFPACK" Tags="|linear-solver|sparse-matrix|memory-management|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7975" PostTypeId="2" ParentId="7973" CreationDate="2013-07-14T02:59:29.930" Score="0" Body="&lt;p&gt;I take it you are getting an error that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;UMFPACK_ERROR_out_of_memory&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With the 32 bit int version this happens if you try to use more than ~2gb. The 64 bit version might be better otherwise you might have to move to mumps or alike. &lt;/p&gt;&#xA;" OwnerUserId="4296" LastActivityDate="2013-07-14T02:59:29.930" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7976" PostTypeId="2" ParentId="45" CreationDate="2013-07-14T10:44:03.567" Score="3" Body="&lt;p&gt;The answer to this question is that the system matrix $A$ of the linear equation system&#xA;$$&#xA;Au = b&#xA;$$&#xA;that we obtain from a discretization scheme must be a M-matrix. This means in particular that $A$ is inverse monotone and this means that inverse matrix $A^{-1}$ is a monotone matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A monotone matrix $B\in\mathbb{R}^{n \times n}$ is a matrix, which has solely nonnegative entries and the symbolic expression for this is $B\geq0$. Furthermore, for a monotone matrix $B$ holds:&#xA;$$&#xA;(B\geq 0) \qquad\Leftrightarrow\qquad (u\leq v ~\Rightarrow~ Bu\leq Bv, ~~\forall u,v \in \mathbb{R}^n )&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This condition applied to the inverse monotone system matrix $A$ means that it holds for the above linear equation system&#xA;$$&#xA;0 \leq b ~\Rightarrow~ 0=A^{-1}0 \leq A^{-1}b = u&#xA;$$&#xA;Hence, the solution remains nonnegative, in case the right hand side $b$ of our system is nonnegative. Usually it is $b\geq 0$ in case of nonnegative initial values and nonnegative boundary data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Commonly, discretization schemes that lead to a M-matrix are called monotone schemes and this are those schemes, which preserve non negativity.&lt;/p&gt;&#xA;" OwnerUserId="4768" LastActivityDate="2013-07-14T10:44:03.567" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7977" PostTypeId="2" ParentId="7973" CreationDate="2013-07-14T12:53:12.403" Score="3" Body="&lt;p&gt;I took a look at the sizes of the matrices you listed. I'm not sure I would characterize their size as &quot;modest&quot;. ohne2 has only around 181K equations but nearly 7M non-zeros. Hamrle3 has around 1.5M equations which is a significant number for an unsymmetric, direct solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are running a 32-bit executable on a 64-bit OS, you likely have a full 4GB address space for your process. This seems like it should be enough to factor these matrices. But as the solver progresses, and makes repeated requests for large blocks of memory, the address space becomes fragmented. So when a particularly large block is required for the values in the  L or U factors, the OS simply can't find this much contiguous memory in the address space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As has already been suggested, if you are running on a 64-bit OS, I would simply re-build UMFPACK with 64-bit integers. Even with 44GB of memory, factoring Hamrle3 in a reasonable amount of time might be challenging.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2013-07-14T15:29:15.837" LastActivityDate="2013-07-14T15:29:15.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7978" PostTypeId="1" CreationDate="2013-07-14T19:26:50.993" Score="30" ViewCount="2330" Body="&lt;p&gt;The 3.0 version of the MPI standard formally deleted the C++ interface (it was previously deprecated).  While implementations may still support it, features that are new in MPI-3 do not have a C++ interface defined in the MPI standard.  See &lt;a href=&quot;https://web.archive.org/web/20170421220544/http://blogs.cisco.com/performance/the-mpi-c-bindings-what-happened-and-why/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://blogs.cisco.com/performance/the-mpi-c-bindings-what-happened-and-why/&lt;/a&gt; for more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The motivation for removing the C++ interface from MPI was that it had no significant value over the C interface.  There were very few differences other than &quot;s/_/::/g&quot; and many features that C++ users are accustomed to were not employed (e.g. automatic type determination via templates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As someone who participates in the MPI Forum and works with a number of C++ projects that have implemented their own C++ interface to the MPI C functions, I would like to know what are the desirable features of a C++ interface to MPI.  While I commit to nothing, I would be interested in seeing the implementation of a standalone MPI C++ interface that meets the needs of many users.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And yes, I am familiar with Boost::MPI (&lt;a href=&quot;http://www.boost.org/doc/libs/1_54_0/doc/html/mpi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.boost.org/doc/libs/1_54_0/doc/html/mpi.html&lt;/a&gt;) but it only supports MPI-1 features and the serialization model would be extremely difficult to support for RMA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One C++ interface to MPI that I like is that of Elemental (&lt;a href=&quot;https://github.com/poulson/Elemental/blob/master/src/core/imports/mpi.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/poulson/Elemental/blob/master/src/core/imports/mpi.cpp&lt;/a&gt;) so perhaps people can provide some pro and con w.r.t. that approach.  In particular, I think MpiMap solves an essential problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In response to constructive feedback that this is not appropriate for StackExchange, please move this discussion to &lt;a href=&quot;https://github.com/mpi-forum/mpi-issues/issues/288&quot; rel=&quot;nofollow noreferrer&quot;&gt;MPI Forum issues on GitHub&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2020-04-24T14:38:07.013" LastActivityDate="2020-04-24T14:38:07.013" Title="What features do users need from an MPI C++ interface?" Tags="|c++|mpi|" AnswerCount="5" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="7979" PostTypeId="2" ParentId="7972" CreationDate="2013-07-15T01:01:02.090" Score="1" Body="&lt;p&gt;As far as I recall, Matlab runs as 32bit on 64bit Linux up until R2013a. Try 32bit version of computing toolbox?&lt;/p&gt;&#xA;" OwnerUserId="1573" LastActivityDate="2013-07-15T01:01:02.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7980" PostTypeId="2" ParentId="7978" CreationDate="2013-07-15T03:29:17.080" Score="6" Body="&lt;p&gt;To get the ball rolling, here are two of my needs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The interface should be able to eliminate redundant or unnecessary arguments, e.g. MPI_IN_PLACE.&lt;/li&gt;&#xA;&lt;li&gt;The interface should auto-detect built-in datatypes ala Elemental's MpiMap.&lt;/li&gt;&#xA;&lt;li&gt;If/whenever possible, user-defined datatypes should be constructed for classes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-15T03:29:17.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7982" PostTypeId="1" AcceptedAnswerId="7983" CreationDate="2013-07-15T07:52:48.180" Score="2" ViewCount="614" Body="&lt;p&gt;How is the nullspace correction implemented in the PETSc conjugate gradient solver? I searched the source code and documentation, but could not find references on the actual implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For background, I am solving linear systems arising from a bilinear form which is elliptic in a quotient space $\mathbb{R}^n\big/\ker{\mathbf{Z}}$, where $\mathbf{Z}$ is an $m \times n$ matrix having non-trivial null-space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I compute SVD of $\mathbf{Z}$ and feed the right singular vectors corresponding to &quot;big&quot; singular values as nullspace vectors to the PETSc conjugate gradient solver. It works like a charm, except when $\mathbf{Z}$ is not of academic size.&lt;/p&gt;&#xA;" OwnerUserId="4734" LastActivityDate="2013-07-15T08:24:20.247" Title="PETSc Krylov Subspace and nullspace" Tags="|petsc|fenics|conjugate-gradient|svd|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7983" PostTypeId="2" ParentId="7982" CreationDate="2013-07-15T08:24:20.247" Score="3" Body="&lt;p&gt;It is just a projection $1 - ZZ^T$.  If you are digging around in the implementation, look at &lt;code&gt;KSP_PCApplyBAorAB()&lt;/code&gt; and related functions.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-07-15T08:24:20.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7985" PostTypeId="2" ParentId="1194" CreationDate="2013-07-15T11:55:54.550" Score="5" Body="&lt;p&gt;Folks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this discussion very interesting, but I was surprised to see that re-ordering the loops in the Matmul example changed the picture.  I don't have an intel compiler available on my current machine, so I am using gfortran, but a rewrite of the loops in the mm_test.f90 to &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;call cpu_time(start)  &#xA;do r=1,runs  &#xA;  mat_c=0.0d0  &#xA;     do j=1,n  &#xA;        do k=1,n  &#xA;  do i=1,n  &#xA;           mat_c(i,j)=mat_c(i,j)+mat_a(i,k)*mat_b(k,j)  &#xA;        end do  &#xA;     end do  &#xA;  end do  &#xA;end do  &#xA;call cpu_time(finish)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;changed the entire results for my machine.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The previous version timing results were:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#time ./mm_test_f 10000 100&#xA; matmul    time   6.3620000000000001     &#xA; triple do time   21.420999999999999     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;whereas with the triple loops re-arranged as above yeilded:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#time ./mm_test_f 10000 100&#xA; matmul    time   6.3929999999999998     &#xA; triple do time   3.9190000000000005    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is gcc/gfortran  4.7.2 20121109 on a Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compiler flags used were those from the Makefile I got here...&lt;/p&gt;&#xA;" OwnerUserId="4775" LastEditorUserId="4775" LastEditDate="2013-07-15T12:01:28.717" LastActivityDate="2013-07-15T12:01:28.717" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7986" PostTypeId="1" CreationDate="2013-07-15T18:30:24.177" Score="3" ViewCount="884" Body="&lt;p&gt;The title is not the best but since I have no clue what I'm actually searching I simply used something broad. I'm searching for a an algorithm, a class of algorithms or at least keywords I can use for further research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially I'm searching for a hash-like function that takes an input and boils it down to a much more compact representation. I want similar inputs to be somewhat near each other and I want certain parts of the input to be weighted higher/lower than other parts. I don't care how the distance is as long as I have a partial ordering so I can group similar entries together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example lets assume I have the Input strings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = alittlebeer&#xA;B = alittlebear&#xA;C = blittlebear&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And lets assume that I define my weight so that I weight the first 3 characters much higher than the rest. Then I would expect that $|f(A) - f(B)| &amp;lt;&amp;lt; |f(B) - f(C)| &amp;lt; |f(C) - f(A)|$&lt;/p&gt;&#xA;" OwnerUserId="4777" LastActivityDate="2014-01-18T04:15:34.447" Title="Hash-like algorithm with &quot;weighted&quot; input" Tags="|algorithms|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7988" PostTypeId="1" CreationDate="2013-07-16T02:49:51.640" Score="1" ViewCount="278" Body="&lt;p&gt;Let us have simplicial mesh and continuous function $u$ which is piece-wise linear and non-constant on every cell. Then normal vector to level-sets of $u$ is given $$\mathbf{n}=\frac{\nabla u}{|\nabla u|}$$ and is piece-wise constant function. What is then possible definition of curvature $\kappa$ which would be for $u\in{\cal C}^2$ $$\kappa = \mathrm{div}\,\mathbf{n}.$$ This definition is not suitable as it yields $\kappa$ being zero in every cell and a multiple of Dirac $\delta$ on every facet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion: &lt;strong&gt;what is a suitable finite-element space for $\kappa$ and weak formulation for $\kappa = \mathrm{div}\,\mathbf{n}$ with given piece-wise constant $\mathbf{n}$&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: I got another idea: first &lt;strong&gt;project $\mathbf{n}$ on some $H(\mathrm{div})$-conforming space&lt;/strong&gt;; then it is no problem calculating $\mathrm{div}\,\mathbf{n}$ directly. I did some numerical experiments with $u$ being projection of&#xA;$$ u^\mathrm{exact} = (1+x)(1+y) $$&#xA;to CG1 (continuous, piece-wise linear) on domain $\Omega=(0,1)\times(0,1)$. Exact curvature for $u^\mathrm{exact}$ is&#xA;$$ \kappa^\mathrm{exact} = \frac{-2(1+x)(1+y)}{[(1+x)^2+(1+y)^2]^{3/2}} .$$&#xA;Taking Raviart-Thomas degree 1 space (piece-wise linears, continuous on facet midpoints) or CG1 space as space for $\mathbf{n}$ method seems to converge in norms but with annoying wiggles on $\partial\Omega$. Here is calculated $\kappa$&#xA;&lt;img src=&quot;https://i.stack.imgur.com/aulE0.png&quot; alt=&quot;$\kappa$&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and error $\kappa-\kappa^\mathrm{exact}$&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Z3VRB.png&quot; alt=&quot;$\kappa-\kappa^\mathrm{exact}$&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you see the problem?&lt;/p&gt;&#xA;" OwnerUserId="4254" LastEditorUserId="4254" LastEditDate="2013-07-16T05:49:11.200" LastActivityDate="2013-07-16T05:49:11.200" Title="Curvature of level-sets of CG1 function" Tags="|finite-element|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="7990" PostTypeId="1" CreationDate="2013-07-16T13:23:45.150" Score="3" ViewCount="3744" Body="&lt;p&gt;I need to run the pdetool analysis several times for different values of pde coefficients. Repeating this procedure for different meshes is not convenient using the pdetool GUI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to automate the whole process via a .m file? The .m file should read the pde coefficients via the Matlab command prompt and should return the mesh (p, e, t) and solution (u) after the pde solution is complete. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Appreciate help in advance. &lt;/p&gt;&#xA;" OwnerUserId="4171" LastActivityDate="2013-07-16T14:23:12.800" Title="Export mesh and solution from pdetool" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7991" PostTypeId="2" ParentId="7978" CreationDate="2013-07-16T14:11:06.920" Score="19" Body="&lt;p&gt;Let me first answer why I think C++ interfaces to MPI have generally not been overly successful, having thought about the issue for a good long time when trying to decide whether we should just use the standard C bindings of MPI or building on something at higher level:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you look at real-world MPI codes (say, PETSc, or in my case deal.II), one finds that maybe surprisingly, the number of MPI calls isn't actually very large. For example, in the 500k lines of deal.II, there are only ~100 MPI calls. A consequence of this is that the pain involved in using lower-level interfaces such as the MPI C bindings, is not too large. Conversely, one would not gain all that much by using higher level interfaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My second observation is that many systems have multiple MPI libraries installed (different MPI implementations, or different versions). This poses a significant difficulty if you wanted to use, say, boost::mpi that don't just consist of header files: either there needs to be multiple installations of this package as well, or one needs to build it as part of the project that uses boost::mpi (but that's a problem in itself again, given that boost uses its own build system, which is unlike anything else).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think all of this has conspired against the current crop of C++ interfaces to MPI: The old MPI C++ bindings didn't offer any advantage, and external packages had difficulties with the real world.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This all said, here's what I think would be the killer features I would like to have from a higher-level interface:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It should be generic. Having to specify the data type of a variable is decidedly not C++-like. Of course, it also leads to errors. Elemental's MpiMap class would already be a nice first step (though I can't figure out why the heck the &lt;code&gt;MpiMap::type&lt;/code&gt; variable isn't static const, so that it can be accessed without creating an object).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It should have facilities for streaming arbitrary data types.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Operations that require an &lt;code&gt;MPI_Op&lt;/code&gt; argument (e.g., reductions) should integrate nicely with C++'s &lt;code&gt;std::function&lt;/code&gt; interface, so that it's easy to just pass a function pointer (or a lambda!) rather than having to clumsily register something.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;boost::mpi actually satisfies all of these. I think if it were a header-only library, it'd be a lot more popular in practice. It would also help if it supported post-MPI 1.0 functions, but let's be honest: this covers most of what we need most of the time.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-16T14:11:06.920" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="7992" PostTypeId="2" ParentId="7990" CreationDate="2013-07-16T14:23:12.800" Score="1" Body="&lt;p&gt;pdetool is an easy way for first-time users to get started with MATLAB PDE Toolbox. It can also be used for some realistic analyses since it supports a non-trivial subset of the capability of the toolbox. However, I know of no straightforward way to use it as part of an automated analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recommended approach for automating a PDE Toolbox analysis is to perform all operations using function calls from within a MATLAB script. All of the runable examples that are shipped with PDE Toolbox use this approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Geometry, mesh, and coefficient creation using PDE Toolbox function calls are relatively straightforward-- essentially the same as performing the step in pdetool. Assigning boundary conditions from the script is significantly different than within pdetool and, perhaps, it is not immediately obvious how to do this. The recommended approach is by using a user-written function called a &quot;boundary file&quot; in the PDE Toolbox documentation. This is documented here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/ug/boundary-conditions-for-scalar-pde.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/pde/ug/boundary-conditions-for-scalar-pde.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/ug/boundary-conditions-for-pde-systems.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/pde/ug/boundary-conditions-for-pde-systems.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good, example showing a complete analysis is here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/examples/nonlinear-heat-transfer-in-a-thin-plate.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/pde/examples/nonlinear-heat-transfer-in-a-thin-plate.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-07-16T14:23:12.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7993" PostTypeId="1" CreationDate="2013-07-16T15:32:18.163" Score="4" ViewCount="4750" Body="&lt;p&gt;Which easy-to-handle software would you recommend me for creating energy level diagrams? What do you use and what is your experience? I'm recently working on my bachelor thesis and I would like to add some pictures to ilustrate Hund rules, Zeeman splitting, crystal field splitting and so on with option to customize them on my own. Cheers!&lt;/p&gt;&#xA;" OwnerUserId="4786" LastActivityDate="2013-08-22T20:50:22.190" Title="Energy level diagram - software" Tags="|software|quantum-mechanics|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="7995" PostTypeId="1" CreationDate="2013-07-16T19:09:36.780" Score="6" ViewCount="808" Body="&lt;p&gt;I'm looking to write some code to read in a dense matrix from a file, and I was wondering what are the most common storage formats that my code should support?&lt;/p&gt;&#xA;" OwnerUserId="4788" LastActivityDate="2013-07-17T21:21:32.447" Title="What are the most common dense matrix storage formats?" Tags="|io|dense-matrix|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="7996" PostTypeId="2" ParentId="7995" CreationDate="2013-07-16T20:21:36.640" Score="2" Body="&lt;p&gt;Here are two examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://math.nist.gov/MatrixMarket/formats.html&quot; rel=&quot;nofollow&quot;&gt;http://math.nist.gov/MatrixMarket/formats.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://tedlab.mit.edu/~dr/SVDLIBC/SVD_F_DB.html&quot; rel=&quot;nofollow&quot;&gt;http://tedlab.mit.edu/~dr/SVDLIBC/SVD_F_DB.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't claim either is a &lt;em&gt;good&lt;/em&gt; format...&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-16T20:21:36.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7997" PostTypeId="1" CreationDate="2013-07-17T10:48:52.420" Score="5" ViewCount="2417" Body="&lt;p&gt;What's the best solver that can solve a large sparse but indefinite matrix? &lt;/p&gt;&#xA;" OwnerUserId="4795" LastEditorUserId="1804" LastEditDate="2016-05-27T16:33:50.217" LastActivityDate="2016-05-27T16:33:50.217" Title="What is the best solver for solving a large sparse indefinite system" Tags="|matrix|linear-solver|sparse-matrix|linear-system|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7998" PostTypeId="1" AcceptedAnswerId="8000" CreationDate="2013-07-17T10:54:53.763" Score="3" ViewCount="39" Body="&lt;p&gt;I have three sets of data, measured by three different devices: &lt;code&gt;A,B&lt;/code&gt; and &lt;code&gt;C&lt;/code&gt; of air balloon whose fall is influenced by wind and each data sheet looks like;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A:&#xA;&#xA;Longitude    Latitude    Altitude    Weight    Rotation(about main axis)   Time&#xA;    ...       ...           ...       ...       ...                         ...&#xA;&#xA;B:&#xA;&#xA;Longitude    Latitude    Altitude    Weight    Rotation(about main axis)   Time&#xA;    ...       ...           ...       ...       ...                         ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and similarly for &lt;code&gt;C&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there known standard techniques to  compute the error in measurement of &lt;code&gt;B&lt;/code&gt; and &lt;code&gt;C&lt;/code&gt; with respect to &lt;code&gt;A&lt;/code&gt; (considered standard)?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that except time, all other parameter can both increase and decrease and longitude/latitude and rotation are only parameters that can be positive and negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with regression is that I do not have independent/dependent variables. The method of error analysis I generally approach is calculate $Err_{x},Err_{y},Err_{z}$ and combine them suitably when I have a function explaining something. I do not have a function to propagate error from given quantities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ADDED&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By combining using a suitable function, I mean : computing error of all quantities individually and &lt;a href=&quot;http://chemwiki.ucdavis.edu/Analytical_Chemistry/Quantifying_Nature/Significant_Digits/Propagation_of_Error&quot; rel=&quot;nofollow&quot;&gt;propagating the error&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to measure accuracy of the devices &lt;code&gt;B&lt;/code&gt; (let's disregard C for now) taking &lt;code&gt;A&lt;/code&gt; as standard.&lt;/p&gt;&#xA;" OwnerUserId="4796" LastEditorUserId="4796" LastEditDate="2013-07-17T12:01:33.423" LastActivityDate="2013-07-17T12:01:33.423" Title="Known Algorithm to compute errors of given nature" Tags="|error-estimation|accuracy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="7999" PostTypeId="2" ParentId="7997" CreationDate="2013-07-17T11:05:37.057" Score="7" Body="&lt;p&gt;You may want to watch lecture 34 here: &#xA;  &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-17T11:05:37.057" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="8000" PostTypeId="2" ParentId="7998" CreationDate="2013-07-17T11:12:44.777" Score="0" Body="&lt;p&gt;Your objective is not quite clear, however, as per my understanding, if you're just trying to express the errors of the balloon B and C w.r.t A, you shall have many simple models to your disposal like mean squared error within the estimation period,  root mean squared error, white noise standard deviation etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the noise is random, you may try &lt;a href=&quot;https://www.google.co.in/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=3&amp;amp;ved=0CDoQFjAC&amp;amp;url=http://arxiv.org/pdf/math/0508600&amp;amp;ei=03rmUey8HcaGrgf2noGQDg&amp;amp;usg=AFQjCNHFnTlpx6lPMwgADv_Q_AKZ3liDPA&amp;amp;bvm=bv.49405654,d.bmk&amp;amp;cad=rja&quot; rel=&quot;nofollow&quot;&gt;Berkson Error Model&lt;/a&gt; Or you may also express the error as a probability distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may try explaining &quot;combine them suitably when I have a function explaining something&quot;... that will help others explain things better, perhaps.&lt;/p&gt;&#xA;" OwnerUserId="4773" LastActivityDate="2013-07-17T11:12:44.777" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8001" PostTypeId="2" ParentId="7997" CreationDate="2013-07-17T17:04:54.630" Score="7" Body="&lt;p&gt;Prof. Bangerth's lecture has already covered most of it, but I'd recommend you look at &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0613049&quot;&gt;this paper&lt;/a&gt;. The authors take three of the most common methods (GMRES, CGS and CGNE) and give some matrices for which one method will converge in O(1) operations and the other two converge in O(N). The upshot of this is that &lt;strong&gt;there is no uniformly best iterative method for indefinite systems&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some special cases. For example, if your matrix is symmetric but indefinite, you'll want to use MINRES, or if your matrix comes from a constrained optimization problem there's a family of methods based on the Uzawa algorithm that you'd want to look at. Nonetheless, no one method works best for everything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All told, you'll be better off asking what's the best preconditioner for the specific problem you have in mind. Prof. Bangerth has a bunch of lectures on this topic already, but I'll mention ILU as a fairly popular approach which works for a lot of problems and is pretty easy to understand (if not to analyze).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want more background, you can refer to &lt;a href=&quot;http://books.google.com/books?id=h9nwszYPblEC&amp;amp;lpg=PR2&amp;amp;ots=PU49HbESOS&amp;amp;dq=saad%20iterative%20methods&amp;amp;lr&amp;amp;pg=PR2#v=onepage&amp;amp;q=saad%20iterative%20methods&amp;amp;f=false&quot;&gt;Yousef Saad's book on iterative methods&lt;/a&gt; for sparse linear systems or &lt;a href=&quot;http://www.ec-securehost.com/SIAM/FA02.html&quot;&gt;Tim Davis's book on direct methods&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-07-17T17:04:54.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8002" PostTypeId="1" CreationDate="2013-07-17T20:19:12.963" Score="7" ViewCount="142" Body="&lt;p&gt;I'm working on some adaptive discontinuous Galerkin codes for time harmonic wave propagation, currently just Helmholtz, but will be branching out once I have a working prototype in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some papers out there demonstrating that residual based a posteriori estimators combined with a proper marking strategy yield a provable error reduction provided the refinement satisfies some conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(e.g. &lt;a href=&quot;http://imajna.oxfordjournals.org/content/early/2012/11/23/imanum.drs028.abstract&quot;&gt;http://imajna.oxfordjournals.org/content/early/2012/11/23/imanum.drs028.abstract&lt;/a&gt;&#xA;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am not an expert on a posteriori estimation. However the conditions of these theorems for error reduction appear to rely strongly on the fact that the solutions satisfy the specified discrete weak forms exactly. Unfortunately a direct solve is only possible for me on a few levels of refinement, then the problem becomes quite large. (The initial mesh has to be fine enough to resolve the wave frequency to ensure the estimators are valid, so it's not necessarily the case that I'm beginning with only a few elements)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't however find any literature detailing the relationship between the error incurred in an inexact linear solve and its impact on the reliability of convergence theorems for adaptivity. Is there any out there I might be missing, or maybe I'm just asking for too much?&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2013-11-29T07:30:24.780" Title="Convergence of adaptive finite elements with inexact solves" Tags="|krylov-method|adaptive-mesh-refinement|discontinuous-galerkin|error-estimation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8003" PostTypeId="2" ParentId="7986" CreationDate="2013-07-17T21:10:18.787" Score="0" Body="&lt;p&gt;This would be a naive suggestion rather than a solution:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider each string a series of numbers from ASCII mapping. This defines the closeness of letters.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Declare a container of input, which is a vector of a max length. Now give a weight vector of the same length.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sort every input based on the given weight vector, then greedily match letters from the starts of the inputs to their ends. The earlier the mismatch occurs, the larger the distance between the inputs is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To quantify the distance, let's say we use this non-interpretable naive formula:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Distance = exp(100 + mismatch-weight) + mismatch-weight * ASCII-difference&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a greedy approach. I'm ignoring the fact that there may be a large number of mismatch in the strings. Neither is the distance metric meaningful at all.&lt;/p&gt;&#xA;" OwnerUserId="4761" LastEditorUserId="4761" LastEditDate="2013-07-18T00:13:53.800" LastActivityDate="2013-07-18T00:13:53.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8004" PostTypeId="2" ParentId="7995" CreationDate="2013-07-17T21:21:32.447" Score="0" Body="&lt;p&gt;Perhaps csv format, since I work mostly on csv so far during data analysis. You can also provide flexibility in delimiter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jeff's suggestion are more technical formats. They are all important if your library targets computational scientists and if you can get to them.&lt;/p&gt;&#xA;" OwnerUserId="4761" LastActivityDate="2013-07-17T21:21:32.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8005" PostTypeId="2" ParentId="8002" CreationDate="2013-07-17T23:11:22.470" Score="2" Body="&lt;p&gt;There is a paper from the 1990s (early 2000s) by Roland Becker on the topic. I think it was co-authored by Rolf Rannacher. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a more recent paper by Rannacher and Vihharev that I proof read last year. It may not have appeared, but you could ask them whether they're willing to send you a copy. It is not this one&#xA;  &lt;a href=&quot;http://numerik.iwr.uni-heidelberg.de/Paper/MeidnerRannacherVihharev_paper.pdf&quot; rel=&quot;nofollow&quot;&gt;http://numerik.iwr.uni-heidelberg.de/Paper/MeidnerRannacherVihharev_paper.pdf&lt;/a&gt;&#xA;but that one may also be of interest to you.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-17T23:11:22.470" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8006" PostTypeId="2" ParentId="7931" CreationDate="2013-07-18T02:04:38.167" Score="0" Body="&lt;p&gt;Consider the following special case (using 0-based indices):&#xA;$$&#xA;A = \begin{bmatrix}0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;1\\0&amp;amp;1&amp;amp;0&amp;amp;0&amp;amp;1&amp;amp;0 \end{bmatrix}, \&#xA;b = \begin{bmatrix} 1\\1 \end{bmatrix},\\&#xA;I = \{ (0,4,2,3), (0,0,0,5), (1,1,1,5), (2,2,2,5), (3,3,3,5) \}.&#xA;$$&#xA;In other words, $x_5=1$ and $x_4=1-x_1$. So if we fix $x_1,x_2,x_3$, then the problem reduces to:&#xA;$$&#xA;\min_{x_0} |x_0 (1-x_1) - x_2 x_3| + \sum_{i=0}^3 |x_i^2 - x_i|&#xA;$$&#xA;The expression equals zero if and only if $x_i \in \{0,1\}$ and $x_1 x_2 x_3=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So by introducing extra variables for each clause, one can reduce any 3-SAT problem to checking whether the minimum of such a program is zero.&lt;/p&gt;&#xA;" OwnerUserId="767" LastEditorUserId="767" LastEditDate="2013-07-18T23:48:53.033" LastActivityDate="2013-07-18T23:48:53.033" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8007" PostTypeId="2" ParentId="7986" CreationDate="2013-07-18T05:01:09.867" Score="1" Body="&lt;p&gt;Usually, k-means clustering is used for grouping words and the technique proposed here: &lt;a href=&quot;https://stackoverflow.com/questions/13769242/clustering-words-into-groups&quot;&gt;https://stackoverflow.com/questions/13769242/clustering-words-into-groups&lt;/a&gt; could be used to calculate distance words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I think a kd-tree may be a more suitable technique to sort the words according to their distances. Here, the maximum character length of A,B or C would be the dimension of the tree. The first character of the string is the first dimension, the second character is the second D... and so on. Search and insert operation would be fast and the first 3 characters will naturally carry more weight than the rest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="4773" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-07-18T05:01:09.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8008" PostTypeId="1" CreationDate="2013-07-18T07:33:05.273" Score="2" ViewCount="1133" Body="&lt;p&gt;I need to solve an advection-diffusion equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{∂u}{∂t}=\frac{1}{x}\frac{∂u}{∂x}+\frac{∂^2 u}{∂x^2 } $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with MATLAB. Could you guide me, please? &#xA;Is the Crank-Nicolson method proper method for this equation?&lt;/p&gt;&#xA;" OwnerUserId="4806" LastEditorUserId="81" LastEditDate="2013-07-18T17:47:21.193" LastActivityDate="2013-07-21T02:15:43.153" Title="How to solve an advection-diffusion equation" Tags="|pde|advection-diffusion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8009" PostTypeId="2" ParentId="7978" CreationDate="2013-07-18T08:25:35.670" Score="2" Body="&lt;p&gt;Personally, I don't really mind calling long C-style functions for the exact reason Wolfgang mentioned; there are really few places you need to call them and even then, they almost always get wrapped around by some higher-level code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only things that really bother me with C-style MPI are custom datatypes and, to a lesser degree, custom operations (because I use them less often). As for custom datatypes, I'd say that a good C++ interface should be able to support generic and efficient way of handling this, most probably through serialization. This is of course the route that &lt;code&gt;boost.mpi&lt;/code&gt; has taken, which if you are &lt;a href=&quot;http://www.boost.org/doc/libs/1_41_0/doc/html/mpi/performance.html&quot; rel=&quot;nofollow&quot;&gt;careful&lt;/a&gt;, is a big time saver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for &lt;code&gt;boost.mpi&lt;/code&gt; having extra dependencies (particularly &lt;code&gt;boost.serialization&lt;/code&gt; which itself is not header-only), I've recently came across a header-only C++ serialization library called &lt;a href=&quot;http://uscilab.github.io/cereal/&quot; rel=&quot;nofollow&quot;&gt;cereal&lt;/a&gt; which seems promising; granted it requires a C++11 compliant compiler. It might worth looking into and using it as a based for something similar to &lt;code&gt;boost.mpi&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-07-18T08:25:35.670" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8010" PostTypeId="1" CreationDate="2013-07-18T09:28:45.800" Score="1" ViewCount="21" Body="&lt;p&gt;Consider the following optimization problem:&#xA;\begin{align}&#xA;\text{Min}_{i\neq j\neq s\neq t} |x_i x_j-x_sx_t|\\&#xA;s.t: Ax=b\\&#xA;x\geq 0;  &#xA;\end{align}&#xA;This problem can be seen as an instance of non convex optimization problems. How can the NP-hardness be shown using a suitable reduction?&lt;/p&gt;&#xA;" OwnerUserId="917" LastActivityDate="2013-07-18T09:28:45.800" Title="Reduction for NP-hardness" Tags="|optimization|convex-optimization|nonlinear-programming|nonconvex|" AnswerCount="0" CommentCount="1" ClosedDate="2013-07-19T04:01:14.613" ContentLicense="CC BY-SA 3.0" />
  <row Id="8011" PostTypeId="2" ParentId="7256" CreationDate="2013-07-18T20:04:50.613" Score="2" Body="&lt;p&gt;After more than 2 months and no one found the answer to my problem. I found myself the errors and the solution. To help the community I'am sharing it.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The source term is wrong&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The correct discrete equation using a source term $S(t)$ is given by :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{multline}&#xA;U_{jk}^{n+1}  =  \left( \frac{\Delta t  V_{jk}}{\Delta s} \right) ^2   \left[ \sum_{a=-N}^N  w_a \left( U_{j+a k}^n + U_{j k+a}^n \right) \right] + S_{jk}^n \Delta t^2 V_{jk}^2 + 2 U_{jk}^{n} - U_{jk}^{n-1} \label{xb1}&#xA;\end{multline}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not by the equation given in the beginning that doesn't take in account the source term. The code doesn't implement (above)  it just set's $ U_{10,10}^n = S(t) $. That's not right.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Python class/pointer error (&lt;em&gt;biggest mistake&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When you do the time stack update (bellow), you are not copying the numpy arrays content. You are just changing the class pointers.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# make the update in the time stack&#xA;Uprevious = Ucurrent #  Uprevious is pointing to Ucurrent (previous ok).&#xA;Ucurrent = Ufuture # Ucurrent is pointing to Ufuture.&#xA;Simulation[i-1] = Ucurrent &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So what the Laplacian loop is doing: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ufuture[k][j] += 2*Ucurrent[k][j]-Uprevious[k][j]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is in fact:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ufuture[k][j] += 2*Ufuture[k][j]-Uprevious[k][j] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Content copy would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ucurrent[:][:] = Ufuture[:][:]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running the code above we get, simple animation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.youtube.com/watch?v=MA7Rwcq3z9E&quot; rel=&quot;nofollow&quot;&gt;http://www.youtube.com/watch?v=MA7Rwcq3z9E&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4092" LastEditorUserId="4092" LastEditDate="2013-07-19T03:09:43.090" LastActivityDate="2013-07-19T03:09:43.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8012" PostTypeId="2" ParentId="8008" CreationDate="2013-07-19T00:44:56.230" Score="4" Body="&lt;p&gt;Well, you can use Crank-Nicolson here but then you'll have to construct and solve a linear system for each time-step. That's easy to do but it would be much easier to use an ODE integrator that is available in MATLAB. Due to the diffusion operator in the RHS the implicit integrator ode23tb seems to be a good choice here but experimenting with different integrators will be helpful. What is needed is (1) discretize your PDE by finite-difference on a grid in x, (2) at each grid point $x_i$ you'll have an ODE for time-evolution of $u_i(t)$, (3) write a function, say $myrhs()$, representing the whole set of these ODEs, one for each grid point, (4) look up a usage example for the chosen ODE integrator in MATLAB manuals and call it with $myrhs$ as an argument (other arguments will be the time step, time limits etc). That's it.&lt;/p&gt;&#xA;" OwnerUserId="4325" LastActivityDate="2013-07-19T00:44:56.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8016" PostTypeId="2" ParentId="7931" CreationDate="2013-07-19T06:09:56.603" Score="2" Body="&lt;p&gt;\begin{align}&#xA;\text{Min}&amp;amp;&amp;amp;\frac{1}{2}\sum_{(i,j,s,t)\in I}\|x_ix_j-x_sx_t\|\\&#xA;s.t.: &amp;amp;&amp;amp; Ax=b\\&#xA;&amp;amp;&amp;amp;x\geq 0&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;has the same optimal solution as (and thus has the same computational complexity as, because this transformation is a polynomial reduction)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\text{Min}&amp;amp;&amp;amp;\frac{1}{2}\sum_{(i,j,s,t)\in I}\|x_ix_j-x_sx_t\|^{2}\\&#xA;s.t.: &amp;amp;&amp;amp; Ax=b\\&#xA;&amp;amp;&amp;amp;x\geq 0.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The latter problem is a polynomial programming problem, which is known to be NP-hard, since this program class contains quadratic programming, which is also NP-hard. (See &lt;a href=&quot;http://www.ecommons.cornell.edu/bitstream/1813/5489/1/93-117.pdf&quot; rel=&quot;nofollow&quot;&gt;Complexity Issues in Global Optimization: A Survey, by Stephen Vavasis&lt;/a&gt;.) Obtaining a reduction (as the other answer seems to do correctly) is useful, but unnecessary, since the problem can be transformed into a polynomial programming problem. &lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-07-19T18:18:31.503" LastActivityDate="2013-07-19T18:18:31.503" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8018" PostTypeId="1" CreationDate="2013-07-18T21:55:16.510" Score="7" ViewCount="647" Body="&lt;p&gt;I am writing a program to compute the ground state energy for any closed shell atom using Hartree Fock Roothaan method, with GTO basis. The code works for the simplest case, the helium, but it fails with beryllium (z=4).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that I have two Roothaan equations for two orthonormal orbitals:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F\vec{c}_i=S\vec{c}_i\epsilon_{i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As usual, I have initial guess for the coefficient matrix. Use it to generate the Fock matrix and find the coefficient for the first orbital by solving a generalized eigenvalue problem, choosing only the smallest energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F^{(0)}\vec{c}_1=S\vec{c}_1\epsilon_{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Than I use the coefficient of the new 1_st orbital (the 2_nd orbital is not changed yet) to generate a new Fock matrix to find the coefficient of the 2_nd orbital&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F^{(1)}\vec{c}_2=S\vec{c}_2\epsilon_{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And use the new 2_nd orbital to find a new 1_st orbital again until energy converges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that with that algorithm I always generate orbitals with the same coefficients. But different orbitals are supposedly orthonormal to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I impose orthonormal condition by gram-schmidting the coefficient vectors, I have an oscillating result but the energy range obtained was not even closed to the right answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code to solve the generalized eigenvalue problem automatically generate the corresponding eigenvector and it is normalized with respect to my basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really hope some knowledgeable people can point me to the right way to do it. &lt;/p&gt;&#xA;" OwnerDisplayName="Cheong" LastEditorUserId="6663" LastEditDate="2013-12-26T16:39:34.503" LastActivityDate="2013-12-26T16:39:34.503" Title="Hartree Fock iteration problem" Tags="|iterative-method|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8020" PostTypeId="1" CreationDate="2013-07-19T16:14:01.117" Score="1" ViewCount="89" Body="&lt;p&gt;The documentation for the indigo module can be found here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ggasoftware.com/opensource/indigo/api#inputoutput&quot; rel=&quot;nofollow&quot;&gt;http://ggasoftware.com/opensource/indigo/api#inputoutput&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for instance if I have a molecule object for some SMILES string, e.g. &lt;code&gt;&quot;[C](=[O])&quot;&lt;/code&gt;, I wish to calculate the valency of each atom, for instance here the desired output would be &lt;code&gt;[atom=C, unbound_electrons=2],[atom=O, valency=0]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I consider the atom &lt;code&gt;&quot;[C]&quot;&lt;/code&gt;&#xA;Can anyone explain why is this code printing &lt;code&gt;[atom=C, unbound_electrons=0]&lt;/code&gt; not &lt;code&gt;[atom=C, unbound_electrons=4]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from indigo import *&#xA;indigo = Indigo()&#xA;&#xA;mol=indigo.loadMolecule(&quot;[C]&quot;)&#xA;&#xA;print(mol.grossFormula(),&quot;\n&quot;)&#xA;&#xA;for atom in mol.iterateAtoms():&#xA;        print([atom.symbol(),atom.radicalElectrons()])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I could work it out if I could generate a list of the types of bonds on the atom in conjuction with &lt;code&gt;atom.atomicNumber()&lt;/code&gt;. E.g. if I could say &lt;code&gt;[C]&lt;/code&gt; has a double bond I could take it's atomic number - 2 (second shell) - 2 (double bond)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might be useful for visualising what i'm talking about&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from indigo_renderer import *&#xA;renderer = IndigoRenderer(indigo)&#xA;renderer.renderToFile(mol,&quot;mol.png&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="Freeman" LastActivityDate="2013-08-07T09:30:05.340" Title="Counting valency of atoms, in a molecule with python" Tags="|simulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8021" PostTypeId="2" ParentId="7997" CreationDate="2013-07-20T12:27:34.683" Score="0" Body="&lt;p&gt;The question cannot be answered in this generality. It depends on the system. If it is related to the discretization of partial differential equations, a well designed multigrid method is of linear complexity. It can be used as a preconditioner in a Krylov-space method for additional robustness.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-07-20T12:27:34.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8022" PostTypeId="2" ParentId="7953" CreationDate="2013-07-20T12:54:06.790" Score="9" Body="&lt;p&gt;You can apply linear stability analysis. That is, for given $u=(x,v)$ compute the linearization $Df(u)$ of the right hand side if the equation is $u‘=f(u)$. The problem is stiff if those differ by orders of magnitude. At a glance, I would not expect this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can determine a good step size by running the problem again with half the size. If the results are close, you are done. How close represents on your requirements. Or you use an embedded method like Dormand-Prince to estimate the local error. This is also ode45 in Matlab.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastEditorUserId="3481" LastEditDate="2013-07-20T19:44:40.620" LastActivityDate="2013-07-20T19:44:40.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8023" PostTypeId="1" CreationDate="2013-07-20T16:21:45.737" Score="3" ViewCount="72" Body="&lt;p&gt;I am estimating parameters on a dataset that would, for the most part, result in a weakly constrained solution. The dataset however also contains a few more data points that make the solution well-conditioned. The issue is illustrated in an example of a simple line fit in the sketch below.&lt;br&gt;&#xA;While the few constraining points may actually be good/inliers the obtained solution clearly still heavily depends on these few data points.&lt;br&gt;&lt;br&gt;&#xA;I would now like to have a direct measure that can tell me the relative amount of evidence that contributed to the 'well-constrainedness' of a solution. (In the illustration below that number would indicate a low amount of evidence).&lt;br&gt;&#xA;Does such a direct measure exist or would I have to go through sampling/clustering/segmenting the dataset?&#xA;&lt;br&gt;&lt;br&gt;&lt;img src=&quot;https://i.stack.imgur.com/wzgDr.png&quot; alt=&quot;Line fit on a problematic dataset&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="740" LastEditorUserId="740" LastEditDate="2013-07-21T14:16:39.267" LastActivityDate="2013-07-21T14:16:39.267" Title="Parameter Fitting: Need measure of data 'support' for a parameter solution" Tags="|regression|condition-number|data-analysis|data-sets|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8024" PostTypeId="2" ParentId="5411" CreationDate="2013-07-20T19:53:11.060" Score="3" Body="&lt;p&gt;The original publications by Harlow and Welch discusses this. If you search for &quot;Harlow Welch marker cell&quot; in your favorite search engine, you should get a few links to copies. If this does not help, my own article in Int. J. Num. Meth. Fluids titled &quot;Divergence-free discontinuous Galerkin schemes for the Stokes equations and the MAC scheme&quot;, available in preprint version at &lt;a href=&quot;http://www.isc.tamu.edu/publications-reports/tr/0705.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.isc.tamu.edu/publications-reports/tr/0705.pdf&lt;/a&gt; may help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the condition u dot n = f, you set the degrees of freedom on the boundary to the values of f.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the condition on the tangential velocity, you apply a penalty formulation, which can for instance be reduced to additional terms of the form alpha (u-f) = 0 for all degrees of freedom which are on edges originating from the boundary, but not subset of the boundary.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-07-20T19:53:11.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8025" PostTypeId="1" CreationDate="2013-07-20T20:00:16.517" Score="7" ViewCount="235" Body="&lt;p&gt;In Theorem 6.2 of their 2003 paper on &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/S0036142901399124?journalCode=sjnaam&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Mixed hp-DGFEM for incompressible flows. SIAM J. Numer. Anal., 40(6), 2171–2194&quot;&lt;/a&gt;, D. Schötzau, Ch. Schwab, and A. Toselli prove a bound of the $\inf$-$\sup$ constant of the Stokes problem which decreases by $1/k$ if $k$ is the polynomial degree of an hp-discontinuous Galerkin method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical examples suggest that this estimate may not be sharp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is anybody aware of an improved estimate, possibly independent of $k$?&lt;/p&gt;&#xA;" OwnerUserId="4108" LastEditorUserId="20688" LastEditDate="2017-06-09T17:34:38.613" LastActivityDate="2017-06-09T17:34:38.613" Title="Has a uniform estimate in k of the inf-sup constant for hp-DG methods for the Stokes problem been established?" Tags="|navier-stokes|discontinuous-galerkin|incompressible|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8027" PostTypeId="2" ParentId="5626" CreationDate="2013-07-20T21:37:55.530" Score="4" Body="&lt;p&gt;The example you give is well-posed. &lt;a href=&quot;http://en.wikipedia.org/wiki/Korn%27s_inequality&quot; rel=&quot;nofollow&quot;&gt;Korn's inequality&lt;/a&gt; holds, if the subset on which the displacement is fixed contains an open (in the topology of the boundary) subset of the boundary, which is true in your case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple test is: if you fix a rigid body at your Dirichlet boundary, can it still be moved. For instance, if you fix a point in two dimensions, your object can rotate around it. If you fix a point or a line in 3 dimensions, the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If in the end you want periodic boundary conditions in $x$ and $y$ directions, you will have to impose an additional constraint, for instance that the mean value of the displacement over the whole rectangle is zero. Possibly, you will have to eliminate rotations as well.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastEditorUserId="4108" LastEditDate="2013-07-21T20:41:42.997" LastActivityDate="2013-07-21T20:41:42.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8028" PostTypeId="2" ParentId="8023" CreationDate="2013-07-21T02:11:12.587" Score="0" Body="&lt;p&gt;You want to look for the concept of &quot;parameter sensitivity&quot; in parameter estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another direction worth looking into is Bayesian inversion that gives you either a very narrow or a broad probability distribution in parameter space. There are many good introductions. My (partial, biased) favorite is number 35 here:&#xA;  &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications.html#x-reviewed&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/publications.html#x-reviewed&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-21T02:11:12.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8029" PostTypeId="2" ParentId="8008" CreationDate="2013-07-21T02:15:43.153" Score="6" Body="&lt;p&gt;You will have a problem if $x=0$ is part of your domain because in that case your advection velocity $u=1/x$ becomes singular. In particular, there will be cells close to $x=0$ where the cell Peclet number is very large, and consequently you will be in the advection-dominated regime. You will need to stabilize your discretization.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-21T02:15:43.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8030" PostTypeId="2" ParentId="8023" CreationDate="2013-07-21T07:53:36.243" Score="1" Body="&lt;p&gt;As pointed out in the previous answer,  the concept you are looking for is sensitivity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You take the formula that computes the parameters of your regression line and compute the derivatives with respect to your point positions. These should be large for the three isolated points, indicating that moving them will have big influence.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-07-21T07:53:36.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8031" PostTypeId="1" CreationDate="2013-07-21T10:11:50.350" Score="4" ViewCount="72" Body="&lt;p&gt;I've got a function of form &#xA;$$f: (\mathbb{Z}_3)^n \rightarrow \mathbb{R}$$ to optimize, where $n$ is relatively large (the order of hundreds). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it there a gradient-like notion for these type of functions? Or does it make sense to define one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="1922" LastActivityDate="2013-07-21T19:05:38.830" Title="gradient for ternary functions?" Tags="|optimization|combinatorics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8032" PostTypeId="2" ParentId="8025" CreationDate="2013-07-21T12:37:04.623" Score="5" Body="&lt;p&gt;For interior penalty type HDG methods we have recently shown that $1/\sqrt{k}$ is possible (Proposition 6.10) and this bound is valid for several element types. In Remark 6.11 there is also a discussion about known results from the literature for other methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge there exist no uniform results for hp-DG methods yet. I would however also be interested in new ideas and recent developments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Egger, Waluga. hp-Analysis of a Hybrid DG Method for Stokes Flow, 2013 (&lt;a href=&quot;http://imajna.oxfordjournals.org/cgi/content/abstract/drs018?ijkey=kwbzr8Sj7m8wI0p&amp;amp;keytype=ref&quot; rel=&quot;noreferrer&quot;&gt;link&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2013-07-21T12:42:57.153" LastActivityDate="2013-07-21T12:42:57.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8033" PostTypeId="2" ParentId="8031" CreationDate="2013-07-21T16:54:46.967" Score="2" Body="&lt;p&gt;In many cases it's possible to extend $f$ to a function that maps $R^{n}$ (actually $[0,2]^{n}$) to $R$.  If that function is convex and preferably differentiable (so that it has a gradient), then you can use conventional continuous optimization techniques to minimize the extended function.  However, rounding off the minimum of the extended function is not sufficient to get a minimum of the original function.  Rather, you would typically embed this into a branch and bound algorithm to find a minimum of the original function.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-07-21T16:54:46.967" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8034" PostTypeId="2" ParentId="2026" CreationDate="2013-07-21T19:03:43.520" Score="3" Body="&lt;p&gt;I would recommend trying CGAL&#xA;&lt;a href=&quot;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Triangulation_3/Chapter_main.html#Section_39.2&quot; rel=&quot;nofollow&quot; title=&quot;CGAL&quot;&gt;http://www.cgal.org/Manual/latest/doc_html/cgal_manual/Triangulation_3/Chapter_main.html#Section_39.2&lt;/a&gt;&#xA;, as Paul suggested above.&#xA;CGAL is a robust and well-supported library that has been around quite some time. I've used it happily in the past, even on point sets with co-linear and co-planar points. I don't know if it's the very fastest today, but it is certainly a good place to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The link above also includes some performance numbers: it can do a million points in about 10 seconds, and 10 million in about 1.5 mins.&lt;/p&gt;&#xA;" OwnerUserId="326" LastEditorUserId="326" LastEditDate="2013-07-26T13:57:20.557" LastActivityDate="2013-07-26T13:57:20.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8035" PostTypeId="2" ParentId="8031" CreationDate="2013-07-21T19:05:38.830" Score="2" Body="&lt;p&gt;You have what is called an &lt;em&gt;integer optimization problem&lt;/em&gt;. Like Brian already mentioned, there are methods that use continuations of your objective function to all of ${\mathbb R}^n$ as part of the algorithm, but ultimately you will have to deal with the integer nature of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many text books on this topic. Find one in your local library to get the basics of solution algorithms for the kind of problem you are considering.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-21T19:05:38.830" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8036" PostTypeId="1" CreationDate="2013-07-21T20:37:17.690" Score="1" ViewCount="68" Body="&lt;p&gt;This might not be the best place to ask, and if its not could someone please point me to where would be better, but here we go anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone out there know of open source software which can be used for speckle interferometry with normal digital cameras? While speckle seams to be the standard, im happy to use other systems as well, but don't have easy access to lasers etc. It is to measure stress/strain curves of 15x15mm orthotropic samples but I don't need the strain fields within the samples so something less precise than speckle may well be appropriate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any ideas&lt;/p&gt;&#xA;" OwnerUserId="4296" LastActivityDate="2013-07-21T20:37:17.690" Title="speckle interferometry (or other interferometry) software" Tags="|software|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8037" PostTypeId="2" ParentId="8025" CreationDate="2013-07-21T22:45:57.630" Score="1" Body="&lt;p&gt;The inf-sup constant for a continuous-velocity space is a lower bound for the corresponding DG space, therefore one can take the discontinuous analogue of any uniformly stable continuous space.  Two natural choices are $Q_k - Q_{\min(k-2, \lambda k)}$ where $0 &amp;lt; \lambda &amp;lt; 1$ and $Q_k - P_{k-1}$.  This was pointed out in Section 9 of &lt;a href=&quot;ftp://magellan-03.math.ethz.ch/hg/pub/sam-reports/reports/reports2002/2002-02.pdf&quot; rel=&quot;nofollow&quot;&gt;Toselli, &lt;em&gt;$hp$ Discontinuous Galerkin Approximations&#xA;for the Stokes Problem&lt;/em&gt;, 2002&lt;/a&gt;.  Note that the latter space is not uniformly stable with respect to aspect ratio.  Recall also the anisotropically-enriched spaces of &lt;a href=&quot;http://dx.doi.org/10.1137/S0036142999365400&quot; rel=&quot;nofollow&quot;&gt;Ainsworth and Coggins (2000)&lt;/a&gt; that provide a logarithmic stability bound, though I think these are of limited practical utility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The spaces above are not very satisfying because normally one would hope to raise the pressure approximation order when going to a discontinuous velocity.  One practical reason for this is to simultaneously obtain optimal approximation order and uniform inf-sup stability with respect to aspect ratio. For example, DG using $Q_k - Q_{k-1}$ is uniformly stable with respect to aspect ratio (see the sequel,&#xA;&lt;a href=&quot;http://dx.doi.org/10.1093/imanum/24.2.273&quot; rel=&quot;nofollow&quot;&gt;Schötzau, Schwab, Toselli, &lt;em&gt;Mixed $hp$-DGFEM for incompressible flows II:&#xA;Geometric edge meshes&lt;/em&gt;, 2004&lt;/a&gt;), for which there is no known analogue in the continuous Galerkin world, causing one to use the suboptimal space $Q_k - Q_{k-2}$ to obtain uniform stability with respect to aspect ratio.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-07-21T22:45:57.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8039" PostTypeId="2" ParentId="8018" CreationDate="2013-07-22T05:47:08.320" Score="0" Body="&lt;p&gt;You might find the Hartree-Fock tutorial on &lt;a href=&quot;http://vergil.chemistry.gatech.edu/resources/programming/&quot; rel=&quot;nofollow&quot;&gt;http://vergil.chemistry.gatech.edu/resources/programming/&lt;/a&gt; useful.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-22T05:47:08.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8040" PostTypeId="2" ParentId="7156" CreationDate="2013-07-22T18:22:03.767" Score="1" Body="&lt;p&gt;After more than 2 months and no answer. I post my own answer this is as far as I could get (not final answer though). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the general formula for stability criteria (in a paper[1]). That is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ r \leq \frac{2}{\sqrt{\sum_{a=-N}^{N} (|w_a^1| + |w_a^2|)}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $ r = \frac{V \Delta t}{\Delta s} $ and $w_a$ is the centered finite differences weights and the indexes 1 e 2 refer to the x and y dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I couldn't get to this general formula, I could just get to the criteria to the 2nd order that was the post N=1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Not certain if this a proof by contradiction. (Also forgive my bad math I am really eager to learn)&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Suppose $ \Delta &amp;gt; 0 $ condition holds&lt;/strong&gt; for $ | \epsilon |  \leq 1 $ that using $ \epsilon^{-1} = \mu $ means $ |\mu| \geq 1 $. Thus this requires for $ r^2 \beta - 1 &amp;gt; 0 $   to be $ r &amp;gt; \frac{1}{ \sqrt{ \beta} } $  that can be satisfied by using $ r = \frac{1}{ \sqrt{ \beta} } + \psi $ with $ \psi &amp;gt; 0 $ positive, real. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going back to for the first root $ \mu{'} $, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\mu^{'} &amp;amp;=&amp;amp; -2r^2\beta+1 + 2\sqrt{r^2\beta(r^2\beta-1)} \nonumber \\&#xA;	&amp;amp;=&amp;amp; -2 \left(1+\frac{2\psi\beta}{\sqrt{\beta}} + \psi^2\beta\right)+1+ 2\sqrt{\left(1+\frac{2\psi\beta}{\sqrt{\beta}} + \psi^2\beta\right)\left[\left(1+\frac{2\psi\beta}{\sqrt{\beta}} + \psi^2\beta\right)-1 \right]} \nonumber \\&#xA;	&amp;amp;=&amp;amp; -2 	\left(1+2\psi\sqrt{\beta}+\psi^2\beta\right) +1 + 2 \sqrt{\left(1+2\psi\sqrt{\beta}+\psi^2\beta\right)\left[\left(1+2\psi\sqrt{\beta}+\psi^2\beta\right)-1\right]} \nonumber \\&#xA;	&amp;amp;=&amp;amp; -2 A + 1 + 2\sqrt{A^2-A} \nonumber&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A = \left(1+2\psi\sqrt{\beta}+\psi^2\beta\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that since $ \psi &amp;gt; 1 $ then $ A &amp;gt; 1 $ always. Using the requirement for stability:&#xA;\begin{eqnarray}&#xA;| \mu^{'} | &amp;amp;\geq&amp;amp; 1 \nonumber \\&#xA;	\left|-2 A + 1 + 2\sqrt{A^2-A}\right| &amp;amp;\geq&amp;amp; 1  \nonumber \\&#xA;\end{eqnarray}&#xA;To satisfy the inequality, two possibilities&#xA;\begin{eqnarray}&#xA;    -2 A + 1 + 2\sqrt{A^2-A} &amp;amp;\leq&amp;amp; -1  \nonumber \\&#xA;	-2 A + 1 + 2\sqrt{A^2-A} &amp;amp;\geq&amp;amp; 1  \nonumber \\ &#xA;	-2 A + 2\sqrt{A^2-A} &amp;amp;\leq&amp;amp; -2  \text{  (2)} \nonumber \\&#xA;    -2 A + 2\sqrt{A^2-A} &amp;amp;\geq&amp;amp; 0  \text{  (3)}&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At (2) for $ A &amp;gt; 1 $ left hand side cannot hold, always $ &amp;gt; -2 $. &#xA;At (3) for $ A &amp;gt; 1 $ also cannot hold, $ -1 &amp;gt; left hand side &amp;gt; -2 $&#xA;We don't even need to look at the second root. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This implies that $ \Delta &amp;gt; 0 $ doesn't satisfy the stability criteria. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now suppose $ \Delta = 0 $ condition holds}&lt;/strong&gt; for $ | \epsilon |  \leq 1 $ that using $ \epsilon^{-1} = \mu $ means $ |\mu| \geq 1 $. Thus this requires that $ r = \frac{1}{ \sqrt{ \beta} } $ \&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For booth roots $ \mu $, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\mu &amp;amp;=&amp;amp; -2r^2\beta+1 + 2\sqrt{r^2\beta(r^2\beta-1)} \nonumber \\&#xA;	&amp;amp;=&amp;amp; -2+1 \\ &#xA;	&amp;amp;=&amp;amp; -1 \nonumber&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That clearly holds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Finally suppose $ \Delta &amp;lt; 0 $ condition holds&lt;/strong&gt; for $ | \epsilon |  \leq 1 $  Thus this requires $ r^2 \beta - 1 &amp;lt; 0 $  that can be satisfied by using $ r = \frac{1}{ \sqrt{ \beta} } - \psi $ with $ \psi &amp;gt; 0 $ positive, real. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again going back for the first root $ \mu{'} $, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\mu^{'} &amp;amp;=&amp;amp; -2r^2\beta+1 + 2\sqrt{r^2\beta(r^2\beta-1)} \nonumber \\&#xA;	&amp;amp;=&amp;amp; -2 	\left(1-2\psi\sqrt{\beta}+\psi^2\beta\right) +1 + 2 \sqrt{\left(1-2\psi\sqrt{\beta}+\psi^2\beta\right)\left[\left(1-2\psi\sqrt{\beta}+\psi^2\beta\right)-1\right]} \nonumber \\&#xA;\end{eqnarray}&#xA;Rearranging due the imaginary part&#xA;\begin{eqnarray}&#xA;	&amp;amp;=&amp;amp; -2 	\left(1-2\psi\sqrt{\beta}+\psi^2\beta\right) +1 + 2 i\sqrt{\left[1-\left(1-2\psi\sqrt{\beta}+\psi^2\beta\right)\right]\left(1-2\psi\sqrt{\beta}+\psi^2\beta\right)} \nonumber \\&#xA;	&amp;amp;=&amp;amp; -2 A + 1 + 2i\sqrt{A^2-A} &#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $i=\sqrt{-1}$ imaginary unit and:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A = \left(1-2\psi\sqrt{\beta}+\psi^2\beta\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that since $ \psi &amp;gt; 1 $ then $ A &amp;lt; 1 $ always. Then using the requirement for stability and complex number modulus (the other root is just conjugate of this so same modulus):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;| \mu | &amp;amp;\geq&amp;amp; 1 \nonumber \\&#xA;	\left|-2 A + 1 + 2i\sqrt{A^2-A}\right| &amp;amp;\geq&amp;amp; 1  \nonumber \\&#xA;	\sqrt{(-2 A + 1)^2 + 4(A^2-A) } &amp;amp;\geq&amp;amp; 1  \nonumber \\&#xA;	\sqrt{ 1 } &amp;amp;\geq&amp;amp; 1  \nonumber &#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this condition also holds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the solution is $ r \leq \frac{1}{ \sqrt{ \beta} } $ that is maximum given $\beta = 2 $ and then $ r \leq \frac{1}{ \sqrt{ 2 } } $ That agrees with general formula presented first for N=1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] A stability formula for Lax-Wendroff methods with fourth-order in time and general-order in space for the scalar wave equation - pg T38 - Geophysics Vol. 76 No. 2 2011&lt;/p&gt;&#xA;" OwnerUserId="4092" LastActivityDate="2013-07-22T18:22:03.767" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8041" PostTypeId="2" ParentId="1873" CreationDate="2013-07-23T01:40:34.573" Score="6" Body="&lt;p&gt;I'm going to write this as an answer although it doesn't directly answer the question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plugging the second equation and the third equation into the first, and plugging the third into the second, together give:&#xA;$$&#xA;\begin{align}&#xA;\frac{\partial^2 c}{\partial t^2} &amp;amp;= \frac{\partial^2}{\partial x^2}\frac{\partial c}{\partial t} + \frac{\partial b}{\partial t} \\&#xA;\frac{\partial b}{\partial t} &amp;amp;= - \left(\frac{\partial c}{\partial t}\right) b&#xA;\end{align}&#xA;$$&#xA;Rearranging these two gives:&#xA;$$&#xA;\begin{align}&#xA;\frac{\partial}{\partial t}\left(\frac{\partial c}{\partial t} - \frac{\partial^2 c}{\partial x^2} - b \right) &amp;amp;= 0 \\&#xA;\frac{1}{b}\left(\frac{\partial b}{\partial t}\right) &amp;amp;= -\frac{\partial c}{\partial t}&#xA;\end{align}&#xA;$$&#xA;Now, we can integrate both of these once in $t$, leaving for the first equation:&#xA;$$&#xA;\frac{\partial c}{\partial t} - \frac{\partial^2 c}{\partial x^2} = b + A(x)&#xA;$$&#xA;Using the third equation, we can express the &quot;constant&quot; of integration as $A(x)=a_0-\frac{\partial^2 c_0}{\partial x^2}-b_0$. The second equation is a bit more tricky. Rewriting a little, we have:&#xA;$$&#xA;\int_0^t \frac{1}{b(x,t')}\left(\frac{\partial b(x,t')}{\partial t'}\right) dt' = -\int_0^t \frac{\partial c(x,t')}{\partial t'} dt'&#xA;$$&#xA;This leads to the solution&#xA;$$&#xA;\ln b(x,t) - \ln b_0(x) = - c(x) + c_0(x) &#xA;$$&#xA;Or &#xA;$$&#xA;\ln \frac{b}{b_0} = -c + c_0&#xA;$$&#xA;Exponentiating gives:&#xA;$$&#xA;b = b_0 e^{c_0-c}&#xA;$$&#xA;And, finally, plugging this into the PDE for $c$ gives&#xA;$$&#xA;\frac{\partial c}{\partial t} - \frac{\partial^2 c}{\partial x^2} = b_0 e^{c_0-c}+A(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replacing $c$ by $c-c_0$, or equivalently using the initial conditions $c_0(x)=0$, this equation simplifies to&#xA;$$&#xA;\frac{\partial c}{\partial t} = \frac{\partial^2 c}{\partial x^2} + a_0 - (1-e^{-c}) b_0&#xA;$$&#xA;Now, you should be able to find considerable literature on the best way to solve this equation. I don't know if Crank-Nicholson is a good choice for the exponential term, but it seems plausible. Some care should probably be taken to guarantee that $c &amp;gt; c_0$ everywhere or the solution may blow up quickly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've only walked through this derivation twice, so there may be an error or two in it, but it has the right feel to me. If $b_0 = 0$ everywhere, then this is clearly the right solution, and it has a whiff of plausibility otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this until $t=1$ and evaluating $c(x,1)$ should give you the answer you are looking for.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="534" LastEditDate="2013-07-23T12:02:09.533" LastActivityDate="2013-07-23T12:02:09.533" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8042" PostTypeId="2" ParentId="7728" CreationDate="2013-07-23T04:04:48.363" Score="2" Body="&lt;p&gt;Yes.  ASE (GPL) supports both Turbomole (&lt;a href=&quot;https://wiki.fysik.dtu.dk/ase/ase/calculators/turbomole.html&quot; rel=&quot;nofollow&quot;&gt;https://wiki.fysik.dtu.dk/ase/ase/calculators/turbomole.html&lt;/a&gt;) and NEB (&lt;a href=&quot;https://wiki.fysik.dtu.dk/ase/ase/neb.html&quot; rel=&quot;nofollow&quot;&gt;https://wiki.fysik.dtu.dk/ase/ase/neb.html&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-23T04:04:48.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8043" PostTypeId="2" ParentId="7993" CreationDate="2013-07-23T06:36:29.500" Score="1" Body="&lt;p&gt;ChemDraw is a proprietary package that gives you easy access to renderings of orbitals, which can be useful here, but I do not recommend this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I draw lots of simple figures in OpenOffice (free and portable) in the ODG format and save as PDF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to have orbitals to go along with your figures, then you should post a new question on orbital rendering programs because that is not within the scope of the question here.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-23T06:36:29.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8044" PostTypeId="2" ParentId="330" CreationDate="2013-07-23T06:42:15.660" Score="1" Body="&lt;p&gt;Documentation that is automatically derived from your source code is an essential component in having up-to-date i.e. correct documentation.  I can't count how many times I've seen documentation that is &lt;em&gt;years&lt;/em&gt; behind the source code because of developer apathy towards documentation.  The easy solution is to make it easy for programmers to write documentation along with new code in the same place at the same time rather than as some a posteriori effort that will inevitably be de-prioritized in favor of more glorious activities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If forced to choose, I'd rather have detailed and accurate (i.e. current) source code comments but &lt;em&gt;no user manual whatsoever&lt;/em&gt; than an out-of-date user manual that's full of wrong information.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-23T06:42:15.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8045" PostTypeId="2" ParentId="399" CreationDate="2013-07-23T06:46:44.867" Score="7" Body="&lt;p&gt;I use only two debuggers for serial and parallel programs:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The Kernighan debugger, i.e. judicious print statements and careful thinking.&lt;/li&gt;&#xA;&lt;li&gt;Multiple instances of GDB as described o &lt;a href=&quot;http://www.open-mpi.org/faq/?category=debugging#serial-debuggers&quot; rel=&quot;noreferrer&quot;&gt;http://www.open-mpi.org/faq/?category=debugging#serial-debuggers&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the case where (2) is not sufficiently scalable, I refer to (1b).&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-07-23T06:46:44.867" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8046" PostTypeId="1" CreationDate="2013-07-23T11:43:01.013" Score="10" ViewCount="508" Body="&lt;p&gt;My question is about extracting observables from QMC methods, as described in &lt;a href=&quot;http://www.itp.phys.ethz.ch/education/fs12/cqp/chapter05.pdf&quot;&gt;this reference&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the formal derivation of various QMC methods like Path Integral Monte Carlo. However, at the end of the day I'm still confused about on how to effectively use these techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea of the derivation of Quantum MC methods is to discretise, via the Trotter approximation, an operator which may be either the density matrix or the time-evolution operator of a quantum system. We then obtain a classical system with an additional dimension which may be treated with MC methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that we can interpret $\beta$ in the quantum operator $e^{−\beta\hat{H}}$ both as an inverse temperature and an imaginary time, the aim of these algorithms should be to compute an approximation of this operator. Indeed, if we would directly measure quantities from the various configurations sampled along a simulation, in the &quot;inverse temperature&quot; case we would have samples respecting a probability density based on $\beta/M$, where $M$ is the number of discrete steps introduced in the Trotter decomposition. Instead, in the &quot;imaginary time&quot; case we would obtain samples at various discrete time-steps, thus getting averages along the time as well. We also wouldn't obtain quantities like $\langle\psi_t|\hat{A}|\psi_t\rangle$ at a given time $t$, with $\hat{A}$ some observable operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in my opinion the quantities we sample &lt;em&gt;directly&lt;/em&gt; from this kind of simulations (taken from (5.34) of the document, page 35):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar{O} \equiv \langle \hat{O}(X) \rangle \equiv \frac{1}{N!} \sum_P \int O(X) \pi(X,P) dX $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;cannot be quantities related to the quantum system, given the additional dimension. Instead, the correct quantum quantities can be computed via formulas like (5.35), which contains in every sample a whole chain of $M$ simulated configurations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{E_{th}}{N}= \left\langle \frac{d}{2 \tau} - \frac{m}{2 (\hbar \tau)^2MN } \sum_{j=1}^{M} (\mathbf{R}_j -\mathbf{R}_{j+1})^2 + \frac{1}{MN} \sum_{j=1}^{M} V(\mathbf{R}_j) \right\rangle $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I right that a series of QMC simulations is required to extract useful information about a given observable?&lt;/p&gt;&#xA;" OwnerUserId="4826" LastEditorUserId="81" LastEditDate="2013-08-03T09:18:15.673" LastActivityDate="2014-08-02T11:05:03.203" Title="Confusion about Quantum Monte Carlo" Tags="|monte-carlo|quantum-mechanics|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="8047" PostTypeId="2" ParentId="81" CreationDate="2013-07-23T21:27:22.157" Score="6" Body="&lt;p&gt;There is an important piece of information missing in your question: where did the matrix originate from. The structure of the problem you were trying to solve has great potential to suggest a solution method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix originated from a partial differential equation with smooth coefficients, a geometric multigrid method will be hard to beat, in particular in three dimensions. If your problem is less regular, algebraic multigrid is a good method. Both usually combined with Krylov-space methods. Other efficient solvers can be derived from fast multipole methods or fast Fourier transform.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-07-23T21:27:22.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8048" PostTypeId="1" AcceptedAnswerId="8051" CreationDate="2013-07-23T20:55:08.710" Score="7" ViewCount="2266" Body="&lt;p&gt;Specifically, ode15i. I have ode15i solving a system of 5 first order implicit odes in 5 variables with an initial condition (made consistent by decic). It's great for what I need, except I need to add a final condition as well. Is this possible? I think (but am not sure) ode15s works too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: I think I figured it out. I'm going to make dummy variables for all the derivatives, add in a bunch of extra equations, and that should make the equations &quot;explicit&quot; so I can use bvp4c. &lt;/p&gt;&#xA;" OwnerUserId="4833" OwnerDisplayName="Samuel Reid" LastEditorUserId="123" LastEditDate="2013-07-24T23:16:49.453" LastActivityDate="2013-07-24T23:16:49.453" Title="How do I solve a boundary value ODE in MATLAB?" Tags="|matlab|ode|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8049" PostTypeId="2" ParentId="7894" CreationDate="2013-07-24T10:56:30.327" Score="1" Body="&lt;p&gt;So, finding a good solution has been instructive. First point is that as Bill suggested, setting up the system in Octave (rather than solving it) takes most of the time - but it is not easy code to vectorise. Also, for my problem formulation with mixed boundary conditions, the system matrix is not symmetric, which rules out some efficient approaches. I looked at different libraries and now use the sparse LU solver in umfpack, included in my Linux system as part of the suitesparse library. I gather both Octave and Matlab use umfpack which I take as a recommendation - it is a traditional library, easy to call from C as well as C++, and has excellent, clear documentation. A straightforward C routine solves a 320x320 FDE grid &gt;100 times faster than in Octave and easily copes with 1000x1000 in 30s on a cheap laptop.&lt;/p&gt;&#xA;" OwnerUserId="4700" LastActivityDate="2013-07-24T10:56:30.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8050" PostTypeId="2" ParentId="8048" CreationDate="2013-07-24T12:17:58.750" Score="5" Body="&lt;p&gt;I assume that you want to solve $$f(t,y',y)=0 \text{ on } (0,T)$$ with two-point boundary values $$y(0)=\alpha \text{ and } y(T)=\beta.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You cannot simply apply ODE solvers to this problen unless you take the heuristic approach of forward-backward iteration (see the list below). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no general approach to these boundary value problems. And I &lt;s&gt;don't&lt;/s&gt; didn't know of any built-in function in Matlab that solves these boundary value problems even for the case with $y' = \tilde f (t,y)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT: There are matlab functions for solving these semi-explicit two point boundary value problems, see David Ketcheson's answer, that use finite differences and collocation. ]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, &lt;strong&gt;my answer is&lt;/strong&gt;, there is no answer to your particular question, how to make Matlab's ODE solvers handle your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve the problem you have the following options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Finite Differences&lt;/strong&gt;: Discretize the interval $[0,T]$, if necessary do a collocation, and solve the resulting algebraic system&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;(Multiple) Shooting&lt;/strong&gt;: parametrize the boundary conditions and solve for the parameters&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Decoupling&lt;/strong&gt;: split the variables so that there is one initial value problem, and one terminal value, i.e. backward in time initial value, problem. In the linear case this is commonly referred to as Riccati decoupling&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Fixed-point Iteration&lt;/strong&gt;: this is the most heuristic approach. For example, you can try to guess the missing intial values, integrate forward in time, set the part of the terminal value to the given values, integrate backward in time, and so on...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The standard reference for these problems is the book by Ascher, Mattheij, &amp;amp; Russell: &lt;em&gt;Numerical solution of boundary value problems for ordinary differential equations.&lt;/em&gt; However, implicit problems are not considered there.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-07-24T20:32:45.770" LastActivityDate="2013-07-24T20:32:45.770" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8051" PostTypeId="2" ParentId="8048" CreationDate="2013-07-24T16:53:27.583" Score="2" Body="&lt;p&gt;The MATLAB routines starting with 'ode', like ode15i, are for solving initial value problems.  If you want to solve a boundary value problem, use bvp4c or bvp5c.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-07-24T16:53:27.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8052" PostTypeId="1" AcceptedAnswerId="8054" CreationDate="2013-07-24T19:16:59.560" Score="31" ViewCount="4492" Body="&lt;p&gt;I have been looking into C++ linear algebra libraries for a project I've been working on. Something that I still don't have any grasp on is the connection of BLAS and LAPACK to other linear algebra libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking through &lt;a href=&quot;http://verdandi.sourceforge.net/doc/linear_algebra_libraries.pdf&quot;&gt;this article on linear algebra libraries&lt;/a&gt; I found it interesting that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;some libraries are independent from BLAS and LAPACK&lt;/li&gt;&#xA;&lt;li&gt;some require BLAS and LAPACK&lt;/li&gt;&#xA;&lt;li&gt;some have optional interfaces to BLAS and LAPACK&lt;/li&gt;&#xA;&lt;li&gt;and, as I understand it, you can use BLAS and LAPACK to solve linear algebra problems directly&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I can imagine that some libraries are simply C++ interfaces to BLAS and LAPACK libraries written in C and Fortran and others have implemented their own substitute routines, but &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the implications of the optional interfaces to BLAS and LAPACK? What are you loosing by opting out, and what are the libraries doing instead?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do any of the libraries provide more than just an interface? For example, UMFPACK is written in C and has optional interfaces to BLAS and LAPACK. What can UMFPACK (or other libraries) do that BLAS and LAPACK can't on their own?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3179" LastActivityDate="2013-07-25T00:58:55.843" Title="What is the relationship of BLAS, LAPACK, and other linear algebra libraries?" Tags="|linear-algebra|software|lapack|blas|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8053" PostTypeId="1" CreationDate="2013-07-24T19:42:04.747" Score="2" ViewCount="74" Body="&lt;p&gt;This is probably going to be an easy question for most people and embarass me. I've been reading the Zhang et al. 08 paper on detection-based tracking through network flows: &lt;a href=&quot;http://iris.usc.edu/outlines/papers/2008/zhang-li-nevatia-cvpr08.pdf&quot; rel=&quot;nofollow&quot;&gt;http://iris.usc.edu/outlines/papers/2008/zhang-li-nevatia-cvpr08.pdf&lt;/a&gt;. In this paper, equation (11), the authors seem to go from: $-\log(1-\beta_i)\cdot f_i - \log\beta_i \cdot (1-f_i)$ to $C_i \cdot f_i$ where $C_i = \log\frac{\beta_i}{1-\beta_i}$. I've been going over this for about an hour now, shifting logarithms and exponents around but the derivation doesn't seem possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing here?&lt;/p&gt;&#xA;" OwnerUserId="4839" LastEditorUserId="1552" LastEditDate="2013-07-25T17:56:46.697" LastActivityDate="2013-07-25T17:56:46.697" Title="Network Flow - based optimization in renowned paper appears to have a wrong step" Tags="|optimization|computer-vision|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8054" PostTypeId="2" ParentId="8052" CreationDate="2013-07-24T20:08:10.717" Score="17" Body="&lt;p&gt;As far as I know, Lapack is the only publicly available implementation of a number of algorithms (nonsymmetric dense eigensolver, pseudo-quadratic time symmetric eigensolver, fast Jacobi SVD). Most libraries that don't rely on BLAS+Lapack tend to support very primitive operations like matrix multiplication, LU factorization, and QR decomposition. Lapack contains some of the most sophisticated algorithms for dense matrix computations that I don't believe are implemented anywhere else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer your questions (at least partially),&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;By opting out of BLAS/Lapack, you are typically not missing functionality (unless the optional interface was designed so that there is no substitute implementation, which is rare). If you wanted to do very sophisticated operations, those other libraries probably don't implement it themselves anyways. Since BLAS can be highly tuned to your architecture, you could be missing out on huge speedups (an order of magnitude speed difference is not unheard of).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You mention UMFPACK, which is for sparse matrix factorization. BLAS/Lapack is only concerned about dense matrices. UMFPACK at some level needs to work on medium size dense problems, which it can do using custom implementations or by calling BLAS/Lapack. Here the difference is only in speed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If speed is of great concern, try to use a library that supports optional BLAS/Lapack bindings, and use them in the end when you want things faster.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-07-24T20:08:10.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8055" PostTypeId="2" ParentId="8052" CreationDate="2013-07-24T20:20:05.127" Score="1" Body="&lt;p&gt;Short version: they are libraries written primarily in Fortran that are used for Numerical operations in many languages - even some C programs due to their sheer speed and optimizations; They are also some of the only open source implementations of many algorithms :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't have to use libraries unless they have dependencies; iirc most of those are pretty independent, and you can always write your own math functions, such as better vectorized ones for your architecture &lt;/p&gt;&#xA;" OwnerUserId="1276" LastActivityDate="2013-07-24T20:20:05.127" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8056" PostTypeId="2" ParentId="8052" CreationDate="2013-07-25T00:58:55.843" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Good implementations of the BLAS and LAPACK routines (most importantly the BLAS routines) can be much faster than naive straight forward implementations of the same functions.   However, efficient implementations typically include optimizations that are very specific to the particular computer that you're running on. Even different models of processors from the same manufacturer (e.g. Intel x86-64 processors) often require very different code to get good performance.  By supplying optimized BLAS/LAPACK libraries to a software package, you can typically speed up the code compared to using unoptimized routines.  However, since many casual users may not have the expertise to install optimized routines, it's common to also provide an option to use generic unoptimized linear algebra routines.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;UMFPACK is a library of routines for linear algebra on sparse matrices (matrices with a high proportion of 0 entries.)  It can use BLAS/LAPACK to handle dense matrices (or dense blocks within matrices) that it encounters.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-07-25T00:58:55.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8057" PostTypeId="1" AcceptedAnswerId="8059" CreationDate="2013-07-25T01:42:15.983" Score="3" ViewCount="6598" Body="&lt;p&gt;I have a n x 3 double containing x, y and z coordinates. Can someone tell me where did I go wrong?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;store_y_temp=real(Y(Y&amp;gt;Iy));&#xA;store_z_temp=Z(store_y_temp);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Error:&#xA;Subscript indices must either be real positive integers or logicals.&lt;/p&gt;&#xA;" OwnerUserId="4840" LastActivityDate="2015-09-14T09:27:01.437" Title="MATLAB Subscript indices must either be real positive integers or logicals" Tags="|matlab|matrix|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8058" PostTypeId="2" ParentId="8057" CreationDate="2013-07-25T02:05:06.727" Score="0" Body="&lt;p&gt;The reason you get the error is because &lt;code&gt;store_y_temp&lt;/code&gt; is not an integer and you are using it to refer to array indices. Check a similar &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7800/matlab-output-in-scientific-notation-error-subscript-indices-must-either-be-re/7802#7802&quot;&gt;question&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="4364" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-07-25T02:05:06.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8059" PostTypeId="2" ParentId="8057" CreationDate="2013-07-25T02:55:02.320" Score="2" Body="&lt;p&gt;I am suggesting you are doing something like &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Find the points such that the $y$-coordinate of this $n\times 3$ array is greater than some $Iy$, then find these points' $z$-coordinates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You have several ways to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most simple command would be (if we assume the same row corresponds to the same point):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z_temp = z(y&amp;gt;Iy);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z_temp = z(find(y&amp;gt;Iy));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;y&amp;gt;Iy&lt;/code&gt; is a logical vector returning &lt;code&gt;1&lt;/code&gt; in the entry if the criterion is met, &lt;code&gt;0&lt;/code&gt; otherwise. &lt;code&gt;find&lt;/code&gt; finds the non-zero entry's index. The first one is faster though (if you use the second one while writing scripts, &lt;code&gt;mlint&lt;/code&gt; will remind you the first method is better...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  x = [1; 2; 3; 4];&#xA;  y = [1/2; -1/2; -3/2; 3/2];&#xA;  z = [5; 6; 7; 8];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Typing the first command results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; z_temp = z(y&amp;gt;0)&#xA;   z_temp = &#xA;           5&#xA;           8   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-07-25T02:55:02.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8060" PostTypeId="1" CreationDate="2013-07-25T10:08:44.040" Score="8" ViewCount="1182" Body="&lt;p&gt;I have a generalised matrix problem: $A x = \lambda B x$ from a spectral method on a linear stability analysis problem. My matrix B is diagonal and positive semi-definite. A is non-hermitian and complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is essentially that when using the SLEPc generalised eigenvalue solver I get the error &quot;zero pivot in LU factorisation&quot;. The rest of the below is details about the problem and things I have tried so far. Thanks for the help!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Details of the problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The matrix will be at its largest about 48000 by 48000, and I want to find the eigenvalues. The eigenvalues I am interested in are ones with the largest real part near 0+0i. Ideally, I want to be able to find them even if they are internal (i.e when there are other eigenvalues with larger positive real part in the spectrum). However, I would be happy if I could get it to work for problems where all eigenvalues have real parts &amp;lt; 0 apart from the eigenvalue of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I have used the scipy linalg.eig and sparse.eigs functions. As far as I know, these use LAPACK and ARPACK respectively to do the heavy lifting. I have decided to see if I can achieve better performance through using the SLEPc library. If this is a bad decision, let me know!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to move onto using PETSc with the SLEPc eigenvalue solvers. I have been trying out SLEPc using the examples provided as part of the tutorial. Exercise 7 (&lt;a href=&quot;http://www.grycap.upv.es/slepc/handson/handson3.html&quot;&gt;http://www.grycap.upv.es/slepc/handson/handson3.html&lt;/a&gt;) reads matricies A and B from a file and outputs the solutions. I got this to work fine using the matrices provided. However, if I substitute a smaller sized test version of my problem (6000x6000), I get a variety of errors depending on the command line arguments I supply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problem I have is the error: &quot;zero pivot in LU factorisation!&quot; when I use the default settings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this might be related to the fact that B contains rows of zeros, although my understanding of linear algebra is somewhat basic. Is this true?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried setting the options suggested on the petsc website, -pc_factor_shift_type NONZERO etc but all I get is an additional warning that these options were not used&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assumed that this was a problem with the preconditioner, so I tried setting -eps_target to 0.1 and both with and without specifying -st_type sinvert and shift. Still I get the same error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I tried -st_pc_type jacobi and st_pc_type bjacobi. jacobi runs, but does not produce any eigenvalues. Block jacobi does an LU factorisation and gives me the same error again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The default method is krylov-schur, so I have experimented with the -eps_type gd and -eps_type jd options. Unfortunately these seem to produce nonsense eigenvalues, which do not appear on the spectrum at all when I solve using LAPACK in scipy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know my matrix problem is not singular, because I can solve it using scipy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know of any books/guides I might need to read besides the PETSC and SLEPC manuals to understand the behaviour of all these different solvers? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output from the case with no command line options is given below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot for taking the time to read my first post!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kind Regards,&#xA;Toby&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Terminal output from SLEPc&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;tobymac:SLEPC toby$ mpiexec ./ex7 -f1 LHS-N7-M40-Re0.0-b0.1-Wi5.0-amp0.02.petsc  -f2 RHS-N7-M40-Re0.0-b0.1-Wi5.0-amp0.02.petsc -eps_view&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generalized eigenproblem stored in file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[0]PETSC ERROR: --------------------- Error Message ------------------------------------&#xA;[0]PETSC ERROR: Detected zero pivot in LU factorization:&#xA;see &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot&quot;&gt;http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot&lt;/a&gt;!&#xA;[0]PETSC ERROR: Empty row in matrix: row in original ordering 2395 in permuted ordering 3600!&#xA;[0]PETSC ERROR: ------------------------------------------------------------------------&#xA;[0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 5, Sat Dec  1 15:10:41 CST 2012 &#xA;[0]PETSC ERROR: See docs/changes/index.html for recent updates.&#xA;[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.&#xA;[0]PETSC ERROR: See docs/index.html for manual pages.&#xA;[0]PETSC ERROR: ------------------------------------------------------------------------&#xA;[0]PETSC ERROR: ./ex7 on a arch-darw named tobymac by toby Thu Jul 25 10:20:40 2013&#xA;[0]PETSC ERROR: Libraries linked from /opt/local/lib&#xA;[0]PETSC ERROR: Configure run at Tue Jul 23 15:11:27 2013&#xA;[0]PETSC ERROR: Configure options --prefix=/opt/local --with-valgrind-dir=/opt/local --with-shared-libraries --with-scalar-type=complex --with-clanguage=C++ --with-superlu-dir=/opt/local --with-blacs-dir=/opt/local --with-scalapack-dir=/opt/local --with-mumps-dir=/opt/local --with-metis-dir=/opt/local --with-parmetis-dir=/opt/local --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 --LDFLAGS=-L/opt/local/lib --CFLAGS=&quot;-O2 -mtune=native&quot; --CXXFLAGS=&quot;-O2 -mtune=native&quot;&#xA;[0]PETSC ERROR: ------------------------------------------------------------------------&#xA;[0]PETSC ERROR: MatLUFactorSymbolic_SeqAIJ() line 334 in /opt/local/var/macports/build/_Users_toby_MyPorts_scienceports_math_petsc/petsc/work/petsc-3.3-p5/src/mat/impls/aij/seq/aijfact.c&#xA;[0]PETSC ERROR: MatLUFactorSymbolic() line 2750 in /opt/local/var/macports/build/_Users_toby_MyPorts_scienceports_math_petsc/petsc/work/petsc-3.3-p5/src/mat/interface/matrix.c&#xA;[0]PETSC ERROR: PCSetUp_LU() line 135 in /opt/local/var/macports/build/_Users_toby_MyPorts_scienceports_math_petsc/petsc/work/petsc-3.3-p5/src/ksp/pc/impls/factor/lu/lu.c&#xA; Number of iterations of the method: 0&#xA; Number of linear iterations of the method: 0&#xA; Number of requested eigenvalues: 1&#xA; Stopping condition: tol=1e-08, maxit=750&#xA; Number of converged approximate eigenpairs: 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[0]PETSC ERROR: PCSetUp() line 832 in /opt/local/var/macports/build/_Users_toby_MyPorts_scienceports_math_petsc/petsc/work/petsc-3.3-p5/src/ksp/pc/interface/precon.c&#xA;[0]PETSC ERROR: KSPSetUp() line 278 in /opt/local/var/macports/build/_Users_toby_MyPorts_scienceports_math_petsc/petsc/work/petsc-3.3-p5/src/ksp/ksp/interface/itfunc.c&#xA;[0]PETSC ERROR: PCSetUp_Redundant() line 176 in /opt/local/var/macports/build/_Users_toby_MyPorts_scienceports_math_petsc/petsc/work/petsc-3.3-p5/src/ksp/pc/impls/redundant/redundant.c&#xA;[0]PETSC ERROR: PCSetUp() line 832 in /opt/local/var/macports/build/_Users_toby_MyPorts_scienceports_math_petsc/petsc/work/petsc-3.3-p5/src/ksp/pc/interface/precon.c&#xA;[0]PETSC ERROR: KSPSetUp() line 278 in /opt/local/var/macports/build/_Users_toby_MyPorts_scienceports_math_petsc/petsc/work/petsc-3.3-p5/src/ksp/ksp/interface/itfunc.c&#xA;[0]PETSC ERROR: STSetUp_Shift() line 94 in src/st/impls/shift/shift.c&#xA;[0]PETSC ERROR: STSetUp() line 280 in src/st/interface/stsolve.c&#xA;[0]PETSC ERROR: EPSSetUp() line 204 in src/eps/interface/setup.c&#xA;[0]PETSC ERROR: EPSSolve() line 109 in src/eps/interface/solve.c&#xA;tobymac:SLEPC toby$ &lt;/p&gt;&#xA;" OwnerUserId="4843" LastActivityDate="2014-04-22T12:47:46.547" Title="Solving a large non-hermitian generalised eigenvalue problem from a linear stability analysis using SLEPc" Tags="|linear-algebra|parallel-computing|petsc|eigensystem|preconditioning|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8061" PostTypeId="2" ParentId="8060" CreationDate="2013-07-25T12:08:38.717" Score="1" Body="&lt;p&gt;In problems of this form, you typically need that $B$ is non-singular, i.e., that it is invertible. You don't say where $A,B$ come from, so I don't know anything about your $B$ but you need to make sure that it is not singular.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-25T12:08:38.717" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8062" PostTypeId="1" CreationDate="2013-07-25T12:58:44.907" Score="5" ViewCount="227" Body="&lt;p&gt;I am solving a laplace equation on a finite-element mesh (tetrahedral, triagonal) and have many say 99% dirichlet conditions compared to the number of unknowns. Is there an efficient way to solve this kind of system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give an example for better understanding:&#xA;Total number of nodes: 100,000&#xA;Nodes Dirichlet: 99,000&#xA;Number of Unknowns not Dirichlet: 1,000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be direct or iterative that doesnt matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be fast and efficient though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1:&lt;/strong&gt; The domain can be regular (box shaped) or irregular (eg. irregular shaped surface in 3D) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2:&lt;/strong&gt; The dirichlet conditions are can be on the boundary and/or inside the domain as a single dirichlet condition is enough to make the solution unique.&lt;/p&gt;&#xA;" OwnerUserId="4844" LastEditorUserId="4844" LastEditDate="2013-07-25T16:21:11.790" LastActivityDate="2014-02-19T00:03:55.437" Title="What Linear Equation Solver should be used for a problem with many dirichlet conditions?" Tags="|linear-algebra|finite-element|matrix|linear-solver|boundary-conditions|" AnswerCount="1" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="8063" PostTypeId="1" CreationDate="2013-07-25T13:43:24.947" Score="2" ViewCount="148" Body="&lt;p&gt;The maximum k-splittable s-t flow problem(MkSF) that aims to find a maximum k-splittable flow between a given source and sink node is NP-hard. We do not require the paths to be disjoint, not even different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a special case, just to be sure, is still computation of the k-splittable s-t  flow in a directed network NP-hard if we consider only k disjoint paths?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, is it true if I say that for a directed network with finite number of nodes and arcs, the number of disjoint paths between two given nodes is still finite?&lt;/p&gt;&#xA;" OwnerUserId="917" LastActivityDate="2013-12-23T09:30:49.847" Title="k-splittable flow problem" Tags="|optimization|linear-programming|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8067" PostTypeId="2" ParentId="6873" CreationDate="2013-07-25T23:53:16.573" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Travelling_salesman_problem&quot; rel=&quot;nofollow&quot;&gt;Traveling Salesman problem&lt;/a&gt; is a great optimization problem. It's also a NP-hard problem which need global optimization. In order to solve the problem you have to try several heuristic optimization algorithm like Simulated Annealing and Evolutionary Algorithm which makes you familiar with the idea of global optimization.&lt;/p&gt;&#xA;" OwnerUserId="1622" LastActivityDate="2013-07-25T23:53:16.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8068" PostTypeId="1" AcceptedAnswerId="8070" CreationDate="2013-07-26T01:40:18.673" Score="1" ViewCount="523" Body="&lt;p&gt;I have X as a 616 x 1 double. I calculated the differences between adjacent elements of X by using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y=diff(X);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and get Y as a 615 x 1 double, containing positive and negative numbers, including 0 (zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y=diff&amp;gt;0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'll get the answer as 615 x 1 logical, which is not the answer I'm expecting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how can I extract only the element of X which the difference is 0?&lt;/li&gt;&#xA;&lt;li&gt;how can I detect where the decrease and increase trends starts?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4840" LastActivityDate="2013-07-26T03:38:05.083" Title="matlab find element with diff equals to zero" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8070" PostTypeId="2" ParentId="8068" CreationDate="2013-07-26T03:05:37.123" Score="2" Body="&lt;p&gt;If you are trying to find the entries with numerical difference being zero, you might end up with nothing because the differences may be non-zero for all entries because they are &lt;code&gt;double&lt;/code&gt;, and assuming these 616 entries are approximation to something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you might wanna try to set a tolerance &lt;code&gt;Tol&lt;/code&gt;: for example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; X = sin(linspace(0,2*pi,616)'); &#xA; % this gives you an X being the grid approximation to sine&#xA; Y = diff(X);&#xA; Tol = 1e-2*2*pi/615;&#xA; indY = find(abs(Y) &amp;lt; Tol);&#xA; % the index of the difference vector that has absolute value &amp;lt; Tolerance&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then &lt;code&gt;X(indY)&lt;/code&gt; and &lt;code&gt;X(indY+1)&lt;/code&gt; will give you the entries of &lt;code&gt;X&lt;/code&gt; that produces these less-than-tolerance differences. If the latter is bigger than the former, then &lt;code&gt;X&lt;/code&gt; starts to increase (approximation), vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  &amp;gt;&amp;gt; X(indY)&#xA;     ans =&#xA;         0.99997&#xA;               1&#xA;        -0.99992&#xA;              -1&#xA;  &amp;gt;&amp;gt; X(indY+1)&#xA;     ans =&#xA;               1&#xA;         0.99992&#xA;              -1&#xA;        -0.99997&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The criteria for choosing &lt;code&gt;Tol&lt;/code&gt; is more like an a priori guess. First you have the length of the interval I set being $2\pi$, there is $616$ grid points, which has a magnitude of &lt;code&gt;1e2&lt;/code&gt;. The true derivative of $\sin x$ is $\cos x$, which has absolute value less than $1$, so the all the numerical differences should be less than or equal to $h = 2\pi/615$. Now I already know sine, when near its extrema, increases or decreases like a linear function (imagine Taylor expansion, or small angle approximation), and it takes about 100 grids to make it the maximum (knowing a priori for sine). Therefore the absolute value of the difference being less than the magnitude of the &lt;code&gt;1e-2&lt;/code&gt; times the grid size would suffice to say that, &lt;code&gt;X&lt;/code&gt; almost remains the same. It is problem dependent.  &lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2013-07-26T03:38:05.083" LastActivityDate="2013-07-26T03:38:05.083" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8071" PostTypeId="2" ParentId="8063" CreationDate="2013-07-26T08:11:13.717" Score="1" Body="&lt;p&gt;The answer of the the first part is yes; restricting the paths to be disjoint makes still the problem NP-hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the Menger theorem, the number of disjoint paths between two given nodes is the minimum number of arcs whose removal makes the given nodes disconnected. This means that the number of disjoint paths between two nodes in a graph with polynomial number of nodes and arcs is finite.&lt;/p&gt;&#xA;" OwnerUserId="917" LastActivityDate="2013-07-26T08:11:13.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8072" PostTypeId="1" CreationDate="2013-07-26T08:38:31.123" Score="7" ViewCount="433" Body="&lt;p&gt;I am looking into full multigrid, FMG, and several sources, including &lt;a href=&quot;http://computation.llnl.gov/casc/people/henson/mgtut/ps/mgtut.pdf&quot;&gt;these slides&lt;/a&gt;, that a lot of people are referring to, state that the prolongation operator used in FMG the first time you visit a finer grid should be of higher order than the prolongation operator used otherwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a concrete example of such a higher order operator? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the prolongation in the other steps of multigrid, I use the operator suggested in the slides page 56. &lt;/p&gt;&#xA;" OwnerUserId="4854" LastActivityDate="2015-03-01T12:58:04.107" Title="Full Multigrid Prolongation Operator" Tags="|interpolation|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8073" PostTypeId="1" AcceptedAnswerId="8074" CreationDate="2013-07-26T11:17:32.760" Score="6" ViewCount="1404" Body="&lt;p&gt;I try to compute eigenvalues of the tridiagonal matrix coming from finite difference scheme. For small mesh size, eigs works well. But for large size it fails. Here is an example where eigs fails. Is there any other way to reliably compute a few eigenvectors in matlab ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 1000; h = 1/n;&#xA;e = ones(n,1);&#xA;A = (1/h)*spdiags([e, -2*e, e], -1:1, n, n);&#xA;e = eigs(A,10,'LA')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2014-02-05T18:56:07.810" Title="matlab eigs: wrong eigenvalues for tridiagonal matrix" Tags="|matlab|eigenvalues|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8074" PostTypeId="2" ParentId="8073" CreationDate="2013-07-26T13:37:29.443" Score="6" Body="&lt;p&gt;I was able to solve your example by using one of the eigs options to increase the number of Lanczos vectors used in the eigensolution from the default (20 in your case) to 50:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;opts.p = 50;&#xA;e = eigs(A,10,'LA',opts)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't believe the algorithm is extremely sensitive to this number but, obviously, some experimentation with your real problem will be necessary to see if you can select a value that will consistently work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any other MATLAB functions for calculating a few eigenvalues from large, sparse matrices.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-07-26T13:37:29.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8075" PostTypeId="1" CreationDate="2013-07-26T14:59:05.063" Score="3" ViewCount="156" Body="&lt;p&gt;I am looking for a tool to specify pipelines of arbitrary multiple jobs with dependencies for the Sun Grid Engine. I can wire things together myself but I am hoping for something more declarative and reusable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone recommend such a tool or has anyone adapted an existing build system/dependency manager for compute jobs on SGE?&lt;/p&gt;&#xA;" OwnerUserId="4856" LastActivityDate="2014-02-26T17:35:09.023" Title="Can anyone recommend a pipeline dependency (&quot;make&quot;) tool for SGE jobs?" Tags="|hpc|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8077" PostTypeId="1" CreationDate="2013-07-26T17:11:31.860" Score="5" ViewCount="84" Body="&lt;p&gt;I'm trying to solve numerically the 1-dim time dependent Schrodinger equation using the Crank Nicolson scheme and the Thomas algorithm to solve the tridiagonal matrix.&#xA;The physical system consists of a free particle moving in a region of costant (zero) potential, far away from the boundary of my domain.&#xA;At this time I am interested in checking the accuracy of the method experimentally, so I am studying the L2 error for different mesh spacing.&#xA;The initial wave function and the analytical solution of the Scrhodinger eq. are the same that one can find in Cohen- Tannoudji (complement G1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Matlab code is here: &lt;a href=&quot;http://tny.cz/1bd1bf94&quot;&gt;http://tny.cz/1bd1bf94&lt;/a&gt;.&#xA;and the code for the Thomas algorithm (already tested) here: &lt;a href=&quot;http://tny.cz/83c04868&quot;&gt;http://tny.cz/83c04868&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I don't get the result I should (in particular, if I study the order of convergence experimetally I get inconsistent results and not the second order accuracy in space and time that I expect).&#xA;Does anyone has any ideas?&#xA;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4857" LastActivityDate="2013-07-26T17:11:31.860" Title="Order of convergence of Scrodinger eq. with CN scheme" Tags="|accuracy|crank-nicolson|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8078" PostTypeId="1" CreationDate="2013-07-26T19:05:20.150" Score="4" ViewCount="1215" Body="&lt;p&gt;I have a matrix &lt;code&gt;A&lt;/code&gt; which is an approximation to the known matrix &lt;code&gt;B&lt;/code&gt;. Both matrices are square, 3x3 matrices and, in this case, are symmetric. Is there a 'best' method for calculating percent error in the approximation? I had been using the sum the 'sum of squares' error for all entries divided by the sum of 'sum of squares' of the true matrix,&#xA;&lt;code&gt;pct_err = sum(sum( (A-B).^2 ))/(sum(sum(B.^2))&lt;/code&gt; &#xA;and realized that this is simply the square of &lt;code&gt;norm(A-B,2)/norm(B,2)&lt;/code&gt; which gives drastically different percent error estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I go about calculating percent error between two matrices, or maybe more generally, two tensors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Thanks to some of the comments, I now recognize the difference between the 2-norm and the Frobenius norm. In either case, however, the type of norm I am taking is of relatively small consequence. The bigger question is how to represent the percent error of a 2nd rank tensor correctly.&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="164" LastEditDate="2013-07-27T14:48:58.667" LastActivityDate="2013-07-27T14:48:58.667" Title="How can you calculate percent error in tensor approximations?" Tags="|error-estimation|matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8079" PostTypeId="2" ParentId="8078" CreationDate="2013-07-26T20:12:27.247" Score="1" Body="&lt;p&gt;&lt;code&gt;norm(A,2)&lt;/code&gt; returns the largest singular value of &lt;code&gt;A&lt;/code&gt; not the square root of the sum of squares of its elements.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-07-26T20:12:27.247" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8080" PostTypeId="2" ParentId="7702" CreationDate="2013-07-26T21:56:32.220" Score="2" Body="&lt;p&gt;There are two factors that are influenced by time step size and the choice of scheme: accuracy and stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accuracy is typically measured by the &quot;local error&quot; or &quot;consistency error&quot; of the scheme. You want to choose your time step such that this error is balanced with a comparable error of the space discretization. That would be a good balance for accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, most timestepping schemes also change the dynamics of your system, which is usually subsumed under the term stability. This question goes beyond explicit or implicit. And this goes both ways: a perfectly stable solution can be converted to an explosion if you use the wrong method with a large timestep. And the opposite holds: if you use a method that is too stable, turbulent, instationary flow might be turned into honey. I know of simulations where a single backward Euler step every 100 Crank-Nicolson steps made an oscillatory solution stationary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms used to categorize the dynamics of timestepping schemes are A-, L-, and B-stability. As far as I know, only the Crank-Nicolson scheme and Gauss-collocation methods preserve the essential dynamics, but even for those, certain features of your solution may be amplified or suppressed in an unphysical way, if your timestep is too large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to be able to predict these effects, you have to know your scheme. Else, you are stuck with test examples, or with computing everything with at least two time step sizes&lt;/p&gt;&#xA;" OwnerUserId="4108" LastEditorUserId="4108" LastEditDate="2013-07-27T11:42:14.973" LastActivityDate="2013-07-27T11:42:14.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8081" PostTypeId="1" CreationDate="2013-07-26T22:26:49.380" Score="3" ViewCount="137" Body="&lt;p&gt;I have written an algorithm to solve a dense system with smoothly varying entries. This means I assume there is no large jump from any entry to its neighbors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would love to use real-application-generated matrix to test my algorithm, but I cannot think of any that satisfies my assumption out of my stupid head. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="4761" LastEditorUserId="4244" LastEditDate="2013-07-27T14:52:38.863" LastActivityDate="2013-07-27T17:15:53.157" Title="Smoothly varying dense matrices arising from computational science" Tags="|pde|linear-solver|matrix|dense-matrix|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8082" PostTypeId="2" ParentId="8081" CreationDate="2013-07-27T00:11:01.857" Score="1" Body="&lt;p&gt;How about a problem of $L^2$-projection on to the polynomial space spanned by &#xA;$$&#xA;\{1,1+\epsilon x, 1+\epsilon x^2, \dots, 1+\epsilon x^n\},&#xA;$$&#xA;for $\epsilon \ll 1$, $x\in [0,1]$. Then the matrix has entry&#xA;$$&#xA;A_{ij} = \int^1_{0} (1+\epsilon x^i)(1+\epsilon x^j) dx = 1 + \frac{\epsilon}{i+1}+\frac{\epsilon}{j+1} + \frac{\epsilon^2}{i+j+1},&#xA;$$&#xA;with right hand side $B$ is&#xA;$$&#xA;b_j = \int^1_0 (1+\epsilon x^j) f(x) dx.&#xA;$$&#xA;Then the projection of $f$ has coefficients $F = (f_1,\dots,f_n)$ can be solved by&#xA;$$&#xA;A F = B.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-07-27T00:11:01.857" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8083" PostTypeId="2" ParentId="8078" CreationDate="2013-07-27T02:07:08.547" Score="1" Body="&lt;p&gt;It appears that what you want in MATLAB is &lt;code&gt;norm(A, 'fro')&lt;/code&gt;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Matrix_norm#Frobenius_norm&quot; rel=&quot;nofollow&quot;&gt;the Frobenius norm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-07-27T02:07:08.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8084" PostTypeId="2" ParentId="8081" CreationDate="2013-07-27T10:43:50.530" Score="2" Body="&lt;p&gt;A differential operator will always result in rows with sum zero. Furthermore, if you want to preserve the locality of the differential operator, the entries with opposite signs will be neighbors (possibly separated by zeros).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Smoothly varying entries come up if you discretize an integral operator with smooth kernel by a collocation or quadrature method. I recommend you look into applications there.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-07-27T10:43:50.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8085" PostTypeId="2" ParentId="8081" CreationDate="2013-07-27T17:15:53.157" Score="1" Body="&lt;p&gt;A method of moments solution to the Helmholtz equation will have smoothly varying coefficients as a function of frequency (we usually want to compute frequency spectra, so this is very common). Also, planewave expansions are dense, and should also vary continuously with respect to frequency. I wrote a package called &lt;a href=&quot;http://www.stanford.edu/group/fan/S4/&quot; rel=&quot;nofollow&quot;&gt;S4&lt;/a&gt;, which is almost always used to compute spectra, and it requires many linear solves and eigenvalue problems of smoothly varying dense matrices.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-07-27T17:15:53.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8086" PostTypeId="2" ParentId="1151" CreationDate="2013-07-28T15:27:19.457" Score="3" Body="&lt;p&gt;Here a most likely incomplete list. But maybe others can help extend it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bisection methods can be applied to simplicial meshes and have the advantage that they are always conforming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Refinement into similar objects can be applied to triangles, quadrilaterals and hexahedra. To tetrahedra with some modifications. It allows for particularly simple setup of finite element multigrid methods, since the local embedding operators are always the same. It is very simple if combined with the technique of hanging nodes. And with &lt;a href=&quot;http://www.p4est.org&quot; rel=&quot;nofollow&quot;&gt;p4est&lt;/a&gt;, there is a very efficient parallel representation for such meshes available.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-07-28T15:27:19.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8087" PostTypeId="1" CreationDate="2013-07-28T16:01:26.350" Score="5" ViewCount="629" Body="&lt;p&gt;I wish to solve an equation of the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial}{\partial t} \left( \frac{\partial \phi}{\partial x} \right) = -\frac{\partial}{\partial x}(\mathcal{F})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the variable &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; (e.g. mass).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the right-hand side is the flux &lt;span class=&quot;math-container&quot;&gt;$\mathcal{F}$&lt;/span&gt; of quantity &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation &lt;strong&gt;&quot;looks like&quot;&lt;/strong&gt; an advection-diffusion equation, but with the rate of change of the spatial derivative of &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; appearing on the left-hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying the finite volume approach we integrate the equation over the cell &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial}{\partial t}\int_{\Omega}\left( \frac{\partial \phi}{\partial x} \right)dx = - \int_{\Omega}\frac{\partial\mathcal{F}}{\partial x} dx&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial}{\partial t}\left( \frac{\phi_{j+1/2}}{h_j} - \frac{\phi_{j-1/2}}{h_j}\right) = -\left( \frac{\mathcal{F}_{j+1/2}}{h_j} - \frac{\mathcal{F}_{j-1/2}}{h_j}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$h_j$&lt;/span&gt; width of the cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this basic approach correct? I have never needed to solve an equation which is the time-derivative of a spatial derivative before, this is the approach I have taken, does anyone have any advice or direction? I have not yet tried to implement this numerically.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="20688" LastEditDate="2019-06-22T20:12:21.823" LastActivityDate="2019-06-22T20:12:21.823" Title="Solving PDE with spatial and temporal derivatives on left hand side" Tags="|finite-volume|numerics|advection-diffusion|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8088" PostTypeId="2" ParentId="8087" CreationDate="2013-07-28T16:58:29.430" Score="4" Body="&lt;p&gt;What I describe below is a version of the method of characteristics. Its applications are limited, but if it works in your case, it would be a fairly simple, fast, and accurate process. If you want to discretize on a mesh, it might not be that helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming your solutions $\phi$ are smooth, you can reduce this to a family of ODE problems. Exchange the order of derivatives on the left first, obtaining&#xA;\begin{gather}&#xA;\frac{\partial}{\partial x}\frac{\partial}{\partial t} \phi = -\frac{\partial}{\partial x}\frac{\partial}{\partial x} \phi.&#xA;\end{gather}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you define $\psi = \partial_x \phi$ and solve&#xA;\begin{gather}&#xA;\frac{\partial}{\partial t} \psi = -\frac{\partial}{\partial x} \psi,&#xA;\qquad\text{or}\qquad (\partial_t+\partial_x)\psi=0,&#xA;\end{gather}&#xA;which is a first order hyperbolic conservation law and linear. Thus,&#xA;\begin{gather}&#xA;\psi(t,x) = \psi(0,x-t),&#xA;\end{gather}&#xA;that is, $\psi$ is already determined by its initial values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, observing that by construction $\psi = \partial_x \phi$, you compute (boundary conditions permitting):&#xA;\begin{gather}&#xA;\phi = \int \psi(t,x) \,dx.&#xA;\end{gather}&lt;/p&gt;&#xA;" OwnerUserId="4108" LastEditorUserId="4108" LastEditDate="2013-07-28T20:40:19.677" LastActivityDate="2013-07-28T20:40:19.677" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8089" PostTypeId="1" AcceptedAnswerId="8096" CreationDate="2013-07-28T19:43:11.490" Score="11" ViewCount="3120" Body="&lt;p&gt;I have a question that is similar to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2771/is-there-a-special-algorithm-for-computing-the-convex-hull-ordering-when-the-can&quot;&gt;this one asked before&lt;/a&gt; except in 3D, and I only need the volume, not the actual shape of the hull.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, I'm given a small set of points (say, 10-15) in 3D, all of which are known to lie on the convex hull of the point set (so they all &quot;matter&quot; and define the hull). I only want to compute the volume of the hull, I don't care about computing the actual polyhedron. Is there an efficient algorithm to do this?&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2019-08-07T14:30:56.540" Title="Volume of 3D convex hull of small point sets all on the hull" Tags="|algorithms|reference-request|computational-geometry|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8090" PostTypeId="1" CreationDate="2013-07-29T02:18:07.230" Score="12" ViewCount="371" Body="&lt;p&gt;Consider a symmetric positive definite tridiagonal linear system&#xA;$$A x = b$$&#xA;where $A \in \mathbb{R}^{n \times n}$ and $b \in \mathbb{R}^n$.  Given three indices $0 \le i &amp;lt; j &amp;lt; k &amp;lt; n$, if we assume only equation rows strictly between $i$ and $k$ hold, we can eliminate intermediate variables to get an equation of the form&#xA;$$u x_i + v x_j + w x_k = c$$&#xA;where $v &amp;gt; 0$.  This equation relates the value of $x_j$ to $x_i,x_k$ independent of &quot;outside&quot; influence (say, if a constraint affecting $x_0$ was introduced).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Is it possible to preprocess the linear system $Ax = b$ in $O(n)$ time so that the linking equation for any $(i,j,k)$ can be determined in $O(1)$ time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the diagonal of $A$ is 2, the offdiagonals are $-1$, and $b = 0$, the desired result is the analytic result for the discretized Poisson equation.  Unfortunately, it is not possible to transform a general SPD tridiagonal system into a constant coefficient Poisson equation without breaking the tridiagonal structure, essentially because different variables can have different levels of &quot;screening&quot; (locally strict positive definiteness).  A simple diagonal scaling of $x$, for example, can eliminate half of the $2n-1$ DOFs of $A$ but not the other half.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, a solution to this problem would require arranging the problem so that the amount of screening could be accumulated into a linear size array and then somehow &quot;cancelled&quot; to arrive at the linking equation for the given triple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update (more intuition)&lt;/strong&gt;: In terms of PDEs, I have a discretized linear elliptic problem in 1D, and I want to know whether I can spend $O(n)$ in precomputation to produce some sort of &quot;analytic&quot; solution that can be looked up in $O(1)$ time, where I am allowed to vary where the boundary conditions are.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2013-08-11T00:59:28.053" LastActivityDate="2013-08-15T18:18:30.360" Title="Given an SPD tridiagonal linear system, can we precompute so that any three indices can be linked in O(1) time?" Tags="|linear-algebra|poisson|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8091" PostTypeId="2" ParentId="8087" CreationDate="2013-07-29T04:54:55.863" Score="3" Body="&lt;p&gt;The equation is&#xA;$\partial_{t} \psi = \partial_{x} F(\phi)$&#xA;where $\psi = \partial_x \phi$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time-integration can be done, e.g., by explicit time-stepping:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\psi^{j+1}_i = \psi^j_i + \frac{\tau}{2 dx} (F^j_{i+1}-F^j_{i-1})$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which requires that $F$ is known at each grid node $i$ at the &quot;old&quot; time slice $j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be achieved by adding a numerical &quot;inversion&quot; operator &#xA;$\phi(x_i)=\int_{x_0}^{x_i} \psi(x)$ used each time before evaluating the function $F$.&#xA;If $\psi$ is known at each grid point $i$ at the old time-slice $j$ then the integral can be evaluated (e.g., by Gaussian quadrature) so $\phi$ can be found at each grid point $i$; therefore $F$ can be found at each grid point as well. Then the time-step can be carried out and $\psi^{j+1}$ is found. This method would work for implicit time-stepping as well.&lt;/p&gt;&#xA;" OwnerUserId="4325" LastEditorUserId="4325" LastEditDate="2013-07-29T13:42:32.747" LastActivityDate="2013-07-29T13:42:32.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8092" PostTypeId="2" ParentId="8090" CreationDate="2013-07-29T18:04:45.037" Score="2" Body="&lt;p&gt;Here's a somewhat unstable solution that only works when the coupling between variables is always nondegenerate.  Assume for simplicity that $b = 0$.  First, precompute the $n$ linking equations for $(0,i,n-1)$ for $0 \le i &amp;lt; n$, say&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_i = a_i x_0 + b_i x_{n-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, given $i &amp;lt; j$, we can combine the $i$th and $j$th linking equations and eliminate $x_{n-1}$ to get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{aligned}&#xA;b_j x_i &amp;amp;= a_i b_j x_0 + b_i b_j x_{n-1} \\&#xA;b_i x_j &amp;amp;= a_j b_i x_0 + b_i b_j x_{n-1} \\&#xA;b_j x_i - b_i x_j &amp;amp;= (a_i b_j - a_j b_i) x_0 \\&#xA;x_i &amp;amp;= \frac{a_i b_j - a_j b_i}{b_j} x_0 + \frac{b_i}{b_j} x_j&#xA;\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This process can be repeated once more to eliminate $x_0$ given $(i,j,k)$.  Unfortunately, we lose stability near $b_j = 0$, or in general if the tridiagonal system decouples into independent blocks.  If $b_j = 0$ this is no problem, but I'm worried about breakdown for tiny but positive values.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-07-29T18:04:45.037" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8093" PostTypeId="1" AcceptedAnswerId="8102" CreationDate="2013-07-29T21:08:06.667" Score="5" ViewCount="5645" Body="&lt;p&gt;I have a transformation matrix which takes in the elastic constants from the local $rtl$ coordinates and then converts the elastic constants to the global $xyz$ coordinates via a rotation about the $z$ axis (i.e., $l$ doesn't change with respect to $z$, at lest at this stage). From what I gather the standard way to do this is to create a $6 \times 6$ matrix of directional cosines, along with a $6 \times 6$ matrix of elastic constants (the stiffness matrix) and then use $G^T C G$. where $G$ is the rotation matrix and $C$ is the stiffness matrix. I did this, and then to validate it I used the analytical solutions in Lekhnitskii's text '&lt;em&gt;Theory of elasticity of an anisotropic elastic body&lt;/em&gt;' for a cylinder rotated around the $z$ axis. To check I had coded Lekhnitskii's matrix correctly I used the 5 rotational invariants presented partially in his book and elsewhere on the web. Now his matrix is working (at lest as far as the 5 invariants hold), however the $G^T C G$ system does not match with his solution, (and most of the invariants don't hold for my $G^T C G$ system either).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions: Is there a more complete way to test that Lekhnitskii's matrix is working correctly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And is there a systematic way to debug my $G^T C G$ system to find out why it is not working?&lt;/p&gt;&#xA;" OwnerUserId="4296" LastEditorUserId="9667" LastEditDate="2016-01-12T16:20:02.840" LastActivityDate="2020-07-21T12:49:02.577" Title="debugging a rotation matrix for elastic constants" Tags="|matrix|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8094" PostTypeId="1" CreationDate="2013-07-29T21:22:59.507" Score="8" ViewCount="1102" Body="&lt;p&gt;&lt;em&gt;What is the policy of multiple overlapping asynchronous transfers in MPI?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a program with several open asynchronous &lt;code&gt;irecv&lt;/code&gt; operations.  I find that transfers that could take place (the corresponding &lt;code&gt;isend&lt;/code&gt; has been called) wait on other transfers that aren't yet ready (the corresponding &lt;code&gt;isend&lt;/code&gt; hasn't yet been called).  To be clear this inefficiency does not stem from network contention; my network is unnecessarily idle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My program looks like the following:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Machine 1&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;call irecv(variable A from machine 2)&#xA;call irecv(variable B from machine 2)&#xA;call irecv(variable C from machine 2)&#xA;call wait(variable C from machine 2)&#xA;call do_important_work_with(variable C)&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Machine 2&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;call isend(variable C to machine 1)&#xA;call isend(variable B to machine 1)&#xA;call do a bunch of costly work&#xA;call isend(variable A to machine 1)&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Problem&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The transfer of &lt;code&gt;C&lt;/code&gt; seems to be unnecessarily blocked by the transfer of &lt;code&gt;A&lt;/code&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find that the &lt;code&gt;wait&lt;/code&gt; on &lt;code&gt;variable C&lt;/code&gt; on Machine 1 does not complete until after the costly work on Machine 2 completes.  This is unfortunate because this transfer could have started at the beginning of my program.  It seems to wait unnecessarily for the transfer of &lt;code&gt;A&lt;/code&gt; to complete.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Questions&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In particular I have a computation like the following.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this expected?&lt;/li&gt;&#xA;&lt;li&gt;What is the policy of multiple overlapping asynchronous transfers?&lt;/li&gt;&#xA;&lt;li&gt;Can this be avoided without rearranging my code (is there some relevant internal setting)?&lt;/li&gt;&#xA;&lt;li&gt;Where should I go to learn more about MPI's policy for multiple live transfers?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19" LastEditorUserId="119" LastEditDate="2013-08-04T15:17:27.663" LastActivityDate="2013-08-04T15:17:27.663" Title="MPI policy for multiple asynchronous transfers" Tags="|parallel-computing|software|mpi|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8095" PostTypeId="2" ParentId="8089" CreationDate="2013-07-29T22:38:57.457" Score="2" Body="&lt;p&gt;From Komei Fukuda's &lt;a href=&quot;ftp://ftp.ifor.math.ethz.ch/pub/fukuda/reports/polyfaq040618.pdf&quot; rel=&quot;nofollow&quot;&gt;Polyhedral Computation FAQ&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;2.23 Is there any efficient algorithm to compute the volume of a&#xA;  convex polytope in $R^d$?&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;It is known that computing the volume of a V-polytope (or H-polytope)&#xA;  is #P-hard, see [DF88] and [Kha93]. There are theoretically efficient&#xA;  randomized algorithms to approximate the volume of a convex body&#xA;  [LS93] but no implementation seems to be available. There is a&#xA;  comparative study [BEF00] of various volume computation algorithms for&#xA;  convex polytopes. It indicates that there is no single algorithm that&#xA;  works well for many different types of polytopes.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0217060?journalCode=smjcat&quot; rel=&quot;nofollow&quot;&gt;[DF88]&lt;/a&gt;&#xA;   M.E. Dyer and A.M. Frieze. The complexity of computing the&#xA;  volume of a polyhedron. &lt;em&gt;SIAM J. Comput.&lt;/em&gt;, 17:967-974, 1988.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[Kha93] L.G. Khachiyan. Complexity of polytope volume computation. In&#xA;  J. Pach, editor, &lt;em&gt;New Trends in Discrete and Computational Geometry&lt;/em&gt;,&#xA;  pages 91-101. Springer Verlag, Berlin, 1993.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[LS93] L. Lovasz and M. Simonovits. Random walks in a convex body and&#xA;  an improved volume algorithm.&#xA;  &lt;em&gt;Random structures &amp;amp; algorithms&lt;/em&gt;, 4:359-412, 1993.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[BEF00] B. Bueler,  A. Enge, and K. Fukuda. Exact volume computation&#xA;  for convex polytopes: A practical study. In G. Kalai and G. M.&#xA;  Ziegler, editors, &lt;em&gt;Polytopes - Combinatorics and Computation&lt;/em&gt;,&#xA;  DMV-Seminar 29, pages 131-154. Birkhauser,&#xA;  2000.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This may seem to bury the specifics of the 3D problem among difficulties&#xA;of higher dimensions, despite the title of the Dyer and Frieze paper.  From their Abstract: &quot;We show that computing the volume of a polyhedron given either as a list of facets or as a list of vertices is as hard as computing the permanent of a matrix.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 1991 paper by Jim Lawrence, &lt;a href=&quot;http://fma2.math.uni-magdeburg.de/~henk/lectures/konvexgeometrie%20analytische%20aspekte/seminar/lawrence&amp;amp;polytope%20volume%20computation.pdf&quot; rel=&quot;nofollow&quot;&gt;Polytope Volume Computation&lt;/a&gt;, can be read online and has some references for the specifically 3D case.  He writes, &quot;The method in the present paper avoids triangulation of $P$ or of its boundary.&quot;  Also the algorithm described in that paper seems appropriate for your situation as it expresses &quot;the volume of $P$ as a sum of numbers $N_v$, one for each vertex $v$ of $P$. These numbers are easy to compute, so the difficulty of the procedure is mainly enumerating the vertices of $P$.&quot;  The difficulty probably translates in your situation to finding an expression for $P$ as the solution of a system of linear inequalities $P = \{x \in \mathbb{R}^3 : Ax \le b \}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you already know such an &quot;intersection of half-spaces&quot; expression for $P$, then perhaps it will permit a fairly fast computation.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2013-07-30T17:14:25.620" LastActivityDate="2013-07-30T17:14:25.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8096" PostTypeId="2" ParentId="8089" CreationDate="2013-07-30T00:43:46.730" Score="5" Body="&lt;p&gt;I'd be surprised if you can beat the suggestion of Shuhao Cao: compute the hull and then the volume once you have a triangulation of the hull. You could compute the hull with the $O(n^2)$ incremental algorithm, or the gift-wrapping algorithm. If you really want easy code, you can simply write an $n^4$ loop over all possible triangles to see if they are on the hull. For $n=15$, this is still pretty fast, and you can easily implement shortcuts. Once you have all the triangle faces, then pick one vertex $v$ and make a tetrahedron with each triangle $T$ and $v$. Its volume is a $4 \times 4$ determinant in the vertex coordinates.&lt;/p&gt;&#xA;" OwnerUserId="673" LastActivityDate="2013-07-30T00:43:46.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8097" PostTypeId="1" AcceptedAnswerId="8230" CreationDate="2013-07-30T06:29:34.457" Score="1" ViewCount="1941" Body="&lt;p&gt;I'm trying to write a perona malik anisotropic diffusion filter for the GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm basing my shader off a matlab implementation of the filter. I'm running into trouble because of what I suspect is floating point arithmetic issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a line in my shader&lt;/p&gt;&#xA;&#xA;&lt;p&gt;float cN  = 1 / pow( 1 + (nablaN / k), 2);&lt;/p&gt;&#xA;&#xA;&lt;p&gt;cN always evaluates to zero, whereas the matlab version does not. I think I am mishandling floating point numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I handle this properly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nabla is a float computed by convolution of the texture with a kernel hN.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float4 nablaN = Convolution(psInput, hN);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The kernel is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;static const float3x3 hN =&#xA;{&#xA;    0,  1, 0,&#xA;    0, -1, 0,&#xA;    0,  0, 0&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and convolution is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float4 Convolution(VertexShaderOutput input, float3x3 kernel)&#xA;{&#xA;    float4 pixel = float4(0.0f, 0.0f, 0.0f, 0.0f);&#xA;&#xA;    for (int i = -1; i &amp;lt;= 1; ++i)&#xA;    {&#xA;        for (int j = -1; j &amp;lt;= 1; ++j)&#xA;        {&#xA;            pixel += kernel[i+1][j+1] * tex2D(Input0Sampler, input.TextureCoordinate + float2(i,j));&#xA;        };&#xA;    };&#xA;&#xA;    return pixel;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I return nablaN from the pixel shader I get an image like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://imageshack.us/photo/my-images/819/dw69.png/&quot; rel=&quot;nofollow&quot;&gt;http://imageshack.us/photo/my-images/819/dw69.png/&lt;/a&gt;&#xA;(nablaN/2 is similar)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If i return ( 1 + (nablaN / k)) * ( 1 + (nablaN / k)) from the pixel shader I get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;something looking like this &#xA;&lt;a href=&quot;http://imageshack.us/photo/my-images/822/s49z.png/&quot; rel=&quot;nofollow&quot;&gt;http://imageshack.us/photo/my-images/822/s49z.png/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This looks similar to an edge detection filter. Most of the pixel values are zero. &#xA;So you can see why 1/(( 1 + (nablaN / k)) * ( 1 + (nablaN / k)) ) just gives me a black screen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK I have three versions of the program now in three languages&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FREEMAT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function diff_im = anisodiff2D(im, num_iter, delta_t, kappa, option)&#xA;%ANISODIFF2D Conventional anisotropic diffusion&#xA;%   DIFF_IM = ANISODIFF2D(IM, NUM_ITER, DELTA_T, KAPPA, OPTION) perfoms &#xA;%   conventional anisotropic diffusion (Perona &amp;amp; Malik) upon a gray scale&#xA;%   image. A 2D network structure of 8 neighboring nodes is considered for &#xA;%   diffusion conduction.&#xA;% &#xA;%       ARGUMENT DESCRIPTION:&#xA;%               IM       - gray scale image (MxN).&#xA;%               NUM_ITER - number of iterations. &#xA;%               DELTA_T  - integration constant (0 &amp;lt;= delta_t &amp;lt;= 1/7).&#xA;%                          Usually, due to numerical stability this &#xA;%                          parameter is set to its maximum value.&#xA;%               KAPPA    - gradient modulus threshold that controls the conduction.&#xA;%               OPTION   - conduction coefficient functions proposed by Perona &amp;amp; Malik:&#xA;%                          1 - c(x,y,t) = exp(-(nablaI/kappa).^2),&#xA;%                              privileges high-contrast edges over low-contrast ones. &#xA;%                          2 - c(x,y,t) = 1./(1 + (nablaI/kappa).^2),&#xA;%                              privileges wide regions over smaller ones. &#xA;% &#xA;%       OUTPUT DESCRIPTION:&#xA;%                DIFF_IM - (diffused) image with the largest scale-space parameter.&#xA;% &#xA;%   Example&#xA;%   -------------&#xA;%   s = phantom(512) + randn(512);&#xA;%   num_iter = 15;&#xA;%   delta_t = 1/7;&#xA;%   kappa = 30;&#xA;%   option = 2;&#xA;%   ad = anisodiff2D(s,num_iter,delta_t,kappa,option);&#xA;%   figure, subplot 121, imshow(s,[]), subplot 122, imshow(ad,[])&#xA;% &#xA;% See also anisodiff1D, anisodiff3D.&#xA;&#xA;% References: &#xA;%   P. Perona and J. Malik. &#xA;%   Scale-Space and Edge Detection Using Anisotropic Diffusion.&#xA;%   IEEE Transactions on Pattern Analysis and Machine Intelligence, &#xA;%   12(7):629-639, July 1990.&#xA;% &#xA;%   G. Grieg, O. Kubler, R. Kikinis, and F. A. Jolesz.&#xA;%   Nonlinear Anisotropic Filtering of MRI Data.&#xA;%   IEEE Transactions on Medical Imaging,&#xA;%   11(2):221-232, June 1992.&#xA;% &#xA;%   MATLAB implementation based on Peter Kovesi's anisodiff(.):&#xA;%   P. D. Kovesi. MATLAB and Octave Functions for Computer Vision and Image Processing.&#xA;%   School of Computer Science &amp;amp; Software Engineering,&#xA;%   The University of Western Australia. Available from:&#xA;%   &amp;lt;http://www.csse.uwa.edu.au/~pk/research/matlabfns/&amp;gt;.&#xA;% &#xA;% Credits:&#xA;% Daniel Simoes Lopes&#xA;% ICIST&#xA;% Instituto Superior Tecnico - Universidade Tecnica de Lisboa&#xA;% danlopes (at) civil ist utl pt&#xA;% http://www.civil.ist.utl.pt/~danlopes&#xA;%&#xA;% May 2007 original version.&#xA;&#xA;% Convert input image to double.&#xA;im = double(im);&#xA;&#xA;% PDE (partial differential equation) initial condition.&#xA;diff_im = im;&#xA;&#xA;% Center pixel distances.&#xA;dx = 1;&#xA;dy = 1;&#xA;dd = sqrt(2);&#xA;&#xA;% 2D convolution masks - finite differences.&#xA;hN = [0 1 0; 0 -1 0; 0 0 0];&#xA;hS = [0 0 0; 0 -1 0; 0 1 0];&#xA;hE = [0 0 0; 0 -1 1; 0 0 0];&#xA;hW = [0 0 0; 1 -1 0; 0 0 0];&#xA;hNE = [0 0 1; 0 -1 0; 0 0 0];&#xA;hSE = [0 0 0; 0 -1 0; 0 0 1];&#xA;hSW = [0 0 0; 0 -1 0; 1 0 0];&#xA;hNW = [1 0 0; 0 -1 0; 0 0 0];&#xA;&#xA;nablaN = zeros(1072,1912);&#xA;nablaS = zeros(1072,1912);&#xA;nablaW = zeros(1072,1912);&#xA;nablaE = zeros(1072,1912);&#xA;nablaNE = zeros(1072,1912);&#xA;nablaSE = zeros(1072,1912);&#xA;nablaSW = zeros(1072,1912);&#xA;nablaNW = zeros(1072,1912);&#xA;&#xA;% Anisotropic diffusion.&#xA;for t = 1:num_iter&#xA;&#xA;        % Finite differences. [imfilter(.,.,'conv') can be replaced by conv2(.,.,'same')]&#xA;        % nablaN = conv2(diff_im,hN)(2:1081,2:1921);&#xA;        nablaN = conv2(diff_im,hN)(2:1073,2:1913);&#xA;        nablaS = conv2(diff_im,hS)(2:1073,2:1913);&#xA;        nablaW = conv2(diff_im,hW)(2:1073,2:1913);&#xA;        nablaE = conv2(diff_im,hE)(2:1073,2:1913);&#xA;        nablaNE = conv2(diff_im,hNE)(2:1073,2:1913);&#xA;        nablaSE = conv2(diff_im,hSE)(2:1073,2:1913);&#xA;        nablaSW = conv2(diff_im,hSW)(2:1073,2:1913);&#xA;        nablaNW = conv2(diff_im,hNW)(2:1073,2:1913);&#xA;&#xA;        % Diffusion function.&#xA;        if option == 1&#xA;            cN = exp(-(nablaN/kappa).^2);&#xA;            cS = exp(-(nablaS/kappa).^2);&#xA;            cW = exp(-(nablaW/kappa).^2);&#xA;            cE = exp(-(nablaE/kappa).^2);&#xA;            cNE = exp(-(nablaNE/kappa).^2);&#xA;            cSE = exp(-(nablaSE/kappa).^2);&#xA;            cSW = exp(-(nablaSW/kappa).^2);&#xA;            cNW = exp(-(nablaNW/kappa).^2);&#xA;        elseif option == 2&#xA;            cN = 1./(1 + (nablaN/kappa).^2);&#xA;            cS = 1./(1 + (nablaS/kappa).^2);&#xA;            cW = 1./(1 + (nablaW/kappa).^2);&#xA;            cE = 1./(1 + (nablaE/kappa).^2);&#xA;            cNE = 1./(1 + (nablaNE/kappa).^2);&#xA;            cSE = 1./(1 + (nablaSE/kappa).^2);&#xA;            cSW = 1./(1 + (nablaSW/kappa).^2);&#xA;            cNW = 1./(1 + (nablaNW/kappa).^2);&#xA;        end&#xA;&#xA;        % Discrete PDE solution.&#xA;        diff_im = diff_im + ...&#xA;                  delta_t*(...&#xA;                  (1/(dy^2))*cN.*nablaN + (1/(dy^2))*cS.*nablaS + ...&#xA;                  (1/(dx^2))*cW.*nablaW + (1/(dx^2))*cE.*nablaE + ...&#xA;                  (1/(dd^2))*cNE.*nablaNE + (1/(dd^2))*cSE.*nablaSE + ...&#xA;                  (1/(dd^2))*cSW.*nablaSW + (1/(dd^2))*cNW.*nablaNW );&#xA;&#xA;        % Iteration warning.&#xA;        fprintf('\rIteration %d\n',t);        &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;HLSL&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;texture2D Input0;&#xA;sampler2D Input0Sampler = sampler_state&#xA;{&#xA;    Texture = &amp;lt;Input0&amp;gt;;&#xA;    MinFilter = Point;&#xA;    MagFilter = Point;&#xA;    MipFilter = Point;&#xA;    AddressU = Clamp;&#xA;    AddressV = Clamp;&#xA;};&#xA;&#xA;struct VertexShaderInput&#xA;{&#xA;    float4 Position : POSITION0;&#xA;    float2 TextureCoordinate : TEXCOORD0;&#xA;};&#xA;&#xA;struct VertexShaderOutput&#xA;{&#xA;    float4 Position : POSITION0;&#xA;    float2 TextureCoordinate : TEXCOORD0;&#xA;};&#xA;&#xA;struct PixelShaderOutput&#xA;{&#xA;    // TODO: Optionally add/remove output indices to match GPUProcessor.numOutputs&#xA;    float4 Index0 : COLOR0;&#xA;};&#xA;&#xA;// input texture dimensions&#xA;static float w = 1920 - 8;&#xA;static float h = 1080 - 8;&#xA;&#xA;static const float2 pixel = float2(1.0 / w, 1.0 / h);&#xA;static const float2 halfPixel = float2(pixel.x / 2, pixel.y / 2);&#xA;&#xA;static const float3x3 hN =&#xA;{&#xA;    0,  1, 0,&#xA;    0, -1, 0,&#xA;    0,  0, 0&#xA;};&#xA;static const float3x3 hS =&#xA;{&#xA;    0,  0, 0,&#xA;    0, -1, 0,&#xA;    0,  1, 0&#xA;};&#xA;static const float3x3 hE =&#xA;{&#xA;    0,  0, 0,&#xA;    0, -1, 1,&#xA;    0,  0, 0&#xA;};&#xA;static const float3x3 hW =&#xA;{&#xA;    0,  0, 0,&#xA;    1, -1, 0,&#xA;    0,  0, 0&#xA;};&#xA;static const float3x3 hNE =&#xA;{&#xA;    0,  0, 1,&#xA;    0, -1, 0,&#xA;    0,  0, 0&#xA;};&#xA;static const float3x3 hSE =&#xA;{&#xA;    0,  0, 0,&#xA;    0, -1, 0,&#xA;    0,  0, 1&#xA;};&#xA;static const float3x3 hSW =&#xA;{&#xA;    0,  0, 0,&#xA;    0, -1, 0,&#xA;    1,  0, 0&#xA;};&#xA;static const float3x3 hNW =&#xA;{&#xA;    1,  0, 0,&#xA;    0, -1, 0,&#xA;    0,  0, 0&#xA;};&#xA;&#xA;VertexShaderOutput VertexShaderFunction(VertexShaderInput vsInput)&#xA;{&#xA;    //VertexShaderOutput output;&#xA;&#xA;    //output.Position = vsInput.Position;&#xA;    //output.TextureCoordinate = vsInput.TextureCoordinate;&#xA;&#xA;    VertexShaderOutput output;&#xA;    vsInput.Position.x =  vsInput.Position.x - 2*halfPixel.x;&#xA;    vsInput.Position.y =  vsInput.Position.y + 2*halfPixel.y;&#xA;    output.Position = vsInput.Position;&#xA;    output.TextureCoordinate = vsInput.TextureCoordinate;&#xA;    return output;&#xA;&#xA;    //return output;&#xA;}&#xA;&#xA;float Convolution(VertexShaderOutput input, float3x3 kernel)&#xA;{&#xA;    //float4 pixel = float4(0.0f, 0.0f, 0.0f, 0.0f);&#xA;    float pixel = 0.0L;&#xA;&#xA;    for (int i = -1; i &amp;lt;= 1; ++i)&#xA;    {&#xA;        for (int j = -1; j &amp;lt;= 1; ++j)&#xA;        {&#xA;            pixel += (kernel[i+1][j+1] * tex2D(Input0Sampler, input.TextureCoordinate + float2(i,j))).x;&#xA;        };&#xA;    };&#xA;&#xA;    return pixel;&#xA;}&#xA;&#xA;PixelShaderOutput PixelShaderFunction(VertexShaderOutput psInput)&#xA;{       &#xA;    PixelShaderOutput output;&#xA;    //output.Index0 = float4(0,0,0,0);//tex2D(Input0Sampler, psInput.TextureCoordinate);&#xA;    //output.Index0 = tex2D(Input0Sampler, psInput.TextureCoordinate);&#xA;&#xA;    float big = 100000.0f;&#xA;    float total = tex2D(Input0Sampler, psInput.TextureCoordinate);&#xA;    float dx, dy, dd;&#xA;    dx = 1; dy = 1; dd = pow(2.0L, 0.5L);&#xA;    float delta_t = 1/7;//0.25f;&#xA;    float k = 30.0;//0.015f;&#xA;&#xA;    float nablaN = Convolution(psInput, hN);&#xA;    float nablaS = Convolution(psInput, hS);&#xA;    float nablaW = Convolution(psInput, hW);&#xA;    float nablaE = Convolution(psInput, hE);&#xA;    float nablaNE = Convolution(psInput, hNE);&#xA;    float nablaSE = Convolution(psInput, hSE);&#xA;    float nablaSW = Convolution(psInput, hSW);&#xA;    float nablaNW = Convolution(psInput, hNW);&#xA;&#xA;&#xA;    float cN  = ( 1 + (nablaN / k)) * ( 1 + (nablaN / k));&#xA;    float cS  = ( 1 + (nablaS / k)) * ( 1 + (nablaS / k));&#xA;    float cW  = ( 1 + (nablaW / k)) * ( 1 + (nablaW / k));&#xA;    float cE  = ( 1 + (nablaE / k)) * ( 1 + (nablaE / k));&#xA;    float cNE = ( 1 + (nablaNE / k)) * ( 1 + (nablaNE / k));&#xA;    float cSE = ( 1 + (nablaSE / k)) * ( 1 + (nablaSE / k));&#xA;    float cSW = ( 1 + (nablaSW / k)) * ( 1 + (nablaSW / k));&#xA;    float cNW = ( 1 + (nablaNW / k)) * ( 1 + (nablaNW / k));&#xA;&#xA;    /*  &#xA;    float4 cN  = exp(-(nablaN/k)*(nablaN/k));&#xA;    float4 cS  = exp(-(nablaS/k)*(nablaS/k));&#xA;    float4 cW  = exp(-(nablaW/k)*(nablaW/k));&#xA;    float4 cE  = exp(-(nablaE/k)*(nablaE/k));&#xA;    float4 cNE = exp(-(nablaNE/k)*(nablaNE/k));&#xA;    float4 cSE = exp(-(nablaSE/k)*(nablaSE/k));&#xA;    float4 cSW = exp(-(nablaSW/k)*(nablaSW/k));&#xA;    float4 cNW = exp(-(nablaNW/k)*(nablaNW/k));&#xA;    */&#xA;&#xA;    total += delta_t * &#xA;    (&#xA;        //(mul(cN, nablaN)/(dy*dy)) + (mul(cS, nablaS)/(dy*dy)) + (mul(cW, nablaW)/(dx*dx)) + (mul(cE, nablaE)/(dx*dx)) + (dd*dd)*(mul(cNE, nablaNE) + mul(cSE, nablaSE) + mul(cSW, nablaSW) + mul(cNW, nablaNW))&#xA;        (nablaN/cN) + (nablaS/cS) + (nablaW/cW) + (nablaE/cE) + 2*((nablaNE/cNE) + (nablaSE/cSE) + (nablaSW/cSW) + (nablaNW/cNW))&#xA;    );&#xA;&#xA;&#xA;    output.Index0 = total;&#xA;&#xA;    return output;&#xA;}&#xA;&#xA;technique PeronaMalik&#xA;{&#xA;    pass pass1&#xA;    {&#xA;        VertexShader = compile vs_3_0 VertexShaderFunction();&#xA;        PixelShader  = compile ps_3_0 PixelShaderFunction();&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OpenCL&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float4 Convolution(__read_only image2d_t srcImg, int2 point, float * kern)&#xA;{&#xA;    const sampler_t smp = CLK_NORMALIZED_COORDS_FALSE | CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_LINEAR;&#xA;    int maskSize = 1;&#xA;    float4 sum = (float4)(0.0f,0.0f,0.0f,0.0f);&#xA;    for (int i = -maskSize; i &amp;lt;= maskSize; i++)&#xA;    {&#xA;        for(int j = -maskSize; j &amp;lt;= maskSize; j++) &#xA;        {&#xA;            int2 delta = (int2)(i+maskSize,j+maskSize);                 &#xA;            sum += kern[(delta.y*maskSize) + delta.x] * convert_float4(read_imageui(srcImg, smp, point + delta));&#xA;        }&#xA;    }&#xA;    return sum;&#xA;}&#xA;&#xA;&#xA;__kernel void imagingTest(__read_only  image2d_t srcImg, __write_only image2d_t dstImg)&#xA;{&#xA;    float k = 30;&#xA;    float delta_t = 1/7;&#xA;&#xA;    float hN[9];&#xA;    hN[0] = 0; hN[1] = 1; hN[2] = 0;&#xA;    hN[3] = 0; hN[4] =-1; hN[5] = 0; &#xA;    hN[6] = 0; hN[7] = 0; hN[8] = 0;&#xA;&#xA;    float hS[9];&#xA;    hS[0] = 0; hS[1] = 0; hS[2] = 0;&#xA;    hS[3] = 0; hS[4] =-1; hS[5] = 0; &#xA;    hS[6] = 0; hS[7] = 1; hS[8] = 0;&#xA;&#xA;    float hE[9];&#xA;    hE[0] = 0; hE[1] = 0; hE[2] = 0;&#xA;    hE[3] = 0; hE[4] =-1; hE[5] = 1; &#xA;    hE[6] = 0; hE[7] = 0; hE[8] = 0;&#xA;&#xA;    float hW[9];&#xA;    hW[0] = 0; hW[1] = 0; hW[2] = 0;&#xA;    hW[3] = 1; hW[4] =-1; hW[5] = 0; &#xA;    hW[6] = 0; hW[7] = 0; hW[8] = 0;&#xA;&#xA;    float hNE[9];&#xA;    hNE[0] = 0; hNE[1] = 0; hNE[2] = 1;&#xA;    hNE[3] = 0; hNE[4] =-1; hNE[5] = 0; &#xA;    hNE[6] = 0; hNE[7] = 0; hNE[8] = 0;&#xA;&#xA;    float hSE[9];&#xA;    hSE[0] = 0; hSE[1] = 0; hSE[2] = 0;&#xA;    hSE[3] = 0; hSE[4] =-1; hSE[5] = 0; &#xA;    hSE[6] = 0; hSE[7] = 0; hSE[8] = 1;&#xA;&#xA;    float hSW[9];&#xA;    hSW[0] = 0; hSW[1] = 0; hSW[2] = 0;&#xA;    hSW[3] = 0; hSW[4] =-1; hSW[5] = 0; &#xA;    hSW[6] = 1; hSW[7] = 0; hSW[8] = 0;&#xA;&#xA;    float hNW[9];&#xA;    hNW[0] = 1; hNW[1] = 0; hNW[2] = 0;&#xA;    hNW[3] = 0; hNW[4] =-1; hNW[5] = 0; &#xA;    hNW[6] = 0; hNW[7] = 0; hNW[8] = 0;&#xA;&#xA;    const sampler_t smp = CLK_NORMALIZED_COORDS_FALSE | CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_LINEAR;&#xA;&#xA;    int2 coord = (int2)(get_global_id(0), get_global_id(1));&#xA;&#xA;    uint4 bgra = read_imageui(srcImg, smp, coord); &#xA;&#xA;    float4 nablaN = Convolution(srcImg, coord, hN);&#xA;    float4 nablaS = Convolution(srcImg, coord, hS);&#xA;    float4 nablaE = Convolution(srcImg, coord, hE);&#xA;    float4 nablaW = Convolution(srcImg, coord, hW);&#xA;&#xA;    float4 nablaNE = Convolution(srcImg, coord, hNE);&#xA;    float4 nablaNW = Convolution(srcImg, coord, hNW);&#xA;    float4 nablaSE = Convolution(srcImg, coord, hSE);&#xA;    float4 nablaSW = Convolution(srcImg, coord, hSW);&#xA;&#xA;    float4 cN  = exp(-(nablaN/k)*(nablaN/k));&#xA;    float4 cS  = exp(-(nablaS/k)*(nablaS/k));&#xA;    float4 cW  = exp(-(nablaW/k)*(nablaW/k));&#xA;    float4 cE  = exp(-(nablaE/k)*(nablaE/k));&#xA;    float4 cNE = exp(-(nablaNE/k)*(nablaNE/k));&#xA;    float4 cSE = exp(-(nablaSE/k)*(nablaSE/k));&#xA;    float4 cSW = exp(-(nablaSW/k)*(nablaSW/k));&#xA;    float4 cNW = exp(-(nablaNW/k)*(nablaNW/k));&#xA;&#xA;    float4 sum = delta_t * ((nablaN/cN) + (nablaS/cS) + (nablaW/cW) + (nablaE/cE) + 2*((nablaNE/cNE) + (nablaSE/cSE) + (nablaSW/cSW) + (nablaNW/cNW)));&#xA;&#xA;    bgra.x = bgra.y = bgra.z = convert_int(sum.x);&#xA;&#xA;    bgra.w = 255;&#xA;    write_imageui(dstImg, coord, bgra);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It works fine in freemat, its just really slow. The OpenCL and HLSL versions are really fast but they don't give the right results. &lt;/p&gt;&#xA;" OwnerUserId="4680" LastEditorUserId="4680" LastEditDate="2013-08-13T06:19:26.577" LastActivityDate="2013-08-13T06:19:26.577" Title="How to handle floating point operations in HLSL?" Tags="|floating-point|gpu|diffusion|opencl|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8098" PostTypeId="1" CreationDate="2013-07-30T07:20:30.897" Score="5" ViewCount="382" Body="&lt;p&gt;We know that if the matrix $A$ is symmetric positive definite, the FOM (full orthogonalization method) and the GMRES are theoretically equivalent to the CG (conjugate gradient) and the CR (conjugate residual). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you introduce some good references in the CR, especially for the convergence properties of the CR? &lt;/p&gt;&#xA;" OwnerUserId="4866" LastEditorUserId="20688" LastEditDate="2017-04-03T07:14:36.537" LastActivityDate="2017-04-19T04:52:25.860" Title="A question about GMRES" Tags="|linear-algebra|linear-solver|reference-request|iterative-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8099" PostTypeId="1" CreationDate="2013-07-30T07:36:49.260" Score="8" ViewCount="1416" Body="&lt;p&gt;I want to compute the &lt;strong&gt;Generalized Singular Value Decomposition&lt;/strong&gt; for sparse matrices with a size of up to 1000000 x 1000000 (not necessarily square). The method is going to be used in machine learning (classification).&lt;/p&gt;&#xA;&lt;p&gt;Currently I'm using the function &lt;strong&gt;ggsvd&lt;/strong&gt; (&lt;a href=&quot;http://www.netlib.org/lapack/explore-html/dd/db4/dggsvd_8f.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) from the &lt;strong&gt;LAPACK&lt;/strong&gt; package which is a &lt;strong&gt;Fortran&lt;/strong&gt; implementation.&lt;/p&gt;&#xA;&lt;p&gt;What it does is computing matrices U, V, X, C, S for given matrices A and B such that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;A = U * C * transposed(X)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;B = V * S * transposed(X)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;where C and S are diagonal matrices containing the singular values of A and B respective. (&lt;a href=&quot;http://www.mathworks.de/de/help/matlab/ref/gsvd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explanation from Matlab&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;With SVD one can do a low-rank approximation meaning that only the r largest singular values with most information are being computed to approximate the input matrix.&lt;/p&gt;&#xA;&lt;p&gt;I am now wondering if this is also possible in the GSVD so that not all the singular values have to be computed. I would like to compute only the r largest values or to stop computing when a value is below a given threshold.&lt;/p&gt;&#xA;&lt;p&gt;My assumption is that it's not possible because the singular values of A and B are computed pairwise and if a pair contains the largest singular value of A its mate of B doesn't have to be necessarily.&lt;/p&gt;&#xA;&lt;p&gt;A short explanation why it is possible (and maybe a link to an implementation that offers this option) or not would be very nice.&lt;/p&gt;&#xA;" OwnerUserId="4867" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2013-08-13T16:37:53.370" Title="Generalized Singular Value Decomposition: only compute the r largest singular values" Tags="|linear-algebra|matrix|fortran|svd|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8101" PostTypeId="2" ParentId="8094" CreationDate="2013-07-30T13:51:27.623" Score="6" Body="&lt;p&gt;There's no guarantee in the standard that any progress is made on the non-blocking sends until you actually call &lt;code&gt;MPI_WAIT&lt;/code&gt;. It's a perfectly valid implementation to just queue up the operations and when you call &lt;code&gt;MPI_WAIT&lt;/code&gt;, all of the &lt;code&gt;MPI_ISEND&lt;/code&gt; operations complete at once. In reality, they usually tend to get a chance to progress anytime you enter the MPI library and if you enable asynchronous progress threads, they have a better chance of progressing in the background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the signature issue, MPI guarantees that messages on the same communicator to the same ranks will be received in the same order in which they were sent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the MPI Standard version 3.0:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Order Messages are non-overtaking: If a sender sends two messages in&#xA;  succession to the same destination, and both match the same receive,&#xA;  then this operation cannot receive the second message if the first one&#xA;  is still pending. If a receiver posts two receives in succession, and&#xA;  both match the same message, then the second receive operation cannot&#xA;  be satisfied by this message, if the first one is still pending.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This doesn't say anything about how the implementation chooses to send the messages, but at least they will be received in the correct order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice would be to first make sure you have progress threads enabled, then make sure you're calling wait where you really need the messages sent (though with progress threads, you'll most likely be fine).&lt;/p&gt;&#xA;" OwnerUserId="4658" LastActivityDate="2013-07-30T13:51:27.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8102" PostTypeId="2" ParentId="8093" CreationDate="2013-07-30T14:22:27.047" Score="7" Body="&lt;p&gt;You are facing one of the most tedious and error prone aspect of elasticity theory: change of reference frame in &lt;a href=&quot;https://en.wikipedia.org/wiki/Voigt_notation&quot; rel=&quot;noreferrer&quot;&gt;engineering (or Voigt) notation&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Recap theory&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If $\boldsymbol{\epsilon}$ and $\boldsymbol{\sigma}$ are the strain and stress engineering components (represented as $6\times 1$ vectors), then you can change reference of frame with the following transformation:&#xA;\begin{gather}&#xA;\boldsymbol{\epsilon}' = T_\epsilon \, \boldsymbol{\epsilon}\\&#xA;\boldsymbol{\sigma}' = T_\sigma \, \boldsymbol{\sigma}&#xA;\end{gather}&#xA;where $T_\sigma, T_\epsilon$ are $6\times 6$ matrices (containing products of directional cosines). Note that these matrices &lt;em&gt;are not&lt;/em&gt; orthogonal; but invoking the invariance of $\boldsymbol{\sigma}^T\boldsymbol{\epsilon}$ we have&#xA;\begin{equation}&#xA;T_\epsilon = T_\sigma ^ {-T}&#xA;\end{equation}&#xA;For the stifness matrix $D$, remembering that $\boldsymbol{\sigma} = D \boldsymbol{\epsilon}$ becomes $\boldsymbol{\sigma}' = D' \boldsymbol{\epsilon}'$ we have&#xA;\begin{equation}&#xA;D' = T_\sigma \, D\,  T_\epsilon^{-1} = T_\sigma \, D\,  T_\sigma^T&#xA;\end{equation}&#xA;Note however that the compliance matrix $C = D^{-1}$ follows a different rule:&#xA;\begin{equation}&#xA;C\,{}' = T_\epsilon \, C\,  T_\sigma^{-1} = T_\epsilon \, C\,  T_\epsilon^T&#xA;\end{equation}&#xA;Expressions for $T_\epsilon$ and $T_\sigma$ are terrible and forgive me if I do not copy them here.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The answer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The easiest way to check for a correct expression of $T_\sigma$ is to compute the components of $\boldsymbol{\sigma}'$ in tensor notation $\sigma_{ij}'=a_{im}a_{jl}\sigma_{ml}$ where $a_{ij}$ are the directional cosines of the given transformation. In matrix notation this amounts to $\Sigma' = A \Sigma A^T$, where $A$ is the usual unitary rotation matrix, so its very hard to do a mistake here. Then you compare the components of $T_\sigma\boldsymbol{\sigma}$ and $\Sigma'$ to see if they match. Test this for a number of arbitrary rotations.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Things to be aware of&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The expression of $T_\sigma$ and $T_\epsilon$ depend on the ordering of $\boldsymbol{\sigma}$ and $\boldsymbol{\epsilon}$: double check that you are consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compliance and stiffness matrix in engineering notation have different transformation laws, owing to the fact that $T$ matrices are not unitary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_\sigma$ and $T_\epsilon$ are almost equal, up to a factor of two, because engineering shear strains are twice the corresponding tensor components.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2013-07-30T21:51:50.770" LastActivityDate="2013-07-30T21:51:50.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8103" PostTypeId="1" CreationDate="2013-07-30T14:32:52.593" Score="8" ViewCount="1344" Body="&lt;p&gt;I'm searching for the minimum path between the minima of a potential surface that is already known on a grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/knRxm.gif&quot; alt=&quot;example&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(source: &lt;a href=&quot;http://www.math.nus.edu.sg/~matrw/string/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.nus.edu.sg/~matrw/string/&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any point on the path is at an potential minimum in all directions perpendicular to the path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any SciPy method or any other python package to compute this path?&#xA;I'm not looking for a method that can search on an unknown surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit:&lt;/strong&gt; I'm searching the path with the lowest potential barrier.&lt;/p&gt;&#xA;" OwnerUserId="3486" LastEditorUserId="22284" LastEditDate="2019-01-06T10:03:17.033" LastActivityDate="2019-01-06T10:03:17.033" Title="Minimum path on known potential surface" Tags="|optimization|python|scipy|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8104" PostTypeId="2" ParentId="8099" CreationDate="2013-07-30T15:01:02.380" Score="4" Body="&lt;p&gt;I'm not an expert in this field, but being your $A$ and $B$ sparse, matlab and LAPACK are not a good choice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sparse matrices a quick literature search confirmed that algorithms for the computation of a few extremal generalized singular values have been described, as one might expect. (Some random results from google scholar: &lt;a href=&quot;http://www.win.tue.nl/~hochsten/pdf/jdgsvd.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.win.tue.nl/~hochsten/pdf/jdgsvd.pdf&lt;/a&gt; and &lt;a href=&quot;http://www.cs.berkeley.edu/~richie/cs294.data/project/reading/zha_sparse_gensvd.pdf.gz&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.berkeley.edu/~richie/cs294.data/project/reading/zha_sparse_gensvd.pdf.gz&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I'm not aware of their implementation in available libraries, but here I hope to be contradicted from the more knowledgeable.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-07-30T15:01:02.380" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8105" PostTypeId="2" ParentId="8089" CreationDate="2013-07-30T16:30:46.190" Score="2" Body="&lt;p&gt;A small test in MATLAB, for number of vertex $N = 100$, each component is a uniform random number in $[0,1]$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 100;&#xA;p=rand(N,3);&#xA;tic;&#xA;T = delaunayTri(p(:,1),p(:,2),p(:,3));&#xA;t = T.Triangulation;&#xA;e1 = p(t(:,2),:)-p(t(:,1),:);&#xA;e2 = p(t(:,3),:)-p(t(:,1),:);&#xA;e3 = p(t(:,4),:)-p(t(:,1),:);&#xA;V = abs(dot(cross(e1,e2,2),e3,2))/6;&#xA;Vol = sum(V);&#xA;time_elapse = toc;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time_elapse =&#xA;              0.014807&#xA;Vol =&#xA;      0.67880219135839&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would say it is reasonably fast, if you wanna run it $10^6$ times, it only takes less than 3 hours. Here is what it is like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PFkhV.png&quot; alt=&quot;convhull&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I want to mention that in Professor O'Rourke's post he mentioned using determinant to compute tetrahedra's volumes, yet here I prefer using triple product. It is a naturally vectorized operation, more scalable than the built-in routine of determinant (or you can expand a $4\times 4$ determinant by hand :p). Here is another test for $N=10^5$, the result is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time_elapse =&#xA;              3.244278&#xA;Vol =&#xA;     0.998068316875714&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with tetrahedra number $\approx 7\times 10^5$. Notice the total volume is pretty closed to $1$ for there are too many points clustered in $[0,1]^3$.&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-07-30T16:30:46.190" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8106" PostTypeId="1" CreationDate="2013-07-30T20:44:28.993" Score="4" ViewCount="467" Body="&lt;p&gt;Sorry if this is a basic problem but I don't know where to start looking (mainly because being an outsider I don't know the terms and nomenclature).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine two perpendicular lines (&quot;profiles&quot;) in a &quot;T&quot; spatial arrangement. The lines are arbitrary (empirical functions should I say?) in the sense that they don't follow any simple formula (but I have the data of each line. Each line is a velocity profile along a transect. One line is parallel to the X axis, the other is parallel to the Y axis. These lines have one point in common, ie there a point i where Xi=Yi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to interpolate between them to guess how the velocity distribution would look in the area defined by these two lines (eg the containing rectangle?). So I want to go from 2 known lines (1D) to a surface (2D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagine that I need a function Z=f(X,Y) such as when plotting this function I can have a 2D representation of the surface containing both lines (&quot;profiles&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lines are perpendicular. The lines are NOT straight lines (except when viewed from above). Imagine you measure velocity over two lines: one from A to B, another from C to D. When viewed from above, these lines are perpendicular and they have a point in common: they look like this: &lt;a href=&quot;https://www.dropbox.com/s/1ju2hzmkvijlz2j/1d.lines.vel.jpg&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/s/1ju2hzmkvijlz2j/1d.lines.vel.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the point they have in common is the beginning of one line, and the middle of the 2nd line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what I need is an algorithm which allows me to &quot;guess&quot; the &quot;surface&quot; (2D velocity vector field) defined by these two lines. I would think on a simple interpolation, but how? I need to obtain this: &lt;a href=&quot;https://www.dropbox.com/s/3is8zzetps4coq3/surface.png&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/s/3is8zzetps4coq3/surface.png&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sure this must be a VERY common problem in many many fields... But I don't know the math world.. Could you please provide some keywords so I start looking??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks very very much!!! &lt;/p&gt;&#xA;" OwnerUserId="4870" LastActivityDate="2013-08-30T11:20:47.777" Title="Surface interpolation from two lines" Tags="|interpolation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8107" PostTypeId="2" ParentId="8103" CreationDate="2013-07-30T23:02:26.217" Score="3" Body="&lt;p&gt;I don't know if you are aware of it but you can find some MATLAB examples (not Python, though) for the Mueller potential at &lt;a href=&quot;http://www.cims.nyu.edu/~eve2/string.htm&quot; rel=&quot;nofollow&quot;&gt;Eric Vanden-Eijnden's string method page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-07-30T23:02:26.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8108" PostTypeId="2" ParentId="5541" CreationDate="2013-07-31T01:26:52.697" Score="3" Body="&lt;p&gt;CESM and COSMOS have been mentioned. A group in Japan has their own model, although I forget what it's called. If you really need a comprehensive list and a comparison, I'd recommend taking a look at any of the papers published in CMIP project. Our's (COSMOS) is in there and you can see where each model has its weaknesses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as a clarification question, what sort of data are you looking for? Climate models differ a lot in what sort of things they do and don't include. Some have ice components, some have chemistry, some have aerosols and dust, etc etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to the follow up question -- Meteorological and Climate models are extremely similar. In fact, you can run an Atmospheric simulation in &quot;weather mode&quot; and get (sortof) realistic weather patterns. The key difference, however, is the time step and length of model integration. Climate deals with long term averages of a system in both space and time, whereas weather is principally concerned with what may be happening over a week or two in a very local region. As an example, you wouldn't bother looking up the weather in Virginia if you lived in Florida.&lt;/p&gt;&#xA;" OwnerUserId="4872" LastEditorUserId="81" LastEditDate="2013-08-03T09:28:13.100" LastActivityDate="2013-08-03T09:28:13.100" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8109" PostTypeId="1" CreationDate="2013-07-31T01:27:24.387" Score="2" ViewCount="78" Body="&lt;p&gt;While I look up programming and technical stuff on Stack Overflow all the time, I figured that would probably be the wrong forum for a question like this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some background: I'm a climate modeler working on paleoclimates and data comparison for warm climate modes, and I've been writing ice sheet models in C and C++ for a while. It's been painful but finally I'm happy enough with how the ice is behaving to incorporate it into the GCM we've been using. The question boils down to the following: Assuming you have a fairly decent simulation of the present day state, how long would you expect a simulation to need to run to adjust to a similar warm climate state, in my case the Eemian. I've run it through 2000 years, but my ocean salt is still climbing (very very slowly, but linearly -- 0.05% per century, more or less) Physically that translates to a little more than one full ocean overturning.&lt;/p&gt;&#xA;" OwnerUserId="4872" LastActivityDate="2013-08-30T10:20:44.977" Title="Model Spinup time" Tags="|stability|model|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8110" PostTypeId="2" ParentId="7356" CreationDate="2013-07-31T09:11:23.943" Score="1" Body="&lt;p&gt;Don't know if you already found an answer yet... Try looking here for some articles here: &#xA;&lt;a href=&quot;http://scholar.google.com/scholar?q=unsteady+compressible+turbomachinery+openfoam&quot; rel=&quot;nofollow&quot;&gt;http://scholar.google.com/scholar?q=unsteady+compressible+turbomachinery+openfoam&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenFoam is probably the most popular open source code that could possibly be used to solve this problem.  This presentation may be of help: &lt;a href=&quot;http://www.personal.psu.edu/dab143/OFW6/Presentations/alex_wouden_slides.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.personal.psu.edu/dab143/OFW6/Presentations/alex_wouden_slides.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a full list of available CFD codes, just google &quot;free cfd codes list&quot;.&lt;/p&gt;&#xA;" OwnerUserId="4873" LastActivityDate="2013-07-31T09:11:23.943" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8111" PostTypeId="2" ParentId="8109" CreationDate="2013-07-31T09:36:40.643" Score="3" Body="&lt;p&gt;Each of your subsystems -- and likely also each coupling between subsystems -- will have an intrinsic time scale. What that is of course depends on your system. The spin up time will then be a small multiple of the maximum of all of the time scales that are relevant in the transition you are considering.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-31T09:36:40.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8112" PostTypeId="2" ParentId="8106" CreationDate="2013-07-31T09:42:18.603" Score="2" Body="&lt;p&gt;Interpolation refers to finding data &lt;em&gt;between&lt;/em&gt; points. In other words, if you had the data on two perpendicular line segments, all you could do with interpolation is fill in something in the triangle enclosed by the two line segments, but not in an entire rectangle. For example, you could use a linear interpolation of the values between points $(i \Delta x, 0)$ and $(0, i \Delta y)$ at which you know the values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the rest of your rectangle, you can't do &lt;em&gt;inter&lt;/em&gt; polation but you could do &lt;em&gt;extra&lt;/em&gt; polation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, however, extrapolation has a lot to do with what you think you know the real data looks like. Should it be smooth? Should it be oscillatory? Whatever assumptions you put in, you will get out.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-31T09:42:18.603" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8113" PostTypeId="2" ParentId="8103" CreationDate="2013-07-31T09:44:23.030" Score="1" Body="&lt;p&gt;What you are looking for is a way to find a geodesic on a known surface with a known metric. This is a classical geometry problem. The place to look for algorithms is in books on computational geometry.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-07-31T09:44:23.030" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8114" PostTypeId="2" ParentId="8097" CreationDate="2013-07-31T09:47:34.840" Score="1" Body="&lt;p&gt;Is &lt;code&gt;nablaN&lt;/code&gt; an integer? Because if it is, then the expression &lt;code&gt;1 / pow( 1 + (nablaN / k), 2);&lt;/code&gt; is considered the division of an integer by an integer, which is going to be zero if &lt;code&gt;nablaN&lt;/code&gt; is greater than 2. These are just the rules of the C programming language.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-08-01T10:48:20.150" LastActivityDate="2013-08-01T10:48:20.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8115" PostTypeId="1" CreationDate="2013-07-31T09:50:52.423" Score="3" ViewCount="159" Body="&lt;p&gt;I'm working on a finite-volume discretization method with implicit time-integration for the incompressible Navier-Stokes equations in Arbitrary Lagrangian-Eularian (ALE) form. So far I've tested the implementation using manufactured solutions similar to the ones by &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2008.11.032&quot; rel=&quot;nofollow&quot;&gt;Etienne e.a., JCP 228 (2009)&lt;/a&gt;. However, I'm also looking for a test case which&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is physically meaningful (e.g. flow around moving objects)&lt;/li&gt;&#xA;&lt;li&gt;has prescribed motion (not true fluid-structure interaction)&lt;/li&gt;&#xA;&lt;li&gt;needs deforming grids (not just rigid-grid motion)&lt;/li&gt;&#xA;&lt;li&gt;is not too complicated (a complete fighter plane is too much), &lt;/li&gt;&#xA;&lt;li&gt;is preferably 3D (because some methods will work in 2D but not in 3D)&lt;/li&gt;&#xA;&lt;li&gt;is more or less standard (results by others and/or experiments available for comparison)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This would be ideal but other suggestions for testing flow simulations on deforming grids are welcome.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-08-02T05:06:57.543" Title="Test case suggestion for incompressible flow with ALE method on deforming grids" Tags="|testing|navier-stokes|incompressible|" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8116" PostTypeId="2" ParentId="8093" CreationDate="2013-07-31T09:53:29.273" Score="3" Body="&lt;p&gt;As a matter of general truth: Life becomes a lot easier and less error prone if you use $3\times 3$ matrices instead of writing tensors as 6-component vectors. The laws of transforming $3\times 3$ are much more obvious than the transformation of 6-component vector representation of these symmetric matrices.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="9667" LastEditDate="2016-01-12T16:24:33.137" LastActivityDate="2016-01-12T16:24:33.137" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8117" PostTypeId="1" CreationDate="2013-07-31T11:04:26.333" Score="9" ViewCount="480" Body="&lt;p&gt;I am wondering whether there is some very good &lt;strong&gt;mailing list&lt;/strong&gt; or &lt;strong&gt;google groups&lt;/strong&gt; for Computational Science, where we can discuss questions instead of only asking and replying questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, I am more interested parallel computing and numerical solutions of PDEs. But I don't know what and how people in this area are doing. I can only read their papers to grasp the road-map of this area. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please give me some guiding information. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4562" LastActivityDate="2014-02-26T22:36:23.450" Title="Is there some good mailing list for `Computational Science`?" Tags="|parallel-computing|reference-request|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8118" PostTypeId="2" ParentId="8115" CreationDate="2013-07-31T12:10:49.817" Score="2" Body="&lt;p&gt;One example that meets some (certainly not all) of your criteria is flow in a flexible tube in Badia, Quaini, and Quarteroni SIAM J. Sci. Comput. 30 (2008) pp. 1778ff.  There are also some straight tube examples in the PhD thesis of Fabio Nobile.  The advantages of these straight flexible tubes is mostly in relation to your last point, in that there are some results in the literature and some experiments, often under the label of &quot;Womersley flow&quot;.&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2013-07-31T12:10:49.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8119" PostTypeId="1" CreationDate="2013-07-31T14:23:56.547" Score="0" ViewCount="64" Body="&lt;p&gt;I am trying to simulate the gravitational interaction between many bodies. I am using a direct PP force calculation and a 4th order symplectic integrator with a variable step size.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The energy of the simulation is calculated by this function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real energy(body* system,int n){&#xA;  int i,k,j;&#xA;  real energy_var=0;&#xA;  for(i=0;i&amp;lt;n;i++){&#xA;    energy_var += 0.5*system[i].mass*pow(mag(system[i].vel),2);&#xA;    for(j=i+1;j&amp;lt;n;j++){&#xA;      if(i==j){continue;}&#xA;      real position_vect[3]={0,0,0};&#xA;&#xA;      for(k=0;k&amp;lt;3;k++){&#xA;    position_vect[k] = system[i].pos[k] - system[j].pos[k];}&#xA;&#xA;      real distance = mag(position_vect);&#xA;&#xA;      energy_var += -Gravitational_constant*system[i].mass*system[j].mass/distance;&#xA;    }&#xA;  }&#xA;  return energy_var;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In a test run with 100 bodies the energy( calculated every 100000's ) appears to be constant apart from one anomaly. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Energy = -3.23197e+41&#xA;Energy = -3.23197e+41&#xA;Energy = -3.23197e+41&#xA;Energy = -4.24807e+41&#xA;Energy = -3.23197e+41&#xA;Energy = -3.23197e+41&#xA;Energy = -3.23197e+41&#xA;Energy = -3.23197e+41&#xA;Energy = -3.23197e+41&#xA;Energy = -3.23197e+41&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If it was a numerical or coding error i.e. the timestep being too high then I would not expect the energy to return to the same constant value. Is there a known phenomenon that might cause such an error? &lt;/p&gt;&#xA;" OwnerUserId="3939" LastActivityDate="2013-07-31T14:23:56.547" Title="Energy anomolies in many body simulation" Tags="|ode|simulation|conservation|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8120" PostTypeId="1" CreationDate="2013-07-31T18:38:31.697" Score="4" ViewCount="832" Body="&lt;p&gt;&lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/metis/metis/overview&quot; rel=&quot;nofollow&quot;&gt;Metis&lt;/a&gt; is purpose-built for partitioning graphs and unstructured meshes one uses in finite element/volume methods, and it works great for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a 3D structured multi-block topology, where each $ni\times nj\times nk$ block can only be partitioned along planes, as opposed to an unstructured topology where the partitions can be drawn anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any simple way to partition this kind of topology with metis?&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-07-31T20:15:58.290" Title="metis partitioning for structured multi-block grids" Tags="|finite-element|mesh-generation|partitioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8123" PostTypeId="2" ParentId="8120" CreationDate="2013-07-31T20:10:32.047" Score="3" Body="&lt;p&gt;Can you construct a graph G that describes the adjacency relationship between blocks and have METIS partition that? E.g. each vertex in G represents an ni/nj/nk block, and each edge in G represents a plane between two such blocks? (where METIS is allowed to cut)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do something similar (but for high order FEM), having METIS partition on the graph induced by a volume mesh (mesh tetrahedra =&gt; graph vertices, mesh triangles =&gt; graph edges). In reality each mesh element represents a larger collection of unknowns, analogous to how each of your blocks presumably represents a collection of ni/nj/nk unknowns.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2013-07-31T20:15:58.290" LastActivityDate="2013-07-31T20:15:58.290" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8124" PostTypeId="2" ParentId="668" CreationDate="2013-08-01T01:10:27.350" Score="0" Body="&lt;p&gt;Would it be possible for you to use adaptive mesh refinement?  A quick Google search will turn up lots of links.  AMR is used, for example, in fluid dynamics to model flow past complicated shapes; as well as many other applications.  Here is an example of solving systems of hyperbolic conservation laws that arise in star formation.  The geometries are very complex.  The first part of the paper is a nice tutorial.  &lt;a href=&quot;http://www.mpa-garching.mpg.de/lectures/ADSEM/SS05_Homann.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mpa-garching.mpg.de/lectures/ADSEM/SS05_Homann.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4881" LastEditorUserId="4881" LastEditDate="2013-08-01T02:04:46.660" LastActivityDate="2013-08-01T02:04:46.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8125" PostTypeId="1" CreationDate="2013-08-01T01:40:19.617" Score="11" ViewCount="708" Body="&lt;h1&gt;Question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Are there libraries out there that implement block Krylov subspace methods? (I was not able to find any from a simple Google search.)&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Background&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Right now, I am working with a code that solves several systems of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;Ax_{i} = b_{i},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is $n$ by $n$, $i = 1, \ldots, m$, and in general, $n \gg m$ ($n$ is at least 10 times larger than $m$, and could be several orders of magnitude larger than $m$). In addition, $A$ is large enough that iterative methods are preferable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, the code solves $Ax_{i} = b_{i}$ with a Krylov subspace method for each separate right-hand side with a given preconditioner, and this approach seems to work well enough. (The code also recomputes the preconditioner, which is unnecessary.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To speed up the code, it seems like it could be worth looking at solving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;AX = B&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;instead, where $B = [b_{1}\, b_{2}\, \ldots\, b_{m}]$ is an $n$ by $m$ matrix gathering up all of the $b_{i}$, and $X$ is also an $n$ by $m$ matrix gathering up the $x_{i}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to try out solving $AX = B$ with block Krylov subspace methods (using the same preconditioner as before) to see if it outperforms the current approach of solving $Ax_{i} = b_{i}$ and looping over $i$. It is both possible and unlikely that the solution to $Ax_{i} = b_{i}$ could then be used as a guess for the system $Ax_{j} = b_{j}$ for $i \neq j$; I don't expect there to be a relationship among the various systems.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="20688" LastEditDate="2022-07-04T03:37:44.147" LastActivityDate="2022-07-04T03:37:44.147" Title="Are there any libraries out there that implement block Krylov subspace methods?" Tags="|linear-solver|iterative-method|krylov-method|software-recommendation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8126" PostTypeId="2" ParentId="8115" CreationDate="2013-08-01T04:15:33.803" Score="2" Body="&lt;p&gt;Check &lt;a href=&quot;http://adhesion.ucd.ie/ccm/CCM_II_files/Turek%20and%20Hron%20-%20Proposal%20for%20numerical%20benchmarking%20of%20fluid-structure%20interaction%20between%20an%20elastic%20object%20and%20laminar%20incompressible%20flow.pdf&quot; rel=&quot;nofollow&quot;&gt;Turek-Hron FSI benchmark&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="4364" LastActivityDate="2013-08-01T04:15:33.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8128" PostTypeId="2" ParentId="8125" CreationDate="2013-08-01T07:35:24.643" Score="5" Body="&lt;p&gt;Here's at least a partial answer, after a few hours of searching:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an implementation of block Krylov subspace methods buried within the &lt;a href=&quot;http://trilinos.sandia.gov/packages/belos/&quot; rel=&quot;noreferrer&quot;&gt;Belos&lt;/a&gt; package of Trilinos that contains:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Block CG&lt;/li&gt;&#xA;&lt;li&gt;Block GMRES&lt;/li&gt;&#xA;&lt;li&gt;Block flexible GMRES&lt;/li&gt;&#xA;&lt;li&gt;Block GCRODR (block recycling GMRES)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Supposedly, MATLAB implementations of some of these methods exist, but I haven't yet come across them.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-08-01T07:35:24.643" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8129" PostTypeId="1" CreationDate="2013-08-01T12:29:48.400" Score="3" ViewCount="293" Body="&lt;p&gt;Let's say I have two molecular dynamics (MD) simulators, A and B, which implement the following interface:&#xA;$$ \left\{\vec{x}_i(t = 0), T_i, t \right\} \rightarrow \left\{\vec{x}_i(t), \vec{F}_i(t) \right\} $$&#xA;where $\vec{x}_i$ is the position of atom/ion $i$, $T_i$ is the type of atom/ion $i$ (e.g. hydrogen or carbon), and $\vec{F}_i$ is the force on atom/ion $i$.  Given a set of times, they can produce discrete trajectories:&#xA;$$ \mathcal{T} = \left\{ \left(t, \vec{x}_i(t), \vec{F}_i(t)\right) : t \in (dt) \mathbb{N} \right\} $$&#xA;Unless A and B are identical, there are going to be differences in the trajectories. Given a pair of trajectories, $(\mathcal{T}_A, \mathcal{T}_B)$, how should I decide if A and B are in agreement (for that particular trajectory)?  or what would I have to do to convince you that $B$ was reproducing the behavior of $A$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give some context, B is an altered version of A, using a slightly different Hamiltonian, so they have identical interfaces.  I understand that it may be useful to sample the neighborhood of initial conditions around $\vec{x}(t=0)$, but producing a trajectory is computationally intensive, so using a small number of trajectories is preferable.  The regime is ab initio molecular dynamics using plane-wave DFT.  I'm particularly interested in the motion of adatoms on surfaces.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2013-10-10T22:22:16.880" LastActivityDate="2013-10-10T22:22:16.880" Title="Reproducibility in molecular dynamics trajectories" Tags="|molecular-dynamics|density-functional-theory|reproducibility|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8130" PostTypeId="2" ParentId="8129" CreationDate="2013-08-01T14:35:52.337" Score="2" Body="&lt;p&gt;First thought is to use A and B on standard validation test cases. I'm not sure what's available for validation on MD (googling &quot;molecular dynamics validation&quot; turned up a lot), but in CFD there's plenty of databases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to validate something very specific (it sounds like you are), then you'll need to compile some statistics to convince me that A and B are reproducing the same behavior. You'd do this by sampling an input space of initial conditions given to both methods. There are some sophisticated sampling methods available in DAKOTA that I've used; things like polynomial chaos expansion or the various design-of-experiments/uncertainty quantification capabilities can dramatically reduce the number of runs you have to do to compile meaningful statistics.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-08-01T14:35:52.337" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8131" PostTypeId="2" ParentId="8129" CreationDate="2013-08-01T18:01:44.147" Score="6" Body="&lt;p&gt;Any numerical differences between A and B will become exponentially larger with time (i.e. the Lyapunov instability, as discussed in &lt;a href=&quot;http://molsim.chem.uva.nl/frenkel_smit/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Frenkel and Smit&quot;&gt;Frenkel and Smit&lt;/a&gt;). Even a small difference due to basis set size could result in dramatic differences in the trajectory over time. So I'm not sure that a comparison between individual trajectories will be meaningful. It may be more relevant to compare some equilibrium quantity as &lt;a href=&quot;https://scicomp.stackexchange.com/users/522/deathbreath&quot;&gt;Deathbreath&lt;/a&gt; suggested, such as average energy or diffusion constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you really wanted to check whether the trajectories were the same, one thing you could try is a time-reversal test. Sometimes as a test people will run a simulation for a fixed period of time, and then run it backwards (i.e. reverse the directions of all momenta). If the integrator is time-reversible (not all are), then you should get back to where you started (or at least close, depending on machine precision and the length of the run). Maybe you could try a similar strategy: run a simulation forwards with A. Then run it reverse, once with A and once with B. If method A and method B are identical, then the final coordinates of these two runs should be more or less equally close to the starting coordinates. This could be measured by the mean square displacement of the atoms, for example.&lt;/p&gt;&#xA;" OwnerUserId="3887" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-08-01T18:01:44.147" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8132" PostTypeId="1" AcceptedAnswerId="8136" CreationDate="2013-07-28T18:24:02.407" Score="6" ViewCount="179" Body="&lt;p&gt;I'm looking to learn more about Sparse Optimization and apply it to machine learning problems. Could you please recommend some books/resources on this topic? Both theoretical and applied are fine.&lt;/p&gt;&#xA;" OwnerUserId="4884" OwnerDisplayName="hattoriace" LastActivityDate="2013-08-01T23:25:24.987" Title="Books/Resources on Sparse Optimization?" Tags="|reference-request|optimization|machine-learning|" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8133" PostTypeId="2" ParentId="8046" CreationDate="2013-08-01T20:53:27.120" Score="0" Body="&lt;p&gt;You're right that people use monte carlo techniques to calculate statistical averages (as opposed to time-resolved information) all the time. It's not necessarily true that this is what &lt;em&gt;should be&lt;/em&gt; calculated: it depends on what kind of information you want. Maybe you have a time-dependent external forcing, for example, and want to see how the system evolves in response. &lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2013-08-01T20:53:27.120" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8134" PostTypeId="2" ParentId="8132" CreationDate="2013-08-01T21:25:56.933" Score="0" Body="&lt;p&gt;It sounds like you're interested in iterative method in optimizing solution for a sparse system. If so, you can consult the following book:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4761" LastActivityDate="2013-08-01T21:25:56.933" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8135" PostTypeId="1" CreationDate="2013-08-01T22:29:01.797" Score="3" ViewCount="70" Body="&lt;p&gt;Say I have an expression involving adds, subtractions, and multiplications.  I know that it is safe to assume commutativity, associativity, distributivity, etc., and would like to automatically generate a simpler expression which involves fewer operations.  All interesting variants of this problem are NP complete, but are there existing tools or algorithms which work well in practice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a baseline, currently I am using Mathematica's FullSimplify command and then running common subexpression elimination on the result.  Unfortunately, the result isn't even minimal (it misses obvious distributive law simplifications), and it'd be nicer to have a tool which works on DAGs natively.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="3481" LastEditDate="2013-08-01T23:18:25.363" LastActivityDate="2013-08-03T21:48:14.827" Title="Minimizing expression DAGs" Tags="|graph-theory|symbolic-computation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8136" PostTypeId="2" ParentId="8132" CreationDate="2013-08-01T23:25:24.987" Score="4" Body="&lt;p&gt;First, just to clarify things, you're talking about solving optimization problems of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| x \|_{1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\| Ax - b \|_{2} \leq \delta$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and related forms, right?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many different applications in which problems are formulated as the minimization of the 1-norm of a vector subject to linear or least squares constraints.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important area of theoretical research is proving conditions under which solving the $L_{1}$ minimization problem will recover the sparsest solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many folks working in convex optimization have gravitated to this area because of the new found demand for solvers for these problems.  The methods that they develop can be used in many different applications, and the solvers are effectively black boxes to most users of the codes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It isn't clear whether you're more interested in the application of sparse optimization to machine learning or in theoretical questions or in methods for actually solving the resulting optimization problems.  These are really very disjoint though related areas of research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a starting point, I'd suggest looking at the list of resources for compressive sensing at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dsp.rice.edu/cs&quot; rel=&quot;nofollow&quot;&gt;http://dsp.rice.edu/cs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can supply more details about what it is you want to learn about, and also give me some idea of your background in optimization and other areas of mathematics, then perhaps I could suggest more specific references.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-08-01T23:25:24.987" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8137" PostTypeId="2" ParentId="8115" CreationDate="2013-08-02T05:06:57.543" Score="0" Body="&lt;p&gt;You can try &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045793010002045&quot; rel=&quot;nofollow&quot;&gt;this example&lt;/a&gt;. Its simple but 2D. All the best.&lt;/p&gt;&#xA;" OwnerUserId="3549" LastActivityDate="2013-08-02T05:06:57.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8139" PostTypeId="1" CreationDate="2013-08-02T06:36:38.160" Score="2" ViewCount="135" Body="&lt;p&gt;I have recently started using Lapack++ which I found convenient for my programming purpose, in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I need to solve a matrix using QR algorithm. I've searched the &lt;a href=&quot;http://lapackpp.sourceforge.net/html/index.html&quot; rel=&quot;nofollow&quot;&gt;user manual&lt;/a&gt;&#xA;and I found a &lt;a href=&quot;http://lapackpp.sourceforge.net/html/classLaGenQRFactComplex.html#_details&quot; rel=&quot;nofollow&quot;&gt;LaGenQRFactComplex Class&lt;/a&gt; for the purpose. But it need a complex matrix type as input parameter (as per my understanding).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other way for me to use the same for solving a real valued matrix??&lt;/p&gt;&#xA;" OwnerUserId="1906" LastEditorUserId="3549" LastEditDate="2013-08-03T03:53:54.350" LastActivityDate="2013-08-03T03:53:54.350" Title="Lapack++ for QR algorithm" Tags="|linear-algebra|linear-solver|c++|lapack|numerical-analysis|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8141" PostTypeId="1" AcceptedAnswerId="8150" CreationDate="2013-08-02T08:54:36.193" Score="2" ViewCount="160" Body="&lt;p&gt;Given a time-dependent measure of error, $e(t)$, and a time-dependent function to normalize it with, $f(t)$, one would report the normalized error as $$E(t) = \left|\frac{e(t)}{f(t)}\right|.$$  However, if $f(t)$ passes through zero at certain times, $E(t)$ will be very large near those instances, despite the fact that the actual error could be very small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one hopes to normalize an error measurement with a function that passes through zero, is there a standard convention to normalized error? &lt;/p&gt;&#xA;" OwnerUserId="3742" LastActivityDate="2013-08-04T03:17:37.870" Title="Normalizing error when data passes through zero" Tags="|convergence|error-estimation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8143" PostTypeId="1" CreationDate="2013-08-02T12:33:58.893" Score="7" ViewCount="859" Body="&lt;p&gt;Say you have a system of ODE's where the Jacobian has one zero eigenvalue; what does that tell you about the &lt;a href=&quot;http://en.wikipedia.org/wiki/Stiff_equation&quot;&gt;stiffness&lt;/a&gt; of the system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This case doesn't seem to be discussed in the cases I have been able to find (&lt;a href=&quot;http://en.wikipedia.org/wiki/Stiff_equation#Stiffness_ratio&quot;&gt;Stiffness ratio&lt;/a&gt; or &lt;a href=&quot;http://www.scholarpedia.org/article/Stiff_systems&quot;&gt;stiffness index&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="742" LastActivityDate="2013-08-06T08:57:11.730" Title="ODE: How to measure stiffness if the Jacobian has zero eigenvalues?" Tags="|ode|stiffness|runge-kutta|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8144" PostTypeId="1" CreationDate="2013-08-02T18:26:50.220" Score="3" ViewCount="708" Body="&lt;p&gt;Please can you help me with my problem? On Wikipedia, in article Discrete sine transform, this is written (chapter Computation):&lt;br/&gt;&lt;br/&gt;&#xA;&quot;&lt;em&gt;Although the direct application of these formulas would require O(N2) operations, it is possible to compute the same thing with only O(N log N) complexity by factorizing the computation similar to the fast Fourier transform (FFT). (&lt;strong&gt;One can also compute DSTs via FFTs combined with O(N) pre- and post-processing steps.&lt;/strong&gt;)&lt;/em&gt;&quot;&lt;br&gt;&lt;br/&gt;&#xA;I would like to compute sine discrete fourier transformation and would like to use the FFT algorithm for that. What I have to do with the input sequence so that the data are transformed through discrete sine fourier transform? Adding the zeroes, I think?&lt;/p&gt;&#xA;" OwnerUserId="4007" LastActivityDate="2018-03-15T21:40:52.090" Title="DST using FFT routine" Tags="|fftw|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8145" PostTypeId="2" ParentId="8144" CreationDate="2013-08-02T20:31:02.467" Score="6" Body="&lt;p&gt;FFTW can &lt;a href=&quot;http://www.fftw.org/fftw3_doc/1d-Real_002dodd-DFTs-_0028DSTs_0029.html#1d-Real_002dodd-DFTs-_0028DSTs_0029&quot; rel=&quot;noreferrer&quot;&gt;do this&lt;/a&gt; using certain options: &lt;code&gt;RODFT00&lt;/code&gt;, etc. It's probably better to call a library like FFTW than coding it yourself unless you have very special needs.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-08-02T20:31:02.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8146" PostTypeId="2" ParentId="21" CreationDate="2013-08-02T21:05:24.253" Score="3" Body="&lt;p&gt;NWChem's built-in B3LYP is supposed to agree with Gaussian's, modulo the grid and tolerance issues noted in Thom's answer.  You can prescribe any functional form for which the constituents are supported using the explicit XC interface: &lt;a href=&quot;http://www.nwchem-sw.org/index.php/Density_Functional_Theory_for_Molecules#XC_and_DECOMP_--_Exchange-Correlation_Potentials&quot; rel=&quot;nofollow&quot;&gt;http://www.nwchem-sw.org/index.php/Density_Functional_Theory_for_Molecules#XC_and_DECOMP_--_Exchange-Correlation_Potentials&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recognize that the question has already been answered but I wanted to add the NWChem details for completeness since the question indicates a desire to make GAMESS=NWChem=Gaussian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side note, Dalton supports B3LYP and B3LYP-G.  The latter agrees with Gaussian while the former is the more canonical version that may agree with GAMESS.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-08-02T21:05:24.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8147" PostTypeId="1" CreationDate="2013-08-02T21:13:54.767" Score="4" ViewCount="1680" Body="&lt;p&gt;My main interest is sparse matrix-vector and matrix-transpose-vector multiplication of the form &lt;code&gt;y=y+AA'x&lt;/code&gt;. Is there any library that performs &lt;code&gt;y=y+AA'x&lt;/code&gt; efficiently? I have investigated SPARSKIT, MKL, ARPACK,... but I was not able to find the related routine. Isn't the &lt;code&gt;y=y+AA'x&lt;/code&gt; operation widely used?&lt;/p&gt;&#xA;" OwnerUserId="3978" LastActivityDate="2014-03-03T05:13:02.680" Title="Library that performs sparse matrix-vector and matrix-transpose-vector multiplication" Tags="|sparse-matrix|matrix|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8148" PostTypeId="2" ParentId="8147" CreationDate="2013-08-02T21:55:07.347" Score="2" Body="&lt;p&gt;If a C++ library is acceptable, I suggest trying Eigen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;http://eigen.tuxfamily.org/index.php?title=Main_Page&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The performance is good and it is easy to use. In particular, your expression&#xA;above can be written like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigen::SparseMatrix&amp;lt;double&amp;gt; a(n,n);&#xA;Eigen::VectorXd y(n), x(n);&#xA;y += a*a.transpose()*x;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-08-02T21:55:07.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8149" PostTypeId="2" ParentId="8147" CreationDate="2013-08-02T21:58:15.307" Score="6" Body="&lt;p&gt;Usually you would do one of two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute $z = Ax$, then $y = y+Az$;&lt;/li&gt;&#xA;&lt;li&gt;If you anticipate needing to do this operation many times, you might compute the matrix $B = AA'$ and then evaluate $y = y+Bx$, but it will be much less sparse than the original matrix $A$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I don't know of any libraries that will compute $AA'x$ directly, and if there are any that do so, they probably use (1) behind the scenes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SPARSKIT has routines for both $A\cdot x$ and $A'\cdot x$, they're called &lt;code&gt;amux&lt;/code&gt; and &lt;code&gt;atmux&lt;/code&gt; respectively; these are for matrices in the compressed sparse row format, which is described in the SPARSKIT documentation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have a language preference? If you're just experimenting and don't need your program to be super fast, you could try the &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/sparse.html&quot;&gt;Scipy sparse matrix modules&lt;/a&gt; in python, which would be less of a hassle than using Fortran. These will also have the functionality you're looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Since you want fast and you mentioned GPU computing, you might want to consider the &lt;a href=&quot;https://github.com/cusplibrary/cusplibrary&quot;&gt;CUSP library&lt;/a&gt;. The ellpack matrix format is particularly amenable to usage on GPU machines, although there are exceptional cases.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2013-08-03T17:10:29.333" LastActivityDate="2013-08-03T17:10:29.333" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8150" PostTypeId="2" ParentId="8141" CreationDate="2013-08-03T20:11:46.867" Score="2" Body="&lt;p&gt;Modifying the denominator to avoid its zeros often involves adding constant factor which is equivalent to using an absolute error test near $f(t) = 0$. That is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E(t) = \frac{\left|e(t)\right|}{\left|f(t)\right| + C} &#xA;$$&#xA;Which is equivalent to normalizing $e(t)$ by $C$ near $f(t) = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's no standard convention for this nor the choice of $C$. There are an infinite number of ways to normalize your error, so it's better to just state what you're doing when you're presenting your results. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-08-03T20:11:46.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8151" PostTypeId="2" ParentId="8143" CreationDate="2013-08-03T21:44:07.337" Score="4" Body="&lt;p&gt;The zero eigenvalue is not particularly harmful. Think of the linear system&#xA;$$&#xA;  \dot x = A x,  x(0) = x_0&#xA;$$&#xA;then the components of $x(t)$ decay in proportion to the (magnitude of the) eigenvalues of $A$ (assuming the eigenvalues are all non-negative), whereas the components of $x(t)$ in direction of eigenvectors that belong to zero eigenvalues simply stay the same. In other words, these components simply don't factor at all in the determination whether a system is stiff or not.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="4230" LastEditDate="2013-08-06T08:57:11.730" LastActivityDate="2013-08-06T08:57:11.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8152" PostTypeId="2" ParentId="8141" CreationDate="2013-08-03T21:46:28.910" Score="1" Body="&lt;p&gt;You could also compute the error with respect to a &lt;em&gt;typical size&lt;/em&gt; of $f(t)$. For example, the average of its magnitude, or the maximum of the magnitude. This avoids the problem you encounter.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="27" LastEditDate="2013-08-04T03:17:37.870" LastActivityDate="2013-08-04T03:17:37.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8153" PostTypeId="2" ParentId="8135" CreationDate="2013-08-03T21:48:14.827" Score="1" Body="&lt;p&gt;This is a classical problem compiler builders face in trying to optimize code. For example, Common Subexpression Elimination (CSE) is one way. I would suggest you look into the books that describe compiler building.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-03T21:48:14.827" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8154" PostTypeId="2" ParentId="8143" CreationDate="2013-08-04T03:10:47.673" Score="1" Body="&lt;p&gt;Like conditioning of linear systems, it is hard to boil down the difficulty of time-stepping a differential equation to a single number. In general a big gap in the spectrum is often held as the accepted definition of &quot;stiffness&quot; because it represents a separation of scales, often a nonphysical separation coming from a discretization technique (spectral methods come to mind).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numbers you give are reasonable measures of stiffness in certain circumstances, given this understanding. The situation you present however show that it breaks down in some important cases. Discretizing hyperbolic problems one often obtains for example a nontrivial nullspace, and there furthermore is not an obvious separation of scales if one looks at the spectrum (even if using a very high order method), nevertheless the problem is still &quot;stiff.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some excellent answers to a similar question which may be of use to you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/891/the-definition-of-stiff-ode-system&quot;&gt;The definition of stiff ODE system&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-08-04T03:10:47.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8156" PostTypeId="2" ParentId="8147" CreationDate="2013-08-04T15:46:46.697" Score="8" Body="&lt;p&gt;The matrix product $B = AA^T$ is generally faster to apply as $A (A^T x)$ even when the product matrix $B$ is already available.  Only for peculiar graphs with extremely low expansion factor would the product matrix be faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PDE graphs in two or more dimensions have higher expansion factors.  For example, if $A$ is a 9-point differencing operation on a 2D rectangular grid, then $A^2$ (or $A A^T$) is a 25-point operation; applying the 9-point operation twice is faster.  In 3D, the analog is a 27-point $A$ and 125-point $A^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the worst case: if $A$ contains a single dense column, then $A A^T$ is dense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All sparse linear algebra libraries have support for applying $A$ and $A^T$.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-08-04T15:46:46.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8157" PostTypeId="1" AcceptedAnswerId="8163" CreationDate="2013-08-04T20:38:03.487" Score="-1" ViewCount="4844" Body="&lt;p&gt;I have been working on a transient simulation, using the SST k-w turbulence model in Fluent(TM). An unsteady boundary condition is used: at the beginning of the simulation the flow is laminar and gradually becomes turbulent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where I am getting a bit confused.&#xA;When describing the numerics of my model, should I give the Navier-Stokes equations (conservation,momentum and energy), RANS equations or URANS equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the equations used throughout the simulation or does the solver switch to them from NS only when the flow becomes turbulent? Would RANS work just fine in a laminar flow?&lt;/p&gt;&#xA;" OwnerUserId="4469" LastEditorUserId="4469" LastEditDate="2013-08-06T10:36:38.380" LastActivityDate="2013-08-06T10:36:38.380" Title="Confusion Over Navier-Stokes, RANS and URANS in a Flow With Laminar-Turbulent Transition" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="3" ClosedDate="2013-08-06T08:54:46.887" ContentLicense="CC BY-SA 3.0" />
  <row Id="8158" PostTypeId="1" CreationDate="2013-08-04T22:32:48.477" Score="5" ViewCount="1565" Body="&lt;p&gt;I need to implement a non-linear fitting algorithm in Fortran and chose to use MINPACK's flavor of the Levenberg-Marquardt algorithm as a basis for the least-squares stuff. However, I seem to misunderstand something very basic about how these routines are to be used. My understanding is that I have to call the routine lmdif1 and define a function fcn to be called by the MINPACK routines that calculates the square of the difference between the data to be fitted and the model function; lmdif1 would then return the best solution for the free fitting parameters in an array x.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written a small test program (given below) to try this out. The program produces a set of &quot;data&quot; by calculating values from the model function (exp(ax) in this case) with a given parameter a (=-0.5 in this case) and changing them by a small random &quot;error&quot;. This dataset is then fitted with the same model function, with a being the free parameter. But it doesn't really work, the fits are usually very bad, unless I almost give the &quot;correct&quot; solution as a starting guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering that Levenberg-Marquardt is said to be quite robust and seeing how well it works e.g. in Gnuplot, I wonder what I'm doing wrong here. Any ideas? The code below should work when linking to the MINPACK routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tom&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;! test lmdif&#xA;program testanf&#xA;implicit none&#xA;integer, parameter :: m=10,n=1,lwa=m*n+5*n+m&#xA;integer :: info,iwa(n)&#xA;double precision :: x(n),fvec(m),wa(lwa),tol=1d-3&#xA;external :: fcn&#xA;x(1)=-0.5d0&#xA;call lmdif1(fcn,m,n,x,fvec,tol,info,iwa,wa,lwa)&#xA;call translate_info(info,n,tol)&#xA;write(*,*) 'a=',x(1)&#xA;end program testanf&#xA;&#xA;subroutine fcn(m,n,x,fvec,iflag)&#xA;implicit none&#xA;! m: number of data points&#xA;! n: number of fitting parameters&#xA;integer, intent(in) :: m,n,iflag&#xA;integer :: i&#xA;real, allocatable, save :: t(:),fdata(:)&#xA;real :: tmin,tmax,dt,rand&#xA;double precision, intent(inout) :: x(n)&#xA;double precision, intent(out) :: fvec(m)&#xA;character(len=1) :: c&#xA;logical, save :: init=.true.&#xA;if (init) then&#xA;!  create &quot;measured data&quot;&#xA;   tmin=0.&#xA;   tmax=10.&#xA;   dt=(tmax-tmin)/m&#xA;   call random_seed(put=(/1,2,3,4,5,6,7,8/))&#xA;   allocate(fdata(m),t(m))&#xA;   do i=1,m&#xA;      t(i)=i*dt&#xA;      fdata(i)=fmodel(-0.5d0,t(i))&#xA;   end do&#xA;   tmax=0.1*maxval(fdata)&#xA;   do i=1,m&#xA;      call random_number(rand)&#xA;      fdata(i)=fdata(i)+(rand-0.5)*tmax ! add measurement errors&#xA;      write(10,*) t(i),fdata(i)&#xA;   end do&#xA;   init=.false.&#xA;end if&#xA;! minimize the residuals&#xA;do i=1,m&#xA;   fvec(i)=(fdata(i)-fmodel(x(1),t(i)))**2&#xA;end do&#xA;if (iflag &amp;lt; 0) deallocate(t,fdata)&#xA;return&#xA;contains&#xA;! model function&#xA;  real function fmodel(a,t)&#xA;  implicit none&#xA;  real, intent(in) :: t&#xA;  double precision, intent(in) :: a&#xA;  fmodel=exp(a*t)&#xA;  return&#xA;  end function fmodel&#xA;end subroutine fcn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4909" LastActivityDate="2013-08-04T22:32:48.477" Title="Using MINPACK for curve fitting: implementation?" Tags="|optimization|least-squares|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8159" PostTypeId="2" ParentId="8157" CreationDate="2013-08-05T14:16:27.223" Score="-1" Body="&lt;p&gt;It seems to me that this is a question you can easily figure out on your own - at least partially. A cursory search leads to the following links that should help clarify NS, RANS, and URANS:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.os-cfd.ru/cfd_docs/chalmers/urans.pdf?PHPSESSID=e16a72bf68c8e833e522b3120bd6ee4b&quot; rel=&quot;nofollow&quot;&gt;http://www.os-cfd.ru/cfd_docs/chalmers/urans.pdf?PHPSESSID=e16a72bf68c8e833e522b3120bd6ee4b&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.iaeng.org/publication/WCECS2011/WCECS2011_pp673-678.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.iaeng.org/publication/WCECS2011/WCECS2011_pp673-678.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you will most likely only be able to figure it out partially because you cannot know for sure what's going on inside Fluent...&lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2013-08-05T14:16:27.223" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8160" PostTypeId="1" CreationDate="2013-08-05T15:56:53.803" Score="5" ViewCount="1466" Body="&lt;p&gt;I wrote the following program to solve the three body problem (Sun, Earth, Jupiter), initializing the system such that the total angular momentum is 0. The result of the simulation is really bad (and the Sun tend to infinity). I looked at the code for more than an hour trying to find the problem. Does anybody see what is wrong with my program?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;using std::cout;&#xA;using std::endl;&#xA;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;using std::ofstream;&#xA;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;using std::sqrt;&#xA;&#xA;int main()&#xA;{&#xA;    double time(0), dt(0.05);&#xA;&#xA;    double G(1.9838e-29);&#xA;&#xA;    double m1(6e24), m2(1.9e27), ms(1.99e30);&#xA;&#xA;    double x1(1), y1(0), x2(5.2), y2(0), xs(0), ys(0);&#xA;&#xA;    double T1(1), T2(11.8618);&#xA;&#xA;    double vx1(0),vy1(2*M_PI*x1/T1), vx2(0), vy2(2*M_PI*x2/T2);&#xA;&#xA;    double Rx( (m1*x1+m2*x2+ms*xs)/(m1+m2+ms) ), Ry( (m1*y1+m2*y2+ms*ys)/(m1+m2+ms) );&#xA;&#xA;    x1 = x1 - Rx;&#xA;    x2 = x2 - Rx;&#xA;    xs = xs - Rx;&#xA;    y1 = y1 - Ry;&#xA;    y2 = y2 - Ry;&#xA;    ys = ys - Ry;&#xA;&#xA;    double Lx1(x1*m1*vx1), Lx2(x2*m2*vx2);&#xA;&#xA;    double vxs(0), vys( -(Lx1+Lx2)/(ms*xs) );&#xA;&#xA;    double r12(0), r1s(0), r2s(0);&#xA;&#xA;    ofstream output(&quot;TB_jupiter-earth.dat&quot;);&#xA;&#xA;    output &amp;lt;&amp;lt; time &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; x1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; x2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; xs &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; ys &amp;lt;&amp;lt;endl;&#xA;&#xA;    while(time &amp;lt; 12)&#xA;    {&#xA;        r12 = sqrt( (x1-x2)*(x1-x2) + (y1-y2)*(y1-y2) );&#xA;        r1s = sqrt( (x1-xs)*(x1-xs) + (y1-ys)*(y1-ys) );&#xA;        r2s = sqrt( (x2-xs)*(x2-xs) + (y2-ys)*(y2-ys) );&#xA;&#xA;        vx1 = vx1 - G*ms*dt*(x1-xs)/(r1s*r1s*r1s) - G*m2*(x1-x2)*dt/(r12*r12*r12);&#xA;        vy1 = vy1 - G*ms*dt*(y1-ys)/(r1s*r1s*r1s) - G*m2*(y1-y2)*dt/(r12*r12*r12);&#xA;&#xA;        vx2 = vx2 - G*ms*dt*x2/(r2s*r2s*r2s) - G*m1*(x2-x1)*dt/(r12*r12*r12);&#xA;        vy2 = vy2 - G*ms*dt*y2/(r2s*r2s*r2s) - G*m1*(y2-y1)*dt/(r12*r12*r12);&#xA;&#xA;        vxs = vxs - G*m1*(xs-x1)*dt*(r1s*r1s*r1s) - G*m2*(xs-x2)*dt*(r2s*r2s*r2s);&#xA;        vys = vys - G*m1*(ys-y1)*dt*(r1s*r1s*r1s) - G*m2*(ys-y2)*dt*(r2s*r2s*r2s);&#xA;&#xA;&#xA;        x1 = x1 + vx1*dt;&#xA;        y1 = y1 + vy1*dt;&#xA;&#xA;        x2 = x2 + vx2*dt;&#xA;        y2 = y2 + vy2*dt;&#xA;&#xA;        xs = xs + vxs*dt;&#xA;        ys = ys + vys*dt;&#xA;&#xA;        time += dt;&#xA;&#xA;        output &amp;lt;&amp;lt; time &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; x1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; x2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; xs &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; ys &amp;lt;&amp;lt;endl;&#xA;    }&#xA;&#xA;    output.close();&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correction of the first answer were good but now I changed the calculations of the angula momentum (they were wrong) and the program doesn't work properly anymore. The code is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;using std::cout;&#xA;using std::endl;&#xA;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;using std::ofstream;&#xA;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;using std::sqrt;&#xA;&#xA;int main()&#xA;{&#xA;    double time(0), dt(0.005);&#xA;&#xA;    double G(1.9838e-29);&#xA;&#xA;    double m1(6e24), m2(500*1.9e27), ms(1.99e30);&#xA;&#xA;    double x1(1), y1(0), x2(5.2), y2(0), xs(0), ys(0);&#xA;&#xA;    double T1(1), T2(11.8618);&#xA;&#xA;    double vx1(0),vy1(2*M_PI*x1/T1), vx2(0), vy2(2*M_PI*x2/T2);&#xA;&#xA;    double Rx( (m1*x1+m2*x2+ms*xs)/(m1+m2+ms) ), Ry( (m1*y1+m2*y2+ms*ys)/(m1+m2+ms) );&#xA;&#xA;    x1 = x1 - Rx;&#xA;    x2 = x2 - Rx;&#xA;    xs = xs - Rx;&#xA;    y1 = y1 - Ry;&#xA;    y2 = y2 - Ry;&#xA;    ys = ys - Ry;&#xA;&#xA;    double Ly1(x1*m1*vy1), Ly2(x2*m2*vy2);&#xA;&#xA;    double vxs(0), vys( -(Ly1+Ly2)/(ms*xs) );&#xA;&#xA;    double r12(0), r1s(0), r2s(0);&#xA;&#xA;    ofstream output(&quot;TB_jupiter-earth.dat&quot;);&#xA;&#xA;    output &amp;lt;&amp;lt; time &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; x1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; x2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; xs &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; ys &amp;lt;&amp;lt;endl;&#xA;&#xA;    cout &amp;lt;&amp;lt; x1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y1 &amp;lt;&amp;lt; ' '&amp;lt;&amp;lt; vx1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; vy1 &amp;lt;&amp;lt; endl;&#xA;    cout &amp;lt;&amp;lt; x2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y2 &amp;lt;&amp;lt; ' '&amp;lt;&amp;lt; vx2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; vy2 &amp;lt;&amp;lt; endl;&#xA;    cout &amp;lt;&amp;lt; xs &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; ys &amp;lt;&amp;lt; ' '&amp;lt;&amp;lt; vxs &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; vys &amp;lt;&amp;lt; endl;&#xA;&#xA;    while(time &amp;lt; 4)&#xA;    {&#xA;        r12 = sqrt( (x1-x2)*(x1-x2) + (y1-y2)*(y1-y2) );&#xA;        r1s = sqrt( (x1-xs)*(x1-xs) + (y1-ys)*(y1-ys) );&#xA;        r2s = sqrt( (x2-xs)*(x2-xs) + (y2-ys)*(y2-ys) );&#xA;&#xA;        vx1 = vx1 - G*ms*dt*(x1-xs)/(r1s*r1s*r1s) - G*m2*(x1-x2)*dt/(r12*r12*r12);&#xA;        vy1 = vy1 - G*ms*dt*(y1-ys)/(r1s*r1s*r1s) - G*m2*(y1-y2)*dt/(r12*r12*r12);&#xA;&#xA;        vx2 = vx2 - G*ms*dt*(x2-xs)/(r2s*r2s*r2s) - G*m1*(x2-x1)*dt/(r12*r12*r12);&#xA;        vy2 = vy2 - G*ms*dt*(y2-ys)/(r2s*r2s*r2s) - G*m1*(y2-y1)*dt/(r12*r12*r12);&#xA;&#xA;        vxs = vxs - G*m1*(xs-x1)*dt/(r1s*r1s*r1s) - G*m2*(xs-x2)*dt/(r2s*r2s*r2s);&#xA;        vys = vys - G*m1*(ys-y1)*dt/(r1s*r1s*r1s) - G*m2*(ys-y2)*dt/(r2s*r2s*r2s);&#xA;&#xA;&#xA;        x1 = x1 + vx1*dt;&#xA;        y1 = y1 + vy1*dt;&#xA;&#xA;        x2 = x2 + vx2*dt;&#xA;        y2 = y2 + vy2*dt;&#xA;&#xA;        xs = xs + vxs*dt;&#xA;        ys = ys + vys*dt;&#xA;&#xA;        time += dt;&#xA;&#xA;        output &amp;lt;&amp;lt; time &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; x1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y1 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; x2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; y2 &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; xs &amp;lt;&amp;lt; ' ' &amp;lt;&amp;lt; ys &amp;lt;&amp;lt;endl;&#xA;    }&#xA;&#xA;    output.close();&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4910" LastEditorUserId="4910" LastEditDate="2013-08-05T20:27:55.683" LastActivityDate="2013-08-25T02:37:18.557" Title="Three-body problem" Tags="|c++|" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8161" PostTypeId="2" ParentId="8160" CreationDate="2013-08-05T16:20:17.557" Score="5" Body="&lt;p&gt;Your problems is that variants Euler is the incorrect method for gravitational systems.  You need to look into &lt;a href=&quot;http://en.wikipedia.org/wiki/Symplectic_integrator&quot;&gt;Symplectic Integrators&lt;/a&gt;.  This will solve the stability issue you are having since this type of integrators hold certain values constant (in this case energy).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, as a comment on your code, you should use temporary variables for holding derivates before you update state variables.  As you have it now, updating variables one by one, mixes information from two time steps.  Right now you are updating the velocities explicitly and the positions implicitly.  Unless you have a good reason to do so, it is better to stick to a single method for a problem so you have known stability characteristics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;As the comments stated, you were using a symplectic integrator, so that is not the issue.  You simply have a couple small typos in your code.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, in the second term of your update for v2, you used &lt;code&gt;x2&lt;/code&gt; and &lt;code&gt;y2&lt;/code&gt; rather than &lt;code&gt;(x2-xs)&lt;/code&gt; and &lt;code&gt;(y2-ys)&lt;/code&gt;.  This shouldn't make much difference though since your sun should stay near 0,0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your main errors was that in the update for the suns velocity, you multiplied by the product of radii rather than dividing by it.  The four lines are fixed here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    vx2 = vx2 - G*ms*dt*(x2-xs)/(r2s*r2s*r2s) - G*m1*(x2-x1)*dt/(r12*r12*r12);&#xA;    vy2 = vy2 - G*ms*dt*(y2-ys)/(r2s*r2s*r2s) - G*m1*(y2-y1)*dt/(r12*r12*r12);&#xA;&#xA;    vxs = vxs - G*m1*(xs-x1)*dt/(r1s*r1s*r1s) - G*m2*(xs-x2)*dt/(r2s*r2s*r2s);&#xA;    vys = vys - G*m1*(ys-y1)*dt/(r1s*r1s*r1s) - G*m2*(ys-y2)*dt/(r2s*r2s*r2s);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-08-05T16:48:02.983" LastActivityDate="2013-08-05T16:48:02.983" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="8162" PostTypeId="1" CreationDate="2013-08-05T16:36:45.713" Score="5" ViewCount="3139" Body="&lt;p&gt;I am trying to calculate all of the eigenvectors/eigenvalues of large (40000x40000), sparse, asymmetric matrix.  I am using MATLAB and have 3 GB of working memory.  The way I am calculating them is by sweeping through (as suggested here -&lt;a href=&quot;http://www.mathworks.com/support/solutions/en/data/1-A0QRGR/index.html?product=SL&amp;amp;solution=1-A0QRGR&quot; rel=&quot;noreferrer&quot;&gt;http://www.mathworks.com/support/solutions/en/data/1-A0QRGR/index.html?product=SL&amp;amp;solution=1-A0QRGR&lt;/a&gt;). All of my eigenvalues are negative. I start from near the most negative eigenvalue, use eigs, then select the maximum. I repeat eigs on that maximum plus 1E-5 to avoid using eigs with a shift equal to an eigenvalue and then collect the eigenvalues (and the corresponding eigenvectors) that are greater than that maximum plus 1E-14 to avoid duplicate eigenvalues due to numerical error. I repeat this until the maximum of a new set of eigenvalues reaches close to 0. However, I always seem to fall a few (~5) eigenvalues short of the total. I have checked for the possibility of a maximum being a complex number and making sure that the complex conjugate has been tabulated, but that does not do anything. Moreover, some (~2) of the zero eigenvalues have corresponding eigenvectors that are entirely NaN.  I am currently at a complete loss of how to proceed, and any suggestions regarding useful texts, etc., would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4911" LastActivityDate="2013-09-04T22:21:23.527" Title="How to calculate ALL of the eigenvalues/eigenvectors of a large, sparse, asymmetric matrix?" Tags="|matrix|sparse-matrix|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8163" PostTypeId="2" ParentId="8157" CreationDate="2013-08-05T19:59:05.927" Score="1" Body="&lt;p&gt;You are solving the RANS equations.  It just so happens that people apply the RANS equations so often to steady problems that people inexplicably think of URANS as some different beast. It's not. You wouldn't call them the UNS equations so don't call them the URANS equations.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2013-08-05T19:59:05.927" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8164" PostTypeId="2" ParentId="8162" CreationDate="2013-08-05T22:14:48.007" Score="2" Body="&lt;p&gt;Over the course of 40,000 eigenvalues/eigenvectors, I wouldn't be surprised at all if at least a couple were closer to each other than the 1e-14 tolerance you're allowing, and that's why you're missing a few. If you really need all of these eigenvectors, the easiest possible solution would be to just find a computer with enough memory to do this in matlab. As Wolfgang pointed out, this will probably take ~20gb+ of memory; pushes the limit of tractability on a single computer but you can try and find one available that works. Another alternative is to set up a lot of virtual memory (swap space), keeping in mind that this will likely be &lt;em&gt;very&lt;/em&gt; slow. So either get a bigger computer, use the eigenpairs you do already have, or just use swap space and wait.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you used the phrase &quot;matrix for which the eigenpairs can be calculated exactly.&quot; Keep in mind you are never computing these &quot;exactly&quot;. They're being determined by iterative methods, and are subject to all floating point errors the same as any other method.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-08-05T22:14:48.007" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8166" PostTypeId="1" AcceptedAnswerId="27615" CreationDate="2013-08-06T18:28:24.510" Score="6" ViewCount="1912" Body="&lt;p&gt;Suppose we are given the Stokes equations with Neumann conditions on part of the boundary:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$-\nabla\cdot\boldsymbol{\sigma} = \mathbf{f},  \quad \text{and} \quad  \nabla\cdot \mathbf{u} = 0 \quad \text{in} \quad \Omega$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here $\boldsymbol{\sigma} := \nu(\nabla \mathbf{u} + (\nabla \mathbf{u})^T) - p I$, and $\mathbf{u}=[u_1,\dots,u_d]^T$. Additionally, we assume the boundary conditions of Dirichlet and Neumann-type&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\mathbf{u} = 0 \quad \text{on} \quad \partial\Omega\backslash\Gamma,  \quad \text{and} \quad \boldsymbol{\sigma}\cdot\mathbf{n} = \mathbf{t}\quad \text{on} \quad \Gamma$,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{n}$ denotes the outward pointing normal vector. Assuming that $\nu$ is &lt;em&gt;constant&lt;/em&gt; in the whole of $\Omega$, we can rewrite the momentum equation as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$-\nu\Delta\mathbf{u} + \nabla p = \mathbf{f},  \quad \text{and} \quad  \nabla\cdot \mathbf{u} = 0 \quad \text{in} \quad \Omega$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;by using the incompressibility $\nabla \cdot \mathbf{u} = 0$ and some elementary manipulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What puzzles me now is the following:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the usual procedure, we obtain the following weak form: find $(\mathbf{u},p) \in \mathbf{H}^1_{0,\partial\Omega\backslash\Gamma}(\Omega)\times L^2(\Omega)$ such that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\int_\Omega (\nu\nabla\mathbf{u} : \nabla\mathbf{v} - p\nabla\cdot\mathbf{v} - q\nabla\cdot\mathbf{u}) \,{\rm d}x - \int_\Gamma (\nu\nabla\mathbf{u}-pI)\cdot\mathbf{n}\cdot\mathbf{v}\,{\rm d}s = \int_\Omega \mathbf{f}\cdot\mathbf{v}\,{\rm d}x$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;for all test functions $(\mathbf{v},q) \in \mathbf{H}^1_{0,\partial\Omega\backslash\Gamma}(\Omega)\times L^2(\Omega)$. At this point, one would normally replace the remaining boundary terms by the Neumann boundary conditions and put them to the right hand side. However, when considering the physical traction boundary conditions given above, we clearly have that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\mathbf{t} = \boldsymbol{\sigma}\cdot\mathbf{n} = (\nu(\nabla \mathbf{u} + (\nabla \mathbf{u})^T)\cdot\mathbf{n} - p\mathbf{n} \ne \nu\nabla \mathbf{u}\cdot\mathbf{n} - p\mathbf{n}$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;unless for some reason $(\nabla \mathbf{u})^T\cdot\mathbf{n} = 0$ of course. This now leads to the weak form: find $(\mathbf{u},p) \in \mathbf{H}^1_{0,\partial\Omega\backslash\Gamma}(\Omega)\times L^2(\Omega)$ such that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\int_\Omega (\nu\nabla\mathbf{u} : \nabla\mathbf{v} - p\nabla\cdot\mathbf{v} - q\nabla\cdot\mathbf{u}) \,{\rm d}x + \int_\Gamma \nu(\nabla\mathbf{u})^T\cdot\mathbf{n}\cdot\mathbf{v}\,{\rm d}s = \int_\Omega \mathbf{f}\cdot\mathbf{v}\,{\rm d}x - \int_\Gamma \mathbf{t}\cdot\mathbf{v}\,{\rm d}s$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;for all test functions $(\mathbf{v},q) \in \mathbf{H}^1_{0,\partial\Omega\backslash\Gamma}(\Omega)\times L^2(\Omega)$. However, what I am really interested in is the problem in finite dimensional spaces $\mathbf{V}_h \times Q_h \subset \mathbf{H}^1_{0,\partial\Omega\backslash\Gamma}(\Omega)\times L^2(\Omega)$ for the velocity and pressure, respectively. Let us assume that these are LBB stable. Following the standard procedure for showing stability, one would still have to show that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$a(\mathbf{u},\mathbf{v}) := \int_\Omega \nu\nabla\mathbf{u} : \nabla\mathbf{v} \,{\rm d}x + \int_\Gamma \nu (\nabla\mathbf{u})^T\cdot\mathbf{n}\cdot\mathbf{v}\,{\rm d}s$. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is uniformly coercive on the kernel space $\mathbf{\tilde V}_h := \{ \mathbf{v}_h \in \mathbf{V}_h : b(\mathbf{v}_h,q_h) = 0, \forall q\in Q_h \}.$ This is something, which I could not find in the literature, and did not manage to show myself. Perhaps some stabilization terms or sophisticated analysis techniques might be needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one wants to discretize the above problem by the finite element method, one can of course always discretize the alternative form&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;${\tilde a}(\mathbf{u},\mathbf{v}) := \int_\Omega \tfrac{\nu}{2}(\nabla\mathbf{u} + (\nabla\mathbf{u})^T) : (\nabla\mathbf{v} + (\nabla\mathbf{v})^T)\,{\rm d}x$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where the traction boundaries are no problem to incorporate during the derivation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for constant viscosities it might sometimes be advantageous to not implement the above form with the cross-derivatives involved. Reasons may be that the effort to implement this in some legacy code might be too large or efficiency concerns due to the additional couplings, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;tl;dr:&lt;/strong&gt; Most people who use the Laplacian-Stokes only consider &lt;em&gt;pseudo-traction&lt;/em&gt; boundary conditions, i.e., they impose $\nu\nabla \mathbf{u}\cdot\mathbf{n} - p\mathbf{n} = \mathbf{t}$, where it assumed that $(\nabla \mathbf{u})^T\cdot\mathbf{n} \approx 0$. I kind of doubt that there is always a physical justification for this, and rather expect that this is mostly done for convenience. However, I would be surprised if nobody ever investigated the use of the Laplacian form with full traction conditions. I was not very successful yet with my literature study and hope that somebody might have additional resources or some more information for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have found so far is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://www.cimec.org.ar/ojs/index.php/mc/article/viewFile/486/464&quot; rel=&quot;noreferrer&quot;&gt;Laplace form of Navier-Stokes equations: A safe path or wrong&#xA;  way?&lt;/a&gt; by A. Limache and S. Idelsohn (2006)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This paper raises some similar questions.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2013-08-06T19:08:26.877" LastActivityDate="2017-08-12T10:14:31.893" Title="Finite elements for Stokes with traction boundary conditions" Tags="|finite-element|fluid-dynamics|stability|navier-stokes|incompressible|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8167" PostTypeId="1" CreationDate="2013-08-06T19:26:38.830" Score="5" ViewCount="1110" Body="&lt;p&gt;I am brand new to slepc4py and petsc4py (and SLEPc and PETSc in general), and I have a few basic questions. I apologize if these have been answered somewhere - I've looked around for a while and haven't had much luck. If you know the answer to one (or more) of these questions, I would very much appreciate your insight. Alternatively, if you could point me to some resource that I should consult, I would appreciate that as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to implement spectral clustering for a relatively large data set using petsc4py and slepc4py. The steps involved are the following (paraphrasing Ulrike von Luxburg's Tutorial on Spectral Clustering):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Supposing that we have a similarity graph represented by a weighted adjacency matrix W, compute the unnormalized graph Laplacian, L = D - W, where D is a diagonal matrix containing the degrees of each node.&lt;/li&gt;&#xA;&lt;li&gt;Compute the first k eigenvectors of L. Let U be the matrix containing these vectors as columns.&lt;/li&gt;&#xA;&lt;li&gt;Each row of U now represents a point in k-dimensional space. Cluster these points using the k-means algorithm.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My questions are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. What is the best way to assemble a PETSc sparse matrix?&lt;/strong&gt; I have a file that contains the nonzero entries of my matrix stored as 3-tuples (i, j, val). I tried modeling the matrix assembly after the example in ex1.py that comes with slepc4py, by doing the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = PETSc.Mat()&#xA;A.create()&#xA;A.setSizes([max_i, max_j])&#xA;A.setUp()&#xA;for line in file:&#xA;    # split line, etc&#xA;    A[i,j] = val&#xA;A.assemble()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This took an unreasonably long time to do. I had more success creating a scipy sparse array from the file and then using PETSc.Mat().createAIJ(scipy_mat) to create a PETSc matrix from the scipy matrix. Am I approaching this the right way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. What is the best way to compute the graph Laplacian?&lt;/strong&gt; On a related note, once I have the weighted adjacency matrix loaded in, it is not clear how I should assemble the graph Laplacian. In Matlab or scipy I would iterate over the rows, summing the weights for the edges in a row to get the diagonal element, etc. Is that kind of thinking appropriate with PETSc matrices, or should I be going about this in a different way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. How can I create the U matrix from the eigenvectors?&lt;/strong&gt; Following the examples that come with slepc4py, I was able to compute the first k eigenvalues/eigenvectors for my sample matrix. The example shows how to print out the eigenvalues. It is less clear how to use the eigenvectors after the eigenproblem has been solved. In Matlab or scipy creating a matrix with the eigenvectors as columns would be straightforward. Is it possible to stick a bunch of PETSc Vector objects together to make a PETSc Matrix? I have not found anything to suggest that it is, but I also haven't been able to find any other good way of doing this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any help you might be able to provide!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers,&#xA;-Rob&lt;/p&gt;&#xA;" OwnerUserId="4919" LastActivityDate="2013-08-06T20:52:29.727" Title="Basic questions regarding slepc4py/petsc4py" Tags="|python|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8168" PostTypeId="2" ParentId="8167" CreationDate="2013-08-06T20:52:29.727" Score="1" Body="&lt;p&gt;When you're setting up a matrix in PETSc, it's best to preallocate where the non-zero entries of the matrix are going to go before you start filling in the values. It looks like you already know the number of non-zero entries, so you can use &lt;code&gt;createAIJ&lt;/code&gt; and &lt;code&gt;assemble&lt;/code&gt; first to build the matrix structure, then fill the entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Internally, PETSc uses a sparse matrix storage scheme, which has the advantage that matrix-vector multiplication is extremely fast, but changing the structure of the matrix once you've already defined it is quite slow. So each time you add a new entry, it has to deallocate the matrix structure, reallocate it to accommodate the new value, then insert it.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-08-06T20:52:29.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8169" PostTypeId="2" ParentId="8166" CreationDate="2013-08-06T21:59:07.837" Score="6" Body="&lt;p&gt;You're learning that the two forms (with $-\Delta u$ and with $-2\nabla \cdot \varepsilon(u)$) are not equivalent. They lead to different boundary terms in the bilinear form after integration by parts. In the first case, you get a term involving $n\cdot \nabla u$, in the latter $2n \cdot \varepsilon(u)$, and the &quot;natural&quot; boundary conditions you can enforce are, respectively, the one where either $\nu n\cdot \nabla u-pn=g$ or $2\nu n \cdot \varepsilon(u)-pn=g$. The respectively other one is, simply, not &quot;natural&quot; for each formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a physical viewpoint, my belief is that the &quot;correct&quot; formulation should be the one with the $-2\nabla \cdot \varepsilon(u)$ term, as it is the one that comes out of modeling flow and also because that's the boundary conditions that make sense. The other formulation may be equivalent when you only have Dirichlet boundary conditions, but it is &lt;em&gt;not&lt;/em&gt; equivalent whenever you have natural boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative point of view is to ask what &quot;natural&quot; boundary conditions are anyway. What does it really mean to say that at an outflow segment of the boundary there is no traction on the fluid. To assume that there are &lt;em&gt;no&lt;/em&gt; forces on the fluid at a particular point is equivalent to assuming certain things about the flow field &lt;em&gt;past the boundary&lt;/em&gt; -- for example, with any boundary condition we may prescribe at such a boundary, we make an assumption that the flow will proceed mostly &lt;em&gt;unobstructed&lt;/em&gt; for a long distance, but what exactly is it that we're trying to say with that? When we use no-traction boundary conditions $n \cdot \sigma = 0$ at these places, then this is mostly for convenience because we can not put into formulas what exactly it is that we want to say about what we think is happening outside the domain. But if this is so, whether we say that $n \cdot (\nu\nabla u - pI)=0$ or that $n \cdot (2\nu [\nabla u + \nabla u^T] - pI)=0$, either is about equally arbitrary and simply expresses slightly different notions about the kinds of forces we believe to be zero at this boundary. Given that either is arbitrary, we may as well choose whichever is more convenient in a given situation, given which form we've chosen for the (Navier-)Stokes equations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-06T21:59:07.837" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8170" PostTypeId="1" CreationDate="2013-08-06T22:30:22.340" Score="5" ViewCount="141" Body="&lt;p&gt;I have produced a CSV file in the following format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;position number | list of positions it links to | 2nd list of positions it links to | ... | 6th list of positions it links to &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All the items are integers and the lists are of varying size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The file forms a kind of tree or web (it's a game tree).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to visualise the data as a web or tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can sort the tree according to distance from opening move (another integer), but maybe I won't need to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like it to look something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1&#xA;| \&#xA;2  4&#xA;|\ |\&#xA;3,7,5,9&#xA;......&#xA;....&#xA;|/&#xA;2000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any ideas on what software I should use and a few hints? &lt;/p&gt;&#xA;" OwnerUserId="4920" LastEditorDisplayName="user10435" LastEditDate="2015-03-08T05:00:41.797" LastActivityDate="2015-03-08T05:00:41.797" Title="Which visualisation software for a layered graph?" Tags="|visualization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8171" PostTypeId="2" ParentId="8170" CreationDate="2013-08-06T23:07:33.747" Score="4" Body="&lt;p&gt;You would need to change the output format, but this sort of data (a layered graph) works well with &lt;a href=&quot;http://www.graphviz.org/&quot; rel=&quot;nofollow&quot;&gt;graphviz&lt;/a&gt;, specifically &lt;a href=&quot;http://www.graphviz.org/content/dot-language&quot; rel=&quot;nofollow&quot;&gt;dot&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2013-08-06T23:07:33.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8173" PostTypeId="1" AcceptedAnswerId="8210" CreationDate="2013-08-07T09:11:48.197" Score="6" ViewCount="225" Body="&lt;p&gt;I have a program which compares the similarity of two images for different positions, so my surface consists of points which correspond to X and Y translations each with a value (mutual information).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of a surface I want to be ranked as very good:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OIZA2.png&quot; alt=&quot;https://dl.dropboxusercontent.com/u/51282958/good.png&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is an example of surface which needs a low rank:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/GuOi0.png&quot; alt=&quot;https://dl.dropboxusercontent.com/u/51282958/bad.png&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the metric is some kind of measure of certainty, lots of similar peaks = bad and one big peak in center of the surface is good. Also a peak on the border of the surface is bad as it might not be a peak at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I've already created some kind of metric to rank these surfaces, and it does work in some cases. What I would really like to know is if there is some mathematical or computational method well defined to do this kind of ranking?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have extracted information such as the location of maxima and standard deviations. Also this surface is sometimes up to 6 dimensional when rotation and scale are included.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: My current metric works as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Finds the coordinates of each local maximum&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Starts with a score of 1 and then reduces this for each maximum it finds by a percentage, this percentage is calculated by looking at how similar the local maximum is to the global maximum. Hence if you have a lot of maxima similar to the global maxima you will get a low score. The issue with this technique at the moment is that it was designed when I had images which only generated typically under 10 maxima but I now have images with over 500 maxima and so my score goes very low when it should not.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally there is a correction depending on how many standard deviations away form the mean the global maximum is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm planning on improving how step two scales and so I'll hopefully be able to get rid of the last step (as it feels fairly subjective). &lt;/p&gt;&#xA;" OwnerUserId="4923" LastEditorUserId="4260" LastEditDate="2013-08-14T12:35:22.710" LastActivityDate="2013-08-14T12:35:22.710" Title="What computational methods would allow me to rank 2D surfaces (with examples)" Tags="|statistics|image-processing|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8174" PostTypeId="2" ParentId="8020" CreationDate="2013-08-07T09:30:05.340" Score="1" Body="&lt;p&gt;I'm not familiar with Indigo, but from reading the documentation, it looks like it should be possible to do the manual calculation you mention, using something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for atom in mol.iterateAtoms():&#xA;    bond_count = 0&#xA;    for nei in atom.iterateNeighbors():&#xA;       bond_count += nei.bond().bondOrder()&#xA;    print([atom.symbol(),atom.atomicNumber() - bond_count])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's just missing the calculation of number of shells; I'm not sure how to do that part.&lt;/p&gt;&#xA;" OwnerUserId="4727" LastActivityDate="2013-08-07T09:30:05.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8175" PostTypeId="1" CreationDate="2013-08-07T10:15:30.617" Score="2" ViewCount="1576" Body="&lt;p&gt;I have a large VTK file with rectangular data, it seems Paraview cannot handle the size of the file, so it crashes and Visit navigation is extremely slow? Can anyone point me to good open source visualization software?&lt;/p&gt;&#xA;" OwnerUserId="4907" LastActivityDate="2013-08-13T02:13:43.680" Title="Best VTK visualizing software?" Tags="|visualization|open-source|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="8176" PostTypeId="1" CreationDate="2013-08-07T11:22:36.887" Score="2" ViewCount="1482" Body="&lt;p&gt;I was reading an article where they have a set of pages. For every page they extract a feature vector of dimention 300. Then they fuse these vectors into one vector by using a method called average pooling: Quoting : &quot; to encode sets of pages into a compact document representation. The main idea consists of performing average pooling of page representations to obtain a single representation of an entire document&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P is the set of pages P = {p1,p2,p3,...,pn} and pi is the vector&lt;/p&gt;&#xA;&#xA;&lt;p&gt;F(p) = 1/|P| * Somme(pi) Such that p E P&lt;/p&gt;&#xA;" OwnerUserId="4918" LastActivityDate="2013-08-07T11:22:36.887" Title="What is average pooling" Tags="|machine-learning|statistics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8177" PostTypeId="2" ParentId="7566" CreationDate="2013-08-07T15:47:40.480" Score="3" Body="&lt;p&gt;Take a look at &lt;a href=&quot;http://scikit-learn.org/dev/modules/generated/sklearn.decomposition.TruncatedSVD.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sklearn.decomposition.TruncatedSVD&lt;/a&gt; in&#xA;&lt;a href=&quot;http://scikit-learn.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit-learn&lt;/a&gt; 0.14-rc.&lt;br&gt;&#xA;(I believe that the scikit-learn people follow&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/tagged/scikit-learn&quot;&gt;stackoverflow.com/questions/tagged/scikit-learn&lt;/a&gt;, so I'd ask detailed questions there.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(How much memory do you have ? 10$^{6+3}$ doubles is already 8G.)&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-08-07T15:47:40.480" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8178" PostTypeId="1" CreationDate="2013-08-07T19:20:13.293" Score="2" ViewCount="398" Body="&lt;p&gt;I have written a molecular dynamics simulator with a single potential function (lennard jones). I initialise the temperature of the  system using a Boltzmann velocity distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a standard way of verifying that it works? E.g. some pressure/temperature calculation for a known element?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="1017" LastEditorUserId="1017" LastEditDate="2013-08-07T22:46:41.467" LastActivityDate="2013-10-09T21:09:25.350" Title="How can I easily verify my molecular dynamics simulation?" Tags="|molecular-dynamics|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8179" PostTypeId="1" CreationDate="2013-08-07T21:04:41.300" Score="2" ViewCount="113" Body="&lt;p&gt;I am learning PETSc following the tutorials of the distribution (version 3.4.1). My question is about the KSPView(ksp,PETSC_VIEWER_STDOUT_WORLD); (line 216) of ex7.c (KSP tutorial). &#xA;It teaches how to use PCBJACOBI, setting different solvers for each PC block. When I use &lt;/p&gt;&#xA;&#xA;&lt;p&gt;mpirun -n 4 ./ex7 -m 20 -ksp_type cg&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the output is kind of a mess, like if there wasn't order on the output (and difficult to understand). I just figured out this when I did the serial test&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mpirun -n 1 ./ex7 -m 20 -ksp_type cg&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone can tell me what is going on, since when using PETSC_VIEWER_STDOUT_WORLD the process [0] should print the outputs of the others processes in some (random) order, but without mixing the outputs?&lt;/p&gt;&#xA;" OwnerUserId="4927" LastActivityDate="2013-08-07T21:04:41.300" Title="PETSc KSP Tutorial ex7 Output" Tags="|petsc|" AnswerCount="0" CommentCount="1" ClosedDate="2013-08-11T21:15:07.480" ContentLicense="CC BY-SA 3.0" />
  <row Id="8180" PostTypeId="1" CreationDate="2013-08-07T21:25:21.343" Score="7" ViewCount="824" Body="&lt;p&gt;I would like to construct a Finite Element basis by using a generalized Vandermonde matrix. The idea is to compute the values of a suitable modal basis ('prime basis') at a set of points in reference space and express the values of Finite Element basis functions as a linear combination of the members of the prime basis as explained in the references [1] or [2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following problem. The primal basis of choice on triangles/tetrahedra is Dubiner basis and uses Jacobi polynomials. When I evaluate this basis, some entries in the Vandermonde matrix are nearly zero ( ~ 10^-17), but it appears that they should be in fact exactly zero. Later on, when I use the Vandermonde matrix to compute the values of Lagrange shape functions, some of the resulting values are again slightly perturbed. The consequence of this is that the sum of all Lagrange shape functions at certain points is not exactly 1, and the sum of derivatives is not exactly 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if anyone has ever encountered a problem like this and if there's a remedy. I searched in books and on google, but with no success. As I would like to use higher-order shape functions in a CFD code, I'm afraid that the errors in the shape function evaluation will introduce instabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give a bit of implementation details: I implemented the computation of Dubiner modes in C++ in two different ways: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) 'direct' implementation using a recursive formula for Jacobi polynomials as explained in [1] and [2]. This is known to fail at the vertex (-1,1) of the reference triangle due to a singularity in the collapsed coordinate transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) a 'workaround' implementation described in [3]. In accordance with the observations made in this paper, the errors obtained by b) are slightly smaller than in case a).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Jan S. Hesthaven, Tim Warburton: Nodal Discontinuous Galerkin Methods: Algorithms, Analysis, and Applications&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] George Em Karniadakis, Spencer J. Sherwin: Spectral/hp Element Methods for Computational Fluid Dynamics&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Robert C. Kirby: Singularity-free evaluation of collapsed-coordinate&#xA;orthogonal polynomials&lt;/p&gt;&#xA;" OwnerUserId="4688" LastEditorUserId="4244" LastEditDate="2013-08-11T04:34:28.957" LastActivityDate="2014-03-06T14:57:10.817" Title="Evaluation of Vandermonde matrix" Tags="|finite-element|numerical-analysis|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8181" PostTypeId="2" ParentId="8180" CreationDate="2013-08-07T22:20:18.540" Score="2" Body="&lt;p&gt;It may help to tell us more about your choice of interpolation points. [1] has a 1D discussion on conditioning issues for poor choices of points; the close-to-zero terms aren't usually an issue unless the matrix itself is poorly conditioned or close to singular.  Warburton has a &lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs10665-006-9086-6&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; on optimal choices for interpolation points in 2D and 3D, which I think the Trilinos library bases its higher order Lagrange shape functions off of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this isn't the issue, you could always check by switching to a modal representation for a simple test case. Your &quot;slightly off&quot; nodal polynomials can themselves be considered basis functions, and most Galerkin schemes are agnostic to choices of basis function. If you tried both modal and nodal approaches you could see if being inexactly nodal causes instabilities.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-08-07T22:20:18.540" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8182" PostTypeId="2" ParentId="8180" CreationDate="2013-08-07T22:57:32.617" Score="2" Body="&lt;p&gt;This happens all the time due to numerical round-off and can't be avoided. In deal.II, we typically just set small elements (relative to some tolerance, say $10^{-14}$) to zero and rescale the matrix rows or columns if we know that their sum should be one.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-07T22:57:32.617" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8185" PostTypeId="1" CreationDate="2013-08-08T10:51:48.190" Score="1" ViewCount="2774" Body="&lt;p&gt;I have written some code, to produce some data. In what I have shown below, the output is just one-dimensional array of numbers. I have an analytical expression for what the autocorrelation function of this data should look like. My problem is, that it doesn't look like that, when I use matlabs built-in function &lt;code&gt;xcorr&lt;/code&gt;. I don't know why that is, but I was hoping some of you might be able to tell me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have attached two graphs, to show you what I mean. The graphs herein are normalized, by the way. The red graph is the expected autocorrelation function, and the blue is the measured one. (Sorry for the horrible images - I don't know why matlab does that).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EnNsO.jpg&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/6vRvK.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, there seem to be some sort of oscillations, or something like that. Is that to be expected?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is actually somewhat of a toy-example. The code I really (really) need to get working, produces a 4D array - actually a discretized 3D vector field. The measured autocorrelation of this output is far uglier than this. I will upload some graphs in a while.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&#xA;Here is a graph showing the measured autocorrelation of the produced 3D vector field. Once again, sorry for the poor quality. The blue graph shows the output of &lt;code&gt;xcorr(V,'coef')&lt;/code&gt;, the purple graph shows the output of &lt;code&gt;xcorr(V,'biased')&lt;/code&gt; and the red is how it is actually supposed to look. If I move the purple graph, so that its maximum is 1, the middle part of the graph is pretty close to the red graph - but is this just a coincidence?&#xA;&lt;img src=&quot;https://i.stack.imgur.com/lnQ0U.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3618" LastEditorUserId="3618" LastEditDate="2013-08-08T12:34:46.473" LastActivityDate="2013-08-09T01:36:36.140" Title="Autocorrelation problems" Tags="|matlab|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8186" PostTypeId="1" CreationDate="2013-08-08T17:14:27.257" Score="3" ViewCount="70" Body="&lt;p&gt;I'm trying to place multiple light sources on a 2D plane, in a fashion that satisfies multiple constraints. The 2D scalar distributions are the irradiance distributions of each light source that are functions of a 3D poisition vector and 2 orientation angles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want the lights to be set up to satisfy certain constraints as well as possible (maximal homogeneity in a certain area, minimal glare in certain area).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not an orthodox packing problem as far as I can tell, since the lit areas have a form that can change substantially by varying the positions and directions of each light source. Additionally, superposition of the irradiance distributions may be needed for an optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only solution I've come up with is a naive minimization of a cost function (using gradient-descent methods, simulated annealing) composed of my constraints. But there may be a more efficient approach to solving such a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does SCICOMP know a more efficient strategy to tackle this problem? Or an applied mathematics/computer science field dealing with similar problems, so that I may do further research?&lt;/p&gt;&#xA;" OwnerUserId="4936" LastActivityDate="2013-08-08T22:36:24.317" Title="Suitable algorithm for superposing variable scalar distributions in 2D area with constraints" Tags="|algorithms|reference-request|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8187" PostTypeId="1" AcceptedAnswerId="8189" CreationDate="2013-08-08T17:23:05.000" Score="8" ViewCount="4065" Body="&lt;p&gt;&lt;br&gt;&#xA;I want to solve the (convex) optimisation task:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$max_{r,z}\quad r$&lt;br&gt;&#xA;subject to the following two constraints&lt;br&gt;&#xA;$r\|x_i\| - x_i^Tz \leq 0 \qquad \forall i=1,\dots, N $&lt;br&gt;&#xA;$\|z\| \leq 1$&lt;br&gt;&#xA;$r\geq0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r$ is a scalar, $z$ is a vector, the $x_i$'s are vectors of the same dimension and $\|\cdot\|$ is the simple eucl. norm. One can assume that the feasible region is non-empty. &#xA;&lt;br&gt;&lt;br&gt;&#xA;Is there an easy way to solve this? I think this should be easy because without the $\|z\|\leq 1$ constraint this is just a linear program. Before refering my to software packages can you give an hint about the general approach which is usefull for this kind of task? &lt;br&gt;&#xA;Thanks DG&lt;/p&gt;&#xA;" OwnerUserId="4937" LastActivityDate="2013-08-09T18:39:07.050" Title="How to deal with norm inequality constraints" Tags="|optimization|convex-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8188" PostTypeId="1" CreationDate="2013-08-08T19:20:34.587" Score="1" ViewCount="61" Body="&lt;p&gt;&lt;a href=&quot;https://mathematica.stackexchange.com/questions/27458/malliavin-derivative-with-mathematica-is-it-possible&quot;&gt;From mathematica.stackexchange&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to define a  &lt;a href=&quot;http://en.wikipedia.org/wiki/Malliavin_calculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Malliavin calculus&lt;/a&gt; with &lt;em&gt;Mathematica&lt;/em&gt; 9?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a random variables on the Wiener-space $\Omega=\mathcal{C}([0,1])$ of the form $$F=F(\omega)=\displaystyle\int_{0}^{T}h_{t}dW_{t}$$ for some deterministic $h(.)$ in $L^{2}[0,T].$&#xA;Here, $\omega$ is a path in the Wiener-space, $\omega \in \Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We define the Malliavin derivative by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;DF=h,\text{ and }D_{t}F=h_{t}.&#xA;$$&#xA;Let say &lt;code&gt;MalliavinDer&lt;/code&gt; is this derivative. We have the following rules&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;MalliavinDer[F_ + G_] := MalliavinDer[F]+MalliavinDer[G]&lt;/code&gt;;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;MalliavinDer[F_G_]:=MalliavinDer[F]G+ MalliavinDer[G]F&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;MalliavinDer[f(F)]:=f'[F]MalliavinDer[F].&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;On the other hand, we have for example:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;MalliavinDer[\int_0^T h(t)dWt] := h(t)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;MalliavinDer[W_s] := Piecewise[{{1, s &amp;lt;t}, {0, s &amp;gt;t}}]&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;MalliavinDer[f(W_s] := f'[W_s])Piecewise[{{1, s &amp;lt;t}, {0, s &amp;gt;t}}]&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4938" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2015-03-19T19:15:50.297" Title="Malliavin Derivative with Mathematica is it possible?" Tags="|mathematica|" AnswerCount="0" CommentCount="1" ClosedDate="2013-08-08T21:07:53.260" ContentLicense="CC BY-SA 3.0" />
  <row Id="8189" PostTypeId="2" ParentId="8187" CreationDate="2013-08-08T21:04:20.700" Score="3" Body="&lt;p&gt;You have a couple options, depending on how crucial it is that you take the Euclidean norm of $z$.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use your formulation as is, with a small tweak:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$max_{r,z}\quad r$&lt;br&gt;&#xA;subject to the following two constraints&lt;br&gt;&#xA;$r\|x_i\| - x_i^Tz \leq 0 \qquad \forall i=1,\dots, N $&lt;br&gt;&#xA;$z^{T}z \leq 1$&lt;br&gt;&#xA;$r\geq0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is a quadratically-constrained program, for which there are many fast solvers out there, such as CPLEX and Gurobi. This particular program is also a second-order cone program, semidefinite program, and convex nonlinear program, so you could also use any of those solvers as well. The reason I replaced the Euclidean norm constraint with a dot product is that the two constraints are equivalent, but the latter is differentiable, whereas the former is not. Nondifferentiable functions require more expensive algorithms, and this problem doesn't require that type of machinery, so it's best to avoid it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Change the norm of $z$. The 1-norm and infinity-norm are both linear functions of the elements of $z$, and replacing the Euclidean norm in your formulation with either of those norms results in a linear program, for which the best solvers tend to be commercial (Gurobi, CPLEX), but slower free solvers exist (GLPK, solvers in the COIN-OR suite). Using the 1-norm means that any solution to that formulation would also be a feasible solution of your current formulation (that is, using the 1-norm would yield a restriction of your current formulation). Using the infinity-norm means that any solution to your current formulation would also be feasible in the infinity-norm formulation (that is, using the infinity-norm would yield a relaxation of your current formulation).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Although it is true that linear programming solvers are very efficient, I would select option 1, because quadratically-constrained programming solvers are also very efficient (relative to convex programming solvers, and other types of nonlinear programming solvers) and can solve large formulations (at least hundreds of thousands of decision variables, last time I looked at the literature). Unless your formulation were astonishingly large, you should be fine using a quadratically-constrained programming solver in serial, and needn't change the norm in your formulation unless you absolutely have to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A final remark: I would scale the vectors $x_{i}$ prior to constructing your formulation so that they all have unit norm, which will likely help with the conditioning of your formulation when you solve it numerically. It's another trick that costs you practically nothing, but guards you against numerical difficulties.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-08-09T16:46:08.533" LastActivityDate="2013-08-09T16:46:08.533" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8190" PostTypeId="2" ParentId="8186" CreationDate="2013-08-08T22:36:24.317" Score="1" Body="&lt;p&gt;Steepest descent or other &quot;simple&quot; methods are not going to converge to the optimal solution since the problem is not convex. This is easy to see: if you swap two lights, you get the same irradiation. In other words, if you've found one optimum, you can find many others just by swapping lights. What this suggests is that you need an algorithm that allows for &lt;em&gt;global optimization&lt;/em&gt; if you really care for finding the &lt;em&gt;best&lt;/em&gt; arrangement, not just a local optimum. Of course, global optimizers are also much more expensive than local ones such as the steepest descent method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-08T22:36:24.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8191" PostTypeId="2" ParentId="8185" CreationDate="2013-08-09T01:36:36.140" Score="1" Body="&lt;p&gt;Your question is not very clear - there may always be differences in the computed autocorrelation and the theoretical (expected) autocorrelation due to noise in the data. Are you sure you are not having any? The more noise your data has, the lesser the chances of you getting a smooth correlation plot. Usually the wiggles in the plot indicate presence of pockets of randomness - which is what noise is. compute the autocorrelation using 2 different software packages - if you see a difference, then it might be a programming issue. Else I do not see what is wrong with this. The analytical autocorrelation does not always match with what you get from the measured data,which is prone to noise and measurement/computation errors.&lt;/p&gt;&#xA;" OwnerUserId="4299" LastActivityDate="2013-08-09T01:36:36.140" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8192" PostTypeId="1" CreationDate="2013-08-09T04:03:24.343" Score="6" ViewCount="996" Body="&lt;p&gt;I don't know much about GPU computing at the moment, so please pardon the simple question. Can one invert local matrices in parallel on the GPU?  CUBLAS doesn't seem to support factorization, and most of the LU/QR/Chol libraries I've found for GPUs aim instead to accelerate a single direct factorization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if mass matrices had to be recomputed for an explicit DG method, is there a way to reinvert them locally on the GPU (i.e. in more of an MPI fashion, computing a factorization in parallel over multiple warps/blocks/etc)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I'm trying to see if it's possible to assemble and invert a large number of small matrices on a GPU. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2013-08-09T17:47:21.007" LastActivityDate="2013-08-09T17:59:42.180" Title="Local inversion of small matrices on GPUs?" Tags="|finite-element|gpu|discontinuous-galerkin|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8193" PostTypeId="1" AcceptedAnswerId="8201" CreationDate="2013-08-09T05:43:08.177" Score="3" ViewCount="5670" Body="&lt;p&gt;Recently I am using Umfpack with Intel MKL BLAS. To link the library to a program one has to link mkl_rt.lib / mkl_rt.so. However there is no word which version: sequential or parallel of library is linked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone could help?&#xA;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="4334" LastActivityDate="2017-08-23T06:04:05.963" Title="How to tell which (sequential or parallel) version of Intel MKL is linked?" Tags="|blas|intel-mkl|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8194" PostTypeId="2" ParentId="8192" CreationDate="2013-08-09T08:04:41.897" Score="2" Body="&lt;p&gt;ViennaCL has multiple frontends (OpenMP/OpenCL/CUDA) and is easy to use. It can perform QR factorization, however I don't know if it suits you, as it does onl;y a part of factorization on CUDA device. But still it is one of the friendliest libs in my opinion.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://viennacl.sourceforge.net/viennacl-manual-current.pdf&quot; rel=&quot;nofollow&quot;&gt;Manual&lt;/a&gt; - page 32&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://viennacl.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;WWW&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4334" LastActivityDate="2013-08-09T08:04:41.897" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8195" PostTypeId="2" ParentId="8170" CreationDate="2013-08-09T08:08:10.953" Score="0" Body="&lt;p&gt;A guess would be &lt;a href=&quot;http://www.cytoscape.org/&quot; rel=&quot;nofollow&quot;&gt;Cytoscape&lt;/a&gt; - I haven't used it in anger but I helped someone look into it for something else and what I saw was promising&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2013-08-09T08:08:10.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8196" PostTypeId="2" ParentId="8175" CreationDate="2013-08-09T08:17:04.163" Score="4" Body="&lt;p&gt;ParaView should be your best bet. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I would try different versions as each behaves differently (go back as far as 3.12)&lt;/li&gt;&#xA;&lt;li&gt;You also need to make sure that parallel is switched on (New versions have a &quot;Use Multi-core&quot; checkbox in settings)&lt;/li&gt;&#xA;&lt;li&gt;You might need to compile your own version if you are unlucky (I had to do this once to get 64 bit headers working)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="564" LastActivityDate="2013-08-09T08:17:04.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8197" PostTypeId="2" ParentId="8193" CreationDate="2013-08-09T09:38:34.053" Score="0" Body="&lt;p&gt;The results of these two versions should be the same. It's up to you to decide whether you want to utilize multiple cores or not.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-09T09:38:34.053" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8198" PostTypeId="1" CreationDate="2013-08-09T10:10:23.640" Score="5" ViewCount="2206" Body="&lt;p&gt;I need to compute the integral $\int_{\mathbb{R}^3}f(x)\, dx$, where $f$ is a matrix-valued function $f:\mathbb{R}^3\to\mathbb{R}^{3\times 3}$. How do I do that using MATLAB? The function $f$ is implemented in a seperate file. It takes as input three vectors, giving the $x$-, $y$-, and $z$-coordinates of the points in which is has to be evaluated. As output it gives a 5D array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I just write &lt;code&gt;integral3(@f,-1,1,-1,1,-1,1)&lt;/code&gt; for instance, I get the error &lt;code&gt;Integrand output size does not match the input size.&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3618" LastActivityDate="2016-01-31T09:01:53.083" Title="Integration of matrix-valued function using MATLAB" Tags="|matlab|quadrature|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8199" PostTypeId="1" CreationDate="2013-08-09T12:36:47.720" Score="5" ViewCount="348" Body="&lt;p&gt;What is the quickest and simplest way to graph or visualize numerical data that is output as text files?&#xA;Transferring to Matlab is slow, Excel is clunky, things like automatic scaling are essential.&lt;/p&gt;&#xA;" OwnerUserId="4945" LastActivityDate="2013-08-11T12:22:43.590" Title="Quick data visualization" Tags="|visualization|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8200" PostTypeId="2" ParentId="8199" CreationDate="2013-08-09T12:56:56.673" Score="6" Body="&lt;p&gt;&lt;a href=&quot;http://gnuplot.info/&quot;&gt;Gnuplot&lt;/a&gt; does a good job here. You might also try this &lt;a href=&quot;https://github.com/dmcdougall/mpl_binutils&quot;&gt;command line interface to Matplotlib&lt;/a&gt;. The interface of the latter resembles that of &lt;a href=&quot;http://www.gnu.org/software/plotutils/&quot;&gt;GNU plotutils&lt;/a&gt;, which gives you a third alternative.&lt;/p&gt;&#xA;" OwnerUserId="1553" LastActivityDate="2013-08-09T12:56:56.673" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8201" PostTypeId="2" ParentId="8193" CreationDate="2013-08-09T16:22:41.647" Score="4" Body="&lt;p&gt;I believe that MKL has the threaded parallel and serial functions in one unified library. You can try setting OMP_NUM_THREADS or MKL_NUM_THREADS to a range of values and see how the performance varies. Setting either to 1 will give you the serial behavior.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-08-09T16:22:41.647" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8202" PostTypeId="2" ParentId="8192" CreationDate="2013-08-09T17:59:42.180" Score="4" Body="&lt;p&gt;The short answer to your question is yes, you can invert a large number of small, independent matrices on a GPU, and more than likely you can do it efficiently.  The best way to go about it, however, is not such a straight forward answer.  I can think of three possible implementations, although I will admit right from the start I have never attempted these, so I may overlook issues in any of them.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The simplest would be to assemble all the matrices into one larger block diagonal matrix on the gpu, then use a single matrix solve on the whole thing.  This will retain the block diagonal form and simply require pulling each block out as the inverse of its respective matrix.  You would need to tune your block size and thread counts to your smaller matrix sizes, and if they are different sizes, it would become fairly difficult.  This is the only option that you may have a chance of a library implementing.&lt;/li&gt;&#xA;&lt;li&gt;Create a higher level GPU kernel that takes a vector of pointers and matrix sizes, and calls a smaller kernel for each matrix.  This would be a more understandable program, but assigning threads/blocks to your smaller matrices would be more complicated since the block and thread id's are assigned in the high level kernel, and not the low level one.&lt;/li&gt;&#xA;&lt;li&gt;Create a low level kernel to solve a single matrix, and call them all in sequence from the CPU.  This would likely be the best way to go about it, however you would need to include checks to ensure that all of the kernels are complete before you let the CPU continue further into the program.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that all of these methods could also be used for kernels that construct the matrices within the GPU memory as well, and not just solving them.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-08-09T17:59:42.180" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8203" PostTypeId="2" ParentId="8187" CreationDate="2013-08-09T18:39:07.050" Score="1" Body="&lt;p&gt;As we see in Geoffs answer, this is a very simple quadratically constrained problem, or more generally a second-order cone program. If you don't have extreme performance requirements or enormous dimensions, solving it using a standard nonlinear solver in the quadratic form $z^Tz \leq 1$ or using an SOCP solver in the norm formulation $\|z\|\leq 1$ will work perfectly fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have to improve performance, there are methods to exploit the single cone feature. Here is one example&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;SIAM J. Optim., 17(2), 459–484. (26 pages) An Active Set Method for&#xA;  Single-Cone Second-Order Cone Programs E. Erdougan and G. Iyengar&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would like to point out that replacing the norm with a 1-norm probably won't work well. The quadratic norm has its origin in the geometric background of this problem (which I interpret as finding a vector which has the smallest angle to a given set of vectors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, a QP approximation of the problem seems to work extremely well. Remove the quadratic constraint, and add a penalty $\alpha z^Tz$ to the objective instead. I would not be surprised if it is possible to prove something about this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code below, implemented using YALMIP (Disclaimer, developed by me) in MATLAB, using CPLEX as the solver, the average distance from the true $z$ and the $z$ computed using the QP heuristics, is on the order of $10^{-6}$, while the distance to the solution from the LP (1-norm) formulation is on the order $10^{-1}$. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z = sdpvar(5,1);&#xA;r = sdpvar(1);&#xA;&#xA;err1 = [];&#xA;err2 = [];&#xA;for i = 1:1000&#xA;    X = randn(5,10);&#xA;    Con = [r*sqrt(sum(X.^2,1)) &amp;lt;= z'*X,norm(z,2) &amp;lt;= 1]&#xA;    sol = solvesdp(Con,-r)&#xA;    if sol.problem == 0 &amp;amp; double(r)&amp;gt;1e-3&#xA;        zSOCP = double(z);&#xA;        Con = [r*sqrt(sum(X.^2,1)) &amp;lt;= z'*X];&#xA;        sol = solvesdp(Con,-r+0.001*z'*z);&#xA;        zQP = double(z/norm(double(z)));&#xA;        err1 = [err1 norm(zQP-zSOCP)];&#xA;        Con = [r*sqrt(sum(X.^2,1)) &amp;lt;= z'*X, norm(z,1)&amp;lt;=1];&#xA;        sol = solvesdp(Con,-r);&#xA;        zLP = double(z/norm(double(z)));&#xA;        err2 = [err2 norm(zLP-zSOCP)];&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, using geometric insight might lead to a much better approach to solve this problem. You are essentially looking for a particularly defined center of a set of points on the unit-sphere.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-08-09T18:39:07.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8204" PostTypeId="1" CreationDate="2013-08-10T03:20:58.030" Score="10" ViewCount="577" Body="&lt;p&gt;I am working on some ill-conditioned large sparse linear system of equations. I want to use double-double arithmetic or quad-double arithmetic to solve them. I know that there is a package named MPACK developed by Nakata, Maho, which can perform numerical linear algebraic computations under quad-double arithmetic. However, it is designed for dense matrix, not sparse matrix. Do you know whether there is any quad-double arithmetic sparse matrix package?&lt;/p&gt;&#xA;" OwnerUserId="4192" LastActivityDate="2013-09-10T12:01:14.373" Title="Are there any quad-double arithmetic sparse matrix package?" Tags="|sparse-matrix|matrix|condition-number|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8205" PostTypeId="2" ParentId="8178" CreationDate="2013-08-10T04:07:02.973" Score="1" Body="&lt;p&gt;This is not my field but perhaps you could numerically check conservation laws, such as ensemble averaged energy.&lt;/p&gt;&#xA;" OwnerUserId="4881" LastActivityDate="2013-08-10T04:07:02.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8206" PostTypeId="2" ParentId="8204" CreationDate="2013-08-10T04:19:49.760" Score="2" Body="&lt;p&gt;I might give the Trilinos library a try.  They have templated sparse matrix libraries under Tpetra (which is supposed to replace Epetra, their original sparse matrix library).  You can template double, complex, quad, etc, and they have possibly the largest selection of solvers (both direct and iterative) next to PETSc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: after reading the comments, the immediate usefulness of Tpetra seems to be a bit questionable w.r.t quad precision...&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2013-08-23T22:08:17.830" LastActivityDate="2013-08-23T22:08:17.830" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8207" PostTypeId="2" ParentId="8199" CreationDate="2013-08-10T05:26:18.423" Score="9" Body="&lt;p&gt;I think the speed and simplicity of visualizing data is largely dependent on how fluent you are with visualization tools and preprocessing tools. I'm a big advocate of scripting for this purpose, because you can figure out how to do a task once and then copy and repurpose the script on similar tasks, which saves you time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For visualization, it really depends on what you mean by &quot;graph numerical data&quot; and &quot;scalability&quot;. If you want to visualize data in 2D or do really simple 3D plots, options like Gnuplot and Matplotlib work well for data sets that don't require any more than a few processors to handle the computations. I give Matplotlib the edge for its outstanding &lt;a href=&quot;http://matplotlib.org/gallery.html&quot;&gt;gallery of different plots (and example scripts)&lt;/a&gt;; Gnuplot has a similar &lt;a href=&quot;http://gnuplot.sourceforge.net/demo_4.4/&quot;&gt;gallery&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If by &quot;scalability&quot;, you mean &quot;larger data sets&quot;, you'll probably want something more along the lines of Paraview or VisIt (or if you have access to them, Tecplot or Ensight), because these tools generally have features that let you use more processors, which can handle scalability if you mean that you're processing very large data sets. If by  &quot;scalability&quot;, you mean, &quot;generating the same plot for a large number of data sets&quot;, you probably just want to use something that you can script, and then just wrap it in some sort of &lt;code&gt;for&lt;/code&gt; loop. You can do something like that in any of the packages I've mentioned; most of them have Python interfaces or interfaces to other languages (with the possible exceptions of Ensight and Tecplot). Creative use of shell scripting may also work for batch processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another advantage to working with Paraview or VisIt is that they have great postprocessing features for working with data, and multiple types of 3D plots. If you're using a language like Python for scripting, you can do manual postprocessing, too. A similar sort of dichotomy exists when it comes to reading in data; more complex visualization packages like Paraview or VisIt have lots of different built-in converters for reading data from text formats (HDF5, VTK, GMV, Ensight format, FLASH astrophysics data), or you can script that by hand with libraries in your favorite scripting language (mine is Python, and there are Python packages that will read in lots of different text formats).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really, though, it's up to you what you like. If you don't like MATLAB, then I'd recommend Python for simpler tasks, because I think it's easier to install the necessary packages on a variety of platforms (for quick setups, Python distributions like &lt;a href=&quot;http://docs.continuum.io/anaconda/install.html&quot;&gt;Anaconda&lt;/a&gt; and &lt;a href=&quot;https://www.enthought.com/canopy-express/&quot;&gt;Canopy Express&lt;/a&gt; are great options; you can also install everything manually if you like). Installing anything in Windows is a tremendous pain in the ass, but the Python distributions usually seem to work out better than manual installation or trying to install some sort of package manager like Cygwin that contains Gnuplot. I also think that Python code is more readable than Gnuplot scripts, that you're more likely to find useful Python code snippets you can copy, and that there are more Python users out there that will help you out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more complex stuff, it's really a toss-up between Paraview and VisIt. I've used both. VisIt is more intuitive to me, not having read the documentation; I liked Paraview's documentation better, and I can do more with Paraview.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever you pick, if you like it, stick with it, and become an expert user.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-08-10T05:26:18.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8208" PostTypeId="1" AcceptedAnswerId="8212" CreationDate="2013-08-10T08:13:21.447" Score="3" ViewCount="425" Body="&lt;p&gt;This is an attempt to extend an idea from Computer Science.. the dynamic programming based solution of the rod cutting problem (given a rod of an integral length and an array of prices for each integral value of the length, find the optimal cuts that maximize profit). Thinking about it made me wonder.. what if we were not constrained to cut the rod at integral values, but could cut it any where we liked (fractional lengths are possible). Further, lets say that instead of a table of prices corresponding to the integral lengths, we are given a continuous (monotonically increasing, tapering) function that represents the value of the rod at different lengths. For example, for a rod of length 1, the value function might be given by the CDF of a beta distribution. Now, we want to find the number of cuts that we should make and at what values to optimize the total value. Has any one heard of such a problem (I couldn't find any thing online)?&lt;/p&gt;&#xA;" OwnerUserId="4672" LastActivityDate="2013-08-10T19:39:43.430" Title="Continuous version of rod cutting problem" Tags="|optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8209" PostTypeId="2" ParentId="8178" CreationDate="2013-08-10T12:20:04.503" Score="3" Body="&lt;p&gt;A common test consists of computing some ensemble averages that you can check analytically and repeating the experiment a number of times while decreasing the time step length.  You should observe that, when you plot the time step length against the error in log-log scale, the data points can be easily fitted to a line with slope equal to the order of the numerical scheme's global error (that would be two in the case of Verlet).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following code computes an integral curve of a Lennard-Jones oscillator using the Verlet algorithm for different time step lengths and collecting the average total energy (which should be constant).  In the plot below a simple linear fitting reveals that the slope is almost two, as expected from theory.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;const double total_time = 1e3;&#xA;&#xA;const double sigma = 1.0;&#xA;const double sigma6 = pow(sigma, 6.0);&#xA;const double epsilon = 1.0;&#xA;const double four_epsilon = 4.0 * epsilon;&#xA;&#xA;inline double force(double q, double&amp;amp; potential) {&#xA;  const double r2 = q * q;&#xA;  const double r6 = r2 * r2 * r2;&#xA;  const double factor6  = sigma6 / r6;&#xA;  const double factor12 = factor6 * factor6;&#xA;&#xA;  potential = four_epsilon * (factor12 - factor6);&#xA;  return -four_epsilon * (6.0 * factor6 - 12.0 * factor12) / r2 * q;&#xA;}&#xA;&#xA;int main() {&#xA;  const double q0 = 1.5, p0 = 0.1;&#xA;  double potential;&#xA;  const double f0 = force(q0, potential);&#xA;  const double total_energy_exact = p0 * p0 / 2.0 + potential;&#xA;&#xA;  for (double dt = 1e-3; dt &amp;lt;= 5e-2; dt *= 2.0) {&#xA;    const long steps = long(total_time / dt);&#xA;&#xA;    double q = q0, p = p0, f = f0;&#xA;    double total_energy_average = total_energy_exact;&#xA;&#xA;    for (long step = 1; step &amp;lt;= steps; ++step) {&#xA;      p += dt / 2.0 * f;&#xA;      q += dt * p;&#xA;      f = force(q, potential);&#xA;      p += dt / 2.0 * f;&#xA;&#xA;      total_energy_average += p * p / 2.0 + potential;&#xA;    }&#xA;&#xA;    total_energy_average /= double(steps);&#xA;&#xA;    const double err = fabs(total_energy_exact - total_energy_average);&#xA;    cout &amp;lt;&amp;lt; log10(dt) &amp;lt;&amp;lt; &quot;\t&quot;&#xA;         &amp;lt;&amp;lt; log10(err) &amp;lt;&amp;lt; endl;&#xA;  }&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OC7xO.png&quot; alt=&quot;Graph validating simulation&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2013-08-11T08:11:20.267" LastActivityDate="2013-08-11T08:11:20.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8210" PostTypeId="2" ParentId="8173" CreationDate="2013-08-10T17:42:37.650" Score="3" Body="&lt;p&gt;The quantity you are measuring currently is something akin to &quot;&lt;a href=&quot;http://en.wikipedia.org/wiki/Topographic_prominence&quot; rel=&quot;nofollow&quot;&gt;prominence&lt;/a&gt;&quot; which is a better formed topographical quantity than numerical one.  The wiki page in that link describes all sorts of strange cases that arise, even without having any image/surface borders to worry about.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to stick with this alignment metric I would suggest using a &lt;a href=&quot;http://en.wikipedia.org/wiki/Watershed_%28image_processing%29&quot; rel=&quot;nofollow&quot;&gt;watershed transform on the gradient image&lt;/a&gt; to find the area that your peaks encompass.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a conceptual motivation of what is going on, imagine turning those plots upside down then filling them with water.  This will give you a segmentation between your peaks (lakes) which you can use to calculate the quantities you actually care about.  I suspect what will be good is the mean peak height, aka peak volume / peak surface area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the flat noisy graph, this quantity will come out largely similar for each peak, in the one with the prominent peak there will be a really significant outlier in the distribution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It sounds like this is being done on satellite/aerial imagery, so my final suggestion would be adding an additional metric on the deformation tensor, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Trace_%28linear_algebra%29&quot; rel=&quot;nofollow&quot;&gt;trace&lt;/a&gt; of the deformation matrix should be constant for a given altitude, and solutions found outside that constant should also be penalized.  You can do similar things with the off-diagonal terms knowing the trajectory of your travel in the images being compared.   &lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2013-08-11T22:05:19.863" LastActivityDate="2013-08-11T22:05:19.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8211" PostTypeId="1" AcceptedAnswerId="8239" CreationDate="2013-08-10T18:12:21.723" Score="2" ViewCount="3905" Body="&lt;p&gt;If a physical model is not listed in the wizard, we can use Comsol's weak form to enter PDE's (governing equations of a system) in their weak form. How can it be done ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example: 2 equations of momentum (2D flow) coupled with continuity equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho \ v.\nabla v=-\nabla p +\mu \nabla^2 v$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla .v=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This model (well-known) is for calculating velocity and pressure fields&lt;/p&gt;&#xA;" OwnerUserId="4950" LastEditorUserId="4964" LastEditDate="2014-09-01T14:07:44.563" LastActivityDate="2019-05-14T10:51:49.087" Title="PDEs in their weak form in Comsol" Tags="|fluid-dynamics|navier-stokes|comsol|modeling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8212" PostTypeId="2" ParentId="8208" CreationDate="2013-08-10T19:39:43.430" Score="4" Body="&lt;p&gt;If the function is monotonically increasing and tapering (i.e., positive derivative, negative second derivative), then the optimal solution is to make infinitely many cuts and see infinitesimally small pieces of the rod.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you assume that the price function is monotonically increasing with positive second derivatives, your optimal solution is to sell a single piece.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you assume a price function that is monotonically increasing with zero second derivative (i.e., a linear function), then any subdivision is optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this shows is that the integer constraints of the original problem really are an essential part of the problem and that removing them let the problem degenerate into one that has either a trivial solution (convex cost function) or no solution (concave cost function, where there is no maximal revenue with finitely many cuts, just a supremum). Of course, you could assume that the cost function is both convex and concave in parts of its domain, but in that case you likely have a problem that is no longer convex and that may have multiple local optima.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-10T19:39:43.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8215" PostTypeId="2" ParentId="8199" CreationDate="2013-08-11T12:22:43.590" Score="1" Body="&lt;p&gt;I like to use R for all plotting. What I'll do is copy the data from the text file (which will typically be delimited). Then, I'll go x1&amp;lt;-readClipboard() in R. After that, &#xA;t(sapply(strsplit(x1,&quot;\t&quot;),function(x)as.numeric(x)))-&gt;res&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And your data is in the &quot;res&quot; object. You can make all kinds of plots. So that simple, copy and enter the two commands in R. Also, remember to replace the &quot;\t&quot; in your second line with the delimiter of your data (which might be a comma, space or tab). And, you have to type in the x1&amp;lt;-readClipboard() command or the clipboard will be replaced with the command. For the other one that puts it into res, you can copy paste it into R.&#xA;You can also copy from excel files, db visualizer, etc.&lt;/p&gt;&#xA;" OwnerUserId="4672" LastActivityDate="2013-08-11T12:22:43.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8216" PostTypeId="1" CreationDate="2013-08-11T16:41:32.990" Score="7" ViewCount="1267" Body="&lt;p&gt;I recently encountered a problem that looks like a variation of bin packing or knapsack problem, but with the objective to maximise the number of bins/knapsacks:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Consider there is a list of &lt;em&gt;M&lt;/em&gt; items with positive values &lt;em&gt;v&lt;/em&gt;&lt;sub&gt;1&lt;/sub&gt; to &lt;em&gt;v&lt;sub&gt;M&lt;/sub&gt;&lt;/em&gt; respectively. There is also a standard size of bins which carries at most &lt;em&gt;C&lt;/em&gt; items. The objective is to distribute items into as many bins as possible, with each bin having at least a total value of &lt;em&gt;V&lt;/em&gt;. The problem is, given &lt;em&gt;v&lt;/em&gt;&lt;sub&gt;1&lt;/sub&gt;..&lt;em&gt;v&lt;sub&gt;M&lt;/sub&gt;&lt;/em&gt;, &lt;em&gt;C&lt;/em&gt; and &lt;em&gt;V&lt;/em&gt;, how to get the maximum possible number of filled bins? And the actual pattern?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My actual real life problem is actually one step more difficult. In addition to &lt;strong&gt;(A)&lt;/strong&gt; filling in at most &lt;em&gt;C&lt;/em&gt; items of a total value of at least &lt;em&gt;V&lt;/em&gt;, there is another choice of &lt;strong&gt;(B)&lt;/strong&gt; filling in at most &lt;em&gt;C&lt;/em&gt; items of a total value of at least 2*&lt;em&gt;V&lt;/em&gt;. Both kinds of bin can be produced for any number of times, but the second type of bin is (automatically) worth twice as the first type of bin. Then the problem is, what is the maximum possible &lt;em&gt;worth&lt;/em&gt; of bins, i.e.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 * number of type (A) bins + 2 * number of type (B) bins,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the actual pattern?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I just made up the name &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Fukubukuro&quot;&gt;fukubukuro&lt;/a&gt; problem&quot; for how I imagine its differences with knapsack problem :) I don't know the actual name of this problem yet, so I appreciate if anyone can give a direction on it. I also guess this can be solved by integer linear programming, but I hope to learn any more specific algorithms or approximations.&lt;/p&gt;&#xA;" OwnerUserId="4953" LastActivityDate="2019-06-16T20:38:58.803" Title="Bin-packing: Maximise number of bins / &quot;Fukubukuro&quot; problem?" Tags="|optimization|algorithms|approximation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8217" PostTypeId="1" AcceptedAnswerId="8218" CreationDate="2013-08-12T00:48:12.193" Score="56" ViewCount="37998" Body="&lt;p&gt;There is an obvious difference between finite difference and the finite volume method (moving from point definition of the equations to integral averages over cells). But I find FEM and FVM to be very similar; they both use integral form and average over cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the FEM method doing that the FVM is not? I have read a little background on the FEM I understand that the equations are written in the weak form, this gives the method a slightly different stating point than the FVM. However, I don't understand on a conceptual level what the differences are. Does FEM make some assumption regarding how the unknown varies inside the cell, can't this also be done with FVM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am mostly coming from 1D perspective so maybe FEM has advantages with more than one dimension?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't found much information available on this topic on the net. Wikipedia has a section on how the FEM is different from finite difference method, but that is about it, &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_element_method#Comparison_to_the_finite_difference_method&quot;&gt;http://en.wikipedia.org/wiki/Finite_element_method#Comparison_to_the_finite_difference_method&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2018-10-31T03:53:11.810" Title="What are the conceptual differences between the finite element and finite volume method?" Tags="|finite-element|finite-volume|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8218" PostTypeId="2" ParentId="8217" CreationDate="2013-08-12T03:51:16.590" Score="55" Body="&lt;h3&gt;Finite Element: volumetric integrals, internal polynomial order&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Classical finite element methods assume continuous or weakly continuous approximation spaces and ask for volumetric integrals of the weak form to be satisfied.  The order of accuracy is increased by raising the approximation order within elements.  The methods are not exactly conservative, thus often struggle with stability for discontinuous processes.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Finite Volume: surface integrals, fluxes from discontinuous data, reconstruction order&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Finite volume methods use piecewise constant approximation spaces and ask for integrals against piecewise constant test functions to be satisfied.  This yields exact conservation statements.  The volume integral is converted to a surface integral and the entire physics is specified in terms of fluxes in those surface integrals.  For first-order hyperbolic problems, this is a Riemann solve.  Second order/elliptic fluxes are more subtle.  Order of accuracy is increased by using neighbors to (conservatively) reconstruct higher order representations of the state inside elements (slope reconstruction/limiting) or by reconstructing fluxes (flux limiting).  The reconstruction process is usually nonlinear to control oscillations around discontinuous features of the solution, see total variation diminishing (TVD) and essentially non-oscillatory (ENO/WENO) methods.  A nonlinear discretization is necessary to simultaneously obtain both higher than first order accuracy in smooth regions and bounded total variation across discontinuities, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Godunov%27s_theorem&quot; rel=&quot;noreferrer&quot;&gt;Godunov's theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Comments&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Both FE and FV are easy to define up to second order accuracy on unstructured grids.  FE is easier to go beyond second order on unstructured grids.  FV handles non-conforming meshes more easily and robustly.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Combining FE and FV&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The methods can be married in multiple ways.  Discontinuous Galerkin methods are finite element methods that use discontinuous basis functions, thus acquiring Riemann solvers and more robustness for discontinuous processes (especially hyperbolic).  DG methods can be used with nonlinear limiters (usually with some reduction in accuracy), but satisfy a cell-wise entropy inequality without limiting and can thus be used without limiting for some problems where other schemes require limiters.  (This is especially useful for adjoint-based optimization since it makes the discrete adjoint more representative of the continuous adjoint equations.)  Mixed FE methods for elliptic problems use discontinuous basis functions and after some choices of quadrature, can be reinterpreted as standard finite volume methods, see &lt;a href=&quot;https://scicomp.stackexchange.com/a/6712/119&quot;&gt;this answer&lt;/a&gt; for more.  Reconstruction DG methods (aka. $P_N P_M$ or &quot;Recovery DG&quot;) use both FV-like conservative reconstruction and internal order enrichment, and are thus a superset of FV and DG methods.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-08-12T11:32:44.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8219" PostTypeId="1" AcceptedAnswerId="8228" CreationDate="2013-08-12T04:46:51.213" Score="3" ViewCount="964" Body="&lt;p&gt;Is there any libraries(python is preffered) for generation rectangular 3d mesh (for using with the finite difference method), which will be consistent with boundaries? (I've asked this question at &lt;a href=&quot;https://stackoverflow.com/questions/18170603/cfd-mesh-generation&quot;&gt;stackoverflow&lt;/a&gt;, but nothing)&lt;/p&gt;&#xA;" OwnerUserId="4955" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-08-12T20:06:41.320" Title="CFD mesh generation" Tags="|fluid-dynamics|mesh-generation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8220" PostTypeId="1" AcceptedAnswerId="8242" CreationDate="2013-08-12T06:40:38.127" Score="3" ViewCount="635" Body="&lt;p&gt;I have a set of points, among which a subset needs to have the same color in vtk file format. For ex.  all agents with table value &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;0.0, 0.0, 0.0, 0.0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;will have same color. Here is my vtk&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;POINT_DATA 11&#xA;SCALARS Gid FLOAT&#xA;LOOKUP_TABLE table&#xA;0&#xA;1&#xA;2&#xA;3&#xA;4&#xA;5&#xA;6&#xA;7&#xA;8&#xA;9&#xA;10&#xA;LOOKUP_TABLE table 11&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;0.0 0.0  0.0 0.0&#xA;1.0 1.0  1.0 1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could anyone tell me how to do the same in paraview, I mean the associated parameters that need to be changed in Paraview, or is it that the vtk itself does not do as I want?&lt;/p&gt;&#xA;" OwnerUserId="4907" LastActivityDate="2013-08-14T04:27:06.587" Title="VTK: Subset of Points with same color" Tags="|visualization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8221" PostTypeId="1" AcceptedAnswerId="8227" CreationDate="2013-08-12T08:06:22.277" Score="4" ViewCount="896" Body="&lt;p&gt;I am solving a coupled system of non-linear PDEs in 1D. Something like,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = F_1(u,v,w) \\&#xA;v_t = F_2(u,v,w) \\&#xA;w_t = F_3(u,v,w)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where each variable is a function of $x$ (the spatial dimension).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for each point in space I wish to calculate the Jacobian of the vector,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\boldsymbol{F} = \pmatrix{F_1(u,v,w) \\ F_2(u,v,w) \\ F_3(u,v,w)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use the PDEs themselves as a starting point to calculate the Jacobian, or is it better to use the discretised form of the equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask because the equations are very unstable without using exponential fitting, so the discretised form is quite non-trivial. My intuition is that Jacobian should follow from the discretised form as this seems more consistent. Is there a standard approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I were to start with the discretised form, I would simply partially differentiate each term with respect to the appropriate variable.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-08-12T08:33:01.117" LastActivityDate="2013-08-12T16:25:08.053" Title="Should the Jacobian of a system of PDEs be calculated from the main equations of the discretised equation?" Tags="|discretization|nonlinear-equations|newton-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8223" PostTypeId="1" AcceptedAnswerId="8224" CreationDate="2013-08-12T11:13:19.103" Score="19" ViewCount="34660" Body="&lt;p&gt;I would like to know if there is a quick way to compute the Euclidean distance of two vectors in Octave. It seems that there is no special function for that, so should I just use the formula with &lt;code&gt;sqrt&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="4958" LastEditorUserId="1804" LastEditDate="2013-08-13T21:01:24.213" LastActivityDate="2017-07-08T11:29:58.270" Title="Euclidean distance in Octave" Tags="|octave|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8224" PostTypeId="2" ParentId="8223" CreationDate="2013-08-12T11:22:04.537" Score="36" Body="&lt;p&gt;The Euclidean distance between two vectors is the two-norm of their difference, hence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d = norm( x1 - x2 , 2 );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;should do the trick in Octave. Note that if the second argument to &lt;a href=&quot;http://www.gnu.org/software/octave/doc/interpreter/Basic-Matrix-Functions.html&quot;&gt;&lt;code&gt;norm&lt;/code&gt;&lt;/a&gt; is omitted, the 2-norm is used by default.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-08-12T11:22:04.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8225" PostTypeId="2" ParentId="8217" CreationDate="2013-08-12T13:21:42.933" Score="8" Body="&lt;p&gt;The conceptual differences of FEM and FVM are as subtle as the differences between a tree and a pine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you compare a certain FEM scheme to the FVM discretization applied to a particular problem then you can speak of fundamental differences that become evident in different implementation approaches and different approximation properties (as @Jed Brown has laid out in his answer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in general I would say that FVM is a special case of FEM, using a grid of cells and piecewise constant test functions. This relation is also used for convergence analysis of FVM as it can be found in the book by Grossmann, Roos &amp;amp; Stynes: &lt;a href=&quot;http://www.springer.com/mathematics/computational+science+&amp;amp;+engineering/book/978-3-540-71582-5&quot;&gt;&lt;em&gt;Numerical Treatment of Partial Differential Equations&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-08-12T13:21:42.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8227" PostTypeId="2" ParentId="8221" CreationDate="2013-08-12T16:25:08.053" Score="6" Body="&lt;p&gt;If you need Jacobian matrix information for a numerical method, you should calculate the Jacobian matrix of the discretized form of the equations, since that will be consistent with the discretized equations you are solving.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-08-12T16:25:08.053" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8228" PostTypeId="2" ParentId="8219" CreationDate="2013-08-12T17:37:16.600" Score="2" Body="&lt;p&gt;I'm partial to &lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;nofollow&quot;&gt;gmsh&lt;/a&gt;, but it's more of a standalone tool rather than a library you can call from another program. Google directed me to &lt;a href=&quot;http://www.salome-platform.org/&quot; rel=&quot;nofollow&quot;&gt;SALOME&lt;/a&gt;, which does have proper python bindings, but again is mostly a standalone program. I haven't used it, but &lt;a href=&quot;http://www.openfoam.com/features/mesh-generation.php&quot; rel=&quot;nofollow&quot;&gt;OpenFOAM&lt;/a&gt; has a mesh generator built in which might suit your needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a long list of mesh generation software &lt;a href=&quot;http://www.robertschneiders.de/meshgeneration/software.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, but you'll find that most of the 3D mesh generators are either (1) tetrahedral, (2) have weird licensing agreements under which you have to provide proof you're an American fruit bat over the age of 36 to get it, (3) no longer maintained, (4) are not free.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-08-12T17:37:16.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8229" PostTypeId="2" ParentId="8219" CreationDate="2013-08-12T20:06:41.320" Score="-1" Body="&lt;p&gt;I would only add to what Daniel Shapero said that gmsh can be used as both an executable and a library. It also has python bindings.&lt;/p&gt;&#xA;" OwnerUserId="4688" LastActivityDate="2013-08-12T20:06:41.320" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8230" PostTypeId="2" ParentId="8097" CreationDate="2013-08-13T02:11:00.403" Score="1" Body="&lt;p&gt;Looks like I have figured it out, at least for OpenCL ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#ifdef cl_khr_fp64&#xA;    #pragma OPENCL EXTENSION cl_khr_fp64 : enable&#xA;#elif defined(cl_amd_fp64)&#xA;    #pragma OPENCL EXTENSION cl_amd_fp64 : enable&#xA;#else&#xA;    #error &quot;Double precision doubleing point not supported by OpenCL implementation.&quot;&#xA;#endif&#xA;&#xA;double4 Convolution(__read_only image2d_t srcImg, int2 point, double * kern)&#xA;{&#xA;    const sampler_t smp = CLK_NORMALIZED_COORDS_FALSE | CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_LINEAR;&#xA;    int maskSize = 1;&#xA;    double4 sum = (double4)(0.0f,0.0f,0.0f,0.0f);&#xA;    for (int i = -maskSize; i &amp;lt;= maskSize; i++)&#xA;    {&#xA;        for(int j = -maskSize; j &amp;lt;= maskSize; j++) &#xA;        {&#xA;            int2 delta = (int2)(i+maskSize,j+maskSize); &#xA;            int2 pos = (int2)(i,j);&#xA;            sum += kern[(delta.y*3) + delta.x] * convert_double4(read_imageui(srcImg, smp, point + pos));&#xA;        }&#xA;    }&#xA;    return sum;&#xA;}&#xA;&#xA;&#xA;__kernel void imagingTest(__read_only  image2d_t srcImg, __write_only image2d_t dstImg)&#xA;{&#xA;&#xA;    double k = 30.0L;&#xA;    double delta_t = 0.14285714285714285714285714285714L; // 1/7&#xA;&#xA;    double hN[9];&#xA;    hN[0] = 0; hN[1] = 1; hN[2] = 0;&#xA;    hN[3] = 0; hN[4] =-1; hN[5] = 0; &#xA;    hN[6] = 0; hN[7] = 0; hN[8] = 0;&#xA;&#xA;    double hS[9];&#xA;    hS[0] = 0; hS[1] = 0; hS[2] = 0;&#xA;    hS[3] = 0; hS[4] =-1; hS[5] = 0; &#xA;    hS[6] = 0; hS[7] = 1; hS[8] = 0;&#xA;&#xA;    double hE[9];&#xA;    hE[0] = 0; hE[1] = 0; hE[2] = 0;&#xA;    hE[3] = 0; hE[4] =-1; hE[5] = 1; &#xA;    hE[6] = 0; hE[7] = 0; hE[8] = 0;&#xA;&#xA;    double hW[9];&#xA;    hW[0] = 0; hW[1] = 0; hW[2] = 0;&#xA;    hW[3] = 1; hW[4] =-1; hW[5] = 0; &#xA;    hW[6] = 0; hW[7] = 0; hW[8] = 0;&#xA;&#xA;    double hNE[9];&#xA;    hNE[0] = 0; hNE[1] = 0; hNE[2] = 1;&#xA;    hNE[3] = 0; hNE[4] =-1; hNE[5] = 0; &#xA;    hNE[6] = 0; hNE[7] = 0; hNE[8] = 0;&#xA;&#xA;    double hSE[9];&#xA;    hSE[0] = 0; hSE[1] = 0; hSE[2] = 0;&#xA;    hSE[3] = 0; hSE[4] =-1; hSE[5] = 0; &#xA;    hSE[6] = 0; hSE[7] = 0; hSE[8] = 1;&#xA;&#xA;    double hSW[9];&#xA;    hSW[0] = 0; hSW[1] = 0; hSW[2] = 0;&#xA;    hSW[3] = 0; hSW[4] =-1; hSW[5] = 0; &#xA;    hSW[6] = 1; hSW[7] = 0; hSW[8] = 0;&#xA;&#xA;    double hNW[9];&#xA;    hNW[0] = 1; hNW[1] = 0; hNW[2] = 0;&#xA;    hNW[3] = 0; hNW[4] =-1; hNW[5] = 0; &#xA;    hNW[6] = 0; hNW[7] = 0; hNW[8] = 0;&#xA;&#xA;    const sampler_t smp = CLK_NORMALIZED_COORDS_FALSE | CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_LINEAR;&#xA;&#xA;&#xA;&#xA;    int2 coord = (int2)(get_global_id(0), get_global_id(1));&#xA;&#xA;    uint4 bgra = read_imageui(srcImg, smp, coord); &#xA;&#xA;    double4 nablaN = Convolution(srcImg, coord, hN);&#xA;    double4 nablaS = Convolution(srcImg, coord, hS);&#xA;    double4 nablaE = Convolution(srcImg, coord, hE);&#xA;    double4 nablaW = Convolution(srcImg, coord, hW);&#xA;&#xA;    double4 nablaNE = Convolution(srcImg, coord, hNE);&#xA;    double4 nablaNW = Convolution(srcImg, coord, hNW);&#xA;    double4 nablaSE = Convolution(srcImg, coord, hSE);&#xA;    double4 nablaSW = Convolution(srcImg, coord, hSW);&#xA;&#xA;    double4 cN  = exp(-(nablaN /k) * (nablaN /k));&#xA;    double4 cS  = exp(-(nablaS /k) * (nablaS /k));&#xA;    double4 cW  = exp(-(nablaW /k) * (nablaW /k));&#xA;    double4 cE  = exp(-(nablaE /k) * (nablaE /k));&#xA;    double4 cNE = exp(-(nablaNE/k) * (nablaNE/k));&#xA;    double4 cSE = exp(-(nablaSE/k) * (nablaSE/k));&#xA;    double4 cSW = exp(-(nablaSW/k) * (nablaSW/k));&#xA;    double4 cNW = exp(-(nablaNW/k) * (nablaNW/k));&#xA;&#xA;&#xA;    double4 sum = 0.5 * (nablaNE * cNE) + (nablaSE * cSE) + (nablaSW * cSW) + (nablaNW * cNW);&#xA;    sum += (nablaN * cN) + (nablaS * cS) + (nablaW * cW) + (nablaE * cE);&#xA;    sum *= delta_t;&#xA;&#xA;    bgra.x = bgra.y = bgra.z = convert_int(sum.x);&#xA;&#xA;    bgra.w = 255;&#xA;    write_imageui(dstImg, coord, bgra);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This seems to give good results. I'm not sure whether this is a limitation of HLSL. &lt;/p&gt;&#xA;" OwnerUserId="4680" LastActivityDate="2013-08-13T02:11:00.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8231" PostTypeId="2" ParentId="8175" CreationDate="2013-08-13T02:13:43.680" Score="0" Body="&lt;p&gt;You could also try opendx, not sure if the performance will be there&lt;/p&gt;&#xA;" OwnerUserId="4680" LastActivityDate="2013-08-13T02:13:43.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8232" PostTypeId="1" AcceptedAnswerId="8347" CreationDate="2013-08-13T03:44:47.893" Score="0" ViewCount="831" Body="&lt;p&gt;I'm new to opencl but I have some experience using HLSL. In HLSL multiple passes are used when you need to finish a computation before moving on to the next step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know how this sort of thing is done in opencl.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am writing an image filter as below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float4 Convolution(__read_only image2d_t srcImg, int2 point, float * kern)&#xA;{&#xA;    const sampler_t smp = CLK_NORMALIZED_COORDS_FALSE | CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_LINEAR;&#xA;    int maskSize = 1;&#xA;    float4 sum = (float4)(0.0f,0.0f,0.0f,0.0f);&#xA;    for (int i = -maskSize; i &amp;lt;= maskSize; i++)&#xA;    {&#xA;        for(int j = -maskSize; j &amp;lt;= maskSize; j++) &#xA;        {&#xA;            int2 delta = (int2)(i+maskSize,j+maskSize); &#xA;            int2 pos = (int2)(i,j);&#xA;            sum += kern[(delta.y*3) + delta.x] * convert_float4(read_imageui(srcImg, smp, point + pos));&#xA;        }&#xA;    }&#xA;    return sum;&#xA;}&#xA;&#xA;&#xA;__kernel void imagingTest(__read_only  image2d_t srcImg, __write_only image2d_t dstImg)&#xA;{&#xA;    float k = 30.0L;&#xA;    float delta_t = 0.14285714285714285714285714285714L; // 1/7&#xA;&#xA;    float hN[9];&#xA;    hN[0] = 0; hN[1] = 1; hN[2] = 0;&#xA;    hN[3] = 0; hN[4] =-1; hN[5] = 0; &#xA;    hN[6] = 0; hN[7] = 0; hN[8] = 0;&#xA;&#xA;    float hS[9];&#xA;    hS[0] = 0; hS[1] = 0; hS[2] = 0;&#xA;    hS[3] = 0; hS[4] =-1; hS[5] = 0; &#xA;    hS[6] = 0; hS[7] = 1; hS[8] = 0;&#xA;&#xA;    float hE[9];&#xA;    hE[0] = 0; hE[1] = 0; hE[2] = 0;&#xA;    hE[3] = 0; hE[4] =-1; hE[5] = 1; &#xA;    hE[6] = 0; hE[7] = 0; hE[8] = 0;&#xA;&#xA;    float hW[9];&#xA;    hW[0] = 0; hW[1] = 0; hW[2] = 0;&#xA;    hW[3] = 1; hW[4] =-1; hW[5] = 0; &#xA;    hW[6] = 0; hW[7] = 0; hW[8] = 0;&#xA;&#xA;    float hNE[9];&#xA;    hNE[0] = 0; hNE[1] = 0; hNE[2] = 1;&#xA;    hNE[3] = 0; hNE[4] =-1; hNE[5] = 0; &#xA;    hNE[6] = 0; hNE[7] = 0; hNE[8] = 0;&#xA;&#xA;    float hSE[9];&#xA;    hSE[0] = 0; hSE[1] = 0; hSE[2] = 0;&#xA;    hSE[3] = 0; hSE[4] =-1; hSE[5] = 0; &#xA;    hSE[6] = 0; hSE[7] = 0; hSE[8] = 1;&#xA;&#xA;    float hSW[9];&#xA;    hSW[0] = 0; hSW[1] = 0; hSW[2] = 0;&#xA;    hSW[3] = 0; hSW[4] =-1; hSW[5] = 0; &#xA;    hSW[6] = 1; hSW[7] = 0; hSW[8] = 0;&#xA;&#xA;    float hNW[9];&#xA;    hNW[0] = 1; hNW[1] = 0; hNW[2] = 0;&#xA;    hNW[3] = 0; hNW[4] =-1; hNW[5] = 0; &#xA;    hNW[6] = 0; hNW[7] = 0; hNW[8] = 0;&#xA;&#xA;    const sampler_t smp = CLK_NORMALIZED_COORDS_FALSE | CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_LINEAR;&#xA;&#xA;    int2 coord = (int2)(get_global_id(0), get_global_id(1));&#xA;&#xA;    uint4 bgra = read_imageui(srcImg, smp, coord); &#xA;&#xA;    float4 nablaN = Convolution(srcImg, coord, hN);&#xA;    float4 nablaS = Convolution(srcImg, coord, hS);&#xA;    float4 nablaE = Convolution(srcImg, coord, hE);&#xA;    float4 nablaW = Convolution(srcImg, coord, hW);&#xA;&#xA;    float4 nablaNE = Convolution(srcImg, coord, hNE);&#xA;    float4 nablaNW = Convolution(srcImg, coord, hNW);&#xA;    float4 nablaSE = Convolution(srcImg, coord, hSE);&#xA;    float4 nablaSW = Convolution(srcImg, coord, hSW);&#xA;&#xA;    float4 cN  = exp(-(nablaN /k) * (nablaN /k));&#xA;    float4 cS  = exp(-(nablaS /k) * (nablaS /k));&#xA;    float4 cW  = exp(-(nablaW /k) * (nablaW /k));&#xA;    float4 cE  = exp(-(nablaE /k) * (nablaE /k));&#xA;    float4 cNE = exp(-(nablaNE/k) * (nablaNE/k));&#xA;    float4 cSE = exp(-(nablaSE/k) * (nablaSE/k));&#xA;    float4 cSW = exp(-(nablaSW/k) * (nablaSW/k));&#xA;    float4 cNW = exp(-(nablaNW/k) * (nablaNW/k));&#xA;&#xA;    float4 sum = 0.5 * (nablaNE * cNE) + (nablaSE * cSE) + (nablaSW * cSW) + (nablaNW * cNW);&#xA;    sum += (nablaN * cN) + (nablaS * cS) + (nablaW * cW) + (nablaE * cE);&#xA;    sum *= delta_t; &#xA;&#xA;    bgra.x = bgra.y = bgra.z = convert_int(sum.x);&#xA;&#xA;    bgra.w = 255;&#xA;    write_imageui(dstImg, coord, bgra);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This performs one pass of anisotropic diffusion, I would like to be able to apply this process multiple times. How do I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the C# code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using System;&#xA;using System.Collections;&#xA;using System.Collections.Generic;&#xA;using System.Drawing;&#xA;using System.Drawing.Imaging;&#xA;using System.IO;&#xA;using System.Runtime.InteropServices;&#xA;&#xA;using Emgu.CV;&#xA;using Emgu.Util;&#xA;using Emgu;&#xA;using Emgu.CV.Structure;&#xA;&#xA;using OpenCL.Net;&#xA;&#xA;namespace HLSLTest&#xA;{&#xA;    public class Computations&#xA;    {&#xA;        private Cl.Context _context;&#xA;        private Cl.Device _device;&#xA;        private Cl.Kernel kernel;&#xA;&#xA;        private void CheckErr(Cl.ErrorCode err, string name)&#xA;        {&#xA;            if (err != Cl.ErrorCode.Success)&#xA;            {&#xA;                Console.WriteLine(&quot;ERROR: &quot; + name + &quot; (&quot; + err.ToString() + &quot;)&quot;);&#xA;            }&#xA;        }&#xA;&#xA;        private void ContextNotify(string errInfo, byte[] data, IntPtr cb, IntPtr userData)&#xA;        {&#xA;            Console.WriteLine(&quot;OpenCL Notification: &quot; + errInfo);&#xA;        }&#xA;&#xA;        public void Setup()&#xA;        {&#xA;            Cl.ErrorCode error;&#xA;            Cl.Platform[] platforms = Cl.GetPlatformIDs(out error);&#xA;            List&amp;lt;Cl.Device&amp;gt; devicesList = new List&amp;lt;Cl.Device&amp;gt;();&#xA;&#xA;            CheckErr(error, &quot;Cl.GetPlatformIDs&quot;);&#xA;&#xA;            foreach (Cl.Platform platform in platforms)&#xA;            {&#xA;                string platformName = Cl.GetPlatformInfo(platform, Cl.PlatformInfo.Name, out error).ToString();&#xA;                Console.WriteLine(&quot;Platform: &quot; + platformName);&#xA;                CheckErr(error, &quot;Cl.GetPlatformInfo&quot;);&#xA;&#xA;                //We will be looking only for GPU devices&#xA;                foreach (Cl.Device device in Cl.GetDeviceIDs(platform, Cl.DeviceType.Gpu, out error))&#xA;                {&#xA;                    CheckErr(error, &quot;Cl.GetDeviceIDs&quot;);&#xA;                    Console.WriteLine(&quot;Device: &quot; + device.ToString());&#xA;                    devicesList.Add(device);&#xA;                }&#xA;            }&#xA;&#xA;            if (devicesList.Count &amp;lt;= 0)&#xA;            {&#xA;                Console.WriteLine(&quot;No devices found.&quot;);&#xA;                return;&#xA;            }&#xA;&#xA;            _device = devicesList[0];&#xA;&#xA;            if (Cl.GetDeviceInfo(_device, Cl.DeviceInfo.ImageSupport, out error).CastTo&amp;lt;Cl.Bool&amp;gt;() == Cl.Bool.False)&#xA;            {&#xA;                Console.WriteLine(&quot;No image support.&quot;);&#xA;                return;&#xA;            }&#xA;&#xA;            _context = Cl.CreateContext(null, 1, new[] { _device }, ContextNotify, IntPtr.Zero, out error); //Second parameter is amount of devices&#xA;            CheckErr(error, &quot;Cl.CreateContext&quot;);&#xA;&#xA;            //Load and compile kernel source code.&#xA;            string programPath = Environment.CurrentDirectory + &quot;/../../../ImagingTest.cl&quot;;  //The path to the source file may vary&#xA;&#xA;            if (!System.IO.File.Exists(programPath))&#xA;            {&#xA;                Console.WriteLine(&quot;Program doesn't exist at path &quot; + programPath);&#xA;                return;&#xA;            }&#xA;&#xA;            string programSource = System.IO.File.ReadAllText(programPath);&#xA;&#xA;            using (Cl.Program program = Cl.CreateProgramWithSource(_context, 1, new[] { programSource }, null, out error))&#xA;            {&#xA;                CheckErr(error, &quot;Cl.CreateProgramWithSource&quot;);&#xA;&#xA;                //Compile kernel source&#xA;                error = Cl.BuildProgram(program, 1, new[] { _device }, string.Empty, null, IntPtr.Zero);&#xA;                CheckErr(error, &quot;Cl.BuildProgram&quot;);&#xA;&#xA;                //Check for any compilation errors&#xA;                if&#xA;                (&#xA;                    Cl.GetProgramBuildInfo&#xA;                    (&#xA;                        program,&#xA;                        _device,&#xA;                        Cl.ProgramBuildInfo.Status,&#xA;                        out error&#xA;                    ).CastTo&amp;lt;Cl.BuildStatus&amp;gt;() != Cl.BuildStatus.Success&#xA;                )&#xA;                {&#xA;                    CheckErr(error, &quot;Cl.GetProgramBuildInfo&quot;);&#xA;                    Console.WriteLine(&quot;Cl.GetProgramBuildInfo != Success&quot;);&#xA;                    Console.WriteLine(Cl.GetProgramBuildInfo(program, _device, Cl.ProgramBuildInfo.Log, out error));&#xA;                    return;&#xA;                }&#xA;&#xA;                //Create the required kernel (entry function)&#xA;                kernel = Cl.CreateKernel(program, &quot;imagingTest&quot;, out error);                &#xA;                CheckErr(error, &quot;Cl.CreateKernel&quot;);&#xA;&#xA;            }&#xA;        }&#xA;&#xA;        public void ImagingTest(Image&amp;lt;Gray, Single&amp;gt; InputImage, out Image&amp;lt;Gray, Single&amp;gt; outputImage)&#xA;        {&#xA;            Cl.ErrorCode error;&#xA;&#xA;            int intPtrSize = 0;&#xA;            intPtrSize = Marshal.SizeOf(typeof(IntPtr));&#xA;&#xA;            //Image's RGBA data converted to an unmanaged[] array&#xA;            byte[] inputByteArray;&#xA;            //OpenCL memory buffer that will keep our image's byte[] data.&#xA;            Cl.Mem inputImage2DBuffer;&#xA;&#xA;            Cl.ImageFormat clImageFormat = new Cl.ImageFormat(Cl.ChannelOrder.RGBA, Cl.ChannelType.Unsigned_Int8);&#xA;&#xA;            int inputImgWidth, inputImgHeight;&#xA;            int inputImgBytesSize;&#xA;            int inputImgStride;&#xA;&#xA;            inputImgWidth = InputImage.Width;&#xA;            inputImgHeight = InputImage.Height;&#xA;&#xA;            System.Drawing.Bitmap bmpImage = InputImage.ToBitmap();&#xA;&#xA;            //Get raw pixel data of the bitmap&#xA;            //The format should match the format of clImageFormat&#xA;            BitmapData bitmapData = bmpImage.LockBits&#xA;            (&#xA;                new Rectangle(0, 0, bmpImage.Width, bmpImage.Height),&#xA;                ImageLockMode.ReadOnly,&#xA;                PixelFormat.Format32bppArgb&#xA;            );&#xA;&#xA;            inputImgStride = bitmapData.Stride;&#xA;            inputImgBytesSize = bitmapData.Stride * bitmapData.Height;&#xA;&#xA;            //Copy the raw bitmap data to an unmanaged byte[] array&#xA;            inputByteArray = new byte[inputImgBytesSize];&#xA;            Marshal.Copy(bitmapData.Scan0, inputByteArray, 0, inputImgBytesSize);&#xA;&#xA;            //Allocate OpenCL image memory buffer&#xA;            inputImage2DBuffer = Cl.CreateImage2D&#xA;            (&#xA;                _context,&#xA;                Cl.MemFlags.CopyHostPtr | Cl.MemFlags.ReadOnly,&#xA;                clImageFormat,&#xA;                (IntPtr)bitmapData.Width,&#xA;                (IntPtr)bitmapData.Height,&#xA;                (IntPtr)0,&#xA;                inputByteArray,&#xA;                out error&#xA;            );&#xA;&#xA;            CheckErr(error, &quot;Cl.CreateImage2D input&quot;);&#xA;&#xA;            //Unmanaged output image's raw RGBA byte[] array&#xA;            byte[] outputByteArray = new byte[inputImgBytesSize];&#xA;&#xA;            //Allocate OpenCL image memory buffer&#xA;            Cl.Mem outputImage2DBuffer = Cl.CreateImage2D&#xA;            (&#xA;                _context,&#xA;                Cl.MemFlags.CopyHostPtr | Cl.MemFlags.WriteOnly,&#xA;                clImageFormat,&#xA;                (IntPtr)inputImgWidth,&#xA;                (IntPtr)inputImgHeight,&#xA;                (IntPtr)0, &#xA;                outputByteArray,&#xA;                out error&#xA;            );&#xA;&#xA;            CheckErr(error, &quot;Cl.CreateImage2D output&quot;);&#xA;&#xA;            //Pass the memory buffers to our kernel function&#xA;            error  = Cl.SetKernelArg(kernel, 0, (IntPtr)intPtrSize,  inputImage2DBuffer);&#xA;            error |= Cl.SetKernelArg(kernel, 1, (IntPtr)intPtrSize, outputImage2DBuffer);&#xA;            CheckErr(error, &quot;Cl.SetKernelArg&quot;);&#xA;&#xA;            //Create a command queue, where all of the commands for execution will be added&#xA;            Cl.CommandQueue cmdQueue = Cl.CreateCommandQueue(_context, _device, (Cl.CommandQueueProperties)0, out error);&#xA;            CheckErr(error, &quot;Cl.CreateCommandQueue&quot;);&#xA;&#xA;            Cl.Event clevent;&#xA;&#xA;            //Copy input image from the host to the GPU.&#xA;            IntPtr[] originPtr = new IntPtr[] { (IntPtr)0, (IntPtr)0, (IntPtr)0 };  //x, y, z&#xA;            IntPtr[] regionPtr = new IntPtr[] { (IntPtr)inputImgWidth, (IntPtr)inputImgHeight, (IntPtr)1 }; //x, y, z&#xA;            IntPtr[] workGroupSizePtr = new IntPtr[] { (IntPtr)inputImgWidth, (IntPtr)inputImgHeight, (IntPtr)1 };&#xA;            error = Cl.EnqueueWriteImage(cmdQueue, inputImage2DBuffer, Cl.Bool.True, originPtr, regionPtr, (IntPtr)0, (IntPtr)0, inputByteArray, 0, null, out clevent);&#xA;            CheckErr(error, &quot;Cl.EnqueueWriteImage&quot;);&#xA;&#xA;            //Execute our kernel (OpenCL code)&#xA;            error = Cl.EnqueueNDRangeKernel(cmdQueue, kernel, 2, null, workGroupSizePtr, null, 0, null, out clevent);&#xA;&#xA;            CheckErr(error, &quot;Cl.EnqueueNDRangeKernel&quot;);&#xA;&#xA;            //Wait for completion of all calculations on the GPU.&#xA;            error = Cl.Finish(cmdQueue);&#xA;            CheckErr(error, &quot;Cl.Finish&quot;);&#xA;&#xA;            //Read the processed image from GPU to raw RGBA data byte[] array&#xA;            error = Cl.EnqueueReadImage&#xA;            (&#xA;                cmdQueue,&#xA;                outputImage2DBuffer,&#xA;                Cl.Bool.True,&#xA;                originPtr,&#xA;                regionPtr,&#xA;                (IntPtr)0,&#xA;                (IntPtr)0,&#xA;                outputByteArray,&#xA;                0,&#xA;                null,&#xA;                out clevent&#xA;            );&#xA;&#xA;            CheckErr(error, &quot;Cl.clEnqueueReadImage&quot;);&#xA;&#xA;            //Clean up memory&#xA;            Cl.ReleaseKernel(kernel);&#xA;            Cl.ReleaseCommandQueue(cmdQueue);&#xA;&#xA;            Cl.ReleaseMemObject(inputImage2DBuffer);&#xA;            Cl.ReleaseMemObject(outputImage2DBuffer);&#xA;&#xA;            //Get a pointer to our unmanaged output byte[] array&#xA;            GCHandle pinnedOutputArray = GCHandle.Alloc(outputByteArray, GCHandleType.Pinned);&#xA;            IntPtr outputBmpPointer = pinnedOutputArray.AddrOfPinnedObject();&#xA;&#xA;            //Create a new bitmap with processed data and save it to a file.&#xA;            Bitmap outputBitmap = new Bitmap(inputImgWidth, inputImgHeight, inputImgStride, PixelFormat.Format32bppArgb, outputBmpPointer);&#xA;&#xA;            outputImage = new Image&amp;lt;Gray, Single&amp;gt;(outputBitmap);&#xA;&#xA;            //outputBitmap.Save(outputImagePath, System.Drawing.Imaging.ImageFormat.Png);&#xA;&#xA;            pinnedOutputArray.Free();&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4680" LastEditorUserId="4680" LastEditDate="2013-08-15T02:43:21.587" LastActivityDate="2013-08-28T06:26:14.317" Title="How to use multiple passes in OpenCL?" Tags="|opencl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8233" PostTypeId="1" AcceptedAnswerId="8237" CreationDate="2013-08-13T06:12:28.920" Score="2" ViewCount="1672" Body="&lt;p&gt;I am looking into upgrading to a High Performance Computing work-station to do mostly computational fluid dynamics computations and some fundamental numerics. The CFD software that is available does not support GPU/CUDA. Aside from sufficient RAM and appropriate number of processors, the major bottleneck for my computations are the sockets of the processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two choices for processors: Intel and AMD. Intel interconnects their Xeon processors with &lt;a href=&quot;http://en.wikipedia.org/wiki/Intel_QuickPath_Interconnect&quot; rel=&quot;nofollow&quot;&gt;QPI&lt;/a&gt;. AMD uses &lt;a href=&quot;http://en.wikipedia.org/wiki/HyperTransport&quot; rel=&quot;nofollow&quot;&gt;HyperTransport&lt;/a&gt; for their connection sockets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any &lt;strong&gt;reliable&lt;/strong&gt; way I can compare the two just by using what is given to me in the data sheets? Has anybody here faced a similar decision? Advise is highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2013-08-13T16:24:29.100" Title="HPC: Intel QPI versus AMD with HyperTransport" Tags="|hpc|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8235" PostTypeId="2" ParentId="8233" CreationDate="2013-08-13T09:51:43.107" Score="0" Body="&lt;p&gt;I think you are asking this question at too low a level. The processor interconnect is at such a fundamental level that you will have a hard time evaluating its effect. You also do not have much of a choice -- you can't just exchange this part of a system. Things that are by far more important (and that you can change by choosing different CPUs from the same vendor) are clock rate, memory bandwidth, cache size, how many cores, etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-13T09:51:43.107" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8237" PostTypeId="2" ParentId="8233" CreationDate="2013-08-13T13:06:54.800" Score="4" Body="&lt;p&gt;No there is no reliable way to predict the impact of interconnect characteristics on code performance solely from numbers on data sheets.  One of the major confounding factors in the way of establishing the simple relationship you seek is the characteristics of your code about which we know nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, purchasers of HPC systems (large and small) face this decision all the time.  The usual answer is simple: benchmark, in particular benchmark your code on the target platforms.  If your purchasing power is too modest to bring a truck load of Intel and AMD kit and a truck load of Intel and AMD engineers flocking to your data centre, scare up some of the most similar hardware you can lay your hands on and do some of your own investigations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I note that the article you pointed to in your comment on Wolfgang's answer is a report on benchmarking, not an argument from data sheet to code performance.&lt;/p&gt;&#xA;" OwnerUserId="1003" LastEditorUserId="1003" LastEditDate="2013-08-13T16:24:29.100" LastActivityDate="2013-08-13T16:24:29.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8238" PostTypeId="2" ParentId="8216" CreationDate="2013-08-13T13:36:18.047" Score="2" Body="&lt;p&gt;I found &lt;a href=&quot;http://math.haifa.ac.il/lea/lbp.pdf&quot; rel=&quot;nofollow&quot;&gt;a paper (circa 2005) by Boyar et al&lt;/a&gt; which studies algorithms for problems of this type, which they call maximum resource bin-packing problems (to contrast with the usual goal of bin-packing to minimize a resource such as the number of bins).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Usually, for bin packing problems, we try to minimize the number of bins&#xA;  used or in the case of the dual bin packing problem, maximize the number&#xA;  or total size of accepted items.  This paper presents results for the opposite&#xA;  problems, where we would like to maximize the number of bins used or minimize&#xA;  the number or total size of accepted items. We consider off-line and on-line&#xA;  variants of the problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By on-line variants are meant problems in which a &quot;current item&quot; must be assigned to a bin before proceeding to the next item, while off-line variants permit selection of items for assignment without regard to such input sequencing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performance of an algorithm is assessed by bounding the optimal solution OPT(I) (maximum number of bins used) with a linear function $c$ ALG(I) + $b$ of the number used by the algorithm, and defining approximation ratio $\mathcal{R}_{ALG}$ to be the infimum of $c$ over all inputs I.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors note a similarity between the Bin-Covering Problem where bin capacity is 1 and Maximum Resource Bin-Packing with twice the bin capacity (2).  In Bin-Covering the goal is assigning items to meet or exceed the capacity of as many bins as possible.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-08-13T13:36:18.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8239" PostTypeId="2" ParentId="8211" CreationDate="2013-08-13T13:42:13.507" Score="3" Body="&lt;p&gt;This &quot;model&quot; is the incompressible (constant density) Navier Stokes problem, the second equation being the mass balance:&#xA;$$&#xA;\frac{\partial\rho }{\partial t}+\nabla\cdot(\rho v)=0&#xA;$$&#xA;I have worked in the past with Comsol, and I believe that the Navier Stokes weak forms are readily implemented in the CFD module as states the Comsol modeling manual in this &lt;a href=&quot;http://filebox.ece.vt.edu/~hbhudi/Laboratory/Distributed%20Temperature%20Sensing/Gasifier%20simulation/modeling.pdf&quot; rel=&quot;nofollow&quot;&gt;LINK&lt;/a&gt;. You would only have to input the initial values of the (velocity,pressure) field and the boundary conditions. Your model input will be the density and the viscosity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br/&gt;&#xA;Comsol supports however adding weak contributions to the boundaries or in local points but this is not what you are asking for i presume&lt;/p&gt;&#xA;" OwnerUserId="4964" LastActivityDate="2013-08-13T13:42:13.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8240" PostTypeId="1" AcceptedAnswerId="8243" CreationDate="2013-08-13T14:02:31.263" Score="0" ViewCount="4099" Body="&lt;p&gt;After reading some stuff on STL files, I realized that they do no contain information about a mesh (connectivity for instance) , but they represent the surface geometry of the part to be meshed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, some available tools like the open source Gmsh, could import the STL file and create the mesh from the defined vertices within the STL file. Can anyone guide to some algorithm that performs such conversion ?&lt;/p&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2013-08-13T14:10:44.533" LastActivityDate="2013-08-13T20:26:49.940" Title="Convert STL file to other formats like .msh (Gmsh) and .inp (Abaqus)" Tags="|mesh|gmsh|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8242" PostTypeId="2" ParentId="8220" CreationDate="2013-08-13T17:18:29.563" Score="2" Body="&lt;p&gt;The best solution I can give you is in the development version of ParaView and won't be available until ParaView 4.1 comes out in a couple of months. It uses the idea of categories. What I did was create a Wavelet source, contour that with isosurface values of 100, 200, 220, and 240, then use the Clip filter to look at the &quot;insides&quot;. After that is done, go to the View menu and select check Color Map Editor. In the color map editor check the Interpret Values As Categories. The click on the button with the 3 overlapping squares (red, yellow and blue). This will bring up the discrete values of your variable. In the first column you can click inside to choose an annotation color (I did green for the first 3 colors and red for the last). I also specified annotations for them: a, b, c and d. I attached an image to show you what it looks like.&lt;img src=&quot;https://i.stack.imgur.com/wKkQj.png&quot; alt=&quot;color categories&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this doesn't work for you maybe you can try playing around a bit with the color map editor to get it to work the way you want.&lt;/p&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2013-08-13T17:18:29.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8243" PostTypeId="2" ParentId="8240" CreationDate="2013-08-13T20:26:49.940" Score="2" Body="&lt;p&gt;What you need to do is to analyze the set of input vertices (each triangle contributes three of these) and assign consecutive indices to them so that vertices that appear multiple times (i.e. are shared by different triangles) are assigned the same index every time. This leaves you with a list of (unique) vertices ${\bf x}_i$, and a list of index triples $(i, j, k)$ indexing into the vertex list so that the $({\bf x}_i, {\bf x}_j, {\bf x}_k)$'s represent the original triangles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle this is easy to do - just use a hash table to associate indices to 3D points and fill it as you iterate over all vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the coordinates of equivalent vertices will possibly not be exactly identical but differ by small amounts due to roundoff problems. In this case, discretize your input data with a grid resolution of $\delta$ (which should be a small number corresponding to the maximum expected roundoff error, e.g. $10^{-6}$). Use the grid coordinates of your vertices thus obtained (rather than their 3D positions) as keys in your hashtable to find the associated vertex index, but make sure to check whether any one of the neighboring grid cells was assigned an index earlier before generating a new index. The last step ensures that close vertices will receive the same index.&lt;/p&gt;&#xA;" OwnerUserId="4968" LastActivityDate="2013-08-13T20:26:49.940" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8244" PostTypeId="2" ParentId="5383" CreationDate="2013-08-13T21:06:37.670" Score="2" Body="&lt;p&gt;As &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/odeset.html#f92-1017470&quot; rel=&quot;nofollow&quot;&gt;described in the documentation&lt;/a&gt;, one can set the &lt;code&gt;'Event'&lt;/code&gt; property using &lt;code&gt;odeset&lt;/code&gt;. The following (in an m-file named &lt;code&gt;ode45EventTest.m&lt;/code&gt;) provides a barebones example of your use case, with the trivial ODE $\frac{dx}{dt} = 1, x(0)=0$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function ode45EventTest&#xA;for tMax = [2, 20]&#xA;    [~,~,~,~,IE] = ode45(@(t,y) 1, [0 tMax],0,odeset('Event',@myEvent));&#xA;    if(isempty(IE))&#xA;        fprintf('Threshold not exceeded for tMax = %g\n',tMax);&#xA;    else&#xA;        fprintf('Threshold exceeded for tMax = %g\n',tMax); &#xA;    end&#xA;end&#xA;&#xA;end&#xA;&#xA;function [value,isterminal,direction] = myEvent(~,y)&#xA;threshold  = 10;&#xA;r          = double(abs(y) &amp;gt; threshold);&#xA;value      = r;&#xA;isterminal = r;&#xA;direction  = 0;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3824" LastActivityDate="2013-08-13T21:06:37.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8245" PostTypeId="2" ParentId="5198" CreationDate="2013-08-14T01:57:16.537" Score="1" Body="&lt;p&gt;One way to think about a differential $k$-form is &quot;something integrable over a $k$-dimensional manifold&quot;. The most familiar example would be the volume form $dx$ in $\int_0^1 x^2 dx$, but also the $x^2$, which a $0$-form.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stokes' Theorem generalizes many of the identities you're familiar with from vector calculus, such as the divergence theorem. These identities are applied to integral conservation laws to compute fluxes across boundaries in Finite Volume Methods so one should, as you suspect, be able to write everything in terms of differential forms. &lt;/p&gt;&#xA;" OwnerUserId="3824" LastActivityDate="2013-08-14T01:57:16.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8246" PostTypeId="1" CreationDate="2013-08-14T04:00:41.023" Score="7" ViewCount="1831" Body="&lt;p&gt;I am solving 3D time-harmonic Maxwell FDFD problems (which result in huge sparse linear systems) using BiCGStab(l). I have tried out a bunch of different methods and for my specific use case, it seems like this algorithm outperforms the other common ones such as BiCG and restarted GMRES in terms of total solution time. However, I have noticed that as I have increased my problem size as well as increased the meshing accuracy (I am using a non-uniform grid, but the smallest $dx$ seems to be important for the convergence behavior) that I am running into an issue where often after many iterations for a specific initial solution, BiCGStab(l) stagnates and cannot reduce the residual further. This seems to happen about 10% of the time and so far I have always been able to solve it by restarting from a different initial solution. Each problem takes a long time to solve, however, so it would be ideal if I could be guaranteed convergence on the first go. I have tried setting l=2,4,5, and even 8 and the same thing happens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if doing something simple like detecting stagnation, taking the current solution and randomly perturbing it a little bit, followed by restarting the solver with that as the initial guess would help. Sounds like a complete hack, but I'm not sure how else I can resolve this without having to completely restart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it is important for anything, I am using a simple preconditioner which is pretty much a diagonal preconditioner. I found that for my specific problems it leads to faster convergence than an ILU(0) preconditioner. I have not tried any other preconditioners than these two.&lt;/p&gt;&#xA;" OwnerUserId="1480" LastActivityDate="2013-08-30T18:13:02.850" Title="Convergence/stagnation of BiCGStab(l)" Tags="|finite-difference|iterative-method|convergence|krylov-method|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8247" PostTypeId="2" ParentId="8220" CreationDate="2013-08-14T04:27:06.587" Score="0" Body="&lt;p&gt;This is not an exact solution but here is what I did, I just used the &lt;em&gt;Default Lookup table&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;POINT_DATA 11&#xA;SCALARS Gid FLOAT&#xA;LOOKUP_TABLE default&#xA;0&#xA;0&#xA;2&#xA;2&#xA;4&#xA;5&#xA;5&#xA;7&#xA;10&#xA;10&#xA;10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here &lt;em&gt;points with the same&lt;/em&gt; &lt;strong&gt;default LOOK_UP&lt;/strong&gt; &lt;em&gt;Values&lt;/em&gt; will have the same color in any color scale chosen in your visualizer.&lt;/p&gt;&#xA;" OwnerUserId="4907" LastActivityDate="2013-08-14T04:27:06.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8248" PostTypeId="2" ParentId="5198" CreationDate="2013-08-14T09:08:07.083" Score="1" Body="&lt;p&gt;Differential geometric techniques are used in formulations / understanding of finite-element (-volume) methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://journals.cambridge.org/action/displayAbstract?fromPage=online&amp;amp;aid=439260&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999197956564&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1285" LastActivityDate="2013-08-14T09:08:07.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8249" PostTypeId="2" ParentId="8246" CreationDate="2013-08-14T18:05:54.087" Score="4" Body="&lt;p&gt;You might be familiar with the following paper already:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-3-642-22061-6_10&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/chapter/10.1007%2F978-3-642-22061-6_10&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problems which are highly indefinite and oscillatory are very difficult to design robust iterative methods for. The paper gives some suggestions which might be helpful to you though, many of them have been extended to the time-harmonic Maxwell case as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: For the benefit of others, though you already have encountered this frustration, I'll mention that the difficulties you describe in proper convergence only increase with increasing wavenumber, as this parameter increases the oscillatory nature of the solution. The paper linked I think describes the nature of this problem far better than I could in a few paragraphs, and I highly recommend reading through it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should mention however for respect's sake that there have been developed some robust iterative methods which have even seen successful parallel implementation. Here's one such paper&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/1007.4291&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/abs/1007.4291&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My (primitive) understanding is this method is kind of like a multiplicative Schwarz preconditioner using PML boundary conditions, with some brilliant insights that allow the preconditioner to be applied with lower computational complexity than normal (using H-matrix algebra).&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2013-08-15T06:26:08.903" LastActivityDate="2013-08-15T06:26:08.903" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8250" PostTypeId="1" AcceptedAnswerId="8281" CreationDate="2013-08-14T20:14:20.203" Score="1" ViewCount="236" Body="&lt;p&gt;I have a program that does some setup work, forks several times, and exits. The child processes do most of the heavy computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will torque kill the child processes when it detects the parent process exiting?&lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2013-08-21T23:30:03.453" LastActivityDate="2013-08-22T19:59:50.153" Title="If a process created by a torque job forks and then exits, does torque kill the child processes?" Tags="|pbs|" AnswerCount="1" CommentCount="1" ClosedDate="2014-04-30T01:15:58.277" ContentLicense="CC BY-SA 3.0" />
  <row Id="8251" PostTypeId="1" CreationDate="2013-08-15T07:23:36.517" Score="6" ViewCount="1148" Body="&lt;p&gt;To summarise this question in advance, I'm looking for a good hash function that is suitable for generating pseudo-random numbers in Monte Carlo simulations. This means it should be reasonably fast (so something like md5 is ruled out) but have statistics that are good enough for numerical applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is that sometimes, when writing simulation code, I've felt the need to generate a function that maps some set of objects to floats between 0 and 1, in a pseudo-random way. Such a function might always map the Numpy array &lt;code&gt;[1.0, 2.0, 3.0]&lt;/code&gt; to 0.11214, but on a different run of the simulation a different function might be generated, which always maps &lt;code&gt;[1.0, 2.0, 3.0]&lt;/code&gt; to 0.92546. The input set might not consist of arrays, but if it does then it's important that order is not ignored, i.e. &lt;code&gt;[1.0, 2.0, 3.0]&lt;/code&gt; and &lt;code&gt;[1.0, 3.0, 2.0]&lt;/code&gt; should return different results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems there are a few different technical terms for generating such a random function: depending on details of the implementation it's either called &lt;a href=&quot;http://en.wikipedia.org/wiki/Universal_hashing&quot; rel=&quot;nofollow&quot;&gt;universal hashing&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Pseudorandom_function_family&quot; rel=&quot;nofollow&quot;&gt;psuedorandom function families&lt;/a&gt;. Of course one way to implement this is to use a good hashing function. Different pseudorandom functions can be generated using &quot;salting&quot;, i.e. prepending a fixed string to every object before running the hashing algorithm. Changing this prefix gives a different pseudorandom function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secure hashing algorithms such as md5 will have excellent statistics, but are too slow to be practical in a Monte Carlo simulation. Of course there exist many &quot;fast hashing&quot; algorithms (i.e. non-secure hashing algorithms) that might be suitable to use instead. &#xA;However, the issue is that I haven't been able to find a good analysis of any such algorithms &lt;em&gt;in terms of their suitability for numerical computation&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is important, because most fast hashing algorithms are designed for use in file systems. Typically, papers describing hash functions evaluate them in terms of how well they will perform in such an application. However, and the requirements for this are rather different from the requirements for a numerical simulation. I'm not an expert, but the essential difference is that for a file system we really care a lot about &lt;em&gt;avoiding collisions&lt;/em&gt;, whereas for a numerical algorithm we care most about &lt;em&gt;uniformity&lt;/em&gt; of the distribution of outputs and &lt;em&gt;statistical independence&lt;/em&gt; of the functions' outputs, even if the function is given inputs that are closely related.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These goals are related to one another (and secure hashing algorithms will score very well on all three) but they are not the same. As a trivial example to illustrate this, using Python's built-in hashing algorithm modulo 1000, &lt;code&gt;&quot;aaa&quot;&lt;/code&gt; is mapped to 340, &lt;code&gt;&quot;aab&quot;&lt;/code&gt; to 343, &lt;code&gt;&quot;aac&quot;&lt;/code&gt; to 342, &lt;code&gt;&quot;aad&quot;&lt;/code&gt; to 337, &lt;code&gt;&quot;aae&quot;&lt;/code&gt; to 336 and &lt;code&gt;&quot;aaf&quot;&lt;/code&gt; to 339. There are no collisions in this data, but the results are clearly neither uniformly distributed nor independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm looking for any advice on which hashing algorithm(s) to use in Monte Carlo simulations, to get a good trade-off between speed and statistics that are suitable for numerics. In an ideal world there would also be an existing C implementation with Python bindings; it would also be ideal if objects like Numpy arrays could be hashed directly, rather than first having to convert them to strings. Ideally I would like an off the shelf solution that I can trust, in the same way that I trust Numpy's random number generator. However, I don't mind implementing it myself if that's what necessary - the important thing is to find an algorithm that has been formally evaluated in terms of it suitability for numerical applications, rather than for use in file systems.&lt;/p&gt;&#xA;" OwnerUserId="3746" LastEditorUserId="20688" LastEditDate="2019-06-22T22:43:46.433" LastActivityDate="2019-06-22T22:43:46.433" Title="Hashing algorithms/implementations for Monte Carlo simulation" Tags="|python|monte-carlo|random-number-generation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8252" PostTypeId="2" ParentId="8251" CreationDate="2013-08-15T09:30:19.327" Score="2" Body="&lt;p&gt;The first thing that comes to my mind is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import random&#xA;&#xA;def randomy(obj):&#xA;    &quot;&quot;&quot;&#xA;    Map objects to uniformly distributed random numbers in [0, 1].&#xA;    The input must be hash-able. You will get the same output for&#xA;    the same input during all calls in the same program run, but&#xA;    outputs may vary between different program runs.&#xA;    &quot;&quot;&quot;&#xA;    try:&#xA;        return randomy._store[obj]&#xA;    except KeyError:&#xA;        value = random.random()&#xA;        randomy._store[obj] = value&#xA;        return value&#xA;&#xA;randomy._store = {} # Initialize memoization map&#xA;&#xA;x = &quot;Test string&quot;&#xA;y = &quot;Another string&quot;&#xA;&#xA;print randomy(x)&#xA;print randomy(x)&#xA;print randomy(y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4230" LastActivityDate="2013-08-15T09:30:19.327" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8254" PostTypeId="2" ParentId="8251" CreationDate="2013-08-15T12:09:35.653" Score="2" Body="&lt;p&gt;If the data object you are trying to map is large enough, then just taking bit patterns is probably enough. For example, you might simply xor all the bytes of your object, giving you a number between 0 and 255 which you can then map onto the reals between 0 and 1.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-15T12:09:35.653" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="8255" PostTypeId="1" CreationDate="2013-08-15T14:53:34.737" Score="11" ViewCount="544" Body="&lt;p&gt;I am solving a non-linear system of coupled equations, and have calculated the Jacobian of the discretised system. The result is really complicated, below are (only!) the &lt;em&gt;first&lt;/em&gt; 3 columns of a &lt;span class=&quot;math-container&quot;&gt;$3\times 9$&lt;/span&gt; matrix,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/joPJe.png&quot; alt=&quot;Partial Jacobian matrix&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The complexity arises, in part, because the numerical scheme requires exponential fitting for stability.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have quite a general question about implementation of numerical codes using Jacobians. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can go ahead and implement this matrix in code. But my intuition is telling me to expect a few days (maybe weeks!) of tedious debugging due to the sheer complexity and the unavoidability of introducing errors. How does one cope with complexity such as this in numerical code, it seems unavoidable?! Do you use automatic &lt;em&gt;code-generation&lt;/em&gt; from symbolic packages (then tweak the code by hand)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I plan to debug the analytical Jacobian with a finite difference approximation, should I be aware of any pitfalls? How do you deal with similar problems in your code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am coding this in Python and used &lt;a href=&quot;http://sympy.org/en/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sympy&lt;/a&gt; to generate the Jacobian. Maybe I can used the &lt;a href=&quot;http://docs.sympy.org/dev/modules/utilities/codegen.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;code-generation&lt;/a&gt; feature?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="20688" LastEditDate="2019-06-22T20:11:47.443" LastActivityDate="2019-06-22T20:11:47.443" Title="How to deal with complexity in numerical code, for example, when dealing with large Jacobian matrices?" Tags="|symbolic-computation|jacobian|design-patterns|sympy|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8257" PostTypeId="2" ParentId="8255" CreationDate="2013-08-15T16:03:12.753" Score="7" Body="&lt;p&gt;One word: &lt;strong&gt;Modularity&lt;/strong&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of repeated expressions in your Jacobian that could be written as their own function.  There's no reason for you to write the same operation more than once, and that will make debugging easier; if you only write it once there's only one place for an error (in theory).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modular code will also make testing easier;  You can write tests for each component of your Jacobian as opposed to trying to test the whole matrix.  For example, if you write your function am() in a modular fashion, you can easily write sanity tests for it, check if you're differentiating it properly, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another suggestion would be to look at automatic differentiation libraries for assembling the Jacobian.  There's no guarantee they're error-free, but there will probably be less debugging/fewer errors than writing your own.  Here are a few you may want to look at it:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://trilinos.sandia.gov/packages/docs/dev/packages/sacado/doc/html/&quot;&gt;Sacado&lt;/a&gt; (Sandia Labs)&lt;/li&gt;&#xA;&lt;li&gt;ADIC (Argonne)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Sorry, just saw that you're using python.  ScientificPython has support for AD.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dirac.cnrs-orleans.fr/plone/software/scientificpython/&quot;&gt;ScientificPython&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4935" LastActivityDate="2013-08-15T16:03:12.753" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8258" PostTypeId="2" ParentId="8255" CreationDate="2013-08-15T17:01:10.613" Score="6" Body="&lt;p&gt;There are several strategies to consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Find the derivatives in symbolic form using a CAS, then export code for computing the derivatives.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use an automatic differentiation (AD) tool to produce code that computes the derivatives from code to compute the functions.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use finite difference approximations to approximate the Jacobian.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Automatic differentiation could produce more efficient code for computing the entire Jacobian then using symbolic computation to produce a formula for each entry in the matrix.  Finite differences are a good way to double check your derivatives.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2013-08-17T17:57:10.357" LastActivityDate="2013-08-17T17:57:10.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8259" PostTypeId="1" AcceptedAnswerId="8262" CreationDate="2013-08-15T17:19:40.657" Score="10" ViewCount="7973" Body="&lt;p&gt;Im trying to understand the difference between these two graphics cards for academic computing, specifically for the DGEMM component. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we look at the raw statistics, both have the same GK110 chip, have comparable statistics in virtually every category, and, I believe, have the same core architecture. Before any discounts the K20X is roughly 4x the cost of a Titan. From a efficiency perspective it seems to make much sense to use Titan's over the K20X's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having a difficult time understanding the difference here, can anyone illuminate the situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a note I am looking at purchasing these cards for a rack server and run at full tilt essentially until they die; however, I do not view the efficiency of using multiple GPU's for a single job to be particularly paramount.&lt;/p&gt;&#xA;" OwnerUserId="4947" LastEditorUserId="4947" LastEditDate="2013-08-15T18:43:54.803" LastActivityDate="2013-10-31T13:19:46.663" Title="Nvidia K20X vs GeForce Titan for GPGPU acceleration" Tags="|performance|gpu|efficiency|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8260" PostTypeId="2" ParentId="8090" CreationDate="2013-08-15T17:53:56.603" Score="1" Body="&lt;p&gt;Here's another attempt, which is more stable than the cancellation method but still not very good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is an SPD tridiagonal matrix, Meurant [1] gives the following stable formula for the entries of $B = A^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B_{ij} = b_{i+1}\cdots b_j \frac{d_{j+1}\cdots d_n}{\delta_i \cdots \delta_n}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $i \le j$, $b_i$ are the negative offdiagonal entries and $d_i,\delta_i$ are derived from $UL$ and $LU$ factorizations of $A$.  The linking formula for $i &amp;lt; j &amp;lt; k$ has the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_j = \left(\begin{array}{c}B_{ji} \\ B_{ki}\end{array}\right)^T \left(\begin{array}{cc}B_{ii} &amp;amp; B_{ik} \\ B_{ki} &amp;amp; B_{kk}\end{array}\right)^{-1} \left(\begin{array}{c}x_i \\ x_k\end{array}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this formula remains unstable.  Intuitively, if $i$ and $k$ are reasonably close a delta source at $i$ is similar to one at $k$, and the inverted $2 \times 2$ matrix is close to singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1]: Gerard Meurant (1992), &quot;A review on the inverse of symmetric diagonal and block tridiagonal matrices&quot;.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2013-08-15T18:01:30.583" LastActivityDate="2013-08-15T18:01:30.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8261" PostTypeId="2" ParentId="8090" CreationDate="2013-08-15T18:18:30.360" Score="2" Body="&lt;p&gt;I wonder if you could do something useful with a cyclic-reduction factorization of A (which I believe is still O(n) size), reusing as much of the blocks that would remain unchanged when factoring a contiguous principal submatrix of A. I doubt it gives you O(1), but maybe O(log n)...&lt;/p&gt;&#xA;" OwnerUserId="4979" LastActivityDate="2013-08-15T18:18:30.360" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8262" PostTypeId="2" ParentId="8259" CreationDate="2013-08-15T18:28:51.590" Score="13" Body="&lt;p&gt;There are some differences, however they aren't necessarily in hardware or specs.  Note that this is all information I have gained from forums or news releases, so take it all with a grain of salt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first is the &quot;scalability and reliability&quot; (&lt;a href=&quot;http://www.anandtech.com/show/6760/nvidias-geforce-gtx-titan-part-1/2&quot; rel=&quot;noreferrer&quot;&gt;source&lt;/a&gt;).  The K20 was designed to sit in a cluster system and run at full tilt 24/7.  The Titan is more designed for gaming, so it will run at this duty cycle, but it may suffer long term lifetime issues if used this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The drivers are also different, however I am not sure of the major differences.  The difference in focus of the cards' design likely leads to relatively small performance gains for the Tesla cards on this front.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Some Tesla-exclusive features include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;NVIDIA GPUDirect RDMA for InfiniBand performance&lt;/li&gt;&#xA;&lt;li&gt;Hyper-Q for MPI (Hyper-Q for CUDA Streams is supported on GeForce GTX TITAN)&lt;/li&gt;&#xA;&lt;li&gt;ECC protection for all internal and external registers and memories&lt;/li&gt;&#xA;&lt;li&gt;Supported tools for GPU and cluster management, such as Bright Computing, Ganglia.&quot; (&lt;a href=&quot;http://blogs.nvidia.com/blog/2013/03/13/geforce-gtx-titan-cuda/&quot; rel=&quot;noreferrer&quot;&gt;source&lt;/a&gt;) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This points to the fact that the main differnce is their scalability.  If you are looking to run on a desktop in your office, it would be hard to argue against a Titan over the K20 for the price difference.  If you need the extra performance of multiple K20's, find yourself a HPC center and buy time with their servers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After looking a bit more into ECC, I am updating this answer to point out the implications of having it on the K20 and not on the Titan.  The following information is a paraphrase of info found &lt;a href=&quot;http://www.realworldtech.com/fermi-ecc/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ECC is error checking on the DRAM and registers for the GPU.  Soft errors are when a bit is incorrectly transfered/stored.  The faster and closer together the circuits, the higher the probility of a soft error.  If you are solving a set of coupled ODE's or solving a linear system, a single number being off by one bit could significantly change the results in a non-reproducable way.  Most standard RAM and caches in the CPU are error checked for these errors using ECC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;GPU's on the other hand, do not, in general, have ECC even though their memory bus are much faster than those on the CPU.  This is because if a pixel on the screen is off by a bit for one frame, the quality of the program is not diminished.  These errors also don't propogate.  Therefore a lot of chip real estate (and cost) can be saved by skipping this feature.  This extra complexity likely causes a large portion of the extra cost of the Tesla line.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-10-31T13:19:46.663" LastActivityDate="2013-10-31T13:19:46.663" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8263" PostTypeId="2" ParentId="8259" CreationDate="2013-08-15T18:37:33.177" Score="2" Body="&lt;p&gt;In my opinion the difference seems to be mostly market segmentation.  If you are a scientist then NVidia wants you to be afraid that your paper will be rejected because you are using a GPGPU without as much error correcting RAM as would be available with K20X.  Similarly if you are a corporation then you might want to pay 4x if it means you are less likely for being sued on suspicion that your calculations are not as error-corrected as possible.  Individual gamers or hobbyist GPGPU'ers are sold Titan because they have less money and they are harder to persuade in these ways.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-08-15T18:37:33.177" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8264" PostTypeId="1" CreationDate="2013-08-15T16:53:44.543" Score="3" ViewCount="259" Body="&lt;p&gt;In the &lt;a href=&quot;http://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm&quot; rel=&quot;nofollow&quot;&gt;Metropolis algorithm&lt;/a&gt;, we accept or reject based on following criterion in the &lt;a href=&quot;http://en.wikipedia.org/wiki/Canonical_ensemble&quot; rel=&quot;nofollow&quot;&gt;$NVT$ ensemble&lt;/a&gt;:&#xA;$$&#xA;e^ {(E_{\text{new}} - E_{\text{old}})\,/ \,kT } \ge \text{random}(0,1).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does the uniformly distributed random numbers work here? How does it work?&lt;/p&gt;&#xA;" OwnerUserId="4023" OwnerDisplayName="cosmicraga" LastActivityDate="2013-08-15T20:18:51.257" Title="Use of random numbers in the Metropolis algorithm" Tags="|simulation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8265" PostTypeId="2" ParentId="8264" CreationDate="2013-08-15T18:21:24.857" Score="3" Body="&lt;p&gt;You stated the acceptance criterion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$e^{(E_\mathrm{new}-E_\mathrm{old})/kT} \geq \mathrm{random}(0,1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The left hand side expresses a ratio of likelihoods $\mathcal{L}_\mathrm{new}/\mathcal{L}_\mathrm{old}$ of a new state and an old state. If the new state is more likely than the old state, $\mathcal{L}_\mathrm{new}/\mathcal{L}_\mathrm{old} \gt 1$, so the acceptance criterion is automatically fulfilled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the new state is less likely than the old state, $0 \lt \mathcal{L}_\mathrm{new}/\mathcal{L}_\mathrm{old} \lt 1$. Now the acceptance criterion is fulfilled randomly. The more unlikely the new state (as compared to the old state), the more unlikely it is to be accepted, but there is always a finite probability of acceptance (well, up to machine precision anyway).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, this means that as the space of states is explored, typically the algorithm will move toward more likely states, but will occasionally move against the gradient of the likelihood. This is important if there are local maxima in the likelihood distribution so that the algorithm can &quot;get out&quot; of a local maximum to keep exploring other regions of the space of states. Without this feature, the algorithm would get stuck at the first little peak in the likelihood distribution that it encountered. It also allows proper exploration of the vicinity of a maximum, with the number of occupations of a state being proportional to the relative likelihood of the state (as the number of iterations approaches $\infty$).&lt;/p&gt;&#xA;" OwnerUserId="4985" OwnerDisplayName="Kyle" LastActivityDate="2013-08-15T18:21:24.857" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8266" PostTypeId="2" ParentId="2604" CreationDate="2013-08-15T22:57:56.957" Score="1" Body="&lt;p&gt;Here is a link to an old paper in J Comp Phys which computes the LE for coupled random dynamical systems &lt;a href=&quot;http://mesoscopic.mines.edu/acoustics-old/preprints/lyapunov.pdf&quot; rel=&quot;nofollow&quot;&gt;http://mesoscopic.mines.edu/acoustics-old/preprints/lyapunov.pdf&lt;/a&gt;  Perhaps you will find it more readable.  It's a stable, well-tested algorithm. The code was developed by my co-author who is now at KU (in case you want the code).  &lt;/p&gt;&#xA;" OwnerUserId="4881" LastActivityDate="2013-08-15T22:57:56.957" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8267" PostTypeId="2" ParentId="8255" CreationDate="2013-08-16T02:48:38.027" Score="3" Body="&lt;p&gt;Here is an example of where we have used automatic differentiation using Sacado in one code:&#xA;&lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_33.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_33.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-16T02:48:38.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8268" PostTypeId="1" CreationDate="2013-08-16T14:10:08.000" Score="3" ViewCount="226" Body="&lt;p&gt;I am trying to model the oscillation of a fluid miniscus in a straw when the miniscus is displaced from its equilibrium level. The results was the following non-linear ODE: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y''= 1/y - 1.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be solved numerically but I have not been able to find anyone who has worked on it.  Is anyone familiar with this equation?&lt;/p&gt;&#xA;" OwnerUserId="4991" OwnerDisplayName="user28321" LastEditorUserId="4244" LastEditDate="2013-10-18T07:37:36.400" LastActivityDate="2013-12-17T08:24:02.500" Title="Non-linear ordinary differential equation in the modeling of the oscillation of a meniscus" Tags="|fluid-dynamics|ode|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8269" PostTypeId="2" ParentId="8268" CreationDate="2013-08-17T02:55:39.230" Score="1" Body="&lt;p&gt;I'm not going to answer this question in complete detail, since it might be homework, but here are some tips on how to proceed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, decide if there's an exact solution available. If it's &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_differential_equation&quot; rel=&quot;nofollow&quot;&gt;linear&lt;/a&gt;, you have a good chance. The class of nonlinear ODE for which there is an exact solution can seem disappointingly small, but it's worth knowing the tricks (maybe it's &lt;a href=&quot;http://en.wikipedia.org/wiki/Exact_differential_equation&quot; rel=&quot;nofollow&quot;&gt;exact&lt;/a&gt;, maybe you can use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Frobenius_method&quot; rel=&quot;nofollow&quot;&gt;Frobenius method&lt;/a&gt;, perhaps there is a change of variables that simplifies things, etc.). If stumped, another tool  is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Computer_algebra_system&quot; rel=&quot;nofollow&quot;&gt;Computer Algebra System&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you decide there isn't an exact solution, you can choose a numerical method to compute an approximate solution. This depends on the boundary/initial conditions. Whether you have an &quot;initial value problem&quot; (information about $y$ and its derivatives at a single point in the domain) or a &quot;boundary value problem&quot; (information about $y$ and its derivatives at two separate points in the domain) makes a lot of difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the comment below, you are interested in an initial value problem, which for concreteness we'll say is&#xA;$ y''(t) = 1/y - 1$, $y(0) = y_0$,  $y'(0) = 0$.  Now one can choose freely from the bestiary of ODE solvers available. Note that if you can show that the solution stays bounded away from $y=0$, things will behave nicely. The form of this second order ODE makes it possible to use second-order methods like &lt;a href=&quot;http://en.wikipedia.org/wiki/Verlet_integration&quot; rel=&quot;nofollow&quot;&gt;Verlet methods&lt;/a&gt;, you can pick an old workhorse like &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta%E2%80%93Fehlberg_method&quot; rel=&quot;nofollow&quot;&gt;Runge-Kutta-Fehlberg&lt;/a&gt;(aka ode45 from MATLAB), or, if it seems likely that the solution will approach $y=0$, an implicit and/or stiff solver like a . Note that by introducing a new variable $v \doteq dy/dt$, you can write the second order ODE as a system of two first order ODEs.&lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2013-08-19T04:06:14.030" LastActivityDate="2013-08-19T04:06:14.030" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8270" PostTypeId="2" ParentId="8255" CreationDate="2013-08-17T13:05:30.820" Score="5" Body="&lt;p&gt;Let me weigh in here with a few words of caution, prefaced with a story. Long ago, I worked with a fellow when I was just starting out. He had an optimization problem to solve, with a rather messy objective. His solution was to generate the analytical derivatives for an optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem that I saw was these derivatives were nasty. Generated using Macsyma, converted to fortran code, they were each dozens of continuation statements long. In fact, the Fortran compiler got upset at that, as it exceeded the maximum number of continuation statements. While we found a flag that allowed us to get around that problem, there were other issues. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In long expressions, as are commonly generated by CA systems, there is a risk of massive subtractive cancellation. Compute lots of big numbers, only to find they all cancel each other out to yield a small number.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Often analytically generated derivatives are actually more costly to evaluate than are numerically generated derivatives using finite differences. A gradient for n variables may take more than n times the cost of evaluating your objective function. (You may be able to save some time because many of the terms can be re-used across the various derivatives, but that will also force you to do careful hand coding, instead of using computer generated expressions. And any time you code nasty mathematical expressions, the probability of an error is not trivial. Make sure you verify these derivatives for accuracy.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The point of my story is these CA generated expressions have issues of their own. The funny thing is my colleague was actually proud of the complexity of the problem, that he was clearly solving a really difficult problem because the algebra was so nasty. What I don't think he considered was if that algebra was actually computing the correct thing, was it doing so accurately, and was it doing so efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Had I been the senior person at the time on this project, I would have read him the riot act. His pride caused him to use a solution that was probably unnecessarily complex, without even checking that a finite difference based gradient was adequate. I'll bet we had spent perhaps a man-week of time to get this optimization running. At the very least, I would have counseled him to carefully test the gradient produced. Was it accurate? How accurate was it, compared to finite difference derivatives? In fact, there are tools around today that will also return an estimate of the error in their derivative prediction. This is certainly true for the &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/13490-adaptive-robust-numerical-differentiation&quot; rel=&quot;noreferrer&quot;&gt;adaptive differentiation code, (derivest)&lt;/a&gt; I've written in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Test the code. Verify the derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But before you do ANY of this, consider if other, better optimization schemes are an option. For example, if you are doing exponential fitting, then there is a very good chance you can use a partitioned nonlinear least squares (sometimes called separable least squares. I think that was the term used by Seber and Wild in their book.) The idea is to break the set of parameters into intrinsically linear and intrinsically nonlinear sets. Use an optimization that works on only the nonlinear parameters. Given those parameters are &quot;known&quot;, then the intrinsically linear parameters can be estimated using simple linear least squares. This scheme will reduce the parameter space in the optimization. It makes the problem more robust, since you don't need to find starting values for the linear parameters. It reduces the dimensionality of your search space, so making the problem run more quickly. Again I have supplied &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/10093-fminspleas&quot; rel=&quot;noreferrer&quot;&gt;a tool for this purpose&lt;/a&gt;, but only in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do use the analytical derivatives, code them to reuse terms. This can be a serious time savings, and may actually reduce the bugs, saving your own time. But then check those numbers!&lt;/p&gt;&#xA;" OwnerDisplayName="user840" LastActivityDate="2013-08-17T13:05:30.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8271" PostTypeId="2" ParentId="8255" CreationDate="2013-08-18T05:07:17.250" Score="2" Body="&lt;p&gt;In addition to BrianBorcher's excellent suggestions, another possible approach for real-valued functions is to use the complex-step derivative approximation (see &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=838251&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt; (paywalled) and &lt;a href=&quot;http://aero-comlab.stanford.edu/Papers/martins.aiaa.01-0921.pdf&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt;). In some cases, this approach yields more accurate numerical derivatives at the cost of changing the values of variables in your function from real to complex. The second article lists some cases where the complex step function approximation might break down.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-08-18T05:07:17.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8272" PostTypeId="1" CreationDate="2013-08-18T11:55:22.457" Score="1" ViewCount="296" Body="&lt;p&gt;I have tried matlab eig command, it results true eigenvalues but wrong eigenvectors.&#xA;I also tried direct iteration with rayleigh qotient which is better but doesn't give all eigenvectors&#xA;also I have tried  QR method which gives eigenvalues but not eigenvectors&#xA;is there any algorithm that i can use to find eigenvalues and corresponding eigenvectors?&lt;/p&gt;&#xA;" OwnerUserId="4990" LastEditorUserId="123" LastEditDate="2013-08-24T04:35:52.667" LastActivityDate="2013-08-24T13:18:28.547" Title="how can I numerically calculate all eigenvectors of a $n \times n$ complex tridiagonal matrix?" Tags="|eigensystem|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8273" PostTypeId="2" ParentId="8272" CreationDate="2013-08-18T14:36:21.927" Score="4" Body="&lt;p&gt;MATLAB's eig() function normalizes the eigenvectors to have unit norm; it doesn't normalize the eigenvalues.  You can check the norm of the vectors to verify their correctness.  Remember, if $\bf x$ is an eigenvector of $A$, then so is $c\,\bf x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can &quot;turn off&quot; the normalization by using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[v,d] = eig(A,'nobalance')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this also disables preconditioning and may give less accurate results.&lt;/p&gt;&#xA;" OwnerUserId="4935" LastActivityDate="2013-08-18T14:36:21.927" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8274" PostTypeId="1" AcceptedAnswerId="8275" CreationDate="2013-08-18T16:31:23.197" Score="5" ViewCount="229" Body="&lt;p&gt;I'm sure this is very important for anyone working in computational astrophysics. I have a &lt;em&gt;.sph&lt;/em&gt; file &amp;amp; it has 10 columns containing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x, y, z, density, temp., velx, vely, velz, smoothing length, particle mass. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Data is in some strange units but I can take care of that. But, I don't know about other details. Anyone who may have worked with SPH code probably will know how to extract physically sensible data out of it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a novice in computational astrophysics &amp;amp; computation as a whole but know the physical equations (MHD, Radiative transport, etc...) very well.&#xA;Any help would be highly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. : Please provide some novice &amp;amp; some advanced references.&lt;/p&gt;&#xA;" OwnerUserId="4992" LastEditorUserId="541" LastEditDate="2013-08-18T18:02:55.297" LastActivityDate="2013-08-18T18:02:55.297" Title="Mapping SPH data on Eulerian Grid for further evolution" Tags="|multiphysics|sph|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8275" PostTypeId="2" ParentId="8274" CreationDate="2013-08-18T17:34:55.890" Score="5" Body="&lt;p&gt;The data in the SPH format is perfectly &quot;physically sensible&quot;, you just need to know how to interpret it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the SPH formulation, you can compute any particle quantity $A$ at any given point using&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A(\mathbf r) = \sum_j m_j \frac{A_j}{\rho_j} W(\|\mathbf r-\mathbf r_j\|,h_j).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathbf r$ is the target point, e.g. a node on your grid, and $\mathbf r_j$ is the location of the $j$th particle, and $\rho_j$, $m_j$, and $h_j$ its density, mass, and smoothing length respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $W(r,h)$ is the &lt;em&gt;kernel function&lt;/em&gt; which is usually zero beyond $r&amp;gt;2h$. This means that for any target point $\mathbf r$, you only need to inspect particles where $\|\mathbf r - \mathbf r_j\| &amp;lt; 2h_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The target quantity $A$ can be the velocity, temperature, density, whatever is stored at the particle level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of this is described in the &lt;a href=&quot;http://en.wikipedia.org/wiki/Smoothed-particle_hydrodynamics&quot;&gt;Wikipedia article on SPH&lt;/a&gt;. A good source for further details is Price's excellent review of SPH available &lt;a href=&quot;http://arxiv.org/abs/1012.1885&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if you're mapping the particle data to a regular grid, it makes much more sense to loop over the particles, not the grid points, as finding the grid points within range of a particle is much easier than finding the particles within range of a grid point.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2013-08-18T18:00:20.457" LastActivityDate="2013-08-18T18:00:20.457" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8276" PostTypeId="1" CreationDate="2013-08-19T04:06:04.860" Score="6" ViewCount="118" Body="&lt;p&gt;There are hundreds of different types of performance measures for numerical models, many of which are applicable to many different types of models. But a good model doesn't just perform well, it performs well while being as simple as possible, and no simpler. Are there any general-ish measures of a model's parsimony?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some fields you can build in some kind of ensured parsimony, like in evolutionary neural-network model generation, where you can punish complexity. I'm thinking more along the lines of physics-based models, where the model is to some extent defined by theory.&lt;/p&gt;&#xA;" OwnerUserId="1232" LastActivityDate="2013-08-20T04:51:37.783" Title="Measures of parsimony for numerical models?" Tags="|performance|complexity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8278" PostTypeId="2" ParentId="8276" CreationDate="2013-08-19T13:00:43.013" Score="4" Body="&lt;p&gt;Not entirely sure this is applicable to your situation, but I think it may be. Statisticians use something called an information criterion for purposes very similar to this. It captures the trade off between good fit of the observations (in particular, the likelihood of the observations given the model and a setting of the parameters) and the complexity of the model (in particular, the number of fitted parameters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several possible choices for this trade off, and therefore several different criteria. I think &lt;a href=&quot;https://en.wikipedia.org/wiki/Akaike_information_criterion&quot; rel=&quot;nofollow&quot;&gt;Akaike's information criterion&lt;/a&gt; (with or without sample size correction) and &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_information_criterion&quot; rel=&quot;nofollow&quot;&gt;Bayes' information criterion&lt;/a&gt; are used most. Wikipedia has formulas and more background info.&lt;/p&gt;&#xA;" OwnerUserId="49" LastEditorUserId="1232" LastEditDate="2013-08-20T04:51:37.783" LastActivityDate="2013-08-20T04:51:37.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8279" PostTypeId="1" AcceptedAnswerId="8280" CreationDate="2013-08-18T22:44:25.607" Score="1" ViewCount="39" Body="&lt;p&gt;This question may be better suited for an Astronomy Stack Exchange site, but I figured I'd ask here. Say I have measurements of something as a function of radius for a number of objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example to make things concrete:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Object 1&#xA;In [92]: r1&#xA;Out[92]: &#xA;array([ 0.05      ,  0.07040816,  0.09081633,  0.11122449,  0.13163265,&#xA;    0.15204082,  0.17244898,  0.19285714,  0.21326531,  0.23367347,&#xA;    0.25408163,  0.2744898 ,  0.29489796,  0.31530612,  0.33571429,&#xA;    0.35612245,  0.37653061,  0.39693878,  0.41734694,  0.4377551 ,&#xA;    0.45816327,  0.47857143,  0.49897959,  0.51938776,  0.53979592,&#xA;    0.56020408,  0.58061224,  0.60102041,  0.62142857,  0.64183673,&#xA;    0.6622449 ,  0.68265306,  0.70306122,  0.72346939,  0.74387755,&#xA;    0.76428571,  0.78469388,  0.80510204,  0.8255102 ,  0.84591837,&#xA;    0.86632653,  0.88673469,  0.90714286,  0.92755102,  0.94795918,&#xA;    0.96836735,  0.98877551,  1.00918367,  1.02959184,  1.05      ])&#xA;# Object 2&#xA;In [93]: r2&#xA;Out[93]: &#xA;array([ 0.04      ,  0.06081633,  0.08163265,  0.10244898,  0.12326531,&#xA;    0.14408163,  0.16489796,  0.18571429,  0.20653061,  0.22734694,&#xA;    0.24816327,  0.26897959,  0.28979592,  0.31061224,  0.33142857,&#xA;    0.3522449 ,  0.37306122,  0.39387755,  0.41469388,  0.4355102 ,&#xA;    0.45632653,  0.47714286,  0.49795918,  0.51877551,  0.53959184,&#xA;    0.56040816,  0.58122449,  0.60204082,  0.62285714,  0.64367347,&#xA;    0.6644898 ,  0.68530612,  0.70612245,  0.72693878,  0.7477551 ,&#xA;    0.76857143,  0.78938776,  0.81020408,  0.83102041,  0.85183673,&#xA;    0.87265306,  0.89346939,  0.91428571,  0.93510204,  0.95591837,&#xA;    0.97673469,  0.99755102,  1.01836735,  1.03918367,  1.06      ])&#xA;# Object 3&#xA;In [94]: r3&#xA;Out[94]: &#xA;array([ 0.045     ,  0.06540816,  0.08581633,  0.10622449,  0.12663265,&#xA;    0.14704082,  0.16744898,  0.18785714,  0.20826531,  0.22867347,&#xA;    0.24908163,  0.2694898 ,  0.28989796,  0.31030612,  0.33071429,&#xA;    0.35112245,  0.37153061,  0.39193878,  0.41234694,  0.4327551 ,&#xA;    0.45316327,  0.47357143,  0.49397959,  0.51438776,  0.53479592,&#xA;    0.55520408,  0.57561224,  0.59602041,  0.61642857,  0.63683673,&#xA;    0.6572449 ,  0.67765306,  0.69806122,  0.71846939,  0.73887755,&#xA;    0.75928571,  0.77969388,  0.80010204,  0.8205102 ,  0.84091837,&#xA;    0.86132653,  0.88173469,  0.90214286,  0.92255102,  0.94295918,&#xA;    0.96336735,  0.98377551,  1.00418367,  1.02459184,  1.045     ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where r1, r2, and r3 are the radial lists which I make measurements for (Object 1, 2, and 3, respectively). The measurements look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [95]: out1&#xA;Out[95]: &#xA;array([ 0.39919261,  0.40078876,  0.40335025,  0.40702841,  0.41194375,&#xA;    0.41818517,  0.42580846,  0.43483499,  0.44525116,  0.45700891,&#xA;    0.4700276 ,  0.48419736,  0.49938363,  0.51543283,  0.53217868,&#xA;    0.54944865,  0.56707028,  0.58487678,  0.60271173,  0.62043267,&#xA;    0.63791355,  0.65504607,  0.67174007,  0.6879231 ,  0.70353938,&#xA;    0.71854842,  0.73292329,  0.74664884,  0.75971994,  0.77213972,&#xA;    0.783918  ,  0.79506992,  0.80561461,  0.81557416,  0.82497273,&#xA;    0.83383575,  0.84218935,  0.85005984,  0.85747334,  0.86445547,&#xA;    0.87103113,  0.87722433,  0.88305811,  0.88855443,  0.89373415,&#xA;    0.898617  ,  0.90322162,  0.90756555,  0.91166525,  0.91553618])&#xA;In [96]: out2&#xA;Out[96]: &#xA;array([ 0.31626381,  0.31638513,  0.31667084,  0.31721226,  0.31811294,&#xA;    0.31948646,  0.32145416,  0.32414259,  0.32768059,  0.33219587,&#xA;    0.33781106,  0.34463926,  0.35277915,  0.36230989,  0.37328607,&#xA;    0.38573305,  0.39964318,  0.41497335,  0.43164433,  0.449542  ,&#xA;    0.46852083,  0.48840914,  0.50901598,  0.53013886,  0.55157189,&#xA;    0.57311338,  0.59457264,  0.61577537,  0.63656761,  0.65681803,&#xA;    0.6764189 ,  0.69528578,  0.71335626,  0.73058811,  0.74695691,&#xA;    0.76245356,  0.77708177,  0.79085565,  0.80379747,  0.81593569,&#xA;    0.82730323,  0.83793606,  0.84787193,  0.85714946,  0.86580729,&#xA;    0.87388357,  0.88141541,  0.88843862,  0.89498747,  0.90109452])&#xA;In [97]: out3&#xA;Out[97]: &#xA;array([ 0.6325231 ,  0.63424383,  0.63647521,  0.63916579,  0.64227168,&#xA;    0.64575317,  0.64957324,  0.65369686,  0.65809065,  0.6627228 ,&#xA;    0.66756296,  0.67258231,  0.67775356,  0.68305097,  0.68845041,&#xA;    0.69392934,  0.69946681,  0.70504349,  0.71064159,  0.71624488,&#xA;    0.72183859,  0.72740939,  0.7329453 ,  0.73843566,  0.74387099,&#xA;    0.749243  ,  0.75454444,  0.75976907,  0.76491154,  0.76996738,&#xA;    0.77493287,  0.77980499,  0.78458137,  0.78926022,  0.79384025,&#xA;    0.79832065,  0.80270103,  0.80698134,  0.81116187,  0.81524321,&#xA;    0.81922618,  0.82311181,  0.82690134,  0.83059613,  0.83419771,&#xA;    0.83770771,  0.84112783,  0.84445988,  0.84770571,  0.8508672 ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sorry for all of the output. Anyway, the question is, what is the proper technique for &quot;adding&quot; these curves? I'm looking for an average measurement vs. radius curve (with error bars), and I'm not entirely sure what that means for different radial measurements of different systems. My first idea was to interpolate between values of object 1, and evaluate them at the location of object 2 and 3, so that they can be evaluated on the same footing. But, I have potentially thousands of these objects, and that doesn't like the right thing to do for that many (plus that also brings up my interpolation method - cubic, linear, etc..). Any ideas would be appreciated! Let me know if further clarification is needed. Thanks! &lt;/p&gt;&#xA;" OwnerUserId="5000" OwnerDisplayName="astromax" LastEditorUserId="123" LastEditDate="2013-08-24T04:34:20.147" LastActivityDate="2013-08-24T04:34:20.147" Title="Adding data with different abscissas" Tags="|interpolation|data-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8280" PostTypeId="2" ParentId="8279" CreationDate="2013-08-19T01:30:22.817" Score="1" Body="&lt;p&gt;Yes, interpolation is usually your best bet. There are three main methods to add them together, but they only fit in some context.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Add interpolated function together&lt;/strong&gt;: Assume you can fit the results as a continuous $f_i(r_i)$, then you can get the average $f(r) = \frac{1}{n}\sum_i^n f_i(r)$. But using this method, you must assume that the original function $f_i$ are smooth (and the error are small) so that you can do interpolation. The error can be found in the similar way. By glimpsing through your data, I think it is the case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Combine all data points&lt;/strong&gt;: Merge the x data points together, and y data points together, i.e. $union({r_i})$ and $union({f_i})$. It works if they are actually the outcome of the same object/ same ensemble, with large errors. Since they represent the same think, you don't really need to take the average and it works well for very small number of samples. In addition, you can add a simple fitting curve for these high noise data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Use binned average&lt;/strong&gt;: In case you have large number of data series (you mentioned), a simple way to do so is to use binned average. The bin can be, say, $r=[0.05, 0.15, ..., 0.95]$ and then you put the corresponding data points in each bin. For each bin, you can then take the average and variance of data points. If you have large number of data points and they belong to the same ensemble, this usually give you the best smooth results and have good error estimate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the limited information you give, I would recommend you to use the last method.&lt;/p&gt;&#xA;" OwnerUserId="3444" OwnerDisplayName="hwlau" LastActivityDate="2013-08-19T01:30:22.817" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8281" PostTypeId="2" ParentId="8250" CreationDate="2013-08-19T19:53:09.037" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://www.clusterresources.com/torquedocs21/2.3cancelingjobs.shtml&quot; rel=&quot;nofollow&quot;&gt;http://www.clusterresources.com/torquedocs21/2.3cancelingjobs.shtml&lt;/a&gt; says that all processes associated with the job are killed.  I believe that is true for all schedulers since it is clearly the desired behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Torque will not kill the job or the child processes when the original process exists, as this would prevent a job from launch one program after another in sequence, for example, since this workflow would cause the job to terminate after the first program exited if Torque initiated cleanup and job termination upon that event.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can, of course, verify any of this experimentally using the Torque system you intend to use.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2013-08-22T19:59:50.153" LastActivityDate="2013-08-22T19:59:50.153" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8283" PostTypeId="2" ParentId="5610" CreationDate="2013-08-20T02:32:37.327" Score="1" Body="&lt;p&gt;It looks like you are looking for the functions &lt;a href=&quot;http://linux.die.net/man/3/fmod&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;fmod&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;http://linux.die.net/man/3/remainder&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;remainder&lt;/code&gt;&lt;/a&gt;, or perhaps &lt;a href=&quot;http://linux.die.net/man/3/remquo&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;remquo&lt;/code&gt;&lt;/a&gt; if you're willing to accept a non-portable maximum number of intervals.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-08-20T02:32:37.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8286" PostTypeId="2" ParentId="5093" CreationDate="2013-08-20T17:44:32.400" Score="0" Body="&lt;p&gt;The transformation recommended by Pedro is a great idea. Have you tried to play around with the parameters in Matlab's &quot;quadgk&quot; function? For example, using Pedro's transformation, you can do the following:&lt;br/&gt;&#xA;&lt;code&gt;quadgk(f, 0.0+eps, 1.0-eps, 'AbsTol', eps, 'MaxIntervalCount', 100000)&lt;/code&gt;&lt;br/&gt;&#xA;Using this gives me a solution of:&lt;br/&gt;&lt;code&gt;-2184689.50220729&lt;/code&gt;&lt;br/&gt;&#xA;and only takes 0.8 seconds (using the values mentioned above: x=10)&lt;br/&gt;&#xA;Walter Gander and Walter Gautschi have a paper on adaptive quadrature with Matlab code you can use as well (link &lt;a href=&quot;http://users.wpi.edu/~walker/MA510/HANDOUTS/w.gander,w.gautschi,Adaptive_Quadrature,BIT_40,2000,84-101.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="5006" LastActivityDate="2013-08-20T17:44:32.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8287" PostTypeId="2" ParentId="8193" CreationDate="2013-08-20T18:12:17.320" Score="3" Body="&lt;p&gt;There is some good information here:&lt;br/&gt;&#xA;&lt;a href=&quot;https://software.intel.com/en-us/mkl-linux-developer-guide-calling-intel-mkl-functions-from-multi-threaded-applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.intel.com/en-us/mkl-linux-developer-guide-calling-intel-mkl-functions-from-multi-threaded-applications&lt;/a&gt;&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;set MKL_NUM_THREADS=n&lt;/li&gt;&#xA;&lt;li&gt;call mkl_set_num_threads(n)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5006" LastEditorUserId="25209" LastEditDate="2017-08-23T06:04:05.963" LastActivityDate="2017-08-23T06:04:05.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8288" PostTypeId="1" AcceptedAnswerId="8513" CreationDate="2013-08-20T20:43:31.120" Score="4" ViewCount="2397" Body="&lt;p&gt;I am trying to compare my finite difference's solution of the scalar (or simple acoustic) wave equation with an analytic solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that purpose I am using the following analytic solution presented in the old paper &lt;a href=&quot;https://drive.google.com/#folders/0B-AAXv24JyOLSVdfb3RtSlR5Wkk&quot; rel=&quot;nofollow&quot;&gt;Accuracy of the finite-difference modeling of the acoustic wave equation - Geophysics 1974 - R. M. Alford et. al&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution, equation (9) in that paper, for a constant velocity medium is given in cylindrical coordinates by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;u_s(\rho, \phi, t) = \frac{1}{2\pi} \int_{-\infty}^{\infty} \{ -i \pi H_0^{(2)}\left(k | \sigma - \sigma_s| \right) F(\omega) e^{i\omega t} d\omega\}&#xA;\end{eqnarray} &lt;strong&gt;(A)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H_0^{(2)}$ is the Hankel function of the second kind with order zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$| \sigma - \sigma_s| $ is the distance from the source to the observation point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also $ k = \frac{\omega}{C_0}$ come from $C_0$ the medium velocity converted to the Fourier domain, defined in the original scalar wave equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\left[ \nabla^2 - \frac{1}{C_0^2}\frac{\partial ^2}{\partial t^2}\right] u(\rho, \phi, t) = -4 \pi \frac{\delta(\rho-\rho_s)\delta(\phi-\phi_s)f(t)}{\rho}&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ (\rho_s, \phi_s) $ defines the source location in the $\phi$ x $\rho$ plane. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ F(\omega)$ comes from the source function $f(t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When implementing numerically &lt;strong&gt;A&lt;/strong&gt; I'm not getting meaningful&#xA;results. The velocity obtained using &lt;strong&gt;A&lt;/strong&gt; is totally different from the $C_0$ used as input&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My code example is bellow where I am evaluating &lt;strong&gt;A&lt;/strong&gt;, using python.&#xA;I calculate the solution at zero &lt;code&gt;solution0&lt;/code&gt; and at a distance D=8000 &lt;code&gt;solutionD&lt;/code&gt;. Then I use the global maximum to calculate an approximation to velocity $C_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably it's a quite simple error that I am doing. If someone can point it out for me.&#xA;The velocity I am getting shows that maybe the equation (9) is wrong or I am implementing it wrongly (what I believe to be the case).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.special import hankel2&#xA;import numpy as np&#xA;&#xA;# a simple source function f(t) gauss derivative normalized&#xA;def gaussource(time, wlength, delay=None):&#xA;        if delay == None: # enough delay time&#xA;            delay = 3*wlength&#xA;        t = time - delay&#xA;        return ((2*np.sqrt(np.e)/(wlength))&#xA;               *t*np.exp(-2*(t**2)/(wlength**2)))&#xA;&#xA;# analytic solution equation (A) paper&#xA;def waves(source, distance, c):&#xA;    n = len(source)&#xA;    sourcew = np.fft.fft(source) # source in the frequency domain&#xA;    hnk_factor = distance/c # distance to source divided by velocity&#xA;    if not distance == 0.0:&#xA;        hankelshift = np.complex(0,-np.pi)*np.array([hankel2(0,hnk_factor*omega) for omega in xrange(n)])&#xA;        hankelshift[0] = 0. # i infinity in limit&#xA;        return np.real(np.fft.ifft(hankelshift*sourcew))&#xA;    return source&#xA;&#xA;c = 4000.&#xA;fc = 5.0&#xA;dt = 0.01&#xA;t = np.arange(0., 2., dt) # 2 seconds to evaluate the solution&#xA;source = gaussource(t, 1./fc) # source&#xA;solution0 = waves(source, 0., c) # solution at zero source (source itself)&#xA;solutionD = waves(source, 8000., c) #solution at certain distance D=8000 metres e.g.   &#xA;t0 = solution0.argmax(); # get max index time at zero&#xA;tD = solutionD.argmax(); # get max index time at distance&#xA;print &quot;phase velocity aproximation &quot;, 8000./((tD-t0)*dt) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4092" LastEditorUserId="5006" LastEditDate="2013-08-22T19:34:18.220" LastActivityDate="2013-09-17T15:41:50.197" Title="Analytic solution 2D scalar wave equation in cylindrical coordinates numerical implementation" Tags="|python|numerical-analysis|numpy|wave-propagation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8289" PostTypeId="1" AcceptedAnswerId="8294" CreationDate="2013-08-21T04:56:03.803" Score="1" ViewCount="116" Body="&lt;p&gt;There's a computational code I'm working with which I'm told requires OpenCL 1.1 or later. I found out I'm saddled with OpenCL 1.0 on my Macbook Pro using some helpful &lt;a href=&quot;http://dhruba.name/2012/08/13/opencl-cookbook-listing-all-platforms-and-their-attributes/&quot; rel=&quot;nofollow&quot;&gt;code snippets&lt;/a&gt;, and Google has been remarkably opaque on how to update versions of OpenCL on OS X. The above code returns that my machine sports the following devices&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;Device: GeForce 9400M&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1.1 Hardware version: OpenCL 1.0 &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1.2 Software version: CLH 1.0&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1.4 Parallel compute units: 2&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;Device: Intel(R) Core(TM)2 Duo CPU     P7550  @ 2.26GHz&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;2.1 Hardware version: OpenCL 1.0 &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;2.2 Software version: 1.0&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;2.4 Parallel compute units: 2&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Has anyone been able to either upgrade OpenCL for Mac, or can anyone explain what might limit (hardware, OS, etc) this computer from supporting OpenCL?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks. Please let me know if there's a better place to post this; the connection to computational science seems a bit more tenuous with this question. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-08-21T18:36:32.367" Title="OpenCL version constraints?" Tags="|gpu|opencl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8290" PostTypeId="1" CreationDate="2013-08-21T06:47:26.403" Score="5" ViewCount="440" Body="&lt;p&gt;I am a postgraduate student with interest in numerical methods for solving nonlinear systems of equations. I have read some papers that discussed about 'computational order of convergence' for some iterative methods for example &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0377042712005249&quot;&gt;Increasing the order of convergence of iterative schemes for solving nonlinear systems&lt;/a&gt;, but still I can't explain it or implement it practically using MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, can somebody help me with a good reference to understand how to compute the computational order of convergence for my proposed iterative method? &lt;/p&gt;&#xA;" OwnerUserId="5009" LastActivityDate="2013-08-21T06:47:26.403" Title="Understanding the meaning of Computational Order of Convergence" Tags="|iterative-method|convergence|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8291" PostTypeId="1" CreationDate="2013-08-20T10:31:16.793" Score="5" ViewCount="1891" Body="&lt;p&gt;Below a piece of code is shown to solve a linear system.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ierr = KSPCreate(PETSC_COMM_WORLD,&amp;amp;ksp); CHKERRQ(ierr);&#xA;ierr = KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN); CHKERRQ(ierr);&#xA;KSPSetType(ksp,KSPBCGS);&#xA;ierr = KSPSetTolerances(ksp,1.e-2/((m+1)*(n+1)),1.e-50,PETSC_DEFAULT, PETSC_DEFAULT); CHKERRQ(ierr);&#xA;ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr);&#xA;ierr = PCSetType(pc,PCSOR);CHKERRQ(ierr);&#xA;ierr = KSPSolve(ksp,b,x);CHKERRQ(ierr);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;According to the PETSC manual, PETSC uses GMRES with an ILU(0) preconditioner to solve SLE. I want to modify code to solve system using the BiCGStab method with a SSOR preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help to do that?&lt;/p&gt;&#xA;" OwnerUserId="3544" OwnerDisplayName="Nurlan Kenzhebekov" LastEditorUserId="210" LastEditDate="2013-08-21T09:02:01.837" LastActivityDate="2013-08-23T12:50:44.783" Title="Solve a linear system with BiCGStab instead of GMRES in PETSC" Tags="|linear-algebra|petsc|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8292" PostTypeId="2" ParentId="8291" CreationDate="2013-08-21T13:50:19.573" Score="4" Body="&lt;p&gt;The PETSc team always recommends that their users control solver options from the command line. The whole package is built with the idea of extreme flexibility in composing solvers and preconditioners, and the only way to achieve that is to use the command line scheme. You could get rid of setting the KSP and PC types in your code and use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ierr = KSPCreate(PETSC_COMM_WORLD,&amp;amp;ksp);&#xA;ierr = KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN);&#xA;ierr = KSPSetTolerances(ksp,1.e-2/((m+1)*(n+1)),1.e-50,PETSC_DEFAULT, &#xA;                        PETSC_DEFAULT);&#xA;ierr = KSPSetFromOptions(ksp);&#xA;ierr = KSPSolve(ksp,b,x);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and run with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-ksp_type bcgsl -pc_type sor -pc_sor_its 10 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;KSPSetFromOptions(ksp)&lt;/code&gt; will override the default options with the options you have supplied here. So, instead of GMRES with ILU, it will use BiCGStab(l) with SSOR. Additional options to control SSOR can be found on the PETSc man pages:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCSOR.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCSOR.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you insist on setting the options from code, use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSetType.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;KSPSetType&lt;/code&gt;&lt;/a&gt;&lt;code&gt;(ksp,&lt;/code&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPBCGS.html&quot; rel=&quot;nofollow&quot;&gt;KSPBCGS&lt;/a&gt;&lt;code&gt;)&lt;/code&gt;.  Almost all PETSc run-time options have an equivalent functional interface, but it is only recommended when your application is making &lt;em&gt;active&lt;/em&gt; decisions about which method to use.&lt;/p&gt;&#xA;" OwnerUserId="1453" LastEditorUserId="119" LastEditDate="2013-08-23T12:50:44.783" LastActivityDate="2013-08-23T12:50:44.783" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8293" PostTypeId="2" ParentId="8291" CreationDate="2013-08-21T18:12:25.103" Score="4" Body="&lt;p&gt;You can either do it from the code by doing this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;KSP  my_solver ;  // define the KSP method&#xA;PC   my_prec   ;  // define the preconditioner &#xA;&#xA;/* Define the matrices to be used ... */ &#xA;&#xA;KSPCreate(comm,&amp;amp;my_solver);&#xA;KSPSetType(my_solver,KSPBCGS);  // Sets BiCGStab as the krylov method&#xA;&#xA;KSPGetPC(my_solver,&amp;amp;my_prec);&#xA;PCSetType(my_prec,PCSOR);     // Sets SSOR as the preconditioner method&#xA;&#xA;/* then continue with the SetType and Solve and other stuff ... */&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or by initializing the KSP method and Preconditioner as done before, then setting them using the command line by adding:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;KSPSetFromOptions(my_solver);&#xA;KSPSetUp(my_solver);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The second way is more flexible of course, as you can specify your KSP method and PC regardless of what the code defines. Check this link to know more about how to set your values from the database&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSetFromOptions.html#KSPSetFromOptions&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPSetFromOptions.html#KSPSetFromOptions&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2013-08-22T03:09:17.520" LastActivityDate="2013-08-22T03:09:17.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8294" PostTypeId="2" ParentId="8289" CreationDate="2013-08-21T18:22:38.633" Score="3" Body="&lt;p&gt;Your GeForce 9400M video card is older and probably doesn't support the later OpenCL 1.1 spec.  Unfortunately, this is a hardware limitation that you can't fix with software- you'll need to get a newer computer if you want to run this software using OpenCL 1.1.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, I believe that OpenCL 1.1 support is new in Mac OS X 10.7 but you also need compatible hardware and I don't think that the 9400M will cut it.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2013-08-21T18:36:32.367" LastActivityDate="2013-08-21T18:36:32.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8295" PostTypeId="1" CreationDate="2013-08-22T07:08:12.860" Score="3" ViewCount="283" Body="&lt;p&gt;I am trying to implement exemplar based in-painting, as proposed by &lt;a href=&quot;http://research.microsoft.com/pubs/67276/criminisi_tip2004.pdf&quot; rel=&quot;nofollow&quot;&gt;Criminsi&lt;/a&gt; et. al. in its original format (before moving into further optimisations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few theoretical doubts which I would like to clarify. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how the patch priority and the determination of fill-order works to propagate linear structures into the user-selected target region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand, Criminsi suggests the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine the fill front dΩ.&lt;/li&gt;&#xA;&lt;li&gt;Compute the patch priorities [C(p)D(p)] on patches lying on the fill&#xA;front.&lt;/li&gt;&#xA;&lt;li&gt;Propagate texture into the patches - in order of their patch&#xA;priorities.&lt;/li&gt;&#xA;&lt;li&gt;Update confidence values.&lt;/li&gt;&#xA;&lt;li&gt;Re-compute fill front dΩ from the remaining unfilled section.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[Repeat these 5 steps till all of the target region is filled].&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if this understanding of the algorithm is correct (and, please do correct me if the interpretation is wrong), I have the following doubt:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I'm always filling patches from the fill front only, and if the&#xA;patches are fixed in their spatial location (w.r.t other patches on&#xA;the fill-front), then I don't see how the order of filling-in&#xA;matters. The patches are filled in by direct sampling of the source&#xA;region, which stays constant for all iterations. Hence, I do not&#xA;understand how the order of filling will result in any pixel level&#xA;difference on the propagated textures. From what I gather, the best match from the source region should remain unaltered, irrespective of the order. What am I missing here?&lt;/li&gt;&#xA;&lt;li&gt;Based on this &lt;a href=&quot;http://www.youtube.com/watch?v=UYk9_fTZbAs&quot; rel=&quot;nofollow&quot;&gt;video&lt;/a&gt; (and assuming it is correct), the patch&#xA;filling does not always occur on the fill-front only. How does this&#xA;work?  Does this mean we are to compute the priorities for all patches in the target region at once and go on filling them based on their priorities? Then, how does the fill-front determination matter and why does Criminsi determine dΩ at all? I understand Criminsi's approach was a step ahead of the traditional onion-peel technique. But how, exactly?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I really wish to clarify these questions before I delve into 2-3 days of intense coding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="4773" LastActivityDate="2013-08-22T07:08:12.860" Title="Patch priority and its effect on Criminsi's Exemplar Based Inpainting" Tags="|image-processing|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8296" PostTypeId="2" ParentId="7993" CreationDate="2013-08-22T07:51:17.170" Score="2" Body="&lt;p&gt;If you are familiar with LaTeX (and I can only recommend it to use it to write your bachelor thesis), you can try out &lt;a href=&quot;http://sourceforge.net/projects/pgf/&quot; rel=&quot;nofollow&quot;&gt;PGF and TikZ&lt;/a&gt;. You can find an example for energy level diagrams in TikZ &lt;a href=&quot;http://www.texample.net/tikz/examples/energy-level-diagram/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-08-22T07:51:17.170" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8297" PostTypeId="1" AcceptedAnswerId="8299" CreationDate="2013-08-22T08:04:37.117" Score="7" ViewCount="2902" Body="&lt;p&gt;I will be blunt: this is an old exam question of a course that covers Molecular Dynamics (MD) and Monte Carlo (MC) simulations.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Explain how to determine the severity of finite-size effects.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My idea: Since we are talking about MD and MC, finite-size effects are related to the fact that we can only simulate systems of finite sizes. Contrarily, in nature systems usually approach very large sizes, i.e. they are in the thermodynamic limit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found these &lt;a href=&quot;http://www.helsinki.fi/~rummukai/simu/fss.pdf&quot; rel=&quot;nofollow&quot;&gt;lecture slides&lt;/a&gt; which gives me some ideas, but I am not sure that this is what is being asked in the question above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts, and pointers to slides / scripts are very welcome. &lt;/p&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="3748" LastEditDate="2013-08-23T14:17:42.327" LastActivityDate="2014-03-03T14:24:03.277" Title="Finite-size effects in molecular dynamics simulation" Tags="|simulation|molecular-dynamics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8298" PostTypeId="1" CreationDate="2013-08-22T09:03:03.657" Score="4" ViewCount="1126" Body="&lt;p&gt;I am looking for simple &lt;a href=&quot;http://en.wikipedia.org/wiki/Generalized_minimal_residual_method&quot; rel=&quot;nofollow&quot;&gt;GMRES&lt;/a&gt; implementations. Unfortunately, I have a few requirements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;should be C or Fortran, Fortran preferred&lt;/li&gt;&#xA;&lt;li&gt;needs a license that permits inclusion in an MIT/BSD licensed package&lt;/li&gt;&#xA;&lt;li&gt;needs to deal with complex data&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So far, the &quot;&lt;a href=&quot;http://www.netlib.org/templates/&quot; rel=&quot;nofollow&quot;&gt;templates&lt;/a&gt;&quot; code on netlib seemed to fit well. However, it doesn't deal with complex data (although I think changing a few declarations from real to complex could fix that), and also it obviously implements operations better left to BLAS (I think e.g. &lt;a href=&quot;http://www.netlib.org/templates/double//GMRESREVCOM.f&quot; rel=&quot;nofollow&quot;&gt;this code's&lt;/a&gt; Givens rotations that'd probably better be &lt;code&gt;drot()&lt;/code&gt; calls). Another alternative, PETSc, seems overkill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking forward to your suggestions. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="1553" LastActivityDate="2013-08-22T09:03:03.657" Title="GMRES implementations" Tags="|gmres|libraries|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8299" PostTypeId="2" ParentId="8297" CreationDate="2013-08-22T13:57:49.170" Score="3" Body="&lt;p&gt;Periodic boundary conditions (PBC) are a common technique to get closer to the thermodynamic limit while keeping the computation feasible.   You could set up an experiment simulating a box of argon with and without PBC and study how the radial distribution function g(r) changes in each case, you should observe a difference in g(r) for large values of r.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach could consist of revisiting the derivation of the  canonical ensemble (or other ensembles).  One way this derivation is typically done is by truncating a Taylor expansion where the remainder is a function of the number of particles N that goes to zero as N goes to infinity.  You could consider N fixed and quantify the error of the approximation that way (see, for example, the derivation of equation 2.23 in &lt;a href=&quot;http://dx.doi.org/10.1088/0034-4885/42/12/002&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-08-22T13:57:49.170" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8300" PostTypeId="1" CreationDate="2013-08-22T15:50:31.807" Score="2" ViewCount="90" Body="&lt;p&gt;There are many variants of non-linear conjugate gradient method available ( Flatcher-Reeves, Polak-Rebiere, Dai-Yuan). In case of minimization of quadratic function when first search direction is chosen to be negative of gradient then all of these algorithms are  equivalent and possess finite termination property. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in case where first search direction is arbitrary (not equal to steepest descent direction), all these algorithms are not equivalent anymore. Moreover, their convergence properties varies as well. I am interested to know what happens to finite termination propety in this case? In particular  which non linear CG still possess finite termination property under this condition.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2013-08-22T15:50:31.807" Title="Which non-linear conjugate gradient possess finite termination property" Tags="|optimization|nonlinear-programming|conjugate-gradient|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8301" PostTypeId="2" ParentId="7993" CreationDate="2013-08-22T20:50:22.190" Score="0" Body="&lt;p&gt;Depending on what you want to show, the matplotlib library for python may also be a good solution. In particular, I would like to direct you to the &lt;a href=&quot;http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.hlines&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;hlines&lt;/code&gt;&lt;/a&gt; function  and this article on &lt;a href=&quot;http://matplotlib.org/users/annotations_intro.html&quot; rel=&quot;nofollow&quot;&gt;annotating your plots&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2013-08-22T20:50:22.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8302" PostTypeId="1" CreationDate="2013-08-22T21:10:58.890" Score="3" ViewCount="220" Body="&lt;p&gt;For a 3D velocity field $\mathbf{u}$ obtained by direct numerical simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that the field is defined on a periodic domain $\mathcal{P}$ of periodicity $L_x$ in the $x$ direction,  $L_y$ in the $y$ direction and  $L_z$ in the $z$ direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that the simulation is computed from time $t_1$ to $t_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How practically estimate the degree of statistical homogeneity of the velocity field?&lt;/p&gt;&#xA;" OwnerUserId="1383" LastEditorUserId="1383" LastEditDate="2013-08-22T21:43:58.140" LastActivityDate="2013-08-22T21:43:58.140" Title="How to estimate if a velocity field is statistically homogeneous?" Tags="|fluid-dynamics|simulation|statistics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8303" PostTypeId="1" CreationDate="2013-08-23T11:34:41.140" Score="2" ViewCount="2259" Body="&lt;p&gt;I'm looking at a global minimisation problem on the $n$-sphere ($n$ of the order 10--50) but with a complete swine of an objective --- it is (effectively) a black box; it is piecewise smooth but with discontinuities and (this is the kicker) the minimum seems to occur at (one side of) a discontinuity.  My best results so-far are with a simulated annealing (&lt;code&gt;scipy.optimize.anneal&lt;/code&gt;, hacked so that updates stay on the sphere) which seems to give me reasonable results with 100,000 evaluations on an $n=10$ problem taking about 2 CPU-hours.  Is there a better way?   &lt;/p&gt;&#xA;" OwnerUserId="1400" LastActivityDate="2013-08-25T19:29:15.480" Title="Global optimisation with discontinuous objective" Tags="|nonconvex|constrained-optimization|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8304" PostTypeId="2" ParentId="8204" CreationDate="2013-08-23T12:24:58.547" Score="2" Body="&lt;p&gt;Ill-conditioned systems are better solved by regularisation than by increasing the numerical precision.  Search for &quot;regularisation ill-posed&quot; for the gory details.&lt;/p&gt;&#xA;" OwnerUserId="1400" LastActivityDate="2013-08-23T12:24:58.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8305" PostTypeId="2" ParentId="8303" CreationDate="2013-08-23T15:13:42.210" Score="1" Body="&lt;p&gt;I suggest you try a &lt;a href=&quot;http://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithm&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genetic Algorithms are robust against discontinuities in the objective function and converge to the global optimum in propability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They might be unfeasible if the evaluation of the objective is expensive. But you have a lot of parameters for tuning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at &lt;a href=&quot;https://stackoverflow.com/questions/4227538/python-module-for-multiple-variable-global-optimization&quot;&gt;this post&lt;/a&gt; for implementations in python. &lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-08-23T15:13:42.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8306" PostTypeId="2" ParentId="8204" CreationDate="2013-08-23T16:38:43.950" Score="3" Body="&lt;p&gt;As of version &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/changes/32.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;3.2&lt;/a&gt;, PETSc supports sparse quad-precision computations on gcc/gfortran 4.6 and newer.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll need a quad-precision BLAS and LAPACK, which PETSc can provide to you (along with quad support) with the following (partial) configure command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./configure --with-precision=__float128 --download-f2cblaslapack&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/faq.html#precision&quot; rel=&quot;nofollow noreferrer&quot;&gt;FAQ&lt;/a&gt; for a little more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I agree with &lt;a href=&quot;https://scicomp.stackexchange.com/a/8304/9&quot;&gt;nOOb&lt;/a&gt;, if at all possible, try to regularize the system before switching out to quad-precision.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-08-23T21:46:36.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8307" PostTypeId="1" CreationDate="2013-08-23T17:42:37.447" Score="2" ViewCount="80" Body="&lt;p&gt;I need to code up a computer graphics algorithm for Surface Registration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly &lt;strong&gt;surface registration&lt;/strong&gt; is the process of finding &quot;optimal&quot; one-one correspondence &#xA;between surfaces, where the meaning of optimal depends on the specific problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have much experience with Computer Graphics software, and so I was looking around&#xA;the web for a good software environment to code my algorithm up in. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One promising option I hit upon was &lt;a href=&quot;http://www.blender.org/&quot; rel=&quot;nofollow&quot;&gt;Blender&lt;/a&gt;, which to my delight also has a Python interface to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is &lt;em&gt;Blender + Python&lt;/em&gt; a good software environment for implementing and testing computer graphics algorithms? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If not, are there better alternatives to it (hopefully using a Pythonic interface)? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there any particular points from your experience&#xA;which I should keep in mind while using Blender (or whatever package you recommend) while implementing/testing my algorithm? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3352" LastActivityDate="2013-08-23T17:42:37.447" Title="Environment for implementing/testing Computer Graphics algorithms" Tags="|python|computational-geometry|computer-vision|geometry|" AnswerCount="0" CommentCount="2" ClosedDate="2013-08-24T06:30:03.793" ContentLicense="CC BY-SA 3.0" />
  <row Id="8308" PostTypeId="1" AcceptedAnswerId="8310" CreationDate="2013-08-23T19:52:06.573" Score="3" ViewCount="12501" Body="&lt;p&gt;I hope this is not too off-topic here -- I've asked it on SO but I'm hoping I'll get better answers here!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I'm a little bit confused about when I should call MPI_Wait (or other variants such as: MPI_Waitall, MPII_Waitsome, etc). Consider the following situations: (Note: pseudo code)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case (1)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MPI_Isend (send_buffer, send_req);    &#xA;// Do local work&#xA;MPI_Probe (recv_msg);&#xA;MPI_Irecv (recv_buffer, recv_req);&#xA;// wait for msgs to finish&#xA;MPI_Wait (recv_req);   // &amp;lt;--- Is this needed?&#xA;MPI_Wait (send_req);   // &amp;lt;--- How about this?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So my confusion stems from &lt;code&gt;MPI_Probe&lt;/code&gt; in this case. Since this is a blocking call, wouldn't that essentially mean it blocks the caller until message is received? If this is the case, then I think MPI_Waits are unnecessary here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How about the following case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case (2)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MPI_Isend (send_buffer, send_req);    &#xA;// Do local work&#xA;MPI_Probe (recv_msg);&#xA;MPI_Recv (recv_buffer);&#xA;// wait for msgs to finish&#xA;MPI_Wait (send_req);   // &amp;lt;--- Is this necessary?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Similar to the first case but &lt;code&gt;MPI_Irecv&lt;/code&gt; is replaced with its blocking version. In this case, the message is definitely received by the time &lt;code&gt;MPI_Wait&lt;/code&gt; is called which means &lt;code&gt;MPI_Isend&lt;/code&gt; must have been finished ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also as a separate question, what do we mean when we say &lt;code&gt;MPI_Probe&lt;/code&gt; is blocking? Does it block until all of the message is received by the process or does it only block until &quot;meta-data&quot; (such as msg size, sender rank, etc) is received? In other words is &lt;code&gt;MPI_Probe&lt;/code&gt; + &lt;code&gt;MPI_Irecv&lt;/code&gt; any better than &lt;code&gt;MPI_Probe&lt;/code&gt; + &lt;code&gt;MPI_Recv&lt;/code&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2018-06-11T16:47:33.753" Title="When is MPI_Wait necessary for non-blocking calls?" Tags="|parallel-computing|mpi|c|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8309" PostTypeId="2" ParentId="8308" CreationDate="2013-08-23T20:13:15.043" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/research/projects/mpi/mpi-standard/mpi-report-1.1/node50.htm&quot; rel=&quot;noreferrer&quot;&gt;MPI_Probe&lt;/a&gt; allows you to test for a message without actually receiving it. You &lt;em&gt;must&lt;/em&gt; complete all non-blocking communications with an appropriate communications completion fuction like MPI_Wait and friends, otherwise the runtime will not free up internal resources associated with the communications leading to resource leaks and other problems. For example, you may not reuse the send message buffer for anything else until the communication has been completed with an appropriate completion call.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-08-23T20:13:15.043" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8310" PostTypeId="2" ParentId="8308" CreationDate="2013-08-23T21:43:26.650" Score="6" Body="&lt;p&gt;Bill answered the first part, so I'll only answer the second question.  An MPI send is blocking if it does not return until it is safe to modify the send buffer and a receive is blocking if it does not return until the receive buffer contains the newly-received message.  In practice, outside of buffered sends (thanks, Hristo Iliev), this implies that communication may be required before returning.  For example, &lt;code&gt;MPI_Send&lt;/code&gt; is blocking because it cannot complete before the message has been buffered or sent.  Implementations will usually &quot;eagerly&quot; buffer short messages, in which case &lt;code&gt;MPI_Send&lt;/code&gt; appears to return immediately.  This means that code looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MPI_Send(&amp;amp;x,count,MPI_INT,(rank+1)%size,1,comm);&#xA;MPI_Recv(&amp;amp;y,count,MPI_INT,(size+rank-1)%size,1,comm,MPI_STATUS_IGNORE);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is expected to deadlock for large messages, although it will likely succeed for small enough messages.  Nonblocking sends and receives return &lt;code&gt;MPI_Request&lt;/code&gt;s that must be completed before buffers are accessed/modified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some operations are a bit different from sends and receives. &lt;code&gt;MPI_Probe&lt;/code&gt; is blocking because it does not return until a message has been found, although the message need not have been received yet.  &lt;code&gt;MPI_Iprobe&lt;/code&gt; is non-blocking in that it always returns even if there is no message.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-08-24T15:57:12.217" LastActivityDate="2013-08-24T15:57:12.217" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8311" PostTypeId="2" ParentId="8303" CreationDate="2013-08-24T04:21:38.643" Score="4" Body="&lt;p&gt;Global optimization algorithms are almost always inefficient. This is true for all variants (simulated annealing, genetic, particle swarm, you name it). You can use them, but it's often worth finding out some properties of your objective function. For example, if you can find out &lt;em&gt;where&lt;/em&gt; the objective function is discontinuous, then you can rephrase the optimization problem as one where you try to find the optimum on each patch of smooth objective function plus constraints to delineate the smooth region, and then take the optimum over the best value from each of the patches. On each patch you use a local optimizer, making for a much faster method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-24T04:21:38.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8312" PostTypeId="2" ParentId="8272" CreationDate="2013-08-24T13:18:28.547" Score="0" Body="&lt;p&gt;&quot;Wrong&quot; eigenvectors is a poor problem statement in terms of how it helps us to help you. Are they merely of the wrong sign? Since the sign is arbitrary, that is impossible to control, but you can easily flip the signs if you don't like them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One not uncommon convention is to always force the sign of the first non-zero element to be positive, but that too can be sometimes an issue if the first element were very near zero. How close is close enough?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there replicated eigenvalues? If so, then the eigenvectors for that set of replicated eigenvalues are not unique. You can essentially rotate the eigenvectors that span that subspace arbitrarily and they will still be equally valid eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So unless you give us more help as to why eig produced the &quot;incorrect&quot; eigenvectors, we can at best guess at the real problem.&lt;/p&gt;&#xA;" OwnerDisplayName="user840" LastActivityDate="2013-08-24T13:18:28.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8313" PostTypeId="2" ParentId="1300" CreationDate="2013-08-24T15:58:48.387" Score="1" Body="&lt;p&gt;I've been working on &lt;a href=&quot;http://github.com/tsbertalan/openmg&quot; rel=&quot;nofollow&quot;&gt;this implementation&lt;/a&gt; for a little while. It's Python/Numpy/Scipy. It's not algebraic multigrid--you have to supply your own restriction operator. But if it's an education implementation in which you're interested, I would welcome some pull requests to add such capability.&lt;/p&gt;&#xA;" OwnerUserId="3976" LastActivityDate="2013-08-24T15:58:48.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8315" PostTypeId="2" ParentId="8160" CreationDate="2013-08-25T02:37:18.557" Score="-2" Body="&lt;p&gt;double G(1.9838e-29); ??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;maybe use: G = 6.673e-11&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and try something lake this:&#xA;q = q*q*q; // one&lt;/p&gt;&#xA;&#xA;&lt;p&gt;f = 1/q; g = 2/q; h = 3/q; // two, three, ...&lt;/p&gt;&#xA;" OwnerUserId="5027" LastActivityDate="2013-08-25T02:37:18.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8316" PostTypeId="1" CreationDate="2013-08-25T18:28:42.807" Score="4" ViewCount="5688" Body="&lt;p&gt;The wiki for machine epsilon says:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Machine epsilon gives an upper bound on the relative error due to rounding in floating point arithmetic&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If machine epsilon is the upper bound on the relative error, why does the spacing between floating point numbers actually get bigger for larger numbers? For example in MATLAB:   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;eps(1) = 2.220446049250313e-016 (machine epsilon)&lt;br&gt;&#xA;eps(2) = 4.440892098500626e-016&lt;br&gt;&#xA;eps(4) = 8.881784197001252e-016&lt;br&gt;&#xA;eps(8) = 1.776356839400251e-015&lt;br&gt;&#xA;...&lt;br&gt;&#xA;eps(realmax) = 1.995840309534720e+292  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how is eps the upper bound on the relative error if it is less than all of those numbers?&lt;/p&gt;&#xA;" OwnerUserId="5030" LastActivityDate="2013-08-25T19:07:27.177" Title="Machine epsilon (eps)" Tags="|matlab|floating-point|precision|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8317" PostTypeId="2" ParentId="8316" CreationDate="2013-08-25T19:07:27.177" Score="8" Body="&lt;p&gt;The Matlab command &lt;code&gt;help eps&lt;/code&gt; says the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;D = EPS(X), is the positive distance from ABS(X) to the next larger in&#xA;  magnitude floating point number of the same precision as X. X may be&#xA;  either double precision or single precision.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words, if $\varepsilon_\mathsf{mach}$ is the &lt;em&gt;relative&lt;/em&gt; error due to floating point, as defined in the Wikipedia article, then $\mbox{eps}(x)$ will, for any normal $x$, be $\hat{x}\varepsilon_\mathsf{mach}$, where $\hat{x}$ is the largest power of two such that $|\hat{x}|\leq|x|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calling &lt;code&gt;eps&lt;/code&gt; without an argument will give you the &lt;em&gt;relative&lt;/em&gt; error, calling &lt;code&gt;eps(x)&lt;/code&gt; with an argument will give you the &lt;em&gt;absolute&lt;/em&gt; error for that argument.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-08-25T19:07:27.177" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8318" PostTypeId="2" ParentId="8303" CreationDate="2013-08-25T19:29:15.480" Score="1" Body="&lt;p&gt;Consider exploring your design space a bit more methodically. &lt;a href=&quot;http://en.wikipedia.org/wiki/Polynomial_chaos&quot; rel=&quot;nofollow&quot;&gt;Polynomial chaos expansion&lt;/a&gt; is good for this; you can find it built into the very impressive DAKOTA package. Essentially you intelligently sample the design space and reconstruct it using an orthogonal polynomial expansion. It works very well in areas with discontinuities. You may be able to get away with using a 2nd order polynomial mapping, so you'd have 3^N samples (59049 for your N=10 case), sample that, and use the minima you identify as an initial guess for a local search method (e.g. gradient-based Newton-Raphson.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, try using the results of the global simulated annealing approach you're already using; then stop it after say, 10,000 results, and use the best case (or best 10) as the initial guess for scipy's local optimization routines.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-08-25T19:29:15.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8319" PostTypeId="1" AcceptedAnswerId="8325" CreationDate="2013-08-25T20:34:38.337" Score="10" ViewCount="3970" Body="&lt;p&gt;I was wondering what is the best way to learn how to program in Fortran ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been looking for books, the examples that seemed OK to me were &lt;a href=&quot;http://global.oup.com/academic/product/modern-fortran-explained-9780199601417;jsessionid=1DA082FF1635D8935AE9DBF993905268?cc=be&amp;amp;lang=en&amp;amp;&quot;&gt;&quot;Fortran 95/2003 Explained&quot;&lt;/a&gt; and &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0073191574&quot;&gt;&quot;Fortran For Scientists And Engineers&quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what you guys think of these?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that I am wondering about is which compiler I should use?&#xA;I work on a Windows 7 machine (64-bit).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope you guys can help me on this one!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for my experience with programming:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Matlab and maple were kind off my first introductions into the programming world. Matlab is a program in which I'm actually pretty good, I got the hang of working with loops and how to combine these to make more effeciënt programs. Maple is more in the basic phase (I know how to search the solution to simple integrals etc.).&lt;/li&gt;&#xA;&lt;li&gt;A year after that we got an introduction to C++ in visual basic. The professor actually went to quick on this one, after 2 months we were already discussing pointers and inheritance. I am able to write simple programs in this language, only the working with classes is something that remains a difficulty, but I believe that's just a matter of practice.&lt;/li&gt;&#xA;&lt;li&gt;And a year later I needed to learn Mathematica 8 which I'm also really getting good at nowadays.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So I have a little bit of programming-expierince, but I won't say it's an incredible amount!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I want to learn Fortran is mainly because you can't do theoretical physics on it's own, you need to test your theories and for that you'll need to be able to write programs which might be able to test those theories numerically. I heard that Fortran is kindoff the language to use when it comes to numerical mathematics, so I think it would come in handy to know this language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ps: what's the difference between Fortran95 and Fortran 2003 ? &lt;/p&gt;&#xA;" OwnerUserId="4811" LastEditorUserId="4811" LastEditDate="2013-08-26T19:08:27.373" LastActivityDate="2013-08-28T09:23:25.367" Title="Learning Fortran" Tags="|fortran|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8320" PostTypeId="2" ParentId="8319" CreationDate="2013-08-25T22:22:25.567" Score="2" Body="&lt;p&gt;Chapmans book I think is more newbie friendly and a good reference. The authors on the second one are all experts but I think the book will be a good addition to your library at a later stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For start I would suggest that you stick to Fortran 95 and then later (if needed) move to Fortran 2003. Fortran 95 is easy to learn (almost Matlab like) and you'll be productive very quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Linux you have a number of free Fortran compilers (GNU, Solaris Studio, Open64 etc.) in addition to a richer set of tools e.g., valgrind, gdb, gnuplot, (for quick viz), svn/hg etc. Additionally installation of pre-compiled packages including numerical libs (blas/lapack/netcdf/fftw etc.) is a command away (at least on Debian with apt-get).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Windows you might be better of with cygwin as it gives you easy and quick access to some of the above tools/libs in addition to the GNU Fortran compiler. So yes go to cygwin.com and install the core package along with gfortran and an editor of choice (vim is good). That should be a good start. Later if you anything else then I would suggest using the &lt;a href=&quot;http://code.google.com/p/apt-cyg/&quot; rel=&quot;nofollow&quot;&gt;apt-cyg&lt;/a&gt; package for installing stuff.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2013-08-25T22:22:25.567" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8321" PostTypeId="1" CreationDate="2013-08-26T03:14:50.503" Score="0" ViewCount="1054" Body="&lt;p&gt;&quot;The traditional local Monte Carlo method is simple, extremely general, and versatile.&quot; -Wang Swendsen, 2002&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does 'local' Monte Carlo mean ? Is there anything called 'global' Monte Carlo? &lt;/p&gt;&#xA;" OwnerUserId="4023" LastActivityDate="2016-08-24T23:05:42.263" Title="What is local Monte Carlo simulation?" Tags="|monte-carlo|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8322" PostTypeId="2" ParentId="8321" CreationDate="2013-08-26T04:20:02.470" Score="0" Body="&lt;p&gt;It's hard to tell without more context from the quote, but my best guess is that the reference refers to the fact that practical MC methods sample in the &lt;em&gt;vicinity&lt;/em&gt; of the previous sample. A &lt;em&gt;global&lt;/em&gt; method would find a new sample at a random location of the search domain, independent of the previous sample's location. But this is inefficient and is a method that is not practical.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-26T04:20:02.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8323" PostTypeId="1" AcceptedAnswerId="8334" CreationDate="2013-08-26T05:42:08.413" Score="4" ViewCount="1812" Body="&lt;p&gt;I am attempting to increase the performance of a legacy Kalman Filter implementation. The state covariance is factored in terms of UDU, i.e. $\mathbf{P} = \mathbf{U}\mathbf{D}\mathbf{U}^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many years ago, &lt;a href=&quot;http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4045429&quot; rel=&quot;nofollow&quot;&gt;Bierman&lt;/a&gt; showed how scalar measurements may be used to update the covariance. Helpfully, it includes an implementation in FORTRAN that I have reproduced in MATLAB (though it combines certain operations in a loop and thus the algorithm is not particularly transparent).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, I would like to replace the routine to take advantage of modern matrix libraries (BLAS, Eigen, etc) to increase performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, the core of the operation is to perform a rank-1 update (downdate) of the UD-factored covariance in the form of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{U_+}\mathbf{D_+}\mathbf{U_+}^T = \mathbf{U_-}\mathbf{D_-}\mathbf{U_-}^T - c\mathbf{v}\mathbf{v}^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{U_-}$ and $\mathbf{D_-}$ is my prior known factorisation, c is a non-negative scalar and $\mathbf{v}$ is (known) a column vector.  The goal is to determine $\mathbf{U_+}$ and $\mathbf{D_+}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For the modified Cholesky update, what is the cost in terms of FLOPS - I understand it is $O(n^2$)?&lt;/li&gt;&#xA;&lt;li&gt;Is the a known efficient implementation in a commonly available library, or so I need to try to roll my own?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(For interest, $n$ is about 40 , but is being implemented on an embedded platform).&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="761" LastEditDate="2013-08-26T05:47:17.480" LastActivityDate="2013-08-28T10:52:28.143" Title="Computational complexity and implementation of UDU Modified Cholesky Rank 1 Update" Tags="|matrix|rank|update|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8324" PostTypeId="1" AcceptedAnswerId="10108" CreationDate="2013-08-26T05:54:37.880" Score="1" ViewCount="1502" Body="&lt;p&gt;I'm new to opencl and I'm trying to figure out how to return a variable length array of numbers. (I'm using Cloo for C# .net)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing image processing and I've written a program to look for bubbles in oil. I want to port some of the code to opencl to take advantage of the GPU. I want my opencl function to return a list of co-ordinates and a radius for each bubble. The problem is, since I don't know how many bubbles their will be ahead of time, I seem to need a variable length array. I'm not sure how this is done (or if this can be done) in opencl.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if there is a way to exchange datastructures between the GPU and the host, this would work too. However from what I've read from other people's experiences, this doesn't sound like it has been a good option.&lt;/p&gt;&#xA;" OwnerUserId="4680" LastEditorUserId="4680" LastEditDate="2013-08-26T06:00:54.230" LastActivityDate="2013-11-20T13:08:29.163" Title="Returning variable length arrays in OpenCL" Tags="|opencl|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8325" PostTypeId="2" ParentId="8319" CreationDate="2013-08-26T07:18:13.277" Score="4" Body="&lt;p&gt;I'm going to second stali's recommendation of Chapman's book. I learned from Chapman's Fortran 90 book, and it was very helpful. It will teach you the Fortran language. It will not teach you about good workflows for programming, and for that, you should look elsewhere &lt;em&gt;after&lt;/em&gt; you've learned some Fortran and programmed for a little bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most online resources to help you with Fortran programming are going to be Linux-based. Similarly, in computational science, most of the written materials are going to be geared towards Linux users. For that reason, I strongly recommend that you install VirtualBox (or some sort of software that enables you to run virtual operating systems) and then install some Linux distribution on there. If you've never used Linux before, a user-friendly distribution like Ubuntu, Mint, or Fedora is probably going to be your best bet. These distributions require a minimum amount of configuration if run in a virtual environment, and have a good GUI to help you navigate the operating system. Each of these distributions has a package manager that you can use to install some additional software. Using the package manager, you can install gfortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're going to want to develop software in a text editor (or an IDE), so it's probably worth learning the basics of the command line. Software Carpentry has good resources for this task. (Disclaimer: I've taught part of a workshop for Software Carpentry.) Then you should be able to move forward with learning to write simple programs and compile them on the command line, and then learn how to use Makefile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with doing anything Windows-based is that most of the online materials you'll see when doing computational science are going to assume you're using Linux, and the advice (if it was any good to begin with) is going to break down, and cost you time. Cygwin is an API layer that adds more Linux-like functionality, but the correspondence with Linux isn't perfect, and software compiled for Linux is not going to work on your machine; you'll still need to get Windows-native versions or recompile the software for Windows. A Windows-native Fortran compiler such as Visual Fortran is going to cost money, and you're still going to have the problem that most Fortran users and computational scientists use Linux for their software development, meaning you'll have to forge ahead on your own. However, if you really need to &lt;em&gt;distribute&lt;/em&gt; Windows-based software for some reason, then it makes sense to go through all the effort to figure out how to use something like Visual Fortran.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-08-26T07:18:13.277" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8326" PostTypeId="5" CreationDate="2013-08-26T09:35:19.707" Score="0" Body="&lt;p&gt;SPH (Smoothed Particle Hydrodynamics) is a computational method to simulate fluid flows. It is a lagrangian method, used in different fields as astrophysics or oceanography. It was first developed by Gingold and Monanghan (1977) in an astrophysical context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It relies on a discretization of the fluid in fluid elements, the particles. These particles are characterized by a &quot;smoothing length&quot; over which their physical properties are smoothed over the other particles laying within this length by a &quot;kernel function&quot;. Kernel functions are typically cubic spline of Gaussian function.&lt;/p&gt;&#xA;" OwnerUserId="3849" LastEditorUserId="3849" LastEditDate="2013-08-26T15:34:11.877" LastActivityDate="2013-08-26T15:34:11.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8327" PostTypeId="4" CreationDate="2013-08-26T09:35:19.707" Score="0" Body="SPH (Smoothed Particle Hydrodynamics) is a computational method to simulate fluid flows. It is a lagrangian method, used in different fields as astrophysics or oceanography." OwnerUserId="3849" LastEditorUserId="3849" LastEditDate="2013-08-26T15:34:18.283" LastActivityDate="2013-08-26T15:34:18.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8328" PostTypeId="2" ParentId="5127" CreationDate="2013-08-26T10:55:25.503" Score="2" Body="&lt;p&gt;There are quite a few unknowns in your question, so I'll try to make this answer as broad as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the sound of it, you're considering writing your own MD code. Writing an MD code is easy, but writing a &lt;em&gt;good&lt;/em&gt;, i.e. efficient and correct, MD code is &lt;em&gt;very, very difficult&lt;/em&gt;, not to mention time-consuming. Unless you're studying some special case of physics or geometry which can not be handled by existing MD codes/libraries, or you are a Computer Scientist looking at some new algorithmic challenges, I would advise you first try to use existing software packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your MD requirements are relatively standard, i.e. no special potential functions or interactions, then I would strongly recommend using one of the most popular Open-Source software packages such as &lt;a href=&quot;http://www.ks.uiuc.edu/Research/namd/&quot; rel=&quot;nofollow&quot;&gt;NAMD&lt;/a&gt; or &lt;a href=&quot;http://www.gromacs.org/&quot; rel=&quot;nofollow&quot;&gt;GROMACS&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have any exotic requirements, you could look at MD libraries such as &lt;a href=&quot;http://dirac.cnrs-orleans.fr/MMTK/&quot; rel=&quot;nofollow&quot;&gt;MMTK&lt;/a&gt; (in Python), &lt;a href=&quot;https://simtk.org/home/openmm&quot; rel=&quot;nofollow&quot;&gt;OpenMM&lt;/a&gt; (in C), or &lt;a href=&quot;http://mdcore.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;mdcore&lt;/a&gt; (in C, note tat this is my own code). These allow you to do most of the nuts and bolts bits of MD simulations, but also let you access and manipulate the particle data directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the exception of MMTK, all of these codes/libraries can use GPUs. If you're looking to run long simulations efficiently, then GPUs are definitely the way to go, especially in terms of their bang/buck ratio. GPUs can also be used to run relatively large simulations as most models currently have several gigabytes of memory on board. We regularly use NAMD's &lt;a href=&quot;http://www.ks.uiuc.edu/Research/STMV/&quot; rel=&quot;nofollow&quot;&gt;STMV setup&lt;/a&gt; as a benchmark, which consists of more than one million particles. Also, if you have a machine with multiple GPUs, you can run several simulations in parallel, one on each GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;strong&gt;summary&lt;/strong&gt;, I would not recommend writing your own code unless that is the ultimate goal of your research. Stick to existing packages and/or libraries, depending on your needs, and keep in mind that GPUs are currently the most efficient workhorses for MD simulations.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-08-26T10:55:25.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8329" PostTypeId="2" ParentId="8319" CreationDate="2013-08-26T11:30:21.670" Score="3" Body="&lt;p&gt;For books, I recommend &quot;Modern Fortran Explained&quot; by Metcalf/Reid/Cohen. It's more a reference than a textbook, but it'll serve you well if you already had some exposure to programming in general (with say C).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides that book, I strongly recommend &lt;a href=&quot;https://scicomp.stackexchange.com/users/236/ondrej-certik&quot;&gt;Ondřej Čertík&lt;/a&gt;'s website &lt;a href=&quot;http://fortran90.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://fortran90.org/&lt;/a&gt;. It won't teach you Fortran in the first place, but it'll provide you a set of best practices. Fortran allows various ways for programming - it's downward compatible after all. The best practices given there will help you develop a consistent, modern Fortran style. Also, the FAQ found there will be of interest to you.&lt;/p&gt;&#xA;" OwnerUserId="1553" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-08-26T11:30:21.670" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8330" PostTypeId="2" ParentId="8321" CreationDate="2013-08-26T15:10:16.357" Score="2" Body="&lt;p&gt;The qutoe is from a 2002 paper by Swensen and Wang in the Journal of Statistical Physics titled &quot;Transition Matrix Monte Carlo Method.&quot;  A preprint is available from the ArXiv:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/pdf/cond-mat/0104418.pdf&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/pdf/cond-mat/0104418.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scanning through the paper quickly, it's apparent that by &quot;local Monte Carlo method&quot;, the authors mean the conventional approach to Monte Carlo simulation of an Ising model (or similiar statistical mechanics system) in which you consider flipping one spin at a time, and the probability of that flip happening depends on the state of nearby spins.  This is the original kind of Markov Chain Monte Carlo simulation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, it's often the case that the algorithm will consider many potential state changes with small probabilities, so that most steps are not actually accepted.  The authors of this paper are proposing an algorithm that speeds things up by estimating a transition matrix for the next state that the system will actually move to.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-08-26T15:10:16.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8331" PostTypeId="1" AcceptedAnswerId="8344" CreationDate="2013-08-26T15:21:35.263" Score="0" ViewCount="1027" Body="&lt;p&gt;How to set image with respect to a point in Paraview? I have a vtk file within a rectangular domain starting with the lower left point -1842,10000, I want to set a background image whose lower left also starts at the same point as stated earlier in order to superimpose the vtk on the image. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="4907" LastActivityDate="2014-02-24T04:40:04.177" Title="Setting background image with respect to a point in Paraview" Tags="|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8332" PostTypeId="1" AcceptedAnswerId="8357" CreationDate="2013-08-26T15:44:14.550" Score="7" ViewCount="1044" Body="&lt;p&gt;The context - I'm working with a spectral FE (higher order interpolation at GLL nodes) code on conforming hexahedral meshes, and our PI is interested in improving mesh quality, possibly with adaptive refinement. However, the only local refinement schemes I know of for hexes involve nonconforming meshes/hanging nodes, which don't tend to work so well for spectral FE methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conforming mesh refinement schemes I'm familiar with involve essentially splitting a 2x2 set of quads into a 3x3 set (or equivalently, splitting the middle node of the 2x2 set of quads into 9 new nodes in 2D), as below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/loXCB.gif&quot; alt=&quot;Original mesh&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ypxJi.gif&quot; alt=&quot;Node-refined mesh&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only know of conforming adaptivity using some mesh metric; is there a way to use instead local error indicators to drive adaptivity in a conforming manner?  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-08-29T23:57:26.527" Title="Conforming mesh refinement for quads/hex elements" Tags="|finite-element|adaptive-mesh-refinement|spectral-method|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8333" PostTypeId="1" AcceptedAnswerId="8350" CreationDate="2013-08-26T17:13:39.610" Score="11" ViewCount="534" Body="&lt;p&gt;There are many well known numerical methods for solving equations of the type&#xA;$$ f(x) = 0, \quad x \in \mathbb{R}^n,$$&#xA;e.g. bisection method, Newton's method, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my application $f(x)$ is calculated with a stochastic method (the result is an average).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any numerical equation solving methods that handle this situation well?  Links to any discussion of similar situations are also appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The precision to which I can calculate $f(x)$ depends strongly on $x$, and I might easily hit a wall where I can't increase the precision without increasing the computation time significantly.  So I can't ignore the fact that the result from $f$ is not precise.  This will also impact the precision to which $x$ can be found in practice.&lt;/p&gt;&#xA;" OwnerUserId="605" LastActivityDate="2013-12-10T14:50:50.543" Title="Numerical method for equation solving that works on stochastically computed functions" Tags="|monte-carlo|nonlinear-equations|stochastic|numerics|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8334" PostTypeId="2" ParentId="8323" CreationDate="2013-08-27T03:54:50.857" Score="4" Body="&lt;p&gt;Yes, rank-one updates of a Cholesky factorization take $O(n^{2})$ time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, there isn't a low rank update for the Cholesky factorization in LAPACK.  There is an update routine for the Cholesky factorization in Linpack, but that routine works with the convention $LL^{T}$ factorization, not the square root free Cholesky factorization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, there isn't enough data reuse in the process of rank-one updating a Cholesky factorization to get the kind of performance boost from blocknig and cache reuse that you get in various $O(n^3)$ operations that operate on $O(n^{2})$ data.  .  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-08-27T03:54:50.857" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8335" PostTypeId="2" ParentId="3523" CreationDate="2013-08-27T05:15:57.450" Score="5" Body="&lt;p&gt;Look for the book &lt;a href=&quot;http://books.google.com/books/about/Vibration_of_Plates.html?id=TfsfAAAAIAAJ&quot; rel=&quot;noreferrer&quot;&gt;Vibration of plates&lt;/a&gt; by Arthur Leissa. It has explicit solutions for square and circular plates. Including tables with approximated eigenvalue for different boundary conditions. &lt;/p&gt;&#xA;" OwnerUserId="5039" LastEditorUserId="1804" LastEditDate="2013-08-27T17:26:49.943" LastActivityDate="2013-08-27T17:26:49.943" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8336" PostTypeId="2" ParentId="8319" CreationDate="2013-08-27T09:10:23.780" Score="6" Body="&lt;p&gt;I think that the Metcalfe book is a good annotated guide to the modern features of Fortran, very useful if you already know an earlier version of the language but not a good introduction to programming in the language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my money the &lt;a href=&quot;http://www.amazon.co.uk/Introduction-Programming-Fortran-Coverage-2003/dp/0857292323/ref=sr_1_1?s=books&amp;amp;ie=UTF8&amp;amp;qid=1377594208&amp;amp;sr=1-1&amp;amp;keywords=fortran%20chivers&quot; rel=&quot;nofollow&quot;&gt;Chiver's and Sleightholme book &lt;/a&gt; and the &lt;a href=&quot;http://www.amazon.co.uk/Guide-Fortran-Programming-Walter-Brainerd/dp/1848825420/ref=sr_1_3?s=books&amp;amp;ie=UTF8&amp;amp;qid=1377594243&amp;amp;sr=1-3&amp;amp;keywords=fortran%20brainerd&quot; rel=&quot;nofollow&quot;&gt;Brainerd book&lt;/a&gt; are both good enough, though I don't think either is as good as the &lt;a href=&quot;http://www.amazon.co.uk/Fortran-Programming-International-Computer-Science/dp/0201544466/ref=sr_1_2?s=books&amp;amp;ie=UTF8&amp;amp;qid=1377594295&amp;amp;sr=1-2&amp;amp;keywords=fortran%20ellis&quot; rel=&quot;nofollow&quot;&gt;book by Ellis &lt;em&gt;et al&lt;/em&gt;&lt;/a&gt; which, unfortunately, has not been updated for Fortran 2003 or later.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a strong aversion to the Chapman book.  I can't remember why I disliked the 90/95 edition so much so can't justify this aversion any longer, you can discount it as a prejudice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the one book I recommend without hesitation is &lt;a href=&quot;http://www.amazon.co.uk/Modern-Fortran-Practice-Arjen-Markus/dp/1107603471/ref=sr_1_1?s=books&amp;amp;ie=UTF8&amp;amp;qid=1377594499&amp;amp;sr=1-1&amp;amp;keywords=fortran%20markus&quot; rel=&quot;nofollow&quot;&gt;Arjen Markus' book&lt;/a&gt;.  It's probably not an ideal introduction for someone still trying to figure out the basic features of the language, but once you've mastered those it's worth it's weight in some valuable substance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for which compiler to use, I use the Intel compiler and am generally happy with it. It integrates with Visual Studio, which might be important to you.  The NAG compiler has better error and warning messages but produces slower (generally and your experience may differ) executables. The gfortran compiler is free and gets good enough reviews but it's not one I use.  If you want to know more &lt;a href=&quot;http://www.polyhedron.com/compare0html&quot; rel=&quot;nofollow&quot;&gt;Polyhedron&lt;/a&gt; will tell you.&lt;/p&gt;&#xA;" OwnerUserId="1003" LastEditorUserId="1003" LastEditDate="2013-08-28T09:23:25.367" LastActivityDate="2013-08-28T09:23:25.367" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8337" PostTypeId="1" AcceptedAnswerId="8340" CreationDate="2013-08-27T09:57:29.880" Score="16" ViewCount="3018" Body="&lt;p&gt;I am trying to understand some results and would appreciate some general comments on tackling nonlinear problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fisher's equation (a nonlinear reaction-diffusion PDE), &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = du_{xx} + \beta u (1 - u) = F(u)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in discretised form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_j^{\prime} = \boldsymbol{L}\boldsymbol{u} + \beta u_j (1 - u_j) = F(\boldsymbol{u})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\boldsymbol{L}$ is the differential operator and $\boldsymbol{u}=(u_{j-1}, u_j, u_{j+1}) $ is the discretisation stencil.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Method&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I wish to apply a implicit scheme because I require stability and unrestricted time step. For this purpose I am using the $\theta$-method, (note that $\theta=1$ gives a fully implicit scheme and $\theta=0.5$ gives the trapezoidal or &quot;Crank-Nicolson&quot; scheme),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_{j}^{\prime} = \theta F(\boldsymbol{u}^{n+1}) + (1-\theta) F(\boldsymbol{u}^{n})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for nonlinear problems this cannot be done because the equation cannot be written in a linear form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get around this problem I have been exploring two numerical approaches,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;IMEX method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;   u_j^{\prime} = \underbrace{\theta\boldsymbol{L}\boldsymbol{u^{n+1}} + (1-\theta)\boldsymbol{L}\boldsymbol{u^{n}}}_{\theta-\text{method diffusion term}} + \underbrace{\beta u_j^{n} (1 - u_j^{n})}_{\text{Fully explicit reaction term}}&#xA;   $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most obvious route is to ignore the nonlinear part of the reaction term and just update the reaction term with the best possible value, i.e. that from the previous time step. This results in the IMEX method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Newton solver &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nu^{k+1} = \nu^{k} - (I - \theta\tau A^{n})^{-1} \left( \nu^{k} - u_{n} - (1-\theta) \tau F(w^n) - \theta\tau F(w^{n+1}) \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full $\theta$-method equation can be solved using the a Newton-Raphson iteration to find the future solution variable. Where $k$ is the iteration index ($k\geq0$) and $A^{n}$ is the Jacobian matrix of $F(w^n)$. Here I use the symbols $\nu^{k}$ for iteration variables such that they are distinguished from solution of the equation at a real time point $u^n$. This is actually a &lt;em&gt;modified&lt;/em&gt; Newton solver because the Jacobian is not updated with every iteration.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Results&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DNawj.png&quot; alt=&quot;Fisher&amp;#39;s equation comparison of numerical methods.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results above are calculated for a reasonably large time step and they show the difference between the time stepping approach and a full Newton iteration solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I don't understand:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I am surprised that the time-stepping method does &quot;OK&quot; but it eventually lags behind the analytical solution as time goes by. (&lt;strong&gt;NB&lt;/strong&gt; if I had chosen a smaller time-step then the time-stepping approach gives results closed to the analytical model). Why does the time-stepping approach give reasonable results to a nonlinear equation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Newton model does much better, but starts to lead the analytical model as time goes forward. Why does the accuracy of the Newton approach decrease with time? Can accuracy be improved?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is there a general feature that after many iterations then numerical model and the analytical model begin to diverge? Is this just because the time step is too large or will this always happen?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-09-02T10:06:49.280" LastActivityDate="2013-09-02T10:06:49.280" Title="Is it possible to solve nonlinear PDEs without using Newton-Raphson iteration?" Tags="|numerical-analysis|nonlinear-equations|implicit-methods|newton-method|explicit-methods|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8340" PostTypeId="2" ParentId="8337" CreationDate="2013-08-27T12:09:28.680" Score="10" Body="&lt;p&gt;I assume, that you have conducted a space discretization, so that you are about solving the (vector-valued) ODE&#xA;$$&#xA;   \dot u_h(t) = F_h(t,u_h(t)), \text{ on [0,T] }, u_h(0) = \alpha.&#xA;$$&#xA;via a numerical scheme $\Phi$ that advances the approximation $u_h^n$ at the current time instance $t=t^n$ to the next value $u_h^{n+1}$ at $t=t^{n+1}:=t^n+\tau$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your questions refer to properties of &lt;strong&gt;explicit&lt;/strong&gt;, where the update writes as&#xA;$$&#xA;u_h^{n+1} = u_h^n + \Phi_e(t^n,\tau,u_h^n),\quad \quad \quad &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;implicit&lt;/strong&gt;, written like&#xA;$$&#xA;u_h^{n+1} = u_h^n + \Phi_i(t^n,\tau,u_h^{n+1},u_h^n), \quad \quad \quad (*)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or a combination of &lt;em&gt;both&lt;/em&gt; ('&lt;strong&gt;IMEX&lt;/strong&gt;', see @Jed Brown's answer) single-step time-stepping schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this setup, the &lt;em&gt;Newton&lt;/em&gt; method simply is an approach to solve the possibly nonlinear in $u_h^{n+1}$ systems resulting from $(*)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my answers base on results from the numerical analysis of single-step methods.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you use convergent schemes, in terms of the convergence order, there is no general advantage of using implicit schemes (see. 2.). However, for stiff systems, e.g. your system containing a Laplacian, there are implicit schemes that are stable without time-step restrictions. Nevertheless, in theory, for the explicit scheme, you get better results with smaller time-steps, as long as your equation itself is stable (e.g., referring to Picard-Lindelof Theorem, if $F_h$ is Lipshitz in the second argument) and your time-step is not too small.&lt;/li&gt;&#xA;&lt;li&gt;You can find examples, where explicit schemes perform better. (Theoretically, you can reverse the time in your example, start from the terminal value, and find implicit and explicit interchanged.) If you make the Newton error sufficiently small, you can still improve accuracy by decreasing the time-step or by using time-stepping schemes of higher order.&lt;/li&gt;&#xA;&lt;li&gt;The constant $C$ in the error estimate for the global error grows exponentially with the length of the time-interval. See, e.g., &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler_method#Global_truncation_error&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for the explicit Euler scheme. This is true for every single-step method. As the estimate is of type $err \approx C \tau^p$, $p&amp;gt;0$, a smaller time-step $\tau$ only postpones this effect. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Some more remarks and the final answer:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;IMEX schemes can be used to treat only the linear part implicitly what avoids the nonlinear solves. See Jed Brown's answer.&lt;/li&gt;&#xA;&lt;li&gt;Crank-Nicolson is a single-step method. The 'multi' in multi-step methods refers to the use of a number of preceding timesteps to define the current update. E.g. like &#xA;$$&#xA;u_h^{n+1} = \Phi_m(t^n,\tau,u_h^{n+1},u_h^n,u_h^{n-1}).&#xA;$$ &#xA;This is very different from single-step and also split-step or IMEX methods, where the update is defined not recurring to previous values.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, my answer is: &lt;strong&gt;Yes&lt;/strong&gt;, you can solve nonlinear PDEs without Newton's method. You can use explicit schemes, 'IMEX' schemes, or socalled &lt;em&gt;linearly implicit&lt;/em&gt; methods (e.g. the Rosenbrock methods). Also, you can employ other approaches to solve the systems from $(*)$ like fixed-point iteration or, in particular cases, algebraic solvers. &lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-09-02T09:33:53.867" LastActivityDate="2013-09-02T09:33:53.867" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8341" PostTypeId="2" ParentId="5127" CreationDate="2013-08-27T12:14:12.613" Score="1" Body="&lt;p&gt;Here are three architectures you could evaluate against your needs. Considering them may help you formulate your needs more concretely. If you, as Pedro suggest, base your work on an existing software package (and I think you should) compatibility with that package will help narrow down the alternatives. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You could consider something from the Intel Xeon Phi family, which you can slot into a PC cabinet and which sets you back about $2700. &lt;/li&gt;&#xA;&lt;li&gt;Go with a GPU-based program&lt;/li&gt;&#xA;&lt;li&gt;Run things initially on virtual machines in the cloud. Easy to scale up and get a feel for just how much parallellism you actually need. (won't go down well with licensing of some software packages)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5034" LastActivityDate="2013-08-27T12:14:12.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8342" PostTypeId="2" ParentId="8337" CreationDate="2013-08-28T02:01:54.890" Score="13" Body="&lt;h3&gt;Short answer&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you only want second order accuracy and no embedded error estimation, chances are that you'll be happy with Strang splitting: half-step of reaction, full step of diffusion, half step of reaction.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Long answer&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Reaction-diffusion, even with linear reaction, is famous for demonstrating splitting error.  Indeed, it can be much worse, including &quot;converging&quot; to incorrect steady-states, mistaking steady-states for limit cycles, confuse stable and unstable configurations, and more.  See Ropp, Shadid, and Ober (2004) and Knoll, Chacon, Margolin, and Mousseau (2003) for the computational physicists perspective on this.  For the mathematician's analysis in terms of order conditions, see Hairer and Wanner's book on stiff ODE (Rosenbrock-W methods are a linearly-implicit IMEX method), Kennedy and Carpenter (2003) for nonlinearly-implicit IMEX &quot;additive&quot; Runge-Kutta, and &lt;a href=&quot;http://www.mcs.anl.gov/~emconsta/TimeStepping.php#IMEXDescription&quot;&gt;Emil Constantinescu's page&lt;/a&gt; for more recent IMEX methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, IMEX methods have more order conditions than the underlying implicit and explicit methods alone.  IMEX method pairs can be designed with desired linear and nonlinear stability and so that they satisfy &lt;em&gt;all&lt;/em&gt; order conditions up to the design order of the method.  Satisfying all order conditions will keep the asymptotic splitting error of the same scale as the error in each scheme separately.  It says nothing about the pre-asymptotic regime (large time steps/low accuracy requirement), but it is rarely more stringent than resolution of each part separately.  In any case, the splitting error is visible to the embedded error estimator (when using adaptive error control).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc has many IMEX methods of the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSROSW.html&quot;&gt;Rosenbrock-W&lt;/a&gt; and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSARKIMEX.html&quot;&gt;additive Runge-Kutta&lt;/a&gt; families, and will have extrapolation and linear multistep IMEX in our next release.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I wrote much of the PETSc time integration support and collaborate with Emil (linked above).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-08-28T02:01:54.890" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8343" PostTypeId="2" ParentId="8332" CreationDate="2013-08-28T02:10:35.780" Score="1" Body="&lt;p&gt;In practice, everyone who uses quads or hexes uses non-conforming mesh refinement because the conforming methods are just too unwieldy. The only alternative I can think of would be that you start with a triangular (tetrahedral) mesh and get your quad (hex) mesh by subdividing the triangles (tets). Then, when you know which quads you want to refine, you go back to the triangular mesh, refine it in some conforming way into another triangular mesh, and get your next quad mesh by subdividing the refined triangles (same for tets).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-28T02:10:35.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8344" PostTypeId="2" ParentId="8331" CreationDate="2013-08-28T02:22:53.820" Score="1" Body="&lt;p&gt;You should be able to apply a transform filter to each item and translate it anyway you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can enter the translate values directly in the properties tab. Or you can middle-mouse and drag the object into position.&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2013-08-28T02:22:53.820" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8345" PostTypeId="1" AcceptedAnswerId="8358" CreationDate="2013-08-28T02:26:51.773" Score="4" ViewCount="3753" Body="&lt;p&gt;I'm simulating the thermo-electro-mechanical behavior of a copper wire which is surrounded by silicon dioxide. In other words, the wire segments is under mechanical and thermal loads and at the same time an electrical current is flowing in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to perform the simulation accurately, I think that I should consider an initial stress due to the mismatch of thermal expansion coefficients between copper and oxide. I am using COMSOL to do my simulation. The options that COMSOL provides me with are &quot;body load&quot; and &quot;boundary load&quot;. Can anyone advise me which one I should choose? Which one is a more realistic assumption. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4858" LastActivityDate="2013-09-08T20:49:05.420" Title="Stress due to the mismatch of thermal expansion coefficients of two different attached materials in COMSOL" Tags="|finite-element|comsol|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8346" PostTypeId="1" AcceptedAnswerId="8369" CreationDate="2013-08-28T04:48:01.163" Score="2" ViewCount="363" Body="&lt;p&gt;The background is that I'm solving a problem in numerical analysis and I seem to be halfway having solved the non-linear system and now getting to the part where I should apply solving differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/467164/how-to-proceed-solving-this-problem&quot;&gt;https://math.stackexchange.com/questions/467164/how-to-proceed-solving-this-problem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code so far is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z=[0:500:4000 5000:1000:12000];&#xA;data=[5050 4980 4930 4890 4870 4865 4860 4860 4865 4875 4885 4905 4920 4935 4950 4970 4990];&#xA;fun=@(p)(4800 + p(1))*ones(size(z)) +p(2)/1000*z+p(3)*exp(p(4)/1000*z)-data;&#xA;x0=[0 0 0 -1];&#xA;opt = optimset('MaxFunEvals',1000);&#xA;p=lsqnonlin(fun,x0,[],[],opt);&#xA;fitf=@(t)(4800 + p(1))*ones(size(t)) + p(2)/1000*t+ p(3)*exp(p(4)/1000*t);&#xA;tt=linspace(0,12000,1000);&#xA;plot(z,data,'r-',tt,fitf(tt),'b-');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the system of equations is stated in the math question. Now I suppose that I should define a function something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function f=fsys(x, z)&#xA;    f= ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But first I suppose that I must rewrite the system from second order to first order which is just math theory, right? So I first rewrite the system to first order and then I define the function in matlab and finally something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xend=...; z0=...&#xA;[X,Z]=ode45(@fsys,[0, xend],z0); zs=Z(end), plot(X,Z,'--')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you help me define fsys, tell me how to proceed, confirm or reject that I'm on the right way? Any hints if you don't want to help me but hint at how to learn matlab well enough to solve this &quot;pretty simple math&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I can transform the system in the usual way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u = z'&#xA;u' = z''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The system becomes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u'=-q0*c'(z)/((c(z)^3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since z is a function of x, u and u' are also functions of x and thus this system till become a first-ordet system but how do I program the solution in matlab? How do I use that u=tan(b0)?&lt;/p&gt;&#xA;" OwnerUserId="3566" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-08-31T21:35:54.427" Title="ode45 usage in this case?" Tags="|matlab|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8347" PostTypeId="2" ParentId="8232" CreationDate="2013-08-28T06:26:14.317" Score="0" Body="&lt;p&gt;OpenCL uses barriers&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to store the results of the first &quot;pass&quot; in a variable then call&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;barrier(CLK_LOCAL_MEM_FENCE);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once all the threads have reached the barrier, the next section of code can be executed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is to enforce data dependencies. &lt;/p&gt;&#xA;" OwnerUserId="4680" LastActivityDate="2013-08-28T06:26:14.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8348" PostTypeId="2" ParentId="8323" CreationDate="2013-08-28T06:32:43.267" Score="1" Body="&lt;p&gt;Since a UDU rank 1 update is hard to find, below is a MATLAB implementation (without warranties of any kind)&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Downdate&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [Ubar, Dbar] = ThorntonRank1Downdate(U, D, c, a)&#xA;&#xA;Ubar = U;&#xA;Dbar = D;&#xA;&#xA;n = length(a);&#xA;&#xA;for j = n : -1 : 2&#xA;&#xA;    s = a(j);&#xA;    d = D(j) - c * s^2;&#xA;    b = c / d;&#xA;    beta = s * b;&#xA;    c = b * D(j);&#xA;    Dbar(j) = d;&#xA;&#xA;    a( 1:(j-1) ) = a( 1:(j-1) ) - s * U( 1:(j-1) ,j);&#xA;    Ubar( 1:(j-1) ,j) = U( 1:(j-1) ,j) - beta * a( 1:(j-1) ) ;&#xA;&#xA;end&#xA;&#xA;Dbar(1) = D(1) - c * a(1)^2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [Ubar, Dbar] = ThorntonRank1Update(U, D, c, a)&#xA;&#xA;Ubar = U;&#xA;Dbar = D;&#xA;&#xA;n = length(a);&#xA;&#xA;for j = n : -1 : 2&#xA;&#xA;    s = a(j);&#xA;    d = D(j) + c * s^2;&#xA;    b = c / d;&#xA;    beta = s * b;&#xA;    c = b * D(j);&#xA;    Dbar(j) = d;&#xA;&#xA;    for i = j-1 : -1 : 1&#xA;&#xA;        a(i) = a(i) - s * U(i,j);&#xA;        Ubar(i,j) = U(i,j) + beta * a(i);&#xA;&#xA;    end&#xA;&#xA;end&#xA;&#xA;Dbar(1) = D(1) + c * a(1)^2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Test Program&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Requires symbolic toolbox&#xA;syms u12 u13 u14 u23 u24 u34 real&#xA;U = [1 u12 u13 u14; 0 1 u23 u24; 0 0 1 u24; 0 0 0 1]&#xA;% U = [1 u12 u13; 0 1 u23; 0 0 1]&#xA;&#xA;syms v1 v2 v3 v4 real&#xA;v = [v1;v2;v3; v4]&#xA;% v = [v1;v2;v3]&#xA;&#xA;syms d1 d2 d3 d4 real&#xA;D = diag([d1,d2,d3,d4]);&#xA;% D = diag([d1, d2, d3]);&#xA;&#xA;syms c positive&#xA;&#xA;%--------------------------------------------------------------------------&#xA;&#xA;P = U * D * U.' + c * (v * v.');&#xA;[Udash, Ddash] = UD(P)&#xA;&#xA;[Ubar, Dbar] = ThorntonRank1Update(U, diag(D), c, v)&#xA;&#xA;update_TestU = simplify(Udash - Ubar, 'seconds', 10)&#xA;update_TestD = simplify(Ddash - diag(Dbar), 'seconds', 10)&#xA;&#xA;update_TestPbar  = simplify((Ubar * diag(Dbar) * Ubar.') - P, 'seconds', 10)&#xA;update_TestPdash = simplify((Udash * Ddash * Udash.') - P, 'seconds', 10)&#xA;&#xA;%--------------------------------------------------------------------------&#xA;&#xA;P = U * D * U.' - c * (v * v.');&#xA;[Udash, Ddash] = UD(P)&#xA;&#xA;[Ubar, Dbar] = ThorntonRank1Downdate(U, diag(D), c, v)&#xA;&#xA;downdate_TestU = simplify(Udash - Ubar, 'seconds', 10)&#xA;downdate_TestD = simplify(Ddash - diag(Dbar), 'seconds', 10)&#xA;&#xA;downdate_TestPbar  = simplify((Ubar * diag(Dbar) * Ubar.') - P, 'seconds', 10)&#xA;downdate_TestPdash = simplify((Udash * Ddash * Udash.') - P, 'seconds', 10)&#xA;&#xA;&#xA;&#xA;%--------------------------------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Reference&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Page 55 of: Bierman, G. J.,  &quot;Factorization Methods for Discrete Sequential Estimation&quot;,  Academic Press, 1977&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also useful was the following paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fletcher, R. &amp;amp; Powell, M. J. D. On the modification of LDLT factorizations Mathematics of Computation, 1974, 28, 1067-1087 (&lt;a href=&quot;http://www.ams.org/journals/mcom/1974-28-128/S0025-5718-1974-0359297-1/S0025-5718-1974-0359297-1.pdf&quot; rel=&quot;nofollow&quot;&gt;PDF&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Complexity&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Without doing a proper FLOPS count, it looks to be in the order of $n^2 + O(n)$.&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="761" LastEditDate="2013-08-28T10:52:28.143" LastActivityDate="2013-08-28T10:52:28.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8349" PostTypeId="2" ParentId="7171" CreationDate="2013-08-28T07:57:16.967" Score="2" Body="&lt;p&gt;After scrutinizing thoroughly the NASTRAN documentation I found the following option to print the stiffness matrix $[K]$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SOL 101&#xA;TIME 20&#xA;DIAG 8&#xA;compile semg&#xA;alter 'kjjz.*stiffness' &#xA;matprn kjjz//&#xA;CEND&#xA;SPC = 1&#xA;LOAD = 1&#xA;SUBCASE 1&#xA;BEGIN BULK&#xA;INCLUDE 'bulk_data.dat'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will print the stiffness matrix into the &lt;code&gt;.f06&lt;/code&gt; file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After printing the stiffness matrix the differential stiffness matrix can be calculated using the approximation above, i.e. dividing the stiffness matrix by the eigenmode: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[K_{G_0}] = \frac{[K]}{\lambda_0}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where  $\lambda_0$ is the already computed eigenvalue (using SOL 105 for example). There will be a different $[K_{G_0}]$ for each eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/7ms7xj4xp06qa5f/finite-element-nastran-how-to-print-the-differential-stiffness-matrix-in-f06.zip&quot; rel=&quot;nofollow&quot;&gt;A sample code is available here&lt;/a&gt; using a Python script to compute the approximated eigenvalue using the equation presented in the question. You have to execute:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python test_01.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Things to be improved here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;how to stop the solution just after the stiffness matrix is printed, to avoid running the SOL 101 (but running it with zero load will be fast)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;how to print the stiffness matrix in a binary file&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4287" LastActivityDate="2013-08-28T07:57:16.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8350" PostTypeId="2" ParentId="8333" CreationDate="2013-08-28T11:15:10.740" Score="11" Body="&lt;p&gt;This is the stochastic root-finding problem, as in &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1921603&quot;&gt;The stochastic root-finding problem: Overview, solutions, and open questions&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5046" LastActivityDate="2013-08-28T11:15:10.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8351" PostTypeId="1" AcceptedAnswerId="8352" CreationDate="2013-08-28T11:42:05.443" Score="2" ViewCount="1280" Body="&lt;p&gt;This is probably a very trivial question but I have not been able to figure it out myself, so here goes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\Gamma$ be a smooth boundary in 2-D divided into $N$ quadratic (3-noded), continuous, finite elements. Consider one such 3-noded element on the boundary $\Gamma$. Let $\phi$ be a continuous function on this boundary. Consider a point $\mathbf{x}$ on $\Gamma$ which resides on one of the finite elements. Now we approximate $\phi(\mathbf{x})$ using the conventional FE &lt;a href=&quot;http://kratos-wiki.cimne.upc.edu/index.php/Shape_Functions&quot; rel=&quot;nofollow&quot;&gt;shape functions&lt;/a&gt;,&#xA;$$&#xA;\phi(\mathbf{x}) = \sum_{j=1}^3 N_j \phi_j&#xA;$$&#xA;where $N_j$ are the polynomial shape functions and $\phi_j$ the nodal values of the function $\phi$ on $\Gamma$. Similarly, we can approximate the normal derivative of the continuous function $\phi$ at point $\mathbf{x}$ on $\Gamma$ as,&#xA;$$&#xA;\frac{\partial \phi(\mathbf{x})}{\partial n(\mathbf{x})}  = \sum_{j=1}^3 N_j \frac{\partial \phi_j}{\partial n_j}&#xA;$$&#xA;where $\frac{\partial \phi_j}{\partial n_j}$ is simply the nodal value of the normal derivative of $\phi$ at node $j$. The question is can we alternatively write,&#xA;$$&#xA;\frac{\partial \phi(\mathbf{x})}{\partial n(\mathbf{x})}  = \sum_{j=1}^3  \frac{\partial N_j}{\partial n_j}\phi_j&#xA;$$&#xA;where the shape functions are differentiated instead of the function $\phi$?&#xA;Is it necessary that the normal (or other) derivatives of a function be modeled with linear combination of the shape functions multiplied with the nodal values of the derivatives?&lt;/p&gt;&#xA;" OwnerUserId="4171" LastActivityDate="2013-08-29T20:34:55.903" Title="approximating normal derivative of a continuous function with shape functions" Tags="|finite-element|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8352" PostTypeId="2" ParentId="8351" CreationDate="2013-08-28T13:41:43.807" Score="6" Body="&lt;p&gt;The way you describe the problem makes no sense, and figuring out how to formulate the problem will likely point you in the right direction towards its solution as well. To explain:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You describe the function $\phi$ as living only on the boundary. But then, $\partial \phi/\partial n$ doesn't make any sense because defining the derivative would require you to know values &lt;em&gt;away from the boundary&lt;/em&gt;. In other words, you can't take the derivative of a function that's only defined on a line in a direction that's perpendicular of the line.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The same is true for your shape functions $N_j(x)$ -- the way you describe them, they are only defined on the boundary and consequently you can't take normal derivatives.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your coefficients $\phi_j$ are just numbers, not functions. Consequently, $\partial \phi_j/\partial n = 0$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-08-28T13:41:43.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8353" PostTypeId="1" CreationDate="2013-08-28T16:57:28.580" Score="1" ViewCount="86" Body="&lt;p&gt;I am trying to use the &lt;a href=&quot;http://cvxr.com/cvx/doc/index.html&quot; rel=&quot;nofollow&quot;&gt;cvx&lt;/a&gt; package for optimization. However, I am having some issues with it. I have a variable X which is a matrix but I cannot add $X^{-1}$ in the objective function. What should I do? Is it even possible to use it or it violates the definition of semi definite programming and hence cannot be included&lt;/p&gt;&#xA;" OwnerUserId="3871" LastActivityDate="2013-08-28T16:57:28.580" Title="Issues with CVX package for optimization" Tags="|optimization|convex-optimization|cvx|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8355" PostTypeId="1" AcceptedAnswerId="8356" CreationDate="2013-08-28T19:20:27.423" Score="11" ViewCount="365" Body="&lt;p&gt;Consider two computers with different hardware and software configurations. When running the exact same serial Navier-Stokes code on each platform it takes x and y time to execute one iteration for computer 1 and 2, respectively. In this case, $\Delta = x-y$, is the iteration time difference between computer 1 and computer 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the impacting the magnitude of $\Delta$? One obvious candidate is the CPU, my main question is whether there are other factors that could be impacting $\Delta$ on the same order as the hardware difference between CPUs?&lt;/p&gt;&#xA;" OwnerUserId="3561" LastEditorUserId="3561" LastEditDate="2013-08-28T19:27:52.727" LastActivityDate="2013-09-02T23:58:00.867" Title="What is the underlying structure of scientific code performance?" Tags="|performance|iterative-method|navier-stokes|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8356" PostTypeId="2" ParentId="8355" CreationDate="2013-08-28T19:37:22.363" Score="13" Body="&lt;p&gt;This list is nowhere near complete, but hopefully the size of it will give a hint as to the scale of possible factors.  I am assuming you are compiling the code from source on your platform of choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Software&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Standard Library Performance&lt;/li&gt;&#xA;&lt;li&gt;Lin. Alg. Library Performance (if the software links to outside libraries)&lt;/li&gt;&#xA;&lt;li&gt;Compiler Choice&lt;/li&gt;&#xA;&lt;li&gt;Compiler Optimization&lt;/li&gt;&#xA;&lt;li&gt;Compiler Flags&lt;/li&gt;&#xA;&lt;li&gt;Background Processes (May vary significantly if OSes are different)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Hardware&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;CPU&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Clock Speed&lt;/li&gt;&#xA;&lt;li&gt;Architecture (the same instruction may take different numbers of cycles on different architectures)&lt;/li&gt;&#xA;&lt;li&gt;Cache Sizes&lt;/li&gt;&#xA;&lt;li&gt;Cache Latency&lt;/li&gt;&#xA;&lt;li&gt;SIMD (Single Instruction, Multiple Data) Capability&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Memory&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of Channels&lt;/li&gt;&#xA;&lt;li&gt;Speed&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;HDD&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Read/Write speed (mostly only important for writing results, so this depends on how often you are writting output to file for a NS solver, but could be important for other programs that do things such as image processing)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is all ignoring the little tricks and features different manufacturers include to give their chips an edge in the market.  The big one though is that many sparse linear algebra libraries are memory bound.  Doing a sparse matrix multiply involves a lot of data moving around without many actual flops.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="541" LastEditDate="2013-08-28T20:26:42.700" LastActivityDate="2013-08-28T20:26:42.700" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8357" PostTypeId="2" ParentId="8332" CreationDate="2013-08-28T21:07:17.603" Score="6" Body="&lt;p&gt;For the &lt;em&gt;how&lt;/em&gt; part referred to in the previous answer, conforming Quad or Hex mesh refinement is most likely going to use an algorithm based on the work of R. Schneiders' 2- and 3- refinement algorithms. These methods are used in mesh generation. Two papers that I happen to have that do adaptive conforming quad refinement are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;A new fast hybrid adaptive grid generation technique for arbitrary two-dimensional domains&quot;  by Ebeida et al, Int. J. Num. Meth. Eng. 84:305-329 (2010) who use 2-refinement algorithms and show time dependent examples, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a preprint &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Conformal Refinement of Unstructured Quadrilateral Meshes&quot; by Garimella, who uses 3-refinement and whose source I don't remember. (Both of these are 2-D, but Schneiders presents 3D algorithms, too.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question refers to applying a conforming algorithm using some local error estimate. The papers I cite show adaptive, local (h-)refinement, but don't say much about the decision of when to subdivide and coalesce. For the spectral approximation, one approach would be to use an element-local interpolation estimate like Cathy Mavriplis did in her dissertation, or an adjoint based estimate like that used by Matt Willyard in his dissertation. These would indicate that an element needs to be refined. You could then apply the conforming h-adaptation as in the papers above to satisfy the prediction that the estimate provides. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The spectral algorithm, however, will prefer to increase the order over subdividing elements if the solution is smooth. That will lead to (functionally) non-conforming meshes locally. If that is the question that is really being asked, then no, I don't see a way of doing conforming order refinement.&lt;/p&gt;&#xA;" OwnerUserId="5049" LastActivityDate="2013-08-28T21:07:17.603" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8358" PostTypeId="2" ParentId="8345" CreationDate="2013-08-29T13:20:27.377" Score="6" Body="&lt;p&gt;&lt;em&gt;Thermal stresses&lt;/em&gt; are &lt;em&gt;self stresses&lt;/em&gt; that arises in two main cases.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If one imposes displacement continuity at the interface between two materials with different thermal expansion subjected to a uniform temperature change;&lt;/li&gt;&#xA;&lt;li&gt;if an homogeneous material is subjected to a non uniform temperature change.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(Here with uniform I mean constant with respect to space, i.e. no gradient.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being self stresses (equilibrated to vanishing external body forces) you &lt;em&gt;cannot&lt;/em&gt; model them as external loads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thermal effects are correctly taken into account if you perform a thermo-mechanical simulation, i.e. solving both for the temperature field and stress field. Of course the stress-strain relation should be written taking into account the inelastic strains due to thermal expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few remarks.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pay great attention to the mechanical boundary conditions: as everyone knows huge stresses may arise in thermally loaded structures subjected to mechanical constraints. So ask your self if the modeled boundary conditions are an accurate representation of your physical prototype.&lt;/li&gt;&#xA;&lt;li&gt;Thermo-mechanical simulations can be uncoupled (first solve for the temperatures, and the for the stresses) or coupled (solve for temperatures and stresses simultaneously). Since you have also electrical behavior to model, accurately analyze how the three field equations are coupled.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using FEM thermal stresses are easily incorporated into the model, provided that thermal expansion is correctly modeled. (Every text book on solid mechanics  and FE analysis should give you the details.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case of $N$ distinct materials each with its own domain $\Omega_i$, $\Omega \equiv \bigcup_{i=1}^{N} \Omega_i$, is simply treated by having &lt;em&gt;continuous&lt;/em&gt; displacement and temperatures across the material interfaces (which is always true for a conforming mesh in $\Omega$) and &lt;em&gt;discontinuous&lt;/em&gt; thermal expansion and elastic constants (which again is simply obtained by assigning to the elements of each subdomain $\Omega_i$ the corresponding material properties). This will result in the correct (discontinuous) thermal stress fields int the whole $\Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No extra or ad-hoc assumptions needed.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2013-09-08T20:49:05.420" LastActivityDate="2013-09-08T20:49:05.420" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8360" PostTypeId="2" ParentId="5534" CreationDate="2013-08-29T14:25:48.103" Score="4" Body="&lt;p&gt;You should definitely check &lt;a href=&quot;https://en.wikipedia.org/wiki/K-d_tree&quot; rel=&quot;nofollow&quot;&gt;K-D trees&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Octree&quot; rel=&quot;nofollow&quot;&gt;octrees&lt;/a&gt; which are the methods of choice for point sets (while BSPs are for general objects, and grids for more or less uniform densities). They can be very compact and fast, minimizing overhead in both memory and computation, and are simple to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When your points are more or less uniformly distributed (even with empty areas, but there must be no density singularity or other high concentration) check sphere packings if you want to try a grid-like non-hierarchical space subdivision.&lt;/p&gt;&#xA;" OwnerUserId="5005" LastActivityDate="2013-08-29T14:25:48.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8361" PostTypeId="2" ParentId="6736" CreationDate="2013-08-29T18:03:33.197" Score="1" Body="&lt;p&gt;I would guess that the main reason for solving the max margin task in the dual is that the dual formulation permits the use of kernel functions. And therefore it is easy to transfer the input points implicitly to some other feature space where the two classes are separable. In your above formulated task (hard margin) there exits only a solution if the two classes where your points come from are linearly separable. If the convex hulls of the points of the two classes intersect then your above task is infeasible. There are two ways to fix this situation: 1) Switch to soft margins by introducing slack variables or 2) map your input points to some other feature space where they are linearly separable and perform your classification there. The second approach can be implicitly done via appropriate kernels e.g. the Gaussian kernel. This is because in the dual task the input points only appear in inner products.&#xA;This can be seen in a more convenient form of the dual&lt;br&gt;&#xA;$\max_{\alpha} \left( \sum_i \alpha_i - \frac{1}{2} \sum_{i,j} y_i y_j\alpha_i\alpha_j\langle x_i,x_j\rangle\right)$ &lt;br&gt; s.t. $\sum_i \alpha_i y_i = 0$ &lt;br&gt; $\qquad \alpha_i\geq 0\quad \forall i$&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more comprehensive/detailed explanation I recommend the book &quot;An introduction to Support Vector Machines and other kernel-based learning methods&quot; by Nello Cristianini and John Shawe-Taylor.&lt;/p&gt;&#xA;" OwnerUserId="4937" LastActivityDate="2013-08-29T18:03:33.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8362" PostTypeId="2" ParentId="8351" CreationDate="2013-08-29T18:19:44.030" Score="0" Body="&lt;ol&gt;&#xA;&lt;li&gt;Your function $ \phi(x)$ depends on $x$ , the nodal values $\phi_j$ do not. This means only the shape functions depend on the position.&lt;/li&gt;&#xA;&lt;li&gt;Same thing applies to the normal vector $n(x)$ and the nodal values $n_j$.&lt;/li&gt;&#xA;&lt;li&gt;I would rather write &#xA;$(\frac{\partial \phi}{\partial n})(x)= \sum\limits_{j=1}^3 N_j(x) (\frac{\partial \phi}{\partial n})_j $&lt;/li&gt;&#xA;&lt;li&gt;Consequently, the answer would be &quot;Yes&quot; to your question&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4964" LastActivityDate="2013-08-29T18:19:44.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8363" PostTypeId="2" ParentId="7142" CreationDate="2013-08-29T19:14:39.477" Score="2" Body="&lt;p&gt;Intuitively for the Gaussian kernel: The decision function of the max-margin task for the two classes $\{-1,1\}$ is the following &lt;br&gt;&lt;br&gt;&#xA;$f_w(x) = \mbox{sign}(\langle x, w\rangle +b)$ &lt;br&gt;&lt;br&gt;&#xA;That is when you want to classify a new observation $x$ you calculate the inner product of $x$ and the parameter vector $w$ and decide for the class according to the sign of the inner product. Given a training set $\{(x_i,y_i)\mid i=1,\dots, L\}$ with $y_i\in\{-1,1\}$ (upon which $w$ is learned) we know from optimisation theory that the parameter vector $w$ can be expressed  as &lt;br&gt;&lt;br&gt;&#xA;$w = \sum_{i}\alpha_i y_i x_i$&lt;br&gt;&lt;br&gt;&#xA;for some $\alpha_i\geq 0$. The index $i$ runs over the elements of the training set . Substituting this into the above we get the decision function&lt;br&gt;&#xA;$f(x) = \mbox{sign}(\sum_i\alpha_i y_i\langle x, x_i\rangle +b)$&lt;br&gt;&#xA;&lt;br&gt;Now if we use the gaussian kernel &lt;br&gt;$k_{Gauss}(x,x_i)=\exp(-\frac{\|x-x_i\|^2}{\sigma^2})$ instead of the linear kernel $k_{linear}=\langle x,x_i\rangle$&lt;br&gt;&#xA;we get &lt;br&gt;&#xA;$f(x) = \mbox{sign}(\sum_i\alpha_i y_i\exp(-\frac{\|x-x_i\|^2}{\sigma^2})+b)$.&#xA;&lt;br&gt;&lt;br&gt;&#xA;Therein you can see that all training elements for which $\alpha_i &amp;gt; 0$ (the support vectors) have influence on the decision, since the element for which $\alpha_i=0$ do not contribute to the sum (this holds for all kernels). What can also be seen is that the support vectors which are close to $x$ give higher contribution to the sum than the ones which are far away because the term $exp(-\frac{\|x-x_i\|^2}{\sigma^2})$ decreased exponentially with the distance of the points $x$ and $x_i$. Especially for small $\sigma^2$ only the very nearby points to $x$ have significant influence on the sum which in the extreme case renders the max-margin classifier into a kind of nearest neighbour classifier. Therefore for a small enough $\sigma^2$ you can classify any given training set (as long as it is consistent) correct, but you might encounter difficulties later because of over-fitting onto the training set. Another thing is that you end up with more support vectors the more &quot;powerful&quot; your kernel is. Hence decreasing $\sigma^2$ gives more support vectors. Up to my knowledge there is no &quot;elegant&quot; way on how to determine the best value for $\sigma^2$. I thing this is mostly done by cross validation. &lt;/p&gt;&#xA;" OwnerUserId="4937" LastActivityDate="2013-08-29T19:14:39.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8364" PostTypeId="2" ParentId="8351" CreationDate="2013-08-29T20:34:55.903" Score="1" Body="&lt;p&gt;Another take: I would guess from your question that you are interested in BE methods for potential problems. If this is the case, $\phi(x)\rvert_\Gamma$ and $\frac{\partial \phi}{\partial n}\bigr\rvert_\Gamma$ are to be discretized on $\Gamma$ as &lt;em&gt;independent&lt;/em&gt; functions, possibly with different shape functions and node positions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the BEM case the answer is therefore no: for the method to work you have a set of shape functions and nodal values for $\phi$ and a distinct set of shape functions and nodal values for $\frac{\partial \phi}{\partial n}$ and you cannot link them by differentiation. The reason was clearly stated in the &lt;a href=&quot;https://scicomp.stackexchange.com/a/8352/1744&quot;&gt;answer&lt;/a&gt; by Wolfgang: let me just add that in BE methods this is not a &quot;problem&quot; but the very core of the procedure.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-08-29T20:34:55.903" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8365" PostTypeId="2" ParentId="8332" CreationDate="2013-08-29T23:57:26.527" Score="1" Body="&lt;p&gt;You can always choose which elements to refine by computing a local error estimator/indicator for each cell, picking a cutoff value, and refining every cell beyond the cutoff. This is independent of whether the refinement is conforming or non-conforming. If you are using a conforming refinement strategy that works on blocks of cells (2x2 to a 3x3 as you mention), then you can choose to refine a 2x2 block of cells if any of the individual cells need refinement, if all of the cells need refinement, or some combination in between. You can also try projecting the cell values onto the nodes and refine nodes instead. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bookkeeping is more complicated, but you can also make a conforming refinement in 3D using basically the same strategy. I.e., you can make a 2x2x2 block of cells into a 3x3x3 block. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real trouble with conforming refinement strategies is that you have these topologically odd elements inserted with potentially lousy shape properties. The question then is what to do when refining one of those elements. Do you just follow the same rule, or do you try to do some tricks to help maintain the shape regularity? Non-conforming refinements don't have these problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, it may be commonplace to refer to them as non-conforming refinements, but most hanging node refinements that I know of, use constraints along the element interfaces to force the refined elements into a conforming space. They &lt;em&gt;look&lt;/em&gt; like non-conforming elements, but for continuous Galerkin schemes, the underlying elements do conform.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-08-29T23:57:26.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8367" PostTypeId="1" CreationDate="2013-08-30T07:20:27.463" Score="4" ViewCount="2611" Body="&lt;p&gt;Is there a way to convert vtk file without visualizing into a image file such as png in the bash terminal itself? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found that for the vtk files that I visualize to take longer time than its corresponding image file (which is made from a different file).&lt;/p&gt;&#xA;" OwnerUserId="4907" LastEditorUserId="4964" LastEditDate="2014-09-05T15:09:21.527" LastActivityDate="2014-09-08T00:23:05.543" Title="VTK file format to image file" Tags="|visualization|vtk|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8368" PostTypeId="2" ParentId="8367" CreationDate="2013-08-30T09:06:42.910" Score="4" Body="&lt;p&gt;I don't know about &quot;in the bash terminal itself&quot;, but I use python scripting with Visit.  See &lt;a href=&quot;http://www.visitusers.org/index.php?title=VisIt-tutorial-Python-scripting&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;http://www.visitusers.org/index.php?title=Python_examples&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for some examples.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below I have pasted a simple script I use to make an image of a 2D slice along y = 0.5 of a 3D cube. The script is named &quot;script.py&quot; and &quot;ysolution&quot; refers to a variable name with the .vtk file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# invoke like so: visit -cli -nowin -s script.py ysolution-19.vtk ysolution&#xA;&#xA;import os,sys&#xA;wholefile = os.getcwd() + &quot;/&quot; + sys.argv[-2]&#xA;fieldname = sys.argv[-1]&#xA;&#xA;OpenDatabase(wholefile)&#xA;AddPlot(&quot;Pseudocolor&quot;,fieldname)&#xA;AddOperator(&quot;Slice&quot;)&#xA;s = SliceAttributes()&#xA;s.originIntercept = 0.5&#xA;SetOperatorOptions(s)&#xA;DrawPlots()&#xA;t = SaveWindowAttributes()&#xA;t.fileName = &quot;y19-&quot;&#xA;SetSaveWindowAttributes(t)&#xA;n = SaveWindow()&#xA;print n&#xA;&#xA;quit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="336" LastActivityDate="2013-08-30T09:06:42.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8369" PostTypeId="2" ParentId="8346" CreationDate="2013-08-30T13:43:10.333" Score="2" Body="&lt;p&gt;I take $p4=1$ and i get the other coefficients as $p_{1}=5.66$,&#xA;$p_{2}=14.58$, $p_{3}=257.4$. So $c(z)$ and $c'(z)$ will be as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c(z)=4800+5.66+(14.58)\frac{z}{1000}+(257.4)exp(-\frac{z}{1000})$&#xA;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c'(z)=(1287exp(\frac{z}{1000}))/5000+729/50000$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $z'=u$ and $z''=u'$ than we may write following differential&#xA;eq.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d^{2}z}{dx^{2}}=\left(\frac{c(2000)}{cos(7.8)}\right)^{2}\frac{c'(z)}{c(z)^{3}},&#xA;\qquad z(0)=2000,\qquad z'(0)=tan(7.8)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;z' &amp;amp; =u,\qquad z(0)=2000\\&#xA;u' &amp;amp; =\left(\frac{c(2000)}{cos(7.8)}\right)^{2}\frac{c'(z)}{c(z)^{3}},\qquad u(0)=tan(7.8)&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the codes for the problem should be as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    function dZ=sys(x,Z)&#xA;    c=@(z)4800 + 5.66 + (14.58)*z/1000+ (257.4)*exp(-z/1000); % c(z)&#xA;    c=c(2000);&#xA;&#xA;% Z(1):=z&#xA;% Z(2):=u&#xA;    dZ=zeros(2,1);    % a column vector&#xA;    dZ(1)=Z(2);&#xA;    dZ(2)=(c/cosd(7.8))^2*((1287*exp(Z(1)/1000))/5000 + 729/50000)/...&#xA;        (4800 + 5.66 + (14.58)*Z(1)/1000+ (257.4)*exp(-Z(1)/1000))^3;&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can call the above function by the following codes and plot the solution&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; x=0:0.5:3000;&#xA;&amp;gt;&amp;gt; [X,Z]=ode45(@sys,x,[2000 tand(7.8)]);&#xA;&amp;gt;&amp;gt; plot(X,Z(:,1),'r',X,Z(:,2),'b')  %Z(:,1) is z(x) and Z(:,2) is z'(x).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You see just z below&#xA;&lt;img src=&quot;https://i.stack.imgur.com/pq7Da.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="4282" LastEditDate="2013-08-31T21:35:54.427" LastActivityDate="2013-08-31T21:35:54.427" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8370" PostTypeId="2" ParentId="8246" CreationDate="2013-08-30T18:13:02.850" Score="3" Body="&lt;p&gt;Convergence of iterative methods are affected by the condition number of the matrix, which tends to increase as the mesh is refined. Benzi has done work showing that improved convergence can be obtained using ILU or approximate inverse preconditioners if one first performs permutations to maximize the diagonal entries of the matrix. Search for &quot;Preconditioning symmetric and highly indeﬁnite problems&quot; by Bollhoffer and &quot;Preconditioning highly indefinite and nonsymmetric matrices&quot; by Benzi et al. They used either the Harwell MC64 algorithm, which I believe free for academic use, or their own variant of the underlying algorithm. Search for &quot;The design and use of algorithms for permuting large entries to the diagonal of sparse matrices&quot; for a description of MC64. Generally, one first reorders to either minimize fill or bandwidth and then permutes the matrix to maximize the diagonal. One can also regularize the matrix using row and column scaling matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a variant of BiCGStab that begins with multiple starting vectors. I am not sure if it will help in this situation. Seach for ML(n)BiCGStab. The author also supplies matlab code at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.uwyo.edu/mathmyeung/r19/index.html&quot; rel=&quot;nofollow&quot;&gt;http://www.uwyo.edu/mathmyeung/r19/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best bet may just be a better more robust preconditioner (e.g., ILUt code from Saad, ILUPACK code from Bollhoffer, Trilinos preconditioners developed at Sandia National Labs, etc). Jack Dongarra of ORNL maintains a nice site with a list of freely available direct and iterative solvers for sparse matrices at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Site includes links to the software's official website.&lt;/p&gt;&#xA;" OwnerUserId="5057" LastActivityDate="2013-08-30T18:13:02.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8371" PostTypeId="1" AcceptedAnswerId="8373" CreationDate="2013-08-30T19:02:04.717" Score="13" ViewCount="1007" Body="&lt;p&gt;The function $f \colon x \mapsto (e^x-1)/x$ has singularity near $x = 0$. That singularity can be lifted, though: for $x = 1$, one should have $f(x) = 1$, since&#xA;$$&#xA;  e^x = \sum_{k=0} \frac{x^k}{k!}&#xA;$$&#xA;and thus&#xA;$$&#xA;  (e^x - 1)/x = \sum_{k=1} \frac{x^{k-1}}{k!}&#xA;$$&#xA;However, the form $(e^x-1)/x$ is not only not defined at $x = 0$, it is also numerically unstable in the vicinity of that point; in order to evaluate $f(x)$ for very small $x$ numerically, one could use a Taylor expansion, i.e. a truncation of the afore-mentioned power series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q&lt;/strong&gt;: Does the function $f$ have a name? In other words, is this a common problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q&lt;/strong&gt;: Is anyone aware of a C/C++ library that handles this situation nicely, i.e. uses the Taylor expansion of an appropriate degree near 0 and the other representation away from zero?&lt;/p&gt;&#xA;" OwnerUserId="3749" LastActivityDate="2017-02-12T16:18:05.133" Title="Compute $(e^x-1)/x$ near $x = 0$" Tags="|c++|c|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8372" PostTypeId="2" ParentId="8371" CreationDate="2013-08-30T19:14:26.120" Score="2" Body="&lt;p&gt;&lt;s&gt;I remember this question having been asked earlier on this site, and surprisingly the answer is that you only need to special-case exact equality to zero.  The errors cancel out near zero.  I don't have the link.&lt;/s&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yeah this answer was completely wrong.  I'm not sure why it was upvoted so much, probably because it was stated so authoritatively.  I found the link I had in mind.  It was on the math stackexchange &lt;a href=&quot;https://math.stackexchange.com/questions/185215&quot;&gt;here&lt;/a&gt;, not on the scicomp stackexchange.  The &lt;code&gt;expm1&lt;/code&gt;-free error cancellation formula is given in the answer by J.M. and uses a &lt;code&gt;u = exp(x)&lt;/code&gt; transformation.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-09-03T15:37:11.243" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8373" PostTypeId="2" ParentId="8371" CreationDate="2013-08-30T20:14:30.553" Score="19" Body="&lt;p&gt;Possibly one could start with the function $\mathtt{expm1}$ which is part of the C99 standard, and calculates $e^x-1$ accurately near $x=0$.&lt;/p&gt;&#xA;" OwnerUserId="1400" LastEditorUserId="1400" LastEditDate="2013-09-01T12:50:41.403" LastActivityDate="2013-09-01T12:50:41.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8374" PostTypeId="2" ParentId="8371" CreationDate="2013-08-30T20:14:40.797" Score="17" Body="&lt;p&gt;This is an instance of cancellation error.  The C standard library (as of C99) includes a function called &lt;code&gt;expm1&lt;/code&gt; that avoids this problem.  If you use &lt;code&gt;expm1(x) / x&lt;/code&gt; instead of &lt;code&gt;(exp(x) - 1.0) / x&lt;/code&gt;, you won't experience this issue (see graph below).&#xA;&lt;img src=&quot;https://i.stack.imgur.com/TqMKn.jpg&quot; alt=&quot;&amp;lt;code&amp;gt;fabs(expm1(x) / x - (exp(x) - 1.0) / x)&amp;lt;/code&amp;gt;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The details and solution of this particular problem are discussed at length in &lt;a href=&quot;http://books.google.com/books?id=epilvM5MMxwC&amp;amp;lpg=PP1&amp;amp;pg=PA19#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;Section 1.14.1 of &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;&lt;/a&gt;.  The same solution is also explained in &lt;a href=&quot;http://www.cs.berkeley.edu/~wkahan/Mindless.pdf&quot;&gt;page 19 of W. Kahan's paper titled &lt;em&gt;How Futile are Mindless Assessments of Roundoff in Floating-Point Computation?&lt;/em&gt;&lt;/a&gt;.  The actual implementation of &lt;code&gt;expm1&lt;/code&gt; in the GNU C library is different from the approach described in the references above and is &lt;a href=&quot;http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/ieee754/dbl-64/s_expm1.c;h=1a4bcd979a5b57d6e6602f93c5c7feea707aed99;hb=HEAD&quot;&gt;thoroughly documented in the source code&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2013-09-07T22:52:11.387" LastActivityDate="2013-09-07T22:52:11.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8375" PostTypeId="2" ParentId="8371" CreationDate="2013-08-30T21:06:28.917" Score="3" Body="&lt;p&gt;To answer your first question, no, the function doesn't have a name (at least not one that is widely known).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others have mentioned, the best way to compute the function is to treat several special cases. This is how any library would compute the function.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Case 0: x = 0, return 1.&lt;/li&gt;&#xA;&lt;li&gt;Case 1: $|x|&amp;lt;\delta$, return $1+x/2$. Choose $\delta$ empirically for your numerical precision and data type. For &lt;code&gt;double&lt;/code&gt;, it should be about &lt;code&gt;2e-8&lt;/code&gt;. For float, it's about &lt;code&gt;5e-4&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Case else: return &lt;code&gt;expm1(x)/x&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can be more sophisticated and special case more things with truncated Taylor series, but it's probably not worth it. In fact, it's not entirely clear that case 1 needs to be handled separately, since as k20 pointed out, the cancellation is safe. However, handling it separately would let me feel more confident about it.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-08-30T21:06:28.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8376" PostTypeId="1" AcceptedAnswerId="8389" CreationDate="2013-08-31T18:13:22.550" Score="6" ViewCount="349" Body="&lt;p&gt;I'm trying to understand the types of numerical errors, to do this I want to calculate $\sin(\pi/2)=1$ numerically. To do this I use the Taylor series of $\sin(x)$ in 0:&#xA;$$\sin(x)=x-\frac{x^3}{6}+\frac{x^5}{120}-\frac{x^7}{5040}+ \ldots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make the calcultaions I did the following Fortran program:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program main&#xA;&#xA;implicit none&#xA;real*8 pi, x, res, error&#xA;&#xA;pi=3.14159265&#xA;x = pi/2&#xA;res = x - x**3/6 + x**5/120 - x**7/5040&#xA;&#xA;error = 1-res&#xA;print*, &quot;Result: &quot;,res&#xA;print*, &quot;Error: &quot;, error&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I get the following results depending of the degreee of the Taylor series and in the decimals of pi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\pi=3.14$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Taylor 3rd Degree:  0.92501782114084341&lt;/li&gt;&#xA;&lt;li&gt;Taylor 5th Degree:  1.0045086615502121&lt;/li&gt;&#xA;&lt;li&gt;Taylor 7th Degree:  0.99984349522599403&lt;/li&gt;&#xA;&lt;li&gt;Taylor 9th Degree:  1.0000032059098956&lt;/li&gt;&#xA;&lt;li&gt;Taylor 11thDegree:  0.99999962708361323&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If $\pi=3.14159265$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Taylor 3rd Degree:  0.92483221907327295&lt;/li&gt;&#xA;&lt;li&gt;Taylor 5th Degree:  1.0045248564076874&lt;/li&gt;&#xA;&lt;li&gt;Taylor 7th Degree:  0.99984310136039689&lt;/li&gt;&#xA;&lt;li&gt;Taylor 9th Degree:  1.0000035425853664&lt;/li&gt;&#xA;&lt;li&gt;Taylor 11thDegree:  0.99999994374102952&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And I observe that the error gets bigger if I put more decimals in my $\pi$ approximation. Why is this happening?&#xA;Edit: Adding more terms to the Taylor expansion (11th degree) makes adding more $\pi$ digits better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, How can I make a better approximation? What method will be better to a problem like this one? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I predict the size of these errors?&lt;/p&gt;&#xA;" OwnerUserId="4458" LastEditorUserId="4458" LastEditDate="2013-09-01T18:06:20.347" LastActivityDate="2013-09-01T18:16:00.280" Title="Computing $\sin(\pi/2)$ numerically" Tags="|error-estimation|approximation|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8377" PostTypeId="1" AcceptedAnswerId="8379" CreationDate="2013-09-01T00:23:01.197" Score="9" ViewCount="337" Body="&lt;p&gt;I am writing a small MATLAB package which will solve a certain class numerical problems. There are 3 stages of the algorithm and the user has 5 choices for each stage. I have implemented the entire problem using $\approx 20$ functions and 3 switch cases (one for each algorithm stage). It works fine but I am considering making it do more things (more than 5 choices and one more stage) and also make a Python port (some people are interested).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if I should convert to a OOP framework (which I am not good at, at all) or whether I should stick with the procedural framework I have (which I am good at). I have made sure in my procedural code that no function does two things and there is minimal overlap (2 code-segments almost never do the same thing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please migrate this to SO if you feel it is more fitting in their domain.&lt;/p&gt;&#xA;" OwnerUserId="5062" LastActivityDate="2013-09-01T13:10:20.000" Title="Can my project survive without Object Orientation?" Tags="|matlab|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8378" PostTypeId="2" ParentId="8377" CreationDate="2013-09-01T00:44:31.867" Score="3" Body="&lt;p&gt;This question might be a good fit for Stack Overflow. I think it's also a good fit here because it's a common issue in computational science.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of composing algorithms, a good example of a library that does this well is PETSc, if you can read C code. Object-oriented programming style may help with encapsulating data, but as a simple first step, you might just want to make each choice for a stage a function with a common interface. Then, for your algorithm, pass in a function for each stage as input and have the main algorithm call the function for each stage.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-09-01T00:44:31.867" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8379" PostTypeId="2" ParentId="8377" CreationDate="2013-09-01T01:35:10.420" Score="6" Body="&lt;p&gt;There is no need to convert to a OOP framework, in order to benefit from object orientation where it helps. Note that the number of functions is less of an indicator than code duplication within and between these functions, or the length of these functions. (A function with more than 100 lines of code would be a typical indication for &quot;improvement potential&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you should be unable to spot the parts where object orientation would help, converting to an OOP framework would not be the best first step. Rather ask somebody familiar with object orientation what could be simplified or improved, and then try to understand why it is a simplification or improvement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use a framework like deal.II or PETSc, use them if the functionality they offer is useful for you, not because you think it will make your own code better or more maintainable. But you are already within the MATLAB framework, so it is unlikely that you want to change to a C++ OOP framework instead. (MATLAB offers full OOP support, as you probably know already.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Python port, just do it based on your current knowledge for a start, and have a second start later after you learned enough to make it significantly better.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2013-09-01T01:35:10.420" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8380" PostTypeId="1" AcceptedAnswerId="8385" CreationDate="2013-09-01T02:22:30.443" Score="2" ViewCount="157" Body="&lt;p&gt;I am currently mulling over the idea of taking a code I currently work with and rebuilding it from the ground up to allow for the use of more efficient programming and numerical techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the early stages of the design I decided I would like to write a wrapper class for matrices and vectors.  This would allow me to (hopefully) switch out lin. alg. libraries and solvers without actually touching the code.  I would simply write a class that has virtual functions for BLAS functionality along with some methods to internalize the initialization (allowing the main code to not worry about initializing in parallel).  Then, when I want to make my code work with a new library, I simply write a new matrix class that implements all those functions using the efficient library functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be working with a finite-volume code, so I would be using sparse matrices with a preknown block memory layout.  I am also looking at approximately a million unknowns although I would like this to not matter (so long as enough memory was available).  The matrix wrapper would also get instructions from a config file to know which library, memory layout, and solvers to use, so it would not have to do any optimization on its own.  It would only check if the requested library was installed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me this seems like a fairly straight forward concept, even if the implementation won't be easy.  As such, my question comes in a couple parts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First off, has this been tried/implemented before, and if so where?  I would much rather be able to grab an open source library that start from nothing.  Any reference or papers would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And secondly, if I implement this sort of system, what sort of issues should I expect?  I can already foresee that making it work transparently with multiple memory models (shared memory, GPU, or distributed memory) will be difficult, but I don't know what other pitfalls to look out for in designing a fairly general linear algebra interface.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-09-01T13:40:35.250" LastActivityDate="2013-09-01T15:49:33.363" Title="General Linear Algebra Wrapper Library" Tags="|linear-algebra|reference-request|c++|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8381" PostTypeId="1" AcceptedAnswerId="8387" CreationDate="2013-09-01T04:56:46.357" Score="3" ViewCount="344" Body="&lt;p&gt;Is there a way to understand what happens when a singular operator is discretized and inverted using the pseudoinverse (say using the SVD Moore-Penrose pseudoinverse)?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we discretize $(\nabla u, \nabla v) $ with a finite element method or something, the operator should usually be singular due to the fact that constants are in the nullspace. Is it known what the pseudoinverse does in such cases - i.e. will it project the solution onto $U/\mathbb{R}$, or does its effect depend other factors such as the choice of basis/discretization? &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-09-01T16:48:23.130" Title="Pseudo-inverse of a discretized operator with a null space?" Tags="|finite-element|linear-solver|discretization|svd|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8382" PostTypeId="2" ParentId="8380" CreationDate="2013-09-01T05:17:09.650" Score="2" Body="&lt;p&gt;Edit: Most of this answer had dealt with dense matrices, I've edited it more for sparse matrices, which is tougher. As far as I can tell, a lot of the issue with sparsity is taking advantage of caching and memory layout the way that you can with dense matrices.  Storage and retrieval of sparse matrices entries is a lot more irregular. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there are a couple examples out there.  There's Matlab and all its open source equivalents (as HighPerformanceMark mentioned), and Eigen is another library you might look at. I know &lt;a href=&quot;http://trilinos.sandia.gov/packages/docs/dev/packages/teuchos/doc/html/index.html&quot; rel=&quot;nofollow&quot; title=&quot;here&quot;&gt;Trilinos&lt;/a&gt; has some examples of this as well - Epetra takes an abstract sparse matrix for iterative and direct solvers. They deal with storage/retrieval issues in memory by having you specify an estimated number of nonzeros per row. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A thought on your note on multiple memory models - there seems to be something like BLAS/LAPACK for most architectures (Eigen, LAPACK/BLAS for serial, ViennaCL, CuBLAS, MAGMA, etc for GPU and OpenMP, Trilinos distributed, etc). It might be possible (though tricky?) to write a matrix class/solver which calls different packages depending on your architecture.  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2013-09-01T13:18:25.040" LastActivityDate="2013-09-01T13:18:25.040" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8383" PostTypeId="2" ParentId="8377" CreationDate="2013-09-01T13:10:20.000" Score="2" Body="&lt;p&gt;Not everything is well modeled by objects.  Consider $1 + 1$, typically an OO system will treat the &lt;code&gt;1&lt;/code&gt;s  as objects and the &lt;code&gt;+&lt;/code&gt; as a method, &lt;code&gt;1.plus(1)&lt;/code&gt;, but this does not correspond to our natural interpretation of $1+1$ as a binary infix function &lt;code&gt;plus(1,1)&lt;/code&gt;; it introduces an unnatural asymmetry between the first &lt;code&gt;1&lt;/code&gt; as being the object for which the method &lt;code&gt;plus&lt;/code&gt; is defined, and the second which is the argument to that method (this gives rise to kludges like &lt;code&gt;__plus__&lt;/code&gt; and &lt;code&gt;__rplus__&lt;/code&gt; in Python).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So don't feel that you need to force the square peg into the round hole, if you algorithm looks good in plain procedural fashion then keep it like that.  If it needs to be included in an OO system you can always put an OO interface on your non-OO code. C++ coders do this every day (and even get paid for it).&lt;/p&gt;&#xA;" OwnerUserId="1400" LastActivityDate="2013-09-01T13:10:20.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8384" PostTypeId="2" ParentId="8381" CreationDate="2013-09-01T15:42:44.927" Score="1" Body="&lt;p&gt;It depends on how you define your pseudoinverse. If you define it as the operator that produces the least-$L_2$-norm element, then you will get that $u=u_0+c$ that has the smallest norm $\|u_0+c\|_{L_2}$ where $u_0$ is an arbitrarily chosen solution and $c$ is a constant. In general, of course, $u$ will not have zero mean value, but it will be unique.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-01T15:42:44.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8385" PostTypeId="2" ParentId="8380" CreationDate="2013-09-01T15:49:33.363" Score="3" Body="&lt;p&gt;deal.II has such wrappers for our own linear algebra implementation, for PETSc and for Trilinos. It's a lot of work to get all of this right because every library provides slightly different abstractions. As far as performance is concerned, deal.II does not use virtual functions since the overhead of calling a virtual function just to read a single element of a matrix or vector is likely non-negligible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation would be to not go down this road. It's simply too much work. Rather, my suggestion would be to carefully choose which library to use and then stick with it. I don't know what you plan on doing with your code, but in all likelihood you're going to want to produce results, not spend forever on implementing interfaces. I say that because among the big two (PETSc and Trilinos), there isn't so much to be gained by making things generic -- they're both very good, and the performance differences are much smaller than what you can gain by changing your algorithms. It seems to me that you can only lose by spending a lot of time making everything generic.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-01T15:49:33.363" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8386" PostTypeId="2" ParentId="8381" CreationDate="2013-09-01T16:44:33.193" Score="1" Body="&lt;p&gt;If your pseudoinverse is defined through SVD, then the corresponding singular vector will span the nullspace in $U_h$, so it exists. But, given the generality of the question, you will have to answer two more questions in order to get the answer to yours:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Does your discrete space actually contain a nullspace? For instance, you could use a finite element space with vanishing values on the boundary (this is not excluded by your current quesation). In that case, your operator would actually be definite.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is the discrete nullspace a subspace of the continuous subspace? I can't come up with an example for the Neumann problem, but if you look at the ample literature on divergence conforming discretizations, you see that indeed there might be a discrete nullspace which is not in the kernel of the continuous operator.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you answer both questions to the affirmative, your finite element is unisolvent, and your discrete subspaces are conforming, the answer is yes. But this does not tell you yet, in what sense it is a projection. That's where the choice of your basis and also of the discrete inner products defining the SVD come into play.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-01T16:44:33.193" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8387" PostTypeId="2" ParentId="8381" CreationDate="2013-09-01T16:48:23.130" Score="3" Body="&lt;p&gt;The Moore-Penrose pseudoinverse of a matrix has the property that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x=A^{\dagger}b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is a least squares solution, and that among all least squares solutions (if there's a nontrivial nullspace of $A$), $x$ will be the least squares solution that minimizes $\| x \|_{2}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to discretizing a PDE and solving the linear system of equations $Ax=b$ that results from that discretization, using the pseudoinverse will get you a least squares solution that minimizes the norm of $x$, but depending on the discretization that you've used (or even a change of basis for the linear system of equations), minimizing the norm of the vector $x$ might not be equivalent to minimizing the norm of the solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easy to construct examples of bases where this doesn't work out.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your change of basis is orthogonal, then you'll actually be OK.  For example, consider using the discrete Fourier transform as a change of basis.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-09-01T16:48:23.130" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8388" PostTypeId="2" ParentId="5493" CreationDate="2013-09-01T17:28:39.783" Score="3" Body="&lt;p&gt;You should also take a look at &lt;a href=&quot;http://mayavi.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;mayavi&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2013-09-01T17:28:39.783" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8389" PostTypeId="2" ParentId="8376" CreationDate="2013-09-01T18:16:00.280" Score="5" Body="&lt;p&gt;It may look like the error is getting larger, but I think you are confusing multiple issues here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first is that you are looking at finite approximating Taylor polynomials $p_3(x), p_5(x), p_7(x)$ evaluated at $x=\pi/2$. Taylor's theorem says that for analytic functions (which the sine is), $p_N(\pi/2)\rightarrow 1$ as $N\to \infty$. Your results appear to confirm this, at least for the small set of examples you consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, you are not considering $p_N(\pi/2)$ but $p_N(x)$ for some $x\approx \pi/2$. Now, you expect that $|p_N(x)-1|$ becomes smaller as $x\to \pi/2$, but there is really no reason why that should be so. All you know is that $p_N(x)\to p_N(\pi/2)$ as $x\to \pi/2$. So what to you looks like the error getting larger is because you think that the error is $|p_N(x)-1|$, but that's not correct. The error is in fact $|p_N(x)-p_N(\pi/2)|$ for which you have not shown results.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-01T18:16:00.280" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8390" PostTypeId="1" CreationDate="2013-09-02T06:56:50.407" Score="0" ViewCount="144" Body="&lt;p&gt;My question seems elementary:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a new numerical method to do fluids dynamics simulation. I use t=0.001s as the time step. But I don´t know how many iterations I need to achieve velocity convergence, because I see the velocity still increases even the iterations was 1000. I want to ask the experienced scientist usually how many iterations do I need and how it is related to time step, typically in &lt;strong&gt;fluids dynamics&lt;/strong&gt; area. I understand it depends on the method you use. But I just want get a general guidance. For example , if I want to model a real world 1 hour, what is the time step and iterations I should use? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="5069" LastEditorUserId="4244" LastEditDate="2013-09-02T13:49:27.450" LastActivityDate="2013-09-02T13:49:27.450" Title="What is the time_step and how many iterations required for convergence purpose in fluids simulation" Tags="|fluid-dynamics|simulation|" AnswerCount="1" CommentCount="1" ClosedDate="2013-09-05T07:50:28.060" ContentLicense="CC BY-SA 3.0" />
  <row Id="8391" PostTypeId="2" ParentId="1274" CreationDate="2013-09-02T07:48:19.137" Score="3" Body="&lt;p&gt;A simple idea for spreading a typical sequential RNG over a decent number of threads is to have a single thread advance the seed as fast as possible and send only every thousandth or so seed out to memory.  Then have each of your other threads pick up one of these spaced out reference seeds and process the 1000 values in that block, i.e. regenerate again the 1000 seeds in the block, generate their psuedorandom draws, and then do whatever other processing your task involves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works because for RNGs that don't compute all that much (LCG is certainly one, but many others should be in this category) the real bottleneck is sending the seeds out to memory (and also probably subsequent processing).  If you run a LCG without sending anything out to memory, the whole thing ought to stay in CPU registers and be extremely fast.  Even for a more complicated RNG, you should stay in L1 cache and be very fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used this very simple approach with an LCG that for legacy reasons we have to keep.  We basically get linear speedup up to the 4-8 threads in a typical multicore workstation.  But now I will try the method from Jack Poulson's answer and hopefully get even faster :).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OTOH, I believe this simple trick should work for other inherently sequential RNGs.&lt;/p&gt;&#xA;" OwnerUserId="5065" LastActivityDate="2013-09-02T07:48:19.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8392" PostTypeId="1" CreationDate="2013-09-02T10:31:28.783" Score="3" ViewCount="115" Body="&lt;p&gt;We're all familiar with traditional least-squares method for constructing a straight line through a set of data points. The question is: suppose I show you a scatter plot which clearly is suggestive of more than one line (suppose two). I need to split the data points into two groups and build a regression line for each. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I split the dataset into two so groups so as to make optimal split? What if I allow to leave out some data points entirely from either dataset? Is there a formal name for this problem in research? Are there any known solutions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="5071" LastActivityDate="2013-09-02T10:31:28.783" Title="Partitioning Data for Multiple Regression Lines" Tags="|optimization|regression|partitioning|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8393" PostTypeId="1" AcceptedAnswerId="8400" CreationDate="2013-09-02T11:15:03.330" Score="5" ViewCount="1649" Body="&lt;p&gt;I am exploring the Method of Lines as a way of time stepping semi-discretised PDEs with ODE time-integration solvers. For an excellent introduction to this technique see the &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines&quot; rel=&quot;noreferrer&quot;&gt;scholarpedia.org&lt;/a&gt; article. A popular ODE solver is VODE from &lt;a href=&quot;http://www.netlib.org/ode/vode.f&quot; rel=&quot;noreferrer&quot;&gt;http://www.netlib.org/ode/vode.f&lt;/a&gt;. It is also possible to access this solver by the &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html#scipy.integrate.ode&quot; rel=&quot;noreferrer&quot;&gt;scipy.integrate.ode&lt;/a&gt; set of solvers if you use Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to solve an &lt;em&gt;extremely&lt;/em&gt; stiff set of non-linear PDEs so I am interested in implicit methods because they offer stability and unrestricted time-step. &lt;strong&gt;I had planned to time-step the nonlinear system using a Newton iteration at every time step. However, I recently became aware of the Method of Lines (MOL).&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MOL technique seems a very powerful technique for time stepping semi-discretised PDEs because one can rely on robust and well tested ODE solver code that are well integrated with modern languages such as Python etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt; How does the standard implicit Newton time-stepping method relate to the implicit Adams-Moulton technique used by the VODE solver?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my current understanding the only difference is that Adams-Moulton is a generalisation of the implicit backwards Euler method which is include previous time points in the time integration up to a require order. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt; Does the algorithm work like this?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Solve the nonlinear system with $\theta$-method to give $u^{n+1}$,&#xA;$$&#xA;w^{\prime}(u^{n+1}, u^{n}) = b_1 F(u^{n+1}) + b_0 F(u^{n})&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Solve the system for the solution variable at the next time step $u^{n+2}$ but include the pervious value and appropriate coefficients to improve the time integration,&#xA;$$&#xA;w^{\prime}(u^{n+2}, u^{n+1}) = b_2 F(u^{n+2}) + \underbrace{b_1 F(u^{n+1}) + b_0 F(u^{n})}_{\text{known}}&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Continue this processes up until the desired order ($N$) at which point we can start dropping the older terms,&#xA;$$&#xA;w^{\prime}(u^{n+k}, u^{n+k-1}) = b_k F(u^{n+k}) + \underbrace{b_{k-1} F(u^{n+k-1}) +   \,\,...\,\, +\, b_{k-N} F(u^{n+k-N})}_{\text{known}}&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 3&lt;/strong&gt; This seems almost identical to the BDF approach, how does these approaches differ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 4&lt;/strong&gt; For stiff nonlinear system of ODEs an implicit scheme (such as Newton iteration time-stepping) seems to be what people recommend. However, for a stiff ODE system why are Backwards Differentiation Formula (BDF) recommended and  not Adams-Moulton?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Perhaps some of these equations don't make sense because I am confused regarding the basic technique, but I will be happy to be put straight. &lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2013-09-04T21:51:13.877" Title="Confusion regarding the Adam-Moulton and Backwards Differentiation Formula (BDF) of the VODE solver" Tags="|nonlinear-equations|method-of-lines|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8394" PostTypeId="1" CreationDate="2013-09-02T11:45:25.433" Score="0" ViewCount="90" Body="&lt;p&gt;DEM: discrete element method.&#xA;XDEM: extended discrete element method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my current project of furnace simulation with granular materials, I am interested in the methods mentioned above. I have not studied these method in the past and now I am looking for general introductory materials. Either monograph or scientific papers would be fine.&lt;/p&gt;&#xA;" OwnerUserId="1152" LastEditorUserId="4244" LastEditDate="2013-09-02T13:49:11.823" LastActivityDate="2013-09-02T13:49:11.823" Title="References on the topic of DEM and XDEM" Tags="|finite-element|reference-request|simulation|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8395" PostTypeId="2" ParentId="8390" CreationDate="2013-09-02T12:10:37.740" Score="1" Body="&lt;p&gt;&lt;strong&gt;Elements of answer:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lots of fluid dynamics methods rely on explicit, time-marching schemes; for this kind of schemes, a classical convergence condition is the Courant-Friedrichs-Lewy condition. It can be understand fairly simply: if you have a wave propagating in a discrete spatial grid, if you want you scheme to remain stable the time for the wave to propagate from one grid point to another should not be shorter than the integration time. Basically, it means that the waves cannot propagate faster than the information (which is reasonable). As you understand from what precedes, this condition is spatial resolution-dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one dimension, is can be express as follow:&#xA;$$&#xA;C = \frac{u \Delta t}{\Delta x} \leq C_{\rm max}&#xA;$$&#xA;$C$ is the Courant number, with $u$ the velocity of your fastest wave, $\Delta t$ your timestep, $\Delta x$ your spatial resolution, and $C_{\rm max}$ a given value which constrains your scheme. Typically, for explicit schemes in fluid dynamics, $C_{\rm max} \sim 1$. Then, for a given resolution $\Delta x$, you can deduce your timestep $\Delta t$.&lt;/p&gt;&#xA;" OwnerUserId="3849" LastActivityDate="2013-09-02T12:10:37.740" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8396" PostTypeId="1" AcceptedAnswerId="8397" CreationDate="2013-09-02T12:48:59.253" Score="7" ViewCount="1539" Body="&lt;p&gt;I want to simulate a nonlinear stochastic differential equation&#xA;$$&#xA;{\rm d}X_t = f(X_t) {\rm d}t + g(X_t){\rm d}B_t&#xA;$$&#xA;where $f,g \in C^{\infty}({\mathbb R}^n ,{\mathbb R})$ and $B_t$ is one-dimensional standard brownian motion. How can I do it by MATLAB? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good simulation file for a similar equation might be helpful.&lt;/p&gt;&#xA;" OwnerUserId="4862" LastActivityDate="2021-08-18T09:26:29.763" Title="Examples of numerical solution of stochastic differential equation(SDE)?" Tags="|numerical-analysis|stochastic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8397" PostTypeId="2" ParentId="8396" CreationDate="2013-09-02T15:17:44.400" Score="12" Body="&lt;p&gt;Check out this paper by &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/S0036144500378302&quot;&gt;Des Higham&lt;/a&gt; and the &lt;a href=&quot;https://github.com/horchler/SDETools&quot;&gt;SDETools MATLAB toolbox&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-09-02T15:17:44.400" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8398" PostTypeId="1" CreationDate="2013-09-02T14:44:41.707" Score="1" ViewCount="227" Body="&lt;p&gt;Looking for some software to deal with 50kx50k sparse matrix applying non-negative matrix factorization. Do you know any? &lt;/p&gt;&#xA;" OwnerDisplayName="kamjel" LastActivityDate="2018-03-13T13:02:46.767" Title="Non-negative matrix factorization for sparse input" Tags="|matrix|matrix-factorization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8399" PostTypeId="1" AcceptedAnswerId="8401" CreationDate="2013-09-02T18:22:49.280" Score="4" ViewCount="4072" Body="&lt;p&gt;I am a student and I came to this question while solving problems regarding the float-points.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a 16 bit floating point number format is given as;&#xA;1 bit for sign&#xA;9 bits for mantissa&#xA;6 bits for exponent.&#xA;&#xA;mantissa is normalized fraction denoted in sign magnitude form and exponent is in 2's complement form.&#xA;base of the system is 2 and and biasing is (2^e-1) - 1. What is the range of positive mantissa?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the ans is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.5 to 1 - 2^-9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the max value of mantissa will be .111111111 ie. 1 - 2^-9. but I am not getting the min value of mantissa. It is given that .100000000 is the min positive mantissa.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;my question is why it is not .000000001? &lt;/p&gt;&#xA;" OwnerUserId="5078" LastActivityDate="2014-01-27T05:33:33.397" Title="range of positive mantissa in given floating-point number representation" Tags="|floating-point|computer-arithmetic|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8400" PostTypeId="2" ParentId="8393" CreationDate="2013-09-02T21:22:03.917" Score="7" Body="&lt;p&gt;To answer your questions in order:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Any implicit method for solving an ordinary differential equation involves solving a system of nonlinear equations. You can do this through variants of Newton's method, successive substitution, full approximation scheme, or any other approach that solves systems of nonlinear equations. (The caveat is, of course, depending on the problem, some methods work better than others, and some may not work at all. The case of successive substitution is a good example.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You are describing, more or less, how some implicit linear multistep methods work. Adams-Moulton and BDF methods are both linear multi step methods. For more background on linear multi step methods, check out the book by Ascher and Petzold, the two-volume work of Hairer and Wanner, Butcher's book, or the book by Lambert (all on numerical methods for ODEs).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;BDF methods have different coefficients than Adams-Moulton methods, and thus, they also have different stability and accuracy properties. The form of BDF methods is different than the equation you mentioned because, in standard form, many of its coefficients are zero. The Wikipedia articles on BDF methods and linear multistep methods illustrate this distinction nicely. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I remember correctly, BDF methods are recommended because they have large stability regions, but they're by no means the only choice for stiff systems. Rosenbrock methods, implicit Runge-Kutta methods, exponential integrators, and general linear methods can be effective as well, depending on the properties of your ODE right-hand side (and the spectrum of its Jacobian matrix), and whether or not you can effectively precondition any of the linear systems you solve during integration. If your stiff system can be decomposed into stiff and nonstiff components (like a reaction-diffusion equation), then implicit-explicit methods can also be worthwhile. The impression I get is that, due to the high-quality implementations of BDF methods out there, they've become the method of choice for people who just want something to work and don't really care about investigating alternatives or understanding their problem more deeply. They are reliable and accurate, but other algorithmic choices may be faster for a given problem. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some final thoughts: VODE is really an old version of the CVODE solver in the SUNDIALS suite, and it might be worth using that instead of VODE to benefit from the algorithmic improvements made over the years (like incorporating inexact Newton methods into the nonlinear solves with good error control). Assimulo is a decent Python interface to SUNDIALS. PETSc also implements many of the methods I mentioned above and has a Python interface (called petsc4py), and Trilinos also implements some of those methods and has a Python interface (called PyTrilinos). If you don't think you need the new features, using scipy and VODE is probably good enough; lots of people still use it. &lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-09-04T21:51:13.877" LastActivityDate="2013-09-04T21:51:13.877" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8401" PostTypeId="2" ParentId="8399" CreationDate="2013-09-02T22:51:19.177" Score="3" Body="&lt;p&gt;You haven't told us the original source of this question, or precisely quoted it, so it's possible that something critical is missing from the statement of the problem.  It's also quite possible that the solution is just wrong.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a likely explanation of your confusion is in the idea of normalized binary floating point numbers with an implicit leading one bit.  This scheme is used most prominently in IEEE-754 floating point.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that although we will only store 9 bits of the mantissa in memory, we will always normalize the exponent so that the first bit of the mantissa is a 1.  We will implicitly assume (without explicitly storing this bit) that the first bit of the mantissa is 1.  This gives us an extra bit in the mantissa at no cost in storage.  The 10 bit mantissas then range from &lt;/p&gt;&#xA;&#xA;&lt;p&gt;.1000000000  (stored in memory as 000000000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;.1111111111  (stored in memory as 111111111)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this scheme, a number with the smallest possible exponent is considered denormalized (or subnormal) so that the implicit leading one bit is not assumed.  This allows for mantissas all the way down to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;.000000001 (note that there are only 9 bits here!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This extends the range of the floating point numbers, but at the cost of precision at the low end.  On many processors, arithmetic on denormalized numbers can be substantially slower than arithmetic on normalized IEEE floating point numbers.  In some cases (this is common on GPU's) the processor will simply round all denormalized results down to 0 rather than doing the extra processing to properly handle them.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-09-02T22:51:19.177" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8402" PostTypeId="2" ParentId="8355" CreationDate="2013-09-02T23:58:00.867" Score="2" Body="&lt;p&gt;First, @Godric's answer is good, but I would suggest you think in terms of $x/y$, not $x-y$, so you don't have to qualify it by the size of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, your question specifically excludes differences in software.&#xA;In my experience, the performance rewards for careful tuning can be large factors, so while you're considering hardware issues, don't forget software issues.&#xA;After all, the hardware can only execute the instructions you give it, and if you give it fewer, it will finish sooner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not to expand on this too much, but for any given problem there is a countable infinity of programs that will solve it.&#xA;Among these, some take less time than all the others, and that is a lower bound.&#xA;Don't assume any program is at &lt;em&gt;or even near&lt;/em&gt; this lower bound if it hasn't been carefully tuned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/2719/1262&quot;&gt;&lt;em&gt;This link explains in detail the unorthodox method I use.&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1262" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-09-02T23:58:00.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8403" PostTypeId="1" CreationDate="2013-09-03T14:13:10.320" Score="9" ViewCount="894" Body="&lt;p&gt;I have $n$-dimensional matrices $\mathrm{\hat{H}}(\vec{k})$ depending on vector parameter $\vec{k}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, eigenvalue routines return eigenvalues in no particular order (they are usually sorted), but I want to trace eigenvalues $E_i$ as smooth functions of $\vec{k}$. Because eigenvalues are not returned in any particular order, just tracing $E_i$ for some particular index $i\in\{1,..,n\}$ will return set of lines which are not smooth, as shown on the picture bellow&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wAAyO.png&quot; alt=&quot;band structure&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea to trace continuous lines was to use eigenvectors. Namely, for two close points $\vec{k}$ and $\vec{k}+d\vec{k}$ eigenvectors should be approximately orthonormal so that $v_i(\vec{k})\cdot v_j(\vec{k}+d\vec{k})\sim \delta_{p_i p_j}$ where $p_i, p_j\in \pi(\{1,...,n\})$, and $\pi$ is some permutation. Then I would use given permutation to reorder the eigenvalues and thus trace smooth lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I other words, I would trace continuity of eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I run into some problems with numerical routines. At a given small subset of points I use, few eigenvectors at nearby points are not almost orthonormal. My first suspicion was that those eigenvectors correspond to a degenerate eigenvalue, but that is not always true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also holds true if I reduce $d\vec{k}$ to be really small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is such thing allowed to happen. Or, is it possible to guarantee that numerical routines return continuous eigenvectors? Routine I use is numpy.linalg.eigh which is an interface for zheevd from LAPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Physicists amongst you will recognize that I am talking about the band structure)&lt;/p&gt;&#xA;" OwnerUserId="5082" LastActivityDate="2013-09-04T02:24:56.337" Title="Continuity of eigenvectors of parametric matrix" Tags="|linear-algebra|matrix|eigensystem|numpy|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8404" PostTypeId="1" AcceptedAnswerId="8405" CreationDate="2013-09-03T15:13:13.593" Score="21" ViewCount="19373" Body="&lt;p&gt;I need to write some data from a computation, that will be read later by Paraview (.vtu or vtk file).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to file size , should I go for the ASCII format or the Binary format ?&lt;/p&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2014-09-05T10:17:57.437" LastActivityDate="2020-05-29T17:55:40.957" Title="Binary vs. ASCII file size" Tags="|visualization|paraview|vtk|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8405" PostTypeId="2" ParentId="8404" CreationDate="2013-09-03T15:22:25.693" Score="24" Body="&lt;p&gt;If your only worry is file size, then you want binary files.  For an illustrative example, lets assume you are writing 1 double precision floating point number to a file.  Let's assume that the file system can handle this perfectly and holding the file, headers, and padding are all 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a binary file, that number would take the exact size of the number in RAM, or 8 bytes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In ASCII format, it would hold:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;16 digits of the base  &lt;/li&gt;&#xA;&lt;li&gt;1 period for the decimal  &lt;/li&gt;&#xA;&lt;li&gt;1 char to delimit the exponent  &lt;/li&gt;&#xA;&lt;li&gt;1 char for the sign of the exponent  &lt;/li&gt;&#xA;&lt;li&gt;2-3 char for the exponent&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Assuming it uses only 1 byte for a character, That is 22 bytes to hold the same number.  This doesn't count the characters required to dilimit between numbers (usually atleast 1).  Therefore file size for ASCII format will be about 3 times larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can trade in file size for the precision in the stored files (only keep 5-6 digits in the base), but that depends on what you are using them for.  The main advantage of ASCII is for debugging or producing human readable data.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-09-03T16:34:04.320" LastActivityDate="2013-09-03T16:34:04.320" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8406" PostTypeId="2" ParentId="8403" CreationDate="2013-09-03T17:41:11.220" Score="2" Body="&lt;p&gt;I work in electromagnetics, so I have to compute photonic band structures. I used to try to make the bands smooth by trying to detect crossover points, but after many attempts and discussions with coworkers, we ultimately concluded there is no really good way or reason to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, if you still insist on doing what you want, you want to look at computing eigenvalue derivatives with respect to k. There is quite a bit of literature on this, mainly on perturbation theory of eigenvalue problems (classic book by Kato), and also work on perturbation analysis in the presence of eigenvalue degeracies (a much harder problem, literature by Roger C E Tan). I would try to do this for the nondegenerate case first, since that's still relatively easy.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-09-03T17:41:11.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8407" PostTypeId="2" ParentId="1274" CreationDate="2013-09-03T18:25:46.847" Score="2" Body="&lt;p&gt;This answer is late in coming, but you should have a look at &lt;a href=&quot;http://www.sprng.org/&quot; rel=&quot;nofollow&quot;&gt;SPRNG&lt;/a&gt;. It's specifically designed for scalability in parallel and supports a handful of types of PRNGs.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-03T18:25:46.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8408" PostTypeId="2" ParentId="8403" CreationDate="2013-09-04T02:24:56.337" Score="3" Body="&lt;p&gt;At points where two lines merge, you have that two eigenvalues are the same and consequently that the eigenspace that corresponds to these two eigenvectors is two-dimensional. What this means is that at that point, the two eigenvectors are no longer unique (not just up to a sign) but can be any of the infinitely many possible orthogonal vectors that span this two-dimensional space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have you tried continuation methods that trace individual lines over different values of $k$?&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-04T02:24:56.337" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8409" PostTypeId="2" ParentId="8404" CreationDate="2013-09-04T02:28:46.120" Score="18" Body="&lt;p&gt;In practice, you rarely need data in visualization files that's more accurate than, say, 3 valid digits. In that case, ASCII is -- maybe surprisingly -- often more compact than binary form. If you're thinking about archiving, then bzip-ing these ASCII files is likely going to yield the smallest files you can get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, Paraview reads VTU format which has a compressed binary form (XML-based, but the data is first libz-compressed and then uuencoded again to yield ASCII text). On typical files, this saves a factor of 4-10. For large files, this is definitely the way to go.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-09-05T02:09:37.447" LastActivityDate="2013-09-05T02:09:37.447" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8411" PostTypeId="1" AcceptedAnswerId="8414" CreationDate="2013-09-04T08:09:49.787" Score="6" ViewCount="1102" Body="&lt;p&gt;Consider a method (e.g., FEM) with variable approximation order $p$. Now, we know that the optimal order of convergence is given by&#xA;$$e = C h^{p+1},$$&#xA;where $h$ denotes the mesh size and a constant $C$ that does not depend on $h$. As a result, we can study the &lt;em&gt;experimental order convergence&lt;/em&gt; (or &lt;em&gt;$h$-convergence&lt;/em&gt;) on a series of grids $h, \frac{h}{2}, \ldots$ because you can directly compare two errors $e_1$ and $e_2$ on different grids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I've always been puzzled by the way people perform and evaluate &lt;em&gt;$p$-convergence&lt;/em&gt; studies in this context. Here, we have a fixed grid with mesh size $h$, but different approximation orders $p=1,2,3,\ldots$. My problem now is that the constant $C$ does not depend on $h$, but it &lt;strong&gt;does&lt;/strong&gt; depend on $p$ (otherwise, we would see that the lines in a log-log-plot of the $h$-convergence results for different values of $p$ would intersect in a single point - which they, in my experience, don't do). Now, what is the significance of a $p$-convergence study having realized that we cannot really compare two errors, $e_1$ and $e_2$, using the above formula for the error, given that they have been obtained using different polynomial degrees?&lt;/p&gt;&#xA;" OwnerUserId="3738" LastEditorUserId="4244" LastEditDate="2013-09-04T15:33:55.013" LastActivityDate="2013-09-04T16:12:44.290" Title="Significance of p-convergence studies" Tags="|finite-element|convergence|numerical-analysis|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8412" PostTypeId="2" ParentId="8411" CreationDate="2013-09-04T11:55:52.897" Score="0" Body="&lt;p&gt;This might be answering a different question that what you asked, but I think a chart like that is more about calling out to a reader whether p-adaption or h-adaption is more effective for converging the underlying model. Although the x-axis is often taken to be mesh granularity (h) or number of unknowns, if you normalize it to something that quantifies &quot;work&quot; (e.g. RAM consumption, flops required, elapsed wallclock), then you could gather at a glance whether p-refinement or h-refinement is the better strategy to hit a given error bound with minimum &quot;work&quot;. Certainly this is problem dependent (in particular, whether or not the underlying fields are smooth or singular). But a couple of graphs like that (&quot;work&quot; vs error) could immediately demonstrate e.g. how coarse-h high-p is a better strategy on smooth problems, but fine-h low-p is better on say a singularity-dominated re-entrant corner. (Granted, on real applications that mix those two regimes, the answer itself is mixed - hence the motivation towards adaptive refinements)&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2013-09-04T11:55:52.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8413" PostTypeId="2" ParentId="8411" CreationDate="2013-09-04T14:31:28.867" Score="2" Body="&lt;p&gt;You can conduct your $p$ convergence experiments for different $h$. Since the &quot;constant&quot; $C$ depends on $p$, but not on $h$, this allows you to separate the effects of the $h^{p+1}$ and the $C(p)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The estimate of the form $C(p) h^{p+1}$ is justified by the fact that the $C(p)$ is typically polynomial in $p$, and thus for small $h$ and large $p$, the estimate is dominated by the exponential term.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-04T14:31:28.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8414" PostTypeId="2" ParentId="8411" CreationDate="2013-09-04T14:42:59.623" Score="10" Body="&lt;p&gt;It isn't so much that we want to compare the $p$-refinement and $h$-refinement errors directly, instead we want to compare the convergence properties (e.g. speed) of each refinement strategy. This requires more knowledge of the constant in the apriori error estimate. We'll illustrate by looking at the apriori error estimate of a discontinuous Galerkin spectral method (which can be interpreted as a high-order FEM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that the constant $C$ in the inequality $$ e \leq Ch^{p+1}, $$ depends on the approximation order $p$. Thus, it is not immediately clear what to expect when we increase the order $p$. However, the error inequality above hides a lot of information about the convergence behaviour in that constant. In $p$-refinement it becomes important &lt;em&gt;how&lt;/em&gt; the $C$ depends on $p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a 1D spectral element method the error estimate on a given element $k$ has a form (derived in a similar fashion as in &lt;a href=&quot;http://books.google.com/books/about/Spectral_Methods.html?id=DFJB0kiq0CQC&quot;&gt;Canuto et. al.&lt;/a&gt;)$$ e \leq \mathcal{C}h^{\min(m,p+1)}p^{\frac{3}{2}-m}\|q^k\|_{H^m(\Omega)}, $$&#xA;where $\mathcal{C}$ is independent of $h$ and $p$, $q^k$ is the true solution on element $k$, and $H^m$ is the $m$ order Sobolev space. In we apply the numerical method to a problem with a &quot;smooth&quot; solution it will generally be true that $p+1\leq m$. Now we can more readily see the difference in $p$ versus $h$ refinement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we fix $p$ then the terms $$ p^{\frac{3}{2}-m}\|q^k\|_{H^m(\Omega)} $$ are a constant that we absorb into $\mathcal{C}$ to obtain the $C$ in the first error estimate. Now we can perform a standard $h$-refinement argument and see that the convergence will be similar to that of a finite difference scheme, i.e., some polynomial order on $h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if we fix the grid $h$ then the term $h^{\min(m,p+1)}$ is constant and the error estimate looks like $$ e \leq \hat{\mathcal{C}}p^{\frac{3}{2}-m}\|q^k\|_{H^m(\Omega)},$$&#xA;where $\hat{\mathcal{C}} = \mathcal{C}h^{\min(m,p+1)}$. Now we immediately see that the rate of convergence changes in the $p$-refinement study. Provided $m&amp;gt;\frac{3}{2}$ the error decays as a power of $p$. For &quot;very smooth&quot; solutions, e.g. infinitely differentiable, the decay in the error is faster than any polynomial order. Sometimes this is referred to as exponential convergence because if you look at a semilog plot of the error against $p$ you see a straight line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a quick aside we note that in spectral methods the convergence rate depends on the smoothness of the solution. Heuristically, we can interpret this as &quot;the larger $m$ is in the error estimate, the smoother the solution,&quot; because the true solution is in a higher order Sobolev space, i.e., the true solution is square integrable up to the $m^{\textrm{th}}$ order derivative. The convergence of a Sobolev norm is often viewed in terms of the decay of Fourier coefficients of the solution $q^k$ (particularly for fractional derivatives). This is one reason the family of methods are referred to as spectral methods.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastEditorUserId="1057" LastEditDate="2013-09-04T16:12:44.290" LastActivityDate="2013-09-04T16:12:44.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8415" PostTypeId="1" AcceptedAnswerId="8416" CreationDate="2013-09-04T17:53:22.003" Score="3" ViewCount="141" Body="&lt;p&gt;What is the best way to handle a constraint of the type $ax_1^2+x_2^2+...+x_n^2=c$ in a gradient descent algorithm? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to solve an optimization problem of the type:&#xA;$$ \min J(x_1,..,x_n)$$ &#xA;with this constraint using gradient descent. I tried to think of a way to alter the gradient so that it doesn't affect the constraint, but I didn't manage to do something useful. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What I did was that after each gradient descent, I projected (in a more or less correct way...) the result onto the constraint. I got the results I wanted, but I don't think my approach was 100% valid...&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="3548" LastEditDate="2013-09-04T17:59:37.800" LastActivityDate="2013-09-04T21:49:29.197" Title="What's the best way to handle a quadratic constraint" Tags="|constraints|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8416" PostTypeId="2" ParentId="8415" CreationDate="2013-09-04T21:49:29.197" Score="3" Body="&lt;p&gt;Yes, that's a valid way of incorporating constraints into nonlinear programming problems. These types of algorithms are called &lt;em&gt;reduced gradient&lt;/em&gt; or &lt;em&gt;gradient projection&lt;/em&gt; methods. The method of that type I'm most familiar with is the &lt;em&gt;generalized reduced gradient&lt;/em&gt; (GRG) method, and there are other methods out there as well.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-09-04T21:49:29.197" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8418" PostTypeId="2" ParentId="6875" CreationDate="2013-09-05T21:48:10.773" Score="-1" Body="&lt;p&gt;Apologies for not adding to the discussion for a while - but I would be glad to post the abstract of the paper - but also gladly go into the reasons why the computations are organized differently than standard KF of EnKF approaches &lt;/p&gt;&#xA;" OwnerUserId="4490" LastActivityDate="2013-09-05T21:48:10.773" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8419" PostTypeId="1" AcceptedAnswerId="8424" CreationDate="2013-09-06T10:35:58.970" Score="3" ViewCount="4098" Body="&lt;p&gt;I have two physical properties which are exact (no noise). If I perform a polynomial regression of any degree, all coefficients are statistically significant. The problem is that the residuals are always &quot;sinusoidal&quot;. For me, it probably means that a polynomial is not the adequate function to fit but I did not find better up to now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this systematic trend of residuals indicate something I am missing ?&lt;/p&gt;&#xA;" OwnerUserId="5095" LastActivityDate="2013-09-08T12:49:31.507" Title="Information from residuals in polynomial regression" Tags="|statistics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8420" PostTypeId="1" AcceptedAnswerId="8451" CreationDate="2013-09-06T15:05:01.603" Score="7" ViewCount="660" Body="&lt;p&gt;When applying Galerkin method, we have two conventions, i.e. multiply the test function $v$ at left/right, $(v,u)/(u,v)$.&#xA;Both ways won't matter for a simple problem like Poisson's equation, since the stiffness matrix $K$ and Mass matrix $M$ are symmetric, and the transpose doesn't change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if there is also first order derivative in the differential equation, then the corresponding assembled matrix is non-symmetric for $(v,du)/(du,v)$,  each one is the transpose of the other, then it is a non-unique problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I got wrong at some point, anyone could help?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="20688" LastEditDate="2018-06-19T13:20:37.643" LastActivityDate="2018-06-19T13:20:37.643" Title="A problem in 1D linear finite element method" Tags="|finite-element|numerics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8421" PostTypeId="2" ParentId="8420" CreationDate="2013-09-06T15:44:39.020" Score="1" Body="&lt;p&gt;We usually write linear algebra equations as $Au=b$, which can be rewritten to&#xA;\begin{equation}&#xA;v^T A u = v^T b \qquad \forall v\in \mathbb R^n.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this, you see that the convention in linear algebra is the test function on the left, while in PDEs, we have it on the right. Thus, for a nonsymmetric bilinear form, the safe bet for integrating the matrix entries is&#xA;\begin{equation}&#xA;a_{ij} = a(u_j, v_i).&#xA;\end{equation}&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-06T15:44:39.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8422" PostTypeId="2" ParentId="8420" CreationDate="2013-09-06T16:57:28.360" Score="4" Body="&lt;p&gt;&lt;em&gt;&lt;strong&gt;Rule of thumb&lt;/em&gt;&lt;/strong&gt;: The rows in the finite element matrix corresponds to the test function that's been multiplied to the PDE, while the columns in the finite element matrix corresponds to the weak solution (trial function). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long answer: Say we consider the convection term in the diffusion-convection-reaction equation in a bounded open domain $\Omega$, with a maybe not divergence free convection vector $\mathbf{b}$:&#xA;$$&#xA;-\nabla\cdot(A\nabla u) + \underbrace{\mathbf{b}\cdot \nabla u}_{\text{Convection}} + cu = f,&#xA;$$&#xA;with zero Dirichlet boundary condition so that the trial function space for $u$ coincides with the test function space for $v$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't matter you multiply the test function $v$ on the left or right:&#xA;$$&#xA;\int_{\Omega} v( \mathbf{b}\cdot \nabla u), \quad \text{compare with } \;&#xA;\int_{\Omega} (\mathbf{b}\cdot \nabla u)v,\tag{1}&#xA;$$&#xA;the corresponding matrix entry will be the same if $v$ is test function. If you perform integration by parts, the convection term will be different:&#xA;$$&#xA;-\int_{\Omega} \nabla \cdot(\mathbf{b}v) u, \quad \text{or } \;&#xA;-\int_{\Omega} u \nabla \cdot(\mathbf{b}v),\tag{2}&#xA;$$&#xA;If we let $u = \sum_{j=1}^N u_j\phi_j$, where $\phi_j$ is the basis function in your finite element space, plugging $u$'s expression into (1), and choose $v = \phi_i$ as the test function to get the $i$-th row the linear equation system:&#xA;$$&#xA;\int_{\Omega} A\nabla\Big( \sum_{j=1}^N u_j\phi_j\Big) \cdot \nabla \phi_i + &#xA; \int_{\Omega} \left( \mathbf{b}\cdot \nabla \Big( \sum_{j=1}^N u_j\phi_j\Big)\right)\phi_i + &#xA;\int_{\Omega} c \Big( \sum_{j=1}^N u_j\phi_j\Big)\phi_i= \int_{\Omega} f\phi_i.&#xA;$$&#xA;The matrix $B$ corresponding to the second convection term has entry:&#xA;$$&#xA;B_{ij} = \int_{\Omega} \phi_i ( \mathbf{b}\cdot \nabla \phi_j) = \int_{\Omega} ( \mathbf{b}\cdot \nabla \phi_j)\phi_i. &#xA;$$ &#xA;It doesn't matter you multiply the test function on the left or on the right. The index in the $u$ corresponds to the column, while the index in $v$ corresponds to the row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it does matter the differentiation is performed on which function, the trial function $u$ or the test function $v$. If you use (2) as your convection term in the bilinear form, the matrix $B$ will be different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that you confuse this difference with the &quot;multiply the test function on left or right&quot;. Because in 1D, using the notation in your comment, $N N_x$ is matrix generated by (2), for this is performing the differentiation on the test function $v$. $N_x N$ is the matrix generated by (1), this is performing the differentiation on the trial function $u$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you may confuse &quot;multiply the test function on left or right&quot; with the formalism of a non-symmtric bilinear form. In case the bilinear form for convection term is already defined as :&#xA;$$&#xA;b[u,v] := \int_{\Omega} v( \mathbf{b}\cdot \nabla u).&#xA;$$&#xA;Notice if you switch $u$ and $v$, it is&#xA;$$&#xA;b[v,u] := \int_{\Omega} u( \mathbf{b}\cdot \nabla v)&#xA;\\&#xA;=-\int_{\Omega} \nabla\cdot(\mathbf{b}u) v+ &#xA;\int_{\partial \Omega} v(\mathbf{b}u)\cdot \mathbf{n}\,dS&#xA;\\&#xA;=-\int_{\Omega} (\nabla\cdot\mathbf{b}) u v-\int_{\Omega} (\mathbf{b}\cdot\nabla u) v &#xA;+ \int_{\partial \Omega} v(\mathbf{b}u)\cdot \mathbf{n}\,dS&#xA;\\&#xA;= -b[u,v]-\int_{\Omega} (\nabla\cdot\mathbf{b}) u v &#xA;+ \int_{\partial \Omega} v(\mathbf{b}u)\cdot \mathbf{n}\,dS.&#xA;$$&#xA;which is totally different, even after integration by parts.  This is not about the position where we multiply this test function $v$. Like your said in the comments, if using &quot;left&quot; corresponds to column, &quot;right&quot; corresponds to row, to assemble the finite element matrix, you will get nowhere near correct matrix if divergence of $\mathbf{b}$ is not zero, also boundary terms doesn't vanishes if there exists Neumann boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe the minus sign in (2) makes you think this matrix is anti-symmtric, however it is not. Because the entry on its diagonal is not zero in general:&#xA;$$&#xA;B_{ii} = \int_{\Omega} \phi_i ( \mathbf{b}\cdot \nabla \phi_i) \neq 0.&#xA;$$&#xA;A matrix with non-zero diagonal can't be anti-symmetric.&lt;/p&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2013-09-09T14:21:43.433" LastActivityDate="2013-09-09T14:21:43.433" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8423" PostTypeId="1" AcceptedAnswerId="8457" CreationDate="2013-09-06T21:34:50.670" Score="1" ViewCount="506" Body="&lt;p&gt;I trying to find a computing algorithm for the &lt;em&gt;Significant wave height&lt;/em&gt; $H_{1/3}$, or $H_\text{s}$ or $H_\text{sig}$ in the time domain.&#xA;This is defined as the average height of the one-third part of the measured waves—which are $N$ in number—having the largest wave heights:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;H_{1/3} = \frac{1}{\frac13 N} \sum_{m=1}^{\frac13 N} H_m&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example set consider $\{100,300,400,400, ..\}$ of height values $H_m$. &lt;/p&gt;&#xA;" OwnerUserId="5100" LastEditorUserId="1744" LastEditDate="2013-09-06T22:10:54.017" LastActivityDate="2013-09-09T17:50:30.343" Title="Significant wave height Algorithm" Tags="|algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8424" PostTypeId="2" ParentId="8419" CreationDate="2013-09-06T21:41:05.683" Score="4" Body="&lt;p&gt;This will probably be a long answer, but stick around. It is a good question that I think many have tripped over once in their careers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are seeing is not at all uncommon. In fact, it is to be expected. By the way, it is arguably not terribly appropriate to use significance tests on the coefficients, when there is no randomness in the data anyway. We use regression models in places where they are not really appropriate, and then it seems natural to apply methods based on statistical tests. But there is no randomness here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets look at an example to understand what happens. I'll do it in MATLAB, using some of my own tools. Try fitting an exponential function, by sampling 101 uniform data points from 0 to 1. The nice thing about my choice of function is the series for it is known and easily written down.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = 0:.01:1;&#xA;Y = exp(X);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is no error at all. So the residuals in the fit will be purely lack of fit.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mdl = polyn2sympoly(polyfitn(X,Y,'constant, x'));&#xA;P = polyn2sympoly(mdl)&#xA;P =&#xA;    0.874119292489012 + 1.69114271520557*x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the first order truncated Taylor series model would have been simply 1+x. But a first order linear regression model through those points will miss above and below the curve. In fact, it will tend to cross the curve in 2 points, and the residuals will tend to be fairly quadratic looking. Since we know the actual Taylor series, the residuals as y-yhat will be...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(1 + x + x^2/2 + x^3/6) - P&#xA;ans =&#xA;    0.125880707510988 - 0.691142715205567*x + 0.5*x^2 + 0.166666666666667*x^3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So in general, the residuals will be a function that will tend to be dominated by the highest order truncated term, in this case, the quadratic term. That does not mean the constant and linear terms were estimated exactly &amp;amp; correctly compared to the Taylor series. If fact, they will not be. A plot of the residuals should make this clear.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/2B46F.jpg&quot; alt=&quot;enter image description here&quot;&gt;&#xA;Jumping ahead to a higher order model, here is a 4th order polynomial model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mdl = polyfitn(X,Y,'constant, x,x^2,x^3,x^4');&#xA;P = polyn2sympoly(mdl)&#xA;P =&#xA;    1.00004767172716 + 0.998491471851398*x + 0.510486867080189*x^2 + 0.139707694443945*x^3 + 0.0694961565648233*x^4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that the lower order coefficients are now starting to look like the known values from the Taylor series, and by the way, all of those coefficients will be highly significant. I'd expect them to be so, as the Taylor series had no zero coefficients. So if I look at the ratio of the parameter coefficients to their approximate standard errors, I get unsurprisingly large numbers.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mdl.Coefficients./mdl.ParameterStd&#xA;ans =&#xA;  119272.343263854  8504.47662299035  1061.5504181253  192.89918511997  193.456498197328&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, look at a residual error plot.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/lLQ1w.jpg&quot; alt=&quot;enter image description here&quot;&gt;&#xA;It should be no surprise if this looks vaguely sinusoidal. Again, I expect the residual curve to cross the x axis in 5 points for a 4th order model. In general, n+1 points for an n'th order polynomial model. It simply MUST look vaguely sinusoidal!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I have done something differently? I think so. While it makes some distorted sense to fit a polynomial model to the curve, polynomials are rarely a good choice for these tasks. They tend to have numerical problems. High order models tend to go crazy if you are not careful. And low order models are simply not sufficiently accurate when you have essentially exact data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when is it a good idea to use a polynomial regression model? Polynomials DO have their uses. Low order polynomial models are nice if you need only encapsulate the basic behavior of your data, in a local region where the low order model is sufficiently accurate. They make the most sense when your data is so noisy that you cannot estimate more than a few terms. Too often people end up then going to a higher order model. After all, any well behaved function will have a series representation, so a polynomial model of some order SHOULD suffice! High order polynomials just have so many problems, with the fit, with accurate evaluation, with extrapolation. They even have what I call &quot;intrapolation&quot; problems, where we see large oscillations or ringing inside the domain of the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better choice of model is often a spline of some ilk. Use an interpolating spline, or smoothing spline, or a regression spline. The exact choice should depend on your goals for this model and what you know about the function. Is there any noise at all to be smoothed out? If so, then you might need a smoothing or regression spline. Can you afford to have many knots in the spline, and can't afford any residual error at all? Then an interpolating spline is arguably right. I've had cases where even though the data was known to have noise in it, the client could not afford to modify the shape of the curve in a specific highly curved region, needing to pin down the shape of that spline carefully. In that case, an interpolating spline was indicated despite the presence of known noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A regression spline fit with 5 knots to the exponential yields reasonably small residual errors, however note that the residuals still have a sinusoidal shape to them. Always look at the Y axis scaling though. The limits are +/- 0.000015, so reasonably tight. The vertical bars show the equally spaced knot locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/De6Do.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, while the residuals still show a pattern, if they had any lower order shape to them, the regression model could have removed it, thus reducing the sum of squares of the errors! And since the data had no noise in it, I cannot expect to see a noisy looking residual plot as we see in traditional regression models on noisy data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you saw exactly what I'd expect to see. In hindsight, it is the only way the residuals could logically look.&lt;/p&gt;&#xA;" OwnerDisplayName="user840" LastEditorDisplayName="user840" LastEditDate="2013-09-08T12:49:31.507" LastActivityDate="2013-09-08T12:49:31.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8425" PostTypeId="1" CreationDate="2013-09-07T11:10:24.820" Score="5" ViewCount="2160" Body="&lt;p&gt;What I am trying to solve is the following&#xA;Rayleigh-quotient-like minimization:&#xA;\begin{eqnarray}&#xA;\begin{split}&#xA;(P_0)\quad\min_x \frac{\left( Ax - b\right)^\top \left( Ax - b\right)}{x^\top&#xA;x}\\&#xA;s.t. \quad\quad Ax-b \geq 0&#xA;\end{split}&#xA;\end{eqnarray}&#xA;where $A\in R^{m\times n}$, $m&amp;gt;n$, is a full rank matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is well known that the solution of the Rayleigh-quotient&#xA;minimization is the eigen vector correponding to the minimum eigen&#xA;value. It seems that it cannot be solved using the same approach as the Rayleigh-quotient&#xA;minimization. &#xA;Thanks very much!&lt;/p&gt;&#xA;" OwnerUserId="5102" LastEditorUserId="5102" LastEditDate="2013-09-09T03:12:25.230" LastActivityDate="2013-10-29T11:44:47.257" Title="How to solve a Rayleigh-quotient-like minimization with inequality constraints" Tags="|optimization|nonlinear-programming|constrained-optimization|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8426" PostTypeId="1" AcceptedAnswerId="8430" CreationDate="2013-09-07T16:25:31.250" Score="3" ViewCount="127" Body="&lt;p&gt;I have a quantity $\beta(\mathbf{x}) \in \mathbb{R}$ that I wish to compute on a curved, smooth surface defined by $\{\mathbf{x}: \Gamma(\mathbf{x})=0\} \subset \mathbb{R}^{3}$. (This surface is approximated by a trimesh.) Since I want to display the quantity as a color on the surface, I need to sample, then bin via $\sum_{\mathbf{x} \in \text{bin cell}} \beta(\mathbf{x})/\text{bin cell area}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does one efficiently choose the bin cells so that they all have the same area, and how does one choose the points at which to evaluate the quantity so that each point corresponds roughly to the same subarea of the bin cell? References and answers appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2013-09-07T18:49:04.473" Title="Equal Area Sampling on Curved Surface:" Tags="|algorithms|reference-request|computational-geometry|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8428" PostTypeId="1" AcceptedAnswerId="8431" CreationDate="2013-09-07T18:26:47.143" Score="1" ViewCount="2547" Body="&lt;p&gt;I am working on Fermion and Boson Hubbard Models, in which the dimension of Hilbert Spaces are quite large (~50k).  Because the Hamiltonian matrix is ~50k X ~50k, to diagonalize these big sparse matrices I am using DSYEV(LAPACK SUBROUTINE) which takes quite good amount of time(8hrs on Xeon Hp workstation Z400 16 GB RAM) for calculation eigenvalues as well as eigenfunctions.  I am interested to reduce time to diagonalizing matrix with available processing power only;  so any suggestion is welcome.  Also, if I use Lanczos method, will this reduce the computation time? &lt;/p&gt;&#xA;" OwnerUserId="5104" LastEditorUserId="549" LastEditDate="2013-09-07T19:08:53.557" LastActivityDate="2013-09-07T20:31:31.630" Title="what is difference “DSYEV(LAPACK SUBROUTINE)” and “Lanczos”?" Tags="|fortran|lapack|banded-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8430" PostTypeId="2" ParentId="8426" CreationDate="2013-09-07T18:49:04.473" Score="5" Body="&lt;p&gt;It's pretty hard to do what you want to do. The method that comes to mind for me is to first calculate a &quot;'blue noise&quot; point distribution on your mesh, then take the bin cells to be the Voronoi regions of those points. Here is a &lt;a href=&quot;http://www.cs.technion.ac.il/~gotsman/AmendedPubl/Yin/BlueNoiseSamplingSurfaces.pdf&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt; that talks about computing such point sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An easier approach is to simply choose the barycenter of each triangle (average of its three vertices) as the sample point, taking the bin cell to be the triangle itself. If you have triangles of vastly different areas, then you may want to remesh it first, using a tool like MeshLab.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-09-07T18:49:04.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8431" PostTypeId="2" ParentId="8428" CreationDate="2013-09-07T20:31:31.630" Score="5" Body="&lt;p&gt;If you are trying to calculate all 50k eigenvalues of these matrices, that is going to &#xA;take you a &quot;quite good amount of time&quot; no matter which algorithm you choose. I'm doubtful&#xA;you can do substantially better than LAPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lanczos (or generally Arnoldi) algorithms are particularly good at calculating a hand full&#xA;of eigenvalues at either end of the spectrum (either lowest or highest). A robust&#xA;implementation with a sophisticated shifting strategy is required to efficiently calculate a large number of eigenvalues in the spectrum without missing any.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ARPACK is generally considered a state-of-the-art Lanczos/Arnoldi eigensolver:&#xA;&lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot;&gt;http://www.caam.rice.edu/software/ARPACK/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the ARPACK main web page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;ARPACK software is capable of solving large scale symmetric, nonsymmetric, and generalized eigenproblems from significant application areas. The software is designed to compute a few (k) eigenvalues with user specified features such as those of largest real part or largest magnitude.&quot;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-09-07T20:31:31.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8432" PostTypeId="1" AcceptedAnswerId="8450" CreationDate="2013-09-08T00:16:26.333" Score="6" ViewCount="302" Body="&lt;p&gt;I have some function of $R^2$, that must be numerically computed. For instance, I might be interested in a real-valued contour integral that begins from (x,y) = 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(x,y) = \Re\left[\int_0^{x + iy} t^3 + t^5 \, dt \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Re$ is for the real part. (In practice I don't have the explicit form for f and it must be numerically computed). The contours $f = 0$ look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OPMcL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there exists some continuous and smooth curve that leaves the origin, and for all points along this curve $x = x(s)$, $y = y(s)$, that $f = 0$ where $s$ is is some parameterization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know a good initial guess to use for the initial direction of this line. For instance, I might know that initially, the curve leaves along some direction, $\theta$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I numerically solve for the curve? &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;One way is to just compute $f$ over a grid of values in $x$ and in $y$. Then create a contour plot and interpolate for all constant-value contours satisfying f = 0. This is not a computationally friendly way. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another way might be to start at (x,y) = (0,0) and take a small step in the direction of your guess of where the curve lies. Then from this new point, take another small step, but in a direction that minimizes $f$. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The problem with 2. is that I'm not sure how to code something adaptive (I'd like to take variable step sizes, particularly around regions where the curve may be highly-curved). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure there must be well-known numerical methods for these problems. This is really just root-finding but with the advantage that you know the solution set (that you are interested in) must be continuous and smooth. Can someone lend a hand?&lt;/p&gt;&#xA;" OwnerUserId="5107" LastEditorUserId="5107" LastEditDate="2013-09-08T08:54:02.310" LastActivityDate="2013-09-09T12:07:52.250" Title="Continuation procedure to solve for a 2D curve that satisfies f(x,y) = 0" Tags="|interpolation|numerical-analysis|roots|complex-analysis|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8433" PostTypeId="2" ParentId="8432" CreationDate="2013-09-08T03:37:59.433" Score="1" Body="&lt;p&gt;I answered a similar question &lt;a href=&quot;https://scicomp.stackexchange.com/a/1562/317&quot;&gt;here&lt;/a&gt;. That method works if you have a good way of solving the segment intersection query (easiest if your $f$ returns a signed value so that you can use sign-based bisection).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, I would try analysis. You can compute the gradient of $f$ with respect to $x$ and $y$ presumably. It's best to carry out the differentiation by hand, but you could approximate it with a finite difference. You could possibly compute higher order derivatives to estimate curvature, but this rapidly gets annoying. The curve goes in the directions of zero gradient. You will have to decide on a large-ish step size (which you can then refine) and then compute the correction to get back onto the curve (either using the line segment query from above (in the direction perpendicular to the step) or using something like Newton's method).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in order to perform adaptive sampling, you have to have some idea of the maximum curvature that is possible or that you'll be happy with. If you know the spatial extent of your curve, then you can take a steps of more reasonable size.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-09-08T03:37:59.433" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8434" PostTypeId="1" CreationDate="2013-09-08T07:09:01.347" Score="6" ViewCount="223" Body="&lt;p&gt;I have a similar question compared to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6784/optimize-an-unknown-function-which-can-be-evaluated-only&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to maximize a function, which is a black-box to me. I have tools of gradient-free methods; in fact, this problem is reasonably smooth that I can even use the numerical approximation of gradients most of the time. However, there is this last problem: the domain where the function is defined is unknown. I cannot know whether the function is defined at this point before I evaluate it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, what I did is if during evaluation I realized this function is undefined, I gave it the value of either &lt;code&gt;-inf&lt;/code&gt; or &lt;code&gt;-nan&lt;/code&gt;. However, the program sometimes got to this point and try to evaluate its derivatives - and then the program just fails. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think this is a rare problem, so I tried to search the web, but with no luck. I would like to bring this question up here, and would really appreciate any ideas that may help.&lt;/p&gt;&#xA;" OwnerUserId="5108" LastEditorUserId="20688" LastEditDate="2019-05-09T20:52:07.153" LastActivityDate="2019-05-09T20:52:55.140" Title="Unknown domain in optimization problem" Tags="|optimization|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8435" PostTypeId="2" ParentId="8434" CreationDate="2013-09-08T12:47:28.160" Score="1" Body="&lt;p&gt;Judging from your description, your problem occurs when using derivative based methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I suggest you try to stay in feasible regions by --if necessary-- adjusting the length of the step in the current gradient direction, such that you always hit a valid point in your search space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other derivative-free methods like Simplex or Genetic Algorithms, the sporadic failure of the evaluation does not make the algorithm fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I have only basic insight in the field of optimization.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-09-08T12:47:28.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8437" PostTypeId="2" ParentId="8425" CreationDate="2013-09-08T23:47:47.763" Score="1" Body="&lt;p&gt;I think the comparison you're trying to draw is to the Rayleigh quotient minimization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\min_x &amp;amp;\frac{\left( Ax \right)^\top \left( Ax \right)}{x^\top&#xA;x},\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to try to find the eigenvalues of $A^{\top}A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The chief difference I see between those problems is that if $x$ is an optimal solution to the Rayleigh quotient minimization, then so is $cx$ for all scalar $c \neq 0$ (which makes sense; $x$ would be an eigenvector of $A^{\top}A$, so $cx$ is also an eigenvector for $c \neq 0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no such scale-invariance for solutions of the constrained optimization problem you've shown. Adding constraints and affine terms to the numerator of the objective destroys the property exploited by Rayleigh quotient iteration. Consequently, I would guess that you are correct: you probably cannot solve this problem using the same methods as Rayleigh quotient iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem you've formulated looks to me like it's a nonlinear (probably nonconvex) program. It might be worth reformulating it so that it looks more like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\min_{x, f} &amp;amp;f \\&#xA;\textrm{s.t.} &amp;amp; (Ax - b)^{\top}(Ax - b) - f x^{\top}x = 0 \\&#xA;&amp;amp; Ax - b \geq 0;&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this sort of reformulation works great for linear fractional programming, but I don't know that it will help you much for your problem, other than to avoid issues occurring if the objective function is evaluated at $x = 0$. The $f$ term is a new variable introduced to stand for the fractional objective function. The term $f x^{\top}x$ is now a potential source of nonconvexity, along with the nonlinear equality constraint.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-09-08T23:47:47.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8438" PostTypeId="1" AcceptedAnswerId="8443" CreationDate="2013-09-09T01:34:36.277" Score="0" ViewCount="6717" Body="&lt;p&gt;CG Algorithm&#xA;&lt;a href=&quot;https://skydrive.live.com/redir?resid=E0ED7271C68BE47C!386&amp;amp;v=3&quot; rel=&quot;nofollow&quot;&gt;https://skydrive.live.com/redir?resid=E0ED7271C68BE47C!386&amp;amp;v=3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;System of equations, the question and the example&#xA;&lt;a href=&quot;https://skydrive.live.com/redir?resid=E0ED7271C68BE47C!387&amp;amp;v=3&quot; rel=&quot;nofollow&quot;&gt;https://skydrive.live.com/redir?resid=E0ED7271C68BE47C!387&amp;amp;v=3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% below result is only accurate at decimal 0.1, but not the tolerate, why? &#xA;% is the conjugate gradient algorithm only for square matrix A's system?&#xA;% about the lambda1 + lambda2 = 1, how embedded this in algorithm?&#xA;% below is Matlab Code&#xA;&#xA;w = CGResult(1);&#xA;lambda1 = CGResult(2);&#xA;lambda2 = CGResult(3);&#xA;(w+13/35*lambda1+47/140*lambda2) &amp;gt;= 2/5&#xA;(w+57/140*lambda1+61/140*lambda2) &amp;gt;= 2/5&#xA;(w+31/140*lambda1+8/35*lambda2) &amp;gt;= 1/5&#xA;&#xA;&#xA;A = [1, 13/35, 47/140; &#xA;1, 57/140, 61/140;&#xA;1, 31/140, 8/35;];&#xA;b = [2/5;2/5;1/5;];&#xA;tol = 0.0000000000001;&#xA;Max = 10000;&#xA;x2 = [0.5;0.5;0.5];&#xA;n2 = 3;&#xA;MartinCG(x2, A, b, Max, tol, n2);&#xA;&#xA;% Why only for Square Matrix, How about Non Square Matrix A&#xA;&#xA;function CGResult = MartinCG(x2, A, b, Max, tol, n3)&#xA;r = zeros(n3);&#xA;x = zeros(n3,Max-1);&#xA;x(:,1) = x2;&#xA;r = zeros(n3,Max-1);&#xA;v = zeros(n3,Max-1);&#xA;t = zeros(n3,Max-1);&#xA;dum = zeros(n,n);&#xA;r(:,1) = b - A*x(:,1);&#xA;v(:,1) = r(:,1);&#xA;for k=1:1:Max-3&#xA;    k&#xA;    testnorm = norm(v(:,k))&#xA;    if (norm(v(:,k)) == 0)&#xA;        break;&#xA;    end &#xA;    t(:,k) = (r(:,k).*r(:,k))./(v(:,k).*(A*v(:,k)));&#xA;    x(:,k+1) = x(:,k) + (t(:,k).*v(:,k));&#xA;    r(:,k+1) = r(:,k) - t(:,k).*(A*v(:,k));&#xA;    testnorm2 = norm(r(:,k+1))&#xA;    if (norm(r(:,k+1)) &amp;lt; tol)&#xA;        break;&#xA;    end&#xA;    v(:,k+1) = r(:,k+1) + (r(:,k+1).*r(:,k+1))./(r(:,k).*r(:,k)).*v(:,k);&#xA;end&#xA;CGResult = x(:,k+1);&#xA;end&#xA;&#xA;%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;when really to calculate the example, my code do not work for 6*3 Matrix A&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = [1, 13/35, 47/140; &#xA;1, -13/35, -47/140;&#xA;1, 57/140, 61/140;&#xA;-2/5, -57/140, -61/140;&#xA;1/5, 31/140, 8/35;&#xA;-1/5, -31/140, -8/35;];&#xA;b = [2/5;-2/5;2/5;-2/5;1/5;-1/5;];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5113" LastActivityDate="2023-07-30T07:55:39.973" Title="How to solve this system with conjugate gradient algorithm in matlab" Tags="|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8439" PostTypeId="1" AcceptedAnswerId="8440" CreationDate="2013-09-09T01:39:39.580" Score="0" ViewCount="1153" Body="&lt;p&gt;After download from &lt;a href=&quot;http://sourceforge.net/projects/reduce-algebra/files/&quot; rel=&quot;nofollow&quot;&gt;http://sourceforge.net/projects/reduce-algebra/files/&lt;/a&gt; and unzip reduce-windows64-20101007.zip &lt;/p&gt;&#xA;&#xA;&lt;p&gt;failed to open reduce, got error in window 8&#xA;libgcc_s_sjlj-1.dll is missing&lt;/p&gt;&#xA;" OwnerUserId="5113" LastActivityDate="2013-09-09T02:16:36.610" Title="libgcc_s_sjlj-1.dll is missing" Tags="|software|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8440" PostTypeId="2" ParentId="8439" CreationDate="2013-09-09T02:16:36.610" Score="0" Body="&lt;p&gt;The software that you're trying to download was compiled with the GCC compiler and requires a run time library that is provided with that compiler.  Installing the gcc compiler on your system will probably resolve the issue.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-09-09T02:16:36.610" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8441" PostTypeId="1" CreationDate="2013-09-09T02:46:34.340" Score="0" ViewCount="3561" Body="&lt;p&gt;I want to compute a symmetric matrix A from a vector b by: A = b * b'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the Eigen library automatically take into account that it does not need to do all calculations to get A (because of the symmetry which repeats most of the matrix entries)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, I could take advantage of the symmetry and write my own function for the computation.&lt;/p&gt;&#xA;" OwnerUserId="5106" LastActivityDate="2013-10-09T17:37:51.230" Title="Symmetric Matrix Computation (Eigen &amp; C++)" Tags="|linear-algebra|c++|eigen|symmetry|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8443" PostTypeId="2" ParentId="8438" CreationDate="2013-09-09T06:15:43.807" Score="3" Body="&lt;p&gt;A cursory glance at &lt;a href=&quot;http://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow&quot;&gt;Wikipedia's page on the conjugate gradient (CG) method&lt;/a&gt; will indicate why CG won't work on your particular system of linear equations: CG is designed to solve the system $Ax = b$ where $A$ is &lt;em&gt;symmetric and positive-definite&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These two conditions naturally also require that $A$ is square. Since your linear system satisfies none of these conditions, there is no reason that conjugate gradient should converge to a valid solution for arbitrary nonsquare matrices with arbitrary right-hand sides.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-09-09T06:15:43.807" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8446" PostTypeId="1" CreationDate="2013-09-09T10:01:37.717" Score="5" ViewCount="1196" Body="&lt;p&gt;I'm solving an electrostatic problem governed by Laplace equation $$-\nabla \cdot (\rho^{-1} \nabla u) = 0$$ in the following domain: a brick ($\Omega_1$) with a cylindrical inclusion ($\Omega_2$), where $\rho$ - electric resistivity, $u$ - potential. There are two plane electrodes attached to opposite faces of the brick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/VBinN.jpg&quot; width=&quot;250&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On top and bottom faces Dirichlet boundary conditions are defined:&#xA;$$u|_{\partial\Omega_{top}} = u_1$$&#xA;$$u|_{\partial\Omega_{bottom}} = u_2$$&#xA;On other faces of the brick homogeneous Neumann boundary conditions are held.&#xA;$$\rho^{-1}\frac{\partial u}{\partial n}|_{\partial\Omega_{side}} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the inclusion is strictly inside the brick, everything is good. Here is a 2D view of potential distribution with isolines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/yX4ET.jpg&quot; alt=&quot;2D view of potential distribution for the case of cylinder lying inside brick&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in such a situation when inclusion goes through the brick. In this case it crosses top and bottom faces, and they become consisting of two subdomains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/WNzPv.jpg&quot; width=&quot;250&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I impose the same boundary conditions as in previous case on both subdomains, I get the following potential distribution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fYMKJ.jpg&quot; alt=&quot;2D view of potential distribution for the case of cylinder crossing brick&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it's wrong. Therefore I would like to ask - what should I change to get right solution - boundary conditions on top/bottom faces, or something else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parameters: $\rho_{\Omega_1} = 10^6 (Ohm \cdot m)$, $\rho_{\Omega_2} = 10^{-3} (Ohm \cdot m)$, $u_1 = 1 (V)$, $u_2 = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why I think that the result I obtain in the second case is wrong. The task I'm solving is a task about conductive inclusion inside non-conductive (or low conductive) medium. I was taught that there is no electric field in conductive bodies, and therefore electrostatic potential is constant there. That can be seen from the result of the first simulation when inclusion is fully inside a brick. And I expect the same behaviour of electric field in the second case. But that doesn't happen. Potential equally changes in conductive and non-conductive media, and electric field is the same in the whole domain. Therefore I think something is wrong there. I would be happy if somebody could say that the model I use is correct, but I'm wrong only in interpretation of my results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thanks to all of those who took part in discussion of my problem. But the main part of the answers was about explaining that the solution of the problem in the second case (when inclusion goes through the brick) is the right solution of the boundary value problem I provided. I agree with all these answers. And as @StefanoM said &quot;The second problem is trivial and does not need a FEM code to be solved&quot;. I agree 100% with that. But as @StefanoM wisely noticed &quot;one should never forget that phenomenological equations are based on a number of simplifying assumptions and cannot be applied if those assumptions are not true&quot;. Therefore the main question that worries me is - what is the right model for the problem I'm trying to solve? Perhaps I need another equation, or another boundary conditions somewhere. Could anyone suggest the model that would correctly describe the electrostatic problem when the computational domain consists of two subdomains with contrasting electrical resistivity, especially in the case when condutive inclusion goes through non-conductive media. I started from cylindrical inclusion because it's the simplest case. In fact I will need to work with more complicated domains (inclusions), therefore I do need to use FEM code.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastEditorUserId="3576" LastEditDate="2013-09-11T04:14:43.160" LastActivityDate="2013-09-11T04:14:43.160" Title="boundary conditions for electrostatic problem" Tags="|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8447" PostTypeId="2" ParentId="8441" CreationDate="2013-09-09T11:39:38.833" Score="2" Body="&lt;p&gt;This is a typical case where it does not make sense to actually store the matrix. Rather, consider the matrix to be an operator where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;if you need to multiply by it, i.e., form $y=Ax$, you do $y=b (b\cdot x)$ instead&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if you need to access an element $A_{ij}$, you instead compute it as $b_ib_j$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In other words, knowing what $A$ is, you should just not store it as a matrix but simply as a single vector. You can't expect libraries to do these things for you, but you should do it yourself.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-09T11:39:38.833" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8448" PostTypeId="2" ParentId="8438" CreationDate="2013-09-09T11:58:03.477" Score="2" Body="&lt;p&gt;Your question does not contain enough details to understand what it is you are doing and what you want to do. But, if I understand your right, then you are trying to apply the conjugate gradient method to a linear system $Ax=b$ where your matrix $A$ is not square. This can not work -- CG can only deal with square matrices. Furthermore, if $A$ is not square, you need to think about what exactly it is you mean when you ask for the &quot;solution&quot; of such a system because there are, in general, either infinitely many (underdetermined) or none (overdetermine).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-09T11:58:03.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8449" PostTypeId="1" CreationDate="2013-09-09T12:00:24.333" Score="8" ViewCount="1686" Body="&lt;p&gt;Suppose that you have a system of PDEs to solve. At least for simplicity, let's assume it's time independent, quasi-linear (linear in its derivatives) solved on a rectangular grid in (x,y) space, and with boundary conditions specified all around. My question is more general, but let's start here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There might be two dependent variables, $u(x,y)$ and $v(x,y)$. The general equation might of the form: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;a(x,y) Y_{xx} + b(x,y) Y_{yy} + cY_{xy} + d(x,y) Y_x + e(x,y) Y_y = f(x, y, Y)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where all the functions $a$ through $e$ are 2x2 matrices, $f$ is a 2x1 matrix, and $Y$ is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Y(x,y) = \begin{pmatrix} u(x,y) \\ v(x,y) \end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that you would like to compute a finite-difference numerical approximation. Assume that the grid-points are evenly spaced in x and y. You might discretize $x$ into $N$ points and $y$ into $M$ points. And then you would construct a solution as a 2(nm) column vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;X = \begin{pmatrix} u_{11} \\ \vdots \\ u_{n1} \\ \vdots \\ u_{nm} \\ v_{11} \\ \vdots \\ v_{nm} \end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would then solve some sort of matrix equation &#xA;$$&#xA;(A + B + C + D + E)X = b,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the 2nm x 2nm matrices $A$ through $E$ are finite difference matrices for the corresponding differential operators. The combined matrix in the brackets will have some sort of block banded structure. Their edge values may be complicated due to boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is simple: is there an easy way (particularly in &lt;strong&gt;Matlab&lt;/strong&gt;) to generate the matrices $A$ through $E$? Or at the very least, a simple guide to creating all the necessary matrices (except perhaps the modifications you need for boundary conditions, which might be a manual input). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I can do this by hand, but it's (i) a lot of algebra; and (ii) subject to a lot of typos and errors when you implement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/matlab/fdcoeffF.m&quot;&gt;here&lt;/a&gt; is a code that gives you all the necessary vectors for a 1D finite difference 'matrix' for different orders of derivatives. Is there an analogy for systems of PDEs on the plane? &lt;/p&gt;&#xA;" OwnerUserId="5107" LastActivityDate="2013-09-09T13:10:23.560" Title="Automatically generating finite difference matrices for systems of PDEs" Tags="|pde|matlab|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8450" PostTypeId="2" ParentId="8432" CreationDate="2013-09-09T12:07:52.250" Score="4" Body="&lt;p&gt;Let us parameterize the curve you are looking for by $(x(t),y(t))$ and let us for a moment assume that at all points on this curve $\nabla f(x(t),y(t)) \neq 0$, i.e., the curve never intersects another isocontour line. Then you know that at each point the tangent to the curve $(x(t),y(t))$ is parallel to the isocontour levels of $f$, i.e., that it is perpendicular to the gradient of $f$. That means that one possible parameterization would be to define the curve as&#xA;$$&#xA;  \frac d{dt} \left({x(t) \atop y(t)}\right) = &#xA;  \begin{pmatrix} 0 &amp;amp; 1 \\ -1 &amp;amp; 0 \end{pmatrix}&#xA;  \nabla f(x(t),y(t))&#xA;$$&#xA;which is an ODE that you can integrate up from an arbitrarily chosen point $(x(0),y(0))$ that you know lies on the curve. Another possible choice for the time parameterization would be to use the form&#xA;$$&#xA;  \frac d{dt} \left({x(t) \atop y(t)}\right) = &#xA;  \begin{pmatrix} 0 &amp;amp; 1 \\ -1 &amp;amp; 0 \end{pmatrix}&#xA;  \frac{\nabla f(x(t),y(t))}{\|\nabla f(x(t),y(t))\|}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either case, you will get into trouble once you get to a point where the gradient of $f$ is zero, i.e., where two zero lines cross. You'll have to treat this case separately.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-09T12:07:52.250" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8451" PostTypeId="2" ParentId="8420" CreationDate="2013-09-09T12:14:41.217" Score="5" Body="&lt;p&gt;I teach my students to always multiply from the left, for exactly the reason you mention. More details here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.676.4.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.676.4.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dealii.org/8.0.0/doxygen/deal.II/step_3.html&quot; rel=&quot;noreferrer&quot;&gt;http://dealii.org/8.0.0/doxygen/deal.II/step_3.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-09T12:14:41.217" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8452" PostTypeId="2" ParentId="8449" CreationDate="2013-09-09T12:40:28.597" Score="3" Body="&lt;p&gt;I'm not aware of any easy solution but it is not too hard to automate this using a symbolic computation package like Maple or &lt;a href=&quot;http://maxima.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;Maxima&lt;/a&gt;.  You can &lt;a href=&quot;http://paste.lisp.org/display/138871&quot; rel=&quot;nofollow&quot;&gt;see here an example of this idea&lt;/a&gt; applied to a 1D diffusion equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, you want the functions &lt;code&gt;LinearAlgebra[GenerateMatrix]&lt;/code&gt; (in Maple) or &lt;code&gt;augcoefmatrix&lt;/code&gt; (in Maxima). MATLAB has a symbolic computation toolbox that I'd think can also be used for this purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regard to automating the implementation, you could use the &lt;code&gt;CodeGeneration&lt;/code&gt; module in Maple to output MATLAB code or the &lt;code&gt;grind&lt;/code&gt; and &lt;code&gt;fortran&lt;/code&gt; functions from Maxima to produce output that's close to MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2013-09-09T13:10:23.560" LastActivityDate="2013-09-09T13:10:23.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8454" PostTypeId="2" ParentId="7846" CreationDate="2013-09-09T15:13:50.070" Score="6" Body="&lt;p&gt;For $-\epsilon \Delta u + \mathbf{v}\cdot\nabla u = f$, where $\epsilon \ll |\mathbf{v}|$. You can refer to this paper:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Adaptive anisotropic meshing for steady convection-dominated problems&lt;/em&gt;, section 6 and section 7.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.math.vt.edu/people/adjerids/research/papers/singular_ams.pdf&quot;&gt;&lt;em&gt;Computational Methods for Singularly Perturbed Systems&lt;/em&gt;&lt;/a&gt;, example 5.1. Known solution, boundary layers. This is also included in the NIST benchmark list.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the all diffusion-convection-reaction equation For $-\nabla\cdot(A\nabla u) + \mathbf{b}\cdot\nabla u + cu= f$, you can refer to &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;A posteriori energy-norm error estimates for advection-diffusion equations approximated by weighted interior penalty methods&lt;/em&gt;, section 4.2, the right hand side $f$ is computed using the true solution.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When the second order term is totally gone  $  \mathbf{b}\cdot\nabla u + \mu u= f$ (meaning the solution is less and less smooth and the problem becomes harder), there are two examples I know:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://hpfem.org/hermes-doc/hermes-examples/html/src/hermes2d/benchmarks-general/stabilized-advection-reaction.html&quot;&gt;http://hpfem.org/hermes-doc/hermes-examples/html/src/hermes2d/benchmarks-general/stabilized-advection-reaction.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;A Posteriori Error Estimation for Interior Penalty Finite Element Approximations of the Advection-Reaction Equation&lt;/em&gt;, section 5. Since this is behind a paywall, I am list it here:&#xA;Let $\Omega = \{(x,y)\in (0,1)\times (0,1)\}$. $\Omega^-$ is the left edge of the square. True solution $u$ satisfies&#xA;$$&#xA;\left\{ \begin{aligned}&#xA;\mu u + \mathbf{b}\cdot \nabla u &amp;amp;= f,&#xA;\\&#xA;u|_{\partial \Omega^-} &amp;amp;=  g,&#xA;\end{aligned} \right.&#xA;$$&#xA;with&#xA;$$&#xA;\mathbf{b} = \binom{y+1}{-x}\frac{1}{\sqrt{x^2 + (y+1)^2}}, \quad  \mu= 0.1&#xA;$$&#xA;and $g(x,y)$ chosen so that the exact solution is:&#xA;$$&#xA;u(x,y) = e^{\mu \sqrt{x^2 + (y+1)^2} \arcsin\left(\frac{y+1}{\sqrt{x^2 + (y+1)^2}}\right)} \arctan\left(\frac{\sqrt{x^2 + (y+1)^2} - 1.5}{\epsilon}\right).&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="350" LastEditorUserId="350" LastEditDate="2013-09-09T20:00:59.967" LastActivityDate="2013-09-09T20:00:59.967" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8455" PostTypeId="2" ParentId="7846" CreationDate="2013-09-09T15:33:31.107" Score="5" Body="&lt;p&gt;The Erikkson-Johnson model problem is useful if you're interested in boundary layer behavior for a steady state equation. It's the only exact solution with a boundary layer for the homogeneous convection-diffusion equation that I've encountered so far. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you take the convection vector $\beta = (1,0)^T$, then convection-diffusion reduces to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;u_{,x} - \epsilon \Delta u = 0&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions on the whole boundary&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\left.u\right|_{\partial \Omega} = \left.u_{\rm exact} \right|_{\partial \Omega}.&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution can be found through separation of variables (the solution is given &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CCoQFjAA&amp;amp;url=http%3A%2F%2Fwww.mat.puc.cl%2Fprepublicaciones%2Findex.php%3Fcontenido%3Darchivo%26idpreprint%3D27&amp;amp;ei=T-gtUs3SA8Tr2QWI44HgBg&amp;amp;usg=AFQjCNGljXAfz4u87VwBwi8StNZuwhhHCw&amp;amp;sig2=1uTHSHSfKSwZvDVsTpJavA&amp;amp;bvm=bv.51773540,d.b2I&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; on page 19 as well).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For highly convection-dominated regimes, manufactured solutions for convection-diffusion often lead to forcing terms with strong gradients themselves, such that evaluation of convergence rates or exact error tends to depend more on numerical integration of the forcing (or the error $u-u_h$!) rather than behavior of the method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A note: you can also generalize the Erikkson-Johnson solution to a convection-diffusion-reaction equation by adding $u_{\rm exact}$ as a zero-order forcing term to the RHS (integration error matters less for a zero order term with small diffusion, since integration error around the boundary layer is an $O(\epsilon)$ error).  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2014-11-27T13:55:54.767" LastActivityDate="2014-11-27T13:55:54.767" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8456" PostTypeId="2" ParentId="8441" CreationDate="2013-09-09T17:12:55.987" Score="0" Body="&lt;p&gt;The Eigen library cannot &quot;know&quot; a priori if the product &lt;code&gt;b*b'&lt;/code&gt; will result in a symmetric matrix, therefore you can write your own method (although it'll be somewhat useless if not working with large matrices)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a routine called &lt;code&gt;isSymmetric&lt;/code&gt;, is used within some linear solvers to check whether the input matrix is symmetric or not, but of course, that does not fulfill your need.&lt;/p&gt;&#xA;" OwnerUserId="4964" LastActivityDate="2013-09-09T17:12:55.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8457" PostTypeId="2" ParentId="8423" CreationDate="2013-09-09T17:50:30.343" Score="2" Body="&lt;p&gt;Significant wave height, $H_{1/3}$, defined as the average height of the highest one-third of all waves, is historical. In the early days, the wave state was observed visually, which resulted in this definition of $H_{1/3}$. This quantity is not necessarily equal to the contemporary notion of significant wave height, $H_{s}$, which is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;H_{s} = 4 \sqrt{m_0},&#xA;$$&#xA;where $m_{0}$ is the zero-order moment of the wave variance spectrum:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;m_0 = \int{F(f)}\ df = &amp;lt;\!\eta^2\!\!&amp;gt;.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\eta$ is the surface elevation, and $&amp;lt;\!\eta^2\!\!&amp;gt;$ is the time average of the surface elevation variance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though $H_{s} \ne H_{1/3}$, Phillips (1977) shows that $H_{s} \approx H_{1/3}$ for a narrow wave variance spectrum $F(f)$. See &lt;em&gt;Dynamics of The Upper Ocean&lt;/em&gt; by Phillips, 1977, Cambridge University Press.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have time series of $\eta$, you can then calculate $H_{s}$ from $m_0$ in a straightforward fashion. Keep in mind that $H_{s}$ as an integrated quantity only makes sense for linear (small slope) waves that are quasi-uniform in space and time, i.e. $F(f)$ and $H_s$ are changing over a significantly longer time scale than that of $\eta$.&lt;/p&gt;&#xA;" OwnerUserId="264" LastActivityDate="2013-09-09T17:50:30.343" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8459" PostTypeId="2" ParentId="8446" CreationDate="2013-09-10T02:12:30.697" Score="6" Body="&lt;p&gt;I don't have electrostatics training but your second picture looks OK to me.  Your conductive inclusion is essentially shorting out the plates u1 and u2.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I was taught that there is no electric field in conductive bodies,&#xA;  and therefore electrostatic potential is constant there.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the conductive body touches both plates u1 and u2 which are forced to have different potentials according to the boundary conditions, then the conductive body cannot have constant electrostatic potential.  It has a potential gradient and has a high current going through it.&lt;/p&gt;&#xA;" OwnerUserId="5046" LastActivityDate="2013-09-10T02:12:30.697" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8460" PostTypeId="2" ParentId="8308" CreationDate="2013-09-10T04:33:39.640" Score="1" Body="&lt;p&gt;Every time that you use an &lt;code&gt;MPI_Request&lt;/code&gt; in an MPI function, you either have to wait for it to finish using any of the &lt;code&gt;MPI_Wait&lt;/code&gt; functions or keep on doing &lt;code&gt;MPI_Test&lt;/code&gt; on it until the flag returns non-zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This includes all functions that start with &lt;code&gt;MPI_I&lt;/code&gt; (non-blocking sends, receives and collectives) and functions that cancel a some non-blocking operation e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int i = 0;&#xA;MPI_Request req = MPI_REQUEST_NULL;&#xA;MPI_Isend( &amp;amp;i, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, &amp;amp;req );&#xA;MPI_Cancel( &amp;amp;req );&#xA;MPI_Wait( &amp;amp;req, MPI_STATUS_IGNORE ); // you have to wait&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5117" LastActivityDate="2013-09-10T04:33:39.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8461" PostTypeId="2" ParentId="8204" CreationDate="2013-09-10T12:01:14.373" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://www.advanpix.com/&quot; rel=&quot;nofollow&quot;&gt;Multiprecision computing toolbox for MATLAB&lt;/a&gt; has support for sparse matrices and specifically optimized for computations with quadruple precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is timing details for quad-precision sparse solvers:&#xA;&lt;a href=&quot;http://www.advanpix.com/2013/09/05/direct-solvers-for-sparse-matrices/&quot; rel=&quot;nofollow&quot;&gt;Direct Solvers for Sparse Matrices&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am the author of the toolbox).&lt;/p&gt;&#xA;" OwnerUserId="1033" LastActivityDate="2013-09-10T12:01:14.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8462" PostTypeId="2" ParentId="7002" CreationDate="2013-09-10T12:23:51.253" Score="7" Body="&lt;p&gt;The simplest/fastest way to solve ill-conditioned problems is to increase precision of computations (by brute force). Another (yet not always possible) way is to re-formulate your problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might need to use quadruple precision (34 decimal digits). Even though 20 digits will be lost in a course (because of condition number) you will still get 14 correct digits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is of any interest, now &lt;a href=&quot;http://www.advanpix.com/2013/09/05/direct-solvers-for-sparse-matrices/&quot;&gt;quad-precision sparse solvers&lt;/a&gt; are available in MATLAB too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am the author of the mentioned toolbox).&lt;/p&gt;&#xA;" OwnerUserId="1033" LastActivityDate="2013-09-10T12:23:51.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8464" PostTypeId="2" ParentId="696" CreationDate="2013-09-10T14:44:47.180" Score="1" Body="&lt;p&gt;Also take a look at John Burkhardt's libraries at &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/&quot; rel=&quot;nofollow&quot;&gt;http://people.sc.fsu.edu/~jburkardt/&lt;/a&gt;&#xA;His code is LGPL licensed, as opposed to the GPL used by the Gnu Scientific Library.&lt;/p&gt;&#xA;" OwnerUserId="5119" LastActivityDate="2013-09-10T14:44:47.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8465" PostTypeId="1" AcceptedAnswerId="8467" CreationDate="2013-09-10T23:22:23.263" Score="10" ViewCount="581" Body="&lt;p&gt;I'm a beginner with FE. My application is the pricing of financial derivatives where the space is five dimensional. So, adding time, the problem has six dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to look around (Fenics, escript, deal.II,...), but my understanding is that those software are limited to 3 + 1 (3d space + 1d time). Is this correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My targeted language are Python or C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Description of my problem&lt;/strong&gt;&lt;br&gt;&#xA;I would like to price an investment product where, each month, the investor has the freedom to re-invests or not. &#xA;I would like to do so with stochastic volatility, stochastic interest rate, and stochastic mortality.&lt;br&gt;&#xA;The stochastic PDEs look like this&#xA;\begin{align}&#xA;dS_t &amp;amp;= \mu^S_t d_t + \sqrt{\sigma_t} dB^S_t  &amp;amp;\text{(stock)}\\&#xA;d\sigma_t &amp;amp;= \mu^\sigma_t dt + \nu^\sigma_t dB^\sigma_t &amp;amp; \text{(volatility)} \\&#xA;dr_t &amp;amp;= \mu^r_t dt + \nu^r_t dB^r_t &amp;amp; \text{(interest rate)} \\&#xA;dq_t &amp;amp;= \mu^q_t dt + \nu^q_t dB^q_t &amp;amp; \text{(mortality)}&#xA;\end{align}&#xA;Where $\mu^S_t$ is a time-dependent constant associated to the stock price $S$, and $B^S_t$ is an independent Levy process which creates noise in the stock price $S$. Similarly for the other quantities: $\nu^\sigma_t$ is a time-dependent quantity associated to the volatility $\sigma$.&lt;br&gt;&#xA;Let $C_\tau$ denotes the admissible investments at time $\tau$. &#xA;The stochastic control problem looks like &#xA;$$&#xA;V_\tau = max \left\{ c \in C_\tau : P(\text{death})E(r_\tau f(S_{\tau+1})) + P(alive)E(r_\tau V_{\tau+1})\right\}.&#xA;$$ &#xA;The above PDEs are continuous, but the value of the product $V_\tau$ is solved only at predefined $\tau$-times, say each month.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess Monte-Carlo can always brute force my problem, but it is very slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Deterministic form of the stochastic PDEs&lt;/strong&gt;&lt;br&gt;&#xA;For this part, assume that the value of the option&#xA;$$&#xA;V : (t, S_t, \sigma_t, r_t, q_t, c_t) \mapsto  (t, V_t),&#xA;$$&#xA;is define on the natural time $t$, not the $\tau$-times, with $c_t$ the investment at time $t$.&lt;br&gt;&#xA;Define the differential operator&#xA;\begin{align}&#xA;L_t &amp;amp;= \partial_{r,S} + \partial_{r,\sigma} + \partial_{\sigma,S} \\&#xA;L^S_t &amp;amp;= \sigma_t \partial_S + r_t \partial_{S,S} \\&#xA;L^r_t &amp;amp;= \partial_r + \partial_{r,r} \\&#xA;L^\sigma_t &amp;amp;= \partial_\sigma + \partial_{\sigma,\sigma} \\&#xA;L^q_t &amp;amp;= \partial_q + \partial_{q,q}&#xA;\end{align}&#xA;where time-dependent constant $\{\mu^S_t,\ldots\}$ are ignored.&#xA;The deterministic PDE is then&#xA;$$&#xA;\partial_t  V_t +\left(L_t+ L^S_t + L^\sigma_t + L^r_t+L^q_t\right)V_t = 0, &#xA;$$&#xA;which can adapted to the optimal control problem on the $\tau$-times.&lt;/p&gt;&#xA;" OwnerDisplayName="user729" LastEditorDisplayName="user729" LastEditDate="2013-09-11T14:14:03.177" LastActivityDate="2013-09-11T18:29:56.563" Title="Are there finite element software who handles more than five dimensions?" Tags="|pde|finite-element|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8466" PostTypeId="2" ParentId="8465" CreationDate="2013-09-11T07:04:37.547" Score="0" Body="&lt;p&gt;Ok, so it looks like what you have are a coupled set of ODEs, since as far as I can tell, there are only derivatives with respect to time, and no derivatives with respect to anything else. There are quite a few packages out there for solving systems of ODEs of arbitrary dimension (Matlab has stuff like &lt;code&gt;ode45&lt;/code&gt;). For Python, look at &lt;a href=&quot;https://stackoverflow.com/questions/16909779/any-way-to-solve-a-system-of-coupled-differential-equations-in-python&quot;&gt;this question&lt;/a&gt; for some suggestions. Finally, there is old Fortran code on &lt;a href=&quot;http://www.netlib.org/odepack/opkd-sum&quot; rel=&quot;nofollow noreferrer&quot;&gt;netlib&lt;/a&gt; that can be interfaced with C++ pretty easily (ease of use is another matter). There are probably better alternatives out there since it's been a while since I've looked (others should chime in).&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-09-11T07:04:37.547" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8467" PostTypeId="2" ParentId="8465" CreationDate="2013-09-11T12:53:04.643" Score="8" Body="&lt;p&gt;Assuming you want to solve the Black-Scholes equations or a variant on a portfolio of 5 assets, then you indeed have 5 spatial plus one time dimension. I don't know any FEM package that can do that off the top of my head (deal.II can't readily do it, but see below) but I think I remember that some people from Chris Schwab's group at ETH Zurich solved such problems using sparse meshes. You may get lucky looking around his publications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other equations that have extra dimensions. One example is the radiative transfer equation that has 3 space + 1 time + 2 angular + 1 energy dimension. The way this is typically solved is to discretize 3-dimensional space as usual, then discretize the angular and energy dimensions on separate 2 and 1-dimensional meshes and at each nodal point of the spatial mesh simply have a lot of variables (one each for each node of the angular mesh times the number of nodes in the energy mesh). We use this scheme in deal.II implementations successfully. This makes sense for the radiative transfer equation, and it may be emulated for your equation even if it isn't natural there. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-11T12:53:04.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8468" PostTypeId="1" AcceptedAnswerId="8491" CreationDate="2013-09-11T17:11:17.717" Score="6" ViewCount="10363" Body="&lt;p&gt;I've been using the basic visualization features of Paraview. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to go further by writing python macros to handle some specific tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question for the advanced users:&#xA;&lt;strong&gt;Are there any known guides to get a proper start ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Already done:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Read the Paraview &lt;a href=&quot;http://paraview.org/Wiki/ParaView/Python_Scripting&quot; rel=&quot;nofollow&quot;&gt;wiki&lt;/a&gt; + found some concise handouts (with which I started)&lt;/li&gt;&#xA;&lt;li&gt;Used &quot;Trace&quot; to record and modify python shell (this is not the best way to learn)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2014-08-20T15:53:09.583" LastActivityDate="2014-08-20T15:53:09.583" Title="Python scripting in Paraview" Tags="|python|visualization|paraview|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8469" PostTypeId="2" ParentId="8465" CreationDate="2013-09-11T18:29:56.563" Score="5" Body="&lt;p&gt;DUNE, the Distributed and Unified Numerics Environment &lt;a href=&quot;http://www.dune-project.org&quot;&gt;http://www.dune-project.org&lt;/a&gt;, features some structured grids of arbitrary dimension (SGrid, and Yaspgrid), see &lt;a href=&quot;http://users.dune-project.org/projects/main-wiki/wiki/Grid-Manager_Features&quot;&gt;DUNE's features&lt;/a&gt;. Currently, there is a branch that turns yaspgrid, one of the above grids, into a tensor product grid, if that is of interest. Since release 2.0 (the current one is 2.2.1, and 2.3 is soon to come) we do have reference elements for various finite element methods that support arbitrary dimensions. Therefore it should be possible set up finite element discretizations of arbitrary dimension with e.g. the disrectization module &lt;a href=&quot;https://dune-project.org/pdelab/&quot;&gt;dune-pdelab&lt;/a&gt;. Albeit this might not be tested to often.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, there is still the curse of dimensionality as Wolfgang pointed out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further information I refer you to the &lt;a href=&quot;https://dune-project.org/mailinglists.html&quot;&gt;DUNE mailinglists&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1605" LastActivityDate="2013-09-11T18:29:56.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8470" PostTypeId="1" AcceptedAnswerId="8474" CreationDate="2013-09-11T18:35:20.853" Score="18" ViewCount="2147" Body="&lt;p&gt;I am teaching a numerical analysis survey class and am seeking motivation for the BFGS method for students with limited background/intuition in optimization!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I don't have time to prove rigorously that everything converges, I'm looking to give a reasonable motivation for why the BFGS Hessian update might appear.  As an analogy, Broyden's root-finding method (my writeup is &lt;a href=&quot;http://graphics.stanford.edu/courses/cs205a-13-fall/assets/notes/chapter7.pdf&quot;&gt;here&lt;/a&gt;) can be motivated by asking that your current approximation of the Jacobian minimizes the difference $\|J_k-J_{k-1}\|^2_{\textrm{Fro}}$ with the old Jacobian subject to the constraint that it takes into account the latest secant:  $J_k(\vec x_k-\vec x_{k-1})=f(\vec x_k)-f(\vec x_{k-1})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Derivations of BFGS updates seem far more involved and murky!  In particular, I'd like not to assume &lt;em&gt;a priori&lt;/em&gt; that the update should be rank-2 or take a particular form.  Is there a short variational-looking motivation for the BFGS Hessian update like the one for Broyden?&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2014-03-31T08:11:47.480" Title="Intuitive motivation for BFGS update" Tags="|optimization|iterative-method|nonlinear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8471" PostTypeId="1" CreationDate="2013-09-11T22:23:34.130" Score="1" ViewCount="44" Body="&lt;p&gt;In direct numerical simulation of turbulence what is the advantage to look at the spectra of the energy flux rather than the transfer spectra?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transfer spectra corresponds to the triple-velocity correlation coming from the non-linear interaction of the Navier-Stokes equations.&#xA;The transfer spectra is generally noted:&#xA;\begin{equation}&#xA;T(\mathbf{k},t)&#xA;\end{equation}&#xA;The energy flux through wave number $\mathbf{k}$ is defined from the transfer spectra by&#xA;\begin{equation}&#xA;\Pi(\mathbf{k},t)=\int^{\infty}_{\mathbf{k}}T(\mathbf{p},t)d\mathbf{p}=-\int^{\mathbf{k}}_{0}T(\mathbf{p},t)d\mathbf{p}&#xA;\end{equation}&lt;/p&gt;&#xA;" OwnerUserId="1383" LastEditorUserId="1383" LastEditDate="2013-09-12T15:21:43.403" LastActivityDate="2013-09-12T15:21:43.403" Title="Spectra of the energy flux VS transfer spectra?" Tags="|fluid-dynamics|simulation|data-analysis|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8473" PostTypeId="2" ParentId="8434" CreationDate="2013-09-12T10:46:32.213" Score="3" Body="&lt;p&gt;Pick a fixed point &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; with a finite function value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then return at points where &lt;span class=&quot;math-container&quot;&gt;$f(x)=\infty$&lt;/span&gt; as gradient the gradient at the auxiliary point &lt;span class=&quot;math-container&quot;&gt;$x_0+t(x-x_0)$&lt;/span&gt; with finite function value, where &lt;span class=&quot;math-container&quot;&gt;$t&amp;lt;1$&lt;/span&gt; is chosen by backtracking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should fix your problems, at least when using a line search method.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="20688" LastEditDate="2019-05-09T20:52:55.140" LastActivityDate="2019-05-09T20:52:55.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8474" PostTypeId="2" ParentId="8470" CreationDate="2013-09-12T12:55:15.080" Score="15" Body="&lt;p&gt;The derivation of the BFGS is more intuitive when one considers (strictly) convex cost functionals:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, some &#xA;&lt;strong&gt;background&lt;/strong&gt;  information is necessary: Assume, one wants to minimize a &lt;strong&gt;convex&lt;/strong&gt; functional &#xA;$$&#xA;f(x) \to \min_{x\in \mathbb R^n}.&#xA;$$ &#xA;Say there is an approximate solution $x_k$. Then, one approximates the minimum of $f$ by the minimum of the truncated Taylor expansion&#xA;$$&#xA;f(x_k+p) \approx f(x_k) +\nabla f(x_k)^Tp + \frac{1}{2}p^T H(x_k)p. \quad(*)&#xA;$$&#xA;That is, one looks for $p$ such that $(*)$ is minimal and sets $x_{k+1} := x_k + p$. Computing the gradient of $(*)$ -- &quot;with respect to $p$&quot; -- and setting it to zero gives the relation&#xA;$$&#xA;H(x_k)[x_{k+1} - x_k] = \nabla f(x_{k+1}) - \nabla f(x_k),&#xA;$$&#xA;where $H$ is the 'Jacobian of the gradient' or the Hessian matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the computation and inversion of the Hessian is expensive...&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;...&lt;strong&gt;a short answer&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;(cf. Broyden's update) might be that the BFGS update $H_{k+1}^{-1}$ minimizes&#xA;$$&#xA;\|H_k^{-1} - H^{-1}\|_W&#xA;$$&#xA;in a smartly chosen weighted Frobenius norm, subject to&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$H[x_{k+1} - x_k] = \nabla f(x_{k+1}) - \nabla f(x_k)$ -- this is what one is out for -- and&lt;/li&gt;&#xA;&lt;li&gt;$H^T = H$, because the Hessian is symmetric.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Then the choice of the weight $W$ in $\|H\|_W := \|W^{1/2}HW^{1/2}\|_F$ &#xA;&lt;strike&gt;as the inverse of&lt;/strike&gt; the averaged Hessian $G:=\int_0^1 H(x_k + \tau p) d\tau$, cf. &lt;a href=&quot;http://www.math.psu.edu/~anovikov/acm113/qn.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; for the statement but without proof, gives the &lt;a href=&quot;http://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm&quot; rel=&quot;noreferrer&quot;&gt;BFGS update formula&lt;/a&gt; (with $\alpha_k = 1$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The major points are:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One tries to approximate the solution for the actual costs by the solution for a quadratic approximation&lt;/li&gt;&#xA;&lt;li&gt;Computation of the Hessian, and its inverse, is expensive. One prefers simple updates.&lt;/li&gt;&#xA;&lt;li&gt;The update is chosen optimal for the &lt;em&gt;inverse&lt;/em&gt; rather than the actual Hessian. &lt;/li&gt;&#xA;&lt;li&gt;That it is a rank-2 update is a consequence of the particular choice of the weights in the Frobenius norm.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;longer answer&lt;/strong&gt;, should include how to choose the weights, how to make this work for nonconvex problems (where a curvature condition appears that requires a scaling of the search direction $p$), and how to derive actual the formula for the update. A reference is &lt;a href=&quot;http://perso.unifr.ch/ales.janka/numeroptim/09_newton1.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; (in German).&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2014-03-31T08:11:47.480" LastActivityDate="2014-03-31T08:11:47.480" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8475" PostTypeId="1" AcceptedAnswerId="8502" CreationDate="2013-09-12T15:38:00.277" Score="6" ViewCount="2753" Body="&lt;p&gt;I want to solve a general nonlinear constrained optimization problem&#xA;$$\min_q\ f(q)\quad \textrm{s.t.}\quad g_i(q) = 0,\ h_j(q) \geq 0.$$&#xA;The problem is that while the equality constraints $g_i(q)$ are individually smooth and have non-vanishing gradients at feasible points, the gradients are not linearly independent. If they were &lt;em&gt;algebraically&lt;/em&gt; dependent I could simply remove the redundant constraints, but unfortunately the rank of $\nabla g$ is not constant over the feasible set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the standard approach to solving such problems? Are there higher-order analogues of the KKT conditions that can be used instead, and are there standard solvers that are based on them?&lt;/p&gt;&#xA;" OwnerUserId="3612" LastActivityDate="2015-10-09T18:03:23.143" Title="linear independence constraint qualification: what to do when they don't hold?" Tags="|optimization|nonlinear-programming|nonconvex|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8476" PostTypeId="1" CreationDate="2013-09-12T16:53:58.563" Score="6" ViewCount="442" Body="&lt;p&gt;Let $M$ be a matrix which has the following properties:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) $M$ is Hermitian&#xA;2) $M$ has only rational entries&#xA;3) $M$ is known to have rational eigenvalues&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithms are there for &lt;em&gt;exactly&lt;/em&gt; computing the eigenvalues of $M$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica seems to be able to diagonalise such matrices very very quickly, so such methods must exist (I am hoping they are not trade secrets of Wolfram Inc!). I am sceptical that Mathematica is using the rational roots test on the characteristic equation since the numbers you'd need to factorise get large quickly.&lt;/p&gt;&#xA;" OwnerUserId="5142" LastActivityDate="2013-09-17T17:28:56.393" Title="What algorithms are known for computing exact eigenvalues for rational matrices?" Tags="|linear-algebra|algorithms|eigenvalues|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8477" PostTypeId="1" CreationDate="2013-09-12T18:57:01.710" Score="2" ViewCount="217" Body="&lt;p&gt;The company I work for has been developing an application for real-time control of sewer networks. Every 5 minutes, a MILP problem is built or updated, then solved using Gurobi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For mid-sized cities, this works well. But for larger cities, the MILP problems get bigger and  of course more time is required to find an acceptable solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question&lt;/strong&gt; : can anyone suggest me methods to find a good approximate solution instead of THE optimal solution? The method must be able to find a solution fast, so calculation time is an issue as important as the quality of the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, we can solve a MILP problem to its optimal value $F$, or we can find an approximate solution by doing this :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We replace half of the binary variables with continuous variables&#xA;varying between 0 and 1;&lt;/li&gt;&#xA;&lt;li&gt;We find a first optimal solution to this modified problem. Let's define $F_1$ as the optimal value of the objective function.&lt;/li&gt;&#xA;&lt;li&gt;In the original problem, we set the binary variables to their optimal values from the previous problem;&lt;/li&gt;&#xA;&lt;li&gt;The approximate solution is found by solving this last problem. Let's define $F_2$ as the optimal value of the objective function.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The quality of the solution depends on the range between $F_1$ and $F_2$ since  $F_1 \leq F \leq F_2$. This method has been giving relatively good results, but I'm pretty sure there may be other methods to find an approximate solution. I googled &quot;approximate solution MILP&quot;, but I'm not sure how reliable my findings are, and how fast a sub-optimal solution can be found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="5143" LastEditorUserId="5143" LastEditDate="2013-09-13T11:38:38.527" LastActivityDate="2013-09-13T11:38:38.527" Title="Good approximate solutions for a MILP problem" Tags="|optimization|algorithms|reference-request|approximation|mixed-integer-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8478" PostTypeId="1" AcceptedAnswerId="8486" CreationDate="2013-09-12T19:13:21.250" Score="11" ViewCount="3637" Body="&lt;p&gt;I often encounter the general adage that interior point methods are difficult to warm start. Is there an intuitive explanation behind this advice? Are there situations in which one can expect benefits from warm starting in an interior point method? Can anyone recommend some helpful references on the topic?&lt;/p&gt;&#xA;" OwnerUserId="1862" LastActivityDate="2013-09-14T02:02:57.040" Title="Why are interior point methods difficult to warm start?" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8479" PostTypeId="2" ParentId="8476" CreationDate="2013-09-12T19:33:00.933" Score="6" Body="&lt;p&gt;The characteristic polynomial of a matrix $M$ can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\chi_M(z) = z^n + \textrm{trace(M)}z^{n-1}+\ldots + \det(M)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you know ahead of time that all the entries of $M$ are rational, you can be assured that $\det(M)$ is rational; say $\det(M) = p/q$. Now, the rational roots theorem assures you that, provided $\chi_M$ has a rational root $\lambda = k/l$, then $k$ divides $p$ and $l$ divides $q$. But, you know ahead of time (somehow) that all eigenvalues of $M$ are rational.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can compute the determinant of $M$ much faster than the full characteristic polynomial by computing the LU-factorization of $M$. Since all the entries are rational, the entries of the triangular factors are also rational; you would of course have to ensure that the factorization is done symbolically so that the determinant is also computed exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying all rational numbers $k/l$ such that $k | p$ and $l | q$ is also prohibitively expensive. However, you could use a conventional eigensolver to get a set of approximate eigenvalues $\hat{\lambda}_1,\ldots,\hat{\lambda}_n$, then look for the exact, rational eigenvalues with the desired divisibility properties nearby to each approximate one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A cursory Google search didn't turn up anything for the exact eigendecomposition of rational matrices, so I doubt that anyone has written a program to do this before. However, you might have luck with the linear algebra features of &lt;a href=&quot;http://docs.sympy.org/0.6.7/modules/matrices.html&quot; rel=&quot;nofollow&quot;&gt;sympy&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2013-09-17T17:28:56.393" LastActivityDate="2013-09-17T17:28:56.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8480" PostTypeId="1" CreationDate="2013-09-12T21:36:36.900" Score="2" ViewCount="464" Body="&lt;p&gt;I am a little confused with PETSc's documentation for &lt;code&gt;MatSetValuesBlocked&lt;/code&gt;. The code below works fine for matrices when I choose small block sizes, but I get errors when using bigger block sizes for the same matrix. The two main questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What is the right way to use &lt;code&gt;MatSetValuesBlocked&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) What is the correct way to generate &lt;code&gt;idxn&lt;/code&gt; and &lt;code&gt;idxm&lt;/code&gt;, global indices for the row and column blocks, respectively. As an example, for a matrix with m=n=20 and block size = 2, are idxn and idxm = {0,1,2,3,4,5,6,7,8,9} ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not sure if there's a mistake in any other function, but I'd like to know what I am doing wrong!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    MatCreateSeqBAIJ(PETSC_COMM_SELF,bs,m,n,nnz*2/m,0,&amp;amp;A);&#xA;    MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE);&#xA;    PetscInt idxm[(m/bs)] ; //row index of total number of blocks&#xA;    PetscInt idxn[(m/bs)] ; //column index of total number of blocks&#xA;&#xA;    /*Values from the sparsematrix.mtx file */&#xA;&#xA;    for (i=0; i&amp;lt;nnz; i++) &#xA;    {&#xA;     fscanf(file,&quot;%d %d %le\n&quot;,&amp;amp;row,&amp;amp;col,(double*)&amp;amp;val);&#xA;     values[i] =  val;&#xA;    }&#xA;    fclose(file);&#xA;&#xA;&#xA;&#xA;/*Constructing idxm */ &#xA;&#xA;   for(m_index=0; m_index&amp;lt; (m/bs); m_index++) //Number of blocks rowwise&#xA;   {&#xA;        idxm[m_index] = m_index; // For the blocks in the first row&#xA;&#xA;   } //idxm construction complete&#xA;&#xA;&#xA; /*Constructing idxn*/&#xA;&#xA;&#xA;&#xA;  for(b_index = 0; b_index &amp;lt; (n/bs) ; b_index ++) //Each column for each block&#xA;  {&#xA;    idxn[b_index] = b_index;&#xA;&#xA;  } //idxn construction complete.&#xA;&#xA;  /*Errors occur..*/&#xA;  MatSetValuesBlocked(A,m/bs,idxm,n/bs,idxn, values, INSERT_VALUES);&#xA;&#xA;  MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);&#xA;  MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);&#xA;&#xA;  PetscPrintf(PETSC_COMM_SELF,&quot;Reading matrix completed....\n&quot;);&#xA;&#xA;  PetscOptionsGetString(NULL,&quot;-fout&quot;,fileout,PETSC_MAX_PATH_LEN,NULL);&#xA;  PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileout,FILE_MODE_WRITE,&amp;amp;view);&#xA;  MatView(A,view);&#xA;  VecView(b,view);&#xA;&#xA;  MatMult(A, b, res); //Actual multiplication&#xA;&#xA;  PetscPrintf(PETSC_COMM_SELF,&quot;Matmult completed...\n&quot;);&#xA;  PetscViewerDestroy(&amp;amp;view);&#xA;  ierr = VecDestroy(&amp;amp;b);CHKERRQ(ierr);&#xA;  ierr = MatDestroy(&amp;amp;A);CHKERRQ(ierr);&#xA;  ierr = VecDestroy(&amp;amp;res);CHKERRQ(ierr);&#xA;  ierr = PetscRandomDestroy(&amp;amp;r);CHKERRQ(ierr);&#xA;  ierr = PetscFinalize();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5145" LastEditorUserId="1804" LastEditDate="2016-05-31T21:31:02.617" LastActivityDate="2016-05-31T21:31:02.617" Title="PETSc: Blocking matrices using MatCreateSeqBAIJ and MatSetValuesBlocked" Tags="|linear-algebra|matrix|sparse-matrix|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8481" PostTypeId="1" CreationDate="2013-09-12T22:45:51.953" Score="20" ViewCount="925" Body="&lt;p&gt;I'm a huge advocate of test-driven development in scientific computing. It's utility in practice is just staggering, and really alleviates the classic troubles that code developers know. However, there are inherent difficulties in testing scientific codes that aren't encountered in general programming, so TDD texts aren't terribly useful as tutorials. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In general you don't know an exact answer for a given complex problem a priori, so how can you write a test?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The degree of parallelism changes; I recently encountered a bug where using MPI tasks as a multiple of 3 would fail, but a multiple of 2 worked. Additionally, common testing frameworks don't seem very MPI-friendly due to the very nature of MPI -- you have to re-execute a test binary to alter the number of tasks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Scientific codes often have a lot of tightly coupled, interdependent and interchangeable parts. We've all seen the legacy code, and we know how tempting it is to forgo good design and use global variables.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Often a numerical method may be an &quot;experiment&quot;, or the coder doesn't fully understand how it works and is trying to understand it, so anticipating results is impossible.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some examples of tests that I write for scientific code:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For time integrators, use a simple ODE with an exact solution, and test that your integrator solves it to within a given accuracy, and the order of accuracy is correct by testing with varying step sizes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Zero-stability tests: check that a method with 0 boundary/initial conditions remains at 0. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Interpolation tests: given a linear function, assure that an interpolation is correct.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Legacy validation: isolate a chunk of code in a legacy application that is know to be correct, and pull some discrete values out to use for testing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It still often comes up that I can't figure out how to properly test a given chunk of code, aside from manual trial and error. &lt;strong&gt;Can you provide some examples of tests you write for numerical code, and/or general strategies for testing scientific software?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2013-09-12T22:56:09.197" LastActivityDate="2020-07-14T16:23:08.050" Title="Strategies for unit testing and test-driven development" Tags="|testing|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8483" PostTypeId="2" ParentId="8481" CreationDate="2013-09-13T01:13:11.380" Score="10" Body="&lt;p&gt;&lt;a href=&quot;https://asmedigitalcollection.asme.org/fluidsengineering/article-abstract/124/1/4/462791/Code-Verification-by-the-Method-of-Manufactured?redirectedFrom=fulltext&quot; rel=&quot;nofollow noreferrer&quot;&gt;Method of manufactured solutions&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Verify through refinement studies that the method achieves the theoretical order of accuracy.&lt;/p&gt;&#xA;&lt;p&gt;Conservation of answer. Bit-wise and norm-wise reproduction of solutions.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2020-07-14T16:23:08.050" LastActivityDate="2020-07-14T16:23:08.050" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8486" PostTypeId="2" ParentId="8478" CreationDate="2013-09-13T04:30:23.030" Score="12" Body="&lt;p&gt;Interior point methods work by following the central path to an optimal solution.  When you change the objective function, the optimal solution from the previous version of the problem is far from the central path for the new problem, so it takes several iterations to get back to the central path and furthermore has to return to a fairly well centered solution.  Then you have to work you way down the path to a new optimal solution.  You might as well just start the interior point method from an arbitrary point.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In comparison, the simplex method (primal or dual) moves from vertex to vertex of the feasible set.  In the typical case, a reasonably small change in the objective will result in a new optimal solution that is only a few simplex pivots away.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;...added to the intuitive explanation above to give more detail...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In computational practice, experience simply hasn't shown any substantial benefit to warm starting primal-dual interior point methods.  It isn't a feature of widely used codes like CPLEX and Gurobi (the companies that produce these packages would be certain to add such a feature if it was worth while), and there are relatively few papers discussing strategies for warm starting interior point methods.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two references that I'll recommend are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E. A. Yildirim and S. Wright.  Warm-Start Strategies in Interior-Point Methods for Linear Programming.  SIAM Journal on Optimization 12:782-810, 2002.  This paper gives some nice theoretical bounds on some warm starting strategies.  See &#xA;&lt;a href=&quot;http://pages.cs.wisc.edu/~swright/papers/YilW02a.pdf&quot;&gt;http://pages.cs.wisc.edu/~swright/papers/YilW02a.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A later paper coauthored by Yildirim gives some computational results, but the authors admit that simply cold starting is often faster in their tests than warm starting:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E. John and E. A. Yildirim.  Implementation of warm-start strategies in interior-point methods for linear programming in fixed dimension.  Computational Optimization and Applications. 41:151-183, 2008.  See &#xA;&lt;a href=&quot;http://link.springer.com/article/10.1007/s10589-007-9096-y&quot;&gt;http://link.springer.com/article/10.1007/s10589-007-9096-y&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2013-09-14T02:02:57.040" LastActivityDate="2013-09-14T02:02:57.040" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8487" PostTypeId="2" ParentId="8480" CreationDate="2013-09-13T06:25:51.990" Score="3" Body="&lt;p&gt;&lt;code&gt;MatSetValuesBlocked&lt;/code&gt; contributes logically dense blocks.  Given the pattern shown above, you should be calling it once per block-row.  You're currently calling it as though the arguments were COO format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also be aware that reading a matrix from a file is a workflow choice that guarantees that your software will never be scalable.  Long-term, you should choose a way to avoid using the file system during matrix assembly.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-09-13T06:25:51.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8488" PostTypeId="2" ParentId="8481" CreationDate="2013-09-13T06:50:41.150" Score="7" Body="&lt;p&gt;Bill has already listed a few methods which address your concerns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addressing your third point, no, there is no reason to introduce strong coupling between parts. Just the opposite: if your functions or classes have well defined interfaces, it will be much easier to exchange for instance a linear solver for another, or a time stepping scheme. Just resist it, and then you will be able to test these components separately. We have done this with deal.II for decades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your fourth point: if your method is an experiment, your experiments with the method constitute a test. As long as you do not have analysis, you will have to take these test results as best available. But usually, you have an expectation for instance for the order of a method, or you would know it is exact for a certain class of solutions, for instance polynomials up to a certain degree. Verifying these should be part of your experiments, and as analysis improves, tests can be added. &lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-13T06:50:41.150" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8489" PostTypeId="1" CreationDate="2013-09-13T08:29:13.037" Score="2" ViewCount="112" Body="&lt;p&gt;Basically I try to compute several smallest eigenvalues of some sparse 50k*50k eigenvalue problems using matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax = \lambda Bx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With matlab eigs, it's not as fast as I expected. So I tried some preconditioned routine like lobpcg, however it said &quot;The problem size is too small, relative to the block size. Try using eig or eigs instead&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone can give some suggestion on how can I circumvent this with lobpcg or using some other routine?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="27" LastEditDate="2013-09-13T13:17:50.400" LastActivityDate="2013-09-13T13:17:50.400" Title="Preconditioner for large size hermitian eigenvalue problems" Tags="|linear-algebra|algorithms|matlab|sparse-matrix|eigensystem|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8490" PostTypeId="1" AcceptedAnswerId="8500" CreationDate="2013-09-13T00:27:32.483" Score="9" ViewCount="10239" Body="&lt;p&gt;I am trying to model heat conduction within a wood cylinder using implicit finite difference methods. The general heat equation that I'm using for cylindrical and spherical shapes is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/jQt82.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where p is the shape factor, p = 1 for cylinder and p = 2 for sphere.  Boundary conditions include convection at the surface. For more details about the model, please see the comments in the Matlab code below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main m-file is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%--- main parameters&#xA;rhow = 650;     % density of wood, kg/m^3&#xA;d = 0.02;       % wood particle diameter, m&#xA;Ti = 300;       % initial particle temp, K&#xA;Tinf = 673;     % ambient temp, K&#xA;h = 60;         % heat transfer coefficient, W/m^2*K&#xA;&#xA;% A = pre-exponential factor, 1/s and E = activation energy, kJ/mol&#xA;A1 = 1.3e8;     E1 = 140;   % wood -&amp;gt; gas&#xA;A2 = 2e8;       E2 = 133;   % wood -&amp;gt; tar&#xA;A3 = 1.08e7;    E3 = 121;   % wood -&amp;gt; char &#xA;R = 0.008314;   % universal gas constant, kJ/mol*K&#xA;&#xA;%--- initial calculations&#xA;b = 1;          % shape factor, b = 1 cylinder, b = 2 sphere&#xA;r = d/2;        % particle radius, m&#xA;&#xA;nt = 1000;      % number of time steps&#xA;tmax = 840;     % max time, s&#xA;dt = tmax/nt;   % time step spacing, delta t&#xA;t = 0:dt:tmax;  % time vector, s&#xA;&#xA;m = 20;         % number of radius nodes&#xA;steps = m-1;    % number of radius steps&#xA;dr = r/steps;   % radius step spacing, delta r&#xA;&#xA;%--- build initial vectors for temperature and thermal properties&#xA;i = 1:m;&#xA;T(i,1) = Ti;    % column vector of temperatures&#xA;TT(1,i) = Ti;   % row vector to store temperatures &#xA;pw(1,i) = rhow; % initial density at each node is wood density, rhow&#xA;pg(1,i) = 0;    % initial density of gas&#xA;pt(1,i) = 0;    % inital density of tar&#xA;pc(1,i) = 0;    % initial density of char&#xA;&#xA;%--- solve system of equations [A][T]=[C] where T = A\C&#xA;for i = 2:nt+1&#xA;&#xA;    % kinetics at n&#xA;    [rww, rwg, rwt, rwc] = funcY(A1,E1,A2,E2,A3,E3,R,T',pw(i-1,:));&#xA;    pw(i,:) = pw(i-1,:) + rww.*dt;      % update wood density&#xA;    pg(i,:) = pg(i-1,:) + rwg.*dt;      % update gas density&#xA;    pt(i,:) = pt(i-1,:) + rwt.*dt;      % update tar density&#xA;    pc(i,:) = pc(i-1,:) + rwc.*dt;      % update char density&#xA;    Yw = pw(i,:)./(pw(i,:) + pc(i,:));  % wood fraction&#xA;    Yc = pc(i,:)./(pw(i,:) + pc(i,:));  % char fraction&#xA;    % thermal properties at n&#xA;    cpw = 1112.0 + 4.85.*(T'-273.15);   % wood heat capacity, J/(kg*K) &#xA;    kw = 0.13 + (3e-4).*(T'-273.15);    % wood thermal conductivity, W/(m*K)&#xA;    cpc = 1003.2 + 2.09.*(T'-273.15);   % char heat capacity, J/(kg*K)&#xA;    kc = 0.08 - (1e-4).*(T'-273.15);    % char thermal conductivity, W/(m*K)&#xA;    cpbar = Yw.*cpw + Yc.*cpc;  % effective heat capacity&#xA;    kbar = Yw.*kw + Yc.*kc;     % effective thermal conductivity&#xA;    pbar = pw(i,:) + pc(i,:);   % effective density&#xA;    % temperature at n+1&#xA;    Tn = funcACbar(pbar,cpbar,kbar,h,Tinf,b,m,dr,dt,T);&#xA;&#xA;    % kinetics at n+1&#xA;    [rww, rwg, rwt, rwc] = funcY(A1,E1,A2,E2,A3,E3,R,Tn',pw(i-1,:));&#xA;    pw(i,:) = pw(i-1,:) + rww.*dt;&#xA;    pg(i,:) = pg(i-1,:) + rwg.*dt;&#xA;    pt(i,:) = pt(i-1,:) + rwt.*dt;&#xA;    pc(i,:) = pc(i-1,:) + rwc.*dt;&#xA;    Yw = pw(i,:)./(pw(i,:) + pc(i,:));&#xA;    Yc = pc(i,:)./(pw(i,:) + pc(i,:));&#xA;    % thermal properties at n+1&#xA;    cpw = 1112.0 + 4.85.*(Tn'-273.15);&#xA;    kw = 0.13 + (3e-4).*(Tn'-273.15);&#xA;    cpc = 1003.2 + 2.09.*(Tn'-273.15);&#xA;    kc = 0.08 - (1e-4).*(Tn'-273.15);&#xA;    cpbar = Yw.*cpw + Yc.*cpc;&#xA;    kbar = Yw.*kw + Yc.*cpc; &#xA;    pbar = pw(i,:) + pc(i,:);&#xA;    % revise temperature at n+1&#xA;    Tn = funcACbar(pbar,cpbar,kbar,h,Tinf,b,m,dr,dt,T);&#xA;&#xA;    % store temperature at n+1&#xA;    T = Tn;&#xA;    TT(i,:) = T';&#xA;end&#xA;&#xA;%--- plot data&#xA;figure(1)&#xA;plot(t./60,TT(:,1),'-b',t./60,TT(:,m),'-r')&#xA;hold on&#xA;plot([0 tmax/60],[Tinf Tinf],':k')&#xA;hold off&#xA;xlabel('Time (min)'); ylabel('Temperature (K)');&#xA;sh = num2str(h);  snt = num2str(nt);  sm = num2str(m);&#xA;title(['Cylinder Model, d = 20mm, h = ',sh,', nt = ',snt,', m = ',sm])&#xA;legend('Tcenter','Tsurface',['T\infty = ',num2str(Tinf),'K'],'location','southeast')&#xA;&#xA;figure(2)&#xA;plot(t./60,pw(:,1),'--',t./60,pw(:,m),'-','color',[0 0.7 0])&#xA;hold on&#xA;plot(t./60,pg(:,1),'--b',t./60,pg(:,m),'b')&#xA;hold on&#xA;plot(t./60,pt(:,1),'--k',t./60,pt(:,m),'k')&#xA;hold on&#xA;plot(t./60,pc(:,1),'--r',t./60,pc(:,m),'r')&#xA;hold off&#xA;xlabel('Time (min)'); ylabel('Density (kg/m^3)');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The function m-file, funcACbar, that creates the system of equations to solve is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Finite difference equations for cylinder and sphere&#xA;% for 1D transient heat conduction with convection at surface&#xA;% general equation is:&#xA;% 1/alpha*dT/dt = d^2T/dr^2 + p/r*dT/dr for r ~= 0&#xA;% 1/alpha*dT/dt = (1 + p)*d^2T/dr^2     for r = 0&#xA;% where p is shape factor, p = 1 for cylinder, p = 2 for sphere&#xA;&#xA;function T = funcACbar(pbar,cpbar,kbar,h,Tinf,b,m,dr,dt,T)&#xA;&#xA;alpha = kbar./(pbar.*cpbar);    % effective thermal diffusivity&#xA;Fo = alpha.*dt./(dr^2);         % effective Fourier number&#xA;Bi = h.*dr./kbar;               % effective Biot number&#xA;&#xA;% [A] is coefficient matrix at time level n+1&#xA;% {C} is column vector at time level n&#xA;A(1,1) = 1 + 2*(1+b)*Fo(1);&#xA;A(1,2) = -2*(1+b)*Fo(2);&#xA;C(1,1) = T(1);&#xA;&#xA;for k = 2:m-1&#xA;    A(k,k-1) = -Fo(k-1)*(1 - b/(2*(k-1)));   % Tm-1&#xA;    A(k,k) = 1 + 2*Fo(k);                    % Tm&#xA;    A(k,k+1) = -Fo(k+1)*(1 + b/(2*(k-1)));   % Tm+1&#xA;    C(k,1) = T(k);&#xA;end&#xA;&#xA;A(m,m-1) = -2*Fo(m-1);&#xA;A(m,m) = 1 + 2*Fo(m)*(1 + Bi(m) + (b/(2*m))*Bi(m));&#xA;C(m,1) = T(m) + 2*Fo(m)*Bi(m)*(1 + b/(2*m))*Tinf;&#xA;&#xA;% solve system of equations [A]{T} = {C} where temperature T = [A]\{C}&#xA;T = A\C;&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And finally the function that deals with the kinetic reactions, funcY, is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Kinetic equations for reactions of wood, first-order, Arrhenious type equations&#xA;% K = A*exp(-E/RT) where A = pre-exponential factor, 1/s&#xA;% and E = activation energy, kJ/mol&#xA;&#xA;function [rww, rwg, rwt, rwc] = funcY(A1,E1,A2,E2,A3,E3,R,T,pww)&#xA;&#xA;K1 = A1.*exp(-E1./(R.*T));    % wood -&amp;gt; gas (1/s)&#xA;K2 = A2.*exp(-E2./(R.*T));    % wood -&amp;gt; tar (1/s)&#xA;K3 = A3.*exp(-E3./(R.*T));    % wood -&amp;gt; char (1/s)&#xA;&#xA;rww = -(K1+K2+K3).*pww;      % rate of wood consumption (rho/s)&#xA;rwg = K1.*pww;               % rate of gas production from wood (rho/s)&#xA;rwt = K2.*pww;               % rate of tar production from wood (rho/s)&#xA;rwc = K3.*pww;               % rate of char production from wood (rho/s)&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running the above code gives a temperature profile at the center and surface of the wood cylinder:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/qsyv9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see from this plot, for some reason the center and surface temperatures rapidly converge at the 2 min mark which isn't correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to fix this or create a more efficient way to solve the problem?&lt;/p&gt;&#xA;" OwnerUserId="5146" OwnerDisplayName="Gavin" LastEditorUserId="5146" LastEditDate="2013-09-13T11:34:05.013" LastActivityDate="2021-03-08T18:02:10.913" Title="Matlab solution for implicit finite difference heat equation with kinetic reactions" Tags="|matlab|matrix|finite-difference|computational-chemistry|implicit-methods|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8491" PostTypeId="2" ParentId="8468" CreationDate="2013-09-13T09:10:03.590" Score="3" Body="&lt;p&gt;Sorry not quite the answer that you are after...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never been able to get the ParaView Python interface to do anything that useful. Though I admit I am only passable at Python and the stuff I want to do is normally outside the box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend you get your head around VTK, you will be able to do much much more. Not only are you not restricted by what ParaView gives you Python interface-wise, you will get access to much more VTK features than ParaView exposes. I now tend to test a few things out in ParaView and then jump into VTK once I know what works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a link to Python examples and a video tut.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.vtk.org/Wiki/VTK/Examples/Python&quot; rel=&quot;nofollow&quot;&gt;VTK Python Code&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.kitware.com/media/protrainingwebinars.php#introvtk&quot; rel=&quot;nofollow&quot;&gt;VTK Python Tut&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2013-09-13T09:10:03.590" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8493" PostTypeId="1" CreationDate="2013-09-13T16:36:38.493" Score="2" ViewCount="853" Body="&lt;p&gt;I am working on a web-based molecular visualization program for a research project (e.g., one that can display XYZ and PDB files).  The current options are outdated (for instance, Jmol uses Java).  My advisor uses MDL Chime but almost no browser supports that plugin nowadays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea has been to use modern web tools so the visualization program is future-proof for a while.  I'm using WebGL through Three.js and it seems to be working pretty well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm already running up against some performance bottlenecks.  I need to be able to display systems of 500,000 to 2,000,000 atoms in realtime (60 fps).  These systems should be rotatable/zoomable/etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entire system is fed into WebGL as one geometry using a BufferGeometry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;500,000 atoms * 24 triangles per sphere * 60 = 720 million triangles per second&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My MBP-retina (GT 650m) chokes on this, so I have no doubt most people's computer would do the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the MDL Chime program I mentioned before renders this scene fine on my advisor's 10 year old laptop without using the GPU.  What kind of trick could it be using to achieve this performance?  The spheres don't look low-quality; there's no way it could be rendering that many triangles per second, right?&lt;/p&gt;&#xA;" OwnerUserId="4362" LastActivityDate="2013-09-13T16:36:38.493" Title="Optimizing display of millions of triangles per second" Tags="|molecular-dynamics|visualization|gpu|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8494" PostTypeId="1" AcceptedAnswerId="8498" CreationDate="2013-09-14T02:17:52.700" Score="6" ViewCount="4948" Body="&lt;p&gt;I am solving an advection-diffusion problem where the solution variable is mostly flat apart from a small region near the centre of the domain where there are shape gradients. I would like to generate mesh faces for 1D finite volume cells (see below) where the cells are clustered towards the centre of the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/UKbT9.png&quot; alt=&quot;Cell centered grid&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not attempted moving or adaptive meshing because, for this application, it will be overkill. I simply want a static but non-uniform mesh. This would seems simple on the surface but I've found it tricky and would like some advice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the following approach. A uniform distribution of cell faces is defined by, $$x_{j+1/2} = \sum\limits_{j=0}^N hn$$ where $h$ is the constant mesh spacing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To generate clustered non-uniform cell faces I plan on simply dividing the uniform mesh sequence  by a mesh density function $\rho$. For example, $$x_{j+1/2} = \sum\limits_{j=0}^N \frac{hn}{\rho}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choosing a Gaussian mesh density function, with constant added so it doesn't become singular when used as a denominator, allows the mesh density to increase near the peak of the curve,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho = ae^{\frac{(x-b)^2}{2c^2}}+1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this approach I can then generate the following mesh. Notice how the mesh points start off constant spacing (because $\rho=1$, then initially begin to &lt;strong&gt;expand&lt;/strong&gt;, and then &lt;strong&gt;contracts&lt;/strong&gt; around the centre points. I have plotted the distance between points (blue line) to highlight the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/yK0RG.png&quot; alt=&quot;Non-uniform grid&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would prefer if the mesh spacing didn't increase above the minimum $h$ value. Is there a way to preserve this property? It seems that I may need a peak function a zero second derivative. Can you suggest a better mesh density function for this problem?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-09-14T06:17:42.070" LastActivityDate="2013-09-14T16:12:48.167" Title="A simple function for generating a nonuniform mesh in 1D with fixed minimum spacing" Tags="|mesh-generation|finite-volume|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8495" PostTypeId="1" AcceptedAnswerId="8497" CreationDate="2013-09-14T07:45:28.683" Score="0" ViewCount="473" Body="&lt;p&gt;I'm solving a &lt;a href=&quot;http://www.csc.kth.se/utbildning/kth/kurser/DN1232/berva13/Lab3BuppgJO.pdf#page=16&quot; rel=&quot;nofollow noreferrer&quot;&gt;problem&lt;/a&gt; (page 16 is in English) in numerical analysis and this is the solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; z=[0:500:4000 5000:1000:12000];&#xA;&amp;gt;&amp;gt; data=[5050 4980 4930 4890 4870 4865 4860 4860 4865 4875 4885 4905 4920 4935 4950 4970 4990];&#xA;&amp;gt;&amp;gt; fun=@(p)(4800 + p(1))*ones(size(z)) +p(2)/1000*z+p(3)*exp(p(4)/1000*z)-data;&#xA;&amp;gt;&amp;gt; x0=[0 0 0 -1];&#xA;&amp;gt;&amp;gt; opt = optimset('MaxFunEvals',1000);&#xA;&amp;gt;&amp;gt; p=lsqnonlin(fun,x0,[],[],opt);&#xA;&#xA;Local minimum possible.&#xA;&#xA;lsqnonlin stopped because the final change in the sum of squares relative to &#xA;its initial value is less than the default value of the function tolerance.&#xA;&#xA;&amp;lt;stopping criteria details&amp;gt;&#xA;&#xA;&amp;gt;&amp;gt; fitf=@(t)(4800 + p(1))*ones(size(t)) + p(2)/1000*t+ p(3)*exp(p(4)/1000*t);&#xA;&amp;gt;&amp;gt; tt=linspace(0,12000,1000);&#xA;&amp;gt;&amp;gt; plot(z,data,'r-',tt,fitf(tt),'b-');&#xA;&amp;gt;&amp;gt; p&#xA;&#xA;p =&#xA;&#xA;  -20.2090   17.3368  272.9057   -0.7528&#xA;&#xA;&amp;gt;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On the plot it looks alright:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/AwD6x.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I asked about it the coefficients to the polynomial in the answer were some other values:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/8346/ode45-usage-in-this-case&quot;&gt;ode45 usage in this case?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my program run, the coeff. appear to be p4=-0.7528 and the other are p1=-20.2090, p2=17.3368, p3=272.9057. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So c(z) becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;c(z)=4800-20.2090+(17.3368)z/1000+(272.9057)exp(0.7528z/1000)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;c’(z)=17.3368/1000-0.2054exp(-z/1000)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then I can make the substitution z’=u and z’’=u’ so the matlab code to continue with will be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dZ=sys(x,Z)&#xA;    c=@(z)4800 -20.2090 + (17.3368)*z/1000+ (272.9057)*exp(0.7528z/1000); % c(z)&#xA;    c=c(2000);&#xA;&#xA;% Z(1):=z&#xA;% Z(2):=u&#xA;    dZ=zeros(2,1);    % a column vector&#xA;    dZ(1)=Z(2);&#xA;    dZ(2)=(c/cosd(7.8))^2*((1287*exp(Z(1)/1000))/5000 + 729/50000)/...&#xA;        (4800 + 5.66 + (14.58)*Z(1)/1000+ (257.4)*exp(-Z(1)/1000))^3;&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do you agree?&lt;/p&gt;&#xA;" OwnerUserId="3566" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-01-30T05:19:54.313" Title="Are these coefficients correctly calculated?" Tags="|matlab|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8497" PostTypeId="2" ParentId="8495" CreationDate="2013-09-14T14:01:52.487" Score="3" Body="&lt;p&gt;Both your coefficients and the coefficients in the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8346/ode45-usage-in-this-case&quot;&gt;answer&lt;/a&gt; are correct but yours are better.&lt;/p&gt;&#xA;&lt;p&gt;The information of fit for the coefficients in the answer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chi-squared: 813.032070619&#xA;R-squared: 0.983460126607&#xA;R-squared adjusted: 0.981097287551&#xA;Root Mean Squared Error (RMSE): 6.91559223196&#xA;p1 = 5.6596316127433939E+00&#xA;     std err squared: 2.17628E+01&#xA;     t-stat: 1.21319E+00&#xA;     p-stat: 2.45129E-01&#xA;     95% confidence intervals: [-4.34594E+00, 1.56652E+01]&#xA;p2 = 1.4581975826123806E+01&#xA;     std err squared: 4.22221E-01&#xA;     t-stat: 2.24412E+01&#xA;     p-stat: 2.24487E-12&#xA;     95% confidence intervals: [1.31883E+01, 1.59756E+01]&#xA;p3 = 2.5743180704436827E+02&#xA;     std err squared: 8.25256E+01&#xA;     t-stat: 2.83379E+01&#xA;     p-stat: 9.14824E-14&#xA;     95% confidence intervals: [2.37948E+02, 2.76916E+02] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The information of fit for your coefficients:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chi-squared: 123.211540448&#xA;R-squared: 0.997493452776&#xA;R-squared adjusted: 0.996915018801&#xA;Root Mean Squared Error (RMSE): 2.69216226729&#xA;p1 = -2.0209016655555111E+01&#xA;     std err squared: 1.78565E+01&#xA;     t-stat: -4.78242E+00&#xA;     p-stat: 3.57957E-04&#xA;     95% confidence intervals: [-2.93381E+01, -1.10800E+01]&#xA;p2 = 1.7336770761968697E+01&#xA;     std err squared: 2.24807E-01&#xA;     t-stat: 3.65648E+01&#xA;     p-stat: 1.70974E-14&#xA;     95% confidence intervals: [1.63125E+01, 1.83611E+01]&#xA;p3 = 2.7290573753852300E+02&#xA;     std err squared: 1.98905E+01&#xA;     t-stat: 6.11913E+01&#xA;     p-stat: 0.00000E+00&#xA;     95% confidence intervals: [2.63271E+02, 2.82541E+02]&#xA;p4 = 7.5277833016177342E-01&#xA;     std err squared: 6.80991E-04&#xA;     t-stat: 2.88467E+01&#xA;     p-stat: 3.59046E-13&#xA;     95% confidence intervals: [6.96402E-01, 8.09155E-01] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These fits calculated by &lt;a href=&quot;http://zunzun.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;zunzun&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Additionally i think you missed a minus before p4 in c(z) and you should change the coefficients in dZ(2) with your coeffs.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2013-09-14T14:01:52.487" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8498" PostTypeId="2" ParentId="8494" CreationDate="2013-09-14T16:12:48.167" Score="5" Body="&lt;p&gt;I suggest the following simple algorithm:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Depending on whether you want symmetry of the mesh, either start at a point $x_0$ at the left end of your interval, or in the center. In the latter case, just mirror the mesh you obtain.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Having performed the algorithm to $x_k$, compute $x_{k+1}$ as $x_k+h_k$, where&#xA;\begin{gather}&#xA; h_k = \min \left\{ h_{\max}, \frac{\bar h}{\rho(x_k)}\right\},&#xA;\end{gather}&#xA;where $h_{\max}$ is your maximal allowed interval size, $\bar h$ is a control parameter determining the average spacing of the mesh, and $\rho$ is your density function. If you want this more fancy, you could use a denominator of $\rho(x_k+h_k/2)$, but that involves solving a nonlinear problem for $h_k$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-14T16:12:48.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8499" PostTypeId="2" ParentId="8476" CreationDate="2013-09-14T20:41:55.817" Score="1" Body="&lt;p&gt;There is a &quot;simple&quot; method, simple in the sense that we need only few lines of procedure&#xA;(here I use Maple, but Mathematica can do it also with PSLQ).&#xA;Step 1: Maple calculates the roots of the characteristic polynomial with (for instance) 200 digits.&#xA;Step 2: Maple uses the LLL method of Lenstra that gives the minimal polynomial of the roots ( If we know the roots with sufficient precision). Here the minimal polynomial has degree $1$ but, for sake of security, we ask for a polynomial of degree $2$. (if you obtain a polynomial of degree $2$, increase accuracy). The time of calculation, in the following random instance, is 0&quot;12.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;roll := rand(-10^10 .. 10^10): &#xA;Digits:= 200:&#xA;f := 1:&#xA; for i to 10 do f := f*(x-roll()*(1/roll())) end do;&#xA; f; &#xA;f := numer(expand(f));&#xA;t := time():&#xA; res := fsolve(f):&#xA;with(PolynomialTools):&#xA;&#xA;for i to 10 do &#xA;print(MinimalPolynomial(res[i], 2)) end do;&#xA;time()-t;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5153" LastEditorUserId="1552" LastEditDate="2013-09-17T12:39:39.547" LastActivityDate="2013-09-17T12:39:39.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8500" PostTypeId="2" ParentId="8490" CreationDate="2013-09-15T08:15:13.183" Score="2" Body="&lt;p&gt;It looks like the model you're trying to solve is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;(1/\alpha(w,c))T_{t}(r,t) &amp;amp;= T_{rr}(r,t) + (p/r) \cdot T_{r}(r,t) \\&#xA;w_{t}(r,t) &amp;amp;= -(k_{1}(T(r,t)) + k_{2}(T(r,t)) + k_{3}(T(r,t)))w(r,t) \\&#xA;g_{t}(r,t) &amp;amp;= k_{1}(T(r,t))w(r,t) \\&#xA;a_{t}(r,t) &amp;amp;= k_{2}(T(r,t))w(r,t) \\&#xA;c_{t}(r,t) &amp;amp;= k_{3}(T(r,t))w(r,t)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$r =$ radius&lt;/li&gt;&#xA;&lt;li&gt;$t =$ time&lt;/li&gt;&#xA;&lt;li&gt;$T =$ temperature&lt;/li&gt;&#xA;&lt;li&gt;$w =$ wood density&lt;/li&gt;&#xA;&lt;li&gt;$g =$ gas density&lt;/li&gt;&#xA;&lt;li&gt;$a =$ tar density&lt;/li&gt;&#xA;&lt;li&gt;$c =$ char density&lt;/li&gt;&#xA;&lt;li&gt;$k_{i}$ is a rate coefficient for $i = 1, 2, 3$&lt;/li&gt;&#xA;&lt;li&gt;$\alpha$ is a thermal diffusivity that is a function of $w$ and $c$&lt;/li&gt;&#xA;&lt;li&gt;$p$ is a constant shape factor&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and subscripts of $t$ and $r$ denote derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You only have boundary conditions for temperature (which are the only boundary conditions you need). Although you mention the convection boundary condition at the surface, your other boundary condition should be a symmetry condition: $T_{r}(r = 0, t) = 0$ for all time, which you should impose as part of the system you form in &lt;code&gt;funcACbar&lt;/code&gt;, rather than omitting that term from the PDE at $r=0$ and discretizing the resulting equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like you're using some sort of two-stage implicit predictor-corrector-like scheme to integrate the equations for temperature, where you do the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\tilde{T}(r, t_{n+1}) &amp;amp;= T(r, t_{n}) + hf(\tilde{T}(r, t_{n + 1})) \\&#xA;T(r, t_{n + 1}) &amp;amp;= \tilde{T}(r, t_{n + 1}) + hf(T(r, t_{n + 1}))&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the function $f$ denotes the right-hand side of the differential equation for $T$ (it's really a function of more variables, but you treat the other variables as essentially constant over each of the two stages of your integrator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within each stage, you advance the species densities using explicit Euler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see a couple potential problems with this scheme:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Since temperature is really coupled to the wood and char densities, you're introducing what is called a (physics) splitting error that could be causing the numerical artifacts you've mentioned. Shrinking your time step will reduce this error.&lt;/li&gt;&#xA;&lt;li&gt;Chemical source terms are sometimes stiff. You're integrating them with explicit Euler, which I wouldn't think to do intuitively (based on the problems I study), due to stability issues. However, for most of your problem, there doesn't seem to be any great instability, and any instabilities you have are damped out, so maybe that's not an issue. Combining explicit and implicit methods in this way usually limits accuracy to first or second-order (depending on splitting), unless you use IMEX (implicit-explicit) methods.&lt;/li&gt;&#xA;&lt;li&gt;Your biggest problem is rolling your own time integrator, so you have no accuracy control. Or rather, your only way to control accuracy is to shrink the time step, look at your solution, and see if the new solution is more accurate.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here's what I would do:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Discretize your equations in space, and solve &lt;em&gt;all&lt;/em&gt; of them simultaneously (for now). If you have twenty points in the radial direction, and five state variables in the continuous formulation of your PDE, you'll only have 100 state variables in your right-hand side. MATLAB should be able to handle that fairly easily. Implementing this suggestion will eliminate splitting errors.&lt;/li&gt;&#xA;&lt;li&gt;For time integration, use something from a library. Given that you have diffusion terms and chemistry terms, you probably want to use something implicit. If you find that none of the built-in MATLAB integrators work, download the MATLAB interface to SUNDIALS, install it, and use CVODE. Implementing this suggestion will give you error-controlled time integration, and these integrators will adapt their time steps to satisfy error tolerances you supply, so you can ask for solutions that are more accurate or less accurate depending on your accuracy requirements.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;After you do those things, it will be easier to troubleshoot your problems.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-09-17T04:19:25.620" LastActivityDate="2013-09-17T04:19:25.620" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8501" PostTypeId="2" ParentId="8475" CreationDate="2013-09-15T17:32:50.073" Score="0" Body="&lt;p&gt;The LICQ is only one condition that ensures solvability of the problem. However, there are weaker conditions that you can find in the literature. They guarantee that a unique solution exists, but typically you do not get unique Lagrange multipliers in these cases.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-15T17:32:50.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8502" PostTypeId="2" ParentId="8475" CreationDate="2013-09-16T01:55:58.983" Score="3" Body="&lt;p&gt;The best reference I've seen on establishing relationships between constraint qualifications is &lt;a href=&quot;http://www.researchgate.net/publication/228978247_Karush-Kuhn-Tucker_Optimality_Conditions_and_Constraint_Qualifications_trhough_a_Cone_Approach&quot; rel=&quot;nofollow&quot;&gt;Constraint Qualification for Nonlinear Programming&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the function $g$ is nonlinear, your problem is likely nonconvex, so the Slater condition will be inapplicable to your problem, but any other constraint qualification should work, as long as you can show it holds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The canonical reference for constraint qualification seems to be &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/1015075&quot; rel=&quot;nofollow&quot;&gt;A Review of Constraint Qualification in Finite-Dimensional Spaces&lt;/a&gt; (paywalled by SIAM), even though the reference is old.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are ways to deduce optimality &lt;em&gt;without&lt;/em&gt; constraint qualifications for convex programs (see &lt;a href=&quot;http://link.springer.com/article/10.1007/BF00933129&quot; rel=&quot;nofollow&quot;&gt;Characterization of optimality in convex programming without a constraint qualification&lt;/a&gt;, paywalled by Springer). The abstract of &lt;a href=&quot;http://link.springer.com/article/10.1007/BF00934107&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt; (also paywalled by Springer) may be what you're looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trouble with generalizing constraint qualifications is that the weaker they get, the harder they are to verify. One of the weakest constraint qualifications is the Abadie Constraint Qualification, but it's very difficult to use in practice. It's considerably easier to verify a condition like the linearly independent constraint qualification or the constant rank constraint qualification.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-10-09T18:03:23.143" LastActivityDate="2015-10-09T18:03:23.143" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8503" PostTypeId="1" CreationDate="2013-09-16T07:00:24.177" Score="2" ViewCount="56" Body="&lt;p&gt;I would like to solve a three-dimensional nonlinear wave equation in an infinite cylindrical waveguide numerically. Since the waveguide is dispersive, shocks are less likely to form. Both the convective and the materila nonlinearity (from the state equation) are considered. Is there any open source CFD codes/softwares (like CLAWPACK) to assist me with this task? Also, please help with identifying appropriate schemes.&lt;/p&gt;&#xA;" OwnerUserId="3467" LastEditorUserId="3467" LastEditDate="2013-09-16T07:09:12.980" LastActivityDate="2013-09-16T07:09:12.980" Title="Solving nonlinear wave equation in a dispersive infinite waveguide" Tags="|fluid-dynamics|nonlinear-equations|wave-propagation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8505" PostTypeId="1" CreationDate="2013-09-17T00:18:38.407" Score="1" ViewCount="98" Body="&lt;p&gt;I'm trying to save a 'big' rational matrix in SAGE, but I'm running into problems. After computing my matrix A, which has size 5 x 10,000 and each entry contains rational numbers in fraction form with total number of digits for numerator and denominator more than 10 pages, I run the following command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;save(A, DATA + 'A'). &#xA;This gives me the following error message:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Traceback(most recent call last): &#xA;... &#xA;RuntimeError: Segmentation fault.&#xA;I tried the same save command with a 'smaller' matrix and that worked fine. I should also note that I'm using a laptop with a 64-bit operating system, x64-based processor, Windows 8, i7 CPU @ 2.40 GHz and 8 GB RAM. I'm running SAGE on a virtual machine to which I allocated 5237 MB. Let me know if you need further information. My questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why can't I save my matrix? Why do I get the above error message? What does it mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I save my matrix A using this command? Is there any other way I can save it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have asked these same questions in another forum which specifically deals with SAGE, but I'm not getting an answer there. I have also spent a lot of time searching online about this question, but haven't seen anyone with this same problem.&lt;/p&gt;&#xA;" OwnerUserId="5165" LastActivityDate="2013-09-17T02:53:19.533" Title="How do you save a large amount of data using SAGE?" Tags="|sage|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8506" PostTypeId="2" ParentId="8505" CreationDate="2013-09-17T02:53:19.533" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why can't I save my matrix? Why do I get the above error message? What does it mean?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most likely is that it means you ran out of memory.  The underlying large integer libraries sometimes segfault instead of giving more useful out of memory errors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might try saving each row of the matrix individually, e.g., row 0 is A[0], so try&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;save(A[0], DATA + 'A0')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If this works, you could load each row and make the matrix from the rows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- william&lt;/p&gt;&#xA;" OwnerUserId="5167" LastActivityDate="2013-09-17T02:53:19.533" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8508" PostTypeId="1" CreationDate="2013-09-17T05:45:28.677" Score="5" ViewCount="5908" Body="&lt;p&gt;I want to work on NVIDIA GPU cluster for my thesis.  But the cost of clusters are too high for me to bear. Although I can buy a couple of latest GPU cards, but buying cluster is impossible for me at least right now.  What is the alternative? Are there any Non profit organization promoting and renting clusters for educational purpose at subsidized pay per use rate? &lt;/p&gt;&#xA;" OwnerUserId="4932" LastActivityDate="2013-09-17T12:31:15.100" Title="Where can I get access to GPU cluster for educational purpose?" Tags="|gpu|hpc|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8509" PostTypeId="2" ParentId="8508" CreationDate="2013-09-17T12:01:44.673" Score="6" Body="&lt;p&gt;If you work at an institution in the US or have collaborators at one, you should look into the &lt;a href=&quot;http://www.xsede.org&quot;&gt;XSEDE&lt;/a&gt; program. This is a federation of the National Science Foundation funded supercomputing centers in the US which provides compute cycles on a variety of large-scale machines to open science research in the US. There are a couple of machines with GPU resources including &lt;a href=&quot;http://keeneland.gatech.edu/&quot;&gt;Keeneland&lt;/a&gt; and &lt;a href=&quot;http://www.tacc.utexas.edu/stampede&quot;&gt;Stampede&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Startup allocations are readily available to academic researchers and can be applied for on the &lt;a href=&quot;http://portal.xsede.org&quot;&gt;XSEDE User Portal&lt;/a&gt;. As a student researcher, you will need to work with your academic/thesis advisor to get them to apply for time. If your application for an allocation of time is successful, the time itself is free.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-17T12:01:44.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8510" PostTypeId="2" ParentId="8508" CreationDate="2013-09-17T12:31:15.100" Score="4" Body="&lt;p&gt;A final alternative is to rent (or apply for time on) &lt;a href=&quot;http://aws.amazon.com/articles/7249489223918169&quot; rel=&quot;nofollow&quot;&gt;a GPU cluster from Amazon EC2&lt;/a&gt;.  As Bill and others have mentioned, most countries provide some national or regional high performance computing resources for free to their scientists.  If you don't have one in your area, it is mostly a matter of finding a collaborator who does. &lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2013-09-17T12:31:15.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8511" PostTypeId="1" CreationDate="2013-09-17T13:14:40.350" Score="5" ViewCount="197" Body="&lt;p&gt;What are good references for time-stepping of the coupled incompressible Navier-Stokes-heat equation (&lt;em&gt;Boussinesq flow&lt;/em&gt;),&#xA;$$&#xA;\begin{cases}&#xA;\rho\left(\dot{\mathbf{u}} + \mathbf{u}\cdot\nabla \mathbf{u}\right) = -\nabla p + \mu\Delta \mathbf{u} + \rho(\theta) \mathbf{g},\\&#xA;\nabla\cdot \mathbf{u} = 0,&#xA;\end{cases}\\&#xA;\begin{cases}&#xA;\rho c_p \left(\dot{\theta} + \mathbf{u}\cdot\nabla\theta\right) = \nabla(\kappa\nabla\theta) + f&#xA;\end{cases}&#xA;$$&#xA;While the solution of either of the equations is well elaborated upon, and (the pitfalls of) the spatial discretizations are understood by and large, articles on time-stepping are harder to find. It is not clear to me, for example, when it makes sense to treat the nonlinear terms implicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question may or may not be treated in a more general sense, e.g.,&#xA;$$&#xA;\dot{u} = \mathcal{L}_1(u, v) + \mathcal{N}_1(u, v),\\&#xA;\dot{v} = \mathcal{L}_2(u, v) + \mathcal{N}_2(u, v).&#xA;$$&#xA;(with linear terms $\mathcal{L}_*$  and nonlinear terms $\mathcal{N}_*$).&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-09-18T11:29:36.697" LastActivityDate="2013-09-18T11:29:36.697" Title="Time-stepping for coupled nonlinear PDEs" Tags="|reference-request|nonlinear-equations|time-integration|navier-stokes|coupling|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8512" PostTypeId="1" AcceptedAnswerId="8518" CreationDate="2013-09-17T13:30:29.327" Score="2" ViewCount="1146" Body="&lt;p&gt;I've used an explicit finite difference scheme to model the 1D time dependent temperature distribution in a friction weld. I want to now verify the consistency and convergence of my algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no exact solution and assume I must use an approximation produced at a very refined mesh spacing to calculate the relative error for the coarser meshes. I plan to calculate the $L^2$ and $L^\infty$ error norms at a number of time intervals (during both heating and cooling). I am under the impression that the $L^2$ norm will provide the best overall description of the error, whereas the $L^\infty$ will allow me to bound my error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is this is a logical and robust method by which to estimate my error and confirm the consistency of my difference scheme?&lt;/p&gt;&#xA;" OwnerUserId="5135" LastActivityDate="2013-09-17T21:16:43.240" Title="Error norm in finite difference calculation" Tags="|finite-difference|convergence|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8513" PostTypeId="2" ParentId="8288" CreationDate="2013-09-17T15:41:50.197" Score="0" Body="&lt;p&gt;After some weeks and no one found the answer to my problem. I found myself the errors and the solution. To help the community I'am sharing it (meta advice).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was indeed a very simple error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really basic sampling theorem, maximum frequency based on sample rate is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f_{max} = 1/\Delta t $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;or using $ \omega $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \omega_{max} = 2\pi/\Delta t $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I changed the &lt;code&gt;def waves(source, distance, c):&lt;/code&gt; function definition. I used  $\Delta t$ to calculate $ \Delta \omega $ to sample the solution in frequency. &#xA;Also removed the calculation at zero distance since the source is the solution itself. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# analytic solution equation (A) paper&#xA;def waves(source, distance, c, dt):&#xA;    n = len(source)&#xA;    sourcew = np.fft.fft(source) # source in the frequency domain&#xA;    hnk_factor = distance/c # distance to source divided by velocity&#xA;    dw = np.pi*2.0*(1./dt)*1/n&#xA;    hankelshift = np.complex(0,-np.pi)*np.array([hankel2(0.0,hnk_factor*k*dw) for k in xrange(n)])&#xA;    hankelshift[0] = 0. # i infinity in limit&#xA;    return np.real(np.fft.ifft(hankelshift*sourcew))&#xA;&#xA;t = np.arange(0., 3.5, dt) # 3.5 seconds to evaluate the solution&#xA;source = gaussource(t, 1./fc) # source and solution at 0&#xA;solutionD = waves(source, 8000., c, dt) # solution at certain distance     D=8000 metres e.g.   &#xA;t0 = source.argmax(); # get max index time at zero&#xA;tD = solutionD.argmax(); # get max index time at distance&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also increased the solution time spam to 3.5 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And bellow is how it did work, it even can give closer results changing the sample rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;phase velocity approximation  3883.49514563&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/LYmxG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4092" LastActivityDate="2013-09-17T15:41:50.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8514" PostTypeId="1" AcceptedAnswerId="8515" CreationDate="2013-09-17T18:30:25.827" Score="7" ViewCount="285" Body="&lt;p&gt;I was wondering why there are no online or offline computational science contests? At least I couldn't find much by googling. I mean, like a topcoder for computational sciences.&#xA;I assume one reason is providing massive computational power for few areas of study which costs lot of money. Other might be to formulate right, practical problems as online contests usually have hypothetical questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is interesting that a computational scientist or engineer applies his knowledge across multiple sciences like aerospace, energy, bio, chemistry, physics, materials et all. But they somehow miss the glamor factor of consumer software.&lt;/p&gt;&#xA;" OwnerUserId="5173" LastActivityDate="2018-08-18T11:58:05.087" Title="Computational science contests. Why arent there any?" Tags="|parallel-computing|software|computational-geometry|hpc|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8515" PostTypeId="2" ParentId="8514" CreationDate="2013-09-17T19:41:00.953" Score="12" Body="&lt;p&gt;I would say that there are a number of reasons why there are no computational science contests besides the potentially massive computational resources required.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Time limits: Writing scientific computing code is usually not something that you want to rush. A lot of emphasis is on making sure it is correct, and thorough consideration of test/corner cases. Also, if the problem being solved is too hard, then you don't want to spend too much of the contest time on waiting for code to run. Also, interesting problems would take quite a long time to solve.&lt;/li&gt;&#xA;&lt;li&gt;High barrier to entry: In order to solve useful problems, you need to have a fairly detailed understanding of the physics (or whatever) as well as the math, and that already severely limits the kinds of people who would participate. For multidisciplinary problems, the set of people shrinks even further.&lt;/li&gt;&#xA;&lt;li&gt;Established best practices: There is often a very small number of good ways of solving a problem (at the low numerical level), so it all comes down to using the right software libraries. Do you allow people to use libraries? If yes, then most of the code is just fluff and wrappers around it. Where do you draw the line at what is a library, and what is a &quot;canned solution&quot;? If you don't allow people to use libraries, then do they have to implement the entire technology stack down to the linear algebra?&lt;/li&gt;&#xA;&lt;li&gt;Verifiability: How do you know that someone solved the problem correctly? You could try to ask them to solve a problem for which there is a known analytic solution. But then, why wouldn't they just figure that out? Comparing it to pre-generated numerical solutions could be problematic if their method is more accurate. Also, how do you decide on the tradeoff between speed and robustness? Shouldn't people be rewarded for writing code that can handle a wider range of problems, but is potentially slower?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It would be interesting if there were such contests. Although, if you managed to get people together to solve an interesting problem, it might be better for the scientific computing community if they just all worked together on it.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-09-17T19:41:00.953" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8516" PostTypeId="1" CreationDate="2013-09-17T20:53:35.907" Score="14" ViewCount="3536" Body="&lt;p&gt;Usually, I write serial code, and when I do, I write unit tests with some xUnit-style testing framework (MATLAB xUnit, PyUnit/nose, or Google's C++ testing framework).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on a cursory Google search, I haven't seen much on how practitioners unit test code that uses MPI. Are there any best practices for that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compared to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8481/strategies-for-unit-testing-and-test-driven-development&quot;&gt;Strategies for unit testing and test-driven development&lt;/a&gt;, I'm looking for answers relating to what software I should use for a testing framework (if any exist -- the answer could very well be &quot;roll your own code&quot;, in which case examples of custom testing code would be helpful).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of what I'm looking to test are right-hand side function evaluations and Jacobian matrix assembly routines for time steppers that will be integrating semi-discretized PDEs. I will be using PETSc, so if there's anything PETSc-specific, that would be helpful in addition to more general testing frameworks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Clarification Edits:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example would be in &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/ts/examples/tutorials/ex2.c.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;${PETSC_DIR}/src/ts/examples/tutorials/ex2.c&lt;/code&gt;&lt;/a&gt;, where I would want to test something like &lt;code&gt;RHSFunction&lt;/code&gt; (a right-hand side function evaluation) and &lt;code&gt;RHSJacobian&lt;/code&gt; (a Jacobian matrix evaluation). I'd be testing against known values for the assembled right-hand side and assembled Jacobian matrix; I can obtain these values analytically for some simple problem instances. These functions are application-specific functions that won't exercise any other application-level function, but they could call MPI if vector or matrix assembly is done within the function (as in the linked PETSc example above). If I write functions that only calculate portions of vectors or matrices local to a processor, I would want to test against the global, assembled version if possible because, being new to parallel programming, it's more intuitive for me to think of global vectors and global matrices. These tests would be run on small problem sizes and small numbers of processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of a few strategies to do this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A strategy that probably won't work well, based on the Google searches I've done on this topic, would be to construct a known output, find the relative/absolute error in parallel, and then do naive comparisons. &lt;a href=&quot;https://groups.google.com/forum/#!topic/googletestframework/nfzu1Ev5D8o&quot; rel=&quot;nofollow noreferrer&quot;&gt;The output will probably be garbled&lt;/a&gt; -- anyone who has written a &quot;Hello, world&quot; program with MPI knows why -- which limits the utility of doing the unit testing. (&lt;em&gt;This was the impetus for asking the question.&lt;/em&gt;) &lt;a href=&quot;https://stackoverflow.com/questions/16526664/unit-testing-mpi-programs-with-gtest&quot;&gt;There also seems to be some potential trickiness in calling the unit-testing framework.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Write output to file (in PETSc, for instance, using &lt;code&gt;VecView&lt;/code&gt; and &lt;code&gt;MatView&lt;/code&gt;), and compare against known output with something like &lt;a href=&quot;http://www.math.utah.edu/~beebe/software/ndiff/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ndiff&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;http://www.nongnu.org/numdiff/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;numdiff&lt;/code&gt;&lt;/a&gt;. My gut feeling with this method from previous experience doing unit testing with file comparisons is that it will be finicky, and it will require some filtering. This method seems like it would be excellent for regression testing, though, because I could replace the utilities above with a plain &lt;code&gt;diff&lt;/code&gt;, and not have to worry about matching up text formats. I've gathered that this strategy is more or less what WolfgangBangerth and andybauer are suggesting. PETSc also appears to use a similar approach for some of the testing it does.&lt;/li&gt;&#xA;&lt;li&gt;Use a unit testing framework, gather everything onto the processor with MPI rank 0, and ask it to execute unit tests only if the processor rank is 0. I could do something similar with norms (it's probably even easier that way), although the tradeoff is that any errors returned will tell me that I have a problem in my calculation, but not which elements are in error. Then I don't need to worry about any unit testing output being garbled; I only need to worry about calling the unit testing framework correctly. PETSc appears to use norm-wise comparisons within its example programs when exact solutions are available, but it does not use a unit testing framework when making those comparisons (nor should it, necessarily).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-01-22T20:27:10.187" Title="Any recommendations for unit-testing frameworks compatible with code/libraries that use MPI?" Tags="|parallel-computing|petsc|mpi|testing|reproducibility|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8517" PostTypeId="2" ParentId="8514" CreationDate="2013-09-17T20:53:44.920" Score="4" Body="&lt;p&gt;There is a lot of food for thought in this question. I would like to differentiate a bit with respect to the character of the contests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The subject of contests I know is fairly inconsequential. There are spelling bees, when it comes to mathematics, there are contests in symbolic integration, marshmallow eating contests and beauty pageants. You rarely hear about vaccination contests for nurses, sentencing contests for judges, or fire fighter competitions going beyond carrying buckets. As Victor has pointed out, computational science is a serious task, which requires skill, understanding, thought, and perseverance. All these cannot take effect in a short time contest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you find more commonly are coding challenges like the HPC Challenge mentioned in Pedro's comment. These take place over extended periods of time, and allow the contestants to use all the tools and information they need to produce a high quality result. They are usually sponsored by corporations (for instance Google) or other big entities because they need the result or use them as a recruiting device. But they have a lot of potential to raise interest in scientific computing in young people and to give them exposure when they succeed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third option are prizes for already existing accomplishments. Scientific awards have seen a considerable inflation during the last years, but by many are still seen as an important indicator for scientific achievement. Compared to all the medals that are awarded for instance in pure mathematics, computational science could definitely do with some more.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-17T20:53:44.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8518" PostTypeId="2" ParentId="8512" CreationDate="2013-09-17T21:16:43.240" Score="2" Body="&lt;p&gt;Both the $L^2$ and the $L^\infty$ norm may be viable measures of the error. And if your problem is well natured, they might even behave the same way as your mesh spacing tends to zero. But if the solution to your problem is not smooth (think of a step function, which cannot be approximated by smooth functions in $L^\infty$), or your method has worse convergence in $L^\infty$, they may converge with different orders, if at all. If you observe that, you will have to look at the mathematics behind your method to see if your implementation is consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, you can simplify your error estimation: if you compute solutions on a sequence of meshes with spacing for instance $h_k = 2^{-k}$, and the difference of two consecutive solutions behaves like $h_k^\alpha$ with positive $\alpha$, you can use the gemoetric series to deduce that your method is of order $h^\alpha$. Comparinng solutions on consecutive meshes is usually easier.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-17T21:16:43.240" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8519" PostTypeId="1" CreationDate="2013-09-17T22:51:25.793" Score="6" ViewCount="267" Body="&lt;p&gt;There's a body of work out there dealing with the discrete convergence of adaptive finite element methods using error estimators. Most deal with proving the property&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\|u-u_{k+1}\|_U \leq (1-\alpha) \|u-u_{k}\|_U, \quad  0&amp;lt;\alpha &amp;lt; 1$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;by relying on Galerkin orthogonality, monotonicity of error, error estimator information, and some other tricks to bound $\| u_{k+1}-u_k\|_U \geq \alpha\|u-u_k\|_U$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious about minimum residual (least squares) finite element methods - there's Galerkin orthogonality, minimization/monotonicity of error, and a built in error &quot;estimator&quot; (the residual in the proper norm), but I haven't found any work proving convergence of adaptive minimum residual finite element methods.  Am I overlooking some papers, or is there a roadblock to proving adaptive convergence for minimum residual FEM?&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2013-09-17T22:56:31.937" LastActivityDate="2013-11-05T16:24:14.250" Title="Proving convergence of adaptive finite elements - min res FEM?" Tags="|finite-element|error-estimation|adaptive-mesh-refinement|least-squares|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8520" PostTypeId="2" ParentId="8514" CreationDate="2013-09-18T00:15:49.793" Score="1" Body="&lt;p&gt;It happens occasionally for related subfields. For instance, in the realm of mathematics and numerical analysis &lt;a href=&quot;https://en.wikipedia.org/wiki/Hundred-dollar,_Hundred-digit_Challenge_problems&quot; rel=&quot;nofollow&quot;&gt;SIAM's Hundred Dollar Hundred Digit Challenge&lt;/a&gt; challenged participants to write a codes to compute mathematical problems to 10 digits of accuracy. It a very nice idea that got a lot of good publicity and entries (not to mention, remains a good lesson to teach students who are learning numerical analysis). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel that large scale numerical problems are not a good vessel for these types of contests. The reason the SIAM 10 digit challenge worked out so well was because the problems were all deceptively simple, but if you brute-forced it, you wouldn't get very far. The trick was to write out a short code (often less than a page) that did the problem in a unique way.&lt;/p&gt;&#xA;" OwnerUserId="5107" LastActivityDate="2013-09-18T00:15:49.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8522" PostTypeId="1" CreationDate="2013-09-18T08:53:29.857" Score="1" ViewCount="1502" Body="&lt;p&gt;I need to implement a matrix inversion of a very big matrix that currently is exceeding the memory limits of my machine (unfortunately I have little memory running on 32-bit machines). I would like to use a few PCs together to get the result. I am using Linux and I have access to several machines with MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you suggest a good C++ library that implements that function? If that's not available, any good article or algorithm on the topic is appreciated as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, I am reading these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://hpc2013.hpclatam.org/papers/HPCLatAm2013-paper-09.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Towards a Distributed GPU-Accelerated&#xA;Matrix Inversion&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/16940/in-place-inversion-of-large-matrices&quot;&gt;M.SE 16940: in-place inversion of large matricies&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5183" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-05-11T22:10:20.130" Title="C+C++ library for inversion of a large scale matrix over cluster" Tags="|linear-algebra|matrix|c++|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8523" PostTypeId="1" AcceptedAnswerId="8525" CreationDate="2013-09-18T09:08:27.227" Score="1" ViewCount="187" Body="&lt;p&gt;I'm trying to code a simple physics simulator in C++ but I'm stuck on the air resistance issue. Surfing the Internet I only find a drag coefficient for very basical area shapes when the air is perpendicular to the area. However, even when simulating a falling parallelepiped, I found this not sufficient to describe the motion of the falling body, as the air is not always perpendicular to each face. More precisely, I'm able to calculate the air drag on each face of the object, obtaining a 3D vector which describes the force acting on the body.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However it's obvious that when such an object falls through the air, even an angular momentum is induced to the body. I'm not able to determine it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about describing the air flow as a matrix of parallels vectors, thus obtaining a resultant force as a weighted sum of the effect of each vector hitting the body. However in this case I'm not able to determine the force of each single wind vector hitting the surfaces of the body.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the simplest way to obtain object rotation due to air resistance? And what is the finest way?&lt;/p&gt;&#xA;" OwnerUserId="5185" LastActivityDate="2013-09-18T11:08:59.357" Title="Simulate aerodynamics of a general shape object" Tags="|fluid-dynamics|simulation|3d|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8524" PostTypeId="1" CreationDate="2013-09-18T10:51:15.227" Score="14" ViewCount="2146" Body="&lt;p&gt;I am not familiar with the file format used in HDF5, but I am wondering if HDF5 files are suitable for revision control with git (or for example Mercurial or Subversion)? I guess what I mean is: are HDF5 files suitable for line-based diff'ing or will git have to treat an HDF5 as one big binary and store an entire copy for each revision?&lt;/p&gt;&#xA;" OwnerUserId="3724" LastEditorUserId="24110" LastEditDate="2017-05-27T02:36:01.413" LastActivityDate="2017-05-27T02:36:01.413" Title="Are HDF5 files suitable for git revision control?" Tags="|data-management|data-storage|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8525" PostTypeId="2" ParentId="8523" CreationDate="2013-09-18T11:08:59.357" Score="3" Body="&lt;p&gt;The correct description (since you want to write something &quot;physics-based&quot;) is via the Navier-Stokes equations, but it is out of the question to solve them with any kind of accuracy in real-time at 30fps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would imagine that a reasonable approximation is to assume that the body simply falls through air without perturbing it, but that the air induces forces on each face of the body in the following way: the perpendicular component of the air velocity vector produces a force normal to a face that you can probably get from the literature (assume a rectangular face, perpendicular flow with a velocity equal to the perpendicular component of the velocity vector); the tangential component of the air velocity induces a drag force that is parallel to the face and proportional to area of the face and whose magnitude is going to be proportional to the viscosity of air. You may need to play a bit with the two proportionality factors for the two components to make things look physical.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-18T11:08:59.357" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8526" PostTypeId="2" ParentId="8522" CreationDate="2013-09-18T11:12:16.290" Score="5" Body="&lt;p&gt;Like HighPerformanceMark already pointed out, one does not need to invert a matrix explicitly to solve linear systems. In the Levenberg-Marquardt algorithm, the matrix is positive definite and symmetric and, consequently, you can use the Conjugate Gradient method to solve the linear system. This avoids the need to compute the exact inverse of a matrix and is generally faster and less memory intensive. It should not be a problem at all to solve linear systems with a few 100,000 unknowns with this even on small PCs (depending on how dense/sparse your matrix is).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-18T11:12:16.290" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8529" PostTypeId="2" ParentId="8516" CreationDate="2013-09-18T11:22:46.677" Score="4" Body="&lt;p&gt;We simply roll our own code in deal.II -- in essence, we tell the framework to execute tests using &lt;code&gt;mpirun -np ...&lt;/code&gt;. We had previously just used a Makefile-based testing scheme (compile, link, execute test, then compare the output with one that had previously been saved) and you can find this here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://svn.dealii.org/branches/releases/Branch-8-0/tests/mpi/Makefile&quot; rel=&quot;nofollow&quot;&gt;https://svn.dealii.org/branches/releases/Branch-8-0/tests/mpi/Makefile&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and for context, the non-MPI targets are here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://svn.dealii.org/branches/releases/Branch-8-0/tests/Makefile.rules&quot; rel=&quot;nofollow&quot;&gt;https://svn.dealii.org/branches/releases/Branch-8-0/tests/Makefile.rules&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We are rewriting things using CMake/CTest, with the current development here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://svn.dealii.org/branches/branch_port_the_testsuite/tests/CMakeLists.txt&quot; rel=&quot;nofollow&quot;&gt;https://svn.dealii.org/branches/branch_port_the_testsuite/tests/CMakeLists.txt&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://svn.dealii.org/branches/branch_port_the_testsuite/tests/cmake/&quot; rel=&quot;nofollow&quot;&gt;https://svn.dealii.org/branches/branch_port_the_testsuite/tests/cmake/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2013-09-19T11:47:05.123" LastActivityDate="2013-09-19T11:47:05.123" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8530" PostTypeId="2" ParentId="8514" CreationDate="2013-09-18T11:29:45.717" Score="3" Body="&lt;p&gt;In addition to the other comments already made, let me point out that the complexity of challenges has not deterred such contests to happen in other areas: There was a $10M prize to the first group building a vehicle to go up 100km. Clearly not something you can do in a week in your garage. Several of the other X-Prize challenges are equally difficult. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There have also been contests in areas of scientific computing that just happen to not quite be solving PDEs. For example, Netflix has sponsored a competition to predict which movies people would like, based on their ranking of previously watched movies. I believe I've heard of other similar challenges in machine learning before. In numerical methods, the 100-digit and 10-digit challenges have already been mentioned.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-18T11:29:45.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8531" PostTypeId="2" ParentId="8516" CreationDate="2013-09-18T16:21:14.950" Score="4" Body="&lt;p&gt;There are several MPI-enabled software packages that use the &lt;a href=&quot;http://cmake.org&quot; rel=&quot;nofollow&quot; title=&quot;CMake&quot;&gt;CMake&lt;/a&gt; set of tools for testing. The ones that I can think of off the top of my head are Trilinos, VTK and ParaView. I would think that you don't want to assume that the executable needs to be launched with mpirun and/or mpiexec. CMake has support for specifying how to properly launch the executable along with different options such as the maximum number of processes to use and pre- and post-flags, if needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to look at the HPC Sites section of the &lt;a href=&quot;http://open.cdash.org/index.php?project=ParaView&quot; rel=&quot;nofollow&quot; title=&quot;ParaView dashboard&quot;&gt;ParaView dashboard&lt;/a&gt; where the tests are run on a variety of NERSC and Argonne supercomputers. Buried in there are also most of the settings that you'd need to specify in order to get it working on those machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, the &lt;a href=&quot;http://testing.sandia.gov/cdash/index.php?project=Trilinos&quot; rel=&quot;nofollow&quot; title=&quot;Trilinos Dashboard&quot;&gt;Trilinos dashboard&lt;/a&gt; has a wide variety of packages listed and to me is rather impressive in its organization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Full disclosure: I am a Kitware employee and CMake is one of the open source projects that Kitware is involved with.&lt;/p&gt;&#xA;" OwnerUserId="3469" LastEditorUserId="3469" LastEditDate="2013-09-19T16:49:11.093" LastActivityDate="2013-09-19T16:49:11.093" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8532" PostTypeId="1" CreationDate="2013-09-18T19:23:58.773" Score="1" ViewCount="107" Body="&lt;p&gt;I have a data set that I'm  trying to cluster, and I know that there will be at most N clusters (based on some physical properties of the thing the data set represents). However, there could be as few as 1 cluster. The clusters themselves are pretty tight, and the cluster separation is pretty large (the inter-cluster distance is an order of magnitude larger than the intra-cluster distance, at least). This is down-stream of other processing steps, so I'm (reasonably) confident that extreme outliers have been removed. N is generally 1, 2, or 4. Is there a well-known algorithm for this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about using K-Means, run it with $K \in \{1,...,N\}$, and somehow choose the best result (maybe the result which maximizes the intercluster to intracluster distance ratio, or something), but the criteria I've come up with for distinguishing the best run have all seemed ad hoc to me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4188" LastActivityDate="2013-11-19T00:49:05.003" Title="Clustering algorithm for a data set with at most N clusters" Tags="|algorithms|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8534" PostTypeId="1" CreationDate="2013-09-19T04:56:40.313" Score="2" ViewCount="2187" Body="&lt;p&gt;I am trying to make a system that will sort a list individuals and their preferred list of others. This may not make complete sense, but bear with me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a list of people, each with their list of other people in the main list. The idea is to group the people in to &lt;em&gt;g&lt;/em&gt; number of groups, with &lt;em&gt;n&lt;/em&gt; number of people per group, while optimizing based on preferences for other people. If one person's preferences request a different person, the algorithm would have to weight the groups so that they are paired together (if possible). If two individuals request each other, it would need to weight that higher than if it was just one way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been puzzling over this for a while, and am hoping that someone here can help find a solution to this, as I am stumped. If it makes any difference, I am developing this in JavaScript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any ideas on an algorithm to determine the optimal groupings? Pseudo-code, English description or sample code would all be very helpful.&lt;/p&gt;&#xA;" OwnerUserId="5187" LastActivityDate="2013-09-19T06:10:34.457" Title="Assigning Groups Based on Preference List" Tags="|approximation|" AnswerCount="1" CommentCount="0" ClosedDate="2013-09-19T21:00:03.557" ContentLicense="CC BY-SA 3.0" />
  <row Id="8535" PostTypeId="1" AcceptedAnswerId="8537" CreationDate="2013-09-19T06:09:01.883" Score="0" ViewCount="75" Body="&lt;p&gt;I have an equation that says essentially, $$J = A_1 * ( \exp( -V * A_2 ) -1 ) + A_3$$ where the output $J$ is what is desired and depends on three constants and the output depends on $V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I need to efficiently solve the following equation for $\eta$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J = B * ( \exp( C * \eta) - \exp(-D * \eta) ) + A_3 $$ where I have $J, B, C, D, A_3$. $J$ is the output from the first formula. $A_3$ is the same parameter as in formula one.  My solution is Newton-Raphson, but the time it takes these calculations is very slow when my step size is linearly set. The general procedure of my solution is that it plots formula 1 using known variables, resulting in pairs of points (V,J) of interest. For each J, formula 2 is solved for $\eta$ to create a new series of (V+$\eta$, J) points of interest for further analysis. I've outlined an example script of the procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody think of a way to set the grain or stp_size in a more efficient way or suggest a much faster computational method? My desired output is also shown below and its general form is very important for later analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/0giZ4.png&quot; alt=&quot;JV&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;function COMPUTATION&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;upperlimit=1.8531; lowerlimit=-1.8531;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;stp_size=0.001;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;V=[lowerlimit:stp_size:upperlimit];&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;for j = 1 : 1 : length(V)&#xA;      J(1,j) = 5.5801*10^(-31) * ( exp(- 1 * V(j) / (1 * 8.6174*10^(-5) * 300) )-1 )*100 - 7.5265; %this is formula (1)&#xA;      if J(1,j) &gt; 0&#xA;          J(1,j) = 0;&#xA;      end&#xA;       end&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;for j = 1: length(V)&#xA;      J_BV(j) = (10^(-5) * ( exp( ( 1.7 * 1 * 9.6485*10^4 *V(j) )/(8.3145*300) ) - exp( -(0.1 * 1* 9.6485*10^4 * V(j)/ (8.3145*300) )&#xA;  ) )); end&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;GIA = V + 1.2290;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;OUTPUT=OP( GIA, -J );&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;for i = 1 : length(OUTPUT)&#xA;      if isnan(OUTPUT(i))&#xA;          GIA(i) = NaN;&#xA;          J(i) = NaN;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;end end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;plot(GIA+OUTPUT, -J, 'p', 'displayname', 'desired output') hold on;&#xA;  plot(GIA, -J, 'r', 'displayname', 'input') legend('toggle') xlim([-0.7&#xA;  0.3]) end&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;function OUTPUT=OP(V, J)&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;OUTPUT = zeros( 1, length(V) );&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;for j = 2: length(V)&#xA;      B = 10^(-5);&#xA;      C = 1.7 * 1 * 9.6485*10^4/(8.3145*300);&#xA;      D = 0.1 * 1 * 9.6485*10^4/(8.3145*300);&#xA;      x_0 = real( -(8.3145*300 / (C * 1 * 9.6485*10^4)) * log( - J(1,j)/B )); %first guess&#xA;      clear x&#xA;      f_=@(x)B*( exp(C*(x) ) - exp(-D*(x)) ) - J(1,j); %this is formula (2)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if isinf(x_0)&#xA;    x_0=0;&#xA;end&#xA;&#xA;options=optimset('TolX', 1*10^(-1));&#xA;&#xA;&#xA;if isinf( f_(x_0) )&#xA;    OUTPUT(1,j) = NaN;&#xA;else&#xA;    [ OUTPUT(1,j), fval, exitvalue ] = fzero( f_, x_0, options);&#xA;    if exitvalue == -3&#xA;        OUTPUT(1,j) = NaN;&#xA;    end&#xA;&#xA;end&#xA; end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;end&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4318" LastActivityDate="2013-09-19T11:11:34.910" Title="Picking grain size or efficiently solving nonlinear equation via another way in MATLAB" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8536" PostTypeId="2" ParentId="8534" CreationDate="2013-09-19T06:10:34.457" Score="1" Body="&lt;p&gt;You're probably looking for algorithms to solve the &lt;a href=&quot;http://en.wikipedia.org/wiki/Stable_marriage_problem&quot; rel=&quot;nofollow&quot;&gt;Stable Marriage Problem&lt;/a&gt;, also called the Stable Matching Problem, for which many algorithms and implementations are available, depending on the particular problem variant.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-09-19T06:10:34.457" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8537" PostTypeId="2" ParentId="8535" CreationDate="2013-09-19T06:48:03.347" Score="5" Body="&lt;p&gt;Why not let Matlab do all the work for you? If I understood it correctly, you have a single (because you can combine both equations by eliminating $J$) non-linear equation in either one ($\eta$) or two variables ($\eta$ and $V$). For the two variables case, define your function as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [res] = F(X)&#xA;eta = X[1]&#xA;V = X[2]&#xA;A1 = 1.0; # these are just dummy values for your constants&#xA;A2 = 2.0;&#xA;A3 = 3.0;&#xA;B = 4.0;&#xA;C = 5.0;&#xA;D = 6.0;&#xA;res = A1*(exp(-V*A2)-1)+A3 - B*(exp(C*eta)-exp(-D*eta))-A3;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then you ask Matlab (or Octave) to solve the equation for you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;fsolve(@F,X0)&lt;/code&gt; where &lt;code&gt;X0&lt;/code&gt;is an initial guess for $\eta$ and $V$. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2013-09-19T11:11:34.910" LastActivityDate="2013-09-19T11:11:34.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8538" PostTypeId="1" AcceptedAnswerId="8558" CreationDate="2013-09-19T08:47:26.403" Score="11" ViewCount="4886" Body="&lt;p&gt;I would like to know how Dirichlet conditions are normally applied when using the finite volume method on a cell-centered non-uniform grid,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tcVw8.png&quot; alt=&quot;Left hand side of the cell centered grid.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current implementation simply imposes the boundary condition my fixing the value of the first cell,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\phi_1 = g_D(x_L)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\phi$ is the solution variable and $g_D(x_L)$ is the Dirichlet boundary condition value at the l.h.s. of the domain (&lt;strong&gt;NB&lt;/strong&gt; $x_L \equiv x_{1/2}$). However this is incorrect because the boundary condition should fix the value of the cell &lt;strong&gt;face&lt;/strong&gt; not the value of the &lt;strong&gt;cell&lt;/strong&gt; itself. What I should really apply is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\phi_{L} = g_D(x_L)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, lets solve the Poisson equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;0 = (\phi_x)_x + \rho(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial condition and boundary conditions,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho=-1\\&#xA;g_D(x_L)=0 \\&#xA; g_N(x_R)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(where $g_N(x_R)$ is a Neumann boundary condition on the right hand side).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/oLgPQ.png&quot; alt=&quot;Numerical solution of the Poisson equation&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice how the numerical solution has fixed the value of the cell variable to the boundary condition value ($g_D(x_L)=0$) at the left hand side. This has the affect of shifting the whole solution upwards. The effect can be minimized by using a large number of mesh points but that is not a good solution to the problem.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In what ways are Dirichlet boundary conditions applied when using the finite volume method? I assume I need to fix the value of $\phi_1$ by interpolating or extrapolating using $\phi_0$ (a ghost point) or $\phi_2$ such that the straight line going through these points has the desired value at $x_L$. Can you provide any guidance or an example of how to do this for a non-uniform cell-centered mesh?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Here is my attempt at using a ghost cell approach you suggested, does it look reasonable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation for cell $\Omega_1$ is (where $\mathcal{F}$ represents the flux of $\phi$),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathcal{F}_{3/2} - \mathcal{F}_{L} = \bar{\rho}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We need to write $\mathcal{F}_{L}$ in terms of the boundary condition using a ghost cell $\Omega_0$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{F}_{L} = \frac{\phi_1 - \phi_0}{h_{-}} \quad\quad \text{[1]} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But we ultimately need to eliminate the $\phi_0$ term from the equation. To do this we write a second equation which is the linear interpolation from the centre of cell $\Omega_0$ to the centre of cell $\Omega_1$. Conveniently this line passes through $x_L$, so this is how the Dirichlet conditions enters the discretistaion (because the value at this point is just $g_D(x_L)$),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g_D(x_L) = \frac{h_1}{2h_{-}}\phi_0 + \frac{h_0}{2h_{-}}\phi_1 \quad\quad \text{[2]}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Combining equations 1 and 2 we can eliminate $\phi_0$ and find an expression for $\mathcal{F}_L$ in terms of $\phi_1$ and $g_D(x_L)$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{F}_L = \frac{1}{h_{{-}}} \left(\phi_{1} - \frac{1}{h_{1}} \left(2 g_{D} h_{{-}} - h_{1} \phi_{1}\right)\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that we are free to choose the volume of the ghost cell we can set $h_0 \rightarrow h_1$ to give,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{F}_L = -\frac{2 g_{D}}{h_{1}} + \frac{2 \phi_{1}}{h_{{-}}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be simplified further because if the cells $\Omega_0$ and $\Omega_1$ are the same volume then we can set $h_{-}\rightarrow h_1$ finally giving,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{F}_L = \frac{2}{h_{1}} \left( \phi_1 - g_D \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;However, &lt;strong&gt;this approach has recovered the definition that is unstable&lt;/strong&gt; so I'm not too sure how to proceed? Did I interpret your advice incorrectly (@Jan)? The strange thing is that is seems to work, see below,&lt;/strike&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See below, it works,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/axVok.png&quot; alt=&quot;Updated computation, new approach agrees very well with analytical approach.&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-09-21T09:47:38.603" LastActivityDate="2013-09-21T09:48:11.210" Title="Applying Dirichlet boundary conditions to the Poisson equation with finite volume method" Tags="|boundary-conditions|finite-volume|poisson|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8540" PostTypeId="2" ParentId="8538" CreationDate="2013-09-19T12:55:33.853" Score="4" Body="&lt;p&gt;Your &quot;straight line&quot; approach would mean that $\phi_1-\frac{\phi_2-\phi_1}{x_2-x_1}(x_1-x_0)=0$, where $x_0$ is the location of the boundary and $x_i$ are the locations of the places where you define the $\phi_i$. This means that you can eliminate $\phi_1$ in terms of $\phi_2$, in the same way as in your first approach you eliminated $\phi_1$ right away by setting it to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you're finding here is why finite volumes aren't frequently used for the elliptic equations for which one poses Dirichlet conditions. They are used for conservation laws where the more natural conditions are stated in terms of fluxes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-19T12:55:33.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8541" PostTypeId="2" ParentId="8532" CreationDate="2013-09-19T14:18:41.970" Score="0" Body="&lt;p&gt;You could try a recursive approach in which given a set of points $S$, you first compute the &lt;a href=&quot;http://en.wikipedia.org/wiki/Covariance_matrix&quot; rel=&quot;nofollow&quot;&gt;covariance matrix&lt;/a&gt; $C_S$ of the points' coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then use the largest eigenvector $v_\mathsf{max}$ of $C_S$ as your search direction, i.e. for each point $x_i$, compute $\xi_i = v_\mathsf{max}\cdot x_i$, which reduces your problem to a one-dimentional clustering of the $\xi_i$ values, which can be solved by sorting the $\xi_i$ and splitting them at the largest gap between two entries, giving you two subsets $S_\mathsf{left}$ and $S_\mathsf{right}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then proceed recursively as long as the clusters produced are sufficiently well separated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this approach will fail for pathological cases, e.g. if the particles are on circle, but then so will most approaches. The main advantage is that it is independent of the number of dimensions used.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-09-19T14:18:41.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8542" PostTypeId="2" ParentId="8538" CreationDate="2013-09-19T14:24:53.367" Score="3" Body="&lt;p&gt;Let's say that your finite-volume form of the Poisson equation &#xA;$$&#xA;  \frac{d^2 \phi}{dx^2} = f&#xA;$$&#xA;can be written as &#xA;$$&#xA; \Bigl(\frac{d\phi}{dx}\Bigr)_{3/2}-\Bigl(\frac{d\phi}{dx}\Bigr)_{1/2} = \int_{x_{1/2}}^{x_{3/2}}f\, dx&#xA;$$&#xA;Of course, you can write the usual approximation&#xA;$$&#xA;  \Bigl(\frac{d\phi}{dx}\Bigr)_{3/2} = \frac{\phi_2-\phi_1}{h_+}&#xA;$$&#xA;There are some subtleties related to non-uniform grids that I'm ignoring here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question is about how to you approximate $(d\phi/dx)_{1/2}$ such that you can incorporate the b.c. $\phi_{1/2}$. One possibility is to change your approximation stencil compared to the usual stencils so that it includes the points $x_{1/2}$, $x_1$ and $x_2$. If my memory serves me correctly, that approximation is (assuming a uniform grid with spacing $h$, so you will need to extend this) &#xA;$$&#xA;  \Bigl(\frac{d\phi}{dx}\Bigr)_{1/2} = \frac{1}{h}\Bigl(-\frac{1}{3}\phi_2+3\phi_1-\frac{8}{3}\phi_{1/2}\Bigr)&#xA;$$ &#xA;but this should be checked. In that approximation, which is second-order accurate, you can incorporate your b.c. Of course, an even simpler and only first order accurate approximation would be&#xA;$$&#xA;  \Bigl(\frac{d\phi}{dx}\Bigr)_{1/2} = \frac{2}{h_1}\Bigl(\phi_1-\phi_{1/2}\Bigr)&#xA;$$&#xA;In this way, the Dirichlet boundary condition is only enforced &quot;weakly&quot; (meaning through a flux). As already commented by Wolfgang, this is one of (the?) reason why finite volume methods are not used for elliptic problems as much as finite element methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, one thing that also needs to be checked is the stability of your discretization with the second order approximation at the boundary. Off the top of my head, I don't know whether it will be stable combined with a centered second order approximation in the interior. A matrix stability analysis will tell you for sure. (I'm virtually certain that the first order approximation at the boundary will be stable.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention the possibility of using ghost points. This leads to the problem that you need to extrapolate from the interior into the ghost point and use the b.c. in the process. I suspect, but have not &quot;proved&quot; it, that at least some ghost point treatments are equivalent to using the kind of approach I've outlined above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps a little bit. &lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2013-09-19T14:24:53.367" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8543" PostTypeId="2" ParentId="8524" CreationDate="2013-09-19T16:02:22.067" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I guess what I mean is: are HDF5 files suitable for line-based diff'ing or will git have to treat an HDF5 as one big binary and store an entire copy for each revision?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The literal answer to this question is that git will not treat HDF5 files efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more useful answers about version control for projects that have some binary files, see this stackoverflow question:&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/540535/managing-large-binary-files-with-git&quot;&gt;https://stackoverflow.com/questions/540535/managing-large-binary-files-with-git&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-09-19T16:02:22.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8544" PostTypeId="2" ParentId="8524" CreationDate="2013-09-19T17:10:03.810" Score="9" Body="&lt;p&gt;&lt;em&gt;You're going to get a much better answer if you provide a few more technical details about what kind of data you're trying to put under version control, how you want to store different versions of the data, what components are likely to change and what components aren't, and whether you're truly going to have tree-like history (branches, merges).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HDF5 files are not suitable for diff-based version control under git.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;git uses a hash-based database under the hood, so it is possible to store the hash of your HDF5 data file without actually storing the file itself.  Three projects, &lt;a href=&quot;https://github.com/jedbrown/git-fat&quot;&gt;git-fat&lt;/a&gt;, &lt;a href=&quot;http://git-annex.branchable.com/&quot;&gt;git-annex&lt;/a&gt; and &lt;a href=&quot;https://github.com/schacon/git-media&quot;&gt;git-media&lt;/a&gt;, greatly simplify this process for you.  I would suggest using this approach if you have large, completely independent chunks of data which you would like to explicitly version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can separate your data storage into non-volatile and volatile regions, this will greatly improve the efficiency of your interaction with the version control database.  You may also want to consider explicitly using a database for your data if you do not need the DVCS features git offers.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2013-12-30T17:40:56.543" LastActivityDate="2013-12-30T17:40:56.543" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8545" PostTypeId="1" AcceptedAnswerId="8551" CreationDate="2013-09-19T20:07:25.100" Score="3" ViewCount="228" Body="&lt;p&gt;When solving Poisson's equation on the unit square $\Omega$ with homogeneous Dirichlet boundary conditions for $x=0$ and Robin-type conditions at the rest of the boundary,&#xA;$$&#xA;\begin{cases}&#xA;-\Delta u = 0 &amp;amp;\quad \text{ in }\Omega,\\&#xA;u = 0 &amp;amp;\quad\text{ on } \Gamma_{\text{left}},\\&#xA;\alpha(u-1) = \mathbf{n}\cdot\nabla u  &amp;amp;\quad\text{ on } \Gamma_{\text{rest}},&#xA;\end{cases}&#xA;$$&#xA;the result will will of course very much depend on the value of $\alpha$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $\alpha\gg 1$, the Robin conditions will essentially enforce $u\approx 1$ on $\Gamma_\text{rest}$, and act as homogeneous Neumann conditions if $\alpha\ll 1$. The behavior for intermediate values is less clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some numerical experiments for $\alpha\in\{10^0, 10^1, 10^2, 10^3\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/UFQYT.png&quot; alt=&quot;alpha1e0&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/19pmP.png&quot; alt=&quot;alpha1e1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/L5lxg.png&quot; alt=&quot;alpha1e3&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/anOfO.png&quot; alt=&quot;alpha1e4&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The oscillations near the boundary stand out. Is there a physical explanation for it or any way this curious behavior could be explained from the equations?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-09-19T23:54:24.093" Title="boundary oscillations with Robin boundary conditions" Tags="|boundary-conditions|poisson|oscillations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8546" PostTypeId="1" CreationDate="2013-09-19T21:07:45.290" Score="3" ViewCount="606" Body="&lt;p&gt;I asked this question over at stackoverflow, and got blasted for it.  I am currently working with legacy MD code written in C that my university uses, and haven't moved to a more modern library like OpenMM or mdcore (even though I fully want to).  Right now, our visualization routines are horribly out of date, and I am beginning to require something more advanced to make movies of my simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current visualization 'library' we use is a combination of OpenGL primitives via 'glew,' and it's X extension.  The researcher I work with a lot is in charge of the visualization routines, but here is my question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a newer, 'easy to use,' visualization library that I can link to that people could recommend?  I really need it for showing 3D simulation movies, and being able to do change perspectives, show multiple perspectives, etc, would be &lt;strong&gt;awesome&lt;/strong&gt;.  Post processing would also be nice, since I write out binary files containing position information for all the objects in my simulation.  What do other people use for molecular dynamics simulations, where you have more than just spheres?  I have spheres, cylinders, squiggly lines, etc in the current incarnation.&lt;/p&gt;&#xA;" OwnerUserId="4545" LastActivityDate="2013-09-20T04:53:27.207" Title="Finding a molecular dynamics visualization library for C" Tags="|simulation|visualization|molecular-dynamics|c|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8547" PostTypeId="1" AcceptedAnswerId="8554" CreationDate="2013-09-19T21:09:22.593" Score="3" ViewCount="79" Body="&lt;p&gt;The behavior of a pump is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If $h \geq h_{start}$, then pumped flow is a constant $Q_p$. The pump works as long as $h &amp;gt; h_{stop}$&lt;/li&gt;&#xA;&lt;li&gt;If $h \leq h_{stop}$, then pumped flow is 0. The pump will be off until $h &amp;lt; h_{start}$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here $h$ is the water level in the pumping well, $h_{start}$ is the height where the pump starts, and $h_{stop}$ is the height where the pump stops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I modeled this using MILP as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$h_{temp} = b\,h_{start} + (1-b) h_{stop}$$&#xA;$$h - h_{start} \leq M (1-b)$$&#xA;$$h_{stop} - h \leq M \,b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q = \left\{\begin{array}l&#xA;Q_p &amp;amp; \text{if } h \geq h_{temp} \\&#xA;0 &amp;amp; \text{otherwise}&#xA;\end{array}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $b$ is  a binary variable and $Q$ is the pumped flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works well when $h \geq h_{start}$ or when $h \leq h_{stop}$. But I'm not sure of the behavior when $h_{stop} \leq h \leq h_{start}$. In that case, both $b=0$ and $b=1$ are valid possibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to model this behavior such that $b$ can't change value except when $h \geq h_{start}$ or when $h \leq h_{stop}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="5143" LastActivityDate="2013-09-20T01:54:39.150" Title="How to model a pump behavior using MILP?" Tags="|mixed-integer-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8548" PostTypeId="2" ParentId="8546" CreationDate="2013-09-19T21:25:35.160" Score="7" Body="&lt;p&gt;Have you had a look at &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot;&gt;VMD&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used it ages ago to produce movies from simulation snapshots. Way back then, it could read a sequence of PDB files, render them (or generate &lt;a href=&quot;http://www.povray.org/&quot;&gt;POV-Ray&lt;/a&gt; scripts to raytrace them), and store them as individual images. I then used &lt;a href=&quot;http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-enc-images.html&quot;&gt;&lt;code&gt;mencoder&lt;/code&gt;&lt;/a&gt; to generate MPEG-4 files out of the stills. Those were the days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't used VMD since, but I can only imagine that it has gotten much, much better. Although it is intimately related to &lt;a href=&quot;http://www.ks.uiuc.edu/Research/namd/&quot;&gt;NAMD&lt;/a&gt;, if your own simulation can produce PDB files, there should be nothing stopping you from using it.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-09-19T21:25:35.160" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8549" PostTypeId="2" ParentId="8546" CreationDate="2013-09-19T21:29:34.217" Score="3" Body="&lt;p&gt;Pedro's answer would have been mine. If you want to manage balls and sticks on your own, though, &lt;a href=&quot;http://www.vtk.org/&quot; rel=&quot;nofollow&quot;&gt;VTK&lt;/a&gt; is a modern library for such things.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-19T21:29:34.217" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8550" PostTypeId="2" ParentId="8546" CreationDate="2013-09-19T22:42:45.307" Score="2" Body="&lt;p&gt;See also &lt;a href=&quot;http://qutemol.sourceforge.net&quot; rel=&quot;nofollow&quot;&gt;QuteMol&lt;/a&gt; and &#xA;&lt;a href=&quot;http://www.umass.edu/microbio/rasmol/othersof.htm&quot; rel=&quot;nofollow&quot;&gt;Free Molecular Visualization Software&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2013-09-19T22:42:45.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8551" PostTypeId="2" ParentId="8545" CreationDate="2013-09-19T23:54:24.093" Score="4" Body="&lt;p&gt;Is the sign accurate?  If so, you may have an issue since your var form should be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(\nabla u, \nabla v) - \langle n\cdot \nabla u, v\rangle_{\Gamma_{\rm rest}} = 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;substituting the Robin condition in gives &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(\nabla u, \nabla v) - \alpha\langle  u, v\rangle_{\Gamma_{\rm rest}} = -\langle 1, v\rangle_{\Gamma_{\rm rest}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can mess with your coercivity if $\alpha$ is too large. We've used $\alpha &amp;lt; 0$ in our modeling cases, but I'm not sure what your problem needs.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-09-19T23:54:24.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8553" PostTypeId="2" ParentId="7986" CreationDate="2013-09-20T01:43:37.463" Score="1" Body="&lt;p&gt;I believe you may want to have a look at string metrics:&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/String_metric&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/String_metric&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These algorithms basically attempt to define a set of similarity metrics.  You could order a list of strings by using an anagram model of the string to calculate an ordering metric. The anagram model is your primitive hash, or primary key. Anagrams are overly simplistic, but hopefully you see what I mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Levenshtein distance is a very well known similarity algorithm.&lt;/p&gt;&#xA;" OwnerUserId="3685" LastActivityDate="2013-09-20T01:43:37.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8554" PostTypeId="2" ParentId="8547" CreationDate="2013-09-20T01:54:39.150" Score="5" Body="&lt;p&gt;You're dealing with a system that has &quot;hysteresis&quot;  that is, the action of the pump depends not only on the current head $h$, but also on the past history of head.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your system has four identifiable states:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: $h \geq h_{start}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2: $h \leq h_{stop}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3: $h_{stop} \leq h \leq h_{start}$ and $h$ was most recently at $h_{start}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4: $h_{stop} \leq h \leq h_{start}$ and $h$ was most recently at $h_{stop}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use two binary variables to encode these four states at any point in time.  One binary variable is not going to be sufficient since it can only encode two states.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to model the dynamic activity of the system, then you'll need to encode this state at each moment (in some discretized sense of &quot;moment&quot;) in time.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-09-20T01:54:39.150" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8555" PostTypeId="1" CreationDate="2013-09-20T04:29:30.113" Score="3" ViewCount="90" Body="&lt;p&gt;For an Stochastic Differential Equation, e.g., &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{du}{dt} = \alpha*\sin(u*t)&#xA;$$&#xA;where $\alpha$ is normally distributed with nonzero mean, I am trying to use a stochastic collocation approach to find simple statistics such as the mean and the variance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand, $\alpha$ is sampled a few times, and the deterministic solution is solved from $t_0$ to $t_F$. Each of these samples is used in a collocation scheme to form the interpolating polynomial. Since in this case, there are multiple time steps between $t_0$ and $t_F$, don't I need to form (possibly) different interpolating polynomials for each of the time points chosen between $t_0$ and $t_F$? Would this not be very slow/inefficient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Raghu&lt;/p&gt;&#xA;" OwnerUserId="5198" LastEditorUserId="4244" LastEditDate="2013-09-20T07:12:02.393" LastActivityDate="2013-09-20T07:12:02.393" Title="Stochastic Collocation for time evolving ODE" Tags="|ode|monte-carlo|stochastic|collocation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8556" PostTypeId="2" ParentId="8546" CreationDate="2013-09-20T04:53:27.207" Score="2" Body="&lt;p&gt;Usually I use &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow&quot;&gt;VMD&lt;/a&gt; and &lt;a href=&quot;http://jp-minerals.org/vesta/en/&quot; rel=&quot;nofollow&quot;&gt;VESTA&lt;/a&gt;. But you also have softwares like &lt;a href=&quot;http://avogadro.openmolecules.net/wiki/Main_Page&quot; rel=&quot;nofollow&quot;&gt;Avogadro&lt;/a&gt; or &lt;a href=&quot;http://www.cmbi.ru.nl/molden/&quot; rel=&quot;nofollow&quot;&gt;Molden&lt;/a&gt;. Since I using VASP to simulate solids, I preferring VMD to visualize &quot;molecular dynamics movies&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very simple format for these movies is the XDATCAR, a example could be find &lt;a href=&quot;https://bitbucket.org/jkitchin/jasp/src/5a235ee2e6af/tests/ref/neb/02/XDATCAR&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. The last 3 blocks (of 13 lines) means 3 frames in your movie, before these blocks you can find the &lt;em&gt;lattice constant&lt;/em&gt;, &lt;em&gt;3 components of the 3 Bravais lattice vectors&lt;/em&gt; (3x3 matrix), &lt;em&gt;atomic species&lt;/em&gt; and &lt;em&gt;number of atoms&lt;/em&gt; for each species.&lt;/p&gt;&#xA;" OwnerUserId="5180" LastActivityDate="2013-09-20T04:53:27.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8557" PostTypeId="1" AcceptedAnswerId="8559" CreationDate="2013-09-20T12:41:21.543" Score="7" ViewCount="380" Body="&lt;p&gt;I am looking to solve large systems of non-linear ODEs. There appears to be a very large list of methods available varying in complexity, and I have a hard time searching through them and picking one. &#xA;Are any of these methods preferred for large systems? Both speed and accuracy play large roles, so I'm hoping that there are methods that are in general considered to be better for large systems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some additional information: The systems usually consist of ~100 ODEs that are quite heavily linked, usually consisting of a lot of quartic terms. (2-loop renormalization group equations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="1264" LastEditorUserId="123" LastEditDate="2013-09-20T21:08:48.733" LastActivityDate="2013-09-20T21:08:48.733" Title="Numerically solving systems of about 100 ODEs" Tags="|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8558" PostTypeId="2" ParentId="8538" CreationDate="2013-09-20T12:53:41.747" Score="3" Body="&lt;p&gt;In the stability-analysis of FVM discretizations for elliptic problems with Dirichlet BC, a central assumption is that the &lt;em&gt;inner&lt;/em&gt; cells, where you state the PDE, have no intersection with the boundary, i.e.&#xA;$$&#xA;\overline \Omega_i \cap \Gamma_D = 0 \quad \quad (*)&#xA;$$&#xA;if seen as a set in $\mathbb R^{n-1}$ if your domain $\Omega \subset \mathbb R^n$, cf., e.g., the book by [&lt;a href=&quot;http://books.google.de/books/about/Numerical_Treatment_of_Partial_Different.html?hl=de&amp;amp;id=Nu7-ZZAyPZYC&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grossmann&amp;amp;Roos&lt;/a&gt;, p. 92]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, if in your setup, the approach &#xA;$$&#xA;  \Bigl(\frac{d\phi}{dx}\Bigr)_{1/2} = \frac{2}{h_1}\Bigl(\phi_1-\phi_{1/2}\Bigr) \quad \quad (**)&#xA;$$&#xA;is unstable, this is &lt;strike&gt;no&lt;/strike&gt; in contradiction to known stability results. &lt;strong&gt;EDIT&lt;/strong&gt;: Using a ghost cell and linear interpolation into it, for a particular choice of volume and distance, one obtains $(**)$ as the flux. Thus, $(**)$ is indeed a stable scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stability and convergence (of first order in the discrete max-norm) for the poisson problem has been proven by Grossmann&amp;amp;Roos for grids, with distinct &lt;em&gt;boundary&lt;/em&gt; cells with their &quot;centers&quot; on the actual boundary as illustrated in my drawing for an 1D case.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/KUVef.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the differential quotient on the interface is approximated in a straight-forward manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would say &lt;strong&gt;ghost cells&lt;/strong&gt; are the common approach, because of two reasons.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They mimick the stable situation described in my drawing but with an interpolated boundary condition &lt;/li&gt;&#xA;&lt;li&gt;They are simply attached to the physical boundary. Thus, one can use a triangulation of the domain, what is also advantageous, since one often has also natural BCs that are directly imposed on the interface [&lt;a href=&quot;http://books.google.de/books/about/Numerical_Treatment_of_Partial_Different.html?hl=de&amp;amp;id=Nu7-ZZAyPZYC&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grossmann&amp;amp;Roos&lt;/a&gt;, p. 101].&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, I suggest you use ghost cells for the Dirichlet boundary. In your example, this will be adding $\phi_0$ to your system and the condition that an interpolant between $\phi_0$, $\phi_1$, and maybe others equals $g_D$ at the boundary.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-09-21T09:48:11.210" LastActivityDate="2013-09-21T09:48:11.210" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8559" PostTypeId="2" ParentId="8557" CreationDate="2013-09-20T12:57:28.410" Score="13" Body="&lt;p&gt;100 equations is not a particular large system. There are certainly many good integrators for this out there -- starting with Matlab's ode45 which should have no problems with a system of 100 equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge with ODEs is not typically the size, but the character. For example, is your system stiff? If so, you may want to look at CVODE. Do you need to preserve certain invariants? If so, you may want to look into symplectic integrators.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-20T12:57:28.410" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8561" PostTypeId="1" AcceptedAnswerId="8587" CreationDate="2013-09-20T16:06:40.223" Score="5" ViewCount="1407" Body="&lt;p&gt;For my series on &lt;a href=&quot;http://martin-ueding.de/en/physics/lagrange-examples/index.html&quot; rel=&quot;nofollow&quot;&gt;mechanical systems with Lagrange&lt;/a&gt;, I would like to add the double pendulum and a rolling pendulum. &lt;a href=&quot;http://martin-ueding.de/en/physics/lagrange-examples/sliding-pendulum/index.html&quot; rel=&quot;nofollow&quot;&gt;I set up the Lagrangian $L$ and solved for the ODE of motions, and got the following&lt;/a&gt;:&#xA;$$ \ddot x = - \frac l{2m} \cos(\phi) \ddot \phi + \frac l{2m} \sin(\phi) \dot \phi^2 $$&#xA;$$ \ddot \phi = - \frac m{2l} \sin(\phi) \dot \phi + \frac{mg}l \sin(\phi) - \frac{m}{2l} \ddot x \cos(\phi) + \frac{m}{2l} \dot x \sin(\phi) \dot \phi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the ODE solver &lt;code&gt;scipy.integrate.odeint&lt;/code&gt;, and it takes a function $\vec f$ so that:&#xA;$$ \frac{\mathrm d}{\mathrm dt} \vec y(t) = \vec f\left((\vec y(t), t\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the functions were not implicit, I would just do the following approach:&#xA;$$&#xA;\frac{\mathrm d}{\mathrm dt}&#xA;\begin{pmatrix}&#xA;x \\ \phi \\ \dot x \\ \dot \phi&#xA;\end{pmatrix}&#xA;=&#xA;\begin{pmatrix}&#xA;\dot x \\ \dot \phi \\ \text{RHS of first equation} \\ \text{RHS of second equation}&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I do not know $\ddot x$ and $\ddot\phi$ to plug in into the the calculations. So far, I have tried putting the second (hand written) equation into the first, then calculating $\ddot x$. Then I would calculate $\ddot \phi$ with that value of $\ddot x$. The result were strange results and stability issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I integrate this entangled system of equations?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Update 2013-09-22 14:56:21+02:00&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;My current function $\vec f$ is the following. I plugged $\ddot \phi$, i. e. the second equation, into the first equation and solved for $\ddot x$ so that I have an equation in $\ddot x$ without any other second derivatives on the RHS. &lt;a href=&quot;https://github.com/martin-ueding/lagrange-simulator/blob/master/lagrangesimulator.py&quot; rel=&quot;nofollow&quot;&gt;My Python code&lt;/a&gt; is the following, where I just divided by $1 - \cos^2(\phi)/4$ in the first equation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = y[0]&#xA;phi = y[1]&#xA;dot_x = y[2]&#xA;dot_phi = y[3]&#xA;&#xA;ddt_x = dot_x&#xA;ddt_phi = dot_phi&#xA;&#xA;ddt_dot_x = (- self.l / (2 * self.m) * np.cos(phi) * (- self.m / (2 * self.l) * np.sin(phi) * dot_phi + self.m * self.g / self.l * np.sin(phi) + self.m / (2 * self.l) * dot_x * np.sin(phi) * dot_phi) + self.l / (2 * self.m) * np.sin(phi) * dot_phi**2) / (1 - 1/4 * np.cos(phi)**2)&#xA;&#xA;ddt_dot_phi = - self.m / (2 * self.l) * np.sin(phi) * dot_phi + self.m * self.g / self.l * np.sin(phi) - self.m / (2 * self.l) * ddt_dot_x * np.sin(phi) + self.m / (2 * self.l) * dot_x * np.sin(phi) * dot_phi&#xA;&#xA;return [ddt_x, ddt_phi, ddt_dot_x, ddt_dot_phi]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I try to integrate this, I &lt;a href=&quot;http://chaos.stw-bonn.de/users/mu/uploads/2013-09-22/Sliding_Pendulum.js&quot; rel=&quot;nofollow&quot;&gt;get results&lt;/a&gt; that do not really reflect any physical effect. You can see that in the &lt;a href=&quot;http://chaos.stw-bonn.de/users/mu/app/lagrange-simulator/animation.html#Trajectories/Sliding_Pendulum.js&quot; rel=&quot;nofollow&quot;&gt;animation&lt;/a&gt; that does a sudden twitch.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2013-09-23T18:23:13.033" LastActivityDate="2013-09-23T23:59:35.000" Title="Convert an implicit ODE system to an explicit ODE set to use Runge-Kutta" Tags="|ode|runge-kutta|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8562" PostTypeId="2" ParentId="8561" CreationDate="2013-09-20T17:24:37.690" Score="3" Body="&lt;p&gt;My first thought would be to convert this system to a first-order system of differential algebraic equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, you'd set something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;y &amp;amp;= \dot{x}, \\&#xA;\dot{y} &amp;amp;= \ddot{x}, \\&#xA;\gamma &amp;amp;= \dot{\varphi}, \\&#xA;\dot{\gamma} &amp;amp;= \ddot{\varphi}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then rearrange your implicit ODE so it looks something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;f(x, \dot{x}, y, \dot{y}, \varphi, \dot{\varphi}, \gamma, \dot{\gamma}) = 0,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is a DAE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The potential pitfall of this approach is that your resulting DAE system might be high-index. You should see if the DAE system can be expressed as a &lt;a href=&quot;http://www.scholarpedia.org/article/Differential-algebraic_equations&quot; rel=&quot;nofollow&quot;&gt;Hessenberg index-1 or Hessenberg index-2 DAE&lt;/a&gt;, for which there good numerical methods available (for instance, IDA in the SUNDIALS package, accessed in Python from &lt;code&gt;scikits.odes&lt;/code&gt;, from Assimulo, or another Python interface to IDA). If it can't, there will likely be stability issues in calculating a solution. Most methods are geared towards solving problems with an index of less than 3. In particular, mechanical systems are known for high-index formulations. (See the book by Ascher and Petzold for some good examples.) There are many definitions of index, so I'm just going to point you to the &lt;a href=&quot;http://www.scholarpedia.org/article/Differential-algebraic_equations&quot; rel=&quot;nofollow&quot;&gt;Scholarpedia article on DAEs&lt;/a&gt; for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also DAE solvers in PETSc accessible through &lt;code&gt;petsc4py&lt;/code&gt;, but it's not clear to me from the documentation if they can solve problems with index greater than 1.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-09-20T17:24:37.690" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8563" PostTypeId="1" CreationDate="2013-09-20T19:16:17.560" Score="1" ViewCount="2135" Body="&lt;p&gt;I'm trying to calculate the divergence of a vector quantity $L(L_x, L_y, L_z)$ in COMSOL. I calculate it using the built-in differentiation operator as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla L = d(L_x,x)+d(L_y,y)+d(L_z,z)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering what is the notation or the function for divergence. Does COMSOL calculate the divergence of $L$ correctly in the way that I did in above?&lt;/p&gt;&#xA;" OwnerUserId="4858" LastEditorUserId="4858" LastEditDate="2013-09-24T00:55:26.067" LastActivityDate="2019-11-14T12:25:01.777" Title="Divergence in COMSOL" Tags="|comsol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8564" PostTypeId="1" AcceptedAnswerId="8567" CreationDate="2013-09-20T19:47:25.463" Score="0" ViewCount="1802" Body="&lt;p&gt;My default mesh geometry length is in 80 mm. For OpenFoam, I scaled the mesh in meters as 0.008 m.  Once I import the mesh in OpenFOAM, it is a collection of point coordinates. Now, the location of these points and spatial difference between different points will be the same either if they are in mm or m. I feel spatial unit is arbitrary here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to understand when do I have to scale the geometry in Openfoam and when not? &lt;/p&gt;&#xA;" OwnerUserId="5201" LastActivityDate="2013-12-23T13:10:29.573" Title="mesh spatial dimension in OpenFOAM" Tags="|mesh-generation|openfoam|mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8567" PostTypeId="2" ParentId="8564" CreationDate="2013-09-21T02:37:47.300" Score="4" Body="&lt;p&gt;In your problem description, note that $80\text{mm} \neq 0.008 \text{m}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, every quantity in a computer program is just a number. It's up to you to interpret it. So of course you can run a simulation where the domain has an edge length of 80 or 0.08 -- they refer to the same domain, after all. But depending on what you use as your base unit, you have to scale material constants as well. For example, at room temperature, the viscosity of air is $15.68\cdot 10^{-6}$ when expressed in meters (with units $m^2/s$ in this case), but $15.68$ when expressed in millimeters (with units $mm^2/s$ in that case). In other words, depending on what unit you choose as a base, your material parameters will be different.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-21T02:37:47.300" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8570" PostTypeId="1" CreationDate="2013-09-22T11:34:20.050" Score="-1" ViewCount="874" Body="&lt;p&gt;For parts of a talk, I need a note on &quot;Immersed boundary finite difference method&quot;, mainly about the reason of appearing this branch in the finite difference methods, considering mathematical principles and algorithm of implementation related to it. A survey paper, chapter of a book, lecture note or any other material concerning this topic discussing the mathematics behind is very good. Also, I need a MATLAB simulation of this method.&lt;/p&gt;&#xA;" OwnerUserId="4862" LastActivityDate="2017-06-27T21:33:37.787" Title="Any note on Immersed boundary finite difference method?" Tags="|matlab|finite-difference|reference-request|simulation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8571" PostTypeId="2" ParentId="8563" CreationDate="2013-09-22T12:45:13.913" Score="1" Body="&lt;p&gt;&lt;strike&gt; First of all, your definition of the divergence is different from the &lt;a href=&quot;http://en.wikipedia.org/wiki/Divergence#Application_in_Cartesian_coordinates&quot; rel=&quot;nofollow&quot;&gt;standard definition&lt;/a&gt;. (Assuming you are in cartesian coordinates $(x,y,z)$, and $d(\cdot,x)$ means the partial derivative wrt $x$.&lt;/strike&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding Comsol. When in cartesian coordinates, $\nabla \cdot L$ always gives you the divergence of a vector quantity $L$ &lt;strike&gt;in the sense linked above&lt;/strike&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In different coordinate systems, $\nabla \cdot$ acts differently depending on the interface where you apply it. I would love to link to the corresponding entries of the Comsol documentation, but it is &lt;a href=&quot;http://www.comsol.com/community/forums/general/thread/23683/&quot; rel=&quot;nofollow&quot;&gt;not commonly available&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-09-24T13:23:16.573" LastActivityDate="2013-09-24T13:23:16.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8572" PostTypeId="2" ParentId="8519" CreationDate="2013-09-22T15:15:22.363" Score="2" Body="&lt;p&gt;I didn't read &lt;a href=&quot;http://www.math.purdue.edu/~zcai/pdf-paper/CKS05.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; in detail, but I think it should have the gist of the kind of proof you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-22T15:15:22.363" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8573" PostTypeId="1" CreationDate="2013-09-22T15:48:53.673" Score="6" ViewCount="641" Body="&lt;p&gt;While reading many research-papers comparing parallel implementations of algorithms on different machines/architectures, I have noticed that the performance comparison is always listed in terms of &lt;strong&gt;&lt;em&gt;GFlop/s&lt;/em&gt;&lt;/strong&gt; and not the actual wall-clock time for the run in seconds. I am curious why this convention is used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My only guess is that since every company  &lt;em&gt;advertises&lt;/em&gt; its device as having a certain peak &lt;em&gt;flop-counts/second&lt;/em&gt; such research papers investigate how much of its &quot;potential&quot;&#xA;has been achieved by listing the performance as &quot;GFlop/s&quot; for the particular application at hand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, say the performance of a &lt;strong&gt;$m$ x $n$&lt;/strong&gt; Matrix -- &lt;strong&gt;$n$ x $1$&lt;/strong&gt; Vector multiply has been stated as 4 GFlop/s. Is it &lt;em&gt;reasonable&lt;/em&gt; to obtain the wall clock time in seconds by the following formula?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{m(2n-1)}{4 * 10^9} \hspace{3mm} \text{seconds}$$ &#xA;where $m(2n-1)$ is the number of floating point operations for the matrix-vector multiplication&lt;/p&gt;&#xA;" OwnerUserId="5210" LastActivityDate="2013-09-22T20:52:31.177" Title="Why performance is given in Gflop/s rather than actual time in seconds" Tags="|parallel-computing|performance|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8574" PostTypeId="2" ParentId="8573" CreationDate="2013-09-22T16:00:34.237" Score="6" Body="&lt;p&gt;Traditionally, I'd say that people more or less understood the number of floating-point operations required to solve a problem ($O(n^3)$ and the like), and so reporting performance as a rate had meaning. Readers could then obtain the run time via the method you describe, but they could also compare that to the theoretical peak performance of the hardware used to find the efficiency of the method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I like to see run time and performance rate reported. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing that might be unreasonable about your method for determining the run time is that you have to be sure you have the right operational complexity formula. If the authors are using scheme with a different operation count, then you might get the calculation very wrong. For example, what you give is probably OK for dense matrix-vector multiplication, but if this was a sparse example, some sort of approach where the zeros were not multiplied was probably used. If you use your approach to calculate the run time without trying to account for the sparsity, then you'll run into trouble.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-22T16:00:34.237" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8575" PostTypeId="2" ParentId="8573" CreationDate="2013-09-22T20:52:31.177" Score="2" Body="&lt;p&gt;With the Linpack benchmark used in ranking supercomputers (and in many other situations where people are benchmarking high performance computing systems), it's common to scale the size of the problem and report the performance in Gflop/s for the particular value of the problem size $n$ at which the machine hits peak performance.  In the case of the Linpack benchmark, the operation count is well known and standardized, so there's no question about how many floating point operations were actually performed in solving the benchmark problem at that size.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If two different machines have peak performance at very different problem sizes, then comparing the run times at these sizes (or even at one common problem size) wouldn't give a reasonable comparison.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if all you want is to solve a 100x100 system of linear equations, don't expect a machine that has a peak performance in the PetaFlop/s range to be much faster at solving your tiny system of equations than a good desktop machine!&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-09-22T20:52:31.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8576" PostTypeId="1" AcceptedAnswerId="8585" CreationDate="2013-09-22T21:42:12.460" Score="3" ViewCount="6555" Body="&lt;p&gt;Computational Science people:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The title is the question: exactly how does Matlab's &quot;interp2&quot; command (with the &quot;cubic&quot; option) perform bicubic interpolation?  I tried the Mathworks documentation online and I couldn't find the answer there.  I found a nice Wikipedia article at &#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Bicubic_interpolation&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Bicubic_interpolation&lt;/a&gt; ,&#xA;but it describes an algorithm that requires values of $f$, $f_x$, $f_y$, and $f_{xy}$, while &quot;interp2&quot; requires only function values, not derivatives.  Maybe Matlab estimates the partial derivatives somehow, but I don't know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My apologies if the answer actually is in the Mathworks documentation, but I looked pretty hard and I couldn't find it.   &lt;/p&gt;&#xA;" OwnerUserId="3821" LastEditorUserId="4244" LastEditDate="2013-09-23T12:15:44.503" LastActivityDate="2013-09-23T20:32:01.257" Title="How does Matlab's &quot;interp2&quot; compute bicubic interpolation?" Tags="|matlab|interpolation|documentation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8577" PostTypeId="1" AcceptedAnswerId="8578" CreationDate="2013-09-23T02:55:23.077" Score="9" ViewCount="1155" Body="&lt;p&gt;I have been trying to debug this error the last few days I wondered if anybody has advice on how to proceed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving the Poisson equation for a step charge distribution (a common problem in electrostatics/semiconductor physics) on a non-uniform finite volume mesh where the unknown are defined on cell centres and the fluxes on the cell faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;0 = (\phi_x)_x + \rho(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the charge profile (the source term) is given by,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    \rho(x)= &#xA;\begin{cases}&#xA;    -1,&amp;amp; \text{if } -1 \leq x \leq 0\\&#xA;     1,&amp;amp; \text{if } 0 \leq x \leq 1\\&#xA;    0,              &amp;amp; \text{otherwise}&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the boundary conditions are,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\phi(x_L)=0 \\&#xA;\frac{\partial\phi}{\partial x}\bigg|_{x_R}=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the domain is $[-10,10]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using code developed to solve the advection-diffusion-reaction equation (I have written myself see my notes here, &lt;a href=&quot;http://danieljfarrell.github.io/FVM&quot; rel=&quot;noreferrer&quot;&gt;http://danieljfarrell.github.io/FVM&lt;/a&gt;). The advection-diffusion-reaction equation is a more general case of the Poisson equation. Indeed the Poisson equation can be recovered by setting the advection velocity to zero and removing the transient term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code has been tested against a number of situations for uniform, nonuniform and random grids and always produces a reasonable solutions (&lt;a href=&quot;http://danieljfarrell.github.io/FVM/examples.html&quot; rel=&quot;noreferrer&quot;&gt;http://danieljfarrell.github.io/FVM/examples.html&lt;/a&gt;) for the advection-diffusion-reaction equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To show where the code breaks down I have made the following example. I setup a &lt;em&gt;uniform&lt;/em&gt; mesh of 20 cells and then make it &lt;em&gt;nonuniform&lt;/em&gt; by removing a single cell. In the left figure I have removed cell $\Omega_8$ and in the right $\Omega_9$ has been removed. The 9th cell covers the region where the source term (i.e. the charge) changes sign. The bug appears &lt;strong&gt;when the grid is nonuniform in a region where the reaction term changes sign&lt;/strong&gt;. As you can see below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas what could possibility be causing this issue? Let me know if more information regarding the discretisation would be helpful (I didn't want to pack too much detail into this question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OlSQC.png&quot; alt=&quot;Peculiar error when solving the Poisson equation&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-09-23T12:48:32.547" LastActivityDate="2013-09-24T14:12:51.953" Title="Peculiar error when solving the Poisson equation on a non-uniform mesh (1D only) finite volume method" Tags="|discretization|finite-volume|poisson|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="8578" PostTypeId="2" ParentId="8577" CreationDate="2013-09-23T05:57:01.677" Score="9" Body="&lt;p&gt;Just as an aside, your github documentation is fantastic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just a guess from DG methods, which can have similar issues if numerical fluxes aren't chosen carefully (I figure FV methods are a subset of DG methods). If you're using interpolation from cell centers to define your fluxes, then this should be equivalent to using the average as a numerical flux in DG and using piecewise constant basis. For standard DG methods for Poisson, this leads to numerically non-unique solutions - you can get a non-trivial null space for the discrete operator, which I think is what's causing your issues in the 2nd example.  See &lt;a href=&quot;http://www.imati.cnr.it/brezzi/papers/bcms.pdf&quot;&gt;this DG paper&lt;/a&gt; for their theory on it from the DG side. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll try to mock up an example for FV which shows how this comes into play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: so here's a small example of what's going on. Consider cells 1-9 and 11-20 in which $\rho(x) = 0$.  From the right side (11-20), we have $f(x_{20}) = 0$ due to the Neumann condition, which tells us from conservation for that cell that $f(x_{19}) = \ldots = f(x_{11}) = 0$. Since the flux is the average of cell values, this tells us that $\phi(x)$ is constant over all these cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the left side (1-9), we have $f(x_{i+1})-f(x_i) = 0$. If $f(-10) = 0$ and we use ghost cells, then $f(-10) = \phi_{9.5} - \phi_{\rm ghost} = \phi_{9.5}$. Conservation over the next few cells gives that $f(x_i) = f(-10) = \phi_{9.5}$ (i.e. constant slope). However, note that this can be any slope, just constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue comes about in the middle cell.  Like Jan mentioned, you undersample the forcing in the second mesh. This throws off the balance equations at that point, gives you an error in $f(10)$, which then propagates backwards and messes up both the slope in the left half of the domain as well as the value of $\phi(9.5)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sensitivity to errors in forcing is what's problematic - unlike FEM or FD methods which explicitly enforce the Dirchlet condition at $x=-10$, FV enforces it weakly using ghost nodes. Intuitively, ghost node weak imposition is like setting a Neumann condition at your left boundary as well. If you have two Neumann conditions for a diffusion problem, your problem is ill posed and has a nonunique solution (you can add any constant to that problem and still have a solution).  You don't quite get that at the discrete level here, but you do get very sensitive and mesh-dependent behavior as you see in your experiments. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2013-09-24T14:12:51.953" LastActivityDate="2013-09-24T14:12:51.953" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8579" PostTypeId="1" CreationDate="2013-09-23T07:35:25.807" Score="0" ViewCount="146" Body="&lt;p&gt;I am doing a numerical simulation of a elasticity problem. It is very simple. A cuboid elastic body with the right end fixed on the wall, under the gravity(but here I set it to be 1 along the z-axis always), what is the motion of this elastic body? The initial displacement and velocity are 0 everywhere, that is, the static case. I use time step 100s, and the displacement converges, and I got the picture of the displacement of the elastic body when it converges. The mesh has 1482 tets with average edge length 4.00. I use a post-processing tool gmsh and got the picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/GnUKO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not quite sure if this result is correct or not. Actually I am simulating an isotropic linear elastic body. Could anyone here give me some guidance about this, and if it is not correct, what the correct picture should looks like?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Re: Stefan M&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think my model is very clear, a cuboid isotropic linear elastic body with a fixed right end, and we want to know what is its motion when time evolves. I am using a new method, but anyone can use, for example, finite element method to simulate this and got the solution. Even the mesh are different, but the solution (if it converges) should be at least same in quality. That's why I am trying to ask this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I see I got something wrong, the solution I get now will not converge, this seems plausible.&lt;/p&gt;&#xA;" OwnerUserId="5181" LastEditorUserId="5223" LastEditDate="2013-09-24T18:11:52.727" LastActivityDate="2013-09-24T18:11:52.727" Title="Simulating the motion of a elastic body under gravity" Tags="|simulation|numerical-analysis|" AnswerCount="1" CommentCount="3" ClosedDate="2013-11-06T23:39:06.640" ContentLicense="CC BY-SA 3.0" />
  <row Id="8580" PostTypeId="1" AcceptedAnswerId="8582" CreationDate="2013-09-23T08:23:42.620" Score="5" ViewCount="474" Body="&lt;p&gt;I am looking for help in understanding the use of discrete Fourier transform as an approximation to continuous Fourier transforms.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an exercise, I considered a Gaussian $$f(x) = \exp\left(-2x^2\right),$$ the (continuous) Fourier transform of which is $$F(k)=\frac{1}{2}\exp\left(\frac{-k^2}{8}\right).$$  I sampled the Gaussian at 10,000 discrete points equally spaced between 0 and 100 and took the discrete Fourier transform of the resulting set of numbers. The question I wanted to ask was how well the discrete Fourier transform compare with the continuous one. Results of this calculation are shown in this figure (thick curve is the analytical, continuous, result; thin one is the numerical, discrete, result):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OJVCq.png&quot; alt=&quot;Comparison of discrete and continuous Fourier transforms of a Gaussian.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evidently, the discrete and Fourier transform are quite different from each other, especially at small $k$.  Could somebody help understand why? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: It was pointed out that I wasn’t really comparing apples to apples in the plots above. I had assumed different scalings the continuous and discrete cases. I corrected that, but the problem won’t go away. Here is the updated version of the Gaussian calculation above: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DqoJm.png&quot; alt=&quot;Better comparison of discrete and continuous Fourier transforms of a Gaussian.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check this on a bandwidth-limited function, I also tried working with the sinc function: $$f(x) = \frac{\sin(x)}{x}.$$ And I have the same problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/SkK5r.png&quot; alt=&quot;Comparison of discrete and continuous Fourier transforms of the sinc function.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I seem to be getting the exact same factor of difference in both cases, which suggests that I have missed some scaling factor. But I just cannot find one!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks! &lt;/p&gt;&#xA;" OwnerUserId="5213" LastEditorUserId="5213" LastEditDate="2013-09-24T08:29:09.670" LastActivityDate="2013-09-24T08:29:09.670" Title="When is discrete Fourier transform a good approximation to the continuous one?" Tags="|discretization|numerical-analysis|fourier-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8581" PostTypeId="2" ParentId="8579" CreationDate="2013-09-23T10:29:35.040" Score="1" Body="&lt;p&gt;I assume that your model is well-posed, i.e. there is only one solution to the problem, and there is only the gravity as an external force. If it converges -- I assume this means that the solution doesn't change in time any more -- then it converges to the steady-state solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The steady state solution of what you describe is known to be &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler%E2%80%93Bernoulli_beam_theory#Cantilever_beams&quot; rel=&quot;nofollow&quot;&gt;this, (last case with uniformly distributed load)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, your result is not correct.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-09-23T11:44:42.037" LastActivityDate="2013-09-23T11:44:42.037" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8582" PostTypeId="2" ParentId="8580" CreationDate="2013-09-23T12:06:17.617" Score="3" Body="&lt;p&gt;I haven't done the algebra, but I think you have the formula wrong. &lt;a href=&quot;http://mathworld.wolfram.com/FourierTransformGaussian.html&quot; rel=&quot;nofollow&quot;&gt;http://mathworld.wolfram.com/FourierTransformGaussian.html&lt;/a&gt;. There's some factors of $\pi$ missing from your answer. Which scaling of the Fourier Transform did you use?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-23T12:06:17.617" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8583" PostTypeId="2" ParentId="8577" CreationDate="2013-09-23T18:05:14.750" Score="0" Body="&lt;p&gt;The first thing to notice are your boundary conditions. Since you can change the slope and the value, you have neither Dirichlet, nor Neumann conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, every straight line is a solution where the right hand side is zero. You got that part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your fluxes are probably dependent on $h$. Do you use the correct $h$ where you eliminate a cell?&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-23T18:05:14.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8585" PostTypeId="2" ParentId="8576" CreationDate="2013-09-23T20:32:01.257" Score="5" Body="&lt;p&gt;Running the command &lt;code&gt;edit interp2&lt;/code&gt; allows you to see the source code of this particular function and then you can read the piece of code that deals with bicubic interpolation.  In MATLAB R2011, there is even a paper being cited: &quot;Cubic Convolution Interpolation for Digital Image Processing&quot;, Robert G. Keys, IEEE Trans. on Acoustics, Speech, and Signal Processing, Vol. 29, No. 6, Dec. 1981, pp. 1153-1160.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-09-23T20:32:01.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8587" PostTypeId="2" ParentId="8561" CreationDate="2013-09-23T23:59:35.000" Score="3" Body="&lt;p&gt;You certainly have a number of cursoriness mistakes in your derivation, but often only part of the mistake is propagated to later equations.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$V=mgly_2$ should be $V=mgy_2$.&lt;/li&gt;&#xA;&lt;li&gt;$L=\frac{m}{2}\left(2\dot x^2+\dot x l\cos(\phi)\dot\phi\right)-mgl\cos(\phi)$ should be $L=\frac{m}{2}\left(2\dot x^2+2\dot x l\cos(\phi)\dot\phi\right)-mgl\cos(\phi)$.&lt;/li&gt;&#xA;&lt;li&gt;$\pi_x=\frac{\partial L}{\partial\dot x}=2m\dot x+l\cos(\phi)\dot\phi$ should be  $\pi_x=\frac{\partial L}{\partial\dot x}=2m\dot x+ml\cos(\phi)\dot\phi$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The following equations are already tainted by these mistakes, so this should be enough for the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding DAEs and their index, the Lagrangian formulation for your problem is uncritical, and leads to a system of index $\leq 1$. So I can assure you that the main problem are the cursoriness mistakes all over the place.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2013-09-23T23:59:35.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8591" PostTypeId="1" CreationDate="2013-09-25T02:39:49.560" Score="9" ViewCount="1215" Body="&lt;p&gt;I am just wondering if Discrete Exterior Calculus, as a new numerical method , is good at numericall solving problems in elasticity, fluids or other physical/real area.&lt;/p&gt;&#xA;" OwnerUserId="5226" LastEditorUserId="4244" LastEditDate="2013-09-25T08:16:57.440" LastActivityDate="2016-11-24T13:11:32.390" Title="Is Discrete Exterior Calculus currently a focusing point in numerial computing world or simulation in industry," Tags="|simulation|numerical-analysis|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8592" PostTypeId="2" ParentId="8591" CreationDate="2013-09-25T02:59:49.680" Score="1" Body="&lt;p&gt;I would say there appears to be some interest, but it's not exploding. It's a bit too finite volume-y for my tastes, but I'm a finite elements person. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-25T02:59:49.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8593" PostTypeId="1" CreationDate="2013-09-25T06:46:31.253" Score="1" ViewCount="155" Body="&lt;p&gt;What exactly is &lt;a href=&quot;http://www.geforce.com/hardware/technology/gpu-boost/technology&quot; rel=&quot;nofollow&quot;&gt;GPU Boost&lt;/a&gt; and how it helps? What is the improvement in GPU Boost 2.0? I did not understand much reading from the link. Can somebody explain the main concept and details?   &lt;/p&gt;&#xA;" OwnerUserId="4932" LastActivityDate="2013-09-25T13:16:09.147" Title="What is GPU Boost in NVIDIA GPU" Tags="|gpu|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8594" PostTypeId="1" AcceptedAnswerId="8619" CreationDate="2013-09-25T09:39:19.780" Score="1" ViewCount="1270" Body="&lt;p&gt;I've been using LAPACK &lt;code&gt;dgeev&lt;/code&gt; in FORTRAN in the last months spending hours to diagonalize ~4000*4000 matrices. It takes about 2'75 hours to find eigenvalues and eigenvectors this way. I thought FORTRAN should be faster than Mathematica as it usually is for this kind of tasks. After all this hard work and coding effort I have just realized Mathematica does the same in &amp;lt;15 minutes. What is happening?&lt;/p&gt;&#xA;" OwnerUserId="5089" LastActivityDate="2013-09-26T20:27:47.160" Title="Eigenvectors: Mathematica vs. LAPACK dgeev" Tags="|fortran|lapack|eigenvalues|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8595" PostTypeId="1" AcceptedAnswerId="8600" CreationDate="2013-09-25T11:08:32.200" Score="6" ViewCount="2472" Body="&lt;h2&gt;Problem statement&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am trying to simulate a &lt;a href=&quot;http://www.maths.surrey.ac.uk/explore/michaelspages/documentation/Spherical.pdf&quot; rel=&quot;nofollow&quot;&gt;spherical pendulum&lt;/a&gt;, with&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;rod length $r$&lt;/li&gt;&#xA;&lt;li&gt;south-polar angle $\theta$ and azimuthal angle $\phi$ &lt;/li&gt;&#xA;&lt;li&gt;initial values $(\theta_0,\phi_0)= (0,0)$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My particular pendulum is hanging in a construction, which can be moved around by external forces (say, an experimenter). This construction has well-defined coordinate axes, which therefore also serve as the coordinate axes for the pendulum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Pendulum movement takes place in a non-inertial reference frame, with:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;non-zero angular velocity $\mathbf{\Omega}$ and angular acceleration $\mathbf{\dot{\Omega}}$&lt;/li&gt;&#xA;&lt;li&gt;non-zero linear velocity $\mathbf{v}_{\text{rel}}$ and linear acceleration $\mathbf{a}_{\text{rel}}$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Because of the nature of my problem, the gravitational acceleration $\mathbf{g}$ is &lt;em&gt;always&lt;/em&gt; pointing in the negative $z$-axis of the pendulum frame. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am experiencing severe numerical instabilities when integrating the equations of motion, when I &lt;em&gt;include&lt;/em&gt; the non-inertial forces &lt;em&gt;and&lt;/em&gt; my initial values are such that $|\theta| \lesssim 1^\circ$. The type of the numerical integrator just seems to have an effect on the moment of divergence, but divergence is always there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strangely enough, I can integrate everything perfectly fine for any set of initial values, when I set $\mathbf{\Omega}=\dot{\mathbf{\Omega}}=\mathbf{v}_{\text{rel}}=\mathbf{a}_{\text{rel}}=\mathbf{0}$.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Mathematical description&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The equations of motion for a &quot;normal&quot; spherical pendulum are &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\ddot{\theta} &amp;amp;= \left(\dot{\phi}^2\cos\theta - g/r \right)\sin\theta\\&#xA;\ddot{\phi}   &amp;amp;= \frac{-2\dot\theta\dot\phi\cos\theta}{\sin\theta}&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I manually set $\ddot\phi=0$ when non-finite values are encountered (precaution for division-by-zero when $\theta=0$), I have no problem whatsoever to numerically integrate these equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The total force on this particular pendulum is an addition of the pendulum weight and non-inertial forces: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{F}_{\text{tot}} = m\left( \mathbf{g} - \dot{\mathbf{\Omega}}\times\mathbf{r} - \mathbf{\Omega}\times\left(\mathbf{\Omega}\times\mathbf{r} \right) - 2\mathbf{\Omega}\times\mathbf{v}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which are the gravitational force, Euler force, centripetal force and Coriolis force, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These cross products are much easier to do in Cartesian coordinates. Therefore, in my software implementation of the differential equation, I convert $r$, $\theta$ and $\phi$ to Cartesian coordinates $x$, $y$, $z$, carry out the cross products, add the forces to the final acceleration, and convert back to the angular accelerations in spherical coordinates $\ddot\theta$ and $\ddot\phi$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equations to convert to spherical coordinates (using $\dot{r} = \ddot{r} = 0$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;\begin{align}&#xA;    \mathbf{r} &amp;amp;= r\mathbf{e}_r \\&#xA;    \mathbf{v} &amp;amp;= r\dot{\phi}\sin\theta \mathbf{e}_\phi + r\dot{\theta}\mathbf{e}_\theta&#xA;\end{align}        &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the unit vectors of the spherical coordinates have been used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;    \mathbf{e}_{r}      &amp;amp;= \mathbf{r}/|\mathbf{r}|  \\ &#xA;    \mathbf{e}_{\theta} &amp;amp;= \left[ \begin{matrix}\cos\theta\cos\phi\\ \cos\theta\sin\phi\\ \sin\theta\end{matrix} \right]\\&#xA;    \mathbf{e}_{\phi}   &amp;amp;= \left[ \begin{matrix}-\sin\phi\\  \cos\phi\\ 0\end{matrix} \right]    &#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as well as their time derivatives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}  &#xA;    \dot{\mathbf{e}_{r}}      &amp;amp;=  \dot{\theta}\mathbf{e}_{\theta} +  \dot\phi\sin\theta\mathbf{e}_{\phi} \\&#xA;    \dot{\mathbf{e}_{\theta}} &amp;amp;= -\dot\theta\mathbf{e}_{r}  + \dot\phi\cos\theta\mathbf{e}_{\phi} \\&#xA;    \dot{\mathbf{e}_{\phi}}   &amp;amp;= -\dot\phi\sin\theta\mathbf{e}_{r}  -  \dot\phi\cos\theta\mathbf{e}_{\theta}    &#xA;\end{align}    &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deriving the velocity $\mathbf{v}$ once more gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;    \mathbf{a} = &amp;amp;\left(-r\left(\dot\theta^2 + \dot\phi^2\sin^2\theta\right) \right) \mathbf{e}_{r} + \\&#xA;        &amp;amp;\left( r\left(\ddot\theta - \dot\phi^2\sin\theta\cos\theta\right) \right) \mathbf{e}_{\theta} + \\&#xA;        &amp;amp;\left( r\left(2\dot\phi\dot\theta\cos\theta + \ddot\phi\sin\theta\right) \right) \mathbf{e}_{\phi}&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;    \ddot\theta &amp;amp;= \frac{\mathbf{a_{\text{tot}}}\cdot\mathbf{e}_{\theta} + r\dot\phi^2\sin\theta\cos\theta}{r}\\&#xA;    \ddot\phi   &amp;amp;= \frac{\mathbf{a_{\text{tot}}}\cdot\mathbf{e}_{\phi}   - 2r\dot\phi\dot\theta\cos\theta }{r\sin\theta}&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{a_{\text{tot}}}$ is the acceleration of the pendulum mass, including gravity &lt;em&gt;and&lt;/em&gt; all the non-inertial forces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am pretty sure these equations are correct, as I have made many assertions, plots, etc. for each and every step of the way. I can also simulate a &quot;regular&quot; spherical pendulum, as well as one in a rotating/accelerating frame, and all the outcomes are very realistic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, all just for initial values well outside the south-polar region :) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some strange reason, it is numerically OK to have extremely large values for $\ddot\phi$ when there is only gravity in $\mathbf{a}_{\text{tot}}$, but not if there are additional forces in there...&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What I have tried&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Usually I try to avoid spherical coordinates like the plague for precisely this sort of reason, but I fear in the case of a spherical pendulum there is no real way around it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the problem is not physical (there is nothing &quot;physically special&quot; about the direct-down position of the pendulums) but just due to the choice of the coordinate system, I thought I'd try to dump the spherical coordinates and integrate everything directly in Cartesian coordinates. This resulted in a much simpler formulation, and much more elegant code, but there was another problem: I could not keep the length of the rod constant :) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When integrating the pendulum like this, you have to include these constraint equations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;length of the rod is constant&lt;/li&gt;&#xA;&lt;li&gt;velocity is perpendicular to the rod&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;as well as calculate additional terms, like the centripetal acceleration of the pendulum mass towards the hinge point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there are ways to do that, but these usually place demands on the type of integrator used to solve the problem. In my final application, I can only use one of a small subset of pre-defined integrators, so keeping the constraints satisfied is virtually impossible. For any of the integrators I can work with, roundoff error quickly accumulates and starts violating the length constraints, leading to divergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing I tried (although I didn't like it much) is to define two &quot;zones of avoidance&quot; at the poles. That is, if the angle $\theta$ is within a small region around the pole, the value for $\ddot\phi$ is defined to be zero. The problem with this approach is: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how big should the region be in order to &lt;strong&gt;guarantee&lt;/strong&gt; stability?&lt;/li&gt;&#xA;&lt;li&gt;it leads to non-physical behavior&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That is: while it does seem to be more stable this way, there are &quot;strange discontinuities&quot; visible in the solutions. Moreover, the results completely disagree with  results obtained without this &quot;zone of avoidance&quot;...&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What I am looking for&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;My final application will need to be space-graded. That means a lot of things, but the most important ones are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it has to produce meaningful results under &lt;em&gt;all&lt;/em&gt; circumstances&lt;/li&gt;&#xA;&lt;li&gt;it has to still produce meaningful results after 15 days of continuous operation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Therefore, I am looking for ways to &quot;get rid&quot; of this singularity at the poles somehow, preferably preserving physical correctness :) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also looking for a good explanation of &lt;em&gt;why&lt;/em&gt; this occurs, as I have been staring at it for three days now and I am totally blinded by my own derivations :)  &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Minimum working example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A (not-so-)minimum working example in MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function MWE&#xA;&#xA;    %% Pendulum parameters&#xA;&#xA;    clc&#xA;&#xA;    g = [0; 0; -9.8];&#xA;    L = 2;&#xA;&#xA;    tspan = [0 50];&#xA;&#xA;    %% Plot Omega forcing term&#xA;&#xA;    tt = 0:0.1:25;&#xA;    Omega = zeros(numel(tt),3);&#xA;    for ii = 1:numel(tt)&#xA;        Omega(ii,:) = Omega_fcn(tt(ii)); end&#xA;    plot(...&#xA;        tt, Omega(:,1), 'r',...&#xA;        tt, Omega(:,2), 'g',...&#xA;        tt, Omega(:,3), 'b');&#xA;    xlabel('t'), ylabel('Angular rate')&#xA;    t = title('$\mathbf{\Omega}(t)$');&#xA;    set(t, 'interpreter', 'LaTeX');&#xA;    l = legend('$\mathbf{\Omega}_x$', '$\mathbf{\Omega}_y$', '$\mathbf{\Omega}_z$');&#xA;    set(l, 'interpreter', 'LaTeX');&#xA;&#xA;&#xA;    %% Normal spherical pendulum, &quot;good&quot; initial values&#xA;&#xA;    Omega     = zeros(3,1);&#xA;    Omega_dot = zeros(3,1);&#xA;&#xA;    y0 = [pi/4 0 0 0.4];&#xA;    f  = @(t,y)equationsOfMotion(y, L,g,Omega,Omega_dot);&#xA;&#xA;    [~, y] = ode113(f, tspan, y0);&#xA;&#xA;    theta = y(:,1);&#xA;    phi   = y(:,2);&#xA;&#xA;    X = +L*cos(phi).*sin(theta);&#xA;    Y = +L*sin(phi).*sin(theta);&#xA;    Z = -L*cos(theta);&#xA;&#xA;    figure(2)&#xA;    plot3(X,Y,Z, 'r')&#xA;    axis tight equal&#xA;    t = title('$\mathbf{\Omega} = \dot{\mathbf{\Omega}} = \mathbf{0}$, &quot;good&quot; initial values');&#xA;    set(t, 'interpreter', 'LaTeX');&#xA;&#xA;&#xA;    %% Spherical pendulum in rotating frame, &quot;good&quot; initial values&#xA;&#xA;&#xA;    Omega_dot = zeros(3,1);&#xA;&#xA;    y0 = [pi/4 0 0 0.4];&#xA;    f  = @(t,y)equationsOfMotion(y, L,g,Omega_fcn(t),Omega_dot);&#xA;&#xA;    [~, y] = ode113(f, tspan, y0);&#xA;&#xA;    theta = y(:,1);&#xA;    phi   = y(:,2);&#xA;&#xA;    X = +L*cos(phi).*sin(theta);&#xA;    Y = +L*sin(phi).*sin(theta);&#xA;    Z = -L*cos(theta);&#xA;&#xA;    figure(3)&#xA;    plot3(X,Y,Z, 'r')&#xA;    axis tight equal&#xA;    t = title('$\mathbf{\Omega} = \mathbf{\Omega}(t),\ \dot{\mathbf{\Omega}} = \mathbf{0}$, &quot;good&quot; initial values');&#xA;    set(t, 'interpreter', 'LaTeX');&#xA;&#xA;&#xA;    %% Spherical pendulum in rotating frame, &quot;BAD&quot; initial values&#xA;&#xA;    Omega_dot = zeros(3,1);&#xA;&#xA;    y0 = [0 0 0 0];&#xA;    f  = @(t,y)equationsOfMotion(y, L,g,Omega_fcn(t),Omega_dot);&#xA;&#xA;    % FAILURE!&#xA;    [~, y] = ode113(f, tspan, y0);&#xA;    % FAILURE!    &#xA;&#xA;    theta = y(:,1);&#xA;    phi   = y(:,2);&#xA;&#xA;    X = L*cos(phi).*sin(theta);&#xA;    Y = L*sin(phi).*sin(theta);&#xA;    Z = -L*cos(theta);&#xA;&#xA;    figure(4)&#xA;    plot3(X,Y,Z, 'r')&#xA;    axis tight equal&#xA;    t = title('$\mathbf{\Omega} = \mathbf{\Omega}(t),\ \dot{\mathbf{\Omega}} = \mathbf{0}$, &quot;BAD&quot; initial values');&#xA;    set(t, 'interpreter', 'LaTeX');&#xA;&#xA;end&#xA;&#xA;% Forcing function&#xA;function omega = Omega_fcn(t)&#xA;&#xA;    omega = [0; 0; 0];&#xA;&#xA;    t1Start = 1;&#xA;    t2Start = 10;&#xA;    Duration  = 1;&#xA;&#xA;    if (t &amp;gt;= t1Start) &amp;amp;&amp;amp; (t &amp;lt; t1Start + Duration)&#xA;        omega = [&#xA;            +0.0175&#xA;            -3.2e-4&#xA;            -3.35e-4]*(t-t1Start);&#xA;    end&#xA;&#xA;    if (t &amp;gt;= t1Start + Duration) &amp;amp;&amp;amp; (t &amp;lt; t2Start)&#xA;        omega = [&#xA;            +0.0175&#xA;            -3.20e-4 + 7.500e-006*(t-t1Start-Duration)&#xA;            -3.35e-4 - 8.125e-006*(t-t1Start-Duration) ];&#xA;    end&#xA;&#xA;    if (t &amp;gt;= t2Start) &amp;amp;&amp;amp; (t &amp;lt; t2Start + Duration)&#xA;        omega= [...&#xA;            +0.0175 - 0.01750*(t-t2Start)&#xA;            -2.6e-4 + 3.10e-4*(t-t2Start)&#xA;            -4e-4   + 3.75e-4*(t-t2Start) ];&#xA;    end&#xA;&#xA;    if (t &amp;gt;= t2Start + Duration)&#xA;        omega= [...&#xA;            0&#xA;            0.5e-004&#xA;            -0.25e-004];&#xA;    end&#xA;&#xA;end&#xA;&#xA;function dy = equationsOfMotion(y, L, g, Omega, Omega_dot)&#xA;&#xA;    % Extract spherical angles and angular rates&#xA;    theta = y(1);    theta_dot = y(3);&#xA;    phi   = y(2);    phi_dot   = y(4);&#xA;&#xA;    % pre-compute sines/cosines&#xA;    sT = sin(theta);   sP = sin(phi);&#xA;    cT = cos(theta);   cP = cos(phi);&#xA;&#xA;    % Unit vectors of the local spherical coordinates&#xA;    eTheta = [cT.*cP; cT.*sP; sT];&#xA;    ePhi   = [-sP;  cP;  0];&#xA;    eR     = [sT*cP;  sT*sP;  -cT];&#xA;    % ASSERT:&#xA;    % eR = r/L;&#xA;    % |eR| = |ePhi| = |eTheta| = 1&#xA;    % cross(eR,     ePhi)   = eTheta&#xA;    % cross(ePhi,   eTheta) = eR       (orthonormal basis)&#xA;    % cross(eTheta, eR)     = ePhi&#xA;&#xA;    % Pendulum vector: vector to bob mass, with hinge point as origin&#xA;    r = L*eR;&#xA;&#xA;    % Time derivatives of unit vectors&#xA;    % eR_dot     =       0         +  theta_dot *eTheta  +  phi_dot*sT*ePhi;&#xA;    % eTheta_dot = -theta_dot *eR  +         0           +  phi_dot*cT*ePhi;&#xA;    % ePhi_dot   = -phi_dot*sT*eR  -  phi_dot*cT*eTheta  +         0       ;&#xA;    %&#xA;    % ASSERT:&#xA;    %&#xA;    % deRdth = [cT*cP; cT*sP; sT];&#xA;    % deRdph = [-sT*sP; sT*cP; 0];&#xA;    % deRdth*theta_dot + deRdph*phi_dot % Should equal eR_dot&#xA;    %&#xA;    % dePhidph = [-cP; -sP; 0];&#xA;    % dePhidph*phi_dot % Should equal ePhi_dot&#xA;    %&#xA;    % deThetadth = [-sT*cP;  -sT*sP;  cT];&#xA;    % deThetadph = [-cT*sP;  cT*cP;  0];&#xA;    % deThetadth*theta_dot + deThetadph*phi_dot % Should equal eTheta_dot&#xA;&#xA;    % Velocities&#xA;    v = L*phi_dot*sT*ePhi + L*theta_dot*eTheta;&#xA;    % ASSERT: v must equal:&#xA;    %  v = R*eR_dot&#xA;&#xA;    % Pendulum weight&#xA;    arel = g;&#xA;&#xA;    % d(Omega)/dt × r&#xA;    % The Euler force&#xA;    Euler = cross(Omega_dot, r);&#xA;&#xA;    % 2·(Omega × v)&#xA;    % The Coriolis force&#xA;    Coriolis = 2*cross(Omega, v);&#xA;&#xA;    % Omega × (Omega × r)&#xA;    % The local centripetal force&#xA;    Centripetal = cross(Omega, cross(Omega, r));&#xA;&#xA;    % Ficticious forces due to non-intertialness of the frame:&#xA;    a = arel - Centripetal - Coriolis - Euler;&#xA;&#xA;    % All components parallel to the rod are absorbed by the tension/compression of&#xA;    % the rod. Moreover, they are irrelevant for theta_dot_dot and phi_dot_dot.&#xA;    % Thus, just to be sure, subtract those components:&#xA;    a = a - dot(a,eR)*eR;&#xA;&#xA;    % Now convert the acceleration to spherical coordinates&#xA;    theta_dot_dot = ( dot(a,eTheta) + L*phi_dot^2*sT*cT )/L;&#xA;    phi_dot_dot   = ( dot(a,ePhi)   - 2*L*phi_dot*theta_dot*cT ) / (L*sT);&#xA;&#xA;    % division by zero&#xA;    if ~isfinite(phi_dot_dot)&#xA;        phi_dot_dot = 0; end    &#xA;&#xA;    % Thus, the derivative of the state vector:&#xA;    dy = [&#xA;        theta_dot&#xA;        phi_dot&#xA;        theta_dot_dot&#xA;        phi_dot_dot&#xA;    ];&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2132" LastEditorUserId="2132" LastEditDate="2013-09-26T08:27:29.047" LastActivityDate="2013-09-30T19:05:49.980" Title="Numerical instability of spherical pendulum" Tags="|stability|time-integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8596" PostTypeId="1" AcceptedAnswerId="8604" CreationDate="2013-09-25T11:44:43.363" Score="2" ViewCount="449" Body="&lt;p&gt;Say we have a discretised a coupled nonlinear system of two PDEs to give a system of ODEs which approximates the original system,&#xA;$$&#xA; \frac{\partial u}{\partial t} = F_1(t,\boldsymbol{u,v}) \\&#xA; \frac{\partial v}{\partial t} = F_2(t,\boldsymbol{u,v}) \\&#xA;$$&#xA;where $\boldsymbol{u} = [u_1 \cdots u_N]^{T}$ and $\boldsymbol{v} = [v_1 \cdots v_N]^{T}$ are a vector of solution variables with the same number of elements as discretised points in the system. $F$ is a vector which depends on how the PDEs has been discretised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When PDEs are left in this semi-discrete form (i.e. the transient term is not discretised) they can be solved using conventional ODE solvers, this technique is known as the &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines&quot; rel=&quot;nofollow&quot;&gt;Method of Lines&lt;/a&gt; (MOL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally the ODE solver can be used to step the system forward in time until a desired time point or until the system reaches steady-state. However, can the MOL technique be used to solve &lt;em&gt;directly&lt;/em&gt; for the steady-state value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Internally the ODE solver will be computing a Jacobian and using Newton's method to solve the system, these are the prerequisites for solving the system directly for steady state. So it would seem that it has all the necessary information to do so. Moreover, to solve the above system for the steady state one would apply Newton's method but would set the transient term to zero,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA; 0 = F_1(t,\boldsymbol{u,v}) \\&#xA; 0 = F_2(t,\boldsymbol{u,v}) \\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this with MOL solvers (e.g. in MATLAB or Python)?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-09-25T13:31:07.713" LastActivityDate="2013-09-25T15:12:32.320" Title="Can the method of lines technique be used to solve a ODE directly for the steady-state value?" Tags="|pde|ode|method-of-lines|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8597" PostTypeId="1" AcceptedAnswerId="8601" CreationDate="2013-09-25T11:47:16.400" Score="1" ViewCount="130" Body="&lt;p&gt;I have a non-linear equation that converges, and reaches suitable accuracy after around 20 steps, however each step is very expensive to calculate. The series are never quite the same, but they are similar; there are 4 examples below. Ideally, I would like to be able to guess the final answer (getting as close as possible), knowing only the first few (say 5?) steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example series:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;22.1365571473&#xA;39.1003909811&#xA;60.8702429029&#xA;87.3193552323&#xA;117.479117043&#xA;149.298938154&#xA;180.099415362&#xA;207.478769226&#xA;230.033683334&#xA;247.487707416&#xA;260.35893824&#xA;269.520601585&#xA;275.88012284&#xA;280.218467629&#xA;283.143170475&#xA;285.099206545&#xA;286.400448741&#xA;287.263033108&#xA;287.833493354&lt;/p&gt;&#xA;&#xA;&lt;p&gt;43.8168649647&#xA;119.274730769&#xA;228.310938956&#xA;332.800659827&#xA;406.364683614&#xA;449.036198231&#xA;471.258177801&#xA;482.206944282&#xA;487.456776422&#xA;489.94148426&#xA;491.110264146&#xA;491.658457589&lt;/p&gt;&#xA;&#xA;&lt;p&gt;13.7709904649&#xA;17.4517204683&#xA;20.862400148&#xA;23.9136500982&#xA;26.5768107479&#xA;28.8599125041&#xA;30.7910913202&#xA;32.4079080805&#xA;33.7507827028&#xA;34.8591665245&#xA;35.7694839095&#xA;36.5141882847&#xA;37.1214961348&#xA;37.6155120979&#xA;38.016560732&#xA;38.3416106934&#xA;38.6047214436&#xA;38.8174732756&#xA;38.9893605432&#xA;39.1281383997&#xA;39.2401237101&#xA;39.3304495994&#xA;39.4032795394&lt;/p&gt;&#xA;&#xA;&lt;p&gt;19.5526511118&#xA;31.7547501442&#xA;45.9545707135&#xA;61.7006795681&#xA;78.515831743&#xA;95.8079163675&#xA;112.898129782&#xA;129.114767641&#xA;143.90065881&#xA;156.888846484&#xA;167.923989061&#xA;177.034730079&#xA;184.379068592&#xA;190.185787294&#xA;194.706624478&#xA;198.184227677&#xA;200.8346208&#xA;202.840305035&#xA;204.349975449&#xA;205.481710536&#xA;206.327550943&#xA;206.958285211&#xA;207.427820219&#xA;207.776913165&lt;/p&gt;&#xA;" OwnerUserId="5228" LastEditorUserId="5228" LastEditDate="2013-09-25T12:19:28.303" LastActivityDate="2013-09-26T08:45:36.927" Title="Guess the final term of a converging series" Tags="|convergence|" AnswerCount="3" CommentCount="5" ClosedDate="2013-10-08T04:05:02.007" ContentLicense="CC BY-SA 3.0" />
  <row Id="8598" PostTypeId="2" ParentId="8594" CreationDate="2013-09-25T12:14:22.260" Score="4" Body="&lt;p&gt;The LAPACK routine dgeev in turn calls routines from the BLAS library to perform more basic linear algebra such as matrix-matrix multiplications.  It's likely that the slow performance of dgeev on your problem is caused by using a BLAS library that isn't highly optimized.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know which implementation of BLAS you're using with your Fortran code?  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-09-25T12:14:22.260" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8599" PostTypeId="2" ParentId="8593" CreationDate="2013-09-25T13:16:09.147" Score="3" Body="&lt;p&gt;If I read correctly, GPU Boost 2.0 ramps up the clock speed of the GPU cores until a target temperature is reached. It sounds like a closed-loop controller than runs the GPU as hot as possible but no hotter. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-25T13:16:09.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8600" PostTypeId="2" ParentId="8595" CreationDate="2013-09-25T13:17:16.823" Score="4" Body="&lt;p&gt;Interesting problem. But this will be only a hint to a possible solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not so much in the details, but I assume, you can write your system in the coordinates of the endpoint of the pendulum $p=[p_1,p_2,p_3]$ and velocities $v=[v_1,v_2,v_3]$ in cartesian coordinates subject to the system of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot p &amp;amp;= v + f_1 \\&#xA;\dot v + g_p(p)^T\lambda &amp;amp;= f_2, \\&#xA;g(p) := p_1^2 + p_2^2 + p_3^2 - r^2 &amp;amp;= 0&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f_*$ contain all external forces, $\lambda$ is a multiplier, and $r$ is the length of the rod. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a &lt;a href=&quot;http://www.scholarpedia.org/article/Differential-algebraic_equations&quot; rel=&quot;nofollow&quot;&gt;DAE&lt;/a&gt; and reduces to a multibody system in an inertial coordinate system. Standard integration will inevitably lead to deviations from the constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For multibody system there is the approach of &lt;em&gt;minimal extension&lt;/em&gt;, that reduces the &lt;em&gt;index&lt;/em&gt; of the DAE. From a practical point of view, this means finding an equivalent reformulation of the system, for which standard integrators, e.g. BDF methods work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;http://www3.math.tu-berlin.de/ePrints/papers/pdf/KunM04a.pdf&quot; rel=&quot;nofollow&quot;&gt;paper by Kunkel&amp;amp;Mehrmann&lt;/a&gt;, in which minimal extension was introduced, there is a concrete example of how to simulate the movement induced by gravity of a masspoint inside a paraboloid (Eqn. 62). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is quite similar to what you are trying, except from your additional difficulties that come with the &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;accelerated frame and&lt;/li&gt;&#xA;&lt;li&gt;that the paraboloid is easier to parametrize than the sphere .&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The authors of the mentioned paper still use a specialized DAE solver. However, if you are fine with lower accuracy - Backward Euler should work well for the reformulated system, as we have shown for semi-discretized Euler equations in a recent &lt;a href=&quot;http://www3.math.tu-berlin.de/preprints/files/Preprint-11-2013.pdf&quot; rel=&quot;nofollow&quot;&gt;preprint&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: Volker Mehrmann is the supervisor of my PhD thesis.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2013-09-25T14:57:30.687" LastActivityDate="2013-09-25T14:57:30.687" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8601" PostTypeId="2" ParentId="8597" CreationDate="2013-09-25T13:52:29.567" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Shanks_transformation&quot; rel=&quot;nofollow&quot;&gt;Shanks Transform&lt;/a&gt; is a good way to improve the convergence of a series. There are some other &lt;a href=&quot;http://en.wikipedia.org/wiki/Series_acceleration&quot; rel=&quot;nofollow&quot;&gt;series acceleration&lt;/a&gt; methods described on Wikipedia too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387989315&quot; rel=&quot;nofollow&quot;&gt;Bender and Orzsag's book&lt;/a&gt; also has an entire chapter on summation of series (although I don't have it handy now).&lt;/p&gt;&#xA;" OwnerUserId="12" LastActivityDate="2013-09-25T13:52:29.567" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8602" PostTypeId="1" CreationDate="2013-09-25T14:30:28.367" Score="2" ViewCount="112" Body="&lt;p&gt;What architecture are current computers based on?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Princeton_architecture&quot; rel=&quot;nofollow&quot;&gt;Princeton Architecture&lt;/a&gt; or the &lt;a href=&quot;http://en.wikipedia.org/wiki/Harvard_architecture&quot; rel=&quot;nofollow&quot;&gt;Harvard Architecture&lt;/a&gt;?  Some notes I found online state the Princeton architecture, but this creates the Princeton bottleneck.  The Harvard architecture was suppose to alleviate this.  Have all systems transferred to a more &quot;Harvard&quot; approach?&lt;/p&gt;&#xA;" OwnerUserId="5231" LastEditorUserId="414" LastEditDate="2013-09-25T15:30:01.210" LastActivityDate="2013-09-25T20:03:48.877" Title="What architecture are current computers based on?" Tags="|architecture|" AnswerCount="1" CommentCount="0" ClosedDate="2013-09-29T00:40:35.777" ContentLicense="CC BY-SA 3.0" />
  <row Id="8603" PostTypeId="1" CreationDate="2013-09-23T13:45:42.140" Score="3" ViewCount="109" Body="&lt;p&gt;I have a function defined by three parameters (or variables): $f(a,b,c)$ This function is not explicitly defined but is actually a piece of code which returns the result of fitting a curve $g(a,b,c)$ to a set of $(x,y)$ points in a 2D space: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;O = {$(x_1,y_1,w_1), (x_2,y_2,w_2), ...$}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $w$ are &lt;em&gt;weights&lt;/em&gt; associated with each point. The smaller $f$, the better the fit of $g$ with those points. Each one of these points has a weight associated to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of these three parameters can only assume values from a given discrete set:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A = {$a_1, a_2, a_3, ..., N_a$}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B = {$b_1, b_2, b_3, ..., N_b$}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C = {$c_1, c_2, c_3, ..., N_c$}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every set $(a_i,b_j,c_k)$ implies a value for $f(a_i,b_j,c_k)$ and as I said, I need to find the $(a_i,b_j,c_k)$ set that returns the &lt;strong&gt;minimum&lt;/strong&gt; value for $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I do to evaluate $f$ is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take a set of $(a_i,b_j,c_k)$ values and construct a $g_{ijk} (a_i,b_j,c_k)$ curve in 2D space.&lt;/li&gt;&#xA;&lt;li&gt;For each $(x,y)$ point in $O$, find the point in $g_{ijk}$ that is closest to it (euclidean distance)&lt;/li&gt;&#xA;&lt;li&gt;Multiply that distance by the weight ($0&amp;lt;w&amp;lt;1$) associated to that $(x,y)$ point. This gives me a &lt;em&gt;weighted distance&lt;/em&gt; for each point.&lt;/li&gt;&#xA;&lt;li&gt;After processing all points in the $O$ set, sum all weighted distances.&lt;/li&gt;&#xA;&lt;li&gt;The sum obtained above is what I call $f(a_i,b_j,c_k)$, ie: a measure of how good $g_{ijk} (a_i,b_j,c_k)$ fitted $O$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So far I've managed to do this through a &lt;em&gt;brute force&lt;/em&gt; approach where I test every possible combination of $a,b,c$ (ie: $N_a*N_b*N_c$ combinations) one after another and at the end I simply isolate the minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better/more efficient algorithm to perform this &lt;em&gt;minimization&lt;/em&gt;, given the restrictions on my function and parameters?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Add&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;After a bit of research it looks like this falls into the category of &lt;em&gt;combinatorial optimization problems&lt;/em&gt; for which methods such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-entropy_method&quot; rel=&quot;nofollow&quot;&gt;Cross-Entropy&lt;/a&gt; seem to be the standard approach.&lt;/p&gt;&#xA;" OwnerUserId="5222" OwnerDisplayName="Gabriel" LastEditorUserId="5222" LastEditDate="2013-09-25T22:09:24.940" LastActivityDate="2013-09-25T22:09:24.940" Title="Find a function's minimizing set of parameters" Tags="|algorithms|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="8604" PostTypeId="2" ParentId="8596" CreationDate="2013-09-25T15:12:32.320" Score="5" Body="&lt;p&gt;A timestepping method can be considered always as an iterative solver for the steady state problem if the solution is guaranteed to run into a steady state. There are several aspects here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If the timestepping scheme is implicit and sufficiently stable, you can increase the time steps when you approach the limit. This is often referred to as pseudo timestepping and can be considered a stabilization of the Newton method you use to solve the nonlinear system. This might be necessary if several steady states exist and you want to stay close to the initial value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the timestepping scheme is explicit, it amounts to a defect correction scheme of the form&#xA;\begin{gather}&#xA;x^{(k+1)} = x^{(k)} - \omega \mathcal F(x^{(k)}),&#xA;\end{gather}&#xA;where the damping parameter $\omega$ coresponds to the timestep. This might work nicely, if the problem is not stiff. For stiff problems, the time step will be very small and you will need excruciatingly many iterations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In both cases, there are more efficient methods, depending on the actual system you solve. In particular, if a Newton method with globalization converges to the desired solution, it will be faster by orders of magnitude.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-25T15:12:32.320" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8605" PostTypeId="2" ParentId="8597" CreationDate="2013-09-25T16:53:44.160" Score="1" Body="&lt;p&gt;Given by how you state the problem, I take it accuracy is not a really important issue here :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your data sets: the fist-order differences ($a_2-a_1, a_3-a_2, ...$) look like they could be part of a &lt;a href=&quot;http://en.wikipedia.org/wiki/Normal_distribution&quot; rel=&quot;nofollow&quot;&gt;Gaussian distribution&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you could detect whether a peak occurs in the first-order differences, and stop evaluating your generator function when it does. Fit a Gaussian to the data points you have thus far to obtain $\mu$ and $\sigma$, and extrapolate to infinity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result should be close to the value of the Gaussian CDF for those values of $\mu$ and $\sigma$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It'll be a nice ballpark estimate, so if that suffices for the rest of your application, it should be OK. But please don't expect this to be anywhere near accurate -- fitting a Gaussian accurately through 5 data points is, well...you'll have to be extremely lucky :)&lt;/p&gt;&#xA;" OwnerUserId="2132" LastEditorUserId="2132" LastEditDate="2013-09-26T08:45:36.927" LastActivityDate="2013-09-26T08:45:36.927" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8606" PostTypeId="1" CreationDate="2013-09-25T15:48:54.327" Score="2" ViewCount="1246" Body="&lt;p&gt;I would like to do a numerical quantum dynamics of a displaced gaussian in harmonic oscillator using split-operator method (see bottom of &lt;a href=&quot;http://hep.physics.indiana.edu/~hgevans/p411-p610/material/08_pde_iv/schrodinger.html#fft&quot; rel=&quot;nofollow&quot;&gt;these notes by Hal Evans&lt;/a&gt; for the algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a problem with the step (3). I'm not sure how to represent the momentum term $p^2$. Is it just an index of the loop?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;creation of the initial state:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  unsigned int n=1400; ///&amp;gt; rank of the state vector psi(x). (x_0, x_1, ..., x_1399)^T&#xA;  const double dx=0.01;///&amp;gt; step of the state vector psi(x). dx = x_1-x_0;&#xA;&#xA;  array1&amp;lt;Complex&amp;gt; psix(n,align); ///&amp;gt; array1 is the array class from fftw package that is wrapped into fftwpp. psi(x) is the state vector in the position space&#xA;&#xA;  double x0 = 4; ///&amp;gt; the mean (center) of the gaussian of the initial state.&#xA;&#xA;  for(unsigned int i=0; i &amp;lt; n; i++) { ///&amp;gt; go over all points in the position space&#xA;    double x = (i-0.5*n)*dx; ///&amp;gt; for each point, calculate x&#xA;    x+=x0; ///&amp;gt; shift the gaussian back by x0&#xA;    psix[i]=exp(-pow(x,2)); ///&amp;gt; create the gaussian for the initial state&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;here is the part of the code that is doing the mentioned algorithm.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  /// step 1&#xA;  for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;    double x = (i-0.5*n)*dx; ///&amp;gt; get value of x for each element of position space vector&#xA;    ax[i]=exp(-ii*u(x)*dt*0.5*invhb)*psix[i];///&amp;gt; propagate half of the step using potential energy operator U:  a(x)=e^(-i*u*dt/(2*hb))*s(x,t)&#xA;  }&#xA;  /// step 2&#xA;  Forward.fft(ax,fs);///&amp;gt; do a fft into momentum space.&#xA;  /// step 3&#xA;  for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;    double p=i;///&amp;gt; THE PROBLEM IS HERE!!! , assign the value of the momentum to be used in kinetic energy operator. I just used the omega from http://en.wikipedia.org/wiki/Discrete_Fourier_transform#Definition&#xA;    bp[i]=exp(-ii*p*p*dt*0.5*invm)*fs[i];///&amp;gt; B(p) ≡ e^( −i p2 Δt / 2m) A(p) ///&amp;gt; act with a kinetic energy operator onto the momentum-space state vector.&#xA;  }&#xA;  /// step 4&#xA;  Backward.fftNormalized(bp,gfs);///&amp;gt; inverse fourier transform back into the position space&#xA;  /// step 5&#xA;  for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;    double x = (i-0.5*n)*dx;///&amp;gt; convert indices of the position-space state vector into values of position&#xA;    psix[i]=exp(-ii*u(x)*dt*0.5*invhb)*gfs[i];///&amp;gt; propagate with potential energy operator through the other half-step: psi(x, t+Δt)  =  e^(−i U Δt / 2ℏ) B(x)&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is my full c++ code (based on the tutorial to fftwpp wrapper &lt;a href=&quot;http://fftwpp.sourceforge.net/example1.cc&quot; rel=&quot;nofollow&quot;&gt;example&lt;/a&gt;  to fftw):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;limits&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;complex&amp;gt;&#xA;//#include &quot;gnuplot_i.hpp&quot;&#xA;#include &quot;Array.h&quot;&#xA;#include &quot;fftw++.h&quot;&#xA;&#xA;// instal fftw and &#xA;// Compile with:&#xA;// g++ -std=c++11 -I fftw++-1.13 -fopenmp main.cc fftw++-1.13/fftw++.cc -lfftw3 -lfftw3_omp; ./a.out&#xA;&#xA;using namespace std;&#xA;using namespace Array;&#xA;using namespace fftwpp;&#xA;&#xA;double u(double x) {&#xA;  return x*x;&#xA;}&#xA;int main()&#xA;{&#xA;  fftw::maxthreads=get_max_threads();&#xA;  const Complex ii(0.0,1.0);&#xA;  const double invhb = 1.0;&#xA;  const double invm = 1.0;&#xA;&#xA;  unsigned int n=1400;&#xA;  const double dx=0.01;&#xA;  const double dt=0.000001;&#xA;  unsigned int np=n/2+1;&#xA;  size_t align=sizeof(Complex);&#xA;&#xA;  array1&amp;lt;Complex&amp;gt; psix(n,align);&#xA;  array1&amp;lt;Complex&amp;gt; fs(n,align);&#xA;  array1&amp;lt;Complex&amp;gt; gfs(n,align);&#xA;&#xA;  fft1d Forward(n,-1,psix,fs);&#xA;  fft1d Backward(n,1,fs,gfs);&#xA;&#xA;  double x0 = 4;&#xA;&#xA;  for(unsigned int i=0; i &amp;lt; n; i++) {&#xA;    double x = (i-0.5*n)*dx;&#xA;    x+=x0;&#xA;    psix[i]=exp(-pow(x,2));&#xA;  }&#xA;&#xA;  ofstream ofs(&quot;psix.dat&quot;);&#xA;  for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;    double x = (i-0.5*n)*dx;&#xA;    ofs &amp;lt;&amp;lt; x &amp;lt;&amp;lt; &quot;  &quot; &amp;lt;&amp;lt; real(psix[i]) &amp;lt;&amp;lt; &quot;  &quot; &amp;lt;&amp;lt; imag(psix[i]) &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;  }&#xA;&#xA;  int tn=10;&#xA;  //// following http://hep.physics.indiana.edu/~hgevans/p411-p610/material/08_pde_iv/schrodinger.html&#xA;  array1&amp;lt;Complex&amp;gt; ax(n,align); ///&amp;gt; a(x)=e^(-i*u*dt/(2*hb))*s(x,t)&#xA;  array1&amp;lt;Complex&amp;gt; bp(n,align); ///&amp;gt; B(p) ≡ e^( −i p2 Δt / 2m) A(p)&#xA;  for (size_t l = 0; l &amp;lt; 100; ++l) {&#xA;    for (size_t j = 0; j &amp;lt; tn; ++j) {&#xA;      /// step 1&#xA;      for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;        double x = (i-0.5*n)*dx;&#xA;        ax[i]=exp(-ii*u(x)*dt*0.5*invhb)*psix[i];///&amp;gt; a(x)=e^(-i*u*dt/(2*hb))*s(x,t)&#xA;      }&#xA;      /// step 2&#xA;      Forward.fft(ax,fs);&#xA;      /// step 3&#xA;      for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;        double p=i;&#xA;        //double p=i*dx;&#xA;        bp[i]=exp(-ii*p*p*dt*0.5*invm)*fs[i];///&amp;gt; B(p) ≡ e^( −i p2 Δt / 2m) A(p)&#xA;      }&#xA;      /// step 4&#xA;      Backward.fftNormalized(bp,gfs);&#xA;      /// step 5&#xA;      for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;        double x = (i-0.5*n)*dx;&#xA;        psix[i]=exp(-ii*u(x)*dt*0.5*invhb)*gfs[i];///&amp;gt;psi(x, t+Δt)  =  e^(−i U Δt / 2ℏ) B(x)&#xA;      }&#xA;    }&#xA;    //// to plot gnuplots on the fly using gnuplot++, uncomment the lines below.&#xA;    //vector&amp;lt;double&amp;gt; xv,psir;&#xA;    //for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;    //  xv.push_back((i-0.5*n)*dx);&#xA;    //  psir.push_back(real(psix[i]));&#xA;    //}&#xA;    //Gnuplot g(&quot;lines&quot;);&#xA;    //g.plot_xy(xv,psir);&#xA;    //std::cin.ignore( std::numeric_limits &amp;lt;std::streamsize&amp;gt; ::max(), '\n' );&#xA;  }&#xA;&#xA;  ofstream offs(&quot;fs.dat&quot;);&#xA;  for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;    double p=i*dx;&#xA;    offs &amp;lt;&amp;lt; p &amp;lt;&amp;lt; &quot;  &quot; &amp;lt;&amp;lt; real(fs[i]) &amp;lt;&amp;lt; &quot;  &quot; &amp;lt;&amp;lt; imag(fs[i]) &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;  }&#xA;&#xA;  ofstream ofgfs(&quot;psixt.dat&quot;);&#xA;  for (size_t i = 0; i &amp;lt; n; ++i) {&#xA;    double x = (i-0.5*n)*dx;&#xA;    ofgfs &amp;lt;&amp;lt; x &amp;lt;&amp;lt; &quot;  &quot; &amp;lt;&amp;lt; real(psix[i]) &amp;lt;&amp;lt; &quot;  &quot; &amp;lt;&amp;lt; imag(psix[i]) &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5233" OwnerDisplayName="kirill_igum" LastActivityDate="2013-10-07T16:01:14.157" Title="Split operator FFT quantum dynamics for a harmonic oscillator" Tags="|quantum-mechanics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8607" PostTypeId="2" ParentId="8606" CreationDate="2013-09-25T16:44:34.407" Score="2" Body="&lt;p&gt;I have not looked at your code and this is not the place for it (though &lt;a href=&quot;https://scicomp.stackexchange.com/&quot;&gt;Computational Science&lt;/a&gt; might be), but your question looks simple in essence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have already transformed your wavefunction $\psi(x)$ into a momentum space representation $A(p)$, and you want to propagate it using the kinetic energy part of the hamiltonian, $p^2/2m$. You are then asked to&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Evaluate&#xA;  $$B(p) ≡ e^{−i p^2 Δt / 2m} A(p)$$&#xA;  in momentum space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here $A(p)$ and $B(p)$ are Hilbert space vectors, in a momentum representation, and the exponential is a matrix which is also diagonal in that representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any numerical implementation, of course, $A(p)$ will be discretized to a vector with a finite number of complex entries representing a set of amplitudes $(A(p_i))_{i=1}^N$. You then need to multiply each of these amplitudes by the relevant phase, to leave&#xA;$$(B(p_i))_{i=1}^N=(e^{−i p_i^2 Δt / 2m}A(p_i))_{i=1}^N.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this makes it clear what you need to do. If you have a specific question of how to implement this you should ask at Computational Science, or provide a detailed outline of what your code is doing and which part of the code implements the troublesome step.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Added&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Ok, so it looks like your problem is with the meaning of the index on the Fourier-transformed vector. This is impossible to resolve in a general manner as it will depend on the specific conventions used by your FFT algorithm but the general advice is independent of implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most implementations (and yours looks like one of these) you will change the continuous function $A(x)$ for a vector &lt;code&gt;a(i)&lt;/code&gt; which has a discrete index &lt;code&gt;i=0,...,n&lt;/code&gt; and which samples the continuous function in a finite range and with a finite resolution, &lt;code&gt;a(i)&lt;/code&gt;$=A(-x_0+i\delta x)$. You are then doing a discrete Fourier transform which is of the form&#xA;$$b(j)=\sum_{i=0}^n e^{2\pi \text i \,ij/n^2}a(i),$$&#xA;but unfortunately this is relatively different to your original intention of doing a continuous transform as&#xA;$$B(p)=\int_{-\infty}^\infty\text dx e^{\text i p x/\hbar} A(x). $$&#xA;As a result of this, the computed DFT &lt;code&gt;b(j)&lt;/code&gt; will in general be a reasonable approximation of the continuous Fourier transform, but the relationship between them can vary depending on conventions. The momentum resolution and the momentum range you'll have available will, in general, be inversely proportional to the position range and position resolution at which you sampled your original function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one further provision which is very important. The DFT is &lt;a href=&quot;http://en.wikipedia.org/wiki/Discrete_Fourier_transform#Periodicity&quot; rel=&quot;nofollow noreferrer&quot;&gt;periodic in its transformed variable&lt;/a&gt;, and this means that &lt;strong&gt;most DFT implementations will have the negative frequencies in the second half of the range&lt;/strong&gt;. This is what follows from the definition above, though of course you need to check with your implementation. As a result, one typically has to perform some sort of &quot;folding&quot; of the domain, and the interpretation of &lt;code&gt;b(i)&lt;/code&gt; will typically be of the form&#xA;$$&#xA;b(j)=\begin{cases}&#xA;\vphantom{\sum}B(j\delta p), &amp;amp;0\leq j\leq n/2 \\&#xA;\vphantom{\sum}B((j-n)\delta p),&amp;amp; n/2\leq j \leq n.\end{cases}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="3551" OwnerDisplayName="episanty" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-10-07T16:01:14.157" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8608" PostTypeId="2" ParentId="8602" CreationDate="2013-09-25T20:03:48.877" Score="5" Body="&lt;p&gt;Most modern systems are Princeton in the sense that the same memory system is used for data and instructions, although memory management systems often keep instructions and data in separate pages and user code may not be able to write into memory that contains executable instructions for security reasons.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a different level, there are typically separate caches for instructions and data.  This separation of instruction and data caches helps to overcome the Princeton bottleneck.  Furthermore, most modern systems don't ensure cache coherency between the data cache and the instruction cache.  This means that if you write new instructions into memory they may not immediately effect execution since the instruction cache might be relying on previously cached instructions.  Typically, operating system software has to clear the instruction cache after a new or modified program has been loaded into memory.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-09-25T20:03:48.877" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8609" PostTypeId="1" AcceptedAnswerId="8631" CreationDate="2013-09-25T22:00:46.853" Score="3" ViewCount="1214" Body="&lt;p&gt;I am using the implicit finite difference method to discretize the 1-D transient heat diffusion equation for solid spherical and cylindrical shapes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{\alpha}\frac{\partial T}{\partial t} = \frac{\partial^2 T}{\partial t^2} + \frac{p}{r} \frac{\partial T}{\partial r} \; \; \; \text{for} \; r\neq0 \\&#xA;\frac{1}{\alpha}\frac{\partial T}{\partial t} = (1+p)\frac{\partial^2 T}{\partial r^2} \; \; \; \text{for} \; r=0 \\&#xA;\text{note that }\; \; \alpha = \frac{k}{\rho C_p}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $p=1$ for cylinder and $p=2$ for sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions are:&#xA;$$&#xA;\left.\begin{matrix}&#xA;\frac{\partial T}{\partial r}&#xA;\end{matrix}\right|_{r=0} = 0 \; \; \; \text{for center node}\\&#xA;\left.\begin{matrix}&#xA;k\frac{\partial T}{\partial r}&#xA;\end{matrix}\right|_{r=R} = h(T_\infty - T_S) \; \; \; \text{for surface node}&#xA;$$&#xA;where $T_S$ is temperature at surface node and $R$ is outer radius of cylinder or sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the above equations and boundary conditions I arrived at the following discretized approximations for the temperatures at radial points from the center to surface:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the center node where $i=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left [ 1 + 2(1+p) Fo \right ]T_0^{\;n+1} - 2(1+p)FoT_1^{\;n+1} = T_0^\;n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the internal nodes where $i=1,2,...,M-1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Fo\left ( 1-\frac{p}{2i} \right )T_{i-1}^{\;n+1}+(1+2Fo)T_i^{\;n+1}-Fo\left ( 1+\frac{p}{2i} \right )T_{i+1}^{\;n+1}=T_i^{\;n}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and finally for the surface node where $i=M$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-2FoT_{M-1}^{\;n+1}+\left [ 1+2Fo \left (1+Bi+Bi\frac{p}{2M}  \right ) \right ] T_M^{\;n+1} = T_M^{\;n} + 2FoBi\left ( 1+\frac{p}{2M} \right )T_\infty&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $n$ is the present time while $n+1$ in the next time level and $Fo=\alpha\Delta t / \Delta r^2$, $Bi=h\Delta r / k$, $\alpha = k/\rho c_p$, $h$ is heat transfer coefficient, $\rho$ is density, $k$ is thermal conductivity, $c_p$ is heat capacity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So using the numerical equations, one can solve for the temperatures inside the sphere or cylinder by creating a system of equations in the form of $[A]\left \{ T \right \}=\left \{ B \right \}$ and solve the temperature at each node by using the Matlab operation T = A \ B&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{bmatrix}&#xA;1+2(1+p)Fo &amp;amp; -2(1+p) &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\ &#xA;Fo\left ( 1-\frac{p}{2i} \right ) &amp;amp; 1+2Fo &amp;amp; Fo\left ( 1+\frac{p}{2i} \right ) &amp;amp; 0 &amp;amp; 0\\ &#xA;0 &amp;amp; Fo\left ( 1-\frac{p}{2i} \right ) &amp;amp; 1+2Fo &amp;amp; Fo\left ( 1+\frac{p}{2i} \right ) &amp;amp; 0\\ &#xA;0 &amp;amp; 0 &amp;amp; Fo\left ( 1-\frac{p}{2i} \right ) &amp;amp; 1+2Fo &amp;amp; Fo\left ( 1+\frac{p}{2i} \right )\\ &#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -2Fo &amp;amp; 1+2Fo \left (1+Bi+Bi\frac{p}{2M}  \right )&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;T_0^{\;n+1}\\ &#xA;T_1^{\;n+1}\\ &#xA;T_2^{\;n+1}\\ &#xA;T_3^{\;n+1}\\&#xA;T_4^{\;n+1} &#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;T_0^{\;n}\\ &#xA;T_1^{\;n}\\ &#xA;T_2^{\;n}\\ &#xA;T_3^{\;n}\\ &#xA;T_4^{\;n}+2FoBi\left ( 1+\frac{p}{2M} \right )T_\infty&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do I include kinetic reactions into this system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following reaction rates for $w$ wood and $c$ char:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial \rho_w}{\partial t} = -(K_1+K_2)\rho_w \\&#xA;\frac{\partial \rho_{c1}}{\partial t} = K_2\rho_w \\&#xA;\frac{\partial \rho_{c2}}{\partial t} = K_3\rho_{c1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rate constants $K$ are represented by the Arrhenius equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;K=A\,e^{\frac{-E}{RT}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ = pre-factor, $E$ = activation energy, $R$ = universal gas constant, and $T$ = temperature at that node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to try to incorporate these reaction equations into my system of equations for the temperatures I have discretized the reactions using the implicit method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left [ 1+(K_1+K_2)\Delta t \right ] \rho_{wi}^{\;n+1} = \rho_{wi}^{\;n} \\&#xA;\rho_{c1i}^{\;n+1}-K_2\rho_{wi}^{\;n+1}\Delta t = \rho_{c1i}^{\;n} \\&#xA;\rho_{c2i}^{\;n+1}-K_3\rho_{c1i}^{\;n+1}\Delta t = \rho_{c2i}^{\;n}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion on how to incorporate these kinetic reactions into the system of temperature equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or should I solve for the temperatures first, then use the new temperatures in the reaction rates to update the $\rho$, then use the updated $\rho$ for the next iteration?&lt;/p&gt;&#xA;" OwnerUserId="5146" LastEditorUserId="5146" LastEditDate="2013-09-28T15:51:20.240" LastActivityDate="2013-09-28T22:46:26.810" Title="Implicit heat diffusion with kinetic reactions" Tags="|matlab|finite-difference|numerical-analysis|computational-chemistry|implicit-methods|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8610" PostTypeId="2" ParentId="8597" CreationDate="2013-09-26T01:33:53.073" Score="2" Body="&lt;p&gt;The series you show appear to have roughly the first three significant digits stable by the last term you show.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most of those same series, the first 5 steps have not gotten you to a correct first significant digit. On a log scale, this is a huge distance. To ask for a prediction that far out from so few terms is, to put it charitably, insanely optimistic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As well, be incredibly careful with multiple iterations of methods such as Shanks (as one response suggested.) That method tends to massively amplify floating point noise in your computations, so that after several iterations of that method, the noise has grown enough to overwhelm the signal. The point is, IF you do apply Shanks, only a few iterations will be of any value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it impossible to predict a limit from such a sequence after only a few terms? I won't say that, but to do any kind of a decent job, YOU need to provide input. Extrapolation (which is what you are doing) is a NASTY process, fraught with problems. However, if you can supply useful information that restricts the family of curves one might expect to see in this process, then the problem becomes far more tractable. (This is not always easy to do.)&lt;/p&gt;&#xA;" OwnerDisplayName="user840" LastEditorDisplayName="user840" LastEditDate="2013-09-26T01:55:25.897" LastActivityDate="2013-09-26T01:55:25.897" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8611" PostTypeId="2" ParentId="8594" CreationDate="2013-09-26T01:53:46.610" Score="4" Body="&lt;p&gt;dgeev is meant for smallish matrices where you need all eigenvalues. It may be that Mathematica is using a smarter but completely different method, such as Krylov subspace methods to generate the eigenvalues for your matrices.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-26T01:53:46.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8612" PostTypeId="1" CreationDate="2013-09-26T02:02:36.077" Score="1" ViewCount="115" Body="&lt;p&gt;I need to solve a quadratic program in Java which minimizes a sum of squares. The problem is that there are constraints to be satisfied, and so a quadratic optimizer must be used. I've been thinking of using &lt;a href=&quot;http://www.joptimizer.com/quadraticProgramming.html&quot; rel=&quot;nofollow&quot;&gt;JOptimizer&lt;/a&gt; for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem is that the $x^T A x$ term in the quadratic program. There are approximately 1000 variables, so the $A$ matrix will be dense with a million terms. What a waste of memory! Is there a way around having to make this big matrix in any other Java library that solves quadratic programs?&lt;/p&gt;&#xA;" OwnerUserId="4672" LastEditorUserId="522" LastEditDate="2013-10-01T20:08:42.397" LastActivityDate="2013-10-01T20:08:42.397" Title="Quadratic optimization without any cross terms" Tags="|optimization|quadratic-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8615" PostTypeId="2" ParentId="577" CreationDate="2013-09-26T12:16:16.043" Score="3" Body="&lt;p&gt;Here is a very good introductory book chapter on Monte Carlo simulations of classical spin systems, which also introduces and discusses the critical exponents:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;W. Janke,&#xA;Monte Carlo Methods in Classical Statistical Physics,&#xA;invited lectures, in: Computational Many-Particle Physics, Wilhelm &amp;amp; Else Heraeus Summerschool, Greifswald, Germany, 18-29 September 2006, edited by H. Fehske, R. Schneider, and A. Weiße, Lect. Notes Phys. 739 (Springer, Berlin, 2008), pp. 79-140&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.physik.uni-leipzig.de/~janke/Paper/lnp739_079_2008.pdf&quot; rel=&quot;nofollow&quot;&gt;You can get it for free from the author's homepage&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5236" LastActivityDate="2013-09-26T12:16:16.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8616" PostTypeId="1" CreationDate="2013-09-26T17:29:02.517" Score="2" ViewCount="58" Body="&lt;p&gt;I'm attempting to model energies of a transition metal complex using hybrid DFT but having issues with the description of my metal centers.  The problem is alleviated by adding extra polarization functions only to the metal centers.  Is this in general a safe procedure, or should I look into applying these extra polarization functions to all atoms?  Early tests seem to suggest that the computation time becomes prohibitively large when the polarization functions are applied to all atoms, but I'm concerned that there may be drawbacks to using a mixed basis set.&lt;/p&gt;&#xA;" OwnerUserId="5244" LastActivityDate="2014-07-01T18:28:08.883" Title="Quantum chemistry - are there potential issues to be concered about when using mixed basis sets in a calculation?" Tags="|computational-chemistry|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8617" PostTypeId="1" CreationDate="2013-09-26T20:23:23.067" Score="7" ViewCount="2695" Body="&lt;p&gt;Does anyone know a compiler for Fortran 77 available as a free download? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have pre-written 77 code from a source published in the early 90's that I just need to compile, build, and run. But I don't know enough about Fortran to update it myself.&lt;/p&gt;&#xA;" OwnerUserId="5246" LastActivityDate="2014-06-26T14:42:23.860" Title="Need Fortran 77 compiler" Tags="|fortran|" AnswerCount="4" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8618" PostTypeId="2" ParentId="8617" CreationDate="2013-09-26T20:25:57.407" Score="19" Body="&lt;p&gt;&lt;a href=&quot;http://gcc.gnu.org/wiki/GFortran&quot;&gt;GFortran&lt;/a&gt; is free and open source and usually works pretty well.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-09-26T20:25:57.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8619" PostTypeId="2" ParentId="8594" CreationDate="2013-09-26T20:27:47.160" Score="2" Body="&lt;p&gt;The LAPACK/BLAS implementation surely matters, but even with the reference BLAS (via the Numpy python library) it took less than 15 minutes (on a Core i7 laptop) to compute the eigenvalues for a random 4000*4000 real matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Was your matrix real or complex? Are you able to check the performance from eg. R or python, just to exclude the chance of errors in the Fortran code?&lt;/p&gt;&#xA;" OwnerUserId="5245" LastActivityDate="2013-09-26T20:27:47.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8620" PostTypeId="2" ParentId="8617" CreationDate="2013-09-26T20:46:36.507" Score="8" Body="&lt;p&gt;While not as free or available as GFortran that Bill mentions, the &lt;a href=&quot;http://software.intel.com/en-us/fortran-compilers&quot;&gt;Intel Fortran Compiler&lt;/a&gt; also works F77 (or atleast works for the legacy code I work with).  I am just putting this here to give an alternative, I would still recomment GFortran for most use cases simply due to the cost of ifort.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-09-26T20:46:36.507" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8621" PostTypeId="1" AcceptedAnswerId="8623" CreationDate="2013-09-26T22:00:46.170" Score="6" ViewCount="619" Body="&lt;p&gt;I'm trying to find the eigenvector corresponding to the second smallest eigenvalue of a large $(4,000,000 \times 4,000,000)$ matrix $L$. $L$ is a graph Laplacian, with the following structure: $L = D - SBS^\intercal$, where $D$ is a diagonal matrix and $S$ and $B$ are a large sparse matrices. $S$ has dimension $4,000,000 \times 10,000,000$, but only about $40,000,000$ non-zero entries. $B$ has similar scale and sparsity. So I can rapidly perform matrix-vector multiplication: $Lv = Dv - S(B(S^\intercal v))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I'm using Scipy (which calls an Arnoldi algorithm implemented in ARPACK) to find the smallest eigenvalues and corresponding eigenvectors of $L$. Rather than directly finding the smallest eigenpairs of $L$, I find the largest eigenpairs of $M^{-1}$, where $M = L + cI$. (Adding $cI$ and inverting doesn't change the eigenvectors.) To compute $M^{-1}v$, where $v$ is an arbitrary vector, I use the conjugate gradient algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this approach is slow: the Arnoldi algorithm often takes hundreds of iterations to converge, and each Arnoldi iteration requires a few hundred conjugate gradient (CG) iterations. Since I'm solving hundreds of CG problems with the same design matrix $M$, I was thinking there could be much efficiency gained by preconditioning. But what preconditioner would be suitable for this problem? I've read that provably good sparse approximations of Laplacians exists (&quot;ultrasparsifiers&quot;), but can they be shifted and inverted, without destroying their sparsity, to form good approximations of $M^{-1}$? Or are there techniques for preconditioning that exploit the structure of $M$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks much.&lt;/p&gt;&#xA;" OwnerUserId="5247" LastActivityDate="2013-10-02T15:45:53.570" Title="Preconditioner for finding the smallest eigenpairs of a large, but structured, matrix" Tags="|petsc|linear-solver|sparse-matrix|eigensystem|preconditioning|" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8622" PostTypeId="1" CreationDate="2013-09-27T00:52:43.433" Score="3" ViewCount="419" Body="&lt;p&gt;In certain cases, boundary integral methods are preferred for elliptic partial differential equations as opposed to finite difference methods. For instance, for solving the Poisson equation in a domain $D$&#xA;$$\nabla^2 u = \phi_0 \tag{$\star$}$$&#xA;with boundary condition $u = f$ on $\partial D$, instead of discretizing $(\star)$, I often see the boundary integral formulation being solved, i.e.,&#xA;$$u(x) = \int_{\partial D} \sigma(s) \dfrac{\partial G(x,y(s))}{\partial n_y} ds + \int_{D} G(x,y) \phi_0(y) dy$$&#xA;where the unknown density $\sigma(s)$ is obtained by enforcing the boundary condition, i.e.,&#xA;$$\dfrac{\sigma(x)}2 + \int_{\partial D} \sigma(s) \dfrac{\partial G(x,y(s))}{\partial n_y} ds + \int_{D} G(x,y) \phi_0(y) dy = f(x) \tag{$\dagger$}$$&#xA;What is the advantage of solving $(\dagger)$ instead of $(\star)$? Note that in $(\dagger)$, one integral is done over the entire domain, while the other is done over the boundary. Below are the list of questions, which I have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What dictates the choice of boundary integral method and finite difference method, i.e., under what circumstances are boundary integral methods preferred over finite difference?&lt;/li&gt;&#xA;&lt;li&gt;Is one more computationally accurate and/or easier and/or inexpensive than the other?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Overall, I would like to have a comparison between boundary integral methods and finite difference methods highlighting the pros and cons of both the methods.&lt;/p&gt;&#xA;" OwnerUserId="5248" LastEditorUserId="5248" LastEditDate="2013-09-27T02:14:02.950" LastActivityDate="2013-10-29T10:44:47.333" Title="Finite difference equations versus boundary integral equations for elliptic pdes" Tags="|finite-difference|integral-equations|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8623" PostTypeId="2" ParentId="8621" CreationDate="2013-09-27T02:08:25.500" Score="4" Body="&lt;p&gt;Assuming your overall matrix is positive definite (it is definitely symmetric), then I would suggest looking into algebraic multigrid (AMG) methods as preconditioners. They compute hierarchies of sparsified matrices themselves. If you're already using PETSc, take a look at the hypre preconditioner. Using this may force you to actually multiply out the elements of the matrix, but this may be worth doing. (It may also be possible that hypre has a way of accepting a fine level matrix that is unassembled.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-27T02:08:25.500" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8624" PostTypeId="2" ParentId="8621" CreationDate="2013-09-27T06:46:18.507" Score="4" Body="&lt;p&gt;If you are only interested in the smallest eigenvalue, the conjugate gradient method applied to the matrix $L$ gives you a good approximation after a reasonably small number of steps, and you won't have to solve any linear systems. The details are in Y. Saad's book on iterative methods, but here a short summary:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;From the coefficients that are computed in the conjugate gradient (cg) iteration you can construct a tridiagonal matrix $T$ of dimension number of cg steps. This matrix is the Galerkin projection of the matrix $L$ into the current Krylov space.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When the number of cg steps increases, the two extremal eigenvalues of $T$ approximate those of $L$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since $T$ is small and tridiagonal, it is fairly cheap to compute its eigenvalues, for instance with LAPACK.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once you have the eigenvalue, computing the corresponding eigenvector is fairly cheap.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As a side node: the cg solver implemented in &lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;nofollow&quot;&gt;deal.II&lt;/a&gt; has an option that allows you to compute the spectrum of $T$ in every step and thus to observe convergence of the eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-09-27T06:46:18.507" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8627" PostTypeId="1" CreationDate="2013-09-27T15:29:17.530" Score="5" ViewCount="121" Body="&lt;p&gt;In (hydrostatic) atmospheric general circulation models, for example the so-calle Primitive Equations, consisting of the horizontal momentum equation, the hydrostatic balance, the continuity equation, and the energy balance &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{eqnarray}&#xA;\nonumber&#xA;\frac{\partial}{\partial t}\vec{v} &amp;amp; = &amp;amp;  \vec{v}\times(f+\xi)\vec{e}_z -  \omega\frac{\partial}{\partial p}\vec{v}&#xA;                                           -\nabla \left(\Phi+\frac{\vec{v}^2}{2}\right) + \vec{R} \\&#xA;\frac{\partial }{\partial p}\Phi &amp;amp; = &amp;amp;  -\frac{1}{\rho} =  -\frac{RT}{p}\\&#xA;\nonumber&#xA;0 &amp;amp; = &amp;amp; \nabla\cdot\vec{v} +  \frac{\partial}{\partial p} \omega \\&#xA;\nonumber&#xA;c_p\frac{d}{d t} T &amp;amp; = &amp;amp; \frac{\omega}{\rho} + Q_{rad} + Q_{lat} + \eta + \epsilon .&#xA;\end{eqnarray}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are solved numerically. How are these equations modified and complemented, when the atmospheric model is to be coupled to a full ocean general circulation model, what additional effects and equations have to be included?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the salinity plays an important role in driving the oceanic circulations among other things, for example a balance equation for the salinity, appropriately coupled to the other governing equations will certainly be needed.&lt;/p&gt;&#xA;" OwnerUserId="3850" LastActivityDate="2013-09-27T15:29:17.530" Title="What are the governing equations solved in coupled atmosphere-ocean models?" Tags="|fluid-dynamics|simulation|model|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8628" PostTypeId="1" CreationDate="2013-09-27T20:01:21.287" Score="4" ViewCount="1847" Body="&lt;p&gt;I originally posted this in the mathematics StackExchange, but I was told that this is more of an operations research question.  I would like to know if anyone has any thoughts on how to go about finding an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following scenario that I want to find an algorithm for in order to put into a c# program: Transportation Inc is contracted to ship 2 ton crates from site A to site B and they want to find the most efficient way to do so with the following constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;All 4 trucks are initially at site B.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Only 2 trucks can be on the road at a time, regardless of the&#xA;direction of travel.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Each truck can carry no more than 6 crates&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Going from one site to another takes 30 minutes&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Putting a crate on a truck takes 5 minutes, and removing a crate from&#xA;a truck takes 5 minutes&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An algorithm needs to be provided in order to transport the most crates for a 6 hour period.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought I might have to write up a simulation in c#, but I would like to know other people's thoughts on how to write a more efficient algorithm for this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing the algorithm, then different variables could be made flexible so that they could test to see what adding more trucks or more efficient loading and unloading would have on the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="5254" LastActivityDate="2014-06-28T21:36:59.607" Title="Shipping algorithm" Tags="|algorithms|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8631" PostTypeId="2" ParentId="8609" CreationDate="2013-09-28T22:46:26.810" Score="2" Body="&lt;p&gt;As I said in my answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8490/matlab-solution-for-implicit-finite-difference-heat-equation-with-kinetic-reacti&quot;&gt;your previous question&lt;/a&gt;, it's probably better if you don't try to write your own ODE solver, which is what you are doing now. There are a lot of good libraries out there that will solve a system of ordinary differential equations much better than if you roll your own implementation. If you can use them, you should; since your previous post uses MATLAB, you have many different ODE solvers available to you, and there are additional ODE solvers you can install that improve upon these (such as SUNDIALS).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll have a system of equations that looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\mathrm{d}T_{i}}{\mathrm{d}t} = \ldots, \\&#xA;\frac{\mathrm{d}w_{i}}{\mathrm{d}t} = \ldots, \\&#xA;\frac{\mathrm{d}c_{i}}{\mathrm{d}t} = \ldots, \\&#xA;\frac{\mathrm{d}a_{i}}{\mathrm{d}t} = \ldots, \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$T_{i}(t) = T(x_{i}, t)$ is temperature&lt;/li&gt;&#xA;&lt;li&gt;$w_{i}(t) = \rho_{w}(x_{i}, t)$ is the wood density&lt;/li&gt;&#xA;&lt;li&gt;$c_{i}(t) = \rho_{c1}(x_{i}, t)$ is the char density&lt;/li&gt;&#xA;&lt;li&gt;$a_{i}(t) = \rho_{c2}(x_{i}, t)$ is the ash density (if memory serves, this was another component in your kinetics from your previous question)&lt;/li&gt;&#xA;&lt;li&gt;$i$ is an index for each grid point, so $i = 1, \ldots$ up to some number of grid points -- note that you'll need to handle boundary conditions appropriately&lt;/li&gt;&#xA;&lt;li&gt;and you need to fill in the right-hand side with the appropriate values&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you can pose your equations in this way, then you can use built-in ODE solvers, which should enable you to attempt to solve these equations without having to explicitly form any matrices yourself (they will be assembled within any implicit ODE solver that calculates finite difference approximations of the Jacobian matrix). However, if you need to supply a Jacobian matrix, you can do so by differentiating the right-hand side of the ODEs above with respect to the dependent variables; this matrix is the one you're looking to form in your question.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-09-28T22:46:26.810" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8633" PostTypeId="1" CreationDate="2013-09-29T01:41:04.063" Score="4" ViewCount="262" Body="&lt;p&gt;I have a system of nonlinear equations $F(x,a) = 0$ and I know that at a specific point $a_c$ a bifurcation occurs, thus the Jacobian becomes singular. How can I switch branches and start following a new solution path as I increment $a$?&lt;/p&gt;&#xA;" OwnerUserId="1059" LastEditorUserId="3980" LastEditDate="2013-10-17T04:14:29.780" LastActivityDate="2014-02-14T06:28:39.540" Title="Switch branch in bifurcation" Tags="|nonlinear-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8634" PostTypeId="2" ParentId="8622" CreationDate="2013-09-29T10:29:12.460" Score="2" Body="&lt;p&gt;The question asked is too broad, in my opinion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assessment of BIE methods vs. finite difference methods (or other domain methods) require careful analysis of many different points. Among them&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;is $D \subset \mathbb{R}^2$ or $D \subset \mathbb{R}^3$ or even $D \subset \mathbb{R}^N$ with $N&amp;gt;3$?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;is $D$ finite or infinite? is $D$ simply connected?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;which is the $\partial D$-surface to $D$-volume ratio compared to a domain diameter?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;how smooth is $\partial D$? is the analytical solution singular at the boundary or not?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;are you interested in an overall picture of the solution on $D$ ore you are looking for a very precise solution in just a few points? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if the solution is singular at some point, are interested in obtaining a very good asymptotic approximation at those points?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This said, I will not go into summarizing BIE methods (this could be the subject of a book chapter!) but just add some random remarks.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For sure FD on $(\star)$ is easier than say collocation on $(\dagger)$: $G(x,y)$ is singular for $y\rightarrow x$, you have to master CPV and FP integration to correctly implement a BIE method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With FD you compute an approximation of $u$ in $D$. With BIE you compute quantities on $\partial D$ and you have to recover values on $D$ by an integral representation. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;FD give raise to sparse matrices, BIE methods to dense ones.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So there are specific applications (class of problems) in which BIE methods are superior to FD methods, and situations in which they are just terrible, but I don't think it is possible to summarize them in a short answer.   &lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-09-29T10:29:12.460" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8635" PostTypeId="2" ParentId="8425" CreationDate="2013-09-29T10:59:14.150" Score="0" Body="&lt;p&gt;Let $f(x)=\dfrac{||Ax-b||^2}{||x||^2}$. We assume that $b$ is not in the image of $A$, otherwise there is nothing to do. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The first thing to do is to minimize $f(x)$ under the &quot;free&quot;  constraint $Ax-b&amp;gt;0$. $Df_x:h\rightarrow \dfrac{2((Ax-b)^T(Ah)||x||^2-||Ax-b||^2x^Th)}{||x||^4}$. $Df_x=0$ iff, for every $h$, $(||x||^2(Ax-b)^TA-||Ax-b||^2x^T)h=0.$ That is equivalent to : $||x||^2(Ax-b)^TA=||Ax-b||^2x^T$. That implies $(Ax-b)^TAx=||Ax-b||^2$ and then $(Ax-b)^Tb=0$. Remark that if $b&amp;gt;0$ or $b&amp;lt;0$, then there is no solution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In a second time, we must add constraints in the form $g_i(x)=e_i^T(Ax-b)=0$. Using Lagrange method, one has the condition $Df_x+2\sum_i\lambda_iDg_i(x)=0$, that is $||x||^2(Ax-b)^TA-||Ax-b||^2x^T+||x||^4\sum_i\lambda_ie_i^TA=0$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5153" LastActivityDate="2013-09-29T10:59:14.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8636" PostTypeId="2" ParentId="8223" CreationDate="2013-09-29T16:40:16.680" Score="3" Body="&lt;p&gt;You can also try distancePoints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://octave.sourceforge.net/geometry/function/distancePoints.html&quot; rel=&quot;nofollow&quot;&gt;http://octave.sourceforge.net/geometry/function/distancePoints.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5262" LastActivityDate="2013-09-29T16:40:16.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8638" PostTypeId="1" CreationDate="2013-09-29T22:51:23.460" Score="6" ViewCount="1891" Body="&lt;p&gt;I'm given a smooth probability density function via its values on a reasonable fine grid. I assume that cubic spline interpolation (or cubic spline interpolation of the logarithm of the density) will be sufficient to evaluate  it at arbitrary points with high accuracy. I wonder how to generate random numbers that reproduce this distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first shot was to approximate the cumulative distribution function of this distribution by a piecewise linear function $F$ (on the original grid), draw a number $r$ from $[0,1)$ uniformly at random, and take the $x$ with $F(x)=r$. However, I noticed that the accuracy of my final results is not great, and I suspect that I lose accuracy because the piecewise constant probability density of my numerical random variable doesn't approximate the real smooth probability density function well enough. What options do I have?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some of my ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to the library and look for a book about Monte Carlo simulation. Or try to ask an expert.&lt;/li&gt;&#xA;&lt;li&gt;Integrate the cubic spline analytically, which gives a piecewise quartic function $F$. There would still be an analytic formula for the $x$ with $F(x)=r$, but it will probably be complicated to implement and slow to evaluate. &lt;/li&gt;&#xA;&lt;li&gt;Approximate the smooth probability density function by a piecewise linear function, which gives a piecewise quadratic function $F$. The analytic formula for the $x$ with $F(x)=r$ should be simple to implement and reasonably fast to evaluate.&lt;/li&gt;&#xA;&lt;li&gt;Approximate the logarithm of the smooth probability density function by a piecewise linear function, which gives a piecewise &quot;simple&quot; analytic function $F$. The analytic formula for the $x$ with $F(x)=r$ should be simple to implement and reasonably fast to evaluate.&lt;/li&gt;&#xA;&lt;li&gt;Approximate the smooth probability density function $g$ by a piecewise constant function $f$ such that $g \leq 1.1 f$. Now use &lt;a href=&quot;http://en.wikipedia.org/wiki/Rejection_sampling&quot;&gt;rejection sampling&lt;/a&gt; by first sampling $x$ via $F(x)=r_1$, and then rejecting $x$ if $g(x) &amp;lt; 1.1 f(x) r_2$.&lt;/li&gt;&#xA;&lt;li&gt;Approximate $F^{-1}(r)$ by a suitable piecewise analytic function. But what does suitable mean here?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2013-09-30T07:56:37.567" LastActivityDate="2013-11-01T10:19:03.230" Title="How to sample numerically from an arbitrary smooth distribution?" Tags="|monte-carlo|probability|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8639" PostTypeId="1" CreationDate="2013-09-29T22:58:57.063" Score="1" ViewCount="117" Body="&lt;p&gt;I am not sure if I can post these kinds of questions on this site, but I was wondering if you could help me with a question that's been bothering me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have started learning about analysis of recursive algorithms and I got the gist of it. However there are some questions, like the one I'm going to post, that confuse me a little.&#xA;Can you please help me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the problem of multiplying two big integers, i.e. integers represented by a large number of bits that cannot be handled directly by the ALU of a single CPU. This type of multiplication has applications in data security where big integers are used in encryption schemes. The elementary-school algorithm for multiplying two n-bit integers has a complexity of . To improve this complexity, let x and y be the two n-bit integers, and use the following algorithm&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Recursive-Multiply(x,y)&#xA;  Write  x = x1 * 2^(n/2)+x0  //x1 and x0 are high order and low order n/2 bits&#xA;       y = y1 * 2^(n/2)+y0//y1 and y0  are high order and low order n/2 bits&#xA;  Compute x1+x0  and y1+y0&#xA;  p = Recursive-Multiply (x1+x0,y1+y0)&#xA;  x1y1 = Recursive-Multiply (x1,y1)&#xA;  x0y0 = Recursive-Multiply (x0,y0)&#xA;  Return  x1y1*2^n + (p-x1y1-x0y0)*2^(n/2)+x0y0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(a)Explain how the above algorithm works and provides the correct answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b)Write a recurrence relation for the number of basic operations for the above algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(c)Solve the recurrence relation and show that its complexity is O(n^lg3)&lt;/p&gt;&#xA;" OwnerUserId="5264" LastActivityDate="2013-09-29T22:58:57.063" Title="Recursive-Multiply Analysis" Tags="|algorithms|" AnswerCount="0" CommentCount="3" ClosedDate="2013-09-30T17:35:46.257" ContentLicense="CC BY-SA 3.0" />
  <row Id="8640" PostTypeId="1" CreationDate="2013-09-30T01:23:57.413" Score="0" ViewCount="335" Body="&lt;p&gt;Does anyone know why I'm seeing these errors? I'm not giving it a matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; t = 0:0.1:2*pi;&#xA;&amp;gt;&amp;gt;&amp;gt; quiver3(4*t,3*cos(t),3*sin(t));&#xA;error: __quiver__: operator +: nonconformant arguments (op1 is 3969x1, op2 is 6&#xA;3x1)&#xA;error: called from:&#xA;error:   /usr/share/octave/3.6.2/m/plot/private/__quiver__.m at line 179, colum&#xA;n 10&#xA;error:   /usr/share/octave/3.6.2/m/plot/quiver3.m at line 75, column 11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5265" LastActivityDate="2013-09-30T06:14:04.313" Title="quiver3 gives non-conformant arguments?" Tags="|visualization|octave|vectorization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8641" PostTypeId="2" ParentId="8638" CreationDate="2013-09-30T03:16:57.507" Score="1" Body="&lt;p&gt;I would go with your option 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it would have helped if you elaborated on your statement &quot;However, I noticed that the accuracy of my final results is not great&quot;. I say so because if the mesh on which your PDF is defined is fine enough, then I see no reason why your approach should not work. What I would do is try to debug things by starting with a PDF you know analytically, say a Gaussian, and evaluate the steps you do one by one. For example, start with a very fine mesh and piecewise constant approximation -- does the resulting set of samples look ok? If not, does it get better by using a piecewise linear approximation? If not, then the error must be somewhere else. Etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-09-30T03:16:57.507" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8642" PostTypeId="2" ParentId="8616" CreationDate="2013-09-30T03:25:02.707" Score="2" Body="&lt;p&gt;It depends on your criteria for what a good basis set is, but I don't that this is a problem.  If nothing else, in some cases it is impossible to use the same basis set on every atom because some basis sets are designed to be used with ECPs and some are all-electron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important thing is for you to try to converge the measured properties as much as possible.  If your properties are unlikely to be perturbed by polarization functions on the ligands, then I see no issue with excluding them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're worried about how a paper will be reviewed, then try to find examples of others doing the same scheme and cite those as evidence in your favor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you provide more detail, I can try to improve my answer.  &lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-09-30T03:25:02.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8643" PostTypeId="2" ParentId="7357" CreationDate="2013-09-30T03:29:27.857" Score="1" Body="&lt;p&gt;I don't know any production implementation that is released in a free and open-source package.  However, a few codes appear to be developing this capability.  You might try to write your own or modify an existing robust code that supports CI already to do this.  I'd recommend PSI4 if you decide to go that route.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-09-30T03:29:27.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8644" PostTypeId="1" AcceptedAnswerId="8648" CreationDate="2013-09-30T05:24:58.080" Score="9" ViewCount="2040" Body="&lt;p&gt;I am looking for GPU libraries to accelerate a code I have, whose primary &quot;high performance&quot; regions have singular value decomposition, QR factorization and Eigen values, Eigenvectors computation. I have looked through Nvidia's website on its tools like CuBLAS,Magma etc., but I do not want to write the code in Cuda.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it would be wonderful if there are any libraries out there which I can just &quot;call&quot; from my main C code for things like SVD,QR etc., and the computation would be done on the GPUs. So in summary, I have a normal C code, in which the intensive linear algebra operations are accelerated. Does anyone know of an open source (preferably) library which can do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks. &lt;/p&gt;&#xA;" OwnerUserId="4299" LastActivityDate="2013-09-30T19:09:13.293" Title="GPU libraries to use for Linear Algebra operations" Tags="|parallel-computing|gpu|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8645" PostTypeId="2" ParentId="8640" CreationDate="2013-09-30T06:14:04.313" Score="1" Body="&lt;p&gt;Matlab is a bit more explicit than Octave:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;quiver3(4*t,3*cos(t),3*sin(t))&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;??? Error using ==&gt; quiver3 at 53&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Not enough input arguments.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is no way of using &lt;code&gt;quiver3&lt;/code&gt; with only 3 arguments. What did you want to do?&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-09-30T06:14:04.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8646" PostTypeId="2" ParentId="8644" CreationDate="2013-09-30T06:17:37.477" Score="4" Body="&lt;p&gt;Karl Rupp, who writes ViennaCL (&lt;a href=&quot;http://viennacl.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;http://viennacl.sourceforge.net/&lt;/a&gt;) and is on Computational Sci StackEx, might be able to chime in here - their library has multiple matrix decompositions including SVD, LU, eigendecomp, etc. It's also a header-only library, and should play well with C++ (not sure about C) code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of it in use with LU factorizations - &lt;a href=&quot;http://viennacl.sourceforge.net/viennacl-examples-dense-matrix.html&quot; rel=&quot;nofollow&quot;&gt;http://viennacl.sourceforge.net/viennacl-examples-dense-matrix.html&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-09-30T06:17:37.477" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8647" PostTypeId="2" ParentId="8644" CreationDate="2013-09-30T09:10:33.520" Score="3" Body="&lt;p&gt;You may want to have a look at the &lt;a href=&quot;http://www.culatools.com/downloads/dense/&quot; rel=&quot;nofollow&quot;&gt;CULA library&lt;/a&gt;, which implements a number of the most common LAPACK/BLAS operations in single precision (free edition) and double precision (full edition, i.e. paying version).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The library works as a direct replacement for LAPACK/BLAS, so if you already use these functions in your original code, you shouldn't have to change anything.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-09-30T09:10:33.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8648" PostTypeId="2" ParentId="8644" CreationDate="2013-09-30T11:33:02.833" Score="8" Body="&lt;p&gt;For [CU]BLAS, there is a wrapper called 'thunking' in the CUDA toolkit (src/fortran_thunking.{c,h}) that takes pointers from CPU memory and does all the GPU allocation/copying for you.  You can plug it into your code with a preprocessor statements like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#define ZGEMV CUBLAS_ZGEMV&#xA;#define ZGEMM CUBLAS_ZGEMM&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For LAPACK, Magma has CPU-side interfaces for most, if not all, of its functionality (they tend to be implemented first) so, like with thunking, you just swap calls.  In some cases the interfaces or workspace requirements might be a bit different, so you'll want to be a bit more careful than with thunking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The necessary disclaimer is that there is no free lunch: if you're not willing to at least handle the memory copies explicitly, your performance is going to be limited to large problems with high computational intensity (flops / bytes).  If your problems are small and/or batched, I'd highly recommend that you write some control code to stream the memory copies and library calls to overlap memory copies and computation.  This isn't really CUDA, as much as an API, since you don't need to write kernels.  Both CUBLAS and Magma provide API calls for specifying streams.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2013-09-30T19:09:13.293" LastActivityDate="2013-09-30T19:09:13.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8649" PostTypeId="1" AcceptedAnswerId="8650" CreationDate="2013-09-30T14:33:38.693" Score="6" ViewCount="3001" Body="&lt;h3&gt;Background on finite volume method&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;When discretising the flux with a central difference stencil of the the advection-diffusion equation restriction $\frac{ah}{d} &amp;lt; 2$ must be observed for the finite volume method to remain numerical stability. Where $a$ is the velocity, $h$ is the cell width (considering 1D only) and $d$ is the diffusion coefficient. The term $\frac{ah}{d}$ is known as the Peclet number. For more information there is a nice description here, &lt;a href=&quot;http://www.ctcms.nist.gov/fipy/documentation/numerical/scheme.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.ctcms.nist.gov/fipy/documentation/numerical/scheme.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When solving the advection-diffusion equation in 1D, exponential fitting is used because it preserves the above property and allows the numerical scheme to automatically adapt from using second order accurate central differences where diffusion transport dominates and first order upwind differences where advection dominates.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Questions about finite element method&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does the finite element method also have a numerical stability criteria with respect to the Peclet number? &lt;/li&gt;&#xA;&lt;li&gt;If so what are the standard techniques to maintain stability? &lt;/li&gt;&#xA;&lt;li&gt;Do exponentially fitted scheme exist for the finite element method?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2013-10-01T17:42:56.073" Title="Does the finite element method impose any restrictions on the Peclet number for numerical stability?" Tags="|finite-element|numerical-analysis|finite-volume|advection-diffusion|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8650" PostTypeId="2" ParentId="8649" CreationDate="2013-09-30T14:52:40.800" Score="5" Body="&lt;p&gt;The finite element method has similar problems to FD with regards to stability in solving the advection-diffusion equation, i.e. the same restrictions on Peclet number apply. One remedy, also similar to that used in FD, is to use a formulation that includes &quot;upwinding&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice set of lecture notes that discusses how upwinding is added to FE formulations is here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ta.twi.tudelft.nl/users/vuik/burgers/fem_notes.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://ta.twi.tudelft.nl/users/vuik/burgers/fem_notes.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bill&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2013-09-30T14:52:40.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8651" PostTypeId="1" CreationDate="2013-09-30T15:21:47.100" Score="0" ViewCount="1455" Body="&lt;p&gt;Let&#xA;$$f(x) = \left\{ \begin{array}{l}&#xA;{a_1}x + {b_1} &amp;amp; if\,0 \le {x_1} \le x \le {x_2}\\&#xA;{a_2}x + {b_2} &amp;amp; if\,{x_2} &amp;lt; x \le {x_3}\\&#xA; \vdots \\&#xA;{a_n}x + {b_n} &amp;amp; if\,{x_{n - 1}} &amp;lt; x \le {x_n}&#xA;\end{array} \right.$$&#xA;be a piecewise continuous function and $0\leq a_1\leq a_2\leq \cdots \leq a_n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formulate the problem of miniimizing $f(x)$ in terms of&#xA;linear programming&lt;/p&gt;&#xA;" OwnerUserId="5268" LastActivityDate="2013-09-30T15:38:22.860" Title="minimization of a convex piecewise linear function" Tags="|optimization|convex-optimization|" AnswerCount="1" CommentCount="1" ClosedDate="2013-10-08T19:09:07.723" ContentLicense="CC BY-SA 3.0" />
  <row Id="8652" PostTypeId="2" ParentId="8638" CreationDate="2013-09-30T15:22:20.993" Score="3" Body="&lt;p&gt;If your PDF is bounded, you could try approximating its inverse with a high-degree polynomial interpolant. This is usually considered a bad thing, but that's &lt;a href=&quot;http://eprints.maths.ox.ac.uk/1352/1/NA-11-07.pdf&quot; rel=&quot;nofollow&quot;&gt;just a myth&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some things to keep in mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of using an equispaced grid, interpolate at the &lt;a href=&quot;http://en.wikipedia.org/wiki/Chebyshev_nodes&quot; rel=&quot;nofollow&quot;&gt;Chebyshev nodes&lt;/a&gt; of the first kind, i.e. $x_i = \cos\left(\pi\frac{2i-1}{2N}\right)$, for $F(x)$ defined in $[-\infty,\infty]$, or second kind, i.e. $x_i = \cos\left(\pi\frac{i-1}{N-1}\right)$, for $F(x)$ defined on a finite interval.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If $F(x)$ is on an infinite interval, don't interpolate $F^{-1}(r)$, as it will have singularities at the endpoints, but interpolate $F^{-1}(r)/(r^2-r)$, as this will cancel-out the singularities at $r=0$ and $r=1$. Using the Chebyshev nodes of the first kind will avoid evaluating $F^{-1}(r)$ at these singular points.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can evaluate your interpolant using &lt;a href=&quot;http://en.wikipedia.org/wiki/Barycentric_Interpolation#Barycentric_interpolation&quot; rel=&quot;nofollow&quot;&gt;Barycentric interpolation&lt;/a&gt;. Note that if you evaluated $F^{-1}(r)$ on Chebyshev nodes of the first or second kind, the Barycentric weights $w_j$ have closed-form expressions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For a faster evaluation that vectorizes well, use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Vandermonde_matrix&quot; rel=&quot;nofollow&quot;&gt;Vandermonde&lt;/a&gt;-like matrix $V$ with $V_{ij}=T_{j}(x_i)$ to compute the Chebyshev coefficients of your interpolant once (if you used the Chebyshev nodes, $V$ should be well conditioned) and use &lt;a href=&quot;http://en.wikipedia.org/wiki/Clenshaw_algorithm#Special_case_for_Chebyshev_series&quot; rel=&quot;nofollow&quot;&gt;Clenshaw's algorithm&lt;/a&gt; to evaluate it for more than one $r$ at a time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The method described here is more or less what the &lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/&quot; rel=&quot;nofollow&quot;&gt;Chebfun system&lt;/a&gt; does (disclaimer: I used to be part of the Chebfun developer team). Most of the basic Chebyshev technology is described in Nick Trefethen's book &quot;&lt;a href=&quot;http://www2.maths.ox.ac.uk/chebfun/ATAP/&quot; rel=&quot;nofollow&quot;&gt;Approximation Theory and Approximation Practice&lt;/a&gt;&quot;, of which the first six chapters are available online.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-09-30T15:22:20.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8653" PostTypeId="2" ParentId="8651" CreationDate="2013-09-30T15:38:22.860" Score="1" Body="&lt;p&gt;Note that $f(x)=\max_{i} a_{i}x+b_{i}$.  You can then reformulate the problem as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_{t,x} t $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ t \geq a_{i}x + b_{i}$,  $i=1, 2, \ldots, n$.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-09-30T15:38:22.860" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8654" PostTypeId="2" ParentId="8595" CreationDate="2013-09-30T19:05:49.980" Score="2" Body="&lt;p&gt;Have you tried using a pseudo-Cartesian (not sure of the proper terminology) coordinate system like $u = \theta \cos \phi$, $v = \theta \sin \phi$? This will avoid the instability at the south pole but is not continuous at the north pole - I don't know whether that is OK for your purposes.&lt;/p&gt;&#xA;" OwnerUserId="3458" LastActivityDate="2013-09-30T19:05:49.980" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8656" PostTypeId="2" ParentId="8617" CreationDate="2013-09-30T19:31:00.990" Score="4" Body="&lt;p&gt;G95 is another alternative free fortran compiler, available for quite a few platforms. I think it is a bit easier to install on Windows compared to Gfortran (the OP did not mention what platform they were interested in).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://g95.org/&quot; rel=&quot;nofollow&quot;&gt;http://g95.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;G95 has seen less updating and maintenance compared to Gfortran (whihch is part of the main GCC distribution since 4.0), but if you just need F77, G95 should work great. Gfortran actually began as a fork of G95. &lt;/p&gt;&#xA;" OwnerUserId="7477" LastActivityDate="2013-09-30T19:31:00.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8659" PostTypeId="1" AcceptedAnswerId="8660" CreationDate="2013-09-30T21:35:11.587" Score="3" ViewCount="391" Body="&lt;p&gt;My question is probably very simple and deals with separation of degrees of freedom in a Monte Carlo Brownian dynamics simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dealing with a particle in an external potential, I want to simulate a Brownian motion in three spatial dimensions. Is it possible to evaluate the individual degrees of freedom separately i.e. calculate displacement in x, evaluate the probabilities, and accept/reject the step and then repeat for y and z? Or do I have to evaluate all degrees of freedom in a single MC step?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now what if the degrees of freedom are not independent, i.e. the displacement in x will change the potential (probability) in y etc.? Can the individual degrees of freedom be still calculated separately? Let's say I calculate displacement in x which gets accepted but displacement in y gets rejected. Do I have to throw out both of the displacements? Or can I keep the displacement in x?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that in multi-particle systems, the MC method doesn't evaluate all particles at the same time, therefore it should be possible to evaluate the degrees of freedom separately even if they influence the probability distribution of other degrees of freedom.&lt;/p&gt;&#xA;" OwnerDisplayName="bioEngineer" LastEditorUserId="4244" LastEditDate="2013-10-01T06:59:26.290" LastActivityDate="2013-10-03T12:50:45.523" Title="Separation of degrees of freedom in Monte Carlo simulation" Tags="|monte-carlo|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8660" PostTypeId="2" ParentId="8659" CreationDate="2013-09-30T21:54:00.117" Score="2" Body="&lt;p&gt;I assume you are writing a Metropolis-Hastings Monte Carlo algorithm.  So, you are trying to generate moves to sample the configuration space and accept/reject these moves based on energy considerations (MHMC), right?  If so, then you are free to define your moves however you choose.  There comes eventually the question of efficiency, as proposing unlikely moves will result in small acceptance probabilities over simulation time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per your question about a displacement in x- and y-, I suppose this depends upon your method of proposing/testing moves.  Generally, we do not sample configuration space this way, you should just propose one move (say -5 Angstroms in the y direction and 2 Angstroms in the x direction), test whether it should be accepted or rejected based on the MH algorithm, and move on without breaking it down into its components.  This is really a matter of efficiency and you are free to sample as you see fit as long as you preserve ergodicity and detailed balance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that it is also perfectly acceptable to grab a few atoms (or molecules, clusters, whatever you are studying) and move them all at once and then compute the acceptance probability.&lt;/p&gt;&#xA;" OwnerUserId="5279" OwnerDisplayName="MaxGraves" LastEditorUserId="5279" LastEditDate="2013-10-01T18:06:20.990" LastActivityDate="2013-10-01T18:06:20.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8661" PostTypeId="2" ParentId="8659" CreationDate="2013-10-01T13:34:08.913" Score="0" Body="&lt;p&gt;Leaving efficiency considerations aside, your moves are OK if they satisfy &lt;a href=&quot;http://en.wikipedia.org/wiki/Detailed_balance&quot; rel=&quot;nofollow&quot;&gt;detailed balance&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-10-01T13:34:08.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8662" PostTypeId="1" CreationDate="2013-10-01T13:46:27.967" Score="3" ViewCount="101" Body="&lt;p&gt;I am trying to implement ellipsoid-ellipsoid collision in my C++ code. &#xA;Briefly this task can described as next: Let's assume that we have two arbitrarily oriented ellipses in the in space and this ellipses are intersected. This ellipses initially located in the center of coordinate system. We need to find how far we need to move one ellips relative to the other along the line joining their centres until the would not be intersected more. It is quite simple mathematical task. I use next equations: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)ellipsoid equation in matrix form&#xA;$$\vec{x}^TA1\vec{x}-1 = 0$$ &#xA;$$(\vec{x}-d\vec{n})^TA2(\vec{x}-d\vec{n})-1 = 0$$&#xA;2)equality of normals&#xA;$$A1\vec{x} = \lambda A2(\vec{x}-d\vec{n})$$&#xA;Where $$\vec{n}$$ is the unit vector in the direction from one center to another.&#xA;I am looking for variable $$d$$ from this system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found this system and its solution on the last page of this article &lt;a href=&quot;http://www.matthiasmueller.info/publications/orientedparticles.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.matthiasmueller.info/publications/orientedparticles.pdf&lt;/a&gt; It seems like everything is fine, this algorithm works well for most cases, however I have noticed, that for some ellipses there is an error  occurs (I mean that this algorithm gives not exact result). I also have noticed, that error occurs primarily when the ratio between the ellipse radii more than 2.&#xA;In order to solve this problem I have changed all variables in my code from float to double, but it did not give any result. It seems like small error in input data results in very big error in output data. Does anybody know how I can overcome this kind of instability?  &lt;/p&gt;&#xA;" OwnerUserId="5278" LastActivityDate="2013-10-01T15:33:03.940" Title="numerical inaccuracy ellipsoid-ellipsoid collision" Tags="|linear-algebra|simulation|numerical-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8663" PostTypeId="2" ParentId="8662" CreationDate="2013-10-01T14:20:12.230" Score="2" Body="&lt;p&gt;In effect you want to &quot;run backwards&quot; until two about-to-intersect ellipses are just touching, to find the time and location of collisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think &quot;very simple mathematical task&quot; is how I would describe finding distances between two ellipses in general positions.  The concise write-up by David Eberly in &lt;a href=&quot;http://www.geometrictools.com/Documentation/DistanceEllipse2Ellipse2.pdf&quot; rel=&quot;nofollow&quot;&gt;Distance Between Ellipses in 2D&lt;/a&gt; promises a numerically stable algorithm, and he's well-known for his contributions in this field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem of minimum distance between a circle and an ellipse is easier, involving only a fourth-degree polynomial's roots.  So it seems reasonable that the approximate methods you've tried are less reliable when the ellipses have higher ratios between major and minor radii.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some 3D notation/machinery, see also David Eberly's &lt;a href=&quot;http://www.geometrictools.com/Documentation/DistancePointToEllipsoid.pdf&quot; rel=&quot;nofollow&quot;&gt;Distance from a Point to an Ellipse, an Ellipsoid, or a Hyperellipsoid&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2013-10-01T15:33:03.940" LastActivityDate="2013-10-01T15:33:03.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8664" PostTypeId="2" ParentId="8649" CreationDate="2013-10-01T17:42:56.073" Score="2" Body="&lt;p&gt;It depends on what you refer to as numerical stability criteria - with the CFL condition, there's a clear requirement on eigenvalues to ensure that the timestepping method does not diverge. With the Peclet number, it's a bit less clear - you know that you don't want large oscillations, but it's hard to quantify it any more than that, especially in 2 and 3 dimensions and with high order methods for FD and FEM. You can get exact nodal stability with the right Peclet number or stabilization scheme in 1D, but it's usually hard to extend these ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm approaching this from a FEM perspective, so keep in mind there may be other methods to deal with this instability for FD/FV. Typical approaches include&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Artificial diffusion - you add a fake diffusion term to make your Peclet number $\approx 1$.  These are fishy though - they change the problem that you solve, and can give you an answer that's way off (this typically gets worse for nonzero forcing). Advanced versions of this only add diffusion in areas with high gradients (see shock capturing artificial diffusion). &lt;/li&gt;&#xA;&lt;li&gt;Stabilization terms - this is akin to Bill's upwinding, and is a bit like artificial diffusion. However, unlike artificial diffusion, it also modifies the way the forcing term is taken into account, so it avoids some of the above issues. For FEM, this can be interpreted as changing your weighting functions. The difficulty for convection-diffusion problems is that you don't want to be &lt;em&gt;too&lt;/em&gt; diffusive; your solution may look good but it might be completely wrong. &lt;/li&gt;&#xA;&lt;li&gt;Flux limiting/TVD/MUSCL schemes - these I know less about, but they seem very popular with the FD and FV crowd. As far as I can tell, you dynamically change your stencil or numerical fluxes to achieve certain properties (local max principles, variation diminishing solutions, etc). &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are a couple good places to start with this - &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782597002302&quot; rel=&quot;nofollow&quot;&gt;this tutorial paper&lt;/a&gt; on a priori and a posteriori estimation for FEM details how Peclet number of $O(1)$ is required for good a prior estimates in FEM, and the ideas should carry over to FD too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another good paper with a fun title is &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0045793081900268&quot; rel=&quot;nofollow&quot;&gt;Don't suppress the wiggles! They're telling you something&lt;/a&gt; by Gresho.  It warns of the danger of too much upwinding and just going for &quot;good looking solutions&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for papers on the 3 methods, I don't know of especially landmark ones off the top of my head, and can only recommend Googling around. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Btw, if this seems a bit confusing or arbitrary, IMO, the field sort of is arbitrary and confusing - numerical methods for convection-diffusion problems, as simple as they seem, still seem to hold a lot of open problems.)&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-10-01T17:42:56.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8665" PostTypeId="2" ParentId="8628" CreationDate="2013-10-01T18:28:54.040" Score="2" Body="&lt;p&gt;Here is a solution I came up with. I wrote a truck simulation. But I am open to better solutions&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using System;&#xA;using System.Collections.Generic;&#xA;using System.Linq;&#xA;using System.Text;&#xA;&#xA;namespace Infrastructure&#xA;{&#xA;    public enum TruckState&#xA;    {&#xA;        Loading,&#xA;        Unloading,&#xA;        TravalingToPickupSite,&#xA;        TravelingToDropOffSite,&#xA;        WaitingToLeavePickupSite,&#xA;        WaitingToLeaveDropOffSite,&#xA;        WaitingToBeLoaded,&#xA;        WaitingToBeUnloaded&#xA;    }&#xA;&#xA;    public class MainStart&#xA;    {&#xA;        static void Main()&#xA;        {&#xA;            var truckSimulation = new TruckSimulation();&#xA;            var result = truckSimulation.RunSimulation();&#xA;            Console.WriteLine(&quot;Crates delivered: &quot; + result);&#xA;            Console.WriteLine(&quot;Press any key to exit.&quot;);&#xA;            Console.ReadKey();&#xA;        }&#xA;    }&#xA;&#xA;    public class TruckSimulation&#xA;    {&#xA;        public const int MaxLoad = 6;&#xA;        public const int TimeToUnloadACrate = 5;&#xA;        public const int TimeToLoadACrate = 5;&#xA;        public const int TimeToPickUpSite = 30;&#xA;        public const int TimeToDropOffSite = 30;&#xA;        public const int TotalTime = 360;&#xA;        public const int TotalTrucks = 6;&#xA;        public const int MaxTrucksOnRoad = 6;&#xA;&#xA;        public TruckSimulation()&#xA;        {&#xA;            Trucks = new List&amp;lt;Truck&amp;gt;();&#xA;            for (int i = 0; i &amp;lt; TotalTrucks; i++)&#xA;            {&#xA;                var Truck = new Truck();&#xA;                Truck.LoadCount = 0;&#xA;                Truck.CurrentTruckState = TruckState.WaitingToLeaveDropOffSite;&#xA;                Truck.TimeToNextTruckState = 0;&#xA;                Trucks.Add(Truck);&#xA;            }&#xA;            CurrentTime = 0;&#xA;            TotalCratesDroppedOff = 0;&#xA;        }&#xA;&#xA;&#xA;        public List&amp;lt;Truck&amp;gt; Trucks { get; set; }&#xA;        public int TotalCratesDroppedOff { get; set; }&#xA;        public int CurrentTime { get; set; }&#xA;        public int TrucksOnTheRoad { get; set; }&#xA;&#xA;        public int RunSimulation()&#xA;        {&#xA;            // start time&#xA;            do&#xA;            {&#xA;                TruckWaitingToBeLoaded();&#xA;                TruckLoading();&#xA;                TruckWaitingToLeavePickupSite();&#xA;                TruckTravelingToDropOffSite();&#xA;                TruckWaitingToBeUnloaded();&#xA;                TruckUnloading();&#xA;                TruckWaitingToLeaveDropOffSite();&#xA;                TruckTravalingToPickupSite();  &#xA;&#xA;                // increment time&#xA;                CurrentTime++;&#xA;            } while (CurrentTime &amp;lt;= TotalTime);&#xA;&#xA;            return TotalCratesDroppedOff;&#xA;        }&#xA;&#xA;        public void TruckWaitingToBeLoaded()&#xA;        {&#xA;            // look for Truck waiting to be loaded and change to start loading and change TruckState and set time to next TruckState&#xA;            var TruckWaitingtoBeLoaded = FindFirstTruckWithTruckState(TruckState.WaitingToBeLoaded);&#xA;            if (TruckWaitingtoBeLoaded != null &amp;amp;&amp;amp; FindFirstTruckWithTruckState(TruckState.Loading) == null)&#xA;            {&#xA;                TruckWaitingtoBeLoaded.CurrentTruckState = TruckState.Loading;&#xA;                TruckWaitingtoBeLoaded.TimeToNextTruckState = CurrentTime + TimeToLoadACrate;&#xA;            }&#xA;        }&#xA;&#xA;        public void TruckLoading()&#xA;        {&#xA;            // look for Truck that is loading and check if ready for next Crate&#xA;            var TruckThatIsLoading = FindFirstTruckWithTruckState(TruckState.Loading);&#xA;            if (TruckThatIsLoading != null)&#xA;            {&#xA;                if (TruckThatIsLoading.TimeToNextTruckState == CurrentTime)&#xA;                {&#xA;                    TruckThatIsLoading.LoadCount++;&#xA;                    if (TruckThatIsLoading.LoadCount &amp;lt; MaxLoad)&#xA;                    {&#xA;                        TruckThatIsLoading.TimeToNextTruckState = CurrentTime + TimeToLoadACrate;&#xA;                    }&#xA;                    else&#xA;                    {&#xA;                        TruckThatIsLoading.CurrentTruckState = TruckState.WaitingToLeavePickupSite;&#xA;                    }&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        public void TruckWaitingToLeavePickupSite()&#xA;        {&#xA;            // look for Truck that is waiting to leave pick up site, check to see if can send to drop off site (if less than two Trucks on the Road)&#xA;            var TruckWaitingToLeavePickupSite = FindFirstTruckWithTruckState(TruckState.WaitingToLeavePickupSite);&#xA;            if (TruckWaitingToLeavePickupSite != null)&#xA;            {&#xA;                if (TrucksOnTheRoad &amp;lt; MaxTrucksOnRoad)&#xA;                {&#xA;                    TruckWaitingToLeavePickupSite.CurrentTruckState = TruckState.TravelingToDropOffSite;&#xA;                    TruckWaitingToLeavePickupSite.TimeToNextTruckState = CurrentTime + TimeToDropOffSite;&#xA;                    TrucksOnTheRoad++;&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        public void TruckTravelingToDropOffSite()&#xA;        {&#xA;            // look for Truck that is on the Road and if ready to unload&#xA;            foreach (var TruckTravelingToDropOffSite in FindAllTrucksWithTruckState(TruckState.TravelingToDropOffSite))&#xA;            {&#xA;                if (TruckTravelingToDropOffSite != null)&#xA;                {&#xA;                    if (TruckTravelingToDropOffSite.TimeToNextTruckState == CurrentTime)&#xA;                    {&#xA;                        TruckTravelingToDropOffSite.CurrentTruckState = TruckState.WaitingToBeUnloaded;&#xA;                        TrucksOnTheRoad--;&#xA;                    }&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        public void TruckWaitingToBeUnloaded()&#xA;        {&#xA;            // look for Truck that is waiting to be unloaded&#xA;            var TruckWaitingToBeUnloaded = FindFirstTruckWithTruckState(TruckState.WaitingToBeUnloaded);&#xA;            if (TruckWaitingToBeUnloaded != null &amp;amp;&amp;amp; FindFirstTruckWithTruckState(TruckState.Unloading) == null)&#xA;            {&#xA;                TruckWaitingToBeUnloaded.CurrentTruckState = TruckState.Unloading;&#xA;                TruckWaitingToBeUnloaded.TimeToNextTruckState = CurrentTime + TimeToUnloadACrate;&#xA;            }&#xA;        }&#xA;&#xA;        public void TruckUnloading()&#xA;        {&#xA;            // look for Truck that is unloading and is ready to unload another Crate&#xA;            var TruckThatIsUnloading = FindFirstTruckWithTruckState(TruckState.Unloading);&#xA;            if (TruckThatIsUnloading != null)&#xA;            {&#xA;                if (TruckThatIsUnloading.TimeToNextTruckState == CurrentTime)&#xA;                {&#xA;                    TruckThatIsUnloading.LoadCount--;&#xA;                    TotalCratesDroppedOff++;&#xA;                    if (TruckThatIsUnloading.LoadCount == 0)&#xA;                    {&#xA;                        TruckThatIsUnloading.CurrentTruckState = TruckState.WaitingToLeaveDropOffSite;&#xA;                    }&#xA;                    else&#xA;                    {&#xA;                        TruckThatIsUnloading.TimeToNextTruckState = CurrentTime + TimeToUnloadACrate;&#xA;                    }&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        public void TruckWaitingToLeaveDropOffSite()&#xA;        {&#xA;            // look for a Truck that is waiting to leave drop off site&#xA;            var TruckWaitingToLeaveDropOffSite = FindFirstTruckWithTruckState(TruckState.WaitingToLeaveDropOffSite);&#xA;            if (TruckWaitingToLeaveDropOffSite != null)&#xA;            {&#xA;                if (TrucksOnTheRoad &amp;lt; MaxTrucksOnRoad)&#xA;                {&#xA;                    TruckWaitingToLeaveDropOffSite.CurrentTruckState = TruckState.TravalingToPickupSite;&#xA;                    TruckWaitingToLeaveDropOffSite.TimeToNextTruckState = CurrentTime + TimeToPickUpSite;&#xA;                    TrucksOnTheRoad++;&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        public void TruckTravalingToPickupSite()&#xA;        {&#xA;            // look for a Truck that has arrived at the pickup site&#xA;            foreach (var TruckTravelingToPickupSite in FindAllTrucksWithTruckState(TruckState.TravalingToPickupSite))&#xA;            {&#xA;                if (TruckTravelingToPickupSite != null)&#xA;                {&#xA;                    if (TruckTravelingToPickupSite.TimeToNextTruckState == CurrentTime)&#xA;                    {&#xA;                        TrucksOnTheRoad--;&#xA;                        TruckTravelingToPickupSite.CurrentTruckState = TruckState.WaitingToBeLoaded;&#xA;                    }&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        public Truck FindFirstTruckWithTruckState(TruckState TruckState)&#xA;        {&#xA;            return Trucks.FirstOrDefault(x =&amp;gt; x.CurrentTruckState == TruckState);&#xA;        }&#xA;&#xA;        public List&amp;lt;Truck&amp;gt; FindAllTrucksWithTruckState(TruckState TruckState)&#xA;        {&#xA;            return Trucks.Where(x =&amp;gt; x.CurrentTruckState == TruckState).ToList();&#xA;        }&#xA;    }&#xA;&#xA;    public class Truck&#xA;    {&#xA;        public Truck()&#xA;        {&#xA;            LoadCount = 0;&#xA;            CurrentTruckState = TruckState.WaitingToLeaveDropOffSite;&#xA;            TimeToNextTruckState = 0;&#xA;        }&#xA;        public int LoadCount { get; set; }&#xA;        public TruckState CurrentTruckState { get; set; }&#xA;        public int TimeToNextTruckState { get; set; }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5254" LastActivityDate="2013-10-01T18:28:54.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8666" PostTypeId="1" CreationDate="2013-10-01T22:51:42.687" Score="3" ViewCount="1186" Body="&lt;p&gt;How can one determine the interfacial tension at a solid-liquid interface using molecular dynamics? What are some good articles on this topic?&lt;/p&gt;&#xA;" OwnerUserId="3887" LastActivityDate="2013-10-05T18:52:17.233" Title="Molecular dynamics: how to calculate interfacial tension" Tags="|computational-chemistry|molecular-dynamics|" AnswerCount="1" CommentCount="0" ClosedDate="2013-10-05T12:57:43.743" ContentLicense="CC BY-SA 3.0" />
  <row Id="8667" PostTypeId="1" CreationDate="2013-10-02T09:45:46.577" Score="1" ViewCount="94" Body="&lt;p&gt;I have to find a solution to a problem and I don't know where to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets imagine I have a list of n containers and m processes. Each process resides already within a container and have 4 attributes, ex: cpu, memory, IO and network utilization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sum of each attribute type of all processes within a host cannot exceed the physical limitation of resources of that host, ex: 5 processes consuming 10% of cpu within a host can be together within a host, but 15 can't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to find the best (or a good one) solution that assigns those processes to the hosts with the following restrictions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use the minimum containers/hosts possibles.&lt;/li&gt;&#xA;&lt;li&gt;Minimize processes exchanges, from one host to another, if a process is already in one host it would be desirable for it to stay there.&lt;/li&gt;&#xA;&lt;li&gt;The processes attributes can changes dynamically but there might be a trend based on time or prior behavior.&lt;/li&gt;&#xA;&lt;li&gt;Maximize a host CPU utilization&lt;/li&gt;&#xA;&lt;li&gt;Maximize memory utilization&lt;/li&gt;&#xA;&lt;li&gt;Minimize IO utilization within a host so processes don't conflict to each other.&lt;/li&gt;&#xA;&lt;li&gt;Minimize network utilization.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have seen a proposed solution of a simpler version of this problem (just using CPU attribute), using Markov chains evaluating the past values of the CPU utilization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first thought applying a multi-objectives Knapsack approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help please.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="5284" LastEditorUserId="5284" LastEditDate="2013-10-02T20:28:19.617" LastActivityDate="2013-10-02T20:28:19.617" Title="Assignment optimization problem" Tags="|optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8670" PostTypeId="1" AcceptedAnswerId="8671" CreationDate="2013-10-02T12:59:48.530" Score="4" ViewCount="282" Body="&lt;p&gt;Restricted to one dimensional problem, is it possible to dynamically adapt the finite element method (FEM) discretisation based on the local value of the Péclet number ($P_e$) for advection-diffusion problems? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For advection-diffusion problems central discretisation becomes unstable when $P_e&amp;gt;2$, that is, when the problem becomes advection dominated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common way to solve this problem for the finite volume method (FVM) is with exponential fitting. For example, when solving the advection-diffusion equation the diffusion term is discretised with a central difference stencil, $$d(x)u_{xx} \approx \boldsymbol{D}_2\boldsymbol{u}(u_{j-1}, u_j, u_{j+1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advection term is discretised as a weighted sum of a central difference and a upwind stencil, $$a(x)u_x \approx \kappa \boldsymbol{D}_1\boldsymbol{u}(u_{j-1}, u_j, u_{j+1}) + (1-\kappa) \boldsymbol{U}_1 \boldsymbol{u}(u_j, u_{j-1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NB&lt;/strong&gt; There are much easier ways to incude exponential fitting with the FVM, the above illustrates the concept. To see how it is done in practice I refer you to my notes, &lt;a href=&quot;http://danieljfarrell.github.io/FVM/advection_diffusion.html#adaptive-upwinding-exponential-fitting&quot; rel=&quot;nofollow&quot;&gt;http://danieljfarrell.github.io/FVM/advection_diffusion.html#adaptive-upwinding-exponential-fitting&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This nice feature about exponential fitting is that it chooses the value of $\kappa$ such that the discretisation weights in favour of the stable upwind stencil when $P_e$ is large and in favour or the higher accuracy central stencil when $P_e$ is small. This is done locally based on the value of $P_e$. It also has the nice property of preserving monotonicity of the solution (at least in 1D) which is a nice feature for physical simulations (conservation laws).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can exponential fitting be introduced with the FEM in 1D? Does the approach also preserve monotonicity as with the FVM?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2013-10-02T13:05:52.950" Title="How can exponential fitting be used with the finite element method?" Tags="|finite-element|numerical-analysis|finite-volume|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8671" PostTypeId="2" ParentId="8670" CreationDate="2013-10-02T13:05:52.950" Score="5" Body="&lt;p&gt;There are whole books written on this, but you should investigate (search the web, really) upwind diffusion and Streamline Upwind Petrov Galerkin finite element methods first. There many more methods that extend these ideas or approach the problem from a different direction. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-10-02T13:05:52.950" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8672" PostTypeId="2" ParentId="8621" CreationDate="2013-10-02T15:45:53.570" Score="2" Body="&lt;p&gt;You should definitely try the Jacobi-Davidson method. One advantage of this method is that an exact $M^{-1} v $ is not necessary. Only an approximate $M^{-1} v $ is needed. &#xA;The convergence of the Jacobi-Davidson method may be improved by using a preconditioner. If it is possible to compute $L=D−SBS^T$ explicitly, then you can try a level '0' incomplete cholesky factorization of this matrix. Possibly you need a small shift to avoid that the incomplete factorization breaks down on a zero or negative pivot. You can also use Jaobi-Davidson without preconditioner, but the convergence may be less well in that case.&#xA;There exists a matlab code of Jacobi-Davidson that is optimized for symmetric matrices:&lt;a href=&quot;http://mntek3.ulb.ac.be/pub/docs/jdcg/&quot; rel=&quot;nofollow&quot;&gt;JDCG&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5286" LastActivityDate="2013-10-02T15:45:53.570" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8673" PostTypeId="1" CreationDate="2013-10-02T17:50:44.217" Score="1" ViewCount="274" Body="&lt;p&gt;I am trying to solve the following 1D inhomogeneous wave equation. &#xA;Forgive me if I a miss any rigorous mathematical concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial^2 u}{\partial x^2} - \frac{1}{c^2}\frac{\partial^2 u}{\partial t^2} = \delta(x)f(t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c$ is the wave velocity, $f(t)$ any force function and $\delta (x)$ is the dirac delta function. The $\delta(x)$ is used to simulate a point source at $x=0$. I used the Fourier Transform $\mathcal{F}$ first to get to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  \frac{\partial^2 U_{\omega}(\omega,x)}{\partial x^2} + \frac{\omega^2}{c^2} U_{\omega}(\omega,x) =  \delta(x)F(\omega)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  \frac{\partial^2 U_{\omega}(\omega,x)}{\partial x^2} + k^2 U_{\omega}(\omega,x) =  \delta(x)F(\omega)$$ &lt;strong&gt;(A)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $ k^2 = \frac{\omega^2}{c^2} $ and the subscript $ U_{\omega}(\omega,x) = \mathcal{F_t}(u(t,x)) $ meaning Fourier Transformed from $t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this is the Helmholtz equation I used the already known Green's function solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u(t,x) = \mathcal{F}^{-1}\{ G(x) \ast \left[ F_{\omega} \delta(x) \right] \} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u(t,x) = \mathcal{F}^{-1}\{F_{\omega} \frac{j e^{jkx}}{2k}\} $$ &lt;strong&gt;(B)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $j = \sqrt{-1} $ imaginary unit and $ G(x) = \frac{j e^{jkx}}{2k} $ is the Green's function for the &lt;a href=&quot;http://en.wikipedia.org/wiki/Helmholtz_equation&quot; rel=&quot;nofollow&quot;&gt;inhomogeneous Helmoltz equation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forgive-me for the simple question, but what would be the best way to evaluate &lt;strong&gt;B&lt;/strong&gt; numerically, for any discrete source &lt;strong&gt;f(t)&lt;/strong&gt; in a discrete grid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was using FFT and IFFT but I was getting weird results, your guidance will help a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another condition I want the energy vanishes at infinity $ x^+_-$ .&lt;/p&gt;&#xA;" OwnerUserId="4092" LastEditorUserId="4092" LastEditDate="2013-10-03T14:23:09.483" LastActivityDate="2013-10-03T14:23:09.483" Title="Numerically evaluate 1D inhomogeneous wave equation solution" Tags="|numerical-analysis|hyperbolic-pde|wave-propagation|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8676" PostTypeId="1" AcceptedAnswerId="8680" CreationDate="2013-10-03T12:20:36.463" Score="3" ViewCount="232" Body="&lt;p&gt;I have a function that is not only infinitely differentiable, but it is also very cheap to calculate any of those derivatives. It looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(\boldsymbol{C}, \boldsymbol{x})=\sum_{i} C_{i} \prod_{j} a_{j}^{x_{i,j}} -\sum_{i}C_{i}  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where both i run over the same length, and where $a_{j}$ are constants. This function is easily differentiable because after the first, it's either just removing a $C_{i}$ or multiplying with $ln(a_{j})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for the most efficient method of finding the zeroes of such a function, that takes full advantage of the differentiability. Are there any methods that are preferred in this scenario?&lt;/p&gt;&#xA;" OwnerUserId="1264" LastEditorUserId="1264" LastEditDate="2013-10-03T13:38:12.597" LastActivityDate="2013-10-04T10:46:37.863" Title="Finding zeroes of an infinitely differentiable function of ~100 to ~1000 variables" Tags="|optimization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8677" PostTypeId="2" ParentId="8659" CreationDate="2013-10-03T12:50:45.523" Score="0" Body="&lt;p&gt;What you describe is commonly called &quot;Gibbs sampling&quot;, see &lt;a href=&quot;http://en.wikipedia.org/wiki/Gibbs_sampler&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Gibbs_sampler&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-03T12:50:45.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8678" PostTypeId="2" ParentId="83" CreationDate="2013-10-03T20:03:26.080" Score="3" Body="&lt;p&gt;fmincon now can be used from Python via OpenOpt framework, optionally with dense/sparse automatic differentiation by FuncDesigner&#xA;&lt;a href=&quot;http://openopt.org/fmincon&quot; rel=&quot;nofollow&quot;&gt;http://openopt.org/fmincon&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5299" LastActivityDate="2013-10-03T20:03:26.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8680" PostTypeId="2" ParentId="8676" CreationDate="2013-10-03T22:10:23.030" Score="6" Body="&lt;p&gt;Your function can be written as&#xA;$$&#xA;f(C, x) = C^T d(x), \quad\text{with } (d(x))_i = \prod_j a_j^{x_{i,j}} - 1,&#xA;$$&#xA;so for any fixed $x$, the roots of $f$ are exactly the vectors $C$ in the subspace orthogonal to $d(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reverse is also true: For any given $C$, find an orthogonal vector $d$ and solve&#xA;$$&#xA;d_i = \prod_j a_j^{x_{i,j}} - 1&#xA;$$&#xA;for $x$. One obvious solution would be&#xA;$$&#xA;x_{i,1} = \ln(d_i+1) / \ln(a_1),\\&#xA;x_{i,j} = 0, j &amp;gt; 1.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-10-04T10:46:37.863" LastActivityDate="2013-10-04T10:46:37.863" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="8681" PostTypeId="1" CreationDate="2013-10-04T02:28:31.850" Score="1" ViewCount="1286" Body="&lt;p&gt;I modeled the heat equation,&#xA;$$&#xA;u_t = au_{xx}&#xA;$$&#xA;using the common 2nd-order Crank-Nicolson scheme,&#xA;$$&#xA;\frac{u^{n+1}_i-u^{n}_i}{dt} = \frac{a}{2\,dx}\left(u_{i-1}^{n+1}+u_{i+1}^{n+1}-2u_i^{n+1} + u_{i-1}^n+u_{i+1}^n-2u_i^n\right)&#xA;$$&#xA;and the standard 4th-order Crank-Nicolson scheme,&#xA;$$&#xA;\frac{u^{n+1}_i-u^{n}_i}{dt} = \frac{a}{24\,dx}\left(-u_{i-2}^{n+1}+16u_{i-1}^{n+1}-30u_{i}^{n+1}+16u_{i+1}^{n+1}-u_{i+2}^{n+1}\right) \\&#xA;   \quad+\frac{a}{24\,dx}\left(-u_{i-2}^{n}+16u_{i-1}^{n}-30u_{i}^{n}+16u_{i+1}^{n}-u_{i+2}^{n}\right)&#xA;$$&#xA;Outside of figuring out how to solve the &lt;a href=&quot;http://en.wikipedia.org/wiki/Pentadiagonal_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;pentadiagonal matrix&lt;/a&gt; (without resorting to libraries like LAPACK), this was fairly straight-forward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But my results were strange: the 2nd-order scheme was &lt;em&gt;more accurate&lt;/em&gt; to the exact solution (in 3 different test cases!) than the 4th-order scheme. &lt;strong&gt;Is there any reason we should expect a 2nd-order accurate scheme to perform better than the 4th-order scheme?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it matters, $a=1$, $x\in(0,1)$ and $u(0,t)=u(1,t)=0$ for all examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$u(x,0)=6\sin(\pi x)$ leading to the analytic solution $u(x,t)=6\sin(\pi x)\exp(-\pi^2t)$&lt;/li&gt;&#xA;&lt;li&gt;$u(x,0)=12\sin(9\pi x) - 7\sin(4\pi x)$ leading to $u(x,t)=12\sin(9\pi x)\exp(-9\pi^2t) - 7\sin(4\pi x)\exp(-9\pi^2t)$&lt;/li&gt;&#xA;&lt;li&gt;$u(x,0)=10$ leading to $u(x,t)=\sum_{k=1}^{100}\frac{40}{(2k-1)\pi}\exp\left(-(2k-1)^2\pi^2t\right)\sin\left((2k-1)\pi x\right)$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With this last one, though, both did pretty poorly at the boundaries, but the 2nd-order method was closer to the exact in the range $x\in(0.1,0.9)$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that it's an order of magnitude difference between the two schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;two sine term&#xA;&lt;img src=&quot;https://i.stack.imgur.com/ZV4ZJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;single sine term&#xA;&lt;img src=&quot;https://i.stack.imgur.com/H9mcq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;constant 10&#xA;&lt;img src=&quot;https://i.stack.imgur.com/bdpFY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran this with different $N$'s and still found the 2nd order version to be more accurate than the 4th order. &lt;/p&gt;&#xA;" OwnerUserId="4762" LastEditorUserId="4762" LastEditDate="2013-10-04T14:43:05.023" LastActivityDate="2013-10-04T14:43:05.023" Title="Crank-Nicolson for 2nd- and 4th-order finite differences" Tags="|finite-difference|crank-nicolson|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8683" PostTypeId="1" CreationDate="2013-10-04T05:32:05.873" Score="11" ViewCount="1176" Body="&lt;p&gt;For example the C++ sparse matrix libraries I used -- Eigen and SuiteSparse, they seem not to have any SVD funcitionality for sparse matrix. So just curious, is SVD more difficult than QR/LU for sparse matrix?&lt;/p&gt;&#xA;" OwnerUserId="5302" LastActivityDate="2013-10-04T05:53:41.087" Title="Why SVD is talk about less than QR and LU for sparse matrix?" Tags="|linear-algebra|sparse-matrix|matrix|svd|eigen|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8684" PostTypeId="2" ParentId="8683" CreationDate="2013-10-04T05:53:41.087" Score="13" Body="&lt;p&gt;The LU factors of a sparse matrix are at least somewhat sparse. The $Q$ matrix in QR can also somewhat preserve sparsity, and is typically used when the matrix is very long and skinny. The SVD of a sparse matrix will almost always have fully dense $U$ and $V$ factors, so it destroys any reason to perform the computations treating the matrix sparsely.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-04T05:53:41.087" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8686" PostTypeId="2" ParentId="8666" CreationDate="2013-10-04T12:04:42.380" Score="1" Body="&lt;p&gt;It would appear that section C of &lt;a href=&quot;http://jcp.aip.org/resource/1/jcpsa6/v124/i20/p204715_s1&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; JCP article is what you are looking for.  I see you are affiliated with a university, so you should be able to access that.  They seem to be doing this for gas-liquid interfaces, so you will obviously want to start the solid part (in your case) in some lattice configuration depending upon the material you are studying and carry out the simulation accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if this article contains what you were looking for, as I started to write my own molecular dynamics package (aiming towards path integral molecular dynamics for simulation of quantum fluids) but had to put it down to continue with my thesis research for the time being.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What systems are you studying?  And under what physical conditions (temp., density, etc..)  This might affect the answer to this question and also might dictate what kind of molecular dynamics simulation you will need to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try &lt;a href=&quot;http://www.utdallas.edu/~son051000/my_publications/contactangle_jmathchem2009.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; publication.  I only read the abstract, so I am not sure that it will explicitly show you how to do it with MD, but from the abstract, it claims to give a method for measuring the surface tension between a planar solid face and a liquid.&lt;/p&gt;&#xA;" OwnerUserId="5279" LastEditorUserId="5279" LastEditDate="2013-10-05T18:52:17.233" LastActivityDate="2013-10-05T18:52:17.233" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8687" PostTypeId="1" AcceptedAnswerId="8688" CreationDate="2013-10-04T12:52:29.223" Score="5" ViewCount="285" Body="&lt;p&gt;I was looking at the documentation for &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.4/docs/manualpages/DM/DMDAVecGetArray.html&quot; rel=&quot;noreferrer&quot;&gt;DMDAVecGetArray&lt;/a&gt; and was surprised that it could create a plain ordinary C array whose indices somehow ranged from, say, &lt;code&gt;istart&lt;/code&gt; to &lt;code&gt;istart + size - 1&lt;/code&gt;, rather than from 0 to &lt;code&gt;size - 1&lt;/code&gt;, or a multidimensional array (also an ordinary C array) whose indices ranged from &lt;code&gt;istart&lt;/code&gt; to &lt;code&gt;istart + isize - 1&lt;/code&gt;, &lt;code&gt;jstart&lt;/code&gt; to &lt;code&gt;jstart + jsize - 1&lt;/code&gt;, etc. I wasn't sure how that worked, so I looked into the code and found that DMDAVecGetArray uses the functions VecGetArray1d, VecGetArray2d, etc. behind the scenes to create these oddly indexed arrays. The code for VecGetArray1d looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PetscErrorCode  VecGetArray1d(Vec x,PetscInt m,PetscInt mstart,PetscScalar *a[])&#xA; {&#xA;   PetscInt       N;&#xA;&#xA;   VecGetLocalSize(x,&amp;amp;N);&#xA;   if (m != N) SETERRQ2(PETSC_COMM_SELF,PETSC_ERR_ARG_OUTOFRANGE,&quot;Local array size %D does not match 1d array dimensions %D&quot;,N,m);&#xA;   VecGetArray(x,a);&#xA;   *a  -= mstart;&#xA;   return(0);&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At least superficially, this looks similar to the &lt;a href=&quot;http://c-faq.com/aryptr/non0based.html&quot; rel=&quot;noreferrer&quot;&gt;trick for doing 1-indexed arrays in C&lt;/a&gt;, that is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int realarray[10];&#xA;int *array = &amp;amp;realarray[-1];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;except that the pointer &lt;code&gt;a&lt;/code&gt; in the PETSc code above is directly decremented instead of having a second pointer variable like &lt;code&gt;array&lt;/code&gt; above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I looking at what I think I'm looking at? I find it hard to believe that a well-used library code like PETSc would relying on undefined behavior, so I'm not sure if that PETSc code is really illegal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is actually going on?&lt;/p&gt;&#xA;" OwnerUserId="5307" LastEditorUserId="5307" LastEditDate="2013-10-04T13:12:51.233" LastActivityDate="2013-10-04T14:56:20.080" Title="Is PETSc's DMDA relying on undefined behavior?" Tags="|petsc|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8688" PostTypeId="2" ParentId="8687" CreationDate="2013-10-04T14:56:20.080" Score="5" Body="&lt;p&gt;Heh, fun question.  If you read the C standard carefully, you'll find wording like (C99, 6.5.6.8) about pointer arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If both the pointer&#xA;  operand and the result point to elements of the same array object, or one past the last&#xA;  element of the array object, the evaluation shall not produce an overflow; otherwise, the&#xA;  behavior is undefined.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As I understand it, the rationale for this restriction is segmented memory models, for which pointers reside in different registers and merely creating a pointer to an invalid memory location (without dereferencing it) could crash or corrupt the program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We &lt;a href=&quot;http://lists.mcs.anl.gov/pipermail/petsc-dev/2010-March/002539.html&quot; rel=&quot;noreferrer&quot;&gt;discussed the issue on petsc-dev&lt;/a&gt; a few years ago.  The conclusion was that segmented memory systems are more-or-less dead and that the &quot;trick&quot; was too useful to give up.  If you need to run on a system with segmented memory in which this pointer arithmetic will fail, you should use plain &lt;code&gt;VecGetArray()&lt;/code&gt; and do the indexing yourself.  You might use C99 VLA-pointers to preserve structured array indexing, but those arrays will necessarily start at zero, so you wouldn't get uniform indexing between the local and global vectors.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-10-04T14:56:20.080" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8689" PostTypeId="2" ParentId="4" CreationDate="2013-10-04T15:33:46.770" Score="1" Body="&lt;p&gt;Here are some references that might be useful:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://pubs.acs.org/doi/abs/10.1021/cr00074a002&quot; rel=&quot;nofollow&quot;&gt;http://pubs.acs.org/doi/abs/10.1021/cr00074a002&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0167797785900036&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/0167797785900036&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?id=Gt4pnp-UFhUC&amp;amp;lpg=PA853&amp;amp;vq=helgaker&amp;amp;pg=PA725#v=snippet&amp;amp;q=helgaker&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;http://books.google.com/books?id=Gt4pnp-UFhUC&amp;amp;lpg=PA853&amp;amp;vq=helgaker&amp;amp;pg=PA725#v=snippet&amp;amp;q=helgaker&amp;amp;f=false&lt;/a&gt; (if you see nothing, click next then prev).  See &lt;a href=&quot;http://www.worldscientific.com/doi/abs/10.1142/9789812832115_0001&quot; rel=&quot;nofollow&quot;&gt;http://www.worldscientific.com/doi/abs/10.1142/9789812832115_0001&lt;/a&gt; to find the analog version in your library.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://vergil.chemistry.gatech.edu/courses/chem6485/pdf/basis-sets.pdf&quot; rel=&quot;nofollow&quot;&gt;http://vergil.chemistry.gatech.edu/courses/chem6485/pdf/basis-sets.pdf&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-10-04T15:33:46.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8690" PostTypeId="1" AcceptedAnswerId="8750" CreationDate="2013-10-04T19:50:33.893" Score="3" ViewCount="245" Body="&lt;p&gt;I would like to map each point in $[0,1]$ to $[0,1]^n$ with a Hilbert curve, where $n=10,000$. That is&#xA;$$&#xA;f: [0,1] \to [0,1]^n,&#xA;$$&lt;br&gt;&#xA;is the $n$-dimensional Hilbert curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the library of &lt;a href=&quot;https://github.com/cortesi/scurve&quot; rel=&quot;nofollow&quot;&gt;Cortesi&lt;/a&gt;,&#xA;or the code given in &lt;a href=&quot;http://books.google.ca/books?id=ZXM8AAAAQBAJ&amp;amp;pg=PA119&amp;amp;dq=sergeyev+strongin+lera&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=gRhPUvmqLKGGyAGGxIHQDA&amp;amp;redir_esc=y#v=onepage&amp;amp;q=sergeyev%20strongin%20lera&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Sergeyev, Strongin and Lera&lt;/a&gt; book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cortesi library used a system of index, so I don't have a map $f$.&lt;br&gt;&#xA;The code from the book gives a map $f$, but it has trouble with large $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If possible, I would like a pointer to a freely avaible code.&lt;br&gt;&#xA;Or some papers/advices on how to write such an $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My use case is that I have a function&#xA;$$&#xA;G: [0,1]^n  \to \mathbb R,&#xA;$$&#xA;that I would like to visualize in 2D with&#xA;$$&#xA;H = G \circ f : [0,1] \to \mathbb R. &#xA;$$&#xA;So I want $f$ to be a Hilbert curve, so that set containments are preserved.&lt;/p&gt;&#xA;" OwnerDisplayName="user729" LastActivityDate="2013-10-14T15:28:28.403" Title="Is there an Implementation of the Hilbert curve from $[0,1]$ to $[0,1]^n$, where $n$ is large? ($n=10,000$, say)" Tags="|visualization|high-dimensional|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8692" PostTypeId="1" CreationDate="2013-09-23T20:17:38.887" Score="3" ViewCount="94" Body="&lt;p&gt;I'm trying to carry out the following integral numerically&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{r_\mathrm{in}}^{r_\mathrm{out}} \Sigma\left(r'\right) \frac{r'}{r} \left( \frac{1}{r-r'}\, E(L) + \frac{1}{r+r'}\, K(L) \right) \mathrm{d}r',$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L = \frac{4 r r'}{(r+r')^2},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $K$ and $E$ are the complete elliptical integrals of the first and second kind. $r$, $r_\mathrm{in}$, and $r_\mathrm{out}$ are all $&amp;gt;0$, $r_\mathrm{in}\leq r \leq r_\mathrm{out}$ and $\Sigma(r)$ is some positive, well behaved function (no singularities, or discontinuities, ...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, I need to evaluate the result numerically for a given $\Sigma(r')$, but my usual python integrators &lt;code&gt;scipy.integrate.quad&lt;/code&gt; or &lt;code&gt;scipy.integrate.romberg&lt;/code&gt; fail to converge or give unreliable results due to the two singularities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$1/(r-r')$ in the first summand&lt;/li&gt;&#xA;&lt;li&gt;$K(L)\rightarrow \infty$ as $r'\rightarrow r$ in the second summand ($L \rightarrow 1$)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me to evaluate this numerically or to rewrite it such that the usual methods don't fail?&lt;/p&gt;&#xA;" OwnerDisplayName="John Smith" LastEditorUserId="6663" LastEditDate="2013-12-26T16:39:51.880" LastActivityDate="2013-12-26T16:39:51.880" Title="analytic or numeric integral of diverging function" Tags="|convergence|integral-equations|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8693" PostTypeId="1" CreationDate="2013-10-05T12:09:37.720" Score="7" ViewCount="189" Body="&lt;p&gt;How can I generate realizations of random complex variable $x(t)$ with a given autocorrelation function $C(s)$, defined by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C(s) = \langle x(s) x(0) \rangle$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and obeying the condition $C(-s) = C^*(s)$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any link related to similar random variable generation would be great.&lt;/p&gt;&#xA;" OwnerUserId="5311" LastEditorUserId="20688" LastEditDate="2017-04-04T18:03:03.117" LastActivityDate="2017-04-04T18:03:03.117" Title="Generation of variable with given auto-correlation function" Tags="|algorithms|statistics|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8696" PostTypeId="1" AcceptedAnswerId="8747" CreationDate="2013-10-02T01:41:16.247" Score="0" ViewCount="304" Body="&lt;p&gt;I'm looking for a piece of software for performing molecular dynamics simulations with the following specifications:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;required&lt;/strong&gt;: well published (as in there are papers citing it in one or more &lt;em&gt;reasonably&lt;/em&gt; respectable journals)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;required&lt;/strong&gt;: incorporates some degree of non-determinism in how it runs (ex. each run from the same initial state may result in a slightly different end state for the system) &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;required&lt;/strong&gt;: should be able to generate initial states from a distribution (specifically the Maxwell-Boltzmann distribution)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;required&lt;/strong&gt;: at the end of any particular trial run, it should be possible to get &lt;strong&gt;either&lt;/strong&gt; a local temperature in a region I define, or the locations and velocities of all particles in the system&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;optional&lt;/strong&gt;: ideally we would be simulating gas particles as hard spheres with elastic collisions&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;optional&lt;/strong&gt;: it would be an added bonus if it's written in something similar to either Java or Python as I'm quite familiar with those languages&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;note&lt;/strong&gt;: two or three dimensions are both fine for the purposes of my simulations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am already aware of a number of pieces of deterministic MD software (ex. LAMMPS, NAMb, VMD, etc) but as far as I can tell, none of these incorporate the second point I want addressed and I'm loath to make any serious modifications to someone else's code as I want to get onto running simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your time.&lt;/p&gt;&#xA;" OwnerUserId="5412" OwnerDisplayName="WizzPhiz" LastActivityDate="2013-10-12T08:24:57.840" Title="Non-deterministic/probabilistic molecular dynamics simulations" Tags="|simulation|software|molecular-dynamics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8697" PostTypeId="2" ParentId="8046" CreationDate="2013-10-06T05:26:37.220" Score="2" Body="&lt;p&gt;There is a lot of confusion in your question. The most important for me is that you miss that &quot;naive&quot; QMC which is Monte-Carlo calculation of integrals in some variational method and diffusion Monte-Carlo are different methods with different argumentation and derivation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main point however is about imaginary time. In diffusion Monte-Carlo imaginary time is a trick to convert time-independent Schroedinger equation into time-dependent diffusion-like equation which solution in the infinite &quot;time&quot; limit tends to a solution of original Schroedinger equation. That's it. Time in DQMC is not real. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relatively good but simple explanation is given in &lt;a href=&quot;http://rmp.aps.org/abstract/RMP/v73/i1/p33_1&quot; rel=&quot;nofollow&quot;&gt;Reviews of Modern Physics, 73, 33(2001)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. By the way, what you mean by &quot;Trotter approximation&quot; in your question?&lt;/p&gt;&#xA;" OwnerUserId="949" LastActivityDate="2013-10-06T05:26:37.220" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8698" PostTypeId="1" CreationDate="2013-10-06T13:34:44.643" Score="9" ViewCount="338" Body="&lt;p&gt;Consider the strictly convex unconstrained optimization problem $\mathcal{O} := \min_{x \in \mathbb{R}^n} f(x).$ Let $x_\text{opt}$ denote its unique minima and $x_0$ be a given initial approximation to $x_\text{opt}.$We will call a vector $x$ an $\epsilon-$ close solution of $\mathcal{O}$ if &#xA;\begin{equation}&#xA;\frac{||x - x_{\text{opt}}||_2}{||x_0 - x_\text{opt}||_2} \leq \epsilon.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that there exists two iterative algorithms $\mathcal{A}_1$ and $\mathcal{A}_2$ to find an $\epsilon-$ close solution of $\mathcal{O}$ with the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For any $\epsilon &amp;gt; 0,$ the total computational effort, i.e. effort required per iteration $\times$ total number of iterations, to find an $\epsilon-$ close solution is same for both the algorithms.&lt;/li&gt;&#xA;&lt;li&gt;The per iteration effort for $\mathcal{A}_1$ is $O(n),$ say, while that of $\mathcal{A}_2$ is $O(n^2).$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are there situations, where one would prefer one algorithm over the other? Why?&lt;/p&gt;&#xA;" OwnerUserId="5316" LastActivityDate="2013-10-07T04:15:46.347" Title="Computation Effort of Algorithms" Tags="|convex-optimization|complexity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8699" PostTypeId="2" ParentId="8698" CreationDate="2013-10-06T14:00:51.040" Score="7" Body="&lt;p&gt;I can think of some possibilities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If both algorithms monotonically reduce the error with each iteration, then it might be preferable to some to have more, cheaper iterations since it gives you more choices about when to stop iterating. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\mathcal{A}_1$ is $O(n)$ work and time but $O(n^k)$ memory, you might prefer $\mathcal{A}_2$ if $k$ is large. $k=2$ might even be large enough to make you select $\mathcal{A}_2$ since memory use is more likely to constrain you here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This probably applies whether we're talking about optimization or any other class of iterative problem. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-10-06T14:00:51.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8700" PostTypeId="2" ParentId="8698" CreationDate="2013-10-06T16:42:04.810" Score="14" Body="&lt;p&gt;It's typically very hard if not impossible to implement a parallel version of an iterative algorithm that paralellizes across iterations.  The completion of one iteration is a natural sequence point.  If one algorithm requires fewer iterations but more work per iteration, then it's more likely that this algorithm can be effectively implemented in parallel.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of this is linear programming, where the primal-dual barrier (interior point) method typically uses only a few dozen iterations even for very large problems, but the work per iteration is quite extensive.  In comparison various versions of the simplex method typically require far more iterations, but the work per iteration is less.  In practice, parallel implementations of interior point methods have shown far better parallel efficiency than parallel implementations of the simplex method.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2013-10-07T04:15:46.347" LastActivityDate="2013-10-07T04:15:46.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8701" PostTypeId="1" CreationDate="2013-10-06T17:26:40.813" Score="0" ViewCount="2278" Body="&lt;p&gt;What do you actually type into the *ORIENTATION entry in an input file?&#xA;Is it a rotation matrix w.r.t. the Global axes?&lt;/p&gt;&#xA;" OwnerUserId="5317" LastEditorUserId="1744" LastEditDate="2013-10-09T21:27:34.883" LastActivityDate="2014-02-07T00:19:29.747" Title="Abaqus *ORIENTATION" Tags="|finite-element|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8702" PostTypeId="2" ParentId="8693" CreationDate="2013-10-06T17:47:39.607" Score="5" Body="&lt;p&gt;You haven't specified the distribution of $x(t)$.  I'll assume that you want to use a complex normal distribution, since that choice makes it reasonably easy to solve the problem and because this assumption is quite common in signal processing.   I'll also discretize the problem so that you're generating a vector $X$ of $N$ entries with a specified complex normal distribution (your computer doesn't have enough storage for an infinite number of values.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the Wikipedia entry &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Complex_normal_distribution&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Complex_normal_distribution&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complex normal distribution is characterized by its mean $\mu$, covariance matrix $\Gamma$, and relation $C$, where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mu=E[X]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Gamma=E[(X-\mu)(X-\mu)^{H}]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C=E[(X-\mu)(X-\mu)^{T}]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $\Gamma$ is Hermitian and must be positive semidefinite, while $C$ is symmetric.  There are further restrictions on $\Gamma$ and $C$ that ensure that some of the calculations below work out.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also common to have $C=0$.  In this case we have a &quot;circularly symmetric&quot; distribution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Write $X$ as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X=\mu + U+ iV$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mu=E[X]$ and where $U$ and $V$ are jointly distributed real multivariate normal random vectors with mean 0 and covariance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{Cov}\left(\left[&#xA;\begin{array}{c}&#xA;U \\&#xA;V&#xA;\end{array}&#xA;\right]\right)=\Sigma=\left[&#xA;\begin{array}{cc}&#xA;\Sigma_{UU} &amp;amp; \Sigma_{UV} \\&#xA;\Sigma_{VU} &amp;amp;  \Sigma_{VV}&#xA;\end{array}&#xA;\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma_{UU}=Re(\Gamma+C)/2 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma_{UV}=Im(-\Gamma+C)/2 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma_{VU}=Im(\Gamma+C)/2 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma_{VV}=Re(\Gamma-C)/2 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we've reduced your original problem to the problem of generating real multivariate normal vectors of length $2N$, with mean 0 and covariance $\Sigma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many techniques for generating such real multivariate normal random numbers.  These are discussed in many textbooks on Monte Carlo simulation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One very simple technique that works well as long as $N$ is reasonably small is to compute the Cholesky factorization of $\Sigma$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma=R^{T}R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then generate a $N(0,I)$ random vector $Z$, and let $W=R^{T}Z$.  $W$ will also be multivariate normal with mean 0.  By the formula for the covariance of a matrix times a random vector, we get that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{Cov}(W)=R^{T}\mbox{Cov}(Z)R=R^{T}IR=\Sigma$.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus $W$ has the desired multivariate normal distribution, and we can construct $X$ as $X=\mu+W_{U}+iW_{V}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The big problem with this technique is that $N$ gets large, it may not be possible to store the $2N$ by $2N$ matrix $\Sigma$ or its Cholesky factor.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your autocovariance matrices have Toeplitz structure, it turns out that there are specialized spectral techniques that can be used to reduce the required storage to $O(N)$ rather than $O(N^2)$.   Spectral methods for generation of multivariate normals with specified stationary autocorrelation are discussed in many textbooks on geostatistics.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2013-10-07T01:21:50.463" LastActivityDate="2013-10-07T01:21:50.463" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8704" PostTypeId="1" CreationDate="2013-10-07T14:27:46.617" Score="5" ViewCount="258" Body="&lt;p&gt;I am trying to solve a &quot;Navier-Stokes&quot;-type problem where the viscosity is negative. Of course this renders the equation unstable and thus I add a fourth order term, so the entire equation becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dv}{dt} + (v\cdot\nabla)v = - \nabla p+\nu_1 \nabla^2 v - \nu_2 \nabla^4 v&#xA;$$&#xA;with $\nu_1&amp;lt;0, \nu_2&amp;gt;0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the fourth order term should stabilize the solution with negative viscosity - I can show this by stability analysis - but I am not able to solve it numerically. I am using the finite element code Fenics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am able to solve the system with $\nu_1&amp;gt;0, \nu_2 = 0$ using a simple Picard technique, see e.g. &lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/dev/python/demo/pde/navier-stokes/python/documentation.html&quot; rel=&quot;nofollow&quot;&gt;http://fenicsproject.org/documentation/dolfin/dev/python/demo/pde/navier-stokes/python/documentation.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am not able to make the system stable with negative viscosity. I believe this is a problem with my Finite element approach, but I have no idea how to fix it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit in response to comments: My implmentation is done via fenics. I have tried with both a Lagrange and a CG space. I've done only very simple implementations. I've tried doing a the fourth order derivative on the previous time step's solution. I've also tried doing partial integration twice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By stable I mean that the field diverges. The velocity field quickly becomes infinite - just as it would if $\nu_2$ is set to zero. It basically behaves just as a Navier Stokes system with negative viscosity and not stabilized by the fourth order term.&lt;/p&gt;&#xA;" OwnerUserId="5325" LastEditorUserId="-1" LastEditDate="2013-10-08T20:24:32.677" LastActivityDate="2013-10-08T20:24:32.677" Title="Negative viscosity stabilized by fourth order terms" Tags="|pde|finite-element|fluid-dynamics|python|numerical-analysis|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8705" PostTypeId="1" CreationDate="2013-10-07T15:09:10.893" Score="3" ViewCount="379" Body="&lt;p&gt;I am currently fitting above mentioned functions to my data and I can observe, that both lognormal and Weibull are better fits than powerlaw. In literature, it is often suggested, that it is hard to distinguish these functions, but I can in my data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not completely understand the differences between them though. What might be an explanation for the underlying process that generates my data that I can infer from these results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope someone can help me.&lt;/p&gt;&#xA;" OwnerUserId="4527" LastActivityDate="2013-10-08T18:51:59.363" Title="Difference powerlaw, lognormal and streteched exponential (Weibull) function" Tags="|statistics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8706" PostTypeId="1" CreationDate="2013-10-07T18:35:26.890" Score="9" ViewCount="1044" Body="&lt;p&gt;I've inherited a finite volume code that does a second-order discretization of flux terms for a set of mixed parabolic-elliptic equations with discontinuous diffusion coefficients. The impression I get is that Discontinuous Galerkin (DG) methods are preferable for this sort of problem; however, since I have some time constraints, and know more about finite volume methods than any type of finite element methods (or DG), for now, I'm forging ahead with finite volume methods. As a disclaimer, I don't know a lot about the finite volume method, either (I've read up to Chapter 8 in LeVeque's Finite Volume Methods for Hyperbolic Problems on my own), so feel free to correct any misconceptions I have here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since my colleagues will be interested in unsteady problems, and results will be compared against previous work in more regular geometries that used the method of lines with high order temporal discretizations, I would like to investigate using higher order spatial discretizations to enable the use of higher order time integrators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good textbook-like or tutorial-like source that someone can recommend on ENO/WENO methods that covers limiters in more than one dimension? The internal interfaces between materials will create, in essence, transmission boundary conditions that I don't want to handle explicitly in the code. These interfaces will probably cause oscillations that will need to be addressed by a limiter. The best source I've seen is &lt;a href=&quot;http://math.la.asu.edu/~gardner/ENO.pdf&quot; rel=&quot;noreferrer&quot;&gt;Essentially Non-oscillatory and Weighted Essentially Non-Oscillatory Schemes for Hyperbolic Conservation Laws by C. W. Shu&lt;/a&gt;; the book by LeVeque seems to focus mostly on Lax-Wendroff schemes from a wave propagation standpoint (which isn't being used in the code), and the book by Toro seems to focus on higher order schemes in the one-dimensional case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a follow-up question: given that the problem is mainly diffusive, is it particularly important to use an strong stability-preserving (SSP) integrator for this type of problem? I know that for hyperbolic problems employing higher-order discretizations, preserving the total variation diminishing property is important, which is why SSP integrators are used. This problem isn't hyperbolic, so the only place where being total variation bounded might actually matter will be any spurious oscillations that arise as a result of the interfaces between materials of different diffusivities.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-10-08T02:21:04.230" LastActivityDate="2013-10-08T03:46:24.377" Title="Is there a good tutorial or textbook-like source on implementing ENO/WENO with limiters in one (and more than one) dimension?" Tags="|pde|finite-volume|parabolic-pde|elliptic-pde|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8707" PostTypeId="1" CreationDate="2013-10-04T09:21:54.817" Score="0" ViewCount="138" Body="&lt;p&gt;Say I'd like to implement a numerical model of a physical system (e.g. a simple wave equation).&#xA;Of course the &lt;a href=&quot;https://en.wikipedia.org/wiki/Conservation_of_energy&quot; rel=&quot;nofollow&quot;&gt;energy conservation law&lt;/a&gt; is valid for this system.&#xA;Is there a way to validate that my implementation of the wave equation is implemented in accordance to the law of energy conservation in general?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not want to implement some test scenarios and check whether the energy is constant at all time. Instead I'd like to use some sort of symbolical solver to verify this property of my implementation. Something like &lt;a href=&quot;http://corp.galois.com/blog/2013/9/11/high-assurance-base64.html&quot; rel=&quot;nofollow&quot;&gt;Cryptol&lt;/a&gt; but instead for physics and of course GPL.&lt;/p&gt;&#xA;" OwnerDisplayName="Manuel" LastEditorUserId="6663" LastEditDate="2013-12-26T16:46:40.297" LastActivityDate="2013-12-28T08:13:21.753" Title="Validating conservation laws from physics" Tags="|computational-physics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8708" PostTypeId="2" ParentId="8707" CreationDate="2013-10-07T23:53:30.020" Score="5" Body="&lt;p&gt;The typical way ODE and PDE numerical methods are tested is using something like the method of &lt;a href=&quot;http://www.personal.psu.edu/faculty/j/h/jhm/ME540/lectures/VandV/MMS_summary.pdf&quot; rel=&quot;noreferrer&quot;&gt;manufactured solutions&lt;/a&gt;; a more extensive report on validating large physics codes can be found &lt;a href=&quot;http://prod.sandia.gov/techlib/access-control.cgi/2000/001444.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an ODE solver, the basic idea for testing a method for solving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\dot{u}(t) = f(u(t))&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;would be to pick a solution, say $u(t) = e^{-t}$, and then calculate what the source term $f$ should be using a symbolic algebra system. For this example, $f(u) = -e^{-t} = -u$. You can also calculate initial conditions and boundary conditions by direct substitution. Here, $u(0) = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are GPL- and BSD-licensed symbolic manipulation programs out there that will take care of the algebra for you. In Python, for instance, there &lt;a href=&quot;http://www.sagemath.org/&quot; rel=&quot;noreferrer&quot;&gt;Sage&lt;/a&gt; and &lt;a href=&quot;http://sympy.org/en/index.html&quot; rel=&quot;noreferrer&quot;&gt;Sympy&lt;/a&gt; can each do symbolic manipulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though you say you'd prefer not to, testing explicitly for things like symmetry or energy conservation (to within a tolerance) is something also done in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternative approaches exist, but they're typically impractical. For instance, you could use a computer assisted proof system like HOL Light or Coq to reason about your program, but the computer assisted proof would take an impractically long time to execute. The Flyspeck project aims to do a computer-assisted proof of Kepler's conjecture, and the computation is estimated to take roughly 2.5 years, which is longer in serial than some &quot;hero&quot; computations in high-performance computing. You could also use something like interval arithmetic to ensure that your results lie within a given range, but things like dependency effects would likely make the output intervals very large relative to the precision you want, and you could suffer from a lack of infrastructure (i.e., there aren't many libraries that do interval arithmetic, and these libraries don't necessarily implement enough of what an average nonspecialist might need to do, like, say, solve systems of nonlinear equations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I gather from your remark about Cryptol that you might want to do some sort of computer-assisted proof. Based on the above, I wouldn't recommend it, but if it could be done efficiently, that would be a very interesting contribution to the numerical methods literature.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-10-07T23:53:30.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8709" PostTypeId="2" ParentId="8706" CreationDate="2013-10-08T02:27:19.813" Score="5" Body="&lt;p&gt;The sources you are looking at are all looking at hyperbolic problems.  The issues are different for elliptic problems and &quot;limiters&quot; are generally not the preferred tool.  I outlined some of the methods and tradeoffs in &lt;a href=&quot;https://scicomp.stackexchange.com/a/6712/119&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for time integration, $L$-stability is the important property to prevent bad overshoots for parabolic systems.  Integrators that are not $L$-stable (such as Crank-Nicolson) will show spurious oscillations even on smooth problems, so long as you start with a steep initial value.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-10-08T02:27:19.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8710" PostTypeId="2" ParentId="8706" CreationDate="2013-10-08T03:46:24.377" Score="5" Body="&lt;p&gt;As Jed says, limiters are not usually an efficient approach for parabolic/elliptic problems.  WENO is &lt;strong&gt;much&lt;/strong&gt; more expensive than simple piecewise-polynomial interpolation, so I would first try vanilla interpolation and see if you actually have oscillations.  WENO is really designed for situations in which the solution is discontinuous; yours is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case someone interested in hyperbolic PDEs reads this question, I would say that the report you have linked is the most practical introduction to WENO, and better than Shu's more recent review paper in terms of helping one to implement it.  Another useful tool for understanding and implementing WENO is &lt;a href=&quot;http://pyweno.readthedocs.org/&quot;&gt;Matthew Emmett's PyWENO code&lt;/a&gt;, which can generate routines in other languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a multidimensional finite volume implementation, Shu's report suggests doing expensive multidimensional quadratures, but I would not go that route.  The first option is to just implement dimension-by-dimension WENO (as if for a finite difference scheme), which will be formally second order but &lt;a href=&quot;http://www.dam.brown.edu/scicomp/scg-media/report_files/BrownSC-2009-42.pdf&quot;&gt;can still give high-order-like resolution&lt;/a&gt;.  Better still, there is a new technique for recovering full accuracy without multidimensional quadratures, which is forthcoming work by Christiane Helzel.  I will add a link here when it appears.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2013-10-08T03:46:24.377" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8711" PostTypeId="1" AcceptedAnswerId="8712" CreationDate="2013-10-08T04:15:02.467" Score="8" ViewCount="161" Body="&lt;p&gt;In many areas of application, one needs to solve a nonlinear system of equations&#xA;$$&#xA;F(x) = 0.&#xA;$$&#xA;Sometimes, the formulation&#xA;$$&#xA;\|F(x)\|^2 \to\min&#xA;$$&#xA;is used. Clearly, every solution $\hat{x}$ of $F(x)=0$ is also a solution of the second problem; the converse is also true (if a solution exists).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is if one can tell a-priori which formulation is better suited for a given problem. Have people worked on this before?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;One example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Consider the function&#xA;$$&#xA;F(x, y) =&#xA;\begin{pmatrix}&#xA;x^3 - 3x y^2 - 1\\&#xA;3 x^2 y - y^3&#xA;\end{pmatrix}.&#xA;$$&#xA;It has the three roots $x_1=(1,0)$ (green in the figure below), $x_2=(-0.5,\sqrt{3}/2)$ (blue), $x_3=(-0.5,-\sqrt{3}/2)$ (red). When applying Newton's method to $F$, the starting point will determine to which of the three solutions we converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/C3n7M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The darker the color, the more Newton iterations were required. The typical &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton_fractal&quot; rel=&quot;noreferrer&quot;&gt;Newton fractals&lt;/a&gt; appear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When finding critial points $\nabla (\|F(x)\|^2) = 0$, again with Newton's method, the picture is a little different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/t1GMC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the point $(0,0)$ is a critical point of $\|F(x)\|^2$, but no solution of $F(x)=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This highlights one possible problem with the $\min$-formulation.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2013-10-08T07:53:42.937" LastActivityDate="2013-10-08T07:53:42.937" Title="F(x) = 0 vs. ||F(x)||^2-&gt;min" Tags="|reference-request|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8712" PostTypeId="2" ParentId="8711" CreationDate="2013-10-08T04:36:51.130" Score="4" Body="&lt;p&gt;You used nice graphics in the question, but I think I answered the question fairly clearly in &lt;a href=&quot;https://scicomp.stackexchange.com/a/2446/119&quot;&gt;this answer&lt;/a&gt;, which contains another worked example.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;To summarize, we started with an optimization problem that had a unique solution that we could guarantee that a method would find. We reformulated as a nonlinear root finding problem that had a unique solution that we could identify locally, but a rootfinding method (like Newton) might stagnate before reaching it. We then reformulated the root finding problem as an optimization problem that had multiple local solutions (no local measure can be used to identify that we are not at the global minimum).&lt;/p&gt;&#xA;&lt;p&gt;In general, each time we convert a problem from optimization to rootfinding or vice-versa, we make the available methods and associated convergence guarantees weaker. The actual mechanics of the methods are often very similar so it is possible to reuse a lot of code between nonlinear solvers and optimization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Feel free to refine your question if you meant to ask something more specific.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2013-10-08T04:36:51.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8713" PostTypeId="1" CreationDate="2013-10-08T12:31:17.717" Score="1" ViewCount="77" Body="&lt;p&gt;As far as I understand, &lt;a href=&quot;http://neuralensemble.org/sumatra/&quot; rel=&quot;nofollow&quot;&gt;Sumatra&lt;/a&gt; does some sort of versioning of simulation scripts and output files. If I start to use Sumatra inside a git working directory, can Sumatra end up interfering with git or vice versa?&lt;/p&gt;&#xA;" OwnerUserId="3724" LastActivityDate="2013-10-08T12:31:17.717" Title="Will Sumatra interfere with git?" Tags="|hpc|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8714" PostTypeId="2" ParentId="8705" CreationDate="2013-10-08T18:51:59.363" Score="2" Body="&lt;p&gt;See &lt;a href=&quot;http://www.uvm.edu/~pdodds/teaching/courses/2013-01UVM-300/docs/2013-01UVM-300lognormals-flat.pdf&quot; rel=&quot;nofollow&quot;&gt;these lecture notes&lt;/a&gt; from the complex systems course taught by Prof. Peter Dodds.  It has some great links to literature, etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a nutshell, random multiplicative growth can lead to lognormal distributions and there are certain things one can enforce within the dataset (minimum number of occurrences among other things, see some of this wonderful literature in the references!) that can lead to a power-law distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned by AJK, this can be hairy business and as you probably know there are entire research groups (generally categorized as complex systems groups) that are dedicated to determining what gives rise to different distributions and how to classify them.  There is a rich history here.&lt;/p&gt;&#xA;" OwnerUserId="5279" LastActivityDate="2013-10-08T18:51:59.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8715" PostTypeId="1" AcceptedAnswerId="8716" CreationDate="2013-10-09T03:27:13.063" Score="0" ViewCount="588" Body="&lt;p&gt;I want to use the quad function to do the integration today. I wrote a function file:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;function z=pdf_y1(x)&#xA;    z=100;&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And then, in the command line, I type:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;quad('pdf_y1',0,0.5)&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The output is:&#xA;??? Error using ==&gt; quad at 79&#xA;The integrand function must return an output vector of the same length as the input vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried z=2*x. It works. I don't know why quad is not working for constant case. I actually want to use it on some other functions involving if-else statements. But it keeps showing the same message: The integrand function must return an output vector of the same length as the input vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know why. Could somebody help me? Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="5335" LastActivityDate="2013-10-09T14:16:28.860" Title="MATLAB quad function not work for constant function?" Tags="|matlab|" AnswerCount="1" CommentCount="0" ClosedDate="2013-12-02T22:09:42.647" ContentLicense="CC BY-SA 3.0" />
  <row Id="8716" PostTypeId="2" ParentId="8715" CreationDate="2013-10-09T03:30:27.077" Score="2" Body="&lt;p&gt;Try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function z=pdf_y1(x) z=100*ones(size(x))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;MATLAB requires the output to be the same shape/dimension as the input for functions passed to &lt;code&gt;quad()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-10-09T14:16:28.860" LastActivityDate="2013-10-09T14:16:28.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8717" PostTypeId="1" CreationDate="2013-10-09T07:48:10.953" Score="5" ViewCount="3617" Body="&lt;p&gt;I am trying to solve 1D viscous Burger's equation numerically and I cannot apply von Neumann analysis because the equation is non-linear.&#xA;How do I predict the stability criteria for my system?&#xA;I also need to predict the criteria when its inviscid(which basically makes it non-linear advection equation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FD schemes that I am using are 1st order FT for time, 1st order BS for the advection term and 2nd order CS for diffusion term.&#xA;Though I know there are better FD schemes, I still want to analyze stability of this scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;br&gt;&#xA;Forgot to include initial and boundary condition. Everything is so meaningless without it..&lt;br&gt;&#xA;The domain of flow is between -2&amp;lt;=x&amp;lt;=8&lt;br&gt;&#xA;Initial conditions are such that 1&amp;lt;=u&amp;lt;=2 for the entire flow domain&lt;br&gt;&#xA;Boundary conditions define constant positive velocity at left and right boundaries.&lt;/p&gt;&#xA;" OwnerUserId="5336" LastEditorUserId="5336" LastEditDate="2013-10-10T07:16:34.863" LastActivityDate="2013-12-09T10:11:10.377" Title="Stability of numerical method for 1D Burger's equation" Tags="|pde|finite-difference|fluid-dynamics|stability|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8718" PostTypeId="1" CreationDate="2013-10-09T09:41:48.973" Score="1" ViewCount="405" Body="&lt;p&gt;Below is a totally made up example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rRNr8.jpg&quot; alt=&quot;Example&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's say on the left we have a weird black-white image or, in other words, a matrix of zeros and ones. We then apply a specific algorithm to the given matrix. The result is shown on the right - it is still represented by a pretty weird image but this time a circle can be visually distinguishable in our outcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that the circle (or any other simple figure for that matter) was planned to appear there all along. Yet that same figure was not visually obvious to be there in the beginning.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How could I realistically recreate a similar result? Maybe some have seen sources referring to like-minded ideas? Essentially I'd need a simple model which relies on underlying patterns that are not (easily) visually distinguishable.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that visual cryptography is almost synonymous with having multiple layers which when properly combined reveal the result. I'd love to go with a single layer even if it is not safe in terms of hiding information. I was thinking about functions and subtle algorithms (such as cellular automata) which potentially could be used in order to recalculate every element of a matrix but no luck so far.&lt;/p&gt;&#xA;" OwnerUserId="4491" LastActivityDate="2014-04-08T13:53:47.060" Title="How to produce visually unexpected results?" Tags="|algorithms|matrix|reference-request|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8720" PostTypeId="2" ParentId="8718" CreationDate="2013-10-09T16:14:21.237" Score="0" Body="&lt;p&gt;This does not address any specific algorithm that would accomplish something of this nature, but as far as tools, I would use Python.  See &lt;a href=&quot;http://scipy-lectures.github.io/advanced/image_processing/&quot; rel=&quot;nofollow&quot;&gt;this Numpy/Scipy&lt;/a&gt; documentation on how to process images with Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a few short lines of code, you can load in an image into numpy data structures, which can then have mathematical operations done on them and be plotted again as images.  I have used this technique for various tasks, but never the specific one you have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would imagine that if you start playing around with this a little then you could come up with some sort of algorithm (probably basic linear algebraic operations on the numpy arrays) that would achieve this result you are looking for.&lt;/p&gt;&#xA;" OwnerUserId="5279" LastActivityDate="2013-10-09T16:14:21.237" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8722" PostTypeId="2" ParentId="8718" CreationDate="2013-10-09T16:44:33.487" Score="0" Body="&lt;p&gt;I'm not sure if you're aware of the work on &lt;a href=&quot;http://graphics.stanford.edu/~niloy/research/emergence/emergence_image_siga_09.html&quot; rel=&quot;nofollow&quot;&gt;Emerging Images&lt;/a&gt;. It has some of the characteristics of what you want, although it doesn't seem to completely answer your question.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-09T16:44:33.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8723" PostTypeId="2" ParentId="8129" CreationDate="2013-10-09T20:56:31.373" Score="0" Body="&lt;p&gt;Reproducibility of the trajectory is generally impossible; even different compiler versions on the same code implementing an integrator will not reproduce each other's first step - chaotic divergence follows. Reproducibility of the resulting thermodynamic observables is a much more relevant and achievable criterion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One simple approach is to recognize that the distribution of (say) potential energies over the trajectory is known a priori, and varies in a known way with (say) temperature. A statistically significant observation that the same (and the expected) temperature dependence of the distribution is observed with the two integrators is an excellent start to demonstrating their equivalence. See &lt;a href=&quot;http://dx.doi.org/10.1021/ct300688p&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1021/ct300688p&lt;/a&gt; (article also available on arxiv.org; code on &lt;a href=&quot;https://simtk.org/home/checkensemble&quot; rel=&quot;nofollow&quot;&gt;https://simtk.org/home/checkensemble&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2013-10-09T20:56:31.373" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8724" PostTypeId="2" ParentId="8297" CreationDate="2013-10-09T21:06:25.790" Score="1" Body="&lt;p&gt;Presupposing an observable of interest, simulate over different finite size scales (with whatever initial and boundary conditions interest you), and demonstrate that the size scale affects the values of the estimate of the observable (taking care to converge the estimate with respect to sampling). In the limit of a sufficiently large size scale, you can see the estimate of the observable approach the true value for that model, and thus measure the severity of finite-size effects.&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2013-10-09T21:06:25.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8725" PostTypeId="2" ParentId="8178" CreationDate="2013-10-09T21:09:25.350" Score="2" Body="&lt;p&gt;One simple approach is to recognize that the distribution of (say) potential energies over the trajectory is known a priori, and varies in a known way with (say) temperature. A statistically significant observation that the same (and the expected) temperature dependence of the distribution is observed with the two integrators is an excellent start to demonstrating their equivalence. See &lt;a href=&quot;http://dx.doi.org/10.1021/ct300688p&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1021/ct300688p&lt;/a&gt; (article also available on arxiv.org; code on &lt;a href=&quot;https://simtk.org/home/checkensemble&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://simtk.org/home/checkensemble&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also discussion at &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8129/reproducibility-in-molecular-dynamics-trajectories&quot;&gt;Reproducibility in molecular dynamics trajectories&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3649" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-10-09T21:09:25.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8726" PostTypeId="2" ParentId="8701" CreationDate="2013-10-09T21:24:58.857" Score="1" Body="&lt;p&gt;The meaning of the *ORIENTATION keyword in Abaqus can be checked in the &quot;Abaqus Keyword Reference Manual&quot;, which unfortunately is not made publicly available by Simulia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As most Abaqus keywords, *ORIENTATION has a lot of optional parameters, which change its meaning and syntax. In most cases however the lines following this keyword are the $X, Y, Z$ coordinates of three points $a,b,c$: $c$ (optional if coincident with the global axes origin,i.e. $(0,0,0)$) is the origin, $a$ lies on the $x$ axis, $b$ is a point on the $xy$ plane.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-10-09T21:24:58.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8727" PostTypeId="2" ParentId="2187" CreationDate="2013-10-09T21:36:23.973" Score="5" Body="&lt;p&gt;By &quot;Domain decomposition is a better choice only when linear system size considerably exceeds the range of interaction, which is seldom the case in molecular dynamics&quot; the authors of that (very old) GROMACS paper mean that if the spatial size of the neighbour list is of the order of 1 nm, and the simulation cell is only several nanometers, then the overhead from doing domain decomposition is too high. You may as well accept an all-to-all information distribution in particle decomposition, and not need to spend time on all the book-keeping for domain decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with particle decomposition as GROMACS implemented it was that over time the particles assigned to each processor diffuse through space. Since responsibility for computing each interaction was fixed by their initial location, the diffusion gradually increased the volume of the total space each processor needed to know in order to build its neighbour list, even if the total computation described by the neighbour list was constant. In practice, you would periodically re-start the simulation to reset the data and communication locality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your supposition that &quot;particle decomposition has the advantage that one does not have to deal with particles moving across domain boundaries&quot; does not hold if diffusion is significant over the time scale of the simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Domain decomposition deals with this &quot;up front&quot; by migrating responsibility for the interaction along with the diffusion, thereby improving data locality on each processor, and minimizing communication volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I help develop GROMACS, and will probably rip out the particle decomposition implementation next week ;-)&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2013-10-09T21:36:23.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8728" PostTypeId="2" ParentId="2168" CreationDate="2013-10-09T22:13:37.390" Score="0" Body="&lt;p&gt;There can be performance enhancements, but first one should profile to know that computing the reciprocal of the sqrt is the bottle-neck (and not, say, loading the positions and saving the forces).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GROMACS MD project grew out of an idea to exploit the details of the IEEE floating-point format to seed a Newton-Raphson iteration scheme for computing an acceptable approximation to the reciprocal of the square root (see Appendix B.3 of &lt;a href=&quot;http://www.gromacs.org/Documentation/Manual&quot; rel=&quot;nofollow&quot;&gt;http://www.gromacs.org/Documentation/Manual&lt;/a&gt;), but there are no HPC CPUs in use where GROMACS still uses this idea.&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2013-10-09T22:13:37.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8729" PostTypeId="2" ParentId="6853" CreationDate="2013-10-09T23:02:45.653" Score="2" Body="&lt;p&gt;You should also read the follow-up post from the GROMACS mailing list: &lt;a href=&quot;http://lists.gromacs.org/pipermail/gmx-users/2013-April/080604.html&quot; rel=&quot;nofollow&quot;&gt;http://lists.gromacs.org/pipermail/gmx-users/2013-April/080604.html&lt;/a&gt;. Whether or not the GPU implementation in the MD supports the use of double precision to a comparable extent is at least as important as whether double precision is available on the hardware.&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2013-10-09T23:02:45.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8731" PostTypeId="2" ParentId="8717" CreationDate="2013-10-10T06:01:32.903" Score="5" Body="&lt;p&gt;A useful rule of thumb (though it is not always a sufficient condition for stability) is stability of the linearized scheme.  Since you have a method of lines discretization, you can think of this geometrically as the condition that the eigenvalues of the jacobian of your spatial discretization, multiplied by the time step size, lie inside the region of absolute stability of your time discretization.  In your case, this will give you the same condition that you would have for stability of an advection-diffusion problem, but with the advection speed given by the largest value of the initial data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, your backward difference in space will only be stable if the initial data is non-negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sufficient conditions for stability of discretizations of nonlinear PDEs, see &lt;a href=&quot;http://dx.doi.org/10.1007/BF01386051&quot; rel=&quot;noreferrer&quot;&gt;Strang's paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the inviscid case, it's well known (and follows from the reasoning above) that this scheme is stable for CFL number $\le 1$.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2013-10-10T09:47:57.920" LastActivityDate="2013-10-10T09:47:57.920" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8732" PostTypeId="1" CreationDate="2013-10-10T08:05:45.283" Score="11" ViewCount="2244" Body="&lt;p&gt;I have a question concerning coding boundary conditions for solid mechanics (linear elasticity). In the special case I have to use finite differences (3D).&#xA;I am very new to this topic, so perhaps some of the following questions may be very basic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To lead to my specific problem, first of all I want to show what I already implemented (To keep it clear, I will only use 2D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) I have the following discretization of &lt;span class=&quot;math-container&quot;&gt;$div(\sigma) = 0$&lt;/span&gt;, showing the first component of the divergence &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial\sigma_{xx}}{\partial x} + \frac{\partial\sigma_{xy}}{\partial y} = 0$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bYiAj.png&quot; alt=&quot;disc&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a non-staggered grid, so Ux and Uy are defined at the same place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) The next step was to treat the boundaries, where I use &quot;ghost nodes&quot;. According to &lt;span class=&quot;math-container&quot;&gt;$\sigma \bullet n = t^*$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$t^*$&lt;/span&gt; is the stress on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xbt84.png&quot; alt=&quot;boundary&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Here I use &lt;span class=&quot;math-container&quot;&gt;$(\lambda + 2\mu)\frac{\partial U_x}{\partial x} + \lambda \frac{\partial U_y}{\partial y} = \sigma_{xx}^*$&lt;/span&gt; to obtain Ux at the ghost point as all other values of Ux and Uy are given (inside the body). &lt;span class=&quot;math-container&quot;&gt;$\sigma_{xx}^*$&lt;/span&gt; is the value of this stress on the boundary (normally zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Same procedure, only via &lt;span class=&quot;math-container&quot;&gt;$\mu\frac{\partial U_x}{\partial y} + \mu \frac{\partial U_y}{\partial x} = \sigma_{xy}^*$&lt;/span&gt; I obtain Uy at the ghost point. Again &lt;span class=&quot;math-container&quot;&gt;$\sigma_{xy}^*$&lt;/span&gt; is the value of this stress on the boundary (normally zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.) I think until now all my steps seem to be logic, if not, &lt;strong&gt;please correct me&lt;/strong&gt;.&#xA;But now there are also the &quot;corner nodes&quot;, where I don't have a clue how to handle them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ax6cd.png&quot; alt=&quot;Corner&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To keep my scheme for &lt;span class=&quot;math-container&quot;&gt;$div(\sigma) = 0$&lt;/span&gt; working on the corner node, I need Ux and Uy at the node in the lower left. But here my previously procedure like in 2.) doesn't work, as the node is not orthogonal to the boundary.&#xA;I already tried to extrapolate the displacements, but it seems that this will result in stability problems (I am solving the whole problem implicit with an iterative solver).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is what's the correct way to handle these &quot;corner nodes&quot;? I am happy for every idea.&lt;/p&gt;&#xA;" OwnerUserId="5349" LastEditorUserId="22284" LastEditDate="2019-01-07T10:46:27.937" LastActivityDate="2019-01-07T10:46:27.937" Title="Solid mechanics with finite differences: How to handle &quot;corner nodes&quot;?" Tags="|finite-difference|boundary-conditions|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8734" PostTypeId="1" AcceptedAnswerId="8736" CreationDate="2013-10-10T21:55:25.410" Score="6" ViewCount="484" Body="&lt;p&gt;CS people:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The title is the question, as I will explain.  As everyone reading this probably knows, if $A$ is a square matrix of real or complex numbers, then $e^A$, or $\exp(A)$, is the matrix of the same dimensions as $A$ defined by the obvious power series (the only thing that might not be obvious is that $A^0 =I$, but one would have to be very contrary not to accept this definition).  One could use fields, rings, or other algebraic structures (such as quaterions) rather than $\mathbb{R}$ or $\mathbb{C}$, but I would guess that the vast majority of people who work with matrices use real or complex matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This spring I taught a linear algebra class using a textbook that did not define the exponential of a matrix or mention it at all.  I taught the topic anyway, using another text as a resource.  The textbook lacking the matrix exponential seems like a popular, mainstream textbook, it contains a lot of material at least as difficult as the matrix exponential, and the author probably knows a lot more about linear algebra than I do.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The total absence of the exponential of a matrix seemed to me like a major flaw of this book. In pure mathematics, any definition that is nontrivial and does not lead to a contradiction is important to someone.  I know that some matrix Lie groups are important in quantum mechanics.  So my question is, how important is the matrix exponential in computational science?   &lt;/p&gt;&#xA;" OwnerUserId="3821" LastActivityDate="2013-10-11T00:09:40.040" Title="How important is the exponential of a matrix in computational science?" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8735" PostTypeId="2" ParentId="8734" CreationDate="2013-10-11T00:02:21.033" Score="9" Body="&lt;p&gt;It's an important concept when developing numerical methods for ordinary differential equations, at least as long as they are linear. Consider, for simplicity, a homogenous system of the form $\dot x(t)=Ax(t), x(0)=x_0$, then the solution is $x(t)=e^{At}x_0$. This is easily extended to the case $\dot x(t)=Ax(t)+f(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point now is not that one may actually want to compute, say $x(2)=e^{2A}x_0$. The reason is that computing $e^{2A}$ is typically rather difficult. But, in the typical form of time stepping schemes, we may want to compute $x(t+\Delta t)=e^{A\Delta t}x(t)$ for some small $\Delta t$, and this is of course a much more manageable undertaking since $e^{\text{something small}}$ can readily be approximated. It is not difficult to show that many of the usual time stepping schemes can in fact be derived this way.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-11T00:02:21.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8736" PostTypeId="2" ParentId="8734" CreationDate="2013-10-11T00:09:40.040" Score="8" Body="&lt;p&gt;The matrix exponential has uses in the theory of systems of linear ordinary differential equations and in various applications in control theory, but these all lie beyond what would be discussed in an introductory course in linear algebra, and the topic is not needed to develop the rest of introductory linear algebra, so it really doesn't make any sense to introduce the topic in such a course.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-10-11T00:09:40.040" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="8737" PostTypeId="1" CreationDate="2013-10-11T03:25:30.887" Score="5" ViewCount="4620" Body="&lt;p&gt;I have the following system of equations which I'm trying to solve using Matlab's &lt;code&gt;pdepe&lt;/code&gt; solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 1-D spherical heat diffusion equation with heat generation (source term):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\rho \, C_p\frac{\partial T}{\partial t} = &#xA;\frac{1}{r^2} \frac{\partial}{\partial r} \left(r^2 \, k \frac{\partial T}{\partial r} \right) + \Delta H \frac{\partial \rho_w}{\partial t} \\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;k \frac{\partial T}{\partial r} = h\,(T_\infty - T_s) \; \; \text{at r = surface}\\&#xA;\frac{\partial T}{\partial r} = 0 \; \; \text{at r = 0}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;also note that $\Delta H$ is a constant, i.e. $\Delta H = -255,000$. The initial value for $\rho_w = \rho_{initial} = 700$ and the ambient temperature $T_\infty = 773$ (see Matlab code below for more details).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kinetic reactions of the system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial \rho_w}{\partial t} = -(K_1+K_2+K_3) \rho_w \\&#xA;\frac{\partial \rho_g}{\partial t} = K_1 \rho_w \\&#xA;\frac{\partial \rho_T}{\partial t} = K_2 \rho_w \\&#xA;\frac{\partial \rho_c}{\partial t} = K_3 \rho_w \\&#xA;\frac{\partial \rho_g}{\partial t} = K_4 \rho_T \\&#xA;\frac{\partial \rho_c}{\partial t} = K_5 \rho_T \\&#xA;\text{where}\; \; K = A\,e^{\frac{-E}{RT}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thermal parameters $\rho, C_p, k$ vary according to the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\rho = \rho_w + \rho_c \\&#xA;Y_w = \frac{\rho_w}{\rho_{initial}} \\&#xA;C_p = Y_w C_{pw} + (1-Y_w) C_{pc} \\&#xA;k = Y_w k_w + (1-Y_w) k_c &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C_{pw} = 1500$, $C_{pc} = 1100$, $k_w = 0.105$, and $k_c = 0.071$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can solve the single PDE for heat diffusion (no source term) using the &lt;code&gt;pdepe&lt;/code&gt; function in Matlab as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function PDEexampleSphere&#xA;&#xA;d350 = 0.035e-2; % diameter&#xA;r350 = d350/2;   % radius&#xA;Ti = 300;&#xA;Tinf = 773;&#xA;tmax = 0.8;&#xA;&#xA;m = 2;&#xA;x = linspace(0,r350,20);&#xA;t = 0:0.01:tmax;&#xA;&#xA;sol = pdepe(m,@pdefunc,@icfunc,@bcfunc,x,t);&#xA;u = sol;&#xA;&#xA;% surface plot&#xA;figure(3)&#xA;surf(x,t,u) &#xA;xlabel('Distance (m)')&#xA;ylabel('Time (s)')&#xA;&#xA;% temperature profile&#xA;figure(4)&#xA;plot(t,u(:,1),'b',t,u(:,end),'r')&#xA;hold on&#xA;plot([0 tmax],[Tinf Tinf],':k')&#xA;hold off&#xA;axis([0 tmax Ti-20 Tinf+20])&#xA;xlabel('Time (s)')&#xA;ylabel('Temperature (K)')&#xA;&#xA;% --------------------------------------------------------------&#xA;function [c,f,s] = pdefunc(x,t,u,dudx)&#xA;rho = 700;  % density&#xA;cp = 1500;  % heat capacity&#xA;k = 0.105;  % thermal conductivity&#xA;&#xA;c = rho*cp;&#xA;f = k.*dudx;&#xA;s = 0;&#xA;&#xA;% --------------------------------------------------------------&#xA;function u0 = icfunc(x)&#xA;Ti = 300;   % initial temperature&#xA;&#xA;u0 = Ti;&#xA;&#xA;% --------------------------------------------------------------&#xA;function [pl,ql,pr,qr] = bcfunc(xl,ul,xr,ur,t)&#xA;h = 375;    % heat transfer coefficient&#xA;Tinf = 773; % ambient temperature&#xA;&#xA;pl = 0;&#xA;ql = 0;&#xA;pr = -h*(Tinf-ur);&#xA;qr = 1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I include the kinetic reactions and heat generation term into the Matlab function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note - for more information on the &lt;code&gt;pdepe&lt;/code&gt; function in Matlab, please see the following documentation: &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/pdepe.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/matlab/ref/pdepe.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5146" LastEditorUserId="5146" LastEditDate="2013-10-16T01:49:15.727" LastActivityDate="2015-08-19T03:45:58.160" Title="How to add reaction and source terms to a diffusion PDE solver written with MATLAB's pdepe function?" Tags="|pde|matlab|finite-difference|parabolic-pde|implicit-methods|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="8738" PostTypeId="1" CreationDate="2013-10-11T08:36:31.567" Score="7" ViewCount="361" Body="&lt;p&gt;I'm looking for a library to exploit parallelism within a single heterogeneous computing node (possibly using Accelerators like Xeon Phi or nVidia's GPGPU's) in a C++ FV/DG code using hierarchical octree-like grids. It should&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;support multiple back-ends (e.g. OpenCL, CUDA, OpenMP, OpenACC, ...)&lt;/li&gt;&#xA;&lt;li&gt;hopefully be generic enough to support back-ends from the future,&lt;/li&gt;&#xA;&lt;li&gt;be easy to install/configure,&lt;/li&gt;&#xA;&lt;li&gt;be easy to use.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Linear algebra would be nice, but the library should at least be able to do a simple transform with a user defined kernel on a computing device:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;auto vd = device_vector&amp;lt;double&amp;gt;{ 11., 22., 33., 44. };&#xA;transform(vd, begin(vd), [](double vd_i){ return 2. * vd_i; });&#xA;host_vector&amp;lt;double&amp;gt; vh = vd;  // no-op if the device is the CPU&#xA;for (auto vh_i&amp;amp;&amp;amp; : vh) { cout &amp;lt;&amp;lt; vh_i &amp;lt;&amp;lt; &quot;\n&quot;; } // 22, 44, 66, 88&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've looked at Intel TBB, openMP, openACC, AMD's bolt, and nVidia's Thrust. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thrust seems to be the best fit for my application because:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it provides different backends: CUDA, TBB, and OpenMP (no OpenCL),&lt;/li&gt;&#xA;&lt;li&gt;it has a familiar STL-like interface: host/device containers, iterators, and algorithms,&lt;/li&gt;&#xA;&lt;li&gt;the documentation seems nice.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, I have no experience at all (and don't know anyone who has) building an hybrid MPI-Thrust application. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to my question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there any other library worth looking into that might fit my needs better?&lt;/li&gt;&#xA;&lt;li&gt;Does anyone has experience with hybrid MPI-Thrust applications that can comment on how good of a fit Thrust is for such a thing?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2013-10-14T14:32:20.427" LastActivityDate="2013-10-14T14:32:20.427" Title="Exploiting parallelism within a single heterogeneous computing node" Tags="|parallel-computing|c++|mpi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8739" PostTypeId="2" ParentId="8516" CreationDate="2013-10-11T08:46:02.560" Score="9" Body="&lt;p&gt;I'm a happy user of GoogleTest with a C++ MPI code in a CMake/CTest build environment:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;CMake automatically installs/links googletest from svn!&lt;/li&gt;&#xA;&lt;li&gt;adding tests is a one-liner!&lt;/li&gt;&#xA;&lt;li&gt;writing the tests is easy! (and google mock is very powerful!)&lt;/li&gt;&#xA;&lt;li&gt;CTest can pass command-line parameters to your tests, and exports data to CDash!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is how it works. A batch of unit-tests that require mpi are written into some &lt;code&gt;my_mpi_test.cpp&lt;/code&gt; file which looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;gtest/gtest.h&amp;gt;&#xA;#include &amp;lt;boost/mpi.h&amp;gt;&#xA;&#xA;/// Most testing libraries allow to define main yourself to override initialization.&#xA;int main(int argc, char* argv[]) {&#xA;    ::testing::InitGoogleTest(&amp;amp;argc, argv);  /// Set gtest environment&#xA;    mpi::environment env(argc, argv);  /// Set mpi environment&#xA;    return RUN_ALL_TESTS();  /// Execute all gtest tests&#xA;}&#xA;&#xA;TEST(test_batch_name, test_name) {  /// Then you can create tests as usual,&#xA;  using namespace mpi;&#xA;  communicator world;  /// and use MPI inside your tests.&#xA;  /* ... test stuff here ... */&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The CMakeLists.txt that adds this test is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;add_mpi_test(my_mpi 2)  # Uses 2 MPI processes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;add_mpi_test&lt;/code&gt; wraps CMake's &lt;code&gt;add_test&lt;/code&gt; inside my root CMakeLists.txt:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function(add_mpi_test name no_mpi_proc)&#xA;  include_directories(${MY_TESTING_INCLUDES})&#xA;      # My test are all called name_test.cpp&#xA;      add_executable(${name} ${name}_test.cpp)&#xA;      add_dependencies(${name} googletest)&#xA;  # Make sure to link MPI here too:&#xA;  target_link_libraries(${name} ${MY_TESTING_LIBS})&#xA;  set(test_parameters ${MPIEXEC_NUMPROC_FLAG} ${no_mpi_proc} &quot;./${name}&quot;)&#xA;      add_test(NAME ${name} COMMAND ${MPIEXEC} ${test_parameters})&#xA;endfunction(add_mpi_test)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This last part is not necessary but allows you to easily add mpi tests in one line. Then you can decide if you want to hard-code the numer of MPI processes for each test or read it via a command line parameter to ctest.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2013-10-11T13:39:12.410" LastActivityDate="2013-10-11T13:39:12.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8740" PostTypeId="1" CreationDate="2013-10-11T09:01:17.560" Score="3" ViewCount="487" Body="&lt;p&gt;I have a question regarding the FMG (Full Multi Grid) performance while computing Poisson's equation using Higher Order Compact discretization. I am using a sixth order compact scheme to discretize the Poisson's equation and use a Gauss Seidel method for smoothing/relaxing. And I employ Neumann boundary conditions on all sides of the 2D domain I am dealing with. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed Multigrid tutorial by Briggs et al to deal with the Neumann BCs by making the null-space vector orthogonal to the RHS vector in the equation Ax = b. I do not pin a value in the domain as it would lead to complications in transfers between multigrid levels. Instead I impose the zero mean condition on every level of the FMG cycle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 128X128 grid the L2 norm of the residual stands at 32.00 at the end of FMG cycle which is very high for the entire domain. But it is less when distributed evenly among all the interior points. I would like converge the L2 norm atleast to 1e-2 for the Neumann case. Are there any techniques that I can use to speed this up to the said convergence without changing the number of times I relax on the fine grid. The higher order scheme itself has bad smoothing properties I think. It is not as effective in culling the high frequencies present in the domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an effective and clear strategy to pin the solution at one point int he fine grid and continue with FMG?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, will it help if we use a higher order interpolation operator from coarse to fine grids when we go from one V-cycle to the other? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know your suggestions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best&#xA;Trinath &lt;/p&gt;&#xA;" OwnerUserId="5357" LastEditorUserId="5357" LastEditDate="2013-10-11T09:07:48.377" LastActivityDate="2013-12-12T20:21:31.333" Title="Full Multigrid Performance for Poisson's equation using Higher Order Compact scheme as a Gauss Seidel smoother" Tags="|multigrid|poisson|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8741" PostTypeId="2" ParentId="8740" CreationDate="2013-10-11T14:09:50.670" Score="2" Body="&lt;p&gt;The regular Gauss-Seidel method is not a good smoother for higher order discretizations. From my experience with finite element multigrid, I expect that a block Gauss-Seidel with block size comparable to the width of the stencil should do the job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: if your stencil involves neighbors and second neighbors, use blocks of size 4 or 9. These blocks should contain the degrees of freedom of a small square patch of 4 or 9 discretization points. You can run the method with overlapping patches (expensive with more than 4 points per patch) or nonoverlapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Higher order mesh transfers can also improve your performance.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2013-10-11T14:09:50.670" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8742" PostTypeId="1" AcceptedAnswerId="8751" CreationDate="2013-10-11T14:22:01.527" Score="0" ViewCount="76" Body="&lt;p&gt;I'm doing research for academic credential on the use of metaheuristic optimisation of building envelopes/enclosures, focusing on the aspect of its thermal performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The subject was handled with heavy emphasis on the integration of algorithm into the architectural design process by evaluating building thermal performance through energy simulation and feeding back the results into the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Question is:&lt;/strong&gt; having somewhat limited knowledge of mathematical optimisation, and very limited knowledge of programming languages; how would you advise me to extend my research? i.e. What references should I start looking into? Which computational programmes/software would best serve my purpose? (all within the general direction of my research topic mentioned above).&lt;/p&gt;&#xA;" OwnerUserId="5360" LastActivityDate="2013-10-13T16:23:09.440" Title="Extension of research on optimisation to a relatively complex level" Tags="|optimization|heuristics|" AnswerCount="1" CommentCount="0" ClosedDate="2013-11-20T08:01:11.463" ContentLicense="CC BY-SA 3.0" />
  <row Id="8745" PostTypeId="1" CreationDate="2013-10-11T17:41:12.930" Score="4" ViewCount="185" Body="&lt;p&gt;I am trying to count the number of cache misses, total cycles, etc. per iteration of a &lt;code&gt;for&lt;/code&gt; loop inside &lt;code&gt;MatMult_SeqSBAIJ_2()&lt;/code&gt; . I've configured PETSc to use PAPI and I am using the PAPI API like shown in the skeletal code below. PAPI always fails (!PAPI_OK) while running ./main . Is there a setting in PAPI or PETSc that I am missing, which enables proper counter sampling inside PETSc? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;main.c&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PetscInitialize(&amp;amp;argc,&amp;amp;args,(char*)0,help);&#xA;PAPI_library_init(PAPI_VER_CURRENT); //Initialize PAPI&#xA;/*Matrix and vector setup code here. */&#xA;MatMult_SeqSBAIJ_2(A,xx,zz)&#xA;/*Destroy matrix here*/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sbaij2.c&lt;/code&gt; :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PetscErrorCode MatMult_SeqSBAIJ_2(Mat A,Vec xx,Vec zz)&#xA;{&#xA;long long counters[2];&#xA;int PAPI_events[2] = {PAPI_TOT_CYC, PAPI_L1_MISSES};&#xA;int retval=0;   &#xA;&#xA;PAPI_start_counters( PAPI_events, 2 );&#xA;&#xA;for (i=0; i&amp;lt;mbs;i++)&#xA;{   &#xA;/*SpMV operations happen here*/&#xA;PAPI_read_counters( counters, 2 );&#xA;&#xA;    if ( (retval = PAPI_read_counters(counters, 2)) != PAPI_OK)&#xA;    {&#xA;    printf(&quot;Papi fails!\n&quot;);&#xA;    }&#xA;&#xA;}&#xA;&#xA;PAPI_stop_counters( counters, 2 );&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5145" LastActivityDate="2013-12-11T16:19:59.113" Title="Using PAPI in PETSc code" Tags="|linear-algebra|petsc|performance|libraries|profiling|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8746" PostTypeId="2" ParentId="8738" CreationDate="2013-10-12T00:34:30.317" Score="4" Body="&lt;p&gt;I suggest you take a look at &lt;a href=&quot;http://viennacl.sourceforge.net&quot; rel=&quot;nofollow&quot;&gt;ViennaCL&lt;/a&gt;. It's written in &lt;code&gt;c++&lt;/code&gt;, leverages template meta programming, and supports OpenCL, CUDA, and OpenMP as backends. Just like &lt;code&gt;Thrust&lt;/code&gt;, it has abstraction layers for device and host containers and implements several algorithms, mostly in the area of linear algebra. Although, device integer containers are missing as of version 1.4.2 (apparently they will be included in 1.5.0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, what I like about it is that it nicely wraps around OpenCL API and simplifies the task of writing custom OpenCL kernels. I have not used ViennaCL or Thrust in a hybrid environment so I cannot comment on that. However, I know &lt;code&gt;PETSc&lt;/code&gt; has interface to &lt;code&gt;ViennaCL&lt;/code&gt; and supports hybrid mpi-viennacl vec types. This should simplify the hybrid approach -- Its actually one of my future projects to utilize this feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all, I recommend you take a look at their webpage. Oh, I almsot forgot to mention that &lt;code&gt;ViennaCL&lt;/code&gt; is very nicely documented :).&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-10-12T00:34:30.317" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8747" PostTypeId="2" ParentId="8696" CreationDate="2013-10-12T08:24:57.840" Score="1" Body="&lt;p&gt;I would suggest any of the biomolecular MD packages as potentially suitable (GROMACS, NAMD, DESMOND, AMBER, CHARMM, GROMOS, ACEMD, TINKER, Espresso,... and doubtless another dozen). Those are all&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;well published,&lt;/li&gt;&#xA;&lt;li&gt;likely can all generate a Maxwell-Boltzmann velocity distribution (which you could do yourself so long as they'll use your velocities), &lt;/li&gt;&#xA;&lt;li&gt;will write positions and velocities to taste,&lt;/li&gt;&#xA;&lt;li&gt;incorporate non-determinism by design through their use of floating-point computation (as well as any stochastic integration schemes, as AJK points out),&lt;/li&gt;&#xA;&lt;li&gt;work in three dimensions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Non-determinism is easy to generate - just shuffle the order of the particles in your input. This will change the order of the accumulation of the force and energy calculations, which will typically cause enough numerical difference for divergence of the trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each code will have ways its implementation leads to non-determinism. For example, running GROMACS in parallel with domain decomposition and dynamic load balancing is non-deterministic because it depends on timing measurements that are not reproducible on a real-world computer. You can suppress that with its reproducible mode, but even that will not be reproducible with respect to the input particle order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know that any of the above do hard spheres with elastic collisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well-cited pretty much means written in a serious HPC programming language (C, C++, Fortran), because people use and cite the codes that run fast. So you can't have recently developed programming languages and good publication history.&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2013-10-12T08:24:57.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8748" PostTypeId="2" ParentId="8745" CreationDate="2013-10-12T15:23:17.513" Score="2" Body="&lt;p&gt;In general you want to know as much about your error as possible when debugging these sorts of problems.  In this case, you're not providing enough information with your &lt;code&gt;retval&lt;/code&gt; check.  See the PAPI &lt;a href=&quot;http://icl.cs.utk.edu/projects/papi/wiki/Error_Handling#CONVERTING_ERROR_CODES_TO_ERROR_MESSAGES&quot; rel=&quot;nofollow&quot;&gt;documentation on error-handling&lt;/a&gt; for more information, but to start, you may want to try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;retval = PAPI_read_counters(counters, 2));&#xA;if (retval != PAPI_OK) {&#xA;    fprintf(stderr, &quot;PAPI error %d: %s\n&quot;, ret, PAPI_strerror(retval));&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also increase the level of debugging information with &lt;a href=&quot;http://icl.cs.utk.edu/projects/papi/wiki/PAPIC%3aPAPI_set_debug.3&quot; rel=&quot;nofollow&quot;&gt;PAPI_Set_debug&lt;/a&gt;, but you need to make sure that you've got a DEBUG-enabled build of PAPI.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2013-10-12T15:23:17.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8749" PostTypeId="1" AcceptedAnswerId="8753" CreationDate="2013-10-12T17:22:45.613" Score="3" ViewCount="127" Body="&lt;p&gt;I want to compute $a = \tan(f \theta)$ for $f\in [0,1]$, given $g = \tan\theta$. Obviously, I can compute $a = \tan(f\tan^{-1}g)$, but I'm wondering if there's a more efficient way that avoids having to evaluate two transcendental functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I happen to also want the quantity $b = \tan[(1-f)\theta]$, and I believe the identity&#xA;$$ a+b+gab-g=0 $$&#xA;holds. I can be reasonably assured that $0\le g \lesssim \frac{1}{2}$, so that tangent and arctangent are both somewhat linear. I figure this assumption should greatly help, if there happens to be any efficient method. The emphasis here is on accuracy and robustness.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-13T22:10:52.567" Title="Efficient computation of tangent of fraction of angle" Tags="|numerical-analysis|special-functions|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8750" PostTypeId="2" ParentId="8690" CreationDate="2013-10-12T19:56:13.617" Score="4" Body="&lt;p&gt;I don't have time to get all the details down, but maybe this answer can give some helpful intuition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, something that will work is taking the first $N$ binary binary of your number in $[0,1]$ and assigning them as the most significant digits for each of the $N$ dimensions (so, the first decimal digit is the most significant digit in the first dimension, the second decimal digit is the most significant digit in the second dimension, and so on). Then, you perform a change of coordinates in the $[0,1]^N$ interval you are mapping to involving translations/rotations/reflections &lt;em&gt;that depends on which ones were the previously read $N$ digits&lt;/em&gt;, and repeat the procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A procedure in the case of $N=2$ that looks to me like it is following this model is given here: &lt;a href=&quot;http://bit-player.org/2013/mapping-the-hilbert-curve&quot; rel=&quot;nofollow&quot;&gt;http://bit-player.org/2013/mapping-the-hilbert-curve&lt;/a&gt;. The operators for the change of coordinates are what they call the $H_i$. The main computational difference with your situation is that instead of considering quadits (groups of two bits, or which is the same, digits from $0$ to $3$), you are considering groups of $N$ bits (or which is the same, digits from $0$ to $2^N - 1$. Your $H_i$ operators will operate on size $N$ vectors, not size $2$ ones. And there will be $2^N$ of the $H_i$ operators, so you will not be able to precompute all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this clarifies more than it confuses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how much of this is a problem depends on the function you are trying to plot, but an issue you are likely to face is that to get a single digit of precision in each of the N coordinates, you must have $N$ digits of precision in your real number between $0$ and $1$. And 10,000 digits of precision is quite a bit : )&lt;/p&gt;&#xA;" OwnerUserId="3760" LastEditorUserId="3760" LastEditDate="2013-10-14T15:28:28.403" LastActivityDate="2013-10-14T15:28:28.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8751" PostTypeId="2" ParentId="8742" CreationDate="2013-10-13T16:23:09.440" Score="5" Body="&lt;p&gt;You will typically get the most significant steps forward if you can concisely state a problem in terms of mathematics. When you have a concise formulation in terms of &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what the free variables are&lt;/li&gt;&#xA;&lt;li&gt;what the objective function is&lt;/li&gt;&#xA;&lt;li&gt;what the constraints are&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;then the next step is to find algorithms that are well-suited to the problem at hand (e.g., can you show that the problem is convex? equality/inequality-constrained? continuous or integer?). The book I always recommend to learn about optimization methods is the one by Nocedal and Wright, &quot;Numerical Optimization&quot;. You don't need to understand everything in it, but it's an easy read and it will give you an idea of how to classify algorithms and how to approach problems.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-13T16:23:09.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8752" PostTypeId="1" CreationDate="2013-10-13T19:50:39.290" Score="1" ViewCount="366" Body="&lt;p&gt;I have a bunch of polygons and a coarse uniform grid. I want to implement two different range queries, for a rectangle aligned with the uniform grid:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does the rectangle intersect with any polygon at all?&lt;/li&gt;&#xA;&lt;li&gt;Give me all the polygons which intersect the rectangle.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A simple data-structure would be to have for each cell a list of pointers to the polygons that overlap the cell. This data-structure is appropriate for the first range query. It seems less appropriate for the second range query, because the same polygon may overlap multiple cells, and we don't want to report the same polygon multiple times. One solution would be to use a &lt;code&gt;std::set&amp;lt;polygon_t*&amp;gt;&lt;/code&gt; container to collect the pointers to polygons that are delivered by a query. However, let's assume I mistrust &lt;code&gt;std::set&lt;/code&gt; in the sense that it will do too many memory allocations even if I reuse the same container for multiple queries. (This mistrust may be unfounded, so I should definitively try this first.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder whether the problem can be solved more efficiently, if I replace the polygons by their &quot;grid aligned&quot; bounding boxes. During a range query, the knowledge of the lower left point of the bounding box should be sufficient for determining whether the polygon corresponding to the box has already been reported. But even in this case, the complexity of a query returning $m$ polygons won't be $O(m)$, because some work is done for every cell intersecting the bounding box of the polygon. I wonder whether creating three additional lists with the polygons whose bounding box starts in this cell in &quot;x-direction&quot;,  &quot;y-direction&quot; and &quot;x- and y-direction&quot; allowing to avoid this additional work would be a good idea. Instead of creating new lists, I could also try to just partition the existing list appropriately.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In general, the coarse uniform grid based data-structures discussed above offer essentially constant lookup time, at the cost of potentially high memory consumption. What is the state of the art regarding such data-structures? Is there a way to avoid the potential memory bloat caused by elements with huge bounding boxes which overlap many grid cells? Also, I don't like the fact that only the simple data-structure can handle polygons directly, while the more efficient data-structures only handle the bounding boxes of the polygons.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2013-10-13T20:10:31.287" LastActivityDate="2013-10-13T20:10:31.287" Title="Practical implementation of spatial binning for rectangular range queries" Tags="|c++|data-structures|spatial-data|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8753" PostTypeId="2" ParentId="8749" CreationDate="2013-10-13T22:10:52.567" Score="4" Body="&lt;p&gt;I cannot think of any useful trigonometric identity that could help evaluating&#xA;\begin{equation}&#xA;a = \tan ( f \tan^{-1} g)&#xA;\end{equation}&#xA;so I would try a series expansion&#xA;\begin{equation}&#xA;fg + \frac{1}{3}f(f^2-1)g^3 + \frac{1}{15}f(2 f^4 - 5f^2 + 3) g^5 + \ldots&#xA;\end{equation}&#xA;but I doubt that if focus is on accuracy and robustness you can do much better than calling &lt;code&gt;tan(f*atan(g))&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you know $a$ and $g$ you can compute $b$ by the &lt;a href=&quot;http://mathworld.wolfram.com/TrigonometricAdditionFormulas.html&quot; rel=&quot;nofollow&quot;&gt;tangent angle subtraction formula&lt;/a&gt;&#xA;\begin{equation}&#xA;\tan(\alpha - \beta) = \frac{\tan\alpha - \tan\beta}{1+\tan\alpha \, \tan\beta}&#xA;\end{equation}&#xA;with $\alpha = \theta$, $\beta = f\theta$, leading to&#xA;\begin{equation}&#xA;b = \frac{g-a}{1+ga}&#xA;\end{equation}&#xA;which proves your identity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I'm not an expert in this field, but at high school I was very good in trigonometry, so I could not resist answering the question...)&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2013-10-13T22:10:52.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8754" PostTypeId="1" CreationDate="2013-10-14T11:43:17.080" Score="0" ViewCount="248" Body="&lt;p&gt;Given an input parameters for the MARTINI force field such as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;;;;;;; HYDRATED CHLORIDE ION&#xA;&#xA;[moleculetype]&#xA;; molname   nrexcl&#xA;  CL-       1&#xA;&#xA;[atoms]&#xA;;id     type    resnr   residu  atom    cgnr    charge&#xA; 1  Qa  1   ION     CL-     1   -1.0 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I wish to know which parameters go into which term in the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm after the either the source code of the MARTINI force field, or just the equation. I believe that it's just a combination of 12-6 LJ and electrostatic interatction, but I want to make sure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="1017" LastActivityDate="2013-11-14T20:06:53.510" Title="Implementation of the MARTINI force field" Tags="|molecular-dynamics|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8755" PostTypeId="1" CreationDate="2013-10-14T11:46:18.903" Score="2" ViewCount="251" Body="&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve&#xA;$$\frac{\partial u}{\partial t}+u\frac{\partial u}{\partial x}=\nu\frac{\partial^{2}u}{\partial x^{2}}\\&#xA;u(0,t)=0\nonumber \\&#xA;u(1,t)=0\nonumber \\&#xA;u(x,0)=\sin(\pi x)\\&#xA;0&amp;lt;x&amp;lt;1,\; t&amp;gt;0$$&#xA;I take $u_n(x,t)$ as an approximation to $u(x,t)$&#xA;$$u_{n}(x,t)=\sum_{n=-4}^{2^j-1}\phi_{n}(x)c_{n}(t),\quad j\in \mathbb{N}$$&#xA;Here $\phi_n(x)$ are basis functions and i need to find the coeffiicients $c_n$. If i take the weight functions as $\phi_k(x)$ then weighted integral form of my problem will be as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{0}^{1}\phi_{k}(x)\sum_{n}\phi_{n}(x)c_{n}^{'}(t)dx+\int_{0}^{1}\phi_{k}(x)\left(\sum_{n}\phi_{n}(x)c_{n}(t)\right)\left(\sum_{l}\phi'_{l}(x)c_{l}(t)\right)dx\\-\int_{0}^{1}\phi_{k}(x)\nu\sum_{n}\phi''_{n}(x)c_{n}(t)dx=0$$&#xA;Now, set &#xA;$$\delta_{k,n}=\int_{0}^{1}\phi_{k}\phi_{n}dx,\quad\Omega_{k,n}^{0,2}=\int_{0}^{1}\phi_{k}\phi''_{n},\quad\Omega_{k,n,l}^{0,0,1}=\int_{0}^{1}\phi_{k}\phi_{n}\phi'_{l}dx$$&#xA;Therefore discretized version of my problem becomes.&#xA;$$\sum_{n=-4}^{0}c'_{n}(t)\delta_{k,n}+\sum_{n=-4}^{0}c_{n}(t)\sum_{l=-4}^{0}c_{l}(t)\Omega_{k,n,l}^{0,0,1}-\nu\sum_{n=-4}^{0}c_{n}(t)\Omega_{k,n}^{0,2}=0$$&#xA;$$\left\{ \mathbf{c}_{n}'(t)\right\} \left[\delta_{k,n}\right]+\left\{ \mathbf{c}_{n}(t)\right\} \left[\mathbf{c}_{l}(t)\Omega_{k,n,l}^{0,0,1}\right]-\nu\left\{ \mathbf{c}_{n}(t)\right\} \left[\Omega_{k,n}^{0,2}\right]=0\\&#xA;$$&#xA;Here i know the values of $\delta_{k,n}$, $\Omega_{k,n,l}^{0,0,1}$, $\Omega_{k,n}^{0,2}$ and $\nu$. Also $-4 \leq k,n,l \leq 2^j-1$. &#xA;Approximating $c'_{n}(t),c_{n}(t)$ by $$c'_{n}(t)=\frac{c_{n}^{i+1}-c_{n}^{i}}{h}$$&#xA;$$c_{n}(t)=\frac{c_{n}^{i+1}+c_{n}^{i}}{2}$$&#xA;I get &#xA;$$\mathbf c_{n}^{i+1}\left(\frac{\delta_{k,n}}{h}+\frac{\mathbf c_{l}\Omega_{k,n,l}^{0,0,1}}{2}-\nu\frac{\Omega_{k,n}^{0,2}}{2}\right)=\mathbf c_{n}^{i}\left(\frac{\delta_{k,n}}{h}-\frac{\mathbf c_{l}\Omega_{k,n,l}^{0,0,1}}{2}+\nu\frac{\Omega_{k,n}^{0,2}}{2}\right)$$&#xA;I know $c_n^i$ for $i=1$ from initial condition and i want to find $c_{n}^{i+1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $c_l$ not get into the job i could find the $c_n$. But i am confused with  $c_l$. &lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="4282" LastEditDate="2013-10-15T06:02:55.313" LastActivityDate="2013-10-15T06:02:55.313" Title="How can i solve this first order system of differential equations" Tags="|finite-difference|ode|numerical-analysis|nonlinear-equations|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="8756" PostTypeId="2" ParentId="8755" CreationDate="2013-10-14T13:05:52.087" Score="5" Body="&lt;p&gt;As it looks to me your formulation of the discrete system is incomplete. You need to&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;state your equation at each (grid?) point associated with $n$&lt;/li&gt;&#xA;&lt;li&gt;put back the summation over $l$ (grid) points in your last equation&lt;/li&gt;&#xA;&lt;li&gt;assign a time index $i+1$ or $i$ to your $c_l$s &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In this way, you get a nonlinear (if you take $c_l^{n+1}$) or linear (if you take $c_l^{n}$) system of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on your discretization, the resulting system will have &lt;strong&gt;no&lt;/strong&gt; / &lt;strong&gt;one&lt;/strong&gt; / or &lt;strong&gt;multiple&lt;/strong&gt; solutions. If it has one, you can use standard linear solvers or a Newton scheme to solve it. If not, you need more considerations and maybe a remodelling of your system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also try, to write your equation system in matrix/vector form, which I find easier to handle than the formulation with the summations and indices.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-10-14T13:05:52.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8757" PostTypeId="2" ParentId="8755" CreationDate="2013-10-14T13:12:35.960" Score="7" Body="&lt;p&gt;What you describe as your time discretization is called the Crank-Nicolson scheme. For nonlinear differential equations it leads, as you have observed, to a nonlinear algebraic system that needs to be solved at each time step. The typical approach is to solve it with a Newton method -- in your case, that requires to solve a nonlinear system in 5 variables. Any introductory book on numerical methods will explain several different methods of doing that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equations such as yours are pretty straightforward and without great difficulty. Rather than implementing a numerical scheme yourself, may I suggest you simply describe the system of differential equations to Maple, Mathematica or Matlab and let them solve it to essentially any accuracy you desire?&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-14T13:12:35.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8758" PostTypeId="1" AcceptedAnswerId="8760" CreationDate="2013-10-14T21:30:57.327" Score="4" ViewCount="168" Body="&lt;p&gt;When you have fine-scale features (e.g. boundary layers) in the solution, its FEM approximation on coarse meshes converge at strange apparent rates. Looking at Cea's lemma, is this behaviour because of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the best approximation error behaving strangely?&lt;/li&gt;&#xA;&lt;li&gt;or the constant changing with h?&lt;/li&gt;&#xA;&lt;li&gt;or should I actually look at Strang's first lemma? (errors in numerical integration of very fine-scale data)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5378" LastActivityDate="2015-09-30T21:34:22.100" Title="What is the origin of the preasymptotic convergence behaviour in FEM?" Tags="|finite-element|numerical-analysis|convergence|advection-diffusion|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8760" PostTypeId="2" ParentId="8758" CreationDate="2013-10-15T01:11:06.777" Score="10" Body="&lt;p&gt;It is because we typically neglect higher order terms in error estimates. For example, we can show that&#xA;$$&#xA;  \|e\| \le C(u) h^2 + {\cal O}(h^3).&#xA;$$&#xA;The point is that when $h$ is small, the cubic term is small and can be neglected. In fact, when $h$ is small, you can observe quadratic convergence. But whenever $h$ is not small (where &quot;small&quot; is relative to the features of the solution), the cubic and possibly even higher order terms can be large compared to $Ch^2$ and will lead to behavior that is not as easily predictable.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2015-09-30T21:34:22.100" LastActivityDate="2015-09-30T21:34:22.100" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8761" PostTypeId="1" CreationDate="2013-10-15T14:27:21.783" Score="2" ViewCount="679" Body="&lt;p&gt;I want to use lanczos method to calculate the few lowest eigenvalue and eigen-vector of a large sparse symmetric matrix(~50k x ~50k). In &lt;a href=&quot;http://www.netlib.org/lanczos/index.html&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lanczos/index.html&lt;/a&gt; I found the codes leval and levec are suitable for my situation. However, the instruction is too complicated to find how to use it and I don't know how to set the input and output files.&#xA;Who used it and please gives some advice.&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="5104" LastActivityDate="2014-04-13T17:24:56.913" Title="how to use lanczos code from netlib for large sparse symmetric matrix?" Tags="|eigensystem|fortran|lapack|banded-matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8762" PostTypeId="2" ParentId="8761" CreationDate="2013-10-15T14:31:49.133" Score="1" Body="&lt;p&gt;This does not directly answer your question, but maybe you want to use something like ARPACK, for example &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.eigsh.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;scipy.sparse.linalg.eigsh&lt;/code&gt;&lt;/a&gt; in python.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-10-15T14:31:49.133" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8763" PostTypeId="1" AcceptedAnswerId="8768" CreationDate="2013-10-15T19:17:39.390" Score="3" ViewCount="203" Body="&lt;p&gt;Given a one-dimensional function (let's say infinitely differentiable) and a prescribed accuracy of an L2 (or H1) norm, what is the optimal mesh and (in general arbitrary) polynomial orders on each element so that the approximation has the least degrees of freedom? (For example if I have three elements, of orders $p_1=2$, $p_2=4$ and $p_3=10$, then there are $p_1+p_2+p_3+1=2+4+10+1=17$ degrees of freedom.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer are the polynomial orders and coordinates of the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, does the most efficient representation have equal polynomial orders on all elements or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the algorithm to find this optimal representation?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Naive algorithm&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For each total number of elements N=1, 2, 3, ..., we pick all combinations of polynomial orders $p_i$ (e.g. for N=3, we have (1, 1, 1), (2, 1, 1), (1, 2, 1), (1, 1, 2), (2, 2, 1), ..., (100, 1, 1), ...) and we optimize the coordinates of the mesh in order to minimize the L1 norm. A candidate is such a combination of mesh+orders, that has a lower L1 norm than prescribed. We only keep the candidate with the lowest degrees of freedom. We skip combinations of N and $p_i$ which have more degrees of freedom, so the algorithm must eventually terminate.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Motivation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Examples of functions that I have in mind are numerical solutions of radial Schroedinger or Dirac equations on a finite interval (either as part of DFT or Hartree-Fock), as well as the various corresponding potentials. All these functions are infinitely differentiable and the numerical solver can solve those to arbitrary numerical accuracy. Then I set e.g. $10^{-6}$ in L2 norm and I want to know the most efficient representation in terms of $hp$-FEM degrees of freedom.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2013-11-15T03:06:56.880" Title="What is the most efficient way to represent a 1D function using $hp$-finite element basis functions" Tags="|optimization|finite-element|polynomials|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8764" PostTypeId="2" ParentId="8754" CreationDate="2013-10-15T19:57:11.347" Score="2" Body="&lt;p&gt;That file is input for GROMACS, which the Martini &lt;a href=&quot;http://md.chem.rug.nl/cgmartini/index.php/faq&quot; rel=&quot;nofollow&quot;&gt;http://md.chem.rug.nl/cgmartini/index.php/faq&lt;/a&gt; hints was the software with which the reference implementation was developed. The Martini &lt;a href=&quot;http://md.chem.rug.nl/cgmartini/index.php/tutorial&quot; rel=&quot;nofollow&quot;&gt;http://md.chem.rug.nl/cgmartini/index.php/tutorial&lt;/a&gt; page also suggests GROMACS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Qa atom type is defined elsewhere by the forcefield files. You can read about the file formats and the equations whose parameters they describe in the GROMACS &lt;a href=&quot;http://www.gromacs.org/Documentation/Manual&quot; rel=&quot;nofollow&quot;&gt;http://www.gromacs.org/Documentation/Manual&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2013-10-15T19:57:11.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8765" PostTypeId="1" CreationDate="2013-10-15T20:51:17.003" Score="1" ViewCount="188" Body="&lt;p&gt;Could anybody please help with voro++ cause i am new to this software?My problem is how can i find the coordination  number of an atom ,cause i checked it for a bcc lattice and it gave me 24 for all atoms which is correct cause the wigner-seitz polyhedron of bcc is a truncated octahedron with indeed 24 vertices. But what if i don't know the initial lattice and i want the coordination number for example for bcc is 8?Does anynody knows the option to do this???&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="5384" LastActivityDate="2013-10-15T20:51:17.003" Title="How can i find the coordination number with voro++?" Tags="|software|simulation|computational-geometry|" AnswerCount="0" CommentCount="2" ClosedDate="2018-03-13T19:34:05.970" ContentLicense="CC BY-SA 3.0" />
  <row Id="8766" PostTypeId="1" AcceptedAnswerId="8767" CreationDate="2013-10-15T23:21:34.767" Score="2" ViewCount="226" Body="&lt;p&gt;I have a large, yet very sparse, matrix that I'd like to diagonalize. Both my own Lanczos implementation and the ARPACK that's built in with scipy fail to converge properly, though. I know that my Lanczos code itself is correct, and the ARPACK implementation should be correct too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that I note about my matrix is that the entries in the $n$-th row grow like $\mathcal{O}(n^2)$. Could that contribute to the problem? Especially since - for physical reasons - I expect the eigenvector associated with the smallest (algebraic) eigenvalue to have little weight at these indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;If I start with a totally not random starting vector that just has weight 1 in the first element and is zero everywhere else, then both my own Lanczos and ARPACK converge to the correct (LAPACK dense matrix diagonalization) solution. In a general setting where I wouldn't already have some physical intuition what eigenvector I'm looking for, what can one do to combat large matrix elements (both diagonal and off-diagonal)?&lt;/p&gt;&#xA;" OwnerUserId="105" LastEditorUserId="105" LastEditDate="2013-10-15T23:32:02.233" LastActivityDate="2013-10-16T18:12:10.217" Title="Does Lanczos have trouble with large matrix elements?" Tags="|eigensystem|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8767" PostTypeId="2" ParentId="8766" CreationDate="2013-10-16T02:19:23.193" Score="3" Body="&lt;p&gt;It sounds like for your general case, you should be doing some kind of column and row scaling, effectively left and right preconditioning with diagonal scaling matrices. You would computing these scaling matrices to try to get all matrix elements on the same order of magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&#xA;Let's work with an example $2\times 2$ matrix&#xA;$$ \begin{bmatrix} 2 &amp;amp; 1 \\ 10^{10} &amp;amp; 2\times 10^{10} \end{bmatrix}$$&#xA;It has eigenvectors (approximately) $[5\times 10^{-11},1]^T$ and $[-2,1]^T$.&#xA;Suppose we scale it as&#xA;$$ \begin{bmatrix}1 &amp;amp; 0 \\ 0 &amp;amp; 10^{-5} \end{bmatrix}\begin{bmatrix} 2 &amp;amp; 1 \\ 10^{10} &amp;amp; 2\times 10^{10} \end{bmatrix} \begin{bmatrix}1 &amp;amp; 0 \\ 0 &amp;amp; 10^{5} \end{bmatrix}$$&#xA;Obviously the eigenvalues don't change, but the eigenvectors are now $[5\times 10^{-6},1]^T$ and $[-200000,1]$. Notice that the elements of the first eigenvector are within 6 orders of magnitude of each other, instead of 11. Of course, the second eigenvector has worse scaling, but no worse than the first. This scaling should improve things overall since the accuracy of the small elements of the eigenvectors depends on how they small they are relative to the largest elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit for symmetry&lt;/strong&gt;&#xA;In the case that $A$ is symmetric, then the original eigenvalue problem is $Ax=\lambda x$. If we apply &lt;em&gt;symmetric&lt;/em&gt; scaling matrices $S$ on both sides to balance $A$, then&#xA;$$ SASy = \lambda S^2 y$$&#xA;for $y=S^{-1}x$. This preserves symmetry at the cost of having to solve a generalized eigenvalue problem (but the $B$ matrix is simply diagonal, so this may not be too bad).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $S$ should be chosen to have nice round elements, like exact powers of 2. This makes it so that $S$ can be applied exactly without rounding error, and $S^2$ can be stored accurately so long as it is within the floating point range.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2013-10-16T18:12:10.217" LastActivityDate="2013-10-16T18:12:10.217" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8768" PostTypeId="2" ParentId="8763" CreationDate="2013-10-16T02:48:48.120" Score="3" Body="&lt;p&gt;I don't know whether that's already published, but Peter Minev of the University of South Carolina has developed algorithms that produce $hp$ meshes that are provably within a fixed factor of the optimal number of degrees of freedom to represent a given function with a prescribed accuracy $\varepsilon$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, it is quite clear that even for $C^\infty$ or analytic functions, the optimal mesh will not use the same polynomial degree everywhere. It is easy to conceive functions that will have large derivatives somewhere and be very smooth elsewhere, and if you don't ask for very small tolerances $\varepsilon$, then it's quite clear that you should use low and high polynomial degrees in these regions, respectively.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-16T02:48:48.120" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8769" PostTypeId="1" CreationDate="2013-10-16T03:56:23.090" Score="2" ViewCount="1103" Body="&lt;p&gt;I have a need to get a variable value from another process rank of which I know. This happens in context of a parallel solver of a &quot;A x = b&quot; equation, for which process with rank 0 knows matrix A, and the other processes put some values (i,j) from this matrix into the matrix of a parallel solver data type (Petsc's Mat). This means that these other processes walk through the range delegated to them, calculate i and j, retrieve ij's element, and call MatSetValue. There is no way to avoid such way - it is a finite differencing method and the parent process has variable values in central points and in neighbouring points (left,right,top,bottom) For example the same happens in &lt;strong&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex13f90.F.html&quot; rel=&quot;nofollow&quot;&gt;ex13F90.F&lt;/a&gt;&lt;/strong&gt; in petsc library examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I don't know a proper MPI subroutine to retrieve the a_central, a_top, a_bottom, ..., values from the parent processes. Right now I tried to broadcast them (MPI_Bcast) but this means that each process has the entire matrix and runs out of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Per the answer below, the essence of this question is &quot;How do I do one-sized MPI communication?&quot;...&lt;/p&gt;&#xA;" OwnerDisplayName="user5273" LastEditorDisplayName="user5273" LastEditDate="2013-10-17T01:08:05.720" LastActivityDate="2013-10-20T21:05:51.290" Title="mpi retrieve a variable value from process with known rank to process that made an mpi_something call" Tags="|petsc|mpi|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8770" PostTypeId="1" CreationDate="2013-10-16T04:24:40.903" Score="0" ViewCount="751" Body="&lt;p&gt;I have the following Max Min optimization problem that appears to be non convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.google.com/file/d/0B39N8ztL-yN3VFNDaG9lZEhBZFE/edit?usp=sharing&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where t p c are my variables and all others are constants. I took the eigen values of the hermetian part of the hessian matrix of the 1st constraint(keeping all the constants as 1) and it comes out to be negative(hessian is indefinite for some values of t p and c). does this mean that the problem is non convex altogether. can it be converted into convex. is there any tool for solving such non convex problems?&lt;/p&gt;&#xA;" OwnerUserId="5387" LastActivityDate="2013-10-16T11:01:28.867" Title="Non convex optimization" Tags="|optimization|nonconvex|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="8771" PostTypeId="2" ParentId="8770" CreationDate="2013-10-16T11:01:28.867" Score="5" Body="&lt;p&gt;The problem is a standard nonlinear nonconvex problem, so any solver for this problem class is suitable to solve the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, the following code implements the problem in the MATLAB Toolbox YALMIP (disclaimer, developed  by me) and solves the problem using the local nonlinear solver ipopt.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 3;&#xA;K = 3;&#xA;h = rand(N,K);&#xA;R = rand(K,1);&#xA;sigma = 1;&#xA;beta  = 1;&#xA;Pmax = 10;&#xA;Constraints = [];&#xA;P = sdpvar(N,K,'full');&#xA;C = sdpvar(N,K,'full');&#xA;sdpvar t&#xA;Constraints = [P&amp;gt;=0, C&amp;gt;=0, sum(sum(P)) &amp;lt;= Pmax,sum(C,1)&amp;lt;=1]&#xA;for k = 1:K&#xA;   Constraints = [Constraints, t&amp;lt;=(beta/(N*R(k)))*sum(C(:,k).*log2(1+P(:,k).*h(:,k)/sigma))];&#xA;end    &#xA;solvesdp(Constraints,-t,sdpsettings('solver','ipopt'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;ipopt is a local solver, so all you can hope for here is a locally optimal solution. However, it looks as if your problem actually is rather easy, as the solution often seems to be globally optimal. You can see this by solving the problem globally using the global solver available in YALMIP. It implements a b&amp;amp;b strategy based on linear relaxations for lower bounds and nonlinear local solvers for upper bounds. Of course, it is only applicable to small problems (unless you have a lot of time to spare...)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solvesdp(Constraints,-t,sdpsettings('solver','bmibnb'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, I would like to add that the structure of your problem actually allows you to eliminate the c-variables. It can be seen that the optimal choice of c is to have all elements in each column equal to zero, execpt the element which corresponds to the largest term in the vector of logarithmic terms. Hence, conceptually, you can solve&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Constraints = [P&amp;gt;=0, sum(sum(P))&amp;lt;= Pmax]&#xA;for k = 1:K&#xA;    Constraints = [Constraints, t&amp;lt;=(beta/(N*R(k)))*max(log2(1+P(:,k).*h(:,k)/sigma))];&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This leads to a mixed-integer nonlinear nonconvex program when you model it in YALMIP (binary variables are introduced to model the nonconvex use of the max operator), and it seems to be a less efficient approach than the straightforward nonlinear approach. &lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-10-16T11:01:28.867" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8772" PostTypeId="1" CreationDate="2013-10-16T11:02:49.650" Score="5" ViewCount="782" Body="&lt;p&gt;Given $a\in\mathbb{R}^{mn\times n}$, find a $C\in\mathbb{R}^{n}$, $x\in\mathbb{R}^{m\times n}$ such that&#xA;$$&#xA;0 = f_{k}(\boldsymbol{C}, \boldsymbol{x}):=\sum_{i=1}^{m} C_{i} \left(\prod_{j=1}^{n} a_{kj}^{x_{i,j}} - 1\right)&#xA;$$&#xA;for all $k\in\{1,\dots,mn\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The rows of $a_{kj}$ are vectors of solutions to a system of differential equations, and the goal of finding the roots of the above system is to find invariant quantities of those diff. eq.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To show that the system has a singular Jacobian, I have a test setup where $a_{k1} = (1-2t_{k})^{-0.5}$, $a_{k2} = (1-2t_{k})^{-1}$ and the rest zero (only $i=1$, but it is still singular if you include the sum). It is in this case easy to check that the Jacobian is singular irregardless of the values of $\boldsymbol{C}$ and $\boldsymbol{x}$. The solution is any $x$ with $2x_{11} - x_{12}=0$ and $C$ anything, and I'm guessing that the Jacobian is singular because this does imply infinite solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just need to be able to find a couple of them (that are non-trivial such as $C_{i} =0$, and I can go from there. Most root-finding methods I found that can deal with singular Jacobians only mention singularities at the solution, not everywhere. Are there any methods that can tackle this problem? I should also mention that the system should eventually get quite large (order of 1000 equations).&lt;/p&gt;&#xA;" OwnerUserId="1264" LastEditorUserId="3980" LastEditDate="2013-10-21T13:42:37.640" LastActivityDate="2013-10-21T13:42:37.640" Title="Finding roots of systems of equations with a Jacobian that is singular everywhere" Tags="|nonlinear-equations|roots|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="8773" PostTypeId="2" ParentId="8769" CreationDate="2013-10-16T11:40:59.063" Score="3" Body="&lt;p&gt;There are generally three approaches to this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1/ If processor P has a deterministic algorithm by which it needs to know which matrix elements it needs, then processor 1 can replay this same algorithm to find out which elements processor P will need. It can then send these elements to processor P (and P alone, not everyone).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2/ Processor P can &quot;simulate&quot; the algorithm by which it does computations and record which matrix elements it will need to do its work. It can then send this list of elements it will need to processor 1 and processor 1 can reply by sending these elements to P. P can then perform its algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3/ Processor P can use &quot;one-sided&quot; MPI communication to simply fetch data elements from the memory of processor 1.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-16T11:40:59.063" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8774" PostTypeId="1" AcceptedAnswerId="8816" CreationDate="2013-10-16T16:28:21.290" Score="4" ViewCount="451" Body="&lt;p&gt;I'm following &lt;a href=&quot;http://inside.mines.edu/~pconstan/docs/constantine-primer.pdf&quot; rel=&quot;nofollow&quot;&gt;Paul Constantine's Primer on Stochastic Galerkin Method&lt;/a&gt;, Section 3.1 (2D Poisson Example).  In this matlab code, the example attempts to solve the PDE&#xA;$$\alpha(w)(u_{xx}+u_{yy})=1 \text{ in } \Omega=[-1,1]^2$$&#xA;$$u=0 \text{ on } \partial\Omega$$&#xA;with $\alpha(w)$ a uniformly randomly distributed variable whose values lie in $[1,3]$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was able to follow the derivation of the method and its implementation into the matlab available &lt;a href=&quot;http://www.stanford.edu/~paulcon/projects/SGS_Primer/Poisson_2D.tgz&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.  Except for one small detail...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the matlab code, the linear operator $Lu=u_{xx}+u_{yy}$ was discretized by finite difference method using $n=16$ equispaced intervals.  In my understanding of the problem, this would mean that $$\Delta x=\frac{2}{n}$$.  However, the main program file *poisson_2d.m*, (line 38), he used&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$dx=1/n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm almost certain that this is a typo.  Even though the stochastic galerkin code is supposedly verified by monte-carlo simulations, even the monte-carlo simulations use a finite difference discretization of $dx=1/n$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find any other reason to suspect that the spatial discretization of the laplacian for this problem should be $\frac{1}{n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone, with some experience with this sort of thing, confirm/refute my suspicions?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-10-17T02:40:28.357" LastActivityDate="2013-10-21T13:50:50.283" Title="Slight mistake in Stochastic Galerkin code" Tags="|finite-element|stochastic|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8775" PostTypeId="1" CreationDate="2013-10-16T22:45:17.830" Score="7" ViewCount="1004" Body="&lt;p&gt;I'm looking at doing a finite-element simulation of air flow essentially for the purposes of approximating the response to an external audio impulse of a smallish (~10-30 cm scale), stationary 3D-printed structure containing a microphone. I'm mostly interested in the human-audible spectrum of 20-20000 Hz. The purpose is to use this information to guide the structure design to have the desired acoustic properties. I think a good analogous problem of similar complexity would be simulating a human ear starting from a good model, although I'm willing to use a cruder model if that makes the problem feasible.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Googling reveals fairly little about acoustic simulation with FEM. Is something like this even currently computationally feasible? Without supercomputer resources? Or is there another better approach?&lt;/li&gt;&#xA;&lt;li&gt;I think I basically understand the finite element method, but I don't have practical experience. I'm also relatively new to fluid dynamics. I think a generic Navier-Stokes simulation (minus viscosity, since its effect should be negligible) should be a good and sufficient way to approach this problem. Am I right?&lt;/li&gt;&#xA;&lt;li&gt;There seems to be lots of software/libraries for FEM, and I suspect I'm missing the big distinguishing features. I've now taken a cursory look at Elmer, which even has some acoustics and vibroacoustics examples, and OpenFOAM. In addition to those there are at least Rheolef, Gerris, Code_Saturne, Code_Aster, SyFi, Feel++, DUNE, FEniCS, ALBERTA, GetDP and FreeFEM available in Debian alone. What should I look for in FEM software?&lt;/li&gt;&#xA;&lt;li&gt;Is there anything I should read for more information? I've been looking at (computational) aeroacoustics, but it seems to be more geared towards analyzing things like turbulent noise from a jet engine at high speeds.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5386" LastActivityDate="2013-10-21T01:28:22.107" Title="Feasibility and software for acoustic simulation" Tags="|finite-element|fluid-dynamics|software|simulation|navier-stokes|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8776" PostTypeId="2" ParentId="8633" CreationDate="2013-10-17T04:17:18.320" Score="1" Body="&lt;p&gt;From &lt;a href=&quot;http://www.scholarpedia.org/article/Numerical_continuation#Branch_switching&quot; rel=&quot;nofollow&quot;&gt;http://www.scholarpedia.org/article/Numerical_continuation#Branch_switching&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When the Jacobian matrix becomes singular and there is no simple fold, so the continuation point is not regular, multiple branches emanate from such a point. In that case, to switch to a different branch than the one that follows the direction vi (as defined above) the continuation must be perturbed. The algebraic bifurcation equation gives the necessary direction. One can often also perturb the system and then try to converge parallel to the original branch; typically the second branch is then found.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are also some packages mentioned on the site which implement automatic branch switching (e.g., AUTO).&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2013-10-17T04:17:18.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8777" PostTypeId="1" CreationDate="2013-10-17T10:52:40.490" Score="1" ViewCount="1072" Body="&lt;p&gt;I have the following optimization problem and I am trying to check for its convexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.google.com/file/d/0B39N8ztL-yN3Xy00aFpObEFvM3c/edit?usp=sharing&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;&#xA;As per the definition of convexity, &quot;a continuous twice differentiable function is convex ON a convex set, iff the hessian is positive semi definite on the interior of the convex set.&quot;&#xA;The feasible region where I am trying to check for its convexity its convexity is defined by..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B, N, sigma are constants &lt;/p&gt;&#xA;&#xA;&lt;p&gt;B = 20000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;N = 50&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sigma = 3.7678e-17&lt;/p&gt;&#xA;&#xA;&lt;p&gt;w, p are variables(vectors of K elements)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;w can vary from 1 to 50&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p, 0 to 46 in dbm(unit of power..)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to convert in into watt before using it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p in watt = (10^(p in dbm/10))/1000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hessian comes out to be indefinite for MANY of the feasible points..But I am not sure if I am right. &#xA;Is the problem CONVEX, can any one check please..? &lt;/p&gt;&#xA;" OwnerUserId="5387" LastActivityDate="2014-01-15T13:32:54.857" Title="Convexity Check" Tags="|optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8778" PostTypeId="1" CreationDate="2013-10-17T11:32:26.053" Score="6" ViewCount="952" Body="&lt;p&gt;For example I have 2 matrices that can't fit in RAM.&#xA;I need algorithm or library which can handle this.Preferably Matlab or Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it can be some block matrix multiplication? Also I think there is an analogy hard drive&amp;lt;-&gt;ram, gpu ram&amp;lt;-&gt;cpu ram, cpu ram&amp;lt;-&gt;cpu cache, so we can take some cpu cache optimized techniques?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems in python I can use numpy.memmap but I don't understand memory consumption of this approach and maybe it isn't optimal solution at all.&lt;/p&gt;&#xA;" OwnerUserId="3290" LastActivityDate="2013-10-21T12:31:45.663" Title="Big matrix multiplication on single machine" Tags="|matlab|python|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8779" PostTypeId="2" ParentId="8777" CreationDate="2013-10-17T11:45:28.063" Score="1" Body="&lt;p&gt;For the problem to be convex, the nonlinear expression in your first constraint has to be &lt;em&gt;concave&lt;/em&gt;. Indeed, this is the case at it essentially is a negated &lt;a href=&quot;http://mathworld.wolfram.com/RelativeEntropy.html&quot; rel=&quot;nofollow&quot;&gt;relative entropy&lt;/a&gt; (known to be convex), $w\log(1+\frac{p}{w}) = -w\log(\frac{w}{p+w})=-relentr(w,p+w)$. Alternatively, we note that it is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Convex_function&quot; rel=&quot;nofollow&quot;&gt;perspective&lt;/a&gt; of $f(p)=\log(1+p)$, and it thus follows that $wf(p/w)$ is concave.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2013-10-17T12:43:04.857" LastActivityDate="2013-10-17T12:43:04.857" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8780" PostTypeId="2" ParentId="8778" CreationDate="2013-10-17T12:50:14.393" Score="9" Body="&lt;p&gt;I think you should have a look at &lt;a href=&quot;http://www.pytables.org&quot;&gt;PyTables&lt;/a&gt;. Especially the &lt;a href=&quot;http://www.pytables.org/docs/PyData2012-NYC.pdf&quot;&gt;tutorial&lt;/a&gt; given at PyData 2012. PyTables combines hierarchical datasets with a computational engine. It uses the &lt;a href=&quot;http://blosc.pytables.org/trac&quot;&gt;Blosc&lt;/a&gt; compresser to avoid I/O bottlenecks and an optimized evaluator for expressions &lt;code&gt;tables.Expr&lt;/code&gt; (based on &lt;a href=&quot;http://code.google.com/p/numexpr/&quot;&gt;Numexpr&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-10-17T12:50:14.393" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8781" PostTypeId="2" ParentId="1639" CreationDate="2013-10-17T16:21:10.013" Score="3" Body="&lt;p&gt;My quick vote is for QtiPlot.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though it isn't perfect, it provides the best combination of GUI easiness, along with python scripting so what you're really doing then is &quot;Qtiplot/Python/(Illustrator or Inkscape)&quot;.  Qtiplot is extensible, so you can create your own scripts/macros which can run python commands to open and process data and then dump them into a table.  You can then manually or automatically plot them.  It is also possible to configure it to work with LaTeX typesetting with an internet available typesetter, or a local typesetter, so you can put math formulas in your plot titles/axes/markups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plots can then be exported to any format you want, but the format that works best is undoubtedly SVG.  The SVG files can be touched up in inkscape if you want, which also has LaTeX typesetting abilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While MatPlotLib is great, its biggest draw-back is having to hand-code absolutely everything (just like any scripting plotting solution).  QtiPlot is great because you can script/automate data processing and plotting all with great Python functionality, and then pass over a rough cut plot to a gui that will allow you to tweak it by hand and do all kinds of gui cutting/pasting/manipulating that is much faster on a case by case basis for data analysis.  You can also set dimensions precisely with the scripting, and also manipulate the dimensions with the GUI, something you can't do with matplotlib.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose it depends on how you feel about QtiPlot plotting solutions.  I have found it to be very capable, but it is all a pretty steep learning curve.&lt;/p&gt;&#xA;" OwnerUserId="5395" LastActivityDate="2013-10-17T16:21:10.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8782" PostTypeId="1" AcceptedAnswerId="8783" CreationDate="2013-10-17T16:53:02.353" Score="5" ViewCount="991" Body="&lt;p&gt;I need to find the zero of a function &lt;span class=&quot;math-container&quot;&gt;$f(\lambda)$&lt;/span&gt; which is of the form &lt;span class=&quot;math-container&quot;&gt;$\sum \frac{c_i^2}{(1+\lambda d_i)^2} -1 $&lt;/span&gt;. I tried using Newton's method, and it works sometimes, but it is highly dependent on the initial choice, and the chances of divergence are very high since the function is &amp;quot;almost&amp;quot; flat.&lt;/p&gt;&#xA;&lt;p&gt;What would be the best method to find the zero of such a function?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;For now, I replaced Newton with the secant method, and it seems to work better.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="20688" LastEditDate="2021-12-17T00:53:45.510" LastActivityDate="2022-04-29T18:24:17.060" Title="Best method to find the zero of a decreasing function numerically" Tags="|newton-method|rootfinding|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8783" PostTypeId="2" ParentId="8782" CreationDate="2013-10-17T17:48:46.783" Score="9" Body="&lt;p&gt;Off the top of my head, there are a number of things that may be going wrong. You might want to verify the following&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Are the function values or derivatives you are computing numerically stable? Is the rounding error in their evaluation smooth? You can verify this visually by plotting an extremely small interval around a known root such that the function values are $\pm \varepsilon_\mathsf{mach}$. If your function or derivative changes sign near the root, Newton's method won't work.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What termination criteria are you using? If you're looking for a root $f(\lambda)=0$ with $\lambda \in [a,b]$, do you stop when $|b-a|&amp;lt;\tau$ or when $|f(\lambda)|&amp;lt;\tau$ where $\tau$ is your tolerance? In either case, what is your rationale for your choice of $\tau$ and how does it fit with the error/interval you observe in the previous point?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Newton's method is a good hammer, but there are many functions that will look a lot like your thumb. The Secant method is a good choice for functions that are almost linear in the search interval, but will perform poorly as soon as you depart from that assumption. A good alternative is the so-called &lt;a href=&quot;http://en.wikipedia.org/wiki/False_position_method#Illinois_algorithm&quot;&gt;Illinois algorithm&lt;/a&gt;. A major advantage of interval methods is that they don't rely on the function or any of its derivatives being &lt;em&gt;numerically&lt;/em&gt; smooth as you can run them until $fl(f(\lambda))=0$ or until the search interval $[a,b]$ is numerically zero, i.e. $fl(a)=fl(b)$. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-10-17T17:48:46.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8784" PostTypeId="1" CreationDate="2013-10-17T18:04:12.343" Score="1" ViewCount="1435" Body="&lt;p&gt;I would like to simulate a capacitor in 2d with freefem++. This is the code I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int C0=0;&#xA;int C1=1;&#xA;int C2=2;&#xA;&#xA;&#xA;border C01(t=0,4.25){x=t; y=0; label=C0;}&#xA;border C02(t=0,2.25){x=4.25; y=t;label=C0;}&#xA;border C03(t=4.25,0){x=t; y=2.25; label=C0;}&#xA;border C04(t=2.25,0){x=0; y=t; label=C0;}&#xA;&#xA;&#xA;border C11(t=0.25,0){x=1.75+t; y=0.75; label=C1;}&#xA;border C12(t=0,0.75){x=1.75; y=0.75+t; label=C1; }&#xA;border C13(t=0,0.25){x=1.75+t; y=1.50;label=C1; }&#xA;border C14(t=0,0.75){x=2; y=1.5-t;  label=C1;}&#xA;&#xA;&#xA;border C21(t=0.25,0){x=2.25+t; y=0.75; label=C2;}&#xA;border C22(t=0,0.75){x=2.25; y=0.75+t; label=C2; }&#xA;border C23(t=0,0.25){x=2.25+t; y=1.50;label=C2; }&#xA;border C24(t=0,0.75){x=2.5; y=1.5-t;  label=C2;}&#xA;&#xA;Th=buildmesh(C01(50)+C02(50)+C03(50)+C04(50)+C11(+20)+C12(+40)+C13(+20)+C14(+40)+C21(+20)+C22(+40)+C23(+20)+C24(+40));&#xA;&#xA;&#xA;fespace Vh(Th,P2);&#xA;Vh uh,vh;&#xA;&#xA;real voltc1=1;&#xA;real voltc2=-1;&#xA;&#xA;//1 e -1 volt dirichlet conditions&#xA;problem Poisson(uh,vh)=int2d(Th)(dx(uh)*dx(vh)+dy(uh)*dy(vh))+on(C0,uh=0)+on(C1,uh=voltc1)+on(C2,uh=voltc2);&#xA;&#xA;Poisson;&#xA;&#xA;plot(uh,wait=true,fill=true,value=1);&#xA;&#xA;&#xA;real energy=int1d(Th) ((dx(uh))^2+(dy(uh))^2);&#xA;&#xA;cout&amp;lt;&amp;lt;&quot;Energy (double)&quot;&amp;lt;&amp;lt;endl&amp;lt;&amp;lt;&quot;E =  &quot;&amp;lt;&amp;lt;energy&amp;lt;&amp;lt;endl;&#xA;&#xA;cout&amp;lt;&amp;lt;&quot;Capacitance&quot;&amp;lt;&amp;lt;endl&amp;lt;&amp;lt;&quot;C =  &quot;&amp;lt;&amp;lt;energy/(voltc1-voltc2)^2&amp;lt;&amp;lt;endl;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Runnng this code it gives me a value for the capacitance of 36.319 F. Is it this value reliable? Using the simplest formula $C=\epsilon \frac{A}{d}$ with $A$ area and $d$ the distance between the two plates the result is completely different. What is wrong in all of this?&lt;/p&gt;&#xA;" OwnerUserId="5396" LastActivityDate="2013-10-18T14:59:20.330" Title="Capacitance in freefem++" Tags="|pde|finite-element|numerical-analysis|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8785" PostTypeId="2" ParentId="8782" CreationDate="2013-10-17T19:30:20.590" Score="5" Body="&lt;p&gt;Can you bound your zero? If you can come up with a good heuristic for the approximate location of the zero and then compute bounds for it over an interval that is not too large, the performance of many methods is improved. Here's some Matlab code I use to perform basic &lt;a href=&quot;https://github.com/horchler/SHCTools/blob/master/SHCTools/stoneholmes/private/bracketroot.m&quot; rel=&quot;noreferrer&quot;&gt;root bracketing for monotonic functions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of interval methods, @Pedro's suggestion to use the Illinois algorithm is a good simple possibility. If you want a big hammer, there is of course &lt;a href=&quot;http://en.wikipedia.org/wiki/Brent%27s_method&quot; rel=&quot;noreferrer&quot;&gt;Brent's method&lt;/a&gt;, which is more complex to implement. It's the basis of Matlab's &lt;code&gt;fzero&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="4474" LastEditDate="2013-10-17T19:49:21.043" LastActivityDate="2013-10-17T19:49:21.043" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8787" PostTypeId="1" AcceptedAnswerId="8788" CreationDate="2013-10-17T23:34:10.353" Score="1" ViewCount="77" Body="&lt;p&gt;I have a function $f$ I'd like to determine numerically and I have a bunch of $(x, y)$ pairs which approximate the function in the following sense: all of the points satisfy $f(x) \leq y$, most of the points have $f(x) = y$ (at least to the accuracy I need), a few of the points have $y$ significantly bigger than $f(x)$. (The points come from a simulated annealing which occasionally fails.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that $f$ can be well approximated by a low-order polynomial, so I'd like to find such which lies below all of the points and which hits as many as possible, but does not mind missing a few. That's a rather woolly thing to want, so I guess I should look for the &lt;em&gt;largest&lt;/em&gt; which lies below all of the points, where &lt;em&gt;largest&lt;/em&gt; would be with respect to the $L^p$ norm, say $p = 1$ of $2$, on an interval containing all of the $x$s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers or references on how to find such a polynomial would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1400" LastActivityDate="2013-10-18T06:00:01.210" Title="Determine low-order polynomial lower bound" Tags="|polynomials|approximation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8788" PostTypeId="2" ParentId="8787" CreationDate="2013-10-18T06:00:01.210" Score="1" Body="&lt;p&gt;In the language of regression, you want to be robust against outliers. Penalizing the 1-norm of the approximation error is a simple way to achieve this. This penalty, together with the constraint that the approximation satisfies $f(x_i)\leq y_i$ leads to a linear program (can probably be solved more efficient with specialized methods, but this is the obvious model). Proof-of-concept code below in MATLAB (using the toolbox YALMIP (disclaimer, developed by me))&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = (-1:0.1:1)';&#xA;y = x.^2+x.^3+.5*x.^4;&#xA;y(3) = 3;&#xA;y(11)= 7;&#xA;plot(x,y,'*')&#xA;&#xA;sdpvar a b c&#xA;yApprox = a+b*x+c*x.^2;&#xA;Error = norm(y-yApprox,1);&#xA;Below = [yApprox &amp;lt;= y];&#xA;solvesdp(Below,Error);&#xA;hold on;&#xA;plot(x,double(yApprox))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the model and data here basically yields the same solution if you switch to a quadratic objective&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error = (y-yApprox)'*(y-yApprox);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The reason is that the upper bound constraint on the approximation protects the solution from being pulled towards the outliers (which typically happens when using quadratic error measures)&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-10-18T06:00:01.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8789" PostTypeId="1" CreationDate="2013-10-18T12:06:02.997" Score="7" ViewCount="1987" Body="&lt;p&gt;What open source tool can be recommended for solving 2D instationary Navier Stokes equations (in simple geometries, but with high Reynolds numbers)? Most packages I found, I'm not very lucky with. OpenFOAM seems to be very powerful, but I think it would take some time to learn using it. FEniCS has a Navier Stokes demo, but only for the stationary case and I'm not sure if it will work for large Reynolds number. Are there any other good options? In the best case, I would like not to write new code, but just provide the geometry, boundy conditions, and data for the initial timestep.&lt;/p&gt;&#xA;" OwnerUserId="1090" LastActivityDate="2013-10-19T21:52:27.370" Title="Open source codes for 2D instationary Navier Stokes equations" Tags="|fluid-dynamics|navier-stokes|open-source|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8790" PostTypeId="2" ParentId="8784" CreationDate="2013-10-18T12:26:38.127" Score="1" Body="&lt;p&gt;One of the problems (except for the 1d integration in the code) is that you use a formula that is not valid for your test case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, the formula that you use is made for two parallel plates&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;infinitly thin&lt;/li&gt;&#xA;&lt;li&gt;with a finite area&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;so, they should be like the two super thin slices of bread of a square burger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you represent with your 2D case are two parallel plates&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;which are thick (0.25 thick)&lt;/li&gt;&#xA;&lt;li&gt;with an inifite area, because you neglect the third dimension (so it's like it would be infinite in that direction).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So the formula just does not apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check your 2D implementation, you should better create a test for the coaxial cable (formula found &lt;a href=&quot;http://en.wikipedia.org/wiki/Capacitance&quot; rel=&quot;nofollow&quot;&gt;here on wiki&lt;/a&gt;) for which it is much easier to restrict to a 2D test case.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-10-18T12:26:38.127" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8791" PostTypeId="2" ParentId="8789" CreationDate="2013-10-18T13:13:54.173" Score="4" Body="&lt;p&gt;Before I answer your question, I just want to clarify one key point:  Solving PDE's, like Navier Stokes equations, requires a two step process:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Geometry Discretization (meshing)  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Solution of the linear equations  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Usually this is done with two separate programs.  Meshing can be a very costly operation by itself and should be done carefully according to the user's accuracy requirements.  There are many freeware programs that accomplish this.  If your geometry is not too complicated, and prefer using a GUI interface, I'd recommend something like NETGEN or GMSH (both available as apps on Ubuntu).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solvers also require a lot of user input beyond the basic boundary and initial conditions.  Particularly for the Navier Stokes equations, you'll have to the specify the type of time stepping scheme and parameters for the linear solver.  Both of these can have a significant impact on the accuracy of your solution.  There are, in fact, many many many different freeware solvers out there for the navier stokes equations.  The trick is finding a solver that can input your mesh data type.  If I'm not mistaken, I believe NETGEN and GMSH both produce file types that can be imported onto ELMER, a free finite element solver available also on UBUNTU.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, of course, many other options and probably better ones that what I've recommended here.  But this should be able to get you started.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-10-18T13:31:58.200" LastActivityDate="2013-10-18T13:31:58.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8792" PostTypeId="2" ParentId="8789" CreationDate="2013-10-18T13:15:20.833" Score="2" Body="&lt;p&gt;If this is all you need, you might have a look at &lt;a href=&quot;http://www.freefem.org/ff++/&quot; rel=&quot;nofollow&quot;&gt;FreeFem++&lt;/a&gt;. There are several working examples (look at the &lt;a href=&quot;http://www.freefem.org/ff++/ftp/freefem++doc.pdf&quot; rel=&quot;nofollow&quot;&gt;manual&lt;/a&gt;, sections 3.9 for a simple example, 3.11 with turbulence), it is very easy to use and you can change the geometry easily, without having to create a mesh with another software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(However, you have to stick with 2D, and it's not a real library).&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-10-18T13:15:20.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8793" PostTypeId="2" ParentId="8784" CreationDate="2013-10-18T13:32:35.537" Score="0" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PKyvy.jpg&quot; alt=&quot;Coaxial mesh&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use the new geometry for the coaxial case defined like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;border C01(t=0,5){x=t; y=0; label=C0;}&#xA;border C02(t=0,5){x=5; y=t;label=C0;}&#xA;border C03(t=5,0){x=t; y=5; label=C0;}&#xA;border C04(t=5,0){x=0; y=t; label=C0;}&#xA;border C11(t=0,2*pi){x=2.5+sin(t);y=2.5+cos(t);label=C1;};&#xA;border C21(t=0,2*pi){x=2.5+1.1*sin(t);y=2.5+1.1*cos(t);label=C1;};&#xA;border C31(t=0,2*pi){x=2.5+0.3*sin(t);y=2.5+0.3*cos(t);label=C2;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I obtained with the same code a value for the capacitance of $C=6.96877 F$. I still don't see the relation with the formula in the wiki...&lt;/p&gt;&#xA;" OwnerUserId="5396" LastEditorUserId="5396" LastEditDate="2013-10-18T14:46:11.157" LastActivityDate="2013-10-18T14:46:11.157" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="8795" PostTypeId="2" ParentId="8784" CreationDate="2013-10-18T14:59:20.330" Score="0" Body="&lt;p&gt;Ok for the plate capacitor I solved following the suggestion of  @rchilton1980 using a new geometry like this one. The thickness is really small compared to the side of the plate so the approximation works and I obtained a capacitance of around 3.&lt;img src=&quot;https://i.stack.imgur.com/NO5FJ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="5396" LastActivityDate="2013-10-18T14:59:20.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8796" PostTypeId="2" ParentId="8789" CreationDate="2013-10-18T18:01:32.590" Score="3" Body="&lt;p&gt;OpenFOAM has turbulence models built in, which is helpful. I think it's easier to use OpenFOAM as a black-box solver than to modify its internals (which I found was very difficult given the lack of developer documentation and OpenFOAM coding style). If you're solving the Navier-Stokes equations, there's probably already a solver available in OpenFOAM, in which case, most of the work would be importing a geometry and writing input files. The documentation for these steps is decent, so OpenFOAM might be a good option for you if you want to use a finite volume code.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-10-18T18:01:32.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8797" PostTypeId="2" ParentId="8782" CreationDate="2013-10-18T20:15:16.030" Score="4" Body="&lt;p&gt;Using a line search or trust region method along with Newton's method typically does the trick of finding the zero reliably. Check any advanced numerical methods or optimization book.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-18T20:15:16.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8798" PostTypeId="2" ParentId="8775" CreationDate="2013-10-18T20:19:02.620" Score="2" Body="&lt;p&gt;deal.II is another finite element library, see &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/&lt;/a&gt; . It has tutorial programs for both fluid flow and acoustics. (Disclaimer: I'm one of the authors of this library.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-18T20:19:02.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8799" PostTypeId="1" AcceptedAnswerId="8805" CreationDate="2013-10-18T20:28:15.477" Score="2" ViewCount="897" Body="&lt;p&gt;I am trying to reformulate the following problem to be solved efficiently (by MOSEK)&#xA;$$&#xA;\min_{X} \text{Tr}(CX)+\lambda\sum_{i,j}|x_{i,j}| \\&#xA;\text{s.t.} \quad ||X||_F\le1 \quad \text{and} \quad X\ge 0 \text{,}&#xA;$$&#xA;where Tr() is the trace, $C \in \mathcal{S}^{n\times n}$ a real symmetric matrix, $X \in \mathcal{S}_{+}^{n\times n}$, the unknown positive-semidefinite (PSD) matrix, $\lambda$ the regularization parameter, and $x_{i,j}$ the $(i,j)^{th}$ entry of $X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two formulations: one obtained by myself and one obtained via YALMIP. The one obtained by YALMIP is solved more efficiently, and I'd like to know how that specific conversion is done. Below I present both formulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity in presentation, my formulation below does not show $x_{i,j}=x_{j,i}$, but this is incorporated in actual implementation. My simple formulation is&#xA;$$&#xA;\min_{X} \text{Tr}(CX)+\lambda\sum_{i,j}t_{i,j} \\&#xA;\text{s.t.}\quad&#xA;-t_{i,j}+x_{i,j}&amp;lt;0 \\&#xA;-t_{i,j}-x_{i,j}&amp;lt;0 \\&#xA;||X||_F\le 1 \\&#xA;X\ge 0 \text{,} \\&#xA;$$&#xA;which involves $n(n+1)$ inequalities, a quadratic cone, a PSD cone, and $n(n+1)$ variables. (I hope I have these numbers right.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;YALMIP produces the following problem for MOSEK (using &lt;code&gt;export.m&lt;/code&gt;), which I assume is equivalent up to some simple conversion because its solution is not the solution for the original problem. Because my understanding of this formulation is limited, I will use $n=2$ as a specific example.&#xA;$$&#xA;\min_{X} t \\&#xA;\text{s.t.}\quad  t\ge \sqrt{y_{10}^2+y_{11}^2+y_{12}^2} \\&#xA;y_{10}=c_{1,1}+y_1-y_2-x_{1,1} \\&#xA;y_{11}=\sqrt{2}\,c_{1,2}+\frac{1}{\sqrt{2}}(y_3-y_4+y_5-y_6)-\sqrt{2}\,x_{1,2} \\&#xA;y_{12}=c_{2,2}+y_7-y_8-x_{2,2} \\&#xA;y_{i}+y_{i+1}=\lambda \quad \text{for}\quad i=1,3,5,7 \\&#xA;y_i\ge 0 \quad \text{for}\quad i=1,2,...,8 \\&#xA;X\ge 0 \text{,}&#xA;$$&#xA;which involves a quadratic cone, a PSD cone, about $n(n+1)$ equalities, $n(n+1)$ inequalities, and $3\times n(n+1)/2$ variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;YALMIP problem is solved much more efficiently. It seems like it tries to complete a square, and implement $\lambda|x_k|$ by $(y_i-y_{i+1})x_k$. Any insight on how the YALMIP problem is formulated will be very helpful. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="5403" LastActivityDate="2013-10-20T15:49:50.587" Title="Efficient formulation of an SDP involving L1 norm" Tags="|optimization|efficiency|semidefinite-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8800" PostTypeId="2" ParentId="8737" CreationDate="2013-10-18T21:06:52.200" Score="2" Body="&lt;p&gt;Since &lt;code&gt;pdepe&lt;/code&gt; accepts systems of PDEs through vector-valued capacity, flux, and source terms, one way to accommodate your request would be to set the fluxes for all of the $\rho$ variables equal to zero. The capacity terms for the $\rho$ variables will all be 1, and the source terms for each variable are the non-flux terms on the right-hand side (the $\rho$ variables have no flux terms, so for these variables, the source term is also the &lt;em&gt;only&lt;/em&gt; term on the right-hand side of these equations). Without knowing more about the methods used by &lt;code&gt;pdepe&lt;/code&gt;, it's difficult to say whether or not this kludge will be numerically stable, but my guess is that it would probably work.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-10-18T21:06:52.200" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8801" PostTypeId="2" ParentId="8789" CreationDate="2013-10-18T22:36:59.220" Score="1" Body="&lt;p&gt;I can recommend the one that I'm working on - &lt;a href=&quot;http://sourceforge.net/projects/caffasst/&quot; rel=&quot;nofollow&quot; title=&quot;caffa-sst&quot;&gt;caffa-sst&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a 2d finite volume code written in Fortran. It works for structured, non-orthogonal body-fitted meshes (mesh generator included).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started with the CAFFA code from Ferziger and Peric book, and added $k-\omega$ SST model by Menter, Automatic Wall Treatment, least-square gradient calculation, flux limited schemes for convection terms (like MUSCL), Paraview output, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Haven't work on it for almost a year, but I still plan to upgrade it more.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2013-10-18T22:36:59.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8804" PostTypeId="2" ParentId="8789" CreationDate="2013-10-19T21:52:27.370" Score="1" Body="&lt;p&gt;I would suggest &lt;a href=&quot;http://gfs.sourceforge.net/wiki/index.php/Main_Page&quot; rel=&quot;nofollow&quot;&gt;gerris&lt;/a&gt;. It does not have a very steep learning curve for simple problems/configurations and it is still being actively developed. It also has adaptive mesh refinement and you can import complex geometry from CAD software. For simpler geometry like boxes, cylinders etc. you can directly define them in the input file. Try starting with &lt;a href=&quot;http://gfs.sourceforge.net/tutorial/tutorial/tutorial1.html#htoc10&quot; rel=&quot;nofollow&quot;&gt;tutorial&lt;/a&gt; page and you may want to check the examples section for more complex problems. It also comes as a package in debian systems so installation should not be a problem if you are on a linux system.&lt;/p&gt;&#xA;" OwnerUserId="5411" LastActivityDate="2013-10-19T21:52:27.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8805" PostTypeId="2" ParentId="8799" CreationDate="2013-10-20T15:49:50.587" Score="2" Body="&lt;p&gt;YALMIP derives the model you do (except adding a dummy variable leading to $||X||_F\leq z, z\leq 1$ and adding some redundant constraints). However, YALMIP works in the dual space, i.e., $X$ is parameterized by variables, so to understand the model, it only makes sense to it interpret it as such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if you look at the data in representing the dual $\max b^Ty \text{ subject to }C-A^Ty \succeq 0$  for your case&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C = [7 2;2 19];&#xA;X = sdpvar(2);&#xA;obj = trace(C*X) +  sum(sum(abs(X)))&#xA;Constraint = [norm(X,'fro') &amp;lt;= 1, X&amp;gt;=0]&#xA;M = export(Constraint,obj,sdpsettings('solver','sedumi'));&#xA;M.b'&#xA;-7    -4   -19    -1    -1    -1    -1     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem has three variables defining $X$, 4 variables to model the absolute values $t_{ij}$ (that's a performance bug, should be three, redundancy due to symmetry is missed) and one variable to represent the Frobenius norm. You see directly how these enter the objective through $b$. Studying $A$ and  $C$ reveals that it is exactly the model you expect (9 linear inequalities to describe the 4 absolute values and the upper bound on the norm, involving YALMIPs dummy variable, an SOCP, and an SDP). Your confusion is most likely caused by the fact that you are trying to match the data to the primal representation  of an SDP, $\min CX \text{ subject to } Ax=b,X\succeq 0$ thus giving a non-intuitive model.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-10-20T15:49:50.587" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8806" PostTypeId="1" AcceptedAnswerId="8834" CreationDate="2013-10-20T19:07:51.060" Score="7" ViewCount="804" Body="&lt;p&gt;I'm currently working on my MS in CS and have developed an interest in astrophysics. Luckily one of my professors is a astrophysicist and is currently doing research through computational physics and he agreed to help me get started.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently reading through Tao Pang's &quot;An Introduction to Computational Physics&quot; (&lt;a href=&quot;http://www.physics.unlv.edu/~pang/cp2_cont.html&quot;&gt;TOC&lt;/a&gt;), but the book's lack of detail on the mathematical methods used leaves me pretty lost at times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, are there any books that cover the topics in a more in-depth fashion or any books that I could use as a reference for the mathematical methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The best way to describe my mathematical background is that I've studied calculus, linear algebra and differential equations, but remember very little.&lt;/p&gt;&#xA;" OwnerUserId="5415" LastEditorUserId="541" LastEditDate="2013-10-21T19:03:34.370" LastActivityDate="2014-10-20T10:40:35.310" Title="Introductory book on computational physics" Tags="|reference-request|computational-physics|" AnswerCount="4" CommentCount="7" ClosedDate="2014-10-20T17:19:40.890" ContentLicense="CC BY-SA 3.0" />
  <row Id="8807" PostTypeId="2" ParentId="8769" CreationDate="2013-10-20T21:05:51.290" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;Step 1: Hack inside of the PETSc matrix object and add an MPI_WIN_CREATE call when the data is allocated.&lt;/li&gt;&#xA;&lt;li&gt;Step 2: Write an address translation routine that converts from matrix row and column coordinates (i,j) to a process and a remote offset to the window (proc,offset).  A ranged version of this function is highly desirable.&lt;/li&gt;&#xA;&lt;li&gt;Step 3: Use the function from Step 2 to do MPI_{PUT,GET} against your PETSc matrix.  This is probably best done with new API functions for said matrix object.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the case of dense matrices, you are basically implementing Global Arrays 3.0 :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want an answer to your ultimate question (&quot;How do I do one-sided MPI communication&quot;), please make a separate post about that so it is clear to others what the question really is.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-10-20T21:05:51.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8808" PostTypeId="1" CreationDate="2013-10-20T21:12:17.983" Score="2" ViewCount="386" Body="&lt;p&gt;I'm writing prototypes for solving the Liouville Equations with Mathematica and C++. Perhaps the question about this may not be suited for this forum in a strict way, but it suits the people here because they're researchers who work with this kind of problems, I presume, and hope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm looking for a model to store and manipulate angular momenta and density matrices that support:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Storing angular momentum number and projection as quantum states (something which internally would look like a bra/ket which can take more than 1 quantum number (matrices can take only 1 quantum number per element);&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;using Clebsch-Gordan conversions efficiently to switch between the coupled and uncoupled representations;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;building Hamiltonian operators to act on those states;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;building density matrices for a subset of states.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In C++, I can imagine how this would look like (though I haven't worked it out), and since I'm a beginner in this kind of numerical calculations with Quantum Mechanics (everything I've done so far is classical), I'm testing my work in Mathematica, which is a functional-programming language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In C++, I could, probably, create a class which represents an angular momentum state (mag. and proj.), and then have other objects of this class get combined in another class which will store their pointers, and that would define a mixed state with multiple quantum numbers. I'm not sure whether this could work, but it's a problem which I have to deal with later, till I get familiar with this problem practically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm looking for a way to do those kinds of operations in Mathematica. What would you suggest? Are there tools for those calculations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though a word about how to do it in C++ (to get an idea) wouldn't be so bad :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any efforts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note to mentors: add the keyword &quot;Density-matrix&quot; and &quot;clebsch-gordan&quot; to the list of keywords.&lt;/p&gt;&#xA;" OwnerUserId="5410" LastEditorUserId="8168" LastEditDate="2014-05-11T22:09:57.190" LastActivityDate="2014-05-11T22:09:57.190" Title="A programming model for Quantum Mechanics angular momenta in Mathematica" Tags="|matrix|c++|quantum-mechanics|mathematica|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8809" PostTypeId="1" CreationDate="2013-10-20T22:48:49.083" Score="1" ViewCount="3016" Body="&lt;p&gt;I've seen that in Newton's method for interpolating polynomials, the coefficients can be found algorithmically using (in Python-ish): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = Y_DataPoints.copy()&#xA;m = length(X_DataPoints)&#xA;for k in range(1,m):&#xA;       a[k: m] = (a[k:m] - a[k-1]) / (X_Data[k:m] - X_Data[k-1]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I don't really understand the model is subtracting all points past $k_{i}$ by $k_{i-1}$. It seems like you would only subtract $k_{i}$  by $k_{i-1}$, not the entire vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone shed some light on this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Spelling&lt;/p&gt;&#xA;" OwnerUserId="5417" LastEditorUserId="5417" LastEditDate="2013-10-21T14:22:49.333" LastActivityDate="2013-10-21T14:22:49.333" Title="Newton's method in interpolation" Tags="|python|numerical-analysis|interpolation|newton-method|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8810" PostTypeId="1" CreationDate="2013-10-20T23:07:18.067" Score="4" ViewCount="298" Body="&lt;p&gt;I am currently testing some new (embedded) processors for their double-precision numeric performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FPU capabilities vary wildly between platforms; For example, the ARM Cortex-A8 has a double precision FPU, takes 9 cycles per operation. The Cortex-A9 also takes 9 cycles, but is able to be fully pipelined, whereas the Cortex-A15 apparently (finally!) has a single cycle double.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up until now, I have just used linear algebra (e.g. Eigen or ATLAS) for benchmarking, which represents a good subset of our own computational demands, but is deficient as a benchmark for &lt;a href=&quot;http://www.utk.edu/tntoday/2013/07/10/professor-jack-dongarra-announces-supercomputer-benchmark/&quot; rel=&quot;nofollow&quot;&gt;well&lt;/a&gt;-&lt;a href=&quot;http://en.wikipedia.org/wiki/LINPACK_benchmarks#Criticism&quot; rel=&quot;nofollow&quot;&gt;documented&lt;/a&gt; &lt;a href=&quot;http://blogs.mathworks.com/cleve/2013/06/24/the-linpack-benchmark/#880d5a73-d032-4e5f-be3c-29ae2d792957&quot; rel=&quot;nofollow&quot;&gt;reasons&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was looking at something like the &lt;a href=&quot;http://en.wikipedia.org/wiki/HPC_Challenge_Benchmark&quot; rel=&quot;nofollow&quot;&gt;HPC Challenge&lt;/a&gt;, or one of the &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/SLIDES/isc-2013-new-metric.pdf&quot; rel=&quot;nofollow&quot;&gt;many others&lt;/a&gt; (see slide 15), but it's a bit like giving a starving man a menu!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what numerical benchmarks, if any, meet the following criteria?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Which other benchmarks (even if flawed/limited) are generally accepted?  A good example for &quot;general acceptance&quot; would be the ability to find results for common (commodity) processors without much time investment using google.&lt;/li&gt;&#xA;&lt;li&gt;As I am generally interested in embedded systems, I much prefer applications that can cleanly be cross-compiled - Self-tuning packages are a pain on embedded systems (e.g. ATLAS is a good example)&lt;/li&gt;&#xA;&lt;li&gt;Even better, packages that only minimally depend on operating system services (or better yet, just libc or similar) allows for testing on a much wider range of systems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update: Some context&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linear algebra does indeed cover a large subset of our computational demands (which was listed in the OP). It does, however, benefit greatly where the operations can be pipelined. This isn't universally true for &lt;em&gt;all&lt;/em&gt; our computational demands - there are multiple smaller solvers and algorithms of various types scattered throughout our code base. I doubt I am aware of all of them. Plus, since we typically maintain an embedded system for 5 years (or more!) after release, our current suite of applications may not represent future demands, the nature of which I cannot entirely predict (Note 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second reasons is than I cannot purchase development kits, port code, build a cross-compiler, run benchmarks, etc for the literally hundreds of different &lt;a href=&quot;http://en.wikipedia.org/wiki/System_on_a_chip&quot; rel=&quot;nofollow&quot;&gt;SoCs&lt;/a&gt; that exist on the market.  If there are few, well-known benchmarks that become ubiquitous (&lt;a href=&quot;http://www.eembc.org/coremark/&quot; rel=&quot;nofollow&quot;&gt;Coremark&lt;/a&gt; is an example, but it is not numeric-specific), then it makes the job of processor selection that much easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note 1:&lt;/strong&gt; A good example is half-way through the life of a product, a floating point application that had significant amounts of branching was developed (by an ex-HPC guy, so I hope he know what he was doing!).  Had we selected a processor that was very heavily pipelined, we may have been caught out.&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="761" LastEditDate="2013-10-21T07:15:51.760" LastActivityDate="2013-10-21T07:15:51.760" Title="Useful well-known numeric performance benchmarks other than Linear Algebra?" Tags="|benchmarking|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8811" PostTypeId="2" ParentId="8775" CreationDate="2013-10-21T01:28:22.107" Score="2" Body="&lt;p&gt;There are a large number of &lt;a href=&quot;https://www.google.com/search?q=fdtd%20acoustic%20simulation&amp;amp;oq=FTDT%20ac&amp;amp;aqs=chrome.1.69i57j0l3.4210j0&amp;amp;sourceid=chrome&amp;amp;ie=UTF-8#q=fdtd%20acoustic%20&quot; rel=&quot;nofollow&quot;&gt;FDTD solvers&lt;/a&gt; targeting acoustic problems.  I would start by playing with these, as the boundary conditions you need to impose in acoustics are a bit complicated, and these packages are likely to be better suited to playing with them.  &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2013-10-21T01:28:22.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8812" PostTypeId="2" ParentId="8810" CreationDate="2013-10-21T02:12:37.140" Score="1" Body="&lt;p&gt;I would suggest to look at the SPEC CPU2006 benchmarks, specifically the Floating Point benchmarks. They are many and complex, so you may not necessarily want to run them yourself (the also cost money) but you may be able to find results at &lt;a href=&quot;http://www.spec.org&quot; rel=&quot;nofollow&quot;&gt;http://www.spec.org&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-21T02:12:37.140" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8813" PostTypeId="2" ParentId="8806" CreationDate="2013-10-21T07:45:57.680" Score="5" Body="&lt;p&gt;&lt;strong&gt;Short answer&lt;/strong&gt;: No, at least not that I know of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a Computer Scientist who, for the past year and a half, has been working with Astrophysicists on writing faster simulation codes. I spent quite a bit of time looking for books on the topic to provide some sort of overview, and found nothing convincing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of what I now know about Astrophysics, I learnt from asking Astrophysicists directly. There are a number of good review papers on specific topics, e.g. Daniel Price's papers on &lt;a href=&quot;http://en.wikipedia.org/wiki/Smoothed-particle_hydrodynamics&quot;&gt;SPH&lt;/a&gt;, e.g. &lt;a href=&quot;http://scholar.google.ch/scholar?cluster=15225120919672653172&amp;amp;hl=en&amp;amp;as_sdt=0,5&quot;&gt;here&lt;/a&gt;,  or &lt;a href=&quot;http://scholar.google.ch/scholar?cluster=5726569702248161437&amp;amp;hl=en&amp;amp;as_sdt=0,5&quot;&gt;Walter Dehnen's papers&lt;/a&gt; on N-body solvers, e.g. &lt;a href=&quot;http://en.wikipedia.org/wiki/N-body_simulation&quot;&gt;here&lt;/a&gt;, but I still rely on my colleagues in that field for the details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I think you should keep in mind, though, is that most of the interesting computational problems don't involve that much physics or even maths, but algorithms and data structures. Keep in mind that most physics codes (or books on physics codes) were written not by Computer Scientists, but by Physicists. That doesn't necessarily mean they are bad, but given the choice, would you rather buy a physics book written by a Computer Scientist, or by a Physicist?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a Computer Scientist, you have an edge with regards to algorithms and data structures, and there are loads of low-hanging fruit there. In most of my own work in this area, I have not had to tweak the physics of any problem at all, just the algorithms implementing them. Simply getting the algorithms right can already lead to &lt;a href=&quot;https://www.dur.ac.uk/pedro.gonnet/?p=290&quot;&gt;massive speedups&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In summary&lt;/strong&gt;: If you have close collaborators who are Astrophysicists, rely on them to get the physics right, but keep in mind that you will probably be able to make the largest contributions in the &lt;em&gt;algorithms and data structures&lt;/em&gt; used to implement said physics.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-10-21T07:45:57.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8814" PostTypeId="1" AcceptedAnswerId="8820" CreationDate="2013-10-21T09:05:07.963" Score="0" ViewCount="323" Body="&lt;p&gt;I have a set of points &lt;code&gt;P1(x1,y1), P2(x2,y2), P3(x3,y3), P4(x4,y4), P5(x5,y5)&lt;/code&gt; in the Cartesian plane. I am trying to find the closest point to P1 based on the orientation and Euclidean distance. What I mean is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that P3 is the closet to P1 and P5 is the second closest, but the angle between P1 and P3 is greater than the angle between P1 and P5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I find this balance between the points?&lt;/p&gt;&#xA;" OwnerUserId="4918" LastEditorUserId="4474" LastEditDate="2013-10-21T13:45:21.400" LastActivityDate="2013-10-21T17:41:17.233" Title="Find closest Point based on orientation and Euclidean distance" Tags="|heuristics|nearest-neighbors|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8815" PostTypeId="2" ParentId="8778" CreationDate="2013-10-21T12:31:45.663" Score="2" Body="&lt;p&gt;More info about your matrix type would help.&#xA;The easiest way to work with large matrices like this is to distribute them across multiple machines and use ScaLAPACK to do the operations in parallel. It will be faster too. If you need to do it on one machine, the out-of-core techniques you were alluding to will work. pytables mentioned by mrgloom supports this. Generally you want to take advantage of the nature of your matrix. &lt;/p&gt;&#xA;" OwnerUserId="3450" LastActivityDate="2013-10-21T12:31:45.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8816" PostTypeId="2" ParentId="8774" CreationDate="2013-10-21T13:50:50.283" Score="5" Body="&lt;p&gt;I have just corresponded with the author of the software, and my suspicions were confirmed.  The spatial stepsize for $n$ equispaced intervals for this problem should be $$\frac{x_{n+1}-x_0 }{n}=\frac{y_{n+1}-y_0}{n}=\frac{2}{n}$$.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-10-21T13:50:50.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8817" PostTypeId="2" ParentId="8809" CreationDate="2013-10-21T14:04:01.750" Score="2" Body="&lt;p&gt;If you compute the coefficients of the Newton polynomial via &lt;a href=&quot;http://de.wikipedia.org/wiki/Polynominterpolation#Bestimmung_der_Koeffizienten%3a_Schema_der_dividierten_Differenzen&quot; rel=&quot;nofollow&quot;&gt;the scheme of divided differences&lt;/a&gt; linewise (what is often a good choice since then one can simply add additional data), in every line, you have to compute the differences with a fixed &lt;code&gt;X_data&lt;/code&gt; point. Also, depending on how you store the &quot;new&quot; abscissae in &lt;code&gt;Y_DataPoints&lt;/code&gt;, in every line you have to take the differences with a fixed function value.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-10-21T14:04:01.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8819" PostTypeId="2" ParentId="8808" CreationDate="2013-10-21T14:34:03.670" Score="1" Body="&lt;p&gt;How about the C++ (with python bindings) Gamma library for NMR simulations:&#xA;&lt;a href=&quot;http://scion.duhs.duke.edu/vespa/gamma&quot; rel=&quot;nofollow&quot;&gt;The Gamma Library&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;http://scion.duhs.duke.edu/vespa/gamma/wiki/GammaDetailedDescription&quot; rel=&quot;nofollow&quot;&gt;GammaDetailedDescription&lt;/a&gt;, the authors mention the various components of the library, which in turn would solve many aspects of your question.&lt;/p&gt;&#xA;" OwnerUserId="5419" LastActivityDate="2013-10-21T14:34:03.670" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8820" PostTypeId="2" ParentId="8814" CreationDate="2013-10-21T17:41:17.233" Score="0" Body="&lt;p&gt;Unless you have a huge number of points, you can just do a linear search through the sorted list to evaluate your error metric, trying to find the smallest value. You might be able to do slightly better if you sort them all by distance away from your query point, in increasing order, then you likely only have to search the beginning portion of the sorted list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as I mentioned above in the comments, you don't seem to know exactly how you want to gauge &quot;nearness,&quot; so I can't give you a more detailed solution.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-21T17:41:17.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8821" PostTypeId="1" CreationDate="2013-10-21T17:49:34.023" Score="3" ViewCount="265" Body="&lt;p&gt;Let $D = \left\{0, t_1, t_2, \ldots, t_n\right\} \times [0,1]$&#xA;and &#xA;$$&#xA;f:  D\to [0,1],&#xA;$$ be a function of time and a one-dimensional space.&lt;br&gt;&#xA;There is no analytical formula for $f$, but $f(t_i, \cdot)$ is Lipschitz.&lt;br&gt;&#xA;The function $f$ can be computed with arbitrary precision&#xA;at a finite number of points $\{x_{t_i, j}\}_{j=0}^m$, $(t_i, j) \in D$.&#xA;Alternatively, the Fourier coefficients &#xA;$$&#xA;\hat f(t_i,k) = \int_0^1 f(t_i,x) e^{2\pi i kx} dx&#xA;$$ &#xA;can be computed for each time slice $f(t_i,\cdot)$, up to a finite order $K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the time slices $f(t_i,\cdot)$ look like this  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/IRb4m.jpg&quot; alt=&quot;time slice&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What will be a good choice of basis functions to approximate as accurately as possible $f$?&lt;br&gt;&#xA;Is the approximation better with a finite sample $\{f(x_{t_i, j})\}_{j=0}^m$,&#xA;or with a truncated Fourier series?&lt;br&gt;&#xA;Can the particular basis functions used give bound on the approximation error? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think a truncated Fourier series is the way to good&#xA;as the error is of order $O(1/K)$.&lt;br&gt;&#xA;But I'm not too familiar with gibbs phenomenom, so I don't know if the graph of $f$ discard Fourier series at once.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I know Chebyshev polynomial are for non periodic function.&lt;br&gt;&#xA;Would they perform better?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically, the easiest thing is to use a polynomial of some order, and use a collocation methods on the finite sample $\{f(x_{t_i, j})\}_{j=0}^m$.&lt;br&gt;&#xA;Is this viable?&lt;br&gt;&#xA;I think the main problem of this approach is that there is no bound on the approximation error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My function $f$ is the solution of a global optimization problem,&#xA;which is Lipschitz itself&#xA;$$&#xA;|V\left(f_1\right) - V\left( f_2\right) | \le L \|f_1 - f_2\|_{L^2(D)}.&#xA;$$&#xA;I know $f$ lives in a compact subset of $L^2(D)$,&#xA;so by constructing a fine enough grid, $f$ can be approximate arbitrarily well.&lt;/p&gt;&#xA;" OwnerDisplayName="user729" LastEditorDisplayName="user729" LastEditDate="2013-10-21T20:12:05.170" LastActivityDate="2013-10-21T20:12:05.170" Title="Choosing good basis functions to approximate a Lipschitz function" Tags="|finite-element|numerical-analysis|fourier-analysis|spectral-method|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8822" PostTypeId="5" CreationDate="2013-10-21T19:04:29.547" Score="0" Body="" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2013-10-21T19:04:29.547" LastActivityDate="2013-10-21T19:04:29.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8823" PostTypeId="4" CreationDate="2013-10-21T19:04:29.547" Score="0" Body="Computational physics is the study and implementation of numerical algorithms to solve problems in physics for which a quantitative theory already exists." OwnerUserId="541" LastEditorUserId="541" LastEditDate="2013-10-21T19:04:29.547" LastActivityDate="2013-10-21T19:04:29.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8824" PostTypeId="1" CreationDate="2013-10-21T19:21:47.703" Score="3" ViewCount="1737" Body="&lt;p&gt;I'm trying to apply the stochastic galerkin method to partial differential equation with multiple uniform random coefficients.  I'm puzzled as to how to extend the corresponding orthogonal (legendre) polynomial basis into higher dimensions.  Is there a systematic method for doing this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there is a paper/book that discusses an algorithm to generate orthogonal polynomials (hermite, laguerre, jacobi,...) into higher dimensional spaces ($d\ge 2$) ? &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-10-21T23:28:13.253" Title="Multivariate Orthogonal Polynomial Generation" Tags="|algorithms|high-dimensional|polynomials|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8825" PostTypeId="2" ParentId="8824" CreationDate="2013-10-21T23:28:13.253" Score="5" Body="&lt;p&gt;Suppose that you know the orthogonal polynomial basis in a single dimension $(x)$ of each degree $i$ up to some desired order $K$.  That is, we know $$p_0(x),p_1(x),...p_i(x),...,p_K(x)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To extend this into a two dimensions $(x,y)$, we need only consider the product between 1D polynomials in (x) and (y) and collect only the products whose &lt;em&gt;total degree&lt;/em&gt; is less than or equal to $K$.  That is, we collect the term $p_i(x)\cdot p_j(y)$ if $i+j\leq K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To extend this further into any arbitrary dimension $d$, simply created one dimensional orthogonal polynomial basis in each direction $1,2,...d$.  Then, to obtain the multidimensional basis functions, collect mutual products of these bases $$p_{i_1}(x_1)\cdot p_{i_2}(x_2)\cdot ... \cdot p_{i_d}(x_d)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;such that $i_1+i_2+...i_d\leq K$.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2013-10-21T23:28:13.253" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8826" PostTypeId="1" AcceptedAnswerId="8827" CreationDate="2013-10-22T02:26:03.547" Score="1" ViewCount="92" Body="&lt;p&gt;The Legendre polynomials satisfy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{-1}^{1}\phi_j(x)\phi_k(x)dx = \begin{cases}&#xA;0 &amp;amp;j\neq k\\\\&#xA;\dfrac{2}{2j+1} &amp;amp;j=k&#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that the best fit problem is given on the interval $[a,b]$. Show that with the transformation $t=\dfrac{1}{2}[(b-a)x+(a+b)]$ and a slight change of notation, we have&#xA;$$\int_{a}^{b}\phi_j(t)\phi_k(t)dt = \begin{cases}&#xA;0 &amp;amp;j\neq k\\\\&#xA;\dfrac{b-a}{2j+1} &amp;amp;j=k&#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $t=\dfrac{1}{2}[(b-a)x+(a+b)]$, then $x=\dfrac{2t-a-b}{b-a}$, and I am stuck at this step, can anyone give me some hints?&lt;/p&gt;&#xA;" OwnerUserId="5178" LastActivityDate="2013-10-22T04:02:44.350" Title="Transfrom a Legendre polynomial from $\int_{-1}^{1}\phi_j(x)\phi_k(x)dx $ into $\int_{a}^{b}\phi_j(t)\phi_k(t)dt$ given $t=\dfrac{1}{2}[(b-a)x+(a+b)]$" Tags="|numerical-analysis|polynomials|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8827" PostTypeId="2" ParentId="8826" CreationDate="2013-10-22T04:02:44.350" Score="6" Body="&lt;p&gt;You need to apply the Jacobian of the variable transformation. In other words, since you know how $x$ and $t$ are related, you can figure out how $dx$ and $dt$ are related. Obviously, they should differ by a factor of $(b-a)/2$ since that's the only substantive difference between the two expressions. In terms of more basic calculus, this is simply a $u$-substitution.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-22T04:02:44.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8828" PostTypeId="1" CreationDate="2013-10-22T08:05:17.770" Score="3" ViewCount="221" Body="&lt;p&gt;I was not quite sure what the right SE for this was, so I posted this also &lt;a href=&quot;https://dsp.stackexchange.com/questions/11226/how-to-detect-specific-behavior-in-time-series&quot;&gt;here&lt;/a&gt; on DSP. Please tell me which one to remove :)&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Problem statement&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I have a few hundred unrelated time series, say $P_i(t)$, with $i=1,2,...,N$. The sample times are all unequally spaced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is known that during certain (unknown) intervals, the trend in some of the time series is well-approximated by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    P_i(t) \approx \alpha - \beta \left(t_f-t\right)^\gamma\left(1 + \delta\cos\left(\omega\ln\left(t_f-t\right)+\phi\right)\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;t_0 \leq t &amp;lt; t_f\\&#xA;\alpha,\beta &amp;gt; 0\\&#xA;0\leq\gamma,\delta\leq1\\&#xA;0\leq\phi&amp;lt;2\pi&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\omega&amp;gt;0$ usually &quot;small&quot;, but not really restricted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of what such a function looks like: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4OfiM.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this interval ($t &amp;gt; t_f$), there is nearly always a rapid decline, after which the signal continues as it did before the interval ($t&amp;lt;t_0$). Of course, each series also has zero-mean, normally distributed noise of varying strength superimposed on it, hence the approximate sign. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My task: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;to determine the intervals where this behavior occurs,&lt;/li&gt;&#xA;&lt;li&gt;find best-estimates for all the parameters listed above. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;My first stab at it&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Since each of these time series can be quite long and voluminous, and I have quite many series to analyze (and that number will likely grow over time), visual inspection is out of the question. Moreover: often, it is quite easy to miss this pattern by just looking at the signal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pattern consists of two main components:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a power-law (the $(t_f-t)^\gamma$-term)&lt;/li&gt;&#xA;&lt;li&gt;an oscillation with logarithmically varying period (the $\cos\left(\omega\ln\left(t_f-t\right)\right)$-term)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To detect the power law, I thought of the following procedure: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;smooth the signal to get rid of the noise and the periodic component&lt;/li&gt;&#xA;&lt;li&gt;take the first two derivatives numerically.&lt;/li&gt;&#xA;&lt;li&gt;Periods of consecutive positive first derivative are a necessary condition for the given power law -- continue analyzing only these periods ($t_0$ estimated)&lt;/li&gt;&#xA;&lt;li&gt;The division of the first derivative by the second makes it possible to estimate $t_f$. &lt;/li&gt;&#xA;&lt;li&gt;The standard deviation and mean of each element's estimate for $t_f$, as well as the demand that $t_f &amp;gt; t$ and $t &amp;lt; t_E$ (the final time at which the first derivative is positive), give a measure for the reliability for this estimate.&lt;/li&gt;&#xA;&lt;li&gt;If it is reliable enough, it is fairly straightforward to backtrack everything and come up with estimates for $\gamma$, $\beta$ and $\alpha$ (in that order).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To detect the oscillating component, I came up with the following: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take an initial $t_f^{\text{trial}} &amp;gt; t_B$, so some time after the beginning of the time series.&lt;/li&gt;&#xA;&lt;li&gt;Compute $\ln(t_f^{\text{trial}}-t)$, from $t=t_B$ to just before $t_f^{\text{trial}}$. &lt;/li&gt;&#xA;&lt;li&gt;Compute the Fourier transform of the transformed time series $P_T(t) = P_i(\ln(t_f^{\text{trial}}-t))$ (via &lt;a href=&quot;http://en.wikipedia.org/wiki/Least-squares_spectral_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lomb/Scargle&lt;/a&gt;, because the new sample times are unequally and logarithmically spaced).&lt;/li&gt;&#xA;&lt;li&gt;Determine all the peaks in the frequency domain and save them. &lt;/li&gt;&#xA;&lt;li&gt;Repeat from the top with $t_f^{\text{trial}}\leftarrow t_f^{\text{trial}} + \Delta t$&lt;/li&gt;&#xA;&lt;li&gt;The progression of the peaks will roughly follow a parabola-shaped path in the frequency domain if there is a periodic component somewhere. The &quot;right&quot; $t_f$ will be found when the maximum power in the frequency has been found. &lt;/li&gt;&#xA;&lt;li&gt;For all peaks and $t_f$ thus found, find the corresponding interval that maximizes the power in the peak ($t_0$ estimated). &lt;/li&gt;&#xA;&lt;li&gt;with all this information, it is fairly straightforward to come up with initial estimates of $t_f$, $\omega$ and $\phi$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then, these two approaches are to be combined:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The estimates for $t_0$ and $t_f$ from both approaches will generally differ. determine the smallest overlapping interval $(t_0, t_f)$.&lt;/li&gt;&#xA;&lt;li&gt;Throw the data from this interval, as well as all the initial estimates, into a non-linear least squares fitter to improve the fit.&lt;/li&gt;&#xA;&lt;li&gt;Compute a couple of measures related to the goodness of fit, which will accompany the results. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;Why I'm looking for another approach&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Well, that all sure &lt;em&gt;sounds&lt;/em&gt; very nice, but of course I wouldn't be asking a question here if it all worked as well as it sounds :) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method I use to detect the presence and parameters of the power law: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;seems to be &lt;em&gt;extremely&lt;/em&gt; sensitive to noise&lt;/li&gt;&#xA;&lt;li&gt;I don't know how to determine automatically what a &quot;good enough&quot; smoothing is&lt;/li&gt;&#xA;&lt;li&gt;All smoothing algorithms I have tried are not very good at removing the periodic component, throwing all the estimates &lt;em&gt;way&lt;/em&gt; off. Moreover, the (log-)periodic component can make the derivatives &lt;em&gt;negative&lt;/em&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I could take the log of the data and detect linearity, but that suffers from the same problem -- the periodic component (as well as the unknown offset $t_f$) seems to make that unreliable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method I use to detect the periodic component: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;is &lt;em&gt;very&lt;/em&gt; computationally intensive; Lomb/Scargle is certainly not as fast as an FFT would be. And as I mentioned earlier, each time series can be quite long, so the number of times the Fourier transform needs to be computed can also be quite large &lt;/li&gt;&#xA;&lt;li&gt;detecting which peak correspond to which other peak from one estimate for $t_f$ to the next, is rather difficult to automate.&lt;/li&gt;&#xA;&lt;li&gt;transforming the sample times into (roughly) logarithmically-spaced sampling times makes it very hard to detect the start/end of the interval with a decent accuracy.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm kind of stuck, and I need some new inspiration. Any suggestions? &lt;/p&gt;&#xA;" OwnerUserId="2132" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2013-11-02T15:59:25.980" Title="How to detect specific behavior in time series?" Tags="|algorithms|regression|pattern-recognition|smoothing|oscillations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8830" PostTypeId="1" AcceptedAnswerId="8832" CreationDate="2013-10-22T11:51:37.290" Score="2" ViewCount="2035" Body="&lt;p&gt;Let's start off by &lt;strong&gt;NUMERICALLY&lt;/strong&gt; solving a 1-D steady-state heat transport problem using &lt;strong&gt;IMPLICIT&lt;/strong&gt; FDM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$DT_{xx}=0; ~T(x=0)=T_{BL}; ~T(x=l)=T_{BR}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ is diffusion; $T$ is temperature; subscript  $ _{x}$ is the gradient with respective to spatial coordinate $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To implicitly solving the equation, we discretise as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{align}D\frac{\frac{T_{x+1}-T_{x}}{\Delta x}-\frac{T_{x}-T_{x-1}}{\Delta x}}{\Delta x}=0\\&#xA;\frac{D}{\Delta x^{2}}T_{x+1}-\frac{2D}{\Delta x^{2}}T_{x}+\frac{D}{\Delta x^{2}}T_{x-1}=0\end{align}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At both boundaries, we enforce the boundary condition by allocating a &lt;strong&gt;ghost node&lt;/strong&gt; outside of the boundary. Therefore, on the left&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{align}D\frac{\frac{T_{2}-T_{1}}{\Delta x}-\frac{T_{1}-T_{BL}}{r_{BL}}}{\Delta x}=0\\&#xA;-D(\frac{1}{\Delta x^{2}}+\frac{1}{\Delta x r_{BL}})T_{1}+\frac{D}{\Delta x^{2}}T_{2}=\frac{D}{\Delta xr_{BL}}T_{BL}\end{align}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $T_{BL}$ is the temperature applied at the left ghost node (which is a known value), $r_{BL}$ is the VIRTUAL DISTANCE between the first node to the ghost node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly the right boundary condition can be implemented using $T_{BR}$ (known value);$r_{BR}=r_{BL}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation can be written as $Ax=b$ form solved by tridiagonal solver. Apparently $A$ and $b$ are function of $r_{BL}=r_{BR}$ which one has to fiddle with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question come to that : &lt;strong&gt;HOW TO IMPLIMENT A PHYSICALLY MEANINGFULL $r_{BL}=r_{BR}$, SO THAT THE SOLUTION WILL NOT BE DISTURBED BY ARBITRARILY CHOOSING VALUES&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually it is easy to obtain an analytical solution for the problem above:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T=\frac{T_{BR}-T_{BL}}{l}x+T_{BL}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the flux can be calculated by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$q=-DT_{x}=-D\frac{T_{1}-T_{0}}{l}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following figure shows the sensitivity analysis of $r_{BL}=r_{BR}$, and analytical solution are also plotted as comparison (we use $D=0.5,\Delta x=1,l=10,T_{BL}=20,T_{BR}=0$ as example):&#xA;&lt;img src=&quot;https://i.stack.imgur.com/GIPxZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one can see that when $r_b$ is between 1e-9 and 1e-2, the result is close enough to the analytical solution. However, if $r_b$ is too large, the result will deviate from right solution ($r_b$=1000). In contrast, if $r_b$ is too small($r_b$=1e-15), the temperature profile may be correct, but the boundary flux will be wrongly calculated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth to point out that finding suitable $r_b$ in this example problem may not be complicated (still between 1e-9 and 1e-2 is a huge range). However, The real problem comes when &lt;strong&gt;time-dependent boundaries into play at transient process&lt;/strong&gt;. At that stage, a physical based $r_b$ seems to be very important as a fixed $r_b$ may not hold for all time-dependent values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SOME of my thoughts:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I have read some of the ghost boundary problems &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7650/how-should-boundary-conditions-be-applied-when-using-finite-volume-method&quot;&gt;ghost boundary&lt;/a&gt; but havn't found any clear answers on this issue.&lt;/li&gt;&#xA;&lt;li&gt;It seems that this boundary condition may be considered as Robin type($T_{B}+\frac{\partial T_{B}}{\partial x}=constants$). However, it seems there is no way to find out the derivative values (although analytical solution here offers one, but most of the problems may not have analytical solutions). &lt;/li&gt;&#xA;&lt;li&gt;I have heard about this may result from matching significant digits of $T_{b}$ and $T_1$ &lt;a href=&quot;http://water.usgs.gov/nrp/gwsoftware/sour/faq/faq.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;search 'boundary conductances' in this page&lt;/a&gt;. in short, the selected $r_B$ should make $T_{b}$ and $T_1$ partially matched (the first half of the digits match, and the rest half of the digits mismatch). Then the mismatched values are responsible for calculating the outflow. Apparently, if $r_b$ is too big, all the digits are matched, then there is no flow going out. but still such fixed $r_b$ value may be venerable for non-linear boundary conditions.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any ideas? Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="4868" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-10-22T12:59:28.710" Title="How to implement boundary condition in this case?" Tags="|boundary-conditions|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8832" PostTypeId="2" ParentId="8830" CreationDate="2013-10-22T12:59:28.710" Score="2" Body="&lt;p&gt;There is absolutely no reason not to choose $r_{BL}=r_{BR}=\Delta x$. You phrase your question in terms of ghost nodes, but this is a concept one uses for &lt;em&gt;Neumann boundary conditions&lt;/em&gt;. It is not necessary for Dirichlet conditions: simply put a node at each $i\Delta x$, $i=0\ldots N$ and in the two equations where the values $T_0, T_N$ appear, you will replace their respective values by the Dirichlet values. That's all that's necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, since your solution is linear, your finite difference scheme must &lt;em&gt;exactly&lt;/em&gt; reproduce the exact solution. This is true for a uniform mesh ($r_{BL}=r_{BR}=\Delta x$) as well as any non-uniform choice ($r_{B*}\neq \Delta x$). The fact that you get a solution that is different from what you should points to the fact that there must be a bug in your code.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-22T12:59:28.710" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8833" PostTypeId="1" AcceptedAnswerId="8885" CreationDate="2013-10-22T15:06:05.903" Score="7" ViewCount="3243" Body="&lt;p&gt;For the function $f(x)=x$, how to compute the wavelet approximation using Haar basis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm new to wavelet, I'm looking for a package which will do something like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from mpmath import *&#xA;mp.dps = 15; mp.pretty = True&#xA;nb_coeff = 3&#xA;interval = [0, 10]&#xA;fct = lambda x : x&#xA;fourier_coef = fourier(fct, interval, nb_coeff)&#xA;fourier_series_apx = lambda x: fourierval(fourier_coef, interval, x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can this be done with &lt;a href=&quot;http://www.pybytes.com/pywavelets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pywavelets&lt;/a&gt; on an Haar basis?&lt;br&gt;&#xA;Or, is there another library which will produce the functionality above for Haar wavelet?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;br&gt;&#xA;Suspecting that their maybe no package which provides the functionality above,&#xA;I tried to roll my own.&lt;br&gt;&#xA;My implementation currently doesn't work and I asked a question &lt;a href=&quot;https://dsp.stackexchange.com/q/11234/5746&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user729" LastEditorUserId="-1" LastEditorDisplayName="user729" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2013-10-28T09:02:23.223" Title="How to compute the wavelet approximation of a function?" Tags="|python|fourier-analysis|wavelet|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="8834" PostTypeId="2" ParentId="8806" CreationDate="2013-10-22T22:04:12.813" Score="9" Body="&lt;p&gt;From skimming the table of contents to the book you listed, I'd say that computational books of that type for physics (or in my case, engineering, since that is my background) tend to sacrifice depth and quality of explanation for breadth. The best book of this ilk I can think of is probably Strang's &lt;a href=&quot;http://math.mit.edu/cse/&quot;&gt;Computational Science and Engineering&lt;/a&gt;, because he's a great writer of mathematics. Many &quot;numerical methods&quot; classes geared towards scientists and engineers tend to teach just enough math in whatever topics they feel the intended audience needs to get by, and cobble together a bunch of notes. (Or they punt, and teach out of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0495114766&quot;&gt;Burden and Faires&lt;/a&gt;, or worse, rely on &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521880688&quot;&gt;Numerical Recipes&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get better explanations and depth, you'll probably need to get access to more specialized textbooks, such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;dense linear algebra: probably &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898713617&quot;&gt;Numerical Linear Algebra&lt;/a&gt; by Trefethen and Bau, though I've also heard good things about &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898713897&quot;&gt;Applied Numerical Linear Algebra&lt;/a&gt; by Demmel, and &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1421407949&quot;&gt;Matrix Computations&lt;/a&gt; by Golub and van Loan&lt;/li&gt;&#xA;&lt;li&gt;sparse linear algebra: for iterative methods, Saad's &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot;&gt;Iterative Methods for Sparse Linear Systems&lt;/a&gt;, for direct methods, Davis' &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898716136&quot;&gt;Direct Methods for Sparse Linear Systems&lt;/a&gt;, for multigrid, &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898714621&quot;&gt;A Multigrid Tutorial&lt;/a&gt;, among others&lt;/li&gt;&#xA;&lt;li&gt;solving ODEs: &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898714125&quot;&gt;Computer Methods for Ordinary Differential and Differential-Algebraic Equations&lt;/a&gt; by Ascher and Petzold; also see the book by Lambert, the two volume series by Hairer and Wanner, and Butcher's book&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A list could be created for optimization (Nocedal's textbook is a good start), finite volume methods (LeVeque's books), finite difference methods (again, LeVeque has another good book, or the book by Strikwerda), finite element methods (not an expert here), and so on. The tradeoff is depth (and quality of explanation) for time. I agree with Pedro that you'll probably be able to make the largest contributions right away in the algorithms and data structures used to implement physics. If you really want to dig into the math, though, I'd pick one of the specialty books that you think will be most useful to you and start reading it.  &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-10-22T22:04:12.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8835" PostTypeId="1" CreationDate="2013-10-23T02:56:02.137" Score="1" ViewCount="89" Body="&lt;p&gt;Using the recursion formula for Chebyshev polynomials, show that $T_n(x)$ can be written as &#xA;$$T_n(x)=2^{n-1}(x-x_1)(x-x_2)...(x-x_n)$$ &#xA;where $x_i$ are the $n$ roots of $T_n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recurrence relation: $T_0(x)=1$,$T_1(x)=x$, and $T_{n+1}(x)=2xT_n(x)-T_{n-1}(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an intuition that I need to use induction,but what is my inductive hypothesis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume $T_n(x)=2^{n-1}(x-x_1)(x-x_2)...(x-x_n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prove $T_{n+1}(x)=2^{n}(x-x_1)(x-x_2)...(x-x_n)(x-x_{n+1})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use the definition of Chebyshev polynomial, $T_n(x) = cos(n\theta),x =cos(\theta)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_0(x)=1$ and $T_1(x)=x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then use the recursion formula $T_{n+1}(x)=2xT_n(x)-T_{n-1}(x)$, I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_{n+1}(x) = 2x[2^{n-1}(x-x_1)(x-x_2)...(x-x_n)]-[2^{n-2}(x-x_1)(x-x_2)...(x-x_{n-1})]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_{n+1}(x) = 2x(2^{-1})(x-x_n)[2^n(x-x_1)(x-x_2)...(x-x_{n-1})]-2^{-2}[2^n(x-x_1)(x-x_2)...(x-x_{n-1})]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rearrange I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_{n+1}(x) = [2x(2^{-1})(x-x_n)-2^{-2}][2^n(x-x_1)(x-x_2)...(x-x_{n-1})]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do next(assume what I did above is correct) so that I can get $T_{n+1}(x)=2^{n}(x-x_1)(x-x_2)...(x-x_n)(x-x_{n+1})$ ??&lt;/p&gt;&#xA;" OwnerUserId="5178" LastEditorUserId="541" LastEditDate="2013-10-23T07:26:27.640" LastActivityDate="2013-10-23T07:26:27.640" Title="Prove $T_n(x)$ of Chebyshev Polynomial given the recurrence relation" Tags="|numerical-analysis|polynomials|" AnswerCount="0" CommentCount="5" ClosedDate="2013-10-24T04:42:35.620" ContentLicense="CC BY-SA 3.0" />
  <row Id="8836" PostTypeId="1" CreationDate="2013-10-23T05:20:12.567" Score="1" ViewCount="346" Body="&lt;p&gt;Many of the problems in Image processing are embaressingly parallel. But I want to know is it worth solving Image processing using parallel computing? Are there real practical problems in image processing that can benefit form parallel computing? If so what are those? I am not taking about mliticore CPUs, instead I am talking about parallelism offered by GPUs, and clusters of GPUs.    &lt;/p&gt;&#xA;" OwnerUserId="4932" LastActivityDate="2013-10-23T06:47:22.550" Title="Image processing problems that can be solved using parallel computing?" Tags="|gpu|hpc|" AnswerCount="0" CommentCount="3" ClosedDate="2013-10-28T19:34:00.497" ContentLicense="CC BY-SA 3.0" />
  <row Id="8838" PostTypeId="2" ParentId="265" CreationDate="2013-10-23T06:49:06.637" Score="5" Body="&lt;p&gt;I realize this answer is probably far, far too late, but for aggressive optimization, the &quot;-fast&quot; option seems like the easiest solution. Excerpts from the man page:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-fast&#xA;    Maximizes speed across the entire program.&#xA;    Description:&#xA;    This option maximizes speed across the entire program.&#xA;&#xA;    It sets the following options:&#xA;     - On Linux* systems: -ipo, -O3, -no-prec-div, -static, and -xHost&#xA;     - On OS X* systems: -ipo, -mdynamic-no-pic, -O3, -no-prec-div, and -xHost&#xA;     - On Windows*  systems:  /O3,  /Qipo,  /Qprec-div-,  and /QxHost&#xA;&#xA;    NOTE: Option fast sets some aggressive optimizations that may not be appropriate for all&#xA;    applications. The resulting executable may not run on processor types different from the&#xA;    one on which you compile. You should make sure that you understand the individual optimi-&#xA;    zation options that are enabled by option fast.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5428" LastActivityDate="2013-10-23T06:49:06.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8839" PostTypeId="1" CreationDate="2013-10-23T12:49:01.247" Score="3" ViewCount="347" Body="&lt;p&gt;Or can they just be used as an interpolation points and use some other &quot;transported property&quot; which are just evolved and propagated from boundary conditions like for eg. heat conduction through a solid metal bar. It seems to be possible, but isn't clearly elaborated. The force computation step is always used for fluid flow to move particles forward according to Newton's laws, but what if heat is transported purely by conduction in a solid? Here, nothing moves but still, heat flows and destroys temperature gradients. What equations are to be considered here (apart from the heat diffusion equation) to adapt this problem for this kind of discretization?&lt;/p&gt;&#xA;" OwnerUserId="5096" LastActivityDate="2017-11-08T10:57:16.893" Title="Should particles in Smoothed Particle Hydrodynamics (SPH) always move during a simulation?" Tags="|advection-diffusion|sph|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8840" PostTypeId="1" CreationDate="2013-10-23T13:29:38.607" Score="0" ViewCount="117" Body="&lt;p&gt;I'm working on a heat diffusion problem,&#xA;$$&#xA;\frac{\partial T}{\partial t}=\vec{\nabla}\cdot\left(\kappa T^{5/2}\,\vec{\nabla}T\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that, after discretization, the time step for the 1D case becomes,&#xA;$$&#xA;dt= \frac{\rho\,dx^2}{\kappa T_{max}^{5/2}}&#xA;$$&#xA;Does this imply/mean that the maximum wave speed is&#xA;$$&#xA;\frac{dx}{dt} = \frac{\kappa T_{max}^{5/2}}{\rho\,dx}&#xA;$$&#xA;or would I have to analyze this differently to get the wavespeed?&lt;/p&gt;&#xA;" OwnerUserId="4762" LastActivityDate="2013-11-23T12:13:00.630" Title="max speed &lt;--&gt; time discretization" Tags="|finite-difference|diffusion|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="8841" PostTypeId="2" ParentId="395" CreationDate="2013-10-23T15:03:18.327" Score="2" Body="&lt;p&gt;NWChem includes a script to turn a list of xyz coordinates into a movie.  NWChem is OSS under Apache-style license so you can reuse however you like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For visualization of molecular vibrations, I use Molden, ECCE, Avogadro and Jmol.  All are OSS (ECCE is only recently).  You might be able to hack them to do what you want.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2013-10-23T15:03:18.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8842" PostTypeId="1" CreationDate="2013-10-23T15:35:48.923" Score="8" ViewCount="206" Body="&lt;p&gt;If you have a large multivariable ODE system, and certain processes occur at a much shorter time scale, how can you implement a solver that uses smaller time steps for state variables involved in fast processes, and large time steps for state variables involved in slow processes, while ensuring the result is still mathematically sound?&lt;/p&gt;&#xA;&lt;p&gt;For simplicity, let's assume that we are using the explicit Euler solver method per state variable.&lt;/p&gt;&#xA;&lt;p&gt;I ask because I am concerned, for example, about ensuring certain quantities remain conserved. There is a lot of literature about variable time step size depending on the error estimation, but I am asking about assuming different time steps for different state variables.&lt;/p&gt;&#xA;&lt;p&gt;This is an optimization to reduce runtime on a computing architecture that does not do well with runtime adaptive time stepping.&lt;/p&gt;&#xA;" OwnerUserId="5432" LastEditorUserId="23791" LastEditDate="2021-12-31T18:19:02.697" LastActivityDate="2021-12-31T18:19:02.697" Title="How do you numerically solve a multivariable ODE system with different time steps per state variable?" Tags="|ode|numerics|stiffness|adaptive-timestepping|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8845" PostTypeId="1" AcceptedAnswerId="8846" CreationDate="2013-10-24T00:51:01.573" Score="4" ViewCount="1632" Body="&lt;p&gt;generally, it is natural to use procedural programming approach (PP) to solve a partial differential equation by finite difference method (FDM). That is, one (1) defines matrices to store the properties at nodes and fluxes in between the nodes, (2) assembles linear systems based on discretization to the controlling equation , (3) solves linear equations and (4) moves into next time step. As each controlling equation solves one property across the whole domain, defining a matrix of one property is very convenient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but so far I realized some (maybe) disadvantages of using this methods, which is that the modules (or subroutines) used for process (2) do not easy to be ported to another program solving another fundamental equation. basically if one wants to solving another problem, it is required to start from scratch (except for matrix solver of course).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been adviced to use object-oriented programming method (OOP) to handle the problem, which is very convenient to modulize the code, and the modules are quite reusable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for my understanding, by OOP method, one may not define a matrix storing the property for the whole domain, but define each node with many properties (using objects). This method is quite robust in some Lagrangian methods like smooth particle hydrodynamics and lattice Boltzmann method. but this method may not quite convenient to solve linear system as the matrix has been disassembled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, Nowadays many programming starts to support OOP, even fortran starts to support objects. In addition, currently there are many FDM, Finite element method (FEM), Finite volume method (FVM) packages using OOP concept (e.g. openfoam), one only requires to define the fundamental equations and then desensitization seems to be done using the same codes. I think that may not be easy to use PP to handle.I must have something missing about OOP ideas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;of course this problem may not limited to use finite different method, but also to use FVM and FEM which to me, seems more difficult to construct by OOP concept. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;any ideas? thanks in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ps: generlly I use fortran, matlab, R, python deal with CFD and multi-phase flow problems in porous media using PP. &lt;/p&gt;&#xA;" OwnerUserId="4868" LastEditorUserId="4244" LastEditDate="2013-10-24T05:31:10.147" LastActivityDate="2013-10-24T11:50:35.900" Title="Object-oriented programming on finite difference method" Tags="|finite-difference|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8846" PostTypeId="2" ParentId="8845" CreationDate="2013-10-24T01:56:02.233" Score="10" Body="&lt;p&gt;OOP has great utility in scientific programming, but the natural way one would write an &quot;OOP&quot; scientific program after education in non-scientific software engineering may be a bad way to tackle the problem. Anecdote:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My first semi-serious CFD code as an undergrad, I wrote a finite-volume 2D Euler equation solver with excessive use of OOP. Intuitively, I figured a &quot;cell&quot; would be a good class, each with pointers to &quot;face&quot; classes that are shared amongst cells. Then I iterated through cell and face objects. i.e., I had a few million tiny objects of &quot;cell&quot; and &quot;face&quot; type. It was incredibly slow -- every access of these elements resulted in tons of dereferenced pointers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As a general rule of thumb, I'd suggest taking the advice of the CUDA people (and for the same reasons -- it's not news to people who know about high performance code):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Small structures of large arrays are better than Large arrays of small structures.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Check out Damian Rouson's &quot;Scientific Software Design: The Object Oriented Way&quot; for a good take on ways to use OOP for high performance code. I have some serious issues with details in there, but it's still pretty good.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-10-24T01:56:02.233" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8847" PostTypeId="1" CreationDate="2013-10-24T03:08:24.417" Score="3" ViewCount="357" Body="&lt;p&gt;Issue of &lt;a href=&quot;https://en.wikipedia.org/wiki/Data_dependency&quot; rel=&quot;nofollow&quot;&gt;data dependency&lt;/a&gt; with &lt;a href=&quot;https://en.wikipedia.org/wiki/Stencil_code&quot; rel=&quot;nofollow&quot;&gt;stencil code&lt;/a&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to parallelise this using openmp? I looked at the openmp manual, I figured out how to use DO ORDERED to get the same result as the serial version, but when run with openmp, it is x200 or so times slower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is example code from the internet doing the update from neighboring cells just to demonstrate the kind of problem I am after. I am just looking for a conceptual description of the correct approach, with examples if you like, but you don't have to spend time on implementing for this specific case.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  // Computation&#xA;  for ( i=1 ; i &amp;lt; N-1 ; i++ ) {&#xA;     for ( j=1 ; j &amp;lt; N-1 ; j++ ) {&#xA;        prev(i,j) = ((prev(i-1,j-1)+foo*prev(i-1,j)+bar*prev(i-1,j+1)+prev(i,j-1)+prev(i,j)+prev(i,j+1)+prev(i+1,j-1)+prev(i+1,j)+prev(i+1,j+1)));&#xA;     }&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You can also see an example of another similar problem &lt;a href=&quot;https://nec2c.googlecode.com/files/Parallelizing%20NEC%E2%80%99s%20Equation%20Solver%20Algorithm%20with%20OpenMP.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, on page 6.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;And another example from &lt;a href=&quot;http://wissrech.iam.uni-bonn.de/research/projects/NaSt3DGP/download.htm&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (in C):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inline void NavierCalc::PSOR(int i,int j,int k,NS_REAL omg1,NS_REAL lomg) {&#xA;&#xA;// Relaxiertes Gaus-Seidel-Verfahren&#xA;//&#xA;IFFLUID(flag[i][j][k]) {&#xA;SetPLocalBorder(P,i,j,k);&#xA;P[i][j][k]=omg1*P[i][j][k] - lomg/(S.ddPstar[0][3][i]+&#xA;     S.ddPstar[1][4][j]+S.ddPstar[2][5][k]) *  &#xA;  (S.ddPstar[0][6][i]*P[i+1][j][k] + S.ddPstar[0][0][i]*P[i-1][j][k] + &#xA;   S.ddPstar[1][7][j]*P[i][j+1][k] + S.ddPstar[1][0][j]*P[i][j-1][k] +&#xA;   S.ddPstar[2][8][k]*P[i][j][k+1] + S.ddPstar[2][0][k]*P[i][j][k-1] &#xA;   - RHS[i][j][k]);&#xA;}&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;A relevant example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://www.cesm.ucar.edu/models/cesm1.0/cesm/cesmBbrowser/html_code/pop/POP_SolversMod.F90.html#PRECONDITIONER&quot; rel=&quot;nofollow&quot;&gt;this page&lt;/a&gt;, except in my case it is not PX, it is X at the left-hand side, so it does depend on itself, and the loop is split into two (one walks from left to right and uses SOMETHING-1 indices at the right-hand side, while the other walks from right to left and uses SOMETHING+1 indices at the right-hand side): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; do j=2,size(X,dim=2)-1&#xA; do i=2,size(X,dim=1)-1&#xA;    PX(i,j,bid) = precondNE    (i,j,bid)*X(i+1,j+1,bid) + &amp;amp;&#xA;                  precondNW    (i,j,bid)*X(i-1,j+1,bid) + &amp;amp;&#xA;                  precondSE    (i,j,bid)*X(i+1,j-1,bid) + &amp;amp;&#xA;                  precondSW    (i,j,bid)*X(i-1,j-1,bid) + &amp;amp;&#xA;                  precondNorth (i,j,bid)*X(i  ,j+1,bid) + &amp;amp;&#xA;                  precondSouth (i,j,bid)*X(i  ,j-1,bid) + &amp;amp;&#xA;                  precondEast  (i,j,bid)*X(i+1,j  ,bid) + &amp;amp;&#xA;                  precondWest  (i,j,bid)*X(i-1,j  ,bid) + &amp;amp;&#xA;                  precondCenter(i,j,bid)*X(i  ,j  ,bid)&#xA; end do&#xA; end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;One more relevant example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www4.ncsu.edu/eos/users/w/white/www/book/picpcg.f90&quot; rel=&quot;nofollow&quot;&gt;this page&lt;/a&gt; - SSOR preconditioner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!   Execute SSOR preconditioner.&#xA;    do j= 1,n-1&#xA;      do i = 1,n-1&#xA;        rhat(i,j) = w*(r(i,j)-aw(i,j)*rhat(i-1,j)   &amp;amp;&#xA;                     -as(i,j)*rhat(i,j-1))/ac(i,j)&#xA;      end do&#xA;    end do&#xA;    do j= 1,n-1&#xA;      do i = 1,n-1&#xA;        rhat(i,j) =  ((2.-w)/w)*ac(i,j)*rhat(i,j)&#xA;      end do&#xA;    end do&#xA;    do j= n-1,1,-1&#xA;      do i = n-1,1,-1&#xA;        rhat(i,j) = w*(rhat(i,j)-ae(i,j)*rhat(i+1,j)  &amp;amp;&#xA;                      -an(i,j)*rhat(i,j+1))/ac(i,j)&#xA;      end do&#xA;    end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;And &lt;a href=&quot;http://nkl.cc.u-tokyo.ac.jp/13e/05-Multicore/omp-02.pdf&quot; rel=&quot;nofollow&quot;&gt;this presentation&lt;/a&gt; outlines the concept or reordering by coloring. More examples may be needed to understand the concept fully...&lt;/p&gt;&#xA;" OwnerDisplayName="user5273" LastEditorDisplayName="user5273" LastEditDate="2013-11-15T00:41:23.637" LastActivityDate="2013-11-15T00:41:23.637" Title="openmp update matrix from neighboring elements (parallelise preconditioner)" Tags="|parallel-computing|linear-solver|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8848" PostTypeId="2" ParentId="8847" CreationDate="2013-10-24T05:56:56.667" Score="3" Body="&lt;p&gt;Since you're traversing your mesh left to right, top to bottom, most steps will depend on results computed in the directly previous step. If you want to maintain the order, this leaves very little room for parallelism. The best you could do is to get each thread to do a row of your matrix, i.e. parallelize only over &lt;code&gt;i&lt;/code&gt;, and somehow try to prevent the threads overtaking each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, synchronization points, i.e. parts of the parallel code where one thread waits for another, will always have a negative impact on parallel performance, and should be avoided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may therefore want to ask yourself how crucial it is that your parallelized code replicates the serial results exactly? Such stencil updates are usually used as part of a &lt;a href=&quot;http://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method&quot; rel=&quot;nofollow&quot;&gt;Gauss-Seidel iteration&lt;/a&gt;, or any other iterative method, which should converge to the correct result and is usually not dependent on the exact order of the updates.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-10-24T05:56:56.667" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8849" PostTypeId="1" AcceptedAnswerId="8854" CreationDate="2013-10-24T08:59:46.383" Score="1" ViewCount="848" Body="&lt;p&gt;What is exactly the definition of &quot;Straddle&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please explain what do they mean exactly or a sketch? A segment P1P2 Straddles a line if point P1 lies on the one side of the line and point P2 lies on the other side. A boundary case of P1 or P2 lies directly on the line. &lt;strong&gt;(Taken from Introduction to algorithms 3rd)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a couple of answers:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/IrQn8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4918" LastEditorUserId="4918" LastEditDate="2013-10-24T15:59:15.573" LastActivityDate="2014-03-01T02:08:33.627" Title="Line segment straddle" Tags="|algorithms|computational-geometry|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8850" PostTypeId="2" ParentId="8845" CreationDate="2013-10-24T11:50:35.900" Score="7" Body="&lt;p&gt;Absolutely nothing is &quot;natural&quot; about using procedural programming for finite differences or finite elements. In fact, &lt;em&gt;all&lt;/em&gt; widely used software packages for finite element methods that have come up in the past 15 years are object-oriented (e.g., libmesh, openfoam, my own library deal.II, fenics, ...) and the same is true for linear algebra packages (e.g., petsc, trilinos). That doesn't mean that every single thing (like a cell) should be an object, but certainly that things like meshes, matrices, finite elements, are all implemented as classes. There is absolutely no question in our community that this is the only possible way to deal with large-scale software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several other questions on the SciComp StackEchange site that deal with similar issues.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-24T11:50:35.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8851" PostTypeId="2" ParentId="8842" CreationDate="2013-10-24T11:54:26.210" Score="8" Body="&lt;p&gt;You will want to read up on operator splitting methods. In essence, in every &quot;macro time step&quot; you would treat fast processes by doing many &quot;micro time steps&quot; in one half of the algorithm, and then do a single macro time step for the slow processes in the other half.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For higher order, you will want to use what's known as &quot;Strang splitting&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-24T11:54:26.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8852" PostTypeId="2" ParentId="8840" CreationDate="2013-10-24T11:55:57.857" Score="2" Body="&lt;p&gt;For the heat equation, the wave speed is infinite and this is reflected by the fact that you are solving a globally coupled differential equation if you are using an implicit time stepping scheme.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-24T11:55:57.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8853" PostTypeId="2" ParentId="8842" CreationDate="2013-10-24T13:08:45.747" Score="6" Body="&lt;p&gt;The term that you want to search for is &lt;em&gt;multiple timestepping&lt;/em&gt; (see, for instance, [1-3]).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.cs.unc.edu/Research/nbody/pubs/external/Berne/tuckerman-berne-rossi91.pdf&quot;&gt;http://www.cs.unc.edu/Research/nbody/pubs/external/Berne/tuckerman-berne-rossi91.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://www3.nd.edu/~izaguirr/papers/newM3paper.pdf&quot;&gt;http://www3.nd.edu/~izaguirr/papers/newM3paper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://arxiv.org/abs/1307.1167&quot;&gt;http://arxiv.org/abs/1307.1167&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-10-24T13:08:45.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8854" PostTypeId="2" ParentId="8849" CreationDate="2013-10-24T16:51:02.013" Score="3" Body="&lt;p&gt;A line L separates the plane into two half-planes.  That is, removing L leaves two (unbounded) regions on either side of L.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We may use the word &lt;em&gt;straddle&lt;/em&gt; to describe a line segment that connects these two half-planes by having an endpoint P1 or P2 in each of the two separated regions.  Visually we see that line segment P1P2 crosses line L and defines a point of intersection strictly between the two endpoints P1 and P2.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-10-24T16:51:02.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8855" PostTypeId="1" CreationDate="2013-10-24T18:21:02.507" Score="1" ViewCount="2691" Body="&lt;p&gt;I have a system of 256 differential equations that I want to solve numerically. The system represents the Liouville equation, which is a first order, linear differential equation with complex numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the best method to solve this equation in parallel? Please advise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a physicist, so I'm familiar with the basic solvers (Euler and Runge-Kutta), which are basically sequential by definition.&lt;/p&gt;&#xA;" OwnerUserId="5410" LastEditorUserId="1804" LastEditDate="2016-05-31T21:30:15.207" LastActivityDate="2016-05-31T21:30:15.207" Title="Best PARALLEL numerical solver of first order differential equation" Tags="|parallel-computing|simulation|runge-kutta|quantum-mechanics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8856" PostTypeId="2" ParentId="8855" CreationDate="2013-10-24T18:57:56.363" Score="4" Body="&lt;p&gt;I would disagree with your statement that Euler and Runge-Kutta have to be sequential.  I know that you are saying this because you cannot parallelize across time steps (or across inner time steps in the case of something like RK4), however both can be parallel as long as you do so within a single time step.  You would simply evaluate the derivatives of your 256 variables in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on your system characteristics, it is possible that RK methods would be the best for you, or perhaps a simple stiff solver.  I personally am not familiar with parallel implementations so I will leave the recommendations to others with much more experience than myself.  Unless your system is something incredibly special though, you will be using algorithms that you are more than likely familiar with.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-10-24T18:57:56.363" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8857" PostTypeId="1" AcceptedAnswerId="8862" CreationDate="2013-10-24T19:47:54.040" Score="6" ViewCount="2556" Body="&lt;p&gt;How can we define the precision we require in a numerical differential equation solver? What is it that I have to optimize to know? And how do I know that I'm at a sufficient time-step value?&lt;/p&gt;&#xA;&lt;p&gt;For example, in those programs like Mathematica and Matlab, how does the differential equations solver know the suitable step size for your problem? How could that be automatically determined?&lt;/p&gt;&#xA;&lt;p&gt;A full explanation of the approach would be nice (or maybe a term that can be googled).&lt;/p&gt;&#xA;" OwnerUserId="5410" LastEditorUserId="20688" LastEditDate="2021-12-17T00:57:39.113" LastActivityDate="2021-12-17T00:58:48.057" Title="Determine the step size in a differential equation numerical solver" Tags="|differential-equations|time-integration|precision|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8858" PostTypeId="1" CreationDate="2013-10-24T20:18:01.833" Score="0" ViewCount="578" Body="&lt;p&gt;In general, is there a partial trace algorithm (ideally for systems of any size) that can be coded using basic matrix operations found in software like Mathematica or Maple? All of the methods I'm aware of seem to be suited for humans, and I am struggling to think of a way of implementing them in any programming language I know. I would appreciate any kind of help with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="5442" LastActivityDate="2013-10-25T14:59:07.497" Title="Partial trace algorithm" Tags="|linear-algebra|quantum-mechanics|" AnswerCount="1" CommentCount="2" ClosedDate="2013-10-27T00:11:14.023" ContentLicense="CC BY-SA 3.0" />
  <row Id="8859" PostTypeId="2" ParentId="8855" CreationDate="2013-10-24T20:38:42.217" Score="2" Body="&lt;p&gt;You may want to look at parareal methods, that is, a parallel-in-time approach.  There can be huge gains in speedup, but there are some issues with stability.  The paper by Lions, Maday &amp;amp; Turinici (2001) would be a good place to start if you're not familiar with the method.&lt;/p&gt;&#xA;" OwnerUserId="4935" LastActivityDate="2013-10-24T20:38:42.217" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8860" PostTypeId="1" CreationDate="2013-10-24T22:38:13.697" Score="2" ViewCount="400" Body="&lt;p&gt;This question interests me for a while.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At one extreme, I found papers, and some thesis that were outcome of change of a single parameter in a CFD code, like in the case of &quot;Atmospheric boundary layer&quot; turbulence model, that was produced by a single change in $c_\mu$ constant of the $k-\epsilon$ model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other end there are papers, made after whole library of hundred thousand or more lines of code have been written, as in the case of big libraries like deal-II.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find this important for understanding what is actual effort invested in writting a single sci comp paper, and to understand what this effort is consisted of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one approach for finding some answers I am considering to send questionaries to authors of a number of papers, and maybe plot a correlation between approximate number of code lines written and citation of the paper. This result can be published in figshare for curious ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to hear some troughts. &lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2013-10-30T01:57:28.813" Title="How many lines of code one needs to write to be able to write a paper?" Tags="|publications|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8861" PostTypeId="2" ParentId="8857" CreationDate="2013-10-24T22:55:44.010" Score="7" Body="&lt;p&gt;for 1-D transport problem, under implicit method, we use &lt;a href=&quot;https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Courant number&lt;/a&gt;, a dimensionless number to choose appropriate time steps:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{u\Delta t}{\Delta x}\le C_{\max}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$C_{\max}$&lt;/span&gt; should be less than 1, if we make it as 1, we get&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Delta t\le \frac{\Delta x}{u}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The basic idea is that, at one time step, a particle placed in the problem domain should never transport through two nodes where solving properties are defined.&lt;/p&gt;&#xA;&lt;p&gt;for 2-D one can expand it as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{u_{x}\Delta t}{\Delta x}+\frac{u_{y}\Delta t}{\Delta y}\le C_{\max}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="4868" LastEditorUserId="20688" LastEditDate="2021-12-17T00:58:48.057" LastActivityDate="2021-12-17T00:58:48.057" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8862" PostTypeId="2" ParentId="8857" CreationDate="2013-10-25T01:52:05.977" Score="9" Body="&lt;p&gt;What is typically done for ODEs is to solve at each timestep using two methods of different order.  The discrepancies between the results are then used to estimate the error and then decide whether to decrease or increase the timestep length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find more information on the subject &lt;a href=&quot;http://books.google.com/books?id=F93u7VcSRyYC&amp;amp;lpg=PR2&amp;amp;vq=automatic%20step%20size&amp;amp;pg=PA167#v=onepage&amp;amp;q=automatic%20step%20size&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. Two examples of these types of schemes are &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta%E2%80%93Fehlberg_method&quot; rel=&quot;noreferrer&quot;&gt;RKF45&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Dormand%E2%80%93Prince_method&quot; rel=&quot;noreferrer&quot;&gt;Dormand-Prince&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-10-25T01:52:05.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8863" PostTypeId="2" ParentId="8855" CreationDate="2013-10-25T03:38:21.903" Score="9" Body="&lt;p&gt;256 equations is a relatively small number. All of the usual integrators, such as those included in Matlab, Maple or Mathematica should have no real problem with equations of this size and should be able to return answers in a fraction of the time it would take an algorithm you would implement yourself, because they use sophisticated explicit/implicit and adaptive time stepping methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ODE solvers are, for the most part, a solved problem. Don't waste your time implementing one yourself -- just use what others have done before you.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-25T03:38:21.903" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8864" PostTypeId="2" ParentId="8860" CreationDate="2013-10-25T03:43:49.063" Score="8" Body="&lt;p&gt;You're looking at the wrong metric. In most areas of science, people don't write code at all but still get their papers published. The same is true for more theoretical papers in numerical analysis, proving for example the convergence of a particular scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, there are papers that discuss particular libraries or implementations. These oftentimes require writing tens of thousands of lines of code. I have several of those (the one on deal.II discusses what at the time was probable ~300k lines of code, the one on hp methods ~20k, the one on parallel distributed computing and the one on mantle convection each required 10-15k). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In between, there are papers that are more about the development of algorithms. Again, I have several where we got away with just a few hundred to maybe 2-3000 lines of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I feel that your question is not well posed. It's not that you get a paper published because you wrote so many lines of code. Rather, it is what you do in terms of theoretical development plus what you can show with your code that determines whether the paper will be published.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-25T03:43:49.063" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8865" PostTypeId="1" AcceptedAnswerId="8868" CreationDate="2013-10-25T09:40:51.350" Score="3" ViewCount="534" Body="&lt;p&gt;I'm stuck on this PDE:&#xA;$$\rho \cdot C \cdot \dfrac{\partial T}{\partial \tau} + u \cdot \rho \cdot C \cdot \dfrac{\partial T}{\partial r} = \dfrac{\lambda}{r^{2}} \cdot \dfrac{\partial}{\partial r}\left(r^{2} \cdot \dfrac{\partial T}{\partial r}\right) + K - \dfrac{\rho \cdot C \cdot T}{V} \cdot \dfrac{\partial V}{\partial \tau}$$&#xA;This should describe the variation of enthalpy during a flash pyrolysis. I know all the parameters ($ \rho, C, \lambda, K$) and I can obtain $u$ and $V$ from analytic expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I know the initial condition and two boundary conditions, I'd like to solve it numerically using matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody help me out? Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IC and BC:&#xA;$$T_{\tau = 0} = T_{0} \hspace{2cm} 0 \leq r \leq R $$&#xA;$$\dfrac{\partial T}{\partial r}\bigg|_{r=0} = 0$$&#xA;$$\dfrac{\partial T}{\partial r}\bigg|_{r=R} = C1 \cdot (T_{m} - T_{R})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, $\tau$ is a time variable and $r$ is the space coordinate. I'm expecting a profile of $T$ with respect to both variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sorry if I can't provide so much useful information, but I was given a scientific paper to study where these equations (there is a lot of equations, but this one is the trickiest) are displayed but there is no mention of how they solved and got to the results.&lt;/p&gt;&#xA;" OwnerUserId="5446" LastEditorUserId="5446" LastEditDate="2013-10-25T12:40:23.487" LastActivityDate="2013-10-28T02:13:30.483" Title="Stuck on a hideous differential equation" Tags="|pde|matlab|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8866" PostTypeId="1" CreationDate="2013-10-25T10:01:01.163" Score="2" ViewCount="77" Body="&lt;p&gt;I have hjb equation where $V=V(x,t)$ and $u=u(x,t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_t + \sup(u) [A(x,u)V_x + B(x,u)V_{xx}]=0$ for $x$ in $[0,1]$ and $t$ in $[0,1]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been able to successfuly resolve it numerically having theese boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V(0,t)=0$ all t in [0,1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V(1,t)=1$ all t in [0,1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V(x,1)=x$ all x in [0,1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used finite differences to compute $V_t$, $V_x$ and $V_{xx}$ and iterative process at each temporal step to find the optimal control $u^*$ recomputing the solution $V$ until differences among the steps are negligible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now i would like to solve it on a infinite space boundary, so letting $x$ go from $-\inf$ to $+\inf$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the condition:&#xA;$V(x,1)=max(x,0)$ for all $x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought to use:&#xA;$V(x,t) \rightarrow 0$ for $x \rightarrow -\inf$ for all $t$ in $[0,1]$&#xA;$V(x,t)/x \rightarrow 1$ for $x \rightarrow \inf$ for all $t$ in $[0,1]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;how do you think is the best way to &quot;discretize&quot; these latter two conditions, if possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks in advance, and i am sorry for the mistakes or imprecisions i can have made.&lt;/p&gt;&#xA;" OwnerUserId="5445" LastEditorUserId="3481" LastEditDate="2013-10-25T21:03:22.397" LastActivityDate="2013-10-25T21:03:22.397" Title="Finite Difference for Hamilton Jacobi Belman" Tags="|finite-difference|boundary-conditions|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8868" PostTypeId="2" ParentId="8865" CreationDate="2013-10-25T12:40:07.677" Score="1" Body="&lt;p&gt;It's just an advection diffusion equation in one space dimension. I don't know how to do it in matlab, but there are tutorial program in most of the open source finite element libraries (deal.II, fenics, libmesh, ...) that demonstrate how to solve such problems. Solving this equation with a finite element code isn't going to be particularly difficult as long as either (i) the diffusion dominates the advection, or (ii) you properly stabilize the equation, or (iii) you can afford a sufficiently fine mesh, which you likely can because your model is only 1d.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I'm one of the authors of deal.II.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-25T12:40:07.677" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8869" PostTypeId="2" ParentId="8833" CreationDate="2013-10-25T13:56:59.393" Score="4" Body="&lt;p&gt;Thanks to Jan. I made my implementation work.&lt;br&gt;&#xA;The code below compare: Haar vs Fourier vs Chebyshev.&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/zjvFW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;from mpmath import *&#xA;&#xA;# --------- Haar wavelet approximation of a function&#xA;# algorithm from : http://fourier.eng.hmc.edu/e161/lectures/wavelets/node5.html&#xA;# implementation only handle [0,1] for the moment: scaling and wavelet fcts need to be periodice&#xA;&#xA;phi = lambda x : (0 &amp;lt;= x &amp;lt; 1) #scaling fct&#xA;psi = lambda x : (0 &amp;lt;= x &amp;lt; .5) - (.5 &amp;lt;= x &amp;lt; 1) #wavelet fct&#xA;phi_j_k = lambda x, j, k : 2**(j/2) * phi(2**j * x - k)&#xA;psi_j_k = lambda x, j, k : 2**(j/2) * psi(2**j * x - k)&#xA;&#xA;def haar(f, interval, level):&#xA;    c0 = quadgl(  lambda t : f(t) * phi_j_k(t, 0, 0), interval  )&#xA;&#xA;    coef = []&#xA;    for j in xrange(0, level):&#xA;        for k in xrange(0, 2**j):&#xA;                djk = quadgl(  lambda t: f(t) * psi_j_k(t, j, k), interval  )&#xA;                coef.append( (j, k, djk) )&#xA;&#xA;    return c0, coef&#xA;&#xA;def haarval(haar_coef, x):&#xA;    c0, coef = haar_coef&#xA;    s = c0 * phi_j_k(x, 0, 0)&#xA;    for j, k ,djk in coef:&#xA;            s += djk * psi_j_k(x, j, k)&#xA;    return s&#xA;&#xA;# --------- to plot an Haar wave&#xA;interval = [0, 1]&#xA;plot([lambda x : phi_j_k(x,1,1)],interval)&#xA;&#xA;# ---------- main&#xA;# below is code to compate : Haar vs Fourier vs Chebyshev&#xA;&#xA;nb_coeff = 5&#xA;interval = [0, 1] # haar only handle [0,1] for the moment: scaling and wavelet fcts need to be periodice&#xA;&#xA;fct = lambda x : x&#xA;&#xA;haar_coef = haar(fct, interval, nb_coeff)&#xA;haar_series_apx = lambda x : haarval(haar_coef, x)&#xA;&#xA;fourier_coef = fourier(fct, interval, nb_coeff)&#xA;fourier_series_apx = lambda x: fourierval(fourier_coef, interval, x)&#xA;chebyshev_coef = chebyfit(fct, interval, nb_coeff)&#xA;chebyshev_series_apx = lambda x : polyval(chebyshev_coef, x)&#xA;&#xA;&#xA;print 'fourier %d chebyshev %d haar %d' % ( len(fourier_coef[0]) + len(fourier_coef[1]),len(chebyshev_coef), 1 + len(haar_coef[1]))&#xA;print 'error:'&#xA;print 'fourier', quadgl(  lambda x : abs( fct(x) - fourier_series_apx(x) ), interval  )&#xA;print 'chebyshev', quadgl(  lambda x : abs( fct(x) - chebyshev_series_apx(x) ), interval  )&#xA;print 'haar', quadgl(  lambda x : abs( fct(x) - haar_series_apx(x) ), interval  )&#xA;&#xA;plot([fct, fourier_series_apx, chebyshev_series_apx, haar_series_apx], interval) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user729" LastEditorDisplayName="user729" LastEditDate="2013-10-25T14:34:10.560" LastActivityDate="2013-10-25T14:34:10.560" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8871" PostTypeId="2" ParentId="8806" CreationDate="2013-10-25T14:58:18.283" Score="4" Body="&lt;p&gt;You might look at James Nearing's book.  It's available &lt;a href=&quot;http://www.physics.miami.edu/~nearing/mathmethods/&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt;.  I don't think it's very academic, but I learned a lot from it.  &lt;a href=&quot;http://math.ucr.edu/home/baez/week260.html&quot; rel=&quot;nofollow&quot;&gt;One reviewer&lt;/a&gt; said about it: &lt;em&gt;Unlike the usual dry and formal textbook, it reads like a friendly uncle explaining things in plain English, trying to cut through the red tape and tell you how to actually think about this stuff.&lt;/em&gt;  The graphic about the Taylor's series of a sine function is actually quite moving.  For free, what could go wrong?&lt;/p&gt;&#xA;" OwnerUserId="5448" LastActivityDate="2013-10-25T14:58:18.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8872" PostTypeId="2" ParentId="8858" CreationDate="2013-10-25T14:59:07.497" Score="0" Body="&lt;p&gt;For an implementation in matlab please see this code -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dr-qubit.org/download.php?file=code/TrX.m&quot; rel=&quot;nofollow&quot;&gt;http://www.dr-qubit.org/download.php?file=code/TrX.m&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4016" LastActivityDate="2013-10-25T14:59:07.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8873" PostTypeId="1" CreationDate="2013-10-25T17:53:36.663" Score="1" ViewCount="90" Body="&lt;p&gt;I am trying to solve a set of couple ODEs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_l(r) - r W_l(r) -  f1(r) W_l'  = 0\tag 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r^2 h''_l(r) + f2 r h_l'(r) + f3 h_l(r) - f4 U_l(r) = 0 \tag 2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\kappa (U_l + h_l) + V_{l+1} + W_{l+1}  = 0\tag 3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\kappa (U_{l-2} + h_{l-2}) + f5V_l' + f6W_l' + f7 U'_{l-1} +f8U'_{l-2} = 0 \tag 4$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $V_l,W_l,h_l,U_l$ and $\kappa$. These functions and the $fn$ are only functions of $r$. Note: These are not the actual ODEs, just a representation of them. There are actually not 4 equations, but $4*l$, where $l$ is the one that appears in the spherical harmonics $Y^l_m$, and $l$ must start at $l=2$.  Also I know what the $fn$ functions are, I do not need to solve for them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method I have used to solve this is a spectral approximation. I take know and unknown functions in the ODEs and expand them in Chebyshev polynomials, for example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f1(r) = \Sigma_{i=0}^{\infty}f1_i T_i[y] - \frac{1}{2} f1_0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U_2 = \Sigma_{i=0}^{\infty}U_{2_i} T_i[y] - \frac{1}{2} U_{2_0} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying this to equation (2) I get something like: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$[\kappa * (U_{2_N} + h_{2_N}) + V_{3_N} + W_{3_N} ]*T_N + \ldots - 1/2[\kappa * (U_{2_0} + h_{2_0}) + V_{3_0} + W_{3_0} ]*T_0  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N$ is the maximum order of the Chebyshev polynomial. So if I extract the coefficients of the Chebyshev polynomials I get $4*l*N$, non-linear equations. I say nonlinear because there will be terms like $\kappa * U_{2_N}$, etc. From now on I will refer to the vector of $U_{2_N}, h_{2_N}, \ldots$ as $X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to solve this non-linear set of equations I am first using the condition that the matrix of equations, $a(\kappa,X)$ needs to satisfy $det(a) = 0$. To makes use of this I am using a singular value decomposition (SVD) to find a $\kappa$ that makes $a$ as singular as possible. To get the values for $X$ I make a new matrix $a'$ in which I remove a row from $a$ and replace it with a &quot;normalization&quot; condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma U_{2_i} T_i = 1$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then do LinearSolve[a',B]. All the elements of B are zero, expect the one that corresponds to the row that I delete, that element of B is 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The main issue I am having is that my value for $\kappa$ does not converge very well. That is, when I increase $l$ and increase $N$ I would expect to get one value of $\kappa$, however at best I can get about 4 digits of convergence, but no matter what I try to do the fifth digit is just random&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried messing with the physical model, and trying to increase the accuracy of the calculations, but nothing seems to help. The matrix is poorly conditioned: 1^8 to 1^10. And so I'm sure this is causing some issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempts at a solution&lt;/strong&gt;&#xA;1st Attempt: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried  to implement a Newton-Raphson method to try and fix the convergence. What I do is start with the matrix $a$, and treat $\kappa$ as a variable like one of the variables in $X$. Then I create the Jacobian&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J = \frac{\partial A_i }{\partial x_j}$&#xA;where $A_i$ represents an element of a vector that is made from the rows of my matrix $a$ with the additional equation that comes from the &quot;normalization&quot; condition. I include this so that I have $4*l*N +1 $ equations since now I'm trying so solve for $\kappa$ and $X$ all at once. And $x_j$ has all the elements of $X$ and $\kappa$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I use the values of $\kappa$ and $X$ I got from doing the SVD and row replacement stuff above to use as the initial guess for the Newton's method, and then do&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z = LinearSolve[J, -A]$, and I take $z$ and add it to guess $X$, and repeat. However there is no convergence, in fact the the vector $X$ goes to zero like I'm stuck at the root, basically the $z$ I get is just $-X$. I tried giving LinearSolve the Krylov option, and there the $z$ that I get just flies off to infinity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure where the problem is. I'm thinking it has to do with the fact that my Jacobian might be close to singular ($det (J) \approx.00001$). I read some other stackexchange forums that said to use a QR decomposition instead of LinearSolve, and I tried that and got no convergence there either. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2nd Attempt: &#xA;I tried using a QR algorithm for finding eigenvalues on matrix $a(\kappa,X)$. The idea is that I will look for the value of $\kappa$ which give me an eigenvalue as close to zero as possible. Again, this method does not do any better than the SVD stuff I was already using. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long, long, story short, I really need some help, figuring out just how to make this result converge better. I would want something like 10 digits, but seriously 6 or 7 would be a miracle. What's more I don't care how slow the code gets, if it works it works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1st Edit&lt;/strong&gt;&#xA;The functions are defined on the domain ($0,R$), where physically zero is the center of a sphere and $R$ is the radius. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions are:&#xA;$W_l(r = R) = 0$&#xA;$h_{in}(r=R) = h_{out}(r=R)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &#xA;$h_{l_{out}(r)} = constant*(R/r)^l$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&#xA;$h_l \rightarrow (r/R)^{l+1}$ as $r \rightarrow 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U_l \rightarrow (r/R)^{l+1}$ as $r \rightarrow 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_l \rightarrow (r/R)^l$ as $r \rightarrow 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$W_l \rightarrow (r/R)^l$ as $r \rightarrow 0$&lt;/p&gt;&#xA;" OwnerUserId="1970" LastEditorUserId="1970" LastEditDate="2013-10-26T16:30:12.033" LastActivityDate="2013-10-26T16:30:12.033" Title="Increase convergence of non-linear equations resulting from ODEs" Tags="|matrix|ode|iterative-method|convergence|newton-method|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8874" PostTypeId="1" CreationDate="2013-10-25T20:07:44.003" Score="3" ViewCount="153" Body="&lt;p&gt;I have implemented Levinson algorithm for toeplitz matrix by book: Blahut &quot;Fast algorithms for digital signal processing&quot;. Book said - modification of this algorithm for Hermitian matrices is simple job. But I could not do it. How I can modify this algorithm for Hermitian Toeplitz matrix?&lt;/p&gt;&#xA;" OwnerUserId="5449" LastEditorUserId="4575" LastEditDate="2014-01-18T22:30:03.777" LastActivityDate="2014-01-18T22:30:03.777" Title="Modification of Levinson algorithm for hermitian toeplitz matrix" Tags="|linear-algebra|algorithms|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8875" PostTypeId="2" ParentId="8874" CreationDate="2013-10-25T20:22:17.320" Score="2" Body="&lt;p&gt;Wikipedia has a comment about symmetry at the end of the following section of the description of the Levinson recursion algorithm:&lt;br&gt;&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Levinson_recursion#Introductory_steps&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Levinson_recursion#Introductory_steps&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-10-25T20:22:17.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8876" PostTypeId="2" ParentId="8591" CreationDate="2013-10-25T21:21:30.640" Score="1" Body="&lt;p&gt;I'd be very interested in the answer to this question as far a HPC and general scientific computing is concerned, but there have certainly been lots of nice results in computational geometry, for example in many of the publications and references here : &lt;a href=&quot;http://www.geometry.caltech.edu/pubs.html&quot; rel=&quot;nofollow&quot;&gt;http://www.geometry.caltech.edu/pubs.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3824" LastActivityDate="2013-10-25T21:21:30.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8877" PostTypeId="2" ParentId="8865" CreationDate="2013-10-25T21:23:09.247" Score="1" Body="&lt;p&gt;Given the way they the boundary conditions are given I would suggest a finite difference code as hinted by Paul up above. It seems that you can set up a grid for $\tau$ and $r$ and pick a forward centered scheme for $\tau$ and possibly a centered difference scheme for $r$. I can go into more detail if you know finite differencing, of if you eventually want to use this method. Just as a first step though you may want to get all of the derivative with respect to to $r$ on one side and those  w.r.t to $\tau$ on the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that Matlab has built in ODE solvers but I'm not sure about PDE solvers. I think you will learn the most by making a first attempt by yourself, by writing some code in a .m file. &lt;/p&gt;&#xA;" OwnerUserId="1970" LastActivityDate="2013-10-25T21:23:09.247" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8878" PostTypeId="2" ParentId="8591" CreationDate="2013-10-25T21:53:38.710" Score="5" Body="&lt;p&gt;Speaking from a computational electromagnetics background, I think it is a very elegant way to discretize problems. I have used it with success in eigenmode and boundary value problems. It is probably less accurate than a strict finite element discretization if you go with diagonal Hodge stars (lumped mass approximation), but I think it still achieves the same asymptotic convergence rate if you compute your Hodge stars carefully (it's probably trickier in electromagnetics than in continuum mechanics). So it is probably just a small constant factor worse (in theory, in practice it may be negligible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DEC greatly simplifies the formulations of the problems, and allows you to focus more on the physics of the problem. The construction of the Hodge stars forces you to think about the meaning of the constitutive relations and what is the physically meaningful way of performing spatial averages. It also seems to preserve many of the important symmetries of the continuous problems in the discrete setting, and it may be easier to prove these than in a finite element setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, as someone who writes code, I appreciate not having to perform quadrature during matrix assembly. Instead, you can typically compute the Hodge stars using analytic spatial averages using assumed forms of spatial variation. In electromagnetics where we have piecewise constant material properties over space, these averages can be computed exactly, making the entire problem smooth with respect to small perturbations in the spatial geometry. This greatly aids any optimization you may want to wrap around your method.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-25T21:53:38.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8879" PostTypeId="1" AcceptedAnswerId="8880" CreationDate="2013-10-24T20:01:07.720" Score="5" ViewCount="2948" Body="&lt;p&gt;In general, is there a partial trace algorithm (ideally for systems of any size) that can be coded using basic matrix operations found in software like Mathematica or Maple? All of the methods I'm aware of seem to be suited for humans, and I am struggling to think of a way of implementing them in any programming language I know. I would appreciate any kind of help with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="5442" OwnerDisplayName="sps" LastEditorUserId="651" LastEditDate="2013-10-26T19:22:54.177" LastActivityDate="2013-10-26T19:22:54.177" Title="Partial trace algorithm (original)" Tags="|quantum-mechanics|software|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8880" PostTypeId="2" ParentId="8879" CreationDate="2013-10-25T16:50:06.817" Score="2" Body="&lt;p&gt;$\newcommand{\ket}[1]{\lvert#1\rangle}$&#xA;I don't know why the methods are more suited for humans than for computer, and you can easily find a MatLab implementation by googling, for example on &lt;a href=&quot;http://www.dr-qubit.org/matlab.php&quot; rel=&quot;nofollow&quot;&gt;Toby Cubitt's webpage&lt;/a&gt;. (I haven't checked the algorithm, but I trust Toby to get such a thing correctly.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll try below to give you an explanation “suited for computers”. Let $\rho_{AB}$ be the density matrix you're interested in, with the $d_A d_B$ base vectors in the following order : &#xA;$$\ket{1,1},\ket{1,2},\dots,\ket{1,d_B},\ket{2,1},\dots,\ket{d_A,d_B}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain $\rho_B=\text{Tr}_A(\rho_{AB})$, you just have to add the $d_A$ diagonal submatrices of $\rho_AB$. Something like $\rho_B=\sum_{j=0}^{d_A-1}\rho_{AB}[jd_B+1\dots jd_B+d_A;jd_B+1\dots jd_B+d_A]$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get $\rho_A$, you can reorder the indices to go back to the previous algorithm. Or you directly take the relevant submatrices, but they are somehow scattered. $\rho_A[k;l]=\sum_{i}^{d_B}\rho_{AB}[(k-1)d_B+i;(l-1)d_B+i]$&lt;/p&gt;&#xA;" OwnerUserId="799" OwnerDisplayName="Frédéric Grosshans" LastActivityDate="2013-10-25T16:50:06.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8881" PostTypeId="1" CreationDate="2013-10-26T16:55:56.123" Score="3" ViewCount="366" Body="&lt;p&gt;lets say I have a 5 dimensional grid where each dimension has 10 points. There are 100000 combinations. Let's say I want a subset of 10000, is there a deterministic algorithm that will choose a set of points that &quot;covers&quot; the region the best and converges to the limiting case of choosing every point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I had a weighting function for each point, would there be a deterministic algorithm that would tend to choose points that are equidistant in a weighting sense as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of some possible ways in lower dimension, just wondering if there is a well defined solution (area of math) for these types of problems that extends to higher dimensions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried searching sampling methods, multi-dimensional sampling and the like, but no luck&lt;/p&gt;&#xA;" OwnerUserId="777" LastActivityDate="2013-10-26T17:35:43.207" Title="sequentially sampling an n-dimensional space" Tags="|algorithms|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8882" PostTypeId="2" ParentId="8881" CreationDate="2013-10-26T17:26:00.053" Score="2" Body="&lt;p&gt;Let's talk about the uniform sampling case. Suppose you define some spatially regular ordering for the points on the grid. For the 5 dimensional points, say the first coordinate increases the slowest, then the second increases the next slowest, and so on, and the last coordinate increases the fastest. Denote the ordered set $p_i$ for $i=0,\ldots,N-1$ where $N=100000$ is the total number of points. Let $n$ be the number of points we want to sample. If you take the set of points&#xA;$$ { p_j: j = (a+iM) \mod N } $$&#xA;for arbitrary $a$ and $i=0,\ldots,n$ where $M$ and $N$ are coprime, then the $p_j$ should not repeat until $n = N$. If you choose $a$ and $M$ reasonably ($M$ being quite a bit larger than $N$), then these points will also be relatively uniformly distributed, but perhaps with too much structure and no guarantee that they have the blue noise property.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the weighted case, you are essentially asking for a combinatorial optimization, so I doubt there would be efficient ways of solving it. On the other hand, your problem size of $10^5$ is not that big.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-26T17:26:00.053" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8883" PostTypeId="2" ParentId="8881" CreationDate="2013-10-26T17:35:43.207" Score="2" Body="&lt;p&gt;You enumerate your grid points along a &lt;a href=&quot;http://en.wikipedia.org/wiki/Space-filling_curve&quot; rel=&quot;nofollow&quot;&gt;space-filling curve&lt;/a&gt;. If you then want $M$ out of $N$ points, choose the points with indices $k\frac{N}{M}$, where $k=1\dots M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the curve you use there may be some pathological cases, but in general this should give you a good sampling of your space, assuming the grid point density is more or less homogeneous to begin with.&lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2013-10-26T17:35:43.207" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8884" PostTypeId="1" CreationDate="2013-10-27T05:57:39.693" Score="1" ViewCount="2185" Body="&lt;p&gt;Does Paraview or Visit have any filters to draw a smooth surface for a set of points? &lt;/p&gt;&#xA;" OwnerUserId="4907" LastActivityDate="2014-07-10T04:16:07.597" Title="Converting VTK point dataset of scalar values to a smooth surface" Tags="|visualization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8885" PostTypeId="2" ParentId="8833" CreationDate="2013-10-27T10:55:02.523" Score="5" Body="&lt;p&gt;This is pure haar scaling function approximation of $f(t)$&#xA;$$f(t)=\sum_{k=-\infty}^{\infty}a_k\phi(2^jt-k)$$&#xA;where $$\phi(t) = \begin{cases}1 \quad; 0 \leq  t &amp;lt; 1,\\0\quad;\mbox{otherwise.}\end{cases}$$&#xA; is haar scaling function, $j$ scaling aparameter, $k$ translation parameter and  the coefficients $$a_k=\int f(t)\phi(2^jt-k)dt$$ &#xA;So on our case if $f(t)=t$, for $k\in \mathbb{Z}$ and for $j=0$ we have to compute $a_k=\int f(t)\phi(t-k)dt$. We know $\phi (t-k)=1$ for $t\in [k,k+1)$ and zero elsewhere. So we only integrate over $[k,k+1)$.&#xA;$$a_k=\int_{k}^{k+1} t \phi(t-k)dt=\frac{2k+1}{2}$$&#xA;Hence we have &#xA;$$f(t)=\frac12\sum_{k=-\infty}^{\infty}(2k+1)\phi(t-k)$$&#xA;The following script plots haar approximation versus given function $f(x)=x$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;def init(x,j,k):&#xA;    y=np.piecewise(x,[x &amp;lt; 0, x &amp;gt; 1],[lambda x: 0, lambda x: 0, lambda x:1]) &#xA;    t=(x+k)/2**j&#xA;    ii=np.where(y==1)&#xA;    return y[ii], t[ii]&#xA;&#xA;a=-4 #lower limit of sum&#xA;b=4 #upper limit of sum&#xA;j=0&#xA;k=np.arange(a*2**j,b*2**j)&#xA;y=(2*k+1)/2 # the coefficients a_k&#xA;T=np.arange(0,2,0.01)&#xA;[p,dummy]=init(T*2**j,j,0)&#xA;phi=np.ones((len(k),len(p)))&#xA;t=np.zeros((len(k),len(p)))&#xA;&#xA;for i in range (1,len(k)+1):&#xA;    [p,t1]=init(T*2**j,j,i-1)&#xA;    t[i-1,:]=t1+a&#xA;&#xA;for l in range (1,len(k)+1):&#xA;    plt.plot(t[l-1,:],2**j*y[l-1]*phi[l-1,:],'r') &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/AHPbp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Haar approximation of $f(x)=cos(x)$ for $j=2$&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/NvrNG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="4282" LastEditDate="2013-10-28T09:02:23.223" LastActivityDate="2013-10-28T09:02:23.223" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8886" PostTypeId="1" AcceptedAnswerId="8889" CreationDate="2013-10-27T17:05:35.427" Score="11" ViewCount="2228" Body="&lt;p&gt;The netlib BLAS implementation is an excellent reference, being mostly un-optimized and well documented (e.g. &lt;a href=&quot;http://www.netlib.org/blas/zgemm.f&quot;&gt;zgemm&lt;/a&gt;).  However, it is in Fortran 77, making it somewhat inaccessible to those with a more modern programming education.  Is there a reference-level implementation of BLAS, like netlib, in C/C++?&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2018-04-01T22:35:01.780" Title="Is there a reference-level implementation of BLAS in C/C++?" Tags="|blas|education|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8887" PostTypeId="2" ParentId="8886" CreationDate="2013-10-27T17:09:04.990" Score="3" Body="&lt;p&gt;Netlib also produces &lt;a href=&quot;http://www.netlib.org/clapack/&quot; rel=&quot;nofollow&quot;&gt;CLAPACK&lt;/a&gt;, which includes BLAS, but it is just the fortran code run through f2c and is therefore a bit clunky (e.g. &lt;a href=&quot;http://www.netlib.org/clapack/cblas/zgemm.c&quot; rel=&quot;nofollow&quot;&gt;zgemm&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2013-10-27T17:09:04.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8889" PostTypeId="2" ParentId="8886" CreationDate="2013-10-27T17:31:37.177" Score="10" Body="&lt;p&gt;Have you looked at &lt;a href=&quot;http://www.gnu.org/software/gsl/&quot;&gt;GNU Scientific Library's&lt;/a&gt; &lt;a href=&quot;http://git.savannah.gnu.org/cgit/gsl.git/tree/cblas&quot;&gt;implementation&lt;/a&gt;?  I find the source code to be sufficiently readable and the routines are well documented.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-10-27T17:31:37.177" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8890" PostTypeId="2" ParentId="8886" CreationDate="2013-10-27T18:50:23.380" Score="0" Body="&lt;p&gt;I have implementations of some bits of BLAS/LAPACK in &lt;a href=&quot;https://github.com/victorliu/RNP&quot; rel=&quot;nofollow&quot;&gt;RNP&lt;/a&gt; and &lt;a href=&quot;https://github.com/victorliu/RNP2&quot; rel=&quot;nofollow&quot;&gt;RNP2&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-27T18:50:23.380" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8891" PostTypeId="2" ParentId="8886" CreationDate="2013-10-28T01:33:18.353" Score="6" Body="&lt;p&gt;A notable, C language implementation of BLAS is ATLAS. Among useful features:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Algebra routines implemented both as straightforward C as well as highly-optimized assembler assisted versions for multiple architectures and variants.&lt;/li&gt;&#xA;&lt;li&gt;The build system features an &quot;auto-tuner&quot; which compiles multiple variants of the ATLAS library to establish which one will be the fastest on the given machine.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://math-atlas.sourceforge.net/&quot;&gt;http://math-atlas.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5464" LastActivityDate="2013-10-28T01:33:18.353" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8892" PostTypeId="2" ParentId="8865" CreationDate="2013-10-28T02:13:30.483" Score="1" Body="&lt;p&gt;I'd also suggest a finite difference code (as suggested by Paul and AshikIdrisy), but rather than discretizing in both $r$ and $\tau$, I would only discretize in $r$. Semi-discretizing the differential equations in $r$ will yield a system of ODEs in $\tau$, which can then be solved in MATLAB using one of MATLAB's built-in ODE solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as PDE solvers go, you could try using MATLAB's &lt;code&gt;pdepe&lt;/code&gt; solver, since your equation looks parabolic.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-10-28T02:13:30.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8893" PostTypeId="1" AcceptedAnswerId="8896" CreationDate="2013-10-28T05:51:11.543" Score="0" ViewCount="484" Body="&lt;p&gt;This is a peice of code written in Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while flag==1&#xA;    flag = 0;&#xA;    disp('end of loop');&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Surprisingly, this does not display the string. If I remove the &lt;code&gt;disp&lt;/code&gt; statement a figure1 window appears. Can anyone explain why?&lt;/p&gt;&#xA;" OwnerUserId="5303" LastEditorUserId="4474" LastEditDate="2013-10-28T14:58:35.423" LastActivityDate="2013-10-28T14:58:35.423" Title="Problem with while-end loop" Tags="|matlab|" AnswerCount="1" CommentCount="1" ClosedDate="2013-10-29T07:36:13.393" ContentLicense="CC BY-SA 3.0" />
  <row Id="8894" PostTypeId="1" CreationDate="2013-10-28T10:12:24.837" Score="5" ViewCount="268" Body="&lt;p&gt;I want to do a simulation of sound waves including wave propagation, absorption, and reflection in 3D space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some research and I found &lt;a href=&quot;https://stackoverflow.com/questions/4956331/wave-simulation-with-python&quot;&gt;this question&lt;/a&gt; in stackoverflow but it talks about electromagnetic waves not sound waves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I can reimplement the FDTD method for sound waves but how about the sources and does it act like electromagnetic waves? Are there any resources to start with?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="5467" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-10-28T21:19:04.723" Title="Sound Waves Simulation in 3D Environment" Tags="|algorithms|simulation|wave-propagation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8895" PostTypeId="1" AcceptedAnswerId="8909" CreationDate="2013-10-28T10:47:50.107" Score="0" ViewCount="9141" Body="&lt;p&gt;&lt;strong&gt;Introduction:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a vertical segment &lt;strong&gt;S&lt;/strong&gt; That i want to move across a plane (&lt;strong&gt;Left --&gt; Right&lt;/strong&gt;), and find intersections with horizontal lines.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/jOdaw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem which i am having is the following: if the vertical segment intersects with multiple horizontal lines, as a result i'll get multiple intersections as you can you see in the following figure. Want I want is that to have just 1 intersection. Or in another way Multiple intersections, should be combined to form 1 group. The &lt;strong&gt;8&lt;/strong&gt; intersections Should be considered as &lt;strong&gt;1&lt;/strong&gt;.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/gKvVi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As a priori information I have the &lt;strong&gt;START&lt;/strong&gt; and &lt;strong&gt;END&lt;/strong&gt; points of every horizontal line.&#xA;I was thinking if I can Move the line by using the sweep algorithm, but i can't really figure out how to model the &lt;strong&gt;EVENTS&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is to have the following result, as shown in the following figure:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/lf3zg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4918" LastEditorUserId="4918" LastEditDate="2013-10-28T11:52:58.960" LastActivityDate="2013-10-30T14:07:13.247" Title="Vertical and horizontal segments intersection (Line Sweep)" Tags="|algorithms|c++|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8896" PostTypeId="2" ParentId="8893" CreationDate="2013-10-28T12:32:20.073" Score="5" Body="&lt;p&gt;That is simply because &lt;code&gt;flag&lt;/code&gt; is a defined command in matlab. Try typing &lt;code&gt;flag&lt;/code&gt; just after having started Matlab and you will see a non-empty matrix appearing. If you type &lt;code&gt;help flag&lt;/code&gt; you will see the meaning of that function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the function is selected by Matlab because you do not declare a variable called &lt;code&gt;flag&lt;/code&gt; before your code. If you write &lt;code&gt;flag=1&lt;/code&gt; before your code, it works as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, the best way is to change the name of your variable in order not to hide the Matlab function. So, use the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myflag = 1;&#xA;while( myflag == 1)&#xA;    myflag = 0;&#xA;    disp('end of loop');&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2013-10-28T12:32:20.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8899" PostTypeId="1" CreationDate="2013-10-28T17:56:17.073" Score="30" ViewCount="10765" Body="&lt;p&gt;What is a simple algorithm for computing the SVD of $2 \times 2$ matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I'd like a numerically robust algorithm, but I'll like to see both simple and not-so-simple implementations. C code accepted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references to papers or code?&lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="20417" LastEditDate="2018-04-02T19:41:00.180" LastActivityDate="2020-08-04T06:58:40.947" Title="Robust algorithm for $2 \times 2$ SVD" Tags="|linear-algebra|matrix|reference-request|svd|matrix-factorization|" AnswerCount="11" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8901" PostTypeId="2" ParentId="8899" CreationDate="2013-10-28T21:33:06.250" Score="2" Body="&lt;p&gt;I have pure C code for the 2x2 real SVD &lt;a href=&quot;https://github.com/victorliu/Cgeom/blob/master/geom_la.c&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. See line 559.&#xA;It essentially computes the eigenvalues of $A^TA$ by solving a quadratic, so it's not necessarily the most robust, but it seems to work well in practice for not-too-pathological cases. It's relatively simple.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-10-28T21:33:06.250" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8902" PostTypeId="2" ParentId="8847" CreationDate="2013-10-29T01:57:39.740" Score="3" Body="&lt;p&gt;I think you're overlooking a major point in parallelizing your code, and that is the issue of data dependencies.  All of the examples you've given are of the form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1..N&#xA;   for j=1..N&#xA;      A(i,j) = foo*A(i-1,j-1) + foo*A(i-1,j+1) + foo*A(i+1,j+1) + ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This won't parallelize since updating A depends on other loop iterations.  One approach to dealing with the dependency would be to use temporary arrays.  For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma omp parallel&#xA;   #pragma omp for&#xA;      for i=1..N&#xA;         for j=1..N&#xA;            temp(i,j) = foo*A(i-1,j-1) + foo*A(i-1,j+1) + foo*A(i+1,j+1) + ...&#xA;&#xA;   #pragma omp for&#xA;      for i=1..N&#xA;         for j=1..N&#xA;            A(i,j) = temp(i,j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Besides restructuring your algorithm, I would recommend using some kind of reporting tool with your compiler to find what is being vectorized.  Intel's compilers use -vec-report, and I believe GNU has -opt-report (double-check that).&lt;/p&gt;&#xA;" OwnerUserId="4935" LastEditorUserId="4935" LastEditDate="2013-10-29T21:07:43.143" LastActivityDate="2013-10-29T21:07:43.143" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8903" PostTypeId="2" ParentId="8899" CreationDate="2013-10-29T20:36:49.097" Score="8" Body="&lt;p&gt;The &lt;a href=&quot;http://www.gnu.org/software/gsl/&quot;&gt;GSL&lt;/a&gt; has a 2-by-2 SVD solver underlying the QR decomposition part of the main SVD algorithm for &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Singular-Value-Decomposition.html&quot;&gt;&lt;code&gt;gsl_linalg_SV_decomp&lt;/code&gt;&lt;/a&gt;. See the &lt;a href=&quot;http://fossies.org/dox/gsl-1.16/svdstep_8c_source.html&quot;&gt;&lt;code&gt;svdstep.c&lt;/code&gt;&lt;/a&gt; file and look for the &lt;code&gt;svd2&lt;/code&gt; function. The function has a few special cases, isn't exactly trivial, and looks to be doing several things to be numerically careful (e.g., using &lt;code&gt;hypot&lt;/code&gt; to avoid overflows).&lt;/p&gt;&#xA;" OwnerUserId="4474" LastActivityDate="2013-10-29T20:36:49.097" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8904" PostTypeId="1" CreationDate="2013-10-30T01:40:24.530" Score="1" ViewCount="218" Body="&lt;p&gt;Have FETI-DP or BDDC methods been applied to alternative FEM discretizations - for example, least squares finite elements? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Google searching doesn't seem to yield many results, so I'm wondering if I've missed any field-specific papers.  If not, is there a reason for this (beyond &quot;lack of interest or funding&quot;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I meant FETI-DP or BDDC as preconditioners, primarily.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2013-10-31T01:33:24.633" LastActivityDate="2013-10-31T01:33:24.633" Title="FETI-DP or BDDC with least squares FEM?" Tags="|finite-element|least-squares|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8905" PostTypeId="2" ParentId="8860" CreationDate="2013-10-30T01:57:28.813" Score="2" Body="&lt;p&gt;As Wolfgang said, your metric is wrong. Writing code alone simply isn't ever sufficient to publish a paper. A meaningful paper that is; obviously a lot of garbage gets published. I can think of 3 major areas of computational science publications:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Numerics: You developed a new numerical method, or otherwise enhanced/modified an existing one. Results show that this method is better in some respect than what exists (faster, more accurate, whatever). This could be as simple as your example of someone changing a closure constant in a turbulence model, or as complex as the invention of discontinuous galerkin methods.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Physics: You used existing numerical methods to tackle a new/unexplored physical phenomenon. In CFD, this is something like using a wind tunnel to explore a flow problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Software: You developed an entirely new software framework. There's no new numerical methods, but it's a large endeavor and helps development of other work. This is something like Wolfgang's deal.ii framework, or PETSc, or Trilinos, or the various MATLAB publications. They don't have any new numerical methods in them, but they're of obvious utility.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;None of these are better than any of the others, but the 3rd example there is unquestionably the least common. No quantity of code by itself justifies publication, and frankly most of the old guard in computational physics doesn't care about code at all - it's just a necessary evil. Examples of things that may have a lot of code, but IMO aren't worth publishing:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You wrote 500k lines of code essentially duplicating the content of Fluent's theory guide. You can tackle all the same problems, but there's no new numerics or applications. My former advisor would call this &quot;an academic exercise&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You coded up every single example out of your favorite CFD textbook, or a bunch of papers. The number of lines you coded simply doesn't matter because it simply doesn't contribute anything to discourse in the field.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-10-30T01:57:28.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8906" PostTypeId="1" AcceptedAnswerId="8945" CreationDate="2013-10-30T06:33:38.280" Score="9" ViewCount="443" Body="&lt;p&gt;This questions may be a bit off-topic in comp-sci. if it is needed please suggest where does it fit with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is regarding on how to manage all the simulation runs efficiently. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;let's say, for instance, a simulation requires fixing 2 parameters which has to be defined at a certain suggested value range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find a better outcome produced by a pair of the two parameters (by comparing the simulation outcome with experimental data, for example), one can do sensitivity analysis by defining three values for each parameter, then formulating 9 runs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;previously i use &lt;em&gt;sed&lt;/em&gt; to change inputs for each run, and tag each run by writing the value and parameter name on the folder that stores inputs and outcomes of this run. but I found this is very inefficient once the number of parameters grows (e.g. access the name of the folders in the scripts for plotting). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;then I decided to use simple numbers as folder names and store the detail by some other spreadsheets. this way is ok so far but requires some laborious work. also with the growth of the runs, it becomes common to make mistakes, such as conduct another run which has already been done a couple of days ago. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any good idea about managing these runs? I think it would be extremely important for someone doing Monte Carlo analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance! &lt;/p&gt;&#xA;" OwnerUserId="4868" LastEditorUserId="4868" LastEditDate="2013-11-03T10:14:28.663" LastActivityDate="2013-11-03T10:14:28.663" Title="suggestion for managing simulation runs?" Tags="|monte-carlo|data-management|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8907" PostTypeId="2" ParentId="242" CreationDate="2013-10-30T09:45:09.043" Score="2" Body="&lt;p&gt;There is something called the ODD (Overview, Design, and Details) protocol, proposed by Volker Grimm and others for describing an agent based model. It consists of a list of elements that are needed for understanding the functioning of an ABM and aims at making descriptions of such models more standardised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The checklist of what has to be described consists of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Overview&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Purpose&lt;/li&gt;&#xA;&lt;li&gt;Entities, state variables, and scales&lt;/li&gt;&#xA;&lt;li&gt;Process overview and scheduling&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Design&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Basic principles&lt;/li&gt;&#xA;&lt;li&gt;Emergence&lt;/li&gt;&#xA;&lt;li&gt;Adaptation&lt;/li&gt;&#xA;&lt;li&gt;Objectives&lt;/li&gt;&#xA;&lt;li&gt;Learning&lt;/li&gt;&#xA;&lt;li&gt;Prediction&lt;/li&gt;&#xA;&lt;li&gt;Sensing&lt;/li&gt;&#xA;&lt;li&gt;Interaction&lt;/li&gt;&#xA;&lt;li&gt;Stochasticity&lt;/li&gt;&#xA;&lt;li&gt;Collectives&lt;/li&gt;&#xA;&lt;li&gt;Observation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Details&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialisation&lt;/li&gt;&#xA;&lt;li&gt;Input data&lt;/li&gt;&#xA;&lt;li&gt;Submodels&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;More details can be found in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S030438001000414X&quot; rel=&quot;nofollow&quot;&gt;Grimm, V., Berger, U., DeAngelis, D. L., Polhill, J. G., Giske, J., &amp;amp; Railsback, S. R. (2010). The ODD Protocol: a Review and First Update. Ecological Modelling, 221, 2760–2768.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5488" LastActivityDate="2013-10-30T09:45:09.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8908" PostTypeId="1" AcceptedAnswerId="8916" CreationDate="2013-10-30T13:26:32.590" Score="2" ViewCount="947" Body="&lt;p&gt;I am trying to use the nonlinear fitting routines of MINPACK for fitting a rather complicated equation of state to a set of experimental data. A subset of the data is fitted fairly well to a simplified form of the EoS, which gives me some confidence that the implementation is, in principle, working. However, it seems to be impossible to fit the entire dataset to the extended form of the EoS, although the dataset is quite smooth and of good quality, and although my initial guesses are close to the expected solution and already give a decent fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what irritates me most:&#xA;The return status of the fit encoded by the INFO parameter is not too elucidating. I have the impression that INFO=1 or 2 is deemed ok; according to the comments in the code it means &quot;algorithm estimates that the relative error in the sum of squares is at most tol= 0.149E-07&quot; or &quot;algorithm estimates that the relative error between x and the solution is at most tol= 0.149E-07&quot;, respectively. However, I get this return status for very different solutions (using different starting guesses).&#xA;Then there is the status INFO=4, &quot;fvec is orthogonal to the columns of the jacobian to machine precision&quot;, which I don't really understand. I often get it when there are only 5 iterations (for 4 fitting parameters, where it seems that 4 iterations vary each of the parameters individually by a small amount to probe the local surroundings of the solution). This can happen for different initial guesses, close or not so close to the expected solution for a good fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody explain this behavior and tell me if it possible, for instance, to change the size of those initial test variations of each parameter or to put constraints (bounds) on the range of each parameter? What I do right now is simply to force each parameter to lie between given bounds with max and min statements before each call of the model function by lmdif1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thomas&lt;/p&gt;&#xA;" OwnerUserId="4909" LastActivityDate="2013-10-31T00:33:36.717" Title="Some questions about MINPACK usage and messages" Tags="|optimization|nonlinear-programming|regression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8909" PostTypeId="2" ParentId="8895" CreationDate="2013-10-30T14:07:13.247" Score="1" Body="&lt;p&gt;I put this python script together from various codes I found on the web,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/563198/how-do-you-detect-where-two-line-segments-intersect?rq=1&quot;&gt;https://stackoverflow.com/questions/563198/how-do-you-detect-where-two-line-segments-intersect?rq=1&lt;/a&gt;, there are plenty of good answers here, this one is from Kris.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So basically,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create input list of line segments&lt;/li&gt;&#xA;&lt;li&gt;Create input list of test lines (the red lines in your diagram).&lt;/li&gt;&#xA;&lt;li&gt;Iterate though the intersections of every line&lt;/li&gt;&#xA;&lt;li&gt;Create a set which contains all the intersection points.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have recreated you diagram and used this to test the intersection code. It gets the two intersection points in the diagram correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C and C++ has equivalents to these data structures, and you can code the intersection algorithm in C too (it's fairly simple).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bAFdL.png&quot; alt=&quot;Line segments&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;&#xA;# Thanks to @Kris for the intersection algorithm in python&#xA;# https://stackoverflow.com/questions/563198/how-do-you-detect-where-two-line-segments-intersect&#xA;def find_intersection( p0, p1, p2, p3 ) :&#xA;&#xA;    s10_x = p1[0] - p0[0]&#xA;    s10_y = p1[1] - p0[1]&#xA;    s32_x = p3[0] - p2[0]&#xA;    s32_y = p3[1] - p2[1]&#xA;&#xA;    denom = s10_x * s32_y - s32_x * s10_y&#xA;&#xA;    if denom == 0 : return None # collinear&#xA;&#xA;    denom_is_positive = denom &amp;gt; 0&#xA;&#xA;    s02_x = p0[0] - p2[0]&#xA;    s02_y = p0[1] - p2[1]&#xA;&#xA;    s_numer = s10_x * s02_y - s10_y * s02_x&#xA;&#xA;    if (s_numer &amp;lt; 0) == denom_is_positive : return None # no collision&#xA;&#xA;    t_numer = s32_x * s02_y - s32_y * s02_x&#xA;&#xA;    if (t_numer &amp;lt; 0) == denom_is_positive : return None # no collision&#xA;&#xA;    if (s_numer &amp;gt; denom) == denom_is_positive or (t_numer &amp;gt; denom) == denom_is_positive : return None # no collision&#xA;&#xA;&#xA;    # collision detected&#xA;&#xA;    t = t_numer / denom&#xA;&#xA;    intersection_point = [ p0[0] + (t * s10_x), p0[1] + (t * s10_y) ]&#xA;    return intersection_point&#xA;&#xA;# Create input data.&#xA;# black lines&#xA;line_segments = [[(1,4), (4,4)], [(2,3), (5,3)], [(3,2), (6,2)], [(6.5, 1), (7,1)], [(7.5, 0), (8.5,0)]]&#xA;# red lines&#xA;test_segments = [[(4.5,0), (4.5,4.5)], [(6.25, 0), (6.25, 4.5)]]&#xA;&#xA;# Check all lines for intersections&#xA;intersections = set()&#xA;for test_segment in test_segments:&#xA;    for line_segment in line_segments:&#xA;        p0, p1 = test_segment[0], test_segment[1]&#xA;        p2, p3 = line_segment[0], line_segment[1]&#xA;        result = find_intersection(p0, p1, p2, p3)&#xA;        if result is not None:&#xA;            intersections.add(tuple(result))&#xA;&#xA;print intersections&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-10-30T14:07:13.247" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8910" PostTypeId="2" ParentId="8904" CreationDate="2013-10-30T14:27:45.793" Score="1" Body="&lt;p&gt;I don't see any reason why they wouldn't work. That said, domain decomposition as a discretization technique is dead -- it was used for a while in the 1990s and early 2000s when we didn't know how to solve very large linear systems that are distributed across processors. However, with good partitioners and libraries such as PETSc and Trilinos that can store linear systems and provide ghost elements on vectors, everybody has moved to solving PDEs as one big system across all processors, rather than one small system per processor.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-30T14:27:45.793" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8911" PostTypeId="2" ParentId="8899" CreationDate="2013-10-30T14:30:47.710" Score="7" Body="&lt;p&gt;When we say &quot;numerically robust&quot; we usually mean an algorithm in which we do things like pivoting to avoid error propagation. However, for a 2x2 matrix, you can write the result down in terms of explicit formulas -- i.e., write down formulas for the SVD elements that state the result &lt;em&gt;only in terms of the inputs&lt;/em&gt;, rather than in terms of &lt;em&gt;intermediate values previously computed&lt;/em&gt;. That means that you may have cancellation but no error propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point simply is that for 2x2 systems, worrying about robustness is not necessary.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-10-30T14:30:47.710" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8912" PostTypeId="2" ParentId="8259" CreationDate="2013-10-30T16:23:51.727" Score="0" Body="&lt;p&gt;It really depends on the application you run.  GPUGRID.net runs on machines that do not have ECC on and everything is fine.  The results are as good as they are on any other platform.  Acellera also sells hardware with GeForce cards and in only very few cases the GPUs have failed.  GeForce is all you need.&lt;/p&gt;&#xA;" OwnerUserId="5491" LastActivityDate="2013-10-30T16:23:51.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8913" PostTypeId="1" AcceptedAnswerId="8917" CreationDate="2013-10-30T17:10:35.240" Score="5" ViewCount="1614" Body="&lt;p&gt;I am starting a self study course in computational fluid dynamics.&#xA;I use ( An introduction to computational fluid dynamics : the finite volume method)&#xA;I am learning Python programming language.&#xA;The problem is that I cannot understand most things in the book&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;what do I need before beginning this course ?&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;My problem in the textbook (An introduction to computational fluid dynamics: the finite volume method):&lt;br&gt;&#xA;I didn't understand the governing eqns. in ch.2 so I had to use another textbook(Fluid Mechanics Frank White)for refresh my information about fluid mechanics's eqns.&lt;/p&gt;&#xA;" OwnerUserId="5040" LastEditorUserId="5040" LastEditDate="2013-10-31T14:55:51.683" LastActivityDate="2017-05-23T02:27:31.497" Title="beginning Computational Fluid Dynamics" Tags="|fluid-dynamics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8914" PostTypeId="1" AcceptedAnswerId="8975" CreationDate="2013-10-30T17:52:39.393" Score="4" ViewCount="116" Body="&lt;h2&gt;Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Let's say I can write a model as the Hermitian eigensystem:&#xA;$$ A x = \lambda x $$&#xA;where $A \in \mathbb{C}^{n\times n}$ is Hermitian, or as the generalized Hermitian eigensystem:&#xA;$$ \tilde A \tilde x = \lambda \tilde B \tilde x $$&#xA;where $\tilde A, \tilde B \in \mathbb{C}^{\tilde n \times \tilde n}$ are Hermitian and $\tilde B$ is positive definite.&#xA;How much smaller must $\tilde n$ be than $n$ for the generalized model to be more efficient?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Qualifications&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I understand that there are many other factors that influence the answer.  A general intuition would be best, but, if added complexity is unavoidable, assume:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Serial solver&lt;/li&gt;&#xA;&lt;li&gt;Direct solver&lt;/li&gt;&#xA;&lt;li&gt;$A,\tilde A, \tilde B$ are dense&lt;/li&gt;&#xA;&lt;li&gt;The $m$ eigen-pairs with lowest eigenvalues are desired, where $n / m \approx 10$&lt;/li&gt;&#xA;&lt;li&gt;$A$ and $\tilde A, \tilde B$ model the same system, so the relative differences in the $m$ eigen-pairs are small&lt;/li&gt;&#xA;&lt;li&gt;$\tilde B = I + \hat{B}$ where $\text{rank}(\hat{B}) = p$, and $p \approx 2 m = n/5$ &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Answers for a different set of qualifications might be useful to other readers as well.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2013-11-06T16:49:09.390" LastActivityDate="2013-11-06T16:49:09.390" Title="What is the added cost of generalizing an eigensystem?" Tags="|linear-algebra|performance|eigensystem|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8915" PostTypeId="1" CreationDate="2013-10-30T22:05:07.573" Score="3" ViewCount="260" Body="&lt;p&gt;I would like use N-body simulation for measure a torque on an object in a liquid (helium). It's a 2D study. On the site : &lt;a href=&quot;http://www.browndeertechnology.com/docs/BDT_OpenCL_Tutorial_NBody-rev3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.browndeertechnology.com/docs/BDT_OpenCL_Tutorial_NBody-rev3.html&lt;/a&gt; I can understand the method. But it's only for free object. How can I do for compute particles when they reached fixed wall ? and for compute object that have only one degree of freedom ? Could you help me to start the equations for the simulation ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider red lines with no mass, black disk with no mass. All walls are not computed. Object I would like to compute torque is composed of one part with vacuum and another part with helium (exactly density of helium if there was no walls). Gravity of Earth is perpendicular to the screen, and it is considered perfectly the same (parallel). So like there is only one layer of particles to compute it's possible to forget gravity from Earth I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to compute helium :  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Density (the object change the density in helium due to the lack of attraction), it's very important  &lt;/li&gt;&#xA;&lt;li&gt;Movement of particles  &lt;/li&gt;&#xA;&lt;li&gt;Torque  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't compute (superfluid):  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;friction (helium)  &lt;/li&gt;&#xA;&lt;li&gt;temperature (helium)  &lt;/li&gt;&#xA;&lt;li&gt;walls&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/9JBxd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/iMGcY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The worst is the density, how take it in equation ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe this sort of program exist ? Could you help me ?&lt;/p&gt;&#xA;" OwnerUserId="5494" LastEditorUserId="5494" LastEditDate="2013-11-01T14:51:46.540" LastActivityDate="2013-11-01T14:51:46.540" Title="N-body simulation for particles" Tags="|simulation|c|fluid-dynamics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8916" PostTypeId="2" ParentId="8908" CreationDate="2013-10-31T00:33:36.717" Score="0" Body="&lt;p&gt;You haven't said which of the MINPACK routines you're using to solver your least squares problem.  I'll assume that you're using LMDIF1 (which uses finite difference approximations for the dervatives with an automatically determined step size for the finite difference derivatives.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could also be a result of non-smoothness of the user supplied functions on the scale of the step size used in the finite difference approximations.  In particular, if your function evaluations involve any approximations (e.g. evaluating an integral to some limited precision or simply by being done in single precision rather than double precision), so that they aren't accurate to full double precision, then this can cause the INFO=4 failure.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;if this is your problem, then consider using the LMDIF function and supplying a parameter that tells MINPACK how imprecise your function evaluations are.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-10-31T00:33:36.717" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8917" PostTypeId="2" ParentId="8913" CreationDate="2013-10-31T02:40:44.900" Score="9" Body="&lt;p&gt;Generally, the prerequisites for computational fluid mechanics is introductory courses to partial differential equations and numerical analysis. I haven't used &quot;An introduction to computational fluid dynamics : the finite volume method&quot;, so I can't say anything about the textbook.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would suggest checking out an online course by Lorena Barba:&#xA;         &lt;a href=&quot;http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/&quot;&gt;http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;She, also, has lectures posted on iTunesU. It is a useful resource.&lt;/p&gt;&#xA;" OwnerUserId="1031" LastActivityDate="2013-10-31T02:40:44.900" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8919" PostTypeId="1" AcceptedAnswerId="8921" CreationDate="2013-10-31T08:37:07.807" Score="3" ViewCount="156" Body="&lt;p&gt;I have a system of nonlinear equations of which I know it has a single root I am interested in, and has a continuum of roots I am not interested in. I am currently using Newton with line-searching in an attempt to find that single root, but the algorithm will obviously always converge to one of the roots from the continuum, as there are infinity of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I go about excluding such roots from my system of equations?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first idea was to add sharp Gaussian functions to my system of equations that are non-zero at those roots, and almost zero everywhere else. I am currently implementing this, but it already seems like convergence is now very slow and inconsistent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would I be better off rewriting my problem as an optimization problem? I would have to find a global minimum of a constrained function. &lt;/p&gt;&#xA;" OwnerUserId="1264" LastActivityDate="2013-10-31T20:48:35.307" Title="Excluding roots from a system of nonlinear equations" Tags="|optimization|convergence|nonlinear-programming|newton-method|constrained-optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8920" PostTypeId="2" ParentId="8913" CreationDate="2013-10-31T10:38:57.887" Score="2" Body="&lt;p&gt;I have attempted to document some resources for a newbie interested in automotive CFD here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.vespalabs.org/Projects/Vespa_CFD//3D_Model#Additional_Resources&quot; rel=&quot;nofollow&quot;&gt;http://www.vespalabs.org/Projects/Vespa_CFD%2F%2F3D_Model#Additional_Resources&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I highly recommend the Hucho book as a starting point for describing the problem space from a practical point of view.&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2013-10-31T10:38:57.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8921" PostTypeId="2" ParentId="8919" CreationDate="2013-10-31T20:48:35.307" Score="3" Body="&lt;p&gt;You could try using an optimization formulation to exclude undesirable roots via constraints. However, global optimization is a notoriously difficult problem and solving such a problem deterministically could be challenging. If you can exclude all suboptimal local minima in some way, that would make it easier. For deterministic approaches, you're essentially limited to calling BARON from GAMS (unless you want to write your own branch-and-bound framework with convex and concave relaxations implemented using interval arithmetic and automatic differentiation). Nondeterministic approaches will converge with probability one; you could try using such an approach, and if the solution it returns is not a root, use that solution as an initial guess for a Newton-like method. I doubt this approach will work especially well; to work, the guess for your Newton method would have to be within the basin of convergence of the root you want, and it can't preferentially converge to other roots. I think you're better off exploiting &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8676/finding-zeroes-of-an-infinitely-differentiable-function-of-100-to-1000-variabl&quot;&gt;the structure Nico discussed in his answer to your other question&lt;/a&gt;; if you can express the desired root as a simpler closed-form expression without an infinity of roots (for instance, without all of the $C_i$), you'll have a more tractable problem.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-10-31T20:48:35.307" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="8922" PostTypeId="1" CreationDate="2013-10-31T22:30:52.613" Score="3" ViewCount="198" Body="&lt;p&gt;Can anyone tell me some very good/efficient numerial algorthims for GPU/CUDA to compute multiplication/ between sparse matrices (its good if you can recommend me some research papers)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I googled some papers on sparse vectors, but it looks like they are more interested in the operations involves sparse matrix and dense vectors, but what I am dealing with is some math operations involves only sparse matrices and sparse vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4170" LastActivityDate="2013-11-01T02:22:16.133" Title="Numerically efficient way to compute sparse-matrix arithmetic on GPU?" Tags="|sparse-matrix|gpu|cuda|vector|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8923" PostTypeId="2" ParentId="8922" CreationDate="2013-11-01T02:22:16.133" Score="2" Body="&lt;p&gt;Nvidia's cuSPARSE library implements the functions in the standard sparse blas library, but the sparse blas doesn't include a sparse matrix times sparse vector multiply.  However, cuSPARSE also has a sparse matrix times sparse matrix multiplication routine that goes beyond the sparse blas.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-11-01T02:22:16.133" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8924" PostTypeId="2" ParentId="8638" CreationDate="2013-11-01T10:19:03.230" Score="0" Body="&lt;p&gt;I have solved my problem now. The reason why I lost accuracy doesn't even occur in the question. Let's first address the proposed solution ideas from the question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Trying to learn more was a good idea in this case.&lt;/li&gt;&#xA;&lt;li&gt;Analytical formulas are attractive when they are reasonably simple. This is not the case here.&lt;/li&gt;&#xA;&lt;li&gt;Straightforward linear interpolation sounds like a good idea, at least for verification. Perhaps I will implement it one day, it can't be too difficult. See also the answer by Wolfgang Bangerth.&lt;/li&gt;&#xA;&lt;li&gt;Note sure. I have done something related instead. I approximate the smooth probability density function $f(x)$ by a piecewise analytical function of the form $a \cdot x^b$. Its antiderivative $\frac{a}{b+1}x^{b+1}$ has the same simple form, and the moments $\int f(x) x^n dx$ also lead to integrals of the same form.&lt;/li&gt;&#xA;&lt;li&gt;Rejection methods shouldn't be outright rejected, but I haven't tried this one.&lt;/li&gt;&#xA;&lt;li&gt;Approximating $F^{-1}(r)$ is one of the &quot;correct&quot; proceedings. What does suitable mean here? The inverse cumulative distribution function is monotonically increasing, and a near zero probability density over an extended range translates into a very steep slope of $F^{-1}(r)$. Nonuniform rational interpolation is one option able to cope with these features. The nonuniform grid leads to a $O(\log n)$ effort for a single function evaluation, but this is normally still fast enough in practice. See also the answer by Pedro. (However, the $O(n)$ effort for a single function evaluation with Chebyshev interpolation instead of a $O(1)$ or $O(\log n)$ effort has &lt;a href=&quot;https://scicomp.stackexchange.com/q/3415&quot;&gt;made me uneasy in the past&lt;/a&gt;.)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;But what about the implicit &quot;real&quot; question?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I noticed that the accuracy of my final results is not great, and I suspect that I lose accuracy because ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I actually wasn't just given a single probability density function, but a one parameter family of probability density functions, tabulated on a sufficiently fine grid relative to the one parameter. I handled this by linear interpolation between the inverse cumulative distribution functions I had precomputed. However, even in case the probability density function is well approximated by a linear interpolation between the tabulated probability density functions, my above proceeding can lead to unacceptable errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;correct&quot; solution is much simpler. It uses the composition method. First draw a number uniform at random to select between the available precomputed distributions, and then sample the value from the selected distribution. This actually generates the &quot;exactly correct&quot; distribution, in case the probability density function is really given as a weighted sum of available precomputed distributions.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2013-11-01T10:19:03.230" CommentCount="4" CommunityOwnedDate="2013-11-01T10:19:03.230" ContentLicense="CC BY-SA 3.0" />
  <row Id="8925" PostTypeId="1" CreationDate="2013-11-01T11:59:34.337" Score="20" ViewCount="30354" Body="&lt;p&gt;I have a mixed integer programming problem. And I am current using GLPK as my solver. But I found that GLPK is good for Linear Programming problem, but for Mixed Integer programming, it requires much longer time, therefore doesn't meet our requirement. I am so seeking other software.  Is there any other good open source tools to solve mixed integer programming problem with fast speed? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="5223" LastActivityDate="2021-12-22T11:06:59.563" Title="What's the fastest software(open source) to solve mixed integer programming problem" Tags="|optimization|software|linear-programming|mixed-integer-programming|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8926" PostTypeId="2" ParentId="8906" CreationDate="2013-11-01T13:57:36.900" Score="2" Body="&lt;p&gt;I think we'd need to know a little bit more about your workflow to make any serious recommendations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest treating your runs like a key-value store.  Create a simple database for all of your metadata for each run, and then hash any relevant information from your run into a key that you assign to each output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the simplest situation, you would use a text file for your metadata store, and safely append lines of metadata about each run to your text file.  You can then store your output runs however you like (a single directory, backups with a listing of the contents, etc...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can implement this strategy in any language you like, but this would be trivial in Python.  You could also take advantage of some nice features as Python's ability to read and write JSON data or interact with SQL databases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach implements a very simple light-weight database.  There are heavier strategies that provide more safety guarantees, a new one you might be interested in is &lt;a href=&quot;http://www.scidb.org/about/index.php&quot; rel=&quot;nofollow&quot;&gt;SciDB&lt;/a&gt;.  Databases provide stronger guarantees about your data and help you scale your approach for larger datasets.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2013-11-01T13:57:36.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8927" PostTypeId="2" ParentId="8925" CreationDate="2013-11-01T14:01:43.527" Score="8" Body="&lt;p&gt;Mixed integer linear programming problems are much harder to solve than linear programming problems.  In terms of computational complexity, LP's can be solved in polynomial time while solving MILP is an NP-Hard problem.  The known algorithms for solving MILP's have exponential worst case complexity.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other software packages for mixed integer linear programming that you could look at, including SCIP (free for academic use), CPLEX (commercial but has an academic licensing option) and GUROBI (also commercial with an academic licensing option.)  One or more of these packages might be substantially faster than GLPK on your problems, but don't expect any of them to be nearly as fast at solving MILP as they are in solving similarly sized LP's.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-11-01T14:01:43.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8928" PostTypeId="1" CreationDate="2013-11-01T14:51:33.630" Score="6" ViewCount="490" Body="&lt;p&gt;&lt;strong&gt;Short version:&lt;/strong&gt; Is it ever a good idea to use multiple languages in scientific codes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Long version:&lt;/strong&gt; May be its just me but these days I often see scientific codes written in multiple languages. The argument is that one language is used for performance and other one for ease of use, pretty syntax, because everyone else is using it and what not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But from a users point of view its twice the work. I mean life is difficult as it is and now you must learn not one but two languages if you wish to adapt or extend the code to suit your own needs (and not everyone is a star programmer who can be productive quickly). There is lot more complexity at every stage, e.g., added dependencies just to configure/build the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my field I see that codes which are written in a single language tend to have many users which contribute to it over time. I am guessing it is also true for successful codes in other fields but I am not sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is that is using multiple languages ever a good idea in traditional scientific computing or is it a case of over engineering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only situation I can imagine when it may be needed is when you have to couple two or more codes, i.e., when there is no easy way to pass/share information between them. But often in those cases it is better to just rewrite. The climate/weather modeling community does a lot of coupling work but most of their codes and even couplers are written in a single language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do down vote then please do comment.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2013-11-01T14:57:38.377" LastActivityDate="2013-12-01T23:35:27.547" Title="Using multiple languages in scientific codes" Tags="|languages|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8929" PostTypeId="2" ParentId="8928" CreationDate="2013-11-01T16:19:06.747" Score="6" Body="&lt;p&gt;Most scientific computing is probably done with a more flexible higher level language and then dropping into a lower level language to deal with speed bottlenecks.  For example python/cython/f2py, R/Rcpp, matlab/mex.  The celebrated (or more cynically, hyped) language Julia is explicitly designed to avoid this multi-language workflow which they call the &quot;two language problem&quot;, for example as explained &lt;a href=&quot;http://www.lanl.gov/orgs/hpc/salishan/salishan2013/Karpinski.pdf&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-11-01T16:19:06.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8930" PostTypeId="1" CreationDate="2013-11-01T16:25:22.800" Score="6" ViewCount="4006" Body="&lt;p&gt;As it known, according to the Polar Decomposition, square matrix can be expressed in the next form $$M=QR$$ ($Q$ - orthogonal matrix $R$ - positive-semidefinite Hermitian matrix) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to find this $Q$ factor for the case of $3\times 3$ matrix. For this purpose I use next well known iterative formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Q_{i+1} = \frac{1}{2}\left[ Q_i+(Q_i^{-1})^T \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $\det Q_0\neq0 $&#xA;However, on practice it works a little bit slowly (it takes, more than 15 iteration before finds right answer). Is there any other, faster algorithms exist to perform Polar Decomposition? I have found exact formula for finding $Q$ factor for the case of $2\times 2$ matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q = M + \mathrm{sign}(\det M)\begin{pmatrix}&#xA;  d &amp;amp; -c\\&#xA;  -b &amp;amp; a\\&#xA;    \end{pmatrix}$$&#xA;where the initial matrix $$M=\begin{pmatrix}&#xA;  a &amp;amp; b\\&#xA;  c &amp;amp; d\\&#xA;    \end{pmatrix}$$&#xA; Does such formula exist for $3\times 3$ matrix?&lt;/p&gt;&#xA;" OwnerUserId="5278" LastEditorUserId="127" LastEditDate="2016-04-22T16:53:42.817" LastActivityDate="2016-04-22T16:53:42.817" Title="Fast algorithm for Polar Decomposition" Tags="|linear-algebra|matrix|approximation|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8931" PostTypeId="2" ParentId="8928" CreationDate="2013-11-01T17:26:24.853" Score="2" Body="&lt;p&gt;My experience matches with k20's on this topic.  When you start a project, you choose a language consistent with one set of performance / flexibility options.  Of course, halfway through, maybe you need additional performance - and have to incorporate lower-level languages.  If you are starting out with a well-developed plan, maybe you can avoid this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't exactly unusual even outside of scientific computation, either - performance-critical code is usually put in the fastest reasonable language (e.g. inline assembler).&lt;/p&gt;&#xA;" OwnerUserId="4494" LastActivityDate="2013-11-01T17:26:24.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8932" PostTypeId="2" ParentId="8928" CreationDate="2013-11-01T22:08:08.903" Score="5" Body="&lt;p&gt;Yes, if you are comfortable with the consequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Engineering always involves tradeoffs. The usual justification for using multiple languages is that a lower-level language (Fortran, C, C++) is used for performance-critical sections of code, and a higher-level language is used to provide a convenient interface and rapid code development. Julia, MATLAB, and NumPy/SciPy are good examples of such projects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This decomposition can also be convenient for users working with such a project; advanced functionality is more likely to be written in a compiled language, whereas basic functionality is more likely to be written in an interpreted language. Furthermore, there is nothing stopping a project from providing &quot;slow&quot; versions of the performance-critical code in the interpreted language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're right, though, that the tradeoff can be maintainability; if you want to modify or maintain the code yourself, you now have to know enough about two languages instead of one. Similarly, if you pick one language, you're stuck with the libraries, performance characteristics, and community of that language. Using two languages gives you slightly more flexibility when it comes to library selection, performance, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the number of languages used in a project increases, maintainability becomes more problematic, and strict modular decomposition is helpful. It's useful if the core code is written in one or two languages and there are interfaces to seven other languages. (e.g., PETSc is written in C, but C is close enough to C++ that it can be called from C++, and it has Fortran, MATLAB, Python, and Mathematica interfaces; I think a Julia interface is even in the works). It's not so helpful if the core code is written in many different languages because it exacerbates the maintainability problem you already identified.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-11-01T22:08:08.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8933" PostTypeId="2" ParentId="8925" CreationDate="2013-11-01T22:17:11.287" Score="18" Body="&lt;p&gt;If you want something open-source, you probably want to try COIN's CBC code (they also have a couple other MILP solvers, like a branch-and-price framework, or SYMPHONY).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gurobi and CPLEX will be considerably faster, and as of the 2011 or 2012 INFORMS meeting, Gurobi was faster than CPLEX (though the performance metrics are of course problem dependent). On the MILPs solved in my thesis, Gurobi was approximately 15-100 times faster than CBC, and CPLEX was almost as fast as Gurobi, but very slightly slower (like 12-80 times faster).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the worst-case performance is indeed exponential, the execution time will depend heavily on problem structure. It's unlikely that you'll be able to solve an MILP with millions of variables unless you exploit special structure (maybe if it's a stochastic program that can be decomposed into many much smaller problems), but it's entirely possible to solve nontrivial MILPs with thousands of variables in under a minute. (Of course, it's also possible for these problems to take an hour or more to solve.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Brian Borchers notes, CPLEX and Gurobi both have free licenses available for some researchers, one of these two software packages would really be the best to use as a general-purpose MILP solver.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-11-01T22:17:11.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8937" PostTypeId="1" AcceptedAnswerId="8939" CreationDate="2013-11-02T10:41:27.167" Score="1" ViewCount="2330" Body="&lt;p&gt;Is there a master degree in HPC (High Performance Computing) in US?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am asking this question because in my early research I found no direct mentioning of HPC instead invidual courese in parallel programming , distributed computing etc in additional to cs generic core course. I did not found so far a focused program in HPC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the programs somtimes are under math or engineering department which is little confuisng for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently waiting for responses from some of those universities..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if you give me hint and refrence to those universities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks....&lt;/p&gt;&#xA;" OwnerUserId="5512" LastActivityDate="2018-08-25T17:51:32.310" Title="Masters in HPC (High Performance Computing)" Tags="|parallel-computing|hpc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8939" PostTypeId="2" ParentId="8937" CreationDate="2013-11-02T15:00:18.747" Score="6" Body="&lt;p&gt;Yes, a number of universities have graduate programs in HPC.  Often they're referred to as computational science or scientific computation programs.  I'm currently pursuing a PhD in scientific computation through the University of Minnesota, and a Master's degree is also offered.  &lt;a href=&quot;https://www.sc.fsu.edu/graduate/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;Florida State&lt;/a&gt; has a department of scientific computing and a graduate program, &lt;a href=&quot;https://www.ices.utexas.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICES&lt;/a&gt; at Texas is a top program, and both offer Master's degree.  &lt;a href=&quot;https://www.cc.gatech.edu/academics/degree-programs/masters/computational-science-engineering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Georgia Tech&lt;/a&gt; has a Computational Science and Engineering degree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you may want to consider when looking at such graduate programs is the structure of the program/department.  For example, at Minnesota, there is no Sci. Comp. department, but rather it's a combination of a number of different departments and certain faculty members are associated with the Sci. Comp. department.  This can have an impact on funding opportunities; a math teaching assistantship may be reserved only for students enrolled in the math department.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HPC is interdisciplinary by nature, so you have a lot of options when it comes to research areas.  Are you interested more in the mathematics and algorithms?  More on the hardware and design?  More computer science-related topics like compilers and language development?  Are you interested more in the applied sciences like chemistry, geosciences, aerospace eng., etc. and using HPC in those settings?  If you have an idea of what you'd like to study, you may want to seek out researchers in those areas and then look at what institutions they're associated with.  Even if the school doesn't have sci. comp. program, you can still pursue a degree in something like Math or Computer Science with research topic related to HPC.&lt;/p&gt;&#xA;" OwnerUserId="4935" LastEditorUserId="12000" LastEditDate="2018-08-25T17:51:32.310" LastActivityDate="2018-08-25T17:51:32.310" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8940" PostTypeId="2" ParentId="8828" CreationDate="2013-11-02T15:59:25.980" Score="1" Body="&lt;p&gt;If there's some model for the signal for $t &amp;lt; t_0$ and $t &amp;gt; t_f$, perhaps it would be possible to try to fit the full signal using a Metropolis-Hastings or Goodman-Weare algorithm. This could then also yield some information about the probability distributions for the parameters involved.&lt;/p&gt;&#xA;" OwnerUserId="5514" LastActivityDate="2013-11-02T15:59:25.980" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8941" PostTypeId="2" ParentId="8930" CreationDate="2013-11-02T16:33:53.243" Score="1" Body="&lt;p&gt;Section 2.5 in &lt;a href=&quot;http://books.google.com/books?id=AJdfQL0rgrgC&amp;amp;printsec=frontcover&quot; rel=&quot;nofollow&quot;&gt;Continuum Mechanics by A.J.M. Spencer&lt;/a&gt; is devoted to the 3x3 polar decomposition.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-11-02T16:33:53.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8942" PostTypeId="1" AcceptedAnswerId="10696" CreationDate="2013-11-02T19:36:15.553" Score="13" ViewCount="318" Body="&lt;p&gt;Suppose I have the following interesting function:&#xA;$$ f(x) = \sum_{k\geq1} \frac{\cos k x}{k^2(2-\cos kx)}. $$&#xA;It has some unpleasant properties, like its derivative not being continous at  rational multiples of $\pi$. I suspect a closed form does not exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can compute it by computing partial sums and using Richardson extrapolation, but the problem is that it is too slow to compute the function to a good number of decimal digits (100 would be nice, for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method that can handle this function better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a plot of $f'(\pi x)$ with some artifacts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/klBIgm.png&quot; alt=&quot;Function&amp;#39;s derivative, $f'(\pi x)$&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2013-11-02T19:44:18.597" LastActivityDate="2014-12-20T19:13:44.403" Title="Computing slightly oscillatory series to high precision?" Tags="|convergence|extrapolation|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="8943" PostTypeId="2" ParentId="8942" CreationDate="2013-11-03T01:06:54.310" Score="0" Body="&lt;p&gt;How about the &lt;a href=&quot;https://math.stackexchange.com/questions/6625/levins-u-transformation&quot;&gt;Levin u-transform&lt;/a&gt;? In addition to Fortan codes, there are several versions in the &lt;a href=&quot;http://www.gnu.org/software/gsl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSL&lt;/a&gt;: &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/gsl-ref_30.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;`gsl_sum_levin_u*'&lt;/a&gt;. Matlab's &lt;a href=&quot;http://www.mathworks.com/help/symbolic/mupad_ref/numeric-sum.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MuPAD&lt;/a&gt; and &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=evalf/Sum&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maple&lt;/a&gt; use this scheme.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-11-03T01:06:54.310" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8944" PostTypeId="2" ParentId="5179" CreationDate="2013-11-03T03:31:10.197" Score="4" Body="&lt;p&gt;&lt;strong&gt;Short answer&lt;/strong&gt;&lt;br&gt;&#xA;Learn about the basics of object oriented code through python, and learn about the basics of computer science through C. When you're at least pretty okay with both of those other languages learn C++, because you can do pretty much anything in C++ and make it run quickly (though it takes forever to write).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Longer answer&lt;/strong&gt;&lt;br&gt;&#xA;So, here's the thing: for your first project you'll be in somebody's lab working on somebody else's code. In which case they'll be the one choosing the programming language. Which I personally think is kind of great! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, as a beginner you're not really going to know your ass from you elbow for a while, and, especially when you don't know what you're doing, learning to program can be borrring. Thus, it's good to have the structure and limits that come from working on somebody else's code, and it's good to have the motivation and excitement that can only come from working on a real project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, regardless of whatever language your lab uses (especially if it's Matlab), you should  probably learn python, C, and C++. In particular, if you're not coming from a computer science background you MUST read Kernighan and Ritchie's &quot;The C Programming Language&quot;. It's 35 years old and gives the distinct impression that its authors were programming on punch cards, but it is that rarest of birds: a timeless computer science book. It will make many things a great deal clearer.&lt;/p&gt;&#xA;" OwnerUserId="34" LastActivityDate="2013-11-03T03:31:10.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8945" PostTypeId="2" ParentId="8906" CreationDate="2013-11-03T05:52:19.320" Score="5" Body="&lt;p&gt;&lt;strong&gt;TLDR&lt;/strong&gt;&lt;br&gt;&#xA;Use Python to manage/modify your input and coral your output, and use HDF5 to organize/store your data. As complex as it might seem at first it'll still be simpler than SQL-anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Longer answer + Example&lt;/strong&gt;&lt;br&gt;&#xA;I personally use a combination of Python scripting and the HDF5 file format to deal with these kinds of situations. Python scripting can handle the text substitutions necessary for altering your runfiles (and can check for duplicate runs), and with some more scripting you can take the output data from your program and put it into an HDF5 file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easiest to think of HDF5 as being more or less exactly like a normal file system (ie the set of directories and subdirectories on your computer), but one that scales easily to large data sets. Each directory/subdirectory can be tagged with metadata (in your case either just the parameters that you're varying, or the entire set of parameters). When it comes time to analyze your data you can search through it based on the metadata.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a short example of how this would work based on some of my simulation data (already in HDF5 format) that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mydata.hdf5&#xA;|___Run01(metadata: {size:13, maxSteps:1e7, maxTime:inf})&#xA;|___Run02(metadata: {size:10, maxSteps:1e6, maxTime:inf})&#xA;|___Run03(metadata: {size:13, maxSteps:1e7, maxTime:inf})&#xA;|___Run04(metadata: {size:9, maxSteps:1e7, maxTime:inf})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;mydata.hdf5&lt;/code&gt; is the HDF5 file, and each of Runxx is a subdirectory that holds the output data from a given simulation, and which is tagged with the associated metadata. A python script that searches through the runs and return a list of those with the desired metadata would look like this:    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import h5py    #the python module that interfaces with HDF5&#xA;&#xA;def GetRuns(hdfRoot, attributeValuePairs):&#xA;    return [subdir for subdir in hdfRoot.values() if not(attributeValuePairs.viewitems() - dict(subdir.attrs).viewitems())]&#xA;&#xA;if __name__==&quot;__main__&quot;:&#xA;    attributeValuePairs = dict(zip(sys.argv[2::2], sys.argv[3::2]))&#xA;    with h5py.File(sys.argv[1]) as hdfRoot:&#xA;        runs = GetRuns(hdfRoot, attributeValuePairs)&#xA;&#xA;        #do something here with runs...&#xA;&#xA;        print runs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So if I was at a command line in a directory containing &lt;code&gt;mydata.hdf5&lt;/code&gt; I could run the above script like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python myscript.py mydata.hdf5 maxSteps 1e7 size 13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which would tell the script to find any runs with metadata partially or wholly matching &lt;code&gt;{'maxSteps':'1e7', 'size':'13'}&lt;/code&gt;. The script could then manipulate that data however you liked (in the &quot;do something here&quot; section), and then it would print a list that would look something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[&quot;Run01&quot;, &quot;Run03&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One note though is that HDF5 is going to present a totally natural mapping for your data only if it is possible to represent your data as a set of n-dimensional arrays. It's pretty common for the output of simulations to be in some kind of array, so this probably won't be an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Good starting points&lt;/strong&gt;&lt;br&gt;&#xA;Python: &lt;a href=&quot;http://www.openbookproject.net/thinkcs/python/english2e/&quot;&gt;http://www.openbookproject.net/thinkcs/python/english2e/&lt;/a&gt;&lt;br&gt;&#xA;HDF5: &lt;a href=&quot;http://www.h5py.org/docs/&quot;&gt;http://www.h5py.org/docs/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="34" LastActivityDate="2013-11-03T05:52:19.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8946" PostTypeId="2" ParentId="5372" CreationDate="2013-11-03T06:42:32.083" Score="6" Body="&lt;p&gt;GPUs are sensitive beasts. Although Nvidia's beefiest card can theoretically execute any of the operations you listed 100x faster than the fastest CPU, &lt;a href=&quot;https://stackoverflow.com/q/17223640/425458&quot;&gt;about&lt;/a&gt; a &lt;a href=&quot;https://stackoverflow.com/q/6500905/425458&quot;&gt;million&lt;/a&gt; &lt;a href=&quot;http://www.azavea.com/blogs/labs/2010/07/gpu-occupancy-and-idling/&quot; rel=&quot;nofollow noreferrer&quot;&gt;things&lt;/a&gt; can get in the way of that speedup. Every part of the relevant algorithm, and of the program which runs it, has to be extensively tweaked and optimized in order to get anywhere near that theoretical maximum speedup. R is generally not known to be a particularly fast language, and so it doesn't surprise me that its default GPU implementation is not that great, at least in terms of raw performance. However, the R GPU functions may have optimization settings that you can tweak in order to regain some of that missing performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're looking into GPUs because you've found that some calculation that you need to run is going to take weeks/months to finish, it may be worth your while to migrate from R to a more performance-friendly language. Python isn't too much harder to work with than R. The NumPy and SciPy packages have most of the same stat functions as R, and PyCuda can be used to implement your own GPU based functions in a fairly straightforward way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you &lt;em&gt;really&lt;/em&gt; want to increase the speed at which your functions run on GPUs, I would consider implementing your own functions in a combination of C++ and CUDA. The CUBLAS library can be used to handle all of the linear algebra-related heavy lifting. However, keep in mind that it can take quite a while to write such code (especially if it's your first time doing so), and so this approach should be reserved only for those computations that take an extremely long time to run (months) and/or that you're going to be repeating hundreds of times.&lt;/p&gt;&#xA;" OwnerUserId="34" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-11-03T06:42:32.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8947" PostTypeId="1" AcceptedAnswerId="8948" CreationDate="2013-11-03T10:25:16.007" Score="1" ViewCount="4394" Body="&lt;p&gt;I have a code similar to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import multiprocessing as mp&#xA;&#xA;a = np.zeros((4, 4))  # 4x4 array containing zeros&#xA;&#xA;def f(x, y):&#xA;    # uses scipy functions&#xA;    # takes long to compute&#xA;    #result = &amp;lt;someValue after calculation&amp;gt;&#xA;    global a&#xA;    a[x][y] = x+y  # simple example function&#xA;&#xA;# since f takes long to compute, I want to run it in parallel&#xA;jobs = []&#xA;for x in range(4):&#xA;    for y in range(4):&#xA;        p = mp.Process(target=f, args=(x, y))&#xA;        p.start()&#xA;        jobs.append(p)&#xA;&#xA;# wait for a to be filled for the next step&#xA;for j in jobs:&#xA;    j.join()&#xA;&#xA;print a  # prints an array of zeros! &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why are the values not assigned to a and how can I fix this? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="5392" LastActivityDate="2013-11-03T15:42:51.093" Title="Using multiprocessing to assign values to a numpy array" Tags="|python|numpy|scipy|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ClosedDate="2013-11-06T23:46:31.000" ContentLicense="CC BY-SA 3.0" />
  <row Id="8948" PostTypeId="2" ParentId="8947" CreationDate="2013-11-03T13:09:59.217" Score="2" Body="&lt;p&gt;Multiprocessing creates separate Python processes (i.e. UNIX or Windows process) for each &lt;code&gt;mp.Process&lt;/code&gt; that you asked it for. These &lt;em&gt;do not&lt;/em&gt; share memory. If you want them to operate on the same data, you must use one of the managed data types in the &lt;code&gt;mp.Manager&lt;/code&gt; class to explicitly communicate between separate tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FWIW, this is pretty slow since whatever object you put into the &lt;code&gt;mp.Manager&lt;/code&gt; containers is pickled before it's communicated.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-11-03T15:42:51.093" LastActivityDate="2013-11-03T15:42:51.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8949" PostTypeId="1" CreationDate="2013-11-03T16:39:16.943" Score="3" ViewCount="2657" Body="&lt;p&gt;I am having trouble with calculating a mean of vector with sufficient accuracy. My current solution which works but it quite slow and has unpredictable performance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;mean_sum = mean = math.fsum(values) / n&#xA;values = values - mean&#xA;mean = math.fsum(values) / n&#xA;while np.abs(mean) &gt; np.finfo(np.float32).eps:      &#xA;      values = values - mean&#xA;      mean_sum += mean&#xA;      mean = math.fsum(values) / n&#xA;return mean_sum, values&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a better way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: All the values are non-negative. The accuracy required is defined by &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.finfo(np.float32).eps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as the mean of the resulting series. I have edited the code a bit to make it more clear. &lt;/p&gt;&#xA;" OwnerUserId="5517" LastEditorUserId="4244" LastEditDate="2013-11-05T07:30:29.473" LastActivityDate="2013-11-05T07:30:29.473" Title="Caculating the mean of vector accurately" Tags="|python|stability|numpy|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8950" PostTypeId="2" ParentId="8949" CreationDate="2013-11-03T17:08:13.283" Score="0" Body="&lt;p&gt;I see that you are using Python, so you could use a high precision library like &lt;a href=&quot;http://mpmath.org&quot; rel=&quot;nofollow&quot;&gt;mpmath&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-11-03T17:08:13.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8951" PostTypeId="2" ParentId="8949" CreationDate="2013-11-03T17:58:50.543" Score="1" Body="&lt;p&gt;If you're concerned about floating point round-off, then have a loop over all elements, add the positive elements to one variable and the negative elements to another, and then add the two together to get the sum over all elements. Divide the result by the number of elements and you have the best mean value you can likely get.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-03T17:58:50.543" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8952" PostTypeId="2" ParentId="8949" CreationDate="2013-11-03T19:08:54.633" Score="6" Body="&lt;p&gt;How accurately? Is there a reason you're not just using &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.mean.html&quot;&gt;numpy.mean&lt;/a&gt;? Is that not sufficient? If you need to compensate for floating-point error, you can try using &lt;a href=&quot;http://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot;&gt;Kahan summation&lt;/a&gt;. Here's some &lt;a href=&quot;http://health.bsd.uchicago.edu/thisted/stat307/Handouts/01-Numerical_analysis_notes.pdf&quot;&gt;pseudocode&lt;/a&gt; (PDF).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a review of floating-point summation techniques see: &lt;a href=&quot;http://dx.doi.org/10.1137%2F0914050&quot;&gt;N.J. Higham, SIAM, 1993&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="4474" LastEditDate="2013-11-03T19:27:51.990" LastActivityDate="2013-11-03T19:27:51.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8953" PostTypeId="1" AcceptedAnswerId="8954" CreationDate="2013-11-03T20:35:20.537" Score="2" ViewCount="498" Body="&lt;p&gt;How does Matlab optimization tools works? It just gets the error function and doesn't need Jacobian (first derivatives) or Hessian (second derivatives)? How it is possible? If it is finite difference how it determines the value of dx for different degree of freedom?&lt;/p&gt;&#xA;" OwnerUserId="3584" LastActivityDate="2013-11-03T21:17:43.323" Title="How Matlab optimization works without Jacobian or Hessian" Tags="|optimization|matlab|jacobian|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8954" PostTypeId="2" ParentId="8953" CreationDate="2013-11-03T21:17:43.323" Score="2" Body="&lt;p&gt;The answer depends very much on what optimization function in MATLAB you're talking about.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The base MATLAB system includes a function fminsearch that uses the simplex method (not the simplex method for LP but rather the Nelder-Mead-Lagarias algorithm of the same name.)  This method does not make use of derivatives.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The functions of the optimization toolbox use a variety of different methods depending on the nature of the problem and its size.   There are different options for problems ranging from small sized to very large, and there are different options depending on the nature of the problem (e.g. constrained vs. unconstraints.)  For unconstrained nonlinear optimization of problems up to medium size, the toolbox uses the BFGS quasi-Newton method and can automatically compute finite difference approximations.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-11-03T21:17:43.323" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8955" PostTypeId="2" ParentId="8928" CreationDate="2013-11-03T22:17:53.930" Score="9" Body="&lt;p&gt;I think that the &quot;two language approach&quot; is sound and I feel very comfortable in using it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you start a new project from scratch you never know beforehand which will be the critical code sections of your implementation. It is important to be able to have a robust and easy to maintain prototypical code that can be validated and benchmarked. Once you have the prototype code working you can start optimising it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The prototyping step is usually better accomplished in expressive and easy to code languages (e.g. python/numpy), while the second one requires rewriting in a low-level language small sections of your code. A common experience is that once you have optimised your bottlenecks there is actually no need rewrite the entire code, since speed-ups would be marginal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A much quoted Knuth &lt;a href=&quot;http://dx.doi.org/10.1145/356635.356640&quot;&gt;passage&lt;/a&gt; says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Programmers waste enormous amounts of time thinking about, or worrying&#xA;  about, the speed of noncritical parts of their programs, and these&#xA;  attempts at efficiency actually have a strong negative impact when&#xA;  debugging and maintenance are considered. We should forget about small&#xA;  efficiencies, say about 97% of the time: premature optimization is the&#xA;  root of all evil. Yet we should not pass up our opportunities in that&#xA;  critical 3%.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So my point is that there is nothing wrong in writing that 3% in a different language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final comment, let me add that it is always desirable to keep the slow &quot;high level&quot; code as a reference to what the fast &quot;low level&quot; code is supposed to do.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2013-11-03T23:05:09.673" LastActivityDate="2013-11-03T23:05:09.673" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8956" PostTypeId="1" CreationDate="2013-11-03T22:53:56.637" Score="1" ViewCount="52" Body="&lt;p&gt;I have an energy dispersion curve obtained from the eigenvalues of &#xA;$$E(k) = \text{eig}(T e^{ik} + T^H e^{-ik} + H_0),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H_0$ and $T$ are $N\times N$ square matrices, $T^H$ is the Hermitian transpose $T$, and $k$ is the wave-vector. So, there are just 2 matrices $T$ and $H_0$ that determine the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to reduce the size of matrices $T$ and $H_0$ in a way that the approximation results in the same eigenvalues for a limited energy range (i.e. $E = [1:3]$). Is it possible to construct new matrices $T_1$ and $H_1$ such that $E_1(k) = \text{eig}(T_1e^{ik} + T_1^H e^{-ik} + H_1)$ be close to $E(k)$ for a limited energy range? How?&lt;/p&gt;&#xA;" OwnerUserId="3584" LastEditorUserId="9667" LastEditDate="2017-04-15T17:23:45.380" LastActivityDate="2017-04-15T17:23:45.380" Title="Size reduction of matrices in dispersion curve calculation" Tags="|eigensystem|eigenvalues|basis-set|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8957" PostTypeId="1" AcceptedAnswerId="8964" CreationDate="2013-11-04T16:21:16.467" Score="1" ViewCount="4396" Body="&lt;p&gt;if i draw a polar plot with Octave the axes are cartessian, not polar axes. But in Matlab it seems polar axes exists for polar plots. Is there anyway to draw with polar axes in polar plots in Octave?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="5522" LastActivityDate="2013-11-05T17:45:52.343" Title="Polar axes in Octave?" Tags="|matlab|octave|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="8958" PostTypeId="2" ParentId="8930" CreationDate="2013-11-05T01:20:53.057" Score="2" Body="&lt;p&gt;You can reduce the problem to computing the &lt;a href=&quot;http://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow&quot;&gt;singular value decomposition&lt;/a&gt;, for which there exist many fast methods and codes. For fast 3x3 SVD, I found &lt;a href=&quot;http://pages.cs.wisc.edu/~sifakis/papers/SVD_TR1690.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To reduce the polar decomposition to the SVD, suppose the polar decomposition is written in the following form, &#xA;$$M = U P,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with orthogonal $U$ and positive semidefinite $P$ (Ie., $Q \rightarrow U$, $R \rightarrow P$ from your notation). Further, denote the SVD of $P$ by&#xA;$$P = V \Sigma V^T.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting the second equation into the first yields, &#xA;$$M = U V \Sigma V^T.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if you compute the SVD of $M$,&#xA;$$M = W \Sigma V^T,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then $P$ is given by the above formula, and &#xA;$$U = W V^T.$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Incidentally, when you want to prove the existence of SVD-like decompositions for infinite dimensional operators, one basic strategy is to start with the polar decomposition which is easier to prove, and then do these steps backwards.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2013-11-06T19:47:05.877" LastActivityDate="2013-11-06T19:47:05.877" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8960" PostTypeId="1" CreationDate="2013-11-05T06:31:20.497" Score="2" ViewCount="374" Body="&lt;p&gt;I am taking a Thesis-based Master degree now and I am going to choose my supervisor soon. I plan to take a PHD degree after graduation, so if possible, I wish my PHD research area could be an extension of my Master Thesis ( so that the researching experience during my Master wouldn't be wasted).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am interested in the research areas of two professors. Professor A focuses on Numerical Solution on PDE. Professor B conducted research on Numerical PDE several years ago (He was the supervisor of Professor A's Master Thesis) but it seems that he has changed his research field to Inverse Problems in the last 5 years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I am not quite familiar with Inverse Problem, currently I am more interested in Numerical PDE. But since Professor B has richer researching experience, I think his recommendation may be more helpful for my PHD application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So may I ask what is the basic knowledge for research on Inverse Problem? Does it share the same research technique with researching on Numerical PDE? Is it possible for me to change from Numerical PDE to Inverse Problem without encountering many difficulties ( in PHD application and researching)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help. Any comment would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="5527" LastEditorUserId="4244" LastEditDate="2013-11-05T07:31:58.290" LastActivityDate="2013-11-05T14:09:53.910" Title="Research in Inverse Problem and Numerical PDE" Tags="|numerical-analysis|inverse-problem|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8961" PostTypeId="2" ParentId="8960" CreationDate="2013-11-05T12:48:35.783" Score="1" Body="&lt;p&gt;If you plan to continue your PhD with the same adviser at the same university, then it seems to me your plan to keep the topics related is a good one. But, since it sounds like you want to do your Masters degree and then find a new (better?) place with a new adviser, I'd suggest that you're likely to find it hard to keep the exact topic going.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't mean that your Masters degree research will have been wasted, though! If you plan carefully, the background you develop while working on your Masters degree will dovetail nicely with your PhD topic. One prepares you for the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You've given a good example in your question, in fact. Numerical solution of PDEs is a part of the numerical solution of inverse problems. You can't to do the latter without the former. You could do a Masters with the PDE adviser and then move on to a PhD in inverse problems if that was interesting to you.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-11-05T12:48:35.783" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8962" PostTypeId="2" ParentId="8960" CreationDate="2013-11-05T13:54:11.910" Score="0" Body="&lt;p&gt;Forward modeling alone is of little use. 'Most' problems, both in engineering and science are really inverse problems. No matter how cool you forward models are unless you can use it to optimize something or estimate some parameters of interest (that have scientific implications) it will be of little use in applied sciences. So yes PDE constrained optimization is a good area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However you also need to learn how to solve forward models (numerically I presume) and it does require some basic skills. So you can do you MS in numerical solution of PDE's (hopefully write a code) and then later pursue a PhD in inverse problems (and apply your code) to solve interesting problems and publish papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is natural for people who do forward modeling to move into inverse modeling later on and it might very well be that Prof A follows the steps taken by Prof B. In fact there are many such examples, specially in the HPC community. A few decades back computers weren't powerful enough to solve forward problems, let alone inverse problems but now solution of inverse problems is routine, both in academia and industry.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2013-11-05T14:09:53.910" LastActivityDate="2013-11-05T14:09:53.910" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8964" PostTypeId="2" ParentId="8957" CreationDate="2013-11-05T15:58:47.620" Score="1" Body="&lt;p&gt;Older versions of &lt;a href=&quot;http://www.gnu.org/software/octave/doc/interpreter/Introduction-to-Plotting.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Octave used gnuplot as the graphics engine&lt;/a&gt;, and you can still fall back on that capability.  A &lt;a href=&quot;https://stackoverflow.com/a/6774704/487781&quot;&gt;StackOverflow post from 2011&lt;/a&gt; illustrates gnuplot's capabilities, including some concentric circles around the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;http://octave.1599824.n4.nabble.com/polar-plots-td1623576.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2005 mailing list thread&lt;/a&gt; discusses Octave commands to &quot;set&quot; a polar grid (the phrase I'd rather use than &quot;polar axes&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll probably want to know how to get those circles with the high-level interface, so you won't have to rely on the gnuplot commands.  However as recently as last year, according to this &lt;a href=&quot;https://mailman.cae.wisc.edu/pipermail/octave-maintainers/2012-July/028938.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;mailing list thread&lt;/a&gt;, support for polar grids in the OpenGL library was still lacking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://octave.1599824.n4.nabble.com/Polar-axes-td1632809.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2008 Help-octave mailing list thread&lt;/a&gt; discusses the missing polar grid but mentions at the end an Octave toolkit, octave-epstk, using encapsulated postscript (EPS), that is supposed to provide the missing polar grid.  Unfortunately &lt;a href=&quot;http://en.wikipedia.org/wiki/EpsTk&quot; rel=&quot;nofollow noreferrer&quot;&gt;EpsTk&lt;/a&gt; seems to have lost their homepage in Germany, but the toolkit itself is still available in Debian and &lt;a href=&quot;https://launchpad.net/ubuntu/+source/octave-epstk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ubuntu&lt;/a&gt;, and likely in other such Linux distros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some &lt;a href=&quot;http://soc.if.usp.br/manual/octave-epstk/files/globalPara-txt.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation for global parameters in EpsTk&lt;/a&gt; indicates &lt;a href=&quot;http://soc.if.usp.br/manual/octave-epstk/files/globalPara-txt.html#Polar_grid&quot; rel=&quot;nofollow noreferrer&quot;&gt;there are several that affect the polar grid&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-11-05T15:58:47.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8967" PostTypeId="2" ParentId="8957" CreationDate="2013-11-05T17:45:52.343" Score="1" Body="&lt;p&gt;I put a simple code using epstk&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;theta = 0:0.01:2*pi;&#xA;rho = sin(2*theta).*cos(2*theta);&#xA;&#xA;eopen('polar.eps') %make a eps file&#xA;&#xA;epolar(theta,rho,'Polar Function',0,[0 0 1],1.5) &#xA;%plot in the eps file our function with polar coordinates&#xA;&#xA;eclose; %close eps file&#xA;eview % we show the eps file with gsview or a browser.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the result &#xA;&lt;img src=&quot;https://i.stack.imgur.com/V6OMM.png&quot; alt=&quot;A math function with polar coordinates using epstk for Octave&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="5522" LastActivityDate="2013-11-05T17:45:52.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8969" PostTypeId="1" AcceptedAnswerId="8972" CreationDate="2013-11-06T00:22:12.807" Score="3" ViewCount="2019" Body="&lt;p&gt;I am trying to solve a system of equations (&lt;strong&gt;A&lt;/strong&gt; x = b) for 3D heat diffusion (i.e. each equation has at most 7 terms not including the constant &quot;b&quot; term) using UMFPACK with boost numeric bindings to C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;namespace umf = boost::numeric::bindings::umfpack;&#xA;&#xA;umf::umf_solve(A,x,b);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can populate the &lt;strong&gt;A&lt;/strong&gt; matrix and the &quot;b&quot; vector for a 200K+ set of equations while using roughly 200MB of memory. Then while in the UMFPACK routines, the memory used by my program jumps up to 2.3GB+.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, at some point above ~211K equations, the routine passes back all zeros for &quot;x&quot;, without throwing an error. I assume this is related to some kind of memory limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the same on every machine I've tested on (my MacBook, my Ubuntu VM, and a node on a Red Hat supercomputer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why is UMFPACK returning zeros without an error?&lt;/li&gt;&#xA;&lt;li&gt;Why does UMFPACK require so much memory?&lt;/li&gt;&#xA;&lt;li&gt;What options do I have for solving the system for larger sets of equations?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3179" LastActivityDate="2013-11-06T13:58:07.130" Title="How Do I solve large systems given UMFPACK memory limitations?" Tags="|linear-algebra|c++|blas|memory-management|umfpack|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8970" PostTypeId="5" CreationDate="2013-11-06T00:24:43.273" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-06T00:24:43.273" LastActivityDate="2013-11-06T00:24:43.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8971" PostTypeId="4" CreationDate="2013-11-06T00:24:43.273" Score="0" Body="UMFPACK is a set of routines for solving unsymmetric sparse linear systems, Ax=b, using the Unsymmetric MultiFrontal method." OwnerUserId="3179" LastEditorUserId="3179" LastEditDate="2013-11-06T02:10:20.980" LastActivityDate="2013-11-06T02:10:20.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8972" PostTypeId="2" ParentId="8969" CreationDate="2013-11-06T00:40:17.270" Score="5" Body="&lt;p&gt;I can't give you a good answer for 1, but I can give you decent answers for 2 and 3.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm not terribly familiar with the Boost interface to UMFPACK. In the C interface, normally, you call UMFPACK routines that will allocate memory. If the memory cannot be allocated because there is not enough free memory available, UMFPACK will return a null pointer. You can then test for routines returning null pointers to handle errors; this approach is standard in C programming. Looking at the documentation, it seems like you'll have to check the return values of each function; it doesn't look like the library throws any exceptions. If you don't check return values, and only examine the solution returned by UMFPACK, that could explain the behavior you observe.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;UMFPACK is a sparse direct solver; it carries out a sparse LU factorization of your $A$ matrix. As $A$ becomes larger, more of the entries of the LU factorization become structurally nonzero -- they are &quot;filled in&quot;, and this phenomenon is called fill-in -- and structurally nonzero entries are represented explicitly in memory because they could be numerically nonzero. Fill-in is why you run out of memory; memory limitations tend to make sparse direct solvers unattractive for problems exceeding around 100K unknowns.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use an iterative method for solving sparse linear equations instead of a sparse direct method. These methods don't necessarily require you to explicitly store $A$; they typically only require a method for calculating the matrix-vector product $Ax$ for any $x$. For best results, you should supply a preconditioner for the iterative method. For a diffusion problem, multigrid is a good preconditioner; multigrid methods are also worth using without a preconditioner for diffusion equations. Other people can probably offer better advice on iterative methods to use for this type of problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-11-06T00:40:17.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8973" PostTypeId="2" ParentId="7646" CreationDate="2013-11-06T01:56:59.597" Score="1" Body="&lt;p&gt;Depends on whether you want free or are happy to pay.  Have used CHARMM in the past.  Here is a decent list of options (&lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_software_for_molecular_mechanics_modeling&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/List_of_software_for_molecular_mechanics_modeling&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="5534" LastActivityDate="2013-11-06T01:56:59.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8974" PostTypeId="2" ParentId="8969" CreationDate="2013-11-06T03:00:50.960" Score="3" Body="&lt;p&gt;Geoff mentioned the limitations of sparse direct solvers.  To solve a 3D problem of size $N = n^3$, an optimal direct solver will need $N^{4/3}$ memory and $N^2$ time.  See &lt;a href=&quot;https://scicomp.stackexchange.com/a/868/119&quot;&gt;this question&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it sounds like you have a structured grid diffusion problem.  If the diffusion coefficient is smooth (or constant), you can solve the problem using about four floating point values worth of memory and 50 flops per grid point, using the celebrated Full Multigrid algorithm (developed by Achi Brandt in the 1970s).  If the diffusion coefficient is very rough, it is easiest to use an algebraic multigrid method.  Both geometric and algebraic multigrid methods are available in &lt;a href=&quot;http://mcs.anl.gov/petsc&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc&lt;/a&gt; and other packages, depending on your needs. &lt;em&gt;[Disclaimer: I am a developer of PETSc.]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-11-06T13:58:07.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8975" PostTypeId="2" ParentId="8914" CreationDate="2013-11-06T12:49:09.820" Score="3" Body="&lt;p&gt;For a dense, direct solver, Golub and Van Loan (&lt;em&gt;Matrix Computations&lt;/em&gt;, 3rd ed) report the following cost estimates for eigenvalues only:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;standard eigenproblem: $10n^3$ (p. 359).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;generalized eigenproblem: $30n^3$ (p. 385).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Costs are in flops (additions and multiplications cost $1$ each).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want eigenvectors as well, you have to compute the orthogonal matrix in the Schur decomposition (or either $Q$ or $Z$ in QZ, respectively) as well and then you can obtain them using inverse iteration. Costs are not explicitly reported in GVL, but my guess is the following (for $m$ eigenvectors out of $n$, I am assuming one step of inverse iteration ($n^2$) on the triangular matrix is enough, and then multiplication by the orthogonal matrix ($2n^2$)):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;standard eigenproblem: $25n^3 + 3n^2m$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;generalized eigenproblem: $46n^3+3n^2m$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All these figures are approximate because they assume a &quot;typical average&quot; number of shifted QR/QZ steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's a sparse eigenproblem or it's larger than it fits in RAM, I doubt you can get realistical &lt;em&gt;a priori&lt;/em&gt; estimates (but I am a dense guy, so I might be wrong).&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2013-11-06T12:49:09.820" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8976" PostTypeId="2" ParentId="8925" CreationDate="2013-11-06T14:42:23.907" Score="3" Body="&lt;p&gt;Following others' suggestions, I have used (the commercial)  &lt;a href=&quot;http://www.gams.com&quot; rel=&quot;nofollow&quot;&gt;GAMS&lt;/a&gt;  for many projects. It is very straight forward; all you have to do is to put the mathematic formulation of your problem. It picks up the variables, constraints, objective functions and all the input data. Then, it provides a range of solvers (optimisers) for any case. Depending on your case, you add more sophisticated solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly, &lt;a href=&quot;http://velos0.ltt.mech.ntua.gr/research/easy.html&quot; rel=&quot;nofollow&quot;&gt;EASY&lt;/a&gt;  is worth a look. Open-source framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &quot;fast&quot; is very vague! You need to be more specific; Fast in terms of number of iterations? number of evaluations? elapsed time? combination of these?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you are not looking for a software, and you just want to solve the problem, I could suggest to use the global optimiser NSGA-II, which is an open-source optimiser of very high reputation and performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you provided more information, I could guide precisely.&lt;/p&gt;&#xA;" OwnerUserId="5538" LastActivityDate="2013-11-06T14:42:23.907" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8979" PostTypeId="2" ParentId="2051" CreationDate="2013-11-06T18:16:27.213" Score="0" Body="&lt;p&gt;As Shuhao Cao said, you definitely need functional analysis. A good one that you might want to try out is &quot;Functional Analysis: Applications to Mechanics and Inverse Problems&quot; by Lebedev, Vorovich and Gladwell. Whatever your background in analysis is, this book should be fine, as it starts with the basics.&lt;/p&gt;&#xA;" OwnerUserId="5541" LastActivityDate="2013-11-06T18:16:27.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8980" PostTypeId="1" AcceptedAnswerId="8983" CreationDate="2013-11-06T18:39:06.813" Score="5" ViewCount="1049" Body="&lt;p&gt;Software such as ParMetis or PTScotch partition a graph. When one wants to use it for mesh partitioning (for example for FEM), a dual graph whose vertices represent cells of the original mesh is usually constructed first. This dual graph is then partitioned. I was wondering why one cannot simply interpret the connectivity of the input mesh as a graph directly - mesh nodes are graph vertices and mesh edges are also graph edges. This would assume that the list of mesh edges is built by client application before the call to the partitioning routine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly, the cut through a dual graph will contain whole faces of adjacent mesh cells belonging to different partitions, while cut through the mesh (seen as a graph) will be less pretty in the sense that only some nodes of an element face might become ghost nodes. In addition, the number of mesh edges (especially in 3D) will probably be much higher than the number of cells. Apart from that, I don't see what are the advantages of using the first or the second approach. Can anyone comment on this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of the fact that (Par)Metis has a function to construct a dual graph. My question is 'why', not 'how'. I'm mainly interested in mesh partitioning for FEM. &lt;/p&gt;&#xA;" OwnerUserId="4688" LastActivityDate="2013-11-06T20:22:54.080" Title="Why dual graph for mesh partitioning" Tags="|mesh|partitioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8981" PostTypeId="1" CreationDate="2013-11-06T19:07:53.140" Score="6" ViewCount="145" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Let $f(x)=\frac{1}{5}x^5+\frac{1}{3}x^3+x-1$&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Show that $f$ has only one zero $r$ in interval $(0,1)$ &lt;/li&gt;&#xA;  &lt;li&gt;To find approximation of $r$ we apply Newton's method $x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}$. Show that for every $x_0\in(0,1)$ this method converges.&lt;/li&gt;&#xA;  &lt;li&gt;Show that order of convergence is exactly quadratic&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;First one is trivial, but I don't know how to solve second and third. For second I was trying to show that error $|x_{n+1}-r|=e_{n+1}=e_n-\frac{f(x_n)}{f'(x_n)}$ is approaching $0$ when $n$ approaches $+\infty$ but it didn't lead me to anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also showing the third seems very complicated. Can anybody help?&lt;/p&gt;&#xA;" OwnerUserId="5540" LastEditorUserId="34" LastEditDate="2013-11-06T23:58:33.560" LastActivityDate="2013-11-09T15:39:20.750" Title="Newton's method for a given polynomial" Tags="|iterative-method|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8982" PostTypeId="2" ParentId="8981" CreationDate="2013-11-06T19:40:45.147" Score="2" Body="&lt;p&gt;You're on the right track for the second and third parts. You might try looking at a Taylor series expansion of $f$ around the root in $(0,1)$, where the remainder term is second order. If you can state conditions required for quadratic convergence, that should also give you insight into some of the conditions required for convergence in general.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-11-06T19:40:45.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8983" PostTypeId="2" ParentId="8980" CreationDate="2013-11-06T20:22:54.080" Score="3" Body="&lt;p&gt;The most common way to write finite element software is to make a non-overlapping partition of elements with interface vertex ownership resolved using some rule (or via hypergraph partitioning, which is more expensive).  To create a globally-assembled stiffness matrix, this involves communication of entries to the process that owns the vertex.  Residual evaluation and matrix assembly does work based on elements, but the solve (using sparse iterative methods) does work based on the vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative is to create a non-overlapping vertex partition and create a ghost layer of elements.  Those interface elements will be integrated redundantly (from each side) but there is no communication to assemble the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This tends to scale somewhat better for low-order elements.  High-order elements have chunky interfaces and are generally more efficient to integrate using a non-overlapping element partition.  Pick your method based on how you manage your mesh and the cost tradeoffs for the discretization you choose.  &lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-11-06T20:22:54.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="8984" PostTypeId="2" ParentId="8981" CreationDate="2013-11-07T03:54:16.423" Score="4" Body="&lt;p&gt;Because $f$ is a polynomial and $f'(x)\ge 1$ for all $x$, it has a unique simple real root $r$. So it can be factored $f(x) = (x-r)g(x)$, where $g(x)$ is a polynomial satisfying $g(x)&amp;gt;0$ for all $x$. Then if $x_n = r + e_n$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_n) = e_ng(x_n)\\ &#xA;f'(x_n)=g(x_n)+e_n g'(x_n)$$&#xA;Plugging this in to Newton's method:&#xA;$$&#xA;r+e_{n+1}=r+e_n-\frac{e_n g(x_n)}{g(x_n)+e_n g'(x_n)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;e_{n+1} = e_n^2 \frac{g'(x_n)}{f'(x_n)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't worked out the details, but you should be able to prove quadratic convergence from this by bounding $|g'(x_n)/f'(x_n)|$.&lt;/p&gt;&#xA;" OwnerUserId="767" LastActivityDate="2013-11-07T03:54:16.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8985" PostTypeId="2" ParentId="8981" CreationDate="2013-11-07T04:16:46.067" Score="7" Body="&lt;p&gt;Newton's Method can be interpreted as a type of fixed point iteration of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_{n+1}=F(x_n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(x)=x-\frac{f(x)}{f'(x)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we know the root $r$ such that $F(r)=r$ (i.e. $r$ the answer to part 1 of your question).  According to fixed point theory, if $F$ is a contractive mapping on a closed set $[a,b]$ and $F'$ exists and is continuous, then the error $e_{n+1}$ can be expressed (with the help of mean value theorem) as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;e_{n+1}&amp;amp;=&amp;amp; x_{n+1}-r\\&#xA;&amp;amp;=&amp;amp; F(x_n)-F(r) \\&#xA;&amp;amp;=&amp;amp; F'(\xi_n)(x_n-r)&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some $\xi_n$ between $x_n$ and $r$.  Thus, to ensure convergence by contractive mapping (your question 2), we require that $|F'(x)|&amp;lt;1$ for all $x$ in some $[a,b]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To prove 2nd order convergence, notice that the error term $e_{n+1}$ can also be expressed (with the help of Taylor Series) as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;e_{n+1}&amp;amp;=&amp;amp;x_{n+1}-r\\&#xA;&amp;amp;=&amp;amp; F(r+e_n)-F(r)\\&#xA;&amp;amp;=&amp;amp;F(r) + e_nF'(r) + \frac{1}{2!} e_n^2F''(r) + \frac{1}{3!}F'''(r)+... - F(r) \\&#xA;&amp;amp;=&amp;amp;e_nF'(r) + \frac{1}{2!} e_n^2F''(r) + \frac{1}{3!}F'''(r)+...&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last expression tells us that &lt;strong&gt;&lt;em&gt;the order of convergence&lt;/em&gt;&lt;/strong&gt; depends on the multiplicity of the root $r$ of $F$.  If $r$ has multiplicity $q$, then all the derivatives of $F$ up to the $q-1$th order (at the root $r$) are all zero and the $q$th derivative $F^{(q)}(r)\neq0$.  Hence, a root $r$ of multiplicity $q$ yields a convergence of order $q$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your particular case, you need to show that the root $r$ of $F$ (note the CAPITAL $F$, not lower case $f$) has multiplicity $2$.  That is, you'll need to show that $F'(r)=0$ and $F''(0)\neq0$&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-11-09T15:39:20.750" LastActivityDate="2013-11-09T15:39:20.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8987" PostTypeId="1" AcceptedAnswerId="8997" CreationDate="2013-11-07T12:14:18.180" Score="5" ViewCount="3925" Body="&lt;p&gt;I want to integrate a function over a radial domain $D=\{r&amp;lt;r(\theta)\}$. The change to polar coordinates yields:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_D f = \int_0^{2\pi} \int_0^{r(\theta)}f(r,\theta)rdr d\theta $$&#xA;so I tried calculating each integral using trapezoidal quadrature:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_a^b f(x)dx \approx \frac{b-a}{m} (f(x_1)+...+f(x_m)) \text{ for } x_i=a+(i-1)\frac{b-a}{m-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I guess this is not the right way to do it, since the density of evaluation points is too great at the origin, and I get a pretty large error for $100\times 100$ discretization grid (approx 0.1) for $f(r,\theta)=r$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other way can I use which converges faster?&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="651" LastEditDate="2013-11-08T15:49:51.573" LastActivityDate="2013-11-09T12:36:04.643" Title="How to integrate numerically over a radial domain" Tags="|quadrature|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8989" PostTypeId="1" CreationDate="2013-11-07T19:00:11.743" Score="16" ViewCount="18606" Body="&lt;p&gt;Suppose I have two VTK files, both in structured grid format. The structured grids are the same (they have the same list of points, in the same order), and there is a field, call it &quot;Phi&quot;, in each VTK file. I want to create a third VTK file, again with the same structured grid, and plot a field that is the difference between Phi in the first VTK file and Phi in the second VTK file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to do this manually; I can parse the raw text in the two VTK files, copy the data into arrays, subtract one array from the other, and then dump the data in the right format into a new file. Is there a better way to calculate this difference and export it to VTK? A solution in Python, or in visualization software like VisIt or Paraview would be preferable to using a compiled language like C++. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of calculating this difference is to compare different numerical methods for calculating the solution of a PDE; since I'm using the same software to generate the solutions, I can guarantee that all data but the field Phi will be the same in each file I generate.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="4964" LastEditDate="2014-09-05T15:09:18.120" LastActivityDate="2014-09-05T15:09:18.120" Title="How do I calculate the numerical difference between two fields stored in two different VTK files with the same structure?" Tags="|visualization|vtk|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="8990" PostTypeId="2" ParentId="8989" CreationDate="2013-11-07T19:00:11.743" Score="17" Body="&lt;p&gt;The easiest way I could find to subtract two fields from different VTK files with the same structured grid is to use a programmable filter in Paraview, which lets you manipulate data using Python scripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the programmable filter dialogue box, you can subtract the two arrays and write to output with the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   phi_0 = inputs[0].CellData['Phi']&#xA;   phi_1 = inputs[1].CellData['Phi']&#xA;   output.CellData.append(phi_1 - phi_0, 'difference')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this instance, the field Phi happens to be cell data. If your field is point data, replace &lt;code&gt;CellData&lt;/code&gt; everywhere in the script with &lt;code&gt;PointData&lt;/code&gt;. See &lt;a href=&quot;http://public.kitware.com/pipermail/paraview/2010-April/016667.html&quot;&gt;http://public.kitware.com/pipermail/paraview/2010-April/016667.html&lt;/a&gt; for further details.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-11-07T19:00:11.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8991" PostTypeId="1" AcceptedAnswerId="8992" CreationDate="2013-11-07T19:02:54.540" Score="2" ViewCount="700" Body="&lt;p&gt;I'm reading up on the Finite Element Method (&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0750663200&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zienkiewicz's Book&lt;/a&gt;), so I understand better what I'm doing in FEniCS and COMSOL. Currently, I'm wondering about this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Using FEM to solve fluid flow problems, do I have to re-assemble the entire system in each iteration of the solution process? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would assume not since re-assembly only needs to be done if the coordinates of the nodes of the elements change. I can see how this is important in structural analysis, however, this is not the case in fluid dynamics (unless you work with moving meshes). So I assume in a common CFD computation it should be safe to only assemble the system of equations once and carry that through the entire solution process?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking because, if I understand correctly, &lt;strong&gt;not&lt;/strong&gt; reassembling the system in each iteration of the solution should (&lt;strong&gt;in my current understanding&lt;/strong&gt;) significantly reduce the computation time.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="20688" LastEditDate="2019-06-22T20:44:04.357" LastActivityDate="2019-06-22T20:44:04.357" Title="Basic Finite Element Method (FEM) question: assembly and re-assembly" Tags="|finite-element|fluid-dynamics|fenics|comsol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8992" PostTypeId="2" ParentId="8991" CreationDate="2013-11-07T19:35:24.427" Score="4" Body="&lt;p&gt;You might have to reassemble if your problem is non-linear and your method at a future step incorporates the solution in the formation of the matrix. If you are doing Picard iteration rather than Netwon-Raphson, then you should only have to reform the right-hand-side vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know enough about FEciCS and COMSOL to say what they do, but I suspect, for good convergence rates, you're going to have to reform matrices every Newton step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Jed's absolutely right on Picard. I should have opened a book or written it out myself before I answered. Though I would say you can always lag the preconditioner, but it may be of dubious quality depending on how strong the convective terms are. &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-11-08T12:38:11.930" LastActivityDate="2013-11-08T12:38:11.930" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8993" PostTypeId="2" ParentId="8989" CreationDate="2013-11-08T00:48:44.723" Score="1" Body="&lt;p&gt;I don't have a particularly good approach, but I'd copy the 'phi' field from one VTK file into the other and name it 'phiprime' or something. In both Paraview and Visit, you have the option of defining new fields by a formula that uses the values of other fields. You can then define a field &quot;error&quot; as &quot;error=phi-phiprime&quot; in the field editor, and plot this field &quot;error&quot; as either a surface, a contour plot, or whatever you're interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The step of copying the block of data from one file to the other is clearly awkward, but it's the best I can come up with.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-08T00:48:44.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8995" PostTypeId="1" AcceptedAnswerId="8996" CreationDate="2013-11-08T12:32:51.200" Score="8" ViewCount="1525" Body="&lt;p&gt;My current project is a reprogramming of a protein folding model involving the solution of thousands of ODEs in C++. I've been making some stop and start progress as I'm writing the solver to run totally on the GPU.  I finally got it integrating but when I try to solve dC/dt=-C using the fifth order solution from the RKF45 algorithm with a fixed step size of h=.001, I get decision from the calculator value of e^{-t} on the order of 10^{-4} when is expect global error on the order of 10^{-12}.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it because I'm not using adaptive step sizes and error control using both the fourth and fifth order solutions that this is occurring? My thought was that since an adaptive algorithm looks at the difference between the fourth and fifth order guesses and basically treats the fifth order as the &quot;correct answer&quot;that I'd just use that answer to try out the integrator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT :&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought that since &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Global Error = (Number of Points) * (Local Error) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Local Error = h^O&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where h is the step size and O is the order, I calculate &lt;/p&gt;&#xA;&#xA;&lt;p&gt;O = ln(Global Error/Number of Points)/ln h&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing an order analysis both the way I thought correct and the way suggested in the answer, I get the following : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Number of Points    h       Global Error    My Way      The Answer's Way&#xA;10                  0.1     2.89E-06        6.539E+00   &#xA;20                  0.05    7.09E-08        6.495E+00   5.350E+00&#xA;40                  0.025   1.76E-07        5.216E+00   -1.315E+00&#xA;50                  0.02    1.128E-07       5.089E+00   2.003E+00&#xA;80                  0.0125  4.401E-08       4.866E+00   2.002E+00&#xA;100                 0.01    2.816E-08       4.775E+00   2.001E+00&#xA;500                 0.002   1.126E-09       4.316E+00   2.000E+00&#xA;1000                0.001   2.810E-10       4.184E+00   2.003E+00&#xA;10000               0.0001  3.000E-12       3.881E+00   1.972E+00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, despite my quasi breakthrough last night, where I did indeed have a programming error, the order of my solver is still off.  The aforementioned error associated with the .001 stepsize has decreased dramatically, however, from O(10^-4) to O(10^-10).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2nd EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a graph of LN(E)-LN(a) vs. LN(h). I tossed on a linear fit so I'd expect that to reveal the order to me.  What's really confusing now is that I have three calculations for the order of my method.  The graph is telling me ~3, Godric's method tells me almost exactly 2 and my method shows it changing from 6-&gt;4 for increasing step size.  I am more inclined to believe my calculations because the Global Error is so low (~10^-10) and closest to what I'd expect (10^-15*10^3 = 10^-12) for that stepsize (10^-3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/hMs6R.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 3:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Code. Pardon the noobishness.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;cuda.h&amp;gt;&#xA;#include &amp;lt;cuda_runtime.h&amp;gt;&#xA;#include &amp;lt;device_launch_parameters.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;iomanip&amp;gt;                      //display 2 decimal places&#xA;#include &amp;lt;math.h&amp;gt;&#xA;using namespace std;&#xA;&#xA;/*&#xA;Things to do log:&#xA;1. Decide how adaptive step sizes is handled by the integrator loop.  We can no longer say how many points to integrate if that is to be determined in the kernel&#xA;itself, only what the initial step size is.&#xA;2. Figure out how I'm going to graph all this stuff.&#xA;3. On paper solve for the step size.&#xA;4. Adaptive Step Size&#xA;    a. arrMin kernel to get min of array&#xA;    b. Decide how to implement tolerances&#xA;5. Use multiples of 256 for maxlength &#xA;&#xA;Important things accomplished:&#xA;1. Can create 2D array for storing concentrations over time.&#xA;2. Can store a 1D array in 2D array.&#xA;3. Can calculate all the k-arrays given static step size.&#xA;4. Can take a step given static step size.&#xA;5. Can step through an integration from t0 to tf&#xA;6. Can accomodate fixed time-step&#xA;7. Calculated the current order of integrator using logs.&#xA;8. Learned how to profile the program.&#xA;9. Fixed the GPU timeout problem with WDDM.&#xA;&#xA;Ideas:&#xA;1. Just use large fixed size array&#xA;*/&#xA;__global__ void rkf5(double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, int*, int*, size_t, double*, double*, double*);&#xA;__global__ void calcK(double*, double*, double*);&#xA;__global__ void k1(double*, double*, double*);&#xA;__global__ void k2(double*, double*, double*);&#xA;__global__ void k3(double*, double*, double*);&#xA;__global__ void k4(double*, double*, double*);&#xA;__global__ void k5(double*, double*, double*);&#xA;__global__ void k6(double*, double*, double*);&#xA;__global__ void arrAdd(double*, double*, double*);&#xA;__global__ void arrSub(double*, double*, double*);&#xA;__global__ void arrMult(double*, double*, double*);&#xA;__global__ void arrInit(double*, double);&#xA;__global__ void arrCopy(double*, double*);&#xA;__device__ void setup(double , double*, double*, double*, double*, int*);&#xA;__device__ double flux(int, double*) ;&#xA;__device__ double knowles_flux(int, double*);&#xA;__device__ void calcStepSize(double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, int*);&#xA;__global__ void storeConcs(double*, size_t, double*, int);&#xA;__global__ void takeFourthOrderStep(double*, double*, double*, double*, double*, double*, double*);&#xA;__global__ void takeFifthOrderStep(double*, double*, double*, double*, double*, double*, double*, double*);&#xA;&#xA;//Error checking that I don't understand yet.&#xA;#define gpuErrchk(ans) { gpuAssert((ans), __FILE__, __LINE__); }&#xA;inline void gpuAssert(cudaError_t code, char *file, int line, bool abort=true)&#xA;{&#xA;   if (code != cudaSuccess) &#xA;   {&#xA;      fprintf(stderr,&quot;GPUassert: %s %s %d\n&quot;, cudaGetErrorString(code), file, line);&#xA;      if (abort) exit(code);&#xA;   }&#xA;}&#xA;&#xA;//Main program.&#xA;int main(int argc, char** argv)&#xA;{&#xA;    //std::cout &amp;lt;&amp;lt; std::fixed;          //display 2 decimal places&#xA;    //std::cout &amp;lt;&amp;lt; std::setprecision(12);   //display 2 decimal places&#xA;    const int maxlength = 1;            //Number of discrete concentrations we are tracking.&#xA;    double concs[maxlength];            //Meant to store the current concentrations &#xA;    double temp1[maxlength];                //Used as a bin to store products of Butcher's tableau and k values.&#xA;    double temp2[maxlength];                //Used as a bin to store products of Butcher's tableau and k values.&#xA;    double tempsum[maxlength];          //Used as a bin to store cumulative sum of tableau and k values&#xA;    double k1s[maxlength];&#xA;    double k2s[maxlength];&#xA;    double k3s[maxlength];&#xA;    double k4s[maxlength];&#xA;    double k5s[maxlength];&#xA;    double k6s[maxlength];&#xA;    const int numpoints = 40;       &#xA;    double to = 0;&#xA;    double tf = 1;&#xA;    //double dt = static_cast&amp;lt;double&amp;gt;(.5)/static_cast&amp;lt;double&amp;gt;(64);&#xA;    double dt = (tf-to)/static_cast&amp;lt;double&amp;gt;(numpoints);&#xA;    double mo = 1;&#xA;    double concStorage[maxlength][numpoints];   //Stores concs vs. time                     &#xA;&#xA;    //Initialize all the arrays on the host to ensure arrays of 0's are sent to the device.&#xA;    //Also, here is where we can seed the system.&#xA;    std::cout&amp;lt;&amp;lt;dt;&#xA;    std::cout&amp;lt;&amp;lt;&quot;\n&quot;;&#xA;    concs[0]=mo;&#xA;    std::cout&amp;lt;&amp;lt;concs[0];&#xA;    std::cout&amp;lt;&amp;lt;&quot; &quot;;&#xA;    for (int i=0; i&amp;lt;maxlength; i++)&#xA;    {&#xA;        for (int j=0; j&amp;lt;numpoints; j++)&#xA;            concStorage[i][j]=0;&#xA;        concs[i]=0;&#xA;        temp1[i]=0;&#xA;        temp2[i]=0;&#xA;        tempsum[i]=0;&#xA;        k1s[i]=0;&#xA;        k2s[i]=0;&#xA;        k3s[i]=0;&#xA;        k4s[i]=0;&#xA;        k5s[i]=0;&#xA;        k6s[i]=0;&#xA;        std::cout&amp;lt;&amp;lt;concs[i];&#xA;        std::cout&amp;lt;&amp;lt;&quot; &quot;;&#xA;    }&#xA;    concs[0]=mo;&#xA;    std::cout&amp;lt;&amp;lt;&quot;\n&quot;;&#xA;&#xA;    //Define all the pointers to device array memory addresses. These contain the on-GPU&#xA;    //addresses of all the data we're generating/using.&#xA;    double *d_concs;&#xA;    double *d_temp1;&#xA;    double *d_temp2;&#xA;    double *d_tempsum;&#xA;    double *d_k1s;&#xA;    double *d_k2s;&#xA;    double *d_k3s;&#xA;    double *d_k4s;&#xA;    double *d_k5s;&#xA;    double *d_k6s;&#xA;    double *d_dt;&#xA;    int *d_maxlength;&#xA;    int *d_numpoints;&#xA;    double *d_to;&#xA;    double *d_tf;&#xA;    double *d_concStorage;&#xA;&#xA;    //Calculate all the sizes of the arrays in order to allocate the proper amount of memory on the GPU.&#xA;    size_t size_concs = sizeof(concs);&#xA;    size_t size_temp1 = sizeof(temp1);&#xA;    size_t size_temp2 = sizeof(temp2);&#xA;    size_t size_tempsum = sizeof(tempsum);&#xA;    size_t size_ks = sizeof(k1s);&#xA;    size_t size_maxlength = sizeof(maxlength);&#xA;    size_t size_numpoints = sizeof(numpoints);&#xA;    size_t size_dt = sizeof(dt);&#xA;    size_t size_to = sizeof(to);&#xA;    size_t size_tf = sizeof(tf);&#xA;    size_t h_pitch = numpoints*sizeof(double);&#xA;    size_t d_pitch;&#xA;&#xA;    //Calculate the &quot;pitch&quot; of the 2D array.  The pitch is basically the length of a 2D array's row.  IT's larger &#xA;    //than the actual row full of data due to hadware issues.  We thusly will use the pitch instead of the data &#xA;    //size to traverse the array.&#xA;    gpuErrchk(cudaMallocPitch( (void**)&amp;amp;d_concStorage, &amp;amp;d_pitch, numpoints * sizeof(double), maxlength)); &#xA;&#xA;    //Allocate memory on the GPU for all the arrrays we're going to use in the integrator.&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_concs, size_concs));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_temp1, size_temp1));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_temp2, size_temp1));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_tempsum, size_tempsum));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_k1s, size_ks));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_k2s, size_ks));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_k3s, size_ks));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_k4s, size_ks));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_k5s, size_ks));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_k6s, size_ks));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_maxlength, size_maxlength));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_numpoints, size_numpoints));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_dt, size_dt));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_to, size_to));&#xA;    gpuErrchk(cudaMalloc((void**)&amp;amp;d_tf, size_tf));&#xA;&#xA;    //Copy all initial values of arrays to GPU.&#xA;    gpuErrchk(cudaMemcpy2D(d_concStorage, d_pitch, concStorage, h_pitch, numpoints*sizeof(double), maxlength, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_concs, &amp;amp;concs, size_concs, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_temp1, &amp;amp;temp1, size_temp1, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_temp2, &amp;amp;temp2, size_temp2, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_tempsum, &amp;amp;tempsum, size_tempsum, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_k1s, &amp;amp;k1s, size_ks, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_k2s, &amp;amp;k2s, size_ks, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_k3s, &amp;amp;k3s, size_ks, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_k4s, &amp;amp;k4s, size_ks, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_k5s, &amp;amp;k5s, size_ks, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_k6s, &amp;amp;k6s, size_ks, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_maxlength, &amp;amp;maxlength, size_maxlength, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_numpoints, &amp;amp;numpoints, size_numpoints, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_dt, &amp;amp;dt, size_dt, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_to, &amp;amp;to, size_to, cudaMemcpyHostToDevice));&#xA;    gpuErrchk(cudaMemcpy(d_tf, &amp;amp;tf, size_tf, cudaMemcpyHostToDevice));&#xA;&#xA;    //Run the integrator.&#xA;    rkf5&amp;lt;&amp;lt;&amp;lt;1,1&amp;gt;&amp;gt;&amp;gt;(d_concs, d_concStorage, d_temp1, d_temp2, d_tempsum, d_k1s, d_k2s, d_k3s, d_k4s, d_k5s, d_k6s, d_maxlength, d_numpoints, d_pitch, d_dt, d_to, d_tf);&#xA;    gpuErrchk( cudaPeekAtLastError() );&#xA;    gpuErrchk( cudaDeviceSynchronize() );&#xA;    cudaDeviceSynchronize();&#xA;&#xA;    //Copy concentrations from GPU to Host.  Almost defunct now that transferring the 2D array works.&#xA;    cudaMemcpy(concs, d_concs, size_concs, cudaMemcpyDeviceToHost);&#xA;    //Copy 2D array of concentrations vs. time from GPU to Host.&#xA;    gpuErrchk( cudaMemcpy2D(concStorage, h_pitch, d_concStorage, d_pitch, numpoints*sizeof(double), maxlength, cudaMemcpyDeviceToHost) );   &#xA;&#xA;    //Print concentrations after the integrator kernel runs.  Used to test that data was transferring to and from GPU correctly.&#xA;    std::cout &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;    for (int i=0; i&amp;lt;maxlength; i++)&#xA;    {&#xA;        std::cout&amp;lt;&amp;lt;concs[i];&#xA;        std::cout&amp;lt;&amp;lt;&quot; &quot;;&#xA;    }&#xA;&#xA;    double a[10];&#xA;    double b[10];&#xA;    double c[10];&#xA;    for(int i = 0; i&amp;lt; 10; i++)&#xA;    {&#xA;        a[i]=0;&#xA;        b[i]=0;&#xA;        c[i]=0;&#xA;    }&#xA;&#xA;&#xA;&#xA;    //Print out the concStorage array after the kernel runs.  Used to test that the 2D array transferred correctly from host to GPU and back.&#xA;    std::cout &amp;lt;&amp;lt; &quot;\n\n&quot;;&#xA;    std::cout &amp;lt;&amp;lt; &quot;Calculated Array&quot;;&#xA;    std::cout &amp;lt;&amp;lt; &quot;\n\n&quot;;&#xA;    for (int i=0; i&amp;lt;maxlength; i++)&#xA;    {&#xA;        for(int j=0; j&amp;lt;numpoints; j++)&#xA;        {&#xA;            if (j%(numpoints/10)==0)&#xA;            {&#xA;                a[j/(numpoints/10)]=concStorage[i][j];&#xA;                std::cout&amp;lt;&amp;lt;concStorage[i][j];&#xA;                std::cout&amp;lt;&amp;lt;&quot;   &quot;;&#xA;            }&#xA;        }&#xA;        std::cout &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;    std::cout &amp;lt;&amp;lt; &quot;Exponential&quot;;&#xA;    std::cout &amp;lt;&amp;lt; &quot;\n\n&quot;;&#xA;    for (int i=0; i&amp;lt;10; i++)&#xA;    {&#xA;        b[i]=exp(-i*(tf-to)/10);&#xA;        std::cout&amp;lt;&amp;lt;exp(-i*(tf-to)/10);&#xA;        std::cout&amp;lt;&amp;lt;&quot;   &quot;;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; &quot;\n\n&quot;;&#xA;    std::cout &amp;lt;&amp;lt; &quot;Error Array&quot;;&#xA;    std::cout &amp;lt;&amp;lt; &quot;\n\n&quot;;&#xA;    for (int i=0; i&amp;lt;10; i++)&#xA;    {&#xA;        c[i]=a[i]-b[i];&#xA;        std::cout&amp;lt;&amp;lt;c[i];&#xA;        std::cout&amp;lt;&amp;lt;&quot;   &quot;;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; &quot;\n\n&quot;;&#xA;    cudaDeviceReset();  //Clean up all memory.&#xA;&#xA;    return 0;&#xA;}&#xA;//Main kernel.  This is mean to be run as a master thread that calls all the other functions and thusly &quot;runs&quot; the integrator.&#xA;__global__ void rkf5(double* concs, double* concStorage, double* temp1, double* temp2, double* tempsum, double* k1s, double* k2s, double* k3s, double* k4s, double* k5s, double* k6s, int* maxlength, int* numpoints, size_t pitch, double* dt, double* to, double* tf)&#xA;{&#xA;    /*&#xA;    axy variables represent the coefficients in the Butcher's tableau where x represents the order of the step and the y value corresponds to the ky value &#xA;    the coefficient gets multiplied by.  Have to cast them all as doubles, or the ratios evaluate as integers.&#xA;    e.g. a21 -&amp;gt; a21 * k1&#xA;    e.g. a31 -&amp;gt; a31 * k1 + a32 * k2&#xA;    */&#xA;    double a21 = static_cast&amp;lt;double&amp;gt;(.25);&#xA;&#xA;    double a31 = static_cast&amp;lt;double&amp;gt;(3)/static_cast&amp;lt;double&amp;gt;(32);&#xA;    double a32 = static_cast&amp;lt;double&amp;gt;(9)/static_cast&amp;lt;double&amp;gt;(32);&#xA;&#xA;    double a41 = static_cast&amp;lt;double&amp;gt;(1932)/static_cast&amp;lt;double&amp;gt;(2197);&#xA;    double a42 = static_cast&amp;lt;double&amp;gt;(-7200)/static_cast&amp;lt;double&amp;gt;(2197);&#xA;    double a43 = static_cast&amp;lt;double&amp;gt;(7296)/static_cast&amp;lt;double&amp;gt;(2197);&#xA;&#xA;    double a51 = static_cast&amp;lt;double&amp;gt;(439)/static_cast&amp;lt;double&amp;gt;(216);&#xA;    double a52 = static_cast&amp;lt;double&amp;gt;(-8);&#xA;    double a53 = static_cast&amp;lt;double&amp;gt;(3680)/static_cast&amp;lt;double&amp;gt;(513);&#xA;    double a54 = static_cast&amp;lt;double&amp;gt;(-845)/static_cast&amp;lt;double&amp;gt;(4104);&#xA;&#xA;    double a61 = static_cast&amp;lt;double&amp;gt;(-8)/static_cast&amp;lt;double&amp;gt;(27);&#xA;    double a62 = static_cast&amp;lt;double&amp;gt;(2);&#xA;    double a63 = static_cast&amp;lt;double&amp;gt;(-3544)/static_cast&amp;lt;double&amp;gt;(2565);&#xA;    double a64 = static_cast&amp;lt;double&amp;gt;(1859)/static_cast&amp;lt;double&amp;gt;(4104);&#xA;    double a65 = static_cast&amp;lt;double&amp;gt;(-11)/static_cast&amp;lt;double&amp;gt;(40);&#xA;&#xA;    //for loop that integrates over the specified number of points. Actually, might have to make it a do-while loop for adaptive step sizes &#xA;    for(int k = 0; k &amp;lt; *numpoints; k++)&#xA;    {&#xA;        if (k!=0)&#xA;        {&#xA;            arrCopy&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concs, tempsum);&#xA;            cudaDeviceSynchronize();&#xA;        }&#xA;        arrInit&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, 0);&#xA;        cudaDeviceSynchronize();&#xA;        arrInit&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(temp1, 0);&#xA;        cudaDeviceSynchronize();&#xA;        arrInit&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(temp2, 0);&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        calcK&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concs, k1s, dt);         //k1 = dt * flux (concs)&#xA;        cudaDeviceSynchronize(); //Sync here because kernel continues onto next line before k1 finished&#xA;&#xA;        setup(a21, temp1, tempsum, k1s, concs, maxlength);  //tempsum = a21*k1&#xA;        arrAdd&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concs, tempsum, tempsum);   //tempsum = concs + a21*k1    &#xA;        cudaDeviceSynchronize();&#xA;&#xA;        calcK&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, k2s, dt);       //k2 = dt * flux (concs + a21*k1)&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        arrInit&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, 0);&#xA;        cudaDeviceSynchronize();&#xA;        setup(a31, temp1, tempsum, k1s, concs, maxlength);  //temp1sum = a31*k1&#xA;        setup(a32, temp1, tempsum, k2s, concs, maxlength);  //tempsum = a31*k1 + a32*k2&#xA;        arrAdd&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concs, tempsum, tempsum);   //tempsum = concs + a31*k1 + a32*k2&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        calcK&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, k3s, dt);           //k3 = dt * flux (concs + a31*k1 + a32*k2)&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        arrInit&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, 0);&#xA;        cudaDeviceSynchronize();&#xA;        setup(a41, temp1, tempsum, k1s, concs, maxlength);  //tempsum = a41*k1&#xA;        setup(a42, temp1, tempsum, k2s, concs, maxlength);  //tempsum = a41*k1 + a42*k2&#xA;        setup(a43, temp1, tempsum, k3s, concs, maxlength);  //tempsum = a41*k1 + a42*k2 + a43*k3&#xA;        arrAdd&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concs, tempsum, tempsum);   //tempsum = concs + a41*k1 + a42*k2 + a43*k3&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        calcK&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, k4s, dt);           //k4 = dt * flux (concs + a41*k1 + a42*k2 + a43*k3)&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        arrInit&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, 0);&#xA;        cudaDeviceSynchronize();&#xA;        setup(a51, temp1, tempsum, k1s, concs, maxlength);  //tempsum = a51*k1&#xA;        setup(a52, temp1, tempsum, k2s, concs, maxlength);  //tempsum = a51*k1 + a52*k2&#xA;        setup(a53, temp1, tempsum, k3s, concs, maxlength);  //tempsum = a51*k1 + a52*k2 + a53*k3&#xA;        setup(a54, temp1, tempsum, k4s, concs, maxlength);  //tempsum = a51*k1 + a52*k2 + a53*k3 + a54*k4&#xA;        arrAdd&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concs, tempsum, tempsum);   //tempsum = concs + a51*k1 + a52*k2 + a53*k3 + a54*k4&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        calcK&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, k5s, dt);           //k5 = dt * flux (concs + a51*k1 + a52*k2 + a53*k3 + a54*k4)&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        arrInit&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, 0);&#xA;        cudaDeviceSynchronize();&#xA;        setup(a61, temp1, tempsum, k1s, concs, maxlength);  //tempsum = a61*k1&#xA;        setup(a62, temp1, tempsum, k2s, concs, maxlength);  //tempsum = a61*k1 + a62*k2&#xA;        setup(a63, temp1, tempsum, k3s, concs, maxlength);  //tempsum = a61*k1 + a62*k2 + a63*k3&#xA;        setup(a64, temp1, tempsum, k4s, concs, maxlength);  //tempsum = a61*k1 + a62*k2 + a63*k3 + a64*k4&#xA;        setup(a65, temp1, tempsum, **k4s**, concs, maxlength);  //tempsum = a61*k1 + a62*k2 + a63*k3 + a64*k4 + a65*k5&#xA;        arrAdd&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concs, tempsum, tempsum);   //tempsum = concs + a61*k1 + a62*k2 + a63*k3 + a64*k4 + a65*k5&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        calcK&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, k6s, dt);           //k6 = dt * flux (concs + a61*k1 + a62*k2 + a63*k3 + a64*k4 + a65*k5)&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        //At this point, temp1 and tempsum are maxlength dimension arrays that are able to be used for other things.&#xA;&#xA;        //Calculate acceptable step size before storing the concentrations.&#xA;        calcStepSize(temp1, temp2, tempsum, concs, k1s, k2s, k3s, k4s, k5s, k6s, dt, maxlength);    //temp1 = 4th Order guess, tempsum = 5th Order guess&#xA;        cudaDeviceSynchronize();&#xA;&#xA;        //Store the initial conditions in the first column of the storage array.&#xA;        if (k==0)&#xA;        {&#xA;            storeConcs&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concStorage, pitch, concs, k);  //Store this step's concentrations in 2D array&#xA;            cudaDeviceSynchronize();&#xA;        }&#xA;        //Store future concentration in next column of storage array.&#xA;        storeConcs&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(concStorage, pitch, tempsum, k+1);  //Store this step's concentrations in 2D array&#xA;        cudaDeviceSynchronize();&#xA;    }&#xA;}&#xA;//calcStepSize will take in an error tolerance, the current concentrations and the k values and calculate the resulting step size according to the following equation&#xA;//e[n+1]=y4[n+1] - y5[n+1]&#xA;__device__ void calcStepSize(double* temp1, double*temp2, double* tempsum, double* concs, double* k1s, double* k2s, double* k3s, double* k4s, double* k5s, double* k6s, double* dt, int* maxlength)&#xA;{&#xA;    //do&#xA;    //{&#xA;        takeFourthOrderStep&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(temp1, concs, k1s, k2s, k3s, k4s, k5s);            //Store 4th order guess in temp1&#xA;        takeFifthOrderStep&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, concs, k1s, k2s, k3s, k4s, k5s, k6s);  //Store 5th order guess in tempsum&#xA;        cudaDeviceSynchronize();&#xA;        arrSub&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(temp1, tempsum, temp2)&#xA;        arrMin&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;&#xA;    //arrMult&#xA;    //}&#xA;    //while&#xA;}&#xA;//takeFourthOrderStep is going to overwrite the old temp1 array with the new array of concentrations that result from a 4th order step.  This kernel is meant to be launched &#xA;// with as many threads as there are discrete concentrations to be tracked.&#xA;__global__ void takeFourthOrderStep(double* y4, double* concs, double* k1s, double* k2s, double* k3s, double* k4s, double* k5s)&#xA;{&#xA;    double b41 = static_cast&amp;lt;double&amp;gt;(25)/static_cast&amp;lt;double&amp;gt;(216);&#xA;    double b42 = static_cast&amp;lt;double&amp;gt;(0);&#xA;    double b43 = static_cast&amp;lt;double&amp;gt;(1408)/static_cast&amp;lt;double&amp;gt;(2565);&#xA;    double b44 = static_cast&amp;lt;double&amp;gt;(2197)/static_cast&amp;lt;double&amp;gt;(4104);&#xA;    double b45 = static_cast&amp;lt;double&amp;gt;(-1)/static_cast&amp;lt;double&amp;gt;(5);&#xA;    int idx = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    y4[idx] = concs[idx] + b41 * k1s[idx] + b42 * k2s[idx] + b43 * k3s[idx] + b44 * k4s[idx] + b45 * k5s[idx];&#xA;}&#xA;//takeFifthOrderStep is going to overwrite the old array of concentrations with the new array of concentrations.  As of now, this will be the 5th order step.  Another function can be d&#xA;//defined that will take a fourth order step if that is interesting for any reason.  This kernel is meant to be launched with as many threads as there are discrete concentrations&#xA;//to be tracked.&#xA;//Store b values in register? Constants?&#xA;__global__ void takeFifthOrderStep(double* y5, double* concs, double* k1s, double* k2s, double* k3s, double* k4s, double* k5s, double* k6s)&#xA;{&#xA;    double b51 = static_cast&amp;lt;double&amp;gt;(16)/static_cast&amp;lt;double&amp;gt;(135);&#xA;    double b52 = static_cast&amp;lt;double&amp;gt;(0);&#xA;    double b53 = static_cast&amp;lt;double&amp;gt;(6656)/static_cast&amp;lt;double&amp;gt;(12825);&#xA;    double b54 = static_cast&amp;lt;double&amp;gt;(28561)/static_cast&amp;lt;double&amp;gt;(56430);&#xA;    double b55 = static_cast&amp;lt;double&amp;gt;(-9)/static_cast&amp;lt;double&amp;gt;(50);&#xA;    double b56 = static_cast&amp;lt;double&amp;gt;(2)/static_cast&amp;lt;double&amp;gt;(55);&#xA;    int idx = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    y5[idx] = concs[idx] + b51 * k1s[idx] + b52 * k2s[idx] + b53 * k3s[idx] + b54 * k4s[idx] + b55 * k5s[idx] + b56 * k6s[idx];&#xA;}&#xA;//storeConcs takes the current array of concentrations and stores it in the cId'th column of the 2D concStorage array&#xA;//pitch = memory size of a row&#xA;__global__ void storeConcs(double* cS, size_t pitch, double* concs, int cId)&#xA;{&#xA;    int tIdx = threadIdx.x;&#xA;    //cS is basically the memory address of the first element of the flattened (1D) 2D array.&#xA;    double* row = (double*)((char*)cS + tIdx * pitch);&#xA;    row[cId] = concs[tIdx];&#xA;}&#xA;//Perhaps I can optimize by using shared memory to hold conc values.&#xA;__global__ void calcK(double* concs, double* ks, double* dt)&#xA;{&#xA;    int idx = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    ks[idx]=(*dt)*flux(idx, concs);&#xA;}&#xA;//Adds two arrays (a + b) element by element and stores the result in array c.&#xA;__global__ void arrAdd(double* a, double* b, double* c)&#xA;{                                                 &#xA;    int idx = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    c[idx]=a[idx]+b[idx];&#xA;}&#xA;//Subtracts two arrays (a - b) element by element and stores the result in array c.&#xA;__global__ void arrSub(double* a, double* b, double* c)&#xA;{                                                 &#xA;    int idx = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    c[idx]=a[idx]-b[idx];&#xA;}&#xA;//Multiplies two arrays (a * b) element by element and stores the result in array c.&#xA;__global__ void arrMult(double* a, double* b, double* c)&#xA;{&#xA;    int idx = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    c[idx]=a[idx]*b[idx];&#xA;}&#xA;//Will find the min of errors array.&#xA;__global__ void arrMin(double* errors)&#xA;{&#xA;    //extern _shared_ double[7];&#xA;}&#xA;//Initializes an array a to double value b.&#xA;__global__ void arrInit(double* a, double b)&#xA;{&#xA;    int idx = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    a[idx]=b;&#xA;}&#xA;//Copies array b onto array a.&#xA;__global__ void arrCopy(double* a, double* b)&#xA;{&#xA;    int idx = blockIdx.x * blockDim.x + threadIdx.x;&#xA;    a[idx]=b[idx];&#xA;}&#xA;//Placeholder function for the flux calculation.  It will take the size of the oligomer and current concentrations as inputs.&#xA;__device__ double flux(int r, double *concs) &#xA;{&#xA;    return -concs[r];&#xA;}&#xA;//This function multiplies a tableau value by the corresponding k array and adds the result to tempsum.  Used to&#xA;//add all the a*k terms.&#xA;//e.g. setup(a21, temp1, tempsum, k1s, concs, maxlength) =&amp;gt; tempsum = a21 * k1&#xA;__device__ void setup(double tableauValue, double *temp1, double *tempsum, double *ks, double *concs, int *maxlength) &#xA;{&#xA;    //Sets tempsum to tabVal * k&#xA;    arrInit&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(temp1, tableauValue);      //Set [temp1] to tableau value, temp1 = a&#xA;    cudaDeviceSynchronize();&#xA;    arrMult&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(ks, temp1, temp1);         //Multiply tableau value by appropriate [k], temp1 = a*k&#xA;    cudaDeviceSynchronize();&#xA;    arrAdd&amp;lt;&amp;lt;&amp;lt; 1, *maxlength &amp;gt;&amp;gt;&amp;gt;(tempsum, temp1, tempsum);   //Move tabVal*k to [tempsum], tempsum = tempsum+temp1&#xA;    cudaDeviceSynchronize();&#xA;    //temp1 = tableauValue * kArray&#xA;    //tempsum = current sum (tableauValue * kArray)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FINAL EDIT&lt;/strong&gt;&#xA;I found my error.  It was a typo calculating the K6's.  I put asterisks around the offending &quot;k4s&quot; term which should be a &quot;k5s&quot;.  Anyone who wants a CUDA RKF45 integrator is welcome to use it as it works now and is of the correct order.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Number of Points    h       Global Error    Calculated Order&#xA;        10          0.1         3.59E-09    &#xA;        20          0.05        1.08E-10    5.06E+00&#xA;        40          0.025       3.31E-12    5.03E+00&#xA;        80          0.0125      1.03E-13    5.01E+00&#xA;        160         0.00625     2.20E-15    5.55E+00&#xA;        320         0.003125    1.20E-15    8.74E-01&#xA;        640         0.0015625   1.40E-15    -2.22E-01&#xA;        1280        0.00078125  2.00E-16    2.81E+00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5553" LastEditorUserId="5553" LastEditDate="2013-11-12T03:56:35.017" LastActivityDate="2013-11-12T03:56:35.017" Title="Why am I getting so much error for my Runge Kutta Fehlberg solver?" Tags="|ode|c++|time-integration|cuda|runge-kutta|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="8996" PostTypeId="2" ParentId="8995" CreationDate="2013-11-08T13:32:39.493" Score="8" Body="&lt;p&gt;I am assuming that you are setting the error tolerance at &lt;code&gt;1e-12&lt;/code&gt;.  You are correct that when an adaptive scheme accepts the current step size, it assumes the 5th order scheme was, for all intents and purposes, the &quot;correct&quot; answer.  However this is &lt;strong&gt;only&lt;/strong&gt; when it accepts the current step.  If the difference between the 4th and 5th order steps are too large, it doesn't accept &lt;strong&gt;either&lt;/strong&gt; as good enough.  For testing a single algorithm like this you need to run it with multiple time steps and see the order of convergence to verify it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now on the other hand, a step of &lt;code&gt;0.001&lt;/code&gt; with a 5th order scheme on your test problem (I am assuming was 1D) should be more than sufficient to drive the error to nearly machine epsilon.  My guess is that you have a typo somewhere that reduces your integrator to 1st order.  Like I said in paragraph 1, run it with multiple time steps and verify its convergence order.  This will tell you whether the error you see is reasonable, or if you have an error somewhere in your integrator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the convergence order of an ode solver, run it on a problem with a known solution for two different time steps $h$ and $\frac{h}{2}$.  Your global error should then be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{E(h)}{E(\frac{h}{2})} = 2^n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the 2 comes from the ratio between time steps.  You should get $n=5$ for your 5th order scheme.  &lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-11-09T15:38:08.310" LastActivityDate="2013-11-09T15:38:08.310" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="8997" PostTypeId="2" ParentId="8987" CreationDate="2013-11-08T15:48:11.473" Score="7" Body="&lt;p&gt;Since the outer integral $\int_0^{2\pi} F(\theta) d\theta$ has a periodic integrand, a trapezoidal rule should work nicely, picking up &lt;a href=&quot;http://en.wikipedia.org/wiki/Trapezoidal_rule#Periodic_functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;extra accuracy&lt;/a&gt; when $F(\theta)$ is smooth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the inner integral you want to take advantage of the known &quot;weight function&quot; $r$ that appears in the integrand, $F(\theta) = \int_0^{r(\theta)} f(r,\theta) r\; dr$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to implement this is to set some threshold $r_\epsilon \gt 0$ around the origin and use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Gaussian_quadrature&quot; rel=&quot;nofollow noreferrer&quot;&gt;weighted Gaussian quadrature rule&lt;/a&gt; on the inner radial segment $[0,r_\epsilon]$, combined if necessary with a compound trapezoidal rule on the exterior segment $[r_\epsilon,r(\theta)]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The common Gaussian quadrature schemes are either &lt;a href=&quot;http://pomax.github.io/bezierinfo/legendre-gauss.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;for the constant weight function&lt;/a&gt; $w(x) = 1$ (Gauss-Legendre) or for much fancier weight functions than $w(x) = x$.  I looked around the web but did not find a quick reference for this case, although it appears as &lt;a href=&quot;https://math.stackexchange.com/questions/377008/gaussian-quadrature-three-point&quot;&gt;a homework-type exercise on Math.SE&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps we should be able to modify the computation of the Gauss-Legendre nodes (roots of Legendre polynomials) to get the nodes and weights for this case, but I have done the 1-, 2- and 3-point rules from &quot;scratch&quot; as concisely &lt;a href=&quot;http://kfe.fjfi.cvut.cz/~klimo/nm/c11/gaussian.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;outlined in this note&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an $N$-point rule we need the (monic) polynomial $q(x)$ of degree $N$ which is orthogonal to lower degree polynomials on $[0,1]$ when integrated with weight function $w(x) = x$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_0^1 q(x) x^k\; x dx \text{ for } k=0,1,\ldots,N-1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When $N=1$ we get $q(x) = x - \frac{2}{3}$, so the node is $x=\frac{2}{3}$ and weight $\omega = \frac{1}{2}$ (because $\int_0^1 x dx = \frac{1}{2}$ should be exactly integrated).  This amounts to a quadrature rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_0^1 f(x) x dx \approx \frac{1}{2} f\left(\frac{2}{3}\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adapting to the case of integrating over $[0,r(\theta)]$ amounts to throwing in factors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_0^{r(\theta)} f(r) \; r dr \approx \frac{r(\theta)}{2} f\left(\frac{2r(\theta)}{3}\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $N=2$ we get a quadratic $q(x) = x^2 - 1.2x + 0.3$ whose roots are two points in $[0,1]$, and the respective weights are found by requiring $1,x$ to be exactly integrated:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_\pm = \frac{1}{10} (6 \pm \sqrt{6}) \; , \; \omega_\pm = \frac{1}{36} (9 \pm \sqrt{6}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $N=3$ the cubic polynomial $q(x) = x^3 - \frac{12}{7} x^2 + \frac{6}{7} x - \frac{4}{35}$ turns out to one of those irreducible cases whose three real roots cannot be expressed in terms of radicals without using complex constants.  So in lieu of symbolic expressions we will note the single precision values for the three roots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_0 \approx 0.21234, x_1 \approx 0.59053, x_2 \approx 0.91141 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be refined if necessary by Newton iterations to any desired precision.  The corresponding weights can then be found by solving the linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \omega_0 + \omega_1 + \omega_2 = \frac{1}{2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_0 \omega_0 + x_1 \omega_1 + x_2 \omega_2 = \frac{1}{3} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_0^2 \omega_0 + x_1^2 \omega_1 + x_2^2 \omega_2 = \frac{1}{4} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the single-precision values of nodes $x_i$ gives approximate weights:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \omega_0 \approx 0.06983, \omega_1 \approx 0.22924, \omega_2 \approx 0.20093 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_0^1 f(x) x dx \approx \omega_0 f(x_0) + \omega_1 f(x_1) + \omega_2 f(x_2) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this quadrature can be scaled to interval $[0,r(\theta)]$ in a similar manner to that shown for the 1-point rule above.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-11-09T12:36:04.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="8998" PostTypeId="1" AcceptedAnswerId="8999" CreationDate="2013-11-08T17:13:45.407" Score="54" ViewCount="11761" Body="&lt;p&gt;I apologize if this is a vague question, but here goes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over the past few years, &lt;a href=&quot;http://en.wikipedia.org/wiki/Functional_programming&quot;&gt;functional programming&lt;/a&gt; has received a lot of attention in the Software Engineering community. Many have started using languages such as Scala and Haskell and claimed success over other programming languages and paradigms. My question is: as high performance computing / scientific computing experts, should we be interested in functional programming? Should we be participating in this mini-revolution? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What are the pros and cons of functional programming in the SciComp domain of work?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="529" LastEditorUserId="11781" LastEditDate="2014-11-13T02:19:55.147" LastActivityDate="2024-02-13T02:12:37.453" Title="Functional Programming and Scientific Computing" Tags="|programming-paradigms|" AnswerCount="9" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="8999" PostTypeId="2" ParentId="8998" CreationDate="2013-11-08T19:22:42.963" Score="38" Body="&lt;p&gt;I've only done a little bit of functional programming, so take this answer with a grain of salt.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Pros:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Functional programming looks very mathematical; it's a nice paradigm for expressing some mathematical concepts&lt;/li&gt;&#xA;&lt;li&gt;There are good libraries available for things like formal verification of programs and theorem proving, so it's possible to write programs that reason about programs -- this aspect is good for reproducibility&lt;/li&gt;&#xA;&lt;li&gt;You can do functional programming in Python and C++ via lambda expressions; you can also do functional programming in Julia and Mathematica&lt;/li&gt;&#xA;&lt;li&gt;Not many people use it, so you can be a pioneer. Much like there were early adopters of MATLAB, Python, R, and now Julia, there need to be early adopters of functional programming for it to catch on&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Cons:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Languages that are typically thought of as functional programming languages, like Haskell, OCaml (and other ML dialects), and Lisp are generally thought of as slow relative to languages used for performance-critical scientific computing. OCaml is, at best, around half as fast as C. &lt;/li&gt;&#xA;&lt;li&gt;These languages lack library infrastructure compared to languages commonly used in computational science (Fortran, C, C++, Python); if you want to solve a PDE, it's way easier to do it in a language more commonly used in computational science than one that is not.&lt;/li&gt;&#xA;&lt;li&gt;There isn't as much of a computational science community using functional programming languages as there is using procedural languages, which means you won't get a whole lot of help learning it or debugging it, and people are probably going to give you crap for using it (whether or not you deserve it)&lt;/li&gt;&#xA;&lt;li&gt;The style of functional programming is different than the style used in procedural programming, which is typically taught in introductory computer science classes and in &quot;MATLAB for Scientists and Engineers&quot;-type classes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think many of the objections in the &quot;Cons&quot; section could be overcome. As is a common point of discussion on this Stack Exchange site, developer time is more important than execution time. Even if functional programming languages are slow, if performance-critical portions can be delegated to a faster procedural language and if productivity gains can be demonstrated through rapid application development, then they might be worth using. It's worth noting here that programs implemented in pure Python, pure MATLAB, and pure R are considerably slower than implementations of these same programs in C, C++, or Fortran. Languages like Python, MATLAB, and R are popular precisely because they trade execution speed for productivity, and even then, Python and MATLAB both have facilities for implementing interfaces to compiled code in C or C++ so that performance-critical code can be implemented to execute quickly. Most languages have a foreign function interface to C, which would be enough to interface with most libraries of interest to computational scientists.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Should you be interested in functional programming?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;That all depends on what you think is cool. If you're the type of person who is willing to buck convention and you're willing to go through the slog of evangelizing to people about the virtues of whatever it is you want to do with functional programming, I'd say go for it. I would love to see people do cool things with functional programming in computational science, if for no other reason than to prove all of the naysayers wrong (and there will be a lot of naysayers). If you're not the type of person who wants to deal with a bunch of people asking you, &quot;Why in &lt;em&gt;hell&lt;/em&gt; are you using a functional programming language instead of (insert their favorite procedural programming language here)?&quot;, then I wouldn't bother.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's been some use of functional programming languages for simulation-intensive work. The quantitative trading firm Jane Street uses OCaml for financial modeling and execution of its trading strategies. OCaml was also used in FFTW for generating some C code used in the library. Liszt is a domain-specific language developed at Stanford and implemented in Scala that is used for solving PDEs. Functional programming is definitely used in industry (not necessarily in computational science); it remains to be seen whether it will take off in computational science.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-11-10T09:33:27.180" LastActivityDate="2013-11-10T09:33:27.180" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10002" PostTypeId="1" AcceptedAnswerId="10010" CreationDate="2013-11-10T19:43:33.623" Score="4" ViewCount="1527" Body="&lt;p&gt;I want to do engineering and scientific calculations im &lt;a href=&quot;http://maxima.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;Maxima/wxmaxima&lt;/a&gt;. I do not want to manually check every time if my units make sense, how can I achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking at Maxima because it's free and solves equations etc. symbolically if asked.&lt;/p&gt;&#xA;" OwnerUserId="3993" LastActivityDate="2019-07-19T14:41:25.977" Title="How can I teach Maxima to use units?" Tags="|symbolic-computation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10003" PostTypeId="1" CreationDate="2013-11-10T21:06:56.603" Score="10" ViewCount="809" Body="&lt;p&gt;I would like to hear about scientific codes and commercial packages utilizing meshless methods like Element-Free Galerkin based on Moving Least Squares functions. By &quot;serious&quot; I mean they could be used to solve problems comparable e.g. in size to those solved by FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's already more than fifteen years since their inception. People who developed them considered them to be very promising. I'm simply trying to understand the current state of the art.&lt;/p&gt;&#xA;" OwnerUserId="3637" LastEditorUserId="20688" LastEditDate="2018-05-06T04:31:13.730" LastActivityDate="2018-12-22T22:23:41.510" Title="Could you give examples of serious usage of meshfree methods?" Tags="|finite-element|meshless-methods|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10004" PostTypeId="1" CreationDate="2013-11-10T22:32:44.243" Score="3" ViewCount="1429" Body="&lt;p&gt;There is a least-squares problem. It can be solved using backslash in Matlab.&lt;br&gt;&#xA;If &lt;code&gt;Ax = b&lt;/code&gt;, then &lt;code&gt;x = A \ b&lt;/code&gt;.&lt;br&gt;&#xA;Let's assume that I have the same problem, but all &lt;code&gt;x&lt;/code&gt; must be non-negative (&lt;code&gt;&amp;gt;=0&lt;/code&gt;). How can I solve this problem in Matlab by analogy with the previous one (without non-negativity constraints)? I think it can be somehow connected with the active set method.&lt;br&gt;&#xA;I know that the non-negative least squares problem can be easily solved with Matlab Optimization toolbox or CVX or in many other ways. But still I'm curious about solving it by analogy with a straight-forward least-squares method (with backslash).&lt;br&gt;&#xA;Could anyone help me please?&lt;/p&gt;&#xA;" OwnerUserId="6566" LastActivityDate="2014-02-06T13:52:00.863" Title="Solving non-negative least squares in Matlab (by analogy with least squares)" Tags="|matlab|least-squares|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10005" PostTypeId="1" AcceptedAnswerId="10007" CreationDate="2013-11-10T22:35:51.000" Score="3" ViewCount="267" Body="&lt;p&gt;I am trying to find a good way to handle the following problem: Let C be an N by 3 array (corresponding to points in $\mathbb{R}^3$). There is a method I am interested in testing which requires constructing, for i=1....N, a matrix $A_i$ of size $n_i &amp;lt;&amp;lt;N$ and a vector $b_i$ of size $n_i$ using only data from the row i of C. Then, we seek the solution $x_i = A_i^{-1} b_i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently am running this in MATLAB as a large for loop going from i=1 to i=N, and doing the above operations. For N = 30,000, this takes a while and I was hoping to find some way to speed things up. Since each iteration is independent of the other, I was hoping I could parallelize the for loop. However, I am unfamiliar with parallel programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have access to the matlab parallel toolbox, so I cannot simply invoke parfor to parallelize the loop. If it helps, I am familiar with Python+Numpy+Scipy, and also a bit with C/C++ and eager to learn MPI or CUDA or whatever would be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I correct in thinking that this for loop could be parallelized for faster performance? Can anyone point me to some resources to get started for a parallel (or, faster) implementation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6567" LastActivityDate="2013-11-11T02:16:55.480" Title="Inverting many small matrices in parallel" Tags="|linear-algebra|matrix|parallel-computing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10006" PostTypeId="2" ParentId="8998" CreationDate="2013-11-10T23:20:31.313" Score="9" Body="&lt;p&gt;Geoff has already given a good overview of the reasons to which I have little add other than emphasize one of his points: ecosystem. Whether you're advocating functional programming or any other paradigm, one of the important questions you have to address is that there is an incredible amount of software everyone else can build upon that you have to re-write. Examples are MPI, PETSc or Trilinos for linear algebra, or any of the finite element libraries -- all written in C or C++. There is a huge amount of inertia in the system, maybe not because everyone thinks that C/C++ is in fact the best language to write computational software in, but because a lot of people have spent years of their lives creating something that is useful to a lot of people.&lt;/p&gt;&#xA;&lt;p&gt;I think most computational people will agree that there is a lot of value in trying new programming languages and evaluating their suitability for this problem. But it's going to be a difficult and lonely time because you will not be able to produce results that are competitive with what everyone else is doing. It may also yield you a reputation as someone who started the next move to a different programming paradigm. Hey, it only took C++ about 15 years to replace Fortran!&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2024-02-13T02:12:37.453" LastActivityDate="2024-02-13T02:12:37.453" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="10007" PostTypeId="2" ParentId="10005" CreationDate="2013-11-11T02:16:55.480" Score="3" Body="&lt;p&gt;You could do this in Numpy with Python's Multiprocessing (specfically by building a Pool and using Pool.map over the matrices you want to solve).&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-11-11T02:16:55.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10008" PostTypeId="2" ParentId="10004" CreationDate="2013-11-11T02:40:12.103" Score="5" Body="&lt;p&gt;MATLAB has a built-in function lsqnonneg() which is an implementation of the active set method described in the book &quot;Solving Least Squares Problems&quot; by Lawson and Hanson (1974)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The \ solution of linear least squares problems in MATLAB is done by a variety of different algorithms depending on the exact structure of the problem.  None of them are particularly similar to the active set method used in lsqnonneg().  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-11-11T02:40:12.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10009" PostTypeId="1" AcceptedAnswerId="10015" CreationDate="2013-11-11T07:07:48.020" Score="5" ViewCount="664" Body="&lt;p&gt;I'm reading now&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Shu, C.-W. (1999). High Order ENO and WENO Schemes for Computational Fluid &gt;Dynamics. (T. J. Barth &amp;amp; H. Deconinck, Eds.)Lecture Notes in Computational &gt;Science and Engineering, 9, 439–582. doi:10.1007/978-3-662-03882-6_5&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The author provides two formulations of WENO methods: finite-difference and finite-volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't understand is why in finite-volume formulation we use Riemann solver, but in finite-difference we don't use it?&lt;/p&gt;&#xA;" OwnerUserId="3254" LastActivityDate="2013-11-12T01:30:15.407" Title="Riemann Solver in WENO methods" Tags="|finite-difference|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10010" PostTypeId="2" ParentId="10002" CreationDate="2013-11-11T18:26:46.490" Score="1" Body="&lt;p&gt;I haven't used any of the following packages but they seem to be what you are looking for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://maxima.sourceforge.net/docs/manual/en/maxima_51.html#SEC257&quot; rel=&quot;nofollow&quot;&gt;http://maxima.sourceforge.net/docs/manual/en/maxima_51.html#SEC257&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://maxima.sourceforge.net/docs/manual/en/maxima_79.html#SEC369&quot; rel=&quot;nofollow&quot;&gt;http://maxima.sourceforge.net/docs/manual/en/maxima_79.html#SEC369&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-11-11T18:26:46.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10011" PostTypeId="1" AcceptedAnswerId="10016" CreationDate="2013-11-11T18:48:45.593" Score="1" ViewCount="13621" Body="&lt;p&gt;This was so much easier in Python (so much so that I'm considering going back to matplotlib for data analysis).  But really, trying to figure out ROOT and Boost graph library and mathgl makes me want to die.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm thinking for the short term is to somehow export my 2d array to a text file and then import that file into a numpy array as I already have that infrastructure coded. Then I'll use matplotlib to visualize the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the long term, I'd like to learn the &quot;standard way&quot;to visualize data from c++ to add to my list of marketable skills. I've been scared away from Root by some of my particle physics friends, disgusted away from mathgl by a Linux snob and like how gnuplot looks but am turned off by the fact that it's command line operated. Is there nothing directly synonymous with matplotlib in c++ in functionality and ease of use?&lt;/p&gt;&#xA;" OwnerUserId="5553" LastActivityDate="2020-07-23T00:47:16.133" Title="What's the simplest way to graph a 2d array generated in C++ using Windows 7?" Tags="|c++|plotting|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10012" PostTypeId="2" ParentId="10011" CreationDate="2013-11-11T19:17:47.273" Score="2" Body="&lt;p&gt;Well, for one, you can call gnuplot as a library. Pretty much every command you'd use at the prompt has a similarly-named API function.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-11-11T19:17:47.273" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10013" PostTypeId="1" AcceptedAnswerId="10017" CreationDate="2013-11-11T20:41:25.837" Score="7" ViewCount="1280" Body="&lt;p&gt;On one hand, one may seed the domain with particles and track their trajectories in the Lagrangian sense by implementing a Lagrangian particle tracking model. On the other hand, one may use the Eulerian approach and solve a scalar transport advection/diffusion equation for a passive scalar similar to what is done for the density transport equation (except density is not a passive quantity because it is typically coupled to the momentum equation, whereby it influences the velocity field through the buoyancy term).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my perspective the first option is simpler from a numerical implementation standpoint since all you need is to perform an integration in time, but it is missing diffusion. For most flows, molecular diffusion can be neglected so that may not even be an issue. The second option involves solving the advection/diffusion equation which has stability and other types of concerns. Are there any other considerations that I should think about?&lt;/p&gt;&#xA;" OwnerUserId="3561" LastEditorUserId="3561" LastEditDate="2013-11-13T05:40:15.640" LastActivityDate="2013-11-13T05:40:15.640" Title="What are some of the differences between using a Lagrangian and Eulerian framework to quantify passive scalar dynamics?" Tags="|fluid-dynamics|navier-stokes|advection-diffusion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10014" PostTypeId="1" CreationDate="2013-11-11T21:30:06.087" Score="5" ViewCount="196" Body="&lt;p&gt;I have a function that is expensive to evaluate whose inputs are n-dimensional (n is the order of a dozen or two).  I need the output of this function at each node and each time step for a PDE simulation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the portion of the n-d space sampled is small since n-1 of the inputs are species mole fractions.  I know the outputs are continuous in all dimensions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to have the solver adaptively sample the n-d space as needed based on the inputs it is being provided.  If the new inputs are close to/contained by old inputs, a simple linear interpolation would be sufficient.  If the new inputs are significantly different from old inputs, I want to add points to the &quot;table&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inputs/outputs are relatively compact, so RAM is not an issue.  I am just wondering if there are data structures/algorithms that will make the lookup and interpolation efficient while allowing for the quick addition of new points.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-09-08T23:58:43.833" Title="Adaptive Table Lookup for Expensive Function Evaluation" Tags="|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10015" PostTypeId="2" ParentId="10009" CreationDate="2013-11-12T01:30:15.407" Score="6" Body="&lt;p&gt;They are different methods and presence of a Riemann solver is the distinguishing characteristic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite Difference WENO starts with vertex values, evaluates the fluxes associated with left- and right-traveling waves, and uses weighted averaging to define a flux at staggered points.  (In the paper, the average is done in local characteristic variables.)  FD WENO is inexpensive, but only works on smooth meshes and requires a flux splitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite Volume WENO reconstructs a piecewise discontinuous state and evaluates it at quadrature points on cell interfaces.  A single-valued flux is evaluated using the multivalued state at those quadrature points.  FV WENO is more general, but Riemann solvers are often more expensive and most full-accuracy variants end up needing many more evaluations in multiple directions (due to the need for transverse reconstruction).  See &lt;a href=&quot;http://www.global-sci.com/freedownload/v9_807.pdf&quot;&gt;Zhang, Zhang, and Shu, &lt;em&gt;On the Order of Accuracy and Numerical Performance of Two Classes of Finite Volume WENO Schemes&lt;/em&gt;, 2011&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-11-12T01:30:15.407" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10016" PostTypeId="2" ParentId="10011" CreationDate="2013-11-12T02:03:04.027" Score="9" Body="&lt;p&gt;There are a couple subtleties to your question that I think are important:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You're comparing an interpreted language (Python) to a compiled language (C++).&lt;/li&gt;&#xA;&lt;li&gt;Most scientific and engineering software is developed with a heavy Linux (and UNIX) bias, and is not usually known for cross-platform compatibility or great user support (big libraries, of course, excepted)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you really want to use a C++ library, you probably want something that has a CMake-based build system. PLplot is one such library that looks reputable and has a C++ interface (unfortunately, the others that seem to come up are ROOT and MathGL...).&lt;/p&gt;&#xA;&lt;p&gt;However, given the points above, if using Cygwin or MinGW is going to drive you into a murderous rage, I'd stick with Python or a visualization program you could install from a binary (like Paraview or VisIt, for instance; other programs could also work). Here's why:&lt;/p&gt;&#xA;&lt;h3&gt;Interpreted languages are convenient for tweaking visualizations&lt;/h3&gt;&#xA;&lt;p&gt;The big advantage of doing your visualization in C++ is that you can append it to whatever C++ code you might be using for simulation work. That said, if you have to compile your code after every modification you make, it's going to make tweaking your visualizations that much more time-consuming. With an interpreted language, once the data is loaded, you can post-process and tweak visualizations on the command line. If you plot raw data and decide it might look better normalized, no problem; a few lines of code, and you can bring up a new plot in a minute or two without having to recompile.&lt;/p&gt;&#xA;&lt;h3&gt;Compiling scientific software on Windows is a pain&lt;/h3&gt;&#xA;&lt;p&gt;...because much of the time, whoever wrote it didn't use a cross-platform build system like CMake, so you either have to slog through and hack together a Makefile that works, or use Cygwin or MinGW to compile everything and hope that the software doesn't rely on Linux libraries (because after all, Cygwin is a POSIX compatibility layer). If a project uses CMake, it probably means that the developers have thought about these issues, and CMake will output the necessary build scripts to compile software with Visual Studio, which will make your life less painful. Otherwise, it's probably not worth the time and effort.&lt;/p&gt;&#xA;&lt;p&gt;If you can install the visualization software you need from a binary, you don't have to worry about grappling with Visual Studio. There are at least two Python distributions I can think of that will supply Matplotlib (Continuum's Anaconda and Enthought Canopy) with a binary install of Python on multiple platforms. Paraview and VisIt are both powerful scientific visualization packages that, again, can be installed from binaries (they can also be built from source on Windows).&lt;/p&gt;&#xA;&lt;h3&gt;I don't think there is a &amp;quot;standard C++ way&amp;quot; to plot data&lt;/h3&gt;&#xA;&lt;p&gt;I haven't heard of a &amp;quot;standard C++ way&amp;quot; to plot data (it doesn't mean it doesn't exist); when I talk to data scientists, a lot of them stress how convenient and important it is to be fluent in an interpreted language like Python, R, or MATLAB because visualization and post-processing can be developed rapidly in those languages. Standard practice for a lot of fields is to dump data to some text format (column-delimited, CSV, HDF5, VTK, etc.) and then read in the data using one of those interpreted languages or a dedicated visualization program (like VisIt, Paraview, and for 2-D plots, gnuplot).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2013-11-12T02:03:04.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10017" PostTypeId="2" ParentId="10013" CreationDate="2013-11-12T06:50:14.170" Score="6" Body="&lt;p&gt;I think this greatly depends on what kind of physics you are trying to model even though for some problems both approaches are viable.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Lagrangian vs. Eulerian Framework&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For certain problems Lagrangian frameworks are better suited than their Eulerian counterparts. For instance, if one is interested in studying the current patterns or sedimentation problems in oceanography it is more natural to do Lagrangian particle tracking than solving for a concentration field. Especially since, at least for some of such problems, diffusion would be meaningless and numerical diffusion inherent in Eulerian methods adversely affect the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, certain problems are better formulated in Eulerian framework. Probably the most prominent example here is interface tracking. Evolving interfaces using Lagrangian framework, a.k.a front tracking methods, are very hard to implement, especially for 3D problems. This is because motion of individual particles do not satisfy entropy conditions and often lead to ambiguous definition of &quot;interfaces location&quot;. To prevent this, connectivity must be defined for neighboring points which is hard to handle when the interface undergoes topological changes. However this class of problems are very naturally expressed in the Eulerian framework, e.g. in the level-set framework.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Pros and Cons&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Apart from which formulation would be more natural for a given problem, there are pros and cons to both methods.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Lagrangian formulation is essentially diffusion-free. This could be viewd as a blessing or short-coming depending on what you are doing.&lt;/li&gt;&#xA;&lt;li&gt;Lagrangian formulation is highly parallelizable since essentially all particles are advected independently.&lt;/li&gt;&#xA;&lt;li&gt;Lagrangian formulation is essentially CFL-free as one is integrating along the characteristics. Note that this property may also be exploited in the Eulerian framework through the so-called &quot;Semi-Lagrangian&quot; methods&lt;/li&gt;&#xA;&lt;li&gt;Eulerian framework is more suited for problems that involve interfaces. They can very naturally express complicated topological changes, e.g. as in the level-set method. Moreover, for this class of problems, Lagrangian methods cannot be used if the interface develops shocks and/or rarefaction fans.&lt;/li&gt;&#xA;&lt;li&gt;It is easier to develop high-order accurate methods in the Lagrangian framework.&lt;/li&gt;&#xA;&lt;li&gt;Reconstruction of &quot;concentration fields&quot; from individual particle locations is expensive and the final result may not be very smooth&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-11-12T06:50:14.170" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10018" PostTypeId="1" AcceptedAnswerId="10027" CreationDate="2013-11-12T12:14:02.140" Score="4" ViewCount="154" Body="&lt;p&gt;I'm trying to solve a multipole system. It involves a matrix of 3x3 tensors $A_{ij}$ and a vector of 3-tuples $\mathbf v_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(\begin{matrix} &#xA;A_{11} &amp;amp; A_{12} &amp;amp; \cdots &amp;amp; A_{1n}\\&#xA;A_{21} &amp;amp; A_{22} &amp;amp; \cdots &amp;amp; A_{2n}\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots\\&#xA;A_{n1} &amp;amp; A_{n2} &amp;amp; \cdots &amp;amp; A_{nn}&#xA;\end{matrix}\right) \left(\begin{matrix}&#xA;\mathbf v_1 \\ \mathbf v_2 \\ \vdots \\ \mathbf v_n&#xA;\end{matrix}\right) = \lambda \left(\begin{matrix}&#xA;\mathbf v_1 \\ \mathbf v_2 \\ \vdots \\ \mathbf v_n&#xA;\end{matrix}\right)&#xA;$$&#xA;It doesn't seem that I can trivially solve it using available linear algebra packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is that I expand the matrix of tensors into a $3n\times3n$ matrix, and the vector of vectors into a $3n$ vector,&#xA;$$\mathbf v_i = \left(\begin{matrix}v_{ix}\\v_{iy}\\v_{iz}\end{matrix}\right)$$&#xA;But it seems like I'd get up to a maximum of $3n$ eigenvalues instead of just $n$ if I do it this way. Is there a better way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit1&lt;/strong&gt;: since $v_{ix}$, $v_{iy}$, and $v_{iz}$ are components of a vector, they should fulfill the following constraint:&#xA;$$&#xA;v_{ix}^2 + v_{iy}^2 + v_{iz}^2 = c_i^2&#xA;$$&#xA;So I suppose this reduces the number of eigenvalues from $3n$ to $2n$, right? But is this even a linear algebra problem anymore?&lt;/p&gt;&#xA;" OwnerUserId="4531" LastEditorUserId="4531" LastEditDate="2013-11-13T02:16:36.437" LastActivityDate="2013-11-13T03:23:14.773" Title="Solver for eigensystem of vectors?" Tags="|linear-algebra|eigensystem|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10019" PostTypeId="2" ParentId="8075" CreationDate="2013-11-12T14:45:19.993" Score="0" Body="&lt;p&gt;Have you considered the qmake command that comes with grid engine?  A bit limited as the individual subjobs can't really be parallel but since you mentioned make...  &lt;/p&gt;&#xA;" OwnerUserId="6575" LastActivityDate="2013-11-12T14:45:19.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10020" PostTypeId="2" ParentId="10013" CreationDate="2013-11-12T16:26:59.930" Score="2" Body="&lt;p&gt;Heat is a passive scalar (under assumptions of incompressibility). But there is no heat transfer from boundaries without diffusion. So you can't really do any such heat transfer problems in a purely Lagrangian framework (unless you go stochastic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Same thing holds for permeable boundaries where the scalar diffuses into the domain.&lt;/p&gt;&#xA;" OwnerUserId="6576" LastActivityDate="2013-11-12T16:26:59.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10021" PostTypeId="2" ParentId="8740" CreationDate="2013-11-12T18:32:39.743" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Instead I impose the zero mean condition on every level of the FMG cycle.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't believe you need to (or want to) do this. It should be sufficient to only do this on the finest grid. With FAS, the restriction of the fine-grid residual to the coarse grid should enforce this condition without having to re-apply it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally as @Guido said, GS is a bad smoother for this, and you will most likely see some improvement with higher accuracy mesh transfers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also run across codes that use lower-order discretization schemes on the coarse grids, i.e. a finite volume scheme using 2nd order on the finest grid, and 1st order on the coarse grids. This seems to work quite well, since the purpose of the coarse grid is to smooth out the errors of the fine one so the diffusive low order scheme actually helps you. This seems somewhat analogous to the practice in high order DG methods where people use &quot;P-multigrid&quot; -- instead of geometric multigrid, you use the same grid in your multigrid cycle but with varying order of the polynomial reconstruction.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-11-12T18:32:39.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10022" PostTypeId="2" ParentId="10014" CreationDate="2013-11-12T18:48:28.133" Score="1" Body="&lt;p&gt;Is it feasible to use something like &lt;a href=&quot;http://dakota.sandia.gov/docs/dakota/stable/html-ref/MethodCommands.html#MethodNonDPCE&quot; rel=&quot;nofollow&quot;&gt;Polynomial Chaos Expansion?&lt;/a&gt; With that, you carefully choose discrete points in your n-d space to sample and reconstruct a unique polynomial with it. Depending on what your function looks like, you could construct this analytical expression only once at the beginning of execution. Depending on your tolerance for error and how you've been evaluating your function, this could also just provide you with a good initial guess for an iterative method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also some adaptive sampling techniques nearby that Dakota link I put above that can help you with a literature search.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-11-12T18:48:28.133" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10024" PostTypeId="1" AcceptedAnswerId="10025" CreationDate="2013-11-12T22:21:11.800" Score="6" ViewCount="94" Body="&lt;p&gt;Given iterative method: $x_{n+1}=0.7\sin x_n +5 = \phi(x_n)$ for finding solution for $x=0.7\sin x +5$, I want to estimate $|e_6|=|x_6-r|$ as good as possible, with $x_0=5$, where $r$ is exact solution. This method obviously converges, because $\phi$ is contraction mapping, so $r=\phi(r)$ is a fixed point. So, with mean value theorem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|e_{n+1}|=|x_{n+1}-r|=|\phi(x_n)-\phi(r)|\le \max_{c\in\mathbb{R}}|\phi'(c)|\cdot |x_n-r|$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|e_n|\le 0.7^n \cdot |e_0|$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't know how can I estimate $|e_0|$ without a computer? I suppose there is some simple way to finish it and with clever observation $|e_0|\le 0.7$. Can anybody help?&lt;/p&gt;&#xA;" OwnerUserId="5540" LastActivityDate="2013-11-12T22:51:06.180" Title="Numerical iterative method, estimating error" Tags="|iterative-method|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10025" PostTypeId="2" ParentId="10024" CreationDate="2013-11-12T22:51:06.180" Score="6" Body="&lt;p&gt;Given some $x_0$, you are looking for a bound on the error of the root for the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ 0 = 0.7 \mathrm{sin}(x) - x + 5$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the function is bounded above by $5.7 - x$ and below by $4.3 - x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of these have a trivial root at $5.7$ and $4.3$ respectively.  You know that any roots of your function then fall in $[ 4.3,    5.7]$.  Therefore, for any $x_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e_0 = max\{|x_0 - 5.7|,|x_0 - 4.3|\}$&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2013-11-12T22:51:06.180" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10027" PostTypeId="2" ParentId="10018" CreationDate="2013-11-13T03:23:14.773" Score="1" Body="&lt;p&gt;Either way, the number of generalized eigenvectors of your system should be $3n$; the eigenvectors of your linear system may not form a complete basis. Either way you pose the problem (expanding the matrix of tensors, or not expanding the matrix of tensors), you should have the same number of eigenvalues. Normalizing the eigenvectors (which is what your constraint does, in essence) does not change the number of eigenvalues of a matrix. (You can show that if $v$ is an eigenvector of matrix $A$, then $v / \|v\|$ is also an eigenvector of matrix $A$; by definition, $v \neq 0$.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-11-13T03:23:14.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10028" PostTypeId="2" ParentId="8989" CreationDate="2013-11-13T03:37:48.200" Score="3" Body="&lt;p&gt;In ParaView there is the Append Attributes filter which can be used for this. It requires that the same number of points are in the data set for appending point data properly and the same number of cells are in the data set for appending cell data properly. It will have problems though with arrays of the same name (i.e. Phi in your example). You can copy that array easily with the Calculator filter though before using the Append Attributes filter. Then you can use another Calculator filter to do the subtraction. This is probably less efficient than using ParaView's Python programmable filter though.&#xA;Besides that, you could use the vtkpython executable to do it manually since you'd have direct access to both grids and their attributes.&lt;/p&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2013-11-13T03:37:48.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10029" PostTypeId="1" CreationDate="2013-11-13T10:28:28.057" Score="2" ViewCount="466" Body="&lt;p&gt;I'm searching for a way to generate a mesh that consists mostly of the same elements and i mean not only the same type, but the exact same size to ease the FEM-calculation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a particular algorithm or software package i should look into?&lt;/p&gt;&#xA;" OwnerUserId="6586" LastActivityDate="2018-05-23T01:21:05.843" Title="Mesh generation with mostly the same elements (type/size..)" Tags="|mesh-generation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10030" PostTypeId="5" CreationDate="2013-11-13T11:08:19.967" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-13T11:08:19.967" LastActivityDate="2013-11-13T11:08:19.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10031" PostTypeId="4" CreationDate="2013-11-13T11:08:19.967" Score="0" Body="CUDA™ is a parallel computing platform and programming model invented by NVIDIA. It enables dramatic increases in computing performance by harnessing the power of the graphics processing unit (GPU)." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2013-11-13T13:54:07.660" LastActivityDate="2013-11-13T13:54:07.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10032" PostTypeId="5" CreationDate="2013-11-13T11:09:21.393" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-13T11:09:21.393" LastActivityDate="2013-11-13T11:09:21.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10033" PostTypeId="4" CreationDate="2013-11-13T11:09:21.393" Score="0" Body="OpenCL™ is the first open, royalty-free standard for cross-platform, parallel programming of modern processors found in personal computers, servers and handheld/embedded devices. OpenCL (Open Computing Language) greatly improves speed and responsiveness for a wide spectrum of applications in numerous market categories from gaming and entertainment to scientific and medical software." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2013-11-13T13:54:16.283" LastActivityDate="2013-11-13T13:54:16.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10034" PostTypeId="2" ParentId="10011" CreationDate="2013-11-13T12:22:46.843" Score="2" Body="&lt;p&gt;C++ does not have a standard library that contains the ability to generate plots (or, in fact, anything graphical). Nor does C++ have a system of packages with a package manager that allows you to import this functionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are cross-platform libraries that facilitate what you want. The most widely used one, also available on Windows, is probably Qt. It will be a little bit more work than in Python to open a window and plot something, but it shouldn't be as difficult as with many of the other libraries you mention.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-13T12:22:46.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10035" PostTypeId="1" CreationDate="2013-11-13T13:49:29.410" Score="7" ViewCount="3882" Body="&lt;p&gt;I am a novice in the world of algorithms, ignorant of the taxonomy used.Please pardon me.&#xA;I have two large sets of numbers A and B where A = {x| 0&amp;lt; x&amp;lt; 9999999999 } B= {y | 0 &amp;lt; y &amp;lt; 9999999999 }. The cardinality of these sets is more than a million. Which is the best algorithm to compute the set differences $A \setminus B$ and $B \setminus A$ ? &#xA;The sets are sorted ( should I say ordered )? &#xA;With my pea sized brain , I arrived only at a comparison of every element of A against every element of B , in a linear fashion. &#xA;Can someone point me to the best way?&#xA;I would also appreciate and be thankful for &quot;go and read this textbook first/pick a good so -and so textbook first &quot; kind of answers.&lt;/p&gt;&#xA;" OwnerUserId="6587" LastEditorUserId="1552" LastEditDate="2013-11-13T15:49:35.540" LastActivityDate="2013-11-14T23:29:35.273" Title="Algorithm to compare two large sets" Tags="|algorithms|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10037" PostTypeId="2" ParentId="10035" CreationDate="2013-11-13T16:55:38.650" Score="4" Body="&lt;p&gt;If you're using C++, there is support in the standard library for this (std::set_difference)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cplusplus.com/reference/algorithm/set_difference/&quot; rel=&quot;nofollow&quot;&gt;http://www.cplusplus.com/reference/algorithm/set_difference/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That documentation even includes equivalent &quot;pseudocode&quot; (really, just more C++) that you can use to port the idea to other languages. The algorithm is close to the &quot;merging&quot; part of mergesort. Note that std::set_difference operates upon sorted ranges, not std::sets (this is a good thing - means sorted std::vector's are adequate).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2013-11-13T17:03:40.530" LastActivityDate="2013-11-13T17:03:40.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10038" PostTypeId="2" ParentId="10035" CreationDate="2013-11-13T21:52:39.110" Score="5" Body="&lt;p&gt;Python sets are open-addressing hashtables with a prime probe.  In other words every set value can be looked up quickly because it was inserted in such a way (hashed) to differentiate it and find it (not order it) from other values.  So operations in python like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = set([1,2,3,4]) &#xA;b = set([3,4,5,6]) #etc..&#xA;a&amp;amp;b&#xA;#gives you {3, 4}&#xA;a|b&#xA;#gives you {1,2,3,4,5,6}&#xA;a^b&#xA;#gives you {1,2,5,6}&#xA;a-b&#xA;#gives you {1, 2}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6590" LastActivityDate="2013-11-13T21:52:39.110" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10040" PostTypeId="1" AcceptedAnswerId="10049" CreationDate="2013-11-14T08:21:49.423" Score="3" ViewCount="530" Body="&lt;p&gt;I have a question relating to linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a fluid solver that solves the poisson equation for pressures. When there are areas of the domain that are entirely enclosed by Neumann conditions, the matrix has a nullspace and we must correct for this. We keep around a mask of the enclosed regions, and in each iteration compute the average value and subtract it off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We've noticed that in this case we can deduce that mask from the matrix itself, rather than from the domain. In that way we have a sort of modified numerical method that operates only on the matrix $A$ and the RHS $b$, but nevertheless converges when $A$ has a nullspace (as opposed to the first paragraph where the inputs to the method are $A$, $b$ &lt;em&gt;and&lt;/em&gt; the enclosed regions mask).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if this generalises a little more, or if anybody has thought about this. For example, is there a modified Jacobi method that always converges, even if the input matrix has a nullspace, and where the only inputs are $A$ and $b$? The solution we are interested in is the one where each Neumann-enclosed region has the smallest average value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested because our domains aren't always so simple; sometimes we don't have Neumann conditions but other conditions on the pressure that nevertheless leave $A$ with a nullspace, and it seems more robust from a software point of view to be calculating the nullspace correction from $A$ itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Normally the answers to a stack overflow question metioning Jacobi are &lt;em&gt;why are you using Jacobi??&lt;/em&gt;! FYI our full solver is a multigrid pre-conditioned conjugate gradient, where we use Jacobi and Gauss-Seidel as smoothers for the multigrid (A. McAdams , E. Sifakis , J. Teran, 2010).)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6595" LastActivityDate="2013-11-15T15:41:16.797" Title="Linear Algebra / Numerical Solution Of Matrix With Nullspace" Tags="|linear-algebra|numerical-analysis|matrix|multigrid|fluid-dynamics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10041" PostTypeId="2" ParentId="8998" CreationDate="2013-11-14T11:50:59.143" Score="8" Body="&lt;p&gt;I would like to add one aspect to the other two answers. Ecosystem aside, functional programming provides a great opportunity for parallel execution such as multithreading or distributed computing. Its inherent immutability properties make it suitable for parallelism, which is generally a real pain in the *bleep* when it comes to imperative languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the improvement in hardware performance in recent years has been focused on adding cores to the processors instead of pushing higher frequencies, parallel computation is getting a lot more popular (I bet you all know this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that Geoff mentions is that developer time is often more important than execution time. I work for a company that builds a computationally intensive SaaS and we did an initial performance test when starting out, pitting C++ vs Java. We found that C++ provided approximately a 50% cut in execution time over Java (this was for computational geometry and the figures will most likely vary depending on the application), but we went with Java anyway because of the importance of developer time and hoped that optimizations and future hardware performance improvements would help us make it on the market. I can say with confidence that had we chosen otherwise, we would not still be in business.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, but Java is not a functional programming language, so what does it have to do with anything, you might ask. Well, later on as we employed more proponents of the functional paradigm and stumbled on the need for paralellization, we progressively migrated parts of our system over to Scala, which combines the positive aspects of functional programming with the power of the imperative and merges well with Java. It has helped us tremendously when increasing the performance of our system with minimal headache and will probably continue to reap benefits from further performance increases in the hardware business when more cores are crammed into the processors of tomorrow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I totally agree with the cons mentioned in the other answers, but I thought that the facilitation of parallel execution is such a powerful pro that it could not go unmentioned.&lt;/p&gt;&#xA;" OwnerUserId="6601" LastActivityDate="2013-11-14T11:50:59.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10042" PostTypeId="2" ParentId="10035" CreationDate="2013-11-14T14:54:47.310" Score="2" Body="&lt;p&gt;One cannot find the set differences with less than a linear search, as an entry appearing anywhere in either set might belong to $A\backslash B$ or to $B\backslash A$ (or neither).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An algorithm which achieves this limit (on sorted lists of data) is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Given input sets A,B, each a strictly ascending list. &#xA;Initialize A\B and B\A as empty lists.&#xA;&#xA;Until A or B is empty, compare the heads of both lists&#xA;    If the heads are equal, remove them.&#xA;    If the head of A preceeds the head of B, remove the head from A&#xA;       and include it in A\B.&#xA;    If the head of B preceeds the head of A, remove the head from B&#xA;       and include it in B\A.&#xA;&#xA;Now A or B is empty.&#xA;Transfer entries left in A to A\B or entries left in B to B\A.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In Prolog, assuming sorted lists &lt;code&gt;A,B&lt;/code&gt; of distinct numeric values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/*  set_differ(A,B,AminusB,BminusA)  */&#xA;set_differ([ ],B,[ ],B) :- !.&#xA;set_differ(A,[ ],A,[ ]) :- !.&#xA;set_differ([H|A],[H|B],AminusB,BminusA) :-&#xA;    !,&#xA;    set_differ(A,B,AminusB,BminusA).&#xA;set_differ([Ha|A],[Hb|B],AminusB,BminusA) :-&#xA;    ( Ha &amp;lt; Hb )&#xA;      -&amp;gt; ( AminusB = [Ha|A_B], set_differ(A,[Hb|B],A_B,BminusA) )&#xA;      ;  ( BminusA = [Hb|B_A], set_differ([Ha|A],B,AminusB,B_A) ) .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With tail-recursion this is memory efficient.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2013-11-14T23:29:35.273" LastActivityDate="2013-11-14T23:29:35.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10044" PostTypeId="1" AcceptedAnswerId="12947" CreationDate="2013-11-14T21:41:54.203" Score="5" ViewCount="785" Body="&lt;p&gt;The question is about the first-order exponential integration method described in this &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_ordinary_differential_equations#The_first-order_exponential_integrator_method&quot;&gt;article&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a system of ordinary differential equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y'(t) = -A\,y(t) + \mathcal{N}(t, y), \qquad y(0) = y_0,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal{N}$ is a nonlinear function. The solution can be approximated using the exponential (explicit) Euler method given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_{n+1} = e^{-Ah}y_n + A^{-1}(1-e^{-Ah}) \mathcal{N}(t_n, \, y(t_n)).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is: What can be said about the stability of the above integration scheme provided that A is a real, symmetric matrix? What if A is also positive semi-definite? I am aware that the ordinary (explicit) Euler scheme is known to be &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler_method#Numerical_stability&quot;&gt;unstable&lt;/a&gt;, but what about its exponential analogue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&#xA;Ivan&lt;/p&gt;&#xA;" OwnerUserId="1815" LastEditorUserId="1815" LastEditDate="2013-11-14T22:03:22.193" LastActivityDate="2014-06-21T06:30:17.390" Title="Stability of the first-order exponential integrator method" Tags="|ode|stability|time-integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10045" PostTypeId="1" AcceptedAnswerId="10050" CreationDate="2013-11-15T05:31:37.023" Score="5" ViewCount="264" Body="&lt;p&gt;The conjugate gradient method suffers from an accumulation of errors as it continues. For this reason it is unwise to use it as a direct solver. My question is, would it help to recalculate the residual every 5 or so steps in an attempt to mitigate these sorts of errors even when used as an iterative solver? I've not heard of people doing this but it seems kind of obvious. If it wouldn't help, can you explain why?&lt;/p&gt;&#xA;" OwnerUserId="4783" LastActivityDate="2015-06-18T13:06:18.830" Title="Would recalculating the residual in the conjugate gradient method help?" Tags="|linear-solver|conjugate-gradient|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10046" PostTypeId="5" CreationDate="2013-11-15T13:43:55.713" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-15T13:43:55.713" LastActivityDate="2013-11-15T13:43:55.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10047" PostTypeId="4" CreationDate="2013-11-15T13:43:55.713" Score="0" Body="The Runge–Kutta methods are a set of numerical methods for ordinary differential equations for the approximation of their solutions." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2013-11-19T15:29:18.183" LastActivityDate="2013-11-19T15:29:18.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10048" PostTypeId="2" ParentId="3220" CreationDate="2013-11-15T14:26:54.887" Score="2" Body="&lt;p&gt;You may check &lt;a href=&quot;https://stats.stackexchange.com/a/11934/5025&quot;&gt;my answer on Cross Validated&lt;/a&gt;. I didn't want to copy it here. Basically, you can use fast, randomized SVD to compute PCA basis and coefficients.&lt;/p&gt;&#xA;" OwnerUserId="201" LastEditorUserId="-1" LastEditDate="2017-04-13T12:44:17.923" LastActivityDate="2013-11-15T14:26:54.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10049" PostTypeId="2" ParentId="10040" CreationDate="2013-11-15T15:41:16.797" Score="4" Body="&lt;p&gt;This is a pretty common problem. There are two simpler ways to what you are doing:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Enforce a Dirichlet-kind condition on one pressure node by fixing it to zero. This way, your matrix no longer has a null space. The pressure may not have mean value zero, but you can fix this once (instead of once per iteration) after you have the solution of the linear system by subtracting the mean value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Many iterative Krylov-space solvers will converge even with a null space if your initial residual has no component in this null space -- in other words, if the pressure component of the initial residual has mean value zero. You then do not need to do anything at all during the iterations. The solution process is not going to be stable &lt;em&gt;within this null space&lt;/em&gt; but since you don't care about it, if you accumulate a small nonzero mean value, you can always subtract it at the end. The difficulty is that while the solver will ignore the null space, you need to have a preconditioner that, when given a vector with mean value zero pressure produces a vector with mean value zero pressure.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-15T15:41:16.797" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10050" PostTypeId="2" ParentId="10045" CreationDate="2013-11-15T15:52:39.157" Score="3" Body="&lt;p&gt;Nobody does this. It seems like everyone who has thought about the issue has come to the conclusion that the error one incurs by not computing the residual exactly is small and that computing it every few iterations would be a waste of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason, I believe, is that even for large systems with hundreds of thousands or hundreds of millions of unknowns, we typically only do a few hundred or thousand iterations at most. This is not enough for errors to significantly accrue due to round-off. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-15T15:52:39.157" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10051" PostTypeId="1" CreationDate="2013-11-15T18:29:35.850" Score="2" ViewCount="142" Body="&lt;p&gt;Having multiple streams of pseudo-random numbers known to be independent and with a uniform distribution I want to do Monte Carlo simulations in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, one thread will have a full-period independent and uniformly distributed stream of pseudo-random numbers. Each thread will consume these numbers in four different functions (a,b,c,d).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My concern is about the distribution across threads for each function. Thread.1 func_a.1, thread.2 func_a.2... and so on. Do I still need to make sure this distribution is indeed uniform across func_a1, func_a2, etc? Failing to do so can make my simulation have flaws?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary,if I start using the pseudo-random numbers in a &quot;random&quot; fashion, rejection sampling. Can I still be sure of the uniform distribution among the different parts?&lt;/p&gt;&#xA;" OwnerUserId="6615" LastEditorUserId="6615" LastEditDate="2013-11-18T03:43:54.757" LastActivityDate="2014-02-16T05:30:17.090" Title="Multiple independent random number streams" Tags="|parallel-computing|probability|cuda|random-number-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10052" PostTypeId="2" ParentId="10051" CreationDate="2013-11-15T20:44:33.493" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;with a uniform distribution&lt;/p&gt;&#xA;&lt;p&gt;uniformly distributed stream&lt;/p&gt;&#xA;&lt;p&gt;distribution is indeed uniform&lt;/p&gt;&#xA;&lt;p&gt;still be sure of the normal distribution&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You seem to be using &amp;quot;independent&amp;quot;, &amp;quot;uniform&amp;quot;, and &amp;quot;normal&amp;quot; interchangeably, but in statistics these each have different technical meanings.  The one that is probably of the most concern in your situation is &amp;quot;independent&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2013-11-15T20:44:33.493" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10053" PostTypeId="2" ParentId="6932" CreationDate="2013-11-15T23:39:07.790" Score="5" Body="&lt;p&gt;I'm late to this discussion, but in fact there &lt;em&gt;is&lt;/em&gt; a  well-posed boundary condition for the pressure in&#xA;the incompressible Navier-Stokes equation, which is (almost) derived from&#xA;the equations themselves.  For fluid motions subject to the no-slip condition &#xA;$\mathbf{u}=0$ on the boundary and with&#xA;body force $\mathbf{f}$,&#xA;the  boundary condition for pressure takes the form&#xA;$$&#xA;\frac{\partial p}{\partial \mathbf{n}} = &#xA;-\nu \mathbf{n}\cdot( \nu \nabla\times\nabla\times \mathbf{u}) +\mathbf{n}\cdot\mathbf{f}.&#xA;$$&#xA;This boundary condition appears in work on spectral element methods&#xA;in the 1990s, and recently has been shown to be well-posed. &#xA;What I mean by this is that (i) it necessarily holds for any&lt;br&gt;&#xA;regular solution of the incompressible Navier-Stokes equations,&#xA;and (ii) the Navier-Stokes equations remain well-posed if one  &lt;em&gt;replaces&lt;/em&gt; &#xA;the divergence-free condition $\nabla\cdot\mathbf{u}=0$&#xA;by the Poisson equation for the pressure with the boundary condition above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that the boundary condition is necessary follows from the vector identity&#xA;$$&#xA;-\nabla\times\nabla\times \mathbf{u} = \Delta\mathbf{u}-\nabla\nabla\cdot\mathbf{u}.&#xA;$$&#xA;Of course $\nabla\cdot\mathbf{u}=0$ for the exact Navier-Stokes solution, but &#xA;that does not mean $\nabla\cdot\mathbf{u}$ can be neglected in the well-posedness theory, or in intermediate steps&#xA;in projection methods.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A deeper reason for the curl-curl term to appear is that its contribution to the pressure&#xA;gradient arises exactly from the failure of the Laplacian to commute with the Helmholtz&#xA;decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See [1] below &#xA;for a thorough discussion of these points, how to implement the pressure boundary condition&#xA;in a finite-element discretization, and a derivation of 3rd-order time-discrete projection&#xA;methods based on the pressure-Poisson equation with this boundary condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] J.-G. Liu, J. Liu, R.L. Pego, Stable and accurate pressure approximation for&#xA;unsteady incompressible viscous flow, J. Comp. Phys. 229 (2010) 3428-3453.&lt;/p&gt;&#xA;" OwnerUserId="6617" LastActivityDate="2013-11-15T23:39:07.790" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10054" PostTypeId="2" ParentId="5029" CreationDate="2013-11-16T00:16:04.533" Score="2" Body="&lt;p&gt;This is a late response, but I want to point out the option (for the time-dependent case)&#xA;of using a &#xA;pressure-approximation type of projection method, based on solving&#xA;a Poisson equation for the pressure with a well-posed boundary condition.&#xA;See [1] for derivations and tests, including tests with equal-order finite elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to admit that Jed Brown's warning still largely applies, however --- the &#xA;schemes proposed and discussed in [1] appear very flexible and are supported by some theory&#xA;and some testing,  but have not been widely tested on many different problems.&#xA;They may prove to be robust, but this has to be proved by experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] J.-G. Liu, J. Liu, R.L. Pego, Stable and accurate pressure approximation for&#xA;unsteady incompressible viscous flow, J. Comp. Phys. 229 (2010) 3428-3453.&lt;/p&gt;&#xA;" OwnerUserId="6617" LastActivityDate="2013-11-16T00:16:04.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10055" PostTypeId="1" CreationDate="2013-11-16T12:08:23.787" Score="0" ViewCount="71" Body="&lt;p&gt;Say, I want to use the UPC code Water Supply Fixture Unit tables in my software that I'm writing ... is it okay or there will be copyright issues?&lt;/p&gt;&#xA;" OwnerUserId="6622" LastActivityDate="2013-11-16T13:10:22.713" Title="Can I use Engineering codes like ASHRAE , NFPA and UPC in my software without permission?" Tags="|software|" AnswerCount="1" CommentCount="1" ClosedDate="2013-12-02T22:10:22.887" ContentLicense="CC BY-SA 3.0" />
  <row Id="10056" PostTypeId="1" CreationDate="2013-11-16T12:50:34.277" Score="2" ViewCount="770" Body="&lt;p&gt;Let's assume that I have a comsol model for solving time-dependent problem. In order to find initial value for one variable I need to solve simple algebraic equation before starting simulation, therefore I added Global ODEs and DAEs module to the model. Now it works good, but it seems like equation from Global ODEs and DAEs module is solved on each time iteration and its obviously redundant calculations, because this equation need to be solved only once.&#xA;Questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Does equation from Global ODEs and DAEs module is solved on each time iterarion or it is only my suggestion? Maybe there is some way how I can check it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If this equation is really solved on each iteration again how can I adjust my solver in order to solve it only once?&lt;/p&gt;&#xA;" OwnerUserId="5278" LastActivityDate="2013-11-16T12:50:34.277" Title="How to solve global equation in Comsol only once?" Tags="|comsol|" AnswerCount="0" CommentCount="1" ClosedDate="2016-10-03T17:37:43.403" ContentLicense="CC BY-SA 3.0" />
  <row Id="10057" PostTypeId="2" ParentId="10055" CreationDate="2013-11-16T13:10:22.713" Score="2" Body="&lt;p&gt;This is a question specific to the software you want to use. You will need to read up on their license description or ask the person/company who wrote the software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will not be able to get a good answer from this forum. In particular, you will not be able to get anything that you can rely on from a legal point of view.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-16T13:10:22.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10058" PostTypeId="1" CreationDate="2013-11-16T17:14:18.050" Score="1" ViewCount="965" Body="&lt;p&gt;I want to simulate capacitor with Comsol by using Poisson's equation in genaral and weak form. How can I specify boundary condition for this case?&lt;/p&gt;&#xA;" OwnerUserId="6624" LastEditorUserId="5278" LastEditDate="2013-11-16T18:28:25.143" LastActivityDate="2013-12-16T19:23:51.330" Title="How can I define Poisson’s equation in general and weak form in comsol" Tags="|comsol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10059" PostTypeId="2" ParentId="10058" CreationDate="2013-11-16T17:27:44.580" Score="1" Body="&lt;p&gt;You can use PDE module and add Dirichlet Boundary Condition (in this section choose the value on the boundary. It may be zero value or another function which you want to use). &#xA;For more details see &lt;a href=&quot;http://www.youtube.com/watch?v=sMJTWa-Z9Ho&quot; rel=&quot;nofollow&quot;&gt;http://www.youtube.com/watch?v=sMJTWa-Z9Ho&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5278" LastActivityDate="2013-11-16T17:27:44.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10060" PostTypeId="2" ParentId="7113" CreationDate="2013-11-16T17:54:03.707" Score="1" Body="&lt;p&gt;An improved discussion of issues related to boundary conditions for pressure and stability of projection methods for the incompressible Navier-Stokes equations is in this paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;J.-G. Liu, J. Liu, R.L. Pego, Stable and accurate pressure approximation for unsteady incompressible viscous flow, J. Comp. Phys. 229 (2010) 3428-3453.&lt;/p&gt;&#xA;" OwnerUserId="6617" LastActivityDate="2013-11-16T17:54:03.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10061" PostTypeId="2" ParentId="8998" CreationDate="2013-11-16T17:57:07.660" Score="4" Body="&lt;p&gt;I think it is interesting to note that the use of functional programming in Computational Science is not new.&#xA;For example, &lt;a href=&quot;http://dl.acm.org/citation.cfm?doid=91556.91612&quot; rel=&quot;nofollow&quot;&gt;this paper from 1990&lt;/a&gt; showed how to improve the performance of numerical programs written in Lisp (possibly the earliest functional programming language) using partial evaluation.  This work was part of a tool chain used in a 1992 paper by G J Sussman (of &lt;a href=&quot;http://mitpress.mit.edu/sicp/&quot; rel=&quot;nofollow&quot;&gt;SICP&lt;/a&gt; fame) and J Wisdom which provided &lt;a href=&quot;http://web.mit.edu/wisdom/www/ss-chaos.pdf&quot; rel=&quot;nofollow&quot;&gt;numerical evidence of the chaotic behavior of the Solar System&lt;/a&gt;.&#xA;More details about the hardware and software involved in that computation &lt;a href=&quot;ftp://publications.ai.mit.edu/ai-publications/pdf/AIM-1249.pdf&quot; rel=&quot;nofollow&quot;&gt;can be found here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2013-12-13T21:00:08.080" LastActivityDate="2013-12-13T21:00:08.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10062" PostTypeId="1" CreationDate="2013-11-16T11:12:50.690" Score="3" ViewCount="236" Body="&lt;p&gt;I have a set of equations to integrate something in time $t$. At each time step I compute a scalar field $\phi(t)$ and a potential $V(\phi)$. I should also control the conservation of energy with an equation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E=\frac{1}{2}\left(\dot\phi^2+V(\phi)\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or numerically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E=\frac{1}{2}\left(\left(\frac{\phi_{n}-\phi_{n-1}}{t_{n}-t_{n-1}}\right)^2+V(\phi)\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is it $V(\phi_{n})$ or $V(\phi_{n-1})$ ?&lt;/p&gt;&#xA;" OwnerUserId="6627" OwnerDisplayName="Vincent" LastActivityDate="2014-02-23T20:07:02.880" Title="Numerical scheme with energy conservation?" Tags="|computational-physics|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10063" PostTypeId="2" ParentId="10062" CreationDate="2013-11-16T12:28:00.170" Score="-1" Body="&lt;p&gt;The definition of the derivative is &#xA;$$\dot{\phi}_n=\lim_{\delta t\rightarrow 0}\frac{\phi_{n+1}-\phi_n}{\delta t}$$&#xA;where $\delta t$ is $t_n-t_{n-1}$. Therefore, the derivative as you are taking it is calculated for the $n-1$-th step. Hence, I'd say you should go for $V(\phi_{n-1})$.&lt;/p&gt;&#xA;" OwnerDisplayName="Danu" LastActivityDate="2013-11-16T12:28:00.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10064" PostTypeId="2" ParentId="10062" CreationDate="2013-11-16T14:50:22.867" Score="2" Body="&lt;p&gt;It depends on &lt;em&gt;how&lt;/em&gt; you want to solve it: via &lt;a href=&quot;http://en.wikipedia.org/wiki/Explicit_and_implicit_methods&quot; rel=&quot;nofollow&quot;&gt;implicit or explicit methods&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explicit method is certainly the easiest to code but constrains $dt$ for stability; the implicit method is a bit more difficult to code but has no such constraints on $dt$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we consider a simple advection equation&#xA;$$\frac{\partial w}{\partial t}+u\frac{\partial w}{\partial x}=0$$&#xA;Then the explicit method solves this as (using your notation, I think most people do $n\to n+1$, rather than $n-1\to n$)&#xA;$$&#xA;\frac{w^{n}_j+w^{n-1}_j}{dt}=u\left[\frac{w_{j+1}^{n-1}-w_{j-1}^{n-1}}{2\cdot dx}\right]&#xA;$$&#xA;which is more commonly written as&#xA;$$&#xA;w^n_j=w^{n-1}_j+\chi u\left[w^{n-1}_{j+1}-w_{j-1}^{n-1}\right]&#xA;$$&#xA;where $\chi=dt/2dx$. The implicit method solves this as&#xA;$$&#xA;\frac{w^{n}_j+w^{n-1}_j}{dt}=u\left[\frac{w_{j+1}^{n}-w_{j-1}^{n}}{2\cdot dx}\right]&#xA;$$&#xA;which is more commonly written as&#xA;$$&#xA;-\chi uw_{j+1}^n+w^n_j-\chi uw^n_{j-1}=-w_j^{n-1}&#xA;$$&#xA;which requires solving a &lt;a href=&quot;http://en.wikipedia.org/wiki/Tridiagonal_matrix&quot; rel=&quot;nofollow&quot;&gt;tri-diagonal matrix&lt;/a&gt; (fortunately there is a relatively &lt;a href=&quot;http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;nofollow&quot;&gt;simple algorithm&lt;/a&gt; to solve this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I generally would recommend &lt;em&gt;trying&lt;/em&gt; the &lt;strong&gt;explicit&lt;/strong&gt; method first and seeing if the time-stepping is good enough to be useful; if it's not, then it's time to use the &lt;strong&gt;implicit&lt;/strong&gt; method.&lt;/p&gt;&#xA;" OwnerUserId="4762" OwnerDisplayName="Kyle Kanos" LastActivityDate="2013-11-16T14:50:22.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10065" PostTypeId="1" CreationDate="2013-11-16T19:33:10.000" Score="0" ViewCount="914" Body="&lt;p&gt;Is there an efficient storage format for general, non-symmetric sparse matrices for which one can find all non-zero entries in a given row &lt;em&gt;or&lt;/em&gt; column in $O(d)$ time? ($d$ is the max number of non-zero entries in any row or column.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I store an $m \times n$ sparse matrix $A$ in the compressed row or ellpack formats, I can find all non-zero entries in a given row in $O(d)$ time, but finding all non-zero entries in a given column takes $O(m)$. Of course, I could store $A$ in compressed column format, in which case the column accesses would take $O(d)$ but the row accesses would take $O(n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A blunt approach would be to store both formats, compressed row and column, for the same matrix, which of course entails double the memory. Can one do much better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best I can come up with: For square matrices, one can store it as a structurally symmetric matrix. If you want to find all entries in column j, you instead find all entries in row j and then remove the redundant ones.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2013-11-16T19:43:37.560" LastActivityDate="2013-11-17T03:57:18.367" Title="sparse matrix format with fast row and column access" Tags="|linear-algebra|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10066" PostTypeId="1" AcceptedAnswerId="10071" CreationDate="2013-11-17T02:51:28.627" Score="1" ViewCount="63" Body="&lt;p&gt;I am working on a model of social interactions in mice. I have mice and boxes and a simulation that outputs which mouse stays in which box during which time period (this comes out as a mouse is selected randomly to change a box). The problem is how to obtain, in the end and from this, the meetings of two mice which were in the same box in an overlapping period. I need this to look at the CDF of meeting durations, build a social network for each day, etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I have a MySQL database where the simulation directly inserts each stay result. And then another tool,written in Scala, just retrieves all stay result, in portions of a few hundred, loops through them and for each asks the database which stays were overlapping with it, and inserts each pair into the database, like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;box&lt;/code&gt;,&lt;code&gt;id1&lt;/code&gt;, &lt;code&gt;res_id1&lt;/code&gt;, &lt;code&gt;id2&lt;/code&gt;, &lt;code&gt;res_id2&lt;/code&gt;, &lt;code&gt;from&lt;/code&gt;, &lt;code&gt;to&lt;/code&gt;, &lt;code&gt;dt&lt;/code&gt;, &lt;code&gt;typ&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that mice id1 and id2 were in the box &quot;box&quot; in the interval between &quot;from&quot; and &quot;to&quot; with duration &quot;dt&quot; and the meetings was of type &quot;typ&quot;. There could be four types of meetings depending on when each mouse was in the box (e.g. when the one entered and exited relative to the other). &quot;res_id1&quot; and &quot;res_id2&quot; tell which stay results were used to generate the meeting result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, this is very inefficient. What would be a better way of doing it? I am not constrained to using a RDMS, but I though this would be the easiest as I am reading and further analyzing the data in R afterwards. Would it make sense to output the stays in a text file and then use Hadoop to generate the meetings somehow? Or anything else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the duration of an approximately one quarter of one trial of the simulation I generate around 1.5 million stay results.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2013-11-17T16:29:34.017" Title="Searching in a large dataset" Tags="|data-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10067" PostTypeId="2" ParentId="10065" CreationDate="2013-11-17T03:57:18.367" Score="1" Body="&lt;p&gt;You can store the matrix as usual, in row-compressed sparse format. Then for each column, you store which entries exist in the sparsity pattern. If you need the entries of a column, you can find out from this second data structures which entries exist, and you can then get the value from the row-wise (normal) data structure that stores the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is essentially one integer per entry of the matrix for the column-wise storage (in addition to one integer per entry for the row-wise storage plus one floating point number for each entry). With 32-bit integers for the column or row entries, and 64-bit double precision for the numbers, this gives:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;12 bytes per entry for the regular row-wise data storage scheme&lt;/li&gt;&#xA;&lt;li&gt;16 bytes per entry if you also want to keep the column-wise information.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Accessing all entries in a column requires traversing a column in the column-wise data structure, which is ${\cal O}(d)$, and for each entry you find, you need to look it up in the row-wise format to get at the value, which will cost you ${\cal O}(\log d)$ for each entry. In total, that makes ${\cal O}(d \log d)$ to access all elements of a column.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-17T03:57:18.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10068" PostTypeId="1" CreationDate="2013-11-17T07:56:22.943" Score="13" ViewCount="2164" Body="&lt;p&gt;I am interested in globally maximizing a function of many ($\approx 30$) real parameters (a result of a complex simulation).  However, the function in question is relatively expensive to evaluate, requiring about 2 days for each parameter set.  I am comparing different options, and was wondering if anyone had suggestions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there is a suite of methods for this sort of process that involve developing approximate functions and then maximizing those (e.g. &lt;a href=&quot;http://www.ressources-actuarielles.net/EXT/ISFA/1226.nsf/8d48b7680058e977c1256d65003ecbb5/f84f7ac703bf5862c12576d8002f5259/%24FILE/Jones98.pdf&quot;&gt;Jones et al. &quot;Efficient Global Optimization of Expensive Black-Box Functions&quot;&lt;/a&gt;).  However, this seems to be relatively involved to code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have the ability to run a large number of simulations in parallel (50+).  This seemed to suggest using something like genetic algorithms to do this optimization - since I can create a population of candidate solutions just as quickly as I can make one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are my questions:&#xA;1) Does anyone have experiences with freely available implementations of this sort of global solvers / recommendations?&#xA;2) Are there reasons to either prefer or avoid genetic algorithms here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a physical problem, and my early experiments have shown the figure of merit changes fairly smoothly as I change the parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for the help!  A few more details: I do not need any information beyond the location of the maximum.  The simulation is deterministic, not Monte Carlo, so that complication isn't a big deal.  There are no explicit bounds or constraints on the parameters.  One other piece of information I have (and didn't mention before) is a sense of the size of the maximum required.  While I am looking for a global maximum, I would also be happy with anything of this scale or larger - I don't know if this would provide any help.  Hopefully if I do the screening more systematically (Latin hypercubes as suggested by Brian Borchers), this will show up.  &lt;/p&gt;&#xA;" OwnerUserId="4494" LastEditorUserId="4494" LastEditDate="2013-11-18T05:37:19.473" LastActivityDate="2016-05-18T23:28:38.360" Title="Global maximization of expensive objective function" Tags="|optimization|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10069" PostTypeId="2" ParentId="10068" CreationDate="2013-11-17T09:44:26.590" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't have any experience with these sorts of solvers; some of my co-workers have used them. &lt;a href=&quot;http://dakota.sandia.gov/&quot; rel=&quot;noreferrer&quot;&gt;DAKOTA&lt;/a&gt; seems to be the software package recommended for these sorts of tasks. It includes an interface that allows a user to repeatedly submit jobs to a submission queue and use the output for parameter studies, sensitivity analysis, etc. I'm not familiar enough with it to know whether or not it will take advantage of running many simulations simultaneously.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Assuming that your parameters are continuous, if the figure of merit changes smoothly as parameters change, then a surrogate model should do a reasonable job of fitting the figure of merit, and surrogate derivative information should be helpful for refining convergence. For 30 parameters, deterministic derivative-free optimization methods should also be useful; there again, smoothness should help. In contrast, genetic algorithms won't use derivative information at all, and often require tuning of parameters like mutation rate, recombination rate, and selection parameters in order to achieve good performance. As an algorithmic choice, I'd use genetic algorithms as a fallback, because I'd expect a well-designed surrogate optimization or a deterministic derivative-free optimization method to have better convergence behavior.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-11-17T09:44:26.590" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10070" PostTypeId="2" ParentId="10068" CreationDate="2013-11-17T15:38:02.143" Score="12" Body="&lt;p&gt;Genetic algorithms are a very poor choice when the objective function is extremely expensive to evaluate- these methods require a lot of function evaluations in each generation (which parallelism can help with) and a lot of generations (which are inherently sequential.)  At two days per generation, this would be very slow.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't mentioned where this problem came from.  Are you statistically analyzing a likelihood surface (in which case you'll want more than just the optimal parameters and objective value) or just optimizing an objective function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't mentioned whether the objective function calculation is precise or inaccurate.  It's often the case that when the objective function is calculated by Monte Carlo simulation, the values are quite noisy.  This can mislead many optimization algorithms.  Response surface methods help with this problem by smoothing out the noise.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't mentioned any constraints on the parameters.  Are they bounded?  Are there linear or nonlinear constraints between the parameters?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chances are that most of your 30 parameters aren't really that important to the problem.  I would suggest using an experimental design screening approach to first determine which of the 30 parameters are really important in the optimization, and then after setting reasonable values for the unimportant parameters optimize over the important parameters.  Methods like Latin Hypercube Sampling can be very helpful in screening out the relatively unimportant parameters.  In this screening stage you can easily make use of hundreds of processors.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reducing the number of parameters to a more reasonable size, I'd use a response surface method to optimize over the remaining parameters.  If the response surface really is multi-modal, and you use an overly simple response surface model (typically folks just fit a quadratic model) then you could easily be mislead and miss out on the global maximum.   Be careful!  In this stage you can again make use of lots of processors by using an experimental design that gives very good coverage of the parameter space.  Look for design points where the fitted model is far off from the computed values- this is an indication that the response surface isn't working well in that region.  You may have to build response surfaces in separate regions of the parameter space.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a last step, you can start with the parameters from your response surface optimization and try to improve on the values of the screened out parameters by adjusting them one at a time (coordinate descent.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll second the recommendation of DAKOTA as a framework for this kind of optimization.  If you're going to be doing this optimization only once then it might be easier to organize the computations by hand, but if you're going to do it repeatedly, then DAKOTA would be very helpful.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2013-11-17T16:03:06.423" LastActivityDate="2013-11-17T16:03:06.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10071" PostTypeId="2" ParentId="10066" CreationDate="2013-11-17T16:29:34.017" Score="2" Body="&lt;p&gt;You need to think &quot;out of the box&quot; (pun intended). Because you have a database of stays in the boxes, you think of your algorithm of looping over all stays. But to find overlaps, you need to compare each stay with each other stay -- so your algorithm has ${\cal O}(N_s^2)$ complexity where $N_s$ is the number of stays. This is expensive, as you have noticed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So turn it around. Instead of a loop over all stays, have a loop over all boxes and do this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For each box, extract those stay events that are associated with this box into a list or subset (complexity: ${\cal O}(N_s)$).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sort this subset by start time (complexity: ${\cal O}(N_{s,b} \log N_{s,b})$ where $N_{s,b}$ is the number of stays in box $b$). If your database was filled in ascending start times of stays, this step is not even necessary.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Within this subset, loop over all stays and compare whether the end time of one event is later than the start time of the next event. This indicates an overlap. (Complexity: ${\cal O}(N_{s,b})$.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What I describe is a pretty common strategy: If a loop over a collection of data is inefficient, think whether there is a different way of looking at the data set and loop over something else instead.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-17T16:29:34.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10072" PostTypeId="1" AcceptedAnswerId="10073" CreationDate="2013-11-17T20:40:19.090" Score="3" ViewCount="2921" Body="&lt;p&gt;I have a sparse $5\,656\,236 * 5\,656\,236$ matrix $A$ with $166\,526\,888$ non-zero elements. The matrix comes from using the finite element method on a linear elasticity problem and is positive semi-definite. I'm trying to use the preconditioned conjugate gradient method to solve it, particularly, the pcg() function in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I expect this to converge at all? I tried&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;L = ichol(A);&#xA;[u,flag,res,iter,resvec] = pcg(A,F,1e-6,max_ter,L,L');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it does not seem to converge at all. The residual starts by increasing a lot and after 2000 iterations (which takes some hours) the relative residual is up at $10^4$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I guess my question is, how would you attack a system of this size? I'm sure there are a lot of ways to lure the method into converging faster. Any hints and tips are welcome!&lt;/p&gt;&#xA;" OwnerUserId="1333" OwnerDisplayName="burk" LastEditorUserId="1333" LastEditDate="2013-11-18T08:14:31.160" LastActivityDate="2013-11-19T02:17:05.943" Title="Matlab preconditioned conjugate gradient on big matrix" Tags="|matrix|matlab|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10073" PostTypeId="2" ParentId="10072" CreationDate="2013-11-18T10:51:47.237" Score="7" Body="&lt;p&gt;The MATLAB function &lt;code&gt;ichol&lt;/code&gt; computes by default the zero fill-in variant of the incomplete Cholesky factorisation. You could try to allow more fill-in (and thus try to improve the preconditioner) by using a drop tolerance, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;L = ichol(A,struct('type','ict','droptol',1e-03,'michol','on'));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also note, that single level preconditioners such as ILU (or ICHOL), sparse inverses, simple splittings etc. do not scale well for large problem sizes. So if the above won't work, you could try to use algebraic multigrid (AMG). There are AMG packages which are well-suited for solving linear elasticity, e.g., BoomerAMG from &lt;a href=&quot;http://computation.llnl.gov/casc/hypre/&quot;&gt;Hypre&lt;/a&gt; or ML in &lt;a href=&quot;https://trilinos.sandia.gov&quot;&gt;Trilinos&lt;/a&gt; (as far as I know, ML has also a Matlab interface). If you don't like C/C++, you might consider trying &lt;a href=&quot;http://pyamg.org&quot;&gt;PyAMG&lt;/a&gt; which is in Python (build upon NumPy and SciPy).&lt;/p&gt;&#xA;" OwnerUserId="5342" LastActivityDate="2013-11-18T10:51:47.237" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10074" PostTypeId="2" ParentId="8532" CreationDate="2013-11-19T00:49:05.003" Score="1" Body="&lt;p&gt;You might take a look at Bayesian Clustering. This seems like a good fit because you can start with exaclty &lt;code&gt;N&lt;/code&gt; clusters, and the ones that are not needed will drop out as your solution converges.  If your data is already well separated, I expect that this method would converge rather quickly.  For an example of this, see [1] chapter 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Christopher M. Bishop, &lt;a href=&quot;http://amzn.com/0387310738&quot; rel=&quot;nofollow&quot;&gt;Pattern Recognition and Machine Learning&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6633" LastActivityDate="2013-11-19T00:49:05.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10075" PostTypeId="2" ParentId="10072" CreationDate="2013-11-19T02:17:05.943" Score="1" Body="&lt;p&gt;If your residual increases, then you probably have a matrix that is either not symmetric or at least not positive definite. In those cases, CG will not work, with or without preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two ways to find out:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Try a smaller system that results from the same program but using a much coarser mesh and see if CG converges for that.&lt;/li&gt;&#xA;&lt;li&gt;Inspect your code to see if it really does what it's supposed to do when assembling the matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-19T02:17:05.943" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10078" PostTypeId="1" CreationDate="2013-11-19T12:42:33.497" Score="6" ViewCount="5706" Body="&lt;p&gt;I'm trying to write an efficient bilinear (2D)-interpolation, after reading some recipes, as a fortran-mex for Matlab that is used extensively throughout a long algorithm of solar image processing, and therefore is one of my main bottlenecks. My 1st attempt work but is slow. See code below. &#xA;Any advice that can speed up the code is most welcome. &#xA;I'm using gcc version 4.3.6. &#xA;I cannot use a later one as it the only one fully compatible with my version of Matlab. Any later version prevented the use of some embedded matlab-mex functions necessary for interfacing / displaying error messages in the matlab prompts. &#xA;My mex-setup can compile Fortran 90 and 95 (so far...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have a C++ mex equivalent used so far and was hoping that a well-written Fortran code would be faster, i'm using the C++ as reference for the speedup that Fortran can or cannot offer.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SUBROUTINE L2DINTERPOL3(IntIm,Image,x,y,NPts,M,N)&#xA;&#xA;  INTEGER*8        :: NPts,M,N,jj&#xA;  INTEGER*8        :: x1(NPts),y1(NPts),x2(NPts),y2(NPts)&#xA;  DOUBLE PRECISION :: IntIm(NPts)&#xA;  DOUBLE PRECISION :: Image(M,N)&#xA;  DOUBLE PRECISION :: f1,f2&#xA;  DOUBLE PRECISION :: x(NPts),y(NPts)&#xA;  DOUBLE PRECISION :: wx1(NPts),wx2(NPts),wy1(NPts),wy2(NPts)&#xA;&#xA;c Take the nearest integers around the input. &#xA;&#xA;   x1 = FLOOR(x)&#xA;   x2 = CEILING(x)&#xA;   y1 = FLOOR(y)&#xA;   y2 = CEILING(y)&#xA;&#xA;   wx1 = (x2-x)/(x2-x1)&#xA;   wx2 = (x-x1)/(x2-x1)&#xA;   wy1 = (y2-y)/(y2-y1)&#xA;   wy2 = (y-y1)/(y2-y1)&#xA;&#xA;c Whenever the input are already integers, weights go to infinity,&#xA;c so set each pair to 1 and 0.&#xA;&#xA;   WHERE(x1 .EQ. x2)&#xA;      wx1 = 1.&#xA;      wx2 = 0.&#xA;   END WHERE&#xA;&#xA;   WHERE (y1 .EQ. y2)&#xA;      wy1 = 1.&#xA;      wy2 = 0.&#xA;   END WHERE&#xA;&#xA;c Main calculation, loop over each element of the input. &#xA;&#xA;  DO 10 jj=1,NPts&#xA;&#xA;&#xA;     f1 = wx1(jj)*Image(y1(jj),x1(jj)) + wx2(jj)*Image(y1(jj),x2(jj))&#xA;     f2 = wx1(jj)*Image(y2(jj),x1(jj)) + wx2(jj)*Image(y2(jj),x2(jj))&#xA;&#xA;     IntIm(jj) = wy1(jj)*f1 + wy2(jj)*f2&#xA;&#xA;&#xA; 10   CONTINUE&#xA;&#xA;  RETURN&#xA;  END&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;[EDIT] As i only have regular grid, x2-x1 = 1 or 0, i've made another version that does not require division. This got rid of the where constructs. In addition, it seems that replace the DO loop with a &quot;FORALL&quot; loop improved by about 15% on 2 cpus. My understanding is that FORALL uses multithread when present, and each computation at each jj is independent from the other at jj+k or jj-k which makes the FORALL statement sensible. Doesn't it ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SUBROUTINE L2DINTERPOL(IntIm,Image,x,y,NPts,M,N)&#xA;  implicit none&#xA;&#xA;  mwSize, PARAMETER             :: dp = kind(0.d0) ! Double precision&#xA;  mwSize                        :: NPts, M,N       ! Input&#xA;  REAL(dp),DIMENSION(Npts)      :: x,y             ! Input&#xA;  REAL(dp),DIMENSION(M,N)       :: Image           ! Input&#xA;  mwSize                        :: jj&#xA;  mwSize, DIMENSION(NPts)       :: x1,y1,x2,y2      &#xA;  REAL(dp),DIMENSION(Npts)      :: wx1,wx2,wy1,wy2&#xA;  REAL(dp),DIMENSION(Npts)      :: IntIm           ! Output&#xA;&#xA;&#xA;  x1 = FLOOR(x)&#xA;  x2 = CEILING(x)&#xA;  y1 = FLOOR(y)&#xA;  y2 = CEILING(y)&#xA;&#xA;  wx1 = x2-x&#xA;  wy1 = y2-y&#xA;&#xA;  wx2 = 1 - wx1&#xA;  wy2 = 1 - wy1&#xA;&#xA;  FORALL( jj=1 : NPts)&#xA;&#xA;     IntIm(jj) = wy1(jj)*(wx1(jj)*Image(y1(jj),x1(jj))+wx2(jj)*&#xA; $        Image(y1(jj),x2(jj))) + wy2(jj)*(wx1(jj)*&#xA;     $        Image(y2(jj),x1(jj))+wx2(jj)*Image(y2(jj),x2(jj)))&#xA;&#xA;  END FORALL&#xA;&#xA;&#xA;  RETURN&#xA;  END &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6640" LastEditorUserId="6640" LastEditDate="2013-12-03T09:32:54.230" LastActivityDate="2016-10-28T17:30:02.803" Title="Fortran, making a more efficient bilinear interpolation" Tags="|fortran|interpolation|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="10079" PostTypeId="5" CreationDate="2013-11-19T13:09:01.897" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T13:09:01.897" LastActivityDate="2013-11-19T13:09:01.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10080" PostTypeId="4" CreationDate="2013-11-19T13:09:01.897" Score="0" Body="Cluster computing is a form of distributed computing in which a group of computers are linked together so that they can act like a single entity." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2013-11-20T03:34:25.397" LastActivityDate="2013-11-20T03:34:25.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10081" PostTypeId="5" CreationDate="2013-11-19T13:11:29.673" Score="0" Body="&lt;p&gt;Matrix is a rectangular array of elements (e.q. numbers, symbols, or expressions), arranged in columns and rows. They are typically used to represent linear transformations in a particular basis.&lt;/p&gt;&#xA;" OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-15T16:13:14.120" LastActivityDate="2021-11-15T16:13:14.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10082" PostTypeId="4" CreationDate="2013-11-19T13:11:29.673" Score="0" Body="For questions about using and representing matrices on a computer in order to solve computational problems. Should generally also include a tag about the specific property/problem you are solving (e.g. [tag:linear-algebra], [tag:eigenvalues], [tag:inverse]." OwnerUserId="6585" LastEditorUserId="23791" LastEditDate="2021-11-15T16:13:08.117" LastActivityDate="2021-11-15T16:13:08.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10083" PostTypeId="5" CreationDate="2013-11-19T13:16:36.167" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T13:16:36.167" LastActivityDate="2013-11-19T13:16:36.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10084" PostTypeId="4" CreationDate="2013-11-19T13:16:36.167" Score="0" Body="Geometry is a branch of mathematics. Geometry studies the spatial relationships and forms of objects, as well as other relationships and forms, similar to the spatial in its structure." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2013-11-20T01:09:13.050" LastActivityDate="2013-11-20T01:09:13.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10085" PostTypeId="5" CreationDate="2013-11-19T13:42:34.127" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T13:42:34.127" LastActivityDate="2013-11-19T13:42:34.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10086" PostTypeId="4" CreationDate="2013-11-19T13:42:34.127" Score="0" Body="Singular Value Decomposition (SVD) is a decomposition (factorization) of rectangular real or complex matrix into the product of a unitary rotation matrix, a diagonal scaling matrix, and a second unitary rotation matrix." OwnerUserId="6585" LastEditorUserId="276" LastEditDate="2013-11-20T01:11:50.027" LastActivityDate="2013-11-20T01:11:50.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10087" PostTypeId="5" CreationDate="2013-11-19T13:44:50.027" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T13:44:50.027" LastActivityDate="2013-11-19T13:44:50.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10088" PostTypeId="4" CreationDate="2013-11-19T13:44:50.027" Score="0" Body="SciPy is a Python-based ecosystem of open-source software for mathematics, science, and engineering." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2013-11-19T14:28:53.033" LastActivityDate="2013-11-19T14:28:53.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10089" PostTypeId="5" CreationDate="2013-11-19T13:48:44.453" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T13:48:44.453" LastActivityDate="2013-11-19T13:48:44.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10090" PostTypeId="4" CreationDate="2013-11-19T13:48:44.453" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T13:48:44.453" LastActivityDate="2013-11-19T13:48:44.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10091" PostTypeId="5" CreationDate="2013-11-19T13:56:53.620" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T13:56:53.620" LastActivityDate="2013-11-19T13:56:53.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10092" PostTypeId="4" CreationDate="2013-11-19T13:56:53.620" Score="0" Body="Combinatorics (combinatorial analysis) is a branch of mathematics that studies the discrete objects, the set (a combination, permutation, deployment and transfer of elements) and the relationship to them (eg, partial order)." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2013-11-20T01:11:53.823" LastActivityDate="2013-11-20T01:11:53.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10093" PostTypeId="1" AcceptedAnswerId="10112" CreationDate="2013-11-19T13:57:15.363" Score="6" ViewCount="391" Body="&lt;p&gt;In my last question I noticed a peculiar error when solving the Poisson equation using finite volume method (FVM), &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8577/peculiar-error-when-solving-the-poisson-equation-on-a-non-uniform-mesh-1d-only&quot;&gt;Peculiar error when solving the Poisson equation on a non-uniform mesh (1D only) finite volume method&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is actually only one equation in coupled system of non-linear equations. The other equations in the system are of advection-reaction-diffusion type which I can solved robustly using a FVM scheme (&lt;a href=&quot;http://danieljfarrell.github.io/FVM/advection_diffusion.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://danieljfarrell.github.io/FVM/advection_diffusion.html&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be feasible to use a mixed approach in this case? For example, using a finite difference scheme to solve the Poisson equation and sticking with the FVM for the advection-diffusion equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines&quot; rel=&quot;nofollow noreferrer&quot;&gt;method of lines (MOL)&lt;/a&gt; when solving these equations, so I only need supply a vector which represents the spatial discretisation of each variable. I was hoping that if I use a finite difference method for the Poisson equation I could then interpolate the resulting vector on to the finite volume mesh afterwards?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might sound like a hack, but wanted to get your opinion before trying it. Maybe moving to a finite element approach is needed?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-11-21T03:08:56.093" Title="Is is possible to mix finite-volume and finite-difference discretisations when solving a coupled non-linear system?" Tags="|finite-element|finite-difference|finite-volume|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10094" PostTypeId="1" AcceptedAnswerId="10118" CreationDate="2013-11-19T13:59:07.200" Score="24" ViewCount="5514" Body="&lt;p&gt;There are two general approaches to representing solutions in the discontinuous galerkin method: nodal and modal.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Modal&lt;/strong&gt;: Solutions are represented by sums of modal coefficients multiplied by a set of polynomials, e.g. $u(x,t) = \sum_{i=1}^N u_i(t) \phi_i(x)$ where $\phi_i$ is usually orthogonal polynomials, e.g. Legendre. One advantage of this is that the orthogonal polynomials generate a diagonal mass matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Nodal&lt;/strong&gt;: Cells are comprised of multiple nodes on which the solution is defined. Reconstruction of the cell is then based on fitting an interpolating polynomial, e.g. $u(x,t) = \sum_{i=1}^N u_i(x,t) l_i(x)$ where $l_i$ is a Lagrange polynomial. One advantage of this is that you can position your nodes at quadrature points and quickly evaluate integrals.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the context of a large-scale, complex ($10^6$-$10^9$ DOFs) 3D mixed structured/unstructured parallel application with goals of flexibility, clarity of implementation, and efficiency, &lt;strong&gt;what are the comparative advantages and disadvantages of each method?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure there's good literature already out there, so if someone could point me to something that'd be great as well.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2019-04-20T16:51:46.110" Title="Discontinuous Galerkin: Nodal vs Modal advantages and disadvantages" Tags="|fluid-dynamics|discontinuous-galerkin|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10095" PostTypeId="5" CreationDate="2013-11-19T14:07:07.547" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T14:07:07.547" LastActivityDate="2013-11-19T14:07:07.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10096" PostTypeId="4" CreationDate="2013-11-19T14:07:07.547" Score="0" Body="For questions about determining the error caused by specific computational procedures, approximations, or numerical representations." OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-17T14:38:38.377" LastActivityDate="2021-11-17T14:38:38.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10097" PostTypeId="5" CreationDate="2013-11-19T14:13:02.090" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T14:13:02.090" LastActivityDate="2013-11-19T14:13:02.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10098" PostTypeId="4" CreationDate="2013-11-19T14:13:02.090" Score="0" Body="Eigenvalues are a special set of scalars associated with a linear system of equations (i.e., a matrix equation) that are sometimes also known as characteristic roots, characteristic values (Hoffman and Kunze 1971), proper values, or latent roots (Marcus and Minc 1988, p. 144)." OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2016-07-14T13:48:38.267" LastActivityDate="2016-07-14T13:48:38.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10099" PostTypeId="5" CreationDate="2013-11-19T14:21:50.807" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T14:21:50.807" LastActivityDate="2013-11-19T14:21:50.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10100" PostTypeId="4" CreationDate="2013-11-19T14:21:50.807" Score="0" Body="For questions about AMR, a technique for dynamically updating the precision of a grid to ensure a certain accuracy for a given region." OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-17T17:25:03.140" LastActivityDate="2021-11-17T17:25:03.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10101" PostTypeId="5" CreationDate="2013-11-19T14:24:46.557" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2013-11-19T14:24:46.557" LastActivityDate="2013-11-19T14:24:46.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10102" PostTypeId="4" CreationDate="2013-11-19T14:24:46.557" Score="0" Body="For computation questions involving computing probabilities or simulating probabilistic processes." OwnerUserId="23791" LastEditorUserId="23791" LastEditDate="2021-11-20T17:15:22.293" LastActivityDate="2021-11-20T17:15:22.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10103" PostTypeId="1" CreationDate="2013-11-19T17:05:50.683" Score="3" ViewCount="367" Body="&lt;p&gt;I'm working on translating a discontinuous Galerkin code from MATLAB to C and I'm at the final point where I need to solve a sparse system. I've taken a course in C before but I'm very rusty and wasn't all that good to begin with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used LAPACK before but was wondering if there are better options out there. My professor uses PetSc but I was looking at the documentation and have no idea how to install it let alone use it. The &quot;tutorial&quot; section on the ANL site is written at a pretty high level already. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for sparse solvers in C, is there a &quot;special way&quot; that the array has to be stored? Or can I just input my arrays as is into the solver and get the solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a plot of the matrix structure&#xA;&lt;img src=&quot;https://i.stack.imgur.com/f73re.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2013-11-19T18:18:14.870" LastActivityDate="2013-11-19T18:24:53.733" Title="Sparse linear solvers in C?" Tags="|petsc|linear-solver|c|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10104" PostTypeId="2" ParentId="8018" CreationDate="2013-11-19T18:03:17.473" Score="2" Body="&lt;p&gt;If you choose to do it one orbital at a time, the second orbital needs to be solved for the lowest energy solution &lt;strong&gt;in the space orthogonal to the first orbital&lt;/strong&gt; and for &lt;strong&gt;the same Fock operator&lt;/strong&gt;. You need to solve &lt;em&gt;ALL&lt;/em&gt; the orbitals before you update your Fock operator.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You were basically solving the same orbital over and over again, and took the change of that orbital between iterations to be the second orbital.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of doing it one orbital at a time, it is actually easier to solve all orbitals at the same time. Basically you are solving $ \mathbf{F}\mathbf{C}=\mathbf{S}\mathbf{C}\mathbf{\epsilon} $ every iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard way is to perform decomposition   $ \mathbf{S}=\mathbf{U}^†\mathbf{\Lambda}\mathbf{U}  $ and use $\mathbf{S}^{1/2}=\mathbf{U}^†\mathbf{\Lambda}^{1/2}\mathbf{U} $  to transform your basis and Fock operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way you get a more standard eigenvalue problem $ \mathbf{\tilde{F}}\mathbf{\tilde{C}}=\mathbf{\tilde{C}}\mathbf{\epsilon} $ and you can simply pull out your favorite LAPACK implementation and get the orbitals fairly quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hartree-Fock procedure might oscillate or divergence.   However, it really shouldn't do so for beryllium. The HOMO LUMO gap is fairly large and you will only need s orbitals in your basis.&lt;/p&gt;&#xA;" OwnerUserId="6644" LastActivityDate="2013-11-19T18:03:17.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10105" PostTypeId="2" ParentId="10103" CreationDate="2013-11-19T18:24:53.733" Score="1" Body="&lt;p&gt;A lot depends on whether you're interested in using iterative methods or direct factorization of the matrix to solve the system of equations.  You need to be clear on the relative advantages and disadvantages of these approaches.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in direct factorization, you might want to look at Tim Davis's SuiteSparse software.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For iterative methods, PETSc is highly recommended.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either way, LAPACK is a poor choice, since it works only with dense matrices.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-11-19T18:24:53.733" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10106" PostTypeId="1" AcceptedAnswerId="10110" CreationDate="2013-11-20T01:07:02.720" Score="1" ViewCount="7309" Body="&lt;p&gt;I need to numerically determine the convergence order of Euler's method for various step-sizes. I am unsure how to go about doing this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem statement:&lt;/strong&gt; $\frac{dy}{dt}=\alpha t^{\alpha - 1}, y(0)=0$, where $\alpha &amp;gt; 0$. Use Euler's method to solve the initial value problem for $\alpha = 2.5,1,5,1.1$ with stepsize $h=0.2,0.1,0.05$. Determine numerically the convergence orders of the Euler method for these problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created an Excel spreadsheet with all of the relevant information tabulated (using Excel formulas), with columns $i, t_i=a+ih, a, b, \alpha, N, w_i, y(t_i), |y(t_i)-w_i|$ where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$i=$ ith step in iteration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_i$= time at ith step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a=$ initial time value&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b=$ final time value&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\alpha=$ constant in problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N=$ number of subintervals&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w_i=$ approximation to solution at ith step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y(t_i)=$ true solution at ith step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|y(t_i)-w_i|=$ Magnitude of error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, since I cannot estimate the error $\kappa$ from just one integration as the error term is approximately of the form $error_{t=b}(h) = Ah^{\kappa}$, one step size is not adequate to estimate $\kappa$. But if I do a log plot of the error and step size, the slope of the best line will give the required estimate of $\kappa$, which should be close to 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is how can I do the log plot of the error and step size in Excel 2013? I understand the math behind all this, but don't know how to implement it.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DlHIB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerDisplayName="user5427" LastEditorDisplayName="user5427" LastEditDate="2013-11-20T04:08:59.290" LastActivityDate="2013-11-20T15:43:00.390" Title="Numerically determining convergence order of Euler's method" Tags="|numerical-analysis|convergence|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="10107" PostTypeId="1" AcceptedAnswerId="10109" CreationDate="2013-11-20T10:07:54.890" Score="4" ViewCount="453" Body="&lt;p&gt;I’m going through some considerable effort to translate one of my codes from MATLAB. It’s a type of finite element code and I haven’t implemented the solver yet but comparing CPU times for simply assembling the global system between MATLAB and C, the latter is more than two orders of magnitude faster...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways, I have no idea how to use PETSc but will attempt to learn it in the next few weeks. Just reading over some code and documentation, it seems like it might be pretty hard to translate my code for PETSc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that there’s an option to read a matrix from a “file” (see &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex10.c.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;). I’m just wondering if I go through with translating my code as I’m doing now, can I just save matrix/rhs to a .dat file or something then read from there? I don’t have all that much experience with C so I don’t know if this would be difficult to do or not, especially for larger systems like I’ll have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my code, I’ll have to solve a linear system at every Newton iteration (it’s a nonlinear system), and combine this with the fact that the problem is time-dependent. The only time in my entire code that I’ll need PETSc is for the linear solve (admittedly, it’s a lot of linear solves...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I’m going to have to learn the syntax for PETSc now, I’d just like to know so I’m not frustrated later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On another note, I’m really surprised how much faster the code is in C so far. Maybe my MATLAB coding isn’t that efficient, but I’ve been keeping the for loops at a minimum where possible.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2013-11-20T11:13:48.330" LastActivityDate="2013-11-20T15:33:50.967" Title="PETSc input format for linear solvers" Tags="|petsc|c|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10108" PostTypeId="2" ParentId="8324" CreationDate="2013-11-20T13:08:29.163" Score="2" Body="&lt;p&gt;Since buffer's sizes fixed at the moment they created, the only way is to simulate variable length arrays: Create an array of maximum size and use an extra variable so that OpenCL-kernel can store how much of this (max)array is in use.&lt;/p&gt;&#xA;" OwnerUserId="6585" LastActivityDate="2013-11-20T13:08:29.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10109" PostTypeId="2" ParentId="10107" CreationDate="2013-11-20T15:33:50.967" Score="6" Body="&lt;p&gt;I would recommend looking at a time-dependent example because PETSc can provide a lot more diagnostics if you formulate at that level.  For example, you could use a Rosenbrock method, an additive Runge-Kutta IMEX method, or others.  Some involve Newton iteration, but that is not required and is not always the best approach.  To use those methods, you can start by just providing your residual function, $f(t,u)$, as in $\dot u = f(t,u)$, or alternatively, $g$ in $g(t,u,\dot u) = 0$.  PETSc can automatically compute a Jacobian via coloring, and if you later decide to assemble the Jacobian for efficiency, PETSc can check whether what you assembled was correct and show you the errors.  If you roll your own time integration and your own Newton, these algorithmic variants and debugging features will not be available unless you write them yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf&quot;&gt;Users Manual&lt;/a&gt; explains the interfaces.  Also see the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/index.html&quot;&gt;manual pages&lt;/a&gt; for specific API documentation.  The man pages are cross-referenced to examples.  A good example to start with would be the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/ts/examples/tutorials/ex25.c.html&quot;&gt;Brusselator src/ts/examples/tutorials/ex25.c&lt;/a&gt;.  You may also find &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/tutorials/index.html&quot;&gt;slides from tutorials&lt;/a&gt; useful.  If you get stuck, feel free to ask here or on the &lt;a href=&quot;https://lists.mcs.anl.gov/mailman/listinfo/petsc-users&quot;&gt;petsc-users&lt;/a&gt; mailing list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your proposal to use the file system will be a huge bottleneck even in serial.  It complicates the workflow greatly and will limit the methods that you can use.  But if you just want to test a matrix once, you can use &lt;code&gt;PetscBinaryWrite()&lt;/code&gt; from MATLAB and load the system and solve using &lt;code&gt;src/ksp/ksp/examples/tutorials/ex10.c&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-11-20T15:33:50.967" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10110" PostTypeId="2" ParentId="10106" CreationDate="2013-11-20T15:43:00.390" Score="1" Body="&lt;p&gt;The task is using Excel to illustrate rate of convergence for Euler's method applied to three ordinary differential equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{dy}{dt} = \alpha t^{\alpha - 1} \text{ for } \alpha = 2.5,1.5,1.1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial conditions $y(0) = 0$, by varying the step size $h = 0.2,0,1,0.05$.  The astute Reader will spot that these IVPs are really simple integrations, and since the exact solution $t^{\alpha}$ is available by definite integration, say to $y(1)=1$ for comparison, we can columnate the three data sets in Excel as $\log h$ versus the log of &quot;errors&quot; $E = |y_h(t_i)-1|$ at $t=1$ (or some other convenient endpoint).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not hard to find instructions on the web for how to get Excel to combine a best fit line with a &quot;scatter plot&quot; of data.  Note that the step sizes $h$ here are in geometric progression, so after taking logarithms we'll have equally spaced arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The slope of the line gives us the order of convergence of the method, i.e. the exponent by which error $E$ varies with step size $h$.  While the local truncation error of &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler_method&quot; rel=&quot;nofollow&quot;&gt;(forward) Euler's method&lt;/a&gt; behaves like $O(h^2)$, with smaller $h$, proportionally more steps are required to reach the same endpoint.  Thus the global truncation error behaves only like $O(h)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the &lt;a href=&quot;https://en.wikipedia.org/wiki/Midpoint_method&quot; rel=&quot;nofollow&quot;&gt;modified Euler method&lt;/a&gt; does more work and should obtain a better rate of convergence, analogous to using higher order accurate quadrature rules.  The local truncation error of the modified (midpoint) Euler method is $O(h^3)$ and its global truncation error is thus $O(h^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You report getting slopes for the log-log plot of modified Euler method solution of about $0.5$, but this should be more like $2$.  I suspect a numerator/denominator confusion has crept into the line fitting.  If not then there's something wrong with how the modified Euler method was implemented.  Its rate of convergence should improve on the basic Euler method.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-11-20T15:43:00.390" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10111" PostTypeId="2" ParentId="2630" CreationDate="2013-11-20T23:57:11.580" Score="3" Body="&lt;p&gt;SLICOT's algorithm is not that complicated, it's a reduction to Schur form + some back-substitution. You can check the Bartels-Stewart paper &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=361582&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=361582&lt;/a&gt; which is reasonably readable and explains how it works. The paper is about the nonsymmetric case, but it shouldn't be hard to adapt it to the symmetric one --- you just need one Schur form instead of two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can probably also code it yourself in R if it already has a routine for the Schur form (I'd check myself, but it's always a mess to get meaningful results about R out of Google since due to their unfortunate naming choice).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could settle the dense case. The large and sparse one is more technical.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2013-11-20T23:57:11.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10112" PostTypeId="2" ParentId="10093" CreationDate="2013-11-21T03:08:56.093" Score="7" Body="&lt;p&gt;Of course you can discretize the two equations of your system with two different methods. The challenge will simply be when the solution of one equation enters that of another. At that point, you will have to decide what the finite volume solution should be at a finite difference point should be, or how to integrate the finite difference solution (which is only defined at individual points) over a finite volume integration area (a volume or co-volume).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is relatively easy to define something for these operations, but it is equally simple to get something that reduces the overall order of your scheme. That said, if the question is whether it's possible and whether it's complicated, I would say that the answer is &quot;yes&quot; and &quot;no&quot;, respectively.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-21T03:08:56.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10113" PostTypeId="2" ParentId="10078" CreationDate="2013-11-21T03:11:54.387" Score="2" Body="&lt;p&gt;Your solution looks pretty optimal to me (assuming it works correctly, which I haven't checked). I don't think there is a lot that you can optimize -- switching language, compiler, optimization flags may give you 20% speedup, but nothing that will change the fact that this function is your bottleneck. If you need to make your code faster, then you will need to change the algorithm that calls this function so often.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-21T03:11:54.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10114" PostTypeId="1" AcceptedAnswerId="10121" CreationDate="2013-11-21T08:03:59.970" Score="5" ViewCount="420" Body="&lt;p&gt;As we know, &lt;a href=&quot;http://en.wikipedia.org/wiki/Machine_epsilon&quot;&gt;machine epsilon&lt;/a&gt; limits relative rounding error in the range of normalized floating point numbers. But it is easy to check that this is not true for &lt;a href=&quot;http://en.wikipedia.org/wiki/Denormal_number&quot;&gt;denormalized&lt;/a&gt;  numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is when one should take this subtlety into account and what sort of problems can this issue cause? Any references?&lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2013-11-22T17:50:11.353" Title="Machine epsilon does not limit relative rounding error for denormals. Is this a problem?" Tags="|reference-request|floating-point|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10115" PostTypeId="2" ParentId="10094" CreationDate="2013-11-21T10:08:48.330" Score="5" Body="&lt;p&gt;I was curious to see some answers to this question, but somehow nobody bothers to reply...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding literature, I really like the book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0198528698&quot; rel=&quot;noreferrer&quot;&gt;Spectral/hp Element Methods for Computational Fluid Dynamics&lt;/a&gt; (there's also a cheaper soft-cover version now) and also the book of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387720650&quot; rel=&quot;noreferrer&quot;&gt;Hesthaven and Warburton&lt;/a&gt;. These two go into quite some detail that will help you implement the methods. The book of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3540307257&quot; rel=&quot;noreferrer&quot;&gt;Canuto, Hussaini, Quarteroni and Zang&lt;/a&gt; is more theoretical. This one also has a second volume &quot;Spectral Methods: Evolution to Complex Geometries and Applications to Fluid Dynamics&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't work on DG methods and I'm not an expert to judge the advantages of nodal vs. modal. The book of Karniadakis &amp;amp; Sherwin is more focused on methods with &lt;em&gt;continuous&lt;/em&gt; modal expansions. In this type of method, you are obliged to reorder the modes in two neighbouring elements in such fashion that the corresponding modes on the interface match in order to preserve the continuity of the global expansion. In addition, imposing boundary conditions requires extra attention since your modes are not associated with a specific location on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope someone familiar with this type of methods will add more detail.&lt;/p&gt;&#xA;" OwnerUserId="4688" LastActivityDate="2013-11-21T10:08:48.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10117" PostTypeId="2" ParentId="10114" CreationDate="2013-11-22T00:46:14.070" Score="1" Body="&lt;p&gt;I've been doing numerical methods for 15 years and had to read up on what denormals actually are. Given that I converse a lot with others who are also doing numerical methods and had never heard anyone talk about it, it's safe to assume that everyone just ignores this oddity -- denormals play no role in everyday numerics.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-22T00:46:14.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10118" PostTypeId="2" ParentId="10094" CreationDate="2013-11-22T03:19:08.590" Score="15" Body="&lt;p&gt;The tradeoffs below apply equally to DG and to spectral elements (or $p$-version finite elements).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing the order of an element, as in $p$-adaptivity, is simpler for modal bases because the existing basis functions do not change.  This is generally not relevant to performance, but some people like it anyway.  Modal bases can also be filtered directly for some anti-aliasing techniques, but that is also not a performance bottleneck.  Modal bases can also be chosen to expose sparsity within an element for special operators (usually the Laplacian and mass matrices).  This does not apply to variable coefficient or non-affine elements, and the savings are not huge for the modest order typically used in 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nodal bases simplify the definition of element continuity, simplify implementation of boundary conditions, contact, and the like, are easier to plot, and lead to better $h$-ellipticity in discretized operators (thus allowing use of less expensive smoothers/preconditioners).  It is also simpler to define concepts that are used by solvers, such as rigid body modes (just use nodal coordinates), and to define certain grid transfer operators such as arise in multigrid methods.  Embedded discretizations are also readily available for preconditioning, without needing a change of basis.  Nodal discretizations can efficiently use collocated quadrature (as with spectral element methods), and the corresponding under-integration can be good for energy conservation.  Inter-element coupling for first-order equations is sparser for nodal bases, though otherwise-modal bases are often modified to obtain the same sparsity.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2013-11-23T19:18:36.873" LastActivityDate="2013-11-23T19:18:36.873" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10119" PostTypeId="1" CreationDate="2013-11-22T10:35:53.113" Score="9" ViewCount="6825" Body="&lt;p&gt;I have a sparse and symmetric matrix A(n x n). The method Lanczos  tranforms matrix A into tridiagonal and symmetric matrix T and the Lanczos vectors in matrix V.&#xA;From there how do I compute k smallest or largest eigenvalues and corresponding eigenvectors?&lt;/p&gt;&#xA;" OwnerUserId="1381" LastActivityDate="2019-10-19T07:38:27.113" Title="How to use Lanczos method to compute eigenvalues and eigenvectors" Tags="|matlab|eigensystem|sparse-matrix|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10120" PostTypeId="1" AcceptedAnswerId="10130" CreationDate="2013-11-22T11:47:50.000" Score="3" ViewCount="904" Body="&lt;p&gt;I am not sure if this is the correct place to ask this question!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a data set such as the &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/matrices/&quot; rel=&quot;nofollow&quot;&gt;University of Florida Sparse Matrix Collection&lt;/a&gt; which  is produced from stencil operations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is there a way to generate such sparse matrices, maybe using Matlab?&lt;/p&gt;&#xA;" OwnerUserId="6663" LastActivityDate="2013-11-23T14:29:31.503" Title="Sparse matrices that represent common stencil operations" Tags="|matlab|matrix|sparse-matrix|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10121" PostTypeId="2" ParentId="10114" CreationDate="2013-11-22T14:31:50.127" Score="7" Body="&lt;p&gt;Back in the era of Intel 387 math coprocessors I had to maintain an interrupt handler for floating point exceptions.  Apart from that, I agree that pretty much everyone ignores denormals (or &lt;em&gt;subnormals&lt;/em&gt; as they are sometimes called), and this is successful in part because &lt;a href=&quot;https://en.wikipedia.org/wiki/Subnormal_numbers&quot; rel=&quot;nofollow noreferrer&quot;&gt;the IEEE 754 standard default handling of &quot;gradual underflow&quot;&lt;/a&gt; hides their (presumably rare) occurrance (better, at least, than overflow exceptions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denormals are an edge case of &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_of_significance&quot; rel=&quot;nofollow noreferrer&quot;&gt;loss of significance&lt;/a&gt;, from exponent underflow.  As a matter of theoretical treatment in numerical methods, it is most likely to arise by subtractive cancellation.  But we will often have subtractive cancellation (whether benign or harmful) without creating denormals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some studies have been done on the &lt;em&gt;slower performance&lt;/em&gt; caused by operations on denormals, such as Dooley and Kale (2013), &lt;a href=&quot;http://charm.cs.illinois.edu/newPapers/06-13/paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantifying the Interference Caused by Subnormal Floating-point Values&lt;/a&gt;, and by the same authors with additional coauthors, &lt;a href=&quot;http://research.ihost.com/osihpa/osihpa-govind.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Performance Degradation in the Presence of Subnormal Floating-Point Values&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's &lt;a href=&quot;http://rcl-rs-vvg.blogspot.com/2012/02/denormal-floats-across-architectures.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a blog article from 2012&lt;/a&gt; that links back to &lt;a href=&quot;https://stackoverflow.com/questions/9314534/why-does-changing-0-1f-to-0-slow-down-performance-by-10x&quot;&gt;a highly upvoted StackOverflow thread&lt;/a&gt; on the performance hit from denormalized floats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quantifying the effects on numerical accuracy are apt to be specific to the application, and in many cases the luxury of doing a comparison with multiple or arbitrary precision arithmetic may be the only way of getting a definitive measure.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2013-11-22T14:31:50.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10122" PostTypeId="2" ParentId="10114" CreationDate="2013-11-22T17:50:11.353" Score="7" Body="&lt;p&gt;It's worth a bit of a discussion as to &lt;em&gt;why&lt;/em&gt; denormals don't matter in typical scientific computation.  Usually, we are given some input $x$ and want to compute a function $f(x)$.  Due to numerical errors, we instead compute $\tilde{f}(x) \approx f(x)$.  There are two kinds of accuracy typically sought:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Relative: $$|f(x)-\tilde{f}(x)| &amp;lt; \alpha |f(x)|$$&lt;/li&gt;&#xA;&lt;li&gt;Absolute: $$|f(x)-\tilde{f}(x)| &amp;lt; \beta $$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Denormals break relative, but usually do not break absolute.  However, in most situations absolute accuracy is sufficient, and the only reason to establish relative accuracy is that it doesn't require knowledge of $\beta$ (this helps code and analysis modularity).  Since denormals interfere with absolute accuracy typically only for $\beta$ much smaller than we ever need, all is well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not always true: the canonical example is normalizing a vector where the output must have unit length to machine precision.  Here relative accuracy really is what is required, since the magnitudes will be amplified up to detectable levels.  In this case, denormals should be treated as equivalent to zero.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2013-11-22T17:50:11.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10123" PostTypeId="1" AcceptedAnswerId="10125" CreationDate="2013-11-22T22:42:20.963" Score="2" ViewCount="57" Body="&lt;p&gt;I have an oscillatory, decaying function that can be decomposed as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_k e^{iz_kt} $$where $z_k$ are complex. What I want is the imaginary parts of all of the $z_k$'s with some range of real parts. I could do a fourier transform and extract the widths of the lines, but I don't know how to handle the case that the lines overlap (which I expect to happen in at least a few cases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm to extract the imaginary parts of the $z_k$, like the Fouier transform would extract the real parts?&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2013-11-23T05:15:23.220" Title="Algorithm to extract the decaying parts of complex exponentials" Tags="|algorithms|fourier-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10124" PostTypeId="1" CreationDate="2013-11-23T00:33:51.283" Score="2" ViewCount="177" Body="&lt;p&gt;This question popped up today in our group meeting. Suppose you are given a cloud of N points in 2D and each is associated with a velocity vector. These points are associated with particles on a 2D interface and velocities are obtained via simple particle tracking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interface we are studying is not &quot;clean&quot; in the sense that there are solid &quot;island&quot; spread over the liquid interface. As a results, some of the particles actually sit on the said islands while remaining ones are spread on the liquid interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the question: Given the velocity information can you determine and isolate the particles on the solid islands? I did a quick search but could not find anything useful but came up with a simple algorithm based on computing relative angular velocities between all particle pairs. If particle $\mathbf{p}_i$ and $\mathbf{p}_j$ belong to the same rigid body, then one can write:&#xA;$$\mathbf{v}_j = \mathbf{v}_i + \Omega_{ij} \:\hat{\mathbf{k}} \times \mathbf{r}_{ji},$$&#xA;where $\mathbf{r}_{ji} = \mathbf{r}_j - \mathbf{r}_i$ is the relative position. For any particle $\mathbf{p}_k$ that belongs to the same rigid body, one requires that $\Omega_{ik} = \Omega_{jk} = \Omega_{ij}$. Finally I detect the islands by generating a histogram of the matrix entries and looking of the peaks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a sample made-up examples, were a bunch of random points are considered. Red and blue particles belong to two different rigid bodies with $\Omega_{red} = 5$ and $\Omega_{blue} = 10$. Black particles are assumed to be Brownian and have random velocities. As you can see the histogram detects two peaks at 5 and 10 but also fictitious peaks around 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OXAMU.png&quot; alt=&quot;Random sample points&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/gECud.png&quot; alt=&quot;Histogram of matrix $\Omega_{ij}$&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you think of a better algorithm or point me to some references?&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2013-11-23T14:54:15.130" Title="Detect rigid body motions in a cloud of points" Tags="|algorithms|simulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10125" PostTypeId="2" ParentId="10123" CreationDate="2013-11-23T05:15:23.220" Score="2" Body="&lt;p&gt;Check out &lt;a href=&quot;http://scitation.aip.org/content/aip/journal/jcp/107/17/10.1063/1.475324&quot; rel=&quot;nofollow&quot;&gt;harmonic inversion&lt;/a&gt;. There is &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/Harminv&quot; rel=&quot;nofollow&quot;&gt;software&lt;/a&gt; for it.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-11-23T05:15:23.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10126" PostTypeId="1" CreationDate="2013-11-23T11:12:35.670" Score="5" ViewCount="147" Body="&lt;p&gt;I’m implementing a finite element code (translating from a working MATLAB version, so I have results to compare to) and for some odd reason, some of my computations are only accurate to around 6 decimal places.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m working with doubles and PetscScalars and I’ve double and quadruple checked my code and am fairly certain there is nothing wrong with my implementation. I went back and made sure all of my multiplications are with doubles or PetscScalars, though I’m not sure if that changes anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I test on an exact solution  where I should get machine precision error. On a refinement study, the coarsest mesh (two-element triangular mesh on $[0,1]^{2}$) I get machine precision error but for every refinement after that, the error bumps up to around $10^{-7}$ and completely flatlines there. For exact solutions that can’t be exactly expressed in terms of the function space, I’m obtaining proper convergence rates for basis degree 1 and 2, and then the rates rapidly drop off for any high degrees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’ve been trying to hunt down the error for a while now. This is my first time programming such a large project in C so I’m sure there are some nuances that I just don’t know about, although I think I’m experience enough with C to have not made any errors when translating the working code. Do I have to specify that the PetscScalar’s are doubles? I reconfigured &lt;code&gt;--with-precision=double&lt;/code&gt; but it didn’t change the results.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2013-11-23T11:31:31.390" LastActivityDate="2013-12-23T20:31:12.323" Title="Odd accuracy barrier in C/PETSc regarding finite elements" Tags="|petsc|c|precision|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10127" PostTypeId="1" AcceptedAnswerId="10128" CreationDate="2013-11-23T11:44:57.043" Score="2" ViewCount="162" Body="&lt;p&gt;You can calculate the electric potential over every point in a defined space by solving Laplace's equation. To do this in a computer program you set up an 2-d array/ matrix and loop the internal points applying the following formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\phi(x,y)= \frac{\phi(x_{i-1},y_j)+\phi(x_{i+1},y_j)+\phi(x_i,y_{j-1})+\phi(x_i,_{j+1})}{4}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meaning that the electric potential at $\phi(x,y)$ is just the average of the 4 neighbouring points. Included in the array are areas of constant potential i.e. the algorithm does not alter them. These are the boundary conditions (the outer points of the grid/ matrix, set to 0) and the two plates (at a user defined location, set to -1 and 1). Now with the grid set up as it is currently: evenly spaced in both x and y direction, only a parallel plate capacitor can be modelled. If i try to put the plate at an incline it will just look like a stair case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how can i set this up in terms of a computer program in order to calculate the electric potential over all points with one plate rotated at an arbitrary angle $\theta$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks. &lt;/p&gt;&#xA;" OwnerUserId="6669" LastActivityDate="2013-11-23T12:40:43.053" Title="Inclined plate capacitor grid/ mesh" Tags="|computational-geometry|geometry|computational-physics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10128" PostTypeId="2" ParentId="10127" CreationDate="2013-11-23T12:40:43.053" Score="3" Body="&lt;p&gt;What you're describing is a very specific way to solve the Laplace equation: using the Jacobi iteration to solve the five-point stencil when using finite differences. But there are much more general methods that would be difficult to derive from your starting point. A more promising approach is to start with the Poisson equation and ask how it can be approximated with finitely many points. One way would be to apply the finite element method using an unstructured grid that allows you not only to consider inclined plates but, in fact, any kind of geometry. This then gives you a finite dimensional linear system that you can, if you want, invert using the Jacobi iteration, but again, if you start with the linear system you will realize that there are many other methods that allow you to solve it: Direct solvers such as Gaussian elimination or LU decomposition, or if you want to exploit the fact that the system is sparse, solvers like Conjugate Gradients or multigrid.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-23T12:40:43.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10129" PostTypeId="2" ParentId="10126" CreationDate="2013-11-23T12:42:31.250" Score="1" Body="&lt;p&gt;Are you solving your linear systems iteratively, using solvers such as CG or GMRES? If so, you need to set the tolerance within these solvers appropriately or your overall accuracy will be limited by the accuracy you get from the linear solver.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-23T12:42:31.250" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10130" PostTypeId="2" ParentId="10120" CreationDate="2013-11-23T14:29:31.503" Score="1" Body="&lt;p&gt;I assume that you are looking for matrices that come from, e.g., finite differences applied to a PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may try the &lt;a href=&quot;http://math.nist.gov/MatrixMarket/&quot; rel=&quot;nofollow&quot;&gt;matrix market&lt;/a&gt;. There you can search for matrices from common PDE applications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab there is the function &lt;a href=&quot;http://www.mathworks.de/de/help/matlab/ref/del2.html&quot; rel=&quot;nofollow&quot;&gt;del2&lt;/a&gt; that returns a matrix representing a discrete Laplace operator.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2013-11-23T14:29:31.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10131" PostTypeId="2" ParentId="10124" CreationDate="2013-11-23T14:54:15.130" Score="1" Body="&lt;p&gt;Your algorithm makes sense, but is going to be too slow if you have many particles because you do pairwise comparisons. But there are other ways of looking at your data. Let us imagine for a moment that if you had islands that only moved but did not rotate. Then imagine each particle to be a point in 4d space: its two spatial coordinates plus the two components of its velocity vector. In effect, then, you will find islands as clusters of points that aren't just closely located but also are close in their velocities -- in other words, you will need to find &lt;em&gt;clusters of particles in 4d space&lt;/em&gt;. There is a whole field in statistics that deals with this question -- cluster analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation is a bit more complex if you allow islands to also rotate. In that case, the particles on islands are characterized by 7-dimensional vectors: the two location coordinates, the two components of the linear velocity of the island, the two components of the rotation center of the island, and its angular velocity. The problem is that if you only analyze two subsequent pictures all you get are four pieces of information for each particle: its position and its velocity. In other words, each particle occupies a three-dimensional manifold within the 7-dimensional space. Identifying islands then comes down to finding locations of your 7-dimensional space where multiple of these manifolds cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If you can analyze more than 2 frames, you may be able to determine whole arcs of particle motion, and in that case you will get more than 4 pieces of data if the arcs are not straight, since then only certain rotation centers and angular velocities are compatible with the arc. You should then be able to reduce the dimensionality of the manifolds, which should also make finding clusters easier.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-23T14:54:15.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10133" PostTypeId="2" ParentId="10126" CreationDate="2013-11-23T20:20:57.537" Score="1" Body="&lt;p&gt;For archival purposes, note that the resolution to the problem is found in the comments below the question. Data was being read from a file containing lower-precision data.&lt;/p&gt;&#xA;" OwnerUserId="3824" LastActivityDate="2013-11-23T20:20:57.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10134" PostTypeId="1" CreationDate="2013-11-24T11:18:43.553" Score="1" ViewCount="91" Body="&lt;p&gt;I am reading Stephen Wolfram's &quot;A New Kind of Science&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At present, I cannot understand how the cellular automata illustrations on &lt;a href=&quot;https://www.wolframscience.com/nksonline/page-79&quot; rel=&quot;nofollow&quot;&gt;p79&lt;/a&gt; are created.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the patterns, the active cell, representing the head, appears to change orientation between up and -45 degrees (towards bottom right). However, none of the rules seem to include an active cell with an orientation other than up or down. How does the active cell orientation of -45 degrees come about in the patterns?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something obvious (I am a beginner in this area)?&lt;/p&gt;&#xA;" OwnerUserId="6672" LastEditorUserId="6672" LastEditDate="2013-11-24T11:54:07.067" LastActivityDate="2013-12-24T13:31:52.630" Title="How to interpret Turing Machine illustrations on p79 of Stephen Wolfram's &quot;A New Kind of Science&quot; book?" Tags="|visualization|complexity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10135" PostTypeId="2" ParentId="10134" CreationDate="2013-11-24T13:00:43.357" Score="1" Body="&lt;p&gt;I think that it is an inconsistency between the printing of the rules and the diagrams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the downward (-90 degrees) arrows in the rules are replaced with arrows pointing to the bottom right (-45 degrees) then the rules seem to make sense.&lt;/p&gt;&#xA;" OwnerUserId="6672" LastActivityDate="2013-11-24T13:00:43.357" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10137" PostTypeId="1" AcceptedAnswerId="10139" CreationDate="2013-11-24T14:29:38.243" Score="9" ViewCount="2380" Body="&lt;p&gt;I have this problem in my textbook:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Suggest efficient algorithm for solving system of linear equations with cyclic three-diagonal matrix, that is of the form:&#xA;   \begin{bmatrix} a_1&amp;amp;b_1&amp;amp;0&amp;amp;\cdots&amp;amp;0&amp;amp;d_1\\c_2&amp;amp;a_2&amp;amp;b_2&amp;amp;0&amp;amp;\vdots&amp;amp;0\\0&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;0&amp;amp;\vdots\\\vdots&amp;amp;\vdots&amp;amp;c_{n-2}&amp;amp;a_{n-2}&amp;amp;b_{n-2}&amp;amp;0\\0&amp;amp;\cdots&amp;amp;\cdots&amp;amp;c_{n-1}&amp;amp;a_{n-1}&amp;amp;b_{n-1}\\d_2&amp;amp;0&amp;amp;\cdots&amp;amp;0&amp;amp;c_n&amp;amp;a_n\end{bmatrix}&#xA;  without swapping any rows and columns. Estimate complexity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And I don't know how to approach this. Classic elimination would work in very efficient $O(n)$ time with this matrix, but the problem is when, let's say, I want to eliminate $c_{2}$ with $1$-st row that is add to second row $\frac{-c_{2}}{a_{1}}\cdot \begin{bmatrix} a_1&amp;amp;b_1&amp;amp;0&amp;amp;\cdots&amp;amp;0&amp;amp;d_1 \end{bmatrix}$ and when $a_1=0$. I can't do that, and even if $a_1\neq 0$ then the same problem can occur somewhere in the middle of this proccess. Moreover, as the problem text states, I am not allowed to swap any rows or columns, so I don't know if this approach can be somehow fixed. Can anybody help?&lt;/p&gt;&#xA;" OwnerUserId="5540" LastEditorUserId="276" LastEditDate="2013-11-26T02:22:07.357" LastActivityDate="2013-11-26T02:22:07.357" Title="Solving system of linear equations with cyclic tridiagonal matrix" Tags="|matrix|linear-solver|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10139" PostTypeId="2" ParentId="10137" CreationDate="2013-11-25T04:06:12.220" Score="4" Body="&lt;p&gt;It is not difficult to determine the complexity of a straightforward elimination/reduction to upper triangular form.  Note that the initial matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{bmatrix} a_1&amp;amp;b_1&amp;amp;0&amp;amp;\cdots&amp;amp;0&amp;amp;d_1\\c_2&amp;amp;a_2&amp;amp;b_2&amp;amp;0&amp;amp;\vdots&amp;amp;0\\0&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;0&amp;amp;\vdots\\\vdots&amp;amp;\vdots&amp;amp;c_{n-2}&amp;amp;a_{n-2}&amp;amp;b_{n-2}&amp;amp;0\\0&amp;amp;\cdots&amp;amp;\cdots&amp;amp;c_{n-1}&amp;amp;a_{n-1}&amp;amp;b_{n-1}\\d_2&amp;amp;0&amp;amp;\cdots&amp;amp;0&amp;amp;c_n&amp;amp;a_n\end{bmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;becomes after one step of elimination:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{bmatrix} a_1&amp;amp;b_1&amp;amp;0&amp;amp;\cdots&amp;amp;0&amp;amp;d_1\\ 0 &amp;amp;a_2 - \frac{b_1 c_2}{a_1}&amp;amp;b_2&amp;amp;0&amp;amp;\vdots&amp;amp;-\frac{d_1 c_2}{a_1}\\0&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;0&amp;amp;\vdots\\\vdots&amp;amp;\vdots&amp;amp;c_{n-2}&amp;amp;a_{n-2}&amp;amp;b_{n-2}&amp;amp;0\\0&amp;amp;\cdots&amp;amp;\cdots&amp;amp;c_{n-1}&amp;amp;a_{n-1}&amp;amp;b_{n-1}\\0&amp;amp;-\frac{b_1 d_2}{a_1}&amp;amp;\cdots&amp;amp;0&amp;amp;c_n&amp;amp;a_n-\frac{d_1 d_2}{a_1}\end{bmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cost of this step is constant $O(1)$, and it leaves a $(n-1)\times(n-1)$ cyclic three-diagonal submatrix in the lower right corner.  Full reduction will thus entail $O(n)$ cost.  To solve a linear system one might perform the same operations on the right-hand side, also an $O(n)$ cost.  One also has then to perform a backsolve with an upper triangular matrix having at most three nonzero entries per row, another $O(n)$ task.  So that represents the complexity of solving systems with such coefficient matrices.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-11-25T04:06:12.220" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10140" PostTypeId="1" AcceptedAnswerId="10149" CreationDate="2013-11-25T17:23:38.397" Score="4" ViewCount="393" Body="&lt;p&gt;Hej,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to simulate a partial differential equation (a modified Cahn-Hilliard equation, but the details do not matter much. The questions also applies to the diffusion equation). I'm looking for a good framework to do heavy numerics. My requirements are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No license costs (matlab is no option, because we do not have enough licenses to run many jobs on our cluster)&lt;/li&gt;&#xA;&lt;li&gt;Fast code (ideally it would be compiled code. Using an interpreted language like python at runtime might be too slow)&lt;/li&gt;&#xA;&lt;li&gt;Flexible geometries. In fact I'd like to simulate a three dimensional problem with cylindrical symmetry. For computational efficiency I want to express my concentration field as a function of radius and the z coordinate, which makes of course the Laplacian a little bit more difficult to implement.&lt;/li&gt;&#xA;&lt;li&gt;Neumann boundary conditions (periodic boundary conditions are ok, too, but they cannot be used for the aforementioned cylindrical geometry). Neumann boundary conditions essential ensure no-flux conditions at the boundary, such that the total amount of material in the system is conserved.&lt;/li&gt;&#xA;&lt;li&gt;Ideally multiprocessing would be supported.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Currently, I'm using &lt;a href=&quot;http://www.xmds.org&quot; rel=&quot;nofollow&quot;&gt;XMDS2&lt;/a&gt;, which is a great toolset. I use it to automatically generate C code, which I can compile and then run. It works like a charm, except for the boundary conditions. Unfortunately, XMDS only supports Dirichlet boundary conditions for cylindrical geometry, which does not help much in my case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know alternative frameworks, which support my requirements and are easy to use? If everything fails, I have to develop my own code, but I'm afraid that I cannot write as efficient code as experienced people would be able to do.&lt;/p&gt;&#xA;" OwnerUserId="6683" LastActivityDate="2013-11-26T13:42:22.583" Title="Is there a general framework for solving PDEs on uniform grid in parallel" Tags="|pde|parallel-computing|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10141" PostTypeId="1" AcceptedAnswerId="10143" CreationDate="2013-11-25T17:41:39.560" Score="1" ViewCount="136" Body="&lt;h1&gt;Background&lt;/h1&gt;&#xA;&lt;p&gt;I am solving the unsteady heat equation in 3D using an alternating direction implicit (ADI) method. This means that I am solving three different tridiagonal systems within a single timestep (once for each cartesian axis). The size of these matrices is on the order of 10^5 x 10^5.&lt;/p&gt;&#xA;&lt;p&gt;Although each direction's &lt;em&gt;A&lt;/em&gt; matrix has all non-zeros along the three diagonals, there are situations where one direction will have a zero value along one of the diagonals where another direction does not.&lt;/p&gt;&#xA;&lt;p&gt;My original plan was to set up an AIJ matrix and preallocate for 3 non-zero values and reuse the same memory allocation for each of the three directions. This works fine for solving the first direction, but when I hit the second direction PETSc complains because I am attempting to add a new non-zero value where there wasn't one before (related to differences in boundary conditions).&lt;/p&gt;&#xA;&lt;h1&gt;Questions&lt;/h1&gt;&#xA;&lt;p&gt;What is the most efficient way to proceed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a way to allocate differently that will allow me to have values change between zero and non-zero?&lt;/li&gt;&#xA;&lt;li&gt;OR should I create a new matrix and new allocation for each direction? (This I know works, but I'm worried about the overhead of destroying and creating a new matrix for every solution.)&lt;/li&gt;&#xA;&lt;li&gt;Other options?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In pseudo code, my program looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;// Create and preallocate A matrix:&#xA;MatCreateAIJ(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,M,M,3,NULL,3,NULL,&amp;amp;Amat);&#xA;    &#xA;// Create x vector&#xA;VecCreate(PETSC_COMM_WORLD,&amp;amp;x);&#xA;&#xA;// Create b vector&#xA;VecDuplicate(x,&amp;amp;b);&#xA;&#xA;// Solve for each timestep:&#xA;for (each timestep)&#xA;{&#xA;    for (X,Y,Z)&#xA;    {&#xA;        MatSetValues(Amat,...,INSERT_VALUES);&#xA;        VecSetValues(b,...,INSERT_VALUES);&#xA;&#xA;        MatAssemblyBegin(Amat,MAT_FINAL_ASSEMBLY);&#xA;        MatAssemblyEnd(Amat,MAT_FINAL_ASSEMBLY);&#xA;        &#xA;        VecAssemblyBegin(b);&#xA;        VecAssemblyEnd(b);   &#xA;     &#xA;        KSPSetOperators(ksp, Amat, Amat, DIFFERENT_NONZERO_PATTERN);&#xA;        KSPSetUp(ksp);&#xA;        KSPSolve(ksp,b,x);&#xA;&#xA;        MatZeroEntries(Amat);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3179" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2013-11-25T22:06:23.613" Title="How to allocate memory for successive iterative solutions with potentially different non-zero structure?" Tags="|finite-difference|petsc|memory-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10142" PostTypeId="1" AcceptedAnswerId="10175" CreationDate="2013-11-25T21:58:56.690" Score="3" ViewCount="427" Body="&lt;p&gt;I’m currently having some issues with my routine for the linear advection-diffusion problem. The model problem is as follows:&#xA;$$&#xA;\nabla\cdot(\mathbf{s} u) - \nabla\cdot(\kappa\nabla u) = f, \;\;\;\text{in}\;\Omega \subseteq \mathbb{R}^{2}&#xA;$$&#xA;$$&#xA;u = g_{D} \;\;\;\text{on}\;\Gamma_{D}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary is purely Dirichlet in this case. The formulation is:&#xA;$$&#xA;a(u,v) = \sum_{K \in \mathcal{M}_{h}}\int_{K}\kappa\nabla u\cdot\nabla v - \sum_{e \in \mathcal{E}_{h}}\int_{e}\left\{\kappa\nabla u\cdot\mathbf{n}\right\}[[v]] + \sum_{e \in \mathcal{E}_{h}}\left\{\kappa\nabla v\cdot\mathbf{n}\right\}[[u]] + \sum_{e \in \mathcal{E}_{h}}\frac{\sigma}{|e|}\int_{e}[[u]][[v]]&#xA;$$&#xA;$$&#xA;b(u,v) = -\sum_{K \in \mathcal{M}_{h}}\int_{K}\mathbf{s}u\cdot\nabla v + \sum_{e \in \mathcal{E}_{\text{int}}}\int_{e}\left\{\mathbf{s}\cdot\mathbf{n}\right\}u^{up}[[v]] + \sum_{e \in \Gamma_{\text{out}}}\int_{e}(\mathbf{s}\cdot\mathbf{n})uv&#xA;$$&#xA;$$&#xA;\ell(v) = \int_{\Omega}fv + \sum_{e \in \Gamma_{D}}\int_{e}\left( \kappa\nabla v\cdot\mathbf{n} + \frac{\sigma}{|e|}v\right)g_{D} - \sum_{e \in \Gamma_{\text{in}}}\int_{e}(\mathbf{s}\cdot\mathbf{n})g_{D}v&#xA;$$&#xA;So, we have&#xA;$$&#xA;a(u,v) + b(u,v) = \ell(v)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inflow and outflow boundaries are defined as $\Gamma_{\text{in}} = \left\{e \in \partial\Omega| \;\mathbf{s}\cdot\mathbf{n} &amp;lt; 0\right\}$ and $\Gamma_{\text{in}} = \partial\Omega\backslash\Gamma_{\text{in}}$. And $\mathcal{M}_{h}$ is the set of all mesh elements (triangular in my case), $\mathcal{E}_{h}$ the set of all edges of the mesh elements, and $\mathcal{E}_{\text{int}}$ just the set of interior edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the linear diffusion problem, my routine has no issues at all. Now if I use the formulation above for the exact solution $u(x,y) = x(x-1)y(y-1)$, for some values of $\kappa$ and $\mathbf{s}$ yield suboptimal convergence when I’m quite positive that they shouldn’t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source I’m primarily using is “Discontinuous Galerkin Methods for Solving Elliptic and Parabolic Equations” by B. Riviere. For this formulation, the convergence rate (for penalization of $\sigma \ge 0$) in the $L^{2}$ norm should be $p$ if $p$ is even and $p+1$ if $p$ is odd, where $p$ is the degree of the basis functions used. In the energy norm, we should have convergence rate of $p$ in all cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now here are the examples I am testing and the results. The exact solution is taken to be $u(x,y) = x(x-1)y(y-1)$ and the source term is chosen accordingly. I take $\Omega = [0,1]^{2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) $\kappa = 10^{-10}$, $\mathbf{s} = [1,1]^{T}$. $L^{2}$ convergence is $p$ for all $p$ (suboptimal). Energy norm convergence is $p-1$ for all $p$ (suboptimal) (with the exception of $p=1$, which has convergence rate $p$). This is for $\sigma = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Same example as above, but this time with $\sigma = 0$. I obtain the proper convergence rates that I stated in the paragraph above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Same example as in 1), with $\sigma = 1$, $\kappa=10^{-10}$ and $\mathbf{s} = [1000,1000]^{T}$. I obtain the proper convergence rates here too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) In general, for $\mathbf{s} = [1,1]^{T}$, if I take $\kappa=10^{-k}$ for some positive integer $k &amp;gt; 3$ (approximately speaking), I attain the suboptimal convergence rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) If I multiply the entire model equation by $10^{10}$ on both sides and use the resulting formulation (so $\kappa = 1$ and $\mathbf{s} = [10^{10}, 10^{10}]^{T}$ and the source term $f$ is modified accordingly), I obtain proper convergence rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas what is going on here? Is this to be expected or a bug in the code? I’ve tested on a number of a examples and can’t really find other examples where the rate is suboptimal. It’s very perplexing to me that in examples 1) and 2), if I increase the advection it actually gives me the expected convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can any DG experts chime in here?&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="3691" LastEditDate="2013-11-29T10:00:57.960" LastActivityDate="2013-11-29T10:00:57.960" Title="Convergence of interior penalty DG methods" Tags="|finite-element|advection-diffusion|discontinuous-galerkin|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="10143" PostTypeId="2" ParentId="10141" CreationDate="2013-11-25T22:06:23.613" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;ADI is not a very good parallel algorithm.  You should seriously consider formulating the problem in 3D and solving with multigrid.  You could get a start with &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex45.c.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;src/ksp/ksp/examples/tutorials/ex45.c&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you insist on using ADI, you should seriously consider allocating the three matrices separately.  It's more memory, but then you won't have to reassemble on each iteration, so you amortize setup costs.  This uses more memory.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you insist on ADI and on reusing the same matrix for each direction, you should allocate for three nonzeros per row and fill in explicit zeros even when the direction is missing an entry.  Otherwise the first assembly compacts out the unused entries and they can't be cheaply inserted later.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-11-25T22:06:23.613" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10144" PostTypeId="2" ParentId="10140" CreationDate="2013-11-25T22:18:59.597" Score="2" Body="&lt;p&gt;From your description, PETSc's &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMDACreate3d.html&quot; rel=&quot;nofollow&quot;&gt;DMDA&lt;/a&gt; is probably sufficient.  It is easy to use with geometric multigrid (or many other solvers) and should meet your other requirements.  There are many examples (some linked at the bottom of the man page) and even some examples for Cahn-Hilliard.  Feel free to ask further questions here or on the &lt;a href=&quot;https://lists.mcs.anl.gov/mailman/listinfo/petsc-users&quot; rel=&quot;nofollow&quot;&gt;petsc-users mailing list&lt;/a&gt;.  &lt;em&gt;[Disclaimer: I am a PETSc developer.]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-11-25T22:18:59.597" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10146" PostTypeId="2" ParentId="10140" CreationDate="2013-11-26T01:11:15.403" Score="2" Body="&lt;p&gt;deal.II (&lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/&lt;/a&gt;) satisfies all of your requirements. Like Jed, let me suggest to ask your question on the &lt;a href=&quot;http://dealii.org/mail.html&quot; rel=&quot;nofollow&quot;&gt;deal.II mailing lists&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-26T01:11:15.403" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10148" PostTypeId="1" AcceptedAnswerId="10151" CreationDate="2013-11-26T09:57:54.203" Score="4" ViewCount="461" Body="&lt;p&gt;Can the Levenberg-Marquardt algorithm be used for minimization and not fitting? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually we input the derivative of the function we want to fit in the minimizer. Now if I assume I have an objective function $f(t;a,b,c)$ (which I want to minimize with respect to the parameters $a,b,c$). Then I would not see a way to input something equivalent to the derivatives that I input for fitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what should I do in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why am I asking this question? I have developed some C++ code for fitting with Levenberg-Marquardt, and it's a piece of art that I don't want to lose by starting from scratch with a new minimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think?&lt;/p&gt;&#xA;" OwnerUserId="5410" LastActivityDate="2013-11-27T00:38:20.853" Title="Can the Levenberg-Marquardt algorithm be used for minimization and not fitting" Tags="|optimization|c++|regression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10149" PostTypeId="2" ParentId="10140" CreationDate="2013-11-26T13:42:22.583" Score="1" Body="&lt;p&gt;The authors of XMDS2 added support for Neumann conditions in connection with the Bessel transform (i.e. radial symmetric coordinates) in the latest development version (revision 2913). I will therefore continue using XMDS2, since it now contains all features that I need. Thanks again to the very helpful XMDS2 authors!&lt;/p&gt;&#xA;" OwnerUserId="6683" LastActivityDate="2013-11-26T13:42:22.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10150" PostTypeId="1" AcceptedAnswerId="10159" CreationDate="2013-11-26T13:55:57.247" Score="0" ViewCount="214" Body="&lt;p&gt;I am working on numerical method of Multigrid. What's the available implementation(solver) (actually used in scientific computation) of multigrid method?&lt;/p&gt;&#xA;" OwnerUserId="6694" LastEditorUserId="123" LastEditDate="2013-11-27T05:09:50.207" LastActivityDate="2013-11-27T07:16:31.683" Title="What libraries provide an implementation of multigrid?" Tags="|pde|software|multigrid|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10151" PostTypeId="2" ParentId="10148" CreationDate="2013-11-26T15:06:27.527" Score="9" Body="&lt;p&gt;The Levenberg-Marquardt method can be used to minimize any problem of the form:&#xA;$&#xA;\min f(x)=\sum_{i=1}^{m} f_{i}(x)^{2}&#xA;$&#xA;However, if the objective functino to be minimized is not a sum of squares, then the method is no longer applicable.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-11-26T15:06:27.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10152" PostTypeId="1" CreationDate="2013-11-26T19:54:33.333" Score="3" ViewCount="502" Body="&lt;p&gt;I'm looking for analytic solutions to time dependent fluid flow problems (can be compressible or incompressible, Euler or Navier-Stokes equations). The main thing though is that I'd like there to be 'interesting' physics going on and for this 'interesting' physics to develop as time proceeds. I'm fine with having source terms to force the solution to be 'interesting'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give an idea of what I mean by interesting -- I'm thinking along the lines of something that appears like transitioning to turbulence, developing shocks and/or vortex shedding.&lt;/p&gt;&#xA;" OwnerUserId="3469" LastEditorUserId="3469" LastEditDate="2013-11-27T03:58:20.847" LastActivityDate="2013-12-04T03:06:00.860" Title="Looking for analytic solutions to time dependent fluid flow problems" Tags="|fluid-dynamics|testing|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10154" PostTypeId="2" ParentId="10152" CreationDate="2013-11-26T22:43:37.537" Score="5" Body="&lt;p&gt;Try looking into the solutions available within &lt;a href=&quot;https://github.com/manufactured-solutions/MASA&quot; rel=&quot;noreferrer&quot;&gt;MASA&lt;/a&gt;.  MASA is a library of manufactured solutions containing many steady and transient problems for the flow physics you describe.  These don't tend to be &quot;interesting&quot; in the sense that one stares at the results because they describe canonical engineering flows.  However, they're &quot;interesting&quot; in the sense that many or all of the terms in the governing equations are active.  For example, the library contains &lt;a href=&quot;http://users.ices.utexas.edu/~rhys/papers/WCCM2012-16661.pdf&quot; rel=&quot;noreferrer&quot;&gt;a transient manufactured solution for the compressible Navier--Stokes equations with a power law viscosity&lt;/a&gt; that I helped to develop.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2013-11-26T22:43:37.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10155" PostTypeId="2" ParentId="10148" CreationDate="2013-11-27T00:38:20.853" Score="3" Body="&lt;p&gt;Take a look at the book by Nocedal and Wright, &quot;Numerical Optimization&quot;, to see the Levenberg-Marquardt method in more context than just fitting.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-27T00:38:20.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10156" PostTypeId="1" AcceptedAnswerId="10160" CreationDate="2013-11-27T01:12:03.347" Score="7" ViewCount="111" Body="&lt;p&gt;In general (and in real application), suppose I am using a randomized algorithm (e.g. Use MCMC to sample from a distribution and then compute $E(f(x))$ for some function $f$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume my algorithm will face every possible input and I use a pseudo-random generator to provide randomness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shall I derandomize my algorithm by fixing a random seed every time the algorithm start? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason that I come up with this question is that by derandomizing it, I can make sure every input will have a corresponding output, which makes me feel more comfortable for practical use. However, I believe I will lose something (maybe the algorithm works very bad for worst case input) if I derandomizing it like this.&lt;/p&gt;&#xA;" OwnerUserId="4211" LastActivityDate="2013-11-27T04:01:04.783" Title="Shall I derandomize a randomized algorithm in real application?" Tags="|monte-carlo|random-number-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10158" PostTypeId="2" ParentId="10152" CreationDate="2013-11-27T02:50:04.000" Score="2" Body="&lt;p&gt;Frankly, what you want doesn't exist. If we could generate arbitrary, interesting solutions with simple techniques, we wouldn't have to write sophisticated Navier-Stokes solvers to find them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can manufacture arbitrary solutions as Rhys notes, but they will likely be arbitrarily tuned to your feature extractor. As such, they won't tell you much. Your best bet will be to develop a catalog of solutions to real fluids problems using a CFD solver and save them. Loading these from disk should be orders of magnitude faster than regenerating them for most interesting flows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also try finding some CFD communities to solicit some interesting solutions from.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-11-27T02:50:04.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10159" PostTypeId="2" ParentId="10150" CreationDate="2013-11-27T02:57:57.300" Score="3" Body="&lt;p&gt;There are many libraries that provide widely used implementations of multigrid methods. Most of the available finite element libraries do to the best of my knowledge (certainly the one I work on, &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal.II&lt;/a&gt;, does). There are also widely used implementations of algebraic multigrid methods, most notably from the ML and &lt;a href=&quot;http://acts.nersc.gov/hypre/&quot; rel=&quot;nofollow&quot;&gt;hypre&lt;/a&gt; packages that you can most easily access via the &lt;a href=&quot;http://trilinos.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;Trilinos&lt;/a&gt; and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt; projects.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="3481" LastEditDate="2013-11-27T07:16:31.683" LastActivityDate="2013-11-27T07:16:31.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10160" PostTypeId="2" ParentId="10156" CreationDate="2013-11-27T04:01:04.783" Score="5" Body="&lt;p&gt;You definitely want to derandomize your program during development. Otherwise you will not be able to debug it since problems are not reproducible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the same time, once you know the algorithm is working, you need to run it for multiple seeds or with different random number generators to ensure that your results (such as ensemble averages, standard deviations, etc) are independent of the actual sequence of random numbers.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-27T04:01:04.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10161" PostTypeId="1" AcceptedAnswerId="10162" CreationDate="2013-11-27T06:07:31.063" Score="11" ViewCount="1129" Body="&lt;p&gt;I'm trying to numerically invert the following integral transform:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(y) = \int_{0}^{\infty} y\exp{\left[-\frac{1}{2}(y^2 + x^2)\right]} I_0\left(xy\right)f(x)\;\mathrm{d}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for a given $F(y)$ I need to approximate $f(x)$&#xA;where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;$f(x)$ and $F(y)$ are real and positive&lt;/strong&gt; (they are continuous probability distributions)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;$x,y$ are real and positive&lt;/strong&gt; (they are magnitudes)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have a very messy and brute force method for doing this at the minute: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I define $f(x)$ and the spline over a series of points, the values of the splined points are 'guessed' by random sampling, which yields a predicted $F(y)$. A basic genetic algorithm I wrote up minimises the difference between the predicted and measured $F(y)$ array. I then take the $f(x)$ which the algorithm converges to as my answer for the inversion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach works fairly well for some simple cases, but it feels messy to me and not particularly robust.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can anyone give me guidance on better ways of solving this problem?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your time &amp;amp; help!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[x-posted at computerscience]&lt;/p&gt;&#xA;" OwnerUserId="6700" LastEditorUserId="4244" LastEditDate="2013-11-27T12:41:31.107" LastActivityDate="2013-11-28T10:56:33.170" Title="Numerical methods for inverting integral transforms?" Tags="|algorithms|numerical-analysis|approximation|inverse-problem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10162" PostTypeId="2" ParentId="10161" CreationDate="2013-11-27T07:30:02.767" Score="13" Body="&lt;p&gt;A fairly simple method would be to choose a basis in function space and convert the integral transformation to a matrix. Then you can just invert the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, here's how that works: you need some set of orthonormal basis functions $T_i(x)$. (You can get away without them being normalized too, but it's easier to explain this way.) Orthonormal means that the inner product $\langle T_i,T_j\rangle = \delta_{ij}$, where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\langle T_i,T_j\rangle \equiv \int_a^b W(x) T_i(x)T_j(x)\,\mathrm{d}x = \delta_{ij}\tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $W(x)$ is some weight function. That and the limits $a$ and $b$ are tied to your choice of $T_i$. Once you pick which set of basis functions to use, you can hard code the limits and weight function into your program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the orthonormality, you can express any function, such as $f(x)$ and $F(y)$, as linear combinations of these basis functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;f(x) &amp;amp;= \sum_{i} c_i T_i(x) &amp;amp;&#xA;F(y) &amp;amp;= \sum_{j} C_j T_j(y) \tag{2}&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the coefficients are calculated as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;c_i &amp;amp;= \langle f,T_i\rangle = \int_a^b W(x) f(x)T_i(x)\,\mathrm{d}x \tag{3}\\&#xA;C_j &amp;amp;= \langle F,T_j\rangle = \int_a^b W(y) F(y)T_j(y)\,\mathrm{d}y \tag{4}&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can verify that these expressions are consistent with the definitions of the coefficients, eq. (2), and the orthonormality, eq. (1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, compute the transform of each of the basis functions; let's call it $\tilde{T}_i(y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\tilde{T}_i(y) \equiv \int_0^\infty y \exp\biggl[-\frac{1}{2}(y^2 + x^2)\biggr]I_0(xy)T_i(x)\,\mathrm{d}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{T}_i(y)$ is a function, and so you can express it as a linear combination of the basis functions just as we did with $f(x)$ and $F(y)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\tilde{T}_i(y) = \sum_k A_{ik}T_k(y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the matrix elements $A_{ik}$ are determined the same way we found $c_i$ and $C_j$ above:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{ik} = \langle \tilde{T}_i, T_k\rangle = \int_a^b W(y) \tilde{T}_i(y)T_k(y)\,\mathrm{d}y\tag{5}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, this is a rather icky double integral, but you only have to do it once (ever) for each combination of $i$ and $k$. You can do the integrals numerically and then hard-code the resulting values in your program. (Side note: With a clever choice of $T_i(x)$ and $W(x)$, you might be able to make it so the integral can be done symbolically. Whether this is possible depends on your transform. You can do it with the Fourier transform, but I'm inclined to think it's not possible for the transform you're asking about here.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of the matrix elements $A_{ik}$ and coefficients $c_i$ and $C_j$, the relation between $f(x)$ and $F(y)$ reduces to a linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\overbrace{\sum_{j} C_j T_j(y)}^{F(y)}&#xA;&amp;amp;= \int_0^\infty y \exp\biggl[-\frac{1}{2}(y^2 + x^2)\biggr]I_0(xy)\overbrace{\sum_{i} c_i T_i(x)}^{f(x)}\,\mathrm{d}x\\&#xA;&amp;amp;= \sum_{i} c_i \int_0^\infty y \exp\biggl[-\frac{1}{2}(y^2 + x^2)\biggr]I_0(xy)T_i(x)\,\mathrm{d}x\\&#xA;&amp;amp;= \sum_{i} c_i \sum_{k} A_{ik}T_k(y)&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the orthogonality of the basis functions, you can isolate any particular coefficient $C_{\ell}$ by taking the inner product of both sides with $T_{\ell}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\left\langle \biggl(\sum_{j} C_j T_j\biggr), T_\ell\right\rangle &amp;amp;= \left\langle \biggl(\sum_{i} c_i \sum_{k} A_{ik}T_k\biggr), T_\ell\right\rangle \\&#xA;\int_a^b W(y) \sum_{j} C_j T_j(y)T_\ell(y)\,\mathrm{d}y &amp;amp;= \int_a^b W(y) \sum_{i} c_i \sum_{j} A_{ik}T_k(y)T_\ell(y)\,\mathrm{d}y \\&#xA;\sum_{j} C_j \int_a^b W(y) T_j(y)T_\ell(y)\,\mathrm{d}y &amp;amp;= \sum_{i} c_i \sum_{k} A_{ik}\int_a^b W(y) T_k(y)T_\ell(y)\,\mathrm{d}y \\&#xA;\sum_{j} C_j \delta_{j\ell} &amp;amp;= \sum_{i} c_i \sum_{k} A_{ik}\delta_{k\ell} \\&#xA;C_\ell &amp;amp;= \sum_{i} c_i A_{i\ell}&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course $\ell$ is just a dummy index so I'll go back to calling it $C_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just a linear algebra problem. $C_j$ is a component of a vector, as is $c_i$, and $A_{ij}$ are components of a matrix. You can calculate $c_i$ from the function you're trying to transform using eq. (3), and you know $A_{ij}$ from the one-time calculation you did for this particular integral transform, eq. (5), so you can get the $C_j$s by doing a matrix multiplication (which computers are very good at), and then reconstruct the function $F(y)$ from those using eq. (2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, to do the inverse transform, you start with the function $F(y)$, calculate the $C_j$s from it using eq. (4), and then you would need to solve the linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C_j = \sum_{i} c_i A_{ij}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done by multiplying both sides by the inverse of the matrix $A$, but in practice there are more efficient ways to do it. Use the linear system solver of whatever linear algebra library you have at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that everything I've written so far has left the limits of the sums over $i$, $j$, etc. unspecified. In practice, you will need to choose some limits, say $1$ to $N$ (you choose $N$), such that any $f(x)$ you may encounter can be sufficiently well approximated by a linear combination of $T_1(x),\ldots,T_N(x)$, and also a range like $1$ to $M$ such that any $F(y)$ can be sufficiently well approximated by a linear combination of $T_1(y),\ldots,T_M(y)$. It's probably easiest to choose $M = N$. Here, what &quot;sufficiently well&quot;  means is up to you. Generally, the larger you make $M$ and $N$, the better your approximation will be, but the more memory (and time) you'll need to do the calculation. Remember that you need to calculate $N$ coefficients $c_i$, and for the matrix $A$ you need to have $M\times N$ coefficients calculated, from $A_{11}$ up to $A_{NM}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For functions defined on a finite interval which can be linearly rescaled to $[-1,1]$, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Chebyshev_polynomials&quot;&gt;Chebyshev polynomials&lt;/a&gt; are a common choice for $T_i$. There is a sense in which Chebyshev polynomials produce the most accurate approximation of a given function using a fixed number of terms, which makes them especially well suited for this kind of application. If you can truncate the domain of your probability distributions, you might be able to use those. They are associated with the weight $W(x) = \frac{1}{\sqrt{1 - x^2}}$ and the limits $a = -1$ and $b = 1$. (Note that in the form they're often given, the normalization is such that $\langle T_i, T_j\rangle = \delta_{ij}\pi/2$ for $i = j\neq 0$ and $\langle T_0, T_0\rangle = \pi$, so you may have to account for this in your code.)&lt;/p&gt;&#xA;" OwnerUserId="103" LastEditorUserId="103" LastEditDate="2013-11-28T10:56:33.170" LastActivityDate="2013-11-28T10:56:33.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10163" PostTypeId="2" ParentId="705" CreationDate="2013-11-27T07:38:14.733" Score="1" Body="&lt;p&gt;Given the apparent dearth of quasi Monte Carlo integrators for C++ (or C), I wrote &lt;a href=&quot;https://github.com/diazona/quasimontecarlo&quot; rel=&quot;nofollow&quot;&gt;my own implementation&lt;/a&gt; to be used with GSL. It's not especially well-tested, nor does it implement Mathematica's algorithm, but it should be better than nothing.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2013-11-27T07:38:14.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10164" PostTypeId="1" CreationDate="2013-11-27T12:14:22.103" Score="3" ViewCount="324" Body="&lt;p&gt;I was wondering if the use of incompressible flow approximation for fluid flow in heated pipes is reasonable. A previous question (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/6880/definition-of-incompressible-flow?newreg=29a19ca2603f4ff6a5573318176ad68f&quot;&gt;Definition of incompressible flow&lt;/a&gt;) seemed to focus on Natural Convection heat transfer. It does not seem to apply to Forced Convection problems where the fluid is heated as it moves in a pipe with a heated wall?&#xA;In this situation the temperature is increasing overtime (non steady state) and hence the density of the fluid is changing, In my case of a solar thermal system this heating time can be in order of hours and the temperature can vary between 80 and 250 °C. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model I'm creating is a run-time numerical model, and it has to be fast enough for that purpose, so what I'm doing now is using incompressible flow differential equations for each time step (assuming density is constant) and then modify the density after the solution is obtained. I'm not sure if this is really ok to do.&lt;/p&gt;&#xA;" OwnerUserId="6701" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-11-27T15:29:04.973" Title="Can one use incompressible flow approximation for fluid flow in heated pipes?" Tags="|fluid-dynamics|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10165" PostTypeId="2" ParentId="10164" CreationDate="2013-11-27T15:29:04.973" Score="5" Body="&lt;p&gt;It depends on the temperature range and the thermal expansion coefficient of the fluid. You can't treat a gas as incompressible if you heat it from 300K to 1000K, but if you heat water from 20C to 80C the density variations are so small that you can treat the fluid as incompressible (even though the density variations of course drive the convection -- this then leads to the Boussinesq approximation).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-11-27T15:29:04.973" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10166" PostTypeId="1" AcceptedAnswerId="10171" CreationDate="2013-11-27T20:54:10.453" Score="5" ViewCount="429" Body="&lt;p&gt;I'm writing a code using continuous piecewise linear finite elements on triangular&#xA;grids to solve the diffusion-reaction problem. the source function f is a Dirac mass at the center. How can i compute the mesh in order to have the dirac in the baricenter of almost one triangle of the mesh?&#xA;thanks  &lt;/p&gt;&#xA;" OwnerUserId="6703" LastEditorUserId="6703" LastEditDate="2013-11-29T17:53:14.337" LastActivityDate="2013-11-29T17:53:14.337" Title="Finite element discretization of Reaction-diffusion problem with Dirac source term" Tags="|finite-element|numerical-analysis|mesh-generation|elliptic-pde|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10167" PostTypeId="1" CreationDate="2013-11-28T00:21:48.350" Score="6" ViewCount="874" Body="&lt;p&gt;I'm solving a heat conduction problem in cylindrical coordinates with a composite cylinder made of two different materials. Essentially the cylinder is split into a central cylinder of material A, surrounded by an annulus of material B. My mesh is set up so that a control volume boundary aligns with the interface between material A and material B. When discretizing the conduction equation, I need to come up with an &quot;interface conductivity&quot; for this control volume edge, since the two materials have different thermal conductivities. The method normally employed in Cartesian coordinates is the harmonic mean conductivity, which basically ensures that the heat flux is the same on either side of the control volume surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as I am working in cylindrical coordinates, I am somewhat unsure of the expression I came up with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the standard notation of P being the node of interest and N being the adjacent node radially outward from P, this is what I came up with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k_{n}=(\frac{{\Delta}r}{k_{P}{\delta}r_{n}(1+r_{P}/r_{n})}+\frac{{\Delta}r}{k_N{\delta}r_{n}(1+r_{N}/r_{n})})^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above equation, ${\Delta}r$ is the radial width of a cell, ${\delta}r_n$ is the distance between nodes P and N, $r_P$ is the radial coordinate of node P, $r_N$ is the radial coordinate of node N, and $r_n$ is the radial coordinate of the interface between nodes P and N.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone confirm my expression? I can walk through my derivation if that would help.&lt;/p&gt;&#xA;" OwnerUserId="6705" LastEditorUserId="6705" LastEditDate="2013-12-02T05:33:42.423" LastActivityDate="2014-10-26T14:55:24.370" Title="Interface Conductivity for Finite Volume Method Heat Transfer in Cylindrical Coordinates" Tags="|finite-volume|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10168" PostTypeId="1" AcceptedAnswerId="10179" CreationDate="2013-11-28T00:20:49.847" Score="3" ViewCount="996" Body="&lt;p&gt;I have a numerical computation problem which requires solving nonlinear equations (with long integers) in multiple precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried an MPFR C++ wrapper from this link by Pavel:&#xA;&lt;a href=&quot;http://www.holoborodko.com/pavel/mpfr/&quot; rel=&quot;nofollow&quot;&gt;mpfr C++ wrapper by Pavel&lt;/a&gt;&#xA;The wrapper can be downloaded here:&#xA;&lt;a href=&quot;http://www.holoborodko.com/pavel/wp-content/plugins/download-monitor/download.php?id=4&quot; rel=&quot;nofollow&quot;&gt;mpfrc++-3.5.6.zip&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is precision loss in the solution when handling very long integers (equations with small integers worked well).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried three options as in the sample code below:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;to use the code immediately does not work with &quot;constant integer too&#xA;long&quot; error. &lt;/li&gt;&#xA;&lt;li&gt;below the two options &quot;worked&quot; with precision loss&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How can I handle such equations in multiple precision without precision loss?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a sample of my nonlinear equations with the long integers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;mpreal.h&quot;&#xA;&#xA;VectorXmp testfun126(VectorXmp x0)&#xA;{&#xA;    VectorXmp value(3);&#xA;    mpreal x,y,z;&#xA;    x=x0(0);&#xA;    y=x0(1);&#xA;    z=x0(2);&#xA;&#xA;// Working option 1: write all the long integers into &quot;double&quot; ( but there are precision loss )&#xA;value(0)=(256.*((256.*(-24610976415716501050652227.*x+256.*(-10153609683556422184100.+374519398571124540883.*y-4145573659500944095488.*z))*(29427736469514379027531261659072347.+58899562724319710108573382000184640.*y-1732944474195510410991057714955859184.*z))/pow(5042560366642267.*x-256.*(2446745837411900.+4901398098088043.*y-144207654645973248.*z),3)-(256.*(-308518681989548429992935348850261.+41445095210006425938788783390458.*y-1638970396838251453451269879637336.*z)*(-801790542801929135637671.-732048260009923946735424.*x+56975701334774517040256.*y-187552638032246240630656.*z))/pow(-3075770275504817.+198931044892562752.*x+14199788245258112.*y-1122852841901814912.*z,3)+(5*(-89303793175477833893354121208000.+6533090911353242906294143748495.*y-32276910383172707359896832089932.*z)*(-61468981380127448102256.-5328427636421850183140.*x+4647710007810227520885.*y-13344414478836548348450.*z))/pow(-46366672189358032.-18896234711237580.*x+3927118781169095.*y+14705346416259850.*z,3)-(3*(9101665097092871812176.+3063507166600182944940.*x+6945927557350563805665.*y+1052001549322007294950.*z)*(19493858980629008651267653094056.+93282964805436900100617577630195.*y+42271355681070699741325611572830.*z))/pow(46366672189358032.+18896234711237580.*x-3927118781169095.*y-14705346416259850.*z,3)-(4*(39553725461800043367392.+17203831108841472538824.*x+45483386678520344593037.*y+2703260049547565568088.*z)*(52830583937680669669892057655944.+303023948138837354463602341532495.*y+134962043561465977901954677856080.*z))/pow(92856945980914656.+51329763147513032.*x-8586501277743859.*y-56199770659759016.*z,3)-((22670037111266004087968.+12461845278544574559640.*x+39219302812923818032157.*y-46563087562792926056.*z)*(95973949246309465842551069546976.+723429769797021053206211106031819.*y+317530466286898645427564085427048.*z))/pow(50159316775994592.+36243094308305160.*x-4827156544231217.*y-52318895858217464.*z,3)-(80.*(4157117722725769078952.+4534359335248895646832.*x+26193979470458655189977.*y-2382852476120229696128.*z)*(205429639975670471114284923188348.+2095815907391732802212116237430935.*y+883539023887333564964405237094400.*z))/pow(45070329471431608.+130124049256651728.*x-5583613021604317.*y-387630670566282112.*z,3)-(16.*(9439334964924689507817.+17499514376929345709248.*x+187907876794815451253888.*y-21704870055089718153088.*z)*(943164674716649969807523653958385.+18130967224506023673179633045358720.*y+7486136216172114262568716503454336.*z))/pow(-3075770275504817.+198931044892562752.*x+14199788245258112.*y-1122852841901814912.*z,3)+(80.*(2304705299858575630109.*x-256.*(204828849006588248100.+19508530860149228990861.*y-2445924471668591306496.*z))*(-179928369646271075844345534739549.+3401432279430696137250330740801392.*y+12500875943051297916024009205116096.*z))/pow(5042560366642267.*x-256.*(2446745837411900.+4901398098088043.*y-144207654645973248.*z),3)+(80.*(-805507884940017483975376678503744.+52529278437993151034132605337909.*y-620040027953848498781390188900552.*z)*(-716026618045942942760.*x+243780804476456624597.*y-8*(408351630952413337484.+89777022692195474597.*z)))/pow(-50159316775994592.-36243094308305160.*x+4827156544231217.*y+52318895858217464.*z,3)+(768.*(61889933231497708820968.+30294916915069669525488.*x-4484037822343607626207.*y+13934625423713945278848.*z)*(16858970779944867265671037333379.*y-176.*(1546216290476124632111328928258.+3134171189636832381705249359145.*z)))/pow(45070329471431608.+130124049256651728.*x-5583613021604317.*y-387630670566282112.*z,3)-(40.*(1717566388539311579248.*x+7025931019459451548321.*y+48.*(46537098413809906919.-8301700878138964680.*z))*(3434616943638241443585000648954199.*y+320.*(1107265969195848092307625165761.+4643932844541992753284837619195.*z)))/pow(85141430232132048.+97951351741329392.*x-8855616621991191.*y-199920422688690560.*z,3)+(12.*(88457226224862447127008.+13504083955712971035976.*x-6622138801690554356387.*y+19322683651036147287512.*z)*(36451820000039413375829754767131.*y-8*(66864837166560711793644210325852.+35619205657210451197984743698883.*z)))/pow(92856945980914656.+51329763147513032.*x-8586501277743859.*y-56199770659759016.*z,3)+(512.*(45619694076424722199344.+14936846773318822792976.*x-3365788117861218576473.*y+10130491989577935272320.*z)*(12048859085295019197936041733505.*y-6*(32519187452933223586671104614156.+40471151781636260063426632487709.*z)))/pow(85141430232132048.+97951351741329392.*x-8855616621991191.*y-199920422688690560.*z,3)))/125.;&#xA;value(1)=(128.*((32768.*(24610976415716501050652227.*x-256.*(-10153609683556422184100.+374519398571124540883.*y-4145573659500944095488.*z))*(98990697209366584150952278657452.+920305667567495470446459093752885.*x-65799721166407263195366683527104.*z))/pow(5042560366642267.*x-256.*(2446745837411900.+4901398098088043.*y-144207654645973248.*z),3)+(1024.*(-10864227594859409007678067839115.+566592725765813239786863532667460.*x-3214793226869529893757297514562848.*z)*(9439334964924689507817.+17499514376929345709248.*x+187907876794815451253888.*y-21704870055089718153088.*z))/pow(-3075770275504817.+198931044892562752.*x+14199788245258112.*y-1122852841901814912.*z,3)+(40.*(2938923392457131154149055759247753.+8383263629566931208848464949723740.*x-24821520393182477390523323699174560.*z)*(4157117722725769078952.+4534359335248895646832.*x+26193979470458655189977.*y-2382852476120229696128.*z))/pow(45070329471431608.+130124049256651728.*x-5583613021604317.*y-387630670566282112.*z,3)+(80.*(1717566388539311579248.*x+7025931019459451548321.*y+48.*(46537098413809906919.-8301700878138964680.*z))*(3017477155357435955713408172820441.+3434616943638241443585000648954199.*x-6875761229715351344214913955270620.*z))/pow(85141430232132048.+97951351741329392.*x-8855616621991191.*y-199920422688690560.*z,3)+(2*(1013986939222028224203834326214704.+723429769797021053206211106031819.*x-1002019231842824621894736024449560.*z)*(22670037111266004087968.+12461845278544574559640.*x+39219302812923818032157.*y-46563087562792926056.*z))/pow(50159316775994592.+36243094308305160.*x-4827156544231217.*y-52318895858217464.*z,3)+(2*(698833722744934775627393528218146.+279848894416310700301852732890585.*x-191427609122898840477329914007915.*z)*(9101665097092871812176.+3063507166600182944940.*x+6945927557350563805665.*y+1052001549322007294950.*z))/pow(46366672189358032.+18896234711237580.*x-3927118781169095.*y-14705346416259850.*z,3)+(8*(557016173590538671691101855964863.+303023948138837354463602341532495.*x-309197308873592242001670976702725.*z)*(39553725461800043367392.+17203831108841472538824.*x+45483386678520344593037.*y+2703260049547565568088.*z))/pow(92856945980914656.+51329763147513032.*x-8586501277743859.*y-56199770659759016.*z,3)-(128.*(-57335208466953058729715954197164.+96390872682360153583488333868040.*x-372364031472286149332017066304111.*z)*(45619694076424722199344.+14936846773318822792976.*x-3365788117861218576473.*y+10130491989577935272320.*z))/pow(85141430232132048.+97951351741329392.*x-8855616621991191.*y-199920422688690560.*z,3)-(5*(-5058036108182894712997605343704.+13066181822706485812588287496990.*x-23584235630998237996607750176151.*z)*(61468981380127448102256.+5328427636421850183140.*x-4647710007810227520885.*y+13344414478836548348450.*z))/pow(46366672189358032.+18896234711237580.*x-3927118781169095.*y-14705346416259850.*z,3)-(256.*(-35027435322808897803896166913833.+101153824679669203594026224000274.*x-443348667941077090029000877418626.*z)*(61889933231497708820968.+30294916915069669525488.*x-4484037822343607626207.*y+13934625423713945278848.*z))/pow(45070329471431608.+130124049256651728.*x-5583613021604317.*y-387630670566282112.*z,3)-(24.*(-23539469566855513950637813409344.+36451820000039413375829754767131.*x-87577625291530403453057402554096.*z)*(88457226224862447127008.+13504083955712971035976.*x-6622138801690554356387.*y+19322683651036147287512.*z))/pow(92856945980914656.+51329763147513032.*x-8586501277743859.*y-56199770659759016.*z,3)-(112.*(97743545586690977941666831119873.+189463292388600804291605866927808.*x-534599264249120709692835475330432.*z)*(801790542801929135637671.+732048260009923946735424.*x-56975701334774517040256.*y+187552638032246240630656.*z))/pow(-3075770275504817.+198931044892562752.*x+14199788245258112.*y-1122852841901814912.*z,3)-(2560.*(2304705299858575630109.*x-256.*(204828849006588248100.+19508530860149228990861.*y-2445924471668591306496.*z))*(-29205293090710790323990469408790736.+212589517464418508578145671300087.*x+1750806894610755007047140949242022912.*z))/pow(5042560366642267.*x-256.*(2446745837411900.+4901398098088043.*y-144207654645973248.*z),3)-(160.*(3266813047619306699872.+716026618045942942760.*x-243780804476456624597.*y+718216181537563796776.*z)*(52529278437993151034132605337909.*x-4*(8646336391489439377118003754263.+39602745269819371968458588313429.*z)))/pow(50159316775994592.+36243094308305160.*x-4827156544231217.*y-52318895858217464.*z,3)))/125.;&#xA;value(2)=(128.*((-24576.*(3839508863935892182987929073642496.+36103009879073133562313702394913733.*x-87732961555209684260488911369472.*y)*(24610976415716501050652227.*x-256.*(-10153609683556422184100.+374519398571124540883.*y-4145573659500944095488.*z)))/pow(5042560366642267.*x-256.*(2446745837411900.+4901398098088043.*y-144207654645973248.*z),3)-(30720.*(65108728870058843312625047943313.*x-256.*(4791937744017588738333042319232.+569924119339438478856491194414721.*y))*(2304705299858575630109.*x-256.*(204828849006588248100.+19508530860149228990861.*y-2445924471668591306496.*z)))/pow(5042560366642267.*x-256.*(2446745837411900.+4901398098088043.*y-144207654645973248.*z),3)+(256.*(650985307933227267490679218098413.+935767027021514282821089562931792.*x+12859172907478119575029190058251392.*y)*(9439334964924689507817.+17499514376929345709248.*x+187907876794815451253888.*y-21704870055089718153088.*z))/pow(-3075770275504817.+198931044892562752.*x+14199788245258112.*y-1122852841901814912.*z,3)+(1280.*(114748411888321695540849692963124.+110442377985916695620550654636800.*x+775672512286952418453853865599205.*y)*(4157117722725769078952.+4534359335248895646832.*x+26193979470458655189977.*y-2382852476120229696128.*z))/pow(45070329471431608.+130124049256651728.*x-5583613021604317.*y-387630670566282112.*z,3)+(1600.*(100744894915663705876272277122960.+74302925512671884052557401907120.*x+343788061485767567210745697763531.*y)*(1717566388539311579248.*x+7025931019459451548321.*y+48.*(46537098413809906919.-8301700878138964680.*z)))/pow(85141430232132048.+97951351741329392.*x-8855616621991191.*y-199920422688690560.*z,3)+(16.*(72249495731635781189477972681776.+39691308285862330678445510678381.*x+125252403980353077736842003056195.*y)*(22670037111266004087968.+12461845278544574559640.*x+39219302812923818032157.*y-46563087562792926056.*z))/pow(50159316775994592.+36243094308305160.*x-4827156544231217.*y-52318895858217464.*z,3)+(640.*(505227745581172894057712966825000.+155010006988462124695347547225138.*x-39602745269819371968458588313429.*y)*(3266813047619306699872.+716026618045942942760.*x-243780804476456624597.*y+718216181537563796776.*z))/pow(50159316775994592.+36243094308305160.*x-4827156544231217.*y-52318895858217464.*z,3)+(2*(356681541401645116923690413208956.+126814067043212099223976834718490.*x+191427609122898840477329914007915.*y)*(9101665097092871812176.+3063507166600182944940.*x+6945927557350563805665.*y+1052001549322007294950.*z))/pow(46366672189358032.+18896234711237580.*x-3927118781169095.*y-14705346416259850.*z,3)+(8*(301993014170585471859024964195112.+134962043561465977901954677856080.*x+309197308873592242001670976702725.*y)*(39553725461800043367392.+17203831108841472538824.*x+45483386678520344593037.*y+2703260049547565568088.*z))/pow(92856945980914656.+51329763147513032.*x-8586501277743859.*y-56199770659759016.*z,3)+(128.*(4874430224431350455160317539284048.+1942615285518540483044478359410032.*x-372364031472286149332017066304111.*y)*(45619694076424722199344.+14936846773318822792976.*x-3365788117861218576473.*y+10130491989577935272320.*z))/pow(85141430232132048.+97951351741329392.*x-8855616621991191.*y-199920422688690560.*z,3)+((1486971442137244004077030949061728.+322769103831727073598968320899320.*x-117921178154991189983038750880755.*y)*(61468981380127448102256.+5328427636421850183140.*x-4647710007810227520885.*y+13344414478836548348450.*z))/pow(46366672189358032.+18896234711237580.*x-3927118781169095.*y-14705346416259850.*z,3)+(512.*(3005184872892536482128059816733656.+1654842388128247497540371661628560.*x-221674333970538545014500438709313.*y)*(61889933231497708820968.+30294916915069669525488.*x-4484037822343607626207.*y+13934625423713945278848.*z))/pow(45070329471431608.+130124049256651728.*x-5583613021604317.*y-387630670566282112.*z,3)+(192.*(137644881571986015841084811827840.+35619205657210451197984743698883.*x-10947203161441300431632175319262.*y)*(88457226224862447127008.+13504083955712971035976.*x-6622138801690554356387.*y+19322683651036147287512.*z))/pow(92856945980914656.+51329763147513032.*x-8586501277743859.*y-56199770659759016.*z,3)+(64.*(13728575451141247570683309821008705.+13111763174706011627610159037098688.*x-935548712435961241962462081828256.*y)*(801790542801929135637671.+732048260009923946735424.*x-56975701334774517040256.*y+187552638032246240630656.*z))/pow(-3075770275504817.+198931044892562752.*x+14199788245258112.*y-1122852841901814912.*z,3)))/125.;&#xA;&#xA;// This will work: all long interger being written into a string ( but there are precisiion loss)&#xA;//value(0)=(256*((256*(&quot;-24610976415716501050652227&quot;*x+256*(&quot;-10153609683556422184100&quot;+&quot;374519398571124540883&quot;*y-&quot;4145573659500944095488&quot;*z))*(&quot;29427736469514379027531261659072347&quot;+&quot;58899562724319710108573382000184640&quot;*y-&quot;1732944474195510410991057714955859184&quot;*z))/pow(&quot;5042560366642267&quot;*x-256*(&quot;2446745837411900&quot;+&quot;4901398098088043&quot;*y-&quot;144207654645973248&quot;*z),3)-(256*(&quot;-308518681989548429992935348850261&quot;+&quot;41445095210006425938788783390458&quot;*y-&quot;1638970396838251453451269879637336&quot;*z)*(&quot;-801790542801929135637671&quot;-&quot;732048260009923946735424&quot;*x+&quot;56975701334774517040256&quot;*y-&quot;187552638032246240630656&quot;*z))/pow(&quot;-3075770275504817&quot;+&quot;198931044892562752&quot;*x+&quot;14199788245258112&quot;*y-&quot;1122852841901814912&quot;*z,3)+(5*(&quot;-89303793175477833893354121208000&quot;+&quot;6533090911353242906294143748495&quot;*y-&quot;32276910383172707359896832089932&quot;*z)*(&quot;-61468981380127448102256&quot;-&quot;5328427636421850183140&quot;*x+&quot;4647710007810227520885&quot;*y-&quot;13344414478836548348450&quot;*z))/pow(&quot;-46366672189358032&quot;-&quot;18896234711237580&quot;*x+&quot;3927118781169095&quot;*y+&quot;14705346416259850&quot;*z,3)-(3*(&quot;9101665097092871812176&quot;+&quot;3063507166600182944940&quot;*x+&quot;6945927557350563805665&quot;*y+&quot;1052001549322007294950&quot;*z)*(&quot;19493858980629008651267653094056&quot;+&quot;93282964805436900100617577630195&quot;*y+&quot;42271355681070699741325611572830&quot;*z))/pow(&quot;46366672189358032&quot;+&quot;18896234711237580&quot;*x-&quot;3927118781169095&quot;*y-&quot;14705346416259850&quot;*z,3)-(4*(&quot;39553725461800043367392&quot;+&quot;17203831108841472538824&quot;*x+&quot;45483386678520344593037&quot;*y+&quot;2703260049547565568088&quot;*z)*(&quot;52830583937680669669892057655944&quot;+&quot;303023948138837354463602341532495&quot;*y+&quot;134962043561465977901954677856080&quot;*z))/pow(&quot;92856945980914656&quot;+&quot;51329763147513032&quot;*x-&quot;8586501277743859&quot;*y-&quot;56199770659759016&quot;*z,3)-((&quot;22670037111266004087968&quot;+&quot;12461845278544574559640&quot;*x+&quot;39219302812923818032157&quot;*y-&quot;46563087562792926056&quot;*z)*(&quot;95973949246309465842551069546976&quot;+&quot;723429769797021053206211106031819&quot;*y+&quot;317530466286898645427564085427048&quot;*z))/pow(&quot;50159316775994592&quot;+&quot;36243094308305160&quot;*x-&quot;4827156544231217&quot;*y-&quot;52318895858217464&quot;*z,3)-(80*(&quot;4157117722725769078952&quot;+&quot;4534359335248895646832&quot;*x+&quot;26193979470458655189977&quot;*y-&quot;2382852476120229696128&quot;*z)*(&quot;205429639975670471114284923188348&quot;+&quot;2095815907391732802212116237430935&quot;*y+&quot;883539023887333564964405237094400&quot;*z))/pow(&quot;45070329471431608&quot;+&quot;130124049256651728&quot;*x-&quot;5583613021604317&quot;*y-&quot;387630670566282112&quot;*z,3)-(16*(&quot;9439334964924689507817&quot;+&quot;17499514376929345709248&quot;*x+&quot;187907876794815451253888&quot;*y-&quot;21704870055089718153088&quot;*z)*(&quot;943164674716649969807523653958385&quot;+&quot;18130967224506023673179633045358720&quot;*y+&quot;7486136216172114262568716503454336&quot;*z))/pow(&quot;-3075770275504817&quot;+&quot;198931044892562752&quot;*x+&quot;14199788245258112&quot;*y-&quot;1122852841901814912&quot;*z,3)+(80*(&quot;2304705299858575630109&quot;*x-256*(&quot;204828849006588248100&quot;+&quot;19508530860149228990861&quot;*y-&quot;2445924471668591306496&quot;*z))*(&quot;-179928369646271075844345534739549&quot;+&quot;3401432279430696137250330740801392&quot;*y+&quot;12500875943051297916024009205116096&quot;*z))/pow(&quot;5042560366642267&quot;*x-256*(&quot;2446745837411900&quot;+&quot;4901398098088043&quot;*y-&quot;144207654645973248&quot;*z),3)+(80*(&quot;-805507884940017483975376678503744&quot;+&quot;52529278437993151034132605337909&quot;*y-&quot;620040027953848498781390188900552&quot;*z)*(&quot;-716026618045942942760&quot;*x+&quot;243780804476456624597&quot;*y-8*(&quot;408351630952413337484&quot;+&quot;89777022692195474597&quot;*z)))/pow(&quot;-50159316775994592&quot;-&quot;36243094308305160&quot;*x+&quot;4827156544231217&quot;*y+&quot;52318895858217464&quot;*z,3)+(768*(&quot;61889933231497708820968&quot;+&quot;30294916915069669525488&quot;*x-&quot;4484037822343607626207&quot;*y+&quot;13934625423713945278848&quot;*z)*(&quot;16858970779944867265671037333379&quot;*y-176*(&quot;1546216290476124632111328928258&quot;+&quot;3134171189636832381705249359145&quot;*z)))/pow(&quot;45070329471431608&quot;+&quot;130124049256651728&quot;*x-&quot;5583613021604317&quot;*y-&quot;387630670566282112&quot;*z,3)-(40*(&quot;1717566388539311579248&quot;*x+&quot;7025931019459451548321&quot;*y+48*(&quot;46537098413809906919&quot;-&quot;8301700878138964680&quot;*z))*(&quot;3434616943638241443585000648954199&quot;*y+320*(&quot;1107265969195848092307625165761&quot;+&quot;4643932844541992753284837619195&quot;*z)))/pow(&quot;85141430232132048&quot;+&quot;97951351741329392&quot;*x-&quot;8855616621991191&quot;*y-&quot;199920422688690560&quot;*z,3)+(12*(&quot;88457226224862447127008&quot;+&quot;13504083955712971035976&quot;*x-&quot;6622138801690554356387&quot;*y+&quot;19322683651036147287512&quot;*z)*(&quot;36451820000039413375829754767131&quot;*y-8*(&quot;66864837166560711793644210325852&quot;+&quot;35619205657210451197984743698883&quot;*z)))/pow(&quot;92856945980914656&quot;+&quot;51329763147513032&quot;*x-&quot;8586501277743859&quot;*y-&quot;56199770659759016&quot;*z,3)+(512*(&quot;45619694076424722199344&quot;+&quot;14936846773318822792976&quot;*x-&quot;3365788117861218576473&quot;*y+&quot;10130491989577935272320&quot;*z)*(&quot;12048859085295019197936041733505&quot;*y-6*(&quot;32519187452933223586671104614156&quot;+&quot;40471151781636260063426632487709&quot;*z)))/pow(&quot;85141430232132048&quot;+&quot;97951351741329392&quot;*x-&quot;8855616621991191&quot;*y-&quot;199920422688690560&quot;*z,3)))/125;&#xA;//value(1)=(128*((32768*(&quot;24610976415716501050652227&quot;*x-256*(&quot;-10153609683556422184100&quot;+&quot;374519398571124540883&quot;*y-&quot;4145573659500944095488&quot;*z))*(&quot;98990697209366584150952278657452&quot;+&quot;920305667567495470446459093752885&quot;*x-&quot;65799721166407263195366683527104&quot;*z))/pow(&quot;5042560366642267&quot;*x-256*(&quot;2446745837411900&quot;+&quot;4901398098088043&quot;*y-&quot;144207654645973248&quot;*z),3)+(1024*(&quot;-10864227594859409007678067839115&quot;+&quot;566592725765813239786863532667460&quot;*x-&quot;3214793226869529893757297514562848&quot;*z)*(&quot;9439334964924689507817&quot;+&quot;17499514376929345709248&quot;*x+&quot;187907876794815451253888&quot;*y-&quot;21704870055089718153088&quot;*z))/pow(&quot;-3075770275504817&quot;+&quot;198931044892562752&quot;*x+&quot;14199788245258112&quot;*y-&quot;1122852841901814912&quot;*z,3)+(40*(&quot;2938923392457131154149055759247753&quot;+&quot;8383263629566931208848464949723740&quot;*x-&quot;24821520393182477390523323699174560&quot;*z)*(&quot;4157117722725769078952&quot;+&quot;4534359335248895646832&quot;*x+&quot;26193979470458655189977&quot;*y-&quot;2382852476120229696128&quot;*z))/pow(&quot;45070329471431608&quot;+&quot;130124049256651728&quot;*x-&quot;5583613021604317&quot;*y-&quot;387630670566282112&quot;*z,3)+(80*(&quot;1717566388539311579248&quot;*x+&quot;7025931019459451548321&quot;*y+48*(&quot;46537098413809906919&quot;-&quot;8301700878138964680&quot;*z))*(&quot;3017477155357435955713408172820441&quot;+&quot;3434616943638241443585000648954199&quot;*x-&quot;6875761229715351344214913955270620&quot;*z))/pow(&quot;85141430232132048&quot;+&quot;97951351741329392&quot;*x-&quot;8855616621991191&quot;*y-&quot;199920422688690560&quot;*z,3)+(2*(&quot;1013986939222028224203834326214704&quot;+&quot;723429769797021053206211106031819&quot;*x-&quot;1002019231842824621894736024449560&quot;*z)*(&quot;22670037111266004087968&quot;+&quot;12461845278544574559640&quot;*x+&quot;39219302812923818032157&quot;*y-&quot;46563087562792926056&quot;*z))/pow(&quot;50159316775994592+36243094308305160&quot;*x-&quot;4827156544231217&quot;*y-&quot;52318895858217464&quot;*z,3)+(2*(&quot;698833722744934775627393528218146&quot;+&quot;279848894416310700301852732890585&quot;*x-&quot;191427609122898840477329914007915&quot;*z)*(&quot;9101665097092871812176&quot;+&quot;3063507166600182944940&quot;*x+&quot;6945927557350563805665&quot;*y+&quot;1052001549322007294950&quot;*z))/pow(&quot;46366672189358032&quot;+&quot;18896234711237580&quot;*x-&quot;3927118781169095&quot;*y-&quot;14705346416259850&quot;*z,3)+(8*(&quot;557016173590538671691101855964863&quot;+&quot;303023948138837354463602341532495&quot;*x-&quot;309197308873592242001670976702725&quot;*z)*(&quot;39553725461800043367392&quot;+&quot;17203831108841472538824&quot;*x+&quot;45483386678520344593037&quot;*y+&quot;2703260049547565568088&quot;*z))/pow(&quot;92856945980914656&quot;+&quot;51329763147513032&quot;*x-&quot;8586501277743859&quot;*y-&quot;56199770659759016&quot;*z,3)-(128*(&quot;-57335208466953058729715954197164&quot;+&quot;96390872682360153583488333868040&quot;*x-&quot;372364031472286149332017066304111&quot;*z)*(&quot;45619694076424722199344&quot;+&quot;14936846773318822792976&quot;*x-&quot;3365788117861218576473&quot;*y+&quot;10130491989577935272320&quot;*z))/pow(&quot;85141430232132048&quot;+&quot;97951351741329392&quot;*x-&quot;8855616621991191&quot;*y-&quot;199920422688690560&quot;*z,3)-(5*(&quot;-5058036108182894712997605343704&quot;+&quot;13066181822706485812588287496990&quot;*x-&quot;23584235630998237996607750176151&quot;*z)*(&quot;61468981380127448102256&quot;+&quot;5328427636421850183140&quot;*x-&quot;4647710007810227520885&quot;*y+&quot;13344414478836548348450&quot;*z))/pow(&quot;46366672189358032&quot;+&quot;18896234711237580&quot;*x-&quot;3927118781169095&quot;*y-&quot;14705346416259850&quot;*z,3)-(256*(&quot;-35027435322808897803896166913833&quot;+&quot;101153824679669203594026224000274&quot;*x-&quot;443348667941077090029000877418626&quot;*z)*(&quot;61889933231497708820968&quot;+&quot;30294916915069669525488&quot;*x-&quot;4484037822343607626207&quot;*y+&quot;13934625423713945278848&quot;*z))/pow(&quot;45070329471431608&quot;+&quot;130124049256651728&quot;*x-&quot;5583613021604317&quot;*y-&quot;387630670566282112&quot;*z,3)-(24*(&quot;-23539469566855513950637813409344&quot;+&quot;36451820000039413375829754767131&quot;*x-&quot;87577625291530403453057402554096&quot;*z)*(&quot;88457226224862447127008&quot;+&quot;13504083955712971035976&quot;*x-&quot;6622138801690554356387&quot;*y+&quot;19322683651036147287512&quot;*z))/pow(&quot;92856945980914656&quot;+&quot;51329763147513032&quot;*x-&quot;8586501277743859&quot;*y-&quot;56199770659759016&quot;*z,3)-(112*(&quot;97743545586690977941666831119873&quot;+&quot;189463292388600804291605866927808&quot;*x-&quot;534599264249120709692835475330432&quot;*z)*(&quot;801790542801929135637671&quot;+&quot;732048260009923946735424&quot;*x-&quot;56975701334774517040256&quot;*y+&quot;187552638032246240630656&quot;*z))/pow(&quot;-3075770275504817&quot;+&quot;198931044892562752&quot;*x+&quot;14199788245258112&quot;*y-&quot;1122852841901814912&quot;*z,3)-(2560*(&quot;2304705299858575630109&quot;*x-256*(&quot;204828849006588248100&quot;+&quot;19508530860149228990861&quot;*y-&quot;2445924471668591306496&quot;*z))*(&quot;-29205293090710790323990469408790736&quot;+&quot;212589517464418508578145671300087&quot;*x+&quot;1750806894610755007047140949242022912&quot;*z))/pow(&quot;5042560366642267&quot;*x-256*(&quot;2446745837411900&quot;+&quot;4901398098088043&quot;*y-&quot;144207654645973248&quot;*z),3)-(160*(&quot;3266813047619306699872&quot;+&quot;716026618045942942760&quot;*x-&quot;243780804476456624597&quot;*y+&quot;718216181537563796776&quot;*z)*(&quot;52529278437993151034132605337909&quot;*x-4*(&quot;8646336391489439377118003754263&quot;+&quot;39602745269819371968458588313429&quot;*z)))/pow(50159316775994592+&quot;36243094308305160&quot;*x-&quot;4827156544231217&quot;*y-&quot;52318895858217464&quot;*z,3)))/125;&#xA;//value(2)=(128*((-24576*(&quot;3839508863935892182987929073642496&quot;+&quot;36103009879073133562313702394913733&quot;*x-&quot;87732961555209684260488911369472&quot;*y)*(&quot;24610976415716501050652227&quot;*x-256*(&quot;-10153609683556422184100&quot;+&quot;374519398571124540883&quot;*y-&quot;4145573659500944095488&quot;*z)))/pow(&quot;5042560366642267&quot;*x-256*(&quot;2446745837411900&quot;+&quot;4901398098088043&quot;*y-&quot;144207654645973248&quot;*z),3)-(30720*(&quot;65108728870058843312625047943313&quot;*x-256*(&quot;4791937744017588738333042319232&quot;+&quot;569924119339438478856491194414721&quot;*y))*(&quot;2304705299858575630109&quot;*x-256*(&quot;204828849006588248100&quot;+&quot;19508530860149228990861&quot;*y-&quot;2445924471668591306496&quot;*z)))/pow(&quot;5042560366642267&quot;*x-256*(&quot;2446745837411900&quot;+&quot;4901398098088043&quot;*y-&quot;144207654645973248&quot;*z),3)+(256*(&quot;650985307933227267490679218098413&quot;+&quot;935767027021514282821089562931792&quot;*x+&quot;12859172907478119575029190058251392&quot;*y)*(&quot;9439334964924689507817&quot;+&quot;17499514376929345709248&quot;*x+&quot;187907876794815451253888&quot;*y-&quot;21704870055089718153088&quot;*z))/pow(&quot;-3075770275504817&quot;+&quot;198931044892562752&quot;*x+&quot;14199788245258112&quot;*y-&quot;1122852841901814912&quot;*z,3)+(1280*(&quot;114748411888321695540849692963124&quot;+&quot;110442377985916695620550654636800&quot;*x+&quot;775672512286952418453853865599205&quot;*y)*(&quot;4157117722725769078952&quot;+&quot;4534359335248895646832&quot;*x+&quot;26193979470458655189977&quot;*y-&quot;2382852476120229696128&quot;*z))/pow(45070329471431608+&quot;130124049256651728&quot;*x-&quot;5583613021604317&quot;*y-&quot;387630670566282112&quot;*z,3)+(1600*(&quot;100744894915663705876272277122960&quot;+&quot;74302925512671884052557401907120&quot;*x+&quot;343788061485767567210745697763531&quot;*y)*(&quot;1717566388539311579248&quot;*x+&quot;7025931019459451548321&quot;*y+48*(&quot;46537098413809906919&quot;-&quot;8301700878138964680&quot;*z)))/pow(&quot;85141430232132048&quot;+&quot;97951351741329392&quot;*x-&quot;8855616621991191&quot;*y-&quot;199920422688690560&quot;*z,3)+(16*(&quot;72249495731635781189477972681776&quot;+&quot;39691308285862330678445510678381&quot;*x+&quot;125252403980353077736842003056195&quot;*y)*(&quot;22670037111266004087968&quot;+&quot;12461845278544574559640&quot;*x+&quot;39219302812923818032157&quot;*y-&quot;46563087562792926056&quot;*z))/pow(&quot;50159316775994592&quot;+&quot;36243094308305160&quot;*x-&quot;4827156544231217&quot;*y-&quot;52318895858217464&quot;*z,3)+(640*(&quot;505227745581172894057712966825000&quot;+&quot;155010006988462124695347547225138&quot;*x-&quot;39602745269819371968458588313429&quot;*y)*(&quot;3266813047619306699872&quot;+&quot;716026618045942942760&quot;*x-&quot;243780804476456624597&quot;*y+&quot;718216181537563796776&quot;*z))/pow(&quot;50159316775994592&quot;+&quot;36243094308305160&quot;*x-&quot;4827156544231217&quot;*y-&quot;52318895858217464&quot;*z,3)+(2*(&quot;356681541401645116923690413208956&quot;+&quot;126814067043212099223976834718490&quot;*x+&quot;191427609122898840477329914007915&quot;*y)*(&quot;9101665097092871812176&quot;+&quot;3063507166600182944940&quot;*x+&quot;6945927557350563805665&quot;*y+&quot;1052001549322007294950&quot;*z))/pow(&quot;46366672189358032&quot;+&quot;18896234711237580&quot;*x-&quot;3927118781169095&quot;*y-&quot;14705346416259850&quot;*z,3)+(8*(&quot;301993014170585471859024964195112&quot;+&quot;134962043561465977901954677856080&quot;*x+&quot;309197308873592242001670976702725&quot;*y)*(&quot;39553725461800043367392&quot;+&quot;17203831108841472538824&quot;*x+&quot;45483386678520344593037&quot;*y+&quot;2703260049547565568088&quot;*z))/pow(&quot;92856945980914656&quot;+&quot;51329763147513032&quot;*x-&quot;8586501277743859&quot;*y-&quot;56199770659759016&quot;*z,3)+(128*(&quot;4874430224431350455160317539284048&quot;+&quot;1942615285518540483044478359410032&quot;*x-&quot;372364031472286149332017066304111&quot;*y)*(&quot;45619694076424722199344&quot;+&quot;14936846773318822792976&quot;*x-&quot;3365788117861218576473&quot;*y+&quot;10130491989577935272320&quot;*z))/pow(&quot;85141430232132048&quot;+&quot;97951351741329392&quot;*x-&quot;8855616621991191&quot;*y-&quot;199920422688690560&quot;*z,3)+((&quot;1486971442137244004077030949061728&quot;+&quot;322769103831727073598968320899320&quot;*x-&quot;117921178154991189983038750880755&quot;*y)*(&quot;61468981380127448102256&quot;+&quot;5328427636421850183140&quot;*x-&quot;4647710007810227520885&quot;*y+&quot;13344414478836548348450&quot;*z))/pow(&quot;46366672189358032&quot;+&quot;18896234711237580&quot;*x-&quot;3927118781169095&quot;*y-&quot;14705346416259850&quot;*z,3)+(512*(&quot;3005184872892536482128059816733656&quot;+&quot;1654842388128247497540371661628560&quot;*x-&quot;221674333970538545014500438709313&quot;*y)*(&quot;61889933231497708820968&quot;+&quot;30294916915069669525488&quot;*x-&quot;4484037822343607626207&quot;*y+&quot;13934625423713945278848&quot;*z))/pow(&quot;45070329471431608&quot;+&quot;130124049256651728&quot;*x-&quot;5583613021604317&quot;*y-&quot;387630670566282112&quot;*z,3)+(192*(&quot;137644881571986015841084811827840&quot;+&quot;35619205657210451197984743698883&quot;*x-&quot;10947203161441300431632175319262&quot;*y)*(&quot;88457226224862447127008&quot;+&quot;13504083955712971035976&quot;*x-&quot;6622138801690554356387&quot;*y+&quot;19322683651036147287512&quot;*z))/pow(&quot;92856945980914656&quot;+&quot;51329763147513032&quot;*x-&quot;8586501277743859&quot;*y-&quot;56199770659759016&quot;*z,3)+(64*(&quot;13728575451141247570683309821008705&quot;+&quot;13111763174706011627610159037098688&quot;*x-&quot;935548712435961241962462081828256&quot;*y)*(&quot;801790542801929135637671&quot;+&quot;732048260009923946735424&quot;*x-&quot;56975701334774517040256&quot;*y+&quot;187552638032246240630656&quot;*z))/pow(&quot;-3075770275504817&quot;+&quot;198931044892562752&quot;*x+&quot;14199788245258112&quot;*y-&quot;1122852841901814912&quot;*z,3)))/125; &#xA;&#xA;&#xA;return value;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6693" OwnerDisplayName="LCFactorization" LastActivityDate="2013-11-30T17:00:51.950" Title="How to avoid precision loss when handling very long integer constant involved multiple precision computation?" Tags="|c++|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10169" PostTypeId="2" ParentId="10166" CreationDate="2013-11-28T15:52:35.503" Score="2" Body="&lt;p&gt;You cannot use the quadrature points to sample the Dirac data. You would either get zero or infinity as the solution. You could choose to use a smooth approximation (e.g. a narrow Gaussian) of the source term if you don't have access to the piece of code performing the integration over elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can add a &quot;Dirac-loop&quot; to your FEM code, then you can find the entries $f_i$ of the rhs of your linear system $\mathrm{K} \mathbf{u} = \mathbf{f}$, by directly evaluating your shape functions at the dirac points, i.e. $f_i = \int \delta_{\xi_0} N_i = N_i(\xi_0)$. For this, you'd have to determine the local coordinate $\xi_0$ of the dirac load. This can be done by first detecting on which element the dirac load falls (see convex hull algorithms) and then inverting the geometric map of that element either directly (straight elements) or by Newton iteration (curved elements).&lt;/p&gt;&#xA;" OwnerUserId="5378" LastActivityDate="2013-11-28T15:52:35.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10170" PostTypeId="1" CreationDate="2013-11-28T17:57:23.210" Score="4" ViewCount="1469" Body="&lt;p&gt;There is one 3D-diffusion IBVP for sphere (Dirichlet problem with zero conditions on the surface of sphere).&#xA;The equation has the following view:&#xA;$$\frac{\partial u}{\partial t}=\operatorname{div}\left(A(r,\varphi,\psi)\nabla u(r,\varphi,\psi,t)\right)+f(r,\varphi,\psi,t) ,$$ where the matrix $A$ is skew with variable coefficients:&#xA;$$\left(\matrix{1 \quad a \quad c\\-a \quad 1 \quad b\\-c\quad -b \quad 1 }\right) $$&#xA;I've written a numerical scheme for this problem, but the program works strangely: if all non-diagonal coefficients are zeros, everything is cool, but when at least one coefficient is non-zero, my scheme begins to show great error in neighbourhood of points $\varphi=0$ and $\varphi=\pi$. I've checked program many times, moreover, I've shown it to expert --he said, that program code is correct. He also said that this problem with big errors is connected with singularities in coefficients of terms $\frac{\partial u}{\partial r},\frac{\partial u}{\partial \varphi},\frac{\partial u}{\partial \psi}$. Has anyone faced with similar problems? How can I avoid those singularities?&lt;img src=&quot;https://i.stack.imgur.com/zDWSg.jpg&quot; alt=&quot;Example of great error&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is one example of error. The surfaces' equation is $u(r,\varphi,\psi,t)=\cos(t)(1-r^2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[upd] Here is my numerical scheme:&#xA;$$\frac{u^{n+1}_{i,j,k}-u^{n}_{i,j,k}}{\Delta t}=AA_{i,j,k}\frac{u^{n+1}_{i+1,j,k}-u^{n+1}_{i-1,j,k}}{2\Delta r}+BB_{i,j,k}\frac{u^{n+1}_{i,j+1,k}-u^{n+1}_{i,j-1,k}}{2\Delta \varphi}+CC_{i,j,k}\frac{u^{n+1}_{i,j,k+1}-u^{n+1}_{i,k,k-1}}{2\Delta \psi}+\frac{u^{n+1}_{i+1,j,k}-2u^{n+1}_{i,j,k}+u^{n+1}_{i-1,j,k}}{\Delta r^2}+\frac{u^{n+1}_{i,j+1,k}-2u^{n+1}_{i,j,k}+u^{n+1}_{i,j-1,k}}{r_{i}^{2}\Delta \varphi^2}+\frac{u^{n+1}_{i,j,k+1}-2u^{n+1}_{i,j,k}+u^{n+1}_{i,j,k-1}}{r_{i}^{2}\sin^{2}\varphi_{j}\Delta \psi^2}+f_{i,j,k} $$&#xA;Coefficients $AA,BB,CC$ are the following:&#xA;$$AA_{i,j,k}=\frac{2}{r_{i}}+a_{i,j,k}\frac{\cot \varphi_{j}}{r_{i}}+\frac{1}{r_{i}}\left(\frac{\partial a}{\partial \varphi}\right)_{i,j,k}-\frac{1}{r_{i}\sin\varphi_{j}}\left(\frac{\partial c}{\partial \psi}\right)_{i,j,k} $$&#xA;$$BB_{i,j,k}=-\frac{a_{i,j,k}}{r_{i}^2}+\frac{\cot\varphi_{j}}{r_{i}^2}-\frac{1}{r_{i}}\left(\frac{\partial a}{\partial r}\right)_{i,j,k}+\frac{1}{r_{i}^2\sin\varphi_{j}}\left(\frac{\partial b}{\partial \psi}\right)_{i,j,k} $$&#xA;$$CC_{i,j,k}=\frac{c_{i,j,k}}{r_{i}^2\sin\varphi_{j}}+\frac{1}{r_{i}\sin\varphi_{j}}\left(\frac{\partial c}{\partial r}\right)_{i,j,k}-\frac{1}{r_{i}^2\sin\varphi_{j}}\left(\frac{\partial b}{\partial \varphi}\right)_{i,j,k} $$&lt;/p&gt;&#xA;" OwnerUserId="4676" LastEditorUserId="4676" LastEditDate="2013-12-03T11:25:00.647" LastActivityDate="2013-12-03T11:25:00.647" Title="Finite difference method (diffusion equation) for 3D spherical case" Tags="|finite-difference|numerical-analysis|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10171" PostTypeId="2" ParentId="10166" CreationDate="2013-11-28T18:37:34.260" Score="3" Body="&lt;p&gt;Since you are creating the mesh specifically for this right-hand side in order to have a correct solution, it would make more sense to have the Dirac located on a &lt;em&gt;node&lt;/em&gt; instead: On the one hand, this is likely to be easier to realize in the mesh generator, and on the other hand the assembly is easier: Since you are using continuous piecewise linear elements and (presumably) nodal basis functions $\psi_i$, you have for $f=\delta_{x_i}$ (where $x_i$ is one of the nodes of your triangulation) that&#xA;$$&#xA;\langle f,\psi_j\rangle_{C^*,C} = \psi_j(x_i) = &#xA;\begin{cases} &#xA;1 &amp;amp;\text{if } i=j,\\&#xA;0 &amp;amp;\text{if } i\neq j,&#xA;\end{cases}&#xA;$$&#xA;i.e., your load vector is just the vector with $1$ in the $i$th component and $0$ else. (This also works if you have linear combinations of Dirac measures located on the nodes. This also explains what happens when you forget the mass matrix on the right hand side...)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2013-11-28T18:37:34.260" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10172" PostTypeId="1" AcceptedAnswerId="10174" CreationDate="2013-11-28T21:46:01.163" Score="4" ViewCount="1029" Body="&lt;p&gt;For the backward Euler discretization in time:&#xA;$$&#xA;\left( \frac{u^{(k)}-u^{(k-1)}}{\Delta t}, v\right) + a(u^{(k)},v) = \ell(v)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a(\cdot,\cdot)$ is the bilinear operator associated with the discretization in space. I’m considering linear elliptic problems in 2D. So I rearrange the above equation:&#xA;$$&#xA;\left(\frac{1}{\Delta t}u^{(k)},v\right) + a(u^{(k)},v) = \ell(v) + \left(\frac{1}{\Delta t}u^{(k-1)},v\right)&#xA;$$&#xA;since $u^{(k-1)}$ is a known quantity from the previous time step. For non-time dependent problems, the convergence in space is typically on the order of $\mathcal{O}(h^{p+1})$, with $p$ being the basis degree. Now if I have a time-dependent problem as given above, how small does my time step have to be in order to test for convergence in space? If I take a large time step, the method is stable but yields error that is large, on the order of $10^{3}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking my time step extremely small yields proper convergence rates, but I don’t think this tells me anything since for small $\Delta t$, the time-dependent terms will dominate and I’m essentially solving&#xA;$$&#xA;\left(\frac{1}{\Delta t}u^{(k)},v\right) = \left(\frac{1}{\Delta t}u^{(k-1)},v\right)&#xA;$$&#xA;and if the time step is small, we can most likely expect $u^{(k)} \approx u^{(k-1)}$, so solving the system will yield the “correct solution”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m just wondering if my implementation error of Backwards euler is incorrect. I know typically we solve the linear system:&#xA;$$&#xA;(\mathbf{M} + \Delta t\mathbf{K})u^{(k)} = \mathbf{M}u^{(k-1)} + \Delta t\mathbf{f}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\mathbf{M}$ and $\mathbf{K}$ being the mass and stiffness matrices and $\mathbf{f}$ being the source term. But is the implementation I gave above incorrect? I originally implemented the first way I listed, then tried the second for comparison but they yield the same result in both cases, so I'm guessing that's not the issue.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2013-11-28T22:25:32.047" LastActivityDate="2013-11-29T07:58:34.090" Title="Backward Euler time step for finite elements" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10173" PostTypeId="2" ParentId="8002" CreationDate="2013-11-29T07:30:24.780" Score="0" Body="&lt;p&gt;You might get some idea from Stevenson's original paper published in FOCM, because while there you have a continuous Galerkin method, the analysis is done with inexact solvers.&lt;/p&gt;&#xA;" OwnerUserId="356" LastActivityDate="2013-11-29T07:30:24.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10174" PostTypeId="2" ParentId="10172" CreationDate="2013-11-29T07:58:34.090" Score="3" Body="&lt;p&gt;Remember that the backward Euler is first-order in time so your error will look something like:&#xA;\begin{equation*}&#xA;\|u-u_{h,\Delta t}\| = c_1 h^{p+1} + c_2 \Delta t + \cdots&#xA;\end{equation*}&#xA;If you want to test for spatial accuracy then $c_2 \Delta t$ needs to be an order smaller than $c_1 h^{p+1}$ on the finest grid and vice versa. If you obtain the theoretical orders for a manufactured solution it confirms that the implementation is correct.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-11-29T07:58:34.090" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10175" PostTypeId="2" ParentId="10142" CreationDate="2013-11-29T08:07:35.173" Score="5" Body="&lt;p&gt;Since you haven't had an answer yet, I'll reformulate my comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Saying that a method is $p$-th order accurate implies that a polynomial manufactured solution of lesser order can be captured exactly. For example, a 2nd order method will represent a linear solution up to machine precision. This has often helped me in finding implementation issues. It's easier and faster than checking convergence rates, although you need to do that afterwards for non-polynomial solutions (sines and such).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider the case of constant basis functions, which is similar but not equal to finite volume methods. In that case, note that the only remaining diffusion term is the penalty term! So it's highly unlikely that a constant $\sigma$ can give satisfactory results for such a wide range of $\kappa$. Search the literature for suggestions about $\sigma(\kappa)$.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2013-11-29T08:07:35.173" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10176" PostTypeId="1" AcceptedAnswerId="10178" CreationDate="2013-11-29T13:19:21.100" Score="1" ViewCount="3365" Body="&lt;p&gt;I'm trying to find the potential given some boundary conditions using the successive over-relaxation method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 2 solutions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-One iterates over all elements and applies the formula &lt;code&gt;field[y,x] = (1-alpha)*field[y,x] + (field[max(y-1,0),x] + field[min(y+1,field.shape[0]-1),x] + field[y,max(x-1,0)] + field[y,min(x+1,field.shape[1]-1)]) * alpha/4&lt;/code&gt; in place. This is slow because it doesn't access memory in a nice way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-The other one, I create 4 matrices shifted in the 4 directions by 1. I Apply the same formula by then adding the matrices up. This however doesn't take into account modifications done during the current iteration. This is significantly faster then the previous one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With alpha = 1.9 the first algorithm converges while the second one doesn't. For alpha = 1.0 both converge but very slowly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me what I'm doing wrong? And how can I fix the fast solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Full code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#! python3&#xA;&#xA;import numpy&#xA;import math&#xA;import time&#xA;&#xA;def solve_laplace(boundary, mask, file = None, alpha = 1.0, threshold = 0.0001):&#xA;    &quot;&quot;&quot;&#xA;    We are using the successive over-relaxation method. We iterate until our solution changes less than some threshold value.&#xA;&#xA;    Vm+1(x,y,...) = alpha*( ((Vm(x-1,y,...) + Vm(x+1,y,...) + Vm(x,y-1,...) + Vm(x,y+1,...) + ...)/(2*nr dimensions) ) + (1-alpha)*Vm(x,y,...)&#xA;    &quot;&quot;&quot;&#xA;&#xA;    dim = boundary.ndim&#xA;&#xA;    threshold = 0.0001&#xA;    field = numpy.zeros_like(boundary)&#xA;    numpy.copyto(field, boundary, casting = &quot;safe&quot;, where = mask)&#xA;    last_diff = float(&quot;infinity&quot;)&#xA;&#xA;    for iter_nr in range(10000):#max number of iterations&#xA;        prev = field.copy() #make a copy of the field at the start of the iteration (python always stores pointers unless you explicitly copy something)&#xA;&#xA;        for d in range(dim): #can be scaled to arbitrary dimensions, using 2D for testing&#xA;&#xA;            #these 2 blocks are hard to follow but they work, read the comments&#xA;            front = prev[tuple(0 if i==d else slice(None) for i in range(dim))] #select front face of cube/whatever&#xA;            front = front[tuple(numpy.newaxis if i==d else slice(None) for i in range(dim))] #prepare it for next step&#xA;            front = numpy.concatenate((front,prev),d) #add it the previous iteration's result&#xA;            front = front[tuple(slice(-1) if i==d else slice(None) for i in range(dim))] #remove the back side of the previous iteration's result&#xA;            #we now have the volume shifted right by 1 pixel, x now corresponds to the x-1 term&#xA;&#xA;            back = prev[tuple(-1 if i==d else slice(None) for i in range(dim))] #select back face of cube/whatever&#xA;            back = back[tuple(numpy.newaxis if i==d else slice(None) for i in range(dim))] #prepare it for next step&#xA;            back = numpy.concatenate((prev,back),d) #add it the previous iteration's result&#xA;            back = back[tuple(slice(1,None) if i==d else slice(None) for i in range(dim))] #remove the front side of the previous iteration's result&#xA;            #we now have the volume shifted left by 1 pixel, x now corresponds to the x+1 term&#xA;&#xA;            field += (front + back) * alpha/(2*dim) #this part of the formula: alpha*( ((Vm(x-1,y,...) + Vm(x+1,y,...) + Vm(x,y-1,...) + Vm(x,y+1,...))/(2*nr dimensions)&#xA;            #numpy.copyto(field, boundary, casting = &quot;safe&quot;, where = mask)&#xA;&#xA;        field -= alpha*prev #this part of the formula: (1-alpha)*Vm(x,y,...)&#xA;        #reset values at boundaries&#xA;        numpy.copyto(field, boundary, casting = &quot;safe&quot;, where = mask) &#xA;&#xA;        #check if the difference is less than threshold&#xA;        average = math.sqrt(numpy.average(field**2)) #sqrt of average of squares, just so i get a positive number&#xA;        diff = math.sqrt(numpy.average((field-prev)**2)) #standard deviation&#xA;&#xA;        if last_diff &amp;lt; diff/average:&#xA;            print(&quot;Solution is diverging.&quot;)&#xA;            break&#xA;&#xA;        if diff/average &amp;lt; threshold:&#xA;            print(&quot;Found solution after&quot;, iter_nr,&quot;iteratiorn.&quot;)&#xA;            break&#xA;&#xA;        last_diff = diff/average&#xA;&#xA;    if file is not None:&#xA;        numpy.save(file,field)&#xA;    return field&#xA;&#xA;&#xA;&#xA;def solve_laplace_slow_2D(boundary, mask, file = None, alpha = 1.9,threshold = 0.0001):&#xA;    &quot;&quot;&quot;&#xA;    We are using the successive over-relaxation method. We iterate until our solution changes less than some threshold value.&#xA;&#xA;    Vm+1(x,y,...) = alpha*( ((Vm(x-1,y,...) + Vm(x+1,y,...) + Vm(x,y-1,...) + Vm(x,y+1,...) + ...)/(2*nr dimensions) ) + (1-alpha)*Vm(x,y,...)&#xA;    &quot;&quot;&quot;&#xA;&#xA;    assert boundary.ndim == 2&#xA;&#xA;    field = numpy.zeros_like(boundary)&#xA;    numpy.copyto(field, boundary, casting = &quot;safe&quot;, where = mask) &#xA;    last_diff = float(&quot;infinity&quot;)&#xA;    start_time = time.time()&#xA;&#xA;    for iter_nr in range(10000):#max number of iterations&#xA;        prev = field.copy()&#xA;        for y in range(field.shape[0]):&#xA;            for x in range(field.shape[1]):&#xA;                if not mask[y,x]:&#xA;                    field[y,x] = (1-alpha)*field[y,x] + (field[max(y-1,0),x] + field[min(y+1,field.shape[0]-1),x] + field[y,max(x-1,0)] + field[y,min(x+1,field.shape[1]-1)]) * alpha/4&#xA;&#xA;        #check if the difference is less than threshold&#xA;        average = math.sqrt(numpy.average(field**2)) #sqrt of average of squares, just so i get a positive number&#xA;        diff = math.sqrt(numpy.average((field-prev)**2)) #standard deviation&#xA;&#xA;        if last_diff &amp;lt; diff/average:&#xA;            print(&quot;Solution is diverging.&quot;)&#xA;            break&#xA;&#xA;        if diff/average &amp;lt; threshold:&#xA;            print(&quot;Found solution after the&quot;, iter_nr,&quot;iteratiorn.&quot;)&#xA;            break&#xA;&#xA;        if time.time() - start_time &amp;gt; 3600:&#xA;            print(&quot;Completed in an hour time at iteration:&quot;, iter_nr)&#xA;            break&#xA;&#xA;        last_diff = diff/average&#xA;&#xA;        #print(time.time() - start_time, iter_nr, last_diff)&#xA;&#xA;    if file is not None:&#xA;        numpy.save(file,field)&#xA;    return field&#xA;&#xA;def test():&#xA;    boundary = numpy.zeros((51,51))&#xA;    boundary[25,25] = 1&#xA;    for i in range(51):&#xA;        boundary[0,i] = -1&#xA;        boundary[50,i] = -1&#xA;        boundary[i,0] = -1&#xA;        boundary[i,50] = -1&#xA;    mask = (boundary != 0)&#xA;&#xA;    print(&quot;Trying fast method:&quot;)&#xA;    solve_laplace(boundary,mask,alpha = 1.5) #diverges&#xA;    print(&quot;Trying slow method:&quot;)&#xA;    solve_laplace_slow_2D(boundary,mask,alpha = 1.5) #converges but is very slow&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6711" LastActivityDate="2015-09-22T16:43:48.837" Title="Successive over-relaxation not converging (when not done in-place)" Tags="|python|approximation|numpy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10177" PostTypeId="1" AcceptedAnswerId="10191" CreationDate="2013-11-29T15:35:23.347" Score="17" ViewCount="5642" Body="&lt;p&gt;Recently I have been comparing different &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/optimize.nonlin.html&quot;&gt;non-linear solvers from scipy&lt;/a&gt; and was particularly impressed with the &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.12.0/reference/optimize.nonlin.html&quot;&gt;Newton-Krylov example in the Scipy Cookbook&lt;/a&gt; in which they solve a second order differential equation equation with non-linear reaction term in about 20 lines of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I modified the example code to solve the non-linear Poisson equation (&lt;a href=&quot;https://engineering.purdue.edu/~ee606/downloads/T3.PDF&quot;&gt;also called the Poisson-Boltzmann equation&lt;/a&gt;, see page 17 in these notes) for semiconductor heterostructures, which has the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d^2\phi}{dx^2} - k(x) \left(p(x,\phi) - n(x,\phi) + N^{+}(x)\right) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is the residual function that is passed to the solver.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an electrostatics problem where $n(x,\phi)$ and $p(x,\phi)$ are nonlinear functions for the form $n_i(x) e^{-(E_i(x,\phi) - E_f)}$. The details here are not important, but the point is that the non-linear function vary exponentially with $\phi$ so the residual function can vary over a huge range ($10^{-6} - 10^{16})$ with a slight change in $\phi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I numerically solve this equation with scipy's Newton-Krylov, but it would never converge (in fact it would always report an error with the calculating the Jacobian). I switched from a &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.newton_krylov.html&quot;&gt;Newton-Krylov&lt;/a&gt; solver to &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fsolve.html&quot;&gt;fsolve&lt;/a&gt; (which is based on MINPACK hybrd) and it worked first time!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there general reasons why Newton-Krylov is not a good fit to certain problems? Do the input equations need to be conditioned somehow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe more information is needed to comment, but why do you think fsolve worked in this case?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2013-11-30T02:21:55.467" LastActivityDate="2017-12-01T14:24:04.370" Title="When is Newton-Krylov not an appropriate solver?" Tags="|python|nonlinear-equations|poisson|scipy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10178" PostTypeId="2" ParentId="10176" CreationDate="2013-11-29T15:52:32.243" Score="5" Body="&lt;p&gt;The &quot;fast&quot; version is (damped/over-relaxed) Jacobi instead of SOR, which is a multiplicative method.  Even with damping factor of 1.0, Jacobi is not guaranteed to converge, as you would see if you applied it to a 1D problem, or added extreme anisotropy to your multi-dimensional problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your SOR (&quot;slow&quot;) implementation is slow because it is written in pure Python rather than because of the memory access.  Indeed, the vectorized code accesses memory more times, and thus is actually worse in terms of memory bandwidth.  But the interpreter is so much slower that it hides the bandwidth effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a fact of life that writing numerical kernels in a dynamic language without a JIT will give slow performance, the variability of which may not correlate with machine characteristics.  You should be able to speed up the looped code using &lt;a href=&quot;http://cython.org&quot;&gt;Cython&lt;/a&gt; or &lt;a href=&quot;http://numba.pydata.org&quot;&gt;Numba&lt;/a&gt;, or just write it in a compiled language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now it is true that a different sort of local vectorization can be applied to Jacobi, but not to SOR.  That can be partly accommodated, especially for SPD problems, by using polynomial acceleration such as Chebyshev (for known spectral bounds) or with Krylov methods (which are spectrally adaptive).  Alternatively, you can use multicolor relaxation (e.g., &quot;red-black Gauss-Seidel&quot;) to have a method with relatively fast guaranteed convergence.  Incorporating into a multigrid algorithm will also greatly speed up your convergence (to a small constant number of iterations independent of problem size).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-11-29T15:52:32.243" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10179" PostTypeId="2" ParentId="10168" CreationDate="2013-11-29T17:14:20.050" Score="3" Body="&lt;p&gt;Libraries like MPFR &amp;amp; GMP  [ &lt;a href=&quot;http://gmplib.org/&quot; rel=&quot;nofollow&quot;&gt;http://gmplib.org/&lt;/a&gt; ] are designed exactly for this requirement.  Native datatypes for C and C++ have defined, fixed limits, GMP, MPFR,and other bignum libraries have extensible limits on precision. Each handled in its own way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MPFR is also such a library, like GMP, it requires that it be installed.  It is designed to have a fixed precision (declared at the beginningand the limits are smaller than GMP), GMP limits can be changed during runtime.  MPFR has native (that is built-in) support for transcendental functions.  Which can be a win if you need them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However you choose to go, read the documentation, do not simply copy somebody else's code and expect good results. &lt;/p&gt;&#xA;" OwnerUserId="3685" LastActivityDate="2013-11-29T17:14:20.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10180" PostTypeId="1" AcceptedAnswerId="10182" CreationDate="2013-11-29T18:55:53.043" Score="0" ViewCount="899" Body="&lt;p&gt;I am trying (using MATLAB) to generate the following image (from the section &lt;strong&gt;4.1 Numerical Examples&lt;/strong&gt;) of Wu Tian Chen research article '&lt;a href=&quot;http://users.encs.concordia.ca/~tian/index_files/Papers/QREI_Wu-Tian-Chen_2012.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Condition-based Maintenance Optimization Using Neural Network-based Health Condition Prediction&lt;/a&gt;':&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Iens.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model is derived from the equation:&#xA;$$&#xA;L(t)=θ′+β′t+ε(t)&#xA;$$&#xA;where $θ′$ is a normal random variable with mean $5$ and variance $1{^2}$, and $β′$ also be a normal random variable with mean $5$ and variance $1.5{^2}$. $ε(t)=σW(t)$ is a centered Brownian motion such that the mean of $ε(t)$ is zero and the variance of $ε(t)$ is  $σ{^2}t$.The parameters are set according to the article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MATLAB code I have written is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Initialization:&#xA;Ts          = 0;&#xA;Te          = 150;&#xA;Tn          = 101;&#xA;mu0         = 5;&#xA;sigma0      = 1;&#xA;mu1         = 5;&#xA;sigma1      = 1.5;&#xA;sigma       = .5;&#xA;D           = 500; % failure threshold&#xA;paths = 50; % Number of paths in accordance with the article and graph&#xA;&#xA;figure, hold on, box on, grid off&#xA;&#xA;t = linspace(Ts, Te, Tn)';&#xA;&#xA;for i = 1:paths &#xA;   % Generate θ':&#xA;    teta1 = randn() * sigma0 + mu0;&#xA;   % Generate β':&#xA;    beta1 = (randn() * sigma1 + mu1) .* t;&#xA;&#xA;   % Generate Brownian path&#xA;    dW = sqrt(Te / Tn) * randn(Tn, 1);&#xA;    W = cumsum(dW, 1); % cumulative sum&#xA;    e_t = sigma * W;&#xA;&#xA;   L = teta1 + beta1 + e_t;   &#xA;   plot(t, L);&#xA;   xlim([Ts, Te])&#xA;   ylim([0, 600])&#xA;&#xA;end&#xA;&#xA;% Draws threshold&#xA;plot([Ts, Te], [D, D], 'k', 'LineWidth', 2.5)&#xA;&#xA;title('Degradation Signal', 'FontWeight', 'bold', 'FontSize', 14);&#xA;xlabel('Time (day)', 'FontSize', 12);&#xA;ylabel('Amplitude', 'FontSize', 12);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output of this code is different from the original one, but I don't understand why. In have already applied the changes from A. Donda, but the plot is still different, due to the abscence of fluctuations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="6713" LastEditorUserId="6713" LastEditDate="2013-11-30T14:32:22.913" LastActivityDate="2014-08-22T18:28:13.093" Title="Degradation Model in MATLAB" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10181" PostTypeId="2" ParentId="10170" CreationDate="2013-11-29T19:44:43.023" Score="4" Body="&lt;p&gt;An elegant approach to avoid singularities of this type was proposed in Journal of Computational Physics, Volume 124, Issue 1, 1 March 1996, Pages 93–114&#xA;'The “Cubed Sphere”: A New Method for the Solution of Partial Differential Equations in Spherical Geometry'.&lt;/p&gt;&#xA;" OwnerUserId="4325" LastActivityDate="2013-11-29T19:44:43.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10182" PostTypeId="2" ParentId="10180" CreationDate="2013-11-29T20:13:57.287" Score="2" Body="&lt;p&gt;First, your code for the linear part is wrong. As far as I understand, $\theta'$ and $\beta'$ vary randomly over instantiations, but are constant over time. To implement this, you have to write&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Generate θ':&#xA;theta1 = randn() * sigma0 + mu0;&#xA;% Generate β':&#xA;beta1 = (randn() * sigma1 + mu1) .* t;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Secondly, there are several strange things in your code to generate a Brownian path: There's an $\exp()$ without motivation, you use the constant &lt;code&gt;0.5&lt;/code&gt; instead of the defined parameter &lt;code&gt;sigma&lt;/code&gt;, and you have an additional factor 0.5 in the code for &lt;code&gt;dW&lt;/code&gt;. IMHO, that section should look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dW = sqrt(Te / Tn) * randn(Tn, 1);&#xA;W = cumsum(dW);&#xA;e_t = sigma * W;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There'd be also a problem here if &lt;code&gt;Ts&lt;/code&gt; is not zero, so I'd recommend to just remove that parameter and replace it by 0 everywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this modified code, the result looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2c5lp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is still not what you aim at. However, I think the parameters you use are not consistent with your plot anyway. For one, if the variance of $\beta'$ is supposed to be 1.5, then you have to write&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sigma1 = sqrt(1.5);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;since $\sigma_1$ is the standard deviation. Moreover, if $\sigma = 0.5$ for the Brownian motion, then the variance of $\epsilon(150)$ is 37.5, and the standard deviation 6.12. The fluctuations in your plot however are quite a bit larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="5492" LastEditorUserId="5492" LastEditDate="2013-11-29T20:22:59.663" LastActivityDate="2013-11-29T20:22:59.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10183" PostTypeId="1" CreationDate="2013-11-29T21:09:16.143" Score="1" ViewCount="197" Body="&lt;p&gt;I am attempting to model the steady state behavior of a cylinder using the finite volume method (FVM) subjected to a variety of boundary conditions in Matlab. First off, I am treating the cylinder as being axisymmetric so I am only determine the temperature profile in the r-Z plane. I set up a 2D grid so that the r coordinate is on the 'x' axis and the Z coordinate is the 'y' axis. Since the cylinder is axisymmetric then the heat flux at r = 0 is 0 (insulated). I am also applying an insulated boundary condition to the bottom surface at Z=0. The side surface has convection with a constant convection coefficient and air temperature of 310 K. The top of the cylinder, z=Z, there is a heat flux of 1000 W/m^2, convection from the air at 310K, and also surface to ambient radiation with the surroundings being 298 K. The initial temperature of cylinder is 298 K (initial guess value). The problem that I am having is that the temperature distribution matches Comsol's results with a grid of 40x40 and a thermal conductivity of 1. When I increases the number of cells to 50x50, 60x60, etc, or if i change the thermal conductivity to something other than 1 then the temperature profile is still what it should be but all the temperature are significantly lower than expected. My control loop in Matlab is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use initial values and problem constants to solve for initial surface temperature of top and side.&lt;/li&gt;&#xA;&lt;li&gt;Use surfacte temperatures and problem geomety to determine aE, aN, aS, aW, aP, and b for every cell in the grid.&lt;/li&gt;&#xA;&lt;li&gt;Use line by line TDMA in both directions to determine the temperature profile.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the error between the new temperature profile and the previous one.&lt;/li&gt;&#xA;&lt;li&gt;Update the surface temperatures and aP and b because of the nonlinear radiation term that uses previous iterate values.&lt;/li&gt;&#xA;&lt;li&gt;Perform line by line TDMA again and check for convergence. If the error is not less than 0.001 then repeat the procedure.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the boundary conditions, I used an energy balance for the top surface and the side surface. The top surface includes surface to ambient radiation and I calculated the surface temperature to be &#xA;$$T_{b}=\frac{\alpha G+hT_{f}+\frac{kT_{p}}{\delta z}+\epsilon \sigma(T_{sur}^{4}+3T_{b}^{*4})}{h+4 \epsilon \sigma T_{b}^{*3}+\frac{k}{\delta z}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also do the same thing with the surface temperature on the side the is experiencing heating through convection. Since the top surface temperature depends on its previous iterate value, I use an initial guess value of 298 for the top surface. I then calculate the initial surface temperature based on that and the equation i provided for the first iteration,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem lies in the fact that the surface tempreature depends on both the thermal conductivity and the grid spacing. When I use many cells like 80x80, the top surface temperature for the first TDMA iteration is almost 298 which is hard to believe because its being heated by a 1000 W/m^2 heat flux and its also being heated by the 310 K air temperature. This causes the first iteration to have very little error so convergence only produces a max temperature of like 325 when it should be 450. But if I use a grid of like 40x40 then the surface temperature for the first iteration is much much higher, and convergence causes the final temperature profile to look almost exactly like the expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same exact thing happens with the thermal conductivity, if i change the value then the initial surface temperatures change dramatically which causes the convergence values to be much lower than the expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like it all has to do with that initial surface temperature but I am not sure how else to treat it. Any help would be amazing as I have spent many nights til 5am trying to figure out how I could fix this. My contact email is in my profile if you would like to see my matlab code of if you would like to talk to me more about my problem.&lt;/p&gt;&#xA;" OwnerUserId="6714" LastActivityDate="2013-11-29T21:09:16.143" Title="Problem with cell size and boundary conditions in transient cylindrical conduction" Tags="|finite-volume|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10184" PostTypeId="1" AcceptedAnswerId="10188" CreationDate="2013-11-29T22:48:01.217" Score="3" ViewCount="1199" Body="&lt;p&gt;The application is a simple non-linear advection diffusion problem (steady state) using DGFEM. My error at each iteration is given by&#xA;$$&#xA;e_{n+1} = ||\mathbf{J}^{-1}(\mathbf{u}_{n})\mathbf{F}(\mathbf{u}_{n})||_{L^{2}(\Omega)} = ||\mathbf{u}_{n+1} - \mathbf{u}_{n}||_{L^{2}(\Omega)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For testing purposes, I'm using an $L^{2}$ projection of the exact solution as an initial guess, i.e. solve:&#xA;$$&#xA;\sum_{K\in\mathcal{M}_{h}}\int_{K}uv = \sum_{K\in\mathcal{M}_{h}}\int_{K}u_{\text{exact}}v \implies \mathbf{M}\mathbf{u}_{0} = \mathbf{f}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see perfect quadratic convergence for the steady state nonlinear diffusion problem, and for most cases of nonlinear advection diffusion. For some cases it takes a VERY large number of iterations before I see quadratic convergence. It's perplexing since the initial guess is essentially the solution I want to end up at...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Does the advective term mess with the convergence rate of Newton's Method? Or is this just a sign that I've computed the Jacobian part of the advective terms incorrectly? I know the quadratic rate is only locally, but seeing as how the initial guess IS the solution...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If I start with an initial guess of $\mathbf{u}_{0}$ using the $L^{2}$ projection as defined above, should the method converge immediately or only in the case of the exact solution being exactly represented by the DG basis? Some numerical experiments point to the latter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is one simple example. The model problem is&#xA;$$&#xA;\nabla\cdot(\mathbf{s}(u)u) - \nabla\cdot(\kappa(u)\nabla u) = f \;\;\;\text{in}\;\Omega\subseteq\mathbb{R}^{2}\\&#xA;u = g_{D} \;\;\;\text{on}\;\partial\Omega&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I take the exact solution $u(x,y) = x(x-1)y(y-1)$, $\mathbf{s} = [1+u,1-u]$ and $\kappa(u) = 1+u$ on the domain $\Omega = [-1,1]^{2}$. The initial guess is taken to be $\mathbf{u}_{0}$ from the $L^{2}$ projection as described above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For basis degree p=1, here is the result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Refinement Level 0&#xA;Iteration: 1, Error: 4.001769e+00&#xA;Iteration: 2, Error: 1.828048e+00&#xA;Iteration: 3, Error: 6.720810e+00&#xA;Iteration: 4, Error: 5.483628e+00&#xA;Iteration: 5, Error: 6.469472e+00&#xA;Iteration: 6, Error: 2.341527e+00&#xA;Iteration: 7, Error: 4.989268e+00&#xA;Iteration: 8, Error: 4.963194e+01&#xA;Iteration: 9, Error: 2.479127e+01&#xA;Iteration: 10, Error: 1.239452e+01&#xA;Iteration: 11, Error: 6.190117e+00&#xA;Iteration: 12, Error: 3.034077e+00&#xA;Iteration: 13, Error: 1.342876e+00&#xA;Iteration: 14, Error: 8.747657e-01&#xA;Iteration: 15, Error: 5.244396e-01&#xA;Iteration: 16, Error: 1.740700e-01&#xA;Iteration: 17, Error: 5.298694e-03&#xA;Iteration: 18, Error: 1.344089e-05&#xA;Iteration: 19, Error: 2.878889e-11&#xA;Iteration: 20, Error: 8.722883e-16&#xA;Iterations for convergence: 20&#xA;&#xA;Refinement Level 1&#xA;Iteration: 1, Error: 1.241453e+00&#xA;Iteration: 2, Error: 4.959870e-01&#xA;Iteration: 3, Error: 1.097905e-01&#xA;Iteration: 4, Error: 6.880627e-03&#xA;Iteration: 5, Error: 5.445979e-05&#xA;Iteration: 6, Error: 3.168481e-09&#xA;Iteration: 7, Error: 1.874668e-15&#xA;Iterations for convergence: 7&#xA;&#xA;Refinement Level 2&#xA;Iteration: 1, Error: 2.627395e-01&#xA;Iteration: 2, Error: 2.936349e-02&#xA;Iteration: 3, Error: 6.427106e-04&#xA;Iteration: 4, Error: 3.024645e-07&#xA;Iteration: 5, Error: 2.557842e-13&#xA;Iterations for convergence: 5&#xA;&#xA;Refinement Level 3&#xA;Iteration: 1, Error: 5.830850e-02&#xA;Iteration: 2, Error: 1.113766e-03&#xA;Iteration: 3, Error: 8.567317e-07&#xA;Iteration: 4, Error: 5.108461e-13&#xA;Iterations for convergence: 4&#xA;&#xA;Refinement Level 4&#xA;Iteration: 1, Error: 1.372896e-02&#xA;Iteration: 2, Error: 6.526761e-05&#xA;Iteration: 3, Error: 2.784184e-09&#xA;Iteration: 4, Error: 1.217915e-16&#xA;Iterations for convergence: 4&#xA;&#xA;Refinement Level 5&#xA;Iteration: 1, Error: 3.357581e-03&#xA;Iteration: 2, Error: 4.495462e-06&#xA;Iteration: 3, Error: 1.148118e-11&#xA;Iteration: 4, Error: 1.162268e-16&#xA;Iterations for convergence: 4&#xA;&#xA;Convergence in L2-norm:&#xA;1.511590 &#xA;1.809664 &#xA;2.135559 &#xA;2.080259 &#xA;2.030937 &#xA;&#xA;Convergence in H1-norm:&#xA;0.748175 &#xA;1.164069 &#xA;1.155149 &#xA;1.067994 &#xA;1.029409 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then for $p=2$, Newton's method on the coarsest mesh takes an absurd number of iterations to converge...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Refinement Level 0&#xA;Iteration: 1, Error: 9.021838e-01&#xA;Iteration: 2, Error: 6.832202e-01&#xA;Iteration: 3, Error: 2.971411e-01&#xA;...&#xA;Iteration: 683, Error: 1.499679e+00&#xA;Iteration: 684, Error: 8.019582e-01&#xA;Iteration: 685, Error: 4.119762e-01&#xA;Iteration: 686, Error: 1.051567e-01&#xA;Iteration: 687, Error: 5.846994e-03&#xA;Iteration: 688, Error: 1.780973e-05&#xA;Iteration: 689, Error: 4.898476e-11&#xA;Iteration: 690, Error: 1.755067e-15&#xA;Iterations for convergence: 690&#xA;&#xA;Refinement Level 1&#xA;Iteration: 1, Error: 8.871096e-02&#xA;Iteration: 2, Error: 8.290874e-03&#xA;Iteration: 3, Error: 8.002375e-05&#xA;Iteration: 4, Error: 4.729235e-09&#xA;Iteration: 5, Error: 4.802363e-16&#xA;Iterations for convergence: 5&#xA;&#xA;Refinement Level 2&#xA;Iteration: 1, Error: 1.225342e-02&#xA;Iteration: 2, Error: 9.032665e-05&#xA;Iteration: 3, Error: 1.068461e-08&#xA;Iteration: 4, Error: 1.175537e-15&#xA;Iterations for convergence: 4&#xA;&#xA;Refinement Level 3&#xA;Iteration: 1, Error: 3.305251e-03&#xA;Iteration: 2, Error: 3.496646e-06&#xA;Iteration: 3, Error: 5.546949e-12&#xA;Iterations for convergence: 3&#xA;&#xA;Refinement Level 4&#xA;Iteration: 1, Error: 9.875502e-04&#xA;Iteration: 2, Error: 3.438791e-07&#xA;Iteration: 3, Error: 5.191102e-14&#xA;Iterations for convergence: 3&#xA;&#xA;Refinement Level 5&#xA;Iteration: 1, Error: 2.724040e-04&#xA;Iteration: 2, Error: 2.672095e-08&#xA;Iteration: 3, Error: 3.691891e-16&#xA;Iterations for convergence: 3&#xA;&#xA;Convergence in L2-norm:&#xA;4.403329 &#xA;2.867100 &#xA;2.012923 &#xA;1.775206 &#xA;1.863933 &#xA;&#xA;Convergence in H1-norm:&#xA;3.108488 &#xA;2.144271 &#xA;2.073729 &#xA;2.035578 &#xA;2.017033 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And for $p=4$, convergence is in a single iteration for the given initial guess. The error is just about zero as expected.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Refinement Level 0&#xA;Iteration: 1, Error: 5.726012e-12&#xA;Iterations for convergence: 1&#xA;&#xA;Refinement Level 1&#xA;Iteration: 1, Error: 1.410337e-11&#xA;Iterations for convergence: 1&#xA;&#xA;Refinement Level 2&#xA;Iteration: 1, Error: 2.317826e-12&#xA;Iterations for convergence: 1&#xA;&#xA;Refinement Level 3&#xA;Iteration: 1, Error: 5.361553e-12&#xA;Iterations for convergence: 1&#xA;&#xA;Refinement Level 4&#xA;Iteration: 1, Error: 4.668317e-13&#xA;Iterations for convergence: 1&#xA;&#xA;Refinement Level 5&#xA;Iteration: 1, Error: 2.536992e-13 &#xA;Iterations for convergence: 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2013-11-30T07:17:27.053" LastActivityDate="2013-11-30T16:07:53.037" Title="Slow convergence of Newton's method for finite elements" Tags="|finite-element|nonlinear-equations|newton-method|advection-diffusion|discontinuous-galerkin|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10185" PostTypeId="1" AcceptedAnswerId="10186" CreationDate="2013-11-30T00:29:10.140" Score="7" ViewCount="2082" Body="&lt;p&gt;I want to find a solution for $xA=0$, where $A$ is a square matrix. I know that most of the LAPACK routines solve for $Ax=b$. So I take $A^T$ as a, and set $b=0$. I have an additional restriction of $\Sigma x=1$.Thus, I add an additional row of $ones$  to $A^T$ and a single $one$ to $b$. This also allows to avoid the trivial solution in which $x=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach works fine in Octave, but I am struggling to implement this with LAPACK. I keep getting &lt;code&gt;info=4&lt;/code&gt;, which means that my fourth factor $U$ in the $LU$ factorization of $A^T$ with a row of $ones$ is &lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-4CC4330E-D9A9-4923-A2CE-F60D69EDCAA8.htm&quot; rel=&quot;noreferrer&quot;&gt;singular&lt;/a&gt;. This is happening because my last row is $ones$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good way to solve $xA=0$? I should add that in my real data, $A$ is sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my failing attempt in C, where &lt;code&gt;b=[0,0,0,1]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float *A, *b;&#xA;/* Ax = b&#xA; * Ax = xT(A)   //transpose&#xA; * A is m x m   //matrix   &#xA; * b is m x 1  //vector&#xA; * x is m x 1   //vector&#xA; */&#xA;int m = 3;&#xA;int scale = 1;&#xA;A = (float *)mkl_malloc(m*m,32);&#xA;A[0]=-5;A[1]= 2;A[2]= 3;&#xA;A[3]= 4;A[4]= -10;A[5]= 6;&#xA;A[6]= 7;A[7]=8;A[8]= -15;&#xA;&#xA;b = (float*)mkl_malloc(m+1,32);&#xA;for (int i = 0; i &amp;lt; m; i++) {&#xA;    b[i]=0.0;&#xA;}&#xA;b[m]=1.0;&#xA;&#xA;int matrix_order = LAPACK_ROW_MAJOR;&#xA;&#xA;int ipiv[m+1];&#xA;&#xA;int info;&#xA;//transpose and add a row of ones&#xA;float * Acopy = (float*)mkl_malloc((m+1)*m,32);&#xA;mkl_somatcopy('R','T',m,m,scale,A,m,Acopy,m);&#xA;//assign last row of ones&#xA;for (int i = 0; i &amp;lt; m; i++){&#xA;    int p = m*m+i;&#xA;    Acopy[p]=1.0;&#xA;}&#xA;&#xA;printf(&quot;Original\n&quot;);&#xA;print_matrix(A, m, m );&#xA;printf(&quot;Transpose with ones\n&quot;);&#xA;print_matrix(Acopy, m+1, m );&#xA;&#xA;info = LAPACKE_sgesv(matrix_order,m+1,1,Acopy,m+1,ipiv,b,1);&#xA;&#xA;printf(&quot;$?=%d\n&quot;,info);&#xA;&#xA;print_matrix(b, m+1, 1 );&#xA;return 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="6716" LastActivityDate="2013-12-03T07:30:04.247" Title="Solving for null space of a matrix with mkl LAPACK" Tags="|linear-solver|lapack|intel-mkl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10186" PostTypeId="2" ParentId="10185" CreationDate="2013-11-30T03:46:16.697" Score="9" Body="&lt;p&gt;There's no reason to append a row of 1's. You should just perform a rank-revealing QR factorization (like with routine SGEQP3) on $A^T$, and the last column of $Q$ should be in the nullspace. This has the added advantage that the relative magnitude of the last element on the diagonal of $R$ gives you some idea of how singular the solution is. Even better would be to perform a singular value decomposition of $A=U\Sigma V^T$. Then, the last column of $U$ is a singular solution to $xA=0$. You can simply normalize to achieve $\sum x = 1$ afterwards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you can't just append a row of ones at the bottom of $A$ since then you have a non-square system. You can probably solve that by least squares using SGELS (or any of the other variants). This generally is less robust than the previous two suggestions I gave, although it enforces the normalization explicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Note on speed and stability. I would guess QR is slightly faster than SVD, while SVD is more numerically stable (although practically, they are sufficiently stable that people don't worry about it). I don't have a good sense of speed since more sophisticated pivoting in QR is slower. Note finally that these approaches only work for dense matrices. If you want something that works in the sparse case, you can try power iteration on the inverse of $A$. This should work as long as $A$ is not exactly singular (which basically never happens numerically).&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2013-12-03T07:30:04.247" LastActivityDate="2013-12-03T07:30:04.247" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10187" PostTypeId="1" AcceptedAnswerId="10274" CreationDate="2013-11-30T14:59:38.160" Score="15" ViewCount="3367" Body="&lt;p&gt;I'm trying to learn about numerically solving PDE by myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been beginning with finite difference method(FDM) for some time because I heard that FDM&#xA;is the fundament of numerous numerical methods for PDE. So far I've got some basic understanding for FDM and been able to write codes for some simple PDE lay in regular region with the materials I found in the library and Internet, but what's strange is, those materials I've got usually talks little about the treatment of irregular, curved, strange boundary, like &lt;a href=&quot;https://mathematica.stackexchange.com/questions/18921/how-do-i-solve-a-pde-with-a-strange-boundary-condition&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's more, I've never seen a easy way to deal with the curved boundary. For example, the book &lt;a href=&quot;http://math.science.cmu.ac.th/docs/qNA2556/ref_na/Morton_Numerical%20Solution%20of%20PDE.pdf&quot; rel=&quot;noreferrer&quot;&gt;Numerical Solution of Partial Differential Equations - An Introduction (Morton K., Mayers D)&lt;/a&gt;, which contains the most detailed discussion (mainly in &lt;em&gt;3.4&lt;/em&gt; from p71 and &lt;em&gt;6.4&lt;/em&gt; from p199) I've seen until now, has turned to a extrapolation that is really cumbersome and frustrating for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as the title asked, as to the curved boundary, usually how do people deal with it when using FDM? In other words, what's the most popular treatment for it? Or it depends on the type of PDE?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a (at least relatively) elegant and high-precision way to deal with the curved boundary? Or it's just an inevitable pain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I even want to ask, do people actually use FDM for curved boundary nowadays? If not, what's the common method for it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="5331" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2019-04-26T10:45:00.037" Title="How to deal with curved boundary condition when using finite difference method?" Tags="|pde|finite-difference|boundary-conditions|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10188" PostTypeId="2" ParentId="10184" CreationDate="2013-11-30T15:21:49.343" Score="5" Body="&lt;p&gt;Your observed quadratic convergence indicates that the Jacobian is likely correct.  Have you looked at the solutions for your under-resolved configurations?  Galerkin optimality uses the operator norm, which contains only the symmetric part, thus the solution of the discrete system could be quite different from the projection of the exact solution.  This would be very likely if you use an unstable flux, or perhaps just lack a limiter, leading to solutions with numerical artifacts.  What is the maximum cell Péclet number and what numerical fluxes are you using?&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-11-30T15:21:49.343" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10189" PostTypeId="1" CreationDate="2013-11-30T15:28:55.313" Score="2" ViewCount="398" Body="&lt;p&gt;I'm trying to construct the right–hand side of my 2D Poisson's equation in Matlab. I used the vertex rule in order to approximate the integral:&#xA;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function F1 = rightVer(p,t,f)&#xA;&#xA;%area triangle&#xA;it1=t(1,:);&#xA;it2=t(2,:);&#xA;it3=t(3,:);&#xA;area = ((p(1,it2)-p(1,it1)).*(p(2,it3)-p(2,it1))-(p(1,it3)-p(1,it1)).*(p(2,it2)-p(2,it1)))/2;&#xA;&#xA;area = abs(area)/3;&#xA;vt1 = area.*f(p(1,it1),p(2,it1));&#xA;vt2 = area.*f(p(1,it2),p(2,it2));&#xA;vt3 = area.*f(p(1,it3),p(2,it3));&#xA;&#xA;vt = [vt1';vt2';vt3'];&#xA;elem = t';&#xA;elem(:,4)=[];&#xA;N=size(p,2);&#xA;F1  =  accumarray(elem(:),vt,[N  1]);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also construct it with the &lt;a href=&quot;http://www.mathworks.com/products/pde/%22pde%20toolbox%22&quot; rel=&quot;nofollow noreferrer&quot;&gt;pde toolbox&lt;/a&gt; with the function &lt;code&gt;assema()&lt;/code&gt; but the two f which I obtained are the same except for the last 4 numbers. Can anyone help me? &lt;/p&gt;&#xA;" OwnerUserId="6703" LastEditorUserId="20688" LastEditDate="2019-06-22T20:40:45.570" LastActivityDate="2019-06-22T20:40:45.570" Title="assembly matrices in finite element method" Tags="|finite-element|matlab|poisson|elliptic-pde|" AnswerCount="0" CommentCount="2" ClosedDate="2019-06-16T23:10:29.277" ContentLicense="CC BY-SA 4.0" />
  <row Id="10190" PostTypeId="2" ParentId="10184" CreationDate="2013-11-30T16:07:53.037" Score="1" Body="&lt;p&gt;Looks like you are seeing some oscillations in the run that you encounter large number of iterations.  Maybe the Jacobian itself changed too rapidly or you have a small eigenvalue.  When that happens there are a few things that did help me: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;implement a trust radius to limit your step size to a reasonably small number;  ideally the trust radius will be dynamically adjusted by the program, but it is also OK to do it manually as long as convergence problem is rare.&lt;/li&gt;&#xA;&lt;li&gt;shift all eigenvalues by a small amount at the beginning of the convergence.  this tend to remove oscillation but you will lose quadratic convergence, so like the first one you need to scale this shift to 0 once it is on the right track.&lt;/li&gt;&#xA;&lt;li&gt;add a line search step to ensure you find the minimum along the direction predicted by Newton's method.  Basically, use the direction provided by Newton's method but try to get a better step size.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For me, method 3 often worked extraordinarily well, although it does take the most effort.&lt;/p&gt;&#xA;" OwnerUserId="6644" LastActivityDate="2013-11-30T16:07:53.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10191" PostTypeId="2" ParentId="10177" CreationDate="2013-11-30T16:32:28.067" Score="19" Body="&lt;p&gt;There are two issues that you are likely to be encountering.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Ill-conditioning&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;First, the problem is ill-conditioned, but if you only provide a residual, Newton-Krylov is throwing away half your significant digits by finite differencing the residual to get the action of the Jacobian:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ J[x] y \approx \frac{F(x+\epsilon y) - F(x)}{\epsilon}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you provide an analytic Jacobian, you get to keep all the digits (e.g., 16 in double precision).  Krylov methods also rely on inner products, so if your Jacobian is ill-conditioned to the tune of $10^{16}$, it is effectively singular and Krylov can stagnate or return erroneous solutions.  This can also prevent direct solvers from converging.  Sometimes you can use multigrid methods to homogenize into a coarse grid with tractable conditioning.  When a problem cannot be formulated with better conditioning, it can be worth working in quad precision.  (This is supported by PETSc, for example.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the same issues apply to quasi-Newton methods, though without finite differencing.  All scalable methods for problems with non-compact operators (e.g., differential equations) must use Jacobian information for preconditioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is likely that &lt;code&gt;fsolve&lt;/code&gt; either did not use Jacobian information or that it used a dogleg method or a shift to make progress with a &quot;gradient descent&quot; method, despite an essentially singular Jacobian (i.e., finite differencing would have a lot of &quot;noise&quot; from finite precision arithmetic).  This is not scalable and &lt;code&gt;fsolve&lt;/code&gt; likely gets slower as you increase the problem size.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Globalization&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If the linear problems are solved accurately, we can rule out problems relating to the linear problem (Krylov) and focus on those due to nonlinearity.  Local minima and nonsmooth features slow convergence or cause stagnation.   Poisson-Boltzmann is a smooth model so if you start with a good enough initial guess, Newton will converge quadratically.  Most globalization strategies involve some sort of continuation to produce a high-quality initial guess for the final iterations.  Examples include grid continuation (e.g., Full Multigrid), parameter continuation, and pseudotransient continuation.  The latter is generally-applicable to steady-state problems, and offers some global convergence theory, see &lt;a href=&quot;http://dx.doi.org/10.1137/S106482750241044X&quot; rel=&quot;noreferrer&quot;&gt;Coffey, Kelley, and Keyes (2003)&lt;/a&gt;.  A search turns up this paper, which may be useful to you: &lt;a href=&quot;http://dx.doi.org/10.1006/jcis.2001.8033&quot; rel=&quot;noreferrer&quot;&gt;Shestakov, Milovich, and Noy (2002) &lt;strong&gt;Solution of the nonlinear Poisson-Boltzmann equation using pseudo-transient continuation and the finite element method&lt;/strong&gt;&lt;/a&gt;.  Pseudotransient continuation is closely related to the Levenberg-Marquardt algorithm.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Further reading&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/31/119&quot;&gt;Why is Newton's method not converging?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/514/119&quot;&gt;Why is my iterative linear solver not converging?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-11-30T16:32:28.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10192" PostTypeId="2" ParentId="10187" CreationDate="2013-11-30T16:55:42.453" Score="11" Body="&lt;p&gt;Curved boundaries are covered in most CFD books, e.g., &lt;a href=&quot;http://dutita0.twi.tudelft.nl/nw/users/wesseling/cfdbook.html&quot;&gt;Chapter 11 of Wesseling&lt;/a&gt; or &lt;a href=&quot;http://link.springer.com/book/10.1007/978-3-642-97651-3&quot;&gt;Chapter 8 of Ferziger and Peric&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While not a fundamental theoretical problem, the practical complexity of implementing boundary conditions for high-order methods on curved boundaries is a significant reason for interest in more geometrically-flexible methods such as the finite element method (including discontinuous Galerkin).  Structured finite difference and finite volume grids are still used in some CFD simulations, but unstructured methods are gaining popularity and the local operations used by high-order unstructured methods are actually quite efficient, and thus may not suffer much loss in efficiency compared to similar FD methods.  (Indeed, the geometric flexibility often makes them more efficient.)&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-11-30T16:55:42.453" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10193" PostTypeId="2" ParentId="10168" CreationDate="2013-11-30T17:00:51.950" Score="3" Body="&lt;p&gt;Without knowing the magnitude of components &lt;code&gt;x0 = (x,y,z)&lt;/code&gt;, it's not possible to bound the size of results given by the three expressions &lt;code&gt;value(0),value(1),value(2)&lt;/code&gt;.  As @jimmcnamara explains, knowing the necessary limits of precision in advance is critical to using libraries like MPFR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may have gotten a sufficient approximation, by either of your two techniques, to be able to bound the limits on magnitude, despite any &quot;loss of precision&quot;.  It depends partly on whether intermediate results are significantly larger than your final results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case having an exact arithmetic environment will be useful, if only for the sake of validating a wide range of results obtained with C and MPFR.  I'll throw out some ideas on how to compute integer expressions exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interpreted tool that will do this is &lt;a href=&quot;https://en.wikipedia.org/wiki/Rexx&quot; rel=&quot;nofollow&quot;&gt;IBM's Rexx&lt;/a&gt;, available on many platforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_algebra_system&quot; rel=&quot;nofollow&quot;&gt;CAS&lt;/a&gt; (computer algebra system) packages support arbitrary precision integers, e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Maxima_%28software%29&quot; rel=&quot;nofollow&quot;&gt;Maxima&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Sage_%28mathematics_software%29&quot; rel=&quot;nofollow&quot;&gt;Sage&lt;/a&gt;.  &lt;a href=&quot;http://sympy.org/en/index.html&quot; rel=&quot;nofollow&quot;&gt;Sympy&lt;/a&gt; has a more modest set of features but incorporates arbitrary precision arithmetic through &lt;a href=&quot;http://docs.sympy.org/dev/modules/mpmath/index.html&quot; rel=&quot;nofollow&quot;&gt;mpmath&lt;/a&gt;, a Python library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A useful case study for implementing algorithms with long integers is &lt;a href=&quot;http://www.shoup.net/ntl/doc/tour.html&quot; rel=&quot;nofollow&quot;&gt;Victor Shoup's NTL&lt;/a&gt; (a library for doing number theory) which for portability is written in C++.  By default it uses its own customized version of &lt;a href=&quot;http://www.win.tue.nl/~klenstra/&quot; rel=&quot;nofollow&quot;&gt;LIP&lt;/a&gt;, but can also be configured to use &lt;a href=&quot;http://gmplib.org/&quot; rel=&quot;nofollow&quot;&gt;GMP&lt;/a&gt; either as a supplemental or a complete replacement for exact integers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I wanted to do something quick and easy but allowing for version control/regression testing with programming tools already on my desktop, I'd likely turn to &lt;a href=&quot;http://www.amzi.com/manuals/amzi/pro/ref_math.htm#NumberTypes&quot; rel=&quot;nofollow&quot;&gt;Amzi Prolog&lt;/a&gt; which supports arbitrary precision integers as part of an &quot;infinite precision real&quot; data type. Also &lt;a href=&quot;http://www.swi-prolog.org/pldoc/man?section=artypes&quot; rel=&quot;nofollow&quot;&gt;SWI-Prolog&lt;/a&gt; supports unbounded integers (and rational arithmetic on top of that) &lt;em&gt;if&lt;/em&gt; it is built with GMP.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-11-30T17:00:51.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10194" PostTypeId="1" CreationDate="2013-11-29T02:59:42.280" Score="0" ViewCount="826" Body="&lt;p&gt;I am attempting to model the steady state behavior of a cylinder using the finite volume method (FVM) subjected to a variety of boundary conditions in Matlab. First off, I am treating the cylinder as being axisymmetric so I am only determine the temperature profile in the r-Z plane. I set up a 2D grid so that the r coordinate is on the 'x' axis and the Z coordinate is the 'y' axis. Since the cylinder is axisymmetric then the heat flux at r = 0 is 0 (insulated). I am also applying an insulated boundary condition to the bottom surface at Z=0. The side surface has convection with a constant convection coefficient and air temperature of 310 K. The top of the cylinder, z=Z, there is a heat flux of 1000 W/m^2, convection from the air at 310K, and also surface to ambient radiation with the surroundings being 298 K. The initial temperature of cylinder is 298 K (initial guess value). The problem that I am having is that the temperature distribution matches Comsol's results with a grid of 40x40 and a thermal conductivity of 1. When I increases the number of cells to 50x50, 60x60, etc, or if i change the thermal conductivity to something other than 1 then the temperature profile is still what it should be but all the temperature are significantly lower than expected. My control loop in Matlab is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use initial values and problem constants to solve for initial surface temperature of top and side.&lt;/li&gt;&#xA;&lt;li&gt;Use surfacte temperatures and problem geomety to determine aE, aN, aS, aW, aP, and b for every cell in the grid.&lt;/li&gt;&#xA;&lt;li&gt;Use line by line TDMA in both directions to determine the temperature profile.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the error between the new temperature profile and the previous one.&lt;/li&gt;&#xA;&lt;li&gt;Update the surface temperatures and aP and b because of the nonlinear radiation term that uses previous iterate values.&lt;/li&gt;&#xA;&lt;li&gt;Perform line by line TDMA again and check for convergence. If the error is not less than 0.001 then repeat the procedure.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the boundary conditions, I used an energy balance for the top surface and the side surface. The top surface includes surface to ambient radiation and I calculated the surface temperature to be &#xA;$$T_{b}=\frac{\alpha G+hT_{f}+\frac{kT_{p}}{\delta z}+\epsilon \sigma(T_{sur}^{4}+3T_{b}^{*4})}{h+4 \epsilon \sigma T_{b}^{*3}+\frac{k}{\delta z}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also do the same thing with the surface temperature on the side the is experiencing heating through convection. Since the top surface temperature depends on its previous iterate value, I use an initial guess value of 298 for the top surface. I then calculate the initial surface temperature based on that and the equation i provided for the first iteration,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem lies in the fact that the surface tempreature depends on both the thermal conductivity and the grid spacing. When I use many cells like 80x80, the top surface temperature for the first TDMA iteration is almost 298 which is hard to believe because its being heated by a 1000 W/m^2 heat flux and its also being heated by the 310 K air temperature. This causes the first iteration to have very little error so convergence only produces a max temperature of like 325 when it should be 450. But if I use a grid of like 40x40 then the surface temperature for the first iteration is much much higher, and convergence causes the final temperature profile to look almost exactly like the expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same exact thing happens with the thermal conductivity, if i change the value then the initial surface temperatures change dramatically which causes the convergence values to be much lower than the expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like it all has to do with that initial surface temperature but I am not sure how else to treat it. Any help would be amazing as I have spent many nights til 5am trying to figure out how I could fix this. My contact email is in my profile if you would like to see my matlab code of if you would like to talk to me more about my problem.&lt;/p&gt;&#xA;" OwnerUserId="6714" OwnerDisplayName="Greg Harrington" LastEditorUserId="276" LastEditDate="2017-10-10T00:37:25.667" LastActivityDate="2017-10-10T00:37:25.667" Title="Finite volume method to determine steady state temp in cylinder" Tags="|finite-volume|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10195" PostTypeId="2" ParentId="10194" CreationDate="2013-12-01T05:38:48.353" Score="3" Body="&lt;p&gt;&lt;em&gt;Verification&lt;/em&gt; is the first step with any new code.  I like using the method of manufactured solutions (MMS) because it permits the use of rich solutions, but there are also closed-form analytic solutions for similar problems.  I would verify the diffusion solver first, before adding the radiation term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to pick any smooth function $u(r,z)$ (does not need to be &quot;physical&quot;) and feed it to your differential operator to &quot;manufacture&quot; a forcing term for which your chosen $u(r,z)$ is a solution.  Then run your numerical code and perform a grid convergence study, evaluating $\lVert u^h - u \rVert$ on each grid $h$.  Confirm that the design order of the method (e.g., quadratic) has been reached.  If not, simplify further until you see the design order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your symptoms suggest that there may be a scaling bug, but whatever it is, MMS should diagnose the problem.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-12-01T05:38:48.353" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10197" PostTypeId="2" ParentId="8928" CreationDate="2013-12-01T20:10:14.523" Score="3" Body="&lt;p&gt;I think it depends on what you are doing.  I do typically mix languages.  I'll describe my personal experience below, perhaps this'll help you decide for your application.  While I'll be talking about my own way of working below, and my tool of choice is Mathematica, I believe the same advantages apply to other high level languages too.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I typically write some type of simulation in a low level (C++) language.  It has some (numerical) input and produces some potentially complex numerical output.  Originally I passed the input through command line parameters and input files, and wrote the output into text files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually it became necessary to run the program for many different input parameters, which is best automated.  I used to use some mix or shell scripting and scripting languages (Python) do call my program with various parameters / input-files, and processed/visualized the output in Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually I figured out that it's much better to write a Mathematica interface for calling my C++ code instead of writing a command line interface and using input files.  Now this is my standard workflow.  The Mathematica interface turned out to be simpler and easier to write (once I learned how!), mainly because of how easy it is to transfer structured numerical data without the need to convert it to text first.  Consider e.g. having to pass a list of 3D arrays.  One would need to invent a special storage format to transfer these, or use a C++ library for some common data format such as HDF5, which Mathematica already has a high level interface for anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a Mathematica interface also made a number of things very easy, such as: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;smarter mapping of the parameter space (e.g. applying various &lt;a href=&quot;https://mathematica.stackexchange.com/questions/216/adaptive-sampling-for-slow-to-compute-functions-in-2d&quot;&gt;adaptive&lt;/a&gt; &lt;a href=&quot;https://scicomp.stackexchange.com/q/923/605&quot;&gt;sampling&lt;/a&gt; algorithms, which are much easier to implement and test in a high level language);  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;easily save the simulation state and resume later (because transferring structured numerical data between Mathematica and C is easy enough that I can afford to take the time to implement this)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;run various numerical algorithms with my simulation, such as multivariate optimization, root finding, etc. (because Mathematica already has many of these built in)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Visualizing the results immediately and interactively, without having to go through several steps (write the data to a file, process it for visualization, read it into the plotting program).  Suppose that a simple root finding method would fail, and it's not clear why.  If I can play with the simulation interactively, and visualize results immediately, it's much easier to figure out what's going on.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Easily running simulations in parallel using Mathematica's parallel tools, even on multiple computers (thanks to Mma's built-in communication protocol).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the output of the simulation is very large (hundreds of MB), I can't afford to store it all for all parameter values.  I'd usually run some analysis on it (averages, variances, etc.), and store only the results.  Using the high level language to drive the simulation allows me to run many more non-trivial types of analyses without having to implement them myself in C++.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The focus here is on the &lt;em&gt;ease of implementing things using a high level language&lt;/em&gt; with lots of useful built-in functionality.  All of this could be done differently too, but it would be much more work, so I likely wouldn't do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I typically write the Mathematica interface &lt;em&gt;before&lt;/em&gt; writing a command line interface, and only do the latter if I actually need it (e.g. I have to share the code with people who don't use Mathematica).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This worked well for me because I already knew Mathematica well, and I already used it for processing and visualizing the output anyway.  I am also familiar with Mathematica's convenient C interface, which does take some time to learn.  Also, I usually write these simulations alone, which doesn't restrict the tools I can choose.  If I were collaborating with someone on the code, I would choose to use tools that we both know well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same advantages apply when using any other similar high level language with a convenient C interface, such as &lt;strong&gt;Python/MATLAB/Julia/R/etc&lt;/strong&gt;.  I'm mentioning Mathematica here because that's what I am familiar with and I'm describing my own workflow.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I also regularly use &lt;a href=&quot;http://matlink.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MATLAB&lt;/a&gt; and &lt;a href=&quot;https://github.com/szhorvat/IGraphR&quot; rel=&quot;nofollow noreferrer&quot;&gt;R&lt;/a&gt; functions (and occasionally Java) from within Mathematica.  I do this because convenient high level interfaces already exist between these languages.  Otherwise I wouldn't bother.  I believe &lt;strong&gt;Python&lt;/strong&gt; is one of the most convenient languages to use in this manner, &lt;a href=&quot;http://en.wikipedia.org/wiki/Glue_language&quot; rel=&quot;nofollow noreferrer&quot;&gt;gluing together&lt;/a&gt; functionality from various packages.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2013-12-01T23:35:27.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10198" PostTypeId="1" CreationDate="2013-12-01T21:17:28.997" Score="0" ViewCount="1311" Body="&lt;p&gt;I'm working on some MATLAB code to perform something called the Index Calculus attack on a given cryptosystem (this involves calculating discrete log values), and I've gotten it all done except for one small thing. I cant figure out (in MATLAB) how to solve a linear system of congruences mod p, where p is &lt;em&gt;not&lt;/em&gt; prime. Also, this system has more than 1 variable, so, unless I'm missing something, the Chinese remainder theorem wont work. I'm trying to solve the following system $\pmod 8$. The end goal is that I'm solving it mod each factor of 10930888, and then using the Chinese remainder theorem to construct each $L_i$ value:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{pmatrix} 0 &amp;amp; 5 &amp;amp; 4 &amp;amp; 1 \\ 1 &amp;amp; 7 &amp;amp; 0 &amp;amp; 2\\ 8 &amp;amp; 1 &amp;amp; 0 &amp;amp; 2\\ 10 &amp;amp; 5 &amp;amp; 1 &amp;amp;0 \end{pmatrix}$*$\begin{pmatrix} L_1\\L_2\\L_3\\L_4 \end{pmatrix}$=$\begin{pmatrix} 2946321 \\ 5851213 \\ 2563617\\10670279 \end{pmatrix}$ $\pmod{10930888}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I asked a question on the mathematics stackexchange with more detail/formatted mathjax &lt;a href=&quot;https://math.stackexchange.com/questions/585609/having-trouble-using-the-chinese-remainder-theorem-to-solve-a-system-of-congruen&quot;&gt;here&lt;/a&gt;. I solved the issue in my question at that link, and now I'm attempting to find a utility that will allow me to solve the system of congruences modulo a non-prime. I did find a suite that includes a solver supporting modular arithmetic, but the modulus must be prime (&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/22725-variable-precision-integer-arithmetic&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). I also tried stepping through to modify it to work with non-primes, but whatever method is used doesn't work, because it requires all elements of the system have inverses modulo p.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked into using the ability in MATLAB to call MuPAD functions, but from my testing, the MuPAD function linsolve (which seemed to be the best candidate) doesn't support non-prime modulus values either. Additionally, I've verified with Maple that this system is solvable modulo my integer of interest (8), so it can be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated- I feel like I've run out of options here!&lt;/p&gt;&#xA;" OwnerUserId="6719" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-07-02T01:53:13.053" Title="Solving a system of linear congruences mod p, where p is not prime, and the system has 2+ variables" Tags="|linear-algebra|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10199" PostTypeId="1" AcceptedAnswerId="14463" CreationDate="2013-12-02T01:43:35.017" Score="1" ViewCount="1818" Body="&lt;p&gt;Is it computationally safe and accurate to use the following equation in COMSOL to compute the divergence of the vector quantity J (instead of using its general built-in equations that have $\nabla$ operations):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$div_J = d(J_x,x)+d(J_y,y)+d(J_z,z)$$&lt;/p&gt;&#xA;" OwnerUserId="4858" LastActivityDate="2015-09-17T00:03:33.827" Title="Calculating Divergence in COMSOL" Tags="|pde|finite-element|comsol|automatic-differentiation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10200" PostTypeId="1" AcceptedAnswerId="10201" CreationDate="2013-12-02T03:07:48.967" Score="3" ViewCount="181" Body="&lt;p&gt;For a computational representation theory program, I have to calculate the trace of $A^+B$ for a fixed $A$ and many different $B$, where $A^+$ is the pseudoinverse of $A$ ($A,B$ have full column rank and the same image). Currently, I compute the LU decomposition of $A$ and use this to solve $Ax_i=b_i$ for each column $b_i$ of $B$, and use $\mathrm{tr}(A^+B)=\sum_i x_{ii}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A factor of 2 or so speedup could be achieved by halting backsolve once $x_{ii}$ has been computed rather than computing all of $x_i$. However, this is not enough to make a difference in my code and would require reimplementing UMFPACK methods, which would probably give me slower code in the end because my implementation wouldn't be as good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a (hopefully asymptotically) faster approach? For example, is it possible to compute $x_{ii}$ asymptotically faster than $x_i$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it matters, all matrices and methods in question are sparse.&lt;/p&gt;&#xA;" OwnerUserId="682" LastEditorUserId="682" LastEditDate="2013-12-02T05:50:58.950" LastActivityDate="2013-12-03T22:48:35.933" Title="Calculating the trace of $A^+B$" Tags="|linear-algebra|umfpack|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10201" PostTypeId="2" ParentId="10200" CreationDate="2013-12-02T06:04:58.063" Score="6" Body="&lt;p&gt;Since $A$ has full column rank, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A^{+}=(A^{T}A)^{-1}A^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could handle this computation by computing the Cholesky factorization of $(A^{T}A)=R^{T}R$, then compute $(A^{T}A)^{-1}=R^{-1}R^{-T}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can afford to store $A^{+}$ (which is probably dense), then once you have $A^{+}$, computing the trace of $A^{+}B$ takes $O(mn)$ time for each $B$ matrix (and a lot less if $B$ is sparse- you haven't said whether the $B$ matrices are sparse.)   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can't afford to store $A^{+}$ but can store $(A^{T}A)^{-1}$ then it may be helpful to precompute the inverse of $A^{T}A$.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-12-02T06:04:58.063" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="10203" PostTypeId="1" AcceptedAnswerId="10208" CreationDate="2013-12-02T07:58:45.340" Score="6" ViewCount="6578" Body="&lt;p&gt;On my triangular mesh, I have the $(x,y,z)$ coordinates of each vertex of each triangle. For higher order elements, I refine each element a few times so I have more points to work with. If I just need to visualize the solution for myself, I use &lt;code&gt;fill3&lt;/code&gt; but I'm not entirely sure how to insert my own colorbar. Ideally, I want the smallest z value to correspond to blue, and the largest to correspond to red, like the typical colorbar in the &lt;code&gt;surf&lt;/code&gt; plots. Here's an example of what I'm trying to achieve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DacjC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;fill3&lt;/code&gt; works fine, I'm just not really sure how to set the custom colormap to my needs, and the documentation hasn't helped much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm open to other suggestions as well, it's just that MATLAB is what I'm familiar with for visualization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, just curious if anyone knows what software this plot was made in as I've seen plots like it in a number of papers.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="350" LastEditDate="2013-12-02T17:19:17.340" LastActivityDate="2013-12-02T19:49:10.477" Title="Visualizing finite element solutions in MATLAB" Tags="|finite-element|visualization|" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10204" PostTypeId="2" ParentId="10203" CreationDate="2013-12-02T08:17:00.920" Score="1" Body="&lt;p&gt;It's simpler to use the functions &lt;code&gt;pdemesh&lt;/code&gt; or &lt;code&gt;pdesurf&lt;/code&gt; since there are done exactly for that purpose. You can then change the colorscale with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;colormap('jet')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This changes the color map to what you need. There are other preset colormap and you can also define yours, see the colormap function help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have not the PDE package in Matlab, you can maybe use Octave which has the same functionalities.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="3684" LastEditDate="2013-12-02T14:39:47.897" LastActivityDate="2013-12-02T14:39:47.897" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10205" PostTypeId="2" ParentId="10203" CreationDate="2013-12-02T13:26:33.803" Score="3" Body="&lt;p&gt;You could output your data in one of the widely used file formats, for example VTK, and then use either VisIt or Paraview to visualize. These programs are used a lot for visualizing PDE solutions and are made for this purpose.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-02T13:26:33.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10206" PostTypeId="1" AcceptedAnswerId="10210" CreationDate="2013-12-02T13:31:56.313" Score="4" ViewCount="626" Body="&lt;p&gt;I have implemented the 2D heat equation with what I thought was the Crank-Nicolson algorithm in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function[newDomain] = heatCN(domain, dt, Nx, Ny, hx, hy, spanX, spanY)&#xA;aux = domain;&#xA;newDomain = domain;&#xA;for i=2:Nx-1&#xA;    for j=2:Ny-1&#xA;        derX1 = dt * (-2.0 * domain(i,j) + domain(i+1, j) + domain(i-1,j)) / hx^2;&#xA;        derY1 = dt * (-2.0 * domain(i,j) + domain(i, j+1) + domain(i, j-1)) / hy^2;&#xA;        aux(i,j) = derX1 + derY1 + domain(i,j);&#xA;    end;&#xA;end;&#xA;&#xA;for i=2:Nx-1&#xA;    for j=2:Ny-1&#xA;        derX2 = dt * (-2.0 * domain(i,j) + domain(i+1, j) + domain(i-1,j)) / hx^2;&#xA;        derY2 = dt * (-2.0 * domain(i,j) + domain(i, j+1) + domain(i, j-1)) / hy^2;&#xA;        derXT = dt * (-2.0 * aux(i,j) + aux(i+1, j) + aux(i-1,j)) / hx^2;&#xA;        derYT = dt * (-2.0 * aux(i,j) + aux(i, j+1) + aux(i, j-1)) / hy^2;&#xA;        newDomain(i,j) = domain(i,j) + 0.5 * (derX2 + derY2 + derXT + derYT);&#xA;    end;&#xA;end;&#xA;end;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From my knowledge, the CN method should be unconditionally stable, meaning that for any (decent) time step it should give me the right output. But the behavior I see with this program is unstable.&lt;/p&gt;&#xA;" OwnerUserId="6722" LastEditorUserId="123" LastEditDate="2013-12-04T04:26:44.897" LastActivityDate="2013-12-04T04:26:44.897" Title="My algorithm for the heat equation is unstable" Tags="|advection-diffusion|crank-nicolson|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10208" PostTypeId="2" ParentId="10203" CreationDate="2013-12-02T19:49:10.477" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For higher order elements, I refine each element a few times so I have more points to work with. If I just need to visualize the solution for myself.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let's use quadratic Lagrange elements as examples. You need the mesh data, points &lt;code&gt;p&lt;/code&gt; and triangles &lt;code&gt;t&lt;/code&gt;, also the numerical solution $u_h$. For visualization purpose, we merely need the nodal values from the numerical solution using quadratic Lagrange elements (quadratic Lagrange has edge DoFs and node DoFs). Same for the higher order elements, we only need the nodal DoF's value to visualize the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: your solution column vector is &lt;code&gt;u&lt;/code&gt; which is gonna be the $z$ value of the &lt;code&gt;surf&lt;/code&gt; plot. For quadratic element, the most natural data structure is to assign the first &lt;code&gt;NNodes = size(p,1)&lt;/code&gt; rows the value of nodal DoFs, then &lt;code&gt;NEdges = size(e,1)&lt;/code&gt; rows the value of edge DoFs. Because you use triangular mesh, we can use MATLAB's trisurf to do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h = trisurf(t, p(:,1), p(:,2), u(1:size(p,1))');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;colormap&lt;/code&gt;'s range can be manually assigned, see MATLAB's documents &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/colormap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. There are some built-in schemes like jet, you can use an $m\times 3$ RGB matrix as well (you can retrieve the color matrix by typing &lt;code&gt;c = colormap;&lt;/code&gt; then check what &lt;code&gt;c&lt;/code&gt; is like).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is another very helpful MATLAB doc on how to manually setting the &lt;code&gt;colormap&lt;/code&gt; for a patch object (&lt;code&gt;surf&lt;/code&gt; and &lt;code&gt;trisurf&lt;/code&gt; are both using patch to draw things): &lt;a href=&quot;http://www.mathworks.com/help/matlab/visualize/coloring-mesh-and-surface-plots.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coloring Mesh and Surface Plots&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what the numerical solution using quadratic elements is like for $-\Delta u = 1$ with homogeneous Dirichlet boundary data on an L-shape domain. There are no discontinuous gap between the nodes like your plot yields. Unless you use $hp$-Discontinuous Galerkin, your plot is wrong for usually continuous finite element solutions for any elliptic equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/83hpT.png&quot; alt=&quot;Lshape&quot;&gt;:&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2013-12-02T19:49:10.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10209" PostTypeId="1" CreationDate="2013-12-02T22:00:36.977" Score="4" ViewCount="310" Body="&lt;p&gt;I'm having a problem with multigrid code I wrote. If I solve Laplace's equation in 2D and use more than 5 grid levels, the V-cycles stop converging after a few cycles (see below, convergence factor &gt; 1). At grid level 3, you can see that the recursive V-cycle actually increases the error (L2 norm of the defect). If I use fewer grid levels, my code converges as expected. I suspect it's an issue with the boundary conditions, because the defect is very high close to the boundaries after V-cycle 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Dirichlet boundaries I added ghost points to the finest grid to hold the boundary values. When I restrict the defect, I set the values of the defect at the coarse-grid ghost points to zero. Is there something more I need to worry about?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;V-CYCLE 1&#xA;LEVEL   ACTION                  ERROR&#xA;1       Initial                 4.566101e-01&#xA;1       Relaxation sweep 1      1.375412e-01&#xA;2       Initial                 4.264610e-02&#xA;2       Relaxation sweep 1      2.082439e-02&#xA;3       Initial                 1.580465e-02&#xA;3       Relaxation sweep 1      9.881322e-03&#xA;4       Initial                 8.008788e-03&#xA;4       Relaxation sweep 1      5.150501e-03&#xA;5       Initial                 4.121322e-03&#xA;5       Relaxation sweep 1      2.643638e-03&#xA;5       Relaxation sweep 400    1.525026e-18&#xA;5       Coarse grid corr.       4.253979e-03&#xA;5       Relaxation sweep 1      1.356265e-03&#xA;4       Coarse grid corr.       1.301027e-02&#xA;4       Relaxation sweep 1      4.956990e-03&#xA;3       Coarse grid corr.       3.427880e-02&#xA;3       Relaxation sweep 1      1.421168e-02&#xA;2       Coarse grid corr.       8.532111e-02&#xA;2       Relaxation sweep 1      3.721634e-02&#xA;1       Coarse grid corr.       2.409762e-01&#xA;1       Relaxation sweep 1      1.022146e-01&#xA;CONVERGENCE FACTOR: 0.223855&#xA;&#xA;V-CYCLE 2&#xA;LEVEL   ACTION                  ERROR&#xA;1       Initial                 1.022146e-01&#xA;1       Relaxation sweep 1      6.796567e-02&#xA;2       Initial                 5.807979e-02&#xA;2       Relaxation sweep 1      4.311037e-02&#xA;3       Initial                 3.733848e-02&#xA;3       Relaxation sweep 1      2.717026e-02&#xA;4       Initial                 2.302219e-02&#xA;4       Relaxation sweep 1      1.627633e-02&#xA;5       Initial                 1.341552e-02&#xA;5       Relaxation sweep 1      9.233742e-03&#xA;5       Relaxation sweep 400    5.981442e-18&#xA;5       Coarse grid corr.       1.660509e-02&#xA;5       Relaxation sweep 1      5.472094e-03&#xA;4       Coarse grid corr.       4.952103e-02&#xA;4       Relaxation sweep 1      1.951415e-02&#xA;3       Coarse grid corr.       1.255826e-01&#xA;3       Relaxation sweep 1      5.395197e-02&#xA;2       Coarse grid corr.       2.973566e-01&#xA;2       Relaxation sweep 1      1.349677e-01&#xA;1       Coarse grid corr.       6.812406e-01&#xA;1       Relaxation sweep 1      3.210040e-01&#xA;CONVERGENCE FACTOR: 3.140491&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6702" LastEditorUserId="6702" LastEditDate="2013-12-03T16:13:44.353" LastActivityDate="2013-12-09T20:19:40.403" Title="Multigrid stops converging when more grid levels are used" Tags="|boundary-conditions|convergence|multigrid|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10210" PostTypeId="2" ParentId="10206" CreationDate="2013-12-03T04:00:41.923" Score="6" Body="&lt;p&gt;The algorithm you have implemented is &lt;strong&gt;explicit&lt;/strong&gt;.  Crank-Nicolson is an &lt;strong&gt;implicit&lt;/strong&gt; method, and thus requires a solve.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-12-03T04:00:41.923" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10213" PostTypeId="1" CreationDate="2013-12-03T15:33:11.193" Score="6" ViewCount="618" Body="&lt;p&gt;How can show that the prerequisites for the Lax-Milgram Lemma holds if I have different test and trial spaces (which I think is the natural thing to have if at least part of the boundary is non-homogenously Dirichlet)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To take a simple example (in my actual case I have a non-homogenous Neumann part as well, but that is unproblematic), let's say that I have the Poisson equation,&#xA;\begin{align}&#xA;-\nabla^2 u &amp;amp;= f &amp;amp;u&amp;amp;\in\Omega \\&#xA;u &amp;amp;= u_D &amp;amp;u&amp;amp;\in\partial\Omega&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and write the weak form as: find $u\in H_D^1(\Omega)$ such that for all $v\in H_0^1(\Omega)$ it holds that $a(u,v) = l(v)$ where&#xA;\begin{align}&#xA;a(u,v) &amp;amp;= (\nabla u, \nabla v)_{L^2(\Omega)}\\&#xA;l(v) &amp;amp;= (f,v)_{L^2(\Omega)}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how can I show that $a(u,v)$ is coercive and bounded? If $u_D$ had been $0$ everywhere, then I would have had $a(u,v):H_0^1\times H_0^1\rightarrow \mathbf{R}$ so it would have been possible to show coercivity and continuity and then LM would guarantee a unique solution. But I have $a(u,v):H_D^1\times H_0^1\rightarrow \mathbf{R}$. Can I still show continuity and coercivity in the exact same way as usual and then Lax-Milgram (is it still called Lax-Milgram if it isn't from $X\times X$?) will hold? Or what should I do?&lt;/p&gt;&#xA;" OwnerUserId="4669" LastActivityDate="2013-12-03T15:49:40.327" Title="The Lax-Milgram Lemma in FEM with non-homogenous Dirichlet BC" Tags="|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10214" PostTypeId="2" ParentId="10213" CreationDate="2013-12-03T15:49:40.327" Score="8" Body="&lt;p&gt;Typically, you would decompose your solution into $u = u_D + u_0$, where $u_D$ satisfies inhomogeneous Dirichlet conditions. You can then solve for $u_0 \in H^1_0$ subject to $$\Delta u_0 = f-\Delta u_D.$$ &#xA;You would then recover a variational formulation over $H^1_0$ again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $u_D$ is non-unique. To remedy this in proofs, $u_D$ is often taken to be the minimum-energy extension of the trace satisfying boundary conditions. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-12-03T15:49:40.327" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10216" PostTypeId="1" AcceptedAnswerId="10218" CreationDate="2013-12-03T16:02:21.320" Score="4" ViewCount="2670" Body="&lt;p&gt;I’m attempting to write a C program to gather values from a coupled spring system: There is a wall, connected to a mass $m_1$ by a spring, then this mass is connected to a second mass $m_2$ by another spring.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values I require are the positions of the two masses and their velocities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using initial values for position and velocity I intend to calculate the movement of the two masses using the 4th-order Runge–Kutta method. I have successfully done this for positions, but I can’t figure out how to do it for the velocities. The differential equations for the velocities are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{alignat}{2}&#xA;\dot v_1 &amp;amp;= -\omega_1^2(x_1-R_1) &amp;amp;+ \omega_2^2(x_2-x_1-w-R_2)\\&#xA;\dot v_2 &amp;amp;=                      &amp;amp;- \omega_2^2(x_2-x_1-w-R_2)&#xA;\end{alignat}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $R$ is the rest length of the springs, $x$ is the position, $w$ is the width. The equations are correct; that isn’t the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I have is that I just don’t know how to express the Runge–Kutta algorithm for the velocities when &lt;strong&gt;both&lt;/strong&gt; $x$ positions are changing, do I increase them both at the same time in the algorithm? Do I increase one and then take the average?&lt;/p&gt;&#xA;" OwnerUserId="6734" OwnerDisplayName="Carterini" LastEditorUserId="7574" LastEditDate="2017-08-09T10:56:39.460" LastActivityDate="2017-08-09T10:56:39.460" Title="4th-order Runge-Kutta method for coupled harmonic oscillator" Tags="|ode|runge-kutta|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10217" PostTypeId="2" ParentId="10216" CreationDate="2013-12-03T18:23:33.257" Score="1" Body="&lt;p&gt;What you intend to do is not possible nor needed. Within Runge-Kutta you should already have an expression for your velocity. If not, you can calculate the velocity by using finite difference to approximate the first derivative of the position with respect to the time.&lt;/p&gt;&#xA;" OwnerUserId="6736" LastActivityDate="2013-12-03T18:23:33.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10218" PostTypeId="2" ParentId="10216" CreationDate="2013-12-03T18:32:56.483" Score="8" Body="&lt;p&gt;&lt;em&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/10217/7574&quot;&gt;pressure is on the right track&lt;/a&gt;, but I will elaborate a bit.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are solving a system of coupled ODEs, however it appears you are viewing this as two systems of two equations.  You need to combine them so that your system looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left(\begin{array}{c}&#xA;	\dot{x}_1 \\&#xA;	\dot{x}_2 \\&#xA;	\dot{v}_1 \\&#xA;	\dot{v}_2&#xA;\end{array}\right)&#xA;= \left(\begin{alignat}{1}&#xA;	&amp;amp; v_1 \\&#xA;	&amp;amp; v_2 \\&#xA;	-\omega_1^2(x_1-R_1) &amp;amp;+ \omega_2^2(x_2-x_1-w-R_2)\\&#xA;	&amp;amp;-\omega_2^2(x_2-x_1-w-R_2)&#xA;   \end{alignat}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this system of four ODEs with RK4 will solve for all your state variables simultaneously. Position and velocity do not need to be handled separately (and shouldn’t be).&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="7574" LastEditDate="2017-08-09T10:52:24.227" LastActivityDate="2017-08-09T10:52:24.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10219" PostTypeId="2" ParentId="10152" CreationDate="2013-12-03T19:37:55.097" Score="1" Body="&lt;p&gt;For the time-dependent 1D compressible Euler equations, &lt;a href=&quot;http://en.wikipedia.org/wiki/Sod_shock_tube&quot; rel=&quot;nofollow&quot;&gt;Sod's shock tube&lt;/a&gt; is a standard test case with an exact solution from method of characteristics. It has all the interesting elements of 1D Euler equations: shocks, expansions, and thermodynamic discontinuities. For 2D Euler, the &lt;a href=&quot;http://www.youtube.com/watch?v=mQDvuQsIC2E&amp;amp;noredirect=1&quot; rel=&quot;nofollow&quot;&gt;mach 3 flow over a step&lt;/a&gt; problem is popular in CFD and presents some extremely complex features (but no exact solution I'm aware of).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stokes' problems for 2D unsteady incompressible low-Re viscous flow admit exact solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vortex &lt;em&gt;shedding&lt;/em&gt; is not something you're going to find an exact analytical solution to, although there is active research in proper orthogonal decomposition for simple geometries. Turbulent transition is also not something you'll find an analytical solution for. There is some well-developed theory for homogeneous isotropic turbulence.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-12-03T19:37:55.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10220" PostTypeId="2" ParentId="10198" CreationDate="2013-12-03T21:20:55.983" Score="1" Body="&lt;p&gt;You can solve the four equations in four unknowns ignoring the modulus.  If any intermediate steps result in a number greater than the modulus, you can reduce by it or not as you wish.  That will get you to four equations of the form $a_iL_i=b_i \pmod {10930888}$  Now they will be soluable or not, depending on whether $a_i$ is coprime to $10930888$  If it is coprime, there will be a unique solution.  If not, there will be none or $\gcd (a_i,10930888)$&lt;/p&gt;&#xA;" OwnerUserId="6737" LastActivityDate="2013-12-03T21:20:55.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10221" PostTypeId="1" AcceptedAnswerId="10226" CreationDate="2013-12-03T22:10:50.260" Score="2" ViewCount="194" Body="&lt;p&gt;What is the best way to impose a &quot;constant constraint&quot; for a PDE? Specifically, I want to solve an eigenvalue problem $Au=\lambda u$ on the rectangle $(0,2\pi)\times(-\pi/2,\pi/2)$ with periodicity constraints horizontally and constant constraint on the lines $y=\pi/2, y=-\pi/2$. Here a constant constraint means that $u(x,\pi/2)$ is constant &lt;em&gt;and the constant is not fixed&lt;/em&gt;. The same in $-\pi/2$ with possibly a different constant. I use a finite difference grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to impose the constant constraint in the matrix $M$ which is the discretized of $A$?&lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2013-12-04T02:07:47.807" Title="How to impose a constant constraint PDE" Tags="|finite-difference|boundary-conditions|constraints|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10222" PostTypeId="1" AcceptedAnswerId="10224" CreationDate="2013-12-03T22:16:34.303" Score="3" ViewCount="851" Body="&lt;p&gt;Is it possible to solve the eigenvalue problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax = \lambda Mx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using ARPACK when $A$ and $M$ are both non-symmetric complex matrices? According to &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/UG/node138.html&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; documentation, the function &lt;code&gt;znaupd&lt;/code&gt; only works when $M$ is symmetric positive definite. I have verified that the results are only correct when $M$ meets this requirement. I have heard that Matlab's &lt;code&gt;eigs&lt;/code&gt; function is loosely based on ARPACK and it is able to handle non-symmetric $M$ matrices. Do they just convert it to a standard eigenvalue problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not possible using ARPACK, what are some alternatives?&lt;/p&gt;&#xA;" OwnerUserId="4524" LastActivityDate="2013-12-03T23:06:33.490" Title="Generalized eigenvalue problem using ARPACK" Tags="|sparse-matrix|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10223" PostTypeId="2" ParentId="10200" CreationDate="2013-12-03T22:48:35.933" Score="1" Body="&lt;p&gt;I agree with BrianBorchers' general approach. To summarize my discussion with him in the comment thread below his answer, I suggest some modifications to his approach:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To recap the gist of Brian's answer, the quickest way to calculate the trace of $A^{+}B$ for many values of $B$ is to precompute $A^{+}$, and then perform only the matrix-vector products needed for the diagonal elements of $A^{+}B$, and sum these diagonal elements. In the limit of large numbers of $B$, the expense of those matrix-vector products will dominate the cost of forming $A^{+}$. $A$ is also sparse, so you could exploit sparsity for speed, too. Memory becomes a concern when $A$ is large. You mentioned that $A$ is no larger than $5000 \times 10000$, so with double precision arithmetic, it could require up to 400 MB to store $A$. That's a lot, but storing it as a dense matrix is possible; also storing each $B$ as a dense matrix would not be possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Brian suggests calculating $A^{+}$ as $(A^{T}A)^{-1}A = A^{+}$, since $A$ has full column rank. We can rearrange this problem into $(A^{T}A)A^{+} = A$. This immediately suggests three approaches, all based on the normal equations of linear regression (i.e., least squares):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Use sparse Cholesky factorization of $A^{T}A$ to find a sparse matrix $R$ such that $R^{T}R = A^{T}A$; this step can be carried out using a call to CHOLMOD. Call $RA^{+} = X$. Solve the equation $R^{T}X = A$ for $X$ using a sparse LU factorization such as in UMFPACK; $R^{T}$ is lower triangular, so this solve should be quick, and only use symbolic factorization and some pivoting. Then solve $RA^{+} = X$ for $A^{+}$, again using a sparse LU factorization. $R$ is upper triangular, and it might be possible to reuse some of the information from the previous LU factorization in this solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Use a sparse QR factorization to find $R$ -- it's the same matrix as the one above. SuiteSparseQR is one implementation of sparse QR by Tim Davis, author of UMFPACK and CHOLMOD. This part saves you a matrix multiply (which doesn't really matter in the grand scheme of things), and you can either use the two sparse LU factorization approach above, or treat it as a least squares problem directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Use an SVD factorization of $A$, and the formula in &lt;a href=&quot;http://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse#Singular_value_decomposition_.28SVD.29&quot; rel=&quot;nofollow&quot;&gt;this Wikipedia article&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of these three approaches, the sparse QR decomposition and SVD approach are what I used in my thesis. In your case, I wouldn't necessarily recommend the SVD approach because I'm not aware of a good recent implementation of sparse SVD; if one is available, it's worth trying. Cholesky factorization is supposed to do a better job of exploiting sparsity than QR, and is probably fastest. QR is generally used in practice. SVD has the best numerical stability, followed by Cholesky and QR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume in Brian's answer that when he suggests calculating $(A^{T}A)^{-1} = (R^{T}R)^{-1}$, he probably really means, &quot;solve the system $(R^{T}R)A^{+} = A$&quot;. &lt;a href=&quot;http://www.johndcook.com/blog/2010/01/19/dont-invert-that-matrix/&quot; rel=&quot;nofollow&quot;&gt;Solving a linear system by calculating the inverse explicitly and then multiplying is conventionally considered bad practice&lt;/a&gt;. &lt;a href=&quot;http://arxiv.org/pdf/1201.6035.pdf&quot; rel=&quot;nofollow&quot;&gt;Evidence suggests it's really not that bad for accuracy&lt;/a&gt;, but the inverse of a sparse matrix is frequently dense, so inverting the matrix is likely to require more storage and more operations than a sparse factorization. Furthermore, $R$ is better conditioned than $R^{T}R$, so recasting the linear system $(R^{T}R)A^{+} = A$ as two separate linear systems is at least as accurate as solving the single linear system; solving it as two separate systems is the conventional advice given in least squares problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Brian also rightly pointed out that there are approaches to calculate $A^{+}B$ directly using QR factorizations, and that these approaches would be unfavorable because they would require more computation and probably more memory. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-03T22:48:35.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10224" PostTypeId="2" ParentId="10222" CreationDate="2013-12-03T23:06:33.490" Score="4" Body="&lt;p&gt;After reading the documentation closer, it recommends converting the problem to a standard eigenvalue problem when $A$ and $M$ are non-Hermitian positive semi-definite. Section 3.2.2 from the ARPACK documentation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/BazHg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4524" LastActivityDate="2013-12-03T23:06:33.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10225" PostTypeId="2" ParentId="10221" CreationDate="2013-12-04T01:20:35.070" Score="2" Body="&lt;p&gt;$$&#xA;\left.\frac{\partial u}{\partial x}\right|_{y=\pi/2} = 0 &#xA;$$&#xA;would be one strategy, though it's a little unusual to impose a tangential derivative boundary condition, and your system will likely be under-constrained if you don't fix the value of $u$ at at least one point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your $A$ is a decent operator, then your combination of periodic and fixed constant conditions will likely remove the $y$ dimension from the problem all together, allowing you to solve a 1-D problem instead. Can you say a little more about what $A$ is?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-12-04T01:20:35.070" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10226" PostTypeId="2" ParentId="10221" CreationDate="2013-12-04T02:07:47.807" Score="4" Body="&lt;p&gt;Let's say you have numbered your vertices along the line $y=-\pi/2$ as $u_1,u_2,\ldots,u_n$ (so that you'll have a total of $N=nm$ vertices, with $m$ rows of vertices covering your domain). Then your constraint can be written as&#xA;$$&#xA;  u_2=u_1, \\ u_3=u_1, \\ \vdots \\ u_n=u_1.&#xA;$$&#xA;(And the same, obviously, for the vertices at the upper boundary.) This can be written as a homogenous constraint $CU=0$ where $C$ is a rectangular matrix where the constraints above will yield $m-1$ rows and $N$ columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving linear systems $AU=F$ where $CU=0$ is exactly the kind of operation we always do when we deal with hanging nodes, for example. Typically, we would eliminate $u_2, \ldots, u_n$ from the linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a number of descriptions of this in the literature. The earliest, I believe, is in Babuska and Reinboldt from the 1970s. I'm quite sure it's in Graham Carey &quot;Computational Grids&quot; book. I know it's in mine and Oliver Kayser-Herold's 2009 paper on hp, see # 27 at &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications.html#x-reviewed&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/publications.html#x-reviewed&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-04T02:07:47.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10227" PostTypeId="1" AcceptedAnswerId="10229" CreationDate="2013-12-04T02:29:15.613" Score="2" ViewCount="185" Body="&lt;p&gt;I have the same single thread problem, which is something like a simple&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include&amp;lt;stdlib.h&amp;gt;&#xA;#include&amp;lt;iostream&amp;gt;&#xA;#include&amp;lt;stdio.h&amp;gt;&#xA;#include&amp;lt;time.h&amp;gt;&#xA;//#include&amp;lt;ctime.h&amp;gt;&#xA;&#xA;int main(){&#xA;&#xA;clock_t start=clock();&#xA;for (int i=0;i&amp;lt;100 000 000; i++){&#xA;//do something here&#xA;}&#xA;&#xA;double duration = (clock()-start)/(double)CLOCKS_PER_SEC;&#xA;cout&amp;lt;&amp;lt;endl&amp;lt;&amp;lt;&quot;Total consumption time is : &quot;&amp;lt;&amp;lt;duration&amp;lt;&amp;lt;&quot;seconds &quot;&amp;lt;&amp;lt;endl;&#xA;// or use &#xA;// printf(&quot;\n Total consumption time is : %lf seconds&quot;,duration);&#xA;system(&quot;pause&quot;);&#xA;&#xA;return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Occasionally when I was running them on different computer: older one with XP OS and Pentium V CPU, newer one with Win7 OS x86 and i7 CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is very strange that I found the older computer runs the same program significantly faster. Then I found it is general for other similar programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why I got such result? Is it because XP OS is faster than Win7? &lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6693" LastEditDate="2013-12-04T04:38:36.593" LastActivityDate="2013-12-04T04:38:36.593" Title="Why the same program runs faster on an older computer?" Tags="|efficiency|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10228" PostTypeId="2" ParentId="10152" CreationDate="2013-12-04T03:06:00.860" Score="0" Body="&lt;p&gt;Aside from those mentioned above, there are a couple more well-known hydrodynamics problems that are convenient for code testing: (i) strong explosion in atmosphere (spherical shock wave) - described well in Landau,Livshits &quot;Fluid dynamics&quot;; and (ii) Rayleigh-Taylor instability - described well in &quot;Hydrodynamic and Hydromagnetic Stability&quot; by Chandrasekhar.&lt;/p&gt;&#xA;" OwnerUserId="4325" LastActivityDate="2013-12-04T03:06:00.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10229" PostTypeId="2" ParentId="10227" CreationDate="2013-12-04T03:31:23.367" Score="6" Body="&lt;p&gt;There are so many variables that play into the speed with which a program runs that it is impossible to tell from just your description. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Did other programs run at the same time?&lt;/li&gt;&#xA;&lt;li&gt;What is the clock speed of your processor?&lt;/li&gt;&#xA;&lt;li&gt;How was the program compiled and which processor was it optimized for?&lt;/li&gt;&#xA;&lt;li&gt;How long does the program run to begin with -- for example, is it within the accuracy of the timer?&lt;/li&gt;&#xA;&lt;li&gt;Is the loop simple enough for the compiler to optimize it away?&lt;/li&gt;&#xA;&lt;li&gt;Are you timing the entire program execution including loading it, or only the actual run time of the loop?&lt;/li&gt;&#xA;&lt;li&gt;If you time the entire program execution including loading, how does the set of shared libraries that the program links to differ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are many other questions that make it impossible to answer your question in this generality.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-04T03:31:23.367" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10231" PostTypeId="2" ParentId="10227" CreationDate="2013-12-04T03:32:29.903" Score="1" Body="&lt;p&gt;The faster CPU may be more than made up for by the heavier operating system. If you wanted a true comparison, you'd run the test with the same operating system on both computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, depending on what you're actually doing inside that loop, the program may spend more time on memory reads/writes than floating point operations, in which case the performance is more dependent on the cache than the processor clock speed.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-12-04T03:32:29.903" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10232" PostTypeId="1" AcceptedAnswerId="10233" CreationDate="2013-12-04T15:10:50.473" Score="3" ViewCount="5422" Body="&lt;p&gt;I have a while loop, in which I use a Matrix &lt;code&gt;A&lt;/code&gt;, vectors &lt;code&gt;B&lt;/code&gt; and &lt;code&gt;x&lt;/code&gt; with varying  dimensions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int count=0;&#xA;while(true){&#xA;count+=1;&#xA;&#xA;MatrixXd A(count,2*count);&#xA;VectorXd B(2*count);&#xA;&#xA;// assignment of A, and B&#xA;//&#xA;&#xA;VectorXd x = A.householderQr().solve(B);&#xA;&#xA;// output of x&#xA;&#xA;if (x.norm()&amp;gt;30)&#xA;break;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However the program built this way always collapses.&#xA;I tried to use pointers to such matrices or vector, and got similar failures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I handle such issue?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2013-12-04T17:13:31.627" Title="How to change the dimensions of an Eigen Matrix in a loop?" Tags="|c++|matrix|memory-management|eigen|vector|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10233" PostTypeId="2" ParentId="10232" CreationDate="2013-12-04T17:13:31.627" Score="7" Body="&lt;p&gt;You have a size mismatch issue:  &lt;code&gt;A&lt;/code&gt; is a &lt;code&gt;count x 2*count&lt;/code&gt; matrix, and you are trying to solve &lt;code&gt;Ax=B&lt;/code&gt; with &lt;code&gt;B&lt;/code&gt; a &lt;code&gt;2*count x 1&lt;/code&gt; vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, if you compile without &lt;code&gt;-DNDEBUG&lt;/code&gt;, you should get a nice assertion telling you this is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To resize a matrix or vector:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A.resize(count, 2*count);&#xA;B.resize(count);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4337" LastActivityDate="2013-12-04T17:13:31.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10235" PostTypeId="1" AcceptedAnswerId="10240" CreationDate="2013-12-04T21:54:37.570" Score="11" ViewCount="11176" Body="&lt;p&gt;I am trying to plot the wave function for a particle in a 3D box. This requires me to plot 4 variables: x, y, z axes and the probability density function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The probability density function is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;abs((np.sin((p*np.pi*X)/a))*(np.sin((q*np.pi*Y)/b))*(np.sin((r*np.pi*Z)/c)))**2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using &lt;code&gt;np.arange()&lt;/code&gt; for the X, Y and Z.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I have read that to do this you need to plot the surface of a 4D plot.&lt;/strong&gt;&#xA;Here is what it is supposed to look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XZxxC.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="6744" LastActivityDate="2013-12-05T18:23:28.010" Title="How do I plot the surface of a 4D plot?" Tags="|probability|plotting|computational-physics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10236" PostTypeId="2" ParentId="10235" CreationDate="2013-12-05T01:18:17.487" Score="9" Body="&lt;p&gt;The traditional approach for scalar field-based data (temperature, velocity magnitude, pressure, density, etc.) plotted over two or three space dimensions uses color. It's important to note that choice of color scheme can distort your impressions of the data. For this reason, &lt;strong&gt;do not use a rainbow color scheme.&lt;/strong&gt;  (For why, see &lt;a href=&quot;http://www.jwave.vt.edu/~rkriz/Projects/create_color_table/color_07.pdf&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://blog.visual.ly/rainbow-color-scales/&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://nbviewer.ipython.org/gist/olgabot/5357268&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;http://blog.ggplot2.org/post/24607351280/choosing-colour-palettes-part-ii-educated-choices&quot;&gt;here&lt;/a&gt;.) Unfortunately, rainbow is the default color scheme in MATLAB and matplotlib.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're trying to highlight changes in intensity, using a scheme that varies in saturation works well, like one that ranges from white (zero density) to black (maximum density). Transparency can also work well. A tricky problem with 3-D plots when using color is that you'll need to look at the data from multiple perspectives to get a fuller picture of trends and features; you may also need to plot slices.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-05T01:18:17.487" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10237" PostTypeId="1" CreationDate="2013-12-05T04:15:03.907" Score="1" ViewCount="216" Body="&lt;p&gt;I am trying to find the Nusselt number for a small element of the outside of a cylinder that has a height of $\Delta z$. I found the average Grashof number of a surface as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Gr_{L}=\frac{\beta \rho (T_{s}-T_{\infty})L_{c}^{3} }{\nu^{2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I wanted to find $Gr_{x}$ of a certain element, would I use a characteristic length of $\Delta z$ or could I find the z-location of the elements centroid and use that as the $L_{c}$. For example, If the cell was located at a z coordinate of 1 m, would that be the characteristic length or would the characteristic length be the height of the differential element?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking that I could treat each element as a small isothermal cylinder and have the characteristic length of each element be $\Delta z$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, does anyone know the a Nusselt number correlation for a vertical cylinder with a Prandtl number of less than one? I have only found one for when it is greater than 1. Thank you&lt;/p&gt;&#xA;" OwnerUserId="6714" LastEditorUserId="27" LastEditDate="2013-12-05T23:05:29.067" LastActivityDate="2013-12-05T23:05:29.067" Title="Characteristic length of differential element of cylinder surface?" Tags="|finite-element|finite-volume|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10238" PostTypeId="1" AcceptedAnswerId="10239" CreationDate="2013-12-05T09:04:52.407" Score="2" ViewCount="156" Body="&lt;p&gt;Suppose that I have the following vectors $v_1$, $v_2$, $v_3$ $\in R^n$,$n= 9$. What I want to do exactly is to combine these $3$ vectors into $1$ representative vector $V$.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;According to the following article: &lt;em&gt;Document Classification and Page Stream Segmentation for Digital Mailroom Applications&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;they Perform at first average pooling or just the mean of the $3$&#xA;vectors to obtain a single representation of the $3$ vectors&lt;/li&gt;&#xA;&lt;li&gt;The new vector is then &lt;em&gt;L2-Normalized&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is how I interpreted these steps by using the following example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_1 = \ [ x_1,x_2,...,x_n \ ]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_2 = \ [ y_1,y_2,...,y_n \ ]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_3 = \ [ z_1,z_2,...,z_n \ ]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1: Average Pooling&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V = \frac{v_1 + v_2 + v_3}{3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V = \ [ w_1,w_2,...,w_n \ ]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V$ is the vector after the average pooling or mean&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2: L2-Norm&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Norm = \sqrt{w_1^2+\ldots+w_n^2}$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 3 : divide the components $w_i$ of $V$ by the $Norm$&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these steps correct?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Code Sample : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;algorithm&amp;gt;&#xA;using namespace std;&#xA;&#xA;int main()&#xA;{&#xA;     const int size = 5;&#xA;     vector&amp;lt;float&amp;gt; v1{1,1,1,1,1};&#xA;     vector&amp;lt;float&amp;gt; v2{1,1,1,1,1};&#xA;     vector&amp;lt;float&amp;gt; v3{1,1,1,1,1};&#xA;     vector&amp;lt;vector&amp;lt;float&amp;gt;&amp;gt; V;&#xA;     V.push_back(v1);&#xA;     V.push_back(v2);&#xA;     V.push_back(v3);&#xA;     vector&amp;lt;float&amp;gt; doc(size,0.0);&#xA;     //============================================//&#xA;     //STEP 1: Average Pooling&#xA;     for(size_t i = 0; i &amp;lt; V.size();i++)&#xA;     {&#xA;         for(size_t j = 0; j &amp;lt; 5;j++)&#xA;         {&#xA;             doc[j] += V[i][j];&#xA;         }&#xA;     }&#xA;&#xA;     for(size_t i = 0; i &amp;lt; doc.size();i++)&#xA;     {&#xA;        doc[i]= doc[i]/(float) V.size();&#xA;     }&#xA;    //============================================//&#xA;    //STEP 2: L2-Normalization&#xA;     float y =  *max_element(std::begin(doc), std::end(doc));&#xA;     float m_sum = 0.0;&#xA;&#xA;     for (int k = 0; k &amp;lt; doc.size(); k++)&#xA;     {&#xA;        m_sum +=  pow(doc[k]/y,2);&#xA;     }&#xA;     //STEP 3: Divide components by the Norm (m_sum)&#xA;     for(size_t i = 0; i &amp;lt; V.size();i++)&#xA;     {&#xA;        cout &amp;lt;&amp;lt; doc[i]/sqrt(m_sum)&amp;lt;&amp;lt;&quot;  &quot;;&#xA;     }&#xA;     cout &amp;lt;&amp;lt; endl;&#xA;     return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4918" LastEditorUserId="4918" LastEditDate="2013-12-06T15:27:35.757" LastActivityDate="2013-12-06T15:27:35.757" Title="combine $n$ vectors using $L2-normalization$" Tags="|algorithms|vector|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10239" PostTypeId="2" ParentId="10238" CreationDate="2013-12-05T12:37:40.147" Score="3" Body="&lt;p&gt;My interpretation of this sentence is the same as yours, and I would have implemented the algorithm the same way.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-05T12:37:40.147" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10240" PostTypeId="2" ParentId="10235" CreationDate="2013-12-05T17:42:01.090" Score="13" Body="&lt;p&gt;This is not really 4D data.  As Geoff said, it's 3D scalar data, i.e. you're visualizing a scalar function of three variables: $f(x,y,z)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ways to visualize this kind of data, and many tools that will help you.  I'll show you a few styles of plots you can make.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Contour plot showing one or more $f(x,y,z) = \text{(const.)}$ surfaces, possibly with transparency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Mathematica,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ContourPlot3D[&#xA; Abs[Sin[\[Pi] x] Sin[\[Pi] y] Sin[\[Pi] z]]^2 == 1/2,&#xA; {x, -1, 1}, {y, -1, 1}, {z, -1, 1}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/s0GKo.png&quot; width=&quot;360&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Show the surfaces of constant probability 0.2, 0.5 and 0.8:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ContourPlot3D[&#xA; Abs[Sin[\[Pi] x] Sin[\[Pi] y] Sin[\[Pi] z]]^2,&#xA; {x, -1, 1}, {y, -1, 1}, {z, -1, 1}, Contours -&amp;gt; {0.2, 0.5, 0.8}, &#xA; ContourStyle -&amp;gt; (Directive[#, Opacity[0.25]] &amp;amp; /@ {Yellow, Orange, Red}), &#xA; Lighting -&amp;gt; &quot;Neutral&quot;, Mesh -&amp;gt; None]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/eluHH.png&quot; width=&quot;360&quot;&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can do some type of &lt;a href=&quot;https://en.wikipedia.org/wiki/Volume_rendering&quot; rel=&quot;noreferrer&quot;&gt;volume visualization&lt;/a&gt;, possibly with cutouts and slicing.  You'll be able to assign a colour and an opacity to each point in 3D.  More advanced tools will also let you choose a transfer function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;imgdata = &#xA;  Table[Abs[Sin[\[Pi] x] Sin[\[Pi] y] Sin[\[Pi] z]]^2, &#xA;    {x, -1., 1, .01}, {y, -1., 1, .01}, {z, -1., 1, .01}];&#xA;&#xA;img = Image3D[imgdata, ClipRange -&amp;gt; {{150, 200}, {0, 100}, {0, 200}}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wRgzA.png&quot; width=&quot;360&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Slicing often helps, especially if you can interactively control which slice to display.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Image3DSlices[img, Range[1, 200, 10]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xosa3.png&quot; width=&quot;500&quot;&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These examples were meant as ideas for what types of visualizations you can try to create.  There are many different free and commercial tools that you can use to make the plots.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2013-12-05T18:23:28.010" LastActivityDate="2013-12-05T18:23:28.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10241" PostTypeId="1" CreationDate="2013-12-05T19:09:34.063" Score="-2" ViewCount="80" Body="&lt;p&gt;How can I generate a list of all the variables that have been assigned in an R language session?&lt;/p&gt;&#xA;" OwnerUserId="6754" LastActivityDate="2017-04-03T21:11:27.433" Title="R language - how to list variables in use" Tags="|r|" AnswerCount="1" CommentCount="2" ClosedDate="2017-04-04T11:41:37.313" ContentLicense="CC BY-SA 3.0" />
  <row Id="10242" PostTypeId="2" ParentId="10241" CreationDate="2013-12-05T19:49:53.030" Score="3" Body="&lt;p&gt;For the sake of completeness of the answer, you can use &lt;code&gt;ls()&lt;/code&gt; to list all the assigned variables and data frames. This is similar to &lt;code&gt;who&lt;/code&gt; and &lt;code&gt;whos&lt;/code&gt; in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to get the list data.frames only you can use something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;ls()[grepl('data.frame', sapply(ls(), function(x) class(get(x))))]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should probably ask questions like this on R forums.&lt;/p&gt;&#xA;" OwnerUserId="6716" LastEditorUserId="23684" LastEditDate="2017-04-03T21:11:27.433" LastActivityDate="2017-04-03T21:11:27.433" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10244" PostTypeId="2" ParentId="7624" CreationDate="2013-12-05T23:40:19.607" Score="3" Body="&lt;p&gt;&lt;em&gt;I don't know why I put this answer in a comment...&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I needed this I would probably use &lt;a href=&quot;http://mathema.tician.de/software/pyopencl/&quot; rel=&quot;nofollow&quot;&gt;PyOpenCL&lt;/a&gt;. General purpose GPU coding is still quite low level (try OpenCL C interface, it's tough going). Yet PyOpenCL seems to abstract as much as possible and appears to have considerable momentum behind it. &lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2013-12-05T23:40:19.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10247" PostTypeId="1" CreationDate="2013-12-06T08:25:49.507" Score="0" ViewCount="101" Body="&lt;p&gt;Is there any tool to plot 3d dimensional histogram in a spatial grid? Preferably are there any filters or operators in Paraview or Visit for this?&lt;/p&gt;&#xA;" OwnerUserId="4907" LastActivityDate="2014-07-17T03:18:14.890" Title="Plotting 3d dimensional histogarm in a spatial grid" Tags="|visualization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10250" PostTypeId="1" CreationDate="2013-12-06T15:53:31.093" Score="2" ViewCount="596" Body="&lt;p&gt;I heard that subspace iteration plus Ritz acceleration could improve the performance a lot for solving clustered eigenvalues, for the eigenvalues and eigenvectors could converge linearly with ratio $\lambda_{p+1}/\lambda_j$, $j=1,\ldots,p$.  For Hermitian Matrix this is even faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is really fantastic.  Now for any $n\times n$ Complex Matrix $C$ with sufficient eigenvectors that is not deficit, we can construct an algorithm to make all the eigenvalues be solved in an extremely efficient way:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Add a big number to the diagonal elements to make the eigenvalue large in magnitude.&lt;/li&gt;&#xA;&lt;li&gt;Add a zero column together with a zero row to make it an $n+1\times n+1$ matrix $C'$.&lt;/li&gt;&#xA;&lt;li&gt;Modify $C'_{n+1\times n+1}$ with a very small nonzero number.&lt;/li&gt;&#xA;&lt;li&gt;Use Subspace Iteration Algorithm with $n$ orthogonal $n+1$ dimensional vectors.  According to the proof of the Algorithm it should converge very fast, for the first $n$ eigenvalues are respectively huge to the last one.&lt;/li&gt;&#xA;&lt;li&gt;Subtract the big number to the solved $n$ eigenvalues to get the original ones.&#xA;Does anyone think it possible?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6765" OwnerDisplayName="Zhong" LastEditorUserId="3481" LastEditDate="2013-12-07T15:19:56.140" LastActivityDate="2013-12-08T14:28:06.963" Title="About Subspace Iteration for Eigenvalues" Tags="|linear-algebra|eigensystem|matrix|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10251" PostTypeId="1" AcceptedAnswerId="10252" CreationDate="2013-12-06T19:03:21.237" Score="8" ViewCount="967" Body="&lt;p&gt;I would like to compute, as part of the solution of the Laplace equation using the Fast Multipole Method, the second derivative of &lt;a href=&quot;http://mathworld.wolfram.com/AssociatedLegendrePolynomial.html&quot;&gt;the associated legendre functions of the first kind&lt;/a&gt; .&#xA;Specifically, I am looking for C implementations or just the correct recurrence relation to use to write the function myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to look for recurrence relations and found &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S1464189500001010&quot;&gt;this&lt;/a&gt; but I suspect that it might not be correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also tried to search for ready-to-use functions like &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Associated-Legendre-Polynomials-and-Spherical-Harmonics.html&quot;&gt;these&lt;/a&gt; but I have only stumbled upon functions for the first derivative of the associated functions, not the second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If somebody can point me in the right direction, I will be really grateful!&#xA;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="3558" LastActivityDate="2018-03-27T06:56:02.490" Title="Second derivative of the Associated Legendre functions" Tags="|special-functions|numerics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10252" PostTypeId="2" ParentId="10251" CreationDate="2013-12-06T20:05:24.497" Score="3" Body="&lt;p&gt;You can find the recurrence relations for the first derivative (with respect to the variable) in the Digital Library of Mathematical Functions (&lt;a href=&quot;http://dlmf.nist.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;DLMF&lt;/a&gt;) in &lt;a href=&quot;http://dlmf.nist.gov/14&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chapter 14&lt;/a&gt;, sections &lt;a href=&quot;http://dlmf.nist.gov/14.10&quot; rel=&quot;nofollow noreferrer&quot;&gt;14.10&lt;/a&gt; and &lt;a href=&quot;http://dlmf.nist.gov/14.11&quot; rel=&quot;nofollow noreferrer&quot;&gt;14.11&lt;/a&gt;. You'll have to take care of the stability of the recurrence, though. You should consult the references mentioned in this chapter (see final section on &lt;a href=&quot;http://dlmf.nist.gov/14.34&quot; rel=&quot;nofollow noreferrer&quot;&gt;software&lt;/a&gt; and the references it points to).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the GSL library you mention does compute higher order derivatives (with respect to the variable). You can find another implementation in &lt;a href=&quot;http://scipy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciPy&lt;/a&gt; where it is explicitly mentioned that the routine calculates higher order derivatives (see &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.special.lpmn.html#scipy.special.lpmn&quot; rel=&quot;nofollow noreferrer&quot;&gt;man page&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="-1" LastEditDate="2018-03-27T06:56:02.490" LastActivityDate="2018-03-27T06:56:02.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10253" PostTypeId="1" CreationDate="2013-12-07T14:26:08.370" Score="9" ViewCount="528" Body="&lt;p&gt;I have a dense matrix A and its corresponding inverse $A^{-1}$. The Woodbury matrix identity states:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (A + UCV)^{-1} = A^{-1} - A^{-1}U(C^{-1} + VA^{-1}U)^{-1}VA^{-1} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to perform small updates to the diagonal of $A$ and compute corresponding $A^{-1}$. The updates can actually be described as a constant $k$ multiplying an identity matrix: $kI$ and in my case $U=V^T$ where $V$ is a tall matrix with just ones and zeros that selects the desired diagonal positions of $A$ to modify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything is fine so far, except $k$ needs to be quite large (e.g., $10^6$). I am noticing that for large $k$, changes made are not &quot;reversible&quot; without significant error. What I mean by reversible is performing two update steps, one with $kI$ on the original $A$ and another with $-kI$ on the updated matrix to recover the original $A^{-1}$. If $k$ is small enough, it works fine, but otherwise I run into serious accuracy issues and the resulting matrix does not match the initial one closely. For my target application, I need to be able to apply $k$ to diagonal elements or $-k$ at will, so my question is: Is there a clever way to do this which preserves numerical accuracy even for large values of $k$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am using double precision for all values stored and for computation. Furthermore, $k$ is a real scalar, but the matrix $A$ is complex, although that should not really matter in this case.&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="1804" LastEditDate="2016-05-27T20:41:27.920" LastActivityDate="2016-05-27T20:41:27.920" Title="Updating matrix diagonal with Woodbury matrix identity and maintaining numerical accuracy" Tags="|matrix|precision|accuracy|inverse|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10254" PostTypeId="2" ParentId="10250" CreationDate="2013-12-07T15:37:52.410" Score="2" Body="&lt;p&gt;My gut instinct says no. Suppose that hypothetically one of the eigenvalues of your original matrix $C$ is $\lambda_k = 1.0$, while with your method you add an artificial eigenvalue $\lambda_{n+1} = 10^{17}$. Using your method, you'll find an eigenvalue $10^{17}+1.0$ for the matrix $C'$ which, in floating point arithmetic, gets rounded to $10^{17}$. You'd then subtract off $10^{17}$ for all the eigenvalues you found for $C'$ and get that $\lambda_1 = 0$, which is pretty far off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's an extreme case, but even for the eigenvalues whose magnitudes are not so small compared to $\lambda_{p+1}$, you'll have poor numerical precision. In floating point arithmetic, computing $(a+b)-b$ when $b$ is much larger than $a$ will lose many digits of accuracy, giving you an unstable algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this is just my first guess and I don't have a formal proof for you. It's possible that there's a numerically stable variant of this algorithm (like Gram-Schmidt vs. Householder orthogonalization) or that I'm entirely wrong.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-12-07T15:37:52.410" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10256" PostTypeId="2" ParentId="1617" CreationDate="2013-12-08T12:32:47.787" Score="1" Body="&lt;p&gt;This page mentions some packages: &lt;a href=&quot;https://wiki.python.org/moin/FlowBasedProgramming&quot; rel=&quot;nofollow&quot;&gt;https://wiki.python.org/moin/FlowBasedProgramming&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another promising package not mentioned here is Snakemake.&lt;/p&gt;&#xA;" OwnerUserId="6772" LastActivityDate="2013-12-08T12:32:47.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10257" PostTypeId="2" ParentId="10250" CreationDate="2013-12-08T14:28:06.963" Score="1" Body="&lt;p&gt;Adding the (decoupled) extra dimension to the vector space cannot possibly help discriminate among the clustered eigenvalues in the original $n$ dimensions.  Any subspace iterations you do will maintain the partitioning of the first $n$ dimensions and the last (extra) one dimensional subspace.  While the theory tells you that &quot;separation&quot; can rapidly be obtained between a basis for the larger (original) subspace and the small one, you start out with perfect separation, so there's nothing to be gained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shifting the spectrum can (and often does) help in separating clustered eigenvalues, but it has to be done more skillfully than just adding an arbitrarily large real number to all eigenvalues.  The effect here would be to cluster those eigenvalues together more tightly.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-12-08T14:28:06.963" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10260" PostTypeId="1" CreationDate="2013-12-09T01:23:06.923" Score="2" ViewCount="1000" Body="&lt;p&gt;I'm interested in taking some partial derivatives of a 3 dimensional array in Matlab - say $A(i,j,k)$ approximates $f(x_i,y_j,z_k)$.  I need to approximate things like $\partial_{xy}f$, $\partial_{yz}f$, etc.  I've already built the finite difference matrices for a few of these derivatives, but ultimately this is going to be slow and inaccurate (I'd eventually like to operate on $256^3$ or even $512^3$ arrays).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have a good suggestion for computing these partials with FFT?  I've tried the following naive thing and it doesn't seem to work: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N=128;&#xA;L=2*pi;&#xA;&#xA;x=L/N*(-N/2:N/2-1);y=x;z=x;&#xA;[X,Y,Z]=meshgrid(x,y,z); &#xA;f=sin(X).*sin(Y).*sin(Z);  % A 2-pi periodic function to test&#xA;fp_exact=cos(X).*sin(Y).*sin(Z); % Its exact x-partial&#xA;&#xA;ik=1i*(2*pi/L)*[0:nx/2-1 0 -nx/2+1:-1]; % The spectral differentiation vector&#xA;&#xA;fhat=fftn(Z);  % Compute the 3-D FFT &#xA;&#xA;C=mat2cell(fhat,nx,ones(1,nx),ones(1,nx));  % So I can act on each column &#xA;result=cellfun(@(x) x.*ik',C,'UniformOutput',false);  % Multiply each column of Z by ik&#xA;fprimehat=cell2mat(result);  % Convert back to 3D array&#xA;fprime=ifftn(fprimehat);  % IFFT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above code seems hopeless - I think I'm missing something big and haven't really thought about this hard enough.  With the above definition of ik, I can compute a 1D spectral derivative with just &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fprime=ifft(ik.*fft(f));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any tips.&lt;/p&gt;&#xA;" OwnerUserId="3303" LastActivityDate="2013-12-09T13:30:59.847" Title="Partial derivatives of a 3D array in Matlab" Tags="|matlab|finite-difference|spectral-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10261" PostTypeId="2" ParentId="10260" CreationDate="2013-12-09T13:30:59.847" Score="1" Body="&lt;p&gt;It is far more complicated to compute derivatives with the FFT than necessary. If you just use the first order finite difference quotient, then you can approximate $\partial_x A(i,j,k)\approx \frac{A(i+i,j,k)-A(i,j,k)}{\Delta x}$ and similarly for the other derivatives. This can be done with a loop over all $i,j,k$ at a cost proportional to the number of elements, whereas the approach with the FFT is definitely more expensive.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-09T13:30:59.847" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10262" PostTypeId="1" AcceptedAnswerId="10263" CreationDate="2013-12-09T14:03:26.357" Score="4" ViewCount="93" Body="&lt;p&gt;So I am basically trying to solve the simplest case possible for the heat equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y3bSp.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on the domain [0;1] x [0,1] with boundary conditions 1.5 on one face of the square and 1.0 on all other faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using octave and I am solving the mathematical system from scratch, for learning purposes. (I know it is inefficient)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix I am getting:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-4   1   0   0   1   0   0   0   0   0   0   0   0   0   0   0&#xA;1  -4   1   0   0   1   0   0   0   0   0   0   0   0   0   0&#xA;0   1  -4   1   0   0   1   0   0   0   0   0   0   0   0   0&#xA;0   0   1  -4   0   0   0   1   0   0   0   0   0   0   0   0&#xA;1   0   0   0  -4   1   0   0   1   0   0   0   0   0   0   0&#xA;0   1   0   0   1  -4   1   0   0   1   0   0   0   0   0   0&#xA;0   0   1   0   0   1  -4   1   0   0   1   0   0   0   0   0&#xA;0   0   0   1   0   0   1  -4   0   0   0   1   0   0   0   0&#xA;0   0   0   0   1   0   0   0  -4   1   0   0   1   0   0   0&#xA;0   0   0   0   0   1   0   0   1  -4   1   0   0   1   0   0&#xA;0   0   0   0   0   0   1   0   0   1  -4   1   0   0   1   0&#xA;0   0   0   0   0   0   0   1   0   0   1  -4   0   0   0   1&#xA;0   0   0   0   0   0   0   0   1   0   0   0  -4   1   0   0&#xA;0   0   0   0   0   0   0   0   0   1   0   0   1  -4   1   0&#xA;0   0   0   0   0   0   0   0   0   0   1   0   0   1  -4   1&#xA;0   0   0   0   0   0   0   0   0   0   0   1   0   0   1  -4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;WRONG RHS VECTOR FROM MY QUESTION&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the RHS vector is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f =&#xA; -1.00000&#xA; -1.00000&#xA; -1.00000&#xA; -1.00000&#xA; -1.00000&#xA; -0.00000&#xA; -0.00000&#xA; -1.00000&#xA; -1.00000&#xA; -0.00000&#xA; -0.00000&#xA; -1.00000&#xA; -1.00000&#xA; -1.50000&#xA; -1.50000&#xA; -1.00000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;CORRECT UPDATED RHS VECTOR FROM THE ANSWER BELOW&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f =&#xA; -2.00000&#xA; -1.00000&#xA; -1.00000&#xA; -2.00000&#xA; -1.00000&#xA; -0.00000&#xA; -0.00000&#xA; -1.00000&#xA; -1.00000&#xA; -0.00000&#xA; -0.00000&#xA; -1.00000&#xA; -2.50000&#xA; -1.50000&#xA; -1.50000&#xA; -2.50000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is that I am getting some artifacts near the corners, like in the following print screen with added boundary conditions: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gI7oD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="6722" LastEditorUserId="6722" LastEditDate="2013-12-09T15:02:12.373" LastActivityDate="2013-12-11T21:57:51.147" Title="Problem with Static 2D Heat Eqaution" Tags="|poisson|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10263" PostTypeId="2" ParentId="10262" CreationDate="2013-12-09T14:45:54.987" Score="3" Body="&lt;p&gt;Your matrix is correct, but your boundary conditions are not being enforced on your RHS vector.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the row corresponding to an interior corner point that will use the two distinct boundary values.  The RHS vector of this row should be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-1.0 - 1.5$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because the corners require two values to be subtracted to the RHS.  The same is true for your other (interior) corner points where two distinct boundary condition values meet.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-12-11T21:57:51.147" LastActivityDate="2013-12-11T21:57:51.147" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10265" PostTypeId="1" AcceptedAnswerId="10271" CreationDate="2013-12-09T18:48:03.543" Score="3" ViewCount="81" Body="&lt;p&gt;I'm working with error estimates for Poisson's equation of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{E}^2_T = h_T^2\|-\Delta u - f\|_{L^2(T)} + \sum_{e\in \partial T} h_e\|n\cdot \nabla u\|_{L^2(e)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $T$ is an element and $h_T, h_e$ are representative sizes of the element volume and face, respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For skewed elements, I'm unclear on what is used for $h_T$ in practice. I can imagine several options (radius of the largest circumscribable sphere/circle, $d$th root of the volume $|T|^{1/d}$, max edge length) for $h_T$ (though I believe $h_e$ is always just given to be the size of the face?). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a preferred choice of for (slightly) skewed elements? &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2013-12-10T14:08:34.813" LastActivityDate="2013-12-10T14:08:34.813" Title="a posteriori error estimation for skewed elements" Tags="|error-estimation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10266" PostTypeId="1" CreationDate="2013-12-09T19:36:41.820" Score="3" ViewCount="138" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zn53O.jpg&quot; alt=&quot;Image attached is where I am at the moment. Blue dots=points/cities, Black x&amp;#39;s represent central points in each box that contains at least one city, and pink dots represent the midpoint of these centre points&quot;&gt;&#xA;Image attached is where I am at the moment. Blue dots=points/cities, Black x's represent central points in each box that contains at least one city, and pink dots represent the midpoint of these centre points&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing a short project using Lab Windows to approximate the minimal path between points on a 2D plane. (Graphically only, I don't need an actual path length)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this I am using real space re-normalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://scitation.aip.org/docserver/fulltext/aip/journal/cip/10/6/1.168588.pdf?expires=1387106816&amp;amp;id=id&amp;amp;accname=guest&amp;amp;checksum=350319AE8AA74E9002D87940052A0140&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://scitation.aip.org/docserver/fulltext/aip/journal/cip/10/6/1.168588.pdf?expires=1387106816&amp;amp;id=id&amp;amp;accname=guest&amp;amp;checksum=350319AE8AA74E9002D87940052A0140&lt;/a&gt; - An article by U. Yoshiyuki - &quot;Solving the Traveling Salesman Problem By a Statistical Physics Method&quot; describes exactly what I am trying to do&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have:&#xA;Plotted random integer valued points on a graph in my user interface&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Divided my plotting space evenly into boxes, i.e. a 2x2 graph (Just by plotting lines)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Created a new point in the centre of each box if there was a random point generated in that box&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Found the nearest edge point (for the 2x2 graph only)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     /*&#xA;     * Travelling Salesman Problem&#xA;     * A Graphical Approximation using Space Re-normalization&#xA;&#xA;     * Assuming there are  3 or more cities&#xA;     */&#xA;&#xA;     #include &quot;toolbox.h&quot;&#xA;     #include &amp;lt;cvirte.h&amp;gt;        &#xA;     #include &amp;lt;userint.h&amp;gt;&#xA;     #include &quot;TSP.h&quot;&#xA;&#xA;     static int panelHandle;&#xA;     void draw_borders(int,int,int,int);/*function prototype*/&#xA;&#xA;     int main (int argc, char *argv[])&#xA;     {&#xA;if (InitCVIRTE (0, argv, 0) == 0)&#xA;    return -1;  /* out of memory */&#xA;if ((panelHandle = LoadPanel (0, &quot;TSP.uir&quot;, PANEL)) &amp;lt; 0)&#xA;    return -1;&#xA;DisplayPanel (panelHandle);&#xA;RunUserInterface ();&#xA;DiscardPanel (panelHandle);&#xA;return 0;&#xA;     }&#xA;&#xA;     int CVICALLBACK Quit (int panel, int control, int event,&#xA;    void *callbackData, int eventData1, int eventData2)&#xA;     {&#xA;switch (event)&#xA;         {&#xA;    case EVENT_COMMIT:&#xA;        QuitUserInterface (0);&#xA;        break;&#xA;}&#xA;return 0;&#xA;     }&#xA;&#xA;     int CVICALLBACK ClearPlot (int panel, int control, int event,&#xA;    void *callbackData, int eventData1, int eventData2)&#xA;     {&#xA;switch (event)&#xA;{&#xA;    case EVENT_COMMIT:&#xA;        DeleteGraphPlot(panelHandle,PANEL_2X2GRAPH,-1,1);&#xA;        DeleteGraphPlot(panelHandle,PANEL_4X4GRAPH,-1,1); &#xA;        DeleteGraphPlot(panelHandle,PANEL_8X8GRAPH,-1,1); &#xA;        DeleteGraphPlot(panelHandle,PANEL_16X16GRAPH,-1,1); &#xA;        DeleteGraphPlot(panelHandle,PANEL_GRAPH_PATH,-1,1); &#xA;        break;&#xA;}&#xA;return 0;&#xA;     }&#xA;&#xA;     int CVICALLBACK Go (int panel, int control, int event,&#xA;    void *callbackData, int eventData1, int eventData2)&#xA;     {&#xA;switch (event)&#xA;{&#xA;    case EVENT_COMMIT:&#xA;&#xA;        int x[1024],y[1024]; /* City co-ordinates */&#xA;        int i,j,k,num=10; /* num is the number of cities to be plotted */&#xA;        int xi=0,xf=1024,yi=0,yf=1024; /* boundary points om graph */&#xA;        int flag=0;&#xA;&#xA;        /* Structures to store central points for each box containing a city*/&#xA;        typedef struct{&#xA;            int x,y;&#xA;        } Segment1;&#xA;        Segment1 Seg1[100][100];&#xA;&#xA;        typedef struct{&#xA;            int x,y;&#xA;        } Segment2;&#xA;        Segment2 Seg2[100][100];&#xA;&#xA;        typedef struct{&#xA;            int x,y;&#xA;        } Segment3;&#xA;        Segment3 Seg3[100][100];&#xA;&#xA;        typedef struct{&#xA;            int x,y;&#xA;        } Segment4;&#xA;        Segment4 Seg4[100][100];&#xA;&#xA;        /*2x2*/&#xA;        int quad1[2][2]; /*count for cities*/&#xA;        int grid1x2d[2][2],grid1y2d[2][2];/*grid containing x and y coordinate of central point locations in each box which contains a city*/          &#xA;        int grid1xx[4], grid1yy[4];/*same grid stretched to 1d for plotting*/ &#xA;        int mid1x2d, mid1y2d;&#xA;&#xA;        /*4x4*/&#xA;        int quad2[4][4];&#xA;        int grid2x2d[4][4],grid2y2d[4][4]; &#xA;        int grid2xx[16], grid2yy[16];&#xA;&#xA;        /*8x8*/&#xA;        int quad3[8][8];&#xA;        int grid3x2d[8][8],grid3y2d[8][8]; &#xA;        int grid3xx[64], grid3yy[64];&#xA;&#xA;        /*16x16*/&#xA;        int quad4[16][16];&#xA;        int grid4x2d[16][16],grid4y2d[16][16]; &#xA;        int grid4xx[256], grid4yy[256];&#xA;&#xA;        draw_borders(xi,xf,yi,yf);/*Draw borders for clarity*/&#xA;&#xA;        /* Plot Cities */&#xA;        for(i=0;i&amp;lt;num;i++){&#xA;&#xA;            /*represent each city as a random point*/&#xA;            x[i] = (int)Random(0,1024);&#xA;            y[i] = (int)Random(0,1024);&#xA;            PlotPoint(panelHandle,PANEL_2X2GRAPH,x[i],y[i],VAL_CROSS,VAL_BLUE); /*plot city on each graph*/&#xA;            PlotPoint(panelHandle,PANEL_4X4GRAPH,x[i],y[i],VAL_CROSS,VAL_BLUE); &#xA;            PlotPoint(panelHandle,PANEL_8X8GRAPH,x[i],y[i],VAL_CROSS,VAL_BLUE);&#xA;            PlotPoint(panelHandle,PANEL_16X16GRAPH,x[i],y[i],VAL_CROSS,VAL_BLUE);&#xA;            PlotPoint(panelHandle,PANEL_GRAPH_PATH,x[i],y[i],VAL_CROSS,VAL_BLUE);&#xA;&#xA;            /*Print out city coordinates*/&#xA;            char resultx[100];&#xA;            sprintf(resultx,&quot;x co-ordinate of city %d is %d\n&quot;,i+1,x[i]);&#xA;            SetCtrlVal(panelHandle,PANEL_CITYCOORDINATES,resultx);&#xA;&#xA;            char resulty[100];&#xA;            sprintf(resulty,&quot;y co-ordinate of city %d is %d\n&quot;,i+1,y[i]);&#xA;            SetCtrlVal(panelHandle,PANEL_CITYCOORDINATES,resulty); &#xA;        }&#xA;&#xA;        SetCtrlVal(panelHandle,PANEL_MESSAGE,&quot;Cities Plotted&quot;);&#xA;&#xA;        ////////////*2x2 Graph*////////////////////////////////////////////////////////////////////////////&#xA;        /* First find centre point of each box which contains a city&#xA;         * Second find midpoint of centre point locations of each box that contains at least one city&#xA;         */&#xA;&#xA;        /* Set count in all 4 boxes to be zero*/&#xA;        for (i=0;i&amp;lt;2;i++){ /*x direction*/                  &#xA;            for (j=0; j&amp;lt;2; j++) /*y direction*/&#xA;                quad1[i][j]=0;  &#xA;        }       &#xA;&#xA;        /* For every city, add 1 to count */ &#xA;        for (i=0;i&amp;lt;num;i++)&#xA;            quad1[x[i]/512][y[i]/512]++;&#xA;&#xA;        /* Set up a grid containing the x,y coordinates of central point locations of each box that contains at least one city*/ &#xA;        for (i=0;i&amp;lt;2;i++){&#xA;            for (j=0; j&amp;lt;2; j++){&#xA;                if (quad1[i][j]&amp;gt;0){&#xA;                    grid1x2d[i][j]=i*512+256;  grid1y2d[i][j] =j*512+256;&#xA;&#xA;                    /* Assign these values to a structure to call on them later when finding midpoint*/&#xA;                    Seg1[i][j].x = grid1x2d[i][j]; Seg1[i][j].y = grid1y2d[i][j];&#xA;                }      &#xA;                else{&#xA;                    /* Set up a grid containing dummy x,y coordinates if there are no cities in a particular box */ &#xA;                    grid1x2d[i][j]=-1;  grid1y2d[i][j] =-1;&#xA;&#xA;                    Seg1[i][j].x = grid1x2d[i][j]; Seg1[i][j].y = grid1y2d[i][j];&#xA;                }    &#xA;            }   &#xA;        }&#xA;&#xA;        /* Find edge points which are found if representative points are connected*/&#xA;        /* If the 2X2 Graph has an empty box then there will be an edgepoint in centre of graph*/&#xA;        for(i=0;i&amp;lt;2;i++){&#xA;            for(j=0;j&amp;lt;2;j++){ &#xA;                if (quad1[i][j]&amp;lt;=0){&#xA;                                      PlotPoint(panelHandle,PANEL_2X2GRAPH,512,512,VAL_BOLD_X,VAL_DK_GREEN);&#xA;                    PlotPoint(panelHandle,PANEL_4X4GRAPH,512,512,VAL_BOLD_X,VAL_MAGENTA);&#xA;                    PlotPoint(panelHandle,PANEL_8X8GRAPH,512,512,VAL_BOLD_X,VAL_MAGENTA);&#xA;                    PlotPoint(panelHandle,PANEL_16X16GRAPH,512,512,VAL_BOLD_X,VAL_MAGENTA);&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        for(i=0;i&amp;lt;2;i++){&#xA;            j=0;&#xA;            if( quad1[i][j]&amp;gt;0 &amp;amp;&amp;amp; quad1[i][j+1]&amp;gt;0){&#xA;                mid1x2d= (Seg1[i][j].x + Seg1[i][j+1].x) /2 ;&#xA;                mid1y2d= (Seg1[i][j].y + Seg1[i][j+1].y) /2 ;&#xA;                PlotPoint(panelHandle,PANEL_2X2GRAPH,mid1x2d,mid1y2d,VAL_BOLD_X,VAL_DK_GREEN);&#xA;                PlotPoint(panelHandle,PANEL_4X4GRAPH,mid1x2d,mid1y2d,VAL_BOLD_X,VAL_MAGENTA);&#xA;                PlotPoint(panelHandle,PANEL_8X8GRAPH,mid1x2d,mid1y2d,VAL_BOLD_X,VAL_MAGENTA);&#xA;                PlotPoint(panelHandle,PANEL_16X16GRAPH,mid1x2d,mid1y2d,VAL_BOLD_X,VAL_MAGENTA); &#xA;            }&#xA;        }&#xA;        for(j=0;j&amp;lt;2;j++){&#xA;            i=0;&#xA;            if( quad1[i][j]&amp;gt;0 &amp;amp;&amp;amp; quad1[i+1][j]&amp;gt;0){&#xA;                mid1x2d= (Seg1[i][j].x + Seg1[i+1][j].x) /2 ;&#xA;                mid1y2d= (Seg1[i][j].y + Seg1[i+1][j].y) /2 ;&#xA;                PlotPoint(panelHandle,PANEL_2X2GRAPH,mid1x2d,mid1y2d,VAL_BOLD_X,VAL_DK_GREEN);&#xA;                PlotPoint(panelHandle,PANEL_4X4GRAPH,mid1x2d,mid1y2d,VAL_BOLD_X,VAL_MAGENTA);&#xA;                PlotPoint(panelHandle,PANEL_8X8GRAPH,mid1x2d,mid1y2d,VAL_BOLD_X,VAL_MAGENTA);&#xA;                PlotPoint(panelHandle,PANEL_16X16GRAPH,mid1x2d,mid1y2d,VAL_BOLD_X,VAL_MAGENTA); &#xA;            }&#xA;        } &#xA;&#xA;        /*stretch into 1d array for plotting*/ &#xA;        for (k=0;k&amp;lt;4;k++){ &#xA;            grid1xx[k]=grid1x2d[k%2][(int)k/2];&#xA;        }   &#xA;        for (k=0;k&amp;lt;4;k++){ &#xA;            grid1yy[k]=grid1y2d[k%2][(int)k/2];&#xA;        }&#xA;&#xA;        PlotXY (panelHandle, PANEL_2X2GRAPH, grid1xx , grid1yy ,4, VAL_INTEGER, VAL_INTEGER, VAL_SCATTER,VAL_BOLD_X, VAL_THIN_LINE, 1, VAL_BLACK);&#xA;        ////////////////////////////////////////////////////////////////////////////////////////////////////&#xA;&#xA;        ////////////*Optimal Paths*////////////////////////////////////////////////////////////////////////////&#xA;&#xA;        //////////////////////////////////////////////////////////////////////////////////////////////////// &#xA;&#xA;        ////////////*4x4 Graph*////////////////////////////////////////////////////////////////////////////&#xA;        /* Set count in all 4 boxes to be zero*/&#xA;        for (i=0;i&amp;lt;4;i++){ /*x direction*/                  &#xA;            for (j=0; j&amp;lt;4; j++) /*y direction*/&#xA;                quad2[i][j]=0;  &#xA;        }       &#xA;&#xA;        /* For every city, add 1 to count */ &#xA;        for (i=0;i&amp;lt;num;i++)&#xA;            quad2[x[i]/256][y[i]/256]++;&#xA;&#xA;        /* Set up a grid containing the x,y coordinates of central point locations of each box that contains at least one city*/ &#xA;        for (i=0;i&amp;lt;4;i++){&#xA;            for (j=0; j&amp;lt;4; j++){&#xA;                if (quad2[i][j]&amp;gt;0){&#xA;                    grid2x2d[i][j]=i*256+128;  grid2y2d[i][j] =j*256+128;&#xA;&#xA;                    /* Assign these values to a structure to call on them later when finding midpoint*/&#xA;                    Seg2[i][j].x = grid2x2d[i][j]; Seg2[i][j].y = grid2y2d[i][j];&#xA;                }      &#xA;                else{&#xA;                    /* Set up a grid containing dummy x,y coordinates if there are no cities in a particular box */ &#xA;                    grid2x2d[i][j]=-1;  grid2y2d[i][j] =-1;&#xA;&#xA;                    Seg2[i][j].x = grid2x2d[i][j]; Seg2[i][j].y = grid2y2d[i][j]; &#xA;                }    &#xA;            }   &#xA;        }&#xA;&#xA;        /*stretch into 1d array for plotting*/ &#xA;        for (k=0;k&amp;lt;16;k++) &#xA;            grid2xx[k]=grid2x2d[k%4][(int)k/4];&#xA;&#xA;        for (k=0;k&amp;lt;16;k++) &#xA;            grid2yy[k]=grid2y2d[k%4][(int)k/4];&#xA;&#xA;&#xA;        PlotXY (panelHandle, PANEL_4X4GRAPH, grid2xx , grid2yy ,16, VAL_INTEGER, VAL_INTEGER, VAL_SCATTER, VAL_BOLD_X, VAL_THIN_LINE, 1, VAL_BLACK);&#xA;        ////////////////////////////////////////////////////////////////////////////////////////////////////&#xA;&#xA;        ////////////*8x8 Graph*////////////////////////////////////////////////////////////////////////////&#xA;        /* Set count in all 4 boxes to be zero*/&#xA;        for (i=0;i&amp;lt;8;i++){ /*x direction*/                  &#xA;            for (j=0; j&amp;lt;8; j++) /*y direction*/&#xA;                quad3[i][j]=0;  &#xA;        }       &#xA;&#xA;        /* For every city, add 1 to count */ &#xA;        for (i=0;i&amp;lt;num;i++)&#xA;            quad3[x[i]/128][y[i]/128]++;&#xA;&#xA;        /* Set up a grid containing the x,y coordinates of central point locations of each box that contains at least one city*/ &#xA;        for (i=0;i&amp;lt;8;i++){&#xA;            for (j=0; j&amp;lt;8; j++){&#xA;                if (quad3[i][j]&amp;gt;0){&#xA;                    grid3x2d[i][j]=i*128+64;  grid3y2d[i][j] =j*128+64;&#xA;&#xA;                    /* Assign these values to a structure to call on them later when finding midpoint*/&#xA;                    Seg3[i][j].x = grid3x2d[i][j]; Seg3[i][j].y = grid3y2d[i][j];&#xA;                }      &#xA;                else{&#xA;                    /* Set up a grid containing dummy x,y coordinates if there are no cities in a particular box */ &#xA;                    grid3x2d[i][j]=-1;  grid3y2d[i][j] =-1;&#xA;&#xA;                    Seg3[i][j].x = grid3x2d[i][j]; Seg3[i][j].y = grid3y2d[i][j]; &#xA;                }    &#xA;            }   &#xA;        }&#xA;&#xA;        /*stretch into 1d array for plotting*/ &#xA;        for (k=0;k&amp;lt;64;k++) &#xA;            grid3xx[k]=grid3x2d[k%8][(int)k/8];&#xA;&#xA;        for (k=0;k&amp;lt;64;k++) &#xA;            grid3yy[k]=grid3y2d[k%8][(int)k/8];&#xA;&#xA;&#xA;        PlotXY (panelHandle, PANEL_8X8GRAPH, grid3xx , grid3yy ,64, VAL_INTEGER, VAL_INTEGER, VAL_SCATTER, VAL_BOLD_X, VAL_THIN_LINE, 1, VAL_BLACK);&#xA;        ////////////////////////////////////////////////////////////////////////////////////////////////////&#xA;&#xA;        ////////////*16x16 Graph*////////////////////////////////////////////////////////////////////////////&#xA;        /* Set count in all 4 boxes to be zero*/&#xA;        for (i=0;i&amp;lt;16;i++){ /*x direction*/                 &#xA;            for (j=0; j&amp;lt;16; j++) /*y direction*/&#xA;                quad4[i][j]=0;  &#xA;        }       &#xA;&#xA;        /* For every city, add 1 to count */ &#xA;        for (i=0;i&amp;lt;num;i++)&#xA;            quad4[x[i]/64][y[i]/64]++;&#xA;&#xA;        /* Set up a grid containing the x,y coordinates of central point locations of each box that contains at least one city*/ &#xA;        for (i=0;i&amp;lt;16;i++){&#xA;            for (j=0; j&amp;lt;16; j++){&#xA;                if (quad4[i][j]&amp;gt;0){&#xA;                    grid4x2d[i][j]=i*64+32;  grid4y2d[i][j] =j*64+32;&#xA;&#xA;                    /* Assign these values to a structure to call on them later when finding midpoint*/&#xA;                    Seg4[i][j].x = grid4x2d[i][j]; Seg4[i][j].y = grid4y2d[i][j];&#xA;                }      &#xA;                else{&#xA;                    /* Set up a grid containing dummy x,y coordinates if there are no cities in a particular box */ &#xA;                    grid4x2d[i][j]=-1;  grid4y2d[i][j] =-1;&#xA;&#xA;                    Seg4[i][j].x = grid4x2d[i][j]; Seg4[i][j].y = grid4y2d[i][j]; &#xA;                }    &#xA;            }   &#xA;        }&#xA;&#xA;        /*stretch into 1d array for plotting*/ &#xA;        for (k=0;k&amp;lt;256;k++) &#xA;            grid4xx[k]=grid4x2d[k%16][(int)k/16];&#xA;&#xA;        for (k=0;k&amp;lt;256;k++) &#xA;            grid4yy[k]=grid4y2d[k%16][(int)k/16];&#xA;&#xA;&#xA;        PlotXY (panelHandle, PANEL_16X16GRAPH, grid4xx , grid4yy ,256, VAL_INTEGER, VAL_INTEGER, VAL_SCATTER,VAL_BOLD_X, VAL_THIN_LINE, 1, VAL_BLACK);&#xA;        ////////////////////////////////////////////////////////////////////////////////////////////////////&#xA;&#xA;&#xA;        break;&#xA;}       &#xA;return 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     void draw_borders(int xi,int xf,int yi,int yf){ /*function definition*/&#xA;&#xA;        /*Draw borders for clarity*/ &#xA;&#xA;        /*2x2*/&#xA;        PlotLine(panelHandle,PANEL_2X2GRAPH,xi,(yf+yi)/2,xf,(yf+yi)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_2X2GRAPH,(xf+xi)/2,yi,(xf+xi)/2,yf,VAL_RED);&#xA;&#xA;        /*4x4*/&#xA;        PlotLine(panelHandle,PANEL_4X4GRAPH,xi,(yf+yi)/2,xf,(yf+yi)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_4X4GRAPH,(xf+xi)/2,yi,(xf+xi)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_4X4GRAPH,(xi+(xf+xi)/2)/2,yi,(xi+(xf+xi)/2)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_4X4GRAPH,(xf+(xf+xi)/2)/2,yi,(xf+(xf+xi)/2)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_4X4GRAPH,xi,(yf+(yf+yi)/2)/2,xf,(yf+(yf+yi)/2)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_4X4GRAPH,xi,(yi+(yf+yi)/2)/2,xf,(yi+(yf+yi)/2)/2,VAL_RED);&#xA;&#xA;        /*8x8*/&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,xi,(yf+yi)/2,xf,(yf+yi)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,(xf+xi)/2,yi,(xf+xi)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,(xi+(xf+xi)/2)/2,yi,(xi+(xf+xi)/2)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,(xf+(xf+xi)/2)/2,yi,(xf+(xf+xi)/2)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,xi,(yf+(yf+yi)/2)/2,xf,(yf+(yf+yi)/2)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,xi,(yi+(yf+yi)/2)/2,xf,(yi+(yf+yi)/2)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,(xi+(xi+(xf+xi)/2)/2)/2,yi,(xi+(xi+(xf+xi)/2)/2)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,(xf+(xi+(xf+xi)/2)/2)/2,yi,(xf+(xi+(xf+xi)/2)/2)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,(xf+(xf+(xf+xi)/2)/2)/2,yi,(xf+(xf+(xf+xi)/2)/2)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,(xi+(xf+(xf+xi)/2)/2)/2,yi,(xi+(xf+(xf+xi)/2)/2)/2,yf,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,xi,(yi+(yi+(yf+yi)/2)/2)/2,xf,(yi+(yi+(yf+yi)/2)/2)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,xi,(yi+(yf+(yf+yi)/2)/2)/2,xf,(yi+(yf+(yf+yi)/2)/2)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,xi,(yf+(yf+(yf+yi)/2)/2)/2,xf,(yf+(yf+(yf+yi)/2)/2)/2,VAL_RED);&#xA;        PlotLine(panelHandle,PANEL_8X8GRAPH,xi,(yf+(yi+(yf+yi)/2)/2)/2,xf,(yf+(yi+(yf+yi)/2)/2)/2,VAL_RED); &#xA;&#xA;        /*16x16*/&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,(yf+yi)/2,xf,(yf+yi)/2,VAL_RED);//y=midline&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xf+xi)/2,yi,(xf+xi)/2,yf,VAL_RED);//x=midline&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xi+(xf+xi)/2)/2,yi,(xi+(xf+xi)/2)/2,yf,VAL_RED);//x=1/4 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xf+(xf+xi)/2)/2,yi,(xf+(xf+xi)/2)/2,yf,VAL_RED);//x=3/4 line &#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,(yf+(yf+yi)/2)/2,xf,(yf+(yf+yi)/2)/2,VAL_RED);//y=3/4 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,(yi+(yf+yi)/2)/2,xf,(yi+(yf+yi)/2)/2,VAL_RED);//y=1/4 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xi+(xi+(xf+xi)/2)/2)/2,yi,(xi+(xi+(xf+xi)/2)/2)/2,yf,VAL_RED);//x=1/8 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xf+(xi+(xf+xi)/2)/2)/2,yi,(xf+(xi+(xf+xi)/2)/2)/2,yf,VAL_RED);//x=5/8 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xf+(xf+(xf+xi)/2)/2)/2,yi,(xf+(xf+(xf+xi)/2)/2)/2,yf,VAL_RED);//x=7/8 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xi+(xf+(xf+xi)/2)/2)/2,yi,(xi+(xf+(xf+xi)/2)/2)/2,yf,VAL_RED);//x=3/8 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,(yi+(yi+(yf+yi)/2)/2)/2,xf,(yi+(yi+(yf+yi)/2)/2)/2,VAL_RED);//y=1/8 line &#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,(yi+(yf+(yf+yi)/2)/2)/2,xf,(yi+(yf+(yf+yi)/2)/2)/2,VAL_RED);//y=3/8 line &#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,(yf+(yf+(yf+yi)/2)/2)/2,xf,(yf+(yf+(yf+yi)/2)/2)/2,VAL_RED);//y=7/8 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,(yf+(yi+(yf+yi)/2)/2)/2,xf,(yf+(yi+(yf+yi)/2)/2)/2,VAL_RED);//y=3/8 line  &#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi+((xi+(xi+(xf+xi)/2)/2)/2)/2,yi,xi+((xi+(xi+(xf+xi)/2)/2)/2)/2,yf,VAL_RED);//x=1/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi+((xi+(xf+(xf+xi)/2)/2)/2)/2,yi,xi+((xi+(xf+(xf+xi)/2)/2)/2)/2,yf,VAL_RED);//x=3/16 line &#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi+((xf+(xi+(xf+xi)/2)/2)/2)/2,yi,xi+((xf+(xi+(xf+xi)/2)/2)/2)/2,yf,VAL_RED);//x=5/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi+((xf+(xf+(xf+xi)/2)/2)/2)/2,yi,xi+((xf+(xf+(xf+xi)/2)/2)/2)/2,yf,VAL_RED);//x=7/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xi+xf)/2+((xi+(xi+(xf+xi)/2)/2)/2)/2,yi,(xi+xf)/2+((xi+(xi+(xf+xi)/2)/2)/2)/2,yf,VAL_RED);//x=9/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xi+xf)/2+((xi+(xf+(xf+xi)/2)/2)/2)/2,yi,(xi+xf)/2+((xi+(xf+(xf+xi)/2)/2)/2)/2,yf,VAL_RED);//x=11/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xi+xf)/2+((xf+(xi+(xf+xi)/2)/2)/2)/2,yi,(xi+xf)/2+((xf+(xi+(xf+xi)/2)/2)/2)/2,yf,VAL_RED);//x=13/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,(xi+xf)/2+((xf+(xf+(xf+xi)/2)/2)/2)/2,yi,(xi+xf)/2+((xf+(xf+(xf+xi)/2)/2)/2)/2,yf,VAL_RED);//x=15/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,yi+((yi+(yi+(yf+yi)/2)/2)/2)/2,xf,yi+((yi+(yi+(yf+yi)/2)/2)/2)/2,VAL_RED);//y=1/16 line &#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,yi+((yi+(yf+(yf+yi)/2)/2)/2)/2,xf,yi+((yi+(yf+(yf+yi)/2)/2)/2)/2,VAL_RED);//y=3/16 line &#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,yi+((yf+(yf+(yf+yi)/2)/2)/2)/2,xf,yi+((yf+(yf+(yf+yi)/2)/2)/2)/2,VAL_RED);//y=7/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,yi+((yf+(yi+(yf+yi)/2)/2)/2)/2,xf,yi+((yf+(yi+(yf+yi)/2)/2)/2)/2,VAL_RED);//y=5/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,((yi+yf)/2)+((yf+(yi+(yf+yi)/2)/2)/2)/2,xf,((yi+yf)/2)+((yf+(yi+(yf+yi)/2)/2)/2)/2,VAL_RED);//y=13/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,((yi+yf)/2)+((yf+(yf+(yf+yi)/2)/2)/2)/2,xf,((yi+yf)/2)+((yf+(yf+(yf+yi)/2)/2)/2)/2,VAL_RED);//y=15/16 line&#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,((yi+yf)/2)+((yi+(yf+(yf+yi)/2)/2)/2)/2,xf,((yi+yf)/2)+((yi+(yf+(yf+yi)/2)/2)/2)/2,VAL_RED);//y=11/16 line &#xA;        PlotLine(panelHandle,PANEL_16X16GRAPH,xi,((yi+yf)/2)+((yi+(yi+(yf+yi)/2)/2)/2)/2,xf,((yi+yf)/2)+((yi+(yi+(yf+yi)/2)/2)/2)/2,VAL_RED);//y=9/16 line&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I need to code up the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Save the optimal paths for any 2x2 box as a look up table&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Draw the optimal path for each graph, and save where it hits the edge of its box. This edge point must then be used in the next step (when I make the graph 8x8, then 16x16, etc. until I have found minimum path to a good approximation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And how can I pass through the graph in an ordered way to do this, and how can I save the optimal paths?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zn53O.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3954" LastEditorUserId="3954" LastEditDate="2013-12-17T10:05:36.293" LastActivityDate="2013-12-17T10:05:36.293" Title="The traveling salesman problem - Using Space Renormalization" Tags="|c|graph-theory|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10267" PostTypeId="2" ParentId="10209" CreationDate="2013-12-09T20:19:40.403" Score="3" Body="&lt;p&gt;Thank you to the posters for encouraging me to look for a bug. I found one, a subtle issue related to restriction and interpolation. I am using ghost points to treat the boundaries, and so the first interior point is at index (0,0) (boundary points are stored on negative indices). This can work, but you have to remember that the interior coarse grid points must be centered on the fine gridpoints (1,1), (3,1), (1,3), (3,3), etc. My mistake was centering them on (0,0), (2,0), (0,2), (2,2) etc, which are points immediately adjacent to the boundary. This slight shift causes problems at the boundaries.&lt;/p&gt;&#xA;" OwnerUserId="6702" LastActivityDate="2013-12-09T20:19:40.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10268" PostTypeId="2" ParentId="4882" CreationDate="2013-12-10T02:03:30.043" Score="0" Body="&lt;p&gt;In this scenario I would suggest to use matlab livelink. I'm not aware of how to include these global parameters in the comsol figures without loading it into MATLAB. If you do opt to do it in MATLAB, you could use the following code.:&#xA;&lt;br&gt;&lt;code&gt;plot(x,y)&lt;/code&gt;&#xA;&lt;br&gt;&lt;code&gt;title(['Your Title',num2str(plot_z)])&lt;/code&gt;&#xA;&lt;br&gt;You may also need to change your code to address the different parameters, maybe using a for loop.&lt;/p&gt;&#xA;" OwnerUserId="6758" LastActivityDate="2013-12-10T02:03:30.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10270" PostTypeId="1" CreationDate="2013-12-10T09:47:20.330" Score="1" ViewCount="298" Body="&lt;p&gt;I wrote a simple ray tracer, and now I try to implement an uniform grid. There is a lot of documentation on how to traverse the grid, but I don't know how to construct the grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have my uniform grid and my triangles. How could I know, which cell(s) in the grid correspond(s) to my triangles ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks fo your answers. &lt;/p&gt;&#xA;" OwnerUserId="6788" LastActivityDate="2013-12-11T16:45:23.970" Title="Ray tracer : create an uniform grid" Tags="|algorithms|grid|" AnswerCount="1" CommentCount="2" ClosedDate="2013-12-25T19:18:38.393" ContentLicense="CC BY-SA 3.0" />
  <row Id="10271" PostTypeId="2" ParentId="10265" CreationDate="2013-12-10T10:33:04.727" Score="1" Body="&lt;p&gt;You need to trace where this term comes from in the derivation of these error estimates. In your case, you get this from an interpolation estimate of the form $\|z-I_h z\|_{L^2(e)} \le h^{?} \|\nabla^2 z\|_{L^2(T)}$, if I recall correctly. You'll need to look up what exactly the estimate is, what it involves, and where the transformation from reference to real cell/edge enters here. You will then find that $h$ is related to the norm of the mapping from reference to real cell. What it is here, I don't know, but it can be found out.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-10T10:33:04.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10272" PostTypeId="1" AcceptedAnswerId="10273" CreationDate="2013-12-10T10:49:30.547" Score="2" ViewCount="803" Body="&lt;p&gt;I have an array of floating point values $F$. I want to input my array into an algorithm that only takes integer values. How can I efficiently determine the &lt;em&gt;smallest&lt;/em&gt; multiplier $m$ such that all values in $m \centerdot F$ are integers within a given tolerance $tol$?&lt;/p&gt;&#xA;" OwnerUserId="3301" LastEditorUserId="3301" LastEditDate="2013-12-10T13:53:25.267" LastActivityDate="2013-12-10T17:27:55.797" Title="Re-scaling array of floats so that all items are approximately integer" Tags="|floating-point|scaling|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10273" PostTypeId="2" ParentId="10272" CreationDate="2013-12-10T11:36:04.953" Score="3" Body="&lt;p&gt;Sure -- just take $m=\frac{1}{tol}\frac{1}{\min_i |F_i|} 10^{16}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easy to see that if you take $m$ just large enough, you will always achieve this. For simplicity, assume that your numbers are all larger than one and have at most 3 digits after the decimal point, then if you multiply them by a thousand, you will get only integers. Of course, multiplying by anything larger than a thousand will also work. If you don't need to exactly hit integers, you can get away with a smaller factor, but in general you'll need something that's probably proportional to one over the tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, as a general rule, multiplying by some large number like I suggest above is of course not a particularly useful thing. Integers are just as arbitrary as any other numbers if they have physical units -- there is nothing special about one, two, three meters. My question then would be what you want these integers to represent?&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-10T11:36:04.953" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10274" PostTypeId="2" ParentId="10187" CreationDate="2013-12-10T13:10:38.513" Score="8" Body="&lt;p&gt;Answering your last question first, &lt;em&gt;do people actually use FDM for curved boundary nowadays&lt;/em&gt; I'd say the answer is no. In the commercial CFD world, 2nd order accurate finite volume schemes are the de-facto industry standard. One of the advantages of FV (and finite element/discontinuous galerkin approaches Jed mentioned) over FD is the much more natural handling of complex boundaries. FD does provide the foundation of a lot of numerical methods (FV included) and it's necessary to learn as a first step, but it's not advisable for large-scale complex problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for dealing with complex boundaries in FD, I can think of two canonical ways, one of which is the interpolation/extrapolation method you mentioned. The other is to use body-fitted grid points in physical $(x,y)$ space with a conformal mapping to &quot;computational&quot; $\xi = \xi(x,y),\eta = \eta(x,y)$ space where $\Delta \xi = \Delta \eta = constant$. Then one can re-write terms like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial x}=\frac{\partial u}{\partial \xi} \frac{\partial \xi}{\partial x} + \frac{\partial u}{\partial \eta} \frac{\partial \eta}{\partial x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $\frac{\partial (\xi,\eta)}{\partial (x,y)}$ terms are called &lt;em&gt;metric terms&lt;/em&gt; and can be computed at the beginning of a problem (or for a simple domain you might have an exact conformal mapping available), and the $u$ derivatives can be computed on a logically simple computational domain. This process makes the implementation of boundary conditions straightforward, but it requires generating a sufficiently smooth, nominally orthogonal curvilinear mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd say this body-fitted grid approach is the &quot;most popular treatment&quot; for dealing with curved boundaries in FD, with the caveat that FD methods themselves are not very &quot;popular&quot; anymore for complex applications. It's rare to see them still come up in CFD literature except for on very simple domains.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2013-12-10T13:17:33.163" LastActivityDate="2013-12-10T13:17:33.163" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10275" PostTypeId="2" ParentId="10270" CreationDate="2013-12-10T14:13:11.737" Score="0" Body="&lt;p&gt;You don't have to form and store a uniform mesh explicitly since you can do arithmetic to compute cell indicies. If you know you have $N_x$ cells in the $x$ direction on a grid over $\left[0,L_x\right]$, then you know the $x$-index of the cell is&#xA;$$&#xA;i=\left\lfloor \frac{x N_x}{L_x}\right\rfloor&#xA;$$&#xA;You can do the same for each of the three coordinates for each of the three corners of your triangle. The indices you generate uniquely define the cell than contains each corner. A cell contains an entire triangle if it contains all three corners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This index is 0-based. Add 1 if your arrays are 1-based (e.g. Fortran or MATLAB).&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2013-12-10T14:13:11.737" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10277" PostTypeId="2" ParentId="8333" CreationDate="2013-12-10T14:50:50.543" Score="1" Body="&lt;p&gt;The keyword here is &lt;em&gt;stochastic approximation&lt;/em&gt; which refers both to root finding and optimization.  As usual, knowing the keyword makes it easy to find lots of resources.  Here's &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_approximation&quot; rel=&quot;nofollow&quot;&gt;the Wikipedia page&lt;/a&gt; for a start.&lt;/p&gt;&#xA;" OwnerUserId="605" LastActivityDate="2013-12-10T14:50:50.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10278" PostTypeId="1" CreationDate="2013-12-10T17:06:23.730" Score="4" ViewCount="1095" Body="&lt;p&gt;If I have a psd, symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt; and I need to do LU decomps on &lt;span class=&quot;math-container&quot;&gt;$\mathbf{B_i}= \mathbf{A} + \mathbf{D_i}$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{D_i}$&lt;/span&gt; is a diagonal psd matrix, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{D_i}$&lt;/span&gt; changes randomly). Is there an easy way to go from the LU of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt; + some function of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{D_i}$&lt;/span&gt; to get the LU of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{B_i}$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references in the literature would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="6794" LastEditorUserId="23791" LastEditDate="2018-12-19T11:27:24.120" LastActivityDate="2018-12-19T11:27:24.120" Title="LU Decomposition of PSD Matrix + Diagonal Matrix" Tags="|linear-algebra|dense-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10279" PostTypeId="2" ParentId="10272" CreationDate="2013-12-10T17:27:55.797" Score="3" Body="&lt;p&gt;If we treat the input array as $d$ &lt;em&gt;real numbers&lt;/em&gt; rather than floating-point values, then this is a problem of best simultaneous diophantine approximation.  The differences between this and the one-dimensional version, well-known to be solved by continued fraction approximations (see e.g. Knuth AOCP vol. 2, Seminumerical Algorithms) were explored by J.C. Lagarias (&lt;a href=&quot;http://www.ams.org/journals/tran/1982-272-02/S0002-9947-1982-0662052-7/&quot; rel=&quot;nofollow&quot;&gt;part I (TAMS)&lt;/a&gt;, &lt;a href=&quot;http://msp.org/pjm/1982/102-1/p08.xhtml&quot; rel=&quot;nofollow&quot;&gt;part II(Pac.J.Math)&lt;/a&gt;) in 1982 papers, and the complexity of solution in &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0214016&quot; rel=&quot;nofollow&quot;&gt;a 1985 paper (SIAM J.Comp.)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;His result is that while the &lt;em&gt;best&lt;/em&gt; simultaneous sup-norm approximation problem is NP-complete, the LLL integral lattice algorithm can be used to get in polynomial-time approximations that are within a factor $\sqrt{5d} 2^{(d + 1)/2}$ of the optimal error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doug Hensley has &lt;a href=&quot;http://www.math.tamu.edu/~doug.hensley/SimultaneousDiophantine.pdf&quot; rel=&quot;nofollow&quot;&gt;a more recent (2005) paper&lt;/a&gt; detailing the tradeoffs when one advances from one-dimensional to the more difficult $d \ge 2$ problems.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-12-10T17:27:55.797" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10280" PostTypeId="1" AcceptedAnswerId="10281" CreationDate="2013-12-10T19:24:20.863" Score="14" ViewCount="612" Body="&lt;p&gt;We are working on a Bayesian model for a space-time process, and are using a No-U-Turn sampler (NUTS) that requires a model for the log-probability and it's gradient with respect to the model parameters.  More succinctly, we have fairly complicated log-probability function $f : \mathbb{R}^n \rightarrow \mathbb{R}$, involving statistical distributions, kronecker products, exponentials, ratios, if-else statements etc, and need to provide it and it's gradient to NUTS.  Several packages (&lt;a href=&quot;http://mc-stan.org/&quot;&gt;Stan&lt;/a&gt; and  &lt;a href=&quot;https://github.com/JuliaStats/MCMC.jl&quot;&gt;Julia's MCMC&lt;/a&gt;) use operator-overloading (to the best of my knowledge) to obtain the gradient automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we were able to create our own gradient function, perhaps using a source-code-transformation auto-diff tool, would we get better performance, or is OO just as good or better?&lt;/p&gt;&#xA;" OwnerUserId="486" LastActivityDate="2013-12-15T17:12:41.227" Title="Regarding automatic differentiation, is source-code-transformation (STC) more efficient than operator-overloading (OO)?" Tags="|monte-carlo|automatic-differentiation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10281" PostTypeId="2" ParentId="10280" CreationDate="2013-12-10T23:23:01.320" Score="10" Body="&lt;p&gt;Source-to-source transformation is considered the gold standard in terms of performance. OO approaches seem to be almost as good, in that there are more OO packages out there, and performance is not mentioned as a significant drawback. If you find an OO library you like for the language you're working in, I'd use it first, and then figure out later if you absolutely need source-to-source transformation, and if such a tool meeting your needs exists. The typical cost of an automatic-differentiation-generated derivative is roughly three to five times that of a function evaluation, to put things in context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more OO packages out there because it's easier to implement automatic differentiation tools using operator overloading than it is to use source-to-source translation. Implementing a source-to-source translator is tantamount to writing a compiler: source code must be parsed and tokenized, then transformation rules must be applied to the resulting expression tree. Andreas Griewank's book, Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Second Edition, goes into more detail regarding the tradeoffs.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-10T23:23:01.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10282" PostTypeId="2" ParentId="10278" CreationDate="2013-12-10T23:33:47.447" Score="11" Body="&lt;p&gt;Use a Cholesky decomposition or an LDL decomposition instead of LU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judging from answers to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/5101/diagonal-update-of-a-symmetric-positive-definite-matrix?rq=1&quot;&gt;Diagonal update of a symmetric positive definite matrix&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2415/solving-a-system-with-a-small-rank-diagonal-update&quot;&gt;Solving a system with a small rank diagonal update&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/503/can-diagonal-plus-fixed-symmetric-linear-systems-be-solved-in-quadratic-time-aft/505#505&quot;&gt;Can diagonal plus fixed symmetric linear systems be solved in quadratic time after precomputation?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;there's no good way to update any of those decompositions. You'd do no worse to recompute the decomposition from scratch each time unless the rank of the diagonal matrix is really low, in which case, maybe you could do something similar to Sherman-Morrison(-Woodbury)-type updates.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2013-12-10T23:33:47.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10283" PostTypeId="1" CreationDate="2013-12-11T03:02:12.103" Score="5" ViewCount="4231" Body="&lt;p&gt;My long-term goal is to numerically solve the 1D advection-diffusion equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t}=\frac{\partial }{\partial x}\left( v(x,t) u+D\frac{\partial u}{\partial x}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start I've set set $v(x)$ instead of $v(x,t)$ and I've been using an implicit finite difference method with various upwind approximations (I've tried first order and also second order linear). While the behaviour of the various solutions appears reasonable (and agrees with Matlab's pdepe solver) I've been unable to get conservation of mass, and in all cases the total mass eventually just reduces to 0 (this doesn't happen with Matlab's pdepe).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm confident it's not leaking out of my boundaries (I've tried both periodic BC's and Robin BC's), and I'm beginning to think this is a more fundamental problem with my approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is it reasonable to use an upwind-based FDM to solve such a conservation equation, or am I always going to run into conservation-of-mass problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6799" LastActivityDate="2014-03-09T17:09:26.557" Title="Conservation of Mass in 1D Advection-Diffusion Equation" Tags="|finite-difference|advection-diffusion|conservation|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10284" PostTypeId="2" ParentId="10283" CreationDate="2013-12-11T05:05:43.223" Score="2" Body="&lt;p&gt;A lot of this will depend on the magnitude of $D$ and mesh size $h$. For pure convection, upwinding works well; however, upwinding ends up too diffusive for $D \approx h$. If $h \ll D$, then upwinding should behave less diffusively, though only will $D=0$ will the upwind scheme show minimal numerical diffusivity. As $D$ increases relative to mesh size, an effective finite difference scheme should transition from upwinding to a central difference scheme (which is optimal for the case $v=0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0045793081900268&quot; rel=&quot;nofollow&quot;&gt;Gresho&lt;/a&gt; talks a bit about how upwinded methods are generally over-diffusive, and this same idea is what is used to design stabilized schemes for finite elements (i.e. SUPG, etc). In both cases, respecting the balance between $h$, $D$, and $v$ is crucial to the design of the scheme.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-12-11T05:05:43.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10285" PostTypeId="2" ParentId="10283" CreationDate="2013-12-11T05:13:55.973" Score="3" Body="&lt;p&gt;Use a first order upwind (for the convection component) and a second order central difference (for the diffusion component). So the end result would be equivalent to discretising the equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} = \frac{\partial \boldsymbol{v}}{\partial x} + D\frac{\partial^2 u}{\partial x^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So using the $\theta$-method you will end up with,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    \frac{u_{j}^{n+1} - u_{j}^{n}}{\Delta t} = &#xA;    \boldsymbol{v} \left[ \frac{1-\theta}{2\Delta x} \left( u_{j+1}^{n} - u_{j-1}^{n} \right) + \frac{\theta}{2\Delta x} \left( u_{j+1}^{n+1} - u_{j-1}^{n+1} \right) \right] + \\ D \left[ \frac{1 - \theta}{(\Delta x)^2} \left( u_{j-1}^{n} - 2u_{j}^{n} + u_{j+1}^{n} \right) +  \frac{\theta}{(\Delta x)^2} \left( u_{j-1}^{n+1} - 2u_{j}^{n+1} + u_{j+1}^{n+1} \right) \right] &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u^n$ and $u^{n+1}$ terms are the present and future time step, respectively. So a fully implicit scheme would be recovered by setting $\theta=1$ and the Crank-Nicolson by setting $\theta=1/2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this equation you need to write as a linear system so group in terms of the solution variable $u$ and move the unknowns and known to different sides and you should be able to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the end result would have the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\underbrace{\boldsymbol{A}}_{A}\cdot\underbrace{\boldsymbol{u^{n+1}}}_{x} = \underbrace{\boldsymbol{M}\cdot\boldsymbol{u^{n}}}_{d}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    \boldsymbol{A}=\begin{pmatrix}&#xA;      1+2s\theta         &amp;amp; -\theta(s + r)  	&amp;amp;     				&amp;amp;       	0		\\&#xA;      \theta(r-s)  		&amp;amp; 1+2s\theta 		&amp;amp; -\theta (s + r) 	&amp;amp;     	    		\\&#xA;              			&amp;amp;  \ddots   		&amp;amp;   \ddots  		&amp;amp; \ddots			\\&#xA;						&amp;amp; \theta(r-s)  		&amp;amp; 1+2s\theta			&amp;amp; -\theta (s + r) 	\\&#xA;                 0      &amp;amp;      				&amp;amp; \theta(r-s)  		&amp;amp; 1+2s\theta  		\\&#xA;   \end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    \boldsymbol{M}=\begin{pmatrix}&#xA;      1-2s(1-\theta)        &amp;amp; (1 - \theta)(s + r)  		&amp;amp;     						&amp;amp;       	0		 	\\&#xA;      (1 - \theta)(s - r)   &amp;amp; 1-2s(1-\theta)  			&amp;amp; (1 - \theta)(s + r) 		&amp;amp;		     	    	\\&#xA;              			   &amp;amp;  \ddots   					&amp;amp;   \ddots  				&amp;amp; \ddots				\\&#xA;						   &amp;amp; (1 - \theta)(s - r)  		&amp;amp; 1-2s(1-\theta)				&amp;amp; (1 - \theta)(s + r)	\\&#xA;                 0         &amp;amp;      						&amp;amp; (1 - \theta)(s - r) 		&amp;amp; 1-2s(1-\theta)			\\&#xA;   \end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally $\boldsymbol{u^n}$ and $\boldsymbol{u^{n+1}}$ are just column vectors of the solution variable discretised over the space. And very finally, $s=D\frac{\Delta t}{(\Delta x)^2}$ and $r=\boldsymbol{v}\frac{\Delta t}{2 \Delta x}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A side note.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 1D conservation problems you could also consider the finite volume method, it is just as easy to implement. It has some advantages in the fact that the conservative property is preserved to the discretisation level. You have to be a little be careful because discretisation will be come unstable for advection dominated problems so you need to introduce adaptive-upwinding or exponential fitting. &lt;a href=&quot;http://danieljfarrell.github.io/FVM/&quot; rel=&quot;nofollow&quot;&gt;For a step-by-step guide see my notes.&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2013-12-11T05:13:55.973" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10289" PostTypeId="1" CreationDate="2013-12-11T20:35:03.360" Score="2" ViewCount="197" Body="&lt;p&gt;In order to balance atom count in chemical equation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; a O2 + b C12H24O11 → c CO2 + d H2O&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I make linear equation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;O: 2*a + 11*b = 2*c + d&#xA;C: 12*b = c&#xA;H: 24*b = 2*d&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I must find smallest positive non-zero integer that satisfy that question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In GNU Octave I bit this problem with human interaction:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[2,11,-2,-1; 0,12,-1,0; 0,24,0,-2; 0,1,0,0]\[0;0;0;1]&#xA;ans =&#xA;12.5000&#xA; 1.0000&#xA;12.0000&#xA;12.0000&#xA;&#xA;[2,11,-2,-1; 0,12,-1,0; 0,24,0,-2; 0,1,0,0]\[0;0;0;2]&#xA;ans =&#xA;25&#xA; 2&#xA;24&#xA;24&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;by searching across &lt;strong&gt;b&lt;/strong&gt; coefficient until find first integer solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any math package that find solution automatically?&lt;/p&gt;&#xA;" OwnerUserId="6808" LastActivityDate="2013-12-12T02:47:03.777" Title="Solve chemical formula (number of molecules in reaction)" Tags="|linear-algebra|octave|mixed-integer-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10290" PostTypeId="2" ParentId="10289" CreationDate="2013-12-11T21:44:43.947" Score="1" Body="&lt;p&gt;Generally, the algorithm that would be used would be solving a system of equations of the form you show in your question, and then multiplying by the smallest integer that yields integral coefficients for each species. This algorithm would not be carried out via mixed-integer programming, since mixed-integer programming would use inefficient algorithms to solve this sort of problem.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-11T21:44:43.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10291" PostTypeId="1" CreationDate="2013-12-11T21:53:05.020" Score="3" ViewCount="535" Body="&lt;p&gt;I aim to find $x \in \mathbb{R}^n$ that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_x |D \cdot F \cdot x|^2$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to $x_i =  X_i$ and $x_j \geq X_j$ , &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$i \in I, j \in J$ and I and J partition ${1\cdots N}$ into two sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it is easy to do $D \cdot F \cdot x$ because D is diagonal and F is the discrete Fourier transform. But actually storing and computing on F is not practical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB's lsqlin works for small cases. It looks like it uses a direct method, qpsub, if the problem has a mix of equality and inequality constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any solvers or quick implementations you could refer me to?&lt;/p&gt;&#xA;" OwnerUserId="1217" LastActivityDate="2013-12-12T02:15:25.653" Title="indirect method for least-squares with inequality constraints" Tags="|optimization|linear-solver|sparse-matrix|least-squares|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10292" PostTypeId="2" ParentId="10289" CreationDate="2013-12-11T22:21:35.933" Score="1" Body="&lt;p&gt;As far as I know, the only additional pieces you need are functions to convert to a rational approximation and find the least common multiple. Octave (and Matlab) have &lt;a href=&quot;http://www.gnu.org/software/octave/doc/interpreter/Rational-Approximations.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;rat&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;http://www.gnu.org/software/octave/doc/interpreter/Utility-Functions.html#index-lcm-1903&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;lcm&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x1 = A\[0;0;0;1];&#xA;[n,d] = rat(x1);&#xA;x = x1*lcm(d(1),lcm(d(2),lcm(d(3),d(4))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Matlab's &lt;code&gt;lcm&lt;/code&gt; at least can't handle sets, so you'll need to customize this to other dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible that error will be introduced in solving some systems – especially for larger ones. In which case the rational approximation will return a wacky value. You can set a tolerance as a second argument to &lt;code&gt;rat&lt;/code&gt;. Another way around this is to solve the system &lt;a href=&quot;http://octave.sourceforge.net/symbolic/&quot; rel=&quot;nofollow&quot;&gt;symbolically&lt;/a&gt; and use &lt;a href=&quot;http://octave.sourceforge.net/symbolic/function/numden.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;numden&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x1 = sym(A)\sym([0;0;0;1]);&#xA;[n,d] = numden(x1);&#xA;d = double(d);&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I should also mention that it's not necessary to create an augmented &lt;code&gt;A&lt;/code&gt; matrix by adding a fourth row when using either of these methods.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="4474" LastEditDate="2013-12-12T02:47:03.777" LastActivityDate="2013-12-12T02:47:03.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10294" PostTypeId="2" ParentId="10291" CreationDate="2013-12-12T02:15:25.653" Score="1" Body="&lt;p&gt;Iterative methods for bound-constrained least squares problems exist; one example I found pretty quickly was &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.36.9511&quot; rel=&quot;nofollow&quot;&gt;On Iterative Algorithms for Linear Least Squares Problems With Bound Constraints&lt;/a&gt; by Bierlaire, Toint, and Tuyttens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should still be able to use MATLAB's &lt;code&gt;lsqlin&lt;/code&gt; by supplying some options to the function. To use an iterative algorithm, prior to calling &lt;code&gt;lsqlin&lt;/code&gt;, set the options struct using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options = optimoptions(@lsqlin, 'Algorithm', 'trust-region-reflective', ...&#xA;                       'JacobMult', YourMatrixFreeObjectiveFunction)&#xA;% Your lsqlin call should look something like...&#xA;x = lsqlin(C,d,A,b,Aeq,beq,lb,ub,x0,options)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Consult the &lt;code&gt;lsqlin&lt;/code&gt; &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/lsqlin.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll need to alter your problem slightly in order to use the trust region algorithm; the trust region reflective algorithm does not allow any equalities, so you'll have to simply set $x_{i} = X_{i}$ for $i \in I$ within your objective function, and only solve for $x_{j}$ such that $j \in J$.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-12T02:15:25.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10295" PostTypeId="1" AcceptedAnswerId="10296" CreationDate="2013-12-12T08:56:07.240" Score="2" ViewCount="95" Body="&lt;p&gt;Firstly I would like to achieve a final goal as can be found in the links below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.youtube.com/watch?v=ixkQumdlQ-0&quot; rel=&quot;nofollow&quot;&gt;Link 1&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.youtube.com/watch?v=3toTkQ3DPVs&quot; rel=&quot;nofollow&quot;&gt;Link 2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I started doing a little bit of research and found that Stereophotogrammetry is the field I should be looking to study should I want to achieve the desired result in the links above. Now I know there is software like Catch 123d that can help me do this but I would like to write an application hat takes images ad constructs the 3d model. Now is Stereophotogrammetry the right place to start looking to study and if so can anyone recommend where to begin to start learning this as I have googled but nothing constructive pops up  &lt;/p&gt;&#xA;" OwnerUserId="6816" LastActivityDate="2013-12-12T13:22:33.997" Title="Best place to start learning Stereophotogrammetry" Tags="|model|3d|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10296" PostTypeId="2" ParentId="10295" CreationDate="2013-12-12T13:22:33.997" Score="2" Body="&lt;p&gt;That is known as 3D reconstruction.  I don't know if there is anything better in the bookstores these days but &lt;a href=&quot;http://www.robots.ox.ac.uk/~vgg/hzbook/&quot; rel=&quot;nofollow&quot;&gt;Multiple-view geometry by Hartley and Zisserman&lt;/a&gt; is a good textbook on the subject.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2013-12-12T13:22:33.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10297" PostTypeId="1" CreationDate="2013-12-12T16:38:14.140" Score="5" ViewCount="258" Body="&lt;p&gt;I am looking for a clean way to compute preciseley, when $x$ is very close to zero: $$\exp(-1/x^2)$$ using C. What is the best way (speed, precision, etc.) ?&lt;/p&gt;&#xA;" OwnerUserId="1990" LastActivityDate="2014-02-12T02:58:12.820" Title="A clean way to compute exp(-1/x^2) near x=0 in C?" Tags="|c|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10300" PostTypeId="1" AcceptedAnswerId="10312" CreationDate="2013-12-12T20:49:10.493" Score="3" ViewCount="153" Body="&lt;p&gt;Let $x\in \mathbb{R}^{n}$, $Y\in \mathbb{R}^{mxn}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can then define:   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$row_{i}(Y)=$ $i^{th}$ row of $Y$&lt;br&gt;&#xA;$column_{i}(Y)=$ $i^{th}$ column of $Y$&lt;br&gt;&#xA;$x_{i}=i^{th}$ element of $x$&lt;br&gt;&#xA;$sum(x)=$ sum of the element of $x$&lt;br&gt;&#xA;$card(x)=$ number of non-zero element of $x$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Goal is to devise a method to determine if it is possible to construct a matrix $k\in \mathbb{R}_{\geq 0}^{mxn}$  given some $a\in \mathbb{R}^{n}$, $b, c\in\mathbb{R}^{m}$ for some $m, n \in \mathbb{N}_{&amp;gt;0}$ where $sum(a)=sum(b)$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t. for all $ i, j \in \mathbb{N}_{\geq0}$ with $0&amp;lt;i&amp;lt;m$ and $0&amp;lt;j&amp;lt;n$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bullet$  $sum(row_{i}(K))=b_{i}$&lt;br&gt;&#xA;$\bullet$  $sum(column_{j}(K))=a_{j}$&lt;br&gt;&#xA;$\bullet$  $card(row_{i}(K))=c_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="6822" LastEditorUserId="6822" LastEditDate="2013-12-13T22:37:54.213" LastActivityDate="2013-12-14T09:20:26.583" Title="Constrained System / Combinatorial Problem" Tags="|matlab|linear-solver|mixed-integer-programming|r|constrained-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10301" PostTypeId="2" ParentId="4927" CreationDate="2013-12-12T21:25:41.553" Score="4" Body="&lt;p&gt;If all you need is a C++ library to solve nonlinear optimization problems, you can use &lt;a href=&quot;http://roboptim.net/&quot; rel=&quot;nofollow&quot;&gt;RobOptim&lt;/a&gt;. Even though RobOptim was initially developed with robotics optimization problems in mind, it is suitable for any nonlinear optimization problems. It provides a simple &lt;a href=&quot;http://roboptim.net/roboptim-core/doxygen/v2.0/&quot; rel=&quot;nofollow&quot;&gt;C++ interface&lt;/a&gt; with plugins for multiple nonlinear solvers (&lt;a href=&quot;http://www.coin-or.org/Ipopt/&quot; rel=&quot;nofollow&quot;&gt;Ipopt&lt;/a&gt;, &lt;a href=&quot;http://www.nag.com/numeric/numerical_libraries.asp&quot; rel=&quot;nofollow&quot;&gt;NAG&lt;/a&gt;, etc.). Using that kind of wrappers makes it easy to use another NLP solver. If you cannot provide gradients, finite-difference computation can be done automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's open source so you can check out the source code on GitHub: &lt;a href=&quot;https://github.com/roboptim/&quot; rel=&quot;nofollow&quot;&gt;https://github.com/roboptim/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analysis done by @Geoff Oxberry is essential for the choice of nonlinear solver that will be called by RobOptim. Note that when dealing with that kind of solvers, parameter tweaking can have a huge impact on performance, and you may still get stuck in local minimas (it really depends on the kind of problem you are dealing with).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am one of the developers of this project.&lt;/p&gt;&#xA;" OwnerUserId="4204" LastEditorUserId="4204" LastEditDate="2013-12-12T21:53:40.397" LastActivityDate="2013-12-12T21:53:40.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10302" PostTypeId="2" ParentId="541" CreationDate="2013-12-12T21:59:18.503" Score="1" Body="&lt;p&gt;As @Geoff Oxberry pointed out, several packages allow you to find a local solution. If you want to be able to compare these different NLP solvers for a same problem, you can use &lt;a href=&quot;http://roboptim.net/&quot; rel=&quot;nofollow&quot;&gt;RobOptim&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though RobOptim was initially developed with robotics optimization problems in mind, it is suitable for any nonlinear optimization problems. It provides a simple C++ interface with plugins for multiple NLP solvers (e.g. Ipopt, NAG). If you cannot provide gradients, finite-difference computation can be done automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's open source so you can check out the source code on GitHub: &lt;a href=&quot;https://github.com/roboptim/&quot; rel=&quot;nofollow&quot;&gt;https://github.com/roboptim/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am one of the developers of this project.&lt;/p&gt;&#xA;" OwnerUserId="4204" LastActivityDate="2013-12-12T21:59:18.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10303" PostTypeId="1" CreationDate="2013-12-12T22:31:02.483" Score="4" ViewCount="177" Body="&lt;p&gt;I am solving a problem using an iteratively-reweighted least squares method:  &lt;a href=&quot;http://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially this requires solving a number of least-squares problems of the form $$D_i A\vec x\approx D_i\vec b,$$ where each $D_i$ is a different diagonal weighting matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to pre-factor $A$ or do other calculations to speed up iterations of this algorithm?  Right now, I'm resolving the least-squares problem from scratch during each iteration.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2013-12-18T20:06:13.013" Title="Factorization for reweighted least squares" Tags="|linear-algebra|iterative-method|matrix|least-squares|matrix-factorization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10304" PostTypeId="1" CreationDate="2013-12-13T00:39:16.287" Score="5" ViewCount="1448" Body="&lt;p&gt;I am trying to implement a simple finite volume method solver. I had a class on FVM a while back, but am still aware of the principal concepts. But implementing the FVM for non-cartesian or 1D meshes is not straightforward to me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. I have a mesh representing my control volumes. I compute the cell centroids to be used as my DOF later on. How would I compute the connectivity between cell centroids? I can compute this by searching for cells that share a cell surface, but this is very naive and hence computationally very inefficient. Further, I just don't see sophisticated softwares doing so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for anything that might help me understand some of the tricks involved in implementing a FVM solver efficiently! Things that are not directly involved with the underlying mathematical/numerical schemes, but rather the set-up.&lt;/p&gt;&#xA;" OwnerUserId="6826" LastActivityDate="2013-12-13T08:44:26.317" Title="Finite Volume Implementation" Tags="|finite-volume|mesh-generation|numpy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10306" PostTypeId="2" ParentId="10304" CreationDate="2013-12-13T08:44:26.317" Score="4" Body="&lt;p&gt;I have 1D finite-volume code written in python for a &lt;em&gt;cell-centred&lt;/em&gt; mesh,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/W1jgY.png&quot; alt=&quot;Cell centred mesh&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First generate a sequence which is the location of the &lt;em&gt;faces&lt;/em&gt;, $&#xA;\{ x_{j-1/2} \}$. For example, for uniform spacing over the domain [0,1] this is a simple as, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = 0&#xA;b = 1&#xA;J = 50 &#xA;faces = numpy.linspace(a, b, J}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After you have the faces the rest is easy because the faces uniquely determine the cell centres,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x_j = \frac{1}{2}\left(x_{j-1/2} + x_{j+1/2} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the cell volumes (only a &lt;em&gt;true&lt;/em&gt; volume in 3D but we stick with the wording),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;h_j = x_{j+1/2} - x_{j-1/2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For connivence you can define the centroid spacing, this is useful when interpolating values to/from centres/faces. Following &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3540034404&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hundsorfer's approach&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;h_{-} = x_j - x_{j-1} = \frac{1}{2}\left( h_{j-1} + h_j \right) \\&#xA;h_{+} = x_{j+1} - x_j = \frac{1}{2}\left( h_{j} + h_{j+1} \right) &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following project on github might be useful, &lt;a href=&quot;https://github.com/danieljfarrell/FVM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/danieljfarrell/FVM&lt;/a&gt; to see how this can be implemented in practice. I use a &lt;code&gt;Mesh&lt;/code&gt; and &lt;code&gt;CellVariable&lt;/code&gt; objects (&lt;a href=&quot;http://www.ctcms.nist.gov/fipy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;inspired by Fipy&lt;/a&gt;). You can instantiate the mesh and the query it for centroid position, mesh volume etc. The &lt;code&gt;CellVariable&lt;/code&gt; class using the mesh to do transparent linear interpolation of solution variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the mesh class and an example of how to use it. It might be a useful basis for your implementation,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Mesh(object):&#xA;    &quot;&quot;&quot;A 1D cell centered mesh defined by faces for the finite volume method.&quot;&quot;&quot;&#xA;&#xA;    def __init__(self, faces):&#xA;        super(Mesh, self).__init__()&#xA;&#xA;        # Check for duplicated points&#xA;        if len(faces) != len(set(faces)):&#xA;            raise ValueError(&quot;The faces array contains duplicated positions. No cell can have zero volume so please update with unique face positions.&quot;)&#xA;        self.faces = np.array(faces)&#xA;        self.cells = 0.5 * (self.faces[0:-1] + self.faces[1:])&#xA;        self.J = len(self.cells)&#xA;        self.cell_widths = (self.faces[1:] - self.faces[0:-1])&#xA;&#xA;    def h(self, i):&#xA;        &quot;&quot;&quot;Returns the width of the cell at the specified index.&quot;&quot;&quot;&#xA;        return self.cell_widths[i]&#xA;&#xA;    def hm(self, i):&#xA;        &quot;&quot;&quot;Distance between centroids in the backwards direction.&quot;&quot;&quot;&#xA;        if not check_index_within_bounds(i,1,self.J-1):&#xA;            raise ValueError(&quot;hm index runs out of bounds&quot;)&#xA;        return (self.cells[i] - self.cells[i-1])&#xA;&#xA;    def hp(self, i):&#xA;        &quot;&quot;&quot;Distance between centroids in the forward direction.&quot;&quot;&quot;&#xA;        if not check_index_within_bounds(i,0,self.J-2):&#xA;            raise ValueError(&quot;hp index runs out of bounds&quot;)&#xA;        return (self.cells[i+1] - self.cells[i])&#xA;&#xA;import numpy&#xA;faces = numpy.linspace(0,1,6)&#xA;mesh = Mesh(faces)&#xA;mesh.cells # i.e. cell centres&#xA;# array([ 0.1,  0.3,  0.5,  0.7,  0.9])&#xA;mesh.cell_widths # i.e. cell volumes&#xA;# array([ 0.2,  0.2,  0.2,  0.2,  0.2])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2013-12-13T08:44:26.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10307" PostTypeId="1" AcceptedAnswerId="10308" CreationDate="2013-12-13T19:07:09.767" Score="0" ViewCount="1800" Body="&lt;p&gt;Matlab ttest &lt;a href=&quot;http://www.mathworks.com/help/stats/ttest.html#btrj1bc&quot; rel=&quot;nofollow&quot;&gt;&quot;returns a test decision for the null hypothesis that the data in x comes from a normal distribution with mean equal to zero and unknown variance, using the one-sample t-test.&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone clarify what exactly is output by the command ttest when run on a vector of numbers $a_1, a_2, \ldots, a_n$?  OK to answer in a form such as &#xA;&quot;it outputs the value $x$ that maximizes $y$.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize this is probably a trivial question for the experts, but I can't easily find a clear answer.&lt;/p&gt;&#xA;" OwnerUserId="4507" LastActivityDate="2013-12-13T20:52:06.320" Title="Matlab ttest does what exactly?" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10308" PostTypeId="2" ParentId="10307" CreationDate="2013-12-13T20:52:06.320" Score="1" Body="&lt;p&gt;Matlab's &lt;code&gt;ttest&lt;/code&gt; takes your vector of data and performs a &lt;a href=&quot;http://en.wikipedia.org/wiki/Student%27s_t-test#One-sample_t-test&quot; rel=&quot;nofollow&quot;&gt;Student's (one-sample) t-test&lt;/a&gt; on it, assuming that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the population mean you're testing against, $\mu_{0}$, is zero&lt;/li&gt;&#xA;&lt;li&gt;$n$ is equal to &lt;code&gt;length(x)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;the level of statistical signficance, or &lt;a href=&quot;http://en.wikipedia.org/wiki/Type_I_error#Type_I_error&quot; rel=&quot;nofollow&quot;&gt;Type I error&lt;/a&gt;, you're willing to accept is 5%; you can change the amount of Type I error you're willing to accept in the arguments of the function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The $t$-test calculates the mean of the data in &lt;code&gt;x&lt;/code&gt; (i.e., $\bar{x} =$ &lt;code&gt;sum(x)/length(x)&lt;/code&gt;), and its &lt;a href=&quot;http://en.wikipedia.org/wiki/Standard_deviation#Estimation&quot; rel=&quot;nofollow&quot;&gt;sample standard deviation&lt;/a&gt;, $s$, typically with the formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;s = \sqrt{\frac{1}{n - 1}\sum_{i = 1}^{n}(x_{i} - \bar{x})^{2}},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which corrects for the fact that $s$ estimates the true standard deviation of the population from which &lt;code&gt;x&lt;/code&gt; samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the $t$-statistic is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;t = \frac{\bar{x} - \mu_{0}}{s/\sqrt{n}} = \frac{\bar{x}}{s/\sqrt{n}},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because $\mu_{0}$ is assumed equal to zero. The documentation doesn't say, so I assume that the test is a bidirectional $t$-test, which means that &lt;code&gt;ttest&lt;/code&gt; returns 1 if $t$ is greater than &lt;code&gt;tinv(0.95, length(x))&lt;/code&gt; or less than &lt;code&gt;tinv(0.05, length(x))&lt;/code&gt; (these are the t-statistics corresponding to a 5% level of significance; it should be the case that &lt;code&gt;tinv(0.05, length(x))&lt;/code&gt; equals &lt;code&gt;-tinv(0.95, length(x))&lt;/code&gt;). Otherwise, &lt;code&gt;ttest&lt;/code&gt; returns 0.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-13T20:52:06.320" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10309" PostTypeId="1" AcceptedAnswerId="10310" CreationDate="2013-12-13T21:17:08.063" Score="2" ViewCount="172" Body="&lt;p&gt;I would like to minimize:&#xA;$$J = \int_{\Omega} \|\nabla u - \nabla g\|^2 + \lambda \|\frac{\partial u}{\partial t} + \nabla u.v||^2 ~\text{dx dy dt}$$&#xA;where $u(x,y,t)$ is the unknown function, the integral is over a fixed spatio-temporal domain $\Omega$, $\nabla$ is the spatial gradient, $v(x,y,t)$ is a velocity (vector) field. $g(x,y,t)$ and $v(x,y,t)$ are known. $\lambda$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing and cancelling the variation of $J$ using a neighborhood $u+\epsilon h$ of $u$, I obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ ``\frac{\partial J}{\partial u}&quot; =  \int_{xyt} -\Delta (u-g).h + \lambda (\frac{\partial u}{\partial t} + \nabla u.v)(\frac{\partial h}{\partial t} + \nabla h.v) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If only the term $\Delta (u-g).h$ was present, I could directly say that if the integral is zero for all $h$, then $\Delta (u-g)$ is zero locally. I'd obtain the standard Poisson equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the addition of the other term makes me wonder how I can turn this expression into a local PDE to solve via finite differences. &#xA;Any idea ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1440" LastEditorUserId="1440" LastEditDate="2013-12-14T01:22:42.963" LastActivityDate="2013-12-14T01:22:42.963" Title="variational formulation" Tags="|pde|finite-difference|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10310" PostTypeId="2" ParentId="10309" CreationDate="2013-12-13T22:53:28.117" Score="2" Body="&lt;p&gt;You already have a variational formulation of your problem. You can get the strong form if you integrate back by parts so that you get a formula of the form&#xA;$$&#xA;  \int (something) h \; dx \; dt = 0.&#xA;$$&#xA;(In fact, you will probably get multiple integrals to take into account boundary and initial terms coming out of the integration by parts.) The strong form is then&#xA;$$&#xA;  something = 0.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-13T22:53:28.117" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10311" PostTypeId="2" ParentId="10297" CreationDate="2013-12-13T23:30:16.143" Score="1" Body="&lt;p&gt;As has been pointed out in the comments on the question, the function is not analytic at $x=0$ and a Taylor expansion around this point is not possible. However, the function is analytic everywhere else. If you have an idea of the size of your (small) values for $y=x^2$, then you can of course still do one of the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A Taylor expansion around a representative value for $y$.&lt;/li&gt;&#xA;&lt;li&gt;A low-degree polynomial approximation for the range of values $y$ you care about.&lt;/li&gt;&#xA;&lt;li&gt;A low-degree rational approximation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Because the function is very smooth, you will likely good a fairly good approximation using any of these techniques.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-13T23:30:16.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10312" PostTypeId="2" ParentId="10300" CreationDate="2013-12-14T09:20:26.583" Score="1" Body="&lt;p&gt;So you are essentially asking how you can constrain the cardinality of a vector. The answer below is based on a standard approach called big-M modelling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $x$ be a vector for which you want exactly $c$ non-zeros. Introduce a new binary vector $y$ which indicates if an element is zero or non-zero. Let $M$ be an upper bound on the possible value of an element in $x$ at optimality (from domain knowledge)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $y_i=0$ then $x_i$ should be $0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x \leq  My$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also ensures $y_i=1$ when $x_i$ is non-zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $y_i=1$ then $x_i$ non-zero. This is a bit tricky, since you have to define what non-zero is in the numerical precision you are working with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ x \geq 0.00001y$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also ensures $y_i=0$ if $x_i=0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Exactly $c$ elements are non-zero&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum y = c$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All remaining constraints are standard linear programming constraints, and the logic above is applied on every row of $K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code below implements an example in the MATLAB Toolbox YALMIP (disclaimer:developed by me)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m = 5;&#xA;n = 10;&#xA;% create a known solution&#xA;K0 = sprand(m,n,.25)&#xA;a = sum(K0,2);&#xA;b = sum(K0,1);&#xA;c = sum(K0&amp;gt;0,2);&#xA;&#xA;% Define decision variables&#xA;K = sdpvar(m,n,'full')&#xA;Y = binvar(m,n,'full');&#xA;M = max(max(a),max(b));&#xA;&#xA;Model = [sum(K,1)==b, sum(K,2)==a, K&amp;gt;=0];&#xA;Model = [Model, K &amp;lt;= M*Y, K &amp;gt;= 0.00001*Y,sum(Y,2)==c];&#xA;&#xA;% Let us find a solution with smallest largest element&#xA;Objective = max(max(K));&#xA;solvesdp(Model,Objective);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-12-14T09:20:26.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10313" PostTypeId="2" ParentId="10297" CreationDate="2013-12-14T14:41:54.643" Score="4" Body="&lt;p&gt;Use $\exp\left(-\tfrac1{x^2}\right) =\frac1{\exp\left(\tfrac1{x^2}\right)}$ and expand in the denominator, for example as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\exp\left(-\tfrac1{x^2}\right) \approx \frac1{1+1/x^2+1/(2x^4)}=\frac{4x^4}{(1+2x^2)^2+1}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is correct for $1/x^6$ below machine precision, i.e. for $x&amp;gt;1000$ and has the qualitatively correct behavior for smaller x.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use the halving and squaring technique to get the cross-over from numerically to qualitatively exact closer to zero,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\exp\left(-\tfrac1{x^2}\right)=\exp\left(-\tfrac1{nx^2}\right)^n \approx \left[\frac1{1+1/(nx^2)+1/(2n^2x^4)}\right]^{n} = \left[\frac{4n^2x^4}{1+(1+2nx^2)^2}\right]^{n} $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is numerically exact for $1/(nx)^6$ below machine precision, $nx^2&amp;gt;10^5$, for example with $n=10$ for $x&amp;gt;100$, and qualitatively exact below that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the numerator in the last example to give a value above machine precision, $400x^4&amp;gt;10^{-2}$ or $x&amp;gt;0.02$ is necessary. Since this approximation is an upper bound for the true value, this is as close as it gets, i.e., one could alternatively return $0$ for $x&amp;lt;0.02$ and $\exp(-1/x^2)$ for $x\ge 0.02$.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2013-12-14T15:18:00.603" LastActivityDate="2013-12-14T15:18:00.603" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10314" PostTypeId="2" ParentId="7812" CreationDate="2013-12-14T16:15:52.600" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://page.mi.fu-berlin.de/rote/Papers/pdf/Division-free+algorithms.pdf&quot;&gt;Division-free algorithms&lt;/a&gt; for the determinant are the Samuelson-Berkowitz algorithm (see german wikipedia) and the Leverrier-Faddeev algorithm (see german wikipedia). The latter requires division by integers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a variant of the Gauß algorithm, the Gauß-Bareiss algorithm, that does book-keeping on numerators, i.e., keeping it in a factored form and reducing the fractions if possible. Wikipedia only gives integer matrices as example, but the method applies to integral (euclidean?) rings in general, so the polynomial ring over the integers is covered.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2013-12-14T16:15:52.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10315" PostTypeId="2" ParentId="5007" CreationDate="2013-12-14T16:28:01.847" Score="2" Body="&lt;p&gt;To complete the enumeration of numerical methods, if the known roots really are good approximations for the roots of the next polynomial, then you can use Weierstraß' method, commonly known as Durand-Kerner method. It updates all roots at once, and is as fast as Newton's method, in fact is is Newton's method for a multidimensional problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aberth-Ehrlich method is related, but it is not certain that the higher computational effort gives an overall faster convergence.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Additionally, since you are doing some kind of ray tracing, the univariate polynomials resulting from the intersection of a polynomial surface with a ray, you can divide the image into 6x6 or 8x8 blocks, compute the roots at a central pixel and then compute the gradients of the surface at the intersection points. You may get inspiration on how to do that by playing with the derivative code generator tapenade.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the point $x^*$ and the gradient $\nabla f(x^*)$ for the tangent plane equation $\langle \nabla f(x^*),\,x\rangle=\langle \nabla f(x^*),\,x^*\rangle$, you get better initial approximations by intersecting the rays $x_v(t)=\vec c+t\vec v$ of the other pixels first with the tangent plane. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check for $\langle \nabla f(x^*),\,v\rangle\approx 0$, this occurs close to the visual contours. These are the critical cases in any approach since they have multiple or very close roots.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2013-12-15T02:05:48.737" LastActivityDate="2013-12-15T02:05:48.737" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10316" PostTypeId="2" ParentId="10216" CreationDate="2013-12-14T17:06:52.683" Score="4" Body="&lt;p&gt;Since your system is Hamiltonian, no frictions involved, you can also use the Newton-Stoermer-Verlet-Leapfrog (there were more inventors, see the paper of Hairer etal. mentioned in the wikipedia article) method for a fast simple method with reasonable physical properties. These so-called symplectic integrators treat position and velocity separately, in lockstep.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2013-12-14T17:06:52.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10317" PostTypeId="1" AcceptedAnswerId="10318" CreationDate="2013-12-14T20:55:24.083" Score="-1" ViewCount="632" Body="&lt;p&gt;I have my pde 2D problem with robin condition (form: du/dn +ku=g) to solve with matlab. i have the exact function u and I want to find the function g in robin condition. How can i do it?&#xA;thanks for the help :)&lt;/p&gt;&#xA;" OwnerUserId="6703" LastActivityDate="2013-12-14T21:46:38.167" Title="Pde problem with robin boundary condition" Tags="|pde|finite-element|boundary-conditions|numerics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10318" PostTypeId="2" ParentId="10317" CreationDate="2013-12-14T21:46:38.167" Score="4" Body="&lt;p&gt;You can compute $\nabla u$ and the outward normal $n$ to get $\frac{\partial u}{\partial n}$. Add that and $\kappa u$ on the Robin boundary and get $g$. Is that what you're looking for?&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2013-12-14T21:46:38.167" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10319" PostTypeId="1" CreationDate="2013-12-15T08:56:02.510" Score="1" ViewCount="44" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jb1o1.jpg&quot; alt=&quot;My problem&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote the code to fix this problem; now  I should validate it using a test function to see the error in my code. I can't figure out how starting from u can derive the function g on the edge, in particular I don't manage to compute the outward normal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code to calculate g: &lt;a href=&quot;https://dpaste.de/hzYm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dpaste.de/hzYm&lt;/a&gt; (p,t,e are output of initmesh)&lt;/p&gt;&#xA;" OwnerUserId="6717" LastActivityDate="2013-12-15T08:56:02.510" Title="how to find outward normal for robin codition" Tags="|pde|finite-element|matlab|boundary-conditions|" AnswerCount="0" CommentCount="1" ClosedDate="2013-12-25T19:20:43.653" ContentLicense="CC BY-SA 3.0" />
  <row Id="10320" PostTypeId="2" ParentId="10280" CreationDate="2013-12-15T17:12:41.227" Score="1" Body="&lt;p&gt;For gradient computation, you use the reverse mode of AD. This requires in both cases to build an operand stack, the OO version also needs to build up an operations stack, which has to be interpreted in the reverse traversal of the code. Source transformed code writes out the reverse-ordered operations as additional source code that is compiled. The overhead to have the operations interpreter in the code can be significant. There are comparisons of Tapenade generated code and Adol-C that come out in favor of Tapenade.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2013-12-15T17:12:41.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10321" PostTypeId="1" CreationDate="2013-12-15T18:28:04.933" Score="7" ViewCount="2306" Body="&lt;p&gt;Is there an efficient way to find the largest negative eigenvalue of a matrix? The matrix in question is a Markov matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing the full spectrum of the matrix by decomposing it is not an acceptable solution.&lt;/p&gt;&#xA;" OwnerUserId="533" LastEditorUserId="1804" LastEditDate="2016-05-27T15:47:28.770" LastActivityDate="2016-05-27T15:47:28.770" Title="Largest negative eigenvalue" Tags="|linear-algebra|eigenvalues|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10322" PostTypeId="2" ParentId="10321" CreationDate="2013-12-15T18:42:09.050" Score="1" Body="&lt;p&gt;Here is one solution: Add $\|A\|_{inf}$ to the diagonal elements of $A$. Compute the smallest eigenvalue of the resulting matrix using inverse power iterations. Subtract $\|A\|_{inf}$ from the resulting eigenvalue to get the largest negative eigenvalue of $A$.&lt;/p&gt;&#xA;" OwnerUserId="533" LastEditorUserId="533" LastEditDate="2013-12-16T09:17:04.747" LastActivityDate="2013-12-16T09:17:04.747" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10323" PostTypeId="2" ParentId="10321" CreationDate="2013-12-15T21:39:01.937" Score="6" Body="&lt;p&gt;@VictorMay has provided an answer using the inverse power iteration, but this is of course expensive. If you have an estimate that $\bar\lambda &amp;gt; \lambda_i$, i.e., it is an upper bound to all eigenvalues including the positive ones, then $A-\bar\lambda I$ is a negative definite matrix. You can then apply the power iteration (instead of the &lt;em&gt;inverse&lt;/em&gt; power iteration) to finding the largest eigenvalue by magnitude. Call it $\mu$. Since the matrix is negative definite, you then know that the most negative eigenvalue of $A$ is $\lambda_\text{min}=-\mu+\bar\lambda$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-15T21:39:01.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10324" PostTypeId="2" ParentId="10321" CreationDate="2013-12-15T22:41:05.237" Score="4" Body="&lt;p&gt;While the inverse power iteration requires an initial matrix decomposition of any sensible type, which requires $O(n^3)$ operations, the following vector iterations are $O(n^2)$ like the forward power iteration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, to make this initial step overly complicated, one may start with a transformation to Hessenberg form, use Gerschgorin circles to locate the eigenvalues and then modify the diagonal, since $Q^TAQ+cI=Q^T(A+cI)Q$. Then factor the resulting tridiagonal matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expense in initial cost may be justified by faster convergence. The forward iterate converges linearly with factor &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q_D=\frac{|\lambda_2|+\bar\lambda}{|\lambda_1|+\bar\lambda}=1-\frac{|\lambda_1|-|\lambda_2|}{|\lambda_1|+\bar\lambda}$$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\lambda_1&amp;lt;\lambda_2&amp;lt;\lambda_k$ are the smallest and second smallest eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inverse power iteration with only the initial shift converges with factor&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q_I=\frac{\bar\lambda-|\lambda_1|}{\bar\lambda-|\lambda_2|}=1-\frac{|\lambda_1|-|\lambda_2|}{\bar\lambda-|\lambda_2|}$$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which because of the structurally smaller denominator will be smaller than $q_D$. One only needs an advantage of $O(n)$ iteration that the inverse PIu uses less to justify the initial costs.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="276" LastEditDate="2013-12-16T18:42:54.853" LastActivityDate="2013-12-16T18:42:54.853" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10325" PostTypeId="2" ParentId="7812" CreationDate="2013-12-16T00:07:45.763" Score="0" Body="&lt;p&gt;I've actually come up with a nice trick!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compute the determinants obtained by evaluating one of the variables (call it $a$) at a small number of large primes $\{p_i\}$. These are rational functions in the other variable (call it $b$). Assume for simplicity they are actually irreducible polynomials $f_i(b)$ (but there's an easy variation of this scheme for the general case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the constant terms satisfy $f_i(0) = Det(p_i, 0) \pmod{p_i}$, and using the Chinese remainder theorem we can compute $Det(a, 0)$ as a polynomial in $a$, up to an ambiguity in each coefficient of $\prod_i p_i$. Ignore that ambiguity for now, subtract off $Det(a,0)$, divide by $b$, and continue until you're done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, we have a guess for the determinant as a two-variable polynomial. Use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Schwartz%E2%80%93Zippel_lemma&quot; rel=&quot;nofollow&quot;&gt;Schwartz-Zippel lemma&lt;/a&gt;, which only requires evaluating the determinant at rational values of the two variables, to efficiently obtain arbitrarily good bounds on the probability you've got the wrong answer.&lt;/p&gt;&#xA;" OwnerUserId="4645" LastActivityDate="2013-12-16T00:07:45.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10326" PostTypeId="1" CreationDate="2013-12-16T20:23:04.050" Score="2" ViewCount="143" Body="&lt;p&gt;First off I would like to mention that I am absolutely terrible at statistics... so bear with me please.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The question:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;a sha1 hash is a hexadecimal string of 40 characters, the largest number being:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ffffffffffffffffffffffffffffffffffffffff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which if you convert it to a decimal, it becomes around &lt;code&gt;1.2830013993246E+48&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what the exact value is or how to obtain it, but lets say that it is &lt;code&gt;X&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the probability that X number of unique strings when hashed with the sha1 algorithm will have every single possible character combination of a sha1 string?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And does this apply for other hashing algorithms too??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically is a sequence of 2^160 unique strings guaranteed to generate every possible combination of a sha1 hash?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. if someone has a better title for this question it would be much appreciated, also I am unsure if this is the best place to ask this :/&lt;/p&gt;&#xA;" OwnerUserId="6860" LastEditorUserId="6860" LastEditDate="2013-12-17T19:09:42.580" LastActivityDate="2013-12-19T17:14:55.380" Title="Hash in set probability?" Tags="|probability|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10327" PostTypeId="2" ParentId="10326" CreationDate="2013-12-16T22:34:16.443" Score="1" Body="&lt;p&gt;If you repeatedly select elements from a bin with $n$ objects, replacing the objects each time, the expected time to select each element at least once is&#xA;$$E = n \left(\log n + \gamma + o(1)\right).$$&#xA;For $n = 2^{160}$, this is $1.6 \times 10^{50}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see this, note that the probability of choosing a new object if $k$ elements are already chosen is $(n-k)/n$.  Summing gives&#xA;$$E = \sum_{k=0}^{n-1} \frac{n}{n-k} = n \sum_{k=1}^n \frac{1}{k} = n H_n$$&#xA;where $H_n$ is the $n$th harmonic number.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2013-12-19T17:14:55.380" LastActivityDate="2013-12-19T17:14:55.380" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10328" PostTypeId="2" ParentId="10326" CreationDate="2013-12-17T01:32:24.700" Score="1" Body="&lt;p&gt;The probability is zero, because X is less (by one) than the number of possible character combinations of a sha1 string.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-12-17T01:32:24.700" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10329" PostTypeId="1" AcceptedAnswerId="10330" CreationDate="2013-12-17T01:53:52.513" Score="7" ViewCount="2794" Body="&lt;p&gt;I'm implementing a mass-spring solid mechanics solver and I'd like to use the Velocity-Verlet time integration scheme. However, I cannot find anything about the maximum stable time step -- either saying there is none and it's unconditionally stable like an implicit scheme or that there is one and it's bound by something like the explicit schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would think the maximum time step is related to the maximum force and the minimum distance between the particles if there is one, but I can't seem to find anything. What is the stable time step limit for the Velocity-Verlet integration?&lt;/p&gt;&#xA;" OwnerUserId="391" LastActivityDate="2013-12-17T17:04:11.143" Title="Stable time step limits for Velocity-Verlet integration" Tags="|stability|time-integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10330" PostTypeId="2" ParentId="10329" CreationDate="2013-12-17T10:25:13.600" Score="6" Body="&lt;p&gt;In the following, I am basically rephrasing p. 24 of &lt;a href=&quot;http://arxiv.org/abs/1206.1901&quot; rel=&quot;noreferrer&quot;&gt;&quot;MCMC using Hamiltonian Dynamics&quot;&lt;/a&gt; by Radford Neal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least for leapfrog integration, one can analytically calculate the maximum time step for quadratic Hamiltonians. For such systems, a leapfrog step is a linear mapping $(q(t), p(t) \mapsto (q(t+\epsilon), p(t+\epsilon))$. Stability then depends on the magnitude of the eigenvalues of the matrix representing this mapping.&#xA;For a harmonic oscillator with force constant $k$, these eigenvalues are given by $(1 - \frac{k\epsilon^2}{2}) \pm \epsilon \sqrt k \sqrt{\frac{k \epsilon^2}{4} - 1}$. So for $\sqrt{k}{\epsilon} &amp;lt; 2$, the eigenvalues are complex and have a squared magnitude of $1$. The system simulated with this timestep will therefor be stable. On the other hand, if $\sqrt{k} \epsilon &amp;gt; 2$, the eigenvalues are real and at least one of them will have an absolute value $&amp;gt; 0$. This means a simulation with this time step will be unstable when you iterate this scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this is for leapfrog integration, but as Velocity Verlet integration is closely related, I think one can calculate a similar stability criterion and if not, then I think the result for leapfrog integration should also be comparable to the maximum time step for Velocity Verlet integration.&#xA;Specifically for your problem, I guess the maximum time step will be given by the spring constant of the stiffest spring; &quot;mass-spring solid mechanics&quot; sounds suspiciously like harmonic potentials to me.&lt;/p&gt;&#xA;" OwnerUserId="4520" LastEditorUserId="4520" LastEditDate="2013-12-17T17:04:11.143" LastActivityDate="2013-12-17T17:04:11.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10331" PostTypeId="1" CreationDate="2013-12-17T14:22:27.393" Score="7" ViewCount="1463" Body="&lt;p&gt;So I'm trying to implement a Gauss-Kronrod adaptive quadrature. That is, I want to calculate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_a^b f(x) dx = \sum_i f(x_i) w_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where f(x) is evaluated at multiple points at once for efficiency. A key point to bear in mind is that $f$ is also &lt;em&gt;vector-valued&lt;/em&gt; (possible risk of confusion here with the double meaning, and use of, &lt;em&gt;vectorised&lt;/em&gt;); in practice, I'm integrating at once the real part and imaginary part of a complex integrand, or the integrand could be a 3D vector (electric field), etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've worked out the basic quadrature step in R (ultimately to be ported to C++ using the Armadillo library, any Matlab-like matrix syntax will do for the sake of discussion), where I first divide the integral into N portions, and evaluate the weighted sums corresponding to the Gauss and Kronrod nodes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## nodes and weights borrowed from Octave source code&#xA;&#xA;abscissa = c(-0.9914553711208126e+00, -0.9491079123427585e+00, &#xA;             -0.8648644233597691e+00, -0.7415311855993944e+00, &#xA;             -0.5860872354676911e+00, -0.4058451513773972e+00, &#xA;             -0.2077849550078985e+00,  0.0000000000000000e+00, &#xA;             0.2077849550078985e+00,  0.4058451513773972e+00, &#xA;             0.5860872354676911e+00,  0.7415311855993944e+00, &#xA;             0.8648644233597691e+00,  0.9491079123427585e+00, &#xA;             0.9914553711208126e+00)&#xA;&#xA;weights15 = c(0.2293532201052922e-01,  0.6309209262997855e-01, &#xA;              0.1047900103222502e+00,  0.1406532597155259e+00, &#xA;              0.1690047266392679e+00,  0.1903505780647854e+00, &#xA;              0.2044329400752989e+00,  0.2094821410847278e+00, &#xA;              0.2044329400752989e+00,  0.1903505780647854e+00, &#xA;              0.1690047266392679e+00,  0.1406532597155259e+00, &#xA;              0.1047900103222502e+00,  0.6309209262997855e-01, &#xA;              0.2293532201052922e-01)&#xA;&#xA;weights7  = c(0.1294849661688697e+00,  0.2797053914892767e+00, &#xA;              0.3818300505051889e+00,  0.4179591836734694e+00, &#xA;              0.3818300505051889e+00,  0.2797053914892767e+00, &#xA;              0.1294849661688697e+00)&#xA;&#xA;gauss_kronrod &amp;lt;- function(f, a, b, startN=10, dimf = 2, eps = 1e-3, ...){&#xA;&#xA;  Nquad &amp;lt;- length(abscissa)&#xA;  intervals &amp;lt;- seq(a, b, length=startN)&#xA;  ## change of variable from [a, b] to [-1, 1]&#xA;  shifts &amp;lt;- (intervals[-length(intervals)] + intervals[-1]) / 2&#xA;  scalings &amp;lt;- diff(intervals) / 2&#xA;&#xA;  ## scale the weights accordingly&#xA;  weightsG &amp;lt;- tcrossprod(weights15, scalings)&#xA;  weightsK &amp;lt;- tcrossprod(weights7, scalings)&#xA;&#xA;  ## transformed node positions&#xA;  x.scaled &amp;lt;- tcrossprod(abscissa, scalings) +&#xA;    matrix(shifts, ncol=length(shifts), nrow=Nquad, byrow=TRUE)&#xA;&#xA;  ## evaluate the function at all points&#xA;  fvals &amp;lt;- f(c(x.scaled))&#xA;&#xA;  ## function evaluations are reshaped into a 3D array&#xA;  ## rows correspond to dimf values of integrand&#xA;  ## columns are the Nquad evalution points in the sub-intervals&#xA;  ## slices correspond to the (startN - 1) sub-intervals&#xA;  dim(fvals) = c(dimf, Nquad, startN-1)&#xA;&#xA;  ## select which subset of fvals are used for the 7-points kronrod sum&#xA;  ikron &amp;lt;- rep(c(FALSE, TRUE), length.out=Nquad)&#xA;&#xA;&#xA;  gauss &amp;lt;- kronrod &amp;lt;- matrix(0, nrow=dimf, ncol=startN-1)&#xA;  for (ii in seq.int(startN-1)){ # integrals for each sub-interval&#xA;    gauss[,ii] &amp;lt;- fvals[,,ii] %*% weightsG[,ii]&#xA;    kronrod[,ii] &amp;lt;- fvals[,ikron,ii] %*% weightsK[,ii]&#xA;  }&#xA;  ## relative difference between the two rules &#xA;  errors &amp;lt;- abs(gauss - kronrod) / abs(gauss)&#xA;  test &amp;lt;- which(apply(errors, 2, max) &amp;gt; eps)&#xA;&#xA;  ## net integral, just sum the partial sums...&#xA;  rowSums(gauss)&#xA;&#xA;}&#xA;&#xA;# a vector-valued integrand...&#xA;f &amp;lt;-  function(x) rbind(exp(x/(x+2))*sin(x), cos(x))&#xA;&#xA;gauss_kronrod(f, 2, 110, startN=10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My problem comes when trying to implement an adaptive scheme; basically, at each iteration I would like to compare the values of &lt;code&gt;gauss&lt;/code&gt; vs &lt;code&gt;kronrod&lt;/code&gt; for each of the subintervals, and from there decide where to subdivide further. What would be a good strategy to store these intermediate results during the iteration, and keep track of which subintervals sums go where etc. in the final summation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More general advice on this endeavour will also be appreciated, I'm not usually keen to implement such generic operations myself, but it seems it might be easier than linking different libraries together (I was contemplating &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/Cubature&quot;&gt;cubature&lt;/a&gt;, initially, because my end goal is 2D integration).&lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2013-12-19T07:11:38.907" Title="adaptive Gauss-Kronrod quadrature with vector-valued integrand" Tags="|quadrature|data-structures|r|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10332" PostTypeId="1" CreationDate="2013-12-17T15:46:22.820" Score="1" ViewCount="1620" Body="&lt;p&gt;I'd like to find the normalized ground state wavefunction for the anharmonic oscillator (Duffing) whose potential for which there is no analytic solution; an oscillator with a quartic potential, in addition to the quadratic potential. Any comments would be highly appreciated&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$V(x_{-}) = \frac{x^2}{2}+\lambda \frac{x^4}{4}$$&lt;/p&gt;&#xA;" OwnerUserId="6871" LastEditorUserId="27" LastEditDate="2013-12-17T17:00:34.757" LastActivityDate="2014-01-08T12:53:49.327" Title="finding wave function for anharmonic oscillator" Tags="|nonlinear-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10334" PostTypeId="2" ParentId="10332" CreationDate="2013-12-17T17:56:11.673" Score="2" Body="&lt;p&gt;What have you tried so far?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We already know the wave functions and energy levels when $\lambda=0$, they're Hermite functions and integers + a half, respectively. If you know that the parameter $\lambda$ determining the anharmonicity is small, you could use perturbation theory to expand the eigenfunctions and energy levels in a power series in $\lambda$, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\psi = \pi^{-1/4}e^{-x^2/2}+\lambda\psi_1+\lambda^2\psi_2+\ldots$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E = \frac{1}{2}+\lambda E_1+\lambda_2 E_2+\ldots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm happy to elaborate, but the &lt;a href=&quot;http://en.wikipedia.org/wiki/Perturbation_theory_%28quantum_mechanics%29&quot; rel=&quot;nofollow&quot;&gt;wiki article&lt;/a&gt; on quantum mechanical perturbation theory has a better explanation than I could give. It also uses the anharmonic oscillator as an example, although just to find the ground-state energy. The Landau &amp;amp; Lifshitz book has a pretty good chapter on perturbation theory, as does Baym's book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, it's probably just a hypergeometric function, because experience has demonstrated that everything is some kind of hypergeometric function.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-12-17T17:56:11.673" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10335" PostTypeId="2" ParentId="10331" CreationDate="2013-12-17T19:29:13.213" Score="0" Body="&lt;p&gt;Well, I've thought of a strategy, which is the following. First, I create a function to take care of calculating the partial sums over a list of intervals, and checking which are good enough to keep. Second, for those intervals that were not good, a helper function splits them in two. Finally, at each step the good contributions are added to the final result. Here's an implementation in R,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gk_segments &amp;lt;- function(f, segments=list(c(2, 50), c(50, 100)),  dimf = 2, &#xA;                        eps = 1e-3, ...){&#xA;&#xA;  Nquad &amp;lt;- length(abscissa)&#xA;  Nsegments &amp;lt;- length(segments)&#xA;  shifts &amp;lt;- sapply(segments, sum) / 2&#xA;  scalings &amp;lt;- sapply(segments, diff) / 2&#xA;  weightsG &amp;lt;- tcrossprod(weights15, scalings)&#xA;  weightsK &amp;lt;- tcrossprod(weights7, scalings)&#xA;&#xA;  x.scaled &amp;lt;- tcrossprod(abscissa, scalings) +&#xA;    matrix(shifts, ncol=length(shifts), nrow=Nquad, byrow=TRUE)&#xA;&#xA;  fvals &amp;lt;- f(c(x.scaled), ...)&#xA;&#xA;  ## function evaluations are reshaped into a 3D array&#xA;  ## rows correspond to dimf values of integrand&#xA;  ## columns are the Nquad evalution points in the sub-intervals&#xA;  ## slices correspond to the Nsegments sub-intervals&#xA;  dim(fvals) = c(dimf, Nquad, Nsegments)&#xA;&#xA;  ikron &amp;lt;- rep(c(FALSE, TRUE), length.out=Nquad)&#xA;&#xA;  gauss &amp;lt;- kronrod &amp;lt;- matrix(0, nrow=dimf, ncol=Nsegments)&#xA;  for (ii in seq.int(Nsegments)){ # integrals for each sub-interval&#xA;    gauss[,ii] &amp;lt;- fvals[,,ii] %*% weightsG[,ii]&#xA;    kronrod[,ii] &amp;lt;- fvals[,ikron,ii] %*% weightsK[,ii]&#xA;  }&#xA;&#xA;  errors &amp;lt;- abs(gauss - kronrod) / abs(gauss)&#xA;  test &amp;lt;- which(apply(errors, 2, max) &amp;gt; eps)&#xA;&#xA;  if(!length(test))&#xA;    return( list(converged = rowSums(gauss),&#xA;                 todo = list()))&#xA;&#xA;  list(converged = rowSums(gauss[ ,-test]),&#xA;       todo = segments[test])&#xA;}&#xA;&#xA;# f &amp;lt;-  function(x) rbind(exp(x/(x+2))*sin(x), cos(x))&#xA;# gk_segments(f, segments=split_interval(c(2, 110), 10))&#xA;&#xA;# split a given interval [x0, x1] into a list of N subintervals&#xA;split_interval &amp;lt;- function(x, N=2){&#xA;&#xA;  breaks &amp;lt;- seq(x[1], x[2], length=N+1)&#xA;  lapply(seq.int(N), function(ii) c(breaks[ii], breaks[ii+1]))&#xA;}&#xA;&#xA;gk_adaptive &amp;lt;- function(fun, a, b, startN=10, dimf = 2, &#xA;                         eps = 1e-3, maxiter = 100, ...){&#xA;&#xA;  ## first step&#xA;  segments &amp;lt;- split_interval(c(a, b), startN)&#xA;  tmp &amp;lt;- gk_segments(f=fun, segments=segments, dimf=dimf, eps=eps, ...)&#xA;&#xA;  result &amp;lt;- tmp$converged # partial sums that have already converged&#xA;      segments &amp;lt;- unlist(lapply(tmp$todo, split_interval, N=2), recursive=FALSE)&#xA;  iter &amp;lt;- 1&#xA;&#xA;  ## we stop if there's no subinterval left to split&#xA;  ## or we've reached the maximum number of iterations&#xA;  finished &amp;lt;- (!length(tmp$todo)) || (iter &amp;gt; maxiter)&#xA;&#xA;  while(!finished){&#xA;    message(&quot;iteration #&quot;, iter)&#xA;    ## work on the new subsegments&#xA;    tmp &amp;lt;- gk_segments(f=fun, segments=segments, dimf=dimf, eps=eps, ...)&#xA;    result &amp;lt;- result + tmp$converged ## add the good chunks&#xA;        segments &amp;lt;- unlist(lapply(tmp$todo, split_interval, N=2), recursive=FALSE)&#xA;&#xA;    iter &amp;lt;- iter + 1&#xA;    stopping &amp;lt;- iter &amp;gt; maxiter&#xA;    if(stopping)&#xA;      message(&quot;Maximum number of iteration reached, result is likely wrong&quot;)&#xA;    finished &amp;lt;- (!length(tmp$todo)) || stopping&#xA;  }&#xA;&#xA;    return(result)&#xA;}&#xA;&#xA;f &amp;lt;-  function(x) rbind(sin(x), cos(x))&#xA;gk_adaptive(f, 2, 110)&#xA;gk_adaptive(f, 2, 110, startN=10, eps=1e-10)&#xA;gk_adaptive(f, 2, 110, startN=3, eps=1e-10, maxiter=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't think it will be easy to generalise to 2 or more dimensions, because choosing the right square / cubes to focus on iteratively seems harder to code. As a less efficient alternative, I can just chain multiple integrations in 1D, keeping all but one of the variables constant at a time.&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="39" LastEditDate="2013-12-17T19:37:27.670" LastActivityDate="2013-12-17T19:37:27.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10336" PostTypeId="2" ParentId="10326" CreationDate="2013-12-18T18:18:14.653" Score="2" Body="&lt;p&gt;To answer your edit...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Basically is a sequence of 2^160 unique strings guaranteed to generate every possible combination of a sha1 hash?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, there is almost no chance.  The probability is some double exponential like exp(-exp(100)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say that by some miracle you have seen no collision among your first 2^160 - 1 unique strings.  Then your last unique string will still have about a (2^160-1)/(2^160) probability of hashing to a sha1 value that you have already seen.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-12-18T18:18:14.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10337" PostTypeId="1" CreationDate="2013-12-18T19:33:07.177" Score="1" ViewCount="3059" Body="&lt;p&gt;So, I am running this openfoam simulation and I need areaAverage velocity and pressure on a surface,currently the entire plane is being read, I wan't to specify my own in sampleSurfaceDict,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;new_plane&#xA;{&#xA;    type            faceSource;&#xA;    functionObjectLibs (&quot;libfieldFunctionObjects.so&quot;);&#xA;    enabled         true;&#xA;    outputControl   timeStep;&#xA;    log             true;&#xA;    valueOutput     true;&#xA;    source          sampledSurface;&#xA;    //setFormat raw;&#xA;    surfaceFormat   vtk;&#xA;&#xA;&#xA;    sampledSurfaceDict&#xA;    {&#xA;        type            plane;    // always triangulated&#xA;        basePoint       (0   -1.5 0);&#xA;        normalVector    (0 -1 0);&#xA;    }&#xA;    operation       areaAverage;&#xA;    fields&#xA;    (&#xA;        p&#xA;        U&#xA;    );&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is there a type circle? or a way to specify radius? in sampledSurfaceDict&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sampledSurfaceDict&#xA;    {&#xA;        type            plane;    // always triangulated&#xA;        basePoint       (0   -1.5 0);&#xA;        normalVector    (0 -1 0);&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;does anyone know about planeEquationDict?&lt;/p&gt;&#xA;" OwnerUserId="6878" LastEditorUserId="6878" LastEditDate="2013-12-18T19:43:58.213" LastActivityDate="2014-07-31T15:49:00.673" Title="how to specify an arbitrary surface for field average in openfoam?" Tags="|fluid-dynamics|openfoam|error-estimation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10338" PostTypeId="2" ParentId="10303" CreationDate="2013-12-18T20:06:13.013" Score="1" Body="&lt;p&gt;Perhaps you can use an LQ factorization instead of the usual QR approach to solve the least squares problem. Say you already have $A = LQ$ where $L$ is lower-triangular and $Q$ is orthogonal. For some diagonal matrix $D$, we'll have that $DA = DLQ$, and the matrix $DL$ is still lower triangular; we didn't need to re-compute the factorization of $DA$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can edit my answer to elaborate if you need more detail.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2013-12-18T20:06:13.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10339" PostTypeId="2" ParentId="10187" CreationDate="2013-12-18T22:55:34.680" Score="0" Body="&lt;p&gt;I have worked on high precision fdm for the past n years. and I have used electrostatics -2 dim laplace's equation as the example for explicitly developing the high precision algorithms. until about 4 years ago the problems were constructed with horizontal or vertical lines points of potential discontinuity.  if you google my name and fdm high precision you should find the references.  but this is not your question. your question is fdm and curved boundaries. &#xA;about a year ago I presented an order 8 solution in hong kong(see &lt;em&gt;A Finite Difference Method for Cylindrically Symmetric Electrostatics having Curvilinear Boundaries&lt;/em&gt;) which created order 8 algorithms for interior points close to the boundary and these would require of course points on the other side of the boundary.  the points on the other side of the boundary were put there by simply extending the mesh to the other side.  having done this the question was how do you find the values of these points when relaxing the mesh.  it was accomplished by integrating from the boundary (known potential) to the point using the algorithms.  it was reasonably successful and reasonably accurate ~&amp;lt;1e-11, BUT required 103 algorithms each individually crafted  and it was somewhat brittle, unstable geometries could be found. &#xA;to remedy the above a solution has been found(order 8 and below) using (one!) minimal algorithm and the solution exhibits a considerable robustness.  it has been submitted but would be available as a preprint by emailing me.&#xA;I believe this technique would be extensible to time independent pde's(linear required) other than laplace and to dimensions higher than 2. I have not considered the time dependent problem but the technique being a power series technique should be adaptable and applicable.  david&lt;/p&gt;&#xA;" OwnerUserId="6880" LastEditorUserId="6880" LastEditDate="2013-12-22T14:39:45.350" LastActivityDate="2013-12-22T14:39:45.350" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10340" PostTypeId="1" CreationDate="2013-12-19T03:22:00.020" Score="3" ViewCount="4164" Body="&lt;p&gt;I am interested in solving a sparse complex linear system Ax=b where A is a square matrix of complex numbers and b is vector of complex numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If possible I would like such a library to be templated (for the ease of installation and use)&#xA;sth in the spirit of &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked out Eigen but it does not, I think,  look like it supports solving linear equations with complex sparse matrices, (although one can create and do elementary operations on complex matrices.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another trick someone suggested to me was one can work around this, by solving an extended system of twice the dimension using the fact that (A1 + iA2)(x1 + ix2) = (b1 + ib2)&#xA;but I would prefer some simple black box which gets the job done. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="3352" LastActivityDate="2013-12-19T11:32:21.967" Title="C++ Libraries for solving Complex Linear systems Ax=b" Tags="|linear-algebra|c++|" AnswerCount="3" CommentCount="0" ClosedDate="2014-01-07T17:05:57.243" ContentLicense="CC BY-SA 3.0" />
  <row Id="10341" PostTypeId="2" ParentId="10340" CreationDate="2013-12-19T06:27:34.523" Score="2" Body="&lt;p&gt;I think Eigen can do it, but also check out &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/umfpack/&quot; rel=&quot;nofollow&quot;&gt;UMFPACK&lt;/a&gt; which is written in C. Eigen has templated C++ &lt;a href=&quot;http://eigen.tuxfamily.org/dox/group__UmfPackSupport__Module.html&quot; rel=&quot;nofollow&quot;&gt;wrappers&lt;/a&gt; for UMFPACK. Also see the &lt;a href=&quot;http://www.netlib.org/linalg/html_templates/node4.html&quot; rel=&quot;nofollow&quot;&gt;book&lt;/a&gt; on templates for solving linear systems and the &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot; rel=&quot;nofollow&quot;&gt;survey&lt;/a&gt; of linear algebra libraries.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2013-12-19T06:27:34.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10342" PostTypeId="2" ParentId="10340" CreationDate="2013-12-19T06:39:33.457" Score="1" Body="&lt;p&gt;Elemental (written in C++11) and PETSc (written in C, interfaces with UMFPACK, Elemental, and other linear algebra packages) can both solve systems of linear equations over the field of complex numbers. PETSc is geared more towards sparse linear algebra and has some dense linear algebra capabilities as well.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-19T06:39:33.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10343" PostTypeId="2" ParentId="10331" CreationDate="2013-12-19T07:11:38.907" Score="3" Body="&lt;p&gt;I suggest to read the Quadpack book (Quadpack, Piessens, Robert; de Doncker-Kapenga, Elise; Überhuber, Christoph W.; Kahaner, David (1983). QUADPACK: A subroutine package for automatic integration. Springer-Verlag. ISBN 978-3-540-12553-2) and Pedro Gonnet's PhD thesis &quot;Adaptive Quadrature Re-revisited (available as pdf &lt;a href=&quot;http://e-collection.library.ethz.ch/eserv/eth:65/eth-65-02.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;). Pedro is a contributor to SciComp SE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What typically is done is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Priority_queue&quot; rel=&quot;nofollow&quot;&gt;priority queue&lt;/a&gt; data structure: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You apply Gauss-Kronrod on the whole initial interval and estimate the error&lt;/li&gt;&#xA;&lt;li&gt;You split the interval symmetrically in two and again apply Gauss-Kronrod on the intervals, estimate the errors and add a pair (or tuple) &quot;interval, error&quot; to the priority queue&lt;/li&gt;&#xA;&lt;li&gt;If the total error is smaller than the requested tolerance (or if you reached the maximal amount of subdivisions), you stop&lt;/li&gt;&#xA;&lt;li&gt;If not, you pop the top element from the queue (highest error), you split it in two and apply Gauss-Kronrod, add those two new elements to the priority queue and go back to 3.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="414" LastActivityDate="2013-12-19T07:11:38.907" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10344" PostTypeId="1" CreationDate="2013-12-19T07:46:55.000" Score="1" ViewCount="1386" Body="&lt;p&gt;I want to solve Lid driven cavity flow with the help of projection method i want to make own fortran program to solve two dimensional unsteady flow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I propose to solve the equations in the following way:  apply a second order Adam Bashforth method for convection terms and Crank nicloson scheme for viscous term by neglecting pressure. I also want to calculate intermediate velocity first. i need help to develop a code for this in fortran 90.&lt;/p&gt;&#xA;" OwnerUserId="6882" LastEditorUserId="549" LastEditDate="2013-12-26T14:05:12.233" LastActivityDate="2013-12-26T14:05:12.233" Title="Lid driven cavity flow problem to solve with the help of projection method and use coding in fortran 90" Tags="|fluid-dynamics|fortran|projection|" AnswerCount="1" CommentCount="3" ClosedDate="2013-12-25T19:14:36.723" ContentLicense="CC BY-SA 3.0" />
  <row Id="10345" PostTypeId="2" ParentId="10337" CreationDate="2013-12-19T08:41:41.220" Score="2" Body="&lt;p&gt;As far as I know, there is non such thing. But what you aim for should be fairly straight forward to implement, as long as you happen to have some experience with C++.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Locate the &lt;strong&gt;circleSet&lt;/strong&gt; in the OpenFOAM source directory.&lt;/li&gt;&#xA;&lt;li&gt;Make a local copy of it and rename it to something like &lt;strong&gt;faceSetSet&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Read the name of the &lt;strong&gt;faceSet&lt;/strong&gt; you'd like to sample on.&lt;/li&gt;&#xA;&lt;li&gt;Read the &lt;strong&gt;faceSet&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;In each timestep, loop only over those faces and average the fields you'd like.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is a very rudimentary and basic algorithm, you'd have to fill in the gaps. If you need more help, let me know.&lt;/p&gt;&#xA;" OwnerUserId="1960" LastActivityDate="2013-12-19T08:41:41.220" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10346" PostTypeId="2" ParentId="10344" CreationDate="2013-12-19T09:19:27.983" Score="0" Body="&lt;p&gt;In your case, since you want to code yourself, I would start with a paper that comprehensively describes discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your needs &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S002199910096546X&quot; rel=&quot;nofollow&quot;&gt;D. Kim, S. Choi - A Second-Order Time-Accurate Finite Volume Method for Unsteady Incompressible Flow on Hybrid Unstructured Grids, JCP, 162 (2000)&lt;/a&gt; would probably be the right thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There you will find time-integration description, interpolations are shown with nice illustrations, a few words on matrix solver, and finally lid driven cavity flow is solved, so you can compare your results with theirs and verify your code.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2013-12-19T09:19:27.983" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10347" PostTypeId="2" ParentId="2028" CreationDate="2013-12-19T10:13:35.893" Score="1" Body="&lt;p&gt;Beyond hwloc there are a few tools that can report on a HPC cluster's memory environment and which can be used to set a variety of NUMA configurations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend LIKWID as one such tool as it avoids a code based approach allowing you for instance to pin a process to a core. This approach of tooling to address machine specific memory configuration will help ensure the portability of your code across clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a short presentation outlining it from ISC'13 &quot;&lt;a href=&quot;http://wiki.likwid.googlecode.com/hg/BOF-slides.pdf&quot; rel=&quot;nofollow&quot;&gt;LIKWID - Lightweight Performance Tools&lt;/a&gt;&quot; and the authors have published a paper on Arxiv &quot;&lt;a href=&quot;http://arxiv.org/pdf/1206.3738.pdf&quot; rel=&quot;nofollow&quot;&gt;Best practices for HPM-assisted performance engineering on modern multicore processor&lt;/a&gt;&quot;. This paper describes an approach to interpreting the data from hardware counters to develop performant code specific to your machine's architecture and memory topology.&lt;/p&gt;&#xA;" OwnerUserId="6883" LastActivityDate="2013-12-19T10:13:35.893" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10348" PostTypeId="2" ParentId="10340" CreationDate="2013-12-19T11:32:21.967" Score="3" Body="&lt;p&gt;You can do it by transforming that system to equivalent real one and then use usual real solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a Trilinos package called Komplex which does precisely that, you may find it &lt;a href=&quot;http://trilinos.sandia.gov/packages/komplex/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also look for the paper: D. Day, M.A. Heroux - Solving Complex-Valued Linear Systems via Equiavalent Real Transformations, SIAM J. Sci. Comput, Vol.23, No.2, 2001.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2013-12-19T11:32:21.967" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10349" PostTypeId="1" AcceptedAnswerId="10350" CreationDate="2013-12-19T14:24:56.987" Score="3" ViewCount="701" Body="&lt;p&gt;I'm trying to get a deeper understanding of the Finite Element Method (FEM) to better understand what I do in COMSOL and FEniCS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sketch below shows the problem that I want to solve in FEM. The rectangular domain resembles a flow reactor. In the flow reactor there is a laminar velocity field in the z direction, $U_Z$. Particles of the diameter D are traveling along this flow field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The particles change in size according to the relation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d D}{dz} = \frac{1}{D U_Z} exp (\alpha / D)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do know the size of the particles at the left side of the domain, i.e. I have their initial values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would actually like to find is the size of the particles at the right side of the domain (labeled outlet).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ynruu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now this is the my question:&lt;/strong&gt;&#xA;I'm not sure if FEM is actually suitable to solve such a problem, which may be due to my (possibly wrong?) understanding of the method. My current understanding of FEM is that in setting up the matrix equation that solves the system, I need to specify all boundary conditions. Since in my case, I'm actually trying to find D (i.e. the particle diameters) on the right boundary of the domain, my problem seems to be ill-conditioned for FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've solved this problem in finite differences, where I just started on the left side of the domain and integrated a system of equations (corresponding to the above equation) in the z direction. This was much more straight-forward than FEM (it seems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know, if my understanding is correct or incorrect. I'd be more than happy to get references to read up on this.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2013-12-19T19:19:53.973" Title="Integrating an Ordinary Differential Equation over a domain in Finite Element Methods" Tags="|finite-element|fenics|comsol|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10350" PostTypeId="2" ParentId="10349" CreationDate="2013-12-19T17:06:17.387" Score="3" Body="&lt;p&gt;FEM is for solving boundary value problems. What you have here is an initial value problem (assuming $\alpha$ and $U_z$ are constant or functions of $z$), the same as solving a time-dependent ODE, except here $z$ is your time-like coordinate. The appropriate way to solve this kind of problem is with time-integration methods, e.g. Runge-Kutta schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In FEM applications where time-accuracy is required, the FEM aspect only handles the spatial discretization, while time-integration methods handle the time-dependent aspects -- they're distinct parts.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2013-12-19T17:06:17.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10351" PostTypeId="2" ParentId="8732" CreationDate="2013-12-19T17:37:12.677" Score="1" Body="&lt;p&gt;You might be trying to solve a system of equations that does not have a unique solution. Imagine you have a bunch of nodes connected by springs, floating in space, and you want to find the equilibrium position of each node. If the system is not anchored to something fixed (or no force is applied), there are many possible solutions. Any one solution can always be translated or rotated and it is still a solution. Have you tried fixing the displacements at one corner node to eliminate translation, and fixing one displacement at another corner to eliminate rotations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I once tried this approach of fixing some nodes and adjusting normal forces at others, but it seemed to focus large amounts of force at individual boundary nodes, resulting in instability. What ended up working was to not try to anchor just a few of nodes, but to anchor all of the nodes relative to a homogeneous strain. Essentially you strain the entire system homogeneously, but then include the homogeneous component in the local definition of strain at each node, so it does not contribute any additional elastic energy. You can read more about it in this paper and the cited references: &lt;a href=&quot;http://pubs.acs.org/doi/abs/10.1021/nn204177u&quot; rel=&quot;nofollow&quot;&gt;http://pubs.acs.org/doi/abs/10.1021/nn204177u&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This instability problem is probably a good reason to choose finite elements for mechanics problems when possible.&lt;/p&gt;&#xA;" OwnerUserId="6702" LastActivityDate="2013-12-19T17:37:12.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10352" PostTypeId="1" CreationDate="2013-12-19T17:52:39.973" Score="1" ViewCount="93" Body="&lt;p&gt;So I have a 2D histogram, in the third dimention are the counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to achieve is by choosing two random position, make a plot of a 1D histogram along the line that passes through the two points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also want to be able to change the distance from the line to which points are included in the 1D histogram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea what is the best way to do this?&lt;/p&gt;&#xA;" OwnerUserId="3382" LastActivityDate="2013-12-19T23:07:34.710" Title="Line scan in a 2D histogram" Tags="|algorithms|data-analysis|plotting|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10353" PostTypeId="2" ParentId="10349" CreationDate="2013-12-19T19:19:53.973" Score="4" Body="&lt;p&gt;You seem to be confused about which equation to solve. You have two: (i) the flow equations, (ii) the equations for your particle property $D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite element method is suitable for part 1 of this. Or, if you really just have a pipe with laminar flow, then you actually know what the flow field is (namely, Poisseuille flow) and you don't need to solve anything at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second part, you just need the ability to integrate along the trajectory of a particle. For Poisseuille flow, this is a straight line with constant velocity depending on the spanwise position of the line within the pipe, but in general you will need to be able to evaluate your flow field at arbitrary points. This is equally simple or difficult in the finite element or finite difference context, depending on whether you have a uniform or unstructured mesh. In any case, if you can evaluate the velocity field, then you know the trajectory of the particle and you can integrate your ODE along it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-19T19:19:53.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10354" PostTypeId="1" AcceptedAnswerId="10365" CreationDate="2013-12-19T20:18:02.840" Score="35" ViewCount="13864" Body="&lt;p&gt;How do CUDA and OpenCL compare to each other as of late 2013 from a programmer's perspective?  My group is thinking about trying to make use of GPU computing.  Would we be limiting ourself significantly by choosing hardware that only supports OpenCL but not CUDA?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be a bit more specific, are the following assumptions correct?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Everything that's possible in CUDA is also possible in OpenCL&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For as long as we're not using libraries, a given task is not significantly easier (or more difficult) to do in either of them&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;CUDA's main advantage is the availability of libraries&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Both have good support for all three main platforms (Win/OSX/Linux)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2013-12-19T23:15:40.110" LastActivityDate="2014-07-16T22:29:16.923" Title="CUDA vs OpenCL as of late 2013" Tags="|hpc|gpu|cuda|opencl|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="10355" PostTypeId="2" ParentId="10352" CreationDate="2013-12-19T20:35:19.520" Score="1" Body="&lt;p&gt;Sorry for the easy question but I looked for a long time for the proper way to do this and was unable to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now someone around here taught me how this is normally done and it is shamefully easy. Still I rather post the answer than delete the post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the solution is given the line, divide it so that the space between each point is the same as the space between bins in the original histogram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each point consider the 4 nearest neighbours and interpolate the value at the point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To integrate over a certain distance perpendicular to the line, create several lines parallel to the first up to the desired distance and with space between each line equal to the spacing between bins in the original histogram. Sum the lines.&#xA;I think this should maintain the total number of counts equal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have any opinions please share.&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3382" LastEditorUserId="3382" LastEditDate="2013-12-19T23:07:34.710" LastActivityDate="2013-12-19T23:07:34.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10356" PostTypeId="1" CreationDate="2013-12-19T23:26:43.487" Score="8" ViewCount="28813" Body="&lt;p&gt;I am using image filtering for an image processing algorithm I'm developing. I'm using a predefined Matlab function to do the convolution, but I'd like to know what the computational complexity is for this algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple way of thinking of this is that an $M \times N$ (grayscale) image and a $m \times n$ filter, each pixel needs $\mathcal{O}(mn)$ computations, so the 2D convolution would have an approximate complexity of $\mathcal{O}(MNmn)$. (Of course it would matter is you're padding the sides of the images or not, but assuming that $m$ and $n$ are small compared to $M$ and $N$, it shouldn't make too much of a difference.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questionis : Is this true? Is the complexity of 2D convolution $\mathcal{O}(MNmn)$, or are there optimizations that make it less?&lt;/p&gt;&#xA;" OwnerUserId="3892" LastActivityDate="2022-11-11T23:53:51.570" Title="Computational Complexity of 2D Convolution" Tags="|complexity|image-processing|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10357" PostTypeId="1" AcceptedAnswerId="10358" CreationDate="2013-12-19T23:27:11.863" Score="4" ViewCount="458" Body="&lt;p&gt;I apologize in advances if this is the wrong place to be asking this question, (I had considered putting it on CV first). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am studying financial (S&amp;amp;P, Dow Jones, etc) data, and would like to compute some quantiles of an array of numbers. I have been reading about quantiles, so I know what they are, but does the computation of quantiles ALWAYS mean that I need to sort my data first? Or are there methods where quantiles can be computed without first sorting your array of numbers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="6891" LastActivityDate="2013-12-19T23:55:17.880" Title="Is it possible to compute quantiles of a set of numbers, without first sorting those numbers?" Tags="|linear-algebra|statistics|sorting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10358" PostTypeId="2" ParentId="10357" CreationDate="2013-12-19T23:55:17.880" Score="3" Body="&lt;p&gt;You don't need to sort your data first, if you have access to additional temporary storage, in which case, you should use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Selection_algorithm&quot; rel=&quot;nofollow&quot;&gt;selection algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-19T23:55:17.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10359" PostTypeId="1" AcceptedAnswerId="10360" CreationDate="2013-12-20T06:17:27.803" Score="2" ViewCount="83" Body="&lt;p&gt;For a positive definite symmetric linear system, Cholesky decomposition based method should be the best solver which has a rough &lt;code&gt;n^3/3&lt;/code&gt; flops requirement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the fomula of flops including &lt;code&gt;n^2&lt;/code&gt;, &lt;code&gt;n&lt;/code&gt; items? Is there any such reference?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2013-12-20T06:34:25.740" Title="What is the fomula of polynomial time of solving positive definite symmetric linear system" Tags="|linear-solver|efficiency|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10360" PostTypeId="2" ParentId="10359" CreationDate="2013-12-20T06:34:25.740" Score="3" Body="&lt;p&gt;From &lt;em&gt;Numerical Linear Algebra&lt;/em&gt; by Trefethen and Bau, page 175, it seems that the formula is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\sum_{k=1}^{n}\sum_{j = k + 1}^{n} (2(n - j + 1) + 1).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eyeballing it, it seems to agree with the formula given by Boyd in his &lt;a href=&quot;http://www.ee.ucla.edu/ee236b/lectures/num-lin-alg.pdf&quot; rel=&quot;nofollow&quot;&gt;convex optimization notes&lt;/a&gt;: $(1/3)n^{3} + 2n^{2}$.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-20T06:34:25.740" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10361" PostTypeId="2" ParentId="10356" CreationDate="2013-12-20T07:29:47.073" Score="7" Body="&lt;p&gt;A convolution becomes a product if you apply it in Fourier space, so you can express the operation $I * m$ ($I$ being the image, $m$ being the mask with which you filter) as $I * m = F^{-1}(F(I) \cdot F(m))$ (where $F$ is the Fourier transform) which will cost you $O(MN \log (MN))$ operations. Whether this is cheaper than what you have depends on the size of your filter mask: For small filters, a direct application is cheaper, but if the filter applies the mask over an area that is a significant subset of the image, then the Fourier transform may become cheaper.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-20T07:29:47.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10362" PostTypeId="1" AcceptedAnswerId="10366" CreationDate="2013-12-20T09:36:23.020" Score="2" ViewCount="1569" Body="&lt;p&gt;In a problem I am trying to model with a MIP program, the following scenario occurs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am given binary variables $x_1,\ldots,x_n$ and $y_1,\ldots,y_n$ which can really be regarded as $n$-vectors. What I need to compute is the number of bits in which they differ. In particular I just need to make sure they are only equal in at most one bit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done by adding the following quadratic constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{i=1}^n x_i y_i \leq 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the very specific nature of this condition, I am wondering &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a better way to express this? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a trick to convert this expression into a linear one?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4582" LastActivityDate="2018-05-20T22:22:19.550" Title="Modeling a quadratic constraint with a linear expression" Tags="|optimization|linear-programming|quadratic-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10363" PostTypeId="2" ParentId="10356" CreationDate="2013-12-20T09:39:40.823" Score="9" Body="&lt;p&gt;There are a lot of ways to speed up convolution is specialized contexts, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If your filter is separable, i.e. $h = h_1 * h_2$ with $h_1\in\mathbb{R}^{n\times 1}$ and $h_2\in\mathbb{R}^{1\times m}$, calculating $(u*h_1)*h_2$ is much faster than $u*(h_1*h_2)$.&lt;/li&gt;&#xA;&lt;li&gt;Convolving with $h = [1\ \dots\ 1]$ (for any length) can be done recursively with basically two operations per pixel. Combining this with the previous point gives a huge speedup for a rectangular moving average.&lt;/li&gt;&#xA;&lt;li&gt;Sometimes factoring filters is beneficial: E.g. convolving with $[\tfrac14\ \tfrac12\ \tfrac14]$ can be done by convolving with $[1\ 1]$ twice and then dividing by 4. The latter convolution only need additions (no multiplications) and a bit shift and that may be much faster in certain environments.&lt;/li&gt;&#xA;&lt;li&gt;Combining the previous point with the first one, one gets good approximations to convolution with Gaussians by convolving $n$ times with $[1\ 1]$ and $n$ times with $[1\ 1]^T$ (with the appropriate bit shifts).&lt;/li&gt;&#xA;&lt;li&gt;When using the FFT (as Wolfgang Bangerth mentioned) for the convolution of a large image with a small filter, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Overlap%E2%80%93add_method&quot;&gt;overlap add method&lt;/a&gt; further improves speed.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, how much speedup is actually observed in practice depends a lot on the specific architecture and language .&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2013-12-20T09:39:40.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10365" PostTypeId="2" ParentId="10354" CreationDate="2013-12-20T10:27:05.197" Score="37" Body="&lt;p&gt;I'll try to summarize my experiences obtained in the course of developing ViennaCL, where we have CUDA and OpenCL backends with mostly 1:1 translations of a lot of compute kernels. From your question I'll also assume that we are mostly taking about GPUs here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Performance Portability.&lt;/strong&gt; First of all, there is no such thing as performance-portable kernels in the sense that you write a kernel once and it will run efficiently on every hardware. Not in OpenCL, where it is more apparent due to the broader range of hardware supported, but also not in CUDA. In CUDA it is less apparent because of the smaller range of hardware supported, but even here we have to distinguish at least three hardware architectures (pre-Fermi, Fermi, Kepler) already. These performance fluctuations can easily result in a 20 percent performance variation depending on how you orchestrate threads and which work group sizes you choose, even if the kernel is as simple as a buffer copy. It's probably also worth mentioning that on pre-Fermi and Fermi GPUs it was possible to write fast matrix-matrix multiplication kernels directly in CUDA, while for the latest Kepler GPUs it seems that one has to go down to the PTX pseudo-assembly language in order to get close to CUBLAS' performance. Thus, even a vendor-controlled language such as CUDA appears to have issues to keep the pace with hardware developments. Moreover, all CUDA code gets compiled statically when you run nvcc, which somewhat requires a balancing act via the -arch flag, while OpenCL kernels get compiled at run-time from the just-in-time compiler, so you can in principle tailor kernels down to the very specifics of a particular compute device. The latter is, however, quite involved and usually only becomes a very attractive option as your code matures and as your experience accumulates. The price to pay is the O(1) time required for just-in-time compilation, which can be an issue in certain situations. OpenCL 2.0 has some great improvements to address this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Debugging and Profiling.&lt;/strong&gt; The CUDA debugging and profiling tools are the best available for GPGPU. AMD's tools are not bad either, but they do not include gems like cuda-gdb or cuda-memcheck. Also, still today NVIDIA provides the most robust drivers and SDKs for GPGPU, system freezes due to buggy kernels are really the exception, not the rule, both with OpenCL and CUDA. For reasons I probably do not need to explain here, NVIDIA no longer offers debugging and profiling for OpenCL with CUDA 5.0 and above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Accessibility and Convenience.&lt;/strong&gt; It is a lot easier to get the first CUDA codes up and running, particularly since CUDA code integrates rather nicely with host code. (I'll discuss the price to pay later.) There are plenty of tutorials out there on the web as well as optimization guides and some libraries. With OpenCL you have to go through quite a bit of initialization code and write your kernels in strings, so you only find compilation errors during execution when feeding the sources to the jit-compiler. Thus, it takes longer to go through one code/compile/debug cycle with OpenCL, so your productivity is usually lower during this initial development stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Software Library Aspects.&lt;/strong&gt; While the previous items were in favor of CUDA, the integration into other software is a big plus for OpenCL. You can use OpenCL by just linking with the shared OpenCL library and that's it, while with CUDA you are required to have the whole CUDA toolchain available. Even worse, you need to use the correct host compilers for nvcc to work. If you ever tried to use e.g. CUDA 4.2 with GCC 4.6 or newer, you'll have a hard time getting things to work. Generally, if you happen to have any compiler in use which is newer than the CUDA SDK, troubles are likely to occur. Integration into build systems like CMake is another source of headache (you can also find ample of evidence on e.g. the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot;&gt;PETSc&lt;/a&gt; mailinglists). This may not be an issue on your own machine where you have full control, but as soon as you distribute your code you will run into situations where users are somewhat restricted in their software stack. In other words, with CUDA you are no longer free to choose your favourite host compiler, but NVIDIA dictates which compilers you are allowed to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other Aspects.&lt;/strong&gt; CUDA is a little closer to hardware (e.g. warps), but my experience with linear algebra is that you rarely get a significant benefit from it. There are a few more software libraries out there for CUDA, but more and more libraries use multiple compute backends. &lt;a href=&quot;http://viennacl.sourceforge.net/&quot;&gt;ViennaCL&lt;/a&gt;, &lt;a href=&quot;https://github.com/ddemidov/vexcl&quot;&gt;VexCL&lt;/a&gt;, or &lt;a href=&quot;http://www.paralution.com/&quot;&gt;Paralution&lt;/a&gt; all support OpenCL and CUDA backends in the meanwhile, a similar trend can be seen with libraries in other areas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;GPGPU is not a Silver Bullet.&lt;/strong&gt; GPGPU has been shown to provide good performance for structured operations and compute-limited tasks. However, for algorithms with a non-negligible share of sequential processing, GPGPU cannot magically overcome &lt;a href=&quot;http://en.wikipedia.org/wiki/Amdahl%27s_law&quot;&gt;Amdahl's Law&lt;/a&gt;. In such situations you are better off using a good CPU implementation of the best algorithm available rather than trying to throw a parallel, but less suitable algorithm at your problem. Also, PCI-Express is a serious bottleneck, so you need to check in advance whether the savings from GPUs can compensate the overhead of moving data back and forth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Recommendation.&lt;/strong&gt; Please consider CUDA &lt;em&gt;and&lt;/em&gt; OpenCL rather than CUDA &lt;em&gt;or&lt;/em&gt; OpenCL. There is no need to unnecessarily restrict yourself to one platform, but instead take the best out of both worlds. What works well for me is to set up an initial implementation in CUDA, debug it, profile it, and then port it over to OpenCL by simple string substitutions.( You may even parametrize your OpenCL kernel string generation routines such that you have some flexibility in tuning to the target hardware.) This porting effort will usually consume less than 10 percent of your time, but gives you the ability to run on other hardware as well. You may be surprised about how well non-NVIDIA hardware can perform in certain situations. Most of all, consider the reuse of functionality in libraries to the largest extent possible. While a quick&amp;amp;dirty reimplementation of some functionality often works acceptable for single-threaded execution on a CPU, it will often give you poor performance on massively parallel hardware. Ideally you can even offload everything to libraries and don't ever have to care about whether they use CUDA, OpenCL, or both internally. Personally I would never dare to write vendor-locked code for something I want to rely on in several years from now, but this ideological aspect is should go into a separate discussion.&lt;/p&gt;&#xA;" OwnerUserId="966" LastActivityDate="2013-12-20T10:27:05.197" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="10366" PostTypeId="2" ParentId="10362" CreationDate="2013-12-20T11:51:15.530" Score="3" Body="&lt;p&gt;Intropduce a new variable $z_i$ to represent $x_iy_i$ (logical and). Your constraint is $\sum_{i=1}^n z_i \leq 1$, and the product is modelled by $x_i + y_i -1 \leq z_i$.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-12-20T11:51:15.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10367" PostTypeId="1" AcceptedAnswerId="10368" CreationDate="2013-12-20T14:57:41.363" Score="2" ViewCount="104" Body="&lt;p&gt;Is there any analytical (exact, closed-form solution) or numerical method to solve an equation such as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p(x) = r^x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $p(x)$ is a polynomial whose coefficients are drawn from a finite field, and $r$ is a primitive root of the field modulus ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, is this problem known to be equivalent to solving the discrete logarithm problem ?&#xA;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="3533" LastActivityDate="2013-12-20T17:41:34.070" Title="Methods to solve this equation on finite fields?" Tags="|reference-request|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10368" PostTypeId="2" ParentId="10367" CreationDate="2013-12-20T16:33:20.637" Score="2" Body="&lt;p&gt;If the finite field $\mathbb{F}$ has a number of elements that is small enough, you can obtain the Lagrange interpolation polynomial $f \in \mathbb{F}[x]$ such that $f(x_i) = r^{x_i}$ for all elements $x_i \in \mathbb{F}$. This polynomial is &lt;em&gt;identical&lt;/em&gt; to $r^{x}$ precisely because we are working on a finite field.  Then, the problem is reduced to finding the roots of the polynomial $g = p - f \in \mathbb{F}[x]$.  A brute-force approach consisting of trying every possible element in the field would work.  Another possibility would be to factor $g$ using &lt;a href=&quot;http://en.wikipedia.org/wiki/Berlekamp%27s_algorithm&quot; rel=&quot;nofollow&quot;&gt;Berlekamp's algorithm&lt;/a&gt; and read the roots off the factors.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find further details on these types of algorithms (such as computational complexity, etc) in the computer algebra literature (see, for instance, &lt;a href=&quot;https://cosec.bit.uni-bonn.de/science/mca/&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;http://books.google.com/books/about/Polynomial_Algorithms_in_Computer_Algebr.html?id=rqhhlV046kMC&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2013-12-20T17:41:34.070" LastActivityDate="2013-12-20T17:41:34.070" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10369" PostTypeId="1" AcceptedAnswerId="10377" CreationDate="2013-12-20T16:54:08.903" Score="2" ViewCount="668" Body="&lt;p&gt;For a second order PDE, for example heat conduction equation &#xA;$\frac{\partial T}{\partial t} = \frac{\alpha}{C_p} \nabla^2 T$, is it possible to determine the steady-state (or even transient) solution with two Dirichlet conditions? I have two different questions regarding this&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;From my understanding, the solution is non unique for all equal valued ratios of $\alpha$ and $C_p$. so two Dirichlet conditions say nothing about how fast the disturbance propagates with a temporal change of one boundary condition. So only the knowledge of $T$ and $\nabla T$ together can fix the solution curve for specific values of $\alpha$ and $C_p$ instead of the ratio. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Integrating the 1-D second order (steady-state) equation gives &#xA;$T=C_1x+C_2$ where $C_1 = \frac{\partial T}{\partial x}$. So, two Dirichlet conditions are two values of $C_2$ and therefore still don't give us the value of $C_1$ which is required to fix the solution curve. So in this case, how is it possible to assume we know the solution with two just Dirichlet conditions?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5096" LastActivityDate="2013-12-21T22:26:52.447" Title="Boundary conditions for second order PDE" Tags="|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10370" PostTypeId="1" CreationDate="2013-12-20T17:01:14.670" Score="6" ViewCount="610" Body="&lt;p&gt;Quoting from Solenthaler et. al. &lt;em&gt;Predictive-Corrective Incompressible SPH&#xA; (ACM Transactions on Graphics, Vol. 28, No. 3, Article 40, Publication date: August 2009)&lt;/em&gt; (PDF &lt;a href=&quot;https://sph-sjtu-f06.googlecode.com/files/a40-solenthaler.pdf&quot;&gt;link here&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;These incompressible SPH (ISPH) methods first integrate the velocity field in time without enforcing incompressibility. Then, either the intermediate velocity field, the resulting variation in particle density, or both are projected onto a divergence-free space to satisfy incompressibility through a pressure Poisson equation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does the author mean by &quot;project?? Is there a simpler way of understanding this operation? I tried reading other articles, but they go even more deep saying solenoidal vectors etc. I am looking for a simple explanation at first to understand the concept.&lt;/p&gt;&#xA;" OwnerUserId="5096" LastEditorUserId="276" LastEditDate="2017-10-10T00:48:09.993" LastActivityDate="2017-10-10T03:36:13.233" Title="What are projection methods" Tags="|linear-algebra|particle|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10371" PostTypeId="2" ParentId="10370" CreationDate="2013-12-20T20:49:43.943" Score="8" Body="&lt;p&gt;Projection methods typically split up the solution of transient Stokes or Navier-Stokes methods into the solution of two separate problems - one for the velocity, one for the pressure. For Stokes, the simplest version (by Chorin/Temam) goes as follows for a timestep $k$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute an intermediate velocity $u^*$ by solving $$\frac{(u^*-u^k)}{dt} - \nu\Delta u^* = f(t^k)$$&lt;/li&gt;&#xA;&lt;li&gt;Correct the velocity by solving &#xA;$$\begin{cases}&#xA;\frac{(u^{k+1}-u^*)}{dt} + \nabla p^k &amp;amp;= 0\\&#xA;\nabla \cdot u^{k+1} &amp;amp;= 0\\&#xA;u^{k+1}\cdot n &amp;amp;= 0 \quad \text{on the boundary}&#xA;\end{cases}$$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This above step is the reason these splitting schemes are referred to as projection methods - the weak form of (2) can be interpreted as the scaled $L^2$ projection of $u^*$ onto a divergence-free $u^{k+1}$ (after multiplying by a test function, $p$ can then be viewed as a Lagrange multiplier to enforce the divergence free constraint). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A note: since pressure is undetermined in the second step, we need to determine it.  Based on the assumption that $\nabla \cdot u^{k+1}$ must be $0$, we can take the divergence of the second equation to get $$\nabla \cdot \frac{u^*}{dt} + \nabla\cdot \nabla p^k = 0,$$ which yields a Poisson equation for $p$ given $u^*$. The overall scheme is then &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve for $u^*$ through $$\frac{(u^*-u^k)}{dt} - \nu\Delta u^* = f(t^k)$$&lt;/li&gt;&#xA;&lt;li&gt;Solve for $p$ with $$\Delta p = \frac{1}{dt}\nabla \cdot u^*$$&lt;/li&gt;&#xA;&lt;li&gt;Solve for $u^{k+1}$ through $$\frac{(u^{k+1}-u^*)}{dt} + \nabla p^k = 0.$$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://www.math.ust.hk/~mawang/teaching/math532/guermond-shen-2006.pdf&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt; for a great overview of available projection methods. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2013-12-21T08:32:03.947" LastActivityDate="2013-12-21T08:32:03.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10372" PostTypeId="2" ParentId="10370" CreationDate="2013-12-21T05:45:31.483" Score="2" Body="&lt;p&gt;If you're confused why this operation is called &quot;projection&quot;, note that the equation &#xA;$$&#xA;  \text{div} \, \mathbf u = 0&#xA;$$&#xA;means that we only consider velocity fields that are divergence free. This is no different than asking to find a solution of some problem subject to a constraint of the form $Bx=0$ where $B$ is a matrix with fewer rows than columns. The set of all vectors $x$ that satisfy this form a subspace (think of a plane in 3-space, or a hyperplane in $n$-space). What projection algorithms do is to find some approximation for a simpler subproblem, and then &quot;project&quot; back to the hyperplane of functions that are divergence free, where the projection is really just to be considered what one does in regular, finite dimensional algebra or geometry.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="27" LastEditDate="2013-12-21T15:50:27.793" LastActivityDate="2013-12-21T15:50:27.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10373" PostTypeId="2" ParentId="10370" CreationDate="2013-12-21T20:15:55.383" Score="4" Body="&lt;p&gt;To give an easier example, consider the ODE for rotation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot x=-ay\\ \dot y=ax&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one solves that with the Euler method, the next point is found in the direction of the tangent of the circle, increasing the radius. &#xA;\begin{align}&#xA;x(t+h)^2+y(t+h)^2&amp;amp;=(x(t)-ahy(t))^2+(y(t)+ahx(t))^2\\&#xA;&amp;amp;=(1+a^2h^2)(x(t)^2+y(t)^2)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since one knows that the exact solution of this ODE preserves the initial radius&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x(t)^2+y(t)^2=r^2=x(0)^2+y(0)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one can correct this drifting away by projecting the numerical solution back to that circle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\tilde x(t+h),\tilde y(t+h))&#xA;=\frac{r}{\sqrt{x(t+h)^2+y(t+h)^2}}(x(t+h),y(t+h))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The angular velocity will still be different, but the approximation will still be better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you have a system of first integrals for the problem, you may want to return, after a certain number of steps, to the set satisfying the first integrals. Using Newton's method with the Moore-Penrose pseudo-inverse will usually find a rather short path to that set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the closest point on some set usually is called &quot;projection&quot;. This is conform to the characteristic of orthogonal projections in linear algebra.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2013-12-21T20:15:55.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10374" PostTypeId="1" CreationDate="2013-12-21T20:34:05.187" Score="0" ViewCount="743" Body="&lt;p&gt;I have few particle's position and velocity information as a function of time.&#xA;Particle is following a trajectory path, while doing so, one particle may disappear, other particle may appear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Particle position can be represented as a point and velocity as a vector at that point. If we think of particle position, it is nothing but a point in the flow path or pipe or channel &#xA;(I lack that information or I dont know about it), so I want to reverse engineer point position as kind of geometry position and map or interpolate velocity contour on those points, and see how flow is changing with time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dont want to know how to visualize using Matlab or Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;is there any example or link in Matlab or Python that has reversed engineer Lagrangian particle tracking method.&lt;/p&gt;&#xA;" OwnerUserId="6901" LastEditorUserId="6901" LastEditDate="2013-12-25T22:39:21.580" LastActivityDate="2017-09-23T08:23:32.297" Title="how to visualize velocity from Lagrangian particle tracking method" Tags="|fluid-dynamics|numerical-analysis|molecular-dynamics|microfluidics|" AnswerCount="0" CommentCount="9" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10375" PostTypeId="1" AcceptedAnswerId="10376" CreationDate="2013-12-21T21:11:43.750" Score="18" ViewCount="9737" Body="&lt;p&gt;A common problem in statistics is computing the square root inverse of a symmetric positive definite matrix. What would be the most efficient way of computing this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across &lt;a href=&quot;http://link.springer.com/article/10.1007/BF02257775&quot;&gt;some literature&lt;/a&gt; (which I haven't read yet), and some incidental R code &lt;a href=&quot;http://www4.stat.ncsu.edu/~li/software/SparseSDR.R&quot;&gt;here&lt;/a&gt;, which I will reproduce here for convenience&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# function to compute the inverse square root of a matrix&#xA;fnMatSqrtInverse = function(mA) {&#xA;  ei = eigen(mA)&#xA;  d = ei$values&#xA;      d = (d+abs(d))/2&#xA;      d2 = 1/sqrt(d)&#xA;      d2[d == 0] = 0&#xA;      return(ei$vectors %*% diag(d2) %*% t(ei$vectors))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not entirely sure I understand the line &lt;code&gt;d = (d+abs(d))/2&lt;/code&gt;. Is there a more efficient way of computing the matrix square root inverse? &lt;a href=&quot;http://stat.ethz.ch/R-manual/R-devel/library/base/html/eigen.html&quot;&gt;The R &lt;code&gt;eigen&lt;/code&gt; function calls LAPACK&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1969" LastActivityDate="2015-08-07T17:36:23.007" Title="Efficient computation of the matrix square root inverse" Tags="|linear-algebra|numerical-analysis|matrix|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10376" PostTypeId="2" ParentId="10375" CreationDate="2013-12-21T22:18:25.023" Score="12" Body="&lt;p&gt;The code that you've posted uses the eigenvalue decomposition of the symmetric matrix to compute $A^{-1/2}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statement &lt;/p&gt;&#xA;&#xA;&lt;p&gt;d=(d+abs(d))/2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;effectively takes any negative entry in d and sets it to 0, while leaving non-negative entries alone.  That is, any negative eigenvalue of $A$ is treated as though it was 0.  In theory, the eigenvalues of A should all be non-negative, but in practice it's common to see small negative eigenvalues when you compute the eigenvalues of a supposedly positive definite covariance matrix that is nearly singular.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really need the inverse of the symmetric matrix square root of $A$, and $A$ is reasonably small (no bigger than say 1,000 by 1,000), then this is about as good as any method you might use.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many cases you can instead use a Cholesky factor of the inverse of the covariance matrix (or practically the same, the Cholesky factor of the covariance matrix itself.)  Computing the Cholesky factor is typically an order of magnitude faster than computing the eigenvalue decomposition for dense matrices and vastly more efficient (both in computational time and required storage) for large and sparse matrices.  Thus using the Cholesky factorization becomes very desirable when $A$ is large and sparse.         &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2013-12-21T22:18:25.023" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10377" PostTypeId="2" ParentId="10369" CreationDate="2013-12-21T22:19:27.860" Score="1" Body="&lt;p&gt;Before I begin answering your question, I just need to clarify one key point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;On the notion of &quot;Uniqueness&quot;&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way you use the word &quot;unique&quot; in your question is not correct. &quot;Uniqueness&quot; has a very precise meaning in a mathematical context and is very different from the way you are using it.  When $\alpha$ and $C_p$ are of equal ratio, they produce the same solution curve.  However, this does not imply that the solution of the PDE is not unique.  Uniqueness refers to whether two completely different functions solve the same PDE with the same data (i.e. data = coefficients and initial/boundary conditions).   It is more appropriate to say that the problems with the same ratio $\frac{\alpha}{C_p}$ are equivalent problems and must have the same  solution, if one exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please keep this in mind as I answer your questions more directly below:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Both the steady state and transient PDE's are &lt;em&gt;well posed&lt;/em&gt; (under sufficient assumptions on the coefficients &amp;amp; initial/boundary conditions) for both &lt;em&gt;pure dirichlet&lt;/em&gt; and mixed boundary conditions.  Thus, a solution exists, is unique, and depends continuously on the data.  Therefore, you can &lt;strong&gt;uniquely&lt;/strong&gt; determine the steady state solution using only dirichlet boundary conditions.  The transient solution can also be uniquely determined from a combination of the dirichlet boundary conditions &lt;strong&gt;and&lt;/strong&gt; the initial condition.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can determine the coefficients of the analytical solution (1D case) by setting up a system of equations.  Suppose the boundary conditions of the PDE at $x_1$ and $x_2$ are given as $g(x_1)=g_1$ and $g(x_2)=g_2$, respectively.  Then, the coefficients $C_1$ and $C_2$ are uniquely determined by the system of equations &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;\begin{align}C_1x_1 + C_2 = g_1\\ C_1x_2+C_2=g_2\end{align}&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2013-12-21T22:26:52.447" LastActivityDate="2013-12-21T22:26:52.447" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10378" PostTypeId="1" CreationDate="2013-12-21T23:58:11.967" Score="1" ViewCount="2072" Body="&lt;p&gt;I have a &lt;code&gt;numpy&lt;/code&gt; function &lt;code&gt;f&lt;/code&gt; that takes arrays as arguments and a 3D array &lt;code&gt;x[a,b,c]&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to evaluate the function &lt;code&gt;f&lt;/code&gt; along a specific column.  A long-winded way could be with comprehensions:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;y = [ [ f(x[a][b])  for a in range(len(x)) ] for b in range(len(x[0]))]&#xA;y = np.array(y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a numpy way of doing this with broadcasting?&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="119" LastEditDate="2013-12-22T06:35:50.227" LastActivityDate="2013-12-25T03:15:18.470" Title="evaluating a function along an axis in numpy" Tags="|numpy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10379" PostTypeId="1" AcceptedAnswerId="10401" CreationDate="2013-12-22T02:10:38.747" Score="2" ViewCount="247" Body="&lt;p&gt;It is known that Numpy basic matrix slicing will generate a view, whereas advanced slicing a copy. Is this true in cvxopt? I tried&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from cvxopt import spmatrix&#xA;import numpy as np&#xA;A = spmatrix([2,-1,2,-2,1,4,3], [1,2,0,2,3,2,0], [0,0,1,1,2,3,4])&#xA;A_view = A[0:3,0:3]           # basic slicing&#xA;A_cp = A[[0,1,2],[0,1,2]]     # advanced slicing&#xA;np.may_share_memory(A,A_view) # returned false&#xA;np.may_share_memory(A,A_cp)   # returned false&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6903" LastActivityDate="2013-12-24T19:22:35.907" Title="Is slicing matrix a view or copy in cvxopt?" Tags="|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10380" PostTypeId="2" ParentId="5238" CreationDate="2013-12-22T03:25:43.153" Score="1" Body="&lt;p&gt;I'll give an update based on what I've done in the time since asking this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A First Course in Computational Physics&lt;/strong&gt; - DeVries. This covers many fundamental topics (Root finding/interpolation/integration [incl. Monte Carlo methods], linear algebra methods, ODEs, PDEs, and Fourier analysis) in a conversational tone so that everything seems motivated, and was very interesting for starting out in numerical methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Computational Physics - An Introduction&lt;/strong&gt; - Vesely. I've found that the bulk of this book covers the same things the DeVries book covers, but it does so a bit more formally and uses more powerful and general (specialized?) tools. There are too few practice problems, in my opinion. It also has a chapter on hydrodynamics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An Introduction to Scientific Computing&lt;/strong&gt; - Danaila. The projects in this are excellent and definitely complement the material in the other two books I mentioned. It discusses aspects of each problem more in depth and provides challenges. (meaning that the tools in DeVries and Vesely don't just trivially solve each problem)&lt;/p&gt;&#xA;" OwnerDisplayName="user121" LastActivityDate="2013-12-22T03:25:43.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10382" PostTypeId="2" ParentId="10378" CreationDate="2013-12-22T06:59:11.773" Score="0" Body="&lt;p&gt;This is a simpler comprehension:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;numpy.array([[f(row) for row in layer] for layer in x]).T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or more concisely,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;numpy.array([map(f,layer) for layer in x]).T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that I reproduced the transpose from your sample code.  You can avoid the nested comprehension with cryptic reshaping:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;numpy.reshape(map(f,x.reshape(numpy.prod(x.shape[:2]), x.shape[2])),x.shape[:2]).T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="119" LastActivityDate="2013-12-22T06:59:11.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10383" PostTypeId="1" AcceptedAnswerId="10385" CreationDate="2013-12-22T07:59:38.187" Score="7" ViewCount="373" Body="&lt;p&gt;I know that for general matrices, Cramer's rule is far from ideal for the numerical computation of the matrix inverse. However, can it be outperformed in the case of a $3 \times 3$ matrix? One particular advantage is that the algorithm is non-branching. For example, the transpose of the inverse can be implemented as follows, assuming the matrix $\bf{a}$ is invertible and the matrices $\bf{a}$ and $\bf{b}$ are stored as contiguous arrays.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void inverse3(double* a, double* b) {&#xA;    // cofactors&#xA;    b[0] = a[4]*a[8] - a[5]*a[7];&#xA;    b[1] = a[5]*a[6] - a[3]*a[8];&#xA;    b[2] = a[3]*a[7] - a[4]*a[6];&#xA;    b[3] = a[7]*a[2] - a[8]*a[1];&#xA;    b[4] = a[8]*a[0] - a[6]*a[2];&#xA;    b[5] = a[6]*a[1] - a[7]*a[0];&#xA;    b[6] = a[1]*a[5] - a[2]*a[4];&#xA;    b[7] = a[2]*a[3] - a[0]*a[5];&#xA;    b[8] = a[0]*a[4] - a[1]*a[3];&#xA;    // determinant&#xA;    double det = b[0]*a[0] + b[1]*a[1] + b[2]*a[2];&#xA;    // inverse&#xA;    b[0] /= det;&#xA;    b[1] /= det;&#xA;    b[2] /= det;&#xA;    b[3] /= det;&#xA;    b[4] /= det;&#xA;    b[5] /= det;&#xA;    b[6] /= det;&#xA;    b[7] /= det;&#xA;    b[8] /= det;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can this be improved in terms of efficiency or numerical robustness?&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="348" LastEditDate="2013-12-22T14:33:51.557" LastActivityDate="2013-12-28T16:38:33.700" Title="Can one outperform Cramer's rule for the inversion of a 3 by 3 matrix" Tags="|linear-algebra|algorithms|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10384" PostTypeId="1" AcceptedAnswerId="11617" CreationDate="2013-12-22T15:42:04.247" Score="2" ViewCount="2512" Body="&lt;p&gt;I successfully ran a RANS simulation of industrial fan with all wall BCs,&#xA;pimpleDyMFoam solver and spallart-almaras RAS model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got results upto some time, I would like to continue that simulation with LES instead of RAS, what are the options how I need to refine the mesh. (one parameter is way off the experimental value)&#xA;&lt;em&gt;specifically&lt;/em&gt; I would like to continue the simulation from where RAS ended say from 100s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance if I am using one equation eddy model, how to accommodate nuSgs for the solution with RAS?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;quick help would be appreciated&lt;/p&gt;&#xA;" OwnerUserId="6878" LastActivityDate="2020-04-12T07:42:38.007" Title="openFOAM RAS to LES" Tags="|fluid-dynamics|openfoam|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10385" PostTypeId="2" ParentId="10383" CreationDate="2013-12-22T16:20:30.760" Score="8" Body="&lt;p&gt;I don't know if another factorization would be faster overall, but you can (almost) certainly speed the above code up by inverting the determinant once and multiplying by the inverse 9 times rather than performing the division over and over. It's possible the compiler will figure this out for you, but not guaranteed (look at the assembly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edited to add: For the record, a couple of attempts with the Intel 13 compiler give different results depending on what it knows about the divisor. If it comes from command line input, then the compiler dutifully divides each one (vectorized on my Sandy Bridge machine).  &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2013-12-22T19:35:41.767" LastActivityDate="2013-12-22T19:35:41.767" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10386" PostTypeId="1" AcceptedAnswerId="10387" CreationDate="2013-12-23T01:14:33.950" Score="1" ViewCount="355" Body="&lt;p&gt;I need to determine whether a real dense symmetric matrix is positive definite or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible way is to obtain all the eigen values and check the sign of the minimum eigen value but requires computing all eigen values which seems unnecessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was said &lt;code&gt;congruent transform&lt;/code&gt; should be more efficient to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does C++ Eigen template library have such &lt;code&gt;congruent transform&lt;/code&gt; implementation or efficient way to immediately determine whether a symmetric matrix is positive definite or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or any other C++ implementation to do this?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2013-12-23T02:51:23.847" Title="Is there congruent transform implementation for dense symmetric matrix in Eigen(C++)?" Tags="|matrix|eigenvalues|eigen|symmetry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10387" PostTypeId="2" ParentId="10386" CreationDate="2013-12-23T02:51:23.847" Score="1" Body="&lt;p&gt;An LDLT decomposition will give you the information you need. If the diagonal $D$ matrix from the LDLT decomposition of your matrix is positive definite, your matrix is also positive definite. Here is a link to some documentation discussing the LDLT decomposition in Eigen: &lt;a href=&quot;http://eigen.tuxfamily.org/dox-2.0/TutorialAdvancedLinearAlgebra.html&quot; rel=&quot;nofollow&quot;&gt;http://eigen.tuxfamily.org/dox-2.0/TutorialAdvancedLinearAlgebra.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-23T02:51:23.847" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10388" PostTypeId="1" AcceptedAnswerId="10390" CreationDate="2013-12-23T10:06:59.110" Score="0" ViewCount="228" Body="&lt;p&gt;I downloaded the bundle adjustment data from this link:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://grail.cs.washington.edu/projects/bal/data/ladybug/problem-49-7776-pre.txt.bz2&quot; rel=&quot;nofollow&quot;&gt;original data for bundle adjustment&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is the supporting data for a paper titled:&#xA;&lt;a href=&quot;http://grail.cs.washington.edu/projects/bal/bal.pdf&quot; rel=&quot;nofollow&quot;&gt;Bundle Adjustment in the Large&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use the data for triangulation algorithm testing, which requires &lt;code&gt;3x4&lt;/code&gt; pinhole camera matrices; however the pinhole cameras were all calibrated into 9 parameters:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.5741515942940262e-02 //first three represent the `R` rotation,&#xA;-1.2790936163850642e-02&#xA;-4.4008498081980789e-03&#xA;-3.4093839577186584e-02// 4~6 are the `t' translation vector&#xA;-1.0751387104921525e-01&#xA;1.1202240291236032e+00&#xA;3.9975152639358436e+02 // the 7th is focal length&#xA;-3.1770643852803579e-07 // the last two are radial distortion parameters.&#xA;5.8820490534594022e-13.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;though there is already a reference on how to recover the &lt;code&gt;R&lt;/code&gt; from the first three numbers:&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Rodrigues%27_rotation_formula&quot; rel=&quot;nofollow&quot;&gt;Rodrigues's vector&lt;/a&gt; per the original authors:&#xA;&lt;a href=&quot;http://grail.cs.washington.edu/projects/bal/&quot; rel=&quot;nofollow&quot;&gt;description here&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still feel it is difficult for me to understand it, especially the how to recover the &lt;code&gt;R&lt;/code&gt; from the first three numbers. Anyone has suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Camera Model&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;We use a pinhole camera model; the parameters we estimate for each camera area rotation R, a translation t, a focal length f and two radial distortion parameters k1 and k2. The formula for projecting a 3D point X into a camera R,t,f,k1,k2 is:&#xA;P  =  R * X + t       (conversion from world to camera coordinates)&#xA;p  = -P / P.z         (perspective division)&#xA;p' =  f * r(p) * p    (conversion to pixel coordinates)&#xA;where P.z is the third (z) coordinate of P. In the last equation, r(p) is a function that computes a scaling factor to undo the radial distortion:&#xA;r(p) = 1.0 + k1 * ||p||^2 + k2 * ||p||^4.&#xA;This gives a projection in pixels, where the origin of the image is the center of the image, the positive x-axis points right, and the positive y-axis points up (in addition, in the camera coordinate system, the positive z-axis points backwards, so the camera is looking down the negative z-axis, as in OpenGL).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Data Format&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Each problem is provided as a bzip2 compressed text file in the following format.&#xA;&#xA;&amp;lt;num_cameras&amp;gt; &amp;lt;num_points&amp;gt; &amp;lt;num_observations&amp;gt;&#xA;&amp;lt;camera_index_1&amp;gt; &amp;lt;point_index_1&amp;gt; &amp;lt;x_1&amp;gt; &amp;lt;y_1&amp;gt;&#xA;...&#xA;&amp;lt;camera_index_num_observations&amp;gt; &amp;lt;point_index_num_observations&amp;gt; &amp;lt;x_num_observations&amp;gt; &amp;lt;y_num_observations&amp;gt;&#xA;&amp;lt;camera_1&amp;gt;&#xA;...&#xA;&amp;lt;camera_num_cameras&amp;gt;&#xA;&amp;lt;point_1&amp;gt;&#xA;...&#xA;&amp;lt;point_num_points&amp;gt;&#xA;&#xA;Where, there camera and point indices start from 0. Each camera is a set of 9 parameters - R,t,f,k1 and k2. The rotation R is specified as a Rodrigues' vector.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2013-12-23T14:31:30.800" Title="How to recover the 3x4 pinhole camera from 9 parameters" Tags="|least-squares|data-sets|computer-vision|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10389" PostTypeId="2" ParentId="8564" CreationDate="2013-12-23T13:10:29.573" Score="1" Body="&lt;p&gt;I would like to answer &quot;when to scale&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I say keep your geometry .stl , .3ds etc unscaled,in blockMesh give the bounding cuboid points in same units you have in your geometry, keep&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;convertToMeters 1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;when the entire meshing is done (including baffle and patch creation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;scale your geometry using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;transformPoints -scale ('x' 'y' 'z')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;x y z being factors by which each point in the mesh to multiplied, so if you want to scale uniformly 1/1000 , x=y=z=0.001.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this way you can keep the most accurate data of geometry while meshing and also scale to appropriate units without getting confused.&#xA;there is cost though as mentioned by @Wolfgang Bangerth&lt;/p&gt;&#xA;" OwnerUserId="6878" LastActivityDate="2013-12-23T13:10:29.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10390" PostTypeId="2" ParentId="10388" CreationDate="2013-12-23T14:31:30.800" Score="1" Body="&lt;p&gt;To convert from Rodrigues vector to a rotation matrix (and back) please check the MATLAB code here:&#xA;&lt;a href=&quot;http://www.cs.ucla.edu/~soatto/vision/courses/268/rodrigues.m&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.ucla.edu/~soatto/vision/courses/268/rodrigues.m&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this gives you the rotation matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You use the translation parameters directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, you might want to form the camera matrix. It would be a good idea to get a fully calibrated K (which involves the camera center). If you can't, just do as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors claim to have it as the image center, so will use it as our center. Again in MATLAB notation: K=[f, 0, w/2; 0, f, h/2; 0, 0, 1], where w and h are image width and height respectively. Then you can form P = K*[R | t] which you could directly use for projecting to image coordinates (if you ignore the distortion only). If you need to take into account the distortion, then do the computation as the authors describe. &lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2013-12-23T14:31:30.800" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10391" PostTypeId="1" CreationDate="2013-12-23T19:02:16.143" Score="2" ViewCount="147" Body="&lt;p&gt;I need to solve the following problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xLO51.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a given p=(x0,y0,z0,w0) and arbitrary T. For example , let p=(0.8,0.1,0.06,0.04) and T=-1.2. I need to find a vector q=(x,y,z,w) with the minimum distance from p, under the given constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a numerical solution, I tried using MATLAB but I had a lot of problems with that. I will be glad for some help / ideas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks! &lt;/p&gt;&#xA;" OwnerUserId="6912" LastActivityDate="2013-12-24T09:23:21.157" Title="minimize function with convex constraints" Tags="|convex-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10392" PostTypeId="1" CreationDate="2013-12-23T19:14:07.150" Score="1" ViewCount="197" Body="&lt;p&gt;I'm tryng to learn to use a lapack subroutine but I got stuck. I hope this is the right forum... In this fortran program I'd like as a test to find the Shur form of the matrix ((0,1)(1,0)) using cgees, but the matrix &quot;m&quot; after the call in which the Shur form should be written, is made only of zeros (other subrotunes like dgemm work and I don't get errors so I don't think is a problem with the linking). Do you know where I do wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.........................................................&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program example    &#xA;implicit none     &#xA;integer :: d,sdim,info    &#xA;real(kind=8),allocatable ::rwork(:)    &#xA;logical, external :: sel    &#xA;complex(kind=8), allocatable :: m(:,:),w(:),vs(:,:),work(:)&#xA;logical,allocatable ::bwork(:)&#xA;d=2    &#xA;!    &#xA;allocate(m(d,d))    &#xA;allocate(w(d))    &#xA;allocate(vs(d,d))    &#xA;allocate(work(2*d))    &#xA;allocate(rwork(d))    &#xA;allocate(bwork(d))    &#xA;!    &#xA;m(1,1)=(0.d0,0.d0)    &#xA;m(1,2)=(1.d0,0.d0)    &#xA;m(2,1)=(1.d0,0.d0)    &#xA;m(2,2)=(0.d0,0.d0)    &#xA;!    &#xA;print*,'m_before',m    &#xA;!    &#xA;print*,'w_before',w        &#xA;call cgees('V','S',sel,d,m,d,sdim,w,vs,d,work,-1,rwork,bwork,info)    &#xA;print*,'m',m    &#xA;print*,'w',w    &#xA;print*,'vs',vs    &#xA;print*,info        &#xA;end program example&#xA;&#xA;&#xA;logical function sel(x)    &#xA;  complex(kind=8) ::x    &#xA;  if (dimag(x)&amp;gt;1.d0) then    &#xA;     sel = .true.       &#xA;  else    &#xA;     sel = .false.    &#xA;  end if    &#xA;  return    &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;.........................................&lt;/p&gt;&#xA;" OwnerUserId="6913" LastEditorUserId="549" LastEditDate="2013-12-24T15:13:46.393" LastActivityDate="2013-12-26T17:23:02.500" Title="cgees subroutine" Tags="|lapack|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10393" PostTypeId="2" ParentId="10391" CreationDate="2013-12-23T19:24:14.337" Score="0" Body="&lt;p&gt;I guess you could well use CVX package to minimize that in MATLAB: &lt;a href=&quot;http://cvxr.com/cvx/&quot; rel=&quot;nofollow&quot;&gt;http://cvxr.com/cvx/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2013-12-23T19:24:14.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10394" PostTypeId="1" AcceptedAnswerId="11131" CreationDate="2013-12-23T23:29:11.413" Score="5" ViewCount="401" Body="&lt;p&gt;I would like to know how to compute the statistics of the discrete Fourier transform of a noise signal. To illustrate what I mean, I will first explain in detail a computation I have managed to do myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have a discrete time series of values $x_n$ with $n$ from 0 to $N-1$. Each $x_n$ is a random variable, uncorrelated with the others, and Gaussian distributed with width $\sigma$. If I define the discrete Fourier transform&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X_k = \frac{1}{N}\sum_{n=0}^{N-1} x_n e^{-2 \pi i n  k / N}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then I find that $X_k$ is a complex random variable with real and imaginary parts Gaussian distributed with width $\sigma/\sqrt{2 N}$. I did the computation by using the fact that the distribution of a sum is the convolution of the distributions, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to know how to do this computation in the case that $x_n$ are correlated. How does one approach this problem? I can make the assumption that the process is Markovian.&lt;/p&gt;&#xA;" OwnerUserId="6915" LastEditorUserId="6915" LastEditDate="2013-12-24T00:52:39.577" LastActivityDate="2014-04-18T20:44:49.723" Title="How can we compute statistics of the DFT of a random signal?" Tags="|fourier-analysis|signal-processing|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10395" PostTypeId="1" AcceptedAnswerId="10396" CreationDate="2013-12-24T05:25:45.357" Score="2" ViewCount="1049" Body="&lt;p&gt;First some background, I am a mathematician by training with very little formal training in programming. I am most comfortable in MATLAB and have been working with it for a long time. My second choice would be C++. But in either language the longest most complicated code I have written is maybe between a hundred and two hundred lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will soon start writing some code for tracking charged particles in various electromagnetic field configurations, at first in 2D and then in 3D. This sparked a debate with my friend if I should write this in MATLAB or C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions to this community it, what exactly (if any) are the benefits of doing computations like this in C++ over MATLAB? Considering all of the functions/toolboxes available in MATLAB&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.I can do in MATLAB in one line what might take dozens in C++ (old classic argument I know)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.I don't have to implement any ODE solvers for example (like RK45). All such things I will need are already efficiently tested and implemented so one less thing for me to code/debug/worry about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.Same thing for interpolation routines, interpolation in 2D-3D gets very messy, super super easy in MATLAB&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4.Same thing for visualization. Once I have the data, super easy to just look at it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5.A big chunk of this can be vectorized so MATLAB code would be very efficient I think. Not too many loops or anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6.I only need small scale shared memory parallelization (in C++ I would use OpenMP) and MATLAB already has a nice easy to use parallel toolbox.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7.MATLAB scripts can be run from command line and our local super computer has it installed (with more than enough licenses) so I can actually run my MATLAB scripts on our super computer. Just invoke multiple instances, each with 6-8 threads if necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is there any real reason why I should use C++ instead of MATLAB? I will be the only one developing and then the only one using it. I'll be using it for the next year or two for sure and after that I don't know yet. All &quot;real&quot; programmers tell me that all &quot;real&quot; programming should be done in &quot;real&quot; programming languages (i.e. FORTRAN or C). But today it just seems very archaic and outdated opinion at least for mathematical/scientific computation problems. Is there a real reason which I just didn't consider or is it just historical irrational bias?&lt;/p&gt;&#xA;" OwnerUserId="3534" LastActivityDate="2013-12-24T09:14:46.143" Title="Particle Tracking Code - MATLAB vs. C++" Tags="|matlab|simulation|c++|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10396" PostTypeId="2" ParentId="10395" CreationDate="2013-12-24T09:14:46.143" Score="3" Body="&lt;p&gt;Your question teeters on the edge of topicality for this site but here's my ha'pennyworth&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally (see, this is an opinion and that's where your question is close to off-topic), writing as a &lt;em&gt;real&lt;/em&gt; programmer (Fortran, Matlab, Mathematica, C when I have to, C++ when a gun is held to my head) I can't fault the reasons you advance for sticking with Matlab.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if any &lt;em&gt;real&lt;/em&gt; programmers scoff at your choice of Matlab, why, just respond &lt;em&gt;Well, of course, I can code the performance-critical sections in C++ and compile them into MEX-files, if that becomes desirable&lt;/em&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another opinion of mine is that it is actually very difficult to write performance-critical computational kernels in C++ which are (a) faster and (b) more reliable/robust/accurate than the Matlab intrinsic functions.  The Mathworks have put a lot of effort into optimising the execution of much of their basic functionality; oft-times Matlab functions call a tailored version of BLAS (or similar) which is the product of a compilation of C++ (or some other language).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, add another item to your list of points in Matlab's favour:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8 Since R2013a Matlab's core toolbox now includes &lt;a href=&quot;http://www.mathworks.co.uk/help/matlab/matlab-unit-test-framework.html&quot; rel=&quot;nofollow&quot;&gt;a unit-testing framework&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1003" LastActivityDate="2013-12-24T09:14:46.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10397" PostTypeId="2" ParentId="10391" CreationDate="2013-12-24T09:23:21.157" Score="1" Body="&lt;p&gt;I tried it (using YALMIP (disclaimer, developed by me))&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x0 = [0.8,0.1,0.06,0.04]';&#xA;x = sdpvar(4,1);&#xA;objective = (x-x0)'*(x-x0);&#xA;constraints = [sum(x)==1, x&amp;gt;=0, sum(x.*log(x)) &amp;lt;= -1.2];&#xA;solvesdp(constraints,objective,sdpsettings('solver','fmincon'))&#xA;double(x)&#xA;&#xA;ans =&#xA;&#xA;     5.356398659440911e-01&#xA;     1.666760280554157e-01&#xA;     1.522838063170345e-01&#xA;     1.454002996834589e-01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Absolutely no problems for fmincon to solve the problem. Your statement &quot;a lot of problems&quot; is a bit vague. My guess is that you are not being careful with the definition of the entropy at zero, i.e., you cannot compute xlog(x) directly, you have to check for zero and return the analytic value there, as you will get 0*(-inf)=NaN otherwise&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if x==0&#xA;  y = 0;&#xA;else&#xA;  y = x*log(x);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Some versions of fmincon might even require a safeguard against negative values of x, but the version above is an absolute minimum.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2013-12-24T09:23:21.157" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10398" PostTypeId="1" AcceptedAnswerId="10400" CreationDate="2013-12-24T14:04:20.300" Score="3" ViewCount="1145" Body="&lt;p&gt;My problem needs to solve dense symmetric linear systems something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;A x = b&lt;/code&gt;,&#xA;&lt;code&gt;A y = x&lt;/code&gt;,&#xA;&lt;code&gt;A z = y+x&lt;/code&gt;,...in sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Eigen C++, if I take advantage of the symmetry of &lt;code&gt;A&lt;/code&gt; by using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=A.ldlt().solve(b);&#xA;y=A.ldlt().solve(x);&#xA;z=A.ldlt().solve(y+x);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The same matrix &lt;code&gt;A&lt;/code&gt; has to be &lt;code&gt;Cholesky&lt;/code&gt; factorized many times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar issue may exists for &lt;code&gt;householderQR&lt;/code&gt; and &lt;code&gt;LU&lt;/code&gt; algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I reuse the coefficient matrix decomposition result for other systems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems this is equivalent to ask whether there is a simple back-substitution solver for upper triangular linear systems.&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6693" LastEditDate="2013-12-24T14:11:35.170" LastActivityDate="2017-07-25T21:11:22.350" Title="How to re-use the coefficient matrix decomposition result when solving linear systems by Eigen C++" Tags="|linear-solver|c++|efficiency|eigen|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10400" PostTypeId="2" ParentId="10398" CreationDate="2013-12-24T15:02:03.910" Score="6" Body="&lt;p&gt;You need to declare a LDLT object like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LDLT&amp;lt;MatrixXd&amp;gt; ldlt(A);&#xA;x = ldlt.solve(b);&#xA;y = ldlt.solve(x);&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4337" LastEditorUserId="3441" LastEditDate="2017-07-25T21:11:22.350" LastActivityDate="2017-07-25T21:11:22.350" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10401" PostTypeId="2" ParentId="10379" CreationDate="2013-12-24T19:22:35.907" Score="2" Body="&lt;p&gt;That is a sparse matrix.  Numpy doesn't have sparse matrices, and I think neither &lt;code&gt;scipy.sparse&lt;/code&gt; nor &lt;code&gt;cvxopt&lt;/code&gt; use memoryviews for sparse matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even for dense matrices it appears that &lt;code&gt;cvxopt&lt;/code&gt; uses a low level Python API instead of something like Cython.  I don't see any references to memoryview interfaces so perhaps even dense matrices in &lt;code&gt;cvxopt&lt;/code&gt; use copying instead of views.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might be the relevant function for sparse matrix indexing in the cvxopt source, where you will find a lot of copying.&#xA;&lt;a href=&quot;https://github.com/cvxopt/cvxopt/blob/master/src/C/sparse.c#L2791&quot; rel=&quot;nofollow&quot;&gt;https://github.com/cvxopt/cvxopt/blob/master/src/C/sparse.c#L2791&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2013-12-24T19:22:35.907" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10402" PostTypeId="1" CreationDate="2013-12-22T19:46:39.480" Score="1" ViewCount="654" Body="&lt;p&gt;I have a question about matrix diagonalization. I don't know if this is the right forum... Is there a way to compute the smallest real eigenvalue (and eigenvector if possible) of a general real nxn matrix? (with n small say n=5). Is there a routine in fortran 90 that does this? And, more generally, what is the situation on numerical computing all existing eigenvalues (even for non diagonalizable matrices)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: after Gerry's comment, I believe it's better to consider n as an odd number. In this case a real eigenvalue always exists because a polynomial of odd order with realcoefficients, the characteristic polynomial, has always a zero and so the smallest real eigenvalue is well defined.&lt;/p&gt;&#xA;" OwnerDisplayName="Thomas" LastEditorUserId="6663" LastEditDate="2013-12-26T16:45:50.887" LastActivityDate="2013-12-26T20:41:40.580" Title="Real eigenvalues finding" Tags="|matrix|eigenvalues|" AnswerCount="3" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="10403" PostTypeId="2" ParentId="10402" CreationDate="2013-12-22T20:44:06.617" Score="3" Body="&lt;p&gt;For numerical linear algebra, there's no point in reinventing the wheel.  There are very well-written linear algebra libraries (e.g. LAPACK) that will find eigenvalues and eigenvectors.&lt;/p&gt;&#xA;" OwnerDisplayName="Robert Israel" LastActivityDate="2013-12-22T20:44:06.617" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10404" PostTypeId="2" ParentId="10402" CreationDate="2013-12-23T00:34:24.473" Score="0" Body="&lt;p&gt;In addition to what Robert Israel wrote, I noticed that you mentioned that you are dealing with incredibly tiny square matrices, with $n=5$ being typical. I don't know what application you have in mind for this, but Mathematica is perfectly capable of computing the eigenvalues of small (ie, $n&amp;lt;5000$) matrices. Just type&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigenvalues[N[A]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $A$ is your matrix, and it will give you a list of all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you are looking for the smallest eigenvalue of a very large matrix for which typical $O(n^3)$ algorithms would be unfeasible, you should look for variants of the Lanczos algorithm. The Lanczos algorithm is used for finding the largest eigenvalues of a matrix, but it can easily be adapted to find the smallest ones as well.&lt;/p&gt;&#xA;" OwnerUserId="6928" OwnerDisplayName="DumpsterDoofus" LastActivityDate="2013-12-23T00:34:24.473" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10405" PostTypeId="2" ParentId="10378" CreationDate="2013-12-25T03:15:18.470" Score="1" Body="&lt;p&gt;&lt;code&gt;np.apply_along_axis(f, 2, x).T&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="5046" LastActivityDate="2013-12-25T03:15:18.470" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10406" PostTypeId="2" ParentId="10402" CreationDate="2013-12-25T04:41:52.767" Score="3" Body="&lt;p&gt;If $n$ is small, LAPACK is your best bet. Instead of using &lt;code&gt;cgees&lt;/code&gt;, I'd use &lt;code&gt;zgeev&lt;/code&gt;, which is a routine that will calculate eigenvalues and optionally, the left and/or right eigenvectors of a general matrix. Compared to &lt;code&gt;zgeev&lt;/code&gt;, &lt;code&gt;cgees&lt;/code&gt; uses single-precision complex numbers, rather than double-precision complex numbers, and will return the Schur form of the matrix rather than the eigenvectors you would like to calculate . There exist other, similar routines that will take advantage of symmetry (or Hermitian symmetry) if it is available for your problem. LAPACK uses dense linear algebra, which is best for small matrices (say, less than 1000 by 1000 or so, as a rough estimate; you might be able to accommodate more or less depending on how much RAM you have available).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $n$ is large, algorithms that combine a some variant of: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a good estimate of an eigenvalue&lt;/li&gt;&#xA;&lt;li&gt;a shift-and-invert strategy&lt;/li&gt;&#xA;&lt;li&gt;and power iteration&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;are used to calculate eigenvalues and eigenvectors. (More advanced algorithms use Krylov subspace iteration, transformation to Hessenberg form, and other features; broadly speaking, these can be thought of as power iteration-like algorithms with some more desirable algorithmic properties.) If you want to find the smallest eigenvalue of a large matrix, you're best off using a package like SLEPc in concert with a specialized package for eigenvalue problems, like ARPACK, BLOPEX, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, one does not calculate &lt;em&gt;all&lt;/em&gt; eigenvectors and eigenvalues of a large, sparse matrix. Usually, eigenvalues at the extremes -- the eigenvalues with the largest and smallest magnitudes -- are easier to calculate accurately than eigenvalues in the middle of the spectrum. As alluded to earlier, these are also possible to calculate accurately if estimates of these eigenvalues are available (using shift-and-invert). It's also expensive to calculate all of these eigenvalues, because iterative methods calculate one eigenvalue and eigenvector at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for diagonalization, the Jordan canonical form is numerically unstable, so it is not calculated. The Schur decomposition is typically calculated instead, as alluded to in one of the comments above. If $A = QUQ^{*}$ is the Schur decomposition of $A$, then the main diagonal of $U$ gives the eigenvalues of $A$. Once these are known, it's possible to calculate the eigenvectors of $A$. If $A$ is a normal matrix (that is, $A^{*}A = AA^{*}$, where the asterisk denotes the conjugate transpose, or Hermitian transpose), then the Schur decomposition is also an eigendecomposition, and diagonalizes the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As many noted, it's possible for a matrix not to be diagonalizable over the reals. However, as you correctly point out, the fundamental theorem of algebra implies that for $n$ odd, an $n \times n$ matrix must have a real eigenvalue, thus a smallest eigenvalue must exist.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2013-12-26T20:41:40.580" LastActivityDate="2013-12-26T20:41:40.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10407" PostTypeId="1" AcceptedAnswerId="10576" CreationDate="2013-12-26T06:01:19.063" Score="3" ViewCount="1405" Body="&lt;p&gt;My question is about Finite Element Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how to implement &quot;gradient zero&quot; conditions to advection-diffusion equations in conservative form like,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \rho}{\partial t} + \nabla \cdot \mathbf{\Gamma} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&#xA;$\mathbf{\Gamma} = \mathbf{u}\rho - k\nabla \rho$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{u}$ and $k$ is not constant variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary integral part of weak-form can be written as follows,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{\Gamma^N} \rho^* \mathbf{\Gamma} \cdot \mathbf{n} d\Gamma^N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, $\mathbf{n}$ is normal vector and $\rho^*$ is weighed function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it seems that this integral is specialized to &quot;Mixed boundary condition&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is how to modify this integral to that of &quot;Gradient zero conditon&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I beg your kindness.&lt;/p&gt;&#xA;" OwnerUserId="6930" LastActivityDate="2014-01-19T23:46:16.617" Title="Implementation of gradient zero boundary conditon in advection-diffusion equation" Tags="|pde|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10408" PostTypeId="1" AcceptedAnswerId="10412" CreationDate="2013-12-26T06:53:56.393" Score="3" ViewCount="104" Body="&lt;p&gt;What library can one use to compute efficiently the lowest eigenvalues of the Laplace operator in a polyhedral domain in $R^3$?&#xA;For the application I have in mind one has to consider very acute polyhedra (which moreover are non-convex) so it would probably be necessary to use a smart enough algorithm that can deal well with what happens near the vertices.﻿&lt;/p&gt;&#xA;" OwnerUserId="6931" LastActivityDate="2013-12-26T14:46:14.293" Title="Library to compute eigenvalues of the Laplace operator in a polyhedral domain" Tags="|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10409" PostTypeId="2" ParentId="4784" CreationDate="2013-12-26T13:49:07.820" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://developer.nvidia.com/cusparse&quot; rel=&quot;nofollow&quot;&gt;cuSPARSE library&lt;/a&gt; is the official library developed by Nvidia and comes with the CUDA SDK, it implements a lot of sparse matrix operations.  &lt;/p&gt;&#xA;" OwnerUserId="6663" LastEditorUserId="6663" LastEditDate="2013-12-26T13:57:32.167" LastActivityDate="2013-12-26T13:57:32.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10410" PostTypeId="1" AcceptedAnswerId="10411" CreationDate="2013-12-26T13:55:37.030" Score="3" ViewCount="147" Body="&lt;p&gt;I am using the sparse matrices provided by the  &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/matrices/&quot; rel=&quot;nofollow&quot;&gt;University of Florida Sparse Matrix Collection&lt;/a&gt; and most matrices are accompanied with little description of the problem or discipline from which the matrix was originated, is there a way to find more about each matrix origins, and how it was generated?&lt;/p&gt;&#xA;" OwnerUserId="6663" LastActivityDate="2019-10-31T13:11:20.827" Title="Sparse matrices origins" Tags="|matrix|sparse-matrix|collocation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10411" PostTypeId="2" ParentId="10410" CreationDate="2013-12-26T14:43:02.810" Score="2" Body="&lt;p&gt;Asking &lt;a href=&quot;http://faculty.cse.tamu.edu/davis/publications.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tim Davis&lt;/a&gt; would be the best approach to finding out.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="33125" LastEditDate="2019-10-31T13:11:20.827" LastActivityDate="2019-10-31T13:11:20.827" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10412" PostTypeId="2" ParentId="10408" CreationDate="2013-12-26T14:46:14.293" Score="2" Body="&lt;p&gt;Any of the finite element libraries will provide a generic way to do this. You can take a look at &lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/step_36.html&quot; rel=&quot;nofollow&quot;&gt;this example program of deal.II&lt;/a&gt; to see how this can be done. (Disclaimer: I am one of the authors of deal.II.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be techniques that deal specifically with the situation of polyhedral domains, for example by decomposing them into triangles and doing something special with them. However, I am unaware of such methods.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2013-12-26T14:46:14.293" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10413" PostTypeId="1" AcceptedAnswerId="10894" CreationDate="2013-12-26T16:53:50.787" Score="4" ViewCount="327" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to understand the Multiscale Finite Element Method and I'm reading Effendiev &amp;amp; Hou (2009) &lt;a href=&quot;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-0-387-09495-3&quot; rel=&quot;nofollow&quot;&gt;Multiscale Finite Element Method&lt;/a&gt;.  Suppose I'm working with a poisson equation of the form \begin{align}-\nabla\cdot(k(x)\nabla u)=f \text{ in }\Omega\\ u=g \text{ in } \partial\Omega_1 \\ k(x)\nabla u\cdot n = h \text{ in } \partial\Omega_2\end{align}  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm to solve this pde using Multiscale FEM seems fairly straightforward to implement:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Partition the domain $\Omega$ into a triangulation $T_H$ consisting of finite coarse-scale (i.e. large) elements $K_i$ and assign a piece-wise polynomial basis $\Phi_i$ for this scale.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Each element $K_i$, we solve for a local fine-scale basis function $\phi_i$ by forcing them to satisfy \begin{align}-\nabla\cdot(k(x)\nabla\phi_i)=0 \text{ in } K_i \\ \phi_i=\Phi_i \text{ on } \partial K_i \end{align}  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Let the coarse scale solution be a linear combination of the (resolved) fine scale basis functions $u_{coarse}=\sum \alpha_i \phi_i$ and solve the variational form  of the original PDE with &lt;em&gt;coarse-scale&lt;/em&gt; test functions $\Phi_j$ (i.e. petrov-galerkin method).  Thus, we obtain a system of equations  $$\sum_i \int_\Omega k(x)\nabla\phi_i\nabla\Phi_j = \int_\Omega f\Phi_j$$ for each $j.$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Question:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While this algorithm seems easy to implement, I'm struggling trying to understand how step 2 is derived.  The authors seem to present this localized problem as a given rather than something derived from theory.  While I would really love to have a full account of its derivation (or summary thereof), it may be too much to ask for in this forum.  Instead, I'd like to focus my question on two key points:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why solve a laplace equation in each element instead of the poisson equation?  How does having zero on the RHS help us resolve the fine scale nature of the basis functions?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why use boundary conditions from the coarse scale basis $\Phi_i$?  Why not interpolate them from the BC's given on $\partial\Omega$?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-04-12T16:50:37.797" Title="Localization of Sub-problems in the Multiscale Finite Element Method" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10414" PostTypeId="2" ParentId="10392" CreationDate="2013-12-26T17:23:02.500" Score="3" Body="&lt;p&gt;Reading this answer &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10402/real-eigenvalues-finding&quot;&gt;Real eigenvalues finding&lt;/a&gt;  I understood that the problem is that cgees works with single precision arithmetic and not double as I thought!!! This solves the problem :) ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general what I understood is that lapack subroutines are named so that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;subroutines beggining with &quot;d&quot; (like DGEMM, DGETRI, DGEES,...) deal with real double precision numbers;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;subroutines beggining with &quot;c&quot; (like CGEMM, CGETRI, CGEES,...) deal with complex single precision numbers;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;subroutines beggining with &quot;z&quot; (like ZGEMM, ZGETRI, ZGEES...) deal with complex double precision numbers;   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thx a lot,&lt;/p&gt;&#xA;" OwnerUserId="6913" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2013-12-26T17:23:02.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10415" PostTypeId="1" CreationDate="2013-12-26T19:59:04.253" Score="1" ViewCount="142" Body="&lt;p&gt;Does anyone know a good algorithm for quickly finding an approximate solution to the following problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given two square matrices $A$ and $B$, minimize $\| P A P^\top - B \|$ over all permutation matrices $P$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have heard that there are several types of algorithms for these kinds of problems, like iterative improvement, simulated annealing, tabu search, genetic algorithms, evolution strategies, ant algorithms, and scatter search. I am looking for existing software.&lt;/p&gt;&#xA;" OwnerUserId="6936" LastActivityDate="2013-12-26T19:59:04.253" Title="Quadratic programming problem involving permutation matrices" Tags="|optimization|algorithms|approximation|quadratic-programming|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10416" PostTypeId="1" AcceptedAnswerId="23655" CreationDate="2013-12-27T07:57:13.133" Score="5" ViewCount="1755" Body="&lt;p&gt;perhaps the most idiotic question I ever asked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;people throw around the parameter $y^+$, as if it is the be all and end all of CFD (Computational Fluid Dynamics) parameters and use it as if everyone and their dog know how to calculate it.&#xA;(Specially in CFD literature)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_\tau = (\tau/\rho)^{1/2}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $ u_\tau $ = frictional velocity and $\tau$ is wall stress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;also $\tau = \mu*\left(\dfrac{du}{dy}\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mu$ = co-efficient of viscosity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{du}{dy}$ = velocity gradient normal to wall (this formula holds near wall)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y^+ = u_\tau \dfrac{y}{\nu}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nu = \mu/\rho$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am totally down with all this (please correct any of the equations if I am wrong).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is there a way to calculate values of $y^+$ before running the simulation?&#xA;I mean can we get good guess for $u_\tau$? If not what is the point of all this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK, maybe we can't reliably, it has to be done after/while simulation runs, again after how many time-steps, should I consider that I am getting overall correct values of $y^+$ (both (min,max) and avg) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;openFoam provides such function called yPlusRAS / yPlusLES.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if we need to calculate all the values to get $y^+$, what is the point of log law? (What are we modelling exactly? because we calculated $u$ and $du/dy$ anyway) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u^+ = (1/\kappa) \ln(y^+) + C^+$$&lt;/p&gt;&#xA;" OwnerUserId="6878" LastEditorUserId="123" LastEditDate="2016-04-11T10:35:18.383" LastActivityDate="2016-04-12T00:00:28.523" Title="CFD: Can one calculate the dimensionless wall distance $y^+$ a priori?" Tags="|fluid-dynamics|openfoam|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10417" PostTypeId="1" CreationDate="2013-12-27T19:25:47.517" Score="5" ViewCount="1827" Body="&lt;p&gt;I would like use Python to verify the following identities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$ \frac{1}{1-z}  = 1 + z + z^2 + z^3 + \dots $&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$ \frac{1}{1-z - z^2}  = 1 + z + 2z^2 + 3z^3 + \dots $&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$ q \prod_{n \geq 1} (1 - q^n)^{24} = q - 24 q^2 + 252 q^3 + \dots$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The last one could be evaluated by checking the &lt;code&gt;poly1d&lt;/code&gt; class in numpy.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python doesn't seem to have a Laurent series or rational functions capability for the first two examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps there are numerical methods for generating these coefficients very quickly.  I am not sure what a good data type for storing series with potentially negative exponents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility might be to manipulate &lt;a href=&quot;https://wiki.python.org/moin/Generators&quot; rel=&quot;nofollow&quot;&gt;generators&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="42534" LastEditDate="2022-02-25T20:17:35.567" LastActivityDate="2022-02-26T04:05:07.303" Title="Rational functions in Python" Tags="|python|numpy|polynomials|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10418" PostTypeId="2" ParentId="10417" CreationDate="2013-12-27T19:38:40.710" Score="9" Body="&lt;p&gt;I think you are looking for capabilities not available in Python itself nor Numpy, but that are available in &lt;a href=&quot;http://sympy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt;. Sympy is a computer algebra package developed in Python. You can try it &amp;quot;live&amp;quot; &lt;a href=&quot;http://live.sympy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;in its on-line shell&lt;/a&gt;. Documentation on series development can be found &lt;a href=&quot;https://docs.sympy.org/latest/modules/series/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="23791" LastEditDate="2022-02-26T04:05:07.303" LastActivityDate="2022-02-26T04:05:07.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10419" PostTypeId="2" ParentId="8707" CreationDate="2013-12-28T03:37:46.830" Score="1" Body="&lt;p&gt;If your differential equations support energy conservation in some form then you should be able to prove it analytically first. So you write the energy of the system as some kind of volume integral, and take time derivative of it, and use your differential equation for time-evolution of involved quantities (a wave equation in your case) to show that the time-derivative of total energy is identically zero - that would be proving that the model obeys energy conservation. But even if it does, it is a separate issue if your discretized in space and time equations solved on a computer still satisfy this conservation law to machine accuracy (look up conservative finite-difference or finite-volume methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the equations representing physical laws should be consistent with energy conservation but in formulating a numerical model there are all kinds of approximations made, so the actual equations in the model are by no means guaranteed to satisfy the &quot;real&quot; conservation laws. For example, you model the motion of a body in viscous medium, so there is a friction force $F = - k V$. You can write and solve the equation of motion for the body, $m \dot{V}=- k V$, but obviously there is no energy conservation here since the thermal energy of the medium is not in the model. This does not mean that the model of body motion is not &quot;right&quot;, it is perfectly correct for calculating the body trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, it may happen that a model may satisfy conservation of something that is not a full physical energy but some kind of proxy of it. A good example is the &quot;energy&quot; in plasma turbulence, $E = \int ( {n}^2 + |\nabla \phi|^2 ) dV$, where $n, \phi$ are plasma density and electric potential fluctuations, see papers on Hasegawa-Wakatani drift turbulence, e.g., &lt;a href=&quot;http://iri.columbia.edu/~suzana/papers/camargo_biskamp_scott95.pdf&quot; rel=&quot;nofollow&quot;&gt;http://iri.columbia.edu/~suzana/papers/camargo_biskamp_scott95.pdf&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="4325" LastEditorUserId="4325" LastEditDate="2013-12-28T08:13:21.753" LastActivityDate="2013-12-28T08:13:21.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10420" PostTypeId="1" AcceptedAnswerId="10430" CreationDate="2013-12-28T06:35:01.917" Score="6" ViewCount="193" Body="&lt;p&gt;This is more of a conceptual question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have learned about Neural Nets, and I have some clue as to how Support Vector Machines work. I read somewhere however that given the appropriate kernel (is that right?), the SVM is identical to the Neural Net. Could someone who understands this please enlighten me as to how that's possible?&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2014-01-01T22:12:48.410" Title="Support Vector Machines as Neural Nets?" Tags="|linear-algebra|machine-learning|support-vector-machines|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10421" PostTypeId="1" CreationDate="2013-12-28T10:29:57.667" Score="0" ViewCount="2265" Body="&lt;p&gt;&lt;strong&gt;Firstly&lt;/strong&gt;, About the Hierarchical Model And X (HMAX model), i still don't understand what is the difference between the &quot;Original model&quot; and &quot;Standard model&quot;. Does in Standard, we use imfilter not conv2 ? is that correct ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Secondly&lt;/strong&gt;, i didn't understand the following phrase : &quot;&lt;em&gt;The filters in S1 layer are sum-normalized to zero and square-normalized to 1, and the result of the convolution of an image patch with a filter is divided by the power (sum of squares) of the image patch.&lt;/em&gt; &quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What was meant by sum-normalized and square normalized ?&#xA;for example : &lt;code&gt;f=f./ sqrt(sum(sum(f.^2))).&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please i need your help and explanations. Any help will be very appreciated.&lt;/p&gt;&#xA;" OwnerUserId="6950" LastActivityDate="2013-12-28T15:08:12.840" Title="A Filter is Sum normalized to 0 and square normalized to 1" Tags="|linear-algebra|matlab|machine-learning|image-processing|computer-vision|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10422" PostTypeId="2" ParentId="10421" CreationDate="2013-12-28T15:08:12.840" Score="3" Body="&lt;p&gt;I think I can address the second part of your Question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phrase &quot;sum-normalized to zero&quot; is a fancy way of saying &quot;subtract the mean (average)&quot;, i.e. subtract the constant needed to give a zero sum over the resulting function (filter) values.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phrase &quot;square-normalized to 1&quot; applies to the result of the first phrase and means dividing by a (positive) constant such that as a result the sum of squares of function values will be 1.  That divisor is the square root of the sum of squares of values (i.e. their difference from the now zero mean produced by the first step).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second step preserves the zero mean (&quot;sum-normalized to zero&quot;) property achieved by the first step.  Normalizing the &quot;filter&quot; for use in a convolution helps in controlling the magnitude of the numerical results.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2013-12-28T15:08:12.840" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10423" PostTypeId="1" CreationDate="2013-12-28T16:17:50.647" Score="0" ViewCount="307" Body="&lt;p&gt;In &lt;a href=&quot;http://arxiv.org/ftp/arxiv/papers/1112/1112.5463.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Establishing, versus maintaining, brain function: a neurocomputational model of cortical reorganisation after injury to the immature brain&lt;/a&gt;, Varier et al develop a neural network model of motor control and then &quot;lesion&quot; it as it is training. I am new to neural networks and really do not even know where to begin in emulating them so that I can investigate this further for myself. Any hints would be appreciated. The details of the model can be found on page 16 of the linked arxiv paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Figure 1 from the linked paper:&lt;/strong&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/bwVic.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="6952" LastEditorUserId="20688" LastEditDate="2019-05-25T09:53:51.023" LastActivityDate="2019-05-25T09:53:51.023" Title="How could I implement this neural network in R?" Tags="|machine-learning|r|" AnswerCount="1" CommentCount="4" ClosedDate="2014-01-09T03:57:19.473" ContentLicense="CC BY-SA 3.0" />
  <row Id="10424" PostTypeId="2" ParentId="10383" CreationDate="2013-12-28T16:38:33.700" Score="2" Body="&lt;p&gt;In terms of numerical robustness, yes, for sure. For instance, &lt;code&gt;det&lt;/code&gt; could overflow in your code; a method based on a Givens QR factorization won't instead. I guess also that there could be significant cancellation in your cofactor computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am afraid that you can't have perfect stability and minimal number of operations at the same time; you'll have to choose in which of the two directions you wish to improve.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2013-12-28T16:38:33.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10425" PostTypeId="1" AcceptedAnswerId="10436" CreationDate="2013-12-28T19:14:56.917" Score="0" ViewCount="76" Body="&lt;p&gt;I'm studying support vector machines and came across &lt;a href=&quot;http://www.tristanfletcher.co.uk/SVM%20Explained.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. The following equation doesn't make sense to me, especially the part with the 0 ∀i. Any help understanding the basics of SVMs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;yi * (xi * w + b) - 1 &gt;= 0 ∀i&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;xi - one input &lt;/li&gt;&#xA;&lt;li&gt;yi - the classification (+1 or -1) &lt;/li&gt;&#xA;&lt;li&gt;w and b - variables to find in order to &quot;train up&quot; the SVM (?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Please forgive my ignorance on SVMs, a lot is still unclear to me and I'm trying to learn these bit by bit. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6948" LastEditorUserId="6948" LastEditDate="2013-12-28T20:27:20.557" LastActivityDate="2013-12-30T07:08:49.833" Title="SVM Math Question" Tags="|machine-learning|support-vector-machines|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10426" PostTypeId="2" ParentId="7280" CreationDate="2013-12-28T21:50:45.353" Score="2" Body="&lt;p&gt;In general, radial distribution functions $g(r)$ have &lt;em&gt;not&lt;/em&gt; been computed with uncertainty bars attached to them. In principle, they do exist, since the $g(r)$ (computationally or through experimental observation) is effectively an average over many different measurements of the same system. Typically, the average would also be performed over all of the different particles of a given type. Consequently, the average is comprised of many different contributions, leading to a standard error in the mean that is quite small, relatively speaking. A larger error probably comes from discretization of the data to form the $g(r)$ in the first place.&lt;/p&gt;&#xA;" OwnerUserId="348" LastActivityDate="2013-12-28T21:50:45.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10427" PostTypeId="2" ParentId="8732" CreationDate="2013-12-29T00:23:37.037" Score="2" Body="&lt;p&gt;I have had similar problems with the corner boundary conditions, especially in solving structural plate problems with a uniformly applied transverse pressure. In particular if one is trying to obtain the Shear loads on the edges (including the corners).  The shear loads are a function of the ∂^3 w/∂^2 x∂y.  Using a central difference scheme this causes one to need the the &quot;ghost&quot; node that is diagonal to the corner node to determine this derivative.  I don't believe that averaging based on adjacent nodes is appropriate.  What I did was to use the twisting moment Mxy that I calculated at the corner node and equated it to the finite difference &quot;molecule&quot; for the twisting moment as a function of the displacements.  Since I already knew the displacements of all other adjacent nodes (based on the boundary conditions along the edges of the plate) it was a simple matter to solve for this &quot;tricky&quot; corner node.  Ii hope this helps. &lt;/p&gt;&#xA;" OwnerUserId="6956" LastActivityDate="2013-12-29T00:23:37.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10428" PostTypeId="1" CreationDate="2013-12-29T05:57:25.007" Score="5" ViewCount="95" Body="&lt;p&gt;I have a finite series that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(t) = \sum^n_{i=0} A_i cos(\Theta_i + \omega_i t) + B_i sin(\Theta_i + \omega_i t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, a finite series of pairs of phasors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the state of the art for numerically calculating the value of the series at a given $t$?  I'm interested in numerically finding the root of this series, so it's important to minimize numerical inaccuracies, particularly around the zeroes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just naively performing the cosine and sine calls produces a lot of numerical noise, which causes my root finding to fail.  I've built up a solution that works using Kahan summation and various trig identities (see &lt;a href=&quot;https://math.stackexchange.com/a/97090&quot;&gt;this&lt;/a&gt;), but it's really, really slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The N I have is not very large, so an answer doesn't have to necessarily scale well to more terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone knows any links or papers that are relevant I'd appreciate it.&lt;/p&gt;&#xA;" OwnerUserId="1365" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2013-12-29T05:57:25.007" Title="Computing linear combinations of sines and cosines (phasors)" Tags="|numerical-analysis|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10430" PostTypeId="2" ParentId="10420" CreationDate="2013-12-29T11:04:34.570" Score="6" Body="&lt;p&gt;If I understand the link of hardmath correctly, a &quot;support vector machine&quot; in its most simple form is just a glorious name for a linear function $f(x)=b^Tx+b_0$ dividing the space into two half spaces according to the sign of the values. And separating two given sets (with disjoint convex hulls for existence of a separating hyperplane).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original perceptron that started all that neural network business was just such a linear function chained together with a sigmoid function $p(x)=\sigma(f(x))$, for instance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(x)=\frac1{1+e^{-b_0-b^Tx}},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that now you can say that a point $x$ is of class 1 if $p(x)\approx 0$, which happens when $f(x)$ is very clearly negative, and of class 2 if $p(x)\approx 1$, which happens whenever $f(x)$ is very positive. Points with values of $p(x)$ in between are not clearly classifiable. By scaling $(b_0,b)$ the separating hyperplane stayes fixed, but the region of uncertainty changes width.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we got fuzzy logic before the age of fuzzy logic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classical back-propagation neural network consists of several layers of such perceptrons, where the input of each layer is the output of the previous layer, the first input is $x$ and the last output the (hopefully, after training) separating function value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still works best if the convex hulls of the classes are disjoint, but also non-convex shapes are trainable.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Essentially, neural networks and the more complex realizations of SVM are just non-linear functions with a huge parameter set making them very flexible to adapt to any desired shape, that is, they do interpolation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, take a neural network, take a linear combination of gaussian kernels, take any polynomial, all with unspecified parameters collected into one huge parameter vector $w$, and call the resulting function $f(w,x)$. Then one can pose the following non-linear optimization problem in the spirit of SVM&lt;/p&gt;&#xA;&#xA;&lt;p&gt;minimize $\frac12 \|w\|_2^2$ so that $y_i\cdot f(w,x_i)\ge 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;minimize $\frac12 \|w\|_2^2+C\sum\xi_i$ so that $y_i\cdot f(w,x_i)\ge 1-\xi_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The training points constituting the active constraints are again the support vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this to work one has to make certain &quot;niceness&quot; conditions on the function. The parameters in $w$ should all have about the same influence on the function value in the region of the training set. For $w=0$, $f(0,x)=0$ or $f(0,x)$ very small across the training set. For $\|w\|\to\infty$, the function values should also grow largely above the level of $1$ in absolute value.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The easiest way to achieve this niceness is to have $f(w,x)$ linear in $w$. Then $f(w,x)=\langle w,\phi(x)\rangle$, with $w,\phi(x)$ in some Hilbert space with its scalar product $\langle\cdot,\cdot\rangle$. The whole machinery of the simple linear case still applies, $w=\sum\alpha_iy_i\phi(x_i)$, so that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(w,x)=\sum_{α_i&amp;gt;0}α_iy_i\langle \phi(x_i),\phi(x)\rangle$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that then everything is reduced to the kernel $\kappa(x,y)=\langle \phi(x),\phi(y)\rangle$. Common examples are the the polynomials up to some degree $d$ and the gaussian functions,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{c|c}&#xA;\phi(x)=(x^e:e\in\mathbb N^n, \;|e|\le d), &amp;amp;&#xA;\phi(x)=\Bigl[u\mapsto \exp(\tfrac{\|u-x\|^2}{\sigma^2})\Bigr],\\ f(w,x)=\sum_{|e|\le d}c_ew_e\,x^e,&amp;amp; &#xA;f(w,x)=\left[\frac{2}{\pi\sigma^2}\right]^{\frac n2}\int_{\mathbb R^n}w(u)\exp(\tfrac{\|u-x\|^2}{\sigma^2})\,du,\\&#xA;\kappa(x,y)=(1+\tfrac1{R^2}\langle x,y\rangle)^d;&amp;amp;&#xA;\kappa(x,y)=\exp(\tfrac{\|x-y\|}{2\sigma^2})&#xA;\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $R$ represents some scale of the data points $x_i$ and $\sigma$ the granularity of the data points, especially of the support vectors among them. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In this sense, the parametrization of neural networks is much too non-linear to directly fit into the idea of kernel based SVM. The flexibility however should be nearly the same.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2014-01-01T22:12:48.410" LastActivityDate="2014-01-01T22:12:48.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10433" PostTypeId="1" AcceptedAnswerId="10434" CreationDate="2013-12-29T15:58:46.200" Score="2" ViewCount="25223" Body="&lt;p&gt;Suppose I have the following 3 by 3 matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p&amp;lt;-3&#xA;X&amp;lt;-matrix(1/p,p,p)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;--$\pmb X$ is just a $p$ by $p$ matrix where every entry is $1/p$.&#xA;Now I want to solve the system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\pmb a\pmb X=\pmb 1_p$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\pmb 1_p$ is a vector of 1 of length $p$.&#xA;I know $\pmb 1_p$ is the solution to such a system, &#xA;but when I do, in &lt;code&gt;R&lt;/code&gt;,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solve(X,rep(1,p))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in solve.default(X, rep(1, p)) : &#xA;  Lapack routine dgesv: system is exactly singular: U[2,2] = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;instead of $\pmb 1_p$. My question is why&lt;/p&gt;&#xA;" OwnerUserId="479" LastActivityDate="2013-12-29T17:20:59.910" Title="why Lapack routine dgesv doesn't solve this?" Tags="|r|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10434" PostTypeId="2" ParentId="10433" CreationDate="2013-12-29T17:20:59.910" Score="5" Body="&lt;p&gt;Your matrix does not have an inverse, which is why &lt;code&gt;dgesv&lt;/code&gt; returns an error.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-29T17:20:59.910" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10436" PostTypeId="2" ParentId="10425" CreationDate="2013-12-29T19:03:41.517" Score="2" Body="&lt;p&gt;You have two classes of points. Instead of managing them in two sets, one just assigns each point in the first class the value $-1$ and in the second class the value $+1$. So in fact you have point-value pairs $(x_i,y_i)$. To classify future points in a consistent way you now want to construct a function $f(x)$ that has not exactly $f(x_i)=y_i$ as in interpolation, but the still sufficient condition $f(x_i)\le -1$ for points in the first class and $f(x_i)\ge +1$ for points in the second class. These two kinds of inequality can be compressed into one single class of inequalities by multiplying with the sign $y_i$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_if(x_i)\ge 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all $i=1,...,N$, where $N$ is the number of training points. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&quot;for all&quot; has the symbolic sign $\forall$, an inverted letter &quot;A&quot;. The inverted letter &quot;E&quot;, $\exists$, is the symbol for &quot;exists&quot;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now to find such a function, you select a parametrized class of functions $f(w,x)+b$ with some parameter vector $(w,b)$ and strive to find a compromise between having a simple form of $f$ and small function values on the test set, or rather, $f(w,x_i)=y_i$, which defines the support vectors, on as many points as possible. Simplicity includes that the parameters in $w$ are small numbers.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So we come to the linear SVM where $f(w,x)=w^Tx$ and minimal paramters means to minimize $\|w\|_2^2=w^Tw$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In optimization, this task is encoded via a Lagrange function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L(w,b,α)=\tfrac12\|w\|_2^2-\sum_{i=1}^Nα_i(y_i(w^Tx_i+b)-1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the restriction $α_i\ge 0$. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Standard optimization techniques solve this problem via its KKT system.&#xA;\begin{align}&#xA;0=\frac{\partial L}{\partial w}&amp;amp;=w-\sum_{i=1}^Nα_iy_ix_i\\&#xA;0=\frac{\partial L}{\partial b}&amp;amp;=-\sum_{i=1}^Nα_i y_i\\&#xA;α_i&amp;amp;\ge 0\\&#xA;y_i(w^Tx_i+b)-1&amp;amp;\ge 0\\&#xA;α_i\,(y_i(w^Tx_i+b)-1)&amp;amp;=0&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The last three equations again for all $i$. They can be combined using NCP functions like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$N(u,v)=2uv-(u+v)_-^2$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $(u+v)_-=\min(0,u+v)$ to one condition per $i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$N(α_i,\, y_i(w^Tx_i+b)-1)=0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This now is smooth enough so that Newton's method or quasi-Newton methods may be applied.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2013-12-30T07:08:49.833" LastActivityDate="2013-12-30T07:08:49.833" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10437" PostTypeId="1" CreationDate="2013-12-30T08:36:48.797" Score="2" ViewCount="116" Body="&lt;p&gt;I want to know the best optimal algoritm of gcd with its complexity if you have a any useful source I will be glad to have a look at it.&lt;/p&gt;&#xA;" OwnerUserId="6964" LastEditorUserId="3630" LastEditDate="2013-12-30T13:42:20.677" LastActivityDate="2013-12-31T12:11:02.957" Title="Optimal algoritm of gcd with complexity" Tags="|optimization|algorithms|reference-request|complexity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10439" PostTypeId="2" ParentId="10423" CreationDate="2013-12-30T15:24:12.220" Score="2" Body="&lt;p&gt;Actually, there are two considerable packages in &lt;strong&gt;R&lt;/strong&gt; to handle neural networks with easiness. Here they are: &lt;code&gt;nnet&lt;/code&gt; and &lt;code&gt;neuralnet&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Install them via&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;install.packages('nnet')&lt;/p&gt;&#xA;&lt;p&gt;install.packages('neuralnet')&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;in &lt;strong&gt;R&lt;/strong&gt;. To get help and see examples, see&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;?neuralnet::neuralnet&lt;/p&gt;&#xA;&lt;p&gt;?nnet::nnet&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can look at a neural network as a function &lt;code&gt;f(x)&lt;/code&gt; where &lt;code&gt;x&lt;/code&gt; is a vector of inputs. You put a vector and the neural network answers you a number.&lt;/p&gt;&#xA;&lt;p&gt;The first two problems you will find are&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Define an architecture&lt;/strong&gt;. For simplicity, take a single node at first, this will be equal to a linear regression.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Train the network&lt;/strong&gt;. There are algorithms such as Levenberg-Marquardt.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This is the absolute minimum you have to know in order to program/run a neural network in &lt;strong&gt;R&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6966" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2013-12-30T15:24:12.220" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10440" PostTypeId="1" AcceptedAnswerId="10441" CreationDate="2013-12-30T18:13:15.870" Score="12" ViewCount="250" Body="&lt;p&gt;Let $\vec{x} = (x_1, x_2, \dots, x_n) \in [0,1]^n$ and $f(\vec{x}): [0,1]^n \to \mathbb{C}$ be a function in these variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a recursive scheme for this iterated integral?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{[0,1]^n} \prod dx_i \;f(\vec{x}) $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $n = 10$ and I break $[0,1]$ into 100 segments, we have $10^{20}$ points to add up.   There must be a smarter way.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In fact, the function I wish to integrate is the &lt;a href=&quot;https://mathoverflow.net/questions/36025/explicit-computations-using-the-haar-measure&quot;&gt;Haar measure&lt;/a&gt; of the Unitary group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{U(n)} f(A) \ dA = &#xA;\frac{1}{n!} &#xA;\int_{[0,2\pi]^n}&#xA;\prod_{j&amp;lt;k} \big|e^{i \theta_j} - e^{i \theta_k}\big|^2 \cdot f(\theta_1, \ldots, \theta_n) \  \frac{d \theta_1}{2\pi} \ \cdots \ \frac{d \theta_n}{2\pi}$$&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2014-01-01T06:52:51.763" Title="numerical integration in many variables" Tags="|numerical-analysis|fourier-analysis|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10441" PostTypeId="2" ParentId="10440" CreationDate="2013-12-30T18:48:47.990" Score="15" Body="&lt;p&gt;For integrations with many variables, the Monte Carlo method usually is a decent fit.  Its error decreases as $O (\sqrt{N})$ where N is the number of equidistributed points selected.  Of course this is not good for low dimension (1D and 2D) spaces where high order methods exist.  Most of these deterministic methods, however, take a large number of points in higher dimensions. For example, a 1st order 1D scheme is $O(\sqrt{N})$ in 2D and $O(N^{\frac{1}{4}})$ in 3D. The strength of the Monte Carlo method is that the error convergence is independent of space dimension.  No matter whether your space is 1D or 100D, it is $O (\sqrt{N})$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is probabilistic, however, you need to integrate it multiple times using a set number of points to find a standard deviation and an estimate of your error.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2013-12-30T19:06:31.377" LastActivityDate="2013-12-30T19:06:31.377" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10442" PostTypeId="1" CreationDate="2013-12-31T06:16:21.227" Score="2" ViewCount="153" Body="&lt;p&gt;First, an example:&lt;br&gt;&#xA;Given an image (2D array) to write down a mathematical notation for the function of pixelation, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bQcrr.png&quot; alt=&quot;pixelation&quot;&gt;&lt;br&gt;&#xA;&lt;em&gt;Fig. 0: The pixelated version (right) of the given 2D array (left)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What would be a mathematical notation for the above simple function?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part:&lt;br&gt;&#xA;&lt;strong&gt;How about many everyday computing algorithms that can apply on any arbitrary dimensional data if one wants to write them down in mathematical notation? You as a computer scientist how would you start, proceed and finalise?&lt;/strong&gt; Consider the case that your work will be checked by a pure mathematician!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;to whom may can please add a new tag math-notation&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="6969" LastActivityDate="2013-12-31T07:36:48.550" Title="Practices to convert computer algorithms to matheamtical notations" Tags="|algorithms|education|" AnswerCount="1" CommentCount="3" ClosedDate="2014-01-27T16:19:49.693" ContentLicense="CC BY-SA 3.0" />
  <row Id="10443" PostTypeId="2" ParentId="10442" CreationDate="2013-12-31T07:36:48.550" Score="2" Body="&lt;p&gt;To answer the first part of your question: there are probably many functions that achieve the same goal, so &quot;what would be a mathematical notation for the above simple function&quot; will have many answers. For instance, the image could be coarsened by decreasing the resolution by one-half in each dimension so that each pixel in the coarse image corresponds to four pixels in the original image, and the pixel color could be determined by averaging, in which case you could use algebraic formulas with indexing. The few signal processing questions we've gotten on here have used special notation for filters; I am not familiar with signal processing, so I can't really describe it well, or what it means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part of your question on the correspondence between algorithms and mathematical notation is broad. To give a brief overview, the Church-Turing thesis states that any function that is &quot;algorithmically calculable&quot; if and only if it can be computed by Turing machine. So a facile answer would be, &quot;Any algorithm can be expressed in mathematical notation as a Turing machine.&quot; Few people describe algorithms in this manner. You'll see it in a theory of computation class, but you wouldn't see it in an undergraduate introduction to algorithms class. What seems to be more common for algorithmic description is pseudocode. There's no standard pseudocode used everywhere, but one common variant can be found in the algorithms textbook by Cormen, Leiserson, Rivest, and Stein. You can find pseudocode also in several applied mathematics papers in SIAM journals; more mathematical algorithms tend to involve equations with standard notation that augment some sort of pseudocode like that in the Cormen, et al. book. Others simply describe their algorithm in long-form sentences, again using mathematical notation as needed. If it's short enough, some include MATLAB code (or similar sorts of code) in their papers. If the program used to implement an algorithm is long, typically some sort of description is given; the program may or may not be accessible to readers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your comment about pure mathematicians, that really depends on audience as well. In more formal settings (computer science classes, some research settings like certain mathematics journals), a proof may be required to show that a proposed algorithm achieves its stated objectives. In less formal settings, the algorithm may simply be stated without proof; exposition without proof is more common in journals with a less mathematically rigorous readership, such as some applications-oriented engineering journals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So both of your questions have many answers, and the latter is certainly open to interpretation and preference.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2013-12-31T07:36:48.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10444" PostTypeId="2" ParentId="10437" CreationDate="2013-12-31T12:11:02.957" Score="2" Body="&lt;p&gt;Current state of the art uses variants of Euclid's algorithm. Yet, it is proven to be non-optimal:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dspace.ucalgary.ca/bitstream/1880/46601/2/1989-376-38.pdf&quot; rel=&quot;nofollow&quot;&gt;http://dspace.ucalgary.ca/bitstream/1880/46601/2/1989-376-38.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though its variants have significant speed improvements, I have not heard of a sequential optimal one. I guess one of the best modifications is Lehmer's algorithm. Extended Euclidean algorithms are also worth mentioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, I guess optimality, or at least guaranteed bounds are obtained on parallel machines. One of such profound works prove a sequential complexity of O(n^2/log(n)) while achieving a paralel complexity, which is worth taking a look at :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.csie.nuk.edu.tw/~cychen/gcd/Two%20fast%20GCD%20algorithms.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.csie.nuk.edu.tw/~cychen/gcd/Two%20fast%20GCD%20algorithms.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2013-12-31T12:11:02.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10445" PostTypeId="2" ParentId="7001" CreationDate="2013-12-31T15:11:14.250" Score="0" Body="&lt;p&gt;I suggest you to use Levenberg Marquardt (a trust region variant) as it is used in many practical applications and demonstrated very good speed-vs-accuracy performance. Moreover, for GPU there are some libraries (e.g. cuLM &lt;a href=&quot;https://github.com/zitmen/cuLM&quot; rel=&quot;nofollow&quot;&gt;https://github.com/zitmen/cuLM&lt;/a&gt;), which you could try out. If they don't do the job, there are tons of resources for you to implement. Implementing LM is not hard at all. You should only take care about minimizing GPU communication. To get a brief idea:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://on-demand.gputechconf.com/gtc/2012/presentations/S0231-Levenberg-Marquardt-Using-Block-Sparse-Matrices-on-CUDA.pdf&quot; rel=&quot;nofollow&quot;&gt;http://on-demand.gputechconf.com/gtc/2012/presentations/S0231-Levenberg-Marquardt-Using-Block-Sparse-Matrices-on-CUDA.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2013-12-31T15:11:14.250" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10446" PostTypeId="2" ParentId="10440" CreationDate="2014-01-01T06:10:32.353" Score="8" Body="&lt;p&gt;Sparse grid quadrature is an alternative approach to integrate in higher dimensions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quadrature relies on evaluating a weighted sum of function values at specific &quot;optimal&quot; points.  Traditional quadrature uses a tensor product grid construction in higher dimensions, which means that you would have to evaluate the function at an exponentially growing number of points as the dimension increases.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trick to &lt;strong&gt;&lt;em&gt;sparse grid quadrature&lt;/em&gt;&lt;/strong&gt; is that you can obtain the same order accuracy (in the asymptotic sense) using a small subset of the tensor product grid.  The sparse points you choose end up being those that accurately integrate monomials of up to a desired &lt;strong&gt;total degree&lt;/strong&gt;.  The computational savings (compared to the tensor product grid) increase significantly as the dimension increases.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, however, drawbacks to this method that you should be aware.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This method does not work well if your function is not smooth (or otherwise not well approximated by polynomial functions).    &lt;/li&gt;&#xA;&lt;li&gt;While the order of accuracy of sparse grid quadrature may be equivalent to a tensor product grid, the relative accuracy may be much worse.  This is because the constant in front of the sparse grid's order of accuracy can be very large.  &lt;/li&gt;&#xA;&lt;li&gt;Sparse grids work well for relatively small dimensions.  But there comes a dimension after which you'd probably be better off using another method (like monte carlo or its variants).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For more information on sparse grids, I recommend Burkardt's &lt;a href=&quot;http://people.sc.fsu.edu/~%20jburkardt/presentations/sparse_2007_psc.pdf&quot;&gt;Sparse Grids in High Dimensions&lt;/a&gt;.  If you're interested in code to generate sparse grids, you may want to consider &lt;a href=&quot;http://www.sparse-grids.de/&quot;&gt;these matlab files&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-01-01T06:52:51.763" LastActivityDate="2014-01-01T06:52:51.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10447" PostTypeId="1" AcceptedAnswerId="10448" CreationDate="2014-01-01T18:52:41.500" Score="4" ViewCount="258" Body="&lt;p&gt;I am looking to create an algorithm to minimize an  N dimensional problem. I am unsure how to write it in its generic form, so I will show it in 1, 2 and 3 dimensions&#xA;Minimize $ \sum_{i} x_i\left [ f\left ( w_i\right )-H \right ], $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constrained by$ \sum_{I} x_i \left (w_i - R \right) \le 0.01,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where&#xA;$f\left (w\right)\left\{\begin{matrix}&#xA; 0&amp;amp; w=0\\ &#xA; 1&amp;amp; w&amp;gt;0&#xA;\end{matrix}\right.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0 &amp;lt;R\le1,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0 &amp;lt;H\le 1,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x \in \mathbb{R}^{+},$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w \in \mathbb{R}^{+},$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2D&#xA;Minimize $ \sum_{i}\sum_{j} x_{1i}x_{2j}\left [ f\left ( w_{ij}\right )-H \right ], $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constrained by$ \sum_{i}\sum_{j} x_{1i }x_{2j}\left (w_{ij} - R \right) \le 0.01,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And 3D&#xA;Minimize $ \sum_{i}\sum_{j}\sum_{k}x_{1i}x_{2j}x_{3k}\left [ f\left ( w_{ijk}\right )-H \right ], $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constrained by$ \sum_{i}\sum_{j}\sum_{k} x_{1i }x_{2j}x_{3k}\left (w_{ijk} - R \right) \le 0.01,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values of R, H and w are all known, so we are solving for x.&#xA;The first problem I am looking to solve with this is actually in 5D, with each sum over 19. &#xA;In 1D  this looks to me like a basic branch and bounds problem akin to the regularly used example of the knapsack problem, however the unknowns here are all real rather than integer and I'm unsure of how this would generalise to multiple dimensions.&#xA;Any pointers for literature to read or algorithms to try would be massively appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify the problem, in case my notation is iffy, I am a physicist and a bit rusty on standard maths notation, expanding the 2D example with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ w= \begin{bmatrix}&#xA; 5 &amp;amp; 2\\ &#xA; 0&amp;amp; 0&#xA;\end{bmatrix} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R=0.95$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gives for the constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$4.05x_{11}x_{21}+1.05x_{11}x_{22}-0.95x_{12}x_{21}-0.95x_{12}x_{22}\le0.01$ &lt;/p&gt;&#xA;" OwnerUserId="6977" LastEditorUserId="27" LastEditDate="2014-01-02T12:55:26.560" LastActivityDate="2014-01-02T12:55:26.560" Title="constrained minimization in N dimensions" Tags="|optimization|algorithms|nonlinear-programming|constrained-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10448" PostTypeId="2" ParentId="10447" CreationDate="2014-01-01T19:47:11.373" Score="4" Body="&lt;p&gt;The isolated bilinear and trilinear terms make your problem nonconvex. (Occasionally, these terms can be gathered into sums or differences of squares, but that does not appear to be the case here.) If $f$ is a twice continuously differentiable function, and you're interested in deterministic global optimization, you probably want to use a branch-and-bound method in concert with a method for generating convex relaxations and a deterministic local optimization method (any standard method like sequential quadratic programming, interior-point methods, active-set methods, etc. will work). This combination of methods is pretty hard to come by. &lt;a href=&quot;http://archimedes.cheme.cmu.edu/?q=baron&quot; rel=&quot;nofollow&quot;&gt;BARON&lt;/a&gt; is a &lt;a href=&quot;http://gams.com/&quot; rel=&quot;nofollow&quot;&gt;GAMS&lt;/a&gt; solver that has all of these features. It's possible to roll your own with some combination of interval arithmetic, convex analysis, and automatic differentiation. &lt;a href=&quot;http://yoric.mit.edu/libMC&quot; rel=&quot;nofollow&quot;&gt;libMC&lt;/a&gt; (or its successor &lt;a href=&quot;http://www.imperial.ac.uk/AP/faces/pages/read/Research.jsp?person=b.chachuat&amp;amp;_adf.ctrl-state=ty5vlc3e4_107&amp;amp;_afrRedirect=1329814605172000&quot; rel=&quot;nofollow&quot;&gt;MC++&lt;/a&gt;) is one tool that I'm aware of that will compute the convex relaxations you need; it works with the automatic differentiation software &lt;a href=&quot;http://www.fadbad.com/fadbad.html&quot; rel=&quot;nofollow&quot;&gt;FADBAD++&lt;/a&gt;. Rolling your own solver is likely to be a lot of work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't care about obtaining a certified globally $\varepsilon$-optimal solution, you could use a local optimization method with multistart, or nondeterministic methods (simulated annealing). My preference is to use multistart in combination with &lt;a href=&quot;https://projects.coin-or.org/Ipopt&quot; rel=&quot;nofollow&quot;&gt;IPOPT&lt;/a&gt;, which is an interior-point method that has a limited capability for coping with nonconvexity (better than nothing).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-01-02T00:44:29.010" LastActivityDate="2014-01-02T00:44:29.010" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10449" PostTypeId="1" AcceptedAnswerId="10454" CreationDate="2014-01-02T05:43:48.230" Score="1" ViewCount="22745" Body="&lt;p&gt;If a matrix $A_{n\times n}$ is invertible, then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left\|A^{-1}\right\|_2 = \dfrac{1}{\min\limits_{i} \sigma_i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\sigma_i$ is the $i$-th singular value of $A$&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6910" LastEditDate="2017-10-08T15:58:27.347" LastActivityDate="2019-11-10T17:50:14.727" Title="How to prove the 2-norm of an invertible matrix is exactly the reciprocal of its minimum singular value?" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10450" PostTypeId="1" AcceptedAnswerId="10453" CreationDate="2014-01-02T05:49:29.350" Score="2" ViewCount="677" Body="&lt;p&gt;Spectral decomposition of symmetric matrix $A_{n\times n}$, specifically, $n=3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;find the orthogonal matrix $Q$ and diagonal matrix $\Lambda$ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=Q\Lambda Q^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to implement such decomposition in Eigen C++? or any other C++ implementations?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2014-01-02T10:48:37.037" Title="How to implement the spectral decomposition of a symmetric dense matrix via Eigen C++" Tags="|c++|eigensystem|eigen|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10451" PostTypeId="1" CreationDate="2014-01-02T10:17:51.687" Score="2" ViewCount="1348" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sltx1.png&quot; alt=&quot;Excerpt from book - FEM by using Matlab by Kwon et.al.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the attached image, I want to understand &lt;strong&gt;how to arrive at the equation 2.5.1&lt;/strong&gt;, i.e. the variational expression. The problem is defined as best as it could and the further derivation follows smoothly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complete derivation is as shown in the image below: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/JwLJG.png&quot; alt=&quot;Complete Derivation&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in how the eq 2.1.1 leads to eq 2.5.1 ... or rather how does the variational operator (del) work?&lt;/p&gt;&#xA;" OwnerUserId="6981" LastEditorUserId="6981" LastEditDate="2014-01-04T05:21:12.877" LastActivityDate="2014-01-04T05:21:12.877" Title="How to derive the functional for a given differential equation using the variational expression?" Tags="|finite-element|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10452" PostTypeId="1" CreationDate="2014-01-02T10:45:23.433" Score="2" ViewCount="189" Body="&lt;p&gt;I have two 50000 x 50000 binary matrices A and B for solving A*x=lambda*B*x eigenvalue problem. These matrices are sparse. I am trying to solve using PETSC and SLEPC. My memory requirement shoots off like more than 200 GB of RAM ! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used mpirun in 16 cores and 96GB RAM with swap of 130GB. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to solve this problem.? Am I doing something wrong ?&#xA;Kindly let me know.&lt;/p&gt;&#xA;" OwnerUserId="5495" LastActivityDate="2014-01-02T10:45:23.433" Title="How to reduce RAM requirement in PETSC and SLEPCreading large binary matrices" Tags="|petsc|" AnswerCount="0" CommentCount="2" ClosedDate="2014-01-28T12:35:21.650" ContentLicense="CC BY-SA 3.0" />
  <row Id="10453" PostTypeId="2" ParentId="10450" CreationDate="2014-01-02T10:48:37.037" Score="6" Body="&lt;p&gt;You're looking for the &lt;a href=&quot;http://eigen.tuxfamily.org/dox/classEigen_1_1SelfAdjointEigenSolver.html&quot;&gt;SelfAdjointEigenSolver class&lt;/a&gt;, and there is also an &lt;a href=&quot;http://eigen.tuxfamily.org/dox/group__TutorialLinearAlgebra.html#title2&quot;&gt;example&lt;/a&gt; in the user manual that I report here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;Eigen/Dense&amp;gt;&#xA;using namespace std;&#xA;using namespace Eigen;&#xA;int main()&#xA;{&#xA;   Matrix2f A;&#xA;   A &amp;lt;&amp;lt; 1, 2, 2, 3;&#xA;   cout &amp;lt;&amp;lt; &quot;Here is the matrix A:\n&quot; &amp;lt;&amp;lt; A &amp;lt;&amp;lt; endl;&#xA;   SelfAdjointEigenSolver&amp;lt;Matrix2f&amp;gt; eigensolver(A);&#xA;   if (eigensolver.info() != Success) abort();&#xA;   cout &amp;lt;&amp;lt; &quot;The eigenvalues of A are:\n&quot; &amp;lt;&amp;lt; eigensolver.eigenvalues() &amp;lt;&amp;lt; endl;&#xA;   cout &amp;lt;&amp;lt; &quot;Here's a matrix whose columns are eigenvectors of A \n&quot;&#xA;        &amp;lt;&amp;lt; &quot;corresponding to these eigenvalues:\n&quot;&#xA;        &amp;lt;&amp;lt; eigensolver.eigenvectors() &amp;lt;&amp;lt; endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4337" LastActivityDate="2014-01-02T10:48:37.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10454" PostTypeId="2" ParentId="10449" CreationDate="2014-01-02T14:36:45.870" Score="10" Body="&lt;p&gt;Singular value decomposition of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; can be written as &lt;span class=&quot;math-container&quot;&gt;$A=USV^T$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; being a diagonal matrix of singular values of &lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Euclidean norm of a matrix can be written as : &lt;span class=&quot;math-container&quot;&gt;$||A||_2=\sigma_{max}(A)$&lt;/span&gt;, meaning that the norm is the maximum singular value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is invertible, then &lt;span class=&quot;math-container&quot;&gt;$A^{-1}=(USV^T)^{-1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here it follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A^{-1}=(USV^T)^{-1}&#xA;      =(V^T)^{-1}S^{-1}U^{-1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a real valued matrix, then from orthogonality the expression further reduces to:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$A^{-1}=VS^{-1}U^T.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This new matrix &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt; now has singular values &lt;span class=&quot;math-container&quot;&gt;$S^{-1}$&lt;/span&gt;, and its norm would be &lt;span class=&quot;math-container&quot;&gt;$\max(\text{diag}(S^{-1}))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\text{diag}$&lt;/span&gt; takes the diagonal of the matrix. Because &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is a diagonal matrix, its inverse is computed by simply inverting every single element. Therefore,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$||A^{-1}||_2=\sigma_{max}(A^{-1})=\max\limits_{i} \text{diag}(S^{-1})_i = \max\limits_{i}\frac{1}{\sigma_i}=\frac{1}{\min\limits_{i}{\sigma_i}}$&lt;/span&gt; and this completes the proof.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2019-10-18T12:27:17.967" LastActivityDate="2019-10-18T12:27:17.967" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10455" PostTypeId="2" ParentId="10451" CreationDate="2014-01-02T18:16:34.317" Score="1" Body="&lt;p&gt;In general, the variational problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I[u]=\int_a^b \Bigl(\tfrac12 u_x(x)^2+V(x,u(x))\Bigr)\,dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;leads to the functional derivative&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$δ I[u,δu]=\int_a^b \Bigl(u_x(x)δu_x(x)+V_u(x,u(x))δu(x)\Bigr)\,dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and after partial integration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$δ I[u,δu]=[u_x(x)δu(x)]_a^b+\int_a^b \Bigl(-u_{xx}(x)+V_u(x,u(x))\Bigr)δu(x)\,dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions lead to $δu(a)=0=δu(b)$, identifying the terms in $u_{xx}=V_u(x,u)$ with the given equation to $V_u=u-x$, thus $V(x,u)=\tfrac12u^2-xu$.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2014-01-02T18:16:34.317" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10456" PostTypeId="1" CreationDate="2014-01-03T04:48:49.143" Score="5" ViewCount="93" Body="&lt;p&gt;Typically Chebyshev interpolation from $-1$ to $1$ with angle from $0$ to $\pi$: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\xi_j=\cos \left ({\pi j \over N}\right )$&lt;/li&gt;&#xA;&lt;li&gt;$x_j=(1+\xi_j) * {L \over 2}$&lt;/li&gt;&#xA;&lt;li&gt;$w$:&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$w_0=\pi/(2N)$&lt;/li&gt;&#xA;&lt;li&gt;$w_{1,...,N-1}=\pi/(N)$&lt;/li&gt;&#xA;&lt;li&gt;$w_N=\pi/(2N)$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I appear to have a need in the same with angle from $0$ to $\pi/2$ only, while solving a collocation grid problem where mesh is very fine inside of the computational domain and gets coarser near computational domain boundary. Literature review does not appear to be fruitful.&lt;/p&gt;&#xA;" OwnerDisplayName="user5273" LastEditorUserId="276" LastEditDate="2014-01-03T08:11:51.037" LastActivityDate="2014-01-03T08:11:51.037" Title="How do I perform chebyshev interpolation from a to b with custom angle range?" Tags="|interpolation|collocation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10457" PostTypeId="2" ParentId="8524" CreationDate="2014-01-03T08:58:31.603" Score="3" Body="&lt;p&gt;As others said, it would be easier to make useful suggestions if you described your overall goal rather than a precise technical point. Here is one more suggestion that might help you, depending on what your goal is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ActivePapers project (&lt;a href=&quot;http://www.activepapers.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.activepapers.org/&lt;/a&gt;) provides a code and data management system on top of HDF5. An ActivePaper is an HDF5 file that contains datasets AND the code that works on them, with metadata keeping track of which piece of code computed which dataset and using which input data. In combination with version control on source code and/or version control on the whole HDF5 file (using tools such as git-annex, mentioned in another reply), ActivePapers can be used for versioning computations rather than isolated files or datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I am the author of ActivePapers.&lt;/p&gt;&#xA;" OwnerUserId="140" LastActivityDate="2014-01-03T08:58:31.603" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10458" PostTypeId="1" AcceptedAnswerId="10459" CreationDate="2014-01-03T10:16:10.637" Score="2" ViewCount="788" Body="&lt;p&gt;Let $A$ be an $M \times N$ sparse matrix stored in compressed column format, in a C-like programming environment. I am interested in the best solution to get a sub-matrix of $A$. In MATLAB notation, this would read:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A_sub = A(i:i+m-1, j:j+n-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $A_{sub}$ is again a sparse matrix, but this time being $m \times n$.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="276" LastEditDate="2014-01-03T19:04:06.723" LastActivityDate="2014-01-05T22:00:51.543" Title="Cropping in Sparse Matrix" Tags="|matrix|sparse-matrix|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10459" PostTypeId="2" ParentId="10458" CreationDate="2014-01-03T19:17:30.843" Score="4" Body="&lt;p&gt;While you mentioned using C, Yousef Saad's package &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/software/SPARSKIT/&quot; rel=&quot;nofollow&quot;&gt;SPARSKIT&lt;/a&gt; has a lot of sparse matrix algorithms if you're comfortable with Fortran 77.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The compressed column format consists of three arrays, &lt;code&gt;ja&lt;/code&gt;, &lt;code&gt;ia&lt;/code&gt; (integers) and &lt;code&gt;a&lt;/code&gt; (floats/doubles). For each column index &lt;code&gt;j&lt;/code&gt;, &lt;code&gt;ja(j)&lt;/code&gt; stores the location in the array &lt;code&gt;ia&lt;/code&gt; where the list of all non-zero rows in column &lt;code&gt;j&lt;/code&gt; starts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step in extracting a sub-matrix &lt;code&gt;A_sub&lt;/code&gt; from &lt;code&gt;A&lt;/code&gt; is to count how many non-zero entries there are in each of the &lt;code&gt;n&lt;/code&gt; columns; for that purpose, you can make an array &lt;code&gt;count&lt;/code&gt; of all zeros and then do something to the effect of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int k, l;&#xA;for (k=0; k&amp;lt;n; k++) {&#xA;    for (l=A.ja[k+j]; l&amp;lt;A.ja[k+j+1]; l++) {&#xA;        if (A.ia[l]&amp;gt;=i &amp;amp;&amp;amp; A.ia[l]&amp;lt;i+m) {&#xA;            count[k]++;&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Next, you'd fill out &lt;code&gt;A_sub.ja&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A_sub.ja[0] = 0;&#xA;for (k=0; k&amp;lt;n; k++) {&#xA;    A_sub.ja[k+1] = A_sub.ja[k]+count[k];&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The next step is to fill out the arrays &lt;code&gt;ia&lt;/code&gt; and &lt;code&gt;a&lt;/code&gt;, which requires a second pass through &lt;code&gt;A&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're being space-conscious, instead of making a new array &lt;code&gt;count&lt;/code&gt;, you can store the same values starting at the address &lt;code&gt;A_sub.ja[1]&lt;/code&gt; with no effect on the output. SPARSKIT does this if you want to see an example.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-01-03T19:17:30.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10460" PostTypeId="1" CreationDate="2014-01-03T00:24:31.977" Score="15" ViewCount="3644" Body="&lt;p&gt;It is well known that for certain linear systems Jacobi and &lt;a href=&quot;https://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gauss-Seidel&lt;/a&gt; iterative methods have the same convergence behavior, e.g. Stein-Rosenberg Theorem. I am wondering if similar results exist for nonlinear iterations, where at step $k$ the Jacobi iterations on, say $\mathbb{R}^n$, are defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_i^{k+1}=F_i(x_1^{k+1},\cdots,x_{i-1}^{k+1},x_i^k,\cdots,x_n^k),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the Gauss-Seidel iterations are defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_i^{k+1}=F_i(x_1^k,\cdots,x_n^k)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $i=1,\cdots,n$ and we have a set of nonlinear functions $F_i(\cdot): \mathbb{R}^n\to\mathbb{R}$.&lt;/p&gt;&#xA;" OwnerUserId="7026" OwnerDisplayName="hchen" LastEditorUserId="9667" LastEditDate="2018-01-01T22:25:04.810" LastActivityDate="2021-03-13T20:40:35.197" Title="Comparing Jacobi and Gauss-Seidel methods for nonlinear iterations" Tags="|nonlinear-equations|convergence|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10461" PostTypeId="2" ParentId="10458" CreationDate="2014-01-05T22:00:51.543" Score="1" Body="&lt;p&gt;With the Eigen matrix class library (&lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt;),&#xA;you can efficiently extract a submatrix with a single line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigen::SparseMatrix A(m,n);&#xA;Eigen::SparseMatrix B = A.block(startRow, startCol, numRows, numCols);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If A already exists in compressed sparse column (or row) format, you can extract a submatrix with code similar to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigen::MappedSparseMatrix&amp;lt;double&amp;gt; Amap(A.m, A.n, A.nnz, A.colPtrs, A.rowPtrs, A.vals);&#xA;Eigen::SparseMatrix B = Amap.block(startRow, startCol, numRows, numCols);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-01-05T22:00:51.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10462" PostTypeId="1" AcceptedAnswerId="10471" CreationDate="2014-01-05T22:17:37.510" Score="6" ViewCount="965" Body="&lt;p&gt;I have an $n \times n$ unsymmetric matrix $A$ that results from the discretization of an ill-posed Poisson problem, and thus is rank-deficient with null space of dimension one.  I want to compute just the smallest singular triplet &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;(u_n,\sigma_n,v_n)&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*} &#xA; u_n^T A v_n = \sigma_n = 0.&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found, through some experimentation, that applying inverse iteration to the shifted (and thus full-rank) matrix $A^T - \sigma I$ yields the left singular vector very quickly, often in one or two iterations.  In Matlab code:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x(:,1) = rand(n,1); &#xA;for k = 1:MAXIT&#xA;    x(:,k+1) = ( A' - sigma*eye(n) )\x(:,k); &#xA;    x(:,k+1) = x(:,k+1)/norm(x(:,k+1)); &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Usually, this iteration converges to small enough tolerances ($|| x_k^T A|| = 10^{-9}$ or so) in one or two iterations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this converges so rapidly because the largest and second largest singular values of $(A^T - \sigma I)^{-1}$ are spaced by $\sigma^{-1}$ which can be made large, but I have not been able to prove that this inverse iteration should converge.  I'm sure there is some clever way of using the SVD to show this convergence as in the standard proof of the equivalent inverse iteration for the eigenvectors, but I can't come up with it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help, ideas, or thoughts on why this works, and how I could prove this works would be helpful. &lt;/p&gt;&#xA;" OwnerUserId="4879" LastEditorUserId="4879" LastEditDate="2015-04-18T13:39:37.110" LastActivityDate="2015-04-18T13:39:37.110" Title="Inverse iteration to find the null singular vector of a rank-deficient matrix" Tags="|linear-algebra|iterative-method|eigenvalues|svd|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="10463" PostTypeId="2" ParentId="5222" CreationDate="2014-01-06T11:54:41.220" Score="0" Body="&lt;p&gt;Here is what I did: the idea is to find the eigenvalues in a certain interval $[a,b]$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;choose a discretization step and a corresponding discretization of the interval and then calculate the values of the function on that discretization. This discretization must be done such that there is a reasonable equilibrium between the computation time and the precision I want. Each function computation calculates the determinant of a $100\times 100$ matrix which is obtained by evaluating a Bessel function over the entries of a distance matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;find the peaks among the calculated values&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;around each peak make a &lt;a href=&quot;http://en.wikipedia.org/wiki/Golden_section_search&quot; rel=&quot;nofollow&quot;&gt;golden search&lt;/a&gt; in order to find &lt;em&gt;one&lt;/em&gt; precise value close to the peak&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(the method is inspired from the work of Alves and Antunes: &lt;a href=&quot;http://msvlab.hre.ntou.edu.tw/cite/CMC-Alves-2005.pdf&quot; rel=&quot;nofollow&quot;&gt;http://msvlab.hre.ntou.edu.tw/cite/CMC-Alves-2005.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is very accurate (errors of the order $10^{-6}$) and reasonably fast (for this precision) when all the eigenvalues are simple. Things get a bit messy when eigenvalues are double and worse if they are almost double. If an eigenvalue is double then looking at the kernel of $A_\lambda$ can give the multiplicity of $\lambda$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the eigenvalues are almost double, the kernel of $A_\lambda$ does not &lt;em&gt;see&lt;/em&gt; the eigenvalue $\lambda\pm \varepsilon$, so the only way I managed to solve this is by choosing an interval $(\lambda-\delta,\lambda+\delta)$ and make an even smaller discretization in which the almost double eigenvalues will be visible.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, in my work (of optimizing eigenvalues), eigenvalues almost always tend to get double at the optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have any suggestions on how could I improve the presented algorithm, they are more than welcome.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2014-01-06T11:54:41.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10464" PostTypeId="1" AcceptedAnswerId="10468" CreationDate="2014-01-06T13:48:02.570" Score="2" ViewCount="209" Body="&lt;p&gt;Where can I find documentation for the P*GEMR2D routines in Fortran? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I've found:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://netlib.org/scalapack/slug/node165.html#SECTION04851000000000000000&quot; rel=&quot;nofollow&quot;&gt;Scalapack UG&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.netlib.org/scalapack/explore-html/d9/d30/pdgemr2_8c_source.html&quot; rel=&quot;nofollow&quot;&gt;Undocumented related source&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=2&amp;amp;t=611&quot; rel=&quot;nofollow&quot;&gt;Unanswered forum post&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="521" LastActivityDate="2014-01-06T23:35:20.757" Title="*GEMR2D documentation (scalapack)" Tags="|documentation|scalapack|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10465" PostTypeId="2" ParentId="10449" CreationDate="2014-01-06T14:18:20.040" Score="11" Body="&lt;p&gt;I think I'd rather see it done this way. Since $A$ is invertible, then $\min_i\sigma_i&amp;gt;0$, $$&#xA;\min_i\sigma_i = \inf_{x\neq 0}\frac{\|Ax\|_2}{\|x\|_2} \quad\Longleftrightarrow&#xA;\frac{1}{\min_i\sigma_i} = \sup_{x\neq 0}\frac{\|x\|_2}{\|Ax\|_2}.&#xA;$$&#xA;Then we have&#xA;$$&#xA;\frac{1}{\min_i\sigma_i} &#xA;= \sup_{x\neq 0}\frac{\|x\|_2}{\|Ax\|_2}&#xA;= \sup_{A^{-1}y\neq 0}\frac{\|A^{-1}y\|_2}{\|y\|_2}&#xA;= \sup_{y\neq 0}\frac{\|A^{-1}y\|_2}{\|y\|_2}&#xA;= \|A^{-1}\|_2.&#xA;$$&#xA;where we have made the substitution $Ax=y$, and utilized the fact that $A^{-1}y=0\Longleftrightarrow y=0$, again since $A$ is invertible.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2014-01-06T14:18:20.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10466" PostTypeId="1" AcceptedAnswerId="27369" CreationDate="2014-01-06T18:51:44.603" Score="1" ViewCount="310" Body="&lt;p&gt;my question is simple, in openfoam we have the function foamToVTK which creates vtk files for each timestep of the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have a huge VTK tar file (has atleast 100 timesteps), i have system 0 and constant folders can we convert those vtk files back to time folders?&lt;/p&gt;&#xA;" OwnerUserId="6878" LastActivityDate="2017-07-09T10:52:31.323" Title="a reverse of foamToVTK tool?" Tags="|openfoam|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10467" PostTypeId="1" CreationDate="2014-01-06T19:00:48.433" Score="2" ViewCount="162" Body="&lt;p&gt;I am generating random weight as per e.g. below. The I generate a set of 3 values say 100, 250, 300 and I multiple them with the weights below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initial population.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1. 0.1 0.7 0.2&#xA;&#xA;2. 0.4 0.5 0.1&#xA;&#xA;3. 0.3 0.5 0.2&#xA;&#xA;4. 0.6 0.1 0.3&#xA;&#xA;5  0.3 0.3 0.4&#xA;&#xA;6. 0.2 0.2 0.6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Multiplication of weight&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1. 0.1*100 + 0.7*250 + 0.2*300 = 245&#xA;&#xA;2. 0.4*100 + 0.5*250 + 0.1*300 = 195&#xA;&#xA;3. 0.3*100 + 0.5*250 + 0.2*300 = 215&#xA;&#xA;4. 0.6*100 + 0.1*250 + 0.3*300 = 175&#xA;&#xA;5. 0.3*100 + 0.3*250 + 0.4*300 = 225&#xA;&#xA;6. 0.2*100 + 0.2*250 + 0.6*300 = 250&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So after the multiplication we selected the individual 6 as the best combination of weights. This method works fine for us but we are trying to link it to any available algorithm. Does this algorithm have a specific name? Any idea or help on this?  Is this not a genetic algorithm?&lt;/p&gt;&#xA;" OwnerUserId="7007" LastEditorUserId="20688" LastEditDate="2019-06-22T22:38:31.243" LastActivityDate="2019-06-22T22:38:31.243" Title="What is the name of the optimization algorithm that uses random sampling?" Tags="|optimization|algorithms|random-sampling|" AnswerCount="1" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10468" PostTypeId="2" ParentId="10464" CreationDate="2014-01-06T23:35:20.757" Score="2" Body="&lt;p&gt;The Scalapack UG link brings up a misleading web page. It looks as if there isn't any Fortran documentation, when I suppose the page is instead intended to be a title page. Clicking on the &quot;Next&quot; button in the upper left-hand corner brings you to the first page of Fortran interface documentation for &lt;code&gt;P_GEMR2D&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-01-06T23:35:20.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10469" PostTypeId="1" CreationDate="2014-01-07T12:35:38.147" Score="5" ViewCount="207" Body="&lt;p&gt;Suppose I want to set up an experiment to measure the performance of some numerical code on a desktop machine running Linux/Windows/MacOS. What kind of environment should I arrange in order to get correct timings? More precisely,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what applications/services should be closed?&lt;/li&gt;&#xA;&lt;li&gt;what priority the considered process should have?&lt;/li&gt;&#xA;&lt;li&gt;how many runs should I typically perform to average the timing?&lt;/li&gt;&#xA;&lt;li&gt;are there any specific issues with parallel CPU/GPU computations?&lt;/li&gt;&#xA;&lt;li&gt;anything else?..&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, all these issues are platform specific. The interesting question for me is also how Linux and Windows compare at that point.&lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2014-01-07T17:52:20.130" Title="What is the correct way of performing numerical experiments on desktops?" Tags="|performance|benchmarking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10470" PostTypeId="2" ParentId="10469" CreationDate="2014-01-07T14:29:43.653" Score="12" Body="&lt;p&gt;I think it depends on your purpose.  If you are trying to assess the overall performance of the code or environment, then I'd encourage you to run it however you think most people will run it on a desktop environment: leave things open but make sure nothing is crunching in the background or hogging all the memory.  The biggest culprits, in my experience, are backup tools during archive cycles and browsers with lots of and/or runaway tabs.  If you're afraid that the performance difference is getting lost in the noise, then it probably isn't significant from a desktop user point of view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are trying to evaluate the performance of a specific, potentially small code or environment change, then there is a trade-off: you can be more careful with shutting everything else down to reduce noise or you can try to isolate or emphasize that change to boost the `signal' in your experiment.  I tend to prefer the latter, but your mileage may vary.  This is related to profiling, for example &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1449/profiling-cfd-codes-with-callgrind&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1695/reference-request-for-profiling-high-performance-computing-codes&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone else can give a better answer for getting the most precise timings, but for desktop experiments, I think it is overkill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things to look out for:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Hyper-threading&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hyper-threading&lt;/a&gt; can lead to confusing results.  Though, if users won't bother turning it off, it could be relevant for overall time. (disable in BIOS)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Intel_Turbo_Boost&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turbo-boost&lt;/a&gt; poses similar issues to hyper-threading. (disable in BIOS)&lt;/li&gt;&#xA;&lt;li&gt;Threaded BLAS libraries can make 'single core' runs parallel (disable with OMP_NUM_THREADS=1 in environment)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="521" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-01-07T17:52:20.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10471" PostTypeId="2" ParentId="10462" CreationDate="2014-01-07T15:17:24.487" Score="3" Body="&lt;p&gt;Just collecting the answers contained in the comments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The left and right singular vectors of $A$ are exactly the left and right kernel vectors of $A$. Since the assumption is that the kernels have dimension $1$ and $A$ is well conditioned outside the kernel, one can compute the kernel vectors using the usual, pivoted, matrix factorizations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the LU factorization, $A=PLU$, $P$ a permutation matrix, $L$ lower triangular with $1$-diagonal, $Ax=0$ is equivalent to $Ux=0$. The last row of $U$ including the diagonal element is zero, thus choosing $x_n=1$ and substituting backwards results in a kernel vector $x$ and $v_n=x/\|x\|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y^TA=0$ is equivalent to $(y^TPL)R=0$, which has the nontrivial solution $(y^TPL)=e_n^T$ or $L^T(P^Ty)=e_n$. $L^T$ is a regular triangular matrix, thus this is easily solvable and $y=P(L^{-T}e_n)$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the QR decomposition, row pivoting is without effect, so column pivoting needs to be applied, the factorization is then $AP=QR$. $Ax=0$ is equivalent to $R(P^Tx)=0$ which can be solved as before for $P^Tx$, and $y^TA=0$ is equivalent to $(y^TQ)R=0$, again with the obvious solution $y^TQ=e_n^T$ or $y=Qe_n$&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2014-01-07T15:17:24.487" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10472" PostTypeId="2" ParentId="10467" CreationDate="2014-01-07T18:15:30.583" Score="1" Body="&lt;p&gt;The algorithm you have chosen for your optimization problem sounds like a variant of the family of algorithms known as &lt;a href=&quot;http://en.wikipedia.org/wiki/Random_optimization&quot; rel=&quot;nofollow&quot;&gt;random optimization&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to also add that since your objective and constraints seem to both be linear (please correct me if I'm wrong), your problem can be casted as a &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_programming&quot; rel=&quot;nofollow&quot;&gt;Linear Programming&lt;/a&gt; problem, which is a well studied problem with more efficient algorithms than the one you propose.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-01-07T22:09:13.770" LastActivityDate="2014-01-07T22:09:13.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10474" PostTypeId="1" CreationDate="2014-01-08T01:20:50.557" Score="1" ViewCount="179" Body="&lt;p&gt;I am looking for a 2D static problem (i.e. planar stress/strain) with known analytical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose for that is to verify my self-written code in matlab for solving 2D static problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references would be welcome.&lt;/p&gt;&#xA;" OwnerUserId="1152" LastActivityDate="2014-01-08T09:47:52.120" Title="A 2D static problem with known analytical solution" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10476" PostTypeId="2" ParentId="7685" CreationDate="2014-01-08T09:18:10.243" Score="0" Body="&lt;p&gt;I have also found distmesh in Matlab that appears to be able to do this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://persson.berkeley.edu/distmesh/&quot; rel=&quot;nofollow&quot; title=&quot;Distmesh homepage&quot;&gt;Distmesh Homepage&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does tesselations for finite element meshes (via QHull) but with a nice interface for defining areas / surfaces based on distance functions. Better for cases where you want to define a surface mathematically and do not mind where the internal nodes are.&lt;/p&gt;&#xA;" OwnerUserId="4575" LastActivityDate="2014-01-08T09:18:10.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10477" PostTypeId="2" ParentId="10474" CreationDate="2014-01-08T09:47:52.120" Score="2" Body="&lt;p&gt;Use the method of manufactured solutions. Here is a &lt;a href=&quot;http://prod.sandia.gov/techlib/access-control.cgi/2000/001444.pdf&quot; rel=&quot;nofollow&quot;&gt;link to a Sandia technical report&lt;/a&gt; describing the method. It will give you considerably more flexibility in testing your code for the order-of-accuracy of your numerical methods.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-01-08T09:47:52.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10479" PostTypeId="1" AcceptedAnswerId="30983" CreationDate="2014-01-08T11:30:12.307" Score="4" ViewCount="345" Body="&lt;p&gt;Quadrature-free DG method using nodal Lagrangian basis are computationally very efficient. I have seen many papers using this method for linear PDE but almost no literature for non-linear PDE like compressible Euler equations. The book of Hesthaven &amp;amp; Warburton discusses this and talks of errors in quadrature-free method for non-linear problems and the need to use some filter. The few papers I have seen on quadrature-free method like &lt;a href=&quot;http://etd.auburn.edu/etd/bitstream/handle/10415/3830/JF_thesis.pdf&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt; use the same degree basis for solution and flux and dont seem to use any filter (but they have artificial viscosity which is probably enough). I would like to ask the experts if there are any issues in using quadrature-free approach for non-linear problems when I am already using a non-linear limiter or artificial viscosity ? Do they give the same convergence rates as the quadrature-based methods ?&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2014-01-08T11:40:18.053" LastActivityDate="2020-02-27T23:01:50.190" Title="Quadrature and quadrature-free discontinuous galerkin method for non-linear PDE" Tags="|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10480" PostTypeId="1" CreationDate="2014-01-08T11:30:53.053" Score="3" ViewCount="705" Body="&lt;p&gt;I'm trying to use immersed boundary method for the 3D flow problem (Navier-Stokes equations), but I'm maybe misunderstood something in this method. Main principles I took from this &lt;a href=&quot;http://ebookbrowsee.net/peskin-c-s-numerical-analysis-of-blood-flow-in-the-heart-pdf-d42966243&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt;. I use the continuous forcing approach and the quasi-rigid immersed boundary, i.e. the boundary force computed from this equation &lt;img src=&quot;https://i.stack.imgur.com/2kQAN.png&quot; alt=&quot;enter image description here&quot;&gt;, where &lt;strong&gt;&lt;em&gt;k&lt;/em&gt;&lt;/strong&gt; is the stiffness, &lt;strong&gt;&lt;em&gt;x ref&lt;/em&gt;&lt;/strong&gt; is the initial node position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My trouble is that in the first time layer I have zero boundary force (because I don't know this force) and nonzero fluid velocity. So the fluid shifts the all boundary nodes and I have something like this (this is the 3D flow around a cylinder that was projected on the Oz axis)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nvbC5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the all nodes were shifted they all have the same force including the right half circle. So the streamlines twisted through the immersed boundary (a cylinder).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I avoid this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPD&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have steady flow problem and maybe it's the source of troubles. I thought that the series of calculation of steady problem can approximates the unsteady and allow to use immersed boundary method, but now I'm not sure about this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPD&lt;/strong&gt;&#xA;I have reached the more correct flow, when stiffness has been increased. I didn't understand this, because my calculation had not enough time steps. &lt;/p&gt;&#xA;" OwnerUserId="4955" LastEditorUserId="4955" LastEditDate="2014-01-13T13:41:09.197" LastActivityDate="2014-01-13T13:41:09.197" Title="Immersed boundary method" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10481" PostTypeId="1" CreationDate="2014-01-08T11:49:19.577" Score="3" ViewCount="362" Body="&lt;p&gt;I have this problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H_i(x_1,x_2,\dots, x_N) =  a_{ijk} x_j x_k + b_{ij} x_j + c_i = 0 \quad 1\leq i \leq N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I need to show that applying Newton-Raphson can fail to find even one real solution of this system for any N.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well if I apply Newton-Raphon, first I calculate the gradient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial H_i}{\partial x_l} = a_{ilk} x_k + a_{ijl} x_j + b_{il} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and solve for&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\boldsymbol{\Delta x} = -( \frac{\partial \boldsymbol{H}}{\partial \boldsymbol{x}})^{-1} \boldsymbol{H} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evaluating the gradient and the function at the initial estimation. The gradient is a second-order tensor. I do not know how this could fail. Maybe if the initial estimation did not converge, but that's a matter of choosing a good one. I assume the system has a solution, since the question focuses on finding the solutions, not on whether these exist or not. Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2018-12-16T07:22:37.543" Title="System of quadratic algebraic equations" Tags="|nonlinear-equations|newton-method|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10482" PostTypeId="1" AcceptedAnswerId="10485" CreationDate="2014-01-08T12:03:09.943" Score="2" ViewCount="139" Body="&lt;p&gt;Suppose $f$ is a infinite continuously differentiable map: $R^n\to R$, and $x,x_0 \in R^n$, then we have the following second order Taylor expansion of $f(x)$ at $x_0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x)\approx f(x_0)+(x-x_0)^T\nabla f(x_0)+\dfrac{(x-x_0)^T \nabla^2f(x_0)(x-x_0)}{2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the next iterm? &#xA;Do I need to use tensor? what is a simple representation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and how to conduct the multiplication between tensors and matrices and vectors?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6693" LastEditDate="2014-01-09T07:31:48.397" LastActivityDate="2014-01-09T07:31:48.397" Title="What is the more than 3rd order Taylor series approximation for a multi-variate function?" Tags="|matrix|approximation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10483" PostTypeId="2" ParentId="10332" CreationDate="2014-01-08T12:44:05.957" Score="1" Body="&lt;p&gt;I solved using the time independent perturbation theory. The corrected wavefunction in nondegenerate first order perturbation theory is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|\psi{_n}\rangle\approx|\psi_{n}^{(0)}\rangle+\sum_{k\neq n}\frac{\langle \psi_{k}^{(0)}|\hat H^{(1)}|\psi_{n}^{(0)}\rangle}{E_{n}^{(0)}-E_{k}^{(0)}}|\psi_{k}\rangle$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $|\psi_{n}^{(0)}\rangle$ is the original wavefunction and $|\psi_{k}^{(0)}\rangle$ is the wavefunction of the $k$ level. This means that the perturbation could lead the original state to have contributions from all other levels. The denominator shows that contributions will decrease as the energy difference increases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now there are two integrals to evaluate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\langle \psi_{1}^{(0)}|\hat H^{(1)}|\psi_{0}^{(0)}\rangle=\int_{-\infty}^{\infty}\psi_{1}^{(0)}(x)\hat H^{(1)}\psi_{0}^{(0)}(x)dx=0$=odd function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\langle \psi_{2}^{(0)}|\hat H^{(1)}|\psi_{2}^{(0)}\rangle=\int_{-\infty}^{\infty}\psi_{1}^{(0)}(x)\hat H^{(1)}\psi_{0}^{(0)}(x)dx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting all together, the ground state wavefunction interms of the harmonic oscillator's ground state gives&#xA;$|\psi_{0}\rangle\approx|\psi_{0}^{(0)}\rangle-\frac{3}{4\sqrt{2}}\frac{\lambda} {\alpha^2(2\hbar \omega+{\frac{39}{16}}{\frac{\lambda}{\alpha^2})}}|\psi_{2}^{(0)}\rangle$&lt;/p&gt;&#xA;" OwnerUserId="6871" LastEditorUserId="6871" LastEditDate="2014-01-08T12:53:49.327" LastActivityDate="2014-01-08T12:53:49.327" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10484" PostTypeId="1" AcceptedAnswerId="10862" CreationDate="2014-01-08T15:05:32.160" Score="5" ViewCount="383" Body="&lt;p&gt;If I use &lt;a href=&quot;http://www.netlib.org/scalapack/&quot; rel=&quot;nofollow&quot;&gt;scalapack&lt;/a&gt; and &lt;a href=&quot;http://www.netlib.org/scalapack/pblas_qref.html&quot; rel=&quot;nofollow&quot;&gt;pblas&lt;/a&gt;, and the code is run in serial (1x1 &lt;a href=&quot;http://www.netlib.org/blacs/&quot; rel=&quot;nofollow&quot;&gt;blacs&lt;/a&gt; &lt;a href=&quot;http://www.netlib.org/blacs/BLACS/Pgrid.html&quot; rel=&quot;nofollow&quot;&gt;process grid&lt;/a&gt;), how well does scalapack and pblas revert to the performance of &lt;a href=&quot;http://www.netlib.org/lapack/&quot; rel=&quot;nofollow&quot;&gt;lapack&lt;/a&gt;/&lt;a href=&quot;http://www.netlib.org/blas/&quot; rel=&quot;nofollow&quot;&gt;blas&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am particularly interested in the &lt;a href=&quot;http://www.netlib.org/scalapack/explore-html/d7/dff/pzhegvx_8f_source.html&quot; rel=&quot;nofollow&quot;&gt;pzhegvx&lt;/a&gt; and &lt;a href=&quot;http://www.netlib.org/scalapack/explore-html/d8/d3b/pzheevx_8f_source.html&quot; rel=&quot;nofollow&quot;&gt;pzheevx&lt;/a&gt; routines with $n \approx 1000$ and the &lt;a href=&quot;http://www.netlib.org/scalapack/explore-html/d8/d38/pzgemm___8c_source.html&quot; rel=&quot;nofollow&quot;&gt;pzgemm&lt;/a&gt; and &lt;a href=&quot;http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=/com.ibm.cluster.pessl.v4r2.pssl100.doc/am6gr_lsyrk.htm&quot; rel=&quot;nofollow&quot;&gt;pzherk&lt;/a&gt; routines with $m,n \approx 1000$ and $k \approx 100$.  These are the lower bounds in the space of reasonable inputs.  Otherwise, as Jeff points out, scalapack wouldn't make a whole lot of sense.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2014-01-24T14:20:53.837" LastActivityDate="2014-02-23T14:44:04.540" Title="How gracefully does scalapack/pblas revert to lapack/blas in serial" Tags="|blas|scaling|scalapack|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10485" PostTypeId="2" ParentId="10482" CreationDate="2014-01-08T15:13:48.913" Score="1" Body="&lt;p&gt;I think multi-indices are the typical way to expand this out further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://en.wikipedia.org/wiki/Taylor%27s_theorem#Taylor.27s_theorem_for_multivariate_functions&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="103" LastEditDate="2014-01-08T20:43:06.290" LastActivityDate="2014-01-08T20:43:06.290" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10486" PostTypeId="2" ParentId="7901" CreationDate="2014-01-08T15:55:04.327" Score="-1" Body="&lt;p&gt;LU factorization exists only if the matrix is invertible (Not LUP). A common way to do this in Lapack is to try to obtain the LU decomposition. Many Lapack libraries will throw you an info code, stating the reason of failure. If your matrix is not invertible LU decomposition output &quot;info&quot; will not be 0 and you can check this flag to decide if your matrix is invertible or not.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-01-08T15:55:04.327" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10488" PostTypeId="1" AcceptedAnswerId="10491" CreationDate="2014-01-08T18:02:44.353" Score="2" ViewCount="933" Body="&lt;p&gt;I'm looking for an algorithm to solve&#xA;the underdetermined system of linear equations&#xA;$$\mathbf{A}\,\mathbf{x} = \mathbf{b}$$ &#xA;with &#xA;$\mathbf{A} \in \mathbb{R}^{n\times n}$, $\mathbf{b} \in &#xA;\mathbb{R}^{n\times 1}$ and $rank(\mathbf{A}) &amp;lt; n$.&#xA;Clearly, there exists an infinite number of solution vectors &#xA;$\mathbf{x}$ (or no solution at all).&#xA;However, if $\mathbf{A}$ is sparse, &#xA;some components of $\mathbf{x} = [x_1,\ldots,x_n]$ &#xA;might be determined uniquely.&#xA;Example:&#xA;$$ \begin{bmatrix}2 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0\\1&amp;amp;0&amp;amp;2&amp;amp;4\\1&amp;amp;0&amp;amp;2&amp;amp;4\\1 &amp;amp; 2 &amp;amp; 0 &amp;amp; 0\end{bmatrix}&#xA;\,\begin{bmatrix}x_1 \\x_2 \\ x_3 \\ x_4\end{bmatrix} = &#xA;\begin{bmatrix}0 \\ 1 \\ 1 \\ 5\end{bmatrix}$$&#xA;How do I find those components $x_i$ of&#xA;$\mathbf{x}$ which have this property (in this example $x_1$ and $x_2$)?&#xA;I'd be grateful for pointers to the relevant literature.&lt;/p&gt;&#xA;" OwnerUserId="7025" LastActivityDate="2014-01-10T07:27:03.977" Title="Sparse, underdetermined system of linear equations" Tags="|linear-algebra|algorithms|linear-solver|sparse-matrix|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10491" PostTypeId="2" ParentId="10488" CreationDate="2014-01-08T19:54:12.693" Score="4" Body="&lt;p&gt;The problem here is to determine where or not there are any components of $x$ that are uniquely determined by the linear system of equations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's one approach:  Compute a basis for the null space of $A$ by using a rank revealing QR factorization or by SVD or by any of a number of other methods.  Let $U$ be the matrix of size $n$ by $p$ that contains as its columns the basis for $N(A)$.  If row $i$ of $U$ is $0$, then $x_{i}$ is uniquely determined by the equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this won't work very well in practice because of small round off errors in the determination of the basis for $N(A)$.  For example, using MATLAB, &#xA;I computed the null space for your example matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A=[2 -1 0 0; 1 0 2 4; 1 0 2 4; 1 2 0 0]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A =&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 2    -1     0     0&#xA; 1     0     2     4&#xA; 1     0     2     4&#xA; 1     2     0     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;null(A)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ans =&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;-5.5511e-17&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8.9443e-01&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-4.4721e-01&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the null space is one dimensional.  Looking at the basis vector for this null space, it's clear that the $x_{1}$ is uniquely determined by the equations.  However, the component of the null space basis vector corresponding to $x_{2}$ is small but not quite exactly 0.  Thus it isn't clear that $x_{2}$ is uniquely determined by the equations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way of approaching this is to compute the RREF of the matrix $A$ in exact rational arithmetic and then obtain a basis for the null space of $A$.  In this case, we get:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;rref(A)&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ans =&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1     0     0     0&#xA; 0     1     0     0&#xA; 0     0     1     2&#xA; 0     0     0     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thus x=[0; 0; -2; 1] is a basis vector for the one dimensional $N(A)$.  It's clear from this that $x_{1}$ and $x_{2}$ are uniquely determined by the equations.    We were lucky in this case that the RREF computed by MATLAB produced an exact answer- for a much larger matrix we'd probably see some round-off error and then the use of higher (or infinite) precision arithmetic would be necessary.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2014-01-08T23:43:02.407" LastActivityDate="2014-01-08T23:43:02.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10492" PostTypeId="2" ParentId="10488" CreationDate="2014-01-08T20:45:26.040" Score="-1" Body="&lt;p&gt;You might want to minimize $L_0$ norm $||x||_0$ s.t. $Ax=b$, as to find the the sparsest solution. Other $p$-norms also applicable depending on the case. Some resources to follow up are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.vanderbilt.edu/~foucart/FL08.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.vanderbilt.edu/~foucart/FL08.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;http://ftp.math.uga.edu/~mjlai/papers/surveySS.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ftp.math.uga.edu/~mjlai/papers/surveySS.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods should already cover the cases you like to solve.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-01-08T20:45:26.040" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10493" PostTypeId="1" CreationDate="2014-01-08T21:47:06.440" Score="6" ViewCount="1514" Body="&lt;p&gt;I'm acquiring data which looks like this&lt;img src=&quot;https://i.stack.imgur.com/2wCQY.png&quot; alt=&quot;LC data&quot;&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically it should be possible to fit it with a Lissajous curve, which is typically defined in the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x = A \sin(a t + \delta)$, and $y = B \sin(b t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, neither the cftools or optimization toolbox seem ready to do this or at least I do not know how to parse it in such a way as to get an algorithm to try fitting variables $A, a, t, \delta, B, b$ to my data.&#xA;If I try joining these two equations together, then &lt;em&gt;something&lt;/em&gt; breaks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I solve $x$ for $t$, then $$ t = \frac{\sin^{-1}(\frac{x}{A}) - \delta + 2 \pi n}{a} $$&#xA;and then plugging this into $y$ to get: $$y = B \sin(b(\frac{\sin^{-1}(\frac{x}{A}) - \delta + 2 \pi n}{a}))$$, but I had no success using this.&lt;/p&gt;&#xA;" OwnerUserId="4318" LastEditorUserId="4318" LastEditDate="2014-01-09T00:21:07.140" LastActivityDate="2014-01-09T19:29:08.343" Title="Fit my data to Lissajous curve in Matlab" Tags="|data-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10494" PostTypeId="1" CreationDate="2014-01-08T23:01:34.530" Score="3" ViewCount="2015" Body="&lt;p&gt;I have a set of dates, position angles ($\theta$) and angular separations ($\rho$) for visual binary star. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1994.3132  57.0    0.097&#xA;1996.1783  73.4    0.051&#xA;2006.1938 182.0    0.126&#xA;2009.2597 192.9    0.152&#xA;etc.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What are algorithms to calculate orbital elements and to plot the orbit of the binary star?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there available codes (or programs) for this calculations? Python is preferable )&lt;/p&gt;&#xA;" OwnerUserId="7029" LastActivityDate="2014-01-14T01:38:20.620" Title="How to plot orbit of binary star and calculate its orbital elements?" Tags="|python|numerical-analysis|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10495" PostTypeId="1" CreationDate="2014-01-08T23:16:55.403" Score="2" ViewCount="4079" Body="&lt;p&gt;I want to solve the Schrodinger via the Numerov Method but I had some troubles. I'm programing in C++, so here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include&amp;lt;cstdlib&amp;gt;&#xA;#include&amp;lt;iostream&amp;gt;&#xA;#include&amp;lt;cmath&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;double x_min=-4.0 , x_max=4.0;&#xA;int N=2000;             &#xA;double r=(x_max-x_min)/(1.0*N); &#xA;double d=2.0;&#xA;double p=0.4829;    // 2m/(hbar^2)&#xA;double Vo=20.0; // Altura del pozo&#xA;&#xA;double x_m=0.1;                                     //Matching point&#xA;int i_x_m=(x_m-x_min)/r;&#xA;&#xA;double Control=-123456789;&#xA;&#xA;double SlopeLeft,SlopeRight;&#xA;&#xA;double PAR;&#xA;&#xA;double K2(double x, double E);&#xA;double NumerovL(int i, double k21, double k22, double k23, double Y[]);&#xA;double NumerovR(int i, double k21, double k22, double k23, double Y[]);&#xA;double FuncLeft(double E, double Y[]);&#xA;double FuncRight(double E, double Y[]);&#xA;void PrintFunc(double Y[]);&#xA;void Normalizar(double Y[]);&#xA;double f(double E, double Y[]);&#xA;double Biseccion(double a, double b, double Y[]);&#xA;&#xA;//=========================MAIN===============================&#xA;&#xA;int main(int argc, char **argv)&#xA;{  &#xA;&#xA;double Y[N+1];       // Función de Onda&#xA;&#xA;double paso=0.02;   // Escala   en la que se varia la energía&#xA;double Eo=0;&#xA;&#xA;for(double E=0 ; E&amp;lt;=Vo ; E+=paso) //Cálculo de las funciones IMPARES&#xA;{&#xA;    PAR=-1;&#xA;    Eo=Biseccion(E,E+paso,Y);&#xA;&#xA;    if(Eo != Control &amp;amp;&amp;amp; SlopeRight*SlopeLeft&amp;lt;0.) &#xA;    {&#xA;        Y[i_x_m]=FuncRight(Eo,Y);&#xA;        Y[i_x_m]=FuncLeft(Eo,Y);&#xA;&#xA;&#xA;        Normalizar(Y);&#xA;&#xA;        PrintFunc(Y);   &#xA;    }&#xA;&#xA;}&#xA;&#xA;&#xA;for(double E=0 ; E&amp;lt;=Vo ; E+=paso)   //Cálculo de las funciones PARES&#xA;{&#xA;    PAR=1;&#xA;    Eo=Biseccion(E,E+paso,Y);&#xA;&#xA;    if(Eo != Control &amp;amp;&amp;amp; SlopeRight*SlopeLeft&amp;gt;0.)&#xA;    {&#xA;        Y[i_x_m]=FuncRight(Eo,Y);&#xA;        Y[i_x_m]=FuncLeft(Eo,Y);&#xA;&#xA;        Normalizar(Y);&#xA;&#xA;        PrintFunc(Y);   &#xA;    }&#xA;}&#xA;&#xA;&#xA;  return 0;&#xA;}&#xA;&#xA; //=========================FUNCIONES===============================&#xA;&#xA; double K2(double x, double E) &#xA; {&#xA;  double k2;&#xA;&#xA;if(fabs(x)&amp;lt;=d)&#xA;{&#xA;    k2=p*E;&#xA;    return k2;&#xA;}&#xA;else&#xA;{&#xA;    k2=p*(E-Vo);&#xA;    return k2;&#xA;}&#xA;}&#xA;&#xA;double NumerovL(int i, double k21, double k22, double k23, double Y[]) &#xA;{ // Para la función de Onda Izquierda&#xA;double A1,B1,C1,N;&#xA;A1=2.0*(1.0-(5.0/12.0)*r*r*k21)*Y[i-1];&#xA;B1=(1.0+(1.0/12.0)*r*r*k22)*Y[i-2];&#xA;C1=1.0+(1.0/12.0)*r*r*k23;&#xA;N=(A1-B1)/(C1);&#xA;return N;&#xA;}&#xA;&#xA;double NumerovR(int i, double k21, double k22, double k23, double Y[]) &#xA;{ // Para la función de Onda Derecha&#xA;double A1,B1,C1,N;&#xA;A1=2.0*(1.0-(5.0/12.0)*r*r*k21)*Y[i+1];&#xA;B1=(1.0+(1.0/12.0)*r*r*k22)*Y[i+2];&#xA;C1=1.0+(1.0/12.0)*r*r*k23;&#xA;N=PAR*(A1-B1)/(C1);&#xA;return N;&#xA;}&#xA;&#xA;double FuncLeft(double E, double Y[])&#xA;{&#xA;double k21,k22,k23,Yleft,b;&#xA;&#xA;b=sqrt(p*(Vo-E));&#xA;&#xA;Y[0]=exp(b*x_min);&#xA;Y[1]=exp(b*(x_min+r));&#xA;&#xA;&#xA;for(int i=2 ; i&amp;lt;i_x_m ; i++) // Se calcula la función de Onda Izquierda&#xA;{&#xA;    k21=K2(x_min+(i-1)*r,E);&#xA;    k22=K2(x_min+(i-2)*r,E);&#xA;    k23=K2(x_min+i*r,E);&#xA;&#xA;    Y[i]=NumerovL(i,k21,k22,k23,Y);&#xA;&#xA;    if(i==i_x_m-1) //Función de Onda Izquierda en el Matching point&#xA;    {&#xA;        k21=K2(x_min+(i)*r,E);&#xA;        k22=K2(x_min+(i-1)*r,E);&#xA;        k23=K2(x_min+(i+1)*r,E);&#xA;&#xA;        Yleft=NumerovL(i+1,k21,k22,k23,Y);&#xA;    }&#xA;}&#xA;&#xA;SlopeLeft=(Yleft-Y[i_x_m-1])/r;&#xA;&#xA;return Yleft;&#xA;}&#xA;&#xA;double FuncRight(double E, double Y[])&#xA;{&#xA;double k21,k22,k23,Yright,b;&#xA;&#xA;b=sqrt(p*(Vo-E));&#xA;&#xA;Y[N]=PAR*exp(-b*(x_min+N*r));   &#xA;Y[N-1]=PAR*exp(-b*(x_min+(N-1)*r));&#xA;&#xA;for(int i=N-2 ; i&amp;gt;i_x_m; i--) // Se calcula la función de Onda Derecha&#xA;{&#xA;    k21=K2(x_min+(i+1)*r,E);&#xA;    k22=K2(x_min+(i+2)*r,E);&#xA;    k23=K2(x_min+i*r,E);&#xA;&#xA;    Y[i]=PAR*NumerovR(i,k21,k22,k23,Y);&#xA;&#xA;    if(i==i_x_m+1) //Función de Onda Derecha en el Matching point&#xA;    {&#xA;        k21=K2(x_min+(i)*r,E);&#xA;        k22=K2(x_min+(i+1)*r,E);&#xA;        k23=K2(x_min+(i-1)*r,E);&#xA;&#xA;        Yright=NumerovR(i-1,k21,k22,k23,Y);&#xA;    }&#xA;}&#xA;&#xA;SlopeRight=PAR*(Y[i_x_m+1]-Yright)/r;&#xA;&#xA;return Yright;&#xA;}&#xA;&#xA;void PrintFunc(double Y[])&#xA;{&#xA;  for(int i=0 ; i&amp;lt;=N+1 ; i++)&#xA; {&#xA;    cout &amp;lt;&amp;lt; x_min+i*r &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; Y[i] &amp;lt;&amp;lt; endl;&#xA; }&#xA;}&#xA;&#xA;void Normalizar(double Y[])&#xA;{&#xA;  double S=0;&#xA;&#xA; for(int i=0 ; i&amp;lt;=N+1 ; i++)&#xA; {&#xA;     S += Y[i]*Y[i]*r;&#xA; }  &#xA;&#xA;S=sqrt(S);&#xA;&#xA;  for (int i=0 ; i&amp;lt;=N+1 ; i++)&#xA; {&#xA;     Y[i]=Y[i]/S;&#xA; }&#xA;&#xA;}&#xA;&#xA;double f(double E, double Y[])&#xA;{&#xA;double F;&#xA;&#xA;F=FuncLeft(E,Y)-PAR*FuncRight(E,Y);&#xA;&#xA;return F;&#xA;}&#xA;&#xA; double Biseccion(double a, double b, double Y[])&#xA; {&#xA;&#xA;  double Tol=0.00001; //Tolerancia para encontrar la raiz&#xA;&#xA; double RET=-123456789;&#xA;&#xA; if(f(a,Y)*f(b,Y)&amp;lt;0)&#xA; {&#xA;     while(fabs(a-b)&amp;gt;Tol)&#xA;    {&#xA;         double x_m,fa,fm;&#xA;&#xA;        fa=f(a,Y);&#xA;        x_m=(a+b)/2.0;&#xA;        fm=f(x_m,Y);&#xA;        //fb=f(b);&#xA;&#xA;        if(fa*fm&amp;lt;0)&#xA;        {&#xA;            b=x_m;&#xA;            //RET=b;&#xA;        }&#xA;        else&#xA;        {&#xA;            a=x_m;&#xA;            //RET=a;&#xA;        }&#xA;    }&#xA;    RET=a;&#xA;}   &#xA;&#xA;return RET;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically the code takes all the energies, i.e. $0&amp;lt;E&amp;lt;Vo$ and the fuction &quot;Biseccion&quot; applies the Bisection algorithm between an energy $E$ and $E+step$. So the fuction finds the eigen-energy for wich the left and right (from the &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerov%27s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerov Method&lt;/a&gt;) wave functions matches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code compiles perfectly but the problem arises when I want to plot the odd solutions. I obtain two satisfactory solutions but another two that it's function is continous but not it's derivative. Here is an example of the plot that I obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pjwj1.png&quot; alt=&quot;Eigenfunctions that the program calculates&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, there are two graphs that are not a satisfactory solution to the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be very thankful if somebody can help me with this problem.&lt;/p&gt;&#xA;" OwnerUserId="7030" LastEditorUserId="7030" LastEditDate="2014-01-09T16:37:14.967" LastActivityDate="2014-08-30T09:29:52.717" Title="Implementation of the Numerov Method for the 1D square well" Tags="|c++|quantum-mechanics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10496" PostTypeId="2" ParentId="10493" CreationDate="2014-01-09T01:59:37.887" Score="2" Body="&lt;p&gt;It seems that both $x$ and $y$ have an offset, since a Lissajous curve should oscillate around the origin, so: $x=A\sin{\left(at+\delta\right)}+x_0$ and $y=B\sin{\left(bt\right)}+y_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides that, your test equations do not seem to be correct. Because the &lt;a href=&quot;http://en.wikipedia.org/wiki/Lissajous_curve&quot; rel=&quot;nofollow&quot;&gt;ratio $\frac{a}{b}$&lt;/a&gt; determines the number of &quot;lobes&quot; (points where the curve crosses itself) of the cyclic curve. When assuming that the data contains noise/uncertainty and that at least one complete oscillation has been complete. It seems that there are no lobes, which means that $a=b$ and that there would be only be a phase shift between $x$ and $y$. However this should produce an &lt;a href=&quot;http://en.wikipedia.org/wiki/Lissajous_curve#Application_for_the_case_of_a_.3D_b&quot; rel=&quot;nofollow&quot;&gt;elliptical curve&lt;/a&gt;, which does not seems to be the case. This curve does look similar to a &lt;a href=&quot;http://en.wikipedia.org/wiki/Hysteresis&quot; rel=&quot;nofollow&quot;&gt;hysteresis&lt;/a&gt; curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;br&gt;&#xA;But to answer how you might fit data to a Lissajous curve we first have to look at why you can't calculate the vector $t$. The inverse of the sine function, $\sin^{-1}$, has actually &lt;a href=&quot;http://www.wolframalpha.com/input/?i=sin%28x%29%3Da&quot; rel=&quot;nofollow&quot;&gt;another solution&lt;/a&gt; beside the one you mentioned: $\sin{x}=a\rightarrow x=\sin^{-1}{a}+2\pi n\bigvee x=-\sin^{-1}{a}+2\pi+\pi$. These two slopes can be seen by &lt;a href=&quot;http://www.wolframalpha.com/input/?i=asin%28sin%28x%29%29&quot; rel=&quot;nofollow&quot;&gt;plotting $\sin^{-1}\left(\sin{(x)}\right)$&lt;/a&gt;, which gives a triangle wave. However the absolute value of the slopes of both data sets should be the same, so for most data points, using index $i$, the following should be true:&#xA;$$&#xA;\left|\frac{d}{di}t(i)\right|=\left|\frac{d}{di}\frac{1}{a}\left(\sin^{-1}\left(\frac{x(i)-x_0}{A}\right)-\delta\right)\right|&#xA;$$&#xA;Instead of taking the absolute value, you could also square both sides.&lt;br&gt;&#xA;However $t$ is still unknown, but this should also hold for $y$, so:&#xA;$$&#xA;\left|\frac{d}{di}\frac{b}{a}\left(\sin^{-1}\left(\frac{x(i)-x_0}{A}\right)-\delta\right)\right|-\left|\frac{d}{di}\sin^{-1}\left(\frac{y(i)-y_0}{B}\right)\right|=0&#xA;$$&#xA;However I could only think of one solution to be able to use $\frac{d}{di}$ while fitting in MATLAB. And that is taking the equivalent of &lt;code&gt;diff(x)&lt;/code&gt;, namely splitting $x$ up into two vectors: &lt;code&gt;x1 = x(1:end-1); x2 = x(2:end);&lt;/code&gt;. So the custom equation would look like this:&lt;br&gt;&#xA;&lt;code&gt;abs(b / a * (asin(x1 / A) - asin(x2 / A))) - abs(asin(y1 / B) - asin(y2 / B)) = 0&lt;/code&gt;&lt;br&gt;&#xA;This method loses information about $\delta$, but once you know all the other parameters it would be much easier to find $\delta$ with another method. Another downside of this approach is that you will end up with a total of 5 data sets, &lt;code&gt;x1&lt;/code&gt;, &lt;code&gt;x2&lt;/code&gt;, &lt;code&gt;y1&lt;/code&gt;, &lt;code&gt;y2&lt;/code&gt; and a vector of zeros. However MATLAB cftool can only handle 3 data inputs, so I am not sure if you could perform a fit using cftool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However according to &lt;a href=&quot;http://iopscience.iop.org/1367-2630/12/9/095009/fulltext/#nj347122s4&quot; rel=&quot;nofollow&quot;&gt;this report&lt;/a&gt; it is possible to fit a Lissajous curve using a least squares algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;http://research.microsoft.com/pubs/67845/ellipse-pami.pdf&quot; rel=&quot;nofollow&quot;&gt;report&lt;/a&gt; might also be interesting, since it is about least square fitting of ellipses (so $a=b$). I have only taken a quick look at it, so I am not 100% sure how useful it would be to you.&lt;/p&gt;&#xA;" OwnerUserId="7031" LastEditorUserId="7031" LastEditDate="2014-01-09T19:29:08.343" LastActivityDate="2014-01-09T19:29:08.343" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10497" PostTypeId="2" ParentId="7566" CreationDate="2014-01-09T10:13:15.637" Score="3" Body="&lt;p&gt;Maybe you can try this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/jakevdp/pypropack&quot; rel=&quot;nofollow&quot;&gt;https://github.com/jakevdp/pypropack&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a Python wrapper for the PROPACK package, which implements efficient partial singular value decompositions of large sparse matrices and linear operators.&lt;/p&gt;&#xA;" OwnerUserId="7034" LastActivityDate="2014-01-09T10:13:15.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10498" PostTypeId="2" ParentId="7566" CreationDate="2014-01-09T12:26:09.480" Score="5" Body="&lt;p&gt;Intel MKL implements the new Jacobi-SVD algorithm.&#xA;Here are the implemention details:&#xA;&lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn169.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.netlib.org/lapack/lawnspdf/lawn169.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.fernuni-hagen.de/MATHPHYS/veselic/downloads/j02.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.fernuni-hagen.de/MATHPHYS/veselic/downloads/j02.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the LAPACK routine:&#xA;&lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-732F9EE1-BCEC-4D9B-9B93-AF5499B21140.htm#DRMAC08-1&quot; rel=&quot;noreferrer&quot;&gt;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-732F9EE1-BCEC-4D9B-9B93-AF5499B21140.htm#DRMAC08-1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Work size is of course adjustable. You could call C functions from Python easily using Cython, SWIG or any other wrapping mechanism.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-01-09T12:26:09.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10499" PostTypeId="1" AcceptedAnswerId="10502" CreationDate="2014-01-09T14:52:15.597" Score="1" ViewCount="60" Body="&lt;p&gt;If the analytical form global minimum value of a nonconvex $C^{\infty} f:R^n\to R$ is already known, will it be easier to find its global minimizer $x^*\in R^n$?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2014-01-09T17:45:47.597" Title="If the global minimum value of a nonconvex $C^{\infty} f: R^n\to R$ is known,can it be easier to find the global minimizer?" Tags="|optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10500" PostTypeId="1" AcceptedAnswerId="10501" CreationDate="2014-01-09T17:32:27.163" Score="2" ViewCount="7050" Body="&lt;p&gt;I am looping an array using MPI. The problem is, i think that some processes are moving onto their next iteration before other precesses have finished theirs. This is causing me problems because data calculated by each process at each iteration is needed in the next. Is their a way that i can pause the processes at the end of the loop until all other have finished their current iteration? (Im using Fortran90 if it makes any difference)&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6669" LastActivityDate="2014-01-09T19:36:15.097" Title="Stop process from continuing until others have finished MPI" Tags="|fortran|mpi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10501" PostTypeId="2" ParentId="10500" CreationDate="2014-01-09T17:35:01.107" Score="5" Body="&lt;p&gt;An &lt;a href=&quot;http://www.open-mpi.org/doc/v1.5/man3/MPI_Barrier.3.php&quot;&gt;MPI_Barrier&lt;/a&gt; can be used to synchronize all processes in a communicator. Each of the processes has to wait till all other processes reach the barrier before all of them can proceed further:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MPI_BARRIER(COMM, IERROR)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here COMM is the communicator handle and IERROR the error status.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that sometimes there are more clever ways to deal with such dependencies algorithmically, but this is highly problem-dependent.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2014-01-09T17:41:32.033" LastActivityDate="2014-01-09T17:41:32.033" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10502" PostTypeId="2" ParentId="10499" CreationDate="2014-01-09T17:45:47.597" Score="5" Body="&lt;p&gt;Within a branch and bound scheme, having a good lower bound on the optimal value can be very helpful in speeding up the algorithm.  If your function is Lipschitz continuous with a known Lipschitz constant, and the region of interest can be bounded then you can readily apply branch and bound to the problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, the problem is hopeless, since it's possible to hide the global minimum in a $C^{\infty}$ bump that could be located anywhere in $R^{n}$ and which could have arbitrarily small support.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-09T17:45:47.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10503" PostTypeId="2" ParentId="10500" CreationDate="2014-01-09T19:36:15.097" Score="2" Body="&lt;p&gt;While Christian has given you the right answer, it's almost certainly not what you want. Unnecessary synchronization is guaranteed to limit the scalability of your code. If all processors need data from all other processors, then using a collective like MPI_Alltoall() might be more appropriate. If one task has received all the required data from other tasks, there's usually no harm in that task continuing even if others are still working on the previous iteration. In that case, you can probably use point-to-point communications only.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-01-09T19:36:15.097" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10504" PostTypeId="2" ParentId="10488" CreationDate="2014-01-09T20:22:14.497" Score="1" Body="&lt;p&gt;Use a standard sparse reordering of the equations and variables. If all goes well, the matrix then decomposes into diagonal blocks (and possibly some remains) that can be easier examined for partial uniqueness of the solution.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.labri.fr/perso/pelegrin/scotch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scotch project&lt;/a&gt; provides &quot;sequential and parallel sparse matrix block ordering&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/PetterS/SuiteSparse/tree/master/BTF&quot; rel=&quot;nofollow noreferrer&quot;&gt;The btf tool of SuiteSparse&lt;/a&gt; uses &quot;maximal matching which maximizes the number of&#xA;nonzeros on the diagonal&quot; and &quot;a method for finding the strongly connected components of a graph&quot; to &quot;give the permutation to block upper triangular form.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.google.com/search?q=sparse+matrix+block+reordering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google&lt;/a&gt; provides more links to papers discussing the theory and other stackexchange and overflow questions, like (&lt;a href=&quot;https://mathoverflow.net/questions/68041/showing-block-diagonal-structure-of-matrix-by-reordering&quot;&gt;https://mathoverflow.net/questions/68041/showing-block-diagonal-structure-of-matrix-by-reordering&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;These algorithm interpret the sparsity structure of the matrix as a bipartite graph and construct edge weights from the entries. For instance log magnitude (bit length for integers) plus number of other row entries times number of other column entries. For this matrix it would give the weight matrix&#xA;$$\begin{bmatrix}&#xA;4&amp;amp;1&amp;amp;-&amp;amp;-\\&#xA;6&amp;amp;-&amp;amp;3&amp;amp;4\\&#xA;6&amp;amp;-&amp;amp;3&amp;amp;4\\&#xA;3&amp;amp;1&amp;amp;-&amp;amp;-&#xA;\end{bmatrix}$$&#xA;where then the Hungarian algorithm or a variant is applied to find a max weight matching defining the reordered diagonal. Which can be seen here to be $(1,1),(2,3),(3,4),(4,2)$. Then semi-heuristic algorithms of clustering and partial graph ordering are applied to get as many entries as possible above the diagonal. Which would result here in moving the $(2:3, 3:4)$ block to the topleft diagonal position.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2014-01-10T07:27:03.977" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10505" PostTypeId="2" ParentId="10481" CreationDate="2014-01-09T20:26:52.483" Score="1" Body="&lt;p&gt;You can show that any system of polynomial equations can be written as a larger system of quadratic equations by introducing new variables for intermediate terms. Now it is known that global convergence of Newtons method gets worse the higher the degree and dimension, so there should be many examples that show this property.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2014-01-09T20:26:52.483" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10506" PostTypeId="1" AcceptedAnswerId="10510" CreationDate="2014-01-09T21:52:59.503" Score="4" ViewCount="4874" Body="&lt;p&gt;Is there any reason why SciPy's integrate.odeint() should become less precise&#xA;when the number of equations increase?  I'm trying to solve these two&#xA;sets of differential equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dy_1}{dx} = x^2$     — (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dy_2}{dx} = x^3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dy_1}{dx} = x^2$     — (2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the two equations in (1) are uncoupled, $y_1$ from integrating (1)&#xA;and (2) should be the same.  I'm implementing the two in the following&#xA;snippet:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.integrate import odeint&#xA;&#xA;def f(y, x):&#xA;    return [x**2, x**3]&#xA;&#xA;def g(y, x):&#xA;    return x**2&#xA;&#xA;a = odeint(f, [0.0, 0.0], np.arange(0, 5, 0.0001))&#xA;b = odeint(g, 0.0, np.arange(0, 5, 0.0001))&#xA;&#xA;print a[-1][0], b[-1][0], abs(a[-1][0] - b[-1][0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running the above code gives me:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;41.6641667161 41.6641667354 1.93298319573e-08&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The difference seems to be very insignificant here.  But in cases where&#xA;the number of equations becomes large (for e.g. in Lyapunov exponent&#xA;calculations), it appears to cause significant differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be going on here?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;UPDATE: Like @horchler explained, using &lt;code&gt;numpy.linspace&lt;/code&gt; instead of &lt;code&gt;numpy.arange&lt;/code&gt; did increase the accuracy of the final values, but the difference between two answers is of the same order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;41.6666666618 41.6666666811 1.93298248519e-08&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7042" LastEditorUserId="7042" LastEditDate="2014-01-11T19:53:01.063" LastActivityDate="2014-01-11T19:53:01.063" Title="Number of equations and precision of SciPy's integrate.odeint()" Tags="|ode|precision|scipy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10507" PostTypeId="1" CreationDate="2014-01-10T01:11:37.657" Score="1" ViewCount="223" Body="&lt;p&gt;In multiple view geometry of computer vision, there is a geometric error called &lt;code&gt;Sampson error&lt;/code&gt; which is very useful in the nonlinear estimation of fundamental matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this &lt;code&gt;Sampson error&lt;/code&gt; be generalize to multiple view cases so as to use it for multiple focal tensor estimation?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2014-01-10T01:11:37.657" Title="Is it possible to generalize the two view Sampson error to multiple view cases in computer vision?" Tags="|computer-vision|geometry|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10508" PostTypeId="2" ParentId="10506" CreationDate="2014-01-10T02:18:55.730" Score="4" Body="&lt;p&gt;&lt;code&gt;scipy.integrate.odeint&lt;/code&gt; is really a wrapper to LSODA, part of ODEPACK, written in the 1980s. It uses some heuristics to switch between Adams (nonstiff) and BDF (stiff) methods, depending on diagnostics used to determine the stiffness of the ODE system. Since BDF methods are implicit, presumably the Adams methods used are Adams-Moulton methods (which are also implicit, as opposed to Adams-Bashforth methods, which are explicit).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without a more concrete example comparing two large systems, the errors you see could be caused by:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;stiffness/ill-conditioned Jacobian matrix in your right-hand side&lt;/li&gt;&#xA;&lt;li&gt;roundoff from doing substantially more floating point operations (say, in the limit as your larger system becomes infinitely larger than the smaller system)&lt;/li&gt;&#xA;&lt;li&gt;error tolerances, depending on how you set them for any &quot;new&quot; variables&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-01-10T02:18:55.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10509" PostTypeId="2" ParentId="1639" CreationDate="2014-01-10T02:20:26.670" Score="0" Body="&lt;p&gt;Tecplot 360 is good for processing large data and producing publication-quality plots. It has powerful add-on and scripting interfaces to process the data. It appears to me that it does everything you'd need for post-processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common mistake is to resort to using Excel. No. It does not remember layout, X range, colour map. It does not animate. It uses ugly bulky spreadsheets to work with the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course you would not find a tool which does both post-processing and text editing, and choosing a reasonable text editor for your publications is another important question. Don't use MS Word, its citations manager is ugly. I use LyX or any other Latex tool available for the operating system I'm using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oh, and when you have to publish PDF, include a link to LaTeX, Word, or whatever format with readable math. In PDF, it's not.&lt;/p&gt;&#xA;" OwnerDisplayName="user5273" LastEditorDisplayName="user5273" LastEditDate="2014-01-10T02:28:58.137" LastActivityDate="2014-01-10T02:28:58.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10510" PostTypeId="2" ParentId="10506" CreationDate="2014-01-10T02:22:20.253" Score="4" Body="&lt;p&gt;To begin, it might be a better idea to use &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.linspace.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;linspace&lt;/code&gt;&lt;/a&gt; in this case if you care about the final value or just let the integrator choose the intermediate steps. That will give you accurate final values, but the difference in precision between the two cases will remain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few interrelated reasons why this is occurring. Just because two equations are uncoupled does not mean that &lt;code&gt;odeint&lt;/code&gt; solves them separately. The solver has to choose one time step on each iteration and this will depend on the behavior of all of the equations. In the case of your simpler scalar quadratic equation, the solver may be able to take much larger time steps, whereas the cubic equation in the other case grows much more quickly and may cause the solver to reduce step sizes or even switch between modes. The time steps chosen depend on (amongst other things) the absolute and relative tolerances which you can adjust.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastActivityDate="2014-01-10T02:22:20.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10511" PostTypeId="2" ParentId="1639" CreationDate="2014-01-10T08:28:03.270" Score="3" Body="&lt;p&gt;I'm surprised nobody has mentioned Nico Schlömer's excellent tools &lt;a href=&quot;https://github.com/nschloe/matlab2tikz&quot; rel=&quot;nofollow noreferrer&quot;&gt;matlab2tikz&lt;/a&gt; and &lt;a href=&quot;https://github.com/nschloe/matplotlib2tikz&quot; rel=&quot;nofollow noreferrer&quot;&gt;matplotlib2tikz&lt;/a&gt; yet. If you are using LaTeX for document preparation and either Matlab or Python for data processing, you can easily get high quality vector plots which you can post process to your heart's content:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Prepare your plots in Matlab or Python, including axes, labels, legends, etc. Don't worry about TeX formatting.&lt;/li&gt;&#xA;&lt;li&gt;Call &lt;code&gt;matlab2tikz('figure.tex')&lt;/code&gt; (or &lt;code&gt;matplotlib2tikz&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Edit &lt;code&gt;figure.tex&lt;/code&gt; to taste (replace label text with proper math, change colors, line width, legend placement, axis style, etc.), see the &lt;a href=&quot;http://mirrors.ctan.org/graphics/pgf/contrib/pgfplots/doc/pgfplots.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pgfplots manual&lt;/a&gt; (or the many &lt;a href=&quot;https://tex.stackexchange.com/questions/tagged/pgfplots&quot;&gt;questions on pgfplots&lt;/a&gt; on the TeX stackexchange). &lt;/li&gt;&#xA;&lt;li&gt;In your figure, instead of &lt;code&gt;includegraphics&lt;/code&gt;, put &lt;code&gt;\input{figure.tex}&lt;/code&gt; (and don't forget to include &lt;code&gt;\usepackage{pgfplots}&lt;/code&gt; in the preamble).&lt;/li&gt;&#xA;&lt;li&gt;For submission, use the externalization library (see chapter 7 of the manual) to bake your figure into a pdf and simply replace &lt;code&gt;\input{figure.tex}&lt;/code&gt; by &lt;code&gt;\includegraphics{figure.pdf}&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:36:11.990" LastActivityDate="2014-01-10T09:42:15.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10512" PostTypeId="1" CreationDate="2014-01-10T09:22:28.140" Score="3" ViewCount="634" Body="&lt;p&gt;I am looking at a problem of constrained minimization, where the function to be minimized contains the Heaviside function, and as such is not twice continuously differentiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what effect would the use of a twice continuously differentiable approximation to the Heaviside function have on the accuracy and efficiency of the optimization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the problem takes the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$min \sum_{i}^{N} k_{i}\left [  H\left ( x_{i} \right ) -T\right ],$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s.t. \sum_{i}^{N} k_{i}\left [  x_{i}  -R\right ] \le 0.01,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H\left(x \right ) $ is the Heaviside function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $x,k,T,R\in\mathbb{R}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would the use of the approximate Heaviside &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H\left ( x \right )\approx \frac{1}{2}+\frac{1}{2}tanh(kx)=\frac{1}{1+e^{-2kx}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;help with the minimization and give a sufficiently accurate result&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or would a Legendre polynomial expansion (similar to &lt;a href=&quot;http://www.phys.ufl.edu/~fry/6346/legendrestep.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.phys.ufl.edu/~fry/6346/legendrestep.pdf&lt;/a&gt;) be more successful?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach will need to be extended into multiple dimension before implementation, for example in 3 dimensions the minimization becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i}\sum_{j}\sum_{l}k_{1,i}k_{2,j}k_{3,l}\left [ H\left ( x_{ijk}\right )-T \right ],$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i}\sum_{j}\sum_{l} k_{1,i }k_{2,j}k_{3,l}\left [x_{ijk} - R \right] \le 0.01,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimization is covered by the question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10447/constrained-minimization-in-n-dimensions&quot;&gt;constrained minimization in N dimensions&lt;/a&gt;, this question is regarding the effect of using an approximation to the step function in the algorithm (and ultimately the choice of algorithm to use)&lt;/p&gt;&#xA;" OwnerUserId="6977" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-01-13T09:26:41.593" Title="Affect of approximating a non-differentiable function on optimisation of minimisation" Tags="|optimization|algorithms|nonlinear-programming|constrained-optimization|nonconvex|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10514" PostTypeId="1" CreationDate="2014-01-10T17:01:27.643" Score="3" ViewCount="1603" Body="&lt;p&gt;In one sentence (thanks to @Brian Borchers), I want to minimize the function f(x, y, ...), with gradient g(x, y, ...), subject to constraints that aren't given explicitly, but are defined by situations where the routines that compute f and g return an error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The domain for &lt;code&gt;f&lt;/code&gt; isn't linear and it is not simple to change variables. I approximated a boxed constraints within which the parameters only occasionally (still infinitely many) fall out of the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(think of &lt;code&gt;f&lt;/code&gt; and &lt;code&gt;g&lt;/code&gt; as a black box, it throws exceptions when parameters are not in the domain. I am only able to create wrappers so as to catch exceptions and returns some values.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I used L-BFGS-B, and &lt;code&gt;f&lt;/code&gt; to be very large when the parameters not in the domain. And I set &lt;code&gt;g&lt;/code&gt; at those points to be 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most case the optimization just works fine. However, sometimes it would trigger error like &quot;ABNORMAL_TERMINATION_IN_LNSRCH&quot;, I guess it is just because my &lt;code&gt;g&lt;/code&gt; doesn't agree with &lt;code&gt;f&lt;/code&gt; at those nasty points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the best practice in general in such case?&lt;/p&gt;&#xA;" OwnerUserId="7047" LastEditorUserId="7047" LastEditDate="2014-01-10T23:47:55.790" LastActivityDate="2014-01-11T10:23:47.447" Title="Workaround for BFGS with non simple constraints?" Tags="|optimization|constrained-optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10515" PostTypeId="2" ParentId="10512" CreationDate="2014-01-10T23:24:56.457" Score="2" Body="&lt;p&gt;If the number of variables you have is modest, then you can just subdivide the domain into the half-lines/quadrants/octants/etc of the space. In each of them, the Heaviside function is constant. For example, for a single optimization variable, your original problem was &#xA;$$&#xA;  \min_x k[H(x)-T] \\&#xA;  \text{s.t.} \ k[x-R] \le c.&#xA;$$&#xA;You can split this into two optimization problems: first, solve&#xA;$$&#xA;  x_1^* = \arg \min_x -kT \\&#xA;  \qquad \text{s.t.} \ k[x-R] \le c \quad \text{and} \quad x\le 0.&#xA;$$&#xA;then solve &#xA;$$&#xA;  x_2^* = \arg \min_x k[1-T] \\&#xA;  \qquad \text{s.t.} \ k[x-R] \le c \quad \text{and} \quad x\ge 0.&#xA;$$&#xA;Note that both of these are linear optimization problems and are, consequently rather simple to solve. It is possible that one of these problems has no solution because the feasible set is empty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If only one of the problems has a solution, then you're done. If both have a solution, you simply have to compare the value of the objective function at these points to find which one is better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have more than one variable $x_i$, then the problem becomes slightly more complicated. For example, if you have two, then you need to consider four cases of signs of the variables $x_i$. For general, $N$-dimensional problems, the number of cases to compare is $2^N$. This will quickly become impractical, but for modest numbers you should be able to deal with it. In particular, you can use the usual tricks of integer programming, e.g. branch and bound: if you have already found a solution $x^\ast$ from one of the sub-problems, then this provides you with a value of the objective function that you need to get below to find a better solution. I believe you can use this in the dual formulation of linear programs to rule out that a given one of the other subproblems has a better solution, without even solving it. In effect, you may get away with solving many fewer than $2^N$ linear programs.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-10T23:24:56.457" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10516" PostTypeId="1" CreationDate="2014-01-11T00:13:18.600" Score="8" ViewCount="2237" Body="&lt;p&gt;I'm looking at speeding up matrix-vector products but everything I read is about how to do it for very large matrices. My case, the matrices are small but the number of times it must be done is very large. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What methods, if any, are there to optimize this? Would it be faster to construct a really big diagonal block matrix out of the small matrices and one big vector made of the smaller vectors and use the techniques for the large matrix-vector speedups? Or would setting up the global matrix and vector kill any benefit there?&lt;/p&gt;&#xA;" OwnerUserId="391" LastActivityDate="2018-06-07T13:34:06.503" Title="Optimizing matrix-vector multiplication for many small matrices" Tags="|linear-algebra|optimization|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10517" PostTypeId="2" ParentId="10514" CreationDate="2014-01-11T03:05:56.667" Score="4" Body="&lt;p&gt;Unfortunately, L-BFGS-B simply isn't designed to handle this kind of problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a couple of suggestions that might be helpful.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you can define a smooth extension to your function $f(x)$ which increases very rapidly in value outside of the feasible region, then you may be able to get reasonable results using L-BFGS-B or some other unconstrained or bounds constrained optimization code.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you can define a smooth function $g(x)$ such that $g(x)&amp;lt;0$ outside of the feasible region and $g(x) \geq 0$ inside the feasible region then you can use a constrained optimization routine to try to solve the problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the objective function $f(x)$ is convex and the feasible region is convex, then there are lots of methods that can be applied.  You could use the method of steepest descent for example.  You could also use L-BFGS-B and simply restart the method with a steepest descent step every time the method tries to step outside of the feasible region.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the general case where $f$ is nonconvex or the feasible region is nonconvex, the problem is hopeless.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-11T03:05:56.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10518" PostTypeId="2" ParentId="10512" CreationDate="2014-01-11T09:38:10.163" Score="3" Body="&lt;p&gt;If you're optimizing with respect to the $k_{i,j}$, $k_{i,j}$ isn't an argument to the Heaviside function (for each $i, j$), and the $x_{i}$ and $T$ are known parameters, your problem is continuous and (twice continuously) differentiable with respect to $k$. Your problem still looks to be nonconvex at first glance, so deterministic global optimization would require a branch-and-bound method with convex relaxations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the $x_{i}$ are decision variables (with or without the $k_{i,j}$), then your problem isn't just non-differentiable: it's discontinuous. Consequently, again, it's nonconvex and requires a branch-and-bound method. Generally speaking, as Wolfgang points out, if you know locations of discontinuities, you can subdivide your problem into a number of smaller subproblems on which you may be able to use more efficient methods (read: not branch-and-bound, preferable some second-order optimization algorithm). The number of subproblems may be unacceptably large (as Wolfgang points out, in this case, $2^{n}$, where $n$ is the number of decision variables that are arguments to Heaviside functions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you replace the Heaviside function with the hyperbolic tangent approximation, you gain twice-differentiability, but given that the hyperbolic tangent is nonconvex over any subinterval containing 0, you may also introduce nonconvexity into your problem. As a result, I don't think you gain much by introducing that approximation, and I would only use it in the event that it is impractical to divide the problem into many subproblems over which all functions are convex, continuous, and twice-differentiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the problem were just about nondifferentiability and not about discontinuities, I'd say you could try looking at subgradient methods or bundle methods. These are slower than methods that require some degree of differentability. It's worth noting that even derivative-free methods require some degree of differentiability -- they merely assume that derivative information is unavailable.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-01-13T09:26:41.593" LastActivityDate="2014-01-13T09:26:41.593" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10519" PostTypeId="2" ParentId="10514" CreationDate="2014-01-11T10:23:47.447" Score="0" Body="&lt;p&gt;It depends on what you care about. Another possibility is to pick the largest box in your feasible that you know doesn't contain any points outside the domain of $f$ and $g$. Assuming that $f$ and $g$ are smooth and convex, solving this problem will get you an upper bound on the optimal objective function value. Sometimes that upper bound can be helpful. Other times, it's simply not enough information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, though, I would try some combination of 2 and 3 from BrianBorchers' approach. If you feel very strongly about using a BFGS-type method, using his suggestions, you could augment it with a gradient projection method, where BFGS steps are projected onto the feasible set to ensure that each iterate remains feasible. If $f$ is extended so that it is artificially very large outside of its domain, then restarting with gradient descent is probably a fine approach and gradient projection is a little overkill.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-01-11T10:23:47.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10520" PostTypeId="1" AcceptedAnswerId="10524" CreationDate="2014-01-11T12:43:44.917" Score="17" ViewCount="7145" Body="&lt;p&gt;I would like to farm out solving systems of ODEs onto GPUs, in a 'trivially parallelisable' setting. For example, doing a sensitivity analysis with 512 different parameter sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally I want to do ODE solving with a smart adaptive timestep solver like CVODE, rather than a fixed timestep like Forward Euler, but running it on an NVIDIA GPU instead of CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone done this? Are there libraries for it? &lt;/p&gt;&#xA;" OwnerUserId="4575" LastEditorUserId="4575" LastEditDate="2019-12-03T08:32:28.673" LastActivityDate="2020-07-19T17:51:05.117" Title="Options for solving ODE systems on GPUs?" Tags="|ode|gpu|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10521" PostTypeId="2" ParentId="10516" CreationDate="2014-01-11T13:10:52.123" Score="7" Body="&lt;p&gt;Before trying to optimize your code, it is worth asking if there is anything to optimize to begin with. Libraries that optimize matrix-vector products do so by working around two issues: limitations on the size of the cache and the latency to load data from memory. The first is done by using the data that's currently in cache to its fullest extent for everything it needs to be used for before replacing it by other data, the latter is done by prefetching data into the cache before actually making use of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you have relatively little arithmetic intensity of your data -- you load data from memory, you use it exactly once, and then you move on to the next matrix. This leaves only the second avenue to optimize for: prefetch data before you use it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, as I said, before trying to optimize things, it may be worth figuring out what you already have: time how many matrix-vector products you are doing per second, calculate how many bytes this requires to load from memory onto your processor, and then compare this with the bandwidth of the processor you happen to have in your machine. You may find that there is nothing you can do to make things faster.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-11T13:10:52.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10522" PostTypeId="2" ParentId="10494" CreationDate="2014-01-11T13:31:47.360" Score="2" Body="&lt;p&gt;If you google &quot;calculate orbital elements binary star&quot; you find these two as the first hits:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ugastro.berkeley.edu/infrared10/adaptiveoptics/binary_orbit.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ugastro.berkeley.edu/infrared10/adaptiveoptics/binary_orbit.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.professores.uff.br/diomar_cesar_lobao/material/Astro-Dio/SABP94-CERCA.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.professores.uff.br/diomar_cesar_lobao/material/Astro-Dio/SABP94-CERCA.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am sure the following hits would also be helpful.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-11T13:31:47.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10523" PostTypeId="2" ParentId="10516" CreationDate="2014-01-11T14:33:17.830" Score="4" Body="&lt;p&gt;It may not actually matter since your matrices are cache contained already, but you should be calling &lt;code&gt;dgemv()&lt;/code&gt; or &lt;code&gt;sgemv()&lt;/code&gt; or the equivalent from the best BLAS library you can get your hands on. You should try the Intel MKL if you can get access to it, and also BLIS or ATLAS or one of the many other optimized BLAS libraries out there.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-01-11T14:33:17.830" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10524" PostTypeId="2" ParentId="10520" CreationDate="2014-01-11T18:32:18.057" Score="6" Body="&lt;p&gt;You may want to look into &lt;a href=&quot;http://www.boost.org/doc/libs/1_55_0/libs/numeric/odeint/doc/html/&quot;&gt;Boost's odeint library&lt;/a&gt; and &lt;a href=&quot;http://docs.nvidia.com/cuda/thrust/&quot;&gt;Thrust&lt;/a&gt;.  They can be combined as &lt;a href=&quot;http://www.boost.org/doc/libs/1_55_0/libs/numeric/odeint/doc/html/boost_numeric_odeint/tutorial/using_cuda__or_openmp__tbb_______via_thrust.html&quot;&gt;discussed here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2014-01-11T18:38:19.853" LastActivityDate="2014-01-11T18:38:19.853" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10525" PostTypeId="2" ParentId="10119" CreationDate="2014-01-12T03:01:16.043" Score="12" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For &lt;strong&gt;eigenvalues&lt;/strong&gt;, simply take $k$ largest or smallest eigenvalues of $T$.&#xA;They are good approximations of $A$, provided that the number of Lanczos iterations is large compared to $k$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Things are a little trickier if we want &lt;strong&gt;eigenvectors&lt;/strong&gt; as well.&lt;br&gt;&#xA;The simplest way is to multiply each eigenvector $\mathbf{u}_i$ of&#xA;$T$ by $V$ to the left, where $V$ is, as you said, the collection of&#xA;Lanczos vectors. But this approach falls short for many classes of&#xA;matrices, as round-off errors cause the Lanczos vectors to lose their&#xA;orthogonality (1).&lt;br&gt;&lt;br&gt;&#xA;A better way is to re-orthogonalize the Lanczos vectors in $V$ by doing a Gram-Schmidt step.&lt;br&gt;&#xA;Let $\mathbf{z}$ be the $i$-th Lanczos vector being computed, and let $\mathbf{q}_1, \mathbf{q}_2, \cdots, \mathbf{q}_{i-1}$ be the previous Lanczos vectors. We mutate $\mathbf{z}$ so as to eliminate all the components of $\mathbf{z}$ that are parallel to any of $\mathbf{q}_1, \mathbf{q}_2, \cdots, \mathbf{q}_{i-1}$:&#xA;$$&#xA;\mathbf{z}' = \mathbf{z} - \sum_{j=1}^{i-1}(\mathbf{z}^{T}\mathbf{q}_j)\mathbf{q}_j&#xA;$$&#xA;It turns out that we need to re-orthogonalize twice to guarantee the numerical orthogonality of the Lanczos vectors (2).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Reference&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;J. Demmel, &lt;em&gt;Applied Numerical Linear Algebra&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;B. Parlett, &lt;em&gt;The Symmetric Eigenvalue Problem&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7052" LastActivityDate="2014-01-12T03:01:16.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10527" PostTypeId="2" ParentId="2152" CreationDate="2014-01-13T09:05:57.767" Score="2" Body="&lt;p&gt;You want to keep an indexed table of variable values. The index corresponds to a folder where you keep each simulation input and output. So it's just an index and you don't have to worry about naming convention or folder hierarchies because you'll look up what parameters values correspond to each folder. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now you can use this table to organize your post-processing, plotting (analysis), logging, and commenting. The table is central to the workflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the basic idea, and I'm describing what you might want to do only conceptually. In my initial response, I suggested looking into the framework that I've developed. More recently I've discovered &lt;a href=&quot;http://pythonhosted.org/Sumatra/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sumatra&lt;/a&gt;. It's much more developed than my individually-developed, struggling grad student, and new to python effort but I think it tries to do too much. It's focused on provenance info while my framework focuses on workflow efficiency. There is also &lt;a href=&quot;http://deeplearning.net/software/jobman/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jobman&lt;/a&gt;, &lt;a href=&quot;http://sacred.readthedocs.org/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sacred&lt;/a&gt;, and &lt;a href=&quot;https://github.com/ioam/lancet&quot; rel=&quot;nofollow noreferrer&quot;&gt;lencet&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever you choose to do, I strongly recommend python to tackle these types of tasks since you can manage your whole workflow with python. Just as a little story, I watched my colleagues work with DAKOTA, bash, GNUplot, file naming conventions, sed/awk octave...etc. to do their computational work. Each of these tools are fine on their own but the power of python as an integrating glue language really shines when you use python for managing your work along with the python scientific stack. I literally had zero issues managing my computational work after I developed my framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;/my initial response follows/&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe I have solved this problem using python. I have thought of all these issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out my repo &lt;a href=&quot;http://msdresearch.blogspot.com/2012/01/parameter-study-management-with-python.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://msdresearch.blogspot.com/2012/01/parameter-study-management-with-python.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of now though, I'm working on better documenting my framework. (it's more involved than filling in a readme!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Majid alDosari&lt;/p&gt;&#xA;" OwnerUserId="7059" LastEditorUserId="7059" LastEditDate="2017-05-17T15:06:41.743" LastActivityDate="2017-05-17T15:06:41.743" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10528" PostTypeId="2" ParentId="3229" CreationDate="2014-01-13T10:06:56.993" Score="3" Body="&lt;p&gt;If you have a sparse matrix stored in &lt;code&gt;COO&lt;/code&gt; format, the following might be helpful&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    A.row = perm[A.row];&#xA;    A.col = perm[A.col];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;assuming that &lt;code&gt;A&lt;/code&gt; contains the &lt;code&gt;COO&lt;/code&gt; matrix, and &lt;code&gt;perm&lt;/code&gt; is a &lt;code&gt;numpy.array&lt;/code&gt; containing the permutation. This will only have $m$ memory overhead, where $m$ is the number of non-zero elements of the matrix.&lt;/p&gt;&#xA;" OwnerUserId="7061" LastActivityDate="2014-01-13T10:06:56.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10531" PostTypeId="1" AcceptedAnswerId="10536" CreationDate="2014-01-13T17:12:53.403" Score="0" ViewCount="279" Body="&lt;p&gt;I'm trying to perform a trasient simulation of a subsonin flow over a flat plate in CFD++.&#xA;I want to impose a ramp profile on the velocity at the inlet, like for instance a smooth step, and look at the evolution of the boundary layer over the plate due to this acceleration.&#xA;I want to perform this simulation without pressure gradient so I'm imposing a back pressure value in outflow BC. The problem is that once I start the acceleration at the inlet it creates a pressure gradient that travels to the outlet, and since the pressure there is kept constant, the pressure wave bounces creating instabilities&#xA;Does any one know how to perform this simulation without creating pressure gradients?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7064" LastActivityDate="2014-01-13T23:12:25.727" Title="zero pressure gradient flat plate CFD++" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10532" PostTypeId="1" AcceptedAnswerId="10534" CreationDate="2014-01-13T17:27:48.387" Score="3" ViewCount="2105" Body="&lt;p&gt;Would anyone please explain me what is the mathematical algorithm to solve a IVP system of ODE with a time-dependent parameter. e.g. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   dy1/dt = a1(t) * y2 -b1(t)* y3 -c1(t) y1;&#xA;   dy2/dt = a2(t) * y1 -b2(t)* y1 -c2(t) y3;&#xA;   dy3/dt = a3(t) * y2 -b3(t)* y3 -c3(t) y1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;y(0)=c;&#xA;My idea so far is that for fixed time-step I should form a vector of a,b,c and assume a,b,c to be constant over that time-period. what about if the system is stiff?&#xA;Would anyone plaese give me an idea how to solve it using c++ boost library? Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="3695" LastActivityDate="2014-01-14T04:42:44.497" Title="Algorithm for solving an ODE with time-dependent parameter numerically" Tags="|ode|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10533" PostTypeId="1" CreationDate="2014-01-13T18:03:58.153" Score="5" ViewCount="1055" Body="&lt;p&gt;I am looking for a computationally cheap way to compute $x$ such that $$(L L^T + \mu^2 I)x = y$$&#xA;where $L \in \mathbb{R}^{n \times n}$ is a lower triangular definite positive matrix (with some very small eigenvalues), $y \in \mathbb{R}^n$ and $\mu \in \mathbb{R}$ are known. If it is necessary, I can assume that&#xA;$$\mu \ll 1$$&#xA;But $\mu^2$ is larger than the smallest eigenvalue of $LL^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I would like to make the most of my knowledge of the Cholesky decomposition $L L^T$. Eventually, I hope to be able to compute $x$ in $\mathcal{O}(n^2)$. Approximate approaches are also welcomed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen &lt;a href=&quot;https://mathoverflow.net/questions/131481/solve-abx-y-given-cholesky-decomposition-of-a-and-b&quot;&gt;here&lt;/a&gt; that this does not seem to be doable in a more general situation, but I hope the smallness of $\mu$ may help...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea, reference or warning?&#xA;Thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="7065" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2015-07-14T22:49:47.270" Title="Perturbation of Cholesky decomposition for matrix inversion" Tags="|linear-algebra|matrix|inverse|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10534" PostTypeId="2" ParentId="10532" CreationDate="2014-01-13T20:16:59.040" Score="5" Body="&lt;p&gt;Given an ODE $\dot{y}(t) = f(y(t), t)$, most -- if not all -- algorithms reduce to function evaluations and Jacobian evaluations at specified points in $t$ and $y$. To illustrate, a step of explicit Euler is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;y_{n + 1} = y_{n} + (t_{n + 1} - t_{n}) \cdot f(y_{n}, t_{n}).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a time-dependent parameter, treat it as part of $f$, the function evaluation (respectively, for implicit methods, also treat it as part of the Jacobian evaluation). A similar strategy applies to more complicated methods for solving ODEs (multistage methods such as Runge-Kutta, implicit methods for stiff systems, etc.). This strategy is different than assuming your parameters are constant over a time step -- for a multistage method, assuming your parameters are constant over a time step will result in inaccurate function evaluations in intermediate stages and reduce the order of your method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for solving it using a C++ Boost library, you might try looking at the &lt;a href=&quot;http://headmyshoulder.github.io/odeint-v2/&quot;&gt;Boost odeint&lt;/a&gt; documentation.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-01-13T20:16:59.040" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10535" PostTypeId="2" ParentId="10533" CreationDate="2014-01-13T20:36:58.053" Score="5" Body="&lt;p&gt;Unfortunately, although it is easy to update the Cholesky factorization after a low-rank update, updating the Cholesky factorization after a full rank update like this isn't any easier than recomputing the Cholesky factorization.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative to consider would be to compute the eigenvalue decomposition of your matrix, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=LL^{T}=UDU^{T}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding $\mu^{2}I$ to $A$ simply adds $\mu^{2}$ to the eigenvalues of $A$, so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A+\mu^{2}I=UVU^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $V=D+\mu^{2}I$.  Then, to solve $(A+\mu^{2}I)x=y$, we let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$UVU^{T}x=y$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x=UV^{-1}U^{T}y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each value of $\mu$ you're interested in, this takes only $O(n^{2})$ additional work (three matrix-vector multiplications and one of them is just a diagonal scaling matrix.  Be careful not to multiply the matrices!) In comparison with computing the Cholesky factorization of $A$, computing the eigenvalue decomposition is also $O(n^{3})$, but it's typically an order of magnitude slower than the Cholesky factorization.  On an amortized basis this will pay for itself if you have enough (say dozens to hundreds) of values of $\mu$ to consider.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-13T20:36:58.053" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10536" PostTypeId="2" ParentId="10531" CreationDate="2014-01-13T23:12:25.727" Score="1" Body="&lt;p&gt;I don't think this is a physically reasonable problem to try to run. You can't have a ramping velocity at the inflow without pressure changes throughout the domain and at the outflow. If you want a zero pressure gradient boundary layer flow, you'll have to equilibrate it after you get the boundary layer formed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be better off using a taller domain with an imposed top boundary velocity equal to your inflow velocity, and using whatever kind of outflow boundary condition CFD++ supports. In a 2-D, steady boundary layer (in the right units), the pressure obeys&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dp}{dx}=-\rho U \frac{dU}{dx}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U(x)$ is the x-direction velocity boundary condition at the top of the domain. This isn't true for unsteady flows, but if you ramp slowly, you may be able to have a nearly constant pressure during your ramp.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-01-13T23:12:25.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10537" PostTypeId="2" ParentId="10494" CreationDate="2014-01-14T01:38:20.620" Score="1" Body="&lt;p&gt;The good software for orbit calculation is &quot;Binary&quot; (Binary Star Combined Solution Package) by CHARA team:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.chara.gsu.edu/~gudehus/binary.html&quot; rel=&quot;nofollow&quot;&gt;http://www.chara.gsu.edu/~gudehus/binary.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7029" LastActivityDate="2014-01-14T01:38:20.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10540" PostTypeId="1" CreationDate="2014-01-14T13:45:57.557" Score="4" ViewCount="2397" Body="&lt;p&gt;I have a problem in physics formulated via an ODE. Now I like to solve it numerically using Pythons scipy.integrate and the therein complex_ode. I figured out how and it works but now I like to optimize my code and I ran into a question: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a massive amount of grid points so that the solution converges. So I tried setting another integrator method with stepsize control (dop853). I am not really familiar with that feature but it works as well and is much faster than my approach. Here's a simple MWE which does not reflect my complicated ODE but the structure of the setup (complex valued etc.) is the same:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy import *&#xA;from scipy.integrate import *&#xA;from pylab import *&#xA;&#xA;grid = linspace(0, 8, 10e3)&#xA;dgrid = (grid[1]-grid[0])*ones(100e3)&#xA;&#xA;def RHS(t, x):&#xA;    return -x&#xA;&#xA;y = zeros(len(grid), dtype=complex)&#xA;y[0] = 1.0&#xA;&#xA;solver = complex_ode(RHS)&#xA;solver.set_initial_value(y[0], grid[0]).set_integrator('dop853')&#xA;&#xA;for t in range(len(grid)):&#xA;    solver.integrate(solver.t+dgrid[t])&#xA;    y[t] = solver.y[0]&#xA;&#xA;plot(grid, y.real)&#xA;show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But now the question: I now of course want to relate the time grid to the solution. What grid is now connected to the solution? Is it still the grid I was introducing (grid) or does the stepsize control calculate at different time steps and I would need another time array?&lt;/p&gt;&#xA;" OwnerUserId="7040" LastActivityDate="2014-01-14T19:22:06.317" Title="Python: Grid with step control ODE solver" Tags="|python|ode|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10541" PostTypeId="1" AcceptedAnswerId="20244" CreationDate="2014-01-14T17:24:55.027" Score="9" ViewCount="749" Body="&lt;p&gt;I have a large cubic eigenvalue problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(\mathbf{A}_0 + \lambda\mathbf{A}_1 + \lambda^2\mathbf{A}_2 + \lambda^3\mathbf{A}_3\right)\mathbf{x} = 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could solve this by converting to a linear eigenvalue problem but it would result in a system $3^2$ as large:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix}&#xA;-\mathbf{A}_0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; \mathbf{I} &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; \mathbf{I}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\mathbf{x} \\&#xA;\mathbf{y} \\&#xA;\mathbf{z}&#xA;\end{bmatrix}&#xA;=&#xA;\lambda&#xA;\begin{bmatrix}&#xA;\mathbf{A}_1 &amp;amp; \mathbf{A}_2 &amp;amp; \mathbf{A}_3 \\&#xA;\mathbf{I} &amp;amp; 0 &amp;amp; 0 \\&#xA;  0 &amp;amp; \mathbf{I} &amp;amp; 0&#xA; \end{bmatrix}&#xA; \begin{bmatrix}&#xA; \mathbf{x} \\&#xA; \mathbf{y} \\&#xA; \mathbf{z}&#xA; \end{bmatrix},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{y} = \lambda\mathbf{x}$ and $\mathbf{z} = \lambda\mathbf{y}$. What other techniques are available to solve a cubic eigenvalue problem? I've heard that there is a version of Jacobi-Davidson that will solve it but haven't found an implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I need to be able to target specific eigenvalues similarly to the shift-and-invert method of ARPACK and find the associated eigenvectors.&lt;/p&gt;&#xA;" OwnerUserId="4524" LastActivityDate="2015-07-21T13:54:15.210" Title="Implementation of Jacobi-Davidson method for cubic eigenvalue problem" Tags="|c++|eigensystem|eigenvalues|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10542" PostTypeId="1" AcceptedAnswerId="10543" CreationDate="2014-01-14T19:01:50.850" Score="8" ViewCount="261" Body="&lt;p&gt;When I learned about SOR, it was mostly given as one of the first examples of iterative methods, and then later the iterative methods that I would end up using would be Krylov subspace methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are any of the iterative methods like Gauss-Seidel and SOR ever used in practice? Do you know of any real packages that use them &quot;seriously&quot;, for something other than demonstration purposes?&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-01-15T03:11:39.887" Title="Gauss-Seidel, SOR in practice?" Tags="|linear-algebra|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10543" PostTypeId="2" ParentId="10542" CreationDate="2014-01-14T19:14:04.370" Score="10" Body="&lt;p&gt;Yes, but not as stand-alone solvers for linear systems of equations.  These days, they are used as &lt;a href=&quot;http://www.math.ust.hk/~mawang/teaching/math532/mgtut.pdf&quot;&gt;smoothers in multigrid&lt;/a&gt; or as &lt;a href=&quot;http://www.cs.ucdavis.edu/~bai/ECS231/returnsfinal/Song.pdf&quot;&gt;preconditioners in krylov methods&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-01-14T19:14:04.370" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10544" PostTypeId="2" ParentId="10540" CreationDate="2014-01-14T19:22:06.317" Score="4" Body="&lt;p&gt;In this situation, you need an integrator with dense output, which means it has a capability to calculate the solution to the ODE at any point in time between the time steps taken by the integrator. &lt;code&gt;dop853&lt;/code&gt; (from Hairer and Wanner's eighth-order Dormand-Prince integrator) has this capability, in addition to adaptive time-stepping. Likely what happens is that the integrator keeps track of enough time steps internally to be able to interpolate solutions at intermediate times -- that is, the integrator has an &quot;internal discretization&quot;. When you call the integrator using &lt;code&gt;solver.integrate(solver.t + dt)&lt;/code&gt;, it adjusts the internal discretization (if necessary) so that the solution at &lt;code&gt;solver.t + dt&lt;/code&gt; can be interpolated from solution values calculated in the internal discretization. If you can read the &lt;a href=&quot;http://www.unige.ch/~hairer/prog/nonstiff/dop853.f&quot; rel=&quot;nofollow&quot;&gt;original Fortran source&lt;/a&gt;, the part around &quot;SAVE THE FIRST FUNCTION EVALUATION&quot; is probably the relevant code illustrating this strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to answer your question, the stepsize control generally does calculate at different time steps, but you don't need another time array for that. Just call the SciPy interface to &lt;code&gt;dop853&lt;/code&gt; directly to calculate the solution at the time points you want, and it will take care of everything.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-01-14T19:22:06.317" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10545" PostTypeId="2" ParentId="3229" CreationDate="2014-01-14T23:57:05.713" Score="6" Body="&lt;p&gt;&lt;strong&gt;Warning:&lt;/strong&gt; The below example works properly, but &lt;strong&gt;using the full set of parameters suggested at the post end exposes a bug&lt;/strong&gt;, or at least an &quot;undocumented feature&quot; in the numpy.take() function.  See comments below for details.  &lt;a href=&quot;https://github.com/numpy/numpy/issues/4285&quot; rel=&quot;nofollow&quot;&gt;Bug report filed&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this in-place with &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.take.html&quot; rel=&quot;nofollow&quot;&gt;numpy's take() function&lt;/a&gt;, but it requires a bit of hoop jumping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of doing a random permutation of an identity matrix's rows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;i = np.identity(10)&#xA;rr = range(10)&#xA;np.random.shuffle(rr)&#xA;np.take(i, rr, axis=0)&#xA;array([[ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],&#xA;       [ 0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.],&#xA;       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.],&#xA;       [ 0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.],&#xA;       [ 0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.],&#xA;       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.],&#xA;       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.],&#xA;       [ 1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],&#xA;       [ 0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],&#xA;       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To do it in-place, all you need to do is specify the &quot;out&quot; parameter to be the same as the input array &lt;strong&gt;AND&lt;/strong&gt; you have to set the mode=&quot;clip&quot; or mode=&quot;wrap&quot;.  If you don't set the mode it will make a copy to restore the array state on a Python exception &lt;a href=&quot;https://github.com/numpy/numpy/blob/aff396d13ad2e64dc0df1a095e53e7dcb67710d7/numpy/core/src/multiarray/item_selection.c#L108&quot; rel=&quot;nofollow&quot;&gt;(see here)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a final note, take seems to be a array method, so instead of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.take(i, rr, axis=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you could call&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;i.take(rr, axis=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if that is more to your taste.  So in total you call should look something like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Inplace Rearrange&#xA;arr = makeMyBixMatrix()&#xA;pVec0, pVec1 = calcMyPermutationVectors()&#xA;arr.take(pVec0, axis=0, out=arr, mode=&quot;clip&quot;)&#xA;arr.take(pVec1, axis=1, out=arr, mode=&quot;clip&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To permute both rows and columns I think you either have to run it twice, or pull some ugly shenanigans with &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.unravel_index.html&quot; rel=&quot;nofollow&quot;&gt;numpy.unravel_index&lt;/a&gt; that hurts my head to think about.   &lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2014-02-12T16:45:34.050" LastActivityDate="2014-02-12T16:45:34.050" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="10546" PostTypeId="2" ParentId="10542" CreationDate="2014-01-15T03:11:39.887" Score="6" Body="&lt;p&gt;Paul already gave the short answer (that all PDE and linear algebra packages do implement these methods, but that they are most frequently used only as smoothers in multigrid methods). The long answer can be found in lectures 34-38 here:&#xA;  &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-15T03:11:39.887" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10547" PostTypeId="1" CreationDate="2014-01-15T15:54:12.403" Score="1" ViewCount="60" Body="&lt;p&gt;I have a system of reactions that are governed by differential equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are reacting inside of a volume with known dimensions i.e l&lt;em&gt;b&lt;/em&gt;h. I don't have any other information on their position inside that volume. I simply know there are 200 of A, 300 of B etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when molecules pass out of this volume they no longer react in the same way and move around in a Brownian fashion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how often molecules leave this volume and I want to simulate this system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose I want a flux, or a probability distribution showing how likely an individual molecule leaves. I want to be able to &quot;Count&quot; the number of molecules left in V at the end of a timestep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ways I have thought about solving this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fick's first law: &#xA;I get a flux with this, but I thought you couldn't apply it in systems where the concentration isn't constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fick's second law:&#xA;I don't really know how I would put it into my simulation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Smirnov density:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(t)=\frac{x_0}{\sqrt{4\pi Dt^3}}\exp\left(-\frac{x_0^2}{4Dt}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;got this from &lt;a href=&quot;https://physics.stackexchange.com/questions/93498/simulating-diffusion-from-bulk-to-individual-particles&quot;&gt;https://physics.stackexchange.com/questions/93498/simulating-diffusion-from-bulk-to-individual-particles&lt;/a&gt; haven't really seen any other information on this. Maybe it's called something different?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this equation there is a problem in that I don't know where the particles are in the volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other approach I have thought of is using gibson-bruck next reaction and treating a transition as a reaction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please provide computational details in your answer, I need to be able to implement this.&lt;/p&gt;&#xA;" OwnerUserId="1017" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2014-01-15T16:45:48.900" Title="Calculating the number of molecules diffusing out of a volume" Tags="|simulation|diffusion|" AnswerCount="0" CommentCount="3" ClosedDate="2019-06-07T17:28:31.783" ContentLicense="CC BY-SA 3.0" />
  <row Id="10549" PostTypeId="1" AcceptedAnswerId="10662" CreationDate="2014-01-15T19:43:21.597" Score="7" ViewCount="1184" Body="&lt;p&gt;I am seeking to understand DDM's and their application to Maxwell's equations, though I am settling for the scalar Helmholtz as a baby step. Unfortunately I have hit a conceptual snag that I could use help with. My model problem is a 2D rectangle [-1,1]x[-1/2,1/2] split at x = 0 into left/right domains ($\Omega_L$ and $\Omega_R$). After expanding $u_L$ with scalar nodal interpolants, and testing with $v_L$ (galerkin, same space for $v_L$), and integration by parts over $\Omega_L$, I am left with the usual:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int \left(\nabla u_L \cdot \nabla v_L - k^2 u_L \cdot v_L \right) d\Omega_L - \int \left(v_L \frac{\partial u}{\partial n_L} \right) d\Gamma_L$ = 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\Gamma_L$ is the boundary of $\Omega_L$ at x = 0. For simplicity, I've omitted all the forcing terms (a point/line source) and boundary terms on surfaces other than $\Gamma_L$ (they're all ABC's or dirichlet scatterers). Because it's a wave equation, a first order Robin TC is used at the DDM interface:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u_L}{\partial n_L} + jk\cdot u_L = -\frac{\partial u_R}{\partial n_R} + jk\cdot u_R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here, there are two implementations I've pursued, (1) and (2). In strategy (1), which is adapted from a reference document (old colleagues thesis), a lagrange multiplier/auxiliary variable is introduced for each of the fluxes (surface currents) $\frac{\partial u_L}{\partial n_L}$ and $\frac{\partial u_R}{\partial n_R}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u_L}{\partial n_L} = g_L$,&#xA;$\frac{\partial u_R}{\partial n_R} = g_R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lagrange multipliers $g_L$ are discretized in space over $\Gamma$ with 1D nodal interpolants. Furthermore, the TC itself is tested over $\Gamma$ using the same space for tests $h_L$. This yields:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int \left(\nabla u_L \cdot \nabla v_L - k^2 u_L \cdot v_L \right) d\Omega_L - \int \left(v_L \cdot g_L \right) d\Gamma_L$ = 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int \left(h_L\cdot g_L\right)d\Gamma_L + jk \int \left(h_L\cdot u_L\right) d\Gamma_L + \int \left(h_L\cdot g_R\right)d\Gamma_L - jk \int \left(h_L\cdot u_R\right) d\Gamma_L = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Repeating the same testing process over the other domain $\Omega_R$ gives you the desired 4 equation/4 unknown system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[&#xA;\begin{array}{cccc}&#xA;\mathbf H_{vL,uL} &amp;amp; +k \mathbf T_{vL,gL} &amp;amp; \mathbf 0 &amp;amp; \mathbf 0\\&#xA;+k\mathbf T_{hL,uL} &amp;amp; +jk \mathbf T_{hL,gL} &amp;amp; -k \mathbf T_{hL,uR}&amp;amp;+jk \mathbf T_{hL,gR}\\&#xA;\mathbf 0 &amp;amp; \mathbf 0 &amp;amp; \mathbf H_{vR,uR} &amp;amp; +k \mathbf T_{vR,gR}\\&#xA;-k\mathbf T_{hR,uL} &amp;amp; +jk \mathbf T_{hR,gL} &amp;amp; +k \mathbf T_{hR,uR}&amp;amp;+jk \mathbf T_{hR,gR}\\&#xA;\end{array}&#xA;\right]&#xA;\left[&#xA;\begin{array}{c}&#xA;u_L \\&#xA;g_L \\&#xA;u_R \\&#xA;g_R \\&#xA;\end{array}&#xA;\right]&#xA;=&#xA;\mathbf 0&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the $\mathbf H$'s are the 2D integrals of the $\nabla^2-k^2$ operator, and all the $\mathbf T$'s are basically 1D mass/gramian matrices. Note there has been a little bit of tricky rescaling applied to the TC's to make the 2x2 L/R subdomain problems complex-symmetric, to ease the memory requirements to direct solve them. This system can be solved w/ either jacobi splitting or krylov/GMRES method (with block diagonal preconditioning, because the L and R subproblems resemble FEM's with low order ABC's enforced via multiplier's, well posed for all k). This method works as expected, in the following ways:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;the eigenvalues of the $\mathbf M^{-1} \mathbf N$ splitting are all on or within the unit circle, which matches the reflection coefficient behavior of the &quot;continuous&quot; ddm algorithm, where you inject plane wave &quot;error&quot; solutions for $u_L$, $u_R$ and analyze how these errors decay to zero as a function of $k_x$ and $k_y$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;on a conformal mesh, the fully discrete algorithm yields the exact same answer (to machine precision, or at least the residual tolerance of your iterative solver) as the non-ddm/monolithic FEM approach.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Bottom line is, this (1) approach works and mimics the results from my reference thesis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My confusion arises because I tried an alternate approach, (2), that I think should yield the same results but in fact does not. In strategy (2), lagrange multipliers are not used, and instead we solve the TC for the $\frac{\partial u_L}{\partial n_L}$... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u_L}{\partial n_L} = -jk\cdot u_L -\frac{\partial u_R}{\partial n_R} + jk\cdot u_R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... and then plug it into the boundary term in the volume test to yield:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int \left(\nabla u_L \cdot \nabla v_L - k^2 u_L \cdot v_L \right) d\Omega_L + jk \int \left(v_L \cdot u_L \right) d\Gamma_L - \int \left(v_L \cdot \frac{\partial u_R}{\partial n_R}\right) d\Gamma_L + jk\int \left(v_L \cdot u_R\right) d\Gamma_L$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you repeat the process on $\Omega_R$, you get the following system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[&#xA;\begin{array}{cccc}&#xA;\mathbf H &amp;amp; \mathbf H &amp;amp; \mathbf 0 &amp;amp; \mathbf 0\\&#xA;\mathbf H &amp;amp; \mathbf H+jk \mathbf T &amp;amp; \mathbf 0 &amp;amp; \mathbf D+jk \mathbf T\\&#xA;\mathbf 0 &amp;amp; \mathbf 0 &amp;amp; \mathbf H &amp;amp; \mathbf H\\&#xA;\mathbf D+jk \mathbf T &amp;amp; \mathbf 0 &amp;amp; \mathbf H &amp;amp; \mathbf H+jk \mathbf T \\&#xA;\end{array}&#xA;\right]&#xA;\left[&#xA;\begin{array}{c}&#xA;u_L^{\Omega} \\&#xA;u_L^{\Gamma} \\&#xA;u_R^{\Omega} \\&#xA;u_R^{\Gamma} \\&#xA;\end{array}&#xA;\right]&#xA;=&#xA;\mathbf 0&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is actually more like what I expected to see, because the 2x2 subdomain blocks are now clearly classic FEM's with ABC's on their boundaries (the $\mathbf H+jk\mathbf T$ stuff). The $\mathbf D$ &quot;derivative&quot; operator discretizes the $\int \left(v_L \cdot \frac{\partial u_R}{\partial n_R}\right) d\Gamma_L$, kinda like you're using it for forcing (Robin) data. This approach works in that it converges and yields an answer that appears correct, but there are some things I don't like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;both the preconditioned krylov and block jacobi iterations converge - but actually faster than they should. In brief, the first order Robin TC should yield unit reflection coefficients for evanescent fourier modes - which motivates the quest for higher order tc's. But for this algorithm all the eigenvalues of $\mathbf M^{-1} \mathbf N$ are considerably within the unit circle - NONE are actually on it, while this is good, it's  better than theory suggests, which makes me distrust it/think it's wrong.&lt;/li&gt;&#xA;&lt;li&gt;On a conformal mesh, the converged DMM result is quite close to the monolithic FEM result, but the error is far above machine precision. I drive GMRES to 10 digits, the backwards residual of the DDM Ax=b is 10 digits down, but the DDM and the monolithic FEM only agree to 3-4 digits. This is giving me heartburn / trustworthiness issues.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So the question (at long last) is this: have I commited some crime in strategy (2) when I forgoed the use lagrange multipliers to replace the flux of u, and instead just differentiated the primal variable directly via $\mathbf D$? If strategy (1) is the only way that really works, and (2) is junk, that's fine, I'll do (1) - it's more just understanding why (2) doesn't work, so that I won't commit the same crime again unknowingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I am not super familiar with a lot of DDM literature/techniques, so this issue might be covered in a pretty basic text on a simpler model problem (e.g. laplace/poisson?). Thanks in advance - I know this question is very long but just wanted to be as thorough as possible. I am pretty careful (normally) about implementation correctness, so I suspect there is some misunderstanding in the (2) formulation, not it's implementation (but no promises)&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2017-11-08T11:49:35.890" Title="FEM+DDM applied to scalar Helmholtz - necessity of lagrange multipliers?" Tags="|finite-element|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10550" PostTypeId="1" CreationDate="2014-01-16T09:21:32.377" Score="2" ViewCount="87" Body="&lt;p&gt;I am currently trying to calculate the geographic distance between two regions as I want to correlate it with their similarity of another aspect (e.g., similarity in word usage).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I have used a simple approach of determining the geographic center point of each region (a region corresponds to a province/state of a country) and then calculate the Haversine distance [1] between these two points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach seems rather ad-hoc to me and as I am no expert in this area, I am curious if you guys know a more reasonable approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://en.wikipedia.org/wiki/Haversine_formula&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Haversine_formula&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4527" LastActivityDate="2014-01-16T14:52:38.423" Title="Geographic distance between two regions" Tags="|geometry|spatial-data|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10552" PostTypeId="1" CreationDate="2014-01-16T14:33:47.740" Score="5" ViewCount="248" Body="&lt;p&gt;Let's say we want to solve $Ax=b$ ($A$ symmetric positive /semi/definite) with the conjugate residual/gradient method. $A$ comes from FEM where the mesh is being refined. The exact solution is $x_*$ and the approximate solution after $m$ iterations is $x_m$. Say that the initial guess is $x_0=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have that&#xA;$$\varepsilon=\left(\frac{(x_*-x_m, A(x_*-x_m))}{(x_*-x_m, A(x_*-x_m))}\right)^{1/2}=\left(\frac{(A^{1/2}(x_*-x_m), A^{1/2}(x_*-x_m))}{(A^{1/2}(x_*-x_m), A^{1/2}(x_*-x_m))}\right)^{1/2}\le2\left(\frac{\sqrt{\kappa}-1}{\sqrt{\kappa}+1}\right)^m$$&#xA;with $\kappa$ being the condition number of $A$. But how can I calculate or estimate this in practice? I don't have $x_*$, all I have is $x_m$ and thus $r_m=b-Ax_m$. Sure, I can calculate&#xA;\begin{align}&#xA;\tilde{\varepsilon}=\left(\frac{(r_m,r_m)}{(r_0,r_0)}\right)^{1/2}&amp;amp;=\left(\frac{(b-Ax_m,b-Ax_m)}{(b-Ax_0,b-Ax_0)}\right)^{1/2}=\left(\frac{(b-Ax_m,b-Ax_m)}{(b-Ax_0,b-Ax_0)}\right)^{1/2} \\&#xA;&amp;amp;=\left(\frac{(A(x_*-x_m),A(x_*-x_m))}{(A(x_*-x_0),A(x_*-x_0))}\right)^{1/2}=\left(\frac{(A(x_*-x_m),A(x_*-x_m))}{(b,b)}\right)^{1/2}&#xA;\end{align}&#xA;but then what? For a fix $A$ I think I'll still have the same asymptotic decrease in the $A$-norm when using $\tilde{\varepsilon}$ as stopping criterion, that is $\left(\frac{\sqrt{\kappa}-1}{\sqrt{\kappa}+1}\right)^m$, so if I continue until $\tilde{\varepsilon}&amp;lt;\tilde{\varepsilon}_{\mathrm{max}}$ then the number of iterations needed are $m=c\sqrt{\kappa}$. But even so, $c$ depends on $A$! That means my iteration count could behave oddly if I'm using the algorithm for different matrices $A$ (corresponding to a mesh that has been refined a different number of times), does it not? Using $\tilde{\varepsilon}$ might not give the wanted $\sqrt{\kappa}$ growth of the iteration count. (Now, I'm not sure why $\varepsilon$ is a better stopping criterion than $\tilde{\varepsilon}$, but using one for theory and the other for implementation without understanding how they're connected feels very immoral.)&lt;/p&gt;&#xA;" OwnerUserId="4669" LastActivityDate="2014-01-16T14:33:47.740" Title="Conjugate residual/gradient convergence checking in practice" Tags="|finite-element|iterative-method|convergence|conjugate-gradient|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10553" PostTypeId="2" ParentId="10550" CreationDate="2014-01-16T14:52:38.423" Score="3" Body="&lt;p&gt;The Haversine formula is really the best way to calculate distances on the Earth unless you have outstanding accuracy concerns.  The next step passed the Haversine involves taking into account that the Earth is not a sphere but ellipsoidal and which involves a much more expensive computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a good page with some embedded javascript at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.movable-type.co.uk/scripts/latlong.html&quot; rel=&quot;nofollow&quot;&gt;http://www.movable-type.co.uk/scripts/latlong.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that provides some explanation of issues involved and some alternatives such as the law of cosines (less accurate but faster than the Haversine formula).&lt;/p&gt;&#xA;" OwnerUserId="567" LastActivityDate="2014-01-16T14:52:38.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10554" PostTypeId="1" AcceptedAnswerId="10559" CreationDate="2014-01-16T15:30:39.880" Score="2" ViewCount="155" Body="&lt;p&gt;I have a very large(up to 9 000 000) &lt;code&gt;VectorXi&lt;/code&gt; vector defined in Eigen C++ which is read from a data file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vector consists of 0, 1, 2, ...n segments with randomized lengths like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0 0 ... 0 1 1 1 ... 1 2 2 ...2 3 ... 3 ... k k ... k k+1 k+1 ... k+1 .... n n ...n]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to count the lengths of each segement and store them into a new &lt;code&gt;n+1&lt;/code&gt; VectorXi, and index the starting positions of each segment which is also stored into a new &lt;code&gt;n+1&lt;/code&gt; VectorXi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I implement it faster? I found sometimes it may take me up to 1 hour or so to finish it!!&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6693" LastEditDate="2014-01-17T02:53:13.007" LastActivityDate="2014-01-17T02:53:13.007" Title="How can I efficiently position the different segments of a large vector in Eigen C++?" Tags="|c++|eigen|vector|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10555" PostTypeId="1" AcceptedAnswerId="10556" CreationDate="2014-01-16T15:33:06.107" Score="5" ViewCount="1535" Body="&lt;p&gt;I have a C++ program/model that in some parts already use IPOPT (with ADOL-C and ColPack) to solve some pretty large nonlinear problems.&lt;/p&gt;&#xA;&lt;p&gt;Now in another part of the program, I need to solve a large amount (tens of thousands) of small quadratic problems.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to have your suggestion if you think it should be more efficient to use again IPOPT, that it is very good for large generic non-linear problems but it seems it takes too much to initialise or if you think it should be better to use a specialized quadratic programming solver that is particularly fast for small problems (and in that case, a name would be appreciated).&lt;/p&gt;&#xA;&lt;p&gt;For completeness my quadratic problem is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;min: sum_i sum_j a_i,j x_i x_j&#xA;sub&#xA;sum_i x_i = 1&#xA;x_i &amp;gt;= 0&#xA;sum_i x_i b_i = c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with &lt;code&gt;i=j ~ 10&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Problems are independent.&lt;/p&gt;&#xA;" OwnerUserId="7087" LastEditorUserId="4325" LastEditDate="2021-12-20T02:29:34.813" LastActivityDate="2021-12-20T02:29:34.813" Title="Solver suggestion for many small quadratic problems in C++" Tags="|optimization|c++|libraries|quadratic-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10556" PostTypeId="2" ParentId="10555" CreationDate="2014-01-16T16:18:48.927" Score="5" Body="&lt;p&gt;If your problems are small, you could try a simple solver with dense data structures and use stack allocation (old Fortran codes). IPOPT uses sparse data structures and indeed, it takes a &lt;em&gt;relatively&lt;/em&gt; long time to initialize IPOPT if your problems are small. Old Fortran codes with stack allocation are surprisingly fast if you have small problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the license is not an issue for you, you could look at the &lt;a href=&quot;http://www.hsl.rl.ac.uk/archive/index.html&quot; rel=&quot;noreferrer&quot;&gt;Harwell Subroutine Library&lt;/a&gt;. I belive &lt;a href=&quot;http://www.hsl.rl.ac.uk/archive/specs/ve02.pdf&quot; rel=&quot;noreferrer&quot;&gt;VEO2 Minimize a quadratic function, linear constraints&lt;/a&gt; is likely to be good for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another appealing option is the MNHL subroutine: &lt;em&gt;general optimization, requiring gradient and Hessian, linear constraints&lt;/em&gt; from &lt;a href=&quot;http://open-innovation.alcatel-lucent.com/frs/?group_id=30&amp;amp;release_id=10&quot; rel=&quot;noreferrer&quot;&gt;PORT3&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.yolinux.com/TUTORIALS/LinuxTutorialMixingFortranAndC.html&quot; rel=&quot;noreferrer&quot;&gt;Interfacing C++ and Fortran codes&lt;/a&gt; is painful and boring but really not a rocket science. I believe these old Fortran codes are likely to be fast enough for your application. (Take it from someone who loves C++.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If performance is an issue, I wouldn't use automatic differentiation on your quadratic program. Instead, I would hand-code the derivatives.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My I ask: What is your application? How do you get that many small quadratic programs?&lt;/p&gt;&#xA;" OwnerUserId="679" LastEditorUserId="679" LastEditDate="2014-01-16T16:24:49.560" LastActivityDate="2014-01-16T16:24:49.560" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10557" PostTypeId="1" CreationDate="2014-01-16T18:17:37.707" Score="2" ViewCount="189" Body="&lt;p&gt;OK, I have a FORTRAN code which numerically integrates equations of motion for large data sets of initial conditions. I run this program in my PC and it requires about 1 day of computations per data set. So, I was wandering if there is any site with super-fast PCs (or better a grid of processors) in which I could upload the .exe and perform my calculations much faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?!&lt;/p&gt;&#xA;" OwnerUserId="7088" LastEditorUserId="27" LastEditDate="2014-01-16T19:08:11.497" LastActivityDate="2014-01-17T09:21:39.577" Title="Ways to speed up the computations" Tags="|performance|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10558" PostTypeId="2" ParentId="10557" CreationDate="2014-01-16T18:37:59.510" Score="3" Body="&lt;p&gt;There are commercial services that do exactly this.  See for example &lt;a href=&quot;http://aws.amazon.com&quot; rel=&quot;nofollow&quot;&gt;Amazon Web Services&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're doing this as a research project, you may be able to get time at one of the many supercomputing centers.  See for example the &lt;a href=&quot;http://www.ncsa.illinois.edu/&quot; rel=&quot;nofollow&quot;&gt;National Center for Supercomputing Applications&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-16T18:37:59.510" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10559" PostTypeId="2" ParentId="10554" CreationDate="2014-01-16T20:49:28.333" Score="4" Body="&lt;p&gt;This is a &lt;code&gt;O(m)&lt;/code&gt; problem where &lt;code&gt;m&lt;/code&gt; is the size of the big array. So this should be fast:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int n = v[v.size()-1];&#xA;int k=0;&#xA;VectorXi lengths[n+1], starts[n+1];&#xA;for(int i=0; i&amp;lt;v.size(); ++i) {&#xA;  int i0 = i;&#xA;  while(v[i]==k &amp;amp;&amp;amp; i&amp;lt;n) ++i;&#xA;  lengths[k] = i-i0;&#xA;  starts[k] = i;&#xA;  ++k;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4337" LastActivityDate="2014-01-16T20:49:28.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10561" PostTypeId="2" ParentId="10557" CreationDate="2014-01-17T09:21:39.577" Score="4" Body="&lt;p&gt;I have the feeling you are confused about how parallel processors work. You state your &lt;em&gt;serial&lt;/em&gt; code runs faster on your dual core machine than on another single core CPU. That has nothing to do with the fact that your CPU dual-core, but that it's clock rate is higher (and probably also performs more instructions per clock cycle). A dual-core 2.2 GHz CPU indeed doesn't sound like a state-of-the-art CPU, so you would gain some performance running on a more modern CPU (see &lt;a href=&quot;http://www.cpubenchmark.net/singleThread.html&quot; rel=&quot;nofollow&quot;&gt;http://www.cpubenchmark.net/singleThread.html&lt;/a&gt; for a single threaded benchmark). On top of that, if you want to process multiple datasets, a multi-core CPU does indeed give you some benefits. In fact, if you do that a lot you might even opt for a many-core CPU such as the AMD Opteron 6000 series (up to 16 cores), even though their single-threaded performance is lower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have to process &lt;em&gt;a lot&lt;/em&gt; of datasets, such an upgrade wont cut it. In this case you can try to get computing time on a computing grid [1], where you can use hundreds of cores simultaneously. Note that again, you will only benefit from this when processing a lot of different datasets. If on the other hand you have only a few very large datasets, you will see no benefit. In this case you'll have to work on dividing your problem to utilize multiple cores at the same time (parallelization e.g. using &lt;a href=&quot;https://en.wikipedia.org/wiki/Message_Passing_Interface&quot; rel=&quot;nofollow&quot;&gt;MPI&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] From my personal experience, most national computing centres run their grids on Linux/UNIX. Since you mention you have an .exe, you'll have to shop around a bit to find a Windows HPC or you'll have to port your program.&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2014-01-17T09:21:39.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10562" PostTypeId="1" AcceptedAnswerId="10592" CreationDate="2014-01-17T12:30:47.347" Score="11" ViewCount="3345" Body="&lt;p&gt;Is there a truncated SVD algorithm that computes the singular values one at a time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem: I would like to compute the first $k$ singular values (and singular vectors) of a large dense matrix $M$, but I don't know what an appropriate value of $k$ would be. $M$ is large, so for efficiency reasons, I would rather not evaluate the full SVD only to truncate off the smallest SV's afterwards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, there would be a way to compute the singular values $\sigma_1, \sigma_2,\ldots$ serially, from largest ($\sigma_1$) to smallest ($\sigma_n$). That way, I could simply halt the computation after computing the $k$th singular value if $\sigma_k/\sigma_1$ falls below some threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does such an algorithm exist (preferably with a Python implementation)? In my googling around, I've only found truncated SVD functions that take k as a parameter, thus forcing you to guess it a priori. &lt;/p&gt;&#xA;" OwnerUserId="7093" LastEditorUserId="20688" LastEditDate="2017-04-04T18:02:54.530" LastActivityDate="2017-04-04T18:02:54.530" Title="computing the truncated SVD, one singular value/vector at a time" Tags="|linear-algebra|algorithms|svd|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10563" PostTypeId="1" AcceptedAnswerId="10565" CreationDate="2014-01-17T13:28:03.387" Score="15" ViewCount="388" Body="&lt;p&gt;I'm currently writing up my PhD thesis. I spent a significant fraction of my PhD cleaning up and extending existing scientific code, applying software engineering best practices which were previously not used, and would like to write about this in my thesis. Rather than simply saying &quot;I added unit tests&quot;, I want to be able to write something like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;J. Doe invented unit tests in 1975$^{[23]}$.&#xA;  A recent study by Bloggs et al$^{[24]}$ showed that unit tests reduce the incidence of software errors by 73%...&#xA;  234 separate unit tests were added to the code base, managed by the xUnit framework created by Timpkins et al$^{[25]}$ &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm looking for citable academic references (preferably articles in peer-reviewed journals where I can get DOIs, BibTeX etc) to widely accepted software engineering best practices, specifically:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;unit tests&lt;/li&gt;&#xA;&lt;li&gt;version control&lt;/li&gt;&#xA;&lt;li&gt;modularisation / separation of concerns&lt;/li&gt;&#xA;&lt;li&gt;performance profiling / optimisation based on profiling information&lt;/li&gt;&#xA;&lt;li&gt;bug/issue tracking&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm looking for information both about the initial invention and on subsequent evaluations of effectiveness. If there's a review article that lists all of this stuff in one place then so much the better.&lt;/p&gt;&#xA;" OwnerUserId="3458" LastActivityDate="2014-02-11T20:09:52.360" Title="Citable references for software best practices" Tags="|reference-request|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10564" PostTypeId="2" ParentId="10563" CreationDate="2014-01-17T17:50:38.037" Score="6" Body="&lt;p&gt;I don't have references for the origin of each of these ideas/practices.  But here are some very recent, relevant references:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications/2013-software.pdf&quot;&gt;What makes computational open source software&#xA;libraries successful?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://arxiv.org/abs/1210.0530&quot;&gt;Best Practices for Scientific Computing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-01-17T17:50:38.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10565" PostTypeId="2" ParentId="10563" CreationDate="2014-01-17T18:49:25.643" Score="13" Body="&lt;p&gt;Steve McConnell's book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0735619670&quot;&gt;Code Complete, 2nd edition&lt;/a&gt; has an extensive bibliography discussing these issues from more of the standpoint of software developers than computational scientists. The book is starting to become a little dated, in that it's approaching a decade old, so it doesn't cover more recent testing methodologies like behavior-driven development. Nevertheless, it is the closest thing to a comprehensive review article on software construction that I am aware of. You could also look for articles in IEEE Software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the computational science side of things, I think the best article is probably the &lt;a href=&quot;http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001745&quot;&gt;PLoS version&lt;/a&gt; of the arXiv preprint DavidKetcheson cited on &quot;Best Practices for Scientific Computing&quot;. I say this coming from an engineering background, where fewer people cite arXiv references or post arXiv preprints, and thus, citing a &quot;real journal article&quot; (setting aside, of course, all of the issues about scientific publishing that are being debated right now) is looked upon more favorably (and I get the impression that is why those authors chose to publish it in a journal).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors of the PLoS paper that DavidKetcheson and I cited are part of an organization called &lt;a href=&quot;http://software-carpentry.org/index.html&quot;&gt;Software Carpentry&lt;/a&gt; that puts on (usually 2 day) &quot;boot camps&quot; to teach scientists about some best practices for software development and useful computational skills for scientists (not just computational scientists). The Software Carpentry web site has &lt;a href=&quot;http://software-carpentry.org/bib/bib.html&quot;&gt;an extensive bibliography&lt;/a&gt; related to software development in science. If you're interested in these issues, I encourage you to reach out to them; they're always looking for more advocates of best practices in software development to do volunteering in various capacities. (&lt;em&gt;Disclaimer&lt;/em&gt;: I volunteer with Software Carpentry.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another common justification for engaging in software development best practices is reproducibility. Victoria Stodden has curated a &lt;a href=&quot;http://reproducibleresearch.net/index.php/RR_links&quot;&gt;long list of reproducible research references&lt;/a&gt; that may be of interest, depending on what you want to say.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-01-17T18:49:25.643" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10566" PostTypeId="2" ParentId="10562" CreationDate="2014-01-17T19:02:49.240" Score="2" Body="&lt;p&gt;(Edited, because I misread the question at first; you already know that there are routines available to calculate the first $k$ singular values.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you exclude the approach of calculating the whole SVD, partial SVD algorithms reduce to using iterative methods to solve a related Hermitian eigenvalue problem. So, one strategy you could take would be to hand-code this sort of thing yourself, and keep solving for the largest remaining unsolved singular value until you want to stop, using something like a shift-and-invert strategy. There may be elegant ways of doing this sort of thing in sophisticated packages like &lt;a href=&quot;http://www.grycap.upv.es/slepc/&quot; rel=&quot;nofollow&quot;&gt;SLEPc&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another strategy would be the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Calculate the largest singular value $s_{1}$.&lt;/li&gt;&#xA;&lt;li&gt;Set the absolute tolerance of the sparse SVD routine to $\tau \cdot s_{1} \cdot f$, where $\tau$ is your threshold, and $0 &amp;lt; f \leq 1$ is some safety factor to determine how many possibly extraneous singular values you want to calculate.&lt;/li&gt;&#xA;&lt;li&gt;Call the sparse SVD routine.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If the sparse SVD routine calculates a thin SVD (and I can't see why it wouldn't), then this strategy gives you all of the singular values you want (plus possibly some extra ones), because values below the absolute tolerance will be treated as zero. In that case, you can use &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.svds.html&quot; rel=&quot;nofollow&quot;&gt;scipy.sparse.linalg.svds&lt;/a&gt;, noting that $k$ is an optional parameter, and that you don't have to specify it &lt;em&gt;a priori&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-01-17T19:37:44.210" LastActivityDate="2014-01-17T19:37:44.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10567" PostTypeId="1" CreationDate="2014-01-17T20:22:53.230" Score="3" ViewCount="1456" Body="&lt;p&gt;I apologize if this question in a more general form has been asked before. I have a tridiagonal Toeplitz matrix $K$, whose eigenvalues and eigenvectors are known analytically for any dimension $N$ [&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0377042705000464&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt;]. Specifically, in the notation of Section 2.2 of this paper, the matrix I wish to diagonalise is such that $a_1 = a_2$ $= -2$, and $ b_1 = b_2=c_1=c_2=1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;K = \begin{pmatrix}&#xA;-2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0&amp;amp;..&amp;amp;0 \\&#xA;1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0&amp;amp;..&amp;amp;0 \\&#xA;0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1&amp;amp;..&amp;amp;0 \\&#xA;. \\&#xA;.&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then theorem 2.1 gives its eigenvalues and eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are: $\lambda_i = -4 \cos^2\left({\frac{(i+1)\pi}{2(N+1)}}\right)$ and eigenvectors $v_i = (.,.,\left[U(\frac{j}{2}, \frac{(\lambda_i+2)^2-2}{2}) + U(\frac{j}{2}-1, \frac{(\lambda_i+2)^2-2}{2})\right],.,.)^T$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U(j,x)$ is the Chebyshev polynomial of the second kind of order $j$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I further wish to do is to use the matrix of eigenvectors, say $A$, to implement similarity transforms. (Note that $A$ is not sparse at all in this example). How best can I do it computationally? In Python, for $ N = 10^4 $, the program takes a lot of time to even calculate $A$ and store the matrix, presumably because of the presence of the Chebyshev polynomial. On the other hand, using the sparse linear algebra module scipy.sparse.linalg.eigs finishes the task faster, even though I don't give it any information! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is this- how do I use a combination of my knowledge of the true eigenvalues and eigenvectors and the efficient linear algebra routines in SciPy to implement the similarity transform on a diagonal matrix D?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if I use just the information about eigenvalues, can I efficiently compute the eigenvectors? Are there routines that compute the eigenvectors after taking an input of eigenvalues if they are known?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the following code was my trial in Python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.special import eval_chebyu&#xA;from scipy import sparse as sp&#xA;&#xA;N = 10**3&#xA;&#xA;def Mat(x,j):&#xA;if j%2==0:&#xA;    if j==0:&#xA;        return 1&#xA;    else:&#xA;        return eval_chebyu(j/2, ((x+2)**2-2)/2) + eval_chebyu(j/2-1, ((x+2)**2-2)/2)&#xA;else:&#xA;    return (x+2)*eval_chebyu((j-1)/2, ((x+2)**2-2)/2)&#xA;&#xA;def norm(A):&#xA;A=A.transpose()/np.sqrt(np.einsum('...i,...i', A,A ))&#xA;return A&#xA;&#xA;def tridiag(a, b, c, N):&#xA;return sp.diags(a*np.ones(N-1), -1) + sp.diags(b*np.ones(N), 0) +sp.diags(c*np.ones(N-1), 1)&#xA;&#xA;K = tridiag(1,-2,1,N)&#xA;A = np.zeros([N,N])&#xA;lam = np.zeros(N)&#xA;&#xA;for i in range(0,N):&#xA;    lam[i] = -4*(np.cos((i+1)*np.pi/(2*(N+1))))**2&#xA;    for j in range(0,N):&#xA;        A[i,j]= Mat(lam[i],j)&#xA;&#xA;A = norm(A)&#xA;&#xA;Tq_new = sp.diags(1/(2*omega*np.sqrt(-lam)),0)&#xA;&#xA;Tq_old = A.transpose().dot(Tq_new.todense()).dot(A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7096" LastEditorUserId="7096" LastEditDate="2014-01-18T12:18:57.350" LastActivityDate="2014-01-18T12:18:57.350" Title="How to use eigenvalue information to efficiently diagonalize matrices?" Tags="|linear-algebra|matrix|eigenvalues|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10568" PostTypeId="2" ParentId="10567" CreationDate="2014-01-18T00:48:06.713" Score="2" Body="&lt;p&gt;Gilbert Strang explains the matrix square root of the second difference matrix &lt;a href=&quot;http://math.mit.edu/highdegree/TplusH.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;import numpy&#xA;import scipy.linalg&#xA;&#xA;def f(N, p):&#xA;    a = 2 * (-1)**(p-1)&#xA;    b = 1/numpy.tan((numpy.pi * (2*p - 1)) / (4 + 4*N))&#xA;    c = 1/numpy.tan((numpy.pi * (2*p + 1)) / (4 + 4*N))&#xA;    return a - b + c&#xA;&#xA;N = 10000&#xA;K_tri = numpy.eye(N) - numpy.eye(N, k=1)&#xA;K = K_tri + K_tri.T&#xA;&#xA;s = numpy.arange(N)&#xA;T = scipy.linalg.toeplitz(f(N, s))&#xA;H = scipy.linalg.hankel(f(N, s+2), f(N, N+1+s))&#xA;K_sqrt = (0.5 / (N+1)) * (T - H)&#xA;&#xA;print(numpy.max(numpy.abs(numpy.dot(K_sqrt, K_sqrt) - K)))&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&#xA;2.14717132963e-13&#xA;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2014-01-18T03:12:04.660" LastActivityDate="2014-01-18T03:12:04.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10569" PostTypeId="2" ParentId="10563" CreationDate="2014-01-18T06:27:52.780" Score="-2" Body="&lt;p&gt;IMHO I would take great care citing &quot;Best Practices&quot; in the context of scientifically proven approaches. Most practices are derived from &quot;what seems to work for a set of projects by someone perceived as guru involved in those projects&quot;, rather than derived from rigorous testing of different approaches. There are just too many variables and human factors in software engineering to state there is a referable list of &quot;best practices&quot; (e.g. a practice that works on one project will completely fail on another).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would approach it by stating what your project needed, why it needs it and add references to methods used and why you used them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also lean towards reporting quantifiable results rather than references to state your point. For example, if your unit tests uncovered 100 bugs, 10 of which serious enough to cast doubt on previously published result. This is a much more powerful statement to have in your PhD than a statement that you know the origin of unit tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: (fixed typo) - Answering the following - I often give raising children as an analogy to software projects. There there are many methods and tested ways to raise them, trying to raise your children with one method because it works for the average or a tested subsample, will work as long as your child is the same as the one tested. It is better to know many methods and apply the ones that work in your instance. Yes unit testing may be proven, but applying it based on that alone could mean your project gets to market late and therefore fails it's objective (if that is the objective). I am saying that applying a method to get an outcome and giving the result of that outcome, in my opinion is better in a thesis, than listing things that you tried based on other projects - unless of course the topic of the thesis is measuring methodologies :) &lt;/p&gt;&#xA;" OwnerUserId="564" LastEditorUserId="564" LastEditDate="2014-02-11T20:09:52.360" LastActivityDate="2014-02-11T20:09:52.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10570" PostTypeId="1" CreationDate="2014-01-18T10:19:25.900" Score="3" ViewCount="818" Body="&lt;p&gt;I have a nxn covariance matrix (so, real, symmetric, dense, nxn). 'n' may be very very very big! I'd like to solve complete eigenvalue (+eigenvectors) problem for this matrix. Could somebody tell me what the fastest algorithm to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.&#xA;I'd like to make GPGPU implementation using OpenCL. Typical sizes is 10000x10000 or even bigger. That's why I'm looking for fastest algorithm&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.S.&#xA;I'd like to write own implementation (just for education) so please don't advise already existing libraries&lt;/p&gt;&#xA;" OwnerUserId="7100" LastEditorUserId="7100" LastEditDate="2014-01-21T07:14:28.373" LastActivityDate="2014-01-21T07:14:28.373" Title="Algorithm for Complete Eigenvalue Problem of a Real Symmetric nxn Matrix" Tags="|linear-algebra|algorithms|eigensystem|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10571" PostTypeId="2" ParentId="10570" CreationDate="2014-01-18T17:34:25.143" Score="5" Body="&lt;p&gt;10000 by 10000 isn't really very big.  A double precision matrix of this size requires 800 megabytes of memory.  You'll need at least as much memory to hold the matrix of resulting eigenvectors, and you'll need additional working storage, but this is well within the capability of a typical desktop machine with 8 gigabytes or more of memory.  It would also fit into the graphics memory on a higher end GPGPU.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of time, this also isn't too bad.  I just computed the eigenvalues and eigenvectors for a random symmetric 100000 by 10000 matrix on a desktop machine (using MATLAB/LAPACK and running with 8 cores), and it took about 3 minutes to run.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only need to do this computation for a small number of matrices of this size, then it's likely not worth your time and effort to move this to a GPGPU.  However, if you plan on doing these computations 24/7, then putting some effort into speeding them up might be appropriate.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the sort of computation that you might be able to speed up with a high end GPGPU, but don't be surprised if it doesn't perform that well on a more basic graphics card.  The issues here are that your graphics card might not have sufficient memory, and that double precision performance is not a strong point of many GPGPU's.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since eigenvalue computation is commonly available through the LAPACK libraries, I'd suggest that you look for an implementation of the LAPACK libraries that has been optimized for your GPGPU.  In particular, check out the OpenCL version of the MAGMA library:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://icl.cs.utk.edu/magma/software/index.html&quot;&gt;Magma&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-18T17:34:25.143" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10572" PostTypeId="1" CreationDate="2014-01-19T04:16:13.047" Score="4" ViewCount="145" Body="&lt;p&gt;I would like to numerically integrate a smooth function (to be precise, I am computing differential entropies of Gaussian mixtures of the form $\int f(x) \log(1/f(x)) dx$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any software package or library that gives me the exact answer up to X digits (and potentially more), where X can be rather large? The GSL numerical integration seems to provide a solution with its claim:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The algorithms &lt;strong&gt;attempt&lt;/strong&gt; to estimate the absolute error ABSERR = |RESULT&#xA;  - I| in such a way that the following inequality holds,&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;|RESULT - I| &amp;lt;= ABSERR &amp;lt;= max(epsabs, epsrel |I|)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;but first I don't understand what is exactly meant by &quot;attempt&quot; (boldface added above) and second I would like to go beyond machine floating point if possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am OK if the software gives up (if the requested accuracy/precision is too severe), but once an answer is out, it ought to be rather reliable. (Otherwise, what would be the point of computing it to begin with?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This must be a very basic problem in numerical integration, but my search over the past few weeks have been futile. Any thread would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="7106" LastEditorUserId="4244" LastEditDate="2014-01-20T08:21:28.367" LastActivityDate="2014-01-20T08:21:28.367" Title="Reliable quadrature software" Tags="|quadrature|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10573" PostTypeId="2" ParentId="10572" CreationDate="2014-01-19T05:24:12.010" Score="6" Body="&lt;p&gt;You can't bound the accuracy of any numerical integration scheme without some control over how crazy the integrand can be.  Smoothness is obviously helpful, but if the derivatives aren't bounded, then it's possible to introduce a $C^{\infty}$ bump into the integrand that can have an arbitrarily large impact on the value of the integral and you might not ever sample from that portion of the integration interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, adaptive quadrature schemes attempt to estimate the error and these error estimates are quite reliable for most practical situations in which the integrand is smooth and the derivatives are bounded (even if you don't know the bounds), but there are no theoretical guarantees for a general smooth integrand.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest that you look at symbolic computation software packages like Maple and Mathematica that offer arbitrarily high precision floating point arithmetic and have robust numerical quadrature routines.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-19T05:24:12.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10574" PostTypeId="1" CreationDate="2014-01-19T21:28:34.683" Score="1" ViewCount="114" Body="&lt;p&gt;Consider one initial boundary value problem for sphere.&#xA;$$\frac{\partial u}{\partial t}=\operatorname{div}A\nabla u +f$$&#xA;Here is explicit numerical scheme (we consider that it is stable):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u^{n+1}_{i,j,k}-u^{n}_{i,j,k}}{\Delta t}=A_{i,j,k}{\frac{\partial u}{\partial r}}_{i,j,k,+/-}^{n}+B_{i,j,k}{\frac{\partial u}{\partial \varphi}}_{i,j,k,+/-}^{n}+C_{i,j,k}{\frac{\partial u}{\partial \psi}}_{i,j,k,+/-}^{n}+&#xA;R_{i,j,k}{\frac{\partial^{2} u}{\partial r^2}}_{i,j,k}^{n}+\Phi_{i,j,k}{\frac{\partial^{2} u}{\partial \varphi^2}}_{i,j,k}^{n}+\Psi_{i,j,k}{\frac{\partial^{2} u}{\partial \psi^2}}_{i,j,k}^{n}+f^{n}_{i,j,k} ;$$&#xA;$u(r,\varphi,\psi,t=0)=u_{0}(r,\varphi,\psi)$;$u(r=r_{0},\varphi,\psi,t)=0;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under $+/-$ for operator it is understood that for operator was used upwind scheme; the operators are standart.&#xA;It is clear that such scheme works for all domain, except points on $r=r_{0}$(actually, we don't need to compute values here, because they are known because of boundary condition) and for point $r=0$ in the center of the ball.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following method for evaluating $u^{n+1}$ in the center of the ball: we use the definition of divergence&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\operatorname{div}(A(x,y,z))=\lim\limits_{R\to 0}\frac {\oint_{B_{R}(x,y,z)}{(A,n)ds}}{V_{R}}, $$&#xA;therefore,we can write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u^{n+1}_{1,j,k}-u^{n}_{1,j,k}}{\Delta t}=\frac {\oint_{B_{r_2}}{(A\nabla u,\overline{n})ds}}{V_{r_2}} +\overline{f}^{n}_{1,j,k} ,$$&#xA;where $B_{r_2}$ -- a ball of radius equal to the value of unit step for $r$, $\overline{f}$ -- mean value of $f$ on the surface of $B_{r_2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: will this method be stable for the center of the point?&#xA;If not, what stable method of computing the value in the center of ball you can offer?&lt;/p&gt;&#xA;" OwnerUserId="4676" LastActivityDate="2014-01-19T21:28:34.683" Title="Stability question (finite difference): dealing with corner nodes" Tags="|pde|finite-difference|stability|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10575" PostTypeId="2" ParentId="10549" CreationDate="2014-01-19T21:32:11.980" Score="1" Body="&lt;p&gt;As noone else is replying so far I'll try to make remarks and question that could check helf for your search for deeper insight. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One question from my side concerning your discretization within the subdomains: You say your ansatz functions are &quot;nodal interpolants&quot;. Does that imply that the ansatz functions are continuous in each subdomain or are you using piecewise polynomials which are discontinuous? Which order within each element are you using? &#xA;If you're using continuous functions, that means that in your DD-solution you should have $u_L|_{\Gamma} = u_R|_{\Gamma}$ if the solution coincides with the monolithic approach, right? Did you check for that?&lt;/li&gt;&#xA;&lt;li&gt;Which degrees of freedom do you mean with $u_L^{\Gamma}$? Only the nodes at the interface or all degrees of freedom which are on elements at the boundary? &lt;/li&gt;&#xA;&lt;li&gt;Have you tried the problem in 1D? If you do it in 1D it is perhaps even simpler to investigate certain properties of your discretization, e.g. the system matrix. I suggest you try a simple 1D problem. Than you solve both problems with a direct solver (just to exclude another error source at the beginning). You could further check the eigenvalues of the system matrices for both cases and compare them. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7072" LastActivityDate="2014-01-19T21:32:11.980" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10576" PostTypeId="2" ParentId="10407" CreationDate="2014-01-19T23:46:16.617" Score="2" Body="&lt;p&gt;For the advection-diffusion equation you can't just apply a &#xA;Neumann boundary condition (the 'gradient zero condition' as you called it) because there is are two components to the flux: advection and diffusion. Therefore you are correct to apply Robin boundary conditions (mixed boundary conditions as you called it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want the edges of your domain to prevent mass from leaving then you simply apply the zero flux condition, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Gamma^N} \rho^* \mathbf{\Gamma} \cdot \mathbf{n} d\Gamma^N = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information see theses question/answers:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/5434/conservation-of-a-physical-quantity-when-using-neumann-boundary-conditions-appli&quot;&gt;Conservation of a physical quantity when using Neumann boundary conditions applied to the advection-diffusion equation&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/7650/how-should-boundary-conditions-be-applied-when-using-finite-volume-method&quot;&gt;How should boundary conditions be applied when using finite-volume method?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you were to apply the condition $\frac{\partial n}{\partial x} = 0$ at the boundary this would be equivalent to preventing diffusion through the boundary, but it would be an open boundary condition for the advection component of the flux. This may or may not be what you want.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-01-19T23:46:16.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10578" PostTypeId="2" ParentId="10570" CreationDate="2014-01-20T07:19:28.370" Score="4" Body="&lt;p&gt;Regarding your question about the fastest algorithm for the complete symmetric eigenproblem...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two main phases to the computation. First is reduction to tridiagonal form using (blocked) Householder reflections. This runs in cubic time w.r.t the matrix dimension, and you can easily code up your own &lt;a href=&quot;https://github.com/victorliu/RNP2/blob/master/RNP/LA/Tridiagonal.hpp#L799&quot; rel=&quot;nofollow&quot;&gt;non-blocked&lt;/a&gt; version, while the &lt;a href=&quot;https://github.com/victorliu/RNP2/blob/master/RNP/LA/Tridiagonal.hpp#L124&quot; rel=&quot;nofollow&quot;&gt;blocked&lt;/a&gt; version is somewhat more complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing the eigenvalues of the result tridiagonal matrix can be done in pseudo-quadratic time using the most cutting edge algorithms (&lt;a href=&quot;http://www.cs.utexas.edu/users/inderjit/public_papers/DesignMRRR_toms06.pdf&quot; rel=&quot;nofollow&quot;&gt;MRRR&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Divide-and-conquer_eigenvalue_algorithm&quot; rel=&quot;nofollow&quot;&gt;Divide and Conquer&lt;/a&gt;), or using the traditional &lt;a href=&quot;https://github.com/victorliu/RNP2/blob/master/RNP/LA/Tridiagonal.hpp#L465&quot; rel=&quot;nofollow&quot;&gt;QR&lt;/a&gt; or &lt;a href=&quot;https://github.com/victorliu/RNP2/blob/master/RNP/LA/Tridiagonal.hpp#L348&quot; rel=&quot;nofollow&quot;&gt;QL&lt;/a&gt; iteration. The tricky part is that you have to not only do the iterations (which is &quot;trivial&quot;), but you also have to deal with deflation and various other optimizations that can easily speed things up by an order of magnitude or more. I've only linked to (equivalent) Lapack code for the traditional stuff, the code for MRRR is incredibly subtle and &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn163.pdf&quot; rel=&quot;nofollow&quot;&gt;can often fail&lt;/a&gt;, and the step in Divide and Conquer for solving the secular equation to glue the divisions together is also incredibly subtle. Coding these things up is highly nontrivial even for ordinary CPU execution; I can't imagine how hard it would be on a GPU.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-01-20T07:19:28.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10581" PostTypeId="1" AcceptedAnswerId="10585" CreationDate="2014-01-20T16:26:55.313" Score="3" ViewCount="694" Body="&lt;p&gt;I'm trying to solve the Convection-Diffusion-Reaction (CDR) equation on a rectangular domain, using cylindrical coordinates and Finite Difference Methods (FDM) (this approximates a flow reactor).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I want to solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D ( \frac{\partial^2 u}{\partial r^2} + \frac{1}{r}\frac{\partial u}{\partial r} ) - v(r) \frac{\partial u}{\partial z} = S(r)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, r and z are the coordinates in the respective directions, u is the property I want to solve for, v is the &lt;strong&gt;laminar velocity profile&lt;/strong&gt;, S is a given source and D is the diffusion coefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Boundary conditions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial c}{\partial r} = 0, $ at $r=0$ and at $r=R$ (where R is the length of the rectangle in the r direction). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach is to use the Method of Lines, i.e. to solve the system in the r direction for each step in the z direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When discretizing the above equation, I run into the practical problem that I divide by the velocity v, i.e. my discretized version of the above equation reads:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{du}{dz} = \frac{D}{v_i}(\frac{u_{i+1} - 2u_i + u{i-1}}{2h}+\frac{1}{r_i}\frac{u_{i+1}-u_{i-1}}{2h}) - S_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with this is that $v_i=0$ at one edge of the computational domain, i.e. where $r=R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have approximated my Boundary Conditions by using central differences. This means that I divide entries in my solution matrix by 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking of fixing this problem with using either unsymmetric difference approximations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just wanna ask if that is a good idea or if there is a better fix to this problem.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="3748" LastEditDate="2014-01-31T08:22:48.367" LastActivityDate="2014-01-31T18:09:20.227" Title="Problem in Discretizing Convection-Diffusion-Reaction equation" Tags="|fluid-dynamics|finite-difference|boundary-conditions|cdr-equation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10582" PostTypeId="1" CreationDate="2014-01-20T16:37:25.307" Score="0" ViewCount="520" Body="&lt;p&gt;I'm trying to find some theory to help me explicitly express the inverse of a matrix (or a close approximation of the inverse).  My matrix has the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;invertible&lt;/li&gt;&#xA;&lt;li&gt;positive definite&lt;/li&gt;&#xA;&lt;li&gt;symmetric&lt;/li&gt;&#xA;&lt;li&gt;all diagonal entries $&amp;gt; 0$&lt;/li&gt;&#xA;&lt;li&gt;all off diagonal entries $\le 0$&lt;/li&gt;&#xA;&lt;li&gt;sum across any row or column is nonnegative and further for each row, $i$, the diagonal entry, $d_{ii} \ge \sum_{j\ne i}|d_{ij}|$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;6) tells me that all eigenvalues are $\ge 0$ (Gershgorin theorem) but since I know the matrix is invertible, the eigenvalues must be positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, does anyone know if there's a way to approximate the inverse?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks&lt;/p&gt;&#xA;" OwnerUserId="7122" LastEditorUserId="7122" LastEditDate="2014-01-20T16:49:29.573" LastActivityDate="2018-12-22T13:44:54.033" Title="closed form approximation of matrix inverse with special properties" Tags="|matrix|approximation|inverse|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="10583" PostTypeId="1" AcceptedAnswerId="10588" CreationDate="2014-01-20T19:09:30.333" Score="3" ViewCount="327" Body="&lt;p&gt;Please help me with any/all of the cases below. In the following cases, the named method and basis set are not suitable for the chemical systems. Why aren't they? Could you suggest a suitable method/basis set?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(i)&lt;/em&gt;&lt;/strong&gt;. MP2/6-31G(d) for the calculation of reaction energy of the reaction below (H2C: should be calculated in the triplet state):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Bn5V.png&quot; alt=&quot;1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;carbene H2C: has two singlet configurations that contribute to the ground state energy we need a multireference method to recover the single excitations. MP2 is not good for this open-shell systems since they have large spin contamination and thus large error. The basis set also needs to include diffuse functions in the electron correlation methods.&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(ii)&lt;/em&gt;&lt;/strong&gt;. CISD/cc-pVTZ for the calculation of the reaction energy of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/IhPxs.png&quot; alt=&quot;2&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;we have bond elongation in the Reactant to the point of breackage. CISD is not size-consistent. It recovers less electron correlation energy for the larger reactant as for the products since reactant is larger and thus cannot be used. The basis set however is fine. I don't know what method can work best for this system but would go arbitrarily with CCSD(T)&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(iii)&lt;/em&gt;&lt;/strong&gt;. BLYP/6-31G(d) for geometry optimization and calculation of binding energy of the neon dimer, Ne...Ne.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;Most important interactions in Ne dimer are London forces of van der Waals. BLYP is a hybrid DFT method that includes exact exchange term in exchange-correlation functional. However, the method is local (in LDA, external potential is the uniform electron gas and in GGA its gradients included) but vdW interaction requires fully non-local functionals. The basis set is as well too small and should included diffuse functions. I don't know what method to recommend.&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(iv)&lt;/em&gt;&lt;/strong&gt;. Mp2/6-311++G(2df,p) for the singlet-triplet energy splitting of para-benzyne. Note that for both singlet and triplet states unrestricted MP2 is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/YjjIz.png&quot; alt=&quot;3&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;The para related electrons have low correlation and this means they have small singlet-triplet separations of the frontier orbitals. MPn energy terms are related to the inverse of this separation and thus for a very small s-t separation this term is too large for MP2 to handle. The basis set is probably fine. I think Coupled Cluster methods or DFT would work here&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(v)&lt;/em&gt;&lt;/strong&gt;. CCSD/3-21G for geometry optimization of the DNA base adenine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2GyvB.png&quot; alt=&quot;4&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;The basis set is too small for this molecule and a correlated method (no diffuse or polarization functions are used). Furthermore, CCSD is very costly and not good for this big molecule. Maybe also conjugated pi bonds is of essence. Suggestion?&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(vi)&lt;/em&gt;&lt;/strong&gt;. CISD/6-311G(d,p) to compare the CC bond lengths of oligoenes (2&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/cWnz8.png&quot; alt=&quot;5&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;CISD is not size-extensive so as n increases the calculations become prohibitive.&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(vii)&lt;/em&gt;&lt;/strong&gt;. MP3/6-311G(d,p) for the electron affinity of F2 i.e. the process:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/A0HIX.png&quot; alt=&quot;6&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;MP3 is not good when we have an open-shell species or a system with electron congestion. Convergence will be very slow and not guaranteed at all. We assume in MPn that the zeroeth order wavefunction is reasonable approximation and the add the small perturbation contribution. But if HF describes this molecule very poorly then MPn won't improve it dramatically.&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(viii)&lt;/em&gt;&lt;/strong&gt;. LSDA/6-31G(d) for the relative energies of the various conformations of n-butane obtained through rotation about the central C-C bond.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/VKZNN.png&quot; alt=&quot;7&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;LSDA is equal to LDA for closed shell systems. Thus eternal potential is the uniformly distributed positive charge and can be used only if the &quot;constant&quot; electron density is different for each point in space. But here electron density due to bond twisting is not spatially uniform. I think GGA or B3LYP should be used. Furthermore I think the basis set is too small and should include polarization and diffusion (I don't know why, just saying)&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;What method/basis set would you suggest for the following:&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(ix)&lt;/em&gt;&lt;/strong&gt;. The activation energy of the Diels-Alder reaction between cyclopentadiene and acrylonitrile (reaction does not need static electron correlation treatment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xIKCR.png&quot; alt=&quot;8&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;pure LDA or GGA overestimate bond length so I suggest hybrid B3LYP with small basis set.&lt;/em&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(x)&lt;/em&gt;&lt;/strong&gt;. The interaction energy between two Ar atoms in the Ar...Ar van der Waals complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;MY SUGGESTED ANSWER:&lt;/strong&gt; &lt;em&gt;CCSD with medium basis set. Since vdW is repulsion of electron clouds at large distances as they interpenetrate and CCSD has traces of higher excitations. A small basis set can cause basis set superposition error (BSSE). Inclusion of higher order virtual orbitals as in CCSD improves dispersion description.&lt;/em&gt; )&lt;/p&gt;&#xA;" OwnerUserId="7126" LastEditorUserId="22284" LastEditDate="2019-01-09T08:03:47.210" LastActivityDate="2019-01-09T08:03:47.210" Title="Suggest methods and basis sets for a variety of systems" Tags="|computational-chemistry|density-functional-theory|hartree-fock|" AnswerCount="1" CommentCount="15" ClosedDate="2014-01-31T07:19:31.020" ContentLicense="CC BY-SA 4.0" />
  <row Id="10584" PostTypeId="1" CreationDate="2014-01-20T20:07:46.487" Score="13" ViewCount="2271" Body="&lt;p&gt;Could anyone recommend a method for the following least-squares problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;find $R \in \mathbb{R}^{3 \times 3}$ that minimizes: $\sum\limits_{i=0}^N (Rx_i - b_i)^2 \rightarrow  \min$, where $R$ is a unitary (rotation) matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could get an approximate solution by minimizing $\sum\limits_{i=0}^N (Ax_i - b_i)^2 \rightarrow  \min$ (arbitrary $A \in \mathbb{R}^{3 \times 3}$), taking matrix $A$ and:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;computing SVD: $A = U \Sigma V^T$, dropping $\Sigma$ and approximating $R \approx U V^T$ &lt;/li&gt;&#xA;&lt;li&gt;computing polar decomposition: $A = U P$, dropping scale-only symmetric (and positive definite in my case) $P$ and approximating $R \approx U$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I could also use QR decomposition, but it wouldn't be isometric (would depend on the choice of the coordinate system).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of a way to do this, at least approximately, but with better approximation than the two methods above?&lt;/p&gt;&#xA;" OwnerUserId="7128" LastEditorUserId="20688" LastEditDate="2018-04-01T04:09:58.387" LastActivityDate="2018-04-01T04:09:58.387" Title="Purely rotational least squares match" Tags="|linear-algebra|least-squares|svd|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10585" PostTypeId="2" ParentId="10581" CreationDate="2014-01-20T23:26:04.410" Score="4" Body="&lt;p&gt;The simplest fix here is to treat the problem as a differential-algebraic equation, which is consistent with many treatments of PDE discretizations. Instead of solving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{du}{dz}(r_{i}) = \frac{D}{v_i}(\frac{u_{i+1} - 2u_i + u{i-1}}{2h}+\frac{1}{r_i}\frac{u_{i+1}-u_{i-1}}{2h}) - \frac{S_i}{v_{i}}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;v_{i}\frac{du}{dz}(r_{i}) = D(\frac{u_{i+1} - 2u_i + u{i-1}}{2h}+\frac{1}{r_i}\frac{u_{i+1}-u_{i-1}}{2h}) - S_i,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be done using standard DAE solvers.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-01-31T18:09:20.227" LastActivityDate="2014-01-31T18:09:20.227" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10588" PostTypeId="2" ParentId="10583" CreationDate="2014-01-21T13:17:33.173" Score="1" Body="&lt;p&gt;Most of your answers look good.  For the geometry optimization of single-reference organic systems, B3LYP/big should be fine.  MPn is bad for open-shell systems in general.  MCSCF is the way to go if one has truly pathological multireference effects.  CCSD(T)/huge is good any time you want to treat dispersion, but you'll find in the literature that (He)2 requires essentially exact calculations ala QMC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry I am not giving a more detailed answer, but since you openly admit this material is related to a class, I am reticent to provide improper assistance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other thing you could do is run these problems through Google Scholar and see what the experts have done in the past.  Even better would be to download and install an open-source quantum chemistry code (my personal preference is NWChem, as it has all the methods mentioned here) and do the experiments yourself.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2014-01-21T13:17:33.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10589" PostTypeId="2" ParentId="10584" CreationDate="2014-01-21T17:50:19.277" Score="10" Body="&lt;p&gt;The problem is called &lt;a href=&quot;http://en.wikipedia.org/wiki/Wahba%27s_problem&quot;&gt;Wahba's problem&lt;/a&gt;, one algorithm for it is called &lt;a href=&quot;http://en.wikipedia.org/wiki/Kabsch_algorithm&quot;&gt;Kabsch algorithm&lt;/a&gt;, and the later more popular is called &lt;a href=&quot;https://ia600605.us.archive.org/4/items/nasa_techdoc_20000034107/20000034107.pdf&quot;&gt;Davenport q method&lt;/a&gt;. It's apparently used and studied in aeronautics to determine a craft orientation. There are &lt;a href=&quot;http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19990104598_1999165453.pdf&quot;&gt;lots of reviews&lt;/a&gt; about the methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beware that the best fit may include reflection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kabsch method computes a 3x3 covariance matrix SVD and drops the $\Sigma$ term (modulo one reflection, which usually is accounted for by negating the last column of $U$ in the SVD). It's very straightforward to generalize to other number of dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Davenport q method is often touted as the first practical algorithm, perhaps someone can comment why. It also constructs a 3x3 covariance matrix, but then parametrizes the rotation matrix as a function of a quaternion, and the problem becomes that of computing the max-eigenvalue eigenvector of a symmetrical 4x4 matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Some of) the most popular numerical implementations are called &lt;a href=&quot;http://arxiv-web3.library.cornell.edu/abs/1309.5679&quot;&gt;QUEST and FOMA&lt;/a&gt;. These methods are usually a variation on the theme of computing the max eigenvalue by writing out and optimizing the characteristic polynomial (a quartic), and either solving it analytically (pretty involved computations, going through Kardano formulae), or with Newton iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://malcolmdshuster.com/Pubp_010_031z_J_JAS1290_cquest_MDS.pdf&quot;&gt;Schuster&lt;/a&gt; also developed and analyzed some iterative algorithm variants.&lt;/p&gt;&#xA;" OwnerUserId="7128" LastActivityDate="2014-01-21T17:50:19.277" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10590" PostTypeId="1" CreationDate="2014-01-21T19:40:57.777" Score="0" ViewCount="400" Body="&lt;p&gt;I know the number of solutions to the equation $$x_1+x_2+x_3+...+x_k=n$$&#xA;is given by $\binom{n+k-1}{k-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm to actually find all the solutions to this equation, without having to brute force?&lt;/p&gt;&#xA;" OwnerUserId="7136" LastEditorUserId="276" LastEditDate="2014-02-04T22:12:24.470" LastActivityDate="2014-02-04T22:12:24.470" Title="Algorithm to find non-negative integer solutions to x_1 + x_2 ...=n" Tags="|linear-algebra|algorithms|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10591" PostTypeId="1" AcceptedAnswerId="10595" CreationDate="2014-01-21T19:58:12.207" Score="0" ViewCount="3505" Body="&lt;p&gt;I have an expression of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ACA^{'}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C$ is an invertible, symmetric and positive definite matrix.  I'm trying to figure out if the expression above is invertible.  The $C$ matrix is $n \times n$ and the $A$ matrix is $k \times n$.  Every row of A has exactly one entry =1 and one entry =-1, all other entries are 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help, especially something pointing to a theorem, would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="7138" LastEditorUserId="6706" LastEditDate="2014-01-22T07:56:07.270" LastActivityDate="2014-01-22T09:40:43.840" Title="inverse of a quadratic form" Tags="|matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10592" PostTypeId="2" ParentId="10562" CreationDate="2014-01-21T20:19:40.800" Score="6" Body="&lt;p&gt;There are a couple of options available if you want an approximate rank-k factorizations.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Strongly rank-revealing QR factorizations&lt;/li&gt;&#xA;&lt;li&gt;Interpolative decomposition (ID) and other randomized techniques. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, they provide a factorization of the form &#xA;\begin{equation}\| A - MN^T\| \leq \text{factor}\times \sigma_{k+1}(A) := \epsilon  \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approximate factorization of the above form can be converted into a standard decomposition like QR or SVD using standard techniques. A good review is available in the paper by Halko, Martinsson and Tropp &quot;Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of software an interface to ID algorithms is available in scipy (scipy.linalg.interpolative) &lt;a href=&quot;http://docs.scipy.org/doc/scipy-dev/reference/linalg.interpolative.html&quot; rel=&quot;noreferrer&quot;&gt;http://docs.scipy.org/doc/scipy-dev/reference/linalg.interpolative.html&lt;/a&gt; that allows you the user to specify $\epsilon$.&lt;/p&gt;&#xA;" OwnerUserId="4558" LastActivityDate="2014-01-21T20:19:40.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10593" PostTypeId="2" ParentId="10590" CreationDate="2014-01-22T04:40:44.503" Score="3" Body="&lt;p&gt;Algorithms for listing all solutions are given in the textbook &quot;Constructive Combinatorics&quot; by Dennis Stanton and Dennis White.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-22T04:40:44.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10595" PostTypeId="2" ParentId="10591" CreationDate="2014-01-22T09:40:43.840" Score="1" Body="&lt;p&gt;If $C\in\mathbb{R}^{n\times n}$ is (symmetric and) positive definite and $A\in\mathbb{R}^{k\times n}$ with $k\leq n$, then $ACA^T\in\mathbb{R}^{k\times k}$ is invertible if and only if $A$ has full rank. (You can think of $ACA^T$ as the projection of $C$ onto the subspace spanned by the rows of $A$, so it makes sense to expect them to be linearly independent.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see this, let $x\in\mathbb{R}^k\setminus\{0\}$ be arbitrary. If $A$ has full (row) rank, then $A^T$ has full (column) rank as well and thus is injective. Hence, $y:=A^Tx \in\mathbb{R}^n\setminus\{0\}$, and the positive definiteness of $C$ yields&#xA;$$&#xA;x^T(ACA^T)x = (A^Tx)^TC(A^Tx) = y^TCy &amp;gt; 0,&#xA;$$&#xA;i.e., $ACA^T$ is (symmetric and) positive definite and thus invertible.&#xA;Conversely, if $A$ does not have full rank, $A^T$ is not injective and there exists a vector $x\in\mathbb{R}^k\setminus\{0\}$ with $A^Tx = 0$ and hence $ACA^Tx = 0$. It follows that $ACA^T$ is not injective and thus not invertible.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2014-01-22T09:40:43.840" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10596" PostTypeId="1" AcceptedAnswerId="10613" CreationDate="2014-01-22T13:44:25.177" Score="4" ViewCount="251" Body="&lt;p&gt;I have a nasty non-linear partial differential equation. I wonder if there exists a method that would allow me to examine what numerical errors (like numerical diffusion or dispersion) are introduced by different approximation schemes. Name of the method or a link to a paper would be great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Equation&#xA;$$&#xA;\frac{\partial f}{\partial t}+\frac{\partial}{\partial x} \left(\sqrt{c-\frac{\partial f}{\partial x}} \right)=0&#xA;$$&#xA;$c$ is a constant.&lt;/p&gt;&#xA;" OwnerUserId="4334" LastEditorUserId="4334" LastEditDate="2014-01-22T19:28:55.353" LastActivityDate="2014-01-25T20:42:39.000" Title="Is there a method to examine numerical diffusion for non-linear PDE?" Tags="|numerical-analysis|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10597" PostTypeId="1" CreationDate="2014-01-22T14:24:25.957" Score="1" ViewCount="1187" Body="&lt;p&gt;I would like to ask you for your help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 2D box. I know how to do an 2D cut line and compute an average value of someting over this line. (in Results -&gt; Data Sets -&gt; Cut Line 2D and then Results-&gt;Derived Values-&gt;Line Average and choose Cut Line 2D 1). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would need to know this average value for many of such parallel Cut Lines. I need to compute a curve of average value of something in y for every x. Have you any idea how to get it? I have tried to produce in Cut Line 2D the Additional parallel lines and then do average on it but the result was one number. I wanted a number for every cut line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do it? I would like to get an average value of pressure (in y) as a function of coordinate x.&lt;/p&gt;&#xA;" OwnerUserId="7145" LastActivityDate="2014-01-22T14:24:25.957" Title="COMSOL - 2D box - get for every x an average value over y (line average apply for more lines)" Tags="|comsol|" AnswerCount="0" CommentCount="3" ClosedDate="2014-01-27T16:20:26.793" ContentLicense="CC BY-SA 3.0" />
  <row Id="10598" PostTypeId="2" ParentId="10590" CreationDate="2014-01-22T15:37:54.000" Score="1" Body="&lt;p&gt;Perhaps there's some misunderstanding about the nature of a &quot;brute force&quot; enumeration of the solutions.  Guesswork, at any rate, can be essentially eliminated in navigating a search tree consisting of the solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An efficient production of the solutions is implemented easily enough by nesting $k$ loops, with indexes $i_1,\ldots,i_k$ each going from zero to a &quot;running count&quot; of $n$ minus the sum of the indexes assigned outer to any given level of nesting.  This can be parameterized as to the levels of nesting by making $k$ a formal argument in a recursively called routine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solutions are called &lt;em&gt;ordered&lt;/em&gt; partitions (or &lt;a href=&quot;http://en.wikipedia.org/wiki/Composition_%28number_theory%29&quot; rel=&quot;nofollow&quot;&gt;(weak) compositions&lt;/a&gt;) of $n$, and the count of these as given in the Question is easily established by the &lt;a href=&quot;http://en.wikipedia.org/wiki/Stars_and_bars_%28combinatorics%29&quot; rel=&quot;nofollow&quot;&gt;Stars and Bars argument&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the order of the summands is not important, or may otherwise be accounted for in a calling algorithm, some efficiency may be gained by generating the solutions with summands restricted to an ascending (equiv. descending) arrangement, which are known as the (unordered) &lt;a href=&quot;http://en.wikipedia.org/wiki/Partition_%28number_theory%29&quot; rel=&quot;nofollow&quot;&gt;partitions&lt;/a&gt; of $n$ into at most $k$ summands.  [Strictly speaking a partition is required to be a sum of positive integers, but the restriction to at most $k$ summands allows an easy bookkeeping convention by taking any &quot;missing&quot; summands to be zero.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nested loops or recursive function implementation for &lt;em&gt;weak compositions&lt;/em&gt; (ordered partitions) is easily adapted to generate (unordered) partitions by simply restricting the indexes to an upper bound limited by the next-higher level index (as well as by the remaining portion of $n$ to be allocated) and a lower bound that permits that remaining portion of $n$ to be attained by the number of summands left to fill.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2014-01-22T15:37:54.000" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10599" PostTypeId="1" AcceptedAnswerId="10601" CreationDate="2014-01-22T16:53:21.183" Score="1" ViewCount="842" Body="&lt;p&gt;I have a quadratic programming problem with constraints of the general form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimize w.r.t. x:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(x) = (1/2) x^T * Q * x + c^T * x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;subject to one or more constraints of the following form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A * x &amp;lt;= b (inequality constraint)&#xA;&#xA;E * x = d (equality constraint)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My problem is that I want to have an inequality constraint like x&amp;lt;=u where x is the vector being minimized and u is a vector(!) which is constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also considered something like norm(x)&amp;lt;=norm(u) as a better constraint, but I can't express it/ implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried with &lt;code&gt;JOptimizer&lt;/code&gt; and &lt;code&gt;ojAlgo&lt;/code&gt; but it does only work when u is not a vector but a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how can I achieve an inequality constraint &quot;x&amp;lt;=u&quot; or &quot;norm(x)&amp;lt;=norm(u)&quot; where x and u are vectors?&lt;/p&gt;&#xA;" OwnerUserId="7146" LastActivityDate="2014-01-22T19:17:10.537" Title="Quadratic Programming bound constraints" Tags="|quadratic-programming|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10600" PostTypeId="1" CreationDate="2014-01-22T17:15:50.143" Score="1" ViewCount="130" Body="&lt;p&gt;I'm looking for a simple method to find a linear transform that minimizes&#xA;    $$ \text{argmin}_T F(T): T \in \mathbb{R}^{m \times n}&#xA;,\ T \ge 0&#xA;,\ T \mathbb{1} = c \mathbb{1}&#xA;,\ \mathbb{1}^T T = \mathbb{1} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. over $m \times n$ matrices $\ge 0$ with row sums all the same and column sums all 1.&lt;br&gt;&#xA;(If there's a standard name for this simplex or this problem, please edit.)&lt;br&gt;&#xA;$F(T)$ is min${_x}\ f( T x )$ with $f()$ convex but slow; the gradient at the current minimum should be available.&lt;br&gt;&#xA;$m$ ~ 20 and $n$ ~ 100, so the method should be $O( m + n )$, not $O( m\,n )$ .&lt;br&gt;&#xA;Perhaps line searches that move / clip $T$ along rows / columns ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The problem comes from trying to optimize a filter bank a.k.a. overlapping basis;&#xA;another application area with many heuristics is feature reduction&#xA;in machine learning.)&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2014-01-23T08:51:46.803" LastActivityDate="2014-01-24T14:40:01.083" Title="Optimal linear transform in a simplex" Tags="|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10601" PostTypeId="2" ParentId="10599" CreationDate="2014-01-22T19:17:10.537" Score="2" Body="&lt;p&gt;x&amp;lt;=u is exactly in the form A*x&amp;lt;=b, trivially from A=I and b=u. Most solvers allow you to specify bounds explicitly though (i.e, you specify A, b, E, d, and lower bounds l and upper bounds u)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bound norm(x) &amp;lt;= u (Euclidean norm) cannot be represented using linear constraints. It leads to a so called second-order cone constraint. Changing the norm to the 1-norm (sum of absolute values) or the infinity-norm (largest absolute value) leads to a model which can be represented using linear constraints, and thus allow you to use a simple QP solver.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2014-01-22T19:17:10.537" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10603" PostTypeId="1" CreationDate="2014-01-23T12:01:08.327" Score="3" ViewCount="793" Body="&lt;p&gt;I have a vtk file which has a bunch of points, I would like to delete a few points because, those distort my geometry (basic geometry clean up), is there a way I can do it in paraview? I just want to delete those points , that is it. I haven't found a way to do it.&lt;/p&gt;&#xA;" OwnerUserId="6878" LastActivityDate="2014-01-23T18:15:14.937" Title="how to do geometry clean up in paraview?" Tags="|computational-geometry|" AnswerCount="1" CommentCount="1" ClosedDate="2014-01-28T14:39:17.790" ContentLicense="CC BY-SA 3.0" />
  <row Id="10604" PostTypeId="1" CreationDate="2014-01-23T16:41:53.060" Score="2" ViewCount="35" Body="&lt;p&gt;Is there a way to approximate the convolution of Beta variables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I am trying to find an approximation to $g(x_0)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g(x_0) = \int \delta(x_0-\sum_{i=1}^{n} a_i x_i) \prod_{i=1}^{n} f(\alpha_i,\beta_i;x_i)\mathrm{d}x_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\delta(x)$ is the Dirac delta function and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(\alpha_i,\beta_i;x_i)=\frac{x_i^{\alpha_i - 1}(1-x_i)^{\beta_i - 1}}{\mathrm{B}(\alpha_i,\beta_i)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all $0&amp;lt;x&amp;lt;1$, and $f(\alpha_i,\beta_i;x_i)=0$ otherwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, it is known that the convolution of Gamma functions can be approximated by another Gamma function, whose mean and variance is the sum of the mean and variances of the convoluted Gamma distributions (1). Is there a similar result for the convolution of Beta distributed variables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Stewart, Trevor, et al. &quot;A simple approximation to the convolution of gamma distributions.&quot; (2007).&lt;/p&gt;&#xA;" OwnerUserId="988" LastActivityDate="2014-01-23T16:41:53.060" Title="Approximate convolution of independent Beta variates?" Tags="|probability|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10605" PostTypeId="2" ParentId="10603" CreationDate="2014-01-23T18:15:14.937" Score="2" Body="&lt;p&gt;If you have some way to distinguish between the &quot;good&quot; points and the &quot;bad&quot; points, you can use the Threshold filter to filter on point data to get rid of the undesired points. If you don't have an appropriate point data array to do this on, you can always generate one with either the Calculator filter or the Python Programmable filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to move the &quot;bad&quot; points, options include the Smooth, Warp by Scalar and/or Warp by Vector filters.&lt;/p&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2014-01-23T18:15:14.937" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10606" PostTypeId="2" ParentId="10484" CreationDate="2014-01-23T20:25:33.390" Score="4" Body="&lt;p&gt;For matrices of that size, I'm not sure if you want to use ScaLAPACK at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you've got the ScaLAPACK code already, it shouldn't be hard to implement your own logic to drop into LAPACK instead.  At the very least, doing that will allow you to perform the experiments required to answer your own question.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2014-02-22T16:18:34.127" LastActivityDate="2014-02-22T16:18:34.127" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10609" PostTypeId="1" CreationDate="2014-01-24T08:53:38.487" Score="1" ViewCount="557" Body="&lt;p&gt;I'm solving a nonlinear constrained optimization with constraint of following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A}^T\mathbf{A}-\mathbf{I}=\mathbf{0}, \mathbf{B}^T\mathbf{B}-\mathbf{I}=\mathbf{0}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{A}$ and $\mathbf{B}$ are matrices that I'm solving for. Usually the initial guess of $\mathbf{A}$ and $\mathbf{B}$ are diagonal matrices with entries of 1 or -1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I flatted the entries of $\mathbf{A}$ and $\mathbf{B}$ in column order into a vector as the variable $\mathbf{x}$ to solve for. I tried NLopt and Opt++ to solve my problem, both of them failed. While debugging I found that the gradient of constraint is a singular matrix. $\mathbf{A}$ and $\mathbf{B}$ are not necessarilly square matrices, thus neither is gradient of constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is there any algorithm to solve this nonlinear constrained optimization, which does not require gradient of the constraint?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;=======Edit=========&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textrm{off}(\mathbf{A^T\Lambda_XA})+\textrm{off}(\mathbf{B^T\Lambda_YB})+\mu\left\|\mathbf{F^T\Phi{}A}-\mathbf{G^T\Psi{}B}\right\|_{\mathbf{F}}^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where off() means sum of the squared off-diagonal elements, $\Lambda_X$ and $\Lambda_Y$ are diagonal matrices, $\mathbf{F}$ and $\mathbf{G}$, $\mathbf{\Phi}$ and $\mathbf{\Psi}$ are all known matrices. &lt;/p&gt;&#xA;" OwnerUserId="1976" LastEditorUserId="1976" LastEditDate="2014-01-24T15:01:52.540" LastActivityDate="2014-01-24T15:01:52.540" Title="How to solve nonlinear optimization with constraints that have singular jacobian" Tags="|nonlinear-programming|constrained-optimization|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10610" PostTypeId="1" CreationDate="2014-01-24T10:07:32.017" Score="8" ViewCount="3459" Body="&lt;p&gt;I want to use Quasi Monte Carlo to try and improve the convergence of a simulation I am running.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The random numbers are simply to produce the observation errors for a standard linear regression model, which is then estimated using a number of different regression techniques. This is done repeatedly to estimate the mean square error of each model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm fairly new to Quasi Monte Carlo, but is is likely to help in this situation I am just using it to produce 10k random numbers. It seems that generally I can expect quicker convergence of the order of $1/n$ rather than $n^{-1/2}$: &lt;a href=&quot;http://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method&quot; rel=&quot;noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the above article also states that the QMC numbers are not truly random, so I wonder what the implications might be for any statistical tests I might want to run on the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) What are the pros and cons of MC vs. QMC. (Would you always want to use QMC if it's available?) 2.) What tests can I use to ascertain which is best for my application? (Seems any test that depends on the numbers being truly random will fail?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that this can be done in Matlab using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;q = qrandstream('halton',NSteps,'Skip',1e3,'Leap',1e2); &#xA;RandMat = qrand(q,NRepl); &#xA;z_RandMat = norminv(RandMat,0,1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is taken from &lt;a href=&quot;http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=1519239&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other low discrepancy numbers, such as Sobol sequence also available in Matlab, and again I'd like to know what tests I can use to ascertain which is best for my situation.&lt;/p&gt;&#xA;" OwnerUserId="7161" LastEditorUserId="4474" LastEditDate="2014-01-24T17:08:59.717" LastActivityDate="2014-02-25T22:47:20.740" Title="Quasi Monte Carlo in Matlab" Tags="|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10612" PostTypeId="2" ParentId="10600" CreationDate="2014-01-24T14:40:01.083" Score="1" Body="&lt;p&gt;The problem appears relatively simple: You have a few linear equality constraints and a few linear inequality constraints, together with a convex objective function. Any optimizer should not have great difficulty with this, even the one in Matlab. If you're looking for specific ones, I believe that Lancelot has ways that specifically deal with constraints like yours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, it would help to know what $f(T)$ is.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-24T14:40:01.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10613" PostTypeId="2" ParentId="10596" CreationDate="2014-01-24T14:46:13.310" Score="5" Body="&lt;p&gt;The usual way of evaluating the accuracy of a solution scheme is to compare a numerical solution with an analytically known one. If you can't find a solution to the exact problem you have here (as is likely, given that it's nonlinear), you can always construct one using the Method of Manufactured Solutions. There are many descriptions of this method, one of which is &lt;a href=&quot;http://code.google.com/p/dealii/wiki/FrequentlyAskedQuestions#How_do_I_know_whether_my_finite_element_solution_is_correct?_%28Or&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-24T14:46:13.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10614" PostTypeId="1" AcceptedAnswerId="10615" CreationDate="2014-01-24T17:31:28.107" Score="2" ViewCount="177" Body="&lt;p&gt;There is PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u(r,\varphi,\psi,t)}{\partial t}=\operatorname{div}A(r,\varphi,\psi)\nabla u +f(r,\varphi,\psi,t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We solve numerically IBVP for the ball $B_{1}(0)\subset \mathbb{R}^3$ using the numerical scheme of this PDE. Numerical scheme is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u^{n+1}_{i,j,k}-u^{n}_{i,j,k}}{\Delta t}=\frac{\oint_{\partial V_{i,j,k}}(A_{i,j,k}\nabla u^{n}_{i,j,k},\overline{n})ds}{V_{i,j,k}}+\overline{f}_{i,j,k}^{n}, $$&#xA;where $V_{i,j,k}$ is a cell of grid with center $(i,j,k)$ and vertexes coordinates $(i\pm 1/2, j\pm 1/2, k\pm 1/2)$; $\overline{f}_{i,j,k}^{n}$ is a mean value of $f$ for $V_{i,j,k}$. This scheme is equivalent to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u^{n+1}_{i,j,k}=u^{n}_{i,j,k}+\frac{\Delta t}{V_{i,j,k}}\oint_{\partial V_{i,j,k}}(A_{i,j,k}\nabla u^{n}_{i,j,k},\overline{n})ds+\frac{\Delta t}{V_{i,j,k}}\int\limits_{V_{i,j,k}}f(r,\varphi,\psi,t)dr d\varphi d\psi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) is this scheme conditionally stable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) To integrate over $\partial V_{i,j,k} $ we must know values of $u_{i\pm 1/2,j,k},u_{i,j\pm 1/2,k},u_{i,j,k\pm 1/2}$ on every time step. What are the methods to find this values on every time step?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) How is this scheme called and where can I find more information about it?&lt;/p&gt;&#xA;" OwnerUserId="4676" LastActivityDate="2014-01-25T20:05:33.703" Title="One finite difference scheme" Tags="|pde|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10615" PostTypeId="2" ParentId="10614" CreationDate="2014-01-24T19:13:14.857" Score="4" Body="&lt;p&gt;The scheme you are showing is a fairly standard finite volume formulation coupled with explicit euler time integration.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Finite Volume Derivation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note: I noticed after a bit of writing that I had started to ramble but didn't want to scrap what I had.  If you don't want to see the equation in the derived skip to the 2nd bold header)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will step through the discretization of the unsteady heat equation with source (note the similarity to your equation):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u(t,\vec{r})}{\partial t} = \nabla \cdot (k(\vec{r}) \nabla u(t,\vec{r})) + S(\vec{r})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First you discretize in space.  Your unknowns are defined as the volumetric average of each cell (cells indexed with $i$).  Integrating over each cell you get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\iiint_{V_i}\frac{\partial u_i(t,\vec{r})}{\partial t}dV = \iiint_{V_i} \nabla \cdot (k(\vec{r}) \nabla u_i(t,\vec{r})) + S(\vec{r}) dV$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Break the r.h.s. in two:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\iiint_{V_i}\frac{\partial u_i(t,\vec{r})}{\partial t}dV = \iiint_{V_i} \nabla \cdot (k(\vec{r}) \nabla u_i(t,\vec{r}))dV + \iiint_{V_i}S(\vec{r}) dV$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The unsteady term, in the simplest form, can be estimated by taking the volume of the cell times the time derivative at the center point ($\vec{r}_i$).  Then discretize using Euler:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${V_i}\frac{u_i^{n+1} - u_i^n}{\Delta t} = \iiint_{V_i} \nabla \cdot (k(\vec{r}) \nabla u_i(t,\vec{r}))dV + \iiint_{V_i}S(\vec{r}) dV$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only step left is to apply the divergence thereom to the first term on the rhs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\iiint_V \nabla \cdot \vec{f} dV = \iint_{\partial V} \nabla \vec{f} \cdot \hat{n}  d \partial V$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which implies&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${V_i}\frac{u_i^{n+1} - u_i^n}{\Delta t} = \iint_{\partial V_i} k(\vec{r}) \nabla u_i(t,\vec{r}) \cdot \hat{n}dA + \iiint_{V_i}S(\vec{r}) dV$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is your discretization (minus some trivial algebra).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Spatial Discretization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to actually attempt to answer your questions.  The method is conditionally stable, with the condition depending on the answer to your second question.  The important thing to note is that you don't need to know $u$ at the boundary but $\nabla u \cdot \hat{n}$ which is the gradient normal to the surface.  If you choose your boundary between cells to be half way between the two points, then you can find &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\iint_{V_i} \nabla u \cdot \hat{n} dV = \Sigma_{j \in N_i} \frac{u_j - u_i}{||\Delta \vec{r}||} A_{i,j}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N_i$ is the set of neighbors for node i, and $A_{i,j}$ is the area of their interface.  So while the flux terms in the equation are written as at the one half nodes, you write that flux as a function of the values at the nodes.  Of course this method can be changed, to increase both order of accuracy and stability of the numerical scheme.  Some of these schemes require defining the boundary between cells differently as well.  Depending on what scheme you choose you will have to look for its stability characteristics.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2014-01-25T20:05:33.703" LastActivityDate="2014-01-25T20:05:33.703" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="10616" PostTypeId="2" ParentId="10596" CreationDate="2014-01-24T22:09:52.417" Score="4" Body="&lt;p&gt;To examine the numerical diffusion you need to do the standard consistency analysis in the space variable, that is, you insert the exact solution into your numerical scheme, expand terms of the form $f(x_i+h)$ into a Taylor series, and estimate the terms that you cut off in your approximation. The terms that are of the form $c_d \frac{\partial ^2 f}{\partial x^2}$ and that do not cancel out, are then responsible for numerical diffusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is straight forward for discretizations of convection terms, but I am not sure what this will give for your square-root example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical dispersion is harder to analyse. It occurs when your discretization 'supports' so-called checkerboard modes, i.e. unwanted decoupling of the variables that become visible as oscillations in the numerical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both issues are well understood for upwind discretization (leading to numerical diffusion) and discretization via central differences (leading to numerical dispersion) in the Finite Differences or Finite Volumes approximation of convection-diffusion equations. Maybe, you can transfer some ideas from there to your particular case.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2014-01-25T20:42:39.000" LastActivityDate="2014-01-25T20:42:39.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10617" PostTypeId="1" CreationDate="2014-01-25T17:31:38.180" Score="2" ViewCount="1127" Body="&lt;p&gt;In FEA simulation of simple beams (3D solid and shell/plate), I have taken a simple geometry:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A long beam in the y direction &lt;/li&gt;&#xA;&lt;li&gt;a concentrated load applied on the plane of symmetry&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a pin or roller joint at some point between the symmetry plane and end of the beam:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                      load&#xA;a       b            c↓&#xA;======================⌇  y-symmetry&#xA;        ▲              &#xA;free    pin&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this system, when we solve with the typical beam equation, we assume that the angle of deflection is small. In that case, we see a slight axial distortion between the pin and free end because the total deflection is very small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when we solve this with 3D FEA elements, I would expect the length of the beam from a-b to be conserved. In my understanding, I expect the the shear and moment on elements beyond the joint to be zero. However, in practice I have seen beams with finite-thickness-shell elements and 1 and 2 element thick solid plate (cannot go thicker because of software limitations) meshes result in extreme axial distortions of this region when the load is large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the mesh distortion here unphysical? What is the cause of this behavior? What are some ways to reduce or eliminate this behavior? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I simply not include this region in a simulation? If so, what are some guidelines or considerations for deciding what regions to expect such behavior in order to eliminate/reduce it?&lt;/p&gt;&#xA;" OwnerUserId="7171" LastActivityDate="2014-04-04T00:32:31.590" Title="What is the meaning of distortion in region outside of stressed geometry using FEA?" Tags="|finite-element|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10618" PostTypeId="2" ParentId="10617" CreationDate="2014-01-25T19:59:52.520" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In this system, when we solve with the typical beam equation,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let's first consider the results from the beam equation-- whether&#xA;you solve it analytically or by FEA. Beam theory says that the&#xA;axial deformation is zero along the beam centerline (neutral axis)-- both to the&#xA;left and the right of the pin. At locations away from the centerline&#xA;(e.g. the outer fibers of the beam), you can easily calculate the&#xA;axial deformation from the first derivative of the transverse deformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you observe, there is no bending moment or shear force to the left of the&#xA;pin. The transverse deformation there is basically a rigid-body rotation due&#xA;to the rotation of the beam cross section at the pin. This deformation is&#xA;something you would certainly see in an experiment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation is slightly more complex in 3D and depends on how you model&#xA;the pin in your FEA model. In the vicinity of the pin, there will be a&#xA;3D stress state; for example, there will be stresses to the left of the pin. &#xA;At locations away from the pin and as you make the beam more slender, the&#xA;solution will agree more closely with beam theory.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-01-25T19:59:52.520" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10619" PostTypeId="2" ParentId="1932" CreationDate="2014-01-25T21:40:48.903" Score="1" Body="&lt;p&gt;BLAS1 functions represent a set of kernels that are bandwidth limited because their compute intensity is low.  In particular, these kernels do O(1) flops per memory access.  This means that on modern hardware, they run at a small fraction of peak and there is essentially nothing you can do about.  The best implementation of BLAS1 will check for alignment and modulo the FPU vector length and perform at bandwidth peak, which is likely to be 5-10% of compute peak.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you write these operations explicitly in the source, a good compiler immediately recognizes them and inlines some optimal implementation that is equivalent to the BLAS1 one noted above.  However, because the compiler knows more about the context, it can avoid certain branches (not that these matter that much) and function call overhead, as well as potentially perform higher-order transformations in the code that would be blocked by a function call to an opaque library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a variety of experiments you can perform to determine what is actually affecting the performance of your code.  They are pretty obvious so I won't list them here.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2014-01-25T21:40:48.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10620" PostTypeId="1" AcceptedAnswerId="10621" CreationDate="2014-01-26T12:20:52.383" Score="5" ViewCount="255" Body="&lt;p&gt;I am implementing a fluid simulator as my numerical methods course project and I have to compute pressure at each simulation step. Basically, that means solving an equation $Ap = b$, where $A$ is a sparse matrix: all its entries are zero except:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Its main diagonal&lt;/li&gt;&#xA;&lt;li&gt;Diagonals before and after main diagonal&lt;/li&gt;&#xA;&lt;li&gt;For some $k$, the $k$th diagonal before and the $k$th diagonal after main&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Somewhat like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix}&#xA;a_{11} &amp;amp; a_{12} &amp;amp; 0 &amp;amp; \dots &amp;amp; a_{1k} &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots &amp;amp; 0 \\&#xA;a_{21} &amp;amp; a_{22} &amp;amp; a_{23} &amp;amp; \dots &amp;amp; 0 &amp;amp; a_{2(k+1)} &amp;amp; 0 &amp;amp; \dots &amp;amp; 0 \\&#xA;0 &amp;amp; a_{32} &amp;amp; a_{33} &amp;amp; \dots &amp;amp; 0 &amp;amp; 0 &amp;amp; a_{3(k+2)} &amp;amp; \dots &amp;amp; 0 \\&#xA;0      &amp;amp; 0 &amp;amp; a_{43} &amp;amp; \dots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots &amp;amp; 0 \\&#xA;\dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots \\&#xA;a_{k1} &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots \\&#xA;0 &amp;amp; a_{(k + 1)2} &amp;amp; 0 &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots \\&#xA;0 &amp;amp; 0 &amp;amp; a_{(k + 2)3} &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots \\&#xA;\dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots &amp;amp; \dots\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots &amp;amp; 0 &amp;amp; 0 &amp;amp; a_{(n-1)(n-2)} &amp;amp; a_{(n-1)(n-1)} &amp;amp; a_{(n - 1) n} \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; a_{n(n-1)} &amp;amp; a_{nn}&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I sumulate the fluid without free surface, the matrix is not changing during the simulation, so I want to invert it before the simulation and then just compute $A^{-1} b$ at each step. However, I believe there are faster algorithms for doing it than Gaussian elimination which could exploit sparsity and the structure of such a matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a commonly used name for such a matrix?&lt;/li&gt;&#xA;&lt;li&gt;Are there any effective algorithms for inverting it? (something similar to tridiagonal matrix algorithm, maybe?)&lt;/li&gt;&#xA;&lt;li&gt;If 2 is positive, are there any Java libraries doing that? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7176" LastEditorUserId="7176" LastEditDate="2014-01-27T14:10:44.303" LastActivityDate="2014-01-27T14:10:44.303" Title="Inverting a pressure matrix for fluid simulation" Tags="|fluid-dynamics|sparse-matrix|matrix|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10621" PostTypeId="2" ParentId="10620" CreationDate="2014-01-26T17:30:24.773" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I suppose you are solving the so called &lt;em&gt;Pressure Poisson Equation&lt;/em&gt; and that your $A$ is a discrete representative of a Laplacian.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Typically, you don't want to have the inverse computed and stored, cf. the comments to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10582/closed-form-approximation-of-matrix-inverse-with-special-properties&quot;&gt;this question&lt;/a&gt;. The main reason is that the inverse will be a dense matrix. I rather suggest you compute a sparse LU factorization of $A$, so that $Ax=LUx=b$ can be solved for $x$ very fast. For the 2D Laplacian the factors $L$ and $U$ are known to be computable with little &lt;em&gt;fill in&lt;/em&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Routines for the sparse LU decomposition are available &lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or as a part of high performant libraries like &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/umfpack/&quot; rel=&quot;nofollow noreferrer&quot;&gt;umfpack&lt;/a&gt;. However, I wasn't able to find a java implementation within 3 minutes of googling.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Finally, as your $A$ is typically positive definite, you can go with the CG algorithm which takes only a few lines of code to implement.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-01-26T17:30:24.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10622" PostTypeId="2" ParentId="10620" CreationDate="2014-01-26T22:24:26.870" Score="7" Body="&lt;p&gt;Your matrix is &lt;a href=&quot;https://scicomp.stackexchange.com/a/880/119&quot;&gt;not &quot;banded&quot;&lt;/a&gt; in the sense of &quot;banded&quot; direct solvers, so don't bother with those.  Multigrid is absolutely the best way to solve these &quot;pressure Poisson&quot; problems.  There are lots of libraries and it's not difficult to implement a simple multigrid algorithm for structured grids.  CG and direct solvers are fundamentally non-scalable, so the cost will increase superlinearly as you increase resolution.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-01-26T22:24:26.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10623" PostTypeId="1" AcceptedAnswerId="10624" CreationDate="2014-01-27T01:39:21.353" Score="2" ViewCount="196" Body="&lt;p&gt;I read the paper called Laplacian Surface Editing, but I'm confused about how does the author solve the formula (4). The paper says that 'Solving this quadratic minimization problem results in a sparse linear system of equations', so I have tried to use the quadprog function in matlab, but when the size of the nodes is growing up, my result is becoming messy. And I read the 'MATLAB 2D demo' code of this paper (&lt;a href=&quot;http://igl.ethz.ch/projects/Laplacian-mesh-processing/Laplacian-mesh-editing/&quot; rel=&quot;nofollow&quot;&gt;http://igl.ethz.ch/projects/Laplacian-mesh-processing/Laplacian-mesh-editing/&lt;/a&gt;), it seems that they solve the question just by solving a linear system instead of using method such like 'interior-point' or 'active-set'. If someone has read this paper, Could you please tell me how on earth they solve the formula(4) ? Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="7180" LastActivityDate="2014-01-27T18:12:12.537" Title="Questions about Laplacian Surface Editing" Tags="|optimization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10624" PostTypeId="2" ParentId="10623" CreationDate="2014-01-27T02:50:20.677" Score="3" Body="&lt;p&gt;I'll state this in terms of an analogous problem in a domain where you are looking for a function $u(x)$ (namely the Laplace equation): If you are looking to minimize the problem&#xA;$$&#xA;  \min_{u \in H^1} \frac 12 \|\nabla u\|^2_{L_2(\Omega)} - \int f\; u \; dx&#xA;$$&#xA;then this is equivalent to solving the partial differential equation&#xA;$$&#xA;  -\Delta u = f.&#xA;$$&#xA;This is because the PDE are the &lt;em&gt;optimality conditions&lt;/em&gt; for the optimization problem and, because they are linear, allows for the direct solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same way, the minimization problem given by equation (4) of the paper leads to a partial differential equation -- here posed on the surface of a body -- as the optimality conditions, and this is what the authors solve.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-27T02:50:20.677" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10626" PostTypeId="1" CreationDate="2014-01-27T16:05:57.537" Score="2" ViewCount="103" Body="&lt;p&gt;I am looking at a few reaction-diffusion equations of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dP}{dt} = D\left(\frac{d^2P}{dr^2} + \frac{2}{r}\frac{dP}{dr}\right) - a(P)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the initial conditions and the boundary value at one end. I also know the function steadily falls and eventually hits zero at the second unknown boundary. To model this behaviour and indirectly find this unknown boundary, I coded a solver using an explicit finite difference scheme, with an added condition in the loop that changed forced any negative values to zero. This gave me the correct result, but due to the CFL condition I've had to use a tiny time step ($\Delta t = 0.001$) when I'm more interested in the system's behaviour over several hours or even days. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was looking into using implicit finite difference methods or CN methods so I may increase the time step, but my (limited) understanding of this implies I'd have to solve a system of equations which would include the unknown boundary, which I will not know exactly. Is it possible to work around this, or will implicit methods fail? If it is possible to work around it, could anyone suggest a good method and how I would implement this? Thanks in advance, I'm quite new to numerical methods and would appreciate any suggestions. &lt;/p&gt;&#xA;" OwnerUserId="7182" LastActivityDate="2014-03-28T17:02:13.610" Title="Implicit Finite difference scheme for a PDE with only one boundary" Tags="|linear-solver|numerical-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10627" PostTypeId="2" ParentId="10626" CreationDate="2014-01-27T16:39:20.840" Score="2" Body="&lt;p&gt;I am not entirely certain that your method works correctly.  The answers may look correct but unless you have an analytic solution to check against you could be introducing errors.  Also, how do you handle the 2nd derivative at the far boundary?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary condition you have is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \lim \limits_{x \to \infty} P = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with a domain of $[x_0, \infty)$.  A more common way to handle this numerically is to truncate your domain to $[x_0, x_1]$ and hold the boundary condition that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ P(x_1) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works fine, except for you don't know if your truncated domain has affected your answer.  So then you resolve it on $[x_0, 2 x_1]$ (assuming $x_1 &amp;gt;&amp;gt; x_0$ and this about doubles your domain).  If your answer in the areas of interest are close, then you know you are done.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you make this change, then switching to an implicit solver would be trivial, since you have eliminated the if condition inside your solver.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-01-27T16:39:20.840" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10628" PostTypeId="1" CreationDate="2014-01-27T18:07:18.850" Score="0" ViewCount="149" Body="&lt;p&gt;Consider the following optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{max}_{p} \quad ||p||^2 \\ &#xA;s.t: x\geq 0\\&#xA;p\in D$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ is a convex set. Is this problem $\mathcal{NP}$-hard?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="917" LastEditDate="2014-01-28T09:52:13.853" LastActivityDate="2014-01-28T09:52:13.853" Title="Nonconvex Optimization" Tags="|optimization|convex-optimization|nonlinear-programming|nonconvex|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10629" PostTypeId="2" ParentId="10623" CreationDate="2014-01-27T18:12:12.537" Score="1" Body="&lt;p&gt;To add to Wolfgang's answer, the PDE that they come up with in the paper arises as the optimality condition for an unconstrained optimization problem. You mention using the &lt;code&gt;quadprog&lt;/code&gt; function in MATLAB and methods like interior point and active set; these are more appropriate for optimization problems with inequality constraints. This would be a problem like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \min\frac{1}{2}\int_\Omega |\nabla u|^2 dx - \int_\Omega u\cdot f dx$, over all $u$: $u \ge 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The addition of the constraint $u \ge 0$ is what makes quadratic programming problems with inequality constraints so much harder than unconstrained QP, which boils down to just solving a linear system. By using the interior point method, you would be applying a method normally used for problems with inequality constraints to an unconstrained problem. Hope this clarifies matters!&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-01-27T18:12:12.537" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10630" PostTypeId="1" AcceptedAnswerId="10631" CreationDate="2014-01-27T18:12:59.743" Score="10" ViewCount="4487" Body="&lt;p&gt;Let $A$ be a real, symmetric, positive definite matrix. It has at least 500 rows, possibly much more. I compute its Cholesky decomposition, which allows me to calculate&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$det(A)$&lt;/li&gt;&#xA;&lt;li&gt;$A^{-1}X$ for some vector/matrix $X$&lt;/li&gt;&#xA;&lt;li&gt;$A^{-1}$ (I am aware of the warnings here)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this operation needs to be performed a lot of times (say, a million), on matrices which are closely related but different from $A$. Formally, $A$ and any other similar matrix $A'$ differ by a full-rank update $E$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A' = A+E \quad \frac{\|E\|}{\|A\|}\ll 1 \quad rank(E)=dim(E)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, I am computing the Cholesky decomposition of every matrix at every step. In order to speed up my calculation, I would like to use the information that two consecutive matrices are closely related. How can I do that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for something $O(N^2)$ instead of the $O(N^3)$ of the Cholesky decomposition. Currently the matrices are dense, but I am also interested in solutions for banded sparse matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; Since this question was general and gave a great general answer, I will mark it as accepted. However, as far as my more specific problem is concerned, see &lt;a href=&quot;https://scicomp.stackexchange.com/q/10638/733&quot;&gt;full rank update to cholesky decomposition for multivariate normal distribution&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="733" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2017-07-19T16:35:08.473" Title="full rank update to cholesky decomposition" Tags="|linear-algebra|linear-solver|iterative-method|banded-matrix|dense-matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10631" PostTypeId="2" ParentId="10630" CreationDate="2014-01-27T19:18:25.467" Score="16" Body="&lt;p&gt;In general, there is no shortcut other than completely re-factoring the matrix. There have been a few similar questions on this SE that cover the topic in more depth than I can:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/503/can-diagonal-plus-fixed-symmetric-linear-systems-be-solved-in-quadratic-time-aft/505#505&quot;&gt;Can diagonal plus fixed symmetric linear systems be solved in quadratic time after precomputation?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/10278/lu-decom-of-psd-matrix-diagonal-matrix/10282#10282&quot;&gt;LU Decom of PSD Matrix + Diagonal Matrix&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/10533/perturbation-of-cholesky-decomposition-for-matrix-inversion/10535#10535&quot;&gt;Perturbation of Cholesky decomposition for matrix inversion&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/5101/diagonal-update-of-a-symmetric-positive-definite-matrix&quot;&gt;Diagonal update of a symmetric positive definite matrix&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/8323/computational-complexity-and-implementation-of-udu-modified-cholesky-rank-1-upda&quot;&gt;Computational complexity and implementation of UDU Modified Cholesky Rank 1 Update&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long story short, you can do it in $O(\textrm{rank}(E)\cdot n^2)$. If $E$ is rank-1 or of some rank asymptotically lower than the dimension of the matrix you're trying to update, then you can do better than $n^3$, otherwise you're out of luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For banded matrices, things are a little nicer; you can do a Cholesky factorization in $O(\textrm{bandwidth}^2\cdot n)$, so provided the bandwidth is appreciably smaller than $n$, the cost of doing a single factorization isn't so punitive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is your implementation serial or parallel? If it's serial, you may want to investigate using a package like &lt;a href=&quot;http://libelemental.org/&quot; rel=&quot;noreferrer&quot;&gt;Elemental&lt;/a&gt; to parallelize the decomposition of each individual matrix. Sometimes, the matrices are too small to get much advantage out of parallelism, but if you have 1000-dimensional dense matrices I suspect that won't be the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You've said that the matrices are related; are you given a whole mess of matrices $\{A_i\}$ right at the start of the program, or do you update them sequentially? If it's the former, you may be able to use MPI to get several processors working on the matrices in parallel.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2017-07-19T16:35:08.473" LastActivityDate="2017-07-19T16:35:08.473" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10632" PostTypeId="2" ParentId="10628" CreationDate="2014-01-27T19:54:41.453" Score="0" Body="&lt;p&gt;Show that an $\mathcal{NP}$-complete problem reduces to your problem in polynomial time. A classic example of this for nonconvex optimization is the &lt;a href=&quot;http://www-personal.umich.edu/~murty/np.pdf&quot; rel=&quot;nofollow&quot;&gt;reduction from subset sum to nonconvex optimization by Murty and Kabadi&lt;/a&gt;. For your specific problem, you will have to formulate a similar type of reduction. You might be able to leverage work done on quadratically-constrained quadratic programs.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-01-27T19:54:41.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10633" PostTypeId="2" ParentId="10628" CreationDate="2014-01-27T20:11:43.800" Score="1" Body="&lt;p&gt;Your original problem statement is vague in that you haven't described how the convex set $D$ would be encoded.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem simplifies immediately to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \| p \|^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p \in D$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We'll show that this problem is NP-Hard by reduction from 0-1 Integer Linear&#xA;Programming feasibility.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 0-1 Integer Linear Programming feasibility problem is a well known NP-Complete problem:  &quot;Given integer matrices $A$ and $b$, is there a 0-1 vector $x$ such that $Ax=b$?&quot;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the change of variables $y_{i}=2x_{i}-1$.  Then our original problem becomes &quot;Is there a vector $y$ with $y_{i}=\pm 1$ for $i=1, 2, \ldots, n$, such&#xA;that $\hat{A}y=\hat{b}$?&quot;  It's easy to see that this transformation produces a new instance of size polynomial in the size of the original problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose that we could solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \| y \|^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\hat{A}y=\hat{b}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ -1 \leq y_{i} \leq 1$, for $i=1, 2, \ldots, n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in polynomial time.  The optimal value is $n$ if and only $y$ has a solution in which all entries are $\pm 1$.  The feasible set for this optimization problem is clearly convex.  A polynomial time algorithm to find the optimal $y$ would give us a polynomial time algorithm for the original 0-1 ILP feasibility problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shows that in general, the problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \| p \|^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p \in D$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is NP-Hard, even if $D$ is restricted to being convex.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-27T20:11:43.800" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10635" PostTypeId="1" CreationDate="2014-01-28T08:26:38.280" Score="0" ViewCount="57" Body="&lt;p&gt;I will be interfacing our fluid dynamics code with an optimizer. I expect the optimization process to take 3-4 days for any decent decrease. But I will want to have a look at the intermediate results for trends and to make sure that things does not blow up, both at the optimizer level and at the fluid dynamics level. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way is to go to individual folders containing results for all the run and check them periodically. Another way it to write a code which does that for me and print a result as single place. Another thing that can be done and is more visually appealing is to generate a html file using a script which make some fancy graphs using data from all the run etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to ask whether solution to this task already exists? And what is the most preferred way of doing this? Also any particular data format which will be helpful?&lt;/p&gt;&#xA;" OwnerUserId="3640" LastActivityDate="2014-01-28T17:31:03.467" Title="Best way of getting updates on intermediate simulation results" Tags="|software|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10638" PostTypeId="1" CreationDate="2014-01-28T10:14:48.573" Score="7" ViewCount="975" Body="&lt;p&gt;This question is a specialization of &lt;a href=&quot;https://scicomp.stackexchange.com/q/10630/733&quot;&gt;full rank update to cholesky decomposition&lt;/a&gt;, to which I hope to get a more positive answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When calculating the minus log of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Multivariate_normal_distribution&quot; rel=&quot;noreferrer&quot;&gt;multivariate normal distribution&lt;/a&gt;, the most difficult part is evaluating&#xA;$$f(\epsilon,\Sigma) \equiv \epsilon^\top \Sigma^{-1} \epsilon + \log \det \Sigma$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\epsilon$ is a vector, and $\Sigma$ is a symmetric positive definite matrix. When calculating its gradient, we are also interested in&#xA;$$ \frac{\partial f}{\partial \epsilon}(\epsilon,\Sigma) \equiv 2\Sigma^{-1}\epsilon&#xA;\qquad \frac{\partial f}{\partial \Sigma}(\epsilon,\Sigma) \equiv -(\Sigma^{-1}\epsilon)(\Sigma^{-1}\epsilon)^\top + \Sigma^{-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this question, $\epsilon$ is constant, and $\Sigma$ is such that&#xA;$$\exists (x_1,\ldots,x_N) \quad \Sigma_{ij} = \sigma^2(x_i,x_j; \theta)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sigma^2$ is twice differentiable along $\theta$, which is a parameter, or a parameter vector, depending on the application. Therefore, an infinitesimal change in $\theta$ results in an infinitesimal change in $\Sigma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In trying to optimize the value of $\theta$, I perform minute changes in its value, possibly following the gradient of $f$, and recompute $f$ at each step to see if the value of $\theta$ got better. &lt;strong&gt;I would like to make this step faster by not Cholesky-decomposing $\Sigma(\theta+d\theta)$ from scratch, but instead use the decomposition of $\Sigma(\theta)$ as a starting point.&lt;/strong&gt; If it makes things easier, I would simply like to update the following quantities&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\epsilon\Sigma^{-1}\epsilon$&lt;/li&gt;&#xA;&lt;li&gt;$\log \det \Sigma$&lt;/li&gt;&#xA;&lt;li&gt;$\Sigma^{-1}\epsilon$&lt;/li&gt;&#xA;&lt;li&gt;$\Sigma^{-1}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here's my personal intuition&lt;/strong&gt; as to why there should be a solution. We focus on the simplest case, updating $g(\theta) \equiv \epsilon^\top\Sigma^{-1}\epsilon$ and with $\theta$ a scalar parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is, therefore, &quot;what is $g(\theta+\delta\theta)$ knowing $g(\theta)$ ?&quot; We know that $g(\theta+d\theta) = g(\theta) + g'(\theta)d\theta$, and in this particular case, we have&#xA;$$g'(\theta) \equiv -(\Sigma^{-1}\epsilon)^\top \frac{\partial\Sigma}{\partial \theta}(\Sigma^{-1}\epsilon) \qquad \text{with} \quad \Sigma \equiv \Sigma(\theta)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the Cholesky decomposition of $\Sigma(\theta)$ is known, the calculation of $g'(\theta)$ is $O(N^2)$ and therefore by simple iteration of this gradient descent, we should be able to reach $\Sigma(\theta+\delta\theta)$ in a small number of iterations, making it $O(kN^2)$ with $k\ll N \equiv \dim \Sigma$. So, what should it be, and how to approach this? &lt;a href=&quot;http://en.wikipedia.org/wiki/Steepest_descent&quot; rel=&quot;noreferrer&quot;&gt;Steepest descent&lt;/a&gt; ? &lt;a href=&quot;http://en.wikipedia.org/wiki/Conjugate_gradients&quot; rel=&quot;noreferrer&quot;&gt;Conjugate Gradients&lt;/a&gt; ? And how do I set $k$?&lt;/p&gt;&#xA;" OwnerUserId="733" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2021-01-15T18:42:30.293" Title="full rank update to cholesky decomposition for multivariate normal distribution" Tags="|linear-algebra|iterative-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10639" PostTypeId="1" CreationDate="2014-01-28T11:58:23.150" Score="1" ViewCount="112" Body="&lt;p&gt;I would like to minimize the following quadratic form:&#xA;$$&#xA;f(\mathbf{\theta}) = (\mathbf{y} - \mathbf{\mu}(\mathbf{\theta}))^T \mathbf{\Sigma}({\mathbf{\theta}})^{-1}&#xA;(\mathbf{y} - \mathbf{\mu}(\mathbf{\theta})),&#xA;$$&#xA;wrt $\mathbf{\theta}$. $\mathbf{y}$ is $d$-dimensional, while $\mathbf{\theta}$ is $p$-dimensional. Generally $d \geq p$ and $\mathbf{\Sigma}_{\mathbf{\theta}}$ is positive definite for any $\mathbf{\theta}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The partial derivatives:&#xA;$$&#xA;\frac{\partial \mathbf{\mu}(\mathbf{\theta})_{i}}{\partial \mathbf{\theta}_j}&#xA;$$&#xA;are relatively easy to estimate. On the other hand the derivatives: &#xA;$$&#xA;\frac{\mathbf{\Sigma}({\mathbf{\theta}})_{ij}}{\partial \mathbf{\theta}_k}&#xA;$$&#xA;are nearly impossible to approximate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm doing at the moment is optimizing only along $\mathbf{\mu}(\mathbf{\theta})$ while treating $\mathbf{\Sigma}({\mathbf{\theta}})$ as if it was fixed. In other words I'm using steepest descent where at each step I move in the direction:&#xA;$$&#xA;\frac{\partial \mathbf{\mu}(\mathbf{\theta})}{\partial \mathbf{\theta}_j}^T\frac{\partial f(\mathbf{\theta})}{\partial \mathbf{\mu}(\mathbf{\theta})} =&#xA;\frac{\partial \mathbf{\mu}(\mathbf{\theta})}{\partial \mathbf{\theta}_j}^T\mathbf{\Sigma}({\mathbf{\theta}})^{-1}(\mathbf{y} - \mathbf{\mu}(\mathbf{\theta})).&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is there any guarantee doing this procedure converges to the minimum of $f(\mathbf{\theta})$? Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Information&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two possible situation:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Finite sample: it doesn't exist a $\mathbf{\theta}$ such that $\mathbf{y} = \mathbf{\mu}(\mathbf{\theta})$.&lt;/li&gt;&#xA;&lt;li&gt;In the limit: there is a unique $\mathbf{\theta_0}$ such that $\mathbf{y} = \mathbf{\mu}(\mathbf{\theta_0})$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2140" LastEditorUserId="2140" LastEditDate="2014-01-28T12:17:03.993" LastActivityDate="2014-01-28T12:17:03.993" Title="Minimizing a quadratic form" Tags="|optimization|convex-optimization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10640" PostTypeId="1" AcceptedAnswerId="10642" CreationDate="2014-01-28T15:08:37.553" Score="6" ViewCount="1084" Body="&lt;p&gt;Given a forward-in-time approximation I have the coupled equations:&#xA;$$&#xA;\frac{T^{(n+1)} - T^{(n)}}{\Delta t} = x T^{(n)} - y h^{(n)} \\&#xA;\frac{h^{(n+1)} - h^{(n)}}{\Delta t} = -z h^{(n)} - \alpha T^{(n)}&#xA;$$&#xA;where $x, y, z$ and $\alpha$ are constants, I see from my simulation that the solution is damping. But how can I use Von Neumann analysis to find the amplication factor $A$?&lt;/p&gt;&#xA;" OwnerUserId="7192" LastEditorUserId="549" LastEditDate="2016-12-08T16:23:37.710" LastActivityDate="2016-12-08T16:23:37.710" Title="Stability analysis of coupled ordinary differential equations" Tags="|stability|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10641" PostTypeId="2" ParentId="10635" CreationDate="2014-01-28T17:31:03.467" Score="2" Body="&lt;p&gt;There are no general solutions to this problem, but there are lots of workflow tools that might help you orchestrate your process. Whether you choose to compute derived quantities or make pictures to establish the progress of your runs is up to you to determine based on what's most helpful to you in your work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might look at:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://kepler-project.org/&quot; rel=&quot;nofollow&quot;&gt;Kepler&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.vistrails.org/index.php/Main_Page&quot; rel=&quot;nofollow&quot;&gt;VisTrails&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.taverna.org.uk/&quot; rel=&quot;nofollow&quot;&gt;Taverna&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;and many more.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-01-28T17:31:03.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10642" PostTypeId="2" ParentId="10640" CreationDate="2014-01-28T18:53:43.237" Score="6" Body="&lt;p&gt;You can rewrite your finite difference method into the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}\left[\begin{matrix} T^{t+1} \\ h^{t+1}\end{matrix} \right]=\left[\begin{matrix}A_{11} &amp;amp; A_{12} \\ A_{21} &amp;amp; A_{22} \end{matrix} \right] \left[\begin{matrix} T^n\\h^n\end{matrix} \right] = \left[\begin{matrix}A_{11} &amp;amp; A_{12} \\ A_{21} &amp;amp; A_{22} \end{matrix} \right]^n\left[\begin{matrix} T^0\\h^0\end{matrix} \right]&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rewritten this way, you will find that the matrix $A$ becomes $$A=I+\Delta tJ,$$ where $I$ is the identity matrix and $J$ is the jacobian of the RHS of your original system.  $A$ can be thought of as the &lt;em&gt;amplification matrix&lt;/em&gt;, whose eigenvalues determine the stability of the numerical system.  To ensure absolute stability, you need to find a region in the complex vector space such that the spectral radius $\rho(A)&amp;lt;1$ (i.e. the dominant eigenvalue is less than one).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, since this is a &lt;em&gt;system of ODE's&lt;/em&gt;, you need to be extra careful.  &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_stability&quot; rel=&quot;nofollow&quot;&gt;Stability&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_methods_for_ordinary_differential_equations#Consistency_and_order&quot; rel=&quot;nofollow&quot;&gt;consistency&lt;/a&gt; may not be enough to ensure an accurate solution.  You'll also need to check for &lt;a href=&quot;http://en.wikipedia.org/wiki/Stiff_equation&quot; rel=&quot;nofollow&quot;&gt;Stiffness&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-01-29T17:26:13.300" LastActivityDate="2014-01-29T17:26:13.300" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10644" PostTypeId="1" AcceptedAnswerId="10648" CreationDate="2014-01-28T21:12:40.527" Score="6" ViewCount="616" Body="&lt;p&gt;I am trying to create a simulation to help visualize how different chemical components flow through a network of pipes with associated valves, pumps, and chemical inputs. In this simulation, the pipes will be represented in 1D and the pump flow rates and valve states will ideally be able to be changed by the user during the simulation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the simplest pipe network might consist of three pipes and two pumps. The images below show what the simulation ideally would be able to do in real time. The arrows represent pumps, the circles represent inputs and outputs, and the lines represent pipes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/BcO2Z.png&quot; alt=&quot;Simple pipe network example&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already created a program that can take a representation of such a pipe network with pumps and valves in a given configuration and determine the flow rate in each pipe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difficulty I am having is in simulating the mass transport of the chemicals through the pipes. So far, I have tried an agent-based approach, where discrete boluses of fluid move through the pipes (one bolus for every v_pipe * dt meters in each pipe), and a simple finite difference scheme. The agent-based approach works in some situations, although doesn't work well when the flow rates of the pipes change. I have had some success with the finite difference method for a single pipe, but have problems with numerical diffusion and getting it to work well with a network of pipes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good simulation method for implementing such a simulation of mass transport in a network of pipes where the flow rates in each pipe can vary? Would something like the finite volume method work? &lt;/p&gt;&#xA;" OwnerUserId="6766" LastActivityDate="2014-01-29T03:59:50.153" Title="Simulating advection over a network of 1D pipes" Tags="|fluid-dynamics|simulation|visualization|advection|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10646" PostTypeId="2" ParentId="10644" CreationDate="2014-01-29T00:26:20.027" Score="1" Body="&lt;p&gt;It depends how realistic you want the simulation to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could have a &lt;a href=&quot;http://en.wikipedia.org/wiki/Cellular_automaton&quot; rel=&quot;nofollow&quot;&gt;cellular automaton&lt;/a&gt; approach (maybe similar to your boluses approach). Where the automata behave as little agents moving in the fluid. Give each cellular automata simple rules like: &lt;em&gt;'at each time step move down this pipe at the pipes velocity'&lt;/em&gt;, &lt;em&gt;'if you reach a node and it is closed stop'&lt;/em&gt;, &lt;em&gt;'if the node is open move along both pipes'&lt;/em&gt;, &lt;em&gt;'if speed increases/decreases lighten/darken colour by the fraction change in speed'&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things get a little tricky at the junctions because you will want to apply rules which preserve mass as best as you can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle each pipe could be it's own finite volume or finite difference simulation and connected together with appropriate boundary conditions. As you you only need advection (or maybe in the future diffusion?) finite volume is a good fit. As you noticed numerical diffusion becomes a problem here. To fix that I would move to a finite volume approach and apply a &lt;a href=&quot;http://en.wikipedia.org/wiki/Flux_limiter&quot; rel=&quot;nofollow&quot;&gt;flux limiter&lt;/a&gt; like the &lt;a href=&quot;http://en.wikipedia.org/wiki/MUSCL_scheme&quot; rel=&quot;nofollow&quot;&gt;MUSCL scheme&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2014-01-29T00:26:20.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10648" PostTypeId="2" ParentId="10644" CreationDate="2014-01-29T03:59:50.153" Score="4" Body="&lt;p&gt;This is actually a standard modeling problem if you consider the medium that flows through the network to be incompressible (e.g., liquids, or gases at low velocity). Then, you formulate everything in terms of fluxes (liters or kg per second) rather than in discrete parcels. The key realization is that the flux that goes into one end of the pipe equals the flux that comes out at the other. You then prescribe the flux at those inlets where you have a pump, and at all interior joints you have mass conservation, i.e., the sum of all fluxes into each node is zero. You will likely have to have one or more open outlets (such as the one on the right in your example) to make sure you don't have an overdetermined system. The variables you have are the fluxes $x_i$ through pipe $i$ for all of the pipes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-29T03:59:50.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10649" PostTypeId="1" AcceptedAnswerId="10652" CreationDate="2014-01-29T05:13:49.080" Score="2" ViewCount="124" Body="&lt;p&gt;I'm numerically inverting an integral transform using a &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10161/numerical-methods-for-inverting-integral-transforms&quot;&gt;method suggested by a scicomp user&lt;/a&gt; from an earlier question. The problem is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to estimate $f(x)$ for a given $F(y)$, both of which are expressed as a linear combination of some set of orthogonal basdis functions $\phi_n$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) = \sum_{i}^{M} c_i \phi_i(x) \quad \quad \quad F(y) = \sum_{j}^{M} b_j \phi_j(y) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coefficients needed to reconstruct $f(x)$ are given by the vector $\mathbf{c}$, which can be obtained by solving the linear linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{b} = \mathbf{A}\mathbf{c} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method works well in some theoretical test cases, but when applying it to real data I find that the solution to this system produces an $f(x)$ which is highly oscillatory, with much more power in the higher order basis functions. Adding small amounts of random noise to test cases which do work also produces incorrect oscillatory solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the following about the true solution for $f(x)$:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$f(x)$ is not oscillatory&lt;/li&gt;&#xA;&lt;li&gt;$f(x)$ has a smooth, decaying form - often similar to a half-Cauchy distribution&lt;/li&gt;&#xA;&lt;li&gt;$f(x)$ is positive everywhere (it's a continuous probability distribution)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I've done some reading around the problem and I think I need to use &lt;em&gt;regularisation&lt;/em&gt; to constrain the solution of the linear system such that it produces the correct $f(x)$, but I've seen there are many types of regularisation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can anyone give me advice on which approaches to regularisation would be appropriate and how to apply them?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your time and help!&lt;/p&gt;&#xA;" OwnerUserId="6700" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-01-29T13:24:18.887" Title="Advice on the regularisation of a linear problem" Tags="|linear-algebra|optimization|linear-solver|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10650" PostTypeId="2" ParentId="10649" CreationDate="2014-01-29T05:38:36.263" Score="4" Body="&lt;p&gt;Because your basis functions are orthogonal polynomials (and assuming that they've been properly normalized), $\| f \|_{2}=\| c \|_{2}$.  Thus you could regularize the solution of the linear system of equations in the 2-norm and effectively regularize $f$ in the 2-norm as well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, simply minimize $\| Ac-b\|_{2}^{2}+ \lambda \| c \|_{2}^{2}$.  This can be set up as a least squares problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min &#xA;\left\|&#xA;\left[ &#xA;\begin{array}{c}&#xA;A \\&#xA;\sqrt{\lambda} I&#xA;\end{array} &#xA;\right]&#xA;b-&#xA;\left[&#xA;\begin{array}{c}&#xA;b \\&#xA;0 \\&#xA;\end{array}&#xA;\right]&#xA;\right\|_{2}^{2}&#xA;$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The normal equations for this damped least squares problem give the solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\hat{c}=(A^{T}A+\lambda I)^{-1}A^{T}b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could easily add a constraint to force $f$ to integrate out to 1, if this isn't already implied by the equations $Ac=b$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important aspect of your problem is that $f$ is inherently nonnegative (it could be 0 at some points, so its not necessarily positive everywhere.)  Unfortunately, the orthogonal basis functions that you're using aren't nonnegative everywhere, and there's no reasonable way to enforce nonnegativity of the solution in terms of the coefficients in the orthogonal polynomial expansion of $f$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus you might want to consider getting rid of the orthogonal polynomials and solving the problem in terms of basis functions that allow you to enforce the nonnegativity of $f$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also consider expressing your problem in terms of the CDF rather than the pdf of your probability distribution.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-01-29T05:38:36.263" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10651" PostTypeId="2" ParentId="8855" CreationDate="2014-01-29T06:27:41.627" Score="4" Body="&lt;p&gt;From the comments, it sounds like your problem is stiff, and using an implicit integrator will help you a lot more than trying to parallelize.  But in case someone comes here looking for information on parallel time integration, you can find a discussion of some simple parallel extrapolation and deferred correction methods in &lt;a href=&quot;http://arxiv.org/abs/1305.6165&quot; rel=&quot;nofollow noreferrer&quot;&gt;this preprint of mine&lt;/a&gt;.  It's not a new topic; you can trace the references backward starting from the bibliography there.  For more sophisticated methods, see answers to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2662/examples-of-pde-computations-using-parallelism-in-both-space-and-time&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parallelism in time is possible, but is only useful if you need high accuracy (relative to what a cheap stable integrator will give you).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-01-29T06:27:41.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10652" PostTypeId="2" ParentId="10649" CreationDate="2014-01-29T13:24:18.887" Score="2" Body="&lt;p&gt;Brian already provides you with the general outline of how regularization works (apart from the fact that choosing the regularization parameter $\lambda$ is difficult). However, you can make the regularization work better if you encode your knowledge that you expect the highly oscillatory terms to be small, whereas the low frequency terms can be large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way you do this is to recognize that when you solve&#xA;$$&#xA;  \min \|Ac-b\|_2^2 + \lambda\|c\|_2^2&#xA;$$&#xA;you essentially &lt;em&gt;penalize&lt;/em&gt; large coefficients $c_i$. In other words, what you are doing is trying to find those coefficients $c$ so that $Ac-b$ is small &lt;em&gt;but also so that&lt;/em&gt; $c$ is small (and the balance between the two terms is given by $\lambda$). But that's not what you want, according to your problem description: you only want the high frequency terms to be small. Consequently, you may want to minimize the following instead:&#xA;$$&#xA;  \min \|Ac-b\|_2^2 + \sum_i\lambda_i|c_i|^2&#xA;$$&#xA;where $\lambda_i$ is an &lt;em&gt;increasing&lt;/em&gt; sequence so that higher coefficients are penalized more. What form you choose for the $\lambda_i$ is something you need to decide based on what your &lt;em&gt;expectations&lt;/em&gt; are for the sizes of the coefficients. An example may be something along the lines&#xA;$$&#xA;  \lambda_i = \bar\lambda \sqrt{i}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a general theory on how regularization should be chosen to match what we expect solutions to look like. If you want to spend a significant amount of time on these issues, I suggest to read the books by Tarantola (&quot;Inverse Problem Theory&quot;) or by Kaipio and Somersalo (&quot;Statistical and computational inverse problems&quot;).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-01-29T13:24:18.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10653" PostTypeId="1" CreationDate="2014-01-29T15:28:55.277" Score="3" ViewCount="2239" Body="&lt;p&gt;I've an equation like this to solve with the crank-nicolson method&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$U_t -\frac{y}{2} U_x + \frac{x}{2}U_y = 0,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; are: &lt;code&gt;[-2,5:2,5]&lt;/code&gt; and the time &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$2\pi$&lt;/span&gt;.&#xA;Is there anyone who can help me?&#xA;I have to solve it in a matlab code.&#xA;thanks everyone.&#xA;I'm sorry for my english.&lt;/p&gt;&#xA;" OwnerUserId="7205" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-04-30T18:07:59.987" Title="How to solve the advection equation in 2 dimension using the Crank-Nicolson method?" Tags="|matlab|finite-difference|crank-nicolson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10655" PostTypeId="1" CreationDate="2014-01-29T21:20:22.830" Score="2" ViewCount="446" Body="&lt;p&gt;I'm looking for a library that is either in R or easily wrappable with R, that can do the following things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;construct and subset N-dimensional arrays&lt;/li&gt;&#xA;&lt;li&gt;perform operations such as &lt;code&gt;min&lt;/code&gt;, &lt;code&gt;max&lt;/code&gt;, &lt;code&gt;sum&lt;/code&gt;, &lt;code&gt;sd&lt;/code&gt; along any dimension&lt;/li&gt;&#xA;&lt;li&gt;is fast and ideally has a BLAS backend&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And (this is important), is able to define multilevel-masks along any dimension (and combinations thereof) where the above operations are summarised. E.g.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 3 5&#xA;2 4 6   -(rowmask=[1,3])-(colmask=1,2)-(op=sum)-(axis=rows)-&amp;gt;  8 11&#xA;7 8 9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 3 5                                                      3 7 11&#xA;2 4 6   -(rowlevels=['a','a','b'])-(op=sum)-(axis=rows)-&amp;gt;  7 8 9&#xA;7 8 9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and ideally a combination of the two as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've started coding this in core R but it gets a mess quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="1601" LastEditorUserId="1601" LastEditDate="2014-01-30T00:04:30.077" LastActivityDate="2014-01-31T23:08:59.887" Title="R/C/C++ library for N-dimensional arrays" Tags="|matrix|libraries|data-analysis|r|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10656" PostTypeId="1" AcceptedAnswerId="10705" CreationDate="2014-01-29T23:30:13.233" Score="3" ViewCount="219" Body="&lt;p&gt;comp! I'm trying to solve the advection-reaction problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ dg/dt = dg/dx + x\cdot g \qquad on~~x \in \Omega = (-\infty, +\infty)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;supplemented with the boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \lim_{\lvert x \rvert \to\infty} g(x, t) = 0.0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and subject to the constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g(x=0, t) = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that a non-trivial steady state solution exists for this system and is given by $g_{\infty}(x) = \exp(-0.5x^2)$. (Note that without the constraint at $x=0$ the PDE would be ill-posed as there would be infinitely many admissible solutions, each offset by a multiplicative factor.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm unsure as to how I would go about solving this system numerically. My initial approach was to use an upwind stencil (i.e. $\delta_xg_k \approx \frac{1}{2\Delta x}(-g_{k} + g_{k+1})$) since it's an advection-like equation and then time march using implicit Euler. Moreover, I truncated the infinite domain by approximating it as $\widehat{\Omega} = [-10.0, +10.0]$ and modified the boundary conditions accordingly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this scheme, I get the following results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/p3Urs.png&quot; alt=&quot;Numerical Solution using First Order Forward Differencing and Implicit Euler&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course in retrospect this result isn't too surprising -- we can think of the original problem as two separate subproblems. Problem 'A' exists on the domain $x \in [-10.0, ~0.0]$ and has the outflow boundary conditions $g(-\infty, t) = 0$ and inflow boundary condition $g(0, t) = 1$ -- physically this problem makes sense, which explains why we obtain the correct response.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the remaining problem ('Problem B') exists on the domain $x \in [0.0, +10.0]$ and has the OUTFLOW boundary $g(0, t) = 1$ and INFLOW $g(+\infty, t) = 0.0$. Using my upwind stencil, I can never incorporate the boundary at $x = 0.0$ which explains why on the RHS we obtain the trivial solution. If I try to use a downwind or centered stencil though, the solution blows up since the corresponding semi-discrete form is unstable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Surprisingly, even though an analytical steady state solution exists, it doesn't seem that it's possible to recover it numerically... Am I missing something in my understanding/Have you ever encountered a similar problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice on this matter would be appreciated! Thanks in advance for your help.&lt;/p&gt;&#xA;" OwnerUserId="7208" LastActivityDate="2014-10-11T08:38:31.640" Title="Finite Differencing of a Strange Advection-Reaction Problem" Tags="|finite-difference|advection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10657" PostTypeId="1" CreationDate="2014-01-30T00:40:48.140" Score="2" ViewCount="209" Body="&lt;p&gt;This topic is used in spectral methods, for collocation grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Literature mentions Chebyshev interpolation on a grid (defined by $\xi_j = cos(\pi \cdot j/N)$, $x_j = (\xi_j+1) L/2$, $j=0,...,N$) includes 2 steps&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;1/ Calculation of Chebyshev coefficients (sum over all points $x$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c_n = \sum_{j=0}^N f(x_j)  \cdot T_n (\xi_j)\cdot w_j$ — In the $x$ direction. Does not depend on a specific $x$, only on $n$...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2/ The interpolation itself&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$result(x) = \sum_{n=0}^{N} c_n \cdot T_n\left ( 2\cdot {x \over L}-1\right )$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;How can the same be done in 2-dimensional space?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I could come up with these steps, but no literature supporting such point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is $N_x + N_y$ coefficients, as mentioned in one of the answers.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Calculation of Chebyshev coefficients (sum over all points $x$)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$c_n^{(y_0)} = \sum_{j=0}^{N_x} f(x_j, y_0)  \cdot T_n (\xi_j)\cdot w^{(x)}_j$ — in the $x$ direction&lt;/li&gt;&#xA;&lt;li&gt;$d_n^{(x_0)} = \sum_{k=0}^{N_y} f(x_0, y_k)  \cdot T_n (\eta_k)\cdot w^{(y)}_k$ — in the $y$ direction&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;B. The interpolation itself&#xA;$$&#xA;result(x,y) = \sum_{n=0}^{N_x} \sum_{k=0}^{N_y} c_n^{(y_k)} d_n^{(x_n)} \cdot T_n\left ( 2\cdot {x \over L_x}-1\right ) \cdot T_k\left ( 2\cdot {y \over L_y}-1\right )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;(Or these two steps, which are a quite different thing.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is $N_x \cdot N_y$ coefficients, as mentioned in one of the answers.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Calculation of Chebyshev coefficients (sum over all points $x$)&#xA;$$&#xA;c_{n,k} = \sum_{j=0}^{N_y}  \sum_{i=0}^{N_x} f(x_i, y_j)  \cdot T_n (\xi_i)\cdot T_k (\eta_j)\cdot w^{(x)}_i \cdot w^{(y)}_j&#xA;$$&#xA; B. The interpolation itself&#xA;$$&#xA;result(x,y) = \sum_{i=0}^{N_x} \sum_{j=0}^{N_y} c_{i,j} \cdot T_i\left ( 2\cdot {x \over L_x}-1\right ) \cdot T_j\left ( 2\cdot {y \over L_y}-1\right )&#xA;$$&lt;/p&gt;&#xA;" OwnerDisplayName="user5273" LastEditorDisplayName="user5273" LastEditDate="2014-02-03T00:56:52.313" LastActivityDate="2014-02-03T00:56:52.313" Title="How do I do Chebyshev interpolation in multi-dimentional space?" Tags="|interpolation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10658" PostTypeId="2" ParentId="10657" CreationDate="2014-01-30T02:05:39.193" Score="2" Body="&lt;p&gt;Note that it's most common to use a &lt;em&gt;nodal&lt;/em&gt; (rather than modal) Chebyshev basis, in which case the coefficients are just the values at the Chebyshev nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your definition is not a very useful interpolation in 2D.  You want something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathrm{result}(x,y) = \sum_{n=0}^{N_x} \sum_{k=0}^{N_y} c_{n,k} T_n\left(\frac{2 x}{L_x} - 1\right) T_k\left(\frac{2 y}{L_y} - 1\right).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this has $N_x N_y$ coefficients rather than $N_x + N_y$.  If the functions you want to represent have rapid decay in mixed derivatives, you can use &quot;sparse grids&quot; to reduce the number of coefficients.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-01-30T02:05:39.193" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10659" PostTypeId="1" CreationDate="2014-01-30T11:07:26.193" Score="2" ViewCount="55" Body="&lt;p&gt;In &lt;a href=&quot;http://www.featflow.de/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark1_re20.html&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; simple benchmark describing the flow around a disc in the plane with stationary bc, for low Reynolds numbers the solution is stationary and for higher RN it is &lt;a href=&quot;http://www.featflow.de/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark2_re100.html&quot; rel=&quot;nofollow&quot;&gt;periodic in time&lt;/a&gt;. Is there a theory saying, by which RN periodicity disappears and the solution(s) become(s) completely chaotic (if it happens at all)?&lt;/p&gt;&#xA;" OwnerUserId="7212" LastEditorUserId="7212" LastEditDate="2014-01-30T12:20:36.813" LastActivityDate="2014-01-30T12:20:36.813" Title="When a flow changes its character from time-periodic to chaotic?" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10660" PostTypeId="2" ParentId="10653" CreationDate="2014-01-30T15:10:38.120" Score="3" Body="&lt;p&gt;Here is a tutorial on how to solve this equation in 1D with example code. The code is Python (which is similar to MATLAB so you should be able to translate). To extend this to 2D you just follow the same procedure for the other dimension and extend the matrix equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/7399/how-to-discretize-the-advection-equation-using-the-crank-nicolson-method&quot;&gt;How to discretize the advection equation using the Crank-Nicolson method?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-01-30T15:10:38.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10661" PostTypeId="2" ParentId="10655" CreationDate="2014-01-30T16:37:41.723" Score="0" Body="&lt;p&gt;The array function in base R provides this functionality:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- array(c(1, 2, 7, 3, 4, 8, 5, 6, 9), dim = c(3, 3, 1))&#xA;max(x[1,,])&#xA;min(x[,1,])&#xA;sd(x[,,1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The slam package provides a format for sparse arrays:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s &amp;lt;- as.simple_sparse_array(x)&#xA;max(s[1,,])&#xA;min(s[,1,])&#xA;sd(as.vector(s[,,1]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I haven't ever used &lt;a href=&quot;http://cran.r-project.org/web/packages/slam/index.html&quot; rel=&quot;nofollow&quot;&gt;slam&lt;/a&gt;, so I'm not familiar with the ins and outs.  If your arrays are going to be pretty dense, just use R arrays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Between array indexing and the apply function, you should be able to do most of what you want.&lt;/p&gt;&#xA;" OwnerUserId="152" LastEditorUserId="152" LastEditDate="2014-01-30T16:43:26.263" LastActivityDate="2014-01-30T16:43:26.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10662" PostTypeId="2" ParentId="10549" CreationDate="2014-01-30T17:19:16.350" Score="1" Body="&lt;p&gt;I have not been able to resolve completely why Strategy (2) doesn't work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best intuition is that for a given PDE/mono-fem-solution, there always exists &lt;em&gt;some&lt;/em&gt; choice of boundary dirichlet/neumann/robin data that will give you the same (restricted) solution in each subdomain. But it's not necessarily computable from the $u$ values of adjacent domains, or even the entire $u$ state over &lt;em&gt;every&lt;/em&gt; other domain. The correct robin data could be something totally different, it's unknown. That's what the addition of new unknowns $\lambda$ (lagrange multipliers) represents - the &quot;correct&quot; but unknown robin data. They must be solved concurrently with $u$, and should have similar &quot;galerkin-ness&quot; property as $u$ (in the sense that error in $\lambda$ should be constrained orthogonal to the space that represents it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This answer has a lot of squish, but I have pressed on. Happy to accept something better if it ever appears!&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2014-01-30T17:19:16.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10663" PostTypeId="2" ParentId="10655" CreationDate="2014-01-30T18:32:06.733" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/ContinuumIO/libdynd&quot; rel=&quot;nofollow&quot;&gt;DyND&lt;/a&gt; library might interest you.  It comes out of the Scientific Python ecosystem as a numpy replacement but I believe that it is straight C++ and so should be easily wrappable in R.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, having a single numeric library shared between the two languages might have unintended benefits.&lt;/p&gt;&#xA;" OwnerUserId="19" LastActivityDate="2014-01-30T18:32:06.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10664" PostTypeId="1" CreationDate="2014-01-30T20:46:58.857" Score="0" ViewCount="355" Body="&lt;p&gt;Given a standard advection equation, we write the update as&#xA;$$&#xA;q_i^{n+1}=q_i^n+\frac{\Delta t}{\Delta x}\left(F_i^{n+1/2}-F_{i+1}^{n+1/2}\right)&#xA;$$&#xA;with $F_i^{n+1/2}=F\left(q_i^n,\,q_i^*\right)$ and $q^*_i$ a predicted value, other variables take their normal meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to reconstruct what numeric value $F_i^{n+1/2}$ possessed if I know $q^n_i$ and $q^{n+1}_i$? Or is it once I deallocate the array &lt;code&gt;F&lt;/code&gt;, that information is lost to me?&lt;/p&gt;&#xA;" OwnerUserId="4762" LastActivityDate="2014-10-22T17:58:06.273" Title="Reconstructing fluxes" Tags="|fluid-dynamics|finite-difference|finite-volume|" AnswerCount="2" CommentCount="4" ClosedDate="2014-10-24T02:15:18.357" ContentLicense="CC BY-SA 3.0" />
  <row Id="10665" PostTypeId="1" CreationDate="2014-01-31T08:25:01.453" Score="3" ViewCount="373" Body="&lt;p&gt;I'm modeling mass transport in a flow reactor. The flow reactor is a tube, which allows me to use cylindrical symmetry in solving the Convection-Diffusion-Reaction (CDR) Equation, which governs the mass transport.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation that I'm solving is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D ( \frac{\partial^2 u}{\partial r^2} + \frac{1}{r}\frac{\partial u}{\partial r} ) - v(r) \frac{\partial u}{\partial z} = S(r)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, r and z are the coordinates in the respective directions, u is the property I want to solve for (the mass concentration), v is the velocity field, S is a given source and D is the diffusion coefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Boundary conditions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial c}{\partial r} = 0, $ at $r=0$ and at $r=R$ (where R is the length of the rectangle in the r direction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system is sketched in the figure below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rdmyk.png&quot; alt=&quot;enter image description here&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach is to use the Method of Lines, i.e. to solve the system in the r direction for each step in the z direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I discretised the above equation as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{du}{dz} = \frac{D}{v_i}(\frac{u_{i+1} - 2u_i + u{i-1}}{2h}+\frac{1}{r_i}\frac{u_{i+1}-u_{i-1}}{2h}) - S_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For both boundary conditions I used the central difference approximation, i.e. at r = 0 and r = R:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{u_{i+1} - u_{i-1}}{2h} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a grid with two ghost points, as shown in the figure below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4qSda.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the code works ok, if I do not put the source on any of the boundaries, see the first plot below. However, in the actual experiment the source is in the centre of the flow reactor, i.e. the source is located on the boundary r = 0. When I put the source there, I get quite large oscillations (see the second plot below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that I've either implemented the wrong boundary condition or that I'm using the wrong grid for this kind of computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My thoughts on removing those oscillations are:&lt;/strong&gt;&#xA; 1. use a staggered grid, i.e. do not actually put a source on the boundary&#xA; 2. use a mixed boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What would you recommend? Am I missing something else?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The MatLab code is also attached below.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xtpDm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/uQfxH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all&#xA;close all &#xA;&#xA;%radius of flow reactor in mm&#xA;R = 13&#xA;&#xA;%step-size in r direction &#xA;hr = 0.2&#xA;r = -hr : hr : R + hr&#xA;&#xA;%parameters&#xA;D = 1.0 %diffusion&#xA;v = 1./(13.1 - r)&#xA;v = ones(1,length(r))&#xA;&#xA;&#xA;%build diagonals a, b, c of matrix A&#xA;%&#xA;%      |b c 0 0 0|&#xA;%      |a b c 0 0|&#xA;%  A = |0 a b c 0|&#xA;%      |0 0 a b c|&#xA;%      |0 0 0 a b|&#xA;%&#xA;% with a = alpha - beta, b = -2alpha, c = alpha + beta&#xA;&#xA;alpha = ones(length(r)-2,1) ./ v(2:end-1)' .* (D/hr^2)&#xA;beta = ones(length(r)-2,1) ./ (v(2:end-1).* r(2:end-1))' .* (D/(2*hr))&#xA;&#xA;diag_a = alpha - beta&#xA;diag_b = -2*alpha&#xA;diag_c = alpha + beta&#xA;&#xA;A= spdiags([diag_a diag_b diag_c],[-1 0 1],length(r)-2,length(r)-2)&#xA;&#xA;%implement boundary conditions du/dr = 0 at r = 0&#xA;A(1,1) = -4*alpha(1)&#xA;A(2,1) =  4*alpha(1)&#xA;&#xA;%implement boundary conditions du/dr = 0 at r = R&#xA;A(end-1,end)= A(end-1,end)*2 &#xA;&#xA;ht = 0.00001&#xA;u = zeros(length(r)-2,1);&#xA;&#xA;source = zeros(length(r)-2,1);&#xA;source(1:length(r)/2-2) = 1.&#xA;&#xA;&#xA;n_steps = 500&#xA;&#xA;clear ustored&#xA;&#xA;ustored = []&#xA;&#xA;%step forward in time with explicit Euler&#xA;for i = 1:n_steps   &#xA;&#xA;    ustored = [ustored u];&#xA;    u_next = u(:,1) + ht*(A*u(:,1)+source(:,1));&#xA;    u = u_next;&#xA;&#xA;end&#xA;&#xA;&#xA;ustored = [ustored u]&#xA;&#xA;time = 0:ht:(n_steps)*ht;&#xA;&#xA;mesh(time,r(2:end-1),ustored)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="4244" LastEditDate="2014-01-31T13:02:29.497" LastActivityDate="2014-01-31T13:02:29.497" Title="Transport Equation in a Tube: Source Term on Boundary" Tags="|matlab|fluid-dynamics|boundary-conditions|cdr-equation|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10666" PostTypeId="1" AcceptedAnswerId="10670" CreationDate="2014-01-31T13:53:38.147" Score="2" ViewCount="1857" Body="&lt;p&gt;I have a program that generates mesh for given 3d models. The generated mesh must use quadrangle elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is required to add &quot;shell elements generation capabilities&quot;. The requirement is as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The mesh generator must have the ability to mesh &quot;shell elements&quot;.&#xA;  Shell elements are two-dimensional elements that are created by&#xA;  meshing surfaces with zero-thickness. Shell elements are elements in a&#xA;  mesh that do not form a closed surface. The geometry examples provided&#xA;  are all closed volumes. An example of shell elements can be seen here&#xA;  -&lt;a href=&quot;http://images.machinedesign.com/images/archive/closer0300jpg_00000036105.jpg&quot; rel=&quot;nofollow&quot;&gt;http://images.machinedesign.com/images/archive/closer0300jpg_00000036105.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't know what are shell elements, how are they generated. Thanks for any help&lt;/p&gt;&#xA;" OwnerUserId="7227" LastActivityDate="2014-02-02T21:03:28.543" Title="How can I generate shell elements to a mesh" Tags="|computational-geometry|mesh-generation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10667" PostTypeId="1" AcceptedAnswerId="10690" CreationDate="2014-01-31T17:13:12.100" Score="8" ViewCount="3414" Body="&lt;p&gt;What is the best way to compute:&#xA;$$ Y = D X $$&#xA;where $D \in \mathbb{R}^{m\times m}$ is diagonal and $X \in \mathbb{C}^{m \times n}$ is general.  I am mostly interested in these two cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$m &amp;gt;&amp;gt; n$,  $m &amp;gt; 10^7$&lt;/li&gt;&#xA;&lt;li&gt;$n &amp;gt;&amp;gt; m$,  $m &amp;lt; 10^4$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Options&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I can think of four not-obviously-flawed ways of doing this: loops, forall, loop over &lt;a href=&quot;http://www.netlib.org/blas/zgbmv.f&quot; rel=&quot;nofollow noreferrer&quot;&gt;zgbmv&lt;/a&gt;, loop over &lt;a href=&quot;http://www.netlib.org/blas/zdscal.f&quot; rel=&quot;nofollow noreferrer&quot;&gt;zdscal&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Loop&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do i = 1,n&#xA;  do j = 1,m&#xA;    Y(j,i) = D(j) * X(j,i)&#xA;  enddo&#xA;enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pros: easy to read, reads D, X, Y in order&lt;/li&gt;&#xA;&lt;li&gt;Cons: doesn't re-use D&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Forall&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;forall (i = 1:n, j = 1:m) Y(j,i) = D(j) * X(j,i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pros: concise, gives compiler freedom?&lt;/li&gt;&#xA;&lt;li&gt;Cons: gives compiler freedom?&lt;/li&gt;&#xA;&lt;li&gt;Notes: previous dicsussions of forall &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6842/effecient-cfd-programming-techniques/6843#6843&quot;&gt;in these comments&lt;/a&gt; and &lt;a href=&quot;http://compgroups.net/comp.lang.fortran/forall-vs-do.enddo-vs-where/382733&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;zgbmv&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Dz = cmplx(D)&#xA; do i = 1,n&#xA;   call zgbmv('N', m, m, 0, 0, one, D, 1, X(1,i), 1, zero, Y(1,i), 1)&#xA; enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pros: similar to loop, but could contain BLAS magic&lt;/li&gt;&#xA;&lt;li&gt;Cons: doesn't re-use D, double size of D by casting to complex&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;zdscal&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Y = X&#xA; do i = 1,m&#xA;   call zdscal(n,D(i),Y(i,1),m)&#xA; enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pros: re-uses D, could contain BLAS magic&lt;/li&gt;&#xA;&lt;li&gt;Cons: strided reads of Y, requires copy if not in-place&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Thoughts&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The two major trade-offs seem to be in order reads of X vs re-use of D and use of fortran libraries vs treating D as a real instead of casting to complex.  A custom implementation could get the best of both worlds in both cases, but I'm leery of architecture-specific parameters. Best case would be a way to express the operation natively (e.g. loops or forall) and tell the compiler to do the rest.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-02-03T14:44:23.070" Title="What is the best way to multiply a diagonal matrix (in fortran)" Tags="|linear-algebra|performance|blas|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10668" PostTypeId="2" ParentId="10655" CreationDate="2014-01-31T23:08:59.887" Score="1" Body="&lt;p&gt;By far the best solution for R is DataTable&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cran.r-project.org/web/packages/data.table/index.html&quot; rel=&quot;nofollow&quot;&gt;http://cran.r-project.org/web/packages/data.table/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure about BLAS backend, but DataTable is wickedly fast. I had a job that used aggregation from data.frame that took about 24 hours. I simply replaced everything with data tables and it finished in about 20 minutes. This was 2 years ago when the package was in its infancy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;see timings:&#xA;&lt;a href=&quot;http://cran.r-project.org/web/packages/data.table/vignettes/datatable-timings.pdf&quot; rel=&quot;nofollow&quot;&gt;http://cran.r-project.org/web/packages/data.table/vignettes/datatable-timings.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="777" LastActivityDate="2014-01-31T23:08:59.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10669" PostTypeId="2" ParentId="10664" CreationDate="2014-02-01T08:48:43.720" Score="1" Body="&lt;p&gt;You could solve a linear system in terms of the $F_{i}^{n+1/2}$.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-02-01T08:48:43.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10670" PostTypeId="2" ParentId="10666" CreationDate="2014-02-01T14:30:47.407" Score="2" Body="&lt;p&gt;From the meshing point of view shell elements are just 2D elements. They can represent 3D structures because of the thickness assigned to them. This thickness is not modeled, but given to the analyzer as a property. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to create a shell mesh from a solid model, you have to remove the thickness dimension (ie. create quadrilateral element from a brick element) Mesh the surface and give the length of the removed dimension to the analyzer as a property. &lt;/p&gt;&#xA;" OwnerUserId="7179" LastActivityDate="2014-02-01T14:30:47.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10671" PostTypeId="1" CreationDate="2014-02-01T15:24:03.777" Score="4" ViewCount="10402" Body="&lt;p&gt;I am working on a project that I need to add a regularization into the NNLS algorithm. Is there a way to add the Tikhonov regularization into the NNLS implementation of scipy [1]?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] talks about it, but does not show any implementation. Sklearn has an implementation, but it is not applied to nnls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.nnls.html&quot; rel=&quot;nofollow&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.nnls.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://icses2012.pwr.wroc.pl/article/34.pdf&quot; rel=&quot;nofollow&quot;&gt;http://icses2012.pwr.wroc.pl/article/34.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7236" LastActivityDate="2020-06-16T15:19:55.073" Title="Tikhonov regularization in the non-negative least square - NNLS (python:scipy)" Tags="|optimization|convex-optimization|nonlinear-programming|regression|scipy|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10672" PostTypeId="2" ParentId="10671" CreationDate="2014-02-01T15:30:21.650" Score="0" Body="&lt;p&gt;If the fortran code &lt;a href=&quot;https://github.com/scipy/scipy/blob/master/scipy/optimize/nnls/nnls.f&quot; rel=&quot;nofollow&quot;&gt;https://github.com/scipy/scipy/blob/master/scipy/optimize/nnls/nnls.f&lt;/a&gt; doesn't have a Tikhonov regularization option, then it would probably take some work to add this feature to scipy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I looked at the .pdf.  I guess you want to find the smallest $\lambda$ so that $\text{argmin}_x ||Ax-b||^2 + \lambda^2 ||x||^2$ is positive.  I think the $\text{argmin}_x$ for any given $\lambda$ has a more or less closed form solution, and the .pdf mentions bisection search on $\lambda$ until you find the smallest one so that the best $x$ is positive.  This seems pretty straightforward, in the sense that it shouldn't need any weird decompositions or messy fortran code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: I noticed that this question is tagged &lt;code&gt;convex-optimization&lt;/code&gt;.  If you want to take that approach literally, then you could try cvx (&lt;a href=&quot;http://cvxr.com/cvx/&quot; rel=&quot;nofollow&quot;&gt;http://cvxr.com/cvx/&lt;/a&gt;) or cvxpy (&lt;a href=&quot;https://github.com/cvxgrp/cvxpy&quot; rel=&quot;nofollow&quot;&gt;https://github.com/cvxgrp/cvxpy&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2014-02-01T16:22:59.133" LastActivityDate="2014-02-01T16:22:59.133" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10673" PostTypeId="2" ParentId="10671" CreationDate="2014-02-01T16:31:57.793" Score="20" Body="&lt;p&gt;If what you want is to solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\min \| Ax - b \|_{2}^{2} + \lambda^{2} \| x \|_{2}^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x \geq 0$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then this is easily implemented.  Construct a matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$C=\left[&#xA;\begin{array}{c}&#xA;A \\&#xA;\lambda I &#xA;\end{array}&#xA;\right]$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a vector&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$d=\left[&#xA;\begin{array}{c}&#xA;b \\&#xA;0 &#xA;\end{array}&#xA;\right]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then use your nonnegative least squares solver on &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\min \| Cx - d \|_{2}^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x \geq 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can easily extend this to problems of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\min \| Ax-b \|_{2}^{2} + \lambda^{2} \| L (x-x_{0}) \|_{2}^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by letting &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$C=\left[&#xA;\begin{array}{c}&#xA;A \\&#xA;\lambda L &#xA;\end{array}&#xA;\right]$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$d=\left[&#xA;\begin{array}{c}&#xA;b \\&#xA;\lambda Lx_{0} &#xA;\end{array}&#xA;\right]$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2020-06-16T15:19:55.073" LastActivityDate="2020-06-16T15:19:55.073" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10674" PostTypeId="2" ParentId="8942" CreationDate="2014-02-02T02:38:35.637" Score="3" Body="&lt;p&gt;For $x = 2\pi a/b$ with $a,b$ integer, we have&#xA;$$\begin{aligned}&#xA;f(x) &amp;amp;= \sum_{k \ge 1} \frac{\cos {kx}}{k^2 (2 - \cos {kx})} \\&#xA;   &amp;amp;= \sum_{k=1}^b \frac{\cos {kx}}{2-\cos {kx}} \sum_{n \ge 0} \frac{1}{(k+bn)^2} \\&#xA;   &amp;amp;= \sum_{k=1}^b \frac{\cos {kx}}{2-\cos {kx}} \frac{\psi^1(k/b)}{b^2}&#xA;\end{aligned}$$&#xA;where $\psi^1(z)$ is the trigamma function (&lt;a href=&quot;http://en.wikipedia.org/wiki/Polygamma&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Polygamma&lt;/a&gt;).  Here are plots of the function and its derivative with the artifacts removed:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/jLyBN.png&quot; alt=&quot;Values and derivatives for the series&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2014-02-03T17:16:23.080" LastActivityDate="2014-02-03T17:16:23.080" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10675" PostTypeId="1" CreationDate="2014-02-02T09:43:32.437" Score="2" ViewCount="465" Body="&lt;p&gt;I have a nxn covariance matrix (so, real, symmetric, dense, nxn... I mean positive semi-definite). 'n' may be very very very big! I'd like to solve partial (3 largest) eigenvalue (+eigenvectors) problem for this matrix. Could somebody tell me what the fastest algorithm to do it?&#xA;Is it power iteration method (because it have O(n^2) complexity)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I'd like to make GPGPU implementation using OpenCL. &lt;/p&gt;&#xA;" OwnerUserId="7100" LastEditorUserId="7100" LastEditDate="2014-02-02T16:06:41.860" LastActivityDate="2016-07-01T19:03:56.830" Title="Algorithm for Eigenvalue Problem of a Real Symmetric nxn Matrix" Tags="|linear-algebra|eigensystem|gpu|eigenvalues|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10676" PostTypeId="1" AcceptedAnswerId="10677" CreationDate="2014-02-02T12:34:26.733" Score="3" ViewCount="1342" Body="&lt;p&gt;I have the following quadratic program&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; f(x) = \frac{1}{2} x^T D x + c^T x\\ \text{subject to} &amp;amp; x_{\text{lower}} &amp;lt; x &amp;lt; x_{\text{upper}}\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where matrix $D$ is symmetric &lt;strong&gt;negative&lt;/strong&gt; definite!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no problems with using symmetric positive definite matrices for $D$. I use JOptimizer's QP-solver for it. But with negative $D$ it doesn't work.&#xA;(JOptimizer can deal only with convex optimization.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I solve it? Does it have to be a non-convex problem solver? I found &lt;a href=&quot;https://projects.coin-or.org/Couenne&quot; rel=&quot;nofollow&quot;&gt;Couenne&lt;/a&gt; so far, which can handle non-convex MINLPs.&lt;/p&gt;&#xA;" OwnerUserId="7146" LastEditorUserId="20417" LastEditDate="2016-10-27T17:56:57.477" LastActivityDate="2016-10-28T21:19:14.457" Title="Minimizing a negative definite quadratic function with specified bounds" Tags="|optimization|quadratic-programming|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10677" PostTypeId="2" ParentId="10676" CreationDate="2014-02-02T16:09:51.710" Score="2" Body="&lt;p&gt;Your problem is a non convex bounds constrained quadratic programming problem.  There are lots of software packages that can solve such problems.  See the list of solvers at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://plato.asu.edu/sub/nlores.html#QP-problem&quot; rel=&quot;nofollow&quot;&gt;http://plato.asu.edu/sub/nlores.html#QP-problem&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-02-02T16:09:51.710" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10678" PostTypeId="1" AcceptedAnswerId="10681" CreationDate="2014-02-02T16:19:03.980" Score="23" ViewCount="50819" Body="&lt;p&gt;I just started studying FEM in a more structured basis compared to what I used to do during my undergraduate courses. I am doing this because, despite the fact that I can use the &quot;FEM&quot; in commercial (and other non-commercial) software, I would like to really understand the underground techniques that support the method. That's why I am coming here with such, at least for the experienced user of the technique, basic question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm reading a quite popular (I think) and &quot;engineer-friendly&quot; book called &quot;Finite element method- The basics&quot; from Zienkwicz. I've been reading this book from the first page but I yet can't understand the concept of shape function in the way Zienkwicz explains it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I know about from the things that I'd read is that a &quot;Stiffness&quot; matrix, the one that relates the unknowns with the result ($A$ in: $Ak=b$), has its components from the &quot;relationships between the nodes&quot;, and if that &quot;relationship&quot; changes, (i.e. if we change it to a Higher order interpolant), that stiffness matrix changes, because the relationship between the nodes does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in this book, the definition is quite fuzzy for me, because in some point it says that you can arbitrarily chose the function as, i.e., the identity matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DpPy6.png&quot; alt=&quot;Chapter 2.2.1. FEM-Basics Zienkwicz&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only explanation I found is in &lt;a href=&quot;http://stochasticandlagrangian.blogspot.com.es/2011/07/what-does-shape-function-mean-in-finite.html&quot; rel=&quot;noreferrer&quot;&gt;this blog&lt;/a&gt; , but it is still not so clear for me. So, somebody can give me a simple plain explanation of what is a Shape functon and how it is done to &quot;put it&quot; in the stifness matrix?&lt;/p&gt;&#xA;" OwnerUserId="5083" LastEditorUserId="9667" LastEditDate="2015-06-29T17:23:10.870" LastActivityDate="2020-02-09T00:59:59.320" Title="Basic explanation of shape function" Tags="|finite-element|" AnswerCount="9" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10679" PostTypeId="2" ParentId="10675" CreationDate="2014-02-02T16:31:45.430" Score="6" Body="&lt;p&gt;A 100,0000 by 100,000 symmetric dense matrix in single precision requires 20 gigabytes of memory (storing only the upper triangle) or 40 gigabytes of memory for double precision. Thus it is too large to fit within the memory of available GPU's.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to solve this problem using GPU acceleration you'd have to develop an algorithm that sends smaller chunks of work to the GPU.  Copying parts of the matrix into the GPU in each iteration is unlikely to work well.  Rather, you could try to use multiple GPU's in parallel, with each GPU storing part of the matrix throughout the algorithm. For example, you could use four of the NVIDIA Tesla K20X's to store your single precision matrix.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The appropriate algorithms to consider for this kind of problem are iterative schemes that use matrix-vector multiplications.   Start by looking at the algorithms implemented in ARPACK.  Since ARPACK can use BLAS routines for the matrix-vector multiplications, you should be able to simply link ARPACK with some GPU implementation of the BLAS to get a working code.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-02-02T16:31:45.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10680" PostTypeId="2" ParentId="10678" CreationDate="2014-02-02T16:45:33.917" Score="4" Body="&lt;p&gt;The most important thing to know about &quot;shape functions&quot; is that they&#xA;describe how the dependent variable(s) you want to calculate (e.g. displacement)&#xA;vary as a function of the spatial coordinates of the element (e.g. x and y) in&#xA;terms of some unknown scalar parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often the shape functions are simple polynomials and the scalar parameters are the&#xA;values of the dependent variables at the element nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forming the finite element equations using these shape functions requires a few&#xA;other fundamental concepts such as establishing a &quot;weak form&quot; of the partial&#xA;differential equation you are trying to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a lot of unnecessary &quot;mysticism&quot; associated with the finite element method&#xA;so I encourage your approach of trying to get a thorough understanding of the&#xA;fundamentals.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-02-02T16:45:33.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10681" PostTypeId="2" ParentId="10678" CreationDate="2014-02-02T17:33:38.583" Score="33" Body="&lt;p&gt;I've always found the approach to describing finite element methods that focuses on the discrete linear system and works backward unnecessarily confusing. It is much clearer to go the other way, even if that involves a bit of mathematical notation in the beginning (which I'll try to keep to a minimum). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that you are trying to solve an equation &lt;span class=&quot;math-container&quot;&gt;$A u = f$&lt;/span&gt; for given &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and unknown &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a linear operator that maps functions (e.g., describing the displacement at every point &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; in a domain) in a space &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; to functions in another space (e.g., describing the applied forces). Since the function space &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is usually infinite-dimensional, this system cannot be solved numerically. The standard approach is therefore to replace &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; by a &lt;em&gt;finite-dimensional&lt;/em&gt; subspace &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; and look for &lt;span class=&quot;math-container&quot;&gt;$u_h \in V_h$&lt;/span&gt; satisfying &lt;span class=&quot;math-container&quot;&gt;$Au_h = f$&lt;/span&gt;. This is still infinite-dimensional due to the range space (which we'll assume for simplicity to be &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; as well), so we just ask for the residual &lt;span class=&quot;math-container&quot;&gt;$Au_h-f\in V$&lt;/span&gt; to be orthogonal to &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; -- or equivalently, that &lt;span class=&quot;math-container&quot;&gt;$v_h^T(Au_h-f) = 0$&lt;/span&gt; for every basis vector &lt;span class=&quot;math-container&quot;&gt;$v_h$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt;. If we now write the &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt; as a linear combination of these basis vectors, we are left with a linear system for the unknown coefficients in this combination. (The terms &lt;span class=&quot;math-container&quot;&gt;$v_i^TAu_j$&lt;/span&gt; are exactly the entries of the stiffness matrix &lt;span class=&quot;math-container&quot;&gt;$K_{ij}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$v_j^Tf$&lt;/span&gt; are the entries of the load vector. If &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a differential operator, one usually performs integration by parts at some point, but this is not important here.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of this so far is specific to finite element methods, but applies to any so-called Galerkin method or method of weighted residuals. &#xA;The finite element method is characterized by a special choice of &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt;: The computational domain is decomposed into a number of &lt;em&gt;elements&lt;/em&gt; of the same basic shape (e.g., triangles; the process is often called &lt;em&gt;triangulation&lt;/em&gt;), and the space &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; is chosen such that restricted to each element, functions in &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; are polynomials (e.g, linear in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;). Furthermore, the basis functions are chosen such that they are non-zero only in (the neighborhood of) one of the elements. The point of this choice is that you can build such a basis of &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; fairly easily by finding a basis &lt;span class=&quot;math-container&quot;&gt;$\{\psi_j\}$&lt;/span&gt; of the polynomial space on a single &lt;em&gt;reference element&lt;/em&gt; (such as the triangle with vertices &lt;span class=&quot;math-container&quot;&gt;$(0,0)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$(0,1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(1,0)$&lt;/span&gt;) and then using an affine transformation to map these basis functions to basis functions on each element in the triangulation. &lt;strong&gt;These &lt;span class=&quot;math-container&quot;&gt;$\psi_j$&lt;/span&gt; are the shape functions.&lt;/strong&gt; Usually, one requires that the local basis functions take the value &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; at only one of the vertices and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; at the others (called a &lt;em&gt;nodal basis&lt;/em&gt;), which is what the page you linked is talking about. Like any polynomial, these are uniquely determined by a number of &lt;em&gt;interpolation conditions&lt;/em&gt; (e.g., a polynomial of degree &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; on an interval is determined by its value at two distinct points; if the basis is a nodal basis, these values are taken at the vertices), which are also referred to as the &lt;em&gt;degrees of freedom&lt;/em&gt; of the element. (The total degrees of freedom can be less than the sum of the degrees of all elements if some have to be fixed to ensure global properties of the approximation such as continuity.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Other choices of &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; lead to other methods; in fact, there are spectral methods where the basis functions are chosen such that the stiffness matrix is the identity. Of course, there's no free lunch, so other parts of the procedure become more difficult with this basis. Similarly, there are other interpolation conditions than point evaluations, such as derivative evaluations or averages over elements or sides.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2020-02-09T00:59:59.320" LastActivityDate="2020-02-09T00:59:59.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10683" PostTypeId="2" ParentId="10676" CreationDate="2014-02-02T19:30:56.250" Score="4" Body="&lt;p&gt;You do realize that there are at least two problems with your goal:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;This is, in general, a problem that can have $2^N$ solutions where $N$ is the number of variables. An example is if you tried to find the solution of&#xA;$$&#xA;    \min_{x\in {\mathbb R}^N} \sum_{1\le i\le N} -x_i^2 \\&#xA;    \text{subject to} \quad -1 \le x_i \le 1.&#xA;  $$&#xA;For this problem, &lt;em&gt;every&lt;/em&gt; vertex of the hypercube $[-1,1]^N$ is a solution, and there are $2^N$ such vertices. In other words, you have a problem that is combinatorially difficult.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your problem is ill-posed: you state your constraints as&#xA;$$ l_i &amp;lt; x_i &amp;lt; u_i $$&#xA;but this is not a closed set and so a minimizer may not exist despite the fact that you have a minimizing sequence of points that converge towards the boundary of your feasible set. You need to formulate your problem on a closed domain by requiring $$ l_i \le x_i \le u_i. $$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-02T19:30:56.250" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10684" PostTypeId="2" ParentId="10678" CreationDate="2014-02-02T20:15:20.273" Score="6" Body="&lt;p&gt;In engineering approach to FEM in Structural Mechanics, how it is presented, you lose the feeling that you are solving &lt;em&gt;Partial Differential Equations&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They show you these matrices, they attach some physical meaning, and in my opinion this leads you to developing a dubious physical intuition for the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;It may be helpful to think about the subject it terms of geometry.&lt;/strong&gt; The solution to a boundary value problem for PDE is some shape. V.I. Arnol'd once said praising Newton's accomplishments in the field, to paraphrase -  he did a marvelous thing by creating the field of differential equations by allowing us to reformulate the problems of natural sciences to geometrical problems of curves in plane and surfaces in space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In FEM you approximate the solution (in FD and FVM you approximate the governing equation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enter Boris Gligorievich Galerkin. What did B.G. Galerkin say?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He said: “&lt;em&gt;I want you, not to be able to make residual with the same basis functions, you used to create the solution.&lt;/em&gt;” &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(P.S. This story is completely not true, and I urge my readers to find a better explanation of (Bubnov-) Galerkin method, if it exists.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basis functions, or trial functions are those that you use to build the solution. You use them to approximate the shape of the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Galerkin wants you to think of functions in abstract way analogous to vectors in ordinary Euclidean space, and using the notion of orthogonality to create enough algebraic conditions to find the necessary parameters to build the solution. These algebraic conditions when all put together create $Ku=f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FEM is when you approximate shape of the solution function piece by piece. In each piece - the element - you have some basic shapes (shape functions) that have some flexibility and can approximate various solutions - but only one is the solution to your problem. There are shape functions as much as there are nodes. I-th shape function is equal to one in i-th node, in other nodes it is zero. Solving $Ku=f$ we find parameters that define shape functions - their values at nodes, the mix of shape functions in an element gives the solution over that element, collecting the shapes piece by piece over elements we get global shape-which is a sought solution to our BVP.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="10160" LastEditDate="2016-12-20T11:12:54.567" LastActivityDate="2016-12-20T11:12:54.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10685" PostTypeId="2" ParentId="10666" CreationDate="2014-02-02T21:03:28.543" Score="1" Body="&lt;p&gt;The term &quot;shell&quot; has a very specific meaning in solid mechanics, see for example &lt;a href=&quot;http://en.wikipedia.org/wiki/Plate_theory&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Plate_theory&lt;/a&gt; . Shell elements are simply finite elements that are used to numerically approximate the solution of equations derived in shell or plate theory. There are many books in the engineering literature that will help you understand what exactly all of this means and is.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-02T21:03:28.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10686" PostTypeId="2" ParentId="10617" CreationDate="2014-02-02T23:29:58.727" Score="3" Body="&lt;p&gt;Concentrated forces and constraints are &lt;em&gt;per se&lt;/em&gt; unphysical: have you never seen a force acting on a true geometrical point? And a point-wise roller?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever you use beam theory (whether Euler-Bernoulli or Timoschenko) you should ask yourself about its hypothesis and how the theory maps to real world beams. The same holds true for Continuum Mechanics and shell theories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This said in beam theory all internal forces are vanishing in $ab$ which is subject only to rigid body motion, as outlined by Bill in his &lt;a href=&quot;https://scicomp.stackexchange.com/a/10618/1744&quot;&gt;answer&lt;/a&gt;. Please note that under the small deflection assumption this means linear transversal displacement. (Of course this is not a &lt;em&gt;true&lt;/em&gt; rigid body motion if finite displacement are assumed: is this the origin of your supposed axial deformation?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 3D continuum elements, concentrated loads and constraints can be modelled as nodal forces or constraints. Note however that, in continuum mechanics, solutions are singular under concentrated forces (e.g. &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_elasticity#Solutions_for_elastostatic_cases&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kelvin solution&lt;/a&gt;). This means that near the nodal loads and constraints great element distortion and no point-wise convergence is to be expected. Furthermore, due to numerical problems like hourglassing, a catastrophic breakdown of the solution can occur even far away from the concentrated load.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the engineering point of view, beam models and concentrated forces are OK, provided you understand what they really mean. According to de Saint Venant principle, you can assume that far away from a loaded region, only the load resultant influences the solution, and not the actual load distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you experience stresses and deformations in $ab$ I would tentatively suppose that&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;the beam is not slender enough to apply the de Saint Venant principle, or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;numerical problems (like hourglassing for reduced integration elements) are affecting your results.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-02-02T23:29:58.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10687" PostTypeId="1" CreationDate="2014-02-03T01:16:21.690" Score="1" ViewCount="676" Body="&lt;p&gt;Within Fenics what ordering is used for storing vectors as upper triangular matrices? Th:e question really is I define a green Lagrange strain tensor in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;V = VectorFunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;u  = Function(V)&#xA;I = Identity(V.cell().d)    # Identity tensor&#xA;F = I + grad(u)             # Deformation gradient&#xA;C = F.T*F                   # Right Cauchy-Green tensor&#xA;E = (C-I)/2                 #green strain tensor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But when I accesses the elements later,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;psi = stress1*E[0,0]+stress2*E[1,1]+stress3*(E[2,2])+stress4*E[1,2]+stress5*E[0,2]+stress6*E[0,1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assumed Voigt notation but it occurs to me that this may not be the case. Using this method to initialize the strain tensor is this the notation I should be using or is there a different type or initialization that would be more appropriate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help. &lt;/p&gt;&#xA;" OwnerUserId="4296" LastActivityDate="2014-02-04T02:56:12.367" Title="Tensor notation in fenics" Tags="|fenics|" AnswerCount="1" CommentCount="3" ClosedDate="2014-02-07T15:46:06.120" ContentLicense="CC BY-SA 3.0" />
  <row Id="10689" PostTypeId="1" CreationDate="2014-02-03T10:17:19.133" Score="4" ViewCount="152" Body="&lt;p&gt;Take sparse approximate inverse preconditioner $M \approx A^{-1}$ given by solution of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\underset{M \in S}{\mbox{min}} \; ||I - AM||_F,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $S$ is a set of sparse matrices and $||.||_F$ is the Frobenius norm (see [1] for details).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering which specific characteristics of the matrix would make this preconditioner very efficient? For instance, it is not difficult to see why Jacobi scaling is efficient for diagonally dominant matrices, SOR for tridiagonal matrices, multigrid for discretizated elliptic operators, etc. But I am lacking an insight into how the preconditioner above could implicitly or explicitly exploit any particular (perhaps structure-related?) characteristics of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://mathcs.emory.edu/~benzi/Web_papers/comp.pdf&quot; rel=&quot;nofollow&quot;&gt;http://mathcs.emory.edu/~benzi/Web_papers/comp.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="1089" LastEditDate="2014-02-03T15:03:51.667" LastActivityDate="2014-02-03T16:08:32.090" Title="What type of matrices is approximate inverse preconditioner $||I - AM||_F$ well suited for?" Tags="|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10690" PostTypeId="2" ParentId="10667" CreationDate="2014-02-03T14:43:48.363" Score="10" Body="&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; Use loops&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My numbers indicate that ifort is smart enough to recognize the loop, &lt;code&gt;forall&lt;/code&gt;, and &lt;code&gt;do concurrent&lt;/code&gt; identically and achieves what I'd expect to be about 'peak' in each of those cases.  gfortran, on the other hand, does a bad job (10x or more slower) with &lt;code&gt;forall&lt;/code&gt; and &lt;code&gt;do concurrent&lt;/code&gt;, especially as &lt;code&gt;N&lt;/code&gt; gets large.  Both ifort and gfortran seem to produce identical results for &lt;code&gt;forall&lt;/code&gt; and &lt;code&gt;do concurrent&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using MKL for BLAS with both &lt;code&gt;ifort&lt;/code&gt; and &lt;code&gt;gfortran&lt;/code&gt;.  gbmv achieves a steady 3x slower than 'peak'.  'scal' is close to peak for small problems, especially small &lt;code&gt;N&lt;/code&gt;, but quickly falls behind.  It is never worse than &lt;code&gt;gfortran&lt;/code&gt;'s forall and do concurrent, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On systems like mine (standard workstation configuration), it looks like loops are both the most robust and highest performance for all &lt;code&gt;N&lt;/code&gt;,&lt;code&gt;M&lt;/code&gt;.  &lt;code&gt;do concurrent&lt;/code&gt; has no advantage over &lt;code&gt;forall&lt;/code&gt;: both are bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find result tables and code &lt;a href=&quot;https://gist.github.com/maxhutch/8784895&quot;&gt;here&lt;/a&gt;.  Are results similar for IBM or PGI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, the performance as a multiple of the copy time &lt;code&gt;Y = X&lt;/code&gt; doesn't doesn't depend on &lt;code&gt;N&lt;/code&gt;.  I would have thought re-use of &lt;code&gt;D&lt;/code&gt; would have improved performance with higher &lt;code&gt;N&lt;/code&gt;, similar to GEMMs outperforming GEMVs.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ifort --version&#xA;ifort (IFORT) 14.0.1 20131008&#xA;&#xA;$ gfortran --version&#xA;GNU Fortran (Debian 4.7.2-5) 4.7.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="521" LastActivityDate="2014-02-03T14:43:48.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10691" PostTypeId="2" ParentId="10689" CreationDate="2014-02-03T16:08:32.090" Score="4" Body="&lt;p&gt;I'm not an expert on this topic, but I have read a few papers concerning this in the hopes of finding a useful preconditioner for my problem. I think though that understanding of this topic is rather incomplete. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some work in the direction of proving effectiveness of sparse approximate inverse preconditioners aims to show that off diagonals of (exact) inverse matrices have some kind of rapid decay property. Theorems like this effectively say that the inverse of said matrix has the potential to be approximated sparsely. See for example &lt;a href=&quot;http://www.mathcs.emory.edu/~benzi/Web_papers/BITBenziGolub.pdf&quot; rel=&quot;nofollow&quot; title=&quot;This paper&quot;&gt;This paper&lt;/a&gt; and the citations therein.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2014-02-03T16:08:32.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10692" PostTypeId="1" AcceptedAnswerId="26638" CreationDate="2014-02-03T16:37:31.577" Score="8" ViewCount="1606" Body="&lt;p&gt;I'm attempting to validate my FDTD code against Meep by calculating the Poynting vector field across a simulation consisting of a monochromatic point source within a box (no boundary conditions (Dirichlet) so everything just bounces off the domain boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the simulation is 2D TMz (point source in $E{z}$, coupling into $H{x}$ and $H{y}$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After quite a bit of research and talking with my thesis advisor, I wound up using this for Poynting vector $\hat{P}$ calculation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\hat{E} \times \hat{H}^*$ (complex conjugate is the same as the original $H$ vector in this case since everything is in the real domain).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\begin{bmatrix}&#xA;  \hat{P}_X\\&#xA;  \hat{P}_Y\\&#xA;  \hat{P}_Z&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;E_YH_Z - E_zH_Y\\&#xA;E_zH_X - E_XH_Z\\&#xA;E_XH_Y - E_YH_X\\&#xA;\end{bmatrix}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that &#xA;$\hat{E} = \begin{bmatrix}0 &amp;amp; 0 &amp;amp; E_Z\end{bmatrix}$&#xA;and &#xA;$\hat{H} = \begin{bmatrix}H_X &amp;amp; H_Y &amp;amp; 0\end{bmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(since there are no $E_X$, $E_Y$, or $H_Z$ components in a 2D TMz sim)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\begin{bmatrix}&#xA;  \hat{P}_X\\&#xA;  \hat{P}_Y\\&#xA;  \hat{P}_Z&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;- E_ZH_Y\\&#xA;E_ZH_X\\&#xA;0&#xA;\end{bmatrix}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When calculating the Poynting vector field, I iterate through the entire $E_Z$ array, and use the above formula for obtaining the $P$ vector at each cell. Then, basic algebra to get the magnitude and direction:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|P| = \sqrt{P_X^2 + P_Y^2 + P_Z^2}&#xA;\\&#xA;\\&#xA;P_{normal} = {P \over{|P|}}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, plotting the magnitude before the source wave hits the boundary of the simulation ($E_Z$ pictured below),&#xA;&lt;img src=&quot;https://i.stack.imgur.com/9SFF5.png&quot; alt=&quot;Ez just before the wave hits the boundary&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect the Poynting vector magnitude field to resemble a cone, indicating that the magnitude falls off with $1\over{R^2}$ distance $R$ from the source position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm seeing this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/dJDVc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my question is: does my math look right? Or, am I missing something here? Any suggestions or links to example code are welcome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even a picture of what the Poynting magnitude and direction field should look like for a monochromatic point source would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="7246" LastEditorUserId="7246" LastEditDate="2017-04-15T03:09:21.663" LastActivityDate="2017-04-15T03:09:21.663" Title="FDTD Poynting Vector" Tags="|finite-difference|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10693" PostTypeId="2" ParentId="10678" CreationDate="2014-02-03T19:06:36.040" Score="2" Body="&lt;p&gt;My take is in lecture 4 at &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; . In particular, it gives you an idea of &lt;em&gt;why&lt;/em&gt; we choose the hat functions we usually use when we use the finite element method -- namely, because they lead to the important concept of sparsity, even though many other choices of basis functions would have been equally valid.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-03T19:06:36.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10694" PostTypeId="1" CreationDate="2014-02-01T03:53:34.437" Score="2" ViewCount="1814" Body="&lt;p&gt;I am working on a project that I need to add a regularization into the NNLS algorithm.&#xA;Is there a way to add the Tikhonov regularization into the NNLS implementation of scipy [1]?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] talks about it, but does not show any implementation. Sklearn has an implementation, but it is not applied to nnls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.nnls.html&quot; rel=&quot;nofollow&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.nnls.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://icses2012.pwr.wroc.pl/article/34.pdf&quot; rel=&quot;nofollow&quot;&gt;http://icses2012.pwr.wroc.pl/article/34.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7236" OwnerDisplayName="user3259573" LastActivityDate="2014-02-04T00:07:31.500" Title="Tikhonov regularization in the non-negative least square" Tags="|python|scipy|" AnswerCount="1" CommentCount="0" ClosedDate="2014-02-08T16:42:17.580" ContentLicense="CC BY-SA 3.0" />
  <row Id="10695" PostTypeId="2" ParentId="10694" CreationDate="2014-02-03T01:30:08.680" Score="1" Body="&lt;p&gt;Still use &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;scipy.optimize.nnls(A1, b1)&lt;/code&gt;, where &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A1 = np.concatenate((A, lambda*np.matlib.identity(n)))&#xA;b1 = np.concatenate((b, numpy.zeros(shape=(n,1))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7251" OwnerDisplayName="lennon310" LastActivityDate="2014-02-03T01:30:08.680" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10696" PostTypeId="2" ParentId="8942" CreationDate="2014-02-04T00:29:10.847" Score="7" Body="&lt;p&gt;If analytic techniques are disallowed but the periodic structure is known, here is one approach.  Let&#xA;$$g(x) = \frac{\cos x}{2-\cos x}$$&#xA;be periodic with period $2 \pi$, so that&#xA;$$g(x) = \sum_j w_j e^{ijx}$$&#xA;where&#xA;$$w_j = \frac{1}{2\pi}\int_0^{2\pi} g(x) e^{-ijx} dx$$&#xA;Thus,&#xA;$$\begin{aligned}&#xA;f(x) &amp;amp;= \sum_{k \ge 1} \frac{g(kx)}{k^p} \\&#xA;     &amp;amp;= \sum_{k \ge 1} \frac{1}{k^p} \sum_j w_j e^{ijkx} \\&#xA;     &amp;amp;= \sum_j w_j \sum_{k \ge 1} \frac{\left(e^{ijx}\right)^k}{k^p} \\&#xA;     &amp;amp;= \sum_j w_j \operatorname{Li}_p(e^{ijx})&#xA;\end{aligned}$$&#xA;You can either approximate the integrals $w_j$ directly or compute a bunch of $f(x)$ values and use a DFT.  In either case, you can potentially apply Richardson extrapolation to the result.  Since in your case $g(x)$ is analytic within a neighborhood of $\mathbb{R}$, the final series converges exponentially even without Richardson.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2014-12-20T19:13:44.403" LastActivityDate="2014-12-20T19:13:44.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10697" PostTypeId="1" AcceptedAnswerId="10698" CreationDate="2014-02-04T01:57:25.433" Score="4" ViewCount="191" Body="&lt;p&gt;I know that the Tikhonov regularization of a linear system has an analytical solution given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\hat{\mathbf{x}} = \mathrm{arg\;min}\left( \left| \mathbf{Ax} - \mathbf{b} \right|^{2} + \left| \mathbf{\Gamma x} \right|^{2} \right) = \left( \mathbf{A}^{\top}\mathbf{A} + \mathbf{\Gamma}^{\top}\mathbf{\Gamma} \right)^{-1} \mathbf{A}^{\top} \mathbf{b}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to extend this idea to obtain an analytical solution for a system with an additional weighting? i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\hat{\mathbf{x}} = \mathrm{arg\;min}\left( \left| \mathbf{Ax} - \mathbf{b} \right|^{2} + \left| \mathbf{\Gamma x} \right|^{2} + \left| \mathbf{U  x} \right|^{2} \right) = \; ?&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help!&lt;/p&gt;&#xA;" OwnerUserId="6700" LastActivityDate="2014-02-04T15:28:34.307" Title="Question about extending Tikhonov regularization" Tags="|linear-algebra|optimization|linear-solver|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10698" PostTypeId="2" ParentId="10697" CreationDate="2014-02-04T02:24:11.227" Score="5" Body="&lt;p&gt;Yes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to the Tikhonov regularization is derived by computing the first order optimality condition of functional to be minimized and then inverting the derived linear system. Applying the same process for your functional, let:&#xA;$$&#xA;I(\mathbf{x}) = \left|\mathbf{Ax} - \mathbf{b}\right|^2 + \left|\mathbf{\Gamma x}\right|^2 + \left|\mathbf{Ux}\right|^2&#xA;$$&#xA;Then, taking the derivative w.r.t to $\mathbf{x}$ and setting this equal to zero, we get:&#xA;$$&#xA;\frac{\partial I}{\partial\mathbf{x}}= 0 = 2 \left[\mathbf{A}^{T}\left(\mathbf{Ax}-\mathbf{b}\right) + \mathbf{\Gamma}^T\mathbf{\Gamma x} + \mathbf{U}^{T} \mathbf{Ux} \right]&#xA;$$&#xA;Rearranging and solving for $\mathbf{x}$, gives:&#xA;$$&#xA;\mathbf{x}=\left(\mathbf{A}^{T}\mathbf{A} + \mathbf{\Gamma}^T\mathbf{\Gamma} + \mathbf{U}^{T} \mathbf{U}\right)^{-1} \mathbf{A}^{T} \mathbf{b}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2014-02-04T15:28:34.307" LastActivityDate="2014-02-04T15:28:34.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10699" PostTypeId="2" ParentId="10687" CreationDate="2014-02-04T02:56:12.367" Score="0" Body="&lt;p&gt;Adding the answer here in case others stumble on it. Answer is from Anders Logg, a fenics dev&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your definition of the tensors F, C, E is correct. FEniCS does not know anything about Voigt notation, so you need to access the components directly as E[0, 0] etc.&lt;/p&gt;&#xA;" OwnerUserId="4296" LastActivityDate="2014-02-04T02:56:12.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10700" PostTypeId="1" CreationDate="2014-01-31T02:44:18.690" Score="5" ViewCount="7678" Body="&lt;p&gt;I'm a newbie at modelling and optimization of LPs. My research problem concerns assigning tasks/jobs to virtual machines in a data centre depending on the least energy cost incurred. As you can imagine this is a very large problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My primary question is: Which algorithm (interior point, genetic algorithm....etc) would be efficient in solving very large LPs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was initially using CPLEX as a solver and quickly realised that it was inefficient on very large data. So now I'm planning to solve the LP using C++. Is this wise? Are there any problems that I might encounter if using C++ to solve the LP?&lt;/p&gt;&#xA;" OwnerDisplayName="user3255728" LastActivityDate="2016-04-07T18:18:31.480" Title="Which algorithm to use for solving an LP with a very large number of variables?" Tags="|c++|linear-programming|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10701" PostTypeId="2" ParentId="10700" CreationDate="2014-01-31T03:50:17.040" Score="8" Body="&lt;p&gt;CPLEX is commercial grade and solves very large LPs and IPs. If CPLEX didn't pan out for you, then switching to a different solver may not be the answer.&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/502102/best-open-source-mixed-integer-optimization-solver?rq=1&quot;&gt;Here's one SO Question on good solvers.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, here are three suggestions for you, rather than going for the solver speed or the specific algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Focus on your formulation. How many constraints and variables do you have?&#xA;    Are you modeling each of task/jobs to be assigned and each one of the virtual machines?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, do have binary variables of the type &lt;code&gt;X_ij&lt;/code&gt; where i is the set of jobs and j is the set of virtual machines? If you make the IP very large, then every solver out there will choke. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Formulation related suggestions&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Groups&lt;/strong&gt;: Try to make groups of jobs/tasks, and create groups of virtual machines. Even if it is just to get an initial feasible solution,&#xA;make groups of 100 or even 1000 and that will significantly reduce your problem size. Now, the solver should be able to solve your problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Hierarchical problems&lt;/strong&gt;: This is similar to grouping. Instead of one LP, you solve multiple ones. Make a hierarchy of &#xA;machines and big groups of jobs. Once you have a solution for this higher level LP, you can then solve sub-problems at higher levels &#xA;of granularity. This is a very typical approach when &quot;good&quot; solutions are acceptable. It would be a mistake to go&#xA;for the absolute lowest electricity costs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Problem Relaxations&lt;/strong&gt;: Look into simplifying or decomposing your problem. Compare your formulation to textbook formulations of &quot;&lt;a href=&quot;http://en.wikipedia.org/wiki/Assignment_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;assignment problems&lt;/a&gt;.&quot;&#xA;It is quite possible that in your formulation, there is a nice underlying structure, &#xA;but a few dense constraints are giving the solver trouble. &#xA;If you can identify these constraints, you can &quot;relax&quot; those, and maybe even throw it into the objective function with a multiplier. One option is to consider Lagrangian relaxations for 'troublesome constraints.' &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Try some of these and see if you can make progress.&lt;/p&gt;&#xA;" OwnerUserId="4589" OwnerDisplayName="Ram Narasimhan" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-01-31T07:12:34.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10702" PostTypeId="2" ParentId="10700" CreationDate="2014-01-31T15:01:41.850" Score="4" Body="&lt;p&gt;The big solvers like CPLEX / GUROBI are usually quite good, especially concerning linear programs and not integer linear programs, however they have natural difficulties beyond a certain problem size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tackling the problem with c++ alone will probably not help, since I doubt that it is to easy to beat CPLEX implementations. CLPEX itself provides a c++ api, but it wont help you much on the inbound LP solver, it is rather for the &quot;integer&quot; part (though it can help a lot there if you problem is integer by custom heuristics/cutting-planes)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your (integer) linear problem's size, informally amount of columns / rows, makes up for most of the difficulty, you can try column-generation or cutting plane algorithms, however they are not trivial to implement. I am not sure whether the current versions of GUROBI/ CPLEX support these algorithms, but another solver called SCIP (as well by its c++ api) provides opportunities to do so.  &lt;/p&gt;&#xA;" OwnerUserId="772" OwnerDisplayName="Martin" LastActivityDate="2014-01-31T15:01:41.850" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10704" PostTypeId="1" CreationDate="2014-02-04T12:13:01.817" Score="5" ViewCount="534" Body="&lt;p&gt;I have question connected with finite volume method. Consider equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t}=\operatorname{div}A\nabla u +f . \quad x\in \overline{B}_{1} (0)\subset \mathbb{R}^3 -\text{unit ball with its boundary}$$ I solved IBVP for this equation with a help of finite volume method, for grid I took uniform spherical coordinates grid,. &#xA;(picture lower is related to topic)&#xA;&lt;img src=&quot;https://i.stack.imgur.com/hU2Id.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a problem with finding flows for the faces of cell: how to approximate the value&#xA;$$\oint_{\partial V_{i,j,k}}(A\nabla u, n)dS $$&#xA;(where $A$ is a variable matrix)? When $A$ is an identity matrix everything is clear. Could you help me with this issue or to advise literature where this issue is represented?&lt;/p&gt;&#xA;" OwnerUserId="4676" LastActivityDate="2014-02-04T12:13:01.817" Title="Finite volume method" Tags="|finite-volume|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10705" PostTypeId="2" ParentId="10656" CreationDate="2014-02-04T19:09:24.530" Score="0" Body="&lt;p&gt;The centered difference scheme should be stable for implicit Euler, so I'd try that first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[deleted, misread question]&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2014-02-04T21:00:53.103" LastActivityDate="2014-02-04T21:00:53.103" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10706" PostTypeId="2" ParentId="10700" CreationDate="2014-02-04T19:42:35.083" Score="3" Body="&lt;h1&gt;A brief word on solvers&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Don't roll your own code to start out. People have been working on this problem for decades, and have developed high-quality solvers. Of these, CPLEX and Gurobi are the state-of-the-art. If you decide you must roll your own code (perhaps you need certain features existing solvers don't provide, or you really want to write a solver yourself), consider contributing to a community project like the COIN-OR solvers. For people who can't use CPLEX or Gurobi (say, due to cost, or licensing restrictions), the COIN-OR solvers are a good no-cost alternative, and they're always looking for solver improvements. Much of the advice below is geared towards CPLEX and Gurobi, although a lot of it applies to the COIN-OR solvers as well, with the caveats that the COIN-OR solvers tend to be slower, have worse heuristics, and have fewer features.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Methods you could use to speed up the solution of a large-scale LP&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Assuming you have a large LP with no integer variables the main strategies typically employed are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;change your formulation&lt;/li&gt;&#xA;&lt;li&gt;interior point methods&lt;/li&gt;&#xA;&lt;li&gt;delayed column generation (e.g., Dantzig-Wolfe decomposition)&lt;/li&gt;&#xA;&lt;li&gt;delayed row generation (e.g., Benders' decomposition, generalized Benders' decomposition)&lt;/li&gt;&#xA;&lt;li&gt;Lagrangian relaxation (as mentioned by RamNarasimhan)&lt;/li&gt;&#xA;&lt;li&gt;exploit parallelism&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Change your formulation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I originally put this section last, but it's probably best to put it first. If you haven't explored different methods for modeling your problem, you should, and you should follow RamNarasimhan's advice on grouping and hierarchical formulations. These methods are good exercises anyway, because they will help you understand tradeoffs you're making between mathematical modeling and time-to-solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you settle on a level of modeling detail, it's still worth structuring a large-scale formulation to take advantage of the other methods. Interior point methods are suited to large-scale, sparse problems. If your problem is large-scale and dense, it is less helpful, and your problem may still be intractable. Similarly, delayed column generation works best if you have sets of mostly uncoupled variables, coupled by a small number of constraints. Lagrangian relaxation works best if you can identify a small number of difficult constraints.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Interior point methods&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;CPLEX and Gurobi both have heuristics that select either an interior-point method or a simplex method based on the data supplied to its solvers. These libraries can also switch between the two via &quot;cross-over&quot; steps. Without any information on how large your problem is (number of variables, number of constraints) or any information on exploitable problem structure, I can only assume that CPLEX and Gurobi would use interior point methods automatically.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Delayed column/row generation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Delayed column generation and delayed row generation are both methods that are not generally implemented in CPLEX or Gurobi automatically because they require manual intervention to decompose your problem into a master problem and smaller subproblems. There are examples of how to implement them in CPLEX (and probably Gurobi) available online. The one exception I can think of is cutting plane generation for mixed-integer linear programs; in that case, CPLEX and Gurobi will both generate Gomory cuts, lift-and-project cuts, and so on, using heuristics (and if you know something about the structure of your problem, user intervention).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Lagrangian relaxation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As noted by RamNarasimhan, Lagrangian relaxation works by dualizing a subset of constraints known to be difficult to solve, so it comes up more in the context of mixed-integer and nonlinear programming than it does in linear programming.  It has been used effectively to solve large-scale linear programs also.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Parallelism&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you have access to machines with many cores, or a number of machines with fast interconnects, you might be able to parallelize your problem, if it is large enough. CPLEX and Gurobi both have facilities for exploiting parallelism, although I don't know in detail what algorithms it uses. The impression I get is that there is a lot of room for improvement in parallelizing optimization algorithms, so I would suggest exhausting all other avenues before looking to speed up the solution of your problem via parallelism.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-02-04T19:58:54.227" LastActivityDate="2014-02-04T19:58:54.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10707" PostTypeId="1" AcceptedAnswerId="10709" CreationDate="2014-02-05T05:25:49.023" Score="7" ViewCount="181" Body="&lt;p&gt;The non-linear Poisson equation in one-dimension,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;0 =  \frac{\partial^2u}{\partial x^2} - f(u)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can be discretised as to give,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_{j-1} -2u_{j} + u_{j+1} = h^2 f(u_j)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h$ is the step size of the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any advantage (in general or not) to write the non-linear source term as the average value of the solution variable $\bar{u_j}$ over the neighbouring mesh points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(u_j) \rightarrow f(\bar{u_j})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\bar{u_j} = \frac{1}{2}\left( u_{j-1} + u_{j+1} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have noticed in practice that this sometimes improves the solution stability when using &lt;a href=&quot;http://en.wikipedia.org/wiki/Relaxation_%28iterative_method%29&quot;&gt;relaxation methods&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2014-02-05T06:35:40.793" LastActivityDate="2014-02-16T20:37:15.347" Title="For a non-linear PDEs should the source term be discretised at $u_j$ or averaged over $(u_{j+1} + u_{j-1})/2$?" Tags="|numerical-analysis|nonlinear-equations|poisson|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10708" PostTypeId="1" AcceptedAnswerId="30530" CreationDate="2014-02-05T08:29:46.957" Score="1" ViewCount="1334" Body="&lt;p&gt;[Edited to simplify the question]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to associate the eigenvalues $E$ of a matrix $H$ to the original rows of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, it would be trivial to sort the eigenvalues in ascending order, however this throws away some of the information. For my purpose the matrix is a Hamiltonian where the different rows have physical meaning. I &lt;em&gt;always&lt;/em&gt; want the eigenvalue that corresponds to a given row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to &lt;strong&gt;sort the eigenvalues according to the dominant contribution&lt;/strong&gt;. To do this I get the magnitudes of the eigenvectors (they are complex) for each row and pick the largest. I use this index to sort the eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Get eigenvalues and vector of the Hamiltonian&#xA;E, V = eig(H) &#xA;&#xA;# Now sort the eigenvalues by dominant contribution&#xA;sorted_order = []&#xA;for i in range(len(E)):&#xA;    row = V[i,:]&#xA;    row = np.absolute(row)           # length of complex number &amp;lt;-- correct?&#xA;    largest_index = np.argmax(row)   # the index of the largest value&#xA;    sorted_order.append(largest_index)&#xA;&#xA;E_sorted = E[sorted_order]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The sorting approach works fairly well when the eigenvector matrix is dominated by the diagonal. However, if the matrix is more homogeneous the this sorting method fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads me to believe that my method of sorting is probably incorrect. Or I am running in to accuracy issues with the &lt;code&gt;eig&lt;/code&gt; function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does anyone have suggestion on an alternative way to sort by the dominant contribution?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am doing a $\mathbf{k}\cdot\mathbf{p}$ simulation for semiconductor band structure. The eigenvalues for the  $\mathbf{k}\cdot\mathbf{p}$ Hamiltonian (the 8x8 matrix below) corresponds to the different semiconductor bands and the eigenvectors are the wavefunctions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/9IpM1.png&quot; alt=&quot;10 band Hamiltonian, &amp;lt;code&amp;gt;http://dx.doi.org/10.1103/PhysRevB.73.125348&amp;lt;/code&amp;gt;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;NB&lt;/strong&gt; I have removed two rows and columns because I am not interested in their effect for by purpose.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tOO6R.png&quot; alt=&quot;Sorting errors.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Figure:&lt;/strong&gt; &lt;em&gt;In the plot the y-axis is the eigenvalues, the x-axis corresponds to a k-space vector which changes values in the Hamiltonian. A definition: at $k=0.5$ the system is unperturbed and the eigenvectors are dominated by the diagonal terms. This also illustrates why I cannot do a simple energy sort: because at some points the bands cross. The bands have different physical meaning which I want to preserve.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Comment.&lt;/strong&gt; The sorting approach (the Python code above) works fairly well when $k=0.5$ because the contribution is dominated by the diagonal of the eigenvectors. However, moving away from the centre of k-space ($k&amp;gt;0.5$ and $k&amp;lt;0.5$) the eigenvector matrix becomes more homogeneous and this method of sorting fails. I am surprised that it is failing intermittently.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2014-02-05T12:50:04.543" LastActivityDate="2018-11-12T21:47:00.030" Title="Sorting eigenvalues by the dominant contribution" Tags="|simulation|eigensystem|eigenvalues|sorting|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10709" PostTypeId="2" ParentId="10707" CreationDate="2014-02-05T08:48:55.287" Score="7" Body="&lt;p&gt;For your example equation, taking the average approach, the local consistency error &#xA;$$&#xA;\frac{1}{h^2}[u(x-h) - 2 u(x) + u(x+h)]-f(\frac{1}{2}[u(x-h)+u(x+h)]) = \frac{1}{2}f_uu_{xx}h^2 + hot.&#xA;$$&#xA;will be of order $2$ (instead of order $3$). ($hot.$ means higher order terms)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, if your overall approximation is of order $1$, e.g. if you use upwind somewhere, then it doesn't matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use higher order schemes, then this approach will limit of your convergence rate.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2014-02-06T09:06:45.373" LastActivityDate="2014-02-06T09:06:45.373" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10710" PostTypeId="1" CreationDate="2014-02-04T10:56:11.920" Score="2" ViewCount="338" Body="&lt;p&gt;I would like to normalize a quantum mechanical multi-particle wave function numerically, and since the result is a multidimensional integral I thought Monte Carlo methods might be appropriate. So, I'm looking for $\mathcal{N}$ where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\psi=\frac{1}{\sqrt{\mathcal{N}}}\tilde{\psi}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\psi$ and $\tilde{\psi}$ are the normalized and unnormalized wave function respectively. So, introducing $\langle f(\textbf{x})\rangle$ as the expectation value of $f$ in the normalized state $\psi$, with position coordinates $\textbf{x}$ (a vector containing all components of position vectors for all particles), I find the Monte Carlo estimate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{\mathcal{N}}=\frac{1}{\mathcal{N}A^N_d}\int d^d\textbf{x}=\frac{1}{A^N_d}\int d^d\textbf{x}\big|\frac{\psi(\textbf{x})}{\tilde{\psi}(\textbf{x})}\big|^2\approx\frac{1}{A^N_dn}\sum_{j=1}^n\frac{1}{|\tilde{\psi}(\textbf{x}_j)|^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_d$ is the (finite) $d$-dimensional area of the system and $\textbf{x}_j$ are $n$ position configurations drawn from the (normalized) probability density $|\psi|^2$. I'm using the Metropolis algorithm, which automatically uses a normalized probability, so that I have no choice in that matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I try to do this I get an estimate for $\frac{1}{\mathcal{N}}$ which &lt;em&gt;increases&lt;/em&gt; with number of Monte Carlo samples $n$. And the strange thing is, so does the standard deviation, which one expects to have a behaviour $\sim n^{-1/2}$ in a Monte Carlo. I've checked this up to millions of configurations, for a wave function $\psi$ that is expected to converge relatively fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone find some mistake in my logic? Or does someone have a tip for a better way to calculate the normalization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: In reply to a comment by Isidore Seville I'll explain why I need the normalization of $\tilde{\psi}$ in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My wave function is itself given as an integral over auxiliary coordinates $\textbf{y}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{\psi}(\textbf{x})=\int d^d\textbf{y}\phi_1(\textbf{y})\phi_2(\textbf{y},\textbf{x})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When performing a Metropolis Monte Carlo using $|\tilde{\psi}(\textbf{x})|^2$ as the probability density I basically have two possibilities: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For every random move in the particle coordinates $\textbf{x}$, perform a MC integration    in $\textbf{y}$ to get the resulting $|\tilde{\psi}(\textbf{x})|^2$&lt;/li&gt;&#xA;&lt;li&gt;Introduce another set of auxiliary coordinates $\textbf{z}$ and use $|\tilde{\psi}(\textbf{x})|^2=\int d^d\textbf{y}d^d\textbf{z}\  \phi_1(\textbf{y})^*\phi_1(\textbf{z})\phi_2(\textbf{y},\textbf{x})^*\phi_2(\textbf{z},\textbf{x})$. Then treat the MC as an integral over all three sets of coordinates, randomly moving $\textbf{x}$, $\textbf{y}$ and $\textbf{z}$ for every iteration.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For now I decided to go for the second possibility, which for e.g. the expectation value of a function $f(\textbf{x})$ thus entails calculating&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\langle f(\textbf{x})\rangle=\int d^d\textbf{x}f(\textbf{x})|\tilde{\psi}(\textbf{x})|^2=\int d^d\textbf{x}d^d\textbf{y}d^d\textbf{z}\ f(\textbf{x})\phi_1(\textbf{y})^*\phi_1(\textbf{z})\phi_2(\textbf{y},\textbf{x})^*\phi_2(\textbf{z},\textbf{x})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this introduces a new problem: although of course $|\tilde{\psi}(\textbf{x})|^2$ is real, the integrand above isn't necessarily until the integrals over $\textbf{y}$ and $\textbf{z}$ converge. Therefore I have to use another probability density $P(\textbf{x},\textbf{y},\textbf{z})$ that is real and nonnegative to find the MC estimate (using some shorthand below):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\langle f(\textbf{x})\rangle=\int d\Omega\ f\phi_1^*\phi_1\phi_2^*\phi_2\frac{P}{P}\approx\frac{1}{n}\sum_{i=1}^N\frac{f\phi_1^*\phi_1\phi_2^*\phi_2}{P}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the summand above is evaluated on configurations drawn from the probability $P$ ($n$ in total). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this is the situation: To evaluate the summand I need the normalized $\phi_1$, $\phi_2$ and $P$ (the latter since there is no way to do the Metropolis MC with an unnormalized $P$). Also, there is no way to just do the integral using $|\tilde{\psi}|^2$ in the normal way, since this itself includes an integral.&lt;/p&gt;&#xA;" OwnerUserId="7265" OwnerDisplayName="jorgen" LastEditorUserId="7265" LastEditDate="2014-02-06T08:24:06.390" LastActivityDate="2014-03-08T08:56:49.370" Title="Monte Carlo normalization of a wave function" Tags="|quantum-mechanics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10711" PostTypeId="2" ParentId="10710" CreationDate="2014-02-05T14:01:05.260" Score="1" Body="&lt;p&gt;My understanding is that, you have computed the energy of a given wave function by doing the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E[\phi] = \langle \phi|H|\phi\rangle = \frac{\int E(x) |\phi|^2 dx}{\int |\phi|^2 dx} = \frac{\int E(x) (|\phi|^2/|\psi|^2) |\psi|^2 dx}{\int |\psi|^2 dx}\times \frac{\int |\psi|^2 dx}{\int |\phi|^2 dx}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the following quantity is measured in MC:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A = \frac{\int E(x) (|\phi|^2/|\psi|^2) |\psi|^2 dx}{\int |\psi|^2 dx},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by treating $|\psi|^2$ as unnormalized probability and $E(x) (|\phi|^2/|\psi|^2)$ as a variable. Now we need the value of the following quantity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;B = \frac{\int |\psi|^2 dx}{\int |\phi|^2 dx} = \frac{\int (|\psi|^2/|\phi|^2) |\phi|^2 dx}{\int |\phi|^2 dx},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can measure $B$ by treating $|\psi|^2/|\phi|^2$ as a variable. To make it work, the reference state $\psi$ and the state $\phi$ should be &quot;reasonably close&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope the above helps.&lt;/p&gt;&#xA;" OwnerUserId="7264" LastActivityDate="2014-02-05T14:01:05.260" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10712" PostTypeId="1" CreationDate="2014-02-05T17:51:05.977" Score="3" ViewCount="43" Body="&lt;p&gt;I have a set of points (2D space), and for every point there's a label (like city names on a map).&#xA;I want to find a real-time algorithm that allows labels to avoid overlapping, moving them from their original position if necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard about simulated annealing algorithm, but I can't find a good source to learn how to do this. Do you have some idea where I can find bibliography and practical examples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for your replies.&lt;/p&gt;&#xA;" OwnerUserId="7267" LastActivityDate="2014-02-05T17:51:05.977" Title="Auto labeling algorithm" Tags="|computational-geometry|" AnswerCount="0" CommentCount="2" ClosedDate="2014-04-30T01:21:51.730" ContentLicense="CC BY-SA 3.0" />
  <row Id="10714" PostTypeId="2" ParentId="8073" CreationDate="2014-02-05T18:56:07.810" Score="2" Body="&lt;p&gt;Since your matrices are tridiagonal, you could use an eigenvalue algorithm specifically designed for tridiagonal matrices. Here is one you could try (I haven't tried it though), &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/24068-tridiegs&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;trideigs&lt;/code&gt; by Vasil Yordanov&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="7261" LastActivityDate="2014-02-05T18:56:07.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10715" PostTypeId="1" CreationDate="2014-02-05T21:59:07.997" Score="1" ViewCount="535" Body="&lt;p&gt;I have a question which I think should be pretty simple. I have written a subroutine in &lt;code&gt;Fortran&lt;/code&gt; which will compute the Gradient at a point of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;subroutine get_gradient(xyz, Grad)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;xyz&lt;/code&gt; is a &lt;code&gt;3x1&lt;/code&gt; array and &lt;code&gt;Grad&lt;/code&gt; is also a &lt;code&gt;3x1&lt;/code&gt; array. The gradient method is a fourth order accurate method where the step is chosen inside the algorithm. Based on this, is there a simple way to use it to then turn around and generate the Hessian at xyz?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i = 1,3&#xA;  for j = 1,3&#xA;    txyz = xyz&#xA;    xyz(i) = xyz(i) + h&#xA;    xyz(j) = xyz(j) + h&#xA;    call get_gradient(xyz, Grad)&#xA;    Hess(i,:) = Hess(i,:) + Grad&#xA;    Hess(:,j) = Hess(:,j) + Grad&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know the previous loop will not give the Hessian, but the idea is there - use the gradient method to construct the Hessian. Is there a simple algorithm which will do this?&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2014-02-06T00:43:05.210" Title="How to design a Hessian Algorithm based on Gradient Algorithm?" Tags="|fortran|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10716" PostTypeId="2" ParentId="10715" CreationDate="2014-02-05T22:12:38.590" Score="1" Body="&lt;p&gt;Hessian is in simple terms the Jacobian of the gradient. So if you compute the Jacobian as in:&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;form the gradient, you would obtain Hessian Matrix. While it may not be the best way to compute Hessian, it might be sufficient to do it.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-02-05T22:12:38.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10717" PostTypeId="2" ParentId="10715" CreationDate="2014-02-06T00:43:05.210" Score="1" Body="&lt;p&gt;As it turns out the answer is quite simple. In this example, &lt;code&gt;f1&lt;/code&gt;, &lt;code&gt;f2&lt;/code&gt;, &lt;code&gt;f3&lt;/code&gt;, &lt;code&gt;f4&lt;/code&gt; are all &lt;code&gt;3x1&lt;/code&gt; arrays.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  do ixyz = 1,3&#xA;    ! x+2h&#xA;    txyz = xyz&#xA;    txyz(ixyz) = txyz(ixyz) + two*h&#xA;    call get_grad(txyz, f1)&#xA;    ! x+h&#xA;    txyz = xyz&#xA;    txyz(ixyz) = txyz(ixyz) + h&#xA;    call get_grad(txyz, f2)&#xA;    ! x-h&#xA;    txyz = xyz&#xA;    txyz(ixyz) = txyz(ixyz) - h&#xA;    call get_grad(txyz, f3)&#xA;    ! x-2h&#xA;    txyz = xyz&#xA;    txyz(ixyz) = txyz(ixyz) - two*h&#xA;    call get_grad(txyz, f4)&#xA;    HessianRho(ixyz,:) = HessianRho(ixyz,:) + (-f1 + 8.0_rkind*(f2-f3) + f4)/(h*12.0_rkind)&#xA;    HessianRho(:,ixyz) = HessianRho(:,ixyz) + (-f1 + 8.0_rkind*(f2-f3) + f4)/(h*12.0_rkind)&#xA;  enddo&#xA;  HessianRho = HessianRho/two&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will generate the Hessian in a symmetric manner (which is why we do &lt;code&gt;(ixyz,:)&lt;/code&gt; as well as &lt;code&gt;(:,ixyz)&lt;/code&gt; and then divide by 2)&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2014-02-06T00:43:05.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10718" PostTypeId="1" AcceptedAnswerId="10731" CreationDate="2014-02-06T02:54:28.560" Score="5" ViewCount="505" Body="&lt;p&gt;Suppose I have a symmetric matrix $A_{1000\times 1000}$, which can be represented by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A = J G J^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $J$ in 1000x3 is full column rank dense matrix; $G$ in 3x3 is a nonsingular dense matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the fastest way to obtain ONLY the maximum eigenvalue of $A$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the eigenvalue problem of symmetric matrix can be faster than that of general dense matrix, but can the following features of the problem make it even faster ?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;only the maximum eigenvalue of $A$ is needed;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$A = J G J^T$, rank($A$) = 3, and $A$ has only 3 nonzero eigenvalues&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Can $LDL^T$ decomposition work any good? I would prefer to implement it via Eigen C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does $B=J^TJG$ has the same nonzero eigenvalues as $A$?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6693" LastEditDate="2014-02-06T08:41:13.060" LastActivityDate="2014-02-06T22:40:12.510" Title="What is the most efficient way to obtain the max eigenvalue of a specific symmetric matrix via Eigen C++" Tags="|c++|matrix|eigenvalues|eigen|symmetry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10719" PostTypeId="1" CreationDate="2014-02-06T08:31:19.903" Score="1" ViewCount="294" Body="&lt;p&gt;How is it possible to do something once every solver iteration?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expressions when &lt;code&gt;change(time)&lt;/code&gt; or when &lt;code&gt;time &amp;gt; pre(time)&lt;/code&gt; are not permitted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards,&#xA;Koldo&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The final goal is to do a cosimulation from Modelica with other tool that does not have FMI and where the Modelica side signals the iteration delta time.&#xA;- Approach 1: Is there any way to fire a 'when' under an algorithm section just before or after an iteration step has been successfully finished? - Approach 2: I wonder if there is something similar to fmiEventUpdate(). It appears in the automatically generated .mo file when doing FMI, and &quot;is called after a time, state or step event occurred ... the function returns once a new consistent state has been found&quot;.&lt;/p&gt;&#xA;" OwnerUserId="4607" LastEditorUserId="4607" LastEditDate="2014-02-09T13:40:45.417" LastActivityDate="2014-02-09T13:40:45.417" Title="How to do something in Modelica every solver iteration?" Tags="|openmodelica|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10720" PostTypeId="1" AcceptedAnswerId="10745" CreationDate="2014-02-06T09:00:59.793" Score="1" ViewCount="1516" Body="&lt;p&gt;If $A\in R^{m\times n}, b\in R^m, c\in R^n$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if I need to solve the least square problems via SVD of $A$ and $A^T$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to solve the least square solutions to following linear systems via SVD:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=b_i,\quad i=1,2,3,\cdots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A^Ty=c_j,\;j=1,2,3,\cdots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it is also possible to do these via &lt;code&gt;householderQr().solve()&lt;/code&gt; in Eigen, but it is instable; So I want to do this via &lt;code&gt;SVD&lt;/code&gt; decomposition of $A$ and $A^T$; obviously, the &lt;code&gt;SVD&lt;/code&gt; of $A^T$ can be obtained by the transpose of that for $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how can I implement it in Eigen so that only SVD on $A$  is Ok to solve both the two type least square problems?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int m=100,n=50;&#xA;VectorXd b=VectorXd::randome(m),c=VectorXd::randome(n);&#xA;MatrixXd A=MatrixXd::random(m,n);&#xA;Eigen::JacobiSVD&amp;lt;Eigen::MatrixXd&amp;gt; _svd(A, Eigen::ComputeThinU | Eigen::ComputeThinV);   &#xA;MatrixXd  x = _svd.solve(b);&#xA;&#xA;A.transposeInPlace();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;// How can I save this sentence below which implements the SVD of $A^T$  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigen::JacobiSVD&amp;lt;Eigen::MatrixXd&amp;gt; _svd(A, Eigen::ComputeThinU | Eigen::ComputeThinV);&#xA;&#xA;MatrixXd  y = _svd.solve(c);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6693" LastEditDate="2014-02-07T02:32:58.457" LastActivityDate="2014-02-08T12:46:28.483" Title="How can I reuse the SVD of matrix A to solve LS problems for both A and its transpose via Eigen C++?" Tags="|c++|least-squares|svd|eigen|dense-matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10721" PostTypeId="1" AcceptedAnswerId="19825" CreationDate="2014-02-06T09:42:27.437" Score="7" ViewCount="318" Body="&lt;p&gt;In finite difference and finite volume methods, convection schemes (upwind, central, quick, ...) are usually shown in a normalized variable diagram. The diagram gives the normalized face variable as a function of the normalized upwind or downwind variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/O4Fcb.png&quot; alt=&quot;example of normalized variable diagram&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let $D$ be the donor cell, $A$ the acceptor cell and $U$ the upwind cell, then the normalized variable $\tilde{\phi}_f$ at the face between $D$ and $A$ is given as a function of the normalized donor variable $\tilde{\phi}_D$:&#xA;\begin{equation}&#xA;  \tilde{\phi}_f = F(\tilde{\phi}_D) \quad \text{with} \quad \tilde{\phi}\equiv\frac{\phi-\phi_U}{\phi_A - \phi_U}&#xA;\end{equation}&#xA;Various limiters are also expressed in terms of the normalized variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to define and robustly implement such schemes when $\phi_A \approx \phi_U$? &#xA;(which probably happens somewhere in the flow field).&lt;/p&gt;&#xA;" OwnerUserId="1973" LastEditorUserId="1973" LastEditDate="2014-02-20T08:03:59.197" LastActivityDate="2015-06-02T14:48:04.707" Title="Implementation of convection scheme given by normalized variable diagram" Tags="|finite-difference|finite-volume|advection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10722" PostTypeId="1" AcceptedAnswerId="10737" CreationDate="2014-02-06T10:46:05.133" Score="5" ViewCount="1661" Body="&lt;p&gt;I am dealing with large, sparse, rational matrices that I need to determine the nullspace of. Currently, I have one that is about 12000x12000 (but not square), where one in every 2000ish elements is nonzero. Subsequent matrices will grow both in size and sparsity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most nullspace algorithms rely on bringing the matrix in (some form of) a reduced row-Echelon form, and then reading the nullspace vectors off of that form. Since these algorithms require large amounts of elementary row operations, I would think that using Compressed Row Storage is the best option, since it allows easy acces to full rows, which is what is needed for the row operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as a part of these operations, it is required to insert elements in the sparse matrix occasionally. Now, as I understand it, inserting elements in a CRS matrix becomes increasingly expensive when the size of the matrix grows. Are there other options for the storage or the nullspace algorithm that avoids this problem?&lt;/p&gt;&#xA;" OwnerUserId="1264" LastEditorUserId="1264" LastEditDate="2014-02-06T16:35:32.597" LastActivityDate="2014-02-07T18:08:57.447" Title="Nullspace algorithm for a sparse matrix" Tags="|linear-algebra|sparse-matrix|matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10723" PostTypeId="2" ParentId="10004" CreationDate="2014-02-06T13:52:00.863" Score="0" Body="&lt;p&gt;Not sure if this helps, but the trick is that Lawson &amp;amp; Hanson's algorithm selects a special subset of columns from A such that when you perform regression x = A(:,subset) \ b resulting coefficients are all non-negative. &lt;/p&gt;&#xA;" OwnerUserId="7272" LastActivityDate="2014-02-06T13:52:00.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10724" PostTypeId="1" AcceptedAnswerId="10725" CreationDate="2014-02-06T14:37:07.947" Score="4" ViewCount="2251" Body="&lt;p&gt;Is there any clear classification between different iterative methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between &lt;code&gt;Newton-type&lt;/code&gt; and &lt;code&gt;Newton-like&lt;/code&gt; iterative methods?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="549" LastEditDate="2014-02-06T15:01:20.623" LastActivityDate="2014-02-06T16:40:07.093" Title="What is the difference between &quot;Newton-type&quot; and &quot;Newton-like&quot; iteration?" Tags="|iterative-method|newton-method|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10725" PostTypeId="2" ParentId="10724" CreationDate="2014-02-06T14:59:56.337" Score="5" Body="&lt;p&gt;They're equivalent.  Both imply some variation of the root-finding method by linearization.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-02-06T14:59:56.337" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10726" PostTypeId="1" CreationDate="2014-02-06T01:44:21.723" Score="5" ViewCount="110" Body="&lt;p&gt;Incomplete low-rank matrices can be exactly recovered in most cases, so long as the rank is low enough relative to the number of known entries. This result was famously proved by Candès and Recht in the paper &quot;Exact Matrix Completion via Convex Optimization&quot;. Algorithmically, the nuclear norm minimization that recovers the full matrix requires keeping the entire data matrix in memory, and only after convergence can the matrix be reduced to its low-rank parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we know that our matrix $X\in\mathbb{R}^{n\times n}$ is rank one: $X = uv^T$, $u$, $v\in\mathbb{R}^n$. When $n$ is large, keeping $X$ in memory may be intractable, even if it's very tractable to keep $u$ and $v$. A nice identity about the nuclear norm is that $||X||_* = \min_{u,v\,st\,uv^T=X} \frac{1}{2}||u||^2_2 + \frac{1}{2}||v||^2_2$, so it is possible to recast nuclear norm minimization as a minimization over $u$ and $v$ constrained such that $uv^T$ agrees with known entries of $X$. My question is this: can we carry out this minimization tractably? Will we still converge to the exact solution? How do we perform this minimization?&lt;/p&gt;&#xA;" OwnerUserId="6631" OwnerDisplayName="David Pfau" LastEditorUserId="7275" LastEditDate="2014-02-07T20:13:48.140" LastActivityDate="2014-02-07T20:13:48.140" Title="Exact recovery of large incomplete rank-one matrices?" Tags="|optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10727" PostTypeId="2" ParentId="10724" CreationDate="2014-02-06T16:40:07.093" Score="7" Body="&lt;p&gt;A few specific examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quasi-Newton methods avoid computing second derivatives by using an approximation to the Hessian which is updated at each iteration by a low rank (rank one or rank two) update.  This makes factoring the Hessian (or equivalently keeping the inverse in product form) easy to do computationally.  There are also limited memory Quasi-Newton methods that keep track of only the most recent rank one updates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Gauss-Newton method for minimization of sums of squares takes advantage of the sum of squares structure of the problem to get an approximation to the Hessian that only involves first derivatives (the second order term is dropped.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Levenberg-Marquardt method is a particular approach to stabilizing the Gauss-Newton iteration by regularizing the linear system that is solved in each iteration (either by adding a regularizing term in the equations or by using a trust region method.)&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-02-06T16:40:07.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10728" PostTypeId="2" ParentId="10726" CreationDate="2014-02-06T17:23:14.730" Score="1" Body="&lt;p&gt;I realized that specifically in the rank one case, this can be turned into a convex program by a change of variables. Let $\{i_1,\ldots,i_K\}$ and $\{j_1,\ldots,j_K\}$ be the row and column indices of the observed entries in $X$, our problem is to minimize $\frac{1}{2}||u||_2^2 + \frac{1}{2}||v||_2^2$ under the constraint $u_{i_k}v_{j_k} = X_{i_k j_k}\forall k = 1,\ldots,K$. Then let $p_i = \mathrm{log}(u_i)$ and let $q_j = \mathrm{log}(v_j)$, the transformed problem becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_{p,q} \frac{1}{2}\sum_i \mathrm{exp}(p_i)\overline{\mathrm{exp}(p_i)} + \frac{1}{2}\sum_i \mathrm{exp}(q_j)\overline{\mathrm{exp}(q_j)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with linear equality constraints $p_{i_k} + q_{j_k} = \mathrm{log}(X_{i_k j_k})$, which is a convex problem over complex numbers (or over real numbers if $X$ is nonnegative). Of course this doesn't extend to anything above rank one, as the constraints on $u$ and $v$ go from being products to sums of products. However, it does extend to arbitrary rank one tensors!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For anyone interested in going beyond the rank one case, here are some resources (courtesy of &lt;a href=&quot;http://statweb.stanford.edu/~rahulm/&quot; rel=&quot;nofollow&quot;&gt;Rahul Mazumder&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jmlr.org/papers/volume11/mazumder10a/mazumder10a.pdf&quot; rel=&quot;nofollow&quot;&gt;Spectral Regularization Algorithms for Learning Large Incomplete&#xA;Matrices&lt;/a&gt;: Describes an algorithm for matrix completion that allows you to work with the reduced singular value decomposition of $X$ instead of the entirety of $X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also some results suggesting that, when working with $U$ and $V$ rather than $X$, even if the convergence is to a local minimum in $U$ and $V$, this corresponds to the global minimum in $X$, for instance: &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.100.7257&quot; rel=&quot;nofollow&quot;&gt;Local minima and convergence in low-rank semidefinite programming&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6631" LastEditorUserId="6631" LastEditDate="2014-02-06T17:30:18.667" LastActivityDate="2014-02-06T17:30:18.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10729" PostTypeId="1" AcceptedAnswerId="10730" CreationDate="2014-02-06T18:33:52.507" Score="0" ViewCount="179" Body="&lt;p&gt;I have an optimization problem with the following objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max_{a^{l}_{n,k} b^l_{n,k}}  \sum_{n=1}^{\overline{N_l}} b_{k,n} \frac{C_1}{C_2} \log_2 \bigg(1 +\frac{a_{k,n} h_{k,n}} {b_{n,k} c_3}\bigg) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraints are linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is concave, if I keep all the constants as 1, for simplicity the objective function is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f  = b \cdot \log_2(1+a/b)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is concave right, or does it depends on the actual values of the constants?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if I add another parameter in the denominator of the log term as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max_{a^{l}_{n,k} b^l_{n,k}}  \sum_{n=1}^{\overline{N_l}} b_{k,n} \frac{C_1}{C_2} \log_2 \bigg(1 +\frac{a_{k,n} h_{k,n}} {b_{n,k} c_3 + X}\bigg) $ does it still remain concave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f= b⋅ \log (1+\frac{a}{b+1})$ is not concave right? Or does it depends on the value of X and other constants(which I kept one)?&lt;/p&gt;&#xA;" OwnerUserId="5387" LastEditorUserId="7142" LastEditDate="2014-02-08T13:19:24.670" LastActivityDate="2014-02-08T13:19:24.670" Title="Checking the convexity or the concavity of a function" Tags="|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10730" PostTypeId="2" ParentId="10729" CreationDate="2014-02-06T19:10:24.013" Score="1" Body="&lt;p&gt;$b\log(1+\frac{a}{b})$ is concave (it is the perspective of the concave function $\log(1+a)$. The function $b\log(1+\frac{a}{1+b})$ is not concave (You can compute Hessian to prove if you want, or pick two suitable points and check if the point in between has a lower function value than the average of the function values at the end points. Test the two end points $(a ,b)=(0.01,0.01)$ and $(a, b) = (1,1)$ for instance). I just plotted the function and saw that it is neither convex nor concave. Try simple things first...)&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2014-02-06T19:16:17.993" LastActivityDate="2014-02-06T19:16:17.993" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10731" PostTypeId="2" ParentId="10718" CreationDate="2014-02-06T22:40:12.510" Score="6" Body="&lt;p&gt;We have the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; that can be expressed as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A = JGJ^T$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The first thing is to calculate the &lt;a href=&quot;http://en.wikipedia.org/wiki/QR_decomposition&quot; rel=&quot;noreferrer&quot;&gt;QR decomposition&lt;/a&gt; of matrix &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt;. Because of the low rank of the matrix it can be done very fast with, for instance, modified Gram Schmidt algorithm. Now we can write &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A = QR G R^TQ^T$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is an orthonormal matrix (&lt;span class=&quot;math-container&quot;&gt;$Q^T Q = I$&lt;/span&gt;). We define &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$F = R G R^T$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$3\times3$&lt;/span&gt; matrix. here the eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; will be the same than the ones of your original matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. But you probably want to calculate also the eigenvectors of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, so we continue.&lt;/p&gt;&#xA;&lt;p&gt;Calculate the eigen decomposition of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$F = XDX^T$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is a diagonal matrix, and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is orthonormal (&lt;span class=&quot;math-container&quot;&gt;$X^T X = I$&lt;/span&gt;). Then, inserting that expression for &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; in the formula for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; we obtain&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A = QXDX^TQ^T$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;that can be rewritten as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A = YDY^T$&lt;/span&gt;,.&lt;/p&gt;&#xA;&lt;p&gt;So &lt;span class=&quot;math-container&quot;&gt;$Y = QX$&lt;/span&gt; is a orthonormal matrix whose columns are the eigenvectors of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is the diagonal matrix with the eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. It is unique except for the ordering of the columns of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7277" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-02-06T22:40:12.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10732" PostTypeId="1" AcceptedAnswerId="10733" CreationDate="2014-02-07T05:52:24.730" Score="4" ViewCount="424" Body="&lt;p&gt;I look for a numerical method to solve boundary value problems for systems of differential and algebraic equations of the form F(x,y,y') = 0, G(x,y) = 0, y(a) = ya, y(b) = yb, where y = (y1, y2, ... yn). I have to implement it myself, but I can't find such methods in textbooks. Where can I find description of needed method?&lt;/p&gt;&#xA;" OwnerUserId="7282" LastEditorUserId="7282" LastEditDate="2014-02-07T10:33:07.063" LastActivityDate="2014-02-07T13:54:24.767" Title="Methods for solving BVP for DAE" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10733" PostTypeId="2" ParentId="10732" CreationDate="2014-02-07T13:47:40.450" Score="2" Body="&lt;p&gt;The main methods I'm aware of are collocation methods. The best source I can think of on the topic is &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898713544&quot; rel=&quot;nofollow&quot;&gt;&quot;Numerical Solution of Boundary Value Problems for Ordinary Differential Equations&quot; by Ascher, Mattheij, and Russell&lt;/a&gt;, in which they devote a chapter to methods applicable to DAEs and generalized ODEs. One package for solving BVPs for DAE is &lt;a href=&quot;http://www.cs.ubc.ca/~ascher/coldae.f&quot; rel=&quot;nofollow&quot;&gt;COLDAE&lt;/a&gt; by Ascher and Spiteri; relevant references for the methods implemented therein can be found in the comment block at the start of the file in the link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3037190175&quot; rel=&quot;nofollow&quot;&gt;&quot;Differential-Algebraic Equations: Analysis and Numerical Solution&quot; by Kunkel and Mehrmann&lt;/a&gt; is a more up-to-date reference. I've only skimmed the book; a former colleague of mine recommended it.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-02-07T13:54:24.767" LastActivityDate="2014-02-07T13:54:24.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10734" PostTypeId="1" CreationDate="2014-02-07T14:17:53.040" Score="2" ViewCount="64" Body="&lt;p&gt;Suppose $A,B \in \mathbb{R}^{n\times n}; A=A^T, B=B^T$, let $C = AB, D =BA$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have all the real eigenvalues of $A$ and $B$, e.g. the eigenvalue decomposition of them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=P\Lambda_1 P^T$,&#xA;$B=Q\Lambda_2 Q^T$; both $P$ and $Q$ are orthogonal matrices, and both $\Lambda_1$ and $ \Lambda_2$ are diagnoal matrices;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can we also obtain the eigenvalues of $C$ and $D$ based on these information without solving eigenvalue problems of $C$ and $D$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the eigenvalues of $C$ and $D$ also real?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2014-02-07T14:17:53.040" Title="Are the eigenvalues of the product matrix of two real symmetric square matrices also real values?" Tags="|eigensystem|eigenvalues|dense-matrix|symmetry|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10737" PostTypeId="2" ParentId="10722" CreationDate="2014-02-07T16:50:39.553" Score="4" Body="&lt;p&gt;tl;dr : Use list-of-lists format, reorder your matrix, use sympy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use a different sparse matrix storage scheme like the &lt;a href=&quot;http://en.wikipedia.org/wiki/Sparse_matrix#List_of_lists_.28LIL.29&quot; rel=&quot;nofollow&quot;&gt;list-of-lists format&lt;/a&gt;, which allows for fast insertion of new matrix entries but suffers from poor cache coherency. You can then convert the inefficient storage scheme back to CRS once you're done finding the matrix's echelon form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Permuting the entries of the matrix can make a huge difference by reducing the amount of fill-in that you encounter. The &lt;a href=&quot;http://en.wikipedia.org/wiki/Cuthill-McKee_algorithm&quot; rel=&quot;nofollow&quot;&gt;Cuthill-McKee ordering&lt;/a&gt; is fairly common and works by doing a breadth-first search of the underlying sparse graph, reordering the matrix in the order that the unknowns were visited. This tends to cluster all the unknowns closer to the diagonal of the matrix. Alternatively, you can use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Minimum_degree_algorithm&quot; rel=&quot;nofollow&quot;&gt;approximate minimum degree algorithm&lt;/a&gt;, which you can play around with in MATLAB using the function &lt;code&gt;symamd&lt;/code&gt;. The documentation for the &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal.II library&lt;/a&gt; has a really awesome comparison of a mess of different reordering schemes but I can't seem to find it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ascertaining the connectivity structure of the reduced echelon form of the matrix is basically a problem of graph theory for which you needn't know the actual matrix entries (see &lt;a href=&quot;http://books.google.com/books?isbn=0898716136&quot; rel=&quot;nofollow&quot;&gt;Davis's book&lt;/a&gt; on sparse direct methods). So you can first find which entries are non-zero in the echelon form of the matrix, and only then perform the computations, content in the knowledge that you won't have to add any new non-zero entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge, the more common computational libraries don't have support for exact rational arithmetic, only floating point. If you like Python, the &lt;a href=&quot;http://sympy.org/en/index.html&quot; rel=&quot;nofollow&quot;&gt;sympy&lt;/a&gt; package can do exact rational arithmetic and has some support for sparse matrices.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2014-02-07T18:08:57.447" LastActivityDate="2014-02-07T18:08:57.447" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10738" PostTypeId="1" CreationDate="2014-02-07T19:28:14.627" Score="3" ViewCount="66" Body="&lt;p&gt;Suppose we have incomplete observations of the square matrix $X$. Most matrix completion algorithms assume the matrix is low-rank. What if instead we assume the matrix of eigenvectors is a tensor product: $V = A\otimes B$, $X=VDV^{-1}$? This still dramatically reduces the space of possible matrices, so it may be possible to exactly recover a matrix of this form. How would we solve this algorithmically?&lt;/p&gt;&#xA;" OwnerUserId="7275" LastActivityDate="2014-02-07T19:28:14.627" Title="Matrix completion when the eigenvectors are a tensor product?" Tags="|matrix|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10739" PostTypeId="1" CreationDate="2014-02-07T21:02:15.817" Score="1" ViewCount="116" Body="&lt;p&gt;I want to minimize a cost function for a differential-nonlinear system (dynamic).&#xA;Is it possible to use this software? How can I do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards,&#xA;Haniye&lt;/p&gt;&#xA;" OwnerUserId="7285" LastEditorUserId="7142" LastEditDate="2014-02-08T13:58:27.793" LastActivityDate="2014-03-11T09:03:54.453" Title="Optimization for differential and nonlinear system" Tags="|nonlinear-programming|" AnswerCount="1" CommentCount="5" ClosedDate="2014-03-26T19:05:44.013" ContentLicense="CC BY-SA 3.0" />
  <row Id="10740" PostTypeId="1" AcceptedAnswerId="10743" CreationDate="2014-02-07T21:16:00.663" Score="3" ViewCount="331" Body="&lt;p&gt;Can anyone point me in the direction of literature pertaining to the selection of the coefficient c2 in the curvature condition in the wolfe conditions when performing a line search?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Nocedal, he says that c2 = 0.9 for quasi Newton methods, and that c2 = 0.1 for non linear conjugate gradient methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not been able to find any literature pertaining to this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="7286" LastActivityDate="2014-02-08T03:08:02.693" Title="coefficient c2 for curvature condition of Wolfe Conditions for line search in non linear conjugate gradient" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10741" PostTypeId="1" CreationDate="2014-02-07T22:57:10.117" Score="6" ViewCount="310" Body="&lt;p&gt;I have a logarithmic grid, upon which i have two functions that are similar to this one (this is only the last 100 points):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tXElC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are essentially very similar to a Sin function at this point.  I need to take the integral of two of these functions multiplied together, along with the position:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_0^R f_1(r)\; r\; f_2(r)\; dr&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this gives me functions similar to the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ENpT4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to integrate these with high accuracy, but I'm not sure how to do it.  There are a few options that I know of, but I expect there are more that I'm not sure of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I can interpolate the original functions (is cubic interpolation a good interpolation strategy for sinusoids and products of sinusoids?), then multiply the interpolated functions together at specific points and use something from &lt;a href=&quot;https://scicomp.stackexchange.com/questions/58/whats-the-state-of-the-art-in-highly-oscillatory-integral-computation&quot;&gt;this scicomp post&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I can switch the order of the interpolation and multiplication, multiplying the values I have, then taking the interpolation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I can do a simpler Simpsons rule approximation, though I'm not sure how to do such a thing on a non-uniform grid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="35" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-02-07T22:57:10.117" Title="Integrating highly oscillatory functions" Tags="|interpolation|quadrature|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="10743" PostTypeId="2" ParentId="10740" CreationDate="2014-02-08T03:08:02.693" Score="4" Body="&lt;p&gt;There is no magic to these parameters. Line search is a tool to get from areas where a full-step Newton's method doesn't converge into those areas where it does converge. For this, it is not necessary to be absolutely optimal in your choice of step length acceptance criteria, the parameters in these criteria, or the exactness of the Newton direction. They're all just tools to get to those places where you can take a full step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by all this is that these parameters are somewhat heuristic choices. The values Nocedal and Wright give are widely used, but there is no fundamental theory to back these choices up.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-08T03:08:02.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10744" PostTypeId="1" CreationDate="2014-02-08T05:39:27.513" Score="14" ViewCount="516" Body="&lt;p&gt;I'm dealing with a coupled system of three transient, non-linear convection-diffusion equations. Let's just say to simplify the problem that they take the following form:&#xA;$$&#xA;-\nabla\cdot(D_{1}(u_{2},u_{3})\nabla u_{1}) = -\nabla\cdot\mathbf{\chi}(u_{2},u_{3})&#xA;$$&#xA;$$&#xA;\frac{\partial u_{2}}{\partial t} + \nabla\cdot f_{1}(u_{1},u_{2},u_{3}) - \nabla\cdot(D_{2}(u_{2},u_{3})u_{2}) = 0&#xA;$$&#xA;$$&#xA;\frac{\partial u_{3}}{\partial t} + \nabla\cdot f_{2}(u_{1},u_{2},u_{3}) - \nabla\cdot(D_{3}(u_{2},u_{3})u_{3}) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume a general set of boundary conditions for each equation:&#xA;$$&#xA;u_{i} = g_{i,D}, \;\;\;\text{on}\;\Gamma_{D_{i}}&#xA;$$&#xA;$$&#xA;D_{i}\nabla u_{i} = g_{i,N}, \;\;\;\text{on}\;\Gamma_{N_{i}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My spatial discretization is via DGFEM and the time discretization is just Backward Euler. Anyway, I'll use the superscript $(k)$ to denote the solution at the $k$th time step. This is how I am decoupling the equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Solve for $u_{1}^{(k+1)}$ using information from the previous time step for $u_{2}$ and $u_{3}$. This effectively linearizes the equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-\nabla\cdot(D_{1}(u_{2}^{(k)},u_{3}^{(k)})\nabla u_{1}^{(k+1)}) = -\nabla\cdot\chi(u_{2}^{(k)},u_{3}^{(k)})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Use the new $u_{1}^{(k+1)}$ and information from the previous time step for $u_{3}$ to solve for $u_{2}^{(k+1)}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u_{2}}{\partial t} + \nabla\cdot f_{1}(u_{1}^{(k+1)},u_{2}^{(k+1)},u_{3}^{(k)}) - \nabla\cdot(D_{2}(u_{2}^{(k+1)},u_{3}^{(k)})\nabla u_{2}^{(k+1)}) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Solve for $u_{3}^{(k+1)}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u_{3}}{\partial t} + \nabla\cdot f_{2}(u_{1}^{(k+1)},u_{2}^{(k+1)},u_{3}^{(k+1)}) - \nabla\cdot(D_{3}(u_{2}^{(k+1)},u_{3}^{(k+1)})\nabla u_{3}^{(k+1)}) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the first equation is linear and the last two are non-linear. I am using Newton's method to handle the non-linearity. However, there are some bugs in my code and I'm a bit stuck trying to interpret the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this is long but I'm trying to be as detailed as possible here. Just a couple of questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I test the simple case where $u_{1} = u_{2} = u_{3} = 5+t$ and I get about machine precision error using linear basis functions. Newton's method is also converging quadratically. I'm using a timestep approximately equal to $h^{p+1}$. I can also increase it to $100h^{p+1}$ which surprisingly doesn't seem to affect the convergence much. Using this time step, I have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/* sample Newton iterations */&#xA;******** Current Time: 7.812500 ********&#xA;u2 Iteration: 1, Error: 4.155186e-01&#xA;u2 Iteration: 2, Error: 2.480219e-02&#xA;u2 Iteration: 3, Error: 9.169256e-05&#xA;u2 Iteration: 4, Error: 1.250692e-09&#xA;u2 Iteration: 5, Error: 8.042476e-16&#xA;Iterations for u2 convergence: 5&#xA;&#xA;u3 Iteration: 1, Error: 4.155186e-01&#xA;u3 Iteration: 2, Error: 2.480219e-02&#xA;u3 Iteration: 3, Error: 9.169256e-05&#xA;u3 Iteration: 4, Error: 1.250692e-09&#xA;u3 Iteration: 5, Error: 7.664044e-16&#xA;Iterations for u3 convergence: 5&#xA;&#xA;/* error results */&#xA;*********** L2 Error ***********&#xA;u1 error in L2-norm:&#xA;4.53162e-12 &#xA;4.64396e-12 &#xA;6.15316e-12 &#xA;1.07398e-11 &#xA;&#xA;u2 error in L2-norm:&#xA;4.12483e-13 &#xA;7.30247e-13 &#xA;1.39962e-12 &#xA;2.76142e-12 &#xA;&#xA;u3 error in L2-norm:&#xA;4.09102e-13 &#xA;7.29944e-13 &#xA;1.39935e-12 &#xA;2.76121e-12 &#xA;&#xA;&#xA;*********** H1 Error ***********&#xA;u1 error in H1-norm:&#xA;1.05844e-11 &#xA;2.03320e-11 &#xA;3.84458e-11 &#xA;7.47941e-11 &#xA;&#xA;u2 error in H1-norm:&#xA;6.40033e-13 &#xA;1.33164e-12 &#xA;2.71254e-12 &#xA;5.47814e-12 &#xA;&#xA;u3 error in H1-norm:&#xA;6.34670e-13 &#xA;1.33089e-12 &#xA;2.71211e-12 &#xA;5.47862e-12 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the bizarre part is that if I take $u_{1}=5+t$, $u_{2}=6+t$, and $u_{3}=7+t$, my results are awful. I'm thinking maybe the first case was the exception rather than the norm since it's the only example I've tested so far that seems to work. Anyway, the weird part about this one is that I'm getting about machine precision error for $u_{2}$ while the others are flatlining around &lt;code&gt;1.e0&lt;/code&gt;. I'm not sure how this could possible be since the solution to $u_{2}$ depends on the solutions to $u_{1}$ and $u_{3}$, which are incorrect if my code is to be believed. I observe quadratic convergence for $u_{2}$, but obviously not the others...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time step is around $h^{p+1}$ here, but I observe similar results for smaller time steps.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;******** Current Time: 0.074219 ********&#xA;u2 Iteration: 1, Error: 3.908065e-03&#xA;u2 Iteration: 2, Error: 2.219552e-06&#xA;u2 Iteration: 3, Error: 2.889154e-12&#xA;Iterations for u2 convergence: 3&#xA;&#xA;u3 Iteration: 1, Error: 1.373826e-01&#xA;u3 Iteration: 2, Error: 1.564705e-02&#xA;u3 Iteration: 3, Error: 2.492157e-03&#xA;u3 Iteration: 4, Error: 7.872758e-04&#xA;u3 Iteration: 5, Error: 1.311116e-04&#xA;u3 Iteration: 6, Error: 2.997904e-05&#xA;u3 Iteration: 7, Error: 9.158730e-06&#xA;u3 Iteration: 8, Error: 1.519301e-06&#xA;u3 Iteration: 9, Error: 3.555316e-07&#xA;u3 Iteration: 10, Error: 1.067623e-07&#xA;u3 Iteration: 11, Error: 1.755777e-08&#xA;u3 Iteration: 12, Error: 4.195674e-09&#xA;u3 Iteration: 13, Error: 1.242190e-09&#xA;u3 Iteration: 14, Error: 2.024999e-10&#xA;u3 Iteration: 15, Error: 4.936768e-11&#xA;u3 Iteration: 16, Error: 1.443518e-11&#xA;u3 Iteration: 17, Error: 2.333785e-12&#xA;Iterations for u3 convergence: 17&#xA;&#xA;*********** L2 Error ***********&#xA;u1 error in L2-norm:&#xA;3.02951e+00 &#xA;5.43305e+00 &#xA;5.96871e+00 &#xA;4.25056e+00 &#xA;&#xA;u2 error in L2-norm:&#xA;1.45749e-13 &#xA;2.71039e-13 &#xA;3.69646e-13 &#xA;4.95591e-13 &#xA;&#xA;u3 error in L2-norm:&#xA;6.29385e-01 &#xA;8.30180e-01 &#xA;1.11972e+00 &#xA;1.13907e+00 &#xA;&#xA;&#xA;*********** H1 Error ***********&#xA;u1 error in H1-norm:&#xA;7.86736e+00 &#xA;1.81562e+01 &#xA;3.67674e+01 &#xA;3.97038e+01 &#xA;&#xA;u2 error in H1-norm:&#xA;2.33954e-13 &#xA;2.00202e-12 &#xA;4.95659e-12 &#xA;1.03835e-11 &#xA;&#xA;u3 error in H1-norm:&#xA;2.00438e+00 &#xA;3.91391e+00 &#xA;6.24097e+00 &#xA;7.06979e+00 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think maybe the most telling part is the convergence of the Newton iterations. Namely, what does it mean if I'm not observing quadratic convergence? My initial guess is just the solution at the previous time step, and since the time step is quite small, I'm pretty sure the initial guess is not the issue. I know that non-quadratic convergence can imply that the Jacobian is not being computed properly, but the first example is so similar to the second that I can't imagine that's the case...anyone have any insights here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) I'm a bit iffy on the spatial discretizaton of the convection term:&#xA;$$&#xA;\int_{\Omega}\nabla\cdot f_{1}(u_{1},u_{2},u_{3})z = -\int_{\Omega}\nabla z\cdot f_{1}(u_{1},u_{2},u_{3}) + \int_{\Gamma}z(f_{1}\cdot\mathbf{n})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(via Green's identity). The DG discretization is&#xA;$$&#xA;-\sum_{E\in\mathcal{M}_{h}}\int_{E}\nabla z\cdot f_{1} + \sum_{e\in\Gamma_{\text{int}}}\int_{e}\left\{\left\{f_{1}\cdot\mathbf{n}\right\}\right\}[[z]] + \sum_{e\in\Gamma_{\text{out}}}\int_{e}(f_{1}\cdot\mathbf{n})z + \sum_{e\in\Gamma_{\text{in}}}\int_{e}(f_{1}\cdot\mathbf{n})z&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where {{ }} is the average and [[ ]] is the jump. In all of the literature I've come across, the inflow integral is moved to the right-hand side. So if we're considering the second equation (solving for $u_{2}$), is this correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sum_{e\in\Gamma_{\text{in}}}\int_{e}(f_{1}(u_{1},g_{2,D},u_{3})\cdot\mathbf{n})z&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when computing the Jacobian for the second equation, does this inflow term not contribute to the Jacobian at all since $u_{2,D}$ is known?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm able to solve a single transient, non-linear convection diffusion equation using Newton's method, so I'm not entirely sure what it is that I'm missing.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2014-02-09T09:59:18.187" LastActivityDate="2020-09-26T15:35:52.837" Title="Sequential approach to solving coupled PDEs" Tags="|pde|finite-element|discontinuous-galerkin|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10745" PostTypeId="2" ParentId="10720" CreationDate="2014-02-08T12:46:28.483" Score="1" Body="&lt;p&gt;I do not have too much experience using eigen, but when you are solving the systems with the SVD decomposition, actually you are doing the followin:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A x = USV^T x = b $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and you use the SVD decomposition of A to isolate &lt;em&gt;x&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x = VS^{-1}U^T b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same for the other system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A^T y = VSU^T y = c $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and you use the SVD decomposition of A to isolate &lt;em&gt;x&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y = US^{-1}V^T c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So with eigen you should be able to recover a pointer to the matrices $U$, $S$ and $V$ as something similar to that&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MatrixUType* U        = _svd.matrixU()&#xA;MatrixVType* V        = _svd.matrixV()&#xA;SingularValuesType* S = _svd.singularValues()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then use some functions from the library to apply the operations to $c$ (matrix multiplications, traspose multimplication, and diagonal inversion) to obtain $y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sorry I can not give you more implementation details, but it think that it can help.&lt;/p&gt;&#xA;" OwnerUserId="7277" LastActivityDate="2014-02-08T12:46:28.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10746" PostTypeId="1" AcceptedAnswerId="10756" CreationDate="2014-02-08T20:20:44.260" Score="3" ViewCount="251" Body="&lt;p&gt;Are consumer grade GPUs (like the NVidia Geforce series) used for solving sparse linear equations systems in a professional setting? For example, by engineers performing finite element analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After all, at least for iterative methods, memory bandwidth is the bottleneck, and a Geforce 780 Ti provides a lot more of this than a Tesla C2075, while also costing a lot less.&lt;/p&gt;&#xA;" OwnerUserId="7289" LastActivityDate="2014-02-12T21:47:18.147" Title="Consumer GPUs for sparse matrices (e.g. FEA)" Tags="|finite-element|sparse-matrix|gpu|" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10747" PostTypeId="1" AcceptedAnswerId="10816" CreationDate="2014-02-08T20:28:20.110" Score="8" ViewCount="508" Body="&lt;p&gt;I've used &lt;code&gt;MT19937&lt;/code&gt; in a test harness to generate uniformly (unsigned) 32-bit [0, $2^{32}$- 1] values, based on the original Authors' &lt;strong&gt;mt19937.c&lt;/strong&gt; implementation, to generate an (essentially inexhaustible) supply of statistically random stream of bit-octets. This is in lieu of a CSPRNG, which isn't necessary for these particular tests. However, I've recently been considering the &lt;a href=&quot;http://www.iro.umontreal.ca/~panneton/WELLRNG.html&quot;&gt;&lt;code&gt;WELL&lt;/code&gt;&lt;/a&gt; PRNGs - not because of the statistical properties as such (both seem more than adequate for my needs) - but they seem to suggest more a efficient implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I lack the mathematical background for the academic papers, though I could at least follow the 'twist' matrix and tempering transform for the former. However, much of the code provided by the Authors of the &lt;code&gt;WELL-n&lt;/code&gt; functions seem to focus of floating-point generation, with some magic floating point constants (e.g., &lt;code&gt;2.32830643653869628906e-10&lt;/code&gt;). Can steps be omitted from the &lt;code&gt;WELL&lt;/code&gt; code to provide a uniform 32-bit distribution? Or is the algorithm designed / biased specifically for floating-point distributions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or I'm an incorrect in thinking that &lt;code&gt;WELL&lt;/code&gt; will yield a performance gain for 'bulk' uint32 vector generation, while satisfying my requirements?&lt;/p&gt;&#xA;" OwnerUserId="7287" LastEditorUserId="7287" LastEditDate="2014-02-09T14:49:42.403" LastActivityDate="2014-02-17T16:10:24.200" Title="WELL pseudo-random number generations" Tags="|performance|c|random-number-generation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10748" PostTypeId="1" AcceptedAnswerId="10750" CreationDate="2014-02-08T20:48:15.473" Score="3" ViewCount="16569" Body="&lt;p&gt;Have two arrays, in my case $X = \{1,2,\dots, n\}$ or &lt;code&gt;X = np.arange(n)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I get $Y = X \times X = \{ [i,j]: 1 \leq i,j \leq n   \}$ as a 2D array in numPy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In numpy this would be a $n \times n \times 2$ array &lt;code&gt;np.shape(Y)=(n,n,2)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="4006" LastEditDate="2014-02-08T21:52:13.227" LastActivityDate="2019-07-24T10:19:06.993" Title="cartesian products in numPy" Tags="|linear-algebra|matrix|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10749" PostTypeId="2" ParentId="10748" CreationDate="2014-02-08T22:23:47.610" Score="1" Body="&lt;p&gt;A possibility would be &lt;a href=&quot;http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html&quot; rel=&quot;nofollow&quot;&gt;broadcast&lt;/a&gt; your &lt;code&gt;x&lt;/code&gt; as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xx = numpy.zeros(n) + x.reshape(n,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to obtain the 'x-grid'. Do the same with the 2nd component and then use numpy's &lt;code&gt;dstack&lt;/code&gt; &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.dstack.html&quot; rel=&quot;nofollow&quot;&gt;function&lt;/a&gt; to tile them in the 3rd dimension.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-02-08T22:23:47.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10750" PostTypeId="2" ParentId="10748" CreationDate="2014-02-09T02:57:01.320" Score="4" Body="&lt;p&gt;You can use itertools.product:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from numpy import *&#xA;&amp;gt;&amp;gt;&amp;gt; import itertools&#xA;&amp;gt;&amp;gt;&amp;gt; array([x for x in itertools.product(arange(2), arange(3))])&#xA;array([[0, 0],&#xA;       [0, 1],&#xA;       [0, 2],&#xA;       [1, 0],&#xA;       [1, 1],&#xA;       [1, 2]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1665" LastActivityDate="2014-02-09T02:57:01.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10753" PostTypeId="2" ParentId="10739" CreationDate="2014-02-09T08:58:09.087" Score="0" Body="&lt;p&gt;In theory, you &lt;em&gt;could&lt;/em&gt; do it in MATLAB. In practice, you probably can't, and shouldn't, because it would probably be prohibitively slow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two dominant approaches I'm aware of for dynamic optimization are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;collocation; fully discretize your differential equations using a collocation method (Radau collocation is a common approach) and then solve a very large linear program&lt;/li&gt;&#xA;&lt;li&gt;interval arithmetic + automatic differentiation + branch-and-bound; essentially, use interval arithmetic and automatic differentiation to generate convex and concave relaxations of your dynamic optimization problem, and then use a branch-and-bound approach to solve&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In practice, neither of these approaches is likely to be viable in MATLAB; the former approach is a more realistic possibility than the latter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The large-scale nonlinear programming solver IPOPT was essentially written to solve nonlinear programs arising from collocation, and a compiled language was used (first Fortran, then C++) because large-scale nonlinear programs are computationally demanding. IPOPT does have a MATLAB interface affiliated with it, so if you were to try this approach in MATLAB, you might be able to use the interface and solve your problem. It's not clear that the interface is maintained; for instance, &lt;a href=&quot;http://www.coin-or.org/Ipopt/documentation/node18.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; suggests that it has been tested in MATLAB 7.2 (R2006a) to 7.7 (R2008b), which means it hasn't been tested against a version of MATLAB released in the past 5 years. If this interface works, it is probably your only viable approach for using MATLAB to solve a dynamic optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rigorous approach (interval arithmetic, etc.) is already time-consuming, even if everything is implemented in C++. Problems with 10 or so state variables take an hour or two to complete, and the algorithm scales exponentially with the number of state variables. Although interval arithmetic and automatic differentiation packages exist for MATLAB, you would probably have to write all of the remaining solver infrastructure (I'm not aware of any libraries that implement this approach). The performance of MATLAB is usually one to three orders of magnitude slower, assuming that you're not relying heavily on compiled libraries, so solving your problem would likely take an unacceptably long time (and that's assuming your problem is small enough for this approach).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-02-09T08:58:09.087" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10754" PostTypeId="1" AcceptedAnswerId="10759" CreationDate="2014-02-09T16:16:14.440" Score="3" ViewCount="288" Body="&lt;p&gt;I need to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mbox{min}||Ax - b||_2^2 \quad \mbox{s.t.} \quad l \leq x \leq u,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A \in R^{m \times n}$, $m \ll n$, $n \approx 10^4-10^5$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BVLS [1] based on active-set method works fine for small problems, but it is too slow for large-scale problems I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other (maybe approximate) methods to deal with such problems? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.stat.berkeley.edu/~stark/Preprints/bvls.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.stat.berkeley.edu/~stark/Preprints/bvls.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1089" LastEditorUserId="276" LastEditDate="2014-02-09T21:27:09.860" LastActivityDate="2014-02-09T21:27:09.860" Title="Large-scale box-constrained linear least-squares" Tags="|least-squares|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10756" PostTypeId="2" ParentId="10746" CreationDate="2014-02-09T18:40:07.850" Score="3" Body="&lt;p&gt;In a &quot;professional&quot; setting usually you run commercial software only on &quot;certified&quot; hardware, i.e. hardware on which the vendor has tested his software and is willing to provide user support. &quot;Certified and supported&quot; GPU's lists are usually very short...: there is really no technical reason, just commercial ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, high end FEA solution vendors charge such horrible high sums for licenses that clients do not care for the cost of the GPU cards and just buy the supported ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This said, if you are going to use &quot;consumer grade&quot; GPUs, you should test if, under sustained load, accuracy of the solution and performance is preserved.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2014-02-09T18:40:07.850" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10757" PostTypeId="1" AcceptedAnswerId="10758" CreationDate="2014-02-09T21:02:41.903" Score="2" ViewCount="11443" Body="&lt;p&gt;The &lt;a href=&quot;http://mitpress.mit.edu/sicp/full-text/sicp/book/node41.html&quot; rel=&quot;nofollow&quot;&gt;Huffman code&lt;/a&gt; is a way of compressing data streams by encoding the more frequent items with shorter words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a distribution on the letters &lt;strong&gt;A&lt;/strong&gt; thru &lt;strong&gt;J&lt;/strong&gt; and the code I obtained:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.00066872  0.01864353  0.0474964   0.18568327  0.11587298 &#xA;0.13329128  0.08580358  0.0881718   0.1847246   0.13964385&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that &lt;strong&gt;A&lt;/strong&gt; is least frequent ant gets a word of length six, while &lt;strong&gt;I&lt;/strong&gt; occurs more often and has length 2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'B': '100011'&#xA;'A': '000011'&#xA;'D': '111'&#xA;'C': '10011'&#xA;'F': '001'&#xA;'E': '110'&#xA;'H': '010'&#xA;'G': '1011'&#xA;'J': '101'&#xA;'I': '00'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Huffman algorithm builds a tree out of the different keys.  From &lt;a href=&quot;http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-046j-design-and-analysis-of-algorithms-spring-2012/lecture-notes/MIT6_046JS12_lec19.pdf&quot; rel=&quot;nofollow&quot;&gt;Pseudcode&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Begin with the set of leaf nodes, containing symbols and their frequencies, as determined by the initial data from which the code is to be constructed. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now find two leaves with the lowest weights and merge them to produce a node that has these two nodes as its left and right branches. The weight of the new node is the sum of the two weights. Remove the two leaves from the original set and replace them by this new node. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now continue this process. At each step, merge two nodes with the smallest weights, removing them from the set and replacing them with a node that has these two as its left and right branches. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Pseudocode&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Algorithm: HUFFMAN-TREE(C)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;n ← |C|&lt;/li&gt;&#xA;&lt;li&gt;Q ← C a min–&lt;a href=&quot;http://courses.csail.mit.edu/6.897/spring05/lec/lec14.pdf&quot; rel=&quot;nofollow&quot;&gt;priority queue&lt;/a&gt; keyed by frequency&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;for i ← 1 to n-1 do&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Allocate new node z&lt;/li&gt;&#xA;&lt;li&gt;z.left ← x ← EXTRACT-MIN(Q)&lt;/li&gt;&#xA;&lt;li&gt;z.right ← y ← EXTRACT-MIN(Q)&lt;/li&gt;&#xA;&lt;li&gt;z.freq ← x.freq ← y.freq&lt;/li&gt;&#xA;&lt;li&gt;Q.INSERT(z)&lt;/li&gt;&#xA;&lt;li&gt;B Return the root of the tree&lt;/li&gt;&#xA;&lt;li&gt;return EXTRACT-MIN(Q)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here's my implmentation in Python.  It uses a lot of sorting and does not use any of the data types above.  &lt;em&gt;Is there a way to do it using numPy and/or priority queues?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;f  = lambda a,b: int(100*(a[1]-b[1]))&#xA;&#xA;p = np.random.random(10)*np.arange(10)+0.01&#xA;p = p/sum(p)&#xA;q = p&#xA;p = sorted([(str(x[0]), x[1]) for x in enumerate(p)], f)&#xA;&#xA;code = { x[0]: '' for x in p}&#xA;&#xA;&#xA;for k in range(10-1):&#xA;    for x in p[0][0]:&#xA;        code[x] += '0'&#xA;    for x in p[1][0]:&#xA;        code[x] += '1'&#xA;&#xA;    p = sorted(p[2:] + [(p[0][0]+p[1][0], p[0][1]+p[1][1])],f)&#xA;&#xA;print q&#xA;print code&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2014-02-10T14:17:24.180" Title="Implement the Huffman code in Python or Numpy?" Tags="|python|data-structures|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10758" PostTypeId="2" ParentId="10757" CreationDate="2014-02-09T21:06:42.487" Score="2" Body="&lt;p&gt;you can use &lt;a href=&quot;http://docs.python.org/2/library/heapq.html&quot; rel=&quot;nofollow&quot;&gt;http://docs.python.org/2/library/heapq.html&lt;/a&gt; as a priority queue&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-02-09T21:06:42.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10759" PostTypeId="2" ParentId="10754" CreationDate="2014-02-09T21:14:49.380" Score="3" Body="&lt;p&gt;In the bound-constrained case, MATLAB defaults to using a trust-region reflective method found in Coleman, T.F. and Y. Li, &quot;A Reflective Newton Method for Minimizing a Quadratic Function Subject to Bounds on Some of the Variables,&quot; SIAM Journal on Optimization, Vol. 6, Number 4, pp. 1040-1058, 1996. You might try that method, and see how it works out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For very large scale problems, I'd probably use algorithms in &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/tao/&quot; rel=&quot;nofollow&quot;&gt;TAO&lt;/a&gt; tailored to quadratic problems, such as GPCG or BQPIP.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-02-09T21:14:49.380" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10760" PostTypeId="1" CreationDate="2014-02-09T17:37:46.800" Score="3" ViewCount="1368" Body="&lt;p&gt;I'd like to model the fluid flow through a porous medium using finite differences. Since I am new to this numerical technique, I have a simple question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the following set of equations to calculate the change in water volume or porosity over time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q_x = K \frac{dP}{dx}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q_y = K \left(\frac{dP}{dy} + \varphi g\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d\phi}{dt} = \frac{q_x}{dx} + \frac{q_y}{dy} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K$, $P$, $g$, $q$, $\varphi$, $\phi$ are a constant, pressure, gravity, flow velocity, fluid density and porosity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to solve the groundwater flow equation for fluid flow along the x-axis:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d\phi_{i,j}}{dt} = K \frac{P_{i-1,j} - 2P_{i,j} + P_{i+1,j}}{dx^2 } + ??? $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do I incorporate the gravity term for fluid flow along the y-axis?&#xA;Would this be the correct solution?&#xA;$$\frac{d\phi_{i,j}}{dt} = K \frac{P_{i-1,j} - 2P_{i,j} + P_{i+1,j}}{dx^2 } + K \frac{P_{i,j-1} - 2P_{i,j} + P_{i,j+1} + \varphi g}{dy^2} $$&lt;/p&gt;&#xA;" OwnerDisplayName="Marius" LastEditorUserId="53" LastEditDate="2014-02-12T07:29:06.460" LastActivityDate="2014-02-12T16:13:27.230" Title="Darcy Flow - Solution by Finite Difference" Tags="|fluid-dynamics|computational-physics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10761" PostTypeId="2" ParentId="10760" CreationDate="2014-02-10T02:02:33.437" Score="3" Body="&lt;p&gt;No, that's not right. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I assume you mean partial derivatives everywhere you have written $d$. Second, your formula:&#xA;$$&#xA;\frac{d \phi}{dt} = \frac{q_x}{dx} + \frac{q_y}{dy}&#xA;$$&#xA;should probably be:&#xA;$$&#xA;\frac{\partial \phi}{\partial t} = \frac{\partial q_x}{\partial x} + \frac{\partial q_y}{\partial y}&#xA;$$&#xA;or&#xA;$$&#xA;\frac{\partial \phi}{\partial t} = \nabla \cdot \boldsymbol{q} &#xA;$$&#xA;Next, I'm assuming that where you write the gravity term with $\varphi$ you meant to write $\phi$, otherwise, you need to give a definition of $\varphi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, given all of the above, you can see that your $y$-direction term should have a first derivative of $\phi$ in it:&#xA;$$&#xA;\frac{\partial q_y}{\partial y} = \frac{\partial}{\partial y} \left(K\frac{\partial P}{\partial y} + \phi g\right) = \frac{\partial}{\partial y}\left(K \frac{\partial P}{\partial y}\right) + g\frac{\partial \phi}{\partial y}&#xA;$$&#xA;Here I've assumed that $K$ was only supposed to multiply the first term in the definition of $q_y$, but if it's both, then you should have a $K$ in the coefficient of second term as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on this, you should have an approximation of the first-derivative of $\phi$ appearing in your formula. Given that this makes this problem a convection-diffusion problem, you may need to be very careful with how you implement that. &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2014-02-10T04:54:34.853" LastActivityDate="2014-02-10T04:54:34.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10762" PostTypeId="1" AcceptedAnswerId="10763" CreationDate="2014-02-10T02:52:26.583" Score="4" ViewCount="2504" Body="&lt;p&gt;According to the Perron-Frobenius theorem, a real matrix with only positive entries (or one with non-negative entries with a property called irreducibility) will have a unique eigenvector that contains only positive entries. Its corresponding eigenvalue will be real and positive, and will be the eigenvalue with greatest magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a situation where I'm interested in such an eigenvector. I'm currently using numpy to find all the eigenvalues, then taking the eigenvector corresponding to the one with largest magnitude. The trouble is that for my problem, when the size of the matrix gets large, the results start to go crazy, e.g. the eigenvector found that way might not have all positive entries. I guess this is due to rounding errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of this, I'm wondering if there's an algorithm that can give better results by making use of the facts that $(i)$ the matrix has non-negative entries and is irreducible, and $(ii)$ we're only looking for the eigenvector whose entries are positive. Since there are algorithms that can make use of other matrix properties (e.g. symmetry), it seems reasonable to think this might be possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While writing this question it occurred to me that just iterating $\nu_{t+1} = \frac{A\nu_t}{|A\nu_t|}$ will work (starting with an initial $\nu_0$ with positive entries), but imagine with a large matrix the convergence will be very slow, so I guess I'm looking for a more efficient algorithm than this. (I'll try it though!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if the algorithm is easy to implement and/or has been implemented in a form that can easily be called from Python, that's a huge bonus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Incidentally, in case it makes any kind of difference, my problem is &lt;a href=&quot;https://math.stackexchange.com/questions/661801/infinite-matrix-leading-eigenvalue-problem&quot;&gt;this one&lt;/a&gt;. I'm finding that as I increase the matrix size (finding the eigenvector using Numpy as described above) it looks like it's converging, but then suddenly starts to jump all over the place. This instability gets worse the smaller the value of $\lambda$.&lt;/p&gt;&#xA;" OwnerUserId="3746" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-02-10T11:53:15.670" Title="Algorithm for directly finding the leading eigenvector of an irreducible matrix" Tags="|linear-algebra|eigensystem|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10763" PostTypeId="2" ParentId="10762" CreationDate="2014-02-10T04:43:31.560" Score="3" Body="&lt;p&gt;The algorithm you describe that computes $x^{(k+1)}=\frac{Ax^{(k)}}{\|Ax^{(k)}\|}$ is of course what is called the the &lt;em&gt;power method&lt;/em&gt;. It will converge in your case if you have a non-degenerate largest eigenvalue. Furthermore, if you start with an initial guess $x^{(0)}$ that has only positive entries, you are guaranteed that all future iterates are also strictly positive and, moreover, that round-off should not be an issue since in every matrix-vector product &lt;em&gt;you only ever add up positive terms&lt;/em&gt;. In other words, this method must work if your problem has the properties you describe.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-10T04:43:31.560" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10766" PostTypeId="1" AcceptedAnswerId="10767" CreationDate="2014-02-10T09:00:23.820" Score="5" ViewCount="5829" Body="&lt;p&gt;For a nonsingular lower or upper triangular square matrix $A$, how to solve such linear system in Eigen:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A x = b$$&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2017-06-16T12:56:15.507" Title="How to use TrangularView class in Eigen C++" Tags="|linear-solver|c++|eigen|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10767" PostTypeId="2" ParentId="10766" CreationDate="2014-02-10T09:47:09.163" Score="6" Body="&lt;p&gt;As documented &lt;a href=&quot;http://eigen.tuxfamily.org/dox/group__QuickRefPage.html#title14&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = A.triangularView&amp;lt;Upper&amp;gt;().solve(b);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = A.triangularView&amp;lt;Lower&amp;gt;().solve(b);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4337" LastActivityDate="2014-02-10T09:47:09.163" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10768" PostTypeId="2" ParentId="10762" CreationDate="2014-02-10T11:45:58.650" Score="1" Body="&lt;p&gt;I tried the power method, as described in my question and in Wolfgang Bangerth's answer. It turns out that the convergence isn't unacceptably slow for my application, though it does depend on the parameters. So in a way this was kind of a dumb question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I noticed that there's a way to exponentially increase the speed of this algorithm, by doing repeated squaring of the matrix. That is, let $B_0=A$ and iterate $B_{n+1}=\frac{B_n^2}{\|B_n^2\|}$, where $\|\cdot\|$ is whatever matrix norm you feel like. (I just summed all the elements, since they're all positive anyway.) This very rapidly converges to a matrix whose columns are each proportional to the leading eigenvector. This is because $B_n \nu_0 \propto A^{2^n}\nu_0$, so iterating this algorithm for $n$ steps is the same as doing power iteration for $2^n$ steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although multiplying large matrices can be slow (especially in numpy, unfortunately), I'm finding that this tends to converge pretty nicely after around 10 to 15 iterations, so on the whole it's pretty fast. &lt;/p&gt;&#xA;" OwnerUserId="3746" LastEditorUserId="3746" LastEditDate="2014-02-10T11:53:15.670" LastActivityDate="2014-02-10T11:53:15.670" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10769" PostTypeId="2" ParentId="10757" CreationDate="2014-02-10T14:17:24.180" Score="1" Body="&lt;p&gt;Are you looking for a fast implementation, or are you doing this to learn about Python/numPy/priority queues/Huffman codes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of different implementations out there already:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://code.activestate.com/recipes/576603-huffman-coding-encoderdeconder/&quot; rel=&quot;nofollow&quot;&gt;http://code.activestate.com/recipes/576603-huffman-coding-encoderdeconder/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://rosettacode.org/wiki/Huffman_coding#Python&quot; rel=&quot;nofollow&quot;&gt;http://rosettacode.org/wiki/Huffman_coding#Python&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;etc. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-02-10T14:17:24.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10770" PostTypeId="1" CreationDate="2014-02-10T17:00:22.983" Score="11" ViewCount="10940" Body="&lt;p&gt;I read somewhere that &lt;a href=&quot;http://pandas.pydata.org/index.html&quot;&gt;Pandas&lt;/a&gt; was first developed for the financial world, at least not especially for natural sciences (physics, biology and so on), so is there any similar data analysis Python package more “natural sciences oriented“?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have just started using Pandas and I have already encountered two problems I couldn’t solve without using other packages or homemade solution:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How to manage uncertainties?&lt;/li&gt;&#xA;&lt;li&gt;How to define easily the units of my data?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Maybe there are other issues but I lack experience to be more precise. For the moment, I think about the &lt;a href=&quot;http://packages.python.org/uncertainties/index.html&quot;&gt;uncertainties&lt;/a&gt; package to solve the first point but I am not sure it will work fine with Pandas and will not decrease the computation speed. Actually, I am not looking for a way of computing with uncertainties, just a simple way of storing uncertainties along with my imported data. For the second point, I didn't find a better solution than creating a dictionary apart from my DataFrame to manage the units associated to each data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have ever experienced the same issues, how did you solve it or which other package(s) do you use for data manipulation/storing/analysis in Python in natural sciences?&lt;/p&gt;&#xA;" OwnerUserId="7303" LastActivityDate="2016-05-17T07:26:16.127" Title="Pandas limitations and its alternatives in Python" Tags="|python|data-analysis|data-management|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10771" PostTypeId="2" ParentId="10002" CreationDate="2014-02-10T17:39:31.840" Score="0" Body="&lt;p&gt;There is also &lt;code&gt;dimension&lt;/code&gt; package by Barton Willis. It is most likely already&#xA;a part of your &lt;code&gt;maxima&lt;/code&gt; installation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://sourceforge.net/p/maxima/code/ci/master/tree/share/physics&quot; rel=&quot;nofollow&quot;&gt;http://sourceforge.net/p/maxima/code/ci/master/tree/share/physics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A direct link to a pdf file with documentation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://sourceforge.net/p/maxima/code/ci/master/tree/share/physics/dimension.pdf?format=raw&quot; rel=&quot;nofollow&quot;&gt;http://sourceforge.net/p/maxima/code/ci/master/tree/share/physics/dimension.pdf?format=raw&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may find this example useful&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  load(&quot;dimension&quot;);&#xA;  depends(v, [t, x]);&#xA;  depends(p, [t, x]);&#xA;  /* put known dimensions */&#xA;  qput(rho, mass/length^3, dimension);&#xA;  qput(v, length/time, dimension);&#xA;  qput(t, time, dimension);&#xA;  qput(x, length, dimension);&#xA;  qput(L, length, dimension);&#xA;&#xA;  /* Navier-Stokes equation */&#xA;  ns: rho*(diff(v, t) + v*diff(v, x)) = -diff(p, x) + mu*diff(v, x, 2);&#xA;&#xA;  /* assume dimensions for `p' and `mu' are unknown and derive them */&#xA;  eq1: dimension(lhs(ns)) = dimension(part(rhs(ns), 1));&#xA;  eq2: dimension(lhs(ns)) = dimension(part(rhs(ns), 2));&#xA;  sol: solve([eq1, eq2], [dimension(p), dimension(mu)])[1];&#xA;&#xA;  /* this line should give an error becouse `ns' is dimensionally inconsistent */&#xA;  /* dimension(ns); */&#xA;&#xA;  /* put dimensions for `p' and `mu'  */&#xA;  put(p, assoc(dimension(p), sol),   'dimension);&#xA;  put(mu, assoc(dimension(mu), sol), 'dimension);&#xA;&#xA;  /* now it is OK */&#xA;  dimension(ns);&#xA;&#xA;  /* define Reynolds number as dimensionless reverse viscosity */&#xA;  one_over_mu_unit: natural_unit(1/mu, [L, rho, v]);&#xA;  Re: (1/mu)/one_over_mu_unit;&#xA;&#xA;  print(&quot;dimension(mu):          &quot;, dimension(mu));&#xA;  print(&quot;dimension(p):          &quot;, dimension(p));&#xA;  print(&quot;Reynolds number:        &quot;, Re);&#xA;  print(&quot;dimension(ns):        &quot;,   dimension(ns));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                            mass&#xA;dimension(mu):           ----------- &#xA;                         length time&#xA;                            mass&#xA;dimension(p):           ------------ &#xA;                                   2&#xA;                        length time&#xA;                          rho v L&#xA;Reynolds number:         [-------] &#xA;                            mu&#xA;                           mass&#xA;dimension(ns):         ------------- &#xA;                             2     2&#xA;                       length  time&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1802" LastActivityDate="2014-02-10T17:39:31.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10772" PostTypeId="1" AcceptedAnswerId="10773" CreationDate="2014-02-10T19:35:02.793" Score="7" ViewCount="427" Body="&lt;p&gt;I was reading up on &lt;a href=&quot;http://www.scholarpedia.org/article/Spectral_methods&quot;&gt;Spectral Methods&lt;/a&gt; for PDEs. In all the descriptions I read, while the position component is approximated via a Fourier series or other methods, the time component is still discretized and solved via a time-step procedure (finite difference, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any reason why the time component is also not approximated via a closed form solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I found one &lt;a href=&quot;http://www.jstor.org/stable/2157448&quot;&gt;paper&lt;/a&gt; which does use a polynomial approximation even for the  time dimension but my question remains as to why it's not done in general. Is it because chaotic dynamics means the number of terms required for the representation will be too large?&lt;/p&gt;&#xA;" OwnerUserId="7306" LastEditorUserId="7306" LastEditDate="2014-02-10T20:05:56.697" LastActivityDate="2014-02-11T05:43:41.080" Title="Spectral Methods in time" Tags="|pde|discretization|time-integration|spectral-method|collocation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10773" PostTypeId="2" ParentId="10772" CreationDate="2014-02-10T21:41:17.127" Score="10" Body="&lt;p&gt;The reason is that with the exception of linear problems, if you do a Fourier (or other) decomposition in time, you end up with a significant number of problems that are coupled globally in time. In other words, you have to solve lots of problems on the entire time interval concurrently. That will typically bust your computational or memory budget. The beauty of time stepping schemes is that you only need to consider one (or a very small number) if time steps at the same time and can forget everything about the more recent past.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-10T21:41:17.127" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10776" PostTypeId="1" AcceptedAnswerId="10806" CreationDate="2014-02-11T02:07:37.710" Score="5" ViewCount="229" Body="&lt;p&gt;The original nonlinear optimization problem I have is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For constant symmetric matrices $A=A^T, B_i=B_i^T(\forall i\in\mathbb{N}) \in \mathbb{R}^{n\times n}, \text{rank}(A)=n,$&#xA;$$\arg\min\limits_{x\in\mathbb{R}^n}\dfrac{x^TAx}{\sum\limits_{i=1}^{m}\left(x^TB_ix\right)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone said this problem can be converted into an eigenvalue problem by Cholesky factorizing $A=LL^T, $ and substituting $y=L^Tx, x=L^{-T}y, Q_i=L^{-1}B_iL^{-T}$, then it is equivalent to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\arg\max\limits_{\left\|y\right\|_2=1}\sum\limits_{i=1}^{m}\left(y^TQ_iy\right)^2=\arg\max\limits_{\|y\|_2=1}y^T\left(\sum\limits_{i}^m(Q_iyy^TQ_i)\right)y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is it strictly or only approximately an eigenvalues problem (EVP)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is an EVP, is there any guaranteed global solver in polynomial time for such problem?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2014-02-13T21:16:42.390" Title="Is there guaranteed global solver for such an eigenvalue problem?" Tags="|optimization|eigensystem|nonlinear-programming|eigenvalues|nonconvex|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10777" PostTypeId="2" ParentId="10772" CreationDate="2014-02-11T05:43:41.080" Score="4" Body="&lt;p&gt;For something with a spectral flavor in time, look at deferred correction methods, starting with &lt;a href=&quot;http://link.springer.com/article/10.1023/A:1022338906936&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt;.  I would argue that they're not spectral in the usual sense of the word, but they give you a family of arbitrary-order Runge-Kutta methods, so if you think of &quot;refining&quot; by increasing the order (by adding more nodes), then the convergence can be spectral.  Of course, you could do the same thing with extrapolation methods, and nobody calls them &quot;spectral&quot;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-02-11T05:43:41.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10778" PostTypeId="1" AcceptedAnswerId="10780" CreationDate="2014-02-11T14:13:30.757" Score="5" ViewCount="488" Body="&lt;p&gt;I have been browsing through the documentation of the &lt;a href=&quot;http://www-03.ibm.com/systems/technicalcomputing/platformcomputing/products/lsf/&quot;&gt;IBM Platform Load Sharing Facility&lt;/a&gt;, to find out if it supports &lt;a href=&quot;http://en.wikipedia.org/wiki/Task_parallelism&quot;&gt;parallel task computing&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is what I think is relevant for this question, Please correct me if my perception is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Focus seems to be on the parallel job execution, where a computation has to be split up and run as different processes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code we have, however, is optimized for parallel task execution. Here is some pseudo C# code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(timeStep=0; timeStep &amp;lt; MaxTimeSteps; timeStep++)&#xA;{&#xA;    Parallel.ForEach(Cube.Cells, currentCell =&amp;gt;&#xA;    {&#xA;        Compute(currentCell, Cube);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code will calculate a time series and perform some computation on each cell in every time step. The computation needs the state of other cells in the same time step, to do the computation. Note the &lt;a href=&quot;http://msdn.microsoft.com/en-us/library/dd460720%28v=vs.110%29.aspx&quot;&gt;Parallel.ForEach&lt;/a&gt; instruction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem cannot easily be split into separately running processes which calculate a time step each. The state of each cell of the previous time step is needed in current time step. Splitting the problem further, into a job for each cell in each time step, would require synchronization and extensive exchange of data, be it inter process or through files or database systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the Parallel.ForEach instruction could benefit extremely from the availability of many CPU's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, could this problem, running as a single process, benefit from a grid running LSF? Would it be transparent for the process, so that it perceives one huge system, with many CPU's and one big memory address space?&lt;/p&gt;&#xA;" OwnerUserId="7313" LastActivityDate="2014-02-14T17:29:59.687" Title="Does IBM LSF facilitate parallel computing?" Tags="|parallel-computing|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10779" PostTypeId="2" ParentId="10778" CreationDate="2014-02-11T14:38:11.270" Score="3" Body="&lt;p&gt;You haven't told us anything about how much work is involved in &quot;Compute(currentCell,Cube)&quot;, so it's difficult to fully answer this question.  Do you just need to execute a few lines of code per cell, or is it a very involved process that takes several minutes (or hours) of CPU time?  What are the rough numbers of cells and time steps?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also haven't said anything about how much data is shared in the cube and how it is accessed.  Does every call to Compute require access to the entire cube or just to cells near the currentCell in a 3D grid?  How much data is there per cell?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the answers to these questions, it might make sense to implement this with parallel threads on a single multiprocessor shared memory system (using e.g. OpenMP), or it might be reasonable to implement this using message passing on a distributed memory cluster (using e.g. MPI for the message passing.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, it's unlikely that LSF will be directly relevant to solving this problem.  LSF is a job scheduler that works at a much higher level than would be appropriate for this.  If you had (for example) a program that used MPI message passing between 100 processes that were working cooperatively, then you could use LSF to schedule a run of the program and LSF would schedule the execution of the program for a time when 100 processors with enough memory were available to run the job.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-02-11T14:38:11.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10780" PostTypeId="2" ParentId="10778" CreationDate="2014-02-11T14:41:15.403" Score="5" Body="&lt;p&gt;LSF definitely does &lt;em&gt;not&lt;/em&gt; provide transparent combination of nodes into a single shared memory environment for the single process running your multi-threaded program. There are tools that can do this (like &lt;a href=&quot;http://www.scalemp.com/&quot;&gt;ScaleMP's&lt;/a&gt; product), but that's not what LSF does. LSF is responsible for queuing up many jobs which wish to run on 1 to many nodes of a cluster and then executing those jobs in some order. It is the responsibility of the jobs themselves to deal with how the nodes in that job are utilized. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is usually done with &lt;a href=&quot;http://en.wikipedia.org/wiki/Message_Passing_Interface&quot;&gt;MPI&lt;/a&gt; which provides a portable interface for sending messages between programs or processes which are running independently on, perhaps, different computers using a variety of interconnect networks (including shared memory). Most HPC clusters use some sort of specialized network like &lt;a href=&quot;http://en.wikipedia.org/wiki/InfiniBand&quot;&gt;Infiniband&lt;/a&gt; for low-latency ($\sim\!1\,{\rm \mu s}$) and high-bandwidth connectivity (56Gb/s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MPI programs are increasingly, themselves, multi-threaded inside each process using such mechanisms as &lt;a href=&quot;http://en.wikipedia.org/wiki/OpenMP&quot;&gt;OpenMP&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/POSIX_Threads&quot;&gt;Pthreads&lt;/a&gt;. Sometimes this leads to better on-node performance and better exploitation of the shared memory architecture within the node. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;LSF is one of many such resource manager and scheduling systems that provide queuing functionality for Linux clusters. Others include: &lt;a href=&quot;http://slurm.schedmd.com/&quot;&gt;SLURM&lt;/a&gt;, &lt;a href=&quot;http://www.adaptivecomputing.com/products/hpc-products/moab-hpc-suite-enterprise-edition/&quot;&gt;Moab/Torque&lt;/a&gt;, and &lt;a href=&quot;http://www.pbsworks.com/&quot;&gt;PBSPro&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-02-11T14:41:15.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10781" PostTypeId="2" ParentId="10776" CreationDate="2014-02-11T14:56:15.020" Score="2" Body="&lt;p&gt;You can look at your problem in a different way. Look at the &lt;a href=&quot;http://en.wikipedia.org/wiki/Norm_%28mathematics%29&quot; rel=&quot;nofollow&quot;&gt;definition of norm&lt;/a&gt; , and then you can define your norm as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \| x \|_{\mathcal{B}} = \sqrt{\sum_i \| x \|_{B_i}^2}  = \sqrt{\sum_i \left( x^T,B_ix \right)^2} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal{B} = \{B_i\}_i$ is the set of matrices $B_i$ that you have. You can do it if your matrices $B_i$ are definite positive, or semidefinite positives and at least one of the definite positive. In other case you are not sure that you can define the norm (the same happens for your matrix $A$). Because they are symmetric and rank complete, I think it is enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can rewrite your problem as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\arg \min_{x\in \mathbb{R}} \frac{x^T A x }{\sum_i \left( x^T,B_ix \right)^2} =&#xA;\arg \min_{\|y\|_{\mathcal{B}}=1}  y^T A y&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where you have used that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;y = \frac{x}{\sqrt{\sum_i \left( x^T,B_ix \right)^2}} .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is strictly an eigenvalue problem. I am not sure about if the eigenvalues for the matrix A are always the same whithout taking into account the norm that you use to define the problem. I dont think so. Anyway, you can use the power iteration method, and define yourself the norm as in the explanation to normalize the eigenvector. It should work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that it will work. You start with a initial guess $y_0$. Then you iterates, solving the systems. At iteration $i$ you do&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat{y} = A^{-1}y_i $$&#xA;$$\lambda_i = \|\hat{y}\|_{\mathcal{B}}$$&#xA;$$y_{i+1} = \frac{y_i}{\lambda_i}$$&#xA;$$\varepsilon = \left\|\frac{\lambda_{i}-\lambda_{i-1}}{\lambda_{i}}\right\|$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;until $\varepsilon$ is small enough. Once you converge, your eigenpair $(\lambda,y)$ is the one with minimum eigenvector satisfying &#xA;$$A y = \lambda y$$&#xA;for a vector $y$ satisfying &#xA;$$\|y\|_{\mathcal{B}} = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is the same as&#xA;$$\lambda = y^T A y $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry, the previous is wrong as @Johan said in his answer, because the different exponent of $x$ in the numerator and the denominator. The answer would be valid if the problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\arg \min_{x\in \mathbb{R}} \frac{x^T A x }{\sum_i \left( x^T,B_ix \right)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we dont have the square in the denominator. Then defining the norm as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \| x \|_{\mathcal{B}} = \sqrt{\sum_i \| x \|_{B_i}}  = \sqrt{\sum_i \left( x^T,B_ix \right)} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the power iteration can be applied. For the original problem this not valid.&lt;/p&gt;&#xA;" OwnerUserId="7277" LastEditorUserId="7277" LastEditDate="2014-02-13T21:16:42.390" LastActivityDate="2014-02-13T21:16:42.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10782" PostTypeId="1" CreationDate="2014-02-11T15:22:13.397" Score="1" ViewCount="196" Body="&lt;p&gt;I know a square self-adjoint matrix $S_{vv}$ and I want to find:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S_{rr} = HS_{vv}H^{\dagger}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\dagger$ denotes conjugate transpose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know $H$ but I do know $H^{-1}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most efficient way to solve something of this form? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have considered rewriting it as: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S_{rr} = H(HS_{vv})^{\dagger}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the LU decomposition of $H^{-1}$, and then doing two lots of forward and back substitutions to solve for $S_{rr}$.  But it feels like there might be a better way? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help much appreciated. &lt;/p&gt;&#xA;" OwnerUserId="7316" LastActivityDate="2014-02-11T15:22:13.397" Title="Efficient way to do congruent transformation using matrix inverse?" Tags="|linear-algebra|matrix|eigenvalues|inverse|matrix-factorization|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10784" PostTypeId="1" CreationDate="2014-02-11T17:19:13.027" Score="3" ViewCount="129" Body="&lt;p&gt;Is there a conservative FV second-order (or first-order) accurate immersed boundary method for compressible flow including moving boundaries (in the literature)?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By compressible flow I mean the Navier-Stokes eqts. for an ideal gas.&lt;/li&gt;&#xA;&lt;li&gt;Most of the results I am able to find deal exclusively with incompressible flows.&lt;/li&gt;&#xA;&lt;li&gt;It would be nice if the publication describes the method in enough detail that one can reproduce their results.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1831" LastActivityDate="2014-03-14T13:05:56.213" Title="Conservative FV Immersed boundary method for compressible flow" Tags="|finite-volume|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10785" PostTypeId="1" AcceptedAnswerId="10827" CreationDate="2014-02-11T21:31:58.310" Score="4" ViewCount="471" Body="&lt;p&gt;I'm searching for software which can solve 3d IBVP like PDEtool in Matlab (user-friendly GUI, easy to learn how to use). What could you advise to me? (ANSYS,COMSOL,OpenFOAM are too difficult for me)&lt;/p&gt;&#xA;" OwnerUserId="4676" LastEditorUserId="549" LastEditDate="2014-02-12T00:01:54.713" LastActivityDate="2014-02-16T17:34:41.590" Title="User Friendly GUI based 3d PDE solving software" Tags="|pde|software|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10786" PostTypeId="1" AcceptedAnswerId="10793" CreationDate="2014-02-11T22:03:25.107" Score="3" ViewCount="548" Body="&lt;p&gt;For periodic functions on a regularly spaced grid, the trapezoidal rule is supposed to converge incredibly quickly ( the error estimate decreasing as $O(e^{-\eta/h})$ for step size $h$ ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't believe this is true for non-uniform grids.  What is the error estimate for a  non-uniform grid, and how does it converge? (specifically for a logarithmic grid, but more generally if possible).&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2014-02-12T14:20:59.880" Title="Trapezoidal rule (linear Newton-Cotes integration) convergence for periodic functions on general non-uniform grids" Tags="|quadrature|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10787" PostTypeId="1" CreationDate="2014-02-11T22:15:19.320" Score="5" ViewCount="180" Body="&lt;p&gt;If i have some function expressed as points on a non-uniform grid (I'm specifically interested in logarithmic grids, but general results are also interesting), and I want to integrate it, I believe there are two general possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Interpolate, and use some method like Gauss Quadrature, or Clenshaw-Curtis Quadrature.  The downside is that you now have stacked errors from the interpolation and the integration method.  The upside is that these quadrature methods are known to be very accurate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use some integration method like the trapezoidal rule, or Simpson's rule, which use the points that you have available to you.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there a general rule of thumb for which method is better?  Does it depend on some basic property of the function (for example: periodic)?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2014-02-11T22:15:19.320" Title="Methods for integrating black box functions on a non-uniform grid" Tags="|quadrature|error-estimation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10788" PostTypeId="2" ParentId="668" CreationDate="2014-02-12T00:10:57.113" Score="2" Body="&lt;p&gt;I think that making the mesh fit the boundary and hence a departure from the standard square mesh of the fdm is probably a solution but nevertheless has serious implications as to the use of high order algorithms - difficult if not impossible.  I have taken a different approach, namely keep the rectangular grid over the curved boundary geometry, create high order algorithms, interpolate from the boundary to set the values &quot;outside&quot; the geometry, and that's all there is.  we have achieved precisions in concentric sphere test geometries of ~1e-12  with this method using an order 8 algorithm.  if you will google &quot;Edwards, fdm curved boundary&quot; you will find references to my work.  &lt;/p&gt;&#xA;" OwnerUserId="6880" LastActivityDate="2014-02-12T00:10:57.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10789" PostTypeId="2" ParentId="10297" CreationDate="2014-02-12T00:19:41.647" Score="1" Body="&lt;p&gt;$$y(x) = \exp(-1/x^2)$$ in C is &lt;code&gt;exp(-1/(x*x))&lt;/code&gt;.  Since &lt;em&gt;most&lt;/em&gt; floating point in C is based on powers of 2, simple keep &lt;code&gt;y&lt;/code&gt; in two parts: an integer power of 2 and a significand.  Reconstitute &lt;code&gt;y&lt;/code&gt; as needed with &lt;code&gt;ldexp()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculation of &lt;code&gt;z&lt;/code&gt; (see below) may lose up to 2 bits.  &lt;code&gt;y_expo2&lt;/code&gt; will not lose anymore.  &lt;code&gt;y_signifcand&lt;/code&gt; may lose 1 bit.  Reconstituting &lt;code&gt;y&lt;/code&gt; should yield no more than a 4 bit loss of precision.  But each &lt;code&gt;x&lt;/code&gt; is halved, when less than 1e-3, another 2 bits is lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The helps maintain usable &lt;code&gt;y_expo2&lt;/code&gt; and &lt;code&gt;y_significand&lt;/code&gt; down to &lt;code&gt;|x|&lt;/code&gt; of about 1e-7.  But smaller than that, &lt;code&gt;y_expo2&lt;/code&gt; has all the precision and &lt;code&gt;y_signifcand&lt;/code&gt; little or none.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double f(double x) {&#xA;  return exp(-1 / (x * x));&#xA;}&#xA;&#xA;void fmellow(double x, double *y_expo2, double *y_signifcand) {&#xA;  static const double ln2 = 0.69314718055994530941723212145818; // log(2.0);&#xA;  double z = -1.0 / (x * x) / ln2;&#xA;  // y = exp(z), but this rapidly rounds to 0.0&#xA;  // Instead, break the floating-point number into a normalized fraction&#xA;  //   and an integral power of 2.&#xA;  double fraction;&#xA;  fraction = modf(z, y_expo2);&#xA;  *y_signifcand = pow(2.0, fraction);&#xA;}&#xA;&#xA;&#xA;void testf() {&#xA;  double x = 1.0;&#xA;  int i;&#xA;  for (i = 0; i &amp;lt; 24; i++) {&#xA;    double y = f(x);&#xA;    double y_expo2;&#xA;    double y_signifcand;&#xA;    fmellow(x, &amp;amp;y_expo2, &amp;amp;y_signifcand);&#xA;    double y_reconstituted = ldexp(y_signifcand, y_expo2);&#xA;    double error = (y - y_reconstituted) / y_reconstituted;&#xA;    printf(&quot;x=%8.2e  y=%9.2e  y_expo=%9.2e  y_signifcand=%9.2e  y_reconstituted=%9.2e  diff=%9.2e\n&quot;,&#xA;        x, y, y_expo2, y_signifcand, y_reconstituted, error);&#xA;    x *= 0.5;&#xA;  }&#xA;}&#xA;&#xA;int main() {&#xA;  testf();&#xA;  return 0;&#xA;}&#xA;&#xA;x=1.00e+00  y= 3.68e-01  y_expo=-1.00e+00  y_signifcand= 7.36e-01  y_reconstituted= 3.68e-01  diff= 0.00e+00&#xA;x=5.00e-01  y= 1.83e-02  y_expo=-5.00e+00  y_signifcand= 5.86e-01  y_reconstituted= 1.83e-02  diff=-1.89e-16&#xA;x=2.50e-01  y= 1.13e-07  y_expo=-2.30e+01  y_signifcand= 9.44e-01  y_reconstituted= 1.13e-07  diff=-2.35e-16&#xA;x=1.25e-01  y= 1.60e-28  y_expo=-9.20e+01  y_signifcand= 7.94e-01  y_reconstituted= 1.60e-28  diff=-8.39e-16&#xA;x=6.25e-02  y=6.62e-112  y_expo=-3.69e+02  y_signifcand= 7.96e-01  y_reconstituted=6.62e-112  diff=-3.63e-15&#xA;x=3.12e-02  y= 0.00e+00  y_expo=-1.48e+03  y_signifcand= 8.01e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=1.56e-02  y= 0.00e+00  y_expo=-5.91e+03  y_signifcand= 8.24e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=7.81e-03  y= 0.00e+00  y_expo=-2.36e+04  y_signifcand= 9.23e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=3.91e-03  y= 0.00e+00  y_expo=-9.45e+04  y_signifcand= 7.26e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=1.95e-03  y= 0.00e+00  y_expo=-3.78e+05  y_signifcand= 5.55e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=9.77e-04  y= 0.00e+00  y_expo=-1.51e+06  y_signifcand= 7.60e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=4.88e-04  y= 0.00e+00  y_expo=-6.05e+06  y_signifcand= 6.69e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=2.44e-04  y= 0.00e+00  y_expo=-2.42e+07  y_signifcand= 7.99e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=1.22e-04  y= 0.00e+00  y_expo=-9.68e+07  y_signifcand= 8.16e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=6.10e-05  y= 0.00e+00  y_expo=-3.87e+08  y_signifcand= 8.89e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=3.05e-05  y= 0.00e+00  y_expo=-1.55e+09  y_signifcand= 6.24e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=1.53e-05  y= 0.00e+00  y_expo=-6.20e+09  y_signifcand= 6.07e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=7.63e-06  y= 0.00e+00  y_expo=-2.48e+10  y_signifcand= 5.44e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=3.81e-06  y= 0.00e+00  y_expo=-9.91e+10  y_signifcand= 7.01e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=1.91e-06  y= 0.00e+00  y_expo=-3.97e+11  y_signifcand= 9.68e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=9.54e-07  y= 0.00e+00  y_expo=-1.59e+12  y_signifcand= 8.78e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=4.77e-07  y= 0.00e+00  y_expo=-6.35e+12  y_signifcand= 5.95e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=2.38e-07  y= 0.00e+00  y_expo=-2.54e+13  y_signifcand= 5.03e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;x=1.19e-07  y= 0.00e+00  y_expo=-1.02e+14  y_signifcand= 5.11e-01  y_reconstituted= 0.00e+00  diff=      nan&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4431" LastEditorUserId="4431" LastEditDate="2014-02-12T02:58:12.820" LastActivityDate="2014-02-12T02:58:12.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10790" PostTypeId="2" ParentId="8075" CreationDate="2014-02-12T07:23:30.463" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://star.mit.edu/cluster/&quot; rel=&quot;nofollow&quot;&gt;Starcluster&lt;/a&gt; uses SGE fairly intelligently.  It is part of a suite of tools that make numpy/scipy installs/deploy operations really easy via EC2.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have moved away from using SGE, as it is in &lt;a href=&quot;http://en.wikipedia.org/wiki/Oracle_Grid_Engine&quot; rel=&quot;nofollow&quot;&gt;developmental flux&lt;/a&gt; at the moment.&lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2014-02-12T07:23:30.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10791" PostTypeId="2" ParentId="10760" CreationDate="2014-02-12T08:08:52.327" Score="1" Body="&lt;p&gt;First up, all the things Bill said in his answer are spot on.  You have to be fairly rigorous about the nomenclature here.  The old-school expression you see for &quot;Darcys Law&quot; is something like the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Q = \frac{-K A}{\mu} \frac{(P_b-P_a)}{L} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is more often presented in the form given by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  \dot{Q} = - \frac{K}{\mu} \nabla P $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which provides a simple stimulus response relationship analogous to Ohm's law in electronics and Fick's law of diffusion. In this presentation, the geometric terms (and gravity!) have been absorbed to produce a &#xA;volumetric flow velocity $\dot{Q}$ in the direction of the gradient of pressure $\nabla P$, linked scalar viscosity and permeability of the media.  Think of gravity as a force that creates a pressure gradient.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Darcy flow is inherently steady state and with a bit of hoop jumping you can get a pair of equations which are solvable to get satisfaction of the Darcy flow law and continuity.  Omitting the math and symbol juggling, you rearrange things to get a pair of equations like this:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \tilde{\nabla} \tilde{P}= \tilde{\nabla}^2 \tilde{v} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \tilde{\nabla}^2 \tilde{P}= 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which you can descritize into a sparse matrix fairly cleanly or solve by some other methods.  All of this is outlined in my thesis, which just happens to be titled &lt;a href=&quot;https://dl.dropboxusercontent.com/u/4525748/masters-thesis-related/ms-thesis.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;Properties of Stochastic Flow and Permeability of Random Porous Media&quot;&lt;/a&gt;.  I have also &lt;a href=&quot;https://github.com/meawoppl/ndsolver&quot; rel=&quot;nofollow&quot;&gt;implemented a finite-difference solver&lt;/a&gt; that solves these equations in 2, 3 and higher dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The secret horrible hard part is getting all the BC's right :)&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="53" LastEditDate="2014-02-12T16:13:27.230" LastActivityDate="2014-02-12T16:13:27.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10792" PostTypeId="2" ParentId="10784" CreationDate="2014-02-12T12:24:08.227" Score="1" Body="&lt;p&gt;There are several research groups that are working on IBMs for compressible flow. A quick Google search leads to the following papers...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.me.jhu.edu/fsag/Publications/Papers/AShapecompressibleJCP.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.me.jhu.edu/fsag/Publications/Papers/AShapecompressibleJCP.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;http://multiscalemodeling.colorado.edu/liu/publications/Liu_JCP_2007.pdf&quot; rel=&quot;nofollow&quot;&gt;http://multiscalemodeling.colorado.edu/liu/publications/Liu_JCP_2007.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;http://ctr.stanford.edu/ResBriefs05/detullio.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ctr.stanford.edu/ResBriefs05/detullio.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045793006000065&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0045793006000065&lt;/a&gt;&#xA;&lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/fld.3872/abstract&quot; rel=&quot;nofollow&quot;&gt;http://onlinelibrary.wiley.com/doi/10.1002/fld.3872/abstract&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.tandfonline.com/doi/abs/10.1080/10618562.2013.791391#.UvtnHnn0jRo&quot; rel=&quot;nofollow&quot;&gt;http://www.tandfonline.com/doi/abs/10.1080/10618562.2013.791391#.UvtnHnn0jRo&lt;/a&gt;&#xA;&lt;a href=&quot;http://dutw1479.wbmt.tudelft.nl/~wim/academy/presentations/c1/pres_sanderse.pdf&quot; rel=&quot;nofollow&quot;&gt;http://dutw1479.wbmt.tudelft.nl/~wim/academy/presentations/c1/pres_sanderse.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of them seem to give enough detail so you can get started with your own implementation. Few people will be ready to give so much detail that you can write your own solver from scratch. When writing such codes, the devil is in the detail... &lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2014-02-12T12:24:08.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10793" PostTypeId="2" ParentId="10786" CreationDate="2014-02-12T14:20:59.880" Score="3" Body="&lt;p&gt;The trapezoidal rule is an interpolating quadrature where you incur an error on each interval that is independent from that of other intervals. In particular, if you have a set of disjoint intervals $I_i$ of size $h_i$ so that $\bigcup_i I_i = I=[a,b]$ is the interval that you want to integrate over, then the total error is given by an expression of the form&#xA;$$&#xA;  e = \sum_i h_i^3 \zeta_i&#xA;$$&#xA;where $\zeta_i$ is a measure of the second derivative of your integrand on interval $i$. In particular, if the function is linear, then the error is zero. Also, since the number of intervals is proportional to $1/h$ in the case of a uniform mesh, you get an error bound of the form &#xA;$$&#xA;  e = {\cal O}(h^2).&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-12T14:20:59.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10796" PostTypeId="1" AcceptedAnswerId="10799" CreationDate="2014-02-12T15:29:24.433" Score="9" ViewCount="3570" Body="&lt;p&gt;What are main differences between FEM and XFEM? When should we (not) use XFEM intead of FEM and vice versa? In other words, when I meet a new problem, how I can know to use which one of them?&lt;/p&gt;&#xA;" OwnerUserId="4670" LastActivityDate="2014-02-13T04:12:54.013" Title="Finite Element Method vs Extended Finite Element Method (FEM vs XFEM)" Tags="|finite-element|numerical-analysis|adaptive-mesh-refinement|numerics|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10797" PostTypeId="2" ParentId="10778" CreationDate="2014-02-12T15:48:32.180" Score="1" Body="&lt;p&gt;What LSF (or any batch scheduler) can offer is the ability to schedule your job on a machine with a large number of cores, so your process can maximize its potential parallelism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an integration between LSF and ScaleMP, so you can submit a job that asks for more cores than any single machine in your grid has (for example, 1024 cores), and LSF will schedule the creation of a vSMP w/ that many cores.  The integration requires that you state the exact number of cores.  I think in your case it might be better to ask for something like &quot;the most cores that are available now&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In regard to optimizing parallelism of a single process, OpenMP or MPI might be what you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="7327" LastEditorUserId="7327" LastEditDate="2014-02-14T17:29:59.687" LastActivityDate="2014-02-14T17:29:59.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10798" PostTypeId="2" ParentId="10796" CreationDate="2014-02-12T18:18:46.237" Score="6" Body="&lt;p&gt;FEM is a subset of XFEM.  XFEM is a methodology for enriching finite-element spaces to handle problems with discontinuities (such as fracture).  With classical FEM, attaining similar accuracy typically requires complicated conformal meshing and adaptive refinement, where as XFEM does it with a single mesh, moving that geometric complexity into the elements (XFEM is very complicated to implement, especially in 3D).  Meanwhile, XFEM results in extremely ill-conditioned matrices that require either direct solvers or very specialized multigrid methods (e.g., &lt;a href=&quot;https://cfwebprod.sandia.gov/cfdocs/CCIM/docs/gerstenberger_tuminaro_2012.pdf&quot;&gt;Gerstenberger and Tuminaro (2012)&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-02-12T18:18:46.237" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10799" PostTypeId="2" ParentId="10796" CreationDate="2014-02-12T21:10:11.253" Score="11" Body="&lt;p&gt;The Finite Element Method (FEM) is the parent method which has inspired many, many other methods and methods which are actually FEM but pretend not to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the finite element method, &quot;shape functions&quot; are used to provide an approximation space so that the solution can be represented by a vector. In the classical FEM, these shape functions are polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Extended Finite Element Method (XFEM), additional &quot;enrichment&quot; functions are used to approximate the solution in addition to the polynomial shape functions. These enrichment functions are chosen to have properties that the solution is known to follow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most obvious XFEM enrichment functions are power functions introduced at cracks sharp corners to represent the singularities in the solution gradient (i.e., the singularity in the stress for solid mechanics problems). The XFEM can be used for other enrichment functions and other solution domains (notably heat transfer), but the name synonymous with fracture analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distinction between various methods -- is this XFEM or not?, etc. -- is tricky and subtle and unimportant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for which to use, XFEM sees very little practical use. There are a handful of applications in real finite element codes, most notably Abaqus, but they have not seen widespread acceptance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For almost all practical problems, the classical FEM would be used. For most fracture analysis problems, classical FEM might still be used with suitable mesh refinement and/or p-refinement in the area of the crack tip. Other, less rigorous, fracture models may also be used.&lt;/p&gt;&#xA;" OwnerUserId="1056" LastEditorUserId="1056" LastEditDate="2014-02-13T04:12:54.013" LastActivityDate="2014-02-13T04:12:54.013" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="10800" PostTypeId="2" ParentId="10748" CreationDate="2014-02-12T21:32:44.403" Score="5" Body="&lt;p&gt;You want &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.meshgrid.html&quot; rel=&quot;noreferrer&quot;&gt;meshgrid()&lt;/a&gt;.  Here is a quick example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [14]: a = arange(5)&#xA;&#xA;In [18]: r = array(meshgrid(a,a))&#xA;&#xA;In [20]: r.shape&#xA;Out[20]: (2, 5, 5)&#xA;&#xA;In [21]: r&#xA;Out[21]: &#xA;array([[[0, 1, 2, 3, 4],&#xA;        [0, 1, 2, 3, 4],&#xA;        [0, 1, 2, 3, 4],&#xA;        [0, 1, 2, 3, 4],&#xA;        [0, 1, 2, 3, 4]],&#xA;&#xA;       [[0, 0, 0, 0, 0],&#xA;        [1, 1, 1, 1, 1],&#xA;        [2, 2, 2, 2, 2],&#xA;        [3, 3, 3, 3, 3],&#xA;        [4, 4, 4, 4, 4]]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since there is a lot of repeated values in there, you can call it with the sparse=True command to save mem and leverage some ndarray tricks.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you actually want some simple function of (i,j) you can do the following trick which avoids a formation of an immidiate.  I use it all the time but didn't see this in any of the other links:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [3]: a = arange(5)&#xA;&#xA;In [4]: a[:,newaxis] * a[newaxis,:]&#xA;Out[4]: &#xA;array([[ 0,  0,  0,  0,  0],&#xA;       [ 0,  1,  2,  3,  4],&#xA;       [ 0,  2,  4,  6,  8],&#xA;       [ 0,  3,  6,  9, 12],&#xA;       [ 0,  4,  8, 12, 16]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The example uses multiplication, but any number of binary functions can go in that place.  This trick also works for higher dimensions thanks to the broadcasting rules:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [6]: a = ones((2,5))&#xA;&#xA;In [7]: a[:,newaxis] * a[newaxis,:]&#xA;Out[7]: &#xA;array([[[ 1.,  1.,  1.,  1.,  1.],&#xA;        [ 1.,  1.,  1.,  1.,  1.]],&#xA;&#xA;       [[ 1.,  1.,  1.,  1.,  1.],&#xA;        [ 1.,  1.,  1.,  1.,  1.]]])&#xA;&#xA;In [8]: a.shape&#xA;Out[8]: (2, 5)&#xA;&#xA;In [9]: b = a[:,newaxis] * a[newaxis,:]&#xA;&#xA;In [10]: b.shape&#xA;Out[10]: (2, 2, 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="53" LastActivityDate="2014-02-12T21:32:44.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10801" PostTypeId="2" ParentId="10746" CreationDate="2014-02-12T21:47:18.147" Score="1" Body="&lt;p&gt;Sparse matrix multiplication (that is, sparse matrix inversion/solution) is a &lt;em&gt;huge&lt;/em&gt; area of interest/research in the GPU community. Currently things are looking pretty good, but it's not anywhere near where it is in some other fields. (This is really hard.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the nuts-and-bolts level of an analyst using a practical FEA code, it's not there. All the big players are waiting to add/push support until the situation is a bit clearer.&lt;/p&gt;&#xA;" OwnerUserId="1056" LastActivityDate="2014-02-12T21:47:18.147" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10803" PostTypeId="2" ParentId="10796" CreationDate="2014-02-12T22:23:27.053" Score="7" Body="&lt;p&gt;Both Mike's &lt;a href=&quot;https://scicomp.stackexchange.com/a/10799/1744&quot;&gt;answer&lt;/a&gt; and Jed's &lt;a href=&quot;https://scicomp.stackexchange.com/a/10798/1744&quot;&gt;one&lt;/a&gt; describe well the XFEM/FEM dichotomy and correctly point out that the most important area of application is 3D Fracture Mechanics, where you have a crack, i.e. a displacement discontinuity across a surface inside your domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cracks are hard to model in classical FEM for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The mesh has to be congruent across the crack: more precisely the crack has to be at the boundary of a subdomain of FE's. The crack &lt;em&gt;cannot&lt;/em&gt; lay inside (pass though) a finite element.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The singular stress field at crack tip requires special elements and/or meshing techniques (quarter point elements, focused mesh) to be modelled with good accuracy.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From an engineering point of view in fracture mechanics you have two main types of problems:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Stress_intensity_factor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stress intensity factor&lt;/a&gt; computation,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;crack propagation analysis, e.g. in fatigue or damage tolerance analysis.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the first type of problem classical FEM is more than adequate and is &lt;em&gt;the&lt;/em&gt; standard engineering tool. (This is because, fortunately, there are energy methods to evaluate the stress intensity factors that are not sensitive to numerical errors near the crack tip.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Crack propagation analysis is a completely different story: in most cases you do not know beforehand the crack path, and therefore frequent remeshing is necessary. The major promise of XFEM is to allow for crack propagation inside a &lt;em&gt;fixed&lt;/em&gt; FEM mesh, the crack cutting his way not only at the boundary between subdomains, but &lt;em&gt;inside&lt;/em&gt; the FE's themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;XFEM is a relatively new technique, still far from being a standard engineering tool. My answer to the the OP question, at least in solid mechanics and engineering analysis, is that XFEM has a very narrow and specialised application field in crack and damage propagation analysis, for complex 3D geometries, when the crack path cannot be estimated &lt;em&gt;a priori&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless let me stress that fracture mechanics is a very important field in engineering: e.g. today's aiplanes are safe also because it is possible to numerically predict damage and crack propagation between maintenance intervals. XFEM, or similar new techniques, are to become important tools in the near future.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-02-12T22:23:27.053" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10804" PostTypeId="1" CreationDate="2014-02-13T02:23:22.563" Score="1" ViewCount="99" Body="&lt;p&gt;Suppose a real, dense and asymmetric square matrix $A\in\mathbb{R}^{n\times n}$,&#xA;all its eigenvalues $\lambda_i \in \mathbb R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to construct a symmetric matrix $B\in\mathbb{R}^{n\times n}$, the eigenvalues of which are $\lambda_i^2$ to compute absolute values of $\lambda_i$ without the need of solving eigenvalues of $A$?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2014-02-13T12:34:25.710" Title="Is it possible to construct such a symmetric matrix with desired eigenvalues?" Tags="|matrix|eigenvalues|dense-matrix|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="10805" PostTypeId="1" AcceptedAnswerId="10808" CreationDate="2014-02-13T08:51:25.540" Score="2" ViewCount="128" Body="&lt;p&gt;Short version of question: We have a Opteron 6272 cluster, an we have recently started bilding FX 9370 based small cluster. Problem is that we are not able to get desired speed, i.e scaling with number or processes is not optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Full story: When we ran CPMD on Opteron 6272 as a single process, on a single node (hence without any parallel component) we got execution time of say 44 sec. when same thing was repeated on FX 9370 time to finish up job was ~27 sec. but as number of processes are increased (max=8) opteron 6272 scales up very nicely as compared to 9370 and at towards the end manages to just better the 9370 performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As processor is almost twice as fast (as given by single process performance), shouldn't i expect similar trend in parallel performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MPICH2 was compiled with ACML, and x86_open64 compilers from amd to avoid any bottleneck. preformance with intel compilers was also same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link to Intel MPI benchmarking results. &lt;a href=&quot;http://pastebin.com/keACK8Qz&quot; rel=&quot;nofollow&quot;&gt;http://pastebin.com/keACK8Qz&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7333" LastEditorUserId="7333" LastEditDate="2014-02-13T09:02:57.363" LastActivityDate="2014-02-13T13:38:43.873" Title="Optimizing MPICH2 on FX 9370 processor" Tags="|parallel-computing|computational-chemistry|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10806" PostTypeId="2" ParentId="10776" CreationDate="2014-02-13T09:03:16.690" Score="4" Body="&lt;p&gt;For generic data the optimal value (in the limit) is zero, which is achieved by any vector with arbitrarily large elements. Just pick a random vector $x_0$ and use $x=\alpha x_0$ where  $\alpha$ tends to $\infty$. Am I missing something?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigenvalue discussion you have is not correct (there is no reason that you should be allowed to add the constraint $y^Ty=1$ as you do, it seems to be blindly done by mimicking what is done when you have a ratio of two quadratics, of which one is $y^Ty$, and the objective thus is homogenous and unaffected by scaling $y$)&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2014-02-13T09:03:16.690" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10808" PostTypeId="2" ParentId="10805" CreationDate="2014-02-13T13:38:43.873" Score="2" Body="&lt;p&gt;The single-node performance of all of the MPI stacks I know of has always been very good. They all used shared memory behind the scenes to move data around between tasks. Your IMB results look fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are two very different processors, and you may be getting killed by the difference in memory bandwidth available to you. The 9370 is a desktop processor, and the 6272 is designed for servers. I haven't done the complete research, but it's very possible that while a single core is better on the 9370, you've run out of memory bandwidth by the time you get to 8 cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Less likely but also possible:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There's some problem with task pinning that preventing optimal layout of the tasks on the node&lt;/li&gt;&#xA;&lt;li&gt;Frequency scaling under Linux is turning down the clock frequency more on the 9370 as you scale up to conserve power or keep the processor cool&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can you post your CPMD results as an update?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-02-13T13:38:43.873" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10809" PostTypeId="1" CreationDate="2014-02-13T17:00:40.040" Score="2" ViewCount="1665" Body="&lt;p&gt;&lt;strong&gt;Original Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiarising myself with OpenFOAM and want to run a simple pipe flow case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PY2QA.png&quot; alt=&quot;Schematic of the test case&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to be able to compare my computational pipe-flow results to a well established case (like the &lt;em&gt;lid-driven cavity flow&lt;/em&gt; or &lt;em&gt;Ahmed's body&lt;/em&gt;): given certain $L,\,D,\,v_{1},\,p_{2}$ (atmosphere) and $\mu,\,\rho$ of the fluid, what ${\bf U}$ and ${\bf p}$ fields would I expect to get.&#xA;There seems to be little (&lt;a href=&quot;http://www.somogyibence.hu/documents/Cornell1/Cornell1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;http://www.mne.psu.edu/cimbala/Learning/CFD/Tutorial_Straight_Pipe_Pointwise_OpenFOAM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;) information online; especially with a clear definition of inputs and the expected output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have set up a case in OpenFOAM using:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$D=0.0008 \, m$&lt;/li&gt;&#xA;&lt;li&gt;$L=0.02 \, m$&lt;/li&gt;&#xA;&lt;li&gt;$v=0.1 \, m/s$&lt;/li&gt;&#xA;&lt;li&gt;$p_{2}=0 Pa$&lt;/li&gt;&#xA;&lt;li&gt;$\mu=0.001 \, N \, s/m^2$ (in OpenFOAM one enters the kinematic viscosity $\nu=0.000001 \, m^{2}/s$)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The solver gives me a solution of $p_{1}=0.1 \, Pa$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $Re=\frac{\rho v D}{\mu}=80$ (the flow is laminar), so (as mentioned by @Bill) I can use the Hagen-Poiseuille equation $dp=\frac{128 \mu L v A}{\pi D^4}$. It gives me a predicted $p_{1}=100 \, Pa$ -- 1000 times higher than the simulation result. The value seems to be correct, but there is a problem with the magnitude, but I can't spot where the mixup is occurring. As far as I know OpenFOAM is operating in SI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the screenshot of the pressure distribution in the pipe.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/jKbB5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4469" LastEditorUserId="4469" LastEditDate="2014-11-19T14:18:05.340" LastActivityDate="2018-03-06T22:45:29.497" Title="A Simple Standard Test Case for Pipe Flow" Tags="|fluid-dynamics|openfoam|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10810" PostTypeId="1" AcceptedAnswerId="10818" CreationDate="2014-02-13T21:42:20.437" Score="2" ViewCount="456" Body="&lt;p&gt;Suppose that $A(x) \in \mathbb{R}^{m \times n}$ is a nonlinear matrix function of $x \in \mathbb{R}^d$. We may assume that $A(x)$ is continuously differentiable. Are there any good ways to estimate $\mathrm{argmin}_{x} \|A(x)\|_2$? Also assume that due to some growth properties of the function $A(x)$, this minimizer can only be achieved within a known compact set. There are well known methods when $A(x)$ is linear with respect to $x$, but I don't have that linearity here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to implement a few quasi-Newton methods with varying success, and I've also tried MATLAB's &lt;code&gt;fminunc&lt;/code&gt; command with reasonable success (stabilizing around local minima is a problem). Unfortunately I'm relatively unfamiliar with the literature on these things (and the literature is quite vast). Are there any general approaches anybody could suggest?&lt;/p&gt;&#xA;" OwnerUserId="3477" LastActivityDate="2014-02-15T21:23:46.363" Title="Nonlinear optimization for minimizing matrix norm" Tags="|optimization|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10811" PostTypeId="1" CreationDate="2014-02-14T01:41:49.277" Score="2" ViewCount="1224" Body="&lt;p&gt;I am trying to solve a least squares problem where the objective function has a least squares term along with L1 and L2 norm regularization. I am unable to find which matlab function provides the ability to perform such an optimization in addition to specifying constraints. I looked into the MATLAB optimization toolbox which also does not provide too much freedom to specify own objective functions (although I hope I am wrong in this case!) in the case of leastSquares optimization problems. If anyone knows how to model such an optimization problem in matlab please do help me out. The optimization problem is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jbl02.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;strong&gt;aj&lt;/strong&gt; is an m X 1 vector and &lt;strong&gt;A&lt;/strong&gt; is an mXn matrix , &lt;strong&gt;wj&lt;/strong&gt; is an n x 1 vector. I don't know how to incorporate the two additional regularization terms into the optimization problems as they only accept 2 matrices and perform the least squares operation on them. I would like to know how to implement this objective function and pass it into matlab optimization toolbox leastSquares function. Any help would be appreciated. If not in the optimization toolbox, alternative implementations of this problem in &lt;strong&gt;CPLEX using matlab&lt;/strong&gt; would also be helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="7339" LastActivityDate="2014-02-14T05:36:35.647" Title="Solve Regularized Least Squares problems using Matlab optimization toolbox" Tags="|matlab|convex-optimization|least-squares|constrained-optimization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10812" PostTypeId="2" ParentId="10811" CreationDate="2014-02-14T04:54:35.580" Score="1" Body="&lt;p&gt;This isn't a linear least squares problem as it is written.  However, the problem does simplify down to a quadratic optimization problem with nonnegativity constraints and you can even write it as a linear least squares problem with nonnegativity constraints.  .  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, since $w_{j,j}$ is constrained to be 0, it has no effect on the objective function.  Let $\bf{w}$ be the $n-1$ element vector consisting of the elements of $\bf{w}_{j}$ other than $w_{j,j}$.  You can simply remove column $j$ from the $A$ matrix and solve for the $n-1$ by $1$ vector $w$.  After you've found $\bf{w}$, you can insert the $0$ in the $j$th position.  Let $\hat{A}$ be the $A$ matrix with the $j$th column removed.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, since $\bf{w}_{j} \geq 0$, $\| \bf{w}_{j} \|_{1}$ is simply the sum of the elements of the $\bf{w}$ vector.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you can write your problem in terms of the $w$ vector as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_{{\bf w}}  \;\;  \| a_{j} - \hat{A} {\bf w} \|_{2}^{2} + \frac{\beta}{2} \| {\bf w} \|_{2}^{2} + \lambda \sum_{k=1}^{n-1} w_{k}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bf{w} \geq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can rewrite the objective function in quadratic form as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_{{\bf w}} \;\; (a_{j}-\hat{A}{\bf w})^{T}(a_{j}-\hat{A}{\bf w})+\frac{\beta}{2}{\bf w}^{T}{\bf w} + \lambda \sum_{k=1}^{n-1} w_{k}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bf{w} \geq 0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Combining the quadratic parts of the first two terms and combining the linear terms and the constant term gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_{{\bf w}}\;\; {\bf w}^{T}(\hat{A}^{T}\hat{A}+\frac{\beta}{2}I){\bf w} + (-2\hat{A}^{T}a_{j}+\lambda {\bf 1})^{T}{\bf w} + a_{j}^{T}a_{j}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bf{w} \geq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, you could &quot;complete the square&quot; to turn this into a nonnegative least squares problem and then use the MATLAB function lsqnonneg to solve the problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simpler solution (that would probably perform equally well) is to use the Optimization Toolbox function quadprog to solve the problem.  CPLEX could also easily solve the quadratic programming problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't said anything about the size of the problem- if the problem is particularly large then it may be very important to pay attention to the algorithm used to solve the problem.  In particular, if $A$ is sparse, then you will want to make sure that your solver can take advantage of that fact.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2014-02-14T05:36:35.647" LastActivityDate="2014-02-14T05:36:35.647" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10813" PostTypeId="1" AcceptedAnswerId="10844" CreationDate="2014-02-15T00:43:57.137" Score="3" ViewCount="1278" Body="&lt;p&gt;I am trying to implement the N-body problem using Fortran 95. Before generalizing it to N bodies, I am first trying to calculate the motion of a single orbiting body (body 2 in my code below) around a fixed one (body 1 in the code below). The problem is the output is not correct, but I can't find the error with my equations! This is what I am trying to implement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_{2,x} \leftarrow V_{2,x} - \frac{ G \times M_1 \times (x_2-x_1)}{r^3} \times dt$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_{2,y} \leftarrow V_{2,y} - \frac{ G \times M_1 \times (y_2-y_1)}{r^3} \times dt$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_2 \leftarrow x_2 + V_{2,x} \times dt$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y_2 \leftarrow y_2 + V_{2,y} \times dt$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program gravitation&#xA;implicit none&#xA;&#xA;integer :: k, total&#xA;real, dimension(2) :: M, x, y, Vx, Vy&#xA;real, parameter :: G = 6.6738E-11, pi = 3.1415926&#xA;real :: time, dt, r&#xA;&#xA;dt = 1000. ! dt = 1000 seconds&#xA;total = 5000 ! total number of points to be output &#xA;&#xA;M(1)=1.9891E+09 ! Sun's mass&#xA;x(1)=0.&#xA;y(1)=0.&#xA;Vx(1)=0.&#xA;Vy(1)=0.&#xA;&#xA;M(2)=5.9736E+03 ! Earth's mass&#xA;x(2)=1.4960E+11 ! Earth's average orbit as initial point&#xA;y(2)=0.&#xA;Vx(2)=0.&#xA;Vy(2)=2.9786E+04 ! Earth's average orbital speed&#xA;&#xA;open(unit=101,file=&quot;a.txt&quot;)&#xA;write(101,*) &quot;time, x, y&quot;&#xA;write(101,&quot;(E10.4,2(A1,1x,E10.4))&quot;) 0., &quot;,&quot;, x(2), &quot;,&quot;, y(2)&#xA;&#xA;do k=1,total&#xA;  r = sqrt( (x(2)-x(1))**2 + (y(2)-y(1))**2 )&#xA;  Vx(2) = Vx(2) - G * M(1) * dt * (x(2)-x(1)) / r**3 ! i.e., Vx2 = Vx2 - G.M.dt.(x2-x1)/r^3&#xA;  Vy(2) = Vy(2) - G * M(1) * dt * (y(2)-y(1)) / r**3&#xA;    x(2) = x(2) + Vx(2) * dt&#xA;    y(2) = y(2) + Vy(2) * dt&#xA;  write(101,&quot;(E10.4,2(A1,1x,E10.4))&quot;) k*dt, &quot;,&quot;, x(2), &quot;,&quot;, y(2)&#xA;enddo&#xA;&#xA;end program gravitation&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output file shows that the x position of the second body does not change at all (it is stuck at the initial value of 0.1496E+12), while the y position changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone see what is the problem here? Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="7348" LastActivityDate="2014-02-22T22:31:31.587" Title="2-body problem with Fortran 95" Tags="|fortran|newton-method|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="10814" PostTypeId="1" CreationDate="2014-02-15T10:36:09.427" Score="8" ViewCount="430" Body="&lt;p&gt;I asked a somewhat similar question previously but perhaps it might have been too specific for anyone to really answer. Here is a bit more general of a question that I am struggling with. Consider the following system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-\nabla\cdot(D_{1}(u_{2})\nabla u_{1}) = \nabla\cdot\mathbf{f}_{1}(u_{2})&#xA;$$&#xA;$$&#xA;\frac{\partial u_{2}}{\partial t} + \nabla\cdot\mathbf{f}_{2}(u_{1},u_{2}) - \nabla\cdot(D_{2}(u_{2})\nabla u_{2}) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;assuming a general set of BCs:&#xA;$$&#xA;u_{i} = u_{i,D}, \;\;\;\text{on}\;\Gamma_{D}&#xA;$$&#xA;$$&#xA;D_{i}\nabla u_{i}\cdot\mathbf{n} = u_{i,N}, \;\;\;\text{on} \;\Gamma_{N}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using DGFEM for spatial discretizations and backwards Euler for the time derivative. We decouple like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Solve for $u_{1}^{k+1}$ using $u_{2}^{k}$:&#xA;$$&#xA;-\nabla\cdot(D_{1}(u_{2}^{k})\nabla u_{1}^{k+1}) = \nabla\cdot\mathbf{f}_{1}(u_{2}^{k})&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Solve for $u_{2}^{k+1}$ using $u_{1}^{k+1}$:&#xA;$$&#xA;\frac{\partial u_{2}}{\partial t} + \nabla\cdot\mathbf{f}_{2}(u_{1}^{k+1},u_{2}^{k},u_{2}^{k+1}) - \nabla\cdot(D_{2}(u_{2}^{k+1})\nabla u_{2}^{k+1}) = 0&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Newton's method is used to handle the non-linearity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the particular example I'm looking at, I have&#xA;$$&#xA;\mathbf{f}_{2}(u_{1},u_{2}) = u_{2}(-\nabla u_{1} + u_{2}\nabla u_{2})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{f}_{2}(u_{1}^{k+1},u_{2}^{k},u_{2}^{k+1}) = u_{2}^{k+1}(-\nabla u_{1}^{k+1} + u_{2}^{k}\nabla u_{2}^{k})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that the evaluation of $\mathbf{f}_{2}$ at $u_{1}^{k+1}$ and $u_{2}^{k}$ is really messing up my solution of $u_{2}^{k+1}$ to the point where Newton's method is not converging quadratically and I'm not observing the correct convergence of the discretization errors. I originally wanted to evaluate $\mathbf{f}_{2}$ at $u_{2}^{k}$ so that the vector part, $-\nabla u_{1}^{k+1} + u_{2}^{k}\nabla u_{2}^{k}$ could be treated as a constant at each Newton iteration. Otherwise, it would have to be added to the second order term, which I wanted to avoid doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To test my code, I'm using method of manufactured solutions. As a test, I'm considering the second equation by itself and evaluating $\mathbf{f}_{2} = \mathbf{f}_{2}(u_{1,\text{exact}}^{k+1}, u_{2}^{k+1}, u_{2,\text{exact}}^{k+1}) = u_{2}^{k+1}(-\nabla u_{1,\text{exact}}^{k+1} + u_{2,\text{exact}}^{k+1}\nabla u_{2,\text{exact}}^{k+1})$ using the exact solutions and there's, no problem here. Newton's method converges quadratically, I'm observing correct convergence of discretization error, etc. so I'm pretty sure the error here lies with the decoupling and not the discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this suboptimal performance in the first scenario expected given the decoupling? I know the decoupling will limit the time step, but I'm pretty sure I'm taking the time step sufficiently small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step I was going to take was to split up $\mathbf{f}_{2}$ and toss the $u_{2}\nabla u_{2}$ into the second order term. This would have the advantage of making the formulation more implicit since the only explicit part would be the use of $u_{1}^{k+1}$ in the convective term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hadn't been able to find much literature on what effect decoupling would have on convergence, so if anyone could point me in the right direction or offer some advice, that would be great.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2014-02-15T10:46:23.610" LastActivityDate="2014-03-20T16:13:18.550" Title="The effect of decoupling a coupled system of PDEs" Tags="|pde|finite-element|reference-request|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10815" PostTypeId="1" AcceptedAnswerId="11407" CreationDate="2014-02-15T11:56:54.047" Score="10" ViewCount="5475" Body="&lt;p&gt;Is there any &lt;strong&gt;simple rule of thumb&lt;/strong&gt; to say if it is worth to do SOR instead of Gauss-Seidel? ( and possible way how to estimate realxation parameter $\omega$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean just by &lt;strong&gt;looking on the matrix&lt;/strong&gt;, or knowledge of &lt;strong&gt;particular problem&lt;/strong&gt; the matrix represents?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was reading answer on this questions:&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/851/are-there-any-heuristics-for-optimizing-the-successive-over-relaxation-sor-met&quot;&gt;Are there any heuristics for optimizing the successive over-relaxation (SOR) method?&lt;/a&gt;&#xA;but it's a bit too sophisticated. I don't see simple heuristics how to estimate spectral radius just looking on the matrix ( or problem which it represents ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like something much simpler - &lt;strong&gt;just few examples&lt;/strong&gt; of matrices (problems) for which SOR converge faster.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I was experimenting with SOR for matrix of this king:&#xA;$A = I + C + R $ &#xA;where $I$ is identity matrix, $C_{ij}=c$ $ \forall i,j$ and $R_{ij}$s are random numbers from unifrom distribution such that $|R_{ij}|&amp;lt;r$. I was thinking that there would be some dependence of optimal $\omega$ on parameters $c,r$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I used very small $c,r$ to make sure tha $A$ is strongly diagonally dominant. ( $|c|&amp;lt;0.1$, $r&amp;lt;2|c|$ for matrix of dimension 5-10 ). I should also say that these $A$ was real and symmetric. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;strong&gt;I found that Gauss-Seidel ($\omega=1$) is almost always the best (?)&lt;/strong&gt;. Does this means that there has to be some more correlation between $A_{ij}$s to get advantage of SOR ? Or I did something wrong?   &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I know, that SOR is not the most efficient solver (in comparison to CG,GMRES ... ) but it is simple to implement and paraelize, and modify for particular problem. Sure good for prototyping.&lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-05-27T20:41:11.700" Title="Sort of problems where SOR is faster than Gauss-Seidel?" Tags="|linear-solver|iterative-method|heuristics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10816" PostTypeId="2" ParentId="10747" CreationDate="2014-02-15T17:58:39.707" Score="4" Body="&lt;p&gt;I'd imagine that most users of random number generators are ultimately interested in floating-point values. This is why the &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index.html#dSFMT&quot; rel=&quot;nofollow&quot;&gt;Double precision SIMD-oriented Fast Mersenne Twister&lt;/a&gt; (dSFMT) exists. However, there is &lt;a href=&quot;http://www3.ocn.ne.jp/~harase/megenerators.html&quot; rel=&quot;nofollow&quot;&gt;newer C code for the WELL RNG&lt;/a&gt; that returns &lt;code&gt;unsigned long&lt;/code&gt; values. Looking at the code, it appears that the earlier version was casting &lt;code&gt;unsigned long&lt;/code&gt; to &lt;code&gt;double&lt;/code&gt; so you may be able to figure out what they were doing by comparing the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mainly care about speed, I don't think that this is the PRNG for you. The speed/performance that the authors refer to is not how fast the algorithm generates random variates, but rather &lt;a href=&quot;http://en.wikipedia.org/wiki/Mersenne_twister#SFMT&quot; rel=&quot;nofollow&quot;&gt;how quickly it recovers from a poor initialization&lt;/a&gt;. This improves the statistical properties of the WELL generator. See section 6 of &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/M062821.pdf&quot; rel=&quot;nofollow&quot;&gt;Mutsuo Saito's Master's thesis&lt;/a&gt; (PDF) for details on what this means. Such tests are performed with artificial worst-case initial states so it's unclear to me what the effect is in general use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Random number generation can be quite expensive—often despite one's intuition—and is a very good candidate for optimization provided that your code is spending a reasonable amount of time producing random values. In my own simple performance tests (Retina MacBook Pro, OS X 10.9) I found the &lt;a href=&quot;http://www3.ocn.ne.jp/~harase/megenerators.html&quot; rel=&quot;nofollow&quot;&gt;WELL RNG&lt;/a&gt; to be about &lt;em&gt;one third as fast&lt;/em&gt; as the current integer-based &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index.html&quot; rel=&quot;nofollow&quot;&gt;SIMD-oriented Fast Mersenne Twister&lt;/a&gt; (SFMT) when producing &lt;code&gt;uint32&lt;/code&gt; values. &lt;code&gt;SFMT&lt;/code&gt; also has support for &lt;code&gt;uint64&lt;/code&gt;. I also found the  classic &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/emt19937ar.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;mt19937ar.c&lt;/code&gt;&lt;/a&gt; code to be about &lt;em&gt;half as fast&lt;/em&gt; as &lt;code&gt;SFMT&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="4474" LastEditDate="2014-02-17T16:10:24.200" LastActivityDate="2014-02-17T16:10:24.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10817" PostTypeId="2" ParentId="10815" CreationDate="2014-02-15T18:28:43.800" Score="0" Body="&lt;p&gt;This side of things isn't really my speciality, but I don't think this is a super-fair test for many realistic applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what values you were using for &lt;em&gt;c&lt;/em&gt; and &lt;em&gt;r&lt;/em&gt;, but I suspect you were working with extremely ill-conditioned matrices. (Below is some Python code showing that these might not be the most invertible matrices.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import numpy&#xA;&amp;gt;&amp;gt;&amp;gt; for n in [100, 1000]:&#xA;...     for c in [.1, 1, 10]:&#xA;...         for r in [.1, 1, 10]:&#xA;...             print numpy.linalg.cond(&#xA;...                 numpy.eye(n) + &#xA;...                 c * numpy.ones((n, n)) + &#xA;...                 r * numpy.random.random((n, n))&#xA;...             )&#xA;... &#xA;25.491634739&#xA;2034.47889101&#xA;2016.33059429&#xA;168.220149133&#xA;27340.0090644&#xA;5532.81258852&#xA;1617.33518781&#xA;42490.4410689&#xA;5326.3865534&#xA;6212.01580004&#xA;91910.8386417&#xA;50543.9269739&#xA;24737.6648458&#xA;271579.469212&#xA;208913.592289&#xA;275153.967337&#xA;17021788.5576&#xA;117365.924601&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you actually needed to invert matrices this ill-conditioned, you'd a) use a specialized method, and b) should probably just go find a new field &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For well-conditioned matrices of any size, SOR is likely to be faster. For real problems where speed matters, it would be rare to use SOR -- on the sophisticated side, there's much better these days; on the slow but reliable side, SOR isn't the best you can do.&lt;/p&gt;&#xA;" OwnerUserId="1056" LastActivityDate="2014-02-15T18:28:43.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10818" PostTypeId="2" ParentId="10810" CreationDate="2014-02-15T21:23:46.363" Score="1" Body="&lt;p&gt;This seems to be exactly the problem addressed in the following paper. Just recast the problem of minimizing $||A||_2$ into a problem of minimizing the maximum eigenvalue of $A^T A$.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Overton, Michael L., and Robert S. Womersley. &quot;Second derivatives for optimizing eigenvalues of symmetric matrices.&quot; SIAM Journal on Matrix Analysis and Applications 16.3 (1995): 697-718.  &lt;a href=&quot;http://ftp.cs.nyu.edu/cs/faculty/overton/papers/pdffiles/eighess.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ftp.cs.nyu.edu/cs/faculty/overton/papers/pdffiles/eighess.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2014-02-15T21:23:46.363" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10819" PostTypeId="1" CreationDate="2014-02-15T22:38:39.923" Score="2" ViewCount="180" Body="&lt;p&gt;Can you show me the main differences between 2 methods? I find out 2 reasons but I don't know they are right or not.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;XFEM is constructed base on enrichment functions whereas P1-bubble is constructed base on bubble functions. Basically, 2 types of these functions are different (what are these differences?).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For $P_k$-bubble: It is not necessary to consider a space of polynomials of degree of $k+1$, it just needs to add more one extra degree of freedom to the barycenter of &lt;strong&gt;every simplex of the triangulation&lt;/strong&gt; $T_h$ of the domain $\Omega$. Whereas,  xFEM is only considers the enrichment function at &lt;strong&gt;some nodes&lt;/strong&gt; relating to the cracks (or the main problem).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4670" LastActivityDate="2017-01-08T09:43:38.450" Title="Extended finite element method vs $P_k$-bubble element" Tags="|finite-element|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10820" PostTypeId="1" AcceptedAnswerId="10822" CreationDate="2014-02-15T23:35:47.063" Score="1" ViewCount="1338" Body="&lt;p&gt;I am working with &lt;a href=&quot;http://en.wikipedia.org/wiki/Biofilm&quot; rel=&quot;nofollow&quot;&gt;biofilm&lt;/a&gt; and there are many documents talking about boudary conditions while others talks about interface or both of boundary and interface. So, boundary and interface are the same (in numerical analysis) or not? why and examples?&lt;/p&gt;&#xA;" OwnerUserId="4670" LastActivityDate="2014-02-16T02:12:55.023" Title="&quot;boundary&quot; vs &quot;interface&quot;?" Tags="|fluid-dynamics|numerical-analysis|computational-biology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10822" PostTypeId="2" ParentId="10820" CreationDate="2014-02-16T01:59:40.990" Score="3" Body="&lt;p&gt;I'm not particularly familiar with biofilm literature.  But in most computational literature, the border of the entire domain of a problem are usually referred to as the &lt;em&gt;boundary&lt;/em&gt;.  Outside of a boundary, there are no nodes, elements, or anything else under consideration.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entire domain may also be subdivided into smaller regions.  Some of these regions share edges on the &lt;em&gt;boundary&lt;/em&gt; of the domain.  However, some of these regions share borders between each other that are not boundaries of the domain.  Usually these regions have different material properties or different relevant physics and may also have different meshes.  The borders between any two regions in a domain are usually referred to as &lt;strong&gt;&lt;em&gt;interfaces&lt;/em&gt;&lt;/strong&gt;. &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-02-16T02:12:55.023" LastActivityDate="2014-02-16T02:12:55.023" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10824" PostTypeId="2" ParentId="10814" CreationDate="2014-02-16T06:48:11.930" Score="1" Body="&lt;p&gt;It's a bit hard to say without knowing precisely what PDE you're solving; that said, a few places to maybe look are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For coupled &lt;em&gt;linear&lt;/em&gt; systems, Uzawa and augmented Lagrangian methods are examples of general splitting schemes. There are known results concerning convergence of these methods based on the spectra of the blocks and the Schur complement of the coupled block system (see Bramble et al, on solving saddle point problems). &lt;/li&gt;&#xA;&lt;li&gt;There's a wealth of material on different &lt;a href=&quot;http://www.mathematik.uni-dortmund.de/~kuzmin/cfdintro/lecture11.pdf&quot; rel=&quot;nofollow&quot;&gt;splitting methods&lt;/a&gt; for PDEs. Though these are largely formulated for linear problems, there is ample use and theory for their extension to nonlinear equations - for example, at least for the nonlinear PDEs governing types of convection-diffusion and incompressible flow, splitting/projection schemes have a pretty rich &lt;a href=&quot;http://www.math.ust.hk/~mawang/teaching/math532/guermond-shen-2006.pdf&quot; rel=&quot;nofollow&quot;&gt;history&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One thought is that solutions to the decoupled PDEs may or may not be close to solutions of the full coupled systems. Thus, solving only one equation &quot;exactly&quot; with Newton might not buy you more (or may be worse) than simply taking one single Newton iteration and updating both solutions (i.e. taking a small implicit step, then updating $u_1$ and $u_2$ again, instead of solving Newton for $u_1$ and $u_2$ separately at each implicit step). But then again, I'm not sure about this.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2014-02-16T07:05:39.447" LastActivityDate="2014-02-16T07:05:39.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10825" PostTypeId="2" ParentId="10815" CreationDate="2014-02-16T13:01:20.857" Score="0" Body="&lt;p&gt;OK, so for symetric matrices of this king:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 t 0 0 0 0 t t 0 0 &#xA;t 1 0 0 0 0 0 0 0 0 &#xA;0 0 1 0 0 0 0 t 0 t &#xA;0 0 0 1 0 0 0 0 0 t &#xA;0 0 0 0 1 t 0 0 0 0 &#xA;0 0 0 0 t 1 0 t 0 0 &#xA;t 0 0 0 0 0 1 0 0 0 &#xA;t 0 t 0 0 t 0 1 0 0 &#xA;0 0 0 0 0 0 0 0 1 t &#xA;0 0 t t 0 0 0 0 t 1 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;SOR does converge faster than Gauss-Seidel if number of $t$s in each row is small ( much smaller than dimension of A ) and if all $t$s are similar. I was using $t$s generated like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_i = c + random(-r,r)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $t$s vary much and are centred arround 0 ( $c=0,r=0.1$ ) than Gauss-Seidel is faster. Gauss-Seidel is also faster if each row is more than half filled by $t$s. This also means that SOR is better for very big and very sparse matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;( This is just emperical observation, nothing rigorous )&lt;/p&gt;&#xA;" OwnerUserId="4696" LastActivityDate="2014-02-16T13:01:20.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10826" PostTypeId="1" AcceptedAnswerId="10885" CreationDate="2014-02-16T13:19:14.927" Score="4" ViewCount="1700" Body="&lt;p&gt;Assume that we have a function $u$ defined in a ball in a discrete way: we know only the values of $u$ in the nodes $(i,j,k)$ of spherical grid, where $i$ is a radius coordinate, $j$ is a coordinate for angle $\varphi$, $k$ is a coordinate for angle $\psi$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a vector-function &#xA;$$\nabla u_{i,j,k}=\left(\frac{\partial u}{\partial r}_{i,j,k},\frac{1}{r_i\sin\psi_k}\frac{\partial u}{\partial \varphi}_{i,j,k},\frac{1}{r_i}\frac{\partial u}{\partial \psi}_{i,j,k}\right)-$$&#xA;gradient of $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to know the values of $\nabla u_{i,j,k}$ on z-axis in cartesian coordinates, which corresponds to $\psi=0$ -- axis in spherical coordinates, but we can not use the formula above, because in case $\psi=0$ the second term turns to infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, we can find the values of $\frac{\partial u}{\partial z}$ with a help of the formula of numerical derivative, but we have a problem with finding $\frac{\partial u}{\partial x}$,$\frac{\partial u}{\partial y}$, because the grid is not rectangular. Could you help me with this stuff and advise me what to do?&lt;/p&gt;&#xA;" OwnerUserId="4676" LastActivityDate="2014-02-23T12:18:10.267" Title="Numerical gradient in spherical coordinates" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10827" PostTypeId="2" ParentId="10785" CreationDate="2014-02-16T16:49:44.323" Score="3" Body="&lt;p&gt;You may want to consider &lt;a href=&quot;http://www.pde2d.com&quot; rel=&quot;nofollow&quot;&gt;PDE2D&lt;/a&gt;.  Despite the name, it also applies to 3D problems.  The advantage of this software is that it does provide a GUI based interface.  You can download and use a free trial version for an unlimited amount of time, but there are limitations to the size of the problems that you can solve in the free version.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-02-16T17:34:41.590" LastActivityDate="2014-02-16T17:34:41.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10829" PostTypeId="2" ParentId="10707" CreationDate="2014-02-16T20:37:15.347" Score="2" Body="&lt;p&gt;If you look at it from a Galerkin perspective, the right side could also be approximated by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac16(f(u_{i-1})+4f(u_i)+f(u_{i+1})),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the Kepler formula, I believe. This would be valid for first sampling $f(u)$ and then approximating by a piecewise linear function. If the sampled $u$ is approximated piecewise linearly, then the non-linearity of $f$ will destroy this property, giving no general rule.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2014-02-16T20:37:15.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10830" PostTypeId="2" ParentId="10819" CreationDate="2014-02-16T22:41:04.517" Score="3" Body="&lt;p&gt;The difference between bubble-enriched and XFEM-enriched spaces is that in the former case, you do &lt;em&gt;the same thing&lt;/em&gt; on the same cell (i.e., you enrich with shape functions created with the same construction on every cell), whereas in the latter you add shape functions that are defined &lt;em&gt;globally&lt;/em&gt;, independent of the mesh. The former therefore fits well into the usual finite element implementations: they work well as they do the same thing on every cell. On the other hand, the XFEM is difficult to implement because it requires doing something different on every cell.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-16T22:41:04.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10832" PostTypeId="1" CreationDate="2014-02-18T03:44:54.203" Score="4" ViewCount="397" Body="&lt;p&gt;I am trying to solve the following system of differential equations numerically over the domain $x=0$ to $x=D$. The main difficulty is that the boundary conditions are interconnected and depend on the solution variable. I'm not too sure how to approach the finding a numerical solution for that reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dI_1}{dx} = -\alpha(x) I_1(x) \\&#xA;-\frac{dI_2}{dx} = -\alpha(x) I_2(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to the interconnected boundary conditions,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_1(0) =  AI_2(0) + C\\&#xA;I_2(D) = BI_1(D)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$, $B$, $C$ are known constants.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an optics problem and describes the multiple reflection of light entering and exiting a dielectric layer with surface reflection coefficients $A$ and $B$, absorption coefficient $\alpha$, $C=(1 - A)I_0$ where $I_0$ is the incident intensity (a known constant). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have divided the propagation into &lt;a href=&quot;http://en.wikipedia.org/wiki/Two-stream_approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;two streams&lt;/a&gt; which propagate downwards and upwards. This is a common approach for solving the &lt;a href=&quot;http://en.wikipedia.org/wiki/Radiative_transfer&quot; rel=&quot;nofollow noreferrer&quot;&gt;radiative transfer equatio&lt;/a&gt;n. The $I_1$ equation is the intensity of light propagating to the right, and $I_2$ is the intensity propagating backwards to the left. So the total intensity is $I(x) = I_1(x) + I_2(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zdgb0.png&quot; alt=&quot;Two-steams approximation&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analytical solution when $\alpha(x)$ is constant&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone is interested there is an analytical solution in the limit that $\alpha(x)\rightarrow\alpha$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I(x) = I_{1}\left(x\right)+I_{2}\left(x\right)=-\frac{{\left(Be^{\left(2\,\alpha x\right)}+e^{\left(2\,D\alpha\right)}\right)}Ce^{\left(-\alpha x\right)}}{AB-e^{\left(2\,D\alpha\right)}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when $A=B=0$ this reduces to the &lt;a href=&quot;http://en.wikipedia.org/wiki/Beer%E2%80%93Lambert_law&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beer-Lambert&lt;/a&gt; law as expected,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I(x) = I_0e^{-\alpha x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a useful expression because this way you don't have to calculate and keep track of the magnitude of the multiple reflections.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2014-02-18T18:57:01.490" Title="Solving system of differential equations with interconnected boundary conditions" Tags="|ode|simulation|numerical-analysis|boundary-conditions|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10833" PostTypeId="2" ParentId="10832" CreationDate="2014-02-18T05:02:36.757" Score="3" Body="&lt;p&gt;Assuming that $\alpha(x)$ has a closed form, I think you can use an integrating factor to integrate this exactly. Even if not, you should be able to write it in a form that's amenable to numeric integration (quadrature) rather than discretization with finite differences.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-02-18T05:02:36.757" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10834" PostTypeId="1" CreationDate="2014-02-18T17:22:10.573" Score="2" ViewCount="1331" Body="&lt;p&gt;I am trying to discretize the differential operator $\frac{d^2}{dx^2}$ acting on $S^1 = [0,1]$ using finitely many points around a circle at $0, \frac{1}{N}, \frac{2}{N}, \dots, \frac{N-1}{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the matrix for $N=5$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[ \begin{array}{cc} -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 \\&#xA;1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1 \\\&#xA;1 &amp;amp;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -2\end{array} \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we solved the equation $\frac{d^2 f}{dx^2} = \lambda f$ we should get $f(x) = e^{\lambda x}$ with $\lambda = 2\pi k$ and $k^2 \in \mathbb{Z}$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;When I implemented this in numpy, the reascaling seems all wrong.  Can someone explain a little bit of the theory behind this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 100&#xA;A = np.zeros((N,N))&#xA;d = np.arange(N)&#xA;A[d,d] = -2&#xA;A[d,(d+1)%N]=1&#xA;A[d,(d-1)%N]=1&#xA;A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The resulting matrix is like I said.  Here $N=100$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([[-2.,  1.,  0., ...,  0.,  0.,  1.],&#xA;       [ 1., -2.,  1., ...,  0.,  0.,  0.],&#xA;       [ 0.,  1., -2., ...,  0.,  0.,  0.],&#xA;       ..., &#xA;       [ 0.,  0.,  0., ..., -2.,  1.,  0.],&#xA;       [ 0.,  0.,  0., ...,  1., -2.,  1.],&#xA;       [ 1.,  0.,  0., ...,  0.,  1., -2.]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I came up with this adhoc scaling for the eigenvalues of $-A$ dividing them by $(2 \pi N)^2$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;L = np.linalg.eig(-A)[0]/(2*np.pi)**2*N**2&#xA;L = np.round(L)&#xA;np.sort(L).astype(int)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result looks pretty close.  I get the sequence of perfect squares.  Does that mean $\frac{1}{2\pi N}A$ is the correct rescaling?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([   0,    1,    1,    4,    4,    9,    9,   16,   16,   25,   25,&#xA;         36,   36,   48,   48,   63,   63,   79,   79,   97,   97,  116,&#xA;        116,  137,  137,  160,  160,  184,  184,  209,  209,  235,  235,&#xA;        263,  263,  291,  291,  320,  320,  350,  350,  381,  381,  412,&#xA;        412,  443,  443,  475,  475,  507,  507,  538,  538,  570,  570,&#xA;        602,  602,  633,  633,  663,  663,  693,  693,  722,  722,  751,&#xA;        751,  778,  778,  804,  804,  830,  830,  853,  853,  876,  876,&#xA;        897,  897,  916,  916,  934,  934,  951,  951,  965,  965,  978,&#xA;        978,  988,  988,  997,  997, 1004, 1004, 1009, 1009, 1012, 1012,&#xA;       1013])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="1804" LastEditDate="2018-06-29T07:19:26.713" LastActivityDate="2018-06-29T07:19:26.713" Title="Finite difference discretization on a circle" Tags="|finite-difference|discretization|numerics|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10835" PostTypeId="1" AcceptedAnswerId="10845" CreationDate="2014-02-18T13:58:09.227" Score="4" ViewCount="150" Body="&lt;p&gt;I am using a multigrid preconditioned GMRES method for a nonsymmetric matrix. The matrix is the discretisation of the derivative of a nonlinear operator. Since multigrid is not the best for nonsymmetric problems I am preconditioning the 'symmetric part' of the matrix, where I am taking the symmetric part of matrix $A$ to be $(A + A^{\text{T}})/2$, where $A^{\text{T}}$ is the transpose of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that the 'symmetric part' of the matrix that I get is not positive definite. I was wondering if there is a way to get a symmetric positive definite part of a matrix which is just some manipulation of the original matrix, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{A} = F(A)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $F$ some function of $A$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; From the comments and answers below I realise that I have not included a large part of the important information required for my question, so here is some supplementary information. I am using a standard geometric multigrid implementation with a pointwise Jacobi / Gauss-Seidel smoother. I am aware that using a smoothing operator that is tailored to the specific operator that I am dealing with will give better results, but I am assessing the effectiveness of the standard linear geometric multigrid implementation for some problems I am interested in. As such I ideally want to be applying the multigrid iteration to a symmetric positive definite matrix. I have information about the linear operator that I can use to get a matrix to precondition which is symmetric positive definite and 'close to' the matrix I am interested in inverting, which works quite well. What I am particularly interested in, though, is knowing if there is some 'black box' function which can give a symmetric positive definite matrix for the preconditioning that will be 'good' for a given matrix, without having knowledge of where the given matrix came from&lt;/p&gt;&#xA;" OwnerUserId="6602" OwnerDisplayName="Keeran Brabazon" LastEditorUserId="6602" LastEditDate="2014-02-19T14:50:42.360" LastActivityDate="2014-02-19T14:50:42.360" Title="Methods for calculating the symmetric part of a matrix" Tags="|linear-algebra|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10836" PostTypeId="2" ParentId="10832" CreationDate="2014-02-18T18:09:15.710" Score="1" Body="&lt;p&gt;You said that for the following problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dI_1}{dx} = -\alpha(x) I_1(x) \\&#xA;-\frac{dI_2}{dx} = -\alpha(x) I_2(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_1(0)=AI_2(0)+C\\&#xA;I_2(D)=BI_1(D)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the coefficient $\alpha(x)$ is piecewise constant. As I can se you can do it in an iterative way. You start with an initial guess for $I_1(0) = I_{1,0}$, and what you can do is iterate in time (with finite differences or with discontinuous Galerkin to have stable solution), until you reach $I_{1,n}\approx I_1(D)$. Then you use that value to calculate the inital guess for $I_{2,n}$ with the boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_{2,n} \approx I_2(D) = B I_1(D)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and you integrate in the oposite direction, reaching the position $0$ with $I_{2,0}$. With this value as an approximation for $I_{2}(0)$, you update the value for $I_{1}(0)$ usgin the boundary condition at $0$. You then iterate again to reach position $D$, use the approximation for the initial guess for $I_{2,n}$, and go againt until $0$. You perform this recursive scheme until you converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the scheme is not garanted to converge, because it depends on the initial guess. My experience is that these kind of schemes usually have good behaviour. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also rewrite your problem as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dI}{dx} = -\overline{\alpha(x)} I(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I(0)=\frac{A}{B} I(2D)+C\\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the new function $I$ and the new coefficient $\overline{\alpha}$ are defined over the domain $[0,2D]$ as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_{[0,D]}(x) = I_1(x) \\&#xA;I_{[D,2D]}(x) = \frac{1}{B} I_2(-x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\overline{\alpha}(x)_{[0,D]} = \alpha(x) \\&#xA;\overline{\alpha}(x)_{[D,2D]} = \alpha(-x) &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your function $I$ will be continuous over the domain $[0,2D]$ (it is continuous at $x=D$ because the different parts satisfy the boundary condition $\lim_{x\to D-}I(x) = I_1(D) = I_2(D)/B = \lim_{x\to D+ }$, and you still have the other boundary condition conecting the points $0$ and $2D$. So you can integrate in time and refeed the system, or you can build the whole system with for example discontinuous galerkin method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Remark:&lt;/strong&gt; If the coefficient function $\alpha$ is piecewise constant as you said, I dont think that you can solve it analyticaly. Only numerical solutions are possible. &lt;/p&gt;&#xA;" OwnerUserId="7277" LastEditorUserId="7277" LastEditDate="2014-02-18T18:57:01.490" LastActivityDate="2014-02-18T18:57:01.490" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10837" PostTypeId="2" ParentId="8062" CreationDate="2014-02-19T00:03:55.437" Score="5" Body="&lt;p&gt;The best thing to do here is to actually reduce the size of your problem after you have assembled the full system matrices. Let's assume you've gotten to that point, so you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{\tilde{K}d}=\mathbf{\tilde{F}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{K}$ is a $N_{dof} \times N_{dof}$ matrix of stiffness coefficients, $\mathbf{d}$ is your full vector of nodal degrees of freedom (most of which are known in your case), and $\mathbf{F}$ is a $N_{dof} \times 1$ vector of &quot;forces&quot; which arose from integration of the source terms and Neumann boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather than solving this very large system, you want to reduce the system down to ONLY the unknowns in $\mathbf{d}$! The method I will describe here is very general, and is not limited to specially-shaped domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, first (for the sake of simplicity) that the &quot;top&quot; half of your $\mathbf{d}$ vector contains your known values from the Dirichlet boundary conditions, and the bottom half contains your unknowns. Immediately, we see that we can simply ignore your first $N_{dof}/2$ equations. In practice, this means chopping off the first $N_{dof}/2$ rows of $\mathbf{\tilde{K}}$ and $\mathbf{\tilde{F}}$. Now we're left with a more manageable set of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{\hat{K}d}=\mathbf{\hat{F}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{\hat{K}}$ is a $\frac{N_{dof}}{2} \times N_{dof}$ matrix, and $\mathbf{\hat{F}}$ is a vector of length $N_{dof}/2$. We haven't yet done anything with $\mathbf{d}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we're going to reduce $\mathbf{d}$ down to only having the unknowns on the left side of the equation. Notice that the first $N_{dof}/2$ columns of $\mathbf{\hat{K}}$ are being multiplied with known values of $\mathbf{d}$. We will take advantage of this fact and move all of the known values to the right-hand side of the equation. That statement may have been a little cryptic, but the best way to illustrate it is with a little bit of code. I'm going to use a MATLAB-like syntax and take advantage of matrix slicing.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for ii=1:Ndof                            //loop through d&#xA;    if(isDirichletDOF(ii))               //proceed if d(ii) is known&#xA;        Fhat = Fhat - d(ii)*Khat(:,ii);&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the small snippet above, I only moved over columns associated with KNOWN values (ie, all of the dirichlet boundary values) and did not touch any of the others. Now that you've done this, you no longer need to worry about the known values of $\mathbf{d}$ any more, so just chop it down to a vector of only unknowns. Simultaneously, make a new matrix $\mathbf{K_{final}}$ from $\mathbf{\hat{K}}$ that uses only the columns that were not associated with known values. Now you finally have the system that you want to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{K_{final}d_{unknowns}} = \mathbf{\hat{F}_{modified}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your particular application, this will be MUCH MUCH faster than some other methods of applying boundary conditions. From here, you pick your favorite linear solver and go to town. I won't try to predict actual performance gains since that depends so much on the linear solver you use, but $\mathbf{K}$ is now $1/10,000$ the size that it used to be (if, for some reason you were not using a sparse matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, in practice, you don't know your node numbering ahead of time (for non-trivial geometries) so you just need to do some bookkeeping to determine which rows to get rid of and which columns to move to the right hand side of the equation.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2014-02-19T00:03:55.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10838" PostTypeId="1" CreationDate="2014-02-19T04:11:38.053" Score="3" ViewCount="951" Body="&lt;p&gt;I need a C# implementation of the gamma function that produces correct exact answers at positive integer inputs.  I took a look at MathNet.Numerics Meta.Numerics.  In both cases, if you calculate something like gamma(5)-4!, evaluating the latter by integer arithmetic, you get an answer whose absolute value is approximately 10^-14, instead of the expected zero.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could code in the integers as a special case.  But that would lead to some strange behavior; for example, my gamma function might not be monotonically increasing for x&gt;=2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is anyone aware of a library that will work?&lt;/p&gt;&#xA;" OwnerUserId="7381" LastActivityDate="2019-01-02T16:40:35.073" Title="C# implementation of the gamma function that produces correct answers at positive integer inputs?" Tags="|numerics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10839" PostTypeId="2" ParentId="10838" CreationDate="2014-02-19T05:47:09.637" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;http://git.savannah.gnu.org/cgit/gsl.git/tree/specfunc/gamma.c&quot; rel=&quot;nofollow&quot;&gt;GSL implementation of the Gamma function&lt;/a&gt; stores the integer factorial cases up to 297 for double precision (integers greater than that overflow), and uses a table of those values directly for integer arguments (see lines 42-555 or so). Any error you see in those values should be due to loss of precision from converting an integer to a double, or because the value is so large, you can't store the exact integer in a double and lose the least significant figures. The estimated absolute error is an machine epsilon times the function value, or roughly an ulp, which is about as good as you can get. For $\Gamma(5) = 4!$, this would also be around $10^{-14}$, at worst (I get $24 \times 2.22 \times 10^{-16} = 5.328 \times 10^{-15}$, which rounds up to $10^{-14}$); it is a cost of using double precision. If you need exact answers for integer arguments, and you will only have integer arguments between 0 and 18 inclusive, consider using a factorial function that returns long integers.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-02-19T05:47:09.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10841" PostTypeId="1" CreationDate="2014-02-19T07:46:08.193" Score="4" ViewCount="670" Body="&lt;p&gt;Following &quot;A conservative DGM for Convection-Diffusion and Navier-Stokes Problems&quot; (Oden and Baumann), if we have a linear convection-diffusion equation of the following form:&#xA;$$&#xA;\nabla\cdot(\mathbf{b}u) - \nabla\cdot(a\nabla u) = f&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with BC's&#xA;$$&#xA;u = u_{D} \;\;\text{on} \;{\Gamma_{D}} \;\;\;\text{and}\;\;\;a\nabla u\cdot\mathbf{n}=u_{N} \;\;\text{on} \;\Gamma_{N}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we have the variational formulation of the convection term as follows:&#xA;$$&#xA;A_{\mathbf{b}}(u,v) = -\sum_{K\in\mathcal{M}_{h}}\int_{K}\nabla v\cdot(\mathbf{b}u) + \sum_{e\in\Gamma_{h}^{i}}\int_{e}\left\{\mathbf{b}\cdot\mathbf{n}\right\}u^{\uparrow}[v] + \sum_{e\in\Gamma_{\text{out}}}\int_{e}(\mathbf{b}\cdot\mathbf{n})uv&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&#xA;$$&#xA;L_{\mathbf{b}}(v) = -\sum_{e\in\Gamma_{\text{in}}}\int_{e}(\mathbf{b}\cdot\mathbf{n})u_{D}v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Gamma_{h}^{i}$ is the set of interior edges, $\Gamma_{\text{in}} = \left\{\mathbf{x}\in\partial\Omega:\;\mathbf{b}\cdot\mathbf{n} &amp;lt; 0\right\}$ and $\Gamma_{\text{out}} = \partial\Omega\backslash\Gamma_{\text{in}}$. Also, we have the following upwind term:&#xA;$$&#xA;u^{\uparrow} = \begin{cases} u|_{K_{-}}, &amp;amp;\left\{\mathbf{b}\cdot\mathbf{n}\right\} \geqslant0\\ u|_{K_{+}}, &amp;amp;\left\{\mathbf{b}\cdot\mathbf{n}\right\} &amp;lt; 0\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K_{-}$ is the element to the &quot;left&quot; of an interior edge $e$ and $K_{+}$ is the element to the &quot;right&quot;, using a fixed convention for defining the neighbors of an edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I can implement this without much trouble. However, I'm trying to solve a non-linear equation in the following form:&#xA;$$&#xA;\nabla\cdot(\gamma(u)\mathbf{b}(u)) - \nabla\cdot(a(u)\nabla u) = f(x,y)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having some trouble figuring out the convective term here. I have:&#xA;$$&#xA;A_{\mathbf{b}}(u,v) = -\sum_{K\in\mathcal{M}_{h}}\int_{K}\nabla v\cdot(\gamma(u)\mathbf{b}(u)) + \sum_{e\in\Gamma_{h}^{i}}\int_{e}\left\{\mathbf{b}(u)\cdot\mathbf{n}\right\}(\gamma(u))^{\uparrow}[v] + \sum_{e\in\Gamma_{\text{out}}}\int_{e}(\mathbf{b}(u)\cdot\mathbf{n})\gamma(u)v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&#xA;$$&#xA;L_{\mathbf{b}}(u) = -\sum_{e\in\Gamma_{\text{in}}}\int_{e}(\mathbf{b}(u)\cdot\mathbf{n})\gamma(u_{D})v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I'm trying to compute a Jacobian from these terms and I think there may be something wrong with my integration by parts. Is my averaging and upwinding correct? Namely, I'm a bit unsure of where the $\uparrow$ should go. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) On the inflow integral, do I evaluate $\mathbf{b}(u_{D})$ AND $\gamma(u_{D})$? Or just $\gamma(u_{D})$? In the first case, this inflow term wouldn't contribute to the Jacobian. In the latter case, it would. Does it matter which way I do it?&lt;/p&gt;&#xA;" OwnerUserId="6643" LastActivityDate="2014-02-19T20:31:12.387" Title="How to handle inflow and outflow boundaries for a non-linear convection-diffusion equation (DGFEM)" Tags="|boundary-conditions|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10842" PostTypeId="1" CreationDate="2014-02-19T09:17:04.343" Score="13" ViewCount="715" Body="&lt;p&gt;I am looking for a fast Eigenvalue and SVD solver for small dense structured matrices (Hankel and Toeplitz). I have searched for efficient implementations in libraries like MKL but I am not able to find anything specific to structured matrices. I found a set of interesting papers on the subject: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1016/S0024-3795(00)00084-7&quot; rel=&quot;noreferrer&quot;&gt;Luk and Qiao, &quot;A Fast Eigenvalue algorithm for Hankel matrices&quot;&lt;/a&gt; which uses Lancoz tridiagonalization based on FFT and a QR like diagonalization of the tridiagonal matrix for computing eigenvalues with a complexity of $O(n^2\log(n))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1016/j.laa.2007.05.027&quot; rel=&quot;noreferrer&quot;&gt;Luk and Qiao, &quot;A fast symmetric SVD algorithm for square Hankel matrices&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do any of the libraries out there or any implementation you know of use this as I would like to reuse as much as possible? Are there any algorithms which achieve bounds better than this for structured matrices?&lt;/p&gt;&#xA;" OwnerUserId="4841" LastEditorUserId="20688" LastEditDate="2017-05-05T11:31:04.060" LastActivityDate="2017-05-05T11:31:04.060" Title="Fast Eigenvalue and SVD Solver for Structured Matrices" Tags="|linear-algebra|eigenvalues|svd|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10843" PostTypeId="1" AcceptedAnswerId="10852" CreationDate="2014-02-19T09:51:29.370" Score="1" ViewCount="501" Body="&lt;p&gt;I'm dealing with Jacobi iterative method for solving sparse system of linear equations. For small matrices it works well and gives right answers even if matrix is not strictly diagonal dominant, however for the case of really big matrices ($100000*100000$) it does not converge because the matrix is not diagonal. Many articles suggest to interchange rows and columns in order to make diagonal dominant matrices, however for the case of my matrix it is always not diagonal dominant. Could anyone please, suggest me how to deal with this problem? Maybe there is some method how to choose right initial approximation or maybe there is more robust algorithm exists. I'm a newcomer in this field and I would be appreciated for any help.&lt;/p&gt;&#xA;" OwnerUserId="5278" LastEditorUserId="5278" LastEditDate="2014-02-19T10:22:38.663" LastActivityDate="2014-02-20T01:49:22.140" Title="Problem with convergence of Jacobi iterative algorithm" Tags="|matrix|linear-solver|convergence|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10844" PostTypeId="2" ParentId="10813" CreationDate="2014-02-19T11:30:59.790" Score="1" Body="&lt;p&gt;The only problem with the code above was with the values of the masses. By simply correcting the masses of both astronomical bodies to their proper value, the simulation runs pretty well. Below, I added the corrected code (still using all units in SI, i.e., kg, m, seconds); but as one of the users previously commented (TomDickens), probably the best approach for this kind of program would be to use astronomical units, such as AU for distance, year for time and $G \times M_{sun}=1$ concerning masses.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program gravitation&#xA;implicit none&#xA;&#xA;integer :: k, total&#xA;real(8), dimension(2) :: M, x, y, Vx, Vy&#xA;real(8), parameter :: G = 6.6738D-11&#xA;real(8) :: time, dt, r&#xA;&#xA;dt = 3600. ! dt = 3600 = 1h&#xA;total = 10000 ! total number of points to be output &#xA;&#xA;M(1)=1.9891D+30 ! Sun's mass&#xA;x(1)=0.&#xA;y(1)=0.&#xA;Vx(1)=0.&#xA;Vy(1)=0.&#xA;&#xA;M(2)=5.9736D+24 ! Earth's mass&#xA;x(2)=1.4960D+11 ! Earth's average orbit as initial point&#xA;y(2)=0.&#xA;Vx(2)=0.&#xA;Vy(2)=2.9786D+04 ! Earth's average orbital speed&#xA;&#xA;open(unit=101,file=&quot;2-body_output.txt&quot;)&#xA;write(101,*) &quot;time, x, y&quot; ! CSV file header&#xA;write(101,*) 0., &quot;,&quot;, x(2), &quot;,&quot;, y(2) ! initial position&#xA;&#xA;do k=1,total&#xA;  r = sqrt( ( x(2) - x(1) )**2 + ( y(2) - y(1) )**2 ) ! distance between object 1 and 2&#xA;  Vx(2) = Vx(2) - G * M(1) * dt * ( x(2) - x(1) ) / r**3 ! i.e., Vx2 = Vx2 - G.M.dt.(x2-x1)/r^3&#xA;  Vy(2) = Vy(2) - G * M(1) * dt * ( y(2) - y(1) ) / r**3&#xA;  x(2) = x(2) + Vx(2) * dt ! calculating new x position&#xA;  y(2) = y(2) + Vy(2) * dt&#xA;  time = k * dt&#xA;  write(101,*) time, &quot;,&quot;, x(2), &quot;,&quot;, y(2)&#xA;enddo&#xA;&#xA;end program gravitation&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7348" LastEditorUserId="7348" LastEditDate="2014-02-22T22:31:31.587" LastActivityDate="2014-02-22T22:31:31.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10845" PostTypeId="2" ParentId="10835" CreationDate="2014-02-19T13:47:20.063" Score="4" Body="&lt;p&gt;You seem to be under the misconception that multigrid only works for symmetric and positive definite matrices, and that you can apply a multigrid preconditioner if only you can replace the matrix by some SPD matrix. But that's not true. Multigrid works if the error components of your linear system live on all scales and if the matrix allows to construct a smoother that smoothes the error components on the finest levels. But this is not a result of matrix properties such as symmetricity or positive definiteness. Rather, it follows from the underlying differential operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, if I understand you right, you want to apply the multigrid preconditioner to a matrix $f(A)$ and use this to precondition the solution of a linear system involving the matrix $A$. This will in general only bring any reduction in iteration counts if $f(A)$ and $A$ are in fact closely related. Just because you can find a function $f$ that maps &lt;em&gt;any&lt;/em&gt; matrix $A$ to a symmetric and positive matrix $f(A)$ and for which you have a multigrid implementation does not mean that this is then automatically a good preconditioner for $A$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-19T13:47:20.063" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10846" PostTypeId="1" CreationDate="2014-02-19T19:14:35.267" Score="1" ViewCount="309" Body="&lt;p&gt;I am dealing with a large sparse integer matrix that I need to find the nullspace of. I've seen Markowitz Pivoting come up in several places discussing similar problems such as here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mapleprimes.com/posts/41191-Solving-Sparse-Linear-Systems-In-Maple&quot; rel=&quot;nofollow&quot;&gt;http://www.mapleprimes.com/posts/41191-Solving-Sparse-Linear-Systems-In-Maple&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in the MAGMA documentation under the sparse null space algorithm. How can one use this technique (or similarly perhaps, the Structured Gaussian Elimination mentioned above it) to reduce a sparse matrix to a dense, small matrix? I understand that you can eliminate variables with regular Gaussian Elimination, but how does one retrieve those after solving the new system?&lt;/p&gt;&#xA;" OwnerUserId="1264" LastEditorUserId="1264" LastEditDate="2014-02-19T19:58:42.013" LastActivityDate="2014-02-19T19:58:42.013" Title="Markowitz Pivoting to reduce size of a dense integer system" Tags="|linear-algebra|sparse-matrix|matrix|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10847" PostTypeId="1" AcceptedAnswerId="10851" CreationDate="2014-02-19T20:00:39.837" Score="4" ViewCount="654" Body="&lt;p&gt;I have a problem with parallelization and the brownian dynamics (molecular dynamics) code that I am using.  We have our own home-grown framework at the university, and recently we've made the change to using OpenMP for the more intensive calculations.  We are definitely CPU bound, so I don't necessarily have to worry about the total memory or disk speed of the computation, only the processor.  There are multiple portions of the code that are parallelized, and they appear in the two basic forms.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;No shared variables, so no critical operations needed.&lt;/li&gt;&#xA;&lt;li&gt;An omp lock, so that we can access shared variables.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Also relevant is the architecture that these are running on.  The cluster that I am using has 2 Intel Xeon hex-core processors, so 12 cores per node, and I only run on 1 node for a sim at a time.  The problem is that when I move beyond using 6 cores on a node, I immediately see a performance hit, so I see worse performance using 12 cores than 6.  Initially I thought that this was probably a synchronization issue between the two processors, but only 1 of my functions has the critical section, the other 3 don't have any shared variable, and also synch up their main results at the end.  In this case, each thread has it's own array that it writes to, and then after they all complete the results are computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did just move from using gcc to icc (intel compiler), which has definitely improved my times, but I was wondering if anybody had any thoughts on what else I could do to check out what is going on.  Unfortunately, we don't have VTune installed on the cluster, so I can't profile using the Intel tools, but what does everybody here think?  Does this sound like a caching issue to you?  Or is it something more sinister.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just don't quite understand that, given how little of the code has to synchronize, there is worse performance with more cores once I pass one processor.&lt;/p&gt;&#xA;" OwnerUserId="4545" LastActivityDate="2014-02-20T05:10:10.127" Title="Increasing thread number degrades performance" Tags="|parallel-computing|simulation|performance|hpc|molecular-dynamics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10848" PostTypeId="2" ParentId="10841" CreationDate="2014-02-19T20:31:12.387" Score="2" Body="&lt;p&gt;I do not quite see why you separate $\gamma(u)$ and $\mathbf b(u)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do the integration by parts, that should yield on interior faces the consistency term&#xA;\begin{equation}&#xA;\{\gamma(u)\mathbf b(u)\cdot\mathbf n\}[v]&#xA;\end{equation}&#xA;To this, you can add a consistent stabilization term like multiples of $[\gamma(u)\mathbf b(u)\cdot\mathbf n][v]$, possibly of $[\gamma(u)\{\mathbf b(u)\cdot\mathbf n\}[v]$. But only if your interface term is the sum of the consistency term above and a stabilization term which vanishes if applied to continuous functions, will you discretization be conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this holds, you might still be in trouble with stability. Have you tried local Lax-Friedrichs fluxes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, you know about the stability issues with the Baumann-Oden method?&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2014-02-19T20:31:12.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10849" PostTypeId="1" AcceptedAnswerId="10850" CreationDate="2014-02-19T20:53:16.387" Score="0" ViewCount="98" Body="&lt;p&gt;At the moment, I'm installing the &quot;&lt;a href=&quot;http://www.mpa-garching.mpg.de/gadget/&quot; rel=&quot;nofollow&quot;&gt;GADGET-2&lt;/a&gt;&quot; application. In its documentation it says &quot;Note that the MPI-capable version 2.x of FFTW is required, the new version 3 lacks MPI capability at this point. &lt;a href=&quot;http://www.fftw.org/&quot; rel=&quot;nofollow&quot;&gt;FFTW&lt;/a&gt; is only needed for simulations that use the TreePM algorithm.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this still true for the current version 3.3.3? Should I use 2.1.5 or 3.3.3?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="173" LastActivityDate="2014-02-19T21:00:19.357" Title="Which version of FFTW" Tags="|libraries|mpi|fftw|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10850" PostTypeId="2" ParentId="10849" CreationDate="2014-02-19T21:00:19.357" Score="2" Body="&lt;p&gt;FFTW3 now supports MPI, but the APIs between 2 and 3 have many differences. Thus, I'm 99% certain you should use FFTW2. However, if you ask around, you may be able to find a usable version of Gadget-3 which has been slated for but not yet officially released for awhile now. We see a lot of both on our systems.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-02-19T21:00:19.357" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10851" PostTypeId="2" ParentId="10847" CreationDate="2014-02-19T22:24:47.347" Score="4" Body="&lt;p&gt;It's possible that all the memory being used by your code is on one &lt;em&gt;socket&lt;/em&gt; and that up to 6 cores all the tasks are running on that socket. When you get to 7+ sockets, then there are transfers between sockets to get at the memory. You may need to investigate memory affinity options for your threads. The default policy in Linux is first-touch (I think), so if you have one thread initializing things and then all the threads go to work, you may need to change this.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-02-19T22:24:47.347" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="10852" PostTypeId="2" ParentId="10843" CreationDate="2014-02-20T01:49:22.140" Score="3" Body="&lt;p&gt;The Jacobi iteration is the worst possible solver for linear systems. Furthermore, contrary to your belief (but easy to show), it is entirely independent of the ordering of the unknowns, so reordering rows and columns of the system makes absolutely no difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many better methods for solving linear systems, among them CG and GMRES, and there are many good books on the subject (e.g., the one by Y. Saad). My take on many of the issues with solver and preconditioners is given in lectures 34-38 at &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-20T01:49:22.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10855" PostTypeId="1" AcceptedAnswerId="10858" CreationDate="2014-02-20T07:45:39.670" Score="1" ViewCount="310" Body="&lt;p&gt;I have generated a list of 3 random number where each summed to 1. I would like to assess the quality of randomness. What is the best mechanism to assess this randomness? E.g my random numbers are.  Any idea what tools can I use excel for this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.4 0.5 0.1&#xA;0.2 0.3 0.5&#xA;0.6 0.2 0.2&#xA;0.5 0.2 0.3&#xA;0.2 0.4 0.4&#xA;0.2 0.1 0.7&#xA;0.3 0.3 0.4&#xA;0.8 0.1 0.1&#xA;0.1 0.5 0.4&#xA;0.4 0.4 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7007" LastEditorUserId="549" LastEditDate="2014-02-20T19:06:52.450" LastActivityDate="2014-02-20T19:06:52.450" Title="Quality of randomness in generated random number" Tags="|statistics|random-number-generation|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10856" PostTypeId="2" ParentId="10855" CreationDate="2014-02-20T09:21:21.267" Score="-1" Body="&lt;p&gt;What do you mean by &quot;Quality of Randomness&quot;? In simple terms, if you want to see if the generated set of numbers is random enough, their mean should be close to 0, i.e. a pure gaussian distribution. Note that this is a good option only when you have sufficiently large no. of random samples, in which case getting a mean close to zero indicates that the rand generator has done a pretty good job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, purely random numbers are uncorrelated with each other, although it almost never happens in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be other mathematically rigorous ways to test it, but for simple tasks this would suffice. &lt;/p&gt;&#xA;" OwnerUserId="4299" LastActivityDate="2014-02-20T09:21:21.267" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10858" PostTypeId="2" ParentId="10855" CreationDate="2014-02-20T10:12:31.893" Score="-1" Body="&lt;p&gt;Lets say that you have your numbers in a matrix&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = [...&#xA;0.4 0.5 0.1&#xA;0.2 0.3 0.5&#xA;0.6 0.2 0.2&#xA;0.5 0.2 0.3&#xA;0.2 0.4 0.4&#xA;0.2 0.1 0.7&#xA;0.3 0.3 0.4&#xA;0.8 0.1 0.1&#xA;0.1 0.5 0.4&#xA;0.4 0.4 0.2];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;x(i,:)&lt;/code&gt; is the i-th row (you 3D random variable). You have to calculate the mean row (I am using matlab notation)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_mean = mean(x);&#xA;disp(x_mean);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can calculate the residual matrix (the difference between your rows and the mean one)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res = x - repmat(x_mean,10,1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this matrix &lt;code&gt;res&lt;/code&gt; should have the rows distributed as a 3D gaussian variable with zero mean (or uniform, or whatever it is the random distribution you have used to generate your sample).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have some dependence between your variables, and I dont know how to deal with that. It is probably a good idea to ask this question here &lt;a href=&quot;https://stats.stackexchange.com/&quot;&gt;CrossValidated&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7277" LastEditorUserId="-1" LastEditDate="2017-04-13T12:44:17.923" LastActivityDate="2014-02-20T10:12:31.893" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10859" PostTypeId="2" ParentId="10855" CreationDate="2014-02-20T18:26:04.937" Score="3" Body="&lt;p&gt;Testing whether or not the mean is correct, or even if the histogram of your generated random variants &quot;looks&quot; like a certain distribution is not sufficient. Stick with much more rigorous test suites such as &lt;a href=&quot;http://www.iro.umontreal.ca/~simardr/testu01/tu01.html&quot; rel=&quot;nofollow&quot;&gt;TestU01&lt;/a&gt; or &lt;a href=&quot;http://www.stat.fsu.edu/pub/diehard/&quot; rel=&quot;nofollow&quot;&gt;Diehard&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you really only have TWO random numbers in each row, because of the constraint that they sum to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This requires more thought, but I think you may want to test the randomness of the first column, then test the randomness of the second column conditioned on the numbers in the first column.&lt;/p&gt;&#xA;" OwnerUserId="7322" LastActivityDate="2014-02-20T18:26:04.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10862" PostTypeId="2" ParentId="10484" CreationDate="2014-02-21T14:53:20.760" Score="5" Body="&lt;p&gt;Not well.  If serial is a common case, it is important to wrap and drop down to lapack for serial execution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented this in my code.  For 2013 MKL pzhegvx with $n \approx 100 (1000)$ seems to incur 30% (100%) overhead compared to zhegvx when executed in serial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems high to me, so I'm a little worried about &lt;a href=&quot;https://github.com/maxhutch/qe-srb/blob/master/PW/src/srb-matrix_mod.f90#L307&quot; rel=&quot;nofollow&quot;&gt;my implementation&lt;/a&gt;.  Note that I inline zhegvx to enable re-use of the factorization of $B$.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2014-02-23T14:44:04.540" LastActivityDate="2014-02-23T14:44:04.540" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10863" PostTypeId="1" CreationDate="2014-02-21T15:54:22.857" Score="3" ViewCount="93" Body="&lt;p&gt;I'm using PETSc's SNES linesearch (NEWTONLS) to solve a system of nonlinear PDEs. SNES+KSP successfully solved a test problem of the same type before without any trouble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, with my current problem, if I use the KSP to invert the Jacobian, the intermediate solution develops a &quot;ridge&quot; at the boundary between two MPI processes. The image below shows the solution after one SNES step running with &lt;code&gt;-ksp_gmres_modifiedgramschmidt -pc_type asm -sub_pc_type lu -sub_pc_factor_shift_type NONZERO&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/HyJY9.png&quot; alt=&quot;KSP gives a discontinuity&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem seems to go away if I execute the exact same binary, but using the MUMPS direct solver with &lt;code&gt;-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps&lt;/code&gt;, as shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/c5EQJ.png&quot; alt=&quot;No issue with MUMPS&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases above SNES decided to start by taking a full Newton step. However, in the KSP run the ridge continues to grow and the step size gets significantly suppressed over the next few steps before failing completely. In the MUMPS run, SNES converged straightforwardly after taking 5 full steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any idea what is happening here?&lt;/p&gt;&#xA;" OwnerUserId="7404" LastActivityDate="2014-02-21T15:54:22.857" Title="PETSc: KSP gives strange discontinuity at the edge between MPI processes" Tags="|petsc|" AnswerCount="0" CommentCount="1" ClosedDate="2014-03-26T19:08:01.957" ContentLicense="CC BY-SA 3.0" />
  <row Id="10864" PostTypeId="1" CreationDate="2014-02-21T20:02:44.663" Score="3" ViewCount="1959" Body="&lt;p&gt;I have a C++ program that has a double &lt;code&gt;for&lt;/code&gt; loop that I'd like to parallelize using opeMP. It is simple (a polynomial product) of varying length (between 100 and 8000 coefficients) and the function is called millions of times per execution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the parallel &lt;code&gt;pragma&lt;/code&gt; around the outer loop and the &lt;code&gt;for&lt;/code&gt; pragma just below the &lt;code&gt;parallel&lt;/code&gt; pragma. I have found that if the number of elements is big, near 8000, the execution time is good, lower than without parallelization. However, for small number of coefficients, the performance is degraded, and the parallel version is slower than the serial one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose it is due to the creation/destruction of parallel threads every time the function is called. I can limit the creation of threads with an &lt;code&gt;if&lt;/code&gt; in the pragma, so it is only parallelized for big polynomials, but I was wondering if there is a way to avoid creation/destruction of the threads, so I could create them once and leave them &quot;waiting&quot; between calls, so they would be ready when needed. That function is the only one I'm parallelizing and I cannot move the parallelization to the caller function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using g++.&lt;/p&gt;&#xA;" OwnerUserId="7405" LastActivityDate="2014-02-25T15:59:27.240" Title="Leave openMP theads ready" Tags="|parallel-computing|c++|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10865" PostTypeId="2" ParentId="10864" CreationDate="2014-02-21T20:38:08.810" Score="6" Body="&lt;p&gt;The Intel Compiler stack uses &lt;code&gt;KMP_BLOCKTIME=&amp;lt;time in milliseconds, infinite&amp;gt;&lt;/code&gt; to set the time that the threads wait to go to sleep after the end of the loop (or infinite). I don't know if GCC supports this. You might trying setting &lt;code&gt;OMP_WAIT_POLICY=active&lt;/code&gt; to force the threads to always spin. This will keep the threads running after the end of the loop, but it might also impact your performance between OpenMP calls.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-02-21T20:38:08.810" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10866" PostTypeId="1" CreationDate="2014-02-22T00:52:09.707" Score="4" ViewCount="235" Body="&lt;p&gt;This question was original posted on SO but it was suggested that I post it here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on a program in which I have a banded matrix &lt;strong&gt;M&lt;/strong&gt; and a vector &lt;strong&gt;b&lt;/strong&gt;, and I want to maintain an approximate solution vector &lt;strong&gt;x&lt;/strong&gt; such that &lt;strong&gt;Mx&lt;/strong&gt; ≃ &lt;strong&gt;b&lt;/strong&gt;. Is there a speedy algorithm or way of modeling this so that I can change individual elements of &lt;strong&gt;M&lt;/strong&gt; and correspondingly update &lt;strong&gt;x&lt;/strong&gt;, without having to do a full matrix inversion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I'm considering is maintaining an approximate inverse of &lt;strong&gt;M&lt;/strong&gt;, using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Sherman%E2%80%93Morrison_formula&quot; rel=&quot;nofollow&quot;&gt;Sherman Morrison Algorithm&lt;/a&gt; in combination with a fast approximate matrix multiplication algorithm like &lt;a href=&quot;http://www.cs.rpi.edu/~drinep/Papers/Drineas_FOCS_01.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7409" LastActivityDate="2014-03-24T05:57:51.280" Title="Updating an approximate solution to a linear system in response to a small change" Tags="|algorithms|linear-solver|matrix|approximation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10867" PostTypeId="2" ParentId="10866" CreationDate="2014-02-22T04:58:54.223" Score="2" Body="&lt;p&gt;Apologies for what will probably be a comment-answer hybrid (and a comment probably too long to fit in the comment box).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you say in your question is that you want an approximate solution vector. Have you considered iterative methods? There, the approximate inverse of $M$ could be used as a preconditioner in concert with some iterative method (GMRES, BiCGStab, etc.) to maintain approximate solution vectors, and the approximate solution for one linear system could then be used as an initial guess for the solution of a perturbed linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To update the preconditioner in response to single-element (or low-rank) perturbations, you could use Sherman-Morrison(-Woodbury). Depending on the perturbation, the approximate inverse may still be an effective preconditioner even without updating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how fast approximate matrix multiplication might degrade the solution of an iterative linear system; I would guess that it would be more forgiving in forming the preconditioner, since that can be approximate. After forming the preconditioner, I would stick to standard matrix-vector products for the solver iterations (for GMRES iterations, or whatever iterative method you consider using).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this entire discussion might be moot, depending on the size of $M$. If it's sufficiently small, it might be worth just solving every time; I assume it's relatively large, or we wouldn't be having this discussion.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-02-22T04:58:54.223" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10868" PostTypeId="1" CreationDate="2014-02-22T08:01:55.843" Score="1" ViewCount="283" Body="&lt;p&gt;I am working on a numerical method for time fractional PDE. One problem is that I must compute a numerical integral of the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{equation}&#xA;\int_0^{t_m} (t_m-s)^{-\beta}f(s)ds&#xA;\end{equation}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$t_m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0&amp;lt;\beta&amp;lt;1$&lt;/span&gt; are constants. Noting that the integral has a weakly singular kernel, so classical quadrature, such as Gaussian quadrature, will invalid for the singularity of &lt;span class=&quot;math-container&quot;&gt;$(t_m-s)^{-\beta}$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$s=t_m$&lt;/span&gt;. So my question is how to compute this kind of integral numerically with a given precision (because I need a high precision when computing the convergence ratio)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My try:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have deduced the Gaussian quadrature with the kernel of &lt;span class=&quot;math-container&quot;&gt;$(t_m-s)^{-\beta}$&lt;/span&gt; which have a highest algebraic accuracy. But the formula is too complex so that when I compute the coefficients and nodes MATLAB loss precision which result in a bad precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another method I have tried is to compute this by using integral by parts. Thus, the kernel will be &lt;span class=&quot;math-container&quot;&gt;$(t_m-s)^{1-\beta}/(1-\beta)$&lt;/span&gt;, which is without singularity. This method is valid when &lt;span class=&quot;math-container&quot;&gt;$f(s)$&lt;/span&gt; have a good regularity, say in &lt;span class=&quot;math-container&quot;&gt;$C^1$&lt;/span&gt;. But when &lt;span class=&quot;math-container&quot;&gt;$f(s)$&lt;/span&gt; does not have a continuous derivative, such as &lt;span class=&quot;math-container&quot;&gt;$|s|$&lt;/span&gt; or Heaviside function, this method is also invalid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is more, &lt;span class=&quot;math-container&quot;&gt;$(t_m-s)^{-\beta}$&lt;/span&gt; is only one kind of weakly singular kernel. How about other situations? Is there any general method to conquer this problem? &lt;/p&gt;&#xA;" OwnerUserId="7410" LastEditorUserId="20688" LastEditDate="2019-05-12T09:06:03.837" LastActivityDate="2019-05-12T09:06:03.837" Title="Numerical integral with a weakly singular kernel with a satisfactory precision" Tags="|quadrature|numerics|integration|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10869" PostTypeId="1" AcceptedAnswerId="10886" CreationDate="2014-02-22T14:29:20.953" Score="26" ViewCount="9489" Body="&lt;p&gt;Given is an increasing finite sequence of positive numbers $z_{1} ,z_{2},.....z_{n}$. Which of the following two algorithms is better for computing the sum of the numbers?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s=0; &#xA;for \ i=1:n &#xA;    s=s + z_{i} ; &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s=0; &#xA;for \ i=1:n &#xA;s=s + z_{n-i+1} ; &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my opinion it would be better to start adding the numbers from the largest to the smallest number, because the error gets smaller and smaller. We also know that when we add a very large number to a very small number, the approximate result can be the large number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct? What else can be said?&lt;/p&gt;&#xA;" OwnerUserId="7414" LastEditorUserId="53" LastEditDate="2014-02-23T03:10:57.043" LastActivityDate="2019-09-19T14:50:21.563" Title="Which algorithm is more accurate for computing the sum of a sorted array of numbers?" Tags="|algorithms|floating-point|" AnswerCount="10" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10870" PostTypeId="2" ParentId="10869" CreationDate="2014-02-22T15:21:39.600" Score="27" Body="&lt;p&gt;Are these integers or floating point numbers? Assuming it's floating point, I would go with the first option. It's better to add the smaller numbers to each other, then add the bigger numbers later. With the second option, you'll end up adding a small number to a big number as &lt;em&gt;i&lt;/em&gt; increases, which can lead to problems. Here's a good resource on floating point arithmetic: &lt;a href=&quot;http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html&quot;&gt;What Every Computer Scientist Should Know About Floating-Point Arithmetic&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7322" LastActivityDate="2014-02-22T15:21:39.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10871" PostTypeId="1" CreationDate="2014-02-22T15:48:14.440" Score="5" ViewCount="171" Body="&lt;p&gt;I'm working with an elliptic reaction diffusion PDE of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-k\nabla^2u+cu=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've noticed that when the reaction term dominates over the diffusion (i.e. $c&amp;gt;&amp;gt;k$), the true (exact) solution has sharp gradients near the boundaries. I've also noticed that my numerical approximations via finite difference, finite volume, and standard galerkin method do not approximate the solution well using equi-spaced points with a spatial stepsize significantly larger than the sharp boundary layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that I can overcome this problem by adapting my mesh/nodes to be more densely near the boundary layers, but this presupposes that I know exactly where these layers are.  Is there a more generalized approach to this problem which can yield good approximations without apriori knowledge.  Ideally, I'm hoping there is a method that allows for a uniformly spaced grid that can get a better approximation than the naive implementations of FDM, FVM, and FEM, without making the mesh extremely fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if this is possible, but if anyone knows of any alternative methods that I can try, I'm open to suggestions! :)&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-02-24T20:53:09.430" Title="Discretization method for a reaction dominated elliptic PDE" Tags="|pde|discretization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10872" PostTypeId="2" ParentId="10871" CreationDate="2014-02-22T15:56:43.417" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Adaptive_mesh_refinement&quot;&gt;Adaptive refinement&lt;/a&gt;. There are even optimal error estimators for your exact problem, though you can't go too wrong on this problem by dividing cells where the magnitude of the gradient is largest. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-02-22T15:56:43.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10873" PostTypeId="2" ParentId="10864" CreationDate="2014-02-22T16:34:19.453" Score="1" Body="&lt;p&gt;A more involved solution is to lift the parallel region above this routine and wrap the code you don't want to be parallel with a single/master clause.  The advantage here is that you might figure out how to increase the amount of threading in your code, which is good in the battle against Amdahl's Law.  On the other hand, doing this might be rather involved and not worth much benefit relative to the solutions discussed already.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2014-02-22T16:34:19.453" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10875" PostTypeId="2" ParentId="10869" CreationDate="2014-02-22T20:54:47.827" Score="24" Body="&lt;p&gt;animal_magic's answer is correct that you should add the numbers from smallest to largest, however I want to give an example to show why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we are working in a floating point format that gives us a staggering 3 digits of accuracy.  Now we want to add ten numbers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[1000, 1, 1, 1, 1, 1, 1, 1, 1, 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course the exact answer is 1009, but we can't get that in our 3 digit format.  Rounding to 3 digits, the most accurate answer we get get is 1010.  If we add smallest to largest, on each loop we get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Loop Index        s&#xA;1                 1&#xA;2                 2&#xA;3                 3&#xA;4                 4&#xA;5                 5&#xA;6                 6&#xA;7                 7&#xA;8                 8&#xA;9                 9&#xA;10                1009 -&amp;gt; 1010&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So we get the most accurate answer possible for our format.  Now lets assume that we add from largest to smallest.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Loop Index        s&#xA;1                 1000&#xA;2                 1001 -&amp;gt; 1000&#xA;3                 1001 -&amp;gt; 1000&#xA;4                 1001 -&amp;gt; 1000&#xA;5                 1001 -&amp;gt; 1000&#xA;6                 1001 -&amp;gt; 1000&#xA;7                 1001 -&amp;gt; 1000&#xA;8                 1001 -&amp;gt; 1000&#xA;9                 1001 -&amp;gt; 1000&#xA;10                1001 -&amp;gt; 1000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since the floating point numbers are rounded after each operation, all of the additions are rounded away, increasing our error from 1 to 9 from the exact.  Now imagine if your set of numbers to add had a 1000, and then a hundred 1's, or a million.  Note that to be truly accurate, you would want to sum the smallest two numbers, then resort the result into your set of numbers.  &lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-02-22T20:54:47.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10876" PostTypeId="1" CreationDate="2014-02-07T21:23:46.237" Score="42" ViewCount="9162" Body="&lt;p&gt;I would like to run some simple simulations of scattering of wavepackets off of simple potentials in one dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there simple ways to numerically solve the one-dimensional TDSE for a single particle?&lt;/strong&gt; I know that, in general, trying to use naïve approaches to integrate partial differential equations can quickly end in disaster. I am therefore looking for algorithms which&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;are numerically stable,&lt;/li&gt;&#xA;&lt;li&gt;are simple to implement, or have easily accessible code-library implementations,&lt;/li&gt;&#xA;&lt;li&gt;run reasonably fast, and hopefully&lt;/li&gt;&#xA;&lt;li&gt;are relatively simple to understand.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would also like to steer relatively clear of spectral methods, and particularly of methods which are little more than solving the time-independent Schrödinger equation as usual. However, I would be interested in pseudo-spectral methods which use B-splines or whatnot. If the method can take a time-dependent potential then that's definitely a bonus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, any such method will always have a number of disadvantages, so I would like to hear about those. When does it not work? What are common pitfalls? Which ways can it be pushed, and which ways can it not?&lt;/p&gt;&#xA;" OwnerUserId="3551" OwnerDisplayName="episanty" LastEditorUserId="3551" LastEditDate="2018-07-31T08:36:38.283" LastActivityDate="2018-07-31T08:36:38.283" Title="Are there simple ways to numerically solve the time-dependent Schrödinger equation?" Tags="|quantum-mechanics|computational-physics|" AnswerCount="6" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10877" PostTypeId="2" ParentId="10876" CreationDate="2014-02-07T22:16:18.707" Score="4" Body="&lt;p&gt;The most straightforward finite difference method is fast and easy to understand, but is not unitary in time - so probability is not conserved.  Crank-Nicholson-Crout averages the forward and backward finite difference methods to produce a hybrid implicit/explicit method that is still pretty easy to understand and to implement and is unitary in time.  This site explains the method well, provides pseudocode, and gives the relevant properties:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.physics.utah.edu/~detar/phycs6730/handouts/crank_nicholson/crank_nicholson/&quot; rel=&quot;noreferrer&quot;&gt;http://www.physics.utah.edu/~detar/phycs6730/handouts/crank_nicholson/crank_nicholson/&lt;/a&gt;&#xA;Note: There is a - sign missing from the LHS of equation one of this link, which propagates throughout the page.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where does the nonunitarity come from?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a nut shell, solving the TDSE comes down to figuring out how to deal with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$| \psi(x,t)\rangle = e^{-iHt}|\psi(x,0)\rangle$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which contains a differential operator in an exponential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying a forward finite difference turns the differential operator into a tridiagonal matrix (converting the Reals to a grid) and the exponential into the first two terms of its Taylor series &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e^{-iHt}\approx1-iHt $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This discretization and linearization is what gives rise to the nonunitarity.  (You can show that the tridiagonal matrix is not unitary by direct computation.)  Combining the forward finite difference with the backward finite difference produces the approximation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e^{-iHt}\approx \frac {1-\frac{1}{2} iHt} {1+\frac{1}{2} iHt} $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which, kindly, happens to be unitary (again you can show it by direct computation).&lt;/p&gt;&#xA;" OwnerDisplayName="Wally" LastActivityDate="2014-02-08T23:44:44.277" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10878" PostTypeId="2" ParentId="10876" CreationDate="2014-02-08T03:17:37.937" Score="25" Body="&lt;p&gt;The Schroedinger equation is effectively a &lt;a href=&quot;http://en.wikipedia.org/wiki/Reaction%E2%80%93diffusion_system&quot; rel=&quot;noreferrer&quot;&gt;reaction-diffusion&lt;/a&gt; equation&#xA;$$&#xA;i\frac{\partial\psi}{\partial t}=-\nabla^2\psi+V\psi\tag{1}&#xA;$$&#xA;(all constants are 1). When it comes to any partial differential equation, there's two ways to solve it:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Implicit method (adv: large time steps &amp;amp; unconditionally stable, disadv: requires matrix solver that can give bad data)&lt;/li&gt;&#xA;&lt;li&gt;Explicit method (adv: easy to implement, disadv: requires small timesteps for stability)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For parabolic equations (linear in $t$ and 2nd order in $x$), the implicit method is often the better choice. The reason is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&quot; rel=&quot;noreferrer&quot;&gt;condition for stability&lt;/a&gt; for the explicit method requires $dt\propto dx^2$, which will be very small. You can avoid this issue by using the implicit method, which has no such limitation on the time-step (though in practice you don't normally make it insanely large because you can lose some of the physics). What I describe next is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Crank-nicholson_method&quot; rel=&quot;noreferrer&quot;&gt;Crank-Nicolson method&lt;/a&gt;, a common second order accurate (space &amp;amp; time) implicit scheme.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;&lt;strong&gt;Starters&lt;/strong&gt;&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In order to computationally solve a PDE, you need to discretize it (make the variables fit onto a grid). The most straight-forward is a rectangular, Cartesian grid. Here on, $n$ represents the time index (and is always a super-script) and $j$ the position index (always a subscript). By employing a &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference#Finite_difference_in_several_variables&quot; rel=&quot;noreferrer&quot;&gt;Taylor expansion&lt;/a&gt; for the position-dependent variable, Equation (1) becomes&#xA;$$&#xA;i\frac{\psi^{n+1}_j-\psi_j}{dt} = -\frac12\left(\frac{\psi_{j+1}^{n+1}-2\psi_j^{n+1}+\psi_{j-1}^{n+1}}{dx^2}+\frac{\psi_{j+1}^{n}-2\psi_j^{n}+\psi_{j-1}^{n}}{dx^2}\right) \\ +\frac12\left(V_j\psi_j^{n+1} +V_j\psi_j^n\right)&#xA;$$&#xA;Where we have assumed that $V=V(x)$. What happens next is a grouping of like spatial and temporal indices (you may want to double check the math):&#xA;$$&#xA;\frac12\frac{dt}{dx^2}\psi_{j+1}^{n+1}+\left(i-\frac{dt}{dx^2}-\frac12V_j\right)\psi_j^{n+1}+\frac12\frac{dt}{dx^2}\psi_{j-1}^{n+1}=\\&#xA;i\psi_j^n-\frac12\frac{dt}{dx^2}\left(\psi_{j+1}^n-2\psi_j^n+\psi_{j-1}^n\right)+\frac12V_j\psi_j^n\tag{2}&#xA;$$&#xA;This equation has the form&#xA;$$&#xA;\left(\begin{array}{ccccc}A_0 &amp;amp; A_- &amp;amp; 0 &amp;amp; 0 &amp;amp;\cdots\\ &#xA;                         A_+ &amp;amp; A_0 &amp;amp; A_- &amp;amp; 0 &amp;amp;\cdots\\&#xA;                         0 &amp;amp; A_+ &amp;amp; A_0 &amp;amp; A_- &amp;amp;\cdots \\&#xA;                         \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp;  \vdots\end{array}\right)\left(\begin{array}{c}\psi_0^{n+1} \\ \psi_1^{n+1}\\ \vdots \\ \psi_{J-1}^{n+1}\end{array}\right)=\left(\begin{array}{c}\psi_0^{n} \\ \psi_1^{n}\\ \vdots \\ \psi_{J-1}^{n}\end{array}\right)&#xA;$$&#xA;Which is called a tri-diagonal matrix and has &lt;a href=&quot;http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;noreferrer&quot;&gt;a known solution&lt;/a&gt; (plus working examples, including one written by me!). The explicit method scratches out the entire left side (or should I say top line?) of Equation (2) except for the $i\psi_j^{n+1}$ term.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;&lt;strong&gt;Issues&lt;/strong&gt;&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The biggest issue that I have found with implicit methods is that they are strongly dependent on the boundary conditions. If you have poorly defined/implemented boundary conditions, you can get spurious oscillations in your cells that can lead to bad results (see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8681/crank-nicolson-for-2nd-and-4th-order-finite-differences&quot;&gt;my SciComp post&lt;/a&gt; on a similar topic). This leads to actually having 1st order accuracy in space, rather than 2nd that your scheme &lt;em&gt;ought&lt;/em&gt; to give.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implicit methods are also supposedly difficult to parallelize, but I have only used them for 1D heat equations and not needed parallel support, so I can neither verify nor deny the claim.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also unsure how the complex nature of the wave function will affect the calculations. The work that I have done uses the &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler_equations_%28fluid_dynamics%29&quot; rel=&quot;noreferrer&quot;&gt;Euler fluid dynamic equations&lt;/a&gt;, and are thus entirely real with non-negative magnitudes.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;&lt;strong&gt;Time-dependent potential&lt;/strong&gt;&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;If you have an analytic time-dependent potential (e.g. $V\propto \cos(\omega t)$), then you'd simply use the current time, $t$, for the $V_j$ on the RHS of (2) and the future time, $t+dt$, on the LHS. I don't believe that this would create any problems, but I have not tested this so I cannot verify or deny this aspect too.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;&lt;strong&gt;Alternatives&lt;/strong&gt;&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;There are some interesting alternatives to the Crank-Nicolson method as well. The first up is the so-called &quot;super-time-stepping&quot; method. In this explicit method, you take the time step ($dt\propto dx^2$) and use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Chebyshev_polynomials#Roots_and_extrema&quot; rel=&quot;noreferrer&quot;&gt;roots of Chebyshev polynomials&lt;/a&gt; to get an optimized set of time-steps that quickly sum to $dt$ faster than doing $dt/N$ steps $N$ times (effectively you get $\Delta T=N^2dt$ so that each step $N$ advances you $Ndt$ in time). (I employ this method in my research because you have a well-defined &quot;flux&quot; from one cell to another that is used for merging data from one processor to another, using the Crank-Nicolson scheme I was unable to do this). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; &lt;em&gt;One thing to note is that this method is first order accurate in time, but if you use a Runge-Kutta 2 method in conjunction, it will give you a 2nd order accurate scheme in time.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other is called an alternating-direction &lt;em&gt;explicit&lt;/em&gt;. This method requires you to have known and well-defined boundary conditions. It then proceeds to solve the equation by &lt;em&gt;using the boundary directly&lt;/em&gt; in the computation (no need to apply it after each step). What happens in this method is you solve the PDE twice, once in an upward sweep and once in a downward sweep. The upward sweep uses&#xA;$$&#xA;\frac{\partial^2\psi}{\partial x^2}\approx\frac{\psi_{j-1}^{n+1}-\psi_j^{n+1}-\psi_j^n+\psi_{j+1}^n}{dx^2}&#xA;$$&#xA;while the downward sweep uses&#xA;$$&#xA;\frac{\partial^2\psi}{\partial x^2}\approx\frac{\psi_{j+1}^{n+1}-\psi_j^{n+1}-\psi_j^n+\psi_{j-1}^n}{dx^2}&#xA;$$&#xA;for the diffusion equation while the other terms would remain the same. The time-step $n+1$ is then solved by averaging the two directional sweeps.&lt;/p&gt;&#xA;" OwnerUserId="4762" OwnerDisplayName="Kyle Kanos" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-02-08T18:17:01.900" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="10879" PostTypeId="2" ParentId="10876" CreationDate="2014-02-09T08:38:12.477" Score="10" Body="&lt;p&gt;Kyle Kanos's answer looks to be very full, but I thought I'd add my own experience. The split-step Fourier method (SSFM) is extremely easy to get running and fiddle with; you can prototype it in a few lines of Mathematica and it is, extremely stable numerically. It involves imparting only unitary operators on your dataset, so it automatically conserves probability / power (the latter if you're solving Maxwell's equations with it, which is where my experience lies). For a one-dimensional Schrödinger equation (i.e. $x$ and $t$ variation only), it is extremely fast even as Mathematica code. And if you need to speed it up, you really only need a good FFT code in your target language (my experience lies with C++). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you'd be doing is a disguised version of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Beam_propagation_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beam Propagation Method&lt;/a&gt; for optical propagation through a waveguide of varying cross section (analogous to time varying potentials), so it would be helpful to look this up too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I look at the SSFM/BPM is as follows. Its grounding is the Trotter product formula of Lie theory:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\tag{1}\lim\limits_{m\to\infty}\left(\exp\left(\mathcal{D}\,\frac{t}{m}\right)\,\exp\left(\mathcal{V}\,\frac{t}{m}\right)\right)^m = \exp((\mathcal{D+V}) t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is sometimes called the operator splitting equation in this context. Your dataset is an $x-y$ or $x-y-z$ discretised grid of complex values representing $\psi(x,y,z)$ at a given time $t$. So you imagine this (you don't have to &lt;em&gt;do&lt;/em&gt; this; I'm still talking conceptually) whopping grid written as an $N$-element column vector $\Psi$ (for a $1024\times1024$ grid we have $N=1024^2=1\,048\,576$) and then your Schrödinger equation is of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\tag{2}\mathrm{d}_t \Psi = K\Psi = (\mathcal{D+V}(t)) \Psi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K = \mathcal{D+V}$ is an $N\times N$ skew-Hermitian matrix, an element of $\mathfrak{u}(N)$, and $\Psi$ is going to be mapped with increasing time by an element of the one parameter group $\exp(K\,t)$. (I've sucked the $i\hbar$ factor into the $K = \mathcal{D+V}$ on the RHS so I can more readily talk in Lie theoretic terms). Given the size of $N$, the operators' natural habitat $\mathfrak{U}(N)$ is a thoroughly colossal Lie group so PHEW! yes I am still talking in wholly theoretical terms!. Now, what does $\mathcal{D+V}$ look like? Still imagining for now, it could be thought of as a finite difference version of $i\,\hbar\,\nabla^2/(2\,m) - i\hbar^{-1}V_0 + i\hbar^{-1}(V_0-V(x,y,z,t_0))$, where $V_0$ is some convenient &quot;mean&quot; potential for the problem at hand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We let:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\tag{3}\begin{array}{lcl}\mathcal{D} &amp;amp;=&amp;amp; i\frac{\hbar}{2\,m} \nabla^2 - i\hbar^{-1}V_0\\&#xA;\mathcal{V}&amp;amp;=&amp;amp;i\hbar^{-1}(V_0-V(x,y,z,t))\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why I have split them up like this will become clear below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point about $\mathcal{D}$ is that it can be worked out analytically for a plane wave: it is a simple multiplication operator in momentum co-ordinates. So, to work out $\Psi\mapsto\exp(\Delta t\,\mathcal{D}) \Psi$, here are the first three steps of a SSFM/BPM cycle:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Impart FFT to dataset $\Psi$ to transform it into a set $\tilde{\Psi}$ of superposition weights of plane waves: now the grid co-ordinates have been changed from $x,\,y,\,z$ to $k_x,\,k_y,\,k_z$;&lt;/li&gt;&#xA;&lt;li&gt;Impart  $\tilde{\Psi}\mapsto\exp(\Delta t\,\mathcal{D}) \tilde{\Psi}$ by simply multiplying each point on the grid by $\exp(i\,\Delta t (V_0-k_x^2+k_y^2+k_z^2)/\hbar)$;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Impart inverse FFT to map our grid back to $\exp(\Delta t\,\mathcal{D}) \Psi$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.Now we're back in position domain. This is the better domain to impart the operator $\mathcal{V}$ of course: here $\mathcal{V}$ is a simple multiplication operator. So here is your last step of your algorithmic cycle:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Impart the operator $\Psi\mapsto\exp(\Delta t\,\mathcal{V}) \Psi$ by simply multiplying each point on the grid by the phase factor $\exp(i\,\Delta t\,(V_0-V(x,y,z,t))/\hbar)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;....and then you begin your next $\Delta t$ step and cycle over and over. Clearly it is very easy to put time-varying potentials $V(x,y,z,t)$ into the code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you see you simply choose $\Delta t$ small enough that the Trotter formula (1) kicks in: you're simply approximating the action of the operator $\exp(\mathcal{D+V}\,\Delta t)\approx\exp(\mathcal{D}\,\Delta t)\,\exp(\mathcal{V}\,\Delta t)$ and you flit back and forth with your FFT between position and momentum co-ordinates, i.e. the domains where $\mathcal{V}$ and $\mathcal{D}$ are simple multiplication operators. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that you are only ever imparting, even in the discretised world, unitary operators: FFTs and pure phase factors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One point you do need to be careful of is that as your $\Delta t$ becomes small, you must make sure that the spatial grid spacing shrinks as well. Otherwise, suppose the spatial grid spacing is $\Delta x$. Then the physical meaning of the one discrete step is that the diffraction effects are travelling at a velocity $\Delta x/\Delta t$; when simulating Maxwell's equations and waveguides, you need to make sure that this velocity is much smaller than $c$. I daresay like limits apply to the Schrödinger equation: I don't have direct experience here but it does sound fun and maybe you could post your results sometime!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second &quot;experience&quot; point with this kind of thing - I'd be almost willing to bet this is how you'll wind up following your ideas. We often have ideas that we want to do simple and quick and dirty simulations but it never quite works out that way! I'd begin with the SSFM as I've described above as it is very easy to get running and you'll quickly see whether or not its results are physical. Later on you can use your, say Mathematica SSFM code check the results of more sophisticated code you might end up building, say, a Crank Nicolson code along the lines of &lt;a href=&quot;https://physics.stackexchange.com/a/98285/26076&quot;&gt;Kyle Kanos's answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr/&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Error Bounds&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The Dynkin formula realisation of the Baker-Campbell-Hausdorff Theorem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\exp(\mathcal{D}\Delta t)\,\exp(\mathcal{V})\Delta t) = \exp\left((\mathcal{D}+\mathcal{V})\Delta t + \frac{1}{2} [\mathcal{D},\,\mathcal{V}]\,\Delta t^2 + \cdots\right)$$&#xA;converging for some $\Delta t&amp;gt;0$ shows that the method is accurate to second order and can show that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\exp(\mathcal{D}\Delta t)\,\exp(\mathcal{V})\Delta t)\,\exp\left(-\frac{1}{2} [\mathcal{D},\,\mathcal{V}]\,\Delta t^2\right) = \exp\left((\mathcal{D}+\mathcal{V})\Delta t + \mathcal{O}(\Delta t^3)\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can, in theory, therefore use the term $\exp(\mathcal{V})\Delta t)\,\exp\left(-\frac{1}{2} [\mathcal{D},\,\mathcal{V}]\,\Delta t^2\right)$ to estimate the error and set your $\Delta t$ accordingly. This is not as easy as it looks and in practice bounds end up being instead rough estimates of the error. The problem is that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\Delta t^2}{2}[\mathcal{D},\,\mathcal{V}] = -\frac{i\,\Delta t^2}{2\,m}\,\left(\partial_x^2 V(x,\,t) + 2 \partial_x V(x,\,t)\,\partial_x\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and there are no readily transformed to co-ordinates wherein $[\mathcal{D},\,\mathcal{V}]$ is a simple multiplication operator. So you have to be content with $\exp\left(-\frac{1}{2} [\mathcal{D},\,\mathcal{V}]\,\Delta t^2\right) \approx e^{-i\,\varphi\,\Delta t^2}\left(\mathrm{id} -\left(\frac{1}{2} [\mathcal{D},\,\mathcal{V}]\,-i\,\varphi(t)\right)\,\Delta t^2\right)$ and use this to estimate your error, by working out $\left(\mathrm{id} -\left(\frac{1}{2} [\mathcal{D},\,\mathcal{V}]\,-i\,\varphi(t)\right)\,\Delta t^2\right) \,\psi$ for your currently evolving solution $\psi(x,\,t)$ and using this to set your $\Delta t$ on-the-fly after each cycle of the algorithm. You can of course make these ideas the basis for an adaptive stepsize controller for your simulation. Here $\varphi$ is a global phase pulled out of the dataset to minimise the norm of $\left(\frac{1}{2} [\mathcal{D},\,\mathcal{V}]\,-i\,\varphi(t)\right)\,\Delta t^2$; you can of course often throw such a global phase out: depending on what you're doing with the simulation results often we're not bothered by a constant phase global $\exp\left(\int \varphi\,\mathrm{d}t\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A relevant paper about errors in the SSFM/BPM is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF00619865#page-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lars Thylén. &quot;The Beam Propagation Method: An Analysis of its Applicability&quot;, &lt;em&gt;Optical and Quantum Electronics&lt;/em&gt; &lt;strong&gt;15&lt;/strong&gt; (1983) pp433-439&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lars Thylén thinks about the errors in non-Lie theoretic terms (Lie groups are my bent, so I like to look for interpretations of them) but his ideas are essentially the same as the above. &lt;/p&gt;&#xA;" OwnerUserId="7425" OwnerDisplayName="WetSavannaAnimal aka Rod Vance" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2014-02-12T01:51:00.710" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10880" PostTypeId="2" ParentId="10876" CreationDate="2014-02-09T21:27:14.780" Score="29" Body="&lt;p&gt;In the early 90s we were looking for a method to solve the TDSE fast enough to do animations in real time on a PC and came across a surprisingly simple, stable, explicit method described by PB Visscher in &lt;em&gt;Computers in Physics&lt;/em&gt;: &quot;&lt;a href=&quot;http://www.hunter.cuny.edu/physics/courses/physics485/repository/files/reference-papers/Visscher%20NumSol%20Schrodinger%20Eqt.pdf&quot;&gt;A fast explicit algorithm for the time-dependent Schrödinger equation&lt;/a&gt;&quot;.  Visscher notes that if you split the wavefunction into real and imaginary parts, $\psi=R+iI$, the SE becomes the system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}\frac{dR}{dt}&amp;amp;=&amp;amp;HI \\&#xA;\frac{dI}{dt}&amp;amp;=&amp;amp;-HR \\&#xA;H&amp;amp;=&amp;amp;-\frac{1}{2m}\nabla^2+V\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you then compute $R$ and $I$ at staggered times ($R$ at $0,\Delta t,2\Delta t,...$ and $I$ at $0.5\Delta t, 1.5\Delta t,...)$, you get the discretization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R(t+\frac{1}{2} \Delta t)=R(t-\frac{1}{2} \Delta t)+\Delta t HI(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I(t+\frac{1}{2} \Delta t)=I(t-\frac{1}{2} \Delta t)-\Delta t HR(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &#xA;$$\nabla^2\psi(r,t)=\frac{\psi(r+\Delta r,t)-2\psi(r,t)+\psi(r-\Delta r,t)}{\Delta r^2}$$ (standard three-point Laplacian).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is explicit, very fast to compute, and second-order accurate in $\Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining the probability density as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P(x,t)=R^2(x,t)+I(x,t+\frac{1}{2} \Delta t)I(x,t-\frac{1}{2} \Delta t)$$ at integer time steps and, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P(x,t)=R(x,t+\frac{1}{2} \Delta t)R(x,t-\frac{1}{2} \Delta t)+I^2(x,t)$$ at half-integer time steps&lt;/p&gt;&#xA;&#xA;&lt;p&gt;makes the algorithm unitary, thus conserving probability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With enough code optimization, we were able to get very nice animations computed in real-time on 80486 machines.  Students could &quot;draw&quot; any potential, choose a total energy, and watch the time-evolution of a gaussian packet.&lt;/p&gt;&#xA;" OwnerDisplayName="Wally" LastActivityDate="2014-02-10T18:44:07.997" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="10881" PostTypeId="2" ParentId="10869" CreationDate="2014-02-23T03:01:55.283" Score="15" Body="&lt;p&gt;For the general case, I'd use &lt;a href=&quot;http://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot;&gt;compensated summation&lt;/a&gt; (or Kahan summation). Unless the numbers are already sorted, &lt;a href=&quot;http://www.drdobbs.com/floating-point-summation/184403224&quot;&gt;sorting them will be much more expensive than adding them&lt;/a&gt;. Compensated summation is also more accurate than sorted summation or naive summation (see the previous link).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for references, &lt;a href=&quot;http://floating-point-gui.de/&quot;&gt;What every programmer should know about floating-point arithmetic&lt;/a&gt; covers the basic points in enough detail that someone could read it in 20 (+/- 10) minutes and understand the basics. &quot;What every computer scientist should know about floating-point arithmetic&quot; by Goldberg is the classical reference, but most people I know who recommend that paper haven't read it in detail themselves, because it's around 50 pages (more than that, in some printings), and written in dense prose, so I have trouble recommending that as a first-line reference for people. It is good for a second look at the subject. An encyclopedic reference is Higham's &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898715210&quot;&gt;Accuracy and Stability of Numerical Algorithms&lt;/a&gt;, which covers this material, as well as the accumulation of numerical errors in many other algorithms; it's also 680 pages, so I wouldn't look at this reference first either.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-02-23T03:10:08.953" LastActivityDate="2014-02-23T03:10:08.953" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10882" PostTypeId="1" AcceptedAnswerId="10883" CreationDate="2014-02-23T03:38:39.273" Score="15" ViewCount="585" Body="&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that you have two different (factored) preconditioners for a symmetric positive definite matrix $A$:&#xA;$$A \approx B^TB$$&#xA;and&#xA;$$A \approx C^TC,$$&#xA;where the inverses of the factors $B, B^T, C, C^T$ are easy to apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When is it possible to use information from &lt;em&gt;both&lt;/em&gt; $B$ and $C$ to build a better preconditioner than either $B$ or $C$ alone? &lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2014-06-06T03:45:29.737" Title="Is there any way to do &quot;double preconditioning&quot;" Tags="|linear-algebra|krylov-method|preconditioning|condition-number|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10883" PostTypeId="2" ParentId="10882" CreationDate="2014-02-23T09:01:33.133" Score="13" Body="&lt;p&gt;You can use additive&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ P_a^{-1} x = (B^T B)^{-1} x + (C^T C)^{-1} x, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;multiplicative&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ P_m^{-1} x = (B^T B)^{-1} x + (C^T C)^{-1} \Big(x - A (B^T B)^{-1} x \Big), $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or symmetric multiplicative.  Methods of this class are available in PETSc using &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCCOMPOSITE.html&quot;&gt;PCCOMPOSITE&lt;/a&gt; in PETSc.  For example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;petsc/src/ksp/ksp/examples/tutorials$ ./ex2 -m 100 -n 100 -ksp_monitor \&#xA;      -pc_type composite -pc_composite_type multiplicative \&#xA;      -pc_composite_pcs ilu,gamg&#xA;  0 KSP Residual norm 7.088415699389e+01 &#xA;  1 KSP Residual norm 1.271768323411e+01 &#xA;  2 KSP Residual norm 1.529853612054e+00 &#xA;  3 KSP Residual norm 1.214841683459e-01 &#xA;  4 KSP Residual norm 8.341606406485e-03 &#xA;  5 KSP Residual norm 6.471990946051e-04 &#xA;  6 KSP Residual norm 8.082672366030e-05 &#xA;  7 KSP Residual norm 6.111138513482e-06 &#xA;Norm of error 6.93786e-06 iterations 7&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf&quot;&gt;users manual&lt;/a&gt; has a section on &quot;Combining Preconditioners&quot;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-02-23T09:01:33.133" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10884" PostTypeId="2" ParentId="2583" CreationDate="2014-02-23T10:42:35.943" Score="0" Body="&lt;p&gt;See the paper &quot;On Sampling-based Approximate Spectral Decomposition&quot; by Sanjiv  Kumar, Mehryar Mohri &amp;amp; Ameet Talwalkar (ICML 2009.).  It uses sampling of columns of your matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your matrix is symmetric you should do the following:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let A be your n*n matrix.   You want to reduce the computation of the eigenvalues of an n*n matrix to the computation of the eigenvalues of an k*k matrix.  First choose your value of k.  Let's say you choose k=500, since you can easily compute the eigenvalues of a 500*500 matrix.  Then, randomly choose k columns of the matrix A.  Contruct the matrix B that keeps only these columns, and the corresponding rows.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;B = A(x,x) for a random set of k indexes x&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B is now a k*k matrix.  Compute the eigenvalues of B, and multiply them by (n/k).  You now have k values which are approximately distributed like the n eigenvalues of A.  Note that you get only k values, not n, but their distribution will be correct (up to the fact that they are an approximation).&lt;/p&gt;&#xA;" OwnerUserId="7424" LastActivityDate="2014-02-23T10:42:35.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10885" PostTypeId="2" ParentId="10826" CreationDate="2014-02-23T12:18:10.267" Score="0" Body="&lt;p&gt;There are 3 ways to avoid this situation, but before use one must check if this way is suitable due to computation error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Green-Gauss cell method: here the definition of gradient is used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla u_i \approx\frac{1}{V_{i}} \int\limits_{\partial V_i}u d\overline{S}\approx \sum\limits_{k=1}^{n}{u_{f_k} S_k \overline{n}_k} , $$ where $k$ - numbers of neighbours of cell $V_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Least squares method: the error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum\limits_{k=1}^{n}{\frac{1}{d_{ik}}E_{i,k}^{2}}, E_{i,k}=\nabla u_i \cdot\Delta r_{i,k}+u_i-u_k $$ must be minimized, hence we get the components of $\nabla u_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Interpolation method. The value of gradient is interpolated from the values of gradient vector-function.&lt;/p&gt;&#xA;" OwnerUserId="4676" LastActivityDate="2014-02-23T12:18:10.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10886" PostTypeId="2" ParentId="10869" CreationDate="2014-02-23T17:58:19.887" Score="19" Body="&lt;p&gt;Adding arbitrary floating point numbers will usually give some rounding error, and the rounding error will be proportional to the size of the result. If you calculate a single sum and start by adding the largest numbers first, the average result will be larger. So you would start adding with the smallest numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you get better result (and it runs faster) if you produce four sums, for example: Start with sum1, sum2, sum3, sum4 and add four array elements in turn to sum1, sum2, sum3, sum4. Since each result is on average only 1/4th of the original sum, your error is four times smaller. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Better still: Add the numbers in pairs. Then add the results in pairs. Add those results in pairs again, and so on until you are left with two numbers to add. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very simple: Use higher precision. Use long double to calculate a sum of doubles. Use double to calculate a sum of floats. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Close to perfect: Look up Kahan's algorithm, described before. Best still used by adding starting with the smallest number. &lt;/p&gt;&#xA;" OwnerUserId="7429" LastActivityDate="2014-02-23T17:58:19.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10887" PostTypeId="2" ParentId="10869" CreationDate="2014-02-23T19:42:55.160" Score="7" Body="&lt;p&gt;The previous answers already discuss the matter at large and give sound advice, but there is an additional quirk that I'd like to mention. On most modern architectures, the &lt;code&gt;for&lt;/code&gt; loop that you have described would be performed anyway in 80-bit &lt;em&gt;extended precision&lt;/em&gt;, which guarantees additional accuracy, since all temporary variables will be put in registers. So you already have some form of safeguard from numerical errors. However, in more complicated loops, the intermediate values will be stored in memory in between the operations, and hence truncated to 64 bits. I guess that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s=0; &#xA;for \ i=1:n &#xA;    printf(&quot;Hello World&quot;);&#xA;    s=s + z_{i} ; &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;suffices to get lower precision in your summation (!!). So be very careful if you want to printf-debug your code while checking for accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the interested, &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1377616&quot;&gt;this paper&lt;/a&gt; describes a problem in a widely used numerical routine (Lapack's rank-revealing QR factorization) whose debugging and analysis was very tricky precisely because of this issue.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-02-23T19:42:55.160" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10888" PostTypeId="2" ParentId="10062" CreationDate="2014-02-23T20:07:02.880" Score="2" Body="&lt;p&gt;I will just sum up the useful remarks made in the comments.  Both approximations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E=\frac{1}{2}\left(\left(\frac{\phi_{n}-\phi_{n-1}}{t_{n}-t_{n-1}}\right)^2+V(\phi_n)\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E=\frac{1}{2}\left(\left(\frac{\phi_{n}-\phi_{n-1}}{t_{n}-t_{n-1}}\right)^2+V(\phi_{n-1})\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;will give you first-order accurate approximation of the energy.  If you wish to get higher than 1st order accuracy, you should use a more accurate finite difference approximation of $\dot\phi$.  For instance, you could get second order with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E=\frac{1}{2}\left(\left(\frac{\phi_{n+1/2}-\phi_{n-1/2}}{t_{n}-t_{n-1}}\right)^2+V(\phi_{n})\right)$.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-02-23T20:07:02.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10889" PostTypeId="1" AcceptedAnswerId="10892" CreationDate="2014-02-23T21:29:33.313" Score="12" ViewCount="321" Body="&lt;p&gt;Say I have slow and fast versions of some code, and want to report a speedup number comparing the two.  I run the slow version $n$ times and the fast version $m$ times, producing times $(s_1, \ldots, s_n)$ and $(f_1, \ldots, f_m)$.  The simplest way to produce a speedup is to average the means:&#xA;$$\frac{\bar{s}}{\bar{f}} = \frac{m \sum_{i&amp;lt;n} s_i}{n \sum_{j&amp;lt;m} f_j}$$&#xA;However, this does not take outliers into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: What is the best statistic to use when reporting speedup numbers?&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2014-02-25T14:35:18.147" Title="Correct statistics for reporting speedup results" Tags="|statistics|publications|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10890" PostTypeId="1" CreationDate="2014-02-24T01:39:09.810" Score="1" ViewCount="137" Body="&lt;p&gt;I have to approximate the smallest eigenvalue of the following generalized eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;- \nabla \cdot  D(x) \nabla p(x) + \alpha(x) p(x) = \lambda \beta(x) p(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;over a domain like follows (many more subdomains, this is only an approximation to illustrate the problem)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ib3l4.gif&quot; alt=&quot;Domain&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigenvalues are all positive, so to approximate the smallest one I invert the left side of the equation after the spatial discretization, and use Arnoldi method. The functions $D(x)$, $\alpha(x)$ and $\beta(x)$ are piecewise constant over the whole domain, being probably different over different subdomains. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have a code to approximate the smallest eigenvalue and associated eigenvector with a Continuous Galerkin FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering which locally conservative method can I use to solve the eigenvalue problem (i.e., I want the solution to satisfy the balance equation over each subdomain). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I know:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OPTION 1:&lt;/strong&gt; I know that using discontinuous Galerkin FEM the balance equation will be satisfied over each subdomain, but it usually deal with the mixed formulation of the problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;- \nabla \cdot  q(x) + \alpha(x) p(x) = \lambda \beta(x) p(x) \\&#xA;q(x) - D(x) \nabla p(x) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and it introduces a lot of degrees of freedom to the system. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OPTION 2:&lt;/strong&gt; I have read that it can also be done using a Hybridizable discontinuous Galerkin, because the we can apply a static condensation technique and the degrees of freedom related to $q$ are the only ones that we have to solve in a coupled way. It works fine for source problems, but I think that we can not apply this technique with an eigenvalue problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit: (why static condensation will not work for eigenproblems?)&lt;/strong&gt;&#xA;If dealing with a source problem, we form the linear system for the hybridizable discontinuous Galerkin as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\begin{array}{c c}&#xA;A &amp;amp; B \\&#xA;C &amp;amp; D&#xA;\end{array}\right]&#xA;\left[\begin{array}{c}&#xA;p \\&#xA;q&#xA;\end{array}\right]&#xA;=&#xA;\left[\begin{array}{c}&#xA;r \\&#xA;s&#xA;\end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know that matrix $A$ is block diagonal because the degrees of freedom are completelly decoupled (from element to element), so we perform the static condensation to solve the following problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;(D-CA^{-1}B)q = s-CA^{-1} r&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then we recover $p$ with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Ap = r-Bq&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we form the algebraic eigenvalue problem with discontinuous galerking we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\begin{array}{c c}&#xA;A &amp;amp; B \\&#xA;C &amp;amp; D&#xA;\end{array}\right]&#xA;\left[\begin{array}{c}&#xA;p \\&#xA;q&#xA;\end{array}\right]&#xA;=&#xA;\lambda \left[\begin{array}{c c}&#xA;E &amp;amp; 0 \\&#xA;0 &amp;amp; 0&#xA;\end{array}\right]&#xA;\left[\begin{array}{c}&#xA;p \\&#xA;q&#xA;\end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, as in a linear system, we try to isolate $p$ in the first equation, and we obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p = (A-\lambda E)^{-1}B q&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and after substitution in the second equation we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;(D + C(A-\lambda E)^{-1}B ) q = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrices $A$ and $E$ are block diagonal, so easy to invert, but the eigenvalue $\lambda$ now is inside the matrix to invert, so the eigenvalue problem has been converted into a nonlinear eigenvalue problem. Different linear approximations of  $(A-\lambda E)^{-1}$ (taylor expansion on $\lambda = 0$ or $\lambda = \lambda_0$, for instance) will solve the problem of the nonlinear eigenvalue problem, but then the algorithm will converge much more slower and succesive corrections will be necessaries. Thus static condensation is not a good idea here.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OPTION 3:&lt;/strong&gt; There are some papers that talk about locally conservative continuos Galerkin methods: &lt;a href=&quot;http://dx.doi.org/10.1006/jcph.2000.6577&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Continuous Galerkin Method Is Locally Conservative&lt;/a&gt;, and &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/060666305&quot; rel=&quot;nofollow noreferrer&quot;&gt;Locally Conservative Fluxes for the Continuous Galerkin Method&lt;/a&gt;. The first one seems to perform some extra calculations to approximate the fluxes, that for my is equivalent to the mixed formulation. And the second one says that is posible to deal with the primal formulation, but also develop the article about a formulation that includes the fluxes $q(x)$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;QUESTION:&lt;/strong&gt; it is no clear for me that locally conservation can be obtained with a primal formultaion (like in the first equation I posted), moreover taking into accout that we are solving an eigenvalue problem. Can it be done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea or advice about literature on the topic would be greatly appreciated. I am mainly interested in FEM methods, but any idea is welcome.&lt;/p&gt;&#xA;" OwnerUserId="7277" LastEditorUserId="7277" LastEditDate="2014-02-24T16:35:06.553" LastActivityDate="2014-02-24T16:35:06.553" Title="Locally conservative method for differential generalized eigenvalue problem" Tags="|pde|finite-element|eigensystem|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10891" PostTypeId="2" ParentId="10869" CreationDate="2014-02-24T02:37:10.237" Score="2" Body="&lt;p&gt;Since you didn't restrict the data-type to be used, to achieve a perfectly accurate result, simply use arbitrary length numbers... in which case the order won't matter.  It will be much slower, but obtaining perfection does take time.&lt;/p&gt;&#xA;" OwnerUserId="7432" LastActivityDate="2014-02-24T02:37:10.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10892" PostTypeId="2" ParentId="10889" CreationDate="2014-02-24T05:28:06.567" Score="9" Body="&lt;p&gt;In addition to all that Bill Barth has already said above, let me mention that people often report the &lt;em&gt;fastest&lt;/em&gt; of several runs. The rationale is that the actual run time is the &lt;em&gt;ideal&lt;/em&gt; run time plus any number of slow downs resulting from other processes running, OS delays, network delays, etc. Since these are all noise we are not interested in, using the &lt;em&gt;fastest&lt;/em&gt; run time comes closest to the one we really want to know.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-24T05:28:06.567" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10893" PostTypeId="2" ParentId="10889" CreationDate="2014-02-24T08:07:17.217" Score="1" Body="&lt;p&gt;I suggest you use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Median&quot; rel=&quot;nofollow&quot;&gt;median&lt;/a&gt; to give a statistical estimate. Unlike the mean, the median is not corrupted by outliers. &lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-02-24T08:07:17.217" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10894" PostTypeId="2" ParentId="10413" CreationDate="2014-02-24T16:12:46.547" Score="6" Body="&lt;p&gt;Comment regarding your Step 3: the original paper from Hou and Wu uses a Galerkin method. So only the basis $(\phi_i)$ is used. In that case, the matrix remains symmetric. My reply will consider the case of a Galerkin method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $a(\cdot,\cdot)$ be the variational form associated with your problem. I assume that $k$ satisfies $k(x) \geq \kappa &amp;gt; 0$. To simplify the discussion, I will assume homogeneous Dirichlet condition (and no Neumann condition). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem becomes: find $u \in H^1_0(\Omega)$ such that &#xA;$$&#xA;a(u,v) = \int_{\Omega} fv, \quad \forall v \in H^1_0(\Omega).&#xA;$$&#xA;Note that $a$ defines an inner-product on $H^1_0(\Omega)$ and that the solution $u$ satisfies also &#xA;$$&#xA;\frac{1}{2} a(u,u) - \int_{\Omega} fu = \inf_{v \in H^1_0(\Omega)} \left[ a(v,v) - \int_{\Omega} fv \right].&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A key result is &#xA;$$&#xA;H^1_0(\Omega) = \bigoplus_{i=1}^{I} ``H_0^1(K_i)&quot; \oplus E(\Gamma)&#xA;$$&#xA;where $``H_0^1(K_i)&quot;$ denotes functions from $H^1_0(K_i)$ extended by 0 over $\Omega$. $\Gamma$ denotes the interior skeleton of the mesh, i.e. &#xA;$$&#xA;\Gamma = \left( \bigcup_{i=1}^{I} \partial K_i \right) \backslash \partial \Omega.&#xA;$$&#xA;and $E(\Gamma)$ is the set of energy-minimizing extensions of function in $H^{1/2}_{00}(\Gamma)$, i.e.&#xA;$$&#xA;w \in E(\Gamma) \Leftrightarrow w_{|\partial K_i} \in H^{1/2}(\partial K_i) \mbox{ and } \nabla \cdot ( k(x) \nabla w ) = 0 \mbox{ in } K_i&#xA;$$&#xA;The (previously written) decomposition of $H^1_0(\Omega)$ is an orthogonal decomposition for the inner-product $a$.&#xA;With this orthogonality, minimizing the energy over $H^1_0(\Omega)$ is equivalent to minimizing the energy over each subspace $H^1_0(K_i)$ and $E(\Gamma)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can show that the projection of $u$ into $H^1_0(K_i)$ are &quot;small&quot;. MsFEM usually neglects those components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The functions $\phi_i$ of MsFEM span a subspace of $E(\Gamma)$. You can show that when $k$ is a constant (or 1), the classical P1 or Q1 finite element basis functions span a subspace for the corresponding $E(\Gamma)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding Question 2, to the best of my knowledge, the choice of traces $\Phi_i$ remains a &quot;heuristic&quot; parameter in these methods. When a homogeneous Dirichlet problem is posed, interpolating the boundary condition does not yield anything. &lt;/p&gt;&#xA;" OwnerUserId="7440" LastEditorUserId="7440" LastEditDate="2014-04-12T16:50:37.797" LastActivityDate="2014-04-12T16:50:37.797" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10895" PostTypeId="2" ParentId="10871" CreationDate="2014-02-24T20:53:09.430" Score="3" Body="&lt;p&gt;IF you know a priori where your boundary layer is located (it could be a big IF when $k$ and $c$ become more general), you could enrich your approximation space with a function capturing the layer (see GFEM or MSFEM).&lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2014-02-24T20:53:09.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10896" PostTypeId="1" CreationDate="2014-02-25T02:24:11.803" Score="3" ViewCount="1815" Body="&lt;p&gt;I am trying to build a complex simulator for the transport of mass &amp;amp; heat in porous media. I am currently following coarsely the algorithm laid out in an older software and have got the simulator to work, but the iterative solvers that I use run into trouble at large timesteps and prevent the convergence of the system to steady state. I was wondering if somebody could help me to improve my method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simulator builds the Jacobian matrix by numeric differentiation using double precision numbers. The desired final matrix size is in the order of 500,000x500,000 with each row having about 15-20 entries. The structure of the Jacobian is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The diagonal entries stem from the thermodynamic properties of the elements in the system.&lt;/li&gt;&#xA;&lt;li&gt;The off-diagonal elements describe the transport of mass/heat between the elements. They scale with the time step. Due to the formulation currently used they are non-symmetric but only differ by a volume ratio between the elements; if need be they could be made symmetric.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The system J * (-dx) = r is solved using iterative solvers; in particular BICGSTAB with ILU0 works ok but I have tried others as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem arises when the timesteps get larger. The diagonal elements stay the same, but the off-diagonal elements scale with the timestep and the solvers used appear to run into problems once the timestep reaches ~1e+8 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to proceed? I was thinking about one or more of the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use a solver with higher precision. But will this work, given that my derivative is done numerically to only ~8 significant digits?&lt;/li&gt;&#xA;&lt;li&gt;Is decomposing the matrix an option? If so, what decomposition would work?&lt;/li&gt;&#xA;&lt;li&gt;Are other solvers an option?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any help is highly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Peter&lt;/p&gt;&#xA;" OwnerUserId="7444" LastActivityDate="2014-02-25T17:09:51.630" Title="Solving Newton-Raphson step with ill-conditioned sparse matrix" Tags="|matrix|linear-solver|sparse-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10897" PostTypeId="2" ParentId="10869" CreationDate="2014-02-25T02:39:42.127" Score="4" Body="&lt;p&gt;Of the 2 options, adding from smaller to larger will produce less numerical error then adding from larger to smaller.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &gt;20 years ago in my &quot;Numerical Methods&quot; class the instructor stated this and it occurred to me that this was still introducing more error than necessary because of the relative difference in value between the accumulator and the values that were being added.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logically, a preferable solution is to add the 2 smallest numbers in the list, then re-insert the summed value into the sorted list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To demonstrate it, I worked out an algorithm that could do that efficiently (in space and time) by using the space freed up as elements were removed from the primary array to build a secondary array of the summed values which were inherently ordered since the additions were of the sums of values that were always increasing.  On each iteration the &quot;tips&quot; of both arrays are then checked to find the 2 smallest values.&lt;/p&gt;&#xA;" OwnerUserId="7443" LastActivityDate="2014-02-25T02:39:42.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10898" PostTypeId="2" ParentId="10896" CreationDate="2014-02-25T09:04:16.653" Score="2" Body="&lt;p&gt;For a general list of strategies, see:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/30/276&quot;&gt;Why is Newton&amp;#39;s method not converging?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/513/276&quot;&gt;Why is my iterative linear solver not converging?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I ask a bunch of questions throughout the answer; please respond to them in your question as edits, because some of the answers to those questions are information that will help other users answer your question.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What is the best way to proceed? I was thinking about one or more of the following:&lt;/p&gt;&#xA;&lt;p&gt;Use a solver with higher precision. But will this work, given that my derivative is done numerically to only ~8 significant digits?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I believe the main consequence of using higher precision in the solver will be that you preserve more significant digits within the solver. If the finite differencing is done outside the solver, this will (obviously) be unaffected; if the derivative is calculated in higher precision, it gives you more options in terms of selecting the numerical differencing parameter. Have you tried varying that differencing parameter and examining its effect on the conditioning of your system?&lt;/p&gt;&#xA;&lt;p&gt;Is using an analytical Jacobian matrix (or a reasonably good analytical approximation thereof) an option? Something that can be problematic is when the numerical differencing parameter is of the same order as relevant numerical parameters (or scales) in your problem. This issue was one I encountered in debugging a reaction-diffusion simulation: a matrix-free approximation of the Jacobian was not sufficiently accurate and caused ill-conditioning issues. Replacing the matrix-free approximation of the Jacobian matrix solved my issues. (See also: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10177/when-is-newton-krylov-not-an-appropriate-solver&quot;&gt;When is Newton-Krylov not an appropriate solver?&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is decomposing the matrix an option? If so, what decomposition would work?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, it's not. Using something like an sparse LU decomposition will likely require too much memory (you already have ~10 million nonzeroes, and fill-in will only increase that number).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are other solvers an option?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Probably. What have you tried? Something that would also help for analysis would be an estimate of the conditioning of your system. GMRES (without restarts) provides one such estimate, and it would be helpful to know if the linear systems you're solving are ill-conditioned.&lt;/p&gt;&#xA;&lt;p&gt;Specifically, it would be helpful to know what:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;time-steppers&lt;/li&gt;&#xA;&lt;li&gt;preconditioners&lt;/li&gt;&#xA;&lt;li&gt;linear solvers&lt;/li&gt;&#xA;&lt;li&gt;libraries&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;you are using. (Mentioning ILU(0) and BiCGStab is a good start.)&lt;/p&gt;&#xA;&lt;p&gt;It would also be helpful to see equations, and understand more about your problem. Is it parabolic? Are there likely to be the same sorts of errors at multiple scales? Multigrid methods might be helpful for preconditioning, and scale better than ILU(0).&lt;/p&gt;&#xA;&lt;p&gt;1e8 is a really large time step, without any other context. Why such a large number? Is there a scale issue? Can the problem be rescaled or nondimensionalized?&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-02-25T17:09:51.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10900" PostTypeId="2" ParentId="10889" CreationDate="2014-02-25T14:35:18.147" Score="0" Body="&lt;p&gt;If the standard deviation is not negligible, you could use two &lt;a href=&quot;https://en.wikipedia.org/wiki/Box_plot&quot; rel=&quot;nofollow&quot;&gt;box plots&lt;/a&gt; side-by-side, constructed each with the timing of one of the algorithms. They are by all means not standard in numerical analysis, but they do a great job in displaying this kind of information.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-02-25T14:35:18.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10901" PostTypeId="1" CreationDate="2014-02-25T14:58:14.133" Score="3" ViewCount="62" Body="&lt;p&gt;Let $\Sigma$ be a covariance matrix (e.g. symmetric positive definite). For arbitrary vectors $\epsilon$, I need to compute $\chi^2 \equiv \epsilon^\top\Sigma^{-1}\epsilon$, which I do using a Cholesky decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\Sigma=I$, then $\chi^2$ would be the 2-norm of $\epsilon$. I would then say that each coefficient of $\epsilon$ had the same weight in $\chi^2$. Similarly, if $\Sigma$ is a diagonal matrix with positive diagonal elements $(\lambda_1,\ldots,\lambda_N)$, $\epsilon_i$ would be weighted by $\lambda_i^{-1}$ :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \chi^2 = \sum_{i=1}^N \frac{\epsilon_i^2}{\lambda_i} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is then trivial to conclude that, in general (i.e. irrespective of the value of $\epsilon$), &lt;em&gt;the coefficients of $\epsilon$ that contribute most to $\chi^2$ are those where $\lambda_i$ is biggest.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, $\Sigma$ is not diagonal. We can of course diagonalize it: $PDP^\top \equiv \Sigma$ ($D$ is diagonal and $P$ is the matrix whose columns are the normalized eigenvectors of $\Sigma$; $P$ is unique if we order the eigenvalues from largest to smallest in $D$). Diagonalization allows us to isolate independent contributions to $\chi^2$, and we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\chi^2 = \sum_{i=1}^N \frac{(p_i^\top\epsilon)^2}{\lambda_i} \qquad \text{where} \quad \Sigma p_i \equiv \lambda_i p_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What previously was a scalar weight now becomes a vector weight, and is $p_i/\sqrt\lambda_i$&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am only interested in dominant contributions to $\chi^2$, i.e. where $\lambda_i$ is small and $p_{ij}$ is large.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Analytically, what is a proper way to determine dominant contributions to $\chi^2 \equiv \epsilon^\top\Sigma^{-1}\epsilon$? Is it possible to tell that some region of $\epsilon$ will be weighted more or less than another one, accounting for the covariances?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Numerically, the $1/\lambda$ dependence of the discussion suggests that the smaller the eigenvalue, the larger the contribution. That is very unfortunate: my $\Sigma$ matrix is very ill-conditioned, and computation of the smallest eigenvalues is probably just gibberish. Is there a way to do implement this discussion either robustly or by avoiding completely the calculation of eigenvalues?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="733" LastActivityDate="2014-02-25T14:58:14.133" Title="Dominant contributions of a quadratic form" Tags="|eigensystem|matrix|inverse|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10902" PostTypeId="2" ParentId="10864" CreationDate="2014-02-25T15:59:27.240" Score="0" Body="&lt;p&gt;I suppose you are looking for using openmp thread pool. I am not sure which runtme implementation you are using but normally you can use omp_set_num_threads(int) to set the thread pool size on initialization. Only if runtime is unable to create a threadpool, then it will create threads dynamically at run time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am not sure how you identified your bottleneck is thread start and stop. I advise you to profile your code to understand where you are actually hitting the bottleneck.&lt;/p&gt;&#xA;" OwnerUserId="4841" LastActivityDate="2014-02-25T15:59:27.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10903" PostTypeId="2" ParentId="10896" CreationDate="2014-02-25T17:02:01.680" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is decomposing the matrix an option? If so, what decomposition would work?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, I think that is a very reasonable option. 500000 is not a huge number of&#xA;equations and your system is quite sparse. For your current, unsymmetric case&#xA;you would need to perform an $LU$ decomposition. If you can symmetrize it, as you&#xA;suggest, you can save almost 1/2 the memory by performing an $LL^T$ or $LDL^T$&#xA;decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest trying the MUMPS sparse solver:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mumps.enseeiht.fr/&quot; rel=&quot;nofollow&quot;&gt;http://mumps.enseeiht.fr/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has options for all three types of factorizations. It is designed to run in&#xA;parallel on distributed memory computers but also runs quite efficiently on&#xA;single-CPU or multicore machines. It has an out-of-core option that you can use&#xA;to reduce memory usage.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-02-25T17:02:01.680" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10904" PostTypeId="2" ParentId="10610" CreationDate="2014-02-25T22:47:20.740" Score="4" Body="&lt;p&gt;Quasi random numbers are not statistically independent so if your algorithm requires this, you should not use them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One area where you can usually use quasi random numbers instead of pseudo random numbers is in the evaluation of integrals via monte carlo integration. In this application, the fact that quasi random numbers are not truly random is not an issue since the statistical properties don't matter. What matters is how well the sequence samples the integrand space and quasi random numbers fill space better than pseudo random numbers, leading to improved convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find the following article by the Numerical Algorithms Group (NAG) useful in describing this in more detail&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.nag.co.uk/IndustryArticles/introduction_to_quasi_random_numbers.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.nag.co.uk/IndustryArticles/introduction_to_quasi_random_numbers.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also note that Mathworks suggest a couple of other applications of quasi-random numbers along with monte-carlo integration: space filling experimental designs and global optimisation &lt;a href=&quot;http://www.mathworks.co.uk/help/stats/generating-quasi-random-numbers.html#br5k9hi-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.co.uk/help/stats/generating-quasi-random-numbers.html#br5k9hi-9&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One issue you need to be wary of is how many dimensions a quasi-random sequence retains its desirable properties for. The standard Halton sequence, for example, does not do well in high dimensions.  You can see this directly using MATLAB&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a 15 dimensional halton sequence and evaluate 1000 points in each dimension:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; rng default&#xA;&amp;gt;&amp;gt; p=haltonset(15);&#xA;&amp;gt;&amp;gt; x0=net(p,1000);&#xA;&amp;gt;&amp;gt; size(x)&#xA;&#xA;ans =&#xA;        1000          15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Plot dimensions 1 and 2 against each other&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; scatter(x0(:,1),x0(:,2),5,'b')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ecIeN.png&quot; alt=&quot;Dimensions 1 and 2&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looks good so far. Not so good when we plot dimension 14 against 15&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; scatter(x0(:,14),x0(:,15),5,'b')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tNffk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various ways of dealing with this such as scrambling (see &lt;a href=&quot;http://www.mathworks.co.uk/help/stats/qrandset.scramble.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.co.uk/help/stats/qrandset.scramble.html&lt;/a&gt; for example) but you may also choose to select different sequences that are known to be good for higher dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have more experience with using the NAG Toolbox for MATLAB (&lt;a href=&quot;http://www.nag.co.uk/numeric/MB/start.asp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.nag.co.uk/numeric/MB/start.asp&lt;/a&gt;) which provides 2 sobol generators (one that's good for 10,000 dimensions, another thats good for 1111 dimensions), a Niederreiter generator (318 dimensions) and a Faure generator (40 dimensions).  &lt;/p&gt;&#xA;" OwnerUserId="7454" LastActivityDate="2014-02-25T22:47:20.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10905" PostTypeId="1" CreationDate="2014-02-26T03:50:41.413" Score="4" ViewCount="701" Body="&lt;p&gt;I am trying to solve Poisson-Nernst-Planck system of equations for ions diffusion problem using finite volume method. Nernst-Planck equation for mass transport and Poisson equation for electrostatic charge distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bYoCC.jpg&quot; alt=&quot;Where c is concentration, t, time, D, diffusion coef., beta = const., z, valance number and psi is electric field, epsilon is media permittivity and F is Faraday constant&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My jacobian is sparse and ill-conditioned (condition number ~ 1E+14). For a system of 4 ions and three control volumes an schematic Jacobian looks like this:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/hUzWo.jpg&quot; alt=&quot;the first four rows for mass and the fifth row potential&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use Newton-Raphson method to solve it. The timesteps don't go larger than 1E-7 seconds and eventually it doesn't converge and crashes. In the mean time when I pause the run and check the results they are not correct.&lt;br&gt;&#xA;One solution that I found was to assume $\epsilon/F = 0$ as $\epsilon/F \sim 1E-14$). In this case condition number becomes ~1E+12 although after ~ 2000 timesteps condition number jumps to 1E+30. But the code goes to larger timesteps (up to 1E-3 second) and the result seems right but not very precise. I need to speed up the code as this timestep is not practical. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do you think using direct solver is an option?&lt;/li&gt;&#xA;&lt;li&gt;Do you think higher quality initial guess for electric potential will help?&lt;/li&gt;&#xA;&lt;li&gt;Is using solvers with quad precision helpful? &lt;/li&gt;&#xA;&lt;li&gt;Is it possible that jacobian assembly is not right?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;many thanks&lt;/p&gt;&#xA;" OwnerUserId="7456" LastEditorUserId="53" LastEditDate="2014-03-30T01:51:57.357" LastActivityDate="2014-06-28T08:30:50.137" Title="Poisson-Nernst-Planck equations with ill-conditioned sparse matrix" Tags="|sparse-matrix|poisson|newton-method|jacobian|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10908" PostTypeId="1" AcceptedAnswerId="10909" CreationDate="2014-02-26T12:31:13.940" Score="1" ViewCount="421" Body="&lt;p&gt;I have real symmetric positive definite matrix $M = \left(\begin{matrix} a &amp;amp; b \\ b &amp;amp; c \end{matrix}\right)$ where $a,b,c \in R,\  a,c&amp;gt;0,\ \left|b\right|&amp;lt;2\sqrt{a c}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to define CONVEX objective function of $M$ (preferably polynomial) which has minimum value for prescribed value of trace and determinant of $M$, i.e. $tr(M) = L$ and $det(M) = V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Natural definition of such function is probably&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(M) = f(a,b,c) = \left( \frac{tr(M)}{L} - 1 \right)^2 + \left( \frac{det(M)}{V} - 1 \right)^2 = \left( \frac{a + c}{L} - 1 \right)^2 + \left( \frac{a c - b^2}{V} - 1 \right)^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that this function is not convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody have an idea how to define this function to be convex or can it be proved that this is not possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help.&lt;/p&gt;&#xA;" OwnerUserId="7461" LastActivityDate="2014-02-26T14:52:19.430" Title="Convex objective function of matrix with prescribed determinant and trace" Tags="|optimization|convex-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10909" PostTypeId="2" ParentId="10908" CreationDate="2014-02-26T14:52:19.430" Score="2" Body="&lt;p&gt;You only need a convex function that involves the determinant. You can then minimize it subject to the constraint $\text{tr}\; M = L$ which is a linear constraint and, therefore, easy to handle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I believe that no such function exists so that $\phi(M)$ is minimal where $\text{det}\; M=V$ and only there, in general. Take, for example, the case where you want $V=0$ and shift your function so that $\phi(M)=0$ for all matrices with zero determinant. Then you are looking for a function that is zero for all degenerate matrices and positive for all other matrices. Now consider $M_1,M_2$ so that $\text{det}\; M_i=0$, i.e., $\phi(M_i)=0$. Convexity of $\phi$ would imply that $\phi((M_1+M_2)/2)=0$, but even if $\text{det}\;M_i=0$ you typically do not have $\text{det}\;(M_1+M_2)/2=0$ -- in other words, the set of matrices where $\phi$ is minimal (the set of matrices with zero determinant) is not convex itself, and consequently the objective function could only be convex if it is zero not only on the degenerate matrices but indeed their convex hull. But then, the objective function would also be zero on some matrices that do not have zero determinant -- not what you probably want.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-26T14:52:19.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10910" PostTypeId="1" AcceptedAnswerId="10925" CreationDate="2014-02-26T14:55:54.930" Score="2" ViewCount="2230" Body="&lt;p&gt;I have a master equation for $P(N_A^+,N_B^+,N_C^+,t)$, with $N_A^+,N_B^+,N_C^+$ all discrete. The numerical integration is done by this Matlab &lt;a href=&quot;https://dl.dropboxusercontent.com/u/7383429/Archive.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;program&lt;/a&gt; using Euler's method. Despite the crude Euler's method, the results from numerical integration are in agreement with the simulation, so the correctness of the program is not the focus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to see how $P(N_A^+,N_B^+,N_C^+,t)$ evolves over time. Since $P(N_A^+,N_B^+,N_C^+,t)$ has 3 more coordinates in addition to time, it is really hard to visualize the changes. Ideally, I want to have something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/1GqoN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I cannot find any function in Matlab that can does something similar. The closest thing I can find is Matlab's &lt;code&gt;contourslice&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By running the Matlab program mentioned above with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ABC_driver(0.5,100,20,200,10,true);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you will get something like the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/yOmZE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/kE1oP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can guess what happens to $P(N_A^+,N_B^+,N_C^+,t)$, but it is nowhere as obvious as the atomic orbital graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how can I plot $P(N_A^+,N_B^+,N_C^+,t)$ in way similar to the atomic orbital density plot? Or can anyone suggest better way to visualize the time evolution of $P(N_A^+,N_B^+,N_C^+,t)$?&lt;/p&gt;&#xA;" OwnerUserId="3512" LastActivityDate="2014-02-27T11:01:23.280" Title="visualization of 3D probability flow" Tags="|matlab|visualization|stochastic|probability|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10911" PostTypeId="2" ParentId="10910" CreationDate="2014-02-26T15:09:28.230" Score="4" Body="&lt;p&gt;Programs like Visit and Paraview can do &quot;volume rendering&quot;, which is what you show in your figure. You just need to export the data you have in a format that either of these programs can read.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-26T15:09:28.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10912" PostTypeId="2" ParentId="10905" CreationDate="2014-02-26T15:21:06.463" Score="0" Body="&lt;p&gt;If you write your system as a single matrix, you may need to scale the two equations so that they have roughly the same order of magnitude. Rather than describing in great detail what to do and why, let me simply link to an example where we have done this: &lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/step_32.html#Thescalingofdiscretizedequations&quot; rel=&quot;nofollow&quot;&gt;http://dealii.org/developer/doxygen/deal.II/step_32.html#Thescalingofdiscretizedequations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-02-26T15:21:06.463" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10913" PostTypeId="1" CreationDate="2014-02-26T17:30:08.490" Score="3" ViewCount="99" Body="&lt;p&gt;As you will know there are different numerical integrals (I believe Levin's method is the most popular one) for the numerical quadrature of oscillating integrands which may roughly speaking be written as a product of oscillating part and non-oscillating amplitude $exp(i \omega t) A(t)$ (the form can of course be slightly more general). Integrands of the form $exp(i \omega t) A(t)+B(t)$ may be treated using a method for oscillatory and one for non-oscillatory for the first and second part, respectively. I wonder if there are routines also for the efficient quadrature of integrands which cannot be analytically decomposed as $exp(i \omega t) A(t)+B(t)$ but which nevertheless feature a rapidly oscillatory behavior (you can assume fixed frequency) . I.e. they behave like $exp(i \omega t) A(t)+B(t)$ with some unknown $A$ and $B$. It appears to me that I could in principle do better than using a conventional quadrature for non-oscillating integrands as I could try to find $A$ and $B$ using smooth (and successively refining) fits and then run a quadrature for oscillating integrals in order to obtain a numerical estimate for the integral. I wonder if there are any known efficient methods for such problems.  &lt;/p&gt;&#xA;" OwnerUserId="1943" LastActivityDate="2014-02-26T17:30:08.490" Title="Numerical Quadrature of Oscillating Integral With Non-oscillating part" Tags="|quadrature|oscillations|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10914" PostTypeId="2" ParentId="8075" CreationDate="2014-02-26T17:35:09.023" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://www3.nd.edu/~ccl/software/makeflow/&quot; rel=&quot;nofollow&quot;&gt;Makeflow&lt;/a&gt; describes dependencies using the Make syntax.  It natively supports SGE as well.&lt;/p&gt;&#xA;" OwnerUserId="7466" LastActivityDate="2014-02-26T17:35:09.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10915" PostTypeId="1" CreationDate="2014-02-26T18:51:05.740" Score="0" ViewCount="137" Body="&lt;p&gt;I am looking mainly for c/c++ implementations but also for theoretical algorithms to compute gaps between smallest positive eigenvalues of symmetric, singular matrix or real numbers. To be precise, I want to find index n of the eigenvalue such that between $\lambda_n$ and $\lambda_{n+1}$ (both are nonzero) there is the largest gap (I assume that eigenvalues are sorted). The matrix I am dealing with is huge, so I am allowed to use only sparse representation. I cannot store all the zero entries due to memory constraints.&#xA;Can you recommend any c/c++ software which can compute it to me and uses sparse matrix representations? I have tried to install fortran ARPACK, but with no success. &#xA;Maybe you know some papers with algorithms solving this problem. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="7468" LastEditorUserId="27" LastEditDate="2014-02-26T19:20:46.703" LastActivityDate="2014-02-26T19:20:46.703" Title="Algorithms to compute largest gap between smallest nonzero eigenvalues of sparse symmetric matrix" Tags="|linear-algebra|numerical-analysis|eigenvalues|banded-matrix|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10916" PostTypeId="2" ParentId="8117" CreationDate="2014-02-26T22:36:23.450" Score="2" Body="&lt;p&gt;I recommend ResearchGate's computational science topic for general discussion about computational science. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/topic/computational_science/&quot; rel=&quot;nofollow&quot;&gt;https://www.researchgate.net/topic/computational_science/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7467" LastActivityDate="2014-02-26T22:36:23.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10918" PostTypeId="1" AcceptedAnswerId="10963" CreationDate="2014-02-26T23:48:14.707" Score="10" ViewCount="5203" Body="&lt;p&gt;I have an optimization problem that looks like the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{rl}&#xA;\min_{J,B} &amp;amp; \sum_{ij} |J_{ij}|\\&#xA;\textrm{s.t.} &amp;amp; MJ + BY =X&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, my variables are &lt;em&gt;matrices&lt;/em&gt; $J$ and $B$, but the entire problem is still a linear program; the remaining variables are fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I attempt to enter this program into my favorite linear programming tools, I run into some trouble.  Namely, if I write this in &quot;standard&quot; linear program form, the parameter matrices $M$ and $Y$ end up getting repeated a ton of times (once for each column of $X$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm and/or package that can deal with optimizations of the form above? Right now I'm running out of memory because $M$ and $Y$ have to be copied so many times!&lt;/p&gt;&#xA;" OwnerUserId="5129" LastEditorUserId="5129" LastEditDate="2014-02-28T23:14:45.377" LastActivityDate="2014-03-12T09:29:47.080" Title="Linear programming with matrix constraints" Tags="|optimization|convex-optimization|linear-programming|" AnswerCount="4" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10919" PostTypeId="1" AcceptedAnswerId="10920" CreationDate="2014-02-26T23:49:38.387" Score="6" ViewCount="370" Body="&lt;p&gt;I recently discovered that, in MATLAB 2013b at least, it is significantly faster to do repeated multiplication rather than integer powers.  That is, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tic;test4p = a.^4;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is much slower than&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tic;test4m = a.*a.*a.*a;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://www.walkingrandomly.com/?p=5377&quot;&gt;http://www.walkingrandomly.com/?p=5377&lt;/a&gt; for full details.  Is there any numerical reason (accuracy etc) why repeated multiplication might be a bad idea?&lt;/p&gt;&#xA;" OwnerUserId="7454" LastEditorUserId="7454" LastEditDate="2014-02-27T00:18:06.313" LastActivityDate="2014-02-27T03:25:29.790" Title="Is there any numerical reason for not using repeated multiplication instead of integer powers?" Tags="|matlab|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10920" PostTypeId="2" ParentId="10919" CreationDate="2014-02-27T02:18:56.800" Score="4" Body="&lt;p&gt;I'm surprised that MATLAB does not do &lt;a href=&quot;http://en.wikipedia.org/wiki/Exponentiation_by_squaring&quot; rel=&quot;nofollow&quot;&gt;exponentiation-by-squaring&lt;/a&gt; for positive integer powers, since that algorithm should require fewer multiplications. I think a consequence of that should be that numerical errors accumulate less quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Addition-chain_exponentiation&quot; rel=&quot;nofollow&quot;&gt;Optimal addition-chain multiplication&lt;/a&gt; of exponents is an NP-complete problem, so it's not worth it to figure out the sequence of multiplies and exponentiations that minimizes the total number of multiplicative operations.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-02-27T02:18:56.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10921" PostTypeId="2" ParentId="10919" CreationDate="2014-02-27T03:19:27.150" Score="6" Body="&lt;p&gt;the numeric reason would be the loss precision. at each multiplication you lose a little bit of precision. If you want to implement power by multiplication, then it's better to look up the algorithm. I once coded the algorithm from Stepanov's &quot;Elements of Programming&quot; book, it's the fastest possible way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: I found the earlier version of Stepanov's book as &lt;a href=&quot;http://www.cs.rpi.edu/~musser/gsd/notes-on-programming-2006-10-13.pdf&quot;&gt;Notes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Search for fast_power algorithm in section 10.3. It's so beautiful, one of the most elegant pieces of code ever&lt;/p&gt;&#xA;" OwnerUserId="7470" LastEditorUserId="7470" LastEditDate="2014-02-27T03:25:29.790" LastActivityDate="2014-02-27T03:25:29.790" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10922" PostTypeId="1" CreationDate="2014-02-27T04:34:20.947" Score="61" ViewCount="10603" Body="&lt;p&gt;I'm considering learning a new language to use for numerical/simulation modelling projects, as a (partial) replacement for the C++ and Python that I currently use. I came across &lt;a href=&quot;http://julialang.org/&quot; rel=&quot;noreferrer&quot;&gt;Julia&lt;/a&gt;, which sounds kind of perfect. If it does everything it claims, I could use it to replace both C++ &lt;em&gt;and&lt;/em&gt; Python in all my projects, since it can access high-level scientific computing library code (including PyPlot) as well as running for loops at a similar speed to C. I would also benefit from things like proper coroutines that don't exist in either of the other languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's a relatively new project, currently at version 0.x, and I found various warnings (posted at various dates in the past) that it's not quite ready for the day to day use. Consequently, I would like some information about the status of the project right now (February 2014, or whenever an answer is posted), in order to help me assess whether I personally should consider investing the time to learn this language at this stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate answers that focus on &lt;em&gt;specific relevant facts about the Julia project&lt;/em&gt;; I'm less interested in opinions based on experience with other projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, a comment by Geoff Oxberry suggests that the Julia API is still in a state of flux, requiring the code to be updated when it changes. I would like to get an idea of the extent to which this is the case: which areas of the API are stable, and which are likely to change?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess typically I would mostly be doing linear algebra (e.g. solving eigenproblems), numerical integration of ODEs with many variables, and plotting using PyPlot and/or OpenGL, as well as low-level C-style number crunching (e.g. for Monte Carlo simulations). Is Julia's library system fully developed in these areas? In particular, is the API more or less stable for those types of activities, or would I find that my old code would tend to break after upgrading to a new version of Julia?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, are there any other issues that would be worth considering in deciding whether to use Julia for serious work at the present time?&lt;/p&gt;&#xA;" OwnerUserId="3746" LastEditorUserId="20688" LastEditDate="2019-05-19T03:59:45.067" LastActivityDate="2020-07-19T17:46:19.960" Title="How mature is the &quot;Julia&quot; scientific computing language project?" Tags="|parallel-computing|languages|julia|" AnswerCount="4" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10923" PostTypeId="2" ParentId="10922" CreationDate="2014-02-27T08:24:27.463" Score="28" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If not, is it possible to give a rough order-of-magnitude estimate for how long I should wait before considering it again?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My rough, order-of-magnitude estimate of how long it takes computational science languages to mature is around a decade.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example 1: SciPy started in 2001 or so. In 2009, Scipy 0.7.0 was released, and the ODE integrator had an interface to VODE (which is the equivalent of &lt;code&gt;ode15s&lt;/code&gt;, roughly; &lt;code&gt;ode15s&lt;/code&gt; is NDF-based, VODE is BDF/Adams-Bashforth, depending). With SciPy 0.10.0, an interface to &lt;code&gt;dopri5&lt;/code&gt;, which is roughly the equivalent of MATLAB's &lt;code&gt;ode45&lt;/code&gt;, a Runge-Kutta 4th order method typically introduced as the first practical numerical integration method to undergraduates. SciPy 0.10.0 was released in December 2011, and it took about 10 years for them to include a feature of MATLAB introduced to every engineering undergrad I know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example 2: Mathworks was founded in 1984. In their first release, they used a LAPACK port to C named JACKPAC (after Jack Little, a MathWorks engineer who wrote it). They did not replace it with LAPACK until 2000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Julia may take less time, but I'd estimate about 10 years from founding to become mainstream. (It's already been out a year or so; maybe 9-10 years, then?)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is Julia's library system fully developed in these areas? In particular, is the API more or less stable for those types of activities, or would I find that my old code would tend to break after upgrading to a new version of Julia?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't use Julia, so take what I say with a grain of salt, since I've only seen Jeff Bezanson give presentations about Julia. They've bent over backwards to make it easy to link and use libraries from C, Python, and Fortran. If you can't find a Julia library that does what you want, write a Julia shim for a library in a more established language. Consequently, I don't think a lack of libraries will be a concern. I think a concern will be making sure that changes to core language features don't bite you in the ass. If you look at the milestones in the Julia Git repo, you'll see that the &quot;breaking&quot; tag is used quite a bit (12 times in the 0.2 release, 5 times in the 0.3 release). To me, that suggests that the core language is still evolving, which is why I hesitate to use the language right now. Julia looks very cool, so I anticipate that I may use it in the future, maybe a few years from now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Aurelius brings up a good point:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What makes you assume Julia will ever actually become mainstream, and not just die off in obscurity like so many other languages? SciPy/numpy had/has the backing of an ever-growing python community, which Julia doesn't have.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the original answer, I decided to avoid the question of &quot;Will Julia succeed in becoming mainstream?&quot; as much as possible. Failing is easy; success is difficult. I think the best comparison of Julia is to technical computing languages like MATLAB, R, and Octave. HPC languages (Chapel, Fortress, UPC, etc.) have a narrower audience than technical computing languages, and general purpose languages (C, Python, C++, etc.) have a broader audience than  technical computing languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something I think that helps Julia is design for performance without sacrificing expressiveness. Julia is much more competitive with compiled languages like C than MATLAB, R, or even Python. This design goal is also a feature that can draw people from existing languages, like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;People who care a lot about performance and come from languages like C and Fortran, but are willing to sacrifice a &lt;em&gt;tiny&lt;/em&gt; bit of performance (maybe a factor of 2ish) to go from compiled language to fewer lines of interpreted language (along with a REPL for more rapid development and testing).&lt;/li&gt;&#xA;&lt;li&gt;People who care about high productivity and come from languages like Python, R, and MATLAB, but want more performance. When it comes to execution, pure Python, pure MATLAB, and pure R are slow. Developers in those languages have gone out of their way to wrap libraries in compiled languages, but you can't wrap everything, and at some point, the core language is going to slow you down. Core Julia is faster, which lets you do more science faster.&lt;/li&gt;&#xA;&lt;li&gt;People who care about free software. Julia is interpreted and free (so is Python, Octave, etc.); MATLAB is not.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Julia is also attempting to facilitate parallelism; I don't feel terribly qualified to expand upon that point, and I don't think it's the main draw of the language, but I think it's a selling point that they are working on, and I hope others can shed light on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, even with technical merit on their side, the language creators have to do the legwork to promote the language and evangelize. Jeff Bezanson, Alan Edelman, Stephen Karpinski, and Viral Shah are working very hard to make the language succeed. Alan Edelman has deep ties to the computational science community, and he's worked on language-level projects before (notably, the Star-P extension to MATLAB). Jeff Bezanson has been doing the conference circuit to promote Julia to computational scientists and engineers for a while. At MIT, they've been doing a good job of recruiting students and staff (notably, Steven G. Johnson) to contribute by adding libraries to Julia. They've got an article in Wired, and managed to get a Wikipedia article for themselves, all after only a year. Their Git repo has thousands of stars, hundreds of forks, and hundreds of watches, so by open-source standards, their project has been a success. I think they've done all the right things so far, so it's a matter of sustaining that effort and building community. They could still fail, but getting this far suggests to me that they have a reasonable chance at success.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-02-27T19:45:18.630" LastActivityDate="2014-02-27T19:45:18.630" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10925" PostTypeId="2" ParentId="10910" CreationDate="2014-02-27T11:01:23.280" Score="3" Body="&lt;p&gt;There is a nice volume rendering toolbox for MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/22940-vol3d-v2&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/22940-vol3d-v2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you could tweak it for your purposes.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-02-27T11:01:23.280" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10926" PostTypeId="2" ParentId="10918" CreationDate="2014-02-27T11:41:32.083" Score="2" Body="&lt;p&gt;You could try using &lt;a href=&quot;http://cvxr.com/cvx/&quot; rel=&quot;nofollow&quot;&gt;CVX&lt;/a&gt;, which would allow you to code it in exactly the form you've written it (i.e., with $X$ as a matrix rather than a vector).  It probably would be solved using a more general convex solver instead of an LP solver, but if the convex solver succeeds then I suppose you won't mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility: use a solver that allows your constraint matrices to be stored as sparse matrices.  This will still require a lot more memory than what you ought to need, but far less than if you stored them as dense matrices.  In CVX, if you use &lt;code&gt;kron&lt;/code&gt; you get a sparse matrix, so it would be trivial to try this.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-02-28T10:45:20.743" LastActivityDate="2014-02-28T10:45:20.743" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="10928" PostTypeId="1" AcceptedAnswerId="10934" CreationDate="2014-02-27T14:50:55.723" Score="3" ViewCount="701" Body="&lt;p&gt;I know that the finite volume method (based around a central different stencil) is unstable for advection dominated advection-diffusion problems. This leads to different adaptive schemes to can be applied to stabilise the basic approach (such as exponential fitting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't have broad understanding of stability issue with finite elements method, in particular I am interesting in comparing the continuous and discontinuous Galerkin method. From what I have read the continuous Galerkin method can have stability problems for advection dominated problems. But I haven't managed to find an exact criteria for stability, is there one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the discontinuous Galerkin method fundamentally more stable in these situations, or does it also have a stability criteria?&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2014-02-28T03:35:55.420" Title="In what regime do the continuous and discontinuous Galerkin method become unstable for advection-diffusion systems?" Tags="|finite-element|stability|advection-diffusion|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10929" PostTypeId="2" ParentId="10928" CreationDate="2014-02-27T15:03:50.630" Score="1" Body="&lt;p&gt;The stability criteria for continuous Galerkin FEM with piecewise linear basis functions in 1-D is that the cell Peclet number should be less than 2. This is the same as the central difference finite-difference formulation and probably the same as some finite volume formulations of similar order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYI, this half of the question was answered similarly in your &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8649/does-the-finite-element-method-impose-any-restrictions-on-the-peclet-number-for&quot;&gt;previous question on the topic&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll let a DG person speak to the criteria there.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-02-27T15:03:50.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10930" PostTypeId="1" CreationDate="2014-02-27T22:52:38.040" Score="1" ViewCount="207" Body="&lt;p&gt;I am currently trying to port my code over to Trilinos because the problems that I am working on are too big for LAPACK/ARPACK. Specifically I am computing the generalized eigenvalues/eigenvectors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A}x = \lambda\mathbf{B}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where both $\mathbf{A}$ and $\mathbf{B}$ are large and sparse. I am mainly interested in targeting specific interior eigenvalues using a shift-invert method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A}x - \sigma\mathbf{B}x = \lambda\mathbf{B}x - \sigma\mathbf{B}x$$&#xA;$$\mathbf{A}x - \sigma\mathbf{B}x = (\lambda - \sigma)\mathbf{B}x$$&#xA;$$\frac{1}{\lambda - \sigma}x = (\mathbf{A}x - \sigma\mathbf{B})^{-1}\mathbf{B}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This maps eigenvalues that are close to $\sigma$ towards $\infty$ so that they are easy to find by iterative methods. To avoid computing the inverse in the previous expression we instead solve the linear system at every iteration:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w = \mathbf{B}v_i$$&#xA;$$(\mathbf{A}x - \sigma\mathbf{B})v_{i+1} = w$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previously I was solving this system using LU decomposition but as the matrices become large, this method is unwieldy. Instead I am trying to use Belos to solve this linear system for the Anasazi eigensolver package. An important point to note is that my matrices are complex-valued and so Epetra is a less than ideal solution, I would rather get Tpetra working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been looking at &lt;a href=&quot;http://trilinos.sandia.gov/packages/docs/r11.4/packages/anasazi/doc/html/BlockKrylovSchur_2BlockKrylovSchurEpetraExGenBelos_8cpp-example.html&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; Epetra example which defines an &lt;code&gt;Anasazi::EpetraGenOp&lt;/code&gt; generalized operator to solve the linear system. This generalized operator basically computes $\mathbf{A}^{-1}\mathbf{B}v_i$ for a shift of $\sigma = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Anasazi::EpetraGenOp&lt;/code&gt; is defined in the Epetra adapter for Anasazi but the Tpetra adapter does not seem to have an equivalent generalized operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the correct way to solve a shift-inverted generalized eigenvalue problem using Tpetra/Belos/Anasazi?&lt;/p&gt;&#xA;" OwnerUserId="4524" LastActivityDate="2014-02-27T22:52:38.040" Title="Shift-Invert in Anasazi/Belos using Tpetra Sparse Matrices" Tags="|eigenvalues|trilinos|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10931" PostTypeId="2" ParentId="10918" CreationDate="2014-02-28T00:03:56.900" Score="5" Body="&lt;p&gt;You probably want to use a matrix-free method for linear programming. I don't know of any method specifically geared towards linear programming, but there exist matrix-free interior point methods for quadratic programs and for general nonlinear programs. The quadratic program case corresponds exactly to your problem, where the quadratic form coefficients are all zeroes. (You could also tailor methods that use exact linear solves to the structure of your problem, but that sort of bespoke implementation may not be worth it, and is less practical than using a matrix-free method.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any commercial optimization package that implements matrix-free variants of interior point methods. IPOPT is supposed to implement a matrix-free interior point method for nonlinear programming, but I haven't been able to track down the API calls that enable you to use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to CVX, you could probably use GAMS or AMPL to input the matrix once and set up your constraints in the modeling language to use that matrix. However, the methods used by the solver backends to CVX, GAMS, and AMPL do not use matrix-free solvers; all will require the full coefficient matrix for the linear program in standard form, which will be huge (it will be a Kronecker product of matrices). What will probably happen is that you input your linear program in the form above using the modeling language, and then the modeling language will translate the data into a form usable by backend solvers. This form will require huge matrices, and I suspect you will run into the same sorts of errors (unless you run on a machine with enough memory).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-02-28T00:11:26.823" LastActivityDate="2014-02-28T00:11:26.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10932" PostTypeId="2" ParentId="10922" CreationDate="2014-02-28T02:03:53.377" Score="24" Body="&lt;p&gt;I believe Julia is worth learning. I have used it to produce a few research finite element codes, and produce them very quickly. I have been over all very pleased with my experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Julia has enabled a workflow for me that I have found difficult to achieve with other languages. You may use it as a prototyping language like MATLAB, but unlike MATLAB when you have a working code and are going through profiling iterations to speed it up, replacing hotspots with C code is painless. They have made interfacing with C (and Python) a priority in design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the language has enabled me to very rapidly move from my mathematical formulations to functional code and then from functional code to highly performant code. I think this feature of Julia is undersold, but it adds tremendous value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many instances I have obtained C performance (compared to my own C code) in a matter of hours after producing a functional finite element code. So far if I use only Julia features I usually get to within ~3X slower than C. After this I replace hotspots with C functions (Julia comes with a stack sampling profiler to help identify these). Often this requires only replacing the offending hotspot lines of code with a &quot;ccall,&quot; with Julia handling any marshalling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the main things Julia is missing right now though that makes me hesitate to consider it for bigger projects is the lack of a fully supported debugger, and poor support for plotting (right now your best bet in plotting is actually just an interface into matplotlib that I've had break more often than not). Immediate feedback on code is really important. I also don't know how to survive without interactive debugging, and in this respect I'm very spoiled by MATLAB and GDB.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2014-02-28T02:15:34.443" LastActivityDate="2014-02-28T02:15:34.443" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10934" PostTypeId="2" ParentId="10928" CreationDate="2014-02-28T03:35:55.420" Score="5" Body="&lt;p&gt;In the advection dominated case the problem can develop physics which is invisible to a computational mesh that is too coarse (say by having elements which contain many wavelengths). This leads to instability because neighboring elements could disagree considerably on what they believe are the right values to represent this physics (think interpolating high frequency waves with low order polynomials, you can get wildly different interpolants depending on how you sample.). This is problematic to continuous Galerkin because despite this potential disagreement, it forces continuity between elements. This is why you have mesh refinement restrictions, basically you only see stability once you have correctly resolved the advection (or add a bit of magic to the formulation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;DG resolves this by not requiring continuity and instead adding a term to penalize jump discontinuities, this penalty term essentially adds artificial dissipation to the method if the solution develops large jumps. This typically yields unconditionally stable methods (unconditional on the mesh size,polynomial order, and advection speed). Whether or not this is a desirable property depends on your application though, I think. Although the formulations are stable, it does not mean they are accurate. One could be stuck in the preasymptotic part of the convergence theory until they refine the mesh (resp. increase polynomial order), and then the level of mesh refinement used for convergence to kick in often ends up being exactly the same amount of refinement used for stability in the continuous cousin. Also be very careful about trusting stability results that depend on integrals being evaluated exactly.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2014-02-28T03:35:55.420" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10935" PostTypeId="1" CreationDate="2014-02-28T09:01:06.997" Score="1" ViewCount="168" Body="&lt;p&gt;I dont understand how this Higher Order Compact ADI scheme, which is fourth order in time and space, for the wave equation is derived:&#xA;I go through the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Tylor's expansion $u(t+h,x,y)$ and $u(t-h,x,y)$ about the point (t,x,y) we have:\&#xA;(1)&#xA;\begin{equation}&#xA;\label{311}&#xA;u(t+k,x,y)-2u(t,x,y)+u(t-k,x,y)=k^2\left(\frac{\partial^2 u(t,x,y)}{\partial t^2}\right)+\frac{k^4}{12}\left(\frac{\partial^4 u(t,x,y)}{\partial t^4}\right)+\cdots&#xA;\end{equation} &#xA;If $u$ is a solution of $u_{tt}=u_{xx}+u_{yy}$ then we have the following:\&#xA;(2)&lt;br&gt;&#xA;\begin{equation}&#xA;\label{312}&#xA;\frac{\partial^2 }{\partial t^2}=\frac{\partial^2 }{\partial x^2}+\frac{\partial^2 }{\partial y^2}&#xA;\end{equation}&lt;br&gt;&#xA; And \&#xA; (3)&#xA;\begin{equation}&#xA;\label{313}&#xA;\frac{\partial^4 }{\partial t^4}=\frac{\partial^4 }{\partial x^4}+\frac{\partial^4 }{\partial y^4}&#xA;\end{equation} (3)&#xA;Substituting equations (2) and (3)  in to equation (1) and $u(t+k,x,y)=u^{n+1}_{i,j}$,$u(t-k,x,y)=u^{n-1}_{i,j}$ and $u(t,x,y)=u^{n}_{i,j}$ shows:\&#xA;(4)&#xA;\begin{equation}&#xA;\label{314}&#xA;u^{n+1}_{i,j}-2u^{n}_{i,j}+u^{n-1}_{i,j}=k^2\left(\frac{\partial^2u(t,x,y) }{\partial x^2}+\frac{\partial^2 u(t,x,y)}{\partial y^2}\right)+\frac{k^4}{12}\left(\frac{\partial^4 u(t,x,y)}{\partial x^4}+\frac{\partial^4 u(t,x,y)}{\partial y^4}\right)+\cdots&#xA;\end{equation} &#xA;Again using Taylor's expansion up to the fourth order difference for the following:\&#xA;(5)&#xA;\begin{equation}&#xA;\label{315}&#xA;\frac{\partial^2 u}{\partial x^2}=\frac{1}{h^2}\left(\delta^2_x-\frac{1}{12}\delta_x^4\right)u&#xA;\end{equation} &#xA;And \&#xA; (6)&#xA;\begin{equation}&#xA;\label{316}&#xA;\frac{\partial^2 u}{\partial y^2}=\frac{1}{h^2}\left(\delta^2_y-\frac{1}{12}\delta_y^4\right)u&#xA;\end{equation}&lt;br&gt;&#xA;And we also have :&#xA;\(7)&#xA;\begin{equation}&#xA;\label{317}&#xA;\frac{\partial^4 u}{\partial x^4}=\frac{\partial^2}{\partial x^2}\left( \frac{\partial^2 u}{\partial x^2}\right)=\frac{1}{h^4}\delta^4_xu&#xA;\end{equation} &#xA;And \(8)&#xA;\begin{equation}&#xA;\label{318}&#xA;\frac{\partial^4 u}{\partial y^4}=\frac{\partial^2}{\partial y^2}\left( \frac{\partial^2 u}{\partial y^2}\right)=\frac{1}{h^4}\delta^4_yu&#xA;\end{equation} &#xA;Substituting equations (5)-(8) in to equation (4) shows to the fourth order accuracy:\(9)&#xA;\begin{equation}&#xA;\label{319}&#xA;u^{n+1}_{i,j}-2u^{n}_{i,j}+u^{n-1}_{i,j}=\left(r^2\left( 1+\frac{r^2-1}{12}\delta_x^2\right)\delta^2_x +r^2\left( 1+\frac{r^2-1}{12}\delta_y^2\right)\delta^2_y\right) u_{i,j}^n&#xA;\end{equation}&#xA;The next step may be applying some operator,but I dont know how to apply and to reach the following ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(1+\frac{1-r^2}{12}\delta_x^2)u^*_{ij}=[2+\frac{1+5r^2}{6}\delta_x^2-\frac{2(1+r^2)}{1-r^2}\delta_y^2]u_{ij}^n-[1+\frac{1-r^2}{12}\delta_x^2]u_{ij}^{n-1}$$&#xA;$$(1+\frac{1-r^2}{12}\delta_y^2)u^{n+1}_{ij}=u^*_{ij}+\frac{1-r^2}{12}\delta_y^2[\frac{2(1+10r^2+r^4)}{(1-r^2)^2}u_{ij}^n-u_{ij}^{n-1}]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h_x=h_y=h$&lt;/p&gt;&#xA;" OwnerUserId="7482" LastEditorUserId="7482" LastEditDate="2014-04-14T17:00:22.170" LastActivityDate="2014-04-14T17:00:22.170" Title="Derivation of a Higher Order Compact Alternating Direction Implicit Method" Tags="|pde|finite-difference|operator-splitting|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10936" PostTypeId="1" AcceptedAnswerId="10937" CreationDate="2014-02-28T20:30:28.593" Score="1" ViewCount="1877" Body="&lt;p&gt;I have a fairly sparse adjacency matrix showing connections between approx 5,000 points in my dataset. I'm looking at various ways to analyze the relationships between the data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach looks interesting:&#xA;&lt;a href=&quot;https://www.cs.purdue.edu/homes/dgleich/demos/matlab/spectral/spectral.html&quot; rel=&quot;nofollow&quot;&gt;https://www.cs.purdue.edu/homes/dgleich/demos/matlab/spectral/spectral.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author sorts the adjacency matrix by the Fiedler vector and obtains what look like interesting results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for reference material about this approach. I've found plenty of references describing using the sign of the Fiedler vector to partition data into two sets (and this can be applied recursively). A direct sort using the Fiedler vector is something I haven't found in the literature. Can anyone tell me whether this is a known technique, and if so, point me towards some reference material about it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&#xA;Jeremy&lt;/p&gt;&#xA;" OwnerUserId="7491" LastActivityDate="2017-10-08T06:38:57.817" Title="sorting adjacency matrix by the Fiedler vector" Tags="|linear-algebra|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10937" PostTypeId="2" ParentId="10936" CreationDate="2014-02-28T21:18:20.367" Score="2" Body="&lt;p&gt;Not sure if this has a name, but if $v_1$ is the Fiedler vector (for the first nonzero eigenvalue), and $v_2$, $v_3$ are the eigenvectors for the next two smallest eigenvalues, you can plot them against each other to possibly reveal partitioning into multiple blocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I constructed a simple adjacency matrix with 4 groups that I later permute randomly. &#xA;&lt;img src=&quot;https://i.stack.imgur.com/FkyU7.png&quot; alt=&quot;adjacency&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we plot the Fiedler vector, we can see that we can't really tell more than 2 groups (it may look like 3, but that's just cause we've plotted the indices in order). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/t6u4w.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we plot instead now the Fiedler vector $v_1$ against $v_2$ and $v_3$, it reveals 4 clusters in 3D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/38ynV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, of course, still means you have to somehow detect clusters in the resulting data that you have, so it may not be too useful in practice. However, it's a way to force out a little more information from the eigenvectors of the graph Laplacian. What you &lt;em&gt;definitely&lt;/em&gt; don't want to do is to use the Fiedler vector $v_1$ in partitioning into more than two groups. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More often, folks will just use recursive spectral partitioning, along with enough hardcoded safeguards to force your resulting partitions to work well (i.e. being careful about interfaces, for example).  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2014-02-28T21:18:20.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10938" PostTypeId="1" CreationDate="2014-03-01T00:05:30.287" Score="2" ViewCount="512" Body="&lt;p&gt;For a serendipity element, the shape function of a middle pt on a edge can be formed using the tensor product in each direction, e.g. point 5 on Quad8&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\phi_5(\xi,\eta)=\frac{1}{2}(1-\xi^2)(1-\eta)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but the shape function on corner node (point 1) is&#xA;$$&#xA;\phi_1(\xi,\eta)=\frac{1}{4}(1-\xi)(1-\eta)-\frac{1}{4}(1-\xi^2)(1-\eta)-\frac{1}{4}(1-\eta^2)(1-\xi)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why the shape function of the corner point from a serendipity element (Q8)  can not be built by the tensor products like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\phi_1(\xi,\eta)=\frac{1}{4}\xi\eta(1-\xi)(1-\eta)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean I just pick up the wrong basis?&lt;/p&gt;&#xA;" OwnerUserId="1965" LastEditorUserId="9667" LastEditDate="2016-11-17T01:02:55.993" LastActivityDate="2016-11-18T12:38:00.123" Title="Shape functions of serendipity element as tensor products" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10940" PostTypeId="1" CreationDate="2014-03-01T12:12:38.050" Score="8" ViewCount="4758" Body="&lt;p&gt;I have a generalized eigenvalue problem in the standard form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lambda \mathbf{B} \mathbf{x} = \mathbf{A} \mathbf{x} $,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;resulting from a finite difference discretization of a coupled system of two linear stability equations, so the system is large $(10^5 $ x $10^5)$ and sparse. I know that $\mathbf{B}$ is indefinite, and not symmetric. Since I am interested in the stability of the system, I need to find the first, say ten, eigenvalues with the largest real part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it the preferred way to tackle such kind of problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve it using scipy linear algebra package for sparse matrices, but it appears that the eigs function only works when $\mathbf{B}$ is positive definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is appreciated.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Edit&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I post a minimal example code that generate my issue. The matrices that I use in the example &lt;strong&gt;are not&lt;/strong&gt;  the same I have in my application, but they illustrate the problem.&#xA;A working Matlab snippet, producing the correct answer is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = diag([-5, -4, -3, -2, -1]);&#xA;B = diag([1, 1, -1, 1, 1]);&#xA;eigs(A, B, 2, 'LR')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which produces the correct answer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; msolution&#xA;ans =&#xA;    3.0000&#xA;   -1.0000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;An equivalent python version to this problem  is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.sparse.linalg import eigs&#xA;&#xA;A = np.diag([-5, -4, -3, -2, -1]).astype(np.float64)&#xA;B = np.diag([1, 1, -1, 1, 1]).astype(np.float64)&#xA;vals, vecs = eigs(A, 2, B, which='LR')&#xA;&#xA;print vals&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;producing the clearly wrong result&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ 83.66243085+163.44457559j  83.66243085-163.44457559j]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, this might look like a bug in Scipy, but the docs explicitly ask for a positive definite matrix $\mathbf{B}$ and the one I have here is not, as the one I have in my application.&lt;/p&gt;&#xA;" OwnerUserId="6836" LastEditorUserId="6836" LastEditDate="2014-03-04T14:37:23.673" LastActivityDate="2016-10-20T10:45:51.743" Title="Solving a generalised eigenvalue problem" Tags="|eigenvalues|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="10941" PostTypeId="2" ParentId="10918" CreationDate="2014-03-02T04:12:12.963" Score="4" Body="&lt;p&gt;Can you afford those SVDs Geoffrey Irving mentioned? If you can, I would consider an &lt;a href=&quot;http://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares&quot; rel=&quot;nofollow&quot;&gt;iteratively reweighted least squares (IRLS)&lt;/a&gt; approach. This approach would solve problems of the form $$\begin{array}{ll}\text{minimize}&amp;amp;\sum_{ij} W_{ij}J_{ij}^2\\\text{subject to}&amp;amp;MJ+BY=X\end{array}$$where $W$ is a weight matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The iterations begin with $W^{(0)}$ as the all ones matrix; this yields an optimum $J^{(0)}$. The iterations proceed with $$W_{ij}^{(k+1)}=\left|\max\{J_{ij}^{(k)},\epsilon\}\right|^{-1}$$ where $\epsilon$ is a small constant that prevents division by zero. I'm not totally sure about the convergence criteria, but perhaps the Wikipedia link I offered above can give you references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also consider a smoothed first-order method. TFOCS, which I co-authored, could handle this using its &quot;smoothed conic dual&quot; (SCD) solver, but it's not going to be as easy to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do want to try a matrix-free interior point method, read the work of Jacek Gondzio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: hmm, it might be the case that IRLS won't be able to use the SVD to compute solutions. If so I would fall back to one of the other choices.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2014-03-02T17:01:59.640" LastActivityDate="2014-03-02T17:01:59.640" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10942" PostTypeId="1" AcceptedAnswerId="11004" CreationDate="2014-03-01T18:44:27.993" Score="2" ViewCount="153" Body="&lt;p&gt;If while distributing particles to their corresponding nodes, two particles comes closer to a level smaller than the machine precision (or $\Delta d \to 0$), how is the situation to be treated? Should the recursive distribution stop and both of them be added to a single node?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This happens because of treating particles as point objects rather than 2D objects. Imagine a machine having a precision level of 3 decimal points (mm in SI) and a situation in which the bodies come within $10^{-6}$ (micro level) of one another.&lt;/p&gt;&#xA;" OwnerUserId="7590" OwnerDisplayName="user3058846" LastEditorUserId="4489" LastEditDate="2014-05-27T06:47:58.463" LastActivityDate="2014-05-27T06:47:58.463" Title="Barnes-Hut algorithm and recursion limit" Tags="|simulation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10943" PostTypeId="1" AcceptedAnswerId="10945" CreationDate="2014-03-02T20:23:21.233" Score="10" ViewCount="613" Body="&lt;p&gt;A lot of adaptive FEM libraries use more advanced mesh data structures to handle adding/removing nodes, edges, triangles, tetrahedra, etc. For example, the &lt;a href=&quot;http://www.p4est.org/&quot;&gt;p4est&lt;/a&gt; library uses octree data structures for adaptive mesh refinement; you wouldn't often find octrees used for computations on a static mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What changes on the linear algebra side for adaptive FEM?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most blunt way I can conceive of would be to completely rebuild all the system matrices whenever the mesh is refined or coarsened. If mesh adaptation is a sufficiently infrequent operation, then the expense of doing so is ultimately amortized over the rest of the computation. One could easily leverage existing sparse linear algebra software (PETSc, Trilinos, etc.) with this approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this blunt method the most commonly used, or are there libraries that manage to reuse or modify the old matrix during refinement? After all, most of the mesh and the corresponding matrices are unchanged during a mesh adaptation.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-03-05T00:13:13.497" Title="Which novel data structures are used in adaptive FEM?" Tags="|linear-algebra|finite-element|sparse-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10944" PostTypeId="1" CreationDate="2014-03-02T22:58:04.887" Score="7" ViewCount="1962" Body="&lt;p&gt;I have a Simplex problem $ A y \ge b $, where some of the elements of $b$ are positive and some are negative, and thus setting $y = 0$ does not give a basic feasible solution (BFS). By previous work, however, I happen to know a specific BFS. How do I then set up the table to start at &lt;em&gt;this BFS&lt;/em&gt; instead of the standard $y=0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note: I don't want to use phase one of the simplex method because I want to use a specific BFS that I already know, and phase one finds me a BFS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that I have to do some row operations, but I can't work out what ones I would need to do. (Also, my specific case is reasonably large so I don't want to have to do it by hand but want some method that I can implement!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Sorry for the poor tagging - I'm not allowed to use better tags until I have a higher reputation! Also, I realise that there is a similar question posted elsewhere, but that focuses on a different point: I am interested in how to set up the table initially, which is not the focus of the other post.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance! :)&lt;/p&gt;&#xA;" OwnerUserId="7504" LastEditorUserId="276" LastEditDate="2014-03-03T22:55:48.903" LastActivityDate="2021-01-29T14:38:11.880" Title="Starting at a Given Basic Feasible Solution in the Simplex Method" Tags="|optimization|linear-programming|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10945" PostTypeId="2" ParentId="10943" CreationDate="2014-03-02T23:09:16.583" Score="8" Body="&lt;p&gt;Yes, the most common approach is to rebuild.  Data structures that are modifiable in-place tend to be less efficient once set up, and reallocation is actually quite cheap compared to reassembly (e.g., due to nonlinearity) so it's really a fine solution.  Outside of relatively rare niches with very easy solves, attempts to use dynamic data structures in the solvers will only make your application slower.  It is a common perception among those who forget to measure or model performance, however.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-03-02T23:09:16.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10946" PostTypeId="1" AcceptedAnswerId="10977" CreationDate="2014-03-03T00:10:19.893" Score="13" ViewCount="1456" Body="&lt;p&gt;Say I run a supercomputer computation on 100k cores for 4 hours on &lt;a href=&quot;http://www.nersc.gov/users/computational-systems/edison/configuration&quot;&gt;http://www.nersc.gov/users/computational-systems/edison/configuration&lt;/a&gt;, exchanging about 4 PB of data over the network and performing about 4 TB of I/O.  The calculation is all integer, so the results are either right or wrong (no intermediate numerical errors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the code is correct, I would like to estimate the probability that the computation is wrong due to a hardware failure.  What is a good way to go about this?  Are there good sources for the numbers required to make such an estimate?&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2014-03-03T01:20:46.693" LastActivityDate="2014-03-05T00:05:39.707" Title="Estimating hardware error probability" Tags="|error-estimation|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10947" PostTypeId="2" ParentId="10944" CreationDate="2014-03-03T00:23:13.057" Score="6" Body="&lt;p&gt;If your problem is in standard form (that is, with constraints $Ax = b$, $x \geq 0$), and you know a BFS, then you should know which columns of the standard form $A$ matrix to select to form a basis, from which you can set up your initial tableau by calculating reduced costs, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you have to implement the simplex algorithm yourself, I would use a library. In modeling languages, you can supply an initial guess; presumably, if this were a basic feasible solution for a linear program, it would be immediately usable. If it were not, there should be procedures for using that information to find one (e.g., Phase I simplex, crossover procedures for converting interior-point method iterate to a BFS, or just using an interior-point algorithm instead of simplex).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have to implement simplex yourself, convert the problem to standard form. The algorithms in Bertsimas and Tsitsiklis' book are easy to follow (but probably not efficient, since it's a textbook on linear programming theory); a library would be faster, and probably would save you time.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-03-03T00:23:13.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10948" PostTypeId="2" ParentId="10946" CreationDate="2014-03-03T03:28:44.307" Score="9" Body="&lt;p&gt;I guess, you start by collecting error rates of components, such as DRAM, like this Google research on &lt;a href=&quot;http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf&quot;&gt;DRAM Errors in the Wild: A Large-Scale Field Study&lt;/a&gt; They found ~1% chance to get one uncorrectable error per year. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if that's what you're interested. I'd be more interested in undetectable errors. Errors such that typical error checking methods would not detect. For instance, when you send packets over the optics, they're accompanied by some sort of CRC, which allows for a small chance of an error slipping through.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: this paper &lt;a href=&quot;http://rsim.cs.illinois.edu/Pubs/11DATE.pdf&quot;&gt;Architectures for Online Error Detection and Recovery in Multicore Processors&lt;/a&gt; talks about reliable multicore architecture, but they also cover different aspects of reliability of the system and has bibliography&lt;/p&gt;&#xA;" OwnerUserId="7470" LastEditorUserId="7470" LastEditDate="2014-03-03T13:08:19.607" LastActivityDate="2014-03-03T13:08:19.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10949" PostTypeId="2" ParentId="8147" CreationDate="2014-03-03T05:13:02.680" Score="2" Body="&lt;p&gt;If you want good parallel performance in shared memory, you can try out the Compressed Sparse Blocks (CSB) code:&#xA;&lt;a href=&quot;http://gauss.cs.ucsb.edu/~aydin/csb/csb.html&quot; rel=&quot;nofollow&quot;&gt;http://gauss.cs.ucsb.edu/~aydin/csb/csb.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is specifically designed to give equally good parallel performance for SpMV and SpMV_T (the transposed matrix case) without explicitly transposing the matrix. It also has provable parallelism guarantees regardless of the nonzero distribution (it can be as skewed as it gets)&lt;/p&gt;&#xA;" OwnerUserId="6782" LastActivityDate="2014-03-03T05:13:02.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10950" PostTypeId="2" ParentId="10946" CreationDate="2014-03-03T06:08:20.497" Score="2" Body="&lt;p&gt;Sounds epic.  If no one has done this experiment, you might consider running 100k separate cores doing something like rehashing a sha1 input over and over, seeing what the error rate is.  (Unmeasureable I suspect), from there do the same, but have them trade hash-chain results every so often to get your network error rates.  This I imagine is also very small, but I suspect you can get at least a couple using your supercluster over a few hours :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach insures that &lt;em&gt;every&lt;/em&gt; calculation is correct, as hashing is extremely sensitive to single-bit swaps, whereas even an integer only calculation might hide errors in branches, ie the entire calculation would not be elliptic on each consecutive memory state.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been working on a way to ensure that code has been run correctly by a external cluster who's motivation is to cheat by submitting faked results.  The solution I converged on is integrating the hash into the computation with some frequency that makes cheating less efficient than doing the work.   &lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2014-03-03T06:08:20.497" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10951" PostTypeId="1" CreationDate="2014-03-03T11:27:30.827" Score="3" ViewCount="42" Body="&lt;p&gt;I need to solve two point bondary problem for ODE. The solution of the problem itself is very easy. The real issue is that when solving arising non linear system of equations it always converges to one solution. However there exist two (physically justified) solutions. To illustrate this problem in detail consider the following case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discrete form of ODE is:&#xA;$$&#xA;y_{i+1}=y_i+\frac{\Delta x}{2} (a\cdot y_i^2+a\cdot y_{i+1}^2)&#xA;$$&#xA;I want to find $a$ and $y_i$ values. As one can see in each node $x_i$ there are two possible solutions of this equation. Sometimes I want to pick the ones that are placed on the left of the minimum of the parabol, sometimes those on the right. It depends on the value of some additional function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_{i+1},y_{i+1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that is computed in every step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I formulate a problem? Should I look into optimization techniques?&lt;/p&gt;&#xA;" OwnerUserId="4334" LastActivityDate="2014-03-03T11:27:30.827" Title="Solving ODE boundary problem with additional conditions" Tags="|ode|convergence|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10952" PostTypeId="2" ParentId="8297" CreationDate="2014-03-03T14:24:03.277" Score="1" Body="&lt;p&gt;You can simulate a range of systems varying in size, from smaller to larger, and show that a given property of interest mades a plato as system size increases (note that You'll have to equilibrate longer, as the size of the system grows).&lt;/p&gt;&#xA;" OwnerUserId="6750" LastActivityDate="2014-03-03T14:24:03.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10953" PostTypeId="2" ParentId="10944" CreationDate="2014-03-03T15:28:51.250" Score="4" Body="&lt;p&gt;Can you simply modify your problem from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ay\geq b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax \geq b−Ay_0,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the new unknown is $x=(y−y_0)$ and $y_0$ is your known specific BFS? Then you can use $x=0$ to start, and recover the solution of the initial problem by means of $y = x+y_0$.&lt;/p&gt;&#xA;" OwnerUserId="7277" LastActivityDate="2014-03-03T15:28:51.250" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10954" PostTypeId="2" ParentId="10946" CreationDate="2014-03-03T15:33:03.470" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there good sources for the numbers required to make such an&#xA;  estimate?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You might try asking the admins of the cluster you're computing on.  I imagine as part of their validation process they have confronted the problem of estimating the likelihood of hardware errors. &lt;/p&gt;&#xA;" OwnerUserId="4879" LastActivityDate="2014-03-03T15:33:03.470" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10956" PostTypeId="1" CreationDate="2014-03-03T16:19:53.223" Score="4" ViewCount="105" Body="&lt;p&gt;I'm dealing with physical simulation (position based dynamic). Now I'm trying to realize tracking for two meshes. In order to explain what does it mean, let's assume that we have two similar meshes: one with fine resolution and other with coarse. A need to make in the way that mesh with fine resolution followed the coarse mesh moving. But it should not be exact following, instead it should follow only in general. Let's consider this task in terms of mathematics:&#xA; First we need to build Laplace-Beltrami discrete operator. I use next definition&#xA;$$L=V^{-1/2}CV^{-1/2},$$&#xA;where $V$ is a diagonal matrix of Voronoi areas, $C$ is a sparse symmetric matrix of cotangent weights.&#xA;After that I perform an eigenvalue decomposition&#xA;$$L=QAQ^{T}$$&#xA;The next step is to build the matrix of harmonic functions $T$;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Internet I have found that it could be built in the next way&#xA;$$T=(V^{-1/2}Q)^{T}$$&#xA;So now I have $n*n$ matrix T, where n - is a number of vertices's in the fine mesh.&#xA;Now, if have T matrix I can build the next constraint $$T*p_f==TBp_c$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p_f$ is a vector of vertices's in the fine mesh &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p_c$ is a vector of vertices in the coarse mesh&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B$ such matrix that $Bp_c=p_f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this constraint is satisfied fine mesh exactly follows for the coarse mesh.&#xA;Now, if we discard from matrix $T$ those eigenvectors, which correspond to the biggest eigenvalues we obtain the $k*n$ matrix $T$, and it means that we discarded information about high frequent harmonics in the mesh spectral representation. As result mesh with fine resolution will follow for the coarse only in general but such small details like wrinkle will be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my implementation it works only if I use full $T$ matrix, but when I discard at least one eigenvector, behavior becomes unpredictable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question:&#xA;Is there any fundamental errors in this approach? I'm confident in all, except of definition for $T$ matrix.  &lt;/p&gt;&#xA;" OwnerUserId="5278" LastEditorUserId="5278" LastEditDate="2014-03-03T17:03:34.007" LastActivityDate="2014-03-03T17:03:34.007" Title="Tracking for two meshes" Tags="|linear-algebra|matrix|simulation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10957" PostTypeId="1" AcceptedAnswerId="10976" CreationDate="2014-03-03T23:34:31.750" Score="7" ViewCount="762" Body="&lt;p&gt;I have a one dimensional convex function&#xA;$$f : [a,b] \to \mathbb{R}$$&#xA;and want to find the minimum value&#xA;$$\min_{a \le x \le b} f(x)$$&#xA;I know all derivatives of $f$, so the problem could easily be solved with any 1D minimization method even ignoring the convexity.  However, I would like to not ignore the convexity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: How can I best take advantage of convexity to solve my 1D minimization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the values $f(a),f(b),f'(a),f'(b)$ define a triangular lower bound on the values of $f(x)$ on $[a,b]$, and the lowest vertex of this triangle is likely a good next guess.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2014-03-05T02:57:13.863" Title="Minimizing 1D convex functions" Tags="|convex-optimization|" AnswerCount="2" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="10958" PostTypeId="1" CreationDate="2014-03-04T01:26:11.100" Score="1" ViewCount="162" Body="&lt;p&gt;I am trying to maximize the following function (variable: $\theta$, a vector ($\theta_1$,...,$\theta_K$). most likely $K\leq5$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;F(\theta, c_0, c_1) = &amp;amp;P_{\theta}(T_0 &amp;gt; c_0, \max T_i &amp;gt;c_1, \min T_i &amp;lt;-c_1)  \\&#xA;&amp;amp;+ \sum_{S \subset \{1,2,.,K\},\\ 1\leq |S|\leq K-1} P_{\theta}(T_0 &amp;gt; c_0, \min_{i\in S} |T_i|  &amp;gt;c_1, \max_{i \in \bar{S}} |T_i| \leq c_1) \\&#xA;&amp;amp;\mspace{100mu}\mathcal{I}(\max_{i\in S} |\theta_i| \leq  \min_{i\in \bar{S}}  |\theta_i|)&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $T_i$ is normally distributed with mean $\theta_i$ and variance 1. &#xA;$T_0 = \sum_{1&amp;lt;= i&amp;lt;j &amp;lt;=K} (T_i-T_j)^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically this function is &#xA;First, a sum of Gaussian integrals which I don't believe has any closed form. &#xA;Second, discontinuous due to the multiplication of indicator $\mathcal{I}(\max_{i\in S} |\theta_i| \leq  \min_{i\in \bar{S}}  |\theta_i|)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach now: use quasi-Monte Carlo to numerically calculate the integral. Use simulated annealing to optimize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seems quite clumsy. Is there a better way to do this?&lt;/p&gt;&#xA;" OwnerUserId="7516" LastEditorUserId="20688" LastEditDate="2018-03-28T12:36:12.983" LastActivityDate="2018-03-28T12:36:12.983" Title="optimizing a discontinous function" Tags="|optimization|monte-carlo|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10960" PostTypeId="2" ParentId="4928" CreationDate="2014-03-04T04:03:13.313" Score="0" Body="&lt;p&gt;I'd offer the &lt;a href=&quot;http://en.wikipedia.org/wiki/Frank%E2%80%93Wolfe_algorithm&quot; rel=&quot;nofollow&quot;&gt;Frank Wolfe algorithm&lt;/a&gt; and related methods for your consideration. Basically, you linearize the objective function and solve the resulting LP at each iteration. I do think, however, that you would need to add bounds on $x$ to make this approach effective.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2014-03-04T04:03:13.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10962" PostTypeId="2" ParentId="541" CreationDate="2014-03-04T04:34:56.947" Score="1" Body="&lt;p&gt;Here is a partial list of PDE-constrained optimization packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dolfin Adjoint is part of FEniCS FEM:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dolfin-adjoint.org/&quot; rel=&quot;nofollow&quot;&gt;http://dolfin-adjoint.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ROL, MOOCHO, Sundance are parts of Trilinos:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/trilinos/trilinos/tree/master/packages/rol/&quot; rel=&quot;nofollow&quot;&gt;https://github.com/trilinos/trilinos/tree/master/packages/rol/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/trilinos/trilinos/tree/master/packages/Sundance/&quot; rel=&quot;nofollow&quot;&gt;https://github.com/trilinos/trilinos/tree/master/packages/Sundance/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://trilinos.org/packages/moocho/&quot; rel=&quot;nofollow&quot;&gt;http://trilinos.org/packages/moocho/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PYOMO example for PDE-constrained optimization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://software.sandia.gov/trac/pyomo/browser/pyomo/trunk/examples/dae&quot; rel=&quot;nofollow&quot;&gt;https://software.sandia.gov/trac/pyomo/browser/pyomo/trunk/examples/dae&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TAO manual gives examples of solving PDE-constrained optimization problems:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.5/docs/tao_manual.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-3.5/docs/tao_manual.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5401" LastEditorUserId="5401" LastEditDate="2015-03-16T03:59:14.127" LastActivityDate="2015-03-16T03:59:14.127" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10963" PostTypeId="2" ParentId="10918" CreationDate="2014-03-04T06:14:56.917" Score="12" Body="&lt;p&gt;&lt;strong&gt;Overview&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to try a variant of the &lt;em&gt;Alternating Directions Method of Multipliers&lt;/em&gt; (ADMM), which has been found to converge surprisingly quickly for $l_1$ lasso type problems. The strategy is to formulate the problem with an augmented Lagrangian and then do gradient ascent on the dual problem. It is especially nice for this particular $l^1$ regularized problem because the nonsmooth part of each iteration of the method has an exact solution you can simply evaluate element by element, while the smooth part involves solving a linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this post we&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;derive an overall ADMM formulation for a generalization of your problem,&lt;/li&gt;&#xA;&lt;li&gt;derive the subproblems for each ADMM iteration and specialize them to your situation, and then&lt;/li&gt;&#xA;&lt;li&gt;investigate the resulting linear system that needs to be solved each iteration, and develop a fast solver (or preconditioner) based on precomputing the eigenvalue decompositions (or low rank approximations thereof) for $M^TM$ and $YY^T$.&lt;/li&gt;&#xA;&lt;li&gt;summarize with a few concluding remarks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Most of the big ideas here are covered in the following superb review paper,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Boyd, Stephen, et al. &quot;Distributed optimization and statistical learning via the alternating direction method of multipliers.&quot; Foundations and Trends® in Machine Learning 3.1 (2011): 1-122. &lt;a href=&quot;http://www.stanford.edu/~boyd/papers/pdf/admm_distr_stats.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.stanford.edu/~boyd/papers/pdf/admm_distr_stats.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Before going into details, I want to note that this is a method/algorithm answer not a practical existing code answer - if you want to use this method, you would need to roll your own implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ADMM formulation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, suppose you want to solve&#xA;$$&#xA;\begin{array}{rl}&#xA;\min_{x} &amp;amp; \sum_{i} |x_i|\\&#xA;\textrm{s.t.} &amp;amp; Ax = b&#xA;\end{array}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem in the original post falls in this category after appropriate vectorization. (this is only in principle - we will see that the vectorization doesn't need to be performed in practice) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could instead solve the equivalent problem, &#xA;$$&#xA;\begin{array}{rl}&#xA;\min_{x,z} &amp;amp; \sum_{i} |x_i| + \frac{\alpha}{2}||x-z||^2 + \frac{\beta}{2}||Az-b||^2 \\&#xA;\textrm{s.t.} &amp;amp; Az = b \\&#xA;\textrm{&amp;amp;} &amp;amp; x = z,&#xA;\end{array}&#xA;$$&#xA;which has Lagrangian&#xA;$$\begin{align}&#xA;L(x,z,\lambda,\gamma) =&amp;amp; \sum_{i} |x_i| + \frac{\alpha}{2}||x-z||^2 + \frac{\beta}{2}||Az-b||^2 + \lambda^T(Az-b) + \gamma^T(x-z) \\&#xA;=&amp;amp; \sum_{i} |x_i| + \frac{\alpha}{2}||x-z + \frac{1}{\alpha}\gamma||^2 + \frac{\beta}{2}||Az-b + \frac{1}{\beta}\lambda||^2 \\&#xA;&amp;amp;+ \frac{\alpha}{2}||\frac{1}{\alpha}\gamma||^2 + \frac{\beta}{2}||\frac{1}{\beta}\lambda||^2.&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternating direction method of multipliers solves the dual problem,&#xA;$$\max_{\lambda,\gamma} \min_{x,z} L(x,z,\lambda,\gamma),$$&#xA;via gradient ascent on the dual variables, except with inexact alternating projections on the dual subproblems. Ie., one does the iteration&#xA;$$&#xA;\begin{align}&#xA;x^{k+1} &amp;amp;= \mathrm{argmin}_x L(x,z^k,\lambda^k,\gamma^k) \\&#xA;z^{k+1} &amp;amp;= \mathrm{argmin}_z L(x^{k+1},z,\lambda^k,\gamma^k) \\&#xA;\gamma^{k+1} &amp;amp;= \gamma^k + \alpha(x^{k+1}-z^{k+1}) \\&#xA;\lambda^{k+1} &amp;amp;= \lambda^k + \beta(Az^{k+1}-b).&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under certain mild conditions on the parameters $\alpha$ and $\beta$ (explained in the Boyd &amp;amp; Parikh paper linked above), the ADMM method will converge to the true solution. The convergence rate is linear, as it is at the core a gradient ascent method. Often it can be accelerated to be superlinear by 1) changing the parameters $\alpha$ and $\beta$ as you go based on heuristics, or 2) using Nesterov acceleration. For notes on changing the penalty parameters, see the Boyd survey paper, and for using Nesterov acceleration with ADMM see the following paper,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Goldstein, Tom, Brendan O’Donoghue, and Simon Setzer. &quot;Fast alternating direction optimization methods.&quot; CAM report (2012): 12-35. &lt;a href=&quot;ftp://ftp.math.ucla.edu/pub/camreport/cam12-35.pdf&quot; rel=&quot;noreferrer&quot;&gt;ftp://ftp.math.ucla.edu/pub/camreport/cam12-35.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, even if the overall convergence rate is only linear, for $l^1$ problems the method has been observed to find the sparsity pattern very quickly, and then converge more slowly on the exact values. Since finding the sparsity pattern is the hardest part, this is very fortuitous! The exact reasons why seem to be an area of current research. Everyone sees the sparsity pattern converge fast, but no one seems to know exactly why it happens. A while ago I asked Boyd and Parikh about this over email and Parikh thought it might be explained by interpreting the method in a control systems context. Another heuristic explanation of the phenomenon is found in the appendix of the following paper,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Goldstein, Tom, and Stanley Osher. &quot;The split Bregman method for L1-regularized problems.&quot; SIAM Journal on Imaging Sciences 2.2 (2009): 323-343. &lt;a href=&quot;ftp://ftp.math.ucla.edu/pub/camreport/cam08-29.pdf&quot; rel=&quot;noreferrer&quot;&gt;ftp://ftp.math.ucla.edu/pub/camreport/cam08-29.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Of course now the difficulty lies in solving the $x$ and $z$ update subproblems for your specific situation. Since the Lagrangian is quadratic in $z$, the $z$ update subproblem simply requires solving a linear system. The $x$ subproblem seems harder since it is nondifferentiable, but it turns out that there is an exact formula for the solution that can be applied element by element! We now discuss these subproblems in more detail and specify them to the problem in the original post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Setup for the $z$ update subproblem (linear system)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the $z$ update, we have&#xA;$$\mathrm{argmin}_z L(x_k,z,\lambda_k,\gamma_k) = \mathrm{argmin}_z \frac{\alpha}{2}||x-z + \frac{1}{\alpha}\gamma||^2 + \frac{\beta}{2}||Az-b + \frac{1}{\beta}\lambda||^2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specialized to your problem this becomes,&#xA;$$\begin{align}&#xA;\mathrm{argmin}_{Z_J,Z_B} &amp;amp;\frac{\alpha}{2}||J^{k+1}-Z_J + \frac{1}{\alpha}\Gamma_J||_{Fro}^2 + \frac{\alpha}{2}||B^{k+1}-Z_B + \frac{1}{\alpha}\Gamma_B||_{Fro}^2 \\&#xA;&amp;amp;+\frac{\beta}{2}||MZ_J + Z_BY - X + \frac{1}{\alpha}\Lambda||^2_{Fro},&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $||\cdot||Fro$ denotes the Frobenius (elementwise $l_2$) norm. This is a quadratic minimization problem, where the first order optimality conditions can be found by taking partial derivatives of the objective with respect to $Z_J$ and $Z_B$ and setting them to zero. This is,&#xA;$$\begin{align}&#xA;0 &amp;amp;= -\frac{\alpha}{2}(J^{k+1} - Z_J + \frac{1}{\alpha}\Gamma_J) + \frac{\beta}{2}M^T(MZ_J + Z_BY - X + \frac{1}{\beta}\Lambda), \\&#xA;0 &amp;amp;= -\frac{\alpha}{2}(B^{k+1} - Z_B + \frac{1}{\alpha}\Gamma_B) + \frac{\beta}{2}(MZ_J + Z_BY - X + \frac{1}{\beta}\Lambda)Y^T.&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As noted in the comments by the original poster Justin Solomon, this system for $Z_J,Z_B$ is symmetric so conjugate gradient is an ideal matrix-free method. A later section discusses this system and how to solve/precondition it in greater detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solving $x$ update subproblem (analytic thresholding solution)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we turn to the $x$ subproblem,&#xA;$$\mathrm{argmin}_x L(x,z^k,\lambda^k,\gamma^k) = \mathrm{argmin}_x \sum_{i} |x_i| + \frac{\alpha}{2}||x-z^k + \frac{1}{\alpha}\gamma^k||^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing to see is that the sum can be broken up element by element,&#xA;$$\sum_{i} |x_i| + \frac{\alpha}{2}||x-z^k + \frac{1}{\alpha}\gamma^k||^2 = \sum_{i} |x_i| + \frac{\alpha}{2}\sum_i (x_i-z_i^k + \frac{1}{\alpha}\gamma_i^k)^2,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can solve the optimization problem element by element in parallel, yielding&#xA;$$x_i^{k+1} = \mathrm{argmin}_{x_i} |x_i| + \frac{\alpha}{2}(x_i-z_i^k + \frac{1}{\alpha}\gamma_i^k)^2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general form of this equation is,&#xA;$$\min_s |s| + \frac{\alpha}{2}(s-t)^2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The absolute value function is trying to pull the optimal point towards $s=0$, whereas the quadratic term is trying to pull the optimal point towards $s=t$. the true solution therefore lies somewhere on the segment $[0,t)$ between the two, with increasing $\alpha$ tending to pull the optimal point towards $t$, and decreasing $\alpha$ pulling the optimal point towards $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a convex function but it is not differentiable at zero. The condition for a minimizing point is that the subderivative of the objective at that point contains zero. The quadratic term has derivative $\alpha(s-t)$, and the absolute value function has derivative $-1$ for $s &amp;lt; 0$, set-valued subderivative as the interval $[-1,1]$ when $s=0$, and derivative $1$ for $s &amp;gt; 0$. Thus we get the subderivative for the overall objective function,&#xA;$$\partial_s \left(|s| + \frac{\alpha}{2}(s-t)^2\right) = \begin{cases}&#xA;1 + \alpha (s-t)\, &amp;amp; s &amp;gt; 0 \\&#xA;[-1,1] + \alpha t, &amp;amp; s = 0, \\&#xA;-1 + \alpha (s-t), &amp;amp; s &amp;lt; 0.&#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this we see that the subderivative of the objective at $s=0$ contains $0$ if and only if $|t| \le \frac{1}{\alpha}$, in which case $s=0$ is the minimizer. On the other hand, if $s=0$ is not the minimizer, then we can set the single-valued derivative equal to zero and solve for the minimizer. Doing this yields,&#xA;$$\mathrm{argmin}_s |s| + \frac{\alpha}{2}(s-t)^2 = \begin{cases}&#xA;t - \frac{1}{\alpha}, &amp;amp; t &amp;gt; \frac{1}{\alpha}, \\&#xA;0, &amp;amp; |t| \le \frac{1}{\alpha}, \\&#xA;t + \frac{1}{\alpha}, &amp;amp; t &amp;lt; -\frac{1}{\alpha}&#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specializing this result again to the real problem we are trying to solve in the original question where $t = Z_{ij}^k - \frac{1}{\alpha}\Gamma_{ij}^k$ yields,&#xA;$$J_{ij}^{k+1} = \begin{cases}&#xA;Z_{ij}^k - \frac{1}{\alpha}\Gamma_{ij}^k - \frac{1}{\alpha}, &amp;amp; Z_{ij}^k - \frac{1}{\alpha}\Gamma_{ij}^k &amp;gt; \frac{1}{\alpha}, \\&#xA;0, &amp;amp; |Z_{ij}^k - \frac{1}{\alpha}\Gamma_{ij}^k| \le \frac{1}{\alpha}, \\&#xA;Z_{ij}^k - \frac{1}{\alpha}\Gamma_{ij}^k + \frac{1}{\alpha}, &amp;amp; Z_{ij}^k - \frac{1}{\alpha}\Gamma_{ij}^k &amp;lt; -\frac{1}{\alpha}.&#xA;\end{cases}$$&#xA;The update for $B$ is simply&#xA;$$B^{k+1} = Z_B - \frac{1}{\alpha}\Gamma_B,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as noted by original poster Justin Solomon in the comments. Overall, doing the update for $J,B$ just requires looping through the entries of your matrices and evaluating the above formulas for each entry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Schur complement for the $Z_J,Z_B$ system&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most costly step of the iteration is solving the system,&#xA;$$\begin{align}&#xA;0 &amp;amp;= -\frac{\alpha}{2}(J^{k+1} - Z_J + \frac{1}{\alpha}\Gamma_J) + \frac{\beta}{2}M^T(MZ_J + Z_BY - X + \frac{1}{\beta}\Lambda), \\&#xA;0 &amp;amp;= -\frac{\alpha}{2}(B^{k+1} - Z_B + \frac{1}{\alpha}\Gamma_B) + \frac{\beta}{2}(MZ_J + Z_BY - X + \frac{1}{\beta}\Lambda)Y^T.&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To that end, it is worth some effort to construct a good solver/preconditioner for this system. In this section we do so by &lt;a href=&quot;http://en.wikipedia.org/wiki/Vectorization_%28mathematics%29&quot; rel=&quot;noreferrer&quot;&gt;vectorizing&lt;/a&gt;, forming a &lt;a href=&quot;http://en.wikipedia.org/wiki/Schur_complement&quot; rel=&quot;noreferrer&quot;&gt;Schur complement&lt;/a&gt;, doing some &lt;a href=&quot;http://en.wikipedia.org/wiki/Kronecker_product&quot; rel=&quot;noreferrer&quot;&gt;Krnoecker product&lt;/a&gt; manipulations and then unvectorizing. The resulting Schur complement system is a slightly modified &lt;a href=&quot;http://en.wikipedia.org/wiki/Sylvester_equation&quot; rel=&quot;noreferrer&quot;&gt;Sylvester equation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In what follows the following identities about vectorization and Kronecker products are absolutely key:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\mathrm{vec}(ABC) = (C^T \otimes A)\mathrm{vec}(B),$&lt;/li&gt;&#xA;&lt;li&gt;$(A \otimes B)(C \otimes D) = AC \otimes BD$,&lt;/li&gt;&#xA;&lt;li&gt;$(A \otimes B)^{-1} = A^{-1} \otimes B^{-1}$, and&lt;/li&gt;&#xA;&lt;li&gt;$(A \otimes B)^T = A^T \otimes B^T$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These identities hold whenever the matrix sizes and invertibility are such that each side of the equation is a valid expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vectorized form of the system is,&#xA;$$\left(\alpha I +\beta\begin{bmatrix}I \otimes M^TM &amp;amp; (Y \otimes M)^T \\ Y \otimes M &amp;amp; YY^T \otimes I\end{bmatrix}\right)\begin{bmatrix}\mathrm{vec}(Z_J) \\ \mathrm{vec}(Z_B)\end{bmatrix} = \begin{bmatrix}\mathrm{vec}(\alpha J + \beta M^TX + \Gamma_J - M^T\Lambda) \\ \mathrm{vec}(\alpha B + \beta XY^T + \Gamma_B - \Lambda Y^T)\end{bmatrix},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or,&#xA;$$\begin{bmatrix}I \otimes (\alpha I + \beta M^TM) &amp;amp; \beta (Y \otimes M)^T \\ \beta Y \otimes M &amp;amp; (\alpha I + \beta YY^T) \otimes I\end{bmatrix} \begin{bmatrix}\mathrm{vec}(Z_J) \\ \mathrm{vec}(Z_B)\end{bmatrix} = \begin{bmatrix}\mathrm{vec}(F) \\ \mathrm{vec}(G)\end{bmatrix},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $F$ and $G$ are condensed notation for the right-hand-side. Now we perform block-gaussian-elimination/Schur complement to eliminate the lower left block of the matrix, in the process condensing the Kronecker products. This is,&#xA;$$\begin{bmatrix}I \otimes (\alpha I + \beta M^TM) &amp;amp; \beta (Y \otimes M)^T \\ 0 &amp;amp; (\alpha I + \beta YY^T) \otimes I - \beta^2 YY^T \otimes M(\alpha I + \beta M^TM)^{-1} M^T\end{bmatrix} \dots \\ \cdot \begin{bmatrix}\mathrm{vec}(Z_J) \\ \mathrm{vec}(Z_B)\end{bmatrix} = \begin{bmatrix}\mathrm{vec}(F) \\ \mathrm{vec}(G) - \beta Y \otimes M(\alpha I + \beta M^TM)^{-1}\mathrm{vec}(F)\end{bmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unvectorizing, the two equations we have to solve in sequence are,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$$Z_B (\alpha I + \beta YY^T) - (\beta M (\alpha I + \beta M^TM)^{-1} M^T)Z_B(\beta YY^T) \dots \\ = G - \beta M (\alpha I + \beta M^TM)^{-1} F Y^T$$&lt;/li&gt;&#xA;&lt;li&gt;$$(\alpha I + \beta M^TM) Z_J = F - \beta M^T Z_B Y.$$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution of Schur complement system when $Y,M$ are squareish, high rank&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this section we solve the Schur complement system for $Z_B$ (equation 1. above) by using precomputed &lt;em&gt;full&lt;/em&gt; SVD's of the matrices $YY^T, MM^T, M^TM$ and applying a modified version of the Bartels-Stewart algorithm for the Sylvester equation. The algorithm is slightly modified from the standard version to account for the extra $\beta YY^T$ on the second term, which makes it not quite the Sylvester equation. Once $Z_B$ is found via the first equation, $Z_J$ can be found from the second equation easily. The second equation is trivial to solve via any method you like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method requires an upfront cost to precompute two full SVDs before the ADMM process starts, but then is fast to apply in the actual ADMM iterations. Since the method deals with full SVDs of the constraint matrices, it is appropriate when they are close to square and high rank. A more complicated method using low rank SVD's is also possible, but is presented in a later section. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method develops as follows. Let &#xA;$$Q D Q^T = YY^T, \\&#xA;W\Sigma W^T = MM^T, \\&#xA;VTV^T = M^TM$$&#xA;denote precomputed full singular value decompositions, and condense the right hand side to be $H$. Then the first equation becomes,&#xA;$$Z_B Q (\alpha I + D) Q^T - W \beta \Sigma (\alpha I + \Sigma)^{-1}\Sigma W^T Z_B Q D Q^T = H.$$&#xA;Multiplying by the orthogonal factors to clear out the left and right and setting a new temporary unknown $A = W^T Z_B Q$, this further becomes,&#xA;$$A (\alpha I + D) - \beta \Sigma (\alpha I + \Sigma)^{-1}\Sigma A D = W H Q^T.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we can find $A$ by solving the &lt;em&gt;diagonal&lt;/em&gt; system,&#xA;$$\left((\alpha I + D) \otimes I + D \otimes \beta \Sigma (\alpha I + \Sigma)^{-1}\Sigma \right)\mathrm{vec}(A) = \mathrm{vec}(W H Q^T).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having found $A$, we compute $Z_B = W A Q^T$, and knowing $Z_B$ we solve the second equation above for $Z_J$, which is trivial since we already have the eigenvalue decomposition for $M^TM$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The upfront cost is computing two symmetric positive definite eigenvalue decompositions of $M^TM$ and $YY^T$, and then the cost-per-iteration for a complete solve is dominated by a handful of matrix-matrix multiplications, which is on the same order of magnitude as doing 1 CG subiteration. If the upfront eigenvalue decompositions are too costly, then they can be computed inexactly by, for example, terminating the &lt;a href=&quot;http://en.wikipedia.org/wiki/Lanczos_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Lanczos iteration&lt;/a&gt; early and keeping the largest eigenvectors. Then the method can be used as a good preconditioner for CG rather than a direct solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution method when $M,Y$ are very rectangular or have low rank approximation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we turn our attention to solving or preconditioning the $Z_J,Z_B$ when either a) the input matrices $M,Y$ are very rectangular - meaning they have many more rows than columns or vice versa - or b) they have low rank approximation. The derivation below involves extensive use of Woodbury formula, Schur complement, and other similar manipulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We start with our Schur complement system, &#xA;$$(\alpha I + \beta YY^T) \otimes I - \beta^2 YY^T \otimes M(\alpha I + \beta M^TM)^{-1} M^T.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few manipulations transform this system into a more symmetric form,&#xA;$$(\alpha I + \beta I \otimes MM^T + \beta YY^T \otimes I)\mathrm{vec}(Z_B) = \left(I \otimes (I + \frac{\beta}{\alpha}MM^T)\right)\mathrm{vec}(H).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we bring in the low rank approximations. Let &#xA;$$Q D^{1/2} Q_2^T = Y \\&#xA;W \Sigma^{1/2} V^T = M$$&#xA;be either the reduced SVD's or low rank approximations of $Y$ and $M$ ($Q_2$ is a placeholder and is not used). Substituting these into our system yields the following matrix inverse we wish to apply,&#xA;$$(\alpha I + \beta I \otimes W \Sigma W^T + \beta YY^T \otimes I)^{-1}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the matrix we with to invert is a low-rank update to the identity, the logical strategy is to try to use the Woodbury formula,&#xA;$$(A + UCU^T)^{-1} = A^{-1} - A^{-1}U(C^{-1}+U^TA^{-1}U)^{-1}U^TA^{-1}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, some care is needed since the low rank pieces $I \otimes W$ and $Y \otimes I$ are not orthogonal. Thus to apply the Woodbury formula we collect both low rank updates into a single big update. Doint so and applying the Woodbury formula yields,&#xA;$$\left(\frac{1}{\alpha} I + \beta \begin{bmatrix}I\otimes W &amp;amp; Q \otimes I\end{bmatrix}\begin{bmatrix}I \otimes \Sigma &amp;amp; \\ &amp;amp; D \otimes Y\end{bmatrix}\begin{bmatrix}I \otimes \Sigma^T \\ Q^T \otimes I\end{bmatrix}\right)^{-1} \\&#xA;= \alpha I - \frac{\beta}{\alpha^2}\begin{bmatrix}I\otimes W &amp;amp; Q \otimes I\end{bmatrix}\begin{bmatrix}I \otimes (\Sigma^{-1}+\frac{\beta}{\alpha}I) &amp;amp; \frac{\beta}{\alpha}Q \otimes W^T\\ \frac{\beta}{\alpha}Q^T\otimes W &amp;amp; (D^{-1} + \frac{\beta}{\alpha}I) \otimes Y\end{bmatrix}^{-1}\begin{bmatrix}I \otimes \Sigma^T \\ Q^T \otimes I\end{bmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The core inverse can be computed by the blockwise 2x2 inverse formula,&#xA;$$\begin{bmatrix}A &amp;amp; B \\ B^T &amp;amp; C\end{bmatrix}^{-1} = \begin{bmatrix}(A-BC^{-1}B^T)^{-1} &amp;amp; -A^{-1}B(C-B^TA^{-1}B)^{-1} \\ -C^{-1}B^T(A-BC^{-1}B^T)^{-1} &amp;amp; (C-B^TA^{-1}B)^{-1}\end{bmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This post is already long enough so I'll spare the lengthy details of the calculation, but the end result is that plugging the necessary submatrices into the blockwise inverse and multiplying everything through yields the following explicit form for the overall inverse,&#xA;$$(\alpha I + \beta I \otimes MM^T + \beta YY^T \otimes I)^{-1} = \frac{1}{\alpha} I - \frac{\beta}{\alpha^2}(t_{11} + s_{11} + t_{12} + s_{12} + t_{21} + s_{21} + t_{22} + s_{22}),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&#xA;$$\begin{align}&#xA;t_{11} &amp;amp;= \frac{\alpha}{\beta}I \otimes W l^{-1} W^T \\&#xA;s_{11} &amp;amp;= (Q \otimes W l^{-1})D_{11}(Q^T \otimes l^{-1}W^T) \\&#xA;t_{12} &amp;amp;= -\frac{\alpha}{\beta} Q h^{-1} Q^T \otimes W l^{-1} W^T \\&#xA;s_{12} &amp;amp;= -(Q h^{-1} \otimes W l^{-1})D_{22}(h^{-1} Q^T \otimes W^T) \\&#xA;t_{21} &amp;amp;= t_{12} \\&#xA;s_{21} &amp;amp;= -(Q h^{-1} \otimes W)D_{22}(h^{-1} Q^T \otimes l^{-1} W^T) \\&#xA;t_{22} &amp;amp;= \frac{\alpha}{\beta}Q h^{-1} Q^T \otimes I \\&#xA;s_{22} &amp;amp;= (Q h^{-1} \otimes W)D_{22}(h^{-1}Q^T \otimes W^T) \\&#xA;D_{11} &amp;amp;= \frac{\alpha}{\beta}\left(h \otimes I - I \otimes l^{-1} \right)^{-1} \\&#xA;D_{22} &amp;amp;= \frac{\alpha}{\beta}\left(I \otimes l - h^{-1} \otimes I \right)^{-1} \\&#xA;l &amp;amp;= \frac{\alpha}{\beta} \Sigma^{-1} + I \\&#xA;h &amp;amp;= \frac{\alpha}{\beta} D^{-1} + I.&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this form, we can apply the inverse and find $Z_B$ term by term through 8 left and right matrix multiplication sandwiches. The general formula for applying the sum of Kronecker products is,&#xA;$$\left((A_1 \otimes B_1) + (A_2 \otimes B_2) + \dots\right)\mathrm{vec}(C) = \mathrm{vec}(B_1^T C A_1 + B_2^T C A_2 + \dots ).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that all the explicit inverses we ended out with are diagonal, so there is nothing to be &quot;solved&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Linear solver code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented the above two $z_J,Z_B$ solvers in Matlab. The seem to work well. The solver code is here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/NickAlger/MeshADMM/blob/master/zkronsolve.m&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/NickAlger/MeshADMM/blob/master/zkronsolve.m&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A test script for checking that the solvers work is here. It also shows by example how calll the solver code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/NickAlger/MeshADMM/blob/master/test_zkronsolve.m&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/NickAlger/MeshADMM/blob/master/test_zkronsolve.m&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Concluding remarks&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADMM-type methods are well suited for problems like this, but you would need to roll your own implementation. The overall structure of the method is pretty simple so implementation is not too difficult in something like MATLAB. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The piece missing from this post that would be need to be specified to fully define the method for your problem is a choice of penalty parameters $\alpha,\beta$. Luckily the method is generally pretty robust as long as the parameter vales aren't crazy. The Boyd and Parikh paper has a section about the penalty parameters as do the references therein, but I would just experiment with the parameters until you get reasonable convergence rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $Z_J,Z_B$ solver strategies presented are highly effective if the constraint matrices are either a) dense, squareish, and high rank, or b) have a good low rank approximation. Another useful solver that could be a topic of future work would be a solver optimized for the following case -  the constraint matrix $M$ is sparse and squareish and high rank, but there exists a good preconditioner for $\alpha I + MM^T$. This would be the case if, for example, $M$ is a discretized Laplacian.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2014-03-12T09:29:47.080" LastActivityDate="2014-03-12T09:29:47.080" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="10964" PostTypeId="1" AcceptedAnswerId="10965" CreationDate="2014-03-04T11:46:34.503" Score="2" ViewCount="2773" Body="&lt;p&gt;I have case of a rotating fan at Low- speed ie about 300-400 rpm. I have successfully carried out RAS simulations using cyclicAMI approach where, the motion of the fan is modeled as rigid body rotation of a cylindrical AMI mesh section, which has same speed as our fan. I have tried various levels of mesh refinement and differing y+ and so forth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there are any other methods of modelling this problem, as I am not able to predict some important parameters accurately. For the attached mesh I used solidBodyMotionFvMesh for rotating inner domain/cyclicAMI. I am looking into &#xA;dynamicFvRefineFvMesh and dynamicMotionSolverFvMesh for rotating the fan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;please let me know which other approach is suitable and how to set-up the mesh for such case (pointers to examples would be great)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS : Motion of the fan is strictly rotation about a fixed axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know any other way (except below accepted answer) of conducting this simulation.[cry for more answers]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/AXER5.jpg&quot; alt=&quot;AMI mesh&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="6878" LastEditorUserId="6878" LastEditDate="2014-03-06T11:21:10.263" LastActivityDate="2014-03-06T11:21:10.263" Title="Modelling rotating machinery other than AMI rotating meshes" Tags="|openfoam|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10965" PostTypeId="2" ParentId="10964" CreationDate="2014-03-04T14:14:43.860" Score="3" Body="&lt;p&gt;You can do a simulation using the MRF (moving/multiple reference frame) approach. For this you need to create a cylindrical cell set that is aligned with the axis of rotation. There are some cases in the OpenFOAM tutorials that make use of this approach. You can also consult e.g. the Fluent Manual for some theory behind MRF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With MRF there is no actual mesh motion, so the simulation will be faster than with a moving mesh. However, this method is less accurate than a simulation with a moving mesh.&lt;/p&gt;&#xA;" OwnerUserId="3510" LastActivityDate="2014-03-04T14:14:43.860" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="10966" PostTypeId="1" CreationDate="2014-03-04T14:28:58.520" Score="2" ViewCount="125" Body="&lt;p&gt;Am quite new to wavelet analysis and would like some help. I am performing a spatio-temporal analysis of monthly gridded rainfall data. With PCA, I can reduce the dimension of the rainfall data into a few leading modes, yielding EOFs (which explain spatial variability) and principal components (explaining temporal variability).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to perform a similar analysis with wavelets using the Matlab Wavelet Toolbox. As of now, I am able to decompose a 2D data (spatial decomposition in my case) but unable to take into account the temporal variability in the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first course of action has been to first compress the data with PCA and then perform wavelet decomposition of the leading modes in both the spatial (EOFs) and temporal (PCs) domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if this is the right way to perform such an analysis and would like suggestions as to how to proceed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="7521" LastActivityDate="2014-03-04T14:28:58.520" Title="Spatio-temopral wavelet analysis" Tags="|matlab|data-analysis|wavelet|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10967" PostTypeId="1" CreationDate="2014-03-04T16:16:32.223" Score="1" ViewCount="53" Body="&lt;p&gt;I have a large array of zeros and ones and I need to count instances of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 1     1 0     0 0     1 1&#xA;0 1,    1 0,    1 1,    0 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I would like to exclude all other $2 \times 2$ submatrices... such as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 1     1 1&#xA;0 0,    1 0,    etc.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the space of $2 \times 2$ binary matrices has $2^4 = 16$ elements, and I could like to single out these 4 matrices.  I can't use a linear condition since&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 1     0 0   0 1&#xA;0 1  +  1 1 = 1 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is possible to explicitly write a boolean that checks these explicitly, but then I lose the wonderful numpy quickness...&lt;/p&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2014-03-04T19:28:50.653" Title="detecting special $2 \times 2$ matrices in a large array of zeros and ones" Tags="|matrix|numpy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10968" PostTypeId="1" CreationDate="2014-03-04T17:25:46.597" Score="6" ViewCount="351" Body="&lt;p&gt;I'm working with non-linear optimization for imaging, such as MRI and CT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our problem is of the form $\|Ax-b \|_2^2+\lambda \|Wx\|_1$. $A$ is never formed explicitly, so we're limited to approaches that only use $Ax$ and $A^Hf$. Even so, these evaluations are quite expensive, whereas $Wx$ and $W^Tx$ are quite inexpensive to compute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been using a non-linear conjugate gradient (NCG) for this problem, but my backtracking line search tends to fail as $A^H$ is not the exact conjugate transpose of $A$, but an approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ignoring the non-linear term and using the conjugate gradient on the normal equations, everything works beautifully, and so does the Split-Bregman algorithm. The latter is however something like an order of magnitude slower than the NCG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if anyone knew if an iterative solver that allowed for the gradient ($A^H(Ax-f)$ to be approximate? &lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-08-18T13:30:57.100" Title="Non-linear optimization using approximate gradient" Tags="|convex-optimization|nonlinear-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10969" PostTypeId="2" ParentId="10967" CreationDate="2014-03-04T19:28:50.653" Score="1" Body="&lt;p&gt;Assuming you are trying to rule out adjacent submatrices, you can build an array that represents the $2 \times 2$ submatrices in binary and then check against a list of the valid types.  I.e.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Represent 2 by 2 submatrices of A in binary&#xA;subs = A[::2,::2]+2*A[::2,1::2]+4*A[1::2,::2]+8*A[1::2,1::2]&#xA;allowed = ... # List of submatrices to count expressed in binary&#xA;for a in allowed:&#xA;  print('count %d = %d'%(a,sum(subs==a))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="447" LastActivityDate="2014-03-04T19:28:50.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10971" PostTypeId="2" ParentId="10958" CreationDate="2014-03-04T20:22:52.827" Score="2" Body="&lt;p&gt;It's not possible to optimize rigorously a discontinuous objective function without resorting to all sorts of trickery.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know where all of the discontinuities are, you can subdivide your problem into a collection of problems, all of which have continuous objective functions. Hopefully, this collection is small; otherwise, it is intractable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't know where all of the discontinuities are, in some cases, you can use automatic differentiation-like tools to detect where they are, and if they are step discontinuities, it's possible to construct relaxed optimization problems and solve them in a branch-and-bound procedure. This sort of algorithm has exponential time complexity in the number of decision variables, and requires lots of infrastructure (automatic differentiation, the ability to calculate convex and concave relaxations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most deterministic algorithms require that a function be Lipschitz continuous, at minimum. Stochastic algorithms like simulated annealing are probably a good practical compromise in situations where you need an answer, have severe time constraints, and aren't picky about the quality of your answer (i.e., it has to be &quot;good enough&quot;). If the set of discontinuities is a set of measure zero (which I suspect holds in a lot of situations), then any stochastic method will almost surely avoid those discontinuities.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-03-04T20:22:52.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10972" PostTypeId="1" CreationDate="2014-03-04T22:25:37.027" Score="3" ViewCount="736" Body="&lt;p&gt;This should be easy, but...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to express the singular value decomposition of a 2 x 2 complex matrix $A$ as function of its coefficients $A_{ij}$. In &quot;closed form&quot;, no intermediate values, straight up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean is that if I express the SVD in terms of eigenvalues, say, then I have to write those eigenvalues in terms of the $A_{ij}$'s and substitute. In other words, I don't want to write the solution to SVD in terms of the solution of something else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibly useful background: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Wikipedia page on SVD, if you write $A$ as a linear combination of the Pauli matrices (and identity), the singular values are expressed in terms of the corresponding coefficients. But then I don't know how you find those coefficients, so I can't tell if they themselves are in &quot;closed form&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Singular_value_decomposition&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7529" LastEditorUserId="20162" LastEditDate="2016-10-07T06:26:00.587" LastActivityDate="2016-10-07T06:26:00.587" Title="singular value decomposition of a 2 x 2 complex matrix" Tags="|svd|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10973" PostTypeId="2" ParentId="10940" CreationDate="2014-03-04T23:19:32.017" Score="2" Body="&lt;p&gt;It si very weird. It should be a bug in scipy.sparse.linalg.eigs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at matlab, we see that opening the function with &lt;code&gt;open eigs&lt;/code&gt;, at line 822-825, they say that&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% eigs(A,B,k,stringSigma), stringSigma~='SM'&#xA;% A*x = lambda*B*x&#xA;% =&amp;gt; OP = inv(B)*A and use standard inner product.&#xA;mode = 2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have been looking for the same piece of code inside scipy, and it seems that they are doing the same at lines 541-543 in &lt;a href=&quot;https://github.com/scipy/scipy/blob/v0.13.0/scipy/sparse/linalg/eigen/arpack/arpack.py#L541&quot; rel=&quot;nofollow&quot;&gt;scipy/sparse/linalg/eigen/arpack/arpack.py#L541&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;elif self.mode == 2:&#xA;    self.workd[xslice] = self.OPb(self.workd[xslice])&#xA;    self.workd[yslice] = self.OPa(self.workd[xslice])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the OPb and OPa have been defined before as the product by $A$ and the inverse of $B$. So I did not found the difference, so it should be a bug.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab and also in Scipy, they seem to be solving $B^{-1}Ax = \lambda x$. Because of that I was wondering why $B$ is required to be positive definite and not just nonsingular (I think that $B$ must be positive definite for the shift-and-invert algorithm, because there it is used to define a norm). Thus, you can simply modify your code to do the same:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.sparse.linalg import eigs&#xA;from scipy.sparse.linalg import LinearOperator&#xA;from scipy.sparse.linalg import factorized&#xA;&#xA;A = np.diag([-5, -4, -3, -2, -1]).astype(np.float64)&#xA;B = np.diag([1, 1, -1, 1, 1]).astype(np.float64)&#xA;&#xA;B_inv = factorized(B);&#xA;def mv(v):&#xA;    temp = np.dot(A, v)&#xA;    return B_inv(temp)&#xA;&#xA;BA = LinearOperator(A.shape, matvec = mv) # BA.matvec(x) is np.dot(inv(B),np.dot(A,x))&#xA;vals, vecs = eigs(BA, 2, which='LR')&#xA;print vals&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;obtaining the right solution&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/lib/python2.7/dist-packages/scipy/sparse/linalg/dsolve/linsolve.py:258: SparseEfficiencyWarning: splu requires CSC matrix format&#xA;  warn('splu requires CSC matrix format', SparseEfficiencyWarning)&#xA;[ 3.+0.j -1.+0.j]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got a warning because $B$ is not in csr format when performing the sparse LU decomposition (splu) by means of the call to &lt;code&gt;factorized&lt;/code&gt;, so the factorization function perform the conversion to csr and do the sparse LU. I guess that your matrices already are in csr format, so it will not be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you can also play with an incomplete LU factorization, solving the system with $B$ iteratively, and lots of choices for the reordering of $B$ before the LU factorization to make it efficient.&lt;/p&gt;&#xA;" OwnerUserId="7277" LastEditorUserId="7277" LastEditDate="2014-03-04T23:25:25.807" LastActivityDate="2014-03-04T23:25:25.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10974" PostTypeId="2" ParentId="10972" CreationDate="2014-03-04T23:42:46.030" Score="6" Body="&lt;p&gt;The Pauli matrices and the identity matrix form an orthogonal basis of the space of $2\times 2$ matrices, so finding the expansion coefficients amounts to just a projection onto this basis (i.e., you need to form the inner product of your matrix onto each element of this basis).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, for $2\times 2$ matrices, the answer is simple enough to write things down right away. The eigenvalues are those values $\lambda$ for which&#xA;$$&#xA;  \det (A-\lambda I) = \det \begin{pmatrix} a_{11}-\lambda &amp;amp; a_{12} \\ a_{21} &amp;amp; a_{22}-\lambda \end{pmatrix} = 0.&#xA;$$&#xA;This is a quadratic equation in $\lambda$:&#xA;$$&#xA; (a_{11}-\lambda)(a_{22}-\lambda)-a_{12}a_{21}&#xA; = 0&#xA;$$&#xA;for which you can write down the roots immediately as a function of the elements of $A$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-10-04T17:55:32.843" LastActivityDate="2016-10-04T17:55:32.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10975" PostTypeId="1" CreationDate="2014-03-04T23:54:40.570" Score="0" ViewCount="2458" Body="&lt;p&gt;I have an orthotropic material with a (6x1) stress vector known in the global coordinate system and yield surfaces known in a local coordinate system. So far I have only needed to convert from local to the global system and this has been done in the standard way of C = G.T*D*G where C is the stiffness matrix in the global coordinates, G is a 6x6 transformation matrix and D is the stiffness matrix in the local coordinate system. Now I need to convert my stresses living in the global system into stresses in the local system. The way I attempted to do this is using Sl = G.I*Sg, where Sl is the stress vector in the local coordinates and Sg is the stress vector in the globals, and G.I is the inverse of G. However I find that in some cases G is singular, ie not invertible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest a way around this problem, preferably by not inverting G?&lt;/p&gt;&#xA;" OwnerUserId="4296" LastEditorUserId="276" LastEditDate="2014-03-05T00:03:55.027" LastActivityDate="2014-03-06T08:02:41.053" Title="What to do with singular (non-invertible) rotation matrix" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10976" PostTypeId="2" ParentId="10957" CreationDate="2014-03-04T23:55:38.013" Score="6" Body="&lt;p&gt;If you have derivatives available, no method can beat Newton's method in practice unless you use very specific features of your objective function. This is true whether you want to solve one or a billion problems: solving each one of them is most efficiently done using Newton's method since it is the only one that guarantees quadratic convergence, and this in turn typically leads to convergence to practical accuracies within less than 10 iterations, often significantly less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Newton's method gets into a bit of trouble occasionally if your objective function is not convex, in which case you need to modify the Hessian appropriately. But, as you say, this is not important in your application.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-04T23:55:38.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10977" PostTypeId="2" ParentId="10946" CreationDate="2014-03-05T00:05:39.707" Score="5" Body="&lt;p&gt;Have you looked at the various exascale reports that have come out? Hardward failures are not a significant concern today -- sure, they happen, but their frequency is not sufficiently high to cause grave worry. But they are estimated to be sufficiently frequent on exascale systems with $O(10^8)$ or more cores that codes need to be prepared to react appropriately. I believe that these issues have been laid out in the reports on roadmaps towards exascale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recollection is that among the various failure modes, single bit flips in memory or on processor cores were not the most significant concerns. Rather, it was entire nodes going down, e.g. due to disk failure, operating system faults, etc. The current exascale designs therefore all call for periodic checkpointing of codes into flash RAM, preferably transmitting the checkpoint data off-node. Codes will then need to be able to restart on the fly from a previously saved state if the system encounters that one node has disappeared, replacing this node with a hot-start node elsewhere in the system.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-05T00:05:39.707" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="10978" PostTypeId="2" ParentId="10943" CreationDate="2014-03-05T00:13:13.497" Score="3" Body="&lt;p&gt;Like Jed already said, reuse of linear algebra components such as matrices and vectors is not commonly done. It is also not necessary: setting up these components is comparably very cheap relative to the cost of solving linear systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look for things that change when going from static to adaptively refined meshes, then the biggest obstacle is dealing with hanging nodes. In &lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;nofollow&quot;&gt;deal.II&lt;/a&gt;, this is handled by the &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/classConstraintMatrix.html&quot; rel=&quot;nofollow&quot;&gt;ConstraintMatrix&lt;/a&gt; class, which takes several 1000 lines of code. You can find some of the description of what this class does in the paper by myself and Oliver Kayser-Herold (linked to from &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications.html&quot; rel=&quot;nofollow&quot;&gt;my publications page&lt;/a&gt;). No other component (other than handling the mesh, of course) has required so much adaptation when going from fixed to adaptive meshes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-05T00:13:13.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10979" PostTypeId="2" ParentId="10957" CreationDate="2014-03-05T02:57:13.863" Score="5" Body="&lt;p&gt;As requested I'm upgrading my comments to an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer the original question it's necessary to understand how much you know about the function. How many derivatives can you readily compute? For instance, strong convexity matters with first-order methods. For second-order methods a similar characteristic of the third derivative probably applies (e.g., self-concordance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For first-order methods, if you have strong convexity, then gradient search can do quite a good job. If you don't, then consider the so-called &quot;accelerated first-order methods&quot;. Theoretically, these methods require Lipschitz continuity, but in practice, you can estimate and adapt the Lipschitz constant and do fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For second-order methods, you really can't beat Newton, unless you exploit specific knowledge of your function. That's a big &quot;unless&quot; though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing we don't know is the computational complexity of computing the values and derivatives. My intuition here is that gradient descent or Newton will be as good as you can expect, with the choice depending on the cost of computing f′′. Unless the second derivative is wickedly expensive, and in your case it sounds like it's not, then Wolfgang wins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the billions of problems are closely related, then a warm start from a nearby solution will probably help, especially if you can start within the region of quadratic convergence for Newton.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2014-03-05T02:57:13.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10980" PostTypeId="1" AcceptedAnswerId="10984" CreationDate="2014-03-05T09:51:20.103" Score="7" ViewCount="9320" Body="&lt;p&gt;The question is if Python Numpy library can use back subsitution to solve Ax=b if possible, that is, if A is lower triangular? Do numerical linear algebra packages do this? I would think Numpy would detect the triangular state and use the proper approach, but a Google search returns things like scipy.linalg.lu_solve or scipy.linalg.cho_solve, which I assume are to be used in case when we know we have a triangular matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance,&lt;/p&gt;&#xA;" OwnerUserId="621" OwnerDisplayName="user423805" LastEditorUserId="621" LastEditDate="2014-03-05T11:34:46.353" LastActivityDate="2014-03-07T22:09:15.117" Title="Can numpy.linalg.solve use back substitution when possible?" Tags="|linear-algebra|numpy|scipy|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="10981" PostTypeId="1" CreationDate="2014-03-05T13:21:25.660" Score="4" ViewCount="376" Body="&lt;p&gt;In light of &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10943/which-novel-data-structures-are-used-in-adaptive-fem&quot;&gt;this&lt;/a&gt; question and some stuff I read online I am wondering if large FE libs (e.g., deal.ii, libmesh etc.) that do AMR keep the entire mesh or possibly a coarse version of the entire mesh on each core. Also, does this limit the size of the problems that can be solved on machines with low RAM per core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally how is load balancing performed? Is the entire mesh repartitioned after each refinement/coarsening step? &lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-03-05T14:16:43.197" Title="Meshes in codes that do AMR" Tags="|adaptive-mesh-refinement|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10982" PostTypeId="2" ParentId="10981" CreationDate="2014-03-05T14:16:43.197" Score="8" Body="&lt;p&gt;deal.II keeps the entire &lt;em&gt;coarse&lt;/em&gt; mesh on every processor, but of course we cannot do that with the actual mesh after many refinement steps. It is true that this somewhat limits the size of problems we can solve to maybe a few hundred thousand coarse mesh cells on typically-sized cluster nodes. However, this is plenty for most realistic cases for a reasonable coarse approximation of the geometry. Adaptive mesh refinement can then resolve the details -- and for the refined mesh you can go about as far as you want, we have done meshes with several billion cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as load balancing is concerned, yes, we do that after every refinement/coarsening step. It turns out that this is not an operation that requires a noticeable fraction of the overall run time of programs, even on large numbers of processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I admit that I do not know whether the observations above are true for other libraries. The authors of these libraries need to speak for their own products.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-05T14:16:43.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10983" PostTypeId="2" ParentId="10980" CreationDate="2014-03-05T20:23:48.597" Score="7" Body="&lt;p&gt;No. The &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.solve.html&quot; rel=&quot;noreferrer&quot;&gt;numpy.linalg.solve&lt;/a&gt; method uses LAPACK's DGESV, which is a general linear equation solver driver. If you know that your matrix is triangular, you should use a driver specialized for that matrix structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.solve.html&quot; rel=&quot;noreferrer&quot;&gt;scipy.linalg.solve&lt;/a&gt; does something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB detects triangularity in a solve if you use the backslash operator; see &lt;a href=&quot;http://www.mathworks.com/matlabcentral/answers/96919&quot; rel=&quot;noreferrer&quot;&gt;this page&lt;/a&gt; for pseudocode.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-03-05T21:01:26.460" LastActivityDate="2014-03-05T21:01:26.460" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="10984" PostTypeId="2" ParentId="10980" CreationDate="2014-03-05T20:29:09.370" Score="10" Body="&lt;p&gt;Looking at the information of &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.solve.html&quot; rel=&quot;noreferrer&quot;&gt;nympy.linalg.solve&lt;/a&gt; for dense matrices, it seems that they are calling &lt;a href=&quot;http://www.netlib.no/netlib/lapack/double/dgesv.f&quot; rel=&quot;noreferrer&quot;&gt;LAPACK subroutine gesv&lt;/a&gt;, which perform the LU factorization of your matrix (without checking if the matrix is already lower triangular) and then solves the system. So the answer is NO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, it makes sense. If you do not have an easy (cheap) way to verify that your matrix is triangular (lower or upper), it can be very expensive to check this things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know that your matrix is lower triangular, it is better to solve it in scipy with &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.solve_triangular.html&quot; rel=&quot;noreferrer&quot;&gt;solve_triangular&lt;/a&gt;, while the matrix is still dense square matrix (so you are consumming a lot of effort).&lt;/p&gt;&#xA;" OwnerUserId="7277" LastActivityDate="2014-03-05T20:29:09.370" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10985" PostTypeId="2" ParentId="10975" CreationDate="2014-03-06T05:33:40.660" Score="-1" Body="&lt;p&gt;If G is singular, use an approximation, like the pseudo inverse.&lt;/p&gt;&#xA;" OwnerUserId="5106" LastActivityDate="2014-03-06T05:33:40.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10986" PostTypeId="2" ParentId="10975" CreationDate="2014-03-06T07:53:19.267" Score="1" Body="&lt;p&gt;Rotation matrices being orthogonal should always remain invertible. However in certain cases (e.g. when estimating it from data or so on) you might end up with non-invertible or non-orthogonal matrices. There are some ways to get around with this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) If your issues are numerical, you might just add some small random noise to this matrix - or its diagonal to purturb it. This is not an ideal operation and disrupts the orthogonality. But you can now proceed to step 2, to recover it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) One way to orthogonalize your rotation matrix is to use SVD as in MATLAB notation $[U,S,V]=svd(G)$. And you should check the singular values $S$ to see if they correspond to the identity matrix. If not replace them by the identity matrix and recompose the matrix. This would just equate to $G=U*V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way you guarantee the orthogonality and thus invertibility (For orthogonal matrices the inverse is equal to the transpose). So, you might just use the transpose operation to get the inverse of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are really keen on learning some methods in detail, here is a relatively easy to read tutorial:&#xA;&lt;a href=&quot;http://www.cis.upenn.edu/~cis610/invers.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cis.upenn.edu/~cis610/invers.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers, &lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2014-03-06T08:02:41.053" LastActivityDate="2014-03-06T08:02:41.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10988" PostTypeId="1" CreationDate="2014-03-06T12:48:56.343" Score="2" ViewCount="2194" Body="&lt;p&gt;In which problem category can I put a quadratic programming problem with only continuous values whereas the matrix A should be symmetric but needs not to be positive semi-definite?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minimize/maximize x' * A * x&#xA;whereas&#xA;lowerbound &amp;lt;= x &amp;lt;= upperbound&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can I reformulate it as a Mixed Integer Problem for example?&#xA;Or is there some algorithm or method that can cope with this problem?&lt;/p&gt;&#xA;" OwnerUserId="7146" LastEditorUserId="7568" LastEditDate="2014-03-07T16:14:52.960" LastActivityDate="2014-03-07T16:14:52.960" Title="How to solve (continuous) quadratic programming problem with non-PSD matrix" Tags="|optimization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="10990" PostTypeId="2" ParentId="10988" CreationDate="2014-03-06T14:16:39.283" Score="2" Body="&lt;p&gt;There is no need for $A$ to be positive definite. All you need is that your constraints cut off that part of the domain where the objective function becomes negative. For example, the problem&#xA;$$&#xA;  \min x_1^2 - x_2^2&#xA;$$&#xA;subject to constraints $-1\le x_2\le 1$ has a perfectly good solution (in fact, it has two). The only problem you have is that in general your problem may not be convex at the solution, so a straight up Newton method will not work. However, there are plenty of modifications to Newton's method that allow it to continue working even in situations like yours. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method to use such cases is the active set SQP method. Take a look at the book by Nocedal and Wright on Numerical Optimization.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-06T14:16:39.283" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10993" PostTypeId="1" AcceptedAnswerId="10994" CreationDate="2014-03-06T15:35:34.257" Score="4" ViewCount="1547" Body="&lt;p&gt;I am interested, how can we get CFL condition for every type of PDE? It's known that for 1st order linear equation&#xA;$$\frac{\partial u}{\partial t}+a\frac{\partial u}{\partial x}=0 $$&#xA;CFL is get from characteristics method, but how can we get in general? Any your advise will be welcome.&lt;/p&gt;&#xA;" OwnerUserId="4676" LastEditorUserId="4676" LastEditDate="2014-03-07T16:17:35.197" LastActivityDate="2014-03-07T16:17:35.197" Title="Courant Friedrichs Lewy condition - how to get it?" Tags="|pde|numerical-analysis|stability|cfl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10994" PostTypeId="2" ParentId="10993" CreationDate="2014-03-06T16:29:57.007" Score="9" Body="&lt;p&gt;For a hyperbolic &lt;em&gt;system&lt;/em&gt; of equations, you can write your equation as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial \mathbf{u}}{\partial t}  + [\mathbf{A}] \frac{\partial \mathbf{u}}{\partial x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then perform an eigendecomposition $\mathbf{A} = \mathbf{Q} \mathbf{\Lambda} \mathbf{Q}^{-1} $ where $ \mathbf{\Lambda} $ is a diagonal matrix of the eigenvalues, then defining $\mathbf{w} = \mathbf{Q}^{-1} \mathbf{u}$ you get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial \mathbf{w}}{\partial t}  + [\mathbf{\Lambda}] \frac{\partial \mathbf{w}}{\partial x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the diagonal elements (eigenvalues) are analogous to the wave speed $a$ in your 1D equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for general non-hyperbolic PDEs, you take the same approach and determine the eigenvalues of your operator to determine a stability limit. I think Hirsch's CFD book has a pretty lucid elaboration on the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a matter of semantics, authors often refer to &lt;em&gt;any&lt;/em&gt; non-dimensional step size limit as a &quot;CFL number&quot; (e.g. even for an elliptic equation).&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2014-03-07T12:49:26.990" LastActivityDate="2014-03-07T12:49:26.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10995" PostTypeId="1" AcceptedAnswerId="10996" CreationDate="2014-03-06T17:06:37.867" Score="1" ViewCount="178" Body="&lt;p&gt;I use &lt;strong&gt;gpbsv&lt;/strong&gt; command from &lt;strong&gt;Intel MKL&lt;/strong&gt; to solve symmetric positive band system. But unfortunately when the system is large I get an error &lt;strong&gt;Access violating writing location&lt;/strong&gt; in VisualStudio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone give an advice how could I solve this system. Maybe I could use more optimized library? Maybe it's simplier to write my own solver for large problems ( where can I read about realizations ) ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I have only one notebook with Core i3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best wishes.&lt;/p&gt;&#xA;" OwnerUserId="3956" LastActivityDate="2014-03-06T17:59:59.430" Title="large symmetric positive band matrix" Tags="|linear-algebra|finite-difference|banded-matrix|intel-mkl|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="10996" PostTypeId="2" ParentId="10995" CreationDate="2014-03-06T17:59:59.430" Score="1" Body="&lt;p&gt;If this is a large band matrix, I assume it is sparse and you could well use the sparse solvers in MKL (such as Pardiso or iterative such as conjugate gradients)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-78889273-7E77-426A-9B5E-23A7C2378D78.htm&quot; rel=&quot;nofollow&quot;&gt;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-78889273-7E77-426A-9B5E-23A7C2378D78.htm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-03-06T17:59:59.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="10997" PostTypeId="1" CreationDate="2014-03-06T19:03:44.913" Score="2" ViewCount="1136" Body="&lt;p&gt;What software or sites might simulate an &lt;strong&gt;N-body gravitational problem&lt;/strong&gt;? Application is related to &lt;a href=&quot;http://en.wikipedia.org/wiki/Klemperer_rosette&quot; rel=&quot;nofollow noreferrer&quot;&gt;Klemperer Rosette&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Iridium_satellite_constellation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iridium Satellites&lt;/a&gt;.  Bias for C, C++, PHP, or Java. Nice visuals a big plus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar questions are here in Computation Science for &lt;a href=&quot;https://scicomp.stackexchange.com/q/10494/7556&quot;&gt;binary stars&lt;/a&gt;, &lt;a href=&quot;https://scicomp.stackexchange.com/q/7448/7556&quot;&gt;satellites&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/q/7584/7556&quot;&gt;comets&lt;/a&gt; but I need to simulate several  planet-size celestial bodies over long timescales (1K - 1M years) to determine stability. Asked questions related to my application on &lt;a href=&quot;https://space.stackexchange.com/q/3872/213&quot;&gt;Space Exploration&lt;/a&gt; and &lt;a href=&quot;https://physics.stackexchange.com/q/101354/20241&quot;&gt;Physics&lt;/a&gt; but haven't gotten much interest or answers. Turning here to consider researching myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibilities so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sourceforge.net/projects/orsa/files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ORSA&lt;/a&gt; sounds promising but is 10 years old, has six errors on install, and crashes.  I have it showing a cute little Earth going around a cute little Sun at about 20 samples per second.  It will export thousands of PNGs but not an AVI.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sourceforge.net/apps/trac/odtbx/wiki/WikiStart&quot; rel=&quot;nofollow noreferrer&quot;&gt;ODTBX&lt;/a&gt; seems more for artificial satellites around Earth and may require MATLAB.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sourceforge.net/projects/celestia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Celestia&lt;/a&gt; seems more for visual rendering than long-term simulation.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://gmatcentral.org/display/GW/GMAT+Wiki+Home&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMAT&lt;/a&gt; seems more for interplanetary spacecraft of a negligible mass.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7556" LastEditorUserId="-1" LastEditDate="2017-04-13T12:58:56.060" LastActivityDate="2014-05-12T02:44:49.493" Title="Celestial mechanics simulation software? The 'N-Body Problem'" Tags="|software|simulation|computational-physics|open-source|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="10998" PostTypeId="1" AcceptedAnswerId="11014" CreationDate="2014-03-07T01:04:36.793" Score="5" ViewCount="183" Body="&lt;p&gt;I have a nasty system of coupled integral equations, which I managed to discretize and recast a non-linear system, i.e. something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \vec{w} = F \left( \vec{w} \right) \hspace{32pt} w \in \mathbb{R}^n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am employing a naive iterative solution method. For some reasonable initial choice of $\vec{w}$ I repeatedly calculate $ \vec{w}^{(n+1)} = F \left( \vec{w}^{(n)} \right) $. I define at each step the quantity $x= || \vec{w}^{(n+1)} - \vec{w}^{n} ||$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I observe that $x$ is decreasing (as expected!) iteration after iteration until I am close the the solution of the system (I know the solution for some particular values of the parameters...) Then $x$ starts rapidly increasing, and it seems like my method diverges progressively from the real solution of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I solve my problem? My guess would be taking the solution which produced the lowest $x$, and maybe trying some small perturbation to achieve better convergence starting from there, but that's just a guess. Just taking the solution with the lowest $x$ won't work, because I would like to have better convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance! &lt;/p&gt;&#xA;" OwnerUserId="7560" LastActivityDate="2014-04-17T15:29:44.997" Title="Convergence issues for a non-linear system" Tags="|nonlinear-equations|integral-equations|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="10999" PostTypeId="2" ParentId="10988" CreationDate="2014-03-07T02:48:48.783" Score="0" Body="&lt;p&gt;I would suggest that Wolfgang's and Geoff's answers are more general than necessary. This is a quadratic program with box constraints, and this problem has been the subject of a fair amount of &lt;em&gt;specific&lt;/em&gt; study. For instance, &lt;a href=&quot;http://dollar.biz.uiowa.edu/~sburer/papers/027-onboxqp.pdf&quot; rel=&quot;nofollow&quot;&gt;here is a PDF&lt;/a&gt; of &quot;On Nonconvex Quadratic Programming with Box Constraints&quot; by S. Burer and A. Letchford. I would suggest that's the perfect place to start, and the bibliography therein will provide more direction if needed.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2014-03-07T04:04:27.300" LastActivityDate="2014-03-07T04:04:27.300" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11000" PostTypeId="2" ParentId="10988" CreationDate="2014-03-07T03:29:21.630" Score="3" Body="&lt;p&gt;Like WolfgangBangerth says, it is a quadratic program. Without loss of generality, I will assume that you are minimizing your quadratic form. (If you are maximizing your quadratic form, replace that problem with minimizing the negative of your quadratic form, and what I say will still apply; where I refer to $A$ below, replace with $-A$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is positive semi-definite, your program is convex, so any optimizer that converges to a local minimum will converge to a global minimum, because the two sets of minima coincide for convex programs. Furthermore, this can be accomplished in polynomial time, and can exploit the quadratic structure of your problem, so it will be fast in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ has at least one negative eigenvalue, your problem is nonconvex. This statement is true no matter what your box constraints are, because the Hessian matrix of your quadratic form will be $2A$, since $A$ is symmetric. This constant Hessian matrix implies that the convexity properties of your objective function will be the same everywhere. For the case where $A$ has at least one negative eigenvalue, the problem is known to be $\mathcal{NP}$-hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I'm aware, most methods for quadratic programming (or more generally, nonlinear programming) converge to local optima. If local optima suffice for your application, then any standard method -- active set SQP being one common approach to small- or medium-sized problems -- will work. On the other hand, if you wish to solve your nonconvex optimization problem to global optimality, then you should select another approach. Usually, a convex optimization method is augmented by methods for branching and bounding to solve nonconvex problems deterministically; this class of algorithms is slower than algorithms for convex optimization, but has the advantage of returning a global optimum instead of a local one. Libraries that will solve nonconvex problems deterministically to global optimality include BARON and LINDOGlobal, which can be accessed as part of GAMS. If you do not have a copy of GAMS, you can submit a GAMS job through the NEOS server. You could also use a nondeterministic global optimization method, if you prefer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For examples of algorithms more tailored to nonconvex quadratic programs, see &lt;a href=&quot;http://dollar.biz.uiowa.edu/~sburer/papers/034-quadprogbb.pdf&quot; rel=&quot;nofollow&quot;&gt;Globally solving nonconvex quadratic programs via completely positive programming&lt;/a&gt;, &lt;a href=&quot;http://dollar.biz.uiowa.edu/~sburer/papers/017-qpbb.pdf&quot; rel=&quot;nofollow&quot;&gt;A finite branch-and-bound algorithm for nonconvex quadratic programming via semidefinite relaxations&lt;/a&gt;, and &lt;a href=&quot;http://www.researchgate.net/publication/225319278_Globally_solving_box-constrained_nonconvex_quadratic_programs_with_semidefinite-based_finite_branch-and-bound/file/d912f508148a8a21d0.pdf&quot; rel=&quot;nofollow&quot;&gt;Globally solving box-constrained quadratic programs with semidefinite-based finite branch-and-bound&lt;/a&gt;. These algorithms are much more tailored to your particular problem, but it looks like you'll need to roll your own implementation, which is why I suggest off-the-shelf libraries first.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-03-07T06:43:26.517" LastActivityDate="2014-03-07T06:43:26.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11001" PostTypeId="1" AcceptedAnswerId="11015" CreationDate="2014-03-07T14:39:03.450" Score="2" ViewCount="490" Body="&lt;p&gt;I am trying to write a 2d pseudo-spectral DNS code with random initial velocities. This is kind of a classic simulation where the very tiny vortices group together forming larger and larger vortices as the spectral energy cascades upward. So far I have done what I think is the intuitive thing to do: I have done a splitting scheme where the nonlinear term is treated explicitly using a 2nd order adams bashforth time stepping, while the diffusive term is completed implicitly by trapezoid time stepping. Finally the incompressibility condition is enforced by the projection method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far this seems to work well for other initial conditions, for example zero initial velocity  but with a forcing term added. However for the problem I am trying to tackle, the flow is just too diffusive and smears everything out for low Reynolds numbers, or becomes unstable for anything over a couple hundred (Re). I have tried hyper viscosity, but this then becomes unstable as well even after applying various filters (sharp, box, gaussian etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I was wondering whether anyone can tell me what standard methods are usually employed for this kind of problem. It is a common toy simulation set-up, so I am assuming there is a common introductory way that people solve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am solving the non-dimensionalized Navier-Stokes where the nonlinear term is written in conservative form.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2014-03-08T04:17:02.490" Title="2d pseudo-spectral turbulence simulation with random initial velocities" Tags="|fluid-dynamics|navier-stokes|spectral-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11002" PostTypeId="5" CreationDate="2014-03-07T16:19:39.423" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2014-03-07T16:19:39.423" LastActivityDate="2014-03-07T16:19:39.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11003" PostTypeId="4" CreationDate="2014-03-07T16:19:39.423" Score="0" Body="Courant-Friedrichs-Lewy number is a condition of stability of numerical discretization scheme for a time dependent PDE" OwnerUserId="4676" LastEditorUserId="549" LastEditDate="2014-03-07T16:21:57.143" LastActivityDate="2014-03-07T16:21:57.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11004" PostTypeId="2" ParentId="10942" CreationDate="2014-03-07T17:25:03.573" Score="0" Body="&lt;p&gt;Is the issue the division by zero (or close to zero) when computing the forces? I think what is often done (and what I did when implementing Barnes-Hut) was to include what is known as a softening factor. In other words when computing the radius between the centre of mass of the box and the specific particle you do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r=sqrt((bx-px)^2+(by-py)^2+epsilon)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where (bx,by) is the location of the box centre of mass, and (px,py) is the location of the particle, epsilon is the softening factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I have only done Barnes-Hut for particles rotating around a large mass (black-hole for example). This method my not be good for star clusters where collisions are common.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2014-03-07T17:25:03.573" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11005" PostTypeId="1" CreationDate="2014-03-07T17:47:32.023" Score="3" ViewCount="2420" Body="&lt;p&gt;The model I'm working on is a multinomial logit choice model. It's a very specific dataset so other existing MNLogit libraries don't fit with my data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically, it's a very complex function which takes 11 parameters and returns a loglikelihood value. Then I need to find the optimal parameter values that can minimize the loglikelihood using scipy.optimize.minimize. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the problems that I encounter with different methods: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Nelder-Mead’: it works well, and always give me the correct answer. However, it's EXTREMELY slow. For another function with a more complicated setup, it takes 15 hours to get to the optimal point. At the same time, the same function takes only 1 hour on Matlab using fminunc (which uses BFGS by default)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;‘BFGS’: This is the method used by Matlab. It works well for any simply functions. However, for the function that I have, it always fails to converge and returns 'Desired error not necessarily achieved due to precision loss.’. I've spent lots of time playing around with the options but still failed to work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Powell': It quickly converges successfully but returns a wrong answer. The code is printed below (x0 is the correct answer, Nelder-Mead works for whatever initial value), and you can get the data here: &lt;a href=&quot;https://www.dropbox.com/s/aap2dhor5jyxy94/data.csv&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/s/aap2dhor5jyxy94/data.csv&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;from scipy.optimize import minimize&#xA;&#xA;# https://www.dropbox.com/s/aap2dhor5jyxy94/data.csv&#xA;df = pd.read_csv('data.csv', index_col=0)&#xA;dfhh = df.hh&#xA;B = df.ix[:,'b0':'b4'].values # NT*5&#xA;P = df.ix[:,'p1':'p4'].values # NT*4&#xA;F = df.ix[:,'f1':'f4'].values # NT*4&#xA;SDV = df.ix[:,'lagb1':'lagb4'].values&#xA;&#xA;def Li(x):&#xA;    b1 = x[0] # coeff on prices&#xA;    b2 = x[1] # coeff on features&#xA;    a = x[2:7] # take first 4 values as alpha&#xA;    E = np.exp(a + b1*P + b2*F) # (1*4) + (NT*4) + (NT*4) build matrix (NT*J) for each exp()&#xA;    E = np.insert(E, 0, 1, axis=1) # (NT*5)&#xA;    denom = E.sum(1)&#xA;    return -np.log((B * E).sum(1) / denom).sum()&#xA;&#xA;&#xA;x0 = np.array([-32.31028223, 0.23965953, 0.84739154, 0.25418215,-3.38757007,-0.38036966])&#xA;np.random.seed(0)&#xA;x0 = x0 + np.random.rand(6)&#xA;&#xA; minL = minimize(Li, x0, method='Nelder-Mead',options={'xtol': 1e-8, 'disp': True})&#xA;# minL = minimize(Li, x0, method='BFGS')&#xA;# minL = minimize(Li, x0, method='Powell', options={'xtol': 1e-12, 'ftol': 1e-12})&#xA;print minL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Update: 03/07/14 Simpler Version of the Code&#xA;Now Powell works well with very small tolerance, however the speed of Powell is slower than Nelder-Mead in this case. BFGS still fails to work. &lt;/p&gt;&#xA;" OwnerUserId="7565" LastActivityDate="2014-03-07T17:47:32.023" Title="BFGS Fails to Converge" Tags="|optimization|python|convergence|scipy|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11006" PostTypeId="1" AcceptedAnswerId="11012" CreationDate="2014-03-07T19:35:27.153" Score="25" ViewCount="6355" Body="&lt;p&gt;Going to teach students of undergraduate level a course titled Introduction to Computer Programming. I am confused a bit. In Computational Physics scientists use C/C++ or Python or Fortran,CUDA etc..... this is time to build their base. What should I use? I know you can learn new programming language anytime in your life but which is wiser choice for me to elaborate them all basic programming concepts and OOP concepts later on.&lt;/p&gt;&#xA;" OwnerUserId="7572" LastEditorUserId="123" LastEditDate="2014-10-01T08:39:16.543" LastActivityDate="2017-06-28T13:09:45.957" Title="What language should I use when teaching an undergraduate course in computer programming?" Tags="|python|c++|computational-physics|languages|" AnswerCount="5" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="11007" PostTypeId="2" ParentId="11006" CreationDate="2014-03-07T20:26:41.270" Score="1" Body="&lt;p&gt;C,C++, &amp;amp; Fortran (listed in no particular order) are the three main programming languages used for computational mathematics/physics if you want to solve large problems on supercomputers. I think CUDA is considered a library that is used in conjunction with other languages for accelerated GPU computing. Matlab and python are great to learn for running output diagnostics and creating prototype models. They are also easier to learn and may be better for a course where you want to get across algorithms verses learning how to program. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, if your course is purely about programming I would choose C++ or, if this is the students first time programming, Python. Both of those languages have high utility outside the world of scientific computing. If the course is centred around learning algorithms to solve physics based problems then I think Matlab is undoubtedly the winner. &lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2014-03-07T20:37:51.297" LastActivityDate="2014-03-07T20:37:51.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11008" PostTypeId="2" ParentId="11006" CreationDate="2014-03-07T20:43:21.127" Score="2" Body="&lt;p&gt;Speaking as a not-too-far-removed undergraduate, and assuming that you are not teaching in the CS department, I think it would be a disaster to introduce students to computer programming with something like C, C++, or Fortran (or god forbid CUDA), even though others have pointed out that they are probably the status quo in scientific computing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're expecting to teach students scientific computing &lt;em&gt;and&lt;/em&gt; introduce them to programming in the same course, I would bet that's too much to cover in a semester unless you stick to an interpreted language like matlab or python.  In my experience, most classes in scientific computing at the undergraduate level are taught in one of those two any how, and python is becoming more and more useful as a production level language every day, so it still has some utility as a practical skill (beyond just teaching programming basics, I mean).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just my two cents.&lt;/p&gt;&#xA;" OwnerUserId="4879" LastActivityDate="2014-03-07T20:43:21.127" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11009" PostTypeId="1" AcceptedAnswerId="11013" CreationDate="2014-03-07T22:04:49.660" Score="0" ViewCount="428" Body="&lt;p&gt;I have constructed a function in Matlab that is convex and increasing (qualitatively similar to an exponential function but I am hoping to avoid the successive approximation requirement of exp). In my specification of the function I take a minimum of two convex functions which of course does not, in general, result in a convex function; however for the functions I am using, the result is convex. My function is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function y = cost( x )&#xA;y = min( inv_pos( -x ), max( square_pos( x + 3 )/4, 1 ) );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For those unfamiliar with CVX syntax, the function is min( 1/max(-x,0), max(max(x+3,0)^2/4, 1) ). CVX throws the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Disciplined convex programming error:&#xA;    Cannot perform the operation min( {convex}, {convex} )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any way to convince CVX that my function is indeed convex? Talking to it hasn't been working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7573" LastActivityDate="2014-03-08T02:51:00.027" Title="Custom CVX Functions - Overriding Errors" Tags="|matlab|convex-optimization|cvx|" AnswerCount="1" CommentCount="1" ClosedDate="2014-03-26T18:56:28.223" ContentLicense="CC BY-SA 3.0" />
  <row Id="11010" PostTypeId="2" ParentId="10980" CreationDate="2014-03-07T22:09:15.117" Score="2" Body="&lt;p&gt;I needed this for $y^T\Sigma^{-1}y$ calculation which can be solved by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ y^T\Sigma^{-1}y= y^TL^{-T}L^{-1}y $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Sigma = LL^T $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ = (y^TL^{-T})L^{-1}y $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ = (L^{-1}y)^TL^{-1}y $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  = |L^{-1}y|^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we need $L^{-1}y$ which is the linear solution of $Lx=y$. Based on the two previous answers, I used (first vanilla solution)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy.linalg as lin&#xA;Sigma = np.array([[10., 2.],[2., 5.]])&#xA;y = np.array([[1.],[2.]])&#xA;print np.dot(np.dot(y.T,lin.inv(Sigma)),y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now with solve_triangular&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.linalg as slin&#xA;L = lin.cholesky(Sigma)&#xA;x = slin.solve_triangular(L,y,lower=True)&#xA;print np.dot(x.T,x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both give 0.804.&lt;/p&gt;&#xA;" OwnerUserId="621" LastActivityDate="2014-03-07T22:09:15.117" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11012" PostTypeId="2" ParentId="11006" CreationDate="2014-03-08T00:19:31.673" Score="35" Body="&lt;p&gt;First, if your undergraduates are like ours and had no prior introduction to computers, expect to spend some time teaching them how to use basic stuff like using a proper editor (i.e., not MS Word), the command line, etc.&lt;/p&gt;&#xA;&lt;p&gt;I think the answer somewhat depends on where you set the focus of your course (or what you are required to teach). For example: How relevant are the internal workings of the computer? Do you need classes and other advanced OOP structures? Do you want to teach them how to produce efficient programs or are you happy if they produce working programs at all? Also, do not forget that you most probably will need capable tutors.&lt;/p&gt;&#xA;&lt;p&gt;But now something to advantages and disadvantages of the languages, I am familiar with. Note that this is mainly from my experience as a computational physicist and some of this may depend on the particular field, workgroup, university, etc.&lt;/p&gt;&#xA;&lt;h1&gt;Python&lt;/h1&gt;&#xA;&lt;p&gt;I generally recommend using Numpy from almost the very beginning and I am assuming it to be used in the following.&lt;/p&gt;&#xA;&lt;p&gt;Advantages:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It’s easy to learn and so is reading other people’s code (e.g., your example code, but also the students’ code for the tutors).&lt;/li&gt;&#xA;&lt;li&gt;Input and output (which should not be the focus of your course) can be fully covered by &lt;code&gt;print&lt;/code&gt;, Numpy’s &lt;code&gt;savetxt&lt;/code&gt; and &lt;code&gt;loadtxt&lt;/code&gt;, and maybe &lt;code&gt;sys.argv&lt;/code&gt;. It can be introduced on the fly and it does not eat much programming time.&lt;/li&gt;&#xA;&lt;li&gt;You do not need to deal with or only need to deal little with such details as number representation, memory management, data types. Thus it’s fast to program and you can focus on the actual algorithms.&lt;/li&gt;&#xA;&lt;li&gt;It‘s not a compiled language. This has two advantages: Students do not need to deal with a compiler and students can test stuff directly in the console without having to compile, restart and rerun the program. Relatedly, debugging is easier.&lt;/li&gt;&#xA;&lt;li&gt;There are easy-to-use libraries for almost everything.&lt;/li&gt;&#xA;&lt;li&gt;You do not need to learn additional script languages like shell scripts, Make, Gnuplot and so on – all this can be done from Python.&lt;/li&gt;&#xA;&lt;li&gt;There are a lot of good tutorials (for free).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Disadvantages:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It’s not compiled. Therefore Python programs may be drastically slower than compiled programs in some cases relevant to computational physics. In other cases, however, libraries (especially Numpy) can yield a comparable performance. Another way, to get good performances with Python is to write the relevant code snippets in another language like C¹. Obviously you need to learn this language for this, but this can be done later and your time learning Python is not wasted.&lt;/li&gt;&#xA;&lt;li&gt;It’s more difficult to teach such details as number representation, memory management, data types and their pitfalls, since they are somewhat obfuscated.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h1&gt;C/C++&lt;/h1&gt;&#xA;&lt;p&gt;Advantages:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It is compiled and therefore it’s easier to produce efficient code.&lt;/li&gt;&#xA;&lt;li&gt;You are directly dealing with number representation, memory management, data types and thus it is more intuitive to teach these – your students will get closer to what is really happening in their computer.&lt;/li&gt;&#xA;&lt;li&gt;There are libraries for basically everything but understanding and using a library takes some work.&lt;/li&gt;&#xA;&lt;li&gt;There is a relevant amount of existing code in C/C++ and thus students need to learn the language if they want to work with this code.&lt;/li&gt;&#xA;&lt;li&gt;If you already know C/C++, you can learn Python (for example) very fast.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Disadvantages:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It is compiled and your students have to deal with the compiler, the preprocessor, headers and so on. You would be surprised how much students fail at this step, even at the end of the semester.&lt;/li&gt;&#xA;&lt;li&gt;It is slower too learn and it takes longer to produce working code.&lt;/li&gt;&#xA;&lt;li&gt;Dealing with marginal stuff such as input and output takes some time as well in teaching as in programming. In C++, there is an extra syntax for input and output.&lt;/li&gt;&#xA;&lt;li&gt;Compiler and operating-system dependencies.&lt;/li&gt;&#xA;&lt;li&gt;You have to deal with the C/C++ confusion.&lt;/li&gt;&#xA;&lt;li&gt;Reading the code of other’s especially in C++ can be quite difficult due to the vast amount of syntax features.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The main advantages of C++ over C (Classes, templates) should not be relevant for your course and are only becoming relevant for larger projects. Therefore I would choose C of the two, since it is more concise.&lt;/p&gt;&#xA;&lt;h1&gt;Others&lt;/h1&gt;&#xA;&lt;p&gt;Some comments on the other languages:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Fortran:&lt;/strong&gt; This is still used by a lot of groups and there is a lot of legacy code, but you cannot get around dealing with the old standards and their huge limitations and pitfalls (a lot of people are still working with Fortran 77). Also, it will be much harder to find tutorials, help on the Internet and so on.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Matlab/Mathematica:&lt;/strong&gt; All the problems of proprietary software. Consider in particular that your students are likely to collaborate with people who do not have access to this software and the ensuing problems.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Cuda:&lt;/strong&gt; This is only relevant for certain problems, if performance matters. Also, after all I know, you do not want to learn programming this way.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;¹ Which is the standard workflow at least in our group.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-03-08T00:19:31.673" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11013" PostTypeId="2" ParentId="11009" CreationDate="2014-03-08T02:51:00.027" Score="2" Body="&lt;p&gt;No, there is not. CVX doesn't have the rules it does for arbitrary reasons. They are directly related to the approach it uses to convert problems to solvable form. So it is non-negotiable: if you cannot express a function according to the rules, then CVX cannot handle it, because it cannot convert it to solvable form.  Unless you can find a way to express this function according to the ruleset, CVX will not accept it.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2014-03-08T02:51:00.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11014" PostTypeId="2" ParentId="10998" CreationDate="2014-03-08T03:44:32.193" Score="5" Body="&lt;p&gt;A common strategy is to employ a damping strategy, i.e., to compute $\vec{w}^{\ast} = F \left( \vec{w}^{(n)} \right)$ and then set&#xA;$\vec{w}^{(n+1)} = \alpha \vec{w}^{(\ast)} + (1-\alpha)\vec{w}^{(n)}$ where $\alpha\in[0,1]$. You typically choose $\alpha$ in such a way that it minimizes $\|[\alpha \vec{w}^{(\ast)} + (1-\alpha)\vec{w}^{(n)}] - F(\alpha \vec{w}^{(\ast)} + (1-\alpha)\vec{w}^{(n)})\|$ in some sense. Damping strategies like this are often more robust than just taking a full step $\alpha=1$, which is what you are currently doing.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-08T03:44:32.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11015" PostTypeId="2" ParentId="11001" CreationDate="2014-03-08T04:17:02.490" Score="2" Body="&lt;p&gt;Goofy question, are you dealiasing in some fashion or is it unnecessary in your setup?  The issue sounds convective in some sense otherwise I'd expect your diffusion dominated limit to misbehave.  If not, you might see if using the skew-adjoint form of the convective operator improves things somewhat (see &lt;a href=&quot;https://github.com/RhysU/suzerain/blob/master/writeups/skewadjoint.tex&quot; rel=&quot;nofollow&quot;&gt;https://github.com/RhysU/suzerain/blob/master/writeups/skewadjoint.tex&lt;/a&gt; and the Zang1991 reference in &lt;a href=&quot;https://github.com/RhysU/suzerain/blob/master/writeups/references.bib&quot; rel=&quot;nofollow&quot;&gt;https://github.com/RhysU/suzerain/blob/master/writeups/references.bib&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have you tried cranking the time step through the floor?  If you're pushing the CFL too much with the numerics you described, bad convective things happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, have you tried throwing more resolution at it than you'd anticipate needing?  In 2D that's cheap, and insufficient resolution can cause &quot;grid noise&quot; to take down things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, for whatever reason, 2nd order schemes give me the willies on problems with convection.  I can't speak to the particular IMEX setup you've got, but I've personally had good luck with the 3rd-order-nonlinear/second-order-linear scheme documented in the appendix of Spalart, Moser, and Rogers 1991.  If you can already do an implicit diffusion solve, you're not far off.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2014-03-08T04:17:02.490" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11017" PostTypeId="1" CreationDate="2014-03-08T16:57:31.680" Score="1" ViewCount="11744" Body="&lt;p&gt;I'm trying to get a frequency plot, or PDF (probability density function) plot for my biometrics project in MATLAB.&#xA;I have two vectors &lt;code&gt;genuine_scores&lt;/code&gt; and &lt;code&gt;impostor_scores&lt;/code&gt;, and I need to have a graph like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x-axis : value of elements&#xA;y-axis : frequency of that value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the graph should look like this :&#xA; &lt;img src=&quot;https://i.stack.imgur.com/txWr6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried &lt;code&gt;probplot()&lt;/code&gt; but it gives the cumulative distrubition function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried &lt;code&gt;normpdf();&lt;/code&gt; but it does not make sense to me. Because I'm not interested in a mean value. I just need how frequent each of the value appears in the vector(s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help !&lt;/p&gt;&#xA;" OwnerUserId="7580" LastActivityDate="2014-11-05T02:48:53.383" Title="How to plot probability density function in MATLAB?" Tags="|matlab|probability|plotting|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11020" PostTypeId="1" CreationDate="2014-03-08T19:28:31.363" Score="2" ViewCount="2276" Body="&lt;p&gt;I am trying to solve an assignment on solving the Bogoliubov de Gennes equations self-consistently in Matlab. BdG equations in 1-Dimension are as follows:- &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(\begin{array}{cc}&#xA;-\frac{\hbar^{2}}{2m}\frac{\delta^{2}}{\delta z^{2}}-\mu+V\left(z\right) &amp;amp; \triangle(z)\\&#xA;\triangle(z) &amp;amp; \frac{\hbar^{2}}{2m}\frac{\delta^{2}}{\delta z^{2}}+\mu-V(z)&#xA;\end{array}\right)\left(\begin{array}{c}&#xA;u_{n}(z)\\&#xA;v_{n}(z)&#xA;\end{array}\right)= \epsilon_{n}\left(\begin{array}{c}&#xA;u_{n}(z)\\&#xA;v_{n}(z)&#xA;\end{array}\right)$$&#xA;along with the equations for gap function  $\triangle(z)$ and number density $n(z)$. &#xA;   $$\triangle(z)=U\sum_{n}\left(1-2f_{n,}\right)u_{n}(z)v_{n}^{\star}(z)$$&#xA;and &#xA;$$n(z)=2\sum_{n}|{u_{n}(z)}|^{2}f_{n}+|{v_{n}(z)}|^{2}\left(1-f_{n}\right).$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case of solving the BdG equations in Fourier space in Matlab for the case of a periodic potential and periodic gap function (assumed), we can take $$u_{n}(z)=\sum_{k}\exp\left[ikz\right]U_{n,k}, $$ &#xA;$$\triangle(z)=\sum_{K}\exp(iKz)T_{K},$$ &#xA;and &#xA;$$ V(z)=\sum_{K}\exp(iKz)P_{K} $$&#xA;where the sum is over the reciprocal lattice vectors $K$ leading to the number equation &#xA;$$N=2\sum_{n,k}\left[f_{n}|{U_{n,k}}|^{2}+\left(1-f_{n}\right)|{V_{n,k}}|^{2}\right]$$&#xA;with $ f_{n}$ as the Fermi distribution function $f_{n}=\frac{1}{\exp(\beta(\epsilon-\mu))+1}$. Solving the set of equations self-consistently for a fixed $N$, I am trying to get a value of chemical potential from the number equation each time after solving the eigenvector components $U_{n,k}$ and $V_{n,k}$, but due to the form of the exponentials in the number equation and sum over large number of them, I am unable to get a correct value of chemical potential out of them using Matlab routines as the root of the equation to put it back into the equations for eigenvector components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases, I get random values of chemical potential since the equation is more or less insoluble.  How can I avoid this error ? Is there a better way to numerically solve the BdG equations self-consistently ? I also want to do this assignment in real space avoiding finite size effects but started with the Fourier space case to avoid errors associated with discretizing the differential. Please guide and ask for any details you might need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is my MATLAB code to solve the equations in real space but the code does not work as fsolve does not find the mu value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://postimg.org/image/v7amx5vd9/full/&quot; rel=&quot;nofollow&quot;&gt;http://postimg.org/image/v7amx5vd9/full/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://postimg.org/image/do9pyyk7z/full/&quot; rel=&quot;nofollow&quot;&gt;http://postimg.org/image/do9pyyk7z/full/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7584" LastEditorUserId="7584" LastEditDate="2014-03-17T12:00:49.563" LastActivityDate="2015-09-07T04:41:20.350" Title="Self-consistent numerical solution of a set of equations" Tags="|matlab|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11021" PostTypeId="2" ParentId="11017" CreationDate="2014-03-08T19:43:42.457" Score="1" Body="&lt;p&gt;I think hist (histogram plot) or bar (bar graph) may do the job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.co.uk/help/matlab/ref/bar.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.co.uk/help/matlab/ref/bar.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.co.uk/help/matlab/ref/hist.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.co.uk/help/matlab/ref/hist.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7576" LastActivityDate="2014-03-08T19:43:42.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11022" PostTypeId="2" ParentId="10998" CreationDate="2014-03-08T20:13:30.553" Score="1" Body="&lt;p&gt;Look into Anderson mixing. It is used for self consistent quantum chemistry applications which are nonlinear. &lt;strong&gt;It is faster and more robust than simple damping.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The headline is that you keep a history of n previous iterations and calculate the residual for each, then perform a linear solve on the set of residuals using the approximation that the solution space is linear near the solution. This works very well, and far faster than the similar Broyden update method, which Eyert proved was equivalent to Anderson. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It fell down slightly when I used it, because my transform was so sensitive to small deltas even near the solution, i.e. the linear approximation did hold hold too well.&lt;/p&gt;&#xA;" OwnerUserId="1372" LastEditorUserId="1372" LastEditDate="2014-04-17T15:29:44.997" LastActivityDate="2014-04-17T15:29:44.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11023" PostTypeId="1" AcceptedAnswerId="11502" CreationDate="2014-03-09T13:01:58.967" Score="3" ViewCount="635" Body="&lt;p&gt;I am stuck with a problem about finding the condition number of an algorithm. I tried to find an example, but i couldn't. Can anybody help me, please? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given is $f(x)=\ln(x)$. We have the algorithm $f_{A}(x)=[\ln(x)](1+\varepsilon )$, where $\left | \varepsilon  \right |\leq 5\mathrm{eps}$ and $\mathrm{eps}$ is the machine epsilon. What could be said about the condition number of this algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, i know that $x\in \mathbb{R}_{m}^{M}$ (the set of the machine numbers) there exists an $x_{A}$ such that $f_{A}(x)=f(x_{A})$. I also know that the condition number of an algorithm we compute with $\mathrm{cond}_{A}(f)= \mathrm{inf} \left \{\frac{\left \| x- x_{A} \right \|}{\left \| x \right \| \mathrm{eps}}  \right \}$. The problem is how to find this $x_{A}$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody help me with this problem, please? Thank you in advance! &lt;/p&gt;&#xA;" OwnerUserId="7414" LastEditorUserId="1553" LastEditDate="2014-04-29T14:29:39.303" LastActivityDate="2014-04-29T15:42:44.660" Title="Condition number of an algorithm" Tags="|numerical-analysis|condition-number|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11024" PostTypeId="2" ParentId="10283" CreationDate="2014-03-09T17:09:26.557" Score="1" Body="&lt;p&gt;What is &quot;u&quot; in your advection-diffusion equation? If it represents the mass-fraction of a species then the &lt;strong&gt;total mass of that species&lt;/strong&gt; will likely vary over time. Also, in this case the advection-diffusion equation itself is the continuity equation of that species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also believe that the equation should have a negative sign in front of the first term in the parenthesis of the right hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conservation of total mass should be satisfied from the initial condition unless you plan on solving a pressure-velocity coupling as well.&lt;/p&gt;&#xA;" OwnerUserId="7592" LastActivityDate="2014-03-09T17:09:26.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11027" PostTypeId="1" CreationDate="2014-03-10T08:15:02.693" Score="1" ViewCount="301" Body="&lt;p&gt;The objective function is $L = D(B)$, here $D$ is a function of $B$ and $D$ is a convex function. We have a prior information of $B$: it is rank one and we can write $B$ into $B = pq^T$. So we can change the objective function into $L=D(p, q)$. Obviously, for $p$ and $q$, this problem is not a convex problem anymore and we can have many sets of $p$ and $q$. My question is that even though we have a lot of local minimum regarding $p$ and $q$, will we have a global minimum $B$ even though there are many $p$ and $q$? Thanks so much for your answering.   &lt;/p&gt;&#xA;" OwnerUserId="7600" LastActivityDate="2014-03-10T08:15:02.693" Title="Rank one constraint in optimization, which is nonconvex, but will we find the global maximum?" Tags="|optimization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11031" PostTypeId="1" CreationDate="2014-03-10T10:37:07.263" Score="5" ViewCount="1917" Body="&lt;p&gt;Is there a way of estimating Pi with the Buffon's method without assuming Pi known?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more precise: in a Monte Carlo simulation of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Buffon%27s_needle&quot; rel=&quot;noreferrer&quot;&gt;experiment invented by Buffon&lt;/a&gt; I would (ideally) generate 2 random numbers with uniform distribution within [0,1] and [0,Pi] respectively (the two numbers being the distance of the center of the needle from the border of the strip and the orientation of the needle). Unfortunately this is sort of cheating because one must already know the value of Pi in order to generate the angle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible reasonable -at least in principle- solution is to work around this problem by generating 2 points in a rectangle and then extract the probability distribution for needle's center and orientation from them (imposing the fixed length of the needle). However analytic calculations to get some usable formulas seem a complete mess to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some smarter way of doing that?&lt;/p&gt;&#xA;" OwnerDisplayName="cherbert" LastActivityDate="2014-03-10T15:09:52.110" Title="Monte Carlo estimation of Pi with Buffon's needle" Tags="|simulation|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11032" PostTypeId="2" ParentId="11031" CreationDate="2014-03-10T10:48:46.160" Score="1" Body="&lt;p&gt;Why don't you check it intersects the curve $x^2+y^2=1$, that's a circle and you didn't need to know $\pi$.&lt;/p&gt;&#xA;" OwnerUserId="7619" OwnerDisplayName="Sean D" LastActivityDate="2014-03-10T10:48:46.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11033" PostTypeId="2" ParentId="11031" CreationDate="2014-03-10T10:58:58.667" Score="1" Body="&lt;p&gt;For a given $(y,\theta)$ pair you really only need to decide whether $y\pm\frac{1}{2}sin\theta$ (the endpoints) crosses a line:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gaJHq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possiblity is to sample $sin(\theta)$ directly, either using a lookup table or its Taylor series to whatever precision you care for:&#xA;$$&#xA;sin(x) = x - \frac{x^3}{3!} + \frac{x^5}{5!} + \cdots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, you can avoid $\pi$ at the expense of using a non-uniform PDF. Draw a pair of numbers $(y, y')$ where $y'$ is now sampled from the sinusoidal distribution. Now check whether $y\pm\frac{1}{2}y'$ crosses a line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, there are approximations to $\pi$ based on Taylor series, so it' still a bit of a chicken-and-egg problem...&lt;/p&gt;&#xA;" OwnerUserId="4089" OwnerDisplayName="chase" LastActivityDate="2014-03-10T11:06:52.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11034" PostTypeId="2" ParentId="11031" CreationDate="2014-03-10T11:53:16.157" Score="2" Body="&lt;p&gt;Here's an alternative game (code in &lt;code&gt;R&lt;/code&gt;; hope it's understandable). Basically the same concept as SeanD's answer.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# from http://giventhedata.blogspot.com/2012/09/estimating-pi-with-r-via-mcs-dart-very.html&#xA;est.pi &amp;lt;- function(n){&#xA;&#xA;# drawing in  [0,1] x [0,1] covers one quarter of square and circle.&#xA;&#xA;# draw random numbers for the coordinates of the &quot;dart-hits&quot;&#xA;a &amp;lt;- runif(n,0,1)&#xA;b &amp;lt;- runif(n,0,1)&#xA;# use the pythagorean theorem&#xA;c &amp;lt;- sqrt((a^2) + (b^2) )&#xA;inside &amp;lt;- sum(c&amp;lt;1)&#xA;#outside &amp;lt;- n-inside&#xA;pi.est &amp;lt;- nside/n*4&#xA;&#xA; return(pi.est)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7606" OwnerDisplayName="Carl Witthoft" LastActivityDate="2014-03-10T11:53:16.157" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11035" PostTypeId="1" AcceptedAnswerId="11045" CreationDate="2014-03-10T17:09:58.347" Score="3" ViewCount="1179" Body="&lt;p&gt;I need to get a stability criterion for the numerical scheme for equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t}-\frac{\partial^2 u}{\partial x^2}-\frac{\partial^2 u}{\partial y^2}-\frac{\partial^2 u}{\partial z^2} -\alpha\frac{\partial u}{\partial x}-\beta\frac{\partial u}{\partial y}-\gamma\frac{\partial u}{\partial z}-f=0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scheme is explicit:&#xA;$$ \frac{u^{n+1}_{i,j,k}-u^{n}_{i,j,k}}{\Delta t}=\frac{u^{n}_{i+1,j,k}-2u^{n}_{i,j,k}+u^{n}_{i-1,j,k}}{\Delta x^2}+\frac{u^{n}_{i,j+1,k}-2u^{n}_{i,j,k}+u^{n}_{i,j-1,k}}{\Delta y^2}+$$&#xA;$$+\frac{u^{n}_{i,j,k+1}-2u^{n}_{i,j,k}+u^{n}_{i,j,k-1}}{\Delta z^2}+\alpha\frac{u^{n}_{i+1,j,k}-u^{n}_{i-1,j,k}}{2\Delta x }+$$&#xA;$$+\beta\frac{u^{n}_{i,j+1,k}-u^{n}_{i,j-1,k}}{2\Delta y }+\gamma\frac{u^{n}_{i,j,k+1}-u^{n}_{i,j,k-1}}{2\Delta z }+f_{i,j,k}.$$&#xA;After that I found an expression with a help of von Neumann analysis for amplification coefficient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho=1-4\Delta t\left(\frac{\sin^2{\theta /2}}{\Delta x^2}+\frac{\sin^2{\eta /2}}{\Delta y^2}+\frac{\sin^2{\zeta /2}}{\Delta z^2}\right)+i\Delta t\left(\alpha\frac{\sin{\theta}}{\Delta x}+\beta\frac{\sin{\eta}}{\Delta y}+\gamma\frac{\sin{\zeta }}{\Delta z}\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find a condition for stability criterion I must find a condition for $\Delta t,\Delta x,\Delta y,\Delta z$ so that for all $\theta,\eta,\zeta$ an inequality $|\rho|&amp;lt;1$ is true. Here I stuck.Could you help me with this stuff?&lt;/p&gt;&#xA;" OwnerUserId="4676" LastEditorUserId="4676" LastEditDate="2014-03-11T16:34:16.780" LastActivityDate="2014-03-11T16:53:31.393" Title="Von Neumann stability analysis in 3d" Tags="|pde|numerical-analysis|stability|cfl|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11037" PostTypeId="1" CreationDate="2014-03-11T06:56:03.133" Score="1" ViewCount="173" Body="&lt;p&gt;I want to discretize the LWR(Lighthill-Whitham-Richards)-v partial differential equation (as shown in 1) &#xA;$$&#xA; \frac{\partial u}{\partial t} + \frac{\partial R(u)}{\partial x} = 0 \quad \text{i.e.,} \quad \frac{\partial u}{\partial t} + \frac{dR}{du}\,\frac{\partial u}{\partial x} = 0 \quad \dots\dots \quad \text{(1)}&#xA;$$&#xA;In the equation $u(x,t) \equiv u_x^t$ is the speed at time $t$ at section $x$ and $R(u)$ is given by Greenshield’s speed-density relationship&#xA;$$&#xA;R(u) = u^2 - u\,u_f \quad \implies \quad \frac{dR}{du} = 2u - u_f =: a(u)&#xA;$$&#xA;where $u$ is the stream speed and $u_f$ is the free flow speed or the maximum speed in the section.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use an implicit finite difference scheme (backward time, back ward space as shown in (2) to discretize equation (1):&#xA;$$&#xA;  \frac{u_x^{t+1}-u_x^t}{\Delta t} + a(u_x^{t+1})\,\frac{u_x^{t+1}-u_{x-1}^{t+1}}{\Delta x} = 0 \quad \dots\dots \quad \text{(2)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In literature I could find solutions for explicit schemes alone like upwind scheme, downwind scheme, Godunov scheme etc. Can anyone suggest a good discretization scheme or some references in literature for the implicit case shown above in equation (2)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help!&lt;/p&gt;&#xA;" OwnerUserId="7614" LastEditorUserId="7626" LastEditDate="2014-03-13T00:32:59.627" LastActivityDate="2014-03-14T05:41:38.747" Title="Implicit finite difference scheme for LWR-v PDE" Tags="|finite-difference|implicit-methods|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11039" PostTypeId="1" AcceptedAnswerId="11040" CreationDate="2014-03-11T11:02:41.587" Score="0" ViewCount="310" Body="&lt;p&gt;I'm dealing with SVDLIBC library. In order to compute singular values I need to specify my matrix in the Harwell-Boeing format. On Internet I have found next example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dense Text Format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4 3&#xA;&#xA;2.3  0  4.2&#xA;&#xA;0   1.3 2.2&#xA;&#xA;3.8  0  0.5&#xA;&#xA;0    0   0 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sparse Text Harwell-Boeing Format:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SVDLIBC v. 1.32                   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;3             1             1             1             0&#xA;rra     &#xA;              4             3             6             0&#xA;&#xA;       (8i)            (8i)            (8e)            (8e)&#xA;&#xA;1 3 4 7 &#xA;&#xA;1 3 2 1 2 3 &#xA;&#xA;2.3 3.8 1.3 4.2 2.2 0.5 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In other libraries for sparse matrices I specified matrix as an array of non-zero values and their indices, but I can't understand how do they specify matrix in Harwell-Boeing format. Can anybody explain me how do they form two array of indices in an example above? I'm a newcomer in this field, so I'm sorry if the question seems too obvious. &lt;/p&gt;&#xA;" OwnerUserId="5278" LastActivityDate="2014-03-11T12:01:51.463" Title="Harwell-Boeing format for sparse matrices" Tags="|libraries|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11040" PostTypeId="2" ParentId="11039" CreationDate="2014-03-11T12:01:51.463" Score="1" Body="&lt;p&gt;The Harwell-Boeing Format is described &lt;a href=&quot;http://math.nist.gov/MatrixMarket/formats.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-03-11T12:01:51.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11041" PostTypeId="1" AcceptedAnswerId="11058" CreationDate="2014-03-11T14:12:44.727" Score="0" ViewCount="842" Body="&lt;p&gt;I want to compute an integral with the following code written in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;function f = LO_scheme2a(r,kk)&#xA;% Run the Prog by :  q = integral(@(r)LO_scheme2a(r,kk), 0, inf)&#xA;&#xA;%% System parameters &#xA;lambda_m= 10^(-2);&#xA;lambda_f= kk.*lambda_m; %can be changed based on scenario&#xA;mu= 10^-4; %user density (find the proper user density or scenario based) &#xA;Pdbm_m=43;&#xA;P_m=10.^(Pdbm_m./10);&#xA;Pdbm_f=20;&#xA;P_f=10.^(Pdbm_f./10);&#xA;gammadBm_m=0;&#xA;gamma_m=10.^(gammadBm_m./10);&#xA;gammadBm_f=0;&#xA;gamma_f=10.^(gammadBm_f./10);&#xA;alpha=4;&#xA;k=(P_f/P_m)^(1/alpha);% choose proper value&#xA;%k=.4&#xA;%% Busy Burst parameters &#xA;&#xA;Pdbm_req=0; % the optimum one should be calculated&#xA;P_req=10.^(Pdbm_req./10);&#xA;gammadBm_0=-40;&#xA;gamma_0=10.^(gammadBm_0./10);&#xA;%%&#xA;%% scheme 2&#xA;P_t=P_f;&#xA;C0=1; %By changing C_0 you can investigate the performance of system &#xA;Theta_0=C0.* P_f;&#xA;%%&#xA;D1=(P_req./gamma_0).^(1./alpha);&#xA;D2=(Theta_0./gamma_0./P_t).^(1./alpha).*r;&#xA;D=D2;&#xA;lambda_fnew=lambda_f.*exp(-pi.*mu.*D.^2);&#xA;%%&#xA;PM_u=lambda_m./(lambda_m+k.^2.*lambda_fnew);&#xA;PF_u=lambda_fnew.*k.^2./(lambda_m+k.^2.*lambda_fnew);&#xA;%%&#xA;fm_M=2.*pi.*r.*(lambda_m+k.^2.*lambda_fnew).*exp(-pi.*r.^2.*(lambda_m+k.^2.*lambda_fnew));&#xA;ff_F=2.*pi.*r.*(lambda_m./k.^2+lambda_fnew).*exp(-pi.*r.^2.*(lambda_m./k.^2+lambda_fnew));&#xA;%%&#xA;%% Interference from MBSs to MU an FU&#xA;A_i= 1;&#xA;S_i=gamma_m .*r.^alpha;&#xA;LIm_m=exp(-pi.*lambda_m.*(A_i.*r).^2.*rho1(gamma_m,alpha));&#xA;%LIm_m=exp(-pi.*lambda_m.*(A_i.*r).^2.*rho0(S_i./(A_i.*r).^alpha,alpha));&#xA;A_i= 1./k;&#xA;S_i=gamma_f .*r.^alpha *P_m/P_f;&#xA;LIf_m=exp(-pi.*lambda_m.*(A_i.*r).^2.*rho1(gamma_f.*P_m.*k.^alpha./P_f,alpha));&#xA;%LIf_m=exp(-pi.*lambda_m.*(A_i.*r).^2.*rho0(S_i./(A_i.*r).^alpha,alpha));&#xA;&#xA;%%&#xA;max_x= 10^5;&#xA;%% Expectation over h in lemmma 3 :  Interference from FAPs to MU an FU&#xA;S_i=gamma_f .*r.^alpha .*P_f./P_m;&#xA;z=(P_req/gamma_0).^(1/alpha);&#xA;%MIm_f=(exp(-h).*exp(-mu.*pi.*z^2.*h.^(2./alpha))).*((-((z.^2).*h^(2/alpha)).*(1-exp(-S_i.*z.^(-alpha))))+(S_i.*h).^(2./alpha).*real(gammainc(1-(2./alpha),S_i.*z.^(-alpha))));&#xA;for idx = 1:numel(r)&#xA;    MIm_f(idx)=  integral(@(h)exp(-h).*((exp(-mu.*pi.*z^2.*h.^(2./alpha))).*((-((z.^2).*h.^(2./alpha)).*(1-exp(-S_i.*z.^(-alpha))))+(S_i.*h).^(2./alpha).*real(gammainc(1-(2./alpha),S_i.*z.^(-alpha))))),0,max_x);&#xA;end&#xA;% % &#xA;S_i=gamma_f .*r.^alpha ;&#xA;z=(P_req./gamma_0).^(1/alpha);&#xA;&#xA;for idx = 1:numel(r)&#xA;    MIf_f(idx)=  integral(@(h)(exp(-h).*exp(-mu.*pi.*z^2.*h.^(2./alpha))).*((-((z.^2).*h.^(2./alpha)).*(1-exp(-S_i.*z.^(-alpha))))+(S_i.*h).^(2./alpha).*real(gammainc(1-(2./alpha),S_i.*z.^(-alpha)))),0,max_x);&#xA;end&#xA;%%&#xA;&#xA;f=PM_u.*LIm_m.*exp(-2.*pi.*lambda_f.*MIm_f).*fm_M+PF_u.*LIf_m.*exp(-2.*pi.*lambda_f.*MIf_f).*ff_F;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [ f_rho ] = rho1( x,alpha)&#xA;f_rho=x.^(2./alpha).*(pi./2-atan(x.^(-2./alpha)));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can run it for various values of kk, but for some kk it gives me an error. For example, when kk=10, I can run this program with the command &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; q = integral(@(r)LO_scheme2a(r,kk), 0, inf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but for kk=100, it gives me an error.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(Error using  .* Matrix dimensions must agree.) &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Would you please help me to find the source of this error?&lt;/p&gt;&#xA;" OwnerUserId="7618" LastEditorUserId="-1" LastEditDate="2014-03-12T11:25:33.797" LastActivityDate="2014-03-12T15:55:33.603" Title="Matlab: Error in integral function" Tags="|matlab|integral-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11042" PostTypeId="1" AcceptedAnswerId="11043" CreationDate="2014-02-07T12:31:16.560" Score="2" ViewCount="102" Body="&lt;p&gt;Unless I'm mistaken, deep neural networks are good for learning functions that are nonlinear in the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In such cases, the input set is linearly inseparable, so the optimisation problem that results from the approximation problem is not convex, so it cannot be globally optimised with local optimization. Support Vector Machines (try to) get around this by choosing features such that the projection of the input space into the feature space is linearly separable, and we have a convex optimisation problem once again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't suppose that a deep neural network always learns features that make the projection of the input set in the feature space linearly separable. So what's the point in it learning features?&lt;/p&gt;&#xA;" OwnerDisplayName="Alexandre Holden Daly" LastActivityDate="2014-03-11T16:47:08.370" Title="Neural Networks: what's the point of learning features that don't linearly separate?" Tags="|machine-learning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11043" PostTypeId="2" ParentId="11042" CreationDate="2014-02-07T13:33:24.800" Score="3" Body="&lt;p&gt;In theory, when you use a non-linear SVM you might be motivated by choosing a space in which the classes are linearly separable. However, nothing about recent SVMs requires separability: slack variables allow for misclassification of some examples at the expense of a better solution overall, as measured by the loss function and margin. Convexity of the optimisation problem has nothing to do with the linear separability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multi-layer neural nets allow for the same idea: the non-linearity allows for a more flexible separation, as measured by the loss function and regulariser. It allows you to capture (induced, not necessarily real) class boundaries that are not linear in the input space.&lt;/p&gt;&#xA;" OwnerDisplayName="Ben Allison" LastActivityDate="2014-02-07T13:33:24.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11044" PostTypeId="2" ParentId="11042" CreationDate="2014-02-07T16:14:58.200" Score="1" Body="&lt;p&gt;This depends on the activation function in neural network. A two-layer neural network with a single output node and Sigmoid activation function at decoding stage is equivalent to an SVM classifier with a Sigmoid kernel regarding the form of function to be learned. So the neural network, as @BenAllison mentioned, also gets you the boundaries that are non linear in the input space. On the other hand, the perception rule only works for the data that are linearly separable (which is somehow equivalent to SVM with linear kernel) since the perception is only a linear combination of input and weights (before the sign is applied), but without the Sigmoid decoding stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;a href=&quot;https://stackoverflow.com/questions/8963937/svm-and-neural-network&quot;&gt;the difference between a SVM and a Neural network is in how they decide what these parameters should be set to&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7251" OwnerDisplayName="lennon310" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-02-07T16:14:58.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11045" PostTypeId="2" ParentId="11035" CreationDate="2014-03-11T16:53:31.393" Score="3" Body="&lt;p&gt;Though your problem is in 3 dimensions, you can gain a lot of insight by considering the 1D case.  Let's omit the y and z terms and concentrate on x and t.  Then, your problem would be equivalent to the classic advection-diffusion equation with an explicit Forward-in-Time, Centered-in-Space (FTCS) finite difference method.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing the name of the problem and the method, it's fairly easy to find derivations online.  I personally found &lt;a href=&quot;http://www.personal.psu.edu/jhm/ME540/lectures/ConvDiff/ConvDiff.pdf&quot; rel=&quot;nofollow&quot;&gt;this stability analysis for the advection diffusion equation&lt;/a&gt; particularly insightful.  Following this approach, you should be able to extrapolate a method for the 3D case.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-03-11T16:53:31.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11046" PostTypeId="1" CreationDate="2014-03-11T17:21:05.353" Score="1" ViewCount="632" Body="&lt;p&gt;Within the manual, the effective/von Mises stress or Hill's potential for anisotropic bodies is calculated in Abaqus in cartesian rectangular coordinates as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sigma_{eff}=\sqrt{I_{1}^{2}-3I_{2}} \\&#xA;f({\sigma})_{Hill}=\sqrt{\big( F(\sigma_{22}-\sigma_{33})^{2}+ G(\sigma_{33}-\sigma_{11})^{2}+H(\sigma_{11}-\sigma_{22})^{2}+\\ 2L\sigma_{23}^{2}+2M\sigma_{31}^2+2N\sigma_{12}^{2}\big) }$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $I_{1}$ and $I_{2}$ are the first and second invariants of the stress tensor and F,G,H,L,M,N are constants you define. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas it is clear that in non-rectangular systems the effective stress is calculated in the coordinate system you work in, this is not clear when one applies Hill's potential.&#xA;I am trying to implement anisotropy with axial symmetry/axisymmetric model, but Hill's potential function is given in terms of rectangular coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does Abaqus carry out the transformation from rectangular to polar in non rectangular coordinate systems automatically? Do I have to supply the values that correspond to a transformed coordinate system? Or are do the ${1,2,3}$ coordinates correspond to ${r,z,\theta}$ in Hill's potential function? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help appreciated,&#xA;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="5317" LastActivityDate="2014-08-11T04:59:50.997" Title="How does Abaqus calculate Hill's function for non-rectangular coordinate systems?" Tags="|optimization|finite-element|computational-physics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11047" PostTypeId="1" CreationDate="2014-03-12T01:48:28.120" Score="4" ViewCount="67" Body="&lt;p&gt;I am working on analyzing a data set and I was wondering what would be the most statistically valid method of demonstrating that there is a strong spatial correlation between images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a data set with about 50 pairs of images of cancerous tissue samples. The first image in each pair shows the locations of gold nanoparticles, and the second image shows the locations of the blood vessels in the same tissue sample. By looking at the images it is easy to see that the locations of the nanoparticles match up with the blood vessels, but I would like to prove this statistically in the paper. This is an important point because it demonstrates that the nanoparticles bind specifically to the cancerous areas instead of the normal tissue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been looking at different statistics such as a simple linear correlation or something more sophisticated like the Moran's I statistic. However, I haven't found anything that would work well for correlation between images.&lt;/p&gt;&#xA;" OwnerUserId="7624" LastActivityDate="2014-03-25T23:04:55.667" Title="What statistical method should be used for spatial correlation between images?" Tags="|image-processing|spatial-data|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11048" PostTypeId="2" ParentId="11046" CreationDate="2014-03-12T04:58:47.943" Score="1" Body="&lt;p&gt;This is a tricky issue because the parameters used in Hill's criterion are determined on the basis of tests with respect to a particular set of orthogonal world coordinates (not necessarily Cartesian).  Therefore, you should express the stress in that coordinate system before you compute the value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As long as you are working in an orthogonal curvilinear coordinate system, the easiest way of dealing with the issue is to transform your stress components at each point so that they are with respect to the experimental coordinates system.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, you are limited to transverse isotropy if the model is axisymmetric and Hill's criterion takes the form&#xA;$$&#xA;f(\boldsymbol{\sigma}) = \sqrt{F\,[\sigma_{rr}^2 + \sigma_{\theta\theta}^2 + 2\sigma_{zz}^2 - 2(\sigma_{rr}+\sigma_{\theta\theta})\sigma_{zz}] + H\,(\sigma_{rr}-\sigma_{\theta\theta})^2 + 2\,L\,\sigma_{rz}^2 }&#xA;$$&#xA;where $1\rightarrow r, 2 \rightarrow \theta, 3 \rightarrow z$.  In some axisymmetric tests, the load is applied in the form of pressure in the $r-\theta$ plane and we have $\sigma_{rr} = \sigma_{\theta\theta}$.  For that situation, &#xA;$$&#xA;f(\boldsymbol{\sigma}) = \sqrt{2F\,(\sigma_{rr} - \sigma_{zz})^2  + 2\,L\,\sigma_{rz}^2 } \,.&#xA;$$&#xA;A good description of the treatment of axisymmetric problems in FEA can be found &lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch02.d/AFEM.Ch02.pdf&quot; rel=&quot;nofollow&quot; title=&quot;here&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Update:  The previous axisymmetric expression was wrong and has been corrected.)&lt;/p&gt;&#xA;" OwnerUserId="7625" LastEditorUserId="7626" LastEditDate="2014-03-14T02:12:04.123" LastActivityDate="2014-03-14T02:12:04.123" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11049" PostTypeId="1" CreationDate="2014-03-12T08:42:06.573" Score="8" ViewCount="203" Body="&lt;p&gt;I need to calculate the integral&#xA;$$I = \int_0^R f(r)J_n\left(\frac{z_{nm}r}{R}\right)rdr$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $J_n$ is the $n^{\mathrm{th}}$ order Bessel functions of the first kind, $z_{nm}$ is its $m^{\mathrm{th}}$ zero and $f(r)$ is a real function which is somewhat similar to $J_n$ (but not the same, it is quite complicated and usually involves terms with $J_n^2$ and sometimes $\exp(J_n)$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As $f(r)$ is extremely expensive and this integral must be evaluated very many times, I am looking for the best (very fast, but still reasonably accurate) numerical method to solve it. Currently, I am using the trapezoidal rule with 11 points. But I am investigating other methods, such as Clenshaw–Curtis and Gauss–Kronrod (with low order).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am wondering if there is a method particularly suited to such integrals, especially given that it is similar to those required to compute Hankel transforms.&lt;/p&gt;&#xA;" OwnerUserId="7627" LastEditorUserId="20688" LastEditDate="2018-07-07T18:47:25.377" LastActivityDate="2019-11-14T16:55:26.257" Title="Radial integration of expensive function with Bessel weights" Tags="|quadrature|integration|special-functions|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11052" PostTypeId="1" CreationDate="2014-03-12T11:11:25.333" Score="1" ViewCount="135" Body="&lt;p&gt;The parallel FFTW3 distributes different portions of a large data to different processors, so that each processor obtains and manipulates only a small fraction of the large size data that wants to be fourier transformed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering, at the end of the transformation, as the transformed data are locating at different processors, is there any built-in routine in FFTW3 to gather all these small portions from different processors to recover the entire data set, for example, at the root processor. Hope this question is clear to you guys and I am looking forward to your suggestions. &lt;/p&gt;&#xA;" OwnerUserId="7630" LastActivityDate="2014-03-13T13:08:34.843" Title="gather data from processors in FFTW3 MPI" Tags="|mpi|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11054" PostTypeId="2" ParentId="11052" CreationDate="2014-03-12T11:57:01.393" Score="1" Body="&lt;p&gt;I don't think that there's an FFTW3 routine that does this, but there's always &lt;code&gt;MPI_Gather()&lt;/code&gt; in the MPI library. Of course, it's possible that you may not be able to gather all the data to a single processor because it is too big to fit there.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-03-12T11:57:01.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11055" PostTypeId="1" AcceptedAnswerId="11150" CreationDate="2014-03-12T12:54:13.957" Score="3" ViewCount="161" Body="&lt;p&gt;I am trying to calculate the numbers $n$ for which the $n$-th Fibonacci number $F_n$ is a multiple of $n$; that is fib(n)%n==0. &#xA;Here is the best PARI code I could come up with (for the counting function).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fibmod(n)={return(lift(((Mod([1,1;1,0],n))^n)[1,2]));};&#xA;S(n)=sum(k=1,floor(n),if(fibmod(k),0,1));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this, I can go up to 10E9 in a reasonable time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you think of a more efficient implementation? &#xA;For instance, I guess writing it in C, or adding multithreading, could give a significant speed improvement.&lt;/p&gt;&#xA;" OwnerDisplayName="user7632" LastActivityDate="2014-03-22T17:11:40.980" Title="Fibonacci, variation on the theme" Tags="|computer-arithmetic|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11057" PostTypeId="2" ParentId="6943" CreationDate="2014-03-12T15:20:54.400" Score="2" Body="&lt;p&gt;That's what I usually do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define streamline function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Psi = \left[&#xA;\begin{array}{c}&#xA;\psi_x\\&#xA;\psi_y\\&#xA;\psi_z\\&#xA;\end{array}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the velocity equals:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{u} = \nabla\times\Psi = &#xA;\left[&#xA;\begin{array}{c}&#xA;u_x =\partial_y \psi_z - \partial_z \psi_y\\&#xA;u_y =\partial_z \psi_x - \partial_x \psi_z\\&#xA;u_z =\partial_x \psi_y - \partial_y \psi_x\\&#xA;\end{array}&#xA;\right].&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can pick any reasonable zero-averaged pressure and construct a forcing term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I post a SymPy example code for $\Omega = [0,1]^3$ and homogeneous boundary conditions, enjoy:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; from sympy import *&#xA;&#xA; x,y,z = symbols('x y z')&#xA;&#xA; X = Matrix([[x],[y],[z]])&#xA;&#xA; psi = zeros(3,1)&#xA; psi[0,0] = sin(2*pi*x)*y**2*(1-y)**2*z**2*(1-z)**2&#xA; psi[2,0] = x**2*(1-x)**2*y**2*(1-y)**2*sin(2*pi*z)&#xA;&#xA; curl_psi = zeros(3,1)&#xA; curl_psi[0] = diff(psi[2],X[1]) - diff(psi[1],X[2])&#xA; curl_psi[1] = diff(psi[0],X[2]) - diff(psi[2],X[0])&#xA; curl_psi[2] = diff(psi[1],X[0]) - diff(psi[0],X[1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6574" LastEditorUserId="6574" LastEditDate="2014-03-24T17:32:31.350" LastActivityDate="2014-03-24T17:32:31.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11058" PostTypeId="2" ParentId="11041" CreationDate="2014-03-12T15:43:56.577" Score="1" Body="&lt;p&gt;When you call &lt;code&gt;q = integral(@(r)LO_scheme2a(r,kk), 0, inf)&lt;/code&gt;, the &lt;code&gt;integral function chooses&lt;/code&gt;r`, which is a vector that discretizes the integration bounds. The &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/integral.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For scalar-valued problems, the function y = fun(x) must accept a vector argument, x, and return a vector result, y. This generally means that fun must use array operators instead of matrix operators. For example, use .* (times) rather than * (mimes).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Your error(s) occur where you call &lt;code&gt;integral&lt;/code&gt; again from within your code. At this point &lt;code&gt;r&lt;/code&gt; is a vector (a vector that can change size) and you don't have a scalar-valued problem. You have tried to deal with this with your two &lt;code&gt;for&lt;/code&gt; loops. However, the right hand sides of both loops don't depend on your index variable &lt;code&gt;idx&lt;/code&gt;, i.e., &lt;code&gt;r(idx)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you don't even need &lt;code&gt;for&lt;/code&gt; loops as the documentation above continues:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you set the 'ArrayValued' option to true, fun must accept a scalar and return an array of fixed size.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thus, using the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/integral.html#inputarg_ArrayValued&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;'ArrayValued'&lt;/code&gt;&lt;/a&gt; option, your &lt;code&gt;for&lt;/code&gt; loops can be replaced with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;fun = @(h,S_i,z)exp(-h).*exp(-mu*pi*z^2*h.^(2/alpha)) ...&#xA;                       .*(-z^2*h.^(2/alpha).*(1-exp(-S_i*z^-alpha)) ...&#xA;                       +(S_i.*h).^(2/alpha).*real(gammainc(1-2/alpha,S_i*z^-alpha)));&#xA;MIm_f = integral(@(h)fun(h,S_i,z),0,max_x,'ArrayValued',true);&#xA;&#xA;S_i = gamma_f.*r.^alpha ;&#xA;z = (P_req./gamma_0).^(1/alpha);&#xA;&#xA;MIf_f = integral(@(h)fun(h,S_i,z),0,max_x,'ArrayValued',true);&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've also cleaned up your anonymous function and added two extra parameters so you can re-use it (&lt;a href=&quot;http://www.mathworks.com/help/optim/ug/passing-extra-parameters.html#brhkghv-9&quot; rel=&quot;nofollow&quot;&gt;see here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="4474" LastEditDate="2014-03-12T15:55:33.603" LastActivityDate="2014-03-12T15:55:33.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11060" PostTypeId="1" CreationDate="2014-03-12T19:36:54.690" Score="2" ViewCount="114" Body="&lt;p&gt;I'm using modal decomposition to predict the steady state response of a beam structure to harmonic loading. The structure itself is very lightly damped, but we know from experiments that the aerodynamic drag force as it moves through the air is very significant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the mass and stiffness matrices for the beam ($\boldsymbol{[M]}$ and $\boldsymbol{[K]}$) and so I am able to obtain the eigenvectors and eigenvalues ($\boldsymbol{[V]}$ and $\boldsymbol{[D]}$). The eigenvalues yields a vector containing the natural frequencies of the system (the square root of the diagonal entries of $\boldsymbol{[D]}$, I'll call it $\boldsymbol{\omega}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system is excited by a force vector in the global reference frame $\boldsymbol{F_0}$ at an excitation frequency $\omega_F$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a rotation matrix which rotates the beams from the local to the global reference frame $\boldsymbol{[R]}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have successfully used the procedure to calculate the steady state response without aerodynamic damping:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Get eigenvalues and eigenvectors $\boldsymbol{[V]}$ and $\boldsymbol{[D]}$&lt;/li&gt;&#xA;&lt;li&gt;Get generalised force vector $\boldsymbol{Q}=\boldsymbol{[V]}^T \boldsymbol{F_0}$&lt;/li&gt;&#xA;&lt;li&gt;Calculate modal participation factor vector $\boldsymbol{q}$. The individual elements are calculated as follows:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$q_i=\frac{Q_i}{{\omega_i}^2}\frac{1}{\sqrt{[(1-(\omega_F/\omega_i)^2)^2+(2\zeta_i\omega_F/\omega_i)^2]}}$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Calculate response from $\boldsymbol{x}=\boldsymbol{[V]q}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, this drastically overpredicts the amplitude of the oscillations because there is no aerodynamic damping. I've found a few examples for single d.o.f. systems in which the energy lost from the system is calculated (by integrating the drag force x the velocity). The integration results in the following equation for energy loss:&#xA;$$\Delta E=\frac{8}{3}\frac{1}{2}\rho A C_D \omega^2 X^3$$&#xA;This can then be converted to an equivalent damping coefficient as follows:&#xA;$$C_{eq}=\frac{\Delta E}{\pi \omega X^2}$$&#xA;Which can be converted to a damping coefficient by doing:&#xA;$$\zeta=\frac{C_{eq}}{2 \omega_n}$$&#xA;I've modelled a simple spring-mass system with air resistance and it works fine (I have to iterate to find the final position because the amplitude calculation requires knowledge of the damping which requires knowledge of the amplitude so there is a circular reference, but that's no biggy!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please explain the equivalent for a multi-d.o.f. system? My initial forays have been unsuccessful, with much higher drag coefficients than would really occur.&lt;/p&gt;&#xA;" OwnerUserId="7637" LastActivityDate="2014-03-12T19:36:54.690" Title="Modal analysis of structure with aerodynamic damping" Tags="|finite-element|iterative-method|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11064" PostTypeId="2" ParentId="10997" CreationDate="2014-03-13T01:03:15.490" Score="1" Body="&lt;p&gt;Some n-body simulators I aware of are:&#xA;1. Gadget: &lt;a href=&quot;http://www.mpa-garching.mpg.de/gadget/&quot; rel=&quot;nofollow&quot;&gt;http://www.mpa-garching.mpg.de/gadget/&lt;/a&gt;&#xA;2. nbody6: &lt;a href=&quot;http://www.ast.cam.ac.uk/~sverre/web/pages/nbody.htm&quot; rel=&quot;nofollow&quot;&gt;http://www.ast.cam.ac.uk/~sverre/web/pages/nbody.htm&lt;/a&gt;&#xA;3. swift: &lt;a href=&quot;http://www.boulder.swri.edu/~hal/swift.html&quot; rel=&quot;nofollow&quot;&gt;http://www.boulder.swri.edu/~hal/swift.html&lt;/a&gt; (but this may be hard to install)&lt;/p&gt;&#xA;" OwnerUserId="1437" LastActivityDate="2014-03-13T01:03:15.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11067" PostTypeId="1" AcceptedAnswerId="11077" CreationDate="2014-03-13T16:34:45.620" Score="9" ViewCount="1448" Body="&lt;p&gt;I've always heard that easy parallelization was one of the advantages of DG methods, but I don't really see why any of those reasons don't also apply to continuous Galerkin.&lt;/p&gt;&#xA;" OwnerUserId="3309" LastActivityDate="2014-03-14T04:30:21.147" Title="Is discontinuous Galerkin really any more parallelizable than continuous Galerkin?" Tags="|finite-element|parallel-computing|discontinuous-galerkin|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11068" PostTypeId="1" AcceptedAnswerId="11072" CreationDate="2014-03-13T17:48:59.993" Score="2" ViewCount="69" Body="&lt;p&gt;I'm working on this application where &#xA;I get the spectral decomposition (O,D) of a matrix A for free (A is of full rank) but not A itself, and I need to only recover the last column of A. I was wondering if there was a more efficient way to do this than reconstructing the whole of A first?&lt;/p&gt;&#xA;" OwnerUserId="479" LastEditorUserId="479" LastEditDate="2014-03-13T17:54:45.783" LastActivityDate="2014-03-13T20:04:18.480" Title="last column of SPD matrix given it's spectral decomposition" Tags="|eigensystem|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11069" PostTypeId="1" CreationDate="2014-03-13T18:42:49.197" Score="0" ViewCount="2340" Body="&lt;p&gt;I have installed OpenCV and IntraFace, now the code can detect landmarks on the face, like this :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/oBBzQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now how can I convert this to a smile detector? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the landmark points as an array, so I should probably first find which ones belong to the mouth. How can I do that? Are there any tutorials for IntraFace? I could not find any.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated. Thanks !&lt;/p&gt;&#xA;" OwnerUserId="7580" LastActivityDate="2014-12-11T13:01:33.393" Title="How to detect smiles using OpenCV and IntraFace?" Tags="|machine-learning|computer-vision|pattern-recognition|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11070" PostTypeId="2" ParentId="11067" CreationDate="2014-03-13T18:59:54.603" Score="5" Body="&lt;p&gt;Just as it is the case with most general statements about numerical schemes, the answer depends on the exact circumstances you're looking at. First of all, the advantages of DG concerning parallelization mainly pay off in case of &lt;strong&gt;explicit&lt;/strong&gt; time-integration schemes because of&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The cell-local mass matrix of DG schemes (so you don't have to apply the inverse of the mass matrix globally)&lt;/li&gt;&#xA;&lt;li&gt;A favorable ratio of CPU-local work (volume integrals) to communication related work (edge integrals), especially for higher orders&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While these statements apply to generic DG discretizations, &lt;em&gt;real&lt;/em&gt; HPC applications (say, using several thousand processors) require some more effort concerning the parallelization strategy in order maintain a good scaling. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S004579301200093X?v=s5&quot; rel=&quot;noreferrer&quot;&gt;This&lt;/a&gt; paper shows, for example, how one can achieve almost perfect scaling up to &lt;strong&gt;one cell per processor&lt;/strong&gt;. This is certainly something you cannot expect from a continuous FEM, but as mentioned before, implicit schemes are totally different thing.&lt;/p&gt;&#xA;" OwnerUserId="3738" LastActivityDate="2014-03-13T18:59:54.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11071" PostTypeId="2" ParentId="11069" CreationDate="2014-03-13T19:46:01.240" Score="2" Body="&lt;p&gt;IntraFace has an expression detector built in. However, if you would like to do a custom smile detector, then what you should do is to convert your spatial facial fiducial points to some kind of a feature vector, which could be input to a non-linear classifier such as SVM or MLP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A trivial way to do this would be to use angular relations (if the face is fronto-parallel) or else you could use projective invariants such as cross ratio to find out the geometric relationships. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two types of classifiers can be trained :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two-Class: You would first prepare a training set of smiling and non-smiling people. Then train your learning machine with this input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Novelty-Detection : If you use something like GMM or a variant of SVM you could model only the smile features (build a hyper-envolope around those features). And then during the runtime, you could query if the test feature falls into this hyper-envelope or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, regarding the mouth part : You are right that it can help you, and IntraFace would have that built in. In the worst scenario, you might not need to separate anything. If you have a good training set you could as well run a dimensionality reduction on your features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course there are more advanced methods published to accomplish what you like to do. Just as an example &lt;a href=&quot;http://research.microsoft.com/en-us/um/people/zhang/Papers/IJPRAI.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2014-12-11T13:01:33.393" LastActivityDate="2014-12-11T13:01:33.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11072" PostTypeId="2" ParentId="11068" CreationDate="2014-03-13T20:04:18.480" Score="6" Body="&lt;p&gt;Yes, for matrices A, B, the last column of matrix product A*B can be written as A*(the last column of B).  You can use this fact to get the last column of the reconstruction using only O(N^2).&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-03-13T20:04:18.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11073" PostTypeId="2" ParentId="11067" CreationDate="2014-03-13T22:15:46.557" Score="1" Body="&lt;p&gt;When assembling a stiffness matrix, the data stored in an element in continuous (nodal) FEM has to be communicated to all its nodal neighbours. In contrast, DGFEM requires the element data to be communicated to all its face neighbours. In 1D nodal and face neighbours are identical, but in 3D the difference can be quite large: For a regular hexahedral mesh, there are 26 nodal neighbours but only 6 face neighbours. For irregular meshes with many high-valence vertices, the situation gets worse for CG, while it stays the same for DG.&lt;/p&gt;&#xA;" OwnerUserId="5378" LastActivityDate="2014-03-13T22:15:46.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11075" PostTypeId="2" ParentId="11067" CreationDate="2014-03-14T00:22:33.833" Score="8" Body="&lt;p&gt;From my many years writing FEM software, I believe that the statement that DG schemes are better suited to parallelization than CG schemes is apocryphal. It is frequently used in introductions of DG papers as a justification for DG methods, but I have never seen it substantiated with a reference that actually investigated the question. It is similar to every NSF proposal on a number theory project referencing &quot;cryptography&quot; as an area of broader impact, a statement that in this generality is also never substantiated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, I believe that with the one exception of explicit time stepping schemes and, possibly, problems where you have to invert a mass matrix, DG schemes are no better or worse than CG schemes if one investigated the cost of communication involved in either. I do mean this to be in a practical sense: sure, one may have to communicate less data, but I would imagine the difference in wallclock time to be negligible to all the other operations programs do on this data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, I would be delighted if anyone took up the challenge to prove me wrong!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-14T00:22:33.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11077" PostTypeId="2" ParentId="11067" CreationDate="2014-03-14T02:06:13.907" Score="9" Body="&lt;p&gt;One reason DG methods may receive more attention as a parallel method is that it is readily seen that the method is inherently local to an element. The coupling in DG methods is weak, as it only occurs through adjacent edges (or faces in 3d). So, for triangles or quads DG will communicate to three or four processers at most, respectively. Whereas CG methods will include element corners, thus the valence of an element's corner becomes important. Depending on the mesh generator used, this valence can be eight processers (possibly higher). So the cost of assembling the time derivative can be considered &quot;higher&quot; for CG methods. This is paricularly of concern for spectral methods, where the volume of information to be communicated can be quite large (and latency hiding can become more difficult as the size of each partition shrinks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this extra cost for CG to assemble it's time derivative could be compensated for by a different load balancing strategy. Different mesh partitioning strategies (I'm most farmiliar with METIS) allow the user to balance the load through various metrics, e.g., ensuring each partitiom has approximately the same number of elements or limiting the amount of communication between partitions. I feel a reason the colloquisium of DG being readily parallelizable is that a naive splitting of the problem into equal pieces can give a very efficient parallel implementation, even presenting super linear speedup in certain cases due to cache effects (see for example &lt;a href=&quot;http://www.cs.odu.edu/~keyes/papers/pcfd99_dgm.pdf&quot;&gt;Baggag et. al.&lt;/a&gt; or &lt;a href=&quot;http://link.springer.com/chapter/10.1007/978-3-642-35893-7_4&quot;&gt;Altmann et. al.&lt;/a&gt;). Whereas CG may require a more clever partitioning method. So it may be the case that changing spacial discretizations from DG to CG, say, would require one to also reconsider how to divide the mesh into subproblems.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastEditorUserId="1057" LastEditDate="2014-03-14T02:13:38.807" LastActivityDate="2014-03-14T02:13:38.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11078" PostTypeId="1" AcceptedAnswerId="11080" CreationDate="2014-03-14T02:59:40.333" Score="1" ViewCount="58" Body="&lt;p&gt;I have a function called f(x) which is convex and I can have access to its first order derivative , my objective function is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ J(\bf{x}) = f(\bf{x}) + \lambda |\bf{x}|_0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ 	\bigtriangledown f(\bf{x}) \space is \space available$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please provide a list of state of the art methods in the literature that can solve this sparse regularization problem?&lt;/p&gt;&#xA;" OwnerUserId="7645" LastActivityDate="2014-03-14T04:53:06.703" Title="an over view of sparsity promoting inversion techniques" Tags="|sparse-matrix|inverse-problem|" AnswerCount="1" CommentCount="1" ClosedDate="2014-04-30T01:22:41.747" ContentLicense="CC BY-SA 3.0" />
  <row Id="11079" PostTypeId="2" ParentId="11067" CreationDate="2014-03-14T04:30:21.147" Score="1" Body="&lt;p&gt;DG for hyperbolic PDE can be used as a replacement for finite volume schemes. In finite volume as in finite difference, when you increase the order of the scheme, your stencil increases. This makes parallelization more difficult, since for each scheme order, you have different stencil. At partition boundaries, you must ensure that all required cells from the neigbouring partition for the particular order of the scheme are available. But with DG, no matter what order the scheme is, each cell communicates only with its face neighbours. So compared between finite volume/difference and DG, one can say the DG is easier to parallelize.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2014-03-14T04:30:21.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11080" PostTypeId="2" ParentId="11078" CreationDate="2014-03-14T04:53:06.703" Score="3" Body="&lt;p&gt;The short answer is that in general this problem is intractable (NP-Hard) with the $\| x \|_{0}$ regularization.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are you willing to consider minimizing &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f(x) + \lambda \| x \|_{1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;instead?  There are many methods for the 1-norm regularized problem.  There are also lots of theoretical results that give conditions under which 1-norm minimization produces solutions that are very close to the 0-norm regularized solution.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-03-14T04:53:06.703" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11081" PostTypeId="2" ParentId="11037" CreationDate="2014-03-14T05:41:38.747" Score="1" Body="&lt;p&gt;Nobody solves this implicitly because it would be very inefficient.  If you step with large CFL number, you will smear the shocks and get very poor accuracy.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-03-14T05:41:38.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11083" PostTypeId="1" CreationDate="2014-03-14T10:22:36.547" Score="3" ViewCount="539" Body="&lt;p&gt;All the papers I have seen on DG methods for hyperbolic conservation laws together with limiters to compute discontinuous solutions make use of Taylor polynomial basis (Pk basis) or Legendre polynomials. In these cases, limiters have been developed. But if I want to use nodal basis on quadrilaterals/rectangles, then I have not seen any papers which develop limiters for this case. My attempts at writing a nodal DG code and extending the existing limiters in a &quot;natural&quot; way does not lead to accurate solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am looking for any existing literature that uses nodal DG (Qk basis) on quadrilaterals and limiters to compute discontinuous solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&#xA;praveen&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2014-03-17T02:41:46.160" Title="Nodal DG method and limiters for hyperbolic conservation laws" Tags="|discontinuous-galerkin|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11084" PostTypeId="1" CreationDate="2014-03-14T10:36:21.910" Score="0" ViewCount="156" Body="&lt;p&gt;I am working on a project involving some pattern recognition. For this I need to find the &lt;strong&gt;maximum value&lt;/strong&gt; in a huge multidimensional dataset. For example I have a discrete &lt;strong&gt;5-dimensional space containing 10^10 data points&lt;/strong&gt;. Of course I can just do an exhaustive search but time is of the essence so I am searching for an time (and memory) efficient algorithm that can help me with this. Probably something like gradient descent. FYI, the project is done is Java. &lt;/p&gt;&#xA;" OwnerUserId="7650" LastActivityDate="2014-03-14T10:57:24.307" Title="Finding maximum value in huge dataset" Tags="|optimization|algorithms|data-analysis|data-sets|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11085" PostTypeId="2" ParentId="11084" CreationDate="2014-03-14T10:57:24.307" Score="1" Body="&lt;p&gt;Without knowing more about your data, it is not possible to do better than the exhaustive search. This would be memory efficient, but linear in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data set does not contain local minima, you could simply follow the steepest descent. This still has a worst case linear time however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possibly better solution would be the &lt;a href=&quot;http://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method&quot; rel=&quot;nofollow&quot;&gt;downhill-simplex algorithm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-03-14T10:57:24.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11086" PostTypeId="1" AcceptedAnswerId="11154" CreationDate="2014-03-14T10:58:07.213" Score="7" ViewCount="2936" Body="&lt;p&gt;I would like to perform the numerical integration of an integral of the form&#xA;$$ \int_{-\infty}^\infty e^{i \omega 0+} G(i \omega, \mathbf{v}) d \omega ,$$&#xA;or, using the symmetry $G(i\omega)^* = G(-i \omega)$,&#xA;$$ \int_0^\infty \Re(e^{i \omega 0+} G(i \omega, \mathbf{v}) )  d\omega ,$$&#xA;where $\mathbf{v}$ is a vector of additional parameters.&#xA;$G(i \omega)$ is a rational function and behaves for large $\omega$ as $\omega^5/\omega^6 = 1/\omega$ and it can not be integrated analytically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I split the integral, i.e. $\int_0^\infty = \int_0^a + \int_a^\infty$, where $a$ is a large positive constant, such that the approximation $G(i \omega) \approx 1/i \omega$ holds for $\omega \geq a$. The infinite integral gives an additive constant. The convergence factor in $\int_0^a$ can now be dropped because it is a finite integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now there is this problem: I use the quad() function from SciPy to evaluate the finite integral for set of different $\mathbf{v}$. If I set the upper limit in quad() as $a=10^8, 10^9$ or $a = \infty$ then it gives consistent and correct results. But for $a = 10^{10} - 10^{14}$, the results are rubbish. For $a = \infty$ quad() gives a warning message that the integrand may be slowly convergent and that there is a roundoff error detected in the extrapolation table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So obviously there is a numerical problem and I don't know what quad() exactly does, when one enters $\infty$ as a limit. From the source code it seems that quad() maps the big finite interval onto $[0,1]$, rescales the integrand and uses a Gauss-Kronrod method for integration, but it is not clear (to me) how big this scaling respectively the finite interval is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other reliable methods to perform a numerical integration over large intervals? Is a method where one divides the big interval into a number of subintervals with fixed width and subsequent integration (e.g. with Simpson's rule) recommended?&lt;/p&gt;&#xA;" OwnerUserId="7649" LastEditorUserId="7649" LastEditDate="2014-03-14T16:13:51.363" LastActivityDate="2015-03-30T17:24:05.803" Title="Numerical Integration with Convergence Factor with SciPy: Problem with Improper Integral" Tags="|quadrature|scipy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11087" PostTypeId="1" AcceptedAnswerId="11089" CreationDate="2014-03-14T12:14:06.930" Score="0" ViewCount="557" Body="&lt;p&gt;I need the Truncated SVD implementation in java.&#xA;I need to pass a matrix of doubles and an integer value representing the rank where to filter out noise. In output i need a filtered matrix of doubles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="7651" LastActivityDate="2014-04-13T16:21:38.527" Title="Truncated SVD implementation in Java" Tags="|linear-algebra|svd|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11088" PostTypeId="1" CreationDate="2014-03-14T14:10:27.717" Score="1" ViewCount="244" Body="&lt;p&gt;I am looking for finding the velocity distribution using Principle of Maximum entropy when applied to Tsallis entropy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tsallis entropy is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;S_{T} = \frac{1}{q-1}\left(1-\int{f(p)^qdp}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following constraints:&#xA;$$&#xA;\int{f(p)dp}=1\\&#xA;\int{pf(p)dp}=0\\&#xA;\int{p^2f(p)dp}=1\\&#xA;\int{p^3f(p)dp}=0\\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using these constraints and incorporating Lagrange multipliers, the equation of ${S_{T}}$ becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;S_{T} = \frac{1}{q-1}\left(1-\int{f(p)^qdp}\right) + \lambda_0 \left(\int{f(p)dp}-1\right) + \lambda_1 \left(\int{pf(p)dp}\right) + \lambda_2 \left(\int{p^2f(p)dp-1}\right)+ \lambda_3 \left(\int{p^3f(p)dp}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking variation of ${S_{T}}$ with respect to ${f(p)}$ and equating it to 0, the distribution function becomes:&#xA;$$&#xA;f(p)=\left[\left(\frac{q-1}{q}\right)\left(\lambda_0+\lambda_1p+\lambda_2p^2+\lambda_3p^3\right)\right]^{\frac{1}{q-1}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entire problem then boils down to solving for the Lagrange multiplier. However, I am unable to solve this problem using Newton-Raphson. Can anyone give me some pointers as to how to proceed with solving this problem. My MATLAB Newton Raphson code incorporating first three constraints is shown below. Any help would be appreciated:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%Here we use Newton_Solver&#xA;&#xA;clear all;&#xA;clc;&#xA;&#xA;&#xA;%Initial Trials&#xA;lambda(1,1) = -1.5;&#xA;lambda(2,1) = 0.0;&#xA;lambda(3,1) = 0.09;&#xA;lambda(4,1) = 0.09;&#xA;&#xA;ll = -5;&#xA;ul = 5;&#xA;steps = 10000;&#xA;iter = 1500;&#xA;i = 0;&#xA;q = 1.5; %Tsallis q index&#xA;while(i&amp;lt;iter)&#xA;    func_val = zeros(4,1);&#xA;    Jacobian_mat = zeros(4,4);&#xA;&#xA;    avg_v0 = 0.0;&#xA;    avg_v1 = 0.0;&#xA;    avg_v2 = 0.0;&#xA;    avg_v3 = 0.0;&#xA;    avg_v4 = 0.0;&#xA;    avg_v5 = 0.0;&#xA;    avg_v6 = 0.0;    &#xA;    for(j=1:steps)&#xA;        p = ll+(ul-ll)*j/steps;&#xA;        temp = (lambda(1) + lambda(2)*p + lambda(3)*p*p + lambda(4)*p*p*p)*(q-1)/q;&#xA;        temp = temp^(1/(q-1));&#xA;        temp = temp*(ul-ll)/steps;&#xA;&#xA;        func_val(1) = func_val(1) + temp;&#xA;        func_val(2) = func_val(2) + temp*p;&#xA;        func_val(3) = func_val(3) + temp*p*p;&#xA;        func_val(4) = func_val(4) + temp*p*p*p;&#xA;&#xA;        temp1 = lambda(1) + lambda(2)*p + lambda(3)*p*p + lambda(4)*p*p*p;&#xA;        temp1 = temp1^((2-q)/(q-1));&#xA;        temp1 = temp1*((q-1)/q)^(1/(q-1))/(q-1);&#xA;        temp1 = temp1*(ul-ll)/steps;&#xA;&#xA;        avg_v0 = avg_v0 + temp1;&#xA;        avg_v1 = avg_v1 + temp1*p;&#xA;        avg_v2 = avg_v2 + temp1*p*p;&#xA;        avg_v3 = avg_v3 + temp1*p*p*p;&#xA;        avg_v4 = avg_v4 + temp1*p*p*p*p;&#xA;        avg_v5 = avg_v5 + temp1*p*p*p*p*p;&#xA;        avg_v6 = avg_v6 + temp1*p*p*p*p*p*p;&#xA;&#xA;    end&#xA;&#xA;    Jacobian_mat(1,1) = avg_v0;    &#xA;    Jacobian_mat(1,2) = avg_v1;   &#xA;    Jacobian_mat(1,3) = avg_v2;&#xA;    Jacobian_mat(1,4) = avg_v3;&#xA;&#xA;    Jacobian_mat(2,1) = avg_v1;    &#xA;    Jacobian_mat(2,2) = avg_v2;&#xA;    Jacobian_mat(2,3) = avg_v3;&#xA;    Jacobian_mat(2,4) = avg_v5;&#xA;&#xA;    Jacobian_mat(3,1) = avg_v2;    &#xA;    Jacobian_mat(3,2) = avg_v3;&#xA;    Jacobian_mat(3,3) = avg_v4;&#xA;    Jacobian_mat(3,4) = avg_v5;&#xA;&#xA;    Jacobian_mat(4,1) = avg_v3;    &#xA;    Jacobian_mat(4,2) = avg_v4;&#xA;    Jacobian_mat(4,3) = avg_v5;&#xA;    Jacobian_mat(4,4) = avg_v6;&#xA;&#xA;    func_val(1) = func_val(1) - 1.0;&#xA;    func_val(2) = func_val(2) - 0.0;&#xA;    func_val(3) = func_val(3) - 1.0;&#xA;    func_val(4) = func_val(4) - 0.0;&#xA;&#xA;    Jacobian_inv = Jacobian_mat^(-1);&#xA;    lambda = lambda - Jacobian_inv*func_val*1.0;&#xA;&#xA;    Disp_val = ['Step is: ',num2str(i), ' Lambda 1 values:', num2str(lambda(1,1))];&#xA;    Disp_val1 = ['Step is: ',num2str(i), ' Lambda 2 values:', num2str(lambda(2,1))];&#xA;    Disp_val2 = ['Step is: ',num2str(i), ' Lambda 3 values:', num2str(lambda(3,1))];&#xA;    Disp_val3 = ['Step is: ',num2str(i), ' Lambda 4 values:', num2str(lambda(4,1))];&#xA;    Disp_val4 = ['Step is: ',num2str(i), ' AVG 1 values:', num2str(func_val(1) + 1.0)];&#xA;    Disp_val5 = ['Step is: ',num2str(i), ' AVG 2 values:', num2str(func_val(2) + 0.0)];&#xA;    Disp_val6 = ['Step is: ',num2str(i), ' AVG 3 values:', num2str(func_val(3) + 1.0)];&#xA;    Disp_val7 = ['Step is: ',num2str(i), ' AVG 4 values:', num2str(func_val(4) + 0.0)];&#xA;&#xA;    disp(Disp_val);&#xA;    disp(Disp_val1);&#xA;    disp(Disp_val2);&#xA;    disp(Disp_val3);&#xA;    disp(Disp_val4);&#xA;    disp(Disp_val5);&#xA;    disp(Disp_val6);&#xA;    disp(Disp_val7);&#xA;    disp(' ');&#xA;    i = i+1;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7653" LastEditorUserId="7653" LastEditDate="2014-03-15T08:24:52.460" LastActivityDate="2014-04-14T19:45:32.750" Title="Constrained Minimization of Tsallis Entropy" Tags="|algorithms|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11089" PostTypeId="2" ParentId="11087" CreationDate="2014-03-14T15:25:17.770" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://acs.lbl.gov/software/colt/&quot; rel=&quot;nofollow&quot;&gt;Colt&lt;/a&gt; has SVD, lookup &lt;code&gt;cern.colt.matrix.linalg.SingularValueDecomposition&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that &lt;a href=&quot;http://commons.apache.org/proper/commons-math/&quot; rel=&quot;nofollow&quot;&gt;Apache Commons Math&lt;/a&gt; has its port too&lt;/p&gt;&#xA;" OwnerUserId="7470" LastEditorUserId="7470" LastEditDate="2014-03-14T15:32:03.880" LastActivityDate="2014-03-14T15:32:03.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11090" PostTypeId="1" CreationDate="2014-03-14T15:48:01.633" Score="3" ViewCount="1387" Body="&lt;p&gt;$\def\pd{\partial}$&#xA;$\def\l{\left}\def\r{\right}$&#xA;$\def\mdot{{\dot{m}}}$&#xA;$\def\eps{\varepsilon}$&#xA;Consider a tube with longitudinal coordinate $x$ from $0$ to $l$ and&#xA;varying cross-section $A(x)$.  Derek S. Bale (a former student of&#xA;Prof. Randall J. LeVeque) gives in his PhD (&lt;a href=&quot;http://faculty.washington.edu/rjl/students/dbale/thesis.ps.gz&quot; rel=&quot;nofollow&quot;&gt;http://faculty.washington.edu/rjl/students/dbale/thesis.ps.gz&lt;/a&gt;) the Euler equations for density&#xA;$\rho$, impulse density $(\rho u)$ and total energy density $e$ in such a tube in the form&#xA;$$&#xA;\begin{array}{ccccl}&#xA;  \pd_t \rho &amp;amp;+&amp;amp; \frac1{A}\pd_x(A(\rho u)) &amp;amp;=&amp;amp; 0\\&#xA;  \pd_t (\rho u) &amp;amp;+&amp;amp; \frac1{A}\pd_x\l(A\l(\frac{(\rho u)^2}\rho+p\r)\r) &amp;amp;=&amp;amp; \frac{A'}{A}p\\&#xA;  \pd_t e &amp;amp;+&amp;amp; \frac1{A}\pd_x\l(A(e+p)\frac{(\rho u)}{\rho}\r)&amp;amp;=&amp;amp; 0&#xA;\end{array}&#xA;$$&#xA;as an example for a spatially varying flux function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thereby, the pressure is determined by the gamma-law $p&#xA;=(\gamma-1)\l(e-\frac12 \frac{(\rho u)^2}{\rho}\r)$.  Note, that I&#xA;have slightly reformulated the system such that beside the pressure $p$ only conserved&#xA;quantities are used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, is it not better to change the conserved variables to the mass per length&#xA;$m':=\rho A$, the mass flow $\mdot:=A\rho u$ and the energy per length&#xA;$E':=Ae$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an intermediate variable one could introduce a pressure force $P:=A p=(\gamma-1)\l(E'-\frac12\frac{\mdot^2}{m'}\r)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Euler's equations reformulated with these new conserved quantities are:&#xA;$$&#xA;\begin{array}{ccccl}&#xA;  \pd_t m' &amp;amp;+&amp;amp; \pd_x\mdot &amp;amp;=&amp;amp; 0\\&#xA;  \pd_t \mdot &amp;amp;+&amp;amp; \pd_x\l(\frac{\mdot^2}{m'}+P\r) &amp;amp;=&amp;amp; \frac{A'}{A}P\\&#xA;  \pd_t E' &amp;amp;+&amp;amp; \pd_x\l((E'+P)\frac{\mdot}{m'}\r)&amp;amp;=&amp;amp; 0&#xA;\end{array}&#xA;$$&#xA;In this way one obtains a system with the structure of the original Euler&#xA;equations (including a source term) where the flux function does&#xA;&lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt; depend on the spatial variable.&#xA;Therefore, the numerics should work fine. I could use Roe's method on this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I miss here something? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bale uses the first form to calculate supersonic waves through a tube with a narrowing halfway. The speed-up in the narrowing causes a supersonic wave and the corresponding shock wave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variation of $A(x)$ is smooth:&#xA;$$&#xA;A(x) = \begin{cases}&#xA;1&amp;amp; \text{ for }x&amp;lt;1\\&#xA;1+\frac\eps2\bigl(\cos(\pi(x-1))-1\bigr)&amp;amp;\text{for }1\leq x \leq 3\\&#xA;1&amp;amp;\text{for }x&amp;gt;3&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of the fact that smooth transformations of the conserved quantities, such as $(m',\mdot,E')=(A\rho,A(\rho u),A e)$ have the potential to change the structure of discontinusous solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, is this really the case here?&lt;/p&gt;&#xA;" OwnerUserId="7652" LastEditorUserId="7652" LastEditDate="2014-03-14T17:12:02.550" LastActivityDate="2014-03-17T14:59:32.893" Title="Euler's equations for a tube with varying cross-section" Tags="|simulation|hyperbolic-pde|fluid-dynamics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11091" PostTypeId="2" ParentId="7626" CreationDate="2014-03-14T16:07:20.913" Score="3" Body="&lt;p&gt;This may not come as a surprise to you given that I develop deal.II, but here's my perspective: When I talk to students, I typically tell them to develop their own prototype in the beginning so they can see how it's done. But then, once they've got something small running, I make them use a library that allows them to go so much further because they don't have to reinvent the wheel with basically every step they take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you've already seen how to implement a simple Helmholtz solver. But you'll spend the next 6 months writing the code necessary to do it in parallel, you'll spend another 3 months if you want to use more complicated geometries. You'll then spend 6 more months if you want an efficient solver. And all of this time you're writing code that's already been written by someone else and that, in a sense, doesn't get you any closer to what you actually need to do for your PhD: develop something new that hasn't been done before. If you go down this road, you'll spend 2-3 years of your PhD time re-doing what others have done, and maybe 1 year doing something new.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternative is that you now spend 6 months learning one of the existing libraries, but after that you'll have 2-3 years where you really do new stuff, things where every other week you can walk into your adviser's office and show him/her something that's truly new, that runs on massively large scales, or is just very cool in other regards. I think you probably see where I'm going with this by now.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-14T16:07:20.913" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11092" PostTypeId="2" ParentId="7836" CreationDate="2014-03-14T16:22:55.963" Score="1" Body="&lt;p&gt;I believe VW has a bug with loss_function=hinge. I always see that the first iteration of bfgs showing derivative = 0 issue. For sgd it didn't seem to report this anomaly but I tend not to trust it. Presumably the error is caused by the nondifferentiability of the hinge loss function (logistic and least square are both smooth).&lt;/p&gt;&#xA;" OwnerUserId="7654" LastActivityDate="2014-03-14T16:22:55.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11093" PostTypeId="2" ParentId="11083" CreationDate="2014-03-14T16:39:33.177" Score="4" Body="&lt;p&gt;Hesthaven and Warburton's textbook &quot;Nodal Discontinuous Galerkin Methods&quot; has in chapter 5 descriptions of how to handle limiting. One interpretation of this is that through the Vandermonde matrix you can recover the Legendre polynomial coefficients directly from the nodal values and then use those limiters and then pass back to the nodal values.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-03-14T16:39:33.177" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11094" PostTypeId="1" CreationDate="2014-03-14T17:29:12.483" Score="1" ViewCount="26" Body="&lt;p&gt;I want to simulate the diffusion of initial dye concentration in a small estuary with a 2D depth averaged model. Would a radiation boundary condition for concentration of tracers alone be enough or do I need to define any additional conditions at the open boundary? &lt;/p&gt;&#xA;" OwnerUserId="3786" LastActivityDate="2014-03-14T17:29:12.483" Title="What type of boundary condition(s) do I need to define for a process with diffusion of tracer concentration?" Tags="|boundary-conditions|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11096" PostTypeId="1" AcceptedAnswerId="11099" CreationDate="2014-03-14T18:44:06.897" Score="4" ViewCount="97" Body="&lt;p&gt;I had always assumed that the covariance matrix depends upon the amount and quality of your input data, but I am finding out that this is not the case. Is this true?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to fit $f(t) = \Sigma_{i=1}^{M} ( x_i t )  $ to the $N$ datapoints $(t_i,y_i)$ by using the matrix&#xA;$A_{i,j} = \phi_j(t_i)$ where $\phi_j$ are the basis functions. Then $A$ is $N$x$M$ so we find the best fit by $X = (A^T .A)^{-1}.(A^T.Y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, here the variance covariance matrix is $V = (A^T .A)^{-1}$, but it does not contain any information about the input data trying to be fitted $\{y_i\}$, only the form of the fitting function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this really the case? Regardless of the input data, the covariance between the fitting parameters will be the same?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible the see the effect of the data on the covariance if we take into account $ \{ \sigma_i \}$, the standard deviation of observations $\{ y_i \}$, but where does this appear in your covariance matrix? Do you create $A$ as $A_{i,j} = \sigma_i * \phi_j (t_i) $?&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2014-03-14T22:12:25.163" Title="Does the covariance matrix in Least Squares depend upon the input data?" Tags="|linear-algebra|least-squares|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11097" PostTypeId="2" ParentId="11090" CreationDate="2014-03-14T19:33:31.820" Score="3" Body="&lt;p&gt;$\def\rmC{{\mathrm C}}$&#xA;$\def\vr{{\vec r}}$&#xA;$\def\vA{{\vec A}}$&#xA;$\def\ve{{\vec e}}$&#xA;$\def\l{\left}\def\r{\right}$&#xA;In the following I try to show that Bale's form is correct. If you complain, leave a comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations we &lt;strong&gt;can&lt;/strong&gt; trust are the integral equations.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Mass Balance:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \l[\int_0^x \rho(x',t') A(x') dx'\r]_{t'=0}^t + \int_{t'=0}^t \l[(\rho u)(x',t')A(x')\r]_{x'=0}^x d t' =0&#xA;$$&#xA;with the density $\rho(x,t)$ and the momentum flow $(\rho u)(x,t)$ as averages over $A(x)$.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Momentum balance:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\l[\int_0^x (\rho u)(x',t') A(x')dx'\r]_{t'=0}^t + \ve_x\cdot\int_{t'=0}^t\l( \int_{\partial V}(\rho \vec{v})\vec{v}\cdot d\vec{A} + \int_{\partial V} p d\vec{A}\r)dt =0&#xA;$$&#xA;with the volume $V=\bigcup_{x'\in[0,x]}A(x')$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no flow out of complementary part $A_\rmC:=\pd V\setminus&#xA;\bigl(\{0\}\times A(0)\cup \{x\}\times A(x)\bigr)$ of the tube&#xA;cross-sections $\{0\}\times A(0)$ and $\{x\}\times A(x)$. Furthermore, we approximate $\int_{A(x)} (\rho u)(\vr,t) \vec{v}(\vr,t)\cdot d\vA \approx (\rho u)(x,t) u(x,t) A(x)$&#xA;and remark that for integration in x-direction we have $\ve_x\cdot d\vA = d A_x(x')$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That gives:&#xA;$$&#xA;\begin{array}{l}&#xA;\l[\int_0^x (\rho u)(x',t') A(x')dx'\r]_{t'=0}^t+&#xA;\int_{t'=0}^t \l[(\rho u)(x',t') u(x',t') A(x') + pA(x')\r]_{x'=0}^{x} dt =\\&#xA;=\quad-\int_{t'=0}^t \int_{x'=0}^x p(x',t')  d A(x') dt'&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thereby $\int_{x'=0}^x p(x',t')  d A(x')$ can be interpreted as Riemann-Stieltjes integral which can be re-written as $\int_{x'=0}^x p(x',t')  A'(x') dx'$ in the case that $A(x)$ depends smoothly on $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Riemann-Stieltjes integral suggests to use&#xA;$$&#xA;p_k\frac{A_{k+\frac12}-A_{k-\frac12}}{A_k}&#xA;$$&#xA;as a 1st-order update for the source term to capture discontinuities of $A(x)$. The 1st-order update of the flux function in the FVM-method is already designed such that discontinuities of $A(x)$ are captured.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Energy balance:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;With a similar approximation for the averages over the cross-section as for the momentum balance and with the zero flux out of $A_\rmC$ we get:&#xA;$$&#xA;\l[\int_0^x e(x',t')A(x')dx'\r]_{t'=0}^t + \int_{t'=0}^t \biggl[(e(x',t') + p(x',t'))u(x')A(x')\biggr]_{x'=0}^x dt = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Interpretation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;LeVeque describes in chapter 6.16 &quot;Capacity-Form Differencing&quot; of his FVM-book that conservation laws of the form&#xA;$$&#xA;\frac{d}{dt}\int_{x_1}^{x_2} \kappa(x) q(x,t) dx = f(q(x_1,t))-f(q(x_2,t))&#xA;$$&#xA;with a capacity function $\kappa(x)$ can be treated with the help of&#xA;capacity-form differencing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more general integral form is&#xA;$$&#xA;\l[\int_{x_1}^{x_2} \kappa(x) q(x,t) dx\r]_{t_1}^{t_2} = \int_{t_1}^{t_2}(f(q(x_1,t))-f(q(x_2,t))) dt.&#xA;$$&#xA;In the above conservation laws $A(x')$ can be seen as capacity function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We see that the equations in Bale's form are correct even for shock&#xA;waves with capacity-form differencing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other form I have given in my question is also correct. For that case the conserved quantities stand right under the spatial integral on the left hand side. Everything else is just combination of terms without differentiation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I believe that Bale used his form of the equations to have a&#xA;simple example for a system with spatially varying flux function.&lt;/p&gt;&#xA;" OwnerUserId="7652" LastEditorUserId="7652" LastEditDate="2014-03-17T14:59:32.893" LastActivityDate="2014-03-17T14:59:32.893" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11098" PostTypeId="2" ParentId="7626" CreationDate="2014-03-14T22:10:59.443" Score="0" Body="&lt;p&gt;This is not a complete answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For implementation of parallel domain decomposition methods, I encountered some complications. First, one can use many processors for one subdomain or feed one processor with many subdomains and one might want to implement both paradigms. Second, substructured form of domain decomposition methods requires separating out subdomains' (not element's) faces, edges, vertices. I do not think these complications are readily included in the parallel mesh management. The situation becomes simpler if you consider one processor for one subdomain and using the overlapping RAS/RASHO method. Even in this case, you would better manage your parallel layout yourself, because when your subdomains have overlap the usual parallel mesh management may require communications for the entire overlapping regions while the necessary data for RAS/RASHO are only on the boundaries.&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2014-03-14T22:10:59.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11099" PostTypeId="2" ParentId="11096" CreationDate="2014-03-14T22:12:25.163" Score="5" Body="&lt;p&gt;I can't quite read your notation, but suppose we tried to find a set of parameters $x$ that give rise to predictions $y(x)=Ax$ by comparing with measurements $\bar y$. Then, if you try to find your parameters $x$ by solving&#xA;$$&#xA;  \min_x \frac 12 \|\bar y - y(x)\|^2 = \frac 12 \|\bar y - Ax\|^2&#xA;$$&#xA;then it is indeed true that your covariance matrix is $(A^TA)^{-1}$ and is independent of both the data $\bar y$ and the data uncertainty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in general, what we do is weight the misfit functional to use&#xA;$$&#xA;  \frac 12 \sum_i \frac 1{\sigma_i} |\bar y_i - y_i(x)|^2&#xA;$$&#xA;instead (this can be shown to be the correct formulation from a statistical viewpoint where you try to maximize the posterior probability), where $\sigma_i$ is the uncertainty in the $i$th measurement, and the covariance matrix that corresponds to this formulation than depends on the uncertainty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, if you had a nonlinear model, i.e. not just $y(x)=Ax$, then the covariance matrix would be&#xA;$$&#xA;  V = \left(\nabla y(x^\ast)^T \nabla y(x^\ast)\right)^{-1}&#xA;$$&#xA;where you take the derivative &lt;em&gt;at the solution $x^\ast$&lt;/em&gt; of the minimization problem, and this makes the covariance matrix depend on the actual data as well because the solution $x^\ast$ of course depends on the data.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-14T22:12:25.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11100" PostTypeId="1" AcceptedAnswerId="11101" CreationDate="2014-03-15T02:55:07.423" Score="1" ViewCount="1388" Body="&lt;p&gt;I am trying to solve the time dependent heat equation with backward euler timestepping and second order space finite differences. This results in a Poisson system needing to be inverted. In serial this is trivial to solve, but I am not sure how one would go about doing this parallel with MPI. Does anyone know how to solve the 2d heat equation (implicit) with MPI? I am using a 1d parallelization. I have looked online for different parallel solvers, but it has been difficult to find a paper that actually explains clearly how to implement this problem. Any references would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: What I am really solving is the 2d heat equation where I used spectral differentiation (fft's) in the periodic horizontal and second order finite differences in the vertical (so I specify dirichlet BC's on the top and bottom boundaries). I parallelize my domain by decomposing it in strips of rows in the horizontal that way computing the global fft's is trivial on each process. I am then left with trying to solve what is basically a modified heat equation system for the vertical direction. If my domain was decomposed as groups of columns insted of strips of rows, then this would be trivial as it would amount to solving a bunch of tridiagonal systems. One thing that I could do is do an MPI_ALLTOALL call, solve the tridiagonal systems on each process, then MPI_ALLTOALL call back, but this is communication instensive.  &lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2014-03-15T15:45:53.900" LastActivityDate="2014-03-15T15:45:53.900" Title="Parallel 2d heat equation (implicit timestepping) using MPI" Tags="|pde|fortran|mpi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11101" PostTypeId="2" ParentId="11100" CreationDate="2014-03-15T03:31:37.210" Score="4" Body="&lt;p&gt;Use a library such as PETSc or Trilinos. They offer you data structures for parallel linear systems and algorithms that can work on them, such as parallel implementations of CG or GMRES. The PETSc manual provides extensive examples on how to use all of this. I'm quite certain that there's in fact even a tutorial program that already gets you 90% there.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-15T03:31:37.210" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11102" PostTypeId="1" CreationDate="2014-03-15T12:21:47.440" Score="3" ViewCount="61" Body="&lt;p&gt;I am working on a fluid simulation using vortex methods. For this I must compute the vortex sheet on my boundaries given as:&#xA;$$&#xA;\gamma(\mathbf{x}) - \frac{1}{\pi}\int_S \frac{\partial}{\partial\mathbf{n}}(\text{log}|\mathbf{x} - \mathbf{x'}|)\gamma(\mathbf{x'})d\mathbf{x'} = -2\mathbf{u}\cdot \mathbf{s}&#xA;$$&#xA;I have two questions on how to compute the vortex sheet $\gamma$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;a href=&quot;http://lorenabarba.com/wp-content/uploads/2013/01/CooperBarbaAIAA-2009-3546.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; of Cooper and Barba state this can be solved using radial basis functions.&#xA;I think the hardest part to compute is&#xA;$$&#xA;\oint \frac{\partial}{\partial n}[\text{log}|\mathbf{x}-\mathbf{x'}|]\phi(|\mathbf{x'} - \mathbf{x_i}) d\mathbf{x'}&#xA;$$&#xA;What would the discrete version of this be? So how do I compute this term?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I would use the panel method, how does a discretisation look like?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7658" LastActivityDate="2014-03-15T12:21:47.440" Title="Discretizing boundary conditions for vortex methods" Tags="|simulation|boundary-conditions|fluid-dynamics|vorticity|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11103" PostTypeId="1" AcceptedAnswerId="11105" CreationDate="2014-03-15T14:52:41.260" Score="2" ViewCount="3661" Body="&lt;p&gt;I want to calculate ${n \choose k}/2^n$ for moderate $n$ and $k$. In Matlab, use &lt;code&gt;nchoosek(n,k)&lt;/code&gt; with $n=60$ and $k=30$ will give a warning: &quot;Warning: Result may not be exact. Coefficient is greater than 9.007199e+15&#xA;and is only accurate to 15 digits &quot;. I understand that $60\choose30$ is a huge number, but what I really need is ${60\choose30}/2^{60}$, which is around 0.1026.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is there a way to compute ${n \choose k}/2^n$ without sacrificing numerical precision in Matlab?&lt;/p&gt;&#xA;" OwnerUserId="3512" LastEditorUserId="27" LastEditDate="2014-03-15T15:15:38.023" LastActivityDate="2015-07-01T10:51:45.627" Title="calculate Binomial coefficient over $2^n$ in Matlab" Tags="|matlab|combinatorics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11104" PostTypeId="2" ParentId="11103" CreationDate="2014-03-15T17:23:06.127" Score="4" Body="&lt;p&gt;I'd take the log of your expression, calculate the log of your expression using built-in functions that are well-behaved (i.e., don't underflow or overflow), and then exponentiate at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;{n \choose k} = \frac{n!}{k!(n-k)!}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\log\left(\frac{{n \choose k}}{2^{n}}\right) = \log{n!} - \log{k!} - \log{(n - k)!} - n \log{2}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can calculate $\log{n!}$ via &lt;code&gt;gammaln(n+1)&lt;/code&gt;, since $n!$ is $\Gamma(n+1)$, where $\Gamma$ is the Gamma function. (Don't calculate $\log{n!}$ as &lt;code&gt;log(gamma(n+1))&lt;/code&gt;, since it won't avoid overflow issues!) This method should be better behaved. You could also get a quick estimate by using Stirling's approximation to $\log{n!}$, which is $n \log{n} - n$; I suspect you want something more accurate, but Stirling's approximation will give you a reasonable idea as to whether or not your exponential operation at the end will overflow. &lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-03-15T17:32:07.380" LastActivityDate="2014-03-15T17:32:07.380" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11105" PostTypeId="2" ParentId="11103" CreationDate="2014-03-15T17:26:02.797" Score="5" Body="&lt;p&gt;If you have MATLAB's Symbolic Math Toolbox installed, then it is just a matter of writing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;evalin(symengine, 'binomial(60, 30) / 2^60')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could write your own version of &lt;code&gt;nchoosek&lt;/code&gt; (see &lt;a href=&quot;http://www.csl.mtu.edu/cs4321/www/Lectures/Lecture%2015%20-%20Dynamic%20Programming%20Binomial%20Coefficients.htm&quot;&gt;this&lt;/a&gt;) using multiple precision arithmetic (available in MATLAB through third party toolboxes &lt;a href=&quot;http://sourceforge.net/projects/mptoolbox/&quot;&gt;like this one&lt;/a&gt;).  You may also consider writing your own code for addition of arbitrary precision integer arithmetic (this is &lt;a href=&quot;http://books.google.com/books?id=rqhhlV046kMC&amp;amp;lpg=PP1&amp;amp;dq=winkler.%20polynomial%20algorithms%20in%20computer%20algebra&amp;amp;pg=PA27#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;not too hard&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2014-03-15T17:36:08.507" LastActivityDate="2014-03-15T17:36:08.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11106" PostTypeId="2" ParentId="11103" CreationDate="2014-03-15T18:06:55.653" Score="3" Body="&lt;p&gt;If utmost speed is not a concern, I'd go for rewriting &lt;code&gt;nchoosek&lt;/code&gt; interspersing the divisions with the computation so that the temporary values stay bounded. What follows is a sample implementation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function accumulator=dividedbinomial(n,k)&#xA;  num=n; %factor to multiply at the numerator&#xA;  den=k; %factor to multiply at the denominator&#xA;  powers=n; %powers of 2 still left to divide&#xA;  accumulator=1; &#xA;  for i=1:k&#xA;    accumulator=accumulator*n/k;&#xA;    n=n-1;&#xA;    k=k-1;&#xA;    %divides now by enough powers of two so that the result stays below 1&#xA;    while accumulator&amp;gt;1 &amp;amp;&amp;amp; powers&amp;gt;0 &#xA;      accumulator=accumulator/2;&#xA;      powers=powers-1;&#xA;    end&#xA;  end&#xA;  %divides by the remaining powers of two&#xA;  while powers&amp;gt;0&#xA;    accumulator=accumulator/2;&#xA;    powers=powers-1;&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2014-03-15T18:31:37.513" LastActivityDate="2014-03-15T18:31:37.513" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11107" PostTypeId="2" ParentId="11088" CreationDate="2014-03-15T18:59:56.883" Score="1" Body="&lt;p&gt;Presumably, you want to maximize the entropy, so after the exposition you've presented above, you have something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\min_{\lambda_{0}, \ldots, \lambda_{3}} -\frac{1}{q - 1}\left(1-\int{\left[\left(\frac{q-1}{q}\right)\left(\lambda_0+\lambda_1p+\lambda_2p^2+\lambda_3p^3\right)\right]^{\frac{q}{q-1}}dp}\right)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and you want to do unconstrained minimization. Here, I'm using the fact that most optimization problems are posed as minimization problems, and $\max_{x} g(x) = \min_{x} -g(x)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The big comments I'd make about your code are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Don't do Newton's method (for equation solving) by hand, if you can help it.&lt;/strong&gt; People mess it up, and library routines are much more robust when it comes to setting convergence tolerances. MATLAB includes it in the &lt;code&gt;fsolve&lt;/code&gt; routine as part of the Optimization Toolbox. You may not have that toolbox (toolboxes are expensive, so individuals and institutions don't necessarily have them), in which case, I'd encourage you to find another tool that does (Python definitely does, Octave might).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Don't do Newton's method (for unconstrained optimization) by hand, for the same reasons.&lt;/strong&gt; MATLAB includes this routine, again, as part of the Optimization Toolbox. You can call it via &lt;code&gt;fminunc&lt;/code&gt;. Mistakes made in numerical algorithms can be very subtle, and affect your results in a profound way. Using libraries is a big time saver by eliminating another source of error.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Generally speaking, don't calculate the explicit inverse of a matrix.&lt;/strong&gt; Set up and solve an equivalent linear system. In MATLAB, use the backslash operator for this purpose.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If convergence is a problem and you have to write Newton's method by hand, consider building in something like a line search for robustness, and add some error tolerance checks, so you don't necessarily have to run all 1500 iterations.&lt;/strong&gt; These features are generally included in a library, so like I said, get access to one rather than write these yourself (which is another source of error, and time-consuming).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Vectorize your evaluation of the integrals. For that matter, vectorize all your function evaluations, if possible.&lt;/strong&gt; You're explicitly looping over everything, and constructing arrays element-by-element. These processes are slow, because you can't take advantage of any of the fast MATLAB routines that are linked from compiled libraries.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Modularize your code.&lt;/strong&gt; You should be able to move your objective function, gradient, and Hessian evaluations into their own functions (or maybe a combined function that calculates two or three of these quantities, if you want to reuse information from the integration). Refactoring your code in this way would make it much easier to read and test.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Make your code self-documenting.&lt;/strong&gt; Add comments judiciously. People will spend more time reading your code than it takes for you to write it. It's hard to go through parts of your code because I have no clue what &lt;code&gt;avg_v0&lt;/code&gt; is supposed to mean, other than, well, it's some sort of average. If your main audience is someone in your application area, maybe this sort of naming is fine. You've posted your code to a general computational science message board, so I encourage you to make it accessible to general computational scientists. I have no idea what &lt;code&gt;temp&lt;/code&gt; means either, other than it's a temporary variable.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Test your code.&lt;/strong&gt; If you've made your code modular, you should be able to test it in an automated way using unit tests. MATLAB xUnit is a good library for this purpose; later versions of MATLAB have unit testing functionality built in. You should be able to construct examples with values of $q$ and $\lambda_{i}$, $i = 0, \ldots, 3$ so that you can check evaluations of your objective function, gradient, and Hessian against values you can compute by hand analytically. You could also use a computer algebra system like Maple, Mathematica, Sage, or SymPy for this purpose (and in the process, actually derive your gradient and Hessian functions, mitigating another source of error: bad math). Testing means writing more code, but it also means debugging is easier, because you have more diagnostics with which to find errors, and a more systematic means of identifying these errors.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Look at &lt;a href=&quot;http://software-carpentry.org/&quot; rel=&quot;nofollow&quot;&gt;Software Carpentry&lt;/a&gt;&lt;/strong&gt;, which discusses a lot of the points on testing, modularization, self-documenting code, etc. (&lt;em&gt;Disclaimer:&lt;/em&gt; I do some volunteer work for Software Carpentry.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-03-15T18:59:56.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11112" PostTypeId="1" CreationDate="2014-03-17T00:12:47.043" Score="1" ViewCount="1472" Body="&lt;p&gt;I have here &lt;a href=&quot;http://dx.doi.org/10.2139/ssrn.729886&quot; rel=&quot;nofollow&quot;&gt;equations from a paper by E. Bradlow&lt;/a&gt;. They're for counting the events in a Weibull-distributed data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\Pr(N(t)=n) &amp;amp;= \sum_{j=n}^\infty{\frac{(-1)^{j+n}(\lambda t^c)^j \alpha_j^n}{\Gamma(cj+1)}} \\&#xA;E(N(t)) &amp;amp;= \sum_{n=1}^\infty{\sum_{j=n}^\infty{\frac{n(-1)^{j+n}(\lambda t^c)^j \alpha_j^n}{\Gamma(cj+1)}}}&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $Pr(N(t) = n)$ is the probability that the count will be $n$, and $E(N(t))$ is the expected value. Below are the other vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\alpha_m^l &amp;amp;= \sum_{m=0}^{l-1}{\frac{\Gamma(cm+1)\Gamma(cl-cm+1)}{\Gamma(m+1)\Gamma(l-m+1)}} \\&#xA;\alpha_j^0 &amp;amp;= \frac{\Gamma(cj+1)}{\Gamma(j+1)}, \quad j=0,1,2,\ldots \\&#xA;\alpha_j^{n+1} &amp;amp;= \sum_{m=n}^{j-1}{\alpha_m^n \frac{\Gamma(cj-cm+1)}{\Gamma(j-m+1)}}&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've programmed these into C++, but I'm not getting the right results. I know because when I get the expected value, I get an extremely negative number (too many zeroes to count). I'd like some input on where I got my code wrong?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#ifndef M_PI&#xA;#define M_PI 3.14159265358979323846&#xA;#endif&#xA;&#xA;using namespace std;&#xA;&#xA;double alpha_m(int n, double shape){&#xA;       double sum = 0;&#xA;       for(int m = 0; m &amp;lt; n-1; m += 1){&#xA;             //printf(&quot;%i &amp;lt; %i\n&quot;, m, n-1);&#xA;             double num = tgamma((shape*m) + 1) * tgamma((shape*n) - (shape*m) + 1);&#xA;//             printf(&quot;num %f , shape %f, m %i\n&quot;, num, shape, m);&#xA;//             system(&quot;pause&quot;);&#xA;             double denom = tgamma(m + 1) * tgamma(n - m + 1);&#xA;             //printf(&quot;denum %f \n&quot;, denom);&#xA;             sum += num/denom;&#xA;             //printf(&quot;sum%f\n&quot;, sum);&#xA;       }&#xA;//       printf(&quot;sum%f\n&quot;, sum);&#xA;//         printf(&quot;done&quot;);&#xA;       return sum;&#xA;}&#xA;&#xA;double alpha_j(int n, int j, double shape){&#xA;       //printf(&quot;n: %i&quot;, n);&#xA;       //system(&quot;pause&quot;);&#xA;       double sum = 0;&#xA;       if(n == 0){&#xA;            double result = tgamma((shape*j) + 1)/tgamma(j + 1);&#xA;            return result;&#xA;       } else if(n &amp;gt;= 1) {&#xA;            for(int m = n-1; m &amp;lt; j-1; m += 1){&#xA;                  //printf(&quot;%i &amp;lt; %i\n&quot;, m, j-1);&#xA;                  double result = tgamma((shape*j) - (shape*m) + 1)/tgamma(j - m + 1);&#xA;//                  printf(&quot;result %f\n&quot;, result);&#xA;                  sum += result;&#xA;//                  printf(&quot;sum %f\n&quot;, sum);&#xA;            }&#xA;            double alpha = alpha_m(n,shape);&#xA;//            printf(&quot;sum %f, alpha %f\n&quot;, sum, alpha);&#xA;            sum = sum*alpha;&#xA;//            printf(&quot;sum %f\n&quot;, sum);&#xA;            return sum;&#xA;       }&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;/*  double x;&#xA;&#xA;&#xA;  printf(&quot;libm&quot;);&#xA;  for(x=1.0; x &amp;lt;= 10.0; x+=1.0) {&#xA;    printf(&quot;%15.8lf\n&quot;, tgamma(x/3.0));&#xA;  }&#xA;  system(&quot;pause&quot;);&#xA;  return 0;&#xA;*/&#xA;  double scale, shape;&#xA;  int freq;&#xA;&#xA;  cout &amp;lt;&amp;lt; &quot;Enter scale param: &quot;;&#xA;  cin &amp;gt;&amp;gt; scale;&#xA;  cout &amp;lt;&amp;lt; &quot;Enter shape param: &quot;;&#xA;  cin &amp;gt;&amp;gt; shape;&#xA;  cout &amp;lt;&amp;lt; &quot;Enter Frequency of PM: &quot;;&#xA;  cin &amp;gt;&amp;gt; freq;&#xA;&#xA;  printf(&quot;Scale: %f\nc: %f\nFrequency: %i\n\n&quot;, scale, shape, freq);&#xA;  system(&quot;pause&quot;);&#xA;&#xA;&#xA;  double sum = 0;&#xA;&#xA;  if(freq &amp;gt; 0){&#xA;      for(int n=1; n &amp;lt;= 99; n++){&#xA;            //printf(&quot;%i\n&quot;, n);&#xA;            for(int j=n; j&amp;lt;=99; j++){&#xA;                  double t = 365/freq;&#xA;                  double power = pow (t, shape);&#xA;                  double num = n*((-1)^(j+n))* pow (scale*power, j);&#xA;                  double denom = tgamma((shape*j) + 1);&#xA;                  //printf(&quot;%f&quot;, alpha_j(n, j, shape));&#xA;                  sum += num*alpha_j(n, j, shape)/denom;&#xA;//                  printf(&quot;t %f, power %f, num %f, denom %f, sum %f\n, alpha %f&quot;, t, power, num, denom, sum, alpha_j(n, j, shape));&#xA;//                  system(&quot;pause&quot;);&#xA;                  //printf(&quot;n = %i ;; j = %i\n&quot;, n, j);&#xA;            }&#xA;      }&#xA;  } else if(freq == 0){&#xA;         sum = (365*24)/(scale*tgamma((1/shape) + 1));&#xA;         //printf(&quot;Mean TTF gamma %f&quot;, tgamma((1/shape) + 1));&#xA;  }&#xA;&#xA;  printf(&quot;The final sum is %f\n&quot;, sum);&#xA;  system(&quot;pause&quot;);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7674" LastActivityDate="2014-03-17T21:51:18.470" Title="Coding complex equations into C++" Tags="|c++|statistics|probability|vector|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11113" PostTypeId="2" ParentId="11083" CreationDate="2014-03-17T02:41:46.160" Score="2" Body="&lt;p&gt;Well, you say: &quot;My attempts at writing a nodal DG code and extending the existing limiters in a &quot;natural&quot; way does not lead to accurate solutions.&quot;  So that may be a problem.  Limiters are going to affect accuracy.  Most limiters are going to reduce you to first order accurate, even at higher order.  The terrible fact is, the answer to this question is extremely problem/application dependent.  I would recommend checking out&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.overtureframework.org/banks/publications/sublinearContacts_JCP2008.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.overtureframework.org/banks/publications/sublinearContacts_JCP2008.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathematik.uni-dortmund.de/~kuzmin/dglim.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mathematik.uni-dortmund.de/~kuzmin/dglim.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The BDS limiter is a good multi-functioning limiter (see DOI:10.1016/j.jcp.2011.07.009), which can be used in both settings (nodal or modal) fairly easily.  As a general rule, I would say the first step in choosing a limiter is understanding very clearly what your high level goals are going to be.  For example, do you need to preserve positivity as well?  Are you after an optimal limiter, in terms of computational cycles, or are you aiming for the best accuracy you can find?  Do you need a robust solution, or is convergence order more important to you?  Are you going to be benchmarking against finite difference, finite volume, or spectral methods solutions, such that having too little numerical diffusion might actually cause you problems?  Understanding the setting is actually maybe the most important part of choosing a limiter.&lt;/p&gt;&#xA;" OwnerUserId="7675" LastActivityDate="2014-03-17T02:41:46.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11114" PostTypeId="1" CreationDate="2014-03-17T12:41:16.530" Score="4" ViewCount="2276" Body="&lt;p&gt;Can anybody recommend me a good introduction to Crouzeix-Raviart Finite Elements? Their motivation is not obvious and the body of literature is hard to overlook.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2014-03-17T13:45:44.760" Title="Crouzeix-Raviart Finite Element" Tags="|finite-element|reference-request|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11115" PostTypeId="2" ParentId="11114" CreationDate="2014-03-17T12:59:10.290" Score="5" Body="&lt;p&gt;I'd take a look at Chapter 3 of the &lt;a href=&quot;http://fenicsproject.org/pub/book/book/fenics-book-2011-06-14.pdf&quot; rel=&quot;nofollow&quot;&gt;FEniCS book&lt;/a&gt;. They cover common and unusual finite elements&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, a Crouziex-Raviart element is a non-conforming finite element used typically for $H^1$ or $C_0$ discretizations. The nonconformity comes from the fact that continuity is only enforced on a CR element's midpoint as opposed to vertices. For scalar $P_1$ Crouziex-Raviart elements, this reduces down to the condition that the average of the solution is continuous, i.e. over an element edge $e$, $\int_{e}[[u]] = 0$.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2014-03-17T13:45:44.760" LastActivityDate="2014-03-17T13:45:44.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11116" PostTypeId="2" ParentId="4812" CreationDate="2014-03-17T15:27:19.060" Score="7" Body="&lt;p&gt;The Scharfetter-Gummel (SG) formulation is commonly used to solve the current density equations.  This is a special formulation that overcomes difficulties in solving the nonlinear dependence between potential and current density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A standard text discussing how these equations using box integration methods is in this book:&#xA;Selberherr, S., Analysis and simulation of semiconductor devices.  Springer-Verlag 1984&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This type of simulation called Technology Computer Aided Design (TCAD).  As opposed to the Finite Element Method (FEM), Finite Volume Method (FVM) is used to calculate the currents.  This is since it fits into the SG formulation that has been shown (by practioners of this method) to work when solving the current density equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to solve this using generalized PDE's, COMSOL has a Semiconductor Module which solves this problem using a hybrid FEM/FVM method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, commercial and open source TCAD simulators are listed here:&#xA;&lt;a href=&quot;http://www.tcadcentral.com&quot; rel=&quot;noreferrer&quot;&gt;http://www.tcadcentral.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge generalized-PDE TCAD solvers are DEVSIM, FLOOPS, PROPHET.  The commercial tools tend to have most of the physical equations are hard coded in a compiled language such as C++.&lt;/p&gt;&#xA;" OwnerUserId="7681" LastActivityDate="2014-03-17T15:27:19.060" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11117" PostTypeId="2" ParentId="11112" CreationDate="2014-03-17T21:51:18.470" Score="1" Body="&lt;p&gt;There are several issues that you will need to resolve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) In your function alpha_m the loop should be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (int m = 0; m &amp;lt; n; m++) {&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;because the sum goes from 0 to n-1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) I'm not sure what you tried in the function alpha_j.  But I believe the loop should not end at m = j-2.  Try to match your variable names and starting indices with those of the equation you are trying to code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Also, function alpha_j does not return anything if n &amp;lt; 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) The main routine does not deal with the freq &amp;lt; 0 case.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2014-03-17T21:51:18.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11119" PostTypeId="1" AcceptedAnswerId="11120" CreationDate="2014-03-18T14:54:22.807" Score="5" ViewCount="2441" Body="&lt;p&gt;I am trying to understand an example from a book, but I seem to get different answers depending on which spectral decomposition function I use in NumPy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to find a spectral decomposition $W V W^\top$ of a given symmetric matrix $G$ using NumPy functions &lt;code&gt;eig&lt;/code&gt; and &lt;code&gt;eigh&lt;/code&gt;, where the latter is for symmetric matrices (which $G$ is):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;eig&lt;/code&gt; returns columns arranged in a different order from the answer in the book. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;eigh&lt;/code&gt; returns a matrix similar to that in the book, but with a different sign in columns $2$ and $4$. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Are these matrices equivalent? They yield different results in downstream use. Any help is appreciated. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;G = np.array([[-0.55,  0.3  , 0.   , 0.    ,0.25],&#xA;              [ 0.3 , -0.95 , 0.   , 0.4   ,0.25],&#xA;              [ 0.  ,  0.   ,-0.55 , 0.3   ,0.25],&#xA;              [ 0.  ,  0.4  , 0.3  ,-0.95  ,0.25],&#xA;              [ 0.25,  0.25 , 0.25 , 0.25  ,-1.  ]])&#xA;&#xA;v1,w1 = np.linalg.eigh(G)&#xA;v2,w2 = np.linalg.eig(G)&#xA;&#xA;print w1&#xA;array([[ 0.224,  0.224,  0.5  , -0.671,  0.447],&#xA;       [-0.671,  0.224, -0.5  , -0.224,  0.447],&#xA;       [-0.224,  0.224,  0.5  ,  0.671,  0.447],&#xA;       [ 0.671,  0.224, -0.5  ,  0.224,  0.447],&#xA;       [ 0.   , -0.894, -0.   ,  0.   ,  0.447]]))&#xA;&#xA;print w2&#xA;array([[-0.447, -0.671, -0.5  ,  0.224, -0.224],&#xA;       [-0.447, -0.224,  0.5  , -0.671, -0.224],&#xA;       [-0.447,  0.671, -0.5  , -0.224, -0.224],&#xA;       [-0.447,  0.224,  0.5  ,  0.671, -0.224],&#xA;       [-0.447,  0.   ,  0.   ,  0.   ,  0.894]]))&#xA;&#xA;&quot;Answer in book&quot;&#xA;array([[ 0.224, -0.224,  0.5  ,  0.671,  0.447],&#xA;       [-0.671, -0.224, -0.5  ,  0.224,  0.447],&#xA;       [-0.224, -0.224,  0.5  , -0.671,  0.447],&#xA;       [ 0.671, -0.224, -0.5  , -0.224,  0.447],&#xA;       [ 0.   ,  0.894, -0.   ,  0.   ,  0.447]]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7687" LastEditorUserId="20417" LastEditDate="2018-06-11T10:30:32.273" LastActivityDate="2018-06-11T10:30:32.273" Title="Sign differences in spectral decomposition in NumPy" Tags="|linear-algebra|python|eigensystem|numpy|spectral-method|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11120" PostTypeId="2" ParentId="11119" CreationDate="2014-03-18T15:58:37.903" Score="5" Body="&lt;p&gt;Let $w_k$ be the k-th column of W (the kth eigen-vector) and $v_k$ be the k-th element of v (the kth eigen-value).  Then we can write:&#xA;$$ G = \sum_k w_k v_k w_k^T $$&#xA;which, element-wise, is equivalent to:&#xA;$$ G_{i,j} = \sum_k v_k w_{i,k} w_{j,k} $$&#xA;where the indices are column-major.&#xA;Since we always get $w_k$ twice, an overall sign on a column of $W$ cancels.&#xA;Since the sum over $k$ can occur in any order, the order of the columns of $W$ also doesn't matter as long as the elements of $v$ are in the same order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they are yielding different results down-stream, you likely have an error.  Were you consistent with using &lt;code&gt;v1&lt;/code&gt; with &lt;code&gt;w1&lt;/code&gt; and &lt;code&gt;v2&lt;/code&gt; with &lt;code&gt;w2&lt;/code&gt;?  &lt;code&gt;v1&lt;/code&gt; and &lt;code&gt;v2&lt;/code&gt; should have the same elements but a different order.  Neither &lt;code&gt;eig&lt;/code&gt; nor &lt;code&gt;eigh&lt;/code&gt; guarantee an ordering of the eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2014-03-18T18:24:18.703" LastActivityDate="2014-03-18T18:24:18.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11121" PostTypeId="1" CreationDate="2014-03-18T18:51:14.553" Score="1" ViewCount="66" Body="&lt;p&gt;I am trying to find the effect on turbulence characteristics when flow passes through a converging section. For this experiment I will use a PIV system to gather data about the flow. &#xA;However, I could not find any specific literature on flow through symmetric converging channel. (The converging channel I would be using is type of a converging hopper with a circular hole exit.) I have attached a 2d drawing of this &lt;img src=&quot;https://i.stack.imgur.com/oJCIg.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please advise if you know any similar papers relating to this.&lt;/p&gt;&#xA;" OwnerUserId="7692" LastActivityDate="2014-03-18T18:51:14.553" Title="Turbulence: Suggest research papers similar to this experiment" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11122" PostTypeId="1" AcceptedAnswerId="11605" CreationDate="2014-03-18T21:57:43.850" Score="1" ViewCount="881" Body="&lt;p&gt;I used &lt;a href=&quot;http://en.wikipedia.org/wiki/Math_Kernel_Library&quot; rel=&quot;nofollow&quot;&gt;MKL Lapack&lt;/a&gt; &lt;a href=&quot;http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-659297EE-D35B-428C-ABBE-1A7EBE2B0F6E.htm&quot; rel=&quot;nofollow&quot;&gt;SBGVX&lt;/a&gt; because it can solve for &quot;selected&quot; eigenvalues/modes (both positive and negative), thinking it would be efficient, but it is extremely slow when compared with &lt;a href=&quot;http://web.mit.edu/kjb/www/Principal_Publications/The_subspace_iteration_method_Revisited.pdf&quot; rel=&quot;nofollow&quot;&gt;Bathe's subspace iteration&lt;/a&gt; eigensolver, which unfortunately can solve only for positive eigenvalues. I need only the smallest positive eigenvalue and its mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve the problem &#xA;$$&#xA;(\mathbf{K} - e\,\mathbf{M})\,\mathbf{u} = \mathbf{0}&#xA;$$&#xA;with $\mathbf{M}$ not +definite, and $\mathbf{K}$&lt;del&gt;-- A-- &lt;/del&gt; that is +definite.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $\mathbf{K}$ is has the form of a stiffness matrix and is invertible while $\mathbf{M}$ is like a mass matrix, some of the components of which can be negative (for instance, for negative-mass metamaterials).  It is possible that some of the diagonal terms are zero, but let us assume that the matrix is invertible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since SBGVX needs the second matrix to be +definite, I multiply by $-1/e$ and rearrange it as &#xA;$$&#xA;(\mathbf{M} - e'\,\mathbf{K})\,\mathbf{u} = \mathbf{0}&#xA;$$&#xA;then look for the highest $e'$, which is positive, to then compute the lowest positive eigenvalue $e=1/e'$ . It works, but it is very slow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to modify Bathe's subspace iteration eigensolver to compute the highest eigenvalue instead of the smaller?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, how can I transform &#xA;$$&#xA;(\mathbf{K} - e\,\mathbf{M})\,\mathbf{u} = \mathbf{0}&#xA;$$&#xA;to solve for eigenvalues $e^2$ instead of $e$, so I can use Bathe's solver to find the smallest $e$? Bathe's solver crashes trying to solve for the smallest $e$ when the problem has negative eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, does anyone know another solver that I can try? A typical system has 5000 equations. &lt;/p&gt;&#xA;" OwnerUserId="7693" LastEditorUserId="7626" LastEditDate="2014-03-19T08:51:40.733" LastActivityDate="2014-05-11T18:49:41.300" Title="How to find the smallest positive eigenvalue of a large general system if they are all in +/- pairs of real eigenvalues" Tags="|linear-algebra|finite-element|eigensystem|eigenvalues|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11123" PostTypeId="1" AcceptedAnswerId="11124" CreationDate="2014-03-19T03:23:17.900" Score="4" ViewCount="333" Body="&lt;p&gt;For my problem I need to extract the lower convex hull of a set of 3D points (X,Y,Z). In Matlab, one can find the convex hull using the convhull function as follows: K = convhull(X,Y,Z).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anybody please suggest me a way to extract the lower hull from the convex hull? I found many references/code for 2D case but for 3D it seems to be not very popular (I'm new to computational geometry).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your help.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastEditorUserId="549" LastEditDate="2014-03-19T03:58:52.397" LastActivityDate="2014-03-19T03:59:37.150" Title="Extract 3D lower hull from convex hull" Tags="|computational-geometry|convex-hull|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11124" PostTypeId="2" ParentId="11123" CreationDate="2014-03-19T03:53:02.533" Score="4" Body="&lt;p&gt;Every facet in a 3D convex hull has a normal vector.  The key is to determine which facets have an outward normal vector pointing downward.  You can determine this easily by looking at the z-component of each normal vector.  If it is negative, it's part of the lower hull.  If it's positive, it's part of the upper hull.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you'll have to decide whether or not to include with facets with zero z-components.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-03-19T03:59:37.150" LastActivityDate="2014-03-19T03:59:37.150" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11125" PostTypeId="1" AcceptedAnswerId="11126" CreationDate="2014-03-19T11:25:13.997" Score="5" ViewCount="1350" Body="&lt;p&gt;I have tinkered around with the implementation of hybridizable Discontinuous Galerkin (HDG) methods in DUNE-PDELab and DUNE-FEM for my PhD but since then I have not had the time to work on these codes very much. Some things were possible back then, but others such as element-wise assemblies of the facet contributions or static condensation did not fit well into the concept and required a lot of work which is most probably not compatible anymore with current versions, since it involved reimplementing certain internals. That being said, this applied to almost any framework I tried out during that time, the only exception being NGSolve (which unfortunately had some compatibility issues with Mac OS X that lead me to give up on it after a while).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two years have passed since then and I have certain things in mind which I want to try out in the future, but I still could not find THE framework for HDG; in fact it seems like there are loads of papers published on the method and its analysis for a whole zoo of problems, but the class of HDG methods is for the most part ignored by framework developers. Am I wrong?&lt;/p&gt;&#xA;" OwnerUserId="4260" LastActivityDate="2022-05-23T06:26:21.287" Title="Which finite-element framework allows the easy implementation of HDG methods?" Tags="|finite-element|discretization|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11126" PostTypeId="2" ParentId="11125" CreationDate="2014-03-19T11:25:13.997" Score="4" Body="&lt;p&gt;The newest &lt;a href=&quot;https://web.archive.org/web/20141017095749/http://www.dealii.org:80/8.1.0/doxygen/deal.II/step_51.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;step 51&lt;/a&gt; of the deal.ii tutorial describes the implementation and some tests of an HDG method for the convection-diffusion equation. As far as I can see there is no support for other elements than hexahedrals but the documentation of the implemented method looks excellent.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://sourceforge.net/projects/ngsolve/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NGSolve&lt;/a&gt; has support for HDG which was demonstrated for convection-diffusion as well as incompressible and compressible flow problems by various authors.&lt;/p&gt;&#xA;&lt;p&gt;It is possible to implement HDG in DUNE-PDELab (the &lt;a href=&quot;https://doi.org/10.1007/978-3-642-28589-9_13&quot; rel=&quot;nofollow noreferrer&quot;&gt;implementation&lt;/a&gt; is however incompatible with current releases and has some limitations). There also exist some implementations of HDG in DUNE-FEM which are all - to my best knowledge - not publicly released.&lt;/p&gt;&#xA;&lt;p&gt;Also I keep hearing every now and then that FEniCS has some rudimentary support for HDG but I could not find working examples.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="22284" LastEditDate="2022-05-23T06:26:21.287" LastActivityDate="2022-05-23T06:26:21.287" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11127" PostTypeId="1" CreationDate="2014-03-19T16:51:12.900" Score="5" ViewCount="228" Body="&lt;p&gt;I am reposting here a question previously asked in stackoverflow: as suggested in a reply to my question, scicomp could be a better place to obtain some useful comments/suggestions for my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to make some tests with BLACS/Scalapack (C interface, Intel MKL version) making use multiple MPI communicators: particularly, what I am trying to obtain is a set of BLACS contexts/grids corresponding (one-to-one) to a set of (disjoint) MPI communicators. While I have no problems in working with a single &quot;global&quot; communicator (&lt;code&gt;MPI_COMM_WORLD&lt;/code&gt;) I have some difficulties with the multiple communicator case. I hope that you could give me some suggestions for the following problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a bit confused about the BLACS context variable value update after a call to &lt;code&gt;Cblacs_gridinit&lt;/code&gt;: let's suppose to start with a &quot;global&quot; context corresponding to the &lt;code&gt;MPI_COMM_WORLD&lt;/code&gt; communicator. I can obtain the corresponding BLACS context with the call:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MPI_Comm globalCommunicator(MPI_COMM_WORLD);&#xA;&#xA;MKL_INT globalContext(Csys2blacs_handle(globalCommunicator));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and create a grid on it with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Cblacs_gridinit(&amp;amp;globalContext,&#xA;                &amp;amp;c_blacsGridOrdering,&#xA;                MKL_INT i_nTaskRow,&#xA;                MKL_INT i_nTaskCol);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with, for example, &lt;code&gt;char c_blacsGridOrdering('R')&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The globalContext value in this case is 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At some point in my code, which is making use of 6 MPI tasks,&#xA;an MPI communicator (&lt;code&gt;localCommunicator&lt;/code&gt;) corresponding to the group of 4 tasks with id &lt;code&gt;[0;3]&lt;/code&gt; is being created: at this point I would like to create a new BLACS context (&lt;code&gt;localContext&lt;/code&gt;) for this &quot;local&quot; communicator and a local grid on it. I can do that with the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MKL_INT localContext(Csys2blacs_handle(localCommunicator));&#xA;&#xA;Cblacs_gridinit(&amp;amp;localContext,&#xA;                &amp;amp;c_blacsGridOrdering,&#xA;                MKL_INT i_nTaskRowLocal,&#xA;                MKL_INT i_nTaskColLocal);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the above function call sequence is performed only by the tasks included in the local communicator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;localContext&lt;/code&gt; value, after the call to &lt;code&gt;Csys2blacs_handle&lt;/code&gt;, is equal to 1 (for each task of the local communicator) but it is modified and set to 0 by the subsequent call to &lt;code&gt;Cblacs_gridinit&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously this causes some problems in the rest of my code since, for example, if I try to retrieve the MPI communicator corresponding to &lt;code&gt;localContext&lt;/code&gt; with a call to &lt;code&gt;Cblacs2sys_handle&lt;/code&gt; I get a six task communicator corresponding to the initial &lt;code&gt;globalCommunicator&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most likely I am making some very stupid error (missing call to some BLACS functions??) or it is not clear to me the interplay between multiple MPI communicators and BLACS context/grids but I can not find what is wrong with my code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have some suggestions concerning the above problem? Many thanks for your help!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;/////////&#xA;UPDATE 1&#xA;/////////&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an update for my question which could be useful to find a solution or at least an explanation for the observed problem: the described problem is no more present if I initialize the first grid (the one making use of the global context) on a task grid whose size is such that all available MPI tasks are included, for example with MKL_INT i_nTaskRow(1) and MKL_INT i_nTaskCol(MPI_COMM_WORLD size). Is this behavior expected from BLACS? Many thanks again for your support!   &lt;/p&gt;&#xA;" OwnerUserId="7699" LastActivityDate="2014-03-19T16:51:12.900" Title="BLACS context value and multiple MPI communicators" Tags="|linear-algebra|parallel-computing|mpi|intel-mkl|scalapack|" AnswerCount="0" CommentCount="1" FavoriteCount="0" ClosedDate="2018-01-07T23:03:38.173" ContentLicense="CC BY-SA 3.0" />
  <row Id="11128" PostTypeId="1" CreationDate="2014-03-19T18:01:21.273" Score="1" ViewCount="898" Body="&lt;p&gt;$\def\rmin{{\mathrm{in}}}$&#xA;$\def\l{\left}\def\r{\right}$&#xA;$\def\tagl#1{\tag{#1}\label{#1}}$ &#xA;I am using the one-dimensional finite volume method to calculate the air flow in some tube.&#xA;For subsonic flow one can predefine two quantities at the inlet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inlet is a wide opening to the ambient.&#xA;The overall system deeply inside the tube is a bit more complicated. But, this does not really matter here. The system is described in some PhD and I am coding along the lines of this PhD. The author assumes isentropic flow at the inlet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the inlet pressure $p_\rmin$ he gave the formula&#xA;\begin{align}&#xA;  p_\rmin + \frac{\gamma-1}\gamma\rho_0 \l(\frac{p_\rmin}{p_0}\r)^{\frac1\gamma}\frac{u_\rmin^2}2&amp;amp;= p_0&#xA;  \tagl{A20amb}&#xA;\end{align}&#xA;with the ambient pressure $p_0=10^5 \rm Pa$, the ambient air density $\rho_0=1.23\rm \frac{kg}{m^3}$ and the given inlet velocity $u_\rmin$ .&#xA;After solving this equation for $p_\rmin$ one can also determine the inlet density $\rho_\rmin$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pittyingly, I do not get this equation.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He motivated \eqref{A20amb} through the ideal gas law&#xA;\begin{align}&#xA;  p=\rho(c_p-c_v)T,&#xA;  \tagl{A1amb}&#xA;\end{align}&#xA;the assumption of isentropic state transition&#xA;\begin{align}&#xA;  \frac{p_\rmin}{p_0} = \l(\frac{\rho_\rmin}{\rho_0}\r)^\gamma&#xA;  \tagl{A3amb}&#xA;\end{align}&#xA;where&#xA;\begin{align}&#xA;  \gamma := \frac{c_p}{c_v},&#xA;  \tagl{A2}&#xA;\end{align}&#xA;and the energy balance&#xA;\begin{align}&#xA;  c_p T_0 = c_pT_\rmin + \frac12 u_\rmin^2&#xA;  \tagl{A5amb}&#xA;\end{align}&#xA;with the ambient temperature $T_0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substitution of $T=\frac{p}{(c_p-c_v)\rho}$ from \eqref{A1amb} into \eqref{A5amb} delivers with $\rho_\rmin = \rho_0\l(\frac{p_\rmin}{p_0}\r)^{1/\gamma}$ from \eqref{A3amb}  the equation&#xA;\begin{align}&#xA;  c_p\frac{p_0}{(c_p-c_v)\rho_0} = c_p\frac{p_\rmin}{(c_p-c_v)\rho_0\l(\frac{p_\rmin}{p_0}\r)^{1/\gamma}} +\frac12 u_\rmin^2&#xA;\end{align}&#xA;which can be transformed into&#xA;\begin{align}&#xA;  1 &amp;amp;= \l(\frac{p_\rmin}{p_0}\r)^{\frac{\gamma-1}\gamma} + (\gamma-1) \frac{\rho_0}{\gamma p_0}\cdot\frac{u_\rmin^2}2\\&#xA;  1 &amp;amp;= \l(\frac{p_\rmin}{p_0}\r)^{\frac{\gamma-1}\gamma} + \frac{\gamma-1}2\l(\frac{u_\rmin}{c_0}\r)^2&#xA;  \tagl{p0}&#xA;\end{align}&#xA;where&#xA;\begin{align}&#xA;c_0=\sqrt{\frac{\gamma p_0}{\rho_0}}&#xA;\tagl{speedOfSound}&#xA;\end{align}&#xA;is the speed of sound in the ambient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This formula can be resolved for the inlet pressure:&#xA;\begin{align}&#xA;  p_\rmin &amp;amp;= p_0 \l(1-\frac{\gamma-1}2\l(\frac{u_\rmin}{c_0}\r)^2\r)^{\frac{\gamma}{\gamma-1}}&#xA;\tagl{solution}&#xA;\end{align}&#xA;&lt;strike&gt;I did not check whether \eqref{solution} is equivalent to \eqref{A20amb}. But, I do not think so because then the author would have used the explicite form \eqref{solution} to calculate $p_\rmin$.&lt;/strike&gt; EDIT: I did check for differences. See below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What am I missing??&lt;/strong&gt;&#xA;Did I miss-interprete the assumptions?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that also \eqref{A20amb} can be re-formulated with the help of $c_0$:&#xA;\begin{align}&#xA;\l(\frac{p_\rmin}{p_0}\r) + \frac{\gamma-1}2 \l(\frac{p_\rmin}{p_0}\r)^{\frac1\gamma}\l(\frac{u_\rmin}{c_0}\r)^2 = 1&#xA;\tagl{A20mod}&#xA;\end{align}&#xA;If one divides by $\l(\frac{p_\rmin}{p_0}\r)^{\frac1\gamma}$ one nicely recognizes the difference between \eqref{A20amb} and \eqref{p0}:&#xA;\begin{align}&#xA;\l(\frac{p_\rmin}{p_0}\r)^{\frac{\gamma-1}\gamma} + \frac{\gamma-1}2\l(\frac{u_\rmin}{c_0}\r)^2 = \l(\frac{p_\rmin}{p_0}\r)^{-\frac1\gamma}&#xA;\tagl{A20mod2}&#xA;\end{align}&#xA;Instead of the 1 on the left-hand side of \eqref{p0} the formula from the PhD-thesis has a $\l(\frac{p_\rmin}{p_0}\r)^{-\frac1\gamma}$.&lt;/p&gt;&#xA;" OwnerUserId="7652" LastEditorUserId="7652" LastEditDate="2014-03-31T18:01:39.123" LastActivityDate="2014-03-31T18:01:39.123" Title="Euler's equations 1d for pipe, Inlet boundary conditions" Tags="|boundary-conditions|finite-volume|fluid-dynamics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11129" PostTypeId="2" ParentId="11122" CreationDate="2014-03-19T19:43:19.360" Score="2" Body="&lt;p&gt;I am not an expert in this area, so there may be better methods for handling this problem, but what I suggest should work in principle; I don't know if it will converge faster than the methods you are currently using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $M$ is invertible, then you can transform your generalized eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;Ku = \lambda Mu&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;into an ordinary eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;M^{-1}Ku = \lambda u&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then solve it. Furthermore, if $K$ is positive definite, then $M^{-1}K$ should be invertible, and all (generalized) eigenvalues should be nonzero. If all (generalized) eigenvalues occur in pairs with opposite signs, then your problem reduces to finding the (generalized) eigenvalue with smallest magnitude, which you can do with something like an inverse power method. Once you find the smallest magnitude (generalized) eigenvalue, it should be straightforward to calculate the corresponding (generalized) eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a problem with 5000 degrees of freedom, it should be possible to invert $M$ with direct methods, calculate $M^{-1}K$ explicitly, and then use standard methods for calculating eigenvalues and eigenvectors. If $M$ is not invertible, you will need to use methods specialized for generalized eigenvalue problems. One software package that can handle these problems is SLEPc; it interfaces to a number of iterative eigenvalue solvers (ARPACK, BLOPEX, TRLAN, etc.). &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-03-19T19:43:19.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11130" PostTypeId="1" AcceptedAnswerId="11133" CreationDate="2014-03-19T19:50:26.223" Score="6" ViewCount="132" Body="&lt;p&gt;Typical finite element problems assume $L^2$ which is a Hilbert space, but I've heard that $L^1$ for Navier-Stokes results in less overshoots/undershoots, but $L^1$ is not Hilbert. The dual space for $L^1$ is $L^\infty$, so it seems that we might want some sort of Petrov-Galerkin formulation. But practically, do these different spaces result in different choices for the basis functions?&lt;/p&gt;&#xA;" OwnerUserId="3309" LastActivityDate="2014-03-20T01:19:24.137" Title="L1 functional setting for Navier-Stokes with finite elements" Tags="|finite-element|navier-stokes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11131" PostTypeId="2" ParentId="10394" CreationDate="2014-03-19T20:29:54.283" Score="4" Body="&lt;p&gt;I &lt;a href=&quot;https://dsp.stackexchange.com/questions/13346/how-to-compute-the-statistics-of-the-dft-of-noise&quot;&gt;asked this question on the Signal Processing exchange&lt;/a&gt; as well. Eventually I found the answer myself and posted it there. Here I copy/paste my answer as posted there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After many weeks I give the answer to my own question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a limit in which we can solve this problem in a reasonably simple way. Suppose we sum enough points in our DFT that the &lt;a href=&quot;https://dsp.stackexchange.com/questions/13346/how-to-compute-the-statistics-of-the-dft-of-noise&quot;&gt;central limit theorem&lt;/a&gt; guarantees that the distribution of the sum's real and imaginary parts are Gaussian distributed. Then we only need to compute the variance. If we specialize the case of the variances of the real part of the DFT we can write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\langle (\textrm{Re} X_k) (\textrm{Re} X_l) \rangle = \frac{1}{N^2}\sum_{n=0}^{N-1}\sum_{m=0}^{N-1}\langle x_n x_m \rangle \cos(2\pi n k / N)\cos(2 \pi m l / N)&#xA;$$&#xA;The thing to note is that $\langle x_n x_m \rangle$ is just the correlation function of the time domain samples, which we can denote $\rho(n-m)$. Putting this in, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\langle (\textrm{Re} X_k) (\textrm{Re} X_l) \rangle = \frac{1}{N^2}\sum_{n=0}^{N-1}\sum_{m=0}^{N-1} \rho(n-m) \cos(2\pi n k / N)\cos(2 \pi m l / N)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This form in terms of the correlation function is useful because the correlation function is frequently known; By the &lt;a href=&quot;http://en.wikipedia.org/wiki/Wiener%E2%80%93Khinchin_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wiener-Khinchin theorem&lt;/a&gt; the correlation function is the Fourier transform of the spectral density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sum can be computed numerically, or even analytically for some particular forms of $\rho$. To compute covariance with imaginary parts of $X$ just put $\sin$ instead of $\cos$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the idea for this in J. Schoukens and J. Renneboog, IEEE Transactions on Instrumentation and Measurement, Vol. IM-35, No. 3, September (1986).&lt;/p&gt;&#xA;" OwnerUserId="6915" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2014-03-19T20:29:54.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11132" PostTypeId="2" ParentId="11122" CreationDate="2014-03-19T22:31:37.733" Score="1" Body="&lt;p&gt;$\def\m#1{\mathbf{#1}}$&#xA;I know my answer below is a bit off. Somehow, I misread the question that the  eigenvalues would come in pairs $\pm\lambda$. On second reading you did not say so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, I do not delete the answer yet since it shows the principle how to do a double-shift. You can apply the same principle to calculate $\lambda^2$ instead of $\lambda$. Just solve the system twice in each step. To get $\lambda^2$ is even simplier than what I wrote below. You can choose the identity matrix $\m1$ for $f$ and shift two times with $\lambda$, i.e., $\bar K = K-\lambda\m1$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you apply an iterative method with zero shift to a matrix with eigenvalues of the same absolute value you can run into problems.&#xA;A good example is the matrix&#xA;\begin{align}&#xA;K:=&#xA;\begin{pmatrix}&#xA;0&amp;amp;1\\&#xA;1&amp;amp;0&#xA;\end{pmatrix}&#xA;\end{align}&#xA;with eigenvalues $\lambda_{12}=\pm 1$ and eigen directions $e^{(1)}:=\frac1{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix}$, $e^{(2)}:=\frac1{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix}$.&#xA;If you iterate&#xA;\begin{align}&#xA;v^{(k+1)} := K v^{(k)}&#xA;\end{align}&#xA;with initial guess $v^{(0)}=\begin{pmatrix}1\\0\end{pmatrix}$ you get the oscillating sequence $v^{(2k)}=v^{(0)}, v^{(2k+1)}=\begin{pmatrix}0\\1\end{pmatrix}$ for $(k=0,1,\ldots)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The iterated for the eigenvalues remain zero:&#xA;$$&#xA;\lambda^{(k+1)} = \frac{(v^{(k)})^T K v^{(k)}}{|v^{(k)}|^2}=\frac{(v^{(k)})^T v^{(k)}}{|v^{(k)}|^2}=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In such cases it is important to have a good initial guess for the eigenvalue and apply the corresponding shift to the matrix $K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are looking for the smallest eigenvalue you should apply inverse iteration.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You told us that the eigenvalues come in pairs $\pm\lambda$. This looks like there is structure in the problem. If you have some (linear) mapping $f:\mathbb{R}^n\rightarrow\mathbb{R}^n$ which maps the eigen vector $e$ for $\lambda$ to the eigen vector $\bar e$ for $-\lambda$, you could apply a double shift strategy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lambda^{(k)}$ is the current approximation for the eigenvalue, $x^{(k)}$ the current guess for the corresponding eigen vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shift: $\bar K^{(k)} := K+\lambda^{(k)}\m1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve&#xA;$$&#xA;\bar K^{(k)} y^{(k)} = M f(x^{(k)})&#xA;$$&#xA;for $y^{(k)}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $\tilde K^{(k)} := K-\lambda^{(k)}\m1$ solve&#xA;$$&#xA;\tilde K^{(k)} z^{(k)} = M f(y^{(k)})&#xA;$$&#xA;for $z^{(k)}$ and set&#xA;$$&#xA;x^{(k+1)}:=\frac{z^{(k)}}{|z^{(k)}|}&#xA;$$&#xA;The new guess for the eigen value should be&#xA;$$&#xA;\lambda^{(k+1)} = \bigl((x^{(k)})^T z^{(k)} + (\lambda^{(k)})^{-2}\bigr)^{-\frac12}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first guess $x^{(0)}$ should be normalized $|x^{(0)}|=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can try it with $M=\m1$, $K$ from above and $f(v)=\begin{pmatrix}v_1\\-v_2\end{pmatrix}$.&lt;/p&gt;&#xA;" OwnerUserId="7652" LastEditorUserId="7652" LastEditDate="2014-03-20T08:44:56.147" LastActivityDate="2014-03-20T08:44:56.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11133" PostTypeId="2" ParentId="11130" CreationDate="2014-03-20T01:19:24.137" Score="6" Body="&lt;p&gt;Take a look at some of the papers by Jean-Luc Guermond over the past 10 or so years that deal with solving PDEs my minimizing the residual in the L1 norm. His website is &lt;a href=&quot;http://www.math.tamu.edu/~guermond/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-20T01:19:24.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11134" PostTypeId="2" ParentId="11128" CreationDate="2014-03-20T04:04:41.950" Score="1" Body="&lt;p&gt;I lack the time to dig into it at the moment (sorry), but any chance sections 2.9, 2.10, and 2.11 from &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0486419630&quot; rel=&quot;nofollow&quot;&gt;Liepmann and Roshko&lt;/a&gt; (available in the linked Amazon preview) can help resolve your question?  The exposition and derivations in that book are quite good.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2014-03-20T04:04:41.950" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11135" PostTypeId="1" AcceptedAnswerId="11137" CreationDate="2014-03-20T08:39:45.233" Score="3" ViewCount="1465" Body="&lt;p&gt;I am trying to find specific eigenvalues and -vectors of a large complex symmetric tridiagonal matrix (at least 10000x10000, and ideally larger). I know roughly which eigenvalues I am looking for, so I've been using scipy.linalg.sparse.eigs (really ARPACK) to find them. Unfortunately, this has turned out to be slow for large matrices - solving a 7000x7000 matrix took approximately 30 minutes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel that there must be some way of using the symmetry properties of the matrix I am dealing with - it's not just any sparse complex matrix. As a comparison, when I use hermitian matrices of the same size, linalg.sparse.eigsh allows me to reduce the solution time by approximately a factor of 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any freely available eigensolvers out there which implement efficient algorithms for diagonalising the kinds of matrices I am dealing with? Do such algorithms even exist? &lt;/p&gt;&#xA;" OwnerUserId="7705" LastActivityDate="2014-03-21T21:41:14.960" Title="Finding eigenvalues of a complex symmetric tridiagonal matrix" Tags="|python|matrix|eigenvalues|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11136" PostTypeId="2" ParentId="10814" CreationDate="2014-03-20T16:13:18.550" Score="3" Body="&lt;p&gt;This is definitely whats known as a splitting scheme and are very popular in nonlinear optics as well as Quantum simulations, an example being the dirac equation but you can also do it on nonlinear PDES'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the issue you're having is that you're solving for $u_1^{k+1}$ first in terms of $u_2^k$, when you should really be solving for $u_1^{k+1}$ in terms of $u_2^{k+1}$, (if you want to use an upwind scheme in (2). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see this, notice that the first equation is linear in $u_1$, with coefficients and an inhomogeneity only depending on $u_2$. So 'in principle' (it could be brutally tough), you could find you're solution to $u_1$ from the first equation (1), analytically in terms of a Greens function (which also may depend on $u_2$) convoluted with the LHS of one. ie. let $ Lu_1$ be the RHS of (1) and $f=\nabla \cdot f_2(u_2)$ and let $G(x;u_2)$ satisfy $LG=\delta(x-s)$ and the boundary conditions for $u_1$, then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_1=\int_{\Omega} G((x-s);u_2)fds$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you've reduced the coupled set to an integro-differential equation for (2). However, now look what happens if you want to do an upwind scheme for $u_2^{k+1}$, you find that $u_1^{k+1}$ depends on $u_2^{k+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is definitely flexibility though in this decoupling or splitting though that could make for a nice scheme in your equations. A nice introduction to splitting in the context you're interested in can be found in Chapter 13 'Splitting and its cousins' of Boyd 'Chebyshev and Fourier Spectral Methods'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps&lt;/p&gt;&#xA;" OwnerUserId="7710" LastActivityDate="2014-03-20T16:13:18.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11137" PostTypeId="2" ParentId="11135" CreationDate="2014-03-20T22:10:03.217" Score="3" Body="&lt;p&gt;You can apply a banded cholesky-like algorithm to generate L*L.', although the stability is questionable because you cannot bound the pivot growth (unlike the LL' positive definite case, which is well behaved). So perhaps a shift-invert strategy (since you know which eigenvalues you want) might be able to give you an answer. In the event you have to slog through a cluster to find a particular eigenvector nearby your chosen shift, you can exploit the fact that distinct eigenvectors (vi,vj) of a complex-symmetric matrix are orthogonal under a quirky non-hermitian &quot;inner product&quot;, vi.'*vj = 0. You can use that fact to setup an outer loop that deflates your arpack residual against converged but unwanted eigenvectors on every arnoldi step, via gram-schmidt. Note this part is dodgy too, because that &quot;inner product&quot; $&amp;lt;vi,vi&amp;gt;$ can be negative/zero, so your deflation/projection/normalization gram-schmidt coefficients can go haywire, too. Ultimately, complex symmetry is a nice storage-saving property but does not really impart the same robustness as real/hermitian symmetry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So lots of caveats, but FWIW I have written working software based on this - wouldn't suggest it otherwise.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2014-03-20T23:21:07.747" LastActivityDate="2014-03-20T23:21:07.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11140" PostTypeId="1" CreationDate="2014-03-21T14:02:11.393" Score="1" ViewCount="231" Body="&lt;p&gt;I looked at the list of publications related to deal.II, but it seems that when it comes to Navier-Stokes equations, everyone speaks about Discontinuous Galerkin solver in their papers. I know there's also a Continuous Galerkin method implemented in deal.II. Has it been used for high-order accurate solution of Navier-Stokes equations? Could someone comment on performance/accuracy differences between CG and DG for fluid flow problems in deal.II?&lt;/p&gt;&#xA;" OwnerUserId="4688" LastEditorUserId="4688" LastEditDate="2014-03-21T15:03:46.913" LastActivityDate="2014-03-21T15:03:46.913" Title="High-order continuous Galerkin Navier-Stokes solver in deal.II" Tags="|fluid-dynamics|navier-stokes|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11142" PostTypeId="2" ParentId="11135" CreationDate="2014-03-21T21:41:14.960" Score="0" Body="&lt;p&gt;LAPACK seems to have several methods for this &lt;a href=&quot;http://www.netlib.org/lapack/lug/node48.html&quot; rel=&quot;nofollow&quot;&gt;case&lt;/a&gt;. You would have to wrap the fortran code with f2py however. Either way, the algorithms certainly exist, and the LAPACK documentation provides references to the relevant articles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a slightly different note, make sure you're using a new version of gfortran or (better yet) Intels fortran compiler ifort. It might save you 20% or more (on runtime).&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-03-21T21:41:14.960" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11143" PostTypeId="1" CreationDate="2014-03-21T22:39:06.443" Score="1" ViewCount="103" Body="&lt;p&gt;I've got a little conjecture I need to prove for a theoretical result related to causal Bayes net search with latent variables under sparsity constraints.  If you're interested in the application please feel free to email me at adambrod@andrew.cmu.edu.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have a $p\times q$ real matrix $A$ such that 1) $A$ is rank deficient so $\textrm{rk}(A)=r&amp;lt;\min(p,q)$, and 2) there are no zero elements in $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to prove that there is no factorization of $A$ into two matrices with&#xA;fewer total non-zero elements than some factorization of $A$ into two&#xA;matrices $B$ and $C$ of dimensions $p\times r$ and $r\times q$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proof strategy I'm employing is proof by induction on rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The base case, $\textrm{rk}(A)=1$ is trivial ($B$ and $C$ must have at least $p+q$ non-zero&#xA;elements between the two of them in order for every element of $A$ to be&#xA;non-zero and that is the maximum number of elements in a pair of matrices&#xA;with dimensions $p\times r$ and $r\times q$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The induction step is as follows:  Subtract a matrix $D$ of rank 1 from $A$&#xA;such that the resulting matrix $S$ is of rank $r-1$.  This corresponds to&#xA;removing all of the &quot;influence&quot; of some specific row vector $v$ from $A$,&#xA;such that $v$ is linearly independent of the remainder, $S$.  $D$ can be expressed&#xA;as the outer product of some column vector $w$ with $v$.  Suppose $wv$ is the&#xA;sparsest factorization of $D$ (the possibility of this is implied by our&#xA;base case).  Let $RT$ be the sparsest factorization of $S$ of dimensions $p\times &#xA;r-1$ and $r-1\times q$ respectively.  My conjecture is that the sparsest&#xA;factorization of $A$ is achieved by appending these two sparsest&#xA;factorizations, that is, the sparsest factorization of $A$ is the product of&#xA;$R$ with $w$ appended as the $r$th column and $T$ with $v$ appended as the $r$th row.&#xA;Because $D$ and $S$ are linearly independent this seems quite plausible, but a&#xA;rigorous demonstration eludes me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any solutions, suggestions, or references would be greatly appreciated, and of course, if you provide me with personal information (name/department affiliation), I would be happy to acknowledge your assistance in any publications I apply this result in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7727" LastActivityDate="2014-03-21T22:39:06.443" Title="Sparse matrix factorization of a rank deficient matrix by decomposition into linearly independent components" Tags="|linear-algebra|sparse-matrix|rank|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11144" PostTypeId="1" AcceptedAnswerId="11145" CreationDate="2014-03-22T01:08:32.260" Score="4" ViewCount="3526" Body="&lt;p&gt;I need to develop a FVM code in C (The multiscale FVM method for heterogeneous media). I know that: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Only uniform rectangular grids will be considered (2d now, later 3d)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sparse systems will be large&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Need to parallelize my code eventually (will be solving local&#xA;problems on fine grid and use them to solve PDE on coarse grid)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Speed is key&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I used Matlab  exclusively during my PhD. I am now faced with the challenge of C programming which is new to me.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fully intend to use existing libraries as much as possible, but I am quite puzzled  about my options and to what extent I should use them.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Must I write my own program  and assemble the matrices and then look for solver libraries, or are there alternatives ? &lt;/li&gt;&#xA;&lt;li&gt;Which library would you recommend for this? Is PETSC my best choice&#xA;? (I couldnt find  many sparse solvers in C)&lt;/li&gt;&#xA;&lt;li&gt;I looked at GSL, LApack, etc and my question is does using a library entail using only  the data types/headers  that they offer ?&#xA;For instance GSL has a vector and matrix header file which seems to&#xA;have its own data type. &lt;/li&gt;&#xA;&lt;li&gt;Even simple steps, like setting a matrix A= B requires several lines of code in C, is there any library that offers some of these basic &quot;matlab&quot;&#xA;luxuries, basically readymade functions in C for such tasks ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I realize some of my questions might be very elementary, considering my  lack of  familiarity with C and the non-adhoc way to program  numerical methods. I apologize. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the question is subjective in nature, but your answers will point me in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="7728" LastActivityDate="2014-03-23T22:45:13.770" Title="Developing finite volume (FVM) code in C . General advice" Tags="|finite-difference|petsc|libraries|finite-volume|c|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11145" PostTypeId="2" ParentId="11144" CreationDate="2014-03-22T01:47:01.947" Score="5" Body="&lt;p&gt;Some thoughts from someone who has worked a fair amount in compiled languages, and has done a tiny bit of FVM:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Typically, if you have experience programming in C, you sketch out a high-level description (pseudocode) of what you would like to do. Then you look for libraries that might implement the data structures and capabilities you need for your high-level description. Then you learn how to use those libraries by writing simple programs that do simpler versions of what you want to do. Sometimes, libraries have example programs. Use them. If you can't find a library that does what you want, you write code to do it yourself. Then, once you have some handle on what you want to do, and how you use the libraries you want to use, you write your program. This process may be iterative; for instance, you may find that you neglected some crucial step in your high-level description, and then you have to find a library that implements the capabilities you need for that step, and repeat the process. Write programs to solve small problems first, then work up to bigger problems. Trying to write the program that will solve the big problem in the first attempt is just asking for problems.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;From your description, PETSc is probably a reasonable choice. It's well-documented, the developers are responsive and helpful, and it has many examples. PETSc's certainly not your only choice, but it's probably the most convenient library that implements standard numerical methods from time steppers to linear solvers, and it makes it relatively straightforward to write a code to run in parallel (assuming you have some basic idea of how to do so).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you're asking this question, you should read a book on C and do the exercises. The book by Kernighan and Ritchie is good, as is Learn C The Hard Way, by Zed Shaw. Also, read a book like Code Complete by Steve McConnell, or Clean Code by Robert Martin, so you learn decent programming style, and so you learn to write something that other people can read.&lt;/p&gt;&#xA;&lt;p&gt;Typically, you include the library header files and link any static or dynamic library    files accompanying those headers. Libraries provide an Application Programming Interface (API), which includes data types and functions (typically operating on those data types). As you program more, you'll see what I mean.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Not really, no. In using C, you're trading off programmer productivity and the convenience of automatic memory management for execution speed and manual memory management. If you want something that's more MATLAB-like, use Python. There is a Python interface for PETSc called petsc4py. It is not as well-documented as the PETSc library. For finite volume methods, if you're willing to live with explicit time integrators, you could try PyClaw, also written in Python. Pure Python programs will be slower than pure C programs; however, many Python libraries write performance-critical portions of their code in C, so you get most of the performance benefits of C along with the convenience and productivity benefits of using Python. (You could do something similar in MATLAB with MEX files.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-03-23T22:45:13.770" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11147" PostTypeId="1" AcceptedAnswerId="11148" CreationDate="2014-03-22T04:56:53.707" Score="4" ViewCount="3824" Body="&lt;p&gt;I have been writing some code in C for &lt;a href=&quot;http://en.wikipedia.org/wiki/Particle-in-cell&quot; rel=&quot;nofollow&quot;&gt;particle-in-cell&lt;/a&gt; simulation.&#xA;One of the steps of the PIC algorithm requires to solve (numerically) Poisson's equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Delta \varphi = - 4 \pi \rho.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially I want to limit the program to 2d case ( instead of 3d ) and use the finite difference method ( instead of finite elements ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Much to my surprise, I was not able to find any free open source C library for this task ( i.e. to solve 2d Poisson's equation using the finite difference method ).&#xA;The only thing I've found is a FISHPACK - collection of FORTRAN77 routines, part of a famous SLATEC library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm fine with fortran, but a call to a fortran routine from C requires some unnecessary array transformations and &quot;native&quot; C solution seems more convenient. &#xA;Another reason I hesitate to use FISHPACK is that it uses single precision floats with implicit type declaration. &#xA;Using the gcc '-fdefault-real-8' flag I was able to convert it to double precision, but I'm not sure whether or not everything still works correctly after this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is: are there any free ( as in freedom ) C/C++ libraries to solve 2d Poisson equation using the finite difference method?&#xA;And since I do intend to switch to 3d and finite elements, advices on C library for this purpose are also welcome. &lt;/p&gt;&#xA;" OwnerUserId="7730" LastActivityDate="2014-03-22T08:22:35.213" Title="Free open source C/C++ library to solve 2d Poisson equation using the finite difference method" Tags="|finite-difference|c|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11148" PostTypeId="2" ParentId="11147" CreationDate="2014-03-22T08:22:35.213" Score="2" Body="&lt;p&gt;2-D Poisson by finite differences looks to be a few examples in PETSc (KSP: ex29, ex32, ex50). I'd probably use a nonlinear solver (SNES) or time-stepper (TS) in PETSc anyway for the added flexibility. 2-D Poisson via finite differences is probably an example in other packages (for instance, probably Trilinos). I know it's an example given in Using MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many free C/C++ libraries to solve Poisson via finite elements. You could try PETSc again, deal.II, DUNE, FEniCS, &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_finite_element_software_packages&quot; rel=&quot;nofollow&quot;&gt;any other library in this list on Wikipedia&lt;/a&gt;, or (probably) Trilinos. Using a library like deal.II or FEniCS, it looks to be relatively easy to change the dimensionality of a code (i.e., go from 2-D to 3-D), so you might consider writing a 2-D finite element code to start, and then converting it to 3-D.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-03-22T08:22:35.213" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11149" PostTypeId="1" CreationDate="2014-03-22T15:04:21.287" Score="0" ViewCount="202" Body="&lt;p&gt;I have a (very) large sparse matrix in CSC form and I'm supposed to factorize it. I've read that between Givens and Householder transformations, Givens is better for a sparse matrix. The problem is that I don't really know how I should do it (everything's fine for a full matrix, but now all I have are three vectors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want to use built-in Matlab functions. &lt;/p&gt;&#xA;" OwnerDisplayName="user7729" LastActivityDate="2014-03-22T15:04:21.287" Title="Givens method for sparse matrix" Tags="|matlab|sparse-matrix|matrix|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11150" PostTypeId="2" ParentId="11055" CreationDate="2014-03-22T15:10:20.830" Score="1" Body="&lt;p&gt;The first thing to say is that such numbers exist.  The &quot;trivial&quot; cases are where $n = F(n)$, namely $n=1,5$.  After that the next solution is $n=12$, and many multiples of $12$ are also solutions.  We will show below that infinitely many solutions exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second thing to note is that computing Fibonacci numbers $F(n)$ by the recurrence relation modulo $n$ seems inefficient.  These are basically integer powers of the &quot;golden ratio&quot; $\tau = 1.618\ldots$ and its reciprocal, so the computation of $F(n)$ can be done in $O(\log n)$ steps, rather than $n$ (very simple, add/subtract) steps using the recurrence from scratch each time.  Knowing that PARI is performing the power of a small matrix mod $n$, it seems like that this is already being taken advantage of...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sieving idea, using the strong divisibility property of the Fibonacci numbers, seems attractive as well.  There should be a lot of structure to solutions of $n|F(n)$ because $m|n \iff F(m)|F(n)$.  In particular if $n|F(n)$, and $m=F(n)$, then $m|F(m)$ will be another solution provided $n \gt 5$.  This shows an infinite number of solutions exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally if $m,n$ are solutions, so too will $\operatorname{lcm}(m,n)$ be a solution.  I will add more details, but as an example, $5\cdot 12$ is a solution since both $5$ and $12$ are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will also show that by looking at prime divisors, a large amount of search space can be pruned away.  The simplest case of this is that any solution divisible by $2$ or by $3$ must actually be divisible by $12$.  This alone reduces the search space by a little more than half, since we need check only residues $0,1,5,7,11$ mod $12$ for possible solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we should check is the periodic divisibility of Fibonacci numbers by $n$.  It turns out that for each positive integer $n$, there exists a positive integer $a(n)$ such that $n|F(m)$ if and only if $a(n)|m$.  See &lt;a href=&quot;https://oeis.org/A001177&quot; rel=&quot;nofollow noreferrer&quot;&gt;OEIS Sequence A001177&lt;/a&gt; as well as David Speyer's fine Answer (and others) to this &lt;a href=&quot;https://math.stackexchange.com/q/60340/3111&quot;&gt;Math.SE Question&lt;/a&gt; on divisibility in Fibonacci numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only every third Fibonacci number is even, $a(2)=3$, which implies that if $n$ is an even solution to $n|F(n)$, then also $3|n$, and thus $3|F(n)$.  But three divides only every fourth Fibonacci number, $a(3)=4$, so even $n|F(n)$ implies $12|n$.  A similar argument starting from a solution where $3|n$ would give again that $12|n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the prime divisor $5$ is a bit of a wash, because $a(5)=5$.  We get no new restrictions by sieving for multiples of $5$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we check for larger prime divisors, we do find ever more stringent restictions.  Consider a solution $n|F(n)$ such that $7|n$.  Thus, since $7|F(n)$ we have also $a(7)=8$ dividing $n$.  Since $56 = \operatorname{lcm}(7,8)$ divides $n$ and $F(n)$, we have $a(56)=24$ divides $n$.  Taking the least common multiple once more gives us that $168|n$, and now (since $a(168)=24$) the reasoning stabilizes.  Note that $168|F(168)$, the first solution divisible by $7$, and to find any solution divisible by $7$, it suffices to check $n$ which are divisible by $168$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This suggests a scheme in which we compute (or look up) $a(p)$ for all primes $p$ under the square root of the threshold $N$ you wanted to search, e.g. $N=10^{10}$ if I understood the original problem, and we need the primes below $10^5$.  We carry out the chain of reasoning for $p$ similar to the case $p=7$ above entails, and that will both find a &quot;primitive&quot; solution and reduce searching for solutions that are multiples of $p$ to searching multiples of what generally is a much larger number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an interesting open problem in mathematics, &lt;a href=&quot;http://en.wikipedia.org/wiki/Wall%E2%80%93Sun%E2%80%93Sun_prime&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wall's conjecture&lt;/a&gt;, connected to the computation of $a(p^k)$ for prime $p$ and exponent $k \gt 1$.  The search of examples has been carried out in the negative beyond $10^{16}$, which allows us for computational purposes to assume $a(p^k) = p^{k-1} a(p)$.  For example, $a(5^k) = 5^{k-1} a(5) = a(5)$ (this is known to be true).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Armed with these insights we should be able to make sense of the families of solutions that appear in our search.  For example, if $n|F(n)$ is a solution, and $k$ is any factor of $F(n)/n$, then $kn$ will also be a solution.  By &quot;primitive&quot; solutions earlier I meant solutions, such as $12$ and $168$ that &lt;em&gt;do not&lt;/em&gt; arise in this fashion, e.g. there is no solution $n$ dividing $12$ (resp. $168$) such that $12/n$ is a factor of $F(n)/n$ (mutatus mutandi for $168$).  The search can be refined in this way to a search only for primitive solutions, and it would be interesting to know their frequency.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-03-22T17:11:40.980" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11151" PostTypeId="1" CreationDate="2014-03-22T15:32:35.267" Score="1" ViewCount="247" Body="&lt;p&gt;I am measuring the electric field in the closure box with aperture and using probe to achieve that. But I have a problem. When I use the probe for measurement, results are not accurate in transient solver. So, I changed the settings in the transient solver &gt; Specials and I use the program with high number of pulses option ( For example, 100000 and more pulses ). But simulation time is really bad and it takes nearly four days to complete with my desktop pc ( and I know that my pc is not as fast as enough, but this is my situation). How can I drop the simulation time to measuring the E-Field with probe?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="7731" LastActivityDate="2014-03-22T15:32:35.267" Title="How can simulation be fast when Electric Field is being measured with probe in CST Microwave Studio?" Tags="|electromagnetism|" AnswerCount="0" CommentCount="1" ClosedDate="2014-03-26T04:13:44.457" ContentLicense="CC BY-SA 3.0" />
  <row Id="11152" PostTypeId="2" ParentId="11144" CreationDate="2014-03-22T19:20:00.127" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://devsim.org&quot; rel=&quot;nofollow&quot;&gt;DEVSIM&lt;/a&gt; is a finite volume code that I've implemented in C++ with a Python interface.  It uses a generalized PDE solver, meaning you can easily implement the Poisson equation in it (Please see the capacitor example in the program documentation).  It can create a rectangular grid (as triangles) or read in a mesh using created in &lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;nofollow&quot;&gt;gmsh&lt;/a&gt;.  You are welcome to try it out to see if it suits your needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are restricted to C (not C++), you could try &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/umfpack/&quot; rel=&quot;nofollow&quot;&gt;UMFPACK&lt;/a&gt; or &lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot; rel=&quot;nofollow&quot;&gt;SuperLU&lt;/a&gt; solvers for doing the sparse matrix calculations with your own implementation.  If you can use C++, you can use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSC&lt;/a&gt;, which could use these libraries as underlying solvers.&lt;/p&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="7681" LastEditDate="2014-03-22T19:33:20.997" LastActivityDate="2014-03-22T19:33:20.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11153" PostTypeId="1" AcceptedAnswerId="11158" CreationDate="2014-03-22T23:27:52.097" Score="11" ViewCount="1621" Body="&lt;p&gt;I am taking a course on scientific computation, and we just went over least squares approximation. My question is specifically about approximating using polynomials. I understand that if you have n+1 data points, you can find a unique polynomial of degree n that describes all of these points. But I can also see why this is not always ideal. You can get a lot of noise in between data points with such an approach. I suppose it's nice to get a lower degree polynomial that estimates your data well enough. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how do you decide in practice what degree of polynomial you are going to use? Is there a rule of thumb, or does it depend solely on the problem at hand? Do we have to take into account various tradeoffs when deciding between more or less degrees? Or am I misunderstanding something here? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="7736" LastActivityDate="2019-01-10T12:32:09.367" Title="Least squares approximation question" Tags="|matlab|least-squares|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11154" PostTypeId="2" ParentId="11086" CreationDate="2014-03-23T01:18:56.487" Score="3" Body="&lt;p&gt;Ok, I fiddled around with it and think I got a workaround. In fig. &lt;a href=&quot;https://i.stack.imgur.com/KmvnJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; we see a graph of $G(i \omega)$ for a typical set of parameters (printed on the graph, you can ignore them) and realize that $G(i \omega)$ falls off as $1/\omega^2$ for $\omega \geq 10^6$. So the biggest contribution to the integral comes from the interval $[0, 10^6]$. Here is a list of values for the integral that I get for this function when I use $\tt{scipy.integrate.quad()}$ with different values for the upper limit $a$:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$a$, result&lt;/li&gt;&#xA;&lt;li&gt;$10^8$, $1.54368741837$&lt;/li&gt;&#xA;&lt;li&gt;$10^9$, $1.54428315319$&lt;/li&gt;&#xA;&lt;li&gt;$10^{10}$, $1.54434272669$&lt;/li&gt;&#xA;&lt;li&gt;$10^{11}$, $-6.61927531679 \cdot 10^{-7}$&lt;/li&gt;&#xA;&lt;li&gt;$10^{12}$, $-6.61927690454 \cdot 10^{-8}$&lt;/li&gt;&#xA;&lt;li&gt;$10^{14}$, $-6.61927691368 \cdot 10^{-10}$&lt;/li&gt;&#xA;&lt;li&gt;numpy.inf, $1.54435019948$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I suppose that the rubbish results starting at $10^{11}$ arise because the $10^6$ is only a very very small percentage of $10^{11}$ and the quad()-algorithm does not sample enough values of $G$ in the essential region of the interval and therefore gives a severly underestimated value of the integral. This problem is somehow circumvented when one uses numpy.inf as an upper limit. But I want to avoid the use of it, because I do not really understand how quad() handles this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to this problem is the following. We rewrite the integral as an initial value problem (IVP) of a first order ODE:&#xA;\begin{align*} I(x) &amp;amp;:= \int_0^x~G(i\omega, \mathbf{v}) d\omega \\&#xA;\Rightarrow \frac{dI}{dx} &amp;amp;= G(i x, \mathbf{v}), \; I(0) = 0,&#xA;\end{align*}&#xA;and we are interested in the value $I(a)$ for $a$ very large. Now we can use an ODE integrator to solve this IVP. I used a Runge-Kutta Cash-Karp method (with adaptive stepsize!) that I translated from the book &quot;Numerical Recipes&quot; (2nd ed.) and the standard ODE solver $\tt{scipy.integrate.odeint()}$. Here are the results for $10^8 - 10^{14}$ in this order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Runge-Kutta Cash-Karp (RKCK):&#xA;1.54368746,  1.5442832 ,  1.54434278,  1.54434874,  1.54434934, 1.5443494&lt;/p&gt;&#xA;&#xA;&lt;p&gt;odeint:&#xA;1.54368718,  1.54428289,  1.54434245,  1.5443484 ,  1.54434901, 1.54434908&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RKCK needs around 100 steps on average to solve the integral, odeint() more than 200 steps. Using an ODE solver seems to resolve the problem because it can adapt its stepsize and sample enough relevant function values to give the correct result of the integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/KmvnJ.png&quot; alt=&quot;The graph of $G(i \omega)$ for a typical set of parameters&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7649" LastActivityDate="2014-03-23T01:18:56.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11156" PostTypeId="1" AcceptedAnswerId="11175" CreationDate="2014-03-23T17:18:14.760" Score="2" ViewCount="378" Body="&lt;p&gt;I have a stochastic process (a Markov chain actually) that has two absorbing states. I am using a difference equation to calculate the first passage time to either of the absorbing states. There are several parameters in the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For set of parameters, the empirical cumulative distribution function looks like the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EZ1jk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;whose probability density function will only one peak. For another set of parameters, I will have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/jE6Jm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;whose PDF will be the superposition of &quot;two distributions&quot;, meaning having two peaks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is there a &quot;reliable&quot;/&quot;good&quot; algorithms/methods to infer the number of distributions, the statistics of each of the well (or not so well) separated distribution?&lt;/p&gt;&#xA;" OwnerUserId="3512" LastActivityDate="2014-03-25T17:01:04.717" Title="Extracting time scales information from empirical cumulative distribution function" Tags="|matlab|data-analysis|stochastic|probability|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11157" PostTypeId="2" ParentId="11153" CreationDate="2014-03-23T17:54:12.093" Score="2" Body="&lt;p&gt;Very unscientific, but a good rule of thumb is that 3rd degree polynomials are typically a good start, and in practice I have never seen anyone use more than a 6th degree one a get good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;ideal&quot; polynomial would be the lowest order one that represents the noiseless data well enough for your purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data is small enough to allow it (it usually is), you can simply try fitting with higher order polynomials until you start seeing oscillations, which tend to be the  sign of &quot;overfitting&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative would be a &lt;a href=&quot;http://en.wikipedia.org/wiki/Smoothing_spline&quot; rel=&quot;nofollow&quot;&gt;smoothing spline&lt;/a&gt;, but it very much depends on the application. Splines and smoothing splines are only good for interpolation. For noisy data I tend to prefer polynomial fits to smoothing splines, but the data I work with are typically well approximated by the polynomials.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-03-23T17:54:12.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11158" PostTypeId="2" ParentId="11153" CreationDate="2014-03-23T18:01:45.930" Score="18" Body="&lt;p&gt;The most important aspect of interpolation and curve fitting is to understand &lt;em&gt;why&lt;/em&gt; high order polynomial fits can be an issue and what the other options are and then you can understand when they are/are not a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few issues with high order polynomials:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Polynomials are naturally oscillatory functions. As the order of the polynomial increases, the number of oscillations increases and these oscillations become more severe. I'm simplifying here, the possibility of multiple and imaginary roots makes it a bit more complex, but the point is the same.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Polynomials approach +/- infinity at a rate equal to the polynomial order as x goes to +/- infinity. This is often not a desired behavior. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Computing polynomial coefficients for high order polynomials is typically an ill conditioned problem. This means that small errors (such as rounding in your computer) can create large changes in the answer. The linear system that must be solved involves a &lt;a href=&quot;http://en.wikipedia.org/wiki/Vandermonde_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vandermonde Matrix&lt;/a&gt; which can easily be ill conditioned.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think that perhaps the heart of this issue is the distinction between &lt;strong&gt;curve fitting&lt;/strong&gt; and &lt;strong&gt;interpolation&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Interpolation&lt;/strong&gt; is used when you believe that your data is very accurate so you want your function to exactly match the data points. When you need values between your data points it's typically best to use a smooth function that matches the local trend of the data. Cubic or Hermite splines are often a good choice for this type of problem since they are much less sensitive to non-local (meaning at data points far away from a given point) changes or errors in the data and are less oscillatory than a polynomial. Consider the following data set:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = 1   2   3   4   5   6   7   8   9  10&#xA;y = 1   1 1.1   1   1   1   1   1   1   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rKL9W.png&quot; alt=&quot;Interpolation&quot;&gt;&lt;br&gt;&#xA;A polynomial fit has much larger oscillations, especially near the edges of the data set, than a Hermite spline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, least squares approximation is a &lt;strong&gt;curve fitting&lt;/strong&gt; technique. Curve fitting is used when you have some idea of the expected functionality of your data, but you don't need your function to exactly pass through all the data points. This is typical when the data may contain measurement errors or other imprecisions or when you wish to extract the general trend of the data. Least squares approximation is most often introduced in a course by using polynomials for the curve fitting because this results in a linear system that is relatively simple to solve using the techniques you likely learned earlier in your course. However, least squares techniques are much more general than just polynomial fits and can be used to fit any desired function to a data set. For example, if you expect an exponential growth trend in your data set, you can fit an exponential curve using least squares or linearized least squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, choosing the correct function to fit your data is as important as correctly performing the interpolation or least squares computations. Doing so even allows the possibility of (cautious) extrapolation. Consider the following situation. Given population data (in millions of people) for the US from 2000-2010:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Year:  2000   2001   2002   2003   2004   2005   2006   2007   2008   2010&#xA;Pop.: 284.97 287.63 290.11 292.81 295.52 298.38 301.23 304.09 306.77 309.35&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using an exponential linearized least squares fit &lt;code&gt;N(t)=A*exp(B*t)&lt;/code&gt; or a 10th order polynomial interpolant gives the following results:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/fMMua.png&quot; alt=&quot;Population plot&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;US population growth is not quite exponential, but I'll let you be the judge of the better fit.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="22284" LastEditDate="2019-01-10T12:32:09.367" LastActivityDate="2019-01-10T12:32:09.367" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11159" PostTypeId="2" ParentId="7912" CreationDate="2014-03-24T00:40:55.873" Score="3" Body="&lt;p&gt;To find the weaks forms we use the identity&#xA;$$&#xA;  \nabla\cdot(\rho\,\nabla\phi) = \rho\,\nabla^2\phi + \nabla\rho \cdot \nabla\phi&#xA;$$&#xA;and its vector counterpart&#xA;$$&#xA;  \nabla\cdot(\varphi\,\mathbf{v}) = \varphi\,(\nabla\cdot\mathbf{v}) + (\nabla\varphi) \cdot \mathbf{v} \,.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Then the first equation can be written as&#xA;$$&#xA;  \partial_{t}\rho+2\,\nabla\cdot(\rho\,\nabla\phi)+2\sigma\rho\left(\rho-1\right)=0 \,.&#xA;$$&#xA;If we consider only the spatial terms and use a weighting function $w$, the weak form is&#xA;$$&#xA; \int_\Omega (\partial_{t}\rho)\,w\,d\Omega + 2\int_\Omega \left[\nabla\cdot(\rho\,\nabla\phi)+\sigma\rho\left(\rho-1\right)\right]w\,d\Omega=0 \,.&#xA;$$&#xA;Using the vector identity above, we get&#xA;$$&#xA; \int_\Omega (\partial_{t}\rho)\,w\,d\Omega + 2\int_\Omega \left[\nabla\cdot(w\,\rho\,\nabla\phi)-\nabla w\cdot(\rho\,\nabla\phi) + w\sigma\rho\left(\rho-1\right)\right]\,d\Omega=0 \,.&#xA;$$&#xA;Apply the divergence theorem to get&#xA;$$&#xA; \int_\Omega (\partial_{t}\rho)\,w\,d\Omega + 2\int_\Gamma (w\,\rho\,\nabla\phi)\cdot\mathbf{n}\,d\Gamma - 2\int_\Omega \left[\nabla w\cdot(\rho\,\nabla\phi) - w\sigma\rho\left(\rho-1\right)\right]\,d\Omega=0 \,.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Using the same identity as before, we also have&#xA;$$&#xA;  \nabla\phi\cdot\nabla\phi = \nabla\cdot(\phi\,\nabla\phi) - \phi\,\nabla\cdot(\nabla\phi)&#xA;$$&#xA;and&#xA;$$&#xA;  \nabla\cdot\left(\frac{1}{\rho}\,\nabla\rho\right) = \frac{\nabla^2\rho}{\rho} -\frac{\nabla\rho \cdot \nabla\rho}{\rho^2}\,.&#xA;$$&#xA;Assuming that the extra factor of $1/2$ in the second equation is a typo, we can write that equation as&#xA;$$&#xA;\partial_{t}\phi+\nabla\cdot(\phi\,\nabla\phi) - \phi\,\nabla\cdot(\nabla\phi)-\frac{1}{2}\nabla\cdot\left(\frac{1}{\rho}\,\nabla\rho\right)+\rho-1=0&#xA;$$&#xA;It is here that the appropriate choice of terms to consider at the boundaries becomes important.  The choice is typically determined by the ease of application of the boundary conditions and how the two sets of equations are coupled in the numerical method.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2014-03-24T00:40:55.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11160" PostTypeId="1" CreationDate="2014-03-24T05:11:01.493" Score="3" ViewCount="6289" Body="&lt;p&gt;I am plotting a saturation field (as the solution of a finite element problem, using &lt;code&gt;trisurf&lt;/code&gt; in MATLAB). Ideally, I would like to obtain the two-dimensional profile of this surface along $y=0$. See Figure 4 of this &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.100.4825&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; for an example. I am not really familiar with MATLAB's plotting routines but I noticed there's a feature to grab a data point on the surface using the cursor, so I'm hoping this is a built-in feature.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastActivityDate="2014-11-15T09:10:08.083" Title="Obtaining a 2D profile from a 3D surface" Tags="|matlab|plotting|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11161" PostTypeId="1" AcceptedAnswerId="11162" CreationDate="2014-03-24T10:31:35.783" Score="0" ViewCount="468" Body="&lt;p&gt;I searched for Gilbert-Peierls algorithm, but I haven't found anything useful (well, I found &lt;a href=&quot;http://sugandima.wikia.com/wiki/Methods_of_solving_Ax=b_and_their_Matlab_implementations&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;, but it's not working as it should). I think the problem is the second part, and also that those lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;U(1:k, k) = x(1:k);&#xA;L(k:N, k) = x(k:N)/U(k, k);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;should actually be (according to &lt;a href=&quot;http://sugandima.wikia.com/wiki/Category%3aGilbert-Peierls_algorithm_for_LU_Decomposition&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; example):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;U(1:N, k) = x(1:N);&#xA;L(k:N, k) = x(k:N)/U(k, k);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also, in that example, L is the identity matrix, which I find a bit strange. Could someone describe the algorithm, please? (with or without code)&lt;/p&gt;&#xA;" OwnerDisplayName="user7729" LastActivityDate="2014-03-24T11:30:43.050" Title="Gilbert-Peierls algorithm for LU Decomposition" Tags="|algorithms|matlab|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11162" PostTypeId="2" ParentId="11161" CreationDate="2014-03-24T11:30:43.050" Score="0" Body="&lt;p&gt;Why don't you check this out:&#xA;&lt;a href=&quot;http://cci.lbl.gov/cctbx_sources/scitbx/sparse/lu_factorization.h&quot; rel=&quot;nofollow&quot;&gt;http://cci.lbl.gov/cctbx_sources/scitbx/sparse/lu_factorization.h&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-03-24T11:30:43.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11163" PostTypeId="2" ParentId="11160" CreationDate="2014-03-24T12:44:04.220" Score="1" Body="&lt;p&gt;The simplest method would be to use the built in &lt;code&gt;griddata&lt;/code&gt; function. The one issue is that this function does not take a triangle list as input and instead uses a Delaunay triangulation for interpolation. If your mesh is not a Delaunay triangulation the results may be slightly different than directly interpolating from your triangles. Usage is follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xv = linspace(xmin,xmax,nx); %//desired x points&#xA;yv = zeros(size(x)); %// desired y points&#xA;&#xA;zv = griddata(x,y,z,xv,yv,'linear');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Take a look at the help documentation for options other than linear interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to be sure to use your mesh rather than a Delaunay triangulation, you may need something more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;__EDIT__&lt;/strong&gt;&lt;br&gt;&#xA;Unless you have an efficient search tree to find which element contains a given point, I think any efficient solution should probably be based on finding the elements that straddle the desired plane (y=0) and then interpolating only on those elements rather than choosing points and then trying to figure out which element they're in. This will eliminate your nested loop which can be quite slow in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One such solution would be to compute the intersections of your triangle edges with the plane y=0. If you need more resolution and have nonlinear elements you could also interpolate within those triangles using your finite element solution within that element. This can be done as follows (and could be much faster than your posted solution for large numbers of elements):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%//loop over elements&#xA;xsave=[]; zsave=[];&#xA;for i=1:n_elements&#xA;    [x y z]=getVertices(elements,i);&#xA;    if min(y)*max(y)&amp;lt;=0 %//check if the element intersects y=0&#xA;        %// Find intersections between edges and y=0 plane&#xA;        %// The edges can be parameterized for s in [0, 1] as:&#xA;        %//   x(s) = x(1) + s*(x(2)-x(1));&#xA;        %//   y(s) = y(1) + s*(y(2)-y(1));&#xA;        %//   z(s) = z(1) + s*(z(2)-z(1));&#xA;        %// Then just solve for s setting y(s)=0&#xA;        s1 = -y(1)/(y(2)-y(1));&#xA;        s2 = -y(2)/(y(3)-y(2));&#xA;        s3 = -y(3)/(y(1)-y(3));&#xA;&#xA;        %// Save the desired points:&#xA;        if s1&amp;gt;=0 &amp;amp;&amp;amp; s1&amp;lt;=1&#xA;            xsave(end+1) = x(1) + s1*(x(2)-x(1));&#xA;            zsave(end+1) = z(1) + s1*(z(2)-z(1));&#xA;        end&#xA;&#xA;        if s2&amp;gt;=0 &amp;amp;&amp;amp; s2&amp;lt;=1&#xA;            xsave(end+1) = x(2) + s2*(x(3)-x(2));&#xA;            zsave(end+1) = z(2) + s2*(z(3)-z(2));&#xA;        end&#xA;&#xA;        if s3&amp;gt;=0 &amp;amp;&amp;amp; s3&amp;lt;=1&#xA;            xsave(end+1) = x(3) + s3*(x(1)-x(3));&#xA;            zsave(end+1) = z(3) + s3*(z(1)-z(3));&#xA;        end&#xA;    end&#xA;end&#xA;&#xA;%//sort the resulting points, throw out duplicates and plot&#xA;[xsave,I,~] = unique(xsave);&#xA;zsave = zsave(I);&#xA;&#xA;plot(xsave,zsave)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As a fair warning, I haven't tested the above code and it could be re-written slightly to remove some of the duplication, but I think it's most clear as is.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-03-25T21:16:03.460" LastActivityDate="2014-03-25T21:16:03.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11164" PostTypeId="2" ParentId="382" CreationDate="2014-03-24T17:14:15.900" Score="0" Body="&lt;p&gt;The reason to choose the following is because it does not depend on additional libraries. So, this makes it easier to support multiple targets such as arm or blackfin processors and reduces dependencies and ABI trouble.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Multinomial&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;&amp;lt;random&amp;gt;&lt;/code&gt; there is a class: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;class IntType = int&amp;gt; class discrete_distribution;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The term discrete distribution is actually incorrect, but we cannot expect STL programmers to be experts in statistics. :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, this is a &lt;strong&gt;multinomial distribution&lt;/strong&gt; without the preceding $1/B(x)$ factor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = \prod_i p_i^{x_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The class accepts values $w_i$ which will be normalized such they form an actual probability $\sum_i p_i = 1$. Quite convenient in many cases.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Dirichlet&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I haven't found a header file for the Dirichlet distribution. However, the Dirichlet is of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = \frac{1}{B(\alpha)} \prod_i x_i^{\alpha_i-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you might want to check if you also can use the above class for this. As indicated in one of the comments. If the Dirichlet is used as a prior for a multinomial distribution, there are closed forms available for the posterior.&lt;/p&gt;&#xA;" OwnerUserId="7749" LastEditorUserId="7749" LastEditDate="2014-04-02T19:49:37.397" LastActivityDate="2014-04-02T19:49:37.397" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11165" PostTypeId="1" CreationDate="2014-03-24T18:30:34.180" Score="8" ViewCount="204" Body="&lt;p&gt;I asked a similar question on &lt;a href=&quot;http://physics.stackexchange.com&quot;&gt;physics.stackexchange&lt;/a&gt;, being ignorant about this website. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am basically looking for an efficient way to implement gravitational fields. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a huge 2D space, with thousands of objects in it. I then need to simulate how these objects are effected by each other's gravity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought it'd be possible to sort the objects into collections, and check every object outside that collection against that collection, and not every individual object inside of it. I soon came to the realization that this simply wasn't possible. The gravitational field of multiple objects cannot represented as one uniform field, calculated with only one mass and distance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every object inside the simulation can be considered a sphere. I am fine with approximations, as long as it looks reasonably realistic. &lt;/p&gt;&#xA;" OwnerUserId="7752" LastEditorUserId="7752" LastEditDate="2014-03-24T18:38:23.410" LastActivityDate="2014-03-24T18:43:42.767" Title="Efficient Gravitational Field Implementation" Tags="|optimization|simulation|approximation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11166" PostTypeId="2" ParentId="11165" CreationDate="2014-03-24T18:43:42.767" Score="5" Body="&lt;p&gt;You could try using a library that implements the Fast Multipole Method (FMM), which should drastically reduce the amount of memory you need and will decrease the complexity of matrix-vector products from $\mathcal{O}(N^{2})$ to $\mathcal{O}(N)$. It is difficult to implement, but there should be some libraries out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another algorithm for N-body simulation is Barnes-Hut, which is easier to implement, and probably also has library implementations available. It is considered less efficient (in the asymptotic sense) than FMM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your domain is periodic, maybe you could do something like particle mesh Ewald summation? (I know less about this approach.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-03-24T18:43:42.767" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11167" PostTypeId="1" AcceptedAnswerId="11178" CreationDate="2014-03-24T19:17:36.890" Score="9" ViewCount="4582" Body="&lt;pre&gt;&lt;code&gt;double trap(double func(double), double b, double a, double N) {&#xA;  double j;&#xA;  double s;&#xA;  double h = (b-a)/(N-1.0); //Width of trapezia&#xA;&#xA;  double func1 = func(a);&#xA;  double func2;&#xA;&#xA;  for (s=0,j=a;j&amp;lt;b;j+=h){&#xA;    func2 = func(j+h);&#xA;    s = s + 0.5*(func1+func2)*h;&#xA;    func1 = func2;&#xA;  }&#xA;&#xA;  return s;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above is my C++ code for a 1D numerical integration (using the extended trapezium rule) of &lt;code&gt;func()&lt;/code&gt; between limits $[a,b]$ using $N-1$ trapezia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am actually doing a 3D integration, where this code is called recursively. I work with $N = 50$ giving me decent results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than reducing $N$ further, is anybody able to suggest how to optimise the code above so that it runs faster? Or, even, can suggest a faster integration method?&lt;/p&gt;&#xA;" OwnerUserId="7753" LastEditorUserId="8168" LastEditDate="2014-05-11T22:10:32.783" LastActivityDate="2014-05-11T22:10:32.783" Title="Is it possible to optimise this integration code so that it runs faster?" Tags="|c++|performance|" AnswerCount="10" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11168" PostTypeId="2" ParentId="11167" CreationDate="2014-03-24T19:30:03.693" Score="9" Body="&lt;p&gt;Chances are that the evaluation of the functions is the most time consuming part of this computation.  If that's the case, then you should focus on improving the speed of func() rather than trying to speed up the integration routine itself.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the properties of func(), it's also likely that you could get a more precise evaluation of the integral with fewer function evaluations by using a more sophisticated integration formula.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-03-24T19:30:03.693" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11169" PostTypeId="2" ParentId="11156" CreationDate="2014-03-24T19:40:36.657" Score="2" Body="&lt;p&gt;The cumulative distribution function is the integral (antiderivative) of the probability distribution function. In other words, the PDF is the derivative of the CDF. You can therefore compute the PDF by computing the derivative of your data, for example by forming a difference quotient to approximate the derivative from a finite set of points.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-24T19:40:36.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11170" PostTypeId="1" CreationDate="2014-03-24T23:43:34.067" Score="1" ViewCount="41" Body="&lt;p&gt;Suppose I have a box &lt;span class=&quot;math-container&quot;&gt;$D \subset \Bbb{R}^2$&lt;/span&gt; (compact set). Denote &lt;span class=&quot;math-container&quot;&gt;$\mathcal{P}= \{ (\Omega_1,...,\Omega_n) : \bigcup_{i=1}^n \Omega_i = D,\ \Omega_i \cap \Omega_j =\emptyset\}$&lt;/span&gt; the family of partitions of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; into open sets (the sets can overlap on their boundaries...)&lt;/p&gt;&#xA;&lt;p&gt;Consider a functional &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; defined on the subsets of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$\lim_{|\Omega|\to 0}F(\Omega)=+\infty$&lt;/span&gt;. Suppose that there exists a partition &lt;span class=&quot;math-container&quot;&gt;$(\Omega_i)$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$\sum F(\Omega_i) &amp;lt; F(D)$&lt;/span&gt; (so that the whole box is not optimal).&lt;/p&gt;&#xA;&lt;p&gt;Then I consider the problem of finding the partition &lt;span class=&quot;math-container&quot;&gt;$(\Omega_i)$&lt;/span&gt; which solves&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \min_{(\Omega_i) \in \mathcal{P}}\sum_{i=1}^n F(\Omega_i).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The difference between this problem and the problems I've seen and worked until now is that here the number of members of the partition is not fixed.&lt;/p&gt;&#xA;&lt;p&gt;The first condition tells us that &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is bounded above, since many parts means smaller volumes, which mean great &lt;span class=&quot;math-container&quot;&gt;$F(\Omega)$&lt;/span&gt;. The second condition says that &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is not the solution. In what I have in mind &lt;span class=&quot;math-container&quot;&gt;$F(\Omega)=G(\Omega)+\alpha \frac{Per(\Omega)}{|\Omega|}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$G \geq 0$&lt;/span&gt; is something complicated.&lt;/p&gt;&#xA;&lt;p&gt;And now the question...&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you know some references of articles which treat this kind of problems of optimal partitioning with variable number of parts?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you know any available programs/toolboxes which are good for manipulating partitioning problems?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-03-24T23:43:34.067" Title="Optimal partition - variable number of parts" Tags="|optimization|reference-request|partitioning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11171" PostTypeId="2" ParentId="11167" CreationDate="2014-03-25T02:02:53.667" Score="7" Body="&lt;p&gt;Possible? Yes. Useful? No. The optimizations I'm going to list here are unlikely to make more than a tiny fraction of a percent difference in the runtime. A good compiler may already do these for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, looking at your inner loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    for (s=0,j=a;j&amp;lt;b;j+=h){&#xA;        func2 = func(j+h);&#xA;        s = s + 0.5*(func1+func2)*h;&#xA;        func1 = func2;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At every loop iteration you perform three math operations that can be brought outside: adding &lt;code&gt;j + h&lt;/code&gt;, multiplication by &lt;code&gt;0.5&lt;/code&gt;, and multiplication by &lt;code&gt;h&lt;/code&gt;. The first you can fix by starting your iterator variable at &lt;code&gt;a + h&lt;/code&gt;, and the others by factoring out the multiplications:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    for (s=0, j=a+h; j&amp;lt;=b; j+=h){&#xA;        func2 = func(j);&#xA;        s += func1+func2;&#xA;        func1 = func2;&#xA;    }&#xA;    s *= 0.5 * h;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Though I would point out that by doing this, due to floating point roundoff error it is possible to miss the last iteration of the loop. (This was also an issue in your original implementation.) To get around that, use an &lt;code&gt;unsigned int&lt;/code&gt; or &lt;code&gt;size_t&lt;/code&gt; counter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    size_t n;&#xA;    for (s=0, n=0, j=a+h; n&amp;lt;N; n++, j+=h){&#xA;        func2 = func(j);&#xA;        s += func1+func2;&#xA;        func1 = func2;&#xA;    }&#xA;    s *= 0.5 * h;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As Brian's answer says, your time is better spent optimizing the evaluation of the function &lt;code&gt;func&lt;/code&gt;. If the accuracy of this method is sufficient, I doubt you'll find anything faster for the same &lt;code&gt;N&lt;/code&gt;. (Though you could run some tests to see if e.g. Runge-Kutta lets you lower &lt;code&gt;N&lt;/code&gt; enough that the overall integration takes less time without sacrificing accuracy.)&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2014-03-25T02:02:53.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11172" PostTypeId="2" ParentId="11167" CreationDate="2014-03-25T05:19:28.507" Score="3" Body="&lt;p&gt;If your function is a polynomial, possibly weighted by some function (e.g. a gaussian), you can do an exact integration in 3d directly with a cubature formula (e.g. &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/c_src/stroud/stroud.html&quot; rel=&quot;nofollow&quot;&gt;http://people.sc.fsu.edu/~jburkardt/c_src/stroud/stroud.html&lt;/a&gt; ) or with a sparse grid (e.g. &lt;a href=&quot;http://tasmanian.ornl.gov/&quot; rel=&quot;nofollow&quot;&gt;http://tasmanian.ornl.gov/&lt;/a&gt; ). These methods simply specify a set of points and weights to multiply the function value by, so they are very fast. If your function is smooth enough to be approximated by polynomials, then these methods can still give a very good answer.  The formulas are specialized to the type of function you're integrating, so it may take some digging to find the right one.&lt;/p&gt;&#xA;" OwnerUserId="1665" LastActivityDate="2014-03-25T05:19:28.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11174" PostTypeId="1" CreationDate="2014-03-25T06:51:31.857" Score="2" ViewCount="81" Body="&lt;p&gt;The number 1 response to this question was great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/114/is-algorithmic-analysis-by-flop-counting-obsolete&quot;&gt;Is algorithmic analysis by flop-counting obsolete?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the only problem is that I don't really understand it. Here's my example of why it's not clear to me. Please let me know what I'm doing wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying the above analysis to the FFT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I want to compute the fft of a vector of doubles length n. Then I need to do approximately nlog(n) flops. The bytes required to do the computation are 8n, assuming a 64bit system. So $\beta$ = log(n)/8. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose my processor works at 2GHz=Fmax and I have 8Gbytes=Bmax of RAM. Then, I have equality when: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;log(n)/8 (flops/byte) = (Bmax/Fmax) (bytes/Hz) = 4  This implies&lt;/p&gt;&#xA;&#xA;&lt;p&gt;log(n) = 32 =&gt; n = 2^32 = 4,294,967,296&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I should be able to compute the fft of a vector of size n? I don't know. Also, why don't the units cancel?  &lt;/p&gt;&#xA;" OwnerUserId="7756" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-03-25T12:39:57.080" Title="Response to quantifying algorithm bottlenecks memory vs. computation." Tags="|algorithms|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11175" PostTypeId="2" ParentId="11156" CreationDate="2014-03-25T07:12:12.900" Score="1" Body="&lt;p&gt;By taking differences of the cumulative distribution function $F$, you can find the probability density function $p$. What you're looking for are local maxima of $p$; from your second plot, it looks like there is one big maximum at $t = 7$ and another at $t \approx 1200$. You can spot these from the CDF directly as inflection points, where $F$ goes from being convex to being concave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Bimodal_distribution&quot; rel=&quot;nofollow&quot;&gt;bimodal distribution&lt;/a&gt;, and each peak is called a &quot;mode&quot;. You can count the number of modes reliably by looking for the number of local maxima in the PDF. Of course, some very small local maxima might just be statistical artifacts, so you need a criterion for ruling out spurious ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to say more about the statistics of the consensus time $\tau$, you can reasonably guess that your PDF is of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p(t) = (1-\lambda)q(\tau; \theta_1) + \lambda q(\tau; \theta_2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $q$ is a unimodal distribution depending on some parameters $\theta$. Then you can try and find a best value of $\theta_1$, $\theta_2$ and $\lambda$. For example, $q$ could be the hitting time PDF for the gambler's ruin problem, and the parameters $\theta$ represent the fairness of the game and the gambler's initial capital.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2014-03-25T17:01:04.717" LastActivityDate="2014-03-25T17:01:04.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11176" PostTypeId="2" ParentId="11160" CreationDate="2014-03-25T07:27:19.323" Score="2" Body="&lt;p&gt;Well I've figured out a way to do it. It's a bit convoluted but I figured I'd share it here anyway. I'm positive there is a faster way to do this but since I only need to really do this a couple of times per simulation, it's not a huge deal. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% the line y = 0&#xA;xy = [linspace(0,1,100)', zeros(100,1)];&#xA;zz = zeros(100,1);&#xA;&#xA;for i=1:100&#xA;    for j=1:nElements&#xA;        % check if the point xy(i,:) is in or on the jth element (I used barycentric coordinates to do this)&#xA;        if (xy(i,:) in element j)&#xA;            % compute zz(j) (in my case, using finite element solution on element j)&#xA;        end&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;nElements&lt;/code&gt; is the total number of mesh elements. Determining whether the point is in the element or on its boundary is very simple via barycentric coordinates. See &lt;a href=&quot;http://en.wikipedia.org/wiki/Barycentric_coordinate_system#Determining_whether_a_point_is_inside_a_triangle&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, it's going to check over all the elements until it finds the correct one but it gets the job done. Below is an example I made taken from a surface plot along $y=0$.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/nvLcc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QFLxF.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has a faster/more elegant solution, please do share.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastActivityDate="2014-03-25T07:27:19.323" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11177" PostTypeId="2" ParentId="11167" CreationDate="2014-03-25T09:26:42.330" Score="4" Body="&lt;p&gt;There are several changes I would recommend to improve the computation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For performance and accuracy, use &lt;a href=&quot;http://en.cppreference.com/w/cpp/numeric/math/fma&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;std::fma()&lt;/code&gt;&lt;/a&gt;, which performs a &lt;a href=&quot;http://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation&quot; rel=&quot;nofollow&quot;&gt;fused multiply-add&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;For performance, defer multiplying the area of each trapezoid by 0.5 — you can do it once at the end.&lt;/li&gt;&#xA;&lt;li&gt;Avoid repeated addition of &lt;code&gt;h&lt;/code&gt;, which could accumulate round-off errors.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In addition, I would make several changes for clarity:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Give the function a more descriptive name.&lt;/li&gt;&#xA;&lt;li&gt;Swap the order of &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; in the function signature.&lt;/li&gt;&#xA;&lt;li&gt;Rename &lt;code&gt;N&lt;/code&gt; → &lt;code&gt;n&lt;/code&gt;, &lt;code&gt;h&lt;/code&gt; → &lt;code&gt;dx&lt;/code&gt;, &lt;code&gt;j&lt;/code&gt; → &lt;code&gt;x2&lt;/code&gt;, &lt;code&gt;s&lt;/code&gt; → &lt;code&gt;accumulator&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Change &lt;code&gt;n&lt;/code&gt; to an &lt;code&gt;int&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Declare variables in a tighter scope.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;cmath&amp;gt;&#xA;&#xA;double trapezoidal_integration(double func(double), double a, double b, int n) {&#xA;    double dx = (b - a) / (n - 1);   // Width of trapezoids&#xA;&#xA;    double func_x1 = func(a);&#xA;    double accumulator = 0;&#xA;&#xA;    for (int i = 1; i &amp;lt;= n; i++) {&#xA;        double x2 = a + i * dx;      // Avoid repeated floating-point addition&#xA;        double func_x2 = func(x2);&#xA;        accumulator = std::fma(func_x1 + func_x2, dx, accumulator); // Fused multiply-add&#xA;        func_x1 = func_x2;&#xA;    }&#xA;&#xA;    return 0.5 * accumulator;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7758" LastActivityDate="2014-03-25T09:26:42.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11178" PostTypeId="2" ParentId="11167" CreationDate="2014-03-25T10:06:39.560" Score="5" Body="&lt;p&gt;Mathematically, your expression is equivalent to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I = h \left(\frac{1}{2}f_1 + f_2 + f_3 +...+f_{n-1} + \frac{1}{2}f_n \right) + O\left(\frac{(b-a)^3 f''}{n^2} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you could implement that. As it was said, the time is probably dominated by the function evaluation, so to get the same accuracy, you can use a better integration method that requires less function evaluations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gaussian quadrature is, in modern days, bit more than a toy; only useful if you require &lt;em&gt;very&lt;/em&gt; few evaluations. If you want something easy to implement, you can use Simpson's rule, but I wouldn't go further than order $1/N^3$ without a good reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the function's curvature changes a lot, you could use an adaptative step routine, that would select a larger step when the function is flat, and a smaller more accurate one when the curvature is higher.&lt;/p&gt;&#xA;" OwnerUserId="7759" LastActivityDate="2014-03-25T10:06:39.560" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11179" PostTypeId="2" ParentId="11174" CreationDate="2014-03-25T11:33:39.610" Score="1" Body="&lt;p&gt;Matt's answer over there isn't about whether or not you &lt;em&gt;can&lt;/em&gt; compute a problem of a particular size. It's about whether your algorithm is likely to be memory access limited or compute limited on a particular architecture. So, backing $n$ out of this bound doesn't  make sense. $n$ is an input along with $B_{max}$ and $F_{max}$.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-03-25T11:33:39.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11180" PostTypeId="2" ParentId="11167" CreationDate="2014-03-25T12:06:45.893" Score="1" Body="&lt;p&gt;So, as has been pointed out in other answers, this depends heavily on how expensive your function is. Optimizing your trapz code is only worth it if it is really your bottleneck. If it's not completely obvious, you should check this by profiling your code (tools like Intels V-tune, Valgrind or Visual Studio can do this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would however suggest a completely different approach: &lt;a href=&quot;http://en.wikipedia.org/wiki/Monte_Carlo_integration&quot; rel=&quot;nofollow&quot;&gt;Monte Carlo integration&lt;/a&gt; . Here you simply approximate the integral by sampling your function at random points adding the results. See &lt;a href=&quot;http://owww.phys.au.dk/~fedorov/Numeric/now/montecarlo.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; pdf in addition to the wiki page for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is works extremely well for high dimensional data, typically much better than the quadrature methods used in 1-d integration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple case is very easy to implement (see the pdf), just be careful that the standard random function in c++98 is quite bad both performance and quality wise. In c++11, you can use the Mersenne Twister in .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your function has a lot of variation in some areas and less in others, consider using stratified sampling. I would recommend using the  &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Monte-Carlo-Integration.html&quot; rel=&quot;nofollow&quot;&gt;GNU scientific library&lt;/a&gt;, rather than writing your own though.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-03-25T12:06:45.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11181" PostTypeId="2" ParentId="11174" CreationDate="2014-03-25T12:39:57.080" Score="2" Body="&lt;p&gt;You misunderstood what Matt wanted to say about $\beta$. It's not the ratio of total number of operations divided by amount of memory required for the algorithm (or the other way around) but instead the ratio of total number of operations divided by total number of memory accesses. In your case, you may need $8n$ memory locations, but you will most certainly need to read every location more than just once. In fact, in most applications, the number of memory accesses is proportional to the number of floating point operations, and this ratio is then $\beta$. This factor is different for different algorithms, however.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-25T12:39:57.080" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11182" PostTypeId="2" ParentId="11167" CreationDate="2014-03-25T13:42:32.377" Score="1" Body="&lt;blockquote&gt;I am actually doing a 3D integration, where this code is called recursively.&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&quot;recursively&quot; is the key.  You are either going through a large data set and considering a lot of data more than once, or you are actually generating your data set yourself from (piecewise?) functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recursively evaluated integrations will be ridiculously expensive, and ridiculously imprecise as the powers increase in recursion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a model for interpolating your data set and do a piecewise symbolic integration.  Since a lot of data is then collapsing into coefficients of base functions, the complexity for deeper recursion grows polynomially (and usually rather low powers) rather than exponentially.  And you get &quot;exact&quot; results (you still need to figure out good evaluation schemes to get reasonable numeric performance, but it should still be rather feasible to get better than trapezoidal integration).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you take a look at the error estimates for trapezoidal rules, you'll find that they are related to some derivative of the involved functions, and if the integration/definition is done recursively, the functions will not tend to have well-behaved derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your only tool is a hammer, every problem looks like a nail.  While you barely touch upon the problem in your description, I have the suspicion that applying the trapezoidal rule &lt;em&gt;recursively&lt;/em&gt; is a bad match: you get an explosion of both inaccuracy and computational requirements.&lt;/p&gt;&#xA;" OwnerUserId="7762" LastActivityDate="2014-03-25T13:42:32.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11183" PostTypeId="2" ParentId="11167" CreationDate="2014-03-25T13:46:19.717" Score="1" Body="&lt;p&gt;the original code evaluates the function at each N points, then adds the values up, and multiplies the sum by the step size. the only trick is that the values at the beginning and the end are added with weight $1/2$, while all points inside are added with full weight. actually, they are also added with weight $1/2$ but twice. instead of adding them twice, add them only once with full weight. factor out the multiplication by the step size outside of the loop. that's all that can be done to speed this up, really.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    double trap(double func(double), double b, double a, double N){&#xA;double j, s;&#xA;double h = (b-a)/(N-1.0); //Width of trapezia&#xA;&#xA;double s = 0;&#xA;j = a;&#xA;for(i=1; i&amp;lt;N-1; i++){&#xA;  j += h;&#xA;  s += func(j);&#xA;}&#xA;s += (func(a)+func(b))/2;&#xA;&#xA;return s*h;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7470" LastEditorUserId="7470" LastEditDate="2014-03-26T20:11:45.997" LastActivityDate="2014-03-26T20:11:45.997" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11184" PostTypeId="2" ParentId="11167" CreationDate="2014-03-25T18:30:09.637" Score="3" Body="&lt;p&gt;When you try to calculate an integral numerically, you try to get the precision that you want with the smallest possible effort, or alternatively, try to get the highest possible precision with a fixed effort. You seem to ask how to make the code for one particular algorithm run as fast as possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That may give you some little gain, but it will be little. There are much more efficient methods for numerical integration. Google for &quot;Simpson's rule&quot;, &quot;Runge-Kutta&quot;, and &quot;Fehlberg&quot;. They all work quite similar by evaluating some values of the function and cleverly adding multiples of those value, producing much smaller errors with the same number of function evaluations, or the same error with a much smaller number of evaluations. &lt;/p&gt;&#xA;" OwnerUserId="7765" LastActivityDate="2014-03-25T18:30:09.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11188" PostTypeId="2" ParentId="11047" CreationDate="2014-03-25T23:04:55.667" Score="2" Body="&lt;p&gt;You may interpret both images as individual clustering problems: You can then calculate an a posteriori probability of beeing marked/ blood vessels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case both images are properly aligned, you may now calculate a distance measure between the two matrices of a posteriori probabilities. A common distance measure is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence&quot; rel=&quot;nofollow&quot;&gt;Kullback-Leibler&lt;/a&gt; divergence.&lt;/p&gt;&#xA;" OwnerUserId="7769" LastActivityDate="2014-03-25T23:04:55.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11189" PostTypeId="1" AcceptedAnswerId="11190" CreationDate="2014-03-26T00:31:48.773" Score="3" ViewCount="2249" Body="&lt;p&gt;How can a sparse matrix - matrix product be calculated? I know the 'classic' / mathematical way of doing it, but it seems pretty inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about storing the first  matrix in CSR form and the second one in CSC form, so since the row and column vectors are sorted I won't have to search for a specific row / column I need, but I guess that doesn't help much.&lt;/p&gt;&#xA;" OwnerDisplayName="user7729" LastEditorDisplayName="user7729" LastEditDate="2014-03-26T00:43:56.200" LastActivityDate="2014-03-26T01:52:50.213" Title="Sparse matrix - matrix multiplication" Tags="|sparse-matrix|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11190" PostTypeId="2" ParentId="11189" CreationDate="2014-03-26T01:52:50.213" Score="5" Body="&lt;p&gt;It is almost always inefficient to do a sparse-sparse matrix product $C=AB$. It is almost always more efficient to simply think of $C$ as an operator that you can apply to vectors, i.e., to compute $y=Cx$. Whenever such an operation is required, you compute it in two steps: $y=A(Bx)$, i.e., you simply do two matrix-vector products in a row to get the result. For this, it is not necessary to actually form the product matrix $AB$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you absolutely must form the product, take a look at &lt;a href=&quot;https://svn.dealii.org/trunk/deal.II/include/deal.II/lac/sparse_matrix.templates.h&quot; rel=&quot;noreferrer&quot;&gt;https://svn.dealii.org/trunk/deal.II/include/deal.II/lac/sparse_matrix.templates.h&lt;/a&gt; and search for the mmult() function.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-26T01:52:50.213" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11191" PostTypeId="2" ParentId="11153" CreationDate="2014-03-26T02:40:37.083" Score="1" Body="&lt;p&gt;A decently-effective approach that I've used with polynomial approximation is to calculate the least-squares polynomials for various degrees (e.g., from 1 to 10) and then choose the curve that minimizes the mean squared error at points &lt;em&gt;halfway between&lt;/em&gt; consecutive (when sorted by x) points in your data set.  This helps to rule out curves with too-severe oscillations.&lt;/p&gt;&#xA;" OwnerUserId="7771" LastActivityDate="2014-03-26T02:40:37.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11193" PostTypeId="1" CreationDate="2014-03-26T15:57:28.180" Score="6" ViewCount="163" Body="&lt;p&gt;I would like to numerically solve an equation of a type as shown below. Does anyone of you have an idea how to approach such a problem? Any links to literature or for further reading would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;I search a function $f(\vec{x})$ with $f \in \mathbb{R}$ and $\vec{x} \in \mathbb{R}^n$, where $n \in\{2,3\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ is positive:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f &amp;gt; 0~ \forall ~\vec{x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and radial-symetric with $r=\left|\vec{x}\right|$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(\vec{x}) = f(\left| \vec{x}\right|) = f(r)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is bound in infinity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lim_{r\rightarrow \infty} f(r) = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The differential equation looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta f(\vec{x}) = \nabla \cdot \left[ a \vec{G}(\vec{x})f(\vec{x}) + b f(\vec{x}) \left( \left( \vec{G}(\vec{x})f(\vec{x}) \right) *  f(\vec{x})  \right) \right]$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Delta$ is the Laplace Operator, $\nabla$ the Nabla operator, both acting on $\vec{x}$. $\cdot$ denotes the dot-product, and $*$ a convolution, with $a,b \in \mathbb{R}$, and $\vec{G}(\vec{x}) \in \mathbb{R}^n$&lt;/p&gt;&#xA;" OwnerUserId="7776" LastActivityDate="2014-03-26T15:57:28.180" Title="Strategy for solving a non-trivial differential equation" Tags="|pde|ode|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11194" PostTypeId="1" AcceptedAnswerId="11198" CreationDate="2014-03-26T17:05:23.833" Score="5" ViewCount="273" Body="&lt;p&gt;I am trying to compute $x^2$ where $x$ is a double precision floating point number.  I need conservative rounding, which means I need $x \cdot x$ rounded up and $x \cdot x$ rounded down.  If the rounding mode is set to FE_UPWARDS, the code is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double lo = -(x * -x);&#xA;double hi = x * x; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, this does not work for me on clang 3.4: the compiler notices the &quot;common subexpression&quot; and does only one multiply.  Does anyone know how to convince clang to not do this, ideally without harming other optimizations?  In particular, I would like to avoid noninline functions.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2014-04-14T11:13:58.433" Title="Computing a conservatively rounded square of a double on broken compilers" Tags="|interval-arithmetic|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11195" PostTypeId="2" ParentId="7535" CreationDate="2014-03-26T17:33:48.767" Score="5" Body="&lt;p&gt;Parabolic PDE's such as those in the book can usually be solved using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Method_of_lines&quot; rel=&quot;nofollow noreferrer&quot;&gt;Method of Lines&lt;/a&gt;.  First you create some mesh for the $x$ direciton.  I will assume that you used some uniform spacing since the plots don't show any characteristics that show the need of non-uniformity.  Next you recast your equations with only the time derviative on the left hand side and change the derivatives in $x$ to finite difference approximations.  Here is the first equation for a general interior point:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial u_{1,i}}{\partial t} = \varepsilon (u_{2,i} - u_{1,i}) - \frac{u_{1,i+1} - 2 u_{1,i} + u_{1,i-1}}{\Delta x^2} - \frac{u_{1,i+2} - 4 u_{1,i+1} + 6 u_{1,i} - 4 u_{1,i-1} + u_{1,i-2}}{\Delta x^4} - u_{1,i} \frac{u_{1,i+1} - u_{1,i-1}}{2 \Delta x}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am going to leave the second equation, boundary equations, and points near the boundary to you.  Now you have a coupled set of ODE's for $i = 1 ... N$.  From your initial conditions you can assign $u_{1,i}$ and $u_{2,i}$ at the first time step.  Now at each time step you satisfy the above discretized equations at different times based on which time integration algorithm you are using.  If it is explicit Euler, you satisfy it at the beginning of each time step.  If it is implicit Euler, the end.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In matlab, however, there is an easy way to handle all of these (and many more complicated) methods.  What you want is a function that returns a vector of values equal to the right hand side of the above equation given a vector of $u_{j,i}$.  If you assume periodic boundary conditions you get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [ u_prime ] = derivative( t, u, delta_x )&#xA;&#xA;u_prime = zeros(length(u),1);&#xA;u = [u(end-3:end); u; u(1:4)];&#xA;&#xA;if t &amp;lt; 200&#xA;    epsilon = 0;&#xA;else&#xA;    epsilon = 0.1;&#xA;end&#xA;&#xA;&#xA;for i = 5:2:length(u) - 4;&#xA;    u_prime(i-4) = epsilon*(u(i+1) - u(i)) - ...&#xA;        (u(i+2) - 2*u(i) + u(i-2))/delta_x^2 - ...&#xA;        (u(i+4) - 4*u(i+2) + 6*u(i) - 4*u(i-2) + u(i-4))/delta_x^4 - ...&#xA;        u(i)*(u(i+2) - u(i-2))/(2*delta_x);&#xA;    j = i+1;&#xA;    u_prime(j-4) = epsilon*(u(j-1) - u(j)) - ...&#xA;        (u(j+2) - 2*u(j) + u(j-2))/delta_x^2 - ...&#xA;        (u(j+4) - 4*u(j+2) + 6*u(j) - 4*u(j-2) + u(j-4))/delta_x^4 - ...&#xA;        u(j)*(u(j+2) - u(j-2))/(2*delta_x); &#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you can feed this to any of matlabs built-in ODE solvers.  I found that ode15s performed ratherly well.  I also assumed sinusoidal ICS, but it doesn't appear to matter.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 1000;   % Number of space discretizations&#xA;&#xA;x = linspace(0, 150, N);&#xA;u_0 = zeros(2*N,1);&#xA;u_0(1:2:end-1) = sin(2*x/10);   % u_1&#xA;u_0(2:2:end) = -sin(4*x/10);    % u_2&#xA;delta_x = x(2) - x(1);&#xA;&#xA;[t,u] = ode15s(@(t,u) derivative(t,u,delta_x), [0 400], u_0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results give: &lt;img src=&quot;https://i.stack.imgur.com/NcXR0.jpg&quot; alt=&quot;Simulation Results&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2014-03-26T18:47:28.783" LastActivityDate="2014-03-26T18:47:28.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11196" PostTypeId="2" ParentId="11167" CreationDate="2014-03-26T18:31:36.790" Score="3" Body="&lt;p&gt;There are lots of ways to do integration, of which the trapezoidal rule is about the simplest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know anything at all about the actual function you're integrating, you can do better if you exploit that.&#xA;The idea is to minimize the number of grid points within acceptable levels of error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, trapezoidal is making a linear fit to consecutive points.&#xA;You could make a quadratic fit, which if the curve is smooth would fit better, which could allow you to use a coarser grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Orbital simulations are sometimes done using conics, because orbits are very much like conic sections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my work, we are integrating shapes that approximate bell-shaped curves, so it is effective to model them as that (&lt;a href=&quot;http://en.wikipedia.org/wiki/Adaptive_quadrature&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;adaptive Gaussian quadrature&lt;/em&gt;&lt;/a&gt; is considered the &quot;gold standard&quot; in this work).&lt;/p&gt;&#xA;" OwnerUserId="1262" LastEditorUserId="1262" LastEditDate="2014-03-26T18:37:03.047" LastActivityDate="2014-03-26T18:37:03.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11197" PostTypeId="2" ParentId="11194" CreationDate="2014-03-26T19:18:41.840" Score="2" Body="&lt;p&gt;This is a sad answer, so I am not going to accept it in hopes that someone else has a better solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double hi = x * x;&#xA;double lo = (2*epsilon-1)*hi;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is safe because if we're rounding up,&#xA;$$\begin{aligned}&#xA;-\left((2\epsilon-1) \odot (x \odot x)\right) &amp;amp;\le (1 + \epsilon)^2(1-2\epsilon) x^2 \\&#xA;&amp;amp;\le (1+2\epsilon+\epsilon^2)(1-2\epsilon)x^2 \\&#xA;&amp;amp;\le (1-\epsilon^2-\epsilon^3)x^2 \\&#xA;&amp;amp;\le x^2&#xA;\end{aligned}$$&#xA;so our code correctly computes the lower bound.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2014-03-26T19:18:41.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11198" PostTypeId="2" ParentId="11194" CreationDate="2014-03-26T22:30:51.530" Score="4" Body="&lt;p&gt;I believe this can be fixed by packing the interval bounds into SSE registers and performing all the interval operations with SSE as described in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hal.inria.fr/docs/00/28/84/56/PDF/intervals-sse2-long-paper.pdf&quot; rel=&quot;nofollow&quot;&gt;http://hal.inria.fr/docs/00/28/84/56/PDF/intervals-sse2-long-paper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should be faster than my current code anyways, and clang shouldn't apply SSE within the same instruction.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2014-03-26T22:30:51.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11200" PostTypeId="1" AcceptedAnswerId="11226" CreationDate="2014-03-27T05:46:09.510" Score="2" ViewCount="169" Body="&lt;p&gt;Given a plane in 3D Euclidean space is  $\pi$: $ax+by+cz+d=0$&#xA;and a point $P$:$(X,Y,Z)\in \mathbb{R}^3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find a point $Q:(X^*,Y^*,Z^*)\in \pi$ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q= \arg\min\limits_{Q^*\in\pi}\left\|P-Q\right\|$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it difficult to find the closed form solution to $Q$?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2014-03-29T13:20:00.483" Title="The closed form solution of a point on a plane" Tags="|computational-geometry|geometry|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="11201" PostTypeId="2" ParentId="114" CreationDate="2014-03-27T12:03:38.363" Score="-1" Body="&lt;p&gt;Not really answering your question, but more adding another variable to consider: something to take into account is the features of the programming language. For example, Python's &lt;code&gt;sort&lt;/code&gt; uses the &lt;a href=&quot;https://en.wikipedia.org/wiki/Timsort&quot; rel=&quot;nofollow&quot;&gt;Timsort&lt;/a&gt; algorithm, that is designed (among other nice properties) to minimise the number of comparisons, that could be potentially slow for Python objects. On the other hand, compare two floats in C++ is blazing fast, but swapping them is more costly, so they use &lt;a href=&quot;http://en.wikipedia.org/wiki/Sort_%28C++%29&quot; rel=&quot;nofollow&quot;&gt;other&lt;/a&gt; algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other examples are dynamic memory allocation (trivial in a Python list, fast in both runtime and developers time, just &lt;code&gt;.append()&lt;/code&gt;), vs FORTRAN or C, where, though possible and faster when properly implemented, it takes significantly more programming time and brain. See &lt;a href=&quot;https://ep2013.europython.eu/conference/talks/python-is-faster-than-fortran&quot; rel=&quot;nofollow&quot;&gt;Python is faster than FORTRAN.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7759" LastActivityDate="2014-03-27T12:03:38.363" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11202" PostTypeId="2" ParentId="10770" CreationDate="2014-03-27T13:54:59.937" Score="2" Body="&lt;p&gt;Managing uncertainties is actually a quite delicate statistics problem. The known expression for error propagation using squared partial derivatives is good when the errors are normally distributed, independent, and small. This is usually the case; and in fact, even if the normality or independence are not fully satisfied, for most practical cases the result could be reasonably close to the real one, provided you are only interested in an reasonable estimation of the interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another posibility is to repeat the operation with the upper and the lower bound, so that $\sin(2\pm0.1) = \sin(2)^{\sin(2.1)}_{\sin(1.9)}$, but this is only correct if the intervals are hard thresholds (like the output of a non noisy instrument with accuracy 0.1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the cases where more accuracy is needed, if you have a nice analytical model for your uncertainties, there are methods, like maximum likelihood, that can be used to derive the correct estimators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you want a correct result valid for any possible input (imagine having to analytically model a noisy detector where the noise is of the same order as the precision), taking into account all correlations, and being able to explore all the relevant parameter space, you need Monte Carlo methods. Add random noise to several of instances your inputs and run the full analysis. Of course, this may mean that you have to multiply your computational time by hundreds or thousands, but it is always parallelisable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in the end, it all depends on what you want to do, to what degree of accuracy, and how many resources you have.&lt;/p&gt;&#xA;" OwnerUserId="7759" LastActivityDate="2014-03-27T13:54:59.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11203" PostTypeId="1" AcceptedAnswerId="11209" CreationDate="2014-03-27T16:03:04.157" Score="3" ViewCount="749" Body="&lt;p&gt;I had a general query related to calculation of stiffness ratio evolution for a set of coupled odes over a certain time interval. My question is, while calculating the eigen values from the jacobian matrix do we need to know the exact solution at the different time points or can we use solutions approximated by an integration method?&lt;/p&gt;&#xA;" OwnerUserId="7790" LastEditorUserId="276" LastEditDate="2014-03-27T19:56:34.937" LastActivityDate="2016-02-06T05:52:14.083" Title="Stiffness ratio evolution for odes" Tags="|stiffness|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11204" PostTypeId="1" AcceptedAnswerId="11223" CreationDate="2014-03-27T16:05:17.643" Score="4" ViewCount="554" Body="&lt;p&gt;In times of parallel computing, it seems to me that algorithms (also basic ones, like matrix-vector multiplication) should be measured by their dependent steps (that use results from steps before) rather than in total number of Floating Point Operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, disregarding memory access, matrix-vector multiplication has $O(n^3)$ Floating Point Operations, but only 2 dependent steps (first do the multiplications, then the additions)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course even with modern GPUs you cannot compute infinitely many steps in parallel, add up infinitely many numbers, or cache infintely many values, but to compensate for this you can multiply the 'dependent steps length' of your algorithm  by a factor (dependent on number of FPUs, cachesize etc. and the respecitve maximal needs of your algorithm) to obtain reasonable estimates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your opinions/references/objections/caveats/pitfalls&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I'm no computer specialist (especially not concerning hardware), just a mathematician working with MATLAB every now and then&lt;/p&gt;&#xA;" OwnerUserId="7789" LastEditorUserId="7789" LastEditDate="2014-03-28T13:12:18.777" LastActivityDate="2014-08-22T15:22:38.740" Title="Why is computational cost measured in Floating Pt. Ops. in times of parallel computing?" Tags="|algorithms|parallel-computing|performance|complexity|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11205" PostTypeId="1" CreationDate="2014-03-27T17:59:27.283" Score="2" ViewCount="118" Body="&lt;p&gt;I'm going to ask some beginners' questions.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Adaptive mesh can save many DOFs than a uniform mesh. But it also needs to solve linear systems changing with mesh adaptive process. Is this not problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now suppose we are going to solve the problem a lot of times with different multiple source/boundary/initial data. What techniques are useful for adaptive mesh to further reduce costs?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I once heard a lecture given by Hackbusch who proposed to just use a very fine uniform mesh and make use of some low rank property of the matrix to compute its inverse. The number of DOFs sounds crazy but the method seems to work well. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Would someone shares his/her experiences?&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2014-03-27T17:59:27.283" Title="Adaptive Mesh vs Uniform Mesh for multiple source/boundary/initial Data" Tags="|linear-solver|adaptive-mesh-refinement|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11206" PostTypeId="1" CreationDate="2014-03-27T19:27:52.393" Score="2" ViewCount="876" Body="&lt;p&gt;I'm in the middle of installing GPAW on a cluster but I'm running into problems building the &quot;gpaw-python&quot; interpreter, its job is to allow users to execute parallelised GPAW calculations. Following the Developer Installation guide &lt;a href=&quot;https://wiki.fysik.dtu.dk/gpaw/devel/developer_installation.html#developer-installation&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, I've downloaded the most recent stable package and got it working fully on my local desktop, including the gpaw-python interpreter, and currently have the gpaw.tar file unzipped and ready to go on the cluster, which I am accessing through ssh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specific part where the problem occurs is building the C extensions, as seen at the start of the developer guide. I've ensured that I have mpicc installed as required and it links to the command line ok, albeit this is through a software module on the cluster which is available to all users (this version is gcc/4.6.3/openmpi/64/1.6.1). Note the caveat in the developer installation guide requiring mpicc in order to construct the gpaw-python interpreter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;python setup.py build_ext 2&gt;&amp;amp;1 | tee build_ext.log&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Everything looks okay and the expected &lt;em&gt;location&lt;/em&gt; of the &quot;gpaw-python&quot; executable is built in &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;   $GPAW_HOME/build/lib.&amp;lt;platform&amp;gt;-&amp;lt;python_version&amp;gt;/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;but doesn't anything, so I would guess the installation is missing mpicc somehow. When I tried installing GPAW locally there wasn't a problem doing this so maybe I'm missing some subtlety when working on the cluster?&lt;/p&gt;&#xA;" OwnerUserId="3344" LastActivityDate="2014-03-28T12:58:26.157" Title="Installing GPAW (Python DFT package) on a computing cluster" Tags="|python|parallel-computing|computational-chemistry|density-functional-theory|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11208" PostTypeId="1" CreationDate="2014-03-27T20:00:30.260" Score="3" ViewCount="748" Body="&lt;p&gt;I am trying to solve the following set of equations using a finite-difference approach and an iterative solver (Gauss-Seidel).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuity Equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial V_x}{\partial x} + \frac{\partial V_y}{\partial y} = 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simplified Stokes-Equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \eta (\frac{\partial^2 V_x}{\partial x^2} + \frac{\partial^2 V_x}{\partial y^2}) - \frac{\partial P}{\partial x} = 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \eta (\frac{\partial^2 V_y}{\partial x^2} + \frac{\partial^2 V_y}{\partial y^2}) - \frac{\partial P}{\partial y}+ \rho g = 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use both Stokes equations to solve for velocity ($v_x,v_y$). However, how can I solve then for pressure ($P$)? Or do I first solve for velocity using the continuity equation and eventually I solve for pressure using the stokes equations?&lt;/p&gt;&#xA;" OwnerUserId="7792" LastActivityDate="2014-03-30T19:22:22.543" Title="Solving Navier Stokes Eq using Gauss–Seidel and Finite Difference" Tags="|navier-stokes|" AnswerCount="2" CommentCount="2" ClosedDate="2014-04-30T01:15:36.053" ContentLicense="CC BY-SA 3.0" />
  <row Id="11209" PostTypeId="2" ParentId="11203" CreationDate="2014-03-27T20:19:05.033" Score="1" Body="&lt;p&gt;I made &lt;a href=&quot;https://scicomp.stackexchange.com/a/892/276&quot;&gt;a number of comments on stiffness in response to another question&lt;/a&gt; that are relevant here; I won't repeat them here in their entirety. The most important point is that stiffness ratio is only correlated with stiffness, and not a definition of stiffness. Consequently, it does not seem critical to calculate eigenvalues of a Jacobian matrix based on an exact solution; calculating approximate eigenvalues (for example, via a Krylov subspace method that provides eigenvalue estimates, like GMRES) of a reasonably accurate approximate solution, then calculating the stiffness ratio is probably fine for diagnostic purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another criterion that could work, if you can calculate the necessary data, is to compare the time steps your integrator is taking at the end of each iterate to the maximum stable time step it could take. If stability is limiting time step size rather than accuracy, your system is probably stiff. (Again, this sort of behavior tends to occur in stiff systems, but it is not a definition.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-03-27T20:19:05.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11210" PostTypeId="2" ParentId="11208" CreationDate="2014-03-27T21:16:15.873" Score="1" Body="&lt;p&gt;In the incompressible Stokes (and Navier-Stokes) equations, pressure serves to enforce incompressibility. You have 3 equations for three unknown field variables ($V_x$, $V_y$, and $P$) so your system is fully determined once you choose boundary conditions. There is no additional equation for pressure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In solving the time-dependent Navier-Stokes equations, a popular technique is to use a predictor/corrector type method where the velocities are advanced based on the old pressure values and then corrected using a &quot;pressure projection&quot; technique. This involves solving a Poisson equation to find the required pressure to enforce incompressibility. This essentially means projecting the updated velocity field onto the space of incompressible velocity fields where the pressure field serves to enforce this projection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the Stokes equations can be discretized using your desired method, resulting in a system of linear equations that can be solved simultaneously using your desired matrix inversion technique or iterative scheme. I'm not sure how well behaved the resulting linear system will be in terms of convergence under Gauss-Seidel iteration. It may work fine or you may need to perform some type of pre-conditioning.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-03-27T21:16:15.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11212" PostTypeId="1" AcceptedAnswerId="11213" CreationDate="2014-03-28T00:35:04.100" Score="2" ViewCount="2231" Body="&lt;p&gt;I am trying to solve a system of coupled ODEs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \begin{align}&#xA;    \frac{dn_A}{dt} &amp;amp; = e\left[j(t) - f\, θ_H\sinh\left(\frac{g\,n_A}{T}\right)\right] \\&#xA;    \frac{dθ_H}{dt} &amp;amp; = a\left[bP\,(1 - θ_H )^2 - c~(θ_H)^2 - f\,θ_H\,\sinh\left(\frac{g\,n_A}{T}\right)\right] \\&#xA;    \frac{dP}{dt} &amp;amp; =h(i - P) + d\,T\,(P-P_a)\left[bP~(1 - θ_H )^2 - c~(θ_H)^2\right].&#xA;  \end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $e, f, g, T, a, b, c,h,i,d$ are constants, $j$ is a time dependent variable, and $P_a$ = 101325. The initial values for the three parameters $n_A, θ_H, P$ are $0, 0.5$, and $8106$ respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was able to solve the equations using the explicit Euler method, but I am having trouble applying the implicit Euler method to do the same. For the first equation I can use the explicit Euler method to estimate the initial value for $n_A$ at time point $t_{n+1}$, but I am not able to figure out what should I take as a value for the $θ_H$ term that also gets involved in the Newton-Raphson iteration scheme for calculation at $t_{n+1}$.&lt;/p&gt;&#xA;" OwnerUserId="7790" LastEditorUserId="123" LastEditDate="2015-04-22T17:27:24.193" LastActivityDate="2015-04-22T17:27:24.193" Title="How can I implement the implicit Euler method for a small nonlinear system of ODEs?" Tags="|matlab|ode|implicit-methods|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11213" PostTypeId="2" ParentId="11212" CreationDate="2014-03-28T02:36:58.557" Score="3" Body="&lt;p&gt;Define&#xA;$$&#xA; \begin{align}&#xA;  A^{n+1} &amp;amp; = e\left[j^{n+1} - f\, θ_H^{n+1}\sinh\left(\frac{g\,n_A^{n+1}}{T}\right)\right] \\&#xA;  B^{n+1} &amp;amp; = a\left[bP^{n+1}\,(1 - θ_H^{n+1} )^2 - c~(θ_H^{n+1})^2 - f\,θ_H^{n+1}\,\sinh\left(\frac{g\,n_A^{n+1}}{T}\right)\right] \\&#xA;  C^{n+1} &amp;amp; = h(i - P^{n+1}) + d\,T\,(P^{n+1}-P_a)\left[bP^{n+1}~(1 - θ_H^{n+1} )^2 - c~(θ_H^{n+1})^2\right]&#xA;  \end{align}&#xA;$$&#xA;Then backward Euler gives you&#xA;$$&#xA; \begin{align}&#xA;   n_A^{n+1} &amp;amp; = n_A^n + \Delta t\,A^{n+1} \\&#xA;   \theta_H^{n+1} &amp;amp; = \theta_H^n + \Delta t\,B^{n+1} \\&#xA;   P^{n+1} &amp;amp; = P^n + \Delta t\,C^{n+1} \,.&#xA; \end{align}&#xA;$$&#xA;Reorganize into the form&#xA;$$&#xA; \begin{align}&#xA; F^{n+1} &amp;amp;:= n_A^{n+1} - \Delta t\,A^{n+1} -  n_A^n = 0 \\&#xA; G^{n+1} &amp;amp;:= \theta_H^{n+1} - \Delta t\,B^{n+1} - \theta_H^n = 0 \\&#xA; H^{n+1} &amp;amp;:= P^{n+1} -  \Delta t\,C^{n+1} - P^n = 0 &#xA; \end{align}&#xA;$$&#xA;Let us assume that you have an equation for $j^{n+1}$ that can be solved explicitly.  Then the above system of equations has the form&#xA;$$&#xA;  \mathbf{r}(\mathbf{u}^{n+1}) = 0&#xA;$$&#xA;where $\mathbf{u}^{n+1} = (n_A^{n+1}, \theta_H^{n+1}, P^{n+1})$.  Newton's method can then be expressed as&#xA;$$&#xA;  \mathbf{u}^{n+1}_{k+1} = \mathbf{u}^{n+1}_k - \mathbf{T}^{-1}(\mathbf{u}^{n+1}_k)\,\mathbf{r}(\mathbf{u}^{n+1}_k)&#xA;$$&#xA;where&#xA;$$&#xA;  \mathbf{T}(\mathbf{u}^{n+1}_k) = \frac{\partial \mathbf{r}(\mathbf{u}^{n+1}_k)}{\partial \mathbf{u}^{n+1}_k} \,.&#xA;$$&#xA;The components of the matrix $\mathbf{T}$ are found using the usual relations&#xA;$$&#xA;  T_{ij} = \frac{\partial r_i}{\partial u_j} \,.&#xA;$$&#xA;You already have initial values for $\mathbf{u}$.  I'm not sure why you can't use those for your Newton iterations.  If Newton iterations don't converge you may have to use an alternative algorithm (such as bisection/line search etc.)&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2014-03-28T02:36:58.557" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11215" PostTypeId="2" ParentId="10770" CreationDate="2014-03-28T09:24:53.597" Score="5" Body="&lt;p&gt;I agree with Davidmh that calculating uncertainties should not be handled by an automatic library. You will very quickly run into a case where the automatics fail (try doing a Fourier transform for instance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say however that you just want to keep the uncertainties with your data. Why not just add them as an extra column in your dataframe? This is how I typically manage uncertainties with pandas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pandas has no support of units, however anything can go into a dataframe so you could use the &lt;a href=&quot;https://pypi.python.org/pypi/quantities&quot; rel=&quot;noreferrer&quot;&gt;quantities&lt;/a&gt; package directly. Not all functionality will work in pandas however (though a surprising amount still will) and there will be a performance penalty. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been some &lt;a href=&quot;https://github.com/pydata/pandas/issues/2485&quot; rel=&quot;noreferrer&quot;&gt;discussion&lt;/a&gt; to allow for the attachment of metadata in pandas, but so far nothing seems to have come of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the possible exception of C++11, I know of no language or library that would give you really good, first class unit support. There is always a loss of performance and a lack of compatibility&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-03-28T09:24:53.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11216" PostTypeId="2" ParentId="11204" CreationDate="2014-03-28T10:11:21.560" Score="5" Body="&lt;p&gt;First of all, a bit on the terminology. &lt;a href=&quot;https://en.wikipedia.org/wiki/FLOPS&quot; rel=&quot;noreferrer&quot;&gt;FLOPS&lt;/a&gt; are &quot;floating point operations &lt;strong&gt;per second&lt;/strong&gt;&quot;, and they are typically not used to measure the performance of algorithms, but of machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that you mean that the complexity of algorithms is measured in the total number of floating point operations required to do the job (probably one could write that as FLOPs with lower case &quot;s&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that even in times of parallel computing, this is a useful measure. FLOPs simply tell you how much work needs to be done. When I have a problem that requires more FLOPs than even my parallel machine can provide within reasonable time, I'm lost. Insofar, the FLOPs give you a &lt;em&gt;lower limit&lt;/em&gt; of the time you will need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I agree that with parallel machines, the measure has become far less useful, as almost no algorithm is actually able to use the full performance of the machine on a single problem. Therefore, your idea of an alternative measure seems reasonable to me. I am not aware of any alternative, but I have never looked for one. Maybe you want to elaborate on your idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as a side remark: the crazy thing is that supercomputing centers still mostly look at the number of FLOPS of their machines &lt;em&gt;that can be used on a single problem&lt;/em&gt;, and that is what the &lt;a href=&quot;http://www.top500.org/&quot; rel=&quot;noreferrer&quot;&gt;TOP 500&lt;/a&gt; benchmarks are all about. Therefore, they buy extremely expensive network interconnects to connect the whole machine.&#xA;However, with the coming generation of exascale computers, there will be only a handful of algorithms that will be able to really use it.&#xA;On the other hand, there are lots of problems that could profit from the computing power when splitting up the machine into smaller subsystems. That is what actually happens in supercomputing centers in more than 90% of the time anyway. Still, this kind of problems are shunned by the supercomputing community as being &quot;trivially parallelizable&quot;, as they cannot be used to argue for the expensive interconnects.&lt;/p&gt;&#xA;" OwnerUserId="1568" LastActivityDate="2014-03-28T10:11:21.560" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11217" PostTypeId="1" CreationDate="2014-03-28T11:31:05.867" Score="2" ViewCount="743" Body="&lt;p&gt;Given a triangular plane formed by three points in R3 space {p1, p2, p3}, I want to iterate through all points on the triangle plane by using two variables, x0,y0, something like in this example: &lt;a href=&quot;http://demonstrations.wolfram.com/DirectionalDerivativesIn3D/&quot; rel=&quot;nofollow&quot;&gt;http://demonstrations.wolfram.com/DirectionalDerivativesIn3D/&lt;/a&gt; The example in that website uses directional derivatives. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this was in 2D, it is easy to iterate through the triangle, by using 2 for-loops (one each for x &amp;amp; y axis) and looping through a rectangular bounding box formed by that triangle. &#xA;I could then use barycentric coordinates to isolate the points that fall within the triangle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for 3D space, I have no clue how to do it. Any help or idea would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="7796" LastActivityDate="2017-03-08T12:49:16.740" Title="Iterating through a 3D triangle" Tags="|linear-algebra|delaunay-triangulation|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11218" PostTypeId="2" ParentId="11217" CreationDate="2014-03-28T12:55:58.687" Score="5" Body="&lt;p&gt;Why not transform your coordinates into the plane of the triangle and do your iterations there?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-03-28T12:55:58.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11219" PostTypeId="2" ParentId="11206" CreationDate="2014-03-28T12:58:26.157" Score="1" Body="&lt;p&gt;Problem can be fixed by adding this around line 395 in the config.py file, inside the build_interpreter function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;define_macros.append(('_GNU_SOURCE', '1'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the other define_macros.append commands reside.&lt;/p&gt;&#xA;" OwnerUserId="3344" LastActivityDate="2014-03-28T12:58:26.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11220" PostTypeId="1" AcceptedAnswerId="11234" CreationDate="2014-03-28T14:27:54.870" Score="3" ViewCount="1257" Body="&lt;p&gt;&lt;strong&gt;Brief explanation of QM geometry optimization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quantum mechanics packages are often tasked with optimizing a chemical structure. The problem is essentially this: Given a set of points in 3D space and potentials that are strongly correlated with chemical structure (bond lengths, bond angles, torsion angles), minimize the total potential with the points as degrees of freedom. Let's just assume you know who's bonded to who already. The potentials relate pairs of points that are bonded &lt;em&gt;and&lt;/em&gt; nonbonded but the bonded ones are especially strong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;QM programs often use redundant internal coordinates for geometry optimization. For a single molecule, conformations can be parameterized nonredundantly in internal coordinates in a tree structure. The degrees of freedom are bond lengths, bond angles, and torsion angles. (Some chemical bonds may not have an explicit bond length parameter in this scheme.) Redundant internal coordinates have additional parameters that are not strictly necessary to describe the chemical topology, ex: nonbonded distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Redundant internal coordinates have apparently proven more efficient empirically, especially for systems that have bonded cycles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimizations are typically done with BFGS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Qualitatively, how can this be? Why would adding unnecessary information improve optimization efficiency?&lt;/p&gt;&#xA;" OwnerUserId="7745" LastActivityDate="2014-03-31T13:01:57.140" Title="Why do QM programs use redundant internal coordinates for geometry optimization?" Tags="|optimization|quantum-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11223" PostTypeId="2" ParentId="11204" CreationDate="2014-03-29T01:07:01.670" Score="8" Body="&lt;p&gt;In fact, the precise total number of operations is very rarely used as a measure of computational cost. Instead, you will most often see the &lt;strong&gt;computational order&lt;/strong&gt; (i.e. $\mathcal{O}(n^3)$). This &quot;big O&quot; notation roughly means that the number of operations is proportional to $n^3$ and tells you how the total number of operations scales as the number of unknowns is changed. This is an extremely useful measure of computational complexity because it gives you an estimate of the relative cost as you scale up the size of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a simple example: solving a linear system of equations&lt;br&gt;&#xA;&lt;strong&gt;Gaussian elimination&lt;/strong&gt; is $\mathcal{O}(n^3)$ so if you double the number of unknowns it will take $2^3=8$ times as long. If you're performing a 3D computation and half your grid spacing you'll have $2^3=8$ times as many grid points (unknowns) and need to do $8^3=512$ times more work to solve your linear system.&lt;br&gt;&#xA;&lt;strong&gt;Algebraic multigrid&lt;/strong&gt; (AMG) solves linear systems in $\mathcal{O}(n)$ time. Even though it may take significantly longer to solve a small system of equations using AMG, as you scale up your number of unknowns, the work scales much slower. In this case 8 times the unknowns means 8 times the work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above discussion is entirely independent of parallel versus serial execution. In general, parallel performance is discussed in terms of &lt;strong&gt;parallel efficiency&lt;/strong&gt;. Parallel efficiency generally depends both on problem size and number of processes and is defined as&#xA;$$\textit{Efficiency(n,p)} = \dfrac{T_{1}(n)}{p\cdot T_{p}(n,p)}$$&#xA;where $n$ is the number of unknowns, $p$ is the number of processes, and $T_p$ is the runtime using $p$ processes. For example, an algorithm with a 50% parallel efficiency will run 5x as fast on 10 cores compared to a single core (rather than 10x as fast on 10 cores).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the parallel efficiency does not affect the computational order or vice versa. The computational complexity reflects the number of floating point operations that must be performed which does not depend on the number of processes used. On the other hand, parallel efficiency is largely determined by the communication requirements of the algorithm as is closely tied to the communication bandwidth and latency. If every process must communicate with every other all the time you will have very poor parallel efficiency. On the other hand, algorithms that require no communication between processes are known as &quot;embarrassingly parallel&quot; and achieve nearly 100% parallel efficiency. This is typical of Monte Carlo algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although it is perhaps not as widely discussed outside of large computations, parallel efficiency is quickly becoming almost as important as the computational order due to the prevalence of many core computers.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-08-22T15:22:38.740" LastActivityDate="2014-08-22T15:22:38.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11224" PostTypeId="1" AcceptedAnswerId="11227" CreationDate="2014-03-29T01:59:10.167" Score="1" ViewCount="707" Body="&lt;p&gt;I want to understand how the RATTLE algorithm works.&#xA;Can somebody give me an example (in pseudocode or using any programming language like python or matlab) of how would I implement a numerical integrator for the simple pendulum problem? &lt;/p&gt;&#xA;" OwnerUserId="6611" LastActivityDate="2014-03-29T20:44:52.213" Title="RATTLE numerical integrator example" Tags="|algorithms|time-integration|numerics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11225" PostTypeId="2" ParentId="11217" CreationDate="2014-03-29T08:32:21.280" Score="0" Body="&lt;p&gt;You could compute the normal of your plane (by cross product) and rotate the plane in opposite direction. This will make the &amp;amp;z&amp;amp; coordinates identical. Then you could project the plane onto 2D cartesian by simply ignoring &amp;amp;z&amp;amp;. Yet you still have a direct map (correspondence) of each triangle coordinate in 3D to the ones in 2D. You can iterate in 2D grid and always map to 3D.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-03-29T08:32:21.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11226" PostTypeId="2" ParentId="11200" CreationDate="2014-03-29T13:07:30.307" Score="1" Body="&lt;p&gt;Given a plane $\pi:  (a,b,c,d)$ , without loss of generality,  let $a^2+b^2+c^2=1$, then the orthographic projection to $\pi$ in homogeneous matrix is:&#xA;$$T=\left(&#xA;\begin{array}{cccc}&#xA; 1-a^2 &amp;amp; -a b &amp;amp; -a c &amp;amp; -a c \\&#xA; -a b &amp;amp; 1-b^2 &amp;amp; -b c &amp;amp; -b c \\&#xA; -a c &amp;amp; -b c &amp;amp; 1-c^2 &amp;amp; -c^2 \\&#xA;\end{array}&#xA;\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $P=(x,y,z,1)^T$, then the desired point $Q=T\cdot P$ is:&#xA;$$\left\{\quad&#xA;\begin{array}{c}&#xA; \left(1-a^2\right) x-a b y-ac z -a c\\&#xA;-a b x+\left(1-b^2\right) y  -b c z-b c\\&#xA;-a x c-b cy+\left(1-c^2\right) z  -c^2\\&#xA;\end{array}&#xA;\right.$$&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6693" LastEditDate="2014-03-29T13:20:00.483" LastActivityDate="2014-03-29T13:20:00.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11227" PostTypeId="2" ParentId="11224" CreationDate="2014-03-29T16:36:12.787" Score="5" Body="&lt;p&gt;The following snippet of code is an implementation of RATTLE on a system with the constraint $g(x, y) = K x^2 + y^2 - 1 = 0$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Constraint.&#xA;double g(const double2&amp;amp; r) {&#xA;  return K * r.x * r.x + r.y * r.y - 1.0;&#xA;}&#xA;&#xA;// Gradient of constraint.&#xA;double2 G(const double2&amp;amp; r) {&#xA;  return double2(2.0 * K * r.x, 2.0 * r.y);&#xA;}&#xA;&#xA;void rattle(double2&amp;amp; q, double2&amp;amp; p, double&amp;amp; lambda, double h) {&#xA;  // Declare auxiliary constants.&#xA;  const double2 Gqprev = G(q);&#xA;&#xA;  // Deal with constraint on the configuration manifold.&#xA;  q += h * p;&#xA;  double lambda_r = 0.0;&#xA;  // Solve using Newton's method.&#xA;  for (size_t k = 1; k &amp;lt;= max_iters; k++) {&#xA;    if (k == max_iters)&#xA;      abort();&#xA;&#xA;    const double2 r = q - lambda_r * Gqprev;&#xA;    const double phi = g(r);&#xA;    const double dphi_dl = -dot(G(r), Gqprev);&#xA;    const double update = phi / dphi_dl;&#xA;&#xA;    if (fabs(phi) &amp;lt; tol &amp;amp;&amp;amp; fabs(update) &amp;lt; tol)&#xA;      break;&#xA;&#xA;    lambda_r -= update;&#xA;  }&#xA;&#xA;  q -= lambda_r * Gqprev;&#xA;  p -= lambda_r / h * Gqprev;&#xA;&#xA;  // Deal with constraint on the tangent space.&#xA;  const double2 Gq = G(q);&#xA;  double lambda_v = dot(Gq, p) / dot(Gq, Gq);&#xA;  p -= lambda_v * Gq;&#xA;&#xA;  lambda = (lambda_r + lambda_v) / 2.0;&#xA;&#xA;  assert(fabs(g(q)) &amp;lt; tol &amp;amp;&amp;amp; fabs(dot(G(q), p)) &amp;lt; tol);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I use the notation from chapter 7 in the book &lt;a href=&quot;http://www.cambridge.org/br/academic/subjects/mathematics/computational-science/simulating-hamiltonian-dynamics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simulating Hamiltonian Dynamics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your convenience, I have uploaded a standalone program that you can compile,  run, and play with at &lt;a href=&quot;https://gist.github.com/jmbr/9857614#file-rattle-cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/jmbr/9857614#file-rattle-cpp&lt;/a&gt;  The program produces output that can be used to plot the graphs shown below (an example trajectory and its corresponding plot of total energy as a function of time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/NatsO.png&quot; alt=&quot;Example of constrained trajectory&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/36I9l.png&quot; alt=&quot;Total energy as function of time&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2014-03-29T20:44:52.213" LastActivityDate="2014-03-29T20:44:52.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11230" PostTypeId="2" ParentId="10905" CreationDate="2014-03-30T07:49:20.773" Score="2" Body="&lt;p&gt;For the past few months, I've been working on a similar problem. I only solve the steady-state potential equation for the moment, assuming homogeneous, steady-state ion concentrations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scaling your equations is going to be a rudimentary preconditioner. I found that I got good results using algebraic multigrid (AMG) for preconditioning instead, since my problem reduces to a coupled reaction-diffusion equation (i.e., a Poisson problem with nonlinear source term). You might try incomplete LU (ILU) to start, because it is a robust, slower preconditioner, and if that works, then try other preconditioners like AMG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a direct solver might be helpful, but it really depends on the size of your problem. Even sparse LU decompositions will exhaust available memory due to fill-in for problems that are sufficiently large (that point occurs somewhere in the neighborhood of 100,000-1,000,000 variables). If your Jacobian matrix is rank-deficient (as was the case in my problem, due to a nonconductive filler material in some cells), then direct methods will also have problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A single higher quality guess for the nonlinear solver won't help. Your problem is transient, so you'll need a guess for every time step, if you're using an implicit method (and given that it's a diffusion problem, you probably should be, unless you want your time steps to be severely limited by the diffusion stability limit). Established libraries that implement numerical integration methods (i.e., time-steppers) do a good job of constructing initial guesses for each time step, and if the guesses are not great (they're typically derived from a problem-independent heuristic), the time steps will be reduced to preserve accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quad precision might be helpful, if your problem defies preconditioning. There aren't many libraries I'm aware of that use quad precision. PETSc does, but I believe if you configure with quad precision, you're limited to PETSc-native solvers and preconditioners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible. You should test your assembled Jacobian against a finite difference approximation for debugging purposes. If you are using a finite difference approximation in your actual simulation, you will roughly halve the number of significant figures in your problem, which could severely erode your accuracy because your problem is ill-conditioned; that said, it doesn't seem like you're using that approximation in your simulation.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-03-30T07:49:20.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11231" PostTypeId="2" ParentId="11208" CreationDate="2014-03-30T19:22:22.543" Score="0" Body="&lt;p&gt;It seems that you want to solve a simple two-dimensional problem. In this case I would strongly recommend you to use a formulation of the equation which use the out-of-plane component of the vorticity and the streamfunction. &lt;/p&gt;&#xA;" OwnerUserId="6836" LastActivityDate="2014-03-30T19:22:22.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11232" PostTypeId="1" AcceptedAnswerId="11268" CreationDate="2014-03-31T06:43:09.080" Score="3" ViewCount="454" Body="&lt;p&gt;I have an array of 2D points and a known curve geometry (it consists of a straight line segment and a circle segment). The points might rotate slightly and are somewhat noisy. I need to find points in array, which correspond to that curve. I have a feeling, that it should be a pretty common task in image processing, which is a new area to me. So, is there a common algorythm for dealing with such tasks? Right now I am reading about Hough Transform, which i think with some modifications can be applied to my task. Am i moving to the right direction? Are there any pitfalls? Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here goes sample image where (1) is points array, (2) is curve geometry and (3) is what i would like to achieve:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/MLT3p.png&quot; alt=&quot;data sample&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7812" LastEditorUserId="7812" LastEditDate="2014-04-01T10:08:04.563" LastActivityDate="2014-04-04T08:11:08.467" Title="Curve detection in cloud of points" Tags="|image-processing|pattern-recognition|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11233" PostTypeId="1" AcceptedAnswerId="11236" CreationDate="2014-03-31T10:49:54.830" Score="0" ViewCount="1946" Body="&lt;p&gt;I am trying to understand why the amplitude of the FFT (computed with numpy) of a Gaussian differs from its analytic solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $\mathcal{F}\{e^{-\pi t^2}\} = e^{-\pi f^2}$. However if I calculate it with&#xA;the FFT function in numpy the resulting Gaussian's amplitude is not 1?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already done the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I do divide the fft result by the number of samples (normalize).&lt;/li&gt;&#xA;&lt;li&gt;I have even tried shifting the Gaussian so that its first sample is its &#xA;height. This corrects the sinusoidal behaviour, and removes the maginary&#xA;part, but does not improve the amplitude result (same as absolute value of&#xA;unshifted case). I will however be taking the absolute value in any case.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def test_gauss_1D(self,a,f_c,delta_f):&#xA;   delta_t = 1.0/(2.0*f_c)&#xA;   N = int(np.ceil(1/(delta_f*delta_t)))+1&#xA;   if (N % 2) == 0:&#xA;      N = N + 1&#xA;&#xA;   t = np.linspace(-(N-1)/2*delta_t,(N-1)/2*delta_t,N)&#xA;   f = np.linspace(-1/(2*delta_t),1/(2*delta_t),N)&#xA;&#xA;   x_t = np.exp(-np.pi*t**2)&#xA;&#xA;   #x_t = np.roll(x_t,-1*(N-1)/2)&#xA;&#xA;   #x_t = 1/(np.sqrt(2*np.pi)*a)*np.exp(-t**2/(2*a**2))&#xA;&#xA;   #print &quot;std = &quot;, 1/(4*a**2*np.pi**2)&#xA;   #print &quot;ampl =&quot;, np.sqrt(2*np.pi)*a&#xA;&#xA;   plt.plot(t,x_t)&#xA;   plt.show()&#xA;&#xA;   #x_f = np.fft.fft(x_t)/N&#xA;   #print &quot;N = &quot;,N&#xA;   x_f = np.fft.fft(x_t)/N&#xA;&#xA;   x_f = np.roll(x_f,1*(N-1)/2)&#xA;&#xA;   plt.plot(f,np.absolute(f))&#xA;   plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are the images:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wjhvg.png&quot; alt=&quot;Before Fourier transform&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/SpmTE.png&quot; alt=&quot;After Fourier transform&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see the amplitude is incorrect in the second image (should be one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is this?&lt;/p&gt;&#xA;" OwnerUserId="7815" LastActivityDate="2014-03-31T14:27:58.640" Title="Amplitude of discrete Fourier transform of Gaussian is incorrect" Tags="|fourier-analysis|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11234" PostTypeId="2" ParentId="11220" CreationDate="2014-03-31T13:01:57.140" Score="3" Body="&lt;p&gt;If you imagine the equations of just two bodies that are bound to each other along a fixed connection and float through space without external forces, and if you denote their centers as $x_1(t),x_2(t)$ both of which are 3-dimensional coordinates, then you can write the equations of motion as&#xA;$$&#xA;  \ddot x_1(t) = \lambda(x_2-x_1),&#xA;  \\&#xA;  \ddot x_2(t) = -\lambda(x_2-x_1),&#xA;  \\&#xA;  \|x_1-x_2\|^2 - d^2 = 0.&#xA;$$&#xA;This is known as a Differential-Algebraic Equation (DAE). Note its pleasant simplicity -- the worst part of it is the quadratic constraint. The force on the right hand side is in the direction $x_2-x_1$, i.e., in the direction of the fixed connection. We don't know the Lagrange multiplier $\lambda$ yet, but that comes out of the constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine how to write the same equations with non-redundant variables. These would be, for example the position $x_1(t)$ and, because the distance between the two bodies is fixed, the two angles by which to describe their relative position. An alternative would be the center of mass $X(t)$ and the relative position of one of the bodies to the center of mass, again in terms of angles (the other body's position then is opposite). In the latter coordinates, we have the equation&#xA;$$&#xA;  \ddot X(t) = 0&#xA;$$&#xA;for the center of mass. We also need to write down the equations for the two angles, which I will not do here because they have a rather complex form. The take-away message if one did, however, would be that in these non-redundant coordinates, the equations will turn out to be (i) highly nonlinear, (ii) have singularities in the coordinate system such as the north pole when describing relative positions with angles, and (iii) are correspondingly difficult to solve. This is opposed to the rather simple form of the DAE for the redundant coordinates which only have to work with a quadratic constraints and computing the force (the Lagrange multiplier) that results from it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-03-31T13:01:57.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11235" PostTypeId="2" ParentId="11233" CreationDate="2014-03-31T14:02:28.157" Score="0" Body="&lt;p&gt;I think you have used an incorrect normalisation: the factor of 1/N is the result of applying both the transform and the inverse transform. For just the forward normalisation you therefore want 1/(sqrt(N)). When I run your code with this normalisation, I see a peak of sqrt(2), so the correct normalisation is therefore 1/(sqrt(2*N)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that the final factor of two comes from the exact definition of N, and whether or not positive and negative frequencies should count twice. In any case, I can't seem to find it on the numpy documentation page: &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/routines.fft.html&quot; rel=&quot;nofollow&quot;&gt;http://docs.scipy.org/doc/numpy/reference/routines.fft.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7705" LastActivityDate="2014-03-31T14:02:28.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11236" PostTypeId="2" ParentId="11233" CreationDate="2014-03-31T14:16:09.760" Score="1" Body="&lt;p&gt;The normalization needs to include the sampling frequency.  Multiple your x_f by delta_t and you should be all set.&lt;/p&gt;&#xA;" OwnerUserId="7257" LastEditorUserId="7257" LastEditDate="2014-03-31T14:27:58.640" LastActivityDate="2014-03-31T14:27:58.640" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11238" PostTypeId="2" ParentId="11217" CreationDate="2014-03-31T20:28:36.470" Score="2" Body="&lt;p&gt;If you use barycentric coordinates then the number of dimensions in which the triangle is embedded does not matter; the conversion from barycentric coordinates to euclidean coordinates is just a convex combination of the euclidean points at the vertices of the triangle.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-03-31T20:28:36.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11239" PostTypeId="1" CreationDate="2014-03-31T22:45:25.713" Score="3" ViewCount="76" Body="&lt;p&gt;In stepwise regression, you step predictor by predictor, each time selecting the one with the greatest correlation with the measurement, subtracting greedily to leave a residual with no correlation to any of the previous predictors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why go stepwise? Why not just pick the predictors with best correlation to the measurement right now, however many you need, and use those? Why is a stepwise approach better? Does it yield smaller 1-norms?&lt;/p&gt;&#xA;" OwnerUserId="7819" LastActivityDate="2014-03-31T22:45:25.713" Title="Why not use this simpler variant of Stepwise Regression?" Tags="|optimization|regression|constrained-optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11240" PostTypeId="1" CreationDate="2014-04-01T07:07:47.853" Score="1" ViewCount="178" Body="&lt;p&gt;I need a hand with the numerical evaluation, in Mathematica, for this integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(t)=\int_{-\infty}^\infty Exp\{it(\omega_H-\omega_l-\omega_k) - \sum _{j\neq(l,k)} S_j [1-e^{-it\omega_j}]\}\, dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the plot of the real and imaginary part:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EVtAP.jpg&quot; alt=&quot;Real and Imaginary plots of $f(t)$ from $-150 &amp;lt; t &amp;lt; 150$&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this are the values for the parameters of $f(t)$ employed in the graph:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega_H=0.408194 ~~;~~ \omega_l= \omega_k =0.102036 ~~;~~\omega_j=0.169499~~;~~S_j=1.05569$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the all parameters are reals; $t$ is the integration time (s), $\omega$ are oscillation frequencies (1/s) and $S_j$ is an adimensional factor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find that this type of non-convergent integrals are approximated by the &lt;a href=&quot;http://en.wikipedia.org/wiki/Method_of_steepest_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;steepest descendent method&lt;/a&gt; or stationary phase approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Employing this approximation, the integral can be approximated to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(t) \approx \sqrt{\left(\frac{2 \pi}{\sum _{j\neq(l,k)} S_j \omega_j^2 e^{-it^*\omega_j}} \right)}  Exp\{it^*(\omega_H-\omega_l-\omega_k) - \sum _{j\neq(l,k)} S_j [1-e^{-it^*\omega_j}]\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the saddle points value $t^*$, are determined numerically by the transcendental equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega_H-\omega_l-\omega_k=\sum _{j\neq(l,k)} S_j \omega_j e^{-it^*\omega_j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this transcendental equation in Mathematica, for the parameters above, I get this result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C[1] \in Integers \, \&amp;amp;\&amp;amp; \, t ==(0.- 5.89974 i) (0.131673 + (0.+6.28319 i) \, C[1])$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the transcendental equation has number of infinity solutions, which also means that this equation has an infinity number of saddles points. Then the unity partition is used to get a convergent approximation to the integral $f(t)$. In this link make an appointment for this case (&lt;a href=&quot;http://en.wikipedia.org/wiki/Method_of_steepest_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;steepest descendent method&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can anyone give me a hand with formation of the cut off functions, using the partition unity, to get a convergent form of $f(t)$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!!&lt;/p&gt;&#xA;" OwnerUserId="7823" LastActivityDate="2014-04-01T07:07:47.853" Title="Stationary phase approximation for an integral with infinity saddle points?" Tags="|convergence|time-integration|mathematica|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11243" PostTypeId="1" AcceptedAnswerId="11248" CreationDate="2014-04-01T16:10:19.640" Score="3" ViewCount="2870" Body="&lt;p&gt;I have the following LP problem:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min \limits_{\varepsilon, x_{1}, \ldots, x_{n}}f(\varepsilon, x_{1}, \ldots, x_{n}) = \varepsilon \;\;\;\;\; \mathrm{s.t.} \;\;C x \geq 0, \;\; x_{i}^{0} - \varepsilon \leq x_{i} \leq x_{i}^{0} + \varepsilon&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$m \times n$&lt;/span&gt; matrix, and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x^{0}$&lt;/span&gt; are vectors of dimension &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a very sparse matrix, each of its rows contains only &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; non-zero values.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The typical case is &lt;span class=&quot;math-container&quot;&gt;$m = 15000$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n = 5000$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can anybody advise any LP-solver that can provide the best performance for this problem?&lt;/p&gt;&#xA;" OwnerUserId="7716" LastEditorUserId="20688" LastEditDate="2021-12-23T17:48:49.583" LastActivityDate="2021-12-23T17:48:49.583" Title="What sparse linear programming solver it is better to use?" Tags="|linear-algebra|sparse-matrix|iterative-method|linear-programming|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11244" PostTypeId="2" ParentId="11243" CreationDate="2014-04-01T20:01:45.717" Score="1" Body="&lt;p&gt;Your problem seems fairly easy, I would say that any reasonable LP solver should solve it in seconds. So it might be more a choice related to API's, price, licensing. Try out MOSEK, CPLEX or GUROBI among the commercial ones, or CLP from the coin-or project. You can find a list on Wikipedia&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_programming&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Linear_programming&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7396" LastActivityDate="2014-04-01T20:01:45.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11245" PostTypeId="1" AcceptedAnswerId="11250" CreationDate="2014-04-01T20:33:28.160" Score="1" ViewCount="81" Body="&lt;p&gt;I'm teaching myself this regression stuff, so forgive me if this is a basic question. I can't seem to find a discussion of my particular problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm least-squares-ing this overdetermined system $\textbf{Ax}=\textbf{b}$, and I'm interested in minimizing $||\textbf{x}||_1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem is, my $\textbf{A}^T\textbf{A}$ is singular, so I can't use Least Angle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how many of the columns of $\textbf{A}$ are redundant without computing rank explicitly, and I don't really care anyway. I'd rather use that redundancy to get the best 1-norm I can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd settle for a minimized $||\textbf{x}||_2$.&lt;/p&gt;&#xA;" OwnerUserId="7819" LastEditorUserId="7819" LastEditDate="2014-04-01T21:45:42.793" LastActivityDate="2014-04-02T03:32:28.040" Title="Least Angle when $\textbf{A}^T\textbf{A}$ is singular" Tags="|regression|least-squares|constrained-optimization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11246" PostTypeId="2" ParentId="2486" CreationDate="2014-04-01T21:35:55.890" Score="1" Body="&lt;p&gt;Have a look at &lt;a href=&quot;http://spark.apache.org/&quot; rel=&quot;nofollow&quot;&gt;Spark&lt;/a&gt; it's a distributed framework for computations in memory which takes advantage of functional programming. Structure of a program in Spark is very different when compared to MPI, basically you write a code like for single computer, which is automatically distributed as functions to data located in memory. It supports Scala, Java and Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logistic Regression (scala):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//load data to distributed memory&#xA;val points = spark.textFile(...).map(parsePoint).cache()&#xA;var w = Vector.random(D) // current separating plane&#xA;for (i &amp;lt;- 1 to ITERATIONS) {&#xA;  val gradient = points.map(p =&amp;gt;&#xA;    (1 / (1 + exp(-p.y*(w dot p.x))) - 1) * p.y * p.x&#xA;  ).reduce(_ + _)&#xA;  w -= gradient&#xA;}&#xA;println(&quot;Final separating plane: &quot; + w)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There's an extension to called &lt;a href=&quot;http://spark.apache.org/mllib/&quot; rel=&quot;nofollow&quot;&gt;MLib&lt;/a&gt; (Machine Learning library) which uses a Fortran library for some low level computations (for Python I guess numpy is used). So, the idea is simple, concentrate on your algorithm and leave optimizations to lower levels (order of processing, data distribution, etc.).&lt;/p&gt;&#xA;" OwnerUserId="7829" LastActivityDate="2014-04-01T21:35:55.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11247" PostTypeId="1" CreationDate="2014-04-01T23:14:04.930" Score="1" ViewCount="2282" Body="&lt;p&gt;I'm attempting to implement the &lt;a href=&quot;http://arc.aiaa.org/doi/abs/10.2514/3.12946?journalCode=aiaaj&quot; rel=&quot;nofollow&quot;&gt;Weiss and Smith preconditioner&lt;/a&gt; in an existing finite volume code and I am struggling with the idea of dual time stepping. My inner time steps are predictor-corrector, second order explicit with local time step acceleration. The time step is based on the stable CFL number based on the convective speed since it is using the preconditioned values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My outer time step is a second order backward difference implicit method. This is all straight from the Weiss and Smith 1995 AIAA paper. The outer time step is freely chosen to resolve whatever features of interest are desired. The inner time steps over pseudo-time are performed until convergence, in theory when the pseudo-time is marched to infinity. When this happens, the solution should be at the physical time step used for the outer scheme. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two issues with this that are related. Right now in my code, the flow evolves to the time determined by the number of iterations times the inner time step rather than simply becoming my outer time step. For example, if I run a vortex convection case convecting at 1 m/s with an outer time step of 1 s and an inner time step of, say, 0.001 s, I would expect the vortex to move 1 meter each outer timestep. However, it actually convects &lt;code&gt;0.001*&amp;lt;number of inner steps&amp;gt;&lt;/code&gt; meters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I feel like I'm missing something with the whole concept. How does my inner step converge to a physical time of 1s per outer time step rather than a physical time of the number of inner steps times the inner time step? &lt;/p&gt;&#xA;" OwnerUserId="391" LastActivityDate="2014-04-02T14:57:07.477" Title="Dual time stepping for fluid dynamics" Tags="|fluid-dynamics|time-integration|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11248" PostTypeId="2" ParentId="11243" CreationDate="2014-04-02T01:37:57.447" Score="5" Body="&lt;p&gt;The best performance solvers are probably Gurobi or CPLEX; last I checked, Gurobi is slightly faster, but both are competitive. These two commercial solvers are roughly ten times faster than the best open-source solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, as AC_MOSEK points out, your problem is small enough that essentially any functioning LP solver based on sparse data structures (i.e., most of them) will work. Instead of locking your implementation into a specific solver, if you can, I suggest implementing your problem in an optimization framework like GAMS or AMPL, or using an LP-specific modeling framework such as PuLP or Coopr. These frameworks are geared towards making it easy for you to input your problem in a solver-independent, math-like format, so you can experiment with different formulations and solvers rapidly. Then, after settling on a formulation and solver combination that works best for you, you can implement it in a lower-level language for speed for that formulation and specific solver.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-04-02T01:37:57.447" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11249" PostTypeId="1" CreationDate="2014-04-02T02:28:21.350" Score="12" ViewCount="2312" Body="&lt;p&gt;When one want to compute numerical derivatives, the method presented by Bengt Fornberg &lt;a href=&quot;http://amath.colorado.edu/faculty/fornberg/Docs/MathComp_88_FD_formulas.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; (and reported &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference_coefficients&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;) is very convenient (both precise and simple to implement). As the original paper date from 1988, I would like to know whether there is a better alternative today (as (or almost as) simple and more precise) ? &lt;/p&gt;&#xA;" OwnerUserId="6627" LastActivityDate="2019-12-20T08:34:33.523" Title="Numerical derivative and finite difference coefficients: any update of the Fornberg method?" Tags="|finite-difference|precision|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11250" PostTypeId="2" ParentId="11245" CreationDate="2014-04-02T03:32:28.040" Score="2" Body="&lt;p&gt;Minimizing the 2-norm of $x$ among all least squares solutions is relatively easy to do- this is the pseudoinverse solution.  It can be computed using either a rank revealing version of the QR factorization of $A$ (there are specialized versions of this that work well on large and sparse $A$ matrices) or by using the Singular Value Decomposition.  Since you're really more interested in minimizing $\| x \|_{1}$, I'll move on to answering that question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| x \|_{1} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\| Ax-b \|_{2} \leq \delta$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;is equivalent (with a suitable choice of $\lambda$) to the basis pursuit denoising problem (BPDN) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| Ax-b \|_{2}^{2} + \lambda \| x \|_{1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimizing $\| x \|_{1}$ has (in many circumstances) the effect of finding a sparse solution to the problem of approximating $Ax=b$.  These sparse approximation problems have been extensively studied because of their application to problems of compressive sensing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a practical matter, you probably don't want to limit your $x$ to being an actual least squares solution but rather to set some tolerance on $\| Ax-b \|_{2}$.  If for some reason you do need to have an $x$ that is actually a least squares solution, you can start by finding any least squares solution $\hat{x}$, then let $\delta=\| A \hat{x} - b\|_{2}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms for solving these sparse norm approximation problems are specialized enough that you're probably best off making use of existing software rather than trying to implement the algorithms yourself.  I would recommend the list of software packages (as well as the many references to books and papers) at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dsp.rice.edu/cs&quot; rel=&quot;nofollow&quot;&gt;http://dsp.rice.edu/cs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-04-02T03:32:28.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11251" PostTypeId="1" CreationDate="2014-04-02T08:06:46.437" Score="2" ViewCount="214" Body="&lt;p&gt;To compute the evolution of a free surface between two incompressible, immiscible liquids, two tightly coupled equations have to be solved, the volume fraction advection and the Navier-Stokes equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial\alpha}{\partial t}=\nabla \cdot (U\alpha)$$&#xA;$$\frac{\partial \rho U}{\partial t} + \nabla \cdot (\rho U U)-\nabla\cdot (\mu \nabla U) = -\nabla p + \sigma\kappa\nabla\alpha$$&#xA;with $\kappa = -\nabla \cdot \frac{\nabla \alpha}{|\nabla \alpha|}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These equations are tightly coupled, and by far the most common approach is to couple them &lt;s&gt;explicitly&lt;/s&gt; using a very small time step, i.e. solve for $\alpha$ once, then once for $U$ and then proceed to the next time step. Regardless of the time marching scheme used to solve $\alpha$ and $U$ very small time steps are required to maintain the coupling of the two equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this is probably a naive question, but why is it not helpful to wrap the solution in an outer loop, relax U and $\alpha$ and solve with larger time steps?&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="7626" LastEditDate="2014-04-02T22:07:18.757" LastActivityDate="2014-04-02T22:07:18.757" Title="Segregated solving of a tightly coupled system of PDEs" Tags="|pde|operator-splitting|coupling|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="11252" PostTypeId="2" ParentId="11247" CreationDate="2014-04-02T14:37:34.040" Score="7" Body="&lt;p&gt;I believe you're switching around the definitions of &quot;inner&quot; and &quot;outer&quot; stepping (this nomenclature gets worse in segregated schemes where you have at least 3 separate iterative schemes going on.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The core of what you're missing is that &quot;pseudo-time&quot; isn't a time at all; it's just a convenient way to achieve an iterative solution. Take a 1st order backward difference in time equation (spatial discretization doesn't matter, except that we're evaluating it at time n+1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} + f(u)\approx \frac{u^{n+1}-u^n}{\Delta t} + f(u)^{n+1} \equiv R(u) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that expression is an &lt;em&gt;implicit&lt;/em&gt; equation and you generally need some kind of iterative method to solve it for time $n+1$. Here you can use any (non-)linear algebraic solver to find $u$ such that $R(u)=0$. An alternative is to cast that problem as an unsteady one $\frac{du}{d\tau}=R(u)$ and step forward until the residual $R$ is 0 (&lt;em&gt;I think of this as equivalent to using Jacobi iteration for solving R(u)=0&lt;/em&gt;). You may have seen this as a first method for solving laplace's elliptic equation $\nabla^2 u=0$ by posing it as a parabolic equation $\frac{du}{dt} = \nabla^2 u$ and stepping till convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can modify that expression to this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R^{\star}(u) = \frac{u-u^n}{\Delta t} + f(u)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{u^{\star}-u}{\Delta \tau} + R^{\star}(u) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u^{\star}$ is the value at some pseudo time and $\Delta \tau$ is the pseudo-timestep. This is equal to the above whenever $u^{\star}=u$ which implies $u=u^{n+1}$.  That we can step forward explicitly in pseudo-time. The subscripts/superscripts for dual time-stepping always end up getting a little cloudy - see &quot;Multigrid Unsteady Navier-Stokes Calculations with Aeroeleastic Applications&quot; for example of 2nd order. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In implementation, dual time-stepping is really easy once you wrap your head around how it works. It's algorithmically equivalent to your original explicit time-stepping scheme &lt;strong&gt;except now you have an additional source term representing the implicit unsteady term&lt;/strong&gt;, and for each time step you iterate until the residual is near-zero. Another way of looking at this is to realize that each physical unsteady timestep is equivalent to solving a single &quot;steady&quot; problem.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2014-04-02T14:57:07.477" LastActivityDate="2014-04-02T14:57:07.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11257" PostTypeId="1" CreationDate="2014-04-02T22:25:32.903" Score="2" ViewCount="504" Body="&lt;p&gt;In Jacobi method's Wikipedia article there's a section that describes Weighted Jacobi method: &lt;a href=&quot;http://en.wikipedia.org/wiki/Jacobi_method#Weighted_Jacobi_method&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Jacobi_method#Weighted_Jacobi_method&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to implement the Weighted version of Gauss-Seidel but I can't find any references of the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you provide some information about it?&lt;/p&gt;&#xA;" OwnerUserId="7843" LastActivityDate="2014-04-09T13:24:12.083" Title="Weighted Gauss-Seidel Algorithm" Tags="|linear-algebra|optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11258" PostTypeId="1" CreationDate="2014-04-03T09:52:20.907" Score="1" ViewCount="1031" Body="&lt;p&gt;I am trying to solve the 2D convection-diffusion equation, which in non-dimensional variables can be expressed for my problem as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial c}{\partial t} = \lambda^2\frac{\partial^2c}{\partial x^2} + \frac{\partial^2c}{\partial y^2} - 6\, Pe\, y (1 - y)\frac{\partial c}{\partial x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $\lambda$ is a geometrical parameter (an aspect ratio of sorts) and $Pe$ is the Peclet number. The $y\,(1 - y)$ appears due to a parabolic velocity profile at the inlet (left wall). The boundary conditions are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Left wall: Dirichlet type&lt;/li&gt;&#xA;&lt;li&gt;Top and Right walls: No flux, so $y$ and $x$ gradients are zero respectively&lt;/li&gt;&#xA;&lt;li&gt;Bottom wall: Robin type boundary conditions. Basically a surface reaction that follows Langmuir kinetics.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have discretized the equation using central differences for space and am using the Crank-Nicholson scheme to implicitly march in time. My code blows up (extremely large values of c, ~ $10^{134}$) for $Pe &amp;gt; 100$. It works fine when $Pe$ is below that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does this happen? What do I do?&lt;/p&gt;&#xA;" OwnerDisplayName="user7847" LastActivityDate="2014-05-03T12:13:01.680" Title="Solving the convection-diffusion equation using finite differences at high Peclet numbers" Tags="|finite-difference|advection-diffusion|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11259" PostTypeId="2" ParentId="11249" CreationDate="2014-04-03T10:33:44.567" Score="1" Body="&lt;p&gt;I'm not aware of anyone having improved Fornberg's algorithm (see also his slightly more recent &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/S0036144596322507&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;). As an aside, it seems to me that looking at his algorithm as a way to compute numerical derivatives is not right. All he's done is derive an efficient algorithm to compute &lt;em&gt;weights&lt;/em&gt; for finite-difference methods. The advantage of his method is that it gives you the weights for all derivatives up to the desired derivative in one go.   &lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2014-04-03T10:33:44.567" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11260" PostTypeId="1" CreationDate="2014-04-03T10:40:15.103" Score="5" ViewCount="394" Body="&lt;p&gt;In short, I'm looking for either: &#xA;(1) Publications or other sources dealing with contour/isosurface finding algorithms, so that I can write my own implementation (and parallelize as best I can), or &#xA;(2) Suggestions about high-level libraries in Fortran, C/C++, or other languages, that may be useful for finding such surfaces from a scalar 3D grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently analyzing data in checkpoint files from a very large hydrodynamics simulation (AMR w/ Lagrangian sink particles -- data in HDF5 format), and would like to find an algorithm for efficiently computing equipotential surfaces (a.k.a. countours, or isosurfaces) from the gravitational scalar potential grid. In particular, I need to be able to find the largest &lt;em&gt;closed&lt;/em&gt; equipotential surface around a chosen local minimum (to within some acceptable step error), &lt;em&gt;that may not enclose any other minima&lt;/em&gt;. This surface will be used as a selection criteria for further analysis of the volume formed by the enclosed cells. This link (particularly section 3) provides a description of the procedure I'm roughly attempting to replicate, though the IDL code provided takes 2D FITS maps of column density as input, and their data is much smaller: &lt;a href=&quot;http://www.astro.umd.edu/~hgong/GRID_core.htm&quot; rel=&quot;nofollow&quot;&gt;http://www.astro.umd.edu/~hgong/GRID_core.htm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to be very similar to the problem of computing and rendering isosurfaces in graphics (i.e. some variant of the marching cubes algorithm seems relevant), but since I'm interested in using the surface as a boundary for selecting all interior grid cells, rather than visualizing it, most of the libraries I'm familiar with seem ill suited (e.g. VTK). I would happily be proven wrong, however, if someone knows how I might use a graphics algorithm for this purpose -- maybe there's a way to find the isosurface with a graphics library routine and then simply extract information about the surface location or boundary cells?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions about how to approach this problem, or ideas about potentially useful libraries are much appreciated. If I write the algorithm more or less from scratch, I will likely use Fortran (easier for me to parallelize), but I would be happy to use C/C++ or other languages if they support libraries that are better suited to the task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: After a bit more research, I'm optimistic that a contour tree will be the perfect data structure for this problem. &lt;a href=&quot;http://www.csi.ucd.ie/staff/hcarr/home/research/contourtrees/contourtrees.html&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is a rather informal description. It seems to provide a wonderfully abstract graph representation of how the topology of the set of isosurfaces varies with isovalue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Local maxima and minima are represented as leaf nodes, and are said to create or terminate a &quot;component&quot; (graph edge). Interior vertices/nodes represent the joining or splitting of components, or the changing of genus, at critical points -- topological events, in other words. It is also possible to sweep through the space between the isosurfaces associated with two events, and assign all data points in this region to the component connecting the two events. &lt;a href=&quot;http://graphics.stanford.edu/courses/cs468-03-winter/Papers/carr-contour-soda.pdf&quot; rel=&quot;nofollow&quot;&gt;This&lt;/a&gt; more thorough publication calls this an &lt;em&gt;augmented&lt;/em&gt; contour tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this in mind, my current strategy is pretty straightforward: once an augmented contour tree has been produced, then the cells I want to associate with a local minimum are simply those associated with the component between the local minimum leaf node and the closest topological event vertex.&lt;/p&gt;&#xA;" OwnerUserId="7845" LastEditorUserId="7845" LastEditDate="2014-04-04T22:47:37.317" LastActivityDate="2014-04-04T22:47:37.317" Title="Finding closed equipotential surfaces on a 3D grid" Tags="|algorithms|computational-geometry|libraries|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11261" PostTypeId="2" ParentId="11258" CreationDate="2014-04-03T11:47:00.913" Score="1" Body="&lt;p&gt;To quote &lt;a href=&quot;http://www.osti.gov/scitech/biblio/5603843&quot; rel=&quot;nofollow&quot;&gt;Gresho&lt;/a&gt;: &quot;The wiggles are telling you something.&quot; Namely, you need more mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without seeing your solution, or knowing $\lambda$, that's my best guess. I suspect that if you look at your first time step for $Pe$ greater than but near 100, the solution hasn't completely blown up, but it does appear oscillatory. If that's the case, you either need to stabilize, or you need more mesh in the boundary layer(s). &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-04-03T11:47:00.913" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11262" PostTypeId="2" ParentId="11260" CreationDate="2014-04-03T13:00:48.170" Score="3" Body="&lt;p&gt;I would still try to see if it is possible to extract the information from (or possibly to extend) VTK. VTK is such a widely used and high quality library that you would probably spend the better part of a year just reimplementing the pieces you need from VTK, and you'd still be only a fraction as fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the very least, you should try to understand the code in VTK that computes isosurfaces.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-03T13:00:48.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11263" PostTypeId="2" ParentId="5423" CreationDate="2014-04-03T14:17:50.293" Score="0" Body="&lt;p&gt;Start from sparse matrix $E^T$ of size elements$\times$dofs.&#xA;$$&#xA;E_{ij}^T = &#xA;\left\{&#xA;\begin{array}{cc}&#xA;1 &amp;amp; \mathrm{if\ dof}\ j \in \mathrm{element}\ i \\&#xA;0 &amp;amp; \mathrm{elsewhere}&#xA;\end{array}&#xA;\right.&#xA;$$&#xA;Matrix $A = E\, E^T$ has the sparsity pattern you are looking for.&#xA;Keep in mind that implementing $E^T$ is easier, this is why i&#xA;defined $E^T$ instead of $E$.&lt;/p&gt;&#xA;" OwnerUserId="6574" LastActivityDate="2014-04-03T14:17:50.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11264" PostTypeId="1" CreationDate="2014-04-03T14:41:27.407" Score="1" ViewCount="180" Body="&lt;p&gt;I have implemented a simple logistic regression function with IRLS algorithm using the armadillo linear algebra libray&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;#include &amp;lt;boost/math/distributions/normal.hpp&amp;gt;&#xA;#include &amp;lt;boost/math/distributions/students_t.hpp&amp;gt;&#xA;#include &amp;lt;armadillo&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;&#xA;using namespace boost::math;&#xA;arma::mat getW(&#xA;    arma::mat&amp;amp; beta,&#xA;    arma::mat&amp;amp; X,&#xA;    std::string family,&#xA;    std::string link&#xA;)&#xA;{&#xA;    arma::mat w;&#xA;    if(family == &quot;poisson&quot;) {&#xA;        if(link == &quot;identity&quot;) {&#xA;            w = arma::diagmat(1/(X * beta));&#xA;        }&#xA;    }&#xA;    else if(family == &quot;binomial&quot;) {&#xA;        if(link == &quot;logit&quot;) {&#xA;            arma::colvec tmp = exp(X * beta);&#xA;            w = arma::diagmat(tmp/pow(1+tmp, 2));&#xA;        }&#xA;    }&#xA;    else {&#xA;        throw 1;&#xA;    }&#xA;    return w;&#xA;}&#xA;&#xA;arma::mat getz(&#xA;    arma::mat&amp;amp; y,&#xA;    arma::mat&amp;amp; beta,&#xA;    arma::mat&amp;amp; X,&#xA;    std::string family,&#xA;    std::string link&#xA;)&#xA;{&#xA;    arma::mat z;&#xA;    if(family == &quot;poisson&quot;) {&#xA;        if(link == &quot;identity&quot;) {&#xA;            z = y;&#xA;        }&#xA;    }&#xA;    else if(family == &quot;binomial&quot;) {&#xA;        if(link==&quot;logit&quot;) {&#xA;            arma::mat tmp = exp(X * beta);&#xA;            z = X*beta + y % (pow(1+tmp, 2)/tmp) - 1 - tmp;&#xA;        }&#xA;    }&#xA;    else {&#xA;        throw 1;&#xA;    }&#xA;    return z;&#xA;}&#xA;&#xA;inline arma::mat glmMat(&#xA;    arma::mat&amp;amp; y,&#xA;    arma::mat&amp;amp; x,&#xA;    std::string family,&#xA;    std::string link&#xA;)&#xA;{&#xA;&#xA;    int n = x.n_rows;&#xA;    int k = x.n_cols;&#xA;&#xA;    // add a col of all ones&#xA;    arma::mat allOne(n, 1, arma::fill::ones);&#xA;    x.insert_cols(0, allOne);&#xA;    ++k;&#xA;&#xA;    arma::mat res(k, 4);&#xA;&#xA;    if(family==&quot;binomial&quot; and link==&quot;logit&quot;)&#xA;    {&#xA;        arma::mat coef(k, 1, arma::fill::zeros);&#xA;        arma::mat W = getW(coef, x, family, link);&#xA;        arma::mat z = getz(y, coef, x, family, link);&#xA;&#xA;        try {&#xA;            arma::mat J = x.t() * W * x;&#xA;            arma::colvec coef1 = arma::solve(J, x.t()*W*z);&#xA;            double coefdiff = max(abs(coef - coef1));&#xA;            while(coefdiff &amp;gt;= 0.00001) {&#xA;                coef = coef1;&#xA;                W = getW(coef, x, family, link);&#xA;                z = getz(y, coef, x, family, link);&#xA;                J = x.t() * W * x;&#xA;                coef1 = arma::solve(J, x.t()*W*z);&#xA;                coefdiff = max(abs(coef - coef1));&#xA;            }&#xA;&#xA;&#xA;            arma::mat coefVarMatrix = J.i();&#xA;            arma::colvec coefVar = coefVarMatrix.diag();&#xA;            arma::colvec coefSe = pow(coefVar, .5);&#xA;            arma::colvec zscore = coef / coefSe;&#xA;            res.col(0) = coef;&#xA;            res.col(1) = coefSe;&#xA;            res.col(2) = zscore;&#xA;&#xA;            // calculate p values&#xA;            auto d = normal_distribution&amp;lt;&amp;gt;();&#xA;            for(int i=0; i&amp;lt;k; i++) {&#xA;                double p = 2 * (1 - cdf(d, fabs(res(i, 2))));&#xA;                if(p &amp;lt; 0 or p &amp;gt; 1) {&#xA;                    std::cerr &amp;lt;&amp;lt; &quot;Pval is abnormal from glm, dumping data to /tmp/tmpx.csv and /tmp/tmpy.csv&quot; &amp;lt;&amp;lt; std::endl;&#xA;                    x.save(&quot;/tmp/tmpx.csv&quot;, arma::csv_ascii);&#xA;                    y.save(&quot;/tmp/tmpy.csv&quot;, arma::csv_ascii);&#xA;                    throw 1;&#xA;                }&#xA;                res(i, 3) = p;&#xA;            }&#xA;        }&#xA;        catch(...) {&#xA;            std::cout &amp;lt;&amp;lt; &quot;something wrong...&quot; &amp;lt;&amp;lt; std::endl;&#xA;        }&#xA;    }&#xA;    else {&#xA;        throw 1;&#xA;    }&#xA;    return res;&#xA;}&#xA;&#xA;int main(int argc, char const* argv[])&#xA;{&#xA;    {&#xA;        int nr = 5000;&#xA;        int ncx = 50;&#xA;        arma::mat x(nr, ncx, arma::fill::randu);&#xA;        arma::mat y = arma::randi&amp;lt;arma::mat&amp;gt;(nr, 1, arma::distr_param(0, 1));&#xA;        arma::mat xcol;&#xA;        arma::mat res(ncx, 4);&#xA;        for(int i=0; i&amp;lt;ncx; i++) {&#xA;            xcol = x(arma::span::all, i);&#xA;            res.row(i) = (glmMat(y, xcol, &quot;binomial&quot;, &quot;logit&quot;)).row(1);&#xA;        }&#xA;        res.print(&quot;res..........&quot;);&#xA;    }&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Compiled it like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g++ glm.cpp --std=c++11 -larmadillo -llapack -lblas -o bin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The main function simulates a 5000x50 data set and performs logistic regression on each of them, the whole process takes about 23 seconds on my laptop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing largely the same thing in R, it takes about 2 seconds:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;testglm = function() {&#xA;        x = matrix(rnorm(5000*50), 5000)&#xA;        y = matrix(sample(0:1, 5000, repl=T), 5000)&#xA;        res = apply(x, 2, function(coli) summary(glm(y~coli, family=binomial))$coef[2, ])&#xA;        # print(res)&#xA;        }&#xA;&#xA;system.time(testglm())&#xA;   user  system elapsed &#xA;  2.049   0.000   2.049 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am wondering what went wrong with my implementation?&lt;/p&gt;&#xA;" OwnerUserId="7853" LastActivityDate="2014-05-14T01:23:22.743" Title="what went wrong with my logistic regression implementation in c++?" Tags="|c++|regression|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11265" PostTypeId="1" CreationDate="2014-04-03T19:54:52.800" Score="1" ViewCount="185" Body="&lt;p&gt;On a recommendation from &lt;a href=&quot;https://mathematica.stackexchange.com/&quot;&gt;Mathematica.SE&lt;/a&gt;, I am posting this on Computational Science.SE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to quantify stiffness of an ODE by relating it to the fine-ness with which NDSolve  treats it's time step.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;BACKGROUND&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For the uninitiated:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, when an ordinary differential equation is stiff and we attempt to solve it with NDSolve in &lt;em&gt;Mathematica&lt;/em&gt;, it would employ a stiff solver to detect and reconcile stiffness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Stiff_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stiff equation shown on wikipedia&lt;/a&gt; to demonstrate this. This equation is solved using BDF (backward difference formulation/formula) which is a recognized method to solve stiff equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Equation and solution using NDSolve with BDF&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tMax=100;&#xA;rSol=&#xA;r/.NDSolve[&#xA;r'[t]==-15 r[t]&amp;amp;&amp;amp;r[0]==1,r,&#xA;{t,0,tMax},Method-&amp;gt;{&quot;BDF&quot;,&quot;MaxDifferenceOrder&quot;-&amp;gt;5}][[1]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Plots of solution r vs t&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{Plot[&#xA;rSol[t],&#xA;{t,0,100},&#xA;PlotRange-&amp;gt;{{0,tMax},Automatic}&#xA;],&#xA;LogPlot[&#xA;rSol[t],&#xA;{t,0,100},&#xA;PlotRange-&amp;gt;{{0,tMax},Automatic}&#xA;]}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fig 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/l6lSM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fig 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uz2Uw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fig 1 shows that up to t=29.9 or so, there is considerable stiffness (unstable oscillations) that is reconciled by ensuring that the time step is sufficiently small.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;THE REAL QUESTION:&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;What would be a good plot to quantify this stiffness and internal reconciliation/changing of time step? How can I best quantify stiffness graphically in &lt;em&gt;Mathematica&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="7857" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2014-04-04T04:52:15.253" Title="Stiff Equations - What to plot as a qualitative or quantitative measure of stiffness" Tags="|finite-difference|stiffness|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11266" PostTypeId="2" ParentId="11265" CreationDate="2014-04-04T04:52:15.253" Score="2" Body="&lt;p&gt;Stiffness has no precise quantitative definition. (See my answer &lt;a href=&quot;https://scicomp.stackexchange.com/a/892/276&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Common numerical proxies would be things like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;stiffness ratio: find the (unsigned) ratio of largest magnitude eigenvalue to smallest magnitude eigenvalue of the Jacobian matrix of your right-hand side; estimates are probably fine&lt;/li&gt;&#xA;&lt;li&gt;ratio of time step taken to maximum stable time step: a ratio closer to 1 indicates a stability-limited time step, so your problem is probably stiff; a ratio closer to 0 indicates an accuracy-limited time step, and probably a non-stiff problem&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There may be other metrics out there; these are the first two that come to mind.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-04-04T04:52:15.253" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11268" PostTypeId="2" ParentId="11232" CreationDate="2014-04-04T08:11:08.467" Score="1" Body="&lt;p&gt;This should probably be a comment rather than an answer but I can't comment...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can define your curve geometry (2) with some function like (its just an example, it can of course be modified) :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = (a x + b) \theta(x-x_0) +  \left(\sqrt{R^2 - (x+x_c)^2} - y_c\right)\theta(x_0-x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a,b,x_0,x_c,y_c$ are free parameters such that &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x_0$ is the $x$ coordinate where (2) starts to bent&lt;/li&gt;&#xA;&lt;li&gt;$(x_c,y_c)$ is the coordinate of the center of the circle of radius $R$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then you could use a Levenberg-Marquardt algorithm to fit $f(x)$ to your datas.&lt;/p&gt;&#xA;" OwnerUserId="4626" LastActivityDate="2014-04-04T08:11:08.467" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11269" PostTypeId="1" CreationDate="2014-04-04T13:02:50.217" Score="3" ViewCount="1473" Body="&lt;p&gt;I want to integrate a particle path in 2D using the &lt;code&gt;integrate.ode module&lt;/code&gt;. Things that are a bit different in my case are that, I only want to integrate up to a certain position, determined by the maximum allowed x coordinate of the particle: &lt;code&gt;x_max&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main issue I have is that the particle may first move very slowly and then gather more speed later on. Hence I don't want to waste effort with small time steps in this region. The algorithm should be able to adjust such that smaller time steps are used when the particle velocity becomes high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end result if I plot the particle trajectory in the &quot;phase-space&quot; I should have a smooth line.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some rough pseudo-code below for this purpose:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;backend = &quot;dopri5&quot;&#xA;x_max = 1&#xA;solver = ode(f)&#xA;solver.set_integrator(backend)  &#xA;solver.set_initial_value(y0, t0)&#xA;t, y = [t0], [y0]&#xA;k = 1.2&#xA;while solver.successful() and solver.y[0] &amp;lt; x_max&#xA;   solver.integrate(solver.t+dt)&#xA;   t.append(solver.t)&#xA;   y.append(solver.y)&#xA;   v_current = numpy.linalg.norm(y[-1])&#xA;   v_previous = numpy.linalg.norm(y[-2])&#xA;   if numpy.abs( v_current-v_previous ) &amp;gt; k * v_previous:&#xA;       dt = 0.8*dt&#xA;       del y[-1]&#xA;   else:&#xA;       dt = dt*1.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Trouble is this algorithm may not be that robust, as choosing the values &lt;code&gt;k, 1.2, 0.8&lt;/code&gt; is somewhat arbitrary and may cause some stability issues with the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I also want to be able to plot points that are equally spaced in time on the trajectory, to give an indication of how the speed of the particle changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest a better way of doing this?&lt;/p&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="7864" LastEditDate="2014-04-04T13:29:34.833" LastActivityDate="2015-11-04T01:37:24.970" Title="Adaptive ODE algorithm in Python" Tags="|algorithms|python|ode|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11270" PostTypeId="1" CreationDate="2014-04-04T13:04:02.733" Score="9" ViewCount="297" Body="&lt;p&gt;In the solution of sequences of linear systems&#xA;$$A_ix_i=b_i\quad\text{for}\quad i=1,2,\dots$$&#xA;with Krylov subspace methods, data can be recycled from already solved linear systems in order to speed up the computation of subsequent linear systems (complementary to preconditioning). For the analysis of recycling strategies I developed in my PhD thesis and which are implemented in &lt;a href=&quot;https://github.com/andrenarchy/krypy&quot;&gt;KryPy&lt;/a&gt;, I'm looking for interesting cases of sequences of linear systems that satisfy the following conditions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The (possibly preconditioned) matrices exhibit only a few eigenvalues close to zero that are separated from the remaining spectrum. Often, the presence of a few eigenvalues of small magnitude is responsible for an initial slow convergence of CG, MINRES or GMRES.&lt;/li&gt;&#xA;&lt;li&gt;The matrices are sparse.&lt;/li&gt;&#xA;&lt;li&gt;The problem can be boiled down to small scale (dim&amp;lt;1000).&lt;/li&gt;&#xA;&lt;li&gt;The matrices and/or the right hand sides of the linear systems may change but the difference should be within limits.&lt;/li&gt;&#xA;&lt;li&gt;The problem can be implemented in a few lines of code, preferably based on well-known Python packages such as NumPy, SciPy, FEniCS, PyAMG, ...&lt;/li&gt;&#xA;&lt;li&gt;Symmetric/Hermitian matrices would be nice but are not a must.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Candidates for such sequences are time-dependent problems, nonlinear problems (e.g. with Newton's method), parameter-dependent problems, shifted linear systems, ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One application that satisfies the above criteria and benefits from recycling Krylov solvers are nonlinear Schrödinger equations, see &lt;a href=&quot;http://arxiv.org/abs/1208.0264&quot;&gt;this paper on arxiv&lt;/a&gt;. I'm now looking for a simpler model problem in order to demonstrate the mathematical idea without being computationally cumbersome.&lt;/p&gt;&#xA;" OwnerUserId="532" LastActivityDate="2014-04-04T13:04:02.733" Title="Wanted: sequences of linear systems for recycling Krylov solver analysis" Tags="|linear-algebra|matrix|linear-solver|krylov-method|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11271" PostTypeId="1" CreationDate="2014-04-04T13:14:01.797" Score="1" ViewCount="471" Body="&lt;p&gt;Gaussian09 prints coordinates where the local coordinate system as center at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C = (\Sigma n_i*X_i)/(\Sigma n_i)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $n_i$ is the Mass Numer of the atom. However, the Mass Number is not actually the mass of the nucleus. If we instead choose $m_i$, the true mass of each atom, as the weighting factor, we do not get the origin as the center of mass.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it ok to choose the origin that Gaussian09 gives us as the center of mass? Why or why not?&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2014-04-04T13:14:01.797" Title="Center of Mass for molecule in Gaussian09" Tags="|computational-chemistry|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11272" PostTypeId="1" CreationDate="2014-04-04T13:39:22.110" Score="3" ViewCount="457" Body="&lt;p&gt;I'm looking for a specific type of Delaunay tessellation algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm should be: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;incremental so that I can add new sites inside known simplexes (i.e. no searching for the right simplex is necessary)&lt;/li&gt;&#xA;&lt;li&gt;usable with a high number of dimensions (100 or more) &lt;/li&gt;&#xA;&lt;li&gt;usable with a high number of sites (10000 or more)&lt;/li&gt;&#xA;&lt;li&gt;(bonus) parallelizable to multiple cores&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So basically I want to start off with a single simplex, add a site to split the simplex, and then iteratively select a simplex, add a site into it, and fix the Delaynay tesselation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I looking for a unicorn here, or does something like this actually exist?&#xA;I've been using &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0167865583900697&quot; rel=&quot;nofollow&quot;&gt;Devijver and Dekesel&lt;/a&gt;'s algorithm, but it's time complexity in number of sites and dimensions is no good enough.&lt;/p&gt;&#xA;" OwnerUserId="7863" LastActivityDate="2014-12-31T17:18:03.203" Title="Best incremental multidimensional Delaunay tessellation algorithm" Tags="|algorithms|parallel-computing|computational-geometry|high-dimensional|delaunay-triangulation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11273" PostTypeId="2" ParentId="11269" CreationDate="2014-04-04T18:21:03.053" Score="1" Body="&lt;p&gt;Yes, the event detection capability in SUNDIALS should work for you. I do not know if it is implemented in the unofficial Python interfaces to SUNDIALS; you should look at Assimulo, scikits-odes, CasADi, and PySUNDIALS, roughly in that order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for equally spaced times, you should be able to specify times to which you want to integrate the system in your integrator; the integrator should store the internal state of its time steps, and interpolate its current solution array to give you solutions at the times you query.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-04-04T18:21:03.053" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11274" PostTypeId="1" CreationDate="2014-04-04T22:51:07.973" Score="8" ViewCount="400" Body="&lt;p&gt;I'm asking about the solution of the Helmholtz equation on a periodic domain with piecewise constant wavespeed in different frequency regimes. One possible approach is to solving this problem is to write down integral equations on the boundary surfaces in terms of the Green's function of the system. Since the domain is periodic, this will be a periodic Green's function like&#xA;$$ G(r,r') = \sum_L G_0(r,r'+L) $$&#xA;where $L$ is a lattice vector, and $G_0$ is something like&#xA;$$ G_0(r,r') = \frac{e^{ik(r-r')}}{|r-r'|} $$&#xA;My question concerns how the computational cost of this method scales with frequency ($k$). Does it become computationally more difficult at low or high frequencies due to needing to include more terms in the lattice sum?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: People seem to be answering different questions than the one I'm asking. I should clarify that I'm not interested in implementing such a method. I'm merely asking about the theoretical difficulties, as background for understanding the strengths and (moreso) weakness of the method in different frequency regimes. The kinds of problems I have in mind are (more or less) computing modes of periodic arrays of waveguides.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2014-04-07T16:44:36.597" LastActivityDate="2017-01-22T07:48:41.860" Title="Periodic Green's functions in integral equation methods in different frequency regimes" Tags="|integral-equations|helmholtz-equation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11275" PostTypeId="2" ParentId="11272" CreationDate="2014-04-05T07:38:30.907" Score="0" Body="&lt;p&gt;Not sure if this is what you're looking for but try &lt;a href=&quot;http://www.qhull.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.qhull.org/&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's the library used by Matlab, Octave and scipy.&lt;/p&gt;&#xA;" OwnerUserId="7869" LastActivityDate="2014-04-05T07:38:30.907" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11276" PostTypeId="1" CreationDate="2014-04-05T07:52:08.810" Score="29" ViewCount="22016" Body="&lt;p&gt;I am looking for a C++ tensor library that supports dimension-agnostic code. Specifically, I need to perform operations along each dimension (up to 3), e.g. calculating a weighted sum. The dimensions is a template parameter (and thus a compile-time constant). Another constraint is that the library should be relatively lightweight, so rather Eigen/Boost-style than Trilinos/PETSc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I have had a look at Eigen and think it almost fits the profile exactly, if it weren't limited to 2D tensors. If I am mistaken by this, please correct me.&lt;/p&gt;&#xA;" OwnerUserId="1724" LastActivityDate="2022-05-11T14:04:42.887" Title="Fast, lightweight C++ tensor library for dimension-agnostic code" Tags="|c++|libraries|tensor|" AnswerCount="10" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11277" PostTypeId="2" ParentId="11276" CreationDate="2014-04-05T12:40:49.880" Score="6" Body="&lt;p&gt;For what its worth, Eigen does have a Tensor class as an unsupported module.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://eigen.tuxfamily.org/dox-devel/unsupported/group__CXX11__Tensor__Module.html&quot; rel=&quot;noreferrer&quot;&gt;http://eigen.tuxfamily.org/dox-devel/unsupported/group_&lt;em&gt;CXX11&lt;/em&gt;_Tensor__Module.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't used it myself so can't say more about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Armadillo class library has a 3rd-order tensor class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arma.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;http://arma.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't used the tensor capabilities of Armadillo either but have experimented with the standard matrix classes and they seem relatively easy to use and have good performance.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-04-05T12:40:49.880" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11278" PostTypeId="2" ParentId="11276" CreationDate="2014-04-05T13:27:45.547" Score="1" Body="&lt;p&gt;I have not used it myself but &lt;a href=&quot;https://github.com/juanjosegarciaripoll/tensor&quot; rel=&quot;nofollow&quot;&gt;libtensor&lt;/a&gt; seems to fulfill your requirements.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2014-04-05T13:27:45.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11279" PostTypeId="2" ParentId="11276" CreationDate="2014-04-05T16:56:19.007" Score="1" Body="&lt;p&gt;this is a multidimensional array C++ library &lt;a href=&quot;https://github.com/ContinuumIO/libdynd&quot; rel=&quot;nofollow&quot;&gt;https://github.com/ContinuumIO/libdynd&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5046" LastActivityDate="2014-04-05T16:56:19.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11280" PostTypeId="1" AcceptedAnswerId="11283" CreationDate="2014-04-05T20:59:57.240" Score="1" ViewCount="197" Body="&lt;p&gt;Every CPU vendor seems to make BLAS libraries that are specialized to run on their hardware. Do they do the same for LAPACK? Or is that a non-issue because LAPACK is written entirely in terms of BLAS library calls? &lt;/p&gt;&#xA;" OwnerUserId="1420" LastActivityDate="2014-04-05T23:38:15.847" Title="Do vendors release their own LAPACK library?" Tags="|lapack|blas|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11281" PostTypeId="2" ParentId="11280" CreationDate="2014-04-05T22:21:04.063" Score="2" Body="&lt;p&gt;So both Intel and AMD provide LAPACK-like software with their BLAS, called MKL and ACML respectively. I don't know if any of the ARM developers provide similar software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NETLIB version of LAPACK depends heavily on BLAS, and most of the performance gain could probably be had by simply linking NETLIB LAPACK against MKL or ACML BLAS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would guess the difference between NETLIB and MKL or ACML LAPACK would quite small, however as LAPACK is already part of the vendor BLAS distribution, there's no reason NOT to use that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR: They do distribute LAPACK, but it is mainly a non issue.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-04-05T22:21:04.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11282" PostTypeId="1" CreationDate="2014-04-05T23:13:56.850" Score="5" ViewCount="182" Body="&lt;p&gt;I have a dense Hermitian matrix that is approximately unitary, so it has eigenvalues that are $\sim \pm1$. I would like to compute all the eigenvectors corresponding to the $+1$ eigenvalue (not necessarily a small fraction of the total number of eigenvalues). Is there some more efficient way to accomplish this than tridiagonal QR?&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-04-10T06:40:05.243" Title="Computing eigendecomposition of a Hermitian matrix that is almost unitary" Tags="|linear-algebra|eigensystem|eigenvalues|dense-matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11283" PostTypeId="2" ParentId="11280" CreationDate="2014-04-05T23:38:15.847" Score="2" Body="&lt;p&gt;Vendor BLAS like Intel's MKL or AMD's ACML, and even GotoBLAS (now OpenBLAS) provide tuned versions of a tiny subset of Lapack, like linear solvers and least squares solvers. I imagine this is only because these routines are among the most commonly used in Lapack. Besides this, I think there might also be routines for eigensystems and such, but they certainly are not a complete Lapack implementation.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-04-05T23:38:15.847" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11284" PostTypeId="1" CreationDate="2014-04-06T03:08:01.333" Score="2" ViewCount="647" Body="&lt;p&gt;I am a novice at PETSC, and I have been trying to  write an FVM code for steady heat conduction in 2D using  PETSC (square, regular grid, Dirichlet boundaries)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the large matrix , say A, will be sparse I  declare 5 non-zeros per row and use &lt;code&gt;MatCreateSeqAIJ()&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about assembling the entries of the matrix. In each CV, I call &lt;code&gt;MatSetValues()&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and add the 5 coefficients into A using local arrays II, JJ and Values_IJ&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is this the &quot;right&quot; or &quot;usual&quot; way to do this? I ask because, it seems to me, am assembling by implicitly using a &lt;a href=&quot;http://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29&quot; rel=&quot;nofollow&quot;&gt;COO type representation&lt;/a&gt; of sparsity in my loop while the matrix representation in PETSC is &lt;a href=&quot;http://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR_or_CRS.29&quot; rel=&quot;nofollow&quot;&gt;CSR&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If not, is there a way to create the local arrays II, JJ and Values_IJ to fit the CSR format of the matrix and then call &lt;code&gt;MatSetValues()&lt;/code&gt;? &lt;/p&gt;&#xA;" OwnerUserId="7728" LastEditorUserId="549" LastEditDate="2014-04-06T04:40:18.537" LastActivityDate="2020-02-06T11:01:40.283" Title="Assembling sparse matrix in PETSC for Poisson equation" Tags="|finite-difference|petsc|sparse-matrix|finite-volume|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11285" PostTypeId="1" CreationDate="2014-04-06T04:10:27.037" Score="3" ViewCount="546" Body="&lt;p&gt;My question is about implementation alone.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Consider a square domain with regular square, cell centred finite&#xA;volumes.  This is for the multiscale finite volume method (Jenny and Lunati)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I need to  solve the Poisson equation on each cell of the &quot;dual mesh&quot;, i.e.,&#xA;the mesh constructed by joining centroids of each cell. (the dual mesh will look like a translation of the original mesh; 4 cells of the dual mesh make up one cell of primary mesh)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I will need to use this to construct the &quot;transmissibilities&quot; for each cell of the primary mesh.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My question is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What kind of date structure should I use that will help me go from primary to dual mesh easily&lt;/li&gt;&#xA;&lt;li&gt;I am confused what I shld store, the coordinates of the cell centres, or the locations of the faces ? &lt;/li&gt;&#xA;&lt;li&gt;How do I store the data that will tell me which 4 dual cells make up a given primary cell ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It seems to me I will need to do a kind of &quot;assembly&quot; for my transmissibility matrix as I visit each dual cell and solve a local Poisson problem.&lt;/p&gt;&#xA;" OwnerUserId="7728" LastActivityDate="2015-11-14T09:46:16.623" Title="data structures for efficient/easy implementation of finite volume method for 2D Poisson equation" Tags="|finite-volume|poisson|mesh|data-structures|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11286" PostTypeId="1" CreationDate="2014-04-06T14:14:47.277" Score="4" ViewCount="756" Body="&lt;p&gt;I'm looking for a way to optimize this procedure. This is the problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have a list of bit vectors $\mathbf{A} = [ a_1, a_2, a_3, ..., a_n ]$&lt;/li&gt;&#xA;&lt;li&gt;I have a list of bit vectors $\mathbf{B} = [ b_1, b_2, b_3, ..., b_m ]$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For every $b \in \mathbf{B}$, I need to find the following minimum hamming distance:&#xA;$$\min \left\{ \forall x,y\in \mathbf{A}: hamming_d(x \land y, b) \right\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if we define &lt;em&gt;popcount&lt;/em&gt; as a function that counts the number of ones in a bit vector:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min \left\{ \forall x,y\in \mathbf{A}: popcount(x \land y \oplus b) \right\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Python pseudo-code, it should look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;results = list()&#xA;for b in B:&#xA;    m = min(popcount((x &amp;amp; y) ^ b) for x in A for y in A)&#xA;    results.append(m)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A and B are both lists containing vectors of 0 or 1.&#xA;&amp;amp; (and) and ^ (xor) are operations that are applied to every element of the bit vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="7870" LastEditorUserId="7870" LastEditDate="2014-04-06T14:44:55.063" LastActivityDate="2014-04-22T18:03:54.583" Title="Finding the minimum hamming distance between a bit vector and any pairwise intersection of multiple bit vectors" Tags="|algorithms|python|performance|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="11287" PostTypeId="2" ParentId="11286" CreationDate="2014-04-06T18:29:07.137" Score="1" Body="&lt;p&gt;This is not a full answer, but it's possible to get bounds that could possibly be turned into a more efficient solution.  For example if you have&#xA;&lt;code&gt;&#xA;b : 00001111&#xA;x : 01010101&#xA;        ^ ^&#xA;&lt;/code&gt;&#xA;then regardless of &lt;code&gt;y&lt;/code&gt; you will not get a lower popcount than 2 for this &lt;code&gt;b&lt;/code&gt; and &lt;code&gt;x&lt;/code&gt;.  This ability to bound partial solutions makes me think that in practice you can make an algorithm that will be much faster than your brute force algorithm, even if the problem can be shown to be 'hard' in the worst case.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-04-06T18:29:07.137" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11288" PostTypeId="1" CreationDate="2014-04-06T18:33:59.763" Score="1" ViewCount="1544" Body="&lt;p&gt;I would like to compute the inverse of some large block diagonal sparse matrix. The number of rows and columns is somewhat over 50,000. The blocks are 12 by 12 and are sparse (27 non zero elements).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to compute the inverse of the entire matrix (using solve). This was not possible, the entire matrix is too big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, i use a for-loop. Within each iteration, i take out one block, compute its inverse and place it back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That method works, but iT takes about 5 minutes. I wonder if there is some faster way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="7876" LastActivityDate="2015-07-04T06:26:24.280" Title="computing the inverse of a large block diagonal sparse matrix in r" Tags="|sparse-matrix|matrix|r|inverse|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11289" PostTypeId="1" CreationDate="2014-04-06T18:51:47.317" Score="0" ViewCount="70" Body="&lt;p&gt;Figure below depicts a cross section of a creek for which I am trying to measure the water flow for that section. What we have as inputs are a bunch of sample points on the river. For each sample points we measured the depth of creek at that point (e.g. h1, etc) the velocity of water at that point(e.g. v1) and the distance of that sample point from the shore. For each area the water flow is defined as the multiplication of area times the velocity. Since we are given the h1,..hn and d1 ,.. dn we can easily measure each area thereby the water flow of that section.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;areaFlow1 = [(v1+v2)/2][(h1+h2)/2][(d2-d1)]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However what I need to know here is how to aggregate all of those multiple area water flows to define the global water flow for that section?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it the best to say Global Water Flow ( for a cross section) = Sigma(areaFlow(1)..areaFlow(n))/ n ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tY9Q6.png&quot; alt=&quot;Creek flow section&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7877" LastActivityDate="2014-04-07T03:41:53.977" Title="How to model waterflow when only a couple of sample points available" Tags="|optimization|algorithms|fluid-dynamics|computational-geometry|computational-physics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11290" PostTypeId="1" AcceptedAnswerId="11304" CreationDate="2014-04-06T20:53:42.050" Score="8" ViewCount="2614" Body="&lt;p&gt;I'm currently working on &lt;a href=&quot;https://stackoverflow.com/questions/21125316/simulating-physics-for-voxel-constructions-minecraft-dwarf-fortress-etc/21126078&quot;&gt;this project&lt;/a&gt; and I have a basic structural analyzer that uses the finite element method. Essentially, I turn each block into a set of trusses, construct a stiffness matrix relating each truss's force to each other, and feed it into Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next task is for me to speed it up, ideally by using some GPU acceleration. Since Matlab does not yet support spare matrix computations in the GPU, I'm looking into the other libraries available. One approach I'm considering taking is to use &lt;a href=&quot;http://www.culatools.com/sparse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CULA Sparse&lt;/a&gt;'s solvers to speed things up. But there are also a multitude of &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_finite_element_software_packages&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite element software packages&lt;/a&gt; that may do a better job, and save me the performance overhead of constructing matrices and retrieving results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm wondering is, &lt;strong&gt;which of these finite element software packages is right for me?&lt;/strong&gt; That's a hard question, so here are a few qualifiers, in order of importance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It's part of a program, so a library is preferred over a complete modelling package.&lt;/li&gt;&#xA;&lt;li&gt;Established and well-documented&lt;/li&gt;&#xA;&lt;li&gt;GPU acceleration&lt;/li&gt;&#xA;&lt;li&gt;Easy-to-use&lt;/li&gt;&#xA;&lt;li&gt;It should quickly compute results for relatively small data sets (solve a 10,000 element system in about a second)&lt;/li&gt;&#xA;&lt;li&gt;The price for a student should be under $300.&lt;/li&gt;&#xA;&lt;li&gt;An interface in both Java and C.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Does anyone have a recommendation? Or would it be better to go forward with my CULA idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Jed Brown requested that I give the computation I'm making in more detail. I am not a civil engineer, but I spoke with one about the possibility of a making a structural analysis system that would look and feel good. I already have a prototype. Here's how it works.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Within the setting of a game like Minecraft, a subroutine targets some contiguous region of blocks R to be tested. The program creates a truss mesh out of the region of blocks, and assigns a weight to each node in the mesh (based on how heavy the blocks would be). There are base nodes bordering the region.&lt;/li&gt;&#xA;&lt;li&gt;From this mesh, the program makes a wide stiffness matrix &lt;em&gt;T&lt;/em&gt; and a weight vector &lt;em&gt;w&lt;/em&gt;. From there, it needs to solve the equation &lt;em&gt;Tf = w&lt;/em&gt;, where &lt;em&gt;f&lt;/em&gt;, the unknown, is a vector indicating the forces exerted by each truss and base node. In other words, a row of &lt;em&gt;T&lt;/em&gt; dictates how the truss and base forces add up to counteract the weights of the nodes.&lt;/li&gt;&#xA;&lt;li&gt;Currently, my program solves this by solving the equation &lt;em&gt;TT'u=b&lt;/em&gt; using a conjugate gradient solver, then getting &lt;em&gt;f = T'u&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Once I have the forces on each of the trusses, I can determine which blocks break under the strain. Rather than using a detailed deformation model, I just have each block if one of its comprising trusses exerts a force beyond a threshold.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Because my structural analysis system is to be used in a game, and not a serious engineering context, speed is much more important than accuracy. A lot of the work in making this will consist of tweaking variables in order to give the game a good feel.&lt;/p&gt;&#xA;" OwnerUserId="7409" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2017-12-19T10:20:37.120" Title="Good Finite Element Library for a small project" Tags="|linear-algebra|finite-element|sparse-matrix|gpu|cuda|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11291" PostTypeId="1" CreationDate="2014-04-06T23:00:45.870" Score="4" ViewCount="447" Body="&lt;p&gt;Is it possible to find interior eigenvalues closer to some lambda using Davidson method? I was searching online but found that most people use Jacobi-Davidson method for that. &lt;/p&gt;&#xA;" OwnerUserId="7880" LastEditorUserId="20688" LastEditDate="2019-11-24T22:01:44.977" LastActivityDate="2019-11-24T22:01:44.977" Title="Finding interior eigenvalues using Davidson algorithm" Tags="|eigenvalues|iterative-method|eigensystem|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11292" PostTypeId="2" ParentId="11274" CreationDate="2014-04-07T00:13:17.703" Score="2" Body="&lt;p&gt;The decay properties of the Green's function depend, among other things, on the coefficients in your equations. For example, lower-dimensional wave guides typically transport information for long distances, and you may have to add up a lot of terms for good accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatives are to write the Green's function as a sum of sines and cosines that already satisfy the periodic boundary conditions, or as a sum of eigenfunctions of the operator.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-07T00:13:17.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11293" PostTypeId="2" ParentId="11276" CreationDate="2014-04-07T00:14:55.020" Score="5" Body="&lt;p&gt;The deal.II library (&lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;noreferrer&quot;&gt;http://www.dealii.org&lt;/a&gt;), while written for much larger purposes, also has a sub-library of tensor classes that likely does a lot of what you want to do. In particular, it uses templates for the dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I am one of the principal authors of this library.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-07T00:14:55.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11294" PostTypeId="1" AcceptedAnswerId="11295" CreationDate="2014-04-07T00:40:36.043" Score="7" ViewCount="9984" Body="&lt;p&gt;The problem is to find a 2nd order finite difference approximation of the partial derivative &lt;em&gt;u&lt;sub&gt;xy&lt;/sub&gt;&lt;/em&gt;, where &lt;em&gt;u&lt;/em&gt; is a function of &lt;em&gt;x&lt;/em&gt; and &lt;em&gt;y&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Page 5 of &lt;a href=&quot;http://www.mathematik.uni-dortmund.de/~kuzmin/cfdintro/lecture4.pdf&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; pdf I found does a centered difference approximation it in two steps. It first does the 2nd order centered finite-difference approximation of one of the partials, and then inserts the approximation of the second partial into it (using the same formula):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wMeJk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inserting lines 2 and 3 into 1 gives (according to the pdf) the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gnZzz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last &lt;em&gt;O&lt;/em&gt;[(Δ &lt;em&gt;x&lt;/em&gt;)&lt;sup&gt;2&lt;/sup&gt;,(Δ &lt;em&gt;y&lt;/em&gt;)&lt;sup&gt;2&lt;/sup&gt;] is what I have a problem with. Notice that when the O(Δ &lt;em&gt;y&lt;/em&gt;)&lt;sup&gt;2&lt;/sup&gt; terms of lines 2 and 3 go into the numerator of 1, they are being divided by the Δ &lt;em&gt;x&lt;/em&gt; in the denominator. So how come the residual terms in line 3 are of O(Δ &lt;em&gt;y&lt;/em&gt;)&lt;sup&gt;2&lt;/sup&gt; instead of O(Δ &lt;em&gt;y&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt;/Δ &lt;em&gt;x&lt;/em&gt;)? Would this be a '2nd order' approximation any more? (If, say, grid-spacing along both axes are the same  (Δ &lt;em&gt;x&lt;/em&gt; = Δ &lt;em&gt;y&lt;/em&gt; = &lt;em&gt;h&lt;/em&gt;), the term is of order &lt;em&gt;h&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt;/&lt;em&gt;h&lt;/em&gt; = &lt;em&gt;h&lt;/em&gt;, not &lt;em&gt;h&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion would be to use a higher order approximation (3rd or more) in lines 2 and 3 in order to survive the division by Δ &lt;em&gt;x&lt;/em&gt; and still have the final expression in 2nd order. But I may be missing something here.&lt;/p&gt;&#xA;" OwnerUserId="7881" LastEditorUserId="7881" LastEditDate="2014-04-07T05:44:29.507" LastActivityDate="2014-04-09T12:51:10.397" Title="2nd order centered finite-difference approximation of $u_{xy}$" Tags="|finite-difference|discretization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11295" PostTypeId="2" ParentId="11294" CreationDate="2014-04-07T01:02:20.483" Score="5" Body="&lt;p&gt;Section 2.5 of this &lt;a href=&quot;http://www.springer.com/cda/content/document/cda_downloaddocument/9780387236193-c2.pdf&quot; rel=&quot;nofollow&quot;&gt;PDF document&lt;/a&gt; goes through some additional details and the error does in fact work out to be second order. The key is that the term multiplying the $\Delta y^2/\Delta x$ term is equal to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta x\left( \dfrac{\partial^4u}{\partial x\partial y^3} +\mathcal{O}(\Delta x^2) \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $1/\Delta x$ cancels the $\Delta x$ in this term and the error is formally second order in $\Delta x$ and $\Delta y$ independently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find other resources by searching for &quot;cross derivatives&quot; and &quot;finite difference&quot;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-04-07T01:14:12.123" LastActivityDate="2014-04-07T01:14:12.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11296" PostTypeId="2" ParentId="11289" CreationDate="2014-04-07T03:41:53.977" Score="1" Body="&lt;p&gt;I'm not a river-ologist but you may want to account for the depth at which you've measured the velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://galileo.phys.virginia.edu/classes/152.mf1i.spring02/RiverViscosity.htm&quot; rel=&quot;nofollow&quot;&gt;http://galileo.phys.virginia.edu/classes/152.mf1i.spring02/RiverViscosity.htm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5046" LastActivityDate="2014-04-07T03:41:53.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11297" PostTypeId="2" ParentId="11276" CreationDate="2014-04-07T04:41:54.307" Score="1" Body="&lt;p&gt;LTensor (&lt;a href=&quot;https://code.google.com/p/ltensor/&quot; rel=&quot;nofollow&quot;&gt;https://code.google.com/p/ltensor/&lt;/a&gt;) is a VERY easy to use C++ template library for tensors up to rank 4 (based on indical notation), fast and lightweight too. You don't need to compile anything only need to include the main header file. I have used it on several projects and worked ok. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has some built-in features for rank-2 tensors like linear solvers, svd, LU and Cholesky decompositions, etc. I didn't use any of them (I use other libraries for that).&lt;/p&gt;&#xA;" OwnerUserId="7882" LastActivityDate="2014-04-07T04:41:54.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11298" PostTypeId="2" ParentId="11274" CreationDate="2014-04-07T13:19:24.777" Score="1" Body="&lt;p&gt;Though I have never implemented this myself, the conventional wisdom I've heard is that the spatial summation converges very slowly and it's preferable to use the Ewald transform, e.g. &lt;a href=&quot;http://w3.uniroma1.it/lovat/giampiero/Documentipdf/IJ24.pdf&quot; rel=&quot;nofollow&quot;&gt;http://w3.uniroma1.it/lovat/giampiero/Documentipdf/IJ24.pdf&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's probably boatloads of papers on this subject in IEEE transactions.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2014-04-07T13:19:24.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11299" PostTypeId="2" ParentId="11291" CreationDate="2014-04-07T14:35:26.550" Score="3" Body="&lt;p&gt;It should be possible, but it might not be the most efficient.  Here's how I would do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Davidson method builds a subspace to represent a desired region of the eigenspectrum.  The important parts of Davidson are the methods to add and remove vectors from that subspace.  When comparing to 'normal' Davidson, I'll assume it is looking for the lowest eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, Davidson adds the residuals from the lowest eigenvalue approximate solutions:&#xA;$$ K (A - \tilde \lambda_i I) \tilde v_i \quad : \quad i &amp;lt; n$$&#xA;where $K$ is the preconditioner (approximate inverse), $\tilde \lambda_i$ is an approximate eigenvalue, $\tilde v_i$ is the corresponding approximate eigenvector, $i$ sorts in ascending order, and $n$ is the number of desired eigenvectors.  You would want to change the sorting to be by distance from your desired eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Davidson adds vectors to the subspace at every iteration.  If the starting guesses are bad and many iterations are required, the subspace gets polluted with the high eigenspaces.  Normally, the approximate eigenvectors with large eigenvalues are trimmed from the subspace periodically.  You would want to trim the eigenvectors that are the farthest away from your desired eigenvalue.  The easiest way is to restart the procedure using  your best approximations of the desired eigenpairs as the starting guess.&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2014-04-07T14:35:26.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11300" PostTypeId="1" AcceptedAnswerId="11301" CreationDate="2014-04-07T17:50:50.920" Score="1" ViewCount="1288" Body="&lt;p&gt;I've got some problems solving (numerically) this system of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array}{l}&#xA;40 \cdot \cos (2t) + 105 \cdot \cos ({\theta _3}) - 75 \cdot \cos ({\theta _4}) - 91.924 \cdot \cos ({337.62}) = 0\\&#xA;40 \cdot \sin(2t) + 105 \cdot \sin({\theta _3}) - 75 \cdot \sin({\theta _4}) - 91.924 \cdot \sin({337.62}) = 0&#xA;\end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now t is an array of numbers (variable?) ranging from 0 to 0.785 like this t = 0:0.01:0.785.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if it is possible to find $\theta_3$ and $\theta_4$ for every t (like t=0 --&gt; $\theta_3$=something, $\theta_4$=something...t=0.4--&gt;$\theta_3$=&quot;something else&quot;, $\theta_4$=&quot;something else&quot;) put those values(solutions) in an array (vector)so that I can plot them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to do it symbolically with solve but MATLAB couldn't find any (useful) solution. I tried to solve it numerically but I couldn't make it spit out more than two solutions at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done it in Mathcad but I need to solve this with MATLAB (I'm new to MATLAB).&lt;/p&gt;&#xA;" OwnerUserId="7889" LastEditorUserId="20545" LastEditDate="2016-12-01T14:50:43.677" LastActivityDate="2016-12-01T14:50:43.677" Title="System of nonlinear equations in MATLAB" Tags="|matlab|nonlinear-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11301" PostTypeId="2" ParentId="11300" CreationDate="2014-04-07T18:15:59.543" Score="2" Body="&lt;p&gt;I would loop over t and solve the system using the fsolve command. For $t=0$, it shouldn't be too difficult with a reasonable initial guess. And then solve each problem using as initial value the solution of the previous value for $t$...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define your function as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [res] = problem(x,t)&#xA;&#xA;t1 = 337.62/180.0*pi;&#xA;t3 = x(1);&#xA;t4 = x(2);&#xA;&#xA;res = zeros(2,1);&#xA;res(1) = 40.0*cos(2*t)+105*cos(t3)-75*cos(t4)-91.924*cos(t1);&#xA;res(2) = 40.0*sin(2*t)+105*sin(t3)-75*sin(t4)-91.924*sin(t1);&#xA;&#xA;return;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Define the values of the parameter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t = [0:0.01:0.785] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Make room for the solutions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sols = zeros(2,length(t))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Solve for $t=0$ with an initial guess of $\theta_3 = \theta_4 = 1$ (arbitrary)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;theta0 = [1,1];&#xA;sols(:,1) = fsolve(@(x) problem(x,t(1)),theta0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then loop over the remaining values for $t$ using the previous solution as initial guess:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=2:length(t),&#xA;  sols(:,i) = fsolve(@(x) problem(x,t(i)),sols(:,i-1))&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and plot&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot(t,sols(1,:),t,sols(2,:))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Caveat: my Matlab is a bit rusty (my switch to Python/Numpy/Scipy is already a couple of years ago...)&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2014-04-07T18:53:37.903" LastActivityDate="2014-04-07T18:53:37.903" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11302" PostTypeId="1" CreationDate="2014-04-07T22:06:29.810" Score="2" ViewCount="748" Body="&lt;p&gt;As we know, computing the authority (or hub) score of HITS ranking method, means to use the following matrix equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \textbf{a}^{k}=A^T A\textbf{a}^{(k-1)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and apply the power iteration method, but with significant problems: actually, AFAIK, the convergence in not guaranteed, for the matrix $A^T A$ is sometimes not irreducible (so we cannot use Perron-Frobenius theorem and the power method may converge to nonunique solutions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'm wondering: why not apply the SVD to $A$, in order to find the dominant eigenvector(s) of $A^T A$? Do you notice any possible issue with this approach? Will the computational cost be prohibitive?&lt;/p&gt;&#xA;" OwnerUserId="7893" LastEditorUserId="1804" LastEditDate="2016-05-27T16:35:31.027" LastActivityDate="2016-05-27T16:35:31.027" Title="SVD and HITS Algorithm Power Iterations" Tags="|matrix|eigensystem|svd|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11303" PostTypeId="2" ParentId="114" CreationDate="2014-04-07T22:13:45.710" Score="2" Body="&lt;p&gt;Yes, obsolete. Algorithmic analysis by flops, or any other method, is only as useful as the abstract model of the machine when considering the size of the problem at hand. Actual performance depends both on the implementation and the hardware, and the applicability of any abstract model for the latter to reality is decreasing over time. For example, as you further parallelize the  implementation of a complex algorithm, like molecular dynamics, different aspects become rate limiting on different hardware, and algorithmic analysis has nothing to do with the observations. In one sense, the only important thing is to measure the performance of the implementation(s) of the algorithm(s) on the hardware type(s) in question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are such abstractions useful as a learning tool? Yes, like lots of models used for teaching, they are useful as long as they are placed alongside understanding of the limitations of the model. Classical mechanics is fine so long as you appreciate that it won't work on scales of small distance or large velocity...&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2014-04-07T22:13:45.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11304" PostTypeId="2" ParentId="11290" CreationDate="2014-04-07T23:26:27.007" Score="6" Body="&lt;p&gt;There are plenty of finite element libraries out there that satisfy most of your criteria. In no particular order, I would mention deal.II (my own project), libMesh, and FEniCS. All three are large, are libraries, are well documented, are well established with large user bases. All three are actively maintained and are about as fast as you would a general purpose library to be. All will solve elasticity equations. All are free of charge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The criteria you probably won't be able to meet using any of the established projects (the three above as well as the second tier ones) are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In scientific computing, we have been using C++ for quite a long time. All three of the libraries above are written in C++ (though FEniCS has a Python interface) and this is generally true for many of the other libraries as well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;They do not directly use GPUs. Some of them have interfaces to solver libraries that can use GPUs to accelerate computations, but even in those cases, the evidence that doing this on GPUs is faster is not really conclusive. The evidence that it is incredibly more difficult and non-portable is very conclusive, though.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-07T23:26:27.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11305" PostTypeId="2" ParentId="11302" CreationDate="2014-04-07T23:32:25.093" Score="1" Body="&lt;p&gt;The method is typically applied to matrices that are so incredibly large that you cannot do things like an SVD. Note that in the SVD you would also compute the matrix whose vectors correspond to the singular values, and this matrix is full. This is not feasible unless $A$ is small, nor can it be done in a stable way.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-07T23:32:25.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11306" PostTypeId="1" AcceptedAnswerId="11310" CreationDate="2014-04-08T05:35:56.343" Score="21" ViewCount="76345" Body="&lt;p&gt;I would like to determine the theoretical number of FLOPs (Floating Point Operations) that my computer can do. Can someone please help me with this. (I would like to compare my computer to some supercomputers just to get an idea of the difference between them)&lt;/p&gt;&#xA;" OwnerUserId="7895" LastEditorUserId="7895" LastEditDate="2014-04-10T02:10:04.523" LastActivityDate="2020-12-14T06:11:53.170" Title="How to determine the amount of FLOPs my computer is capable of" Tags="|floating-point|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11308" PostTypeId="2" ParentId="8718" CreationDate="2014-04-08T10:30:57.120" Score="0" Body="&lt;p&gt;Let's see, if you like this example. You may take a random black and white image (i took a random text) and noisify it by adding Gaussian noise. This leads to the following image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/6tjWT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can now hardly detect any structure in the picture. I now applied a 2D-HMM decoding to try to reconstruct the original image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/dQtDe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this, what you wanted to see?&lt;/p&gt;&#xA;" OwnerUserId="7769" LastActivityDate="2014-04-08T10:30:57.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11309" PostTypeId="2" ParentId="11306" CreationDate="2014-04-08T11:59:09.083" Score="4" Body="&lt;p&gt;You will need to know the model and vendor of the CPUs in your machine. Once you have that, you can look up on the vendor's website (or maybe on Wikipedia) the clock rate, number of chips/sockets, number of cores per chip, number of floating point operations per cycle, and the vector width of those operations. Then, you simply multiply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take, for example, the Intel Xeon E5-2680 &quot;Sandy Bridge&quot; processors in &lt;a href=&quot;https://www.tacc.utexas.edu/user-services/user-guides/stampede-user-guide&quot; rel=&quot;nofollow&quot;&gt;Stampede&lt;/a&gt; where I work. The specs are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2.7GHz&lt;/li&gt;&#xA;&lt;li&gt;2 chips/node, 8 cores/chip&lt;/li&gt;&#xA;&lt;li&gt;2 vector instructions/cycle&lt;/li&gt;&#xA;&lt;li&gt;256-bit wide AVX instructions (4 simultaneous double-precision operands)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Multiplying those gives 345.6 GF/node or 2.2 PF for the un-accelerated part of the system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We usually think in terms of double-precision (64-bit) operations, because that's the precision required for the vast majority of our users, but you can can redo the calculation in single-precision terms if you like. This usually only changes the last factor, say 8 SP Flops/instruction instead of 4 DP Flops/inst, but it can be wildly different from that. Older GPUs, for example, only did DP at about 1/8th the rate of SP. If you ever quote a number for your system, you should be explicit about which you used if it's not double-precision because people will assume it was, otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if your chip supports fused multiply-add (FMA) instructions, and it can do them at full rate, then most people count this as 2 floating-point operations though a hardware performance counter might count it as only one instruction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, you can also do this for any accelerators that might exist in your system (like a GPU or Xeon Phi) and add that performance to the CPU performance to get a theoretical total.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2014-04-08T13:32:52.050" LastActivityDate="2014-04-08T13:32:52.050" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11310" PostTypeId="2" ParentId="11306" CreationDate="2014-04-08T12:00:40.233" Score="15" Body="&lt;p&gt;The theoretical peak FLOP/s is given by:&#xA;$$ \text{Number of Cores} * \text{Average frequency} * \text{Operations per cycle} $$&#xA;The number of cores is easy.  Average frequency should, in theory, factor in some amount of Turbo Boost (Intel) or Turbo Core (AMD), but the operating frequency is a good lower bound.  The operations per cycle is architecture-dependent and can be hard to find (8 for SandyBridge and IvyBridge, see &lt;a href=&quot;http://media.wix.com/ugd/e53cc7_d9d115bb3548496481e893af130cf943.pdf&quot; rel=&quot;noreferrer&quot;&gt;slide 26&lt;/a&gt;).  It is the subject of this &lt;a href=&quot;https://stackoverflow.com/questions/15655835/flops-per-cycle-for-sandy-bridge-and-haswell-sse2-avx-avx2&quot;&gt;stack overflow question&lt;/a&gt;, which includes numbers for a bunch of modern architectures.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-04-10T05:59:38.583" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11311" PostTypeId="1" AcceptedAnswerId="11319" CreationDate="2014-04-08T12:07:47.807" Score="4" ViewCount="356" Body="&lt;p&gt;I want to solve the diffusion equation, i.e.&#xA;$$&#xA;\dot{f} - f'' = 0 &#xA;$$&#xA;with a boundary condition $f(0) = f(1) = 0$ and with an initial condition that $f$ is a boxcar function concentrated over some small region of size $d \ll 1$. The units are scaled so that the diffusion constant is equal to unity. Then I want to use the resulting solution to compute an integral of the form&#xA;$$&#xA;\int dt \int dx F(f(x), f'(x)) ,&#xA;$$&#xA;where $F$ is some function (it is actually quadratic in $f$ with some $x$-dependent coefficients if this makes any difference).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried two different ways to do this. First is to solve the diffusion equation analytically in Fourier space and then write my integral also in Fourier space and calculate it numerically. The second is to solve the diffusion equation numerically (I've tried both explicit and implicit methods and the well-known Crank-Nicolson method). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all approaches, I end up with the same problem: The initial condition for $f$ is very concentrated which means I need a ridiculously small space and time discretization (corresponds to taking into account very large Fourier modes in F-space solution). On the other hand, since I need to solve the equation also for large times (to calculate the integral), I need to take into account also solutions in large times. Fourier-space method is not faster because in that I actually need to compute three integrals numerically (two Fourier sums with a kernel that is calculated from the $x$-integral). This makes my calculations very slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any better methods to do what I just described? Since diffusion equation is very simple, I feel pretty stupid for not being able to do this faster. I've been thinking about implementing some kind of adaptive grid that would be concentrated to small times and close to the initial condition, but this seems like a complicated thing to do tho solve such a simple problem.&lt;/p&gt;&#xA;" OwnerUserId="7899" LastActivityDate="2014-04-08T20:35:26.300" Title="The most efficient way to solve diffusion equation with concentrated initial condition" Tags="|advection-diffusion|diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11312" PostTypeId="1" CreationDate="2014-04-08T13:33:12.687" Score="5" ViewCount="107" Body="&lt;p&gt;When using the dual mesh (vertex-centered) for finite volume methods, you end up with a cell center at the boundaries between materials.  It is possible that the equations being solved in each material are different, or at the very least don't have continuous derivatives at the boundary (for example: poisson's equation for electric potential moving from gas to metal).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how do I handle these cells.  Do I split the cell in half so each partial cell is entirely contained in one material or do I treat the boundaries of the cell as the material boundaries (i.e. purely gas cells see the cell boundaries as metal and metal cells see the cell boundaries as gas)?  If I split the cell, do I move the cell center for each off of the new boundary?  Any book or reference handling this would be greatly appreciated, since all the references I have found for dual mesh FVM limit themselves to dirichlet conditions, and don't hanlde interfaces at all.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-04-08T13:33:12.687" Title="Interface Formulation at Finite Volume Boundaries when using the Dual Mesh" Tags="|reference-request|boundary-conditions|finite-volume|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11313" PostTypeId="2" ParentId="8718" CreationDate="2014-04-08T13:53:47.060" Score="1" Body="&lt;p&gt;i suggest to look at &lt;a href=&quot;http://en.wikipedia.org/wiki/Digital_watermarking&quot; rel=&quot;nofollow&quot;&gt;digital watermarking&lt;/a&gt;. it's the way to embed information into the data in such a way that it's indistinguishable from noise. you can subsequently extract this information. it's used for copyrighting content, where you may create the copies of content each with a watermark, then track down the leakers.&lt;/p&gt;&#xA;" OwnerUserId="7470" LastActivityDate="2014-04-08T13:53:47.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11314" PostTypeId="2" ParentId="7706" CreationDate="2014-04-08T14:47:20.670" Score="2" Body="&lt;p&gt;Thanks Nick, for the excellent instructions! I had been beating my head against the wall for awhile trying to get .pdbs into Blender (I'm haven't had much luck with some of the .pdb importers, such as ePMV (they feel sluggish and buggy on my mid-2013 MacBook Pro). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I did want to mention of couple of things that I thought might be helpful to anyone who runs across this post (by now, you probably already know this, I certainly didn't when I started!): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) When exporting the Wavefront object, I have found that I need the ribbons (New Cartoon) to have a resolution of at least 25 and licorice should be higher still. Otherwise, when you import into Blender, it won't look smooth. However, don't crank it up crazy high (like 50+), or your object/mtl files will be insanely huge (2-30 Gb--unmanageable!). Still trying to find just the right balance. :) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Once imported, make sure you remove duplicate vertices (select object, click &quot;Tab&quot; to go into edit mode, and under &quot;Tools&quot; (on the left) click &quot;Remove Doubles&quot;. After that, click &quot;Tab&quot; again to go back to Object mode and click &quot;Smooth&quot; under shading. It is important to remove duplicate vertices--if you don't, you get weird and ugly results if you try to use the modifier &quot;Subdivision Surface&quot; (which can make renders look better).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this is helpful! &lt;/p&gt;&#xA;" OwnerUserId="7900" LastActivityDate="2014-04-08T14:47:20.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11315" PostTypeId="1" AcceptedAnswerId="11318" CreationDate="2014-04-08T18:03:17.593" Score="13" ViewCount="5692" Body="&lt;p&gt;What's the difference between these two methods? Can a problem be solved by one method will be able to solved by the other? Can both/or one of them be parallelized with OpenMP and/or MPI?&lt;/p&gt;&#xA;" OwnerUserId="7905" LastActivityDate="2014-04-08T23:14:19.107" Title="What's the difference between conjugate gradient method and biconjugate gradient method" Tags="|mpi|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11316" PostTypeId="2" ParentId="11306" CreationDate="2014-04-08T18:06:03.590" Score="5" Body="&lt;p&gt;I understand that you asked for the theoretical value, but as this is nearly always inaccessible by any real code, even LINPACK, you might want to just run (optimized) DGEMM for very large matrices.  The reason that I prefer this method is that it exposes some of the shortcomings of certain processors that prevent them from achieving their theoretical peak flop value.&lt;/p&gt;&#xA;&lt;p&gt;For example, circa 2014, NVIDIA GPUs did integer and floating-point operations on the same pipeline.  This means that you could only achieve the theoretical peak flop/s if you do &lt;em&gt;no integer computation whatsoever&lt;/em&gt;.  As array indexing and any other form of data access requires integer arithmetic somewhere, no code would achieve the theoretical peak flop/s on an NVIDIA GPU.  In most cases, one saw ~80% as the upper bound.  For CPUs that issue integer and floating-point operations simultaneously, this is a non-issue.&lt;/p&gt;&#xA;&lt;p&gt;On some GPU-like multicore processors like Intel Knights Corner and Blue Gene/Q, it is harder to achieve the peak flop/s than on traditional CPUs for similar pipeline issues (although both can achieve ~90% of peak in large DGEMM at least).&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2020-12-14T06:11:53.170" LastActivityDate="2020-12-14T06:11:53.170" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11317" PostTypeId="2" ParentId="11315" CreationDate="2014-04-08T18:28:51.493" Score="6" Body="&lt;p&gt;The conjugate gradient method only works to solve the system &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A x = b $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if $A$ is symmetric and positive-definite (also works for negative definite).  The reason it must be symmetric is that conjugate gradient works by minimizing (or maximizing) the function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f(x) = \frac{1}{2} x^T A x - b^T x $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the derivative is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f'(x) = \frac{1}{2} A^T x + \frac{1}{2} A x - b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and if $A$ is symmetric $A^T = A$ so the above reduces to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f'(x) = A x - b $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the minimum, $f'(x) = 0$ and $x$ is the solution to your system.  This last step should make it obvious why $A$ must be symmetric.  The positive/negative definite property is more subtle, but it is required so that the extrema exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The biconjugate gradient method will work for any system.  It does so by solving both &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A x = b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;along with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A^T x = b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;simultaneously.  The details of which I am not familiar with, so I won't pretend to know.  It is sufficient to know that biconjugate gradient is the more general of the two.  It does have stability issues, and if $A$ is symmetric, then conjugate gradient will perform less work to get to the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far a parallelism, if you systems are large, you can gain a lot of parallelism in the linear algebra involved in the solver iterations, so there should be no reason a parallel linear algebra library wouldn't lead to performance gains.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-04-08T18:28:51.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11318" PostTypeId="2" ParentId="11315" CreationDate="2014-04-08T18:30:44.637" Score="14" Body="&lt;p&gt;The conjugate gradient method is the provably fastest iterative solver, but only for symmetric, positive-definite systems. What would be awfully convenient is if there was an iterative method with similar properties for indefinite or non-symmetric matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CG method seeks approximate solutions at each step $k$ within the Krylov subspace&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K_k(A,b) = \{b, Ab, A^2b,\ldots,A^kb\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The essential idea of the biconjugate gradient method is to maintain a second Krylov subspace&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K_k^*(A,b) = \{b, A^*b, (A^*)^2b,\ldots,(A^*)^kb\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and seeking a recurrence with similar orthogonality properties to that of CG, but without the stability issues of solving $A^*Ax = A^*b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, that fails if you apply it naively. However, by performing one step of the generalized minimum residual (GMRES) algorithm after each BiCG step, the resulting iteration is stable; this is usually referred to as BiCG-Stab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, BiCG-Stab is (in principle) a more general solver than CG but suffers worse efficiency when applied to the problems for which CG was intended. BiCG or BiCG-stab require more matrix-vector multiplications and more dot products, so if you parallelize them via distributed-memory multiprocessing you'll incur more communication overhead, but nonetheless they can be scaled up as much as you like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two things worth noting here which are more important than all that other junk I just said:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;For every iterative method (BiCG, GMRES, QMR...), there is a matrix that will make it fail to converge in finite-precision arithmetic.&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Therefore, &lt;strong&gt;coming up with a good preconditioner for your specific matrix is probably more important than using the optimal outer-level iterative solver.&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;EDIT: For open-source libraries, the two most popular are &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot;&gt;PETSc&lt;/a&gt; and &lt;a href=&quot;http://trilinos.org/&quot;&gt;Trilinos&lt;/a&gt;. I highly recommend you also get the Python bindings, respectively petsc4py and PyTrilinos. You can also try &lt;a href=&quot;http://eigen.tuxfamily.org/&quot;&gt;Eigen&lt;/a&gt;. On the one hand, it doesn't have many features, but on the other hand, it has just what you need and no more; if you intend to read the code rather than just use it, Eigen might be the easiest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also: Yousef Saad, &lt;a href=&quot;http://books.google.com/books?id=qtzmkzzqFmcC&amp;amp;lpg=PR2&amp;amp;dq=saad%20iterative%20methods&amp;amp;pg=PR2#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;Iterative Methods for Sparse Linear Systems&lt;/a&gt;; Nachtigal et al, &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/publication/PDF/1992_52.pdf&quot;&gt;How Fast Are Nonsymmetrix Matrix Iterations?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2014-04-08T23:14:19.107" LastActivityDate="2014-04-08T23:14:19.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11319" PostTypeId="2" ParentId="11311" CreationDate="2014-04-08T20:35:26.300" Score="2" Body="&lt;p&gt;You should never use explicit method for the diffusion equation. Implicit is unconditionally stable and just as easy to implement. Also if you use an implicit method (like backward Euler or Crank-Nicolson) it will not matter how small d is. In fact you could use a dirac delta function if you wanted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as speed goes, doing a implicit method with a tridiagonal solver (O(n) complexity) this should be very fast especially for 1D. I can send an easy Matlab script if necessary.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2014-04-08T20:35:26.300" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11320" PostTypeId="1" CreationDate="2014-04-08T21:14:24.027" Score="4" ViewCount="199" Body="&lt;p&gt;In a high-dimensional setting, say $d \gg 5$, what is a recommended way of evaluating a spherical integral of a smooth (non-symmetric) function $f(\mathbf{x})$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \int_\mathcal{S_r} f(\mathbf{x}) \mathrm{d}(\mathbf{x})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal{S_r} = \{ \mathbf{x} \; | \; \|\mathbf{x}\|^2 = r \}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to avoid Monte Carlo integration due to high requirement on the number of samples. There seem to be various quadrature methods, but I don't know which one to use. Is there a recent survey of these integrals perhaps?&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2014-06-08T14:32:59.477" Title="Numerical spherical integration" Tags="|numerical-analysis|quadrature|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11321" PostTypeId="1" CreationDate="2014-04-08T21:50:10.107" Score="1" ViewCount="256" Body="&lt;p&gt;I am looking for a routine (or set of routines) in the Intel MKL that that can replace &lt;code&gt;dslucs&lt;/code&gt; (Incomplete LU BiConjugate Gradient Squared Ax=b Solver) in &lt;code&gt;SLATEC&lt;/code&gt; (&lt;a href=&quot;http://www.netlib.org/slatec/lin/&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/slatec/lin/&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="7905" LastActivityDate="2014-12-31T19:18:03.007" Title="SLATEC rouitne dslucs() and MKL correspondence" Tags="|conjugate-gradient|intel-mkl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11322" PostTypeId="1" AcceptedAnswerId="11389" CreationDate="2014-04-09T02:49:31.057" Score="6" ViewCount="3112" Body="&lt;p&gt;I have a function $f(r,\theta,\phi)$ which I am expressing in terms of spherical harmonics&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(r,\theta,\phi) = \sum_{l=0}^{\infty} \sum_{m=-l}^{l} g_{l,m}(r) d_{l,m}(\theta,\phi)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $d_{l,m}$ are real spherical harmonics (RSH - formula below taken from Wikipedia).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;d_{\ell m} &amp;amp;=&#xA;\begin{cases}&#xA;\displaystyle {i \over \sqrt{2}} \left(Y_\ell^{m} - (-1)^m\, Y_\ell^{-m}\right) &amp;amp; \text{if}\ m&amp;lt;0\\&#xA;\displaystyle  Y_\ell^0 &amp;amp; \text{if}\ m=0\\&#xA;\displaystyle  {1 \over \sqrt{2}} \left(Y_\ell^{-m} + (-1)^m\, Y_\ell^{m}\right) &amp;amp; \text{if}\ m&amp;gt;0.&#xA;\end{cases}\\&#xA;&amp;amp;=&#xA;\begin{cases}&#xA;\displaystyle {i \over \sqrt{2}} \left(Y_\ell^{-|m|} - (-1)^{m}\, Y_\ell^{|m|}\right) &amp;amp; \text{if}\ m&amp;lt;0\\&#xA;\displaystyle  Y_\ell^0 &amp;amp; \text{if}\ m=0\\&#xA;\displaystyle  {1 \over \sqrt{2}} \left(Y_\ell^{-|m|} + (-1)^{m}\, Y_\ell^{|m|}\right) &amp;amp; \text{if}\ m&amp;gt;0.&#xA;\end{cases}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I would like to take the Fourier Transform (FT) of $f(r,\theta,\phi)$, is there a special relationship between $d_{l,m}$ and the FT? I feel like there is some nice property like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;FT(f(r,\theta,\phi)) = \sum \sum FT(g_{l,m}(r)) d_{l,m}(\theta, \phi)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I can't remember it or find it. Is there a nice relationship between the FT and RSH which will make the FT of $f(r,\theta,\phi)$ easy to compute when expanded in RSH?&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="164" LastEditDate="2014-04-17T16:44:17.300" LastActivityDate="2020-05-17T13:55:28.500" Title="Fourier Transform of function in Spherical Harmonics" Tags="|fourier-transform|spherical-harmonics|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11323" PostTypeId="1" CreationDate="2014-04-09T08:48:02.127" Score="4" ViewCount="3435" Body="&lt;p&gt;I am trying to implement BFGS. The purpose is to approximate Hessian matrix only (not using the quasi-newton optimization steps), so i am using steepest ascent for optimization. What I observe is that the final Hessian approximate is very sensitive to the initial guess of Hessian. If I start with Identity matrix, most of the singular values of final hessian is closed to 1. Similarly if I start with any multiple of identity (let's say 5*I), the singular values of final Hessian is closed to 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a maximization problem, and also the solution is not unique so i expect a negative semi-definite Hessian matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope my question is clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="7908" LastActivityDate="2014-05-09T11:22:41.530" Title="Effect of Initial guess B (approximate Hessian) on BFGS algorithm" Tags="|optimization|approximation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11324" PostTypeId="2" ParentId="11323" CreationDate="2014-04-09T11:17:04.917" Score="4" Body="&lt;p&gt;There are two standard choices for the initial approximation of $B$ in BFGS: Either you choose $B=\frac{\|g_0\|}{\delta} I$ where $\|g_0\|$ is the gradient in the very first iteration and $\delta$ a &quot;typical step size&quot; from $x_k$ to $x_{k+1}$. Or you choose $B=\frac{y_1^T y_1}{y_1^T s_1}I$ using the standard notation used for BFGS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The (excellent) book by Nocedal and Wright has more details.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-09T11:17:04.917" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="11325" PostTypeId="2" ParentId="11294" CreationDate="2014-04-09T12:51:10.397" Score="-1" Body="&lt;p&gt;The discretization presented is of second order accuracy. To look at it intuitively, both the first derivatives in x and y are taken using central difference, which puts their order of accuracy at 2. So, a combination of these two 2nd order central difference terms should atleast be 2nd order accurate.&lt;/p&gt;&#xA;" OwnerUserId="4596" LastActivityDate="2014-04-09T12:51:10.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11326" PostTypeId="2" ParentId="11257" CreationDate="2014-04-09T13:24:12.083" Score="2" Body="&lt;p&gt;I will try to provide an intuitive understanding - The jacobi updates typically overshoot the original solution and hence we weight the updates to converge better. The Gauss Seidel updates typically do the opposite, i.e. underestimate the updates and hence we apply an over-relaxation to compensate for it. You can search for SOR method and you will find what you need.&lt;/p&gt;&#xA;" OwnerUserId="4596" LastActivityDate="2014-04-09T13:24:12.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11327" PostTypeId="2" ParentId="11320" CreationDate="2014-04-09T13:55:11.923" Score="1" Body="&lt;p&gt;I'd use one of the &lt;a href=&quot;http://mathworld.wolfram.com/HyperspherePointPicking.html&quot; rel=&quot;nofollow&quot;&gt;hypersphere random point pickers&lt;/a&gt; and integrate with Monte Carlo.&lt;/p&gt;&#xA;" OwnerUserId="7470" LastActivityDate="2014-04-09T13:55:11.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11328" PostTypeId="1" CreationDate="2014-04-09T21:36:52.403" Score="4" ViewCount="184" Body="&lt;p&gt;I have multiple plane constraints in $\mathbb{R}^3$ of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$n_i \cdot x \ge \delta_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $n_i$ is the $i$th plane normal (in form (x, y, z)), $x$ is a point in space, and $\delta_i$ is the plane constant (distance of plane from origin).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find $x$ such that the above is satisfied and the distance from $x$ to some other point $p$ is minimized.  I believe this is a convex quadratic programming problem (the union of half spaces is always convex, IIRC).  Normally I'd try to find an off-the-shelf library that could solve generalized quadratic programming problems, but I need a solution I can hand over to a client, and most of the solvers I know are either license restrictive (GPL or proprietary), gigantic libraries or both (CGAL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just need to solve this very specific problem form, and I'm willing to be relatively obtuse about how I do it.  Does anyone know of any resources or way to approach this?  I'm willing to take an off-the-shelf solver if it's approximately a single file and MIT-license equivalent, and I'm willing to write my own based on an algorithm description.&lt;/p&gt;&#xA;" OwnerUserId="1365" LastEditorUserId="27" LastEditDate="2014-04-09T21:51:29.310" LastActivityDate="2014-04-17T21:04:41.060" Title="Plane constraints in R3" Tags="|optimization|computational-geometry|convex-optimization|quadratic-programming|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11329" PostTypeId="2" ParentId="11288" CreationDate="2014-04-10T00:04:26.450" Score="1" Body="&lt;p&gt;I made by code much faster, by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) storing the inverse of the block diagonal matrices in a list, rather than 'placing them back in the large matrix. At the end I built the entire matrix from the list by using the bdiag() command. 2) by considering the block diagonal matrices not one by one, but in groups of about 10 So, I repeatedly calculate the inverse of a submatrix consisting of 10 block diagonal matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the original question i did not tell that i do not only take the inverse of each block, but that i also applied some transformation on each block. Now the transformation is done in advance for the entire block diagonal matrix, which also saves time.&lt;/p&gt;&#xA;" OwnerUserId="7876" LastActivityDate="2014-04-10T00:04:26.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11330" PostTypeId="2" ParentId="11328" CreationDate="2014-04-10T00:09:15.247" Score="1" Body="&lt;p&gt;If I'm reading your description correctly, you want to solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align} &#xA;&amp;amp; \min_{x} \|x - p\|^{2}_{2} \\&#xA;\textrm{s.t.} &amp;amp; n_{i} \cdot x \geq \delta_{i}, i = 1,\ldots, N,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;assuming you have $N$ hyperplanes. As you point out, this program is a convex quadratic program (QP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know Python, you could try using the BSD-licensed constrained QP (or nonlinear programming) solvers in SciPy and OpenOpt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this problem is a convex QP, it can also be reformulated as a semidefinite program (SDP). DSDP has a BSD-like license, so you could try using that solver as well (but read the license to make sure; IANAL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regrettably, optimization software lends itself to making money, so any good software is either commercial, or a &quot;free&quot; license comes with many restrictions (GPL, academic use, non-commercial use, personal use, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would not be surprised if there were a fast algorithm that exploited your particular problem structure.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-04-10T01:51:33.583" LastActivityDate="2014-04-10T01:51:33.583" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11331" PostTypeId="1" CreationDate="2014-04-10T03:11:42.220" Score="3" ViewCount="595" Body="&lt;p&gt;Suppose:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have two domains, $\Omega_{1} = [0, 1/2] \times [0, 1]$ and $\Omega_{2} = [1/2, 1] \times [0, 1]$.&lt;/li&gt;&#xA;&lt;li&gt;The domains share an interface $\Gamma = \{1/2\} \times [0, 1] = \partial\Omega_{1} \cap \partial\Omega_{2}$.&lt;/li&gt;&#xA;&lt;li&gt;I mesh both domains in such a way that nodes and faces on the interface match up between the two meshes. &lt;/li&gt;&#xA;&lt;li&gt;For convenience, $\Omega_{0} = \Omega_{1} \cup \Omega_{2} = [0, 1] \times [0, 1]$.&lt;/li&gt;&#xA;&lt;li&gt;The boundary of $\Omega_{0}$ is divided into two disjoint sets, $\partial{\Omega}_{0, D}$ (for Dirichlet conditions) and $\partial{\Omega}_{0, N}$ (for Neumann conditions) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'd like to find $u: \Omega_{0} \rightarrow \mathbb{R}$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;-\nabla \cdot (\kappa_{i}(x)\nabla{u_{i}}) = f_{i}, i = 1, 2, \\&#xA;[\kappa(x)\nabla{u} \cdot n] = 0, \textrm{on $\Gamma$},\\&#xA;[u] = 0, \textrm{on $\Gamma$},\\&#xA;u = u_{D}, \textrm{on $\partial{\Omega}_{0, D}$}, \\&#xA;\kappa(x)\nabla{u} \cdot n = g, \textrm{on $\partial{\Omega_{0, N}}$}, &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x \in \Omega_{0}$, $u_{i} = u|_{\Omega_{i}}$ is the restriction of $u$ to $\Omega_{i}$, $[\cdot]$ is the jump operator, and $u_{D}$ and $g$ are constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to solve this Poisson problem with interfacial conditions using Nitsche's method, as shown in &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782502005248&quot; rel=&quot;nofollow&quot;&gt;A. Hansbo, P. Hansbo, An unfitted finite element method, based on Nitsche's method, for elliptic interface problems, Computer Methods in Applied Mechanics and Engineering, Volume 191, Issues 47-48, , p. 5537-5552, 2002&lt;/a&gt; (paywalled link, unfortunately).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within each subdomain, I plan on using a continuous Galerkin method with Lagrange elements of order 1 to start. If it is not possible to glue together two matching meshes to implement this method, an equivalent formulation uses XFEM and enriches the continuous spaces on $\Omega_{1}$ and $\Omega_{2}$ by using discontinuous Galerkin on the interface $\Gamma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a library (or collection thereof) that is particularly amenable to setting up a problem like the one above? Eventually, I plan on changing the jump conditions to become more complicated (e.g., nonzero jumps; one of them will satisfy a nonlinear relationship in $[u]$ because it is a surface reaction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried using FEniCS (see my similar question on the &lt;a href=&quot;http://fenicsproject.org/qa/3214/how-would-i-implement-nitsches-method-with-matching-meshes&quot; rel=&quot;nofollow&quot;&gt;FEniCS Q&amp;amp;A site&lt;/a&gt;), and I have gotten nowhere. DOLFIN-OLM seems to be a proof-of-concept implementation, but will not have the capability to change the jump conditions easily (or at least, not without me forking the code, and it does not look to be maintained). I don't care about the shape of the elements (triangles vs. quads does not matter). I do care about having access to the assembled stiffness matrix (or its action) to be able to perform sensitivity analyses. Parallelism and good solver backends (e.g., PETSc, Trilinos) are highly desirable, as is the ability to rapidly develop a code that solves the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there's an inelegant way to kludge this sort of thing into an existing framework (e.g., use discontinuous Galerkin everywhere and enforce the jumps at $\Gamma$), that would also be great, because I care more about getting something working than its efficiency. (I can work on the efficiency later.) If someone answering the question chooses that route, you should explain the weak form I should be using instead of the one in the Hansbo-Hansbo paper above.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-04-11T12:24:29.863" Title="Recommendations on FEM software for implementing Nitsche's method on interfaces between matching meshes?" Tags="|optimization|finite-element|fenics|poisson|discontinuous-galerkin|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11332" PostTypeId="2" ParentId="11282" CreationDate="2014-04-10T06:40:05.243" Score="1" Body="&lt;p&gt;A thing you might try:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) perform a few iterations of the matrix sign iteration $A\mapsto \frac{1}{2}(A+A^{-1})$; the eigenvectors are unchanged, while the eigenvalues converge quadratically to $\pm 1$. When $A-A^{-1}$ is small enough, stop the iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) compute the kernel of $A-I$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information on the matrix sign iteration on the book N. Higham, &lt;em&gt;Functions of Matrices&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method trades off eigensolves for QR factorizations; it might be faster than QR, depending on the size of the matrices, of the perturbation, and other machine-related factors difficult to quantify.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-04-10T06:40:05.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11334" PostTypeId="2" ParentId="11331" CreationDate="2014-04-10T10:09:28.250" Score="3" Body="&lt;p&gt;My suggestion is to just write your own code using PETSc instead of using an existing FE library. Parallel assembly/solve is the most complicated part of an FE code and PETSc takes care of it. The rest of it is simple anyway, maybe a few hundred lines of straightforward C/Fortran. Besides PETSc now has DMPLEX for managing meshes (though it is still in development).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't want AMR and are not working on academic type problems then you will be much better off with your own code built directly on top of PETSc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like you, I explored the possibility of using an existing FE lib for my own code a few years back but each FE library came with its own set of quirks/limitations/issues and doing complicated things sometimes is either impossible or much more difficult (and time consuming) than doing it yourself.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="491" LastEditDate="2014-04-11T12:24:29.863" LastActivityDate="2014-04-11T12:24:29.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11335" PostTypeId="1" AcceptedAnswerId="11349" CreationDate="2014-04-10T12:49:32.490" Score="3" ViewCount="1853" Body="&lt;p&gt;recently I have been working on solving some math problems using Fortran. There occurs to me that a linear matrix equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;AX-XA=B&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ and $B$ are known $n\times n$ matrices and $X$ is the one need to be solved. I know this looks like a typical Lyapunov equation. However, in order to solve this equation space friendly, one can use an iterative way to get the numerical result of $X$ when the Kronecker product:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;AI-IA^T&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is non-singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this is not my case. The Kronecker product in my problem is singular. Thus, I cannot use a space friendly iterative way to solve the problem but use a very space consuming method which generates and stores the Kronecker product explicitly!!! Then I used Moore-Penrose pseudoinverse of matrix algorithm to generate the pseudoinverse of this huge matrix and used a library matrix-vector multiplication routine to solve the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For small matrices, this is alright. But when the size of matrices grows (e.g. when n goes up to several hundred) my computer's memory has been completely used up. Any one can help on this so that I can use a space friendly algorithm for such a problem? Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="7920" LastEditorUserId="27" LastEditDate="2014-04-10T13:22:12.063" LastActivityDate="2014-04-11T11:23:03.820" Title="How to solve a Linear Matrix Equation: AX-XA=B efficiently?" Tags="|linear-algebra|algorithms|fortran|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11337" PostTypeId="2" ParentId="11335" CreationDate="2014-04-10T15:16:56.423" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.solve_sylvester.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;solve_sylvester&lt;/code&gt;&lt;/a&gt; function in scipy will solve this.  I don't know whether or not the singularity you mention would affect it.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-04-10T15:16:56.423" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11338" PostTypeId="1" CreationDate="2014-04-10T16:10:17.653" Score="5" ViewCount="398" Body="&lt;p&gt;This is such a basic topic but there are so many different methods proposed for solving a linear system of equations. I recently found a very good &lt;a href=&quot;http://www.stanford.edu/class/cme324/saad.pdf&quot;&gt;source&lt;/a&gt; but couldn't really make sense of all the stuff it contains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I had to pick only 5 methods that I should understand thoroughly for solving big systems of linear equations which would they be? And what would be the most important reason for each choice? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I.e. direct methods?gauss-siedel?conjugate gradient?...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for the help.&lt;/p&gt;&#xA;" OwnerUserId="7923" LastActivityDate="2016-08-13T07:25:36.033" Title="Methods for solving linear systems" Tags="|linear-algebra|matrix|linear-system|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11339" PostTypeId="2" ParentId="11338" CreationDate="2014-04-10T17:00:03.173" Score="5" Body="&lt;p&gt;While it is easy to reach a suboptimal solution for your problem it is usually much harder (and problem-dependent) to come up with an optimal/robust strategy. In this sense your question is quite broad and to give good advice it is crucial to know about the problem you want to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following I will assume you speak of sparse systems and you have the assembled matrix to start with. For large dense systems, I don't really know if you have any choice if you cannot make very strong assumptions on the nature of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sparse systems it pretty much depends on the size, sparsity and structure of your system. For banded systems or moderate sizes up to a few ten thousand unknowns a sparse-direct solver will often be hard to beat. For much larger systems Krylov-subspace solvers are a standard choice. Here you may categorize into symmetric positive definite (-&gt; CG) or just symmetric (MINRES etc.) or not symmetric at all. If you cannot assume symmetry, then GMRES is a widely-used solver. However, since its memory requirements grow with each iteration, you have to restart it after a couple of iterations (this is usually a parameter which you can tune). While this theoretically brings you back to square one, in practice it works quite well most of the time. In terms of memory-requirements another popular choice is BiCGStab, which is often reported to cause difficulties if not preconditioned well. Speaking of nonsymmetric systems it is not easy to say which method is best. In fact there are examples in the literature where certain methods clearly outperform others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many more methods than SparseLU, CG, MINRES, GMRES and BiCGStab, but if you can at least decide whether you can use CG or MINRES on your problem or not, you are probably better off than many others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For iterative methods it is also important to choose a suitable preconditioner for your problem. There are some choices (diagonal, incomplete LU/Cholesky, AMG variants, etc.) which are implemented or interfaced in almost every toolbox, but preconditioning is most effective when it is done in a meaningful (or physics-aware) way. This is clearly nothing which is easy to answer in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottom line is: if you cannot spend much time to read about good solvers/preconditioners for your problem, you have some generic choices to choose from. Many people just try out a couple of solvers built into the package they use and stick with the choice that gives the best results. If you follow this trial and error strategy, you should be prepared to wait unnecessarily long for your solutions to be computed. There are many excellent questions and answers on this site which provide you with further reading in case you decide that learning about optimal strategies for your specific problem will pay off after some time.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2014-04-10T18:07:10.357" LastActivityDate="2014-04-10T18:07:10.357" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11340" PostTypeId="1" CreationDate="2014-04-10T17:30:29.330" Score="2" ViewCount="893" Body="&lt;p&gt;I am trying to solve an equation like $R(x) = 0$, using Newton-Raphson method. To obtain the $x$ increment in each iteration I solve $dx = -(A)^{-1}\cdot R$ where $A = dR/dx$. But the convergence criteria, here $||R||$ reduces to a certain amount, depending on the input, e.g. 1.e-8, and it remains so. After much trying I did not find any special bug, so i am starting to have second thoughts about my implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering now that the approach to calculate the inverse of matrix $A$ might cause inaccuracy so that the convergence fails. What do you think?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;The code is written in fortran; I calculate the inverse using &lt;code&gt;MKL&lt;/code&gt;'s routines: &lt;code&gt;dgetrf&lt;/code&gt; and &lt;code&gt;dgetri&lt;/code&gt;. I am using double-precision variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is to solve $R(\sigma)$ in &lt;code&gt;n&lt;/code&gt; steps using &lt;code&gt;Backward-Euler&lt;/code&gt; method. So functions $Z^p$ and $Z^d$ are also dependent on $\sigma$. In each step $\Delta\epsilon_{n+1}$ is updated and we solve an implicit problem to update $\sigma$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where for a plane strain problem the stress tensor reads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  {\boldsymbol\sigma} = \left[&#xA;  \begin{array}{c}&#xA;\sigma_{11}\\&#xA;\sigma_{22}\\&#xA;\sigma_{33}\\&#xA;\sigma_{12}&#xA;\end{array}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $R$ is:&#xA;$$&#xA;  \mathbf{R}_{\boldsymbol\sigma} = {\boldsymbol\sigma}_{n+1} - {\boldsymbol\sigma}_n - \mathbf{D}\Delta{\boldsymbol\epsilon_{n}}  + \Delta t\,\mathbf{D}\left[\mathbf{Z}^p({\boldsymbol\sigma}^{\rm{dev}}) + \mathbf{Z}^d({\boldsymbol\sigma}^{\rm{dev}})\right]&#xA;$$&#xA;where&#xA;$$&#xA; \begin{align}&#xA; \left[\mathbf{Z}^d({\boldsymbol\sigma}^{\rm{dev}})\right]_i &amp;amp;= \alpha\,\left[{\boldsymbol\sigma}^{\rm{dev}}\right]_i \\&#xA; \left[\mathbf{Z}^p({\boldsymbol\sigma}^{\rm{dev}})\right]_i &amp;amp;= \beta\,||{\boldsymbol\sigma}^{\rm{dev}}||^{\frac{1-m}{m}} \left[{\boldsymbol\sigma}^{\rm{dev}}\right]_i \\&#xA; \left[{\boldsymbol\sigma}^{\rm{dev}}\right]_i &amp;amp;= \left[\mathbf{I}^{\rm{dev}}\right]_{ij}\,\left[{\boldsymbol\sigma}\right]_j&#xA; \end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$I^{dev}$ is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_{ij}^{dev} = \delta_{ij} - \frac{1}{3}m_im_j \quad\text{where}\quad \mathbf{m} = \left[&#xA;\begin{array}{c}&#xA;1\\&#xA;1\\&#xA;1\\&#xA;0&#xA;\end{array}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\delta$ is the &lt;code&gt;Kronecker delta&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tangent modulus is given by&#xA;$$&#xA;  d\mathbf{R}_{\boldsymbol{\sigma}} = \frac{\partial \mathbf{R}_{\boldsymbol{\sigma}}}{\partial {\boldsymbol{\sigma}}}\,d{\boldsymbol{\sigma}}&#xA; = \left[\mathbf{I} + \Delta t\,\mathbf{D}\left(\frac{\partial \mathbf{Z}^d({\boldsymbol\sigma}^{\rm{dev}})}{\partial \boldsymbol{\sigma}} + \frac{\partial \mathbf{Z}^p({\boldsymbol\sigma}^{\rm{dev}})}{\partial \boldsymbol{\sigma}}\right) \right]\,d{\boldsymbol{\sigma}} = \mathbf{A}\,d{\boldsymbol{\sigma}}&#xA;$$&#xA;If we expand $\mathbf{A}$ we get&#xA;$$&#xA; \mathbf{A} = \mathbf{I} + \Delta t\,\mathbf{D}\left[\left(\alpha+\beta\,||{\boldsymbol\sigma}^{\rm{dev}}||^{\frac{1-m}{m}}\right)\,\mathbf{I}^{\rm{dev}} +&#xA; \frac{(1-m)\beta}{m}\,||{\boldsymbol\sigma}^{\rm{dev}}||^{\frac{1-3m}{m}}{\boldsymbol\sigma}^{\rm{dev}}\otimes{\boldsymbol\sigma}^{\rm{dev}}\right]&#xA;$$&#xA;&#xA;Here $x=\sigma_{n+1}$. Also, $\alpha, \beta, \Delta t, I_{ij}^{dev}, D$ and $m$ are constant($m=0.3$ and $i,j=1,...,4$.). $\sigma_n$ and $\Delta{\boldsymbol\epsilon_{n}}$ are constant in step &lt;code&gt;n&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dl.dropboxusercontent.com/u/21907062/result.txt&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; you can see the results for $R$ and $\sigma_{n+1}$.&lt;/p&gt;&#xA;" OwnerUserId="7919" LastEditorUserId="7919" LastEditDate="2014-04-13T08:39:35.260" LastActivityDate="2014-05-14T03:00:00.110" Title="Newton-Raphson method fails!" Tags="|fortran|newton-method|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="11341" PostTypeId="2" ParentId="11335" CreationDate="2014-04-10T18:23:29.530" Score="0" Body="&lt;p&gt;If a least-squares solution to your singular problem suffices, you might try using the iterative methods in &lt;a href=&quot;http://www.ece.ualberta.ca/~tchen/papers/ding_scl05n2.pdf&quot; rel=&quot;nofollow&quot;&gt;Iterative least-squares solutions of coupled Sylvester equations&lt;/a&gt;. These methods should require less memory than your current approach.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-04-10T18:23:29.530" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11344" PostTypeId="2" ParentId="11338" CreationDate="2014-04-11T01:02:12.050" Score="3" Body="&lt;p&gt;Direct vs. iterative is certainly one of the key questions in linear solvers for sparse matrices as you have already observed. There are many misconceptions and lots of misinformation on this topic. So I encourage you to approach the issue from this point of view rather than looking for &quot;5 methods.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a nice introduction from one of the major contributors to direct methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.numerical.rl.ac.uk/reports/duffRAL98076.pdf&quot; rel=&quot;nofollow&quot;&gt;Matrix methods, Duff I.S.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You refer to Professor Saad's classic book on iterative methods for sparse systems.&#xA;In the course he teaches at U. of Minnesota, he spends about equal time on direct and&#xA;iterative methods. His notes are a gold mine of information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www-users.cselabs.umn.edu/classes/Spring-2014/csci8314/&quot; rel=&quot;nofollow&quot;&gt;Sparse Matrix Computations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="20162" LastEditDate="2016-08-13T07:25:36.033" LastActivityDate="2016-08-13T07:25:36.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11345" PostTypeId="2" ParentId="351" CreationDate="2014-04-11T08:00:45.290" Score="1" Body="&lt;p&gt;Armadillo, Boost and others now included as part of Ceemple, a rapid JIT based C++ technical computing environment. Available (free) from &lt;a href=&quot;http://www.ceemple.com&quot; rel=&quot;nofollow&quot;&gt;http://www.ceemple.com&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7925" LastActivityDate="2014-04-11T08:00:45.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11348" PostTypeId="1" AcceptedAnswerId="11360" CreationDate="2014-04-11T10:30:09.143" Score="1" ViewCount="336" Body="&lt;p&gt;I have an integral of the form&#xA;$$&#xA;I(n) = \int_0^1 dx f(x) \cos(n \pi x) ,&#xA;$$&#xA;where $n$ is an integer. In other words, I calculate the cosine Fourier coefficients of function $f$, which is real and continuous on the interval. I need to calculate this for a large number of large values of $n$. Currently I'm just doing this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;iint = array([ integrate.quad(lambda x: f(x)*cos(n*pi*x), 0, 1, limit=1000)[0] for n in range(0,nlim) ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $nlim$ is typically of the order of $\sim 10^3$. I have the feeling that this is not the most optimal way of doing this. How do I make my calculation faster?&lt;/p&gt;&#xA;" OwnerUserId="7899" LastActivityDate="2014-04-12T16:26:13.550" Title="Fast way to compute integral of type $\int dx f(x) \cos(n \pi x)$ in SciPy" Tags="|quadrature|scipy|oscillations|fourier-transform|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11349" PostTypeId="2" ParentId="11335" CreationDate="2014-04-11T11:23:03.820" Score="0" Body="&lt;p&gt;Another idea that might help is mimicking the Bartels-Stewart algorithm for solving general Sylvester equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming that $A$ and $B$ are full $n\times n$ matrices here, so there is no sparsity property to exploit. The resulting algorithm will cost $O(n^3)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to compute Schur forms $A=QUQ^T$ and $A=RLR^T$, with $U$ upper triangular, $L$ lower triangular, $diag(U)=diag(L)$ and $Q,R$ orthogonal. (You can get the latter from the Schur form of $A^T$.) Then you can transform your problem into the equation $UY-YL=F$, with $Y=Q^TXR$, $F=Q^TBR$. Using the fact that $U$ and $L$ are triangular, you can solve the system one entry of $Y$ at a time with a sort of back-substitution. For instance, comparing the $(n,n)$ entries of the LHS and RHS you get $U_{nn}Y_{nn}-Y_{nn}L_{nn}=F_{nn}$, which has only one unknown and can be solved explicitly. Then you solve for $Y_{n-1,n}$ and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference from the &quot;traditional&quot; Bartels-Stewart algorithm is that in your case the equations corresponding to terms on the diagonal will be degenerate, since $U_{nn}=L_{nn}$. So you'll need $F_{ii}=0$ to ensure solvability, and you can set the corresponding elements of $Y$ arbitrarily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I explained myself -- if you want to learn more on the Bartels-Stewart algorithm, their original article from the 70s is short and readable in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-04-11T11:23:03.820" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11350" PostTypeId="2" ParentId="11348" CreationDate="2014-04-11T13:40:15.207" Score="2" Body="&lt;p&gt;Isn't this the same as the real part of the Fourier transform of f(x)&#xA;$$&#xA;\begin{align}&#xA;I(n) &amp;amp;= \int^1_0dxf(x)\cos(n\pi x) &#xA;   \\ &amp;amp;=\Re{{\int^1_0dx f(x) e^{-j n \pi x}}}&#xA;   \\ &amp;amp;= \Re\int^1_0dx f(x) (cos(n\pi x) - j\sin(n \pi x))&#xA;   \\ &amp;amp;= \int^1_0dxf(x)\cos(n\pi x) &#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2014-04-11T13:40:15.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11351" PostTypeId="2" ParentId="8886" CreationDate="2014-04-11T14:37:14.430" Score="1" Body="&lt;p&gt;For a high-performance implementation that is not only among the highest performing (better than 85% of peak on 60 cores of the Intel Xeon Phi), but is also imho the most beautifully written, have a look at BLIS:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/flame/blis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/flame/blis&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="320" LastEditorUserId="15" LastEditDate="2018-04-01T22:35:01.780" LastActivityDate="2018-04-01T22:35:01.780" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11352" PostTypeId="1" CreationDate="2014-04-11T15:08:01.920" Score="0" ViewCount="379" Body="&lt;p&gt;I have a set of modified compressible Euler equations that I would like to solve using a WENO method. The issue is that the modified flux function involves derivative and filtering terms and I'm not sure how to compute the derivative terms in a consistent manner within the WENO scheme. For simplicity, the key aspects of the problem can be illustrated with the modified 1D Burgers equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t + \left( \dfrac{1}{2}u^2+\dfrac{3}{2}\alpha^2\overline{(u_x)^2} \right)_x = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the overbar denotes a filtering operation using the Helmholtz filter&#xA;$\overline{f} = g^\alpha*f$&#xA;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g^\alpha(x) = \dfrac{1}{2\alpha}e^{-|x/\alpha|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\alpha$ is on the scale of the grid spacing, this PDE produces near discontinuities at the scale of the grid spacing so I would like to use a WENO method for simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; given the values of $u$ at the grid points, how should I compute $u_x$ at the grid points so that I can reconstruct the flux function in a WENO fashion since there may be near-discontinuities in $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, would it be easier/more sensible to reconstruct $u$ and $u_x$ at half-grid points and then compute the flux function from those reconstructed values? If so, how should I compute $u_x$ at the half grid points?&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-04-16T18:14:24.733" Title="WENO reconstruction of flux involving derivative terms" Tags="|fluid-dynamics|hyperbolic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11353" PostTypeId="1" CreationDate="2014-04-11T15:38:39.320" Score="2" ViewCount="4140" Body="&lt;p&gt;This is originally a problem in programming, but since almost no one on Stackoverflow know how to solve this I went here instead; &lt;a href=&quot;https://stackoverflow.com/questions/23003612/javascript-angular-velocity-by-vector-2d&quot;&gt;https://stackoverflow.com/questions/23003612/javascript-angular-velocity-by-vector-2d&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to convert X and Y velocities to angular velocity, this is the formula I am currently using to calculate the initial velocity by the x and y values and then turn it into angular velocity for my circle object:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Av = Sqrt(Vx^2 + Vy^2) / R&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Angularvelocity = Squareroot of (Velocity x^2 + Velocity y^2) / Circle's radius&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how it simulates in my programming: &lt;a href=&quot;http://jsfiddle.net/yzb9P/2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://jsfiddle.net/yzb9P/2/&lt;/a&gt; (Click to change the balls position)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now since a square root can't be negative, this won't work when the ball is supposed to rotate anti-clockwise. So, I need a signed version of the initial velocity that also can be negative, how do I calculate that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard about that the Wedge product is working for this, and I've read many articles about it too, but I still don't understand how to use it, please help!&lt;/p&gt;&#xA;" OwnerUserId="7929" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-04-13T01:32:25.843" Title="Angular Velocity by Vector - 2D" Tags="|vector|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11355" PostTypeId="2" ParentId="11353" CreationDate="2014-04-11T16:15:45.777" Score="6" Body="&lt;p&gt;A cross product will tell you both the magnitude and the sign of your angular velocity. In general, angular velocity is defined by a vector as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{\omega} = \dfrac{\vec{r}\times\vec{v}}{|\vec{r}|^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\vec{\omega}$ is the angular velocity, $\vec{r}$ is the vector from the center of rotation to the point under consideration, and $\vec{v}$ is the velocity vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In two dimensions this becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega = \dfrac{r_x v_y - r_y v_x}{r_x^2+r_y^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where a positive value corresponds to counter-clockwise rotation and negative means clockwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;br&gt;&#xA;After looking at your JSfiddle some more, it looks like your implementation has the more serious issue of giving rotation even when the ball hits straight down on a flat surface. If you have non-zero velocity, you're computing a non-zero angular velocity without regard to the vector directions involved. The cross product will eliminate this effect. I believe this is what you want: &lt;a href=&quot;http://jsfiddle.net/yzb9P/7/&quot; rel=&quot;nofollow&quot;&gt;http://jsfiddle.net/yzb9P/7/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2:&lt;/strong&gt;&lt;br&gt;&#xA;If you want your ball to roll more realistically, there is another bug in your posted code. Try this version, the ball rolls down a slope without stopping unnaturally. I only edited the &lt;code&gt;update()&lt;/code&gt; function. &lt;a href=&quot;http://jsfiddle.net/yzb9P/8/&quot; rel=&quot;nofollow&quot;&gt;http://jsfiddle.net/yzb9P/8/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edite 3:&lt;/strong&gt;&lt;br&gt;&#xA;A final update with improved implementation of elasticity for the ball: &lt;a href=&quot;http://jsfiddle.net/yzb9P/11/&quot; rel=&quot;nofollow&quot;&gt;http://jsfiddle.net/yzb9P/11/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-04-13T01:32:25.843" LastActivityDate="2014-04-13T01:32:25.843" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11356" PostTypeId="2" ParentId="11352" CreationDate="2014-04-11T17:02:01.087" Score="2" Body="&lt;p&gt;&lt;em&gt;Caveat:&lt;/em&gt;  I haven't personally implemented WENO before, so take this with a grain of salt.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Alternatively, would it be easier/more sensible to reconstruct $u$ and $u_x$ at half-grid points and then compute the flux function from those reconstructed values?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I &lt;em&gt;think&lt;/em&gt; generally you want to reconstruct $u$ first. The fundamental notion in WENO is to select the best reconstruction of $u$ given a set of nodal values. That reconstruction gives you a $unique$ polynomial. Since that polynomial is unique (and non-oscillatory), you also have a unique value for $u_x$ which is then easily computed directly from that polynomial. It seems to me that if you tried to independently reconstruct both $u$ and $u_x$ from nodal values something would be inconsistent.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-04-11T17:02:01.087" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11357" PostTypeId="1" CreationDate="2014-04-11T15:25:13.017" Score="0" ViewCount="159" Body="&lt;p&gt;This is originally a problem in programming, but since almost no one on Stackoverflow know how to solve this I went here instead; &lt;a href=&quot;https://stackoverflow.com/questions/23003612/javascript-angular-velocity-by-vector-2d&quot;&gt;https://stackoverflow.com/questions/23003612/javascript-angular-velocity-by-vector-2d&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to convert X and Y velocities to angular velocity, this is the formula I am currently using to calculate the initial velocity by the x and y values and then turn it into angular velocity for my circle object:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Av = Sqrt(Vx^2 + Vy^2) / R&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Angularvelocity = Squareroot of (Velocity x^2 + Velocity y^2) / Circle's radius&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how it simulates in my programming: &lt;a href=&quot;http://jsfiddle.net/yzb9P/2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://jsfiddle.net/yzb9P/2/&lt;/a&gt; (Click to change the balls position)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now since a square root can't be negative, this won't work when the ball is supposed to rotate anti-clockwise. So, I need a signed version of the initial velocity that also can be negative, how do I calculate that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard about that the Wedge product is working for this, and I've read many articles about it too, but I still don't understand how to use it, please help!&lt;/p&gt;&#xA;" OwnerUserId="7929" OwnerDisplayName="Murplyx" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-04-11T20:47:10.453" Title="Angular Velocity by Vector - 2D" Tags="|software|" AnswerCount="1" CommentCount="4" ClosedDate="2014-04-11T22:04:21.367" ContentLicense="CC BY-SA 3.0" />
  <row Id="11358" PostTypeId="2" ParentId="11357" CreationDate="2014-04-11T15:43:31.660" Score="1" Body="&lt;p&gt;The angular velocity vector can be computed via the &lt;a href=&quot;http://en.wikipedia.org/wiki/Cross_product&quot; rel=&quot;nofollow&quot;&gt;cross product&lt;/a&gt; of the position and velocity vectors:&#xA;$$&#xA;\boldsymbol\omega=\frac{\mathbf r\times\mathbf v}{|\mathbf r|^2}\tag{1}&#xA;$$&#xA;In two dimensions, this is really&#xA;$$&#xA;\mathbf a\times\mathbf b=\det(\mathbf{a\,b})=a_xb_y - a_yb_x&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't really know Java well at all, but it looks like your code is already equipped with computing the cross product (&lt;code&gt;wedge: function(v)&lt;/code&gt;). But note that your line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ball.av = ball.v.wedge(ball.v.length())/ball.r&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Seems to be the problem area. Given the definition of &lt;code&gt;wedge&lt;/code&gt;, this should have a negative sign in front as $\mathbf r\times\mathbf v=-\mathbf v\times\mathbf r$. It is also unclear to me how &lt;code&gt;ball.v.length()&lt;/code&gt; and &lt;code&gt;ball.r&lt;/code&gt; are related..&lt;/p&gt;&#xA;" OwnerUserId="4762" OwnerDisplayName="Kyle Kanos" LastActivityDate="2014-04-11T15:43:31.660" CommentCount="53" ContentLicense="CC BY-SA 3.0" />
  <row Id="11359" PostTypeId="2" ParentId="11328" CreationDate="2014-04-12T00:21:49.367" Score="2" Body="&lt;p&gt;It's a quadratic program with linear inequality constraints, which are efficiently solved by the active set method. I pose implementing this method as a homework in my optimization courses and it doesn't take my students more than at most a couple of hours, assuming you have a solver for the linear systems you need to solve in each iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest to just follow the description in Nocedal-Wright (Nocedal and Wright: &quot;Numerical Optimization&quot;). The algorithm is sufficiently simple that you will probably not spend much more time to implement it than to get used to the interface of some other library. Even better, if one of the other answers and comments turns up a free QP solver, then wrapping the active set method around it is pretty straight-forward.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-12T00:21:49.367" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11360" PostTypeId="2" ParentId="11348" CreationDate="2014-04-12T03:01:02.927" Score="4" Body="&lt;p&gt;This is part of the (complex-valued) Fourier transform for which there is (provably) no more efficient way than the Fast Fourier Transform (FFT) if you want to compute the integrals for at least a significant fraction of all frequencies $n$ between zero and the largest frequency you care about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, while your intuition may tell you that evaluating the function at a large number of points is probably in efficient, the intuition is wrong in this case.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-12T03:01:02.927" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11363" PostTypeId="2" ParentId="11348" CreationDate="2014-04-12T16:26:13.550" Score="2" Body="&lt;p&gt;If $f$ is very nice, then you can approximate $f$ with a sequence of piecewise polynomials and integrate over the resulting intervals exactly. This may be much, much cheaper than using an FFT. This is also true for approximations of $f$ by any functions where you can easily know or determine the exact antiderivative in each interval. If $f$ is a black box, then you'll have more trouble with this method. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-04-12T16:26:13.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11364" PostTypeId="1" CreationDate="2014-04-12T18:44:55.737" Score="1" ViewCount="36" Body="&lt;p&gt;While calculating the eccentric anomaly of a satellite using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow&quot;&gt;Newton-Raphson method&lt;/a&gt;, what initial estimate of the root for E (eccentric anomaly) should be taken for least number of iterations(best approximation of root) to take place and why?&lt;/p&gt;&#xA;" OwnerDisplayName="user44457" LastActivityDate="2014-04-12T21:46:20.740" Title="Keplerian Orbit Estimator" Tags="|computational-physics|" AnswerCount="0" CommentCount="2" ClosedDate="2019-06-07T17:31:55.010" ContentLicense="CC BY-SA 3.0" />
  <row Id="11366" PostTypeId="1" CreationDate="2014-04-13T00:42:19.243" Score="0" ViewCount="307" Body="&lt;p&gt;Consider the nonlinear system of equations&#xA;$$&#xA;(1) \quad qk^2a_1^2E^2+wna_0a_1AE+pnka_0^2a_1E+rn^2a_0^2A^2-rn^2a_0^3A^2+qk^2a_0a_1ABE-qk^2na_0^2E^2=0,&#xA;$$&#xA;$$&#xA;(2) \quad nwa_1^2AE+nwa_0a_1AB+2pnka_0a_1^2AE+2pnka_0^2a_1AB+2qk^2a_1^2ABE-qk^2na_1^2ABE-2qk^2na_0a_1AmE+qk^2na_0a_1AB^2+2rn^2a_0a_1A^2-3rn^2a_0^2a_1A^2=0,&#xA;$$&#xA;$$&#xA;(3) \quad  rn^2a_1^2A^2+nwa_1^2AB-nwa_0a_1Am+pnka_1^2AE+2pnka_0a_1^2AB-pnka_0a_1^2Am-2qk^2a_1^2m+qk^2a_1^2B^2-3qk^2na_0a_1AmB+qk^2na_1^2AB^2-qk^2na_1^2B^2+2qk^2na_1mE-2qk^2na_1AmE=0,&#xA;$$&#xA;$$&#xA;(4) \quad - nwa_1^2Am-rn^2a_1^3A^3+pnka_1^3AB-pnka_0a_1^2Am-2qk^2a_1^2mB+2qk^2na_1^2mB-3qk^2na_1^2AmE+2qk^2na_0a_1Am^2=0,&#xA;$$&#xA;$$&#xA;(5) \quad qk^2-pnka_1A+2qk^2nAm-qk^2nm=0;&#xA;$$&#xA;My question for this nonlinear system is: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using Matlab symbolic computation solve for $a_0$,$a_1$ and $w$  in terms of the rest variables $(q,k,p,n,A,B,E,r,m)$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here my tag is under&#xA;matlab; nonlinear system; symbolic computation:::: if any please correct me&lt;/p&gt;&#xA;" OwnerUserId="7922" LastEditorUserId="549" LastEditDate="2014-04-13T05:03:13.193" LastActivityDate="2014-04-14T17:58:25.727" Title="Systems of nonlinear equations" Tags="|matlab|nonlinear-equations|symbolic-computation|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="11368" PostTypeId="2" ParentId="10869" CreationDate="2014-04-13T01:02:02.360" Score="0" Body="&lt;p&gt;Use binary tree addition , ie ,&#xA;Choose the mean of the distribution(closest number) as the root of the binary tree,&#xA;and create a sorted binary tree by adding lesser values to the left of the graph and larger ones to the right and so on.&#xA;The add all child nodes of a single parent recursively in a bottom up approach. This will be efficient as the avg error increases with the number of summations and in a binary tree approach, the number of summations are in the order of log n in base 2. Hence the avg error would be lesser.&lt;/p&gt;&#xA;" OwnerUserId="7942" LastActivityDate="2014-04-13T01:02:02.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11369" PostTypeId="1" AcceptedAnswerId="11400" CreationDate="2014-04-13T02:27:15.750" Score="9" ViewCount="757" Body="&lt;p&gt;Let $A$ be a large symmetric positive definite matrix, and suppose that we can efficiently apply $A$ and have a fast solver to apply $A^{-1}$, but we do not have access to the matrix entries for either $A$ or $A^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can one use this to solve the system,&#xA;$$\sqrt{A}x=b?$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As suggested in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10375/efficient-computation-of-the-matrix-square-root-inverse&quot;&gt;this previous question&lt;/a&gt;, the problem can be solved by computing an eigenvalue decomposition of $A$. In my case a full eigenvalue decomposition is infeasible for speed and memory reasons, as are factorizations that focus on elements of the matrix such as Cholesky.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-04-16T21:00:12.700" Title="Applying matrix square root inverse in matrix-free regime" Tags="|linear-algebra|matrix-free|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11370" PostTypeId="2" ParentId="11340" CreationDate="2014-04-13T04:37:50.210" Score="0" Body="&lt;p&gt;As a check we can look at the problem in terms tensors.&#xA;Let&#xA;$$&#xA;  \boldsymbol{\sigma}^{\text{dev}} = \boldsymbol{\sigma} - \frac{1}{3}(\boldsymbol{\sigma}:\mathbf{I})\,\mathbf{I}&#xA;$$&#xA;where $\mathbf{I}$ is the second-order identity tensor.  Then the fourth-order tensor $\mathsf{I}^{\rm dev}$ is given by&#xA;$$&#xA;  \mathsf{I}^{\rm dev} = \mathsf{I}^{(4s)} - \frac{1}{3} \mathbf{I}\otimes\mathbf{I}&#xA;$$&#xA;where $\mathsf{I}^{(4s)}$ is the symmetric fourth-order identity tensor with components $1/2(\delta_{ik}\delta_{jl} + \delta_{il}\delta_{jk})$ and $\delta_{ij}$ is the Kronecker delta.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can also express $\mathbf{Z}^p$ in tensor form as&#xA;$$&#xA;  \boldsymbol{Z}^p = \beta\,\left\|\boldsymbol{\sigma}^{\rm dev}\right\|^{\frac{1-m}{m}} \boldsymbol{\sigma}^{\rm dev} \,.&#xA;$$&#xA;Then&#xA;$$&#xA; \frac{\partial \boldsymbol{Z}^p}{\partial\boldsymbol{\sigma}} = &#xA;  \beta\left[\frac{\partial}{\partial \boldsymbol{\sigma}}\left(\left\|\boldsymbol{\sigma}^{\rm dev}\right\|^{\frac{1-m}{m}}\right)\otimes\boldsymbol{\sigma}^{\rm dev} + \left\|\boldsymbol{\sigma}^{\rm dev}\right\|^{\frac{1-m}{m}} \frac{\partial \boldsymbol{\sigma}^{\rm dev}}{\partial\boldsymbol{\sigma}}\right]&#xA;$$&#xA;Let us look at the first term&#xA;$$&#xA;\frac{\partial}{\partial \boldsymbol{\sigma}}\left(\left\|\boldsymbol{\sigma}^{\rm dev}\right\|^{\frac{1-m}{m}}\right)&#xA; = \frac{1-m}{m}\left\|\boldsymbol{\sigma}^{\rm dev}\right\|^{\frac{1-2m}{m}} \frac{\partial\left\|\boldsymbol{\sigma}^{\rm dev}\right\|}{\partial \boldsymbol{\sigma}}&#xA;$$&#xA;Next look at the part&#xA;$$&#xA;\frac{\partial\left\|\boldsymbol{\sigma}^{\rm dev}\right\|}{\partial \boldsymbol{\sigma}} = \frac{1}{2\left\|\boldsymbol{\sigma}^{\rm dev}\right\|}\,\frac{\partial}{\partial\boldsymbol{\sigma}}(\boldsymbol{\sigma}^{\rm dev}:\boldsymbol{\sigma}^{\rm dev})&#xA;$$&#xA;Finally, we look at the last bit above, using $\boldsymbol{\sigma}^{\rm dev}:\mathbf{I} = 0$, &#xA;$$&#xA; \frac{\partial}{\partial\boldsymbol{\sigma}}(\boldsymbol{\sigma}^{\rm dev}:\boldsymbol{\sigma}^{\rm dev}) = \mathsf{I}^{\rm dev}:\boldsymbol{\sigma}^{\rm dev} + \boldsymbol{\sigma}^{\rm dev}:\mathsf{I}^{\rm dev} = 2\boldsymbol{\sigma}^{\rm dev}&#xA;$$&#xA;Therefore,&#xA;$$&#xA;\frac{\partial \boldsymbol{Z}^p}{\partial\boldsymbol{\sigma}} = \frac{\beta(1-m)}{m}\left\|\boldsymbol{\sigma}^{\rm dev}\right\|^{\frac{1-3m}{m}} \boldsymbol{\sigma}^{\rm dev}\otimes\boldsymbol{\sigma}^{\rm dev} + \beta\,\left\|\boldsymbol{\sigma}^{\rm dev}\right\|^{\frac{1-m}{m}}\mathsf{I}^{\rm dev} \,.&#xA;$$&#xA;&lt;s&gt;This is different from the expression you have used.  So you will need to recheck your algebra.&lt;/s&gt; So the tangent modulus appears to be correct.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastEditorUserId="7626" LastEditDate="2014-04-14T02:49:55.993" LastActivityDate="2014-04-14T02:49:55.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11371" PostTypeId="2" ParentId="11369" CreationDate="2014-04-13T04:46:00.770" Score="9" Body="&lt;p&gt;After looking through some literature I'm answering my own question. I'll wait on accepting for a few days in case anyone else has a better answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, the problem is equivalent to solving,&#xA;$$A x = \sqrt{A}b,$$&#xA;so the only square root part needed is computing $\sqrt{A}b$, after which it is a normal solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following paper discusses two ways to do that,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Numerical approximation of the product of the square root of a matrix with a vector&lt;/em&gt;, &#xA;  Allen, EJ and Baglama, J and Boyd, SK ,&#xA;  Linear Algebra and its Applications 2000,&#xA;  &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0024379500000689&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0024379500000689&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The second method transforms the problem into solving an ODE whose solution at time $t=1$ is $A^{1/2}b$. This ODE is,&#xA;$$\begin{cases}&#xA;\frac{dx}{dt} = -\frac{1}{2}\left(At + (1-t)I\right)^{-1}(I-A)x(t) \\&#xA;x(0) = b,&#xA;\end{cases}$$&#xA;which can be solved through standard techniques such as forward Euler. At each timestep $t_k$, $(I-A)$ must be applied to $x(t_k)$, and then a regularized version of $A^{-1}$:&#xA;$$\left(At_k + (1-t_k)I\right)^{-1}$$&#xA;must be applied to the result. In my particular situation the method for applying $A^{-1}$ can be extended to also apply $(A + \alpha I)^{-1}$. In other situations it should be possible to use $A^{-1}$ to build a preconditioner for $\left(At_k + (1-t_k)I\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; A major downside of this method is that the ODE becomes stiff as the condition number of $A$ increases.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2014-04-16T21:00:12.700" LastActivityDate="2014-04-16T21:00:12.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11372" PostTypeId="1" CreationDate="2014-04-13T06:10:26.287" Score="3" ViewCount="1640" Body="&lt;p&gt;I'm writing a Monte Carlo simulation in which I have to maintain a large collection of items. This collection contains a great many duplicates, and it will most likely be best to store some or all of these in the form of a number that records the number of duplicates, rather than storing each one individually. (The items are essentially strings, so they have a non-negligible memory cost.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On each iteration I remove a random item from the collection, and possibly add one or two new items. Each item has a weight, and when sampling from the container it's important that the probability of drawing a given item is equal to its weight multiplied by the number of items of that type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The items that get added on each iteration may or may not be duplicates of ones that are already in the container. The items will probably be roughly Pareto distributed, with a few items having loads of duplicates and many having none, but it's hard to tell in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems like a fairly common thing to want to do, and it also seems not entirely trivial to do it efficiently. The obvious way is to store the items in some kind of hash table, pairing each with an integer representing its frequency. The problem is that then it's not very efficient to sample from the container, since you'd have to iterate over the items in an essentially random order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if they were stored in something like a balanced binary tree then sampling would be very efficient. However, since the container's contents are changing on every update, this would involve rebalancing the tree all the time, which (at least if done in a naïve way) would be very inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that I need a data structure has a trade-off between speed of sampling and speed of updating. Perhaps some kind of partially-sorted heap-like thing that can be re-sorted with a low average cost when only a few items have changed. But I don't know of such a structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is a standard algorithm / data structure for this purpose then what's it called? If there is there a readily available implementation in Python and/or C++ then that would  be a huge bonus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;(Note that user &lt;code&gt;thus spake a.k.&lt;/code&gt; gave an elegant answer to the question in the non-weighted case, but I've realised that for my application, the weights are really necessary.)&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="3746" LastEditorUserId="3746" LastEditDate="2014-04-20T07:53:06.533" LastActivityDate="2014-04-22T23:43:25.893" Title="Algorithm for efficient weighted sampling from a collection that can efficiently be updated" Tags="|data-structures|random-sampling|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11373" PostTypeId="2" ParentId="11372" CreationDate="2014-04-13T14:26:36.190" Score="0" Body="&lt;p&gt;The &quot;standard&quot; data structures for this in C++ are the multiset&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cplusplus.com/reference/set/multiset/&quot; rel=&quot;nofollow&quot;&gt;http://www.cplusplus.com/reference/set/multiset/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;based on a balanced binary tree and the unordered_multiset&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cplusplus.com/reference/unordered_set/unordered_multiset/&quot; rel=&quot;nofollow&quot;&gt;http://www.cplusplus.com/reference/unordered_set/unordered_multiset/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;based on a hash table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would try those first and see if their performance is adequate for your requirements.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-04-13T14:26:36.190" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11375" PostTypeId="2" ParentId="11372" CreationDate="2014-04-13T18:19:35.193" Score="2" Body="&lt;p&gt;Assuming C++, if you're willing to store duplicate items, a time efficient approach for the unweighted case is to store them unsorted in a &lt;code&gt;std::vector&lt;/code&gt; or &lt;code&gt;std::deque&lt;/code&gt;, &lt;code&gt;v&lt;/code&gt;.&lt;br&gt;&#xA;You can then efficiently add new items by inserting them at the end of &lt;code&gt;v&lt;/code&gt;.&lt;br&gt;&#xA;To draw a sample, simply pick a random index &lt;code&gt;i&lt;/code&gt; into &lt;code&gt;v&lt;/code&gt; and use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample = v[i];&#xA;v[i] = v.back();&#xA;v.pop_back();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the additional storage cost you'll get (amortized in the case of &lt;code&gt;std::vector&lt;/code&gt;) constant time iterations.&lt;/p&gt;&#xA;" OwnerUserId="7945" LastActivityDate="2014-04-13T18:19:35.193" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11376" PostTypeId="1" AcceptedAnswerId="11377" CreationDate="2014-04-13T18:58:36.320" Score="0" ViewCount="150" Body="&lt;p&gt;I've been reading a paper about using &lt;a href=&quot;http://perception.csl.illinois.edu/matrix-rank/Files/robust_v19.pdf&quot; rel=&quot;nofollow&quot;&gt;Matrix Completion for Photometric Stereo&lt;/a&gt; but I am having some troubles in &lt;strong&gt;section 2.2&lt;/strong&gt; trying to understand why irrespective of the number of pixels and the number of images $$rank(\mathbf{O}) = 3$$&#xA;where $\mathbf{O}$ is the matrix with the images linearized and stacked as column vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone give me a hint to understand this?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2014-05-29T13:45:43.783" Title="Rank of image intensity matrix" Tags="|computer-vision|rank|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11377" PostTypeId="2" ParentId="11376" CreationDate="2014-04-13T20:52:15.160" Score="2" Body="&lt;p&gt;The images described in the paper are not independent, but are uniquely defined by a lighting direction which is 3 dimensional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could thus construct any conceivable image (in this model) by lighting the scene from 3 orthogonal directions (x, y and z for instance). This corresponds to the rank being 3. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="3304" LastEditDate="2014-05-29T13:45:43.783" LastActivityDate="2014-05-29T13:45:43.783" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11378" PostTypeId="2" ParentId="11340" CreationDate="2014-04-13T21:58:55.073" Score="1" Body="&lt;p&gt;I suggest you do the following to check your implementation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take your nearly-converged sigma vector and then calculate the 4x4 Jacobian matrix&#xA;numerically, column-by-column using a central difference approximation. You may have&#xA;to experiment a bit with the step size to obtain an accurate result. This numerical&#xA;Jacobian should be very close to your analytical one.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-04-13T21:58:55.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11379" PostTypeId="1" CreationDate="2014-04-14T01:10:03.330" Score="2" ViewCount="419" Body="&lt;p&gt;Consider the scalar  PDE for $u$ with Dirichlet boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathrm{div}(\mathcal{K}\nabla u) = f\; \forall x\;  \in \Omega \subset R^2$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u = 0 \; \forall \; x\;\in \partial\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal{K} \in R^{2\times 2}$ is positive definite symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a start I assume &lt;strong&gt;$\Omega$ is the unit square with a uniform rectangular mesh&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I edited the question following Jed's comment to make it more specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\mathcal{K}$ is diagonal, it is simple enough. When it is not, I see that there it will require  a computation of the &quot;secondary gradient&quot; (gradient along the face). How is that usually done ? And what I should do when I am near the boundary in this case ?&lt;/p&gt;&#xA;" OwnerUserId="7728" LastEditorUserId="7728" LastEditDate="2014-04-14T22:02:54.513" LastActivityDate="2014-04-14T22:02:54.513" Title="finite volume for diffusion equation with anisotropic (tensor) coefficient" Tags="|finite-difference|finite-volume|poisson|advection-diffusion|tensor|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11380" PostTypeId="2" ParentId="11194" CreationDate="2014-04-14T11:06:42.687" Score="3" Body="&lt;p&gt;If your compiler has &lt;code&gt;nextafter&lt;/code&gt;, then you can do something like this, which works regardless of which rounding mode is set:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double x2 = x*x;&#xA;double lo = nextafter(x2,-1);&#xA;double hi = nextafter(x2,x2+1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I don't know whether &lt;code&gt;nextafter&lt;/code&gt; is inlined (probably not).&lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="640" LastEditDate="2014-04-14T11:13:58.433" LastActivityDate="2014-04-14T11:13:58.433" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11381" PostTypeId="1" CreationDate="2014-04-14T14:01:20.167" Score="2" ViewCount="1077" Body="&lt;p&gt;I am trying to solve &lt;strong&gt;2D Poisson equations with Neumann boundary conditions&lt;/strong&gt;. When the mesh is uniform, Poisson equation is &lt;strong&gt;singular and symmetric&lt;/strong&gt;, so the method listed in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7488/null-space-projection-for-singular-systems&quot;&gt;Null Space Projection for Singular Systems&lt;/a&gt; works well. But when the mesh is non uniform and sharp immersed boundary method is implemented to treat the curved boundary, discretized Poisson equation is &lt;strong&gt;singular and asymmetric&lt;/strong&gt; based on finite difference method. In this case, the pervious method seems not working well. So can someone help me and tell me how to solve this kind of problem, or just provide me some references? Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="7948" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-04-14T14:01:20.167" Title="How to solve singular non symmetric poisson equation with Neumann boundary condtions?" Tags="|linear-algebra|finite-difference|sparse-matrix|matrix|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11382" PostTypeId="2" ParentId="11366" CreationDate="2014-04-14T15:42:25.027" Score="2" Body="&lt;p&gt;I think it is unlikely that these equations have a solution. This is already apparent by the fact that you have 5 equations for three variables, but one can look at it in more detail:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Take equation 5. It is linear in $a_1$ and does no involve the other variables. You can solve it for $a_1$ and plug the value into the other equations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you do this, equation 4 becomes linear in the remaining variables, $a_0$ and $w$. Solve for one of them in terms of the other and plug the result into the first three equations. Let's say you solve for $w$ in terms of $a_0$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This makes equation 3 a quadratic equation in $a_0$ because it contains a term proportional to $a_0w$. Being a quadratic equation, it has two solutions that are easily computed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Equation 2 is also quadratic equation in $a_0$. You can again compute its two solutions. Only those values that simultaneously solve equations 2 and 3 can possibly be solutions of the joint system.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Equation 1 is cubic in $a_0$. If any of the joint solutions of equations 2 and 3 also solve this equation (easily verified by plugging it in and seeing whether left and right hand sides of the equation are equal) then you have an answer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-14T15:42:25.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11384" PostTypeId="2" ParentId="11366" CreationDate="2014-04-14T17:58:25.727" Score="0" Body="&lt;p&gt;This is a polynomial system of equations.  You can try to find a closed form solution by computing a &lt;a href=&quot;http://www.mathworks.com/help/symbolic/mupad_ref/groebner-gbasis.html&quot; rel=&quot;nofollow&quot;&gt;Groebner basis using a lexicographic monomial ordering&lt;/a&gt;.  Doing so should give you similar information to that outlined by Wolfgang Bangerth in his answer.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2014-04-14T17:58:25.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11385" PostTypeId="2" ParentId="11322" CreationDate="2014-04-14T19:51:26.017" Score="0" Body="&lt;p&gt;The FT of a three dimensional real function is not in general real, so there is no way you could express it in real spherical harmonics. I think however that if you only include harmonics with even $l$ the result should be real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FT of a spherically symmetric function $g(r)$ is $S(Q) = \int_{0}^{\infty} \frac{r}{Q} \sin(Qr) g(r)\,\mathrm{d}r$ (with some factors of $2\pi$ depending on how you define the FT). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 3D function decomposed into spherical harmonics is a sum of products $g_{lm}(r) d_{lm}(\theta,\phi)$, so the FT will be a sum of convolutions $S(Q) \otimes \mathrm{FT} \left[ d_{lm}(\theta,\phi) \right]$. I can't think of an easy way to calculate these in general.&lt;/p&gt;&#xA;" OwnerUserId="3458" LastActivityDate="2014-04-14T19:51:26.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11386" PostTypeId="2" ParentId="2255" CreationDate="2014-04-14T20:38:33.153" Score="1" Body="&lt;p&gt;There's a Masters thesis &quot;Comparison Between Mimetic and Two-Point Flux-Approximation Schemes on PEBI-grids&quot; that walks through some of the details, and section 7.3 in particular works through a small example by hand.&lt;/p&gt;&#xA;" OwnerUserId="326" LastActivityDate="2014-04-14T20:38:33.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11387" PostTypeId="1" AcceptedAnswerId="11395" CreationDate="2014-04-14T22:38:24.997" Score="6" ViewCount="1593" Body="&lt;p&gt;I will try to give the motivation behind this problem and later the math formality.&lt;br&gt;&#xA;Given a grayscale image (1 Channel - $M \times N$ Matrix).&lt;br&gt;&#xA;Someone marks some pixels as anchors.&lt;br&gt;&#xA;Now, you need to interpolate the other pixels (Which are not anchors) by minimizing a given cost function s.t. the end result is an image which has the original image values at the anchors and interpolated values else were s.t. it minimizes the cost function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given an $M\times N$ matrix (A 1 channel image for that matter) $ I $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A subset of the elements (Pixels) in the matrix are marked as a reference and their location is a group marked as $ S $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimization cost function is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ E = \arg \min_{E} \sum_{\mathbf{r}} \left( E(\mathbf{r}) - \sum_{\mathbf{s} \in N(\mathbf{r})} {w}_{\mathbf{rs}} E(\mathbf{s}) \right)^2, \text{ s.t. }  \forall p \in S \; E(\mathbf{p}) = I(\mathbf{p})\,, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where a bold letter $ \mathbf{p}, \mathbf{r}, \mathbf{s} $ means an element (Pixel) location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The group $ N(r) $ is the neighborhood of $ \mathbf{r} $, which is size $ k $ namely, a $ k  $ by $ k $ rectangle where $ \mathbf{r} $ is in the middle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weights $ {w}_{\mathbf{rs}} $ are defined as following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ {w}_{\mathbf{rs}} \propto \exp\left(-\frac{(I(\mathbf{r}) - I(\mathbf{s}))^2}{2\sigma^2_r} \right )\ \ \text{ s.t. } \sum_{\mathbf{s} \in N(\mathbf{r})} w_{\mathbf{rs}} = 1\,. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Namely, the weights are normalized to 1 within the neighborhood. The variance is calculated on the matrix $ I $ in the neighborhood (You can assume it is given).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the problem is to find a matrix $ E $ which is equal to $ I $ on all reference points and interpolates other places by bringing the cost function to the minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like a weighted least squares per neighborhood, yet I couldn't formalize it in a way that can be easily calculated and solved in e.g. MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="7951" LastEditorUserId="7951" LastEditDate="2018-08-15T13:46:24.153" LastActivityDate="2023-07-15T16:53:48.887" Title="Interpolation by Solving a Minimization Problem (Optimization)" Tags="|optimization|interpolation|constrained-optimization|image-processing|computer-vision|" AnswerCount="2" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="11388" PostTypeId="2" ParentId="11387" CreationDate="2014-04-15T14:58:50.100" Score="2" Body="&lt;p&gt;Using few tricks the equation can be transformed into classic &lt;a href=&quot;https://en.wikipedia.org/wiki/Quadratic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quadratic Programming&lt;/a&gt; problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ E = \arg \max_{E} \sum_{\mathbf{r}} \left( E(\mathbf{r}) - \sum_{\mathbf{s} \in N(\mathbf{r})} {w}_{\mathbf{rs}} E(\mathbf{s}) \right)^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining $ \boldsymbol{e} = vec \left ( E \right ) $, Namely &lt;a href=&quot;https://en.wikipedia.org/wiki/Vectorization_%28mathematics%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;vectorizing&lt;/a&gt; the matrix, and defining a matrix $ A $ defined the proper usage of the weights $ {w}_{\mathbf{rs}} $, Namely put them in the correct entry at each row per pixel the above could be rewritten as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{e} = \arg \max_{\mathbf{e}} {\left ( \mathbf{e} - A\mathbf{e} \right ) }^{T} \left ( \mathbf{e} - A\mathbf{e} \right ) = \arg \max_{\mathbf{e}} {\left( \left ( I - A \right ) \mathbf{e} \right )}^{T} \left( \left ( I - A \right ) \mathbf{e} \right ) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining $ B = I - A $ yields:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{e} = \arg \max_{\mathbf{e}} {\left( \left ( I - A \right ) \mathbf{e} \right )}^{T} \left( \left ( I - A \right ) \mathbf{e} \right ) = \arg \max_{\mathbf{e}} {\mathbf{e}}^{T} {B}^{T} B \mathbf{e} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining $ L = {B}^{T} B $ , namely $ L $, is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Symmetric_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symmetric&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Positive-definite_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Positive Definite Matrix&lt;/a&gt; yields:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{e} = \arg \max_{\mathbf{e}} {\mathbf{e}}^{T} L \mathbf{e}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding the equality constraints using $ C \mathbf{e} = \mathbf{t} $ where $ \mathbf{t} $ is &lt;a href=&quot;https://en.wikipedia.org/wiki/Vectorization_%28mathematics%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;vectorization&lt;/a&gt; of all &quot;Reference Pixels&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all this yields a Classic Quadratic Programming problem given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{e} = \arg \max_{\mathbf{e}} {\mathbf{e}}^{T} L \mathbf{e}, \; s.t. \; C \mathbf{e} = \mathbf{t} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Few notes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The matrix $ L $ is &lt;a href=&quot;https://en.wikipedia.org/wiki/Symmetric_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symmetric&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Positive-definite_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Positive Definite matrix&lt;/a&gt; and usually sparse (Assuming &quot;Small Neighborhood&quot;).&lt;/li&gt;&#xA;&lt;li&gt;The matrix $ L $ wast built by the Matrix $ I - A $ which each row of it was normalized to be zero (Each row of $ A $ is normalized to 1 ).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now two things are needed:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Optimized fast algorithm to solve this problem utilizing its properties.&lt;/li&gt;&#xA;&lt;li&gt;Assuming the image is large, are there methods to solve this problem by blocks to avoid memory limits?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7951" LastEditorUserId="7951" LastEditDate="2018-05-13T13:03:48.403" LastActivityDate="2018-05-13T13:03:48.403" CommentCount="0" CommunityOwnedDate="2014-04-15T14:58:50.100" ContentLicense="CC BY-SA 4.0" />
  <row Id="11389" PostTypeId="2" ParentId="11322" CreationDate="2014-04-15T18:00:45.953" Score="9" Body="&lt;p&gt;You may find the &lt;a href=&quot;http://en.wikipedia.org/wiki/Plane_wave_expansion&quot; rel=&quot;noreferrer&quot;&gt;expansion of a plane wave in spherical waves&lt;/a&gt; to be helpful here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;e^{i\mathbf{k}\cdot\mathbf{x}} = 4\pi\sum_{l=0}^\infty\sum_{m=-l}^l i^lj_l(kr)Y_{lm}(\theta,\phi)Y_{lm}^*(\vartheta,\varphi)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\theta$, $\phi$ are the angular variables for $\mathbf{x}$ and $\vartheta$, $\varphi$ for $\mathbf{k}$; the radial functions $j_l$ are the spherical Bessel functions; and $Y_{lm}$ are the spherical harmonics*. In that case, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathscr{F}f(\mathbf{k}) = (2\pi)^{-3/2}\int d^3x\hspace{2pt} e^{i\mathbf{k}\cdot\mathbf{x}}f(\mathbf{x})  &#xA;$$&#xA;$$ = (2\pi)^{-3/2}\int d^3x\left(4\pi\sum_{l,m}i^lj_l(kr)Y_{lm}(\theta,\phi)Y_{lm}^*(\vartheta,\varphi)\right)\left(\sum_{l',m'}g_{l'm'}(r)Y_{l'm'}(\theta,\phi)\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can use the orthogonality of spherical harmonics to reduce this to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathscr{F}f(k,\vartheta,\varphi) = \sqrt{\frac{2}{\pi}}\sum_{l,m}i^lY_{lm}^*(\vartheta,\varphi) \cdot \int j_l(kr)g_{lm}(r)r^2dr.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You now have an integral in only the radial dimension. If you have a book of special functions handy, some recurrence relations or asymptotics for $j_l$ will prove invaluable no matter how you choose to evaluate those integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*Every time you use this formula, a kitten is born, a child smiles, and your lifespan increases by 8 minutes.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2014-04-15T20:53:33.910" LastActivityDate="2014-04-15T20:53:33.910" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11390" PostTypeId="1" CreationDate="2014-04-15T19:16:45.293" Score="2" ViewCount="289" Body="&lt;p&gt;A dense matrix is distributed for parallel computation column-wise, then multiplied from left &amp;amp; right by sparse matrices. What would be appropriate c++ libraries for these tasks?&lt;/p&gt;&#xA;" OwnerUserId="7962" LastEditorUserId="7962" LastEditDate="2014-04-15T23:32:47.227" LastActivityDate="2014-05-10T03:58:02.150" Title="Dense distributed matrix" Tags="|matrix|parallel-computing|sparse-matrix|dense-matrix|hpc|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11393" PostTypeId="1" CreationDate="2014-03-18T20:26:20.943" Score="2" ViewCount="130" Body="&lt;p&gt;Heres a couple of questions,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Does a CAS exist that can manipulate Cartesian based tensors and produce a simplified output that is hopefully in index notation?&#xA;2) If not, is there some indirect method to accomplish this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm talking about doing rather simple manipulations such as computing,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^{2}_{\mathbf{r}}\left(\frac{a}{r}\left[\mathbb{I} + \frac{\mathbf{r}\mathbf{r}}{r^{2}}\right] + \left(\frac{a}{r}\right)^{3}\left[\mathbb{I} - 3 \frac{\mathbf{r}\mathbf{r}}{r^{2}}\right]\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in a 3D Cartesian basis where $\mathbf{r} = \langle x,y,z\rangle$, $\mathbb{I}$ is the identity 3x3 matrix, $r = |\mathbf{r}|$ etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I've tried:&#xA;I've tried using Maxima to do this the &quot;pedestrian&quot; way (i.e. making vectors and explicitly computing matricies etc...).  This just results in a complex mess of a big 3x3 matrix with long expressions.  This &quot;works&quot; but the main problem is that simplification is not easy in this case (or maybe I'm not doing it right).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of other &quot;Tensor software&quot; (e.g. &lt;a href=&quot;http://en.wikipedia.org/wiki/Tensor_software&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Tensor_software&lt;/a&gt;) that does these sorts of things but they all seem to be made for more general or complex uses and its not clear to me how to control them for my use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I ask if anyone has any experience with this and could perhaps provide an example where simple manipulations like this in a CAS can be performed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7965" OwnerDisplayName="kmarsh" LastActivityDate="2014-04-16T14:25:37.570" Title="CAS to perform &quot;simple&quot; tensor operations in index notation" Tags="|symbolic-computation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11395" PostTypeId="2" ParentId="11387" CreationDate="2014-04-16T00:26:02.220" Score="13" Body="&lt;p&gt;This is the classic &lt;a href=&quot;http://www.cs.huji.ac.il/%7Eyweiss/Colorization/&quot; rel=&quot;noreferrer&quot;&gt;colorization using optimization&lt;/a&gt; problem.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gjTJa.png&quot; alt=&quot;enter image description here&quot; /&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/0oqlt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Optimization and Linear Algebra&lt;/h1&gt;&#xA;&lt;p&gt;To see how this can be expressed as a linear system, it's helpful to use a slightly different notation (and a slightly different objective function). Think of your image as graph &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; with a node for each pixel in the image. There is an edge &lt;span class=&quot;math-container&quot;&gt;$(i,j)$&lt;/span&gt; between two pixels &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, if these pixels are neighbors in the image.&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; be the intensity of pixel &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, then you can write the problem as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA; \begin{array}{ll}&#xA;   \underset{x}{\mathrm{minimize}} &amp;amp; \frac{1}{2} \sum_{(i,j) \in E} w_{ij} (x_i - x_j)^2 \\&#xA;   \mathrm{subject\ to} &amp;amp; x_i = d_i, i \in S.&#xA;  \end{array}&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For each edge in the graph we have weights denoted by &lt;span class=&quot;math-container&quot;&gt;$w_{ij} = w_{ji}$&lt;/span&gt;. Given this weighted graph &lt;span class=&quot;math-container&quot;&gt;$G(V, E, w)$&lt;/span&gt; we can define the adjacency matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$  A_{ij} = \begin{cases} w_{ij} &amp;amp; \text{if}\ (i,j) \in E\\ 0 &amp;amp; \text{otherwise} \end{cases} $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The degree matrix of a weighted graph is a diagonal matrix &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; such that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ D_{ii} = \sum_{j} A_{ij}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The Laplacian matrix of a weighted graph is defined as &lt;span class=&quot;math-container&quot;&gt;$L = D - A$&lt;/span&gt;. With this definition of the Laplacian we can see that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ x^T L x = \sum_{(i,j) \in E} w_{ij} (x_i - x_j)^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thus we can rewrite the above problem in matrix form as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\begin{array}{ll}&#xA; \underset{x}{\mathrm{minimize}} &amp;amp; \frac{1}{2} x^T L x \\&#xA; \mathrm{subject\ to} &amp;amp; x_m = d\\&#xA;\end{array}&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the Laplacian matrix for the image, &lt;span class=&quot;math-container&quot;&gt;$x_m$&lt;/span&gt; is the set of unknowns corresponding to marked pixels, and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the vector of annotated values (anchors).&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; be a permutation matrix that orders &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; so that the marked and unmarked unknowns are grouped such that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ Px = \begin{bmatrix} x_m \\ x_u \end{bmatrix}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then we have that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \frac{1}{2} x^T L x = \frac{1}{2} x^T P^T (PLP^T) Px = \frac{1}{2} \begin{bmatrix} x_m^T  &amp;amp; x_u^T \end{bmatrix} \begin{bmatrix} L_m &amp;amp; R^T \\ R &amp;amp; L_u \end{bmatrix} \begin{bmatrix} x_m \\ x_u \end{bmatrix}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here we have that &lt;span class=&quot;math-container&quot;&gt;$L_u = L_u^T \succ 0$&lt;/span&gt;. Since the values of the variables &lt;span class=&quot;math-container&quot;&gt;$x_m$&lt;/span&gt; are fixed (&lt;span class=&quot;math-container&quot;&gt;$x_m = d$&lt;/span&gt;) we need only optimize over the variables &lt;span class=&quot;math-container&quot;&gt;$x_u$&lt;/span&gt;. Thus we can solve the unconstrained problem&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \underset{x_u}{\mathrm{minimize}}\ F(x_u) = \frac{1}{2} x_u^T L_u x_u + x_m^T R^T x_u$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$L_u \succ 0$&lt;/span&gt;, the unique minimizer &lt;span class=&quot;math-container&quot;&gt;$x_u^\star$&lt;/span&gt; is characterized by &lt;span class=&quot;math-container&quot;&gt;$\nabla F(x_u^\star) = 0$&lt;/span&gt;. We have that &lt;span class=&quot;math-container&quot;&gt;$\nabla F(x_u) = L_u x_u + R x_m$&lt;/span&gt;. Thus we need only solve the linear symmetric positive-definite system&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L_u x_u = -Rd$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can solve this with a direct method (i.e. \ in MATLAB or conjugate gradient). For a class in grad school I implemented PCG in CUDA on a graphics card to color a 1050 x 3360 image which corresponded to a linear systems with 2.1 million unknowns. It took 230s on two Quadro FX 5600 GPUs to solve the system.&lt;/p&gt;&#xA;&lt;h1&gt;Sample MATLAB code&lt;/h1&gt;&#xA;&lt;p&gt;Below is sample MATLAB code which computes a colorization using the above images&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;original = double(imread('example.bmp'))/255;&#xA;marked   = double(imread('example_marked.bmp'))/255;&#xA;&#xA;annotate_thresh = 0.25; % may need to be adjusted&#xA;colorim = sum(abs(original - marked), 3) &amp;gt; annotate_thresh;&#xA;colorim = double(colorim);&#xA;spy(colorim);&#xA;anchorind = find(colorim);&#xA;&#xA;[m,n,p] = size(marked);&#xA;assert(p == 3);&#xA;N = m*n;&#xA;&#xA;% convert from RGB to NTSC using the matrix T&#xA;T = [1.0 0.956 0.621; 1.0 -0.272 -0.647; 1.0 -1.106 1.703];&#xA;yiq = reshape(marked(:), N, 3)/T';&#xA;yiq = reshape(yiq, m, n, 3);&#xA;chromi = yiq(:,:,2);&#xA;chromq = yiq(:,:,3);&#xA;% Get grayscale value from original image&#xA;yiq = reshape(original(:), N, 3)/T';&#xA;yiq = reshape(yiq, m, n, 3);&#xA;gray = yiq(:,:,1);&#xA;&#xA;% Get anchors from the I and Q color components&#xA;anchor(:,1) = chromi(anchorind);&#xA;anchor(:,2) = chromq(anchorind);&#xA;&#xA;edges=[(1:N)' ((1:N)+1)'];        %add downard edge between node i with node i+1&#xA;edges=[edges; (1:N)' (1:N)'+m];   %add leftward edge between node i and i+m&#xA;% exclude edges which link to nodes outside the domain&#xA;excluded=edges(:,1)&amp;gt;N | edges(:,1)&amp;lt;1 | edges(:,2)&amp;gt;N |edges(:,2)&amp;lt;1; &#xA;edges(excluded, :) = [];&#xA;% remove edge from bottom of column to top of next column&#xA;edges((m:m:(m-1)*n)',:)=[]; &#xA;&#xA;beta = 200;&#xA;EPSILON = 1e-5;&#xA;&#xA;% calculate edge weights&#xA;graydistance = abs(gray(edges(:,1)) - gray(edges(:,2)));&#xA;ming = min(graydistance);  % it's helpful to normalize distances&#xA;maxg = max(graydistance);&#xA;graydistance = (graydistance - ming)./(maxg - ming);&#xA;weights=exp(-(beta*graydistance)) + EPSILON;&#xA;&#xA;%Build sparse weighted adjacency matrix&#xA;W=sparse([edges(:,1);edges(:,2)],[edges(:,2);edges(:,1)], ...&#xA;    [weights;weights],N,N);&#xA;% Build Degree matrix&#xA;D = diag(sum(W));&#xA;% Build Laplacian&#xA;L=D-W;&#xA;&#xA;mark = anchorind;&#xA;rest = 1:n*m;&#xA;rest(mark) = [];&#xA;rest = rest';&#xA;% Extract the portion of the Lapalcian matrices&#xA;Lu = L(rest, rest);&#xA;R  = L(rest, mark);&#xA;recover = zeros(m, n, 3);&#xA;recover(:,:,1) = gray;&#xA;xchannel = zeros(m*n,1);&#xA;for i=1:2&#xA;    d  = anchor(:,i);&#xA;    xrest = -Lu\(R*d);&#xA;    xchannel(rest) = xrest;&#xA;    xchannel(mark) = d;&#xA;    recover(:,:,i+1) = reshape(xchannel, m, n);&#xA;end&#xA;% Convert from YIQ back to RGB using T matrix&#xA;recover = reshape(recover, m*n, 3);&#xA;recover = recover*T';&#xA;% Keep RGB values within [0,1]&#xA;recover = min(max(0, recover), 1);&#xA;final   = reshape(recover, m, n, 3);&#xA;&#xA;figure(1);&#xA;image(final);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is the image below&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mQgZk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Why a different objective function&lt;/h1&gt;&#xA;&lt;p&gt;I choose a different objective function than the one stated in your problem.&#xA;There are several reasons for this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The quantity &lt;span class=&quot;math-container&quot;&gt;$w_{ij} (x_i - x_j)^2$&lt;/span&gt; has a clear interpretation in the objective. You pay a weighted penalty based on the square of the difference&#xA;between two adjacent pixels.&lt;/li&gt;&#xA;&lt;li&gt;Using the definition of the Laplacian matrix &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; it's easy to go from this objective function to the matrix equation &lt;span class=&quot;math-container&quot;&gt;$x^T L x$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Using the definition of &lt;span class=&quot;math-container&quot;&gt;$L = A^T C A$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the edge-node adjacency matrix and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a diagonal matrix containing the edge weights) it's easy to see that the Laplacian is positive semi-definite. If we add a small bit of regularization to make it positive definite, we can solve systems with &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; via a Cholesky factorization or preconditioned conjugate gradient.&lt;/li&gt;&#xA;&lt;li&gt;It's possible to do everything with the objective function you defined above. It's just more difficult to figure out how to translate it into a matrix equation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Note the idea of using this different objective function is not my own. I found it in a paper by Leo Grady on &lt;a href=&quot;https://ieeexplore.ieee.org/document/1704833&quot; rel=&quot;noreferrer&quot;&gt;Random Walks for Image Segmentation&lt;/a&gt; (See presentation &lt;a href=&quot;https://pdfs.semanticscholar.org/d957/b9abe50cf965f95ab02d43c49bf9c193f5e8.pdf&quot; rel=&quot;noreferrer&quot;&gt;Random Walks for Image Segmentation&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;h1&gt;Working with large images&lt;/h1&gt;&#xA;&lt;p&gt;An image with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; rows and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; columns will yield a matrix &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; of size &lt;span class=&quot;math-container&quot;&gt;$mn \times mn$&lt;/span&gt;. The number of nonzeros in &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is related to the size of the neighborhood around a pixel. In the code above, each pixel can have up to four neighbors: up, down, left, right. So the number of nonzeros in &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is bounded by &lt;span class=&quot;math-container&quot;&gt;$5mn$&lt;/span&gt;. Holding &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; in memory is equivalent to having a small number of copies of the image in memory.&lt;/p&gt;&#xA;&lt;p&gt;If you want to work with large images, you probably want to use some sort of iterative method. Iterative methods don't require you to form &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; directly, but rather compute matrix-vector products. That is given a vector &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; compute a vector &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$y=Lx$&lt;/span&gt;. As I mentioned above, I used preconditioned conjugate gradient, with diagonal preconditioning, on multiple graphics cards to solve this problem. This worked OK, but was still fairly slow on large images. To work with large images efficiently you may need to come up with a better preconditioner. There's been some research on this, for example see Richard Szeliski's paper on &lt;a href=&quot;http://research.microsoft.com/pubs/75694/Szeliski-SG06.pdf&quot; rel=&quot;noreferrer&quot;&gt;&amp;quot;Locally Adapted Hierarchical Basis Preconditioning&amp;quot;&lt;/a&gt;. I think others have also investigated using &lt;a href=&quot;http://en.wikipedia.org/wiki/Multigrid_method&quot; rel=&quot;noreferrer&quot;&gt;multigrid methods&lt;/a&gt; to solve this problem.&lt;/p&gt;&#xA;" OwnerUserId="7966" LastEditorUserId="7951" LastEditDate="2023-07-15T16:53:48.887" LastActivityDate="2023-07-15T16:53:48.887" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11397" PostTypeId="1" AcceptedAnswerId="11404" CreationDate="2014-04-16T05:22:35.830" Score="3" ViewCount="276" Body="&lt;p&gt;TUCKER_ALS computes the best rank-(R1,R2,..,Rn) approximation of tensor X, according to the specified dimensions. I am using &lt;a href=&quot;http://www.sandia.gov/~tgkolda/TensorToolbox/index-2.5.html&quot; rel=&quot;nofollow&quot;&gt;MATLAB Tensor Toolbox Version 2.5&lt;/a&gt;. I am wondering if I write a paper, how can I refer to the algorithm?&lt;/p&gt;&#xA;" OwnerUserId="7969" LastEditorUserId="276" LastEditDate="2014-04-17T01:30:02.903" LastActivityDate="2018-04-27T18:22:53.383" Title="Is there a reference/source paper for the TUCKER_ALS() in Tensor Toolbox for MATLAB?" Tags="|matlab|tensor|tensor-decomposition|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11398" PostTypeId="1" AcceptedAnswerId="11403" CreationDate="2014-04-16T10:24:18.977" Score="1" ViewCount="231" Body="&lt;p&gt;In one article I faced with such terms as sSVD, cSVD and oSVD. As I understand sSVD - standart SVD, cSVD - svd for block-circulant matrices, but I can't find what is oSVD.&#xA;1) What is oSVD?&#xA;2) Can anyone suggest me a few articles about cSVD, oSVD and their implementations?&lt;/p&gt;&#xA;" OwnerUserId="5278" LastActivityDate="2014-04-16T19:54:40.347" Title="oSVD and cSVD terms" Tags="|algorithms|matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11399" PostTypeId="1" CreationDate="2014-04-16T10:45:25.300" Score="2" ViewCount="156" Body="&lt;p&gt;According to a recommendation, this is a re-post of &lt;a href=&quot;https://mathoverflow.net/questions/163537/lanczos-algorithm-with-thick-restart-on-a-dynamic-matrix&quot;&gt;that&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;currently, I'm working on a way to compute the 2 biggest eigenvalues of a real, symmetric, huge and sparse matrix that changes a few entries from time to time. The problem should be solved using an algorithm, that does not compute solutions from scratch every time a change occurs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My literature:&#xA;&lt;a href=&quot;http://crd-legacy.lbl.gov/~kewu/ps/trlan-siam.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper 1&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.cs.ucdavis.edu/~bai/Winter09/krylov.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper 2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But first of all - the static case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Lanczos algorithm, the desired Eigenvalues can be approximated very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dynamic case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read about the Lanczos algorithm with restart schemes and I picked the thick restart scheme as it seems most applicable. And I know that the term &quot;restart&quot; is not meant the way I'm using this method for. But I was hoping, that during restart the matrix is allowed to change and the approximate eigenvalues (and any other computations) before restart can still be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as my experiments show, modification of the matrix has as good as no effect on the result so far. The eigenvalues being computed still converge to the ones of the original matrix - only with marginal difference compared to the static case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose the problem is the krylov subspace (being computed for the original matrix) does not really change if I just change the matrix. But I thought it would adjust itself, as only approximated eigenvalues are needed in order to restart and the approximate eigenvalues converge appropriately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&#xA;Do you think the method can be modified to meet my needs?&#xA;Or do you think the whole &quot;no computations from scratch&quot; story is pointless and I should start the Lanczos algorithm all over again?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="7971" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2017-10-06T19:12:41.967" Title="Lanczos algorithm with thick restart on a dynamic matrix" Tags="|linear-algebra|algorithms|numerical-analysis|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11400" PostTypeId="2" ParentId="11369" CreationDate="2014-04-16T16:54:04.790" Score="8" Body="&lt;p&gt;Nick, your answer is certainly valid. However, the additional solve using $Ax = \tilde{b}$ where $\tilde{b} = A^{1/2}b$ can be avoided if you directly solve $f(A)x = b$. This is the subject of a book chapter by Henk van der Vorst &lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-3-642-58333-9_2&quot;&gt;http://link.springer.com/chapter/10.1007%2F978-3-642-58333-9_2&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several other options for computing matrix-square root (or its inverse) are available. As you mentioned solving an ODE is one such approach. Broadly speaking, there are a few other approaches - polynomial approximation to square root &lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF02083211#page-1&quot;&gt;http://link.springer.com/article/10.1007%2FBF02083211#page-1&lt;/a&gt;, rational approximations to square root &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.1310&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.1310&lt;/a&gt;, Krylov subspace based approaches &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/130920587&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/130920587&lt;/a&gt; and Extended Krylov subspace method &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/S0895479895292400&quot;&gt;http://epubs.siam.org/doi/pdf/10.1137/S0895479895292400&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have only provided a single example for each case but the literature on computing matrix square root is rich. The particular choice of the method you use will ultimately depend on the matrix and the distribution of eigenvalues. However, I tend to favor the rational approximation based on contour integral &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.1310&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.1310&lt;/a&gt; because even for ill-conditioned matrix, computing the f(A)b boils down to computing about a dozen solves on a shifted system of equations. It may not work if A is only available in matrix-free form, in which case the Extended Krylov method may be your best bet. &lt;/p&gt;&#xA;" OwnerUserId="4558" LastActivityDate="2014-04-16T16:54:04.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11401" PostTypeId="2" ParentId="2255" CreationDate="2014-04-16T17:11:55.813" Score="2" Body="&lt;p&gt;The support operators method (SOM) takes advantage of the fact that most partial differential equations are formulated in terms of the differential operators divergence $\nabla\cdot$, gradient $\nabla$, and curl $\nabla\times$. The SOM provides an approach for spatial differencing by constructing discrete analogs of the aforementioned differential operators. The discrete operators satisfy discrete versions of important differential and integral identities satisfied by the continuum operators. In essence, the SOM constructs a discrete version of the differential operator calculus. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The construction of a discrete calculus proceeds in two steps. First we choose a discrete form for one of the fundamental operators, termed the &lt;em&gt;prime&lt;/em&gt; operator. Then, based on some subset of differential and integral identities we choose to maintain, we construct the other fundamental operator(s), termed the &lt;em&gt;derived&lt;/em&gt; operators. The choice of the prime operator is application and discretization dependent. In a sense, the prime operator &quot;supports&quot; the construction of the derived operators. Conservation laws, solution symmetries and adjoint relationships between differential operators are examples of properties we want the discrete operators to mimic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a SOM discretization of the linear diffusion equation the mimetic discretization would mimic&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The Gauss-Green theorem to enforce the local conservation law&lt;/li&gt;&#xA;&lt;li&gt;The negative adjoint relationship between the flux and divergence operators, $-\textbf{K}\nabla = (\nabla\,\cdot)^{*}$ &lt;/li&gt;&#xA;&lt;li&gt;Guaranteed symmetry and positivity of the product of the discrete divergence and discrete flux&lt;/li&gt;&#xA;&lt;li&gt;The null space of the discrete flux operator is the constant functions.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Full details on the mimetic discretization of the diffusion equation are available in &lt;a href=&quot;http://permalink.lanl.gov/object/tr?what=info%3alanl-repo/lareport/LA-UR-12-24117&quot; rel=&quot;nofollow&quot;&gt;1D&lt;/a&gt; or &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999196902570&quot; rel=&quot;nofollow&quot;&gt;2D&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastActivityDate="2014-04-16T17:11:55.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11402" PostTypeId="2" ParentId="11352" CreationDate="2014-04-16T18:14:24.733" Score="0" Body="&lt;p&gt;Here's what I ended up doing for Burger's equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each grid point:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute the ENO polynomials of the desired order for $u$ and use these to compute $u_x$ at the grid points $x_j$, upwind based on local characteristic speed $u$.&lt;/li&gt;&#xA;&lt;li&gt;Use the appropriate smoothness measures (i.e. those in Jiang and Shu, 2005 &lt;a href=&quot;http://lsec.cc.ac.cn/lcfd/DEWENO/paper/WENO_1996.pdf&quot; rel=&quot;nofollow&quot;&gt;[PDF link]&lt;/a&gt;) to form the (WENO) weighted approximation of $u_x$&lt;/li&gt;&#xA;&lt;li&gt;Compute the flux function at the grid points using the now known values of $u$ and $u_x$&lt;/li&gt;&#xA;&lt;li&gt;Perform WENO reconstruction of the primitive function for the fluxes at half grid points.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So far, this seems to be working very well.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-04-16T18:14:24.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11403" PostTypeId="2" ParentId="11398" CreationDate="2014-04-16T19:54:40.347" Score="2" Body="&lt;p&gt;Looking at the abstract to the paper, this appears to be a term coined by the authors to describe their solution method which uses the singular value decomposition (specialized to make use of the fact that the matrix is a circulant matrix) with some kind of regularization to suppress ringing in the solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the paper is hidden behind a paywall, and my university has no subscription to this journal.  TE;DR (Too Expensive; Didn't Read.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm afraid that either the original poster will have to supply more details, or someone who has access to this journal will have to look at the paper to answer the question.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-04-16T19:54:40.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11404" PostTypeId="2" ParentId="11397" CreationDate="2014-04-17T01:33:17.790" Score="4" Body="&lt;p&gt;Kolda (one of the authors of the MATLAB Tensor Toolbox) also wrote a review paper in SIREV called &lt;a href=&quot;https://doi.org/10.1137/07070111X&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensor Decompositions and Applications&lt;/a&gt; that provides references for algorithms that are at least related. (Without looking at the source, I can't say if they're the same.) You could try looking at Section 4, particularly Section 4.3, for useful references.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="20688" LastEditDate="2018-04-27T18:22:53.383" LastActivityDate="2018-04-27T18:22:53.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11407" PostTypeId="2" ParentId="10815" CreationDate="2014-04-17T20:22:44.933" Score="5" Body="&lt;p&gt;The convergence of classical iterative solvers for linear systems is determined by the spectral radius of the iteration matrix, $\rho(\mathbf{G})$. &lt;strong&gt;For a general linear system, it is difficult to determine an optimal (or even good) SOR parameter due to the difficulty in determining the spectral radius of the iteration matrix.&lt;/strong&gt; Below I have included many additional details, including an example of a real problem where the optimal SOR weight is known.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Spectral radius and convergence&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The spectral radius is defined as the absolute value of the largest magnitude eigenvalue. A method will converge if $\rho&amp;lt;1$ and a smaller spectral radius means faster convergence. SOR works by altering the matrix splitting used to derive the iteration matrix based on the choice of a weighting parameter $\omega$, hopefully decreasing the spectral radius of the resulting iteration matrix.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Matrix splitting&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;For the discussion below, I will assume that the system to be solved is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A}x=b,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with an iteration of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^{(k+1)}=v+\mathbf{G}x^{(k)},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $v$ is a vector, and iteration number $k$ is denoted $x^{(k)}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SOR takes a weighted average of the old iteration and a Gauss-Seidel iteration. The Gauss-Seidel method relies on a matrix splitting of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A} = \mathbf{D} + \mathbf{L} + \mathbf{U}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{D}$ is the diagonal of $\mathbf{A}$, $\mathbf{L}$ is a lower triangular matrix containing all elements of $\mathbf{A}$ strictly below the diagonal and $\mathbf{R}$ is an upper triangular matrix containing all elements of $\mathbf{A}$ strictly above the diagonal. The Gauss-Seidel iteration is then given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^{(k+1)}=(\mathbf{D}+\mathbf{L})^{-1}b+\mathbf{G}_{\rm G-S}x^{(k)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the iteration matrix is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{G}_{\rm G-S}=-(\mathbf{D}+\mathbf{L})^{-1}\mathbf{U}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SOR can then be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^{(k+1)}=\omega(\mathbf{D}+\omega\mathbf{L})^{-1}b+\mathbf{G}_{\rm SOR}x^{(k)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{G}_{\rm SOR} = (\mathbf{D}+\omega\mathbf{L})^{-1}((1-\omega)\mathbf{D}-\omega\mathbf{U}).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining the convergence rate of the iterative scheme really boils down to determining the spectral radius of these iteration matrices. In general, this is a hard problem unless you know something specific about the structure of the matrix. There are very few examples that I'm aware of where the optimal weighting coefficient is computable. In practice, $\omega$ must be determined on the fly based on the observed (presumed) convergence of the running algorithm. This works in some cases, but fails in others.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Optimal SOR&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;One realistic example where the optimal weighting coefficient is known arises in the context of solving a Poisson equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2 u = f~{\rm in}~\Omega\\&#xA;u = g~{\rm on}~\partial\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discretizing this system on a square domain in 2D using second order finite differences with uniform grid spacing results in a symmetric banded matrix with 4 on the diagonal, -1 immediately above and below the diagonal, and two more bands of -1 some distance from the diagonal. There are some differences due to boundary conditions, but that is the basic structure. Given this matrix, the provably optimal choice for SOR coefficient is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega = \dfrac{2}{1+\sin(\pi \Delta x/L)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Delta x$ is the grid spacing and $L$ is the domain size. Doing so for a simple case with a known solution gives the following error versus iteration number for these two methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/aRDY6.png&quot; alt=&quot;Gauss-Seidel and SOR error&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, SOR reaches machine precision in about 100 iterations at which point Gauss-Seidel is about 25 orders of magnitude worse. If you want to play around with this example, I've included the MATLAB code I used below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all&#xA;close all&#xA;&#xA;%number of iterations:&#xA;niter = 150;&#xA;&#xA;%number of grid points in each direction&#xA;N = 16;&#xA;% [x y] = ndgrid(linspace(0,1,N),linspace(0,1,N));&#xA;[x y] = ndgrid(linspace(-pi,pi,N),linspace(-pi,pi,N));&#xA;dx = x(2,1)-x(1,1);&#xA;L = x(N,1)-x(1,1);&#xA;&#xA;%desired solution:&#xA;U = sin(x/2).*cos(y);&#xA;&#xA;% Right hand side for the Poisson equation (computed from U to produce the&#xA;% desired known solution)&#xA;Ix = 2:N-1;&#xA;Iy = 2:N-1;&#xA;f = zeros(size(U));&#xA;f(Ix,Iy) = (-4*U(Ix,Iy)+U(Ix-1,Iy)+U(Ix+1,Iy)+U(Ix,Iy-1)+U(Ix,Iy+1));&#xA;&#xA;figure(1)&#xA;clf&#xA;contourf(x,y,U,50,'linestyle','none')&#xA;title('True solution')&#xA;&#xA;%initial guess (must match boundary conditions)&#xA;U0 = U;&#xA;U0(Ix,Iy) = rand(N-2);&#xA;&#xA;%Gauss-Seidel iteration:&#xA;UGS = U0; EGS = zeros(1,niter);&#xA;for iter=1:niter&#xA;    for iy=2:N-1&#xA;        for ix=2:N-1&#xA;            UGS(ix,iy) = -1/4*(f(ix,iy)-UGS(ix-1,iy)-UGS(ix+1,iy)-UGS(ix,iy-1)-UGS(ix,iy+1));&#xA;        end&#xA;    end&#xA;&#xA;    %error:&#xA;    EGS(iter) = sum(sum((U-UGS).^2))/sum(sum(U.^2));&#xA;end&#xA;&#xA;figure(2)&#xA;clf&#xA;contourf(x,y,UGS,50,'linestyle','none')&#xA;title(sprintf('Gauss-Seidel approximate solution, iteration %d', iter))&#xA;drawnow&#xA;&#xA;%SOR iteration:&#xA;USOR = U0; ESOR = zeros(1,niter);&#xA;w = 2/(1+sin(pi*dx/L));&#xA;for iter=1:niter&#xA;    for iy=2:N-1&#xA;        for ix=2:N-1&#xA;            USOR(ix,iy) = (1-w)*USOR(ix,iy)-w/4*(f(ix,iy)-USOR(ix-1,iy)-USOR(ix+1,iy)-USOR(ix,iy-1)-USOR(ix,iy+1));&#xA;        end&#xA;    end&#xA;&#xA;    %error:&#xA;    ESOR(iter) = sum(sum((U-USOR).^2))/sum(sum(U.^2));&#xA;end&#xA;&#xA;figure(4)&#xA;clf&#xA;contourf(x,y,USOR,50,'linestyle','none')&#xA;title(sprintf('Gauss-Seidel approximate solution, iteration %d', iter))&#xA;drawnow&#xA;&#xA;&#xA;figure(5)&#xA;clf&#xA;semilogy(EGS,'b')&#xA;hold on&#xA;semilogy(ESOR,'r')&#xA;title('L2 relative error')&#xA;xlabel('Iteration number')&#xA;legend('Gauss-Seidel','SOR','location','southwest')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-04-17T20:27:51.133" LastActivityDate="2014-04-17T20:27:51.133" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11408" PostTypeId="2" ParentId="11328" CreationDate="2014-04-17T21:04:41.060" Score="0" Body="&lt;p&gt;Not a great solution, but what I'm doing right now:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the problem is in $R^3$, there can be at most 3 active plane constraints (inequalities turned equalities) at the correct solution.  Any more, and at least one of them would be redundant.  So I iterate through all possible combinations of 3, 2, 1, and 0 planes from among the $n$ inequalities ($\tbinom{n}{3}$ + $\tbinom{n}{2}$ + $\tbinom{n}{1}$ + 1) and find the one that produces a point that satisfies all other inequality constraints &lt;em&gt;and&lt;/em&gt; is closest to the target point (or actually, I transform everything so the target point is the origin, and find the candidate point with the smallest norm that satisfies all constraints).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given up to three plane constraints (the number of plane constraints being $d$), I stack them row by row to produce a $d$x$3$ matrix $A$ and similarly to produce a $d$x$1$ vector $\delta$.  I construct the system $A x = \delta$, and solve it in a least squares way using normal equations.  eg: $x = (A^TA)^+A^T\delta$.  I find the pseudoinverse of $A^TA$ using a spectral decomposition for 3x3 symmetric matrices based on quaternions (see &lt;a href=&quot;http://www.melax.com/diag.html?attredirects=0&quot; rel=&quot;nofollow&quot;&gt;clicky&lt;/a&gt;).  That is, $(A^TA) = Q D Q^T$, for an orthogonal matrix $Q$ and a diagonal matrix $D$, so $(A^TA)^+ = Q D^+ Q^T$.  $D^+_{i,i} = 1/D_{i,i}$ if $D_{i,i} &amp;gt; 0$, and $0$ otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check if a point satisfies all the plane inequalities I just check it against all planes one at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This produces an $O(n^4)$ algorithm: $O(n^3)$ combinations of 3 planes producing $O(n^3)$ points that need to be checked.  Each check needs to be against all $O(n)$ points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not particularly clever, and for large $n$ it's going to be too slow, but it was relatively simple to code and test and was able to leverage existing quaternion and matrix3 code I had.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything more clever I could be doing that doesn't require more sophisticated mathematical tools?  I don't have a general $n$x$n$ matrix library to leverage, just quaternions, 3x3 matrices, 4x4 matrices, and 3d/4d vectors.&lt;/p&gt;&#xA;" OwnerUserId="1365" LastActivityDate="2014-04-17T21:04:41.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11409" PostTypeId="1" CreationDate="2014-04-17T21:40:50.550" Score="1" ViewCount="595" Body="&lt;p&gt;I have a data mining assignment where I make a content-based image retrieval system.&#xA;I have 20 images of 5 animals. So in total 100 images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code returns the 10 most relevant images to an input image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I need to evaluate the performance of my system with a Precision-Recall curve. However, I do not understand the concept of a Precision-Recall curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say my system returns 10 most relevant images of a gorilla, but only 4 of them are gorillas. The other 6 images returned are other animals'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my precision is &lt;code&gt;4/10 = 0.4&lt;/code&gt;&#xA;and my recall is &lt;code&gt;4/20 = 0.2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I only have a point &lt;code&gt;&amp;lt;0.2,0.4&amp;gt;&lt;/code&gt; not a curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I have a curve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By changing the number of images returned (which is fixed = 10 in my case) ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks !&lt;/p&gt;&#xA;" OwnerUserId="7580" LastActivityDate="2014-04-18T18:07:07.120" Title="What is Precision-Recall Curve?" Tags="|matlab|image-processing|computer-vision|precision|plotting|" AnswerCount="1" CommentCount="5" ClosedDate="2014-04-18T21:32:22.440" ContentLicense="CC BY-SA 3.0" />
  <row Id="11410" PostTypeId="1" CreationDate="2014-04-17T23:18:13.203" Score="0" ViewCount="720" Body="&lt;p&gt;Say I have an LP that is unfeasible and that I want to find the solution that makes it feasible without &lt;em&gt;strongly&lt;/em&gt; violating the current constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a principled way of solving this problem, and how can I know &lt;strong&gt;a posteriori&lt;/strong&gt; what constraints would have to be violated?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the &lt;a href=&quot;http://lpsolve.sourceforge.net/5.5/Infeasible.htm&quot; rel=&quot;nofollow&quot;&gt;following example&lt;/a&gt;, but it is unclear to me how would one proceed to choose a good value for the penalty to the elastic variables. More concretely: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is &lt;strong&gt;any&lt;/strong&gt; penalty to the slack variables enough? How can I choose a &quot;significant cost&quot; for my slack variables?&lt;/li&gt;&#xA;&lt;li&gt;What is a principled way of setting up the cost coefficients of the slack variables? &lt;/li&gt;&#xA;&lt;li&gt;Is it &lt;strong&gt;always&lt;/strong&gt; the case that, anytime a slack variable in the solution is positive, the corresponding original constraint was infeasible? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;min: x + y;&#xA;c1: x &amp;gt;= 6;&#xA;c2: y &amp;gt;= 6;&#xA;c3: x + y &amp;lt;= 11;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;This model is clearly infeasible. We can now introduce slack variables&#xA;  at a significant cost&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;min: x + y + 1000 e1 + 1000 e2 + 1000 e3;&#xA;c1: x1 + e1 &amp;gt;= 6;&#xA;c2: y + e2 &amp;gt;= 6;&#xA;c3: x + y - e3 &amp;lt;= 11;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;The result of this model is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Value of objective function: 1011&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;Actual values of the variables:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x                               5&#xA;y                               6&#xA;e1                              1&#xA;e2                              0&#xA;e3                              0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;With this simple example model, multiple solutions were possible.&#xA;  Here, the first constraint was relaxed since e1 is non-zero. Only this&#xA;  one constraint had to be relaxed to make the model feasible. The&#xA;  objective value of 1011 isn't saying very much. However if we subtract&#xA;  1000 e1 + 1000 e2 + 1000 e3 from it, then it becomes 11 which is the&#xA;  value of the original objective function (x + y).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="13" LastActivityDate="2014-04-22T15:18:32.917" Title="Elastic LP Programming" Tags="|optimization|convex-optimization|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11411" PostTypeId="1" CreationDate="2014-04-18T10:30:55.993" Score="3" ViewCount="806" Body="&lt;p&gt;I'm trying to construct the 12 x 12 beam element stiffness matrix from a section constitutive matrix (6 x 6 with shear stiffnesses, axial stiffness, bending stiffnesses and torsional stiffness on the diagonal). I can get a 6 x 6 beam element matrix as would be used in a multibody formulation using the method described here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Malcolm, David J., and Daniel L. Laird. &quot;Modeling of blades as equivalent beams for aeroelastic analysis.&quot; ASME 2003 Wind Energy Symposium. American Society of Mechanical Engineers, 2003.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I want to use the 6 x 6 matrix that comes out of that process in a conventional finite element model which needs a 12 x 12 matrix. Is there a simple method of converting between the two? I've found information for a simple stiffness matrix but mine is fully populated as it's for a composite beam with lots of material couplings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a two-noded beam element, the element stiffness matrix $K$ relates the generalized forces $f$ to the generalized displacements $u$, i.e., $f = Ku$.&#xA;In matrix form&#xA;$$&#xA;  \begin{bmatrix} f_{x1} \\ f_{y1} \\ f_{z1} \\ m_{x1} \\ m_{y1} \\ m_{z1} \\&#xA;        f_{x2} \\ f_{y2} \\ f_{z2} \\ m_{x2} \\ m_{y2} \\ m_{z2}&#xA;  \end{bmatrix}&#xA;  = \begin{bmatrix} K_{1,1} &amp;amp; K_{1,2} &amp;amp; \dots &amp;amp; K_{1,12} \\&#xA;                    K_{1,2} &amp;amp; K_{2,2} &amp;amp; \dots &amp;amp; K_{2,12} \\&#xA;                    \vdots  &amp;amp; \vdots  &amp;amp; \ddots &amp;amp; \vdots  \\&#xA;                    K_{1,12} &amp;amp; K_{2,12} &amp;amp; \dots &amp;amp; K_{12,12} \\&#xA;    \end{bmatrix}&#xA;    \begin{bmatrix} u_{x1} \\ u_{y1} \\ u_{z1} \\ \theta_{x1} \\ \theta_{y1} \\ \theta_{z1} \\&#xA;        u_{x2} \\ u_{y2} \\ u_{z2} \\ \theta_{x2} \\ \theta_{y2} \\ \theta_{z2}&#xA;  \end{bmatrix} &#xA;$$&#xA;where $f_x, f_y, f_z$ are forces, $m_x, m_y, m_z$ are moments, $u_x, u_y, u_z$ are displacements, and $\theta_x, \theta_y, \theta_z$ are rotations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The section stiffness $S$ relates the generalized forces at a point in the beam to the generalized strains at that location, i.e., $f = S\varepsilon$. In matrix form,&#xA;$$&#xA; \begin{bmatrix} f_{x} \\ f_{y} \\ f_{z} \\ m_{x} \\ m_{y} \\ m_{z} &#xA;  \end{bmatrix} = \begin{bmatrix} S_{1,1} &amp;amp; S_{1,2} &amp;amp; \dots &amp;amp; S_{1,6} \\&#xA;                    S_{2,1} &amp;amp; S_{2,2} &amp;amp; \dots &amp;amp; S_{2,6} \\&#xA;                    \vdots  &amp;amp; \vdots  &amp;amp; \ddots &amp;amp; \vdots  \\&#xA;                    S_{6,1} &amp;amp; S_{6,2} &amp;amp; \dots &amp;amp; S_{6,6} \\&#xA;    \end{bmatrix}&#xA;    \begin{bmatrix} \gamma_x \\ \gamma_y \\ \varepsilon_z \\ \kappa_x \\ \kappa_y \\ \kappa_z \end{bmatrix}&#xA;$$&#xA;where $\gamma_x,\gamma_y$ are shear strains, $\varepsilon_z$ is a normal strain, and $\kappa_x,\kappa_y,\kappa_z$ are curvatures.  These strains are defined as&#xA;$$&#xA; \gamma_x = \frac{\partial u_x}{\partial z} - \theta_y \ , \ &#xA; \gamma_y = \frac{\partial u_y}{\partial z} + \theta_x \ , \ &#xA; \varepsilon_z = \frac{\partial u_z}{\partial z} \ , \ &#xA; \kappa_x = \frac{\partial \theta_x}{\partial z} \ , \&#xA; \kappa_y = \frac{\partial \theta_y}{\partial z} \ , \&#xA; \kappa_z = \frac{\partial \theta_z}{\partial z} \ .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integration of the above relations and then evaluation at the two nodes will probably lead to the beam stiffness matrix, but it is not clear how that should be done.&lt;/p&gt;&#xA;" OwnerUserId="7637" LastEditorUserId="7626" LastEditDate="2014-04-23T04:22:23.343" LastActivityDate="2014-04-23T04:22:23.343" Title="Beam finite element stiffness matrix from section constitutive matrix" Tags="|finite-element|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11412" PostTypeId="1" CreationDate="2014-04-18T15:17:34.210" Score="1" ViewCount="153" Body="&lt;p&gt;I am working on good retrieval mechanism for big data systems. I am of view that multidimensional structures could be of great use for aggregations and querying .  Multidimensional structures  as mentioned in theory/products could replace the need of OLAP CUBE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some data size info residing into data-warehouse and it is approx. around 1 trillion per month and some time even more and having rolling window of 1 month or greater, it depends per client. I used to write reports,sub-reports and calculate aggregates using sub-reports , and also used analytical and analytical aggregate function to accomplish the reports/query.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can some one share the thoughts on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&lt;/p&gt;&#xA;" OwnerUserId="7988" LastEditorUserId="7988" LastEditDate="2014-05-01T03:01:28.623" LastActivityDate="2014-05-01T03:01:28.623" Title="SciDB vs OLAP cube" Tags="|data-analysis|" AnswerCount="0" CommentCount="2" ClosedDate="2017-04-17T20:34:14.437" ContentLicense="CC BY-SA 3.0" />
  <row Id="11413" PostTypeId="2" ParentId="11409" CreationDate="2014-04-18T18:07:07.120" Score="1" Body="&lt;p&gt;Please check this link:&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Precision_and_recall&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Precision_and_recall&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check the ROC section as well. This is a very clear description I think. For a more thorough understanding, I would recommend:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://machinelearning.wustl.edu/mlpapers/paper_files/icml2006_DavisG06.pdf&quot; rel=&quot;nofollow&quot;&gt;http://machinelearning.wustl.edu/mlpapers/paper_files/icml2006_DavisG06.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-04-18T18:07:07.120" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11414" PostTypeId="1" AcceptedAnswerId="26732" CreationDate="2014-04-19T01:15:48.797" Score="5" ViewCount="2198" Body="&lt;p&gt;I'm looking for an efficient way to solve a symmetric tridiagonal system $Mx = d$, where the upper and lower diagonals of $M$ are offset from the main diagonal by $k$ rows/columns:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{bmatrix}&#xA;{b_1 } &amp;amp; {0   } &amp;amp; \ldots &amp;amp; {0      } &amp;amp; {a_1 } &amp;amp; {0   } &amp;amp; \ldots &amp;amp; {      0} \\&#xA;{0   } &amp;amp; {b_2 } &amp;amp; {0   } &amp;amp; {       } &amp;amp; {0   } &amp;amp; {a_2 } &amp;amp; {0    } &amp;amp; \vdots    \\&#xA;\vdots &amp;amp; {0   } &amp;amp; {b_3 } &amp;amp; {0      } &amp;amp; {    } &amp;amp; {0   } &amp;amp; \ddots  &amp;amp; {0      } \\&#xA;{0   } &amp;amp; {    } &amp;amp; {0   } &amp;amp; {b_4    } &amp;amp; {0   } &amp;amp; {    } &amp;amp; {     } &amp;amp; {a_{n-k}} \\&#xA;{a_1 } &amp;amp; {0   } &amp;amp; {    } &amp;amp; {0      } &amp;amp; {b_5 } &amp;amp; {0   } &amp;amp; {     } &amp;amp; {0      } \\&#xA;{0   } &amp;amp; {a_2 } &amp;amp; {0   } &amp;amp; {       } &amp;amp; {0   } &amp;amp; {b_6 } &amp;amp; {     } &amp;amp; \vdots    \\&#xA;\vdots &amp;amp; {    } &amp;amp; \ddots &amp;amp; {       } &amp;amp; {    } &amp;amp; {    } &amp;amp; \ddots  &amp;amp; {0      } \\&#xA;{0   } &amp;amp; \ldots &amp;amp; {0   } &amp;amp; {a_{n-k}} &amp;amp; {0   } &amp;amp; \ldots &amp;amp; {0    } &amp;amp; {b_n    } \\&#xA;\end{bmatrix}&#xA;\cdot&#xA;\begin{bmatrix}&#xA;   {x_ 1 }  \\&#xA;   {x_ 2 }  \\&#xA;   {x_ 3 }  \\&#xA;   {x_ 4 }  \\&#xA;   {x_ 5 }  \\&#xA;   {x_ 6 }  \\&#xA;   \vdots   \\&#xA;   {x_n }  \\&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;   {d_ 1 }  \\&#xA;   {d_ 2 }  \\&#xA;   {d_ 3 }  \\&#xA;   {d_ 4 }  \\&#xA;   {d_ 5 }  \\&#xA;   {d_ 6 }  \\&#xA;   \vdots   \\&#xA;   {d_n }  \\&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;nofollow&quot;&gt;TDMA algorithm&lt;/a&gt; for solving non-symmetric tridiagonal systems, but I haven't come across an existing implementation that allows the upper and lower diagonals to be arbitrarily offset from the main diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, I'm wondering whether it would be possible to take advantage of the symmetry of the matrix to solve the system faster than with standard TDMA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please excuse the naivety of my question!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here's a bit more information about $M$:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$M$ is the Hessian matrix for a large nonlinear system I'm trying to solve via an interior point method&lt;/li&gt;&#xA;&lt;li&gt;It is always positive definite&lt;/li&gt;&#xA;&lt;li&gt;It is constructed like this: $ M = B^{T} \cdot A \cdot B + C$, where $A$ and $C$ are $(n-k, n-k)$ and $(n, n)$ diagonal matrices respectively, and $B$ is an $(n-k, n)$ upper bidiagonal matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't think there's much more I can tell you without getting into to much depth, but if you're interested I'm trying to implement the algorithm described in &lt;a href=&quot;http://dx.doi.org/doi:10.1152/jn.01073.2009&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7993" LastEditorUserId="7993" LastEditDate="2014-04-19T10:36:52.157" LastActivityDate="2017-04-30T22:22:28.293" Title="Efficient solver for a symmetric tridiagonal system where the upper/lower diagonals are offset" Tags="|linear-algebra|algorithms|matrix|linear-solver|sparse-matrix|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11415" PostTypeId="2" ParentId="11414" CreationDate="2014-04-19T02:19:58.853" Score="2" Body="&lt;p&gt;Unfortunately, your matrix isn't tridiagonal in the conventional sense and it can't be turned into a tridiagonal matrix by row/column permutations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One conventional way of dealing with such a system would be to treat the matrix as a banded matrix and use a factorization routine designed to handle banded matrices.  Another approach would be to use a general purpose sparse matrix factorization routine.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-04-19T02:19:58.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11416" PostTypeId="2" ParentId="11414" CreationDate="2014-04-19T03:35:50.070" Score="0" Body="&lt;p&gt;One posibility I can think of is by solving many small $2 \times 2$ matrices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{bmatrix}&#xA;{b_1} &amp;amp; {a_1}\\&#xA;{a_1} &amp;amp; {b_k}\\&#xA;\end{bmatrix}&#xA;\cdot&#xA;\begin{bmatrix}&#xA;{x_1}\\&#xA;{x_k}\\&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;{y_1}\\&#xA;{y_k}\\&#xA;\end{bmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all &lt;em&gt;i&lt;/em&gt; solve:&lt;br&gt;&#xA;$\begin{bmatrix}&#xA;{b_i} &amp;amp; {a_i}\\&#xA;{a_i} &amp;amp; {b_{i+k-1}}\\&#xA;\end{bmatrix}&#xA;\cdot&#xA;\begin{bmatrix}&#xA;{x_i}\\&#xA;{x_{i+k-1}}\\&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;{y_i}\\&#xA;{y_{i+k-1}}\\&#xA;\end{bmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you can make some use of the fact that they are symmetric.&lt;/p&gt;&#xA;" OwnerUserId="7871" LastActivityDate="2014-04-19T03:35:50.070" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11417" PostTypeId="2" ParentId="11414" CreationDate="2014-04-19T10:06:06.370" Score="3" Body="&lt;p&gt;After a bit more thought, I think it's actually fairly simple to modify TDMA to deal with arbitrary diagonal offsets:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forward sweep:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;c'_i &amp;amp; =&#xA;\begin{cases}&#xA;  \cfrac{c_i}{b_i}                         &amp;amp; ; &amp;amp; i=1,2,\dots,k \\&#xA;  \cfrac{c_i}{b_i-c'_{i-k} a_i}            &amp;amp; ; &amp;amp; i=k+1,k+2,\dots,n-k&#xA;\end{cases} \\&#xA;\\&#xA;d'_i &amp;amp;=&#xA;\begin{cases}&#xA;  \cfrac{d_i}{b_i}                         &amp;amp; ; &amp;amp; i=1,2,\dots,k \\&#xA;  \cfrac{d_i-d'_{i-k}a_i}{b_i-c'_{i-k}a_i} &amp;amp; ; &amp;amp; i=k+1,k+2,\dots,n. \\&#xA;\end{cases}&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back substitution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;x_n &amp;amp;= d'_n\\&#xA;x_i &amp;amp;= d'_i-c'_ix_{i+k} &amp;amp; ; {}&amp;amp; i=n-1,n-2,\dots,1.&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've written a quick implementation in Cython:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;cimport numpy as cnp&#xA;&#xA;# NB: `b` and `d` are modified in place&#xA;cpdef TDMA_offset(const double[:] a, double[:] b, const double[:] c, double[:] d):&#xA;    cdef size_t i, k, m, n&#xA;    m = a.shape[0]&#xA;    n = b.shape[0]&#xA;    k = n - m&#xA;    # Forward sweep&#xA;    for i in range(m):&#xA;        d[i+k] -= d[i]*a[i]/b[i]&#xA;        b[i+k] -= c[i]*a[i]/b[i]&#xA;    # Back substitution&#xA;    for i in range(m-1, -1, -1):&#xA;        d[i] -= d[i+k]*c[i]/b[i+k]&#xA;    for i in range(n):&#xA;        d[i] /= b[i]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This approach still doesn't make use of the fact that $M$ is symmetric, and a parallel implementation would also be very nice.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Following the comment by Brian Borchers, I made a simple Python test script:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;from cymodules import _solvers&#xA;from scipy import sparse&#xA;&#xA;def test_trisolve(n=10, k=5):&#xA;    a = np.random.randn(n - k)&#xA;    b = np.random.randn(n)&#xA;    c = np.random.randn(n - k)&#xA;    x = np.random.randn(n)&#xA;    M = sparse.diags((a, b, c), (-k, 0, k), format='csc')&#xA;    d = M.dot(x)&#xA;&#xA;    # solve for x in place &#xA;    x_hat = d.copy()&#xA;    _solvers.TDMA_offset(a, b, c, x_hat)&#xA;&#xA;    print &quot;||x - x_hat|| = &quot;, np.linalg.norm(x - x_hat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've tested my solver for a variety of values of $n$ and $k$, including cases where $k \geq (n - 1) / 2$:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;test_trisolve(n=10, k=3)&#xA;# ||x - x_hat|| =  1.67369803253e-15&#xA;&#xA;test_trisolve(n=10, k=6)&#xA;# ||x - x_hat|| =  6.85231143806e-16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It seems to be working OK.&lt;/p&gt;&#xA;" OwnerUserId="7993" LastEditorUserId="7993" LastEditDate="2017-04-27T19:23:59.750" LastActivityDate="2017-04-27T19:23:59.750" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="11418" PostTypeId="1" CreationDate="2014-04-19T21:02:20.400" Score="2" ViewCount="349" Body="&lt;p&gt;I have to minimize a linear function with respect to variables u which take values [0,1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of variables can exceed 10,000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are thousands of linear inequality constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a solution which is good but does not have to be optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any heuristic approaches for doing this ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example I could allow 0 &amp;lt;= u &amp;lt;= 1 and then use an LP solver. I then iterate by pushing the values of u to 0 or 1 depending on their optimal value from the LP solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to do this but if this has been looked at then I would like to know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using CPLEX.&lt;/p&gt;&#xA;" OwnerUserId="7997" LastActivityDate="2014-04-19T22:59:36.647" Title="I have to solve a large binary programming task. Should I avoid branch and bound?" Tags="|optimization|linear-programming|constrained-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11419" PostTypeId="2" ParentId="11418" CreationDate="2014-04-19T22:59:36.647" Score="4" Body="&lt;p&gt;10,000 variables is a lot for an integer programming problem, but everything depends on the details of your particular problem.  With the information provided, there's really no way for us to tell you in advance how easy or hard it might be to solve this problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice would be to try CPLEX's integer programming solver on this problem with its default settings and observe how it behaves.  You might be pleasently surprised by a quick solution that is proven optimal.  If not, you might still be able to get a solution with a good bound on the objective value gap between it and optimal.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If CPLEX runs for many hours without finding an integer solution then watch out- you've got a problem where even finding an integer solution is extremely hard.  If CPLEX runs for a long time and searches many nodes in the branch and bound tree without noticeably reducing the gap between the best known solution and the bound, then the problem might just be too difficult to solve to optimality.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-04-19T22:59:36.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11420" PostTypeId="1" CreationDate="2014-04-20T19:48:02.203" Score="2" ViewCount="464" Body="&lt;p&gt;In Latent Semantic Analysis one uses the SVD to perform a dimensional reduction of the term-document matrix, via the Eckart-Young theorem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the rank $k$ approximation obtained by E-Y is proven to be the &lt;strong&gt;best&lt;/strong&gt; approximation in the Euclidean norm as well as in the Frobenius norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole literature I'm reading about LSA says something like: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;For the purposes of LSA, we consider the best approximation &lt;strong&gt;in the Frobenius norm&lt;/strong&gt;, provided by Eckart-Young.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But nobody says &lt;em&gt;why&lt;/em&gt;. I mean, why not in the Euclidean norm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7893" LastActivityDate="2018-03-05T21:52:56.647" Title="LSA, SVD and the Frobenius norm" Tags="|linear-algebra|matrix|numerical-analysis|svd|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11422" PostTypeId="2" ParentId="11420" CreationDate="2014-04-21T03:10:24.427" Score="1" Body="&lt;p&gt;There is no such thing as the Euclidean norm of a matrix. It is only defined for vectors in ${\mathbb R}^n$. You need to decide whether you want to consider a matrix as an element of the vector space ${\mathbb R}^{n \times m}$, in which case the Frobenius norm is simply the easiest one to compute. Alternatively, you could consider a matrix as an operator ${\mathbb R}^m \rightarrow {\mathbb R}^n$ in which case the $l_2$ norm is very awkward to compute (it is the largest singular value) and one could go with the $l_1$ or $l_\infty$ norms which are the maximal sums of absolute values of matrix elements over rows or columns. In the end, the Frobenius norm is typically just the simplest norm to compute if you don't think of the matrix as an operator.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-04-21T03:10:24.427" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11423" PostTypeId="2" ParentId="11372" CreationDate="2014-04-21T03:36:16.083" Score="4" Body="&lt;p&gt;You can do this with any balanced binary search tree data structure by additionally maintaining the total weight below each node.  To randomly sample, compute a uniform random number between $0$ and the weight of the root node, and traverse down through the tree until you find the leaf whose range contains the random number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, while maintaining the total weight below a given node is straightforward, it does mean you'll likely have to implement your own balanced binary tree structure.  Copying and modifying an existing data structure might be fastest in terms of programmer time; you could even use libstdc++'s red black tree implementation.&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2014-04-21T03:36:16.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11425" PostTypeId="2" ParentId="2868" CreationDate="2014-04-21T09:43:10.153" Score="3" Body="&lt;p&gt;There is also the &lt;a href=&quot;http://link.springer.com/journal/10915&quot; rel=&quot;nofollow&quot;&gt;Journal of Scientific Computing&lt;/a&gt;, which overlaps strongly with JCP in its area of focus.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-04-21T09:43:10.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11426" PostTypeId="2" ParentId="11372" CreationDate="2014-04-21T10:24:18.250" Score="0" Body="&lt;p&gt;You say that you expect to have a lot of items only once, and a few many times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While in general a self-balancing search tree like the one Geoffrey suggested would be fastest, there is an easier to implement alternative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply keep the $k$ items that only occur once in a vector, like thus spake a.k. suggested, and the other $m$ unique items in an unordered map. You then keep track of the total number of items in each structure, and when you have to sample the map, you accept the $O(m)$ time. However as you expect $k &amp;gt;&amp;gt; m$, this is likely not a problem.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-04-21T10:24:18.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11428" PostTypeId="1" CreationDate="2014-04-21T17:15:22.457" Score="2" ViewCount="176" Body="&lt;p&gt;I hope you can help me out with this. I have to find the solution x to an inverse system &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x=A^{-1}b&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This inverse problem is basically a least square problem with a rank-1 update. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x=[uv^{T}+H^{H}H]^{-1}H^{H}b&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two objectives, one can be achieved with the least square problem and the other with the rank-1 update. I need to find the best tradeoff between the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The least square solution (without the rank-1 update) needs to be regularized, so the smallest singular values of the matrix $[H^{H}H]$ would need to be replaced by zeroes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, I should introduce a parameter (real positive scalar) in front of the rank-1 update $uv^{T}$ and hand-tune it until the solution leads to the best compromise between the two objectives. Nevertheless, I am thinking: could I just do a singular value decomposition of the matrix $[H^{H}H]$ and replace one of my useless elements (one that will be replaced by zero after regularization) by my rank-1 update?* Wouldn't that save me from having to include a parameter that has to be tuned? Once I have my new matrix A, I think the solution could be easily regularized by applying Selective Singular Value Decomposition...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The reason why I think this might be done is that there is only one non-zero singular value in $uv^{T}$ and, at the same time, some (the smallest) singular values of $[H^{H}H]$ will be rejected, so why not create a new matrix A such that one of its svd components is the (non-zero) $svd(uv^{T})$?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="8010" LastEditorUserId="8010" LastEditDate="2014-04-22T06:53:52.903" LastActivityDate="2014-04-22T06:53:52.903" Title="Inverse problem with a rank-1 update" Tags="|linear-algebra|optimization|constrained-optimization|inverse-problem|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11429" PostTypeId="1" CreationDate="2014-04-21T21:07:45.377" Score="1" ViewCount="338" Body="&lt;p&gt;Are there optimizers where it is possible to specify ordinal ranking of parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that I have a function of three parameters $f(\theta_1, \theta_2, \theta_3)$. Are there optimizers such that I can specify&#xA;$$&#xA;{\arg\min}_{\theta_1 &amp;gt; \theta_2 &amp;gt;\theta_3}f(\theta_1, \theta_2, \theta_3)&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that $f$ is smooth ($n$-th order differentiable in each of the parameters).&lt;/p&gt;&#xA;" OwnerUserId="1969" LastActivityDate="2014-05-25T22:46:12.047" Title="Optimization with order constraints on parameters" Tags="|optimization|constrained-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11430" PostTypeId="1" CreationDate="2014-04-21T21:33:13.570" Score="1" ViewCount="45" Body="&lt;p&gt;In numerical methods for ODE's, the model problem&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y' = cy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where c is complex is regarded as sufficient in performing error analysis for different methods in solving ODE's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it sufficient, and why? In my opinion, it overlooks many potential functions that can have different error forms.&lt;/p&gt;&#xA;" OwnerUserId="8012" LastActivityDate="2014-04-21T21:33:13.570" Title="Error analysis and the Model Problem" Tags="|numerical-analysis|ode|stability|" AnswerCount="0" CommentCount="1" ClosedDate="2014-04-30T01:12:20.667" ContentLicense="CC BY-SA 3.0" />
  <row Id="11431" PostTypeId="1" CreationDate="2014-04-21T21:59:34.580" Score="1" ViewCount="480" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Minimize  $\sum\limits_{i=1}^{10} L_{i}x_{i}$  subject   to  $Af=p$ and $x \geq|f|$. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here  $L$, $p$  and $A$ are known, $f$ and $x$ unknown.  $Af=p$ is underdetermined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $x$ is minimized when $|f|$ is minimized, I rewrote the problem putting $x  = |f|$:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Minimize $\sum\limits_{i=1}^{10} L_{i}|f_{i}|$  subject  to $ Af=p$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Problem is, how do you solve it with the absolute value. I am using Matlab with the linprog function, but I'm unsure how to put in the parameters. &lt;/p&gt;&#xA;" OwnerUserId="8013" LastEditorDisplayName="user10435" LastEditDate="2015-03-14T04:12:01.160" LastActivityDate="2015-03-14T04:12:01.160" Title="Optimization of a sum of absolute values of vector entries" Tags="|optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11432" PostTypeId="2" ParentId="11431" CreationDate="2014-04-21T23:04:05.390" Score="6" Body="&lt;p&gt;Assuming that the weights $L_{i}$ are nonnegative, you can formulate your problem as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \sum_{i=1}^{10} L_{i}x_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Af=p$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x \geq f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x \geq -f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraints ensure that $x \geq |f|$, and the nonnegative weights insure that $ x  = |f|$.  This is a standard problem transformation discussed in many textbooks on linear programming and convex optimization.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2014-04-22T01:15:15.330" LastActivityDate="2014-04-22T01:15:15.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11433" PostTypeId="1" AcceptedAnswerId="11434" CreationDate="2014-04-22T02:42:40.257" Score="1" ViewCount="104" Body="&lt;p&gt;I have the following equation I came across which was solved using least squares&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x = \sum_{n=1}^{N} A_{n} y_{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $x$ is a $m \times p$ matrix and $y$ would be of size $m \times p$ as well ,where $p=N$.Both $x$ and $y$ are known. $A_n$ are the weights to be computed for every $y_{n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would one do a least squares approximation to solve for $A_n$ so that there would be only 1 value of $A_n$ for every column in $y$? Usually least squares  (in MATLAB) requires a matrix and a column vector, but in this case there are 2 matrices. Is there any standard technique I am missing here?&lt;/p&gt;&#xA;" OwnerUserId="4299" LastActivityDate="2014-04-22T03:38:48.313" Title="Least squares fitting" Tags="|linear-algebra|linear-solver|least-squares|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11434" PostTypeId="2" ParentId="11433" CreationDate="2014-04-22T03:38:48.313" Score="2" Body="&lt;p&gt;The notation here is very confusing, and that's probably the main cause of your difficulty in formulating this as a linear least squares problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introduce the notation $\mbox{vec}(x)$ for the $mp$ by $1$ vector formed by taking the columns of the $m$ by $p$ matrix $x$ consecutively.  e.g. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{vec}\left( \left[&#xA;\begin{array}{ccc}&#xA;1 &amp;amp; 2 &amp;amp; 3 \\&#xA;4 &amp;amp; 5 &amp;amp; 6 \\&#xA;\end{array}&#xA;\right]&#xA;\right)=&#xA;\left[&#xA;\begin{array}{c}&#xA;1 \\&#xA;4 \\&#xA;2 \\&#xA;5 \\&#xA;3 \\&#xA;6 \\&#xA;\end{array}&#xA;\right].&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, two matrix $A$ and $B$ are equal if and only if $\mbox{vec}(A)=\mbox{vec}(B)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, your model can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{vec}(x) = \sum_{i=1}^{N} A_{i} \mbox{vec}(y_{i})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $H$ be the $mp$ by $N$ matrix whose columns are $\mbox{vec}(y_{i})$, for $i=1, 2, \ldots, N$.  Recognizing that this equation involves a matrix multiplication, we can write the model as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{vec}(x)=HA$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H$ is a known matrix of size $mp$ by $N$, $\mbox{vec}(x)$ is a known vector of length $mp$, and $A$ is an unknown vector of length $N$.  This is a conventional linear least squares problem that can be solved with standard software.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-04-22T03:38:48.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11435" PostTypeId="1" CreationDate="2014-04-22T05:29:33.370" Score="1" ViewCount="730" Body="&lt;p&gt;Before I know how to do tucker decomposition, I mistakenly thought the core tensor is only from combining the singular value matrices of the flattening matrices. Yes I know it is not now.&#xA;For the tucker decomposition method for a 3 way data's tensor decomposition, a core tensor is calculated using the original tensor(e.g. 3D tensor) and its 3 flattening matrices' left singular vectors. I am wondering if there is any relation existing between the singular values of the flattening matrices and the core tensor somehow? Just out of intuition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QVSzy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the materials about Tucker decomposition for tensors:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Tucker_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Tucker_decomposition&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.153.2059&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensor Decompositions and Applications&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7969" LastEditorUserId="7969" LastEditDate="2014-04-22T11:55:00.240" LastActivityDate="2015-03-15T11:08:18.373" Title="Any relation between the singular values of each flattening matrices and the core tensor out of Tucker decomposition?" Tags="|svd|tensor|tensor-decomposition|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11436" PostTypeId="2" ParentId="11410" CreationDate="2014-04-22T06:45:53.773" Score="2" Body="&lt;p&gt;No, any penalty will not do the trick. However, this is a well researched area, and goes under the name exact penalty. Basically, if the penalty is larger than a value which depends on the dual of the original problem, the solution from the model with the penalty on the slacks will coincide with the solution from the original model, when possible. I think this can be found in, e.g., &quot;Fletcher - Practical Methods of Optimization&quot;. Otherwise, just do a search for exact penalty for slacks etc.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2014-04-22T15:18:32.917" LastActivityDate="2014-04-22T15:18:32.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11437" PostTypeId="1" AcceptedAnswerId="11438" CreationDate="2014-04-22T09:03:44.573" Score="8" ViewCount="336" Body="&lt;p&gt;I want to publish the software package I've written for my graduate work along with a paper describing the package. So far I've been considering liberal licenses such as BSD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am now aware that publishing your code may involve an agreement to the journal's licensing terms (cmp. e.g. &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2832/for-software-submitted-to-acm-toms-how-does-the-acm-software-license-agreement&quot;&gt;this question&lt;/a&gt;) or even transfer of copyright. All this is undesirable. Are there any obvious choices concerning the journal? My field is physics. Do you have experience founding a BSD-license style open source project and publishing an accompanying paper?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of &lt;a href=&quot;http://www.journals.elsevier.com/computer-physics-communications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer Physics Communications&lt;/a&gt; (unfortunately &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2868/alternatives-to-journal-of-computational-physics/&quot;&gt;published by Elsevier&lt;/a&gt;), cmp. &lt;a href=&quot;http://code.google.com/p/qutip/wiki/Citing&quot; rel=&quot;nofollow noreferrer&quot;&gt;QuTiP as an example&lt;/a&gt;. Thanks for your suggestions!&lt;/p&gt;&#xA;" OwnerUserId="1553" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-04-23T11:56:26.993" Title="Publishing a software package along with a paper: licensing" Tags="|publications|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11438" PostTypeId="2" ParentId="11437" CreationDate="2014-04-22T14:42:34.343" Score="10" Body="&lt;p&gt;The best way to do this to avoid these issues is to publish a paper about what your software package does and the problems it solves and cite your Github repo or website or whatever for access to the software. That way you don't need to give up your copyright on your code, and you can BSD/MIT/GPL it to your heart's content. &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2014-04-23T11:56:26.993" LastActivityDate="2014-04-23T11:56:26.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11439" PostTypeId="2" ParentId="11390" CreationDate="2014-04-22T15:18:19.523" Score="0" Body="&lt;p&gt;The ScaLAPACK library is designed for dense matrix operations on &lt;strong&gt;distributed memory&lt;/strong&gt; machines. It will still work on a shared-memory machine though, provided that you have MPI installed (installation requires mpicc and mpif77 compilers on linux). I haven't used it before, so I can't attest to its user-friendliness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/scalapack/&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/scalapack/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2014-04-22T15:18:19.523" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11440" PostTypeId="2" ParentId="11390" CreationDate="2014-04-22T15:41:39.783" Score="5" Body="&lt;p&gt;For dense distributed memory linear algebra, you can't do better than &lt;a href=&quot;http://libelemental.org/&quot;&gt;Elemental&lt;/a&gt; these days. Although, it does not currently handle sparse matrices, I believe. Dense-sparse operations will probably require you to do some custom coding.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-04-22T15:41:39.783" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11441" PostTypeId="2" ParentId="11286" CreationDate="2014-04-22T18:03:54.583" Score="1" Body="&lt;p&gt;Here's what I would try:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, to count bits in an &lt;code&gt;unsigned int z&lt;/code&gt;, i.e. &quot;weigh&quot; it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = table[ (z &amp;amp; 0xffff) ] + table[ (z &amp;gt;&amp;gt; 16) ];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;unsigned char table[1&amp;lt;&amp;lt;16];&lt;/code&gt; is filled in ahead of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to see if there was a way to simplify the outer loop, but XOR does not distribute over AND.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastActivityDate="2014-04-22T18:03:54.583" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11442" PostTypeId="1" AcceptedAnswerId="11446" CreationDate="2014-04-22T19:06:29.717" Score="3" ViewCount="1449" Body="&lt;p&gt;For a hobby project I need to solve a series of quadratic programming problems each with&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;about 500 variables&lt;/li&gt;&#xA;&lt;li&gt;about 1000 constraints, each of the form $x_i-x_j\le c_{ij}$&lt;/li&gt;&#xA;&lt;li&gt;the objective function is the sum of about 300 terms of the form $w_{ij}^2(x_i-x_j+d_{ij})^2$&lt;/li&gt;&#xA;&lt;li&gt;about a third of the variables are ignored by the objective function, and are just there to help organize the constraints.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I imagine there must be off-the-shelf solvers that can do this, but I'm quite new to this area, and the list of solvers at &lt;a href=&quot;https://en.wikipedia.org/wiki/Quadratic_programming&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt; is not really helpful in terms of selecting which one to look into. Could someone help point me towards a good choice, please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for something that is free and with not too much of a learning curve just to put a problem into it and get a solution. Needing to write some glue code in C/C++, Java, Perl or the like is not a showstopper, however.&lt;/p&gt;&#xA;" OwnerUserId="8017" LastActivityDate="2014-04-22T23:53:17.737" Title="Sparse quadratic programming solver" Tags="|software|quadratic-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11443" PostTypeId="2" ParentId="533" CreationDate="2014-04-22T21:43:45.960" Score="7" Body="&lt;p&gt;To specifically answer the question: The main shortcoming of Density Functional Theory is that even though it is a formally exact reformulation of quantum theory, in the current state of the theory, &lt;strong&gt;approximations are required for the Exchange-Correlation energy functional&lt;/strong&gt;. All the Density-Functional approximations that we have so far fail to exactly reproduce the contributions from different phenomena to the Exchange- and Correlation-Energies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As discussed by Cohen, Mori-Sánchez, and Yang in a paper titled &quot;Insights into Current Limitations of Density Functional Theory&quot; in the journal Science 2008 most of the weaknesses can be traced back to two main errors of standard density-functionals: The delocalization error and the static correlation error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One has to read the paper to understand the details, but in a hand-wavy explanation what they say is that &lt;strong&gt;when using DFT, the electron density (or electron cloud) is artificially spread-out due to an incorrect behavior of the standard functionals&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem has its root in the fact that when using DFT &lt;strong&gt;even if you have only one electron, the density of that electron (a non-local object) interacts with the electron itself (a local object) producing an artificial repulsion of the electron caused by itself.&lt;/strong&gt; An analogous situation happens with spin-spin interaction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an artifact on the formulation of the exchange-energy functional (exact) that the correlation-energy functional (approximated) cannot correct in any of the functionals that we have prepared so far, including the fanciest ones, i.e. the &quot;Minnesota&quot; family.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is reflected in the underestimation of the barriers of chemical reactions, the band gaps of materials, the energies of dissociating molecular ions, and charge transfer excitation energies. Density-Functional approximations also overestimate the binding energies of charge transfer complexes and the response to an electric field in molecules and materials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Another practical issue in DFT is that it is not variational&lt;/strong&gt;, which is a fancy terminology to say that if you use one of the simplest functionals and you get a some answer, you are not guaranteed to improve it by using a more complicated functional. Choosing a functional is a matter of experience and sometimes, luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Even though all of this may sound really bad, it surprising how DFT works much better and/or faster than other computational quantum methods modelling many different properties important for physics, chemistry and materials sciences.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details, I would also recommend the book by Parr and Yang, Density Functional Theory of Atoms and Molecules.&lt;/p&gt;&#xA;" OwnerUserId="8020" LastActivityDate="2014-04-22T21:43:45.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11444" PostTypeId="1" CreationDate="2014-04-22T23:10:47.163" Score="1" ViewCount="90" Body="&lt;p&gt;I want to calculate the &lt;a href=&quot;http://en.wikipedia.org/wiki/Wigner_quasi-probability_distribution&quot; rel=&quot;nofollow&quot;&gt;Wigner quasiprobability distribution function&lt;/a&gt; of a particular wavefunction. The definition suggests a few straightforward ways of calculating it, but I was wondering if there's something more efficient than the naive approaches of either directly evaluating the integral or taking the fft of the autocorrelation. &lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2014-04-22T23:10:47.163" Title="What's the most efficient way to calculate the Wiger quasiprobability distribution?" Tags="|algorithms|efficiency|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11445" PostTypeId="2" ParentId="11372" CreationDate="2014-04-22T23:43:25.893" Score="0" Body="&lt;p&gt;Suppose you have I unique items, each one with multiplicity m_i. You can create a hash table that maps i -&gt; (item, m_i). When you want to sample, construct a partition of [0,1] (i.e. a sorted array of numbers from 0 to 1) where each interval corresponds to an item, and its width is proportional to the multiplicity E.g. if you have 1 cat, 1 dog, 2 frogs, and 3 rats, the array will be [0, 1/7, 2/7, 4/7, 1]. Then, draw from the uniform distribution, and use binary search to find the interval it falls in; this is the sampled item.&lt;/p&gt;&#xA;" OwnerUserId="1665" LastActivityDate="2014-04-22T23:43:25.893" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11446" PostTypeId="2" ParentId="11442" CreationDate="2014-04-22T23:53:17.737" Score="3" Body="&lt;p&gt;cvxopt (for python) does QP, and can take advantage of sparsity (you can provide a custom KKT solver specific to your problem). An example of a custom solver is &lt;a href=&quot;https://groups.google.com/forum/#!topic/cvxopt/W8kd3LHPwwA&quot; rel=&quot;nofollow&quot;&gt;https://groups.google.com/forum/#!topic/cvxopt/W8kd3LHPwwA&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1665" LastActivityDate="2014-04-22T23:53:17.737" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11447" PostTypeId="1" CreationDate="2014-04-23T07:30:37.673" Score="5" ViewCount="159" Body="&lt;p&gt;I'm solving a system of nonlinear ODEs that take the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q_{nm} \ddot{y}_m + S_{nkl}\dot{y}_k\dot{y}_l +V_n = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where Einstein summation is assumed, $y_i$ are the dependent (complex) variables,  $Q_{nm}$,$S_{nkl}$ and $V_n$ are all low order polynomials in the dependent variables, $n =1,...,N$, with $N$ the $resolution$ of the model. These equations, together with the initial conditions $y_i(0)$ and $\dot{y}_i(0)$ completely specify the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scheme is solved using (the Fortran interface to) &lt;a href=&quot;http://computation.llnl.gov/casc/sundials/main.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SUNDIALS&lt;/a&gt;. In particular, because of the nontrivial coefficient $Q$, an implicit differential algrebraic solver is used (IDA) to solve the equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have successfully implemented these equations and test cases yield coherent results. The computation time is constrained by $N$, the resolution of our model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, no matter how precisely I specify my initial conditions for certain physically relevant scenarios, I find a spurious growth in my system, that leads to the equations becoming stiff, and eventually Q is no longer invertible and the integration fails. The problem is (temporarily) alleviated by increasing $N$, but no matter how high I can reasonably make $N$, the instability still appears. So I would like to see if anyone has any recommendations on a smoothing operation, or perhaps adding some type of viscosity, to attenuate this instability, while keeping the results physically relevant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example of this, I put IC that correspond to equations with known solution. The physical space solutions are related to a summation of the fourier series, with coefficients $y_i(t)$. At time t=0, I see that the difference between the numerical solution to my system, and what I expect is good (see fig 1; note this is for $N=16$!),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tdQ0U.png&quot; alt=&quot;Difference between calculated and expected at t=0&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and is order $10^{-8}$, which is the order of my tolerances in the integration. Note, the solutions are order $10^{-1}$. Now, at time t=0.1, the difference between what I expect and the calculated solution is no longer so negligible, namely we see that the difference is 5 order of magnitude larger! This increases at this exponential rate until the system eventually becomes stiff and stops integrating. I am not sure if this is some kind of instability due to the physics, or the numerics, but in either case I want to get ride of it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/AxOyy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively, it seems like some kind of smoothing between integration steps could be useful, but this is has major drawbacks. The IDA uses an internal Jacobian, so it would not be a good idea to impulsively change the solutions between integration steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the most attractive technique would be to add some kind of $viscosity$ to the system, to dampen or get rid of these growing instabilities. The part of the solution that is unstable is growing fastest, so adding something to these equations that can pick this out would be ideal. Exactly how to do this is unclear to me, so any suggestions or references to similar problems would be greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nick &lt;/p&gt;&#xA;" OwnerUserId="8021" LastActivityDate="2014-07-27T14:40:35.567" Title="Scheme to alleviate (numerical?) instability in system of coupled nonlinear ODEs" Tags="|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11448" PostTypeId="1" CreationDate="2014-04-23T08:12:07.130" Score="3" ViewCount="391" Body="&lt;p&gt;I have 10000 variables (each of them is binary), vector of positive coefficients and a matrix $A$ ($10000\times10000$), if $A_{ij}=1$, then $i$th and $j$th variables can take 1 simultaneously, if it is 0, then it is not possible. The goal is to maximize their weighted sum. What algorithms and software could I use to solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array}{l}&#xA;\max F\left( x_{1}..x_{m} \right)=\sum\limits_{i=1}^m {b_{i}x_{i}} &#xA; \end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array}{l}&#xA; x_{i}\in \left\{ 0,1 \right\},\, \, i=1..m \\ &#xA; x_{i}x_{j}\le A_{ij}\,, i,j=1..m \\ &#xA; A_{ij}\in \left\{ 0,1 \right\},\, \, i,j=1..m \\ &#xA; \end{array}&lt;/p&gt;&#xA;" OwnerUserId="8023" LastEditorUserId="9667" LastEditDate="2015-03-23T08:48:28.960" LastActivityDate="2015-03-26T20:39:13.227" Title="Large binary programming problem" Tags="|optimization|algorithms|linear-solver|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11449" PostTypeId="1" CreationDate="2014-04-23T08:20:45.947" Score="1" ViewCount="64" Body="&lt;p&gt;I have a 2D distribution of mass on a sphere given as a matrix of masses in latitude-longitude grid cells. I need these masses projected to another grid &#xA;on the same sphere with different location of pole and different cell sizes, again as masses in grid cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The projection has to be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;fast&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;monotonous, i.e. not create substantial artificial maxima/minima of concentration (mass/cell area)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;strictly mass-conservative (to numerical accuracy)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I could not find any ready made tool for that, though &#xA;I believe, there should be something available (C, fortran or whatever).&#xA;Any clue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8022" LastActivityDate="2014-04-23T08:20:45.947" Title="Mass-conservative reprojection (on a sphere)" Tags="|interpolation|projection|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11451" PostTypeId="2" ParentId="5549" CreationDate="2014-04-23T14:02:28.017" Score="5" Body="&lt;p&gt;I don't know how this benchmark was done, but probably with floating point numbers, that in Python default to doubles. The sizes correspond, respectively, to $4$ and $16 kB$. Those are reasonable values for L1 and L2 cache sizes of a (a bit old) AMD CPU. To be certain, I did my own benchmark:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def timeit(size):&#xA;     t0 = time.time()&#xA;     for _ in xrange(10):&#xA;         np.random.random(size)&#xA;     return time.time() - t0&#xA;&#xA;sizes = np.logspace(1, 6, 40)&#xA;times = [timeit(s) for s in sizes]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are a few things wrong with this benchmark, for a start, I am not disabling garbage collection and I am taking the sum, not the best time, but bear with me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time it takes is quite proportional to the size of the array, but there is a change in the slope around size $8000$. This is an array of $64 kB$, that is the size of the L1 cache of an i5 (my computer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should one worry about the cache size? As a general rule, I say no. Optimizing for it in Python means making the code much more complicated, for dubious performance gains. Don't forget that Python objects add several overheads that are difficult to track and predict. I can only think of two cases where this is a relevant factor:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Basic operations on large arrays (like evaluate a polynomial), limited by memory bandwidth. Use &lt;a href=&quot;https://github.com/pydata/numexpr&quot;&gt;Numexpr&lt;/a&gt; or (if the data is much bigger) &lt;a href=&quot;http://pytables.org&quot;&gt;Pytables&lt;/a&gt;. They are optimised to take the cache size into account amongst other optimisations.&lt;/li&gt;&#xA;&lt;li&gt;Performance critical code: if you want to squeeze every microsecond, you should not be using Python in the first place. Writing vectorized &lt;a href=&quot;http://docs.cython.org/src/tutorial/numpy.html&quot;&gt;Cython&lt;/a&gt; and leaving the compiler do what it does best is probably the painless way to go.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the comments, Evert mentioned CArray. Note that, even working, the development has stopped and it has been abandoned as a standalone project. The functionality will be included in &lt;a href=&quot;http://blaze.pydata.org/docs/index.html&quot;&gt;Blaze&lt;/a&gt; instead, an ongoing project to make a &quot;new generation Numpy&quot;.&lt;/p&gt;&#xA;" OwnerUserId="7759" LastActivityDate="2014-04-23T14:02:28.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11452" PostTypeId="1" CreationDate="2014-04-23T14:56:55.407" Score="9" ViewCount="265" Body="&lt;p&gt;I have a project where I need to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Quadratic_integer&quot;&gt;quadratic fields&lt;/a&gt;   Specifically numbers of the form $a + b \sqrt{-3}$ with $a,b \in \mathbb{Q}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example here are the prime numbers in &lt;a href=&quot;https://en.wikipedia.org/wiki/Eisenstein_integer&quot;&gt;Eisenstein integers&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/1/17/EisensteinPrimes-01.svg&quot; width=&quot;200&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I do not want to use sage.&lt;/em&gt;  I would like to write my own data type to incorporate &lt;code&gt;numpy&lt;/code&gt;.  PARI would be useful - but it's not compatible with Python.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Addition for these objects is pretty clear $$(a_1 + b_1 \sqrt{-3}) + (a_2 + b_2 \sqrt{-3})  = (a_1 + a_2) + (b_1+b_2) \sqrt{-3}$$&lt;/li&gt;&#xA;&lt;li&gt;Multiplication is a little more delicate but we can hard code it as well&#xA;$$(a_1 + b_1 \sqrt{-3}) \times  (a_2 + b_2 \sqrt{-3}) &#xA;= (a_1 a_2 - 3 b_1 b_2) + (a_1 b_2 + a_2 b_1)\sqrt{-3}$$&lt;/li&gt;&#xA;&lt;li&gt;My datatype also needs to accommodate division.  For simplicity let's take the reciprocal:&#xA;$$ \frac{1}{a + b \sqrt{-3}} = \frac{a - b \sqrt{-3}}{a^2 + 3b^2}$$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there a natural matrix-based way to encode these operations, similar to how $\mathbb{C}$ can be written in terms of $2 \times 2 $ matrices? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left( \begin{array}{cc} a &amp;amp; b \\ -b &amp;amp; a \end{array}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe I will just hard-code the operations as triples with the three operations outlined above.  Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="7001" LastEditDate="2014-04-23T17:03:44.930" LastActivityDate="2014-04-23T18:12:14.797" Title="Representing Eisenstein numbers without floats" Tags="|linear-algebra|precision|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11453" PostTypeId="1" AcceptedAnswerId="11454" CreationDate="2014-04-23T15:29:59.873" Score="3" ViewCount="243" Body="&lt;p&gt;I'm trying to simulate a simple diffusion based on &lt;a href=&quot;http://en.wikipedia.org/wiki/Fick&amp;#39;s_laws_of_diffusion#Fick.27s_second_law&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fick's 2nd law&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from pylab import *&#xA;import numpy as np&#xA;gridpoints = 128&#xA;&#xA;def profile(x):&#xA;    range = 2.&#xA;    straggle = .1576&#xA;    dose = 1 &#xA;    return dose/(sqrt(2*pi)*straggle)*exp(-(x-range)**2/2/straggle**2)&#xA;&#xA;x = linspace(0,4,gridpoints)&#xA;nx = profile(x)&#xA;dx = x[1] - x[0] # use np.diff(x) if x is not uniform&#xA;dxdx = dx**2&#xA;&#xA;figure(figsize=(12,8))&#xA;&#xA;plot(x,nx)&#xA;timestep = 0.5&#xA;steps = 21&#xA;diffusion_coefficient = 0.002&#xA;for i in range(steps):&#xA;    coefficients = [-1.785714e-3, 2.539683e-2, -0.2e0, 1.6e0,&#xA;                    -2.847222e0,&#xA;                    1.6e0, -0.2e0, 2.539683e-2, -1.785714e-3]&#xA;    ccf = (np.convolve(nx, coefficients) / dxdx)[4:-4] # second order derivative&#xA;    nx = timestep*diffusion_coefficient*ccf + nx&#xA;    plot(x,nx)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/NYVg7.png&quot; alt=&quot;output with noise&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the first few time steps everything looks fine, but then I start to get high frequency noise, do to build-up from numerical errors which are amplified through the second derivative. Since it seems to be hard to increase the float precision I'm hoping that there is something else that I can do to suppress this? I already increased the number of points that are being used to construct the 2nd derivative.&lt;/p&gt;&#xA;" OwnerUserId="8026" LastActivityDate="2014-04-23T17:07:34.503" Title="High frequency noise at solving diffusion equation" Tags="|finite-element|python|numpy|diffusion|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11454" PostTypeId="2" ParentId="11453" CreationDate="2014-04-23T16:11:27.353" Score="8" Body="&lt;p&gt;Solving partial differential equations with explicit timestepping methods relies on meeting a certain &lt;a href=&quot;http://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&quot;&gt;CFL condition&lt;/a&gt; for stability. Since you are using a &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler_method&quot;&gt;forward Euler&lt;/a&gt; timestepping scheme, you must ensure that your timestep is small enough compared to the grid spacing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the heat equation (Fick's 2nd law is a heat equation), your CFL number $C$ must satisfy &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C= k\dfrac{\Delta t}{(\Delta x)^2}&amp;lt;A$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k$ is the diffusion coefficient and $A$ is some constant that is typically in the range (0,1] and depends on the specific equation and method used. This places a somewhat harsh constraint on your timestep as you increase the number of grid points (decreasing $\Delta x$) and is one of the major reasons implicit methods are used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this condition is necessary for &lt;em&gt;stability&lt;/em&gt;, meaning that the numerical solution will eventually blow up if this condition is not satisfied. Concerns about accuracy are another topic, but if the solution has blown up you're guaranteed to be inaccurate.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-04-23T17:07:34.503" LastActivityDate="2014-04-23T17:07:34.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11455" PostTypeId="2" ParentId="11452" CreationDate="2014-04-23T16:45:39.443" Score="10" Body="&lt;p&gt;For $a+b\sqrt{-3}$ you can use the representation&#xA;$$\begin{pmatrix}a&amp;amp;-3b\\b&amp;amp;a\end{pmatrix}$$&#xA;Addition works obviously. For multiplication, you can verify&#xA;$$\begin{pmatrix}a_1&amp;amp;-3b_1\\b_1&amp;amp;a_1\end{pmatrix}\begin{pmatrix}a_2&amp;amp;-3b_2\\b_2&amp;amp;a_2\end{pmatrix}&#xA;= \begin{pmatrix}a_1a_2-3b_1b_2&amp;amp;-3(a_1b_2+b_1a_2)\\a_1b_2+b_1a_2&amp;amp;a_1a_2-3b_1b_2\end{pmatrix}$$&#xA;which preserves the representation, thus we have a ring homomorphism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking the determinant of the matrix gives the (squared) norm $a^2+3b^2$, thus reciprocals correspond to inverse matrices, as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have already considered using &lt;em&gt;triples&lt;/em&gt;, by which I assume you'd use integers and a common denominator. That approach may be useful in the matrix representation as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;:&#xA;A general method for matrix representations uses the &lt;a href=&quot;http://en.wikipedia.org/wiki/Companion_matrix&quot;&gt;companion matrix&lt;/a&gt;.&#xA;For example, suppose you want to represent $a+b\omega$ instead where&#xA;$\omega=\exp(\frac{2\pi\mathrm{i}}{3})$, thus $\omega^2+\omega+1=0$.&#xA;The companion matrix of $\omega$ is $\begin{pmatrix}0&amp;amp;-1\\1&amp;amp;-1\end{pmatrix}$,&#xA;and this behaves in all its associated ring operations like $\omega$ itself.&#xA;Of course, $1$ can be represented as $\begin{pmatrix}1&amp;amp;0\\0&amp;amp;1\end{pmatrix}$;&#xA;therefore a matrix representation of $a+b\omega$ is&#xA;$$\begin{pmatrix}a&amp;amp;-b\\b&amp;amp;a-b\end{pmatrix}$$&#xA;You may want to verify that this is a ring homomorphism.&#xA;For addition, this is easy to see. For multiplication,&#xA;the corresponding formulae are now&#xA;$$\begin{align}&#xA;(a_1+b_1\omega)(a_2+b_2\omega)&amp;amp;=(a_1a_2-b_1b_2)+(a_1b_2+b_1a_2-b_1b_2)\omega&#xA;\\\begin{pmatrix}a_1&amp;amp;-b_1\\b_1&amp;amp;a_1-b_1\end{pmatrix}&#xA;\begin{pmatrix}a_2&amp;amp;-b_2\\b_2&amp;amp;a_2-b_2\end{pmatrix}&#xA;&amp;amp;= \begin{pmatrix}a_1a_2-b_1b_2&amp;amp;-(a_1b_2+b_1a_2-b_1b_2)\\&#xA;a_1b_2+b_1a_2-b_1b_2&amp;amp;a_1a_2-a_1b_2-b_1a_2\end{pmatrix}&#xA;\end{align}$$&lt;/p&gt;&#xA;" OwnerUserId="7001" LastEditorUserId="7001" LastEditDate="2014-04-23T18:12:14.797" LastActivityDate="2014-04-23T18:12:14.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11456" PostTypeId="2" ParentId="11452" CreationDate="2014-04-23T16:53:35.810" Score="2" Body="&lt;p&gt;I'm guessing you want exact rational arithmetic for everything, as floating point errors could make $1/z$ not in $\mathbb{Q}[\sqrt{-3}]$ even if $z$ is. For that, you might want to have a look at the &lt;a href=&quot;http://sympy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt; package; if you don't use their rational data type directly, it could serve as inspiration for your own hand-rolled version. You could then build your quadratic field type on top of whatever rational number type you choose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No matter how you represent the elements of your field, you can &lt;a href=&quot;https://docs.python.org/2/library/operator.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;overload operators in Python&lt;/a&gt; using &quot;magic methods&quot;. See also &lt;a href=&quot;https://stackoverflow.com/questions/5604713/creating-my-own-integer-object-in-python&quot;&gt;this SO post&lt;/a&gt; on creating your own numeric type in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think there would be that much more work coding a representation of an element of a quadratic field either as a 2 x 2 matrix of rational numbers or as a pair of rational numbers, since the arithmetic operations aren't that complicated. However, I suspect the second approach will be faster.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-04-23T16:53:35.810" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11457" PostTypeId="1" CreationDate="2014-04-23T19:19:24.413" Score="1" ViewCount="603" Body="&lt;p&gt;I'd like to distinguish different types of beers in my fridge using a Raspberry Pi. I saw a &lt;a href=&quot;https://learn.adafruit.com/raspberry-pi-face-recognition-treasure-box?view=all&quot; rel=&quot;nofollow&quot;&gt;very good tutorial&lt;/a&gt; on Adafruit that utilized OpenCV for face recognition. Can these same face recognition algorithms, Eigenfaces &amp;amp; Fisherfaces, be used to train my raspberry pi for recognizing different types of beer cans? &lt;/p&gt;&#xA;" OwnerUserId="8027" LastActivityDate="2014-06-11T08:34:17.653" Title="Image recognition on Raspberry Pi" Tags="|computer-vision|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11458" PostTypeId="2" ParentId="7149" CreationDate="2014-04-23T22:48:18.683" Score="6" Body="&lt;p&gt;In Torque only this works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   qalter &amp;lt;jobid&amp;gt; -W queue=&amp;lt;new queue name&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8028" LastActivityDate="2014-04-23T22:48:18.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11459" PostTypeId="2" ParentId="11448" CreationDate="2014-04-24T18:58:55.960" Score="0" Body="&lt;p&gt;The best idea is to use an algorithm tailored to your particular optimization problem structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without using any specialized algorithms tailored to this problem, you have an MINLP, and there's no really good way of solving general large MINLPs; &quot;large&quot; at the time of this writing is probably somewhere in the hundreds or thousands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some work by Glover on reformulating these problems to MILPs; the basic idea is to note that for $x_{i}, x_{j} \in \{0, 1\}$, some valid inequalities hold, like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;x_{i}x_{j} &amp;amp;\geq x_{i} + x_{j} - 1 \\&#xA;x_{i}      &amp;amp;\geq x_{i}x_{j} \\&#xA;x_{j}      &amp;amp;\geq x_{i}x_{j}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You add those inequalities and replace $x_{i}x_{j}$ with the variable $z_{ij} \in \{0, 1\}$. However, such a reformulation greatly increases the number of binary variables in your problem, so I doubt it's a viable option. Other versions of this idea vary in the size of the reformulation, but the order of magnitude of the sizes of each reformulation is more or less the same.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-04-24T18:58:55.960" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11460" PostTypeId="1" CreationDate="2014-04-24T20:37:13.223" Score="1" ViewCount="1845" Body="&lt;p&gt;I am using a python script to create wirebond interfaces in Ansys HFSS. Variables define the board, wirebond, and package geometry.  To simulate a new wirebond, I just edit the geometry variables and run the script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is, I have to use the computer with HFSS (and it's expensive license) just to verify the geometry is correct. It would be great if I had some free software that I could use to verify the geometry specified by my python script is correct. This would allow me to edit the script on any computer and only use HFSS when I was ready to simulate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script uses the &quot;hfss&quot; python class to send geometry and simulation options to HFSS. The hfss class and all commands specific to hfss would obviously need to be removed if different software was used, but that's ok. The coordinates of a box are specified the same in HFSS as many other CAD programs. Since, the specific coordinates of each geometric shape are what I want to verify, I don't see a reason why I can't use an alternative software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One alternate I have tried is Salome. Geometry can be created with python and the syntax to describe shapes is similar to HFSS. I also found a 3-D viewer from Ansys called &quot;3-D Ansys CFD Viewer&quot;. Looking at the description, I believe it can only view images created in HFSS and does not allow new geometry to be created, but I'm not 100% sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any software suggestions or ideas that would allow me to break free from the computer with HFSS.&lt;/p&gt;&#xA;" OwnerUserId="8037" LastActivityDate="2019-06-11T11:03:30.480" Title="Free HFSS alternative to create geometry using python" Tags="|python|software|electromagnetism|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11461" PostTypeId="1" CreationDate="2014-04-25T03:51:27.290" Score="2" ViewCount="2569" Body="&lt;p&gt;I am trying to implement QR factorization of a non-square matrix in FORTRAN.  I have the algorithm for a square matrix but not for a non-square. I use Housholder matrices. Do you know where I could find the whole algorithm or code for the non-square case so that I can be sure I am doing it the right way (i am new in programming!)? I have to solve (backsubstitute) for the upper trangular R and the orthogonal Q at the last part but the decomposition must be done for the non-square case. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8041" LastActivityDate="2014-04-28T02:42:31.097" Title="fortran code-algorithm for qr decomposition of non-square matrix" Tags="|matrix|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11462" PostTypeId="2" ParentId="11461" CreationDate="2014-04-25T04:13:48.373" Score="6" Body="&lt;p&gt;Trefethen and Bau's book &lt;a href=&quot;http://books.google.com/books?isbn=0898719577&quot;&gt;Numerical Linear Algebra&lt;/a&gt; has the Householder QR algorithm in chapter 10, and it's written considering general rectangular matrices. It's also in &lt;a href=&quot;http://books.google.com/books?isbn=1421407949&quot;&gt;Matrix Computations&lt;/a&gt; by Golub and van Loan. Both books give algorithms more or less in the form of Matlab code, so you'll have to do some translation between the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a Fortran implementation of QR in &lt;a href=&quot;http://www.netlib.org/lapack/double/&quot;&gt;LAPACK&lt;/a&gt;, as well as numerous other algorithms.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-04-25T04:13:48.373" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11463" PostTypeId="1" CreationDate="2014-04-25T08:12:35.927" Score="1" ViewCount="959" Body="&lt;p&gt;I have done a Fourier transformation of two signals (in time) $S_1(t),S_2(t)$ using numpy's &lt;code&gt;fft&lt;/code&gt; which will give me $S_1(f),S_2(f)$. The corresponding frequency grid I get via &lt;code&gt;fftfreq&lt;/code&gt;. In the frequency space, I then interpolated the two signals to a new grid using &lt;code&gt;interp1d&lt;/code&gt;. I need to do this to match the two grids of the signals so that I can divide them at the correct frequencies. However, I now want to transform the divided property $S_1(f)/S_2(f)$ back to time space using &lt;code&gt;ifft&lt;/code&gt;. But: How do I get the new time grid?&lt;/p&gt;&#xA;" OwnerUserId="7040" LastEditorUserId="7040" LastEditDate="2014-04-25T08:27:45.670" LastActivityDate="2014-04-25T17:38:50.107" Title="Time array from frequency array in FFT using Python" Tags="|python|fftw|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11466" PostTypeId="2" ParentId="11463" CreationDate="2014-04-25T17:38:50.107" Score="2" Body="&lt;p&gt;By interpolating in frequency you are extending the length of your signal in time.  You haven't done anything to increase your sampling rate so $\Delta t$ is fixed.  Your new time grid will be of length N+M where N is the original length and M is the number of points added through interpolation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are you uniformly interpolating your frequency domain signal or just interpolating at a few points?&lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2014-04-25T17:38:50.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11468" PostTypeId="2" ParentId="11429" CreationDate="2014-04-25T19:21:34.017" Score="1" Body="&lt;p&gt;An approach I often use when applying unconstrained optimisation algorithms to constrained problems is to transform the parameter space such that the constraints &lt;em&gt;cannot&lt;/em&gt; be violated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your problem I would define&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;g(x_1, x_2, x_3) = f(x_1, x_1-x_2^2, x_1-x_2^2-x_3^2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\arg \min g(x_1, x_2, x_3)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and finally recover the solution to the original problem with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*}&#xA;\theta^*_1 &amp;amp;= x^*_1\\&#xA;\theta^*_2 &amp;amp;= x^*_1 - x^{*2}_2\\&#xA;\theta^*_3 &amp;amp;= x^*_1 - x^{*2}_2 - x^{*2}_3&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this results in $\theta^*_1\geq\theta^*_2\geq\theta^*_3$ which isn't &lt;em&gt;quite&lt;/em&gt; what you asked for. To get a strict ranking you'll need to bump $x_1-x^2_2$ and $x_1-x^2_2-x^2_3$ down at the last digit of precision.&lt;/p&gt;&#xA;" OwnerUserId="7945" LastEditorUserId="7945" LastEditDate="2014-04-25T19:47:28.490" LastActivityDate="2014-04-25T19:47:28.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11469" PostTypeId="2" ParentId="11429" CreationDate="2014-04-25T21:42:47.087" Score="1" Body="&lt;p&gt;If you relax the strictness criterion, then yes, you could pose your question as a constrained optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA; &amp;amp; \min_{\theta_{1}, \theta_{2}, \theta_{3}} f(\theta_{1}, \theta_{2}, \theta_{3}) \\&#xA;\textrm{s.t.} &amp;amp; \theta_{1} \geq \theta_{2} \geq \theta_{3}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To approximate $\theta_{1} &amp;gt; \theta_{2} &amp;gt; \theta_{3}$, you could add a parameter $\varepsilon &amp;gt; 0$ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA; &amp;amp; \min_{\theta_{1}, \theta_{2}, \theta_{3}} f(\theta_{1}, \theta_{2}, \theta_{3}) \\&#xA;\textrm{s.t.} &amp;amp; \theta_{1} \geq \theta_{2} + \varepsilon \\&#xA;&amp;amp; \theta_{2} \geq \theta_{3} + \varepsilon&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These variants of your constraints are linear, so provided that your function $f$ is well-behaved (smooth, easy to calculate, easy to compute derivatives, derivatives are well-conditioned, etc.), any constrained optimization solver should be able to solve your problem without issue.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-04-25T21:42:47.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11470" PostTypeId="1" CreationDate="2014-04-26T10:20:28.033" Score="3" ViewCount="2697" Body="&lt;p&gt;I am trying to teach myself some programming in C by doing the problems of project Euler. I am trying to find the largest prime factor of the number 600851475143 (Euler problem 3). I have written a code which successfully does this for numbers which are not too large:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;int primearray(int,int);&#xA;&#xA;main()&#xA;{&#xA;  int alpha, k, j, i=1, prime[200000], count = 0;&#xA;  prime[0]=2;&#xA;&#xA;/* Define number which counts number of primes in interval [0,alpha] */&#xA;  int totprime=1;&#xA;&#xA;  printf(&quot;Enter the number of which you want the prime factorization\n&quot;);&#xA;  scanf(&quot;%d&quot;,&amp;amp;alpha);&#xA;&#xA;/* This procedure makes a list of all prime numbers up to alpha */&#xA;  for (j=3 ; j&amp;lt;alpha ; j++)&#xA;  {&#xA;    for (k=0 ; k&amp;lt;i ; k++)&#xA;    {&#xA;      if ((j/prime[k])*prime[k]==j)&#xA;        break;&#xA;      else&#xA;        count = count + 1;&#xA;&#xA;    }&#xA;    if (count == i)&#xA;      {&#xA;      prime[i]=j;&#xA;      i = i + 1;&#xA;      count = 0;&#xA;      totprime = totprime + 1;&#xA;      }&#xA;    else&#xA;      count = 0;&#xA;  }&#xA;&#xA;/* Print prime numbers */&#xA;  printf(&quot;The primes are:\n&quot;);&#xA;  for (j=0 ; j&amp;lt;totprime ; j++) printf(&quot;%d\n&quot;,prime[j]);&#xA;&#xA;  printf(&quot;Total number of primes: %d\n&quot;,totprime);&#xA;&#xA;  int primefac[totprime];&#xA;&#xA;  for (j=0 ; j&amp;lt;totprime ; j++)&#xA;  {&#xA;    if ( (alpha/prime[j])*prime[j] == alpha)&#xA;      {&#xA;        primefac[count] = prime[j];&#xA;        count = count + 1;&#xA;      }&#xA;    else&#xA;      continue;&#xA;  }&#xA;printf(&quot;Primefactors are\n&quot;);&#xA;for (j=0 ; j&amp;lt;count ; j++) printf(&quot;%d\n&quot;,primefac[j]);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it fails to work for large numbers, probably because it has been programmed horribly inefficient. I was wondering if you can help me think of a way to improve this program such that it is able to handle large numbers in a reasonable amount of time. All tips are welcome!&lt;/p&gt;&#xA;" OwnerUserId="8051" LastActivityDate="2014-05-03T03:46:53.863" Title="Optimization of prime factorization in C" Tags="|c|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11471" PostTypeId="1" CreationDate="2014-04-26T12:57:25.857" Score="1" ViewCount="598" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Question: I have problems with sidelobes when computing the convolution of two radially symmetric functions. I wonder whether I should just switch from second order to fourth order and refine the grid aggressively, or whether there are more appropriate methods for computing this convolution.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have radially symmetric convolution kernels, given in sufficiently well specified representations like sum of Gaussians, piecewise constant, or piecewise polynomial in $r^2$. &lt;em&gt;I have routines to analytically evaluate the Fourier transform of these kernels either pointwise or integrated over a ring&lt;/em&gt;. See appendix if interested in these details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute the convolution of two such kernels, I evaluate the Fourier transform pointwise on a sufficiently fine grid in Fourier space, multiply the two Fourier transforms pointwise with each other. I use a logarithmically spaced grid (a factor of 10 is sampled by 200 points), with an additional point at the origin. Then I use some interpolation scheme (currently just linear interpolation in $\rho^2$, but I wonder whether I should switch to a cubic spline interpolation) to get a piecewise polynomial in $\rho^2$ representation of the product in Fourier space. Then I evaluate the Fourier transform of this representation integrated over a ring to get a piecewise constant approximation of the convolution of the two kernels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that a double logarithmic plot of the result can reveal spectral leakage in the form of side-lobes. What I don't like about this is that I can make the grid in Fourier space finer to reduce the amplitude of the side-lobes, but a double logarithmic plot can easily cover 12 orders of magnitude, and it seems excessive to me to compute the convolution to an accuracy of $10^{-12}$, just because of artifacts in a double logarithmic plot which are irrelevant to the accuracy of relevant results from the application of the resulting convolution kernel. On the other hand, the artifacts might be more dangerous than I realize, because my kernels are all expected to be positive, but the resulting convolution kernel might have negative values due to the artifacts from the spectral leakage.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As an example, I computed the convolution of a sum of a 2 nm wide Gaussian and a 10 um wide Gaussian with two different very small (0.35 nm and 1 nm) Gaussians. In the images below, the red line is this sum these two Gaussians. The result should be nearly identical to this, i.e. a sum of two Gaussians, where the smaller is 2.03 nm or 2.236 nm wide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image of sidelobes after convolution with a 0.35 nm wide Gaussian: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/6wf4n.png&quot; alt=&quot;sidelobes after convolution with a 0.35 nm wide Gaussian&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image of sidelobes after convolution with a 1 nm wide Gaussian:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DJoDH.png&quot; alt=&quot;sidelobes after convolution with a 1 nm wide Gaussian&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Appendix&lt;/strong&gt;&#xA;&lt;em&gt;Here is some more of the math (equations) involved&lt;/em&gt;: The Fourier transform of $\exp(-r^2)$ is $\pi\exp(-\pi^2\rho^2)$, the Fourier transform of $\operatorname{circ}(r)$ is $\frac{J_1(2\pi\rho)}{\rho}$, and the Fourier transform of $(1-r^2)^n\operatorname{circ}(r)$ isn't much more complicated either. The antiderivative of $\rho$ times these functions can also be computed analytically. Hence it should be clear that it is possible to analytically evaluate the Fourier transform of these kernels either pointwise or integrated over a ring.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2014-04-29T08:20:50.850" LastActivityDate="2015-04-15T21:35:41.033" Title="Convolution of two radially symmetric functions for double logarithmic plot" Tags="|numerical-analysis|fourier-analysis|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11472" PostTypeId="2" ParentId="11470" CreationDate="2014-04-26T13:54:25.633" Score="6" Body="&lt;p&gt;Whether your code is efficient or not, it will not work for any numbers over 32767 as written. This is because the &lt;code&gt;int&lt;/code&gt; data type is a signed type of 16 bit length. One bit is used for the sign and 15 are used for the value of the integer, making the largest storable integer 2&lt;sup&gt;15&lt;/sup&gt;-1=32767. If you wish to support numbers larger than this, you will need to use a different integer type such as &lt;code&gt;long&lt;/code&gt;. Assuming you do not need to store negative integers, you should also use an unsigned type such as &lt;code&gt;unsigned long&lt;/code&gt; or even &lt;code&gt;unsigned long long&lt;/code&gt; which can store integer values up to 2&lt;sup&gt;32&lt;/sup&gt;-1=14294967295 or 2&lt;sup&gt;64&lt;/sup&gt;-1=18446744073709551615 respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;br&gt;&#xA;As @Ahmed pointed out in the comments, the C standards do not specify the sizes of integer types so portability can be a problem. To make sure you get the behavior you expect, I would suggest including the &lt;code&gt;&amp;lt;inttypes.h&amp;gt;&lt;/code&gt; header. &lt;code&gt;&amp;lt;inttypes.h&amp;gt;&lt;/code&gt; subsumes &lt;code&gt;&amp;lt;stdint.h&amp;gt;&lt;/code&gt; and includes additional macros, especially relating to I/O. As described &lt;a href=&quot;http://pubs.opengroup.org/onlinepubs/009695399/basedefs/stdint.h.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, &lt;code&gt;&amp;lt;stdint.h&amp;gt;&lt;/code&gt; defines the types:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int8_t    // 8 bit integer&#xA;int16_t   // 16 bit integer&#xA;int32_t   // 32 bit integer&#xA;uint8_t    // unsigned 8 bit integer&#xA;uint16_t   // unsigned 16 bit integer&#xA;uint32_t   // unsigned 32 bit integer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Most implementations also provide&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int64_t   // 64 bit integer&#xA;uint64_t   // unsigned 64 bit integer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These types are portable and reliable across architectures and compilers. I think using an external library is overkill in this case, but if needed, you can see @Ahmed's link in the comments.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-04-27T00:11:49.823" LastActivityDate="2014-04-27T00:11:49.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11473" PostTypeId="2" ParentId="11461" CreationDate="2014-04-26T14:01:15.787" Score="0" Body="&lt;p&gt;MINPACK uses a non-square QR factorization with pivoting, i.e. it's not a real QR decomposition (like the one in LAPACK), but a generalized more robust QRP decomposition (where P is the permutation matrix representing the pivoting).&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2014-04-26T14:01:15.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11474" PostTypeId="2" ParentId="11470" CreationDate="2014-04-26T18:03:08.443" Score="4" Body="&lt;p&gt;Since Doug has pointed out the need for a bigger integer datatype to do the problem you want to tackle, let's talk about the logic of your program and improvements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your approach is (A) input &lt;code&gt;alpha&lt;/code&gt; as the number to factor, (B) make a list of all prime numbers less than &lt;code&gt;alpha&lt;/code&gt;, and (C) see which of these divide &lt;code&gt;alpha&lt;/code&gt; and add the ones that do to an array &lt;code&gt;primefac[ ]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This factoring technique is a version &lt;a href=&quot;http://en.wikipedia.org/wiki/Trial_division&quot; rel=&quot;nofollow&quot;&gt;trial division&lt;/a&gt;.  That is a good place to start, at least when you know that &lt;code&gt;alpha&lt;/code&gt; itself is not a prime, something you probably want to assume from the context of the Euler Problem.  However there are relatively simple and fast ways to be sure that &lt;code&gt;alpha&lt;/code&gt; has factors (is &lt;em&gt;composite&lt;/em&gt;) before we knock ourselves out trying to factor it, so for general purposes you might want to look into the &lt;a href=&quot;http://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test&quot; rel=&quot;nofollow&quot;&gt;Miller-Rabin primality test&lt;/a&gt;.  But that can be another Question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic thing to fix in your program (after longer datatypes) is checking for repeated prime factors.  Once you determine that a prime divides &lt;code&gt;alpha&lt;/code&gt;, you just continue with the next prime.  This means you never determine &lt;em&gt;how many times&lt;/em&gt; a repeated prime factor occurs in &lt;code&gt;alpha&lt;/code&gt;, and that may be fine for your application, but it also means you never take advantage of making &lt;code&gt;alpha&lt;/code&gt; smaller by any of the prime factors that divide it.  Smaller numbers to check generally mean fewer checks to make, as we explain in the next paragraph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have set the goal to check &lt;code&gt;alpha&lt;/code&gt; for divisibility of all primes less than &lt;code&gt;alpha&lt;/code&gt;, but this is overkill.  You really only want to check for divisibility by primes that are less than or equal to the square root of &lt;code&gt;alpha&lt;/code&gt;.  Any factor of &lt;code&gt;alpha&lt;/code&gt; that is left after checking trial division by primes less than or equal to the square root of &lt;code&gt;alpha&lt;/code&gt; must itself be a prime.  As the size of &lt;code&gt;alpha&lt;/code&gt; drops, so to does the size of its square root, and we can stop when the trial divisor gets to be bigger than this falling target (square root of &lt;code&gt;alpha&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover you are using trial division to build the list of primes to check for divisibility into &lt;code&gt;alpha&lt;/code&gt;.  This is overkill.  You do treat 2 as a special case (known prime), but then you start checking all the consecutive integers starting with 3 for primality.  As Godric commented above, you can easily increment by 2 instead of 1 when creating prime candidates, since there will never be another even prime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed the testing of these candidates for primality wastes too much time.  You can generate some odd numbers that &lt;em&gt;include&lt;/em&gt; all the odd primes quickly, and use these as your trial divisors.  Even though these will include some odd composite numbers, you will save time because you skip testing the trial divisors for primality and go directly into testing &lt;code&gt;alpha&lt;/code&gt; for divisibility by these trial divisors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I recommend these steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Find out if 2 divides &lt;code&gt;alpha&lt;/code&gt;, and if so set &lt;code&gt;alpha = alpha/2&lt;/code&gt; until &lt;code&gt;alpha&lt;/code&gt; is no longer divisible by 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Using the possibly revised value of &lt;code&gt;alpha&lt;/code&gt; from the last step, find out if it is divisible by 3, and if so set &lt;code&gt;alpha = alpha/3&lt;/code&gt; until &lt;code&gt;alpha&lt;/code&gt; is no longer divisible by 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Starting with 5, and continuing until the trial divisors are greater than the square root of the current &lt;code&gt;alpha&lt;/code&gt;, generate trial divisors by alternately adding 2 and 4.  I.e. generate 5,7=5+2,11=7+4,13=11+2,17=13+4, etc.  If &lt;code&gt;alpha&lt;/code&gt; is divisible by the current trial divisor &lt;code&gt;T&lt;/code&gt;, set &lt;code&gt;alpha = alpha/T&lt;/code&gt; until &lt;code&gt;alpha&lt;/code&gt; is no longer divisible by &lt;code&gt;T&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(4) The stopping criterion, that &lt;code&gt;T&lt;/code&gt; is greater than the square root of current &lt;code&gt;alpha&lt;/code&gt;, does not require a square root operation.  Instead it makes sense to check whether &lt;code&gt;alpha/T&lt;/code&gt; is less than &lt;code&gt;T&lt;/code&gt; and stop if that is the case.  This is fairly efficient because you would compute &lt;code&gt;alpha/T&lt;/code&gt; anyway in the course of checking (exact) divisibility by &lt;code&gt;T&lt;/code&gt; as your code already does it, asking if &lt;code&gt;alpha == (alpha/T)*T&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary: bigger datatypes, no list of primes, check only up to the square root of &lt;code&gt;alpha&lt;/code&gt;.  If the final value of &lt;code&gt;alpha&lt;/code&gt; is greater than 1, then that is the largest prime factor of the originally input &lt;code&gt;alpha&lt;/code&gt;.  Otherwise the largest prime factor is the last successful trial divisor &lt;code&gt;T&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2014-04-26T18:03:08.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11475" PostTypeId="2" ParentId="11470" CreationDate="2014-04-27T03:46:36.403" Score="0" Body="&lt;p&gt;you should consider making use of the modulus (%) operator instead of (j/prime[k]*prime[k] == j) say (j%prime[k] == 0). I know this is supported in C++, correct me if it is not present in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore you only need to check primes up to the square root of your target number. So say you want to factor a number N, you only need primes up to sqrt(N) since its guaranteed if the number is not composite that one factor will exist on that interval.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you want to find all prime numbers less than 100. We just need to generate primes 2,3,5,7 through a process of your choice. and then if you write out all the numbers from 8 - 100, and just cross out the multiples of 2,3,5 and 7 (without considering ANY additional primes) the only numbers that won't be crossed out will be the remaining prime numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus to simplify your work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;List every prime number less than 775146&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and check each of these against 600851475143 as you generate them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to handle large numerical datatypes i recommend either importing externally OR:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;take a look at the vector class (well thats c++) and if thats off limits to then you are going to want to create a numerical type function which essentially converts a string ('101101343433') into a linked list of digits and then create functinos for adding, subtracting, dividing, and multiplying over this linked list device.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LASTLY IF YOU REALLY REALLY NEED HIGH SPEED.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you generate your list of primes multiply every k consecutive primes together (choose k as something such that the average size of the numbers yielded has the same number of digits as the number you want to factor). And then find the GCF of this target number and your groups of k primes multiplied together. The punchline here is that instead of having to do k separate modulus checks (is Num % prime == 0) or in your case k seperate divisions and multiplications you do k smaller multiplications (since they are smaller numbers) thereby shaving off some time. The euclidean algorithm should be used to find GCF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;given A, B to find GCF(a,b)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;while(true):&#xA;if( A &gt;= B)&lt;br&gt;&#xA;    A = A %B;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if A == 0 &#xA;   return B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if( B &gt;= A)&#xA;   B = B%A;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if B == 0&#xA;      return A&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus here is an example for the checking the factors of the number 143.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;List&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2,3,5,7,11 (these are all primes less than sqrt(143) generated using the same eratosthenes protocol that you have in your code (the sieve of eratosthenes is the formal name for the trial division algorithm you have implemented) )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;group:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2x3x5 = 30, 5x7x11 = 385&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GCF(143,30) = 1&#xA;GCF(143,385) = 11&lt;/p&gt;&#xA;&#xA;&lt;p&gt;11 = factor&lt;/p&gt;&#xA;&#xA;&lt;p&gt;143/11 = 13&lt;/p&gt;&#xA;&#xA;&lt;p&gt;13 is another factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;13 &gt; 11&lt;/p&gt;&#xA;&#xA;&lt;p&gt;13 is the largest factor.&lt;/p&gt;&#xA;" OwnerUserId="8059" LastActivityDate="2014-04-27T03:46:36.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11477" PostTypeId="1" CreationDate="2014-04-27T13:00:03.477" Score="8" ViewCount="19967" Body="&lt;p&gt;I need help plotting the &lt;a href=&quot;https://en.wikipedia.org/wiki/Heaviside_step_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heaviside function&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/d/d9/Dirac_distribution_CDF.svg&quot; width=&quot;300&quot;&gt;&#xA;&lt;p&gt;Real analysis often involves constructing bizarre functions which are intuitively correct, but ultimately wrong.  See the great book &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0486428753&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Counterexamples in Analysis&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I wrote a &lt;code&gt;numpy&lt;/code&gt; script to plot the function &lt;span class=&quot;math-container&quot;&gt;$\displaystyle \sum_{n=1}^\infty \frac{1}{n^2} H\left(x - \frac{1}{n}\right)$&lt;/span&gt; to illustrate some concepts in real analysis.  This is in a response to another question on math.stackexchange:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/262121&quot;&gt;Demonstrating Continuity Properties of &lt;span class=&quot;math-container&quot;&gt;$f(x) = \sum_{n=1}^\infty [a_n H(x - x_n)]$&lt;/span&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here is an example with an accompanying plot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;B = [ [1,n] for n in range(1,50) ]&#xA;&#xA;N = 1000&#xA;y = np.zeros(N)&#xA;count = 1&#xA;for b in B:&#xA;    x = np.zeros(N)&#xA;    x[ np.arange(b[0]*1.0*N/b[1], N).astype(int)]=count**-2&#xA;    y += x&#xA;    count += 1&#xA;&#xA;plt.plot(np.arange(0,1,1.0/N),y)&#xA;plt.axis(&amp;quot;Equal&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mvxxT.png&quot; alt=&quot;Heaviside function&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I remove the vertical lines and really show the discontinuity?&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="20688" LastEditDate="2022-06-20T23:45:21.060" LastActivityDate="2022-06-20T23:45:21.060" Title="plotting discontinuous functions" Tags="|python|numerics|numpy|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11478" PostTypeId="2" ParentId="11457" CreationDate="2014-04-27T18:00:44.757" Score="1" Body="&lt;p&gt;This &lt;a href=&quot;http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html#id27&quot; rel=&quot;nofollow&quot;&gt;OpenCV documentation page&lt;/a&gt; suggests that the &lt;em&gt;Eigenfaces&lt;/em&gt; algorithm is simply dimensionality reduction using PCA and classification by nearest neighbor search. &lt;em&gt;Fisherfaces&lt;/em&gt; uses Linear Discriminant Analysis instead, which aims to maximize between-class variance while minimizing within-class variance, and so may perform better in some cases. Neither of these dimensionality reduction methods is specific to faces (or even to images) in any meaningful way, so you should be able to use the same algorithms for general image classification problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing to note though, is that the success rate for your run-of-the-mill classification algorithms, like nearest neighbor searches, will be highly dependent on the quality of training data. With faces, you can get away with using grayscale images, because a lot of the variance between images of two different faces is encoded in the spacial distribution of the data. For beer, I would think about taking advantage of the differences in color between bottle types by using color images, even if you have to reduce the number of pixels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And just for fun, I might also look at other classification algorithms to use on your PCA or LDA processed data. As an example, OpenCV includes &lt;a href=&quot;http://docs.opencv.org/doc/tutorials/ml/introduction_to_svm/introduction_to_svm.html#introductiontosvms&quot; rel=&quot;nofollow&quot;&gt;Support Vector Machines&lt;/a&gt;, which will almost always perform better than a simple nearest neighbor search.&lt;/p&gt;&#xA;" OwnerUserId="7845" LastActivityDate="2014-04-27T18:00:44.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11479" PostTypeId="2" ParentId="11477" CreationDate="2014-04-27T18:54:21.943" Score="1" Body="&lt;p&gt;You can ' hold ' the plot and plot each component separately. Loop over the different functions and plot.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2014-04-27T18:54:21.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11480" PostTypeId="2" ParentId="11470" CreationDate="2014-04-27T22:21:54.367" Score="3" Body="&lt;h1&gt;Examination of your attempt&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Since you are trying to teach yourself C, let's actually show you what the issues are in your code and show you how to attack this problem properly utilizing the complete power of C.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The following declaration is useless&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int primearray(int,int);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The declaration below should be ANSI compliant&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;int main() &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You should NOT use such a big array as a stack variable and actually either declare it&#xA;  as a global variable or allocate it dynamically ( much better solution ) &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  int alpha, k, j, i=1, prime[200000], count = 0;&#xA;  prime[0]=2;&#xA;&#xA;/* Define number which counts number of primes in interval [0,alpha] */&#xA;  int totprime=1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;you should divide the code up into separate functions so that you can deal with&#xA;  each piece simply and easily&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For instance a function that takes a number and finds all primes using whatever&#xA;  method, that way you can start out with a simple function and as you learn better&#xA;  methods you can actually make that routine faster.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  printf(&quot;Enter the number of which you want the prime factorization\n&quot;);&#xA;  scanf(&quot;%d&quot;,&amp;amp;alpha);&#xA;&#xA;/* This procedure makes a list of all prime numbers up to alpha */&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The block below has many issues. &lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;First it's suboptimal because it tests far too many numbers for primality.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;The test for primality is incorrect and will not result in the value always equal to &#xA;  j when they're supposed to be. Also it's very CPU intenstive you can use modulo operator&#xA;  to get results more effectively&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;The tests for storing the primes is wierd.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;This entire block should be in its own function.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  for (j=3 ; j&amp;lt;alpha ; j++)&#xA;  {&#xA;&#xA;&#xA;    for (k=0 ; k&amp;lt;i ; k++)&#xA;    {&#xA;      if ((j/prime[k])*prime[k]==j)&#xA;        break;&#xA;      else&#xA;        count = count + 1;&#xA;&#xA;    }&#xA;&#xA;    if (count == i)&#xA;      {&#xA;      prime[i]=j;&#xA;      i = i + 1;&#xA;      count = 0;&#xA;      totprime = totprime + 1;&#xA;      }&#xA;    else&#xA;      count = 0;&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Don't comment obvious code &#xA;  the code shows that you're going to print the primes&#xA;  adding a comment about is frivilous and detracts&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/* Print prime numbers */&#xA;  printf(&quot;The primes are:\n&quot;);&#xA;&#xA;  for (j=0 ; j&amp;lt;totprime ; j++) printf(&quot;%d\n&quot;,prime[j]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using for loop the way you did above is poor form and you should &#xA;  not use the printf statement on the same line.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Generally speaking: Your tabing and block style is inconsistent when you are &#xA;coding you should use a consistent way to mark your blocks code is read far more often than it's written using a consistent style helps follow the logic &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  printf(&quot;Total number of primes: %d\n&quot;,totprime);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dynamic arrays are not supported in C.  The code below is not standard.&#xA;  if your platform/compiler supports it you will get inconsistent results &#xA;  when you port the code.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  int primefac[totprime];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The block below has many problems.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  for (j=0 ; j&amp;lt;totprime ; j++)&#xA;  {&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This test will fail because you are assuming things about how &#xA;  integer division works (and whether or not it results in a fraction)&#xA;  Not to mentions that it's an extremely expensive operation &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    if ( (alpha/prime[j])*prime[j] == alpha)&#xA;      {&#xA;        primefac[count] = prime[j];&#xA;        count = count + 1;&#xA;      }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The else below is completely not required by any logic&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    else&#xA;      continue;&#xA;  }&#xA;&#xA;&#xA;printf(&quot;Primefactors are\n&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Same issue as above&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (j=0 ; j&amp;lt;count ; j++) printf(&quot;%d\n&quot;,primefac[j]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This should line up with the main function&#xA;      }&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;/* STILL EDITING */ The correct approach coming up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay now let's figure out how approach this problem correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stating the problem simply&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You want to find the largest prime factor of 600851475143.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is to say you want to find maximum value of N &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left. 600851475143 \equiv 0 \pmod{N}  \: \middle| &#xA;   \: {N \not \equiv 0} \pmod {p} \; \forall p \in \mathbb{Z}^+ \right. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In C the modulo operator (%) (actually it's the remainder operator) provides the necessary capability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how do we do this, first let's create a function that does that.  We know that if $ a \times b \equiv N \; \forall {a, b} \in \mathbb{Z}^+ $ then either $ a \leq \sqrt{N} $ or $ b \leq \sqrt{N} $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a simple (crude) algorithm would be to see if there is a prime number $p$ such that: $ \left. p \leq \sqrt{600851475143} \; \big| \; p \, \vert \, 600851475143 \right. $ &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;A correct approach&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I will not provide you with a solution but walk you through some basic concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First things first, it's a good idea to split things up into smaller chunks and treat each as a function. If you define your approach step by step then a good rule of thumb is to create one function per step.  If it's a complex step, with simpler substeps, then that function should be divided into multiple sub functions.  You may not get it right the first time, that's okay, you will develop better judgement as you go along as to how things should get divided up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay let's look at the problem step by step using our simple algorithm using a flowchart&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/leoap.png&quot; alt=&quot;Flowchart for a crude algorith calculating the largest prime that divides a composite number&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now essentially each box in the flowchart above is going to the statements of your main function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/** &#xA; * name: primes_upto&#xA; * parameters:&#xA; *    M: finds all primes until M and stores them in zero-terminated list&#xA; *    list: the pointer to an array of ints&#xA; *          if list is NULL then it will allocate enough memory &#xA; *          to store the primes + 1 &#xA; *          if list is not NULL it will truncate insert a 0 at &#xA; *          the index where value of prime is &amp;gt; M&#xA; *&#xA; * return value: returns number of primes on success or 0 on failure&#xA; */&#xA;int primes_upto(int k, int **list); &#xA;&#xA;int main() {&#xA;&#xA;      const int original_N = 600851475143;&#xA;      int N, a, S;&#xA;      int *L = NULL;    /* list of primes we need to fill */&#xA;      int *ap = NULL; /* a memory pointer to integers that will point to the next prime */&#xA;&#xA;      N = original_N;&#xA;&#xA;      do {&#xA;          S = sqrt(N); /* we have to provide this */&#xA;&#xA;          primes_upto(S, &amp;amp;L); /* we have to provide this function */&#xA;&#xA;         /* we will make sure that the primes_upto function &#xA;            injects a zero at the end of the list. Since zero is not a prime&#xA;            we can use it as a marker to indicate the end of the list &#xA;            and that way we don't have to worry about how long the list is. */&#xA;&#xA;&#xA;         for (ap = L; ap &amp;amp;&amp;amp; *ap != 0; ap++) {&#xA;              int a = *ap;&#xA;              if (N % a == 0) {&#xA;                  N = N / a;&#xA;                  break;&#xA;              }&#xA;         }&#xA;&#xA;      } while (ap &amp;amp;&amp;amp; *ap != 0); &#xA;&#xA;      printf (&quot;The largest number that divides %d is %u\n&quot;, original_N, N);&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8057" LastEditorUserId="8057" LastEditDate="2014-05-03T03:46:53.863" LastActivityDate="2014-05-03T03:46:53.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11481" PostTypeId="2" ParentId="11461" CreationDate="2014-04-28T02:42:31.097" Score="0" Body="&lt;p&gt;I have these subroutines the one does the qr factorization for the nxm matrix A and it stores the upper triangular R and the w of the householder matrices in A while it saves the diagonals of R in d vector.then i try to form the Q^Tb so that i can solve the Rx=Q^tb with backsubstitution.I use it to do a data fitting with least square method(so with the backsubstitution i am supposed to take the coefficients) but it gives me  wrong results. If you have any ideas of what i am doing wrong it would be of great help because I cannot see where  the problem is!Thank you for your help!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subroutine qrdecomposition(a,c,d)&#xA;implicit none&lt;/p&gt;&#xA;&#xA;&lt;p&gt;real, dimension(n,m) :: a&#xA;  real, dimension(m) :: c&#xA;real, dimension(m) :: d&#xA;      logical :: sing&#xA;      !integer :: i,j,k,n,m&#xA;      real :: scale1,sigma,sum1,tau&#xA;      sing=.false.&#xA;      do k=1,m&#xA;    scale1=0&#xA;        do  i=k,n&#xA;          scale1=max(scale1,abs(a(i,k)))&#xA;      end do&#xA;        if(scale1.eq.0.)then&#xA;          sing=.true.&#xA;          c(k)=0.&#xA;          d(k)=0.&#xA;        else&#xA;          do  i=k,n&#xA;            a(i,k)=a(i,k)/scale1&#xA;        end do&#xA;          sum1=0.&#xA;          do  i=k,n&#xA;            sum1=sum1+a(i,k)*a(i,k)&#xA;       end do&#xA;          sigma=sign(sqrt(sum1),a(k,k))&#xA;          a(k,k)=a(k,k)+sigma&#xA;          c(k)=sigma*a(k,k)&#xA;          d(k)=-scale1*sigma&#xA;          do  j=k+1,m&#xA;            sum1=0.&#xA;            do i=k,n&#xA;              sum1=sum1+a(i,k)*a(i,j)&#xA;          end do&#xA;            tau=sum1/c(k)&#xA;            do  i=k,n&#xA;              a(i,j)=a(i,j)-tau*a(i,k)&#xA;          end do&#xA;       end do&#xA;        end if&#xA;    end do&#xA;      d(m)=a(m,m)&#xA;      if(d(m).eq.0.)sing=.true.&#xA;      return&#xA;      end subroutine qrdecomposition&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; subroutine qrsolution(a,c,d,b)&#xA;  implicit none&#xA;  !integer :: n,m&#xA; real, dimension(n,m) :: a &#xA;  real, dimension(n) :: b&#xA;  real, dimension(m) :: c&#xA;  real, dimension(m) :: d&#xA;&#xA;  !integer :: i,j&#xA;  real :: sum1,tau&#xA;  do  j=1,m-1&#xA;    sum1=0.&#xA;   do  i=j,n&#xA;      sum1=sum1+a(i,j)*b(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;end do&#xA;        tau=sum1/c(j)&#xA;        do  i=j,m&#xA;         b(i)=b(i)-tau*a(i,j)&#xA;      end do&#xA;   end do&lt;/p&gt;&#xA;&#xA;&lt;p&gt;do  i=m,1,-1&#xA;        sum1=0.&#xA;        do  j=i+1,m&#xA;        sum1=sum1+a(i,j)*b(j)&#xA;      end do&#xA;       b(i)=(b(i)-sum1)/d(i)&#xA;    end do&#xA;      return&lt;/p&gt;&#xA;&#xA;&lt;p&gt;end subroutine qrsolution&lt;/p&gt;&#xA;" OwnerUserId="8041" LastActivityDate="2014-04-28T02:42:31.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11482" PostTypeId="1" CreationDate="2014-04-28T04:57:14.340" Score="1" ViewCount="1332" Body="&lt;p&gt;Previously, I have calculated energy spectrum for 3D DNS data obtained for isotropic turbulence which is equally spaced in all three directions and then to compute the energy spectrum, one performs Fourier transform and then accumulates energy located in different wavenumber bins and then gets the ($k^{-5/3}$) slope and it all works fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if I want to extend the same for a turbulent boundary layer case, how should I modify the technique? I cannot take a Fourier transform in the wall normal direction anymore and I am not sure how to compute the energy spectrum as such. &#xA;There is no problem if I compute the spectrum for streamwise and spanwise plane since they are both defined using Fourier basis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is two-fold: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do we compute energy spectrum for a 3D turbulent boundary layer? &lt;/li&gt;&#xA;&lt;li&gt;If we cannot compute it for a 3D case i.e. if the argument is valid only for a 2D planar cut at various wall normal locations, then how do we define a scale for turbulent boundary layer cases, particularly along wall normal direction?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4364" LastActivityDate="2014-04-28T13:35:45.473" Title="Computing Kolmogorov/Energy spectrum for turbulent boundary layer" Tags="|fluid-dynamics|fourier-analysis|spectral-method|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11483" PostTypeId="1" CreationDate="2014-04-28T07:50:08.990" Score="2" ViewCount="283" Body="&lt;p&gt;I am looking for an algorithm that I can apply for a random tessellation of polygons with different areas. The algorithm can relax the geometry of the polygons to a condition that all of them would have the same area!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone should have faced this problem. The simplest case is that all the polygons have the same number of edges. In that case, one can find the relationship between the periphery and the area. However, the algorithm should work for general case with polygons with different number of edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if you guys give me some hints or link me to some related studies that might exist in the field o&lt;img src=&quot;https://i.stack.imgur.com/Jcju4.jpg&quot; alt=&quot;enter image description here&quot;&gt;f graph, computational geometry and mathematics.&lt;/p&gt;&#xA;" OwnerUserId="8069" LastActivityDate="2014-05-01T21:09:28.807" Title="Algorithm to equalize the area of random tessellation of various polygons" Tags="|optimization|computational-geometry|graph-theory|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11484" PostTypeId="2" ParentId="11477" CreationDate="2014-04-28T08:50:37.157" Score="5" Body="&lt;p&gt;As long as you know the exact positions of the discontinuities, you just have to set the jump positions to &lt;code&gt;nan&lt;/code&gt; in &lt;code&gt;x&lt;/code&gt;, &lt;code&gt;y&lt;/code&gt; or both. You can set this manually in the desired positions or use some criteria - for example, you can use the &lt;code&gt;np.diff&lt;/code&gt; function to calculate the difference between contiguous positions in an array. For the simple case of the Heaviside function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def heaviside(x):&#xA;    &quot;&quot;&quot;See http://stackoverflow.com/a/15122658/554319&quot;&quot;&quot;&#xA;    y = 0.5 * (np.sign(x) + 1)&#xA;    y[np.diff(y) &amp;gt;= 0.5] = np.nan&#xA;    return y&#xA;&#xA;x = np.linspace(-1, 1)&#xA;plt.plot(x, heaviside(x))&#xA;plt.ylim(-1, 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ljXBN.png&quot; alt=&quot;Heaviside function with matplotlib&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this in your specific example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def U(x, n):&#xA;    return sum([heaviside(x - 1 / i) / i ** 2 for i in range(1, n)])&#xA;&#xA;N = 1000&#xA;x = np.linspace(0, 1, N)&#xA;plt.plot(x, U(x, 50))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8aHPs.png&quot; alt=&quot;OP function&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="782" LastEditorUserId="782" LastEditDate="2014-05-04T14:38:16.697" LastActivityDate="2014-05-04T14:38:16.697" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11485" PostTypeId="2" ParentId="11477" CreationDate="2014-04-28T09:01:34.447" Score="3" Body="&lt;p&gt;Getting rid of the vertical lines in this plot would involve finding all the discontinuities and then plotting each segment in a separate call to &lt;code&gt;plt.plot&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another, less elegant, solution is to plot points instead of lines. Since the points are very close together (high value of &lt;code&gt;N&lt;/code&gt;), they will look like they form a line. To do this, replace &lt;code&gt;plt.plot(np.arange(0,1,1.0/N), y)&lt;/code&gt; with &lt;code&gt;plt.plot(np.arange(0,1,1.0/N), y, '.k')&lt;/code&gt; (the &lt;code&gt;.&lt;/code&gt;tells &lt;code&gt;plot&lt;/code&gt; to use dots, &lt;code&gt;k&lt;/code&gt; is for black). This is what you'll get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sDqqq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1126" LastActivityDate="2014-04-28T09:01:34.447" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11486" PostTypeId="2" ParentId="11482" CreationDate="2014-04-28T13:08:34.987" Score="3" Body="&lt;p&gt;1) One usually chooses one or two representative wall-normal distances and presents the spectra for &lt;code&gt;&amp;lt;u'u'&amp;gt;&lt;/code&gt;, &lt;code&gt;&amp;lt;v'v'&amp;gt;&lt;/code&gt;, etc. over the homogeneous directions.  For example, see Figure 11 within Spalart 1988 (&lt;a href=&quot;http://dx.doi.org/10.1017/s0022112088000345&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1017/s0022112088000345&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) By &quot;scale&quot; do you mean a lengthscale?  If so, a lengthscale quantifying what aspect of the flow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Backing up, what's your goal?  To compare against someone's published result?  To convince yourself that you have adequate resolution?  To look at some aspect of the physics in more detail?  Something else?&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2014-04-28T13:08:34.987" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11487" PostTypeId="2" ParentId="11482" CreationDate="2014-04-28T13:35:45.473" Score="0" Body="&lt;p&gt;Generally from what I have seen, people either compute 1D spectra (in span wise and stream wise directions) or compute 2D spectra of horizontal slices in the vertical. Depending on what you are trying to look at, these horizontal spectra can then be averaged in the vertical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im not entirely sure what scale you are looking for, but let me add that in boundary layers it is not obvious that a Kolmogorov inertial range should exist because of the anisotropic nature of the flow and I believe that this is still an active research area (e.g. convective boundary layers). I think what has been found is that at small enough scales pressure fluctuations do result in &quot;isotropotizing&quot; the flow thus resulting in an inertial range but this is not obvious from the 3D triply periodic theory. One important statistic that is often looked at is the ratio of vertical-to-horizontal spectra. For example in convective boundary layers this ratio should be a constant 4/3 in regions of local isotropy.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2014-04-28T13:35:45.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11488" PostTypeId="2" ParentId="11483" CreationDate="2014-04-28T13:39:44.263" Score="3" Body="&lt;p&gt;The fairly simple &lt;a href=&quot;http://en.wikipedia.org/wiki/Lloyd%27s_algorithm&quot; rel=&quot;nofollow&quot;&gt;Lloyd's algorithm&lt;/a&gt; can be used to achieve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The essence of the algorithm is you start with a given tesselation defined by a set of points and a distance metric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The points are then moved to the centroids, allowing the tesselation and the areas to be recomputed. This iterative process is then repeated until the movement is sufficiently small. If you are familiar with k-means clustering from statistics/classification it is essentially the same process.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2014-04-28T13:39:44.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11489" PostTypeId="2" ParentId="11447" CreationDate="2014-04-28T13:56:07.393" Score="1" Body="&lt;p&gt;If you are trying to add some viscosity where the solution grows fastest then maybe something like $\nu = c\left|\frac{dy}{dx}\right|$ would work where c is some constant that you choose (probably by trial and error).&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2014-04-28T13:56:07.393" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11490" PostTypeId="1" AcceptedAnswerId="11501" CreationDate="2014-04-28T14:21:39.183" Score="3" ViewCount="171" Body="&lt;p&gt;I want to impose symmetry plane boundary condition for a solid mechanics problem. I googled around and found out that in many places people say to &quot;forbid displacemnts out of symmentry plane and forbid rotations about any axis lying in this plane if you have rotational degrees of freedom&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have only disps degrees of freedom, how do I do it right? If my simmetry plane is e.g. YZ then I need to set $u_x=0$. It's only one equation, I think I need three to match the number of equations and unknowns. What are the others (if any)?&lt;/p&gt;&#xA;" OwnerUserId="3637" LastActivityDate="2014-04-29T14:47:26.287" Title="Imposing symmetry plane boundary condtition" Tags="|pde|numerical-analysis|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11492" PostTypeId="1" AcceptedAnswerId="11494" CreationDate="2014-04-28T15:17:51.263" Score="13" ViewCount="5125" Body="&lt;p&gt;I am wondering if the Thomas algorithm is the fastest way (provably?) to solve a symmetric diagonally dominate sparse tridiagonal system in terms of algorithmic complexity (not looking for implementation packages like LAPACK etc). I know that both the Thomas algorithm and multigrid are $O(n)$ complexity, but maybe the constant factor for multigrid is less? It doesn't seem to me like multigrid could be faster but I am not positive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am considering the case where the matrices are very large. Either direct or iterative methods are acceptable.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="20688" LastEditDate="2018-09-06T15:03:16.473" LastActivityDate="2019-01-24T21:34:02.913" Title="Is the Thomas algorithm the fastest way to solve a symmetric diagonally dominant sparse tridiagonal linear system" Tags="|linear-solver|sparse-matrix|complexity|multigrid|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11493" PostTypeId="2" ParentId="11492" CreationDate="2014-04-28T15:37:08.963" Score="11" Body="&lt;p&gt;The short answer is that the Thomas algorithm will be faster than any iterative scheme for almost all cases. The exception would perhaps be applying a single iteration of a very simple iterative scheme such as Gauss-Seidel, but this is highly unlikely to give an acceptable solution. Also, this is ignoring parallel processing concerns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multigrid is an especially poor choice in the case of a tri-diagonal matrix because although multigrid is $\mathcal O(n)$, the constant is quite large. In fact, multigrid does not even have an advantage over Gauss-Seidel until the matrices become quite large. This is due to the need for projection, prolongation, and relaxation operations for each multigrid level, each of which requires $\mathcal O(n)$ operations where n is the number of unknowns at that multigrid level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, this question is best addressed via operation counting. For the Thomas algorithm, a total of $5N$ multiplications and $3N$ additions are required for the solution. Iterative schemes require at least as many operations as matrix-vector multiplication and given a tri-diagonal matrix, each matrix-vector multiplication requires $3N-2$ multiplications and $2N-2$ additions. Therefore, even two applications of any (even the very simplest) iterative scheme will be more expensive than the Thomas algorithm.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="20688" LastEditDate="2018-09-06T15:01:46.137" LastActivityDate="2018-09-06T15:01:46.137" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11494" PostTypeId="2" ParentId="11492" CreationDate="2014-04-28T15:37:27.150" Score="12" Body="&lt;p&gt;I believe comparing an iterative method (multigrid) to a direct/exact method (Thomas) in terms of exact operation count isn't really meaningful. IIRC, Thomas operation count is $8N$ for &lt;em&gt;any&lt;/em&gt; tridiagonal system. The only time I can imagine multigrid conceivably beating that is for a trivial case of having a linear solution, and even then the cost of evaluating the residual at each level would be comparable to the cost of Thomas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $O(N)$ usefulness of multigrid lies in the fact that it's general for sparse matrices, and not restricted to tridiagonal systems.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2014-04-30T15:10:49.890" LastActivityDate="2014-04-30T15:10:49.890" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11495" PostTypeId="1" CreationDate="2014-04-28T16:34:47.903" Score="1" ViewCount="2551" Body="&lt;p&gt;I am going through this example problem (picture below) and working it out myself.  The problem is that I do not get the same mass matrix (M) as the example problem.  I am able to get the same stiffness matrix (K).  I used the same process of obtaining the stiffness matrix, which uses the connectivity array, but do not end up with the same mass matrix provided in the example. Can someone help me confirm whether the mass matrix in the given example is correct or incorrect?&#xA;&lt;img src=&quot;https://i.stack.imgur.com/qCS4X.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I obtain: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA; {4, 1, 0, 1, 2, 0, 0, 0, 0, 0, 0, 0},&#xA; {1, 6, 1, 0, 2, 2, 0, 0, 0, 0, 0, 0},&#xA; {0, 1, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0},&#xA; {1, 0, 0, 6, 2, 0, 1, 2, 0, 0, 0, 0},&#xA; {2, 2, 0, 2, 12,2, 0, 2, 2, 0, 0, 0},&#xA; {0, 2, 1, 0, 2, 6, 0, 0, 1, 0, 0, 0},&#xA; {0, 0, 0, 1, 0, 0, 6, 2, 0, 1, 2, 0},&#xA; {0, 0, 0, 2, 2, 0, 2, 12,2, 0, 2, 2},&#xA; {0, 0, 0, 0, 2, 1, 0, 2, 6, 0, 0, 1},&#xA; {0, 0, 0, 0, 0, 0, 1, 0, 0, 2, 1, 0},&#xA; {0, 0, 0, 0, 0, 0, 2, 2, 0, 1, 6, 1},&#xA; {0, 0, 0, 0, 0, 0, 0, 2, 1, 0, 1, 4}&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8072" LastEditorUserId="549" LastEditDate="2014-04-28T18:14:26.413" LastActivityDate="2024-03-05T18:39:34.053" Title="Help with basic Triangular Finite Elements, constructing stiffness/mass matrices" Tags="|finite-element|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="11497" PostTypeId="2" ParentId="7475" CreationDate="2014-04-28T18:32:24.773" Score="1" Body="&lt;p&gt;You should probably take a look at &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Single molecule counting statistics for systems with periodic driving&quot; J. Chem. Phys. 139, 164120 (2013)&#xA;&lt;a href=&quot;http://dx.doi.org/10.1063/1.4826634&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1063/1.4826634&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors there use a truncation of the Floquet matrix to look at semiclassical two-level systems.  Not sure if it covers precisely the case you are looking at, but it might be a good starting point.&lt;/p&gt;&#xA;" OwnerUserId="4494" LastActivityDate="2014-04-28T18:32:24.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11498" PostTypeId="2" ParentId="11495" CreationDate="2014-04-28T19:17:49.757" Score="2" Body="&lt;p&gt;Equation (b) from the example is correct assuming that m is the mass/area. However, when I create the global mass matrix for the model in Figure 9.21, I get the one you show rather than the one in the text (equation e).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you have to be aware of is that integrating the mass matrix terms over the triangle requires a higher-order integration formula than the stiffness matrix terms. For the stiffness terms a single integration point gives an exact value of the integral; for the mass terms, three integration points are needed.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2014-05-02T21:24:09.713" LastActivityDate="2014-05-02T21:24:09.713" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11500" PostTypeId="2" ParentId="11470" CreationDate="2014-04-29T04:24:17.740" Score="0" Body="&lt;p&gt;There's a more advanced version of the Pythagorean sieve that tracks patterns of composite numbers, and uses those patterns to reduce the number of necessary tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a simple example, consider the numbers 0 through 5. Only 1 and 5 are relatively prime to 6=2*3. You can use this information to eliminate some tests for the numbers 6 through 11. 2+6 = 8, and 2 is not relatively prime to 6, so neither can 8 be, so it must be composite. The same goes for 3+6 = 9 and 4+6 = 10. The others, 1+6 = 7, and 5+6 = 11, might be prime. You need to test these, but in this case they turn out to be prime. The point is you don't need to test 8, 9, or 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly for the next batch of 6, the numbers 12 through 17, they operate similarly to the previous interval because they're equivalent to 6 through 11 in mod 6 arithmetic. You need to test 1+12 and 5+12, but not 0+12, 2+12, 3+12, 4+12.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may continue for the intervals [18,23] and [24,29].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point you have covered the interval [0,29]. The next number 30 = 2*3*5. You might be noticing a pattern here. We consider ever expanding groups, all nested within the next larger interval, which will be the product of the first N+1 primes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you have identified which numbers mod 30 must be composite: {0,2,3,4,6,8,9,10,12,14,15,16,18,20,21,22,24,26,27,28}. 25 is the first possible prime that turns out to be composite, but that's OK because you would have tested it according to the pattern you found. Because 25 is composite you add it to the list of numbers you will eliminate when you consider the next larger interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly to the way you eliminated composites in the interval [0,29], you can do the same for [30,59], [60,89], [90,119], and so forth up to [180,209]. You test possible primes along the way and add to the list the ones that turn out to be composite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now your next larger interval 210 = 2*3*5*7, the product of the first 4 primes. You repeat the process up to 2310 = 2*3*5*7*11, then 30030, 510510, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To save space you can record flags, indicating which numbers are composite or not, as individual bits within integers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in other simple algorithms for prime testing, you can check Pollard's Rho algorithm. It's a little abstruse but easy to implement. Wikipedia has a nice article describing it.&lt;/p&gt;&#xA;" OwnerUserId="7819" LastEditorUserId="7819" LastEditDate="2014-04-29T04:29:44.220" LastActivityDate="2014-04-29T04:29:44.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11501" PostTypeId="2" ParentId="11490" CreationDate="2014-04-29T14:47:26.287" Score="2" Body="&lt;p&gt;The other two boundary conditions you need are&#xA;\begin{equation}&#xA;\frac{\partial u_y}{\partial x}=0;\quad&#xA;\frac{\partial u_z}{\partial x}=0.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the symmetry boundary condition is essentially identical to &quot;rigid-smooth&quot; boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="6775" LastActivityDate="2014-04-29T14:47:26.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11502" PostTypeId="2" ParentId="11023" CreationDate="2014-04-29T15:42:44.660" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Condition_number#Non-linear&quot; rel=&quot;nofollow&quot;&gt;The condition number for a nonlinear function $f$&lt;/a&gt; is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ cond(f) = \frac{x f'(x)}{f(x)} $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;so for $f(x) = ln(x)(1+\epsilon)$ that's&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{x (1/x)(1+\epsilon)}{ln(x)(1+\epsilon)} = \frac{1}{ln(x)}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and as Martin stated, don't use &quot;algorithm&quot; and &quot;function&quot; interchangeably. &lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-04-29T15:42:44.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11503" PostTypeId="2" ParentId="11495" CreationDate="2014-04-29T17:54:47.020" Score="2" Body="&lt;p&gt;I didn't check both mass matrices (yours and the one from the picture) element-by-element, but your matrix seems more correct. It can be easily estimated by diagonal elements: since local mass matrix is constant for each element and has only 2 on its diagonal (regardless the matrix coefficient), the diagonal elements of global matrix divided by 2 should give the number of triangles to which the corresponding mesh vertex belongs. Say, the vertex 1 belongs to 2 triangles, so the diagonal element 1 of the global matrix should have 2*2 = 4. Vertex 2 belongs to 3 triangles, so we should have 2*3=6 for the diagonal of the second row of the global matrix. In your matrix it's 6, but in the matrix from the picture it's 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a request to check the whole matrix again to make sure that your matrix is correct, and the matrix from the text is not, I wrote this simple matrix assembling function in Octave. It doesn't take into account the coefficient, since it's the same for the local and the global matrices, and the question is about assembling only:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;globalM = zeros(12, 12);&#xA;localM = [ [2 1 1]; [1 2 1]; [1 1 2] ];&#xA;connect = [ [1 4 5]; [1 5 2]; [2 5 6]; [2 6 3]; [4 7 8]; [4 8 5]; [5 8 9]; [5 9 6]; [7 10 11]; [7 11 8]; [8 11 12]; [8 12 9] ];&#xA;for el = 1:12&#xA;  for i = 1:3&#xA;    ii = connect(el, i);&#xA;    for j = 1:3&#xA;      jj = connect(el, j);&#xA;      globalM(ii, jj) = globalM(ii, jj) + localM(i, j);&#xA;    end&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result clearly shows your matrix is correct:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4    1    0    1    2    0    0    0    0    0    0    0&#xA;1    6    1    0    2    2    0    0    0    0    0    0&#xA;0    1    2    0    0    1    0    0    0    0    0    0&#xA;1    0    0    6    2    0    1    2    0    0    0    0&#xA;2    2    0    2   12    2    0    2    2    0    0    0&#xA;0    2    1    0    2    6    0    0    1    0    0    0&#xA;0    0    0    1    0    0    6    2    0    1    2    0&#xA;0    0    0    2    2    0    2   12    2    0    2    2&#xA;0    0    0    0    2    1    0    2    6    0    0    1&#xA;0    0    0    0    0    0    1    0    0    2    1    0&#xA;0    0    0    0    0    0    2    2    0    1    6    1&#xA;0    0    0    0    0    0    0    2    1    0    1    4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3576" LastEditorUserId="3576" LastEditDate="2014-05-03T21:50:27.010" LastActivityDate="2014-05-03T21:50:27.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11504" PostTypeId="1" CreationDate="2014-04-29T23:42:17.323" Score="0" ViewCount="742" Body="&lt;p&gt;I am trying to solve a small QP problem in CPLEX. The problem has several linear constraints. The optimiser runs and finds a solution which satisfies these constraints and CPLEX returns a success code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the value of the objective function given by this solution is greater than that for the solution if I just supply a vector of 1's (this is my initial state which I trying to improve upon). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So CPLEX is failing to revert to the initial solution even if that is better than the given solution. How can this be possible ? I have tested this in Excel using Excel solver and it does something similar so CPLEX is not the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had always assumed that QP problems possessed a unique solution. I believe that the QP problem has a positive semi-definite matrix. In fact I know that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing incorrectly ? Here is the mps file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;www.filedropper.com/quadraticfail&lt;/p&gt;&#xA;" OwnerUserId="7997" LastEditorUserId="7997" LastEditDate="2014-04-30T07:46:05.580" LastActivityDate="2015-07-24T20:22:57.777" Title="CPLEX claims to have solved QP minimisation but solution is not optimal" Tags="|convex-optimization|nonlinear-programming|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11505" PostTypeId="2" ParentId="11504" CreationDate="2014-04-30T07:27:32.300" Score="1" Body="&lt;p&gt;If the solvers are using interior-point methods, then they might completely ignore your initial solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may try also MOSEK, which is free for academics or in trial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you could provide the mps file, I can try it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;running MOSEK I got these results&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/andreacassioli/11425618&quot; rel=&quot;nofollow&quot;&gt;https://gist.github.com/andreacassioli/11425618&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but there was some error in the mps file, so I post it in the github.&lt;/p&gt;&#xA;" OwnerUserId="7396" LastEditorUserId="7396" LastEditDate="2014-04-30T12:32:19.973" LastActivityDate="2014-04-30T12:32:19.973" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11506" PostTypeId="1" CreationDate="2014-04-30T07:49:57.297" Score="1" ViewCount="120" Body="&lt;p&gt;We am currently working on a project related to computational optimization, in which I have tasked myself with calculating the most efficient solution to one of the game's levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link to the flash game: &lt;a href=&quot;http://trackmill.com/virus-2/&quot; rel=&quot;nofollow&quot;&gt;http://trackmill.com/virus-2/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hexagon grid is randomized in color, and the player starts out in one of the corners of the grid. The player selects a color from a short set of predefined colors corresponding to those present in the grid. When the color of the starting hexagon changes by this action, every neighboring hexagon with the same color now also changes, when the player selects a new color. In this fashion, the player must make the entire grid the same color, within a limited number of moves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplification, we will use squares instead of hexagons, and use only three colors. The algorithm should be able to calculate order in which we should select colors resulting in a win in the least amount of turns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, we are trying to calculate the most efficient solution by way of brute force, testing every single imaginable combination, but how can we do this smarter? Which algorithms can we use / adapt?&lt;/p&gt;&#xA;" OwnerUserId="8086" LastActivityDate="2014-04-30T07:49:57.297" Title="Solving &quot;virus&quot;, a flash game, in the least amount of attempts" Tags="|optimization|algorithms|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11507" PostTypeId="1" AcceptedAnswerId="11542" CreationDate="2014-04-30T09:15:21.980" Score="0" ViewCount="484" Body="&lt;p&gt;I wrote a script that plots the results of the &quot;drunken lamppost&quot; problem in MATLAB. &#xA;Now I need to create a road-width from -3 to +3, length from 0 to infinity but the drunk can walk just ahead. It means that when he reaches the road's edge MATLAB will automatically stop plotting.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r=[0 0];&#xA;for t= 0:0.1:100 &#xA;    B1=rand(1,2);&#xA;    B=B1&amp;gt;0.5;&#xA;    if B==1&#xA;        rnew=r+[0 1];&#xA;&#xA;    elseif B(1)==1&#xA;        rnew=r+[-1 1];&#xA;&#xA;    elseif B(2)==1&#xA;        rnew=r+[1 1];&#xA;    end&#xA;&#xA;    hold on;&#xA;    plot([r(1) rnew(1)],[r(2) rnew(2)]);&#xA;    drawnow ;&#xA;    r=rnew;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q8OjG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7916" LastEditorUserId="20688" LastEditDate="2019-06-22T22:25:36.613" LastActivityDate="2019-06-22T22:25:36.613" Title="Drunken Man in Matlab" Tags="|matlab|algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11508" PostTypeId="1" AcceptedAnswerId="11512" CreationDate="2014-04-30T15:47:37.000" Score="2" ViewCount="35" Body="&lt;p&gt;I am submitting jobs to Open Science Grid from a glidein-wms enabled submit hosts such as osg-xsede. I'd like to stage some data on SE nearest to osg-xsede submit host, then distribute data to various SEs available at each destination clusters (more or less as a cache), and access the data on each SE from the worker nodes during the job submission.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I distribute data to various SE where osg-xsede will be submitting jobs to? How can make sure that my jobs will be submitted to a cluster with SEs that has my data? Finally, is there a good way to discover SE endpoints from various worker nodes? Is there any documentation written on this subject?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&#xA;Soichi&lt;/p&gt;&#xA;" OwnerUserId="7467" LastActivityDate="2014-05-01T18:40:40.837" Title="How can I discover / access various SE (storage elements) in GlideinWMS environment?" Tags="|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11509" PostTypeId="2" ParentId="7001" CreationDate="2014-05-01T07:33:06.627" Score="9" Body="&lt;p&gt;I have implemented quite a wide variety of non-linear solvers on the GPU, including LBFGS, Barzilai Borwein gradient descent and non-linear conjugate gradient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this, &lt;a href=&quot;http://link.springer.com/article/10.1023/A:1012930416777&quot;&gt;the non-linear conjugate gradient of Dai &amp;amp; Yuan&lt;/a&gt; has been the most efficient. In general, other version of the nonlinear conjugate gradient may be more efficient (such as CG-DESCENT), but can also be trickier to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LBFGS is in general a very solid choice, and unless you're really strapped for memory it's probably the best place to start. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both the conjugate gradient and BFGS require line searches though, which is where the fp32 becomes a problem. Rather than using the standard Wolfe conditions for the line search, I would suggest using the approximate Wolfe condition suggested &lt;a href=&quot;https://www.math.lsu.edu/~hozhang/papers/cg_descent.pdf&quot;&gt;here&lt;/a&gt;. The paper is a little involved, but the important stuff is equation 4.1. Essentially they explicitly introduce the precision with which you can calculate your function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considerations for the GPU:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have a lot of small problems, which is slightly different from my use case of one large problem. Consider running 1 problem per GPU block (or warp, rather) if you can parallelize function and gradient evaluations to use all the threads in a block. That way it's not a problem if different problems require a different number of iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not an option, I would go with the LBFGS solver. If your function is well behaved, you might get away with simply using a step size of 1 (avoiding the line search) and just running all the problems for a fixed number of iterations.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-05-01T07:33:06.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11510" PostTypeId="1" CreationDate="2014-05-01T12:06:04.727" Score="3" ViewCount="1325" Body="&lt;p&gt;I have a mathematical program with a constraint involving a maximum function. More specifically, the constraint is: $y = \max\{a_i x_i:1 \leq i \leq n\}$ where $a_i$ are constants and $x_i$ are binary variables. Can we express this constraint as a linear constraint? &lt;/p&gt;&#xA;" OwnerUserId="8093" LastEditorUserId="276" LastEditDate="2016-06-01T21:33:52.203" LastActivityDate="2016-06-01T21:33:52.203" Title="Can we express max constraint as a linear constraint?" Tags="|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11511" PostTypeId="2" ParentId="11510" CreationDate="2014-05-01T14:45:33.293" Score="5" Body="&lt;p&gt;First note that because your $x_{i}$ are binary variables you aren't really in the world of linear programming any more.  Rather, this problem is a mixed integer linear programming problem (MILP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on your other constraints and objective, this  &lt;em&gt;may&lt;/em&gt; be doable by simply minimizing $y$ in the objective and enforcing the constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y \geq a_{i}x_{i}$ for $i=1, 2, \ldots, n$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, you can do this as follows.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to assume that the constants $a_{i}$ are positive.  If not, the same general approach can handle the case where some $a_{i} \leq 0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll also assume that the coefficients have been sorted so that $a_{1} \geq a_{2} \geq \ldots \geq a_{n}$.  This is just to simplify the notation- you can keep them in the original order if you're willing to keep track of the subscripts in writing the constraints that are given below.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Add binary variables $u_{i}$, $i=1, 2, \ldots, n$.  Add the constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i=1}^{n} u_{i}=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This ensures that exactly one of the $u_{i}$ is 1.  We will arrange things so that if $u_{k}=1$, then $k$ is the index of the $x$ variable that corresponds to the maximum of $a_{i}x_{i}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, add the constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i=1}^{k} x_{i} \leq n \sum_{i=1}^{k} u_{k} \;\; $, for $k=1, 2, \ldots, n$.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This ensures that if $u_{1}=u_{2}=\ldots=u_{k}=0$, then $x_{i}=0$ for $i=1, 2, \ldots k$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, add the constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(1-x_{i}) + u_{i} \leq 1$, for $i=1, 2, \ldots, n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This ensures that if $u_{k}=1$, then $x_{k}=1$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The combined effect of the above constraints is that if $u_{k}=1$, then $k$ is the index of the first $x_{i}$ variable that is 1.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, add the constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y=\sum_{i=1}^{n} a_{i}u_{i}$.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2014-05-01T16:31:55.380" LastActivityDate="2014-05-01T16:31:55.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11512" PostTypeId="2" ParentId="11508" CreationDate="2014-05-01T18:40:40.837" Score="1" Body="&lt;p&gt;The simple answer is: There is no uniform and production method to send data to many storage elements.  There isn't even a uniform method for requesting access to storage elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few things to remember:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Not all CE's have corresponding SE's.&lt;/li&gt;&#xA;&lt;li&gt;Not all SE's have corresponding CE's (though, most do).&lt;/li&gt;&#xA;&lt;li&gt;Each SE is managed by a different group of people, with their own authentication methods.  Just because you have access to their computing, does not mean you have access to their storage.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The only successful consumer of opportunistic storage on the OSG that I know of is DZero.  They went out and asked a small set of storage elements to host their data, and they wrote a custom framework to do what you are asking: maintain the cache, data discovery...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary: All the tools are available, SRM, GridFTP, GlideinWMS advertising.  It's a people problem.  You need to request storage space on all the SE's you want to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may be better off doing what SCEC and SBGrid did, cozy up with 1-2 storage elements, and only use them.  Storage elements have a lot of bandwidth available to other compute elements, I wouldn't worry too much about storage being 'close' to the computing.  CMS and Atlas already discovered that's not as important as once thought (see Xrootd projects, AAA and FAX).&lt;/p&gt;&#xA;" OwnerUserId="7466" LastActivityDate="2014-05-01T18:40:40.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11514" PostTypeId="1" AcceptedAnswerId="11524" CreationDate="2014-05-02T10:06:41.067" Score="20" ViewCount="54278" Body="&lt;p&gt;Which one is better: FORTRAN or Python? And I guess that in both cases you need Gnuplot, am I right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on a Windows machine at the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to use it to get numerical solutions for physics-problems, including Monte-Carlo simulations, numerical integration and differentiation, molecular dynamics, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw a course on computational physics which introduces both FORTRAN (77 I believe) and Python. I'm planning to start with one and then learn the other, but I don't know which transition might be the easiest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also which compilers would you recommend?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic question for me comes down to: which one is the easiest to learn, which one is the fastest, which one is most user-friendly and above all which one is most used (so a comparison of these 4)? And next to that, what are the most common (free or paid) compilers in use? I'm currently considering converting an old laptop (early Intel dual core) to Linux; hopefully that's fast enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot for the answers so far! The answers which are in line with what I'm looking for are those of LKlevin and SAAD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the basics of C++, Maple and I master MATLAB and Mathematica9 almost completely if that's any help.&lt;/p&gt;&#xA;" OwnerUserId="4811" LastEditorUserId="8168" LastEditDate="2014-05-11T15:52:00.543" LastActivityDate="2022-10-07T17:08:22.967" Title="Python vs FORTRAN" Tags="|python|fortran|" AnswerCount="6" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="11515" PostTypeId="1" AcceptedAnswerId="11543" CreationDate="2014-05-02T11:50:37.000" Score="1" ViewCount="199" Body="&lt;p&gt;I have dense complex matrices $M(z)$ in which each element $M_{ij} = M_{ij}(z)$&#xA;depends on a complex parameter $z$. I need to find $z$ such that the matrix $M$&#xA;gets singular, i.e. I am looking for null vectors $\vec v$ which satisfy $M(z)&#xA;\vec k = \vec 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I use a Newton method. As the wanted $M(z)$ is singular, I need&#xA;$M$'s determinant to vanish. Starting with an initial guess $z_0$, I iterate&#xA;$$z_{i+1} = z_i - \frac{g(z_i)}{g'(z_i)}$$&#xA;with $g(z) := \det M(z)$. To avoid computing the determinant from definition, I&#xA;use LU factorization. The iteration is stopped when $|z_{i+1} -&#xA;z_{i}|$ is sufficiently small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the matrix identity $\frac{\mathrm{d}}{\mathrm{d}z}\ln \det M(z) = \mathrm{trace}\,(M^{-1}(z) M'(z))$, the reciprocal logarithmic derivative in the iteration formula can replaced to yield&#xA;$$z_{i+1} = z_i - \frac{1}{\mathrm{trace}\,(M^{-1}(z) M'(z))}\,,$$&#xA;which is what I use in the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few details on steps involved: so far, I compute all derivatives $g'(z)$ from&#xA;a forward finite difference scheme&#xA;$g'(z) = \frac{g(z + h) - g(z)}{h} - \mathrm{i}\frac{g(z + \mathrm{i}h) - g(z)}{h}$,&#xA;($h$ real), likewise for derivatives of $M$. Also note I'm assuming that&#xA;singular vectors have multiplicity $m = 1$ (although adaptation of the formulas&#xA;above is possible using $g=(\det M(z))^m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $M$ has no special structure in global (i.e. not Hermitian). Depending on the problem, it may be rectangular or square. In any case, it is dense and well-conditioned. Typical sizes range from ~ $100\times100$ to $10k \times 10k$. $M$'s origin is in the boundary element method. For open systems (that may have &lt;a href=&quot;http://en.wikipedia.org/wiki/Resonance#Optical_resonance&quot; rel=&quot;nofollow&quot;&gt;resonances&lt;/a&gt;), the null vectors I am looking are the resonance wavefunctions on the boundaries of the domains under consideration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to learn about alternative methods of adjusting the parameter $z$ such&#xA;that $M$ gets singular. Do you have any other ideas? Or any comments on the&#xA;method I described? Although the results are fine, I don't like the approach too much as both iterations described here involve a step that is&#xA;essentially $O(N^3)$ (LU or the inverse). Furthermore, for the second scheme, I&#xA;need to compute the inverse of near singular matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your input!&lt;/p&gt;&#xA;" OwnerUserId="1553" LastEditorUserId="1553" LastEditDate="2014-05-05T14:25:48.797" LastActivityDate="2018-12-08T11:58:39.590" Title="Finding null vectors of a parameter-dependent matrix" Tags="|iterative-method|newton-method|nullspace|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11516" PostTypeId="2" ParentId="11514" CreationDate="2014-05-02T12:18:05.173" Score="8" Body="&lt;p&gt;I would stay away from Fortan, or if you must, use a reasonably new version (2003 rather than 77). A lot of physics software (Monte Carlo simulations in particular) is written in Fortran, simply because the projects were originally started in the 80s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, python and Fortran are two very different language, and what they should be used for is quite different. Python is high level and in general not that fast (compared with Fortran &amp;amp; C++). The reason it's being used so much is that it's fast enough for most things and has excellent (Fortran powered) libraries  for many (but not all) of the things you would like to do. It also has the excellent Matplotlib for plotting (so no need for GNUplot) and you can get quite decent performance by using stuff like Cython for writing the expensive bits. It will not be as fast as Fortran or C++ however, and parallelisation is pretty terrible, making it inadequate for high performance numerical computing. If what you want can be handled by calling Fortran or C libraries however, python makes an excellent language for gluing things together and analysing your data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortran is a somewhat lower level language. For numerics the library support is surprisingly good, but is still very low level giving you a hoard of bugs you could otherwise avoid, such as accidentally passing the wrong array size to a method. These bugs are hard to find and you might not notice them at all. Trust me, I spent quite a while writing Fortran 77.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++ is (in my humble opinion) a happy medium. With libraries like Armadillo or Eigen, you can get away with a fairly high level style of coding while getting a low level style of performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speaking of performance, the only real choice for numeric python right now is CPython. If you download something like &lt;a href=&quot;http://winpython.sourceforge.net/&quot;&gt;WinPython&lt;/a&gt; you will also get the majority of the libraries you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Fortran on windows, things are a little harder. I would recommend switching to linux and using either gfortran or Intels ifort compiler. Ifort tends to be faster for numeric code in my experience, but is only free for non-commerical, non-academic use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum up: Unless you want to run really heavy simulations, python is by far the easier choice and much more enjoyable to work with. It should also be fast enough for most student level projects. If you need better performance, start by looking at the wast amounts of libraries already written and let that decide your language.&#xA;If you have to write things from scratch, use C++. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also a warning: most code written by physicists is quite terrible, presumably because physicists have the tendency to assume programming is easy and doesn't require the same rigour that they might use in mathematics. Consider taking a class or buying a book that teaches programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I'm a physicist who has spent quite some time with Fortran 77 based Monte Carlo codes and currently does all his data processing in Python. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-05-02T12:18:05.173" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11517" PostTypeId="1" CreationDate="2014-05-02T15:08:44.187" Score="2" ViewCount="441" Body="&lt;p&gt;I am trying to simulate a moving airfoil with constant speed (Mach=0.755, aoa=1.25). I solve Euler equations with Roe's method. I have two boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Farfield&lt;/li&gt;&#xA;&lt;li&gt;Slip wall (airfoil)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For all faces I gave face velocities as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;vbx = -Mach cos (aoa)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;vby = -Mach sin (aoa)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the farfield, I set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(rho,u,v,p) = (1, 0, 0, 1/gamma)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where gamma is the ratio of specific heats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For slip wall I set velocity as the tangential velocity of the neighbor near boundary cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The flow is initialized with zero (u,v) everywhere. But with these boundary conditions residual is zero so there is no point to iterate. Residual is the summation of fluxes where flux is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fn0 = rho*(un-vb)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fn1 = rho*(un-vb)*u + p*nx&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fn2 = rho*(un-vb)*v + p*ny&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fn3 = E*(un-vb) + p*un&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how should be the proper boundary conditions?&lt;/p&gt;&#xA;" OwnerUserId="8103" LastEditorUserId="4244" LastEditDate="2014-09-05T08:12:05.643" LastActivityDate="2015-04-03T13:01:02.583" Title="Moving airfoil boundary conditions" Tags="|fluid-dynamics|boundary-conditions|finite-volume|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11518" PostTypeId="2" ParentId="11514" CreationDate="2014-05-02T15:18:15.453" Score="1" Body="&lt;p&gt;I would keep using MATLAB, it calls fast math libraries, and you will not see much difference in performance by switching to FORTRAN on Windows. At the same time you'll have a better infrastructure in MATLAB for reporting of results and running your code. The downside of MATLAB is its cost. FORTRAN is basically free, and there's a bunch of free libraries out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FORTRAN is very easy to learn and start programming in. It basically does what the name suggests: translates your formulas into code, which is easy to read and understand. That's why physicists used it a lot in old days. As long as most of your code is about solving physics problems (not building GUIs or doing other cool stuff), FORTRAN code will be easy to maintain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend Python only if you enjoy programming. Think of this: when you code a solution to physics problem do you enjoy programming part of the solution? If you do, then Python is an option, because the language is much better than MATLAB's. &lt;/p&gt;&#xA;" OwnerUserId="7470" LastActivityDate="2014-05-02T15:18:15.453" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11520" PostTypeId="2" ParentId="11514" CreationDate="2014-05-02T16:15:08.007" Score="11" Body="&lt;p&gt;Python is a very slow, high level language. For fast number crunching you'll have to write the main compute kernels in low level languages like C/C++ which means that now you have to learn not one but at least two languages. You'll also have to deal with additional headache associated with debugging/installation/maintenance etc. Most people use Python as a syntactic sugar to hide shortcomings of C/C++.&lt;/p&gt;&#xA;&lt;p&gt;Modern Fortran (90 and later) is both fast and high level with almost MATLAB like syntax. So you can do things like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;k=k+matmul(transpose(B),matmul(D,B))*weight(i)*detj&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;indx(:)=indxmap(indx(:),2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or even simpler&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;indx=indxmap(indx,2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&lt;p&gt;On Linux there are a number of free Fortran compilers. I use&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;GCC&lt;/li&gt;&#xA;&lt;li&gt;Solaris Studio&lt;/li&gt;&#xA;&lt;li&gt;Open64&lt;/li&gt;&#xA;&lt;li&gt;Intel (non-commercial use only)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I don't use Macs/OSX but there is free PGI.&lt;/p&gt;&#xA;&lt;p&gt;And please don't use FORTRAN 77. No one uses it to write new code.&lt;/p&gt;&#xA;&lt;p&gt;Disclaimer: I personally looked at Python for writing my own small unstructured FE code (built top of PETSc) but the amount of work/coding involved was more than just writing it in plain Fortran 95.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="22284" LastEditDate="2022-10-07T17:08:22.967" LastActivityDate="2022-10-07T17:08:22.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11521" PostTypeId="2" ParentId="11514" CreationDate="2014-05-02T18:45:53.323" Score="5" Body="&lt;p&gt;&lt;strong&gt;Python&lt;/strong&gt;&#xA; is very practical for full simulation analysis with well-documented versatile packages: grid generation, array computation and data structure handling (&lt;a href=&quot;http://www.numpy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;numpy&lt;/a&gt; and &lt;a href=&quot;http://pandas.pydata.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pandas&lt;/a&gt;) as well as data visualization with matplotlib.&#xA;For complex simulations with big result files, it's even better to work with the VTK package which allows exporting data to be read by advanced open-source applications (like Paraview or Visit)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fortran&lt;/strong&gt; has been for some time the preferred language for different domains in simulations. It's easily readable (less readable than Python code though). Array handling is one the language's strong points, quite easy to define and use in use all kinds of array operations. It comes handy when debugging also. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comparison gets down to &lt;strong&gt;performance&lt;/strong&gt;: I've only done big scale computations using compiled languages (C++ and Fortran 90) but never with Python. Another thread gives more performance information about interpreted and compiled languages:&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/11006/computational-physics-python-vs-c-c&quot;&gt;What language should I use when teaching an undergraduate course in computer programming?&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I like working with Python in general, especially for post-processing. Python programming is fun ! &lt;/p&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-05-02T18:45:53.323" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11522" PostTypeId="2" ParentId="2150" CreationDate="2014-05-02T19:18:07.893" Score="6" Body="&lt;p&gt;The Wikipedia article on this topic is rather bad (from an educational point of view) and somewhat misleading.  Do not use it.  You'll find a better summary of the method &lt;a href=&quot;http://dx.doi.org/10.1145/1921598.1921603&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  In particular, I find it confusing to call $a_k$ a &lt;em&gt;step sequence&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To get an intuitive understanding of the method, let's look at Newton's method for the (deterministic) function $M(x)$ first.  The iteration looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_{n+1} = x_{n} - \frac{1}{M'(x_n)} M(x_n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that if $a_n = 1/M'(x_n)$ then the Robbins-Monro iteration is just Newton's method.  In fact to make the Robbins-Monro iteration work well, one needs to start with an estimate of $M'(x_n)$ (as described in the paper I linked).  This is the first step towards the intuitive understanding of the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's write $N(x) = M(x) - \eta$ where $\eta$ is a noise term with mean zero.  Then the sequence of iterates will be&#xA;$$x_{n+1} = x_n - a_n N(x_n) =  x_n - a_n M(x_n) + a_n \eta_n.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the iteration progresses, the noise terms $a_n\eta_n$ get added up.  If the $a_n$ sequence is decreasing then the noise will decrease too.  (This might not be entirely obvious from the formula above, but it can be shown with more rigorous calculations.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To better understand what is happening during iteration, let us study the case when $M(x) = \alpha (x - \bar x)$, i.e. $M(x)$ is a linear function with root $\bar x$.  This is usually a good approximation in a narrow enough interval around the root.  Let us choose $a_n = a/M'(x_n) = a/\alpha$.  In other words, the number $a$ tells us how close we are to the exact Newton iteration (sans noise).  If $a=1$ then we re-gain Newton's method, plus the noise term.  If $a$ is in a certain interval around 1, then the iteration will still converge to the root $\bar x$.  Let's find this interval:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sequence will be  $$x_{n+1} = x_n (1-a) + a \bar x + a \eta_n/\alpha,$$ or in terms of the staring value $x_0$, $$x_n = (1 - (1-a)^n)\bar x + x_0 (1-a)^n + \text{noise}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Newton-like part of this iteration converges if $0 &amp;lt; a &amp;lt; 2$, and the convergence is fastest if $a=1$.  But what about the noise term?  It can be shown that the noise in $x_n$ depends monotonically on $a$ for $0&amp;lt;a&amp;lt;2$ and it gets smaller as $a$ decreases.  Thus we have a tradeoff between fast convergence of the Newton-like part and effective averaging of the noise.  If $a$ is small then we get good averaging of the noise but slow convergence of the deterministic part and vice versa.  A decreasing $a_n$ sequence helps with faster convergence initially and more effective averaging of the noise in later steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2017-01-02T08:20:35.007" LastActivityDate="2017-01-02T08:20:35.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11523" PostTypeId="2" ParentId="11514" CreationDate="2014-05-02T20:48:40.797" Score="3" Body="&lt;p&gt;With Python you don't need Gnuplot, you can use, for example, matplotlib and/or use the IPython shell. IPython is an interactive Python shell that, in %pylab mode, provides pretty much the same plotting commands you have available in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is quite likely that scientific computation will to a great extend shift from MATLAB to Python in the next 5+ years. &lt;/p&gt;&#xA;" OwnerUserId="8099" LastEditorUserId="-1" LastEditDate="2014-05-22T15:50:44.200" LastActivityDate="2014-05-22T15:50:44.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11524" PostTypeId="2" ParentId="11514" CreationDate="2014-05-03T10:54:56.823" Score="37" Body="&lt;h1&gt;Ease of learning&lt;/h1&gt;&#xA;&lt;p&gt;Python and Fortran are both relatively easy-to-learn languages. It's probably easier to find good Python learning materials than good Fortran learning materials because Python is used more widely, and Fortran is currently considered a &amp;quot;specialty&amp;quot; language for numerical computing.&lt;/p&gt;&#xA;&lt;p&gt;I believe the transition from Python to Fortran would be easier. Python is an interpreted language, so the number of steps it takes to get your first program running is smaller (open the interpreter, type &lt;code&gt;print(&amp;quot;Hello, world!&amp;quot;)&lt;/code&gt; at the prompt) than it is for Fortran (write a &amp;quot;Hello world&amp;quot; program, compile, run). I also think that there are better materials to teach object-oriented style in Python than in Fortran, and there's more Python code available on GitHub than Fortran code.&lt;/p&gt;&#xA;&lt;h1&gt;Getting up and running on Windows&lt;/h1&gt;&#xA;&lt;p&gt;Installing Python should be less painful; there are Windows distributions available. I recommend using a scientific distribution like Anaconda or Enthought Canopy. There's not really a compiler, per se; the interpreter takes that role. You'll want to use a CPython-based interpreter, because there are more numerical libraries available and it interoperates nicely with C, C++, and Fortran. Other interpreter implementations include Jython and PyPy.&lt;/p&gt;&#xA;&lt;p&gt;On a Windows machine, installing a Fortran compiler is going to be annoying. Typical command-line compilers are programs like gfortran, ifort (from Intel; free for personal use, otherwise costs money), and pgfortran (from PGI; free trial versions, otherwise costs money). To install these compilers, you might need to install some sort of UNIX/POSIX-type compatibility layer, like Cygwin or MinGW. I found it a pain to work with, but some people like that workflow. You could also install a compiler with a GUI, like Visual Fortran (again, you'd have to pay for a license). Windows Subsystem for Linux (WSL) could also be used to install gfortran compiler in Windows.&lt;/p&gt;&#xA;&lt;p&gt;On Linux, it will be easier to install Python and compilers; I would still install Anaconda or Enthought Canopy as a Python distribution.&lt;/p&gt;&#xA;&lt;h1&gt;Speed: a productivity vs. performance tradeoff&lt;/h1&gt;&#xA;&lt;p&gt;In using Python (or MATLAB, Mathematica, Maple, or any interpreted language), you give up performance for productivity. Compared to Fortran (or C++, C, or any other compiled language), you will write fewer lines of code to accomplish the same task, which generally means it will take you less time to get a working solution.&lt;/p&gt;&#xA;&lt;p&gt;The effective performance penalty for using Python varies, and is mitigated by delegating computationally intensive tasks to compiled languages. MATLAB does something similar. When you do a matrix multiplication in MATLAB, it calls BLAS; the performance penalty is virtually zero, and you didn't have to write any Fortran, C, or C++ to get the high performance. A similar situation exists in Python. If you can use libraries (for example, NumPy, SciPy, petsc4py, dolfin from FEniCS, PyClaw), you can write all of your code in Python and get good performance (a penalty of maybe 10-40%) because all of the computationally intensive parts are calls to fast compiled language libraries. However, if you were to write everything in pure Python, the performance penalty would be a factor of 100-1000x. So if you wanted to use Python and had to include a custom, computationally intensive routine, you would be better off writing that part in a compiled language like C, C++, or Fortran, then wrapping it with a Python interface. There are libraries that facilitate this process (like Cython and f2py), and tutorials to help you; it is generally not onerous.&lt;/p&gt;&#xA;&lt;h1&gt;Scope of use&lt;/h1&gt;&#xA;&lt;p&gt;Python is used more widely overall as a general-purpose language. Fortran is largely limited to numerical and scientific computing, and is mainly competing with C and C++ for users in that domain.&lt;/p&gt;&#xA;&lt;p&gt;In computational science, Python typically doesn't compete directly with compiled languages due to the performance penalties I mentioned. You would use Python for cases where you want high productivity and performance is a secondary consideration, such as in prototyping numerically intensive algorithms, data processing, and visualization. You would use Fortran (or another compiled language) when you have a good idea of what your algorithm and application design should be, you're willing to spend more time writing and debugging your code, and performance is paramount. (For instance, performance is a limiting step in your simulation process, or it is a key deliverable in your research.) A common strategy is to mix Python and a compiled language (usually C or C++, but Fortran has been used also), and only use the compiled language for the most performance-sensitive parts of the code; the development cost is, of course, that it's harder to write and debug a program in two languages than a program in a single language.&lt;/p&gt;&#xA;&lt;p&gt;In terms of parallelism, the current MPI standard (MPI-3) has native Fortran and C bindings. The MPI-2 standard had native C++ bindings, but MPI-3 does not, and you would have to use the C bindings. Third-party MPI bindings exist, such as mpi4py. I've used mpi4py; it works well, and is straightforward to use. For large-scale parallelism (tens of thousands of cores), you'd probably want to use a compiled language because things like dynamically loading the Python modules will bite you in the ass at scale if you do it in a naïve way. There are ways to get around that bottleneck, as demonstrated by the PyClaw developers, but it's simpler to avoid it.&lt;/p&gt;&#xA;&lt;h1&gt;Personal opinions&lt;/h1&gt;&#xA;&lt;p&gt;I have roughly a decade of experience in Fortran 90/95, and I've also programmed in Fortran 2003. I have roughly five years of experience programming in Python. I use Python much more than I use Fortran because, frankly, I get more done in Python. The majority of the work I need to do does not require major supercomputing resources and is generally not worth re-developing in another language, so Python is just fine for solving ODEs and PDEs. If I need to use a compiled language, I will use C, C++, or Fortran, in that order.&lt;/p&gt;&#xA;&lt;p&gt;Most of the Fortran code I've seen has been ugly, mainly because most of the computational science community seems unaware of or averse to any best practices discovered by software engineers in the last 30 years. To wit: there is no good unit testing framework in Fortran. (The best I came across is FUnit, by NASA, and that's not maintained anymore.) There are a few good Python unit testing frameworks, good Python documentation generators, and generally many better examples of good programming practices.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="31686" LastEditDate="2020-07-02T20:02:05.320" LastActivityDate="2020-07-02T20:02:05.320" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11525" PostTypeId="2" ParentId="11321" CreationDate="2014-05-03T14:30:44.640" Score="0" Body="&lt;p&gt;you can use in site for more information perhaps this help you.&#xA;&lt;a href=&quot;http://aam.mathematik.uni-freiburg.de/IAM/Research/projectskr/lin_solver/&quot; rel=&quot;nofollow&quot;&gt;http://aam.mathematik.uni-freiburg.de/IAM/Research/projectskr/lin_solver/&lt;/a&gt;   &lt;/p&gt;&#xA;" OwnerUserId="8107" LastActivityDate="2014-05-03T14:30:44.640" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11526" PostTypeId="1" AcceptedAnswerId="11711" CreationDate="2014-05-03T16:58:36.273" Score="1" ViewCount="333" Body="&lt;p&gt;The problem comes from the need to estimate the error propagation of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Matrix_norm#Induced_norm&quot; rel=&quot;nofollow&quot;&gt;spectral norm&lt;/a&gt; computation of a square matrix $A$ of which I know the components' $A_{ij}$ absolute error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fundamental step of the computation of the spectral norm  $||A||_2$ is the computation of the largest eigenvalue of the matrix $A^TA$. In fact the spectral norm is defined as&#xA;$$||A||_2 = \sqrt{\lambda_\mathrm{max}(A^TA)}$$&#xA;where $\lambda_\mathrm{max}(A^TA)$ denotes the maximum eigenvalue of $A^TA$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my algorithm I'm using the GSL library to compute this $\lambda_\mathrm{max}$.&#xA;On it's &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Eigensystems.html&quot; rel=&quot;nofollow&quot;&gt;manual&lt;/a&gt; the GSL states that the algorithm used is a &quot;&lt;em&gt;symmetric bidiagonalization followed by QR reduction&lt;/em&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: given the absolute error of the components $A_{ij}$ how can I estimate the error on the largest eigenvalue of $A^TA$?&#xA;Can you point me on the direction to further investigate the problem?&lt;/p&gt;&#xA;" OwnerUserId="5499" LastEditorUserId="-1" LastEditDate="2015-01-21T15:03:33.857" LastActivityDate="2015-01-21T15:03:33.857" Title="Error propagation on GSL eigenvalues computation" Tags="|matrix|eigenvalues|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11527" PostTypeId="1" CreationDate="2014-05-04T01:15:27.373" Score="1" ViewCount="594" Body="&lt;p&gt;My problem deals with a large $n \times m$ matrix from which I extract and store several square $k \times k$ submatrices. The original matrix may be very large, and I may need to store many thousands of different $k \times k$ submatrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All I need to store is two list of integers, the indices for both rows and columns. For example, it could be a $4 \times 4$ matrix with rows $1, 10, 22, 44$ and columns $10, 90, 110, 111$. If I store all numbers then if I store $10000$ matrices I need to keep $20000k$ numbers in memory, and if possible I would like to avoid that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how I can reduce the storage requirement for a list of indices, or two lists of indices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="1817" LastEditorUserId="348" LastEditDate="2014-05-05T16:43:50.367" LastActivityDate="2014-05-05T16:43:50.367" Title="Most memory-efficient way to store a list of numbers" Tags="|algorithms|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11528" PostTypeId="1" AcceptedAnswerId="12943" CreationDate="2014-05-04T03:23:54.340" Score="2" ViewCount="5576" Body="&lt;p&gt;I would like to import (or parse ??) a &lt;a href=&quot;http://en.wikipedia.org/wiki/VRML&quot; rel=&quot;nofollow noreferrer&quot;&gt;WRL/VRML&lt;/a&gt; file format in python in order to work the data with Matplotlib. Is there any way of converting this file format into plottable numpy arrays ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attached a photo a WRL file visualized with the &lt;a href=&quot;http://freewrl.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FreeWRL&lt;/a&gt; viewer&#xA;&lt;img src=&quot;https://i.stack.imgur.com/tH69O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4964" LastActivityDate="2014-08-21T13:03:49.840" Title="WRL and VRML to Matplotlib/Numpy" Tags="|python|visualization|numpy|scipy|plotting|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11529" PostTypeId="2" ParentId="5110" CreationDate="2014-05-04T06:05:18.830" Score="1" Body="&lt;p&gt;Have you looked at &lt;a href=&quot;http://deeplearning.net/software/theano/&quot; rel=&quot;nofollow&quot;&gt;Theano&lt;/a&gt;? it seems quite &lt;a href=&quot;http://deeplearning.net/tutorial/dA.html&quot; rel=&quot;nofollow&quot;&gt;powerful&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8109" LastActivityDate="2014-05-04T06:05:18.830" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11530" PostTypeId="2" ParentId="11527" CreationDate="2014-05-04T06:06:44.427" Score="1" Body="&lt;p&gt;1) you could use a smaller data type for the index (e.g. a 2-byte integer)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) if you know your sub-matrices have the same relative indices, e.g. rows are always something like (i, i+a, i+b, i+c), you only have to store the index to element (1,1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) similarly, if your sub-matrices are limited in relative index, e.g. your rows are (a, b, c, d) where d-a &amp;lt; 256, you could store the index to element (1,1), then use bytes for the offsets from that element&lt;/p&gt;&#xA;" OwnerUserId="1665" LastActivityDate="2014-05-04T06:06:44.427" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11531" PostTypeId="1" CreationDate="2014-05-04T11:22:35.077" Score="5" ViewCount="3440" Body="&lt;p&gt;I know the Neumann B.C. is implicit in FEM language. However, I have seen at least two ways to impose Dirichlet B.C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. for the following problem 1D,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2 u + \nabla u= 0, u_{left}= 1, u_{right}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) set first and last &lt;em&gt;row&lt;/em&gt; of assembled &quot;A&quot; to &quot;0&quot; at left hand side, set A(1,1)=1,A(end,end)=1, and specify the boundary value &quot;1&quot; and &quot;0&quot; in right hand side vector &quot;b&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) set first &lt;em&gt;row&amp;amp;column&lt;/em&gt;, last &lt;em&gt;row&amp;amp;column&lt;/em&gt; of assembled &quot;A&quot; to &quot;0&quot; at left hand side, then do the same thing as above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;these two methods are different, the first is more intuitive(probably preferred by finite difference user), while the second sounds more rigorous because we are setting the boundary &quot;element&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know these two ways may generate different results for some specific case. Could any body give some insight?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="4532" LastEditDate="2014-05-04T17:07:13.070" LastActivityDate="2014-05-04T21:49:53.703" Title="FEM: which is the correct way to impose Dirichlet B.C" Tags="|finite-element|finite-difference|numerical-analysis|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="11532" PostTypeId="1" AcceptedAnswerId="11533" CreationDate="2014-05-04T14:26:38.150" Score="10" ViewCount="479" Body="&lt;p&gt;I have used FEA for a couple of years now, but using it and using it correctly are two different things, safety factor is not the solution to everything. I have the feeling I won't be using it right unless I have a clear answer to that question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware elements must be close to their ideal shape (based on the Jacobian only?) in order to get accurate results.. But why? Since I understand it comes from a coordinate transform, unless two vectors of the element become colinear shouldn't the results be accurate no matter its shape?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A step-by-step answer based on an illustrated example&lt;/strong&gt; (arbitrary stress distribution) would be ideal, especially given that it is a relatively common question (but never well answered from what I have seen).&lt;/p&gt;&#xA;" OwnerUserId="8112" LastActivityDate="2014-07-10T15:55:56.833" Title="Why do the shape of finite elements matter?" Tags="|finite-element|simulation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11533" PostTypeId="2" ParentId="11532" CreationDate="2014-05-04T15:19:54.263" Score="11" Body="&lt;p&gt;A good introduction to how issues of element shape influence quality and ease of solution, with pictures, is Jon Shewchuk's &quot;What Is a Good Linear Finite Element? Interpolation, Conditioning, Anisotropy, and Quality Measures&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.berkeley.edu/~jrs/papers/elemj.pdf&quot;&gt;http://www.cs.berkeley.edu/~jrs/papers/elemj.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2014-05-04T15:19:54.263" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11534" PostTypeId="2" ParentId="11531" CreationDate="2014-05-04T21:49:53.703" Score="3" Body="&lt;p&gt;I think the typical approach is 1), which can then be reformulated into a smaller system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the approach in &lt;a href=&quot;https://scicomp.stackexchange.com/a/5073/1804&quot;&gt;https://scicomp.stackexchange.com/a/5073/1804&lt;/a&gt;, suppose your finite element basis coefficients make up a vector $\xi$, and that you can partition them into the vectors $\xi_{int}$ and $\xi_{bc}$. Then you should be able to permute your system into something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\left[\begin{array}{cc} A_{int} &amp;amp; A_{int-bc} \\ 0 &amp;amp; I \end{array}\right]&#xA;\left[\begin{array}{c} \xi_{int} \\ \xi_{bc}\end{array}\right] =&#xA;\left[\begin{array}{c} b_{int} \\ b_{bc}\end{array}\right]&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_{int}$ and $A_{int-bc}$ are stiffness-like matrices, $b_{int}$ is the load vector for your system (in this case, a zero vector), and $b_{bc}$ contains your discretized boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this formulation, you can reformulate into a smaller system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second approach sounds like it would give you a singular system, and the smaller system that results would only govern the interior degrees of freedom, which would only work if you had zero Dirichlet conditions.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-05-04T21:49:53.703" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11536" PostTypeId="1" AcceptedAnswerId="11538" CreationDate="2014-05-05T09:10:31.277" Score="2" ViewCount="272" Body="&lt;p&gt;I'm trying to work out an as-general-as-possible 2D Laplace example for Finite Element Analysis. Starting from $\Delta u = 0$ for an unknown $u(x,y)$, I multiply both sides (well, in practice only the left-hand side) by an arbitrary test function $\varphi(x,y)$ and subsequently integrate over the domain $\Omega$. Then, using the divergence theorem (Green's first identity) I obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega \nabla \varphi \nabla u \; d\Omega = \int_\Gamma \varphi \nabla u \cdot \vec{n} \; d\Gamma,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Gamma$ is the boundary of $\Omega$ and $\vec{n}$ the (unit) normal vector on that boundary. Next, I approximate $u$ and $\varphi$ using the same set of shape functions $N$, resulting in $u \approx u^h = \vec{u}^T N(x,y) = N^T(x,y) \vec{u}$, where $\vec{u}$ is the a column of unknown coefficients, and likewise $\varphi \approx \varphi^h = \vec{\varphi}^T N(x,y)$. Substituting this gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\boxed{\vec{\varphi}^T} \int_\Omega \nabla N \nabla N^T \; d\Omega \;\boxed{\vec{u}} = \boxed{\vec{\varphi}} \int_\Gamma N \nabla u \cdot \vec{n} \; d\Gamma.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this has to be valid for arbitrary test functions $\varphi$ and therefore for arbitrary coefficients $\vec{\varphi}$, we can write this as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega \nabla N \nabla N^T \; d\Omega \;\boxed{\vec{u}} = \int_\Gamma N \nabla u \cdot \vec{n} \; d\Gamma.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we need to take the boundary conditions into account. In the case of Dirichlet / essential BCs, we already know some of the coefficients in $\vec{u}$, they are given values for some of the boundary nodes. We can then either partition the system, or use some kind of penalty method to solve the system $K\vec{u} = \vec{q}$, where $K$ is the left-hand side (matrix) and $\vec{q}$ the right-hand side (vector). &lt;strong&gt;However, what to do with Neumann / natural BCs?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I'm referring to general Neumann BCs, so $\nabla u \cdot \vec{n} = g$. If $g = 0$, then it's rather trivial (unfortunately, this is the example almost all FEA books seem to use). However, what to do if $g \neq 0$? I could discretize $\nabla u$ as $\nabla N^T \vec{u}$, and I can evaluate $\vec{n}$ everywhere, but then&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What do I do with this new $\vec{u}$, combine it with the left-hand side $K$?&lt;/li&gt;&#xA;&lt;li&gt;How do I prescribe the Neumann BCs (in discretized form), is it just a value, a difference, something else?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Basically, how do I apply Neumann BCs (and how are they prescribed), and how do I subsequently solve the system $K\vec{u} = \vec{q}$?&lt;/p&gt;&#xA;" OwnerUserId="3879" LastActivityDate="2014-05-05T12:37:40.693" Title="Neumann / natural BCs in FEA" Tags="|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11538" PostTypeId="2" ParentId="11536" CreationDate="2014-05-05T12:07:38.410" Score="4" Body="&lt;p&gt;On the portions of $\Gamma$ where $\nabla u \cdot \vec{n} = g$, you simply substitute that expression into the right-hand-side integral and evaluate it. I.e., you evaluate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_\Gamma N \nabla u \cdot \vec{n} \; d\Gamma = \int_\Gamma N g \; d\Gamma. &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edited to add: You may need to represent $g$ in your finite-element basis in order to evaluate it sanely. It's possible to use exact integration if your $g$ functions are simple, but usually we interpolate $g$ as $g_h$ and then integrate using the same kind of numerical integration that we used to get $K$. There are some tricks here often because $\Gamma$ is one dimension lower than $\Omega$, so you have to be careful when you integrate on such imbedded surfaces.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2014-05-05T12:37:29.967" LastActivityDate="2014-05-05T12:37:29.967" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11539" PostTypeId="2" ParentId="11321" CreationDate="2014-05-05T14:51:00.930" Score="2" Body="&lt;p&gt;Looking at the MKL documentation, I see that MKL has only CG and GMRES solvers, along with ILU preconditioners.  It may be that the GMRES solver will be adequate for your purposes.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-05-05T14:51:00.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11540" PostTypeId="1" CreationDate="2014-05-05T16:25:34.693" Score="6" ViewCount="6381" Body="&lt;p&gt;Recently I started to get interested in GPU-computing.&#xA;I am considering buying a GPU and am inclined to buy an NVIDIA GPU since this allows for coding in both OpenCL and CUDA. But is NVIDIA really the best brand and if so, what type of GPU should I consider buying. I am not looking for a high-end GPU, but for good value for money.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="8121" LastEditorUserId="1552" LastEditDate="2014-05-05T16:38:23.743" LastActivityDate="2014-05-05T20:36:01.967" Title="GPU Computing: Which GPU Brand and Type gives best value for money?" Tags="|parallel-computing|gpu|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11541" PostTypeId="2" ParentId="11540" CreationDate="2014-05-05T16:38:02.493" Score="6" Body="&lt;p&gt;CUDA is a non-trivial advantage for NVIDIA.  Most of the benchmarks I have seen (&lt;a href=&quot;http://viennacl.sourceforge.net/viennacl-benchmarks.html&quot; rel=&quot;noreferrer&quot;&gt;ViennaCL Benchmarks&lt;/a&gt; for reference) show that CUDA does better than OpenCL by up to an order of magnitude.  At the large problem sizes, the two are fairly comparable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, NVIDIA has the advantage that no matter whether you end up using OpenCL or CUDA, you can very quickly upgrade yourself from 400-500 CUDA cores and 1 GB of VRAM at about \$150 up to 2500 cores and 12 GB of RAM at \$5000.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are just starting with GPU computing, I recommend starting in the GTX 500-600 range.  It is enough to get your feet wet, and with the right problem/library you can beat i7 CPUs.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-05-05T16:38:02.493" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11542" PostTypeId="2" ParentId="11507" CreationDate="2014-05-05T17:52:36.027" Score="0" Body="&lt;p&gt;You are already handing every step individually, so as @user2697246 says in the comment you can simply add an if statement checking for your condition, and a break.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words add this right before the &lt;code&gt;end&lt;/code&gt; for your &lt;code&gt;for&lt;/code&gt; loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if ((rnew(2) &amp;lt; -3) || (rnew(2) &amp;gt; 3))&#xA;    break;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do have a comment about your first conditional, &lt;code&gt;B==1&lt;/code&gt;.  &lt;code&gt;B&lt;/code&gt; is a logical matrix, and I can't find in the documentation how matlab handles this.  Please make sure this is doing what you think it is, and consider rewriting it if you can find a better way to express your logic.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-05-05T17:52:36.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11543" PostTypeId="2" ParentId="11515" CreationDate="2014-05-05T20:12:10.017" Score="3" Body="&lt;p&gt;I have used the method you described for a similar problem. I found that a combination of your Newton iteration, along with Brent's method on minimizing the smallest singular value gave pretty good results. I believe I used Newton's method first for a few iterations, then switched over to Brent's. This &lt;a href=&quot;http://www.opticsinfobase.org/josab/abstract.cfm?uri=josab-28-8-1837&quot; rel=&quot;nofollow&quot;&gt;paper of mine&lt;/a&gt; contains the description for the analogous iteration on the SVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem is generally called a &quot;nonlinear eigenvalue problem&quot;. Applying Newton's method as you have done is one standard way of solving it. The literature desribes a lot of methods for it (&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0710059&quot; rel=&quot;nofollow&quot;&gt;for example&lt;/a&gt;). Besides these kinds of iterations to find singular eigenvalues, you can also perform contour integration around the region in the complex plane where you want to compute eigenvalues. There is also a fairly extensive set of literature on this (see &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0010465599004294&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; or &lt;a href=&quot;http://arxiv.org/abs/1003.1580&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;). These links are more or less the first things I found on Google, so please don't think that they are the best examples out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Note that your formula for the finite difference approximation of $g'$ does not need the imaginary correction (just the first term is enough) if your function is analytic. If you can compute the derivatives analytically or automatically (with automatic differentiation), then that would be much better. However, I would guess this is not possible if your matrix comes from a BEM discretization.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2014-05-05T20:18:16.020" LastActivityDate="2014-05-05T20:18:16.020" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11544" PostTypeId="2" ParentId="11540" CreationDate="2014-05-05T20:36:01.967" Score="6" Body="&lt;p&gt;In general, the AMD line of GPUs are faster for integer based calculations, whereas NVIDIA are faster for floating point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NVIDIA also has CUDA (like Godric mentioned) which is a bit easier to work with, and has a very good library support, including cuBLAS, cuFFT and Thrust which make many things far easier to code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CUDA is not itself faster than OpenCL, but NVIDIA cards typically perform better with a CUDA implementation than with an OpenCL one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check the relevant benchmarks at &lt;a href=&quot;http://www.tomshardware.com/charts/workstation-graphics-2013/benchmarks,146.html&quot;&gt;Toms hardware charts&lt;/a&gt;, which will also give you an idea about performance/dollar.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-05-05T20:36:01.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11545" PostTypeId="1" CreationDate="2014-05-06T01:48:03.010" Score="1" ViewCount="1145" Body="&lt;p&gt;I am trying to solve Boltzmann eqn:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{e}{m}\tau\vec{E}_x\cdot \nabla_\vec{v}{f(\vec{v})}=f(\vec{v})-f_0(\vec{v})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_0(v)=\exp{\frac{-m \vec{v}^2}{T}}$ and  $\vec{v}=\{\vec{v}_x,\vec{v}_y\}$ (i.e., $\vec{v}$ is in 2D.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, let's assume $\vec{v}_y=0$.  Assume $\tau$ and $T$ are constants. Then for any given $\vec{v}$ and $|E|$, $f_0(\vec{v})$ is completely known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to set up a self-consistent code in Matlab to solve this equation for any given set of $\vec{v}$. However, I am not sure how to get Matlab to solve this? the ODE command, ode45, is not exactly for this task. Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I need to do a simulation, so I will choose the values of $\vec{v}$ and do the simulation accordingly. All other constants are assumed known ($e,m,\tau,T$ and $|E|$).&lt;/p&gt;&#xA;" OwnerUserId="8124" LastEditorUserId="1552" LastEditDate="2014-05-06T02:35:54.160" LastActivityDate="2014-05-06T15:25:46.770" Title="Self-Consistent solution of Boltzmann eqn. using Matlab" Tags="|matlab|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11546" PostTypeId="2" ParentId="11545" CreationDate="2014-05-06T02:52:55.020" Score="1" Body="&lt;p&gt;To give background of the problem the general &lt;a href=&quot;http://en.wikipedia.org/wiki/Boltzmann_equation&quot; rel=&quot;nofollow&quot;&gt;Boltzmann Transport Equation&lt;/a&gt; for electrons in an electric field is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial f}{\partial t} + \vec{v} \cdot \nabla_{\vec{r}} f - \frac{e}{m} \vec{E} \cdot \nabla_{\vec{v}}f = \left(\frac{\partial f}{\partial t}\right)_{\mathrm{coll}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$ is the phase space velocity distribution function.  $f_0(\vec{v})$ is the steady state solution with no field (i.e. a maxwellian velocity distribution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then a (usually small) field is applied in one direction ($E_x$) which perturbs the velocity distribution of the electrons.  At this point, the distribution changes slightly, and if we apply steady state $\left(\frac{\partial f}{\partial t} = 0\right)$and isotropic $\left(\nabla_{\vec{r}} f = 0\right) $  we are left with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{e}{m} \vec{E} \cdot \nabla_{\vec{v}}f = \left(\frac{\partial f}{\partial t}\right)_{\mathrm{coll}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now we simply assume that $\left(\frac{\partial f}{\partial t}\right)_{\mathrm{coll}}$ is proportional to the perturbation from equilibruim, $f(\vec{v}) - f_0(\vec{v})$, with a proportionality constant of $\frac{1}{\tau}$.  Defining the $x$ axis along the field gives the equation from the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{e}{m} \tau |E| \frac{d f(\vec{v})}{d v_x} = f(\vec{v}) - f_0(\vec{v})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now due to the steady state being isotropic and us defining the x direction along the field we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{e}{m} \tau |E| \frac{d f(v_x)}{d v_x} = f(v_x) - f_0(v_x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This form of the equation has an exact analytic solution (it is just a 1st order ODE).  Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a = \frac{e}{m} \tau |E|$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the ODE becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f'(v_x) - \frac{1}{a} f(v_x) = - \frac{1}{a} f_0(v_x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which has a general solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(v_x) = C \exp(\frac{v_x}{a}) +\exp(\frac{v_x}{a}) \int_1^x - \frac{1}{a} \exp{\frac{\xi}{a}} f_0(\xi) \mathrm{d}\xi$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(shameless pulled from wolfram alpha at the moment)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have my notes with me at the moment to make sure I have done this all exactly correctly, but I will update this tomorrow when I get to the office.  Im also fairly certain that the final equation simplifies but I can't remember how.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I went back through my notes, and the problem I remembered from stat thermo had a few other small assumptions that don't apply in your question (it converted to energy as well).  Using wolframalpha and unity for the constants, I got an exact solution, although I am not happy with it (the inverse error function doesn't seem appropriate).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even still, it should be possible to solve the above equation numerically as you originally asked using a standard finite difference formulation.  The boundary conditions should be that $f$ goes to 0 at both positive and negative infinity and $\int_{-\infty}^\infty f dv_x = 1$.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2014-05-06T15:25:46.770" LastActivityDate="2014-05-06T15:25:46.770" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11547" PostTypeId="1" AcceptedAnswerId="11548" CreationDate="2014-05-05T08:20:51.137" Score="12" ViewCount="6412" Body="&lt;p&gt;I want to write mathematical optimization software. At university, they taught me how to use AMPL+CPLEX/SCIP/MINOS/Couenne etc.. and that was good enough. But I cannot afford the cost of AMPL for my personal use and I don't want to have any limitation in the size of the problem.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there any real free alternative to AMPL? &lt;/li&gt;&#xA;&lt;li&gt;If I write C++ code which uses opensource optimizator libraries like Couenne, can I use that software to make money? (For example, I saw that the Android camera uses an open source LMS optimizer.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5185" OwnerDisplayName="HAL9000" LastEditorUserId="276" LastEditDate="2014-05-06T03:33:50.137" LastActivityDate="2022-11-24T13:15:27.490" Title="Mathematical optimization software free/openSource" Tags="|optimization|linear-programming|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11548" PostTypeId="2" ParentId="11547" CreationDate="2014-05-06T03:32:29.307" Score="7" Body="&lt;h1&gt;Is there any free alternative to AMPL?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;(Edit: I spoke too soon; thanks to mlubin for pointing this out.) If you don't care too much about convenience, there are open-source formats for specifying nonlinear programs (the SIF and NOP formats), but then you have to translate that format into a form your solver can actually use. Since the CUTEr/CUTEst test sets use the SIF format, you might be able to leverage any interface a given solver has with those test sets and adapt it to reading whatever SIF files you want to produce.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as popular modeling frameworks go, GAMS has a free trial version, but the number of decision variables allowed in a formulation is capped. NEOS has GAMS and AMPL interfaces, so you could submit jobs on NEOS and then retrieve the results via, for instance, e-mail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are free modeling frameworks out there, like PuLP (lightweight, used for LPs; I used it in my thesis) and Pyomo, so you might be able to use those. Miles (mlubin) talks about a Julia-based framework in his answer. A partial list of frameworks can be found in &lt;a href=&quot;https://software.sandia.gov/trac/coopr/attachment/wiki/Pyomo/pyomo-jnl.pdf&quot; rel=&quot;nofollow&quot;&gt;this preprint&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also write solver-specific code. Couenne, for instance, is freely available, and you can use its native API to solve problems. Of course, repeating this process for every solver you might want to use is inconvenient.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;If I write C++ code which uses open source optimizers like Couenne, can I use that software to make money?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;IANAL, and it depends on the license. Couenne, for instance, is licensed under the Eclipse Public License (EPL). It doesn't prohibit you from using the software for commercial purpose (for instance, in designing a device). A quick read of the license (again, I should emphasize, IANAL) suggests that you can sell commercial software based on Couenne, and that if you include the Couenne source code as part of that software, then you must also provide the Couenne source code or tell people how they can obtain it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, BSD- and GPL-licensed software may be sold commercially, provided you comply with the terms of the respective licenses. In the case of the GPL, you must also provide the source code freely.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-05-07T19:04:55.613" LastActivityDate="2014-05-07T19:04:55.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11550" PostTypeId="1" CreationDate="2014-05-06T06:34:26.547" Score="5" ViewCount="395" Body="&lt;p&gt;I've been reading Larson and Bengzon's recent book on finite element methods, which has been good for getting an understanding of basic theory and computational procedures. The finite element book by Brenner and Scott has been recommended strongly by a number of people as a book for understanding more of the theory behind FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the introduction, the only prerequisite mentioned is &quot;a course in real variables&quot;; however, the proofs in the book seem to draw heavily from a measure theoretic treatment of the Lebesgue integral not necessarily covered in a standard two-course real analysis sequence. (For instance, exercises in Chapter 1 suggest using the monotone convergence theorem and Fubini's theorem.) Having taken a couple courses on real analysis that used Riemann integration and Jordan measure, plus a course on functional analysis that developed Lebesgue integration &lt;em&gt;without&lt;/em&gt; fully developing the Lebesgue measure, I have proven related versions of those theorems (years ago) without using Lebesgue measure or measure theory, but it's been a while.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do and understand the proofs of the FEM convergence theory in Brenner and Scott, how critical is it to really understand Lebesgue measure theory (say, at the level of Adams and Guillemin)?&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-06T07:39:33.347" Title="How much measure theory should I know to understand the proofs in Brenner &amp; Scott's FEM book?" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11551" PostTypeId="2" ParentId="11550" CreationDate="2014-05-06T07:39:33.347" Score="3" Body="&lt;p&gt;I don't believe you need any measure theory, just enough integration theory to make sense of Lebesgue and Sobolev spaces. If you know the statements of dominated convergence and Fubini's theorem as well as the fundamental lemma of the calculus of variations ($\int u \phi \,dx = 0$ for all $\phi\in C^\infty_0$ implies $u=0$), you should be fine. Chapter 1 is somewhat special since it is a review of these spaces, so the corresponding exercises are more fundamental as well. The remaining book is on a higher level (but does assume you know some functional analysis and the theory of (weak solutions to) partial differential equations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(For what it's worth, I found &lt;a href=&quot;http://ebooks.cambridge.org/ebook.jsf?bid=CBO9780511618635&quot; rel=&quot;nofollow&quot;&gt;Braess' book&lt;/a&gt;, which also treats the mathematical theory of finite elements, to be a friendlier read, and it might be a better place to start.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2014-05-06T07:39:33.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11553" PostTypeId="1" AcceptedAnswerId="11557" CreationDate="2014-05-06T11:29:38.310" Score="0" ViewCount="223" Body="&lt;p&gt;I am currently solving a Non Linear modell in GAMS and I am interested in a sensitivity analysis of the results. When working with a linear program I am able to look at the marginal values for the constraints (a marginal value of 100 means that if the constraint is relaxed by one unit then my objective will increase with 100 units).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when dealing with non linearity I have been informed that this is not the case and I am interested in finding a similar method of analysis for my non linear modell. How should I think? &lt;/p&gt;&#xA;" OwnerUserId="8127" LastActivityDate="2014-05-06T19:17:58.690" Title="Marginal values for non linear optimisation using SNOPT" Tags="|optimization|nonlinear-programming|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11554" PostTypeId="1" CreationDate="2014-05-06T12:36:30.400" Score="2" ViewCount="160" Body="&lt;p&gt;I am trying to find out the magnitude of the acceleration of my object based on non-uniformly sampled 3D position data. I'm using the standard approximation of the 2nd order derivative on a non-uniform grid (see e.g. &lt;a href=&quot;http://iwi.eldoc.ub.rug.nl/FILES/root/1992/JEngMathVeldman/1992JEngMathVeldman.pdf?origin=publication_detail&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, page 121):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;f'' \approx 2 \cdot\frac{h^+ f_0 - (h^- + h^+) f_1 + h^- f_2}{h^+ h^- (h^+ + h^-)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h^-$ is the first sample interval, $h^+$ is the second sample interval, and $f_n$ are the sampled points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to implement it in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class ComputeAcceleration&#xA;{&#xA;    Vector3D prevPosition;&#xA;    Vector3D prevPrevPosition;&#xA;    float prevTimeStep;&#xA;&#xA;    float acceleration;&#xA;&#xA;    // This function gets called every time step with the time step&#xA;    // since the last data input as argument.&#xA;    void Update(const float timeStep)&#xA;    {&#xA;        float doubleTimeStep = prevTimeStep + timeStep;&#xA;        Vector3D currentPosition = getCurrentPosition();&#xA;        Vector3D accelerationVector = prevTimeStep * currentPosition - &#xA;                                      doubleTimeStep * prevPosition + &#xA;                                      timeStep * prevPrevPosition;      &#xA;        accelerationVector  *= 2 / (doubleTimeStep * prevTimeStep * timeStep);&#xA;        acceleration = sqrt(accelerationVector.X * accelerationVector.X +&#xA;                            accelerationVector.Y * accelerationVector.Y+&#xA;                            accelerationVector.Z * accelerationVector.Z);&#xA;&#xA;        prevPrevPosition = prevPosition;&#xA;        prevPosition = currentPosition;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can't find the error in this, but when I plot this in real-time, I get a curve that doesn't resemble acceleration at all, but rather speed, in some jerky way. I plotted it against the speed (which I know to be correct), and it looks roughly like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/D2nux.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I doing something fundamentally wrong here?&lt;/p&gt;&#xA;" OwnerUserId="8129" LastActivityDate="2014-05-06T12:36:30.400" Title="Why does my Finite Difference approximation not work?" Tags="|finite-difference|computational-physics|" AnswerCount="0" CommentCount="12" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11555" PostTypeId="1" CreationDate="2014-05-06T13:48:47.520" Score="1" ViewCount="973" Body="&lt;p&gt;We use Armadillo a lot in our code but there are places where we prefer to use MKL directly from Vector Math Library. We have cx_vec i.e. vector of doubles. But the exp function of armadillo uses std::exp which is not great so we started using MKL for it. The problem comes when converting cx_vec into MKL_Complex16 arrays and back. The method works but I am not sure if there is a better way to do it or a recommended way. Is there a better way to use MKL from Armadillo? Here is the example of code we use currently. Note the cast at the begining and a copy at the end which is what I would like to rewrite. Any suggestion is welcome.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void Exp_MKL(const cx_vec&amp;amp; VectorInput, cx_vec&amp;amp; VectorOutput) {&#xA;  if (VectorInput.size() != VectorOutput.size())&#xA;    VectorOutput.set_size(VectorInput.size());&#xA;&#xA;  const MKL_Complex16* in = reinterpret_cast&amp;lt;const MKL_Complex16*&amp;gt;(VectorInput.memptr());&#xA;  const int size = VectorInput.n_elem;&#xA;  MKL_Complex16* out = (MKL_Complex16*) mkl_malloc(size * sizeof(MKL_Complex16), 64);&#xA;&#xA;  VZEXP(&amp;amp;size, in, out);&#xA;  cx_vec newoutput(reinterpret_cast&amp;lt;cx_double*&amp;gt;(out), VectorInput.size(), false);&#xA;  VectorOutput = newoutput;&#xA;  mkl_free(out);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4841" LastActivityDate="2017-06-29T11:54:30.183" Title="An efficient way to convert between MKL and Armadillo types" Tags="|linear-algebra|software|intel-mkl|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11556" PostTypeId="2" ParentId="11547" CreationDate="2014-05-06T14:44:26.673" Score="3" Body="&lt;p&gt;If you haven't already, check out &lt;a href=&quot;http://dakota.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;DAKOTA.&lt;/a&gt; Very extensive optimization capabilities and well supported. It's free under the LGPL license, which means you can use it in a commercial application subject to some constraints (I believe you have to dynamically and not statically link, and still allow for modification of that LGPL part, but obviously consult with a lawyer if this is an important issue for you.)&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-05-06T14:44:26.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11557" PostTypeId="2" ParentId="11553" CreationDate="2014-05-06T19:17:58.690" Score="0" Body="&lt;p&gt;The marginal values you're describing are duality multipliers. You should be able to obtain duality multipliers from SNOPT in GAMS output; they should still be called marginal values, and associated with each constraint equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Duality theory for nonlinear programming is different than for linear programming. In linear programming, strong duality always holds, whereas in nonlinear programming, strong duality requires technical conditions that are not always satisfied (typically, you need a convex problem plus a constraint qualification, like Slater's condition, or LICQ). Weak duality always holds, though. (I'm also assuming your problem is smooth; if it's not, the situation becomes even more complicated.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-06T19:17:58.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11559" PostTypeId="1" CreationDate="2014-05-06T22:50:34.200" Score="1" ViewCount="406" Body="&lt;p&gt;I'm trying to implement PageRank. I'm reading the description here: &lt;a href=&quot;http://nlp.stanford.edu/IR-book/html/htmledition/markov-chains-1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nlp.stanford.edu/IR-book/html/htmledition/markov-chains-1.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TE0iJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything is very clear to me, however I'm concerned about the construction of the matrix $P$. I find that constructing $P$ the naive way would be very expensive. For example: to implement step 1, one would need to check every row of $A$ and then check every element of that row to see if all elements are zero. For step 2 one would need to compute the number of ones for each row. I can imagine my code to have nasty slow loops. I was wondering if there are smart linear algebra techniques that could efficiently construct $P$. I will be using python numpy for my coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: one way I'm thinking now to solve this is by doing a summation element wise over the columns of $A$. By that I would have a column vector. Now I will go through each element of this vector to check which elements are zeros. Thus I can now know which rows has no 1s and I can multiply those rows with $1/N$. &lt;/p&gt;&#xA;" OwnerUserId="8134" LastActivityDate="2014-05-06T22:50:34.200" Title="Implementing the transition matrix for page rank" Tags="|linear-algebra|matrix|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11560" PostTypeId="1" AcceptedAnswerId="11561" CreationDate="2014-05-07T09:48:38.477" Score="1" ViewCount="447" Body="&lt;p&gt;I will have to solve a large linear system. I'm now looking for a solver that works &quot;matrix-free&quot; (So that I just have to specify a matrix-vector product, but not the matrix). As far as I understand (I'm not an expert.) LAPACK does not provide this option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you recommend any libraries? Or is my approach to implement this matrix-free not reasonable at all?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remark: I plan to use C++ and the matrix comes up from a FEM for a system of PDEs, but is not symmetric. The matrix is sparse, but as I plan to provide the matrix-vector product by myself, and the resulting vector will be dense, sparsity should not play a role here - as far as I understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards, Michael&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2014-05-07T10:06:31.310" Title="Solver library for matrix-free linear equation system" Tags="|linear-solver|libraries|matrix-free|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11561" PostTypeId="2" ParentId="11560" CreationDate="2014-05-07T10:06:31.310" Score="2" Body="&lt;p&gt;Using a matrix-free approach for a sufficiently large linear system sounds reasonable to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably want to take a look at PETSc or Trilinos; those are the two main libraries that serve as abstraction layers for a number of different numerical methods (preconditioners, sparse linear solvers, scalable nonlinear equation solvers, time steppers, optimization). They also build in abstractions that make it easier to incorporate parallelism into an application than using bare MPI calls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also look at higher levels of abstraction, if that is of interest; there are a number of finite element libraries that build on PETSc or Trilinos.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-07T10:06:31.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11563" PostTypeId="1" CreationDate="2014-05-07T14:36:24.393" Score="1" ViewCount="1125" Body="&lt;p&gt;I'm following the book here to compute PageRank: &lt;a href=&quot;http://nlp.stanford.edu/IR-book/html/htmledition/markov-chains-1.html&quot; rel=&quot;nofollow&quot;&gt;http://nlp.stanford.edu/IR-book/html/htmledition/markov-chains-1.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I built the transition matrix P as indicated in the book. Then I computed the PageRank scores by raising the transition matrix P to higher powers, as said in the book. To make sure my computations are correct I followed this example: &lt;a href=&quot;http://nlp.stanford.edu/IR-book/html/htmledition/the-pagerank-computation-1.html&quot; rel=&quot;nofollow&quot;&gt;http://nlp.stanford.edu/IR-book/html/htmledition/the-pagerank-computation-1.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transition matrix is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P = np.array([[1./6, 2./3, 1./6], &#xA;              [5./12, 1./6, 5./12],&#xA;              [1./6, 2./3, 1./6]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results of the PageRank scores are the following (by computing P^40):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0.28, 0.44, 0.28]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as in the book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the book also says that one can find the PageRank scores by computing the principal left eigenvector of the transition matrix P. So I tried this with numpy:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v, V = np.linalg.eig(P.T)&#xA;left_vec = V[:, 0].T&#xA;print left_vec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[-0.46829291 -0.74926865 -0.46829291]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so what's wrong?!&lt;/p&gt;&#xA;" OwnerUserId="8134" LastEditorUserId="3399" LastEditDate="2014-05-07T16:36:32.437" LastActivityDate="2014-05-07T16:36:32.437" Title="PageRank computation as the left principal eigenvector" Tags="|python|numpy|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11565" PostTypeId="2" ParentId="11547" CreationDate="2014-05-07T17:39:53.387" Score="8" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any real free alternative to AMPL?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes! (Sorry to contradict you, Geoff.) There is a real free alternative to AMPL. &lt;a href=&quot;https://github.com/JuliaOpt/JuMP.jl&quot; rel=&quot;noreferrer&quot;&gt;JuMP&lt;/a&gt; is a free and open-source modeling language built on top of Julia. It is capable of modeling large-scale linear, mixed-integer, and most recently nonlinear programming problems. SCIP support will be coming soon, along with mixed-integer nonlinear programming. Check out the &lt;a href=&quot;https://jump.readthedocs.org/en/latest&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt; and let us know if there's any way we can help out. Julia is still a young language, so you shouldn't expect the stability of C++ or AMPL, but if you're willing to be an early adopter, you get a combination of performance and features that historically hasn't been available in free software.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If I write C++ code which uses opensource optimizator libraries like&#xA;  Couenne, can I use that software to make money?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;JuMP is LGPL licensed, which means that you can use it as part of a closed-source commercial product if you comply with the terms of the license. JuMP depends on solvers (both commercial and open-source) which have their own respective licences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit by yeesian:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Looks interesting, thank you! I'll wait to test it (or to know other&#xA;  users' feedbacks) before accept this answer, though.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are some simple examples available, for &lt;a href=&quot;http://iaindunning.com/2013/urban-planning.html&quot; rel=&quot;noreferrer&quot;&gt;urban planning&lt;/a&gt;, &lt;a href=&quot;http://iaindunning.com/2013/sudoku-as-a-service.html&quot; rel=&quot;noreferrer&quot;&gt;sudoku&lt;/a&gt;, and &lt;a href=&quot;http://iaindunning.com/2013/mip-callback.html&quot; rel=&quot;noreferrer&quot;&gt;travelling salesman&lt;/a&gt;. It has also been used in the MIT course 15.S60 &quot;Software Tools for Operations Research&quot; (&lt;a href=&quot;https://github.com/IainNZ/ORSoftwareTools2014&quot; rel=&quot;noreferrer&quot;&gt;github repo&lt;/a&gt;) too. For an updated list of solvers supported, check the &lt;a href=&quot;http://juliaopt.org/&quot; rel=&quot;noreferrer&quot;&gt;JuliaOpt website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8142" LastEditorUserId="8142" LastEditDate="2014-05-11T22:12:26.100" LastActivityDate="2014-05-11T22:12:26.100" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11566" PostTypeId="2" ParentId="11547" CreationDate="2014-05-07T20:15:55.487" Score="3" Body="&lt;p&gt;It was already mentioned but you should have a look at &lt;a href=&quot;https://software.sandia.gov/trac/coopr/wiki/Documentation&quot; rel=&quot;nofollow&quot;&gt;Pyomo-Coopr&lt;/a&gt; as a modelling language similar to AMPL (but using Python) that have interfaces to a variety of third-party solvers.&lt;/p&gt;&#xA;" OwnerUserId="8114" LastEditorUserId="8168" LastEditDate="2014-05-11T15:36:30.777" LastActivityDate="2014-05-11T15:36:30.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11567" PostTypeId="2" ParentId="5617" CreationDate="2014-05-07T21:53:57.993" Score="0" Body="&lt;p&gt;Oh, man. Just came over this old thread. If anybody reads this, don´t use toolboxes. Write it your own. I did it with the method of lines and published boundary conditions. Working well with standard ode solvers. Don´t know why it did not work.&lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2014-05-07T21:53:57.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11568" PostTypeId="1" CreationDate="2014-05-07T22:32:26.750" Score="1" ViewCount="245" Body="&lt;p&gt;&lt;strong&gt;The Problem Domain&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a number of network-connected PCs. I want to be able to simulate and replicate the same simple fluid dynamics simulation (E.g. Navier-Stokes), in real-time, between them. That is to say, the same body of fluid will be simulated at the same time in isolation on each PC. They will all have the same starting conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple so far I hope. Now, each PC with its own simulation can independently add input into the simulation. For example, one PC may add/subtract some fluid at $xyz$ at time $t$ of its own simulation. I need to be able to replicate that input to all other PCs and simulations so they each stay in sync with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose to summarize in other words - I would like to run a single fluid dynamics simulation across a number of network-connected PCs in real-time. I cannot simply replicate the resulting simulation data to all other PCs as the latency and bandwidth requirements would be too great and all other PCs would lag behind. They each need to be in sync as much as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question here is in regards to the fluid dynamics equation: Is it possible to simulate fluid dynamics that can take input at time $t$, that may be in the past by the time the simulation gets the input, and take that into account?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: When I say input I mean a change in the simulation environment be it a change to the container the fluid is held in, a change in forces or a change in fluid volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current assumptions for this question is that it is possible to run simple fluid dynamics in real-time (&lt;a href=&quot;http://www.intpowertechcorp.com/GDC03.pdf&quot; rel=&quot;nofollow&quot;&gt;See Jos Stams implementation&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="8146" LastActivityDate="2014-07-09T02:26:18.413" Title="Is it possible to simulate fluid dynamics in a time-based and deterministic manner?" Tags="|parallel-computing|simulation|computational-physics|fluid-dynamics|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="11569" PostTypeId="2" ParentId="11517" CreationDate="2014-05-08T01:14:31.613" Score="1" Body="&lt;p&gt;I found the problem. I forgot to add face velocity when updating the airfoil surface. It should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u = tangX + vbx&#xA;v = tangY + vby&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where, tangX and tangY are the x and y components of the tangential velocity of the neighbour interior cell. Then, the flux at the airfoil surface should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Fn0 = 0;&#xA;Fn1 = p*nx;&#xA;Fn2 = p*ny;&#xA;Fn3 = un * p;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that, un is not zero due to face velocity but (un-vb) is zero.&lt;/p&gt;&#xA;" OwnerUserId="8103" LastActivityDate="2014-05-08T01:14:31.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11572" PostTypeId="1" AcceptedAnswerId="11608" CreationDate="2014-05-08T09:49:35.070" Score="4" ViewCount="446" Body="&lt;p&gt;I have to solve for x in b = A*x, where a is sparse. This works fine with Matlab's mldivide: x = A \ b. Since I will have to use an iterative algorithm for very large A, I'm currently testing Matlab's iterative algorithms, bicgstab in particular:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[L, U] = ilu(A, struct('type', 'ilutp', 'droptol', 0.0000001, 'udiag', 0));&#xA;[delta2, flag, relres, iter] = bicgstab(A, b, 0.0000001, 2000, L, U);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, this works with a general vector b generated with with randn(...). But it does not work with the vector b from my problem. Not work in this case means: It does one or only a few iterations and returns with flag 1 which says that it did not converge in the allowed number of iterations. The returned vector is then wrong compared to the solution with x = A \b.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vector b, as well as the matrix A have values over a large magnitude range (about 1e10), matrix A is a general sparse matrix with dimension about 2000x2000 and has condition about 1e14&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only idea which I currently have would be to change physical units of the system behind the matrix to make large values smaller and small values larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this is very little information about this problem, but I would be glad for any hint, especially how to find out more about the problem. &lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2014-05-12T08:50:45.987" Title="Linear equation system: Direct solver works, iterative solver does not" Tags="|linear-solver|iterative-method|conjugate-gradient|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11573" PostTypeId="1" CreationDate="2014-05-08T09:49:59.733" Score="1" ViewCount="280" Body="&lt;p&gt;I want to solve the Poisson equation for a 2D polar system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta_r f(r) = u(r)$$&#xA;with the Laplace operator: $ \Delta_r f(r) = \frac{1}{r}\frac{\partial}{\partial r} \left[r  \frac{\partial}{\partial r} f(r) \right] $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have $u(r)$ given as a vector for a non-uniform (quasi-logarithmic) grid. Any ideas how to solve this?&lt;/p&gt;&#xA;" OwnerUserId="7776" LastActivityDate="2014-05-08T19:17:22.493" Title="Numerically solving the polar poisson equation" Tags="|pde|poisson|" AnswerCount="1" CommentCount="1" ClosedDate="2014-05-10T19:56:33.797" ContentLicense="CC BY-SA 3.0" />
  <row Id="11574" PostTypeId="1" AcceptedAnswerId="11599" CreationDate="2014-05-08T10:11:14.580" Score="0" ViewCount="269" Body="&lt;p&gt;I am not a biologist. But highly motivated by the recent advancements and problems in modern biology. I have been shortlisted for a PhD(computational biology programme  at &lt;a href=&quot;http://www.imsc.res.in/graduate_programme_0&quot; rel=&quot;nofollow&quot;&gt;IMSc&lt;/a&gt;, Madras) interview in first week of June. My background is electronics and communication engineering. But prior biology knowledge is not required for this programme as per IMSc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the call letter for interview says like this &quot;The interview will test your aptitude, knowledge and depth of understanding in various areas of computational biology. There is no specific syllabus.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have basics in Information theory, linear algebra, probability and statistics, have read basics of molecular biology. I have got an introduction in machine learning too. But lack knowledge in the theory of Algorithms and data structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With less than a month remaining for interview, what would be your advice on my preparation? What subjects am I suppose to get introduced to? &lt;/p&gt;&#xA;" OwnerUserId="5003" LastActivityDate="2014-05-10T19:22:41.167" Title="What are the areas a student must focus before entering the computational biology research?" Tags="|computational-biology|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11575" PostTypeId="2" ParentId="11573" CreationDate="2014-05-08T14:38:05.140" Score="2" Body="&lt;p&gt;You can use finite differences (this is from GD Smith page 213)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial^2 }{\partial r^2}u+\frac{1}{r}\frac{\partial}{\partial r}u=f$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define the mesh points in the $r-\theta$ as $r=i \delta r$  so now at point $(i)$ the the equation is approximated as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u_{i+1}-2u_{i}+u_{1-i}}{(r(i+1)-r(i-1))^2}+\frac{1}{r(i)} \frac{u_{i+1}-u_{i-1}}{2(r(i+1)-r(i))} =f(i)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point you should be able to see how to construct this into a matrix formulation and include your BC's. &lt;/p&gt;&#xA;" OwnerUserId="7257" LastEditorUserId="7257" LastEditDate="2014-05-08T19:17:22.493" LastActivityDate="2014-05-08T19:17:22.493" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11576" PostTypeId="1" AcceptedAnswerId="11577" CreationDate="2014-05-08T14:41:03.910" Score="0" ViewCount="78" Body="&lt;p&gt;In one step of image evaluations I need to apply a gaussian blur to pixel data. I do this by folding each pixel with a gaussian folding matrix.&#xA;This is no problem unless I try to do this with pixels at the border of the image since for example on the left border I can't include the pixels to the left in my folding calculation. There are multiple ways around this that I can think of:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Don't blur pixels without a complete set of neighbours.&lt;/li&gt;&#xA;&lt;li&gt;Ignore the neighbours that aren't there.&lt;/li&gt;&#xA;&lt;li&gt;Use the center pixel instead of the pixels that are missing.&lt;/li&gt;&#xA;&lt;li&gt;Use the pixel at the other side of the image.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Case 2 isn't right in my opinion because it destroys the normalization of the whole folding matrix. I can't use 4. in my case and it doesn't really make sense anyways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: How should this problem be handled?&lt;/p&gt;&#xA;" OwnerUserId="8153" LastActivityDate="2014-05-08T14:52:44.273" Title="Behaviour of folding matrix on image borders (Gaussian Blur)" Tags="|image-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11577" PostTypeId="2" ParentId="11576" CreationDate="2014-05-08T14:52:44.273" Score="1" Body="&lt;p&gt;There are a number of different way to do this, and the &quot;right&quot; choice depends on the exact application. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your option 1 is probably not what you want, but a similar option is to simply return a smaller image, on which the blur is well defined. This is probably the closest you will get to something &quot;fool proof&quot;, if you can live with the smaller image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your option 2 could work, and is certainly a valid option provided you redo the normalization so it still integrates to 1. I personally prefer this option&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is a mirroring boundary condition, so you count some of your pixels twice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.filters.gaussian_filter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Gaussian blur in SciPy&lt;/a&gt;, you will see a number of different options, all of which are quite valid from a mathematical point of view. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It really very much depends on your application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to have a look at the &lt;a href=&quot;https://dsp.stackexchange.com/&quot;&gt;Signal Processing StackExchange&lt;/a&gt;, as this is probably more their domain. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2014-05-08T14:52:44.273" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11579" PostTypeId="1" CreationDate="2014-05-08T23:05:32.400" Score="1" ViewCount="224" Body="&lt;p&gt;Could someone tell, please, has someone constructed finite difference mixed derivatives on quadrilateral (non-orthogonal) grids, already? Without using aconformal mapping to an orthogonal grid. I mean we have the mixed differential operator and try to represent it through a difference relation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched with the help of Google Scholar, but found nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to constuct it myself, but have problems with the symmetry of mixed second difference derivative: $$ f_{xy} = f_{yx} $$&lt;/p&gt;&#xA;" OwnerUserId="3956" LastEditorUserId="3956" LastEditDate="2014-05-09T06:01:27.907" LastActivityDate="2015-07-10T21:17:39.960" Title="Finite difference mixed derivatives on quadrilateral (non-orthogonal) grids" Tags="|finite-difference|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11580" PostTypeId="1" AcceptedAnswerId="11588" CreationDate="2014-05-09T10:09:13.040" Score="4" ViewCount="1211" Body="&lt;p&gt;[ question reposted from &lt;a href=&quot;https://math.stackexchange.com/questions/786612/solving-a-linearly-constrained-sparse-linear-least-squares-problem&quot;&gt;https://math.stackexchange.com/questions/786612/solving-a-linearly-constrained-sparse-linear-least-squares-problem&lt;/a&gt; ]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the system of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=b$, subject to $Cx\le d$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is an $n\times m$ matrix (with $n&amp;gt;m$) and is very large and sparse. As an example $A$ can have $3126250\times 2740$ elements. Each row of $A$ has only 4 or 5 non-zero numbers which can only be 1 or -1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am on Matlab and I've been using LSQR but I need the inequality constraints to impose monotonicity on $x$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please advise on any solvers to do this with linear constraints? Is there any implementation on Matlab or C for this?&lt;/p&gt;&#xA;" OwnerUserId="8154" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-05-09T15:32:07.317" Title="solving a linearly-constrained sparse linear least-squares problem" Tags="|optimization|least-squares|constraints|linear-system|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11581" PostTypeId="1" AcceptedAnswerId="11584" CreationDate="2014-05-09T10:55:04.303" Score="3" ViewCount="262" Body="&lt;p&gt;I try to solve a system of coupled PDEs using FEM. Unfortunately, the originating matrix has very poor condition. After days of double checking and thinking, I suspect the following reason:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a PDE&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a * Laplacian(u) + b * u = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For FEM, one has to calculate two operator matrices, one for Laplacian(u) and one for u.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When L is the element size, as far as I understand, the matrix entries for Laplacian(u) scale with 1 / L while the ones for u scale with L. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My system has a size of several micrometers, but I use meters as unit for lengths. So the elements for Laplacian(u) are very big, while the ones for u are very small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, could I improve the matrix condition by using a shorter length unit for the elements, maybe millimeters or micrometers? (I mean not the unit of u, but the unit of the geometry.)&lt;/p&gt;&#xA;" OwnerUserId="8138" LastEditorUserId="8138" LastEditDate="2014-05-09T13:24:08.933" LastActivityDate="2014-05-09T13:41:15.080" Title="Does length unit in FEM affect numerical condition?" Tags="|finite-element|linear-solver|condition-number|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11582" PostTypeId="1" AcceptedAnswerId="11602" CreationDate="2014-05-09T12:41:00.600" Score="4" ViewCount="1525" Body="&lt;p&gt;This is a bit of a long shot, but I was hoping somebody might have some insight (not sure of a better forum than here but open to suggestions). I have implemented the optical flow algorithm from the paper &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1615232&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;An Improved Algorithm for TV-L1 Optical Flow&lt;/em&gt;&lt;/a&gt;. I've tried to stick to exactly the same parameters as the article explains (they have quite detailed implementation notes), and yet I can't reproduce their results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my analysis (I'll show images in a moment), it appears that the algorithm works well with small images, so in the coarse-to-fine pyramid you can see the flow is accurately calculated for small-scale version of each image. Then when the images are upscaled beyond a certain point the optimisation seems to converge to some poor local minimum. To confirm this suspicion I reran the algorithm with a scaling factor of 0.9 between pyramid levels (instead of the 0.5 the article uses) and the results are much improved - it seems with a small enough up/downscaling factor we avoid the poor local minima. Here are the flows (Middlebury backyard scene, with scaling factors of 0.5, 0.7, 0.9 in that order):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zhTte.png&quot; alt=&quot;0.5&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/FKK1L.png&quot; alt=&quot;0.7&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/85o6I.png&quot; alt=&quot;0.9&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the iteration-by-iteration sequences of each scaling factor (here we see that even 0.5 works well on small images):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.5: &lt;a href=&quot;http://youtu.be/EHTO7lJeMrA&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://youtu.be/EHTO7lJeMrA&lt;/a&gt;&lt;br&gt;&#xA;0.7: &lt;a href=&quot;http://youtu.be/-PlTU3VioWg&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://youtu.be/-PlTU3VioWg&lt;/a&gt;&lt;br&gt;&#xA;0.9: &lt;a href=&quot;http://youtu.be/lK5EP865u0E&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://youtu.be/lK5EP865u0E&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've fiddled with all the other parameters of the algorithm, and they don't have any major impact on this phenomenon. I can upload my Matlab code if anyone is interested. So my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have an alternate explanation for this phenomenon? The fact that I can't reproduce their results is troubling me greatly.&lt;/p&gt;&#xA;" OwnerUserId="8160" LastActivityDate="2014-12-26T18:56:21.007" Title="TVL1 algorithm for optical flow" Tags="|optimization|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11583" PostTypeId="2" ParentId="11581" CreationDate="2014-05-09T13:13:42.473" Score="2" Body="&lt;p&gt;No. It is the ratio of the scales of these two operators that will drive the condition number not their absolute magnitude. You can multiply the whole equation through by any arbitrary constant without changing the condition number.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-05-09T13:13:42.473" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11584" PostTypeId="2" ParentId="11581" CreationDate="2014-05-09T13:14:13.757" Score="5" Body="&lt;p&gt;No, a different unit will not alter the condition of the system. Say your FEM system is&#xA;$$&#xA;Au + \beta Mu = 0. \quad (*)&#xA;$$&#xA;Then the parameter $\beta$, here something like $b/a$ from your example, will depend on the units in a way that only allows you to add &quot;$A$&quot; and &quot;$M$&quot; in terms of units. A rescaling will then mean a multiplication of $(*)$ by a constant which will not change the condition number of your system matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entries of $A$ and $M$ are intregals involving the shape functions. Their value is independent of the units in which you express them and they scale like $L$ or $1/L$. Say $L=1mm$ and $\beta = \frac{1}{mm^2}$. Expressing $L$ as $L=10^{-3}m$ will result in&#xA;$$&#xA;A \left [\frac{1}{mm}\right] + \beta\left[\frac{1}{mm^2}\right]M\left[mm\right] = 10^3A\left[\frac{1}{m}\right]+10^6\beta\left[\frac{1}{m^2}\right]10^{-3}M\left[m\right]&#xA;$$&#xA;which is a simple scaling of the equations (*):&#xA;$$ 10^3\left \{A\left[\frac{1}{m}\right] + \beta\left[\frac{1}{m^2}\right]M\left[m\right]\right\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if your matrix entries drop below machine precision, then a rescaling will improve accuracy.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2014-05-09T13:41:15.080" LastActivityDate="2014-05-09T13:41:15.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11585" PostTypeId="1" AcceptedAnswerId="11589" CreationDate="2014-05-09T13:56:39.013" Score="4" ViewCount="3174" Body="&lt;p&gt;I have a choice of two options, analysing and implementing Lattice Boltzmann methods or traditional Navier Stokes based methods. I'm a CFD newbie and I have a rough idea (though not rigorous enough to produce a code) from my fluid mechanics classes about Finite Volume, Finite Element and Finite Difference methods. I am not clear about what would be a good starting point for this. There is already a question slightly similar to this : &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2109/is-lattice-boltzmann-suitable-for-simulation-of-incompressible-stokes-flow/2118#2118&quot;&gt;Is lattice Boltzmann suitable for simulation of incompressible Stokes flow?&lt;/a&gt;&#xA;Can anyone comment on:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Whether I need to become comfortable with traditional Navier Stokes based methods to understand and appreciate Lattice Boltzmann Methods?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;This question is based on the choice&lt;/strong&gt; : What are the key pros and cons of NS-based methods vs LBM methods as far as code, efficiency etc. are concerned?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8161" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-05-17T14:44:04.650" Title="Comparison of Lattice Boltzmann Method vs Traditional Navier-Stokes based Methods" Tags="|finite-element|finite-difference|fluid-dynamics|finite-volume|navier-stokes|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11586" PostTypeId="1" CreationDate="2014-05-09T14:29:51.410" Score="2" ViewCount="453" Body="&lt;p&gt;I noticed that Python does not have a good datatype for rational numbers, certainly not for algebraic numbers like $\tfrac{1 + \sqrt{2}}{3}$ or the real root of $x^3 - 5x + 7$.  They have the &lt;code&gt;fractions&lt;/code&gt; data type and I was able to hack a solution to my particular problem from there using elaborate arrangements of matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be really nice to take advance of the &lt;code&gt;PARI&lt;/code&gt; library which is written in &lt;code&gt;C&lt;/code&gt;.  Since &lt;code&gt;GP&lt;/code&gt; is an interface to the PARI, it seems possible to write a Python interface to those libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't seem to install &lt;a href=&quot;http://code.google.com/p/pari-python/&quot; rel=&quot;nofollow&quot;&gt;Pari-Python&lt;/a&gt; so this is a chance to learn to import C libraries directly.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;At the heart of it, I just want to learn how to import C++ libraries into Python.  Specifically, libraries from the &lt;a href=&quot;http://pari.math.u-bordeaux.fr/&quot; rel=&quot;nofollow&quot;&gt;PARI-GP&lt;/a&gt; source code.&lt;/p&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2014-05-12T21:13:48.173" Title="importing PARI libraries ( in C++) with Python" Tags="|linear-algebra|python|c++|" AnswerCount="3" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11588" PostTypeId="2" ParentId="11580" CreationDate="2014-05-09T15:32:07.317" Score="3" Body="&lt;p&gt;If you have access to the MATLAB optimization toolbox then this can easily be done using the quadprog() function.  You'd start by writing the objective in quadratic form as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \| Ax - b \|_{2}^{2} = x^{T}(A^{T}A)x-2(A^{T}b)^{T}x+b^{T}b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then multiply to get $P=A^{T}A$ and $q=-2A^{T}b$.  Then your objective is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x)=x^{T}Px+q^{T}x+b^{T}b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and ready to feed into quadprog().  The $P$ matrix is only 2740 by 2740, so this isn't a very large problem from the point of view of quadprog(). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure there are some free qp solvers for MATLAB if you don't have a copy of the optimization toolbox. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that you may want to reformulate the problem in terms of variables $z_{i}$, where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{1}=z_{1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{2}=z_{1}+z_{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\ldots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can replace your inequality constraints $Cx \leq d$ with $z \geq 0$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-05-09T15:32:07.317" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="11589" PostTypeId="2" ParentId="11585" CreationDate="2014-05-09T17:37:20.650" Score="2" Body="&lt;p&gt;In general, LBM is much easier to implement than FVM/FEM. It is very much like a FDM implementation, but depending on how you are doing it, I would say it is even simpler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't say much with respect to turbulent flows. But I had a simple LBM code running the &quot;lid driven cavity&quot; problem with Re=500 which had a very few lines of code (even with CUDA).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at some Matlab LBM examples here: &#xA;&lt;a href=&quot;http://wiki.palabos.org/numerics%3amatlab_samples&quot; rel=&quot;nofollow&quot;&gt;http://wiki.palabos.org/numerics:matlab_samples&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to efficiency, in general LBM consumes more memory and performs more floating point operations per time step than the others, however, you can implement it in parallel and achieve good performance.&lt;/p&gt;&#xA;" OwnerUserId="484" LastActivityDate="2014-05-09T17:37:20.650" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11590" PostTypeId="1" CreationDate="2014-05-09T18:28:26.840" Score="4" ViewCount="2108" Body="&lt;p&gt;I have a geometry and a finite element method where there are a couple internal interfaces across which I want to enforce some fairly simple jump conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One recommendation I've gotten has been to align the mesh with the interfaces and generate zero thickness elements along the interfaces, so that every nodal point on the interface is associated with a duplicate node. Is there a mesh generator that will do something like this for unstructured meshes? (I cannot use structured meshes in my application.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to do XFEM to account for the interface conditions, and generating zero thickness elements seems like the simplest way to implement it.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-13T05:53:10.387" Title="Is there a mesh generator that will generate zero thickness elements for interfaces?" Tags="|finite-element|mesh-generation|unstructured-mesh|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11591" PostTypeId="1" CreationDate="2014-05-09T22:20:45.610" Score="3" ViewCount="343" Body="&lt;p&gt;I asked the same question on mo.se and it was suggested that scicomp would be a better forum for it. So here it is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am writing code for solving linear equations of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{n\times n}\cdot x=1_n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $n$ is on the order of $10^6$ and $A$ is a symmetric matrix with approx $10^3$ nonzero entries in each row. This makes its size barely manageable, but inverting it is infeasible, and I'm not sure which decomposition suitable for solving linear equations would lead to sparse matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Any hope that $LU$ decomposition of a symmetric sparse matrix would be sparse?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is it possible to take advantage that r.h.s. is a scalar to simplify solution of the above equation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) What would be the best numerically stable algorithm to handle linear equations of that size?&lt;/p&gt;&#xA;" OwnerUserId="5281" LastActivityDate="2014-05-09T22:55:58.917" Title="Decompositions of sparse symmetric matrices and methods for solving large linear equations" Tags="|linear-algebra|numerical-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11592" PostTypeId="2" ParentId="11582" CreationDate="2014-05-09T22:31:36.370" Score="2" Body="&lt;p&gt;It looks to me like you might not be giving as much weight to the regularization (TV and/or L1) as they are. That would give the kind of result you're seeing.&#xA;A good rule of thumb is that the maximum optical flow in each level should be about 1 (in the units of the current scale).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From reading their article, it seems they're rescaling their image to the [-1,1] interval. This is equivalent to using another regularization weight, so maybe you missed this step?&#xA;I hope it helps you anyway.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-05-09T22:31:36.370" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11593" PostTypeId="2" ParentId="11591" CreationDate="2014-05-09T22:55:58.917" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;In your case, almost no hope. Only very specific types of sparse matrices have a sparse $LU$ decomposition. For example, (trivially) diagonal and tridiagonal matrices.&lt;/li&gt;&#xA;&lt;li&gt;Not really. What you're basically saying is you want the sum of all the columns of $A^{-1}$. If instead the RHS was sparse, you could use selective inversion (SelInv) techniques, potentially.&lt;/li&gt;&#xA;&lt;li&gt;Your matrix is real symmetric I presume, so you can use an iterative method like conjugate gradients (CG). All it requires is that you can apply the matrix-vector product onto an arbitrary matrix. You may need a preconditioner, or to apply something like algebraic/geometric multigrid techniques. If you know something about the low rank off diagonal structure of $A$, you could consider hierarchical matrix factorization methods (this is pretty advanced and there is very little code out there for it). Finally, your matrices are not &lt;em&gt;that&lt;/em&gt; sparse. You could consider treating them densely and using an out-of-core sparse LU. This is really more of a last resort if nothing else works.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you know that $A$ is positive definite, then replace LU above with Cholesky, which is slightly more efficient.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-05-09T22:55:58.917" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11594" PostTypeId="2" ParentId="11586" CreationDate="2014-05-09T23:38:19.180" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I noticed that Python does not have a good datatype for rational numbers, certainly not for algebraic numbers like $\tfrac{1 + \sqrt{2}}{3}$ or the real root of $x^3 - 5x + 7$.  They have the &lt;code&gt;fractions&lt;/code&gt; data type and I was able to hack a solution to my particular problem from there using elaborate arrangements of matrices.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the reason you'd prefer not to use Sage is that you don't want to install kitchen sinks, you can do this in Python using &lt;a href=&quot;http://sympy.org/en/index.html&quot; rel=&quot;nofollow&quot;&gt;sympy&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-05-09T23:38:19.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11595" PostTypeId="1" CreationDate="2014-05-10T00:04:21.033" Score="2" ViewCount="214" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Suppose am solving the heat conduction equation in 1d with&#xA;Dirichlet boundary conditions. The thermal conductivity $k$ is a non&#xA;constant function.&#xA;So $-(k(x)u'(x))' = f(x)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The value of $k$ enters the discretization only through its values at&#xA;the faces of the control volume $k_{i+1/2}, \; k_{i-1/2}$, and not at the cell centres $k_i$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;But most books and notes available online suggest that we store the&#xA;value of $k$ at cell centres a priori and use it in the computation. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If $k(x)$ has jumps  across the faces, I understand one uses the harmonic average.  What about control volumes near the boundary ? What is the  standard way to approximate $k(x)$ there; harmonic averaging with neighbour is not possible. Can I simply use the cell centre value for these cells ? &lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am unable to see how approximating $k(x)$ will eventually affect the order of&#xA;accuracy of the method in these  cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about accuracy, not exactly the same as &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7456/how-should-non-constant-coefficients-be-treated-with-finite-volume-first-order-u&quot;&gt;How should non-constant coefficients be treated with finite-volume first order upwind scheme?&lt;/a&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&#xA;When $k(x)$ is smooth, by expanding  the terms $k_{i+1/2}, \; k_{i-1/2}$ in a Taylor's series around $k_i$ in the standard 3 point scheme, I see that using the cell centre  value directly is atleast $O(h^2)$ or second order accurate. But that would violate the conservation property of the scheme since $k_{i+1/2}^+ \neq k_{i+1/2}^-$. But we can  use linear interpolation  with $k_{i}, \; k_{i+1}$  to calculate $k_{i+1/2}$ and maintain $O(h^2)$ accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in case of discontinuous $k$, I am still puzzled by order of accuracy of the  harmonic mean approximation for $k_{i+1/2}, \; k_{i-1/2}$, and how to  calculate $k_{i+1/2}, \; k_{i-1/2}$ at the boundary  cells.&lt;/p&gt;&#xA;" OwnerUserId="7728" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-05-10T16:46:00.707" Title="boundary conditions with non-constant coefficients in cell centred finite volume method" Tags="|numerical-analysis|boundary-conditions|finite-volume|discretization|accuracy|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11596" PostTypeId="2" ParentId="11390" CreationDate="2014-05-10T03:58:02.150" Score="0" Body="&lt;p&gt;I would day it definitely depends on the parallelization architecture and what you want. If using shared memory CPU's, probably Elemental even though I have not used it. Note that PETSc also has tools for sparse matrices. But if you really need it to go as fast as you can, and the sparse matrices have some known structure that you can take advantage of, then I would write the whole thing using MPI. In my experience that is always much faster, but also requires more work. So it depends on your needs and time to code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to go with GPU's, nvidia has a sparse matrix library that you could try using CUDA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CUDA 5 also let's you do some kind of mpi+cuda, so you can have a clusters of gpu's, but I have no idea if the sparse-matrix libraries are capable of that but I don't see why not. It's worth to take a look at. But again, if you need it to fly, I would make my own code.&lt;/p&gt;&#xA;" OwnerUserId="157" LastActivityDate="2014-05-10T03:58:02.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11598" PostTypeId="2" ParentId="11586" CreationDate="2014-05-10T15:09:10.170" Score="1" Body="&lt;p&gt;To import C or C++ code into Python, please see &lt;a href=&quot;http://www.swig.org/&quot; rel=&quot;nofollow&quot;&gt;SWIG&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it might be easier to try to fix the errors you're seeing with installing or importing the &lt;code&gt;pari-python&lt;/code&gt; library, so maybe post the error(s) you're seeing as a separate question and get them resolved.&lt;/p&gt;&#xA;" OwnerUserId="8168" LastActivityDate="2014-05-10T15:09:10.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11599" PostTypeId="2" ParentId="11574" CreationDate="2014-05-10T19:22:41.167" Score="1" Body="&lt;p&gt;Pretty much any biology can be made computational. Now as it is called &quot;computational&quot; I hope you can do some programming, for that's what it is going to be about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For gene stuff I think databases and algorithms are important, for they deal with big data. This is an area I know little about, but it seems what other people have been suggesting are mainly useful here (i.e. basic sort algorithms, databases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Systems biology on the other hand is often more concerned with somewhat specific, smaller systems (how genes inhibit, promote, and interact with each other). One typically models what are essentially chemical reactions. So in a way it is about expanding the Michaelis-Menten equilibrium kinetics to smaller populations with noise. There are many ways to approach this problem, numerically the most popular one is perhaps the Gillespie method. Mastering this along with enzyme (chemical) reaction rate constants is not difficult supposing you remember your differential equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now a lot of the people involved in the programme seemed to have a physics background, and accordingly they had publications on membranes, the favorite model toy of biophysicists. A lot of interesting questions to ask about membranes are related to their elastic behaviour; how much energy does it take to bend and how much do thermal fluctuations cause it to do so, as a textbook example. This is most typically done by the use of the so called Helfrich Hamiltonian. To be able to analyze these, you'd have to remind yourself of equilibrium thermodynamics: Free energy, the equipartition theorem, and the fluctuation-dissipation theorem come to mind. From mathematical tools you'd need Fourier analysis and calculus of variations (i.e. Euler-Lagrange). Still related to membranes is phase separation and basically all of thermodynamics in general. Learning and coding the Ising model gets you a long way in this regard. Hydrodynamics, too, are important not only to membranes, but to a lot of research in computational biology in general. Finally I'd like to point out that DNA is often also related to statistical mechanics, thermodynamics and hydrodynamics, the most simple models of its stretching are in fact based on the Ising model (and polymer physics). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, you have intense computational modeling of small biological structures. Perhaps most computational work on proteins is about homology modeling and here you might be using tools from quantum physics and chemistry, so knowing some of the terminology there might be useful (say at the level of what Hartree-Fock is, what DFT stands for, and what are pi orbitals). More often than not, these results are parametrized to a classical force field, and run using a molecular dynamics simulation. Coding up a simple MD program (say of three planets attracting one another) should again be a simple task. Some keywords here, if you want to delve deeper, might be AMBER, CHARMM, GROMACS (these are programs that are used with biomolecular simulations, and while learning to use them is a whole other matter, their manuals and tutorials might help you understand what they are about).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I did take a very particular route down from genes to single molecules in my explanation here, and there are many things that I did not touch upon (given that my background is in physics, not biology nor in data science, a major pillar in anything computational). In particular, I did not discuss active processes, something that is of course very important in biology: This is why I referenced (equilibrium) thermodynamics so many times above, and it can really only be used in equilibrium. Nevertheless, it is difficult to understand nonequilibrium without first being introduced to equilibrium. I, however, think that given the interests of the researchers affiliated with the programme, some of the topics I mentioned above may well come up. Do also note that I did, in places, go quite a bit into the technical detail, something you might not be willing to do due to the time constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I'd like to mention Coursera.org, for they have had several offerings of systems biology and the like, so you might find that a useful resource.&lt;/p&gt;&#xA;" OwnerUserId="8143" LastActivityDate="2014-05-10T19:22:41.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11600" PostTypeId="1" CreationDate="2014-05-11T10:45:35.233" Score="0" ViewCount="98" Body="&lt;p&gt;In &lt;a href=&quot;http://www.lptms.u-psud.fr/membres/trizac/Ens/npac_vanRoij.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; paper, the theory behind the isotropic-nematic phase transition is discussed. Furthermore, an algorithm is given to calculate some properties of this phase transition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written a program in C which gives me the free energy, pressure, chemical potential and nematic order parameter for a given concentration $c$ of the particles in the system. Now I want to find the concentrations $c_I$ and $c_N$, which define the interval where the system phase separates, i.e., it is partly nematic and partly isotropic in order to lower its free energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the article explains, one needs to solve the system of equations&#xA;$$&#xA;p_{\text{iso}}(c_I) = p(c_N) \\&#xA;\mu_{\text{iso}}(c_I) = \mu(c_N),&#xA;$$&#xA;for $c_{I,N}$. The expressions $p_{\text{iso}}$ and $\mu_{\text{iso}}$ are analytically known and I can calculate $p$ and $\mu$ for certain discrete values of of the concentration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how to solve these two equations in C? I know that one can use Newton-Rhapson to solve zeros of a function numerically. However, as I only know the functions $p$ and $\mu$ for certain values of $c$, I do not really know how to tackle this problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me find a way to do this? (Preferably an algorithm of some sorts.) Disclaimer: I am starting to learn C, but I am not extremely good at it.&lt;/p&gt;&#xA;" OwnerUserId="8051" LastActivityDate="2014-05-12T08:53:31.473" Title="Calculation of isotropic-nematic phase transition" Tags="|c|newton-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11601" PostTypeId="2" ParentId="11600" CreationDate="2014-05-11T11:58:41.770" Score="1" Body="&lt;p&gt;I don't have enough reputation to write a comment, so this one goes down as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I take it you've written a code to solve the angular distribution $\psi$ numerically for a given concentration $c$, using the self-consistent procedure outlined in the paper. Now that you can write the free energy as a function of phi, and the pressure is the $c$-derivative of the free energy, you can, in principle, get the pressure for any value of $c$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am slightly confused by the fact that it seems, in the derivation for pressure, as if the $\psi$s do not depend on $c$. And they obviously do. So I believe there is an implicit approximation here. If you keep to the same approximation, you can write down the derivative of the pressure by hand. You have then a way to evaluate both the pressure and its derivative at any given $c$ (this is why I am not sure what you mean when you say that you only know them for certain values of $c$). Now that you have these functions (and similarly for the chemical potentials), writing down the Newton-Raphson scheme should pose no problem (you have a two variable, two function setting, so you'll have to differentiate both functions with respect to each variable, and then invert the resulting Jacobian matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;So I decided to write the code myself, in &quot;Python&quot;. I say &quot;Python&quot; instead of Python, for I used it as a quick drop in toolbox. I uses ipython with the -pylab switch so it'll load parts of scipy, numpy for computations.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ntheta  = 128&#xA;ks = arange(Ntheta+1)[1:]&#xA;&#xA;thetas = pi/2.*ks/(Ntheta+1)&#xA;&#xA;deltas = zeros_like(thetas)&#xA;deltas[0]    = 1 - (cos(thetas[0]) + cos(thetas[1]))/2.&#xA;deltas[1:-1] = (cos(thetas[:-2]) - cos(thetas[2:]))/2.&#xA;deltas[-1]   = (cos(thetas[-1]) + cos(thetas[-2]))/2.&#xA;&#xA;Nphi = 256&#xA;js = arange(Nphi+1)[1:]&#xA;phis = 2.*pi*js/(Nphi+1)&#xA;Ks = zeros((Ntheta, Ntheta))&#xA;for k in range(Ntheta):&#xA;  for l in range(Ntheta):&#xA;    g = sqrt(1. - (cos(thetas[k])*cos(thetas[l]) + sin(thetas[k])*sin(thetas[l])*cos(phis))**2)&#xA;    Ks[k, l] = 2.*pi/(Nphi+1) * (3./2*g[0] + sum(g[1:-1]) + 3./2*g[-1])&#xA;&#xA;def thefunc(c):&#xA;  taus = (c/pi)**2 * exp(-2*c**2*thetas**2/pi)&#xA;&#xA;  for i in range(128):&#xA;    As = 16.*c/pi * dot(Ks, deltas*taus)&#xA;    Z  = 4.*pi*sum(deltas*exp(-As))&#xA;    psis = 1./Z * exp(-As)&#xA;    taus = psis.copy()&#xA;&#xA;  S = 4.*pi*sum(deltas*psis*(3.*cos(thetas)**2-1.)/2.)&#xA;  theint = dot(deltas*psis, dot(Ks, deltas*psis))&#xA;  p = c + 32.*c**2*theint&#xA;  pder = 1. + 64.*c*theint&#xA;  f  = log(c) - 1. + 4.*pi*sum(deltas*psis*log(psis)) + 32.*c*theint&#xA;  mu = f + p/c&#xA;  muder = pder/c&#xA;  return p, pder, mu, muder&#xA;&#xA;def isofunc(c):&#xA;  p = c + c**2&#xA;  pder = 1. + 2.*c&#xA;  mu = log(c/(4.*pi)) + 2.*c&#xA;  muder = 1./c + 2.&#xA;  return p, pder, mu, muder&#xA;&#xA;def totalfunc(ct, c):&#xA;   pt, pdert, mut, mudert = thefunc(ct)&#xA;   p, pder, mu, muder     = isofunc(c)&#xA;   F = -array([pt - p, mut - mu])&#xA;   J = array([[pdert, -pder], [mudert, -muder]])&#xA;   dc = solve(J, F)&#xA;   return ct+dc[0], c+dc[1]&#xA;&#xA;ct = 4.&#xA;c  = 3.&#xA;for i in range(100):&#xA;  ct, c = totalfunc(ct, c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here &lt;code&gt;thefunc&lt;/code&gt; computes the self consistent solution for $\psi$ returning the pressure, the chemical potential and their derivatives. &lt;code&gt;isofunc&lt;/code&gt; does the same for the isotropic phase. &lt;code&gt;totalfunc&lt;/code&gt; does a Newton-Raphson iteration. Sorry for the names of the functions. The variables should be self explanatory, though.&lt;/p&gt;&#xA;" OwnerUserId="8143" LastEditorUserId="8143" LastEditDate="2014-05-12T08:53:31.473" LastActivityDate="2014-05-12T08:53:31.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11602" PostTypeId="2" ParentId="11582" CreationDate="2014-05-11T14:11:10.107" Score="3" Body="&lt;p&gt;It turns out I wasn't multiplying the dual variable by 4 on upsampling which kept it out of sync with the primal variable (because the upsampling adds zero rows/cols and blurs so to maintain value integrity we need to multiply by the ratio of new zero entires to old entries). So trivial and yet so frustrating.&lt;/p&gt;&#xA;" OwnerUserId="8160" LastActivityDate="2014-05-11T14:11:10.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11603" PostTypeId="2" ParentId="2100" CreationDate="2014-05-11T14:11:38.473" Score="1" Body="&lt;p&gt;There is a deterministic annealing algorithm which solves the one-to-one assignment problem&#xA;or equivalently the dyadic matrix partition problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However instead of using integer [0, 1] values one can use fractional values (so the algorithm remains the same) or even extend it to handle more than one assignment (by adding an inner loop and eseentially the matrix becomes a hyper-dimensional array or tensor)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper is here:&#xA;&lt;a href=&quot;http://www.researchgate.net/publication/2382666_Pairwise_Data_Clustering_by_Deterministic_Annealing/file/d912f50c75945d835b.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.researchgate.net/publication/2382666_Pairwise_Data_Clustering_by_Deterministic_Annealing/file/d912f50c75945d835b.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8172" LastActivityDate="2014-05-11T14:11:38.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11604" PostTypeId="2" ParentId="7001" CreationDate="2014-05-11T14:18:46.900" Score="0" Body="&lt;p&gt;Maybe a simulated annealing procedure can handle round-off errors better (and is easily parallelized).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You start with a crude grid of the search area and an initial &quot;temperature&quot; parameter&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each step you compute posible solution points (one can also accept non-solution points, with some probability inversely analogous to the temperature)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then retain only the solutions at that step and increase the temperature which provides a more fine-grained grid for the next iteration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do this until temperature &amp;lt; a given limit/threshold of accuracy&lt;/p&gt;&#xA;" OwnerUserId="8172" LastActivityDate="2014-05-11T14:18:46.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11605" PostTypeId="2" ParentId="11122" CreationDate="2014-05-11T18:49:41.300" Score="0" Body="&lt;p&gt;This is an engineer's solution :) First, transform the problem like I explained in my question so that we (a) take advantage of $\mathbf{K}$ being invertible (while $\mathbf{M}$ is not), and BTW we look for the largest eigenvalue $e'$, which of course will be positive. Then, the smallest positive eigenvalue is $e=1/e'$, and that is what we want! &#xA;$$&#xA;\mathbf{A}\,\mathbf{u} = e'\,\mathbf{u}&#xA;$$&#xA;with &#xA;$$&#xA;\mathbf{A}=\mathbf{K}^{-1}\,\mathbf{M} &#xA;$$&#xA;Now, the power method finds the largest eigenvalue (and its eigenvector) by iterating on: &#xA;$$&#xA;\mathbf{x}^k = \mathbf{A}\,\mathbf{x}^{k-1}\quad\text{for}\quad k=1,2,...&#xA;$$&#xA;But you do not need to invert $\mathbf{K}$. Instead solve &#xA;$$&#xA;\mathbf{LU}\, \mathbf{x}^k = \mathbf{M}\,\mathbf{x}^{k-1}&#xA;$$&#xA;where $\mathbf{LU}$ is the Cholesky factorization of $\mathbf{K}$. Super fast and easy!&lt;/p&gt;&#xA;" OwnerUserId="7693" LastActivityDate="2014-05-11T18:49:41.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11606" PostTypeId="2" ParentId="11264" CreationDate="2014-05-11T21:04:18.450" Score="1" Body="&lt;p&gt;You are compiling without optimizations enabled:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;g++ glm.cpp --std=c++11 -larmadillo -llapack -lblas -o bin&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, try using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;g++ &lt;b&gt;-O3 -DNDEBUG&lt;/b&gt; glm.cpp --std=c++11 -larmadillo -llapack -lblas -o bin&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, consider using &lt;a href=&quot;http://clang.llvm.org/&quot; rel=&quot;nofollow&quot;&gt;Clang&lt;/a&gt; as an alternative compiler to &lt;code&gt;g++&lt;/code&gt; and see which produces faster code. It accepts the same command-line flags as &lt;code&gt;g++&lt;/code&gt; so it should be a drop-in replacement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I have worked on &lt;a href=&quot;http://llvm.org/&quot; rel=&quot;nofollow&quot;&gt;LLVM&lt;/a&gt; (which Clang uses internally).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A secondary and minor point is that you are passing around &lt;code&gt;std::string&lt;/code&gt; by value, which causes many copies to be made (which requires memory allocation, data copies, and memory deallocation):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arma::mat getz(&#xA;    arma::mat&amp;amp; y,&#xA;    arma::mat&amp;amp; beta,&#xA;    arma::mat&amp;amp; X,&#xA;    std::string family,&#xA;    std::string link&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Instead, pass non-modified data by const reference to make it more efficient:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arma::mat getz(&#xA;    arma::mat&amp;amp; y,&#xA;    arma::mat&amp;amp; beta,&#xA;    arma::mat&amp;amp; X,&#xA;    const std::string&amp;amp; family,&#xA;    const std::string&amp;amp; link&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and similarly for the &lt;code&gt;glmMat()&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may not matter much in this case (the optimization flag is likely more important), but keep this in mind in the future, e.g., if you're passing around large matrices by value and they're constant, they can be passed around by const reference instead.&lt;/p&gt;&#xA;" OwnerUserId="8168" LastEditorUserId="8168" LastEditDate="2014-05-14T01:23:22.743" LastActivityDate="2014-05-14T01:23:22.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11608" PostTypeId="2" ParentId="11572" CreationDate="2014-05-12T08:50:45.987" Score="4" Body="&lt;p&gt;The problem was in the underlying coupled PDE FEM model. This model includes a potential which is not fixed in any point, so any result potential = phi + C would be a solution, where C is an integration constant. I don't know why, but the direct solver (Matlab's \ using UMFPACK) finds a solution, where the iterative solver gets in massive troubles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first checked all the sub matrices on the diagonal - the mapping of each variable in my coupled PDE system to itself. So I found that the matrix for this potential has very poor condition. Second, I found that eliminating any of the rows + corresponding column in that matrix would make condition fine, which is a hint that one of the rows/columns are linearly dependent which tells me, that there's one degree of freedom too much. I suspected that it's the integration constant which proved to be the right guess after fixing one of the points.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2014-05-12T08:50:45.987" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11609" PostTypeId="1" AcceptedAnswerId="11615" CreationDate="2014-05-12T16:45:32.310" Score="1" ViewCount="457" Body="&lt;p&gt;I would like links to resources which provide codes for scripting (preferably in python) for high performance computing. I'm looking for sample codes and examples which can,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) automatically create various input files based on the ranges provided for input values or by performing combinatorics of input values (ex: atomic positions that can be switched in a quantum mechanical simulation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) create batch submission files (ex:PBS) based on the provided input values of no. of nodes/processors provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Compare the output files of standard and non-standard (just column) formats and create plots as needed.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2014-07-06T22:16:30.083" Title="Scripting for High Performance Computing" Tags="|python|parallel-computing|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11610" PostTypeId="2" ParentId="11609" CreationDate="2014-05-12T17:13:28.763" Score="0" Body="&lt;p&gt;Might &lt;a href=&quot;http://www.unicore.eu/&quot; rel=&quot;nofollow&quot;&gt;UNICORE&lt;/a&gt; fit the bill? It is more than a scripting language, as it attempts to provide a uniform computing interface to the user, so one needs not worry about the kind of clusters accepting the job etc (i.e. no need to tune PBS or other queuing formats by hand). It also provides a graphical workflow, which can help the replicability of numerical experiments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will require the cluster admin to install the software on the server side, but I think the people from UNICORE are usually happy to help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I tend to fall back on my old habit of hacking together one-off solutions combining awk and bash scripts very specific to the given task.&lt;/p&gt;&#xA;" OwnerUserId="8143" LastActivityDate="2014-05-12T17:13:28.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11611" PostTypeId="2" ParentId="11590" CreationDate="2014-05-12T20:21:37.067" Score="7" Body="&lt;p&gt;Gmsh (&lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://geuz.org/gmsh/&lt;/a&gt;) will do it for you. Simply create all of the volumes, surfaces, and lines that are required to describe the geometry and interfaces. Then, assign a &quot;physical&quot; id to everything that you want meshed. Once you mesh the geometry, everything to which you have assigned a physical ID will be meshed. The &quot;zero thickness&quot; elements will simply be 2D elements (tri/quad) in a 3d mesh or line elements in a 2D mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gmsh feels a bit clunky at first, but once you get the hang of using the interface or writing input files manually, it can be a very easy way to generate unstructured meshes. Below, I have pasted a simple input file that generates a square with a diagonal line interface. To use this in gmsh, copy the below text into a file with a .geo extension (doesn't matter, this is convention). Then (in Linux) from the command line, run &quot;gmsh -2 (your file name).geo&quot;. This will create (your file name).msh in the same directory. To visualize the mesh, run &quot;gmsh (your file name).msh&quot;. Running gmsh on the .geo file will display the geometry, but not the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an image of a mesh created by the following input file:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/fVTG3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lc=0.1; //characteristic mesh size (optional make smaller to refine mesh)&#xA;&#xA;// Place points&#xA;Point(1) = {0,0,0,lc};&#xA;Point(2) = {1,0,0,lc};&#xA;Point(3) = {1,1,0,lc};&#xA;Point(4) = {0,1,0,lc};&#xA;&#xA;// Create lines from points&#xA;Line(1) = {1,2};&#xA;Line(2) = {2,3};&#xA;Line(3) = {3,4};&#xA;Line(4) = {4,1};&#xA;Line(5) = {3,1};&#xA;&#xA;// Define line loops used to construct surfaces&#xA;Line Loop(1) = {1,2,5};&#xA;Line Loop(2) = {-5,3,4};&#xA;&#xA;// Make surfaces from line loops&#xA;Plane Surface(1) = {1}; //using LL #1&#xA;Plane Surface(2) = {2}; //using LL #2&#xA;&#xA;// Showing off another gmsh feature:&#xA;// Tell gmsh to use mostly quads rather than tris for surface #2&#xA;Recombine Surface(2);&#xA;&#xA;// Give physical IDs to everything you want meshed&#xA;Physical Line(1) = {5}; //Line #5 --&amp;gt; Interface&#xA;Physical Surface(2) = {1}; //Surface #1 --&amp;gt; Lower triangle (tris)&#xA;Physical Surface(3) = {2}; // Surface #2 --&amp;gt; Upper triangle (mostly quads)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7378" LastEditorUserId="7378" LastEditDate="2014-05-13T05:53:10.387" LastActivityDate="2014-05-13T05:53:10.387" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11612" PostTypeId="2" ParentId="3029" CreationDate="2014-05-12T20:53:08.160" Score="0" Body="&lt;p&gt;You can have the best of both worlds. NIST has &lt;a href=&quot;http://www.itl.nist.gov/div898/strd/lls/lls.shtml&quot; rel=&quot;nofollow&quot;&gt;a set of problems&lt;/a&gt; for minimizers, like fitting this &lt;a href=&quot;http://www.itl.nist.gov/div898/strd/lls/data/Filip.shtml&quot; rel=&quot;nofollow&quot;&gt;10th degree polinomial&lt;/a&gt;, with expected results and uncertainties. Of course, proving that these values are the actual best solution, or the existence and properties of other local minima is more difficult than on a controlled mathematical expression.&lt;/p&gt;&#xA;" OwnerUserId="7759" LastActivityDate="2014-05-12T20:53:08.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11613" PostTypeId="2" ParentId="11586" CreationDate="2014-05-12T21:13:48.173" Score="0" Body="&lt;p&gt;An alternative to &lt;a href=&quot;https://github.com/swig/swig&quot; rel=&quot;nofollow&quot;&gt;SWIG&lt;/a&gt; is &lt;a href=&quot;http://docs.cython.org/src/userguide/external_C_code.html&quot; rel=&quot;nofollow&quot;&gt;Cython&lt;/a&gt;. In &lt;a href=&quot;http://pyvideo.org/video/1692/cython-vs-swig-fight&quot; rel=&quot;nofollow&quot;&gt;Cython vs SWIG, fight!&lt;/a&gt; you can see a comparison of the syntax for both of them (TL;DW: both need you to write some boating code, each one on a different corner).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the main advantage of Cython is the syntax, closer to Python, and that you can easily and progressively convert Python code to C. It is under heavy development, but it is quite stable. It is what most major scientific libraries are using (Numpy, Scipy, scikits...), with a lot of documentation oriented to the scientific stack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SWIG is a more mature project, also maintained and compatible with other programming languages. I know of some research projects that make heavy use of it; but as far as I can remember, none of the libraries I have ever used in detail.&lt;/p&gt;&#xA;" OwnerUserId="7759" LastActivityDate="2014-05-12T21:13:48.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11614" PostTypeId="2" ParentId="10384" CreationDate="2014-05-13T02:08:00.137" Score="3" Body="&lt;p&gt;&quot;pimple&quot; is a good solver to work with, as it handles transient cases with RAS or LES turbulence.  To change turbulence models, you'll need to specify this in your &quot;turbulenceProperties&quot; file by changing &quot;simulationType&quot; from &quot;RASModel;&quot; to &quot;LESModel;&quot; and providing an appropriate &quot;lesProperties&quot;file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest checking out the channel395 tutorial in tutorials\incompressible\pimpleFoam to see the usage of LES turbulence models.   You'll also need the same boundary conditions for the solver (such as nuSgs, nuTilda, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That should be about it, and if you're missing something the error should point you toward something else to copy/modify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For grid refinement, you'll want to have a much finer grid in regions with large gradients, because LES needs a finer resolution to resolve the turbulent kinetic energy accurately.  It really depends on your flow conditions though.  Try solving with your initial RAS mesh, then refine it in areas with sharp velocity gradients to improve your results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't use &quot;pimpleDyMFoam&quot; for my research, but I am studying turbulent jets using &quot;pimpleFoam&quot; as a solver.  pimpleDyM just uses a moving mesh.&lt;/p&gt;&#xA;" OwnerUserId="8179" LastEditorUserId="8179" LastEditDate="2014-05-13T02:14:03.967" LastActivityDate="2014-05-13T02:14:03.967" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11615" PostTypeId="2" ParentId="11609" CreationDate="2014-05-13T09:23:51.530" Score="1" Body="&lt;p&gt;Personally I would break the problem into to separate parts, the Python part (if that is your weapon of choice) doing file processing etc and the PBS job launch script part. I would definitely use bash scripting for the PBS part because it sounds like you want to make (easy) use of PBS variables and leverage what PBS gives you to launch in a variety of ways. Your PBS script then calls your Python code from within the bash script with these options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I have used are job arrays, which allow the PBS variable to give each job a different value in a $PBS_ARRAYID environment variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also used the -v option to pass my own user defined variable names. This has the advantage that you define your variables at launch time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use the job name itself to launch your code with a particular parameter. This has an advantage in that the result log file will also has the job name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quickest way to see what you are working with is to launch a job with &quot;env&quot; and you can see what you get to work with in the output. It is worth doing this as I have found what you get depends on the batch system and how the job is launched. Also be warned that what the documentation (man qsub) says an option &quot;should&quot; do and what it &quot;actually&quot; does are often different (so trial and error is called for).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example of how I launch OpenFOAM to adapt to the number of cores/nodes provided. It creates a config file that OpenFOAM interprets for splitting up the calculation. I used the jobname to configure the speed of the simulation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#PBS -N vespa_LES&#xA;#PBS -l walltime=48:00:00&#xA;#PBS -l nodes=2:ppn=10,pmem=4000MB&#xA;#PBS -m abe&#xA;&#xA;cd $PBS_O_WORKDIR&#xA;    echo Here are the Vespa variables used&#xA;    cat ./vespa_variables&#xA;    echo Running with $PBS_NP processors across $PBS_NUM_NODES nodes. $PBS_NUM_PPN procs per node&#xA;rm ./vespa_env&#xA;echo &quot;vespa_env_procs             $PBS_NP;&quot; &amp;gt;&amp;gt; ./vespa_env&#xA;    echo &quot;vespa_env_num_nodes         $PBS_NUM_NODES;&quot; &amp;gt;&amp;gt; ./vespa_env&#xA;echo &quot;vespa_env_num_ppn           $PBS_NUM_PPN;&quot; &amp;gt;&amp;gt; ./vespa_env&#xA;    echo &quot;vespa_env_jobname          $PBS_JOBNAME;&quot; &amp;gt;&amp;gt; ./vespa_env&#xA;cat ./vespa_env&#xA;&#xA;module purge&#xA;module load openfoam/2.2.2&#xA;. $foamDotFile&#xA;./Allclean&#xA;./Allrun&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The short version of the above is that PBS provides some helper variables and I would use those to simplify launching a variety of jobs with the same code.&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2014-05-13T09:23:51.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11616" PostTypeId="1" CreationDate="2014-05-13T11:36:14.433" Score="4" ViewCount="6791" Body="&lt;p&gt;The code below basically illustrates my problem. It is a test code for a pendulum. I solve it using a method suggested on &lt;a href=&quot;https://stackoverflow.com/questions/12926393/using-adaptive-step-sizes-with-scipy-integrate-ode&quot;&gt;https://stackoverflow.com/questions/12926393/using-adaptive-step-sizes-with-scipy-integrate-ode&lt;/a&gt;). Now I want to plot points at a range of evenly spaced times to show the fact the dynamic nature of the pendulum swings. For example I want to get the solution and plot the corresponding solution state space points at the times  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0,0.2,0.4,0.6,0.8,1.0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any automatic way Python can do this (using interpolation of the internal solution points which are obtained during the ODE integration)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the main code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.integrate import ode&#xA;import matplotlib.pyplot as plt&#xA;import warnings&#xA;&#xA;def f(t,y):&#xA;   l = 1&#xA;   m = 1&#xA;   d = 1&#xA;   g = 9.8&#xA;   return [y[1], -np.sin(y[0])*g/l-y[1]*d/m]&#xA;&#xA;&#xA;y0, t0 = [np.pi/2, 0], 0&#xA;t1 = 500&#xA;&#xA;backend = 'vode'&#xA;&#xA;solver = ode(f).set_integrator(backend, nsteps=1)&#xA;solver.set_initial_value(y0, t0)&#xA;# suppress Fortran-printed warning&#xA;solver._integrator.iwork[2] = -1&#xA;&#xA;y, t = [], []&#xA;warnings.filterwarnings(&quot;ignore&quot;, category=UserWarning)&#xA;while solver.t &amp;lt; t1:&#xA;    solver.integrate(t1, step=True)&#xA;    y.append(solver.y)&#xA;    t.append(solver.t)&#xA;warnings.resetwarnings()&#xA;y = np.array(y)&#xA;t = np.array(t)&#xA;&#xA;plt.plot(y[:,0], y[:,1], 'b-')&#xA;plt.plot(y[0::5,0], y[0::5,1], 'b.')&#xA;&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2021-09-05T02:52:27.860" Title="How to get ODE solution at specified time points?" Tags="|algorithms|python|numerical-analysis|ode|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11617" PostTypeId="2" ParentId="10384" CreationDate="2014-05-13T11:36:15.247" Score="0" Body="&lt;p&gt;I used 'mapFields' utility, which can be used to map field values from any arbitrary case to other, irrespective of turbulence model used, just the boundary patches need to be specified in one-to-one correspondence in 'mapFieldsDict' ins system/ directory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for LES case set-up one can see any of the myriads of LES tutorials (in my case incompressible)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information,&lt;a href=&quot;http://www.openfoam.org/docs/user/mapFields.php&quot; rel=&quot;nofollow&quot;&gt;see this&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6878" LastActivityDate="2014-05-13T11:36:15.247" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11618" PostTypeId="2" ParentId="11616" CreationDate="2014-05-13T14:33:10.777" Score="3" Body="&lt;p&gt;The example given in the docs of scipy.integrate.ode hints at using successive integration between your points of interest:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; r = ode(f, jac).set_integrator('zvode', method='bdf', with_jacobian=True)&#xA;&amp;gt;&amp;gt;&amp;gt; r.set_initial_value(y0, t0).set_f_params(2.0).set_jac_params(2.0)&#xA;&amp;gt;&amp;gt;&amp;gt; t1 = 10&#xA;&amp;gt;&amp;gt;&amp;gt; dt = 1&#xA;&amp;gt;&amp;gt;&amp;gt; while r.successful() and r.t &amp;lt; t1:&#xA;&amp;gt;&amp;gt;&amp;gt;     r.integrate(r.t+dt)&#xA;&amp;gt;&amp;gt;&amp;gt;     print(&quot;%g %g&quot; % (r.t, r.y))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The scipy.integrate.odeint function takes an input argument 't': A sequence of time points for which to solve for y. The initial value point should be the first element of this sequence.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2014-05-13T14:33:10.777" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11619" PostTypeId="2" ParentId="11616" CreationDate="2014-05-13T19:08:20.817" Score="5" Body="&lt;p&gt;See also: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10540/python-grid-with-step-control-ode-solver?rq=1&quot;&gt;Python: Grid with step control ODE solver&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GertVdE's answer covers the user-facing implementation well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, you want two different things to happen:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the integrator should take as large a time step as possible&lt;/li&gt;&#xA;&lt;li&gt;you want output at regularly-spaced points in time anyway&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here's how that works in practice:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The integrator has its own internal time step information. If you were to use VODE in &quot;one-step&quot; mode (calling the Fortran library DVODE, or its successor, CVODE, in SUNDIALS), it would take as large a step as possible (subject to constraints on stability and error). It would then output that step. If you were to do this repeatedly, your output would be the time steps of the integrator. However, that output is not what you want -- you want regularly spaced output.&lt;/li&gt;&#xA;&lt;li&gt;The integrator can also interpolate between internal time steps using the polynomial approximation constructed by the integrator. In Hairer and Wanner, this functionality is called &quot;dense output&quot;. So when you query output from the integrator (as it generates it in time) between internal time steps, what it is really doing is interpolating between these time steps to get the output.&lt;/li&gt;&#xA;&lt;li&gt;The interface GertVdE describes seems to use &quot;normal&quot; mode, which does dense output automatically and takes time steps internally, because most users care about the output they get, and aren't concerned about the numerical details in the implementation, so long as they work correctly.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can see all of these details in the &lt;a href=&quot;http://computation.llnl.gov/casc/sundials/support/notes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SUNDIALS Usage Notes&lt;/a&gt; and in the various &lt;a href=&quot;http://computation.llnl.gov/casc/sundials/documentation/documentation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SUNDIALS Users Guides&lt;/a&gt;, which cover the theory behind the integrators as well as their use. Although these sources of documentation cover the successors to VODE (CVODE and CVODES), the theory should be the same, and the interfaces should be similar, except that CVODE and CVODES have additional functionality.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-05-13T23:30:55.720" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11620" PostTypeId="1" CreationDate="2014-05-14T01:18:14.163" Score="11" ViewCount="445" Body="&lt;p&gt;Jacobian-Free Newton-Krylov (JFNK) methods, and Krylov methods in general, can be very useful because they don't require explicit storage or construction of a matrix, only the results of matrix-vector products. If you do actually form the sparse system, there's many preconditioners out there for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is available for true matrix-free methods? Googling turns up some references to &quot;matrix estimation&quot; and some other things indicating that it is possible. How do these methods generally work? How do they compare to traditional preconditioners? Are physics-based matrix-free preconditioners the way to go? Are there any openly available methods in the wild, say in PETSc or some other package?&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-05-15T22:35:11.893" Title="Do black-box preconditioners for matrix-free methods exist?" Tags="|fluid-dynamics|linear-solver|preconditioning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11625" PostTypeId="1" AcceptedAnswerId="11645" CreationDate="2014-05-14T15:33:30.657" Score="3" ViewCount="120" Body="&lt;p&gt;I am writing my undergrad thesis on the harmonic oscillator on a lattice. So&#xA;far I have implemented the Metropolis Monte Carlo algorithm to generate&#xA;trajectories $x_j$ for $0 \leq j &amp;lt; N$, where $N$ is the number of time lattice&#xA;divisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get a pretty good histogram for $|\psi(x)|^2$ and it matches the expectation&#xA;from the theory. Now I have to calculate the energy eigenvalues of the system.&#xA;The ground energy state can be calculated with $E_0 \propto \langle x^2&#xA;\rangle$ which has worked out so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first energy, $E_1$, should be computable using the exponential slope of&#xA;the correlations, &lt;a href=&quot;http://www.itkp.uni-bonn.de/~urbach/bnd/cf.pdf&quot; rel=&quot;nofollow&quot;&gt;Creutz and Freedman&#xA;(1980)&lt;/a&gt; wrote that this would&#xA;be doable with this: $$ E_1 = \frac{-1}{\Delta \tau} \log\left( \frac{\langle&#xA;x(0) x(\tau + \Delta \tau)\rangle}{\langle x(0) x(\tau) \rangle} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the $x_j$ that I have in my computations, I can calculate those&#xA;correlations to some value of $\tau$, limited by the $N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advisor has given me a paper from &lt;a href=&quot;http://arxiv.org/abs/0902.1265&quot; rel=&quot;nofollow&quot;&gt;Blossier et al&#xA;(2009)&lt;/a&gt; where they introduce a correlation&#xA;&lt;em&gt;matrix&lt;/em&gt; like so (2.1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ C_{ij} (t) = \sum_{n=1}^\infty \langle 0 | \hat O_i | n \rangle \langle 0 |&#xA;\hat O_j | n \rangle^* $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $O_i$ are “some interpolating fields $O_i(x_0)$ already projected to a&#xA;definite momentum and other quantum number such as parity”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From that, they show how the eigenvalues to the generalized eigenvalue problem&#xA;(GEVP) consisting of $C(t)$ and $C(t_0)$ will give the energy eigenvalues,&#xA;which I am interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://physics.ucsd.edu/students/courses/spring2013/physics142/Labs/FinalProjects/LuscherWolff.pdf&quot; rel=&quot;nofollow&quot;&gt;Lüscher and Wolff&#xA;(1990)&lt;/a&gt;&#xA;then write on page 245:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In a numerical simulation, the correlation matrix $C(t)$ can be expected to&#xA;  be computable for some range of $t$, and the basic technical problem then is&#xA;  to extract the levels $W_\alpha$. from $C(t)$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I do not see how I can get the $C_{ij}(t)$ out of my data. What are my fields&#xA;$O$ for a simple harmonic oscillator to start with? Are those the ladder&#xA;operators, so that $O_j$ = $(a^\dagger)^j$?&lt;/p&gt;&#xA;&#xA;&#xA;" OwnerUserId="492" LastActivityDate="2014-05-16T18:47:11.630" Title="Extract the correlation matrix from Monte Carlo data" Tags="|monte-carlo|quantum-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11626" PostTypeId="1" CreationDate="2014-05-14T17:04:02.090" Score="4" ViewCount="5713" Body="&lt;p&gt;There are several libraries to (iteratively) solve large sparse linear equation systems in parallel on a number of CPUs. Our parallel cluster also has attached powerful GPUs, but so far, I did not find any solver that can use CPUs and GPUs in parallel to solve a sparse linear equation system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MAIN QUESTION: Are there actually no solvers which use CPUs and GPUs or could you recommend me one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RELATED SIDE QUESTION: Many of the solver libraries can work with an user defined matrix-vector-product which is the basic operation of any iterative solver. So I could use the GPUs to help the CPUs computing the matrix-vector-product, BUT the problem there is: When not specifying the matrix explicitly, but just giving the matrix-vector-product, the (very important) preconditioner routines can not be used. Any ideas how to get over this problem?&lt;/p&gt;&#xA;" OwnerUserId="8138" LastEditorUserId="24110" LastEditDate="2017-05-21T20:46:36.937" LastActivityDate="2017-05-21T20:46:36.937" Title="GPU-accelerated libraries for solving sparse linear systems" Tags="|parallel-computing|linear-solver|libraries|gpu|preconditioning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11627" PostTypeId="1" CreationDate="2014-05-14T17:44:50.637" Score="2" ViewCount="505" Body="&lt;p&gt;Is there a C++/C implementation of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Appell_series&quot; rel=&quot;nofollow&quot;&gt;Appell series&lt;/a&gt;? GSL and Boost do not seem to have this function.&lt;/p&gt;&#xA;" OwnerUserId="988" LastActivityDate="2014-05-15T22:35:06.377" Title="Appell function implementation in C++?" Tags="|c|special-functions|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11628" PostTypeId="2" ParentId="11627" CreationDate="2014-05-14T19:12:20.940" Score="6" Body="&lt;p&gt;You're probably better served writing a C wrapper for the Fortran implementation you linked to: Colavecchia, F. D., Gasaneo, G., &quot;f1: a code to compute Appell's F1 hypergeometric function&quot;, Computer Physics Communications, Volume 157, Issue 1, p. 32-38 (2004), found at &lt;a href=&quot;http://cpc.cs.qub.ac.uk/summaries/ADSJ&quot; rel=&quot;nofollow&quot;&gt;http://cpc.cs.qub.ac.uk/summaries/ADSJ&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The R package &lt;a href=&quot;http://cran.r-project.org/web/packages/appell/index.html&quot; rel=&quot;nofollow&quot;&gt;appell&lt;/a&gt; wraps that implementation of Appell's $F_{1}$. The R package provides additional implementations (all Fortran with R wrappers) and references. The Python package &lt;a href=&quot;http://mpmath.org/&quot; rel=&quot;nofollow&quot;&gt;mpmath&lt;/a&gt; provides &lt;a href=&quot;https://github.com/fredrik-johansson/mpmath/blob/master/mpmath/functions/hypergeometric.py&quot; rel=&quot;nofollow&quot;&gt;its own implementation&lt;/a&gt;; you could try translating that to C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the niche field (special functions), and the lack of Google results for C or C++ implementations, the process of writing the wrapper (even if you have to learn how to do it) will probably take less time than searching another day or two for a native C or C++ implementation that may not even exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To write a wrapper, there are a number of sources available, but &lt;a href=&quot;http://www.fortran90.org/src/best-practices.html#interfacing-with-c&quot; rel=&quot;nofollow&quot;&gt;http://www.fortran90.org/src/best-practices.html#interfacing-with-c&lt;/a&gt; by Ondrej Certik is probably the best and most modern version describing the process of wrapping modern Fortran code; that is, Fortran 2003 or later. If the Colavecchia implementation is Fortran 90 or Fortran 95, you wouldn't have to change much of it beyond type declarations for the variables declared in the interface. The key is to use the &lt;code&gt;iso_c_binding&lt;/code&gt; module; you can Google that term and it will help you write C wrappers (and, by extension, C++ wrappers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the Colavecchia code is uses Fortran 77, Fortran 90, or Fortran 95, then there are a number of links that will describe the kludgey process needed to get your wrapper to work. In that case, you might try following one of these less confusing resources (recommendations welcome!):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://caligari.dartmouth.edu/doc/solaris-forte/manuals/fortran/prog_guide/11_cfort.html&quot; rel=&quot;nofollow&quot;&gt;http://caligari.dartmouth.edu/doc/solaris-forte/manuals/fortran/prog_guide/11_cfort.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.math.utah.edu/software/c-with-fortran.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.utah.edu/software/c-with-fortran.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.yolinux.com/TUTORIALS/LinuxTutorialMixingFortranAndC.html&quot; rel=&quot;nofollow&quot;&gt;http://www.yolinux.com/TUTORIALS/LinuxTutorialMixingFortranAndC.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;with the following caveats:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I would ignore any recommendations of automated language wrappers or language headers for the time being; if they work, they're useful, but I've never gotten them to work for me, plus, you probably want to wrap a small number of functions, so automation won't be that helpful for you. (For a large project, such as PETSc, automated wrapper generation is essential.)&lt;/li&gt;&#xA;&lt;li&gt;Replace any mention of &lt;code&gt;g77&lt;/code&gt; with &lt;code&gt;gfortran&lt;/code&gt;; &lt;code&gt;g77&lt;/code&gt; is no longer actively developed.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm sure there are other caveats, and it might be useful to turn &quot;How do I write a C wrapper to a Fortran function?&quot; into a separate question, in which case I'll migrate my answer and link to it.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-05-15T22:35:06.377" LastActivityDate="2014-05-15T22:35:06.377" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11631" PostTypeId="2" ParentId="11620" CreationDate="2014-05-15T13:28:38.757" Score="5" Body="&lt;p&gt;Maybe not a preconditioning strategy in the traditional sense, but deflation could be useful in this case. In gmres(A) for instance, you can use the eigenpairs of the hessenberg projection H to form ritz vectors that are good estimates for eigenvectors of A. You use that to deflate your residual upon a restart, and give speedups over traditional restarted gmres. [The harmonic ritz values can be used to find small eigenvalues of A and deflate them out, which is more useful IMO than deflating out large eigenvalues of A]. I think deflated variants exist for all kinds of krylov solvers (CG, etc), but I am most familiar with the concept in the context of restarted gmres.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could google for GMRES-DR for more info, I also ran across a matlab implementation of GCRODR written by someone at Sandia, shouldn't be hard to find it again.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2014-05-15T13:28:38.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11632" PostTypeId="1" CreationDate="2014-05-15T16:41:31.567" Score="1" ViewCount="187" Body="&lt;p&gt;I am trying to reconstruct a 3D surface given the normals of the unknown surface. Reading through &lt;a href=&quot;http://www.wisdom.weizmann.ac.il/~vision/photostereo/Photometric%20Stereo%20with%20General%20Unknown%20Lighting%20-%20BasriJacobsKemelmacher_ijcv06.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; on section 4 they say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] denote the surface by $z(x,y)$. The directions of the normals are given by $n(x,y) = (p,q,-1)^T$, with $p = z_x$ and $q = z_y$ which denote the partial derivatives of $z$ with respect to $x$ and $y$ respectively. The recovered scaled surface normal $(n_x, n_y, n_z)$ roughly satisfies $$(n_x,n_y,n_z)=\frac{\rho}{\sqrt{p^2+q^2+1}}(p,q,-1)$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From there, I understand the fact that $p = \frac{-n_x}{n_z}$ and $q = \frac{-ny}{nz}$ and since $p$ and $q$ are the partial derivatives of $z$ with respect to $x$ and $y$ respectively we can approximate them by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p = z(x+1,y)-z(x,y)$$&#xA;$$q = z(x,y+1)-z(x,y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what I don't really get is how to solve for $z$. All the PDEs I've seen so far are like $u_x+u_y = 0$ or $\Delta u = u_{xx}+u_{yy} = 0$ or something of that style. But here I don't see how can I integrate, this seems to be a system of equations (I'm not sure about this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain me how can I solve this numerically? should I just build 2 linear systems? since there is a term $z(x,y)$ in both equations can I equate them so to have only one and then solve the linear system? I believe there's something I am not considering/seeing/forgetting and it could be solved easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found some useful information &lt;a href=&quot;http://www.physicsforums.com/showthread.php?t=38257&quot; rel=&quot;nofollow&quot;&gt;in another forum&lt;/a&gt;. It is very related to what I am trying to do. Given the gradient field $\vec N$, in my case the normals (correct me if I'm wrong) I want to find a function $S(x,y,z(x,y))$ such that $\vec N = \nabla S$ (pretty much the same problem stated in the forum). I understand the way it could be done analytically, but I still cannot get my head around when going into the discrete formulation which is what I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analytically, first I integrate with respect to $x$ and that result I need to get the derivative in order to plug it into the equation where I need to integrate over $y$. How to connect these ideas to a numerical integration scheme?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2014-05-16T13:49:35.807" LastActivityDate="2014-07-16T21:10:48.000" Title="Surface normals integration" Tags="|pde|linear-solver|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11633" PostTypeId="2" ParentId="11632" CreationDate="2014-05-15T17:07:00.750" Score="0" Body="&lt;p&gt;If I am interpreting the question correctly, I think you would basically integrate this like an ODE using the euler method or perhaps something more advanced (backward euler, RK, whatever). Assuming that you know the p and q functions as givens, and are given some initial seed point z(x,y), you could &quot;timestep&quot; your two equations to figure out two new z-points on your surface, z(x+1,y)=z(x,y)+p and z(x,y+1)=z(x,y)+q. By repeating the process from those two new points, you can steadily fill out an entire lattice of z-points for all (x,y).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2014-05-15T17:07:00.750" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11635" PostTypeId="2" ParentId="11632" CreationDate="2014-05-15T19:33:31.227" Score="1" Body="&lt;p&gt;Supposing you know the boundary conditions, you can of course make a linear system and solve it (be it iteratively or by inverting the matrix). This would seem the most straightforward way to solve the other equations that you mentioned, too: $\Delta u = 0$, where up to a constant $\Delta \approx \begin{pmatrix}-1 &amp;amp; 2 &amp;amp; -1 &amp;amp; \cdots \\  0 &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp; \cdots\end{pmatrix}$ and so on for the 1D case (if this is not obvious, see &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_differences&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; Wikipedia article and the related pages), adding two extra lines for the BCs. The 2D case is not much more complicated: just write the points $u_{x, y}$ as a onedimensional vector $u_i$, where e.g. $i = Nx + y$, and the difference equations for each of those points collected to a matrix. I suggest writing this problem up with a source term $\Delta u = \rho$, and making sure that for some constant boundary conditions your solution indeed is a parabola. This is a good, quick exercise for figuring out how the BCs come into play. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was suggested by rchilton1980 that you time step, and this indeed works if you have all the initial conditions. From a quick reading of the paper, however, it seems that this might not be the case, but I must confess that I am not a 100% on this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for the problem you might write $i = Nx + y$, so that $z_i = z_{Nx+y} = z(x, y)$ (assuming that both directions $x$ and $y$ have $N$ points from $0$ to $N-1$, so $i$ runs from 0 to $N^2-1$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now $p = z(x + 1, y) - z(x, y)$ could be written in the form $p_i = z_{N(x+1)+y} - z_{Nx+y} = z_{i+N} - z_{i}$. This is valid for all $i&amp;lt;N^2-1-N$, so it can be written in the matrix form $p = Az$, where $A = \begin{pmatrix}-1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots \\ 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots \\ \vdots \end{pmatrix}$, where the 1 and -1 are separated by a distance $N$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now using the equation for $y$,&#xA;$q_i = z_{Nx+(y+1)} - z_{Nx+y} = z_{i+1} - z_{i}$, which is valid for $y \neq N-1$, you can get more rows to add to $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here you have &quot;too many&quot; lines in the matrix, and therefore the authors of the paper use least squares (or the pseudoinverse of $A$, if you like) to solve for $z$. &lt;/p&gt;&#xA;" OwnerUserId="8143" LastActivityDate="2014-05-15T19:33:31.227" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11636" PostTypeId="1" AcceptedAnswerId="11637" CreationDate="2014-05-15T21:05:53.677" Score="3" ViewCount="1084" Body="&lt;p&gt;I'm looking for software recommendations for an electrostatic simulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a 3d object of uniform, high conductivity, separated from its environment by a high-resistance skin. Inside that object is a dipole, and I want to calculate the strength of the dipole's generated electric field on the surface of the object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been working with a colleague's old thesis, in which he develops an approach to solve for the field using FEM/BEM. I've coded up his BEM solver in Matlab, but it's quite slow to run, and I'm not convinced I'm getting the right answer. I wouldn't be surprised if there was a tool for this already out there, but I haven't had much luck finding one so far. Any suggestions? I already have the mesh (which has ~3300 vertices, though I can edit this), I just want to pop it in with some conductivities and a dipole location, and get back the field on the surface of the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(For those curious: I study a species of fish that generates electric fields, and uses perturbations in these fields to detect objects in its environment; I want to do some simulations to get a better sense of what the sensory signal at the skin actually looks like.)&lt;/p&gt;&#xA;" OwnerUserId="8202" LastActivityDate="2014-05-18T01:10:54.163" Title="Tool for computing electric field on a 3d surface" Tags="|software|electromagnetism|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11637" PostTypeId="2" ParentId="11636" CreationDate="2014-05-15T21:49:56.603" Score="2" Body="&lt;p&gt;I'd write this down as a comment had I the necessary reputation, but here goes my suggestion (on a side note, do you have any references for fish electrostatics? Sounds interesting). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm surprised to see that you were coding up your own solver from scratch for production. When it comes to real systems, especially something like a fish with probably a really intricate geometry and complicated boundary conditions, I usually tend to go with ready-made software (assuming they have all the necessary features, or that I am familiar enough with the source that I can code them up where missing). This is not to say that they are always particularly easy to use, though, but they should be fast and reliable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when I was doing (rather simple) electrostatics calculations a couple of years back, I used the software &lt;a href=&quot;http://www.geuz.org/getdp/&quot; rel=&quot;nofollow&quot;&gt;GetDP&lt;/a&gt; to solve the equations, and the associated &lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;nofollow&quot;&gt;Gmsh&lt;/a&gt; to generate the meshes/geometries. The usage is simple enough through a GUI but when it comes to defining the equations to solve (the .pro file), you &lt;strong&gt;will&lt;/strong&gt; need to read the manual and/or look through some examples. This might take a couple of moments or more. The official website itself does not have that many on electrostatics, but Google searches seemed to land a couple (e.g. &lt;a href=&quot;http://onelab.info/wiki/Laplace_equation_with_Dirichlet_boundary_condition&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;). I'm sure there are other software packages that are easier to use with more out of the box solutions, but this package is very general and maybe of some use even if you work outside electromagnetic calculations.&lt;/p&gt;&#xA;" OwnerUserId="8143" LastActivityDate="2014-05-15T21:49:56.603" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11638" PostTypeId="2" ParentId="11620" CreationDate="2014-05-15T22:35:11.893" Score="2" Body="&lt;p&gt;It's going to be heavily dependent on your problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you mention fluid dynamics, you might look into BFBt approximate commutators which have been very effective for fluid dynamics problems with constraints such as incompresible Navier-Stokes,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/040608817&quot; rel=&quot;nofollow&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/040608817&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2014-05-15T22:35:11.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11639" PostTypeId="1" CreationDate="2014-05-16T03:07:02.980" Score="1" ViewCount="231" Body="&lt;p&gt;I have a distance matrix (square, symmetrical, non-negative, dense).&#xA;I want to split the objects into two well-connected groups.&#xA;Mathematically speaking, I want to group (re-arrange) the rows/columns so that when the matrix is viewed as a 2x2 block-diagonal matrix, the diagonal blocks are &quot;close to zero&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again: Given an square matrix $M$ of size $N$ I want to find a rearranged matrix $M'$ and a division point $k: 1 &amp;lt; k &amp;lt; N$ such that $norm(M_{1..k,1..k})$ and $norm(M_{k+1..N,k+1..N})$ are minimized (where  $norm$ is sum, max, sum of squares or some other norm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. My ultimate goal is to extract a hierarchy of well-connected components from a complete graph.&lt;/p&gt;&#xA;" OwnerUserId="8207" LastActivityDate="2014-05-19T19:24:53.860" Title="Rearrange a dense distance matrix to a 2x2 non-perfect block diagonal form" Tags="|matrix|graph-theory|dense-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11641" PostTypeId="1" CreationDate="2014-05-16T03:52:23.140" Score="3" ViewCount="32" Body="&lt;p&gt;Is there a way to achieve &quot;one cancels the other&quot; semantics (&lt;a href=&quot;http://www.investopedia.com/terms/o/oco.asp&quot; rel=&quot;nofollow&quot;&gt;http://www.investopedia.com/terms/o/oco.asp&lt;/a&gt;) for qsub jobs submitted to Sun Grid Engine? That is, I submit two (or more) jobs and when one of them runs it cancels the others?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two examples...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I can run for 4 hours on N ranks or 2 hours on 2N ranks. If both jobs can start soon, I'd rather get the answer 2 hours earlier by running the bigger job. If I'm going to wait more than 2 additional hours to run the 2N-rank job, however, it would make sense to run the smaller job first and cancel the bigger one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I can run part of a long running problem for 8 or 16 hours. There's some startup cost amortization for running longer batch jobs so 16 hours is a mildly better use of SUs. But if the 8 hour job can start sooner, I'd rather make progress now than wait for a 16 hour window.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibly another way to accomplish these things would be for me to submit a single job with a range of acceptable node counts (hypothetically, &quot;-pe 12way N,2N&quot;) or wall times (&quot;-l h_rt 16-8:00:00&quot;). I haven't been able to tease this out of the SGE man pages, however. &lt;/p&gt;&#xA;" OwnerUserId="327" LastEditorUserId="276" LastEditDate="2017-10-10T00:37:41.160" LastActivityDate="2017-10-10T00:37:41.160" Title="One cancels the other qsub semantics atop SGE?" Tags="|parallel-computing|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11642" PostTypeId="2" ParentId="11639" CreationDate="2014-05-16T04:35:00.790" Score="1" Body="&lt;p&gt;It sounds like you're looking for a &lt;a href=&quot;http://en.wikipedia.org/wiki/Minimum_cut&quot; rel=&quot;nofollow&quot;&gt;Min-cut&lt;/a&gt; of the graph. There is a lot of literature on the subject. You can repeated apply Min-cuts until you have a hierarchy of well connected components.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2014-05-16T04:43:09.523" LastActivityDate="2014-05-16T04:43:09.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11643" PostTypeId="1" CreationDate="2014-05-16T14:02:27.370" Score="5" ViewCount="180" Body="&lt;p&gt;I am trying to implement the algorithm presented in &lt;a href=&quot;http://perception.csl.illinois.edu/matrix-rank/Files/robust_stereo.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; which tries to recover a matrix that represent a less noisier dataset of the intensities of the pixels of a set of images. In this case the noise comes from the specularities and shadows of the material of the objects in the images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have little (or no) experience in convex optimization. My question is, what should the stopping criterion be in the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was using the $||A_{i+1}-A_{i}||_F$ where $F$ denotes the Frobenius norm and checking the difference between the current and previous recovered matrix $A$ but it results that the value of that norm remains the same iteration after iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone give me some advise to be able to implement the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After recoded the algorithm, I check both conditions as suggested in one of the answers and this is what the plots look like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $||E_{i+1}-E_{i}||_F$ I'm getting&#xA;&lt;img src=&quot;https://i.stack.imgur.com/NsUjc.png&quot; alt=&quot;Difference between the norm of E&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $||A_{i+1}-A_{i}||_F$ I'm getting&#xA;&lt;img src=&quot;https://i.stack.imgur.com/zAa7G.png&quot; alt=&quot;Difference between the norm of A&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know where the error could be, I went step by step over the code and it seems right to me.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2014-05-20T10:34:01.837" LastActivityDate="2014-07-19T11:30:05.053" Title="Matrix completion algorithm" Tags="|linear-algebra|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11644" PostTypeId="2" ParentId="11643" CreationDate="2014-05-16T15:29:21.970" Score="1" Body="&lt;p&gt;I am not sure how this algorithm works exactly, but I guess that $||A_i-A_{i+1}||_F \leq \epsilon_A$ is a fairly reasonable stopping criterion, but also $||E_i-E_{i+1}||_F \leq \epsilon_E$ seems to be needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If both  $||A_i-A_{i+1}||_F $ and $||E_i-E_{i+1}||_F $ do not decrease, then you are making some big mistake in the two inner loops of the algorithms.&lt;/p&gt;&#xA;" OwnerUserId="7396" LastActivityDate="2014-05-16T15:29:21.970" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11645" PostTypeId="2" ParentId="11625" CreationDate="2014-05-16T18:47:11.630" Score="0" Body="&lt;p&gt;I talked to my advisor. The fields are just the powers of $x$. In those, the ladder operators exist, and should give me the desired correlations.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2014-05-16T18:47:11.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11646" PostTypeId="2" ParentId="8899" CreationDate="2014-05-16T19:42:14.897" Score="1" Body="&lt;p&gt;Here is an implementation of a 2x2 SVD solve. I based it off of Victor Liu's code. His code was not working for some matrices. I used these two documents as mathematical reference for the solve: &lt;a href=&quot;http://www.ualberta.ca/~mlipsett/ENGM541/Readings/svd_ellis.pdf&quot; rel=&quot;nofollow&quot;&gt;pdf1&lt;/a&gt; and &lt;a href=&quot;http://www.imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/imm3274.pdf&quot; rel=&quot;nofollow&quot;&gt;pdf2&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix &lt;code&gt;setData&lt;/code&gt; method is in row-major order. Internally, I represent the matrix data as a 2D array given by &lt;code&gt;data[col][row]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void Matrix2f::svd(Matrix2f* w, Vector2f* e, Matrix2f* v) const{&#xA;    //If it is diagonal, SVD is trivial&#xA;    if (fabs(data[0][1] - data[1][0]) &amp;lt; EPSILON &amp;amp;&amp;amp; fabs(data[0][1]) &amp;lt; EPSILON){&#xA;        w-&amp;gt;setData(data[0][0] &amp;lt; 0 ? -1 : 1, 0, 0, data[1][1] &amp;lt; 0 ? -1 : 1);&#xA;        e-&amp;gt;setData(fabs(data[0][0]), fabs(data[1][1]));&#xA;        v-&amp;gt;loadIdentity();&#xA;    }&#xA;    //Otherwise, we need to compute A^T*A&#xA;    else{&#xA;        float j = data[0][0]*data[0][0] + data[0][1]*data[0][1],&#xA;            k = data[1][0]*data[1][0] + data[1][1]*data[1][1],&#xA;            v_c = data[0][0]*data[1][0] + data[0][1]*data[1][1];&#xA;        //Check to see if A^T*A is diagonal&#xA;        if (fabs(v_c) &amp;lt; EPSILON){&#xA;            float s1 = sqrt(j),&#xA;                s2 = fabs(j-k) &amp;lt; EPSILON ? s1 : sqrt(k);&#xA;            e-&amp;gt;setData(s1, s2);&#xA;            v-&amp;gt;loadIdentity();&#xA;            w-&amp;gt;setData(&#xA;                data[0][0]/s1, data[1][0]/s2,&#xA;                data[0][1]/s1, data[1][1]/s2&#xA;            );&#xA;        }&#xA;        //Otherwise, solve quadratic for eigenvalues&#xA;        else{&#xA;            float jmk = j-k,&#xA;                jpk = j+k,&#xA;                root = sqrt(jmk*jmk + 4*v_c*v_c),&#xA;                eig = (jpk+root)/2,&#xA;                s1 = sqrt(eig),&#xA;                s2 = fabs(root) &amp;lt; EPSILON ? s1 : sqrt((jpk-root)/2);&#xA;            e-&amp;gt;setData(s1, s2);&#xA;            //Use eigenvectors of A^T*A as V&#xA;            float v_s = eig-j,&#xA;                len = sqrt(v_s*v_s + v_c*v_c);&#xA;            v_c /= len;&#xA;            v_s /= len;&#xA;            v-&amp;gt;setData(v_c, -v_s, v_s, v_c);&#xA;            //Compute w matrix as Av/s&#xA;            w-&amp;gt;setData(&#xA;                (data[0][0]*v_c + data[1][0]*v_s)/s1,&#xA;                (data[1][0]*v_c - data[0][0]*v_s)/s2,&#xA;                (data[0][1]*v_c + data[1][1]*v_s)/s1,&#xA;                (data[1][1]*v_c - data[0][1]*v_s)/s2&#xA;            );&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8196" LastEditorUserId="8196" LastEditDate="2014-05-17T20:22:22.977" LastActivityDate="2014-05-17T20:22:22.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11647" PostTypeId="1" CreationDate="2014-05-17T10:48:53.190" Score="3" ViewCount="431" Body="&lt;p&gt;Is it possible to use test and trial functions from two different function spaces (defined over two different meshes) in a single weak form? Under what conditions can I do this (eg., each term in the form has functions belonging to only one of the spaces)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, I need this to model heat transfer in a block with a channel passing through it - the block is solid and the channel has water flowing through it. The solid domain and fluid domain are two different meshes. Basically I need to implement two equations like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_L \mathbf{u}.\nabla T_l \,\tilde{T_l} + \alpha_l\int_L \nabla T_l. \nabla \tilde{T_l} + \frac{\alpha_l}{k_l} \int_{\Gamma_w} h(T_l - T_s) \tilde{T_l} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\alpha_s\int_S \nabla T_s. \nabla \tilde{T_s} + \frac{\alpha_s}{k_s} \int_{\Gamma_w} h(T_s - T_l) \tilde{T_s} - \alpha_s \int_{\Gamma_1} Q \tilde{T_s} = 0&#xA;$$&#xA;where $T_l$ and $T_s$ are liquid and solid temperatures respectively, $\mathbf{u}$ is the liquid velocity (for which I'm solving a Stokes equation seperately), $L$ and $S$ are liquid and solid domains respectively, $\Gamma_w$ is the wall between the solid and liquid domains and $\Gamma_1$ is a surface of the solid domain where some heat $Q$ is supplied.&#xA;The water inlet temperature is fixed (a Dirichlet BC).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To have a steady state solution, I need to add these equations to get one. Can anyone tell me how I can go about implementing this in Fenics?&lt;/p&gt;&#xA;" OwnerUserId="5541" LastEditorUserId="5541" LastEditDate="2014-06-03T09:19:47.550" LastActivityDate="2016-05-11T13:47:00.760" Title="A Question About Weak Forms in Fenics" Tags="|pde|finite-element|fenics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11649" PostTypeId="1" CreationDate="2014-05-17T23:18:11.037" Score="3" ViewCount="636" Body="&lt;p&gt;After realizing that Gauss-Seidel is extremely slow for my simulation, i wanted to try GMRES and luckily found the C++ code &lt;a href=&quot;http://people.maths.ox.ac.uk/richardsonm/cpp.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; without diving into the theory. The size of the matrix in my case is &lt;code&gt;nxn&lt;/code&gt; where &lt;code&gt;n=50,000&lt;/code&gt;. However, I even cannot initialize the matrix to zero because the computer freezes while initializing it. Is there a memory friendly version of GMRES?&lt;/p&gt;&#xA;" OwnerUserId="8103" LastActivityDate="2014-05-19T19:02:01.050" Title="Not enough memory for GMRES" Tags="|c++|memory-management|gmres|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11650" PostTypeId="2" ParentId="11636" CreationDate="2014-05-18T01:10:54.163" Score="1" Body="&lt;p&gt;I know of two tools that might work; they are both Finite Element Method based:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;COMSOL Multiphysics; your question reminds me of the &lt;a href=&quot;http://www.comsol.com/blogs/reducing-magnetic-signature-submarine/&quot; rel=&quot;nofollow&quot;&gt;Submarine&#xA;example&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.csc.fi/english/pages/elmer/examples&quot; rel=&quot;nofollow&quot;&gt;ELMER FEM&lt;/a&gt;: Which has many of the features of&#xA;Comsol, but is Open Source and has an electrostatics and&#xA;magnetostatics solvers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8220" LastActivityDate="2014-05-18T01:10:54.163" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11651" PostTypeId="1" AcceptedAnswerId="11662" CreationDate="2014-05-18T02:32:22.760" Score="16" ViewCount="1011" Body="&lt;p&gt;In methods like gmres or bicgstab it could be attractive to use another Krylov method as a preconditioner. After all they are easy to implement in a matrix-free way and in a parallel environment. For example, one could use a few (let say ~5) iterations of unpreconditioned bigcstab as a precontioner for gmres, or any other combination of Krylov methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not find much reference to such approach in the literature, so I expect that this is because it is not very effective. I would like to understand why it is not efficient. Are there cases where it is a good choice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my research, I am interested about the solution of 3-D elliptic problems in a parallel (MPI) environment.&lt;/p&gt;&#xA;" OwnerUserId="8222" LastEditorUserId="20688" LastEditDate="2020-04-29T13:27:36.443" LastActivityDate="2020-04-29T13:27:36.443" Title="Preconditioning a Krylov method with another Krylov method" Tags="|linear-algebra|linear-solver|preconditioning|krylov-method|gmres|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11652" PostTypeId="1" CreationDate="2014-05-18T03:45:05.947" Score="0" ViewCount="306" Body="&lt;p&gt;How do we define the matrix for GMRES, if we do not want to solve the boundary elements but only the interior ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using pentagonal elements so in a row there are 6 elements (cell itself + 5 fluxes) which means that in a row boundary elements are also accounted. But I need to make the matrix square according to the code &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/cpp_src/mgmres/mgmres.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and I do not want to solve for boundary elements? What should I do?&lt;/p&gt;&#xA;" OwnerUserId="8103" LastActivityDate="2014-05-18T16:05:42.320" Title="GMRES: Making the matrix square without solving for boundaries" Tags="|sparse-matrix|c++|boundary-conditions|matrix|gmres|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11653" PostTypeId="2" ParentId="11652" CreationDate="2014-05-18T16:05:42.320" Score="5" Body="&lt;p&gt;It seems that you are referring to the fluxes that occur at the edge of your domain. Since these are typically known quantities enforced by your boundary conditions, there is no need to solve for those values. There are two common ways to deal with this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Include an equation for each (known) boundary quantity in the form $f_i = F(x_i)$ for $x_i\in\partial\Omega$. This allows for more flexible boundary conditions, but adds an extra equation at each boundary point.&lt;/li&gt;&#xA;&lt;li&gt;Since you know the boundary terms, you may move any quantities involving only these terms to the vector on the right hand side of your matrix equation. This requires a simple change to the right hand side of you equation for each element on the boundary, but cannot be done for some boundary conditions (i.e. Neumann conditions in a finite difference method).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;As a simple example, consider the following problem:&lt;br&gt;&#xA;$$u_{xx} = x^2~~{\rm for}~~x\in[0,1]\\&#xA;u(0)=1,~~u(1)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this problem is discretized with 2nd order centered finite differences and $\Delta x=0.25$, we get the equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;u_0 = &amp;amp; 1\\&#xA;(u_0 - 2u_{0.25} + u_{0.5} ) / 0.0625 = &amp;amp; 0.0625 \\&#xA;(u_{0.25} - 2u_{0.5} + u_{0.75} ) / 0.0625 = &amp;amp; 0.25 \\&#xA;(u_{0.5} - 2u_{0.75} + u_{1} ) / 0.0625 = &amp;amp; 0.5625 \\&#xA;u_1 = &amp;amp; 0\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can either be written in matrix form as 5 equations with 5 unknowns:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\begin{array}{ccccc}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1&#xA;\end{array}\right]&#xA;\left[\begin{array}{c}&#xA;u_0\\&#xA;u_{0.25}\\&#xA;u_{0.5}\\&#xA;u_{0.75}\\&#xA;u_{1}&#xA;\end{array}\right]&#xA;=&#xA;\left[\begin{array}{c}&#xA;1\\&#xA;0.00390625\\&#xA;0.015625\\&#xA;0.03515625\\&#xA;0&#xA;\end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, the known boundary conditions (equations 1 and 5 ) can be used in equations 2 and 4 and the known terms all moved to the right, giving:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\begin{array}{ccc}&#xA;-2 &amp;amp; 1 &amp;amp; 0\\&#xA;1 &amp;amp; -2 &amp;amp; 1\\&#xA;0 &amp;amp; 1 &amp;amp; -2&#xA;\end{array}\right]&#xA;\left[\begin{array}{c}&#xA;u_{0.25}\\&#xA;u_{0.5}\\&#xA;u_{0.75}&#xA;\end{array}\right]&#xA;=&#xA;\left[\begin{array}{c}&#xA;-0.99609375\\&#xA;0.015625\\&#xA;0.03515625&#xA;\end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;&lt;br&gt;&#xA;In the case above, the two systems are exactly equivalent and will give the same answer. However, if I had imposed Neumann conditions instead, the second form would not work since the value at the boundary points would not be known. Instead, the boundary condition needs to be discretized (e.g. by using one-sided differences at the boundary points) and those equations need to be included in the final matrix equation.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-05-18T16:05:42.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11654" PostTypeId="1" CreationDate="2014-05-18T17:41:02.563" Score="0" ViewCount="705" Body="&lt;p&gt;I have the following non-linear equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mO0Yt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $w0=0.25,w0=0.5,w0=0.75$. I have to prove that if $k$ is a root, then also $−k$ is a root and that there exists only one $k∈(0,1)$ root, but my MATLAB code doesn't return any solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using wolframalpha I have computed the derivative of w0 and obtained, the constants w0=0.25=0.5=0.75 were ignored. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nV9ub.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB I have the following function to compute the roots of non-linear equations: &#xA;&lt;img src=&quot;https://i.stack.imgur.com/fquUP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my test file looks like this: &#xA;&lt;img src=&quot;https://i.stack.imgur.com/vC6PM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea to make my code functional is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8225" LastActivityDate="2014-05-18T18:43:45.413" Title="MATLAB Newton non-linear equation" Tags="|nonlinear-equations|newton-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11655" PostTypeId="2" ParentId="11654" CreationDate="2014-05-18T18:25:16.477" Score="2" Body="&lt;p&gt;You've written &lt;code&gt;w1 = @(k) ((2*k)/(log((1+k)*(1-k))))-0.25;&lt;/code&gt; but your original function means this should be &lt;code&gt;w1 = @(k) ((2*k)/(log((1+k)/(1-k))))-0.25;&lt;/code&gt; and similar for the other anonymous functions. The difference is division within the &lt;code&gt;log&lt;/code&gt; rather than multiplication. Also note that there are singularities in this function at $k=\{-1, 0, 1\}$ and the function is complex valued outside of the range [0,1] so Newton's method will not be very reliable (especially since the roots are very close to the singularities at $\pm1$). You may be better off with a bracketing method such as bisection or false position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if your goal is to &quot;prove&quot; that if $k\in(0,1)$ is a root, then $-k$ is a root, finding the root numerically is not proof at all. You actually need to perform some (relatively simple) manipulations of the equation to show symmetry about $k=0$.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-05-18T18:43:45.413" LastActivityDate="2014-05-18T18:43:45.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11656" PostTypeId="2" ParentId="11651" CreationDate="2014-05-19T12:46:21.977" Score="9" Body="&lt;p&gt;Such a nested Krylov subspace method may work quite well in practice. It may be of interest for non symmetric linear systems for which restarted GMRES stagnates and unrestarted GMRES is too expensive or uses too much memory. Some literature:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;em&gt;GMRESR: A family of nested GMRES methods&lt;/em&gt;, van der Vorst, Vuik&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Flexible inner-outer Krylov subspace methods&lt;/em&gt;, Simoncini, Szyld&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;A flexible inner-outer preconditioned GMRES algorithm&lt;/em&gt;, Saad&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Further experiences with GMRESR&lt;/em&gt;, Vuik&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5286" LastActivityDate="2014-05-19T12:46:21.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11658" PostTypeId="1" AcceptedAnswerId="11666" CreationDate="2014-05-19T15:11:47.400" Score="4" ViewCount="235" Body="&lt;p&gt;I'm looking for a good way to partition a large, fairly high-dimensional dataset in order to perform fast kNN searches not just in the full $N$-dimensional space, but also in lower-dimensional subspaces of that space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider a point $p$ in a 3-dimensional space, $\{X, Y, Z\}$. On the one hand, I want to find its nearest neighbour $i$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\DeclareMathOperator*{\argmin}{argmin}&#xA;\argmin_{i} max{\{\| x_p - x_i \|, \|y_p - y_i\|, \|z_p - z_i\|  }\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I also want to be able to efficiently find the nearest neighbour $j$ in the $\{X, Y\}$ subspace, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\DeclareMathOperator*{\argmin}{argmin}&#xA;\argmin_{j} max{\{\| x_p - x_j \|, \|y_p - y_j\|}\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case there are usually many more than 3 dimensions, and I want to be able to find the $k^{th}$ nearest neighbour distances considering multiple different subsets of those dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you some more background, my goal is to use kNN distances to &lt;a href=&quot;http://link.aps.org/doi/10.1103/PhysRevE.69.066138&quot; rel=&quot;nofollow&quot;&gt;estimate conditional mutual information&lt;/a&gt; from continuous data without having to discretize it and compute histograms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be an appropriate way to partition my data?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Some more information:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My dimensionality will vary between about 3 and 30&lt;/li&gt;&#xA;&lt;li&gt;The number of datapoints will probably be on the order of about 200000&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7993" LastEditorUserId="7993" LastEditDate="2014-05-20T12:16:50.557" LastActivityDate="2017-05-26T18:32:32.080" Title="K-nearest neighbours search in subspaces of a high-dimensional space" Tags="|computational-geometry|high-dimensional|data-structures|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11659" PostTypeId="1" AcceptedAnswerId="11664" CreationDate="2014-05-19T18:29:58.423" Score="7" ViewCount="11335" Body="&lt;p&gt;I have a little bit of problem figuring out the relation between Fourier series and Least Squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, LS is a way of minimizing the quadratic error between a measured value $y_i$ and a linear combination of functions evaluated at points $x_i$. So, in order to solve for least squares we have to have the $(x_i,y_i)$ and the functions $f_j(\cdot)$ that are going to be linearly combined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I would like to approximate a set of points $(x_i,y_i)$ using Fourier basis functions, I could assemble a matrix as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{pmatrix}&#xA;  1 &amp;amp; \cos(\omega x_1) &amp;amp; \sin(\omega x_1) \\&#xA;  1 &amp;amp; \cos(\omega x_2) &amp;amp; \sin(\omega x_2) \\&#xA;  \vdots &amp;amp; \vdots &amp;amp; \vdots\\&#xA;  1 &amp;amp; \cos(\omega x_n) &amp;amp; \sin(\omega x_n) \\&#xA; \end{pmatrix} \begin{pmatrix}&#xA;  A_0 \\&#xA;  A_1 \\&#xA;  A_2 \\&#xA; \end{pmatrix} = \begin{pmatrix}&#xA;  y_1 \\&#xA;  y_2 \\&#xA;  \vdots \\&#xA;  y_n \\&#xA; \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain the values of the coefficients $A_0,A_1,A_2$ using least-squares I can solve for $x = (A^TA)^{-1}A^Tb$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my doubt arises since it seems to me that the coefficients $A_0,A_1,A_2$ computed with the least-squares formula can be found with the Fourier transform. Are the coefficients $A_0,A_1,A_2$ the same as the Fourier coefficients $C_k$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc; close all;&#xA;&#xA;N = 10;&#xA;x = 1:N;&#xA;%y = rand(1,N);&#xA;y = [0.7099 0.6745 0.7374 0.1144 0.7732 0.0021 0.0357 0.4804 0.8107 0.0410];&#xA;&#xA;% Polynomial Fit&#xA;A = [(x.*x)' x' ones(N,1)];&#xA;c = A\y'&#xA;&#xA;% Fourier Fit - K+1 Coefficients&#xA;A = [];&#xA;K = 8;&#xA;for k = 0:K&#xA;    A = [A exp(-1j*2*pi*k*x/N)'];&#xA;end&#xA;c = A\y'&#xA;c = (fft(y)/length(y))'&#xA;% Test the approximation on a bigger range&#xA;x_nodes = linspace(0,10,1000);&#xA;f = c(1);&#xA;for i = 2:K&#xA;    f = f + c(i)*exp(1j*2*pi*i*x_nodes/N);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I try to use the complex form of the Fourier Series $ f = C_0 + \sum_k C_ke^{-j2\pi kx/L}$ and follow the least-squares approach and compare the results to the coefficients of the FFT, the coefficients seem to be different. What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="10134" LastEditDate="2016-08-28T16:46:41.707" LastActivityDate="2017-10-23T16:54:05.787" Title="Least Squares and Fourier Series" Tags="|fourier-analysis|least-squares|fourier-transform|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11660" PostTypeId="2" ParentId="11659" CreationDate="2014-05-19T18:49:19.610" Score="1" Body="&lt;p&gt;They're related, but not the same, for a square system, in which the least squares fit becomes an interpolation. You have the relationship that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c_k = \pm (a_k+ib_k)/2$ if $k\neq 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $c_0 = a_0$ for $k=0$. You can do this by noting that the Fourier expansion over a period of $\tau$ gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(t) = a_0 + \sum_{k=1}^\infty a_k \cos(\frac{2\pi k}{\tau}t) + \sum_{k=1}^\infty b_k \sin(\frac{2\pi k}{\tau}t)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rewriting using Euler's formula gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(t) = \sum_{k=1}^\infty c_k e^{i(2\pi k/\tau)t}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the above relationship between $c$ and $a,b$ above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if the analogy holds for least squares.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2014-05-19T18:49:19.610" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11661" PostTypeId="2" ParentId="11649" CreationDate="2014-05-19T19:02:01.050" Score="3" Body="&lt;p&gt;From what I can see, on a normal machine, you would run out of memory if you are using a Dense matrix. The implementation you indicated in the question uses a dense matrix ( storing all the matrix entries ). Generally in most simulations, each element computes fluxes with its adjacent elements and the resulting matrix structure is sparse ( the number of non-zero entries in the matrix scales only linearly with the size ). So, one way to get around this problem is to use a sparse matrix implementation of the GMRES algorithm. There are several open-source implementations available. I would recommend PETSc - &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/&lt;/a&gt; PETSc provides implementations for the data-structures and algorithms you need and above that, you get parallelism for free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you prefer a stand-alone code, I can point you to this reference - &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/cpp_src/mgmres/mgmres.html&quot; rel=&quot;nofollow&quot;&gt;http://people.sc.fsu.edu/~jburkardt/cpp_src/mgmres/mgmres.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you (for some reason) prefer to stick to a dense matrix implementation, you should go for distributed memory (MPI) parallelism. Once again, PETSc provides you with algorithms for the same.&lt;/p&gt;&#xA;" OwnerUserId="4596" LastActivityDate="2014-05-19T19:02:01.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11662" PostTypeId="2" ParentId="11651" CreationDate="2014-05-19T19:21:41.777" Score="15" Body="&lt;p&gt;Interesting that this question came yesterday, since I just finished an implementation yesterday that does this. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;My Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Just to start of, let me know that while my education background is from scientific computing, all work I have done since graduating, including my current Ph.D. work, has been in computational electromagnetics. So, I guess our backgrounds are somewhat similar, since you also appear to be looking at physics (based on your profile).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;FGMRES&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;First of all, what you are looking for, as Guido Kanschat has already mentioned in a comment, is called Flexible GMRES or FGMRES. The reference, including pseudocode, is in [1]. While I sometimes find numerical SIAM papers a bit hard to read, [1] (and most of Saad's other work, including the brilliant [B1], apparently legally available for free online) is different; the paper is a fascinating read, very clearly written and with a few nice examples and suggestions for applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FGMRES is easy to implement, particularly if you already have a working RIGHT preconditioned GMRES. Note the keyword RIGHT here - if you have a LEFT preconditioned GMRES, i.e. you are used to solving MAx=Mb, then you have to make a few modifications. Compare [B1,Algorithm 9.4 on pg. 282] to [B1,Algorithm 9.5,pg. 284]. You can also find the FGMRES in [B1,Algorithm 9.6, pg. 287], but I would really encourage you to read [1] as it is short, well written and still has many interesting details.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What does it do&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;FGMRES basically allows you to switch preconditioners for every iteration, if you wish. One of the applications for this is that you can use some preconditioner that works very well when you are far away from the solution, and then switch to another one when you get closer. [2], which I have not read in detail, appears to discuss something similar to this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the most interesting application in my case was that you could use a (preconditioned) GMRES as a preconditioner for your FGMRES. This is the reason behind the typical name for FGMRES, &quot;inner-outer GMRES&quot;. Here, &quot;outer&quot; refers to the FGMRES solver, which (as preconditioner) uses an &quot;inner&quot; solver.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;So, how good is this in practice?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In my case, this worked absolutely brilliant. In the inner loop, I &quot;solve&quot; a reduced-complexity formulation of my problem. On its own, this solution is far too inaccurate for our use, but it works absolutely great as a preconditioner. Note the &quot;&quot; around &quot;solve&quot; - there is no need to run the inner solver to convergence, since you are only looking for rough approximations. In my case, I went from using 151 iterations, each costing 64 seconds, to 72 iterations, each costing 79 seconds (I used a fixed 5 iterations in the inner GMRES). That is a total saving of an hour, with no loss of accuracy and very little coding work since we already had a functioning GMRES which we just made recursive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some applications of this stuff, demonstrating the potential performance, see [3] (which actually uses a three-level FGMRES, so FGMRES, with FGMRES as inner, with GMRES as inner-inner) and [4], which might be too application specific for your use, but contains several interesting test cases.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;[1] Y. Saad, “A flexible inner-outer preconditioned GMRES algorithm,” SIAM J. Sci. Comp., vol. 14, no. 2, pp. 461–469, Mar. 1993. &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/PDF/umsi-91-279.pdf&quot;&gt;http://www-users.cs.umn.edu/~saad/PDF/umsi-91-279.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] D.-Z. Ding, R.-S. Chen, and Z. Fan, “SSOR preconditioned inner-outer flexible GMRES method for MLFMM analysis of scattering of open objects,” Progress In Electromagnetics Research, vol. 89, pp. 339–357, 2009. &lt;a href=&quot;http://www.jpier.org/PIER/pier89/22.08112601.pdf&quot;&gt;http://www.jpier.org/PIER/pier89/22.08112601.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] T. F. Eibert, “Some scattering results computed by surface-integral-equation and hybrid finite-element-boundary-integral techniques, accelerated by the multilevel fast multipole method,” IEEE Antennas and Propagation Magazine, vol. 49, no. 2, pp. 61–69, 2007.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] Ö. Ergül, T. Malas, and L. Gürel, “Solutions of Large-Scale Electromagnetics Problems using an Iterative Inner-Outer Scheme with Ordinary and Approximate Multilevel Fast Multipole Algorithms,” Progress In Electromagnetics Research, vol. 106, pp. 203–223, 2010. &lt;a href=&quot;http://www.jpier.org/PIER/pier106/13.10061711.pdf&quot;&gt;http://www.jpier.org/PIER/pier106/13.10061711.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[B1]    Y. Saad, Iterative Methods for Sparse Linear Systems. SIAM, 2003. &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot;&gt;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="370" LastActivityDate="2014-05-19T19:21:41.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11663" PostTypeId="2" ParentId="11639" CreationDate="2014-05-19T19:24:53.860" Score="0" Body="&lt;p&gt;I have encountered a very similar situation, although in a different field. When dealing with sparse matrices, there are few cases where the elements are present only on the diagonals / in bands around the diagonals. They can be stored very efficiently by just storing the diagonal information and very efficient algorithms can be designed. But, there are cases where the sparse-matrix may not be in the diagonal form. But, we can apply some transformations to the Adjacency matrix (matrix has entry 1 if the position has a non-zero entry, else 0). This is a square, symmetric, non-zero, possibly dense matrix, which we want to transform into a block diagonal form, which is exactly what you want to do. The standard algorithm I know of to do the job is called the Cuthill Mckee algorithm. This algorithm intuitively follows the breadth-first search algorithm. The choice of the first element is crucial and the result of the algorithm is heuristic based on the ordering of the neighbors. In typical cases, the Reversed Cuthill Mckee algorithm gives slightly better results. I hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="4596" LastActivityDate="2014-05-19T19:24:53.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11664" PostTypeId="2" ParentId="11659" CreationDate="2014-05-19T20:22:12.343" Score="8" Body="&lt;p&gt;In a word, yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A least squares problem arises in the context of an over determined system. That means that you have more equations than unknowns and would be the case if you had more data points than Fourier coefficients that you're trying to find. This would give you the &quot;best fit&quot; Fourier series of a given order. For example, you could find the best fit of a 4 term Fourier series to a set of 20 data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the discrete Fourier transform of a set of points always gives the same number of Fourier coefficients as input points. Thus, if you start with 20 points you will get 20 Fourier coefficients. Additionally, &lt;strong&gt;the first N Fourier coefficients are exactly the same as a least squares fit of a Fourier series with only N terms&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key here is that the Fourier basis is an orthogonal basis on a given interval. The math works out so that the least squares best fit based of a lower order Fourier series is exactly equivalent to the truncated FFT. See &lt;a href=&quot;http://elibrary.bharathuniv.ac.in/NPTEL/108101039/Power%20System%20Protection/digital_protection/DPLectures/lec32.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this PDF&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To verify, you can easily try this yourself. Pick a set of data points and compute the DFT and the least squares best fit and compare them. Here's an example in MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    clear&#xA;&#xA;%interval length and number of points&#xA;L = 10;&#xA;N = 15;&#xA;&#xA;%data points&#xA;x = (linspace(0,L,N+1))'; x(end)=[];&#xA;y = x.^2; %function values&#xA;&#xA;% FFT:&#xA;yk = fft(y)/length(x); %divide by length(x) for normalization&#xA;&#xA;% pick out the first half of the coefficients&#xA;% (coefficients are symmetric for real valued functions)&#xA;yk = yk(1:ceil((length(x)+1)/2));&#xA;&#xA;% Transform coefficients to the form:&#xA;%       y = A0 + sum( Ak*cos(...) + Bk*sin(...) )&#xA;A0 = yk(1); %constant term&#xA;A = 2*real(yk(2:end)); %cosine coefficients&#xA;B = -2*imag(yk(2:end)); %sine coefficients&#xA;&#xA;%account for difference in highest wave number for odd/even numbers of points&#xA;if mod(numel(x),2)==0&#xA;    A(end) = A(end)/2;&#xA;end&#xA;&#xA;%reconstruct the original function for verification&#xA;yy=A0;&#xA;for k=1:length(A)&#xA;    yy = yy + A(k)*cos(2*pi/L*k*x) + B(k)*sin(2*pi/L*k*x);&#xA;end&#xA;&#xA;&#xA;&#xA;%least squares fit, unknowns C = [A0 A(1) B(1) A(2) B(2) ... ]'&#xA;&#xA;%number of sin() and cos() coefficients to use:&#xA;N = 4;&#xA;&#xA;%set up matrix:&#xA;M = zeros(length(x),1+2*N);&#xA;M(:,1) = 1;&#xA;for k=1:N&#xA;    M(:,2*k) = cos(2*pi/L*k*x);&#xA;    M(:,2*k+1) = sin(2*pi/L*k*x);&#xA;end&#xA;&#xA;C = M\y; %equivalent to (M'*M)\(M'*y)&#xA;&#xA;% least squares coefficients:&#xA;A0L2 = C(1);&#xA;AL2 = C(2:2:end);&#xA;BL2 = C(3:2:end);&#xA;&#xA;%reconstruct the function values&#xA;yyL2=A0L2;&#xA;for k=1:length(AL2)&#xA;    yyL2 = yyL2 + AL2(k)*cos(2*pi/L*k*x) + BL2(k)*sin(2*pi/L*k*x);&#xA;end&#xA;&#xA;% Plot:&#xA;&#xA;figure(1)&#xA;clf&#xA;hold on&#xA;plot(x,y,'ko')&#xA;plot(x,yy,'b-')&#xA;plot(x,yyL2,'r--')&#xA;legend('data','Fourier Series','Least Squares','location','northwest')&#xA;title('Comparison of function values')&#xA;xlabel('x')&#xA;ylabel('y')&#xA;&#xA;figure(2)&#xA;clf&#xA;hold on&#xA;plot(A,'bo')&#xA;plot(B,'bs')&#xA;plot(AL2,'r.')&#xA;plot(BL2,'rx')&#xA;plot(0,A0,'bo')&#xA;plot(0,A0L2,'r.')&#xA;legend('Fourier Series, A_k','Fourier Series, B_k','Least Squares, A_k','Least Squares, B_k','location','north')&#xA;title('Comparison of coefficient''s values')&#xA;xlabel('wave number, k')&#xA;ylabel('coefficients')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code produces the following plots. The second plot in particular shows that the coefficients of a truncated Fourier series are the same as the least squares best fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mtIUA.png&quot; alt=&quot;comparison of FFT and L2 fit function values&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/uE3qs.png&quot; alt=&quot;comparison of Fourier coefficients&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2017-10-23T16:54:05.787" LastActivityDate="2017-10-23T16:54:05.787" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11665" PostTypeId="1" CreationDate="2014-05-20T09:50:46.917" Score="2" ViewCount="413" Body="&lt;p&gt;Say I have a vector field with non-zero curl, therefore the potential function depends on the path I choose to integrate.&#xA;In &lt;a href=&quot;https://www.cs.drexel.edu/~kon/introcompvis/reading/frankot-chellappa.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; the authors proposed to &lt;strong&gt;project&lt;/strong&gt; the vector field into a gradient field of Fourier basis functions in order to make the potential function independent of the path of integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the idea but cannot understand the math of what that projection means. What I think the authors are doing is to approximate the given data using Fourier basis functions (that's why I was &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11659/least-squares-and-fourier-series/11664#11664&quot;&gt;asking about Fourier series&lt;/a&gt; in another thread) in order to have a smooth function that best fits the given points and complies with the zero-curl of a gradient field. What I don't see is how many basis functions they use in order to approximate the data points, it could be $y_i = C_0 + C_1e^{jwx_i}$ or $y_i = C_0 + C_1e^{jwx_i} + C_2e^{2jwx_i}$ or any other. Moreover, it seems to me that they compute the FFT of the vector field and use all the coefficients of it, so, the function will fit all the points but still will be non-integrable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure I am missing something and mixing maybe some concepts, can someone help me unwrapping it all to understand what the idea of this projection is?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-05-20T11:15:06.030" Title="Projection of vector field on to a gradient field" Tags="|fourier-analysis|projection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11666" PostTypeId="2" ParentId="11658" CreationDate="2014-05-20T10:57:36.310" Score="1" Body="&lt;p&gt;Since your dimension is only 3 for the moment, I guess a naive approach would be to create separate trees for each subspace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend you to use Flann (&lt;a href=&quot;http://www.cs.ubc.ca/research/flann/&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.ubc.ca/research/flann/&lt;/a&gt;) as it is very well implemented and reported to be quite fast for the datasets of your concern (&lt;a href=&quot;http://www.cs.ubc.ca/research/flann/uploads/FLANN/flann_visapp09.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.ubc.ca/research/flann/uploads/FLANN/flann_visapp09.pdf&lt;/a&gt; , &lt;a href=&quot;http://www.cs.ubc.ca/research/flann/uploads/FLANN/binary_matching_crv2012.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.ubc.ca/research/flann/uploads/FLANN/binary_matching_crv2012.pdf&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Flann is implemented in a parallel fashion (it even supports MPI), but you could tweak it to retrieve parallel queries for each subspace, which will nevertheless reduce the computation time significantly. So there are already options for you to try out.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-05-20T10:57:36.310" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11667" PostTypeId="1" AcceptedAnswerId="11668" CreationDate="2014-05-20T12:03:18.267" Score="0" ViewCount="95" Body="&lt;p&gt;Could you tell me please how to obtain separate values of &quot;pi&quot; depending on the value of N (code below)? &#xA;For example If I write &quot;pi1&quot; I will get a value of &quot;pi&quot; for N(1) where N(1)=100. If I write &quot;pi2&quot; I will get &quot;pi&quot; for N(2)=1000 etc. I need it to compare the calculation time and the accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My CODE is:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R=1; &#xA;N=[100,1000,10000,100000]; &#xA;&#xA;for k=1:1:4 &#xA;  toc&#xA;   x = 2*rand(N(k), 1)-1; &#xA;   y = 2*rand(N(k), 1)-1; &#xA;   in_circle = find((x.^2+y.^2)&amp;lt;R.^2); &#xA;   N2 = size(in_circle, 1); &#xA;&#xA;   pi=4*N2/(N(k)*R.^2) &#xA;  t=toc&#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Actual OUTPUT is:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pi = &#xA;3.120000000000000 &#xA;t = &#xA;1.088773407512715e-004 &#xA;&#xA;pi = &#xA;3.208000000000000 &#xA;t = &#xA;5.176135871781760e-005 &#xA;&#xA;pi = &#xA;3.145600000000000 &#xA;t = &#xA;6.113194952018113e-005 &#xA;&#xA;pi = &#xA;3.143360000000000 &#xA;t = &#xA;1.124470896283624e-004 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OUTPUT I would like to get:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pi1 = &#xA;3.120000000000000 &#xA;t1 = &#xA;1.088773407512715e-004 &#xA;&#xA;pi2 = &#xA;3.208000000000000 &#xA;t2 = &#xA;5.176135871781760e-005 &#xA;&#xA;pi3 = &#xA;3.145600000000000 &#xA;t3 = &#xA;6.113194952018113e-005 &#xA;&#xA;pi4 = &#xA;3.143360000000000 &#xA;t4 = &#xA;1.124470896283624e-004 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7916" LastActivityDate="2014-05-20T12:33:37.693" Title="How to get values from the loop in MATLAB?" Tags="|matlab|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11668" PostTypeId="2" ParentId="11667" CreationDate="2014-05-20T12:17:32.150" Score="4" Body="&lt;p&gt;If you really want the variables to be called &lt;code&gt;pi1, pi2, ...&lt;/code&gt;, you can build the variable names using &lt;code&gt;eval&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k=1:1:4&#xA;  ...&#xA;  eval(['pi' num2str(k) ' = 4*N2/(N(k)*R.^2)'])&#xA;  ...&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But a more typical thing to do would be to use an array:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k=1:1:4&#xA;  ...&#xA;  pi(k) = 4*N2/(N(k)*R.^2)&#xA;  ...&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you're not planning on using the names of the &lt;code&gt;pi&lt;/code&gt; variables later in your code, you might just use &lt;code&gt;disp&lt;/code&gt; to write the statement that you actually want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k=1:1:4&#xA;  ...&#xA;  pi = 4*N2/(N(k)*R.^2)&#xA;  disp([num2str(k) ': ' num2str(4*N2/(N(k)*R.^2))])&#xA;  ...&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Etc.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2014-05-20T12:33:37.693" LastActivityDate="2014-05-20T12:33:37.693" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11669" PostTypeId="1" CreationDate="2014-05-20T20:11:20.870" Score="1" ViewCount="269" Body="&lt;p&gt;This may very well be a silly question but I'm trying to cement my understanding of the material in the SICP (&lt;a href=&quot;http://goo.gl/QXrbtV&quot; rel=&quot;nofollow&quot;&gt;http://goo.gl/QXrbtV&lt;/a&gt;). My intuition (common sense) says yes, but wondering from a theoretical point of view if this always holds true.&lt;/p&gt;&#xA;" OwnerUserId="7276" LastActivityDate="2014-05-20T23:18:43.477" Title="Is there always a linear iterative alternative to a recursive iterative process?" Tags="|algorithms|" AnswerCount="1" CommentCount="2" ClosedDate="2014-05-21T02:32:11.007" ContentLicense="CC BY-SA 3.0" />
  <row Id="11670" PostTypeId="2" ParentId="11669" CreationDate="2014-05-20T23:18:43.477" Score="0" Body="&lt;p&gt;I've got a simple and elegant answer for you. @k20 hinted at it in his comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, there always is. You can simply emulate the call stack that would be created if you ran a recursive process. &lt;/p&gt;&#xA;" OwnerUserId="8242" LastActivityDate="2014-05-20T23:18:43.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11672" PostTypeId="1" AcceptedAnswerId="11674" CreationDate="2014-05-21T08:18:41.130" Score="2" ViewCount="486" Body="&lt;p&gt;I am trying to implement a simple FEM approach, using p1 triangular elements, for solving the diffusion equation with variable nodal diffusivities and I was wondering how to incorporate the variable nodal diffusivities when computing the element stiffness matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would really appreciate if someone could point me in the right direction. Many thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="8245" LastActivityDate="2014-07-10T03:20:50.317" Title="Computing element stiffness matrices with variable coefficients" Tags="|finite-element|matrix|diffusion|stiffness|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11673" PostTypeId="1" CreationDate="2014-05-21T08:55:32.110" Score="-1" ViewCount="1399" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\displaystyle \frac{\partial u}{\partial t}=\alpha(x,t)\cdot \frac{\partial^2 u}{\partial x^2}+b(x,t)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u(x,0)=f(x)$&lt;/span&gt; Initial condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u_x(0,t)=0$&lt;/span&gt; 2nd type Boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u_x(1,t)=0$&lt;/span&gt; 2nd type Boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is my code in Maple for the 1st type B.C. And I really do not know, what I should change in this  code to have a solution for my problem with 2nd type B.C.&#xA;What should I change in the loop &lt;strong&gt;&lt;em&gt;#Boundary cond&lt;/em&gt;&lt;/strong&gt; ?&#xA;Where do I have to use central finite differences for boundary conditions? &#xA;What will be changed in the explicit scheme (for 2nd type B.C.)?&#xA;My code for explicit method for 1D heat equation with 1st type B.C. is below:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/euiKL.jpg&quot; alt=&quot;code&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be very grateful, if someone helps me.&lt;/p&gt;&#xA;" OwnerUserId="8247" LastEditorUserId="22284" LastEditDate="2019-01-12T09:54:31.243" LastActivityDate="2019-01-12T09:54:31.243" Title="Explicit scheme for heat equation with Neumann boundary conditions in Maple" Tags="|finite-difference|maple|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11674" PostTypeId="2" ParentId="11672" CreationDate="2014-05-21T08:55:37.630" Score="2" Body="&lt;p&gt;I assume you're trying to solve an equation that looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;-\nabla \cdot (a(x)\nabla{u}) = f,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $x$ in some domain $\Omega$, although the same approach would be fine (for a residual evaluation, anyway) if $a$ were also a function of $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stiffness matrix will take the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A_{ij} = \int_{\Omega}a(x)\nabla\varphi_{j}(x)\cdot\nabla\varphi_{i}(x)\,\mathrm{d}x.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $P_{1}$ triangular elements, $\nabla\varphi_{j}$ will be constant over each element $K$, for each $j$. Letting $\mathcal{K}$ be the mesh, the elements of the stiffness matrix become&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A_{ij} = \sum_{K \in \mathcal{K}}\nabla\varphi_{j}^{K}\cdot\nabla\varphi_{i}^{K}\int_{K}a(x)\,\mathrm{d}x,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where all I've done is factor out the constant gradients over each element (these gradients will differ from element to element, which is why they are also indexed over $K$). If $K$ has vertices $N_{1}$, $N_{2}$, and $N_{3}$, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\int_{K}a(x)\,\mathrm{d}x = a\left(\frac{N_{1} + N_{2} + N_{3}}{3}\right)|K| + O(h^{3}),&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h$ is the mesh &quot;size&quot;, and $|K|$ is the area of element $K$. This approximation is also called the center of gravity rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, you'd take the local stiffness matrix for each element (assuming a diffusivity of one), and multiply it by the variable diffusivity $a$ evaluated at the center of gravity of the element.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-21T08:55:37.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11675" PostTypeId="1" CreationDate="2014-05-21T09:07:52.393" Score="1" ViewCount="865" Body="&lt;p&gt;Is it possible to call an external linear solver from Abaqus and/or Ansys?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solver (which is supplied by me) would get the sparse matrix &lt;strong&gt;A&lt;/strong&gt; and the right hand side vector &lt;strong&gt;b&lt;/strong&gt; as inputs, and would return the solution &lt;strong&gt;x&lt;/strong&gt; (with &lt;strong&gt;A&lt;/strong&gt; &lt;strong&gt;x&lt;/strong&gt; = &lt;strong&gt;b&lt;/strong&gt;) to the program. Ideally, Abaqus/Ansys would also inform the solver about the desired algorithm (CG, BiCGstab, GMREs, ...) and preconditioner.&lt;/p&gt;&#xA;" OwnerUserId="7289" LastActivityDate="2014-07-16T14:26:14.013" Title="External solver in Abaqus/Ansys" Tags="|linear-solver|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11676" PostTypeId="1" AcceptedAnswerId="11681" CreationDate="2014-05-21T11:43:30.650" Score="5" ViewCount="1357" Body="&lt;p&gt;I have two coupled nonlinear partial differential equations of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\begin{align}&#xA;\dot{u} -f(u,u',u'',v,v',v'')=0 \\&#xA;\dot{v} -g(u,u',u'',v,v',v'')=0&#xA;\end{align}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions are such that I know $u$ at time $t=0$ and $v$ at time $t=-\infty$. I also know the values of both functions at the two boundaries of coordinate interval. I need the solution for $x \in [0,1]$ and $t \in (-\infty,0]$ For a specific parameter value I also know an analytical solution and this allows me to tune this parameter little by little and then solve the equations by using as an initial guess for solution my solution for previous parameter value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm solving this by writing&#xA;$&#xA;S \equiv (\dot{u} -f(u,u',u'',v,v',v''))^2 + (\dot{v} -g(u,u',u'',v,v',v''))^2&#xA;$&#xA;and then finding the minimum of S and verifying that it is approximately zero. Currently I'm doing the minimization by discretizing $u$ and $v$ on my time and coordinate grid and then using conjugate gradient algorithm from SciPy optimization library. This gives me good results, but is terribly slow for larger grids. For a grid with $10^4$ grid points my solver is already not feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Which minimization algorithm is best for this sort of task? Conjugation gradient seems to require a lot of steps to converge to the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) What external libraries there are for solving this kind of problem? I'd prefer ones with a Python API. I looked at some finite element libraries and they seem like a bit of an overkill for this problem.&lt;/p&gt;&#xA;" OwnerUserId="7899" LastActivityDate="2014-05-21T19:21:44.500" Title="Solving a system of nonlinear PDEs by minimization" Tags="|optimization|pde|conjugate-gradient|scipy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11677" PostTypeId="2" ParentId="11675" CreationDate="2014-05-21T13:54:39.317" Score="2" Body="&lt;p&gt;As far as I know, there is no way to replace the ABAQUS solver with a user-written one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can use the&#xA;*MATRIX OUTPUT&#xA;option to write the stiffness (or mass or damping) matrices or load vectors to a text file.&#xA;Then you can read these text files and call your solver.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-05-21T13:54:39.317" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11678" PostTypeId="2" ParentId="11665" CreationDate="2014-05-21T14:37:15.110" Score="2" Body="&lt;p&gt;In the matrix case, the least square solution of an overdetermined inconsistent system $Ax=b$ requires the solution of $(A^TA)x=A^Tb$. The projection $A(A^TA)^{-1}Ab$ reduces $b$ to the closest vector that is a consistent right side.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As in the matrix case, the solution of the overdetermined inconsistent system $∇z=V$ is reduced to a solvable problem by applying the adjoint operator, here $$∇⋅ V=∇⋅ ∇z=Δz.$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you represent the vector field $V$ by &#xA;$$V(x)=\sum_{i\in I}C(ω_i)e^{jω_ix}$$ &#xA;then the divergence of this vector field is &#xA;$$∇⋅ V(x)=\sum_{i\in I}j(ω_i⋅ C(ω_i))e^{jω_ix}$$&#xA;and if a solution is tried that has the form &#xA;$$z(x)=\sum_{i\in I}A(ω_i)e^{jω_ix},$$ &#xA;one finds for the Laplacian&#xA;$$Δz(x)=-|ω_i|^2\sum_{i\in I}A(ω_i)e^{jω_ix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting this together by comparing coefficients of the same basis functions, the solution has the form&#xA;$$&#xA;A(ω_i)=-j\frac{ω_i⋅ C(ω_i)}{ω_i⋅ω_i}&#xA;$$&#xA;which is formula 21 of the paper, but without (explicitly) using the orthogonality.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that $x$, $ω_i$ and $C(ω_i)$ are two- or higher dimensional.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You can prune the coefficients of the Fourier series by removing all that fall below some threshold. Orthogonality tells you that you can control the error of that procedure by sorting the magnitudes of the coefficients or coefficient vectors, start the pruning with the smallest and add the squares of the coefficients.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2014-05-21T14:37:15.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11679" PostTypeId="2" ParentId="11675" CreationDate="2014-05-21T15:27:55.607" Score="0" Body="&lt;p&gt;Abaqus and ANSYS are &amp;quot;solvers&amp;quot; actually. Since they are closed source, and no modification is possible, so it would be better to shift your focus to open source programs e.g. &lt;strong&gt;Calculix&lt;/strong&gt; for this purpose where you can modify the code to use your own solver, with very small effort.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Calculix&lt;/strong&gt; uses ABAQUS's  input file format, so you can use your input files (originally written for Abaqus), with Calculix.&lt;/p&gt;&#xA;&lt;p&gt;Currently Calculix uses SPOOLES for solving sparse linear equations systems. You can easily replace that by your own solver.&lt;/p&gt;&#xA;" OwnerUserId="7449" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-05-21T15:27:55.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11680" PostTypeId="1" AcceptedAnswerId="11690" CreationDate="2014-05-21T18:48:03.833" Score="1" ViewCount="761" Body="&lt;p&gt;I have a pdb file and I would like to make a pqr one. In GROMACS, I use &lt;code&gt;editconf -mead&lt;/code&gt; to produce the pqr file, with topol.top taken from pdb2gmx and use it to produce pdb file, then use pdb2pqr. Which one is the better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First 10 lines of PQR made from GROMACS&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATOM  1 N    PRO A   1     0.63000  39.63000  17.25000 -0.2020  1.6250      &#xA;ATOM  2 H1   PRO A   1     1.37000  39.93000  17.89000  0.3120  0.5345      &#xA;ATOM  3 H2   PRO A   1     0.24000  40.48000  16.88000  0.3120  0.5345      &#xA;ATOM  4 CD   PRO A   1     1.21000  38.79000  16.18000 -0.0120  1.6998      &#xA;ATOM  5 HD1  PRO A   1     2.30000  38.76000  16.29000  0.1000  0.9800      &#xA;ATOM  6 HD2  PRO A   1     0.96000  39.21000  15.20000  0.1000  0.9800      &#xA;ATOM  7 CG   PRO A   1     0.63000  37.39000  16.30000 -0.1210  1.6998      &#xA;ATOM  8 HG1  PRO A   1     1.28000  36.80000  16.95000  0.1000  1.3248      &#xA;ATOM  9 HG2  PRO A   1     0.52000  36.90000  15.33000  0.1000  1.3248      &#xA;ATOM  10 CB  PRO A   1    -0.72000  37.64000  16.99000 -0.1150  1.6998      &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First 10 lines of PQR made from pqd2pqr:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   PRO     1       0.630  39.630  17.250 -0.2020 1.8240&#xA;ATOM      2  H   PRO     1       1.370  39.930  17.890  0.3120 0.6000&#xA;ATOM      3  H2  PRO     1       0.240  40.480  16.880  0.3120 0.6000&#xA;ATOM      4  CD  PRO     1       1.210  38.790  16.180 -0.0120 1.9080&#xA;ATOM      5  HD3 PRO     1       2.300  38.760  16.290  0.1000 1.1000&#xA;ATOM      6  HD2 PRO     1       0.960  39.210  15.200  0.1000 1.1000&#xA;ATOM      7  CG  PRO     1       0.630  37.390  16.300 -0.1210 1.9080&#xA;ATOM      8  HG3 PRO     1       1.280  36.800  16.950  0.1000 1.4870&#xA;ATOM      9  HG2 PRO     1       0.520  36.900  15.330  0.1000 1.4870&#xA;ATOM     10  CB  PRO     1      -0.720  37.640  16.990 -0.1150 1.9080&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are from 3B7V, a HIV-1 protease. It has been minimized energy.&lt;/p&gt;&#xA;" OwnerUserId="8250" LastEditorUserId="8250" LastEditDate="2016-10-14T11:35:59.943" LastActivityDate="2016-10-14T11:35:59.943" Title="PQR files make from pdb2pqr are different to GROMACS" Tags="|molecular-dynamics|computational-biology|electromagnetism|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11681" PostTypeId="2" ParentId="11676" CreationDate="2014-05-21T19:21:44.500" Score="6" Body="&lt;p&gt;1) If you're just looking to solve the PDEs without any other optimization, then my answer would be &quot;none of them&quot;. Algorithms that discretize partial differential equations and then solve them as algebraic equations are massively parallelizable. It is possible to solve a PDE over a billion point mesh. Algorithms for nonlinear programming have made great advances recently. It is now possible to use preconditioned iterative algorithms somewhat effectively to solve large-scale nonlinear programs with interior point methods, but I don't know of any nonlinear program that has been solved with a billion degrees of freedom, once we exclude special structure (typically, stochastic programs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an approach to PDE-constrained optimization called simultaneous analysis and design (SAND, also called &quot;all-at-once&quot;) that treats the PDEs as constraints to be solved within an optimization formulation. Solution of large-scale problem instances typically hinges on formulating a good preconditioner for the KKT system. This approach is different from your least-squares approach, and without having an optimization problem using your PDEs as constraints, I do not recommend this approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If you're dead set on doing this with optimization, I'd try pyipopt, which is a Python interface to the optimization library IPOPT, which is currently the best open-source nonlinear programming solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really, though, I'd recommend using traditional discretization approaches for solving PDEs, for which there are many good Python libraries available (depending on what you want to do, any of PyClaw, FEniCS, FiPy, sfepy, firedrake, petsc4py, PyTrilinos, and so on...). &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-21T19:21:44.500" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11684" PostTypeId="1" CreationDate="2014-05-21T21:44:44.213" Score="4" ViewCount="403" Body="&lt;p&gt;A small nonlinear equation system (sizes around 12 ✕ 12) needs to be solved repeatedly (millions of times); each time with some variation in parameters/coefficients (although the equation set is always the same).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most solvers are not very optimized for small systems like this, only achieving higher &quot;GFlops&quot; when using larger matrices (e.g., 512 ✕ 512).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently it is being used an implementation based on MINPACK (Powell's hybrid method with Broyden's Jacobian update), without using BLAS (because even BLAS implementations seem slower for small matrices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a solver implementation (such as the ones found in Intel MKL, PETSc or Ceres) that is optimized for this kind of problem?&lt;/p&gt;&#xA;" OwnerUserId="586" LastEditorUserId="20688" LastEditDate="2021-12-28T03:27:28.100" LastActivityDate="2021-12-28T03:27:28.100" Title="Fast way to repeatedly solve a small nonlinear equation system" Tags="|optimization|nonlinear-equations|performance|vectorization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11685" PostTypeId="2" ParentId="11684" CreationDate="2014-05-21T21:54:24.987" Score="2" Body="&lt;p&gt;Probably not, but you should parallelize over the parameters to the greatest extent possible. Use all the cores on your node and as many nodes as you can get your hands on. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-05-21T21:54:24.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11688" PostTypeId="1" AcceptedAnswerId="11689" CreationDate="2014-05-22T09:46:22.133" Score="0" ViewCount="756" Body="&lt;p&gt;I am curently using R package nleqslv for solving a non-linear system of equations with 300 variables. I need to scale this to the system with ~50k variables and naturally this does not scale very well, since the jacobian then takes up to ~30G of RAM, and I do not have access to a machine with such amount of RAM. The jacobian is a sparse matrix, but nleqslv does not have the ability to exploit this feature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are available solvers (preferably open source) which can exploit sparsity of the jacobian and can work with large systems?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've searched around and found lots of software for optimization problems, which does not exactly suit me, since I am reluctant to minimize the sum of squares of the system. I can always hack nleqslv code to use sparse matrices, but first I would like to now what are available ready-made solutions.&lt;/p&gt;&#xA;" OwnerUserId="128" LastActivityDate="2014-05-22T09:57:42.567" Title="Solver for large non-linear system of equations" Tags="|sparse-matrix|libraries|nonlinear-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11689" PostTypeId="2" ParentId="11688" CreationDate="2014-05-22T09:57:42.567" Score="2" Body="&lt;p&gt;PETSc is a solver package that has interfaces to a number of different methods for solving sparse linear systems, and many different nonlinear equation solvers (that make use of the linear solvers as subroutines). Although the framework is involved, the flexibility it gives you is worth it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking advantage of sparsity and possibly parallelism (if you can decompose the vectors and matrices in your problem appropriately) should help mitigate the memory bottleneck you mentioned because a sparse representation should require less memory, and parallelism should enable you to use memory across multiple machines, with should increase the amount of memory you can use for your simulation runs.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-22T09:57:42.567" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11690" PostTypeId="2" ParentId="11680" CreationDate="2014-05-22T14:57:56.070" Score="3" Body="&lt;p&gt;The Van der Walls radius - last column in the output -  is calculated from the force field. Probably pdb2pqr and editconf uses different force fields, hence different radius. I don't use pdb2pqr, but it seems (1) AMBER99 is the default force field, though CHARMM, PARSE and TYL06 are supported. Gromacs' editconf  reads force field from topology file generated by pdb2gmx (2). In my system pdb2gmx defaults to AMBER03. Try&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pdb2gmx -ff select&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which one is better depends on your problem. Are you generating input to APBS? If so, I would use pdb2pqr.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ref: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) &lt;a href=&quot;http://www.poissonboltzmann.org/pdb2pqr/user-guide/using-pdb2pqr&quot; rel=&quot;nofollow&quot;&gt;http://www.poissonboltzmann.org/pdb2pqr/user-guide/using-pdb2pqr&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) &lt;a href=&quot;http://manual.gromacs.org/online/pdb2gmx.html&quot; rel=&quot;nofollow&quot;&gt;http://manual.gromacs.org/online/pdb2gmx.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8260" LastActivityDate="2014-05-22T14:57:56.070" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11691" PostTypeId="1" CreationDate="2014-05-22T15:24:22.457" Score="2" ViewCount="257" Body="&lt;p&gt;I want to add hydrogens to some linear polymer molecules (polyethylenes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know some working methods like using PyMOL internal function h_add. This method works, but hydrogens are added at distances from the backbone carbon atoms that vary from 1.07 to 1.1 Ångstroms. I need to control the hydrogen distances from the backbone atoms and also the HCH and CCH angles to constant values that I choose every time (for example, H–C = 1.5 Å, ∠HCH = 108.5°, ∠CCH = 109.5°).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a geometry algorithm which I can use to understand the process of hydrogen adding? I know that it can be implemented nicely to a Fortran code but is there an easier way to do this, e.g. a script in Python, or better yet, a bash script?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The molecule coordinate files I am using are in the PDB format.&lt;/p&gt;&#xA;" OwnerUserId="8263" LastActivityDate="2014-05-22T15:24:22.457" Title="Algorithms for adding hydrogens on a molecule" Tags="|algorithms|simulation|molecular-dynamics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11692" PostTypeId="2" ParentId="4925" CreationDate="2014-05-22T15:34:03.980" Score="3" Body="&lt;p&gt;Input parameter handling in Matlab is just really annoying. This works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function out = foobar(foo, bar, baz)&#xA;    if ~exist('baz', 'var') ||  isempty(baz)&#xA;        baz = 'default_value';&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This needs to be repeated for all optional input parameters. Pro: it also does not depend on the order of the parameters in the function definition. Another option are all the &lt;code&gt;nargin&lt;/code&gt; hacks:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if nargin &amp;lt; 3 || isempty(baz)&#xA;    baz = 'baz';&#xA;    if nargin &amp;lt; 2 || isempty(bar)&#xA;        bar = 'bar';&#xA;        if nargin &amp;lt; 1 || isempty(foo)&#xA;             foo = 'foo';&#xA;        end&#xA;    end&#xA; end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obviously this is ugly and not very extensible; plus it is dependent on the definition order of the parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last option might be inputParser, but I have no idea how you can easily do the 'isempty' part with it. I can't find this type of input handling in the example posted by @DavidKetcheson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: @FedericoPoloni mentions another alternative: option structures. I think they are okay if you have many input parameters (and if you throw an error if an unknown field name is parsed to circumvent the confusion FredericoPoloni mentions). I would use something like this&#xA;:&#xA;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function test(foobar, opts)&#xA;&#xA;default_opts.foo = 'foo';&#xA;default_opts.bar = 'bar';&#xA;&#xA;if nargin &amp;gt; 1&#xA;    opts = parse_opts(default_opts, opts);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And as a separate function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function parsed_opts = parse_opts(default_opts, in_opts)&#xA;&#xA;in_fields = fieldnames(in_opts);&#xA;n_in_fields = numel(in_fields);&#xA;&#xA;parsed_opts = default_opts;&#xA;&#xA;for i_field = 1:numel(in_fields)&#xA;    cur_field = in_fields{i_field};&#xA;&#xA;    if isfield(default_opts, cur_field)&#xA;        parsed_opts.(cur_field) = in_opts.(cur_field);&#xA;    else&#xA;        error('parse_opts:input', 'Unknown field name `%s`', cur_field);&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8261" LastEditorUserId="8261" LastEditDate="2014-05-26T16:40:15.847" LastActivityDate="2014-05-26T16:40:15.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11694" PostTypeId="2" ParentId="1502" CreationDate="2014-05-22T16:01:30.163" Score="2" Body="&lt;p&gt;Assuming that the issue is that &lt;code&gt;DAEPACK&lt;/code&gt; works in 32-bit mode but not 64-bit mode, here are some approaches to address this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Compile &lt;code&gt;DAEPACK&lt;/code&gt; in 32-bit mode on a 64-bit OS&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;See how to do this in &lt;a href=&quot;https://stackoverflow.com/a/3501880/3618671&quot;&gt;another question&lt;/a&gt;. This may be an easy way to continue to work with it.&lt;/p&gt;&#xA;&lt;p&gt;If the issue is that the &lt;em&gt;generated code&lt;/em&gt; is having issues in 64-bit mode, compile the generated code with similar 32-bit flags or in a 32-bit environment (64-bit OS can run 32-bit code given appropriate libraries, but not &lt;em&gt;vice versa&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Recreate an entire &lt;code&gt;DAEPACK&lt;/code&gt;-safe environment&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a full 32-bit environment in a virtual machine using, e.g., &lt;a href=&quot;https://www.virtualbox.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VirtualBox&lt;/a&gt;, such that &lt;code&gt;DAEPACK&lt;/code&gt; is happy running in it, whether from old binaries, or recompiled in this environment. You can get a release of an open-source OS and compiler as old as you need to get a working version of the software.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Take the output from &lt;code&gt;DAEPACK&lt;/code&gt; and transfer them to the host machine or a separate virtual machine where the environment is appropriate for the other software and continue working. This can be a more recent 64-bit OS, for example, and continue working there.&lt;/p&gt;&#xA;&lt;p&gt;Using the &lt;a href=&quot;https://www.virtualbox.org/manual/ch04.html#sharedfolders&quot; rel=&quot;nofollow noreferrer&quot;&gt;shared folders&lt;/a&gt; feature in VirtualBox should make this easy for you.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Enlist others to help fix &lt;code&gt;DAEPACK&lt;/code&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I noticed that you said that neither you nor the software author have the time to fix this, but no one said you can't enlist anyone else's help! Consider these options:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Hire some undergraduate MIT CS students to help fix the 32-bit vs. 64-bit issues. Note that to entice them to work on it, you may have to compensate them, whether with money or pizza (and/or beer, if they're of legal drinking age), for example. Or consider asking if your advisor can hire some part-time undergraduates to help with this (may be an issue with funding, if your own funding is running out). Or ask for some favors from research programmers in your department.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lobby for an open-source release of &lt;code&gt;DAEPACK&lt;/code&gt;. It looks like its licensing status is &lt;a href=&quot;http://yoric.mit.edu/DAEPACK/Licensing&quot; rel=&quot;nofollow noreferrer&quot;&gt;unclear&lt;/a&gt;, but note that MIT has an &lt;a href=&quot;http://opensource.org/licenses/MIT&quot; rel=&quot;nofollow noreferrer&quot;&gt;eponymous open-source license&lt;/a&gt; that's rather well-known; releasing it under an open license may help get more eyes on the problem and help fix the 32-bit vs. 64-bit issues.&lt;/p&gt;&#xA;&lt;p&gt;Note that depending on your department, legal, administrative, etc. offices may take longer than the 6 months that you have left for your Ph.D., so consider starting this early, while pursuing other alternatives.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="8168" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-05-22T16:01:30.163" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11695" PostTypeId="1" AcceptedAnswerId="11696" CreationDate="2014-05-22T18:21:25.103" Score="5" ViewCount="283" Body="&lt;p&gt;As my experience with CFD grows, I start to gain the feeling that turbomacinery is kind of special in CFD: there are special tools tailor for it. For example, ANSYS has TurboGrid or Vista TF for turbomachinery design. And OpenFOAM has special working group for turbomachinery. I wish to know what makes turbomacinery so 'stand out' in CFD technology, I am sure there is something special about it, otherwise we would just use general solvers. So, can somebody shed some light on this mystery (to me)?&lt;/p&gt;&#xA;" OwnerUserId="8203" LastActivityDate="2014-05-22T20:02:42.897" Title="What makes turbomachinery so special in CFD" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11696" PostTypeId="2" ParentId="11695" CreationDate="2014-05-22T19:39:04.777" Score="4" Body="&lt;p&gt;1) Turbomachinery is ubiquitous, used for the vast majority of power generation in the world. Therefor, there is a huge amount of money and vested interest in it for practical reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Turbomachines are inherently extremely hard to do any kind of detailed experiments on. Visualizing the flowfield or getting any detailed force measurements of an operating turbomachine in order to improve the design is impossible-to-difficult for experiments. This leaves you with CFD as the primary practical tool.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-05-22T19:39:04.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11697" PostTypeId="2" ParentId="11695" CreationDate="2014-05-22T20:02:42.897" Score="2" Body="&lt;p&gt;To add to Aurelius' answer, turbomachinery also requires high-resolution rotating grids with somewhat complicated geometries. This is a very hard problem for mesh generation and solvers both.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-05-22T20:02:42.897" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11698" PostTypeId="1" AcceptedAnswerId="11705" CreationDate="2014-05-23T08:22:47.910" Score="3" ViewCount="444" Body="&lt;p&gt;I want to solve a poisson problem on two domains,(interface problem: Let $\Omega $ be a square, $\Omega=\Omega1 \cup \Omega2$ and let $\Omega1$  be a circle inside the square.&#xA;$\Gamma$ is the boundary of circle.&#xA;$$\Delta u_1=f_1 ~~in ~~\Omega1$$&#xA;$$\Delta u_2=f_2 ~~in ~~\Omega2$$&#xA;$$u_2=g~~on ~~\partial \Omega$$&#xA;$$u_2-u_1=w ~~on~~\Gamma$$&#xA;$$\frac{\partial u_2}{\partial n_2}-\frac{\partial u_1}{\partial n_1}=v~~on~~\Gamma$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should  I do  by the conditions on $\Gamma$, they are boundary conditions? &#xA;If yes, How to enforce these conditions, when the test space is $H^1(\Omega)$.&lt;/p&gt;&#xA;" OwnerUserId="8271" LastActivityDate="2014-05-24T05:04:02.913" Title="How to enforce the boundary conditions" Tags="|finite-element|unstructured-mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11700" PostTypeId="1" CreationDate="2014-05-23T09:42:02.170" Score="2" ViewCount="524" Body="&lt;p&gt;I am trying to solve the Helmholtz equation in some complex unbounded 2D domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left( \frac{\partial^2 \psi}{\partial x^2} + \frac{\partial^2 \psi}{\partial y^2} + E \right) u(x, y) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, to start with, I tried doing that in the simplified domain $\Omega = [-\infty, \infty] \times [0, H]$, that is, just a tube, with Dirichlet boundary conditions on the walls: $\psi(x, 0) = 0, \psi(x, H) = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is known that solutions of this problem are of form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{n \rightarrow}(x, y) = \sin(\frac{\pi n}{H} y) e^{i \sqrt{E - (\pi n / H)^2} x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{n \leftarrow}(x, y) = \sin(\frac{\pi n}{H} y) e^{-i \sqrt{E - (\pi n / H)^2} x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;, for $n = 1 \dots \infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve numerically, I have to bound the domain somehow, so I restricted it to $[0, L] \times [0, H]$ for some big $L$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want the solution to be unique up to normalization, I am doing that by:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;assuming $\left( \frac{\pi}{H} \right)^2 &amp;lt; E &amp;lt; \left( \frac{2 \pi}{H} \right)^2$, that is, getting solution of form $A \psi_{1 \rightarrow}(x, y) + B \psi_{1 \leftarrow}(x, y)$&lt;/li&gt;&#xA;&lt;li&gt;getting rid of the $B$ coefficient and using the following ansatz in the asymptotic region $x \le 0$: $\psi(x, y) = A \sin(\frac{\pi}{H}y) \sqrt{E - \pi^2/H^2} x)$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So now we should get the solution $\psi_{1 \rightarrow}(x, y) = \sin(\frac{\pi}{H}y) \sqrt{E - \pi^2/H^2} x)$ up to the normalization after solving numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am substituting the ansatz, getting boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\psi(0, y) = \sin(\frac{\pi}{H}y)$, (no $x$ dependency due to the freedom in normalization)&lt;/li&gt;&#xA;&lt;li&gt;$\frac{\partial \psi}{\partial x}(0, y) = \sin(\frac{\pi}{H}y) i \sqrt{E - \pi^2/H^2}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Next, to solve using finite differences, I used the grid (I will assume cell size is equal to $1$ for simplicity), so $\psi_{n, m}$ now approximates $\psi(n, m)$, and the central differences scheme, rewriting the equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{n, m + 1} + \psi_{n + 1, m} + \psi_{n, m - 1} + \psi_{n - 1, m} - 4 \psi_{n, m} + E \psi_{n, m} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the values of two first columns of the grid from my boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{0, m} = \sin\left( \frac{\pi}{H}y \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{1, m} = \sin\left( \frac{\pi}{H}y \right) e^{i \sqrt{E - \pi^2/H^2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that we have two consecutive columns $n - 2$, $n - 1$, we should be able to obtain all the columns up to  $L$, since:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{n, m} = 4 \psi_{n - 1, m} - E \psi_{n - 1, m} - \psi_{n - 2, m} - \psi_{n - 1, m - 1} - \psi_{n - 1, m + 1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for some reason, this computation diverges and grows exponentially as $n$ goes from $0$ to $L$. Am I doing something fundamentally wrong (wrong boundary conditions, or I shouldn't use central differences), or that should work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I tried the same for 1D Helmholtz in the domain $[-\infty, \infty]$ and $\psi$ diverges as well in this case, so it does seem as I am missing something fundamental.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.P.S. Turns out that I was just using large step size in the 1D case. However, setting step sizes smaller and smaller in the 2D case does not help :(&lt;/p&gt;&#xA;" OwnerUserId="7176" LastEditorUserId="7176" LastEditDate="2014-05-23T11:49:15.213" LastActivityDate="2014-05-23T11:49:15.213" Title="Solving the Helmholtz equation numerically" Tags="|finite-difference|helmholtz-equation|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="11701" PostTypeId="1" CreationDate="2014-05-23T10:43:40.963" Score="2" ViewCount="302" Body="&lt;p&gt;Consider the following: I would like to solve a differential equation in Fortran (with a Runge-Kutta routine), e.g. &lt;code&gt;x'=-a*x&lt;/code&gt;, where &lt;code&gt;a&lt;/code&gt; is a constant. I then want to solve this for different &lt;code&gt;a&lt;/code&gt;. I now would like to write the code such that it can be applied to other problems. But in general I do not know what additional input my differential equation may need (and what size and type they have). So my question is if it is possible in Fortran to pass all additional parameters to my Runge-Kutta method (at best with a single property) so that it passes it on to my differential equation so that it can be used there? With a known set of parameters this is easy but what about an unknown set?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach so far is this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PROGRAM main&#xA;  USE RKModule&#xA;&#xA;  IMPLICIT NONE&#xA;&#xA;  INTEGER :: l&#xA;&#xA;  REAL*8, PARAMETER :: num_t = 1e3&#xA;  REAL*8, PARAMETER :: min_t = 0.0d0&#xA;  REAL*8, PARAMETER :: max_t = 10.0d0&#xA;&#xA;  REAL*8, DIMENSION(1), PARAMETER :: a = 1.8d0&#xA;&#xA;  REAL*8, DIMENSION(num_t) :: dGrid_t, grid_t&#xA;  COMPLEX*16, DIMENSION(num_t) :: sol&#xA;  COMPLEX*16, DIMENSION(1) :: x_in, x_out&#xA;&#xA;  DO l = 1, num_t&#xA;    dGrid_t(l) = (max_t - min_t) / DBLE(num_t)&#xA;    grid_t(l) = min_t + DBLE(l-0.5d0)*dGrid_t(l)&#xA;  END DO&#xA;&#xA;  x_in(1) = dcmplx(1.0d0, 0.0d0)&#xA;  x_out(1) = dcmplx(0.0d0, 0.0d0)&#xA;&#xA;  DO l = 1, num_t&#xA;    sol(l) = x_in(1)&#xA;    CALL rk45(x_out, DGL, x_in, grid_t(l), dGrid_t(l), a)&#xA;    x_in = x_out&#xA;  END DO&#xA;&#xA;  OPEN(UNIT = 1000, FILE = 'sol.dat', ACTION = 'WRITE')&#xA;  WRITE(1000, FMT = '(e14.7e3, 1x, e14.7e3)', ADVANCE = &quot;NO&quot;) sol&#xA;  CLOSE(1000)&#xA;&#xA;  CONTAINS&#xA;&#xA;  SUBROUTINE DGL(t,f,df,vararg_in)&#xA;    REAL*8, INTENT(IN) :: t&#xA;    COMPLEX*16, DIMENSION(:), INTENT(IN) :: f&#xA;    COMPLEX*16, DIMENSION(:), INTENT(OUT) :: df&#xA;    REAL*8, DIMENSION(:), INTENT(IN), OPTIONAL :: vararg_in&#xA;&#xA;    REAL*8, DIMENSION(1) :: b&#xA;&#xA;    b = 1.0d0&#xA;    IF (PRESENT(vararg_in)) THEN&#xA;      b = vararg_in&#xA;    END IF&#xA;&#xA;    df = -b*f&#xA;&#xA;  END SUBROUTINE DGL&#xA;&#xA;END PROGRAM main&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the Runge Kutta Routine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MODULE RKModule&#xA;  IMPLICIT NONE&#xA;&#xA;  CONTAINS&#xA;&#xA;  SUBROUTINE rk45(x_out, RHS, x_in, t, dt, vararg_in)&#xA;    INTERFACE&#xA;      SUBROUTINE RHS(t,f,df,a)&#xA;    REAL*8, INTENT(IN) :: t&#xA;    COMPLEX*16, DIMENSION(:), INTENT(IN) :: f&#xA;    COMPLEX*16, DIMENSION(:), INTENT(OUT) :: df&#xA;    REAL*8, DIMENSION(:), INTENT(IN), OPTIONAL :: a&#xA;      END SUBROUTINE&#xA;    END INTERFACE&#xA;    REAL*8, INTENT(IN) :: t, dt &#xA;    COMPLEX*16, DIMENSION(:), INTENT(IN) :: x_in&#xA;    COMPLEX*16, DIMENSION(:), INTENT(OUT) :: x_out&#xA;    REAL*8, DIMENSION(:), INTENT(IN), OPTIONAL :: vararg_in&#xA;&#xA;    REAL*8 :: tHlp&#xA;    COMPLEX*16, DIMENSION(SIZE(x_in)) :: k1 ,k2, k3, k4, dx, x_hlp&#xA;    REAL*8, DIMENSION(:), ALLOCATABLE :: vararg_in_copy&#xA;&#xA;    IF (PRESENT(vararg_in)) THEN&#xA;      ALLOCATE(vararg_in_copy(SIZE(vararg_in)))&#xA;      vararg_in_copy = vararg_in&#xA;&#xA;      tHlp = t&#xA;      x_hlp = x_in&#xA;      CALL RHS(tHlp, x_hlp, dx, vararg_in_copy)&#xA;      k1 = dt * dx&#xA;&#xA;      tHlp = t + 0.5d0 * dt&#xA;      x_hlp = x_in + 0.5d0 * k1&#xA;      CALL RHS(tHlp, x_hlp, dx, vararg_in_copy)&#xA;      k2 = dt * dx&#xA;&#xA;      x_hlp = x_in + 0.5d0 * k2&#xA;      CALL RHS(tHlp, x_hlp, dx, vararg_in_copy)&#xA;      k3 = dt * dx&#xA;&#xA;      tHlp = t + dt&#xA;      x_hlp = x_in + k3&#xA;      CALL RHS(tHlp, x_hlp, dx, vararg_in_copy)&#xA;      k4 = dt * dx&#xA;&#xA;      x_out = x_in + (k1 + 2.0d0 * k2 + 2.0d0 * k3 + k4) / 6.0d0&#xA;&#xA;      DEALLOCATE(vararg_in_copy)&#xA;    ELSE&#xA;&#xA;      tHlp = t&#xA;      x_hlp = x_in&#xA;      CALL RHS(tHlp, x_hlp, dx)&#xA;      k1 = dt * dx&#xA;&#xA;      tHlp = t + 0.5d0 * dt&#xA;      x_hlp = x_in + 0.5d0 * k1&#xA;      CALL RHS(tHlp, x_hlp, dx)&#xA;      k2 = dt * dx&#xA;&#xA;      x_hlp = x_in + 0.5d0 * k2&#xA;      CALL RHS(tHlp, x_hlp, dx)&#xA;      k3 = dt * dx&#xA;&#xA;      tHlp = t + dt&#xA;      x_hlp = x_in + k3&#xA;      CALL RHS(tHlp, x_hlp, dx)&#xA;      k4 = dt * dx&#xA;&#xA;      x_out = x_in + (k1 + 2.0d0 * k2 + 2.0d0 * k3 + k4) / 6.0d0&#xA;&#xA;    END IF&#xA;&#xA;  END SUBROUTINE rk45&#xA;&#xA;END MODULE RKModule&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7040" LastActivityDate="2014-05-27T14:28:04.810" Title="'Varargin' in Fortran" Tags="|fortran|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11702" PostTypeId="1" AcceptedAnswerId="11703" CreationDate="2014-05-23T19:17:22.723" Score="8" ViewCount="1632" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Fast_marching_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast Marching Method&lt;/a&gt; is a way of solving the &lt;a href=&quot;http://en.wikipedia.org/wiki/Eikonal_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eikonal Equation&lt;/a&gt; on a discrete grid, essentially just computing a wavefront speading out from initial points, IE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ghi6j.png&quot; alt=&quot;Wavefront&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that we want to compute the time at which a wavefront, moving across a field with a velocity function $F(x, y)$, reaches each point in a grid, IE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/aL9h6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is done by, as described in the &quot;Implementation Details of the Fast Marching Methods&quot; link of the Wikipedia page, setting all beginning nodes as frozen, with T=0. Then every node adjacent (4 neighbor neighborhood) to these nodes is marked as being in the &quot;narrow band,&quot; and their $T(x, y)$ is computed using a quadratic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(T(x, y) - a)^2 + (T(x, y) - b)^2 = (1/F(x, y))^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a = Min(T(x + 1, y), T(x - 1, y))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$b = Min(T(x, y + 1), T(x, y - 1))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;only considering values for $T(...)$ of neighboring frozen nodes. Thus, for example, if the node above and the node below aren't frozen, the $(T(x, y) - b)^2$ term drops out of the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using these newly computed $T(x, y)$ values, each narrow band node is added to a stack, then the one with the lowest $(T(x, y))$ is removed, set as &quot;frozen,&quot; and this process is repeated until all accessible nodes are &quot;frozen.&quot; Because this also means some &quot;narrow band&quot; nodes are recomputed with more information as more of their neighbors become frozen, when this re-computation occurs they will simply be added to the stack again, and if a previously &quot;narrow band&quot; node is removed from the stack that is now frozen it is just ignored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This process typically works very well, and is widely used in many fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I tried implementing this myself, I solved the quadratic as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$firstSolution = \frac{\sqrt{-c^2(a^2c^2-2abc^2+b^2c^2-2)}+ac^2+bc^2)}{2c^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$secondSolution = \frac{-\sqrt{-c^2(a^2c^2-2abc^2+b^2c^2-2)}+ac^2+bc^2)}{2c^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$c = F(x, y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T(x, y) = Max(firstSolution, secondSolution)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet, &lt;strong&gt;there are some cases where this method fails&lt;/strong&gt;, for example, with $a = 53.7432442$, $b = 53.8721733$, and $c = 11.18034$, square root insides equal $-9.730039,$ meaning there is no (real) solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought this may just be because I was changing the value of f(x, y) too quickly (we're approximating a continuous function, and too sharp changes make it not differentiable), but it seemed to happen even when I was trying to change it as smoothly as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a reason for this, or a better way to approximate these T(x, y) values? Or is this just an inherent problem with this method?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article linked to in the Wikipedia page suggested second order approximations, but I don't think this solves the problem. I'm sorry if this is the wrong place to post such a question as well (being somewhat implementation oriented), feel free to migrate as needed.&lt;/p&gt;&#xA;" OwnerUserId="8275" LastEditorUserId="8275" LastEditDate="2014-11-05T21:11:20.397" LastActivityDate="2014-11-05T21:11:20.397" Title="Solving the quadratic in the Fast Marching Method" Tags="|pde|quadrature|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11703" PostTypeId="2" ParentId="11702" CreationDate="2014-05-23T19:27:35.710" Score="5" Body="&lt;p&gt;Physically, this problem arises when the two input nodes are &quot;impossibly far apart&quot; according to the planar wavefront approximation.  Specifically, if two nodes separated by distance $\Delta x \sqrt{2}$ on the grid have $T$ values differing by more than $\Delta x \sqrt{2}$, there is no solution plane that fits both distance values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if the quadratic does have a solution, it might be the case that two inputs $T_0 &amp;lt; T_1$ result in an output value $T_2$ with $T_0 &amp;lt; T_2 &amp;lt; T_1$.  This is physically invalid, as it corresponds to information flowing backwards through time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution in both cases is to fall back to a lower dimension wave front, ignoring the value that is too large, and corresponding to an axis aligned wavefront approximation.  This can be implemented branch free.  If we assume $\Delta x = 1$ and that all distance values are positive, the example code is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double step(double px, double py) {&#xA;  small_sort(px,py);&#xA;  py = min(py,px+1);&#xA;  return .5*(px+py+sqrt(2-sqr(px-py)));&#xA;}&#xA;&#xA;double step(double px, double py, double pz) {&#xA;  small_sort(px,py,pz);&#xA;  py = min(py,px+1);&#xA;  pz = min(pz,.5*(px+py+sqrt(2-sqr(px-py))));&#xA;  return 1./3*(px+py+pz+sqrt(3-sqr(px-py)-sqr(py-pz)-sqr(pz-px)));&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here &lt;code&gt;small_sort&lt;/code&gt; sorts two or three numbers, and each &lt;code&gt;min&lt;/code&gt; operation implements falling back from 3D to 2D or 2D to 1D planar wavefront approximations.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2014-05-23T19:53:52.983" LastActivityDate="2014-05-23T19:53:52.983" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11704" PostTypeId="1" CreationDate="2014-05-23T20:38:28.277" Score="3" ViewCount="856" Body="&lt;p&gt;I need an accurate implementation (for use in C/C++) of the logarithm of the incomplete Beta function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\log \mathrm{B}(x,y;\alpha,\beta) = \log \int_x^y t^{\alpha-1}(1-t)^{\beta-1}\mathrm{d}t$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with domain $\alpha,\beta&amp;gt;0$, $0\le x\le y \le 1$. I need this to avoid overflows with very large values of the incomplete Beta function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Boost and GSL provide direct implementations of $\mathrm{B}(x,y;\alpha,\beta)$ or its regularized version. This won't work for me, because from these there's no way to get the value of $\log \mathrm{B}(x,y;\alpha,\beta)$ without going through the large values I am trying to avoid (or at least I don't know how).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;  I found that &lt;code&gt;R&lt;/code&gt; has an implementation of the logarithm of the incomplete Beta function (see &lt;a href=&quot;https://stat.ethz.ch/R-manual/R-patched/library/stats/html/Beta.html&quot; rel=&quot;nofollow&quot;&gt;https://stat.ethz.ch/R-manual/R-patched/library/stats/html/Beta.html&lt;/a&gt;). Can I use this from C++? Will it be as fast as native C code?&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2015-04-21T21:30:52.327" LastActivityDate="2015-04-22T00:19:41.847" Title="Accurate implementation of the logarithm of the incomplete Beta function in C++?" Tags="|numerical-analysis|c++|c|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11705" PostTypeId="2" ParentId="11698" CreationDate="2014-05-24T05:04:02.913" Score="1" Body="&lt;p&gt;The conditions on $\Gamma$ are called interface conditions. You could incorporate the jump conditions into your weak form. Typically, the trial and test spaces are $H^{1}(\Omega_{1}) \times H^{1}(\Omega_{2})$. Numerically, the interfacial condition is easier to treat if $\Gamma$ is aligned with a mesh of $\Omega$. One example of how you might treat the embedded interface condition can be found in Chapter 4 of the &lt;a href=&quot;http://dukespace.lib.duke.edu/dspace/bitstream/handle/10161/7213/AnnavarapuSrinivas_duke_0066D_11894.pdf?sequence=1&quot; rel=&quot;nofollow&quot;&gt;PhD thesis by Chandrasekhar Annavarapu&lt;/a&gt;, out of John Dolbow's group at Duke.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-24T05:04:02.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11706" PostTypeId="1" AcceptedAnswerId="11736" CreationDate="2014-05-24T08:59:49.640" Score="4" ViewCount="2694" Body="&lt;p&gt;I like to have a list just like in Python, but for Fortran. I know that there is a &lt;a href=&quot;http://fortranwiki.org/fortran/show/gen_list&quot; rel=&quot;nofollow&quot;&gt;generic list type&lt;/a&gt;. But here I would have to define a type for each data I would like into the list, e.g. one for Real*8, one for Complex*16,Dimension(:,:) and so on. I would like to put data into the list for which I do not have to define the dimension and if it is Real, Complex or Integer - just like a list in Python where I can put everything I want into it. Is that possible? Does such a list already exist for Fortran?&lt;/p&gt;&#xA;" OwnerUserId="7040" LastActivityDate="2014-05-28T08:14:06.363" Title="Python like generic List in Fortran" Tags="|fortran|" AnswerCount="2" CommentCount="2" ClosedDate="2014-06-02T21:24:22.787" ContentLicense="CC BY-SA 3.0" />
  <row Id="11707" PostTypeId="2" ParentId="11706" CreationDate="2014-05-24T10:19:31.240" Score="2" Body="&lt;p&gt;This answer is more of a comment; my remarks are too long for a comment and having additional formatting options will be useful.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I know that there is a &lt;a href=&quot;http://fortranwiki.org/fortran/show/gen_list&quot; rel=&quot;nofollow noreferrer&quot;&gt;generic list type&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A quick perusal of the link you cite describes a Fortran module for a &lt;a href=&quot;http://en.wikipedia.org/wiki/Linked_list&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;linked list&lt;/em&gt;&lt;/a&gt;. Random access (that is, look up the element stored at an arbitrary index) of a list in Python is $O(1)$; for a linked list of length $n$, random access is $O(n)$, because to look at the $i$th element, you must start at the head of the list, and traverse $i - 1$ elements (the links of the list) to get to the $i$th element. The data structures are different.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I would like to put data into the list for which I do not have to define the dimension and if it is Real, Complex or Integer - just like a list in Python where I can put everything I want into it. Is that possible?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. Fortran is a Turing-complete language, and what you're suggesting is emulating the features of Python (also a Turing-complete language), so it is possible to define a Fortran &quot;list&quot; data type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One potential way of implementing a &quot;list&quot; data type would be to have three arrays:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the first array is essentially storage for data; you can use the &lt;code&gt;transfer&lt;/code&gt; function to convert data into a common representation, like, say, an array of characters (this strategy more or less emulates C-style void pointers in Fortran, see &lt;a href=&quot;http://jblevins.org/log/transfer&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;the second array stores index offsets for each object you store in the list&lt;/li&gt;&#xA;&lt;li&gt;the third array stores type information for each object in some manner; this step is probably easy for built-in types, but Fortran 90 allows you to define composite types (sort of like structs), and storing information for composite types might require a more involved scheme or more auxiliary data structures.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm not saying this sort of scheme is the best way -- it's probably not, since it's the first idea that popped into my head. I am biased against using pointers in Fortran, since I learned a Fortran 77-biased style of Fortran 90, but Fortran pointers could be used to great effect here. If I were implementing this in C, I would absolutely use pointers, because pointers are essential to the C idiom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would guess that most methods for emulating a &quot;list&quot; data type are going to involve clever data encoding and retrieval schemes, plus a nice interface to hide it away. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might find looking at descriptions of the C implementation of the Python list data type like &lt;a href=&quot;http://www.laurentluce.com/posts/python-list-implementation/comment-page-1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;, or &lt;a href=&quot;http://svn.python.org/projects/python/trunk/Objects/listobject.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;the source code&lt;/a&gt;, or the answers to &lt;a href=&quot;https://stackoverflow.com/questions/3917574/how-is-pythons-list-implemented&quot;&gt;this Stack Overflow question&lt;/a&gt; helpful.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does such a list already exist for Fortran?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I couldn't find one with a quick search.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I like to have a list just like in Python, but for Fortran.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why? What is your use case? If you know the type of data you want to store in advance, it's easier just to use arrays with known data types, and it will save you work, hassle, and probably execution time (storing arbitrary objects requires some indirection and additional memory references). Furthermore, it's more idiomatic Fortran to use standard array data structures (even though, yes, people can and do emulate other data structures using arrays); data structure munging is more characteristically done in other languages (C-style languages, interpreted languages like Lisp, Python, Ruby, etc.).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-05-24T10:19:31.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11709" PostTypeId="2" ParentId="4925" CreationDate="2014-05-24T11:41:31.147" Score="0" Body="&lt;p&gt;Let me add to the list another method that is used a lot in Matlab but that I &lt;strong&gt;do not&lt;/strong&gt; recommend, to point against it: &lt;code&gt;options&lt;/code&gt; structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, your syntax requires an additional options struct &lt;code&gt;opts&lt;/code&gt;, and you check for the presence of specific fields. Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; options.TolX=1-e-3;&#xA;&amp;gt;&amp;gt; x=fzero(fun,x0,options)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% in fzero.m&#xA;function x=fzero(fun,x0,options)&#xA;if not(isfield(options,'TolX'))&#xA;    tol=defaultTolerance;&#xA;else&#xA;    tol=options.TolX;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is what is used in several standard Matlab methods, for instance &lt;a href=&quot;http://www.mathworks.it/it/help/matlab/ref/fzero.html#inputarg_options&quot; rel=&quot;nofollow&quot;&gt;fzero&lt;/a&gt; that I used for this example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main reason why you should not do it is that error checking is terrible: if you have a field &lt;code&gt;options.tolX&lt;/code&gt; (missing capital T), things will often go silently wrong.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-05-24T11:41:31.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11711" PostTypeId="2" ParentId="11526" CreationDate="2014-05-24T19:32:41.887" Score="2" Body="&lt;p&gt;I am not sure how deep you want to get into it, but you may want to check out Z. Bai's &quot;Error Analysis of the Lanczos algorithm for the non-symmetric eigenvalue problem&quot;, &lt;a href=&quot;http://www.ams.org/journals/mcom/1994-62-205/S0025-5718-1994-1201066-7/&quot; rel=&quot;nofollow&quot;&gt;Math. Comp. 62 (1994), 209-226&lt;/a&gt;. It is open access, so you can access it freely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C.C. Paige's &quot;Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem&quot;, &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0024379580901676&quot; rel=&quot;nofollow&quot;&gt;Linear Algebra and its Applications, Volume 34, (1980), Pages 235-258&lt;/a&gt;, may be more appropriate, as it deals with symmetric problems.&lt;/p&gt;&#xA;" OwnerUserId="6564" LastEditorUserId="6564" LastEditDate="2014-05-24T19:45:16.130" LastActivityDate="2014-05-24T19:45:16.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11712" PostTypeId="1" CreationDate="2014-05-25T03:26:43.433" Score="10" ViewCount="1506" Body="&lt;p&gt;In the technical report on Galahad[1], the authors state, in the context of general nonlinear programming problems,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To our minds, there had never really been much doubt that SQP [sequential quadratic programming] methods would be more successful [than Augmented Lagrangian methods] in the long term...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What could be the basis for that belief? I.e., are there any theoretical results that suggest SQP methods should be faster/more reliable than Augmented Lagrangian methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Galahad, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization, by Gould, Orban, and Toint&lt;/p&gt;&#xA;" OwnerUserId="1420" LastActivityDate="2014-05-27T21:54:13.170" Title="Why is SQP better than Augmented Lagrangian for nonlinear programming?" Tags="|nonlinear-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11713" PostTypeId="1" AcceptedAnswerId="11715" CreationDate="2014-05-25T03:47:53.757" Score="6" ViewCount="1393" Body="&lt;p&gt;Reading through Tim Davis' book &lt;em&gt;Direct Methods For Sparse Linear Systems&lt;/em&gt;, he says that matlab can use a supernodal Cholesky decomposition but never uses a multifrontal Cholesky decomposition. At least when using the backslash operator. By contrast, HSL MA57, used in many optimization codes, is a sparse symmetric indefinite solver that uses a multifrontal method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only have a very basic understanding of the differences between supernodal and multifrontal methods. So I'm curious if there are restrictions for when supernodal vs. multifrontal methods can be applied? In cases when they are both valid, are there any trade-offs between them that could lead to one being better than the other?&lt;/p&gt;&#xA;" OwnerUserId="1420" LastActivityDate="2014-05-25T13:38:11.227" Title="Supernodal vs. Multifrontal Matrix Decompositions" Tags="|sparse-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11715" PostTypeId="2" ParentId="11713" CreationDate="2014-05-25T13:38:11.227" Score="5" Body="&lt;p&gt;Both supernodal and multifrontal methods achieve &#xA;high performance using the same idea: performing matrix operations on dense blocks using &#xA;BLAS3-like matrix kernels. There are lots of reports comparing specific implementations of &#xA;these methods on a wide variety of matrices, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.numerical.rl.ac.uk/reports/ghsRAL200505.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.numerical.rl.ac.uk/reports/ghsRAL200505.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any results showing one approach to be fundamentally better than the other, in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are matrices that are so sparse that the dense submatrix approach of supernodal and multifrontal&#xA;methods does not lead to optimal performance. See, for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1824814&quot; rel=&quot;noreferrer&quot;&gt;http://dl.acm.org/citation.cfm?id=1824814&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you point out, MA57 is designed for symmetric indefinite matrices when an $LL^T$ is not always possible&#xA;but an $LDL^T$ decomposition is. MATLAB uses MA57 for symmetric indefinite matrices. See,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ldl.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.mathworks.com/help/matlab/ref/ldl.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By setting&lt;/p&gt;&#xA;&#xA;&lt;p&gt;spparms('spumoni',1);&lt;/p&gt;&#xA;&#xA;&lt;p&gt;before invoking backslash, you can see what algorithms are being used for your particular matrix. For more information, see:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/spparms.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.mathworks.com/help/matlab/ref/spparms.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-05-25T13:38:11.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11716" PostTypeId="1" AcceptedAnswerId="11717" CreationDate="2014-05-25T14:20:26.630" Score="0" ViewCount="115" Body="&lt;p&gt;Is there a common / good way to represent graphs in Matlab and operate on them? I need to read in an image as a graph and use several algorithms of graph-theory like minimal cut.&lt;/p&gt;&#xA;" OwnerUserId="8286" LastActivityDate="2014-05-25T16:15:43.580" Title="Matlab represent graphs" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11717" PostTypeId="2" ParentId="11716" CreationDate="2014-05-25T16:15:43.580" Score="1" Body="&lt;p&gt;Matlab has a limited functionality network analysis toolbox. Yet, I guess it covers what you might need:&#xA;&lt;a href=&quot;http://www.mathworks.com/help/bioinfo/network-analysis-and-visualization.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/bioinfo/network-analysis-and-visualization.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab is not exactly the best way to deal with graphs. However, by using sparse matrices and mex files, many of the operations can be efficiently performed. In that sense, I think the below 3rd parth toolboxes has some good computational capabilities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/4266-grtheory-graph-theory-toolbox&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/4266-grtheory-graph-theory-toolbox&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/10922-matlabbgl&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/10922-matlabbgl&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/5355-toolbox-graph&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/5355-toolbox-graph&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/19218-matgraph&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/19218-matgraph&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-05-25T16:15:43.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11718" PostTypeId="1" AcceptedAnswerId="11721" CreationDate="2014-05-25T16:33:48.427" Score="3" ViewCount="215" Body="&lt;p&gt;I am trying to find the solution for the M variables in the following system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;0 = C_{b} M^{b}_{x} - M^{a}_{x} k_{2a} + M^{a}_{y} \left(\omega - \omega_{a}\right)\\&#xA;0 = C_{a} M^{a}_{x} - M^{b}_{x} k_{2b} + M^{b}_{y} \left(\omega - \omega_{b}\right)\\&#xA;0 = C_{b} M^{b}_{y} + M^{a}_{x} \left(- \omega + \omega_{a}\right) - M^{a}_{y} k_{2a} - M^{a}_{z} \omega_{1}\\&#xA;0 = C_{a} M^{a}_{y} + M^{b}_{x} \left(- \omega + \omega_{b}\right) - M^{b}_{y} k_{2b} - M^{b}_{z} \omega_{1}\\&#xA;0 = C_{b} M^{b}_{z} + \frac{M^{a}_{0}}{T_{1a}} + M^{a}_{y} \omega_{1} - M^{a}_{z} k_{1a}\\&#xA;0 = C_{a} M^{a}_{z} + \frac{M^{b}_{0}}{T_{1b}} + M^{b}_{y} \omega_{1} - M^{b}_{z} k_{1b}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. I am solving for the 6 $M_x$, $M_y$, $M_z$ variables, but I am not solving for the $M^a_0$, $M^b_0$ variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried using Sympy to solve this, but it has been running for over 17 hours, and has not yet returned a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in a fast way to solve this.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Edit:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.wakari.io/sharing/bundle/daviewales/Two%20site%20exchange%203&quot; rel=&quot;nofollow&quot;&gt;Click this link to have a look at my code.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Edit 2:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For anyone who's interested, @Doug Lipinski's solution was very fast.&#xA;However, as he mentioned the solution was &lt;strong&gt;VERY&lt;/strong&gt; large. Below is just one of the solutions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;M^{b}_{z} = \frac{\frac{M^{a}_{0} \left(C_{a} - \frac{\omega_{1}^{2} \left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega - \omega_{a}\right)}{k_{2a} \left(\frac{C_{a}                C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{\left(- k_{2b} - \frac{\left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega - \omega_{a}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} -     k_{2b}\right)}\right) \left(C_{b} - \frac{C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{b}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{- \frac{C_{a} C_{b}   \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right)            \left(\omega - \omega_{a}\right)} - \frac{\left(- \omega + \omega_{b}\right) \left(\omega - \omega_{b}\right)}{\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}}\right) \left(- \frac{C_{a} C_{b} \left(- \omega +       \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right) \left(\omega -              \omega_{a}\right)\right)}\right)}{T_{1a} \left(\frac{\omega_{1}^{2} \left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega - \omega_{a}\right)}{k_{2a} \left(\frac{C_{a}                 C_{b}}{k_{2a}} - k_{2b}\right)}\right) \left(C_{b} - \frac{C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{b}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} -                             k_{2b}\right)}\right)}{\left(- k_{2b} - \frac{\left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega - \omega_{a}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} -                      k_{2b}\right)}\right) \left(C_{b} - \frac{C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{b}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{- \frac{C_{a} C_{b}   \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right)            \left(\omega - \omega_{a}\right)} - \frac{\left(- \omega + \omega_{b}\right) \left(\omega - \omega_{b}\right)}{\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}}\right) \left(- \frac{C_{a} C_{b} \left(- \omega +       \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right) \left(\omega -              \omega_{a}\right)\right)^{2}} + \frac{\omega_{1}^{2}}{- \frac{C_{a} C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} -                k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)} - k_{1a}\right)} - \frac{M^{b}_{0}}{T_{1b}}}{\frac{\omega_{1}^{2}}{- k_{2b} -               \frac{\left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega - \omega_{a}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right) \left(C_{b} - \frac{C_{b} \left(-      \omega + \omega_{a}\right) \left(\omega - \omega_{b}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{- \frac{C_{a} C_{b} \left(- \omega + \omega_{a}\right) \left(\omega -         \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)} - \frac{\left(- \omega +   \omega_{b}\right) \left(\omega - \omega_{b}\right)}{\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}}} - k_{1b} - \frac{\left(C_{a} - \frac{\omega_{1}^{2} \left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right)  \left(\omega - \omega_{a}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{\left(- k_{2b} - \frac{\left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega -       \omega_{a}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right) \left(C_{b} - \frac{C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{b}\right)}{k_{2a} \left(\frac{C_{a}  C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{- \frac{C_{a} C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} +    \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)} - \frac{\left(- \omega + \omega_{b}\right) \left(\omega - \omega_{b}\right)}{\frac{C_{a} C_{b}}{k_{2a}} -             k_{2b}}\right) \left(- \frac{C_{a} C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}}      \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)\right)}\right) \left(C_{b} - \frac{\omega_{1}^{2} \left(C_{b} - \frac{C_{b} \left(- \omega + \omega_{a}\right) \left(\omega -           \omega_{b}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{\left(- k_{2b} - \frac{\left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega -                      \omega_{a}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right) \left(C_{b} - \frac{C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{b}\right)}{k_{2a} \left(\frac{C_{a}  C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{- \frac{C_{a} C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} +    \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)} - \frac{\left(- \omega + \omega_{b}\right) \left(\omega - \omega_{b}\right)}{\frac{C_{a} C_{b}}{k_{2a}} -             k_{2b}}\right) \left(- \frac{C_{a} C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}}      \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)\right)}\right)}{\frac{\omega_{1}^{2} \left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega -                        \omega_{a}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right) \left(C_{b} - \frac{C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{b}\right)}{k_{2a} \left(\frac{C_{a}  C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{\left(- k_{2b} - \frac{\left(C_{a} - \frac{C_{a} \left(- \omega + \omega_{b}\right) \left(\omega - \omega_{a}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} -     k_{2b}\right)}\right) \left(C_{b} - \frac{C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{b}\right)}{k_{2a} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)}\right)}{- \frac{C_{a} C_{b}   \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right)            \left(\omega - \omega_{a}\right)} - \frac{\left(- \omega + \omega_{b}\right) \left(\omega - \omega_{b}\right)}{\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}}\right) \left(- \frac{C_{a} C_{b} \left(- \omega +       \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} - k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right) \left(\omega -              \omega_{a}\right)\right)^{2}} + \frac{\omega_{1}^{2}}{- \frac{C_{a} C_{b} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)}{k_{2a}^{2} \left(\frac{C_{a} C_{b}}{k_{2a}} -                k_{2b}\right)} - k_{2a} + \frac{1}{k_{2a}} \left(- \omega + \omega_{a}\right) \left(\omega - \omega_{a}\right)} - k_{1a}}}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the equation has been forced onto multiple lines by this site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am going to try using some of Sympy's simplify functions to see if it can be simplified at all.&lt;/p&gt;&#xA;" OwnerUserId="8288" LastEditorUserId="8288" LastEditDate="2014-05-26T07:12:00.807" LastActivityDate="2014-05-28T16:03:25.263" Title="Solving linear system with 6 equations and 22 unknowns for six of the unknowns" Tags="|linear-algebra|linear-solver|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11719" PostTypeId="1" CreationDate="2014-05-25T16:48:48.793" Score="2" ViewCount="84" Body="&lt;p&gt;A consistency notion in constraint programming:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $P = (X, D, C)$ be a &lt;a href=&quot;http://en.wikipedia.org/wiki/Constraint_satisfaction_problem&quot; rel=&quot;nofollow&quot;&gt;CSP&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a set of variables $Y \subseteq X$ with $|Y| = k -1$, a locally consistent instantiation $I$ on $Y$ is $k$-consistent iff for any $k$th variable $x_{i_{k}} \in X \setminus Y$ there exists a value $v_{i_{k}} \in D(x_{i_{k}})$ such that $I \cap \{ (x_{i_k}, v_{i_k}) \}$ is locally consistent.&#xA;    The CSP $P$ is $k$-consistent iff for any set $Y$ of $k-1$ variables, any locally consistent instantiation on $Y$ is $k$-consistent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, 3-consistency ensures that any instantiation any pair of variables can be extended to an instantiation involving any third variable without violating any constraint. It is equivalent to path consistency. Similarly, 2-consistency is also known as arc consistency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how convex geometry can ensure global consistency, because it seems that there is a relationship between consistency and convexity.&#xA;Because Row Convex Constraint with path(3)-consistency guarantees global consistency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can any one help me finding a track to research or a way to prove this relationship?&lt;/p&gt;&#xA;" OwnerUserId="8289" LastEditorUserId="4244" LastEditDate="2014-05-26T13:01:36.560" LastActivityDate="2014-05-26T13:01:36.560" Title="About Convex Geometry" Tags="|computational-geometry|convex-optimization|constrained-optimization|combinatorics|constraints|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11721" PostTypeId="2" ParentId="11718" CreationDate="2014-05-26T02:35:59.337" Score="4" Body="&lt;p&gt;I suggest using the &lt;code&gt;solve_linear_system_LU&lt;/code&gt; function in SymPy rather than just the more generic &lt;code&gt;solve&lt;/code&gt;. This works fine for me, although the result is ridiculously long. In fact it's so long that trying to display it nicely via MathJax (used by if you &lt;code&gt;init_printing()&lt;/code&gt;) in iPython locks up my machine. Obviously my comment about solving this by hand in 15 minutes was way off, that would take forever. Try this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sympy import *&#xA;&#xA;omegaA, omegaB, omega1, omega = symbols('\omega_a \omega_b \omega_1 \omega')&#xA;Max, Mbx, May, Mby, Maz, Mbz = symbols('M^a_x M^b_x M^a_y M^b_y M^a_z M^b_z')&#xA;Ma0, Mb0 = symbols('M^a_0 M^b_0')&#xA;k1a, k1b, k2a, k2b = symbols('k_1a k_1b k_2a k_2b')&#xA;Ca, Cb = symbols('C_a C_b')&#xA;T1a, T1b = symbols('T_1a T_1b')&#xA;&#xA;system = Matrix( ( ( -k2a, Cb, -(omegaA-omega), 0, 0, 0, 0 ),&#xA;                   ( Ca, -k2b, 0, -(omegaB-omega), 0, 0, 0 ),&#xA;                   ( (omegaA-omega), 0, -k2a, Cb, -omega1, 0, 0 ),&#xA;                   ( 0, (omegaB-omega), Ca, -k2b, 0, -omega1, 0 ),&#xA;                   ( 0, 0, omega1, 0, -k1a, Cb, -Ma0/T1a ),&#xA;                   ( 0, 0, 0, omega1, Ca, -k1b, -Mb0/T1b ) ) )&#xA;&#xA;solve_linear_system_LU(system, [Max, Mbx, May, Mby, Maz, Mbz])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get an answer very quickly, but it's extremely long. You should also double check my code since I haven't gone through it too carefully.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there's not a way to simplify the solution, it seems incredibly impractical to actually use. I would bet it's faster to solve the system than to evaluate the symbolic solution.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-05-26T02:35:59.337" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11722" PostTypeId="1" AcceptedAnswerId="11728" CreationDate="2014-05-26T06:25:39.063" Score="0" ViewCount="686" Body="&lt;p&gt;To get a numerical evaluation of the first (K) and second (E) complete elliptic integrals:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$K(k)=\int_0^1\frac{dt}{(1-t^2)^{1/2}(1-k^2t^2)^{1/2}}, \ \ \ \ \ E(k)=\int_0^1\frac{(1-k^2t^2)^{1/2}}{(1-t^2)^{1/2}}dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in a left neighbourhood of the point $k=1$. What numerical methods do you recommend to get a &quot;good&quot; approximation of K and E in a left neighbourhood of the point $k=1$?&lt;/p&gt;&#xA;" OwnerUserId="8295" LastActivityDate="2014-05-26T22:18:16.987" Title="Numerical evaluation of the first and second complete elliptic integrals" Tags="|numerical-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11723" PostTypeId="1" AcceptedAnswerId="11726" CreationDate="2014-05-26T11:19:58.597" Score="0" ViewCount="792" Body="&lt;p&gt;I have to solve the problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_x 1^Tx+\frac{\lambda}{2}\|\Omega\mu-x\|_2^2+\frac{\beta}{2}\|x-\bar{\gamma}\|_2^2\quad\text{w.r.t.}\quad Px-c=0,\ \ x\geq0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and in order to do that with Matlab I am currently using &lt;code&gt;fmincon&lt;/code&gt; like that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FMinValue = @(x) sum(x)            + lambda * sum((Omega*mu - x).^2) / 2 + beta * sum((x - gamma_bar).^2) / 2;&#xA;GradFMin  = @(x) ones(length(x),1) + lambda * (Omega*mu - x)             + beta * (x - gamma_bar);&#xA;Fmin = @(x) FMin(x, FMinValue, GradFMin); % merges FMinValue and GradFMin so that fmincon can handle it&#xA;HessianFPlusLagrangianPart = @(x, lagrange_multiplier) spdiags((beta-lambda)*ones(length(x),1),0,length(x),length(x));&#xA;&#xA;options = optimoptions('fmincon', 'Algorithm', 'interior-point' , 'GradObj', 'on', 'Hessian', 'user-supplied', 'HessFcn', HessianFPlusLagrangianPart);&#xA;x = fmincon(Fmin, ones(length(gamma_bar), 1), [], [], P_bar, c_bar, zeros(length(gamma_bar), 1), [], [], options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here $\lambda$ and $\beta$ are fixed parameters which could be equal. If that is the case then $\lambda-\beta=0$ which means my hessian matrix is zero as well. Of course I get the Matlab warning &lt;code&gt;Warning: Matrix is close to singular or badly scaled&lt;/code&gt;, but a zero hessian is correct in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions what to do in this situation? Unfortunately I cannot change to &lt;code&gt;lsqnonlin&lt;/code&gt; because I have got the linear term &lt;code&gt;sum(x)&lt;/code&gt; in my objectiv function. And I cannot use another algorithm instead of &lt;code&gt;interior-point&lt;/code&gt; either, since I have bound and linear constraints at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to overcome this problem and make it work for all choices of $\lambda$ and $\beta$?&lt;/p&gt;&#xA;" OwnerUserId="8299" LastEditorUserId="8299" LastEditDate="2014-05-26T11:59:44.983" LastActivityDate="2014-05-26T14:47:02.420" Title="Matlab fmincon with zero user-supplied hessian" Tags="|optimization|matlab|convex-optimization|constrained-optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11724" PostTypeId="2" ParentId="11723" CreationDate="2014-05-26T13:43:15.767" Score="2" Body="&lt;p&gt;One way you could overcome this situation is to correct the Hessian of your objective function, which should be $(\lambda + \beta)I$, and then see if you still have scaling issues.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-26T13:43:15.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11725" PostTypeId="2" ParentId="7866" CreationDate="2014-05-26T14:37:44.603" Score="2" Body="&lt;p&gt;Density functional theory optimizes the total energy with respect to the (possibly constrained) electronic density. The usual numerical procedure for this optimization is embodied by the KS method. The KS procedure will give you the optimal total energy and the ground-state (gs) density for which this optimal energy is achieved. This optimization procedure is independent of the explicit exchange-correlation (xc) density (or possibly orbital, where OEP-KS method applies) functional provided by you. Although, of course, the better the xc functional, the closer the total energy and gs density will be to the true one.(see Note added to the end)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimization procedure is done in the (in principle) infinite space of real valued functions that represent the density. However, by choosing a finite basis you constrain this search space. The more complete your basis set, the more flexibility you have to find the gs density, because the density is built from the single particle orbitals expanded in the given basis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this reasoning, it seems plausible to think that if you increase your basis set, you will have a bigger search space and the optimal gs density will be better than for a smaller basis set. This behavior would be in general the expected. However, it seems to me possible that you could add more basis functions that could increase the flexibility of the density in high energy regions, while deteriorating the description of the density in the low energy region (here I mean regions of the infinite dimensional space of densities), so one could somehow arrive at a higher energy by increasing the basis set in the wrong way. For example, think of a 1D quantum well in a real space representation of delta functions centered in a dicrete grid of points. If you increase your basis by adding many delta functions in low density regions but remove a few in places where the minimum of the potential occur, then for sure your total energy will increase, because your basis is incapable of describing accumulation of charge in the minimum of the potential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore,  the take-home message is: if your energy is increasing upon increasing your basis set, then you are adding flexibility in the wrong place and deteriorating in the right place. If this is the behavior, then try to increase your basis in a different way, until you see a lowering of the total energy.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: All above is valid for a given fixed xc (explicit) density functional. Changing the functional will change your energy in a non-variational way. I mean, a given xc functional could give a total energy lower than the true one. However, this does not mean that there is a problem in the KS minimization procedure, but that your xc functional is not a very good one.&lt;/p&gt;&#xA;" OwnerUserId="8302" LastEditorUserId="8302" LastEditDate="2014-05-26T14:54:14.773" LastActivityDate="2014-05-26T14:54:14.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11726" PostTypeId="2" ParentId="11723" CreationDate="2014-05-26T14:47:02.420" Score="1" Body="&lt;p&gt;The problem is (likely) that your gradient and Hessian are incorrect:&#xA;$$\nabla \left(\frac\lambda 2 \|\Omega\mu-x\|_2^2 \right)= \lambda(\Omega\mu-x)(-1) = \lambda(x-\Omega\mu)$$&#xA;and hence the corresponding part of the Hessian is&#xA;$$\nabla^2 \left(\frac\lambda 2 \|\Omega\mu-x\|_2^2 \right) = \lambda I.$$&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2014-05-26T14:47:02.420" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11727" PostTypeId="1" AcceptedAnswerId="11730" CreationDate="2014-05-26T15:17:55.443" Score="9" ViewCount="2691" Body="&lt;p&gt;I'm performing a line search as part of a quasi-Newton BFGS algorithm. In one step of the line search I use a cubic interpolation to move closer to the local minimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $f : R \rightarrow R, f \in C^1$ be the function of interest. I want to find an $x^*$ such that $f'(x^*) \approx 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $f(x_k)$, $f'(x_k)$, $f(x_{k+1})$ and $f'(x_{k+1})$ be known. Also assume $0\le x_k&amp;lt;x^*&amp;lt;x_{k+1}$. I fit a cubic polynomial $Q(x)=ax^3+bx^2+cx+d$ so that $Q(0)=f(x_k)$, $Q'(0)=f'(x_k)$, $Q(x_{k+1}-x_{k})=f(x_{k+1})$ and $Q'(x_{k+1}-x_{k})=f'(x_{k+1})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solve the quadratic equation: $(1): Q'(x^*-x_k) = 0$ for my sought $x^*$ using the closed form solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above works well in most cases, except when $f(x)=\mathcal{O}(x^2)$ as the the closed form solution for $(1)$ divides by $a$ which becomes very close to or exactly $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My solution is to look at $a$ and if it is &quot;too small&quot; simply take the closed form solution for the minimizer of the quadratic polynomial $Q_2(x)=bx^2+cx+d$ for which I already have the coefficients $b,c,d$ from the earlier fit to $Q(x)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is:&lt;/strong&gt; How do I devise a good test for when to take the quadratic interpolation over the cubic? The naive approach to test for $a \equiv 0$ is bad due to numerical reasons so I'm looking at $|a| &amp;lt; \epsilon\tau$ where $\epsilon$ is the machine precision, but I'm unable to decide a good $\tau$ that's scale invariant of $f$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bonus question:&lt;/strong&gt; Are there any numerical issues with using the coefficients, $b,c,d$, from the failed cubic fit or should I perform a new quadratic fit with appropriate way of calculating the coefficients?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit for clarification:&lt;/em&gt;&#xA;In my question $f$ is actually what is commonly referred to as $\phi(\alpha)=f(\bar{x}_k+\alpha \bar{p_k})$ in literature. I just simplified the question formulation. The optimization problem I'm solving is non-linear in 6 dimensions. And I'm well aware that Wolfe conditions are enough for BFGS line search hence stating that I was interested in $f'(x^*) \approx 0$; I'm looking for something that'll satisfy strong Wolfe conditions and taking the minimizer of the cubic approximation is a good step along the way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question was not about BFGS, but rather how to determine when the cubic coefficient is small enough that a quadratic approximation is more appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit 2:&lt;/em&gt; Update notation, equations are unchanged.&lt;/p&gt;&#xA;" OwnerUserId="8298" LastEditorUserId="8298" LastEditDate="2014-05-27T09:51:26.673" LastActivityDate="2016-01-28T13:41:44.860" Title="Help deciding between cubic and quadratic interpolation in line search" Tags="|optimization|numerical-analysis|interpolation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11728" PostTypeId="2" ParentId="11722" CreationDate="2014-05-26T21:14:41.160" Score="1" Body="&lt;p&gt;A truncated power series about $k=1$ is one way. You can find several forms at &lt;a href=&quot;http://functions.wolfram.com&quot; rel=&quot;nofollow&quot;&gt;functions.wolfram.com&lt;/a&gt; for &lt;a href=&quot;http://functions.wolfram.com/EllipticIntegrals/EllipticK/06/01/04/01/&quot; rel=&quot;nofollow&quot;&gt;$K(k)$&lt;/a&gt; and &lt;a href=&quot;http://functions.wolfram.com/EllipticIntegrals/EllipticE/06/01/04/01/&quot; rel=&quot;nofollow&quot;&gt;$E(k)$&lt;/a&gt;. The number of terms used should allow you to estimate your error for a given neighborhood size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab's &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ellipke.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;ellipke&lt;/code&gt;&lt;/a&gt; uses a simple arithmetic–geometric mean method (see &lt;a href=&quot;http://people.math.sfu.ca/~cbm/aands/&quot; rel=&quot;nofollow&quot;&gt;Abramowitz &amp;amp; Stegun&lt;/a&gt;) to find values for any $k$.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastActivityDate="2014-05-26T21:14:41.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11729" PostTypeId="2" ParentId="11722" CreationDate="2014-05-26T22:18:16.987" Score="0" Body="&lt;p&gt;The scipy package for Python has &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.special.ellipkm1.html#scipy.special.ellipkm1&quot; rel=&quot;nofollow&quot;&gt;ellipkm1&lt;/a&gt; &quot;The complete elliptic integral of the first kind around m=1.&quot;&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-05-26T22:18:16.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11730" PostTypeId="2" ParentId="11727" CreationDate="2014-05-27T07:50:36.293" Score="4" Body="&lt;p&gt;Hmm... cubic interpolation is not unheard of for line search, but typically overkill.&lt;/p&gt;&#xA;&lt;p&gt;If I'm reading your problem correctly, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is just a scalar? In this case BFGS is probably not the most efficient way of solving your problem. Scalar optimization algorithms like Brenth's method are likely to solve your problem faster.&lt;/p&gt;&#xA;&lt;p&gt;There are a number of line search algorithms for BFGS. For my own applications, using the memory limited BFGS (L-BFGS) &lt;a href=&quot;https://www.math.lsu.edu/%7Ehozhang/papers/cg_descent.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this linesearch&lt;/a&gt; works very well. Remember that you only need to satisfy the Wolfe conditions, and you're likely not gaining much by finding the exact minimizer.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, to actually answer your question:&#xA;I would consider simply switching to the quadratic polynomial if solving the cubic one yields &amp;quot;bad&amp;quot; values such as NaN or Inf (as is done &lt;a href=&quot;http://www.di.ens.fr/%7Emschmidt/Software/minFunc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I'm not quite sure what you mean by using &lt;span class=&quot;math-container&quot;&gt;$b,c,d$&lt;/span&gt;? These coefficients for the cubic fit will not be the same as for the quadratic fit so you can't reuse them.&lt;/p&gt;&#xA;&lt;p&gt;Lastly, you may want to use &lt;span class=&quot;math-container&quot;&gt;$f(x_{k-1})$&lt;/span&gt; , rather than &lt;span class=&quot;math-container&quot;&gt;$f(x_0)$&lt;/span&gt;, as your function will (probably) only be approximately cubic or quadratic locally, and &lt;span class=&quot;math-container&quot;&gt;$x_k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_{k-1}$&lt;/span&gt; should be closer to each other (and the solution) than &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-05-27T07:50:36.293" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11731" PostTypeId="1" CreationDate="2014-05-27T08:10:36.403" Score="4" ViewCount="78" Body="&lt;p&gt;Given a triangle mesh $\mathcal{T}$ with vertices $V=\{\mathbf{v}_i\}_{i=1}^n$ in $\mathbb{R}^3$ and triangles $T_{ijk}=[\mathbf{v}_i, \mathbf{v}_j, \mathbf{v}_k]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each vertex $\mathbf{v}_i$, I have a scalar value $f_v(\mathbf{v}_i)=U_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each edge $\mathbf{e}_j$, connecting vertices $\mathbf{v}_a$ and $\mathbf{v}_b$, I have a scalar function $f_e(\mathbf{e}_j, t) = At^2+Bt+C$. The scalars $A$, $B$ and $C$ are chosen such that for $t\in[0,1]$ the edge function $f_e$ interpolates $f_v$ in its vertices, i.e. $f_e(\mathbf{e}_j,0)=U_a$ and $f_e(\mathbf{e}_j, 1)=U_b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to find a function $f(\mathbf{p}), \mathbf{p}\in\mathcal{T}$ which interpolates all the edge functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose the simplest solution is to seek an interpolating surface locally on each triangle. The surfaces over each triangle would then join with $\mathcal{C}^0$ continuity across edges in $\mathcal{T}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to find a surface on $T_{ijk}$ which interpolates the functions on all three edges?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bonus question: Is it possible to get $\mathcal{C}^1$ continuity between patches across each edge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first asked this question at &lt;a href=&quot;https://math.stackexchange.com/questions/767156/fitting-a-surface-to-scalar-functions-given-on-the-edges-of-a-triangulation&quot;&gt;Mathematics SE&lt;/a&gt;, but got a tip to ask here instead.&lt;/p&gt;&#xA;" OwnerUserId="8306" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-05-27T16:07:55.420" Title="Fitting a surface to scalar functions given on the edges of a triangulation" Tags="|computational-geometry|interpolation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11733" PostTypeId="1" AcceptedAnswerId="11739" CreationDate="2014-05-26T21:19:32.547" Score="6" ViewCount="2819" Body="&lt;p&gt;I'm given a stream of acceleration data with timestamps. The sampling is non-uniform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from Euler, is there a way to integrate the acceleration into velocity? Something more accurate or of higher order?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can store some past acceleration data, but I'd like to output the velocity as quickly as possible, as new data continually arrives.&lt;/p&gt;&#xA;" OwnerUserId="8316" OwnerDisplayName="user1071136" LastEditorUserId="1552" LastEditDate="2014-05-27T18:23:17.280" LastActivityDate="2015-02-01T16:45:14.147" Title="Numerical integration of non-uniform acceleration samples" Tags="|time-integration|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11734" PostTypeId="2" ParentId="11733" CreationDate="2014-05-26T21:28:56.357" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_multistep_method&quot; rel=&quot;nofollow&quot;&gt;Adams-Bashforth methods&lt;/a&gt; are higher-order extensions of Euler's method, which seems to fit what you're looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your situation is a particularly simple ODE, however, you may want to consider other methods, including various methods of &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_integration&quot; rel=&quot;nofollow&quot;&gt;numerical quadrature&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8309" OwnerDisplayName="Omnomnomnom" LastActivityDate="2014-05-26T21:28:56.357" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11735" PostTypeId="2" ParentId="11712" CreationDate="2014-05-27T12:07:14.840" Score="2" Body="&lt;p&gt;SQP methods require that the objective is twice differentiable (cf &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Sequential_quadratic_programming&quot; rel=&quot;nofollow&quot;&gt;https://en.m.wikipedia.org/wiki/Sequential_quadratic_programming&lt;/a&gt;) while Augmented Lagrangians work even when the objective is nondifferentiable (hence their recent resurgence in the image processing community cf &lt;a href=&quot;ftp://arachne.math.ucla.edu/pub/camreport/cam09-05.pdf&quot; rel=&quot;nofollow&quot;&gt;ftp://arachne.math.ucla.edu/pub/camreport/cam09-05.pdf&lt;/a&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know about the galahad software, but if is supposed to solve differentiable optimization problems it will probably do much better by using a method that is allowed to differentiate the objective function. &lt;/p&gt;&#xA;" OwnerUserId="979" LastEditorUserId="1804" LastEditDate="2014-05-27T15:07:47.530" LastActivityDate="2014-05-27T15:07:47.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11736" PostTypeId="2" ParentId="11706" CreationDate="2014-05-27T14:08:00.130" Score="6" Body="&lt;p&gt;Disclaimer: I am the author of the linked modules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I would recommend to post this question at Stackoverflow, many Fortran experts meet there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more possible approaches to this problem. One similar to C void pointers (using &lt;code&gt;transfer&lt;/code&gt;) or safer one using &lt;code&gt;class(*)&lt;/code&gt;. Both theoretically allow heterogeneous lists. You are then responsible to correctly retrieve the items of different types. In the &lt;code&gt;class(*)&lt;/code&gt; case it is done safely using the &lt;code&gt;select type&lt;/code&gt; construct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there approaches using parametric polymorphism similar to C++ templates, but done using a preprocessor and &lt;code&gt;include&lt;/code&gt;. They are IMHO simpler to use, but only items of one type can be in the copy of the list. You can make a separate copy for every type you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find example modules of both at &lt;a href=&quot;https://github.com/LadaF/fortran-list&quot; rel=&quot;nofollow&quot;&gt;https://github.com/LadaF/fortran-list&lt;/a&gt;  . I do not claim it is perfect, but it suits my needs and I use it regularly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;----EDIT-----&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't like a linked list, because you require random access and you can live with more difficult appending and even more difficult inserting and deleting, use a Fortran array of this simple derives type:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;module array_list&#xA;&#xA;  type container&#xA;    class(*), allocatable :: item&#xA;    class(*), allocatable :: items(:)&#xA;  end type&#xA;&#xA;  interface add_item&#xA;    module procedure add_item_scalar&#xA;    module procedure add_item_array&#xA;  end interface&#xA;&#xA;contains&#xA;&#xA;  subroutine add_item_scalar(a, e)&#xA;    type(container),allocatable,intent(inout) :: a(:)&#xA;    class(*),intent(in) :: e&#xA;    type(container),allocatable :: tmp(:)&#xA;&#xA;      if (.not.allocated(a)) then&#xA;        allocate(a(1))&#xA;        allocate(a(1)%item, source = e)&#xA;      else&#xA;        call move_alloc(a,tmp)&#xA;        allocate(a(size(tmp)+1))&#xA;        a(1:size(tmp)) = tmp&#xA;        allocate(a(size(tmp)+1)%item, source = e)&#xA;      end if&#xA;   end subroutine&#xA;&#xA;  subroutine add_item_array(a, e)&#xA;    type(container),allocatable,intent(inout) :: a(:)&#xA;    class(*),intent(in) :: e(:)&#xA;    type(container),allocatable :: tmp(:)&#xA;&#xA;      if (.not.allocated(a)) then&#xA;        allocate(a(1))&#xA;        allocate(a(1)%items(size(e)), source = e)&#xA;      else&#xA;        call move_alloc(a,tmp)&#xA;        allocate(a(size(tmp)+1))&#xA;        a(1:size(tmp)) = tmp&#xA;        allocate(a(size(tmp)+1)%items(size(e)), source = e)&#xA;      end if&#xA;   end subroutine&#xA;&#xA;end module&#xA;&#xA;&#xA;&#xA;  use array_list&#xA;&#xA;  type(container), allocatable :: a_list(:)&#xA;&#xA;  type newtype&#xA;  end type&#xA;&#xA;  integer i&#xA;&#xA;  call add_item(a_list, 1)&#xA;&#xA;  call add_item(a_list, 5.5)&#xA;&#xA;  call add_item(a_list, (4., 5.))&#xA;&#xA;  call add_item(a_list, newtype())&#xA;&#xA;  call add_item(a_list, [1, 2, 3, 4, 5])&#xA;&#xA;  do i = 1, size(a_list)&#xA;    call print(a_list(i))&#xA;  end do&#xA;&#xA;contains&#xA;&#xA;  subroutine print(c)&#xA;    type(container), intent(in) :: c&#xA;&#xA;    if (allocated(c%item)) then&#xA;      select type (x=&amp;gt;c%item)&#xA;        type is (integer)&#xA;          print *, x&#xA;        type is (real)&#xA;          print *, x&#xA;        type is (complex)&#xA;          print *, x&#xA;        type is (newtype)&#xA;          print *, &quot;is newtype&quot;&#xA;        class default&#xA;          print *, &quot;is unknown type&quot;&#xA;      end select&#xA;    else if (allocated(c%items)) then&#xA;      select type (x=&amp;gt;c%items)&#xA;        type is (integer)&#xA;          print *, x&#xA;        type is (real)&#xA;          print *, x&#xA;        type is (complex)&#xA;          print *, x&#xA;        type is (newtype)&#xA;          print *, &quot;is array of newtype&quot;&#xA;        class default&#xA;          print *, &quot;is array of unknown type&quot;&#xA;      end select&#xA;    else&#xA;      write(*,*) &quot;Error, empty container&quot;&#xA;    end if&#xA;  end subroutine&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;compile and run&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sunf90 alist.f90 &#xA;&amp;gt; ./a.out &#xA; 1&#xA; 5.5&#xA; (4.0,5.0)&#xA; is newtype&#xA; 1 2 3 4 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1244" LastEditorUserId="1244" LastEditDate="2014-05-28T08:14:06.363" LastActivityDate="2014-05-28T08:14:06.363" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11737" PostTypeId="2" ParentId="11733" CreationDate="2014-05-27T14:08:28.743" Score="2" Body="&lt;p&gt;Since you have all the data points already, I agree with @Omnomnomnom that you might want to try numerical quadrature here - a simple trapezoidal rule will be second-order accurate as compared to Euler, which is first-order. &lt;em&gt;(This is equivalent to using Euler for integration, but using the average acceleration between two times as your acceleration value for the step, which should also be 2nd order.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest higher order approach that comes to mind is to fit some curve to it (e.g. a cubic spline) and that gives you an approximating polynomial that can be integrated &quot;exactly&quot; by quadrature or other means, or you can use any kind of ODE solver like an adaptive Runge-Kutta method. Doing this should be 2 or 3 lines of matlab or numpy so it's a good first effort to see if it's sufficiently accurate, or if you need to use higher order interpolations.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2014-05-27T16:49:13.500" LastActivityDate="2014-05-27T16:49:13.500" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11738" PostTypeId="2" ParentId="11701" CreationDate="2014-05-27T14:28:04.810" Score="3" Body="&lt;p&gt;This question better suits to StackOverflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;code&gt;class(*)&lt;/code&gt; or &lt;code&gt;type(c_ptr)&lt;/code&gt;. Both come from Fortran 2003. The latter is widely supported, the former only since gfortran 4.8 and other recent versions of other compilers. Note &lt;code&gt;type(c_ptr)&lt;/code&gt; is very useful even inside Fortran, without calling any C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first approach:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine RK4(normal arguments, vararg_in)&#xA;  class(*), intent(in) :: vararg_in(:)  !for scalar delete (:)&#xA;&#xA;  !if you need to preserve a copy (why?)&#xA;  allocate(vararg_copy, source=varrarg_in)&#xA;&#xA;  call RHS(...,vararg_copy) !or directly vararg_in&#xA;end subroutine&#xA;&#xA;&#xA;subroutine DGL(...,vararg_in)&#xA;  class(*), intent(in) :: vararg_in(:)  !for scalar delete (:)&#xA;&#xA;  select type (vararg_in)&#xA;    select type (expected_type)&#xA;     !do something&#xA;    class default&#xA;     !some exception&#xA;  end select&#xA;end subroutine&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can pass anything to &lt;code&gt;class(*)&lt;/code&gt;, possibly a derived type, if you need variables of differing types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other approach:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine RK4(normal arguments, vararg_in)&#xA;  use iso_c_binding&#xA;  type(c_ptr), intent(in) :: vararg_in_ptr&#xA;&#xA;  !Doing a copy of the pointer doesn't make much sense&#xA;&#xA;  call RHS(...,vararg_in_ptr)&#xA;end subroutine&#xA;&#xA;&#xA;subroutine DGL(...,vararg_in_ptr)&#xA;  use iso_c_binding&#xA;  type(c_ptr), intent(in) :: vararg_in_ptr&#xA;  type(expected_type), pointer :: vararg_in(:) !for scalar delete (:)&#xA;&#xA;  !for an array&#xA;  call c_f_pointer(vararg_in_ptr, vararg_in, expected_shape)&#xA;  !for a scalar just&#xA;  call c_f_pointer(vararg_in_ptr, vararg_in)&#xA;&#xA;  !do something&#xA;end subroutine&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1244" LastActivityDate="2014-05-27T14:28:04.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11739" PostTypeId="2" ParentId="11733" CreationDate="2014-05-27T14:28:27.497" Score="7" Body="&lt;p&gt;I think it may be more useful to think of this is as numerical integration of a series of data points rather than as the solution of an ODE. Adams-Bashforth could work as suggested by @Omnomnomnom, but I think there are better methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that your acceleration values are &quot;given&quot; meaning that you have no control over the time step or the times at which the acceleration is known. In contrast to solving ODEs, this also means that you do not need to project forward in time to get future velocities to enable the next computation, you can simply wait until you have the new acceleration value to perform the next computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this context, there are several basic integration methods that would work, but I think the simplest will be based on directly integrating piecewise polynomial approximations of your acceleration values. The simplest example of this is the well-known &lt;a href=&quot;http://en.wikipedia.org/wiki/Trapezoidal_rule&quot;&gt;Trapezoid Rule&lt;/a&gt; which is just integration of a piecewise linear function and gives second order accuracy. If you want higher-order approximations of the data, you can use higher-order polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's possible to delay computing $v(t_i)$ by one timestep, I would suggest waiting until you know $\{a(t_{i-2}),a(t_{i-1}),a(t_{i}),a(t_{i+1})\}$ and then using a cubic polynomial on these four data points to integrate from $t_{i-1}$ to $t_i$. Using Lagrange interpolating polynomials, this can be done as follows:&#xA;$$&#xA;v(t_i) = v(t_{i-1}) + \int_{t_{i-1}}^{t_i}a(t)dt&#xA;$$&#xA;where you can approximate $a(t)$ as&#xA;\begin{align}&#xA;a(t) = &amp;amp; a(t_{i-2})\dfrac{(t-t_{i-1})(t-t_{i})(t-t_{i+1})}{(t_{i-2}-t_{i-1})(t_{i-2}-t_{i})(t_{i-2}-t_{i+1})} \\&#xA; &amp;amp; + a(t_{i-1})\dfrac{(t-t_{i-2})(t-t_{i})(t-t_{i+1})}{(t_{i-1}-t_{i-2})(t_{i-1}-t_{i})(t_{i-1}-t_{i+1})} \\&#xA; &amp;amp; + a(t_{i-2})\dfrac{(t-t_{i-2})(t-t_{i-1})(t-t_{i+1})}{(t_{i}-t_{i-2})(t_{i}-t_{i-1})(t_{i}-t_{i+1})} \\&#xA; &amp;amp; + a(t_{i+1})\dfrac{(t-t_{i-2})(t-t_{i-1})(t-t_{i})}{(t_{i+1}-t_{i-2})(t_{i+1}-t_{i-1})(t_{i+1}-t_{i})} &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then either integrate this function analytically or use a quadrature formula. Two-point Gauss-Legendre quadrature is exact for third order polynomials so that would work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;br&gt;&#xA;Depending on the language you're working with, there are likely available tools for polynomial fitting, e.g. MATLAB's &lt;code&gt;polyfit&lt;/code&gt; that would make this implementation very easy.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get the polynomial coefficients $y = at^3+bt^22+ct+d$&lt;/li&gt;&#xA;&lt;li&gt;Use the definite integral, $\int_{t_1}^{t_2}y(t)dt=\left.\left(\frac{a}{4}t^4+\frac{b}{3}t^3+\frac{c}{2}t^2+dt\right)\right|_{t_1}^{t_2}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-05-27T18:17:29.123" LastActivityDate="2014-05-27T18:17:29.123" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="11741" PostTypeId="2" ParentId="11731" CreationDate="2014-05-27T15:10:06.133" Score="1" Body="&lt;p&gt;You could do this kind of stuff with finite elements. Maybe you're familiar with the linear scalar nodal interpolation functions, you can construct your interpolant from a higher order variant. For C0 continuity and quadratic completeness there should be 6 basis functions per triangle, one at each vertex and one at each midpoint. The vertex functions are of the form $\lambda_0\cdot(1-2\lambda_0)$ and $\lambda_1\cdot(1-2\lambda_1)$ and $\lambda_2\cdot(1-2\lambda_2)$, while the midpoint functions are of the form $4\cdot\lambda_0\cdot\lambda_1$ and $4\cdot\lambda_1\cdot\lambda_2$ and $4\cdot\lambda_2\cdot\lambda_0$. (For all of these, $\lambda_0/\lambda_1/\lambda_2$ denote barycentric coordinates). Basically you would just sample your given data at the vertices/midpoints, and those would be the weights/&quot;unknowns&quot; used to scale each basis function. Sum the weighted functions to form your interpolant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For higher continuity, I would steer you towards Hermite elements - unfortunately I have no implementation experience with them.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2014-05-27T15:10:06.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11742" PostTypeId="1" AcceptedAnswerId="11746" CreationDate="2014-05-27T15:37:01.223" Score="3" ViewCount="3824" Body="&lt;p&gt;Let $\mathbf{1}\in\mathbb{R}^d$ be a vector with all elements equal to $1$. Define:&#xA;$$\mathbf{D} = \mathrm{diag}(\mathbf{1}^\top,\mathbf{1}^\top,\ldots,\mathbf{1}^\top) = &#xA;\begin{bmatrix}&#xA;1 \cdots 1  &amp;amp;  &amp;amp;  \\&#xA; &amp;amp; 1\cdots 1 &amp;amp;  \\&#xA;&amp;amp; &amp;amp;  1\cdots 1 &#xA;\end{bmatrix} \in\mathbb{R}^{d\times d^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to compute $\mathbf{D}\cdot \mathbf{x}$ for any vector $\mathbf{x}\in\mathbb{R}^{d^2}$. Could anybody suggest me a way to efficiently compute this product?  (Since $\mathbf{D}$ has a very special structure, I guess there should be such a way).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2014-05-27T22:08:32.697" Title="Fast vector - &quot;diagonal&quot; matrix multiplication" Tags="|linear-algebra|matrix|lapack|blas|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11743" PostTypeId="2" ParentId="11742" CreationDate="2014-05-27T15:57:05.220" Score="2" Body="&lt;p&gt;This is equivalent to computing sums of consecutive contiguous subvectors of $\mathbf{x}$. You won't do much better than simple hand-coded nested loops if you have an automatically vectorizing compiler since you will probably be memory bandwidth limited for large $d$.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-05-27T15:57:05.220" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11744" PostTypeId="2" ParentId="11731" CreationDate="2014-05-27T16:07:55.420" Score="1" Body="&lt;p&gt;This can be accomplished by using a &quot;total degree surface&quot;, see &lt;a href=&quot;http://graphics.stanford.edu/courses/cs348a-12-winter/Handouts/handout26.pdf&quot; rel=&quot;nofollow&quot;&gt;this reference&lt;/a&gt; (you will also need to look over the rest of the &lt;a href=&quot;http://graphics.stanford.edu/courses/cs348a-12-winter&quot; rel=&quot;nofollow&quot;&gt;course notes&lt;/a&gt;). Note that a piecewise linear surface is defined uniquely by your $U_i$, giving you $C^0$ continuity. The piecewise linear surface interpolation requires only values local to a triangle. To achieve $C^1$ continuity, you need to use a piecewise quadratic surface, which will require information over the neighborhood of an edge. Note that a piecewise quadratic surface is specified by more information than just $U_i$. Since you have the values at the midpoints of the edges, that should be enough. In any case, the right way to think about the problem is in terms of spline surfaces.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-05-27T16:07:55.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11745" PostTypeId="2" ParentId="11742" CreationDate="2014-05-27T18:17:47.663" Score="2" Body="&lt;p&gt;Edit: removed confusing code&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Yes there is a shortcut, here's some Python code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&#xA;&#xA;import numpy as np&#xA;&#xA;d = 3&#xA;&#xA;&lt;del&gt;&#xA;D = np.repeat(np.identity(d), d, axis=1)&#xA;print('D:')&#xA;print D&#xA;print&#xA;&lt;/del&gt;&#xA;&#xA;print('x:')&#xA;x = np.arange(d*d, dtype=float)&#xA;print x&#xA;print&#xA;&#xA;&lt;del&gt;&#xA;print('D * x:')&#xA;print D.dot(x)&#xA;print&#xA;&lt;/del&gt;&#xA;&#xA;print('shortcut:')&#xA;print x.reshape((d, d)).sum(axis=1)&#xA;print&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;&lt;del&gt;&#xA;D:&#xA;[[ 1.  1.  1.  0.  0.  0.  0.  0.  0.]&#xA; [ 0.  0.  0.  1.  1.  1.  0.  0.  0.]&#xA; [ 0.  0.  0.  0.  0.  0.  1.  1.  1.]]&#xA;&lt;/del&gt;&#xA;&#xA;x:&#xA;[ 0.  1.  2.  3.  4.  5.  6.  7.  8.]&#xA;&#xA;&lt;del&gt;&#xA;D * x:&#xA;[  3.  12.  21.]&#xA;&lt;/del&gt;&#xA;&#xA;shortcut:&#xA;[  3.  12.  21.]&#xA;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2014-05-27T19:08:05.970" LastActivityDate="2014-05-27T19:08:05.970" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11746" PostTypeId="2" ParentId="11742" CreationDate="2014-05-27T20:54:11.257" Score="10" Body="&lt;p&gt;This can be interpreted as summing over an index of a tensor when the vector $x$ is reshaped into a box of numbers instead of a list. In particular, if $X$ is the $d\text{-by-}d$ folded version of $x$, then the operation you are doing is,&#xA;\begin{align}&#xA;Dx &amp;amp;= \mathrm{vec}\left((I \otimes \mathbf{1})\mathrm{vec}(X)\right) \\&#xA;&amp;amp;= \mathrm{vec}(\mathbf{1}^T X I) \\&#xA;&amp;amp;= \mathrm{vec}(\mathbf{1}^T X).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matlab code to do this is surprisingly simple. It is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sum(reshape(x,d,d))'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's an example of it in action - you can see that it far outperforms the standard dense multiply, sparse matrix multiply, and for loop versions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; onesmatrixquestion&#xA;dense matrix multiply&#xA;Elapsed time is 0.000873 seconds.&#xA;sparse matrix multiply&#xA;Elapsed time is 0.000115 seconds.&#xA;for loop version&#xA;Elapsed time is 0.000154 seconds.&#xA;tensorized version&#xA;Elapsed time is 0.000018 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here's the code that generated those timing results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%onesmatrixquestion.m&#xA;d = 100;&#xA;onevec = ones(1,d);&#xA;D = kron(eye(d,d),onevec);&#xA;Dsparse = sparse(D);&#xA;&#xA;x = randn(d^2,1);&#xA;&#xA;disp('dense matrix multiply')&#xA;tic&#xA;aa = D*x;&#xA;toc&#xA;&#xA;disp('sparse matrix multiply')&#xA;tic&#xA;bb = Dsparse*x;&#xA;toc&#xA;&#xA;disp('for loop version')&#xA;tic&#xA;cc = zeros(d,1);&#xA;ind = 1;&#xA;for kk=1:d&#xA;    for jj=1:d&#xA;        cc(kk) = cc(kk) + x(ind);&#xA;        ind = ind + 1;&#xA;    end&#xA;end&#xA;toc&#xA;&#xA;disp('tensorized version')&#xA;tic&#xA;dd = sum(reshape(x,d,d))';&#xA;toc&#xA;&#xA;if (norm(aa - bb) &amp;gt; 1e-9 || norm(bb - cc) &amp;gt; 1e-9 ...&#xA;        || norm(cc - dd) &amp;gt; 1e-9)&#xA;    disp('error: different methods give different results')&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2014-05-27T22:08:32.697" LastActivityDate="2014-05-27T22:08:32.697" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11747" PostTypeId="2" ParentId="11712" CreationDate="2014-05-27T21:54:13.170" Score="4" Body="&lt;p&gt;In terms of outer iterations, SQP should win because it includes second derivative information, whereas augmented lagrangian methods such as ADMM do not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one thing to keep in mind is that each iteration for these methods involves solving a linear system, so to do a fair comparison you have to take into account how easy these systems are to solve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For augmented lagrangian (alternating) methods, each iteration you are solving something like,&#xA;$$(A^TA + \rho I)x = b,$$&#xA;where $A$ is a forward operator straight from the objective function that is known and usually easier to deal with or precondition, and $\rho$ is the penalty parameter. (eg, your problem is $\min_x ||Ax-b||^2$ subject to some regularization and constraints).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For SQP methods you are solving something like&#xA;$$Hx = g,$$&#xA;where $H$ is the Hessian (or approximation thereof) which is usually only available implicitly in terms of it's action on vectors, and $g$ is the gradient. The Hessian contains not just $A$, but also a combination of other matrices and matrix inverses coming from linearizing the constraints and regularization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Preconditioning Hessians is a pretty tricky business and is much less studied than preconditioning forward problems. A standard method is to approximate the Hessian inverse with L-BFGS, but this is of limited effectiveness when the Hessian inverse is high-rank. Another popular method is to approximate the Hessian as a sum of a low-rank matrix plus an easy to invert matrix, but this also has limited effectiveness for hard problems. Other popular Hessian estimation techniques are based on sparse approximations, but continuum problems often have Hessians that have poor sparse approximations.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2014-05-27T21:54:13.170" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11748" PostTypeId="2" ParentId="11718" CreationDate="2014-05-28T01:41:07.947" Score="1" Body="&lt;p&gt;Doug Lipisnki demonstrated the power of symbolic calculations (which I use quite often), but I would like to offer a purely numeric approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a system $A x = b$ where &lt;em&gt;A&lt;/em&gt;:[6×22], &lt;em&gt;b&lt;/em&gt;:[6×1] and &lt;em&gt;x&lt;/em&gt;:[22×1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to solve for 6 unknowns $x_U=(x1,x2,x3,..)$ so we split the vector $x$ into known's and unkown's using the orthogonal projection matrices $K$ and $U$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x = K x_K + U x_U $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The structure of $K$ is [22×16] with a single 1 on each column ,on the row representing the known $x$ value. $U$ is [22×6] with a single 1 on each column at the row representing the unkown $x$ value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if $x=(x1,x2,x3)$ had 3 values only, with $x2$ unknown, then $$ K = \begin{bmatrix} 1 &amp;amp; 0 \\ 0 &amp;amp; 0 \\ 0 &amp;amp; 1 \end{bmatrix} \;\;\;U = \begin{bmatrix} 0 \\ 1 \\  0 \end{bmatrix}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve for the unknowns we have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left. A K x_K + A U x_U = b \right\} x_U = (A U)^{-1}(b - A K x_K) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end you get to solve the following linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{bmatrix}&#xA;-k_{2a} &amp;amp; C_b &amp;amp; \omega-\omega_a &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;C_a &amp;amp;-k_{2b} &amp;amp; 0 &amp;amp; \omega-\omega_b &amp;amp; 0 &amp;amp; 0 \\&#xA;\omega_a-\omega &amp;amp; 0 &amp;amp; -k_{2a}&amp;amp; C_b &amp;amp;-\omega_1 \\&#xA;0&amp;amp; \omega_b-\omega &amp;amp; C_a &amp;amp; -k_{2b} &amp;amp; 0 &amp;amp; -\omega_1 \\&#xA;0 &amp;amp; 0 &amp;amp; \omega_1 &amp;amp; 0 &amp;amp; -k_{1a} &amp;amp; C_b \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \omega_1 &amp;amp; C_a &amp;amp; -k_{1b} \end{bmatrix} &#xA;\begin{pmatrix} M_x^a \\ M_x^b \\ M_y^a \\ M_y^b\\M_z^a \\M_z^b \end{pmatrix}&#xA; = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ - \frac{M_0^a}{T_{1a}} \\ - \frac{M_0^b}{T_{1b}} \end{pmatrix} $$&lt;/p&gt;&#xA;" OwnerUserId="1030" LastEditorUserId="7733" LastEditDate="2014-05-28T16:03:25.263" LastActivityDate="2014-05-28T16:03:25.263" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11749" PostTypeId="2" ParentId="11673" CreationDate="2014-05-28T01:54:40.070" Score="1" Body="&lt;p&gt;I want to point out that your equation probably do not have a unique solution in 2nd type B.C. because if $u$ is a solution then $u+C$ is also a solution for your equation where $C$ is an arbitrary constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically, the 2nd type B.C. usually be approximated by Euler's method. For instance, in your problem, you can set $u_{N}^m=u_{N+1}^m$ where $m$ is time step and $N+1$ is partition point at boundary of $x$. You can modify your code my this method.&lt;/p&gt;&#xA;" OwnerUserId="7410" LastActivityDate="2014-05-28T01:54:40.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11750" PostTypeId="1" AcceptedAnswerId="11758" CreationDate="2014-05-28T11:28:43.887" Score="0" ViewCount="187" Body="&lt;p&gt;I want to simulate tidal features in a large body of water using was smooth particle hydrodynamics.  For a system composed of an incompressible fluid, what general scale of problem can be tackled with an home computer? That is, how large of a body of water can I accurately represent using Smooth Particle Hydrodynamics?&lt;/p&gt;&#xA;" OwnerUserId="3939" LastEditorUserId="549" LastEditDate="2014-05-29T14:14:04.330" LastActivityDate="2014-05-30T01:35:22.020" Title="What scale problem can be simulated on an PC using Smooth Particle Hydrodynamics?" Tags="|fluid-dynamics|simulation|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11751" PostTypeId="2" ParentId="11750" CreationDate="2014-05-28T13:35:12.967" Score="4" Body="&lt;p&gt;This depends a great deal on the type of CFD problem and the methods being used.  But as a rough start using a lightweight geometric multigrid for the pressure solve and explicit time stepping for velocity, suppose you have 100 bytes per cell.  If you have 8 GB of memory, you could solve a problem with about 80 million cells.  This will likely be slower than you want because explicit CFD typically needs many time steps, in which case you should move to a smaller problem size.  More storage will be needed if you use assembled matrices or Krylov methods (which may or may not make the code run faster).&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-05-28T13:35:12.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11752" PostTypeId="1" AcceptedAnswerId="11774" CreationDate="2014-05-28T14:24:22.863" Score="8" ViewCount="219" Body="&lt;p&gt;I've been looking through the other answers, and I haven't found a good answer yet.  I'll describe the simulation I am running, and then the problems.  The program simulates particles undergoing Brownian dynamic motion (thermal motion) alternating with stochastic processes (kinetic Monte-Carlo).  When I move to OpenMP code, I gain performance (finally), but I lose the determinism in the system.  Therefore, I have two problems.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I can only repeat the same 'experiment' twice using a single threaded version.  Otherwise, the order in which the threads operate on the particles destroys any determinism.&lt;/li&gt;&#xA;&lt;li&gt;Profiling how much faster the OpenMP code runs as a function of threads, or versus the serial version, requires running 10x sims at every point since I test for a particular ending configuration, and the speed of the program is very sensitive to the configuration of the particles. Basically, the closer more particles are, the longer it takes to run, so different ending configurations can take drastically different amounts of time.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I know that this is a normal problem found in MD, but I am seeing if there are any ways around it rather than running the single threaded version.  If that is the case, then so be it, I was just hoping for something else mostly so that I could optimize the OpenMP code and environment variables.&lt;/p&gt;&#xA;" OwnerUserId="4545" LastEditorUserId="20688" LastEditDate="2019-07-02T00:50:02.460" LastActivityDate="2019-07-02T00:50:02.460" Title="Removing non-determinism from molecular dynamics code" Tags="|parallel-computing|simulation|hpc|molecular-dynamics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11753" PostTypeId="1" AcceptedAnswerId="11757" CreationDate="2014-05-28T15:51:54.040" Score="0" ViewCount="200" Body="&lt;p&gt;It is necessary for me to produce triangular mesh on a sphere membrane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use  MATLAB pdetool or delaunay for produce mesh. &#xA;How could I do this?&lt;/p&gt;&#xA;" OwnerUserId="8271" LastActivityDate="2014-05-29T00:24:15.960" Title="how could I produce triangular mesh for membrane" Tags="|mesh-generation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11754" PostTypeId="2" ParentId="11752" CreationDate="2014-05-28T16:39:45.910" Score="1" Body="&lt;p&gt;Do you know where exactly the non-determinism is stemming from? I don't do MD, but I can think of two scenarios:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The particle updates (time stepping) are using the most recently available particle data, so another OpenMP thread may have already updated particle that wasn't updated in the serial version. Fixing this would require all threads to only update based on the previous time step's values. (I would think this would be required anyway to maintain time accuracy though.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You mentioned &quot;stochastic processes.&quot; How are you seeding the random number generator used here? If you force all threads to use the same seed, that should remove any non-determinism from the RNG.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-05-28T16:39:45.910" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11755" PostTypeId="1" CreationDate="2014-05-28T17:56:59.467" Score="1" ViewCount="127" Body="&lt;p&gt;Not really much more to say;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of several computers available for use, these are two of the larger ones; one has 2x &lt;a href=&quot;http://ark.intel.com/products/64595/&quot; rel=&quot;nofollow&quot;&gt;E5-2670&lt;/a&gt;, and the other has 4x &lt;a href=&quot;http://ark.intel.com/products/64603/Intel-Xeon-Processor-E5-4640-20M-Cache-2_40-GHz-8_00-GTs-Intel-QPI&quot; rel=&quot;nofollow&quot;&gt;E5-4640&lt;/a&gt;. The problems we're looking at essentially boil down to large matrix diagonalisation problems (i.e. i have been dealing with 125K rank matrices, and in the past have looked at larger ones. Both sparse and non sparse, using &lt;a href=&quot;http://www.caam.rice.edu/~kristyn/parpack_home.html&quot; rel=&quot;nofollow&quot;&gt;PARPACK&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All intuition would tell me that the Quad CPU 4640 should be much faster, but it's not, and I can't work out why at all. Both have hyperthreading enabled, both have 250GB ram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could it be that the gains from the parallelisation of PARPACK beyond 16 cores just start to diminish, and are not worth adding the additional processors due to the increase in inter cpu communcation required?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The codes are written in fortran if that makes much difference.&lt;/p&gt;&#xA;" OwnerUserId="8323" LastActivityDate="2014-05-28T17:56:59.467" Title="Why would this a pair of E5-2670 cpus be faster than 4 E5-2640 cpus for large matrix diagonalisation problems?" Tags="|parallel-computing|fortran|mpi|lapack|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="11756" PostTypeId="2" ParentId="11750" CreationDate="2014-05-28T19:55:30.783" Score="1" Body="&lt;p&gt;Wow what an open question to ask!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already mentioned the answer depends on so many factors. Here are just a few: Type of method (i.e. finite difference, spectral, finite volume, finite element etc), the power of your home computer, your skill as a programmer, the language you use, the algorithms you use for solving linear systems, explicit verses implicit time stepping, any parallelization you are able to exploit, the time you are willing to wait, how many time steps you need....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume by &quot;general scale&quot; you are talking about what grid sizes can be solved on a home PC. For 2D problems as a rough estimate I would say O(1024^2) is definitely do-able for many problems with maybe around 100000 time steps.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2014-05-28T20:02:42.300" LastActivityDate="2014-05-28T20:02:42.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11757" PostTypeId="2" ParentId="11753" CreationDate="2014-05-29T00:24:15.960" Score="1" Body="&lt;p&gt;My favorite free mesh generator is &quot;gmsh&quot;. It allows you to create geometry through a variety of means and can produce high-quality 2D and 3D meshes. Gmsh will actually triangulate the spherical shell, which will produce better results than simply triangulating a circle and projecting it up to a hemisphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the site where you can find more info and tutorials. &lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;nofollow&quot;&gt;http://geuz.org/gmsh/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2014-05-29T00:24:15.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11758" PostTypeId="2" ParentId="11750" CreationDate="2014-05-29T03:40:02.067" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;All models are wrong, but some models are useful - George E. P. Box&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let's establish one important fact:  Simulation Models are &lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt; exact replicas of reality.  We use models to understand only a small piece of the real physics that is actually happening.  No model, no matter how good the equations are or how powerful the computer is, can exactly mimic reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Models can be &lt;strong&gt;&lt;em&gt;useful&lt;/em&gt;&lt;/strong&gt; to understand certain physics, whether at the atomic scale (ab-initio models), mesoscopic scale (molecular dynamics or lattice boltzmann models), or macroscopic scale (navier stokes equations).  Depending on which scale you choose (among other factors), you can make certain assumptions to model a glass of water or a lake (or even an ocean).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The word 'scale' (as the OP is using it) does not translate directly into a &lt;em&gt;computational load&lt;/em&gt; directly.  As others have already pointed out, the computational load depends on the total number of the resulting degrees of freedom after &lt;em&gt;discretization&lt;/em&gt;.  Any 'scale' can be simulated on a desktop computer provided that the total number of resulting &quot;degrees of freedom&quot; is small enough to fit in the computer's memory.  Scale is independent of degrees of freedom.  &lt;strong&gt;&lt;em&gt;Accuracy&lt;/em&gt;&lt;/strong&gt;, on the other hand, is highly dependent on the number of degrees of freedom, the type of problem, and the solver being used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key questions to ask are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Which 'physics' are most important for this particular problem?  &lt;/li&gt;&#xA;&lt;li&gt;How much accuracy is needed?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Neither of these two questions is easy to generalize for any problem, no matter what the scale is.  Therefore, it's really not possible to answer your question in any kind of general sense.  It really depends on the kind of problem you want to model and how much accuracy you really need.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-05-29T03:59:12.990" LastActivityDate="2014-05-29T03:59:12.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11759" PostTypeId="2" ParentId="11752" CreationDate="2014-05-29T09:21:01.303" Score="2" Body="&lt;p&gt;I believe that it close to impossible to get binary reproducibility (or &quot;determinsm&quot;) in a stochastic MD simulation. Such simulations are fundamentally chaotic - even when the least significant bit of any variable (e.g. any particle position) in two runs of a simulation is different, the trajectories will diverge after some time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical source of such an error is the summation order of the forces in the main MD loop: when you sum up the forces in a different order, this might already lead to differences in the least significant bits of the sum. To get full determinism, you would have to make sure that the forces are always summed up in the same order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, it is pretty much impossible to ensure the order of summation. When the compiler optimizes code, one of the first things it does is to change the order. This means that when you compile the same code with different compilers (or maybe even different versions of the same compiler, or different optimization levels), you can not expect the results to be identical on a binary level. Furthermore, some modern architectures do instruction reordering even on the hardware level, or they provide faster instructions where the least significant bit is undetermined. So even when you execute the same binary on the same machine, it is not necessarily guaranteed that both runs create the same result. This problem is even enhanced when you parallelize the code in any efficient way, as this will for sure modify the summation order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this means that even when you manage to write a program that creates binary reproducible results on your machine, you cannot expect that anybody else will be able to reproduce your results unless he has the same version of your compiler, the same version of your OS and the same hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, I have to say that for MD, we simply cannot gain binary reproducibility. On the other hand, the results from an MD usually are statistical mean values anyway. So, what we should aim for when doing MD simulations is &lt;em&gt;statistical reproducibility&lt;/em&gt; rather than binary reproducibility. &lt;/p&gt;&#xA;" OwnerUserId="1568" LastActivityDate="2014-05-29T09:21:01.303" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11761" PostTypeId="2" ParentId="2187" CreationDate="2014-05-29T09:59:18.597" Score="0" Body="&lt;p&gt;I would like to add to the answer of Hristo Iliev. While his post mostly talks about the &lt;em&gt;computational complexity&lt;/em&gt;, when it comes to parallelization, the &lt;em&gt;communication complexity&lt;/em&gt; is at least as important - and that it the main reason for domain decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern parallel machines usually have some kind of a torus topology. This means that each CPU has a number of &quot;neighboring&quot; CPUs that it can communicate to very quickly. Communicating to a CPU that is not a neighbor is more costly.&#xA;Therefore, it is always favorable to have an algorithm that only needs to communicate with neighboring CPUs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using a particle decomposition, the interaction partners of a particle are randomly distributed on all other CPUs. To be able to compute the interactions, it needs to know the coordinates of all partners, so it needs to communicate with &lt;em&gt;all other&lt;/em&gt; CPUs. Effectively, this means that when you have $P$ CPUs, $\mathcal{O}(P^2)$ communication steps are required. Furthermore, a lot of the  communication is with non-neighboring CPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, in a domain decomposition scheme, all interaction partners live on a neighboring CPU. This means that it needs to communicate only with its nearest neighbors to get the updated information, i.e. $P$ CPUs require $\mathcal{O}(P)$ communication steps. Furthermore, all of the communication is with neighboring CPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, when the system is non-uniformly distributed, this scheme doesn't work as optimal. When that happens, this either means that some CPUs have significantly more work than others, or that one has to dynamically adapt the domains. This might cause a mismatch between the domain topology and the network topology (large domains have more neighbors). Still, the communication complexity of $\mathcal{O}(P)$ still holds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, however, that non-uniform systems are not as common as it may sound, they only occur when either simulating something in vacuum, or when using an implicit solvent. The densities of crystals and liquids are close enough to run domain decomposition.&lt;/p&gt;&#xA;" OwnerUserId="1568" LastActivityDate="2014-05-29T09:59:18.597" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11762" PostTypeId="1" CreationDate="2014-05-29T10:47:39.900" Score="1" ViewCount="916" Body="&lt;p&gt;Essentially this is the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\hat{F}(\omega) = \int_0^{\infty} f(s)e^{-i\omega s}ds$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $f$ is in general complex valued. I know this looks like the fourier transform but I don't want to view it that way. So in my implementation $f(s)$ and $\omega$ are going to be numpy vectors representing the sampled values of the true function or range. For my purposes I want to evaluate the integral for&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;omega = np.linspace(-30,30,0.1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i.e. $-30\le\omega \le30$ with spacing of $0.1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone suggest a python implementation for this? I have done some quick research and some sources suggest using 'Gauss Laguerre' integration routine. I'm not sure why, but if someone could motivate this I would be grateful. &lt;/p&gt;&#xA;" OwnerUserId="7864" LastActivityDate="2014-05-29T12:25:09.760" Title="How do I integrate this function in python?" Tags="|python|interpolation|quadrature|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11763" PostTypeId="2" ParentId="11762" CreationDate="2014-05-29T12:25:09.760" Score="4" Body="&lt;p&gt;Even if you don't want to interpret the results as a Fourier transform, can't you make use of the builtin FT routines in numpy as an efficient way to evaluate the integral?&lt;/p&gt;&#xA;" OwnerUserId="8328" LastActivityDate="2014-05-29T12:25:09.760" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11764" PostTypeId="1" AcceptedAnswerId="11765" CreationDate="2014-05-29T12:59:13.407" Score="2" ViewCount="1048" Body="&lt;p&gt;We all know the problem that computation time explodes when simulating systems with big matrices. I got just this problem, but I have the advantage that I know that my matrices are symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is thus: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know a way to decrease computation time if I know that my matrices are symmetry?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want in particular are the follwing two things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A representation for my symmetric matrices which uses only the relevant entries&lt;/li&gt;&#xA;&lt;li&gt;An algorithm that lets me efficiently multiply two matrices in this representation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8329" LastActivityDate="2014-05-30T19:24:24.240" Title="How to decrease computation time for symmetric matrices?" Tags="|matrix|complexity|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11765" PostTypeId="2" ParentId="11764" CreationDate="2014-05-29T14:16:55.030" Score="3" Body="&lt;p&gt;Obviously if your matrices are sparse you should move to a sparse matrix implementation. There are packages available that take advantage of symmetry too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Storage:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that your matrices are dense, unless your matrices have a more detailed (known) structure, you won't be able to do better than the factor of 2 savings you get from storing only the lower (or upper) triangular part. Depending on the language you're using, you could do this with a simple array of pointers to vectors representing the first part of each row. For example, in C you could create such a representation as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;unsigned int i=0; //loop index&#xA;unsigned int n=5; //matrix size, nxn&#xA;float *M[n]; //array of pointers to floats&#xA;&#xA;for (i=1; i&amp;lt;=n; i++) {&#xA;    M[i] = malloc( i * sizeof(float) );&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The main drawback of this (and other) sparse representation is increased access time for an arbitrary element in the array. Only you can say whether the trade off between storage and access time is worth it for your application. In the case where you're storing the lower triangular part of the matrix, if you want to access &lt;code&gt;M[i][j]&lt;/code&gt; you need to first check if &lt;code&gt;j&amp;lt;=i&lt;/code&gt;. You could use a helper function such as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float Mval(M,i,j) {&#xA;    if (j&amp;lt;=i) {&#xA;        return M[i][j];&#xA;    }&#xA;    else {&#xA;        return M[j][i];&#xA;    };&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Matrix multiplications:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the result of multiplying symmetric matrices is not symmetric unless the matrices commute under multiplication. This means that you will have to perform the full matrix-matrix multiplication as usual and your result will not be symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're ok with approximate results or you know more about the structure of your matrices you may be able to do better. You could search for randomized linear algebra for one potential approximate option.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-05-29T16:15:04.123" LastActivityDate="2014-05-29T16:15:04.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11766" PostTypeId="1" CreationDate="2014-05-29T14:20:46.647" Score="2" ViewCount="1271" Body="&lt;p&gt;First we have Discretization of the Transport Equation&#xA;$$&#xA;\frac{\partial \rho \phi}{\partial t} + \nabla(\rho U \phi) - \nabla (\rho \Gamma_\phi \nabla \phi) = S_\phi (\phi)&#xA;$$&lt;/p&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Finite Volume Method it looks like:&#xA;$$&#xA;\int_t^{t+\Delta t} \left[ \frac{\partial}{\partial t} \int_{V_p} \rho \phi dV + \int_{V_p} \nabla \cdot (\rho U\phi)dV - \int_{V_p} \nabla \cdot (\rho \Gamma_\phi \nabla \phi)dV \right]dt = \int_t^{t+\Delta t} \left(\int_{V_p}S_\phi(\phi)dV \right)dt&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and after transformations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_t^{t+\Delta t} \left[ \left( \frac{\partial \rho \phi}{\partial t} \right)_p V_p + \sum_f F\phi_f - \sum_f(\rho \Gamma_\phi)_f S.(\nabla\phi)_f \right]dt=\int_t^{t+\Delta t}(SuV_p+S_pV_p\phi_p)dt&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and time discretization:&#xA;$$&#xA;\frac{\rho_p\phi_p^n-\rho_p\phi_p^o}{\Delta t}V_p+ \frac{1}{2}\sum_f F\phi_f^n-\frac{1}{2}\sum_f(\rho\Gamma_\phi)_f S.(\nabla\phi)^n_f+ \frac{1}{2}\sum_f F\phi_f^o-\frac{1}{2}\sum_f(\rho\Gamma_\phi)_f S.(\nabla\phi)^o_f = SuV_p + \frac{1}{2}S_pV_p\phi_p^n+\frac{1}{2}S_pV_p\phi_p^o&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every cell we can make equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;a_p\phi^n_p+\sum_N a_N \phi_N^n = R_p&#xA;$$&lt;/p&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how we can get elements of matrix $A$ if we don't know $\phi$?&#xA;$$&#xA;[A][\phi]=[R]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;p&gt;Here Hrvoje Jasak wrote that every coefficient $ a_p $ includes the contribution from temporal derivative, convection and diffusion terms. But what formula of $a_p = ...$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://powerlab.fsb.hr/ped/kturbo/OpenFOAM/docs/HrvojeJasakPhD.pdf&quot; rel=&quot;nofollow&quot;&gt;http://powerlab.fsb.hr/ped/kturbo/OpenFOAM/docs/HrvojeJasakPhD.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8331" LastActivityDate="2014-06-18T14:44:11.617" Title="How to get discretization coefficients of matrix A in Finite Volume Method (FVM)?" Tags="|fluid-dynamics|openfoam|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11768" PostTypeId="1" CreationDate="2014-05-29T15:05:16.030" Score="3" ViewCount="1068" Body="&lt;p&gt;&lt;strong&gt;Introduction&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've developed a code to solve for generalised, incompressible 2D Stokes flow&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\eta \nabla^2 \mathbf{v} - \nabla p + \mathbf{S} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla . \mathbf{v} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{S}$ can include forces or gradients of an additional stress tensor $\Pi$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{S} = \mathbf{f} + \nabla . \mathbf{\Pi} + \cdots$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this code, I solve the above in Fourier space using the following hydrodynamic propagator,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{\mathbf{v}} = \frac{1}{\eta k^2}\left(\mathbf{\delta} - \hat{\mathbf{k}}\hat{\mathbf{k}}\right) . \tilde{\mathbf{S}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I numerically do this using FFTW, with periodic boundary conditions in the x and y directions. This code has been verified, for example, on the benchmark problem of flow past a periodic array of cylinders, with forces generated by an immersed boundary code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to extend this code to include walls at $y = 0, L_y$, with no slip and no permeation, can I still solve this with a modified version of the above? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One idea might be possibly to only use $sin$ modes in the y-direction for the Fourier transform? Would this amount to zero gradient BC on $\Pi$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you - any help is greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="7941" LastEditorUserId="7941" LastEditDate="2014-05-29T19:37:58.703" LastActivityDate="2014-12-16T04:47:35.773" Title="Solving Stokes flow with walls using Oseen tensor" Tags="|fluid-dynamics|boundary-conditions|fourier-analysis|navier-stokes|spectral-method|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11769" PostTypeId="1" AcceptedAnswerId="11770" CreationDate="2014-05-29T17:52:45.457" Score="2" ViewCount="1220" Body="&lt;p&gt;I have a sparse matrix coming from the discretization of a 3D elliptic &lt;a href=&quot;https://en.wikipedia.org/wiki/Partial_differential_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDE&lt;/a&gt;. The matrix is banded with seven non-zeros diagonals. The sparsity pattern of the matrix looks like this (the actual matrix is much larger):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/o7Dnf.png&quot; alt=&quot;Spy&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is worth noting that it is &lt;a href=&quot;https://en.wikipedia.org/wiki/Positive-definite_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;positive definite&lt;/a&gt;, but not symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to compute the eigenvalues, right eigenvectors and left eigenvectors. I need at least, for a given integer k, the k first eigenvectors corresponding to the k eigenvalues closest to the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most efficient way to do this for such a matrix?&lt;/p&gt;&#xA;" OwnerUserId="8222" LastEditorUserId="625" LastEditDate="2014-06-02T20:57:29.417" LastActivityDate="2014-06-02T20:57:29.417" Title="Eigenvalues of a sparse banded nonsymmetric matrix from an elliptic operator" Tags="|linear-algebra|sparse-matrix|iterative-method|eigenvalues|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11770" PostTypeId="2" ParentId="11769" CreationDate="2014-05-29T18:10:51.027" Score="8" Body="&lt;p&gt;The simple answer is that you would use &lt;a href=&quot;http://en.wikipedia.org/wiki/Inverse_iteration&quot;&gt;inverse iteration&lt;/a&gt; (subspace or with deflation). This is basically the &lt;a href=&quot;http://en.wikipedia.org/wiki/Power_method&quot;&gt;power method&lt;/a&gt; (repeatedly multiplying the matrix by a vector and normalizing, singling out the eigenvector corresponding to the eigenvalue of largest magnitude) applied to $A^{-1}$. Since you desire the $k$ closest to the origin, you need to use some kind of block method or orthogonalization, otherwise the power iteration converges to the same eigenvector each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically, these simple algorithms are too primitive to be useful (they would take too long). More sophisticated variations of these include the (block) &lt;a href=&quot;http://en.wikipedia.org/wiki/Lanczos_algorithm&quot;&gt;Lanczos&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Arnoldi_iteration&quot;&gt;Arnoldi&lt;/a&gt; or &lt;a href=&quot;http://web.eecs.utk.edu/~dongarra/etemplates/node136.html&quot;&gt;Jacobi-Davidson&lt;/a&gt; methods. Be careful to note the restriction on the matrix properties (e.g. classical Lanczos only works on Hermitian matrices). Note that since you want the smallest eigenvalues, you need to be able to apply $A^{-1}x$ in these iterations. So each iteration requires a sparse matrix solve. Your 3D spatial grid structure might be amenable for multigrid techniques to speed up the solve, or if it's of moderate size you can do an (incomplete) LU or Cholesky decomposition. You can also use these as a preconditioner for the solver, which typically the eigenvalue methods take as a parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically, Matlab has &lt;code&gt;eigs&lt;/code&gt;, and &lt;a href=&quot;http://web.eecs.utk.edu/~dongarra/etemplates/&quot;&gt;this free book&lt;/a&gt; has a decent listing of existing code.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-05-29T18:10:51.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11771" PostTypeId="2" ParentId="11768" CreationDate="2014-05-29T18:17:52.277" Score="2" Body="&lt;p&gt;If you want to use nonperiodic boundary conditions, you could change the Fourier basis in the $y$-direction for your spectral method to a Chebyshev basis. Fast Chebyshev Transforms make use of Fast Fourier Transforms; both are linear transformations on function spaces, so switching between the two amounts to a change of basis, and you can reuse significant parts of your implementation.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-05-29T18:17:52.277" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11772" PostTypeId="1" CreationDate="2014-05-29T19:22:29.977" Score="4" ViewCount="436" Body="&lt;p&gt;I would like to numerically solve the following system of 2 coupled partial differential equations for the unknown functions $\psi_X(x,y,t)$ and $\psi_C(x,y,t)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t \psi_X = -i\psi_C - i\vert\psi_X\vert^2 \psi_X$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t \psi_C = -i F(x,y)\exp{[i(k^p_x x+k^p_y y - \omega_p t)]} -i\psi_X -i [\delta - \Delta -i\kappa_C]\psi_C$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $F(x,y) &amp;gt; 0$ (typically a gaussian function), $\kappa_C &amp;gt;0$ and $\delta \in \mathbb{R}$ constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The domain of integration is a square box of length 2L, with PBC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have tried a 5th order Runge Kutta approach, with adaptive time step, and calculating the laplacian term in fourier space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone suggest a more robust/faster alternative?&lt;/p&gt;&#xA;" OwnerUserId="4656" LastEditorUserId="4656" LastEditDate="2014-06-15T12:00:32.737" LastActivityDate="2014-06-15T12:00:32.737" Title="Coupled nonlinear PDEs with time dependence on the RHS" Tags="|pde|nonlinear-equations|time-integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11773" PostTypeId="2" ParentId="11768" CreationDate="2014-05-29T19:28:36.943" Score="1" Body="&lt;p&gt;I think the answer may be obvious once you develop an Oseen tensor that is appropriate for the boundary conditions.  (This assumes you want to continue using an exact Oseen method to do the hydrodynamics, and not solve the Stokes equation explicitly.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Oseen tensor you give above is appropriate for a response to a point force in free space.  You need to find the Oseen tensor for the response to a point force with your boundary conditions - v to zero at the channel walls.  I think you may be able to find this using the method of images, or possibly just factorizing.  I'll take a shot at it if I get a chance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The benefit of the FFT method in your application is that the Oseen tensor is diagonal - you may need to choose a different basis for this to remain true, and it may or may not be efficient to transform into this basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am not sure how well this will work if you have forces on the boundaries - so I'm not sure what you mean by a boundary condition on $\Pi$.)&lt;/p&gt;&#xA;" OwnerUserId="4494" LastActivityDate="2014-05-29T19:28:36.943" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11774" PostTypeId="2" ParentId="11752" CreationDate="2014-05-29T19:44:16.420" Score="12" Body="&lt;p&gt;Both the standard cluster and custom supercomputer (Anton) versions of molecular dynamics at D. E. Shaw Research are both deterministic and parallel invariant.  That is, a test run on a single core generates the same bits as a massively parallel run.  The techniques include&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Integer summation: Although each force term is computed in floating point, the total force on each atom is summed in fixed point.  As a result, the sums are both commutative and associative, and therefore trivial to parallelize without losing determinism.  All state (positions and momenta) are maintained in fixed point as well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any required random numbers are generated using counter mode techniques, as described in &lt;a href=&quot;http://www.thesalmons.org/john/random123/papers/random123sc11.pdf&quot;&gt;http://www.thesalmons.org/john/random123/papers/random123sc11.pdf&lt;/a&gt;.  A counter mode generator amounts to encrypting the integer sequence 0, 1, 2, $\ldots$ using a sufficiently strong cypher.  See the paper for details.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="447" LastActivityDate="2014-05-29T19:44:16.420" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11775" PostTypeId="1" CreationDate="2014-05-29T20:57:03.993" Score="3" ViewCount="254" Body="&lt;p&gt;I am using Galerkin's method to set up some pde solver for my problems(1D/2D) at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some abrupt material interfaces do exist, so far, what I did regarding this is simply:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) no element across the interface, i.e. nodes exactly lie at the interfaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) no special B.C. imposed on interfaces, just natural B.C.(as it is).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) is what I did sufficient to deal with the interfaces?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) some people recommend to use zero thickness elements along the interfaces, so that every nodal point on the interface is associated with a duplicate node, is it necessary? and what is the justification?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="4532" LastEditDate="2014-05-31T07:56:56.667" LastActivityDate="2014-06-01T06:51:43.640" Title="Finite element: handle discontinuity/abrupt interface" Tags="|finite-element|finite-difference|finite-volume|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11776" PostTypeId="2" ParentId="11775" CreationDate="2014-05-29T21:29:12.453" Score="1" Body="&lt;p&gt;For a material property change for elliptic PDEs you shouldn't have to do anything special. It's best to put the nodes on the interface, but you shouldn't even need to impose a BC. It's not a boundary. Just a change in the coefficient.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-05-29T21:29:12.453" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11777" PostTypeId="1" CreationDate="2014-05-29T22:35:34.547" Score="3" ViewCount="195" Body="&lt;p&gt;At the mean-field level, the dynamics of a polariton condensate can be described by a type of nonlinear Schrodinger equation (Gross-Pitaevskii-type), for a classical (complex-number) wavefunction $\psi$. Its form in momentum space reads (in 2+1 dimensions):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{multline}&#xA;i \frac{d}{dt}\psi(k) =\left[\epsilon(k) &#xA;-i\frac{\gamma(k)}{2}\right] \psi(k) &#xA;+F_{p}(k)\,\, e^{-i\omega_{p}t}  \\&#xA;+  \sum_{q_1,q_2} g_{k,q_1,q_2}\, \psi^*(q_1+q_2-k)&#xA;\, \psi(q_1)\, \psi(q_2).&#xA;\end{multline}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $\epsilon(k)$ is the dispersion of the particles (polaritons). The polaritons are a non-equilibrium system, due to their finite lifetime (damping rate $\gamma$). Therefore, they need continuous pumping with amplitude $F_p$ at energy $\omega_p$.&#xA;Finally, there exists a momentum-dependent nonlinear interaction $g_{k,q_1,q_2}$ that depends of the so-called Hopfield coefficients $X$ (simple functions of momentum) as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;g_{k,q_1,q_2}=g\, X^*(k)\, X^*(q_1+q_2-k)\, X(q_1)\, X(q_2)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equivalently, the real-space correspondent of this equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{multline}&#xA;i\frac{d}{dt}\tilde{\psi}(r)=\left[\epsilon(i\partial_{r})-i\frac{\gamma(i\partial_{r})}{2}\right]\tilde{\psi}(r)+F_{p}e^{i(k_{p}r-\omega_{p}t)}\\&#xA;+\iiint dr_{123}\,\tilde{g}(r_1-r,r-r_2,r-r_3)\tilde{\psi}^*\!(r_1)\tilde{\psi}(r_2)\tilde{\psi}(r_3)&#xA;\end{multline}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{multline}&#xA;\tilde{g}\left(r_{1}-r,r-r_{2},r-r_{3}\right)=g\int dq_{3}X(q_{3})e^{iq_{3}\left(r-r_{3}\right)}\int dq_{1}X^*(q_{1})e^{iq_{1}\left(r_{1}-r\right)}\\&#xA;\times \int dq_{2}\, X^*(q_{2}+q_{3}-q_{1})\, X(q_{2})\, e^{iq_{2}\left(r-r_{2}\right)}&#xA;\end{multline}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where i have used that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\psi(k)=\int dr e^{-ikr}\tilde{\psi}(r)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how to best solve this equation numerically. Boundary conditions are not important, one can use PBC if that makes it easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, I was thinking that it would maybe be easier to solve the equation in k-space, wouldn't that be equivalent to using a spectral method? I am not sure how to do it in practice though, and especially how to take care of the nonlinear interaction term containing the 2 sums over $q_1$ and $q_2$.&lt;/p&gt;&#xA;" OwnerUserId="4656" LastEditorUserId="4656" LastEditDate="2014-05-30T17:42:05.343" LastActivityDate="2014-05-30T17:42:05.343" Title="Integration of nonlinear PIDE via spectral methods" Tags="|pde|nonlinear-equations|time-integration|integral-equations|spectral-method|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11778" PostTypeId="2" ParentId="11750" CreationDate="2014-05-30T01:35:22.020" Score="1" Body="&lt;p&gt;In addition to the other answers, let me just note this as a comparison: if you do not smoothed particle hydrodynamics but just a traditional Q2xQ1 (Taylor-Hood) finite element discretization of the Stokes equations in 3d, then you end up with ~400 nonzero entries per row of the matrix. That means ~5000 bytes per degree of freedom in the matrix alone, plus maybe that much again if you want a good preconditioner. So 10kB per degree of freedom. (I know @Jed Brown is going to say that one could get away with less, or do use matrix free methods, and that is all true, but for the sake of argument let us consider a &quot;traditional&quot; approach.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, if you have say 16GB of memory, you can do ~1.6M unknowns in total. Given that every cell in a uniform mesh has 25 unique degrees of freedom, this equates to 64,000 cells -- i.e., for example a 64x32x32 mesh. Whether that's appropriate for your problem is a question that depends on what you want to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How long will it take to solve such a problem? A rule of thumb is that solving 100,000 unknowns on a single core in a parallel context will take 1 minute. So if your computer has 8 cores, then that's 200k unknowns per core and it would take 2 minutes to solve a Stokes system of the size outlined above.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-05-30T01:35:22.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11779" PostTypeId="2" ParentId="11769" CreationDate="2014-05-30T01:55:05.097" Score="3" Body="&lt;p&gt;The easiest way is to use Matlab, Maple or Mathematica if the matrix is small (up to maybe a few 1,000 rows). If you have something bigger, use packages such as ARPACK. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest this because writing eigenvalue solvers from scratch is a most definitely non-trivial endeavor -- and it is completely unnecessary because there are great packages for this task around already!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-05-30T01:55:05.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11780" PostTypeId="2" ParentId="11766" CreationDate="2014-05-30T02:00:30.313" Score="1" Body="&lt;p&gt;The second to last equation you have in your question is linear in the $\phi^n_p$, and since you have one equation for each $p$, this is a linear system. It all comes down to just identifying which $\phi^n_p$ appears in which equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have this for finite volumes, but if you want to see it happen for finite elements, you may want to watch lecture 4 here: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-05-30T02:00:30.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11781" PostTypeId="2" ParentId="11752" CreationDate="2014-05-30T02:10:34.873" Score="1" Body="&lt;p&gt;You need to make sure that every particle update uses the same random number, whether you run on one of many threads. As an extreme case, consider that you have one RNG associated with every particle. Then, when you do the $n$th update for particle $m$, you will get the $n$th random number from RNG $m$, regardless of the thread you do this update on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may be a bit extreme, but you could have one RNG per, say, 1000 particles and each thread can only update blocks of 1000 particles at once, using the associated RNG. Again, this way it makes no difference which thread makes the update since it draws random numbers deterministically.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-05-30T02:10:34.873" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11782" PostTypeId="2" ParentId="11772" CreationDate="2014-05-30T04:34:18.317" Score="4" Body="&lt;p&gt;How are you choosing your time step?  If your Runge-Kutta method is explicit, you will need to take $\Delta t = {\mathcal O}((\Delta x)^2)$, which is the real reason that this method is slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to march faster in time, I would recommend using an operator-splitting approach, in which you alternate between integrating the ODE terms and integrating the spatial derivatives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1&lt;/strong&gt;: Integrate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\partial_t \psi_X = -i\psi_C - i\vert\psi_X\vert^2 \psi_X \\&#xA;\partial_t \psi_C = -i F(x,y) -i\psi_X -i [\delta -i\kappa_C]\psi_C&#xA;$$&#xA;over a time step $\Delta t$ using an explicit Runge-Kutta method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2&lt;/strong&gt;: Integrate&#xA;$$\partial_t \psi_C = i\Delta \psi_C$$&#xA;exactly (over a time step $\Delta t$) by taking the FFT, multiplying by the appropriate phase factor, and taking the inverse FFT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the time integration in step 2 is exact, the time step size will be restricted only based on step 1 (and that part doesn't look stiff).  The overall accuracy will be first order.  You can apply higher order splitting methods if necessary (look up &lt;em&gt;Strang splitting&lt;/em&gt; for second order).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to impose the correct boundary conditions, you should use an implicit time integration method for the spatial part.  The implicit trapezoidal or midpoint method should work well, and will again allow you to take arbitrarily large time steps.  But you'll need to use an efficient linear solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A detailed example (including Python code) of how to implement such methods for a different problem can be found in &lt;a href=&quot;http://nbviewer.ipython.org/github/ketch/AMCS252/blob/master/8_A_reaction-diffusion_model_for_pattern_formation.ipynb&quot; rel=&quot;nofollow&quot;&gt;this IPython notebook&lt;/a&gt; (written by me).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very basic but useful introduction to low-order splitting methods and other alternatives can be found, for instance, in &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow&quot;&gt;Randall LeVeque's finite difference book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-06-01T07:59:28.967" LastActivityDate="2014-06-01T07:59:28.967" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11783" PostTypeId="2" ParentId="11769" CreationDate="2014-05-30T05:52:12.990" Score="-1" Body="&lt;p&gt;There is EIGEN C++ library (&lt;a href=&quot;http://eigen.tuxfamily.org/&quot; rel=&quot;nofollow&quot;&gt;http://eigen.tuxfamily.org/&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="8340" LastActivityDate="2014-05-30T05:52:12.990" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11784" PostTypeId="2" ParentId="11766" CreationDate="2014-05-30T10:39:25.057" Score="1" Body="&lt;p&gt;First of all you know $\phi$ from field initialization or from previous iteration(or timestep).&#xA;When you use field values in such way, considering mass fluxes known at the beggining of each iteration, you create a decoupling in nonlinear term, and Jasak mentioned that in his thesis, also saying that it causes an error, which vanishes when iterations converge. Also he mentioned that this decoupling is less severe than pressure-velocity decoupling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the main diagonal coefficient $a_P$ it is negative sum of neighboring coefficients $a_{nb}$ plus implicit part of non-stationary term.&#xA;Physically it means that flux going out from one cell, goes into another, neighbor cell (share the same cell face).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out Ferziger &amp;amp; Peric book (a recommendation). &lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2014-05-30T10:39:25.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11785" PostTypeId="1" AcceptedAnswerId="11786" CreationDate="2014-05-30T15:06:09.113" Score="0" ViewCount="169" Body="&lt;p&gt;The classical streaming operator used in FHP lattice gas automata is this:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$n_i(r - c_i, t + 1) = n_i(r, t)$$&lt;/span&gt;&#xA;However if you think in terms of the streaming itself, it should be something like &lt;span class=&quot;math-container&quot;&gt;$$n_j(r + c_i, t + 1) = n_i(r, t)\qquad\text{where}\,j=(i+3)\,mod\,6$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This image illustrates my point : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ovr9Q.jpg&quot; alt=&quot;D2Q9-LBM illustration&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically what I am saying is if you consider an occupied cell, say i, it should stream to the next corresponding cell (the next black vector) instead of the same cell in the next node namely j as mentioned above. However this is not so, it streams to the same location (shown by the blue arrow). Can anyone explain what is the reason of such a choice?&lt;/p&gt;&#xA;" OwnerUserId="8161" LastEditorUserId="22284" LastEditDate="2019-01-13T15:03:04.437" LastActivityDate="2019-01-13T15:03:04.437" Title="Regarding streaming operator in Lattice Gas Automata" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11786" PostTypeId="2" ParentId="11785" CreationDate="2014-05-30T16:36:44.700" Score="1" Body="&lt;p&gt;There are some issues of notation here.  First off, the index $i$ is over velocity, not &quot;cell&quot;. $n_i(r,t)$ defines occupancy of velocity direction $i$ at position $r$ at time $t$.  Define $c_i$ as the sound velocity in direction $i$.  Particles moving in direction $i$ with constant velocity $c_i$ will move from $r$ to $r + c_i$ over the course of one time step.  Your statement neglects collision, so the particle will continue to have velocity $c_i$, and therefore be represented as occupying $n_i(r + c_i,t+1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take $c_i$ to be lattice offsets, the sign will be reversed.  In any case, the velocity direction $i$ of the particle stays the same unless changed by a collision operator or boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your diagram shows the correct behavior.  Now, for the sake of argument, let's assume that it goes to the &quot;next vector&quot; like you seem to be so intent on claiming.  The &quot;next vector over&quot; is in the &lt;em&gt;opposite direction&lt;/em&gt; from the one it's coming from, meaning that at $t+1$ the direction would be $-c_i$ and position would be $r + c_i$ and at $t+2$ one would have position $r$ again with direction $c_i$.  It would wildly ping-pong between these two states.  This is obviously not the intended behavior.  You're visualizing this incorrectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/02FYh.png&quot; alt=&quot;An image of streaming Lattice Automata in three timesteps&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the above image as a demonstration of what it means to stream.  We use names instead of indices to make this more clear.  The $c_i$ are defined as the next steps in the named direction $i$. Note that there is $i=still$, which means that the velocity of the particle is nil.  Red dots indicate occupancy of a velocity state $i$, e.g. $n_i(r,t)$ is marked in red.  We start at $t=0$ with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;n_{downwardright}(upperleft,0) &amp;amp;= true\\&#xA;n_{downward}(uppercenter,0) &amp;amp;= true\\&#xA;n_{still}(upperright,0) &amp;amp;= true&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and all others false.  From there, we stream and update to $t=1$.  This gives us&#xA;\begin{align}&#xA;n_{downwardright}(center,1) &amp;amp;= n_{downwardright}(upperleft,0) &amp;amp;= true\\&#xA;n_{downward}(center,1) &amp;amp;= n_{downward}(uppercenter,0) &amp;amp;= true\\&#xA;n_{still}(upperright,1) &amp;amp;= n_{still}(upperright,0) &amp;amp;= true&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with all others false.  If we stream again, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;n_{downwardright}(lowerright,2) &amp;amp;= n_{downwardright}(center,1) &amp;amp;= true\\&#xA;n_{downward}(lowercenter,2) &amp;amp;= n_{downward}(center,1) &amp;amp;= true\\&#xA;n_{still}(upperright,2) &amp;amp;= n_{still}(upperright,1) &amp;amp;= true&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in conclusion, the particles are going in the same direction the entire time, and transmit across the grid one cell at a time in the directions $i$ defined by $c_i$.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="143" LastEditDate="2014-06-01T04:09:11.910" LastActivityDate="2014-06-01T04:09:11.910" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11787" PostTypeId="1" CreationDate="2014-05-30T18:23:11.937" Score="1" ViewCount="96" Body="&lt;p&gt;I am comparing the performance several finite difference methods of solving an initial-boundary value problem. There are several dimensions to this comparison:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of cells&lt;/li&gt;&#xA;&lt;li&gt;Number of timesteps&lt;/li&gt;&#xA;&lt;li&gt;Solution Method:&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Explicit (one sweep on a single thread, no iterations)&lt;/li&gt;&#xA;&lt;li&gt;Alternating Direction Explicit (two sweeps on separate threads, no iterations)&lt;/li&gt;&#xA;&lt;li&gt;Alternating Direction Implicit (two sweeps on one thread [TDMA] with three sub-timesteps, no iterations)&lt;/li&gt;&#xA;&lt;li&gt;Fully Implicit (any number of threads, using an iterative BICGSTAB solver)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Question:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;My question is this: What metric should I use to compare the relative performance for various combinations of number of cells, number of timesteps, and solution method? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some information on performance metrics on &lt;a href=&quot;http://www.arctic.umn.edu/perf-book/supplement.html&quot; rel=&quot;nofollow&quot;&gt;David J. Lilja's website&lt;/a&gt;. He seems to consider execution time (wall and CPU) as the best metric. But I'm wondering if there might be a more suitable metric for my application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I've considered so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Wall time and CPU time&lt;/strong&gt;: This is easy to measure, but the problem is that it is only representative for computer architecture where the program is executed. I'm doing most of my testing on a 24 core machine...which won't be representative of typical applications. It would be nice not to have to include a description of the machine's architecture every time I publish results. Are there meaningful ways to normalize measured time to avoid this problem?&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Number of Operations&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sum of (N_iterations*N_sweeps*N_sub-timesteps*N_cells)/(N_threads) across all timesteps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is repeatable and platform independent, but it doesn't include some of the overhead of setting up the initial-boundary value problem and creating matrices, etc. It also makes some simplifying assumptions about threading overhead.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It would also be nice to give a rough estimate of parallel and serial performance, so that someone could get a sense of what the speed up would be for a given number of processors (i.e., a way to extrapolate from the results on my 24 core machine).&lt;/p&gt;&#xA;" OwnerUserId="3179" LastActivityDate="2014-06-01T19:30:10.103" Title="Performance metrics to compare initial-boundary value problem solutions" Tags="|linear-algebra|pde|performance|iterative-method|explicit-methods|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11788" PostTypeId="2" ParentId="11764" CreationDate="2014-05-30T19:24:24.240" Score="0" Body="&lt;p&gt;If you're using a library like LAPACK, I think you'll find there are specific routines for symmetric matrices, in addition to those that work for completely general inputs. These are intended to be more efficient (in time and/or storage).&lt;/p&gt;&#xA;" OwnerUserId="8328" LastActivityDate="2014-05-30T19:24:24.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11789" PostTypeId="2" ParentId="11787" CreationDate="2014-05-30T22:02:05.537" Score="1" Body="&lt;p&gt;Time to solution with (approximately) equal error is the best metric you have for comparing algorithms like this, but you need to do it across a range of mesh sizes and PDE coefficients. The explicit methods will have time step constraints that are $O(h^p)$ where $h$ is the mesh size and $p$ is related to the discretizations you use. This means that they will require more time steps on finer grids. The implicit methods won't have this problem, but if you use an iterative solver, then there may be a mesh dependence on the number of solver iterations required. This dependence could cancel out any benefit you have of taking arbitrary time steps, and you may not be able to take large timesteps if the physics requires small ones. There may be several points in parameter space where the methods switch their rank order due to these factors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relative rank of the methods should stay the same as you move from machine to machine, but it's not guaranteed. It's best to always run your simulation comparisons on the same architecture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parallel speedup is the standard way to measure and extrapolate parallel performance. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-05-30T22:02:05.537" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11790" PostTypeId="1" AcceptedAnswerId="11791" CreationDate="2014-05-31T21:54:45.210" Score="3" ViewCount="196" Body="&lt;p&gt;I was reading &lt;a href=&quot;http://www.informit.com/articles/article.aspx?p=2213858&quot; rel=&quot;nofollow&quot;&gt;Twenty Questions for Donald Knuth&lt;/a&gt; and was intrigued by Knuth's argument in question 17 for why he suspects P=NP. In the discussion he asks why you couldn't have an algorithm bounded by a finite but incredibly large polynomial order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not a computer scientist, but I don't know of any really high-order polynomial algorithms. I know of a few from computational physics that are bounded by $N^{12}$, and, based on those, I can imagine that there are some that are like $N^{20}$. But I can't think of any &lt;em&gt;useful&lt;/em&gt; algorithms that would be, say, $\mathcal{O}(N^{1000})$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By useful, I mean based on something I would actually want to do, solving some problem from physics or something, not something dreamed up for the sake of creating a high-polynomial-order algorithm. Can someone give me a few examples of real problems with high polynomial order?&lt;/p&gt;&#xA;" OwnerUserId="4817" LastActivityDate="2014-05-31T23:08:37.930" Title="Examples of high polynomial order complexity" Tags="|algorithms|complexity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11791" PostTypeId="2" ParentId="11790" CreationDate="2014-05-31T23:08:37.930" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://cstheory.stackexchange.com/questions/6660/polynomial-time-algorithms-with-huge-exponent-constant/&quot;&gt;A thread on cstheory has a few examples&lt;/a&gt;. So, there are definitely algorithms with very high exponents that were made for purposes other than simply creating a high-polynomial-order algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the tricky bit is that for any high-order algorithm to be useful, $N$ has to be really really small. In physics, I don't know of any high order polynomial algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going to a worse case than polynomial, quantum simulations are exponential in the number of particles simulated. This is why chemistry simulations tend to be a mix of classical and quantum simulations. We simply don't have the machine power to simulate molecular interactions quantum mechanically. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="-1" LastEditDate="2017-04-13T12:32:32.353" LastActivityDate="2014-05-31T23:08:37.930" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11792" PostTypeId="1" CreationDate="2014-06-01T01:53:18.660" Score="1" ViewCount="49" Body="&lt;p&gt;I have an infinite subset $S$ of $\mathbb{N}^k$ which I know (or at least strongly suspect) is a finite union of regions, each of which is defined by finitely many linear equalities and inequalities (with integer coefficients). I would like to find an explicit description of $S$, i.e., sets of (in)equalities that define it. Is there a good algorithm for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A little more information. The representation I have for $S$ is its indicator function, i.e., I have a function $f \colon \mathbb{N}^k \to \{0,1\}$ such that $f(x)=1$ if and only if $x \in S$. In my case, $k \approx 10$ and $f$ can be computed reasonably efficiently. Obviously, in this generality one cannot verify in a finite computation that a hypothetical description of $S$ is correct. However, for my $S$ I expect the answer to be reasonable, and I have other means to prove a purported description is correct.&lt;/p&gt;&#xA;" OwnerUserId="9344" LastActivityDate="2014-06-01T01:53:18.660" Title="Computing a description of a set of lattice points" Tags="|algorithms|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11793" PostTypeId="2" ParentId="11775" CreationDate="2014-06-01T06:51:43.640" Score="2" Body="&lt;p&gt;It depends on the physics of your problem. Let us take a second-order elliptic equation $\nabla\cdot(a\nabla u) + k^2 u = f$ for example, where $a$ and $k$ are discontinuous across interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most common case, across the interface you have $u$ and $(a\nabla u)\cdot \mathbf{n}$ matched from two sides of the interface, so that your do-nothing and assembly (unique dofs on interface) strategy is conforming to the physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are some cases, the physics is different. For example, in electromagnetic, there might be surface (interface) charges, so that for transversal field you get a jump $[(a\nabla u)\cdot \mathbf{n}]=S$ where $[v]=v_1-v_2$ and $v_i$ is the trace from $\Omega_i$ for the interface between $\Omega_1$ and $\Omega_2$ while $u$ keep continuous across interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even weird, for field component normal to interface, you get the different things: $[\nabla u\cdot \mathbf{n}]=S$ and $[ku]=0.$ These are very unusual. So it is not the mathematics but the physics one should ask for.&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2014-06-01T06:51:43.640" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11794" PostTypeId="2" ParentId="11787" CreationDate="2014-06-01T06:51:57.067" Score="2" Body="&lt;p&gt;I agree with Bill Barth that &quot;time to solution&quot; is a decent metric, but I'm afraid it has a couple of downsides:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It's not portable: what's fast on one machine is not guaranteed to be fast on another (modern architectures differ not just in their floating point performance, but also in how their caches work or what their memory bandwidth is).&lt;/li&gt;&#xA;&lt;li&gt;It's implementation specific: maybe algorithm A is just slower than algorithm B, because itwas implemented poorly, or because the compiler didn't optimize it well?&lt;/li&gt;&#xA;&lt;li&gt;Also keep in mind that not every algorithm may scale equally well on every machine -- and that this scalability is again highly dependent on the implementation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you are interested in evaluating the methods themselves, rather than their implementation, then I'd suggest you to build an abstract, predictive performance model, based on the following characteristics:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;number of operations (as you suggested yourself, BTW: your definition is usually known as &quot;Lattice Updates&quot;, and lots of papers measure performance in GLUPS, or Giga Lattice Updates Per Second)&lt;/li&gt;&#xA;&lt;li&gt;expected single-node performance (e.g.  &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1498785&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=1498785&lt;/a&gt; or &lt;a href=&quot;https://www.researchgate.net/publication/244988210_A_Predictive_Performance_Model_for_Stencil_Codes_on_Multicore_CPUs&quot; rel=&quot;nofollow&quot;&gt;https://www.researchgate.net/publication/244988210_A_Predictive_Performance_Model_for_Stencil_Codes_on_Multicore_CPUs&lt;/a&gt; )&lt;/li&gt;&#xA;&lt;li&gt;for multi-node parallelization (e.g. via MPI), you'll need to estimate communication overhead, suggestion: &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=155333&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=155333&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3829" LastEditorUserId="3829" LastEditDate="2014-06-01T19:30:10.103" LastActivityDate="2014-06-01T19:30:10.103" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="11795" PostTypeId="2" ParentId="1374" CreationDate="2014-06-01T07:45:59.903" Score="2" Body="&lt;p&gt;I'm the project lead on &lt;a href=&quot;http://www.libgeodecomp.org/&quot; rel=&quot;nofollow&quot;&gt;LibGeoDecomp&lt;/a&gt;, so I thought I might chime in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, you can implement a FEM with LibGeoDecomp. We're currently working on an improved data container for exactly this use case. But to be fair: completion of that work is still months away, and until then performance will not be optimal. Feel free to contact me via &lt;a href=&quot;http://www3.informatik.uni-erlangen.de/Persons/schaefer/&quot; rel=&quot;nofollow&quot;&gt;e-mail&lt;/a&gt; if you still want to give LibGeoDecomp a try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option would be &lt;a href=&quot;http://fenicsproject.org/&quot; rel=&quot;nofollow&quot;&gt;Fenics&lt;/a&gt;, which has well optimized solvers for irregular problems. But AFAIK you don't get to write your own solver in that case.&lt;/p&gt;&#xA;" OwnerUserId="3829" LastActivityDate="2014-06-01T07:45:59.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11796" PostTypeId="1" CreationDate="2014-06-01T08:25:14.117" Score="2" ViewCount="1204" Body="&lt;p&gt;I am doing an undergraduate research to model ocean waves in shallow waters in real time. I am trying to base my research on Navier-Stokes equations since they produce the most accurate mathematical results. As a start I am supposed to study an existing implementation of the 3D Navier-Stokes equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find a most suitable implementation for the purpose of my research? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also came across a set of equations called the shallow water equations derived from the Navier-Stokes equations. Are there any suitable implementations of them as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also can I know the different approaches that exist in solving these equations with pros and cons....&lt;/p&gt;&#xA;" OwnerUserId="9345" LastEditorUserId="123" LastEditDate="2014-06-01T08:45:53.193" LastActivityDate="2014-06-03T11:10:10.837" Title="Implementation of Navier-Stokes Equation for Ocean Waves" Tags="|simulation|navier-stokes|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11797" PostTypeId="2" ParentId="11796" CreationDate="2014-06-01T08:53:32.477" Score="1" Body="&lt;p&gt;&lt;strong&gt;Numerical solvers for 3D Navier-Stokes&lt;/strong&gt;: I assume you mean for incompressible fluids.  This is a huge question; there are many methods and many codes out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Shallow water equations&lt;/strong&gt;: Again, there are very many methods and codes.  I will point you to the ones that I know best.  Both are part of Clawpack.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://clawpack.github.io/doc/pyclaw/&quot; rel=&quot;nofollow&quot;&gt;PyClaw&lt;/a&gt; - General hyperbolic conservation law solver that includes 1D and 2D shallow water solvers.  The interface is Python.  Includes MPI parallelism.  I would go with this option if you will only handle idealized problems.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://clawpack.github.io/doc/geoclaw.html&quot; rel=&quot;nofollow&quot;&gt;GeoClaw&lt;/a&gt; - A widely used code for modeling tsunamis, storm surges, and flooding.  The interface is partly Python and partly Fortran.  Includes adaptive mesh refinement and OpenMP parallelism.  I would go with this option if you intend to incorporate things like real-world bathymetry in your simulations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-06-01T08:53:32.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11798" PostTypeId="1" CreationDate="2014-06-01T13:42:29.787" Score="10" ViewCount="869" Body="&lt;p&gt;Is there a complexity degree that is bigger than $O(n)$ and smaller than $O(n \log n)$?&lt;/p&gt;&#xA;" OwnerUserId="9347" LastEditorUserId="9355" LastEditDate="2014-06-01T23:24:39.057" LastActivityDate="2014-06-01T23:24:39.057" Title="Is there a complexity between $O(n)$ and $O(n \log n)$" Tags="|algorithms|complexity|efficiency|" AnswerCount="3" CommentCount="3" ClosedDate="2014-06-02T21:23:33.087" ContentLicense="CC BY-SA 3.0" />
  <row Id="11799" PostTypeId="2" ParentId="11798" CreationDate="2014-06-01T13:49:59.430" Score="20" Body="&lt;p&gt;$n \log\log n$ is between $n$ and $n \log n$, and is a relatively common one to find in the wild.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-06-01T13:49:59.430" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11800" PostTypeId="2" ParentId="11798" CreationDate="2014-06-01T15:08:25.493" Score="7" Body="&lt;p&gt;On top of $O(n\log(\log(n)))$, there's also $O(n \log^*(n))$ in which $\log^*$ is the number of times the logarithm function must be applied in order for the result to be less than or equal to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if you already know an Euclidean minimum spanning tree, the Delaunay triangulation may be discovered in $O(n\log^*(n))$ time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More extremely, one can look at the inverse Ackermann function $\alpha(n,n)$, which may be found in the analysis of several algorithms of complexity $O(n\alpha(n,n))$.  There's a good introduction &lt;a href=&quot;http://www.gabrielnivasch.org/fun/inverse-ackermann&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="143" LastEditorUserId="143" LastEditDate="2014-06-01T16:54:34.097" LastActivityDate="2014-06-01T16:54:34.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11801" PostTypeId="2" ParentId="11798" CreationDate="2014-06-01T22:40:01.640" Score="4" Body="&lt;p&gt;There are infinitely many, since $O(n(\log n)^\alpha) \subsetneq O(n(\log n)^\beta)$ for any $\alpha&amp;lt;\beta$. So, in particular, $O(n) = O(n(\log n)^0) \subsetneq O(n(\log n)^\alpha) \subsetneq O(n\log n)$ for any $\alpha\in (0,1)$.&lt;/p&gt;&#xA;" OwnerUserId="9357" LastActivityDate="2014-06-01T22:40:01.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11803" PostTypeId="1" CreationDate="2014-06-02T05:45:55.777" Score="2" ViewCount="253" Body="&lt;p&gt;I am currently working on a 2D finite element code with a mesh that contains duplicate nodes at certain interfaces where jumps occur. In order to set up the appropriate linear systems, I have to take the mesh connectivity information and locations of mesh nodes, and manipulate them to determine which elements are adjacent across an interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get an idea of potential run-time scaling in serial, I've expressed all of the asymptotic complexities of each algorithm in terms of the number of nodes (or the number of degrees of freedom), because that is a natural complexity metric. I'm able to do that using well-known facts about planar graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to be able to extend these algorithms to 3D meshes, and I'd like to have an idea of their complexity in terms of the number of nodes. I'm also sure that 3D meshes are not necessarily planar graphs. Is there a result that bounds the number of edges, facets, and elements of a 3D mesh in terms of the number of mesh nodes? Assuming a simplicial mesh is fine; results for hexahedral meshes would also be interesting, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure that none of what I'm doing is new; I just don't work with meshes very much, so I have no idea where to look for the result.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-06-02T22:14:31.710" Title="Is there a bound on the number of edges, facets, and elements in a 3D simplicial mesh in terms of the number of mesh nodes?" Tags="|finite-element|finite-volume|unstructured-mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11804" PostTypeId="2" ParentId="11803" CreationDate="2014-06-02T15:11:49.070" Score="6" Body="&lt;p&gt;The complexity of simplicial tessellations in $\mathbb{R}^{d}$, for $d&amp;gt;2$ is, unfortunately, not linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, a simplicial tessellation $\mathcal{T}(X)$ for $n$ vertices $X\subset\mathbb{R}^{d}$ can have $\Theta(n^{\lceil d/2\rceil})$ d-simplexes &lt;em&gt;in the worst case&lt;/em&gt;. Similar results exist for the lower dimensional faces of $\mathcal{T}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on practical experience with tetrahedral meshes in $\mathbb{R}^{3}$ though, I think it's generally accepted that &lt;em&gt;most&lt;/em&gt; tetrahedral meshes exhibit linear complexity, such that $|\mathcal{T}|=O(n)$. Considering that you're almost certainly dealing with a high quality finite-element style mesh, rather than some contrived pathological example from computational geometry, I imagine that linear complexity would be observed in practice in your case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be interested to check out &lt;a href=&quot;http://www.cs.berkeley.edu/~jrs/jrspapers.html&quot; rel=&quot;nofollow&quot;&gt;Jonathan Shewchuk's research&lt;/a&gt; on (Delaunay) tessellations. The result quoted above appears in the recent book &lt;em&gt;Delaunay Mesh Generation&lt;/em&gt; by Shewchuk, Cheng and Dey -- it's a great reference for mesh generation theory!&lt;/p&gt;&#xA;" OwnerUserId="1139" LastEditorUserId="1139" LastEditDate="2014-06-02T22:14:31.710" LastActivityDate="2014-06-02T22:14:31.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11805" PostTypeId="2" ParentId="11803" CreationDate="2014-06-02T15:18:07.263" Score="1" Body="&lt;p&gt;For 2D and triangles, there's the traditional Euler formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E=F+V-1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $E$ edges, $F$ faces, and $V$ vertices (assuming no holes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 3D on tetrahedra, there's a similar formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;V-E+F=T+1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where everything is as above with $T$ tets. This is all taken from Carey's book &lt;em&gt;Computational Grids&lt;/em&gt;, Taylor &amp;amp; Francis, 1997 pp. 86 and 90. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Darren's answer points out, though, this can lead to some complexity in 3D, though many meshes are nice.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-06-02T15:18:07.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11806" PostTypeId="1" AcceptedAnswerId="11808" CreationDate="2014-06-02T16:17:34.683" Score="3" ViewCount="105" Body="&lt;p&gt;I'm writing code that uses a lot of graph operations; many higher-level algorithms iterate over all the graph edges and do some processing on them. There's an iterator method that returns a batch of edges, like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;G.get_edges(edges,iterator,num_edges);&#xA;for (k = 0; k &amp;lt; num_edges; k++) {&#xA;    i = edges[k,0];&#xA;    j = edges[k,1];&#xA;    // Do something to (i,j)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Elsewhere in the program, there's a parameter called &lt;code&gt;batch_size&lt;/code&gt; which is the standard number of edges to operate on whenever this algorithm is called. Currently, I have have &lt;code&gt;batch_size&lt;/code&gt; hard-coded to 64, figuring that a cache line can hold 128 integers on my computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What's a sane way to automatically tune the batch size?&lt;/strong&gt; I'd like to do something akin to how &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatically_Tuned_Linear_Algebra_Software&quot; rel=&quot;nofollow&quot;&gt;ATLAS&lt;/a&gt; chooses the block size without having to read the ATLAS source code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best guess for how this should work:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Compile an initial version of the code where the batch size is chosen at run-time&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run a test suite several times with different batch sizes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pick the batch size &lt;code&gt;b&lt;/code&gt; for which the test suite ran fastest.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generate a new &lt;code&gt;.h&lt;/code&gt; file containing a pre-processor directive that sets the batch size to be &lt;code&gt;b&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compile the rest of the code, for which the batch size is now known at compile time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="625" LastEditDate="2014-06-02T20:57:52.530" LastActivityDate="2014-06-03T17:45:51.487" Title="Automated software optimization and tuning" Tags="|software|hpc|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11807" PostTypeId="1" CreationDate="2014-06-02T16:48:52.257" Score="2" ViewCount="1342" Body="&lt;p&gt;Which numerical methods would be optimal to find an extremum of a function with an almost flat &quot;valley&quot; (but a single minimum in the middle of the valley)? In this context optimal means the least number of function evaluations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity's sake, assume that it's a two-dimensional problem.&#xA;Also, of importance may be that I am more interested not in the coordinates of the extremum, but the value of the function in it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Rosenbrock_function&quot; rel=&quot;nofollow&quot;&gt;Rosenbrock function&lt;/a&gt; is &#xA;$$f(x,y) = (1-x)^2 + 100(y-x^2)^2$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the term &quot;'Rosenbrock'-like&quot;, because I have not yet defined the function itself. The problem I am solving is that of nonlinear equality constraint optimization, which I believe I have reduced to a global optimization of a function with a &quot;valley&quot; along the constraint. &lt;/p&gt;&#xA;" OwnerUserId="9362" LastEditorUserId="9362" LastEditDate="2014-06-03T05:04:49.677" LastActivityDate="2014-08-11T23:46:16.033" Title="Optimal numerical method for optimization of “Rosenbrock Banana”-like function" Tags="|optimization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11808" PostTypeId="2" ParentId="11806" CreationDate="2014-06-02T18:28:12.127" Score="4" Body="&lt;p&gt;You could just write an edit, compile, run loop in shell or Python that does it directly in the .h file if you think hard-coding this value is likely to give the compiler an advantage. If the runs are short enough, it ought to be straightforward to iterate over all values of &lt;code&gt;batch_size&lt;/code&gt; in a reasonable range.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-06-02T18:28:12.127" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11809" PostTypeId="2" ParentId="11796" CreationDate="2014-06-03T08:38:00.773" Score="0" Body="&lt;p&gt;I found an implementation of Shallow Water equations in FreeFEM++ using finite volume method in &lt;a href=&quot;http://www.um.es/freefem/ff++/pmwiki.php?n=Main.Finitevolumes&quot; rel=&quot;nofollow&quot;&gt;the FreeFEM++ wiki&lt;/a&gt;. Have a look at the paper.&lt;/p&gt;&#xA;" OwnerUserId="5541" LastActivityDate="2014-06-03T08:38:00.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11810" PostTypeId="1" AcceptedAnswerId="14289" CreationDate="2014-06-03T09:37:07.873" Score="2" ViewCount="88" Body="&lt;p&gt;A PQR file of a protein describes the positions in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and the charges of every atom in it. If you have them, you can construct the charge density &lt;span class=&quot;math-container&quot;&gt;$\rho(x)$&lt;/span&gt; of the protein, then use it to find the potential &lt;span class=&quot;math-container&quot;&gt;$\phi(x)$&lt;/span&gt;. The Boltzmann distribution is useful when you can't or don't know how to build the &lt;span class=&quot;math-container&quot;&gt;$\rho(x)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is it necessary to use Poisson - Boltzmann equation if I only need to build electrostatic potential from a PQR file?&lt;/p&gt;&#xA;" OwnerUserId="8250" LastEditorUserId="20688" LastEditDate="2019-07-02T00:48:56.977" LastActivityDate="2019-07-02T00:48:56.977" Title="Is it necessary to use Poisson - Boltzmann equation if I only need to build electrostatic potential from a PQR file?" Tags="|poisson|electromagnetism|computational-biology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11811" PostTypeId="2" ParentId="11796" CreationDate="2014-06-03T11:10:10.837" Score="0" Body="&lt;p&gt;The most important limitation of the shallow water equation is arguably that overturning waves can not be represented. People in graphics have tried to overcome this limitation, consider e.g. &lt;a href=&quot;http://graphics.ethz.ch/Downloads/Publications/Papers/2007/Thue07b/Thue07b.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. I'm not sure if this model readily extends to physically accurate solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the implementation of the shallow water equation, a good point to start your research might be using the most simple option, namely a finite difference approach. You can find a MATLAB implementation &lt;a href=&quot;http://www.mathworks.com/moler/exm/chapters/water.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. The actual simulation loop fits in one line. &lt;/p&gt;&#xA;" OwnerUserId="9366" LastActivityDate="2014-06-03T11:10:10.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11813" PostTypeId="1" CreationDate="2014-06-03T15:24:38.970" Score="1" ViewCount="104" Body="&lt;p&gt;I am trying to accurately calculate $\vec{j} \cdot \vec{E}$ for an electron energy equation on a finite-volume mesh.  $\vec{j}$ is the electron current density, and $\vec{E}$ is the electric field.  On the vertex centered mesh, I know $\phi$ at the center of each cell (where $\vec{E} = \nabla \phi$) and I also have a function that calculates $\vec{j} \cdot \hat{n}$ at each face of each cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the $\phi$ values, I can easily calculate $\vec{E} \cdot \hat{n}$.  The problem is that multiplying these two normal components leaves out the tangential components.  Taking the maximum over the faces leads to 10-20% error and volume averaging them is completely incorrect (by several factors).  The current solution I am using, is that I also happen to know the direction of $\vec{E}$ although I don't fully trust it.  From the physics, it is a fair assumption in most cases that $\vec{j}$ is close in direction to $\vec{E}$, so I assume that they are in the same direction, and divide $\left(\vec{E} \cdot \hat{n}\right) \left(\vec{j} \cdot \hat{n}\right)$ by $\cos^2{\theta}$ which gets me close although I have to ignore faces where the normal is within 30 degrees of perpendicular to $\vec{E}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also found a solution that works but utilizes a large number of trig functions, which is likely too slow to be done at every cell in every iteration of a unsteady FV solve.  The solution works as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $\vec{v}\cdot \hat{n}_1$, $\vec{v} \cdot \hat{n}_2$, $\hat{n}_1$, and $\hat{n}_2$  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v = 2*rand(1,2) - 1;    % Original vector we want to reproduce&#xA;&#xA;n_1 = 2*rand(1,2) - 1;  % unit normal vectors&#xA;n_1 = n_1/norm(n_1);&#xA;n_2 = 2*rand(1,2) - 1;&#xA;n_2 = n_2/norm(n_2);&#xA;&#xA;vn_1 = v.'*n_1;         % dot products&#xA;vn_2 = v.'*n_2;&#xA;&#xA;thetaN1 = atan2(n_1(2), n_1(1));      % Calculate angle of first normal to x-axis&#xA;phi = thetaN1 - atan2(n_2(2),n_2(1)); % Difference in angle between normals&#xA;&#xA;theta1 = atan((vn_1*cos(phi) - vn_2)/(vn_1*sin(phi)));  %Use both dot products to back out angle between first normal and v&#xA;v_mag = vn_1/cos(theta1);  % Back out magnitude of v&#xA;&#xA;v_out(1) = v_mag*cos(thetaN1 + theta1);  % Calculate components of v&#xA;v_out(2) = v_mag*sin(thetaN1 + theta1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above reproduces &lt;code&gt;v&lt;/code&gt; accurately down to almost double precision.  Even if I introduce small amounts of noise into the original v of about 1% for each of the dot products (representing that the field $\vec{v}$ is changing in space from the center of one face to the center of another), it is still accurate to a few decimal places most cases.  As I said, the cost of this is higher than I would like. I would need to do the above calculation a few million times per time step.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel like I have somewhat reinvented the wheel and was wondering if there was a more efficient and common way to calculate the dot product of two fields when only the normal components at faces are known.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-08-02T17:06:05.540" Title="Accurate dot-product of fields with only knowing normals" Tags="|finite-volume|vector|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11814" PostTypeId="2" ParentId="11813" CreationDate="2014-06-03T15:34:11.523" Score="1" Body="&lt;p&gt;I would have expected a geometric discretization to place $E$ and $j$ along edges of the mesh (more precisely, $j$ would be fluxes through facets dual to the mesh edges), in which case $\phi$ is represented by scalar values at the vertices of the mesh. In this way, $E$ and $J$ always &quot;live&quot; at the same mesh elements and are collinear, allowing the dot product to be trivially evaluated. In your form, it seems like the best thing to do is just to refine the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your snippet of code can be optimized substantially. I common trick in graphics is to notice that $\sin \theta$ is simply the magnitude of the cross product of two unit vectors that span the angle, and $\cos \theta$ is their dot product. This allows you to completely avoid ever calculating trig functions, and reduces everything to vector algebra. Also, you frequently don't need to normalize vectors since if you end up taking ratios the magnitudes cancel out.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-06-03T15:34:11.523" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="11816" PostTypeId="2" ParentId="11806" CreationDate="2014-06-03T17:45:51.487" Score="1" Body="&lt;p&gt;I think it's rather uncommon these days to run an auto-tuning build, but not out of the question.  Looking at your code sample, I don't actually see any cache reuse, so I'm not sure what your anticipated benefit here is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Bill alludes, you should probably make the batch size a run-time parameter unless you've got evidence that your compiler is behaving inefficiently without it statically compiled in.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2014-06-03T17:45:51.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11817" PostTypeId="1" CreationDate="2014-06-03T18:54:51.273" Score="2" ViewCount="2974" Body="&lt;p&gt;I want to minimize two different functions simultaneously who have the same inputs. The functions are both linear and non-exponential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F_1(X_1, X_2) = a_1X_1 + a_2X_2$$&#xA;$$F_2(X_1, X_2) = b_1X_1 + b_2X_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would prefer to do this in R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks again&lt;/p&gt;&#xA;" OwnerUserId="9370" LastEditorUserId="276" LastEditDate="2014-06-03T21:42:56.080" LastActivityDate="2014-06-03T21:42:56.080" Title="How do I simultaneously minimize two different functions who have the same inputs?" Tags="|optimization|linear-solver|linear-programming|r|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11818" PostTypeId="2" ParentId="11817" CreationDate="2014-06-03T21:40:58.407" Score="5" Body="&lt;p&gt;The problem posed is a multiobjective optimization problem, and the usual notion of optimality for these types of problems is Pareto optimality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scalarization (as proposed in the comments by ChristianClason, TheNobleSunfish, Paul, and DougLipinski) is one way to solve the problem. This approach leverages the large body of theory and algorithms for single objective optimization problems, at which point R packages for single objective optimization could be used. A list of these packages can be found &lt;a href=&quot;http://cran.r-project.org/web/views/Optimization.html&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other methods for solving multiobjective optimization problems. I haven't had more than a single class on multiobjective optimization methods, so I don't claim to be an expert, but I would look at that literature for details on solution methods. In terms of R packages, you might try looking at &lt;a href=&quot;http://cran.r-project.org/web/packages/mco/&quot;&gt;mco&lt;/a&gt; or &lt;a href=&quot;http://cran.r-project.org/web/packages/emoa/index.html&quot;&gt;emoa&lt;/a&gt;. Both methods appear to use evolutionary algorithms, which at first glance look to belong to the same class of algorithms as genetic algorithms.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-06-03T21:40:58.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11819" PostTypeId="1" CreationDate="2014-06-04T06:27:13.370" Score="0" ViewCount="264" Body="&lt;p&gt;Would you know what is the condition for stability for the advection-diffusion equation where we treat the diffusion part using Crank-Nicholson and the advection part using FCTS (forward in time centered in space)? I am applying von Neumann analysis but I am not sure about the final condition for stability. Do you know where I could find the proof for the stability of that scheme? Thank you&lt;/p&gt;&#xA;" OwnerUserId="8041" LastEditorUserId="8041" LastEditDate="2014-06-06T23:14:25.237" LastActivityDate="2014-06-06T23:14:25.237" Title="advection diffusion equation" Tags="|advection-diffusion|advection|diffusion|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11822" PostTypeId="1" AcceptedAnswerId="11823" CreationDate="2014-06-04T12:00:30.827" Score="3" ViewCount="431" Body="&lt;p&gt;If one wants to solve a problem in physics, one often has to deal with very small numbers because of the units, e.g. the energy range of interest of semiconductors lies in the region $eV \approx 10^{-19}J$. For a given problem one could derive equations that are using scaled properties, e.g. instead of SI units using a different set of units so the numbers in use are not so small. Since deriving equations in scaled units is another source of error I am interested if there are any drawbacks (speed, precision,...) when computing with SI units and therefore very small numbers compared to scaled units and larger numbers?&lt;/p&gt;&#xA;" OwnerUserId="7040" LastActivityDate="2014-06-04T15:55:53.403" Title="Are scaled equations still needed?" Tags="|precision|scaling|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11823" PostTypeId="2" ParentId="11822" CreationDate="2014-06-04T12:03:18.403" Score="8" Body="&lt;p&gt;It is the difference in scales between terms in your equations that tend to cause numerical difficulties. You may work in any units you like as long as you are consistent. My approach has been to always consistently non-dimensionalize my equations in order to reduce the number of parameters to the minimum required, but this is only for my convenience.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-06-04T12:03:18.403" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11824" PostTypeId="2" ParentId="11822" CreationDate="2014-06-04T13:14:49.533" Score="6" Body="&lt;p&gt;In contrast to Bill Barth, I usually try to keep things in dimensional form. Within a single equation, this of course does not change the relative scaling of terms. However, not doing the scaling requires that one pays attention to the relative scaling between different equations of a system of equations. A discussion of one case we have documented can be found in &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications/2008-boussinesq.pdf&quot;&gt;this paper, section 3.2.4&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-06-04T13:14:49.533" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11825" PostTypeId="2" ParentId="11822" CreationDate="2014-06-04T15:55:53.403" Score="3" Body="&lt;p&gt;In most cases, it does not matter what system of units you use.&#xA;However, physics deals with quite small an quite large numbers (in SI), and particularly if you're using single-precision floating points you can get into trouble very quickly. For instance, the electron charge squared is very close to what can be represented. In these cases using units in which your quantities are close to 1 makes sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why would you use single-precision floating points you ask? Speed! HPC is often done using single-precision for instance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to say that I don't think using scaled units will increase the number of bugs you see. In my field, the common unit of energy is $MeV$ and the common length unit is $cm$ which is usually quite convenient. The trick is to only do the unit conversion once, if you really need the SI quantity. If you're worried about converting physical constants, most are typically quoted in a number of unit systems, depending on how they're most often used. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-06-04T15:55:53.403" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11827" PostTypeId="1" AcceptedAnswerId="11828" CreationDate="2014-06-04T23:43:11.327" Score="0" ViewCount="1877" Body="&lt;p&gt;LAPACK has following 4 routines for calculating eigenvalues of  a real symmetric matrix; namely DSYEV, DSYEVD, DSYEVX and DSYEVR (DSYEVR being the recommended one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were to calculate both eigenvalues and all the eigenvectors, how many floating point operations would each of the routine take. Are all these routines are based on tridiagonalization?  &lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2014-06-05T00:49:31.333" Title="Flop counts for LAPACK symmetric eigenvalue routines DSYEV, DSYEVD, DSYEVX and DSYEVR" Tags="|linear-algebra|eigenvalues|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11828" PostTypeId="2" ParentId="11827" CreationDate="2014-06-05T00:17:28.217" Score="4" Body="&lt;p&gt;First of all, yes, these are all based on an initial tridiagonalization (often quoted to be $\frac{4}{3}n^3$ flops). DSYEV is just an easier to use version of DSYEVX, so let's ignore it for now. The basic breakdown is such:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;DSYEVX: Tridiagonal implicitly shifted QR (bulge chasing)&lt;/li&gt;&#xA;&lt;li&gt;DSYEVD: Divide and Conquer (stitching by rank 1 modifications)&lt;/li&gt;&#xA;&lt;li&gt;DSYEVR: Use the Multiple Relatively Robust Representations (MRRR) algorithm (symmetric dqds)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The flop count of each of these algorithms is highly dependent on the eigenvalue distribution. For example, if your matrix has clusters of closely packed eigenvalues that are nearly degenerate, then these algorithms tend to have reduced order of convergence. At best you have $O(n^2)$ flops in the tridiagonal eigenproblem, and at worst $O(n^3)$ if the sizes of your clusters go as $O(n)$. See this &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn183.pdf&quot; rel=&quot;nofollow&quot;&gt;working note&lt;/a&gt; for a more detailed analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you want eigenvectors, let's look at the flop count for their extraction. Implicitly shifted QR continually updates the orthogonal matrices used for reduction, so it is always runs in $O(n^3)$ when eigenvectors are desired (basically, each QR bulge sweep takes $O(n^2)$, and you need to $O(n)$ sweeps usually).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For D&amp;amp;C and MRRR, the textbook descriptions say to compute eigenvalues first, then extract eigenvectors with inverse iteration. This requires $O(n)$ work per eigenvector since usually 1-2 iterations is enough, so it would be $O(n^2)$ work to get all the eigenvectors after the eigenvalues have been found. However, in practice, Lapack does something more sophisticated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not familiar with D&amp;amp;C, but look at the source, it looks like it keeps the eigenvectors updated as it goes, so it's runtime is likely somewhere between $O(n^2)$ and $O(n^3)$. For MRRR, the failure of inverse iteration has been analyzed in &lt;a href=&quot;http://www.cs.utexas.edu/users/inderjit/public_papers/invit_fail.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. In practice, the eigenvectors must be updated throughout the algorithm and reorthogonalized when clusters are detected. A few of the details of this process can be controlled in the interface to DSYEVR, and like D&amp;amp;C, the practical runtime is somewhere between quadratic and cubic.&lt;/p&gt;&#xA;" OwnerUserId="317" LastEditorUserId="317" LastEditDate="2014-06-05T00:49:31.333" LastActivityDate="2014-06-05T00:49:31.333" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11829" PostTypeId="1" AcceptedAnswerId="11841" CreationDate="2014-06-05T02:36:38.063" Score="0" ViewCount="679" Body="&lt;p&gt;I am using cvx to solve linear programs with constraints of the form $Ax=b,x\ge0$. However the matrix $A$ is rank deficient and cvx returns a warning and finally displays status as 'Infeasible'. Rank deficient systems can have a solution and my guess is that my system does have a solution. Is there a way to make cvx solve this system without making the matrix full rank?&lt;/p&gt;&#xA;" OwnerUserId="9383" LastActivityDate="2014-06-05T16:17:37.220" Title="Solving rank deficient systems with cvx" Tags="|linear-programming|cvx|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11830" PostTypeId="1" AcceptedAnswerId="11832" CreationDate="2014-06-05T02:51:27.767" Score="7" ViewCount="9461" Body="&lt;p&gt;What type equation Navier-Stokes is: Elliptic, parabolic, or hyperbolic? Should it give always the same answer no matter what is the initial condition? How these statements could be proved?&lt;/p&gt;&#xA;" OwnerUserId="8103" LastActivityDate="2014-06-05T04:08:55.140" Title="Type of Navier-Stokes equation" Tags="|navier-stokes|fluid-dynamics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11831" PostTypeId="2" ParentId="11829" CreationDate="2014-06-05T03:08:02.347" Score="1" Body="&lt;p&gt;Depending on the particular $b$ vector that you have, it may be that the system of equations is infeasible or that there are solutions to $Ax=b$ but no solutions with $x \geq 0$.  The solvers used with CVX are perfectly capable of detecting either type of LP infeasibility, so it's most likely the case that your guess is wrong and that the LP actually is infeasible.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that even if $A$ has full row rank and the system of equations $Ax=b$ is feasible, that doesn't mean that the LP including the constraints $x \geq 0$ must have a solution.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2014-06-05T03:47:00.073" LastActivityDate="2014-06-05T03:47:00.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11832" PostTypeId="2" ParentId="11830" CreationDate="2014-06-05T04:08:55.140" Score="19" Body="&lt;p&gt;The question assumes that there is a strict delineation between equations, but there isn't. On paper, of course, the Navier-Stokes equations have a parabolic character because there is a non-zero diffusion term. But, in reality, we say that equations are &quot;hyperbolic&quot; when we mean that they are advection dominated, and &quot;parabolic&quot; when they are diffusion dominated, and the Navier-Stokes equations can be either depending on whether your Reynolds number is large or small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to make this point in lecture #26 at &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; in more detail.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-06-05T04:08:55.140" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="11833" PostTypeId="2" ParentId="11829" CreationDate="2014-06-05T04:16:01.323" Score="0" Body="&lt;p&gt;To expand on Brian's answer: You may think that if the matrix $A$ is rank deficient, then there needs to be a solution of $Ax=b$ with $x\ge 0$ because there is a whole subspace (actually an affine space) of solutions $x=x_0+y, y\in \text{ker}\;A$ that satisfy the equation and surely parts of this affine space must be non-negative in all components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that's not necessarily true. Imagine, for example $x_0=(0,-1)^T, \text{ker}\;A=\{y \in R^2: y=(y_1,0)^T\}$. For example, this would correspond to the linear system with&#xA;$$&#xA;  A = \begin{pmatrix} 0 &amp;amp; 1 \\ 0 &amp;amp; 0 \end{pmatrix},&#xA;  \qquad \qquad &#xA;  b = \begin{pmatrix} -1 \\ 0 \end{pmatrix}.&#xA;$$&#xA;Even though $A$ is rank deficient, the linear system has no non-negative solutions.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-06-05T04:16:01.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11834" PostTypeId="1" AcceptedAnswerId="11843" CreationDate="2014-06-05T04:42:59.423" Score="7" ViewCount="2101" Body="&lt;p&gt;What are the best FGMRES implementations in various languages/frameworks? In particular, are there any good quality Matlab implementations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am referring to the variation of GMRES where a changing or nonlinear preconditioner is allowed by keeping track of the preconditioned vectors as well, as discussed in Saad's paper,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Saad, Youcef. &quot;A flexible inner-outer preconditioned GMRES algorithm.&quot; SIAM Journal on Scientific Computing 14.2 (1993): 461-469. &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.15.4854&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.15.4854&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It looks pretty simple to implement, but I know there can sometimes be numerical problems with Krylov methods if implemented naively (eg., loss of orthogonality in CG).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;Here's my unsophisticated MATLAB implementation that seems to work fine, in case anyone reading this thread needs a matlab version that works but isn't highly optimized.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [x,X] = fgmres(A,b,m,tol,maxit,P,x0)&#xA;%Flexible GMRES (FGMRES) with restart length m.&#xA;%Solves Ax = b to tolerance tol using preconditioner P and initial guess x0&#xA;%If P is a function handle, P(A) =approx= I.&#xA;%If P is a matrix, P\A =approx= I.&#xA;%Optional output X is the whole sequence of FGMRES iterates&#xA;&#xA;%Written by Nick Alger 6/10/2014, released to public domain.&#xA;%Reference:&#xA;%Saad, &quot;A flexible inner-outer preconditioned GMRES algorithm&quot;, SIAM 1993&#xA;&#xA;%Parse input&#xA;N = size(b,1);&#xA;&#xA;if (isa(A,'numeric'))&#xA;    if (size(A,1) == N &amp;amp;&amp;amp; size(A,2) == N)&#xA;        Afct = @(u) A*u;&#xA;    else&#xA;        error(['Dimensions of A and b in Ax = b are not consistent: '...&#xA;            'A is ', num2str(size(A,1)),'-by-', num2str(size(A,2)),', and '...&#xA;            'b is ', num2str(size(b,1)),'-by-', num2str(size(b,2)),'.']);&#xA;    end&#xA;elseif (isa(A,'function_handle'))&#xA;    Afct = A;&#xA;else&#xA;    error('Forward operator needs to be a matrix or function handle.')&#xA;end&#xA;&#xA;if (nargin &amp;lt; 3 || isempty(m))&#xA;    m = 50; %Default restart&#xA;end&#xA;&#xA;if (nargin &amp;lt; 4 || isempty(tol))&#xA;    tol = 1e-6;    &#xA;end&#xA;&#xA;if (nargin &amp;lt; 5 || isempty(maxit))&#xA;    maxit = min(N,200);&#xA;end&#xA;&#xA;if (nargin &amp;lt; 6 || isempty(P))&#xA;    Pfct = @(u) u;&#xA;elseif (isa(P,'numeric'))&#xA;    Pfct = @(u) P\u;&#xA;elseif (isa(P,'function_handle'))&#xA;    Pfct = P;&#xA;else&#xA;    error('Preconditioner needs to be a matrix or function handle.')&#xA;end&#xA;&#xA;if (nargin &amp;lt; 7 || isempty(x0))&#xA;    x0 = zeros(size(b));&#xA;end&#xA;&#xA;%Run FGMRES&#xA;X = [];&#xA;nit = 1;&#xA;while (nit &amp;lt;= maxit)&#xA;    disp('Starting new FGMRES cycle')&#xA;    H = zeros(m+1,m);&#xA;    r0 = b - Afct(x0);&#xA;    beta = norm(r0);&#xA;    V = zeros(N, m+1);&#xA;    V(:,1) = (1/beta)*r0;&#xA;    Z = zeros(N,m);&#xA;    for j=1:m&#xA;        Z(:,j) = Pfct(V(:,j));&#xA;        w = Afct(Z(:,j));&#xA;        for i=1:j&#xA;            H(i,j) = w'*V(:,i);&#xA;            w = w - H(i,j)*V(:,i);&#xA;        end&#xA;        H(j+1, j) = norm(w);&#xA;        V(:, j+1) = (1/H(j+1, j))*w;&#xA;&#xA;        e1 = zeros(j+1,1); e1(1) = 1;&#xA;        y = H(1 : j+1, 1:j)\(beta*e1);&#xA;        x = x0 + Z(:, 1:j)*y;&#xA;        if (nargout &amp;gt; 1)&#xA;            X = [X,x];&#xA;        end&#xA;        resnorm = norm(b-Afct(x));&#xA;        disp(['k= ', num2str(nit), ...&#xA;            ', relative residual= ', num2str(resnorm/norm(b))])&#xA;        if (resnorm &amp;lt; tol*norm(b))&#xA;            disp(['FGMRES converged to relative tolerance ', ...&#xA;                num2str(resnorm/norm(b)), ...&#xA;                ' at iteration ', ...&#xA;                num2str(nit)])&#xA;            return&#xA;        end&#xA;&#xA;        nit = nit + 1;&#xA;        if (nit &amp;gt; maxit)&#xA;            break;&#xA;        end&#xA;    end&#xA;    x0 = x;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2014-06-10T21:00:40.013" LastActivityDate="2014-06-10T21:00:40.013" Title="High quality flexible GMRES (FGMRES) implementation" Tags="|preconditioning|gmres|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11836" PostTypeId="1" CreationDate="2014-06-05T09:06:46.757" Score="9" ViewCount="743" Body="&lt;p&gt;I wish to model a fishing rod (or a rope) by joining short segments. (The segments may have equal (short) length but each segment should be assigned its own individual mass.) One segment will influence the next by the torque between the segments. For the time being the joints can be regarded as plate springs (torque proportional to bending angle (a or alfa), individual k for each joint).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I apply torque to the first segment (the &quot;handle&quot;), the torque will spread to the rest of the segments.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I do not understand how to compute the movements that will occur at segment one (with mass m1) and the following segments, when I apply torque T1 to segment one (during time dt).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/ze7g6dzrzzd6757/DSC_0113.JPG&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/s/ze7g6dzrzzd6757/DSC_0113.JPG&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a (retired) medical doctor with interest in biomechanics, so please use only basic physical terminology. (I wish to migrate the model to biomechanical use. I have written computer programs for models before, so hopefully I can manage that part if I just get the motion equations straight.)&lt;/p&gt;&#xA;" OwnerUserId="9385" OwnerDisplayName="ycc_swe" LastEditorUserId="9385" LastEditDate="2014-06-19T17:33:42.110" LastActivityDate="2014-06-19T17:33:42.110" Title="How to model a fishing rod (or a rope)?" Tags="|ode|modeling|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="11839" PostTypeId="2" ParentId="11836" CreationDate="2014-06-05T14:19:54.877" Score="7" Body="&lt;p&gt;To solve this problem as you have described it, you need to set up a simple system of ordinary differential equations. For each segment in your &quot;fishing rod&quot; you just need to use conservation of linear and angular momentum ($F=ma$ and $\tau = \frac{dL}{dt}$). Each segment will experience forces and torques from its neighbors. There are many ways to formulate this. And many techniques to solve the resulting system of ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a starting point, I would suggest attacking a simpler problem that will give you an idea of what's required: a &lt;a href=&quot;http://en.wikipedia.org/wiki/Double_pendulum&quot;&gt;double pendulum&lt;/a&gt;. There are many online demonstrations that solve the double pendulum problem including a detailed discussion of the math &lt;a href=&quot;http://scienceworld.wolfram.com/physics/DoublePendulum.html&quot;&gt;here&lt;/a&gt;, a Flash implementation &lt;a href=&quot;http://thinkcreative.host22.com/flash/double_pendulum/&quot;&gt;here&lt;/a&gt;, a javascript version &lt;a href=&quot;http://jsdo.it/aont/9svb&quot;&gt;here&lt;/a&gt;, and a MATLAB version &lt;a href=&quot;http://blogs.mathworks.com/pick/2010/08/13/animating-double-pendulum/&quot;&gt;here&lt;/a&gt;. Also, some implementations place masses only at the joints while others have the mass distributed evenly along the segments so you might focus on the one you prefer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you understand the double pendulum problem, it can be very easily extended to any number of segments. Adding a force at a given segment just means adding an additional force term to the acceleration equation for that segment and is very easy to achieve. The last step for your problem would be to include torques via conservation of angular momentum. I suggest implementing everything you need up to that point and then come back and ask more specific questions about implementing the torques if you need help once you're there.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-06-05T14:19:54.877" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="11841" PostTypeId="2" ParentId="11829" CreationDate="2014-06-05T16:17:37.220" Score="2" Body="&lt;p&gt;It's known that CVX itself, and to an extent the solvers it uses, have issues with rank deficiency in the equation matrix. (Hence the warnings.) But what is your aversion to doing some sort of LU factorization here? Also, have you tried all of the solvers, or just one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is to solve a model that is guaranteed to be feasible. For instance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cvx_begin&#xA;    variables x(n)&#xA;    minimize(norm(A*x-b))&#xA;    x &amp;gt;= 0&#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If your original problem is feasible, this should have an optimal value that is near zero (to within roundoff error). If it has a non-negligible, positive optimal value, then your original model was infeasible.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastActivityDate="2014-06-05T16:17:37.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11842" PostTypeId="2" ParentId="10882" CreationDate="2014-06-06T03:45:29.737" Score="1" Body="&lt;p&gt;In addition to Jed's excellent answer, a method I've found recently is to switch between the preconditioners every other step in flexible GMRES (FGMRES), as is done, for example, in &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Tezduyar, T. E., et al. &quot;A new mixed preconditioning method for finite element computations.&quot; Computer Methods in Applied Mechanics and Engineering 99.1 (1992): 27-42. &lt;a href=&quot;http://repository.ias.ac.in/24680/1/320.pdf&quot; rel=&quot;nofollow&quot;&gt;http://repository.ias.ac.in/24680/1/320.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2014-06-06T03:45:29.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11843" PostTypeId="2" ParentId="11834" CreationDate="2014-06-06T03:53:11.690" Score="5" Body="&lt;p&gt;Both PETSc and Trilinos have high quality implementations. Both of these libraries also have bindings to other languages.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-06-06T03:53:11.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11846" PostTypeId="1" CreationDate="2014-06-06T20:04:18.477" Score="4" ViewCount="390" Body="&lt;p&gt;I am trying to solve the 2D Poisson equation numerically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial ^2 \phi}{\partial x^2} + \frac{\partial ^2 \phi}{\partial y^2} = 1 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the Dirichlet boundary condition $\phi = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the finite element and finite difference approach on a regular grid of 6 × 6 points. The model size is 1 × 1 m. The results appear to be quite different of both methods (I also tested larger grids with similar differences between both methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Results finite element: $\phi = $&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 0.          0.          0.          0.          0.          0.        ]&#xA; [ 0.         -0.03631579 -0.04973684 -0.04973684 -0.03631579  0.        ]&#xA; [ 0.         -0.04973684 -0.07105263 -0.07105263 -0.04973684  0.        ]&#xA; [ 0.         -0.04973684 -0.07105263 -0.07105263 -0.04973684  0.        ]&#xA; [ 0.         -0.03631579 -0.04973684 -0.04973684 -0.03631579  0.        ]&#xA; [ 0.          0.          0.          0.          0.          0.        ]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results finite difference: $\phi = $&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 0.          0.          0.          0.          0.          0.        ]&#xA; [ 0.         -0.03333333 -0.04666667 -0.04666667 -0.03333333  0.        ]&#xA; [ 0.         -0.04666667 -0.06666667 -0.06666667 -0.04666667  0.        ]&#xA; [ 0.         -0.04666667 -0.06666667 -0.06666667 -0.04666667  0.        ]&#xA; [ 0.         -0.03333333 -0.04666667 -0.04666667 -0.03333333  0.        ]&#xA; [ 0.          0.          0.          0.          0.          0.        ]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Furtheremore, if I set a no-flux boundary condition on the left side, the results for the first and second column of the grid are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Results finite element: $\phi = $&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 0.          0.        ]&#xA; [-0.07453021 -0.0730567 ]&#xA; [-0.11114969 -0.10876554]&#xA; [-0.11114969 -0.10876554]&#xA; [-0.07453021 -0.0730567 ]&#xA; [ 0.          0.        ]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results finite difference: $\phi = $&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 0.          0.        ]&#xA; [-0.06977283 -0.06977283]&#xA; [-0.1034833  -0.1034833 ]&#xA; [-0.1034833  -0.1034833 ]&#xA; [-0.06977283 -0.06977283]&#xA; [ 0.          0.        ]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results of the finite difference method show there is actually a zero flux at the left boundary. However, this is not the case for the finite element approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From these results, I get the impression that the finite difference is more accurate. Is this correct (in general)? Or have I implemented the numerical approach incorrectly (see code below)? Since both methods are based on different assumptions, I expeted different results. However, these results seem to be &lt;em&gt;too&lt;/em&gt; different to be true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, I provide the Python code I implemented to solve the Poisson equation using finite elements and finite differences.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##################################################################&#xA;### IMPORT ###&#xA;##################################################################&#xA;from numpy import zeros,sqrt,dot,transpose,sqrt&#xA;from numpy.linalg import det,inv&#xA;from scipy.sparse.linalg import spsolve&#xA;from scipy.sparse import csc_matrix&#xA;&#xA;##################################################################&#xA;### SETUP ###&#xA;##################################################################&#xA;nnx = 6 # number of nodes - x axis&#xA;nny = 6 # number of nodes - y axis&#xA;np = nnx*nny # total number of nodes&#xA;&#xA;nelx = nnx-1 # number of elements - x axis&#xA;nely = nny-1 # number of elements - y axis&#xA;nel = nelx*nely # total number of elements&#xA;&#xA;Lx = 1.0 # x axis goes from 0 to Lx&#xA;Ly = 1.0 # x axis goes from 0 to Lx&#xA;&#xA;xstp    =   Lx/(nnx-1) # x step size&#xA;ystp    =   Ly/(nnx-1) # y step size&#xA;&#xA;x=zeros((np,1))&#xA;y=zeros((np,1))&#xA;ind=-1&#xA;for j in range(nny):&#xA;    for i in range(nnx):&#xA;        ind=ind+1&#xA;        x[ind,0]=i*Lx/nelx&#xA;        y[ind,0]=j*Ly/nely&#xA;&#xA;##################################################################&#xA;#****************************************************************#&#xA;#FINITE ELEMENT APPROACH&#xA;#****************************************************************#&#xA;##################################################################&#xA;&#xA;##################################################################&#xA;### CONNECTIVITY OF NODES FOR EACH ELEMENT ###&#xA;##################################################################&#xA;icon=zeros((4,nel))&#xA;ind=-1&#xA;eind=-1&#xA;for j in range(nny):&#xA;    for i in range(nnx):&#xA;        ind=ind+1&#xA;        if j==nny-1 or i==nnx-1:&#xA;            continue&#xA;        eind += 1&#xA;        icon[0,eind]=ind&#xA;        icon[1,eind]=ind+1&#xA;        icon[2,eind]=ind+1+nnx&#xA;        icon[3,eind]=ind+nnx&#xA;&#xA;##################################################################&#xA;### BOUNDARY CONDITIONS SETUP ####&#xA;##################################################################&#xA;bc_fix=zeros((np,1))&#xA;bc_val=zeros((np,1))&#xA;for i in range(np):&#xA;    if x[i,0]==0.0:&#xA;        bc_fix[i,0] = 1&#xA;        bc_val[i,0] = 0.0&#xA;    if y[i,0]==0.0:&#xA;        bc_fix[i,0] = 1&#xA;        bc_val[i,0] = 0.0&#xA;    if x[i,0]==Lx:&#xA;        bc_fix[i,0] = 1&#xA;        bc_val[i,0] = 0.0&#xA;    if y[i,0]==Ly:&#xA;        bc_fix[i,0] = 1&#xA;        bc_val[i,0] = 0.0&#xA;&#xA;##################################################################&#xA;### ASSEMBLY ###&#xA;##################################################################&#xA;A = zeros((np,np)) # GLOBAL MATRIX - LHS&#xA;B = zeros((np,1)) # GLOBAL MATRIX  - RHS&#xA;&#xA;# Weights for quadtratic integrations&#xA;wgts = [1.0]*4&#xA;# Integration points&#xA;intpt_x = [-1.0/sqrt(3),-1.0/sqrt(3), 1.0/sqrt(3), 1.0/sqrt(3)] &#xA;intpt_y = [-1.0/sqrt(3), 1.0/sqrt(3),-1.0/sqrt(3), 1.0/sqrt(3)]&#xA;&#xA;for iel in range(nel): # loop over each element&#xA;    Ael=zeros((4,4)) # element matrix&#xA;    Bel=zeros((4,1)) # element matrix&#xA;&#xA;    for i in range(4): # loop over each integration point&#xA;        wq=wgts[i]&#xA;        rq=intpt_x[i]&#xA;        sq=intpt_y[i]&#xA;&#xA;        # Shape Function&#xA;        N = zeros((4,1))        &#xA;        N[0,0]=0.25*(1.0-rq)*(1.0-sq)&#xA;        N[1,0]=0.25*(1.0+rq)*(1.0-sq)&#xA;        N[2,0]=0.25*(1.0+rq)*(1.0+sq)&#xA;        N[3,0]=0.25*(1.0-rq)*(1.0+sq)&#xA;&#xA;        # Shape function derivatives&#xA;        dNdrs = zeros((4,2))        &#xA;        dNdrs[0,0] = - 0.25*(1.0-sq)  &#xA;        dNdrs[1,0] = + 0.25*(1.0-sq)     &#xA;        dNdrs[2,0] = + 0.25*(1.0+sq)      &#xA;        dNdrs[3,0] = - 0.25*(1.0+sq) &#xA;&#xA;        dNdrs[0,1] = - 0.25*(1.0-rq)&#xA;        dNdrs[1,1] = - 0.25*(1.0+rq)&#xA;        dNdrs[2,1] = + 0.25*(1.0+rq)&#xA;        dNdrs[3,1] = + 0.25*(1.0-rq)&#xA;&#xA;        # Calculate Jacobian&#xA;        cord = zeros((2,4)) # cordinates of element     &#xA;        for j in range(4):&#xA;            cord[0,j]   =   x[icon[j,iel]] &#xA;            cord[1,j]   =   y[icon[j,iel]]  &#xA;        J   =   dot(cord,dNdrs) # jacobian  &#xA;        detJ    =   det(J) # determinant&#xA;        invJ    =   inv(J) # inverse jacobian&#xA;&#xA;        # Local Derivatives&#xA;        dNdrs_l =   dot(dNdrs,invJ)&#xA;&#xA;        # Create Element Matrix&#xA;        Ael     -=  dot(dNdrs_l,transpose(dNdrs_l))*detJ*wq&#xA;        Bel +=  N*detJ*wq&#xA;&#xA;        # Update Global Matrix&#xA;        for k1 in range(4):         &#xA;            ik1=icon[k1,iel]            &#xA;            for k2 in range(4):&#xA;                    ik2=icon[k2,iel]&#xA;                A[ik1,ik2] += Ael[k1,k2]&#xA;&#xA;            B[ik1]=B[ik1]+Bel[k1]&#xA;&#xA;##################################################################&#xA;# SET BOUNDARY CONDITIONS&#xA;##################################################################&#xA;for i in range(np):&#xA;    if bc_fix[i] == 1:&#xA;        for j in range(np):&#xA;                B[j]=B[j]-A[i,j]*bc_val[i]&#xA;            A[i,j]=0.0&#xA;            A[j,i]=0.0&#xA;&#xA;            A[i,i]=1.0&#xA;            B[i]=bc_val[i]&#xA;&#xA;##################################################################&#xA;# SOLVE ...&#xA;##################################################################&#xA;A = csc_matrix(A)&#xA;S_fe = spsolve(A,B)&#xA;&#xA;S_fe = S_fe.reshape(nny,nnx)&#xA;&#xA;print S_fe&#xA;&#xA;##################################################################&#xA;#****************************************************************#&#xA;#FINITE DIFFERENCE APPROACH&#xA;#****************************************************************#&#xA;##################################################################&#xA;A = zeros((np,np)) &#xA;B = zeros((np,1)) &#xA;&#xA;k = -1&#xA;for i in range(nny):&#xA;    for j in range(nnx):&#xA;            k += 1&#xA;        if i==0 or i==nny-1 or j==0 or j==nnx-1:&#xA;                A[k,k] = 1.0&#xA;                B[k,0] = 0.0&#xA;        else:&#xA;                A[k,k-nny]  = 1.0/xstp**2&#xA;            A[k,k-1  ]  = 1.0/ystp**2&#xA;            A[k,k    ]  = -2.0/xstp**2 - 2.0/ystp**2&#xA;            A[k,k+1  ]  = 1.0/ystp**2&#xA;            A[k,k+nny]  = 1.0/xstp**2&#xA;            B[k,0    ]  = 1.0&#xA;&#xA;##################################################################&#xA;# SOLVE ...&#xA;##################################################################&#xA;A = csc_matrix(A)&#xA;S_fd = spsolve(A,B)&#xA;&#xA;S_fd = S_fd.reshape(nnx,nny)&#xA;&#xA;print S_fd&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9403" LastEditorUserId="9403" LastEditDate="2014-06-07T16:19:48.960" LastActivityDate="2014-06-07T16:19:48.960" Title="Finite differences vs. elements: Accuracy and implementation" Tags="|finite-element|finite-difference|poisson|accuracy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11847" PostTypeId="2" ParentId="11846" CreationDate="2014-06-06T20:41:47.580" Score="6" Body="&lt;p&gt;For a 6x6 grid, those are about the error differences I would expect from two different methods.  You have to realize that a 6x6 grid is a very coarse grid, even for a simple problem like yours.  As long as you see the two solutions converge towards each other as you refine your grid, there is likely no implementation error.  Finite-difference has no general advantage over finite-element, although in some problems it may be more accurate up to a constant factor (and the opposite is true for other problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the Neumann condition, if you formulated it correctly in your FE case, then you may not see column 1 equal to column 2.  What you should actually check is that the approximating functions defined over your elements have zero derivative at the boundary.  However, if you are using first order elements (which I am assuming you are), then your function are linear and my intuition would be that the two columns would be equal.  You will just have to check exactly how you implemented your boundary condition.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2014-06-07T04:06:16.887" LastActivityDate="2014-06-07T04:06:16.887" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="11850" PostTypeId="1" CreationDate="2014-06-07T15:52:17.293" Score="2" ViewCount="222" Body="&lt;p&gt;I want to do a 3-dimensional FFT on this function $\frac{\cos (x) \cos (y) \cos (z)-\sin (x) \sin (y) \sin (z)}{\left((1.0001+\sin (y)+\cos (z))^2+(0.0001+\cos (x)+\sin (z))^2+(0.0001+\sin (x)+\cos (y))^2\right)^{3/2}}$ for it looks intractable via analytical Fourier expansion. Let's denote the number of numerical sampling points in each dimension as $N$. Here's the &lt;em&gt;Mathematica&lt;/em&gt; code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nn = 10; step = (2 \[Pi])/nn; mx0 = 1.0001; my0 = 0.0001; mz0 = 0.0001; &#xA;data = Table[ ( Cos[x] Cos[y] Cos[z] - Sin[x] Sin[y] Sin[z])/((mz0 + Cos[y] + Sin[x])^2 + (mx0 + Cos[z] + Sin[y])^2 + (my0 + Cos[x] + Sin[z])^2)^(3/2), {x, 0, 2 \[Pi] - step, step}, {y, 0, 2 \[Pi] - step, step}, {z, 0, 2 \[Pi] - step, step}];&#xA;s = Fourier[data, FourierParameters -&amp;gt; {-1, -1}]; s[[1, 1, 1]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as I've tried, using Fast Fourier Transform routine from either C++ MKL library or &lt;em&gt;Mathematica&lt;/em&gt;, the transformation result doesn't converge even when $N=500$, oscillating extravagantly with respect to $N$ in fact. I checked the programs with many other non-singular functions, they turned out to be good. So I guess the problem may be caused by the special form of the singular function (denominator can be zero at some points). I tried $\frac{1}{0.9-\sin{(x+y+z)}}$. It doesn't oscillate so much, but still considerably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone shed some light on this problem? Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="9408" LastEditorUserId="9408" LastEditDate="2014-06-07T16:28:18.990" LastActivityDate="2014-06-08T00:22:59.050" Title="How to do Fast Fourier transform (FFT) for singular functions?" Tags="|fourier-analysis|fourier-transform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="11851" PostTypeId="2" ParentId="11850" CreationDate="2014-06-08T00:22:59.050" Score="4" Body="&lt;p&gt;It's a question of function spaces. If your function is in $L^2$, then you know that you can approximate the function using a Fourier series because the Fourier basis (i.e., the sines and cosines) are dense in $L^2$. In other words, you know that if you take more and more terms of the series, then you will approximate the function better and better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if your function is so singular that it doesn't belong to $L^2$, then all hope is lost: you cannot expect the Fourier series to converge.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-06-08T00:22:59.050" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="11854" PostTypeId="1" AcceptedAnswerId="12868" CreationDate="2014-06-08T16:51:15.783" Score="1" ViewCount="161" Body="&lt;p&gt;I want to model a fishing rod and received a &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11836/how-to-model-a-fishing-rod-or-a-rope&quot;&gt;suggestion&lt;/a&gt;. I therefore try to follow the mathematics (Lagrange) of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Double_pendulum#Lagrangian&quot; rel=&quot;nofollow noreferrer&quot;&gt;double pendulum&lt;/a&gt;. I do not understand how to proceed in the step that Wikipedia calls &quot;Substituting the coordinates above and rearranging the equation gives ...&quot;.&#xA;What &lt;a href=&quot;https://www.dropbox.com/s/z5dyjtsgnfj9rvy/DSC_0140.JPG&quot; rel=&quot;nofollow noreferrer&quot;&gt;I get&lt;/a&gt; is totally wrong.&#xA;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;17/6 EDIT&lt;/strong&gt;:&#xA;This question has been marked as &quot;on hold&quot;. I therefore rephrase it like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the computations of the Lagrange for the &lt;a href=&quot;http://en.wikipedia.org/wiki/Double_pendulum#Lagrangian&quot; rel=&quot;nofollow noreferrer&quot;&gt;double pendulum&lt;/a&gt;, I do not understand how to obtain &lt;a href=&quot;https://www.dropbox.com/s/p4yjr6ps7a6psy3/Lag_res.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;this equation&lt;/a&gt; from the preceeding equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The question has been solved by the answer below.)&lt;/p&gt;&#xA;" OwnerUserId="9385" LastEditorUserId="276" LastEditDate="2017-10-10T00:38:24.417" LastActivityDate="2017-10-10T00:38:24.417" Title="Trigonometric equation (Lagrange) for the double pendulum" Tags="|computational-physics|" AnswerCount="1" CommentCount="1" ClosedDate="2014-06-12T15:19:06.727" ContentLicense="CC BY-SA 3.0" />
  <row Id="12855" PostTypeId="1" CreationDate="2014-06-09T09:49:05.773" Score="7" ViewCount="2908" Body="&lt;p&gt;If I want to create a high quality figure to illustrate a scientific concept or experiment I usually use &lt;code&gt;tikz&lt;/code&gt; together with &lt;code&gt;pgfplots&lt;/code&gt; or &lt;code&gt;pstricks&lt;/code&gt;. However this is for static content. Both tools are extremely powerfull for it's purpose and give high quality results. Furhtermore there is a large database of examples, very good documentation and a huge, active user community around those projects.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the &quot;equivalent&quot; of this for creating physics animations or simulations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to start with 2d and 3d animations and simulations like found on the following pages (but possibly looking a bit more high quality): &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://jakobvogel.net/legacy/index.php?url=physics/index.xml&quot; rel=&quot;noreferrer&quot;&gt;http://jakobvogel.net/legacy/index.php?url=physics/index.xml&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.surendranath.org/Apps.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.surendranath.org/Apps.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.walter-fendt.de/ph6en/&quot; rel=&quot;noreferrer&quot;&gt;http://www.walter-fendt.de/ph6en/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and then proceed to more complex things when I get how to use it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what would you recommend for this task and why?  I have little bit background in python so I would prefer this but if there is a &quot;killer tool&quot; for this, it would be ok to learn a new language from scratch. Here are the requirements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;free and open source (FOSS) and should run on linux&lt;/li&gt;&#xA;&lt;li&gt;it should be a good interface to recreate the examples from the three cited web sites above (with the potential to make even more complex things)&lt;/li&gt;&#xA;&lt;li&gt;like for tikz in the case of static content there should be good documentation and a large active user base &lt;/li&gt;&#xA;&lt;li&gt;I need something for 2d and also for 3d&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Concerning the difference between animations and simulations, I think that there should be no difference because for a simulation I have to solve a differential equation first (using another library, for example numpy/scipy) and then animating the data. So one could reduce the term &quot;simulation&quot; to &quot;animation&quot; for this question (or did I overlook something?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure I googled for this, but the problem was that got I a very long list of packages and I am not sure where to start. For the static content it was easy as described above, but here I don't see, what would be worth to learn in detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I have a little bit python experience at first I narrowd the search down to this language. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The only tool I found made for this purpose is vpython, which I tried. However the results didn't look very professional compared to what I was used to get for static content with &lt;code&gt;tikz&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then I found some nice plotting tools like &lt;code&gt;mathplotlib&lt;/code&gt;, &lt;code&gt;mayavi&lt;/code&gt;, &lt;a href=&quot;https://github.com/continuumio/bokeh&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;bokeh&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;http://code.enthought.com/chaco/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;chaco&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;http://www.cityinabottle.org/nodebox/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;nodebox&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;http://www.pyqtgraph.org/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;pyqtgraph&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;http://vispy.org/gallery.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;vispy&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;http://mbostock.github.io/protovis/ex/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;protovis&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://code.google.com/p/visvis/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;visvis&lt;/code&gt;&lt;/a&gt; which possibly could be used to make animations but I am not sure if this is a good way since they seem to be more suited for static plotting&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The next category I found were game engines like &lt;code&gt;pygame&lt;/code&gt; or &lt;code&gt;pylet&lt;/code&gt;, &lt;code&gt;panda3d&lt;/code&gt; or &lt;code&gt;python-ogre&lt;/code&gt;. I am not sure if it is a good idea to use a complete gaming environment like this for physics animations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another possibility would be to use just the OpenGl bindings for python directly: &lt;a href=&quot;http://pyopengl.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;pyopengl&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then there is &lt;code&gt;blender&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally there seems to be much specalized tools like &lt;code&gt;pymbs&lt;/code&gt;, &lt;code&gt;pyblocksim&lt;/code&gt; or &lt;code&gt;pyOpTools&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However the most physics animations or simulations I can find on the web are java applets. Most of them closed source so I didn't find out which java libraries were used for them. The only one was &lt;a href=&quot;http://www.um.es/fem/EjsWiki/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;ejs&lt;/code&gt;&lt;/a&gt; which has a graphical user interface but which looks a bit old fashioned. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally I considered using &lt;code&gt;html5&lt;/code&gt; for this.&lt;/p&gt;&#xA;" OwnerUserId="9417" LastEditorUserId="9417" LastEditDate="2014-06-10T09:24:12.110" LastActivityDate="2014-11-26T22:54:11.150" Title="FOSS tools for creating high quality physics animations and simulations in 2d and 3d" Tags="|software|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12856" PostTypeId="2" ParentId="6846" CreationDate="2014-06-09T09:53:19.463" Score="1" Body="&lt;p&gt;Another really nice and simple method is using cellular automatons for discretization. There are plenty of such models, including LBA, FHP and much more. These are really nice since they can provide a realtime simulation on modern computers and can be also nicely paralellized and ran on GPUs. They also have some disadvantages and the results are strongly dependent on the shape of lattice applied. Square lattice is insufficient because it lacks rotational freedom and i.e. von kaarman vortices will be square shaped which is not nice :)&lt;/p&gt;&#xA;" OwnerUserId="9418" LastActivityDate="2014-06-09T09:53:19.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12857" PostTypeId="2" ParentId="12855" CreationDate="2014-06-09T11:51:27.247" Score="3" Body="&lt;p&gt;I have used matplotlib quite a bit though not for animation. However, I think the &lt;a href=&quot;http://matplotlib.org/api/animation_api.html&quot; rel=&quot;nofollow&quot;&gt;matplotlib animation module&lt;/a&gt; allows you to turn any programmatically created sequence of images into an animation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example see &lt;a href=&quot;http://jakevdp.github.io/blog/2012/09/05/quantum-python/&quot; rel=&quot;nofollow&quot;&gt;this animation&lt;/a&gt; of the wavefunction of a particle bouncing off a potential barrier (and partly tunelling). Its creator also wrote a brief &lt;a href=&quot;http://jakevdp.github.io/blog/2012/08/18/matplotlib-animation-tutorial/&quot; rel=&quot;nofollow&quot;&gt;tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5521" LastActivityDate="2014-06-09T11:51:27.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12858" PostTypeId="1" CreationDate="2014-06-09T16:01:00.683" Score="1" ViewCount="129" Body="&lt;p&gt;I am a new learner of Unix, python and Gurobi for MIP calculation. I basically finish reading the quick start guide in Gurobi website but still feel that I need more training to use this combination to automatically write MIP input files. Do anyone have some suggestion on any available trainings (especially in using Unix or Python to automatically generate the input file for Ising model with large grid sizes). Thank you :)&lt;/p&gt;&#xA;" OwnerUserId="7891" LastActivityDate="2014-06-09T16:01:00.683" Title="learning Gurobi for MIP calculation" Tags="|python|mixed-integer-programming|" AnswerCount="0" CommentCount="1" ClosedDate="2014-06-27T18:08:19.277" ContentLicense="CC BY-SA 3.0" />
  <row Id="12859" PostTypeId="1" CreationDate="2014-06-09T23:45:40.370" Score="3" ViewCount="1264" Body="&lt;p&gt;I am trying to fit data to a polynomial using Python - Numpy. The points, with lines sketched above them are as in the &lt;a href=&quot;https://i.stack.imgur.com/H2BkM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;picture&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to fit those points to a polynomial of 4. or 5. degree. However, all I can get is nothing more than a line. The coefficients of other than linear terms are too small. I tried to do that both with Numpy and Scipy. In Numpy, I have used the &lt;code&gt;polyfit&lt;/code&gt; function and followed the example given in &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.polyfit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt; and in Scipy used the &lt;code&gt;optimization.curve_fit&lt;/code&gt; function and followed the page in the comment below. (I cannot post more than two links!). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: why do I get a linear fit for such data? I provide high enough data (about 70 points). And how can I improve my fit? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions are highly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="9389" LastEditorUserId="9389" LastEditDate="2014-06-10T00:07:11.393" LastActivityDate="2014-06-10T00:07:11.393" Title="Polynomial Fitting with Least Squares using Numpy and Scipy" Tags="|optimization|python|regression|least-squares|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="12860" PostTypeId="1" AcceptedAnswerId="12882" CreationDate="2014-06-10T02:31:43.857" Score="18" ViewCount="791" Body="&lt;p&gt;I'm writing a small library for sparse matrix computations as a way to teach myself to make the best use of object-oriented programming. I've worked really hard on having a nice object model, where the parts (sparse matrices and the graphs which describe their connectivity structure) are very loosely coupled. In my own view, the code is much more extensible and maintainable for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's also somewhat slower than if I had used a blunt approach. In order to test the tradeoffs of having this object model, I wrote a new sparse matrix type which broke the encapsulation of the underlying graph to see how much faster that would run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first, it looked pretty bleak; the code I was once proud of ran 60% slower than a version without any elegant software design. But, I was able to make a few low-level optimizations -- inlining a function and changing a loop a tiny bit -- without changing the API at all. With those changes, it's now only 20% slower than the competition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which brings me to my question: &lt;strong&gt;How much of a performance loss should I accept if it means I have a nice object model?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-08-12T06:15:29.983" Title="20% performance penalty for a nice software design" Tags="|linear-algebra|sparse-matrix|oop|" AnswerCount="5" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="12861" PostTypeId="2" ParentId="12855" CreationDate="2014-06-10T03:46:06.323" Score="5" Body="&lt;p&gt;For my work, I tend to have programs output a sequence of still images, and I then convert them to an animated format in a post-processing step. To make a video, I use ffmpeg (&lt;a href=&quot;http://www.ffmpeg.org/&quot;&gt;http://www.ffmpeg.org/&lt;/a&gt;); to make a .gif, I use imagemagick (&lt;a href=&quot;http://www.imagemagick.org/&quot;&gt;http://www.imagemagick.org/&lt;/a&gt;). Both of these tools are easy to script from the command line. There are numerous tutorials available that demonstrate this process for each program.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2014-06-10T03:46:06.323" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12862" PostTypeId="1" AcceptedAnswerId="12863" CreationDate="2014-06-10T06:59:27.403" Score="1" ViewCount="637" Body="&lt;p&gt;I'm working on an image processing project and have written some of my algorithms in OpenCL. At the moment I'm using Cloo. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm at the stage now where I want to optimise my program, as it currently runs too slowly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm also thinking it might be worth investigating some other implementations of OpenCL. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to install &lt;a href=&quot;https://software.intel.com/en-us/vcsource/tools/opencl-sdk&quot; rel=&quot;nofollow&quot;&gt;Intel's SDK&lt;/a&gt; however the installation failed. (I think I may need to get in Intel GPU to use the SDK, I currently am using an NVIDIA Quadro K4000). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;http://supercomputingblog.com/opencl/setting-up-opencl/&quot; rel=&quot;nofollow&quot;&gt;this site&lt;/a&gt; recommending the Intel SDK because of its a mature implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it worth investing in an Intel graphics card for the Intel SDK? &lt;/p&gt;&#xA;" OwnerUserId="4680" LastEditorUserId="1804" LastEditDate="2015-07-23T08:28:51.997" LastActivityDate="2015-07-23T08:28:51.997" Title="Intel's OpenCL SDK for image processing" Tags="|gpu|image-processing|opencl|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12863" PostTypeId="2" ParentId="12862" CreationDate="2014-06-10T08:29:17.313" Score="5" Body="&lt;p&gt;Short answer: No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have a number of options here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your NVIDIA GPU, you will get the best performance by switching to CUDA, rather than OpenCL. You can also upgrade your card, to something like the Geforce Titan, which significanty outperforms the Quadro card for GPU computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My personal opinion is that CUDA is much nicer to work with, has better library support, etc, but there are some disadvantages as well. This includes not being able to use AMD GPUs and ofcourse you would have to reimplement everything. &#xA;CUDA and OpenCL are quite similar however, and translating from OpenCL to CUDA is relatively easy. The opposite is not always the case. &#xA;NVIDIA also has the Performance Primitives (NVPP) library which implements many basic image processing steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting an Intel GPU will NOT improve performance, nor make programming easier. Intel GPUs are generally built for saving power, not for performance. You could consider an Intel Xeon Phi, but from what I understand you would still need to modify your code significantly to get good performance (Disclaimer: I have never programmed a Xeon Phi, so maybe it's less work than I imagine).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is to get an AMD card, and use the AMD libraries for OpenCL. As OpenCL is the default way to do GPU computations on AMD cards (unlike the Xeon Phi and NVIDIAs cards), they have some decent libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing more about your problem, it's difficult to say what you should do. NVIDIA cards are faster for floating point arithmetic, AMD cards are better at integers. I would consider abandoning Cloo, as it has not been updated in a couple of years. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not comfortable with the low-level programming required from CUDA and OpenCL, you could also try something like the &lt;a href=&quot;http://www.accelereyes.com/arrayfire/c/index.htm&quot;&gt;ArrayFire&lt;/a&gt; library, which seems to ease implementation quite a bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you post more about your problem and how you have solved it, maybe someone here will be able to help with optimizing the code.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-06-10T08:29:17.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12864" PostTypeId="1" CreationDate="2014-06-10T10:51:51.380" Score="0" ViewCount="537" Body="&lt;p&gt;I need to create a program that can simulate the optical system of the human eye. The input will be any image, the output would be the image that is projected onto the retina. I need it to be as accurate as possible. I am a little lost at the moment, I have been reading up on optics, and I get that, but on the computational side, I have no idea where to begin. What algorithms can be used to model optical systems physically accurately? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to make the optical system as physically accurate as possible, in relation to the human eye. The system must be able to model eye problems that have to do with optics in the eye, such as myopia or astigmatism. If there is more information necessary to answer, please let me know.&lt;/p&gt;&#xA;" OwnerUserId="6725" LastEditorUserId="6725" LastEditDate="2014-06-10T20:26:53.513" LastActivityDate="2014-06-12T07:30:56.347" Title="How can I create a computational model of the human eye?" Tags="|algorithms|simulation|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="12866" PostTypeId="2" ParentId="12860" CreationDate="2014-06-10T13:56:38.433" Score="4" Body="&lt;p&gt;IMHO a penalty of up to 50% (due to whatever reason) is not that bad. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact I have seen a 0-30% difference in performance just based on the type of compiler. This is for PETSc's sparse MatMult routine on matrices arising from low order FE discretizations.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2014-06-10T13:56:38.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12867" PostTypeId="1" AcceptedAnswerId="12870" CreationDate="2014-06-10T14:19:15.667" Score="7" ViewCount="1848" Body="&lt;p&gt;I want to solve the biharmonic equation numerically, that is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta^2 u=f~~in~~\Omega$$&#xA;$$u=g_1~~on ~~\partial \Omega$$&#xA;$$\frac {\partial u}{\partial n}=g_2~~on ~~\partial \Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Green's formula, we have that if $u$ is smooth enough&#xA; $$\int_{\partial \Omega}\frac {\partial \Delta u}{\partial n}v-\int_{\Omega}\nabla(\Delta u).\nabla v dx&#xA;=\int_{\partial \Omega}\frac {\partial \Delta u}{\partial n}v-\int_{\partial \Omega}\frac {\partial v }{\partial n}\Delta u+\int_{\Omega}\Delta u \Delta v=\int_{\Omega}fv$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the test space is $H^2(\Omega)$, How could I use the given boundary conditions$g_1,g_2$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;how could I find the new terms $\frac {\partial v }{\partial n}$ and $\frac {\partial \Delta u}{\partial n}$&lt;/p&gt;&#xA;" OwnerUserId="8271" LastEditorUserId="8271" LastEditDate="2014-06-10T14:26:05.590" LastActivityDate="2016-11-07T08:51:33.350" Title="biharmonic equation" Tags="|pde|elliptic-pde|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12868" PostTypeId="2" ParentId="11854" CreationDate="2014-06-10T15:23:20.593" Score="1" Body="&lt;p&gt;It finally &lt;a href=&quot;https://www.dropbox.com/s/v5d0vjz2x3lgmeq/DSC_0152.JPG&quot; rel=&quot;nofollow noreferrer&quot;&gt;worked out&lt;/a&gt;.&#xA;I had to construct the derivatives in a sketch. (They were wrong in my previous attempt.) I am sure there is a mathematical way of finding them, but I don't know that one. (OK. the derivative of sine is cosine, but it is more to it than that.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway. If you add several more segments to the pendulum (and then add plate springs), the equations will become very complex, in my opinion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any further suggestions how to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11836/how-to-model-a-fishing-rod-or-a-rope?lq=1&quot;&gt;model a fishing rod (in 2D)&lt;/a&gt; using a series of rod segments connected by plate springs are appreciated, either using this approach (Lagrange, suggesting ideas how to realize the computations) or other approach. Here is the &lt;a href=&quot;https://www.dropbox.com/s/x8te28f5z9hzp6e/20140607_171550.mp4&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulation&lt;/a&gt; of the pendulum.&lt;/p&gt;&#xA;" OwnerUserId="9385" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-06-17T15:06:36.640" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12869" PostTypeId="1" AcceptedAnswerId="12876" CreationDate="2014-06-10T16:25:10.923" Score="4" ViewCount="2418" Body="&lt;p&gt;I have a Voronoi diagram that I need to calculate the area of each cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This Voronoi diagram is produced by Voronoi command in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the vertices of the Voronoi cell I use &lt;code&gt;Voronoin&lt;/code&gt; command in MATLAB, but the problem is that the first row of the v is &lt;code&gt;(inf, inf)&lt;/code&gt;, so the area of all voronoi cells near the boundary of domain are inf, but I need to find the actual area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the area of the domain be $|D|$ and area of the voronoi cells be $|D_i|$, I need to find area such that $\sum_i |D_i|=|D|$&lt;/p&gt;&#xA;" OwnerUserId="8271" LastEditorUserId="549" LastEditDate="2014-06-11T13:33:42.593" LastActivityDate="2014-06-25T13:01:14.670" Title="area of voronoi cell" Tags="|computational-geometry|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="12870" PostTypeId="2" ParentId="12867" CreationDate="2014-06-10T20:21:34.017" Score="10" Body="&lt;p&gt;The biharmonic equation is the Euler-Lagrange equation of the Laplacian energy $\frac{1}{2} \langle \Delta u,\Delta u \rangle$. A systematic approach to discretize higher order problems is to convert the unconstrained problem to a constrained problem: Minimize $\frac{1}{2} \langle v,v\rangle$ s.t. $\Delta u=v$; that is,&#xA;\begin{equation}&#xA;\frac{1}{2} \langle v,v\rangle +\langle \lambda, \Delta u -v\rangle \\&#xA;\frac{1}{2} \langle v,v\rangle + \langle \lambda,\frac{\partial u}{\partial n}\rangle -\langle \lambda,v\rangle  -\langle  \nabla \lambda,  \nabla v\rangle&#xA;\end{equation}&#xA;where we used Green's identity for the second equation and $\frac{\partial }{\partial n}$ is the normal derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The new optimization problem can be discretized using piecewise linear elements as is commonly done for second order problems. This is the common thread in mixed finite-element discretizations. &lt;/p&gt;&#xA;" OwnerUserId="9378" LastEditorUserId="27" LastEditDate="2014-06-11T03:06:12.597" LastActivityDate="2014-06-11T03:06:12.597" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="12872" PostTypeId="2" ParentId="12864" CreationDate="2014-06-10T21:37:28.863" Score="1" Body="&lt;p&gt;Ah now we are getting somewhere. You need to have a firm grasp of focus length. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out this link on some of the math. &lt;a href=&quot;http://hyperphysics.phy-astr.gsu.edu/hbase/geoopt/lenseq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hyperphysics.phy-astr.gsu.edu/hbase/geoopt/lenseq.html&lt;/a&gt; . To get an idea of this I would use a DSLR with a manual focus lens and set it to the focal length as close as you can. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this with code? I don't know. But here's what I think.&#xA;The problem with what you want to do with an image fed into an algorithm, is that the image, once taken, has a set focus, quality and resolution. So similarly like the guys on TV spy shows zooming and &quot;enhancing&quot; and highly zoomed in picture: You can't really do it. The focal length is cast in stone the moment the picture is taken &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need dummy simulation you could link the focal length variable to a blurring factor in something like Python's Image library. &#xA;Check this out too: &lt;a href=&quot;https://stackoverflow.com/questions/2292233/blurring-a-picture-python-jython-picture-editing&quot;&gt;https://stackoverflow.com/questions/2292233/blurring-a-picture-python-jython-picture-editing&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9432" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-06-10T21:44:39.297" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="12874" PostTypeId="1" AcceptedAnswerId="12875" CreationDate="2014-06-10T22:53:18.750" Score="3" ViewCount="215" Body="&lt;p&gt;I'm looking for some general insight on preconditioning. In particular, relevant references/resources/comments would be greatly appreciated. Note, I have been through some of the literature, but am having trouble finding a reference for a problem similar to the one I am solving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system I'm solving takes the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q_{ij} \ddot{y}_j + S_{ijk} \dot{y}_j\dot{y}_k +V_i =0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where Einstein summation is assumed, and the arrays $Q_{ij}, S_{ijk},V_i$ are all low order polynomials in the dependent variables $\{ y_1, ... ,y_N\}$. Here, $N$ is the 'resolution' of the model, and $(i,j,k)\in(1,..,N)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, because $Q_{ij}$ is non trivial to invert, I've been solving this using an initial differential algebraic equation solver, namely the IDA solver that is a part of the SUNDIALS suite, via the FORTRAN interface. This solver uses a &quot;variable-order, variable-coefficient&quot; backwards differentiation formula. I also change variables, to make the system 1st order, i.e. $\dot{y}_k = v_k$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solver, in general, works as follows. Our governing equation can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F(t,\vec{y},\dot{\vec{y}}) = 0; \quad \vec{y}(t_o) = \vec{y}_o; \quad \dot{\vec{y}}=\dot{\vec{y}}_o$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\vec{y} = (y_1,...,y_N)$. Now, the backwards differentiation formula, of qth order, implies &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i=0}^q \alpha(n,i) \vec{y}(n-i) =h(n) \dot{\vec{y}}(n)$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h(n)=t(n)-t(n-1)$ is the step size, and $\alpha(n,i)$ are functions of q and the history of the time steps. Substituting this back into the governing equation, we have equations of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$G(\vec{y}(n)) = F\left(t(n), \vec{y}(n),h(n)^{-1}\sum_{i=0}^q \alpha(n,i) \vec{y}(n-i)\right) = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solver works by finding the solution to this equation via some form of Newton iteration. That is, it looks at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J[\vec{y}(n)^{m+1}-\vec{y}(n)^m] = -G(\vec{y}(n)^m)$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the superscript $(m,m+1)$ refers to the $m$th, and $m+1$th approximation to the variable $\vec{y}$ and the Jacobian $J$ is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J= \frac{\partial G}{\partial y} = \frac{\partial F}{\partial y}+ \alpha \frac{\partial F}{\partial \dot{y}}; \quad \alpha =\frac{\alpha(n,0)}{h(n)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, as the system approaches the phenomena I wish to study, the equations take much longer to integrate. I have optimized the 'low hanging fruit', e.g. the linear algebra is done using OpenBLAS, OpenMP is implemented etc... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that I need to use an iterative linear solver for these larger problems, and this calls for a preconditioner. My understanding is that a preconditioner $P$ is a matrix such $P^{-1}J$ gives the same solutions as would be the case for $P=I$, but is potentially easier to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I know that there exists a permanent progressive solution to my governing equation, such that $y_n = a_n e^{inct}$ where $a_n,c$ are numerically computed constants. The solutions I'm interested in modeling are going to be perturbations from these. So atleast for short times, the calculated solutions will be small deviations from the permanent progressive solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I am interested in using information from these permanent progressive solutions to see if I can improve the efficiency of solving the more general equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a sentence, I am unclear of how to do this, and have struggled to find relevant resources where specific examples are carried out in detail. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively, I would like to output the Jacobian of my permanent progressive solution so that I could get a better idea of its structure, and from here I could pick a preconditioner. I cannot seem to figure out how to do this in SUNDIALS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any information would be greatly appreciated,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nick&lt;/p&gt;&#xA;" OwnerUserId="8021" LastActivityDate="2014-06-11T01:15:49.010" Title="Heuristic help with preconditioning large system ODEs" Tags="|ode|fortran|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12875" PostTypeId="2" ParentId="12874" CreationDate="2014-06-11T01:15:49.010" Score="2" Body="&lt;p&gt;Assuming you have the necessary Jacobian routine, I'd probably use a package that will compute an incomplete LU factorization as a &quot;black-box&quot; preconditioner to get started. In SUNDIALS, you will have to compute the factorization yourself; for that, I'd use something like SuperLU (you could also use MUMPS, or other routines). You can then supply the factored Jacobian matrix as a &quot;user-supplied preconditioner&quot;. The IDA manual should have some information on how to do that in IDA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Frankly, I find it easier to do these types of things in PETSc. It does have a better mechanism for run-time control and composition of&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;time-steppers (including DAE solvers)&lt;/li&gt;&#xA;&lt;li&gt;nonlinear equation solvers&lt;/li&gt;&#xA;&lt;li&gt;nonlinear preconditioners&lt;/li&gt;&#xA;&lt;li&gt;Krylov subspace methods&lt;/li&gt;&#xA;&lt;li&gt;linear preconditioners (includes LU decomposition, multigrid, ILU, etc.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and that run-time capability makes it much easier to experiment with &quot;black-box&quot;-type preconditioners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc has not yet implemented variable-step, variable-order BDF methods (for ODEs or DAEs), nor has it implemented sensitivity analysis (forward or adjoint), so if you need those features, you are still better off using SUNDIALS. You can call SUNDIALS from PETSc, but I believe that only CVODE is implemented, and it calls CVODE using the GMRES linear solver (I don't believe there is any flexibility in using different linear solvers when using SUNDIALS from PETSc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know something about the physics of your system (for instance, removing coupling terms is a decent approximation of the physics), you might be able to construct a &quot;physics-based&quot; preconditioner that yields good performance. Good examples of physics-based preconditioners can be found in &lt;a href=&quot;http://www.inf.ufes.br/~luciac/comcie/JFNK-Knoll.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;Jacobian-Free Newton–Krylov methods: a survey of approaches and applications&quot; by Dana A. Knoll and David E. Keyes, Journal of Computational Physics, Volume 193, Issue 2, pages 357-397&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-06-11T01:15:49.010" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="12876" PostTypeId="2" ParentId="12869" CreationDate="2014-06-11T07:06:08.463" Score="7" Body="&lt;p&gt;Your question seems to imply that the cells you get extend to infinity. But, since cells are just polyhedra, it is easy to intersect them with your domain which, most of the time, is also just a polyhedron. If your domain is finite, then so are all the cells. In other words, if you encounter a cell with a vertex at infinity, then you need to determine the bounding lines that extend to infinity, intersect them with your domain, and replace the vertices at infinity with the intersection point. These cells will all have finite volume/area.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-06-11T07:06:08.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12877" PostTypeId="2" ParentId="12860" CreationDate="2014-06-11T07:11:58.150" Score="14" Body="&lt;p&gt;It's a question on what you spend your time on. For most of us, we spend 3/4 of the time programming and 1/4 of the time waiting for results. (Your numbers may vary, but I think the number is not completely without merits.) So, if you have a design that allowed you to program twice as fast (3/4 of a time unit instead of 1.5 time units), then you can can take a 300% hit in performance (from 1/4 to 1 time unit) and you still come out ahead in terms of real time spent on solving the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you're doing heavy duty computations, your calculations may look differently and you may want to spend more time optimizing your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, 20% seems like a fairly good trade-off if you end up being more productive.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-06-11T07:11:58.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12878" PostTypeId="2" ParentId="11457" CreationDate="2014-06-11T08:34:17.653" Score="2" Body="&lt;p&gt;The Eigenface algorithm transforms image patches (e.g. logo of beer) into a common data space where nearest neighbor search is performed for classification. In order to use Eigenfaces, you will need a training data set to compute the common data space. Therefore it requires some time to define a proper training set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have a proper training set, I believe that Eigenface will definitely work for the recognition of beer types. As an alternative you could use an unsupervised learning approach that needs no training set. For this take a look at &lt;a href=&quot;http://deeplearning.net/&quot; rel=&quot;nofollow&quot;&gt;http://deeplearning.net/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9378" LastActivityDate="2014-06-11T08:34:17.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12880" PostTypeId="1" AcceptedAnswerId="12916" CreationDate="2014-06-11T13:47:53.090" Score="4" ViewCount="215" Body="&lt;p&gt;I am dealing with a system of non-linear equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(\boldsymbol{x}) = \boldsymbol{y}, \;\;\; \boldsymbol{x}, \boldsymbol{y} \in \mathbb{R}^d.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I know that the Jacobian $J(\boldsymbol{x})$ is positive definite:&#xA;$$&#xA;\boldsymbol{z}^T J(\boldsymbol{x}) \boldsymbol{z} \geq \epsilon &amp;gt; 0, \;\;\;\;\;\;\forall  \; \boldsymbol{x}, \, \boldsymbol{z}  \; :\; ||\boldsymbol{z} || = 1 &#xA;$$&#xA; for any $\boldsymbol{x}$. My intuition is that this implies that the system can by solved for any $\boldsymbol{y}$. In the univariate case it seems obvious, but is this still correct in the multivariate case? And is there a way to prove it? &#xA;Many thanks!&lt;/p&gt;&#xA;" OwnerUserId="2140" LastEditorUserId="2140" LastEditDate="2014-06-11T21:33:54.153" LastActivityDate="2014-06-16T21:42:57.190" Title="Non-linear root finding with positive definite Jacobian" Tags="|convex-optimization|nonlinear-equations|roots|jacobian|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12882" PostTypeId="2" ParentId="12860" CreationDate="2014-06-11T19:51:30.423" Score="9" Body="&lt;p&gt;Very few scientific software developers understand good principles of design, so I apologize if this answer is a bit long-winded.  From a software engineering perspective, the goal of the scientific software developer is to design a solution that satisfies a set of constraints that are often &lt;em&gt;conflicting&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some typical examples of these constraints, as might be applied to the design of your sparse matrix library:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Completed in one month&lt;/li&gt;&#xA;&lt;li&gt;Runs correctly on your laptop and several workstations&lt;/li&gt;&#xA;&lt;li&gt;Runs efficiently&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Scientists are gradually paying more attention to some other common requirements from software engineering:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Documentation (User guide, tutorial, code commenting)&lt;/li&gt;&#xA;&lt;li&gt;Maintainability (version control, testing, modular design)&lt;/li&gt;&#xA;&lt;li&gt;Reusability (modular design, &quot;flexibility&quot;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You might need more or less of one of these requirements.  If you're trying to win a Gordon Bell prize for performance, then even fractions of a percent are relevant, and few of the judges will be evaluating the quality of your code (so long as you can convince them it's right).  If you're trying to justify running this code on a shared resource such as a cluster or a supercomputer, frequently you have to defend claims about the performance of your code, but these are rarely very stringent.  If you're trying to publish a paper in a journal describing the performance gains of your approach, then you need to legitimately be faster than your competitors, and 20% performance is a trade-off I would gladly make for better maintainability and reusability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming back to your question, &quot;good design&quot;, granted enough development time, should never sacrifice performance.  If the objective is to make code that runs as fast as possible, then the code should be designed around those constraints.  You might use techniques such as code generation, inline assembly, or take advantage of highly-tuned libraries to help you solve your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if you don't have enough development time?  What's good enough?  Well, it depends, and nobody is going to be able to give you a good answer to this question without more context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FWIW: If you really are interested in writing high-performance sparse matrix kernels, you should be comparing against an optimized PETSc install and working with their team if you are beating them, they'd be happy to incorporate tuned kernels to the library.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="276" LastEditDate="2014-06-14T06:36:38.027" LastActivityDate="2014-06-14T06:36:38.027" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12883" PostTypeId="2" ParentId="12880" CreationDate="2014-06-11T20:18:42.867" Score="4" Body="&lt;p&gt;You could consider $d=1$ and&#xA;$$ f(x) = e^x, \qquad y=0. $$&#xA;I don't think a function's derivative being positive everywhere implies that the function's range is all of $\mathbb{R}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does imply, though, that solutions are unique. If $(x_1,y)$, $(x_2,y)$, $x_1\neq x_2$ are two solutions, then&#xA;$$ 0 = (x_2-x_1)\cdot(f(x_1) - f(x_2))&#xA; = \int_0^1 (x_2-x_1)\cdot J(x_1+t(x_2-x_1))(x_2-x_1)\,dt $$&#xA;so it can't be that $J$ is positive definite everywhere.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-06-11T21:01:05.933" LastActivityDate="2014-06-11T21:01:05.933" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12884" PostTypeId="2" ParentId="12867" CreationDate="2014-06-11T21:05:29.127" Score="4" Body="&lt;p&gt;Using finite element methods, you also have the choice of discretizing the weak formulation directly, using two types of approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;using H&amp;sup2;-conforming elements like the Argyris element or the Hsieh-Clough-Tocher on triangles or the Bogner-Fox-Schmidt element on quadrilaterals.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Using regular H&amp;sup1;-conforming elements in conjunction with the so called C&lt;sup&gt;0&lt;/sup&gt;-interior penalty method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you take the weak formulation you derived through Green's formula, choosing test spaces with $v = \partial_n v = 0$ on the boundary makes the boundary terms go away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach very well suited to high order differential equations is isogeometric analysis.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2014-06-11T21:05:29.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12885" PostTypeId="2" ParentId="10068" CreationDate="2014-06-11T21:39:47.527" Score="1" Body="&lt;p&gt;Have a look at TOMLAB, DAKOTA, and OpenMDAO for black-box optimization. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit #3:&#xA;Bayesian optimization is very similar to EGO:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mwhoffman/pybo&quot; rel=&quot;nofollow&quot;&gt;https://github.com/mwhoffman/pybo&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/hyperopt/hyperopt&quot; rel=&quot;nofollow&quot;&gt;https://github.com/hyperopt/hyperopt&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;limited licenses:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/rmcantin/bayesopt&quot; rel=&quot;nofollow&quot;&gt;https://github.com/rmcantin/bayesopt&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/HIPS/Spearmint&quot; rel=&quot;nofollow&quot;&gt;https://github.com/HIPS/Spearmint&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit #2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first approach is to build a metamodel/surrogate (using kriging/GP) around expensive function and use this additional information to find the global optimum point faster and with fewer evaluations (EGO).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second approach as in MDAS, is to do direct search with some clever adaptions on multiple levels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Heuristic approaches are genetic/randomized in nature and without any guarantees.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit #1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TOMLAB is MATLAB based tool that has the best speed / quality of optimization according to Sahinidis'es paper. But this is commercial tool with significant corporate usage. I'm not using this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DAKOTA is more tailored for Uncertainty Quantification, besides general optimization. Based on c++ and some legacy Fortran code. Though under LGPL license and binaries available for download, very difficult to recompile at least from my experience on Win7 with GCC or MSVS/ifort. Has dependencies on boost, lapack, cmake for build. Basically this is a wrapper for numerous open source solvers and few commercial ones. This is SNL product and is tightly integrated with other  projects from Sandia NL. I was able to successfully integrate this one instead of some IMSL routines. Sahinidis'es paper missed the massive parallelism possible with DAKOTA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenMDAO is optimization based design software developed in Python by NASA under APACHE License. I'm trying this out currently. &lt;/p&gt;&#xA;" OwnerUserId="5401" LastEditorUserId="5401" LastEditDate="2015-03-16T03:28:18.773" LastActivityDate="2015-03-16T03:28:18.773" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12886" PostTypeId="1" AcceptedAnswerId="12890" CreationDate="2014-06-12T05:49:54.477" Score="8" ViewCount="726" Body="&lt;h3&gt;Background:&lt;/h3&gt;&#xA;&lt;p&gt;I am currently running a large amount parameter variation experiments.&#xA;They are being run in Python 2.6+, using numpy.&#xA;These experiments are going to take about 2 weeks to run.&lt;/p&gt;&#xA;&lt;p&gt;Roughly I am varying 3 parameters (independent variables) over a range of values.&#xA;I am fixing 6 further independent variables (for now)&#xA;I am reporting on 4 dependent variables.&lt;/p&gt;&#xA;&lt;p&gt;One of the parameters I am varying is being distributed across several processes (and computers).&#xA;For each of these parameters, I generate a separate &lt;code&gt;csv&lt;/code&gt; files with each row containing the values of all the variables (including independent, fixed and dependent).&#xA;Across all the variation expect to generate about 80,000 rows of data&lt;/p&gt;&#xA;&lt;p&gt;Most of the time I am only looking at the value of one of the dependent variables, however I keep the others around, as they can explain what is going on when something unexpected happens.&lt;/p&gt;&#xA;&lt;p&gt;In a earlier version of this experiment, varying across only 2 parameters (each though only 2 values)&#xA;I was copying pasting this &lt;code&gt;csv&lt;/code&gt; file into a spreadsheet program and doing a bunch of copy pasting to make a table of just the dependent variable I was interested in.&#xA;The doing some awkward things in MS-Excel to let me sort by formulas.&#xA;This was painful enough for the 6 experiment results sets I had.&#xA;By the time this run is finished I am going to have 2 orders of magnitude more results.&lt;/p&gt;&#xA;&lt;h3&gt;Question:&lt;/h3&gt;&#xA;&lt;p&gt;I was thinking once done, I could dump all the results from the &lt;code&gt;csv&lt;/code&gt; files into a database, and the query out the parts that are interesting.&#xA;Then take those results and put them into a spreadsheet for analysis.&#xA;Making graphs, finding scored relative to the control results etc&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Am I thinking along the right lines? (Is this what people do?)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;My database foo is fairly rusty these days, even when it was good I was using MS-Access.&#xA;I was intending on using MS-Access for this as well.&lt;/p&gt;&#xA;" OwnerUserId="6690" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-07-24T04:16:51.793" Title="Should I use a database to handle large amounts of results?" Tags="|software|visualization|data-analysis|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12887" PostTypeId="2" ParentId="12886" CreationDate="2014-06-12T06:01:06.377" Score="1" Body="&lt;p&gt;Yes, you can dump all the results into a database, and yes, some people elect to use databases. I haven't yet had to deal with situations using databases, but I have taught at workshops where other instructors teach about using databases to gather data. For databases that aren't massive, from what I understand, the underlying technology doesn't matter a whole lot. My co-instructor used SQLite3. It's easy to install in Linux, comes standard in OS X, and I believe it's available for Windows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible to access SQLite databases through a terminal in OS X and Linux; I'm not sure how it's done on Windows. It's also possible to leverage Python packages to read from and write to your database programmatically, for instance, using the sqlite3 package in the Python standard library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data sets get really big, other database implementations are better, and at that point, you probably want to consult a database specialist.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-06-12T06:01:06.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12888" PostTypeId="2" ParentId="12864" CreationDate="2014-06-12T07:30:56.347" Score="0" Body="&lt;p&gt;There is quite some research on modeling the human eye. For an overview, check &lt;a href=&quot;http://www.wiley-vch.de/books/sample/3527403809_c01.pdf&quot; rel=&quot;nofollow&quot;&gt;this chapter&lt;/a&gt; from a book series by former Zeiss head of optics R&amp;amp;D H. Gross. You may check the works of R. Navarro, who is well-known for his eye model.&lt;/p&gt;&#xA;" OwnerUserId="1553" LastActivityDate="2014-06-12T07:30:56.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12889" PostTypeId="1" CreationDate="2014-06-12T08:44:26.700" Score="2" ViewCount="403" Body="&lt;p&gt;I would like to know if there is a commercialization of simulations? Or is it only in academic usage? &lt;/p&gt;&#xA;" OwnerUserId="9443" LastActivityDate="2014-06-12T19:17:32.767" Title="Are there companies which create commercial molecular dynamics / monte carlo simulations?" Tags="|monte-carlo|molecular-dynamics|" AnswerCount="1" CommentCount="2" ClosedDate="2014-06-19T23:23:17.010" ContentLicense="CC BY-SA 3.0" />
  <row Id="12890" PostTypeId="2" ParentId="12886" CreationDate="2014-06-12T10:03:42.723" Score="8" Body="&lt;p&gt;I would suggest that a full database may be overkill for your purposes, though it would certainly work.&#xA;Even $5 \cdot 10^5$ rows should be no more than around 25mb of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would strongly recommend doing the analysis/plotting/etc with the same tool that you will use for querying your data. It is my experience that when changing what to analyse only takes changing 1 line of code and waiting 2 seconds, it is much easier to get the most out your data. Copy pasting is also HIGHLY error prone. I have seen several people at the point of desperation because their data did not make sense, only to realise they made a mistake when copying data in their excel sheet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your are at all familiar with python, I would suggest using &lt;a href=&quot;http://pandas.pydata.org/&quot;&gt;pandas&lt;/a&gt; or (if you have more data than you can fit in memory) &lt;a href=&quot;http://pytables.github.io/usersguide/&quot;&gt;pytables&lt;/a&gt;, which will give you all the advantages of a database (including speed). Pandas has a lot of utility functions for plotting and analysing data, and you would have the full scientific python stack as well. Take a look at &lt;a href=&quot;http://nbviewer.ipython.org/github/jvns/talks/blob/master/pyconca2013/pistes-cyclables.ipynb&quot;&gt;this ipython notebook&lt;/a&gt; for an example of pandas use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe similar tools exist for R, as well as commercial software such as Matlab or Stata. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;HDF5 is a good generic way of storing the data initially, and has good library support in many languages.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="7522" LastEditDate="2014-06-12T11:25:09.907" LastActivityDate="2014-06-12T11:25:09.907" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="12891" PostTypeId="2" ParentId="5372" CreationDate="2014-06-12T11:46:40.277" Score="9" Body="&lt;p&gt;In broad terms, algorithms that run faster on the GPU are ones where you are doing the same type of instruction on many different data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An easy example to illustrate this is with matrix multiplication. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we are doing the matrix computation &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$A \times  B = C$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A simple CPU algorithm might look something like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;//starting with C = 0&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (int i = 0; i &amp;lt; C_Width; i++)&#xA;{&#xA;    for (int j = 0; j &amp;lt; C_Height; j++)&#xA;    {&#xA;        for (int k = 0; k &amp;lt; A_Width; k++)&#xA;        {&#xA;            for (int l = 0; l &amp;lt; B_Height; l++)&#xA;            {&#xA;                C[j, i] += A[j, k] * B[l, i];&#xA;            }&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The key thing to see here is that there are a lot of nested for loops and each step must be executed one after the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://upload.wikimedia.org/wikipedia/commons/e/eb/Matrix_multiplication_diagram_2.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;See a diagram of this&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the calculation of each element of C does not depend on any of the other elements. So it does not matter what order the calculations are done in. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So on the GPU, these operations can be done concurrently. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A GPU kernel for calulating a matrix multiplication would look something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__kernel void Multiply&#xA;(&#xA;    __global float * A,&#xA;    __global float * B,&#xA;    __global float * C&#xA;)&#xA;{&#xA;     const int x = get_global_id(0);&#xA;     const int y = get_global_id(1);&#xA;     for (int k = 0; k &amp;lt; A_Width; k++)&#xA;     {&#xA;         for (int l = 0; l &amp;lt; B_Height; l++)&#xA;         {&#xA;             C[x, y] += A[x, k] * B[l, y];&#xA;         }&#xA;     }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This kernel only has the two inner for loops. A program sending this job to the GPU will tell the GPU to execute this kernel for each data point in C. The GPU will do each of these instructions concurrently on many threads.  Just like the old saying &quot;Cheaper by the dozen&quot; GPUs are designed to be faster doing the same thing lots of times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are however some algorithms which will slow the GPU down. Some are not well suited for the GPU. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for example, there were data dependencies, ie: imagine the computation of each element of C depended on the previous elements. The programmer would have to put a barrier in the kernel to wait for each previous computation to finish. This would be a major slow down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, algorithms which have a lot of branching logic ie:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__kernel Foo()&#xA;{&#xA;    if (somecondition)&#xA;    {&#xA;        do something&#xA;    }&#xA;    else&#xA;    {&#xA;        do something completely different&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;tend to run slower on the GPU because the GPU is no longer doing the same thing in each thread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a simplified explanation because there are many other factors to consider. &#xA;For example, sending data between the CPU and GPU is also time consuming. Sometimes it is worth doing a computation on the GPU even when its faster on the CPU, just to avoid the extra send time (And vice versa). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also many modern CPUs support concurrency now as well with hyperthreaded multicore processors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;GPU's also seem to be not so good for recursion, see &lt;a href=&quot;https://stackoverflow.com/questions/14309997/how-to-implement-deep-recursion-on-cuda&quot;&gt;here&lt;/a&gt; which probably explains some of the problems with the QR algorithm. I believe that one has some recursive data dependencies. &lt;/p&gt;&#xA;" OwnerUserId="4680" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-05-16T14:52:01.933" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12892" PostTypeId="1" CreationDate="2014-06-12T16:17:55.337" Score="2" ViewCount="496" Body="&lt;p&gt;Good day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to implement the EGO optimization algorithm. The algorithm itself is rather long to describe here in full. It is presented &lt;a href=&quot;http://www.ressources-actuarielles.net/EXT/ISFA/1226.nsf/0/f84f7ac703bf5862c12576d8002f5259/%24FILE/Jones98.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, with another example of usage &lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/a544460.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But before I implement the algorithm, I have decided to test the DACE model on a simple 1-d function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am receiving strange results; I believe I have located the problem, but am uncertain, whether my implementation is wrong, or if what I found is a weak point in the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To initiate the algorithm, we need first to determine a set of parameters $\theta = \{\theta_h\}$ and $p = \{p_h\}$ from some sample data we measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way we do so is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we have $X = \{X^1, ..., X^n\}$ a set of points we have evaluated the function at, and $Y = \{y_1, ..., y_n\}$ the values at those points. Denote $X^i = (x^i_1, ... x^i_k)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then define &#xA;$$ d_{ij} = \sum_{h=1}^k \theta_h |x_h^i - x_h^j|^{p_h}$$&#xA;$$R_{ij} = e^{-d_{ij}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We estimate the values $\theta, p$ by maximizing the likelihood function&#xA;$$\frac1{(2\pi)^{n/2}(\sigma^2)^{n/2}|R|^{1/2}} e^{-\frac{(...)R^{-1}(...)}{\sigma^2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The exact expression in the exponent does not matter for what I am about to say, but is present in the files linked to in the beginning. Also, $\sigma^2 = \sigma^2(\theta,p)$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I carry the maximization of the likelihood by the Nelder-Meads algorithm (fminsearch in MATLAB). But here is my problem: in some cases, when the initial data set is too large (I am minimizing $-sin(x)$ on $[0,\pi]$, and 'too large' is ~10) what happens, is that the values the algorithm goes for are such, that the matrix $R$ becomes nearly singular ($cond(R)$ ~ $1e^{16}$). This can be explained, since we have a determinant of the matrix in negative power in the likelihood function. Thus, for a singular matrix the determinant becomes zero, and the likelihood function itself diverges to infinity. But because my calculations also feature $R^{-1}$, having a nearly-singular matrix is problematic for the algorithm (and that is indeed the case, the returned estimated function behaves nothing like the original).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that this does not happen always, and for some initial data of ~4 points, the aprroximation to the function behaves very well, so I have reason to believe that the mistake is not simply a coding mistake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I ask again: is there another way to estimate the parameters? Am I doing something wrong? Is this a weak point in the algorithm?&lt;/p&gt;&#xA;" OwnerUserId="9362" LastEditorUserId="9362" LastEditDate="2014-06-12T16:27:18.337" LastActivityDate="2014-06-12T16:27:18.337" Title="Estimating the parameters of the DACE stochastic model (EGO optimization algorithm)" Tags="|optimization|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="12893" PostTypeId="1" AcceptedAnswerId="12895" CreationDate="2014-06-12T17:18:03.040" Score="14" ViewCount="2751" Body="&lt;p&gt;I am solving a system of two coupled PDE's in two spatial dimensions and in time computationally. Since the function evaluations are expensive, I would like to use a multistep method (initialised using Runge-Kutta 4-5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Adams-Bashforth method using five previous function evaluations has a global error of $O(h^5)$ (this is the case where $s=5$ in the Wikipedia article referenced below), and requires one function evaluation (per PDE) per step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Adams-Moulton method on the other hand requires two function evaluations per step: one for the prediction step, and another for the corrector step. Once again, if five function evaluations are used, the global error is $O(h^5)$. ($s=4$ in the Wikipedia article)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is the reasoning behind using Adams-Moulton over Adams-Bashforth? It has an error of the same order, for twice the number of function evaluations. Intuitively it makes sense that a predictor-corrector method should be favourable, but can somebody explain this quantitatively?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_multistep_method#Adams.E2.80.93Bashforth_methods&quot;&gt;http://en.wikipedia.org/wiki/Linear_multistep_method#Adams.E2.80.93Bashforth_methods&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9448" LastActivityDate="2014-06-13T12:36:26.423" Title="What are the relative benefits of using Adams-Moulton over Adams-Bashforth algorithm?" Tags="|pde|algorithms|finite-element|error-estimation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="12895" PostTypeId="2" ParentId="12893" CreationDate="2014-06-12T18:45:47.067" Score="12" Body="&lt;p&gt;Adams-Moulton method is significantly more stable.  The analogy used when I was taught the difference is the same as extrapolation and interpolation.  Interpolation is relatively safe numerically.  Extrapolation can blow up if you happen to have an asymptote or some other odd feature.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, solving the ode&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y'(t) = -y(t)$ with $y(0) = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the 3rd order Adams-Bashforth method actually becomes &lt;em&gt;more&lt;/em&gt; unstable as the timestep is reduced.  By adding the corrector step, you avoid much of this instability.  A plot of the stability regions for the two methods are show here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bucMO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plot taken from &lt;em&gt;The Art of Scientific Computing&lt;/em&gt; by Gregory Baker and Edward Overman.  $\lambda$ is the eigenvalue of your ODE, $h$ is the timestep.  Note that $\lambda$ can be complex, so the plots are on the complex plain.  If $\lambda h$ is inside the stable space, the ode will converge.  If it is outside, eventually the time integration will become unstable.  Note that for stability &lt;em&gt;all&lt;/em&gt; eigenvalues of your ODE, or ODE system must be inside the stable region.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2014-06-13T12:36:26.423" LastActivityDate="2014-06-13T12:36:26.423" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="12896" PostTypeId="2" ParentId="12889" CreationDate="2014-06-12T19:17:32.767" Score="1" Body="&lt;p&gt;One example of a commercial MD code that comes to mind is Desmond, which is licensed by Schrödinger, LLC, and was produced by D. E. Shaw research. It is available for non-commercial use by academics, although the impression I get from talking to colleagues active in MD research is that they prefer LAMMPS and GROMACS, which are open-source, community efforts.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-06-12T19:17:32.767" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="12897" PostTypeId="1" AcceptedAnswerId="12900" CreationDate="2014-06-13T10:27:36.660" Score="5" ViewCount="524" Body="&lt;p&gt;What is a good direct method to compute the spectral decomposition / Schur decomposition / singular decomposition of a symmetric matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Direct&quot; means as in LU decomposition, Cholesky decomposition, or Golub's SVD algorithm, in contrast to iterative methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can, of course, apply the aforementioned algorithm of Golub, but I guess that would be breaking a butterfly on a wheel.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2014-06-13T16:02:19.120" Title="Spectral decomposition of symmetric matrix" Tags="|linear-algebra|dense-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12899" PostTypeId="2" ParentId="12897" CreationDate="2014-06-13T14:39:39.597" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/QR_algorithm&quot; rel=&quot;nofollow&quot;&gt;implicit (Francis) QR iteration&lt;/a&gt;, with several additional tricks, is the standard algorithm. I suggest you to check a book such as Watkins' &lt;em&gt;The Matrix Eigenvalue Problem&lt;/em&gt; for details; this is a classical topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I find it unusual that you are familiar with Golub's SVD but not with QR - the two methods are similar and usually taught together).&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-06-13T14:39:39.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12900" PostTypeId="2" ParentId="12897" CreationDate="2014-06-13T16:02:19.120" Score="4" Body="&lt;p&gt;In addition to the QR algorithm, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Divide-and-conquer_eigenvalue_algorithm&quot; rel=&quot;nofollow&quot;&gt;divide and conquer method&lt;/a&gt; is also worth mentioning. It is applicable to symmetric tridiagonal matrices, but any matrix can be reduced to such a form via the Lanczos method*. It hinges on the observation that a tridiagonal matrix is, up to a rank 1 perturbation, a block diagonal matrix. One can then find the eigendecomposition of the sub-blocks (in parallel!) and glue them back together using a clever trick. Of course, finding the eigenvalues of the blocks must be done with the QR method as Federico alludes to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you've asked for a direct method -- both QR and divide and conquer are iterative methods. Well, there is no direct method akin to the LU decomposition to find the eigendecomposition of a matrix of dimension greater than 5, there are &lt;em&gt;only&lt;/em&gt; iterative methods. If such a direct method existed, one could find the zeros of an arbitrarily high-degree polynomial as an algebraic function of the coefficients, which Galois told us is impossible. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular_value_decomposition#Calculating_the_SVD&quot; rel=&quot;nofollow&quot;&gt;Golub-Kahan SVD algorithm&lt;/a&gt; is also iterative.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-06-13T16:02:19.120" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="12901" PostTypeId="1" CreationDate="2014-06-13T21:24:52.253" Score="3" ViewCount="138" Body="&lt;p&gt;Is it possible to construct a constrained tetrahedral mesh of a domain using Tetgen or similar software? What I mean by constrained is that there are some nodes or edges that are not free but are predefined by the user.&lt;/p&gt;&#xA;" OwnerUserId="9412" LastEditorUserId="549" LastEditDate="2014-06-13T21:56:24.743" LastActivityDate="2020-06-26T05:01:15.053" Title="Mesh with constraints" Tags="|computational-geometry|mesh-generation|unstructured-mesh|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12902" PostTypeId="1" AcceptedAnswerId="12903" CreationDate="2014-06-15T03:27:57.093" Score="6" ViewCount="472" Body="&lt;p&gt;When testing a numerical method for a PDE, I know that it's often useful to compare it to a known analytical solution.  If none is available, one can always 'manufacture' a solution, substitute it into the PDE and obtain the source term, boundary conditions, and initial conditions needed for the problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what guidelines should I follow for choosing an appropriate 'manufactured' solution to test the scheme on?  What qualities should the manufactured solution have?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some thought, I've come up with the following considerations:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Sufficient Smoothness&lt;/strong&gt; - Whatever the highest order derivative is in the PDE (for any variable), the manufactured solution should have at least this degree of smoothness within the domain of the problem.  &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Physically Plausible Coefficients&lt;/strong&gt; - Non-negative values for coefficients that are physically positive.  &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Appropriate Scaling&lt;/strong&gt; - A non-dimensional version of the problem should be used, if possible.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, my main concern is manufacturing a solution that can adequately capture the theoretical order of accuracy.  Is there anything else i should consider in the properties of the manufactured solution to ensure that the theoretical order of accuracy will be captured?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there anything else I should &lt;strong&gt;&lt;em&gt;generally consider&lt;/em&gt;&lt;/strong&gt; when choosing a manufactured solution?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="9667" LastEditDate="2015-08-20T20:14:58.637" LastActivityDate="2015-08-20T20:14:58.637" Title="Guidelines for choosing manufactured solutions for numerical PDE schemes" Tags="|pde|testing|verification|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12903" PostTypeId="2" ParentId="12902" CreationDate="2014-06-15T03:47:02.423" Score="7" Body="&lt;p&gt;The typical approach to this problem is to use infinitely smooth functions like trigonometric, inverse trig, exponential, etc. There's a handy library called &lt;a href=&quot;https://red.ices.utexas.edu/projects/software/wiki/MASA&quot;&gt;MASA&lt;/a&gt; that has a library of manufactured solutions and makes it easy to make your own. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your PDE coefficients will become parameters in the forcing function that your chosen solution leads to, so you should probably choose your PDE coefficients as you would when solving your real problem. The problem here is that often these coefficients lead to interesting solution features that you want to see if your numerical method/discretization captures, like, say, a boundary layer, but no matter what you choose for these coefficients your manufactured solution and its associated forcing function can hide such features completely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MMS is good for testing whether your method is implemented correctly and achieves the correct order of accuracy as you refine the mesh, but it's bad at determining whether or not your method captures the solution features implicit in the PDE and its coefficients (like layers or turbulence). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your #3 is up to you, but it's not a requirement of MMS or even good MMS. You should manufacture solutions that correspond to the form of the PDE you actually want to solve for real problems.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-06-15T03:47:02.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12906" PostTypeId="2" ParentId="2419" CreationDate="2014-06-15T13:31:02.430" Score="-1" Body="&lt;p&gt;I research about numerical solution of conservation equation, what is the newest issue about numerical solution of this equation&lt;/p&gt;&#xA;" OwnerUserId="9464" LastActivityDate="2014-06-15T13:31:02.430" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12907" PostTypeId="2" ParentId="12886" CreationDate="2014-06-15T15:27:39.577" Score="5" Body="&lt;p&gt;I highly recommend using a tool such as &lt;a href=&quot;http://neuralensemble.org/sumatra/&quot;&gt;Sumatra&lt;/a&gt; for this. I used to have a similar &quot;pedestrian&quot; approach to yours for keeping track of many simulation runs with varying parameters, but in the end it just becomes a huge mess because it's next to impossible to design such an ad-hoc approach correctly upfront and to anticipate all the use cases and extensions needed (e.g., what happens if you need to introduce an additional parameter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sumatra keeps track of all your simulation runs and stores them in a database which can later be queried (using its Python API) to filter and analyse the records you are interested in. It is very flexible and doesn't impose a workflow on you, which I find a big plus. Also, it comes with a web interface that allows you to quickly browse results (or inspect/download generated files), which is tremendously useful. The default database uses SQLite and I could imagine that it becomes a bit slow if you use it to store 80,000+ simulation outcomes. There is a PostgreSQL backend but I have never used it so can't vouch for its performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should say that it's still in its early-ish development stages and there are a few things missing, but I have used it for pretty much all my simulations in the past year and it has saved my day so many times that I could not imagine what I would do without it. Personally, I never used it for computations across different computers (or on a cluster), but I think it supports this kind of workflow. Do ask on the mailing list if you're unsure or can't find exactly what you need, it's a small but very friendly and helpful community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Give me a shout if this is something you're interested in and I'm happy to share my workflow and boilerplate code to get you going (or just for inspiration).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the actual data analysis, I agree with LKlevin that pandas and the IPython notebook are extremely useful tools to know about (Sumatra allows you to import the records into pandas, although this is a bit at the moment but I'm sure it'll soon be improved). I could also imagine that saving data/simulation outcomes to HDF5 format could be useful, in which case pytables is a good tool in the toolbox. (I seem to remember that support for HDF5 is planned in Sumatra, but I can't find the information right now and I'm not sure this is implemented yet.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, I'm sure there are other tools that help with these kinds of tasks (see the &quot;short list&quot; on &lt;a href=&quot;http://www.youtube.com/watch?v=OFzYMtHepXg&amp;amp;t=5m51s&quot;&gt;this presentation slide&lt;/a&gt;). But personally I haven't tried any of those because I've been very happy with the functionality and flexibility that Sumatra offers.&lt;/p&gt;&#xA;" OwnerUserId="4263" LastActivityDate="2014-06-15T15:27:39.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12908" PostTypeId="2" ParentId="12886" CreationDate="2014-06-15T18:07:08.447" Score="1" Body="&lt;p&gt;If all your data fits comfortably in memory (say, below 1 GB, so you have margin for the analysis), a DB is overkill. You can just read the whole file in memory and select the pieces you want. On the other hand, when your data starts to grow (or could potentially grow too big), a DB can offer you fast and easy queries (&lt;em&gt;&quot;give me all the speeds for which the energy was exactly 2 and the temperature bigger than 27&quot;&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another topic is the generation of data. As your data takes two weeks I am assuming you are generating them on a computing cluster in parallel. Setting a DB for parallel writing is complex, and can potentially slow down the process, as data is being transferred and locks are in place. As you only need to write things once, you can have each process generate its own temporary text file, write the results there, and have a central process read each one of this and dump it on a master DB. For the simplest case, this script can be a simple &lt;code&gt;cat&lt;/code&gt;, and save the result as plain text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's assume you want to use a DB. If your use case is something slightly more advanced than a text file (what you would do if you had loaded a CSV with Numpy), I recommend HDF5 through PyTables. It is fast, simple to set up, Numpy-aware, and with a bunch of advanced features if you want to tune things. It also supports compression, querying, and saving arrays. It is also easy to install in Linux, Windows, and Mac. HDF5 data layout are nothing more than tables, like a spreadsheet. The resulting &lt;code&gt;.h5&lt;/code&gt; file can be read from many computer languages if they have the proper library installed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand you have SQL. You have one in Python's stdlib, so you would have it already installed, but it is not very well suited for numerical work (you can't save and recover Numpy arrays so easily, for example). This is the best option if you need third parties to interface from other languages, as it is very well known and there are wrappers to almost any language, many of them come by default.&lt;/p&gt;&#xA;" OwnerUserId="7759" LastActivityDate="2014-06-15T18:07:08.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12912" PostTypeId="1" CreationDate="2014-06-16T15:37:51.967" Score="4" ViewCount="97" Body="&lt;p&gt;My program needs to perform pivoted QR decomposition on tall (e.g. 1e9 by 100) matrices. I run into the bottleneck that the major computational time of my program is spent on doing serial pivoted-QR decomposition, while the node has 20 threads. For some reason I am confined to one node to perform the decomposition, so that distributed strategies such as scalapack will not help. I am wondering if there is any library/package that provides threaded version of pivoted-QR decomposition. Right now my code calls lapack's sgeqp3 subroutine. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9468" LastActivityDate="2014-07-17T01:10:49.757" Title="Threaded QR with column pivoting" Tags="|linear-algebra|parallel-computing|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12913" PostTypeId="1" CreationDate="2014-06-16T18:02:22.457" Score="7" ViewCount="5716" Body="&lt;p&gt;I'm trying to solve the Poisson equation with pure Neumann boundary conditions,&#xA;$$&#xA;\nabla^2\phi = \rho \quad in \quad \Omega\\&#xA;\mathbf{\nabla}\phi \cdot \mathbf{n} = 0 \quad on \quad \partial \Omega&#xA;$$&#xA;using a Fourier transform method I found in &lt;a href=&quot;http://www.nr.com/&quot; rel=&quot;noreferrer&quot;&gt;Numerical Recipes&lt;/a&gt;. The method uses a discrete cosine transform, if you don't have access to the book, you can find a derivation &lt;a href=&quot;http://elonen.iki.fi/code/misc-notes/neumann-cosine/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. I tried implementing the algorithm in Python; my code is listed below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;&lt;code&gt;&#xA;import numpy as np&#xA;import scipy.sparse as sparse&#xA;import scipy.fftpack as fft&#xA;&#xA;if __name__ == '__main__':&#xA;    shape = (3, 3)&#xA;    nx, ny = shape&#xA;&#xA;    charges = np.zeros(shape)&#xA;    charges[:] = 1.0 / (nx * ny)&#xA;    charges[nx / 2, ny / 2] = 1.0 / (nx * ny) - 1.0&#xA;    print charges&#xA;&#xA;    charges = charges.flatten()&#xA;&#xA;#Build Laplacian&#xA;    ex  = np.append(np.ones(nx - 2), [2, 2])&#xA;    ey  = np.append(np.ones(ny - 2), [2, 2])&#xA;    Dxx = sparse.spdiags([ex, -2 * np.ones(nx), ex[::-1]], [-1, 0, 1], nx, nx)&#xA;    Dyy = sparse.spdiags([ey, -2 * np.ones(ny), ey[::-1]], [-1, 0, 1], ny, ny)&#xA;&#xA;    L = sparse.kronsum(Dxx, Dyy).todense()&#xA;###############&#xA;&#xA;#Fourier method&#xA;    rhofft = np.zeros(shape, dtype = float)&#xA;    for i in range(shape[0]):&#xA;        rhofft[i,:] = fft.dct(charges.reshape(shape)[i,:], type = 1) / (shape[1] - 1.0)&#xA;    for j in range(shape[1]):&#xA;        rhofft[:,j] = fft.dct(rhofft[:,j], type = 1) / (shape[0] - 1.0)&#xA;&#xA;    for i in range(shape[0]):&#xA;        for j in range(shape[1]):&#xA;            factor = 2.0 * (np.cos((np.pi * i) / (shape[0] - 1)) + np.cos((np.pi * j) / (shape[1] - 1)) - 2.0)&#xA;            if factor != 0.0:&#xA;                rhofft[i, j] /= factor&#xA;            else:&#xA;                rhofft[i, j] = 0.0&#xA;&#xA;    potential = np.zeros(shape, dtype = float)&#xA;    for i in range(shape[0]):&#xA;        potential[i,:] = 0.5 * fft.dct(rhofft[i,:], type = 1)&#xA;    for j in range(shape[1]):&#xA;        potential[:,j] = 0.5 * fft.dct(potential[:,j], type = 1)&#xA;&#xA;################&#xA;    print np.dot(L, potential.flatten()).reshape(shape)&#xA;    print potential&#xA;&lt;/code&gt;&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The charge density is the following,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;[[ 0.11111111  0.11111111  0.11111111]&#xA; [ 0.11111111 -0.88888889  0.11111111]&#xA; [ 0.11111111  0.11111111  0.11111111]]&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while, multiplying the solution with the Laplacian $L$ gives,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;[[ 0.25  0.25  0.25]&#xA; [ 0.25 -0.75  0.25]&#xA; [ 0.25  0.25  0.25]]&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;instead of the same results as above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been staring to the code for some time now and am unable to understand what I'm doing wrong. I've even checked to see if scipy's discrete cosine transform gives the correct results and that seems fine too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone could point out my mistake, I would be really grateful!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: I found out that if I multiply the solution by $L - J$, where $J$ is a matrix filled with ones, instead of just $L$, I do get the expected charge density. Why is that?&lt;/p&gt;&#xA;" OwnerUserId="9361" LastEditorUserId="9361" LastEditDate="2014-06-18T14:07:46.800" LastActivityDate="2018-03-24T16:39:37.753" Title="Poisson equation with Neumann boundary conditions" Tags="|finite-difference|python|poisson|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="12914" PostTypeId="2" ParentId="11632" CreationDate="2014-06-16T20:52:48.290" Score="1" Body="&lt;p&gt;Your equation basically amounts to solving&#xA;$$&#xA;\begin{bmatrix}&#xA;p\\&#xA;q\end{bmatrix} = \nabla z$$&#xA;for $z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already noticed in amlrg's answer, this system is overdetermined and hence, one can go for a least squares solution, i.e. the solution of&#xA;$$&#xA;\min_z \| \begin{bmatrix}&#xA;p\\&#xA;q\end{bmatrix} - \nabla z\|^2.&#xA;$$&#xA;The solution of this is obtained by solving the corresponding normal equations. As the transpose of the gradient operator is the negative divergence you obtain the equation&#xA;$$&#xA;-\textrm{div}\nabla z = -\textrm{div}\begin{bmatrix}&#xA;p\\&#xA;q\end{bmatrix}$$&#xA;which is nothing else than&#xA;$$&#xA;-\Delta z = -\partial_x p - \partial_y q&#xA;$$&#xA;and this can be solved, in principle, like any other Poisson equation (e.g. by CG with appropriate preconditioning).&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2014-06-16T20:52:48.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12915" PostTypeId="1" AcceptedAnswerId="12917" CreationDate="2014-06-16T21:35:24.780" Score="4" ViewCount="124" Body="&lt;p&gt;I know there are similar questions asked on other StackExchange sites, but I think development in Computational Science may work differently because of the computing environments that are common. I often can not run reasonable tests on my local machine due to time or memory constraints (or maybe I'm developing for intel coprocessors). I also can't really test distributed parallelism on my local machine, Thus I log in to a remote cluster and run my software there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Developing locally is good because I usually have root on my own machine and can install nice tools like valgrind, gdb, libclang (for source code completion), ranger, and others. However, my computer likely has different compiler and library versions than are available on a remote machine (which is shared and I don't have privileges on).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Developing remotely ensures that compiler and library versions are the same and that I can edit, compile and run without having to sync my changes in some way. I can sync easily with git, but it can be a pain to commit, push and then pull just to fix a semicolon I forgot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I'd have all my tools on the remote machine, but without privileges some can be really annoying to install when they have several of dependencies and I can't use a package manager without root.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the best way to develop using a remote machine that I don't have privileges on?&lt;/p&gt;&#xA;" OwnerUserId="4420" LastActivityDate="2014-06-18T11:57:46.343" Title="Local versus remote development" Tags="|parallel-computing|libraries|testing|compiling|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12916" PostTypeId="2" ParentId="12880" CreationDate="2014-06-16T21:42:57.190" Score="1" Body="&lt;p&gt;The question changed, so this a new answer. If&#xA;$(z,J(x)z)\geq\epsilon&amp;gt;0$, then take (w.l.o.g.) $f(0)=0$, and consider a&#xA;path $g:[0,1]\to\mathbb{R}^n$ such that&#xA;$$ g(0) = 0, \qquad f(g(0)) = 0, \qquad f(g(t)) = y t, \quad 0\leq&#xA;t\leq 1, $$&#xA;so that $g(1)$ is the solution to $f(x)=y$. Then we differentiate with&#xA;respect to $t$ to get an ODE for $g(t)$:&#xA;$$ g'(t) = J(g(t))^{-1}y, \qquad g(0) = 0. $$&#xA;By the Picard-Lindelof theorem, it is enough to have the function&#xA;$g\mapsto J(g)^{-1}y$ be Lipschitz everywhere on $\mathbb{R}^n$ for a&#xA;unique solution to the ODE to exist. Since we already have&#xA;$\|J^{-1}\|\leq\frac1\epsilon$, it is enough for $J$ to have uniformly&#xA;bounded derivatives:&#xA;$$ \frac{\partial}{\partial g_i}J(g)^{-1}y = -J(g)^{-1}\frac{\partial&#xA;  J}{\partial g_i}J(g)^{-1}y, $$&#xA;so $\|(J^{-1}y)'\| \leq \frac{1}{\epsilon^2}\|y\|\|\partial_i J\|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, even with $n=1$, $J\geq\epsilon$ does not imply $(J^{-1})'$&#xA;is bounded. For example one could take $J=2+\sin x^2$ (whose&#xA;antiderivative $f$ is given in terms of the Fresnel integral $S$),&#xA;which is bounded above and below, but has unbounded derivative, so&#xA;Picard-Lindelof is not applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the Peano existence theorem, on the other hand, if the function&#xA;$\|J(g)^{-1}y\|$ is bounded by $\frac1\epsilon\|y\|$ over the range&#xA;$\|g\|\leq c$, then there exists at least one solution on the interval&#xA;$|t|\leq c \epsilon/\|y\|$. (Also note that this is an autonomous system of&#xA;ODEs, meaning no $t$ dependence.) In this case, $\|J^{-1}\|$ is&#xA;bounded by $\frac1\epsilon$ everywhere independent of $g$, so the theorem guarantees&#xA;existence on an arbitrary interval, including $[0,1]$, which is sufficient to establish $f(g(1))=y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I couldn't think of a simpler answer to your question that wouldn't go&#xA;through ODE theory. Maybe someone else can do better.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-06-16T21:42:57.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12917" PostTypeId="2" ParentId="12915" CreationDate="2014-06-16T21:47:50.373" Score="5" Body="&lt;p&gt;Ask your remote host to install what you need. We do this all the time for folks where I work. Typically they can help you out. Also, it's OK to do some of your development on your machine and then port your results to the remote machine.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-06-16T21:47:50.373" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12918" PostTypeId="2" ParentId="12912" CreationDate="2014-06-17T00:36:13.887" Score="1" Body="&lt;p&gt;The Intel MKL version of sgeqp3 uses multithreads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how well it scales to 20 cores, however.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-06-17T00:36:13.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12921" PostTypeId="1" CreationDate="2014-06-17T10:11:47.817" Score="1" ViewCount="62" Body="&lt;p&gt;I have to write an algorithm/code to reconstruct an unknown 3D scalar potential $ V(\mathbf{r})$ up to a specified threshold $E_c$ (assume that the potential is monotonic)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem has the following rules:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You start from the minimum of the potential and define a grid&lt;/li&gt;&#xA;&lt;li&gt;you can only do &quot;small&quot; steps, i.e. you can only move to the neighboring points with respect to where you are (random sampling is not possible)&lt;/li&gt;&#xA;&lt;li&gt;at the end of the procedure, all the points $\mathbf{r}_i$ of the grid having $V(\mathbf{r}_i)&amp;lt;E_c$ must be known.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is the a algorithm or a keyword that I can look for in google for this kind of problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9476" LastActivityDate="2014-06-17T10:11:47.817" Title="Algorithm to explore a 3D scalar potential" Tags="|algorithms|spatial-data|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="12922" PostTypeId="1" CreationDate="2014-06-17T11:16:18.613" Score="1" ViewCount="359" Body="&lt;p&gt;I am using Matlab to solve Ax=b and my A is very large, sparse, binary and also rectangular. I saw the Matlab backlash \ operator help and it states that if A is rectangular then it will use the QR solver to solve it. I was wondering if this is the right approach, because QR is not efficient for large sparse matrices. If this is true, then what method would you recommend to solve my system of equations?&lt;/p&gt;&#xA;" OwnerUserId="9478" LastActivityDate="2014-08-17T10:30:32.120" Title="solving underdetermined system of equations with a sparse matrix as input" Tags="|linear-algebra|matlab|sparse-matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12924" PostTypeId="1" CreationDate="2014-06-17T14:19:14.810" Score="1" ViewCount="540" Body="&lt;p&gt;I am working on a Universe Simulator. I'm stuck on creating the dark matters webs of the universe that look like this: &lt;a href=&quot;http://www.mpa-garching.mpg.de/galform/data_vis/sim3dnew-highres.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mpa-garching.mpg.de/galform/data_vis/sim3dnew-highres.png&lt;/a&gt; I am generating the particles as solid spheres with the radius of 1. How can I make the particles slowly move toward each other as demonstrated in the picture above to form &quot;webs&quot; of these particles? If anyone has any ideas or might have some examples of how this can be done that would be great. Code in either c/c++/python would be great. (I have the first step of the picture done) Here is a picture of what I have done: &lt;a href=&quot;https://i.stack.imgur.com/TD4YQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/TD4YQ.png&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9479" LastEditorUserId="9479" LastEditDate="2014-06-17T16:53:09.490" LastActivityDate="2015-03-20T10:21:14.840" Title="PyOpenGL/OpenGL Generating Cosmic NBodys" Tags="|python|simulation|c|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12925" PostTypeId="1" AcceptedAnswerId="12926" CreationDate="2014-06-17T15:03:53.467" Score="3" ViewCount="207" Body="&lt;p&gt;I  solved a simple test example by overlapping domain decomposition. The problem domain is a rectangular that is decomposed to two domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the value on the intersection boundary is guessed at the first time and updated at each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to solve this problem by non-overlapping method. But when I use this procedure, the answer in all steps are the same. Am I wrong? or I should solve this, differently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please hint me.&lt;/p&gt;&#xA;" OwnerUserId="8271" LastActivityDate="2014-06-17T16:53:26.490" Title="nonoverlapping domain decomposition" Tags="|domain-decomposition|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12926" PostTypeId="2" ParentId="12925" CreationDate="2014-06-17T16:53:26.490" Score="6" Body="&lt;p&gt;Short answer: yes, you have to use something different, e.g. a &lt;a href=&quot;http://en.wikipedia.org/wiki/Neumann%E2%80%93Neumann_methods&quot;&gt;Neumann-Neumann method&lt;/a&gt;. A good reference is &lt;a href=&quot;http://books.google.com/books?isbn=3540266623&quot;&gt;Widlund's book&lt;/a&gt;. Non-overlapping methods are based on the principle that, if $u$ solves the Poisson equation in a domain $\Omega$ which has been partitioned into two domains $\Omega_1$, $\Omega_2$, at the shared boundary $\Gamma$ of both domains, $u$ and its gradient have to match up across $\Gamma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By contrast, for an overlapping DD method, the intersection of the domains $\Omega_1$ and $\Omega_2$ has a non-empty interior. You can use the interior values of $u$ in $\Omega_1$ as Dirichlet boundary conditions to solve the Poisson equation in $\Omega_2$, then use the interior values in $\Omega_2$ to update the solution in $\Omega_1$, and so forth. If you try and apply an overlapping method to domains that don't actually overlap, there's no room for the solutions on each sub-domain to disagree, and so you'll update neither.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More to the point, the convergence rate of an overlapping DD method goes like $H/h$, where $H$ is the width of the overlap and $h$ is the mesh width*. There's a trade-off: you can have a really small overlap and cheap subdomain solves, but the method won't converge fast; or you can have a large overlap with more expensive subdomain solves, but the whole method converges faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*Unless you use a global coarse solve, but now we're getting off topic.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-06-17T16:53:26.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12927" PostTypeId="1" CreationDate="2014-06-17T21:27:17.767" Score="3" ViewCount="89" Body="&lt;p&gt;I'm looking to solve the Schrödinger equation with a basis made of a tensor product of basis splines. A number of papers describe calculations made with a program designed this way, but they never describe how they got the knot sequence they use to define the basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How does one go about choosing a knot sequence for a set of B-splines to be used as a basis for the solution of a PDE?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2014-06-17T21:27:17.767" Title="How should I choose the knot sequence when using B-splines as a basis for solving a PDE?" Tags="|pde|basis-set|b-spline|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12929" PostTypeId="1" CreationDate="2014-06-18T08:32:37.527" Score="5" ViewCount="670" Body="&lt;p&gt;I have written some code that can do Kalman filtering (using a number of different Kalman-type filters [Information Filter et al.]) for Linear Gaussian State Space Analysis for an n-dimensional state vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us consider the linear Gaussian State Space model&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_t = \mathbf{Z}_{t}\alpha_{t} + \epsilon_{t},$$&#xA;$$\alpha_{t + 1} = \mathbf{T}_{t}\alpha_{t} + \mathbf{R}_{t}\eta_{t},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $y_{t}$ is our observation vector, $\alpha_{t}$ our state vector at time step $t$. The quantities in bold are the transformation matrices of the state space model which are set according to the characteristics of the system under consideration. We also have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\epsilon_{t} \sim NID(0, \mathbf{H}_{t}),$$&#xA;$$\eta_{t} \sim NID(0, \mathbf{Q}_{t}),$$&#xA;$$\alpha_{1} \sim NID(a_{1}, \mathbf{P}_{1}).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $t = 1,\ldots, n$. Now, I have derived and implemented the recursion for the Kalman Filter for this generic state space model by guessing the initial parameters and variance matrices $\mathbf{H}_{1}$ and $\mathbf{Q}_{1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the points are the Nile River water levels for Jan over 100 years, the line is the Kalamn Estimated state, and the dashed lines are the 90% confidence levels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in order to estimate the initial parameters of my model ($\mathbf{H}_{1}$ and $\mathbf{Q}_{1}$) I have adopted the Expectation Maximisation method to maximise the loglikelihood function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\log L(Y_{n}|\psi) = -\frac{np}{2}\log(2\pi) - \frac{1}{2}\sum^{n}_{t = 1}(log|\mathbf{F}_{t}| + v^{\mathsf{T}}_{t}\mathbf{F}_{t}^{-1}v_{t}), \;\;\;\; (1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with respect to parameter vector $\psi$. Where $v_{t}$ is the state error and $\mathbf{F}_{t}$ is the state error variance. The vector $\psi$ contains the unknown parameters (i.e., either the measurement and state disturbance variances themselves, or a re-parametrisation of these same variances, see below)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A crucial role in the maximisation of (1) is played by the gradient or score vector&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_{1}(\psi) = \partial \log L(Y_{n}|\psi)/\partial\psi,\;\;\;\; (2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least two algorithms can be used to maximise (1). The first is the so-called EM (Expectation-Maximisation) algorithm, and the second is the BFGS (Broyden-Fletcher-Goldfarb-Shannon) algorithm. I have successfully implemented the EM algorithm and this is providing correct results but is very slow; in light of this I now wish to implement the BFGS algorithm as this has better performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, from (2) using Taylor's theorem we can write &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_{1}(\psi) \simeq \tilde{\partial}_{1}(\psi) + \tilde{\partial}_{2}(\psi)(\psi - \tilde{\psi}),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\partial_{2}(\psi)$ is the Hessian matrix or second derivative of the loglikelihood function wrt $\psi$ and $\tilde{\partial}_{N}(\psi) = \partial_{N}(\psi)|_{\psi = \tilde{\psi}}$. So now we can write our updated parameter vector as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar{\psi} = \tilde{\psi} - \tilde{\partial}_{2}(\psi)^{-1}\tilde{\partial}_{1}(\psi) = \tilde{\psi} - s\tilde{\pi(\psi)}.\;\;\;\; (3)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where here $\tilde{\pi(\psi)} = - \tilde{\partial}_{2}(\psi)^{-1}\tilde{\partial}_{1}(\psi)$ and $s$ is our step size. So the basic outline of the BFGS algorithm to get our parameter vector and maximize (1) using BFGS is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Initialise parameter vector $\psi = \psi^{*}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then apply the Kalman and disturbance smoothing filters and thus for score vector (2) at $\psi = \psi^{*}$. When running the Kalman Filter evaluate the log-likelihood function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use (3) to obtain new values for $\psi$ given by $\psi^{+}$ and go back to step 2 until the value of loglikelihood function (1) no longer improves.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This seems fine, but my problem is how to calculate the step size $s$ in (3) above. I know a line search is required, but remember $\psi$ is a vector and it is not clear what function I can use in a traditional line search technique in order to evaluate $s$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question: what line search method should I use in order to evaluate $s$?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Ps. Assume that I would like to use the zoom line search algorithm (algorithm 3.5 of &lt;a href=&quot;http://faculty.bracu.ac.bd/~faruqe/books/numericalanalysis/Numerical%20Optimization%202ed.pdf&quot; rel=&quot;nofollow&quot;&gt;Numerical Optimization by Nocedal &amp;amp; Wright&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Further to the very helpful answer by @LKlevin, I want to classify my problems more explicitly...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$^{*}$Most (L-)BFGS methods accept vector functions, that is a function $f$ that depends on a n-dimensional (in this case let us assume two-dimensions) parameter vector $\mathbf{x} = (x_{1}, x_{2})^{\mathsf{T}}$, so if we had $F(x_{1}, x_{2}) = 2x_{2}^{2} + 4x_{2}$ you would pass into the BFGS algo. $F$ and $\nabla{F}$. This is fine and the text I reference as a PDF above tells me all I need to know here in terms of using BFGS to minimise this function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the case I have above, from (3) we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar{\psi} = \tilde{\psi} - \tilde{\partial}_{2}(\psi)^{-1}\tilde{\partial}_{1}(\psi) = \tilde{\psi} - s\tilde{\pi(\psi)},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[\begin{array}{c}&#xA;\psi_{1}\\&#xA;\psi_{2}\\&#xA;\end{array}\right]  = \left[\begin{array}{c}&#xA;\tilde{\psi}_{1}\\&#xA;\tilde{\psi}_{2}\\&#xA;\end{array}\right] - \mathcal{H}\,\partial_{1}(\psi)\left[\begin{array}{c}&#xA;\tilde{\psi}_{1}\\&#xA;\tilde{\psi}_{2}\\&#xA;\end{array}\right] = \left[\begin{array}{c}&#xA;\tilde{\psi}_{1}\\&#xA;\tilde{\psi}_{2}\\&#xA;\end{array}\right] - s\left[\begin{array}{c}&#xA;\tilde{\pi}_{1}(\psi_{1})\\&#xA;\tilde{\pi}_{2}(\psi_{2})\\&#xA;\end{array}\right],$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal{H}$ is the inverse Hessian matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the problem seems to consist of $n$ linear optimizations, not a single equation depending on a vector of parameters. My problem here is the use of the vectors $\psi$ in this case above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can I optimise this without splitting the n-dimensional (2 dimensional in the example above) vector into $n$ equations which I can optimise using functions as described in the paragraph $^{*}$ above? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue is that the loglikelihood function is not explicitly dependent on $\psi$, so how can i use this in the line search method?&lt;/p&gt;&#xA;" OwnerUserId="8237" LastEditorUserId="8237" LastEditDate="2014-06-24T11:40:45.473" LastActivityDate="2014-06-24T21:26:01.750" Title="Maximum Likelihood Estimation for State Space Models using BFGS" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12930" PostTypeId="2" ParentId="12922" CreationDate="2014-06-18T09:34:49.407" Score="1" Body="&lt;p&gt;It depends on the sparsity pattern. As a first point, did you define your matrix as a sparse matrix, because Matlab has a different suite of operations that it applies to sparse matrices, compared to sparsely filled dense matrices. Depending on your problem, you can use also iterative methods, which also have different implementations for sparse matrices. Depending on your sparsity pattern, you can also specify a smarter pre-ordering to be more efficient with the fill-ins, if you use a direct solver.&lt;/p&gt;&#xA;" OwnerUserId="4596" LastActivityDate="2014-06-18T09:34:49.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12931" PostTypeId="2" ParentId="12915" CreationDate="2014-06-18T11:57:46.343" Score="4" Body="&lt;p&gt;I want to add something not mentioned yet here that might be useful (too long for a comment), because you say that syncing with git is a pain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is the tool &lt;a href=&quot;http://fuse.sourceforge.net/sshfs.html&quot; rel=&quot;nofollow&quot;&gt;sshfs&lt;/a&gt; (and I believe its osx equivalent is &lt;a href=&quot;https://github.com/osxfuse/osxfuse&quot; rel=&quot;nofollow&quot;&gt;osxfuse&lt;/a&gt;). If your remote machine allows ssh connections, sshfs will be able to mount the remote filesystem locally over ssh. This is useful if, as I think you say, you can't mount remote directories already. Thus typically you would do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkdir remote-sandbox&#xA;sshfs ssh://user@remote:~/sandbox remote-sandbox&#xA;cd remote-sandbox&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then edit the files in remote-sandbox directly. This would save you from having to synchronize files using git or rsync or scp, or what have you.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-06-18T11:57:46.343" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12932" PostTypeId="2" ParentId="11836" CreationDate="2014-06-18T14:25:28.880" Score="2" Body="&lt;p&gt;Just to point out to a great &lt;em&gt;free&lt;/em&gt; Open Source software used exactly for the purpose of modeling of a multibody system, just like your fishing rod. It's called &lt;a href=&quot;https://www.mbdyn.org/&quot; rel=&quot;nofollow&quot;&gt;MBDyn&lt;/a&gt;, and I've used it to model the dynamics of multicomponent airfoils. There is ample &lt;a href=&quot;https://www.mbdyn.org/?Documentation&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; available, and also slides that describe the physics. See for instance slide 25 of this &lt;a href=&quot;https://www.mbdyn.org/userfiles/documents/MBDyn-Overview.pdf&quot; rel=&quot;nofollow&quot;&gt;presentation&lt;/a&gt;, the mutually connected &lt;em&gt;deformable&lt;/em&gt; elements correspond exactly to the fishing rod. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest that you go through the tutorials and join the mailing list for questions. I've seen a presentation of prof. Masarati where he showed how a large part of the dynamical system of an entire helicopter (blades, rotor transmission, the whole deal) was modeled and analyzed using MBDyn, so I am fairly sure that the people on the list will be able to guide you with your model. This way, you won't have to build a framework just for yourself, that is later maybe stiff when it comes to modifications and extensions. &lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2014-06-18T14:25:28.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12933" PostTypeId="2" ParentId="11766" CreationDate="2014-06-18T14:44:11.617" Score="0" Body="&lt;p&gt;The $a_p$ is assembled from all the coefficients that multiply $\phi_p^n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with the scalar transport equation in the discretized form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\frac{3}{2}\rho_P\phi_P^n - 2\rho_P\phi_P^o + \frac{1}{2}\rho_P \phi_P^{oo}}{\Delta t} + \sum_{f}F\phi_f^n - \sum_{f}(\rho\gamma)_f(\nabla{\phi})_f^n = S_P^o&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the key point to understand where the $a_p$ comes from are the face values $\phi_f^n$, that will be interpolated from the values between the neighboring cells. Looking at the equation above, you can already see that you have the $\phi_P^n$ value in the temporal term, but the interpolation of the values from the new time step will lead to the final &lt;em&gt;implicit&lt;/em&gt; algebraic equation for cell $P$, and more $\phi_P^n$ terms will appear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you choose the weighted Central Differencing Scheme (CDS) to interpolate $\phi_P^n$ for all $\phi_f^n$ of the above equation, leading to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi_f^n = f_x \phi_P^n + (1 - f_x) \phi_N^n$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f_x$ is the CDS weight $P$ is the &lt;em&gt;owner&lt;/em&gt; cell and N is the face &lt;em&gt;neighbor&lt;/em&gt; cell. For every occurence of $\phi_f^n$ in the first equation, the interpolation will introduce a $\phi_P^n$ and $\phi_N^n$. If you do this by hand and sort out everything that multiplies $\phi_P^n$, you'll get the &quot;formula&quot; for $a_p$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another important thing is also to understand that $P$ and $N$ exchange interpretations based on the orientation of the surface are normal vector of the finite volume face. This orientation is defined by the mesh generation step, and is used to optimize the cell sum calculation - check out Hrv's thesis for more info on that. This has no impact in the symbolic calculation of $a_p$, it's an optimization enhancement.&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2014-06-18T14:44:11.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12934" PostTypeId="1" CreationDate="2014-06-18T17:58:52.523" Score="5" ViewCount="172" Body="&lt;p&gt;Working in finite differences, I am using a transformation on the temperature variable &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Theta = \int_{T0}^T \kappa(T)dT$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to linearize the steady-state heat equation into a Poisson equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-\nabla\cdot(\kappa\nabla T)=-S \rightarrow \nabla^2 \Theta= -S$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Dirichlet and Neumann boundary conditions transform in a way such that they have the same form.  However, it seems that a Dirichlet condition for a heat sink pulls out too much heat from the system.  So I'm trying a convective boundary condition, but this Robin boundary condition causes a problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ -\kappa \nabla T = h(T-T_f) \rightarrow -\nabla \Theta=h[T(\Theta) - T_f] $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h$ is a heat transfer coefficient, $T(\Theta)$ is the inverse transform back to the temperature variable, and $T_f$ the ambient temperature outside the boundary.  The inverse transform is nonlinear making the boundary condition nonlinear as well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm just trying to find some ideas on how I could possibly implement this sort of boundary condition in this case.  Can I Just discretize the derivative in the boundary condition as in a Neumann except that instead of a ghost point I have a point outside the domain where the temperature is known ($T_f$) and then solve iteratively?&lt;/p&gt;&#xA;" OwnerUserId="9487" LastActivityDate="2014-06-18T17:58:52.523" Title="Elliptical problem with Robin BC" Tags="|boundary-conditions|elliptic-pde|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="12936" PostTypeId="1" CreationDate="2014-06-18T22:33:55.297" Score="1" ViewCount="66" Body="&lt;p&gt;I'm not sure if this question is appropriate for this site, so I appologize in advance if it's not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the purpose of a certain project I am working on, I would like to have the following sets plotted in 3D: $A = \{(x_1,x_2,x_3) \in \mathbb{R}^3 : x_1 \geq x_2, x_3 = 0\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B = \{(x_1,x_2,x_3) \in \mathbb{R}^3 : x_1 = x_2 \geq x_3 \geq 0\}$ and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C = \{x \in \mathbb{R}^3 : x_1 \geq x_2 = x_3 \geq 0\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that these are not subspaces. I do not know any sort of mathematical software, but if someone could let me know how I could plot these sets in 3D in a (preferably free) program then I would appreciate it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You see, I imagine that plotting 3 different hyperplanes would be easy, but the fact that these sets are not hyperplanes (they are triangle shaped, I believe) makes it harder. I would also like to find the angles in each triangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just read about gnu-plot. Maybe that could help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerDisplayName="user9488" LastActivityDate="2014-06-19T01:29:16.037" Title="Visualizing certain sets" Tags="|visualization|plotting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12937" PostTypeId="2" ParentId="12936" CreationDate="2014-06-19T01:29:16.037" Score="1" Body="&lt;p&gt;I think you're best off trying to understand what these sets are, then you can use any 3D graphing tool (e.g. GNU Plot, MATLAB, Python with matplotlib, etc) to plot them. Here are some tips:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Each part of your set definitions defines a plane. The set is restricted to the plane defined by the equality and bounded by the planes defined by the inequality.&lt;/li&gt;&#xA;&lt;li&gt;Each inequality bounds the set to one half of the plane or the other. You can plot the line represented by the intersection of the &quot;equality plane&quot; and each &quot;inequality plane&quot; to identify the boundary of your set on the plane.&lt;/li&gt;&#xA;&lt;li&gt;You can easily determine which parts of the plane are inside or outside your set based on picking points on the plane and testing them against the inequalities.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The basic idea is to find the equality plane and then the boundaries of the set on this plane. You can then plot the set (or part of it since it's an infinite triangular region in these cases) based on this information. Going through this process for your set $C$ gives the following figures which were made using lines and patches in MATLAB. Python is free and has similar capabilities using matplotlib (e.g. &lt;a href=&quot;http://matplotlib.org/examples/mplot3d/polys3d_demo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://matplotlib.org/examples/mplot3d/polys3d_demo.html&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plot $x_2=x_3$:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/j711b.png&quot; alt=&quot;plane, x2=x3&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plot the set boundaries on this plane (red dashed lines) which are just the inequality planes intersected with the equality plane:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/0EOTu.png&quot; alt=&quot;set boundaries&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Figure out the relevant region (e.g. by picking points and testing the inequalities) and plot just that region, this is your set:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Dnvn8.png&quot; alt=&quot;final set&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-06-19T01:29:16.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12938" PostTypeId="1" AcceptedAnswerId="12939" CreationDate="2014-06-19T03:47:05.587" Score="4" ViewCount="1363" Body="&lt;p&gt;Let us say we have the following equations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dy1/dt = f(y1, t)      [1]&#xA;dy2/dt = g(y2, t)      [2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The equations are such that they are &quot;conservative&quot;, i.e. the following condition should hold:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dy1/dt + dy2/dt = 0    [3]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;scipy.odeint&lt;/code&gt;, I find that I can integrate conservative equations like this just fine for simple systems of ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for larger ones, I get the following issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that this is my derivative function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def deriv_function(y0s, t):&#xA;    ...body defines equations 1, and 2...&#xA;&#xA;    print np.sum(ode)&#xA;    return ode&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note the print statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use &lt;code&gt;scipy.odeint&lt;/code&gt; on &lt;code&gt;deriv_function&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;odeint(deriv_fun, y0s, [0, 0.5])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Due to the print statement, the following gets printed out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-1.38555833473e-13 &amp;lt;--- note, close to zero&#xA;-0.00679107743937&#xA;-0.0067907211796&#xA;-0.0135814423985&#xA;-0.0135810861584&#xA;-0.416522145214&#xA;-0.416523165887&#xA;-0.818209018574&#xA;-0.818211056221&#xA;-1.21864678558&#xA;-1.21864881584&#xA;-2.86735888212&#xA;-2.8673729885&#xA;-2.46855840934&#xA;-2.46856658088&#xA;-3.70632102566&#xA;-3.70631206163&#xA;-4.93200749506&#xA;-4.93200691488&#xA;-6.14577326158&#xA;-6.14577268283&#xA;-8.53799987128&#xA;-8.53799713959&#xA;-10.8839304356&#xA;-10.8839320212&#xA;-13.1845005689&#xA;-13.1845021725&#xA;-15.4406122011&#xA;-15.4406123927&#xA;-17.6531469917&#xA;-17.653147185&#xA;-24.6238415795&#xA;-24.6238498033&#xA;-31.1628867985&#xA;-31.1628947266&#xA;-37.2974784594&#xA;-37.2974547092&#xA;-35.463527103&#xA;-35.4635192949&#xA;-39.5777426955&#xA;-39.5777472677&#xA;-43.5137135424&#xA;-43.5137108017&#xA;-47.2791485087&#xA;-47.2791483993&#xA;-50.881424906&#xA;-50.8814244751&#xA;-54.3275507164&#xA;-54.3275502654    &amp;lt;--- note, not close to zero&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For smaller systems of equations (not the same), the following gets printed out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-1.13686837722e-13&#xA;0.0&#xA;0.0&#xA;0.0&#xA;-1.13686837722e-13&#xA;1.13686837722e-13&#xA;0.0&#xA;0.0&#xA;0.0&#xA;0.0&#xA;0.0&#xA;0.0&#xA;0.0&#xA;-1.13686837722e-13&#xA;0.0&#xA;0.0&#xA;0.0&#xA;1.13686837722e-13&#xA;0.0&#xA;0.0&#xA;0.0&#xA;0.0&#xA;-1.13686837722e-13&#xA;0.0&#xA;0.0&#xA;-1.13686837722e-13&#xA;0.0&#xA;0.0&#xA;0.0&#xA;0.0&#xA;0.0&#xA;0.0&#xA;-1.13686837722e-13&#xA;0.0&#xA;0.0&#xA;5.68434188608e-14&#xA;-5.68434188608e-14&#xA;-5.68434188608e-14&#xA;0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have spent a lot of time trying to figure out if the issue has something to do with the way I have defined the equations, and I am now fairly certain that this is not the case. In order to confirm this, I would like to ask: is it possible at all for a non-conservative system of equations to first print an ode sum that is close to zero, but later print values that are not close to zero?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to think of the question: in the larger system of equations, the ode sum initial prints out to be approximately zero, before increasing. What could be happening in the bowels of the solver for this problem to be occurring?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="9466" LastEditDate="2014-06-19T04:10:33.823" LastActivityDate="2014-06-19T11:26:52.627" Title="scipy odeint: sum of conservative ode equations does not remain zero as it is being solved -- is this normal?" Tags="|python|ode|scipy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12939" PostTypeId="2" ParentId="12938" CreationDate="2014-06-19T04:41:20.220" Score="5" Body="&lt;p&gt;In general, numerical methods do not conserve quantities that may be conserved in a continuous system. The discretized system is typically non-conservative. A smaller time step (equivalently a smaller error tolerance) should help, but if you have a quantity that must be precisely conserved, you will have to use a method that is designed to conserve the desired quantity. &lt;a href=&quot;http://en.wikipedia.org/wiki/Symplectic_integrator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symplectic integrators&lt;/a&gt; are perhaps the most widely known and used examples of such a case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, consider a simple mass-spring oscillator:&lt;br&gt;&#xA;$$\dfrac{dx}{dt} = v$$&#xA;$$\dfrac{dv}{dt} = -\dfrac{k}{m}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The total energy $E = \frac{1}{2} \left( m v^2 + k x^2\right)$ is conserved since&#xA;$$\dfrac{dE}{dt} = m v \dfrac{dv}{dt}+k x \dfrac{dx}{dt} = -kv x + kxv = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you integrate this system using a Runge-Kutta type method the energy will not be conserved. A quick test using MATLAB's &lt;code&gt;ode45&lt;/code&gt; integrator gives the following energy plot which is not as dramatic as your case, but shows a clear decay in the energy:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/XptJE.png&quot; alt=&quot;energy decay&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a Hamiltonian system with Hamiltonian $H(x,v) = E(x,v) = \frac{1}{2} \left( m v^2 + k x^2\right)$ and would be a good candidate for the application of a symplectic integrator which would preserve the total energy.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-06-19T04:41:20.220" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="12941" PostTypeId="2" ParentId="12938" CreationDate="2014-06-19T11:26:52.627" Score="2" Body="&lt;p&gt;There are two possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You have a bug.&lt;/li&gt;&#xA;&lt;li&gt;You are seeing an accumulation of discretization errors.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Without more detail, it is impossible for us to tell you which is actually happening in your simulation.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-06-19T11:26:52.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12943" PostTypeId="2" ParentId="11528" CreationDate="2014-06-20T04:45:33.747" Score="5" Body="&lt;p&gt;{Assuming you still need the code} Let A1.wrl is your wrl file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;import numpy as np&#xA;import re&#xA;holder = []&#xA;with open(&quot;A1.wrl&quot;, &quot;rb&quot;) as vrml:&#xA;    for lines in vrml:&#xA;        a = re.findall(&quot;[-0-9]{1,3}.[0-9]{6}&quot;, lines)&#xA;        if len(a) == 3:&#xA;            holder.append(map(float, a))&#xA;&#xA;holder_array = np.array(holder) #if you want numpy array&#xA;&#xA;#3D Plotting&#xA;x,y,z = zip(*holder)&#xA;fig = plt.figure()&#xA;ax = Axes3D(fig)&#xA;ax.plot(x,y,z)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My A.wrl: &lt;a href=&quot;http://pastebin.com/PeVXbR5C&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pastebin.com/PeVXbR5C&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;File generate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/MSpRL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9499" LastEditorUserId="9499" LastEditDate="2014-08-21T13:03:49.840" LastActivityDate="2014-08-21T13:03:49.840" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="12945" PostTypeId="2" ParentId="10044" CreationDate="2014-06-20T12:02:55.890" Score="2" Body="&lt;p&gt;Exponential time integrators are exact for linear systems of ordinary differential equations with constant coefficients and are thus trivially A-stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See [1] and the numerous references therein.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a general system of ODEs, the basic idea is to split up the terms in such a way that the largest stable time step for the exponential integrator is significantly larger than for a corresponding explicit discretization, thus reducing the computer time required to solve the equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achieve this goal without sacrificing accuracy, three issues must be addressed:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;all terms required for the fastest solution components should be treated in the linear part $A$,&lt;/li&gt;&#xA;&lt;li&gt;the energy in those fast components should be insignificant, and&lt;/li&gt;&#xA;&lt;li&gt;the resulting matrix exponential should be relatively easy to solve.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Points 1 and 3 should be obvious. Point 2 is not always clearly stated, but it is necessary: contrary to implicit method, where the fastest waves are slowed, solution modes treated in the exponential part are not distorted by the time&#xA;discretization, and thus an accurate solution can be obtained only if there is little energy in these modes. The slowest modes are usually treated in $N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Tokman, M. 2006. Efficient integration of large stiff systems of ODEs with exponential propagation iterative (EPI) methods. J. Comput. Phys. 213, 748–776.&lt;/p&gt;&#xA;" OwnerUserId="8222" LastEditorUserId="8222" LastEditDate="2014-06-20T16:09:54.630" LastActivityDate="2014-06-20T16:09:54.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12946" PostTypeId="1" AcceptedAnswerId="12983" CreationDate="2014-06-20T12:05:25.953" Score="9" ViewCount="582" Body="&lt;p&gt;I am interested in computing the solution of a lage system of ODEs using a krylov method as in [1]. Such method involve functions related to the exponential (the so-called $\varphi$-functions). It essentially consists of computing the action of the matrix function by constructing a Krylov subspace using Arnoldi iteration and projecting the function on this subspace. This reduce the problem to compute the exponential of a much smaller Hessenberg matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that there are several algorithms to compute the exponential (see [2][3] and references therein). I wonder if there is a special algorithm to calculate the exponential that can takes advantage of the fact that the matrix is Hessenberg ?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[1] Sidje, R. B. (1998). Expokit: a software package for computing matrix exponentials. ACM Transactions on Mathematical Software (TOMS), 24(1), 130-156.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Moler, C., &amp;amp; Van Loan, C. (1978). Nineteen dubious ways to compute the exponential of a matrix. SIAM review, 20(4), 801-836.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Moler, C., &amp;amp; Van Loan, C. (2003). Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM review, 45(1), 3-49.&lt;/p&gt;&#xA;" OwnerUserId="8222" LastEditorUserId="8222" LastEditDate="2014-06-23T11:13:29.363" LastActivityDate="2014-06-24T14:47:32.760" Title="Algorithm to calculate the exponential of an Hessenberg matrix" Tags="|linear-algebra|matrix|numerical-analysis|time-integration|krylov-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12947" PostTypeId="2" ParentId="10044" CreationDate="2014-06-20T13:56:00.143" Score="2" Body="&lt;p&gt;It depends on your nonlinear function $\mathcal{N}.$  The stability means the solution continuously depends on the data. In particular, for discretised ODEs, it means if you give a perturbation to $y_n,$ the perturbation propagating to $y_{n+m}$ is bounded uniformly w.r.t. $m.$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, we have&#xA;$$&#xA;|\tilde{y}_{n+1}-y_{n+1}|\leq |e^{-Ah}|\cdot|\tilde{y}_n-y_n| &#xA;+ (h + \frac{1}{2}|A|h^2 + ..)|\mathcal{N}(t_n,\tilde{y}_n)-\mathcal{N}(t_n,{y}_n)|.&#xA;$$&#xA;Assuming $\mathcal{N}$ is Lipschitz w.r.t. the second argument, you will need something like&#xA;$$&#xA;|e^{-Ah}| + Ch = 1 + \mathcal{O}(h)&#xA;$$&#xA;for asymptotic stability (stable as $h\rightarrow 0$) or if you want absolute stability&#xA;$$&#xA;|e^{-Ah}| + Ch&amp;lt;1,&#xA;$$&#xA;where $C$ depends on $|A|.$&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2014-06-21T06:30:17.390" LastActivityDate="2014-06-21T06:30:17.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12948" PostTypeId="1" AcceptedAnswerId="14750" CreationDate="2014-06-20T22:54:04.543" Score="6" ViewCount="4377" Body="&lt;p&gt;How can I access a field in Paraview's programmable filter as a numpy array?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Import an existing field as a numpy array&lt;/li&gt;&#xA;&lt;li&gt;Create a similar array for output&lt;/li&gt;&#xA;&lt;li&gt;Register it as a new field for visualization&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm familiar with the &lt;a href=&quot;http://www.paraview.org/Wiki/index.php?title=ParaView/Users_Guide/Python_Programmable_Filter&quot; rel=&quot;nofollow&quot;&gt;wiki&lt;/a&gt;, but it doesn't address the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;elev0 = inputs[0].PointData['Elevation']&#xA;elev1 = inputs[1].PointData['Elevation']&#xA;output.PointData.append(elev1 - elev0, 'difference')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd like to do (assuming elev0 and elev1 are numpy arrays):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;diff = elev1 - elev0&#xA;diff[ diff &amp;gt; 1.0 ] = 1.0&#xA;output.PointData.append(diff, 'difference')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this doesn't work as those are VTKArray objects.&lt;/p&gt;&#xA;" OwnerUserId="9503" LastEditorUserId="9647" LastEditDate="2014-09-02T14:15:39.033" LastActivityDate="2014-10-02T17:26:34.083" Title="Using numpy arrays in Paraview programmable filter" Tags="|python|visualization|numpy|paraview|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12949" PostTypeId="1" CreationDate="2014-06-21T07:14:36.823" Score="0" ViewCount="90" Body="&lt;p&gt;all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have data from a CFD simulation for a wing, and I need to obtain a visualization of the span-wise load distribution. I'm attempting to do this using Tecplot. I've tried using 'slices' and 'vectors', but got stuck. Any ideas on how to do this? Any input from engineers would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9504" LastActivityDate="2014-06-21T07:14:36.823" Title="Tecplot Span-Wise Loading Visualization" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12950" PostTypeId="2" ParentId="11658" CreationDate="2014-06-21T16:41:06.443" Score="0" Body="&lt;p&gt;As the subspace is simply defined as a subset of dimensions, it may be possible to use a projection of your data in a reduced dimensionality &lt;a href=&quot;http://infolab.stanford.edu/~ullman/mmds/ch11.pdf&quot; rel=&quot;nofollow&quot;&gt;(1)&lt;/a&gt;  and a eucliedean distance method to partition it. I had some good results with SVD and kmeans in data problem similar to yours. Taking&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A = U \Sigma  V^{T} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A_{k} = \Sigma_{k}V_{k}^{T} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and applying kmeans to $A_{k}$ matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ref&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://infolab.stanford.edu/~ullman/mmds/ch11.pdf&quot; rel=&quot;nofollow&quot;&gt;infolab.stanford.edu/~ullman/mmds/ch11.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8260" LastActivityDate="2014-06-21T16:41:06.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12951" PostTypeId="1" AcceptedAnswerId="12952" CreationDate="2014-06-21T18:06:22.977" Score="7" ViewCount="696" Body="&lt;p&gt;I'm part of a Computational Science course and come from a non-programming background, so please forgive me my ignorance. I'm working on a set of code in C to numerically solve the Navier Stokes equations for a Computational Fluid Dynamics course. As a part of our task, I have to write code that runs on a single processor (serially) as well as provide the choice to parallelize that same code with OpenMP and/or MPI (to utilize multiple processors, depending on user input at the beginning of run time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping for advice from the community on doing this and I have a few questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I be thinking of an if/else structure for my program? (The program would need to branch towards serial code, or MPI enabled code, or OpenMP code, or a hybrid of MPI+OpenMP, and this would mean that much of the code would be duplicated).&#xA;Should I construct the actions that would be duplicated to be separate functions and call these functions during run time where they are needed? (But from what I understand function calls would increase the overhead of my code in general, regardless of what choice is picked)&#xA;Would there be a way to hide the pre-processor directives, or MPI-related functions (e.g. MPI_Init()) so that they would conveniently appear or disappear depending on the choice the user made at the beginning or run time?&#xA;Finally, the purpose of this task is to measure the performance benefits of using serial code, vs. OpenMP, vs. MPI, vs. MPI+OpenMP. Does it make sense that I am trying to avoid code duplication since combining the code (to increase readability) would unnecessarily add overhead to my serial code? Such as MPI-related function calls, potentially many if/else conditions, which would skew the performance measurements of my serial code, versus an if/else structure at the beginning of the program which directs it down a completely separate path (as in Q1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I've asked a clear question and have abided by the community's standards. Thank you kindly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I originally asked this question on what I believe to be the general page of stackoverflow and was directed here.)&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/24340121/c-openmp-mpi-serial-program&quot;&gt;https://stackoverflow.com/questions/24340121/c-openmp-mpi-serial-program&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9506" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2018-06-08T18:29:56.450" Title="C - OpenMP, MPI, Serial Program" Tags="|fluid-dynamics|parallel-computing|c|mpi|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12952" PostTypeId="2" ParentId="12951" CreationDate="2014-06-21T20:17:02.757" Score="8" Body="&lt;p&gt;I think some of your issues are more important than others and some of&#xA;your emphasis is misplaced. In pursuing overhead, you are in danger of&#xA;making your program unmaintainable. It is easier to write a common&#xA;program and direct surplus effort somewhere more interesting. I apologize for pontificating like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If statements.&lt;/em&gt; From a strict programming perspective, there is no&#xA;problem having large if-else blocks in your code. If you get your code&#xA;right, what does it matter? On the other hand, if you don't get your&#xA;code right there is an issue. With $n$ if-else blocks present, there&#xA;are something on the order of $2^n$ different paths the execution of&#xA;your program can take. The choice of parallelism support ($4$ paths)&#xA;is not the only choice your program will make. These paths can have&#xA;different outcomes/behaviours (and if you think not, you probably&#xA;can't be sure), which means you have to make sure each one of those&#xA;paths is working correctly, which requires a tremendous amount of&#xA;testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the typical advice is to try and minimize the number of different&#xA;paths that can be taken through your program. The fewer there are, the&#xA;fewer need to be checked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Serial/OMP/MPI/OMP+MPI&lt;/em&gt;. Writing four different versions of the same&#xA;program that are all required to be functionally identical sounds to&#xA;me like a daft idea. It is not impossible, but why? The extra effort&#xA;of making sure they &lt;em&gt;all&lt;/em&gt; work exactly the same is wasted. Try to&#xA;maximize the amount of code they share between each other, especially&#xA;mathematically-tricky code. Spend the extra effort on something&#xA;interesting instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Overhead&lt;/em&gt;. In a numerically-intensive program, the overhead of&#xA;calling a few function is absolutely tiny relative to the&#xA;computational work, and the overhead of &quot;not needing to communicate&quot;&#xA;is virtually nil. I would not worry about a few function calls. It&#xA;would matter in a non-optimized tight inner loop, but for API calls&#xA;this is not a consideration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;MPI.&lt;/em&gt; I don't know how to easily write a program in a way that&#xA;exactly the same program would easily compile and work both with MPI&#xA;and without any MPI at all. It is probably far too much effort to try&#xA;and hide all the MPI calls with ifdefs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way is to have two programs, MPI and non-MPI, that rely on the&#xA;same core mathematical routines. Another way is to make sure the&#xA;serial version is identical to the MPI version running in a single&#xA;process. The only overhead here you need to take note of is the&#xA;overhead of setting up the MPI environment: you need to make sure it's&#xA;not included in any timing results for the computation, MPI or non-MPI. The overhead of &quot;communicating $0$ bytes&quot; should be nil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;OpenMP.&lt;/em&gt; Given how much of OpenMP's parallelism is accessed through&#xA;compiler pragmas, rather than explicit API calls, many OpenMP programs&#xA;will compile with and without OpenMP support. It is also probably not&#xA;worth the effort to hide OpenMP API calls with ifdefs: just run the&#xA;program with only one thread.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-06-21T20:17:02.757" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="12953" PostTypeId="2" ParentId="12948" CreationDate="2014-06-21T21:08:18.680" Score="4" Body="&lt;p&gt;Reading the Paraview python API, found the following solution to convert back and forth between VTKArray and numpy arrays. This uses the &lt;a href=&quot;http://www.paraview.org/ParaView3/Doc/Nightly/www/py-doc/paraview.numpy_support.html&quot; rel=&quot;nofollow&quot;&gt;numpy_support&lt;/a&gt; and &lt;a href=&quot;http://www.paraview.org/ParaView3/Doc/Nightly/www/py-doc/paraview.vtk.dataset_adapter.html&quot; rel=&quot;nofollow&quot;&gt;vtk.dataset_adapter&lt;/a&gt; modules :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from paraview.numpy_support import vtk_to_numpy&#xA;from paraview.vtk.dataset_adapter import numpyTovtkDataArray, vtkDataArrayToVTKArray&#xA;import numpy as np&#xA;&#xA;# get paraview.vtk.dataset_adapter.VTKArray object&#xA;vtkArr = inputs[0].PointData['Elevation']&#xA;# convert to numpy array, shape is (nNodes,) for scalars, (nNodes,nComponents) for vectors&#xA;npArr = vtk_to_numpy(vtkArr)&#xA;# do math on a copy so that original field is not altered&#xA;npArr2 = npArr.copy()&#xA;npArr2 *= 2.0&#xA;# convert back to VTKArray&#xA;vtkArr2 = vtkDataArrayToVTKArray(numpyTovtkDataArray(npArr2))&#xA;# append new field to output&#xA;output.PointData.append(vtkArr2, 'New field')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9503" LastActivityDate="2014-06-21T21:08:18.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12954" PostTypeId="1" CreationDate="2014-06-22T05:57:43.460" Score="2" ViewCount="829" Body="&lt;p&gt;I would appreciate help choosing a software for the Finite Elements Method (FEM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I wish to model items like ropes, bull whips and fishing rods. (I intend to transfer the model into bio-mechanics, later.)&lt;/strong&gt; E.g. I wish to study how the force/torque/movement applied to the handle of a fishing rod spreads to the end of the fishing rod.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am &lt;strong&gt;not&lt;/strong&gt; primarily interested in different methods of constructing e.g. fishing rods, like &quot;should they be round or hexagonal, solid or hollow&quot;. &lt;strong&gt;I am interested in how they work, how the force at the handle is transmitted to the bait during a cast&lt;/strong&gt;, and similar questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many recommend &lt;a href=&quot;http://www.code-aster.org/V2/spip.php?rubrique2&quot; rel=&quot;nofollow&quot;&gt;Code_Aster with Salome-Meca&lt;/a&gt; for &quot;FEM problems in general&quot;. It seems very advanced, highly developed and &quot;hot&quot;. (Some more advanced docs only in French, though.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got the suggestion to use &lt;a href=&quot;http://www.aero.polimi.it/mbdyn/&quot; rel=&quot;nofollow&quot;&gt;MBDyn&lt;/a&gt;. It does not have a graphical interface. But it seems adapted to my task. (It considers the movement of multiple bodies and elasticity.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_finite_element_software_packages&quot; rel=&quot;nofollow&quot;&gt;many other software packages&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never used FEM. I know it is often used to analyze structural tension in solid objects, the flow of gases and fluids and electromagnetic fields. But those applications are a little different from my applications. My application is more focused on the movement of connected elastic objects. &lt;strong&gt;(I wish to model the fishing rod as connected elastic objects.)&lt;/strong&gt; From what I have seen so far, FEM is more about &quot;static&quot; tension, triangulation of &quot;uniform&quot; matter. But I am just guessing from examples here. I do not have experience of FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought I would start with Code_Aster, but first I would appreciate to hear what members familiar with FEM think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The program should run on a Windows home PC. (If &quot;absolutely needed&quot; I can also set up Linux dual boot, but I just removed it due to storage limitations.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, this is only a private project so the cost is of concern. I could consider spending up to maybe $1000 if I knew that I got something which definitely did the job better. But to me it seems open source and software free of charge are highly developed in this area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What software is best suited for my task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT 22/6:&#xA;To summarize the question: &lt;strong&gt;How does a fishing rod move when the fisherman casts his bait?&lt;/strong&gt;&#xA;(The fishing rod is modeled as a series of segments, the rigidity/elasticity and weight differs in each segment. A description in two dimensions will suffice.)&lt;/p&gt;&#xA;" OwnerUserId="9385" LastEditorUserId="9667" LastEditDate="2016-09-09T18:40:41.000" LastActivityDate="2016-09-09T18:40:41.000" Title="Help choose Finite Elements (FEM) software for elastic, multi body system!" Tags="|finite-element|solid-mechanics|" AnswerCount="3" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="12956" PostTypeId="1" CreationDate="2014-06-22T15:03:46.200" Score="3" ViewCount="929" Body="&lt;p&gt;How can I implement the computation of the diffusion coefficient $D$ using periodic boundary conditions (PBC)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use molecular dynamics of a set of $nboby$ particles with positions $pos(3,nbody)$ in a box of length $length$. The implemetion of the PBC is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do k = 1, 3&#xA;    pos(k,i) = modulo(pos(k,i),length)&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At now I'm using for $D$ the following code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do it=1,nstep&#xA;diff=0&#xA;do i = 1,nbody&#xA;    pos2(:)=pos2(:)+pos(:,i)&#xA;    diff=diff+dot_product(pos(:,i),pos(:,i))&#xA;end do&#xA;diff=diff/nbody-dot_product(pos2(:),pos2(:))/nbody**2&#xA;end do&#xA;diff=diff/nstep/6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which I think it corresponds to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D=\lim_{t\to\infty}\dfrac{&amp;lt;x^2&amp;gt;-&amp;lt;x&amp;gt;^2}{6t}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I'm not very sure that the PBC are taking into account in the right way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks Matteo&lt;/p&gt;&#xA;" OwnerUserId="9514" LastActivityDate="2016-05-18T18:44:22.610" Title="Molecular Dynamics: Diffusion with PBC" Tags="|fortran|molecular-dynamics|diffusion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12957" PostTypeId="1" CreationDate="2014-06-22T15:15:11.353" Score="0" ViewCount="393" Body="&lt;p&gt;I'm doing molecular dynamics of $N$ bodies interacting through Lennard-Jones potential. I'm studying the hysteresis phenomena in the phase transition solid-liquid and liquid-solid. Which parameters can I use for determine the temperature at which the phase transition occurs? I'm using some parameters, as potential energy, kinetic energy and pressure, to determine the transition from solid to liquid.  They show a strong change a value at melting point. However in the opposite transition this parameters are not a good choice, their change it is smaller that their fluctuations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone knows some different parameters and how to implement them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matteo&lt;/p&gt;&#xA;" OwnerUserId="9514" LastActivityDate="2015-01-24T20:37:00.607" Title="Molecular Dynamics : Phase transition Lennard Jones potential" Tags="|fortran|molecular-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12958" PostTypeId="1" CreationDate="2014-06-22T21:21:58.687" Score="1" ViewCount="518" Body="&lt;p&gt;Given a 3D unstructured grid consisting of mixed types of shapes (hex, tet, ...), is there a method to know how many faces (including boundary faces) are contained in the grid?&lt;/p&gt;&#xA;" OwnerUserId="8103" LastActivityDate="2014-07-23T12:32:22.917" Title="Number of faces in a 3D multi-type unstructured grid" Tags="|computational-geometry|unstructured-mesh|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="12959" PostTypeId="1" AcceptedAnswerId="12972" CreationDate="2014-06-23T03:01:56.473" Score="1" ViewCount="2638" Body="&lt;p&gt;{&lt;strong&gt;Summary&lt;/strong&gt;: I have a symbolic expression DCritnF expressed in terms of two variables x1 and x2. I need to find it's numeric value and I used combination of double and subs as given below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FuncVal = double(subs(DCritnF,[x1,x2],[x(1),x(2)]));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Challenge is that since DCritnF is a very convoluted expression (generated by symbolic toolbox using combination of matrix algebra and some calculus), evaluating FuncVal takes atleast 0.1 seconds (used timeit function). This is unacceptable as I'm performing optimization of a function and there are multiple function calls to above line of code and a single optimization run takes anywhere from 4 seconds to 40 seconds depending on type of algorithm used. I already determined using profiler that the one and only culprit is the above line of code. How to speed up the above function evaluation?}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Details&lt;/strong&gt;: Im working on global optimization of nonlinear and nonconvex functions by supplying different start points for the decision variables through a For Loop (only have the optimization toolbox in matlab and dont have the global optimization toolbox so simulating multistart functionality).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function I have is quite complex, based on a bunch of matrix computations (transpose, multiplication, inverse and finally determinant) using the gradient of function in equation 11 of &lt;a href=&quot;http://www.amstat.org/publications/jse/v12n3/goos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.amstat.org/publications/jse/v12n3/goos.html&lt;/a&gt; . The objective function (DCritn) is listed in the end of this question for reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here q1 and q2 are parameters and x1/x2 are the decision variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any given value of q1 and q2, like let's say 0.6 and 0.3, I try to find the values of x1/x2 for which DCritn is minimum. In order to do the optimization, I supply different start points for x and use different set of algorithms like interior point, SQP etc to see which one generates best possible local mimima.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms x1 x2 q1 q2;&#xA;q=[q1,q2];&#xA;x=[x1,x2];&#xA;qvalue=[0.6,0.3];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;%bunch of symbolic computations to arrive at symbolic form of DCritn which is the objective function (see end result after signature) &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DCRGrad = jacobian(DCritn,x).';%Gradient of the the objective function&#xA;DCritnF = subs(DCritn,q,qvalue.');%DCritn objective function specific to qvalue&#xA;DCritnGradF = subs(DCritnGrad,q,qvalue.');%Gradient of objective function as per qvalue&#xA;&#xA;xstart=[5,10];% supplying start points for the optimization&#xA;&#xA;%fmincon code to optimize DCritn using different start values of x. Options for using gradient specified (fmincon code skipped here)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a seperate function file &quot;Objfungrad&quot;, provided below, which fmincon can tap in to for calculating function value and gradient for any given value of x&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [FuncVal,gradVal] = Objfungrad(x)&#xA;&#xA;global DCritnF DCritnGradF;&#xA;global x1 x2;&#xA;syms x1 x2;&#xA;&#xA;try&#xA;% Objective function&#xA;    FuncVal = double(subs(DCritnF,[x1,x2],[x(1),x(2)]));&#xA;catch exception&#xA;    FuncVal = NaN;&#xA;end    &#xA;% Gradient of the objective function&#xA;if nargout  &amp;gt; 1&#xA;      gradVal= double(subs(DCritnGradF,[x1,x2],[x(1),x(2)]));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As per profiling tool, more than 95% of optimization time is spent in evaluating lines of code pertaining to FuncVal and gradVal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If instead of double and subs functions used above, If I had hard-coded the complete symbolic expression for FuncVal and gradVal using qvalue as fixed parameters, then optimization gets completed in less than 10% of the time. Unfortunately this hard-coding is not possible, as parameters q can change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So question is, how can I speed up evaluation of a symbolic expression based on double and subs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&#xA;Hari&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DCritn = ((q1 - q2)^4*(q1^2*exp(2*q1*x1)*exp(2*q1*x2)*exp(2*q2*x1)*exp(2*q2*x2) + q2^2*exp(2*q1*x1)*exp(2*q1*x2)*exp(2*q2*x1)*exp(2*q2*x2) - 2*q1*q2*exp(2*q1*x1)*exp(2*q1*x2)*exp(2*q2*x1)*exp(2*q2*x2)))/(q1^2*(q1^2*x1^2*exp(2*q1*x2)*exp(2*q2*x1) + q2^2*x1^2*exp(2*q1*x1)*exp(2*q1*x2) + q1^2*x1^2*exp(2*q2*x1)*exp(2*q2*x2) + q1^2*x2^2*exp(2*q1*x1)*exp(2*q2*x2) + q2^2*x1^2*exp(2*q1*x1)*exp(2*q2*x2) + q2^2*x2^2*exp(2*q1*x1)*exp(2*q1*x2) + q1^2*x2^2*exp(2*q2*x1)*exp(2*q2*x2) + q2^2*x2^2*exp(2*q1*x2)*exp(2*q2*x1) - 2*q1^2*x1^2*exp(q1*x2)*exp(q2*x2)*exp(2*q2*x1) - 2*q1^2*x2^2*exp(q1*x1)*exp(q2*x1)*exp(2*q2*x2) - 2*q2^2*x1^2*exp(q1*x2)*exp(q2*x2)*exp(2*q1*x1) - 2*q2^2*x2^2*exp(q1*x1)*exp(q2*x1)*exp(2*q1*x2) + q1^4*x1^2*x2^2*exp(2*q1*x1)*exp(2*q2*x2) + q1^4*x1^2*x2^2*exp(2*q1*x2)*exp(2*q2*x1) - 2*q2^2*x1*x2*exp(2*q1*x1)*exp(2*q1*x2) - 2*q1^2*x1*x2*exp(2*q2*x1)*exp(2*q2*x2) - 2*q1^3*x1*x2^2*exp(2*q1*x1)*exp(2*q2*x2) - 2*q1^3*x1^2*x2*exp(2*q1*x2)*exp(2*q2*x1) + q1^2*q2^2*x1^2*x2^2*exp(2*q1*x1)*exp(2*q2*x2) + q1^2*q2^2*x1^2*x2^2*exp(2*q1*x2)*exp(2*q2*x1) - 2*q1*q2*x1^2*exp(q1*x1)*exp(q2*x1)*exp(2*q1*x2) - 2*q1*q2*x1^2*exp(q1*x1)*exp(q2*x1)*exp(2*q2*x2) - 2*q1*q2*x2^2*exp(q1*x2)*exp(q2*x2)*exp(2*q1*x1) - 2*q1*q2*x2^2*exp(q1*x2)*exp(q2*x2)*exp(2*q2*x1) + 2*q1^2*x1*x2*exp(q1*x1)*exp(q2*x1)*exp(2*q2*x2) + 2*q2^2*x1*x2*exp(q1*x1)*exp(q2*x1)*exp(2*q1*x2) + 2*q1^2*x1*x2*exp(q1*x2)*exp(q2*x2)*exp(2*q2*x1) + 2*q2^2*x1*x2*exp(q1*x2)*exp(q2*x2)*exp(2*q1*x1) - 2*q1*q2^2*x1*x2^2*exp(2*q1*x2)*exp(2*q2*x1) - 2*q1*q2^2*x1^2*x2*exp(2*q1*x1)*exp(2*q2*x2) + 2*q1^2*q2*x1*x2^2*exp(2*q1*x1)*exp(2*q2*x2) + 2*q1^2*q2*x1*x2^2*exp(2*q1*x2)*exp(2*q2*x1) + 2*q1^2*q2*x1^2*x2*exp(2*q1*x1)*exp(2*q2*x2) + 2*q1^2*q2*x1^2*x2*exp(2*q1*x2)*exp(2*q2*x1) + 2*q1^3*x1*x2^2*exp(q1*x1)*exp(q2*x1)*exp(2*q2*x2) - 2*q1^3*x1^2*x2*exp(q1*x1)*exp(q2*x1)*exp(2*q2*x2) - 2*q1^3*x1*x2^2*exp(q1*x2)*exp(q2*x2)*exp(2*q2*x1) + 2*q1^3*x1^2*x2*exp(q1*x2)*exp(q2*x2)*exp(2*q2*x1) - 2*q1*q2*x1*x2*exp(2*q1*x1)*exp(2*q2*x2) - 2*q1*q2*x1*x2*exp(2*q1*x2)*exp(2*q2*x1) - 2*q1^3*q2*x1^2*x2^2*exp(2*q1*x1)*exp(2*q2*x2) - 2*q1^3*q2*x1^2*x2^2*exp(2*q1*x2)*exp(2*q2*x1) + 2*q1^3*x1*x2^2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) + 2*q1^3*x1^2*x2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) + 2*q1*q2*x1*x2*exp(q1*x1)*exp(q2*x1)*exp(2*q1*x2) + 2*q1*q2*x1*x2*exp(q1*x1)*exp(q2*x1)*exp(2*q2*x2) + 2*q1*q2*x1*x2*exp(q1*x2)*exp(q2*x2)*exp(2*q1*x1) + 2*q1*q2*x1*x2*exp(q1*x2)*exp(q2*x2)*exp(2*q2*x1) - 2*q1^4*x1^2*x2^2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) + 4*q1*q2*x1^2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) + 4*q1*q2*x2^2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) - 2*q1^2*x1*x2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) - 2*q2^2*x1*x2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) + 2*q1*q2^2*x1*x2^2*exp(q1*x1)*exp(q2*x1)*exp(2*q1*x2) - 2*q1*q2^2*x1^2*x2*exp(q1*x1)*exp(q2*x1)*exp(2*q1*x2) - 2*q1^2*q2*x1*x2^2*exp(q1*x1)*exp(q2*x1)*exp(2*q1*x2) + 2*q1^2*q2*x1^2*x2*exp(q1*x1)*exp(q2*x1)*exp(2*q1*x2) - 2*q1*q2^2*x1*x2^2*exp(q1*x2)*exp(q2*x2)*exp(2*q1*x1) + 2*q1*q2^2*x1^2*x2*exp(q1*x2)*exp(q2*x2)*exp(2*q1*x1) - 2*q1^2*q2*x1*x2^2*exp(q1*x1)*exp(q2*x1)*exp(2*q2*x2) + 2*q1^2*q2*x1*x2^2*exp(q1*x2)*exp(q2*x2)*exp(2*q1*x1) + 2*q1^2*q2*x1^2*x2*exp(q1*x1)*exp(q2*x1)*exp(2*q2*x2) - 2*q1^2*q2*x1^2*x2*exp(q1*x2)*exp(q2*x2)*exp(2*q1*x1) + 2*q1^2*q2*x1*x2^2*exp(q1*x2)*exp(q2*x2)*exp(2*q2*x1) - 2*q1^2*q2*x1^2*x2*exp(q1*x2)*exp(q2*x2)*exp(2*q2*x1) + 4*q1^3*q2*x1^2*x2^2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) - 2*q1^2*q2^2*x1^2*x2^2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) + 2*q1*q2^2*x1*x2^2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) + 2*q1*q2^2*x1^2*x2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) - 4*q1^2*q2*x1*x2^2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) - 4*q1^2*q2*x1^2*x2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2) - 4*q1*q2*x1*x2*exp(q1*x1)*exp(q1*x2)*exp(q2*x1)*exp(q2*x2)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastEditorUserId="1804" LastEditDate="2014-06-23T13:31:24.160" LastActivityDate="2014-06-26T00:30:55.087" Title="Looking for ways to speed up the numeric evaluation of a symbolic expression in Matlab" Tags="|optimization|matlab|numerics|symbolic-computation|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="12960" PostTypeId="1" AcceptedAnswerId="12962" CreationDate="2014-06-23T03:59:28.557" Score="6" ViewCount="1695" Body="&lt;p&gt;I am looking at the Nvidia GT-860M which comes in both the old Kepler and new Maxwell architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The old one (1152) seems to have almost twice the cores as the new one (640). The new one has a compute capability score of 5.0 compared to the old one of 3.0. Yet, the new cores are more powerful, according to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://devblogs.nvidia.com/parallelforall/5-things-you-should-know-about-new-maxwell-gpu-architecture/&quot; rel=&quot;noreferrer&quot;&gt;https://devblogs.nvidia.com/parallelforall/5-things-you-should-know-about-new-maxwell-gpu-architecture/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is better for computation? E.g. fitting random forest models?&lt;/p&gt;&#xA;" OwnerUserId="9517" LastActivityDate="2017-04-29T21:37:58.793" Title="Maxwell vs Kepler for GPU computation" Tags="|parallel-computing|gpu|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12961" PostTypeId="2" ParentId="12924" CreationDate="2014-06-23T06:09:42.427" Score="1" Body="&lt;p&gt;I think you are looking for this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Adventures in PyOpenCL: (&lt;a href=&quot;http://enja.org/category/tutorial/advcl/&quot; rel=&quot;nofollow&quot;&gt;http://enja.org/category/tutorial/advcl/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&quot;nbody simulation using pyopencl and pyopengl&quot;: (&lt;a href=&quot;http://www.firstpost.com/topic/person/nvidia-nbody-simulation-using-pyopencl-and-pyopengl-video-xeOUxTSjowY-7106-7.html&quot; rel=&quot;nofollow&quot;&gt;http://www.firstpost.com/topic/person/nvidia-nbody-simulation-using-pyopencl-and-pyopengl-video-xeOUxTSjowY-7106-7.html&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want to program it in python, You might need:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;PyOpenCL (&lt;a href=&quot;http://mathema.tician.de/software/pyopencl/&quot; rel=&quot;nofollow&quot;&gt;http://mathema.tician.de/software/pyopencl/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;PyOpenGL (&lt;a href=&quot;http://pyopengl.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;http://pyopengl.sourceforge.net/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9499" LastEditorUserId="9499" LastEditDate="2014-06-23T06:29:17.680" LastActivityDate="2014-06-23T06:29:17.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12962" PostTypeId="2" ParentId="12960" CreationDate="2014-06-23T07:51:29.063" Score="4" Body="&lt;p&gt;While the Kepler card has more cores, the Maxwell ones run at a higher clock speed.&#xA;&lt;a href=&quot;http://www.notebookcheck.net/NVIDIA-GeForce-GTX-860M.107794.0.html&quot; rel=&quot;nofollow&quot;&gt;This review&lt;/a&gt; actually compares the two, although mainly for gaming purposes, and the Maxwell has the Kepler beat in almost all cases. The margins are relatively small however.&#xA;Looking at the numbers, I would hazard the guess that the Maxwell card will perform better for your purposes, particularly if you make sure to compile your GPU code for that compute model (5.0). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that desktop GPUs are always quite a bit faster than their laptop equivalents (i.e. 780M is a lot slower than the 780), so if a desktop computer works for you that will give you better performance.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-06-23T07:51:29.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12963" PostTypeId="2" ParentId="12958" CreationDate="2014-06-23T11:48:27.810" Score="1" Body="&lt;p&gt;You might be interested in &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler_characteristic&quot; rel=&quot;nofollow&quot;&gt;Euler's characteristic&lt;/a&gt;, which relates the number of vertices, edges and faces in a planar graph, that you can also interpret as a 2D mesh. There is a similar relation for 3D meshes (see for example &lt;a href=&quot;http://perso.uclouvain.be/vincent.legat/teaching/documents/meca2170-jfr-cours3.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (section about tetrahedral meshes) or &lt;a href=&quot;http://www.cs.mtu.edu/~shene/COURSES/cs3621/SLIDES/Mesh.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. The problem is that in 3 dimensions, this (single) formula relates the number of vertices, edges, faces and elements in your mesh. Most mesh generators give you immediately number of elements and vertices, but not the number of faces and edges, so you are left with one equation and two unknowns. I'm afraid there's no easier way than counting the edges or faces.&lt;/p&gt;&#xA;" OwnerUserId="4688" LastEditorUserId="4688" LastEditDate="2014-06-23T12:04:35.883" LastActivityDate="2014-06-23T12:04:35.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12965" PostTypeId="1" CreationDate="2014-06-23T13:24:45.460" Score="2" ViewCount="799" Body="&lt;p&gt;I'm trying to calculate the derivative of a function using FFT in Matlab. I've coded the following function to calculate it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dfdx = derivative(x,psi_0,n)&#xA;&#xA; Nx = max(size(psi_0));&#xA; k = 2*pi/(range(x))*[0:Nx/2-1 0 -Nx/2+1:-1];&#xA; dfdx = ifft(((1i*k).^n).*fft(psi_0));&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For higher derivatives (let's say for the $20$th derivative) it doesn't work. More precisely, the plot of the $20$th derivative doesn't make sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea what causes this, and how this can be corrected?&lt;/p&gt;&#xA;" OwnerUserId="9519" LastEditorDisplayName="user10435" LastEditDate="2015-04-08T02:38:34.417" LastActivityDate="2015-04-08T02:38:34.417" Title="Calculating high-order derivatives using FFT in Matlab" Tags="|matlab|fourier-transform|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12966" PostTypeId="2" ParentId="12951" CreationDate="2014-06-23T13:56:55.803" Score="1" Body="&lt;p&gt;Something in your description sounds amiss when you say you need four codes. You may want clarification from your instructor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would one MPI+OpenMP code that produces correct results for any &quot;OMP_NUM_THREADS=n mpiexec -np m&quot; invocation satisfy your assignment? That would permit serial, parallel, multithreaded, and hybrid operation from a single binary at runtime. &lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2014-06-23T13:56:55.803" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="12967" PostTypeId="1" CreationDate="2014-06-23T16:39:52.230" Score="5" ViewCount="9779" Body="&lt;p&gt;I'm using the SciPy &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;sparse.csr_matrix&lt;/code&gt;&lt;/a&gt; format for a finite element code. In applying the essential boundary conditions, I'm setting the desired value in the right hand side vector, and setting the corresponding row and column to the identity (i.e. 1 on the diagonal and zeros elsewhere.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code I'm using is similar to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K = csr_matrix((data,(rows,cols)))&#xA;for idx in boundary:&#xA;    K[:,idx] = 0.0&#xA;    K[idx,:] = 0.0&#xA;    K[idx,idx] = 1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code results in inserting values for all of the entries in the idx row and column which is extremely slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to be able to set the diagonal entry to 1 (which shouldn't alter the sparsity), and only zero the entries in the &lt;code&gt;idx&lt;/code&gt; row and column which aren't already zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this?&lt;/p&gt;&#xA;" OwnerUserId="553" LastEditorUserId="9" LastEditDate="2014-06-24T12:08:11.257" LastActivityDate="2017-11-15T17:27:56.167" Title="scipy.sparse: Set row/column in sparse matrix to the identity without changing sparsity" Tags="|finite-element|python|scipy|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="12968" PostTypeId="2" ParentId="12965" CreationDate="2014-06-23T16:52:11.300" Score="5" Body="&lt;p&gt;Let's suppose you have a function $f$ in $L^{2}$, and for simplicity, let's suppose it's a single-variable function defined over the whole real line. If its Fourier transform $\mathcal{F}(f)$ is $\hat{f}$, then $\mathcal{F}(f')(\xi) = 2\pi i \xi\hat{f}(\xi)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're calculating the $n$th derivative, iterating the process above yields $\mathcal{F}(f^{(n)})(\xi) = (2 \pi i \xi)^{n}\hat{f}(\xi)$, suggesting that the calculation of higher derivatives becomes ill-conditioned. Assuming $\hat{f}$ is a well-conditioned function at $\xi$, the large coefficient in the derivative expression implies that small perturbations in $\xi$ perturb values of the Fourier transform of the $n$th derivative significantly for large $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if $n = 20$, the magnitude of the constant pre-factor will be $(2\pi)^{20}$, or roughly $10^{16}$. With such a large constant factor, it would be surprising if you obtained meaningful results in floating-point arithmetic. You could compare this approach to calculating derivatives numerically through finite-difference approximations; calculating a 20th derivative with this approach would yield garbage in most cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you used arbitrary precision arithmetic, you would probably see more accurate results. As $n$ increases, you'll require more digits of precision in your input to obtain an output accurate to at least 8 significant figures (this cutoff is arbitrary, but presumably, you want many accurate significant figures). &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-06-23T16:52:11.300" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="12971" PostTypeId="1" AcceptedAnswerId="12993" CreationDate="2014-06-23T23:02:23.077" Score="1" ViewCount="397" Body="&lt;p&gt;I would like to ask how is the evaluation of integrals over inter-element interfaces implemented in a typical DG code. I can think of two basic approaches (I assume 2D mesh here): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Perform 1D interpolation of the interface data seen from 'left' and 'right' of each edge and numerically integrate. For this, I don't need to evaluate the element shape functions in the elements adjacent to the edge. This will probably work fine for Lagrange shape functions (element interpolant restricted to edge and 1D edge interpolant should be identical).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Interpret the 1D edge quadrature rule as a quadrature with points placed on the boundary of each element and use the element shape functions to compute edge integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 1) does not require extra storage for jump/average terms: evaluation on each edge is performed on the fly and I can directly add boundary integrals to the weak form. Option 2) will require a lot of extra memory allocation for interface integrals: jump and average terms are only complete after I visit both left and right neighbour of each edge and temporary data evaluated at quadrature points have to wait somewhere before the interface integrals are assembled. I could also loop over all edges and obtain directly the left-and right interpolant from the element shape functions, but that would mean that I re-evaluate the element shape functions every time I visit one of its edges (computational overhead). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the disadvantage of 1) is that it won't be possible to evaluate shape function gradients on element interfaces for higher-order equations. In addition, it would probably not work with some modal bases (all modes from the element interior are needed in the expansion). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot decide which approach is better (or if they are good at all). Maybe there's a more clever way I did not think of. Can you share your opinion?&lt;/p&gt;&#xA;" OwnerUserId="4688" LastActivityDate="2014-06-26T10:17:15.310" Title="Evaluation of interface terms in Discontinuous Galerkin method" Tags="|discontinuous-galerkin|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12972" PostTypeId="2" ParentId="12959" CreationDate="2014-06-24T00:14:52.550" Score="2" Body="&lt;p&gt;Running your expression through Mathematica's FullSimplify produced this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(exp(2*(q1 + q2)*(x1 + x2))*(q1 - q2)^6)/(q1^2*(exp(q2*(x1 + x2))*q1*(x1 - x2) + &#xA; exp(q1*(x1 + x2))*q2*(x1 - x2) + &#xA; exp(q1*x1 + q2*x2)*(-q2*x1 + q1*x2 + q1*(-q1 + q2)*x1*x2) + &#xA; exp(q2*x1 + q1*x2)*(q2*x2 + q1*x1 (-1 + q1*x2 - q2*x2)))^2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is much shorter at least.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-06-24T00:14:52.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12973" PostTypeId="2" ParentId="12967" CreationDate="2014-06-24T01:41:11.610" Score="1" Body="&lt;p&gt;You can use the method &lt;code&gt;nonzero&lt;/code&gt; to return an array of all the non-zero entries of a given row or column, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in boundary:&#xA;    _, J = A[i,:].nonzero()   # ignored the first return value, which is just [i,...,i]&#xA;    for j in J:&#xA;        if j!=i:&#xA;            A[i,j] = 0.0&#xA;    A[i,i] = 1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There's certainly a clever way of getting rid of the conditional, but you get the idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I think you only want to set the row corresponding to an index &lt;code&gt;i&lt;/code&gt; in the boundary to be the identity, not the column. If the nodes are re-ordered so that the boundary nodes are at the beginning, your matrix should look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A = \left[\begin{matrix}I &amp;amp; 0 \\ B &amp;amp; K\end{matrix}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K$ represents the coupling between internal nodes and $B$ the influence of the boundary conditions on the internal nodes.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-06-24T01:41:11.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12974" PostTypeId="2" ParentId="12959" CreationDate="2014-06-24T01:53:09.890" Score="2" Body="&lt;p&gt;If the expression is as &quot;simple&quot; (I realize that's a relative term) as VictorLiu shows, then you might try hand-coding it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If hand-coding an equivalent MATLAB expression is unwieldy (for instance, if you have other, more complicated expressions to work with), you could try using MATLAB's code generation facilities. MATLAB can convert MuPAD symbolic expressions into MATLAB, Fortran, or C code. If speed is truly critical, converting the symbolic expression into C code and then writing a MEX wrapper around the C code to call it from MATLAB might be a viable option.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-06-24T01:53:09.890" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12975" PostTypeId="1" CreationDate="2014-06-24T03:36:39.423" Score="1" ViewCount="150" Body="&lt;p&gt;I have a transient simulation for a case from 0 to 3 sec , actually i interest the solutions on time range from 2 to 3 sec , as my velocity and mesh size , my time step should be 0.0000000001 sec , my question is can I make my time step is 0.0001 sec for time range 0 to 1.9 and the i use 0.0000000001 for time range from 1.9 to 4 sec ( the range i interested ) ? BTW I use ANSYS FLUENT as a commercial CFD software.      &lt;/p&gt;&#xA;" OwnerUserId="7314" LastActivityDate="2014-06-24T09:03:00.987" Title="Importance of change the Time Step Value for the Accuracy of a Transient CFD Simulation" Tags="|fluid-dynamics|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12977" PostTypeId="2" ParentId="12959" CreationDate="2014-06-24T05:18:33.673" Score="3" Body="&lt;p&gt;Have you tried &lt;a href=&quot;http://www.mathworks.com/help/symbolic/matlabfunction.html&quot; rel=&quot;nofollow&quot;&gt;matlabFunction&lt;/a&gt;?  For example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms x y real&#xA;f = matlabFunction(x^2 + y^2, 'vars', {[x, y]})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will create a (non-symbolic) function &lt;code&gt;f&lt;/code&gt; that takes a two-component vector as input (rather than two different input arguments).  The function is vectorized automatically, which potentially allows MATLAB to optimize its evaluation.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2014-06-24T12:57:23.290" LastActivityDate="2014-06-24T12:57:23.290" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="12978" PostTypeId="2" ParentId="12975" CreationDate="2014-06-24T09:03:00.987" Score="1" Body="&lt;p&gt;Certainly not. Since the approximation error is simply accumulating during the time integration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can never compensate the errors you made earlier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, in terms of accuracy, this is not a good idea.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-06-24T09:03:00.987" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="12979" PostTypeId="1" AcceptedAnswerId="12984" CreationDate="2014-06-24T09:31:08.967" Score="12" ViewCount="8744" Body="&lt;p&gt;I have a list ${\cal L}$ of symmetric matrices that I need to check for positive semi-definiteness (i.e their eigenvalues are non-negative.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comment above implies that one could do it by computing the respective eigenvalues and checking if they are non-negative (perhaps having to take care of rounding errors.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing the eigenvalues is quite expensive in my scenario but I have noticed that the library that I am using has quite a fast test for positive definiteness (that is, if the eigenvalues of a matrix are strictly positive.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence the idea would be, that given a matrix $B \in {\cal L}$, one tests if $B + \epsilon I$ is positive definite. If it is not then $B$ is not positive semi-definite, otherwise one can compute the eigenvalues of $B$ to make sure it is indeed positive semi-definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question now is: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a more direct and efficient way of testing whether a matrix&#xA;  is positive semi-definite, provided that an efficient test for&#xA;  positive definiteness is given?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4582" LastActivityDate="2020-04-13T17:07:30.767" Title="Testing if a matrix is positive semi-definite" Tags="|linear-algebra|eigenvalues|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="12980" PostTypeId="2" ParentId="12929" CreationDate="2014-06-24T09:49:37.560" Score="3" Body="&lt;p&gt;Ok, so Kalman filters are not my forte, but non-linear optimization is, so let's give it a go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You want to obtain $\psi$, not by solving your equation (3), but by maximizing (1), so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\psi^+ = arg\max_\psi (\log L(Y_n | \psi))$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your stepsize $s$ is in theory found by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s^+ = arg\max_s (\log L(Y_n | \psi+s\pi))$, however in practice you don't need to find the exact minimum, it simply needs to fulfil the Wolfe conditions, as described in Nocedal &amp;amp; Wright.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BFGS algorithms have the advantage that the step size is typically around 1, and the simple &lt;a href=&quot;http://en.wikipedia.org/wiki/Backtracking_line_search&quot; rel=&quot;nofollow&quot;&gt;Armijo linesearch&lt;/a&gt; works quite well. You will probably get faster convergence with the zoom linesearch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cas.umt.edu/math/reports/bardsley/bfgs_vkf.pdf&quot; rel=&quot;nofollow&quot;&gt;This article&lt;/a&gt; has a nice description of a Kalman filter using the LBFGS algorithm. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="276" LastEditDate="2014-06-24T21:26:01.750" LastActivityDate="2014-06-24T21:26:01.750" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="12981" PostTypeId="1" AcceptedAnswerId="12982" CreationDate="2014-06-21T09:09:51.257" Score="0" ViewCount="649" Body="&lt;p&gt;I'm part of a Computational Science course and come from a non-programming background, so please forgive me my ignorance. I'm working on a set of code in C to numerically solve the Navier Stokes equations for a Computational Fluid Dynamics course. Anyway, as a part of our task, I have to write code that runs on a single processor (serially) as well as provide the choice to parallelize that same code with OpenMP and/or MPI (to utilize multiple processors, depending on user input at the beginning of run time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping for advice from the community on doing this and I have a few questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should I be thinking of an if/else structure for my program? (The program would need to branch towards serial code, or MPI enabled code, or OpenMP code, or a hybrid of MPI+OpenMP, and this would mean that much of the code would be duplicated).&lt;/li&gt;&#xA;&lt;li&gt;Should I constructs the actions that would be duplicated to be separate functions and call these functions during run time where they are needed? (But from what I understand function calls would increase the overhead of my code in general, regardless of what choice is picked)&lt;/li&gt;&#xA;&lt;li&gt;Would there be a way to hide the pre-processor directives, or MPI-related functions (e.g. MPI_Init()) so that they would conveniently appear or disappear depending on the choice the user made at the beginning or run time?&lt;/li&gt;&#xA;&lt;li&gt;Finally, the purpose of this task is to measure the performance benefits of using serial code, vs. OpenMP, vs. MPI, vs. MPI+OpenMP. Does it make sense that I am trying to avoid code duplication since combining the code (to increase readability) would unnecessarily add overhead to my serial code? Such as MPI-related function calls, potentially many if/else conditions, which would skew the performance measurements of my serial code, versus an if/else structure at the beginning of the program which directs it down a completely separate path (as in Q1).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I hope I've asked a clear question and have abided by the community's standards.&#xA;Thank you kindly.&lt;/p&gt;&#xA;" OwnerUserId="9506" OwnerDisplayName="kfkhalili" LastActivityDate="2014-06-24T12:30:59.433" Title="C - OpenMP, MPI, Serial Program" Tags="|c|mpi|" AnswerCount="1" CommentCount="0" ClosedDate="2014-06-24T14:46:23.363" ContentLicense="CC BY-SA 3.0" />
  <row Id="12982" PostTypeId="2" ParentId="12981" CreationDate="2014-06-21T14:03:11.763" Score="1" Body="&lt;p&gt;I am wondering why you want to combine all the methods(serial, MPI, OpenMP) into one C program. I think a better way is to separate them into 3 programs, and call them by a script(like shell). Anyway, they are different, considering the code and compiling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your first question, if you insist on one C program, of course you have to figure out a way to combine them together. Maybe if/else is not enough, as I know, MPI program is called by &quot;mpirun&quot; command. It seems to be a little hard using it in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your second question, I suggest you convert your serial program into OPenMP and MPI first, and then think about avoiding duplicates. There is a great difference between serial and parallel program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your third question, MPI_init is a part of MPI program, as I know, it is necessary because they will do some pre-processing jobs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your last question, combining them together would bring some side effects unavoidably since you may use &quot;mpicc&quot; to compile serial and OpenMP program. I think it can be ignored because it's a tiny part for your entire program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, I suggest you separate them and use a script.&lt;/p&gt;&#xA;" OwnerDisplayName="user3761116" LastActivityDate="2014-06-21T14:03:11.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12983" PostTypeId="2" ParentId="12946" CreationDate="2014-06-24T14:47:32.760" Score="3" Body="&lt;p&gt;Since expokit seems to use a Krylov subspace method, usually (at least, the hope is that) the upper Hessenberg matrices are of small dimension, say $m \sim 100$. For matrices of these sizes, there should not be any major difference in computational time by using any method for dense matrix exponential computation. For example, 'expm' in MATLAB seems to use scaling and squaring method with a Pade approximation near zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the dimension of the Krylov subspace is large, then you might consider either preconditioning &lt;a href=&quot;http://link.springer.com/article/10.1023%2FA%3A1023219016301&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/article/10.1023%2FA%3A1023219016301&lt;/a&gt; or restarting the Krylov subspace method &lt;a href=&quot;http://www.mathe.tu-freiberg.de/~Ernst/PubArchive/eiermannErnstKrylovExp.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mathe.tu-freiberg.de/~Ernst/PubArchive/eiermannErnstKrylovExp.pdf&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="4558" LastActivityDate="2014-06-24T14:47:32.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12984" PostTypeId="2" ParentId="12979" CreationDate="2014-06-24T15:48:09.690" Score="23" Body="&lt;p&gt;What's your working definition of &quot;positive semidefinite&quot; or &quot;positive definite&quot;?  In floating point arithmetic, you'll have to specify some kind of tolerance for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could define this in terms of the computed eigenvalues of the matrix.  However, you should first notice that the computed eigenvalues of a matrix scale linearly with the matrix, so that for example, the matrix I get by multiplying $A$ by a factor of one million has its eigenvalues multiplied by a million.  Is $\lambda=-1.0$ a negative eigenvalue?  If all of the other eigenvalues of your matrix are positive and on the order of $10^{30}$, then $\lambda=-1.0$ is effectively 0 and shouldn't be treated as a negative eigenvalue.  Thus it's important to take scaling into account.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reasonable approach is to compute the eigenvalues of your matrix, and declare that the matrix is numerically positive semidefinite if all eigenvalues are larger than $-\epsilon \left| \lambda_{\max} \right|$, where $  \lambda_{\max}$ is the largest eigenvalue.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, computing all of the eigenvalues of a matrix is rather time consuming.  Another commonly used approach is that a symmetric matrix is considered to be positive definite if the matrix has a Cholesky factorization in floating point arithmetic.  Computing the Cholesky factorization is an order of magnitude faster than computing the eigenvalues.  You can extend this to positive semidefiniteness by adding a small multiple of the identity to the matrix.  Again, there are scaling issues.  One fast approach is to do a symmetric scaling of the matrix so that the diagonal elements are 1.0  and add $\epsilon$ to the diagonal before computing the Cholesky factorization.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should be careful with this though, because there are some problems with the approach.  For example, there are circumstances where the $A$ and $B$ are postive definite in the sense that they have floating point Cholesky factorizations, but $(A+B)/2$ does not have a Cholesky factorization.  Thus the set of &quot;floating point Cholesky factorizable positive definite matrices&quot; isn't convex!  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2018-05-06T15:33:55.687" LastActivityDate="2018-05-06T15:33:55.687" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12986" PostTypeId="1" CreationDate="2014-06-25T15:21:30.613" Score="2" ViewCount="269" Body="&lt;p&gt;I have numerically integrated the (reduced) homogeneous equation of a damped harmonic oscillator in order to see how the error propagates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2 X}{d\phi^2} + \frac{1}{Q}\frac{dX}{d\phi}+X(\phi) = 0$$&#xA;where $\phi = 2 \pi \tau$ and $\tau$ is the reduced time&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the process I have also extracted the logarithm of the energy and plotted it against time, for different values of the quality factor, Q. One can expect the energy to decay slower for larger Q.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ E = P^2 + X^2$$ where $P = \dot{X}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I found is that for very large values of Q, the energy of the oscillator begins to &lt;strong&gt;increase&lt;/strong&gt;. I checked my code several times and I couldn't see anything wrong with it so I suspected that it purely something to do with numbers so I began playing around with them, but first here is the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/9LTVF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I found is odd; say I'm using a step size of $1 \times 10^{-n}$ . If Q's magnitude matches or exceeds $10^{n}$ the energy begins to increase!&#xA;What is causing this? &lt;strong&gt;What is the connection between Q and the step size?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Also, should the energy be oscillating?&lt;/strong&gt; (If you zoom in on the red and green lines on the graph you will see that it is oscillating much slower there.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is how I am integrating the equation for those interested: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Let $\dot{X} = P$ &lt;/li&gt;&#xA;&lt;li&gt;Thus original equation becomes $\dot{P} + \frac{1}{Q} P + X = 0$ &lt;/li&gt;&#xA;&lt;li&gt;Now we have two 1st order DEs. &lt;/li&gt;&#xA;&lt;li&gt;Using discrete steps we can roughly find the next point of $X$ and $P$ by considering that $\dot{X} = \frac{dX}{d\phi} = \frac{X_{n+1} - X_n}{\Delta \phi}$ and approaching the other eqution in the same manner. &lt;/li&gt;&#xA;&lt;li&gt;We can then rearrange for $X_{n+1}$ and $P_{n+1}$ and create a code that calculates those points, given some basic initial conditions. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6744" LastActivityDate="2014-06-25T15:33:04.363" Title="Energy of damped harmonic oscillator begins increasing with very large Q" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12987" PostTypeId="2" ParentId="12986" CreationDate="2014-06-25T15:33:04.363" Score="4" Body="&lt;p&gt;You are basically using &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler_method&quot; rel=&quot;nofollow&quot;&gt;Forward Euler&lt;/a&gt; timestepping, which is not stable. It is much better to use &lt;a href=&quot;http://en.wikipedia.org/wiki/Backward_Euler_method&quot; rel=&quot;nofollow&quot;&gt;Backward Euler&lt;/a&gt;. Even better, if you care about the total energy of the system being conserved, you need to use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Symplectic_integrator&quot; rel=&quot;nofollow&quot;&gt;symplectic integration&lt;/a&gt; method.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-06-25T15:33:04.363" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="12988" PostTypeId="1" AcceptedAnswerId="12998" CreationDate="2014-06-25T22:11:59.027" Score="1" ViewCount="4274" Body="&lt;p&gt;I am developing some parallel CFD code using Fortran 90 and OpenMPI. The processes are structured into two groups such that half perform a certain type of calculations (let's say Velocity, Pressure and Temperature) while the other half performs a different type of calculations (let's say Turbulence modelling using K-omega). As a result the program splits itself arbitrarily into two communicators, let's call them Comm1 and Comm2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to use Process of local rank 0 in Comm 1 to read some parameters from Namelist files and Broadcast them to all processes in Comm1. For some reason, even though it is just a scalar, I get MPI_ERR_TRUNCATE: message truncated as an error message. I'm wondering if anybody's experienced anything similar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code is below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;module myModule&#xA;use mpi&#xA;double precision, public :: KOMEGA_CMU, KOMEGA_ELOG, KOMEGA_CTRANS, TED_INITIAL, UD_INITIAL&#xA;! Other Variable declarations&#xA;! ...&#xA;contains&#xA;subroutine readParameters(MPI_COMMUNICATOR, PROC_RANK_LOCAL, NUM_PROCESSES)&#xA;implicit none&#xA;integer, intent(in) :: MPI_COMMUNICATOR, PROC_RANK_LOCAL, NUM_PROCESSES&#xA;integer :: ierr&#xA;    if(PROC_RANK_LOCAL == 0) then&#xA;    ! Read variables &#xA;    end if&#xA;    call MPI_Bcast(UD_INITIAL,1, MPI_DOUBLE_PRECISION,0, MPI_COMMUNICATOR, ierr)&#xA;end subroutine ReadParameters    &#xA;end module MyModule&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this for some reason results in MPI_ERR_TRUNCATE in all processes but the root process (Rank 0), even though the variable is declared for double precision and message buffer is for double precision as well. This happens with integers too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help.&lt;/p&gt;&#xA;" OwnerUserId="9542" LastActivityDate="2014-08-25T16:53:08.433" Title="MPI_ERR_TRUNCATE appears for no understandable reason" Tags="|fluid-dynamics|fortran|mpi|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="12992" PostTypeId="1" CreationDate="2014-06-26T09:34:53.170" Score="0" ViewCount="101" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gCe6a.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hi guys! I am working on a dynamics problem that I am not really sure how to solve it. Can anyone help me? The professor gave as a hint that we should compute the stiffness matrix of a linear element and then the stiffness matrix of a quadratic element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;THanks!&lt;/p&gt;&#xA;" OwnerUserId="7825" LastEditorUserId="549" LastEditDate="2014-06-26T14:18:53.780" LastActivityDate="2014-06-26T14:18:53.780" Title="Dynamic problem Finite Element" Tags="|finite-element|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12993" PostTypeId="2" ParentId="12971" CreationDate="2014-06-26T10:17:15.310" Score="1" Body="&lt;p&gt;In deal.II (see &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/&lt;/a&gt;), we use your option 2. The reason is that you need to evaluate not just the shape functions that are defined along the interface, but you also need to evaluate all of the other shape functions defined on that cell (even if their interpolation point -- in the case of Lagrange elements -- is not on the given interface: while they will be zero along the interface, their gradient is not).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see how this is implemented in the step-12 tutorial program of deal.II: &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_12.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_12.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-06-26T10:17:15.310" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12995" PostTypeId="2" ParentId="12992" CreationDate="2014-06-26T11:37:51.197" Score="1" Body="&lt;p&gt;To add onto Christian Clason's hint; as you'd do in a fundamental vibrations course, if you have a mass and stiffness matrix, you should be able to express and solve the eigenvalue problem for the system's natural frequencies.&lt;/p&gt;&#xA;" OwnerUserId="9542" LastActivityDate="2014-06-26T11:37:51.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12997" PostTypeId="2" ParentId="8617" CreationDate="2014-06-26T14:42:23.860" Score="1" Body="&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; &lt;em&gt;I'm about to make statements based on my own experience, so it may not necessarily be true for your situation&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, gFortran is a strict compiler when it comes to fortran 77. If the code is written according to the F77 language standard, then everything should work out. This doesn't seem to be the case very often in code obtained from textbooks and other sources, especially in my field (mechanical engineering, fluid structure interaction, dynamics, FEM, CFD). There were many situations where code obtained verbatim from these sources would either not compile, or result in confusing error messages at runtime. Switching to PGFortran or Ifort always seemed to solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has happened to me with code from:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ferziger, Joel H., and Milovan Perić. Computational methods for fluid dynamics. Vol. 3. Berlin: Springer, 2002.&lt;/li&gt;&#xA;&lt;li&gt;Cebeci, Tuncer. Analysis of turbulent boundary layers. Elsevier, 2012.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To summarize, if for some reason gFortran doesn't work out, don't hesitate to try other fortran compilers as well. Alternatively, you can try the various flags to make gFortran a bit less strict.&lt;/p&gt;&#xA;" OwnerUserId="9542" LastActivityDate="2014-06-26T14:42:23.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12998" PostTypeId="2" ParentId="12988" CreationDate="2014-06-26T16:21:33.873" Score="1" Body="&lt;p&gt;It turns out the problem came from a previously called subroutine by the subgroup of processes whose last MPI statement accidentally broadcast an $M \times N$ ($ M\neq 0, N\neq 0$) array with zero count on the message envelope. It looks like the buffer overflew as a result to subsequent MPI collective operations within the subgroup communicator. As soon as I caught that mistake everything seemed to work fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="9542" LastActivityDate="2014-06-26T16:21:33.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="12999" PostTypeId="1" CreationDate="2014-06-26T21:52:17.363" Score="1" ViewCount="255" Body="&lt;p&gt;I have a coupled PDE problem(Poisson-Schrondinger system), i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;first I need to solve an eigenvalue problem (Schrodinger problem discretized by Galerkin method)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax=\lambda x, ~~~A=A(u)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the output $x$ is then used to compute some source term(charge) of a poisson equation(discretized on the same grid)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Ku=b, ~~~b=b(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problems is: I would like to solve this problem in a neumann sense, which means I don't want to enforce x to be 0 at the boundary for the eigenvalue problem although it might be quite small. So I prefer Neumann B.C. for both PDE, then it seems to me that even if the matrix is singular $Ax=\lambda x$ works fine while $Ku=b$ cannot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My solution is to set Neumann B.C. for $Ax=\lambda x$ and Dirichlet B.C. for the poisson problem. the result looks fine, and my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Ku=b won't work when K is singular because the solution is undetermined to a constant, am I right?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why does the eigenvalue problem still works when the A matrix is singular? does it automatically throw away the null space?( I solve the eigenvalue problem by using matlab eigs)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the combination of Neumann and Dirichlet B.C. in this coupled problem still leads to the physical problem posed in a Neumann sense, right? (since x solved from $Ax=\lambda x$ satisfy Neumann and b=b(x) implicitly built this Neumann condition into a poisson problem)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="4532" LastEditDate="2014-06-27T13:15:50.683" LastActivityDate="2014-06-27T13:15:50.683" Title="Coupled PDE: a confusion in boundary condition setup" Tags="|finite-element|finite-difference|computational-physics|coupling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13000" PostTypeId="1" AcceptedAnswerId="13001" CreationDate="2014-06-26T22:07:11.920" Score="10" ViewCount="3404" Body="&lt;p&gt;I understand the crucial importance of freeing memory when certain variables or arrays need to be reused later in the program, or may not be in use for a while. However, in my experience with scientific computing a lot of the code written is meant to be run as a job and produce results, rather than the case of conventional software (a web browser for instance) which is listening for user input not knowing how much memory may be required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if certain arrays and variables are used throughout the entire program is there any point to dedicating a section of the program before exiting that will deallocate the memory used? My understanding is the operating system will take all memory back from a terminated program anyways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking forward to any thoughts and opinions on the subject.&lt;/p&gt;&#xA;" OwnerUserId="9542" LastActivityDate="2014-06-27T01:30:36.733" Title="Fortran 90/95: Deallocating variables" Tags="|numerical-analysis|performance|fortran|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13001" PostTypeId="2" ParentId="13000" CreationDate="2014-06-27T01:30:36.733" Score="12" Body="&lt;p&gt;Fortran's allocatable variables are automatically deallocated when the variable goes out of scope (see &lt;a href=&quot;http://www.fortran90.org/src/best-practices.html#allocatable-arrays&quot;&gt;http://www.fortran90.org/src/best-practices.html#allocatable-arrays&lt;/a&gt;). This means that it is not possible to create a memory leak by failing to deallocate an allocatable array. This is one of the big benefits of using allocatable arrays rather than pointers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, allocating and deallocating memory takes time so it is not advisable to repeatedly allocate and deallocate an array unless there is a good reason to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, as you mentioned, the operating system will re-claim all memory used by the program once the program terminates. This is true regardless of how the memory was allocated. In the case you mentioned (allocatable arrays that are used for the entire life of the program), deallocating before exit may be seen as more of a bookkeeping step than a necessity. It is a form of self-documentation and also makes users of the code aware of the existence of still-allocated arrays. This could be important if the code were to be later modified or incorporated into a larger program.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-06-27T01:30:36.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13002" PostTypeId="1" CreationDate="2014-06-27T11:49:38.147" Score="0" ViewCount="73" Body="&lt;p&gt;I have a system of particles with equal distance with each other and another at random positions which is moving with time. I want to know:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) The method by which I can reduce the number of particles from the first system, as I know the maximum motion of any particle is X.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) How to efficiently calculate the collision between them, I have heard about quadtree and octree but (as per I understood till now) they are for collision of particle from each other.&#xA;In my problem,second system of particles doesn't collide with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: Sorry, if it is a very basic question, I am (very) new to this field.&lt;/p&gt;&#xA;" OwnerUserId="9499" LastActivityDate="2018-12-22T15:03:00.667" Title="Particle Collision to Static paticles" Tags="|optimization|simulation|computational-physics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="13003" PostTypeId="2" ParentId="12999" CreationDate="2014-06-27T12:10:01.213" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;Yes&lt;/li&gt;&#xA;&lt;li&gt;You should see that there is at least one zero (to machine precision) eigenvalue of that system. The eigenvalue problem can be solved because it is prepared to find the zero eigenvalues.&lt;/li&gt;&#xA;&lt;li&gt;The problem with $A$ and $x$ is different from the problem with $K$ and $u$. $x$ will transport whatever Neumann-ness it has to $b$, but $u$ will satisfy Dirichlet conditions you imposed on it since that what you imposed. I could say more if you posted the continuous problem from which this is all derived.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-06-27T12:10:01.213" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="13004" PostTypeId="1" AcceptedAnswerId="13019" CreationDate="2014-06-27T12:40:28.223" Score="4" ViewCount="63" Body="&lt;p&gt;In Monte Carlo simulations, using the Metropolis criterion, one often has to compare a random number $a$,  $0 \leq a &amp;lt; 1$, to the Boltzmann distribution $exp(-\beta\Delta E)$, where $\Delta E$ is the energy cost of a transition between two states in the simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common optimization, is to check if $\Delta E &amp;gt; 0$, in which case, $a$ is certainly smaller than $exp(-\beta\Delta E)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my simulation, it seems, the calculation of the exponential function, seems to take the most time and I'm looking into methods of reducing the cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One method I thought of, is based on the fact that for even $K$, the Taylor series $\sum_{k = 0}^{K}x^k/k!$, is a lower bound to $exp(x)$, while for odd $K$, it is an upper bound. Thus, one can write a function like the following,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;&lt;code&gt;&#xA;static inline int fast_metropolis(double a, double x){&#xA;    double est = 1.0 + x;&#xA;    double f = x;&#xA;    int i = 1;&#xA;    while(true){&#xA;        if(rand_num &amp;lt est) return 1;&#xA;        f *= x / (i + 1);&#xA;        est += f;&#xA;        if(rand_num &amp;gt est) return 0;&#xA;        f *= x / (i + 2);&#xA;        est += f;&#xA;        i += 2;&#xA;    }&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, in hindsight, the above code doesn't work very negative arguments, since the absolute value of the bounds increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any alternatives to the method described above i.e. could we iteratively refine some upper and lower bounds while making sure their absolute value is decreasing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or am I better of using some 'fast' estimate of the exponential function?&lt;/p&gt;&#xA;" OwnerUserId="9361" LastActivityDate="2014-06-30T11:16:48.837" Title="Iteratively refine bounds on exp for Metropolis criterion" Tags="|optimization|simulation|iterative-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13005" PostTypeId="1" CreationDate="2014-06-27T14:59:25.500" Score="3" ViewCount="234" Body="&lt;p&gt;I want to solve the following system of nonlinear reaction-diffusion equations (Schnakenberg Turing) using FEM methods (such as &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal.ii&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \partial_{t} u = \Delta u + \gamma\left(a-u+u²v\right)$$&#xA;$$ \partial_{t} v = d\Delta v + \gamma\left(b-u²v\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where d, $\gamma$, a, b being constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably I need to first apply a time integration scheme (such as Crank-Nicholson, implicit Runge-Kutta), and then FE space discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: How should I choose and perform the time integration for this nonlinear system? Any hints for very similar examples or specific literature?&lt;/p&gt;&#xA;" OwnerUserId="5308" LastEditorUserId="3481" LastEditDate="2014-06-27T15:09:21.770" LastActivityDate="2014-08-26T17:13:08.713" Title="Time Integration of a nonlinear reaction-diffusion system" Tags="|pde|finite-element|nonlinear-equations|time-integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13006" PostTypeId="1" CreationDate="2014-06-27T15:00:03.090" Score="0" ViewCount="173" Body="&lt;p&gt;I'm trying to make my code more efficient so exploring alternatives and providing a simplified example here. Here are 3 different versions I thought of for defining an anonymous function in matlab. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Version 1 definition in script&#xA;&#xA;clc;&#xA;clear;&#xA;&#xA;syms x1 x2;&#xA;&#xA;b = 2;&#xA;&#xA;c = 3.5;&#xA;&#xA;ExampleFunc = @(x1,x2) x1^3 + b*x2 + c;&#xA;&#xA;% Below from command window for version 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;ExampleFunc&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ExampleFunc = &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; @(x1,x2)x1^3+b*x2+c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;ExampleFunc(2.5,5)&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ans =&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;29.1250&#xA;&#xA;&#xA;&#xA;% Version 2 definition in script&#xA;&#xA;clc;&#xA;clear;&#xA;&#xA;syms x1 x2;&#xA;&#xA;b=2;&#xA;&#xA;c=3.5;&#xA;&#xA;TempExpr = x1^3 + b*x2 + c;&#xA;&#xA;ExampleFunc = @(x1,x2) TempExpr;&#xA;&#xA;% Below results from command window  for version 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;ExampleFunc&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ExampleFunc = &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; @(x1,x2)TempExpr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;ExampleFunc(2.5,5)&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ans =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;x1^3 + 2*x2 + 7/2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Version 3 definition in script&#xA;&#xA;clc;&#xA;&#xA;clear;&#xA;&#xA;syms x1 x2 b c;&#xA;&#xA;TempExpr = x1^3 + b*x2 + c;&#xA;&#xA;b=2;&#xA;&#xA;c=3.5;&#xA;&#xA;ExampleFunc = @(x1,x2) TempExpr;&#xA;&#xA;% Below results from command window  for version 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;ExampleFunc&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ExampleFunc = &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; @(x1,x2)TempExpr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;ExampleFunc(2.5,5)&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ans =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;x1^3 + c + b*x2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do the above versions provide different results when I try to query the definition and evaluate for particular values of x1 and x2: -&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In Version 1, the definition does not show the values of b and c in it (though I understand that as per Matlab documentation the values for b and c are stored in back-end as constant values - Btw, how to check what those exact values are. Typing b, c in command window wouldn't be correct as it will show the values assumed by those 2 variables in the workspace at time of querying and not while defining?). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;On the other hand, in Version 2 (and version 3) the definition does not expand out TempExpr (Btw, how to check what TempExpr is? I can type it in but is that the only way?) In case of Version 2, typing TempExpr in command window replaces b and c with values used during its definition. In case of Version 3, typing TempExpr in command window retains b and c in symbolic form?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why does the evaluation at x1=2.5 and x2=5, work correctly for Version 1 (numerical answer provided) while in Version 2 it only shows the underlying definition for ExampleFunc (with b and c substituted based on values used during definition) but does not evaluate at the given point.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In Version 3, neither b and c get substituted and nor is the function evaluated at given values x1, x2?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Intended result&lt;/strong&gt;: My ultimate goal is to force Version 3 to behave like Version 1 in terms of function evaluation. Reason is in my application, b, c can be thought of as &#xA;parameters which when defined in symbolic form enables precise definition of TempExpr and then subsequently for particular values of the parameters b and c, I pass the ExampleFunc to fmincon which then evaluates ExampleFunc at specific values of x, x2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently in order to force Version 3 to behave like Version 1, I have to use subs function for definition of ExampleFunc which increases execution time (significant as I'm running fmincon for huge number of parameter combinations). My hope was that by using anonymous functions the values of b, c would be automatically substituted with values used during the anonymous function definition but that is not the behaviour noticed here.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastEditorUserId="7153" LastEditDate="2014-06-27T15:06:25.197" LastActivityDate="2014-06-29T23:20:17.227" Title="Trying to understand results for anonymous function defined in two different ways" Tags="|matlab|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13007" PostTypeId="1" CreationDate="2014-06-27T15:14:23.427" Score="3" ViewCount="152" Body="&lt;p&gt;I have spent quite some time trying to solve the following quadratic program: &#xA;$$\min \sum_{i=1}^n (\frac{1}{2}x_i^TQx_i+c_i^Tx_i), \quad \mathrm{s.t. } \quad x_i\ge 0 \quad \forall i,$$&#xA;where $n$ is very large and $Q$ is a &lt;strong&gt;sparse&lt;/strong&gt; positive semi-definite matrix (given at the end of this post). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the sub-problem for each $i$ can be easily solved (numerically), since $n$ is very large, it is infeasible to loop and solve the sub-problems separately. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there is any special method to deal with this type of problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $Q$ in my particular problem is given by $Q=A^TA+B^TB$, where $A$ and $B$ are defined as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denote $e=(1,1,\ldots,1)^T\in\mathbb{R}^d$, then &#xA;$$A=\begin{bmatrix}&#xA;e &amp;amp;  &amp;amp;  &amp;amp; \\ &#xA; &amp;amp; e &amp;amp;  &amp;amp; \\ &#xA; &amp;amp;  &amp;amp; \ddots &amp;amp; \\ &#xA; &amp;amp;  &amp;amp;  &amp;amp; e&#xA;\end{bmatrix}$$ and $$B=\begin{bmatrix}\mathrm{diag}(e) &amp;amp; \mathrm{diag}(e) &amp;amp; \cdots &amp;amp; \mathrm{diag}(e)\end{bmatrix}$$&#xA;where $e$ appears $d$ times in $A$ and $d$ times in $B$ (i.e. $A,B\in\mathbb{R}^{d\times d^2}$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if $d=3$ we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A=\begin{bmatrix}&#xA;1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\ &#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\ &#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 1&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B=\begin{bmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\ &#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\ &#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1&#xA;\end{bmatrix}$$&#xA;and thus&#xA;$$Q=\begin{bmatrix}&#xA;M &amp;amp; I &amp;amp; I\\ &#xA;I &amp;amp; M &amp;amp; I\\ &#xA;I &amp;amp; I &amp;amp; M&#xA;\end{bmatrix}$$&#xA;where $$M=\begin{bmatrix}&#xA;2 &amp;amp; 1 &amp;amp; 1\\ &#xA;1 &amp;amp; 2 &amp;amp; 1\\ &#xA;1 &amp;amp; 1 &amp;amp; 2&#xA;\end{bmatrix},\quad &#xA;I = \begin{bmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0\\ &#xA;0 &amp;amp; 1 &amp;amp; 0\\ &#xA;0 &amp;amp; 0 &amp;amp; 1&#xA;\end{bmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Matlab code for creating $Q$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d = 16;&#xA;one = ones(d,1);&#xA;A = kron(eye(d),one');&#xA;B = repmat(diag(one),1,d);&#xA;Q = A'*A + B'*B;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for any suggestions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; according to the comments, it was not clear why I do not want to solve the subproblems separately &lt;strong&gt;using a loop&lt;/strong&gt;. So here are some clarifications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we do a loop and solve each subproblem at each iteration, then it will take too much time if the number of subproblems is huge. For example, if solving a subproblem takes 1 second, then solving 220512 subproblems will take more than 60 hours. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use some iterative method to solve the subproblems, then instead of updating each $x_i$ the ones after the others (using the loop), can we update them simultaneously? We can reformulate the problem as: &#xA;$$\min\quad 1/2x^T\mathrm{diag}(Q,Q,...,Q)x + c^Tx, \quad \mathrm{s.t. } \quad x\ge 0,$$&#xA;where $x=(x_1,...,x_n)$ and $c=(c_1,...,c_n)$ and then use a sparse QP solver. I tried using Matlab's solver (an example is given &lt;a href=&quot;http://www.mathworks.fr/fr/help/optim/ug/large-sparse-quadratic-program-with-interior-point-algorithm.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;), however, the sparse matrix $\mathrm{diag}(Q,Q,...,Q)$ takes ~9GB of memory and thus slows down everything. Moreover, in using (naively) a sparse QP solver, the special structure of $\mathrm{diag}(Q,Q,...,Q)$ is not exploited (although it is considered to be sparse by the solver, I think we can exploit even more than that).&lt;/p&gt;&#xA;" OwnerUserId="7695" LastEditorUserId="7695" LastEditDate="2014-06-27T23:28:53.400" LastActivityDate="2014-06-27T23:28:53.400" Title="A separable nonnegative quadratic program" Tags="|optimization|convex-optimization|nonlinear-programming|quadratic-programming|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="13008" PostTypeId="2" ParentId="13007" CreationDate="2014-06-27T15:59:57.630" Score="0" Body="&lt;p&gt;Before venturing for specialize algorithms, have you tried a good QP solver as MOSEK, CPLEX or GUROBI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the extreme sparsity of your matrices, you could be able to solve this problems.  &lt;/p&gt;&#xA;" OwnerUserId="7396" LastActivityDate="2014-06-27T15:59:57.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13009" PostTypeId="2" ParentId="13005" CreationDate="2014-06-27T16:09:02.163" Score="1" Body="&lt;p&gt;As a first pass I would suggest a splitting scheme. Do an implicit timestepping scheme (e.g. CN) for the diffusion part for both u and v equations. Then do an explicit step for the nonlinear terms (e.g. RK).&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2014-06-27T16:09:02.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13010" PostTypeId="1" AcceptedAnswerId="13013" CreationDate="2014-06-27T17:41:34.430" Score="3" ViewCount="172" Body="&lt;p&gt;For Monte Carlo simulations, or any other numerical methods that rely heavily on the quality of the pseudo-random numbers generated (i.e even/desired distribution on a certain domain) for that matter, why aren't evenly-spaced and perfectly/accurately distributed numbers (not pseudo-random) used as opposed to pseudo-random numbers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I ask this is because I would imagine the main purpose of sampling tons of pseudo-random data as opposed to non-random data would be to speed up the time taken by the program that is sampling that data, much like how throwing a bunch of paintballs at a canvas/wall would, more easily and quickly, cover up that canvas/wall than carefully painting it with a paintbrush would. However, many pseudo-random number generator algorithms that I have seen look more complicated and time-consuming than just using non-pseudo-random, perfectly distributed data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly I have a misunderstanding of this topic. Could anybody help clear this issue up?&lt;/p&gt;&#xA;" OwnerUserId="9562" LastActivityDate="2014-06-27T21:32:38.067" Title="Monte Carlo Simulation - Random Number Motivation" Tags="|monte-carlo|data-analysis|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="13011" PostTypeId="2" ParentId="13007" CreationDate="2014-06-27T18:31:58.047" Score="3" Body="&lt;p&gt;With $d=16$, the $Q$ matrix is just 256 by 256. Thus the individual subproblems are quite small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your $Q$ matrices are singular, so the optimal solution to each of the subproblems is likely to be nonunique.  Do you care which solutions you end up with?  For example, are minimum norm solutions required?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You write that &quot;since n is very large, it is infeasible to loop and solve the sub-problems separately.&quot;  Solving one very large problem &lt;em&gt;shouldn't&lt;/em&gt; be any more efficient than solving the subproblems separately.  If it is, then this indicates that overhead in the solver is excessive and you might want to look for a better solver.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you want to better understand the performance of the QP solver that you're using, then you might try some computational experiments.  You could for example truncate the sum to $n=10000$ terms, then solve 1 subproblem at time, and compare this with solving 10 subproblems at a time and then 100 subproblems at a time to see whether you get any speedup from combining the subproblems.  If this does help things, then you can always adjust the number of subproblems that you solve at a time to match the available memory.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you've got access to a cluster of computers, then you could easily make efficient use of as many processors as you can get by breaking the sum up into parts and parceling these out to the individual processors.     &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-06-27T18:31:58.047" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="13013" PostTypeId="2" ParentId="13010" CreationDate="2014-06-27T21:32:38.067" Score="3" Body="&lt;p&gt;The main purpose of sampling tons of pseudo-random data as opposed to non-random data is related to &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%27s_phenomenon&quot; rel=&quot;nofollow&quot;&gt;Runge's phenomenon&lt;/a&gt; for polynomial interpolation: Uniform spacing of interpolation points is often a bad idea. But choosing better interpolation points require knowledge of the function you want to interpolate (or integrate etc.). If you don't have that knowledge, or if computing the points is infeasible due to the high dimensionality of the function, your best bet (literally) is to draw points at random. You might not hit the &quot;good&quot; points often, but at least you're not systematically missing all of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In your example: If you don't know &lt;em&gt;where&lt;/em&gt; the walls are, throwing brushes at random means you will likely(!) hit them earlier than if you were starting in one corner and methodically working your way around. Remember, you don't have to cover &lt;em&gt;all&lt;/em&gt; the wall, just enough of them to see the shape.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if your points are truly random, it can happen with positive (albeit small) probability that there's a large part of space where none of the points land (or a wall you never hit, because all of your brushes by chance go in the other direction). This is the point of quasi-Monte Carlo methods: The points are still random (meaning there's no possibly bad structure), but there's some guarantee of uniform coverage on a large scale.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2014-06-27T21:32:38.067" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13015" PostTypeId="1" CreationDate="2014-06-28T14:31:49.627" Score="4" ViewCount="129" Body="&lt;p&gt;I am looking for a nice reference (a review, tutorial, or maybe a book) that has tips and their explanations about general issues of floating-point arithmetic in scientific computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some that I've seen are Numerical Recipes and Goldberg, D. E. (1991). What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 23(1), 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical Recipes is a very large book with specific algorithms. I am looking for general rules of thumb, of general applicability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article by Goldberg has a bit too much emphasis on multiple hardware implementations. I really only care about C/C++ programming on modern PCs, which has a pretty standard way of dealing with floats.&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2014-06-28T14:42:54.410" LastActivityDate="2014-06-28T14:42:54.410" Title="Floating-point arithmetic in scientific computations rules of thumb" Tags="|reference-request|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13016" PostTypeId="1" CreationDate="2014-06-28T16:59:34.740" Score="2" ViewCount="745" Body="&lt;p&gt;I understand how to set up the boundary conditions for a steady state problem discretized by Galerkin method, for a time dependent PDE below,&#xA;$$\frac{\partial}{\partial t} u =  c\nabla^2 u + a\nabla u + b,~~~ u_0= 1, u'_1=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supposing I have correctly set up the Dirichlet and Neumann conditions for the steady state part $c\nabla^2 u + a\nabla u + b=0$, which leads to an algebraic system $ C u + A u + B=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then turning to the time dependent problem, I have an additional &quot;mass&quot; matrix at left hand side,&#xA;$$Mu=\frac{\partial}{\partial t} u, ~~~~~~~~ M=M(v,u)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The problem I encountered is&lt;/em&gt;:&#xA;If I don't fix the Dirichlet end at each time iteration(force it to be the constant, seems a little &quot;dirty&quot;), the solution would blow up at the Dirichlet boundary in the end. also some oscillation can be observed(depends on the specific $\theta$ chosen) near the Dirichlet boundary during the integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I don't have to modify the matrix $C, A, B$ (boundary part), right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) what is the Boundary condition for $M$ (if the $\theta$ method is to be implemented), also Dirichlet and Neumann respectively? I.e. use the same basis function as the time-indepedent part. &lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="1804" LastEditDate="2014-06-29T10:15:54.803" LastActivityDate="2014-06-29T10:23:12.937" Title="Method of lines for inhomogeneous Dirichlet conditions" Tags="|finite-element|boundary-conditions|time-integration|parabolic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13017" PostTypeId="2" ParentId="13016" CreationDate="2014-06-28T20:50:55.657" Score="1" Body="&lt;p&gt;In a Galerkin approach for &lt;em&gt;stationary&lt;/em&gt; PDEs, you write your unknown solution as a linear combination of &quot;nice&quot; basis functions $\{\phi_j\}_{j=1,\dots,n}$,&#xA;$$u(x) \approx \sum_{j=1}^n u_j \phi_j(x),$$&#xA;and project the equation onto the space spanned by (these or other) functions, exploiting  the linearity of the differential operators and the inner product. In your case, you get &#xA;$$\sum_{j=1}^n u_j (c\nabla^2 \phi_j(x)+a\nabla \phi_j(x) + b,\psi_i(x))=0\qquad\text{for all }1\leq i\leq n.$$&#xA;This yields a set of linear equations for the unknown coefficients $u_j$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the finite element method, boundary conditions are implemented differently for Dirichlet and for Neumann conditions. Homogeneous Dirichlet conditions are built into the choice of the basis: $\phi_j(0)=\psi_i(0)=0$ for all $1\leq i,j\leq n$. The Neumann conditions, on the other hand, become part of the operator:&#xA;You integrate by parts in the first term of the sum to obtain (I am leaving out the lower order terms since they do not play such a role here)&#xA;$$\sum_{j=1}^n u_j \left(\phi_j'(1)\psi_i(1)-(c\nabla \phi_j(x),\nabla\psi_i(x)) \right)\qquad\text{for all }1\leq i\leq n$$&#xA;(using that all boundary terms for $x=0$ vanish due to the choice of $\psi$).&#xA;Of course, then you use that&#xA;$$\sum_{j=1}^n u_j \phi_j'(1)\psi_i(1) = u'(1)\psi_i(1)$$&#xA;so that becomes part of your right-hand side. In your case, $u'(1)=0$ so the whole term vanishes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the method of lines, you approximate your unknown solution $u(t,x)$ as&#xA;$$u(t,x)\approx \sum_{j=1}^n u_j(t) \phi_j(x),$$&#xA;i.e., the whole space dependence is in the basis &lt;em&gt;functions&lt;/em&gt;, while the whole time dependence is in the &lt;em&gt;scalar coefficients&lt;/em&gt;. Your projected equation is now&#xA;$$\sum_{j=1}^n u_j(t) (c\nabla^2 \phi_j(x)+a\nabla \phi_j(x) + b,\psi_i(x)) = \sum_{j=1}^n \frac{d}{dt}u_j(t)(\phi_j(x),\psi_i(x))\qquad\text{for all }1\leq i\leq n.$$&#xA;Now you do the same steps; since the inner products on the left-hand side are exactly the same as before, you get exactly the same matrices. On the right-hand side, you have a term just like a standard mass matrix $M$ with entries $M_{ij}=(\phi_j(x),\psi_i(x))$, where you respect the Dirichlet conditions by the choice of basis functions, but you don't have to worry about Neumann conditions. Note that the time dependence is purely in the coefficients, so for the (spatial) mass matrix it makes no difference whether this comes from approximating $u$ or $\partial_t u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way inhomogeneous Dirichlet conditions are treated mathematically is to reduce the problem to the homogeneous case: You write&#xA;$$u(t,x) = g(t,x) + \tilde u(t,x)$$&#xA;for some fixed $g$ with $g(t,0)=1$ and $g'(t,1)=0$. Then, $\tilde u$ satisfies&#xA;$$c\nabla^2 \tilde u + a\nabla \tilde u = -b -c\nabla^2 g -a\nabla g,\qquad \tilde u(0) = \tilde u'(1) = 0,$$&#xA;and you can proceed as above to compute $\tilde u(t,x)$ and then $u(t,x) = \tilde u(t,x)+g(t,x)$. If you work with the standard hat functions, you could simply take $g(t,x) = \phi_1(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the initial condition (which you have not stated) satisfies $u_0(0)=1$, you could also replace the first equation of the system of ODEs by $\partial_t u_1 = 0$, i.e., replace the first row in the mass matrix by $(1,0,\dots,0)$, the first row of the remaining matrices by $(0,\dots,0)$, and the first component of the right-hand side by $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If memory serves correctly, the method of lines is discussed in &lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-1-4757-4355-5&quot; rel=&quot;nofollow&quot;&gt;Ern, Guermond: &lt;em&gt;Theory and Practice of Finite Elements&lt;/em&gt;, Springer, 2004&lt;/a&gt;, Chapter 6.1.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-06-29T10:23:12.937" LastActivityDate="2014-06-29T10:23:12.937" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="13018" PostTypeId="2" ParentId="13006" CreationDate="2014-06-29T23:20:17.227" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Intended result: My ultimate goal is to force Version 3 to behave like Version 1 in terms of function evaluation. Reason is in my application, b, c can be thought of as parameters which when defined in symbolic form enables precise definition of TempExpr and then subsequently for particular values of the parameters b and c, I pass the ExampleFunc to fmincon which then evaluates ExampleFunc at specific values of x, x2.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If that is what you want to do, why not use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Closure_%28computer_programming%29&quot; rel=&quot;nofollow&quot;&gt;closure&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ExampleFunc] = function myClosure(b,c)&#xA;return @(x1,x2)(x1.^3 + b * x2 + c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This construct builds upon your existing anonymous function construction by allowing you to bind values of &lt;code&gt;b&lt;/code&gt; and &lt;code&gt;c&lt;/code&gt; to your anonymous function nonlocally. You can use the closure as a factory to generate a bunch of objective functions with different parameter values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your three examples, the differences between Examples 1 and 2 mainly seem to have to do with scope; when using &lt;code&gt;TempExpr&lt;/code&gt; in the anonymous function definition instead of the actual expression, it uses the variables &lt;code&gt;x1&lt;/code&gt; and &lt;code&gt;x2&lt;/code&gt; defined at global scope rather than the anonymous function scope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between Examples 2 and 3 mainly seems to be that in Example 3, you declare &lt;code&gt;b&lt;/code&gt; and &lt;code&gt;c&lt;/code&gt; as symbolic variables. I don't see why you need any variables here to be symbolic, since you're only evaluating functions; perhaps I'm missing something.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-06-29T23:20:17.227" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="13019" PostTypeId="2" ParentId="13004" CreationDate="2014-06-30T11:16:48.837" Score="3" Body="&lt;p&gt;The simplest solution to my question was quite obvious, too bad I didn't see it before posting the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can split the argument of $exp(x)$ into an integer part and a fractional part. For the integer part one can precompute a lookup table, while for the fractional part you can use the algorithm described in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below I show the final code I used,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;&lt;code&gt;&#xA;static inline int fast_metropolis(double rand_num, double x){&#xA;    int exp_i = -(int)x;&#xA;    if(exp_i &amp;gt 256) return rand_num &amp;lt exp(x);&#xA;    double prefactor = (exp_i &amp;gt 0)? exp_table[exp_i - 1]: 1.0;&#xA;    double k = x + exp_i;&#xA;    double f = k;&#xA;    double est = 1.0 + k;&#xA;    int i = 1;&#xA;    while(1){&#xA;        if(rand_num &amp;lt prefactor * est) return 1;&#xA;        f *= k / (i + 1);&#xA;        est += f;&#xA;        if(rand_num &amp;gt prefactor * est) return 0;&#xA;        f *= k / (i + 2);&#xA;        est += f;&#xA;        i += 2;&#xA;    }&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The algorithm described, was quite accurate in my simulations, with only a $8.4 \cdot 10^{-6}\%$ of the cases being different from using the standard library's &lt;code&gt;exp&lt;/code&gt; function. In comparison, the fast exponential function, found &lt;a href=&quot;https://stackoverflow.com/questions/10552280/fast-exp-calculation-possible-to-improve-accuracy-without-losing-too-much-perfo&quot;&gt;here&lt;/a&gt;, was different in $0.3\%$ of the cases, although it had slightly better performance in simulation. Performance tests with a uniform distribution for the argument of $exp(x)$ seem to indicate a 3x performance gain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will not mark my answer correct for a couple of days, in case someone comes up with a faster/more accurate algorithm.&lt;/p&gt;&#xA;" OwnerUserId="9361" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-06-30T11:16:48.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13021" PostTypeId="1" AcceptedAnswerId="13048" CreationDate="2014-06-30T23:03:39.920" Score="8" ViewCount="1086" Body="&lt;p&gt;Are there any mature Python packages that can plot &lt;em&gt;functions&lt;/em&gt;, and possibly use adaptive sampling?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to pass only a function (can be a numerical black box) and a range, and expect a plot as the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am &lt;em&gt;not&lt;/em&gt; looking for: manually generating the sampling points, manually computing the function values, and finally plotting these &lt;em&gt;pre-computed&lt;/em&gt; values.  I know matplotlib can do this. &lt;/p&gt;&#xA;" OwnerUserId="605" LastActivityDate="2014-07-05T22:21:42.003" Title="Python package for (adaptive) function plotting" Tags="|python|plotting|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13022" PostTypeId="2" ParentId="13021" CreationDate="2014-06-30T23:47:46.913" Score="5" Body="&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.sagemath.org/doc/reference/plot3d/sage/plot/plot3d/plot3d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sage can plot functions adaptively&lt;/a&gt;; the link is to a 3d implementation, but presumably, there is an analogous 2d implementation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could use &lt;a href=&quot;http://scipy-central.org/item/53/1/adaptive-sampling-of-1d-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;this function that samples adaptively 1-d functions&lt;/a&gt;. (see &lt;a href=&quot;https://stackoverflow.com/questions/14084634/adaptive-plotting-of-a-function-in-python&quot;&gt;https://stackoverflow.com/questions/14084634/adaptive-plotting-of-a-function-in-python&lt;/a&gt; for discussion) Judging from your comments, this may not be a solution you want, but presumably, if you can do all of the steps individually, cobbling together a convenience wrapper with the interface you want would not take too long.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://docs.sympy.org/dev/modules/plotting.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy has an adaptive plotting interface&lt;/a&gt; included; it appears to only plot SymPy expressions, though, which may not suit your needs. (Ondrej can correct me on this point.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These implementations probably span the extremes of heavyweight (Sage is a big library) and lightweight (a single sampling function is not intrusive, and presumably, you could use it with any graphics backend you like).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-06-30T23:47:46.913" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="13023" PostTypeId="1" AcceptedAnswerId="13025" CreationDate="2014-07-01T00:00:03.637" Score="9" ViewCount="1637" Body="&lt;p&gt;What are the most accurate algorithms for evaluating a polynomial using floating point arithmetic? The internet seems to suggest that Horner's method is commonly used. In particular I have a cubic polynomial of the form f(x) = a x^3 + b x^2 + c x + d, which I am evaluating in the naive way with floating point arithmetic, and was curious whether some recursive approach or some refactoring would give a more accurate result. That is, a result closer to what I would get if I were to evaluate the polynomial exactly or with a metric ton of precision in Mathematica.&lt;/p&gt;&#xA;" OwnerUserId="1862" LastActivityDate="2020-09-01T22:13:01.120" Title="Accurate Polynomial Evaluation in Floating Point" Tags="|floating-point|polynomials|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13025" PostTypeId="2" ParentId="13023" CreationDate="2014-07-01T06:59:21.020" Score="9" Body="&lt;p&gt;Horner is indeed the most stable way to evaluate a polynomial (and you get the bonus of evaluating its derivatives with not too much extra cost). Higham presents a nice error analysis of the algorithm (Accuracy and stability of numerical algorithms, 2nd edition, p.94). He also presents an algorithm that includes a running error bound so you have an idea on the difference between the real value and what Horner calculated (Algorithm 5.1 on p.95).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your case is limited to cubic polynomials and you're worried about loop overhead, you could unroll the loops by hand. But I doubt you will gain much. Stick to the tried and tested.  &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2014-07-01T06:59:21.020" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13026" PostTypeId="1" CreationDate="2014-07-01T14:47:58.943" Score="0" ViewCount="544" Body="&lt;p&gt;I have a bunch of readings that run every 4 hours, however each sensor has a different offset.  One sensor might read at $t = 0,4,8,12,16,20$ and another senor reads at $t = 1,5,9,13,17,21$.  This makes it difficult to compare the reading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be nice to put them all on equal footing - &quot;normalization&quot;.  To do this, I am considering the following problem:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let $f(t)$ be a periodic function with $f(t+24) = f(t)$.  We are given $f(4k)$ for $k = 0,1,\dots,5$.  How can we estimate $f(1)$?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Really I want to estimate $f(t)$ for all $0 \leq t &amp;lt; 24$ but you get the idea.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;One approach I considered is using Fourier Transform.  Let $\omega = e^{\pi i /3}$ be a sixth root of unity and write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(n) = \hat{f}(0)\cdot 1 + \hat{f}(1)\cdot \omega^{n/6} + \hat{f}(2)\cdot \omega^{2n/6}  + \hat{f}(3)\cdot \omega^{3n/6}  + \hat{f}(4)\cdot \omega^{4n/6}  + \hat{f}(5)\cdot \omega^{5n/6}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I am asking for $f(n+\tfrac{1}{4})$ which I can estimate using this Fourier transform.  Not sure if my scaling is correct.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Is this correct or are there alternative ways like linear interpolation that work better?&lt;/p&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2014-07-06T14:50:16.780" Title="interpolating a periodic time series" Tags="|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13027" PostTypeId="2" ParentId="8616" CreationDate="2014-07-01T18:28:08.883" Score="0" Body="&lt;p&gt;Generally people apply polarization function only on the atoms coordinating the metal, but not on others (other parts of the ligand). It is true that you don't want very different basis sets for different part of the molecule. But if you introduce polarization functions for the whole ligand, you will see that polarization actually occurs only on coordinating atoms which are directly exposed to the charge of the metal ion, while the other parts are not influenced. &lt;/p&gt;&#xA;" OwnerUserId="9559" LastActivityDate="2014-07-01T18:28:08.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13028" PostTypeId="1" CreationDate="2014-07-01T23:07:31.263" Score="6" ViewCount="258" Body="&lt;p&gt;I'm looking at FEM discretizations of&#xA;$$u_i - \Delta u_i = f$$ &#xA;for $u_1, u_2$ on subdomains $\Omega_1, \Omega_2$ with interface $\Gamma$. A Neumann-Neumann transmission condition can be formulated by solving for a flux $\lambda$ on $\Gamma$ such that $n\cdot \nabla u_1 = \lambda = -n\cdot \nabla u_2$. One &quot;dual&quot; variational form involves formulating the problem using a Lagrange multiplier unknown $\lambda$, such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align*}&#xA;a_1(u_1,v_1) + a_2(u_2,v_2) + \int_{\Gamma} \lambda [[v]]&amp;amp;= (f,v1)+(f,v2)\\&#xA;\int_{\Gamma} \mu [[u]] &amp;amp;= 0&#xA;\end{align*}$$&#xA;and we can eliminate $u_1, u_2$ to solve only in terms of $\lambda$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this for other transmission conditions? I can repeat the same process with for Robin interfaces and add $\alpha u_i$ to the interface BCs. Redefining a new Lagrange multiplier $\tilde{\lambda}$ gives&#xA;$$&#xA;\begin{align*}&#xA;-\alpha u_1 - n\cdot\nabla u_1 &amp;amp;= \tilde{\lambda} = \lambda - \alpha u_1\\&#xA;\alpha u_2 - n\cdot\nabla u_2 &amp;amp;= \tilde{\lambda} = -(\lambda - \alpha u_2).&#xA;\end{align*}&#xA;$$&#xA;but the second BC is no longer coercive (the boundary term contributes negative $\int_{\Gamma}\alpha u_2v_2$ to $a_2(u_2,v_2)$). Is there a way to do a stable Lagrange multiplier formulation for the Robin-Robin case for elliptic problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'd also be grateful for a reference to previous work if I've overlooked the answer to this in literature.)&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2014-07-08T03:25:21.070" LastActivityDate="2014-08-09T05:28:26.300" Title="Domain decomposition w/Lagrange multipliers" Tags="|domain-decomposition|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13032" PostTypeId="1" AcceptedAnswerId="13033" CreationDate="2014-07-02T13:58:14.443" Score="5" ViewCount="3919" Body="&lt;p&gt;[I previously asked my question on &lt;a href=&quot;https://stackoverflow.com/questions/24505583/python-multisearch-optimization-using-parallel-programming&quot;&gt;StackOverflow&lt;/a&gt; but this site may be more appropriate]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab, I am currently using the &lt;a href=&quot;http://www.mathworks.com/help/gads/multistart-class.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MultiStart&lt;/a&gt; as an optimization algo in a parallel setup for a computer cluster. For example, this is my Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;opts = optimoptions(@fmincon,'Algorithm','sqp','Use Parallel','Always'); %The options for the algo. The key here is the Use Parallel &#xA;problem = createOptimProblem('fmincon','objective',...&#xA; @(x) x.^2 + 4*sin(5*x),'x0',3,'lb',-5,'ub',5,'options',opts); %x is the variable I wish to min/max&#xA;ms = MultiStart;&#xA;[x,f] = run(ms,problem,20) %20 stands for the number of initial random variables&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code works very well when I submit a batch job on the cluster for X processors.&#xA;I am looking online for a similar algo that I could implement in parallel using Python and I am bit loss. Any similar algo is available for Python?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&#xA;As for my cluster requirements, multithreading is not acceptable. Usually I use &lt;code&gt;mpi4py&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9598" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-07-02T20:41:24.293" Title="Python: Multistart optimization using parallel programming" Tags="|optimization|matlab|python|parallel-computing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13033" PostTypeId="2" ParentId="13032" CreationDate="2014-07-02T19:36:48.863" Score="6" Body="&lt;p&gt;From what I can tell, MultiStart is not an optimization algorithm per se, but a framework that runs a given optimization algorithm (in your code, SQP via &lt;code&gt;fmincon&lt;/code&gt;) in parallel for a set of random starting points (i.e., a separate, independent, instance of &lt;code&gt;fmincon&lt;/code&gt; for each starting point).  Since the instances do not need to communicate at all, this does not require MPI (the technical term is &quot;embarrassingly parallel&quot;), so you can use any job server to launch the instances, each on a separate node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Python is (if you so choose) much more low-level than Matlab, you can implement this yourself rather easily: Pick an implementation of a minimization algorithm (the closest thing to &lt;code&gt;fmincon&lt;/code&gt; would be SciPy's &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;scipy.optimize.minimize&lt;/code&gt;&lt;/a&gt;) and wrap it in a &lt;a href=&quot;https://docs.python.org/2/library/multiprocessing.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;multiprocessing&lt;/code&gt;&lt;/a&gt; Pool; the module &lt;a href=&quot;https://pythonhosted.org/joblib/parallel.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;joblib&lt;/code&gt;&lt;/a&gt; seems to provide a convenient wrapper for this. If you run this on a cluster, you could try &lt;a href=&quot;http://dispy.sourceforge.net&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;dispy&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a higher level, Coopr (an optimization modeling framework similar to AMPL) provides capabilities for &lt;a href=&quot;https://software.sandia.gov/downloads/pub/coopr/CooprGettingStarted.html#_solving_multiple_instances_in_parallel&quot; rel=&quot;noreferrer&quot;&gt;Solving Multiple Instances in Parallel&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="276" LastEditDate="2014-07-02T20:33:16.037" LastActivityDate="2014-07-02T20:33:16.037" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="13034" PostTypeId="1" AcceptedAnswerId="13036" CreationDate="2014-07-02T20:33:36.120" Score="1" ViewCount="285" Body="&lt;p&gt;I am trying to compute the weight matrix for a set of straight ray paths through a reconstruction region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would like to be able to do this for both a rectangular grid region, where each grid cell is a rectangular prism, and a geodetic grid region, where grid cell boundaries are specified in latitude, longitude, and altitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked into the ASTRA library, but cannot get it to compile on my system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried implementing the algorithm myself, but it is rather non-robust, buggy, and I feel like there should be a spelt out algorithm somewhere to do this.&lt;/p&gt;&#xA;" OwnerUserId="9600" LastActivityDate="2014-07-03T09:35:29.023" Title="How to compute the weight matrix for tomography applications?" Tags="|algorithms|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13035" PostTypeId="1" CreationDate="2014-07-02T20:58:53.550" Score="18" ViewCount="777" Body="&lt;p&gt;In most (all?) implementations of the Fast Multipole Method (FMM), octrees are used to decompose the relevant domain. Theoretically, octrees provide a simple volumetric bound, which is useful for proving the O(n) runtime of a FMM. Beyond this theoretical rationale, are there benefits to using an Octree over other tree or trie data structures? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining the interaction list might be easier with an octree because a cell would know its immediate neighbors. However, the interaction list is unnecessary using a more dynamic tree traversal like &lt;a href=&quot;http://arxiv.org/abs/1209.3516&quot;&gt;Dual Tree Traversal&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative would be a kd-tree. One possible theoretical downside is that construction requires expensive median finding operations. However, there are versions of kd-trees that do not require median finding during construction -- albeit with less efficient space partitioning. Implemention-wise, a kd-tree is very simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An even more radical alternative might be an &lt;a href=&quot;http://en.wikipedia.org/wiki/R-tree&quot;&gt;R-tree&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is: What is about Octrees that make them the best choice for a FMM?&lt;/p&gt;&#xA;" OwnerUserId="5516" LastEditorUserId="5516" LastEditDate="2014-07-02T21:33:09.433" LastActivityDate="2018-07-23T01:41:55.390" Title="Why are Octrees used for Multipole space decomposition?" Tags="|algorithms|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="13036" PostTypeId="2" ParentId="13034" CreationDate="2014-07-03T09:35:29.023" Score="1" Body="&lt;p&gt;The canonical way to do this is using &lt;a href=&quot;http://clio.rad.sunysb.edu/micl/Proceeding_Articles/MIC_1999_GHan.PDF&quot; rel=&quot;nofollow&quot;&gt;Sidons algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where you go from there depends on your application. If your ray paths have finite width, you may want to &lt;a href=&quot;http://web.eecs.umich.edu/~fessler/papers/lists/files/proc/10/web/long-10-3fa.pdf&quot; rel=&quot;nofollow&quot;&gt;consider that in your reconstruction&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the geodesics, things get more complicated. I'm fairly confident that an analytical solution exists, but I simply don't know it. &#xA;I would suggest solving this by brute-force instead. Simply trace your line through your geometry in small steps, check which region each step is in and add the distance between this step and the previous to corresponding matrix index.. &#xA;This is essentially doing the integration via the trapezoidal method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing something similar in the reverse case (curved ray path, regtangular grid) and it works extremely well in practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data or reconstruction region gets larger than a very modest size (say, 256x256x64), you may want to consider never forming the matrix explicitly, and simply implementing the matrix-vector product directly, for use in iterative reconstruction.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-07-03T09:35:29.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13037" PostTypeId="1" CreationDate="2014-07-03T22:06:06.290" Score="5" ViewCount="890" Body="&lt;p&gt;I would like to implement custom, domain-specific algorithms for sparse matrix orderings. I am looking for Python packages for ordering sparse matrices. It would be nice to have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The underlying datastructures handle sparsity (matrix sizes: $10^4-10^6$)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To be able to handle highly unsymmetric matrices. (Introducing artificial fill-in to make them symmetric is not acceptable; there are many packages for the symmetric and the mildly unsymmetric case.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Execution speed is not a major concern but rapid prototyping certainly is&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To be able to define sparse submatrix views (block views) &lt;em&gt;recursively&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perform row and column permutations on submatrix views, which are then reflected in the original, big matrix as well&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Intuitive, easy to learn and use API&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Permissive license (preferably BSD 3-Clause or similarly permissive)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I do not need any sparse factorization methods, arithmetic or sparse linear solvers. I would like to experiment with my own ordering algorithms and that's all. For example, I would like to order a sparse matrix to recursive bordered block diagonal form (RBBD), and play with heuristics for defining blocks and nested blocks, based on domain-specific knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/sparse.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.sparse&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://pysparse.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pysparse&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/rwl/CSparse.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;CSparse&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Did I miss any major, well-established Python package for this purpose?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I would also greatly appreciate feedback regarding first-hand experience with these or other Python packages&lt;/strong&gt; for such purposes (something like we have in &lt;a href=&quot;https://scicomp.stackexchange.com/q/351/679&quot;&gt;Recommendations for a usable, fast C++ matrix library?&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="679" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-01-06T09:00:15.610" Title="Sparse matrix ordering in Python" Tags="|matrix|python|sparse-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13038" PostTypeId="2" ParentId="10375" CreationDate="2014-07-04T04:58:25.313" Score="0" Body="&lt;p&gt;Optimise your code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 1 - Optimise your R code: &#xA;&lt;br&gt;a. You can &lt;code&gt;apply()&lt;/code&gt; a function to &lt;code&gt;d&lt;/code&gt; that will both &lt;code&gt;max(d,0)&lt;/code&gt; and &lt;code&gt;d2[d==0]=0&lt;/code&gt; in one loop. &#xA;&lt;br&gt;b. Try operating on &lt;code&gt;ei$values&lt;/code&gt; directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 2 - Use C++: &#xA;&lt;br&gt;Rewrite the whole function in C++ with &lt;code&gt;RcppArmadillo&lt;/code&gt;. You will still be able to call it from R.&lt;/p&gt;&#xA;" OwnerUserId="1175" LastActivityDate="2014-07-04T04:58:25.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13039" PostTypeId="1" CreationDate="2014-07-04T07:27:20.107" Score="2" ViewCount="206" Body="&lt;p&gt;I'm developing a tool that allows users to select a closed boundary (a polygon) on the triangle mesh and then from this boundary, generate a NURBS surface fitting the original mesh surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is as following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&#xA;Generate a list of NURBS curves from the closed boundary (polygon)&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&#xA;Combine such curves together using NURBS loft surface technique (Reference: The Nurbs book - Les A. Piegl, Wayne Tiller)&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My problem arise at the first step, where I am trying to interpolate a list of NURBS curves, which require an input of sample points. To get such sample points, I am planing to generate a point grid on region inside the closed boundary, and then, for each row of this point grid, I will take sample points from this. But I don't know how to create a point grid on the mesh. Could you help me out?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just my idea, and if you know more about NURBS fitting for a region of mesh surface, please suggest me some material about it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="9618" LastEditorUserId="11765" LastEditDate="2014-11-12T00:00:21.933" LastActivityDate="2014-11-12T00:00:21.933" Title="NURBS surface fitting for a closed region on mesh" Tags="|computational-geometry|interpolation|b-spline|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13041" PostTypeId="1" AcceptedAnswerId="13062" CreationDate="2014-07-04T12:16:04.557" Score="1" ViewCount="339" Body="&lt;p&gt;Let $u$ be the answer of a PDE.Is there any relationship between $u,\frac{\partial u} {\partial n}$ and $\Delta u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the values of  $u$ and $\frac{\partial u} {\partial n}$ on $\partial \Omega$ but I need the value of $\Delta u$ on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it's impossible, but is there anyone who know how to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact I want to solve biharmonic equation by convert it into two Poisson problems:&#xA;$$\Delta^2u=f$$&#xA;$$u=g_1$$&#xA;$$\frac{\partial u} {\partial n}=g_2$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using $\Delta u=w$ leads to $$\Delta u=w,$$$$ u=g_1 ~~on~~\partial \Omega$$&#xA;$$\Delta w=f ,$$$$w=\Delta u-c(\frac{\partial u} {\partial n}-g_2)~~on~~\partial \Omega$$&#xA;so at the first I have to use an initial guess for $\Delta u$on boundary. But by this way the accuracy is low. and sometimes it is dependent to initial guess.&#xA;Here $c$ is a small constant.&lt;/p&gt;&#xA;" OwnerUserId="8271" LastEditorUserId="549" LastEditDate="2014-07-04T16:03:15.500" LastActivityDate="2014-07-07T09:53:42.183" Title="How to compute $\Delta u$ on the boundary of the biharmonic equation?" Tags="|pde|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="13042" PostTypeId="2" ParentId="13041" CreationDate="2014-07-04T15:51:51.667" Score="2" Body="&lt;p&gt;Yes, this can be done but, as you've observed, specifying the boundary conditions for the two Poisson equations can be challenging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This example from MATLAB PDE Toolbox&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/examples/clamped-square-isotropic-plate-with-a-uniform-pressure-load.html&quot; rel=&quot;nofollow&quot;&gt;Clamped Square Plate&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;solves the plate equation by converting it to a system of two scalar Poisson's equations. It uses a Robin-type BC as a trick to approximately enforce $u=0$ on the boundary. Specifically, the following two Neumann (Robin) BCs are applied:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${{\partial u}\over{\partial n}} = g_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${{\partial w}\over{\partial n}} = -ku$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k$ is a large number. If you want $g_1 \ne 0$, it isn't immediately obvious to me how to use this same technique to achieve that.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2014-07-04T18:44:27.217" LastActivityDate="2014-07-04T18:44:27.217" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13043" PostTypeId="1" AcceptedAnswerId="13044" CreationDate="2014-07-05T01:06:46.807" Score="1" ViewCount="146" Body="&lt;p&gt;Has anyone here had any success (or at least tried) compiling the HSL_ME57 (or similar such as MA57, etc.) matrix factorization libraries? Do any C wrappers exist for the Fortran function calls? I'm not having much luck compiling the library using GCC 4.9 (latest version). Here are the errors I am getting:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gfortran  -g -O2 -c -o sdeps90.o sdeps90.f90&#xA;/tmp/ccmoJQYa.s: Assembler messages:&#xA;/tmp/ccmoJQYa.s:118: Error: unknown .loc sub-directive `discriminator'&#xA;/tmp/ccmoJQYa.s:118: Error: junk at end of line, first unrecognized character is `2'&#xA;/tmp/ccmoJQYa.s:122: Error: unknown .loc sub-directive `discriminator'&#xA;/tmp/ccmoJQYa.s:122: Error: junk at end of line, first unrecognized character is `2'&#xA;/tmp/ccmoJQYa.s:126: Error: unknown .loc sub-directive `discriminator'&#xA;/tmp/ccmoJQYa.s:126: Error: junk at end of line, first unrecognized character is `2'&#xA;/tmp/ccmoJQYa.s:130: Error: unknown .loc sub-directive `discriminator'&#xA;/tmp/ccmoJQYa.s:130: Error: junk at end of line, first unrecognized character is `2'&#xA;/tmp/ccmoJQYa.s:133: Error: unknown .loc sub-directive `discriminator'&#xA;/tmp/ccmoJQYa.s:133: Error: junk at end of line, first unrecognized character is `1'&#xA;make[1]: *** [sdeps90.o] Error 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not sure that this is the right place for this question since it's mainly about compilation issues, but I don't think StackOverflow or some general programming board would be able to help out. I'm hoping that someone here has experience with the library.&lt;/p&gt;&#xA;" OwnerUserId="1480" LastActivityDate="2014-07-05T04:19:48.833" Title="Compiling HSL_ME57 and interfacing with C code" Tags="|fortran|compiling|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13044" PostTypeId="2" ParentId="13043" CreationDate="2014-07-05T04:19:48.833" Score="1" Body="&lt;p&gt;ME57 should come with a &lt;code&gt;configure&lt;/code&gt; script, so the usual &lt;code&gt;./configure; make; make install&lt;/code&gt; should just work. I tried it just now using GCC 4.9.0-7 on Debian testing.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-07-05T04:19:48.833" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="13045" PostTypeId="1" AcceptedAnswerId="13061" CreationDate="2014-07-05T13:23:48.850" Score="1" ViewCount="44" Body="&lt;p&gt;I do have a Diffusion coefficient in a convection diffusion PDE which is discontinuous and looks like (concentration on the x-Axis):&#xA;&lt;img src=&quot;https://i.stack.imgur.com/69fuA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For numerical reasons i use the integrated form:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/BdfQA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I calculate the functions in beforehand and during my calculations i interpolate the diffusion coefficient depending on the actual concentration value on each grid point.&#xA;Note that the function is discontinuous at x = 0.1 and x = 0.5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If i could fit an analytical function to the data i could avoid the interpolation at each time step and may speed up the calculation.  Is that possible and would it make sense ? Which function would make sense ?&#xA;something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myfun = somefunction*((u&amp;gt;=0.1) &amp;amp;&amp;amp; (u&amp;lt;=0.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2014-07-07T09:47:42.093" Title="replace non-smooth discrete values with analytical function" Tags="|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13046" PostTypeId="5" CreationDate="2014-07-05T22:04:38.753" Score="0" Body="&lt;p&gt;More details can be found &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9456" LastEditorUserId="9456" LastEditDate="2014-07-06T22:20:06.470" LastActivityDate="2014-07-06T22:20:06.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13047" PostTypeId="4" CreationDate="2014-07-05T22:04:38.753" Score="0" Body="Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms." OwnerUserId="9456" LastEditorUserId="9456" LastEditDate="2014-07-06T22:20:10.117" LastActivityDate="2014-07-06T22:20:10.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13048" PostTypeId="2" ParentId="13021" CreationDate="2014-07-05T22:21:42.003" Score="4" Body="&lt;p&gt;It sounds like you want the &lt;a href=&quot;http://docs.sympy.org/dev/modules/mpmath/plotting.html&quot; rel=&quot;noreferrer&quot;&gt;multi-precision plot function from SymPy&lt;/a&gt;, which is capable of plotting arbitrary black-box functions over a given range.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;from sympy.mpmath import plot&#xA;plot(lambda x: math.exp(x), [1, 4])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/hcFHH.png&quot; alt=&quot;exponential plot&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2014-07-05T22:21:42.003" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13049" PostTypeId="1" CreationDate="2014-07-06T10:16:46.273" Score="2" ViewCount="357" Body="&lt;p&gt;Is there a good introduction into wavelet Galerkin schemes for numerical partial (and ordinary) differential equations?&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2014-12-03T19:03:58.373" Title="wavelet for numerical partial differential equations" Tags="|pde|ode|wavelet|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13050" PostTypeId="2" ParentId="13026" CreationDate="2014-07-06T14:33:42.440" Score="1" Body="&lt;p&gt;Yes, you can interpolate periodic data using a Fourier transform like you have done. It works best if the periodic function you are trying to interpolate is smooth in the mathematical sense (i.e. all derivatives are continuous). However, this kind of interpolation can be poor if the sample points to do not form a smooth function (as might be true for experimental data). In this case, I think you can use something like &lt;a href=&quot;http://en.wikipedia.org/wiki/Lanczos_resampling&quot; rel=&quot;nofollow&quot;&gt;Lanczos filtering&lt;/a&gt; to improve the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as alternatives, you could also use linear interpolation, cubic spline interpolation, etc. which could also work well depending on the nature of your data and your accuracy requirements. In your case, I think using a Fourier transform based interpolation might be most accurate as it takes advantage of the periodicity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully this is helpful!&lt;/p&gt;&#xA;" OwnerUserId="9601" LastEditorUserId="9601" LastEditDate="2014-07-06T14:50:16.780" LastActivityDate="2014-07-06T14:50:16.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13051" PostTypeId="1" AcceptedAnswerId="14323" CreationDate="2014-07-06T14:35:22.673" Score="3" ViewCount="1129" Body="&lt;p&gt;I would like to perform a surface integration over a portion $D$ of an ellipsoid. A plane arbitrarily intersects the ellipsoid forming two sections, of which one is $D$. I do not know how I can discretize $D$ in order to perform the numerical integration. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Attempt at a Solution&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I parameterize the ellipsoid in spherical coordinates, then for each grid point I check whether that point lies in $D$ or not using the equation of the intersecting plane. If it does, I compute its contribution to a Riemann integral over a rectangular mesh. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Problems with Solution&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Variable mesh size due to parameterization using spherical coordinates (big squares near center, tiny ones near poles).&lt;/li&gt;&#xA;&lt;li&gt;Slow 1st order convergence (not to mention the if loop in python)&lt;/li&gt;&#xA;&lt;li&gt;Inaccuracies near the plane-ellipsoid boundary due to use of square mesh&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;Desired Solution&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I would like to use triangles to approximate the surface. I am aware of a well-developed literature on triangle meshes, but I cannot find how to deal with the irregular domain I have. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that this is a revised version of a question I posted on Math Stack Exchange a week ago:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/851334/numerical-integration-over-two-regions-of-an-ellipsoid&quot;&gt;https://math.stackexchange.com/questions/851334/numerical-integration-over-two-regions-of-an-ellipsoid&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9599" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-08-07T01:54:06.540" Title="Surface integration over a portion of an ellipsoid" Tags="|computational-geometry|discretization|quadrature|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13053" PostTypeId="2" ParentId="13049" CreationDate="2014-07-06T15:18:44.993" Score="-1" Body="&lt;p&gt;Thorsten Raasch has nice notes (well - in German) but also some more literature on his webpage here &lt;a href=&quot;http://www.mathematik.uni-mainz.de/Members/raasch/juniorprof.-dr.-thorsten-raasch?set_language=en&quot; rel=&quot;nofollow&quot;&gt;http://www.mathematik.uni-mainz.de/Members/raasch/juniorprof.-dr.-thorsten-raasch?set_language=en&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2014-07-06T15:18:44.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13054" PostTypeId="2" ParentId="8989" CreationDate="2014-07-06T19:37:07.420" Score="1" Body="&lt;p&gt;I realized this is a bit older, but I though you might be interested in the VisIt solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this in VisIt with something called a Connectively Based Cross Mesh Field Expression. That is a mouthful, but is basically machinery to maps fields between databases (in your case VTK files). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;Connectivity based&quot; (conn_cmfe) is used when the topology is the same between the files -- like in your case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a &quot;Position based&quot; (pos_cmfe) that samples between meshes with different topologies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your case, open the first file &amp;amp; use the Expressions window to define an expression (MyPhi_Diff) :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Phi - conn_cmfe(&amp;lt;file2.vtk:Phi&amp;gt;, mesh)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can plot &quot;MyPhi_Diff&quot; with a Pseudocolor plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also wizard that you can use to help define the expression&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Options Menu-&gt;&quot;Data Level Comparisons&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some more info:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://visitusers.org/index.php?title=Cmfe&quot; rel=&quot;nofollow&quot;&gt;http://visitusers.org/index.php?title=Cmfe&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="766" LastActivityDate="2014-07-06T19:37:07.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13055" PostTypeId="2" ParentId="11609" CreationDate="2014-07-06T22:16:30.083" Score="0" Body="&lt;p&gt;A friend of mine is just about to finish this book which covers many ways of optimising python scripts to achieve high performance speeds:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://shop.oreilly.com/product/0636920028963.do&quot; rel=&quot;nofollow&quot;&gt;http://shop.oreilly.com/product/0636920028963.do&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8055" LastActivityDate="2014-07-06T22:16:30.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13056" PostTypeId="1" AcceptedAnswerId="13064" CreationDate="2014-07-07T01:28:29.990" Score="1" ViewCount="131" Body="&lt;p&gt;I have a working knowledge of calculus and have been able to understand the application of &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow&quot;&gt;Newton-Raphson&lt;/a&gt; technique for unconstrained optimization. Please point me to some of the easy to understand tutorials for advanced techniques like BFGS and LBFGS. I prefer video lectures to books and books to papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My aim is to understand how these methods work and code some of them (python/scala/Theano) for better understanding. My area of interest is application of these techniques rather than proofs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[this was originally posted in SO where I was suggested to post in this site]&lt;/p&gt;&#xA;" OwnerUserId="9640" LastActivityDate="2014-07-07T14:19:35.983" Title="Unconstrained optimization learning and programming resources" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13057" PostTypeId="1" AcceptedAnswerId="13067" CreationDate="2014-07-07T01:50:14.483" Score="4" ViewCount="501" Body="&lt;p&gt;In a constrained optimization problem, I found in a paper a way to define new variables such that the constraints disappear. They only give the new variable definitions, and I would like to understand how they did.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The context is thermodynamics. A mixture of $c$ components can phase separates into $p$ phases.&#xA;$z_i$ is the total mole number of component $i$ and $n_{ij}$ is the number of moles of component $i$ in the phase $j$. The free energy $G(\{n_{ij}\}_{ij})$ is known. The problem is to find the composition $\{n_{ij}\}_{ij}$ that minimizes the free energy $G$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mass conservation gives the constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{j} n_{ij} = z_i$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0 \leq n_{ij} \leq z_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this, the paper defines the new variables $\beta_{ij} $&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;$n_{i1} = \beta_{i1} z_i ~~~~~~~~~~~~i=1,...,c$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{ij} = \beta_{ij} (z_i - \sum_{m=1}^{j-1} n_{im}) ~~~~~~~~~ i=1,...,c ~~~~~~~~ j=2,...,p-1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{ip}=z_i - \sum_{m=1}^{p-1}n_{im} ~~~~~~~ i=1,...,c$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;with $\beta_{ij} \in [0:1]$ and the problem is know unconstrained&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help, direction, that will get me to understand how the authors achieve this variable change would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem source: &lt;em&gt;Srinivas, Rangaiah,&#xA;&quot;Differential Evolution with Tabu List for Global Optimization and Its Application to Phase Equilibrium and Parameter Estimation Problems.&quot;&#xA;Ind. Eng. Chem. Res. 2007, 46, 3410-3421&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="9639" LastEditorUserId="9639" LastEditDate="2014-07-07T01:55:15.170" LastActivityDate="2014-07-07T16:23:48.877" Title="Transform from constrained to unconstrained optimization" Tags="|constrained-optimization|computational-physics|constraints|" AnswerCount="1" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13059" PostTypeId="2" ParentId="13056" CreationDate="2014-07-07T09:35:32.260" Score="1" Body="&lt;p&gt;I don't know whether there are video lectures, but I find the book by Nocedal and Wright on &quot;Numerical Optimization&quot; to be a fantastic reference to everything related to continuous optimization. It is easy to read and understand.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-07T09:35:32.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13060" PostTypeId="2" ParentId="13051" CreationDate="2014-07-07T09:43:27.667" Score="0" Body="&lt;p&gt;Let's assume for a moment that your plane intersects the ellipsoid in such a way that less than one half of the surface of the ellipsoid forms your domain $D$. Then it should not be very difficult to find a transformation that maps a half-sphere (that is, one half of the surface of a ball) to your domain $D$. Now, we know how to triangulate a half-sphere: with triangles, use one half of the 8 triangles that form an octahedron; with quadrilaterals, take the six sides of a cube and refine them into 24 smaller quadrilaterals by pulling the mid-points out to the surface, then take one half of them for the half-sphere. In either of these cases, you will end up with triangles or quadrilaterals that are all of the same size. To mesh $D$, you then just have to map these triangles or quadrilaterals from the half-sphere to $D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case your plane intersects the ellipsoid in a way that $D$ forms &lt;em&gt;more&lt;/em&gt; than one half of the surface, one can employ a similar technique.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-07T09:43:27.667" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13061" PostTypeId="2" ParentId="13045" CreationDate="2014-07-07T09:47:42.093" Score="1" Body="&lt;p&gt;If your data is available on a uniform grid, you are unlikely to accelerate your computations: all you currently have to do is a single division $i=\lfloor x/\Delta x \rfloor$ to find which interval you're in, and a linear interpolation between data point $i$ and $i+1$. Pretty much any function that is not a low-order polynomial will cost far more than these operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you insist, to fit a function to your data it would be helpful where it comes from. The curve looks like a Gaussian, but whether it really is of course depends on what exactly it models.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-07T09:47:42.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13062" PostTypeId="2" ParentId="13041" CreationDate="2014-07-07T09:53:42.183" Score="3" Body="&lt;p&gt;If your question is whether you can compute $\Delta u|_{\partial \Omega}$ just from $u|_{\partial \Omega}$ and $\partial_n u|_{\partial \Omega}$ without solving the actual equation, then the answer is &lt;em&gt;no&lt;/em&gt;. What you are trying to do is the equivalent of computing the Dirichlet-to-Neumann map for the Laplace equation, which also requires you to compute the solution of the Laplace equation. It is not possible to compute $\Delta u|_{\partial \Omega}$ just from $u|_{\partial \Omega}$ and $\partial_n u|_{\partial \Omega}$ purely &lt;em&gt;locally&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-07T09:53:42.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13063" PostTypeId="1" AcceptedAnswerId="13073" CreationDate="2014-07-07T12:23:41.283" Score="3" ViewCount="886" Body="&lt;p&gt;I'm a novice in object-oriented programming, but this design technique seems very interesting for me. I have a Fortran 90 CFD package , not very complex, but it is already very hard to maintain and to add new physics or other features. As I learnt for now it can reduce complexity of program, but most important from my point of view - intrinsic object orientation of mathematical abstractions. Now I'm trying to draw some parallels between CFD mathematics and object-oriented concepts introduced in Fortran 2003 standard. CFD begins with introducing &lt;strong&gt;field&lt;/strong&gt; abstraction. In most general case it is three-dimensional tensor field, which can be described as five-dimensional array &lt;strong&gt;tensor_field(:,:,:,:,:)&lt;/strong&gt;. That array stores nodal values of the tensor field. Similarly , scalar field &lt;strong&gt;scalar_field(:,:,:)&lt;/strong&gt; and &lt;strong&gt;vector_field(:,:,:,:)&lt;/strong&gt;. For now everything is clear and straightforward. But when one starts to try to develop classes for this field types it becomes tricky. There are some questions that arise :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If that arrays are parts of corresponding classes than how to implement &lt;strong&gt;grad&lt;/strong&gt; and &lt;strong&gt;div&lt;/strong&gt; operations ? In elegant mathematical way &lt;strong&gt;grad&lt;/strong&gt; should be procedure in &lt;strong&gt;scalar_field&lt;/strong&gt; class , that gives &lt;strong&gt;vector_field&lt;/strong&gt; as its result. But similarly , &lt;strong&gt;div&lt;/strong&gt; should be in &lt;strong&gt;vector_field&lt;/strong&gt; class and should result &lt;strong&gt;scalar_field&lt;/strong&gt;. How can be achieved such interactions between objects in object oriented paradigm ?&lt;/li&gt;&#xA;&lt;li&gt;From mathematical point of view &lt;strong&gt;tensor_field&lt;/strong&gt; should be parent class to vector and scalar classes. But it seems a bit overwhelming to use only five-dimensional arrays. I thought about using scalar class as a parental, but it seems more like an element of vector class, not a parent. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To conclude, the main idea of my questions is : are there any object-oriented design pattens for CFD programming , that are wide acceptable and tested ? Maybe you can share here yours implementations in pseudo code, in Fortran or in C++ to discuss them ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards, Ivan Y. &lt;/p&gt;&#xA;" OwnerUserId="4084" LastActivityDate="2014-07-08T11:56:03.800" Title="CFD absractions in Modern Fortran" Tags="|fluid-dynamics|fortran|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13064" PostTypeId="2" ParentId="13056" CreationDate="2014-07-07T14:19:35.983" Score="2" Body="&lt;p&gt;Boyd and Vandenberghe's text on Convex Optimization is free, and lectures have been posted online. It's also not incredibly theory heavy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nocedal and Wright is very good at discussing practical aspects of optimization, particularly implementation of algorithms. It too is not theory heavy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For people who care about theory, Bertsekas' book on Nonlinear Programming is a great read. Nocedal's book is better about discussing practical aspects of algorithms, but you could certainly use Bertsekas' book to implement classical optimization algorithms. To understand how the algorithms work at a deep level, understanding the convergence proofs is helpful; however, it requires an understanding of real analysis. I don't think a deep understanding is required to use these algorithms in practice.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-07T14:19:35.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13065" PostTypeId="2" ParentId="8468" CreationDate="2014-07-07T16:09:09.510" Score="1" Body="&lt;p&gt;I think the most powerful use of python scripting are programmable filters, rather than python shells, since they allow to create objects in the pipeline (and your script will be processed by the server, not the client). You can find some examples in &lt;a href=&quot;http://www.paraview.org/Wiki/ParaView/Simple_ParaView_3_Python_Filters&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.paraview.org/Wiki/ParaView/Simple_ParaView_3_Python_Filters&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to try those script in the python shell, you have just to modify the lines referring to &quot;self&quot; . For example, pdi = self.GetInput()  should become something like pdi = servermanager.Fetch(FindSource(&quot;sphere1&quot;)) . I'm still not sure on the proper way to export data produced in that way (see my temptatives  &lt;a href=&quot;https://stackoverflow.com/a/24273259/1136458&quot;&gt;https://stackoverflow.com/a/24273259/1136458&lt;/a&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also save the current state as a python file, instead of pvsm, and check what it has been done there (even if it's not 100% accurate, for example I had problems with some properties of the spreadsheet representation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't need the rendering, but just load some file, process them and save them, you can have a very quick script: &#xA; - create your pipeline in paraview GUI&lt;br&gt;&#xA; - export state as py&lt;br&gt;&#xA; - open the python file and delete all the representation and data&#xA;   visualization code (only if you are not interested in those, of course)&lt;br&gt;&#xA; - for exporting a polydata, set it as ActiveSource, then use something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vtkWriter = DataSetWriter(FileName = 'Surface.vtk' , Input = GetActiveSource(), FileType = 1) &#xA;vtkWriter.UpdatePipeline() &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;- run it multiple times just changing some parameters! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already mentioned by andybauer, you can find the documentation here &lt;a href=&quot;http://www.paraview.org/ParaView3/Doc/Nightly/www/py-doc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.paraview.org/ParaView3/Doc/Nightly/www/py-doc/&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="9647" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-07-07T17:13:52.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13066" PostTypeId="2" ParentId="8884" CreationDate="2014-07-07T16:16:46.017" Score="1" Body="&lt;p&gt;In Paraview you can try the following :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Delaunay2D (for creating a surface - see &lt;a href=&quot;http://www.vtk.org/Wiki/VTK/Examples/Python/Filtering/Delaunay2D&quot; rel=&quot;nofollow&quot;&gt;http://www.vtk.org/Wiki/VTK/Examples/Python/Filtering/Delaunay2D&lt;/a&gt; )&lt;/li&gt;&#xA;&lt;li&gt;Smooth &lt;/li&gt;&#xA;&lt;li&gt;LoopSubdivision (will also create new cells / new points) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9647" LastActivityDate="2014-07-07T16:16:46.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13067" PostTypeId="2" ParentId="13057" CreationDate="2014-07-07T16:23:48.877" Score="0" Body="&lt;p&gt;Ok I got it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take a simple case, with only 3 components and 2 phases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{ij}$ is the number of moles of components i in phase j.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z_i$ is the total number of moles of component i.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We must satisfy the mass conservation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{11}+n_{12}+n_{13} = z_1$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{21}+n_{22}+n_{23} = z_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{31}+n_{32}+n_{33} = z_3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's fill the phases, component by component. We start with the component 1 in the phase 1. The minimum quantity we can put is obviously 0, and the maximum quantity is the total number of mole of component 1 : $z_1$. Therefore one can define the variable $\beta_{11}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{11}=\beta_{11} z_1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And $\beta_{11}$ is free to range between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we still use the component 1, and we want to fill the phase 2. The minimum quantity 0, and the maximum quantity is the total number of mole of component 1, minus the amount we already put in phase 1 : ($z_1-n_{11}$). So we define $\beta_{12}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{12}=\beta_{12} (z_1-n_{11})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\beta_{12}$ is also free to range between 0 and 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we want to fill the 3rd phase with component 1, and there is only one possibility : the remaining quantity of component 1 ($z_1-n_{11}-n_{12}$) and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n_{13}=z_1-n_{11}-n_{12}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And so on.....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then arrive to the general equations.&lt;/p&gt;&#xA;" OwnerUserId="9639" LastActivityDate="2014-07-07T16:23:48.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13068" PostTypeId="2" ParentId="13063" CreationDate="2014-07-07T17:04:48.997" Score="2" Body="&lt;p&gt;You can see the book by Damian Rousen (Scientific Software Design: The Object-Oriented Way). He uses Fortran 2003 extensively and is one of the author of the Trilinos project. You can also see section 3 &lt;a href=&quot;http://jolts.stanford.edu/72/modern_fortran_for_scientists_and_engineers&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; but the book is better and available &lt;a href=&quot;http://books.google.com/books/about/Scientific_Software_Design.html?id=xedE5KkHVn4C&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2014-07-07T17:04:48.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13069" PostTypeId="1" AcceptedAnswerId="13072" CreationDate="2014-07-08T03:27:42.323" Score="3" ViewCount="1507" Body="&lt;p&gt;I am trying to interpolate irregular data $f(\theta, \phi)$ on a sphere and I have so far tried a scipy approach using Kd-Trees and inverse distance weighting, which works ok - however I was wondering if there is an open-source library for spherical interpolation.&#xA;I have come across the &lt;a href=&quot;http://users.isterre.fr/nschaeff/SHTns/&quot; rel=&quot;noreferrer&quot;&gt;SHTns&lt;/a&gt; library, which is suggested to be useful, but it's not immediately clear to me how to use it for interpolation purposes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if someone could please shed some light on this. Many thanks.&lt;/p&gt;&#xA;" OwnerUserId="8245" LastActivityDate="2014-07-08T11:42:33.047" Title="Interpolating irregular data on a sphere" Tags="|interpolation|spherical-harmonics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13070" PostTypeId="1" CreationDate="2014-07-08T07:31:21.263" Score="1" ViewCount="106" Body="&lt;p&gt;For my application, I need factors $\tilde C$ so that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde C{}^T \tilde C = C^TMC&#xA;$$&#xA;where $C$ is a long matrix, i.e. $C$ has much more columns than rows, and $M$ is a small symmetric positive definite (mass) matrix. Since my code assembles $MC$, I will also have to use some inverses of $M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found that with the Cholesky factorization $M=L^TL$ and $\tilde C = LC$ the relative difference in the Frobenius norm&#xA;$$&#xA;\|LM^{-1}MC - L^{-T}MC\|~/~\|MC\|&#xA;$$&#xA;is of order $10^{-4}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Is there a better factorization for this purpose?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I basically use the &lt;code&gt;scipy.inv&lt;/code&gt; and &lt;code&gt;scipy.cholesky&lt;/code&gt; routines of Python's scipy module.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2014-07-09T08:08:46.053" LastActivityDate="2014-07-09T08:08:46.053" Title="Conditioning of matrix factorizations and square roots" Tags="|linear-algebra|conditioning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13071" PostTypeId="2" ParentId="13069" CreationDate="2014-07-08T07:38:07.570" Score="1" Body="&lt;p&gt;On a basic search on Spherical Interpolation I found these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Graphics Math Template Library (GMTL)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Link: &lt;a href=&quot;http://ggt.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;http://ggt.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;How do I perform spherical interpolation with quaternions?&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Use the slerp function. You need an origin quaternion, a target quaternion, and an interpolation amount between 0 and 1. The following example interpolated halfway between the origin and target quaternions. Note that the interpolation path follows the shortest length arc around a sphere.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  gmtl::Quatf resultQuat, originQuat, targetQuat;&#xA;  float amount = 0.5;&#xA;&#xA;  gmtl::slerp( resultQuat, amount, originQuat, targetQuat );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Source: &lt;a href=&quot;http://ggt.sourceforge.net/html/gmtlfaq.html&quot; rel=&quot;nofollow&quot;&gt;http://ggt.sourceforge.net/html/gmtlfaq.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Example Code for Spherical Interpolation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Project Link: &lt;a href=&quot;http://freesourcecode.net/cprojects/2275/Piecewise-Linear-Image-Denoising&quot; rel=&quot;nofollow&quot;&gt;http://freesourcecode.net/cprojects/2275/Piecewise-Linear-Image-Denoising&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++ Link: &lt;a href=&quot;http://freesourcecode.net/cprojects/2275/sourcecode/quaternion_demo.cpp&quot; rel=&quot;nofollow&quot;&gt;http://freesourcecode.net/cprojects/2275/sourcecode/quaternion_demo.cpp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;H Link: &lt;a href=&quot;http://freesourcecode.net/cprojects/2275/sourcecode/quaternion_demo.h&quot; rel=&quot;nofollow&quot;&gt;http://freesourcecode.net/cprojects/2275/sourcecode/quaternion_demo.h&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;OpenGLDemoEngine:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Spherical Interpolation: &lt;a href=&quot;http://gitlab.scss.tcd.ie/gv2/opengldemoengine/blob/bc0b7ab4f7e4fc287dd7905353bf4a783c05813d/Dependencies/assimp--3.0.1270/include/assimp/quaternion.h&quot; rel=&quot;nofollow&quot;&gt;http://gitlab.scss.tcd.ie/gv2/opengldemoengine/blob/bc0b7ab4f7e4fc287dd7905353bf4a783c05813d/Dependencies/assimp--3.0.1270/include/assimp/quaternion.h&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LINK: &lt;a href=&quot;http://gitlab.scss.tcd.ie/gv2/opengldemoengine&quot; rel=&quot;nofollow&quot;&gt;http://gitlab.scss.tcd.ie/gv2/opengldemoengine&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9499" LastActivityDate="2014-07-08T07:38:07.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13072" PostTypeId="2" ParentId="13069" CreationDate="2014-07-08T11:42:33.047" Score="1" Body="&lt;p&gt;After some more searching I've managed to find what I was looking for. They are available on NETLIB as stripack and ssrfpack - fortran routines - that allow for spherical interpolation of irregular data using spherical splines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also an alternative that uses &lt;a href=&quot;http://www.soest.hawaii.edu/wessel/sphspline/&quot; rel=&quot;nofollow&quot;&gt;generalized Green's function for spherical surface splines in tension&lt;/a&gt;. The matlab code (and the associated paper) is available there, although it has GMT as a dependency.&lt;/p&gt;&#xA;" OwnerUserId="8245" LastActivityDate="2014-07-08T11:42:33.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13073" PostTypeId="2" ParentId="13063" CreationDate="2014-07-08T11:56:03.800" Score="3" Body="&lt;p&gt;I don't know if there are design patterns that are valid in the whole CFD field, because there are a lot of different methods available to solve this kind of problem (PDE based, particle based,...). To have concepts that make sense for all these methods, they would probably need to be extremely general, more than what you wrote: a scalar field would be something (probably a method of an object) taking coordinates in input and returning a scalar value (the same for vector/tensor field, with different return types). All this without speaking about &quot;node&quot; or anything related to the method used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you restrict yourself to a particular kind of method (e.g. node based), you can more specify your interface in the way you do. If you manage to stick with this kind of interface, you will end up with a very powerful framework which can allow you to treat a wide variety of problems. The first thing I would however change in the interface that you propose is to hide the &quot;array&quot; part, use instead a virtual (abstract) method: you might want to use an analytical function, in which case you probably do not need to store the values of the function in all the nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I coded a few years ago a similar framework for finite elements, which was quite flexible with the definition of scalar/vector/tensor fields: anything that could give a value (scalar/vector/tensor) for a given quadrature node (~position) could be used in the framework. Non-linearity, exact solutions, coulings between equations... could all be mixed and used very easily in the end. I wrote that in C++ and the code can still be found &lt;a href=&quot;https://github.com/lifev/lifev/tree/master/lifev/eta&quot; rel=&quot;nofollow&quot;&gt;in the ETA module of the LifeV library&lt;/a&gt;. It is quite a tough machinery because it relies heavily on C++ template metaprogramming (for performance reasons), but the user interface was very simple and the same could be achieved with simpler code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your two particular questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I would create functions for div and grad outside the field classes, so that there are no problem with circular dependencies. You just write grad(my_field) instead of my_field.grad()&lt;/li&gt;&#xA;&lt;li&gt;I would keep the classes scalar/vector/tensor field separated. There are interfaces that make sense only for scalars and not for vectors, and vice versa. If you need a base class for all the fields, do not specify the storage (array) in the mother class but keep it for particular implementations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;[Disclaimer: I used to be a developer of the LifeV library]&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2014-07-08T11:56:03.800" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13074" PostTypeId="1" AcceptedAnswerId="13075" CreationDate="2014-07-08T14:33:48.547" Score="4" ViewCount="2551" Body="&lt;p&gt;I'm working on a code that we eventually plan to scale up to 10's or 100's of thousands of processors (using MPI). I have a single HDF5 file (mesh nodes and connectivity, etc) that needs to be read and distributed to every process with identical data. This is only done once during the setup part of the code. The current solution is to have each process independently read the file, but my collaborator is worried this will not scale well and suggests having process 0 read the file then MPI distribute the data to the other processors. So I wanted to get an outside opinion from someone with more extensive HPC experience. Is this actually going to be a significant bottleneck?&lt;/p&gt;&#xA;" OwnerUserId="3309" LastActivityDate="2014-07-09T20:16:20.903" Title="Many processors reading a single HDF5 file" Tags="|hpc|io|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13075" PostTypeId="2" ParentId="13074" CreationDate="2014-07-08T14:59:46.247" Score="6" Body="&lt;p&gt;It will not scale! Use the &lt;a href=&quot;http://www.hdfgroup.org/HDF5/PHDF5/&quot;&gt;parallel HDF5&lt;/a&gt; functionality to read it in parallel with a relatively small number of readers, or do what your colleague suggests. There will be tradeoffs with both methods, and you will have to do some tuning. It depends pretty strongly on how big the file is, how many tasks you have reading it, what the underlying filesystem is (GPFS, Lustre, etc), how the file is striped, and what the underlying disk infrastructure is. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-07-08T14:59:46.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13076" PostTypeId="2" ParentId="13074" CreationDate="2014-07-08T19:26:00.333" Score="1" Body="&lt;p&gt;If your question was &lt;em&gt;&quot;Is it wise to distribute/replicate data among $N$ processors of a big cluster using a distributed file system&quot;&lt;/em&gt; I would answer fore sure not! (As already pointed out by Bill in another &lt;a href=&quot;https://scicomp.stackexchange.com/a/13075/1744&quot;&gt;answer&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GPFS, Lustre, or NFS are not meant for a substitute of efficient memory/communication organisation in a distributed memory/MPI programming paradigm. But my practical answer is: &lt;em&gt;why bother now&lt;/em&gt;? For sure the asymptotics are against re-reading the data on each processor, but if this becomes a real bottleneck can be determined only by experiment or a more thorough analysis, for which we do not have enough data from your post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by Stali in a comment, maybe you hit before a memory limit, or the data reading time remains negligible in your real-world application, where $N$ may be very big, but still bounded. Maybe it's the application that doesn't scale so well...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My 1 cent worth opinion: optimise only when your profiling/timing reveals that you have a real bottle neck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Somehow, against my will, my answer has started a comment war. Just to be clear: I totally agree that it is foolish to let $N\approx 10^4 \div 10^5$ processes read the very same file at the very same time. But when the OP says &lt;em&gt;&quot;we eventually plan to scale up to 10's or 100's of thousands of processors&quot;&lt;/em&gt; I understand that &lt;em&gt;&quot;eventually&quot;&lt;/em&gt; means &quot;at an unspecified later time&quot;. Before hitting these extreme numbers, there is a lot of work that can be done with more simple (albeit inefficient or even banausic) approaches. Just to put things in the right perspective: in the June 2014 top 500 list the median of the &quot;total cores&quot; column is 19280, so running with $N=10^5$ means that you have access to a significative portion of a really a big machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tuning parallel I/O requires a lot of work, so my answers boils down to the very old advice: &quot;avoid premature optimisation&quot;... but here the risk of starting another war is even greater.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Final note: please, do not be afraid of down voting this post, if you think that it is pointless.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-07-09T20:16:20.903" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="13077" PostTypeId="2" ParentId="13070" CreationDate="2014-07-08T21:03:33.133" Score="1" Body="&lt;p&gt;If I understand your question correctly you are looking for a factorisation $$\tilde{C}^T \tilde{C} = C^TMC,$$ where $M$ is symmetric positive definite. Moreover you have at hand $M$ and $B=MC$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $M$ you can compute Cholesky, LDLT, and eigenvalue decompositions:&#xA;$$M = L_1 L_1^T = L_2 D_2 L_2^T = Q \Lambda Q^T,$$ where $L$ are lower triangular, $Q$ orthogonal, $D_2$ and $\Lambda$ diagonal and positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accordingly you can define and compute $\tilde C$ as&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;$\tilde C = L_1^TC$, to be computed as $\tilde C = L_1^{-1} B$, where $L_1^{-1}\cdot$ is a back-substitution on the columns of $B$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$\tilde C = D^{\frac 12} L_2^TC$, to be computed as $\tilde C = D^{-\frac12}L_2^{-1} B$. Here we have back-substitution followed by row-scaling.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$\tilde C = Q\Lambda^{\frac12}Q^T C$, to be computed as $\tilde{C} = Q\Lambda^{-\frac12}Q^T B$. Different approches are possible, but maybe computing $M^{-\frac12}B$, with $M^{-\frac12} = Q\Lambda^{-\frac12}Q^T$ is advisable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It is hard to guess which approach is best, without more information on $M$ and it's properties, but I would first try approach 2., which could be more stable, for ill-conditioned $M$.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2014-07-08T21:03:33.133" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="13079" PostTypeId="1" AcceptedAnswerId="13081" CreationDate="2014-07-09T01:31:41.220" Score="7" ViewCount="144" Body="&lt;p&gt;I am trying to replicate Fortran90 array syntax using a C++ library.  The libraries themselves are discussed at length in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/351/recommendations-for-a-usable-fast-c-matrix-library&quot;&gt;this question&lt;/a&gt;. They can all do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;D = alpha*A + beta*B + gamma*C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for vectors A, B, C and scalars alpha, beta, gamma.  This could be translated as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i = 0; i &amp;lt; D.length; i++)&#xA;  D[i] = alpha*A[i] + beta*B[i] + gamma*C[i]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which traverses each array a single time, or like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i = 0; i &amp;lt; D.length; i++)&#xA;  D[i] = alpha*A[i] + beta*B[i]&#xA;for (i = 0; i &amp;lt; D.length; i++)&#xA;  D[i] += gamma*C[i]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first way is much better for cache performance, and I assume is what fortran compilers do.  Are C++ matrix libraries able to translate compound vector expressions into single loops like fortran, or will they emit multiple simple loops?&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-07-09T06:33:55.580" Title="Do C++ matrix libraries translate compound vector operations to single loops?" Tags="|linear-algebra|software|c++|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13080" PostTypeId="2" ParentId="11568" CreationDate="2014-07-09T02:26:18.413" Score="1" Body="&lt;p&gt;This is a very interesting problem.  If I understand right, you are simulating a body of water in several PCs at the same time.  They all start at the same time and if left alone they should get the same result after some time.  However, your user will create a disturbance (associated with your game I presume) in one of the PCs and then that should be transmitted to the other PCs so they can see the disturbance too.&#xA;Due to the lag in the transmission of info, the other computers will be out of sync if they use the disturbance when they receive it instead of at the time it was created in the original PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your idea is that maybe the data from the PC that caused the disturbance can be weaved into the results of the other PCs to correct the result for the present.  A potential implementation of that could be: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to run your predictive model faster than real time but then you show the results in real time.  So you have one process in the background that runs the next 10 second in 1 second for instance.  Then when something happens in one PC you send the info to the others with the real time stamp.  The other computers will get this info and run the predictive model starting just before the time stamp from the disturbance (which is before the current time).  Since you're running this 10 or more times faster you can actually know the result at the current time including the disturbance data from the other PCs and show that to the player.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the kind of solution your looking for?&lt;/p&gt;&#xA;" OwnerUserId="9659" LastActivityDate="2014-07-09T02:26:18.413" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="13081" PostTypeId="2" ParentId="13079" CreationDate="2014-07-09T06:33:55.580" Score="5" Body="&lt;p&gt;This is possible in C++ via expression templates. Section 1.9 of this &lt;a href=&quot;http://www.cs.indiana.edu/cgi-bin/techreports/TRNNN.cgi?trnum=TR542&quot;&gt;technical report&lt;/a&gt; addresses your question.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2014-07-09T06:33:55.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13082" PostTypeId="1" CreationDate="2014-07-09T07:18:10.557" Score="0" ViewCount="3148" Body="&lt;p&gt;I deal with PDE module in COMSOL and I would like to simulate Poisson's equation. Actually I need to simulate the distribution of the electric filed created by the spherical probe in the   material, hence I need to consider at least two different domains: air and material. Air and material have different properties, so that I need to create a two seperate PDE modules for air and material respectively. I faced with a problem that I don't know how to specify continuity condition on the boundary between the air and material. Comsol allows me to specify only one boundary condition, however I need to specify two:$$\phi_{air}=\phi_{mat}$$ $$\epsilon_1\frac{\partial \phi_{air}}{\partial z}=\epsilon_2\frac{\partial \phi_{mat}}{\partial z}$$I know how to specify one condition but not two. Could anyone suggest me how I can specify both those conditions on the boundary in COMSOL?   &lt;/p&gt;&#xA;" OwnerUserId="5278" LastActivityDate="2014-07-09T08:44:03.750" Title="boundary condition in COMSOL" Tags="|comsol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13083" PostTypeId="1" CreationDate="2014-07-09T07:34:38.540" Score="4" ViewCount="248" Body="&lt;p&gt;What are the subtleties involved in solving vector PDEs on manifolds? Can someone suggest a reference summarizing the problems involved?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically I want to solve a vector Helmholtz equation with a source field on a curved manifold (e.g. a sphere, or a sphere with distortions). How do I go about solving this? Say using finite elements? Is there any such study available?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I must clarify that I am looking for methods to solve vector PDEs on general surfaces which are topologically equivalent to the sphere (e.g., ellipsoid, sphere + spherical harmonic deformations...). The deformations away from the sphere are &lt;em&gt;not&lt;/em&gt; infinitesimal but finite.&lt;/p&gt;&#xA;" OwnerUserId="1285" LastEditorUserId="1285" LastEditDate="2014-07-09T19:18:38.470" LastActivityDate="2014-07-12T11:23:27.967" Title="vector PDEs on manifolds" Tags="|pde|finite-element|fluid-dynamics|reference-request|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13084" PostTypeId="2" ParentId="13082" CreationDate="2014-07-09T08:44:03.750" Score="2" Body="&lt;p&gt;I don't think you are restricted to only one boundary condition. You can specify boundary conditions through constraint, thereby you can set as many as you need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, your problem seems to fit AC/DC module in COMSOL. Why not try it? &lt;/p&gt;&#xA;" OwnerUserId="8265" LastActivityDate="2014-07-09T08:44:03.750" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="13085" PostTypeId="1" AcceptedAnswerId="13087" CreationDate="2014-07-09T09:21:41.593" Score="1" ViewCount="421" Body="&lt;p&gt;I've currently begun studying Finite Element Method (FEM) and I'm finding it a little difficult to find resources that break it down into something comprehensible. All the resources I've found are simply too advanced for me to understand as I do not have a mathematics background (hence the confusion). I'd like to kindly request the community's advice on resources, preferably online and free, for a proper introduction to FEM, as well as other resources with more detailed information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9506" LastActivityDate="2014-07-09T15:34:28.330" Title="Introductory Resources on FEM" Tags="|finite-element|" AnswerCount="3" CommentCount="3" ClosedDate="2014-07-09T15:57:46.663" ContentLicense="CC BY-SA 3.0" />
  <row Id="13086" PostTypeId="2" ParentId="13083" CreationDate="2014-07-09T10:20:16.767" Score="3" Body="&lt;p&gt;You should browse the &lt;a href=&quot;http://www.igpm.rwth-aachen.de/personen/reusken/publications&quot; rel=&quot;nofollow&quot;&gt;recent publications&lt;/a&gt; by Arnold Reusken from Aachen, Germany. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;He works on solving PDEs on surfaces. I am sure you can apply the methods developed there to the case where the surface is pregiven and even parametrized.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-07-09T10:20:16.767" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="13087" PostTypeId="2" ParentId="13085" CreationDate="2014-07-09T14:03:09.580" Score="2" Body="&lt;p&gt;I had the same problem you faced a number of years ago... I found lots and lots of resources online about the finite element method that were way too mathematically advanced for me at that time.  What I eventually discovered is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are really two ways to approach the study of the finite element method:  the engineering approach and the mathematical approach.  Both are valid and arrive at the same conclusion, but have very different starting points.  The mathematical approach requires a lot more background in functional analysis, measure theory, variational calculus, etc...  It can be very daunting to learn it from a mathematical perspective if you don't already have this background.  The engineering perspective uses principles such as virtual work and superposition, and 'shape functions', concepts that are probably more intuitive for a computer scientist's perspective such as yourself.  Particularly, the so-called &quot;Direct Stiffness Method&quot; is the simplest approach for beginners in engineering.  While the examples tend to be skewed towards mechanical engineering applications such as structural deformation problems, the direct stiffness method can also be used to study problems such as resistor network problems (which are somewhat related to computer science via electrical engineering).  It also provides insight into a more generalized approach using the weak formulation and galerkin projection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the FEM books that I've read with an engineering emphasis tend to focus on HOW it works, and place very little focus on WHY it works.  To gain some insight into why it works, you will need some mathematical tools from functional analysis.  While it's now out of print, I found the book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0894645854&quot; rel=&quot;nofollow&quot;&gt;&quot;Applied Functional Analysis and Variational Methods in Engineering&quot; by Reddy&lt;/a&gt; to be particularly insightful and enlightening because it filled in all the &quot;mathematical gaps&quot; from an engineer's persective.  Also, &lt;a href=&quot;http://www.math.mtu.edu/~msgocken/fembook/index.html&quot; rel=&quot;nofollow&quot;&gt;Mark Gockenbach's &quot;Understanding and Implementing the Finite Element Method&quot;&lt;/a&gt; was particularly helpful to me to understand the convergence theory behind the method and provides nice outline into data structures and matlab code to implement the method.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I don't know of any particularly useful online resources, I hope I've at least given you some perspective into how to find good resources from a beginner's perspective.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-07-09T14:03:09.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13088" PostTypeId="2" ParentId="13085" CreationDate="2014-07-09T15:31:01.283" Score="0" Body="&lt;p&gt;If you're not planning to write your own code then why bother learning it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FEM is a means to an end, i.e., a solver for PDEs. It does the same job as any other mesh or particle based method. 15-20 years ago it was somewhat necessary for people to know about FEM in order to use FE software effectively. These days anyone (e.g. a freshman) can use it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example is Comsol which is also marketed as a generalized PDE solver. It indeed uses the FEM internally but hides all the complicated FE related details (including generation of mesh) from the user. In fact their product &lt;a href=&quot;http://www.comsol.com/products&quot; rel=&quot;nofollow&quot;&gt;webpage&lt;/a&gt; doesn't even mention the word &quot;finite element&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if you still want to learn about the finite elements and don't have the required math background then look at books that emphasize implementation. One nice thing about computers is that they only understand numbers which everyone understands easily. Two examples of such books are &quot;The Finite Element Method Using MATLAB by Kwon and Bang&quot; and &quot;Programming the Finite Element Method, 5th Edition by Smith et al&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I would highly recommend (if you have the time) to learn the math behind finite elements. A good start will be to take the FE course offered by your Math department.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2014-07-09T15:31:01.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13089" PostTypeId="2" ParentId="13085" CreationDate="2014-07-09T15:34:28.330" Score="0" Body="&lt;p&gt;Carlos Felippa at the University of Colorado, Boulder has the notes from his Introduction to Finite Element Methods course online:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/Home.html&quot; rel=&quot;nofollow&quot;&gt;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/Home.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This course is definitely taught from a mechanical engineering perspective and emphasizes structural analysis and elasticity. He also has notes online for two more advanced finite element courses you might want to investigate later.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-07-09T15:34:28.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13090" PostTypeId="1" CreationDate="2014-07-09T16:21:34.047" Score="1" ViewCount="176" Body="&lt;pre&gt;&lt;code&gt;    function [R, RI , S ] = Matrix_block (A,N)&#xA;[U,D]= schur (A);&#xA;E=ordeig(D) ;&#xA;[R, S]= ordschur (U,D, abs (E) &amp;lt;1000* eps ) ;&#xA;RI=R';&#xA; m= max(max(R* RI-eye (2*N) ) ) ;&#xA;if m &amp;gt;100* eps;&#xA; sprintf( 'R not unitary , N: %d , Maximum : %d \n ' ,N,m);&#xA; end&#xA;C = R*S*RI-A;&#xA;m= max(max( abs (C) ) ) ;&#xA;if m &amp;gt;10* eps ;&#xA;sprintf ('Rtransformeriet , N: %d , Maximum : %d \n', N,m)&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using the above code to get the block diagonal form of a certain antisymmetric matrix with non-zero entries. I am getting the block diagonal form to be always a null matrix. Is this a precision error ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I correct this ? Here is the matrix if needed :- &lt;a href=&quot;https://drive.google.com/file/d/0BxqrCZfy_mcXLW5PbkVUeHVSODA/edit?usp=sharing&quot; rel=&quot;nofollow&quot;&gt;https://drive.google.com/file/d/0BxqrCZfy_mcXLW5PbkVUeHVSODA/edit?usp=sharing&lt;/a&gt;&#xA;Thanks a lot in advance :-)&lt;/p&gt;&#xA;" OwnerUserId="4756" LastActivityDate="2014-07-09T16:21:34.047" Title="Matlab: why I am getting the block diagonal form of this antisymmetric matrix always as zero?" Tags="|matlab|matrix|block-decomposition|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13091" PostTypeId="2" ParentId="13083" CreationDate="2014-07-09T18:31:55.527" Score="3" Body="&lt;p&gt;If your surface is a sphere, Helmholtz equation can be solved numerically using vector spherical harmonics. Vector spherical harmonics are eigenfunctions of the Laplace Beltrami operator forming a complete basis. In order to obtain a general overview on this topic,  I point you to the book &quot;Lectures on Constructive Approximation&quot; by Michel, Volker. You will find expressions for the differential operators wrtthe basis of Vector spherical harmonics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your spherical surface is noisy (e.g. sphere with distortions) but the noise is not too large, you could still assume a sphere an approximate the PDE in vector spherical harmonics. Of course it depends on the application and the targeted accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a general surface one would choose a different method depending on the type of surface one has.&lt;/p&gt;&#xA;" OwnerUserId="9378" LastActivityDate="2014-07-09T18:31:55.527" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="13092" PostTypeId="2" ParentId="8468" CreationDate="2014-07-09T22:12:56.190" Score="1" Body="&lt;p&gt;You may want to use these for reference:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.paraview.org/Wiki/The_ParaView_Tutorial&quot; rel=&quot;nofollow&quot;&gt;The Paraview Tutorial&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.paraview.org/Wiki/ParaView/Python_Scripting&quot; rel=&quot;nofollow&quot;&gt;Paraview/Python Scripting&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="549" LastEditDate="2014-07-10T04:17:21.930" LastActivityDate="2014-07-10T04:17:21.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13093" PostTypeId="2" ParentId="8884" CreationDate="2014-07-09T22:15:42.210" Score="0" Body="&lt;p&gt;I know that you asked for filters in Paraview or VisIT. But, you could use another open source tool to reconstruct the surface first, such as &lt;a href=&quot;http://meshlab.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;MeshLab&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="549" LastEditDate="2014-07-10T04:16:07.597" LastActivityDate="2014-07-10T04:16:07.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13095" PostTypeId="2" ParentId="12954" CreationDate="2014-07-10T03:09:50.907" Score="3" Body="&lt;p&gt;if you need a program with 3D capabilities I would suggest &lt;a href=&quot;http://www.csc.fi/english/pages/elmer&quot; rel=&quot;nofollow&quot;&gt;Elmer&lt;/a&gt;, that provides a graphical user interface and is easy to use. If 2D capabilities are enough, I would suggest &lt;a href=&quot;http://www.agros2d.org/&quot; rel=&quot;nofollow&quot;&gt;Agros&lt;/a&gt; that has a really nice GUI (better than most of commercial software) and its use is intuitive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to your description you can model your problem as a set of beams, and this can be done easily in most of the programs.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2014-07-10T03:09:50.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13096" PostTypeId="2" ParentId="11672" CreationDate="2014-07-10T03:20:50.317" Score="1" Body="&lt;p&gt;If you are using numerical integration you can simply replace the integral&#xA;$$\int\limits_{\Omega_{el}} a(x) \nabla \varphi_i(x) \nabla \varphi_j(x) dx = \sum_{x_k \in \Omega_{el}} a(x_k) \nabla \varphi_i(x_k) \nabla \varphi_j(x_k)  \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you have to know your diffusivity at each Gauss point. This is easily know if you have an analytic function. If you know the data just in your nodes you can use the interpolators yo compute these values. In the case of linear elements is simpler to do what Geoff suggested.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2014-07-10T03:20:50.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13097" PostTypeId="2" ParentId="11532" CreationDate="2014-07-10T03:31:06.150" Score="3" Body="&lt;p&gt;You're right, the Jacobian would be invertible unless you have colinear/coplanar elements, i.e., degenerate elements. But this is the case when you have exact arithmetic. In real FEM programs you use floating point arithmetic and then, you can have Jacobians that are nearly zero. Probably, you can still invert this $2\times2$ or $3\times3$ matrix, but the error will accumulate in your model as a whole. Finally... getting an ill-conditioned matrix.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="549" LastEditDate="2014-07-10T15:55:56.833" LastActivityDate="2014-07-10T15:55:56.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14095" PostTypeId="1" CreationDate="2014-07-10T10:39:31.580" Score="5" ViewCount="173" Body="&lt;p&gt;For ease of explanation, suppose that you began with a two-dimensional surface in (x1, x2, x3, x4)-space, and the surface begins as a flat planar region in the (x1, x2)-plane. The boundaries of this flat region is not that important to resolve, but it would have a closed boundary. Pretend it is a square. So this surface might be specified by a triangular mesh, so a listing of points and their neighbours. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Through some operation, the plane now gets deformed into 4D. So for instance, I might assign +(0, 0, 1, 0) for some points and -(0, 0, 0, 1) for other points. At this point, you have a 2-manifold in 4D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I need to remesh this surface. I'd like to capture things like folds in the surface, and I'd like to remesh so that triangles that get stretched out are subdivided, and to avoid things like skinny triangles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that there is a lot of info on how this is done for two-dimensional planar surfaces in 2D, like &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot;&gt;Shewchuk's Triangle&lt;/a&gt; program. I've read &lt;a href=&quot;http://persson.berkeley.edu/distmesh/&quot;&gt;Persson and Strang's article&lt;/a&gt; on simple meshing in 2D, but I don't think this is generalizable to 2-manifolds in 4D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, I'm looking for a more-or-less simple algorithm (or program). The need for accuracy is not terribly high, and the boundary regions of my surface are not important to accurately resolve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody suggest literature I can refer to? There is a startlingly large amount of work done on meshing and remeshing, so it's difficult to see what is applicable for my problem. &lt;/p&gt;&#xA;" OwnerUserId="5107" LastEditorUserId="549" LastEditDate="2014-07-10T13:41:08.937" LastActivityDate="2015-07-17T13:51:15.313" Title="How do you mesh and re-mesh a surface (2-manifold) in 4D?" Tags="|computational-geometry|mesh-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14096" PostTypeId="1" CreationDate="2014-07-10T13:00:11.193" Score="9" ViewCount="1163" Body="&lt;p&gt;I am interested in methods for solving the optimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{array}{rl} \arg\min_x &amp;amp; x^T A x \\ \mathrm{s.t.} &amp;amp; x^T x = 1 \\ &amp;amp; Bx = 0 \end{array} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is symmetric and full rank with well-separated eigenvalues, and $B$ has a nontrivial null space.  In other words, I want to restrict $A$ to the null space of $B$ and find the eigenvector corresponding to the smallest eigenvalue.  In the absence of the linear constraint, this problem is a standard eigenvalue problem and can be solved using power iterations.  Ideally I would like to find a similar (iterative) procedure, since in my case I can gain a lot of performance by first computing a sparse factorization of $A$ (and potentially $B$ as well).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not interested in methods for dense problems, nor answers that simply amount to &quot;just use software package X&quot;; I am interested in implementing the method myself.  For this reason, simple methods are preferred, even over methods that are perhaps slightly better/faster/more robust.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of the paper &quot;Some Modified Matrix Eigenvalue Problems&quot; by Gene Golub, but do not see a way to efficiently incorporate the generalized inverse in the sparse case.  I am also aware of &lt;a href=&quot;https://math.stackexchange.com/questions/581080/finding-generalized-eigenvalues-with-linear-constraints&quot;&gt;this question&lt;/a&gt;, but do not want to rely on a sophisticated solver for generalized eigenvalue problems; I'm really looking for something &quot;not so different&quot; from the power method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9670" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2018-10-03T21:27:02.717" Title="Sparse smallest eigenvalue problem on a linear subspace?" Tags="|linear-algebra|eigensystem|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14097" PostTypeId="1" AcceptedAnswerId="14102" CreationDate="2014-07-10T15:07:11.943" Score="4" ViewCount="713" Body="&lt;p&gt;I've spent some time writing a spectral element solver aimed at solving fluid flow problems concerning the motion of self-propelled bodies. I've gotten to the point where the code has grown long enough that I'm having trouble organizing things in a way that is simple and understandable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently, I have written it in python (using numpy/scipy etc.) and I'm trying to take advantage of the use of modules and OOP concepts to break things down better. Unfortunately, I'm still a bit new with OOP concepts as my background is mainly in engineering. I have things organized follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A &quot;preprocess&quot; module that contains two classes: &quot;MasterElement&quot;, &quot;Element&quot; and &quot;Mesh&quot;. Basically, the Mesh class contains an array of nodes and a list of Element objects which are each linked to a MasterElement object. Objects instantiated from MasterElement contain attributes for interpolation, differentiation and integration within each element.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A &quot;gmsh&quot; module that parses files generated by the Gmsh automatic meshing program.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now is where I get a little confused as far as the best way to proceed. Basically, I need 4 things to happen:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;An &quot;initialization&quot; step where all necessary arrays are initialized into memory (a lot of &lt;code&gt;some_array = np.zeros(...)&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A &quot;processing&quot; step where each element in the mesh is looped over. The Jacobian is calculated and operators (weak Laplacian matrix, mass matrix etc.) are subsequently computed for each element. Boundary integrals associated with the &quot;right-hand-side&quot; of the global system are also computed and arranged into a global &quot;contour integral&quot; vector.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A solution step where the global system is assembled and solved (directly for now) inside a Newton-Raphson iteration loop.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Post processing where the solution is plotted etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Currently, I have steps 1, 2, and 3 mashed together in a long python script. Should I just break parts of the script into functions? This is straight forward, but I would be passing a ton of arguments between functions which might be just as confusing. Or is there a better way that I haven't considered?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if you could just refer me to some literature on this, that would be great. Most of the CFD books I have focus on the algorithms and not necessarily how to write good, reusable code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9601" LastEditorUserId="9601" LastEditDate="2014-07-10T15:12:29.293" LastActivityDate="2014-07-10T21:41:49.483" Title="Organizing a CFD program written in python" Tags="|fluid-dynamics|python|oop|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14099" PostTypeId="2" ParentId="11290" CreationDate="2014-07-10T16:28:38.767" Score="1" Body="&lt;p&gt;Since you want your program to be embedded in a game, something like a physics engine for structures, a good idea is to use an explicit FEM. Something similar to this simulations done with &lt;a href=&quot;http://subprotocol.com/verlet-js/&quot; rel=&quot;nofollow&quot;&gt;Verlet&lt;/a&gt; algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explicit FEM can be really fast if you can have a structured mesh, because in that way all the stiffness matrices for the elements are the same (you can compute it once). You can, actually, compute it analytically and hard code it in the program. The reason for the speed is that you don't need to solve a system of equations. Using the Verlet integration --and lumped mass matrices-- you just have to compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u^{(t+1)}_i = 2 u^{(t)}_i - u^{(t-1)}_i + \frac{F_i \Delta t^2}{M_{ii}} \enspace .$$&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2014-07-10T16:28:38.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14100" PostTypeId="1" CreationDate="2014-07-10T16:47:17.497" Score="4" ViewCount="818" Body="&lt;p&gt;I want to numerically solve the diffusion equation $\partial_t u = D \partial_x^2 u$ in Fourier space, and can think of multiple ways to do it.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Setup&lt;/h1&gt;&#xA;&#xA;&lt;h3&gt;Option 1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Differentiating $u$ twice in Fourier space brings down two factors of $ik$ from the exponential $e^{i k x}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_t \tilde{u}_k = D (ik)^2 \tilde{u}_k$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be discretised, for example, like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\tilde{u}^{n+1} - \tilde{u}^n}{\Delta t} = -Dk^2\left(\frac{\tilde{u}^{n+1} + \tilde{u}^{n}}{2} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Option 2&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Taking the second order finite difference equation in real space:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_x^2 u \approx \frac{u_{i+1} - 2 u_i + u_{i-1}}{\Delta x^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{i+1}$ and $u_i$ are $\Delta x $ apart. Fourier transforming this, remembering that phase shift of $\Delta x$ is given by $e^{\Delta x k}$ we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_x^2 \tilde{u}_k \approx \frac{e^{+\Delta x k} -2 + e^{-\Delta x k} }{\Delta x^2} \tilde{u}_k = 2 \frac{\cos(\Delta x k) - 1}{\Delta x^2} \tilde{u}_k\qquad(1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying again to the diffusion equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\tilde{u}^{n+1}_k - \tilde{u}^n_k}{\Delta t} = D\frac{2 (\cos(\Delta x k)-1)}{\Delta x^2} \left(\frac{\tilde{u}^{n+1}_k + \tilde{u}^{n}_k}{2} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across Option 2 in a few places, for example, in &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0898122188901824&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Which of these (if either) should I be using, and why? I'm guessing the discrete nature of the FFT is important, as the approaches are equivalent as $\Delta x \to 0$, seen by Taylor expanding Eq. (1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$2 \frac{\cos(\Delta x k) - 1}{\Delta x^2} = 2 \frac{1 - \frac{(\Delta x k)^2}{2} + \mathcal{O}(\Delta x^4) - 1}{\Delta x^2} = -k^2 + \mathcal{O}(\Delta x^2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Thinking more about this, the properties of the DFT are actually important. The maximum $k$ is $k_{max} =2\pi (N/2)$ where $\Delta x = L / N$. So as $\Delta x \to 0$, $k_{max} \to \infty$. Therefore the two methods are different for large $k$ at any $\Delta x$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2NDyG.png&quot; alt=&quot;plot comparing different methods&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This looks like the latter option is applying some smoothing at high $k$.&lt;/p&gt;&#xA;" OwnerUserId="7941" LastEditorUserId="7941" LastEditDate="2014-07-11T09:14:30.500" LastActivityDate="2014-07-11T09:14:30.500" Title="Two approaches to solving diffusion equation in Fourier space" Tags="|diffusion|spectral-method|fourier-transform|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14101" PostTypeId="2" ParentId="14100" CreationDate="2014-07-10T18:54:10.630" Score="1" Body="&lt;p&gt;Typically, the first approach is used. The range of frequencies used will limit an explicit time integrator, and is &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_solution_of_the_convection%E2%80%93diffusion_equation#Stability_criteria&quot; rel=&quot;nofollow&quot;&gt;well-known for finite-difference discretizations&lt;/a&gt;. The relationship to the finite difference case follows by noting that $D/(\Delta{x}^{2})$ is a frequency, so for low frequencies, you can take large time steps, but for high frequencies, stability limits the time step. Since the Fourier basis consists of eigenfunctions of the diffusion operator, another way to look at it is that the Fourier-transformed spectral discretization yields a collection of scalar test equations that can be integrated independently (it diagonalizes the operator), and then you can use standard arguments from numerical stability theory for ODEs to high frequencies will limit your time step due to stability. You should probably consider an implicit time integrator for the high-frequency modes, unless you are willing to use small time steps due to this stability limitation.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-10T18:54:10.630" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14102" PostTypeId="2" ParentId="14097" CreationDate="2014-07-10T21:41:49.483" Score="2" Body="&lt;p&gt;I suggest you to have steps 1, 2, 3 in different files. Since each step is conceptually different, and can be replaced by external routines/software it is better to have a modular architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Splitting the whole code into functions is useful. Regarding the passing of arguments, Python has different ways of doing this: &lt;a href=&quot;http://www.diveintopython.net/power_of_introspection/optional_arguments.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;optional parameters&lt;/a&gt;, &lt;a href=&quot;https://stackoverflow.com/questions/3394835/args-and-kwargs&quot;&gt;kwargs&lt;/a&gt;. You can take a look at &lt;a href=&quot;http://sfepy.org/doc-devel/index.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;SfePy&lt;/a&gt; &lt;a href=&quot;https://github.com/sfepy/sfepy&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; that is a FEM code written in Python, it could be useful as example.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-07-10T21:41:49.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14103" PostTypeId="2" ParentId="8899" CreationDate="2014-07-10T22:33:52.247" Score="19" Body="&lt;p&gt;See &lt;a href=&quot;https://math.stackexchange.com/questions/861674/decompose-a-2d-arbitrary-transform-into-only-scaling-and-rotation&quot;&gt;https://math.stackexchange.com/questions/861674/decompose-a-2d-arbitrary-transform-into-only-scaling-and-rotation&lt;/a&gt; (sorry, I would have put that in a comment but I've registered just to post this so I can't post comments yet).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But since I'm writing it as an answer, I'll also write the method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E=\frac{m_{00}+m_{11}}{2}; F=\frac{m_{00}-m_{11}}{2}; G=\frac{m_{10}+m_{01}}{2}; H=\frac{m_{10}-m_{01}}{2}\\&#xA;Q=\sqrt{E^2+H^2}; R=\sqrt{F^2+G^2}\\&#xA;s_x=Q+R; s_y=Q-R\\&#xA;a_1=\mathrm{atan2}(G,F); a_2=\mathrm{atan2}(H,E)\\&#xA;\theta=\frac{a_2-a_1}{2}; \phi=\frac{a_2+a_1}{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That decomposes the matrix as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$M=\pmatrix{m_{00}&amp;amp;m_{01}\\m_{10}&amp;amp;m_{11}}=\pmatrix{\cos\phi&amp;amp;-\sin\phi\\\sin\phi&amp;amp;\cos\phi}\pmatrix{s_x&amp;amp;0\\0&amp;amp;s_y}\pmatrix{\cos\theta&amp;amp;-\sin\theta\\\sin\theta&amp;amp;\cos\theta}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing to guard against with this method is that &lt;span class=&quot;math-container&quot;&gt;$G=F=0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$H=E=0$&lt;/span&gt; for atan2. &lt;del&gt;I doubt it can be any more robust than that&lt;/del&gt; (&lt;strong&gt;Update:&lt;/strong&gt; see Alex Eftimiades' answer!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reference is: &lt;a href=&quot;http://dx.doi.org/10.1109/38.486688&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1109/38.486688&lt;/a&gt; (given by Rahul there) which comes from the bottom of this blog post: &lt;a href=&quot;http://metamerist.blogspot.com/2006/10/linear-algebra-for-graphics-geeks-svd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://metamerist.blogspot.com/2006/10/linear-algebra-for-graphics-geeks-svd.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; As noted by @VictorLiu in a comment, &lt;span class=&quot;math-container&quot;&gt;$s_y$&lt;/span&gt; may be negative. That happens if and only if the determinant of the input matrix is negative as well. If that's the case and you want the positive singular values, just take the absolute value of &lt;span class=&quot;math-container&quot;&gt;$s_y$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9673" LastEditorUserId="9673" LastEditDate="2019-10-22T12:01:17.010" LastActivityDate="2019-10-22T12:01:17.010" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14104" PostTypeId="1" CreationDate="2014-07-11T08:19:36.280" Score="1" ViewCount="450" Body="&lt;p&gt;Electromagnetic metamaterials are &lt;strong&gt;resonant&lt;/strong&gt;, &lt;strong&gt;periodic structures&lt;/strong&gt; (repetition of a unit cell) involving both dielectric and conducting elements, and are used and simulated in frequency ranges from microwave (a few hundred MHz) &lt;strong&gt;up to optical&lt;/strong&gt; and more (up to PHz) frequencies. These structures are simulated in frequency domain using FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, considering the above properties of these structures, what properties (qualitatively) should the meshing of the unit cells of metamaterials have generally? More specifically, what are the benefits and drawbacks of using tetrahedral meshing versus hexahedral? (comparison)&lt;/p&gt;&#xA;" OwnerUserId="9677" LastEditorUserId="549" LastEditDate="2014-07-11T13:36:33.503" LastActivityDate="2014-07-12T19:54:10.760" Title="Which type of meshing is more suited for simulation of electromagnetic metamaterial unit cells?" Tags="|finite-element|mesh-generation|electromagnetism|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14105" PostTypeId="1" AcceptedAnswerId="14106" CreationDate="2014-07-11T12:17:41.007" Score="0" ViewCount="2640" Body="&lt;p&gt;I have scatter plot and I want to trace the border of that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fX39Q.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I wanted it traced:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OKso9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or at least wanted to know the border points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is it possible to detect if any point exist on inside or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xH4cf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9499" LastActivityDate="2014-07-11T12:25:56.147" Title="Border Mapping of Scatter plot" Tags="|python|plotting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14106" PostTypeId="2" ParentId="14105" CreationDate="2014-07-11T12:25:56.147" Score="2" Body="&lt;p&gt;You probably want to look into the literature on &lt;a href=&quot;http://en.wikipedia.org/wiki/Alpha_shape&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alpha Shapes&lt;/a&gt;. They are a generalization of convex hulls and should give you what you want. There's a discussion in &lt;a href=&quot;https://stackoverflow.com/questions/6833243/how-can-i-find-the-alpha-shape-concave-hull-of-a-2d-point-cloud&quot;&gt;this&lt;/a&gt; Stack Overflow question about a Python implementation. &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-07-11T12:25:56.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14107" PostTypeId="1" CreationDate="2014-07-11T17:40:21.910" Score="2" ViewCount="114" Body="&lt;p&gt;I have a partiuclar kind of graph problem and (not having a background in graph algorithms) I would like to know how this kind of problem is called in the literature and what algorithms exist for solving it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an undirected graph consisting of vertices $V$ and edges $E$. Each edge has a potential $p_e \in [0..1]$&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Independently of the edge potentials there is also edge weights $w_e \in \{0,1\}$ that turn on/off an edge and connect or disconnect two vertices. The number of 1-weight-edges of a vertex let's call $n$, which is the &quot;connectedness&quot; (or &quot;subgraph-degree&quot;) of a vertex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In our problem edge potentials are given and edge weights are what we look for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's then call $s_e=p_e*w_e$ the &quot;strength&quot; of an edge. Also, for any two vertices, if there's an indirect connection the strength of the indirect connection is the product of all strengths $s_e$ along the path. The strength of the shortest connection between two vertices let's call $t_e$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the problem is: for a given global $n$ (same for all vertices) how can I find weights $w_e$ for each edge resulting in the &quot;best&quot; interconnectivity for the graph, the one that maximizes the sum of $t_e$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words: Having a given (usually small) $n$, I would like to choose $n$ out of many existing edges for each vertex such that the resulting graph is &quot;nicely&quot; connected. That means that the algorithm would favor connecting two vertices with a somewhat lower $p_e$ over connecting vertices with a higher $p_e$ but who already have a pretty good indirect connection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: According to the comment below I am looking for the $n$-regular edge-induced subgraph that maximizes the sum of all $t_e$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a hunch that this might be NP hard but any approximate solutions would be appreciated too.&lt;/p&gt;&#xA;" OwnerUserId="740" LastEditorUserId="740" LastEditDate="2014-07-14T13:24:34.423" LastActivityDate="2014-07-14T13:24:34.423" Title="Algorithm for optimizing graph interconnectivity" Tags="|algorithms|graph-theory|approximation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14108" PostTypeId="1" AcceptedAnswerId="14110" CreationDate="2014-07-11T18:08:48.280" Score="0" ViewCount="852" Body="&lt;p&gt;I have an expression (let's say determinant of matrix A) expressed in symbolic form in terms of 2 decision variables x1, x2 and 2 parameters q1 and q2. I'm minimizing this using fmincon for different start point and parameter value combinations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For certain combinations of x1, x2, q1, q2 the matrix A is not invertible so I would want determinant evaluation to be zero but Matlab just gives a very small number like    9.7166e-91. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I had created the matrix A with numeric values for which inverse doesnt exist and then take the inverse it does give me the usual warning &quot;Warning: Matrix is close to singular or badly scaled. Results may be inaccurate.&quot; This warning is very good as I know that the output can be discarded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I need to work with symbolic expression as this is what needs to be fed to fmincon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what approach should I use in order to know that certain outputs from the numeric evaluation of a symbolic expression should be discarded?Like am happy to have &quot;NaN&quot; when inverse doesn't exist. Hope there is a trick which doesnt slow down my program &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, the Digits setting is 32 so should I use this and infer that if the determinant result is between -1E-32 and 1E-32 then result should be discarded ? (this approach does not seem correct)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions are welcome.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%Providing code below in order to replicate easily;&#xA;&#xA;syms x1 x2 q1 q2;% x1 and x2 are the decision variables while q1 and q2 are the parameters. For different values of parameters I'm trying to minimize a function using fmincon&#xA;&#xA;&#xA;%matrix A is DesignMatrixMult. Example provided below for illustrative purposes but its form can change depending on user input;&#xA; DesignMatrixMult = [                    ((q1*(exp(-q1*x1) - exp(-q2*x1)))/(q1 - q2)^2 - (exp(-q1*x1) - exp(-q2*x1))/(q1 - q2) + (q1*x1*exp(-q1*x1))/(q1 - q2))^2 + ((q1*(exp(-q1*x2) - exp(-q2*x2)))/(q1 - q2)^2 - (exp(-q1*x2) - exp(-q2*x2))/(q1 - q2) + (q1*x2*exp(-q1*x2))/(q1 - q2))^2, - ((q1*(exp(-q1*x1) - exp(-q2*x1)))/(q1 - q2)^2 + (q1*x1*exp(-q2*x1))/(q1 - q2))*((q1*(exp(-q1*x1) - exp(-q2*x1)))/(q1 - q2)^2 - (exp(-q1*x1) - exp(-q2*x1))/(q1 - q2) + (q1*x1*exp(-q1*x1))/(q1 - q2)) - ((q1*(exp(-q1*x2) - exp(-q2*x2)))/(q1&#xA;    - q2)^2 + (q1*x2*exp(-q2*x2))/(q1 - q2))*((q1*(exp(-q1*x2) - exp(-q2*x2)))/(q1 - q2)^2 - (exp(-q1*x2) - exp(-q2*x2))/(q1 - q2) + (q1*x2*exp(-q1*x2))/(q1 - q2));&#xA;     - ((q1*(exp(-q1*x1) - exp(-q2*x1)))/(q1 - q2)^2 + (q1*x1*exp(-q2*x1))/(q1 - q2))*((q1*(exp(-q1*x1) - exp(-q2*x1)))/(q1 - q2)^2 - (exp(-q1*x1) - exp(-q2*x1))/(q1 - q2) + (q1*x1*exp(-q1*x1))/(q1 - q2)) - ((q1*(exp(-q1*x2) - exp(-q2*x2)))/(q1&#xA;    - q2)^2 + (q1*x2*exp(-q2*x2))/(q1 - q2))*((q1*(exp(-q1*x2) - exp(-q2*x2)))/(q1 - q2)^2 - (exp(-q1*x2) - exp(-q2*x2))/(q1 - q2) + (q1*x2*exp(-q1*x2))/(q1 - q2)),                                        ((q1*(exp(-q1*x1) - exp(-q2*x1)))/(q1 - q2)^2 + (q1*x1*exp(-q2*x1))/(q1 - q2))^2 + ((q1*(exp(-q1*x2) - exp(-q2*x2)))/(q1 - q2)^2 + (q1*x2*exp(-q2*x2))/(q1 - q2))^2];&#xA;&#xA;&#xA;% determinant is calculated below;  &#xA;DCritn = simplify(simplify(det(DesignMatrixMult),'IgnoreAnalyticConstraints', true,'steps',500),'full');&#xA;%creating an anonymous function handle to DCritn;&#xA;DCritnF = matlabFunction(DCritn,'vars',{x1,x2,q1,q2},'file','');&#xA;&#xA;&#xA;&#xA;% few lines of code for generating different possible values of the parameter q1 and q2;&#xA;% As an example, one set of values is provided below;&#xA;q1=0.9287; q2 = 0.83;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the above parameter values, the function DCritnF is minimized using fmincon;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge I have is that for certain combinations of x1 and x2, DCritnF gives non-zero result even though I know that the inverse of DesignMatrixMult (matrix A) does not exist. For example, DCritnF is called with value of x1 = 42.5 and x2 = 95, it returns 9.7166e-91 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if I do the following,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TrialCheck = double(subs(DesignMatrixMult));&#xA;TrialCheckInverse = inv(TrialCheck);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Warning: Matrix is close to singular or badly scaled. Results may be&#xA;  inaccurate. RCOND =  5.404300e-18.  ans =&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1.0e+42 *&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;5.4788   -1.3938&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;-1.3938    0.3546&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also one smaller question;if I had done &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TrialCheck_Alternate = double(subs(DesignMatrixMult))&#xA;TrialCheckDeterm_Alternate = det(TrialCheck_Alternate)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I get no warning? If for inverse of matrix it issues a warning then shouldn't it also issue warning while finding determinant of that matrix?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TrialCheck_Alternate =&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1.0e-25 *&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;0.0139    0.0546&#xA;     0.0546    0.2146&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;TrialCheckDeterm_Alternate =&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;3.9175e-69&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2014-07-11T19:22:51.787" Title="Overcoming floating point issues when Inverse does not exist but determinant provides nonzero result" Tags="|matlab|matrix|floating-point|precision|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14110" PostTypeId="2" ParentId="14108" CreationDate="2014-07-11T19:22:51.787" Score="0" Body="&lt;p&gt;I'm guessing what you want is some way to tell that your square matrix $A$ is either singular or non-singular. For that purpose, I'd use the &lt;code&gt;rank&lt;/code&gt; function in MATLAB, which is internally using the SVD with some thresholding to determine the number of nonzero singular values.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-11T19:22:51.787" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14111" PostTypeId="2" ParentId="14104" CreationDate="2014-07-11T21:25:59.470" Score="2" Body="&lt;p&gt;The main advantage of the tetrahedral vs hexahedral is the mesh generation, there are automatic mesh generators for tetrahedral meshes that give &quot;good&quot; elements. This process is not that easy for the hexahedral case and you will need to do a bigger effort to get a nice mesh. Nevertheless, is not a good idea to use linear tetrahedrals since they have linear interpolators (and constant derivatives). In the case of quadratic shape functions I am not sure about which one is better. Although, complete Lagrangian elements are in general better (in terms of accuracy) than their serendipity counterparts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For high-order elements, like in the &lt;a href=&quot;http://www.opticsinfobase.org/josaa/abstract.cfm?uri=josaa-26-7-1598&quot; rel=&quot;nofollow&quot;&gt;Spectral&lt;/a&gt; &lt;a href=&quot;http://journals.aps.org/pre/abstract/10.1103/PhysRevE.80.056702&quot; rel=&quot;nofollow&quot;&gt;Element&lt;/a&gt; &lt;a href=&quot;http://journals.aps.org/pre/abstract/10.1103/PhysRevE.79.026705&quot; rel=&quot;nofollow&quot;&gt;Method&lt;/a&gt;, you will need less nodes per wavelength. These elements are less sensitive to distorsion, but they need special positions for the nodes and the &lt;a href=&quot;http://gji.oxfordjournals.org/content/139/3/806.short&quot; rel=&quot;nofollow&quot;&gt;advantages&lt;/a&gt; are mainly for quadrilateral/hexahedral.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2014-07-12T19:54:10.760" LastActivityDate="2014-07-12T19:54:10.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14112" PostTypeId="2" ParentId="14104" CreationDate="2014-07-11T22:06:40.993" Score="4" Body="&lt;p&gt;As for any other domain, your mesh needs to be fine enough to resolve the features you have. This means that the mesh has to be finer than the geometric details of your unit cell, and it needs to be finer than the wavelengths of the waves you consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond this, the question of tets vs hexes is a minor issue. Hexes are generally more accurate, but if the geometry is difficult, hex meshes are more difficult to generate. Of course, in your case, the unit cell is likely a pretty simple geometry, and in that case I'd go with hex meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more interesting question is: are you going to use any of the existing libraries for simulating your physical model, or are you going to do it all yourself? If the former, you should just go with the kind of cells the library supports, if it is restricted to one kind.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-11T22:06:40.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14115" PostTypeId="2" ParentId="13083" CreationDate="2014-07-12T11:23:27.967" Score="1" Body="&lt;p&gt;I suggest you also take a look at the papers by Andrea Bonito: &lt;a href=&quot;http://www.math.tamu.edu/~bonito/&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bonito/&lt;/a&gt;  He also has a number of papers on solving PDEs on surfaces.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-12T11:23:27.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14117" PostTypeId="1" CreationDate="2014-07-12T15:20:04.703" Score="6" ViewCount="4024" Body="&lt;p&gt;I do not need a complete answer but just some advice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a sparse matrix of points in a volume. I know a surface passing by these points exists and this surface is mostly flat and relatively smooth with some small harmonic contents. See the two images below (this surface of the first one is a bit too complicated and de second one is a bit too simple).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a method (Laplacian surfaces / Splines / etc.) to fit my points to the best surface of minimal order. Actually I am expecting do describe my surface with less than 30 coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice of the method I can use (with Matlab or Mathematica) ? If I can get a working example it would be awesome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/U0x0g.jpg&quot; alt=&quot;Too complicated&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/NiH6k.png&quot; alt=&quot;Too easy&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, It seems my expectations were too high. In fact, it is very difficult to fit a 2D surface without a model. I recently tried with a pretty simple model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f(x,y) = a_0 + a_1\cdot x^3 + a_2\cdot x^2y + a_3\cdot xy + a_4\cdot y^2x a_5\cdot y^3 + b_1 \sin(b_2\cdot x) + b_3 \cos(b_4\cdot x) + c_1 \sin(c_2\cdot x) + c_3 \cos(c_4 \cdot x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this one I can fit my surface with a least squares approach and normal equations. The result is not bad but it would be better to find a &quot;smart&quot; algorithm that can build the output function with an iterative method. I still haven't find anything like this. &lt;/p&gt;&#xA;" OwnerUserId="9688" LastEditorUserId="9688" LastEditDate="2014-11-02T15:50:08.900" LastActivityDate="2014-11-05T02:54:56.967" Title="Surface fitting" Tags="|matlab|regression|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14118" PostTypeId="1" CreationDate="2014-07-12T21:46:31.097" Score="1" ViewCount="836" Body="&lt;p&gt;I am minimising a diagonal quadratic matrix using CPLEX. All off diagonal elements are zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has 500 variables and 20 linear constraints plus each variable is constrained to be within 0 and 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the elements on the diagonal are greater than zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CPLEX complains that it is either not-optimal or unbounded depending on the values of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot see how this problem can be unbounded as it is a minimsation of a convex function. For some values CPLEX says that the solution is not optimal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have posted the lp file if that helps at ... &#xA;&lt;a href=&quot;http://speedy.sh/Ug76K/quadratic-fail.lp&quot; rel=&quot;nofollow&quot;&gt;http://speedy.sh/Ug76K/quadratic-fail.lp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the CPLEX log&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tried aggregator 1 time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;QP Presolve eliminated 15 rows and 0 columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reduced QP has 5 rows, 500 columns, and 1000 nonzeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reduced QP objective Q matrix has 500 nonzeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presolve time = 0.00 sec. (0.29 ticks)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parallel mode: using up to 8 threads for barrier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number of nonzeros in lower triangle of A*A' = 10&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Approximate Minimum Degree ordering&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Total time for automatic ordering = 0.00 sec. (0.00 ticks)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary statistics for Cholesky factor:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Threads                   = 8&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rows in Factor            = 5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integer space required    = 5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Total non-zeros in factor = 15&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Total FP ops to factor    = 55&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Itn      Primal Obj        Dual Obj  Prim Inf Upper Inf  Dual Inf          &lt;/p&gt;&#xA;&#xA;&lt;p&gt;0  2.7831848e+014 -2.7831848e+014 6.65e+001 9.07e+002 5.57e+014&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1  1.8149043e+012 -1.8149043e+012 5.37e+000 7.32e+001 4.49e+013&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2  1.5035906e+012 -1.5035906e+012 4.89e+000 6.67e+001 4.09e+013&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3  1.1322578e+012 -1.1322578e+012 4.24e+000 5.79e+001 3.55e+013&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4  7.9610680e+011 -7.9610680e+011 3.56e+000 4.85e+001 2.98e+013&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5  5.5016492e+011 -5.5016491e+011 2.96e+000 4.03e+001 2.47e+013&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Barrier time = 0.00 sec. (0.96 ticks)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Total time on 8 threads = 0.00 sec. (0.96 ticks)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Status =  2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ERROR STATUS IS  unbounded&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PRINTING LP&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone reproduce my problem using CPLEX ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: I have forced CPLEX to use the PRIMAL algorithm rather than the default. The optimiser now runs but I get a lot of &quot;Markovitz threshold set to X.XX&quot; style warnings.&lt;/p&gt;&#xA;" OwnerUserId="7997" LastEditorUserId="7997" LastEditDate="2014-07-13T14:20:17.057" LastActivityDate="2014-08-12T15:13:38.893" Title="How can a quadratic positive definite minimization be unbounded" Tags="|optimization|python|quadratic-programming|constrained-optimization|" AnswerCount="1" CommentCount="1" ClosedDate="2014-08-12T19:56:53.377" ContentLicense="CC BY-SA 3.0" />
  <row Id="14119" PostTypeId="1" AcceptedAnswerId="14124" CreationDate="2014-07-12T22:56:24.240" Score="2" ViewCount="235" Body="&lt;p&gt;I run Python/C simulations for biological problems, so the range of computational tasks can be broad. Here are some examples:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) continuum model simulations: FEM type&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) agent-based simulations: each agent may be a 2D polygon, on whose vertices ODEs are defined, modelling the time evolution of interesting variables, and interactions may be defined -- such a model may involve lots of computational geometry&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) agent-based simulations: each agent may be modelled using level set methods&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) agent-based simulations: interacting particle systems which are evolved using stochastic methods (e.g. Monte Carlo)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) molecular dynamics simulations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I have access to a computer with an Intel i5-2500K processor, but it is not my own, and I can only use it when free. I am running a model of type 2), and it may take me a few hours if there are lots of agents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a small budget (~$600 CAD) for a computer solely targeted towards scientific computing. What sort of processor should I look into picking up? What are some important terms and definitions I should know? &lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2014-07-14T16:21:36.960" Title="Which processor should I consider buying for a desktop built for scientific computing?" Tags="|simulation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14120" PostTypeId="2" ParentId="14117" CreationDate="2014-07-12T22:59:58.840" Score="1" Body="&lt;p&gt;Looks like it would be a good candidate for &lt;a href=&quot;http://en.wikipedia.org/wiki/Kriging&quot; rel=&quot;nofollow&quot;&gt;Gaussian Process Regression&lt;/a&gt; (also known as Kriging). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I'll explain a little more. Consider the space $X$ of all functions on a grid with $N$ nodes, $X := \mathbb{R}^N$. For example, given a vector $x \in X$, the first component $x_1$ is the value of the function at the first gridpoint, $x_2$ is the value at the second gridpoint, and so forth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is to put a gaussian prior distribution on $X$,&#xA;$$\pi_\text{prior} \sim e^{-x^T C^{-1} x},$$&#xA;and consider making measurements $y_k = x_k$ of the function at some nodes of interest $k$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can combine the prior information and the measurement information to construct a posterior distribution on $X$ using Bayes' theorem. Some care must be taken since the likelihood is an improper distribution, but it can be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then one takes the mean of the posterior to get a good function that fits the data. This mean function is equivalent to what you get from Kriging.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Entries of the prior covariance, $C_{ij}$, measure how similar you expect the function to be at nodes $i$ and $j$. A couple typical examples are,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Exponential decay covariance:&lt;/strong&gt; The similarity between the function at two points decreases as the points get further apart with decay rate $q$ (eg., $q=2$ is the most common) and characteristic length $\rho$:&#xA;$$C_{ij} \propto e^{-\frac{||x_i - x_j||^q}{\rho}}.$$&#xA;This matrix is a dense $n^2$-by-$n^2$ matrix, which may present some computational difficulties.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Laplacian covariance:&lt;/strong&gt; The covariance falls off in the same manner as the greens function of an elliptic PDE, $\Delta^q C(x-x_0) = \delta_x$:&#xA;$$C \propto \Delta^{-q},$$&#xA;where $\Delta$ is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Laplacian_matrix&quot; rel=&quot;nofollow&quot;&gt;(graph) Laplacian&lt;/a&gt;. This covariance is particularly nice since applying $\Delta^{-q}$ can be done efficiently with multigrid. So, you can compute the action of a dense decaying covariance matrix whenever needed (or it's square root), except without actually forming the dense matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2014-11-05T02:54:56.967" LastActivityDate="2014-11-05T02:54:56.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14122" PostTypeId="1" CreationDate="2014-07-12T18:30:52.207" Score="1" ViewCount="116" Body="&lt;p&gt;So, I'm trying to simulate polygon rigid body physics, both linear and angular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the MTV is easy in SAT (Minimum Translation Vector), and to use it to adjust position is also easy:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PolygonA.center -= MTV * .5&#xA;PolygonB.center += MTV * .5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But if you want to adjust velocity and angular velocity with it how would you do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I've thinked about, I haven't tested it yet though in my program:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AccelerationA = -MTV * (2 * PolygonB.mass / (PolygonA.mass + PolygonB.mass))&#xA;AccelerationB = MTV * (2 * PolygonA.mass / (PolygonA.mass + PolygonB.mass))&#xA;&#xA;PolygonA.velocity += AccelerationA&#xA;PolygonB.velocity += AccelerationB&#xA;&#xA;PolygonA.angularVelocity = Cross(Normalize(AccelerationA), PolygonA.velocity) / PolygonA.radius&#xA;PolygonB.angularVelocity = Cross(Normalize(AccelerationB), PolygonB.velocity) / PolygonB.radius&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where radius is the longest distance from the center to the fartherst vertex, the rotation axis length of that polygon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now is my idea correct?, and if not, how do I fix it?&lt;/p&gt;&#xA;" OwnerUserId="7929" OwnerDisplayName="Murplyx" LastEditorUserId="7929" LastEditDate="2014-07-12T23:29:51.680" LastActivityDate="2014-07-12T23:29:51.680" Title="Understanding the use of Minimum Translation Vector (MTV) for an elastic collision response" Tags="|software|computational-physics|vector|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14123" PostTypeId="2" ParentId="14117" CreationDate="2014-07-13T00:19:50.983" Score="1" Body="&lt;p&gt;Along with Kriging using sparse matrix techniques, you could also look up literature on sparse approximation (aka sparse decomposition) methods or sparse Gaussian graphical models. &lt;/p&gt;&#xA;" OwnerUserId="7295" LastActivityDate="2014-07-13T00:19:50.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14124" PostTypeId="2" ParentId="14119" CreationDate="2014-07-13T04:58:16.583" Score="6" Body="&lt;p&gt;&lt;em&gt;Disclaimer: most of this is purely opinion.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the $600 range for a whole computer, I'm not sure that processor matters all that much, as long as the architecture is x86. If you want to run simulations locally, memory would probably be the first thing that I'd look at spending money on, and since the main use of the computer is scientific computing, you'll probably want to skimp on just about everything but memory and maybe CPU. Memory will limit your problem size; clockspeed gains by getting more expensive CPUs are going to be on the order of 10-20%, maybe, and are less worth it for serial (or small-scale parallel) simulations. You'll probably want as much (ideally, fast) memory as you can afford, although if you know you're not going to be working on large problems, or manipulating large data sets, it may not be crucial to get more than 4-12 GB of RAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll need to have a graphics card to drive a monitor, but it's likely not worth spending the money on anything more than a basic GPU with that sort of budget.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-13T04:58:16.583" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="14125" PostTypeId="2" ParentId="14118" CreationDate="2014-07-13T07:57:16.413" Score="0" Body="&lt;p&gt;Your problem seems quite easy indeed. I tried MOSEK, another convex optimizer, instead of CPLEX and it solves in almost no time (see the log at the end).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is well scaled and thee is no evident reasons for CPLEX to get fooled. My guess is that it is not able to certify optimality. Unboundness shouldn't be the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MOSEK Version 7.0.0.114 (Build date: 2014-4-27 19:30:42)&#xA;Copyright (c) 1998-2014 MOSEK ApS, Denmark. WWW: &lt;a href=&quot;http://mosek.com&quot; rel=&quot;nofollow&quot;&gt;http://mosek.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Open file 'quadratic_fail.lp'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read summary&#xA;  Type             : QO (quadratic optimization problem)&#xA;  Objective sense  : min&#xA;  Constraints      : 20&lt;br&gt;&#xA;  Scalar variables : 500&lt;br&gt;&#xA;  Matrix variables : 0&lt;br&gt;&#xA;  Time             : 0.0     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer&#xA;  Platform               : Linux/64-X86&lt;br&gt;&#xA;  Cores                  : 2               &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem&#xA;  Name                   :&lt;br&gt;&#xA;  Objective sense        : min&lt;br&gt;&#xA;  Type                   : QO (quadratic optimization problem)&#xA;  Constraints            : 20&lt;br&gt;&#xA;  Cones                  : 0&lt;br&gt;&#xA;  Scalar variables       : 500&lt;br&gt;&#xA;  Matrix variables       : 0&lt;br&gt;&#xA;  Integer variables      : 0               &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optimizer started.&#xA;Interior-point optimizer started.&#xA;Presolve started.&#xA;Linear dependency checker started.&#xA;Linear dependency checker terminated.&#xA;Eliminator - tries                  : 0                 time                   : 0.00&lt;br&gt;&#xA;Eliminator - elim's                 : 0&lt;br&gt;&#xA;Lin. dep.  - tries                  : 1                 time                   : 0.00&lt;br&gt;&#xA;Lin. dep.  - number                 : 0&lt;br&gt;&#xA;Presolve terminated. Time: 0.00&lt;br&gt;&#xA;Matrix reordering started.&#xA;Local matrix reordering started.&#xA;Local matrix reordering terminated.&#xA;Matrix reordering terminated.&#xA;Optimizer  - threads                : 2&lt;br&gt;&#xA;Optimizer  - solved problem         : the primal&lt;br&gt;&#xA;Optimizer  - Constraints            : 5&#xA;Optimizer  - Cones                  : 0&#xA;Optimizer  - Scalar variables       : 505               conic                  : 0&lt;br&gt;&#xA;Optimizer  - Semi-definite variables: 0                 scalarized             : 0&lt;br&gt;&#xA;Factor     - setup time             : 0.00              dense det. time        : 0.00&lt;br&gt;&#xA;Factor     - ML order time          : 0.00              GP order time          : 0.00&lt;br&gt;&#xA;Factor     - nonzeros before factor : 15                after factor           : 15&lt;br&gt;&#xA;Factor     - dense dim.             : 0                 flops                  : 0.00e+00&lt;br&gt;&#xA;ITE PFEAS    DFEAS    GFEAS    PRSTATUS   POBJ              DOBJ              MU       TIME&lt;br&gt;&#xA;0   1.0e+02  1.1e+03  2.0e+04  0.00e+00   9.762500000e+03   -9.762500000e+03  1.0e+00  0.00&lt;br&gt;&#xA;1   2.5e+00  2.7e+01  1.3e+03  -8.24e-01  1.461547953e+03   -1.063368559e+04  1.1e-01  0.01&lt;br&gt;&#xA;2   2.1e+00  2.3e+01  1.2e+03  5.90e-01   1.926066470e+03   -1.046829917e+04  1.0e-01  0.01&lt;br&gt;&#xA;3   1.3e+00  1.3e+01  7.0e+02  5.10e-01   2.077327854e+03   -6.495728296e+03  6.0e-02  0.01&lt;br&gt;&#xA;4   5.2e-01  5.6e+00  3.4e+02  4.24e-01   3.416151322e+03   -2.006514499e+03  2.8e-02  0.01&lt;br&gt;&#xA;5   6.6e-02  7.0e-01  8.7e+01  6.41e-01   4.786227760e+03   3.147891392e+03   5.9e-03  0.01&lt;br&gt;&#xA;6   6.3e-03  6.7e-02  1.4e+01  9.60e-01   4.744795814e+03   4.485405619e+03   8.3e-04  0.01&lt;br&gt;&#xA;7   8.3e-06  8.8e-05  7.6e-01  9.98e-01   4.726819747e+03   4.712348560e+03   3.9e-05  0.01&lt;br&gt;&#xA;8   8.0e-07  8.4e-06  1.3e-01  1.00e+00   4.723211688e+03   4.720790746e+03   6.6e-06  0.01&lt;br&gt;&#xA;9   2.8e-09  3.0e-08  8.7e-03  1.00e+00   4.722531001e+03   4.722365208e+03   4.5e-07  0.01&lt;br&gt;&#xA;10  1.7e-12  1.9e-11  5.5e-04  1.00e+00   4.722481461e+03   4.722470859e+03   2.9e-08  0.01&lt;br&gt;&#xA;11  3.5e-15  5.9e-14  3.3e-05  1.00e+00   4.722478119e+03   4.722477487e+03   1.7e-09  0.01&lt;br&gt;&#xA;12  5.7e-16  1.1e-13  1.7e-06  1.00e+00   4.722477945e+03   4.722477912e+03   8.9e-11  0.01&lt;br&gt;&#xA;Interior-point optimizer terminated. Time: 0.01. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optimizer terminated. Time: 0.03    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interior-point solution summary&#xA;  Problem status  : PRIMAL_AND_DUAL_FEASIBLE&#xA;  Solution status : OPTIMAL&#xA;  Primal.  obj: 4.7224779451e+03    Viol.  con: 2e-16    var: 0e+00&lt;br&gt;&#xA;  Dual.    obj: 4.7224779122e+03    Viol.  con: 4e-12    var: 4e-13  &lt;/p&gt;&#xA;" OwnerUserId="7396" LastActivityDate="2014-07-13T07:57:16.413" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14126" PostTypeId="1" CreationDate="2014-07-13T12:27:29.197" Score="0" ViewCount="73" Body="&lt;p&gt;Consider solving numerically for roots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$( x_0, y_0): f(x_0, y_0) = 0, g(x_0, y_0) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where you only know that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;f, g continuously differentiable &lt;em&gt;but the theoretical differentiation is not a possible&lt;/em&gt;. So for this rather take it as continuous only. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f_x(x,y) \leq 0$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ g_y(x,y) \leq 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_y $ and $g_x $are per se not known, they can be locally both weakly increasing or decreasing. f,g are interpolations on discretized Bellman iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is impossible to fall back on actual values of the derivatives or cross derivatives, but I suppose, locally, with bad accuracy, these could be computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for any input w.r.t. how to attack this problem. As usual, performance is key. I'll be using Python, if someone wants to even point me at useful packages.&lt;/p&gt;&#xA;" OwnerUserId="9694" LastEditorUserId="549" LastEditDate="2014-07-13T18:50:13.693" LastActivityDate="2014-07-13T18:50:13.693" Title="Finding roots without knowing much about the function" Tags="|algorithms|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14127" PostTypeId="2" ParentId="14126" CreationDate="2014-07-13T12:52:19.003" Score="2" Body="&lt;p&gt;Since $f$ and $g$ are both differentiable, your problem sounds like a prime candidate for an inexact newton method.  Particularly, you can formulate the jacobian matrix by way of finite differences.  Check out Nocedal and Wright's book on &lt;a href=&quot;http://www.bioinfo.org.cn/~wangchao/maa/Numerical_Optimization.pdf&quot; rel=&quot;nofollow&quot;&gt;Numerical Optimization&lt;/a&gt;, particularly chapter 7 on practical approaches to calculating derivatives.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If necessary, you may also want to implement a globally convergent strategy, like line-searching or trust-region methods.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-07-13T12:52:19.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14128" PostTypeId="1" CreationDate="2014-07-13T18:01:38.437" Score="1" ViewCount="82" Body="&lt;p&gt;I'm solving an ODE, where currently the dependent variables are the (time dependent) spatial Fourier coefficients. It turns out that the phenomena I'm interested in describing is spatially localized, and it takes an impractical number of Fourier coefficients to fully describe it. Therefore, I'm interested in looking at different basis to use, instead of the canonical trig functions $e^{ikx}$ (for $k\in \mathbb{Z}$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for this model I need these basis functions to have certain properties. In particular, I need these functions to be differentiable, as well as periodic. Also, I need the functions to have relatively simple Hilbert Transforms, since this is exploited in solving for the governing equations (and is one reason why the trigonometric basis was desirable). Finally, the basis should converge more rapidly than the Fourier series for spatially compact scenarios. Hence, I would like to reduce the number of dependent variables that I'm solving for in my ODE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of any functions fitting the bill? Any tips/references are appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nick &lt;/p&gt;&#xA;" OwnerUserId="8021" LastActivityDate="2014-07-13T18:01:38.437" Title="Basis for spatially localized function" Tags="|basis-set|" AnswerCount="0" CommentCount="1" ClosedDate="2019-06-07T23:08:46.740" ContentLicense="CC BY-SA 3.0" />
  <row Id="14129" PostTypeId="2" ParentId="14096" CreationDate="2014-07-14T00:40:20.260" Score="1" Body="&lt;p&gt;Nobody has answered this question, so let me take a stab at an attempt. I don't quite know how to solve the problem exactly, but here's a method that should work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us assume that you have a basis of the null space and that it forms the columns of a matrix $Z$, i.e., $BZ=0$. Then consider the eigenvalue problem&#xA;$$&#xA;\begin{array}{rl} \arg\min_{x_\varepsilon} &amp;amp; {x_\varepsilon}^T A {x_\varepsilon} + \frac{1}{\varepsilon} {x_\varepsilon}^T Z^T Z {x_\varepsilon} \\ \mathrm{s.t.} &amp;amp; {x_\varepsilon}^T {x_\varepsilon} = 1 \end{array}&#xA;$$&#xA;If you consider the limit $\varepsilon\rightarrow 0$, then any vector not in the null space of $B$ produces an infinite objective function, and you will only get those eigenvalues you care about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond this, let me point out one other idea (though I do not know enough about the numerical realization of eigenspectrum solvers to really say much about this). Namely, when you compute the second, third, etc, $L$th eigenvalue of a matrix $A$, then this is often written as&#xA;$$&#xA;\begin{array}{rl} \arg\min_{x} &amp;amp; {x}^T A {x_\varepsilon}  \\ \mathrm{s.t.} &amp;amp; {x_\varepsilon}^T {x_\varepsilon} = 1 &#xA;\\&#xA;&amp;amp;x^T v_1 = 0&#xA;\\&#xA;&amp;amp; \vdots&#xA;\\&#xA;&amp;amp;x^T v_{L-1} = 0&#xA;\end{array}&#xA;$$&#xA;In other words, you try to minimize in a subspace. This is no different than what you want to do, except that your subspace isn't given by the previous eigenvectors $v_1, \ldots, v_{L-1}$ but by the basis of your null space $Z$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-14T00:40:20.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14130" PostTypeId="1" CreationDate="2014-07-14T02:43:59.917" Score="2" ViewCount="186" Body="&lt;p&gt;I want to construct a MIP formulation for Ising model. For simplicity, I will only include terms involving nearest-neighbor pairs and triangular terms. I propose one formulation and ask whether there would be some stronger formulation form that would be potentially faster in computation. Thank you :D&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the original objective is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum\limits_{i = 1}^{29} {\sum\limits_{j = 1}^{30} {{J_1}{s_{i,j}}{s_{i + 1,j}}} }  + \sum\limits_{i = 1}^{30} {\sum\limits_{j = 1}^{29} {{J_1}{s_{i,j}}{s_{i,j + 1}}} }  + \sum\limits_{i = 1}^{29} {\sum\limits_{j = 1}^{29} {{J_3}{s_{i,j}}{s_{i + 1,j}}{s_{i,j + 1}}} } $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${s_{i,j}} \in \{ 0,1\} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we linearize it so that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[\begin{align}&#xA;  &amp;amp; \min : \\ &#xA; &amp;amp; \sum\limits_{i=1}^{29}{\sum\limits_{j=1}^{30}{{{J}_{1}}{{s}_{i,j,1}}}}+\sum\limits_{i=1}^{30}{\sum\limits_{j=1}^{29}{{{J}_{1}}{{s}_{i,j,2}}}}+\sum\limits_{i=1}^{29}{\sum\limits_{j=1}^{29}{{{J}_{3}}{{s}_{i,j,3}}}} \\ &#xA; &amp;amp; \text{subject to:} \\ &#xA; &amp;amp; \forall i=1...29\,j=1...30\,\ k=1...4 \\ &#xA; &amp;amp; \left[ \begin{matrix}&#xA;   {{s}_{i,j}}  \\&#xA;   {{s}_{i+1,j}}  \\&#xA;   {{s}_{i,j,1}}  \\&#xA;\end{matrix} \right]={{y}_{i,j,1,1}}\left[ \begin{matrix}&#xA;   0  \\&#xA;   0  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,1,2}}\left[ \begin{matrix}&#xA;   1  \\&#xA;   0  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,1,3}}\left[ \begin{matrix}&#xA;   0  \\&#xA;   1  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,1,4}}\left[ \begin{matrix}&#xA;   1  \\&#xA;   1  \\&#xA;   1  \\&#xA;\end{matrix} \right] \\ &#xA; &amp;amp; \sum\limits_{k=1}^{4}{{{y}_{i,j,1,k}}=1} \\ &#xA; &amp;amp; 0\le y\le 1 \\ &#xA; &amp;amp; {{y}_{i,j,1,k}}\text{ is integral} \\ &#xA; &amp;amp;  \\ &#xA; &amp;amp; \forall i=1...30\,j=1...29\,\,k=1...4 \\ &#xA; &amp;amp; \left[ \begin{matrix}&#xA;   {{s}_{i,j}}  \\&#xA;   {{s}_{i,j+1}}  \\&#xA;   {{s}_{i,j,1}}  \\&#xA;\end{matrix} \right]={{y}_{i,j,2,1}}\left[ \begin{matrix}&#xA;   0  \\&#xA;   0  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,2,2}}\left[ \begin{matrix}&#xA;   1  \\&#xA;   0  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,2,3}}\left[ \begin{matrix}&#xA;   0  \\&#xA;   1  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,2,4}}\left[ \begin{matrix}&#xA;   1  \\&#xA;   1  \\&#xA;   1  \\&#xA;\end{matrix} \right] \\ &#xA; &amp;amp; \sum\limits_{k=1}^{4}{{{y}_{i,j,2,k}}=1} \\ &#xA; &amp;amp; 0\le {{y}_{i,j,2,k}}\le 1 \\ &#xA; &amp;amp; {{y}_{i,j,2,k}}\text{ is integral} \\ &#xA; &amp;amp;  \\ &#xA; &amp;amp; \forall i=1...29\,j=1...29\,\,k=1...8 \\ &#xA; &amp;amp; \left[ \begin{matrix}&#xA;   {{s}_{i,j}}  \\&#xA;   {{s}_{i+1,j}}  \\&#xA;   {{s}_{i,j+1}}  \\&#xA;   {{s}_{i,j,3}}  \\&#xA;\end{matrix} \right]={{y}_{i,j,3,1}}\left[ \begin{matrix}&#xA;   0  \\&#xA;   0  \\&#xA;   0  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,3,2}}\left[ \begin{matrix}&#xA;   1  \\&#xA;   0  \\&#xA;   0  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,3,3}}\left[ \begin{matrix}&#xA;   0  \\&#xA;   1  \\&#xA;   0  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,3,4}}\left[ \begin{matrix}&#xA;   1  \\&#xA;   1  \\&#xA;   0  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,3,5}}\left[ \begin{matrix}&#xA;   0  \\&#xA;   0  \\&#xA;   1  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,3,6}}\left[ \begin{matrix}&#xA;   1  \\&#xA;   0  \\&#xA;   1  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,3,7}}\left[ \begin{matrix}&#xA;   0  \\&#xA;   1  \\&#xA;   1  \\&#xA;   0  \\&#xA;\end{matrix} \right]+{{y}_{i,j,3,8}}\left[ \begin{matrix}&#xA;   1  \\&#xA;   1  \\&#xA;   1  \\&#xA;   1  \\&#xA;\end{matrix} \right] \\ &#xA; &amp;amp; \sum\limits_{k=1}^{8}{{{y}_{i,j,3,k}}=1} \\ &#xA; &amp;amp; 0\le {{y}_{i,j,3,k}}\le 1 \\ &#xA; &amp;amp; {{y}_{i,j,3,k}}\text{ is integral} \\ &#xA; &amp;amp;  \\ &#xA; &amp;amp; \forall i=1...30\,j=1...30 \\ &#xA; &amp;amp; 0\le {{s}_{i,j}}\le 1 \\ &#xA; &amp;amp; {{s}_{i,j}}\quad \text{integral} \\ &#xA;\end{align}]&lt;/p&gt;&#xA;" OwnerUserId="7891" LastEditorUserId="7891" LastEditDate="2014-07-15T21:23:27.240" LastActivityDate="2015-03-13T03:03:59.500" Title="enhancing a MIP formulation of Ising model" Tags="|optimization|mixed-integer-programming|combinatorics|" AnswerCount="1" CommentCount="6" ClosedDate="2016-04-29T16:01:17.447" ContentLicense="CC BY-SA 3.0" />
  <row Id="14131" PostTypeId="1" CreationDate="2014-07-14T14:09:07.313" Score="2" ViewCount="1127" Body="&lt;p&gt;In Lapack (zhseqr) and matlab, the eigenvalues of a complex matrix are computed successfully.&#xA;I notice that QR iteration or algorithm is involved with that process.&#xA;QR iteration repeats to call QR decomposition of R*Q from the previous step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my question: &#xA;QR decomposition in Lapack (zgeqrf) produces real diagonal entries.&#xA;QR iteration should make the complex diagonal entries of the upper triangular matrix as a result.&#xA;For example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; A = [0 2; 1 3*i];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This non-symmetric A is known to have eigenvalues of i and 2i.&#xA;Since the magnitudes of eigenvalues are distinct, the QR iteration probably converges well without any special shift.&#xA;So, we have the upper triangular matrix from schur decomposition or QR iteration in Lapack:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; schur(A)&#xA;ans =&#xA;       0 +          1i          -3 + 0.00066061i&#xA;       0 +          0i  1.4366e-18 +          2i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I tried to implement QR iteration to see how the algorithm works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;num_iter = 200;&#xA;[Q,R] = qr (A);&#xA;for ii=1:num_iter,&#xA;    [Q,R] = qr (R*Q);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the result R has&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R =&#xA;      -2 +          0i  2.4535e-13 -          3i&#xA;       0 +          0i           1 +          0i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My code converged to absolute values of eigenvalues, not to complex eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is wrong with my code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9698" LastActivityDate="2014-07-14T15:25:52.260" Title="How can QR iteration with complex matrices produce complex diagonal entries?" Tags="|matlab|eigenvalues|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14132" PostTypeId="2" ParentId="14131" CreationDate="2014-07-14T15:25:52.260" Score="2" Body="&lt;p&gt;Your QR iteration is correct, but your result is not: it's not &lt;code&gt;R&lt;/code&gt; that converges to the Schur form of &lt;code&gt;A&lt;/code&gt;, but &lt;code&gt;R*Q&lt;/code&gt;. The usual formulation of the QR iteration reflects this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:num_iter&#xA;    [Q,R] = qr(A);&#xA;    A = R*Q;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you get the desired result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; A&#xA;&#xA;A =&#xA;&#xA;   0.0000 + 2.0000i  -3.0000 - 0.0000i&#xA;  -0.0000 + 0.0000i  -0.0000 + 1.0000i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2014-07-14T15:25:52.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14133" PostTypeId="2" ParentId="14119" CreationDate="2014-07-14T16:21:36.960" Score="4" Body="&lt;p&gt;&lt;em&gt;(Turning a previous comment into an answer, because I want to say a bit more.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you should keep in mind is that your time is significantly more valuable than any hardware (at least in the workstation range). So if you spend three days assembling your own machine (or fiddling with the voltages and memory timings to get an overclocked machine even halfway stable) to save -- say -- $200 on a 30% faster machine, you have already lost money. &#xA;(The same applies to time spent in researching and picking components, by the way.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence my recommendations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do not worry too much about specific components; in this price range, the performance differences are unlikely to exceed 20%, which is seldom noticeable. (You will spend more time implementing/debugging/parsing output than the difference in calculation speed.) One exception is if you want to go into GPU computing, where the available software stack and application performance depends much more on the specific hardware. (But see above re: your time vs. money.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your group or department or institution has IT professionals on staff, ask them. They might not know as much about high performance computing as some people on this site, but they will know about any deals with specific vendors that would help you get the most bang for your six hundred bucks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2014-07-14T16:21:36.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14134" PostTypeId="1" AcceptedAnswerId="14136" CreationDate="2014-07-14T17:24:14.697" Score="8" ViewCount="5970" Body="&lt;p&gt;I am writing a finite element solver in C++. The main bottle neck is assembling the global stiffness matrix in sparse compressed row storage (so far I am only solving steady problems). Because I don't know how many nonzero entries exist in each row, I am currently assuming a constant upper bound on the number of non-zeros per row. My codes currently looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int nrow[fem.Np+1];          //row vector in CRS format initialized to 0&#xA;int ncol[fem.Np*MAX_NNZ];    //column vector in CRS format initialized to -1&#xA;double A[fem.Np*MAX_NNZ];    //values vector in CRS format initialized to 0.0&#xA;&#xA;//update nrow, ncol, and A &#xA;//MAX_NNZ is the constant upper bound on the number of nonzeros per row&#xA;//Np=='number of points (nodes)&#xA;//Npe=='number of points per element e.g. 3 for linear triangles&#xA;//kmat[Npe][Npe] is the previously computed local element stiffness matrix&#xA;for(int i=0;i&amp;lt;e.Npe;i++)&#xA;{&#xA;  for(int j=0;j&amp;lt;e.Npe;j++)&#xA;  {&#xA;    r = e.nodes[i];&#xA;    c = e.nodes[j];&#xA;    for(int p=MAX_NNZ*r-MAX_NNZ;p&amp;lt;MAX_NNZ*r;p++)&#xA;    {&#xA;      if(ncol[p]==-1)&#xA;      {&#xA;        for(int l=r;l&amp;lt;fem.Np+1;l++)&#xA;          nrow[l]++;&#xA;        ncol[p] = c-1;&#xA;        A[p] = e.kmat[i][j];&#xA;        break;&#xA;      }&#xA;      else if(ncol[p]==c-1)&#xA;      {&#xA;        A[p] = A[p] + e.kmat[i][j];&#xA;        break;&#xA;      }&#xA;    }&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this the most efficient way to assemble the global matrix in CRS format? Are there improvements that I can make? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2014-07-14T17:32:28.190" LastActivityDate="2015-07-17T14:20:56.653" Title="How to efficiently assemble global stiffness matrix in sparse storage format (c++)" Tags="|finite-element|c++|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14135" PostTypeId="2" ParentId="14130" CreationDate="2014-07-14T18:40:18.080" Score="1" Body="&lt;p&gt;Generally speaking, you want to construct formulations such that the convex hull of the linear programming (LP) relaxation is as small as possible, while retaining all potentially optimal feasible solutions. (Of course, feel free to write constraints that exclude currently feasible solutions that are known to be suboptimal.) In the ideal case, this convex hull has integral extreme points, in which case, the integrality constraints can be dropped and solving the cheaper LP relaxation also yields an optimal solution to the MIP.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-14T18:40:18.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14136" PostTypeId="2" ParentId="14134" CreationDate="2014-07-14T19:46:14.803" Score="7" Body="&lt;p&gt;When I do assembly, I use a Coordinate sparse matrix format. This is basically just a list of (row, col, value) tuples. This is especially useful for sparse matrix construction when the exact pattern of non-zeros is unknown (or simply not yet computed). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my implementation, I prohibit myself from actually querying the value of A(i,j), but this allows me to have a constant-time &quot;add&quot; function since the new value is simply appended to the end of the list of tuples. After construction is complete, I sort the list of tuples (summing any duplicate entries). When I want to use matrix-vector operations, I convert from this sorted coordinate-format sparse matrix to CRS. Performing the sort and conversion takes O(N logN) time, bottlenecked by the sorting step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different approach to this problem is to use your mesh to pre-compute where the non-zero entries are going to be prior to starting assembly. This allows you to efficiently update the values of the nonzeros. Trying to assemble directly in CRS without doing this step first is generally considered a bad idea, since you will have to move a lot of data around if you introduce new nonzeros.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2014-07-14T19:46:14.803" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14137" PostTypeId="2" ParentId="14134" CreationDate="2014-07-15T06:31:17.997" Score="4" Body="&lt;p&gt;Have a look at the thread below. Accepted answer gives a good explanation of how to create a CRS structure on the fly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/13026111/fast-accessing-elements-of-compressed-sparse-row-csr-sparse-matrix&quot;&gt;https://stackoverflow.com/questions/13026111/fast-accessing-elements-of-compressed-sparse-row-csr-sparse-matrix&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7179" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-07-15T06:31:17.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14141" PostTypeId="1" AcceptedAnswerId="14144" CreationDate="2014-07-15T13:44:50.973" Score="4" ViewCount="1671" Body="&lt;p&gt;Suppose I have a quadrature method with a theoretical error estimate that scales with the number of points as $e(N)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_a^b f(x)\mathrm{d}x = \sum_{i=1}^N w_i f(x_i) + \mathcal{O}\bigl(e(N)\bigr)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, $e(N) = N^{-4}$ for Simpson's rule, or $e(N) = N^{-2}$ for the trapezoidal rule. In this question I only care about how the error estimate scales with $N$, not its dependence on properties of the function or any leading coefficients. (And yes, I know it's just an &lt;em&gt;estimate&lt;/em&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use this quadrature method to perform a $d$-dimensional integral by repeated 1D integration,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{a_0}^{b_0} \int_{a_1(x_0)}^{b_1(x_0)} \cdots \int_{a_d(x_0,\ldots,x_{d-1})}^{b_d(x_0,\ldots,x_{d-1})} f(\vec{x})\mathrm{d}^d\vec{x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;how does the error estimate for the multidimensional integral scale with $N$, the number of function evaluations along each dimension? Is it just $e(N)$? Or some function of $e(N)$? (like $[e(N)]^d$) Is it impossible to give a general relation without knowing the specific quadrature method in use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done some Google searching and checking in Numerical Recipes but I can't find a straightforward answer to this. I would have thought it would be common.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2014-12-11T20:21:24.433" Title="How do error estimates scale for multidimensional cubature?" Tags="|quadrature|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14143" PostTypeId="2" ParentId="14096" CreationDate="2014-07-15T14:59:30.080" Score="9" Body="&lt;p&gt;Your main concern is not destroying sparsity - a transformation does not necessarily destroy it. Consider the transformed eigenvalue problem, that eliminates the constraint &lt;span class=&quot;math-container&quot;&gt;$x^Tx = 1$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \min_x \frac{x^TAx}{x^Tx} \qquad  s.t.\quad Bx = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a matrix &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; that is a basis for the nullspace of &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$BZ = 0$&lt;/span&gt;. Then, restricting &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to be in the nullspace of &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$x = Zy$&lt;/span&gt;, the eigenvalue problem now becomes an unconstrained optimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \min_{y} \frac{y^TZ^TAZy}{y^TZ^TZy}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimizer of this problem is the eigenvector corresponding to the smallest eigenvalue of the generalized eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ Z^TAZ y = \lambda Z^T Z y$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this eigenvalue problem one does not have to form &lt;span class=&quot;math-container&quot;&gt;$Z^TAZ$&lt;/span&gt; and destroy sparsity (which is your concern). One only needs to form matrix-vector products with &lt;span class=&quot;math-container&quot;&gt;$Z^TAZ$&lt;/span&gt;, this can be done in turn. How to form &lt;span class=&quot;math-container&quot;&gt;$Zx$&lt;/span&gt;? One possibility is to consider a partitioning using a permutation matrix &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$B_p$&lt;/span&gt; is well-conditioned of rank &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$B \in \mathbb{R}^{m\times n}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$BP = [B_p, B_n] \qquad Z = P \begin{bmatrix} -B_p^{-1}B_n \\ I \end{bmatrix}$$&lt;/span&gt; &#xA;In practice one doesn't invert &lt;span class=&quot;math-container&quot;&gt;$B_p$&lt;/span&gt; but computes a sparse LU factorization. Once you know how to form &lt;span class=&quot;math-container&quot;&gt;$Zx$&lt;/span&gt;, any Krylov based eigenvalue solver for generalized eigenvalues can be used (their convergence is often better than power method). Other possibilities are discussed in section 6 (Null space methods) in this paper &lt;a href=&quot;http://mathcs.emory.edu/~benzi/Web_papers/bgl05.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://mathcs.emory.edu/~benzi/Web_papers/bgl05.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a good source for templates for implementing eigensolvers - you can pick your favorite &lt;a href=&quot;http://web.eecs.utk.edu/~dongarra/etemplates/node155.html&quot; rel=&quot;noreferrer&quot;&gt;http://web.eecs.utk.edu/~dongarra/etemplates/node155.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4558" LastEditorUserId="20688" LastEditDate="2018-10-03T21:27:02.717" LastActivityDate="2018-10-03T21:27:02.717" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14144" PostTypeId="2" ParentId="14141" CreationDate="2014-07-15T16:46:57.243" Score="8" Body="&lt;p&gt;Using product quadrature rules for multi-dimensional integrals suffers from the so-called curse of dimensionality.  An $O(N^{-2})$ accurate rule using N evaluations in one-dimension is generally $O(N^{-2})$ accurate when applied as a product rule in multiple dimensions, but there will be $M = N^d$ evaluations required. So the accuracy is $O(M^{-2/d}).$  The required number of evaluations becomes intractable beyond 10 dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Monte Carlo and Quasi-Monte Carlo methods seem to be preferred approaches for many dimensions with an error that diminishes like $1/\sqrt{M}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if $f:[0,1] \rightarrow \mathbb{R}$ and the second-derivative is bounded -- $|f''(x)|\leq K$ -- then the error bound for the mid-point rule applied at the mid-points $\bar{x_i}= (2i-1)/(2N)$ of $N$ evenly spaced points $x_i = i/N$ is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left|\int_{0}^{1}f(x)\,dx - \frac1{N}\sum_{i=1}^{N}f\left(\bar{x_i}\right)\right|\leq \frac{K}{24N^2}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the two-dimensional case,  $f:[0,1]^2 \rightarrow \mathbb{R}$, where $f$ has bounded second-order partial derivatives we have the same order of accuracy with respect to N. Using a Taylor series approximation, we find that the first- and mixed second-order derivative terms vanish after integrating and the remaining error is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left|\int_{0}^{1}\int_{0}^{1}f(x,y)\,dx\,dy - \frac1{N^2}\sum_{i=1}^{N}\sum_{j=1}^{N}f\left(\bar{x_i},\bar{y_i}\right)\right|\\=\left|\sum_{i=1}^{N}\sum_{j=1}^{N}\int_{x_{i-1}}^{x_i}\int_{y_{j-1}}^{y_j}\frac1{2}\left[f_{xx}(\xi_x,\xi_y)(x-\bar{x_i})^2+f_{yy}(\eta_x,\eta_y)(y-\bar{y_i})^2\right]\,dx\,dy \right| \\ = N^2O(N^{-4})= O(N^{-2})$$&lt;/p&gt;&#xA;" OwnerUserId="9707" LastEditorUserId="9707" LastEditDate="2014-12-11T20:21:24.433" LastActivityDate="2014-12-11T20:21:24.433" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14145" PostTypeId="1" CreationDate="2014-07-15T19:22:01.707" Score="4" ViewCount="159" Body="&lt;p&gt;Given a problem like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \text{min } ||Ex-f|| \text{ s.t.}$$&#xA;$$ Gx \ge 0$$&#xA;$$ Cx = d $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And assuming that the matrices are medium sized (dimensions in the low thousands) and dense, what's the state-of-the-art for solving this system (imagine for use in a real time system)?  Preferably using something like an active set algorithm instead of gradient descent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some classic books (eg: &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898713609&quot; rel=&quot;noreferrer&quot;&gt;Numerical Methods for Least Squared Problems&lt;/a&gt; and &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898713560&quot; rel=&quot;noreferrer&quot;&gt;Solving Least Squares Problems&lt;/a&gt;) that list some algorithms, but they're decades old, concentrate on algorithms for simpler problems (with conversions from the more complex form) and tend to only gloss over important details like storage, cache use, and numerical considerations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's some resources on the quadratic programming side (eg: &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/quadprog.html&quot; rel=&quot;noreferrer&quot;&gt;quadprod&lt;/a&gt; in Matlab).  I've ordered the &quot;Practical Optimization&quot; book (listed in the footnotes for quadprod) to see what sort of methods it has.  But again, it's from the 80s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most google searches provide hits for recent research, but they tend to concentrate on either large sparse systems, interior point methods like gradient descent, or both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm curious what methods are considered modern for dense active-set style algorithms?&lt;/p&gt;&#xA;" OwnerUserId="1365" LastActivityDate="2014-07-16T11:03:31.450" Title="State-of-the-art for active set optimization algorithms?" Tags="|linear-algebra|optimization|numerical-analysis|constrained-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14146" PostTypeId="1" CreationDate="2014-07-16T03:27:45.173" Score="3" ViewCount="213" Body="&lt;p&gt;I have a single sparse matrix $A$ for which I can easily compute $Ax$ and $A^*x$ and I would like to solve $\min(||Ax||) \text{ s.t. } ||x||=1$. I know the answer is the right singular vector associated with the smallest singular value, and thus I can use something like Lanczos (or one of the modern bells-and-whistles versions) to find the smallest singular value. However, I've seen a lot of people warning me that Lanczos is numerically unstable especially for rank-deficient matrices (I know that $A$ should, in general, be 1-less than full rank -- that is, in the noiseless case, $Ax=0$ should have a non-trivial solution). Is there something smarter that I should be using? &lt;/p&gt;&#xA;" OwnerUserId="9717" LastEditorUserId="9667" LastEditDate="2014-07-16T22:42:09.650" LastActivityDate="2014-07-16T22:42:09.650" Title="Find a single vector in the null space" Tags="|svd|nullspace|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14147" PostTypeId="2" ParentId="14146" CreationDate="2014-07-16T08:12:57.100" Score="1" Body="&lt;p&gt;If your sparse matrix isn't too large and you can store it in memory, you could use a sparse (rank-revealing) QR (or LU, or SVD) factorization to determine the kernel of your matrix $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately, if you know something about the spectrum of your operator, you could apply a shift to it -- solve for eigenvectors associated with an eigenvalue of $\lambda$ for $A' = A + \lambda I$ -- using (inverse) Lanczos iteration. The rate of convergence depends on the separation of the eigenvalue you're solving for (the largest/smallest eigenvalue of the shifted matrix) from the remainder of the spectrum. The larger the separation, the larger the rate of convergence. Picking $\lambda$ close to zero will probably work; generally, for shift-and-invert strategies, the shift parameter is chosen to be close to the desired eigenvalue, but not the exact eigenvalue. You might have to be careful about conditioning issues, because you'll have to invert the shifted matrix; hopefully you have a good preconditioner, if you're only interested in using Krylov subspace methods.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-16T08:12:57.100" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14148" PostTypeId="1" CreationDate="2014-07-16T10:58:31.427" Score="7" ViewCount="3575" Body="&lt;p&gt;The diffusion equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial T}{\partial t}&#xA;   = \alpha \left( \frac{\partial^2 T}{\partial x^2} \right) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An explicit finite difference approach can be used to solve this, forward in time and central&#xA;differences in space. Approximating the diffusion equation at a node i, yields,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{T_i^{n+1}-T_i^n}{\Delta t} = \alpha\frac{T_{i+1}^n-2T_i^n+T_{i-1}^n}{\Delta x^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which gives &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_i^{n+1} = T_i^n(1-\omega)+\omega(0.5T_{i+1}^n+0.5T_{i-1}^n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\omega = \frac{2\alpha \Delta t}{\Delta x^2}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book states that for stability condition, the coefficients of the right-hand side terms must be positive which implies that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta t \lt \frac{\Delta x^2}{2\alpha}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is why should the coefficients of the right-hand side terms be positive for stability?&lt;/p&gt;&#xA;" OwnerUserId="9721" LastEditorUserId="9721" LastEditDate="2014-08-18T11:35:34.523" LastActivityDate="2016-04-04T15:18:46.027" Title="Finite Difference Method Stability" Tags="|finite-difference|stability|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14149" PostTypeId="2" ParentId="14145" CreationDate="2014-07-16T11:03:31.450" Score="2" Body="&lt;p&gt;Read through the book &quot;Numerical Optimization&quot; by Nocedal and Wright to understand how active set methods work in general. The more modern variations use the primal-dual method by Kunisch et al. to predict which constraints will/will not be active.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-16T11:03:31.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14150" PostTypeId="2" ParentId="14134" CreationDate="2014-07-16T11:11:32.377" Score="5" Body="&lt;p&gt;Just as a reference point, in the deal.II finite element library (see &lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;noreferrer&quot;&gt;http://www.dealii.org&lt;/a&gt;), we typically pre-compute the sparsity pattern once at the beginning by &quot;simulating&quot; what would happen if we did the assembly, and then create the sparse matrix from this sparsity pattern. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first tutorial in which we build a sparsity pattern is step-3:&#xA;  &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_3.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_3.html&lt;/a&gt;&#xA;We do this in the Step3::setup_system() function. Sparsity patterns in general are discussed here:&#xA;  &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/group__Sparsity.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/group__Sparsity.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-16T11:11:32.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14151" PostTypeId="1" CreationDate="2014-07-16T11:54:35.730" Score="1" ViewCount="132" Body="&lt;p&gt;I have done Molecular Dynamics Simulation and now want to venture into Lattice Boltzmann Method. What would be the best reference book/lecture notes/videos for a beginner?&lt;/p&gt;&#xA;" OwnerUserId="9721" LastActivityDate="2014-07-16T11:54:35.730" Title="Lattice Boltzmann Method" Tags="|reference-request|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14152" PostTypeId="2" ParentId="11675" CreationDate="2014-07-16T14:26:14.013" Score="3" Body="&lt;p&gt;Since Ansys 14.5 you can use ACT (Application Customization Toolkit) to connect external solvers and retrieve results.&lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2014-07-16T14:26:14.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14153" PostTypeId="1" CreationDate="2014-07-16T14:35:20.970" Score="5" ViewCount="867" Body="&lt;p&gt;Given symmetric, positive-definite matrix $A$ and its preconditioner $M^{-1}$. What's kind of preconditioner $M^{-1}$ which preserve the symmetry of $M^{-1}A$?&lt;/p&gt;&#xA;" OwnerUserId="7397" OwnerDisplayName="user2863620" LastEditorUserId="3481" LastEditDate="2014-07-16T15:36:22.923" LastActivityDate="2014-07-16T15:36:22.923" Title="Recommendations for symmetric preconditioner" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14154" PostTypeId="2" ParentId="14153" CreationDate="2014-07-16T15:28:12.717" Score="6" Body="&lt;p&gt;Not a real answer probably: typically for positive-definite matrices the problem is avoided by switching to two-side preconditioning: one looks for $M=LL^T$ that approximates the spectrum of $A$ and then uses an iterative method to solve the linear system $L^{-1}AL^{-T}y=L^{-1}b$, with $y=L^{T}x$. See for instance the wikipedia page for &lt;a href=&quot;https://en.wikipedia.org/wiki/Preconditioned_conjugate_gradient_method#The_preconditioned_conjugate_gradient_method&quot;&gt;preconditioned CG&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-07-16T15:28:12.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14155" PostTypeId="2" ParentId="14153" CreationDate="2014-07-16T15:35:11.053" Score="8" Body="&lt;p&gt;Often $M^{-1}A$ is not symmetric, even if $M$ and $A$ are. There are two common approaches to dealing with this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find a Cholesky factorization of $M$ into $LL^\top$, and instead solve for $L^{-1}AL^{-\top}$. If you're using an incomplete Cholesky factorization preconditioner, this involves no extra work. For other preconditioners, this might be impractical.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Exploit the fact that $M^{-1}A$ is a symmetric linear operator with respect to the inner product&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \langle x, y\rangle_M = x^*My$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apply the conjugate gradient method to $M^{-1}A$ with this new inner product. With a bit of book-keeping, you can guarantee that, in each CG iteration, you only compute one matrix multiplication with $A$ and one with $M^{-1}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From what I've seen, (2) is the more commonly used approach. Moreover, they're all equivalent in the end anyway, so there's no particular advantage to one or the other besides ease of implementation. For more details, see &lt;a href=&quot;http://books.google.com/books?isbn=0898715342&quot;&gt;Saad's book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-07-16T15:35:11.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14156" PostTypeId="1" AcceptedAnswerId="14157" CreationDate="2014-07-16T16:16:53.980" Score="8" ViewCount="597" Body="&lt;p&gt;I've coded up a Stokes Flow problem using finite elements and am in the process of verifying that it works. I'm just not sure what convergence rate I should be expecting as I globally refine the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know for scalar problems using linear basis functions I'd expect order $h^2$ convergence ($h$ is element size), and using quadratic basis functions I'd expect order $h^3$ convergence in the $L^2$ norm and one power less in the $H^1$ seminorm. The problem I'm having now is that when coding Stokes flow I used the Taylor-Hood element which uses linears for the pressure and quadratic for the velocity components. Is it as simple as the velocities converging at $h^3$ and the pressure at order $h^2$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I posted this on mathoverflow first and was told it might be better suited to this forum.&lt;/p&gt;&#xA;" OwnerUserId="9728" LastActivityDate="2014-07-16T18:27:55.443" Title="Finite element convergence rates for mixed problems" Tags="|pde|finite-element|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14157" PostTypeId="2" ParentId="14156" CreationDate="2014-07-16T17:37:09.943" Score="7" Body="&lt;p&gt;The Taylor-Hood approximation of the Stokes flow is a mixed finite element method, for which error estimates generally have the form&#xA;$$&#xA;\|u-u_h\|_V + \|p-p_h\|_M \leq C (\inf_{w_h\in V_h}\|u-w_h\|_V + \inf_{q_h\in M_h}\|p-q_h\|_M),&#xA;\tag{1}&#xA;$$&#xA;where $(u,p)\in V\times M$ is the exact solution and $(u_h,p_h)\in V_h\times M_h$ is the approximation. For the Stokes flow, $V=H^1_0(\Omega)^d$ and $M = L^2(\Omega)$ (with mean value zero). For the $P_2$-$P_1$-Taylor-Hood element, $V_h$ consists of continuous piecewise quadratic polynomials and $M_h$ of continuous piecewise linear polynomials, for which both terms on the right hand side can be bounded by quadratic approximation errors using standard arguments (e.g., Bramble-Hilbert lemma and transformation rules):&#xA;$$&#xA;\begin{aligned}&#xA;\inf_{w_h\in V_h}\|u-w_h\|_{H^1} &amp;amp;\leq C h^2\|u\|_{H^3}\\&#xA;\inf_{q_h\in M_h}\|p-q_h\|_{L^2} &amp;amp;\leq C h^2\|p\|_{H^2}\\&#xA;\end{aligned}&#xA;$$&#xA;(rule-of-thumb &quot;number of derivatives on the left $+$ powers of $h$ $=$ number of derivatives on the right&quot;).&#xA;Inserting this into $(1)$ yields&#xA;$$\|u-u_h\|_{H^1} + \|p-p_h\|_{L^2} \leq C h^2(\|u\|_{H^3} + \|p\|_{H^2}).$$&#xA;(assuming that the exact solution is in fact regular enough).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since both the continuous and the discrete Stokes problem are an upper triangular coupled linear system of the form&#xA;$$\begin{aligned} Au + B^* p &amp;amp;= f\\ Bu &amp;amp;=0\end{aligned}$$&#xA;the error estimate (which exploits that the difference of the solutions satisfies a similar system for $A_h$ and $B_h$ and uses the invertibility of $A$ on the kernel of $B$) for $u-u_h$ depends in general on the error in $p-p_h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you look at the proof, there's a loop hole: If the null space of $B_h$ is contained in the null space of $B$, then the coupling term in fact drops out, and you obtain an error estimate involving $u$ only:&#xA;$$\|u-u_h\|_{H^1} \leq C h^2 \|u\|_{H^3}$$&#xA;From there, you can apply the standard Aubin-Nitsche trick (if the adjoint equation is well-posed, which is the case if the domain $\Omega$ is regular enough -- a convex polygon in 2D or has a boundary which can be parametrized by a Lipschitz differentiable function) to obtain a convergence rate for the $L^2$-error of one order higher:&#xA;$$\|u-u_h\|_{L^2} \leq C h^3 \|u\|_{H^3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find these results in &lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-1-4757-4355-5&quot; rel=&quot;noreferrer&quot;&gt;Ern, Guermond: &lt;em&gt;Theory and Practice of Finite Elements&lt;/em&gt;, Springer, 2004&lt;/a&gt;. (The error estimates are collected in Theorem 4.26, while the necessary regularity for $\Omega$ is defined in Lemma 4.17; the proofs are unfortunately scattered across the book, and I think $\ker B_h\subset \ker B$ is nowhere verified explicitly.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-07-16T18:27:55.443" LastActivityDate="2014-07-16T18:27:55.443" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="14158" PostTypeId="2" ParentId="14148" CreationDate="2014-07-16T18:30:45.657" Score="4" Body="&lt;p&gt;For stability you need $\omega \leq 1$, as stated &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_method&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. If your diffusivity $\alpha$ is positive (that should be based on a physical basis),  $\omega$ should also be positive since $\Delta x$ and $\Delta t$ are, by definition, positive numbers.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2014-08-28T14:07:03.460" LastActivityDate="2014-08-28T14:07:03.460" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14159" PostTypeId="1" CreationDate="2014-07-16T18:33:04.697" Score="1" ViewCount="68" Body="&lt;p&gt;I have some technical question concerning the ellipsoid method Referring to the paper :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://paswkshop.comm.utoronto.ca/~weiyu/01658226.pdf&quot; rel=&quot;nofollow&quot;&gt;http://paswkshop.comm.utoronto.ca/~weiyu/01658226.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is mentioned in p.1317 at the last line in the left column that &quot;When the subgradient of the dual objective function has a bounded norm [26], the ellipsoid update is guaranteed to converge&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the question is what if the subgradient doesn't have a bounded norm ? Can convergence to the optimal multiplier value still take place ? In my problem, there is no upper bound on the multiplier value like what the authors did in the attached paper in equation (23).&lt;/p&gt;&#xA;" OwnerUserId="4467" LastActivityDate="2014-07-16T18:33:04.697" Title="Question about ellipsoid method" Tags="|optimization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14161" PostTypeId="1" AcceptedAnswerId="14162" CreationDate="2014-07-17T01:54:01.190" Score="0" ViewCount="77" Body="&lt;p&gt;How many years it would take our laptops to be as fast as the fastest super computer in 2000 according to Moore's law?&lt;/p&gt;&#xA;" OwnerUserId="9734" LastActivityDate="2014-07-17T02:27:28.413" Title="How many years it would take our laptops to be as fast as the fastest super computer in 2000" Tags="|computer-arithmetic|" AnswerCount="1" CommentCount="1" ClosedDate="2014-07-17T02:27:06.123" ContentLicense="CC BY-SA 3.0" />
  <row Id="14162" PostTypeId="2" ParentId="14161" CreationDate="2014-07-17T02:27:28.413" Score="2" Body="&lt;p&gt;Let's suppose that by &quot;fastest supercomputer in 2000&quot;, and as your metric of speed, you choose the High Performance LINPACK benchmark, as used in the &lt;a href=&quot;http://www.top500.org/statistics/perfdevel/&quot; rel=&quot;nofollow&quot;&gt;TOP500 list&lt;/a&gt;. Depending on when &quot;in 2000&quot; means, you're either selecting as your metric of performance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2.4 TFLOP/s, in June 2000&lt;/li&gt;&#xA;&lt;li&gt;4.9 TFLOP/s, in November 2000&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Setting aside all of the issues involved in using the LINPACK benchmark -- namely, that it no longer reflects useful performance metrics for today's HPC systems as they relate to actual application performance -- the assumption of &lt;a href=&quot;http://en.wikipedia.org/wiki/Moore&amp;#39;s_law&quot; rel=&quot;nofollow&quot;&gt;Moore's Law&lt;/a&gt; growth is also...well, not terribly useful, since it relates to the growth in the number of transistors in integrated circuits (more or less, CPUs, which consist of integrated circuit(s)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thermal limitations are going to limit frequency scaling, so that's probably going to be more or less flat (maybe even go down) while number of cores increases. At 4 FLOP/s/cycle, assuming a middle-of-the-road 2.5 GHz processor yields 10 GFLOP/s/core. So, in your laptop, you'd need roughly 250-500 cores @ 2.5 GHz operating at their theoretical peak performance (which also doesn't happen). Efficiency (ratio of actual to peak performance) &lt;a href=&quot;http://www.top500.org/statistics/efficiency-power-cores/&quot; rel=&quot;nofollow&quot;&gt;varies a lot&lt;/a&gt; for the High Performance LINPACK benchmark; it depends on a lot of different factors (memory bandwidth, interconnects, etc.). A pessimistic efficiency would be something like 20%; wildly optimistic would be something like 99%. Taking that into account, anywhere from 250-2500 cores @ 2.5 GHz. I'm not sure there's enough room for that number of cores in a laptop using current form factors, which leads me to believe it would be difficult to forecast. For desktops, Intel Knight's Landing will use 72 Atom cores, and they &lt;em&gt;might&lt;/em&gt; run around 2.0 GHz, guesstimating from the Atom specs; you'd need a slew of &lt;em&gt;those&lt;/em&gt; to reach your target. A &lt;a href=&quot;http://www.parallella.org/board/&quot; rel=&quot;nofollow&quot;&gt;Parallella board is 66 cores&lt;/a&gt;, and their literature claims it reaches 90 GFLOPs. I don't think the technology for this number of cores is mature yet, so...maybe one to two decades? It's far beyond the reach of current technology, and I'm not even sure it would be useful for everyday use.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-17T02:27:28.413" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14163" PostTypeId="2" ParentId="10247" CreationDate="2014-07-17T03:18:14.890" Score="1" Body="&lt;p&gt;Since you mentioned Paraview, you can be interested in do this task with Python. In Matplotlib you can do a &lt;a href=&quot;http://matplotlib.org/1.3.0/examples/mplot3d/hist3d_demo.html&quot; rel=&quot;nofollow&quot;&gt;3D histogram&lt;/a&gt; or a &lt;a href=&quot;http://matplotlib.org/1.3.0/examples/mplot3d/bars3d_demo.html&quot; rel=&quot;nofollow&quot;&gt;3D bar&lt;/a&gt; plot.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2014-07-17T03:18:14.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14164" PostTypeId="1" CreationDate="2014-07-17T08:38:06.733" Score="2" ViewCount="1433" Body="&lt;p&gt;Due to the derivation of an algorithm I tried to use Maple (9.5) to calculate some block matrix expression. Unfortunately Maple seems to ignore the assumption I set on the variable. Lets consider the following minimal example: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;restart; with(LinearAlgebra);&#xA;assume(A,'SquareMatrix'); &#xA;assume(B,'SquareMatrix'); &#xA;E:=Matrix([A, B]); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which results in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E:=\begin{pmatrix} A &amp;amp; B \end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I now try to evaluate &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Transpose(E).E&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I end up with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E:=\begin{pmatrix} A^2 &amp;amp; AB\\AB &amp;amp; B^2 \end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;instead of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E:=\begin{pmatrix} A^TA &amp;amp; A^TB\\B^TA &amp;amp; B^TB \end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is now why does Maple ignore the assumption set to the variables A and B? &lt;/p&gt;&#xA;" OwnerUserId="569" LastEditorUserId="569" LastEditDate="2014-07-17T11:00:31.700" LastActivityDate="2014-07-17T11:36:14.887" Title="Symbolic Computations with Block Matrices in Maple" Tags="|linear-algebra|symbolic-computation|maple|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14165" PostTypeId="5" CreationDate="2014-07-17T09:36:35.900" Score="0" Body="&lt;p&gt;Questions tagged &lt;a href=&quot;/questions/tagged/maple&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;maple&amp;#39;&quot; rel=&quot;tag&quot;&gt;maple&lt;/a&gt; should be concerned with &lt;strong&gt;the use of &lt;a href=&quot;https://en.wikipedia.org/wiki/Maple_(software)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maple&lt;/a&gt; for solving specific problems in computational science&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions about &lt;strong&gt;bugs or problems encountered using Maple&lt;/strong&gt; should be posted to &lt;a href=&quot;http://www.mapleprimes.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MaplePrimes&lt;/a&gt; (&lt;a href=&quot;https://www.maplesoft.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maplesoft&lt;/a&gt;'s own StackExchange-like question site; free registration, but no license information, required.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="21021" LastEditDate="2018-02-02T19:17:07.683" LastActivityDate="2018-02-02T19:17:07.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14166" PostTypeId="4" CreationDate="2014-07-17T09:36:35.900" Score="0" Body="Questions about the computer algebra system Maple." OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-07-17T15:44:23.540" LastActivityDate="2014-07-17T15:44:23.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14167" PostTypeId="2" ParentId="14164" CreationDate="2014-07-17T11:30:18.070" Score="2" Body="&lt;p&gt;This is not precisely an answer to your question, but too long for a comment: In SymPy (called via &lt;code&gt;isympy&lt;/code&gt; to set up the symbols), this works as intended:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = MatrixSymbol('A',n,n)&#xA;B = MatrixSymbol('B',n,n)&#xA;E = BlockMatrix([[A,B]])&#xA;block_collapse(E.T*E)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;results in the desired output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;⎡ T     T  ⎤&#xA;⎢A ⋅A  A ⋅B⎥&#xA;⎢          ⎥&#xA;⎢ T     T  ⎥&#xA;⎣B ⋅A  B ⋅B⎦&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But if I do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;E = Matrix([[A,B]])&#xA;E.T*E&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;⎡A⋅A  A⋅B⎤&#xA;⎢        ⎥&#xA;⎣B⋅A  B⋅B⎦&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;similar to your output (at least SymPy respects that matrix multiplication is non-commutative). This suggests that you should explicitly declare your &lt;code&gt;E&lt;/code&gt; as a &lt;em&gt;block&lt;/em&gt; matrix, but I don't know if this is possible with Maple's &lt;code&gt;LinearAlgebra&lt;/code&gt; (it was with the deprecated &lt;code&gt;linalg&lt;/code&gt; package). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation is to ask your question on Maple's Q&amp;amp;A, &lt;a href=&quot;http://www.mapleprimes.com/&quot; rel=&quot;nofollow&quot;&gt;http://www.mapleprimes.com/&lt;/a&gt; (it's technically off topic on this site, since it about a bug or problem with a specific software package).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;(If this makes you want to switch to SymPy, &lt;a href=&quot;https://github.com/sympy/sympy/wiki/SymPy-vs.-Maple&quot; rel=&quot;nofollow&quot;&gt;this page&lt;/a&gt; may be helpful.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-07-17T11:36:14.887" LastActivityDate="2014-07-17T11:36:14.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14168" PostTypeId="1" CreationDate="2014-07-17T14:52:52.360" Score="1" ViewCount="1010" Body="&lt;p&gt;What is the fastest multithreaded radix sort code out there? I am using radix sort to sort the Morton/Hilbert keys when constructing a tree structure. The number of identical keys is in the range of 10-100. I also need the sorting code to return a permutation index array so that I can sort other elements according to it.&lt;/p&gt;&#xA;" OwnerUserId="9714" LastActivityDate="2014-08-16T19:58:32.457" Title="Fastest multithreaded radix sort" Tags="|sorting|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14169" PostTypeId="1" AcceptedAnswerId="14437" CreationDate="2014-07-17T16:47:45.713" Score="5" ViewCount="1107" Body="&lt;p&gt;I would like to numerically solve a hyperbolic PDE of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial\theta_t}{\partial t}(x,y)+\frac{\partial\left[\theta_t \gamma_t^x\right]}{\partial x}(x,y)+\frac{\partial\left[\theta_t \gamma_t^y\right]}{\partial y}(x,y)=0,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is very similar to the 2D advection equation, except that the partial derivatives w.r.t $x$ and $y$ are products of my dependent variable $\theta_t(x,y)$ with another function $\gamma_t^x(x,y)$ or $\gamma_t^y(x,y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a biologist by training and new to this type of thing, so any advice (at all!) would be much appreciated. I have written a straightforward explicit upwind differencing scheme in Matlab to numerically solve this (difference equation below). My solution almost works (I'm replicating a result from a paper), but the density is not conserved, and much of it seems 'attracted' to the boundaries. I suspect I have implemented my zero-flux boundary conditions incorrectly, and this is what I would like advice on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\theta_{t+\Delta t}(x,y) \approx \theta_{t}(x,y) -\Delta t \left(\frac{\theta_{t}(x+\Delta x,y)\gamma_t^x(x+\Delta x,y)-\theta_{t}(x,y)\gamma_t^x(x,y)}{\Delta x} + \frac{\theta_{t}(x,y+\Delta y)\gamma_t^y(x,y+\Delta y)-\theta_{t}(x,y)\gamma_t^y(x,y)}{\Delta y} \right).$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, because I have a product of two functions that are all dependent on $t$, $x$ and $y$, it seems harder to solve for a zero-flux boundary condition because I have extra unknowns at my ghost points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Clarification&lt;/strong&gt;: Here's how I applied the finite difference product rule (from &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference&quot; rel=&quot;nofollow&quot;&gt;wikipedia&lt;/a&gt;: $\Delta(fg)=f\Delta g + g\Delta f + \Delta g \Delta f $), please correct me if I've used it incorrectly. To keep it short I've just focussed on $x$ and ignored $y$ and dropped the denominator (since everything is over $\Delta x$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial\left[\theta_t \gamma_t^x\right]}{\partial x}(x)\approx\theta_t(x)\left[\gamma_t^x(x+\Delta x)-\gamma_t^x(x)\right]+\gamma_t^x(x)\left[\theta_t(x+\Delta x)-\theta_t(x)\right]+\left[\theta_t(x+\Delta x)-\theta_t(x)\right]\left[\gamma_t^x(x+\Delta x)-\gamma_t^x(x)\right].$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After expanding brackets I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$=\theta_t(x)\gamma_t^x(x+\Delta x)-\theta_t(x)\gamma_t^x(x)+\theta_t(x+\Delta x)\gamma_t^x(x)-\theta_t(x)\gamma_t^x(x)&#xA;-\theta_t(x)\gamma_t^x(x+\Delta x) -\theta_t(x+\Delta x)\gamma_t^x(x)+\theta_t(x+\Delta x)\gamma_t^x(x+\Delta x)+\theta_t(x)\gamma_t^x(x),$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And after cancelling I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$=\theta_t(x+\Delta x)\gamma_t^x(x+\Delta x)-\theta_t(x)\gamma_t^x(x).$&lt;/p&gt;&#xA;" OwnerUserId="9741" LastEditorUserId="9741" LastEditDate="2014-07-18T08:25:31.893" LastActivityDate="2017-02-03T09:09:22.597" Title="Advice on numerical solution for 2D hyperbolic PDE with zero flux boundary conditions" Tags="|matlab|finite-difference|boundary-conditions|hyperbolic-pde|advection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14170" PostTypeId="2" ParentId="14169" CreationDate="2014-07-17T17:32:44.843" Score="0" Body="&lt;p&gt;If we expand the derivatives, the new equation reads&#xA;$$\frac{\partial\theta(x,y)}{\partial t} + \theta(x,y)\frac{\partial \gamma^x(x,y)}{\partial x} + \gamma^x \frac{\partial \theta(x,y)}{\partial x} + \theta(x,y)\frac{\partial \gamma^y(x,y)}{\partial y} + \gamma^y \frac{\partial \theta(x,y)}{\partial y} = 0 \enspace .$$&#xA;If we know use discretization scheme that you proposed, we get&#xA;$$\theta_{t+\Delta t}(x,y) = \theta_{t}(x,y) - \Delta t\left[\frac{\theta_t(x,y)\gamma_t^x(x+\Delta x,y) + \gamma_t^x(x,y)\theta_t(x+\Delta x, y) - 2\theta_t(x,y)\gamma_t^x(x,y)}{\Delta x} + \frac{\theta_t(x,y)\gamma_t^y(x, y + \Delta y) + \gamma_t^y(x,y)\theta_t(x, y+\Delta y)  - 2\theta_t(x,y)\gamma_t^y(x,y)}{\Delta y}\right] \enspace .$$&#xA;I think that you are not taking into account the &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference#Rules_for_calculus_of_finite_difference_operators&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;product rule&lt;/em&gt; of finite differences&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also mentioned that you might be applying the boundary conditions in a wrong way. How are you applying the boundary conditions?&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2014-07-18T02:32:29.950" LastActivityDate="2014-07-18T02:32:29.950" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14171" PostTypeId="2" ParentId="14168" CreationDate="2014-07-17T17:43:11.050" Score="0" Body="&lt;p&gt;Have you looked at &lt;a href=&quot;http://www.p4est.org/&quot; rel=&quot;nofollow&quot;&gt;p4est&lt;/a&gt;? It might have the kind of code you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-07-17T17:43:11.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14172" PostTypeId="1" CreationDate="2014-07-17T18:48:49.950" Score="5" ViewCount="1426" Body="&lt;p&gt;I would like to determine a temporal derivative and a Laplacian by the finite differences method to solve the heat equation in a 1-dimensional case. My aim is to get the sources term that is why I need to get all left terms of the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I plot the sources term as a function of time, the result is surprising... very far from expected. Is this a problem with the np.reshape?  Thanks for help ;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#profil1D_theta.shape[0] : time&#xA;#profil1D_theta.shape[1] : space&#xA;&#xA;#profil1D_theta_reshape.shape                                                                                                                                                           &#xA;#Out[986]: (328, 411)&#xA;&#xA;T_tempo = 1./115. #time period&#xA;&#xA;#Time derivative&#xA;&#xA;dt_theta_list = []&#xA;&#xA;for i in range(profil1D_theta.shape[0]-1):&#xA;    for j in range(profil1D_theta.shape[1]-1):&#xA;        dt_theta = (profil1D_theta[i+1,j] - profil1D_theta[i,j])/T_tempo&#xA;        dt_theta_list.append(dt_theta)&#xA;&#xA;dt_theta = np.asarray(dt_theta_list)&#xA;dt_theta_reshape = dt_theta.reshape(328,411)&#xA;&#xA;&#xA;#Laplacian&#xA;&#xA;T_spatial = 0.000021661 #spatial period&#xA;&#xA;dx2test_theta_list =[]&#xA;for i in range(0,profil1D_theta.shape[0]-1):&#xA;    for j in range(0,profil1D_theta.shape[1]-1):&#xA;        dxtest_theta = (profil1D_theta[i,j+1] - 2*profil1D_theta[i,j] + profil1D_theta[i,j-1])/T_spatial**2&#xA;        dx2test_list.append(dxtest_theta)&#xA;&#xA;dx2test_theta = np.asarray(dx2test_theta_list)&#xA;&#xA;dx2_theta_reshape = dx2test_theta.reshape(328,411)&#xA;&#xA;&#xA;&#xA;rho = 7850. &#xA;C = 500. &#xA;k = 42. &#xA;D = k/(rho*C) &#xA;e = 3.*0.001 &#xA;b = 6.*0.001 &#xA;Boltz = 5.679373*10**(-8) &#xA;Emissivite_echant = 0.97 #&#xA;T0 = 273.15 + 26. &#xA;h = 5.&#xA;&#xA;tau_th = rho*C*e/(2*h+8*Boltz*Emissivite_echant*T0**3)&#xA;&#xA;&#xA;#Sources&#xA;&#xA;profil1D_theta_reshape = profil1D_theta[0:328,0:411]&#xA;&#xA;w1_chaleur_list=[]&#xA;&#xA;for i in range(0,profil1D_theta_reshape.shape[1]):&#xA;    for j in range(0,profil1D_theta_reshape.shape[0]):&#xA;        w_chaleur1D = rho*C*(dt_theta_reshape[j,i]+1./(tau_th)*profil1D_theta_reshape[j,i] - D*dx_theta_reshape[j,i])&#xA;        w1_chaleur_list.append(w_chaleur1D)&#xA;&#xA;d,e = dt_theta_reshape.shape&#xA;w1_chaleur = np.asarray(w1_chaleur_list)&#xA;w1_chaleur_reshape = np.reshape(w1_chaleur,(e,d))&#xA;&#xA;clim_min = 0&#xA;clim_max = 1*10**6&#xA;&#xA;plt.imshow(w1_chaleur_reshape);plt.clim([clim_min,clim_max]);plt.colorbar();&#xA;plt.ylabel(r'Axe longitudinal')&#xA;plt.xlabel(r'Temps (images)')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9742" LastEditorUserId="23136" LastEditDate="2017-02-02T11:57:42.250" LastActivityDate="2017-02-02T11:57:42.250" Title="Python - calculation time derivative and laplacien by finite differences" Tags="|finite-difference|python|numpy|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14173" PostTypeId="1" AcceptedAnswerId="14176" CreationDate="2014-07-17T19:00:14.863" Score="3" ViewCount="450" Body="&lt;p&gt;I am struggling with an equation that represents the Weak form of Galerkin method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \phi^{T}F(\textbf{u})\sim \int_{\Omega}^{ } \phi.f_{0}(\mathit{u},\nabla \mathit{u}) + \nabla\phi:f_{1}(\mathit{u},\nabla \mathit{}u) = 0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the pointwise functions $f_{0}$ and $f_{1}$ capture the problem physics. After discretizing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F(\textbf{u}) \sim \sum_{e}^{} \Large \varepsilon _{e}^{T} \normalsize &#xA;\left[ B^{T}Wf_{0}(u^{q},\nabla u^{q}) +   \sum_{k}^{}D_{k}^{T}Wf_{1}^{k}(u^{q},\nabla u^{q}) \right ]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u^{q}$ is the vector field evaluations at the set $q$ of quadrature points on an element, $W$ is the diagonal matrix of the quadrature weights, B and D are basis function matrices and $\Large\varepsilon _{e}$ is the element restrictor operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand these equations, in particular the first one. Does anyone knows how this was derived? At least the first one, since it represents the weak formulation that I must really understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="9713" LastEditorUserId="9713" LastEditDate="2014-07-18T12:06:07.817" LastActivityDate="2014-07-18T12:06:07.817" Title="Finite Elements Weak Formulation generalization" Tags="|pde|finite-element|fluid-dynamics|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14174" PostTypeId="1" AcceptedAnswerId="14175" CreationDate="2014-07-17T20:29:13.843" Score="3" ViewCount="291" Body="&lt;p&gt;These days, one can have 64 cores in a single node. I wonder how well the dense matrix-matrix product (&lt;code&gt;SGEMM&lt;/code&gt; and &lt;code&gt;DGEMM&lt;/code&gt;) scales with multiple CPUs/cores?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to find some relevant benchmarks, but couldn't.&lt;/p&gt;&#xA;" OwnerUserId="9743" LastEditorUserId="9743" LastEditDate="2014-07-17T20:57:51.453" LastActivityDate="2014-07-17T20:59:03.747" Title="How does matrix-matrix product scale with multiple CPUs?" Tags="|linear-algebra|parallel-computing|matrix|blas|benchmarking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14175" PostTypeId="2" ParentId="14174" CreationDate="2014-07-17T20:59:03.747" Score="5" Body="&lt;p&gt;In comparison with things like matrix vector multiplication (in which there's no cache reuse and everything has to come out of memory), matrix-matrix multiplication allows for lots of cache reuse in a careful implementation.  Performance depends on having a good implementation of BLAS and perhaps depends on how much memory bandwidth is available although is much less of an issue that it was 10-20 years ago.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over the last decade, in my own testing, I've been seeing at least 80% parallel efficiency in DGEMM for reasonably large matrices (say N=5000) on dual socket Xeon servers with up to 8 cores running well tuned BLAS implementations (ATLAS, OpenBlas, MKL, etc.)   I've never had a machine with more than 8 cores that I've tested, so I won't comment further about larger numbers of processors.  Don't expect good parallel efficiency for small matrices (even N=1000 is small for this.)  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-07-17T20:59:03.747" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14176" PostTypeId="2" ParentId="14173" CreationDate="2014-07-17T21:03:39.553" Score="4" Body="&lt;h2&gt;Continuous weak form&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Though I think the weak form is more fundamental, suppose we start with the strong divergence-form representation for a (first- or) second-order quasilinear PDE: find the $m$-component solution $u \in R^m(\Omega)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ -\nabla \cdot f_1(u,\nabla u) + f_0(u,\nabla u) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on the domain $\Omega \subset R^d$ with (for simplicity) homogeneous Dirichlet boundary condition $u|_{\partial \Omega} = 0$.  The function $f_1 : R^m \times R^{m\times d} \to R^{m\times d}$ defines a &quot;conservative&quot; flux and $f_0 : R^m \times R^{m\times d} \to R^m$ is a source term (including non-conservative transport).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the strong form above is true, it must also be the case that for all test functions $\phi \in R^m(\Omega)$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \phi \cdot \Big(- \nabla \cdot f_1 + \cdot f_0 \Big) = 0 . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integrating by parts and using the fact that $u = 0$ on the boundary, we see that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \nabla \phi :\, f_1 + \phi \cdot f_0 = 0,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is the equation in your question.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Discretization&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When we discretize, we choose a finite number of test functions for which to satisfy the weak form.  Each test function produces one equation.  The solution space is also discretized, typically with the same number of degrees of freedom (Galerkin methods equate the test space with the ansatz space).  If we express the test and trial functions in a basis $\{ e_1, e_2, \dotsc, e_n \} \subset R^m(\Omega)$, we have a set of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \nabla e_i :\, f_1(u_h,\nabla u_h) + e_i \cdot f_0(u_h, \nabla u_h) = 0, \quad i \in \{1, \dotsc, n \} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the discrete solution $u_h$ is expressed in the same basis, $u_h =\sum_i u^i e_i$.  We can enumerate the left side of the above equations as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F^i(\mathbf u) = 0, \quad i \in \{1, \dotsc, n\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf u = \{u^1, \dotsc, u^n\} \in R^n$, forming a vector $\mathbf F = \{F^1,\dotsc,F^n\} \in R^n$.  Note that we have selected a basis and now deal only with components.  The discrete equations are thus completely represented by the (linear or nonlinear) function $\mathbf F : R^n \to R^n$, mapping discrete vectors $\mathbf u$ in the ansatz space to residual vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving $\mathbf F(\mathbf u)=0$ is equivalent to saying&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf\phi^T \mathbf F(\mathbf u) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all test functions $\mathbf\phi \in R^n$.  Each of these discrete equations is a statement of the continuous weak form for $\phi_h = \sum_i \phi^i e_i$ and $u_h = \sum_i u^i e_i$.  This relationship inspires my notation that the discrete form corresponds ($\sim$) to the continuous equation&#xA;$$\mathbf\phi^T \mathbf F(\mathbf u) \sim \int_\Omega \nabla \phi_h :\, f_1(u_h,\nabla u_h) + \phi_h \cdot f_0(u_h, \nabla u_h) = 0 .$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Implementation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In practice, the integrals are usually evaluated by quadrature on each element (and the finite-element basis functions $e_i$ have compact support only on the adjacent elements).  I like the notation in the second equation of your question because it highlights the linear and nonlinear components and corresponds to a good way to organize code for a flexible and efficient implementation.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-07-17T21:03:39.553" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14177" PostTypeId="1" CreationDate="2014-07-17T21:21:19.613" Score="1" ViewCount="63" Body="&lt;p&gt;So in my school we have a day where everybody participates in different activities. Each projects can have like 10 members. The whole day is divided in 2 or 3 different blocks, in which the pupils assigned to the activity change.(So in block 1 pupil x takes part in activity a and in the second block in activity d). Before this day starts, we give make lists in which each pupil can tell us his 3 (or 4) favorite activities (he only takes part in two of them, these again are ordered from most &quot;favorite&quot; to least) in which he wants to take part. Now our job is to assign these pupils in a way that we have the best overall satisfaction among the pupils (so everybody did more or less did get his/her chosen activities).What would be a good algorithm to solve this ?(I'm quite familiar with programming (especially java), so the approach would be enough too (although some (pseudo-)code would be great too:) ) Is there any way to do this, apart from calculating such a &quot;satisfaction&quot; value for each possible solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An optional feature would be that if someone can't get in to his/her project, they would get into a similar on (also this sounds kind of sexist, you could for example rate how &quot;female&quot;/&quot;male&quot; this activity is and choose similar activities according to this scale)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm hope this question is fits into this platform, if it is totally off-topic I would be happy to tell me about a more suitable stack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking forward to your suggestions, John&lt;/p&gt;&#xA;" OwnerUserId="9744" LastActivityDate="2014-10-16T23:42:19.977" Title="Algorithm for distributing members over activites (with individual preferences)" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14178" PostTypeId="1" AcceptedAnswerId="14182" CreationDate="2014-07-17T21:44:23.190" Score="0" ViewCount="68" Body="&lt;p&gt;I am fairly new to using SRM, and I am wondering if I should be using srm CLI tools (srmping, srmls, srmcp, etc..) provided by dcache, or bestman2 counter parts (srm-ping, srm-ls).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used to think that either tools should work fine against either dcache, and bestman2 servers, but recently I am seeing this is not the case.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, following works fine..&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; srm-ping srm://cms-se.sdfarm.kr:8443/srm/v2/server&#xA;SRM-PING: Thu Jul 17 21:32:09 UTC 2014  Calling SrmPing Request...&#xA;versionInfo=v2.3.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But not following..&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; srmping srm://cms-se.sdfarm.kr:8443/srm/v2/server&#xA;ERROR org.dcache.srm.client.SRMClientV2 - srmPing : try # 0 failed with error The AXIS engine could not find a target service to invoke!  targetService is managerv2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I don't know which endpoints are dcache v.s bestman2, and I want my script to be generic enough, should I be using bestman2 clients instead of dcache counterparts? Or is this more complicated than that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7467" LastActivityDate="2014-07-18T15:40:14.810" Title="Should I use bestman2 or dcache CLIs (srmping, srmls, etc..)" Tags="|hpc|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14180" PostTypeId="1" AcceptedAnswerId="14265" CreationDate="2014-07-18T12:00:36.903" Score="6" ViewCount="2961" Body="&lt;p&gt;After reading three books about finite element method, with two of them covering also finite volume and grid generation, I found myself lost when I have to discuss these topics with library developers scientists that use an advanced mathematical language (for example, they usually encompass the discussion with Sobolev and Hilbert spaces).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is this: What reference books, beyond the fundamental and standard finite element books, are recommended ?&lt;/p&gt;&#xA;" OwnerUserId="9713" LastEditorUserId="1804" LastEditDate="2014-07-27T13:20:29.000" LastActivityDate="2014-07-27T13:20:29.000" Title="Books on mathematical foundation of finite element methods" Tags="|pde|finite-element|reference-request|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14181" PostTypeId="2" ParentId="14180" CreationDate="2014-07-18T14:23:56.437" Score="3" Body="&lt;p&gt;For a little more mathematical rigor, I like some sections in &quot;&lt;em&gt;Theoretical Numerical Analysis: A Functional Analysis Framework&lt;/em&gt;&quot; by Atkinson and Han.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-07-18T14:23:56.437" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14182" PostTypeId="2" ParentId="14178" CreationDate="2014-07-18T15:40:14.810" Score="2" Body="&lt;p&gt;SRM is a standard &lt;a href=&quot;https://sdm.lbl.gov/srm-wg/doc/SRM.v2.2.html&quot; rel=&quot;nofollow&quot;&gt;protocol&lt;/a&gt;, so it is suppose to work for all storage implementations.  The only things that differ are the protocol implementations.  In addition to bestman2 and dcache srm tools, there is also another popular implementation, lcg-util (lcg-cp, lcg-ls...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know why srmping and srm-ping do not work the same.  To be honest, I have always used lcg-utils.  I suspect that you should just find an SRM tool that works most of the time, and use it.  Or ask around to see what other people are using.  I know RSV uses bestman2 tools, so probably safe to use those.  As with all things, if someone isn't using it, it's probably broken.  And RSV is using the bestman2 tools.&lt;/p&gt;&#xA;" OwnerUserId="7466" LastActivityDate="2014-07-18T15:40:14.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14183" PostTypeId="1" AcceptedAnswerId="14184" CreationDate="2014-07-18T16:22:13.053" Score="0" ViewCount="222" Body="&lt;p&gt;I have a MIP which I know the solution almost for certain. I want to use gurobi to prove that the true solution (even if it is not the one i provide) shall not lie more than 0.5% deviated from the solution I gave. I believe that simply keep cutting without branching would possibly save much more time. Do you know a way that I could simply do cutting without branching in gurobi? thank you! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code performance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Changed value of parameter LogFile to &#xA;   Prev: gurobi.log   Default: &#xA;&#xA;Changed value of parameter MIPFocus to 3&#xA;   Prev: 0   Min: 0   Max: 3   Default: 0&#xA;&#xA;Changed value of parameter Cuts to 3&#xA;   Prev: -1   Min: -1   Max: 3   Default: -1&#xA;&#xA;Optimize a model with 1794 rows, 673 columns and 4180 non zeros&#xA;&#xA;Found heuristic solution: objective -22.8549&#xA;&#xA;Presolve removed 18 rows and 17 columns&#xA;Presolve time: 0.01s&#xA;Presolved: 1776 rows, 656 columns, 4464 nonzeros&#xA;&#xA;Loaded MIP start with objective -342.641&#xA;&#xA;Variable types: 592 continuous, 64 integer (64 binary)&#xA;Presolved: 1776 rows, 656 columns, 4464 nonzeros&#xA;&#xA;Root relaxation: objective -6.775689e+02, 682 iterations, 0.02 seconds&#xA;&#xA;    Nodes    |    Current Node    |     Objective Bounds      |     Work&#xA; Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time&#xA;&#xA;     0     0 -677.56892    0   64 -342.64109 -677.56892  97.7%     -    0s&#xA;     0     0 -666.45290    0   72 -342.64109 -666.45290  94.5%     -    0s&#xA;     0     0 -658.68050    0   72 -342.64109 -658.68050  92.2%     -    1s&#xA;     0     0 -540.92023    0   72 -342.64109 -540.92023  57.9%     -    3s&#xA;     0     0 -503.36031    0   72 -342.64109 -503.36031  46.9%     -    4s&#xA;     0     0 -485.13025    0   72 -342.64109 -485.13025  41.6%     -    6s&#xA;     0     0 -472.73790    0   72 -342.64109 -472.73790  38.0%     -    8s&#xA;     0     0 -461.23185    0   72 -342.64109 -461.23185  34.6%     -    9s&#xA;     0     0 -453.99476    0   72 -342.64109 -453.99476  32.5%     -   10s&#xA;     0     0 -452.23014    0   72 -342.64109 -452.23014  32.0%     -   10s&#xA;     0     3 -452.23014    0   72 -342.64109 -452.23014  32.0%     -   11s&#xA;   642   586 -397.07656   12   54 -342.64109 -429.76289  25.4%   120   15s&#xA;  1425  1290 -397.34606   11   60 -342.64109 -422.53417  23.3%   114   20s&#xA;  1716  1553 -382.83438   18   72 -342.64109 -420.42709  22.7%   111   25s&#xA;  1727  1560 -376.17473   16   72 -342.64109 -420.42709  22.7%   110   30s&#xA;  1733  1564 -410.28764   10   72 -342.64109 -420.42709  22.7%   110   35s&#xA;  1744  1571 -382.83438   18   72 -342.64109 -420.42709  22.7%   109   40s&#xA;  1750  1577 -412.59771   12   69 -342.64109 -416.84728  21.7%   113   45s&#xA;  1817  1602 -380.32997   19   60 -342.64109 -404.73090  18.1%   120   50s&#xA;  2618  2045 -375.99924   18   62 -342.64109 -391.32863  14.2%   126   55s&#xA;  3159  2315 -369.40052   22   59 -342.64109 -386.33088  12.8%   127   60s&#xA;  3808  2595 -362.27693   20   60 -342.64109 -382.29310  11.6%   127   65s&#xA;  4503  2903 -350.90325   24   54 -342.64109 -379.52932  10.8%   126   71s&#xA;  4895  3078 -349.90847   23   55 -342.64109 -378.33598  10.4%   126   78s&#xA;  5339  3242 -363.26836   21   59 -342.64109 -376.77299  10.0%   126   80s&#xA;  6421  3664 -366.32746   21   56 -342.64109 -374.20072  9.21%   126   85s&#xA;  7560  4450 -357.93456   21   59 -342.64109 -371.61876  8.46%   126   90s&#xA;  8849  5297 -355.57657   21   59 -342.64109 -369.33074  7.79%   125   95s&#xA; 10004  6042 -357.02223   24   55 -342.64109 -367.63772  7.30%   124  100s&#xA; 11274  6819 -352.14570   23   55 -342.64109 -365.95440  6.80%   122  105s&#xA; 12362  7437 -357.95155   22   55 -342.64109 -364.73335  6.45%   122  110s&#xA; 13134  7882 -352.18831   25   47 -342.64109 -363.91508  6.21%   121  115s&#xA; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7891" LastEditorUserId="19158" LastEditDate="2016-02-04T16:25:45.017" LastActivityDate="2016-02-04T16:25:45.017" Title="prove optimality only by cutting without branching (gurobi)" Tags="|mixed-integer-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14184" PostTypeId="2" ParentId="14183" CreationDate="2014-07-18T19:58:32.947" Score="0" Body="&lt;p&gt;I don't think it's possible, from looking at the &lt;a href=&quot;http://www.gurobi.com/documentation/5.6/reference-manual/attributes&quot; rel=&quot;nofollow&quot;&gt;list of controllable attributes on Gurobi's website&lt;/a&gt;, and the &lt;a href=&quot;http://www.gurobi.com/documentation/5.6/reference-manual/parameters&quot; rel=&quot;nofollow&quot;&gt;list of controllable attributes on Gurobi's website&lt;/a&gt;. You can turn off cuts, but not branching, probably because branch-and-bound will still yield a convergent algorithm, whereas I'm not aware of a general purpose mixed-integer linear programming algorithm that only cuts. Cuts also make your LP relaxations bigger, whereas branching (without cutting) makes the smaller, at the cost of adding subproblems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might instead try supplying a good initial guess (if you know one; your question suggests you do) and then use Gurobi's parameter tuning heuristics as a first pass. The information from the tuning heuristics could then inform better choices of parameters that control branching direction (and variable choice) and cut selection.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-18T19:58:32.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14185" PostTypeId="2" ParentId="14180" CreationDate="2014-07-18T21:29:08.623" Score="3" Body="&lt;p&gt;Assuming that you already have a background in partial differential equations and basic numerical analysis, functional analysis is helpful to understand FEM theory.  From personal experience, it's a tedious topic and is a bit difficult to really master autodidactically.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said this, I have gained some insights from books like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898716144&quot; rel=&quot;nofollow&quot;&gt;Understanding and Implementing the Finite Element Method&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0471504599&quot; rel=&quot;nofollow&quot;&gt;Introductory Functional Analysis&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1420086626&quot; rel=&quot;nofollow&quot;&gt;Applied Calculus of Variations for Engineers&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0894645854&quot; rel=&quot;nofollow&quot;&gt;Applied Functional Analysis and Variational Methods in Engineering&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each book helped me in some way, shape, or form to understand the mathematics behind the finite element method.  However, none of them really provided a &quot;perfect&quot; or &quot;complete&quot; understanding to me.  This is in no way an exhaustive list of &quot;good references&quot; and I do not think there is any &quot;canonical books&quot; on the subject either.  But at least, these are some books that helped me and I hope they can help you as well.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-07-18T21:29:08.623" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14186" PostTypeId="1" CreationDate="2014-07-18T22:07:22.953" Score="1" ViewCount="825" Body="&lt;p&gt;Is there any fast way to compute the barycentric Lagrange interpolation using matlab?&#xA;something more faster than using repmat instead of for loops&lt;/p&gt;&#xA;" OwnerUserId="9754" LastActivityDate="2014-07-18T22:07:22.953" Title="Fast way to compute barycentric lagrange interpolation" Tags="|matlab|interpolation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14187" PostTypeId="2" ParentId="14177" CreationDate="2014-07-18T22:28:36.897" Score="1" Body="&lt;p&gt;Although you could probably formulate and solve this problem using some sort of optimization algorithm (a mixed-integer formulation immediately comes to mind, and would be expensive), you might first try looking at algorithms for solving variants of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Stable_marriage_problem#Similar_problems&quot; rel=&quot;nofollow&quot;&gt;stable marriage problem&lt;/a&gt;. Your problem sounds close to something that might look like the &lt;a href=&quot;http://en.wikipedia.org/wiki/Hospital_resident#Matching_algorithm&quot; rel=&quot;nofollow&quot;&gt;hospitals/residents problem&lt;/a&gt;, but it would be difficult to know for sure without asking more questions. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-18T22:28:36.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14188" PostTypeId="1" CreationDate="2014-07-19T03:59:53.473" Score="1" ViewCount="245" Body="&lt;p&gt;I'm using interior point algorithm for solving a nonlinear optimization problem and want to provide Hessian of the Lagrangian as part of fmincon to speed up the process (running couple of thousand variations for different sets of parameters)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The workflow that I'm designing has the (nonlinear) objective function, form of linear and nonlinear constraints each in separate function files. Since i'm running different variations of the problems, sometimes there may be zero nonlinear equality &amp;amp; zero nonlinear inequality constraints and in other scenarios, nonzero number of nonlinear equality/inequality constraints. In short the number of nonlinear constraints are dynamic and stored in the 2 variables &quot;NumOfNonLinEqConstr&quot; and &quot;NumOfNonLinInEqConstr&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have taken a look at &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/symbolic-math-toolbox-calculates-gradients-and-hessians.html#brwgbls&quot; rel=&quot;nofollow&quot;&gt;Generating Hessian using Symbolic toolbox&lt;/a&gt; and few other web-pages but cannot see an example where the Hessian of the Lagrangian is constructed for dynamic number of constraints. In the referred matlab webpage example, like in one variation I tried replacing 10 with NumOfNonLinInEqConstr bu it doesn't work as matlabFunction does not work on cell data type. Can anybody provide a working example for constructing hessian of the lagrangian with dynamic number of nonlinear constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relevant code lines from the matlab link provided below: My question is how could I avoid hard-coding 10 in the below code (and also manually add the product of hessian of the constraint and lambda.ineqnonlin 10 times)?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hessc = cell(1, 10);&#xA;for i = 1:10&#xA;    hessc{i} = jacobian(gradc(:,i),x);&#xA;end&#xA;&#xA;&#xA;for i = 1:10&#xA;    ii = num2str(i);&#xA;    thename = ['hessc',ii];&#xA;    filename = [currdir,thename,'.m'];&#xA;    matlabFunction(hessc{i},'file',filename,'vars',{x});&#xA;end&#xA;&#xA;&#xA;function H = hessfinal(X,lambda)&#xA;%&#xA;% Call the function hessenergy to start&#xA;H = hessenergy(X);&#xA;&#xA;% Add the Lagrange multipliers * the constraint Hessians&#xA;H = H + hessc1(X) * lambda.ineqnonlin(1);&#xA;H = H + hessc2(X) * lambda.ineqnonlin(2);&#xA;H = H + hessc3(X) * lambda.ineqnonlin(3);&#xA;H = H + hessc4(X) * lambda.ineqnonlin(4);&#xA;H = H + hessc5(X) * lambda.ineqnonlin(5);&#xA;H = H + hessc6(X) * lambda.ineqnonlin(6);&#xA;H = H + hessc7(X) * lambda.ineqnonlin(7);&#xA;H = H + hessc8(X) * lambda.ineqnonlin(8);&#xA;H = H + hessc9(X) * lambda.ineqnonlin(9);&#xA;H = H + hessc10(X) * lambda.ineqnonlin(10);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2014-07-19T03:59:53.473" Title="Generating Hessian of the Lagrangian with dynamic number of nonlinear constraints in Fmincon" Tags="|matlab|nonlinear-programming|constrained-optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14189" PostTypeId="1" CreationDate="2014-07-19T05:23:54.993" Score="1" ViewCount="271" Body="&lt;p&gt;I'm trying to generate start points for my optimization problem in Matlab. At this point Im not worried about feasibility but only a fast way to generate the points from which I could test the performance of different algorithms available within fmincon. Though am not worried about feasibility I don't want to use random number generator and would prefer to use a more &quot;systematic&quot; approach as below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a simple example, assume I have 2 decision variables x1 and x2 and I can provide the min/max ranges for each of these 2 variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xlow=[2,9];% 2 represents the min possible value of decision variable x1 and 9 represent the minimum possible value of x2;&#xA;xhigh=[5,16];% 5 represents the max possible value of decision variable x1 and so on;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One start point that I can use is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Random_Point1=(x1ow+xhigh)/2;% centre of the rectangle formed by max/min of decision variables;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After running the optimization algorithm with above start point, let's assume I want to generate more start points. Now I can generate 4 additional start points based on the mid-point Random_Point1 and the 4 original vertices (another way to think about this is by dividing the original rectangle in to 4 equal size smaller rectangles and finding the centre of each of the rectangle)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Random_Point21 = (Random_Point1 + [xlow(1),xlow(2)])/2;&#xA;Random_Point22 = (Random_Point1 + [xlow(1),xhigh(2)])/2;&#xA;Random_Point23 = (Random_Point1 + [xhigh(1),xlow(2))/2;&#xA;Random_Point24 = (Random_Point1 + [xhigh(1),xhigh(2)])/2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I again run the fmincon with above 4 start points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After looking at the results, I decide to generate more start points. This time I can generate 16 start points by using the previously generated above start points. The way to think about this is dividing the original rectangle in to 16 equal pieces along the respective axis and finding the centre of each of the 16 rectangles&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would continue the process of generating start points and running fmincon till my solution is deemed satisfactory (I have some criteria on when I want to stop optimization process)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what approach should I use to generate these start points in the most efficient manner (quick). Whatever I wrote (similar to above hard-coding) is definitely not the right approach. I guess I need to use recursion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stating the problem in general terms:-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Assume I have k decision variables and for each decision variable I have the min/max ranges&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) I want to start with generating the mid-point (easy) and then in first iteration generate 2^k points, in 2nd iteration generate 2^(2k) points, in 3rd iteration generate 2^(3K) points and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, in my application the number of decision variable is not expected to be large (may be 3 or maximum 4)..the complexity I have is more in functional form of objective rather than # of decision variables. I'm stating this so that there are no worries about generating unpractical number of start points (like I will apply a condition that if in the pth iteration if 2^((p-1)k) was greater than 10K abort the generation process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I do have a working model albeit of another approach where for let's say 2 decision variables with given min/max I want 400 start points. I divide each of the individual decision variables to Square root of 400 and for all the resulting combinations I will have 400 points. I would prefer to use alternate approach and one of them is described above.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2014-07-19T05:23:54.993" Title="Generating start point in a systematic manner for fmincon" Tags="|matlab|nonlinear-programming|constrained-optimization|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14190" PostTypeId="1" CreationDate="2014-07-19T06:55:18.450" Score="1" ViewCount="83" Body="&lt;p&gt;If I am to solve a symmetric eigenvalue system $A=QDQ^T$, where $A\in\mathcal{R}^{n\times n}$ and $n$ is small (in the range 4 - 64); I want all the eigenvectors and eigenvalues; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two major consideration in my design; I wish my implementation to be as fast as possible; I need $Q$ to be as orthogonal possible, however I can tolerate error in eigenvalue.  What would be a good way to go about solving it? target platform is single core x86 processor;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All MKL routines such as sYEV, sYEVD and sYEVR seems to be very slow ( around 10x than expected based on flop rates). Thus the need of implementing it myself; I hope design space&#xA; is large than just tridiagonal reduction kind of methods;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like suggestion on all three aspects, i.e. algorithm,implementation and existing implementation for such problems; &lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2014-07-19T07:40:29.447" Title="eigenvalue of small symmetric matrices" Tags="|numerical-analysis|eigenvalues|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14191" PostTypeId="2" ParentId="14190" CreationDate="2014-07-19T07:40:29.447" Score="2" Body="&lt;p&gt;In my experience the major BLAS libraries are not well optimized for small $n$, but you should not expect to obtain back the full factor of 10. You might have a look at &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt; to see how it does in comparison. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll be L2 cache contained for your larger sizes and L1 contained for the smaller ones. You'll probably need to take this into account to get good performance.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-07-19T07:40:29.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14192" PostTypeId="1" CreationDate="2014-07-19T08:41:53.610" Score="0" ViewCount="1018" Body="&lt;p&gt;I am numerically simulating the Mathieu equation using ODE45 and I have to keep changing the parameters delta and epsilon for each simulation to get the required peiodic solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is the MATLAB code I am running:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function xdot=trial2(t,x) &#xA;delta=0.1045;epsilon=0.0048685;&#xA;xdot=[x(2);(-delta-epsilon*cos(t))*x(1)-0.7*delta*abs(x(1))];&#xA;&#xA;[t,x]=ode45('trial2',[0 10000000],[0;1]); &#xA;hold on; plot(t,x(:,1),'r'); &#xA;clear all;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is taking me 10 minutes for each simulation if I simulate for 10000000 time and it doubles if I increase the time 5 fold. After each simulation, I analyze the result and change the parameters accordingly so that I can come closer to a periodic solution. I have to do this for maybe about 30-40 parameters maybe more. I am running this on a 64 bit desktop with 8gb ram. The program runs out of memory on a 32 bit desktop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is there somehow I can improve my computational time? I might have to go from 10000000 to 50000000, maybe even more. 2) Can you help me out in implementing the parfor loop so that I can run simultaneous simulations for more number of parameters. That might be of some help too.3) WIll it help if I run this on a faster processor than i5-2400 CPU?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9756" LastEditorUserId="123" LastEditDate="2015-04-22T17:24:36.953" LastActivityDate="2015-04-22T17:24:36.953" Title="Searching for periodic solutions of Mathieu equation using MATLAB's ode45 and a crude shooting algorithm" Tags="|matlab|ode|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14194" PostTypeId="2" ParentId="12954" CreationDate="2014-07-19T14:34:54.883" Score="3" Body="&lt;p&gt;You can model a whip as a solid object (solids or beams) in FEA. You'll need a package that does dynamic(transient) response with nonlinear large displacements. The latter is important because the direction of the force would probably change as the whip bends. For a list of packages showing which support these features, see &lt;a href=&quot;http://feacompare.com&quot; rel=&quot;nofollow&quot;&gt;http://feacompare.com&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9757" LastActivityDate="2014-07-19T14:34:54.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14195" PostTypeId="1" AcceptedAnswerId="14205" CreationDate="2014-07-19T19:16:47.197" Score="19" ViewCount="1107" Body="&lt;p&gt;Given a desired state $y_0$ and a regularization parameter $\beta \in \mathbb R$, consider the problem of finding a state $y$ and a control $u$ to minimize a functional&#xA;\begin{equation}&#xA;  \frac{1}{2} \| y - y_0 \|^2 + \frac{\beta}{2} \| u \|^2&#xA;\end{equation}&#xA;subject to the constraint&#xA;\begin{equation}&#xA;  Ay = u.&#xA;\end{equation}&#xA;where for simplicity we can think of $ y, y_0, u \in \mathbb R^n $ and $ A \in \mathbb R^{n \times n} $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forming the Lagrangian, looking for stationary points, and eliminating the control $u$ we get the first-order conditions&#xA;\begin{align*}&#xA;  A^T \lambda &amp;amp;= y_0 - y \\&#xA;  Ay &amp;amp;= \frac{1}{\beta} \lambda&#xA;\end{align*}&#xA;Premultiplying by $A$ in the first equation and $A^T$ in the second, we can write the normal equations&#xA;\begin{align}&#xA;  (I + \beta A A^T) \lambda &amp;amp;= \beta A y_0 \\&#xA;  (I + \beta A^T A) y &amp;amp;= y_0 &#xA;\end{align}&#xA;We can interpret these as single steps of backward Euler approximations to the differential equations&#xA;\begin{align}&#xA;  \frac{\partial \lambda}{\partial b} &amp;amp;= -A A^T \lambda + A y_0, \quad \lambda(0) = 0  \\&#xA;  \frac{\partial y}{\partial b} &amp;amp;= -A^T A y, \quad y(0) = y_0 &#xA;\end{align}&#xA;with pseudotimestep $\beta$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: Is this connection well known? Is it discussed in standard treatments of either timestepping or optimization? (To me, it seems to provide some kind of intuitive connection between them.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea seems simple enough that it has to be well known, but neither searching the literature or talking to people has given me a good source where this is discussed. The closest I've found is a paper by O. Scherzer and J. Weichert (J. Math Imaging Vision 12 (2000) pp. 43-63) which states the connection in the first sentence of the abstract (!) but does not provide any references or explore the connection in any depth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally I am looking for a reference that not only states the connection but also explores some consequences (for example, one could imagine preconditioning an optimization problem with a cheap forward Euler step).&lt;/p&gt;&#xA;" OwnerUserId="336" LastActivityDate="2015-09-14T19:04:32.123" Title="Is it well known that some optimization problems are equivalent to time-stepping?" Tags="|optimization|reference-request|time-integration|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14196" PostTypeId="1" AcceptedAnswerId="14216" CreationDate="2014-07-19T20:41:43.203" Score="5" ViewCount="1077" Body="&lt;p&gt;Are there any open-source auto-differentiation libraries for MATLAB?&lt;/p&gt;&#xA;&lt;p&gt;I am aware of commercial packages such as &lt;a href=&quot;https://tomopt.com/tomlab/products/mad/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tomlab/MAD&lt;/a&gt; and plenty of C++ libraries, but I can't find many more for MATLAB other than &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/15235-automatic-differentiation-for-matlab&quot; rel=&quot;nofollow noreferrer&quot;&gt;Automatic Differentiation for Matlab package on Matlab File Exchange&lt;/a&gt; which isn't a major package.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Added&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As a bare minimum of functionality, I need to be able to handle vectors as variables. In that MathWorks package, I reference, for example, here is a test case:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;val = [1 2 3]; %Some value&#xA;x = myAD(val); %Creates a vector with that value.&#xA;&#xA;x .* [4 5 6]; %This should give the derivatives [4; 5; 6]&#xA;x_inner = sum(x.* x);% To get the inner product.  The derivative here is [2 4 6] here&#xA;%Ideally, the above would simply be x_inner = x * x'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9692" LastEditorUserId="20688" LastEditDate="2020-10-27T18:13:53.650" LastActivityDate="2021-09-26T15:32:16.260" Title="Open source auto-differentiation for MATLAB?" Tags="|matlab|libraries|automatic-differentiation|" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14198" PostTypeId="2" ParentId="14196" CreationDate="2014-07-19T23:23:45.540" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://academics.davidson.edu/math/neidinger/SIAMRev74362.pdf&quot; rel=&quot;nofollow&quot;&gt;This article in SIAM Review&lt;/a&gt; describes how to implement operator-overloading automatic differentiation in MATLAB, and gives a good introduction to the technology.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-19T23:23:45.540" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14199" PostTypeId="2" ParentId="14195" CreationDate="2014-07-20T03:16:41.060" Score="13" Body="&lt;p&gt;While I haven't seen the exact formulation that you have written down here, I keep seeing talks in which people &quot;rediscover&quot; a connection to integrating some transient system, and proceed to write down an algorithm that is algebraically-equilavent to one form or another of an existing gradient descent or Newton-like method, and fail to cite anyone else.  I think it's not very useful because the conclusion is basically that &quot;as long as you take small enough steps, the method eventually converges to a local minimum&quot;.  Well, 2014 marks the 45th anniversary of Philip Wolfe's paper showing how to do this in a principled way.  There is also good theory for obtaining q-quadratic or q-superlinear convergence from pseudotransient continuation and related methods like Levenberg-Marquardt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want an instance of this rediscovery using a Newton-like formulation for solving algebraic equations (i.e., classical pseudotransient continuation) from a mathematician with more than 600 papers (so maybe he'll prove things you find interesting), look at the &quot;Dynamical Systems Method&quot; by A.G. Ramm [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the intuition gained by considering a transient system led to practical algorithms that were either faster or more reliable, I think we'd see highly-cited articles on that subject.  I think it's no mystery that Nocedal and Wright has over 13000 citations while Ramm's book has about 80 (mostly self-citations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] I can advise you not to inform Prof. Ramm that his DSM is algebraically-equivalent to something that has been in countless engineering packages for decades or you may get yourself yelled out of the room. #gradstudentmemories&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2014-07-20T03:26:10.710" LastActivityDate="2014-07-20T03:26:10.710" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14200" PostTypeId="1" CreationDate="2014-07-20T13:35:09.980" Score="3" ViewCount="1770" Body="&lt;p&gt;So I'm working on a fitting algorithm using the levenberg-marquardt algorithm and I'm a bit stumped as to how to handle fixed parameters. Looking around at other code, like the minpack version of the l-m algorithm, it looks like they are just setting the columns of the jacobian for fixed parameters to be 0.0, which makes sense as they are not changing. $J$ is being computed numerically and is working fine if there are not any fixed parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I always get a singular uninvertible matrix for $J^TJ$ when I do this, for example something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[       1005,          0,    -110500],&#xA; [          0,          0,          0],&#xA; [    -110500,          0,    3.013e7]]  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(The 2nd parameter was held fixed in this case...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have some experience with the L-M algo? Any ideas as to how handle the fixed parameters in the jacobian? Should I just create a jacobian that only has the free parameters and patch in the fixed parameters somewhere down the line later? &lt;/p&gt;&#xA;" OwnerUserId="9762" LastActivityDate="2015-11-28T10:08:20.893" Title="Levenberg-marquardt: How to calculate the jacobian with fixed parameters" Tags="|optimization|jacobian|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14201" PostTypeId="1" AcceptedAnswerId="14206" CreationDate="2014-07-19T16:01:17.493" Score="4" ViewCount="286" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Automatically_Tuned_Linear_Algebra_Software&quot; rel=&quot;nofollow&quot;&gt;ATLAS&lt;/a&gt; is an extremely popular linear algebra library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you install ATLAS from source, it tries to automatically tune a number of parameters to give you the fastest code. Does it make sense to do this tuning when installing ATLAS on a cloud server like AWS or GCE? I know that these services often provide shared VMs, and presumably the performance of these VMs can change based on what other VMs on the same physical hardware are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to get ATLAS installed and tuned correctly when installing on a cloud service?&lt;/p&gt;&#xA;" OwnerUserId="4970" OwnerDisplayName="user939259" LastActivityDate="2014-07-20T22:02:32.387" Title="Does it make sense to build ATLAS from source on a cloud machine?" Tags="|performance|cloud-computing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14202" PostTypeId="1" AcceptedAnswerId="14203" CreationDate="2014-07-20T15:38:23.903" Score="2" ViewCount="1381" Body="&lt;p&gt;Is it possible to run Clp on multiple threads using Julia and JuMP?&lt;/p&gt;&#xA;" OwnerUserId="5185" LastEditorUserId="1804" LastEditDate="2014-07-20T16:23:43.867" LastActivityDate="2014-07-20T16:23:43.867" Title="Solving a problem using multiple threads using CLP" Tags="|parallel-computing|software|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14203" PostTypeId="2" ParentId="14202" CreationDate="2014-07-20T16:02:45.603" Score="5" Body="&lt;p&gt;By Clp, I assume that you're referring to the linear programming code that is part of the COIN-OR project:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.coin-or.org/Clp/&quot; rel=&quot;noreferrer&quot;&gt;http://www.coin-or.org/Clp/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clp's primal and dual simplex codes aren't multithreaded so even if you call Clp from within a multithreaded program in Julia, each LP will be solved by a single thread.  It is possible to use the primal-dual interior point method in Clp with a threaded sparse Cholesky factorization routine.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, it has proven to be extremely difficult to get an efficient implementation of the simplex method in parallel.  Parallel implementations of primal-dual interior point method have been more successful primarily because the major work in each iteration of the primal-dual method is the Cholesky factorization of a large sparse matrix and considerable progress has been made on doing those factorizations in parallel.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-07-20T16:02:45.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14204" PostTypeId="2" ParentId="14200" CreationDate="2014-07-20T16:14:26.210" Score="5" Body="&lt;p&gt;There are a couple of options here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't compute elements of the Jacobian corresponding to the fixed parameters and don't include these parameters in your vector of parameters to be fitted.  In your code this would require a different version of the function and Jacobian for each collection of parameters that might be fixed in a particular call to the LM routine.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute all of the elements of $F$ and $J$ but then remove those elements of $J$ corresponding to the fixed parameters.  This allows you to write one routine for computing the Jacobian but then requires you to modify your LM routine so that you can specify which parameters are fixed.  Once you've deleted columns from $J$ corresponding to the fixed parameters, you can compute $J^{T}J$ without the rows and columns of 0's that you've mentioned.  However, the vector $\Delta p$ that you get back as a solution to the linear system will need to have 0's inserted in positions corresponding to the fixed parameters.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could also modify the linear system of equations to force the solution to do what you want.  If parameter $k$ is fixed, then zero out column $k$ and row $k$ of $J^{T}J$, then set the diagonal entry in row $k$, column $k$ to 1.  Finally, make sure that the $k$th entry in the right hand side vector is 0.  This will ensure that $\Delta p_{k}=0$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-07-20T16:14:26.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14205" PostTypeId="2" ParentId="14195" CreationDate="2014-07-20T17:47:59.313" Score="17" Body="&lt;p&gt;As Jed Brown mentioned, the connection between gradient descent in nonlinear optimization and time stepping of dynamical systems is rediscovered with some frequency (understandably, since it's a very satisfying connection to the mathematical mind since it links two seemingly different fields). However, it rarely turns out to be a &lt;em&gt;useful&lt;/em&gt; connection, especially in the context you describe. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In inverse problems, people are interested in solving the (ill-posed) operator equation $F(u)=y^\delta$ with $y^\delta$ not in the range of $F$. (Your optimal control problem can be seen as one instance of it with $F=A^{-1}$ and $y^\delta = y_0$.) Several regularization strategies (such as Tikhonov or Landweber) can be interpreted as a single pseudo-time step of a certain class. The idea is then to use the interpretation of the regularization parameter as a step length to obtain some (adaptive, a posteriori) choice rules for the parameter -- a fundamental problem in inverse problems -- and possibly to make multiple pseudo-time steps to approach the true, unregularized solution (similarly to &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_continuation&quot; rel=&quot;noreferrer&quot;&gt;numerical continuation&lt;/a&gt;). This is sometimes called &lt;em&gt;continuous regularization&lt;/em&gt;, and is usually discussed in the context of level set methods; see, for example, Chapter 6.1 of Kaltenbacher, Scherzer, Neubauer: &lt;em&gt;Iterative Regularization Methods for Nonlinear Ill-Posed Problems&lt;/em&gt; (de Gruyter, 2008). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second context this idea repeatedly crops up in is nonlinear optimization: If you look at a gradient descent step for $\min_x f(x)$, &#xA;$$ x^{k+1} = x^k - \gamma_k \nabla f(x^k),$$&#xA;then you can interpret this as a &lt;em&gt;forward&lt;/em&gt; Euler step for the dynamical system&#xA;$$ \dot x(t) = -\nabla f(x(t)),\qquad x(0) = x^0.$$&#xA;As Jed Brown pointed out, this at first glance yields only the not very surprising observation that this method converges, provided the pseudo-time steps $\gamma_k$ are small enough. The interesting part comes when you look at the dynamical system and ask yourself what properties the continuous solution $x(t)$ of the so-called &lt;em&gt;gradient flow&lt;/em&gt; has (or should have), independent of the gradient descent, and whether that might not lead to more appropriate time stepping (and hence optimization) methods than standard Euler. Some examples off the top of my head:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a natural function space in which the gradient flow lives? If so, your gradient step should be taken from the same space (i.e., the discretization should be conforming). This leads, e.g., to computing Riesz representations of the gradient with respect to different inner products (sometimes called &lt;em&gt;Sobolev gradients&lt;/em&gt;) and, in practice, to preconditioned iterations that converge much faster.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Maybe $x$ should belong not to a vector space, but to a manifold (e.g., symmetric positive definite matrices), or the gradient flow should conserve a certain norm of $x$. In this case, you could try to apply structure-preserving time-stepping schemes (e.g., involving a pull-back with respect to an appropriate Lie group or a geometric integrator).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If $f$ is not differentiable but convex, the forward Euler step corresponds to a subgradient descent method which can be very slow due to step size restrictions. On the other hand, an implicit Euler step corresponds to a &lt;em&gt;proximal point method&lt;/em&gt;, for which no such restrictions apply (and which thus have become very popular in, e.g.,  image processing).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In a similar vein, such methods can be significantly accelerated by extrapolation steps. One way of motivating these is by observing that  standard first-order methods suffer from having to make many small steps close to minimizers, because the gradient directions &quot;oscillate&quot; (think of the standard illustration for why conjugate gradients outperform steepest descent). To remedy this, one can &quot;dampen&quot; the iteration by not solving a first-order dynamical system, but a damped &lt;em&gt;second-order&lt;/em&gt; system:&#xA;$$a_1 \ddot x(t) + a_2 \dot x(t) = -\nabla f(x(t))$$&#xA;for suitably chosen $a_1,a_2$. With the proper discretization, this leads to an iteration (known as &lt;em&gt;Polyak's heavy ball method&lt;/em&gt;) of the form&#xA;$$x^{k+1} = x^k - \gamma_k \nabla f(x^k) + \alpha_k (x^k - x^{k-1})$$&#xA;(with $\gamma_k,\alpha_k$ depending on $a_1,a_2$). Similar ideas exist for proximal point methods, see, e.g., the paper &lt;a href=&quot;http://arxiv.org/pdf/1403.3522.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://arxiv.org/pdf/1403.3522.pdf&lt;/a&gt; by Dirk Lorenz and Thomas Pock.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;(I should add that to my knowledge, in most of these cases the interpretation as a dynamical system was not strictly necessary for the derivation or the convergence proof of the algorithm; one could argue that ideas like &quot;implicit vs. explicit&quot; or Lie derivatives are actually more fundamental than either dynamical systems or gradient descent methods. Still, it never hurts to have another view point to look at a problem from.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; I just stumbled across an excellent example from the second context, where the ODE interpretation is used to deduce properties of Nesterov's extragradient method and suggest improvements: &#xA; &lt;a href=&quot;http://arxiv.org/pdf/1503.01243.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://arxiv.org/pdf/1503.01243.pdf&lt;/a&gt;&#xA;(Note that this is also an example of Jed Brown's point, in that the authors essentially rediscover the point 4 above without apparently being aware of Polyak's algorithm.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2:&lt;/strong&gt; And as an indication how far you can take this, see page 5 of &lt;a href=&quot;http://arxiv.org/pdf/1509.03616v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://arxiv.org/pdf/1509.03616v1.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-09-14T19:04:32.123" LastActivityDate="2015-09-14T19:04:32.123" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14206" PostTypeId="2" ParentId="14201" CreationDate="2014-07-20T22:02:32.387" Score="4" Body="&lt;p&gt;Sure.  For example, with Amazon EC2, suppose that you've requested a C3.xlarge instance (which has Xeon E5-2680 Ivy Bridge processors), then you can count on running on this hardware, so it would be good to have versions of the libraries optimized for this hardware.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, note that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your OS will be running as a virtual machine under the control of some sort of hypervisor.  This can cause performance variations that might effect the timing that is done during the installation of ATLAS.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It's likely that the &quot;turbo boost&quot; feature will be setup rather than leaving the processor always in the highest performance mode.  ATLAS won't install unless this is turned off.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would suggest contacting Clint Whaley (the author of ATLAS) to ask about experiences in installing ATLAS on EC2 instances.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-07-20T22:02:32.387" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14207" PostTypeId="1" AcceptedAnswerId="14208" CreationDate="2014-07-21T03:14:29.080" Score="3" ViewCount="405" Body="&lt;p&gt;I need to find the root of a nonlinear system (which comes out of collocation, so I will change the order to test).  I will likely have about 50-300 variables, and the Jacobian is going to be somewhere between 1/9 and 1/2 dense, and calculated using auto-differentiation.&lt;/p&gt;&#xA;&lt;p&gt;What do people suggest for the linear system of this type?  I know about trilinos, PETSc, and sundials, but don't know the other alternatives or have exposure to them.  I realize that the large solvers are overkill due to the relative lack of sparsity in the example, but can it hurt?&lt;/p&gt;&#xA;&lt;p&gt;My preference is for a C++-style interface, but I can deal with C if necessary.&lt;/p&gt;&#xA;" OwnerUserId="9692" LastEditorUserId="20688" LastEditDate="2021-12-18T15:14:03.743" LastActivityDate="2021-12-18T15:14:03.743" Title="Suggestions for open source C++ library for medium scale non-linear solver" Tags="|nonlinear-equations|c++|libraries|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14208" PostTypeId="2" ParentId="14207" CreationDate="2014-07-21T04:02:31.470" Score="5" Body="&lt;p&gt;I am a PETSc developer so take my suggestion with a grain of salt, but I would use PETSc because&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the problem sizes are large enough that execution overhead should be minimal,&lt;/li&gt;&#xA;&lt;li&gt;you can trivially switch between various sparse and dense solvers (1/2 sparse should be treated as dense, but it might pay off to use a sparse solver for 1/9 at your sizes),&lt;/li&gt;&#xA;&lt;li&gt;a suite of globalization techniques quick experimentation, which is important for difficult problems, and&lt;/li&gt;&#xA;&lt;li&gt;you get monitoring and debugging tools so you can debug and understand convergence without writing your own scaffolding.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-07-21T04:02:31.470" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14209" PostTypeId="2" ParentId="14207" CreationDate="2014-07-21T04:10:50.933" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What do people suggest for the linear system of this type? I know about trilinos, petsc, and sundials, but don't know the other alternatives or have exposure to them.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Generally speaking PETSc, Trilinos, and KINSOL (from SUNDIALS) are the best-of-breed when it comes to scalability. From an ecosystem standpoint, PETSc seems most flexible, since it does not try to take over &lt;code&gt;main&lt;/code&gt; or attempt to be a framework; many packages (SLEPc, libmesh, deal.II, PyClaw, MOOSE, TAO (before it was folded into PETSc), FEniCS) build on top of PETSc successfully.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trilinos has more of a framework/ecosystem feel, as Trilinos consists of many packages; packages are still built on top of Trilinos, like FEniCS and deal.II. From mailing list traffic, the PETSc interface to FEniCS tends to get more use than the Trilinos interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;KINSOL is the least flexible. It's basically a Newton solver or a fixed-point iteration solver, with a small number of interfaces to iterative linear solvers. It's probably the easiest to learn completely, but doesn't have the runtime flexibility that something like PETSc does. Choosing between KINSOL and PETSc, I'd pick PETSc; the code maintenance and quality alone are better, and PETSc is way more powerful.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I realize that the large solvers are overkill due to the relative lack of sparsity in the example, but can it hurt?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Probably not. There's some overhead due to PETSc initializing data structures, for instance, but I've only run into two people who have actually complained about it with good reason (and in one case, PETSc developers did a good job of reducing the PETSc memory footprint). JedBrown would know better; I'm just a PETSc user.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-21T04:10:50.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14211" PostTypeId="1" CreationDate="2014-07-21T11:10:59.387" Score="2" ViewCount="1014" Body="&lt;p&gt;I am writing a global climate simulation software system. My idea is the following :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the top of everything, I interface to the OS using D, a very powerful language for compile time code generation. The D code gets environmental variables, finds necessary input data, and then according to the config files, initiates Lua codes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lua is a pretty fast language used in many games, and the good thing about it is that it offers certain dynamism, considering the the lanuage is not typed, variables can have any type - this is very nice to have to account for polymorphism. ( Iknow fortran can support polymorphism, but Lua is comfortable)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally I want to convert the polymorphic data back to static data just in time of computation and call Fortran or Python methods, or may be even C. For this, I will use pipes - or fifos, based on the requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question would be:&lt;/strong&gt; can Fortran / C / Python handle pipe and fifo without loosing much of its speed?&lt;/p&gt;&#xA;" OwnerUserId="9634" LastEditorUserId="9667" LastEditDate="2017-03-16T15:46:58.560" LastActivityDate="2017-03-16T15:46:58.560" Title="Lua and Fortran (or Python) I/O" Tags="|performance|fortran|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14212" PostTypeId="2" ParentId="14196" CreationDate="2014-07-21T13:21:38.567" Score="0" Body="&lt;p&gt;If you are really interested in open source and like to promote/support it, then you should probably take a look at the open source matlab clone &lt;a href=&quot;https://www.gnu.org/software/octave/&quot; rel=&quot;nofollow&quot;&gt;octave&lt;/a&gt;. There is an extension to it, called &lt;a href=&quot;http://octave.sourceforge.net/ad/&quot; rel=&quot;nofollow&quot;&gt;ad&lt;/a&gt; that supports automatic forward differentiation. Unfortunately, it seems to be unmaintained currently.&lt;/p&gt;&#xA;" OwnerUserId="1605" LastActivityDate="2014-07-21T13:21:38.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14213" PostTypeId="2" ParentId="14196" CreationDate="2014-07-21T14:29:11.277" Score="0" Body="&lt;p&gt;The wikipedia page for automatic differentiation has links to many software packages or libraries, including several for MATLAB.&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Automatic_differentiation#Software&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Automatic_differentiation#Software&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-07-21T14:29:11.277" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14214" PostTypeId="1" AcceptedAnswerId="14220" CreationDate="2014-07-21T16:46:15.320" Score="0" ViewCount="3281" Body="&lt;p&gt;I am trying to use the &lt;em&gt;diary&lt;/em&gt; function in Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;delete task1.txt &#xA;diary task1.txt&#xA;echo on&#xA;format long&#xA;b = 1:c;&#xA;x = 10.^(-b); &#xA;y = sin(x)./x;&#xA;echo off&#xA;diary off&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run the script I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Cannot open file: permission denied&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I enable the permission in the script?&lt;/p&gt;&#xA;" OwnerUserId="9713" LastActivityDate="2016-03-26T19:51:00.880" Title="Permission denied with diary command" Tags="|matlab|" AnswerCount="2" CommentCount="1" ClosedDate="2016-03-29T15:20:21.870" ContentLicense="CC BY-SA 3.0" />
  <row Id="14215" PostTypeId="1" CreationDate="2014-07-21T16:50:12.630" Score="1" ViewCount="119" Body="&lt;p&gt;I have a problem of the following structure:  For some scalar $g$, functions $F(z)$ and $h(z)$ defined on $[0,\bar{z}]$ , and a &lt;strong&gt;non-linear&lt;/strong&gt; operator $\phi(F,z)$ (in reality, $F$ and $h$ are vector valued)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $F(z)$ and $g$ must fulfill the ODE st. boundary values:&#xA;$$&#xA;0 = -F'(z;g) + \phi(F,z;g)\\&#xA;F(0) = 0\\&#xA;F(\bar{z}) = 1&#xA;$$&#xA;and the equilibrium condition:&#xA;$$&#xA;0 = -c + \int_{0}^{\bar{z}}h(z;g)F'(z;g) d z&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what the appropriate algorithm to solve this numerically?  I can solve the non-linear ODE with finite differences (and some shooting method in a more complicated variation), but a nested iteration with the integral equilibrium condition is difficult and expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about collocation?  For example, pick the $N$th order chebyshev approximation.  The chebyshev roots and polynomials (adapted to the $[0,\bar{z}]$ bounds) are $\{z_n\}_{n=0}^{N}$ such that $z_0 = 0$ and $z_N = \bar{z}$ and the polynomial of order $k$ evaluated at the $n$ root is $T_{nk} = T_n(z_n)$ and the vector of all orders at the $n$th root is $T_n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then if I assume a vector $a_0,\ldots a_N$ and a vector of these &#xA;$F(z) \approx \sum_{a=0}^{N} a_n T_n(z)$ and therefore,&#xA;$$F(z_n) = a \cdot T_n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since chebyshevs are easy to differentiate, assume another set of values such that $F'(z_n) = a \cdot U_n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I can apply the ODE and boundary values to get the usual system in $a$.  For the integral constraint, I already have the chebyshev nodes, so if I assume weights $w_k$ for the usual chebyshev quadrature, then I get a final equation in $g$&#xA;$$&#xA;0 = -c + \sum_{k=0}^N w_k h(z_k;g) \, a \cdot U_k&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that sound like a defensible algorithm (using a nonlinear solver to find the $a,g$)?  Are there other approaches which might work better?&lt;/p&gt;&#xA;" OwnerUserId="9692" LastActivityDate="2014-08-20T18:44:52.733" Title="Appropriate algorithm for (non-linear) ODE with integral equilibrium constraint: collocation?" Tags="|algorithms|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14216" PostTypeId="2" ParentId="14196" CreationDate="2014-07-21T17:23:52.543" Score="5" Body="&lt;p&gt;The &lt;a href=&quot;http://sintef.no/mrst&quot; rel=&quot;noreferrer&quot;&gt;SINTEF Matlab Reservoir Simulation Toolbox&lt;/a&gt; includes a GPL-licensed AD library. The usage is mostly geared towards numerical applications in subsurface flow, but the library itself is usable for more general purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a basic runthrough of your example as you would run it from the base directory of MRST:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;startup;&#xA;% Load ad based module&#xA;mrstModule add ad-fi&#xA;&#xA;&#xA;val = [1; 2; 3];&#xA;x = initVariablesADI(val);&#xA;y = x.*[4; 5; 6];&#xA;disp 'Jacobian:'&#xA;full(y.jac{1})&#xA;&#xA;z = sum(x.*x);&#xA;disp 'Jacobian:'&#xA;full(z.jac{1})&#xA;Jacobian:&#xA;&#xA;% MRST AD also supports multiple vector valued functions&#xA;a = ones(5, 1);&#xA;b = ones(5, 1);&#xA;[a, b] = initVariablesADI(a, b);&#xA;c = a.*b + 2.*b;&#xA;full(c.jac{1})&#xA;full(c.jac{2})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There exists a work-in-progress user guide at &lt;a href=&quot;http://sintef.no/Projectweb/MRST/Publications/&quot; rel=&quot;noreferrer&quot;&gt;the website&lt;/a&gt;, where chapter 7 has some usage of the AD library. This library is primarily geared towards problems that require all Jacobian values and is vectorized using forward mode only. It does include some nice things though, like 2d table interpolation and so on. It has been used for moderate size non-linear problems (order of 500,000 unknowns).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The non-linear solvers and components are heavily focused on reservoir applications at the moment. This part of the library is presently being rewritten and may in the future have some use outside of this specific domain. However, if you just want the AD part, the ADI class and the initVariablesADI function should be sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I am one of the developers attached to this research group.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1273" LastEditorUserId="1273" LastEditDate="2014-07-21T17:31:11.047" LastActivityDate="2014-07-21T17:31:11.047" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14217" PostTypeId="2" ParentId="14215" CreationDate="2014-07-21T18:31:53.897" Score="1" Body="&lt;p&gt;Your proposal sounds like a Galerkin-type approach. Specifically, it looks to be a spectral method rather than a finite element method. Collocation is typically associated with Runge-Kutta like methods for ODEs. Spectral methods are probably a fine approach here as long as you expect your solution to be smooth.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-21T18:31:53.897" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14218" PostTypeId="1" AcceptedAnswerId="14221" CreationDate="2014-07-21T19:06:47.093" Score="2" ViewCount="515" Body="&lt;p&gt;Is there a mathematical optimization branch that explicitly tries to optimize this (type) problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eqalign{&#xA;  &amp;amp; \min   \cr &#xA;  &amp;amp; \sum\limits_{i = 1}^N {(J*s[i] + {J_1}*s[i]*s[i + 1] + {J_2}s[i]*s[i + 1]*s[i + 2] + ... + {J_m}s[i]*s[i + 1]*...s[i + m])}   \cr &#xA;  &amp;amp; s[i] \in \{ 0,1\} \quad \forall i \in \{ 1,2...,N + m\}  \cr} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I only know things about Mixed integer programming (and its implementation using Gurobi)... &lt;/p&gt;&#xA;" OwnerUserId="7891" LastActivityDate="2014-07-21T21:08:52.507" Title="0,1 binary polynomial programming" Tags="|linear-programming|nonlinear-programming|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14219" PostTypeId="2" ParentId="14211" CreationDate="2014-07-21T19:50:11.607" Score="6" Body="&lt;p&gt;This is probably not the answer you are looking for, but I wanted to state it anyway: Your choice of programming languages introduces two difficulties you will encounter as your program grows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you will find that there is a cost to trying to couple different languages. They may interoperate, but there is always some quirk to it and as your software grows, you will find annoying incompatibilities (e.g., layout of structures, name mangling, etc), and they will turn out to cost you a significant amount of time, energy and patience. The fewer languages you couple, the less gray hair you will have in a year. That your program may be 5% slower will, in hindsight seem like a small price to pay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, D and Lua may be fine languages, but they are not widely used. If this is a project that you are sure you will never share with anyone, then this is not a problem. If you anticipate that someone else will eventually collaborate on it, or work with it, you have to anticipate that they do not know D and/or Lua. They may also not agree with your choice and give you grief everytime they have to touch a piece of code written in these languages. You may have to train them in these languages, and you may have to live with the fact that if you wanted to distribute the software, you will lose a significant number of potential users because they are unwilling to deal with non-conventional languages they are not used to.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-21T19:50:11.607" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14220" PostTypeId="2" ParentId="14214" CreationDate="2014-07-21T20:40:50.313" Score="1" Body="&lt;p&gt;This almost certainly means that you do not have write permissions in the current working directory or &lt;code&gt;task1.txt&lt;/code&gt; already exists and you do not have write permissions for that file. This causes an error when MATLAB tries to create or open the file &lt;code&gt;task1.txt&lt;/code&gt;. If this is the case, you need to use your operating system's tools to change the permissions for that directory. For example, you can likely right click on the relevant folder or file and then edit the properties to change the permissions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can view the current working directory using the &lt;code&gt;pwd&lt;/code&gt; command in MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-07-21T20:40:50.313" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14221" PostTypeId="2" ParentId="14218" CreationDate="2014-07-21T20:45:30.757" Score="5" Body="&lt;p&gt;If the $s_{i}$ are integers, there are reformulations of integer polynomial terms that result in mixed-integer (linear) programs, at the cost of introducing additional variables and constraints. Fred Glover has a sequence of papers to that effect in the mid-to-late 1970s, and subsequent work has built upon it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://leeds-faculty.colorado.edu/glover/fred%20pubs/69%20-%20Improved%20Linear%20Integer%20Programming%20Formulations%20of%20Nonlinear%20Integer%20Problems%20_%20%20Fred%20Glover.pdf&quot;&gt;Fred Glover, &quot;Improved Linear Integer Programming Formulations of Nonlinear Integer Problems&quot;, Management Science, 22(4), 455-460, 1975.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-07-21T21:08:52.507" LastActivityDate="2014-07-21T21:08:52.507" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14222" PostTypeId="1" AcceptedAnswerId="14278" CreationDate="2014-07-21T21:50:08.550" Score="11" ViewCount="373" Body="&lt;p&gt;What is the current state of the art for solving higher dimensional (3-10)  parabolic PDEs in the complex domain with simple poles (of the form $ \frac{1}{|\vec{r}_1 - \vec{r}_2|}$) and absorbing boundary conditions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I'm interested in solving the multi-electron Schrödinger equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \left( \sum_i \sum_{j\neq i}\left[ -\frac{\nabla_i^2}{2 m} - \frac{Z_i Z_j}{|\vec{r}_i - \vec{r}_j|}  + V(\vec{r}_i, t) \right]\right)\psi = -i\partial_t \psi $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a diatomic molecule with more than 1 electron.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="9667" LastEditDate="2017-05-10T21:23:31.153" LastActivityDate="2017-05-10T21:25:46.693" Title="What is the current state of the art in solving higher dimensional parabolic PDEs (multi-electron Schrödinger equation)" Tags="|parabolic-pde|quantum-mechanics|high-dimensional|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14224" PostTypeId="1" CreationDate="2014-07-22T09:53:52.733" Score="4" ViewCount="1580" Body="&lt;p&gt;I am finding the eigenvalues of dense NxN Hermitian matrix which is calculated from a density operator in quantum physics. All the eigenvalues are needed as I need to calculate the sum of the absolute value of them. I would like to know whether there are good solvers for this problem that can run in parallel?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem size I am interested should be at least N=40000. The one I am currently using is the gsl library, which has the advantage that it takes no extra working memory except the matrix itself. However, the running time is close to 5 hours for N=20000 because of no parallelization. I am going to run the problem in computing cluster that has walltime limit of 3 days so my program is likely to be killed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried SLEPc, but it does not scale well for the dense matrix. The computation time with 8 cores is almost the same as single core gsl! Not to mention it takes more memory and less accurate result in my case.&lt;/p&gt;&#xA;" OwnerUserId="3444" LastActivityDate="2014-07-22T13:32:21.093" Title="Fast way to compute all eigenvalues of a dense Hermitian matrix" Tags="|parallel-computing|eigenvalues|hpc|dense-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14226" PostTypeId="2" ParentId="14224" CreationDate="2014-07-22T13:32:21.093" Score="5" Body="&lt;p&gt;SLEPc is a library designed for the solution of large sparse eigenvalue problems, while your matrix is dense.  The algorithms used by SLEPc are based on matrix-vector multiplication, and the performance of these algorithms is primarily limited by the available memory bandwidth rather than the number of processor cores.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be better off using a software library designed to solve dense eigenvalue problems.  For example, if you use LAPACK's QR factorization algorithms, it will use BLAS library routines, some of which can make effective use of your eight cores.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the parallelization of dense eigenvalue problems is still an area of active research.  See for example these recent conference talks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.eecs.utk.edu/~luszczek/conf/siamcse2013_eigvalsolv/&quot; rel=&quot;noreferrer&quot;&gt;http://web.eecs.utk.edu/~luszczek/conf/siamcse2013_eigvalsolv/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-07-22T13:32:21.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14227" PostTypeId="1" AcceptedAnswerId="14244" CreationDate="2014-07-22T14:23:17.903" Score="0" ViewCount="61" Body="&lt;p&gt;&lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/23/20/2692.full&quot; rel=&quot;nofollow&quot;&gt;Hibbs et. al&lt;/a&gt; use SVD to balance the strength of different underlying signals in gene expression data using the following decomposition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X_{m*n} = U_{m*n} \Sigma_{n*n} V^T_{n*n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case $U$ has the same dimensions as $X$, so the way that they see $U$ as a balanced projection of $X$ makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to use the same approach in R. However, the &lt;em&gt;svd&lt;/em&gt; package decomposes $X$ into $D_{m}, U_{m*m}, V_{n*m}$. This is not too surprising given that &lt;a href=&quot;http://mathworld.wolfram.com/SingularValueDecomposition.html&quot; rel=&quot;nofollow&quot;&gt;Mathworld&lt;/a&gt; also mentions different conventions, however, I see no equivalent for the $U$ matrix of the first definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an R implementation of &lt;a href=&quot;http://mathworld.wolfram.com/SingularValueDecomposition.html&quot; rel=&quot;nofollow&quot;&gt;Press' definition&lt;/a&gt; available, or is there a way to convert one in the other?&lt;/p&gt;&#xA;" OwnerUserId="1601" LastEditorUserId="1601" LastEditDate="2014-07-22T14:30:39.117" LastActivityDate="2014-07-24T03:18:07.090" Title="Signal balancing using SVD: notations and implementation in R" Tags="|svd|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14228" PostTypeId="1" AcceptedAnswerId="14231" CreationDate="2014-07-22T15:00:52.703" Score="1" ViewCount="448" Body="&lt;p&gt;I want to find the minimum-norm solution to a rank-deficient least-squares problem, subject to positivity constraints, e.g.&#xA;$$\min_x\ \|x\|^2 \quad s.t.\quad Ax = b,\ x \geq 0$$&#xA;where $A$ is large, sparse, rectangular, and rank deficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most efficient way of robustly solving this problem? Is there recommended code out there for doing so? If $A$ were full-rank, I could use NNLS; without the inequality constraints, I could use a sparse QR decomposition. The above optimization problem is obviously a QP, so worst-case I assume an interior point method should work well, although in the past I have had trouble getting e.g. IpOpt to work with rank-deficient equality constraint matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: A typical $A$ is of size 50,000 $\times$ 70,000 with 1,000,000 nonzeroes. I would prefer C code, but FORTRAN, python, etc. is fine too. For performance reasons I would like to avoid Matlab if possible.&lt;/p&gt;&#xA;" OwnerUserId="3612" LastEditorUserId="3612" LastEditDate="2014-07-22T16:38:56.547" LastActivityDate="2014-07-23T02:52:18.053" Title="rank-deficient NNLS" Tags="|linear-algebra|least-squares|quadratic-programming|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14229" PostTypeId="2" ParentId="2328" CreationDate="2014-07-22T15:10:16.860" Score="-1" Body="&lt;p&gt;There is the open source fortran library forDat by ffr Engineering which uses oop 2003 features&lt;/p&gt;&#xA;" OwnerUserId="9782" LastActivityDate="2014-07-22T15:10:16.860" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14230" PostTypeId="1" CreationDate="2014-07-22T16:03:39.357" Score="4" ViewCount="1206" Body="&lt;p&gt;I am trying to create the same outputs in Mathematica and Matlab, however I am running into trouble aligning the eigenvectors with the eigenvalues, I think the Matlab is doing something slightly more complex (Ordering/Sorting) than I anticipated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the example in on the Mathwork website, for &lt;code&gt;[V,D] = eig(A)&lt;/code&gt;.&#xA;&lt;a href=&quot;http://www.mathworks.co.uk/help/matlab/ref/eig.html#btifddh-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.co.uk/help/matlab/ref/eig.html#btifddh-2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TestMatrix = {{1, 2, 3}, {3, 1, 2}, {2, 3, 1}}&#xA;&#xA;[V,D] = eig(A)&#xA;&#xA;V =&#xA;&#xA;-0.5774 + 0.0000i,   0.2887 - 0.5000i,   0.2887 + 0.5000i,&#xA;&#xA;  -0.5774 + 0.0000i,  -0.5774 + 0.0000i,  -0.5774 + 0.0000i,&#xA;&#xA;  -0.5774 + 0.0000i,   0.2887 + 0.5000i,   0.2887 - 0.5000i,&#xA;&#xA;&#xA;D =&#xA;&#xA;   6.0000 + 0.0000i,   0.0000 + 0.0000i,   0.0000 + 0.0000i,&#xA;&#xA;   0.0000 + 0.0000i,  -1.5000 + 0.8660i,   0.0000 + 0.0000i,&#xA;&#xA;   0.0000 + 0.0000i,   0.0000 + 0.0000i,  -1.5000 - 0.8660i,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However running this in Mathematica, I receive the following eigenvector&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m = N[Eigensystem[TestMatrix]]&#xA;&#xA;DV = DiagonalMatrix[m[[1]]]&#xA;&#xA;Vec=N[Eigenvectors[TestMatrix]]&#xA;&#xA;Vec={&#xA;&#xA;{1., 1., 1.}, &#xA;&#xA;{-0.5 - 0.866025 I, -0.5 + 0.866025 I, 1.},&#xA;&#xA; {-0.5 + 0.866025 I, -0.5 - 0.866025 I, 1.}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is perfectly acceptable, I just now want to get both vectors into the same form.&#xA;Note The Diagonal Matrix matches, however I seem unable to recreate V.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From reading the Matwork example, V is right eigenvectors. In this case the &#xA;Form [V,D] = eig(A) returns matrix V, whose columns are the right eigenvectors of A such that A&lt;em&gt;V = V&lt;/em&gt;D. &amp;amp; The eigenvectors in V are normalized so that the 2-norm of each is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to normalise Vec, now in mathematica, I am unable to recreate the results obtained in Matlab. Clearly I am doing something incorrectly, this is where I get a little lost. I have tried the '2-norm' function as described in Matlab, and also the Normalize function, the latter seems a little better but is still incorrect comparing to the results in the matlab example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone explain where I am going wrong with this, or simply how to recreate the same results in mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are my (...I assume...) incorrect attempts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firslty using Normalise.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TestMatrix = {{1, 2, 3}, {3, 1, 2}, {2, 3, 1}}&#xA;&#xA;vec = N[Eigenvectors[TestMatrix]]&#xA;&#xA;Table[Normalize[vec[[i]]], {i, 1, Length[vec]}]&#xA;&#xA;{{0.57735, 0.57735, 0.57735}, &#xA;&#xA;{-0.288675 - 0.5 I, -0.288675 + 0.5 I,  0.57735 + 0. I}, &#xA;&#xA;{-0.288675 + 0.5 I, -0.288675 - 0.5 I,  0.57735 + 0. I}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This quite close, to the Matlab output V, except the matrix seems to have been transposed, times by a negative, and some how the 2nd and 3rd eigenvalues have been switched (Comparing the Egienvalues and Vector columns) &#xA;Clearly something has gone wrong, I probably shouldn't have used Normalise.&lt;br&gt;&#xA;However my attempts using Norm(p,2) are nowhere near to the final resul. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see another thread which touches on this topic but doesn't really help.&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/5648975/matlab-vs-mathematica-eigenvectors&quot;&gt;https://stackoverflow.com/questions/5648975/matlab-vs-mathematica-eigenvectors&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone out there explain this?&lt;/p&gt;&#xA;" OwnerUserId="9785" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-03-19T19:08:10.130" Title="Comparing Eigenvectors, Mathematica vs. Matlab" Tags="|linear-algebra|matlab|eigensystem|eigenvalues|mathematica|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14231" PostTypeId="2" ParentId="14228" CreationDate="2014-07-22T17:00:28.517" Score="2" Body="&lt;p&gt;We solve almost the same problem using the &lt;a href=&quot;ftp://ftp.math.ucla.edu/pub/camreport/cam10-06.pdf&quot; rel=&quot;nofollow&quot;&gt;Split-Bregman algorithm&lt;/a&gt; (see &lt;a href=&quot;http://pure.au.dk/portal/files/55114303/reg_recon.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; for implementation details). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially you convert the problem into $\min_x \|x\|^2+\lambda \|Ax-b^*\|+\gamma\| u -x \|$ where $u$ is $x$ with all negative values removed, and $b^*$ is $b$ with the residual added. &#xA;This is then solved repeatedly, with $u$ and $b^*$ changing every iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can write up the entire algorithm for you if you'd like, but it may be easier to simply look at the pseudo-code in the references.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-07-22T17:00:28.517" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14233" PostTypeId="1" AcceptedAnswerId="14245" CreationDate="2014-07-22T21:24:44.267" Score="4" ViewCount="120" Body="&lt;p&gt;I have a set of points defined by their coordinates $(x_1,y_1)$ after changing some parameters of the problem I obtain a second set of points defined by their coordinates $(x_2, y_2)$. There exists a one-to-one mapping between these two sets of points. Is there a standard way to obtain the most likely mapping between these two sets of points? Consider the following sets of points: (blue, red, and green are different sets)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/lBmjE.png&quot; alt=&quot;3 sets of points.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to obtain this mapping from blue to red to green:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/g5f8K.png&quot; alt=&quot;3 sets of points with the correct mapping shown.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thought for solving this is to compute the nearest neighbor of blue points to each of the red points, sort these nearest neighbors by distance, remove the closest match, recompute the nearest neighbors, and repeat. However, it is easy to think of situations for which this algorithm does not work. (I don't think it works for the sets of points above.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm solving a large eigenvalue problem and attempting to track eigenvalues as I vary parameters.&lt;/p&gt;&#xA;" OwnerUserId="4524" LastActivityDate="2014-07-24T04:07:16.680" Title="Numerically compute one-to-one mapping" Tags="|algorithms|nearest-neighbors|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14234" PostTypeId="2" ParentId="14228" CreationDate="2014-07-22T21:36:33.277" Score="1" Body="&lt;p&gt;You &lt;em&gt;might&lt;/em&gt; be able to form a (sparse) $QR$ decomposition of $A$, and then use the $R$ matrix and the system $Rx = Q^{T}b$ to form an equivalent full rank linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Independent of that, you could try other QP solvers, not just IPOPT. CPLEX and Gurobi have free academic licenses and contain QP solvers. Something like CVX, or SDP solvers like SeDuMi and SDPT3 would work as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Split augmented Lagrangian approaches like LKlevin is proposing have worked really well for specific problems (Boyd has a good paper for $\ell_{1}$ regression using the alternating direction method of multipliers, for instance), and they can be faster than interior-point solvers by taking advantage of problem structure.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-22T21:36:33.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14235" PostTypeId="1" CreationDate="2014-07-23T01:27:44.383" Score="1" ViewCount="702" Body="&lt;p&gt;Not sure if this question is more appropriate for StackOverload, but I'm guessing a bunch of people here have experience with MPI2, so thought I would try here first. I have some large dense complex symmetric matrices that I would like to factor and wanted to try out Elemental for its MPI support. I compiled Open MPI 1.8.1 and a simple &quot;hello world&quot; example, but for some reason when I try to run: mpirun -n 2 -host localhost,machine-2 hello_world it just exits without outputting anything or even starting the hello_world program. If I run it on &lt;em&gt;just&lt;/em&gt; localhost (i.e. mpirun -n 2 -host localhost hello_world), it works just fine. Do you use Open MPI or a different flavor of MPI2? I've searched everywhere online and can't find a working solution or even just a cause for this problem. (I have ssh setup between all the machines such that I do not need to use a password.) The machines are running RedHat 5, and I do not have root access to them. I have read that it might be a firewall problem (what ports does OpenMPI use outside of ssh?). Do I need to have anything placed in /etc/hosts? Thanks so much in advance!&lt;/p&gt;&#xA;" OwnerUserId="1480" LastActivityDate="2014-07-23T01:27:44.383" Title="Issues with Open MPI" Tags="|mpi|compiling|" AnswerCount="0" CommentCount="3" ClosedDate="2014-07-23T14:41:10.960" ContentLicense="CC BY-SA 3.0" />
  <row Id="14236" PostTypeId="2" ParentId="14228" CreationDate="2014-07-23T02:52:18.053" Score="2" Body="&lt;p&gt;Others have already supplied the two most likely answers to this question, but I'll add a bit of comparison and a way to help decide between the two approaches.  I'd suggest either &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A primal-dual interior point method for quadratic programming.  For this approach you'd want to make use of a library that someone else has developed. A lot depends on whether you want/need open source software or whether something that is free for research (but not open source) or commercial and not free for research would do.  Some codes to consider include CPLEX, Gurobi, LOQO, and Clp.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A first order method (Augmented Lagrangian, ADMM, split Bregman, etc.)  These are possible to implement yourself without needing to use a packaged library.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The big advantage of the first order methods is that they can solve very large problems that are too big to solve by the primal-dual interior point method but these methods don't produce very accurate solutions and can converge very slowly.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem is of a size that should be reasonably possible to solve using a primal dual interior point method (assuming that you've got a computer with sufficient memory and CPU.) If so, you'll probably be happier with the quality of solutions produced by the interior point method.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also consider a more general purpose LP/SOCP/SDP solver such as Mosek, SeDuMi, or SDPT3, but since your problem actually is a QP it doesn't make a lot of sense to reformulate your problem as an SOCP.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a quick test that may help you in determining whether a primal-dual interior point method would be usable for this problem.  Compute $H=AA^{T}+I$ (using e.g. MATLAB)  then compute the Cholesky factorization of $H$ (using a suitable sparse Cholesky routine- MATLAB has this built in.)  If the Cholesky factorization can reasonably be computed on your machine, then a primal-dual interior point code should have enough memory to solve the QP.   If $H$ is just way too big and dense for this to be practical, then using a primal-dual method is out of the question and you need to look at first order methods.       &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-07-23T02:52:18.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14237" PostTypeId="1" AcceptedAnswerId="14238" CreationDate="2014-07-23T11:52:15.927" Score="1" ViewCount="2169" Body="&lt;p&gt;Below is a Python code that calculates the periodicity of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Middle-square_method&quot; rel=&quot;nofollow&quot;&gt;middle-square method&lt;/a&gt; for a given 4 digit-numbers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = int(input(&quot;Please enter a four digit number: &quot;))&#xA;already_seen = list()&#xA;while n not in already_seen:&#xA;    already_seen.append(n)&#xA;    n = int(str(n * n).zfill(8)[2:6])&#xA;    print(n)&#xA;print('periodicity = ', len(already_seen) - already_seen.index(n))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example the seed number 9267 yield to a series that enters in a short loop of periodicity 4 [9267, …, 6100, 2100, 4100, 8100, 6100, 2100, 4100, 8100, 6100, ...]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any solution to find out what numbers yield to the longest periodicity without having to simulate? If no, why isn't there a solution? Generally speaking, are there solutions to find out the periodicity of a given seed in different random number generators?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9791" LastEditorUserId="9791" LastEditDate="2014-07-24T07:37:01.713" LastActivityDate="2014-07-24T23:47:15.637" Title="Calculating periodicity of a pseudo random number generator (Middle-square method)" Tags="|python|random-number-generation|" AnswerCount="1" CommentCount="8" ClosedDate="2014-07-24T23:47:28.103" ContentLicense="CC BY-SA 3.0" />
  <row Id="14238" PostTypeId="2" ParentId="14237" CreationDate="2014-07-23T13:50:44.833" Score="6" Body="&lt;p&gt;To the best of my knowledge, no, but maybe other people here know the field more intimately. My knowledge comes primarily from developing Monte Carlo codes in physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knuth, in volume 2 of his Art of Computer Programming, states that Metropolis, using the middle-square method on 20 bits, found 13 cycles to which the method would always degenerate, the longest of which has a period of 142. Sadly I've been unable to find the original paper, but note that he only refers to the period of the cycle, not the time it takes to reach the cycle from a given period.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, most (but not all) random number generators used in numerics today have a guaranteed period, often independent of the seed. This period is however calculated differently from generator to generator.&#xA;See &lt;a href=&quot;http://xorshift.di.unimi.it/&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; for a list of reasonable random number generators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the period experimentally will be very difficult for almost all modern random number generators, simply because the periods are very very long. In many cases it would take far far longer than the expected age of the universe, even if every single atom was turned into a computer to work on the problem in parallel. Just to give you an idea of how difficult it is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, for toy random generators, remember that in order to find a cycle, you need to check if the entire state space of the generator is repeating. In the case of the middle-square method, the state and the output are the same, but this is not the case for most random number generators, and certainly not for any practical ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: The middle-square method is a really bad random number generator and should not be used for generating random numbers. Ever. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="7522" LastEditDate="2014-07-23T21:45:46.827" LastActivityDate="2014-07-23T21:45:46.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14239" PostTypeId="1" AcceptedAnswerId="14241" CreationDate="2014-07-23T14:47:08.727" Score="2" ViewCount="2049" Body="&lt;p&gt;I'm new to the PDE toolbox in &lt;code&gt;Matlab&lt;/code&gt;. From the PDE specification window of the toolbox, it looks like one can only solve PDE with constant coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/aeJ3E.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use &lt;code&gt;Matlab&lt;/code&gt;'s PDE toolbox to solve PDE like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\partial_t P(x,y,t)=x\partial_xP(x,y,t)+(y-1)\partial_yP(x,y,t)+2P(x,y,t)?&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="3512" LastActivityDate="2014-07-24T12:42:05.363" Title="use Matlab's PDE toolbox to solve PDE with variable coefficients" Tags="|pde|matlab|" AnswerCount="2" CommentCount="1" ClosedDate="2014-07-25T00:12:56.807" ContentLicense="CC BY-SA 3.0" />
  <row Id="14240" PostTypeId="2" ParentId="14239" CreationDate="2014-07-23T15:32:46.027" Score="1" Body="&lt;p&gt;I'm not familiar with this PDE toolbox in Matlab, but the software COMSOL Multiphysics is developed from this toolbox. You can dig a bit to find out their close relation. Also if you check out COMSOL you will find how these two look alike. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least in COMSOL, I know your PDE is in principle possible to solve. &lt;/p&gt;&#xA;" OwnerUserId="8265" LastActivityDate="2014-07-23T15:32:46.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14241" PostTypeId="2" ParentId="14239" CreationDate="2014-07-23T15:43:34.230" Score="3" Body="&lt;p&gt;In general, PDE Toolbox is able to solve 2D PDE in the form shown on this page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/ug/types-of-pde-problems-you-can-solve.html&quot; rel=&quot;nofollow&quot;&gt;PDE Form&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any of those coefficients (e.g. c, a, f, d) can be functions of x, y, t, u,&#xA;$\frac{\partial u}{\partial x}, \frac{\partial u}{\partial y}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the coefficients are relatively simple expressions, the simplest way to define them is by typing a string expression in text boxes above (e.g. 1+x.^2+4*y). That approach is documented on this page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/ug/scalar-coefficients-in-string-form.html&quot; rel=&quot;nofollow&quot;&gt;Coefficients defined with string expressions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the coefficients are more complicated, it is usually more straightforward to define&#xA;them as MATLAB functions. That approach is documented on this page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/ug/scalar-coefficients-in-function-form.html&quot; rel=&quot;nofollow&quot;&gt;Coefficients defined with a function&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your equation, f could be defined as the string &lt;code&gt;&quot;x.^ux+(y-1).*uy&quot;&lt;/code&gt; (without the &quot;),&#xA;a=-2, d=1, and c=0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a strong caveat is in order. The algorithms in PDE Toolbox are designed for the case where the second-order c-term is significant relative to the first-order derivative terms. So the success for this equation will depend very much on the boundary and initial conditions. It might also be necessary to set c to some small number; this is sometimes referred to as adding &quot;artificial diffusion.&quot;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2014-07-24T12:42:05.363" LastActivityDate="2014-07-24T12:42:05.363" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14242" PostTypeId="1" AcceptedAnswerId="14279" CreationDate="2014-07-18T23:39:50.870" Score="2" ViewCount="705" Body="&lt;p&gt;I am attempting to translate a dgemm / MPI matrix multiplier onto the GPU through OpenCL C.  My issue is that the code below gives the correct output for a 9x6 matrix being multiplied by a 6x450 matrix until the 6x450 matrix changes slightly and the kernel output is shifted by 315 indeces. Included is the dgemm that I am trying to emulate for reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DGEMM:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    umAxBtrans(A, 450, 6, B, 9, 6, &amp;amp;fQ1);&#xA;    umAxBtrans(double **A, int Arows, int Acols, double *B, int Brows, int Bcols, double **C){&#xA;    op(A) = 'T';&#xA;    op(B) = 'F';&#xA;    dgemm(&amp;amp;opA, &amp;amp;opB, &amp;amp;rowsB, &amp;amp;rowsA, &amp;amp;colsA, &amp;amp;one, *B, &amp;amp;colsB, *A, &amp;amp;colsA, &amp;amp;zero, *C, &amp;amp;rowsB);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;dgemm man page: &lt;a href=&quot;http://www.math.utah.edu/software/lapack/lapack-blas/dgemm.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.utah.edu/software/lapack/lapack-blas/dgemm.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if that dgemm implementation isn't a slight bit confusing then I don't know what is.  I have taken care of the transpose of the B matrix outside of this matrix multiplication code because B is a static matrix. Matrix A, however, updates on every loop of the function call, meaning a new A is passed by value every time to openCL using the below function and kernel code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenCL:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    clAxBtrans(A, 6, 450, Bcl, 9, 6, &amp;amp;C, context, program, queue, device_counter);&#xA;    clAxBtrans(double* A, int Arows, int Acols, double* Bcl, int Brows, int Bcols, double** C, cl_context context, cl_program program, cl_command_queue* queue, cl_uint device_counter){&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this function I pass the value of A with the dimensions used in the previous dgemm call (I've tried both passing A by reference and by value, no difference), and the static Bcl which is already transposed. I then setup the kernel and define:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    globalWorkSize[0]=450;&#xA;    globalWorkSize[1]=9;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then write the kernel like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    __kernel void clAxBtrans(__global double* A,&#xA;                     __global double* B,&#xA;                     int rowsA,&#xA;                     int colsA,&#xA;                     int rowsB,&#xA;                     int colsB,&#xA;                     __global double* C)&#xA;    {&#xA;    int globalx = get_global_id(0);&#xA;    int globaly = get_global_id(1);&#xA;    double tmp = 0;&#xA;    double tmp0 = 0;&#xA;    double tmp1 = 0;&#xA;    double tmp2 = 0;&#xA;    double tmp3 = 0;&#xA;    double tmp4 = 0;&#xA;    double tmp5 = 0;&#xA;    tmp0 = B[globaly * colsB + 0] * A[0 * colsA + globalx];&#xA;    tmp1 = B[globaly * colsB + 1] * A[1 * colsA + globalx];&#xA;    tmp2 = B[globaly * colsB + 2] * A[2 * colsA + globalx];&#xA;    tmp3 = B[globaly * colsB + 3] * A[3 * colsA + globalx];&#xA;    tmp4 = B[globaly * colsB + 4] * A[4 * colsA + globalx];&#xA;    tmp5 = B[globaly * colsB + 5] * A[5 * colsA + globalx];&#xA;    tmp = tmp0 + tmp1 + tmp2 + tmp3 + tmp4 + tmp5;&#xA;    barrier(CLK_GLOBAL_MEM_FENCE);&#xA;    C[globaly * colsA + globalx] = tmp;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The host code has blocking calls everywhere because this is in an MPI parallelized function and the localWorkSize is set as null to allow the gpu to choose the most &quot;optimal&quot; size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output is a 6x450 C matrix that matches the previous implementation until A updates. By that I mean for the first 5 runs A is a static matrix but then changes by a very small amount each run after depending on the resultant C matrix. This leads me to believe that I am interpreting matrix A incorrectly in memory. I am writing it to the kernel using the clBuffer call below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Acl = clCreateBuffer(context, CL_MEM_READ_ONLY | CL_MEM_COPY_HOST_PTR, sizeof(double)*Arows*Acols, A, &amp;amp;err);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For reference matrix B is the 9x6 below, A is an entire 6x450 filled with .111111 doubles, and the resultant matrix C is a 9x450 filled with .111111 doubles for the first 5 runs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrix B:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    [1, 0 ,0 ,0 ,0 ,0]&#xA;    [0, 1 ,0 ,0 ,0 ,0]&#xA;    [0, 0 ,0 ,1 ,0 ,0]&#xA;    [0, 1 ,0 ,0 ,0 ,0]&#xA;    [0, 0 ,1 ,0 ,0 ,0]&#xA;    [0, 0 ,0 ,0 ,1 ,0]&#xA;    [1, 0 ,0 ,0 ,0 ,0]&#xA;    [0, 0 ,1 ,0 ,0 ,0]&#xA;    [0, 0 ,0 ,0 ,0 ,1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Because I am copying the host ptr, I believe my problem either lies in how I think dgemm interprets the dimension flipping on the matrices, with my actual kernel code(most likely), or with how I am writing it as a buffer. I have checked and the very first new A is the same on both the opencl and previously working code, but the resultant C matrices are different(pointing to bad kernel code or memory management); I can include the resulting C comparison if required, but the first C results are shifted incorrectly by 315 indeces which causes the divergent solutions to really grow out of control. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm very grateful for any help you can afford me, even if it's pointing out that I've done something incredibly stupid! Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9827" OwnerDisplayName="Stumpae" LastEditorUserId="9827" LastEditDate="2014-07-28T21:17:57.903" LastActivityDate="2014-08-20T20:17:15.623" Title="OpenCL C Matrix Multiplication over Multiple Runs" Tags="|matrix|opencl|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14243" PostTypeId="2" ParentId="14230" CreationDate="2014-07-24T00:54:47.370" Score="6" Body="&lt;p&gt;MATLAB always uses the LAPACK libraries to calculate eigenvectors which works on double precision, floating point numbers - MATLAB's default data type.  Mathematica's method depends on its input type.  For example, when you do &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TestMatrix = {{1, 2, 3}, {3, 1, 2}, {2, 3, 1}}&#xA;Eigenvectors[TestMatrix]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll get an exact answer involving Sqrt[3] and so on.  Mathematica uses it's exact arithmetic methods to get this and they are different from LAPACK.  When you do &lt;code&gt;N[Eigenvectors[TestMatrix]]&lt;/code&gt; you are taking these exact results and approximating them as floating point numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TestMatrix = {{1., 2., 3.}, {3., 1., 2.}, {2., 3., 1.}}&#xA;Eigenvectors[TestMatrix]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note the decimal point after each number which is telling Mathematica not to interpret these as exact integers but as floating point numbers.  Since you've asked Mathematica to calculate the eigenvectors of a double precision floating point Matrix, it uses the same library as MATLAB - i.e. LAPACK.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what you get - matching MATLAB&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{{-0.5773502691896255 + 0. I, -0.577350269189626 + &#xA;   0. I, -0.5773502691896258 + 0. I}, {0.5773502691896261 + &#xA;   0. I, -0.28867513459481314 - &#xA;   0.4999999999999996 I, -0.2886751345948132 + &#xA;   0.4999999999999998 I}, {0.5773502691896261 + &#xA;   0. I, -0.28867513459481314 + &#xA;   0.4999999999999996 I, -0.2886751345948132 - 0.4999999999999998 I}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7454" LastActivityDate="2014-07-24T00:54:47.370" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14244" PostTypeId="2" ParentId="14227" CreationDate="2014-07-24T03:18:07.090" Score="1" Body="&lt;p&gt;In $X=UDV^t$, $D$ is diagonal, and only the upper-left square of $D$, of size $\min(m,n)\times\min(m,n)$, is non-zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two SVD decompositions are equivalent in the sense that if you drop the right-most zero columns (or bottom-most zero rows) from $D$, and also drop the corresponding rows/columns from $U$ or $V$, you get the other definition of SVD.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-07-24T03:18:07.090" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14245" PostTypeId="2" ParentId="14233" CreationDate="2014-07-24T04:00:47.140" Score="2" Body="&lt;p&gt;One thing to try is this modification of your idea:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for x in blue-points:&#xA;  for y in red-points, sorted by distance to x:&#xA;    find all green points z &quot;close enough&quot; to y + (y-x)&#xA;    if angle between vectors (y-x) and (z-y) is &quot;small enough&quot;:&#xA;      return (x,y,z) as the guess&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, you can try to use your suspicion that the points move along smooth curves at roughly (&quot;locally&quot;) constant velocities. Any smooth curve is approximately close to a line, and the angle between consecutive chords of the curve is likely to be small. In your plot, &quot;small enough&quot; angle might mean something like $&amp;lt;30^\circ$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not specific to eigenvalues; you could do the same thing for roots of polynomials with time-varying coefficients (which are also easier to test with). With very many points you'll need a clever data structure to implement this efficiently, but you don't seem to have many.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another mathematical justification for doing this is that the angle between chords is related to the curvature of a curve (as in the Frenet-Serret formulas). So in some sense this approach tries to minimize or bound the curvature of the curves that it finds.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-07-24T04:07:16.680" LastActivityDate="2014-07-24T04:07:16.680" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14246" PostTypeId="1" AcceptedAnswerId="14288" CreationDate="2014-07-24T12:02:31.560" Score="5" ViewCount="4455" Body="&lt;p&gt;I'm trying to implement the Helmholtz-Hodge Decomposition in 2D, which states that a vector field is composed by a rotational free component, a divergence free component and a harmonic component. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads me to a Poisson Equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{equation}&#xA;\left\{ &#xA;  	\begin{array}{lcl}&#xA;    		\nabla \cdot \mathbf{F}     &amp;amp; = &amp;amp; \Delta \varphi \\&#xA;    		(\nabla \cdot J) \mathbf{F} &amp;amp; = &amp;amp; - \Delta \psi	&#xA;    \end{array}&#xA;\right.&#xA;\end{equation}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to solve it using finite-differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To comput gradients, I'm using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( &#xA;\begin{array}{ccc}&#xA;-1 &amp;amp; 0 &amp;amp; 1&#xA;\end{array} &#xA;\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;code&gt;dx&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( &#xA;\begin{array}{c}&#xA;-1 \\&#xA; 0 \\&#xA; 1&#xA;\end{array} &#xA;\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;code&gt;dy&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;code&gt;Laplacian&lt;/code&gt;, I'm using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( &#xA;\begin{array}{ccc}&#xA; 0 &amp;amp; -1 &amp;amp; 0 \\&#xA; -1 &amp;amp; +4 &amp;amp; -1 \\&#xA; 0 &amp;amp; -1 &amp;amp; 0&#xA;\end{array} &#xA;\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, for a 2 x 2 input, for example, I will have an equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( &#xA;\begin{array}{c}&#xA;\nabla \cdot \mathbf{F}_{11}\\&#xA;\nabla \cdot \mathbf{F}_{12}\\&#xA;\nabla \cdot \mathbf{F}_{21}\\&#xA;\nabla \cdot \mathbf{F}_{22}\\&#xA;\end{array} &#xA;\right)&#xA;$=&#xA;$\left( &#xA;\begin{array}{cccc} &#xA; 4. &amp;amp; -1. &amp;amp;  0. &amp;amp; -1.\\&#xA;-1. &amp;amp; 4.  &amp;amp; -1. &amp;amp;  0.\\&#xA; 0. &amp;amp; -1. &amp;amp;  4. &amp;amp;  0.\\&#xA;-1. &amp;amp;  0. &amp;amp;  0. &amp;amp;  4&#xA;\end{array} &#xA;\right)&#xA;$&#xA;$\left( &#xA;\begin{array}{c}&#xA;\varphi_{11}\\&#xA;\varphi_{12}\\&#xA;\varphi_{21}\\&#xA;\varphi_{22}&#xA;\end{array} &#xA;\right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve $\nabla \cdot \mathbf{F} = \Delta \varphi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this approach, I could solve the system and obtain the vector field components. However, I'm having &lt;em&gt;problem at the boundaries&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Researching, I read about the Pure Neumann Boundary Conditions. However, I don't know how to include them in the system of equations. I'm wondering how to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9799" LastEditorUserId="9799" LastEditDate="2014-07-24T13:53:40.127" LastActivityDate="2015-12-19T21:03:49.363" Title="Discrete Poisson Equation with Pure Neumann Boundary Conditions" Tags="|finite-difference|boundary-conditions|discretization|poisson|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14247" PostTypeId="2" ParentId="14246" CreationDate="2014-07-24T15:58:46.840" Score="3" Body="&lt;p&gt;Pure Neumann boundary conditions give a non-unique solution. You will need to set some other condition to make it unique. The simplest would be to choose some solution value somewhere and set it to any constant, I would choose a corner or the middle and set it to 0. Another common approach is to apply a zero mean condition that the integral of the solution of the entire solution is zero.&lt;/p&gt;&#xA;" OwnerUserId="3309" LastActivityDate="2014-07-24T15:58:46.840" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14248" PostTypeId="1" AcceptedAnswerId="14253" CreationDate="2014-07-24T16:46:35.630" Score="0" ViewCount="115" Body="&lt;p&gt;I want to numerically solve the following system of differential equations at the steady state:&#xA;\begin{equation}&#xA;\begin{aligned}&#xA; \frac{\partial \rho_{11\mathbf{k}}}{\partial t} =&amp;amp;  +\frac{i}{\hbar} V_{0}^{2121}\sum_\mathbf{k'}(\rho_{21\mathbf{k'}}\rho_{21\mathbf{k}}^* - \rho_{21\mathbf{k'}}^*\rho_{21\mathbf{k}}) + \\ &#xA;&amp;amp; -\Gamma (\rho_{1\mathbf{k}}-\rho^0_{11\mathbf{k}})+ \\&#xA;&amp;amp; -\eta\Gamma_\mathbf{k}^{out}\rho_{11\mathbf{k}} + (1-\eta)\Gamma_\mathbf{k}^{in}(1-\rho_{11\mathbf{k}})\\&#xA;\end{aligned}&#xA;\end{equation}&#xA;\begin{equation}&#xA;\begin{aligned}&#xA;\frac{\partial \rho_{22\mathbf{k}}}{\partial t} =&amp;amp; -\frac{i}{\hbar} V_{0}^{2121}\sum_\mathbf{k'}(\rho_{21\mathbf{k'}}\rho_{21\mathbf{k}}^* - \rho_{21\mathbf{k'}}^*\rho_{21\mathbf{k}}) + \\&#xA;&amp;amp; -\Gamma (\rho_{22\mathbf{k}}-\rho_{22\mathbf{k}}^0)+\\&#xA;&amp;amp;+\eta\Gamma_\mathbf{k}^{in}(1-\rho_{22\mathbf{k}}) - (1-\eta)\Gamma_\mathbf{k}^{out}\rho_{22\mathbf{k}}\\&#xA;\end{aligned}&#xA;\end{equation}&#xA;\begin{equation}&#xA;\begin{aligned}&#xA;\frac{\partial \rho_{21\mathbf{k}}}{\partial t} = &amp;amp; - i\omega_{12} \rho_{21\mathbf{k}} + \\&#xA;&amp;amp;+ \frac{i}{\hbar} V_{0}^{2121}\sum_\mathbf{k'}  \rho_{21\mathbf{k'}}(\rho_{22\mathbf{k}}-\rho_{11\mathbf{k}}) +\\&#xA;&amp;amp; -\Gamma_\rho \,\rho_{21\mathbf{k}}&#xA;\end{aligned}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_0^{2121}, \Gamma, \eta, \Gamma_\mathbf{k}^{in}, \Gamma_\mathbf{k}^{out}, \Gamma_\rho$ are parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variables $\rho_{11\mathbf{k}}, \rho_{22\mathbf{k}}$ and $\rho_{21\mathbf{k}}$ are complex, and defined on a discretized space $\mathbf{k}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented a Newton-Raphson method in the complex plane, but I am not sure of the results. Could someone suggest me a package (Python, Matlab, Mathematica?) that I could use to confirm my results? I know that python can solve nonlinear equations, but I am not sure of how to deal with the $\mathbf{k}$ discretization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9803" LastActivityDate="2014-07-24T18:40:35.753" Title="Non linear system of equations with discretization on k-space" Tags="|nonlinear-equations|discretization|newton-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14249" PostTypeId="2" ParentId="12967" CreationDate="2014-07-24T17:01:52.890" Score="1" Body="&lt;p&gt;This is my very simple function to eliminate all the entries in the &lt;code&gt;bc_id&lt;/code&gt; matrix rows, and keeping only the diagonal entry. This how I usually set boundary conditions. I intensively used it. I know it is not optimal, it is not really elegant, but whenever I profiled my code, this function has never been a problem. Probably it would be good idea to include an assertion to check if the matrix is squared.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy import sparse&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;def set_diag(A,bc_id):&#xA;    ndofs = A.shape[0]&#xA;    diago = A.diagonal()&#xA;&#xA;    uno = np.ones((ndofs))&#xA;    uno[bc_id] = 0&#xA;    uno = sparse.dia_matrix((uno,0), shape = (ndofs,ndofs))&#xA;    # up to here I delete the rows&#xA;    # I multiply A by an identity matrix&#xA;    # where i set to zero the rows I want&#xA;    # to delete&#xA;    A = uno.dot(A)&#xA;    new_diag_entries = np.zeros((ndofs))&#xA;    # here I set the diagonal entries&#xA;    # equals to the value on the diagonal.&#xA;    # Use the second line if you want to&#xA;    # to set the diagonal entry to one&#xA;    new_diag_entries[bc_id] = diago[bc_id]&#xA;    #new_diag_entries[bc_id] = uno[bc_id]&#xA;    uno = sparse.dia_matrix((new_diag_entries,0), shape = (ndofs,ndofs))&#xA;    A = A+uno # here I set the diagonal entry&#xA;    return A&#xA;&#xA;A = sparse.csr_matrix([[1, 2, 3, 10], [4,5, 6,11], [7, 8, 9,12],[13,14,15,16]])&#xA;&#xA;bc_id = np.array([2,3])&#xA;&#xA;A = set_diag(A,bc_id)&#xA;&#xA;print A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/GRxw5.png&quot; alt=&quot;Sparsity pattern before.&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/1zkyF.png&quot; alt=&quot;Sparsity pattern after.&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="6574" LastEditorUserId="6574" LastEditDate="2014-07-25T08:25:00.777" LastActivityDate="2014-07-25T08:25:00.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14250" PostTypeId="2" ParentId="10664" CreationDate="2014-07-24T17:06:36.667" Score="0" Body="&lt;p&gt;I am somewhat confused by this question. Why are you evaluating the fluxes at a midpoint of the time step instead of evaluating them at $t = t_{n}$ or $t = t_{n+1}? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I have seen, fluxes are generally using data at the same time step to produce a value $q^{*}$ at the interface of two elements, where $q^{*}$ is the solution to the Riemann problem generated at the interface. This $q^{*}$ is then used as the variable to evaluate the flux function at, at this interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For nonlinear problems, finding $q^{*}$ is difficult and many people opt to use approximate solutions to $F(q^{*})$ via linearization about $\hat{q}$ which is usually some function that generates some sort of average of the right and left values of $q$ at the interface, or other fancier approaches to approximating the correct flux value at the interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether you are aiming for an explicit or implicit scheme, the approach is the same (at least in my experience), even if the value $q^{*}$ ends up helping create a coupled system in the implicit scheme case. I will recognize that I still have plenty to learn, so I definitely don't know everything about how to approach these problems, just the ways I have been taught.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2014-07-24T17:06:36.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14251" PostTypeId="1" AcceptedAnswerId="14277" CreationDate="2014-07-24T17:08:20.287" Score="1" ViewCount="621" Body="&lt;p&gt;I'm transforming cartesian images into polar images.&#xA;(x,y) =&gt; (angle, radius)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fill the polar image by iterating on each of its pixels and filling them by doing the reverse polar transform. For a given (angle, radius) pair, I'm interpolating in the source cartesian image at the following location:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xcartesianpoint = cos( angle ) * radius + center.x;&#xA;ycartesianpoint = sin( angle ) * radius + center.y;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(I do some computation with the polar image)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I transform back the polar image in cartesian coordinates. Again, I'm filling the destination image by doing the inverse transformation. For a given (x,y) point, I'm interpolating the value from the polar image point located at:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;angle  = atan2(ycartesianPxl, xcartesianPxl) + M_PI;&#xA;radius = getRadius(xcartesianPxl, ycartesianPxl);&#xA;&#xA;xpolarpoint = angle /_angleStep; // give the index in the polar image corresponding to the angle&#xA;ypolarpoint = (polarImage.height / maxRadius) * radius;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The points indexes are floating point values that I use to interpolate in the source image of the given transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code works well but give artifacts like those (first image is the source, second is the transformed source in polar transformed back in cartesian. The artifacts are more apparent on the pixels far from the center):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://imageshack.us/a/img539/3131/932528.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source Image http://imageshack.us/a/img539/3131/932528.png&lt;/a&gt;&#xA;&lt;a href=&quot;http://imageshack.us/a/img631/5459/3b3365.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transformed Image http://imageshack.us/a/img631/5459/3b3365.png&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I do with my polar images:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm polar transforming the source image with the goal of blurring it. This is done to get zoom/spiral like blurs. The blurring is done by doing an FFT of the polar image which is transformed back. The artifact wouldn't be a problem if the whole image was blurred, but it is possible for the user to specify a gain matte that modulates the blur radius for each pixel. So, some pixels may be fully blurred and others not at all.&lt;/p&gt;&#xA;" OwnerUserId="9581" LastEditorUserId="9581" LastEditDate="2014-07-29T23:27:50.107" LastActivityDate="2014-07-29T23:27:50.107" Title="How to minimize the artefact of a cartesian to polar transform followed by a polar to cartesian transform?" Tags="|algorithms|c++|interpolation|image-processing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14252" PostTypeId="2" ParentId="7248" CreationDate="2014-07-24T17:14:13.033" Score="0" Body="&lt;p&gt;In the past semester, I was busy writing some CFD code for a simulation I needed for an engineering project. I stumbled upon a few methods that can handle complex geometry and still maintain decent simplicity, such as Finite Difference Approximations. You should look for the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generalized Finite Difference (also look into Least-Squares approaches within this)&lt;/li&gt;&#xA;&lt;li&gt;Radial basis functions for solution interpolation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2014-07-24T17:14:13.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14253" PostTypeId="2" ParentId="14248" CreationDate="2014-07-24T18:40:35.753" Score="1" Body="&lt;p&gt;The discretization in $\mathbf{k}$-space is just a frequency discretization; if you took the inverse Fourier transform of your equations over $\mathbf{k}$-space, you would be back in &quot;real&quot; space (from the looks of it, your coordinates would be spatial coordinates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonlinear equation solvers operate on systems of equations; how you obtain those equations is mostly immaterial, except for properties of the equations (for instance, conditioning of the Jacobian matrix, avoiding roundoff error, etc.). You could use:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;fsolve&lt;/code&gt; in MATLAB&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;NSolve&lt;/code&gt; in Mathematica&lt;/li&gt;&#xA;&lt;li&gt;various functions from &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/optimize.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;scipy.optimize&lt;/code&gt;&lt;/a&gt; in Python&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bitbucket.org/petsc/petsc4py&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;petsc4py&lt;/code&gt;&lt;/a&gt; has more advanced nonlinear solves (Newton-Krylov methods with many different linear and nonlinear preconditioners); it is a Python interface to &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I strongly recommend using nonlinear solvers from a reputable library instead of writing them yourself, to save yourself time.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-24T18:40:35.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14254" PostTypeId="1" AcceptedAnswerId="14257" CreationDate="2014-07-24T20:50:59.210" Score="1" ViewCount="51" Body="&lt;p&gt;I have several matlab functions and one script that call those functions.  Suppose that they are 'a.m' 'f1.m' 'f2.m' and 'f3.m'. The script 'a.m' must open the functions files when it is manually opened and the run command is used. How can I do that with scripting techniques?&lt;/p&gt;&#xA;" OwnerUserId="9713" LastActivityDate="2014-07-25T12:17:14.910" Title="Open several dependent functions in matlab with scripting" Tags="|matlab|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14256" PostTypeId="2" ParentId="12967" CreationDate="2014-07-25T11:26:56.037" Score="2" Body="&lt;p&gt;I think it is most efficient if you work directly on the data of the CSR matrix. Have a look at this explanation of the &lt;a href=&quot;http://de.wikipedia.org/wiki/Compressed_Row_Storage&quot; rel=&quot;nofollow&quot;&gt;CSR data structure&lt;/a&gt; - it is in german, but the example may help you to understand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a python script with a function that sets a specified row of a CSR matrix to match the identity: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [13]: run t_csrdiag.py&#xA;sparse mat A was before&#xA;[[ 0  1  2  3]&#xA; [ 4  5  6  7]&#xA; [ 8  9 10 11]&#xA; [12 13 14 15]]&#xA;&#xA;index of the row that is modified: 2&#xA;&#xA;sparse mat A is now&#xA;[[ 0  1  2  3]&#xA; [ 4  5  6  7]&#xA; [ 0  0  1  0]&#xA; [12 13 14 15]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Feel free to loop it or vectorize it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.sparse as sps&#xA;import numpy as np&#xA;&#xA;&#xA;def main():&#xA;    # index of the row to become the identity row&#xA;    rowind = 2&#xA;&#xA;    A = np.arange(16).reshape((4, 4))&#xA;    A = sps.csr_matrix(A)&#xA;    print 'sparse mat A was before'&#xA;    print A.todense()&#xA;&#xA;    print '\nindex of the row that is modified: {0}'.format(rowind)&#xA;&#xA;    setcsrrow2id(A, rowind)&#xA;    print '\nsparse mat A is now'&#xA;    print A.todense()&#xA;&#xA;&#xA;def setcsrrow2id(amat, rowind):&#xA;    indptr = amat.indptr&#xA;    values = amat.data&#xA;    indxs = amat.indices&#xA;&#xA;    # get the range of the data that is changed&#xA;    rowpa = indptr[rowind]&#xA;    rowpb = indptr[rowind+1]&#xA;&#xA;    # new value and its new rowindex&#xA;    values[rowpa] = 1.0&#xA;    indxs[rowpa] = rowind&#xA;&#xA;    # number of new zero values&#xA;    diffvals = rowpb - rowpa - 1&#xA;&#xA;    # filter the data and indices and adjust the range&#xA;    values = np.r_[values[:rowpa+1], values[rowpb:]]&#xA;    indxs = np.r_[indxs[:rowpa+1], indxs[rowpb:]]&#xA;    indptr = np.r_[indptr[:rowind+1], indptr[rowind+1:]-diffvals]&#xA;&#xA;    # hard set the new sparse data&#xA;    amat.indptr = indptr&#xA;    amat.data = values&#xA;    amat.indices = indxs&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-07-25T11:26:56.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14257" PostTypeId="2" ParentId="14254" CreationDate="2014-07-25T12:17:14.910" Score="4" Body="&lt;p&gt;I'm assuming you mean &quot;open the functions for editing&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;edit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is actually a function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;edit(varargin)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the script can either have a line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;edit f1.m f2.m f3.m&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or can programatically construct a cell array of strings&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cstrFunctions = {'f1.m' 'f2.m' 'f3.m'}&#xA;edit(cstrFunctions{:});&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9810" LastActivityDate="2014-07-25T12:17:14.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14258" PostTypeId="2" ParentId="12956" CreationDate="2014-07-25T14:47:10.523" Score="1" Body="&lt;p&gt;I solve my own problem. As I said in the question I'm using PBC:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do k = 1, 3&#xA;    pos(k,i) = modulo(pos(k,i),length)&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and there is a dynamics for the positions $pos(k,i)$ of the particles&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pos(k,i)=pos(k,i)+vel(k,i)*dt+0.5*force(k,i)*dt**2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the computation of the diffusion coefficient I used a second variable of position, let say $pos0(k,i)$. This new variable evolve like the original one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pos0(k,i)=pos0(k,i)+vel(k,i)*dt+0.5*force(k,i)*dt**2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but to them I don't apply the PBC. Therefore the computation of the diffusion coefficient is something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do it=1,nstep&#xA;    diff=0&#xA;    do i = 1,nbody&#xA;        pos2(:)=pos2(:)+pos0(:,i)&#xA;        diff=diff+dot_product(pos0(:,i),pos0(:,i))&#xA;    end do&#xA;    diff=diff/nbody-dot_product(pos2(:),pos2(:))/nbody**2&#xA;end do&#xA;diff=diff/nstep/6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Other quantities instead are computed using the original variables $pos(k,i)$, for which the PBC holds:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do k = 1, 3&#xA;    pos(k,i) = modulo(pos(k,i),length)&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I hope that this answer can be helpful to who is looking for something similar.&lt;/p&gt;&#xA;" OwnerUserId="9514" LastActivityDate="2014-07-25T14:47:10.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14259" PostTypeId="1" AcceptedAnswerId="14261" CreationDate="2014-07-25T19:01:28.250" Score="13" ViewCount="3187" Body="&lt;p&gt;I'm trying to understand how the adjoint-based optimization method works for a PDE constrained optimization.  Particularly, I'm trying to understand why the adjoint method is more efficient for problems where the number of design variables is large, but the &quot;number of equations is small&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;What I understand:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following PDE constrained optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_\beta \text{ } I(\beta,u(\beta))\\ s.t. R(u(\beta))=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $I$ is a (sufficiently continuous) objective function of a vector design variables $\beta$ and a vector of field variable unknowns $u(\beta)$ which depend on the design variables, and $R(u)$ is the residual form of the PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, we can the first variations of I and R as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta I = \frac{\partial I}{\partial \beta}\delta\beta + \frac{\partial I}{\partial u}\delta u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta R = \frac{\partial R}{\partial \beta}\delta\beta + \frac{\partial R}{\partial u}\delta u = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introducing a vector of lagrange multipliers $\lambda$, the variation in the objective function can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta I = \frac{\partial I}{\partial \beta}\delta\beta + \frac{\partial I}{\partial u}\delta u + \lambda^T\left[ \frac{\partial R}{\partial \beta}\delta\beta + \frac{\partial R}{\partial u}\delta u\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rearranging terms, we can write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta I = \left[\frac{\partial I}{\partial \beta} + \lambda^T\frac{\partial R}{\partial \beta}\right]\delta\beta + \left[\frac{\partial I}{\partial u} + \lambda^T\frac{\partial R}{\partial u}\right]\delta u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, if we are able to solve for $\lambda$ such that $$\frac{\partial I}{\partial u} + \lambda^T\frac{\partial R}{\partial u}=0 \text{ (adjoint equation)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the gradient $\delta I= \left[\frac{\partial I}{\partial \beta} + \lambda^T\frac{\partial R}{\partial \beta}\right]\delta \beta$ is evaluated only in terms of the design variables $\beta$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, an adjoint based optimization algorithm would loop over the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Given current design variables $\beta$&lt;/li&gt;&#xA;&lt;li&gt;Solve for the field variables $u$ (from the PDE)&lt;/li&gt;&#xA;&lt;li&gt;Solve for the lagrange multipliers $\lambda$ (from the adjoint equation)&lt;/li&gt;&#xA;&lt;li&gt;Calculate gradients $\frac{\partial I}{\partial \beta}$&lt;/li&gt;&#xA;&lt;li&gt;Update design variables $\beta$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;My question&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does this adjoint 'trick' improve the cost of the optimization per iteration in the case where the number of design variables is large?  I've heard that the cost of gradient evaluation for the adjoint method is 'independent' of the number of design variables.  But how exactly is this true?   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure there's something very obvious that I'm somehow overlooking.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2023-09-05T15:54:39.610" Title="Understanding the cost of adjoint method for pde-constrained optimization" Tags="|optimization|pde|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14260" PostTypeId="2" ParentId="14259" CreationDate="2014-07-25T20:06:38.073" Score="9" Body="&lt;p&gt;In a nutshell, the advantage comes from the fact that to compute derivatives of the reduced objective $I(\beta,u(\beta))$, you do not really need to know the derivative of $u(\beta)$ with respect to $\beta$ as a separate object, but only that part of it that leads to variations in $I(\beta,u(\beta))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me switch to a notation I'm a bit more comfortable with: &#xA;$$\min_{y,u} J(y,u) \quad\text{subject to}\quad  e(y,u)=0$$&#xA;($u$ being the design variable, $y$ being the state variable, and $J$ being the objective).&#xA;Let's say $e(y,u)$ is nice enough to apply the implicit function theorem, so the equation $e(y,u)=0$ has a unique solution $y(u)$ which is continuously differentiable with respect to $u$, and the derivative $y'(u)$ is given by the solution of&#xA;$$e_y(y(u),u)y'(u) + e_u(y(u),u) = 0\tag{1}$$&#xA;($e_y$ and $e_u$ being the partial derivatives).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means you can define the reduced objective $j(u):=J(y(u),u)$, which is differentiable as well (if $J(y,u)$ is). &#xA;One way to characterize the gradient $\nabla j(u)$ is via directional derivatives (e.g., compute all the partial derivatives with respect to a basis of the design space).&#xA;Here, the directional derivative in direction $h$ is given by the chain rule as&#xA;$$j'(u;h) = \langle J_y(y(u),u),y'(u)h \rangle + \langle J_u(y(u),u),h\rangle.\tag{2}$$&#xA;If $J$ is nice, the only difficult thing to compute is $y'(u)h$ for given $h$. This can be done by multiplying $(1)$ with $h$ from the right and solving for $y'(u)h$ (which the implicit function theorem allows), i.e., computing&#xA;$$[y'(u)h] = e_y(y(u),u)^{-1} [e_u(y(u),u)h]\tag{3}$$&#xA;and plugging this expression into $(2)$.&#xA;In PDE-constrained optimization, this amounts to solving a linearized PDE &lt;em&gt;for every basis vector&lt;/em&gt; $h$ of the design space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if we find an operator $\nabla j$ such that&#xA;$$j'(u;h) = \langle \nabla j,h\rangle\qquad \text{for all }h,$$&#xA;then this must be the desired gradient. Looking at $(1)$, we can write&#xA;$$ \langle J_y(y(u),u),y'(u)h \rangle =  \langle y'(u)^*J_y(y(u),u),h \rangle $$&#xA;(with $y'(u)^*$ being the adjoint operator), so all we need to compute is $y'(u)^*j_y(y(u),u)$. Using that $(AB)^* = B^* A^*$, this can be done using $(3)$, i.e., computing&#xA;$$\lambda:=  e_y(y(u),u)^{-*}J_y(y(u),u)$$&#xA;and setting&#xA;$$\nabla j(u) = e_u(y(u),u)^*\lambda +J_u(y(u),u).$$&#xA;In PDE-constrained optimization, $J_y(y(u),u)$ is usually some sort of residual, and computing $\lambda$ involves solving a &lt;em&gt;single&lt;/em&gt; (linear) adjoint PDE, independent of the dimension of the design space. (In fact, this even works for distributed parameters, i.e., if $u$ is a function in some infinite-dimensional Banach space, where the first approach is infeasible.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-07-25T20:12:39.833" LastActivityDate="2014-07-25T20:12:39.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14261" PostTypeId="2" ParentId="14259" CreationDate="2014-07-25T20:43:46.667" Score="13" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How does this adjoint 'trick' improve the cost of the optimization per iteration in the case where the number of design variables is large? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think about the cost from a linear algebra perspective. (See &lt;a href=&quot;http://www-math.mit.edu/~stevenj//18.336/adjoint.pdf&quot;&gt;these notes by Stephen G. Johnson&lt;/a&gt;, which I find more intuitive than the Lagrange multiplier approach). The forward approach amounts to solving for sensitivities directly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial{u}}{\partial{\beta}} = -\left(\frac{\partial{R}}{\partial{u}}\right)^{-1}\frac{\partial{R}}{\partial{\beta}}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which involves solving a linear system for each parameter in the vector $\beta$, then evaluating&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\mathrm{d}I}{\mathrm{d}\beta} = \frac{\partial{I}}{\partial{\beta}} + \frac{\partial{I}}{\partial{u}}\frac{\partial{u}}{\partial{\beta}},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathrm{d}$ denotes a total derivative, and $\partial$ denotes a partial derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The adjoint approach notes that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\mathrm{d}I}{\mathrm{d}\beta} = \frac{\partial{I}}{\partial{\beta}} - \frac{\partial{I}}{\partial{u}}\left(\frac{\partial{R}}{\partial{u}}\right)^{-1}\frac{\partial{R}}{\partial{\beta}},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the adjoint variable (Lagrange multiplier) $\lambda$ can be defined by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;-\frac{\partial{I}}{\partial{u}}\left(\frac{\partial{R}}{\partial{u}}\right)^{-1} = \lambda^{T},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which corresponds to the adjoint equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial{I}}{\partial{u}} + \lambda^{T}\frac{\partial{R}}{\partial{u}} = 0.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This regrouping of terms requires only one linear solve, instead of a linear solve for each parameter, which makes adjoint evaluation cheap for the many parameter case. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I've heard that the cost of gradient evaluation for the adjoint method is 'independent' of the number of design variables. But how exactly is this true?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's not totally independent; presumably the cost of evaluating $(\partial{I}/\partial{\beta})$ and $(\partial{R}/\partial{\beta})$ will increase with the number of parameters. The linear solves, however, will still be of the same size, as long as the size of $u$ does not change. The assumption is that the solves are much more expensive than the function evaluations.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-25T20:43:46.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14262" PostTypeId="1" CreationDate="2014-07-26T03:51:43.833" Score="4" ViewCount="219" Body="&lt;p&gt;Is there some flag to let the Fortran compiler pre calculate a math expression before compiling it?. I have to write expressions that contain many small 4x4 matrix multiplications. The thing is, most of the elements of these small 4x4 matrices are zero, so in terms of performance it's better to actually do the multiplication manually and write down the resulting expression, that way you eliminate the unnecessary multiplications by 0. Doing this is error prone, and I thought that compiler optimizations like -O3 would take care of this but they don't. I read about tools like symPy and matlab that let you write expressions, manipulate and simplify them and generate the code for Fortran or C, but if there is some compiler flag for this would be even better.&lt;/p&gt;&#xA;" OwnerUserId="7871" LastActivityDate="2014-08-01T13:51:02.823" Title="Pre calculate mathematical expressions in Fortran 90" Tags="|optimization|fortran|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14264" PostTypeId="1" CreationDate="2014-07-27T10:14:01.153" Score="3" ViewCount="407" Body="&lt;p&gt;At our research facility we routinely perform what are called PALM (&lt;a href=&quot;http://en.wikipedia.org/wiki/Photoactivated_localization_microscopy&quot; rel=&quot;nofollow&quot;&gt;Photo-activated localisation microscopy&lt;/a&gt;) or STORM (&lt;a href=&quot;http://en.wikipedia.org/wiki/Photoactivated_localization_microscopy&quot; rel=&quot;nofollow&quot;&gt;stochastic optical reconstruction microscopy&lt;/a&gt;) experiments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read all about it in the links above but in short it boils down to this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Normal optical microscopy is diffraction limited. We can only resolve features bigger than say 200-300 nm because of the very nature of the wavelengths and optics we use.&lt;/li&gt;&#xA;&lt;li&gt;We can circumvent this resolution limitation by finding ways to observe point emitters in a sample and using what is known about their &quot;point spread function&quot; to fit the location of those emitters with sub diffraction limit accuracy.&lt;/li&gt;&#xA;&lt;li&gt;Observing many such point emitters in a sample allows us to reconstruct an image of the entire sample with enhanced resolution.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Practically, we achieve all this by recording long (50k frames) &quot;movies&quot; of samples using EM-CCD cameras which produce 16 bit 512*512 pixel images. As such, these datasets grow quite big, particularly if we run lots of experiments daily/weekly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next we run established fitting routines on these datasets, such as the ones implemented &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/23208219&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently we do all this in what I call &quot;Offline mode&quot;. We record these data sets, often dozens or hundreds of times a week. Store them on USB hard drives and run the above referenced analyses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads to issues:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Yearly, we invest thousands of euros in disks because we need to store all erudite and might want to hang on to it for future reference. However, this data cannot be properly indexed and managed. Everyone uses their own schemes for naming/folders/...&lt;/li&gt;&#xA;&lt;li&gt;Disks can go missing&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Reading so much about &quot;cloud computing&quot; lately, I wanted to try and find better solutions to the issue. I think that it should be possible to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Install some kind of Dropbox like client on each measurement computer.&lt;/li&gt;&#xA;&lt;li&gt;Upload all raw image stacks to a cloud storage facility&lt;/li&gt;&#xA;&lt;li&gt;Expose that cloud storage as a virtual disk to nice software such as &lt;a href=&quot;https://www.openmicroscopy.org/site&quot; rel=&quot;nofollow&quot;&gt;The Open microscopy Environment&lt;/a&gt; which already offers solutions for indexing/annotating ur type of data&lt;/li&gt;&#xA;&lt;li&gt;Leverage the computational power of cloud systems to also speed up the analysis&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In this respect I have contacted several cloud providers (MS Azure, Google, Amazon) but in my experience it is very hard to get in touch with someone knowledgable enough to first &quot;get&quot; our intended application and next to provide viable potential solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, although I grasp most of the concepts involved in modern cloud platforms I lack the time and in depth knowledge to roll my own solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless I cannot imagine that we would be the first people to run into this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody have an idea in this respect?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DISCLAIMER: It could be that this question is out of scope for this particular SA site but I did not seem to find another where this would be better suited (or I missed it).&lt;/p&gt;&#xA;" OwnerUserId="9823" LastEditorUserId="1804" LastEditDate="2014-07-27T10:28:12.753" LastActivityDate="2014-07-27T21:08:24.170" Title="Cloud solutions for microscopy image analysis" Tags="|image-processing|data-management|cloud-computing|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="14265" PostTypeId="2" ParentId="14180" CreationDate="2014-07-27T13:19:46.797" Score="8" Body="&lt;p&gt;I have to disagree with Paul: Functional analysis is a beautiful, elegant topic with an enormous range of applications (but &lt;em&gt;de gustibus...&lt;/em&gt; :)). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in any case, you don't need to know a lot of (pure) functional analysis to understand finite element methods: Normed vector spaces, (strong) convergence, completeness, inner products and their induced norms, orthogonality, dual spaces and adjoint operators. Except for the last two, these topics should be already familiar if you already know some multivariate analysis and linear algebra (modulo some technicalities). They are all treated in (the first four chapters of) &lt;a href=&quot;http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471504599.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kreyszig's &lt;em&gt;Introductory Functional Analysis&lt;/em&gt;&lt;/a&gt; which was already recommended (and which I found to be a very pleasant read).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But much more than the behavior of abstract vector spaces, you need to understand the behavior of Lebesgue and Sobolev spaces, as well as the theory of (weak solutions) of partial differential equations. (From your &lt;a href=&quot;https://scicomp.stackexchange.com/q/14173&quot;&gt;previous question&lt;/a&gt;, I would say that this is precisely what you have problems with.) These topics (which some would classify as (applied) functional analysis) are usually summarized, but not treated in detail, in what I (as a mathematician) would call the fundamental and standard finite element books:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ebooks.cambridge.org/ebook.jsf?bid=CBO9780511618635&quot; rel=&quot;nofollow noreferrer&quot;&gt;Braess: &lt;em&gt;Finite Elements&lt;/em&gt;, Cambridge University Press, 2007&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/mathematics/applications/book/978-0-387-75933-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brenner, Scott: &lt;em&gt;Mathematical Theory of Finite Elements&lt;/em&gt;, Springer, 2008&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/mathematics/analysis/book/978-0-387-20574-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ern, Guermond: &lt;em&gt;Theory and Practice of Finite Elements&lt;/em&gt;, Springer, 2004&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are listed roughly in order of mathematical abstractness, and should be enough to be on the same page as the scientists you're talking to. If you want more detail, you need to look at books on the theory of partial differential equations; the standard references would be (again in order of abstractness)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ams.org/bookstore-getitem/item=gsm-19-R&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evans: &lt;em&gt;Partial Differential Equations&lt;/em&gt;, AMS, 2010&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/mathematics/dynamical+systems/book/978-0-387-00444-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Renardy, Rogers: &lt;em&gt;An Introduction to Partial Differential Equations&lt;/em&gt;, Springer, 2004&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cambridge.org/de/academic/subjects/mathematics/differential-and-integral-equations-dynamical-systems-and-co/partial-differential-equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wloka: &lt;em&gt;Partial Differential Equations&lt;/em&gt;, Cambridge University Press, 1987&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In fact, Chapter 5 and 6, as well as Appendix D and E, of Evans' book should cover most of the theoretical background you need for the (mathematical) theory of finite element methods.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-07-27T13:19:46.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14266" PostTypeId="2" ParentId="14264" CreationDate="2014-07-27T13:47:52.393" Score="1" Body="&lt;p&gt;Based on our discussion above, I'd recommend the following: Find your local, regional, or national supercomputing, HPC, or research computing outfit and learn about their support for these kinds of data storage and computing activities. This kind of workload is one of the many things covered under the umbrella of &quot;Big Data&quot;, and all of the supercomputing centers that I know of are moving to support these kinds of things. They are also being actively pushed to do so by their funding agencies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many cases, they will be able to support your workload for free or at a much reduced cost compared to commercial cloud options because they are subsidized to do so at the University, regional government, or national government level. Multi-terabyte (which this sounds like) data ingest or at-rest storage costs for commercial clouds are astronomical in comparison, and they typically don't have a mandate to support science. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-07-27T13:47:52.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14267" PostTypeId="2" ParentId="14246" CreationDate="2014-07-27T17:01:48.507" Score="3" Body="&lt;p&gt;Your difference stencil is wrong for nodes at the boundary (which in your 2x2 case happens to be every row). A simple way to see this is to realize that if you applied the Laplace operator to a constant function, the result is zero. Similarly, if you applied the Laplace matrix to a vector where all components have the same value (the discretized version of a constant function) then you need to get the zero vector. This implies that the sum of elements in each row must be zero -- which it is for the 5-point stencil you are using at interior nodes, but it isn't the way you wrote it down. You just need to do something different in those rows that correspond to boundary nodes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-27T17:01:48.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14268" PostTypeId="2" ParentId="14222" CreationDate="2014-07-27T17:25:30.297" Score="1" Body="&lt;p&gt;If you have $M$ atoms, your wave function depends on $3M$ variables. If you wanted to discretize this function on a uniform mesh with $N$ nodes in each of these directions (or with $N$ one-dimensional shape functions), you'd need a total of $N^{3M}$ unknowns -- far too many for any interesting number of electrons $M$. To give just one example, if you'd just use $N=10$ nodes in each direction, and you had just 3 electrons, you'd already have a system of size $10^9$, very much at the limit of what one can do today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this consideration follows that it is not possible to consider the problem with all electrons at the same time -- you need to restrict yourself to one or two electrons at a time. This naturally leads to you to methods such as the Hartree Fock method that iterates over electrons while keeping the rest of the system fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know the field well enough but imagine that there are a number of highly cited and well written review papers on the topic.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-07-27T17:25:30.297" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14269" PostTypeId="1" CreationDate="2014-07-27T20:23:33.787" Score="-1" ViewCount="112" Body="&lt;p&gt;I would like to solve the following equation, wrt $n(e)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(n(e))=g(n(e)) + \int_{\alpha}^{e} w(n(x))dx $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral there it confuses me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion on how I can implement this on a the computer (I am mostly Matlab user)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or simplicity assume the integral is between some values $e_1$ and $e_2$ although in general can also be thought as an indefinite integral. Again for simplicity, assume:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(n)=\frac{1}{1-n} \; \rm{and} \;\; g(n) = \frac{n^2}{1-n^3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I picked those function randomly, just for exposition, if it helps i can post the ones i really have but are rather complicated expressions. In the code, i have those &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c= @(n) (1+rtax)*k0+wtax*e*n-(1+grate)*k; &#xA;labor= @(n) n - ((1-eta)*c(n))/eta*wtax*e&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where everything is known apart from n. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;while the $$ w(n) = (1-\eta) \left[c^\eta (1-n)^{(1-\eta)}\right]^{-\mu} \left[\frac{en}{L} - \frac{a}{k}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;assume everything is known apart from $n$, and $c$ is defined as in the code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PS&lt;/strong&gt; : This is a first order condition from a stochastic maximization, and consider e as a particular value in specific state of the world, so n depends on e, implicitly and cannot be taken out of the integral. &lt;/p&gt;&#xA;" OwnerUserId="9826" LastEditorUserId="9826" LastEditDate="2014-07-30T22:58:53.717" LastActivityDate="2014-07-30T22:58:53.717" Title="solving a non-linear equation with integrals involved" Tags="|nonlinear-equations|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="14270" PostTypeId="2" ParentId="14264" CreationDate="2014-07-27T21:08:24.170" Score="3" Body="&lt;p&gt;Awesome to find another microscopist on this site.  Welcome!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short Answer: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;There is no magic-bullet on the market that will solve this problem space for you.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Longer Answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Putting together a few commercailly available offerings gets you pretty far.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At my work we take ~1-10 thousand $4\,096\times25\,000$ images weekly, which puts us in a similar problem-scale space, so I wanted to share my experiences first in the form of some quick bullet points, then a more through system description of what we created to deal with this all.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Compress Early.  Investing time (lossless) compressing your images will save on network IO, storage costs, transfer times, and almost always will speed analysis.  PALM and STORM are great candidates given typically low per-image entropy.  We built a tiny cluster to get this all done.  With 32 cores it still cost less than 2% of a new STORM setup :)&lt;/li&gt;&#xA;&lt;li&gt;Archive/Upload continuously.  We use s3 for storage and have a constant background thread uploading imagery.  Your ability to upload to AWS will be limited only by your local connection.&lt;/li&gt;&#xA;&lt;li&gt;Use s3 or similar competitive products.  You really can not beat the storage costs of this service, unless you have some form of free/subsidized IT, electricity, or hard-drives.  We use s3 buckets for each sample, and eventually migrate them to Glacier storage to reduce costs on already analyzed data.&lt;/li&gt;&#xA;&lt;li&gt;Leverage Cloud IO magic.  Any particular computer pulling a single frame from AWS gets typically 5-15MB/s, however 500 computers pulling a frame each get the same rate, totaling in GB/s.  This is really powerful for parallel analyses.  &lt;a href=&quot;https://www.youtube.com/watch?v=7mmcEl_1CPw&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a great video about this with a microscopy focus.&lt;/a&gt;  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is the system diagram that I drew for some documentation:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/6zIhB.png&quot; alt=&quot;A look at our internals&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, we have a number of microscopes feeding a compression-cluster that all eventually converge to one IO endpoint that syncs with s3 into sample based buckets.  This stack is not yet OS, but I can share it in a limited way fairly easily with anyone interested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a specialized solution, but any fraction of this could be killed by a couple of shell-scripts with &lt;a href=&quot;http://www.gnu.org/software/parallel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gnu parallel&lt;/a&gt;, &lt;a href=&quot;http://www.imagemagick.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;image-magic&lt;/a&gt;, &lt;a href=&quot;https://code.google.com/p/s3fs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;s3fs&lt;/a&gt;, and a reasonably sized compute cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this same stuff goes if you have a supporting HPC facility, it will just be cheaper :)&lt;/p&gt;&#xA;" OwnerUserId="53" LastActivityDate="2014-07-27T21:08:24.170" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14271" PostTypeId="1" CreationDate="2014-07-27T23:11:13.750" Score="7" ViewCount="1747" Body="&lt;p&gt;I'm trying to write a Python program to use Tanh-sinh quadrature to compute the value of &#xA;\begin{equation} \int_{-1}^1 \frac{dx}{\sqrt{1-x^2}} \end{equation} &#xA;but although the program converges to a sensible value with no errors in every case, it's not converging to the correct value (which is $\pi$ for this particular integral) and I can't find the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of asking for a desired level of accuracy, the program asks for the number of function evaluations wanted, to make comparisons of convergence with simpler integration methods easier. The number of evaluations needs to be an odd number as the approximation used is &#xA;\begin{equation} \int_{-1}^1 f(x) dx = \sum_{k=-n}^n w_k f(x_k) \end{equation} Can anyone suggest what I might have done wrong?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;&#xA;def func(x):&#xA;    # Function to be integrated, with singular points set = 0&#xA;    if x == 1 or x == -1 :&#xA;        return 0&#xA;    else:&#xA;        return 1 / math.sqrt(1 - x ** 2)&#xA;&#xA;# Input number of evaluations&#xA;N = input(&quot;Please enter number of evaluations \n&quot;)&#xA;if N % 2 == 0:&#xA;    print &quot;The number of evaluations must be odd&quot;&#xA;else:&#xA;    print &quot;N =&quot;, N  &#xA;&#xA;# Set step size&#xA;h = 2.0 / (N - 1)&#xA;print &quot;h =&quot;, h&#xA;&#xA;# k ranges from -(N-1)/2 to +(N-1)/2&#xA;k = -1 * ((N - 1) / 2.0)&#xA;k_max  = ((N - 1) / 2.0)&#xA;sum = 0&#xA;&#xA;# Loop across integration interval&#xA;while k &amp;lt; k_max + 1:&#xA;&#xA;    # Compute abscissa&#xA;    x_k = math.tanh(math.pi * 0.5 * math.sinh(k * h))&#xA;&#xA;    # Compute weight&#xA;    numerator = 0.5 * h * math.pi * math.cosh(k * h)&#xA;    denominator = math.pow(math.cosh(0.5 * math.pi * math.sinh(k * h)),2)&#xA;    w_k =  numerator / denominator&#xA;&#xA;    sum += w_k * func(x_k)&#xA;&#xA;    k += 1&#xA;&#xA;print &quot;Integral =&quot;, sum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9828" LastActivityDate="2021-10-15T11:11:11.810" Title="Tanh-sinh quadrature numerical integration method converging to wrong value" Tags="|python|quadrature|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14272" PostTypeId="2" ParentId="14271" CreationDate="2014-07-28T01:36:06.380" Score="5" Body="&lt;p&gt;Your approach to selecting $h$ based on $N$ isn't a very sophisticated one and probably isn't getting $h$ to be small enough.  See these notes for a more reasonable way to adjust $h$ and estimate the error in the integral evaluation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.davidhbailey.com/dhbpapers/dhb-tanh-sinh.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.davidhbailey.com/dhbpapers/dhb-tanh-sinh.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-07-28T01:36:06.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14273" PostTypeId="2" ParentId="12956" CreationDate="2014-07-28T03:53:21.277" Score="2" Body="&lt;p&gt;You need to unfold the positions after the simulation is done. I use the following subroutine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine unfold_positions(L, X, Xu)&#xA;! Unwinds periodic positions. It is using the positions of particles from the&#xA;! first time step as a reference and then tracks them as they evolve possibly&#xA;! outside of this box. The X and Xu arrays are of the type X(:, j, i), which&#xA;! are the (x,y,z) coordinates of the j-th particle in the i-th time step.&#xA;real(dp), intent(in) :: L ! Box length&#xA;! Positions in [0, L]^3 with possible jumps:&#xA;real(dp), intent(in) :: X(:, :, :)&#xA;! Unwinded positions in (-oo, oo)^3 with no jumps (continuous):&#xA;real(dp), intent(out) :: Xu(:, :, :)&#xA;real(dp) :: d(3), Xj(3)&#xA;integer :: i, j&#xA;Xu(:, :, 1) = X(:, :, 1)&#xA;do i = 2, size(X, 3)&#xA;    do j = 1, size(X, 2)&#xA;        Xj = X(:, j, i-1) - X(:, j, i) + [L/2, L/2, L/2]&#xA;        Xj = Xj - L*floor(Xj/L)&#xA;        d = [L/2, L/2, L/2] - Xj&#xA;        Xu(:, j, i) = Xu(:, j, i-1) + d&#xA;    end do&#xA;end do&#xA;end subroutine&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think you can replace the line &lt;code&gt;Xj = Xj - L*floor(Xj/L)&lt;/code&gt; with &lt;code&gt;Xj = modulo(Xj, L)&lt;/code&gt;, good point, I didn't realize that you can use &lt;code&gt;modulo&lt;/code&gt; for floating point numbers.&lt;/p&gt;&#xA;" OwnerUserId="236" LastEditorUserId="8094" LastEditDate="2016-05-18T18:44:22.610" LastActivityDate="2016-05-18T18:44:22.610" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14275" PostTypeId="2" ParentId="14269" CreationDate="2014-07-28T14:08:25.603" Score="0" Body="&lt;p&gt;I think the easier way to solve this if the integral have limits is to just use numeric methods.  The easiest today with a computer is to just select a 'de' and do a Riemann sum of the value calculated for each e to solve the integral.  I would go with a de that is say a 10,000th of the integral interval (e2-e1) and decrease that value until you find the solution has the significant digits you want staying the same.  Solving time can be an issue depending on the difficulty of the function and the sensitivity it has to the de interval but it is not easy to predict, so testing it is the best option.  At the very least you will learn a lot about the function behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are in doubt on how to program this, let me know&lt;/p&gt;&#xA;" OwnerUserId="9659" LastActivityDate="2014-07-28T14:08:25.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14276" PostTypeId="1" CreationDate="2014-07-28T14:19:52.323" Score="0" ViewCount="87" Body="&lt;p&gt;Is checking the equivalence of two convex polytopes $p^{s}$ and $p^{t}$ NP-hard?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p^{s}= CH\{ \cup &amp;lt;p^{s,a_1},...., p^{s,a_m}&amp;gt; \} $   // CH is convex hull computed on union of a polynomial number of polytopes $p^{s,a_i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p^{t}= CH\{ \cup &amp;lt;p^{t,b_1},...., p^{t,b_n}&amp;gt; \}$     // CH is computed on union of a polynomial number of polytopes $p^{t,b_i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p^{s,a_i}= \{(x_{i1},.....,x_{ik}) |  l_{ij} \leq x_{ij} \leq u_{ij} (j=1,..., k);  \sum_{j=1}^k x_{ij} =1 ;  l_{ij}, u_{ij} \text{are non-negative rational numbers for} \; j=1,...,k\}$\&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p^{t,b_i}= \{(y_{i1},.....,y_{ir}) |  l'_{ij} \leq y_{ij} \leq u'_{ij} (j=1,..., r);  \sum_{j=1}^r y_{ij} =1 ;  l'_{ij}, u'_{ij} \text{are non-negative rational numbers for} \; j=1,...,r\}$\&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="917" LastEditDate="2014-07-29T13:51:15.970" LastActivityDate="2014-07-29T13:51:15.970" Title="difference of polytopes in $\mathbb{R}^n$" Tags="|computational-geometry|linear-programming|complexity|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14277" PostTypeId="2" ParentId="14251" CreationDate="2014-07-28T18:48:22.577" Score="1" Body="&lt;p&gt;Since the underlying grid for your polar representation becomes coarser in the circumference as you go further from the center, it's not surprising you get artifacts there. My simplest suggestion then would be to just use many more pixels in the polar representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not clear to me how you are &quot;interpolating&quot; from the code you provided. Are you simply taking the value from the nearest pixel? A least-squares interpolation would likely be better - for a given pixel you want to find the color for, find the 4 surrounding pixels of the previous image and compute an average weighted by the inverse of their distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of three options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Significantly increase the resolution of your polar representation.&lt;/li&gt;&#xA;&lt;li&gt;Instead of a linear interpolation, create spectral representations of the images (e.g. by FFT). This will be considerably more accurate.&lt;/li&gt;&#xA;&lt;li&gt;Apply a blurring/gaussian filter to the final image or anti-aliasing. This won't actually improve the quality of the transformation, but it will get rid of the jagged border edges.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2014-07-29T14:28:19.923" LastActivityDate="2014-07-29T14:28:19.923" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14278" PostTypeId="2" ParentId="14222" CreationDate="2014-07-28T19:21:37.513" Score="6" Body="&lt;p&gt;The solutions for the equation are in&#xA;$$\psi \in \mathbb{C}^{3M}\times\mathbb{R}^+ \enspace .$$&#xA;If the number of electrons is &lt;em&gt;small enough&lt;/em&gt; you can just use any &lt;em&gt;traditional&lt;/em&gt; method. Like a domain discretization method (Finite Difference, Finite Element, Boundary Element), or a pseudospectral method. Since solving this equation is not more difficult than solving a multidimensional wave equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of bigger systems some &lt;em&gt;trick&lt;/em&gt; is necessary to get the solution. We replace the electron-electron interaction for the interaction of an electron with a cloud of electrons (a mean field approximation of the rest of them), and then solve in a self-consistent fashion (due to the nonlinearity that come from the mean field term). This is done in &lt;a href=&quot;https://en.wikipedia.org/wiki/Hartree%E2%80%93Fock_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hartree-Fock&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Density_functional_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Density Functional Theory&lt;/a&gt; (DFT). Where the original differential equation is transformed into a variational formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DFT is the most common method nowadays, and the advantage is that all the equations are formulated in terms of the electron density and not in terms of the wave equations. So, the equations lie in a 3 dimensional space. One book that describes both of these methods is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Thijssen, Jos. Computational physics. Cambridge University Press, 2007. &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521833469&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amazon link&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-05-10T21:25:46.693" LastActivityDate="2017-05-10T21:25:46.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14279" PostTypeId="2" ParentId="14242" CreationDate="2014-07-28T21:17:22.603" Score="2" Body="&lt;p&gt;The answer to my question is that dgemm operates in a column major format, where C and the openCL kernel that I was implementing are row major.  I changed my kernel code to the one below and that fixed everything. I found this out because I saw a Cblas_dgemm call where the first operator was CBLAS_ROW_MAJOR, telling the cblas solver to work in a row major format. This prompted me to do further research into the actual way dgemm works and found out that there is no option for making it row major in this implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kernel:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__kernel void clAxBtrans(__global double* A,&#xA;                     __global double* B,&#xA;                     int rowsA,&#xA;                     int colsA,&#xA;                     int rowsB,&#xA;                     int colsB,&#xA;                     __global double* C)&#xA;{&#xA;int globalx = get_global_id(0);&#xA;int globaly = get_global_id(1);&#xA;double tmp = 0;&#xA;double tmp0 = 0;&#xA;double tmp1 = 0;&#xA;double tmp2 = 0;&#xA;double tmp3 = 0;&#xA;double tmp4 = 0;&#xA;double tmp5 = 0;&#xA;tmp0 = B[0 * rowsB + globaly] * A[globalx * rowsA + 0];&#xA;tmp1 = B[1 * rowsB + globaly] * A[globalx * rowsA + 1];&#xA;tmp2 = B[2 * rowsB + globaly] * A[globalx * rowsA + 2];&#xA;tmp3 = B[3 * rowsB + globaly] * A[globalx * rowsA + 3];&#xA;tmp4 = B[4 * rowsB + globaly] * A[globalx * rowsA + 4];&#xA;tmp5 = B[5 * rowsB + globaly] * A[globalx * rowsA + 5];&#xA;tmp = tmp0 + tmp1 + tmp2 + tmp3 + tmp4 + tmp5;&#xA;C[globalx * rowsB + globaly] = tmp;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the global variables are the same as described in the question.&lt;/p&gt;&#xA;" OwnerUserId="9827" LastEditorUserId="9827" LastEditDate="2014-08-20T20:17:15.623" LastActivityDate="2014-08-20T20:17:15.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14280" PostTypeId="2" ParentId="14276" CreationDate="2014-07-29T03:44:28.470" Score="4" Body="&lt;p&gt;That heavily depends on the representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're given $P_1$ and $P_2$ as systems of linear inequalities (or, dually, as the convex hull of a finite set of points) with finite precision, you can reduce each linear system (or finite point set) to an irredundant system of linear inequalities by solving linearly many linear programs.  Then scale each inequality so that it has a canonical representation, sort them, and check whether the two representations are equal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're given $P_1$ and $P_2$ as systems of linear inequalities with real number coefficients (and you're using a complexity theory that has a notion of &quot;NP-hard&quot; that handles real number computation, like the one derived from the real RAM model), then checking polytope equivalence is exactly as hard as linear programming in said model.  (Which is to say that it's still, to my knowledge, an open problem.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you allow other primitives (such as &quot;intersect with the integer lattice then take the convex hull&quot;) you can make the problem NP-hard.&lt;/p&gt;&#xA;" OwnerUserId="3525" LastActivityDate="2014-07-29T03:44:28.470" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14281" PostTypeId="2" ParentId="14222" CreationDate="2014-07-29T17:34:31.183" Score="3" Body="&lt;p&gt;You want to solve for 3 to 10 particle systems (3D per particle)? As far as I am aware, mean field theories do not work especially well for so few particles, but it seems there has been DFT work on diatomic molecules.&lt;/p&gt;&#xA;&lt;p&gt;Is this a system where Born-Oppenheimer is valid? If so, I might be inclined to expand the electronic wavefunction using a linear combination of Slater determinants possibly using sparse grid or spectral sparse grids &lt;a href=&quot;http://www.math.purdue.edu/%7Eshen/pub/SISC12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper perhaps could help&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Another option is to try using a tight-binding approach, although the fact that you mentioned absorbing boundary conditions suggests you may be thinking of problems involving ionization/dissociation. TB would mostly be useful if you were trying to approximate low level states.&lt;/p&gt;&#xA;&lt;p&gt;Possibly something like the multi-configurational time-dependent Hartree-Fock method could work here &lt;a href=&quot;http://arxiv.org/abs/1101.4832&quot; rel=&quot;nofollow noreferrer&quot;&gt;MCTDHF&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Finally, you could look at quantum Monte Carlo methods. These are the methods by which exchange and correlation functional models for single atoms are obtained to do DFT calculations. It looks like there are poly-atomic extensions. (I'm out of link privileges).&lt;/p&gt;&#xA;" OwnerUserId="9838" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-07-29T17:39:41.180" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14285" PostTypeId="1" CreationDate="2014-07-30T03:27:52.563" Score="1" ViewCount="2465" Body="&lt;p&gt;I'm using Matlab's fmincon to solve a constrained nonlinear optimization for varying sets of parameters in the objective function. I have 2 parameters and have 400 unique combination of these 2 parameters. Within a particular combination of parameters I use a range of start points (100 start points). Trying to do rough simulation of multi-start functionality using optimization toolbox.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a given combination of parameters in objective function, majority of the start points would solve the optimization problem and &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/exit-flags-and-exit-messages.html&quot; rel=&quot;nofollow&quot;&gt;exit&lt;/a&gt; with a positive exitflag like 1 and in some cases 2. But for very small number of cases, the program would finish with an exitflag of -3 which corresponds to &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/fmincon.html&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;objective function at current iteration went below options.ObjectiveLimit and maximum constraint violation was less than options.TolCon.&lt;/em&gt;&lt;/a&gt;. In such cases, the output message mentions about problem might be unbounded. One example message is below&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Problem appears unbounded.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;fmincon stopped because the objective function value is less than&#xA;  the default value of the objective function limit and constraints &#xA;  are satisfied to within the default value of the constraint tolerance.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Stopping criteria details:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;fmincon stopped because the objective function value, -5.374294e+40, is less than&#xA;  options.ObjectiveLimit = -1.000000e+20, and the relative maximum constraint&#xA;  violation, 0.000000e+00, is less than options.TolCon = 1.000000e-06.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Optimization Metric                                         Options&#xA;  objective value =  -5.37e+40               ObjectiveLimit =  -1e+20 (default)&#xA;  relative max(constraint violation) =   0.00e+00    TolCon =   1e-06 (default)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here are my questions: -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) If the problem was able to find local minima for certain start points (as indicated by exitflag 1) within a particular combination of parameters, why does few other start points mention about problem being &quot;unbounded&quot; (exitflag -3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) When the exitflag is -3, the objective function value returned by fmincon does NOT match the explicit calculation of function value at the x point returned by fmincon. Matlab link on &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/current-point-and-function-value.html&quot; rel=&quot;nofollow&quot;&gt;Current Point and Function value&lt;/a&gt; mentions &quot;The function value is the value of the objective function at the current point.&quot; which does not state restriction on exitflags. On the other hand, &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/function-arguments.html&quot; rel=&quot;nofollow&quot;&gt;Output Arguments&lt;/a&gt; states this for x &quot;The solution found by the optimization function. If exitflag &gt; 0, then x is a solution; otherwise, x is the value of the optimization routine when it terminated prematurely.&quot; possibly implying that if exitflag is negative then &lt;em&gt;The function value is NOT the value of the objective function at the current point.&lt;/em&gt; I'm not sure whether it is inconsistent documentation or a mis-interpretation on my part?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, will draw attention to the statement &lt;a href=&quot;http://www.mathworks.com/help/optim/ug/exit-flags-and-exit-messages.html&quot; rel=&quot;nofollow&quot;&gt;Exit flags are not infallible guides to the quality of a solution. Many other factors, such as tolerance settings, can affect whether a solution is satisfactory to you. You are responsible for deciding whether a solver returns a satisfactory answer. Sometimes a negative exit flag does not correspond to a &quot;bad&quot; solution. Similarly, sometimes a positive exit flag does not correspond to a &quot;good&quot; solution.&lt;/a&gt;. I can understand +ve exitflags not being necessarily good in all scenarios, but why would a negative exitflag not necessarily mean a &quot;bad&quot; solution?&lt;/p&gt;&#xA;" OwnerUserId="7153" LastEditorUserId="7153" LastEditDate="2014-07-30T03:33:01.110" LastActivityDate="2014-07-30T03:33:01.110" Title="The solution found by the optimization function when exitflag is negative" Tags="|matlab|constrained-optimization|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14286" PostTypeId="1" CreationDate="2014-07-30T04:48:58.340" Score="2" ViewCount="4040" Body="&lt;p&gt;I am applying an iterative method (projected newton) to an optimization problem. Theoretically, the method should converge linearly. &lt;strong&gt;I would greatly appreciate it if you could share how should I test the convergence and which stopping criteria should I use?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the gradient norm (gradient of the objective function) as the stopping criteria but the results look weird. The norm only decreases from 1e-01 to 1e-02 and then stays at 1e-02 forever--the gradient norm does not decrease any more. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the first order necessary condition for the optimization problem says that the gradient should be zero at the solution --&gt; which means we expect to see &lt;em&gt;really small&lt;/em&gt; gradient norm, right? --&gt; &lt;strong&gt;**Is the gradient norm of order 1e-02 small enough or still big?**&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much!&lt;/p&gt;&#xA;" OwnerUserId="9842" LastEditorUserId="1804" LastEditDate="2014-07-30T08:04:47.340" LastActivityDate="2014-08-04T22:30:31.390" Title="How to test convergence of an algorithm for constrained optimization" Tags="|optimization|iterative-method|convergence|constrained-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14287" PostTypeId="2" ParentId="14286" CreationDate="2014-07-30T07:52:31.620" Score="4" Body="&lt;p&gt;The first order necessary optimality conditions for a minimization problem with inequality constraints are &lt;em&gt;not&lt;/em&gt; that the gradient vanishes, since the minimum can be attained at the boundary of the feasible set (where only the directional derivatives into the interior of the set have to be non-negative, i.e., going away from the boundary leads to an increase in function value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, for a problem of the form&#xA;$$ \min_{x\in C} f(x)$$&#xA;for a convex set $C$, the necessary optimality conditions for a minimizer $\bar x \in C$ are&#xA;$$ \langle \nabla f(\bar x), \bar x-x \rangle \leq 0\qquad \text{for all }x\in C.$$&#xA;For this condition, there is no sensible way to define a residual. But one way to reformulate this condition is using the metric projection $P_C$ onto $C$ as&#xA;$$ \bar x = P_C(\bar x - \nabla f(\bar x)),$$&#xA;so you could monitor &#xA;$$ r(x^k) := \| x^k -  P_C(x^k - \nabla f(x^k))\|,$$&#xA;which should go to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, if your minimizer $\bar x$ lies in the interior of $C$, either of these conditions can only hold if the gradient vanishes. If this is the case in your problem, your gradient norm should go to zero -- and $10^{-2}$ is definitely not zero. If your derivatives are correct, then the norm should go to zero up to machine precision (say, $10^{-16}$). Otherwise the method is limited by the accuracy of your gradient and Hessian evaluation, so I'd suspect that there's an error there (which you can check by comparing directional derivatives via inner products of the gradient with a direction and via finite difference approximations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: To test whether your gradient and Hessian are suitably accurate approximations of derivatives of your functional, you can do the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Gradient: Pick $x$ and direction $h$ (e.g., $h=x$ or $h$ random -- always test multiple vectors!) and compare for $\varepsilon \to 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) $(f(x+\varepsilon h)-f(x))/\varepsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) $\langle \nabla f(x),h\rangle$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Action of Hessian: $x,h$ as above and compare for $\varepsilon \to 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) $(f(x+\varepsilon h)-2f(x) +f(x-\varepsilon h))/\varepsilon^2$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) $\langle [\nabla^2 f(x)h],h\rangle$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-07-30T13:18:36.063" LastActivityDate="2014-07-30T13:18:36.063" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14288" PostTypeId="2" ParentId="14246" CreationDate="2014-07-30T08:14:51.037" Score="4" Body="&lt;p&gt;Previous comments gave you good suggestions, I try to add some more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, your example 2x2 really does not correspond to zero Neumann boundary conditions. In fact, one can show that your choice leads to zero Dirichlet boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you domain is a rectangle, then implementing the zero Neumann condition can be done easily. You have to include the values at boundaries to unknowns (you might done that already). For four nodes in the corners of your rectangle, you have to change in the corresponding row of matrix  two -1 to -2, for all other rows corresponding to boundary nodes change only one -1 to -2, it must be exactly the single entry that is inside of rectangle so not on boundary. As noted by someone else previously, the sum off all entries for each row in the matrix must be equal to zero, that is another check for you if you are doing it correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea behind finite difference discretization for zero Neumann boundary conditions is that you imagine that you have one row or one column of nodes next to each side of your rectangle with additional (artificial) unknowns. It means that you may use the standard stencil with 4 on the diagonal and four time -1 offdiagoals. Additional unknowns can be eliminated by adding discrete equations that correspond to zero Neumann boundary condition, e.g. $\partial_x \varphi = 0$, that takes in the discrete form (hopefully the notation is clear) e.g. $(\varphi_{1 j} - \varphi_{-1 j})/dx = 0$, so the artificial unknowns are equal $\varphi_{-1 j} = \varphi_{1 j}$. Plug this into the discrete equation for unknowns $\varphi_{0 j}$ and you get what I did suggest. By the way this corresponds to second order accurate approximation that you should prefer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another point is that applying zero Neumann boundary condition for Poisson equations gives you underdetermined system where there are formally infinitely many solutions, but they differ only by constant (it means if you subtract two solution functions you get constant function). Once you fix one value e.g. to 0 (as proposed by someone else), you choose one of those infinitely many solution, so your problem has then unique solution. It is like you define Dirichlet boundary condition (e.g. $\varphi =0$) in one single point at boundary. Of course, then such node must be excluded from your unknowns, so you do not have a row in your matrix for this node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes your linear solver can take care for this, but this is another story, the suggestion above may be much simpler.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="8174" LastEditDate="2015-12-19T21:03:49.363" LastActivityDate="2015-12-19T21:03:49.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14289" PostTypeId="2" ParentId="11810" CreationDate="2014-07-30T11:45:52.010" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So Is it neccesary to use Poisson - Boltzmann equation if I only need to build electrostatic potential from a PQR file?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. You can use Poisson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you know the positions of each point charge, you know the charge distribution $\rho$, which is a sum of delta functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can thus solve numerically the Poisson's equation that links the charge distribution to the laplacian of the potential:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\begin{equation}&#xA;\Delta V=\rho&#xA;\end{equation}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution will depend upon your boundary conditions.&#xA;Also, many algorithms are available to find this solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example may be found &lt;a href=&quot;http://arxiv.org/pdf/cond-mat/0605371&quot; rel=&quot;nofollow&quot;&gt;here on the arXiv&lt;/a&gt;. This paper is called &quot;Efficient solution of Poisson's equation with free boundary conditions&quot;.&lt;/p&gt;&#xA;" OwnerUserId="9846" LastActivityDate="2014-07-30T11:45:52.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14290" PostTypeId="2" ParentId="1445" CreationDate="2014-07-30T13:46:53.737" Score="4" Body="&lt;p&gt;I would complement this bibliography with, the &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal.ii&lt;/a&gt; library. Probably, if you are interested in functional analysis, error estimates etc., this is not the right place for you. If you want to have an essential, but rigorous, mathematical picture, plus implementation strategy and software, well, there is no better place to check than &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/Tutorial.html&quot; rel=&quot;nofollow&quot;&gt;deal.ii tutorials&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me also add that &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;Wolfgangs's video lectures&lt;/a&gt; are a precious resource.&lt;/p&gt;&#xA;" OwnerUserId="6574" LastEditorUserId="1553" LastEditDate="2014-12-02T13:56:57.573" LastActivityDate="2014-12-02T13:56:57.573" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14291" PostTypeId="1" CreationDate="2014-07-27T10:02:35.463" Score="6" ViewCount="258" Body="&lt;p&gt;Let a system of $n$ polynomial equations of degree $d$ with $m$ variables.&lt;br&gt;&#xA;I'm interested in a sparse system with $d = 3$, $n \sim 2000000$, $m \sim 50000$ and integer coefficients.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What techniques can I use from numerical analysis to show the existence of a solution? What numerical methods exist for finding a solution?   What are the complexities? In others words, do these methods compute a solution to my problem in a reasonable time if we implement it on an efficient computer?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details about my system (it's a pentagon equation), see &lt;a href=&quot;https://mathoverflow.net/questions/177123/are-there-workable-numerical-approaches-for-the-pentagon-equation&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9824" OwnerDisplayName="Sébastien Palcoux" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2014-08-01T07:35:31.667" Title="What are the numerical methods for huge polynomial systems?" Tags="|numerical-analysis|nonlinear-equations|complexity|polynomials|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14292" PostTypeId="2" ParentId="14291" CreationDate="2014-07-30T15:44:07.437" Score="5" Body="&lt;p&gt;&lt;em&gt;Certified homotopy continuation methods&lt;/em&gt; are used both for finding roots and for proving that they indeed exist (inside a certain interval).  A quick web search turned out this  paper: &lt;a href=&quot;http://www.texmacs.org/joris/homotopy/homotopy-abs.html&quot;&gt;Reliable homotopy continuation by Joris van der Hoeven&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2014-07-30T15:44:07.437" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14293" PostTypeId="1" AcceptedAnswerId="14294" CreationDate="2014-07-30T21:08:13.843" Score="0" ViewCount="73" Body="&lt;p&gt;I what to calculate displacements, stress and strain in a soil slope with a FEM script. The slope moves like a laminar flow.&#xA;Can you suggest me some bibliography on this problem?&#xA;I've already look on Zienkiewicz - The finite element method vol 1, 2 and 3-  &lt;/p&gt;&#xA;" OwnerUserId="8105" LastEditorUserId="8105" LastEditDate="2014-07-30T21:13:56.823" LastActivityDate="2014-07-31T04:21:51.697" Title="FEM with soil slope" Tags="|finite-element|fluid-dynamics|laminar-flow|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14294" PostTypeId="2" ParentId="14293" CreationDate="2014-07-31T04:05:57.703" Score="2" Body="&lt;p&gt;The only book I've found that treats a variety of FE methods for deformation in porous media (e.g. soils) is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R. W. Lewis, B. A. Schrefler (1999) &lt;a href=&quot;http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471928097.html&quot; rel=&quot;nofollow&quot;&gt;The Finite Element Method in the Static and Dynamic Deformation and Consolidation of Porous Media&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think it covers modeling soil deformation as if it were laminar fluid.  Normally, soils are modeled as a two phase material (either fully or partially saturated with fluids).  Deformation of the solid skeleton in the soil is intimately coupled with the flow of the fluids within it (whether that be air, water, or both).  It's very hard to model one without the other.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-07-31T04:21:51.697" LastActivityDate="2014-07-31T04:21:51.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14295" PostTypeId="1" AcceptedAnswerId="14301" CreationDate="2014-07-31T11:54:47.757" Score="2" ViewCount="174" Body="&lt;p&gt;I have the following SDP problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;max: $Tr(CX)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to: $X \geq 0, I - X \geq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to convert it into the standard form specified by CSDP (I'm using the callable C interface), which is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;max: $Tr(CX)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to: $\forall_i, Tr(A_iX) = a_i, X \geq 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The SDP is part of an algorithm which iteratively increases the dimensions of $X$ until the value of $\max\ Tr(CX)$ converges. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any efficient way? $X$ is already known to be sparse, if that helps.&lt;/p&gt;&#xA;" OwnerUserId="9855" LastEditorUserId="9855" LastEditDate="2014-07-31T16:52:00.480" LastActivityDate="2014-07-31T23:20:50.147" Title="How can I convert this SDP constraint?" Tags="|convex-optimization|semidefinite-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14297" PostTypeId="1" CreationDate="2014-07-31T19:26:05.767" Score="1" ViewCount="1070" Body="&lt;p&gt;Can anyone explain how the MATLAB functions: stream2(x,y,z,u,v,startx,starty) and stream3(x,y,z,u,v,startx,starty) compute stream lines from vector data u and v? In the stream2.m file, it calls the stream3c function. I assume it is a .c file but unable to see it. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="8191" LastActivityDate="2014-07-31T19:26:05.767" Title="algorithm Matlab function 'stream2' and 'stream3' use to compute streamline" Tags="|matlab|plotting|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14298" PostTypeId="1" AcceptedAnswerId="14299" CreationDate="2014-07-31T20:09:44.747" Score="1" ViewCount="169" Body="&lt;p&gt;I have two animal running trajectories. A regular one with repeated back and forth running between point A and B, like the one on top in the figure. The other one is very irregular, animal paused and turned around a lot in the middle.  Is there any algorithm to measure the regularity of a trajectory, like repeated activity on the top?  And compare the extend of regularity between the two trajectories?  Thanks in advance. &#xA;&lt;img src=&quot;https://i.stack.imgur.com/E7ANA.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="8191" LastActivityDate="2014-08-01T08:54:19.243" Title="How to measure trajectory regularity?" Tags="|algorithms|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14299" PostTypeId="2" ParentId="14298" CreationDate="2014-07-31T20:50:43.633" Score="0" Body="&lt;p&gt;One possibility is measuring the change in slope at each point in the trajectory and then taking the average at the end. In the first one which you've described as &quot;regular&quot; the slope remains constant for a good part of it. That's not the case in the second one which is &quot;irregular&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So at each point in the x-y trajectory find the change in slope and then take the average. At the end the a more &quot;regular&quot; trajectory will have an average change closer to 0 (because the animal walks in a straight line for a good part of it, so his &quot;slope&quot; is constant). Here is some example code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;change_in_slope = 0.0&#xA;for i in 2:no_points-1&#xA;    change_in_slope += (y[i] - y[i-1])/(x[i] - x[i-1]) - (y[i+1] - y[i])/(x[i+1] - x[i])&#xA;end&#xA;change_in_slope = change_in_slope / (no_points - 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9861" LastEditorUserId="9861" LastEditDate="2014-07-31T21:01:08.210" LastActivityDate="2014-07-31T21:01:08.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14300" PostTypeId="1" AcceptedAnswerId="14302" CreationDate="2014-07-31T22:06:36.787" Score="3" ViewCount="700" Body="&lt;p&gt;This is somehow unexpected, but my recent experience with solving a system of nonlinear equations is that treating them as the right hand side of a system of ordinary equations and then evolve the system with an ODE solver can be considerably faster than with the usual Newton-Raphson iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this only applies to some specific types of nonlinear equations.  The equations I am dealing with is from a chemical equilibrium problem.  The chemical evolution equation looks like&#xA;$$\frac{d}{dt}x_i = f_i(\mathbf{x}) = -k_1x_i-k_2x_ix_j -k_3x_ix_k -\cdots + k_4x_j + k_5x_kx_h + \cdots,$$&#xA;$$\cdots$$&#xA;where the $k_i$s are all constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to solve the equations&#xA;$$f_i(\mathbf{x}) = 0,\ \text{for }i=1,\ldots,n.$$&#xA;The approach I am talking about is to start from an initial guess $x_0$, and let it evolve until the $f_i$s are very close to $0$; an equilibrium solution is guaranteed to exist based on physical intuition (though not necessarily unique).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't understand is that the ODE solver also has an Newton-Raphson iteration built-in, which is needed in an implicit scheme (which is the one I am using); how could the ODE approach be faster?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The nonlinear equations have a trivial solution $\mathbf{x}=0$, but this is rarely the one we want.  My understanding is that there is no guarantee the Newton iteration will converge to a ``physical'' solution starting from an initial estimate $\mathbf{x}_0$, while the ODE approach seems to have this property.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I replaced one of the nonlinear equation with a conservation equation, and now the Newton method runs much faster (&lt;del&gt;similar to&lt;/del&gt; roughly 4 times faster than the ODE approach), and it is likely that not doing this will yield the wrong result, since the system of nonlinear equations is not completely independent (which makes the Newton solver complain).&lt;/p&gt;&#xA;" OwnerUserId="7419" LastEditorUserId="7419" LastEditDate="2014-08-01T05:46:20.923" LastActivityDate="2014-08-03T14:39:46.603" Title="Solving a system of nonlinear equations with an ODE solver is faster than with the Newton method?" Tags="|numerical-analysis|ode|nonlinear-equations|newton-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14301" PostTypeId="2" ParentId="14295" CreationDate="2014-07-31T23:20:50.147" Score="0" Body="&lt;p&gt;Suppose that $X \in \mathbb{R}^{n \times n}$, and $X$ is positive semidefinite. For convenience, define&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\langle A, B\rangle = \sum_{k, l} A_{kl}B_{kl}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for square matrices $A, B$ of equal size; this operation corresponds to the trace of the matrix product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your formulation currently looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\max \langle{C, X}\rangle \\&#xA;\mathrm{s.t.}\,\, X \succeq 0,\,\,I - X \succeq 0, &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $X \succeq 0$ means that $X$ is positive semidefinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, I'd create a positive semidefinite matrix of slack variables $S \in \mathbb{R}^{n \times n}$ such that $S + X = I$, and $S \succeq 0$. Let $E_{ij}$ is a matrix of consisting of all zeros except for the $(i,j)$th element, which is set to 1. Also, define the matrices&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;X' = \left[\begin{array}{cc}X &amp;amp; 0 \\ 0 &amp;amp; S\end{array}\right], \\&#xA;C' = \left[\begin{array}{cc}C &amp;amp; 0 \\ 0 &amp;amp; 0\end{array}\right], \\&#xA;A_{ij} = \left[\begin{array}{cc} E_{ij} &amp;amp; 0 \\ 0 &amp;amp; E_{ij}\end{array}\right].&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, (I think,) the following reformulation works:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\max \langle{C', X'}\rangle \\&#xA;\mathrm{s.t.}\,\, X' \succeq 0,\,\,\langle A_{ij}, X' \rangle = \delta_{ij},\,\,i,j=1,\ldots,n &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\delta_{ij}$ is the Kronecker delta symbol. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-07-31T23:20:50.147" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14302" PostTypeId="2" ParentId="14300" CreationDate="2014-07-31T23:51:05.410" Score="7" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This is somehow unexpected, but my recent experience with solving a system of nonlinear equations is that treating them as the right hand side of a system of ordinary equations and then evolve the system with an ODE solver can be considerably faster than with the usual Newton-Raphson iteration.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It sounds like you're doing some sort of pseudotransient continuation (PtC). (See &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14195/is-it-well-known-that-some-optimization-problems-are-equivalent-to-time-stepping/14199#14199&quot;&gt;Is it well known that some optimization problems are equivalent to time-stepping?&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What I don't understand is that the ODE solver also has an Newton-Raphson iteration built-in, which is needed in an implicit scheme (which is the one I am using); how could the ODE approach be faster?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It could be that the sequence of initial guesses the PtC method goes through requires fewer iterations than a Newton-Raphson method for solving the algebraic equations you wish to solve. It would be helpful to give an example of the iterate history you're seeing.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My understanding is that there is no guarantee the Newton iteration will converge to a ``physical'' solution starting from an initial estimate x0, while the ODE approach seems to have this property.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, there are no guarantees that the Newton iteration will converge to a &quot;physical&quot; solution. If the ODE approach converges, it should converge to a stable equilibrium point of the dynamical system, which, in your case, is the physical solution you are talking about.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I replaced one of the nonlinear equation with a conservation equation, and now the Newton method runs much faster (similar to the ODE approach), and it is likely that not doing this will yield the wrong result, since the system of nonlinear equations is not completely independent (which makes the Newton solver complain).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the Jacobian is rank-deficient, then the linear solves may or may not return solutions. For instance, a Krylov subspace solver will typically calculate a Drazin inverse, which yields a least-squares solution. Gaussian elimination may throw an error due to a zero pivot or something like that; a least-squares solution could also be calculated using QR or SVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobian matrices for time steps in the PtC method will usually be full rank; they will be of the form $(I - \Delta{t}J)$, where $J$ is the Jacobian of the left-hand side of $f(x) = 0$.  Rank-deficiency is probably another reason the convergence of the original Newton-Raphson method you used wasn't so great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conservation equation would yield a DAE, which you could also use in a PtC approach, if you like.  See &lt;a href=&quot;http://dx.doi.org/10.1137/S106482750241044X&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coffey, Kelley, and Keyes (2003)&lt;/a&gt; for analysis of PtC for DAE compared to ODE formulations.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-08-03T14:39:46.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14303" PostTypeId="2" ParentId="14298" CreationDate="2014-08-01T00:58:32.200" Score="1" Body="&lt;p&gt;There are ways to check periodicity in a signal.  If you estimate the most likely frequency and approximate the per-period behavior then if you detect that it is a nearly exact match for every period, this would be evidence of a kind of regularity.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-08-01T00:58:32.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14304" PostTypeId="2" ParentId="14291" CreationDate="2014-08-01T07:35:31.667" Score="4" Body="&lt;p&gt;For this kind of large scale problem, using method like Gröbner basis, or other generally used to solve polynomial system, require lots of calculation time and many times your &quot;solution&quot; is so big you can't do anything with it. Also to solve the system, those method required that your system is zero-dimensional which is not all the time true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably a more numerical approach, may be to see this system as an non-linear equation F(x)=0, and solve it with Gauss-Newton method.&#xA;As your system is big, instead of classical Newton method, check inexact Newton like Newton-Krylov.&#xA;With Newton method with globalization, you have super-linear convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance : &#xA;Some explanation in : &lt;a href=&quot;https://www10.informatik.uni-erlangen.de/de/Teaching/Courses/WS2012/SiWiR/seminar/ParPDE1.pdf&quot; rel=&quot;nofollow&quot;&gt;https://www10.informatik.uni-erlangen.de/de/Teaching/Courses/WS2012/SiWiR/seminar/ParPDE1.pdf&lt;/a&gt;&#xA;and this : &lt;a href=&quot;http://www.math.uwaterloo.ca/~tfcolema/articles/K_final_draft.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.uwaterloo.ca/~tfcolema/articles/K_final_draft.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9749" LastActivityDate="2014-08-01T07:35:31.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14306" PostTypeId="1" CreationDate="2014-08-01T10:29:10.850" Score="0" ViewCount="2247" Body="&lt;p&gt;I am dealing with a highly nonlinear system of two PDEs. I already have a code to solve the system in case of Dirichlet boundary conditions. The explicit system is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{eqnarray*}&#xA;\partial_{t}u &amp;amp; = &amp;amp; D_{11}\partial_{x}^{2}u+\partial_{x}D_{11}\cdot\partial_{x}u+D_{12}\partial_{x}^{2}v+\partial_{x}D_{12}\cdot\partial_{x}v\\&#xA;\partial_{t}v &amp;amp; = &amp;amp; D_{21}\partial_{x}^{2}u+\partial_{x}D_{21}\cdot\partial_{x}u+D_{22}\partial_{x}^{2}v+\partial_{x}D_{22}\cdot\partial_{x}v&#xA;\end{eqnarray*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(0,t)=0 \ (\partial_x v)(0,t) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the discretized form of the scheme are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{eqnarray*}&#xA;-D_{21}(x_{j})u_{n+1,j-1}-D_{22}(x_{j})v_{n+1,j-1}\\&#xA;{}[\mu+2D_{11}(x_{j})]u_{n+1,j}+2D_{12}(x_{j})v_{n+1,j}\\&#xA;-D_{11}(x_{j})u_{n+1,j+1}-D_{12}(x_{j})v_{n+1,j+1} &amp;amp; = &amp;amp; \frac{1}{4}[4\mu-D_{11}(x_{j+1})+D_{11}(x_{j-1})]u_{n,j}\\&#xA; &amp;amp;  &amp;amp; -[D_{12}(x_{j+1})-D_{12}(x_{j})]v_{n,j}\\&#xA; &amp;amp;  &amp;amp; +\frac{1}{4}[D_{11}(x_{j+1})-D_{11}(x_{j-1})]u_{n,j+1}\\&#xA; &amp;amp;  &amp;amp; +\frac{1}{4}[D_{12}(x_{j+1})-D_{12}(x_{j-1})]v_{n,j+1}&#xA;\end{eqnarray*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{eqnarray*}&#xA;-D_{21}(x_{j})u_{n+1,j-1}-D_{22}(x_{j})v_{n+1,j-1}\\&#xA;2D_{21}(x_{j})u_{n+1,j}+[\mu+2D_{22}(x_{j})]v_{n+1,j}\\&#xA;-D_{21}(x_{j})u_{n+1,j+1}-D_{22}(x_{j})v_{n+1,j+1} &amp;amp; = &amp;amp; \frac{1}{4}[4\mu-D_{21}(x_{j+1})+D_{21}(x_{j-1})]u_{n,j}\\&#xA; &amp;amp;  &amp;amp; -[D_{22}(x_{j+1})-D_{22}(x_{j})]v_{n,j}\\&#xA; &amp;amp;  &amp;amp; +\frac{1}{4}[D_{21}(x_{j+1})-D_{21}(x_{j-1})]u_{n,j+1}\\&#xA; &amp;amp;  &amp;amp; +\frac{1}{4}[D_{22}(x_{j+1})-D_{22}(x_{j-1})]v_{n,j+1}&#xA;\end{eqnarray*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, I am solving a linear system of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A\boldsymbol{x}=\boldsymbol{b}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ has $2J\times 2J$ entries, according to the structure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(\begin{array}{cccccccccc}&#xA;\mbox{1st equation}\rightarrow &amp;amp; | &amp;amp; c_{u}^{j} &amp;amp; c_{v}^{j} &amp;amp; c_{u}^{j+1} &amp;amp; c_{v}^{j+1} &amp;amp; 0 &amp;amp;  &amp;amp; ... &amp;amp; 0\\&#xA;\mbox{2nd equation}\rightarrow &amp;amp; | &amp;amp; d_{u}^{j} &amp;amp; d_{v}^{j} &amp;amp; d_{u}^{j+1} &amp;amp; d_{v}^{j+1} &amp;amp; 0 &amp;amp;  &amp;amp; ... &amp;amp; 0\\&#xA;\vdots &amp;amp; | &amp;amp; . &amp;amp; . &amp;amp; . &amp;amp; . &amp;amp; . &amp;amp; . &amp;amp; . &amp;amp; .\\&#xA;\mbox{1st}\rightarrow &amp;amp; | &amp;amp; 0 &amp;amp; 0 &amp;amp; c_{u}^{j-1} &amp;amp; c_{v}^{j-1} &amp;amp; c_{u}^{j} &amp;amp; c_{v}^{j} &amp;amp; c_{u}^{j+1} &amp;amp; c_{v}^{j+1}\\&#xA;\mbox{2nd }\rightarrow &amp;amp; | &amp;amp; 0 &amp;amp; 0 &amp;amp; d_{u}^{j-1} &amp;amp; d_{v}^{j-1} &amp;amp; d_{u}^{j} &amp;amp; d_{v}^{j} &amp;amp; d_{u}^{j+1} &amp;amp; d_{v}^{j+1}\vdots&#xA;\end{array}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $c$'s and $d$'s are the the coefficients corresponding to the scheme above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I am trying to implement the Neumann boundary condition (the one on $v$), using the ghost point method as discussed, for instance, &lt;a href=&quot;http://www.math.montana.edu/~vogel/Courses/M581_2006/BC.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for a simpler equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, I would like to change my code in order to add some extra terms to the array, but I can't work out what to put in them. Also, in the respective components of the matrix $A$.&#xA;Because when I write the condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u_1-u_{-1}}{(\Delta x)^2}=0$$&#xA;(and same for $v$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to eliminate $u_{-1}$ and $v_{-1}$, using also the third and fourth equations I wrote in this post. If I inverted for $u_{-1}$ and $v_{-1}$ (assuming I know how to), I do not see how I could write down a code which works for any kind of scheme I have, without putting the explicit inversion in the matrix. &lt;/p&gt;&#xA;" OwnerUserId="9865" LastEditorUserId="9865" LastEditDate="2014-08-04T10:49:49.297" LastActivityDate="2014-08-04T10:49:49.297" Title="Finite differences and Neumann boundary conditions" Tags="|pde|matlab|boundary-conditions|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14307" PostTypeId="2" ParentId="14262" CreationDate="2014-08-01T13:51:02.823" Score="2" Body="&lt;p&gt;I don't believe there is a compiler flag in Fortran for this, but there are Fortran libraries for sparse matrices that may give you some optimizations. This &lt;a href=&quot;https://stackoverflow.com/questions/12328482/fortran-90-95-library-for-sparse-matrices&quot;&gt;StackOverflow post&lt;/a&gt; discusses a BLAS library for use with them, although it seems like the user had some difficulty.&lt;/p&gt;&#xA;" OwnerUserId="9861" LastActivityDate="2014-08-01T13:51:02.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14309" PostTypeId="1" CreationDate="2014-08-02T00:38:12.817" Score="6" ViewCount="663" Body="&lt;p&gt;I have an upper trapezoidal matrix stored in column major format.  That is, my matrix looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/JiYn2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to RQ decompose it, and store Q in the rectangular part of my upper trapezoidal matrix.  For QR decomposition, I can store Q in the lower portion of the matrix, as a sequence of column vectors ($v$) which form Householder transformations ($I - 2 v v^T$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the same approach taken here would have me store the vectors for the householder transformations as &lt;em&gt;rows&lt;/em&gt; in the rectangular portion of the upper trapezoidal matrix.  However, because the matrix isn't stored row major, that's not going to be cache efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to store Q from the RQ decomposition in the rectangular portion of the upper trapezoidal matrix in a way that lets me use it in a column major way?  If not using Householder Transformations, maybe a different method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now my thought is to require the column vectors in the rectangular portion to be padded with extra space so that the rectangular portion is square, and then I store the householder vectors as columns.  But that can potentially be a lot of extra memory.  Is there a better way?&lt;/p&gt;&#xA;" OwnerUserId="1365" LastActivityDate="2014-08-20T21:19:11.870" Title="Efficient RQ decomposition" Tags="|linear-algebra|dense-matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14310" PostTypeId="1" AcceptedAnswerId="14311" CreationDate="2014-08-02T04:51:00.440" Score="3" ViewCount="321" Body="&lt;p&gt;This is a somewhat basic question, I guess. Take the ODE boundary value problem&#xA;$$ \frac1\lambda y''-y'=0, \qquad y(0) = 0, \quad y(1) = 1, $$&#xA;with the solution&#xA;$$ y(x) = \frac{e^{x\lambda}-1}{e^\lambda -1}. $$&#xA;Discretize it on a uniform grid $x_k = kh$, $h=1/n$, $k=0,\ldots,n$, as follows:&#xA;$$ \frac1\lambda \frac{y_{k-1}-2y_k+y_{k+1}}{h^2}-\frac{y_{k+1}-y_{k-1}}{2h} = 0, $$&#xA;and solve the resulting tridiagonal system, for, e.g., $n=100, \lambda=10^4$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ATh5o.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With increasing $n$, the scheme nevertheless converges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;a.&lt;/strong&gt; Can you suggest a good reference that describes this kind of thing? I think that stiffness of initial-value problems is not quite the same thing, which is why I'm asking this question, but maybe I'm wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;b.&lt;/strong&gt; Short of choosing a better method, is there a &quot;filter&quot; I can apply to the output $\mathbf{y}_{0:n}$ to recover a better approximation of the true solution (in particular, make the transformed output positive, as it should be). &lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-08-02T06:18:40.950" Title="ODE boundary value problem relaxation method reference request" Tags="|finite-difference|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14311" PostTypeId="2" ParentId="14310" CreationDate="2014-08-02T06:18:40.950" Score="6" Body="&lt;p&gt;This is a convection dominated problem. Since $1/\lambda$ is small, your equation is approximately $y'=0$. But the solution of this cannot satisfy your boundary conditions. The solution will rapidly change from 0 to 1 as you approach $x=1$, and a boundary layer is created at $x=1$. The term $y''/\lambda$ cannot be neglected in the boundary layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;y_k = \frac{1}{2}(1 + \lambda h/2) y_{k-1} + \frac{1}{2}(1 - \lambda h/2) y_{k+1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the solution will be monotone if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\lambda h &amp;lt; 2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is called the cell Peclet condition. So your mesh must be finer than&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;h &amp;lt; \frac{2}{\lambda}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that increasing $\lambda$ requires very fine mesh. If your mesh satisfies above condition you should get a good solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other alternative is to use an upwind scheme. In your case it is (please check stability as above)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{\lambda} \frac{y_{k-1} - 2 y_k + y_{k+1}}{h^2} - \frac{y_k - y_{k-1}}{h} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but this is only first order accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to analyze this is in terms of an M-matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more, you can refer to some texts, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Strikwerda: Finite Difference Schemes and Partial Differential Equations&lt;/li&gt;&#xA;&lt;li&gt;Wesseling: Principles of Computational fluid dynamics&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2014-08-02T06:18:40.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14315" PostTypeId="1" CreationDate="2014-08-04T15:43:22.243" Score="1" ViewCount="309" Body="&lt;p&gt;I'm reading up on the Parameter ESTimation algorithm, (Link to PEST homepage &lt;a href=&quot;http://www.pesthomepage.org/&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt;), and having trouble understanding the broad strokes, as I keep getting lost in null spaces, different regularized ion techniques, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone who is familiar with this algorithm, often used in the calibration of environment models, provide a broad outline of its steps and the purpose of each step?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is based on the Gauss-Marquadt-Levenberg algorithm and uses many different types of &quot;regularization&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I understand the part of the algorithm relate to the method of Steepest Descent, but not the Gauss-Newton algorithm part or how it combines these two methods into one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Much thanks.&lt;/p&gt;&#xA;" OwnerUserId="9886" LastEditorUserId="9886" LastEditDate="2014-08-04T20:20:00.973" LastActivityDate="2014-08-04T20:20:00.973" Title="Parameter ESTimation (PEST) algorithm? Steps and purposes?" Tags="|algorithms|" AnswerCount="0" CommentCount="4" ClosedDate="2014-08-18T04:29:08.957" ContentLicense="CC BY-SA 3.0" />
  <row Id="14317" PostTypeId="1" CreationDate="2014-08-05T05:18:54.497" Score="1" ViewCount="116" Body="&lt;p&gt;I am writing the pre processing program for a porosity based cfd project we just started. Basically I have a 3 dimensional mesh made of cubes and I need to import a &lt;a href=&quot;http://en.wikipedia.org/wiki/STL_(file_format)&quot; rel=&quot;nofollow noreferrer&quot;&gt;STL file&lt;/a&gt; over it, and calculate how much space of each cell is being used by the object. The STL file is a CAD format that uses triangles to represent a surface, but I will assume that it represents a closed surface (a solid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made a drawing of the 2D problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FyrY7.png&quot; alt=&quot;2D&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the STL object is over the mesh. Each cell of the mesh receives a value ranging from 0 to 1 based on how much of the object is over it. Cells 1,2,3,4,5 have a value of 0 while cell 6, have a value of ~.15 and, cell 7 have a value of 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, where do I start? Is there any &quot;surface casting&quot; algorithm? How do I deal with the triangle-cubes conversion?&lt;/p&gt;&#xA;" OwnerUserId="9808" LastActivityDate="2014-08-05T05:18:54.497" Title="Algorithm for merging mesh with cad file" Tags="|computational-geometry|mesh-generation|mesh|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14318" PostTypeId="2" ParentId="14211" CreationDate="2014-08-05T09:11:37.277" Score="0" Body="&lt;p&gt;I would definitely recommend against LUA.  Despite what you may have heard from the gaming community, it's not necessarily a fast or well designed language for scientific computing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can manage it, CUDA is possibly the best choice, because it has the advantage of being GPU accelerated, dramatically increasing your compute power for an application as complex as a climate simulation.&lt;/p&gt;&#xA;" OwnerUserId="9893" LastActivityDate="2014-08-05T09:11:37.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14319" PostTypeId="1" CreationDate="2014-08-05T09:46:42.883" Score="3" ViewCount="346" Body="&lt;p&gt;I noticed this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5202/efficient-assembly-of-finite-element-matrix-in-matlab&quot;&gt;post&lt;/a&gt;, where &lt;code&gt;spalloc&lt;/code&gt; and &lt;code&gt;sparse&lt;/code&gt; are recommended for efficient assembly in Matlab. I personally use sparse assembling for simple cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when it comes to the case of coupled PDE, say, 3-PDE coupled, then the scalar unknown becomes a &lt;span class=&quot;math-container&quot;&gt;$3\times 3$&lt;/span&gt; tensor. In this case, I can't figure out a way to exploit &lt;code&gt;sparse()&lt;/code&gt;, and have to use &lt;code&gt;for&lt;/code&gt;-loops to assemble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the final assembled sparse matrix is as large as &lt;span class=&quot;math-container&quot;&gt;$30\text{k}\times 30\text{k}$&lt;/span&gt;(on PC), the assembling process becomes really slow (~10min), while the final matrix solving step is still fast(~less than 3 seconds).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any suggestion? Any generic solution is appreciated (not necessarily Matlab)&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="20688" LastEditDate="2019-06-22T20:39:24.813" LastActivityDate="2019-06-22T20:39:24.813" Title="Efficient assembly of finite element matrix(coupled equations case)" Tags="|finite-element|matlab|finite-difference|sparse-matrix|performance|" AnswerCount="0" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14321" PostTypeId="2" ParentId="13051" CreationDate="2014-08-06T19:22:26.993" Score="2" Body="&lt;p&gt;My answer below assumes your integrand is nonsingular over the domain of integration. If you have singularities, you will need more complicated machinery.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A high-level &quot;straightforward&quot; solution I can think of for &quot;simple&quot; domains is to come up with a coordinate transformation (really, a diffeomorphism) to map your domain $D \subset \mathbb{R}^{3}$ (which, from what you've described, is a manifold) to another domain $D' \subset \mathbb{R}^{2}$). Assuming you can translate your description of $D'$ to something like a collection of cubic splines, you should be able to mesh it with triangles or quads using open source meshing software (Gmsh and Triangle come to mind). Good meshing software for finite elements will discretize $D'$ such that the mesh is relatively uniform, and no particular element is skewed, so as long as your integrand is nonsingular over your domain, this approach will probably work. Then you can integrate the surface of $D'$ using quadrature, and use your coordinate transformation to map the integration over $D'$ back to integration over $D$. Obviously, the implementation is tricky, and requires some pencil and paper analytical calculations, but the quadrature and meshing parts should be within the ability of anyone with a little experience in finite element methods&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach that would work would be to mesh the surface directly, again, using finite element method meshing software. Something I searched for that might work for you would be something like CGAL, although it looks a little involved. You'll need a fine enough mesh that it approximates the surface well. The integration should again be doable for anyone with a finite element background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases above, improving the order of convergence with respect to the fineness of the discretization (really, the diameter of the elements) means using higher-order mesh elements to approximate your domain, and higher-order quadrature rules.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-06T19:22:26.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14322" PostTypeId="2" ParentId="14192" CreationDate="2014-08-06T19:36:42.750" Score="0" Body="&lt;p&gt;1) Unless the function you provided in your question changes, the period of the function should be constant. In that case, I suggest using the ability of &lt;code&gt;ode45&lt;/code&gt; to locate &quot;events&quot; -- such as zero crossings (see the documentation for &lt;code&gt;odeset&lt;/code&gt;). That way you can simulate for a shorter stretch of time, and use the accurate estimations of zero-crossings to measure the period. That should reduce your computational time by reducing the number of solver steps required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) You can certainly use a parfor loop to explore parameters:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vfDelta = linspace(0, 1, 100);&#xA;vfEpsilon = linspace(0, 1, 100);&#xA;[mfD, mfE] = ndgrid(vfDelta, vfEpsilon);&#xA;mfParams = [mfD(:) mfE(:)];&#xA;&#xA;parfor nParam = 1:size(mfParams, 1)&#xA;   % - Set up an anonymous function to pass to ode45, that contains your parameters&#xA;   fhSystem = @(t,x)trial2_params(t, x, mfParams(nParam, 1), mfParams(nParam, 2));&#xA;&#xA;   % - Solve the ODE&#xA;   [t,x]=ode45(trial2, [0 1000], [0;1]);&#xA;&#xA;   % - Do your analysis here...&#xA;   vfPeriod(nParam) = ...&#xA;end&#xA;&#xA;function trial2_params(t, x, fDelta, fEpsilon)&#xA;   ...&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;3) A faster CPU will always help...&lt;/p&gt;&#xA;" OwnerUserId="9810" LastActivityDate="2014-08-06T19:36:42.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14323" PostTypeId="2" ParentId="13051" CreationDate="2014-08-07T01:54:06.540" Score="3" Body="&lt;p&gt;&lt;strong&gt;Step 1&lt;/strong&gt;: Map ellipsoid to unit sphere. For a point $r$ on the ellipsoid, you can compute the map $r'=Mr$ where $\left|r'\right|=1$. This should be a linear map (assuming the ellipsoid was centered at the origin). $M$ is a $3\times 3$ matrix that is a composition of scaling and rotation. Note that the cut plane also transforms into another plane. We can choose the mapping to make the plane parallel to the $xy$ plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose your ellipsoid is defined by $r^T A r=1$, where $r\in\mathbb{R}^3$, and your cut plane is $n^T r = 1$, then the mapping $r' = A^{1/2}r$ is sufficient to get you to a sphere, and $r'=QA^{1/2}r$ will also map the cut plane appropriately. The pure rotation matrix $Q$ can be computed as follows. Compute $z'=\mathrm{normalize}\left[(A^T)^{-1/2}n\right]$. Apply a basis construction algorithm (Gram-Schmidt or QR factorization) to compute an orthonormal triad $x',y',z'$. The matrix $Q$ has $z'$ as the first row, and the other two rows are $x'$ and $y'$ in the order to make the determinant $+1$ instead of $-1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: if you've never seen this notation for an ellipsoid, then notice that for an axis aligned ellipsoide with semi-major axes $a,b,c$, the matrix $a$ is simply diagonal with diagonal elements $a^{-2},b^{-2},c^{-2}$. It should generally be symmetric positive definite. Also note that $n$ is not necessarily a &lt;em&gt;unit&lt;/em&gt; vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2&lt;/strong&gt;: Okay, at this point you have your mapping $M = QA^{1/2}$, and $M^{-T} n$ should be parallel to the $+z$ axis, with a length that is less than 1 if your plane actually intersects the ellipsoid. Some basic analytical geometry well tell you the equation of the circle of intersection between the transformed plane and sphere, and you can apply whatever numerical integration technique you want to this spherical cap, which is a much nicer domain. The only problem that remains is the Jacobian of the transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobian of the back transformation is $\det M^{-1}$, but we need it restricted to the tangent space. Fortunately, we know the tangent space; it is the orthogonal complement of the normal vector at any point on the ellipsoid, so for any point $r'$ on the unit sphere, the original point was $r = M^{-1}r'$ and the normal vector there is in the direction $Ar$. Using that vector and doing a full QR decomposition (or Gram-Schmidt) to make an orthonormal triad gives you a pair of orthonormal basis vectors that span the tangent space. Let $U\in\mathbb{R}^{3\times 2}$ be the matrix with those two vectors as columns. The Jacobian is then the projection of $M$ into this space: $J=\det(U^TM^{-1}U)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 3&lt;/strong&gt;: Do the integration. Let your function on the ellipsoid be $f(r)$. Apply a quadrature routine to the spherical cap you obtained in step 2 to evaluate:&#xA;$$ \int_\textrm{your stupid domain} f(r) dr = \int_\textrm{spherical cap} f(M^{-1}r') J \,dr' $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for how to integrate over the spherical cap, you can do another projection into the plane to integrate in a circular disc, where you'll have to work out the Jacobian of the projection (it's pretty nasty if you're near the equator). Alternatively, there are good semi-analytic triangulations you can apply now.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2014-08-07T01:54:06.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14325" PostTypeId="1" AcceptedAnswerId="14344" CreationDate="2014-08-07T13:33:00.063" Score="2" ViewCount="2246" Body="&lt;p&gt;Let $F_1$, $F_2$ be the foci points of an ellipse $\mathcal{E}\colon \mathbf{x}^TA\mathbf{x}=1$, $\mathbf{x}\in\mathbb{R}^2$, $A\in\mathbb{S}_{++}^{2}$. Let also $a$, $b$ be the semi-axes of $\mathcal{E}$. We would like to find the (symmetric positive-definite) matrix $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first constructed the diagonal matrix $D=\operatorname{diag}\{a^{-2},b^{-2}\}$. Then I cunstructed the orthogonal matrix $U=[\mathbf{u}_1\:\:\mathbf{u}_2]$, where $\mathbf{u}_1=F_1F_2$ and $\mathbf{u}_2\perp\mathbf{u}_1$, specificaly $\mathbf{u}_2 = (-u_{12},u_{11})^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally I got $A=UDU^T$. But when I use Matlab's &lt;strong&gt;eig() function&lt;/strong&gt; to get again the eigenvalues and the eigenvectors, while I get correctly the eigenvalues, I get the opposite of the eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the error here? Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="9905" LastActivityDate="2014-08-09T21:26:25.867" Title="Constructing matrix from eigenvalues, eigenvectors (Inconsistency with Matlab's eig())" Tags="|matlab|matrix|eigenvalues|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14326" PostTypeId="2" ParentId="5110" CreationDate="2014-08-07T14:16:11.557" Score="1" Body="&lt;p&gt;I too came from using neural netowrks in Matlab  to Python. One of the most powerful libraries in Python is &quot;Pylearn2&quot; &lt;a href=&quot;http://deeplearning.net/software/pylearn2/&quot; rel=&quot;nofollow&quot;&gt;http://deeplearning.net/software/pylearn2/&lt;/a&gt;. Currently, this is the most active library and has many different features to experiment with. It is based on Theano and as such is fast and can be made run on GPU's. Unfortunately, this is its disadvantage too: the API is constantly changing, and has a high learning curve. You have to configure your neural netowrks using YAML files too. &#xA;I have had more success using PyBrain for creating basic neural networks. I needed a solution to a regression problem, where I had to forecast the load on a power station based on weather factors. The guide here: &lt;a href=&quot;http://fastml.com/pybrain-a-simple-neural-networks-library-in-python/&quot; rel=&quot;nofollow&quot;&gt;http://fastml.com/pybrain-a-simple-neural-networks-library-in-python/&lt;/a&gt;&#xA;gave me 90% of the solution that i needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One issue I found with PyBrain was speed. It is written natively in Python. I have found the training of a neural network to be ~50x slower than Matlab. Some others have found success with speeding up the training process of PyBrain with the arac library. &lt;/p&gt;&#xA;" OwnerUserId="9908" LastEditorUserId="9908" LastEditDate="2014-08-08T09:26:31.300" LastActivityDate="2014-08-08T09:26:31.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14327" PostTypeId="1" AcceptedAnswerId="14329" CreationDate="2014-08-07T18:14:18.933" Score="2" ViewCount="915" Body="&lt;p&gt;If I have N particles how do I assign their x values so that the end result is Gaussian distribution. i.e. particles near the ends are more spread out than particles near the center.&lt;/p&gt;&#xA;" OwnerUserId="3561" LastActivityDate="2014-08-08T16:55:46.720" Title="Python: What is a good way to generate a 1D particle field with a gaussian distribution?" Tags="|python|statistics|probability|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14328" PostTypeId="2" ParentId="14327" CreationDate="2014-08-07T18:29:45.793" Score="2" Body="&lt;p&gt;NumPy comes with a nifty random library with various distributions, including &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.normal.html#numpy.random.normal&quot; rel=&quot;nofollow&quot;&gt;normal (Gaussian)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the Numpy documentation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mu, sigma = 0, 0.1 # mean and standard deviation&#xA;s = np.random.normal(mu, sigma, 1000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which will give you 1000 normally distributed values with mean mu and standard deviation sigma.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-08-07T18:29:45.793" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14329" PostTypeId="2" ParentId="14327" CreationDate="2014-08-07T23:03:39.657" Score="6" Body="&lt;p&gt;This doesn't randomly sample points, but instead chooses representative points deterministically.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scipy.stats.norm.ppf(np.linspace(0, 1, 1000+2)[1:-1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="3561" LastEditDate="2014-08-08T16:55:46.720" LastActivityDate="2014-08-08T16:55:46.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14331" PostTypeId="1" AcceptedAnswerId="14333" CreationDate="2014-08-07T23:55:10.013" Score="4" ViewCount="140" Body="&lt;p&gt;I am trying to derive Galerkin type weak formulation for the Stokes equations. I'm having a bit of a problem reconciling the notation in the integration by parts. I know that the answer I'm looking for is:&#xA;$&#xA;\int_\Omega \Delta\mathbf{u}\cdot\mathbf{v}d\Omega = \int_\Gamma (\mathbf{n}\cdot\nabla\mathbf{u})\cdot\mathbf{v}d\Gamma - \int_\Omega \nabla\mathbf{u}:\nabla\mathbf{v}d\Omega&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I integrate by parts myself I get:&#xA;$&#xA;\int_\Omega \nabla u\cdot\mathbf{v}d\Omega = \int_\Gamma u(\mathbf{v}\cdot\mathbf{n})d\Gamma - \int_\Omega u \nabla\cdot\mathbf{v}d\Omega\\\ \quad\quad\quad \Rightarrow \int_\Omega\Delta\mathbf{u}\cdot\mathbf{v}d\Omega = \int_\Omega (\nabla\cdot (\nabla\mathbf{u}))\cdot\mathbf{v}d\Omega = \int_\Gamma \nabla\mathbf{u} (\mathbf{v}\cdot\mathbf{n})d\Gamma - \int_\Omega\nabla\mathbf{u}\nabla\cdot\mathbf{v}d\Omega&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume I should be using a dot product for the vector/matrix multiplication, but even so I can't reconcile my answer with what I know the correct answer to be. For instance the line integral should be a scalar, but with my answer $\nabla\mathbf{u}$  is a matrix and $\mathbf{v}\cdot\mathbf{n}$ is a scalar so I fail to see how their product could be a scalar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did notice that the formula I used applies for scalar $u$'s. Is there another identity I should be using when $u$ is a vector?&lt;/p&gt;&#xA;" OwnerUserId="9728" LastEditorUserId="9728" LastEditDate="2014-08-08T01:38:13.730" LastActivityDate="2014-08-08T05:05:38.703" Title="Reconciling vector and scalar notation" Tags="|finite-element|vector|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14332" PostTypeId="1" CreationDate="2014-08-08T04:27:12.960" Score="3" ViewCount="93" Body="&lt;p&gt;After studying the process of &lt;a href=&quot;http://books.google.com.au/books?id=EV_Ojz00A9IC&amp;amp;pg=PA44&amp;amp;lpg=PA44&amp;amp;dq=why%20is%20there%20no%20gradual%20overflow&amp;amp;source=bl&amp;amp;ots=APOQj-ThVm&amp;amp;sig=ohKLKH89V7Iq-8sXmEVzt-n93nk&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=bUnkU4jjG9iE8gXn74DgBg&amp;amp;ved=0CBsQ6AEwADgK#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot; title=&quot;Gradual Underflow&quot;&gt;Gradual Underflow&lt;/a&gt;, I'm left a little curious as to why machines don't implement &lt;em&gt;Gradual Overflow&lt;/em&gt;; where numbers exceeding the overflow level would be stored as un-normalized floats.&lt;br&gt;&#xA;Just as the precision of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Significand&quot; rel=&quot;nofollow&quot; title=&quot;mantissa&quot;&gt;mantissa&lt;/a&gt; is decreased for representation of a larger (in magnitude) exponent (and thus a smaller number in scientific notation) for &lt;a href=&quot;http://en.wikipedia.org/wiki/Denormal_number&quot; rel=&quot;nofollow&quot; title=&quot;subnormal numbers&quot;&gt;subnormal numbers&lt;/a&gt;, so too could precision be traded for numbers above the overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only conceivable reason I can place for this is based on number format preservation during gradual underflow; the mantissa digits are 'right-shifted' (preceding zeros are adding after the decimal point) so although the representation is no longer normalized, it still consists of a single digit before the decimal point and the same storage spaces of mantissa and exponent.&lt;br&gt;&#xA;The best conception of a Gradual Overflow implementation I can muster would involve either expanding the exponent storage space (at the cost of mantissa storage space and thus precision), or 'right-shifting' the decimal point down the mantissa (again, a loss of mantissa precision). This would seemingly break the mantissa format of a single digit before the decimal point (or otherwise change the mantissa / exponential allocated storage spaces).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose I can understand that in a binary representation (with no delimiters between representation components (ie sign, mantissa, etc); distinction only made via the constant lengths of the components), breaking the storage pattern as Gradual Overflow would require is unsupportable. This assumes that there isn't a better implementation than the one I personally conceived that can preserve the storage pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this line of thinking correct, or is there a more pertinent reason as for the lack of Gradual Overflow?&lt;br&gt;&#xA;Thanks!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;(I can't really tag this appropriately, since there is a rather small pool of existing tags and I don't have the reputation to create any)&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9914" LastActivityDate="2014-08-08T04:27:12.960" Title="What disables Gradual Overflow?" Tags="|floating-point|data-storage|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14333" PostTypeId="2" ParentId="14331" CreationDate="2014-08-08T05:05:38.703" Score="7" Body="&lt;p&gt;The formula still applies for vectors - I find it helpful to use &lt;a href=&quot;http://www.maths.qmul.ac.uk/%7Eagp/calc3/notes5.pdf&quot; rel=&quot;noreferrer&quot;&gt;index notation&lt;/a&gt;. If&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(\nabla u) = u_{,i}$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; implicitly refers to the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th entry of the gradient (i.e. &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial u}{\partial x_i}$&lt;/span&gt;), and we assume Einstein notation (i.e. repeated indices sum over that index), then we can rewrite&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla \cdot \nabla {\bf u} = (u_j)_{,i,i} = \sum_i \frac{\partial}{\partial x_i}\frac{\partial}{\partial x_i}u_j$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For scalar integration by parts, we have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\int_K\nabla \cdot \nabla u v = \int_Ku_{,i,i} v = \int_{\partial K}u_{,i}n_i v - \int_K u_{,i}v_{,i} = \int_{\partial K} \nabla u\cdot n v - \int_K \nabla u \nabla v$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;An advantage of using index notation is that integration by parts of gradients of vectors and tensors is more straightforward, since it breaks everything down into the scalar case. Using this for a vector means you throw in an extra index for the vector:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\int_K \nabla \cdot \nabla {\bf u v} = \int (u_j)_{,i,i} v_j = \int_{\partial K} (u_j)_{,i}{n_i} v_j - \int_K (u_j)_{,i} (v_j)_{,i} = \int_{\partial K} (\nabla {\bf u}\cdot n) \cdot {\bf v} - \int_K \nabla {\bf u} : \nabla {\bf v}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the last part is from the definition of tensor contraction.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-08-08T05:05:38.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14334" PostTypeId="1" AcceptedAnswerId="14335" CreationDate="2014-08-08T12:09:17.007" Score="1" ViewCount="1152" Body="&lt;p&gt;How can I automatically set the limits of my axis to be e.g. 5% bigger than all data contained in the plot? Of course I could calculate the total max/min values for all the stuff I plot but I would have to do this for each plot separately. Since Matlab somehow must know all min/max values (it computes a decent limit itself) I guess it must be easier to do this.&lt;/p&gt;&#xA;" OwnerUserId="7040" LastActivityDate="2014-08-08T16:55:56.473" Title="Set limits in Matlab plot automatically" Tags="|matlab|" AnswerCount="1" CommentCount="2" ClosedDate="2014-08-21T11:17:26.813" ContentLicense="CC BY-SA 3.0" />
  <row Id="14335" PostTypeId="2" ParentId="14334" CreationDate="2014-08-08T13:22:29.490" Score="4" Body="&lt;p&gt;Something like the following should do what you want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p=0.05;&#xA;axis tight&#xA;a=axis();&#xA;new_limits=[a(1)-p*(a(2)-a(1)) a(2)+p*(a(2)-a(1)) a(3)-p*(a(4)-a(3)) a(4)+p*(a(4)-a(3))];&#xA;axis(new_limits);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Further refinement would be to make this into a function that accepts any figure handle and allows a different expansion in $x$ and $y$, but those should be straightforward.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="9918" LastEditDate="2014-08-08T16:55:56.473" LastActivityDate="2014-08-08T16:55:56.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14336" PostTypeId="1" CreationDate="2014-08-08T15:21:38.040" Score="4" ViewCount="210" Body="&lt;p&gt;I need a numerically stable way to compute the following ratio:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the parameters are real numbers, with $a&amp;lt; 0$,$\ $ $b,c &amp;gt; 0$ and $0&amp;lt;x&amp;lt;1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I am using GSL's implementation of the hypergeometric function, but I keep getting under/over-flows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simplification that I can use? Or an aproximation scheme that remains accurate for a wide range of the parameters (say $-1000 &amp;lt; a,b,c &amp;lt; 1000$)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a particular case, I am also interested in positive values of the parameters, in the range $0&amp;lt;a,b,c&amp;lt;1000$. What approximation can I use in that case?&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2014-08-18T19:29:53.807" LastActivityDate="2014-08-18T19:29:53.807" Title="Compute hypergeometric function ratio: $\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$?" Tags="|floating-point|precision|special-functions|computer-arithmetic|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14337" PostTypeId="1" AcceptedAnswerId="14338" CreationDate="2014-08-08T15:29:27.883" Score="1" ViewCount="2009" Body="&lt;p&gt;In Fortran, given arrays A(1000000) and B(1000000), and to compute the sum you simply write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C = A + B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when I want to implement OpenMP, I have to write an explicit loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!$OMP PARALLEL DO&#xA;DO I = 1, 1000000&#xA;C(I) = A(I) + B(I)&#xA;END DO&#xA;!$OMP END PARALLEL DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to do this? I know there is &lt;code&gt;-parallel&lt;/code&gt; compiler flag but it gives less performance.&lt;/p&gt;&#xA;" OwnerUserId="7871" LastEditorUserId="150" LastEditDate="2018-02-08T13:51:22.957" LastActivityDate="2021-10-27T23:11:44.390" Title="Using OpenMP in Fortran for common array operations" Tags="|parallel-computing|fortran|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14338" PostTypeId="2" ParentId="14337" CreationDate="2014-08-08T15:38:14.613" Score="5" Body="&lt;p&gt;For the record, you might try putting an OpenMP worksharing region around your array operation syntax:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;!$omp workshare&#xA;C=A+B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Don't forget to build with OpenMP enabled (-openmp for the Intel compilers) and to set &lt;code&gt;OMP_NUM_THREADS&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2021-10-27T23:11:44.390" LastActivityDate="2021-10-27T23:11:44.390" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14339" PostTypeId="1" CreationDate="2014-08-08T20:56:25.407" Score="2" ViewCount="214" Body="&lt;p&gt;I am seeking the optimal method for implementing DG on a parallel system. For my research, I come across two types of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first problem, I am solving a time-independent (steady-state) boundary value problem. I end up with the stiffness matrix multiplied by the vector of my unknowns equal to the vector of the volume-integrated source function. In this case, it seems to me I have to solve the system as a whole since the surface fluxes are implicit on the neighbor solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second problem, I am solving a time-dependent initial value problem. After applying an explicit time discretization scheme, I end up with the mass matrix which I must invert and multiply to the vector of the stiffness matrix multiplied by the solution vector of the previous time step plus my volume-integrated source terms. In this case, it seems to me I do not have to solve the system as a whole since the surface fluxes in the last time step are known and can be passed to each local system as needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently solve both scenarios globally. Instead, is it possible to solve both scenarios locally over each cell? This to me would seem to greatly improve parallel efficiency (via domain decomposition) versus trying to solve a global system (though some parallelism can be achieved using Trilinos or another library to do the sparse matrix operations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9922" LastActivityDate="2014-08-09T05:12:23.263" Title="Optimal Discontinuous Galerkin (DG) solver on a parallel system" Tags="|finite-element|parallel-computing|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14340" PostTypeId="2" ParentId="14339" CreationDate="2014-08-09T05:12:23.263" Score="1" Body="&lt;p&gt;Solving a time-dependent problem problem with DG is typically very fast for hyperbolic problems (diffusion-type problems can be tougher due to the CFL condition), and is the subject of a large part of Hesthaven/Warburton's book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387720650&quot; rel=&quot;nofollow&quot;&gt;&quot;Nodal Discontinuous Galerkin Methods&quot;&lt;/a&gt;. This is inherently parallel, and can be made very memory efficient and scalable (see Warburton's papers on nodal DG on GPUs, for example). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the implicit/steady state equation is more subtle, or at least has way more options. Using an iterative method to solve the DG system (CG for symmetric, positive-definite operators, or GMRES in general) and a good preconditioner are usually the approach to achieving good parallel scalability. Some preconditioners are more scalable/effective than others, and the optimal choice of preconditioner may depend on the details of your problem and hardware.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some popular DG preconditioners include overlapping/non-overlapping additive Schwarz methods with some coarse grid solver and algebraic or geometric multigrid. Black-box algebraic solvers like ILU or Sparse Approximate Inverse (SPAI) methods can also do well, though their scalability is a bit more variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned Domain Decomposition (DD); an interesting note is that for DG, non-overlapping Schwarz methods correspond fairly closely to standard DD methods with specific transmission conditions (see &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/dd21DG.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; by Gander and Hajian).&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2014-08-09T05:12:23.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14341" PostTypeId="2" ParentId="13028" CreationDate="2014-08-09T05:28:26.300" Score="1" Body="&lt;p&gt;It looks like the answer to the question is no; however, &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/130912621&quot; rel=&quot;nofollow&quot;&gt;Japhet, Maday and Nataf&lt;/a&gt; have come up with a way to formulate Robin transmission conditions in tandem with a Neumann-Neumann Lagrange multiplier approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach appears to be: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;initialize $u_0, \lambda_0$ by solving the standard Neumann-Neumann problem with Lagrange multipliers. $\lambda$ represents $\pm n\cdot \nabla u_0$ for Poisson.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;given $u_0,\lambda_0$ on a given element face, compute $u_1,\lambda_1$ by solving a local problem &#xA;$$(\nabla u_1,\nabla v_1) + (u_1,v_1) - \int_{\partial K}\lambda_1,v = (f,v)$$&#xA;$$\int_{\partial K}(\lambda_1 + \alpha u_1^-)\mu = \int_{\partial K}(\lambda_0 + \alpha u_0^+)\mu$$&#xA;where $u_0^+$ is the previous $u$ on the neighboring element, so that information is transmitted across element faces.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat until converged.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2014-08-09T05:28:26.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14342" PostTypeId="2" ParentId="5088" CreationDate="2014-08-09T16:44:01.097" Score="3" Body="&lt;p&gt;This is discussed Section 11.1 of &lt;a href=&quot;http://dx.doi.org/10.1007/978-3-662-12616-5&quot; rel=&quot;nofollow&quot;&gt;Kloeden and Platen, &quot;Numerical Solution of Stochastic Differential Equations&quot;&lt;/a&gt;. There it states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using the deterministic Taylor expansion it is easy to show that the ratio&#xA;  $$\frac{1}{\sqrt\Delta}\left\{b(\tau_n,Y_n + a\Delta + b\sqrt\Delta) - b(\tau_n,Y_n)\right\}$$&#xA;  is a forward difference approximation for $b\frac{\partial{b}}{\partial{x}}$ at $(\tau_n,Y_n)$ if we neglect higher order terms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9918" LastActivityDate="2014-08-09T16:44:01.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14343" PostTypeId="2" ParentId="12860" CreationDate="2014-08-09T17:02:54.127" Score="0" Body="&lt;p&gt;A general principle is to go for good design first and then optimize on performance &lt;em&gt;only if needed&lt;/em&gt;. Use-cases where a 20% performance gain is really needed are likely to be rather rare, if they come up at all.&lt;/p&gt;&#xA;" OwnerUserId="8099" LastEditorUserId="8099" LastEditDate="2014-08-11T23:24:14.390" LastActivityDate="2014-08-11T23:24:14.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14344" PostTypeId="2" ParentId="14325" CreationDate="2014-08-09T21:26:25.867" Score="4" Body="&lt;p&gt;As indicated in several of the comments, if $\mathbf{v}$ is an eigenvector of $A$, then so is $\alpha\mathbf{v}$ for any $\alpha \ne 0$. Given an eigenvalue $\lambda$, there are thus infinitely many eigenvectors. Whatever solver you are using normalizes the eigenvectors, reducing this set to two possibilities; $\pm \mathbf{v}$ where $|\mathbf{v}|=1$. The solver has no way of knowing which of these two you want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This raises the question, do you care which you get? In terms of specifying the ellipse, or spanning the relevant space, the answer is no. If $\{\mathbf{v}_i\}$ is the set of eigenvectors, then any combination of sign changes on these vectors, $\{\pm \mathbf{v}_i\}$, spans the same subspace and equivalently defines the same ellipse when you construct your matrix $A$ from them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There &lt;em&gt;are&lt;/em&gt; cases, however, in which it matters which you get. For example, if you wanted to obtain the angular velocity of the ellipse given a time-series of matrices $\{A_i\}$, you would do so using the eigenvectors of each $A_i$. The matrix whose columns are the eigenvectors of $A$ defines a rotation, and in this case taking the opposite eigenvector will give a different rotation (although symmetric with respect to the ellipse). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it matter for your problem what the signs are? If so, we can devise a way to get you the &quot;right&quot; ones. &lt;/p&gt;&#xA;" OwnerUserId="9599" LastActivityDate="2014-08-09T21:26:25.867" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14345" PostTypeId="1" AcceptedAnswerId="14352" CreationDate="2014-08-10T13:14:11.197" Score="4" ViewCount="10862" Body="&lt;p&gt;I asked &lt;a href=&quot;https://physics.stackexchange.com/questions/119776/curl-of-an-electromagnetic-wave&quot;&gt;this&lt;/a&gt; question to help me understand what is going on in one of Maxwell's equations. I am happy with following through the maths on paper now, but would like to use MATLAB to take it one step further, and actually DO the maths on a computer. This is where I really struggle, because most of what I have used computers for is curve fitting, numeric or symbolic integration, and data processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in this case, I have the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla \times E = -\frac{\partial \textbf{B}}{\partial t}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and would like to DO this equation in MATLAB, in full 3D, using the proper term for the electric and magnetic counterparts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E \left( r, t \right) = E_{0} \exp \left[ i \left( k_{0} \cdot r - \omega_{0} t \right) \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to eventually obtain the canonical diagram of an electromagnetic wave (thank you &lt;a href=&quot;https://mathematica.stackexchange.com/questions/1987/how-do-i-plot-a-plane-em-wave&quot;&gt;THIS&lt;/a&gt; thread for this pic):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OH7wF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am pretty sure it has something to do with &lt;code&gt;meshgrid()&lt;/code&gt; and MATLAB's &lt;code&gt;curl()&lt;/code&gt; function but have no idea where to start. I hope this question makes sense. Thanks&lt;/p&gt;&#xA;" OwnerUserId="9939" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2017-05-16T09:46:02.813" Title="Visualising Maxwell's equations using MATLAB" Tags="|matlab|pde|visualization|plotting|electromagnetism|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14347" PostTypeId="2" ParentId="14345" CreationDate="2014-08-10T16:00:00.440" Score="2" Body="&lt;p&gt;From browsing some forum posts, it appears that the numerical solution to these equations is not trivial to obtain. See, for example, &lt;a href=&quot;http://www.mathworks.com/matlabcentral/newsreader/view_thread/126755&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; discussion (admittedly dated now). You indicate in your post that you are relatively new to numerical PDE solvers, and so the following references may be more involved than you were hoping for. In particular, they require you to write your own solver, as opposed to calling ode45 or some other canned routine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathematik.uni-karlsruhe.de/user/~wieners/MaxwellCourse.pdf&quot; rel=&quot;nofollow&quot;&gt;Reference 1&lt;/a&gt;: Following a brief theoretical discussion of the system, this document presents several explicit schemes for solving the system, beginning with a 1D wave equation. These solutions are complete with psuedo-code and thorough discussion, and proceed from the simpler finite-difference schemes for a 1D problem to more advanced 2D and 3D eigenvalue problems. Note that there is no Matlab code here, so you will need to translate the pseudo-code into Matlab if this is the language/software you wish to use. There are several exercises on convergence rates and stability as you proceed through the various solutions. Following this text will likely take you several hours (I know it would for me), but if you are new to numerical PDEs this is a fantastic way to learn a lot relatively quickly. I haven't read through all of it but I believe the numerics are strictly finite differences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www3.mathematik.tu-darmstadt.de/fileadmin/home/groups/10/Paper_Lang/PP08-TeleagaLang.pdf&quot; rel=&quot;nofollow&quot;&gt;Reference 2&lt;/a&gt;: This is more advanced, and presents the finite element method (FEM) to solving Maxwell's equations. It also appears to deal with some advanced applications that are probably not of interest to you. Nonetheless, the original system is completely solved in the first chapter using FEM. If you haven't heard of FEM or finite differences, I recommend starting with the latter as it tends to be more intuitive (although tedious at times). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if you would like any references to texts on finite differences or FEM. Also feel free to ask me to look at any code if you decide to try to write it and get stuck anywhere. &lt;/p&gt;&#xA;" OwnerUserId="9599" LastActivityDate="2014-08-10T16:00:00.440" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14350" PostTypeId="1" CreationDate="2014-08-11T05:43:16.540" Score="1" ViewCount="320" Body="&lt;p&gt;I am trying to solve governing equations of laminar diffusion flame jet for steady state case. In the next step, I will solve for unsteady case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have non-dimensional continuity, axial momentum, mixture fraction and energy equations. Axial momentum, mixture fraction, and energy equations have same form and boundary conditions. Since density ($\rho$) is function of mixture fraction ($mf$), I have to solve Eqns. 1.1, 1.2, and 1.3 together. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't solve any PDEs in Matlab before, and I don't know much about solution methods of coupled PDEs. So far I was able to find some finite volume methods to solve PDE, and I discovered &lt;a href=&quot;http://www.nag.com/numeric/mb/design.asp&quot; rel=&quot;nofollow noreferrer&quot;&gt;NAG Toolbox for MATLAB&lt;/a&gt; and &lt;a href=&quot;http://www.ctcms.nist.gov/fipy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FiPy: A Finite Volume PDE Solver Using Python&lt;/a&gt;. I couldn't find so far an example that is dealing with coupled PDEs with variable coefficients that is dependent on one of the solved variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mass diffusivity ($D$), exit axial velocity($u_{\text{exit}}$), and jet radius at the exit($R_N$) are constant in the governing equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/I8WNR.jpg&quot; alt=&quot;Governing equations and the boundary conditions:&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9851" LastEditorUserId="9851" LastEditDate="2014-08-13T20:11:24.850" LastActivityDate="2014-08-13T20:11:24.850" Title="How to solve coupled steady laminar diffusion flame jet problem?" Tags="|pde|matlab|parabolic-pde|advection-diffusion|" AnswerCount="1" CommentCount="4" ClosedDate="2014-08-21T11:16:42.703" ContentLicense="CC BY-SA 3.0" />
  <row Id="14351" PostTypeId="1" AcceptedAnswerId="14371" CreationDate="2014-08-11T17:57:18.657" Score="4" ViewCount="1731" Body="&lt;p&gt;I'm trying to solve variable density and viscosity Navier-Stokes equation using lagged pressure projection method. I'm solving for cavity problem as a test case now (once I get projection right, I will be solving a multiphase problem). The problem is I get oscillations developed in the solution as I march in time. .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using collocated grid for solving this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm solving variable density, viscosity N-S equation:&#xA;$$&#xA;\dfrac{u^* - u^n}{\Delta t} + \left(u_i\frac{\partial u_i}{\partial x_j} \right)^n = \left(-\frac{1}{\rho}\frac{\partial p}{\partial x_i} \right)^n + \frac{1}{\rho} \left[\frac{\partial}{\partial x_j}\left( \mu \left(\frac{\partial u_i}{\partial x_j} + \frac{\partial u_j}{\partial x_i} \right) \right)  \right]^n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and on boundary, $U^* = U^{n+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm approximating convective terms with 1st order godunov as of now, diffusion term and pressure gradient with central difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I'm solving the projection step,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\nabla.u^*}{\Delta t} = \nabla.(\frac{1}{\rho}\nabla\phi)&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;using boundary condition&#xA;$$&#xA;\frac{\partial \phi}{\partial n} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm discretizing $\nabla.u^*$ as $\dfrac{u^*_{i+1,j} - u^*_{i-1,j}}{2\Delta x} + \dfrac{v^*_{i,j+1} - v^*_{i,j-1}}{2\Delta y}$ and applying boundary condition using ghost node. and for pressure, I'm discretizing as $\dfrac{\left(\frac{1}{\rho}\frac{\partial p}{\partial x} \right)_{i+1/2,j} - \left(\frac{1}{\rho}\frac{\partial p}{\partial x} \right)_{i-1/2,j}}{h} + \dfrac{\left(\frac{1}{\rho}\frac{\partial p}{\partial y} \right)_{i,j+1/2} - \left(\frac{1}{\rho}\frac{\partial p}{\partial y} \right)_{i,j-1/2}}{h}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For solving the singular poisson equation, I'm removing the nullspace from the RHS ($\nabla.u^*$), operator matrix ($\nabla.(1/\rho$)). I'm also removing the nullspace from the solution vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using petsc to solve the poisson equation and using GMRES ksp solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I,m updating velocity using $U^{n+1} = U^* - \Delta t*(\nabla\phi)$ and $P^{n+1} = P^* + \phi$. I'm fine with first order accurate pressure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I get the oscillations in my solution. And then the solver diverges if kept iterating. I strongly feel there is something wrong with the boundary conditions, especially on $U^*$ as the solution is very sensitive to what I specify $U^*$ on boundaries. I also tried with $U^* = 0$ but no improvement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything wrong in the method I'm using here? It'll great help if you can point it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See results here: &lt;img src=&quot;https://i.stack.imgur.com/5rqBG.png&quot; alt=&quot;Results&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Found the solution: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of the fact that the staggered grid is the most straight forward solution to this checkerboard problem but I wanted to stick to the collocated grid as it is easier to code and less prone to error, plus having velocities at cell centers will greatly reduce the level set formulation efforts compared to staggered grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, instead of lagged-pressure projection method, I've changed it to the (Kim and Moin's) pressure-free projection method which is working great as of now. However, I've coded it for constant density and viscosity Navier-Stokes I'm hoping it to work for the variable density and viscosity as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While implementing, now there is no pressure gradient term in the momentum equation and I used following boundary conditions on intermediate velocity field:&#xA;$$&#xA;U^*.\hat{n} = U^{n+1}.\hat{n}&#xA;$$&#xA;$$&#xA;U^*.\hat{t} = (U^{n+1} + \Delta t*\nabla\phi).\hat{t}&#xA;$$&#xA;where $\hat{n}$ and $\hat{t}$ are normal and tangent vectors. And poisson equation has the neumann B.C. as before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I have to get going with my project, I have not thoroughly investigated the error in lagged-pressure projection method But I will do so as soon as I get some time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the solution (at t = 10sec with Re400) with pressure-free projection method:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/XLw8p.jpg&quot; alt=&quot;quiver&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h6Mrf.jpg&quot; alt=&quot;Pressure&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9950" LastEditorUserId="9950" LastEditDate="2014-08-29T04:33:11.140" LastActivityDate="2016-05-16T13:16:59.253" Title="Projection Method: Boundary condition on intermediate velocity field" Tags="|fluid-dynamics|finite-difference|navier-stokes|projection|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14352" PostTypeId="2" ParentId="14345" CreationDate="2014-08-11T19:35:11.970" Score="11" Body="&lt;p&gt;Here I have an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = linspace(-5,5,100);&#xA;y = linspace(-5,5,100);&#xA;z = linspace(-5,5,100);&#xA;&#xA;[X, Y, Z] = meshgrid(x, y, z);&#xA;&#xA;Ex = sin(2*pi/5*Z);&#xA;Ey = 0*X;&#xA;Ez = 0*X;&#xA;&#xA;[Bx, By, Bz, V] = curl(X, Y, Z, Ex, Ey, Ez);&#xA;&#xA;Eplot = 0*x;&#xA;Bplot = 0*x;&#xA;for i=1:100  %% Integration-like procedure&#xA;    Eplot(i) = mean(mean(Ex(:,:,i),1),2);&#xA;    Bplot(i) = mean(mean(By(:,:,i),1),2);&#xA;end&#xA;&#xA;&#xA;plot3(0*x, y, Eplot, 'b', 'LineWidth', 2); hold on&#xA;h = quiver3(0*x(1:3:100), y(1:3:100), 0*z(1:3:100), 0*x(1:3:100), 0*y(1:3:100), Eplot(1:3:100), 0, 'b', 'LineWidth', 1);&#xA;set (h, &quot;maxheadsize&quot;, 0.0);&#xA;plot3(Bplot, y, 0*z, 'g', 'LineWidth', 2);&#xA;h = quiver3(0*x(1:3:100), y(1:3:100), 0*z(1:3:100), Bplot(1:3:100), 0*y(1:3:100), 0*z(1:3:100), 0, 'g', 'LineWidth', 1);&#xA;set (h, &quot;maxheadsize&quot;, 0.0);&#xA;grid on, axis square&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't understand what do you mean with &lt;em&gt;integration&lt;/em&gt;, so I computed the average over each plane (that is the same that plotting over a line in this case). This was done in Octave since I don't have a Matlab license in my laptop, but it should work. Here is the image that I obtained:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/6W618.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2014-08-11T19:35:11.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14353" PostTypeId="2" ParentId="14350" CreationDate="2014-08-11T20:53:43.323" Score="3" Body="&lt;p&gt;Providing a whole detailed solution is out of the scope of this site, but asking for references is on-topic, so here is what I would suggest to get started:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many good books on finite difference methods (for instance, &quot;Finite Difference Methods for Ordinary and Partial Differential Equations: Steady-State and Time-Dependent Problems&quot; by LeVeque, or &quot;Finite Difference Schemes &amp;amp; Partial Differential Equations&quot; by Strikwerda) that would be adequate for the 1-D case, and that are good for a variety of science and engineering problems. Finite difference methods also appear in the combustion literature, as early codes used this approach in calculations for laminar flames. You could also use this approach for diffusion flames as well, even in multiple dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For books on finite volume methods, &quot;Computational Methods for Fluid Dynamics&quot; by Ferziger and Peric was most helpful for me, even though it doesn't cover chemical reaction at all. &quot;Finite Volume Methods for Hyperbolic Problems&quot; by LeVeque is a really good book, and covers the mathematics really well; from the standpoint of combustion, it essentially advocates operator splitting finite volume discretizations, whereas Ferziger and Peric will discuss fully implicit finite volume discretizations. Both approaches can be found in the literature, and in practice. Finite volume approaches tend to be used more in multi-dimensional combustion modeling, so if you want to look at 2-D or 3-D laminar flames, you're better off learning these techniques (you could also use them in 1-D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading and taking notes on the references, I'd work out simple exercise problems to get comfortable with the numerical methods, including small systems of PDEs, and then move to your actual research problem. It's important not to rush this process, because you need to learn what sorts of bugs can arise when working with these numerical methods, and how to fix them, as well as where these numerical methods break down. If you just skip to your research problem, you won't have a good handle on diagnosing bugs you encounter (and there will be bugs; no one implements these methods perfectly the first time).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-11T20:53:43.323" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14354" PostTypeId="2" ParentId="14192" CreationDate="2014-08-11T21:14:12.877" Score="2" Body="&lt;p&gt;How accurate must your propagation be? You can play with the accuracy using the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/odeset.html&quot; rel=&quot;nofollow&quot; title=&quot;odeset&quot;&gt;odeset&lt;/a&gt; function (to create a set of options for ode45). Since ode45 is variable step, the accuracy has a huge influence on the computational speed. The higher the required accuracy, the more steps ode45 will take. By default, the relative accuracy is set to 1e-3 and the absolute accuracy to 1e-6.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also consider integrating using another language (such as C or Fortran) and &lt;a href=&quot;http://www.mathworks.com/help/matlab/matlab_external/introducing-mex-files.html&quot; rel=&quot;nofollow&quot; title=&quot;Mex files&quot;&gt;Mex functions&lt;/a&gt; or loading libraries (see &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/loadlibrary.html&quot; rel=&quot;nofollow&quot; title=&quot;loadlibrary&quot;&gt;loadlibrary&lt;/a&gt;). This will definitely result in huge speedups, since compiled languages are much faster than Matlab for this type of operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: added link to Mex functions (restrictions for new user allowed only 2 links per post)&lt;/p&gt;&#xA;" OwnerUserId="9889" LastEditorUserId="9889" LastEditDate="2014-08-11T22:38:55.767" LastActivityDate="2014-08-11T22:38:55.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14355" PostTypeId="1" AcceptedAnswerId="14356" CreationDate="2014-08-11T22:36:23.383" Score="8" ViewCount="2851" Body="&lt;p&gt;Most numerical algorithms require an epsilon to be chosen in order to be robust and provide meaningful results.  Choosing machine epsilon is usually too aggressive.  Barring any special knowledge about your problem, one rule of thumb I remember is to use $\sqrt{\epsilon_{\text{machine}}}$.  Eg: &lt;a href=&quot;http://en.wikipedia.org/wiki/Numerical_differentiation#Practical_considerations_using_floating_point_arithmetic&quot; rel=&quot;noreferrer&quot;&gt;it's mentioned in this wikipedia article&lt;/a&gt;.  Which basically is the same as assuming that half of the digits in your floating point number are bogus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I can't figure out where I got that rule of thumb.  I just see it crop up, unreferenced, in various texts.  Is there a source for it I could point someone to if they were uninformed about practical issues of numerical algorithms and naively trying to use machine epsilon?&lt;/p&gt;&#xA;" OwnerUserId="1365" LastActivityDate="2014-08-11T22:53:02.173" Title="Choosing epsilons" Tags="|numerical-analysis|floating-point|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14356" PostTypeId="2" ParentId="14355" CreationDate="2014-08-11T22:49:03.487" Score="9" Body="&lt;p&gt;The Wikipedia article cite Numerical Recipes, 3rd edition, Section 5.7, which is pages 229-230 (a limited number of page views is available at &lt;a href=&quot;http://www.nrbook.com/empanel/&quot; rel=&quot;noreferrer&quot;&gt;http://www.nrbook.com/empanel/&lt;/a&gt;). Sure enough, they provide a derivation to demonstrate that using a one-sided difference quotient, $\sqrt{\varepsilon}$ is a good choice of step-size for finite difference approximations of derivatives. I don't know that $\sqrt{\varepsilon}$ would be great for other choices of algorithm in the absence of problem-specific information. For specific algorithms, you're probably better off trying to get information about your problem, using some reasonable strategies for trial-and-error with different choices of tolerance, and maybe looking up more detailed references for the specific algorithms you're using. A good catch-all reference I like to recommend is Accuracy and Stability of Numerical Algorithms by Nicholas Higham.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-11T22:49:03.487" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14357" PostTypeId="2" ParentId="14355" CreationDate="2014-08-11T22:53:02.173" Score="4" Body="&lt;p&gt;In the context of the forward difference method of numerical differentiation, the choice of a stepsize that is about $h=\sqrt{\epsilon_{\mbox{machine}}}$ can be justified by an analysis of the truncation and roundoff errors that occur- this particular choice minimizes the sum of the two errors.  However, if you switch to a centered difference formula it turns out that $h=\epsilon_{\mbox{machine}}^{1/3}$ is optimal.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-08-11T22:53:02.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14358" PostTypeId="2" ParentId="11807" CreationDate="2014-08-11T23:46:16.033" Score="2" Body="&lt;p&gt;If you have an easy access to the Hessian (as would be possible with the Rosenbrock function), then you should use a second-order method, such as the &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton&amp;#39;s_method_in_optimization&quot; rel=&quot;nofollow&quot; title=&quot;Newton&amp;#39;s method&quot;&gt;Newton Raphson method&lt;/a&gt;. It is both easy to implement and quadratically convergent for a sufficiently &quot;nice&quot; function (see the conditions &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton&amp;#39;s_method#Proof_of_quadratic_convergence_for_Newton.27s_iterative_method&quot; rel=&quot;nofollow&quot; title=&quot;Proof of convergence of Newton&amp;#39;s Method&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="9889" LastActivityDate="2014-08-11T23:46:16.033" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14359" PostTypeId="1" AcceptedAnswerId="14362" CreationDate="2014-08-12T04:35:03.603" Score="3" ViewCount="2487" Body="&lt;p&gt;If you call a library like LAPACK or BLAS (which are written in FORTRAN and use column major order) from a C-like language that uses row major order, won't you lose performance and use a lot of memory due to the creation of transposed matrices? Tests are best, of course, but can anyone with experience tell me whether optimized libraries still improve performance, even after that overhead?&lt;/p&gt;&#xA;" OwnerUserId="1113" LastEditorUserId="1113" LastEditDate="2014-08-12T22:23:54.593" LastActivityDate="2023-10-30T08:24:47.330" Title="Impact of frequent row major / column major conversions using LAPACK/BLAS?" Tags="|performance|libraries|lapack|blas|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14360" PostTypeId="2" ParentId="12860" CreationDate="2014-08-12T06:15:29.983" Score="1" Body="&lt;p&gt;The software design won't automagically improve over time. The performance will. You will get the 20% back with your next CPU. Besides, good software design will make it easier to extend or improve the library in the future.&lt;/p&gt;&#xA;" OwnerUserId="9935" LastActivityDate="2014-08-12T06:15:29.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14361" PostTypeId="1" CreationDate="2014-08-12T10:30:19.683" Score="1" ViewCount="53" Body="&lt;p&gt;I am trying to calculate the second and third invariants for a turbulent flow. I have the second order statistics (both transient and averaged). i.e &lt;span class=&quot;math-container&quot;&gt;$uu$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$vv$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$ww$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$uv$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$vw$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$uw$&lt;/span&gt;. These are supposed to be expanded as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$II = b_{ij} / 2 \, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$III = b_{ij} b_{jk} b_{ki}/3 \, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I am stuck at this point on how to expand them further to get the second and third invariants.&lt;/p&gt;&#xA;" OwnerUserId="9956" LastEditorUserId="9667" LastEditDate="2019-03-16T18:04:37.930" LastActivityDate="2019-03-16T18:04:37.930" Title="Anisotropic invariant expansion" Tags="|fluid-dynamics|computational-physics|tensor|tensor-decomposition|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14362" PostTypeId="2" ParentId="14359" CreationDate="2014-08-12T11:58:08.203" Score="3" Body="&lt;p&gt;Many of these libraries have C interfaces that swap the meaning of the ordering internally without swapping the data. Also, using C-style double pointers for 2-D matrices is probably the wrong choice in the first place (due to the double lookup), so you can make your data appear column-major by indexing into a linear C array or use libraries that work around these issues.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-08-12T11:58:08.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14364" PostTypeId="2" ParentId="14361" CreationDate="2014-08-12T14:07:24.330" Score="2" Body="&lt;p&gt;Besides fixing the second invariant, you need to know that repeated indicies imply summation. So $A_{ii}=\sum_{k=1}^d [A]_{kk}$. This is known as the &lt;a href=&quot;http://en.wikipedia.org/wiki/Einstein_notation&quot; rel=&quot;nofollow&quot;&gt;Einstein Summation Convention&lt;/a&gt;. Each of the second-order statistics is a rank 3 tensor that you can sum up at each point of your domain using this convention in order to compute the invariants at those locations.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-08-12T14:07:24.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14365" PostTypeId="2" ParentId="6701" CreationDate="2014-08-12T14:53:21.927" Score="-3" Body="&lt;p&gt;No, you cannot use Thomas algorithm for block matrixes in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go to wikipedia &lt;a href=&quot;http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&lt;/a&gt;, you will see that this formula is also there, but this form is derived from form with ~, ~ form may be used for block tridiagonal problems in some special cases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thomas algorithm is basically Gauss elimination method, problem with block matrixes is that product of two matrixes is not commutative (in general), so Gauss elimination can't work, for block problem you can only multiply from the left! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Ai block are commutative with Bi and Ci and Bi~ blocks (this is the greatest problem since it would be hard to check commutativity because you derive Bi blocks in every iteration) then you can use Thomas ~ algorithm explained on wikipedia (I think you can't use form above because that form is derived from ~ form by multiplying every equation in matrix with 1/Bi, when you deal with matrix blocks that approach is not safe, it is better to use ~, it's not hard to derive formulas for ~ form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cases when this method can be used for tridiagonal block matrix problem are quite rare, because of commutativity with Bi~ blocks is needed. Luckily for you many differential equations have Ai i Ci blocks in a form const*eye(n)=const * I  (in many cases Ai=Ci) and eye matrix is commutative with everything :D &lt;/p&gt;&#xA;" OwnerUserId="9961" LastActivityDate="2014-08-12T14:53:21.927" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14366" PostTypeId="1" AcceptedAnswerId="14444" CreationDate="2014-08-12T16:47:56.667" Score="6" ViewCount="241" Body="&lt;p&gt;I asked a similar question on MathSE but with more added fluff, but didn't really get any straight answers, so I figured I'd ask here. Computing Fourier coefficients of a function using the FFT is &lt;a href=&quot;https://math.stackexchange.com/questions/894381/how-to-compute-fourier-coefficients-using-a-cubic-spline-corrected-fft&quot;&gt;notoriously inaccurate&lt;/a&gt; whenever the magnitude of the coefficients is small, and &lt;a href=&quot;http://www.aip.de/groups/soe/local/numres/bookcpdf/c13-9.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;13.9 of &quot;Numerical Recipes in C&quot;&lt;/a&gt; makes a partial fix which uses spline basis functions, allowing one to still use the FFT, but drastically reducing the number of samples required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author uses trapezoidial and cubic corrections, and I was mainly interested in the cubic correction. I'm interested in the derivation because I want to see how it works for multidimensional corrections (like 2D functions).  The derivation of the $W(\theta)$ and $\alpha_j$ functions is not explicitly shown (since it's a code/recipe book), but the author states &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;For linear interpolation ψ(s) is piecewise linear, rises from 0 to 1&#xA;  for s in (−1, 0), and falls back to 0 for s in (0, 1). For higher-order interpolation, ψ(s) is&#xA;  made up piecewise of segments of Lagrange interpolation polynomials. It has discontinuous&#xA;  derivatives at integer values of s, where the pieces join, because the set of points used in&#xA;  the interpolation changes discretely.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To that end, I tried to reproduce the cubic corrections by defining&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(x)=\begin{cases}0&amp;amp;x&amp;lt;-1\\\text{Lag}_3\left(\begin{bmatrix}0&amp;amp;0&amp;amp;1&amp;amp;0\\-2&amp;amp;-1&amp;amp;0&amp;amp;1\end{bmatrix},x\right)&amp;amp;-1\leq x\leq0\\\text{Lag}_3\left(\begin{bmatrix}0&amp;amp;1&amp;amp;0&amp;amp;0\\-1&amp;amp;0&amp;amp;1&amp;amp;2\end{bmatrix},x\right)&amp;amp;0\leq x\leq1\\0&amp;amp;1&amp;lt;x&#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $$\text{Lag}_3\left(\begin{bmatrix}y_1&amp;amp;y_2&amp;amp;y_3&amp;amp;y_4\\x_1&amp;amp;x_2&amp;amp;x_3&amp;amp;x_4\end{bmatrix},x\right)=\sum_{j=1}^4y_k\prod_{\begin{smallmatrix}k=1\\ k\neq j\end{smallmatrix}}^4\frac{x-x_k}{x_j-x_k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is the cubic Lagrange polynomial which passes the points specified by the $x_k,y_k$. However, this yields (in Mathematica)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CubicLagrangeInterpolation[X_, Y_, x_] := &#xA;  Sum[Y[[j]]*Product[If[j == k, 1, (x - X[[k]])/(X[[j]] - X[[k]])], &#xA;           {k, 1, 4}], {j, 1, 4}]; &#xA;\[Psi][x_] := &#xA;  Piecewise[{{0, &#xA;     x &amp;lt; -1}, {CubicLagrangeInterpolation[{-2, -1, 0, 1}, {0, 0, 1, &#xA;       0}, x], -1 &amp;lt;= x &amp;lt;= 0}, &#xA;         {CubicLagrangeInterpolation[{-1, 0, 1, 2}, {0, 1, 0, 0}, x], &#xA;     0 &amp;lt;= x &amp;lt;= 1}, {0, x &amp;gt; 1}}]; &#xA;FullSimplify[&#xA; Integrate[\[Psi][s]*Exp[I*\[Theta]*s], {s, -Infinity, Infinity}]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$W(\theta)=\int_{-\infty}^\infty\psi(s)e^{i\theta s}\,\mathrm{d}s=\frac{\theta ^2-2 \left(\theta ^2+3\right) \cos (\theta )-2 \theta  \sin (\theta )+6}{\theta ^4}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which does not match the $W(\theta)$ provided in the article, although when plotted the two are somewhat similar in appearance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, I am confused how he obtains his $\varphi$ endpoint kernel functions. Does anyone know what the author meant when he said he uses &quot;Lagrange interpolation&quot; to obtain his result? Or am I simply using Lagrange interpolation incorrectly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minor note: When I use linear Lagrange interpolation ($\text{Lag}_1$ functions) I get the correct result for $W(\theta)$ for the author's &quot;trapezoidial&quot; case, and $\psi(x)$ is just a unit triangle.&lt;/p&gt;&#xA;" OwnerUserId="6928" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-08-20T22:23:07.393" Title="What spline functions are used in Section 13.9 of &quot;Numerical Recipes in C&quot;?" Tags="|interpolation|fourier-analysis|b-spline|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14367" PostTypeId="1" CreationDate="2014-08-12T19:52:14.053" Score="1" ViewCount="104" Body="&lt;p&gt;I have a bunch of points in $\mathbb{R}^3$ that I would like to translate and rotate so that their center is at the origin and the variance along the $x$ and $y$ axes are maximal (greedy, and in that order). To accomplish this I am trying to use python's principal components analysis algorithm. It is not behaving as I expect it to, most likely due to some misunderstanding about what PCA actually does on my part. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Problem&lt;/strong&gt;: &#xA;When I center and then rotate the data, the variance along the third component is greater than along the second. This means that, once centered and rotated, there is more variance in the data along the $z$ axis than there is along the $y$. In other words, the rotation is not the correct one.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I am Doing&lt;/strong&gt;:&#xA;Python's PCA routine returns an object (say myPCA) with several attributes. myPCA.Y is the data array, but centered, scaled, and rotated (in that order). I do not want the data to be scaled. I simply want a translation and a rotation.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np                                                              &#xA;from matplotlib.mlab import PCA                                                 &#xA;&#xA;# manufactured data producing the problem                                       &#xA;data_raw  = np.array([                                                          &#xA;                     [80.0, 50.0, 30.0],                                        &#xA;                     [50.0, 90.0, 60.0],                                        &#xA;                     [70.0, 20.0, 40.0],                                        &#xA;                     [60.0, 30.0, 45.0],                                        &#xA;                     [45.0, 60.0, 20.0]                                         &#xA;                     ])                                                         &#xA;&#xA;# obtain the PCA                                                                &#xA;myPCA = PCA(data_raw)                                                           &#xA;&#xA;# center the raw data                                                           &#xA;centered = np.array([point - myPCA.mu for point in data_raw])                   &#xA;# rotate the centered data                                                      &#xA;centered_and_rotated = np.array([np.dot(myPCA.Wt, point) for point in centered])&#xA;# the variance along axis 0 should now be greater than along 1, so on           &#xA;variances = np.array([np.var(centered_and_rotated[:,i]) for i in range(3)])     &#xA;# they are not:                                                                 &#xA;print(variances[1]&amp;gt;variances[2]) #False; I want this to be True                 &#xA;&#xA;# Now look at the PCA output, Y. This is centered, scaled, and rotated.          &#xA;# The variances decrease in magnitude, as I want them to:                       &#xA;variances2 = np.array([np.var(myPCA.Y[:,i]) for i in range(3)])                 &#xA;# This looks good, but the coordinates have been scaled.      &#xA;# Let's try to get from the raw coordinates to the PCA output Y&#xA;# mu is the vector of means of the raw data, and sigma is the vector of &#xA;# standard deviations of the raw data along each coordinate direction             &#xA;guess = np.array([np.dot(myPCA.Wt, (xxx-myPCA.mu)/myPCA.sigma) for xxx in data_raw])&#xA;print(guess==myPCA.Y) # all true  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last two lines in the above show that we may take a point $\mathbf{x}$ from its representation in the raw data input into its representation $\mathbf{x}'$ in terms of the PCA axes via &#xA;$$&#xA;\mathbf{x}' = \mathrm{R}\cdot\left((\mathbf{x}-\boldsymbol{\mu}) / \boldsymbol{\sigma} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathrm{R}$ is myPCA.Wt, the weight matrix, $\boldsymbol{\mu}$ is the vector of means of the original data along each coordinate axis, $\boldsymbol{\sigma}$ is the vector of standard deviations of the original data along each coordinate axis, and the division is element-wise. In order to write this in standard mathematical notation, let's replace this division by multiplication:&#xA;$$&#xA;\mathbf{x}' = \mathrm{R}\cdot\left(\mathrm{D}\cdot(\mathbf{x}-\boldsymbol{\mu}) \right)&#xA;$$&#xA;where $\mathrm{D}$ is a diagonal matrix whose diagonal entries are $1/\sigma_i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This notation makes clear the problem: to undo the scaling, I need to act on the RHS above with $\mathrm{R}\mathrm{D}^{-1}\mathrm{R}^{-1}$. This will return me to the problem situation, in which the variance is greater along the $z$ axis than the $y$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to use PCA to get what I want, or do I need to use another method? &lt;/p&gt;&#xA;" OwnerUserId="9599" LastActivityDate="2014-08-12T19:52:14.053" Title="Principal Components Analysis Not Behaving as I Expect it to" Tags="|linear-algebra|python|geometry|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14368" PostTypeId="1" AcceptedAnswerId="14377" CreationDate="2014-08-12T22:03:20.800" Score="2" ViewCount="1399" Body="&lt;p&gt;I have some code illustrating my problem below. As you can see from the results- Matlab is significantly faster than python. It seems that the function which performs the levinson durbin recursion in Matlab is a Mex file. Could this be why Matlab is significantly faster? How would I make the Python implementation on par with Matlab, or does anyone know if such a version already exists?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a separate issue, the python aryule function does not seem to be able to handle white noise that is not unit variance. For instance if I were to use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = 0.2*randn(1, 256000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the python estimates for the AR coefficients would be less accurate. Matlab doesn't seem to have this problem. Is there a way to implement this in Python?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Matlab Implementation&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A=[1 -2.7607 3.8106 -2.6535 0.9238];&#xA;% AR(4) coefficients&#xA;y=filter(1,A,0.2*randn(256000,1));&#xA;%filter a white noise input to create AR(4) process&#xA;t = cputime;  &#xA;[ar_coeffs v]=aryule(y,4);&#xA;e = cputime-t;&#xA;disp(['Elapsed Time: ' num2str(e)]);&#xA;%compare the results in ar_coeffs to the vector A.&#xA;ar_coeffs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Elapsed Time: 0.14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Python Implementation&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from pylab import *&#xA;import scipy.signal&#xA;from spectrum import *&#xA;import time&#xA;&#xA;# Create an AR model&#xA;a = [1, -2.7607, 3.8106, -2.6535, 0.9238]&#xA;# create some data based on these AR parameters&#xA;X = randn(1, 256000)&#xA;y = scipy.signal.lfilter([1], a, X)&#xA;# now, let us try to estimate the original AR parameters&#xA;t0 = time.time()&#xA;AR, P, k = aryule(y[0], 4)&#xA;print &quot;Time elapsed: &quot;, time.time()-t0&#xA;%compare the results in ar_coeffs to the vector A.&#xA;print(AR)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Time elapsed:  2.48965501785&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastActivityDate="2014-08-13T21:29:05.590" Title="Parametric spectrum estimation using aryule: difference between Python and Matlab" Tags="|matlab|python|signal-processing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14369" PostTypeId="1" CreationDate="2014-08-12T22:32:51.933" Score="4" ViewCount="9954" Body="&lt;p&gt;I would greatly appreciate it if you could share some reasons the Conjugate Gradient iteration for Ax = b does not converge? My matrix A is symmetric positive definite. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit with more information:&#xA;My matrix is the reduced Hessian in the optimization algorithms for problems with simple constraints. The matrix is 50% sparse and I am using matlab. &lt;/p&gt;&#xA;" OwnerUserId="9968" LastEditorUserId="9968" LastEditDate="2014-08-12T23:25:12.990" LastActivityDate="2014-08-15T13:11:18.263" Title="What are some reasons that Conjugate Gradient iteration does not converge?" Tags="|iterative-method|convergence|conjugate-gradient|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14370" PostTypeId="1" AcceptedAnswerId="14574" CreationDate="2014-08-12T22:38:48.930" Score="5" ViewCount="2683" Body="&lt;p&gt;I'm trying to plot some vector fields using LIC technique.&#xA;More specifically, I'm using the Python &lt;a href=&quot;http://wiki.scipy.org/Cookbook/LineIntegralConvolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;solution&lt;/a&gt; for this kind of plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before applying that approach, I was plotting my vectors as quiver. This give me results like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TYI4P.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for large data, this technique is not recommended because the larger the data, the more compressed the arrows will be, and, consequently, the plot will get confusing and not readable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using LIC, however, I got the opposite result.&#xA;I get a poor resolution for my plot and it is hard to understand the flow behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pnu13.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both plots are from a 61 x 61 vector field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really would like to use LIC, however, I'm wondering if there is a data size requirement to get good results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an approach to solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9799" LastEditorUserId="9799" LastEditDate="2014-09-06T18:29:44.480" LastActivityDate="2019-02-28T18:03:17.690" Title="Line Integral Convolution (LIC) Requirements" Tags="|python|visualization|integral-equations|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14371" PostTypeId="2" ParentId="14351" CreationDate="2014-08-13T00:46:13.780" Score="2" Body="&lt;p&gt;What you are observing is odd-even decoupling between the velocity and pressure. This is a well-known problem on collocated grids. At any point on the grid, the gradient of the pressure is calculated using $p_{i+1}$ and $p_{i-1}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u_i^{n+1}-u_i^n}{\Delta t} + (u_i^n.\nabla)u_i^n = -\frac{p_{i+1}-p_{i-1}}{2\Delta x}+\nu\nabla^2 u_i^n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence the velocity $u_i$ only depends on the pressure at the adjacent grid points, and not on $p_i$ itself. This can cause the numerical pressure to behave strangely. As you can see in your results, a &quot;checkerboard&quot; pattern is observed, where only the pressures at alternate grid points influence each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem can be solved by using a staggered grid, where the velocities and pressures are calculated at different points in the domain. The most commonly used staggered grid on a Cartesian mesh is the one introduced by Harlow and Welch [1], but it will make your code more complex. If you want to make minimal changes to your code, then use the same collocated grid for the velocities, but compute the pressures at the center of each cell. Take care to set up your pressure equation carefully, and interpolate the velocities as required when you calculate the right-hand side $\nabla\cdot u^*$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is another page that discusses this phenomenon: &lt;a href=&quot;http://www.imm.dtu.dk/~pcha/Projekter/Shock/main/node17.html&quot; rel=&quot;nofollow&quot;&gt;http://www.imm.dtu.dk/~pcha/Projekter/Shock/main/node17.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the boundary conditions for the intermediate velocity field, rather than $u^*=0$, you should be using $u^*=u^{n+1}$. In the problem you have considered, this makes a difference on the top surface where the lid is moving. For more information about the boundary conditions using a projection method, read the paper by Perot [2] and follow the references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Harlow, Francis H., and J. Eddie Welch. &quot;Numerical calculation of time-dependent viscous incompressible flow of fluid with free surface.&quot; Physics of fluids 8.12 (1965): 2182.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Perot, J. Blair. &quot;An analysis of the fractional step method.&quot; Journal of Computational Physics 108.1 (1993): 51-58.&lt;/p&gt;&#xA;" OwnerUserId="3886" LastEditorUserId="3886" LastEditDate="2014-08-13T03:47:07.163" LastActivityDate="2014-08-13T03:47:07.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14374" PostTypeId="2" ParentId="14368" CreationDate="2014-08-13T09:56:17.097" Score="1" Body="&lt;p&gt;How to make that process faster? Well, firstly if you feel like having the time you could check the implementation, which I'm doing just right now.&#xA;(Warning: I need to truncate links as stackexchange does not let me have more than 2 links)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code is here:&#xA;assembla . com /code/PySpectrum/subversion/nodes/37/trunk/src/spectrum&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Giving a look at the package it already contains some .c code to fasten up some operations. That may ease the implementation in C of some part to later on import it via Python if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did&#xA;    svn checkout &lt;a href=&quot;https://subversion.assembla.com/svn/PySpectrum/&quot; rel=&quot;nofollow&quot;&gt;https://subversion.assembla.com/svn/PySpectrum/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I got:&#xA;    ~/spectrumlib/PySpectrum/trunk/src/cpp$ ls&#xA;    &lt;strong&gt;init&lt;/strong&gt;.py  mydpss.c&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mydpss.c implements some subroutines. Something could be added there maybe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But let's take a look at the code that is actually being executed:&#xA;assembla . com /code/PySpectrum/subversion/nodes/37/trunk/src/spectrum/yulewalker.py&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from correlation import CORRELATION&#xA;from levinson import LEVINSON&#xA;&#xA;r = CORRELATION(X, maxlags=order, norm=norm)&#xA;A, P, k = LEVINSON(r, allow_singularity=allow_singularity)&#xA;return A, P, k&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is what is being executed. Let's check what is the slow part here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I modified yulewalker.py like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import time&#xA;print &quot;Correlation calculation&quot;&#xA;t0 = time.time()&#xA;r = CORRELATION(X, maxlags=order, norm=norm)&#xA;print &quot;Time elapsed: &quot;, time.time()-t0&#xA;print &quot;Levinson calculation&quot;&#xA;t1 = time.time()&#xA;A, P, k = LEVINSON(r, allow_singularity=allow_singularity)&#xA;print &quot;Time elapsed: &quot;, time.time()-t1&#xA;return A, P, k&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I ran your piece of code of stackexchange and I got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./aryulespeed.py &#xA;Correlation calculation&#xA;Time elapsed:  1.50509214401&#xA;Levinson calculation&#xA;Time elapsed:  0.000110864639282&#xA;Time elapsed:  1.50538802147&#xA;[-2.7616739   3.81292317 -2.65578205  0.9248034 ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the correlation calculation is the slow part! aha!&#xA;This guy: &#xA;assembla . com /code/PySpectrum/subversion/nodes/37/trunk/src/spectrum/correlation.py&#xA;seems to be the fat guy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oh, look, a funny comment in the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Provides two correlation functions. :func:`CORRELATION` is slower than&#xA;:func:`xcorr`. However, the output is as expected by some other functions.&#xA;Ultimately, it should be replaced by :func:`xcorr`.&#xA;&#xA;For real data, the behaviour of the 2 functions is identical. However, for&#xA;complex data, xcorr returns a 2-sides correlation.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Maybe we should try to interchange this CORRELATION for xcorr!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#! /usr/bin/env python&#xA;from pylab import *&#xA;import scipy.signal&#xA;from spectrum import *&#xA;import time&#xA;&#xA;# The original imports&#xA;from spectrum.correlation import CORRELATION&#xA;from spectrum.levinson import LEVINSON&#xA;# The faster corr&#xA;from spectrum.correlation import xcorr&#xA;&#xA;def superawesomearyule(X, order, norm='biased', allow_singularity=True):&#xA;&#xA;    assert norm in ['biased', 'unbiased']&#xA;    import time&#xA;    print &quot;Correlation calculation in superawesomearyule&quot;&#xA;    t0 = time.time()&#xA;    #r = CORRELATION(X, maxlags=order, norm=norm) # Commenting you, slowwww     code!&#xA;    r = xcorr(X, maxlags=order, norm=norm)&#xA;    print &quot;Time elapsed: &quot;, time.time()-t0&#xA;    print &quot;Levinson calculation&quot;&#xA;    t1 = time.time()&#xA;    A, P, k = LEVINSON(r, allow_singularity=allow_singularity)&#xA;    print &quot;Time elapsed: &quot;, time.time()-t1&#xA;    return A, P, k&#xA;&#xA;&#xA;&#xA;# Create an AR model&#xA;a = [1, -2.7607, 3.8106, -2.6535, 0.9238]&#xA;# create some data based on these AR parameters&#xA;X = randn(1, 256000)&#xA;y = scipy.signal.lfilter([1], a, X)&#xA;# now, let us try to estimate the original AR parameters&#xA;t0 = time.time()&#xA;#AR, P, k = aryule(y[0], 4)&#xA;AR, P, k = superawesomearyule(y[0], 4)&#xA;print &quot;Time elapsed: &quot;, time.time()-t0&#xA;#%compare the results in ar_coeffs to the vector A.&#xA;print(AR)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And executed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./aryulespeed.py &#xA;Correlation calculation in superawesomearyule&#xA;Time elapsed:  69.9396679401&#xA;Levinson calculation&#xA;Traceback (most recent call last):&#xA;  File &quot;./aryulespeed.py&quot;, line 38, in &amp;lt;module&amp;gt;&#xA;    AR, P, k = superawesomearyule(y[0], 4)&#xA;  File &quot;./aryulespeed.py&quot;, line 24, in superawesomearyule&#xA;    A, P, k = LEVINSON(r, allow_singularity=allow_singularity)&#xA;  File &quot;/usr/local/lib/python2.7/dist-packages/spectrum/levinson.py&quot;, line     112, in LEVINSON&#xA;    if P &amp;lt;= 0 and allow_singularity==False:&#xA;ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And...... oh noes :( it took 69 seconds!! and also crashed after that on levinson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think giving it a bit of love something may be done here. Maybe sending a mail to it's developer Thomas Cokelaer would help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I'm sorry but I can't invest more time in this issue right now. I hope this helps you in some way :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found this github repo:&#xA;&lt;a href=&quot;https://github.com/RhysU/ar&quot; rel=&quot;nofollow&quot;&gt;https://github.com/RhysU/ar&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which has an ar implementation with some python bindings. It may also help you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="9970" LastActivityDate="2014-08-13T09:56:17.097" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14375" PostTypeId="1" AcceptedAnswerId="14376" CreationDate="2014-08-13T16:00:13.467" Score="0" ViewCount="596" Body="&lt;p&gt;I want to perform gradient descent optimization of the probability of a sample under a multivariate normal probability density function. For your convenience I state the PDF here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{N}(\boldsymbol\mu,\,\boldsymbol\Sigma) = (2\pi)^{-\frac{k}{2}}|\boldsymbol\Sigma|^{-\frac{1}{2}}\, e^{ -\frac{1}{2}(\mathbf{x}-\boldsymbol\mu)'\boldsymbol\Sigma^{-1}(\mathbf{x}-\boldsymbol\mu) }$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The probability of a sample under this model for the special case of zero mean and unit covariance is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(\vec{x}) = \frac{1}{(2\pi)^{-n/2}}\prod_{i=1}^n e^{-\frac{x_i^2}{2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $n$ is the number of entries in $\vec{x}$. $\vec{x}$ is the projection of a sample onto a PCA model in each iteration and so should actually be seen as a function of a different set of variables $\vec{y}$ that are parametrizing the original problem. In addition, I would like to work with the log of the probability for numerical stability. All in all, I'm looking for the derivative&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d}{d\vec{y}}\log p(\vec{x(\vec{y}})) = \frac{d}{d\vec{y}}\log \frac{1}{(2\pi)^{-n/2}}\prod_{i=1}^n e^{-\frac{\vec{x_i(\vec{y}})^2}{2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this where I'm stuck. Help and pointers are really appreciated. So my questions are these:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Is this approach to deriving a gradient descent cost function correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) What is the derivative given by?&lt;/p&gt;&#xA;" OwnerUserId="9975" LastEditorUserId="27" LastEditDate="2014-08-13T16:18:39.927" LastActivityDate="2014-08-13T16:20:34.650" Title="Gradient descent on the PDF of the multivariate normal distribution" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14376" PostTypeId="2" ParentId="14375" CreationDate="2014-08-13T16:20:34.650" Score="3" Body="&lt;p&gt;Start by simplifying your expression by using the fact that the log of a product is the sum of the logarithms of the factors in the product.  The resulting expression is a quadratic form that is easy to differentiate.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-08-13T16:20:34.650" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14377" PostTypeId="2" ParentId="14368" CreationDate="2014-08-13T20:45:55.583" Score="2" Body="&lt;p&gt;The reason that Python code is slower than the equivalent Matlab code is usually that in Matlab, more computations are carried out by libraries that are &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;written in a lower-level language, and&lt;/li&gt;&#xA;&lt;li&gt;heavily optimized.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Luckily, you can do the same in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, Matlab is linked against (and bundles) Intel's hand-tuned MKL library, while Python (that is, NumPy/SciPy) usually links against the generic system BLAS/LAPACK. With a bit of wrangling (and after paying Intel for a license), you can build NumPy/SciPy yourself so that it calls MKL instead; after that, you get roughly the same performance for anything that relies mostly on BLAS/LAPACK calls. You can do the same with &lt;a href=&quot;https://github.com/xianyi/OpenBLAS&quot; rel=&quot;nofollow&quot;&gt;OpenBLAS&lt;/a&gt;, which has similar performance as MKL but is open source and easier to build NumPy with. With this setup, I get 0.66s for your Python code on my machine (which runs your Matlab code in 0.09s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The remaining performance difference is probably due, as you point out, to the fact that Matlab's version is implemented as a MEX file (i.e., written in C and pre-compiled). There are several options to do the same thing in Python, e.g., &lt;a href=&quot;http://cython.org&quot; rel=&quot;nofollow&quot;&gt;Cython&lt;/a&gt;. As awesomebytes so eloquently suggested, you might want to avoid Spectrum (which is no longer actively developed) and try the correlation function in &lt;code&gt;scipy.signal.correlate&lt;/code&gt; if you don't want to get your hands dirty with C. (The LD recursion is implemented in &lt;code&gt;scipy.linalg.solve_toeplitz&lt;/code&gt;). You might also want to take a look at &lt;a href=&quot;http://statsmodels.sourceforge.net&quot; rel=&quot;nofollow&quot;&gt;statsmodel&lt;/a&gt; and &lt;a href=&quot;http://nipy.org/nitime/&quot; rel=&quot;nofollow&quot;&gt;nitime&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-08-13T21:29:05.590" LastActivityDate="2014-08-13T21:29:05.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14378" PostTypeId="1" AcceptedAnswerId="14380" CreationDate="2014-08-14T00:06:31.500" Score="3" ViewCount="1033" Body="&lt;p&gt;I get a distance map output after using a &lt;a href=&quot;http://chintaksheth.wordpress.com/2013/07/09/fast-marching-method/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast Marching Method&lt;/a&gt;. The PDE involved is the Eikonal equation which take the form : $$\begin{cases}&#xA;        c(x).|\nabla u| =1\\&#xA;        u(x) =\phi(x)&#xA;        \end{cases} $$ where c is a speed function and u the unknown distance/time we compute. Below an example of a 2D distance map (in blue the inital points and in red the further points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tJTrz.png&quot; alt=&quot;Distance map example&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given 2 points in the map I would like to show a simple/shortest path between them. I have tried using a naive algorithm which is not very good since I can have complicated problems with obstacles, multiple sources, different speeds.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;find_path(point A, point B)&#xA;{&#xA;  point current_point = B;   // B is ending point&#xA;  while(current_point is not A)  &#xA;  {&#xA;    new_current_point is the the neighbor of current_point with the minimum value   // I use a 5 point stencil&#xA;    current_point = new_current_point&#xA;    tag every current_point found to show the path&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My data structure is an uni dimensional row major array (I am coding in C) representing distance values in a 2D regular grid.&#xA;I think gradient descent might be a good solution but I have difficulties finding a simple algorithm.&#xA;Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&#xA;I join an article &lt;a href=&quot;http://www.researchgate.net/profile/Santiago_Garrido2/publication/257343885_Application_of_the_fast_marching_method_for_outdoor_motion_planning_in_robotics/links/0c960526f78d5b164a000000&quot; rel=&quot;nofollow noreferrer&quot;&gt;Application of the fast marching method for outdoor&#xA;motion planning in robotics&lt;/a&gt; (starting page 12) which use a gradient descent in order to compute that. I have already seen that other papers but I cannot see how it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&#xA;I am trying to add a backtracking algorithm to my naïve one but I have troubles putting it in applications. I feel like I am doing a brute force and this is not what I want to do. If someone has an idea on how to change my naive algorithm.&lt;/p&gt;&#xA;" OwnerUserId="6804" LastEditorUserId="6804" LastEditDate="2014-08-22T13:15:07.153" LastActivityDate="2014-08-22T13:15:07.153" Title="Finding shortest path in a time/distance map" Tags="|algorithms|finite-difference|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14379" PostTypeId="2" ParentId="14378" CreationDate="2014-08-14T08:10:41.383" Score="0" Body="&lt;p&gt;This can be solved using some variant of &lt;a href=&quot;http://en.wikipedia.org/wiki/Dijkstra%27s_algorithm&quot; rel=&quot;nofollow&quot; title=&quot;Dijkstra&amp;#39;s algorithm&quot;&gt;Dijkstra's algorithm&lt;/a&gt;, such as the &lt;a href=&quot;http://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow&quot;&gt;A* algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2014-08-14T08:10:41.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14380" PostTypeId="2" ParentId="14378" CreationDate="2014-08-14T09:50:18.120" Score="4" Body="&lt;p&gt;If you already have a distance map $T(x)$ that describes the distance (with respect to a given metric $d$) from a given point $x_0$ to any other point $x$ in your domain (i.e., $d(x_0,x) = T(x)$), then the shortest path from $x_0$ to any point $y$ -- called the &lt;em&gt;geodesic&lt;/em&gt; between $x_0$ and $y$ -- can be computed by backtracking: Set $X(0) = y$ and then solve (e.g., by explicit Euler) the gradient flow&#xA;$$ \dot X(s) = -\nabla T(X(s)), \qquad s&amp;gt;0,$$&#xA;where $\nabla T$ is the gradient of $T$, until for some $S&amp;gt;0$ you have $X(S) = x_0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your algorithm can be seen as a space-discrete variant of this, but it will only ever compute the shortest path from &lt;code&gt;B&lt;/code&gt; to the &quot;bluest&quot; point in your distance map -- if that is not &lt;code&gt;A&lt;/code&gt;, it will not terminate in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note that $T$ depends on you initial point $x_0$, and has to be recomputed for each initial point. There are more efficient methods for computing all possible pairs of geodesics at once; this is sometimes referred to &lt;em&gt;all-to-all geodesics&lt;/em&gt;, see, e.g., &lt;a href=&quot;http://www.ias-iss.org/ojs/IAS/article/viewFile/891/789&quot; rel=&quot;nofollow&quot;&gt;http://www.ias-iss.org/ojs/IAS/article/viewFile/891/789&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-08-14T16:45:13.523" LastActivityDate="2014-08-14T16:45:13.523" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="14381" PostTypeId="1" AcceptedAnswerId="14394" CreationDate="2014-08-14T12:59:29.770" Score="5" ViewCount="1209" Body="&lt;p&gt;Is there a python library that would keep track of uncertainty in measured data? i.e. if I put in a figure of a±b is there an easy way to track the propagation of error through calculations.&lt;/p&gt;&#xA;" OwnerUserId="3939" LastActivityDate="2014-08-15T19:03:23.257" Title="Measurement error library" Tags="|python|libraries|error-estimation|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14383" PostTypeId="2" ParentId="14381" CreationDate="2014-08-14T14:08:25.523" Score="2" Body="&lt;p&gt;It sounds like what you may need is a library that supports &lt;a href=&quot;http://en.wikipedia.org/wiki/Interval_arithmetic&quot; rel=&quot;nofollow&quot;&gt;interval arithmetic&lt;/a&gt;.  A cursory google search reveals the following library:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://code.google.com/p/pyinterval/&quot; rel=&quot;nofollow&quot;&gt;Pyinterval&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others have suggested, it seems that the pythonhosted.org package &quot;&lt;a href=&quot;https://pythonhosted.org/uncertainties/&quot; rel=&quot;nofollow&quot;&gt;uncertainties&lt;/a&gt;&quot; also accomplishes this task as well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that interval computations may yield overly conservative bounds on the resulting computations, since dependencies between intervals are not taken into consideration.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-08-14T19:00:44.603" LastActivityDate="2014-08-14T19:00:44.603" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14384" PostTypeId="1" CreationDate="2014-08-14T14:24:39.177" Score="6" ViewCount="701" Body="&lt;p&gt;While continuously reading about &lt;a href=&quot;http://en.wikipedia.org/wiki/Dynamic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dynamic Programming&lt;/a&gt; I have a problem, implementing it in a practical application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume we want to optimize our way to school which we go daily by bicycle. Because the way is a bit hilly in addition to resistance due to friction $F_F$ adn air $F_A$ we have also a force dependent on the grade at our current position $F_G(x)$. So our equation might look like this&#xA;$$m_{ub}\ddot x = F_D - F_F - F_A(\dot x) - F_G(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $m_{ub}$ is the mass of us + bicycle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;We know&lt;/strong&gt; that we want to be at school $x_S = 5 km$ at $t_S = 20 min$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;We want to know&lt;/strong&gt; how to do that with minimum energy $E = \int F_D \; dx$. So we want to find a control vector $\bar F_D$ along this way which holds values for accelerating or decelerating our bike so that the total energy required for getting to school is minimal. The discretized problem might look like this &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wLQ4z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta x = 0.5km \quad x_i = i * \Delta x \quad i = 0, 1 \dots N_i\\ &#xA;\Delta v = 5km/h \quad v_j = j * \Delta v \quad j = 0, 1 \dots N_j\\ $$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Recursion&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The first step is the recursion, calculating the required force $F_D$ for all possible transitions $x_i \rightarrow x_{i-1}$ starting from $x_S$ for each $dv_{i} = v_{j, i} - v_{j, i-1}$. So in this step we are going from $x_S$ to $x_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some states will not be hit, for instance our stregth is limited to $ F_D &amp;lt; F_{max}$ and also our brakes won't do more than $ F_D &amp;gt; F_{brake} \quad (F_{brake} &amp;lt; 0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then at the end of this step we have a bunch of transitions with the according transition forces and times &#xA;$$\Delta F_{i,j \rightarrow i+1,j} \\ \Delta t_{i,j \rightarrow i+1,j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also know the energy for each transition $E_{i,j \rightarrow i+1,j}$ since we assumend $F_D$ to be constant along $\Delta x$.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Finding Optimum&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The big question know is: How do we find the optimal path through these transitions which fulfills the constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E = min \sum_{i} E_{i \rightarrow i+1,j} \\&#xA;\underset{i}{\sum} \Delta t \le t_S$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 1:&lt;/strong&gt; We can run through $i = 0, 1 \dots N_i$ taking the transition with the lowes value for energy. If we notice, that we run out of time, we accelerate making sure we get to school in time. BUT: This way, our algorithm will choose the cheapest transitions until time is short being forced to choose very expensive ones later on. So that won't give us the optimal path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 2:&lt;/strong&gt; We could simply check all paths, adding up the transition times of each and throw away those with a total time greater than $t_S = 20 min$ which we aimed at. But this would apparently be a brute force method because at each position $x_i$ we can choose about $N_j$ directions (velocities), so that the complexity would be basically&#xA;$$O(N_i^{N_j})$$&#xA;So for our little example that'd result in approximately $10^6 = 1,000,000$ paths. Is that right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now what are the actual steps to take to solve the problem in finite time?&lt;/strong&gt; How do we find the path of minimum energy for a required time $t_S$? Obviously these constraints contradict each other. How to heed them both?&lt;/p&gt;&#xA;" OwnerUserId="9986" LastActivityDate="2015-12-10T17:38:30.683" Title="Finding optimal velocity profile using Dynamic Programming" Tags="|algorithms|constrained-optimization|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14385" PostTypeId="2" ParentId="14384" CreationDate="2014-08-14T15:20:04.123" Score="0" Body="&lt;p&gt;You're right, the problem is harder than my first comment suggested. I think the trick is to expand the discretized state space to a third dimension, so that the state is (time, position, velocity) not just (position, velocity). Then you want to end in a state (time &amp;lt;= 20min, position = 5km, velocity = don't care) such that the path to that state has minimal energy usage. The dynamic programming recursion should work after this change.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-08-14T15:20:04.123" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14386" PostTypeId="1" CreationDate="2014-08-14T15:31:54.617" Score="1" ViewCount="254" Body="&lt;p&gt;Since upgrading to Octave 3.8 I have been unable to export plots with LaTeX labels. When the figure is displayed the labels render fine but when I try to export the figure using the &lt;code&gt;print&lt;/code&gt; command the labels come out as a raw string.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = 0:0.01:(2*pi);&#xA;y = sin(x.^2 + cos(x)).^3;&#xA;plot(x,y)&#xA;xlabel('x')&#xA;ylabel('sin^3(x^2 + cos(x))')&#xA;print('test1.png')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;produces,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/0dO6x.png&quot; alt=&quot;screenshot of figure.&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and saves,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/YTu7U.png&quot; alt=&quot;exported figure with mangled labels&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to fix this or hack around it?&lt;/p&gt;&#xA;" OwnerUserId="9987" LastActivityDate="2014-08-14T15:31:54.617" Title="Printing a plot in Octave-3.8.1 breaks LaTeX labels" Tags="|visualization|octave|" AnswerCount="0" CommentCount="4" ClosedDate="2014-08-14T17:14:29.583" ContentLicense="CC BY-SA 3.0" />
  <row Id="14387" PostTypeId="1" AcceptedAnswerId="14393" CreationDate="2014-08-14T16:41:41.413" Score="0" ViewCount="257" Body="&lt;p&gt;I am finding minimum of the potential function $f=f_1+f_2$, where $f_i: \mathbb{R}^3\to\mathbb{R}$. I was about to use Levenberg-Marquardt as the quick starting point, since it is already implemented in Eigen which I use. For a reason I don't understand, it requires the dimension of the solution ($\mathbb{R}$ in my case) to be higher or equal to the dimension of the searched space ($\mathbb{R}^3$ here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The functions are well-behaved, with one global minimum, have continuous derivatives (which are to be computed numerically, for now), and a good initial starting solution is provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the algorithm to recommend here, for the initial lazy implementation? Just Newton-Raphson with numerically computed gradient?&lt;/p&gt;&#xA;" OwnerUserId="9988" LastEditorUserId="1804" LastEditDate="2014-08-15T08:40:23.493" LastActivityDate="2014-08-17T12:35:05.240" Title="Numerical minimization of scalar-valued function in 3d" Tags="|convex-optimization|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14388" PostTypeId="2" ParentId="14381" CreationDate="2014-08-14T17:55:54.707" Score="4" Body="&lt;p&gt;The two basic approaches I can think of are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;interval arithmetic&lt;/li&gt;&#xA;&lt;li&gt;something like polynomial chaos expansions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm sure there are other approaches out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In brief, interval arithmetic redefines common arithmetic operations (addition, subtraction, multiplication, etc.) over intervals, so that given an interval $I$, a function $f$, and the interval extension $F$ of $f$, $f(I) \subset F(I)$. In other words, the image of the function is always contained in the interval extension, and the interval extension may drastically overestimate the bounds of your function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paul is right that naive implementations of interval arithmetic may yield overly conservative bounds; the &quot;dependency problem&quot; will also yield overly conservative bounds, even if the implementation of interval arithmetic is state-of-the-art.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Polynomial chaos expansions instead treat uncertain quantities as random variables, and decompose these random variables (and thus, their probability density functions) into the sum of a collection of standard random variables (for instance, with a uniform probability density function, a Gaussian probability density function, etc.) scaled by coefficients. The random variables are selected so that their probability density functions are orthogonal to each other with respect to a weighted inner product, so these expansions look a lot like Fourier series, Chebyshev series, and so on. Propagating this expansion through your calculations will give you a probability density function for your output, and is another way to propagate uncertainty.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-14T17:55:54.707" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14389" PostTypeId="1" CreationDate="2014-08-14T19:18:24.693" Score="8" ViewCount="264" Body="&lt;p&gt;I'm entering my 3rd year of my PhD program, and up until now my research code (numerical inverse problems/medical imaging/image processing/etc) consists mostly of disorganized MATLAB scripts and functions, with a few MEX'd C++ and CUDA routines thrown in for some added speed here and there.  Working with MATLAB has been (mostly) enjoyable, and I probably won't give it up completely, however:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This summer, I've been working on a 'big' code at a national lab, and I'm starting to see the beauty in a well oiled, object oriented, version controlled code with a good use of gmake and other computer science toys.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this: to what extent should I try to incorporate these tools into my research code?  How much time should I spend &quot;planning&quot; the structure and implementation of my code, or should I stop thinking about it and just write good routines?  I feel as though I should be developing a well planned open source code base as a product of my dissertation, for the experience and CV cred, but I'm not sure how to navigate this process.  Any tips, book/article/website recommendations, etc?&lt;/p&gt;&#xA;" OwnerUserId="3303" LastActivityDate="2014-08-16T16:26:57.940" Title="How to plan &amp; incorporate project management tools into research code" Tags="|open-source|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14390" PostTypeId="2" ParentId="14389" CreationDate="2014-08-14T20:50:13.163" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is this: to what extent should I try to incorporate these tools into my research code?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Only as much as you feel will pay off for whatever you're trying to do. If you're mostly doing MATLAB scripts, version control and maybe unit testing are going to be all you need. If you have MEX files, it's probably good to have a Makefile that compiles them, if only for your own sanity, since typing in sequences of compile commands each time you want to build the MEX files is fragile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of the reason these workflow tools get used and incorporated into large projects is because the investment in these tools pays off for their goals. As an example, for one-off research code supporting your thesis, it's probably not worth using a cross-platform build system like CMake. For a library whose stated purpose is to be cross-platform compatible, like, say, Elemental, it makes more sense to invest time in using a cross-platform build system.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How much time should I spend &quot;planning&quot; the structure and implementation of my code, or should I stop thinking about it and just write good routines?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It depends on how familiar you are with the problem you're trying to solve, appropriate algorithms, data structures, programming practices, and so on. Much like outlining and prewriting are helpful for journal articles, a certain amount of brainstorming and pseudocoding is helpful for writing well-structured code. I like the Pseudocode Programming Process in Code Complete, by Steve McConnell; he also includes some references and guidelines in terms of how much time should be spent on the design phase, depending on the type of project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Writing throwaway versions of routines and experimenting with short bits of code is also really helpful. A common aphorism in software development is that you'll always throw away at least one version of your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tend to believe that &quot;agile&quot; development practices tend to work best with most scientific software development, based on Greg Wilson's work at Software Carpentry (disclaimer: I have volunteered with them in the past). Broadly, &quot;agile&quot; means you should set goals you think you'll reach in a short period of time (say, a couple days, a week, maybe a month at most), plan out how to reach those goals by doing some pseudocoding and designing, and then write  code, and repeat. Short cycles will help you react to changes, such as when your adviser decides that he wants you to extend your work in ways you haven't anticipated.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I feel as though I should be developing a well planned open source code base as a product of my dissertation, for the experience and CV cred, but I'm not sure how to navigate this process. Any tips, book/article/website recommendations, etc?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That all depends on what you want to do. For positions involving software development, developing an open source code base is helpful, because it's something you can post on GitHub and point to. That said, if you want to make it a software package people will use, you're going to have to spend some time maintaining it; you may not want to do that. Contributing your research code to relevant existing projects can also be a really good option. Companies seem to want a mix of both. If you can contribute to other people's code, it shows you're a team player and that you can read other people's code and still do something useful with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of references, &lt;a href=&quot;http://software-carpentry.org/bib/bib.html&quot; rel=&quot;nofollow&quot;&gt;Software Carpentry's reference reading list&lt;/a&gt; is well-geared towards scientists, and if you want to dig deeper into software engineering practice, Code Complete (see the list in the previous link) has further references that are starting to become a little dated, but are useful to look up. The papers they put out on best practices in scientific computing are also helpful&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Software Carpentry's &lt;a href=&quot;http://software-carpentry.org/lessons.html&quot; rel=&quot;nofollow&quot;&gt;lessons&lt;/a&gt; are helpful, too. They're Python-centric when it comes to programming, so you might take that with a grain of salt, but the version control parts are worth looking at.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-14T20:50:13.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14391" PostTypeId="2" ParentId="14369" CreationDate="2014-08-14T22:13:33.633" Score="4" Body="&lt;p&gt;Probabily the CG fails to converge because your problem is ill conditioned, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Condition_number&quot; rel=&quot;nofollow&quot;&gt;condition number&lt;/a&gt; of your matrix is too large. For SPD matrix A you can get the condition number calculating the eigenvalues&lt;/p&gt;&#xA;&#xA;&lt;p&gt;codition_number = largest_eigenvalue/smallest_eigenvalue&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ill conditioned means that small changes on some entries of the matrix could give large changes in the result when solving the system of equations (in a well conditioned matrix a small change in any entry of the matrix gives a small change in the result).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To reduce the condition number of the problem you can use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Preconditioned_conjugate_gradient_method#The_preconditioned_conjugate_gradient_method&quot; rel=&quot;nofollow&quot;&gt;preconditioned CG&lt;/a&gt;, there are several preconditioners. The Jacobi preconditioner (also known as diagonal preconditioner) is easy to implement but not robust.&lt;/p&gt;&#xA;" OwnerUserId="9992" LastActivityDate="2014-08-14T22:13:33.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14392" PostTypeId="2" ParentId="14389" CreationDate="2014-08-15T01:38:28.240" Score="4" Body="&lt;p&gt;I'd consider these questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you want your code to become a fairly general purpose code that you can reuse later, or is this just a bit of code that you're using for one research project or paper that you don't plan to reuse?  If you have no plans to reuse the code, then it's probably not worth spending time and effort improving on the code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Have you learned enough about the algorithms and data structures that you're using that you have confidence that you'll want to keep using these algorithms and data structures?  If not, then it might be premature to freeze your research into a stable code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you want to share the code, get others to make use of the code, and ultimately to get other people help develop the code further?  As a researcher, you should understand that this is one of the most effective ways of making sure that your code has an impact on your field of research.  However, to get this started you have to supply a reasonably robust code and you'll want to release it under an open source license so that others can make use of the code in their own projects.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2014-08-16T16:26:57.940" LastActivityDate="2014-08-16T16:26:57.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14393" PostTypeId="2" ParentId="14387" CreationDate="2014-08-15T08:39:28.213" Score="3" Body="&lt;p&gt;Levenberg-Marquardt is for nonlinear least-square solutions of an (underdetermined) system of equations (i.e., solving $f(x)=0$ for $f:\mathbb{R}^n\to \mathbb{R}^m$, $m&amp;lt;n$), not for minimizing a functional (i.e., a scalar-valued function -- if the function is not scalar-valued, you can't minimize it, since there's no sensible order on $\mathbb{R}^m$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your tag, I assume that &quot;well-behaved&quot; means (strictly) convex, so standard methods should work:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have sufficiently accurate second derivatives (either explicitly or as a procedure that evaluates them given a point and direction), Newton-Raphson would be my choice (possibly with a trust-region globalization a la &lt;a href=&quot;http://www.ii.uib.no/~trond/publications/papers/trust.pdf&quot; rel=&quot;nofollow&quot;&gt;Steihaug&lt;/a&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Otherwise (i.e., you need to approximate derivatives by taking finite differences), try a quasi-Newton method such as BFGS (combined with an Armijo line search); there's a very light-weight &lt;a href=&quot;https://github.com/PatWie/LBFGSB&quot; rel=&quot;nofollow&quot;&gt;C++ implementation based on Eigen&lt;/a&gt; (but without line searches).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One thing to watch out for is that methods that use the function values for globalization (trust-region or backtracking line search) can be rather sensitive to errors in derivatives, since they assume the gradient (or the Newton direction) is in fact a descent directions, which might not be the case for the approximate gradient (or Newton direction). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; If at all feasible, it's always worth the effort to compute an exact gradient (even for a lazy implementation). For the Hessian, it's not so critical, but if you don't have exact second derivatives, use BFGS instead of computing second-order finite-difference approximations.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-08-15T09:07:44.673" LastActivityDate="2014-08-15T09:07:44.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14394" PostTypeId="2" ParentId="14381" CreationDate="2014-08-15T09:36:35.090" Score="2" Body="&lt;p&gt;The usual way used by scientist for tracking uncertainties is to keep track of the standard deviation of quantities, so I will assume that this is what you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_uncertainty_propagation_software#Comparison&quot; rel=&quot;nofollow&quot;&gt;a few programs&lt;/a&gt; that can do this automatically so that you don't have to implement &lt;a href=&quot;http://en.wikipedia.org/wiki/Error_propagation&quot; rel=&quot;nofollow&quot;&gt;formulas that are inconvenient to handle&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just had another look at the current offer, and I would recommend the &lt;a href=&quot;http://pythonhosted.org//uncertainties/&quot; rel=&quot;nofollow&quot;&gt;uncertainties package&lt;/a&gt; too. You can perform arbitrary calculations with numbers with uncertainties as if they were simple floats, and the results automatically contains an uncertainty, propagated correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I'm the author of this package.&lt;/p&gt;&#xA;" OwnerUserId="9996" LastActivityDate="2014-08-15T09:36:35.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14395" PostTypeId="2" ParentId="14369" CreationDate="2014-08-15T13:11:18.263" Score="7" Body="&lt;p&gt;If your matrix is symmetric, positive definite, the CG method may converge slowly, but it converges for $n\to\infty$. The only reason it does not converge on a computer are round-off errors, in particular if the condition number of the matrix, the quotient of largest and smallest eigenvalue is large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experience is, that with double precision arithmetic, even with moderate condition numbers convergence stalls at around $10^{-12}$ to $10^{-14}$ of the original error.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2014-08-15T13:11:18.263" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14396" PostTypeId="2" ParentId="14336" CreationDate="2014-08-15T17:23:39.277" Score="4" Body="&lt;p&gt;I think the main issue that makes this tricky is that trying to&#xA;evaluate the power series for the hypergeometric function is&#xA;numerically unstable due to catastrophic cancellation for&#xA;one negative parameter and function value $\ll1$. Otherwise, having real parameters, and a real argument in $[0,1]$ should be fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the DLMF entry for hypergeometric functions, you will&#xA;find &lt;a href=&quot;http://dlmf.nist.gov/15.5.E11&quot; rel=&quot;nofollow&quot;&gt;a relation&lt;/a&gt; between contiguous&#xA;functions, $F(a,b;c;z)$ and $F(a\pm1,b;c;z)$:&#xA;$$ f_a = \frac{F(a+1,b;c;z)}{F(a,b;c;z)} $$&#xA;$$ (c-a) \frac1{f_{a-1}} + (2a-c+(b-a)z) + a(z-1)f_a = 0. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for large negative $a$, you can find $\bar a=a-\lfloor a\rfloor\geq0$&#xA;and then relate $f_a$ to $f_{\bar a}$, which are separated in the above recurrence relation by integer distance $-\lfloor a\rfloor$. This algorithm takes time&#xA;linear in $|a|$, so it would work well for $|a|&amp;lt;1000$, but poorly for&#xA;$|a|$ much larger than $10^6$, say. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\bar a$ is positive, evaluating $f_{\bar a}$ is a lot simpler since it involves&#xA;hypergeometric function of real variable with all positive parameters&#xA;(so no cancellation). In particular, you can do with power series (or&#xA;GSL for that matter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some python code that I wrote to test this idea. (It doesn't work for some values of $a, c$ that cause division by zero, but that's straightforward to fix; you can use limits for example.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from mpmath import *&#xA;&#xA;def hsum(a, b, c, x):&#xA;  ans, ansAbs, t, i = mpf(1), mpf(1), mpf(1), 0&#xA;  while abs(t) / abs(ans) &amp;gt; mp.eps():&#xA;    t *= (a + i) * (b + i) / ((c + i) * (i+1)) * x&#xA;    i += 1&#xA;    ans += t&#xA;    ansAbs += abs(t)&#xA;  print(&quot;hsum(a=%f): %f (%f)&quot; % (a, ans, ansAbs / abs(ans)))&#xA;  return ans&#xA;&#xA;def h3(a, b, c, x):&#xA;  a, b, c, x = mpf(a), mpf(b), mpf(c), mpf(x)&#xA;&#xA;  ans0 = hyp2f1(a+1, b, c, x) / hyp2f1(a, b, c, x)&#xA;  if a &amp;gt;= 0:&#xA;    return ans0&#xA;&#xA;  def f(a):&#xA;    if a &amp;gt;= 0: return hsum(a+1, b, c, x) / hsum(a, b, c, x)&#xA;    return - (c-a-1) / ((2*(a+1)-c+(b-a-1)*x) + (a+1)*(x-1)*f(a+1))&#xA;&#xA;  def f2(a):&#xA;    n = int(floor(a))&#xA;    a0 = a - n&#xA;    ans = hsum(a0+1, b, c, x) / hsum(a0, b, c, x)&#xA;    def alpha(a): return -(c-a-1)/((a+1) * (x-1))&#xA;    def beta(a): return (2*(a+1)-c+(b-a-1)*x) / ((a+1)*(x-1))&#xA;    aa = a0&#xA;    for i in range(0, -n):&#xA;      aa -= 1&#xA;      ans = alpha(aa) / (beta(aa) + ans)&#xA;    return ans&#xA;&#xA;  ans1 = f2(a)&#xA;&#xA;  if (abs(a) &amp;lt;= 200):&#xA;    assert(abs(f2(a)/f(a)-1) &amp;lt; eps() * 1000)&#xA;&#xA;  print(&quot;a %s: %s&quot; % (a, float(abs(ans1/ans0-1))))&#xA;  return ans1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;P.S. You said in a comment &quot;I am computing the numerator and the&#xA;denominator, which are very small or very large, and thus their ratio&#xA;is not very accurate&quot;. This isn't really right. The numerical stability of&#xA;evaluating the expression $a/b$ can be found by computing the&#xA;condition number of the function $(a,b)\mapsto a/b$. It is&#xA;$$ \left|\frac{d\log(a/b)}{d\log a}\right| = 1, \qquad&#xA;\left|\frac{d\log(a/b)}{d\log b}\right| = 1. $$&#xA;Thus the computation of $a/b$ should be well-conditioned, regardless of their values (assuming no overflow/underflow, which is not what's going on for $|a|,|b|,|c|\approx 100$ anyway). You might be&#xA;thinking of $a-b$, which does have a singular condition number at $a\approx&#xA;b$: &#xA;$$ \left|\frac{d\log(a-b)}{d\log a}\right| = \frac{|a|}{|a-b|}. $$&#xA;This doesn't happen to $a/b$. The numerical problems here are of a&#xA;different sort.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-08-15T18:27:11.013" LastActivityDate="2014-08-15T18:27:11.013" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14397" PostTypeId="2" ParentId="14381" CreationDate="2014-08-15T19:03:23.257" Score="0" Body="&lt;p&gt;If the errors are quite small, and you don't need exact conservative estimates, you can estimate such uncertainties with an algorithmic differentiation library, such as &lt;code&gt;ad&lt;/code&gt; (which I never used, but found by searching). Here is an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from ad import adnumber&#xA;import ad.admath as adm&#xA;import math as m&#xA;&#xA;x = adnumber(1.0)&#xA;x_err = 0.01&#xA;&#xA;# Error in starting value for Newton's method&#xA;z = adnumber(1)&#xA;&#xA;# Solve the equation y + sin(y) - x = 0 for y&#xA;y = x - adm.sin(x) + z&#xA;while abs(y + m.sin(y) - x) &amp;gt; 1e-8:&#xA;  y = y - (y + adm.sin(y) - x) / (1 + adm.cos(y))&#xA;  print((y, y.d(x)))&#xA;&#xA;print(&quot;Value: %s\nEstimated error: %s\nSensitivity to starting value: %s&quot; % \&#xA;      (y.x, abs(y.d(x)) * x_err, abs(y.d(z)/y.x)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(ad(0.3912318441940903), 0.48321150365251125)&#xA;(ad(0.5094172293334197), 0.5309477254386529)&#xA;(ad(0.5109731138269538), 0.5341096893266438)&#xA;(ad(0.510973429388556), 0.5341113161301007)&#xA;Value: 0.510973429388556&#xA;Estimated error: 0.005341113161301007&#xA;Sensitivity to starting value: 7.680410775256825e-13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-08-15T19:03:23.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14399" PostTypeId="2" ParentId="1234" CreationDate="2014-08-15T21:19:22.583" Score="1" Body="&lt;p&gt;The complex eigensolution approach is mathematically correct, but it does more work than necessary. Unfortunately, the improved approach I'm about to describe cannot be implemented with LAPACK calls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look at R. C. Ward and L. J. Gray, ACM Trans. Math. Soft. 4, 278, (1978). This describes the software that is available in TOMS algorithm 530, and which you can download from netlib. This describes how to factor the skew symmetric matrix $X$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X = U D U^T$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is real orthogonal and $D$ is real skew-symmetric and block diagonal. The diagonal subblocks are either $2\times 2$ or $1\times 1$. Because it is block diagonal, you can exponentiate each subblock separately. The $1\times 1$ blocks are zero, and $\exp(0)=1$, so those are trivial.  The $2\times 2$ subblocks are done with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\exp \begin{pmatrix} 0 &amp;amp; -t \\ t &amp;amp; 0 \end{pmatrix} = &#xA;\begin{pmatrix} \cos t &amp;amp; -\sin t  \\ \sin t &amp;amp; \cos t \end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exponential matrix that you want is then given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\exp(X) = U \exp(D) U^T$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used this approach in my quantum chemistry codes for several decades and I have never had any problems with any of the software involved.&lt;/p&gt;&#xA;" OwnerUserId="10002" LastEditorDisplayName="user10435" LastEditDate="2015-03-08T05:00:34.443" LastActivityDate="2015-03-08T05:00:34.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14401" PostTypeId="1" AcceptedAnswerId="14403" CreationDate="2014-08-15T23:47:12.440" Score="2" ViewCount="3444" Body="&lt;p&gt;I am having an issue with the implementation of NLOPT in Python. My objective is to minimize a somewhat complicated Maximum Likelihood function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My function is called mle and there are 6 parameters to estimate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the gradient to this MLE is not trivial, so I decided to turn to a numerical gradient function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def numgrad(f, x, step=1e-6):&#xA;    &quot;&quot;&quot;numgrad(f: function, x: num array, step: num) -&amp;gt; num array&#xA;&#xA;    Numerically estimates the gradient of a function f which takes an array as&#xA;    its argument.&#xA;    &quot;&quot;&quot;&#xA;    ary = len(x)&#xA;&#xA;    curr = x * sp.ones((ary, ary))&#xA;    next = curr + sp.identity(ary) * step&#xA;&#xA;    delta = sp.apply_along_axis(f, 1, next) - sp.apply_along_axis(f, 1, curr)&#xA;&#xA;    return delta / step&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then my implementation of NLOPT goes like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def myfunc(x, grad):&#xA;    if grad.size &amp;gt; 0:&#xA;        grad = numgrad(mle, [x[0], x[1], x[2], x[3], x[4], x[5]], step=1e-14)&#xA;    return mle([x[0], x[1], x[2], x[3], x[4], x[5]])&#xA;&#xA;opt = nlopt.opt(nlopt.LD_SLSQP, 6)&#xA;opt.set_lower_bounds([mmin, smin, ming, bmin, vmin, pmin]) #min bound for each of the param.&#xA;opt.set_upper_bounds([mmax, smax, maxg, bmax, vmax, pmax])&#xA;opt.set_min_objective(myfunc)&#xA;opt.set_xtol_rel(1e-15)&#xA;opt.maxeval=10000&#xA;x = opt.optimize([x1, x2, x3, x4, x5, x6])&#xA;minf = opt.last_optimum_value()&#xA;print &quot;optimum at &quot;, x[0], x[1], x[2], x[3], x[4], x[5]&#xA;print &quot;minimum value = &quot;, minf&#xA;print &quot;result code = &quot;, opt.last_optimize_result()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the issue is this .... the minimization process goes wayyy tooo fast. In matlab, it takes approx 1 hour and here in Python 12 seconds ... I don't get the same results in Matlab using fmincon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My feeling is that the code does not recognize the &lt;code&gt;opt.set_xtol_rel(1e-15)&lt;/code&gt; and &lt;code&gt;opt.maxeval=10000&lt;/code&gt; statements because even if I increase the number ... no change in the time process... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or the problem is elsewhere... what am I doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="9598" LastActivityDate="2014-10-03T10:47:46.953" Title="Maximum function evaluation with NLOPT in Python" Tags="|optimization|python|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14403" PostTypeId="2" ParentId="14401" CreationDate="2014-08-16T10:41:44.857" Score="2" Body="&lt;p&gt;You should essentially never estimate the gradient numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say the gradient is difficult to estimate. In general, if it is at all possible to get the gradient exactly you should do so, and use an appropriate algorithm (&lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms#Local_gradient-based_optimization&quot; rel=&quot;nofollow&quot;&gt;NLopt has several&lt;/a&gt;, the one you're using should be fine).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you cannot get the gradient exactly, NLopt features &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms#Local_derivative-free_optimization&quot; rel=&quot;nofollow&quot;&gt;several derivative free algorithm&lt;/a&gt;, which you could use instead and expect to get better results. I think this is probably the easiest solution to your problem, and would give you better results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The speed difference can easily be real however. Differences in optimization algorithm and the fact that python is generally faster than Matlab could explain the difference easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR: Change from nlopt.LD_SLSQP to nlopt.LN_BOBYQA .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-08-16T10:41:44.857" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14404" PostTypeId="2" ParentId="14168" CreationDate="2014-08-16T19:58:32.457" Score="1" Body="&lt;p&gt;If you can code on many core architectures such as GPUs, maybe this paper &lt;a href=&quot;http://www.sci.utah.edu/~csilva/papers/cgf.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;Fast 4-way parallel radix sorting on GPUs&quot;&lt;/a&gt; can interest you.&lt;/p&gt;&#xA;" OwnerUserId="6804" LastActivityDate="2014-08-16T19:58:32.457" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14405" PostTypeId="1" CreationDate="2014-08-16T21:23:52.587" Score="0" ViewCount="3586" Body="&lt;p&gt;I am working on a physics problem which requires obtaining the exact eigenvalues and eigenvectors of Hermitian and Unitary matrices numerically. Naturally I would like to ask the experts what are the most efficient libraries to diagonalize the referred matrices. In particular, I would like to diagonalize efficiently matrices of size 12870*12870. I have been using Mathematica for convenience, but I am open to use any language you might suggest. I also have access to a HPC cluster in which the diagonalization of a unitary matrix of size 12870*12870 takes around 7000 seconds in Mathematica using 12 cores. Can this time be improved? Note: The matrices are numeric (not symbolic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more precise, I diagonalized a random Hermitian matrix and a random Unitary matrix of size 12870 on the cluster using Mathematica. Here is the code I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;With[{mat = RandomReal[1, 12870 {1, 1}]}, AbsoluteTiming[{vals, vecs} = Eigensystem[mat+Transpose[mat]];]] &#xA;&#xA;With[{matu = Transpose[vecs].(Exp[I*vals]*Conjugate[vecs])}, AbsoluteTiming[Eigensystem[matu];]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results were: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[1]:= Out[1]= {265.369827, Null} &#xA;In[2]:= Out[2]= {7042.662152, Null} &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the unitary matrix diagonalization takes a lot more time. Can this be improved using other languages?&lt;/p&gt;&#xA;" OwnerUserId="10007" LastEditorUserId="10007" LastEditDate="2014-08-18T17:19:18.003" LastActivityDate="2014-08-19T12:05:43.917" Title="Most efficient library to diagonalize exactly large hermitian or unitary matrices" Tags="|c++|eigensystem|eigenvalues|computational-physics|mathematica|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14406" PostTypeId="2" ParentId="14405" CreationDate="2014-08-16T23:08:20.007" Score="0" Body="&lt;p&gt;If you have access to a HPC cluster and you want to work on matrix of larger size (12k^2 order is not much), I would suggest to use &lt;a href=&quot;http://www.netlib.org/lapack/lug/node48.html&quot; rel=&quot;nofollow&quot;&gt;Lapack routines&lt;/a&gt;. &#xA;This is a base/reference in numerical linear algebra available in Fortran or C.&lt;/p&gt;&#xA;" OwnerUserId="6804" LastActivityDate="2014-08-16T23:08:20.007" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14407" PostTypeId="2" ParentId="14359" CreationDate="2014-08-17T05:35:56.193" Score="2" Body="&lt;p&gt;You can use the identity C^T = (AB)^T = B^T A^T to use BLAS to compute on row-major arrays from C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, the overhead of transposition is &lt;em&gt;usually&lt;/em&gt; not a bottleneck compared to O(N^3) BLAS3 or LAPACK routines, but I should note that I am oft to point out counterexamples from my own research.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2014-08-17T05:35:56.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14408" PostTypeId="2" ParentId="2762" CreationDate="2014-08-17T05:37:26.350" Score="3" Body="&lt;p&gt;Just write the loop code and forget about BLAS here.  Adding two matrices is bandwidth-limited and the compiler will likely do as good a job as the BLAS implementation &lt;em&gt;in this case&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2014-08-17T05:37:26.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14411" PostTypeId="2" ParentId="14387" CreationDate="2014-08-17T12:35:05.240" Score="1" Body="&lt;p&gt;Well, one of my favorite algorithms to use for optimization of functions is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method&quot; rel=&quot;nofollow&quot;&gt;Nelder-Mead method&lt;/a&gt;.&#xA;It is a derivative-free method for nonlinear optimization. Depending on you functions it may not be the best method to optimize, but it should probably do the job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have access to matlab, it is implemented there under the name 'fminsearch'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also a small note, you say that your two functions both have a single global minimum - be careful, since their sum may not have one.&lt;/p&gt;&#xA;" OwnerUserId="9362" LastActivityDate="2014-08-17T12:35:05.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14412" PostTypeId="1" AcceptedAnswerId="14413" CreationDate="2014-08-17T12:52:40.930" Score="14" ViewCount="4418" Body="&lt;p&gt;I would like to use Runge-Kutta 8th order method (89) in a celestial mechanics / astrodynamics application, written in C++, using a Windows machine. Therefore I wonder if anyone knows a good library / implementation that is documented and free to use ? It is ok if it is written in C, as long as there aren't any compilation problems to be expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have found &lt;a href=&quot;http://www.mymathlib.com/diffeq/&quot; rel=&quot;noreferrer&quot;&gt;this library ( mymathlib )&lt;/a&gt;. The code seems ok, but I haven't found any information on licensing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help me by revealing some of the alternatives you might know and would suit my problem ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT :&lt;/strong&gt;&lt;br&gt;&#xA;I see that there aren't really that many C/C++ source codes available as I expected. Therefore a Matlab/Octave version would be ok too (still has to be free to use).&lt;/p&gt;&#xA;" OwnerUserId="10011" LastEditorUserId="10011" LastEditDate="2014-08-18T21:50:15.500" LastActivityDate="2020-07-19T18:14:28.650" Title="Looking for Runge-Kutta 8th order in C/C++" Tags="|c++|reference-request|libraries|c|runge-kutta|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14413" PostTypeId="2" ParentId="14412" CreationDate="2014-08-17T13:27:57.993" Score="10" Body="&lt;p&gt;Both &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Ordinary-Differential-Equations.html&quot; rel=&quot;noreferrer&quot;&gt;GNU Scientific Library (GSL) &lt;/a&gt; (C) and &lt;a href=&quot;http://www.boost.org/doc/libs/1_55_0/libs/numeric/odeint/doc/html/index.html&quot; rel=&quot;noreferrer&quot;&gt;Boost Odeint&lt;/a&gt; (C++) feature  8th order Runge-Kutta methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both are opensource, and under linux and mac they should be directly available from the package manager. Under windows, it will probably be easier for you to use Boost rather than GSL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GSL is published under the GPL license, and Boost Odeint under the Boost license. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: &#xA;Ok, Boost Odeint does NOT have the Runge-Kutta 89 method, only the 78, but it does provide a &lt;a href=&quot;http://headmyshoulder.github.io/odeint-v2/doc/boost_numeric_odeint/odeint_in_detail/steppers.html#boost_numeric_odeint.odeint_in_detail.steppers.custom_runge_kutta_steppers&quot; rel=&quot;noreferrer&quot;&gt;recipe&lt;/a&gt; for making arbitrary Runge-Kutta steppers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8th order methods are quite high however, and most likely overkill for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0771050X80900133&quot; rel=&quot;noreferrer&quot;&gt;Prince-Dormand&lt;/a&gt; refers to a specific kind of Runge-Kutta, and is not directly related to the order but the most common is 45. Matlabs ode45, which is their recommended ODE algorithm is a Prince-Dormand 45 implementation. This is the same algorithm as implemented in Boost Odeint &lt;a href=&quot;http://www.boost.org/doc/libs/1_53_0_beta1/boost/numeric/odeint/stepper/runge_kutta_dopri5.hpp&quot; rel=&quot;noreferrer&quot;&gt;Runge_Kutta_Dopri5&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="7522" LastEditDate="2014-08-17T18:21:03.073" LastActivityDate="2014-08-17T18:21:03.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14415" PostTypeId="1" AcceptedAnswerId="14416" CreationDate="2014-08-18T03:02:29.253" Score="4" ViewCount="1307" Body="&lt;p&gt;I need to know if there is some function in BLAS/LAPACK or some other Scientific Library that returns a non absolute sum of a vector/matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found the 'asum', but it returns only the absolute sum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="10013" LastActivityDate="2014-08-18T03:50:07.647" Title="BLAS/LAPACK Non absolute sum" Tags="|lapack|blas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14416" PostTypeId="2" ParentId="14415" CreationDate="2014-08-18T03:50:07.647" Score="5" Body="&lt;p&gt;There's nothing in BLAS or LAPACK that does this directly.  You could use the dot product function in the BLAS to take the dot product of your vector and a vector of all 1's.  However, it's probably more efficient to just write your own loop to compute the sum- this operation doesn't allow for any cache reuse so most compilers should be able to optimize the code to very nearly the highest possible performance.  In any case, it's unlikely that your code is spending much of its time computing such sums.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-08-18T03:50:07.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14419" PostTypeId="2" ParentId="14412" CreationDate="2014-08-18T06:47:26.733" Score="18" Body="&lt;p&gt;If you're doing celestial mechanics over long time scales, using a classical Runge-Kutta integrator will not preserve energy. In that case, using a symplectic integrator would probably be better. Boost.odeint also implements a 4th-order symplectic Runge-Kutta scheme that would work better for long time intervals. GSL does not implement any symplectic methods, as far as I can tell.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-18T06:47:26.733" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14420" PostTypeId="2" ParentId="14405" CreationDate="2014-08-18T06:56:04.440" Score="1" Body="&lt;p&gt;If you want to diagonalize significantly larger matrices on large HPC clusters, I would look at &lt;a href=&quot;http://libelemental.org/&quot; rel=&quot;nofollow&quot;&gt;Elemental&lt;/a&gt;, which is a state-of-the-art distributed dense linear algebra library. It's a better alternative than other libraries out there (e.g., ScaLAPACK, PLAPACK), and builds upon BLAS and LAPACK implementations.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-18T06:56:04.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14422" PostTypeId="2" ParentId="14148" CreationDate="2014-08-18T12:47:50.277" Score="5" Body="&lt;p&gt;&lt;strong&gt;Analytical problem :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what you are expecting is &lt;strong&gt;positive diffusion&lt;/strong&gt; : you want the $T_i$ values to spread over your domain as time passes to eventually reach $T_i(t\rightarrow \infty) = cte$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if $\alpha$ was a negative number, you would have what is called &lt;strong&gt;negative diffusion&lt;/strong&gt; : you'll have exactly the opposite i.e. the gradients will get greater through time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sign of $\alpha$ hence dictates the behaviour of your analytical solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;$\alpha &amp;lt; 0$ case :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, the numerical solution should have the same behaviour as the analytical solution. However, the finite difference theory assumes the solution to be smooth : if the solution features gradients that are too sharp, then your numerical method will not be able to handle them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have just said that in the case where $\alpha &amp;lt; 0$, the gradients grow greater with time. The error generated by the simulation will not be smeared out, as would be the case with positive diffusion $\alpha &amp;gt; 0$, but instead will be amplified. For that reason, if α&amp;lt;0, you know for sure your simulation is going to blow up at some point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;$\alpha &amp;gt; 0$ case :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\alpha &amp;gt; 0$, you are however not safe. If your time step is too large, your simulation will not be stable either. The stability condition $\Delta t &amp;lt; \frac{\Delta x ^2}{2 \alpha}$ indicates whether your numerical method has a chance of being stable or not. Note that it is a &lt;strong&gt;necessary&lt;/strong&gt; condition for your numerical method to be stable, not a &lt;strong&gt;sufficient&lt;/strong&gt; condition. Yet in practice, it turns out to be a very powerful tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the mesh Fourier number for a diffusive term can be defined as $\alpha \frac{\Delta t}{\Delta x^2}$. In practice it is more convenient to write the stability condition in terms of the mesh Fourier&#xA;$\alpha \frac{\Delta t}{\Delta x^2} &amp;lt; \frac{1}{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way you can see that the parameters of your simulation $\Delta t$, $\Delta x$ and $\alpha$ are all on the left hand side and $\frac{1}{2}$ is the critical value that must not be exceeded for the simulation to have a chance of being stable. In practice, the value for $\alpha$ is given by your problem and you will have chosen $\Delta x$ already. Hence, $\Delta t$ is the only parameter you can play with so that the stability condition on diffusion is observed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of the critical mesh Fourier number depends on the space and time discretisation you have chosen. Some time integrators have broader stability regions than others, hence they will allow larger mesh Fourier numbers. Practically speaking, this means you'd be able to choose larger time steps while still having a stable numerical method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;To summarise :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if $\alpha &amp;lt; 0$, you will have negative diffusion and your simulation will in any case not be stable.&lt;/li&gt;&#xA;&lt;li&gt;if $\alpha &amp;gt; 0$, your simulation might be stable... or it may not !&lt;/li&gt;&#xA;&lt;li&gt;the stability condition on diffusion (and the mesh Fourier number) helps you choose the time step $\Delta t$ for your numerical method to be stable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I recommend you make a dummy simulation and play with the parameters to see what happens. No need to waste time into programming something : a spreadsheet software is enough for your particular case.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit: partial rewrite of my answer to make it clearer&lt;/p&gt;&#xA;" OwnerUserId="10018" LastEditorUserId="10018" LastEditDate="2016-04-04T08:45:58.170" LastActivityDate="2016-04-04T08:45:58.170" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="14424" PostTypeId="2" ParentId="10968" CreationDate="2014-08-18T13:30:57.100" Score="4" Body="&lt;p&gt;If your gradients are inexact, they are no longer guaranteed to be descent directions, and so any algorithm that uses both functional values and gradients  (via line searches or trust regions) and thus relies on that property will run into problems sooner or later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Splitting methods, on the other hand, allow for inexact evaluation and can be combined with extrapolation and preconditioning to improve the convergence. The classical Chambolle-Pock algorithm (which is a variant of Douglas-Rachford splitting and can be interpreted as a proximal point algorithm) for your problem can be written as&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$x^k = x^{k-1} - \tau (W^T z^{k-1} + A^T y^{k-1})$&lt;/li&gt;&#xA;&lt;li&gt;$\bar x^k = 2x^k - x^{k-1}$&lt;/li&gt;&#xA;&lt;li&gt;$y^k = \frac{1}{1+\sigma}(y^{k-1}+\sigma (A \bar x^k-b))$&lt;/li&gt;&#xA;&lt;li&gt;$w^k = z^{k-1} + \sigma W\bar x^k$&lt;/li&gt;&#xA;&lt;li&gt;$z^k = \lambda \frac{w^k}{\max\{\lambda,|w^k|\}}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;where the step sizes should satisfy $\sigma\tau &amp;lt; \|A\|^2 +\|W\|^2$. Step 3 and 5 correspond to the proximal mappings of the $l^2$ norm and the $l^1$ norm, respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Preconditioners are discussed in &lt;a href=&quot;http://math.uni-graz.at/mobis/publications/SFB-Report-2014-002_2.pdf&quot; rel=&quot;nofollow&quot;&gt;this preprint by Bredies and Sun&lt;/a&gt;, where $K = (A,W)^T$. The case of inexact solution of the normal equations (which is not quite what you are asking about, but allows for errors to remain bounded from below) is discussed after Remark 2.2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, proximal point methods converge as long as the evaluation error of the proximal mapping remains summable (i.e., evaluation has to become exact sufficiently quickly as the iteration progresses), see for example &lt;a href=&quot;http://arxiv.org/pdf/1109.2415v2.pdf&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/pdf/1109.2415v2.pdf&lt;/a&gt; and the references therein.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2014-08-18T13:30:57.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14428" PostTypeId="1" CreationDate="2014-08-18T17:24:09.787" Score="0" ViewCount="23" Body="&lt;p&gt;I need a numerically stable way to compute the following ratio:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the parameters are real positive numbers, with $0 &amp;lt; a,b,c$ and $0 &amp;lt; x &amp;lt; 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I am using GSL's implementation of the hypergeometric function, but I keep getting under/over-flows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simplification that I can use? Or an aproximation scheme that remains accurate for a wide range of the parameters (say $0 &amp;lt; a,b,c &amp;lt; 1000$)?&lt;/p&gt;&#xA;" OwnerUserId="988" LastActivityDate="2014-08-18T17:24:09.787" Title="Compute hypergeometric function ratio: $\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$, for large positive $a,b,c$?" Tags="|floating-point|special-functions|" AnswerCount="0" CommentCount="4" ClosedDate="2014-08-18T18:41:37.047" ContentLicense="CC BY-SA 3.0" />
  <row Id="14429" PostTypeId="1" AcceptedAnswerId="14430" CreationDate="2014-08-18T18:07:57.643" Score="5" ViewCount="778" Body="&lt;p&gt;If you use a FEM (on the variational formulation), you can discretize some continuous eigenvalue problem, $$L u = \lambda u \ \ \text{on} \ \Omega,$$&#xA;into some discrete, generalized eigenvalue problem, $$ K v = \lambda M v, $$&#xA;&lt;em&gt;(here with K the stiffness and M the mass matrix; both symmetric and M additionally positive definite)&lt;/em&gt; &lt;br&gt;&#xA;with $v_i$ being the the value of the eigenvector $v$ on the i'th node of some triangulation of the domain $\Omega$. &lt;br&gt; &lt;br&gt;&#xA;Since every PDE needs proper boundary conditions, we have to force the entries $v_k = 0$ for all $k$ corresponding to nodes with homogenous Dirchlet b.c. &lt;br&gt;&#xA;One way to do such is zeroing all rows and columns of $K$ and $M$ according to these $k$'s, only letting the $k$'s diagonal entries of the matrix $K$ be non-zero (for example setting it to $1$). For Computation, here i use here the SLEPc library with lanzcos or krylov-schur solvers (with or without spectral transformations), working with large matrices $K$ and $M$ in CSR-format, so doing such transformations might be very costly. Probably more serious is the problem that i make the right-hand Matrix $M$ singular; solving the generalized eigenvalue problem basically is about solving the corresponding simple problem:&#xA;$$ M^{-1} K v = \lambda x. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now i wanted to ask some people with more experience in this how to apply the Dirichlet b.c. for this type of problem in an optimal way.&lt;/p&gt;&#xA;" OwnerUserId="10005" LastActivityDate="2014-08-18T18:50:20.970" Title="Applying Dirichlet b.c. to the Eigenvalue-Problem" Tags="|petsc|boundary-conditions|eigenvalues|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14430" PostTypeId="2" ParentId="14429" CreationDate="2014-08-18T18:50:20.970" Score="4" Body="&lt;p&gt;I like &lt;a href=&quot;https://scicomp.stackexchange.com/a/3300/119&quot;&gt;this formulation&lt;/a&gt; for implementing Dirichlet boundary conditions in cases where elimination of boundary dofs is not convenient.  If you apply effectively the same procedure to $M$, the rows and columns corresponding to Dirichlet boundary dofs will be rows and columns of the identity.  This will give you new eigenvalues equal to 1 (or $\alpha$ in the referenced answer).  You can choose $\alpha$ to move them out of the way of the physical eigenvalues you are interested in.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-08-18T18:50:20.970" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14433" PostTypeId="1" AcceptedAnswerId="14486" CreationDate="2014-08-20T05:00:29.777" Score="10" ViewCount="1906" Body="&lt;p&gt;Some older books I've seen say that the minimum number of stages of an explicit Runge-Kutta method of a specified order is unknown for orders $\geq 9$. Is this still true?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What libraries are there for automatically working with high order Runge-Kutta methods?&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="123" LastEditDate="2014-08-25T18:17:20.817" LastActivityDate="2021-03-27T10:59:05.913" Title="Constructing explicit Runge Kutta methods of order 9 and higher" Tags="|ode|runge-kutta|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14435" PostTypeId="1" CreationDate="2014-08-20T01:25:18.323" Score="0" ViewCount="203" Body="&lt;p&gt;I am working in level set method, specially Lankton method &lt;a href=&quot;https://www.dropbox.com/s/yjjlpx0ecyhbo0b/lankton.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;. I try to implement Histogram Separation (HS) Energy problem (Part III.C). It based on Bhattacharyya to control the evolution of contour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand it, the first consider the global method in which we are given an input image and a contour. The contour divides the image into an inside and outside region. The Bhattacharyya distance is calculated by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B_{global}=\sqrt{P_{u}*P_{v}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where ${P_{u}}$ and ${P_{v}}$ are pdf of inside and outside regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we return the Lankton paper which is a local level set.  He divides the image into small region by Ball function B(x,y). Then, the contour will separate these regions into inside and outside region. Each small regions have ${P_{u}}$ and ${P_{v}}$. And we can calculate Bhattacharyya distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F_{HS}=B_{local}=\sqrt{P_{u,x}*P_{v,x}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x$ is center point of $B(x,y)$ in image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that that term. But I cannot understand second term in the formula (integration term, in $\color{red}{\textrm{red}}$ below). Can I represent it by convolution operator?  Here is my &lt;a href=&quot;https://www.dropbox.com/s/myu4teh245sw4u2/Lankton_Bha.zip&quot; rel=&quot;nofollow&quot;&gt;source code&lt;/a&gt;. It works but the result is not correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;\frac{d\phi}{dt}(x) =\ &amp;amp;\delta\phi(x) \int_{\Omega_y}\frac{\mathcal{B}(x,y)\delta\phi(y)}{2} \times \left[F_{HS}\left(\frac{1}{A_v} - \frac{1}{A_u}\right)\right. \\&#xA;&amp;amp;+\left.\color{red}{\int_z K(z - I(y)) \times \left(\frac{1}{A_u}\sqrt\frac{P_{v,x}(z)}{P_{u,x}(z)} - \frac{1}{A_v}\sqrt\frac{P_{u,x}(z)}{P_{v,x}(z)}\right) dz} \right] dy \\&#xA;&amp;amp;+ \lambda\delta\phi(x)\ \mathrm{div}\left(\frac{\nabla\phi(x)}{\left|\nabla\phi(x)\right|}\right)&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K$ is a Gaussian kernel and $A_v$ and $A_u$ are areas inside and outside of these regions.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for its = 1:max_its   % Note: no automatic convergence test&#xA;&#xA;%-- get the curve's narrow band&#xA;idx = find(phi &amp;lt;= 1.2 &amp;amp; phi &amp;gt;= -1.2)';  &#xA;[y x] = ind2sub(size(phi),idx);&#xA;&#xA;%-- get windows for localized statistics&#xA;xneg = x-rad; xpos = x+rad;      %get subscripts for local regions&#xA;yneg = y-rad; ypos = y+rad;&#xA;xneg(xneg&amp;lt;1)=1; yneg(yneg&amp;lt;1)=1;  %check bounds&#xA;xpos(xpos&amp;gt;dimx)=dimx; ypos(ypos&amp;gt;dimy)=dimy;&#xA;&#xA;%-- re-initialize u,v,Ain,Aout&#xA;Ain=zeros(size(idx)); Aout=zeros(size(idx)); &#xA;B=zeros(size(idx));integral=zeros(size(idx));&#xA;%-- compute local stats&#xA;for i = 1:numel(idx)  % for every point in the narrow band&#xA;  img = I(yneg(i):ypos(i),xneg(i):xpos(i)); %sub image&#xA;  P = phi(yneg(i):ypos(i),xneg(i):xpos(i)); %sub phi&#xA;&#xA;  upts = find(P&amp;lt;=0);            %local interior&#xA;  Ain(i) = length(upts)+eps;&#xA;&#xA;  vpts = find(P&amp;gt;0);             %local exterior&#xA;  Aout(i) = length(vpts)+eps;&#xA;&#xA;  %% Bha distance&#xA;  p = imhist(img(upts))/ Ain(i) + eps; % leave histograms unsmoothed&#xA;  q = imhist(img(vpts)) / Aout(i) + eps;&#xA;  B(i) = sum(sqrt(p.* q));&#xA;  term2= sqrt(p./q)/Aout(i) - sqrt(q./p)/Ain(i); %Problem in here===I don't know how to code the integral term &#xA;  integral(i) =sum(term2(:));%integeration can do by this code?&#xA;end   &#xA;&#xA;F =-B./2.*(1./Ain - 1./Aout) - integral./2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10043" OwnerDisplayName="johnmark" LastEditorUserId="1804" LastEditDate="2014-08-20T11:07:08.283" LastActivityDate="2014-08-20T11:07:08.283" Title="Histogram separation energy equation based on Bhattacharyya" Tags="|matlab|image-processing|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14436" PostTypeId="1" AcceptedAnswerId="14448" CreationDate="2014-08-20T07:54:32.820" Score="4" ViewCount="2485" Body="&lt;p&gt;I want to extract the data from the &lt;strong&gt;boundary surface of a cylinder&lt;/strong&gt; (in a &lt;code&gt;.vtu&lt;/code&gt; file) and plot it onto &lt;strong&gt;a plane&lt;/strong&gt;, where the coordinates are theta (rotation angle) and Z(height) of the cylinder. I know that this could be done in many ways, but i am looking for a simple clean solution, using Paraview, python or both (i.e. a python macro in paraview)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I tried using the &lt;code&gt;ExtractSurface&lt;/code&gt; option to export the outer points with their data, but for some reason, some &quot;volume&quot; points (i mean points not on the domain boundary) are also exported&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I wrote a Python script using the &lt;code&gt;VTK&lt;/code&gt; package to parse my input data file,then grab the cartesian coordinates, transform it to polar coordinates and finally take the boundary points using a condition on the radius. Although this works, it is not flawless because boundary points are not positioned &quot;exactly&quot; on the maximum radius but rather within an radius interval (diffuse interface)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there any way to use paraview filters to get the job done ? I also checked the &lt;code&gt;Programmable filter&lt;/code&gt; option, tried some tutorial examples, but I find it complicated somehow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions or ideas ?&lt;/p&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2014-08-20T15:48:45.270" LastActivityDate="2016-06-04T17:08:19.437" Title="Unwrap cylinder to plane in Paraview" Tags="|python|computational-geometry|visualization|local-coordinates|paraview|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14437" PostTypeId="2" ParentId="14169" CreationDate="2014-08-20T08:04:48.600" Score="6" Body="&lt;p&gt;Let me give a part of your answer, I would need some more indications from your side to answer you fully. So please read, and write some comments so that I can complete my answer.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;About notations in numerical methods&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;There are a few mistakes in the way you write your equations. These are only details, but for someone who is used to it, it can a bit disturbing.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\theta_t$, is either written as : $\theta(t)$ if written in an analytical equation or $\theta^n := \theta(t^n)$ if written in a time-discretised equation&lt;/li&gt;&#xA;&lt;li&gt;$q(x,y)$ is fine when working in an analytical equation, but it should be written $q_{i,j}$ if working in a space-discretised equation. $q$ stands for whichever quantity that may apply (that notation is used by a part of the CFD community, mathematicians would use $u$ instead).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;About the conservative and advective forms of an equation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;First, let me clarify your surprise about the form of the equation you are solving.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Advective form&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The advection equation can be written in different forms. The form you seem to be most familiar with is called the &lt;strong&gt;advective form&lt;/strong&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t q + (\underline{u} \cdot \underline{\nabla}) q = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which would give, if written in the partial derivative formulation :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t q + u_x \partial_x q + u_y \partial_y q = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After space-discretising :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t q_{i,j}(t) + u_{i,j}  [D_x q]_{i,j} + v_{i,j} [D_y q]_{i,j} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$q$ : the &quot;working variable&quot; i.e. the variable you want to integrate&lt;/li&gt;&#xA;&lt;li&gt;$\underline{u} := u \underline{e}_x + v \underline{e}_y$ a speed function&lt;/li&gt;&#xA;&lt;li&gt;$D_x$ and $D_y$ your differentiation schemes. For instance : $D_x q := \frac{q_{i+1} - q_{i}}{\Delta x}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Conservative form&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The form of the advection equation you have is called the &lt;strong&gt;conservative form&lt;/strong&gt; or the &lt;strong&gt;divergent form&lt;/strong&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t q + \underline{\nabla} \cdot \underline{f}(q) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using partial derivatives :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t q + \partial_x f(q) + \partial_y f(q) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After space-discretising :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;  \partial_t q_{i,j}(t)&#xA;=&#xA;  - \frac{1}{\Delta x}&#xA;  \left [&#xA;    f(q_{i+\frac{1}{2},j}) - f(q_{i-\frac{1}{2},j})&#xA;  \right ]&#xA;  - \frac{1}{\Delta y}&#xA;  \left [&#xA;    f(q_{i,j+\frac{1}{2}}) - f(q_{i,j-\frac{1}{2}})&#xA;  \right ]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$f(q)$ is a numerical flux. In your case, $f(\theta^n) := \theta^n \gamma^n$. Note that the numerical evaluation of $f(q)$ requires to know values of $q$ located between nodes e.g. $q_{i+\frac{1}{2},j}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am a bit confused that you seem to indicate that $\gamma_t^x \neq \gamma_t^y$... Could you give me more information on this, please ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conservative and advective forms correspond to exactly the same analytical equation i.e. an advection equation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Why different forms ?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are many different forms of the advection equation. All forms correspond to the exact same analytical equation. However, each form may not conserve the quantities as well as others. In particular, the conservative form is very good at conserving the working variable ($\theta$ in your case). Let me get more into details :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $I^n$ be defined as $I^n := \sum_{i=0}^{N_i} \sum_{j=0}^{N_j} \theta_{i,j}^n$&#xA;with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$N_i$, $N_j$ the total number of columns resp. lines in your computational domain&lt;/li&gt;&#xA;&lt;li&gt;$n$ the number of the  time step&lt;/li&gt;&#xA;&lt;li&gt;$i$, $j$ the number of the column resp. line of the current node in the sum&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you use the conservative form, then you will have $I^n = I^0 \forall n$. For example, $I^{1000} = I^0$, which basically means that you have &quot;as much&quot; $q$ at iteration $n=1000$ than when you started your simulation (i.e. $n=0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is perhaps the behaviour you are expecting, but in finite differences strange things happen. In particular, quantity may vanish or appear. Indeed, if you were using the advective form then you would most likely have $I^n \neq I^0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to read a discussion on the conservative properties of an equation, you can refer to : Morinishi 1995, &quot;Conservative properties of finite difference schemes for incompressible flow&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that we are speaking finite differences here. Even when using the conservative form of the advection equation, we discretise in the style of finite differences. It would also be possible to do finite volumes, but we are not doing that here.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;About your particular problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;You seem to want :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;an Euler Explicit time integration scheme&lt;/li&gt;&#xA;&lt;li&gt;a conservative form of the equation&lt;/li&gt;&#xA;&lt;li&gt;no-flux BC&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Euler Explicit time integration scheme&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;$q^{n+1} = q^{n} + rhs \, \Delta t$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A conservative form of the equation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;$&#xA;  \partial_t \theta_{i,j}(t)&#xA;=&#xA;  - \frac{1}{\Delta x}&#xA;  \left [&#xA;    f(\theta_{i+\frac{1}{2},j}) - f(\theta_{i-\frac{1}{2},j})&#xA;  \right ]&#xA;  - \frac{1}{\Delta y}&#xA;  \left [&#xA;    f(\theta_{i,j+\frac{1}{2}}) - f(\theta_{i,j-\frac{1}{2}})&#xA;  \right ]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With: $f(\theta^n) := \theta^n \gamma^n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If second order accuracy is good enough for you, then :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;    \theta_{i+\frac{1}{2},j}&#xA;  =&#xA;    \frac{\theta_{i+1,j} + \theta_{i,j}}&#xA;         {               2             }&#xA;  +&#xA;    O(\Delta x^2)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;No-flux BC&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The best way to apply it is to use a layer of mirror nodes, as you did. That is, we need to set $q_{-1,j} := q_{0,j}$ on the left border and do the same on all four borders. I work in C, hence the bottom left node of my &quot;physical&quot; domain is $(0,0)$ and the top-right node is $(N_i-1,N_j-1)$. In other words, $q_{0,j}$ is a &quot;physical&quot; node and $q_{-1,j}$ is a ghost node (or mirror node in our case).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Scheme chosen&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The scheme that was chosen by OP @MichaelAndrewBentley is 2nd order Lax-Wendroff, which is of the form :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u^{n+1} = b_{-1} u_{i-1}^n + b_0 u_i^n + b_1 u_{i+1}^n + O(\Delta t^2) + O(\Delta x^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With : $b_{-1} := \frac{1}{2} c (1+c)$, $b_0 := 1-c^2$, $b_1 := -\frac{1}{2} c (1-c)$, $c := \frac{a \Delta t}{\Delta x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More details can be found in LeVeque's 2002 book &quot;Finite Volume Methods for Hyperbolic Problems&quot;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit : added the scheme used by @MichaelAndrewBentley, see comments below.&lt;/p&gt;&#xA;" OwnerUserId="10018" LastEditorUserId="10018" LastEditDate="2017-02-03T09:09:22.597" LastActivityDate="2017-02-03T09:09:22.597" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="14439" PostTypeId="1" AcceptedAnswerId="14443" CreationDate="2014-08-20T13:47:25.960" Score="1" ViewCount="71" Body="&lt;p&gt;I'm doing work in computational geometry where the robustness of the algorithm is important. On two separate occasions now have I come across a scenario where I compare the numerical size of two expressions where the final inequality ends up looking something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sqrt{A} + \sqrt{B} &amp;lt; C$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$, $B$ and $C$ are integers. I only care about the value of the predicate. Instinctively I tried squaring both sides but of course that doesn't rid of the square root term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question. Is there a way to determine the truth value of predicates like this using only integer (computer) arithmetic?&lt;/p&gt;&#xA;" OwnerUserId="10047" LastEditorUserId="9667" LastEditDate="2014-08-20T15:32:51.907" LastActivityDate="2014-08-20T21:25:14.970" Title="Integer simplification of irrational inequality" Tags="|computational-geometry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14440" PostTypeId="1" CreationDate="2014-08-20T14:02:03.030" Score="2" ViewCount="69" Body="&lt;p&gt;I need to calculate these ratios:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\mathrm{B}_{x,y}(\alpha + 1,\beta)}{\mathrm{B}_{x,y}(\alpha,\beta)} \tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha,\beta &amp;gt; 0$ and $0\le x\le y \le 1$. Here $\mathrm{B}_{x,y}(\alpha,\beta)$ is the incomplete Beta function, defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{B}_{x,y}(\alpha,\beta) = \int_x^y t^{\alpha-1}(1-t)^{\beta-1}\mathrm{d}t$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large values of the parameters $\alpha,\beta$ , the numerator and the denominator can be very $\mathrm{B}_{x,y}(\alpha,\beta)$ can get very large or very small, resulting in over/underflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way that I can compute the ratio (1) avoiding these over/underflows?&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2014-08-20T20:11:51.560" LastActivityDate="2014-08-20T20:11:51.560" Title="Accurate computation of $\frac{\mathrm{B}_{x,y}(\alpha + 1,\beta)}{\mathrm{B}_{x,y}(\alpha,\beta)}$ for large paramers?" Tags="|special-functions|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14441" PostTypeId="5" CreationDate="2014-08-20T15:52:18.160" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2014-08-20T15:52:18.160" LastActivityDate="2014-08-20T15:52:18.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14442" PostTypeId="4" CreationDate="2014-08-20T15:52:18.160" Score="0" Body="Paraview is an open source scientific visualization application which can import, read, analyse and post-process a wide range of file types." OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2014-08-20T16:18:08.767" LastActivityDate="2014-08-20T16:18:08.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14443" PostTypeId="2" ParentId="14439" CreationDate="2014-08-20T19:12:20.483" Score="2" Body="&lt;p&gt;Assume $a,b,c&amp;gt;0$ (otherwise it's easy). Check that $c-\sqrt{b}&amp;gt;0$, then square $\sqrt{a}&amp;lt;c-\sqrt{b}$ once to get&#xA;$$ c^2+b-a&amp;gt;2c\sqrt{b}. $$&#xA;So check that $c^2+b-a&amp;gt;0$ (if not then it's false because rhs is positive, so lhs must be as well), then check that&#xA;$$ (c^2+b-a)^2&amp;gt;4bc^2. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there is an important issue with this: with integer arithmetic you must guard against overflow. For example, when computing $c^4$ with $32$-bit integers, this will overflow for $c\geq 256$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, you can arrive at the last inequality with just one line of Mathematica:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GroebnerBasis[{c - u - v, u^2 - a, v^2 - b}, {a, b, c}, {u, v}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives:&#xA;$$ \left\{a^2-2 a b+b^2-2 a c^2-2 b c^2+c^4\right\} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is the same in sage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R.&amp;lt;a,b,c,u,v&amp;gt; = PolynomialRing(QQ, 'a b c u v')&#xA;I = R.ideal([c-u-v, u**2-a, v**2-b])&#xA;I.elimination_ideal([u,v])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ideal (c^4 - 2*a*c^2 - 2*b*c^2 + a^2 - 2*a*b + b^2) of \&#xA;   Multivariate Polynomial Ring in a, b, c, u, v over Rational Field&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-08-20T21:25:14.970" LastActivityDate="2014-08-20T21:25:14.970" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14444" PostTypeId="2" ParentId="14366" CreationDate="2014-08-20T19:35:06.063" Score="3" Body="&lt;p&gt;When I change your code to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;psi[x_] := Piecewise[&#xA;  {{0, x &amp;lt; -2}&#xA;   , {CubicLagrangeInterpolation[{-3, -2, -1, 0}, {0, 0, 0, 1}, x], -2 &amp;lt;= x &amp;lt;= -1}&#xA;   , {CubicLagrangeInterpolation[{-2, -1, 0, 1}, {0, 0, 1, 0}, x], -1 &amp;lt;= x &amp;lt;= 0}&#xA;   , {CubicLagrangeInterpolation[{-1, 0, 1, 2}, {0, 1, 0, 0}, x], 0 &amp;lt;= x &amp;lt;= 1}&#xA;   , {CubicLagrangeInterpolation[{0, 1, 2, 3}, {1, 0, 0, 0}, x], 1 &amp;lt;= x &amp;lt;= 2}&#xA;   , {0, x &amp;gt; 2}}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It gives me&#xA;$$ \frac{8 \left(6+\theta ^2\right) \sin^4(\frac\theta2)}{3 \theta ^4}, $$&#xA;which is equivalent to the NR formula&#xA;$$ \frac{6+\theta^2}{3\theta^4}(3-4\cos\theta+\cos2\theta). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think what's going on here is that your kernel's domain is wrong; Lagrange polynomials might be zero on $\pm1,\pm2$, but they don't have to be zero in between, so long as $0$ is one of the interpolation nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Code for endpoint kernels.&lt;/strong&gt;&#xA;I can't quite understand why $\alpha_3$ is not zero, because the kernels only touch points at most a distance $3$ away, so shouldn't the point number $3$ be far away from the endpoints that there is no endpoint contribution? Note that $\alpha_3$ has different domain from $\alpha_{0,1,2},\psi$. They don't quite say &lt;em&gt;which&lt;/em&gt; segments of &lt;em&gt;which&lt;/em&gt; polynomials they use, so this is a bit confusing.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Clear[lag]&#xA;lag[X_, Y_, x_] /; Length[X] == Length[Y] :=&#xA;  Sum[Y[[j]]*&#xA;    Product[If[j == k, 1, (x - X[[k]])/(X[[j]] - X[[k]])], {k, &#xA;      Length@X}], {j, Length@Y}];&#xA;psi[x_] := Piecewise[&#xA;   {{0, x &amp;lt; -2}&#xA;    , {lag[{-3, -2, -1, 0}, {0, 0, 0, 1}, x], -2 &amp;lt;= x &amp;lt;= -1}&#xA;    , {lag[{-2, -1, 0, 1}, {0, 0, 1, 0}, x], -1 &amp;lt;= x &amp;lt;= 0}&#xA;    , {lag[{-1, 0, 1, 2}, {0, 1, 0, 0}, x], 0 &amp;lt;= x &amp;lt;= 1}&#xA;    , {lag[{0, 1, 2, 3}, {1, 0, 0, 0}, x], 1 &amp;lt;= x &amp;lt;= 2}&#xA;    , {0, x &amp;gt; 2}}];&#xA;Plot[psi[x], {x, -2, 2}]&#xA;FullSimplify[&#xA; Integrate[psi[s] Exp[I*\[Theta]*s], {s, -Infinity, Infinity}]]&#xA;alpha[0, x_] := Piecewise[&#xA;   {{0, x &amp;lt; 0}&#xA;    , {lag[{0, 1, 2, 3}, {1, 0, 0, 0}, x], 0 &amp;lt;= x &amp;lt;= 1}&#xA;    , {lag[{0, 1, 2, 3}, {1, 0, 0, 0}, x], 1 &amp;lt;= x &amp;lt;= 2}&#xA;    , {0, x &amp;gt; 2}}];&#xA;alpha[1, x_] := Piecewise[&#xA;   {{0, x &amp;lt; -2}&#xA;    , {lag[{-1, 0, 1, 2}, {0, 1, 0, 0}, x], -1 &amp;lt;= x &amp;lt;= 0}&#xA;    , {lag[{-1, 0, 1, 2}, {0, 1, 0, 0}, x], 0 &amp;lt;= x &amp;lt;= 1}&#xA;    , {lag[{0, 1, 2, 3}, {1, 0, 0, 0}, x], 1 &amp;lt;= x &amp;lt;= 2}&#xA;    , {0, x &amp;gt; 2}}];&#xA;alpha[2, x_] := Piecewise[&#xA;   {{0, x &amp;lt; -2}&#xA;    , {lag[{-2, -1, 0, 1}, {0, 0, 1, 0}, x], -2 &amp;lt;= x &amp;lt;= -1}&#xA;    , {lag[{-2, -1, 0, 1}, {0, 0, 1, 0}, x], -1 &amp;lt;= x &amp;lt;= 0}&#xA;    , {lag[{-1, 0, 1, 2}, {0, 1, 0, 0}, x], 0 &amp;lt;= x &amp;lt;= 1}&#xA;    , {lag[{0, 1, 2, 3}, {1, 0, 0, 0}, x], 1 &amp;lt;= x &amp;lt;= 2}&#xA;    , {0, x &amp;gt; 2}}];&#xA;alpha[3, x_] := Piecewise[&#xA;   {{0, x &amp;lt; -3}&#xA;    , {lag[{-3, -2, -1, 0}, {0, 0, 0, 1}, x], -3 &amp;lt;= x &amp;lt;= -1}&#xA;    , {lag[{-2, -1, 0, 1}, {0, 0, 1, 0}, x], -1 &amp;lt;= x &amp;lt;= 0}&#xA;    , {lag[{-1, 0, 1, 2}, {0, 1, 0, 0}, x], 0 &amp;lt;= x &amp;lt;= 1}&#xA;    , {lag[{0, 1, 2, 3}, {1, 0, 0, 0}, x], 1 &amp;lt;= x &amp;lt;= 2}&#xA;    , {0, x &amp;gt; 2}}];&#xA;Plot[Evaluate@Table[alpha[k, s], {k, 0, 3}], {s, -2, 2}]&#xA;Integrate[&#xA;   Table[(alpha[k, s - k] - psi[s - k]) Exp[I*\[Theta]*s], {k, 0, &#xA;     3}], {s, -Infinity, Infinity}] // ExpToTrig // FullSimplify&#xA;{Series[Re[%] // ComplexExpand, {\[Theta], 0, 6}], &#xA;   Series[Im[%] // ComplexExpand, {\[Theta], 0, 6}]} // &#xA;  Transpose // TableForm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-08-20T22:23:07.393" LastActivityDate="2014-08-20T22:23:07.393" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14445" PostTypeId="2" ParentId="14309" CreationDate="2014-08-20T21:19:11.870" Score="1" Body="&lt;p&gt;You can try an LQ factorization, similar to what you have in mind. It is mathematically equivalent to QR on $A^T$ &lt;a href=&quot;http://www.netlib.org/lapack/lug/node41.html&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lapack/lug/node41.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4558" LastActivityDate="2014-08-20T21:19:11.870" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14446" PostTypeId="1" CreationDate="2014-08-21T09:04:01.400" Score="6" ViewCount="530" Body="&lt;p&gt;I'm working on a Multibody dynamics code using the finite element method to simulate the behaviour of flexible beams (using &lt;a href=&quot;https://bib.irb.hr/datoteka/240342.cm201997507-519.pdf&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; paper if anyone is interested/ it is relevant). I'd like to model joints, and as I only need spherical friction free joints I believe this should be easy using Lagrange multipliers.&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that simple spherical joints are done by adding some extra rows and columns to the tangent stiffness matrix &lt;span class=&quot;math-container&quot;&gt;$K_t$&lt;/span&gt; and extra rows to the residual force vector &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; as below. Lets say we have 6 degrees of freedom at each node, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; nodes, and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; joints. For this example &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is 2 - The first joint links node &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to node &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; and the second node &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; to node &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. The initial geometry places the linked nodes in the same place. The vector &lt;span class=&quot;math-container&quot;&gt;$\delta p$&lt;/span&gt; is the incremental change in node position/ rotation and &lt;span class=&quot;math-container&quot;&gt;$F_J$&lt;/span&gt; is the nodal forces required to keep the nodes coincident.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix} K_t &amp;amp; (\lambda_{2p \times 6n})^T \\ \lambda_{2p \times 6n} &amp;amp; 0 \end{bmatrix}\begin{bmatrix} \delta p \\ F_J \end{bmatrix} = \begin{bmatrix} g \\ 0 \end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is the Lagrange multiplier which for this two joint system is calculated as follows like this:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lambda=\begin{bmatrix} 0 &amp;amp; \cdots &amp;amp;0&amp;amp;1_{1,6i-5} &amp;amp; 0 &amp;amp;\cdots&amp;amp; 0 &amp;amp; -1_{1,6j-5} &amp;amp; 0 &amp;amp; \cdots &amp;amp;0\\ 0 &amp;amp; \cdots &amp;amp;0&amp;amp;1_{2,6i-4} &amp;amp; 0 &amp;amp;\cdots&amp;amp; 0 &amp;amp; -1_{2,6j-4} &amp;amp; 0 &amp;amp; \cdots &amp;amp;0 \\0 &amp;amp; \cdots &amp;amp;0&amp;amp;1_{3,6i-3} &amp;amp; 0 &amp;amp;\cdots&amp;amp; 0 &amp;amp; -1_{3,6j-3} &amp;amp; 0 &amp;amp; \cdots &amp;amp;0\\0 &amp;amp; \cdots &amp;amp;0&amp;amp;1_{4,6k-5} &amp;amp; 0 &amp;amp;\cdots&amp;amp; 0 &amp;amp; -1_{4,6m-5} &amp;amp; 0 &amp;amp; \cdots &amp;amp;0\\ 0 &amp;amp; \cdots &amp;amp;0&amp;amp;1_{5,6k-4} &amp;amp; 0 &amp;amp;\cdots&amp;amp; 0 &amp;amp; -1_{5,6m-4} &amp;amp; 0 &amp;amp; \cdots &amp;amp;0 \\0 &amp;amp; \cdots &amp;amp;0&amp;amp;1_{6,6k-3} &amp;amp; 0 &amp;amp;\cdots&amp;amp; 0 &amp;amp; -1_{6,6m-3} &amp;amp; 0 &amp;amp; \cdots &amp;amp;0\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;When I run the code, the two bodies part company immediately. Any ideas where I'm going wrong? I'm just simulating a double pendulum with 10 elements in each body for now.&lt;/p&gt;&#xA;" OwnerUserId="7637" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-08-21T14:58:44.343" Title="Lagrange Multipliers in Multi-body Finite Element Code" Tags="|finite-element|simulation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14447" PostTypeId="2" ParentId="14446" CreationDate="2014-08-21T14:58:44.343" Score="2" Body="&lt;p&gt;What I've written above is correct. I had made a programming mistake but I will leave it up as I had trouble finding the correct method so it may be useful for someone. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As people in the comments suggested, eliminating degrees of freedom is an alternative approach.&lt;/p&gt;&#xA;" OwnerUserId="7637" LastActivityDate="2014-08-21T14:58:44.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14448" PostTypeId="2" ParentId="14436" CreationDate="2014-08-21T15:29:19.073" Score="2" Body="&lt;p&gt;The best way I can think of doing it without doing any C++ or Python programming is the following steps, all with a serial run of ParaView:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use the Extract Surface filter&lt;/li&gt;&#xA;&lt;li&gt;Use either selections or clip filter to get rid of the caps of the cylinder (optional but better for viewing)&lt;/li&gt;&#xA;&lt;li&gt;Use a Clip filter to cut away the positive X side of the cylinder&lt;/li&gt;&#xA;&lt;li&gt;Use the Calculator filter to unroll this half of the cylinder boundary (check Coordinate Results to do this)&lt;/li&gt;&#xA;&lt;li&gt;From the output of 1 or 2, use a clip filter to cut away the negative X side of the cylinder&lt;/li&gt;&#xA;&lt;li&gt;Use the Calculator filter to unroll the half of the cylinder boundary output from 5&lt;/li&gt;&#xA;&lt;li&gt;Highlight both Calculator filters and then use the Append Datasets filter to stitch them back together (not necessary for viewing but probably useful for analysis)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2014-08-21T15:29:19.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14449" PostTypeId="1" CreationDate="2014-08-21T20:26:15.933" Score="1" ViewCount="780" Body="&lt;p&gt;I was wondering if anyone knows of any algorithms or projects that can detect the frequency of a motion from a video clip.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like if I had a video of a bouncing ball with constant frequency, could an algorithm tell me the approximate frequency (e.g. 1 bounce/s)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers,&lt;/p&gt;&#xA;" OwnerUserId="10061" LastEditorUserId="9667" LastEditDate="2014-08-22T01:38:20.130" LastActivityDate="2014-11-20T10:06:18.843" Title="Algorithm: Extracting motion frequency from video" Tags="|algorithms|machine-learning|computer-vision|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14450" PostTypeId="1" CreationDate="2014-08-21T21:29:38.800" Score="6" ViewCount="2324" Body="&lt;p&gt;I'm implementing basic fluid flow simulator using SPH method basing on e.g. &lt;a href=&quot;https://www10.informatik.uni-erlangen.de/Publications/Theses/2010/Staubach_BA10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www10.informatik.uni-erlangen.de/Publications/Theses/2010/Staubach_BA10.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT (dead url): &lt;a href=&quot;https://www10.cs.fau.de/publications/theses/2010/Staubach_BT_2010.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www10.cs.fau.de/publications/theses/2010/Staubach_BT_2010.pdf&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I've implemented:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;uniform grid constructing&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;density and pressure field calculating&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;pressure force and viscosity force calculating&lt;/li&gt;&#xA;&lt;li&gt;velocity damping force calculating&lt;/li&gt;&#xA;&lt;li&gt;surface tension force (surface normal and curvature value) calculating&lt;/li&gt;&#xA;&lt;li&gt;acceleration and velocity calculating (basing on these forces and gravity force)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I have a problem with calibrating parameters to avoid particles clustering or spreading them in every possible direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way other than random to calculate proper values of simulation parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My parameters are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;particle radius&lt;/strong&gt; - (I'd like it to be 0.3 and to set other parameters corresponding to this one)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;smoothing radius&lt;/strong&gt; - 4*particle radius = 1.2&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;particle mass&lt;/strong&gt; - decides about everything&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;stiffness&lt;/strong&gt; - decides about value of pressure force (k in 2.19 equation)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;rest density&lt;/strong&gt; - decides about pressure field (ρ0 in 2.19 equation)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;dynamic viscosity&lt;/strong&gt; - decides about value of viscosity force (μ in 2.25 equation)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;gravity force&lt;/strong&gt; - (0.0, -9.81, 0.0)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;velocity damping&lt;/strong&gt; - decides about value of damping force (F = -velocity damping * velocity)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;tension coefficient&lt;/strong&gt; - decides about value of surface tension force (σ in 2.30 equation)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;tension treshold&lt;/strong&gt; - minimal length of surface normal to calculate surface tension force (l in 2.31 equation)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;time step&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm pretty sure that my calculations are fine. Unfortunately example values from this or other documents don't work for me and I probably need to calculate them on my own. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll be glad to answer any questions about my implementation to help you help me. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oh, almost forgot, I'm using CUDA 6.0. &lt;/p&gt;&#xA;" OwnerUserId="10063" LastEditorUserId="10063" LastEditDate="2017-11-06T13:59:27.813" LastActivityDate="2017-11-06T13:59:27.813" Title="How to get proper parameters of SPH simulation?" Tags="|algorithms|fluid-dynamics|computational-physics|cuda|sph|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14451" PostTypeId="2" ParentId="14449" CreationDate="2014-08-21T22:03:01.807" Score="1" Body="&lt;p&gt;I think you should take a look at &quot;Eulerian Video Magnification&quot;, a method for magnifying imperceptibly small motions in a video that was invented by the CSAIL lab at MIT. In the process of doing this, they use a variety of signal processing techniques that may be useful to you. By using this technique, the researchers have been able to extract things like pulse rate and breathing rate of people simply by processing a video of the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code to use their algorithm is posted in the link below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://people.csail.mit.edu/mrub/vidmag/&quot; rel=&quot;nofollow&quot;&gt;http://people.csail.mit.edu/mrub/vidmag/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2014-08-21T22:03:01.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14452" PostTypeId="1" AcceptedAnswerId="14460" CreationDate="2014-08-21T23:59:36.157" Score="1" ViewCount="3151" Body="&lt;p&gt;I have calculated the temperature of the section of a cylinder, which is subjected to a heat flow on its upper surface. Getting the temperature distribution in the 2D section. As shown in the following image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/GGnjG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this temperature distribution would represent the upper part of the cylinder as in the picture below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xnjrv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="10054" LastEditorUserId="9667" LastEditDate="2017-06-01T19:44:09.807" LastActivityDate="2017-06-01T19:44:09.807" Title="a circular plot from a vector which represents the temperature along the radius surface, which is the same for every radius" Tags="|matlab|visualization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14453" PostTypeId="2" ParentId="14449" CreationDate="2014-08-22T02:49:23.283" Score="1" Body="&lt;p&gt;Have a look at &lt;a href=&quot;http://opencv.org/&quot; rel=&quot;nofollow&quot;&gt;OpenCV&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It includes &lt;a href=&quot;http://docs.opencv.org/modules/video/doc/motion_analysis_and_object_tracking.html&quot; rel=&quot;nofollow&quot;&gt;motion analysis and tracking&lt;/a&gt;, so depending on how complicated your setup is, it might be as simple as taking one of the sample programs, using it to compute the ball's motion and then looking for sharp changes in direction.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-08-22T02:57:45.247" LastActivityDate="2014-08-22T02:57:45.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14454" PostTypeId="1" CreationDate="2014-08-22T05:34:12.643" Score="5" ViewCount="674" Body="&lt;p&gt;I'm working with an RKF45 integrator that I have programmed using CUDA C++ on my GPU and am pondering a few questions as I'm trying to track down some issues with my code.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm using double precision values in the calculation.  Is it feasible of me to demand of my integrator that the difference between the 4th and 5th order estimates be &amp;lt; 1 x 10 ^-16 if machine precision is that of a double?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I do impose local error to be 10E-16 and I integrate over, for example, 20,000 time steps, should I then be expecting a maximal global error of the product of these two numbers : 2E-12?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Indeed if that is the case, that's not too bad, however I'm trying to track the source of a leak where my numbers become unrealistic and I'm trying to rule out machine precision as the source of that error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My RHS of my equations look like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double frag_term = 0;&#xA;&#xA;    double flux = 0;&#xA;    if (r == ((maxlength)-1))&#xA;        flux = -km*(r)*conc[r]+2*(ka)*conc[r-1]*conc[0];&#xA;&#xA;    else if ( (r &amp;gt; ((nc)-1)) &amp;amp;&amp;amp; (r != ((maxlength)-1)) )&#xA;    {&#xA;        frag_term = conc2[maxlength-1] - conc2[r];  &#xA;&#xA;        flux = -(km)*(r)*conc[r] + 2*(km)*frag_term - 2*(ka)*conc[r]*conc[0] + 2*(ka)*conc[r-1]*conc[0];&#xA;    }&#xA;    else if (r == ((nc)-1))&#xA;    {&#xA;        frag_term = conc2[maxlength-1] - conc2[r];&#xA;&#xA;        flux = (kn)*pow(conc[0],(nc)) + 2*(km)*frag_term - 2*(ka)*conc[r]*conc[0];&#xA;    }&#xA;    else if (r &amp;lt; ((nc)-1))&#xA;        flux = 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where conc is an array of concentrations whose values I am integrating over time, conc2 is an array of partial sums such that conc2[r] = sum of all conc array items less than and including the rth term.&lt;/p&gt;&#xA;" OwnerUserId="5553" LastEditorUserId="5553" LastEditDate="2014-08-22T17:48:29.407" LastActivityDate="2015-11-01T11:15:09.673" Title="Machine precision and local error" Tags="|c++|error-estimation|time-integration|cuda|runge-kutta|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14455" PostTypeId="2" ParentId="14449" CreationDate="2014-08-22T07:48:36.227" Score="0" Body="&lt;p&gt;The reference Tyler Olsen gave in his answer is the one I would have proposed too. Nevertheless I would complement or propose an alternative which consists in a decomposition of the optical flow into oscillating and piecewise constant flow fields. Similarly to the Eulerian approach, you could build a sort of scale space capturing motion a different frequencies by using different regularization parameters in the optimization proposed in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-3-642-03641-5_10#page-1&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/chapter/10.1007%2F978-3-642-03641-5_10#page-1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck&lt;/p&gt;&#xA;" OwnerUserId="9378" LastActivityDate="2014-08-22T07:48:36.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14456" PostTypeId="2" ParentId="14454" CreationDate="2014-08-22T08:09:39.657" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;As I see it, it makes sense to compare on the level of machine precision. In the worst case you will not be able to satisfy your criterion because of rounding errors, but this is a Type I error (the time step is rejected although it meets the requirement) which is 'safe'.&lt;/li&gt;&#xA;&lt;li&gt;No, this depends on how much an error is amplified by your incremental function. This quantity depends mainly on the Lipshitz constant of your right hand side.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Finally, since you are going for high precision, keep in mind, that every single-step method suffers from an inherent instability which is $\mathcal O(\frac{\epsilon}{\tau})$, i.e., an error that scales with machine precision divided by the time-step size. This prevents you from reaching very high precision and, for very small time steps, may destroy any convergence.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-08-22T08:09:39.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14458" PostTypeId="1" AcceptedAnswerId="18668" CreationDate="2014-08-22T13:14:02.283" Score="5" ViewCount="351" Body="&lt;p&gt;I'm wondering if there is something analogous to a PCA for data sets where there is a dependent variable.  (Though I am interested in any method of dimensionality reduction, PCA is just an example.)  A PCA will take a 64-dimensional data set (say) and might return 4 dimensions which explain 95% of the variance.  From a 4-point vector in the new space and the covariance matrix you could recover the original record with pretty good fidelity in this example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what about the case where x has 64 dimensions, and the data have the form (x, y) where y is a measurement?  Is there something similar, that will let me predict y from a lower dimensional set of inputs?  I.e., y $\approx$ g(x'), where dim(x') &amp;lt; dim(x), and there is a map m: x' $\mapsto$ x.  (y' = g(x') and another map p: y' $\mapsto$ y is fine too.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often what people do in this situation is choose a model such as a GLM and do a partial regression, eliminating those dimensions where the regression coefficients are close to 0.  However I'm looking for a method which does not assume any kind of model at all.  Do these exist?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The image below illustrates an extreme case of this.  Nominally the data is 2-dimensional, but the y-dimension is just a bit of 0-mean noise--z can be estimated well enough on the basis of x alone.  So I'm wondering if there's a technique which will derive this, here but also cases where the redundancy is less obvious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q8PUC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="10067" LastEditorUserId="1804" LastEditDate="2014-08-22T13:49:26.307" LastActivityDate="2015-05-07T02:14:30.140" Title="Dimensionality reduction of the domain of f(x)" Tags="|linear-algebra|high-dimensional|data-analysis|model|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="14460" PostTypeId="2" ParentId="14452" CreationDate="2014-08-22T15:46:09.890" Score="3" Body="&lt;p&gt;If I understand, what you want is to make a polar plot of your data presented in the first image. Where, I assume, the axis are the radius and angle. You need to make a change of coordinates from polar to Cartesian to do that. Matlab has a built-in function for that called &lt;code&gt;pol2cart&lt;/code&gt; (&lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/pol2cart.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;). See an example &lt;a href=&quot;http://www.mathworks.com/help/matlab/creating_plots/contour-plot-in-polar-coordinates.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r = linspace(0,1,1000);&#xA;th = linspace(0,2*pi,1000);&#xA;&#xA;[TH,R] = meshgrid(th,r);&#xA;[X,Y] = pol2cart(TH,R);&#xA;&#xA;Z = besselj(0,20*R);&#xA;&#xA;figure&#xA;contourf(X,Y,Z); axis square&#xA;&#xA;figure&#xA;surf(X,Y,Z); shading interp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;here it is the contour plot (in Octave, though):&#xA;&lt;img src=&quot;https://i.stack.imgur.com/bLc93.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;and&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2014-08-22T15:46:09.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14462" PostTypeId="1" CreationDate="2014-08-22T17:24:14.947" Score="0" ViewCount="181" Body="&lt;p&gt;I am trying to do something fairly simple somehow I have made it hard.&#xA;Is there an example of how to send a 3D array to a binary file?&lt;/p&gt;&#xA;" OwnerUserId="10069" LastEditorUserId="276" LastEditDate="2014-08-22T18:24:28.340" LastActivityDate="2014-08-22T18:29:05.640" Title="Writing a 3D array from Petsc" Tags="|matrix|petsc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14463" PostTypeId="2" ParentId="10199" CreationDate="2014-08-22T17:29:38.863" Score="1" Body="&lt;p&gt;Based on so many simulations that I've run using my manual divergence calculation. The answer is yes. I also compared manual calculation with the built-in one and no error or different was observed.&lt;/p&gt;&#xA;" OwnerUserId="4858" LastActivityDate="2014-08-22T17:29:38.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14464" PostTypeId="2" ParentId="14454" CreationDate="2014-08-22T18:16:16.587" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm using double precision values in the calculation. Is it feasible of me to demand of my integrator that the difference between the 4th and 5th order estimates be &amp;lt; 1 x 10 ^-16 if machine precision is that of a double?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assuming you're talking about absolute error tolerances, in general, yes, it makes sense.. A unit in the last place (ulp) would be the 16th decimal place in scientific notation, which does not necessarily correspond to $10^{-16}$. Suppose you have a scalar ODE, and for a given time step, the 5th order method calculates a value of $1.01 \times 10^{-15}$, while the 4th order method calculates a value of $1 \times 10^{-15}$. The estimated local truncation error would be $1 \times 10^{-17}$, without loss of precision due to floating point arithmetic, so you could use an absolute tolerance of $5 \times 10^{-17}$ for this example time step without having to resort to reducing the size of the time step.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If I do impose local error to be 10E-16 and I integrate over, for example, 20,000 time steps, should I then be expecting a maximal global error of the product of these two numbers : 2E-12?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That all depends on how the summation of time steps is implemented. For naive summation, I'd expect the rounding errors to grow roughly linearly with the number of time steps. &lt;a href=&quot;http://books.google.com/books?id=7J52J4GrsJkC&amp;amp;lpg=PA86&amp;amp;ots=B4f5xiqs61&amp;amp;dq=compensated%20summation%20ordinary%20differential%20equation&amp;amp;pg=PA86#v=onepage&amp;amp;q=compensated%20summation%20ordinary%20differential%20equation&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Compensated summation methods&lt;/a&gt; can be used to control the rounding error well when a large number of time steps are taken. (The link references Accuracy and Stability of Numerical Algorithms, 2nd edition by Higham, page 87; see that book for more details regarding how rounding errors propagate for a variety of different numerical methods.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't necessarily expect the global error to grow linearly, however, purely on the basis of local truncation error.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-08-22T18:45:20.233" LastActivityDate="2014-08-22T18:45:20.233" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14466" PostTypeId="2" ParentId="14462" CreationDate="2014-08-22T18:29:05.640" Score="2" Body="&lt;p&gt;It depends on the object representation of your 3D array. If it's a PETSc &lt;code&gt;Mat&lt;/code&gt;, you want to look at &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatView.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;MatView&lt;/code&gt;&lt;/a&gt;; if it's a &lt;code&gt;Vec&lt;/code&gt;, you want to look at &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecView.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;VecView&lt;/code&gt;&lt;/a&gt;. The links are to PETSc documentation, and each of the links has a list of relevant examples. They may not be exactly what you're looking for, but they should be close; in general, you're looking for something like a &quot;View&quot; type command. If these suggestions don't help, I suggest e-mailing the PETSc user list.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-22T18:29:05.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14467" PostTypeId="1" AcceptedAnswerId="14468" CreationDate="2014-08-22T23:50:10.993" Score="0" ViewCount="621" Body="&lt;p&gt;I am using in Python &lt;code&gt;mpi4py&lt;/code&gt; to process in parallel 20 minimization functions. Each of the 20 worker processes the same algorithm but with different random initial starting values. What I don't understand is the time it takes to process my job. I expect each of the workers to take roughly the same amount of time to minimize the function but it varies from 300 seconds to 2000 seconds. Now I am not sure why this is the case. I am wondering if I am specifying my MPI job properly or this happens to be because it is meant to happen. Here's my code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from mpi4py import MPI&#xA;import os&#xA;import random&#xA;import nlopt&#xA;&#xA;data = #load a dataset&#xA;&#xA;#Set the range for each of the variables (parameters)&#xA;X1_ = arange(1.01,1.99,.01)&#xA;X2_ = arange(0.01, 0.9, 0.01)&#xA;&#xA;comm = MPI.COMM_WORLD&#xA;size = comm.Get_size()&#xA;rank = comm.Get_rank()&#xA;&#xA;# Draw random values for each of the parameters&#xA;X1 = random.choice(X1_)&#xA;X2 = random.choice(X2_)&#xA;&#xA;#Set up the lower and upper bound for each variables:&#xA;X1min = 1.05&#xA;X1max = 1.99&#xA;X2min = .01&#xA;X2max = 0.9999999&#xA;&#xA;def myfunc(x, grad):&#xA;    if grad.size &amp;gt; 0:&#xA;        grad = numgrad(myfunction, [x[0], x[1]],&#xA;                       step=1e-8) #numgrad is a function that computes the gradient but irrelevant with a derivative-free algo&#xA;    return myfunction([x[0], x[1]], data)&#xA;&#xA;opt = nlopt.opt(nlopt.LN_NELDERMEAD, 2)&#xA;opt.set_lower_bounds([X1min, X2min])&#xA;opt.set_upper_bounds([X1max, X2max])&#xA;opt.set_min_objective(myfunc)&#xA;opt.set_xtol_rel(1e-8)&#xA;opt.maxeval = 10000&#xA;x = opt.optimize([X1, X2])&#xA;minf = opt.last_optimum_value()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Am I missing something in my MPI specification? &lt;/p&gt;&#xA;" OwnerUserId="9598" LastEditorUserId="9598" LastEditDate="2014-08-23T05:56:52.447" LastActivityDate="2014-08-23T05:56:52.447" Title="Python: Computation time Issue with mpi4py" Tags="|python|mpi|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14468" PostTypeId="2" ParentId="14467" CreationDate="2014-08-23T05:41:49.173" Score="2" Body="&lt;p&gt;The code looks fine, at first glance -- it does what you say you want it to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's virtually no communication in the code you posted, so one thing you could try to isolate the cause of the performance is to run nlopt on each of your 20 starting points in serial, one at a time. When you do this, make sure to set the seed of your random number generator to a fixed value, so that you always get the same random variates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Timing them will either confirm or disprove what I suspect is the main cause: Nelder-Mead requires a different number of iterations to converge for different starting values, leading to the performance differences you observe.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-23T05:41:49.173" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14469" PostTypeId="1" CreationDate="2014-08-23T07:05:21.340" Score="2" ViewCount="123" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to simulate interaction of Smart Fluids with Solids? Is&#xA;  there a software capable of doing so?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I Know:&lt;/strong&gt;&#xA;The two software that I know of are Autodesk Simulation Mechanical and Computational Fluid Dynamics. The brochures and information about these software do not mention Smart Fluids anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I need to Know:&lt;/strong&gt;&#xA;Alternative Open Source Software that can do the job specified. If they can't, then I would like to know a way to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I need to Do:&lt;/strong&gt; Study effects of electric current on Smart Fluids flowing through solid cracks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PS:&lt;/strong&gt;&#xA;Is it possible to include Radiation and Radioactivity in the scenario also?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Thanks to everyone who read this question. Please comment to point out my mistakes as I am relatively new to Stack Exchange and would like to know how to use it properly.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="10074" LastEditorUserId="10074" LastEditDate="2014-08-23T11:39:57.740" LastActivityDate="2014-08-24T22:46:34.357" Title="Simulating interaction of Smart Fluids with Solids" Tags="|fluid-dynamics|simulation|electromagnetism|open-source|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14470" PostTypeId="1" CreationDate="2014-08-23T12:56:18.327" Score="0" ViewCount="102" Body="&lt;p&gt;I was 7 years old when I learnt BASIC. Then I learnt C and Visual Basic till the age of 13. I stopped programming for 4 years continuously, and don't remember much about it now. I have lost the skill, and need to do learn it all over again.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I learn basics and the terminology of programming all over&#xA;  again using C?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Can someone suggest an E-Book for that?&lt;/p&gt;&#xA;" OwnerUserId="10074" LastActivityDate="2014-08-24T22:12:53.727" Title="Beginning Computer Programming" Tags="|c|" AnswerCount="1" CommentCount="1" ClosedDate="2014-08-23T20:38:50.633" ContentLicense="CC BY-SA 3.0" />
  <row Id="14471" PostTypeId="2" ParentId="14469" CreationDate="2014-08-23T12:59:31.177" Score="1" Body="&lt;p&gt;I suggest taking a look at ADINA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.adina.com/index.shtml&quot; rel=&quot;nofollow&quot;&gt;http://www.adina.com/index.shtml&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have functionality for both fluid-structure-interaction and electrodynamic fluids. This paper may give you more insights into the ADINA functionality and also provide some background on computational techniques for the class of problems you are interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.mit.edu/kjb/www/Principal_Publications/The_solution_of_Maxwells_equations_in_multiphysics.pdf&quot; rel=&quot;nofollow&quot;&gt;ADINA Electromagnetics Theory&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-08-23T12:59:31.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14472" PostTypeId="2" ParentId="14470" CreationDate="2014-08-23T13:56:43.330" Score="1" Body="&lt;p&gt;The process of learning or re-learning depends on the reason for which you want to do it. Since your on Scicomp, I assume that your needs are related to a scientific project (BE, Ms, PhD ...) that requires you to understand/write/use computational codes (based on finite volume, finite element ...) to implement a numerical model. Therefore my advice, &lt;em&gt;if this is the case&lt;/em&gt; is that you do not need ebooks to get going in the beginning because they usually contain a lot of information, which sometimes make the learning process boring. You can start with the following:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tutorials&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally think tutorial sites like &lt;a href=&quot;http://www.cprogramming.com/tutorial/c-tutorial.html&quot; rel=&quot;nofollow&quot;&gt;CProgramming&lt;/a&gt; and &lt;a href=&quot;http://en.wikibooks.org/wiki/C_Programming&quot; rel=&quot;nofollow&quot;&gt;C_Programming&lt;/a&gt; are great for their simplicity, the step-by-step approach with nice examples along the way. After that, you should copy the examples and do some &quot;experiments&quot; to grasp the basics (not the basics tought to a 13-year old :) ): changing variable types, defining functions, dealing with input and output ... &#xA;On the side, it's always good to have reference guides. They tend to be heavy in details, but they're useful occasionally for some reminders or some technical detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;StackOverFlow&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For all your easy, moderate and difficult programming tasks, StackOverFlow can be very useful for both solving issues and learning, specially when you begin experimenting things and reach the limit of what you know  &lt;/p&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2014-08-24T22:12:53.727" LastActivityDate="2014-08-24T22:12:53.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14474" PostTypeId="1" AcceptedAnswerId="14475" CreationDate="2014-08-23T20:16:01.533" Score="2" ViewCount="343" Body="&lt;p&gt;I am using &lt;code&gt;scikit-image&lt;/code&gt;'s &lt;code&gt;points_inside_poly&lt;/code&gt; function, and in my code I am calling it enough times that it takes up about 50% of my running time (determined via profiling).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now have two options before me: 1) reduce the number of calls I make to &lt;code&gt;points_inside_poly&lt;/code&gt; and/or 2) find (or make?) a speedier version of &lt;code&gt;points_inside_poly&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I am exploring 1), a solution is not obvious to me at the moment so it will require more thought. In the mean time, I am wondering if there are any speeder alternatives for &lt;code&gt;points_inside_poly&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2014-08-23T22:09:40.273" Title="Speedier alternative to “skimage.morphology._pnpoly import points_inside_poly”?" Tags="|python|computational-geometry|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14475" PostTypeId="2" ParentId="14474" CreationDate="2014-08-23T22:09:40.273" Score="3" Body="&lt;p&gt;What skimage's function does, is check whether a horizontal ray to the left of the point intersects each of the polygon sides. If the number of crossings is odd, it is inside, if it is even it is outside. skimage's implementation is also quite efficient: some time ago, as a learning exercise, I reimplemented skimage's &lt;code&gt;points_inside_polygon&lt;/code&gt; as a numpy gufunc, see &lt;a href=&quot;https://github.com/jaimefrio/gufunc_sampler&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, and I think it ran about 2x faster than skimage's implementation. Nothing to write home about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This algorithm works very well to check if a single point is inside a single polygon, but will be suboptimal for other situations. In particular, if you have many points inside a given polygon, you may want to borrow ideas from polygon rasterization. Information on the web is kind of scattered, but there are several more or less complete slide sets from college courses, see e.g. &lt;a href=&quot;http://www.eng.utah.edu/~cs5600/slides/Wk%203%20Lec04%20polygon%20scan%20conversion.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;. You would want to use a sweep line algorithm, where you first sort your polygon vertices by e.g. y coordinate, and then scan them in order, updating a list of polygon vertices that cross the current y-coordinate. To check if a point with y coordinate y0 is inside the polygon, you would compute the intersection points of the active vertices at that y coordinate, sort them by x-coordinate, and check whether your point x coordinate falls in an inside or outside segment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The skimage algorithm, for $n$ points in a polygon with $m$ vertices, will be an $\mathcal{O}(mn)$ algorithm. The one I described right after would require sorting both the points and the vertices, so $\mathcal{O}(n\log n + m\log m)$, but after that you have work per point proportional to the number of sides crossing at that y-coordinate, which will be smaller than $m$ for most polygons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically, it is highly dependent on the exact characteristics of your points and polygons, but the answer may very well be: yes, there are better methods out there.&lt;/p&gt;&#xA;" OwnerUserId="3399" LastActivityDate="2014-08-23T22:09:40.273" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14477" PostTypeId="1" AcceptedAnswerId="14479" CreationDate="2014-08-24T16:54:15.123" Score="2" ViewCount="226" Body="&lt;p&gt;There is a lot of talk about exascale computing these days, and whether we will be able to reach that goal by 2018, 2019 or whatever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have what is probably a naive question. What are the issues with doing it now?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, today we have the &lt;a href=&quot;http://www.amd.com/en-gb/products/graphics/desktop/r9/295x2&quot; rel=&quot;nofollow&quot;&gt;AMD Radeon 295x2&lt;/a&gt;&#xA;It has a computing power of 11.5 TFLOPS. Combining a hundred thousand of them would give us 1.15 EFLOPS.&lt;br&gt;&#xA;The power consumption of each card is slightly under 500 W, so the total consumption of all of them would be 50 MW (there would probably be some more for cooling etc). I'm only guessing, but lets say all the other stuff (cooling and whatever else) took 20 MW.  Lets say the price of electric power is 60$/MWh, that would amount to slightly more than 35 million \$ per year.&#xA;Price of a single graphics card is 1500 \$, which means the hardware would cost 150 million \$. Lets say infrastructure costs another 50 million \$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare this to current fastest supercomputer &lt;a href=&quot;http://en.wikipedia.org/wiki/Tianhe-2&quot; rel=&quot;nofollow&quot;&gt;Tianhe-2&lt;/a&gt;. It cost 390 million \$ to make, uses 17.6 MW (24 MW with cooling) and has a processing power of 33.86 PFLOPS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tianhe-2&lt;/strong&gt;&lt;br&gt;&#xA;390 million \$&lt;br&gt;&#xA;24 MW&lt;br&gt;&#xA;33.86 PFLOPS  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;AMD Radeon 295x2 x 100000&lt;/strong&gt;&lt;br&gt;&#xA;200 million \$&lt;br&gt;&#xA;70 MW&lt;br&gt;&#xA;1.15 EFLOPS  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for the cost of the Tianhe-2, you could build a computer that is more than 30 times faster and have its running costs covered for more than 5 years. I guess that after 5 years supercomputer mostly become obsolete anyway, so you would build another one :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a difference between the floating point operation that are done by today's supercomputers and these GPUs?&lt;br&gt;&#xA;Is the problem with AMD not being able to produce/supply 100,000 units of 295x2?&lt;br&gt;&#xA;Are there some other practical concerns, like inability to connect 100,000 units into something that would be useful, or inability to cool them properly?&lt;br&gt;&#xA;Would AMD Radeons be for some reason unstable or unreliable?  &lt;/p&gt;&#xA;" OwnerUserId="10083" LastActivityDate="2014-08-25T14:20:40.057" Title="Exascale computer today" Tags="|gpu|exascale|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14479" PostTypeId="2" ParentId="14477" CreationDate="2014-08-24T19:57:23.993" Score="9" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What am I missing here?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most of the broader issues with your proposal are covered in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/769/what-are-the-current-obstacles-to-reaching-exascale-computing?rq=1&quot;&gt;What are the current obstacles to reaching exascale computing?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the cost and power analysis you've done is a lower bound at best: you've calculated the cost it would take to buy 100,000 GPUs, and you can't run anything on a GPU that isn't plugged into anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Operating systems are typically run on CPUs, not GPUs, so for every node in your system, in addition to one (or more) GPU accelerators, you'll need a mainboard with a CPU and some RAM. Furthermore, you've mentioned nothing about interconnects, on node storage, or storage for your entire cluster. All of these things cost money and power, and that's not even counting other necessary components (e.g., cases/racks, cooling fans, heat exchangers for a water cooled system).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a difference between the floating point operation that are done by today's supercomputers and these GPUs?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell, the main difference between a CPU and a GPU is that GPUs are generally built to execute blocks of the same operation on different data across a core, and branching has poor performance. Beyond that, there's really no high-level difference. Some of today's supercomputers use GPUs (for instance, Titan), so I don't think there's a whole lot of difference into you start looking at low-level details.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the problem with AMD not being able to produce/supply 100,000 units of 295x2?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I doubt that.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there some other practical concerns, like inability to connect 100,000 units into something that would be useful, or inability to cool them properly?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Connecting the units isn't an issue. Cooling probably isn't an issue, if you can find the power, and water (if necessary), but it would be expensive. The main practical concern would be reliability (see below).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Would AMD Radeons be for some reason unstable or unreliable?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The main problem is that with so many components, all of them have to be extremely reliable to avoid nodes going down during a computation that uses the whole machine (that is, to avoid hard errors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Soft errors (flipping a bit, for instance) also become a concern at very large scale; for instance, the lead in the solder used to attach components to the motherboards will occasionally emit a small amount of radiation that can flip a bit in memory. Sometimes, bit flips will affect an algorithm, sometimes they won't. Recovering from soft errors is an area of active research.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-08-24T19:57:23.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14480" PostTypeId="2" ParentId="14469" CreationDate="2014-08-24T22:46:34.357" Score="0" Body="&lt;p&gt;First, I think that you are being very specific when you ask about a &lt;strong&gt;smart fluid&lt;/strong&gt;, it's only a fluid with some special property (i.e. variable viscosity). Second, the physics you have mentionned (fluid-solid interaction aka FSI, solid mechanics and heat transfer) are all based on PDEs. So either you go with a multiphysics software (like &lt;a href=&quot;http://www.comsol.com/&quot; rel=&quot;nofollow&quot;&gt;Comsol&lt;/a&gt;) or you can opt for an open source library (like &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal II&lt;/a&gt;) to solve your PDEs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Multiphysics software:&lt;/strong&gt; you have wide choice of existing modules that you can couple easily in weak or strong fashion. However it comes at a price, specially when it's about parallel computing (where extra fees usually apply for the cluster computing license), and you'll always be limited to the existing implemented features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Open source library:&lt;/strong&gt; you have at hand a powerful tool for solving a wide range of PDE systems, you can easily extend it to suit your needs, no extra fuss when running a parallel computation, and most of all it's &lt;strong&gt;&lt;em&gt;free&lt;/em&gt;&lt;/strong&gt;. The only drawback is that you don't usually have the lovely user-friendly interface. Also, some time should be invested to work with the first computations and know how to extend the library, which can be a &lt;em&gt;a bit&lt;/em&gt; tough for someone unfamiliar with programming&lt;/p&gt;&#xA;" OwnerUserId="4964" LastActivityDate="2014-08-24T22:46:34.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14483" PostTypeId="2" ParentId="14477" CreationDate="2014-08-25T14:20:40.057" Score="7" Body="&lt;p&gt;In addition to Geoff's great points:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Single versus double precision&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The Radeon's quoted performance is single precision, but HPC benchmarks generally measure double precision (including the Tianhe-2 number).  The Radeon has poor double precision performance, but if you buy a card focusing on double precision, expect to take at least a factor of 3 hit on performance/Watt for compute-limited operations.  Many cards focusing on double precision performance will also provide ECC and bear an even higher price tag.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;ECC memory&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Server-grade GPUs generally cost 3-5x as much as their consumer brethren due to reliability features like ECC memory and enhanced double precision performance, along with the smaller market.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Peak versus actual performance&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You quote theoretical peak for the GPU, but &lt;a href=&quot;http://top500.org/system/177999&quot;&gt;HPL benchmark performance for Tianhe-2&lt;/a&gt;, which attains only 61% of its theoretical peak.  The limited graphics memory on the Radeon may further limit performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it is likely that &quot;exaflops&quot; will eventually be declared using HPL, it is widely agreed that HPL is not representative of real applications, most of which attain less than 10% of &quot;peak&quot; due to a combination of memory bandwidth limitations, limited data locality, integer instructions/program logic, and irregular computation needed by the application. Efforts such as &lt;a href=&quot;http://software.sandia.gov/hpcg&quot;&gt;HPCG&lt;/a&gt; and &lt;a href=&quot;https://hpgmg.org&quot;&gt;HPGMG&lt;/a&gt; are attempting to create more representative benchmarks.  (I am a developer of HPGMG.)&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-08-25T14:20:40.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14485" PostTypeId="1" CreationDate="2014-08-25T18:05:24.497" Score="3" ViewCount="363" Body="&lt;p&gt;Are there any good books or references on implementing finite difference methods for PDEs?  Specifically, I'm looking for something comparable to Gockenbach's book &lt;a href=&quot;http://books.google.com/books/about/Understanding_and_Implementing_the_Finit.html?id=sP-Acc5d3Q4C&quot; rel=&quot;nofollow&quot;&gt;Understanding and Implementing the Finite Element Method&lt;/a&gt;.  Even more specifically, I'm looking for information on&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Appropriate data structures for regular meshes&lt;/li&gt;&#xA;&lt;li&gt;Good ways to implement stencils&lt;/li&gt;&#xA;&lt;li&gt;How to handle irregular boundaries&lt;/li&gt;&#xA;&lt;li&gt;What order to loop over the nodes/elements&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Really, I'm looking for pragmatic information on implementing the method.  As to the kind of differential equation or programming language, it doesn't really matter.  Mostly, I'm looking for core information that's (mostly) agnostic to the problem.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2014-08-25T18:16:16.753" Title="Books and references on implementing finite difference codes for PDEs" Tags="|finite-element|finite-difference|numerical-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14486" PostTypeId="2" ParentId="14433" CreationDate="2014-08-25T18:08:25.660" Score="16" Body="&lt;h1&gt;Bounds&lt;/h1&gt;&#xA;&lt;p&gt;That is still true.  In &lt;a href=&quot;http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470723351.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Butcher's book&lt;/a&gt;, page 196, it says the following: In &lt;a href=&quot;http://dx.doi.org/10.1007/BF01935372&quot; rel=&quot;nofollow noreferrer&quot;&gt;a 1985 paper, Butcher showed that you need 11 stages to get order 8&lt;/a&gt;, and this is sharp.  For order 10, &lt;a href=&quot;http://imamat.oxfordjournals.org/content/21/1/47.short&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer derived a family of 17-stage methods&lt;/a&gt;, but it's not known if one can do better.  The same information is given in Section II.5 of &lt;a href=&quot;http://www.springer.com/mathematics/analysis/book/978-3-540-56670-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer, Norsett, &amp;amp; Wanner vol. I&lt;/a&gt;.  The latter reference also goes through some of the techniques for developing high-order pairs (up to order 8).&lt;/p&gt;&#xA;&lt;p&gt;There is an upper bound on the minimum number of stages required for any order, since you can construct them by extrapolation.  This has been known for a very long time; see &lt;a href=&quot;http://arxiv.org/abs/1305.6165&quot; rel=&quot;nofollow noreferrer&quot;&gt;this recent paper of mine&lt;/a&gt; for an explanation. However, this bound is quadratic in the order and is certainly quite pessimistic.  &lt;strong&gt;The nodepy software discussed below can generate exact coefficients for these methods, and also for deferred correction methods (which are Runge-Kutta methods) of any order.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I believe @Etienne is correct in saying that the highest order methods that have been constructed by hand are due to Terry Feagin.  Regarding his other comment, this paper contains some 9(8) pairs:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;J.H. Verner, High-order explicit Runge-Kutta pairs with low stage order, Applied Numerical Mathematics, Volume 22, Issues 1–3, November 1996, Pages 345-357&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The number of new conditions at order &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is equal to &lt;a href=&quot;https://oeis.org/A000081&quot; rel=&quot;nofollow noreferrer&quot;&gt;the number of unlabeled rooted trees with n nodes&lt;/a&gt;.&#xA;Here is a table of the (cumulative) number of order conditions &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; required for each order &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;; this table goes further than those provided in the literature and was produced using nodepy:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p | N&#xA;-----&#xA;1 | 1&#xA;2 | 2&#xA;3 | 4&#xA;4 | 8&#xA;5 | 17&#xA;6 | 37&#xA;7 | 85&#xA;8 | 200&#xA;9 | 486&#xA;10| 1205&#xA;11| 3047&#xA;12| 7813&#xA;13| 20299&#xA;14| 53272&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The number of conditions for higher values of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; can easily be computed by taking cumulative sums of &lt;a href=&quot;https://oeis.org/A000081&quot; rel=&quot;nofollow noreferrer&quot;&gt;OEIS sequence A000081&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h1&gt;Software&lt;/h1&gt;&#xA;&lt;p&gt;For very high order methods, the number and complexity of the order conditions becomes impossible to deal with by hand.  Some symbolic packages (Mathematica, at least) have capabilities for generating Runge-Kutta order conditions.  There are probably some other packages out there, but I am aware of the following (both of which I wrote):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/ketch/nodepy&quot; rel=&quot;nofollow noreferrer&quot;&gt;nodepy&lt;/a&gt;: a Python package that can generate symbolic expressions and code for the order conditions to arbitrary order.  It also includes Python code for checking those conditions, computing error coefficients, etc.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://github.com/ketch/RK-opt&quot; rel=&quot;nofollow noreferrer&quot;&gt;RK-Opt&lt;/a&gt;: A MATLAB package that, among many other things, can find high order Runge-Kutta methods with coefficients optimized for a few different purposes.  It currently couldn't handle 9th-order explicit RK (it goes up to 8th order for stage-order-one methods, tenth order for methods with higher stage order).  If that's something you're interested in I could add the 9th-order (and higher) conditions fairly easily.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Another interesting note about order conditions, which becomes significant at such high orders, is that there are two ways to derive them, and they give you different (but collectively equivalent) conditions: one is due to Butcher, the other &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/S0036142994260872&quot; rel=&quot;nofollow noreferrer&quot;&gt;to Albrecht&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2021-03-27T10:59:05.913" LastActivityDate="2021-03-27T10:59:05.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14487" PostTypeId="2" ParentId="14485" CreationDate="2014-08-25T18:16:16.753" Score="1" Body="&lt;p&gt;You might take a look at &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/052187727X&quot; rel=&quot;nofollow&quot;&gt;Trangenstein's book&lt;/a&gt;, which comes with code.  I'm sure there are many others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some quick suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Appropriate data structures for regular meshes&lt;/strong&gt;: That one is simple -- n-dimensional arrays.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Good ways to implement stencils&lt;/strong&gt;: I'm not sure how deep you want to dive into this.  For starters, you just need to know how your array is laid out in memory and order loops accordingly.  If you want to run on GPUs, blocking is important.  If you're talking about cache optimization, it's an area of research and is architecture dependent.  Be prepared to write assembly code.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;How to handle irregular boundaries&lt;/strong&gt;: This is a mathematical question more than an implementation one.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;What order to loop over the nodes/elements&lt;/strong&gt;: I guess I thought this was what you meant by stencils above.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-08-25T18:16:16.753" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14488" PostTypeId="2" ParentId="14148" CreationDate="2014-08-25T18:45:26.150" Score="1" Body="&lt;p&gt;This is one of the canonical examples used to explain stability theory for PDEs, so there are many good explanations in textbooks.  My favorite is in Chapter 8 of &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow&quot;&gt;LeVeque&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other answers that have been given contain some incorrect terms and some information that is not directly relevant.  Most importantly, they use the term &lt;strong&gt;CFL condition&lt;/strong&gt; in an incorrect way.  From LeVeque's text, p. 217, the CFL condition is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A numerical method can be convergent only if its numerical domain of dependence contains the true domain of dependence of the PDE, at least in the limit as $\Delta t$ and $\Delta x$ go to zero.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the context of the heat equations, this implies merely that $\Delta t \propto \Delta x^{1+\epsilon}$ for some $\epsilon&amp;gt;0$.  Of course, this condition is not sufficient for stability; the CFL condition is only &lt;strong&gt;necessary&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-08-26T12:55:39.567" LastActivityDate="2014-08-26T12:55:39.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14491" PostTypeId="2" ParentId="4709" CreationDate="2014-08-26T11:12:50.447" Score="2" Body="&lt;p&gt;The AIC function need an 'lm' or 'glm' object (linear models).&#xA;See functions &lt;code&gt;lm&lt;/code&gt; and &lt;code&gt;glm&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So just do : &lt;code&gt;AIC(lm(logCPK~dataPOW))&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="9802" LastActivityDate="2014-08-26T11:12:50.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14494" PostTypeId="1" AcceptedAnswerId="14495" CreationDate="2014-08-26T16:48:13.000" Score="2" ViewCount="449" Body="&lt;p&gt;I am writing a program that is supposed to use multiple GPUs on a single node using CUDA Fortran. Although I've looked through the &lt;a href=&quot;http://www.pgroup.com/doc/pgicudaforug.pdf&quot; rel=&quot;nofollow&quot;&gt;Portland Group CUDA Fortran Reference&lt;/a&gt;, I am still unclear about how to make memory allocation work in my case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to split a simulation domain between multiple GPUs, such that the GPUs share the load. I have a few constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The field may not be perfectly evenly split, due to partitioning algorithms in use. So a given GPU device usually will end up with a slightly different array than its fellow devices.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The code has to be scalable. Some machines may only have a single GPU available, while other machines may have two, three, our four GPUs, based on PCI ports, etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The field is large enough that I would rather not have simple copies of the field variables, because that would limit my field size to 1/N of the ideal scenario, where N is the number of GPUs attached to the node.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I tried to follow the procedure I would use if  implementing in MPI, given the assumption that each GPU keeps a separate symbol table (distributed memory assumption), and thus can be allocated individually, and used an approach similar to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  program multiGPU&#xA;  implicit none&#xA;  doubleprecision, allocatable, device :: fieldVAR(:)&#xA;  integer, parameter :: NGPUS=3 ! For the sake of this example&#xA;  integer :: istat, i, fieldsizes(NGPUS)&#xA;&#xA;  ! field size assigned to each GPU, which would be done through&#xA;  ! some partitioning algorithm. Arbitrary assignment here:&#xA;  fieldsizes(1)=300&#xA;  fieldsizes(2)=400&#xA;  fieldsizes(3)=350&#xA;&#xA;  do i=1, NGPUS&#xA;      istat= cudaSetDevice(i-1)&#xA;      allocate(fieldVAR(fieldsizes(i)))&#xA;      !&#xA;      !  Assign initial values etc.&#xA;      !&#xA;  end do&#xA;  end program multiGPU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Because in MPI each process keeps a separate symbol table, and the only thing common to all processes are the environment variables such as the ones with the MPI_* prefix&#xA;However, this tactic does not seem to work as the program seems to go with the latest allocation done, and thus suggests that even though there are two separate units they keep a single symbol table, probably due to having the same host code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there is a better way to split the simulation domain than hard coding &quot;GPU1_fieldvar&quot; and &quot;GPU2_fieldvar&quot;.&lt;/p&gt;&#xA;" OwnerUserId="9542" LastActivityDate="2014-08-26T17:21:53.780" Title="CUDA Fortran: Multi GPU Programming and memory allocation" Tags="|fortran|hpc|gpu|cuda|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14495" PostTypeId="2" ParentId="14494" CreationDate="2014-08-26T17:21:53.780" Score="1" Body="&lt;p&gt;I'm not sure if this would be a better fit for the StackOverflow, but here goes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way to do this is to make a new type, which contains the allocatable array for that particular GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at &lt;a href=&quot;https://www.pgroup.com/lit/articles/insider/v3n3a2.htm&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; article from the Portland Group, describing how to do multi-GPU computations with CUDA Fortran.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-08-26T17:21:53.780" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14496" PostTypeId="1" AcceptedAnswerId="14521" CreationDate="2014-08-26T18:01:39.347" Score="1" ViewCount="58" Body="&lt;p&gt;I have a binary volume consisting of a number of disconnected objects. (coming from a noisy, anatomic dataset)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, some of the objects are 'somewhat' connected.  Picture a big cylinder and a small cylinder with a few streamers connecting them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a relatively computationally inexpensive way to measure 'how connected' the big cylinder and the small cylinder are?  I've looked, but may not know the right vocabulary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible measure would be the average number of completely distinct paths to every other pixel in the volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative would be applying morphological operations and seeing what magnitude of, eg, erosion, results in the volumes being disconnected.&lt;/p&gt;&#xA;" OwnerUserId="10104" LastActivityDate="2014-08-28T14:12:33.870" Title="What is a general method for identifying how connected parts of a binary volume are?" Tags="|algorithms|image-processing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14497" PostTypeId="1" AcceptedAnswerId="14498" CreationDate="2014-08-26T19:05:21.483" Score="9" ViewCount="1985" Body="&lt;p&gt;What algorithm is more practically suited in terms of performance for solving the &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Lx=b}$&lt;/span&gt; equation, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{L}$&lt;/span&gt; is a generic Laplacian matrix (associated to a strongly connected graph, its first eigenvalue being null in most cases). &lt;span class=&quot;math-container&quot;&gt;$\mathbf{L}$&lt;/span&gt; can have more than 1 million rows and less than 15 non-null entries per row.&lt;/p&gt;&#xA;&lt;p&gt;In my specific case, I tried using C++ with the Eigen library for a small example. Seeing that most numerical solver libraries tend to offer both conjugate gradient and Cholesky methods, I still need a more educated advice regarding the pros and cons of such sparse solvers. As far as my experience goes, CG should be better, but I suspect the sheer size of the problem can be overwhelming in the end.&lt;/p&gt;&#xA;" OwnerUserId="10106" LastEditorUserId="20688" LastEditDate="2021-12-18T15:09:18.957" LastActivityDate="2021-12-18T15:09:18.957" Title="Solving Lx = b for big sparse Laplacian matrices" Tags="|linear-solver|sparse-matrix|libraries|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14498" PostTypeId="2" ParentId="14497" CreationDate="2014-08-26T19:30:09.793" Score="7" Body="&lt;p&gt;Which one was faster when you tried using Eigen? You may also consider &lt;a href=&quot;http://trilinos.org/&quot; rel=&quot;noreferrer&quot;&gt;Trilinos&lt;/a&gt; if you're using C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By &quot;generic Laplacian matrix&quot;, do you mean a finite difference / finite element discretization of the Laplace operator on some domain, or do you mean the Laplacian of some graph $G$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether or not your solver is overwhelmed as you scale up the number of unknowns depends strongly on the underlying matrix. For example, if $L$ were a band matrix, then direct solution methods like Cholesky require no more space than the original matrix takes up to begin with. On the other hand, if the matrix has a less regular structure -- e.g. it comes from a 5-point stencil in 2D or a random graph -- then matrix factorizations will have a lot of fill-in and CG becomes preferable. In my own experience, the latter case is more common. Generally speaking, for a PDE in 2 dimensions or more, direct methods are faster for problems with less than ~25,000 unknowns, after which iterative methods are the clear winner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either case, there's a lot to gain by &lt;a href=&quot;http://en.wikipedia.org/wiki/Cuthill%E2%80%93McKee_algorithm&quot; rel=&quot;noreferrer&quot;&gt;permuting the matrix&lt;/a&gt; so that its bandwidth is reduced. For direct solution methods, this reduces the amount of fill-in; for iterative solution methods, it can enhance locality of reference when accessing the elements of $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, for iterative methods, choosing a good preconditioner is as important, if not more so, than choosing a good solver. Both Trilinos and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;noreferrer&quot;&gt;PETSc&lt;/a&gt; have implementations of multigrid preconditioners, for which your problem is the textbook application.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-08-26T19:30:09.793" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14499" PostTypeId="1" CreationDate="2014-08-27T01:29:43.777" Score="4" ViewCount="2358" Body="&lt;p&gt;I need to translate a point (&lt;code&gt;P1&lt;/code&gt;) in 3D a certain amount, call it &lt;code&gt;stepSize&lt;/code&gt;, along a vector described by a heading composed of three angles (basically pitch, roll and yaw).  I know that in homogeneous coordinates the following translation matrix would be used&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mTyCa.png&quot; alt=&quot;translation matrix&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;and I can multiply my existing point by &lt;code&gt;Tv&lt;/code&gt; to get the new point.  However, I need something slightly different.  I have &lt;code&gt;P1&lt;/code&gt; and want the new point, &lt;code&gt;P2&lt;/code&gt; to be separated by &lt;code&gt;stepSize&lt;/code&gt; so I need to recover &lt;code&gt;Vx&lt;/code&gt;, &lt;code&gt;Vy&lt;/code&gt; and &lt;code&gt;Vz&lt;/code&gt; as given in the matrix above (from Wikipedia).  Any general suggestions about how to go about that?  I will do this in &lt;code&gt;R&lt;/code&gt; once I know what to do.&lt;/p&gt;&#xA;&lt;p&gt;EDIT:  Here is some additional information.  I am trying to do 3D Turtle Graphics which are very much analogous to aircraft position and headings.  Specifically, I am writing code to implement the 3D L-systems described &lt;a href=&quot;http://algorithmicbotany.org/papers/#abop&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  This figure from the book gives the coordinate system used in that field (rather non-standard if you ask me).&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/x8ZzY.png&quot; alt=&quot;coordsystem&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I have my 'turtle' at the origin of this system, I have a given standard &lt;code&gt;stepSize&lt;/code&gt;, and I have saved the most recent headings, which are the angles around each of the axes H, L and U.  I want to move the turtle forward by &lt;code&gt;stepSize&lt;/code&gt;.  In the graphics system I am using, I have the current Cartesian coordinates of the turtle, and I need to compute the offset in the x, y and z directions to be added to the current position to get the new position (then repeat ...).  A small (?) complication is that the graphics drawing system I am using has the (+)ive x/L axis going to the right, not left as in the book figure.  H in the figure corresponds to the (+)-ive y-axis, and U to the (+)-ive z axis.  Ugh.&lt;/p&gt;&#xA;" OwnerUserId="10108" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-05-14T13:37:47.923" Title="Translate a 3D point along a heading" Tags="|linear-algebra|matrix|computational-geometry|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14501" PostTypeId="2" ParentId="14497" CreationDate="2014-08-27T08:19:41.440" Score="9" Body="&lt;p&gt;Daniel Shapero's answer is excellent, but I felt I should add the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Properly preconditioned iterative methods will almost certainly win here, unless your systems have a very, very special structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a considerable recent literature, unfortunately mostly of the &quot;asymptotic a priori bounds on running time&quot; variety, on solvers for graph Laplacian systems.  See &lt;a href=&quot;http://www.cs.yale.edu/homes/spielman/precon/precon.html&quot;&gt;Dan Spielman's page&lt;/a&gt; for references to papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Spielman's page suggests using the algebraic multigrid method, in particular &lt;a href=&quot;http://code.google.com/p/lamg/&quot;&gt;Oren Livne and Achi Brandt's LAMG&lt;/a&gt; to solve Laplacian systems in practice.  Since it's already implemented and many of the recent Laplacian solvers that come with provable running time bounds aren't, I'd suggest giving it a shot.&lt;/p&gt;&#xA;" OwnerUserId="3525" LastActivityDate="2014-08-27T08:19:41.440" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14502" PostTypeId="1" AcceptedAnswerId="14504" CreationDate="2014-08-27T13:28:25.017" Score="0" ViewCount="950" Body="&lt;p&gt;I tried to research what exactly is a &lt;strong&gt;SOLVER&lt;/strong&gt; only to find a not clear-cut simple answer. My doubts still remain after going through several sites full of discussions about it.&lt;/p&gt;&#xA;&lt;p&gt;I need to clarify as to whether it is a software product, or is it an open-source tool, or is it a commercial software package? Or is it like a library package that one can even design his/her own?&lt;/p&gt;&#xA;&lt;p&gt;All I could find is that people related &amp;quot;solver&amp;quot; to Microsoft Excel or advanced Excel, and there were sources quoting a solver in connection to various programming languages like python, java, C#. I even saw the term's usage with HADOOP.&lt;/p&gt;&#xA;&lt;p&gt;If it happens to be a software package, then, is there any other package comparable to it or better than it, which can perform a wider range of tasks(for statistics) than &amp;quot;solver&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="10093" LastEditorUserId="20688" LastEditDate="2021-12-20T23:10:14.467" LastActivityDate="2021-12-20T23:10:14.467" Title="What is &quot;SOLVER&quot; in R and Statistics/Analytics?" Tags="|statistics|data-analysis|r|terminology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14503" PostTypeId="2" ParentId="14502" CreationDate="2014-08-27T13:37:05.337" Score="0" Body="&lt;p&gt;The word &quot;solver&quot; typically is used to mean a linear equation solver which finds the answer to systems of linear equations usually written in the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Ax=b $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $A$ is a matrix, and $x$ and $b$ are vectors. Sometimes the word is used to mean other kinds of equation solver spanning the range from linear systems to full-blown PDE systems, etc.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-08-27T13:37:05.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14504" PostTypeId="2" ParentId="14502" CreationDate="2014-08-27T13:38:01.110" Score="3" Body="&lt;p&gt;Generally, people refer to software packages as 'solver' that, well, solve some class of equations. For example, it could be stated that R contains a &lt;a href=&quot;http://stat.ethz.ch/R-manual/R-devel/library/base/html/solve.html&quot; rel=&quot;nofollow&quot;&gt;linear system solver&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9366" LastActivityDate="2014-08-27T13:38:01.110" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14505" PostTypeId="1" CreationDate="2014-08-27T14:32:23.377" Score="5" ViewCount="1611" Body="&lt;p&gt;Assuming that a program is written in either C, C++, Fortran or python, what do you think is the best format for the input file given that users could be relatively diverse, i.e. not familiar with the source code? My current interest is a thermodynanic property tool but the discussion could be more general. Here is what I think are the main options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;dumb ASCII file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VARIABLE_NAME&#xA;300&#xA;OTHER_VARIABLE&#xA;101325&#xA;STRING_ARRAY&#xA;N2 O2&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;more involved ASCII file: some custom delimiters to better handle array inputs for example. Essentially you are trying to be smart about your data and have a custom parser that is more flexible than the one for the dumb ASCII&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;JSON&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;xml&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Anything else you would recommend? A lot of tools choose route #2 but I really don't like it, you often end up with weird non-standard syntax that are more confusing than helping.&lt;/p&gt;&#xA;" OwnerUserId="95" LastActivityDate="2014-09-03T01:32:32.223" Title="Recommendations for input file format for science applications" Tags="|data-management|" AnswerCount="7" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14506" PostTypeId="2" ParentId="14505" CreationDate="2014-08-27T15:58:59.247" Score="1" Body="&lt;p&gt;I can recommend based on my personal experience. I had to do a similar task of parsing an input file in a cross-platform C++ program. I needed something simple and most of all something that could be swiftly implemented, so I went with choice #1 (ASCII), because the input data was relatively plain and simple. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, i think that the size and complexity of your input on one hand, and the language you're using on the other hand, should be considered. If a complex input file is needed, choices #3 and #4 are better, nonethless I am not sure which format has a better parsing speed, easier to implement ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another choice is the &lt;a href=&quot;http://en.wikipedia.org/wiki/YAML&quot; rel=&quot;nofollow&quot;&gt;YAML&lt;/a&gt; format which is simple and human readable. If your project uses python, then the &lt;a href=&quot;http://pyyaml.org/wiki/PyYAML&quot; rel=&quot;nofollow&quot;&gt;PyYAML&lt;/a&gt; binding is the way to go.&lt;/p&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2014-08-27T21:16:22.917" LastActivityDate="2014-08-27T21:16:22.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14507" PostTypeId="2" ParentId="14505" CreationDate="2014-08-27T17:53:11.973" Score="1" Body="&lt;p&gt;Don't ignore &lt;a href=&quot;http://www.hdfgroup.org/HDF5/&quot; rel=&quot;nofollow&quot;&gt;HDF5&lt;/a&gt; which is designed for scientific datasets and has bindings for all the languages you mention, more besides.  It's widely used and there are tools for poking around inside files without programming.  If you want to (though I don't know why you would) you can even dump an HDF5 file into XML.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's primary purpose may be for large datasets but it can cope with very small ones too.  Indeed one reason you should consider it is precisely because you can store the parameters and other bits of job setup in the same file as the input and output datasets.&lt;/p&gt;&#xA;" OwnerUserId="1003" LastActivityDate="2014-08-27T17:53:11.973" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14508" PostTypeId="2" ParentId="14505" CreationDate="2014-08-27T18:18:35.033" Score="8" Body="&lt;p&gt;I would perhaps argue that the main virtue of XML is the ease with which one can write a parser, rather than the ease with which you can write an actual XML document.&#xA;JSON seems a slightly better alternative to my mind.&#xA;Both have the advantage of being standards to some degree, meaning people won't have to learn whichever arcane syntax your program of choice has adopted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned Python, and I would argue that a simple Python file will easily be as, or more, accessible than a text file with a custom syntax. People will need to learn a new syntax anyway, which is unlikely to be less complicated than &quot;entry level&quot; python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, input files often end up with some sort of macro capability anyway, which accidentally becomes Turing-complete. In this case, it would be much better to have a &quot;proper&quot; language, with all that entails. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the problem of defining geometry in Monte Carlo particle transport codes, such as FLUKA or Geant4.&#xA;While many such codes use a custom syntax (your case 2), a full scripting language allows you to use loops and conditionals, allowing you to define the geometry in a fraction of the time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other alternatives to Python would be Lua, which is used heavily in the games industry, Julia or even Javascript (it's sub-par handling of numbers aside).&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-08-27T18:18:35.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14509" PostTypeId="2" ParentId="14505" CreationDate="2014-08-27T19:40:16.267" Score="1" Body="&lt;p&gt;This answer is like LKlevin's but too long to add as a comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's OK for input file to not allow fully general (turing complete?) instructions then I'd use json.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to allow the user to specify turing complete instructions in a setting that is tightly controlled by your software then I'd use lua.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to let the user do anything in their input file (downloading stuff from the internet, reading files, hacking the execution sandbox, hacking the host computer) then let them use python.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-08-27T19:40:16.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14511" PostTypeId="2" ParentId="14499" CreationDate="2014-08-27T20:38:57.043" Score="1" Body="&lt;p&gt;&lt;em&gt;Disclaimer, I only know the small amount I've just read about turtle graphics.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the &quot;turtle&quot; in a turtle graphics system is described by a state, $(\vec{P},\vec{H},\vec{L},\vec{U})$, consisting of a point in space, $\vec{P}$, and a set of three unit vectors that denote the orientation in space where $\vec{H}$ is the heading while $\vec{L}$ and $\vec{U}$ specify directions normal to the heading as in your image. You can think of $\vec{L}$ and $\vec{U}$ as standing for left and up for an actual turtle (the animal) located at point $\vec{P}$ with its head pointed in the direction of $\vec{H}$. Motions of the turtle are given by either changing the orientation by specified rotations or by moving in the direction of $\vec{H}$. Moving in a direction other than $\vec{H}$ requires first turning so that $\vec{H}$ points in the desired direction. In terms of a global cartesian coordinate system, this amounts to multiplying the orientation vectors by a rotation matrix for rotations or adding $d\vec{H}$ to the current position $\vec{P}$ to move a distance $d$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In mathematical terms, a rotation operation corresponds to &#xA;$$\mathbf{O}_\text{new}=\mathbf{O}_\text{old}\mathbf{R}$$&#xA;where $\mathbf{O}$ is the orientation matrix with $\vec{H}$, $\vec{L}$, $\vec{U}$ as its columns&#xA;$$\mathbf{O} = \left[ \vec{H}~\vec{L}~\vec{U} \right]$$&#xA;and $\mathbf{R}$ is the rotation matrix. Rotations are given in terms of angular rotations about $\vec{H}$, $\vec{L}$, or $\vec{U}$ which correspond to roll, pitch, and yaw motions. The rotation matrix can be found using the &lt;a href=&quot;http://en.wikipedia.org/wiki/Rotation_matrix#Rotation_matrix_from_axis_and_angle&quot; rel=&quot;nofollow&quot;&gt;formula for a general rotation about a given axis&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I the translations correspond only to motions in the direction of $\vec{H}$. Thus a move of a distance $d$ corresponds to the operation&#xA;$$\vec{P}_\text{new} = \vec{P}_\text{old} + d\vec{H}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that these operations rely on $\vec{H}$, $\vec{L}$, and $\vec{U}$ being unit vectors so you may have to occasionally re-normalize them due to numerical errors from rotation operations.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-08-27T21:26:10.487" LastActivityDate="2014-08-27T21:26:10.487" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14512" PostTypeId="2" ParentId="136" CreationDate="2014-08-27T23:21:22.633" Score="1" Body="&lt;p&gt;See Google Scholar and look for references to HACC and GADGET, among other codes.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2014-08-27T23:21:22.633" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14513" PostTypeId="1" AcceptedAnswerId="14518" CreationDate="2014-08-27T23:32:54.290" Score="6" ViewCount="2060" Body="&lt;p&gt;I need to find the minimum of a function (a log-likelihood from a Potts model) in tens of thousands of dimensions. The function evaluation is quite fast, takes about $10^{-3} s$, and there is a gradient available (not implemented yet, though, but I expect similar timings). The hessian can be computed analytically, but it can be tedious given the number of dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithm will yield the best results in this case? All the documentation I have been able to find gets at most to 100 dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process will be repeated several times for different configurations, so I will be able to benefit from a warm restart. In fact, as I am going to run them in parallel, each individual process doesn't have to be parallelisable on itself.&lt;/p&gt;&#xA;" OwnerUserId="7759" LastActivityDate="2015-05-30T15:11:59.777" Title="Minimisation problem in thousands of dimensions" Tags="|optimization|" AnswerCount="7" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14514" PostTypeId="2" ParentId="14505" CreationDate="2014-08-27T23:33:32.873" Score="1" Body="&lt;p&gt;Use the dumbest format that suits your needs.  Make it ASCII-based if you can.  Make it inflexible and &lt;em&gt;as easy to parse as possible&lt;/em&gt; because there are better things to do with your life than writing input parsers.  That means eschewing JSON and XML and S-expressions and any sort of hierarchy that the code itself will have to strip away.  Look at the DIMACS formats, rather than XML and JSON, for inspiration and try to make something even easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To reiterate, your job is &lt;em&gt;probably not&lt;/em&gt; to build parsers for complicated input formats.  It's to solve a real problem based on whatever input you're given.&lt;/p&gt;&#xA;" OwnerUserId="3525" LastActivityDate="2014-08-27T23:33:32.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14515" PostTypeId="2" ParentId="14513" CreationDate="2014-08-27T23:39:35.020" Score="2" Body="&lt;p&gt;Is your Hessian sparse?  If so, what I write below doesn't really apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Hessian of a general function in tens of thousands of variables will have at least hundreds of millions of entries.  That is, it'll take upwards of a gigabyte of RAM.  Evaluating it will suck enough, and then you'll have to do linear algebra with the thing.  No thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might try using nonlinear conjugate gradient or a limited-memory BFGS variant like L-BFGS here.  Straight BFGS will also probably be too expensive, since it'll store an approximate Hessian inverse that's a gigabyte plus.&lt;/p&gt;&#xA;" OwnerUserId="3525" LastActivityDate="2014-08-27T23:39:35.020" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14516" PostTypeId="2" ParentId="14513" CreationDate="2014-08-28T03:26:35.660" Score="4" Body="&lt;p&gt;In general, for something with maybe 100 variables, I'd probably try sequential quadratic programming. For very large problems, an interior point method would be better. Use an L-BFGS approximation; in practice, it's usually fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most constrained nonlinear programming solvers will be based on sparse direct linear algebra; theory for preconditioning KKT systems is on the bleeding edge of research, so implementations for preconditioned iterative interior point method are on the horizon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem is box-constrained or unconstrained, and your problem is convex, CG is viable, too, and any quasi-Newton-like method (L-BFGS, etc.) or nonlinear iterative methods like NCG, NGMRES, etc. should work; use it with a globalization strategy (line search or trust region) for better results.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-08-28T03:26:35.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14517" PostTypeId="2" ParentId="14513" CreationDate="2014-08-28T04:49:57.913" Score="0" Body="&lt;p&gt;Depending on your problem details you may be able to use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm&quot; rel=&quot;nofollow&quot;&gt;Levenberg-Marquardt&lt;/a&gt; style algorithm. If its not amenable to transformation into this kind of problem then I'd stick with CG, possibly with some kind of preconditioner if you can find one. I've used both successfully in cases with 10's of thousands of dimensions.&lt;/p&gt;&#xA;" OwnerUserId="10125" LastActivityDate="2014-08-28T04:49:57.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14518" PostTypeId="2" ParentId="14513" CreationDate="2014-08-28T07:53:48.450" Score="11" Body="&lt;p&gt;Matrix-free methods will work fine on problems with thousands of variables.  Specifically, a properly implemented Newton-CG or trust-region code does not need to form a dense Hessian in order to compute a Newton iteration, but rather requires only the action of the Hessian on a vector.  These methods rely on a truncated Krylov solver such as truncated-CG or truncated-MINRES in order to partially solve the linear systems.  In my experience, a matrix-free, trust-region algorithm using truncated-CG works the best.  It tends to work better than Newton-CG, which is a line-search method, because the trust-region helps determine how many Krylov iterations are required per iteration.  In Newton-CG, you have to guess and set this number a priori.  For a short description of Newton-CG, see Nocedal and Wright's book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387303030&quot;&gt;Numerical Optimization&lt;/a&gt; on page 169.  That book also contains a description of a trust-region algorithm on page 69.  Use this with truncated-CG, which is described on page 171.  In addition, the big blue &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898719857&quot;&gt;Trust Region Methods&lt;/a&gt; book has the most complete description of the trust-region algorithm.  That book is a little overwhelming.  Start with the basic trust-region algorithm on page 116.  In step 2, use truncated-CG, which is described on page 202.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, these methods are not perfect.  In general, you'll get better performance than a first-order method for a variety of reasons, but a high-order convergence rate is only guaranteed when close to the actual solution and only if we solve the Newton system accurately enough.  If we're using a truncated Krylov solver such as truncated-CG, this is dictated by the clustering of the eigenvalues of the Hessian, which we likely can't compute.  However, like I said above, in general it works well.  If you want me to be frank, I've never seen a situation where a quasi-Newton method worked better than a good matrix-free optimization algorithm.  I'm sure it happens, but not that often.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, in terms of a warm-start, the only real issue with warm-starts would come into play if you had inequality constraints present in the problem.  In unconstrained optimization, I'm not aware of any algorithm that has a problem with warm-starts.  Though, there's a bit of a numerical conundrum if you actually start at the optimal solution.  In any case, inequality constrained algorithms like primal-dual interior-point methods can have trouble with warm starts.  However, other inequality constrained algorithms such as the &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=589186&quot;&gt;Coleman-Li reflective Newton algorithm&lt;/a&gt; have zero problems with this.  If you're close to the solution, just ignore the reflective bits of the code and it works great.  As a note, good equality constrained algorithms also have no issues with warm-starts.  These are things like sequential quadratic programming (SQP), or better, composite step SQP methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, all of these algorithms are already implemented and freely available in &lt;a href=&quot;http://www.optimojoe.com/products/optizelle&quot;&gt;Optizelle&lt;/a&gt;.  It's BSD licensed and has hooks for C++, Python, and MATLAB.  If you want a limited memory quasi-Newton method, it also has BFGS and SR1.  Certainly, you can implement these algorithms yourself.  And, to be honest, the codes for unconstrained optimization aren't that bad.  However, Optizelle already does it and has been used on problems with upwards of half a billion variables.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2014-08-28T07:53:48.450" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14519" PostTypeId="2" ParentId="8198" CreationDate="2014-08-28T09:38:12.570" Score="1" Body="&lt;p&gt;use command &quot;integral&quot; and then set the following input argument:&#xA;'ArrayValued',true&lt;/p&gt;&#xA;" OwnerUserId="10129" LastActivityDate="2014-08-28T09:38:12.570" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14520" PostTypeId="2" ParentId="14513" CreationDate="2014-08-28T12:48:56.263" Score="-1" Body="&lt;p&gt;Perhaps try some genetic algorithms if you are satisfied with a solution that is not perfect?&lt;/p&gt;&#xA;" OwnerUserId="6845" LastActivityDate="2014-08-28T12:48:56.263" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14521" PostTypeId="2" ParentId="14496" CreationDate="2014-08-28T14:12:33.870" Score="0" Body="&lt;p&gt;Apply a connected component labeling to the binary volume (resolve the equivalence classes). Call this step the connection. Every different labeling is now your cluster. The next step is to apply erosion. Perform a connection + erosion + union (merge voxels into a single equivalence class) iteratively until you create new clusters. Note that new clusters emerge after the connection step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the number of iterations which is required to separate A from B as new clusters would be your measure of connectedness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: You could decide wheather A and B are separated by checking the areas of the newly formed clusters. If they are big enough this means you have separated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB is a good software to implement all those.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-08-28T14:12:33.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14522" PostTypeId="2" ParentId="14513" CreationDate="2014-08-28T14:24:55.357" Score="0" Body="&lt;p&gt;I have good experiance with BOBYQA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I don't know if it is the best algorithm, but it was sufficient for my needs (~250 parameter)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are already some implementations (Fortran, C, python) with good descriptions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the author states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The user can substitute modern computation power for mathematical effort.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10131" LastActivityDate="2014-08-28T14:24:55.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14523" PostTypeId="2" ParentId="14351" CreationDate="2014-08-28T15:13:09.177" Score="1" Body="&lt;p&gt;I think your problem lies in the boundary conditions for $\mathbf{u^*}$. Since your grid is collocated, $\mathbf{u^*}$ is stored at cell centers; and the boundary lies half a cell distance from the place where $\mathbf{u^*}$ is actually computed. Thus a higher-order BC for $\mathbf{u^*}$ should do the trick. Also, I would like to mention that the Kim-Moin projection method does not yield a high order of accuracy in pressure [1]. [1] will also help with many other order of accuracy issues you might face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Brown D. L., Cortex R. and Minion M. L. &quot;Accurate projection methods for the incompressible Navier-Stokes equations&quot;&lt;/p&gt;&#xA;" OwnerUserId="10132" LastActivityDate="2014-08-28T15:13:09.177" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14524" PostTypeId="1" CreationDate="2014-08-29T03:47:28.163" Score="5" ViewCount="345" Body="&lt;p&gt;The Brinkman equations for steady flow of an incompressible fluid through rigid porous solid are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-\dfrac{\mu_0}{k}\mathbf{v} - \mathrm{grad}p + \mu_0 \Delta \mathbf{v} =0$&#xA;and $\mathrm{div}(\mathbf{v}) = 0$ in $\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k &amp;gt; 0$ is the permeability of the solid, $\mu_0 &amp;gt;0$ is the viscosity of the fluid, and $p,\mathbf{v}$ are the pressure and velocity fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that the domain is divided into two disjoint regions and $k$ assumes the values $k_0$ and $k_1$ on them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: Is there a jump condition for this problem due to the discontinuity of $k$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ans:&lt;/strong&gt; $\mathrm{jump}(\mathbf{t_n})=0$ where $\mathbf{t_n} := -p\mathbf{n} + \mu_0\nabla \mathbf{v}\;\mathbf{n}$&lt;/p&gt;&#xA;" OwnerUserId="7728" LastEditorUserId="7728" LastEditDate="2014-09-02T16:37:53.357" LastActivityDate="2015-01-30T20:55:29.783" Title="Is there a jump condition for this PDE? ( Brinkman model , piecewise constant permeability)" Tags="|pde|fluid-dynamics|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14525" PostTypeId="1" AcceptedAnswerId="14526" CreationDate="2014-08-29T04:05:44.380" Score="3" ViewCount="782" Body="&lt;p&gt;Consider the elliptic PDE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ -\mathrm{div}(k(x)\nabla u) = f(x) \in \Omega$&#xA;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u = 0$ on $\partial \Omega$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $k(x)$ is piecewise constant across an interface $\Gamma$ in the domain, then we have the jump condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k(x)\nabla u\cdot \mathbf{n}|_{\Gamma+} = k(x)\nabla u \cdot \mathbf{n}|_{\Gamma-}$ where $\mathbf{n}$ is the unit outward normal to the interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When standard Lagrange finite elements are used for this problem, I understand that $u$ is continuous across the interface provided  the mesh is fitted to the interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does the standard formulation enforce the jump condition ?&lt;/p&gt;&#xA;" OwnerUserId="7728" LastEditorUserId="7728" LastEditDate="2014-08-29T13:01:08.847" LastActivityDate="2014-08-29T13:01:08.847" Title="Jump condition for elliptic equation in standard finite element method" Tags="|finite-element|boundary-conditions|poisson|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14526" PostTypeId="2" ParentId="14525" CreationDate="2014-08-29T09:41:07.233" Score="6" Body="&lt;p&gt;&lt;strong&gt;Preliminary remarks:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The natural jump condition is on the normal derivative, not on the full gradient.&lt;/li&gt;&#xA;&lt;li&gt;The fact that the solution is continuous across the interface is part of the formulation of the problem. This is not only due to the finite element formulation. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Formulation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard formulation simply enforces it with... no extra term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start by writing the formulation on the two subdomains (say $\Omega_1$ and $\Omega_2$, separated by the interface $\Gamma$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega_1} k_1 \nabla u_1 \cdot \nabla v_1 - \int_{\Gamma} k_1 (\nabla u_1 \cdot n_1) v_1 = \int_{\Omega_1} f v_1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega_2} k_2 \nabla u_2 \cdot \nabla v_2 - \int_{\Gamma} k_2 (\nabla u_2 \cdot n_2) v_2 = \int_{\Omega_1} f v_2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_i = u|_{\Omega_i}$ and similar for $v$. Now sum the two equations and you get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega} k \nabla u \cdot \nabla v - \int_{\Gamma} k_1 (\nabla u_1 \cdot n_1) v_1 - \int_{\Gamma} k_2 (\nabla u_2 \cdot n_2) v_2 = \int_{\Omega} fv$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now observe that the two normals to $\Gamma$ are in the opposite direction $n_1 = -n_2$ and that on $\Gamma$, $v_1 = v_2$ (due to the continuity condition) and you finally get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega} k \nabla u \cdot \nabla v = \int_{\Omega} fv$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, no extra term, like for homogeneous Neumann conditions. Remark that if the condition was inhomogeneous, there would be an extra term!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approximation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for the approximation, if you do not pay attention and simply apply the standard finite element method, your approximation won't be good close to the interface, since the elements cannot capture this discontinuity. There are in general two ways to overcome this issue:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use a mesh that is fitted with the interface. Then you can use the formulation above and everything should work well. However, this can be tricky if the interface is complicated or if it moves (in a time dependant problem).&lt;/li&gt;&#xA;&lt;li&gt;Use enriched finite elements, aka XFEM, that are built with the purpose of capturing this kind of discontinuities. But the formulation is then a bit different and the implementation more involved.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2014-08-29T09:41:07.233" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14527" PostTypeId="1" CreationDate="2014-08-29T17:13:11.433" Score="3" ViewCount="2675" Body="&lt;p&gt;I have a 2-D mesh of triangles and I have a scalar function $f(x,y)$ defined at all the vertices of this mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to accurately estimate the values of $\frac{\partial f}{\partial x}$ and $\frac{\partial f}{\partial y}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware how to do this when dealing with finite difference methods on the standard orthogonal grid, using the standard formulas. How will I do it on the 2-D mesh? &lt;/p&gt;&#xA;" OwnerUserId="10140" LastEditorUserId="22994" LastEditDate="2017-01-24T19:01:04.240" LastActivityDate="2017-01-25T08:19:38.127" Title="Computing the derivative on a mesh" Tags="|finite-difference|unstructured-mesh|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14528" PostTypeId="2" ParentId="14527" CreationDate="2014-08-29T17:34:16.540" Score="2" Body="&lt;p&gt;You can compute differential operators for this meshes using information around the neighborhood of each vertex (discrete in this case). In this &lt;a href=&quot;http://www.multires.caltech.edu/pubs/diffGeoOps.pdf&quot; rel=&quot;nofollow&quot;&gt;reference&lt;/a&gt; the authors compute differential-geometry operators in the 1-star neighborhood, i.e., the cells around each vertex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is to fit a paraboloid around each vertex and &lt;em&gt;analytically&lt;/em&gt; calculate the derivatives.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2014-08-29T17:34:16.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14529" PostTypeId="2" ParentId="14527" CreationDate="2014-08-30T04:15:28.710" Score="2" Body="&lt;p&gt;The most conceptually simple approach is to select an interpolation scheme, interpolate through the data points that you have, then compute the derivative of the interpolant. This is a natural generalization of the typical finite-difference differentiation formulas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While you don't mention your exact requirements (in particular: do you need the derivatives at the mesh points, or within the mesh triangles?), the simplest approach is to interpolate the function values linearly in each triangle, and estimate the derivative as the derivative of the linear interpolant; the interpolant will have a piecewise constant derivative, and be discontinuous at the mesh points and edges, where you can take linear combinations of adjacent interpolants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also look at other inteprolation methods, like radial basis functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that many software packages support 2d interpolation themselves, in which case it might not be necessary to do the interpolation step yourself. If a library can interpolate a function, but doesn't calculate the interpolant's derivatives, you can recover the derivatives by using a finite-difference approximation on the interpolant.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-08-30T04:15:28.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14530" PostTypeId="1" CreationDate="2014-08-30T06:42:37.667" Score="1" ViewCount="602" Body="&lt;p&gt;For activation function in neural network, I have used the function &#xA;$$\frac{1}{2}(\sin x +1) \enspace .$$&#xA;But this will give me the value of either 1 or 0, allowing me to classify only 2 classes. What if I want to classify more than 2 clusters of data, then what activation function should I use?&lt;/p&gt;&#xA;" OwnerUserId="10066" LastEditorUserId="9667" LastEditDate="2014-08-30T15:31:54.547" LastActivityDate="2014-10-31T14:37:27.473" Title="Neural Network - single layer perceptron for multiple classes" Tags="|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14532" PostTypeId="2" ParentId="10495" CreationDate="2014-08-30T09:29:52.717" Score="1" Body="&lt;p&gt;I think @Ondřej-Čertík already pointed it out: I think you are obtaining the right solutions, but notice that the matching condition between the left and right solutions holds even if you multiply one or both solutions for a constant, so you are free to scale one of your functions as to exactly match the other in the matching point, this is equivalent to change the next-to-boundary initial values. Then you can normalize if you finally need it. In any case I think this is enough as to obtain the right eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="10148" LastActivityDate="2014-08-30T09:29:52.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14535" PostTypeId="1" CreationDate="2014-08-30T14:01:02.070" Score="1" ViewCount="67" Body="&lt;p&gt;First I would like to say that I have searched the for uncertain fitting, robust fitting, linear optimization, convex optimization, etc. But I'm lacking the knowledge to solve this problem, and I need your help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a function gnom(x) = sin(x)*cos(x) and an uncertain part dg(x) = alpha*sin(8x)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and g(x) = gnom(x) + dg(x)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the function sampled, it means that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;k = -47:1:47;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;x = 0.021*k;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't have alpha, but it is bounded: 0 &amp;lt;= alpha &amp;lt;= 0.2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find the upper and lower bounds of g(x). This means finding alpha that maximizes g(x) and alpha that minimizes g(x). Two separate problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm almost sure that these problems (individually) can be written as a optimization problem in the form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;max g(x) = sin(x)*cos(x) + alpha*sin(8x)&#xA;&#xA;s.t. 0 &amp;lt;= alpha &amp;lt;= 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BUT since I have the function SAMPLED it should not be this difficult (non linear optimization). I could write the problem as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;max G1 + alpha*G2&#xA;&#xA;s.t. 0 &amp;lt;= alpha &amp;lt;= 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;G1 = sin(x)*cos(x) SAMPLED with x = 0.021*k; (as defined before). G1 is a &amp;lt;95x1 double&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;G2 = sin(8x) SAMPLED with x = 0.021*k; (as defined before). G2 is a &amp;lt;95x1 double&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&#xA;&lt;p&gt;alpha is the constant to be found&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like this to be solved as an optimization problem (matlab or SeDuMi native commands) and not algorithmically (with for loops, etc). I need this because there are more complicated optimizations that need to be solved...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that this is not hard, but I've searched a lot and could not find an answer... I hope you guys can help me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much&lt;/p&gt;&#xA;" OwnerUserId="10151" LastActivityDate="2014-08-31T21:47:24.980" Title="How to find max and min bounds of a uncertain function" Tags="|optimization|matlab|convex-optimization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14537" PostTypeId="1" CreationDate="2014-08-31T03:12:45.833" Score="5" ViewCount="594" Body="&lt;p&gt;Consider the mixed formulation of the Poisson/Darcy system for a region $\Omega$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\alpha \mathbf{v} + \nabla p = f  \\&#xA; \mathrm{div}[\mathbf{v}] = 0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{v}\cdot \mathbf{n} = v_n$   on  $\Gamma_v$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p = p_0$ on $\Gamma_p$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the weak form:&#xA;$a(\mathbf{v}, \mathbf{w}) := \int_\Omega \mathbf{v} \cdot \mathbf{w}$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;while $b_1(\mathbf{w},p) := \int_\Omega p\;\mathrm{div}[\mathbf{w}]$ or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b_2(\mathbf{w},p) := \int_\Omega \nabla p\cdot\mathbf{w}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the choice for $b_1$ or $b_2$,  the solution spaces can be chosen as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) $\mathbf{v} \in H(div; \Omega)^n $ while $p \in L^2(\Omega)$ for the choice $b_1$. One can choose for this the BDM elements for the velocity and piecewise constant pressures for each element and get a stable solution. In this formulation, the condition on velocity is essential while the boundary condition for pressure is natural.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) $\mathbf{v} \in L^2(\Omega)^n, p \in H^1_0(\Omega)$ for the choice $b_2$. In this case, the velocity can be approximated by the discontinous Raviart-Thomas elements of polynomial order $k$ and the pressure by Lagrangian elements of polynomial order $k+1$, to get a stable solution. In this formulation, the condition on velocity is natural while the boundary condition for pressure is essential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q1. Is one of 1), 2) to be preferred over the other choice? Will the solutions obtained differ ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q2. However, $H^1_{0}(\Omega)^n \subset H(div;\Omega)^n$. So suppose I  choose &#xA;$\mathbf{v} \in H^1_0(\Omega)^n, p \in L^2(\Omega)$ for the choice $b_1$ and use the standard Taylor-Hood element, enforcing velocity boundary condition as essential and the pressure boundary condition as natural. Is this a stable approximation ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;PS: I am unable to create the tags &quot;Sobolev-spaces&quot;, &quot;mixed-formulations&quot;. If you agree that they are apt for this question, could you please create them.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="7728" LastEditorUserId="7728" LastEditDate="2014-08-31T15:09:49.157" LastActivityDate="2014-08-31T21:43:53.107" Title="Choice of spaces for mixed formulation for Poisson Equation Or Darcy equation" Tags="|finite-element|poisson|navier-stokes|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14538" PostTypeId="2" ParentId="14537" CreationDate="2014-08-31T13:54:17.343" Score="2" Body="&lt;p&gt;Q1) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of finite elements approximation the main question is about stability (aka invertibility of the discrete system). For the primal formulation (elliptic systems) usually any subspace $V_h \subset H_0^1$ yields a stable approximation because the bilinear form is coercive. However the condition that the $a$ form be coercive is not satised for most mixed methods. In fact elements that are not chosen with respect to stability will usually prove to be unstable. But $a$ can be made coercive on certain subspaces of $H(div)^n$. For a good reference I recommend &lt;a href=&quot;http://www.ima.umn.edu/~arnold//papers/mixed.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.ima.umn.edu/~arnold//papers/mixed.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q2) if you choose an unstable approximation such as $V_h\subset H_0^1 \subset H(div)^n$ your discrete system is not invertible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the best&lt;/p&gt;&#xA;" OwnerUserId="9378" LastActivityDate="2014-08-31T13:54:17.343" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14539" PostTypeId="2" ParentId="14537" CreationDate="2014-08-31T21:43:53.107" Score="3" Body="&lt;p&gt;Assuming you choose stable pairings of elements, the main consideration is which variable you are more interested in. For example, if you choose the $H(div),L^2$ formulation, then you have the choice of piecewise constants for the pressure and a BDM element for the velocity. This yields only first order accuracy for the pressure in the $L^2$ norm -- in other words, very slow convergence. You would do this if your primary concern is not the pressure, but the velocity, for example because you want to compute the velocity field as the input for an advection equation transporting along some contaminant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you choose the $L^2,H^1$ formulation, then you need to use at least a continuous $P_1$ element for the pressure, and you get second order convergence. If you care about the pressure, then that would be an option. Of course, you could also choose a higher order mixed element and the first formulation in that case to get second order convergence in the pressure. If you do care about the pressure, which formulation to use is then a question of trading off the relative sizes of the velocity spaces.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-08-31T21:43:53.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14540" PostTypeId="2" ParentId="14535" CreationDate="2014-08-31T21:47:24.980" Score="2" Body="&lt;p&gt;You're trying to solve an optimization problem for a function that is not convex nor has any other useful properties. This can only be solved by algorithms for &quot;global optimization problem&quot;, which are generally very complicated. On the other hand, since in your case the problem is one-dimensional, sampling as you do seems like a completely reasonable approach.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-08-31T21:47:24.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14541" PostTypeId="2" ParentId="14524" CreationDate="2014-08-31T21:52:49.710" Score="1" Body="&lt;p&gt;There is a jump condition, but it is automatically satisfied by the solution (i.e., you don't need to do anything special). This is no different than solving $-\nabla \cdot k \nabla u = f$: there is a jump condition discussed in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14525/jump-condition-for-elliptic-equation-in-standard-finite-element-method&quot;&gt;Jump condition for elliptic equation in standard finite element method&lt;/a&gt; but it is automatically satisfied.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-08-31T21:52:49.710" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14542" PostTypeId="2" ParentId="14505" CreationDate="2014-08-31T23:57:48.153" Score="3" Body="&lt;p&gt;I disagree with the &quot;make it as easy as possible&quot; answers to this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Options 1 and 2 lock you into something you will have a hard time changing later on. For example, you will find that if your number of parameters grows, you will want to put them into sections and subsections because that makes it simpler to organize. But your syntax doesn't allow this. This is why Microsoft Windows configuration files have these weird &lt;code&gt;[section]&lt;/code&gt; markers -- but they neither really help to clarify sections (because there is no 'end-section' marker) nor do they allow nesting. In other words, Windows configuration files are a great example of &lt;em&gt;&quot;well, dang, shoulda thought of that some more up front&quot;&lt;/em&gt;. (Of course, that's sort of a running theme with many Microsoft features.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, don't do it, you will regret having gone this way and keep inventing strange workarounds to do things you wish you had considered beforehand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 4, on the other hand, is great if you have a machine write the file, but not if a human is doing it. People have tried writing scientific software where the input files are created by hand and have XML format -- I've been on the board of an organization that did that, and it was a complete disaster because people rebelled: XML is simply not readable nor writable for humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That only leaves you with option 3. Whether you choose JSON or something that is easier on the eye but has otherwise the same expressiveness does not matter that much. But it should be human readable, human writable, and have some sort of syntax. It should also be pretty self-explanatory what an input file wants to say, which is neither true for your option 1 nor 4. Finally, it should be relatively straightforward to implement in your code, which calls for something for which there already is a parser.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-08-31T23:57:48.153" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14543" PostTypeId="2" ParentId="14527" CreationDate="2014-09-01T00:13:45.450" Score="3" Body="&lt;p&gt;If you have a triangular mesh with data defined at each vertex, you can of course associate a linear function with each element that interpolates the data values at the vertices. This also provides you with an estimate of the gradient on each cell by just computing the gradient of the linear interpolant. Of course, it is only an estimate because you are compute the gradient of the &lt;em&gt;interpolant&lt;/em&gt; of $f$, not of $f$ itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is possible to improve the accuracy of this estimate. For example, you could use a process such as the ZZ estimator (short, for Zienkiewicz and Zhou). If you google for &quot;ZZ estimator&quot; you will find quite a number of links to pages that describe how this is done. (Typically, these pages talk about stresses, strains and displacements; in your context, imagine stress=strain=$\nabla f$, and displacement=$f$.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-01T00:13:45.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14545" PostTypeId="1" AcceptedAnswerId="14546" CreationDate="2014-09-01T05:53:42.920" Score="5" ViewCount="1444" Body="&lt;p&gt;I've heard that classical Gram-Schmidt is more amenable to parallelization than modified Gram-Schmidt; apparently the reason has something to do with &quot;level 2 BLAS&quot;, which I'm not familiar with.  Also, in a &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1391/are-there-any-famous-problems-algorithms-in-scientific-computing-that-cannot-be&quot;&gt;comment on this question&lt;/a&gt;, @Jed Brown talks about left-looking and right-looking Gram-Schmidt, which I haven't found a reference to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How is the Gram-Schmidt algorithm parallelized in practice, and how effective is the parallelization&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-09-02T13:48:30.467" Title="Parallel Gram-Schmidt algorithms" Tags="|linear-algebra|parallel-computing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14546" PostTypeId="2" ParentId="14545" CreationDate="2014-09-01T08:59:37.283" Score="2" Body="&lt;p&gt;First, Blas(Basic Linear Algebra Subprograms) is a collection of highly optimized low level functions performing standard linear algebra operations such as scaling, dot product and matrix multiplication. Therefore it is desirable to make use of these functions for optimal performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The major advantage of the modified Gram-Schmidt(MGS) over the classic Gram-Schmidt is that it offers better stability by rearranging the order of operations. Threfore it is often used in practice. Unfortunately, the MGS cannot be expressed by BLAS routines and requires additional communication when implemented in parallel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice you will resort to some parallel programming language such as OpenMP or CUDA, if you own a graphics card. In the article &lt;a href=&quot;http://www.drdobbs.com/go-parallel/article/print?articleId=240006334&amp;amp;siteSectionName=&quot; rel=&quot;nofollow&quot;&gt;http://www.drdobbs.com/go-parallel/article/print?articleId=240006334&amp;amp;siteSectionName=&lt;/a&gt; they mention a speed up of a factor 3 over serial implementation using OpenMP and almost a factor 9 when using OpenACC with a problem size of $10^3 \times 10^3$. You may also take a look at &lt;a href=&quot;http://link.springer.com/article/10.1140/epjst/e2012-01638-7#page-1&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/article/10.1140/epjst/e2012-01638-7#page-1&lt;/a&gt; for a comparison between GPU and CPU implementations.&lt;/p&gt;&#xA;" OwnerUserId="9378" LastEditorUserId="9378" LastEditDate="2014-09-02T13:48:30.467" LastActivityDate="2014-09-02T13:48:30.467" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14548" PostTypeId="2" ParentId="14530" CreationDate="2014-09-01T13:53:05.170" Score="1" Body="&lt;p&gt;To divide the data into K classes you can use K single layer perceptrons. Perceptron k is trained to output 1 if a training data belongs to that one. When using the perceptrons, calculate the output of the K perceptrons and assign the test data to the class with largest perceptron output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can save 1 perceptron by considering the fact that an input should belong to at least one class. For more information look up:&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Multinomial_logistic_regression&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Multinomial_logistic_regression&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10169" LastActivityDate="2014-09-01T13:53:05.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14549" PostTypeId="1" AcceptedAnswerId="14550" CreationDate="2014-09-01T18:40:39.003" Score="0" ViewCount="243" Body="&lt;p&gt;Consider the Darcy equation,&#xA;$$\mathbf{v} + \dfrac{k}{\mu_0}\nabla p = \mathbf{f} \\ \mathrm{div}\; \mathbf{v} = 0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the coefficient $k$ is piecewise constant across an interface $\Gamma$ in the domain,  we have that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(1)&lt;/strong&gt; $$\mathrm{jump}(k \cdot p\mathbf{n}) = 0 \rightarrow  \mathrm{jump}(k\cdot p) =0 $$ over the interface $\Gamma$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the system can also be written as &#xA;$$ \mathrm{div}(k \nabla p) = \mu_0f_1 =  \mathrm{div}\mathbf{f} $$ in which case the jump condition now is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(2)&lt;/strong&gt; $$\mathrm{jump}(k \nabla p\cdot\mathbf{n}) = 0$$ over $\Gamma$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused as to how to reconcile the two. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, does this dictate the choice of finite element spaces used in the solution ? For example, using a piecewise continuous polynomial for  $p$ in 1) would be wrong ? &lt;/p&gt;&#xA;" OwnerUserId="7728" LastActivityDate="2014-09-03T12:02:11.143" Title="jump conditions for Poisson/Darcy equation in primal form versus mixed form" Tags="|finite-element|fluid-dynamics|boundary-conditions|poisson|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14550" PostTypeId="2" ParentId="14549" CreationDate="2014-09-02T00:38:42.410" Score="2" Body="&lt;p&gt;Your jump condition is wrong. To see this, let's assume for a moment that $\mu=1$ because it plays no real role in your formulation. Then, if you want to integrate the $\nabla p$ term and still want to get a symmetric formulation, you need to start with the first equation in the form&#xA;$$&#xA;  k^{-1} \mathbf v + \nabla p = k^{-1} \mathbf f,&#xA;$$&#xA;which ultimately leads to the jump condition $[p]=0$ -- in other words, the pressure must be continuous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you see that this is the right jump condition? Multiply the equation with a test function $\phi$ and integrate over a (arbitrary) part of the domain $\Omega_1$, for example one of the subdomains where $k$ is constant. After integrating by parts, you have&#xA;$$&#xA;  (\phi,k^{-1} \mathbf v)_{\Omega_1} - (\nabla\cdot\phi, p)_{\Omega_1} + (\phi,p\mathbf n)_{\partial\Omega_1}= (\phi,k^{-1} \mathbf f)_{\Omega_1}.&#xA;$$&#xA;Now do the same on $\Omega_2=\Omega\backslash\Omega_1$ and you get&#xA;$$&#xA;  (\phi,k^{-1} \mathbf v)_{\Omega_2} - (\nabla\cdot\phi, p)_{\Omega_2} + (\phi,p\mathbf n)_{\partial\Omega_2}= (\phi,k^{-1} \mathbf f)_{\Omega_2}.&#xA;$$&#xA;In the last equation, the sign of the normal vector is of course outward from $\Omega_2$, whereas in the first equation it is outward from $\Omega_1$. Now add these two equations together and you get&#xA;$$&#xA;  (\phi,k^{-1} \mathbf v)_{\Omega} - (\nabla\cdot\phi, p)_{\Omega} + (\phi,p\mathbf n)_{\partial\Omega} + (\phi,[p]\mathbf n)_{\Gamma} = (\phi,k^{-1} \mathbf f)_{\Omega},&#xA;$$&#xA;where $\Gamma$ is the interface between $\Omega_1$ and $\Omega_2$ and $[p]$ is the jump of $p$ on the interface. $\mathbf n$ is the normal from $\Omega_1$ info $\Omega_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could, on the other hand, also have multiplied the original equation by $\phi$ and instead integrated over the &lt;em&gt;entire&lt;/em&gt; domain right away. This would have shown that $\mathbf v$ must satisfy the equation&#xA;$$&#xA;  (\phi,k^{-1} \mathbf v)_{\Omega} - (\nabla\cdot\phi, p)_{\Omega} + (\phi,p\mathbf n)_{\partial\Omega}  = (\phi,k^{-1} \mathbf f)_{\Omega}.&#xA;$$&#xA;Comparing with the equation immediately above, it is clear that we have the condition&#xA;$$&#xA;  (\phi \cdot \mathbf n,[p])_{\Gamma} = 0.&#xA;$$&#xA;Because the normal traces $\phi \cdot \mathbf n$ of functions in $H(div)$ are in $L^2(\Gamma)$, this implies that $[p]=0$ in $L^2(\Gamma)$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2014-09-03T12:02:11.143" LastActivityDate="2014-09-03T12:02:11.143" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14551" PostTypeId="1" CreationDate="2014-09-02T18:02:12.990" Score="1" ViewCount="233" Body="&lt;p&gt;I have the program &lt;code&gt;four1.c&lt;/code&gt; from Numerical Recipes in C to calculate the Discrete Fourier Transform (DFT) of a one dimensional array. I want to calculate the DFT of a two dimensional array using &lt;code&gt;four1.c&lt;/code&gt;, how can I do it?&lt;/p&gt;&#xA;" OwnerUserId="10176" LastEditorUserId="6910" LastEditDate="2014-09-04T09:44:37.440" LastActivityDate="2014-09-04T09:44:37.440" Title="How can DFT of a two dimensional array be found using program for one dimensional DFT in C?" Tags="|c|fourier-analysis|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14552" PostTypeId="1" CreationDate="2014-09-02T18:21:00.627" Score="2" ViewCount="2817" Body="&lt;p&gt;I am looking for the algorithm of Patankar (for example, SIMPLE, SIMPLER, SIMPLEC and PISO) written in Fortran for the simulation of heat transfer and fluid flow.&lt;/p&gt;&#xA;" OwnerUserId="10177" LastActivityDate="2015-05-21T12:50:23.480" Title="Patankar's algorithms for Numerical Heat Transfer and Fluid Flow" Tags="|algorithms|computational-physics|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14553" PostTypeId="2" ParentId="14370" CreationDate="2014-09-02T22:14:59.830" Score="2" Body="&lt;p&gt;LIC is very good for visualizing such fields, but what is the texture warped here? First of all, use LIC with white noise in the very beginning. You could then try Perlin noise and similar. There are also other variants of information, such as the coherence directions, which would give you a better idea of the flow field. You could check out &lt;a href=&quot;http://tbirdal.me/downloads/tolga-birdal-flow-lic-icip-2010.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this work of mine&lt;/a&gt; for an interesting application.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorDisplayName="user30082" LastEditDate="2019-02-26T07:16:28.080" LastActivityDate="2019-02-26T07:16:28.080" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14554" PostTypeId="2" ParentId="14505" CreationDate="2014-09-03T01:32:32.223" Score="0" Body="&lt;p&gt;The best solution I think is a combination of @lklevin's and @wolfgang-bangerth answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, make sure to have a clean API. This means have one function/class which takes all the needed parameters. Then, if your program is written in a language where string/data manipulation would require some amount of code (i.e. Fortran, C, C++ etc.), wrap it in a dynamic language (python, ruby, lua, perl etc.), and use a standard format (ini, json, xml).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means if you really need to have some kind of Turing complete input format its easy to add, and if the format you choose is not suitable any more, then you can change easily without affecting the main code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The worst thing would be some unique format, as if you end up needing to generate (or read using a different program) the format (say as the output of another program), then you'll (or someone else who is trying to use your work) have to work out how to write your new format.&lt;/p&gt;&#xA;" OwnerUserId="1437" LastActivityDate="2014-09-03T01:32:32.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14555" PostTypeId="2" ParentId="14551" CreationDate="2014-09-03T13:53:21.723" Score="1" Body="&lt;p&gt;You just take the DFT across each row and then across each column, or columns then rows, it doesn't matter.&lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2014-09-03T13:53:21.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14556" PostTypeId="2" ParentId="14552" CreationDate="2014-09-03T14:17:12.813" Score="5" Body="&lt;p&gt;Probably the best thing is to use codes from a book by J.Ferziger and M. Peric &quot;Computational Methods for Fluid Dynamics&quot;, available from &lt;a href=&quot;ftp://ftp.springer.de/pub/technik/peric/&quot; rel=&quot;nofollow&quot; title=&quot;Springer ftp site&quot;&gt;Springer ftp site.&lt;/a&gt; site.&#xA;There you will find F77 codes for 2D Navier-Stokes equations. I suggest starting from 2dc folder and pcol.f code which uses collocated variable arrangement, instead of staggered which was explained in Patankar's book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally I think the Patankar's book appeared three years too early, and it therefore didn't include Finite Volume Method version using collocated variable arrangement enabled after introduction of Rhie-Chow interpolation in 1983. Also Patankar gives a sketch of what is vertex-based median-dual approach, also very important for unstructured meshes, and doesn't go into detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there is a need for a new book on FVM using general unstructured cells from the start and not using square cells that reduce calculations to something similar to finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice-learn from the codes above, and start reading papers as soon as possible. There are good papers explaining FVM for fluid flows in a comprehensive and modern way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References that I recommend are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;S. Muzaferija and D. Gosman, Finite-Volume CFD Procedure and Adaptive Error Control Strategy for Grids of Arbitrary Topology, Journal of Computational Physics,138, pp.766-787 (1997) (discretization procedure, least-square cell-centered gradients, adaptive grids)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I.Demirdzic, S. Muzaferija, Numerical method for coupled fluid flow, heat transfer and stress analysis using unstructured moving meshes with cells of arbitrary topology, Comput. Methods Appl. Mech. Engrg, 125, pp.235-255 (1955) (both of these papers are written by CD-Adapco people)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;D. Kim, H. Choi, A Second-Order Time-Accurate Finite Volume Method for Unsteady Incompressible Flow on Hybrid Unstructured Grids, JCP, 162, pp.411-428 (2000)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;M.Darwish, I. Sraj, F. Moukalled, A coupled finite volume solver for the solution of incompressible flows on unstructured grids, JCP, 228, pp.180-201 (2009) (!!!)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;F.-S. Lien, A pressure-based unstructured grid method for all-speed flows, Int. J. Numer. Meth. Fluids, 33, pp.355-374 (2000)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;S.R. Mathur and J.Y.Murthy, A pressure-based method for unstructured meshes, Numerical Heat Transfer, Part B: Fundamentals, 31(2),pp.195-215 (1997) (Fluent people)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;B. Basara, Employment of the second-moment turbulence closure on arbitrary unstructured grids, Int. J. Numer. Meth. Fluids, 44, pp. 377-407 (2004) (AVL-Fire\Swift developer)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;H.Jasak, H.G.Weller and A.D.Gosman, High resolution NVD differencing scheme for arbitrary unstructured meshes, Int. J. Numer. Meth. Fluids, 31, pp. 431-449 (1999) (introducing Gamma differencing scheme for convection terms, OpenFOAM developers)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The list may extend to PhD theses. In OF community H. Jasak's PhD thesis is often cited and is a good reading.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="1050" LastEditDate="2014-09-04T07:17:45.553" LastActivityDate="2014-09-04T07:17:45.553" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14557" PostTypeId="1" AcceptedAnswerId="14558" CreationDate="2014-09-03T20:35:49.667" Score="0" ViewCount="1147" Body="&lt;p&gt;I currently have a functioning and blazing fast model written in C++ and CUDA.  However, I'd like to use Scipy.minimize to fit the model to some experimental data.  I was hoping it would be easy, but when I try to describe my problem to Google, it keeps telling me that I want to extend Python with C++, the documentation of which makes me want to cry.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used to use PyCUDA back in the day, but I don't think I am able to wrap the entire model, so I need a little nudge in the right direction.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm trying to accomplish.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.optimize import minimize&#xA;import os&#xA;&#xA;#Initial starting parameters&#xA;ics = [ 1, 0, 0 ... 0]&#xA;&#xA;#The objective function is called from my model, which is a built .exe that will accept parameters as maybe a string and somehow output something&#xA;model = os.popen(&quot;directory/mymodel.exe&quot; + params)&#xA;&#xA;difference = minimize(model, ics, method='Nelder-Mead')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is something like this possible?  I'm having a hard time getting around the interface between C++ and Python and how I can make my .exe generate data interpretable by python so that Scipy can adjust the parameters and iterate.&lt;/p&gt;&#xA;" OwnerUserId="5553" LastActivityDate="2014-09-04T00:22:16.777" Title="How can I use Scipy to fit data generated from a C++ model?" Tags="|python|c++|scipy|cuda|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14558" PostTypeId="2" ParentId="14557" CreationDate="2014-09-04T00:22:16.777" Score="3" Body="&lt;p&gt;Other comments have suggested a file-based interface, using an actual C/C++ optimization library, or extending Python with C++.  Those are probably better ways to solve your problem, but here's a more narrow minded answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To begin, here's a C program called mymodel.c which implements the 2d Rosenbrock function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;#include &quot;stdio.h&quot;&#xA;#include &quot;assert.h&quot;&#xA;#include &quot;stdlib.h&quot;&#xA;&#xA;/* minimum is at (a, a*a) = (0.123, 0.015129) */&#xA;double rosenbrock_2d(double x, double y) {&#xA;  const double a = 0.123;&#xA;  const double b = 100.0;&#xA;  double u, v;&#xA;  u = a - x;&#xA;  v = y - x*x;&#xA;  return u*u + b*v*v;&#xA;}&#xA;&#xA;int main(int argc, char **argv)&#xA;{&#xA;  double x, y;&#xA;  assert(argc == 3);&#xA;  x = atof(argv[1]);&#xA;  y = atof(argv[2]);&#xA;  printf(&quot;%.16g\n&quot;, rosenbrock_2d(x, y));&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can build it using something like the following command.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;$ gcc -o mymodel.exe mymodel.c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's a python script named opt.py which repeatedly calls the executable implementing your model, while attempting to find the minimum of the function.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&#xA;from scipy.optimize import minimize&#xA;import subprocess&#xA;&#xA;#Initial starting parameters&#xA;ics = [ 1, 0]&#xA;&#xA;#The objective function is called from my model, which is a built .exe that will accept parameters as maybe a string and somehow output something&#xA;def model(x):&#xA;    params = ['%.16g' % a for a in x]&#xA;    s = subprocess.check_output(['./mymodel.exe'] + params)&#xA;    return float(s)&#xA;&#xA;print minimize(model, ics, method='Nelder-Mead')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then run it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;$ python opt.py                            &#xA;  status: 0&#xA;    nfev: 86&#xA; success: True&#xA;     fun: 2.2359521022451379e-09&#xA;       x: array([ 0.12296347,  0.01512302])&#xA; message: 'Optimization terminated successfully.'&#xA;     nit: 46&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives something close to the answer.&#xA;If you want a closer answer you can use different tolerances or a better method like 'L-BFGS-B'.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-09-04T00:22:16.777" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14559" PostTypeId="1" AcceptedAnswerId="14560" CreationDate="2014-09-04T12:59:36.727" Score="4" ViewCount="289" Body="&lt;p&gt;In Matlab R2013a I have implemented the Implicit Euler (time) integration scheme. To find the $x^{n+1}$ value I use fixed point iterations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x^{n+1} = \Delta t f(x^{n+1}) + x^n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To test this, I use an undamped mass-spring system, which should give a simple harmonic as result. However, I seem to loose a considerable amount of energy over time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this due to the fixed point iterations finding a 'wrong' fixed point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="10193" LastEditorUserId="10193" LastEditDate="2014-09-04T16:49:02.930" LastActivityDate="2014-09-04T17:10:08.603" Title="Non-conservative implementation implicit Euler" Tags="|ode|time-integration|implicit-methods|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14560" PostTypeId="2" ParentId="14559" CreationDate="2014-09-04T15:01:15.223" Score="13" Body="&lt;p&gt;This might seem extreme, but this can be analysed exactly. Take the system&#xA;$$ \dot x_1 = x_2, \qquad \dot x_2=-x_1, \qquad x_1(0) = 1, \qquad x_2(0)=0. $$&#xA;Let $X=(x_1,x_2)$ be the state vector, $\delta t$ the time step, and $X^+$ the state vector for the next time step. Then the implicit Euler scheme is&#xA;$$ X^+ = \delta t\left(\begin{array}{cc}0&amp;amp;1\\-1&amp;amp;0\end{array}\right) X^+ + X, $$&#xA;or&#xA;$$ X^+ = \frac{1}{1+\delta t^2}\begin{pmatrix}1&amp;amp;\delta t\\-\delta t&amp;amp;1\end{pmatrix}X = AX.&#xA;$$&#xA;Now, the conserved quantity is $\|X\|_2$, so take the matrix, call it $A$, that multiplies $X$ every time step, and consider how it changes the norm of $X$. Its 2-norm is&#xA;$$ \|A\|_2 = \frac{1}{(1+\delta t^2)^{1/2}}, $$&#xA;so if there are $n$ steps in a period of length $2\pi$, the reduction in $\|X\|_2$ is on the order of&#xA;$$ \left(1+\frac{4\pi^2}{n^2}\right)^{-n/2} \approx e^{-2\pi^2/n}, $$&#xA;where we would like this to be $1$.&#xA;For $n=100$, this is $0.82$, which is indeed drastic, but shouldn't be surprising in light of the mathematics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want correctness to $d$ digits after one period, the condition on $n$ is that $e^{-2\pi^2/n}\sim 1-10^{-d}$, or $n\sim 2\pi^210^d$. So to get even three digits of accuracy you need $20,000$ steps or so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conclusion: this is a serious issue with the method, so use a symplectic integrator that solves it.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-09-04T17:10:08.603" LastActivityDate="2014-09-04T17:10:08.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14561" PostTypeId="2" ParentId="13049" CreationDate="2014-09-04T17:28:54.703" Score="1" Body="&lt;p&gt;Some good references can be found on Gregory Beylkin's homepage &lt;a href=&quot;http://amath.colorado.edu/faculty/beylkin/papers/index.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://amath.colorado.edu/faculty/beylkin/papers/BEYLKI-1994b.pdf&quot; rel=&quot;nofollow&quot;&gt;G. Beylkin, On wavelet-based algorithms for solving differential equations&#xA;Chapter in the book  Wavelets: Mathematics and Applications, CRC Press, 1994 &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://amath.colorado.edu/faculty/beylkin/papers/BEY-KEI-1997.pdf&quot; rel=&quot;nofollow&quot;&gt;G. Beylkin and J. M. Keiser, On the Adaptive Numerical Solution of Nonlinear Partial Differential Equations in Wavelet Bases,  Journal of Computational Physics, vol. 132, pp. 233-259, 1997  &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://amath.colorado.edu/faculty/beylkin/papers/A-B-G-V-2002.pdf&quot; rel=&quot;nofollow&quot;&gt;B. Alpert, G. Beylkin, D. Gines,  and  L. Vozovoi, Adaptive Solution of Partial Differential Equations in Multiwavelet Bases,  Journal of Computational Physics,  v. 182, pp. 149-190, 2002 &lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2014-09-04T17:28:54.703" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14562" PostTypeId="1" CreationDate="2014-09-04T18:26:52.327" Score="2" ViewCount="360" Body="&lt;p&gt;I want to solve this equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;-\frac{1}{2}f''(x)+2a\ f(x)^3 = f(x)\mu&#xA;$$&lt;/span&gt;&#xA;One exact solution (there are a lot of different kinds) of this equation is &lt;span class=&quot;math-container&quot;&gt;$f(x) = f_\infty \tanh(\sqrt{2a}f_\infty x) $&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$2a f_\infty^2=\mu$&lt;/span&gt; ) but I want to plot an approximation of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the finite difference method to write the derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f''(x) = \frac{f(x+\Delta x)-2f(x)+f(x-\Delta x)}{(\Delta x)^2} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then I have this formula&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x+\Delta x) = 2f(x)-f(x-\Delta x) - 2\Delta x^2f(x)\mu + 4af(x)^3\Delta x^2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what boundary conditions to use. Can you suggest me something?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried solving the equation in the range &lt;span class=&quot;math-container&quot;&gt;$[-4,4]$&lt;/span&gt;, initializing &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; in this way&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x) = &#xA;\begin{cases}&#xA;-1\qquad -4 &amp;lt; x &amp;lt; 0\\&#xA;0\qquad x=0 \\&#xA;1 \qquad 0 &amp;lt; x &amp;lt; 4&#xA;\end{cases}&#xA;$$&lt;/span&gt;&#xA;but I get a costant solution &lt;span class=&quot;math-container&quot;&gt;$$f(x) =&#xA;\begin{cases}&#xA;-1\qquad -4 &amp;lt; x &amp;lt; 0 \\&#xA;1\qquad 0 &amp;lt; x &amp;lt; 4&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Improving the code the result is better&lt;/strong&gt;&#xA;I see that the correct formula is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;F(x,\Delta x) = \frac{1}{2}\left[f^{old}(x+\Delta x) + f^{old}(x-\Delta x) \right] + 2a\Delta x^2f^{old}(x) - 2a\Delta x^2f^{old}(x)^3&#xA;$$&lt;/span&gt;&#xA;that is different from the formula suggested by nicoguaro but it is the same he used in his code. My question is: How did you approximate the second derivative? Why my iteration isn't good?&#xA;Id i use a different step from &lt;span class=&quot;math-container&quot;&gt;$0.8$&lt;/span&gt; (as nicoguaro did) i see that the solution goes to the function &lt;span class=&quot;math-container&quot;&gt;$sign(x)$&lt;/span&gt;. Why? Is the method inconsistent?&#xA;&lt;img src=&quot;https://i.stack.imgur.com/SZ00t.png&quot; alt=&quot;grap&quot;&gt;&#xA;&lt;strong&gt;I used the suggest of nicoguaro and this is my c++ code. Here &lt;span class=&quot;math-container&quot;&gt;$g=2a$&lt;/span&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;using namespace std;&#xA;double norm(double * array,int n)&#xA;{&#xA;    double tmp;&#xA;    double norm;&#xA;    for (int i = 0; i &amp;lt; n; ++i)&#xA;    {&#xA;        tmp = *(array + i);&#xA;        norm += tmp*tmp;&#xA;    }&#xA;    return sqrt(norm);&#xA;}&#xA;&#xA;double norm(double * array1, double * array2,int n)&#xA;{&#xA;    double tmp;&#xA;    double norm;&#xA;    for (int i = 0; i &amp;lt; n; ++i)&#xA;    {&#xA;        tmp = *(array1 + i) - *(array2+i);&#xA;        norm += tmp*tmp;&#xA;    }&#xA;    return sqrt(norm);&#xA;}&#xA;&#xA;double * FD(double *&amp;amp; u, double&amp;amp; g, double&amp;amp; dx, double&amp;amp; mu, int&amp;amp; n)&#xA;{&#xA;    double * tmp = new double[n];&#xA;    *(tmp+n-1) = *(u+n-1);&#xA;    *(tmp)=*(u);&#xA;    for (int i = 1; i &amp;lt; n-1; ++i)&#xA;    {&#xA;        tmp[i] = (u[i+1]+u[i-1])/2+dx*dx*u[i]*mu - g*dx*dx*u[i]*u[i]*u[i];&#xA;    }&#xA;&#xA;    return tmp;&#xA;&#xA;}&#xA;double * dot(double *  u, double&amp;amp; a,int&amp;amp; n)&#xA;{&#xA;    double * tmp = new double[n];&#xA;    for (int i = 0; i &amp;lt; n; ++i) tmp[i] = u[i]*a;&#xA;    return tmp;&#xA;}&#xA;&#xA;double * sum(double * u1, double* u2, int n)&#xA;{&#xA;    double * tmp = new double[n];&#xA;    for (int i = 0; i &amp;lt; n; ++i) tmp[i] = u1[i]+u2[i];&#xA;    return tmp;&#xA;}&#xA;&#xA;int main(int argc, char * argv[])&#xA;{&#xA;    std::cout.setf( std::ios::fixed, std:: ios::floatfield );&#xA;    std::fstream fs;&#xA;    double dx;&#xA;    double xmin,xmax,temp,f0;&#xA;    double precision;&#xA;    double v,c,k,g,mu;&#xA;    int stop = 0;&#xA;    cout &amp;lt;&amp;lt; &quot;xmin: &quot;;&#xA;    cin &amp;gt;&amp;gt; xmin;&#xA;    cout &amp;lt;&amp;lt; &quot;xmax: &quot;;&#xA;    cin &amp;gt;&amp;gt; xmax;&#xA;    cout &amp;lt;&amp;lt; &quot;passo: &quot;;&#xA;    cin &amp;gt;&amp;gt; dx;&#xA;    cout &amp;lt;&amp;lt; &quot;precisione: &quot;;&#xA;    cin &amp;gt;&amp;gt; precision;&#xA;    int n = (xmax-xmin)/dx;&#xA;    cin &amp;gt;&amp;gt; f0; //initial condition in f[xmin]&#xA;    cin &amp;gt;&amp;gt; g; // g = 2a&#xA;    cin &amp;gt;&amp;gt; c; // c = f_infinity&#xA;&#xA;    mu = g*c*c; &#xA;&#xA;    double * fold = new double[n];&#xA;    double * fnew = new double[n];&#xA;    for (int i = 0; i &amp;lt; n; ++i)&#xA;    {&#xA;        if(i&amp;lt;n/2) fold[i]=f0;&#xA;        if(i&amp;gt;n/2) fold[i] = c;&#xA;        if(i==n/2) fold[i] = 0;&#xA;    }&#xA;&#xA;    double alpha = 0.3;&#xA;    double x = 1-alpha;&#xA;while(stop==0)&#xA;{&#xA;    fnew = sum(dot(fold,alpha,n),dot(FD(fold,g,dx,mu,n),x,n),n);&#xA;    if(norm(fold,fnew,n)/norm(fold,n) &amp;lt; precision ) stop = 1;&#xA;    else for (int i = 0; i &amp;lt; n; ++i) fold[i] = fnew[i];&#xA;}&#xA;&#xA;    fs.open(&quot;file.dat&quot;, std::fstream::in | std::fstream::out| fstream::app);&#xA;    for (int i = 0; i &amp;lt; n; ++i) fs &amp;lt;&amp;lt; (xmin+dx*i) &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; fold[i] &amp;lt;&amp;lt; endl;&#xA;&#xA;    fs.close();&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the core of the code is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while(stop==0)&#xA;{&#xA;    fnew = sum(dot(fold,alpha,n),dot(FD(fold,g,dx,mu,n),x,n),n);&#xA;    if(norm(fold,fnew,n)/norm(fold,n) &amp;lt; precision ) stop = 1;&#xA;    else for (int i = 0; i &amp;lt; n; ++i) fold[i] = fnew[i];&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8188" LastEditorUserId="22284" LastEditDate="2019-01-16T15:07:05.140" LastActivityDate="2019-01-16T15:07:05.140" Title="Solve a differential equation with finite difference method" Tags="|finite-difference|integral-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14563" PostTypeId="1" CreationDate="2014-09-04T21:47:07.587" Score="1" ViewCount="1006" Body="&lt;p&gt;I am writing a program that creates a vtk file with a Rectilinear Grid. I am able to save data to each one of the cells using CELL_DATA, but now, I need to save data to each one of the cell faces (x6). How do I do that? Will I will be able visualize it in ParaView?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: In the VTK file format, if one create a 3d rectilinear grid dataset, the default cell will be 3 dimensional (8 points). Is there a way to create 2D cells with a 3D rectilinear grid?&lt;/p&gt;&#xA;" OwnerUserId="9808" LastEditorUserId="9808" LastEditDate="2014-09-06T03:51:44.050" LastActivityDate="2014-10-07T21:34:59.730" Title="VTK File: How do I save data on a cell face?" Tags="|visualization|open-source|paraview|vtk|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14564" PostTypeId="2" ParentId="14562" CreationDate="2014-09-04T22:11:17.147" Score="3" Body="&lt;p&gt;I am turning my comments into this answer. First of all, the analytic expression that you proposed does not satisfy the differential equation. According to Maple, the solution is&#xA;$$f \left( x \right) ={\frac {k_{{2}}\sqrt {\mu}}{\sqrt {a{k_{{2}}}^{2}- a+\mu}}{\rm sn} \left( {\frac { \left( \sqrt {2\mu - 2a}\ x+k_{{1}}  \right) \sqrt {\mu}}{\sqrt {a{k_{{2}}}^{2}-a+\mu}}},{\frac {\sqrt {a} k_{{2}}}{\sqrt {\mu- a}}} \right) } \enspace ,$$&#xA;where $\rm{sn}$ is the sine-like &lt;a href=&quot;http://en.wikipedia.org/wiki/Jacobi_elliptic_functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jacobi elliptic function&lt;/a&gt;. The parameters $k_1$ and $k_2$ depend on the boundary conditions, let's choose $k_1=k_2=1$ that gives $f(-4) \approx -1$ and $f(4) \approx 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula that you used for the finite difference has a problem; the last term should not have the $2$ factor. I prefer another iteration, since yours just approximate the derivatives in one direction. My choose is this one&#xA;$$ FD(x,\Delta x) \equiv \frac{1}{2}\left[f^{\rm{old}}(x + \Delta x) + f^{\rm{old}}(x - \Delta x)\right] + \mu \Delta x^2 f^{\rm{old}}(x) - 2\Delta x^2 a f^{\rm{old}}(x)^3 \enspace ,$$&#xA;and update your function approximation&#xA;$$f^{\rm{new}}(x) = FD(x,\Delta x) \enspace ,$$&#xA;you can also do something like&#xA;$$f^{\rm{new}}(x) = \alpha f^{\rm{old}}(x) + (1-\alpha) FD(x,\Delta x), \quad \alpha \in [0,1] \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This Python code use this approach (although it don't reach the analytic value).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from  scipy.special import ellipj&#xA;&#xA;def FD_iter(u, a, mu, dx):&#xA;    n = len(u)&#xA;    un = np.zeros_like(u)&#xA;    un[0] = u[0]&#xA;    un[-1] = u[-1]&#xA;    alpha = 0.3&#xA;    for k in range(1, n-1):&#xA;        un[k] = u[k-1]/2 + u[k+1]/2 + dx**2*mu*u[k] - \&#xA;                2*dx**2*a*u[k]**3&#xA;&#xA;    return alpha*u + (1 - alpha)*un&#xA;&#xA;plt.close(&quot;all&quot;)    &#xA;n = 1001&#xA;a = 1.&#xA;mu = 2.&#xA;x = np.linspace(-4, 4, n)&#xA;u0 = np.tanh(x)&#xA;u0 = np.sign(u0)&#xA;plt.plot(x, u0, label=&quot;Initial&quot;)&#xA;&#xA;# This BC lead to a solution with (approx) u(1) = -1  u(4)=1.&#xA;k1 = 1&#xA;k2 = 1   &#xA;v = np.sqrt(mu)*(k1 + np.sqrt(2*mu - 2*a)*x)/np.sqrt(a*k2**2 - a + mu)&#xA;m = np.sqrt(a)*k2/np.sqrt(mu - a)&#xA;amp = k2*np.sqrt(mu)/np.sqrt(a*k2**2 -a + mu)&#xA;z = v*0&#xA;ellipj(v, m, z)&#xA;z = amp*z&#xA;plt.plot(x, z, label=&quot;Analytic&quot;)&#xA;&#xA;&#xA;dx = x[1] - x[0]&#xA;N = 10000&#xA;&#xA;tol = 1e-5&#xA;for k in range(N):&#xA;    u = FD_iter(u0, a, mu, dx)&#xA;    err = np.linalg.norm(u - u0)/np.linalg.norm(u0)&#xA;    if err&amp;lt;=tol:&#xA;        break&#xA;    u0 = u&#xA;&#xA;plt.plot(x, u, label=&quot;Final&quot;)&#xA;plt.ylim([-1.1, 1.1])&#xA;plt.grid(True, color=&quot;b&quot;, alpha=0.4)&#xA;plt.legend()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The curves look like&#xA;&lt;img src=&quot;https://i.stack.imgur.com/fHKou.png&quot; alt=&quot;Comparison of solutions&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2014-09-06T17:37:57.990" LastActivityDate="2014-09-06T17:37:57.990" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14566" PostTypeId="1" CreationDate="2014-09-05T17:03:35.820" Score="6" ViewCount="4384" Body="&lt;p&gt;I recently &lt;a href=&quot;https://mathematica.stackexchange.com/questions/41748/how-do-i-plot-a-proper-streamline-plot-including-spacings-and-line-endings&quot;&gt;had trouble producing a proper streamline plot in Mathematica&lt;/a&gt;, and apparently the problem is a good bit harder than I appreciated. I would like to know if there exist general algorithms to produce streamline plots which deal appropriately with line spacings and terminations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me make the problem specific. Consider a vector field $\vec F(\vec x)$, in two or in three dimensions. A streamline is an &lt;a href=&quot;http://en.wikipedia.org/wiki/Integral_curve&quot; rel=&quot;nofollow noreferrer&quot;&gt;integral curve of the vector field&lt;/a&gt;, that is, a curve $\vec\alpha(s)$ whose derivative $\frac{d\vec\alpha}{ds}$ is proportional at every point to the vector field $\vec F(\vec\alpha(s))$ at that point. A streamline plot represents a collection of such streamlines, and &lt;a href=&quot;https://physics.stackexchange.com/questions/82536/electric-field-lines-density&quot;&gt;it is an extremely useful tool for visualizing the field&lt;/a&gt;, and its direction in particular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, however, additional ways to make the diagrams convey information about the field's magnitude and its divergence.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If $\vec F$ is divergence-free (i.e. $\nabla\cdot\vec F=0$), then it is customary to draw streamlines that do not terminate unless they meet a zero or singularity of the field or go out of the plot, and to space the streamlines such that the vector field flow&#xA;$$ \int_L\vec F\cdot \text d\vec x$$&#xA;is constant for $L$ connecting any two adjacent streamlines. (In three dimensions this is harder to define appropriately, but the flow $ \int_S\vec F\cdot \text d\vec a$ should be constant over any 'unit cell' whose corners are streamlines.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This allows for a direct reading of the intensity of the vector field from the line spacing, which is guaranteed to be smaller where the field is more intense.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If $\vec F$ has a nonzero divergence, then it is still customary to keep the line spacing such that the flow between streamlines is approximately constant, though this requires that lines terminate or initiate in the middle of the diagram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This allows one to visualize regions with positive or negative divergence - volume sources and sinks of the field - as those in which streamlines initiate or terminate, respectively.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So: I would like to know if there exist algorithms that will take a vector field $\vec F$ and a region $R$, in two or three dimensions (mostly two), and which will produce a set of streamline termini such that the field flow in between streamlines is (approximately) constant? It should therefore have lines initiate and terminate at and only at regions with positive and negative field divergence.&lt;/p&gt;&#xA;" OwnerUserId="3551" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2015-03-30T14:02:36.020" Title="How can one produce a proper streamline plot?" Tags="|visualization|plotting|vector|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14568" PostTypeId="1" CreationDate="2014-09-05T19:21:13.947" Score="1" ViewCount="220" Body="&lt;p&gt;I am working on an eigenvalue problem in fortran. I have used Lapack to solve the problem and get the eigenvalues and eigenvectors. This is done for $201\times101$ wavenumbers, only half the wavespace due to symmetry, and for each grid-point in a large domain(in the ocean). I am searching for the maximum eigenvalue for each gridpoint, and I would like to not just pick the absolute maximum in the $201\times101$ matrix of eigenvalues, but perform an azimuthal average in wavespace - and then pick the maximum average. I am struggeling with seeing how to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first I coded it like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;! Wavenumber domain&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dx=4000.&#xA;pi = 4.*atan(1.)&#xA;&#xA;DO m=1,ktot &#xA;    kx(m) = -(2.*pi)/(dx) + ((m-1)*2.*pi)/(100.*dx)&#xA;END DO&#xA;&#xA;DO l=1,ltot&#xA;    ly(l)= ((l-1)*2.*pi)/(100.*dx)&#xA;END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;!Radial distance&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DO m=1,ktot&#xA;DO l=1,ltot&#xA;    raddist(m,l)=sqrt(kx(m)**2+ly(l)**2)&#xA;END DO&#xA;END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;! Azimuthal average (omegai are the eigenvalues I have found, a &lt;code&gt;ktot*ltot&lt;/code&gt; large matrix)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DO i=1,ltot-1   &#xA;    ind=(raddist(:,i).GE.ly(i).AND.raddist(:,i).LT.ly(i+1))&#xA;    length=count(ind)&#xA;    WHERE (ind) average_omegai = sum(omegai)/length&#xA;END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it seems I am summing a horizontal chunk for all wavenumbers in k-direction, between &lt;code&gt;ly(i) and ly(i+1)&lt;/code&gt;. I rather need to make a semicircle in wavespace to sum all omegai-values in between it. Can anyone help with this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks so much in advance!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(BTW: I also tried &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; DO m=1,ktot-1&#xA; DO l=1,ltot-1   &#xA;    ind=(raddist(m,l).GE.ly(l).AND.raddist(m,l).LT.ly(l+1))&#xA;    length=count(ind)&#xA;    WHERE (ind) average_omegai = sum(omegai)/length&#xA; END DO&#xA; END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I right now I am blind to what I am doing wrong! My problem stated differently would be: is there another way in Fortran to find indexes in a matrix which fulfill some criteria? The find-function in Matlab would do the work, but I have not found an equivalent intrinsic function in Fortran? )&lt;/p&gt;&#xA;" OwnerUserId="10202" LastEditorUserId="10134" LastEditDate="2014-09-07T21:31:44.580" LastActivityDate="2015-07-05T02:48:56.067" Title="Azimuthal average in Fortran? Find indexes in Fortran?" Tags="|fortran|matrix|eigenvalues|lapack|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14569" PostTypeId="1" AcceptedAnswerId="14623" CreationDate="2014-09-05T19:50:20.183" Score="2" ViewCount="1414" Body="&lt;p&gt;Lets assume I have a iterative linear system solver, e. g. &lt;a href=&quot;http://math.nist.gov/iml++/bicgstab.h.txt&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whats the typical approach on verifying and testing this kind of solvers? Is there a standard test set of linear systems one could use to verify the above BiCGStab-Implementation and if not, how would one create such test data?&lt;/p&gt;&#xA;" OwnerUserId="10201" LastActivityDate="2014-09-15T03:37:40.920" Title="Test set for linear solvers" Tags="|linear-solver|iterative-method|testing|data-sets|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14570" PostTypeId="2" ParentId="14568" CreationDate="2014-09-05T20:38:05.033" Score="1" Body="&lt;p&gt;I haven't spent much time looking at your code, but I would approach this in a slightly different way. Rather than asking which wave numbers correspond to a given radius range, I would divide the space into the (radial) bins you want and then loop over the wave vectors asking which bin contains each wave vector (e.g. using integer division). By summing the values in each bin and counting how many are in each bin you can easily compute the averages. This should be more efficient too.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-09-05T20:38:05.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14571" PostTypeId="2" ParentId="14569" CreationDate="2014-09-06T02:44:51.833" Score="1" Body="&lt;p&gt;Have you looked at the Matrix Market?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://math.nist.gov/MatrixMarket/&quot; rel=&quot;nofollow&quot;&gt;http://math.nist.gov/MatrixMarket/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-09-06T02:44:51.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14572" PostTypeId="2" ParentId="14568" CreationDate="2014-09-06T09:59:20.773" Score="1" Body="&lt;p&gt;To find indices in Fortran arrays: create your sellecting mask, and use &lt;a href=&quot;https://gcc.gnu.org/onlinedocs/gfortran/MAXLOC.html&quot; rel=&quot;nofollow&quot;&gt;MAXLOC&lt;/a&gt; or MINLOC&lt;/p&gt;&#xA;" OwnerUserId="10197" LastActivityDate="2014-09-06T09:59:20.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14573" PostTypeId="1" AcceptedAnswerId="14575" CreationDate="2014-09-06T16:56:27.760" Score="4" ViewCount="340" Body="&lt;p&gt;Is there an analog of perfectly matched layers for finite element methods? References or small examples are much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="7733" LastEditDate="2014-09-08T01:32:06.937" LastActivityDate="2014-09-23T18:33:18.150" Title="Analog of perfectly matched layers for finite element methods" Tags="|finite-element|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14574" PostTypeId="2" ParentId="14370" CreationDate="2014-09-06T18:29:17.877" Score="2" Body="&lt;p&gt;Well, it seems that there is a size requirement to get a good resolution&#xA;I'm not sure if that is the properly way to solve this, but it worked for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I did was to interpolate the x and y components of the vector field. Talking about Python, this can be done using the &lt;code&gt;scipy.ndimage.zoom&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, I had this vector field (100 x 100):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2xw6I.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which results in this LIC image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Bfi4Z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using an interpolation with factor 4, I'm able to get this result from LIC: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/SLDWk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(:&lt;/p&gt;&#xA;" OwnerUserId="9799" LastEditorUserId="9799" LastEditDate="2019-02-28T18:03:17.690" LastActivityDate="2019-02-28T18:03:17.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14575" PostTypeId="2" ParentId="14573" CreationDate="2014-09-06T23:03:18.703" Score="12" Body="&lt;p&gt;A perfectly matched layer (PML) is generally introduced as a continuous concept without reference to the discretization method. A damping term is multiplied by the advection operator:&#xA;$$&#xA;\frac{\partial}{\partial x} \rightarrow \frac{\partial}{\partial x}\left(\frac{1}{1 + i\sigma/\omega}\right)&#xA;$$&#xA;If $\sigma$ is positive, the imaginary component in the above equation drives the standard oscillatory solution of a wave equation to be exponentially decaying. A simple way of implementing this would just modify the discretized equation everywhere in space and then assign the value of $\sigma$ to be zero everywhere except near the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, a PML could be implemented in any volumetric discretization like finite volumes, finite elements or finite differences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an introduction, I like these &lt;a href=&quot;http://math.mit.edu/~stevenj/18.369/pml.pdf&quot;&gt;notes&lt;/a&gt;. Also, try this &lt;a href=&quot;http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&amp;amp;arnumber=504304&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5516" LastActivityDate="2014-09-06T23:03:18.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14576" PostTypeId="1" AcceptedAnswerId="14577" CreationDate="2014-09-07T01:06:57.160" Score="1" ViewCount="174" Body="&lt;p&gt;I'm currently searching for an elaborate referece that covers most of the approximation errors for elliptic second order problems (like, for the laplacian dirichlet problem) by using finite element methods in 2D and 3D. &lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in 2D, you can receive an optimal error estimate:&#xA;$$ \| u - R_hu \|_{H^1} = \mathcal{O}(h^r), $$&#xA;where $R_h: V \to V_h$ is the Ritz-Projector (aka Galerkin-Projector) from the conformal space $V$, where the variational problem is stated, into the FE-space $V_h$ of piecewise polynomials of degree $r$ (with proper zero conditions on the right boundary edges) on a mesh of mesh-size $h$. Of course, this only works if we assume the solution to be regular enough, i.e. $u \in H^{r+1} \cap V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At university, i worked primary with the book of Braess, which is pretty much the standard literature in germany, but it only covers the cases in 2D. I assume similar results can be obtained in 3D, but might get a bit complicated.&lt;/p&gt;&#xA;" OwnerUserId="10005" LastEditorUserId="10005" LastEditDate="2014-09-07T01:15:05.737" LastActivityDate="2014-09-08T18:56:32.773" Title="Reference for approximation errors in 2D and 3D by using FEM" Tags="|reference-request|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14577" PostTypeId="2" ParentId="14576" CreationDate="2014-09-07T12:50:21.987" Score="2" Body="&lt;p&gt;For standard finite elements for elliptic problems we often find no fundamental difference in the n-dimensional analysis (n=2,3,...), since most parts are dimension-agnostic, provided the elements under consideration have optimal approximation properties and we have sufficient regularity (which may be a complicated matter for higher spatial dimensions).  Because the regularity of solutions is linked to the type of the domain, some authors tend to fix n=2 and/or certain types of computational domains (e.g. (convex) polygonal) to simplify the presentation. Usually they make a remark that the analysis carries over to more general cases straightforwardly if the regularity is sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A classical reference covering the n-dimensional case would be e.g. Ciarlet's 1978 book &quot;The Finite Element Method for Elliptic Problems&quot;. There are also contributions by Ciarlet and Lions in the &quot;Handbook of Numerical Analysis&quot; series that cover the n-dimensional case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sufficient regularity is often not proved in the finite element literature but rather either assumed or referenced (e.g. in Grisvards 1985 book &quot;Elliptic Problems in Nonsmooth Domains&quot; or the book by Gilbarg and Trudinger which Wolfgang mentioned).&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2014-09-08T09:54:54.197" LastActivityDate="2014-09-08T09:54:54.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14578" PostTypeId="1" AcceptedAnswerId="14629" CreationDate="2014-09-07T17:00:54.377" Score="4" ViewCount="1318" Body="&lt;p&gt;I would like to get to know more in details about Lax-Richtmyer stability analysis (esp in examples), but I didn't manage to find anything except a definition. Could you advice any sources for this subject?&lt;/p&gt;&#xA;" OwnerUserId="4676" LastActivityDate="2014-09-15T16:26:32.343" Title="Lax-Richtmyer stability analysis" Tags="|stability|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14579" PostTypeId="1" CreationDate="2014-09-07T20:10:37.687" Score="1" ViewCount="1512" Body="&lt;p&gt;How can I calculate the machine epsilon for two numbers then calculate the theoretical limit for machine epsilon in Matlab ?&lt;/p&gt;&#xA;" OwnerUserId="10212" LastEditorUserId="4964" LastEditDate="2014-09-07T21:31:52.187" LastActivityDate="2014-09-07T21:31:52.187" Title="Calculate the machine epsilon in Matlab" Tags="|matlab|precision|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14580" PostTypeId="2" ParentId="14579" CreationDate="2014-09-07T20:34:21.413" Score="2" Body="&lt;p&gt;So, machine epsilon is an upper bound on the relative error due to roundoff. You can calculate this quantity using the &lt;code&gt;eps&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; eps&#xA;        2.2204e-16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to find the distance between a floating point number &lt;code&gt;x&lt;/code&gt; and the next nearest floating point number, you would call &lt;code&gt;eps(x)&lt;/code&gt; instead.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-09-07T20:45:24.313" LastActivityDate="2014-09-07T20:45:24.313" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14581" PostTypeId="2" ParentId="14563" CreationDate="2014-09-07T21:28:58.810" Score="1" Body="&lt;p&gt;The way this is done in the deal.II library (&lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org&lt;/a&gt;; disclaimer: this is a project I am affiliated with) is that we describe an unstructured data set of 3d points and then create cells that happen to be two-dimensional. In other words, just output a set of quadrilaterals that happen to live in a three-dimensional space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give an example, here is a file that outputs a set of lines in a 2d space: &lt;a href=&quot;https://github.com/bangerth/dealii/blob/master/tests/codim_one/data_out_02.output&quot; rel=&quot;nofollow&quot;&gt;https://github.com/bangerth/dealii/blob/master/tests/codim_one/data_out_02.output&lt;/a&gt; . You'd need to do something similar for 2d quads in a 3d space.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-07T21:28:58.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14582" PostTypeId="2" ParentId="14573" CreationDate="2014-09-07T21:31:06.377" Score="1" Body="&lt;p&gt;Take a look at some of the papers by my colleague Joe Pasciak on the PML and its approximation: &lt;a href=&quot;http://www.math.tamu.edu/~joe.pasciak/&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~joe.pasciak/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-07T21:31:06.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14583" PostTypeId="2" ParentId="14576" CreationDate="2014-09-07T21:42:13.187" Score="1" Body="&lt;p&gt;To prove a convergence rate, there are two steps you need to be able to do: first, you need to be able to show that the Galerkin error can be bounded by a fixed multiple of the interpolation error:&#xA;$$&#xA;  \|u-R_h u\|_{H^1} \le C \inf_{\varphi_h \in V_h} \| u-\varphi_h \|_{H^1}.&#xA;$$&#xA;For the Laplace equation, this step is trivial and does not depend on the space dimension. Then, one of course also has&#xA;$$&#xA;  \|u-R_h u\|_{H^1} \le C \| u-I_h u \|_{H^{r+1}}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second step is to show the interpolation estimate&#xA;$$&#xA;  \|u-I_h u\|_{H^1} \le C h^{r} \| u \|_{H^{r+1}}.&#xA;$$&#xA;This step, conceptually, only requires something like a Taylor expansion to degree $r$ and consideration of the remainder term. However, it is often shown in a more abstract way where, again, you do not really make use of the space dimension nor that $u$ is actually continuous -- you only need that the solution is in $H^{r+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when is $u\in H^{r+1}$? You can find questions like this answered in books like the one by Gilbarg and Trudinger. But to provide the most important case already: if the domain is a convex polyhedron, then $u\in H^2$, in any space dimension.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="4260" LastEditDate="2014-09-08T18:56:32.773" LastActivityDate="2014-09-08T18:56:32.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14584" PostTypeId="2" ParentId="8367" CreationDate="2014-09-07T22:23:37.293" Score="0" Body="&lt;p&gt;I would suggest using &lt;a href=&quot;http://docs.enthought.com/mayavi/mayavi/index.html&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;mayavi&lt;/strong&gt;&lt;/a&gt; (distributed in &lt;a href=&quot;https://code.google.com/p/pythonxy/&quot; rel=&quot;nofollow&quot;&gt;pythonxy&lt;/a&gt; for instance) for this task. Assuming the vtk file is &lt;code&gt;my_input_file.vtk&lt;/code&gt; and you wish to display a vector field in a cut plane using glyphs then save as png image, the following methods do the trick:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Save your vtk as an image directly from the terminal:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ mayavi2 -d my_input_file.vtk -m VectorCutPlane -f MaskPoints -m Glyph \&#xA;           -s &quot;scene.save('my_image.png')&quot;&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or, alternatively wrap things up in an executable shell script &lt;code&gt;mayavi_script.sh&lt;/code&gt; containing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;mayavi2 -d my_input_file.vtk \&#xA;         -m VectorCutPlane \&#xA;         -f MaskPoints \&#xA;         -m Glyph \&#xA;         -s &quot;scene.save('my_image.png')&quot;&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the both cases, the switches &lt;code&gt;-d&lt;/code&gt; , &lt;code&gt;-m&lt;/code&gt;, &lt;code&gt;-f&lt;/code&gt; and &lt;code&gt;-s&lt;/code&gt; stand respectively for datafile, &lt;a href=&quot;http://docs.enthought.com/mayavi/mayavi/modules.html#modules&quot; rel=&quot;nofollow&quot;&gt;module&lt;/a&gt;, &lt;a href=&quot;http://docs.enthought.com/mayavi/mayavi/filters.html#filters&quot; rel=&quot;nofollow&quot;&gt;filter&lt;/a&gt; and executable python expression. This is only an example, but many other things can be done after checking the documentation. The &lt;code&gt;-o&lt;/code&gt; is &lt;strong&gt;very useful to parse the commands in off-screen mode&lt;/strong&gt;, therefore no GUI will be opened. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For more advanced offscreen visualization with image output, you can write and python-run a mayavi script (in the same idea suggested by Andrew T. Barker's answer for Visit) using the &lt;a href=&quot;http://docs.enthought.com/mayavi/mayavi/mlab.html&quot; rel=&quot;nofollow&quot;&gt;mlab&lt;/a&gt; and its pipeline module:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;import mayavi.mlab as ml&#xA; ml.options.offscreen = True&#xA; ...&lt;/code&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2014-09-08T00:23:05.543" LastActivityDate="2014-09-08T00:23:05.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14585" PostTypeId="1" CreationDate="2014-09-08T11:04:28.633" Score="1" ViewCount="126" Body="&lt;p&gt;I want to check numerically if a certain vector relation like&#xA;$$ \alpha_1v_1+...+\alpha_kv_k=c \ (1)$$&#xA;holds (where $v_i,c$ are vectors of $100$ or more components). For this, I use least squares approximation, and to see the error, I evaluate the norm of the difference &#xA;$$ \|\alpha_1v_1+...+\alpha_kv_k-c\|.$$&#xA;All $v_i$ and $c$ are computed numerically, so they are subject to errors. The size of the vectors is $100$ components. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When can I say that the error is small enough, i.e. the relation $(1)$ holds numerically? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I ask the question because I obtain errors between $0.2$ and $1$, which seem rather large, but if we look component wise, the average error is small.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="3548" LastEditDate="2014-09-08T12:07:06.747" LastActivityDate="2014-09-09T03:39:13.263" Title="When are two vectors considered &quot;close&quot;?" Tags="|error-estimation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14586" PostTypeId="2" ParentId="14585" CreationDate="2014-09-08T12:33:22.603" Score="1" Body="&lt;p&gt;In &lt;code&gt;scipy.numpy&lt;/code&gt; there is the function &lt;a href=&quot;http://docs.scipy.org/doc/numpy-1.7.0/reference/generated/numpy.allclose.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;allclose&lt;/code&gt;&lt;/a&gt; to compare two arrays. Have a look at the default values there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is problem dependent what you can call numerically close... Apart from the rounding error you may need to consider approximation errors and the like. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, there is a lot of norms you can take. Often the problem comes with a norm. In finite element computations, one typically takes the euclidean norm weighted with a mass matrix &lt;code&gt;M&lt;/code&gt;. Then, &lt;code&gt;M&lt;/code&gt; weights the components of the error vector properly.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-09-08T12:33:22.603" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14587" PostTypeId="1" CreationDate="2014-09-08T13:02:24.323" Score="1" ViewCount="224" Body="&lt;p&gt;Do anyone know what is the algorithm that MATLAB used in its built-in function &amp;quot;pca&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;I have the following data set:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;148.9820   55.8438  210.2150&lt;/p&gt;&#xA;&lt;p&gt;149.3030   56.8891  208.4280&lt;/p&gt;&#xA;&lt;p&gt;151.4400   55.8180  208.9240&lt;/p&gt;&#xA;&lt;p&gt;146.5530   55.9677  211.5800&lt;/p&gt;&#xA;&lt;p&gt;146.5770   57.2682  209.3680&lt;/p&gt;&#xA;&lt;p&gt;145.5330   58.4735  207.6970&lt;/p&gt;&#xA;&lt;p&gt;153.9680   55.8386  207.9600&lt;/p&gt;&#xA;&lt;p&gt;143.6960   57.2371  211.1020&lt;/p&gt;&#xA;&lt;p&gt;152.5960   57.3995  206.2770&lt;/p&gt;&#xA;&lt;p&gt;144.1070   56.1439  212.9730&lt;/p&gt;&#xA;&lt;p&gt;149.6670   58.8746  205.1560&lt;/p&gt;&#xA;&lt;p&gt;142.8440   58.1240  209.7220&lt;/p&gt;&#xA;&lt;p&gt;143.2190   59.3990  207.0410&lt;/p&gt;&#xA;&lt;p&gt;146.3050   60.2445  204.1980&lt;/p&gt;&#xA;&lt;p&gt;156.7100   55.9361  207.4610&lt;/p&gt;&#xA;&lt;p&gt;141.0470   57.3240  212.4660&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;where the number of rows are number of observations and each observation is of dimension 3.&lt;/p&gt;&#xA;&lt;p&gt;I want to perform principal component analysis on this data set so I wrote&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; P = pca(A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where A is the above matrix. The answer I got is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    0.9480    0.2104    0.2387&#xA;&#xA;   -0.0980   -0.5204    0.8483&#xA;&#xA;   -0.3027    0.8276    0.4727&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I use the following program:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;function [evects,evals] = pca_test(dataset)&#xA;&#xA;if (size(dataset,1)&amp;gt;size(dataset,2))&#xA;&#xA;    dataset = dataset';&#xA;&#xA;end&#xA;&#xA;N = size(dataset,2);&#xA;&#xA;mm = mean(dataset,2);&#xA;&#xA;dataset = dataset - mm*ones(1,N);&#xA;&#xA;cc = cov(dataset',1);&#xA;&#xA;[cvv,cdd] = eig(cc);&#xA;&#xA;[~,ii] = sort(diag(cdd));&#xA;&#xA;ii = flip(ii,1);&#xA;&#xA;evects = cvv(:,ii);&#xA;&#xA;cdd = diag(cdd);&#xA;&#xA;evals = cdd(ii);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it gives&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;evects = &#xA;&#xA;-0.9480    0.2104    0.2387&#xA;0.0980   -0.5204    0.8483&#xA;0.3027    0.8276    0.4727    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first column is of opposite sign to the result generated by the built-in pca. Why is there such a change?&lt;/p&gt;&#xA;&lt;p&gt;I ask this because I think the matlab built-in pca is really slow. The pca_test above is around 3 times faster than the built-in function. But I want it to have exactly the same result as the built-in one. Can anyone help?&lt;/p&gt;&#xA;" OwnerUserId="10218" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-09-10T02:47:20.137" Title="What is the algorithm that matlab used in its built-in function 'pca'?" Tags="|linear-algebra|algorithms|matlab|statistics|approximation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14588" PostTypeId="1" CreationDate="2014-09-08T19:38:27.583" Score="1" ViewCount="322" Body="&lt;p&gt;I want to solve&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial^2}{\partial t^2}u(z,t) + a\frac{\partial^2}{\partial z^2}u(z,t) + k\frac{\partial^4}{\partial z^4}u(z,t) = 0 &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$u(z,0) = 1+0.1e^{-\frac{z^2}{2}}$&lt;/span&gt;. I'd like to study the dynamic of the wave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the simplest way to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried with the finite difference method but my solution diverges very quickly!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use this iteration&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_j^{n+1} = 2f_j^n - f_j^{n-1} + \frac{\Delta t^2}{\Delta z^2}a(f_{j+1}^n - 2f_j^{n} + f_{j-1}^n) +k \frac{\Delta t^2}{\Delta z^4}(f^n_{j-2}-4f^n_{j-1}+6f^n_{j}-4f^n_{j+1} + f^n_{j+2})&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; it's the time step and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; the space step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Improving my question after comments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used a very small time step and this is what I obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/F3V2i.png&quot; alt=&quot;shock&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can avoid the reflection with this iteration method?&lt;/p&gt;&#xA;" OwnerUserId="8188" LastEditorUserId="22284" LastEditDate="2019-01-14T20:32:11.270" LastActivityDate="2019-01-14T20:32:11.270" Title="Solve a fourth order differential equation" Tags="|finite-difference|integral-equations|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="14589" PostTypeId="2" ParentId="14585" CreationDate="2014-09-09T03:36:58.733" Score="3" Body="&lt;p&gt;You want to compare two scalars $a=\sum_k \alpha_k v_k$ and $c$ for &quot;equality&quot;. This, of course, does not work in floating point arithmetic, but what one often does is to see whether the difference is small &lt;em&gt;compared to the sizes of the two individual components&lt;/em&gt;. In other words, $a$ and $c$ would be close if the relationship&#xA;$$&#xA; |a-c| \le \varepsilon \frac{|a|+|c|}{2}&#xA;$$&#xA;is true for a suitably chosen $\varepsilon$. Here, you compare the difference against the average size of the objects you take the difference of. A typical size for $\varepsilon$ would be $10^{-12}$ if you deal with floating point numbers: small enough that it really counts as small, but at the same large enough compared to the floating point roundoff error of $10^{-16}$ that you can afford a reasonable number of operations in the computations of $a$ and $c$ (where every one of these operations would of course accumulate to the numerical error in these quantities).&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="276" LastEditDate="2014-09-09T03:39:13.263" LastActivityDate="2014-09-09T03:39:13.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14590" PostTypeId="1" CreationDate="2014-09-09T07:17:11.507" Score="0" ViewCount="657" Body="&lt;p&gt;What is the state of the art for fastest linear solver for sparse, positive semi definite and strictly diagonally dominant matrix with N varies from ~700 to ~3000, and about a 1/16 of the matrix is non zero values?&lt;br&gt;&#xA;(an existing library is preferred over an algorithm - C/C++).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="10226" LastActivityDate="2014-09-09T07:36:54.480" Title="Fastest linear solver for sparse positive semidefinite, striclty diagonally dominant matrix" Tags="|linear-solver|sparse-matrix|c++|c|semidefinite-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14591" PostTypeId="2" ParentId="14590" CreationDate="2014-09-09T07:36:54.480" Score="1" Body="&lt;p&gt;There are two major linear algebra packages for &#xA;scientific computing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trilinos: &lt;a href=&quot;http://trilinos.org/&quot; rel=&quot;nofollow&quot;&gt;http://trilinos.org/&lt;/a&gt; it si purely object oriented c++.&#xA;This is a collection of packages, the one I would start with is Epetra. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc: Not fully object oriented, c. &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lately, I am working great with the linear algebra facilities in&#xA;dealII, (OO library for finite elements, c++). &lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thought, if the dimension of your problem is that small, &#xA;you probably do not need a compiled language. &#xA;Python will do!&lt;/p&gt;&#xA;" OwnerUserId="6574" LastActivityDate="2014-09-09T07:36:54.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14592" PostTypeId="2" ParentId="14587" CreationDate="2014-09-09T10:17:08.337" Score="2" Body="&lt;p&gt;Principal Components Analysis (PCA) is conducted using a Singular Value Decomposition (SVD) algorithm. As Bill Barth says above, the choice of sign of the principal component vectors is entirely arbitrary.&lt;/p&gt;&#xA;" OwnerUserId="9810" LastActivityDate="2014-09-09T10:17:08.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14593" PostTypeId="1" CreationDate="2014-09-09T12:21:14.870" Score="3" ViewCount="1815" Body="&lt;p&gt;I need to find a numerical solution for $-\Delta U = f$, on the $\Omega = [0,1]^2$, with $ U|_{\partial \Omega} = 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a method: &lt;code&gt;POISSONFD&lt;/code&gt; in &lt;code&gt;NMLibforOctave&lt;/code&gt;&#xA;with signature: &lt;code&gt;[U,X,Y]=POISSONFD(A,C,B,D,NX,NY,FUN,BOUND)&lt;/code&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I can't understand how I can specify bound. Something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;BOUND = inline('x * y * (y - 1) * (x - 1)', 'x', 'y')&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give good values on the border (I hope). can I use it? Or it's going to be bad? Or may be I should use some other package?&lt;/p&gt;&#xA;" OwnerUserId="10227" LastEditorUserId="10227" LastEditDate="2014-09-09T12:37:55.523" LastActivityDate="2014-09-11T10:13:32.727" Title="Partial differential equations with octave" Tags="|pde|numerical-analysis|octave|" AnswerCount="1" CommentCount="4" ClosedDate="2014-09-15T17:16:33.217" ContentLicense="CC BY-SA 3.0" />
  <row Id="14595" PostTypeId="1" CreationDate="2014-09-10T04:50:01.087" Score="6" ViewCount="623" Body="&lt;p&gt;I have to compute the accuracy of a new Gaussian mixture fitting algorithm. One of the tests include computing the probabilities in certain intervals in a 6D hyperspace. Also, the integral of the square of the difference between PDF has to be evaluated. But problem is, Matlab doesn't have library functions that handle integrals above 3 dimensions. Googling, I found several results that involve some sort of nesting using wither quad2d or integral2/integral3. Unfortunately, my attempts to implement nesting haven't been successful. All I have to do is to evaluate the Integral of $(N_{1}(x,\mu_{1},P_{1})-N_{2}(x,\mu_{2},P_{2}))^{2}$ over any given interval in $R^{6}$. Thank you for suggestions&lt;/p&gt;&#xA;" OwnerUserId="10232" LastEditorUserId="9667" LastEditDate="2014-09-10T19:13:15.780" LastActivityDate="2015-01-02T17:53:26.780" Title="Evaluating 6D Gaussian Integral in Matlab" Tags="|matlab|numerics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14596" PostTypeId="2" ParentId="14595" CreationDate="2014-09-10T11:17:26.283" Score="6" Body="&lt;p&gt;In general, the standard quadrature codes do not perform well in higher dimensions, which is probably why they are not available in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would propose that you use &lt;a href=&quot;http://en.wikipedia.org/wiki/Monte_Carlo_integration&quot;&gt;Monte Carlo Integration&lt;/a&gt; instead, which works very well in these cases. Here you randomly sample points of your function in order to evaluate the integral.&#xA;In particular, you probably want to use &lt;a href=&quot;http://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method&quot;&gt;Quasi Monte Carlo&lt;/a&gt; integration, as the convergence is faster. The only difference is that the function is not sampled randomly, but quasi-randomly. You can generate quasi-random numbers in &lt;a href=&quot;http://www.mathworks.se/help/stats/generating-quasi-random-numbers.html&quot;&gt;Matlab very easily&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple way to estimate the precision is to simply repeat the integration twice, and look at the difference. However there are also &lt;a href=&quot;http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Online_algorithm&quot;&gt;online algorithms&lt;/a&gt; that allow you to estimate the variance as part of the Monte Carlo integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, when the function is well behaved it is often sufficient to just use a large number of points and not worry unduly.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2014-09-10T11:17:26.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14598" PostTypeId="2" ParentId="14595" CreationDate="2014-09-10T20:31:06.070" Score="3" Body="&lt;p&gt;One approach to higher dimensional integrals is to use a sparse grid quadrature method (smolyak, clenshaw-curtis, etc...).  The idea is to use a small portion of a tensor product of 1D quadrature rules such that certain multivariate monomials are accounted for in the final integration rule.  If the integrand is reasonably approximated by a low dimensional polynomial function, sparse grid quadrature work really well.  In which case, various matlab packages are available for this purpose:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/19063-sparse-grid-quadrature&quot; rel=&quot;nofollow&quot;&gt;Matlab's Sparse Grid Quadrature Package&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sparse-grids.de/&quot; rel=&quot;nofollow&quot;&gt;www.sparse-grids.de&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more (introductory) information about sparse grids, you may want to look at some of &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/presentations/sparse_2014_fsu.pdf&quot; rel=&quot;nofollow&quot;&gt;John Burkardt's Presentations&lt;/a&gt; on high dimensional integration.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-09-10T20:31:06.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14599" PostTypeId="2" ParentId="14593" CreationDate="2014-09-11T10:13:32.727" Score="0" Body="&lt;p&gt;I implemented finite difference scheme myself and solved resulting linear equations system with octave using embedded methods. &lt;/p&gt;&#xA;" OwnerUserId="10227" LastActivityDate="2014-09-11T10:13:32.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14600" PostTypeId="1" CreationDate="2014-09-11T11:18:57.377" Score="1" ViewCount="64" Body="&lt;p&gt;I am facing the following problem. I know nonlinear least squares can provide a solution but I am wondering if a linear way to solve this data fitting problem may exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my input dataset:&#xA;I've got three different dataset composed of scattered points&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/vQneU.png&quot; alt=&quot;Input dataset&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know a linear equation in the form &#xA;$$&#xA;y(x) = bx^2 + cx&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can be used to explain any of my dataset. I know how to fit this function to a given dataset (e.g the blue one) linearly, so  could the above function separately for each dataset, but I am looking for something different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the specific case I have an additional constrain: I also know that the three functions describing the three dataset share the $b$ and $c$ parameters, while they differ by a multiplier, such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{cases}&#xA;y(x) = (bx^2 + cx) \; \text{explains the blue data}\\&#xA;y(x) = a'(bx^2 + cx) \; \text{for red data}\\&#xA;y(x) = a''(bx^2 + cx) \; \text{for green data}&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a (if it exists) linear way to solve for $a', a'', b$ and $c$ minimizing the overall sum of squares for the three functions.&#xA;Any idea on how this problem can be approached?&#xA;Also approximate solutions are welcome...&lt;/p&gt;&#xA;" OwnerUserId="10239" LastActivityDate="2014-09-11T11:18:57.377" Title="linear solution of curve fitting on multiple linear functions differing by a multiplier" Tags="|linear-algebra|regression|least-squares|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14603" PostTypeId="1" CreationDate="2014-09-11T14:59:36.187" Score="4" ViewCount="108" Body="&lt;p&gt;Is there a general approach to finding self-similar solutions; i.e. collapsing several functions onto a single function by some transformation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have data from some experiments, and the functions generated clearly appear to have some kind of self-similar solution. They are the result of varying a small number (4) of dimensionless parameters, and I'd like to find how (or if) I can collapse these onto a single function.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-09-11T19:15:15.490" Title="Finding self-similar solutions" Tags="|curve-fitting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14604" PostTypeId="1" CreationDate="2014-09-11T15:35:45.897" Score="1" ViewCount="189" Body="&lt;p&gt;I have a system of equations of the form: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ l_i^T l_j \cdot m_i^T m_j - m_i^T R l_j \cdot l_i R^T m_j = 0$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$R \in \mathbb{R}^{3\times3}$&lt;/span&gt; is an unknown rotation matrix.&#xA;&lt;span class=&quot;math-container&quot;&gt;$l_i, l_j, m_i, m_j \in \mathbb{R}^{3}$&lt;/span&gt; are known vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation can also be rewritten:&#xA;&lt;span class=&quot;math-container&quot;&gt;$ r^T Q r = 0 $&lt;/span&gt; where&#xA;&lt;span class=&quot;math-container&quot;&gt;$ r \in \mathbb{R}^9 $&lt;/span&gt;&#xA;is the rotation matrix written in vector form and &lt;span class=&quot;math-container&quot;&gt;$Q \in \mathbb{R}^{9\times9}$&lt;/span&gt; is a known symmetric matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to find &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; if possible in general form, else numerically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is a rotation, only 3 variables need to be found and thus 3 equations of the previous form should be enough. Do you think that any software (Mathematica, Maple, Sage, Sympy...) could solve this system in general form?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already tried with Sympy and Sage that gave me memory errors. So, in case a general solution could not be found, what do you think are the best optimization techniques to find a numerical solution of this system?&lt;/p&gt;&#xA;" OwnerUserId="10244" LastEditorUserId="20688" LastEditDate="2019-06-22T21:23:15.310" LastActivityDate="2019-06-22T21:23:15.310" Title="Solving a system of polynomial equations with multiple variables" Tags="|optimization|constrained-optimization|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14605" PostTypeId="2" ParentId="14578" CreationDate="2014-09-11T16:41:29.860" Score="0" Body="&lt;p&gt;You can try to read this: &lt;a href=&quot;http://arxiv.org/abs/math/0507288&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/abs/math/0507288&lt;/a&gt; and the references mentioned in the paper.&lt;/p&gt;&#xA;" OwnerUserId="10248" LastActivityDate="2014-09-11T16:41:29.860" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14606" PostTypeId="2" ParentId="14603" CreationDate="2014-09-11T19:15:15.490" Score="3" Body="&lt;p&gt;I've always relied on physical insights and finding an appropriate nondimensionalization. If your data arises from a physical process where you know the equation or at least the geometry of the problem you may have some luck via the &lt;a href=&quot;http://en.wikipedia.org/wiki/Buckingham_%CF%80_theorem&quot; rel=&quot;nofollow&quot;&gt;Buckingham pi theorem&lt;/a&gt; followed by trial and error. See also this paper:  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Price, James F. &quot;Dimensional Analysis of Models and Data Sets: Similarity Solutions and Scaling Analysis.&quot; (2006). &lt;a href=&quot;http://www.whoi.edu/science/po/people/jprice/class/DA_SSSA.pdf&quot; rel=&quot;nofollow&quot;&gt;PDF&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Four is a non-trivial number of parameters when the functionality is unknown. In general you will need some additional insight to determine the (expected) functionality of the data and then you can perform some curve fitting.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.&quot; - John von Neumann&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-09-11T19:15:15.490" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14607" PostTypeId="1" AcceptedAnswerId="14610" CreationDate="2014-09-11T22:29:25.987" Score="5" ViewCount="1269" Body="&lt;p&gt;I am having issues understanding why different languages are producing different answers for the Schur(QZ) decomposition.  I am working on writing some old stuff from Matlab into Julia and Python and am running into some issues with differences between Matlab's &lt;code&gt;qz&lt;/code&gt;, Julia's &lt;code&gt;schurfact&lt;/code&gt;, and Scipy's &lt;code&gt;qz&lt;/code&gt; (Julia and Scipy agree and Matlab is different).  After some digging, it seems the root of the issue is that Matlab uses the LAPACK &lt;code&gt;zgges&lt;/code&gt; by default (and not even sure if you can use &lt;code&gt;dgges&lt;/code&gt;, but didn't look very hard) whereas both Julia and Scipy default to LAPACK &lt;code&gt;dgges&lt;/code&gt; and can reach &lt;code&gt;zgges&lt;/code&gt; by requesting things to be complex.  I can get the answer that I am &quot;looking&quot; for, but am somewhat unsatisfied with simply &quot;arriving&quot; at the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a reason that Matlab sticks to &lt;code&gt;zgges&lt;/code&gt;?  For real numbers, why do &lt;code&gt;zgges&lt;/code&gt; and &lt;code&gt;dgges&lt;/code&gt; produce different answers(the answers differ by signs on only a few numbers that aren't necessarily related by row or column)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of what I am talking about (In all three languages for completeness and to facilitate looking at it).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Julia&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = reshape(1:9, 3, 3)&#xA;&#xA;B = [ 0.47143516 -1.19097569 1.43270697&#xA;    -0.3126519 -0.72058873  0.88716294&#xA;     0.85958841 -0.6365235 0.01569637]&#xA;&#xA;# This is the answer without using complex numbers&#xA;Freal = schurfact(A, B)&#xA;AAreal, BBreal = Freal[:S], Freal[:T]&#xA;Qreal, Zreal = Freal[:Q], Freal[:Z]&#xA;&#xA;# This is the complex answer turned into reals&#xA;# and matches Matlab's answer&#xA;Fcomplex = schurfact(A+0im, B+0im)&#xA;AAcomplex, BBcomplex = real(Fcomplex[:S]), real(Fcomplex[:T])&#xA;Qcomplex, Zcomplex = real(Fcomplex[:Q]), real(Fcomplex[:Z])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Python&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.linalg as la&#xA;&#xA;A = np.arange(1, 10).reshape(3, 3).T&#xA;B = np.array([[ 0.47143516, -1.19097569,  1.43270697],&#xA;              [-0.3126519 , -0.72058873,  0.88716294],&#xA;              [ 0.85958841, -0.6365235 ,  0.01569637]])&#xA;&#xA;# Answer without using complex&#xA;AAreal, BBreal, Qreal, Zreal = la.qz(A, B)&#xA;&#xA;# Answer using complex and&#xA;# matches Matlab's answer&#xA;Fcomplex = la.qz(A, B, output=&quot;complex&quot;)&#xA;AAcomplex, BBcomplex, Qcomplex, Zcomplex = map(np.real, Fcomplex)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Matlab&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = reshape(1:9, 3, 3)&#xA;B = [ 0.47143516 -1.19097569 1.43270697&#xA;    -0.3126519 -0.72058873  0.88716294&#xA;     0.85958841 -0.6365235 0.01569637]&#xA;&#xA;[AAm, BBm, Qm, Zm] = qz(A, B)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sorry for being a little long, but I'm interested in why this is happening.&lt;/p&gt;&#xA;" OwnerUserId="7723" LastActivityDate="2014-09-12T13:33:15.340" Title="Schur(QZ) Decomposition Differences" Tags="|linear-algebra|lapack|languages|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14608" PostTypeId="2" ParentId="14604" CreationDate="2014-09-11T22:36:04.980" Score="1" Body="&lt;p&gt;Any rotation matrix can be decomposed into the product of a rotation around the $x$-axis by some angle $\alpha$, a rotation around the $y$-axis by some angle $\beta$, and a rotation around the $z$-axis by some angle $\gamma$. These principal rotation matrices are trigonometric functions of their angles of rotation, so I'm not sure how you would obtain these angles symbolically as a function of the input vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, though, you could use that decomposition in a nonlinear equation solver to find $\alpha, \beta, \gamma \in [0, 2\pi)$; Newton's method (or variants thereof) would probably work fine, assuming your system of equations is well-posed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also opt to use an axis-angle representation of your rotation matrix instead. Then, instead of solving for the principal rotation angles $\alpha, \beta, \gamma$, you have to solve for an angle $\theta \in [0, 2\pi)$ and an axis $u$ such that $u_{x}^{2} + u_{y}^{2} + u_{z}^{2} = 1$. Add the constraint to your system of equations and solve for $u$ and $\theta$; even though you'll be solving for four variables, the presence of the unit norm constraint means you effectively have three degrees of freedom. This representation also involves trigonometric functions, so again, I'm not sure how you would obtain these angles symbolically, but numerically, it would probably work just fine, again, assuming your system of equations is well-posed.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-09-11T22:36:04.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14609" PostTypeId="2" ParentId="14607" CreationDate="2014-09-11T22:44:15.777" Score="2" Body="&lt;p&gt;Frequently, this sort of &quot;disagreement&quot; occurs between languages, libraries, versions of libraries, etc. because different conventions are being enforced, and the requested decompositions are equivalent (but not identical). I encountered this sort of issue in implementing an incremental singular value decomposition. The result I obtained with the algorithm I implemented differed from the standard singular value decomposition in the signs of the singular vectors, because the decomposition is unique up to the signs of those vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're best off asking a MATLAB-specific forum or mailing list why MATLAB chooses &lt;code&gt;zgges&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-09-11T22:44:15.777" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14610" PostTypeId="2" ParentId="14607" CreationDate="2014-09-12T06:32:45.113" Score="5" Body="&lt;p&gt;The MATLAB syntax &lt;code&gt;qz(A,B,'real')&lt;/code&gt; is consistent with &lt;code&gt;schur(A,'real')&lt;/code&gt;, so we might as well ask why the default is complex in the Schur form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two reasons spring to mind.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Backward compatibility. Probably there was a time when only the complex Schur form was implemented in Matlab (possibly from the pre-LAPACK times), and the default is retaining that behavior, not to break existing code. Apparently backward compatibility is a big deal in Matlab, even when it generates abominations (cfr. the syntax of &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/logspace.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;logspace&lt;/code&gt;&lt;/a&gt; for &lt;code&gt;b=pi&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Least surprise. When people call &lt;code&gt;schur&lt;/code&gt;, they expect the triangular factor to be, well, triangular. Those who want a real Schur form know that there are two variants, at least.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2014-09-12T13:33:15.340" LastActivityDate="2014-09-12T13:33:15.340" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14611" PostTypeId="2" ParentId="14604" CreationDate="2014-09-12T21:47:18.473" Score="1" Body="&lt;p&gt;Consider the equation in the form $x^tAx=0$, with $A$ symmetric. Since&#xA;$A$ is symmetric we can write $A=VDV^t$, where $V$ is orthogonal, and&#xA;$D$ is the real diagonal matrix of its eigenvalues $\lambda_i$. With the change of&#xA;variables $y=V^tx$, we have the equation&#xA;$$ \sum_i \lambda_i y_i^2 = 0. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives a complete characterization of solutions. In terms of&#xA;$z_i=y_i^2$, (so that $y_i=\pm\sqrt{z_i}$ and $x=Vy$) all the solutions&#xA;of the original problem are in one-to-one correspondence with the&#xA;solutions of&#xA;$$ \sum_i \lambda_i z_i = 0, \qquad \text{subject to}\ z_i\geq0. $$&#xA;The set of all solutions $z$ is convex; it's non-empty whenever at&#xA;least one $\lambda_i$ is zero or whenever at least two nonzero $\lambda_i$&#xA;have different signs. You can also parametrize the set of solutions by&#xA;separating the indices into those with positive, negative, and zero&#xA;$\lambda_i$, which makes it easy to deal with the constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you won't be able to find a closed form expression for the&#xA;eigenvalues $\lambda_i$ in terms of the original matrix $A$, but this&#xA;can easily be done numerically for any given matrix using standard&#xA;software.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-09-14T04:19:54.340" LastActivityDate="2014-09-14T04:19:54.340" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14612" PostTypeId="1" CreationDate="2014-09-13T12:25:33.007" Score="4" ViewCount="1131" Body="&lt;p&gt;I'm trying to solve a PDE in Python of the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial c(\mathbf{x}, t)}{\partial t} = \mathrm{D} \nabla^2 c(\mathbf{x}, t) -\gamma \rho(\mathbf{x}, t) c(\mathbf{x}, t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c$ represents the concentration of a chemical, $\mathrm{D}$ and $\gamma$ are constants, and $\rho$ represents the density field of some large point-particles (which in fact model bacteria). This is all done in 2D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I'm modelling the particles/bacteria represented by $\rho$ as individual agents moving in continuous space, not as a density field obeying a PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How I get round this is to take the positions of all the particles and turn this into a coarse density field $\rho$, then use this to solve the PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up until now I've been doing this by hand, binning particle positions onto a cartesian grid, then iterating the PDE using finite differencing. So far this has worked fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now I want to solve the PDE in a more complicated 2D geometry with curved boundaries, so I think I need to use the finite volume method rather than finite differencing. This is beyond what I can do by hand, so I've been looking into packages like FiPy and FEniCS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me, there seem to be two issues I don't know if they can deal with:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there an easy way to determine in which volume element a particular point lies? If so I can calculate $\rho$ by hand at each time step so that's good.&lt;/li&gt;&#xA;&lt;li&gt;Assuming I can calculate $\rho$, can I supply a variable field like this to a PDE solver, which it will then use in iterating the equation?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10261" LastActivityDate="2014-09-14T19:40:11.877" Title="Numerically solve a PDE in Python with a term calculated by coarse-graining" Tags="|pde|finite-difference|python|simulation|finite-volume|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14613" PostTypeId="1" AcceptedAnswerId="14620" CreationDate="2014-09-13T17:52:47.717" Score="1" ViewCount="86" Body="&lt;p&gt;I'm looking to find the most efficient way to change integers from a random number generator to a different inclusive number range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know of 2 ways so far:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Change the number into a decimal in the range of [0,1) and multiply it by the difference between the minimum and maximum values* in the new range.&lt;/li&gt;&#xA;&lt;li&gt;Find the remainder of the number divided by the difference between the minimum and maximum numbers* in the new range.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;*The difference will have to be incremented by 1 to get the correct inclusive range on the results&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There is however, a problem with both of these methods:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the decimal method involve a lot of floating point calculations, which are slow&lt;/li&gt;&#xA;&lt;li&gt;the remainder method will favor lower numbers in the number range&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To illustrate #2 above, consider getting a random value of a unsigned byte. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You get a random number in the range of 0-255.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you wanted a number in the range of 1-255, you might use the following formula:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;number = random() % 255 + 1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;any number from 0-254 will simply be increased by 1, giving you a range of 1-255.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;255, however will also grant you a 1, giving 1 DOUBLE the probability as the rest of the numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this illustrates the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;probabilty of number in range [newMin, newMin + oldMax % (newMax - newMin) ] is (oldMax - oldMin) / (newMax - newMin) rounded UP&lt;/p&gt;&#xA;&#xA;&lt;p&gt;probability of number in range (newMin + oldMax % (newMax - newMin) , newMax] is (oldMax - oldMin) / (newMax - newMin) rounded DOWN&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In my situation, I am getting an 8 byte value, so the effects of this flaw in the remainder method would require an insanely large sample of numbers before the flaw effects the results noticeably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if these are the only 2 methods available, I would disregard this distribution flaw to increase performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method that has better performance than method #1 but better results than #2?&lt;/p&gt;&#xA;" OwnerUserId="10263" LastActivityDate="2014-09-14T19:22:46.687" Title="optimising changing the range of integers from random number generation" Tags="|optimization|random-number-generation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14614" PostTypeId="1" AcceptedAnswerId="14615" CreationDate="2014-09-13T18:58:09.480" Score="1" ViewCount="114" Body="&lt;p&gt;I come from a computer graphics background without any professional in computational science. But I find my job(game engine development) being related to this field. May I ask some general questions about computational science?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last year, we made a product for deformable object(such as hair, cloth, etc.) simulation.&#xA;It is said to be a successful product since it can produce really convincing result in most of the time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I find it difficult to state clearly which physics model is adopted, and which computation method is applied.&#xA;When our users doubt whether a simulation result they get is incorrect, I cannot assure even I am a author of this product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does our physics model adopted lead to this result? Does our computational method work correctly so that it converges to our physics model? I do not know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in these days, while I am doing some research in SPH(Smoothed-particle hydrodynamics) simulation,&#xA;I cannot understand how those parameters are chosen.&#xA;When I see a simulation result, I am not sure if there is a bug in my code, or if this result is compliant with the physics model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally I think I can describe my confuse in following questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I check a simulation result is compliant with the physics model? How can I know a computation method applied in our program(like discretion or truncation of a continuous model) behaves well?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any introductory book/paper/online course on the whole process from physics model derivation to computation method design and error analysis?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7258" LastEditorUserId="7258" LastEditDate="2014-09-15T13:58:55.500" LastActivityDate="2014-09-15T13:58:55.500" Title="Confusion in modeling and computing" Tags="|simulation|" AnswerCount="1" CommentCount="3" ClosedDate="2014-09-14T18:40:16.773" ContentLicense="CC BY-SA 3.0" />
  <row Id="14615" PostTypeId="2" ParentId="14614" CreationDate="2014-09-13T22:13:24.187" Score="2" Body="&lt;p&gt;The first question deals with what is called consistency and convergence.  It states that when you reduce your time and spatial discretizations, your solution, in the limit, goes towards the analytical solution.  If you use standard, well-known methods this is usually given.  If you design your own method you need to verify this yourself by symbolically taking the limit of your method as your steps shrink and ensure that the original equations are recovered.  This is very much in the realm of numerical analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another part of the question involves whether your continuous equations properly describe the physics.  This portion is the actual modelling.  To answer whether your equations are sufficient you need to have a good understanding of what you want from your simulations.  Again, unless you are coming up with your own equations, the parameters for which certain equations reproduce real world physics well are usually part of the presentation of the equation.  Look back to the original source to see what the equation was designed for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, your request for reference is still very broad.  You could fill a small library with the textbooks that have been written on the subjects of numerical analysis and physics modelling .  If you want to refine your question to ask about specific physical phenomenon or certain types of numerical techniques, we may be able to point you to more canonical references for them.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-09-13T22:13:24.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14617" PostTypeId="1" AcceptedAnswerId="14618" CreationDate="2014-09-14T15:12:54.083" Score="0" ViewCount="242" Body="&lt;p&gt;I'm student of computer science (BS) and considering computational science as the field to major in for MS program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions which might look silly but I'm really confused:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- Are Scientific Computing and Computational Science fields the same?. In Iran we just have Scientific Computing as an MS program in computer science. After searching for days in Google I saw there were some websites stating that Scientific Computing and Computational Science are the same...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Is Computer Science a good major to enter computational science?. We have many courses in Numerical Mathematics, Algorithms, and Programming  in CS but it seems like you need good background in physics or chemistry. I feel that it's a major for Mechanical engineers or physicists.&lt;/p&gt;&#xA;" OwnerUserId="10085" LastEditorUserId="9667" LastEditDate="2014-09-14T19:57:28.120" LastActivityDate="2014-09-16T01:21:40.127" Title="General question about Computational Science" Tags="|computational-physics|" AnswerCount="2" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14618" PostTypeId="2" ParentId="14617" CreationDate="2014-09-14T16:01:40.323" Score="0" Body="&lt;p&gt;Scientific Computing and Computational Science are the same field.  Both involve using computation to answer scientific question through numerical methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Almost all technical fields can lead into computational science.  Very few people at the end of their undergraduate degree have all the skills needed to be successful in this field.  For instance, I got my undergraduate degree in mechanical engineering, which meant that I had a strong understanding of the physics, but my programming and algorithm knowledge was relatively weak.  I had to pick most of this up as I went.  For you it would likely be the opposite, with you needing to learn the physics.  My experience is that this is not significantly more ground to make up than the average graduate student needs to learn when they join any other (non-computational) lab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, computational science as a program is actually more common in computer science and mathematics departments, since the engineering and physical science departments are usually more focused on what question is being answered rather than how it is being answered.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-09-14T16:01:40.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14619" PostTypeId="2" ParentId="14569" CreationDate="2014-09-14T16:16:06.353" Score="0" Body="&lt;p&gt;I would try to reproduce the results from the original paper, in this case &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0913035&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;. Matrix market &lt;a href=&quot;http://math.nist.gov/MatrixMarket/&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt; and the sparse matrix collection &lt;a href=&quot;http://service.re3data.org/repository/r3d100010208&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt; are  also good repositories. You could also try to compare the convergence results of the code, with a well-tested implementation such as MATLAB, PETSc, etc.&lt;/p&gt;&#xA;" OwnerUserId="4558" LastActivityDate="2014-09-14T16:16:06.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14620" PostTypeId="2" ParentId="14613" CreationDate="2014-09-14T19:22:46.687" Score="1" Body="&lt;p&gt;The usual approach to get an integer random number in the range $[0,\ldots,N)$ (a half open range) is a piece of code of the form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;unsigned int rnd()&#xA;{&#xA;  unsigned int k;&#xA;  do {&#xA;    k = rand()&#xA;  } while (k &amp;gt;= RAND_MAX/N*N) &#xA;&#xA;  return k % N;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works because &lt;code&gt;rand()&lt;/code&gt; produces random numbers uniformly in the range &lt;code&gt;[0,RAND_MAX]&lt;/code&gt;. Then, the do-while loop produces random numbers uniformly in the range &lt;code&gt;[0,RAND_MAX/N*N)&lt;/code&gt; where the upper bound is the largest multiple of &lt;code&gt;RAND_MAX&lt;/code&gt; less than or equal to &lt;code&gt;RAND_MAX&lt;/code&gt;, and consequently &lt;code&gt;k % N&lt;/code&gt; is a uniformly distributed random number in the range &lt;code&gt;[0,N)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want random numbers in an interval &lt;code&gt;[a,b)&lt;/code&gt;, use the function above on the interval &lt;code&gt;[0,N=b-a)&lt;/code&gt; and just add &lt;code&gt;a&lt;/code&gt; to every number you get.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-14T19:22:46.687" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14621" PostTypeId="2" ParentId="14612" CreationDate="2014-09-14T19:40:11.877" Score="1" Body="&lt;p&gt;The way it sounds to me is that the key step is really to determine the field $\rho$ by evaluating how many particles you have in each cell. The rest is then just a standard heat equation with a particular source term, which is not overly complicated. For example, you could start at step-26 of the deal.II tutorial: &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_26.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_26.html&lt;/a&gt; (disclaimer: I wrote that program and a significant part of the library behind it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, it is not entirely trivial to find out which cell a particular point lies in if you have an unstructured mesh. There are, however, a number of simpler cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have have a mesh composed of triangles (in 2d) or tetrahedra (3d), then each cell is bounded by linear constraints and finding whether a point is inside a cell only requires to check 3 (in 2d) or 4 (in 3d) linear inequalities.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have quadrilateral meshes in 2d, each cell is bounded by 4 linear inequalities as long as every cell is convex (which you need for other reasons as well).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In 3d with hexahedra, cells are bounded by nonlinear constraints and things become a lot more complicated.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But even in the case of linear constraints, one surprisingly frequently runs into the case where points are within round-off distance from a bounding line/plane and the function that determines whether the point lies in a cell may say &quot;yes&quot; for two or no cell. You will have to figure out how to deal with this.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-14T19:40:11.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14622" PostTypeId="1" AcceptedAnswerId="14624" CreationDate="2014-09-14T20:28:57.150" Score="5" ViewCount="437" Body="&lt;p&gt;Although there are similar questions, I am also struggling with the implementation of the following term in &quot;my own code&quot; by Finite Element Method, namely,  $\nabla \phi \cdot \nabla \phi$. $\phi$ is a state variable such as scalar potential. I think the descritization of this term is much more difficult than that of advection term in N-S equation.  If possible, please tell me how to descritize and implement the term by FEM.&#xA;The equation to solve is as follows,  $\frac{\partial \phi}{\partial t} = (\nabla\phi)^2 + \nabla^2 \phi.$&lt;/p&gt;&#xA;" OwnerUserId="6930" LastEditorUserId="6930" LastEditDate="2014-09-14T21:03:02.827" LastActivityDate="2014-09-16T01:17:11.683" Title="Implementation of nonlinear term in FEM" Tags="|pde|finite-element|nonlinear-equations|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14623" PostTypeId="2" ParentId="14569" CreationDate="2014-09-15T03:37:40.920" Score="3" Body="&lt;p&gt;One thing you can do for smoke testing is to take a known solution $x$ (you could generate it randomly or pick something convenient) and a known matrix $A$, and then set $b$ equal to $Ax$, and solve the equation $Ax = b$ to see if you recover your known solution. This method is known as the method of manufactured solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid the influence of convergence issues, you probably want to make sure that $A$ is well-conditioned. One way to manufacture a well-conditioned $A$ would be to select a collection of eigenvalues that are in a bounded interval, and construct the diagonal matrix $\Lambda$ such that the main diagonal consists of those eigenvalues. Then, if you generate an invertible matrix $V$ (also should be well conditioned), $A = V \Lambda V^{-1}$. It's probably best to let $V$ be orthogonal in some of your test cases, in which case $V^{-1} = V^{T}$ and you avoid conditioning issues at the cost of making $A$ symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By no means will such a procedure substitute for using the matrices in collections like the Matrix Market and the University of Florida sparse matrix collection, but by generating small matrices and using the method of manufactured solutions, you should be able to detect and correct simple errors (say, due to indexing, or faulty logic) without having to run large test cases. It's also a methodology that will generalize to other types of problems (testing ODE solvers, testing PDE solvers, and so on).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-09-15T03:37:40.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14624" PostTypeId="2" ParentId="14622" CreationDate="2014-09-15T08:16:55.437" Score="6" Body="&lt;p&gt;You should rather think of what you will need in the following step, which is probably the numerical time integration of the semi-discrete equations.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you are going to use a (semi) explicit time stepping scheme, all you need is a function that for a given $\phi_0$ assembles the vector $\langle (\nabla \phi_0)^2, v \rangle$, where $v$ are your test functions. In FEM packages like &lt;code&gt;FEniCS&lt;/code&gt; this is straight forward.&lt;/li&gt;&#xA;&lt;li&gt;If you going for an implicit scheme, you will probably apply a Newton or Picard iteration for the given iteration point $\phi_0$, you will need the forms or matrices associated with &#xA;$\langle \nabla \phi_0 \nabla u, v \rangle$ where $u$ are the trial functions. This is a linear form that is easy to assemble with standard FEM methods.&lt;/li&gt;&#xA;&lt;li&gt;For atmost generality you can tensorize your problem, express $(\nabla \phi)^2$ via $D_\nabla (\phi \otimes \phi)$ and assemble the associated linear form (linear in the Kronecker product). For an example considering the convection term $(u \cdot \nabla) u$ in the Navier-Stokes equations have a look at this &lt;a href=&quot;https://github.com/highlando/dolfin_navier_scipy/blob/a1d28da38acf9bffa273fbd661e8fafaaf389fd0/dolfin_to_sparrays.py#L26&quot;&gt;python function&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-09-15T08:16:55.437" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14625" PostTypeId="1" AcceptedAnswerId="14628" CreationDate="2014-09-15T09:21:10.313" Score="10" ViewCount="2338" Body="&lt;p&gt;During my wandering in &lt;a href=&quot;https://mathematica.stackexchange.com/&quot;&gt;Mathematica.se&lt;/a&gt;, I gradually noticed that a certain kind of differential equation solving problem is &quot;troubling&quot; us all the time, that is, the boundary value problem (BVP) of nonlinear ordinary differential equations (ODEs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Shooting_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;shooting method&lt;/a&gt;, which is used by the &lt;a href=&quot;http://reference.wolfram.com/language/tutorial/NDSolveBVP.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Mathematica&lt;/em&gt; function &lt;code&gt;NDSolve&lt;/code&gt;&lt;/a&gt;, seems to be the only method that users of Mathematica.SE know about. Sometimes &lt;a href=&quot;https://mathematica.stackexchange.com/a/18309/1871&quot;&gt;it works well&lt;/a&gt;, but &lt;a href=&quot;https://mathematica.stackexchange.com/a/34533/1871&quot;&gt;in more cases (according to my personal feeling)&lt;/a&gt;, it's &lt;a href=&quot;https://mathematica.stackexchange.com/a/24660/1871&quot;&gt;painful&lt;/a&gt; to &lt;a href=&quot;https://mathematica.stackexchange.com/a/28713/1871&quot;&gt;find a proper initial guess&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's often that case that a proper guess cannot be found. &lt;a href=&quot;https://mathematica.stackexchange.com/q/59709/1871&quot;&gt;Here is an example&lt;/a&gt; that caused me to post this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, &lt;strong&gt;is shooting method the only general numerical method for solving BVP of nonlinear ODEs&lt;/strong&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, is there a good way to get a proper initial guess?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If not, what's the alternative?&lt;/strong&gt; If possible, please give some introduction or links for existing solvers implementing these alternatives.&lt;/p&gt;&#xA;" OwnerUserId="5331" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2014-09-16T03:13:56.053" Title="Is the shooting method the only general numerical method for solving nonlinear boundary value ODEs?" Tags="|ode|nonlinear-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14626" PostTypeId="2" ParentId="14625" CreationDate="2014-09-15T10:34:12.893" Score="5" Body="&lt;p&gt;No it is not. There is also &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;multiple shooting&lt;/li&gt;&#xA;&lt;li&gt;collocation&lt;/li&gt;&#xA;&lt;li&gt;finite differences&lt;/li&gt;&#xA;&lt;li&gt;fixed point iterations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and probably some more.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-09-15T10:34:12.893" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14627" PostTypeId="1" CreationDate="2014-09-15T14:13:24.893" Score="9" ViewCount="1001" Body="&lt;p&gt;I was wondering if anyone has any experience dealing with boundaries when implementing chebyshev differentiation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently trying to implement a no slip boundary condition to solve the incompressible Navier Stokes equations in 3D, in order to ensure that flow is zero at the boundaries is it really just as simple as setting u(:,:,1) and u(:,:,N)=0 at every stage of computation (similarly for v and w) as is indicated in textbooks. This wouldn't seem to take into account how points next to the boundary are affected by there being zero flow at the boundaries and just seems far too simplistic an approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks to anyone who can help.&lt;/p&gt;&#xA;" OwnerUserId="10270" LastActivityDate="2016-06-07T18:55:39.907" Title="Boundary conditions Chebyshev differentiation" Tags="|boundary-conditions|spectral-method|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14628" PostTypeId="2" ParentId="14625" CreationDate="2014-09-15T16:04:44.043" Score="11" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the shooting method the only general numerical method for solving BVP of nonlinear ODE(s)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;No.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most other methods consist of three parts:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Discretization&lt;/strong&gt;.  This may be done with finite differences, finite volumes, finite elements (Galerkin or collocation), spectral methods, and so forth.  This reduces the problem from an infinite-dimensional one to a finite-dimensional system of nonlinear algebraic equations.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;A nonlinear solver&lt;/strong&gt;.  Usually this is a Newton-type method, meaning that you linearize the problem locally and compute an update.  This reduces the problem to a sequence of linear algebraic systems.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;A linear solver&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Unlike shooting, these methods are easily generalized to elliptic problems in higher dimensions.  If you read any introductory book on numerical methods, you will find a description of some method of this kind.  For an existing solver, see e.g. MATLAB's &lt;code&gt;bvp4c&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods still require an initial guess.  Good initial guesses are usually based on some insight specific to the problem.  I believe there is not a general technique for finding good initial guesses for arbitrary BVPs.  You should keep in mind that nonlinear BVPs may have multiple solutions, and which one you obtain will depend on your initial guess.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-09-15T16:12:31.773" LastActivityDate="2014-09-15T16:12:31.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14629" PostTypeId="2" ParentId="14578" CreationDate="2014-09-15T16:26:32.343" Score="3" Body="&lt;p&gt;You could always &lt;a href=&quot;http://www.mathe.tu-freiberg.de/~ernst/Lehre/IVP/Literatur/laxRichtmyer1956.pdf&quot; rel=&quot;nofollow&quot;&gt;read the paper&lt;/a&gt;.  But my favorite explanation is in Chapter 9 of &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow&quot;&gt;LeVeque's finite difference book&lt;/a&gt;.  It includes examples proving stability of several discretizations.  Most introductory textbooks on numerical methods for PDEs will cover this.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-09-15T16:26:32.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14630" PostTypeId="2" ParentId="14627" CreationDate="2014-09-15T16:48:22.480" Score="0" Body="&lt;p&gt;From my limited experience:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It takes into account algebraically but after doing arithmetic - plugging in zero nodal values (supposing that they are the unknowns in your approach) at boundaries - terms containing them vanish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general problem of applying Dirichlet boundary conditions the approach is the same as in any method where nodal values are unknown, and after discretization you get linear system from which you need to eliminate known/fixed DOF's.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something that might be helpful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://code.google.com/p/another-chebpy/source/browse/p36-Laplace-nhBC.py&quot; rel=&quot;nofollow&quot;&gt;https://code.google.com/p/another-chebpy/source/browse/p36-Laplace-nhBC.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2014-09-15T16:48:22.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14631" PostTypeId="1" CreationDate="2014-09-15T16:55:02.573" Score="2" ViewCount="893" Body="&lt;p&gt;My data set is a 60x10 matrix. I performed principal component analysis of this matrix with matlab using the princomp(AdjustedData) after I adjusting my original data set by subtracting the mean of each column. Because I was hoping to obtain a data set described by the top five principal components, I selected the top five vectors in the COEFF output as my feature vectors. I firstly multiply the row feature vectors (vector on each row, 5x10) by my row adjusted data matrix (transposed adjusted data matrix with different dimensions on each row, 10x60) to obtain the so called final data.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;           FinalData (5x60)=Row feature vector(5x10) * row AdjustedData(10x60)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I multiply the transposed row feature vector(10x5) by the FinalData(5x60) to get a matrix as the same dimensions as my transposed original data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TransOriginalAdjustedData(10x60)=TransposedRowFeatureVector(10x5)*FinalData(5x60)&#xA;Finally, I transpose the output matrix and add the mean of each column back to retrieve the final original matrix (PCA data, a data set I thought described by only 5 principal components) that I want to compare with my original data set.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    PCA data=OriginalAdjustedData(60x10)+ original column mean&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I sum up the square difference between the data points of my original data set and the PCA data set, and take a square root of the sum. The result was about 0.022.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I transpose my original data matrix, 10x60 instead of 60x10, and performed the same procedures, the result was only 0.015. I know the dimension of the matrix is changed when performing PCA if I transpose the original matrix, but I cannot figure out what is causing the large difference between the two. The variables in my data set were not defined so I thought either way should work about the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My procedures are based on this PCA tutorial: &lt;a href=&quot;http://www.cs.otago.ac.nz/cosc453/student_tutorials/principal_components.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.otago.ac.nz/cosc453/student_tutorials/principal_components.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone help me with this? Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="10273" LastEditorUserId="549" LastEditDate="2014-09-15T17:15:08.117" LastActivityDate="2014-09-15T17:15:08.117" Title="Does it matter if I use principal component analysis on the transpose instead of the original matrix?" Tags="|linear-algebra|matrix|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14632" PostTypeId="2" ParentId="14622" CreationDate="2014-09-16T01:17:11.683" Score="3" Body="&lt;p&gt;Nonlinear differential equations are really no different than other nonlinear equations that you can solve with a Newton method. You may want to take a look at the discussion how this is done in a typical finite element code for a stationary problem here:&#xA;&lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/step_15.html&quot; rel=&quot;nofollow&quot;&gt;http://dealii.org/developer/doxygen/deal.II/step_15.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code shows how to do it for a stationary problem, but the process is the same for every time step if you use a fully implicit method. Of course, the process becomes a lot simpler if you treat the nonlinear term as explicit in the time stepping scheme.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-16T01:17:11.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14633" PostTypeId="2" ParentId="14617" CreationDate="2014-09-16T01:21:40.127" Score="3" Body="&lt;p&gt;I don't think there are any formal definitions of &quot;computational science&quot; and &quot;scientific computing&quot;. However, informally, people often seem to think of &quot;numerically solving PDEs&quot; when they say &quot;scientific computing&quot;. This is probably a historical vestige resulting from the fact that this is where much of large scale computing started.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, and equally informally, &quot;computational science&quot; seems to be a more recent term and has a wider meaning. It includes the practice of solving PDEs numerically, but also discrete algorithms (for example in computational biology or bioinformatics), questions of software design, etc. It denotes a larger field than just &quot;scientific computing&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-16T01:21:40.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14635" PostTypeId="1" AcceptedAnswerId="14644" CreationDate="2014-09-16T04:49:11.597" Score="2" ViewCount="1197" Body="&lt;p&gt;I have a RKF45 numerical integrator that simulates polymerization of proteins using CUDA.  It does so by tracking the populations of discrete length polymers, e.g. monomers, dimers, trimers, etc. all the way up to 8192-mers using the equations of the type that follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dc_r}{dt}= k_nc_r^{n_c}+2k_a(c_{r-1}-c_r) + k_m(2\sum^{max}_{s=r+2}c_s-(r-1)c_r) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where r represents the length of the polymer and c its concentration at that given time.  The k's are the parameters that I'd like to minimize an objective function with respect to.  These equations are integrated over about 20,000-30,000 timesteps.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to start fitting the model to experimental data.  The first type of fit I'm trying is fitting a value derived from these polymer populations at arbitrary timesteps.  For instance, I want to calculate the average length of polymers at five different times in the simulation/integration and compare them to externally supplied data.  The average length is derived for each timestep from the concentrations like this : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L(t) = \frac{M(t)}{N(t)}$$&#xA;$$M(t) = \sum^{max}_{s=n_c}s*c_s(t)$$&#xA;$$N(t) = \sum^{max}_{s=n_c}c_s(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function I'm trying to minimize thusly looks like this : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y(t)=\sqrt{\sum_{fit data}(L_{calculated}-L_{experimental})^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already implemented a solution to do this and run it through the following norm and minimize said norm/objective function using Scipy's Nelder-Mead optimize method.  It seems to work, but if it's possible to make it perform better, I'd like to make it happen.  I want to try and utilize more robust and swifter converging methods like L-BFGS-B, especially because they also allow me to impose bounds on the parameters, that need to remain positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the Runge-Kutta algorithm, for the unfamiliar, is here : &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the recursive nature of the Runge Kutta calculations such as &#xA;$$k_1=f(y_n)$$&#xA;$$k_2=f(y_n+a_{21}k_1)$$&#xA;$$k_3=f(y_n+a_{31}k_1+a_{32}k_2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having issues figuring out how to calculate the Jacobian, which I believe would be this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J(t) = [\frac{dy}{dk_a} \frac{dy}{dk_m} \frac{dy}{dk_n}]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if I framed this question properly, is it possible at some point during the RKF calculation to construct a Jacobian to pass out to Scipy along with the objective function evaluation?  I'd like to get an optimized optimization script up and running ASAP.&lt;/p&gt;&#xA;" OwnerUserId="5553" LastActivityDate="2014-09-17T17:59:47.517" Title="Is there a relatively simple way to extract the Jacobian from a Runge-Kutta 4/5 integrator?" Tags="|optimization|runge-kutta|jacobian|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14637" PostTypeId="1" CreationDate="2014-09-16T13:08:57.290" Score="8" ViewCount="7709" Body="&lt;p&gt;When writing MPI 3.0 code, should I pass &lt;code&gt;argc&lt;/code&gt; and &lt;code&gt;argv&lt;/code&gt; to the &lt;code&gt;MPI_Init&lt;/code&gt; call or not, and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Since Bill Barth's answer raised some questions I want to make some remarks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Passing &lt;code&gt;argc&lt;/code&gt;/&lt;code&gt;argv&lt;/code&gt; is not required since MPI 1.1.&lt;/li&gt;&#xA;&lt;li&gt;The question is specifically about why should/shouldn't one pass &lt;code&gt;argc&lt;/code&gt;/&lt;code&gt;argv&lt;/code&gt; (&lt;em&gt;why shouldn't you&lt;/em&gt; is then not really an answer).&lt;/li&gt;&#xA;&lt;li&gt;Still, sometimes you cannot pass &lt;code&gt;argc&lt;/code&gt;/&lt;code&gt;argv&lt;/code&gt; to &lt;code&gt;MPI_Init&lt;/code&gt; (writing a library that uses static initialization to start MPI if main is outside your control and MPI is an implementation detail).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;EDIT2: The question &lt;em&gt;why wouldn't you use &lt;code&gt;MPI_Init(argc, argv)&lt;/code&gt;&lt;/em&gt; has come too often now. Some reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Doing it for the sake of compatibility with older/non-compliant/compliant MPI &amp;lt; 1.1 implementations is not possible since if you are using MPI2 or MPI3 features those won't work anyways.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;MPI_Init()&lt;/code&gt; initializes the MPI runtime in the same way as &lt;code&gt;MPI_Init(argc, argv)&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;MPI_Init(argc, argv)&lt;/code&gt; removes arguments passed to the MPI run-time from &lt;code&gt;argc&lt;/code&gt;, and &lt;code&gt;argv&lt;/code&gt; &lt;em&gt;and&lt;/em&gt; initializes MPI. AFAIK it is the only way to clean &lt;code&gt;argc&lt;/code&gt; and &lt;code&gt;argv&lt;/code&gt; so if you need these to have no MPI arguments you need to use it. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;MPI_Init()&lt;/code&gt; can be used in more situations that &lt;code&gt;MPI_Init(argc, argv)&lt;/code&gt;. For example your library using MPI as an implementation detail can test if MPI is initialized, and if not, call &lt;code&gt;MPI_Init()&lt;/code&gt; and the right thing will happen. Your user doesn't know that you are using mpi, doesn't need to pass &lt;code&gt;argc&lt;/code&gt;, &lt;code&gt;argv&lt;/code&gt; to your library, doesn't need to change their main (in case it is taking no arguments) to use your library....&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2014-09-17T13:12:04.307" LastActivityDate="2021-07-01T02:45:07.453" Title="Should I pass command line arguments to MPI_Init or not?" Tags="|c++|c|mpi|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="14639" PostTypeId="2" ParentId="14637" CreationDate="2014-09-16T13:28:10.370" Score="1" Body="&lt;p&gt;I don't know if there's something new in the 3.0 standard that makes it optional in C/C++ to not pass them, but I would definitely pass them. I don't know the current state, but in the past many implementations passed additional command-line arguments to your program when executed and then edited them out in &lt;code&gt;MPI_Init()&lt;/code&gt;. If you want to use command-line arguments to pass options to your program, if you don't let the implementation do its editing, then you will have to interpret both your arguments and a potentially unknown number and style of arguments from the particular implementation you are using. These arguments are also highly likely to vary from implementation to implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's pretty normal to call &lt;code&gt;MPI_Init()&lt;/code&gt; with &lt;code&gt;argc&lt;/code&gt; and &lt;code&gt;argv&lt;/code&gt;, so why wouldn't you?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-09-16T13:28:10.370" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="14640" PostTypeId="2" ParentId="7978" CreationDate="2014-09-16T13:46:07.453" Score="5" Body="&lt;p&gt;My list in no particular order of preference. The interface should:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;be header only, without any dependencies but &lt;code&gt;&amp;lt;mpi.h&amp;gt;&lt;/code&gt;, and the standard library,&lt;/li&gt;&#xA;&lt;li&gt;be generic and extensible,&lt;/li&gt;&#xA;&lt;li&gt;be non-blocking &lt;em&gt;only&lt;/em&gt; (if you want to block, then block explicitly, not by default),&lt;/li&gt;&#xA;&lt;li&gt;allow continuation-based chaining of non-blocking operations,&lt;/li&gt;&#xA;&lt;li&gt;support extensible and efficient serialization (Boost.Fusion like, such that it works with RMA),&lt;/li&gt;&#xA;&lt;li&gt;have zero abstraction penalty (i.e. be at least as fast as the C interface),&lt;/li&gt;&#xA;&lt;li&gt;be safe (the destructor of a non-ready future is called? -&gt; std::terminate!),&lt;/li&gt;&#xA;&lt;li&gt;have a strong &lt;code&gt;DEBUG&lt;/code&gt; mode with tons of assertions,&lt;/li&gt;&#xA;&lt;li&gt;extremely type-safe (no more ints/void* for everything, heck I want tags to be types!),&lt;/li&gt;&#xA;&lt;li&gt;it should work with lambdas (e.g. all reduce + lambda),&lt;/li&gt;&#xA;&lt;li&gt;use exceptions consistently as error-reporting and error-handling mechanism (no more error codes! no more function output arguments!),&lt;/li&gt;&#xA;&lt;li&gt;MPI-IO should offer a non-blocking I/O interface in the style of Boost.AFIO,&lt;/li&gt;&#xA;&lt;li&gt;and just follow good modern C++ interface design practices (define regular types, non-member non-friend functions, play well with move semantics, support range operations, ...)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Extras:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;allow me to chose the executor of the MPI environment, that is, which thread pool it uses. Right now you can have applications with a mix of OpenMP, MPI, CUDA, and TBB... all at the same time, where each run-time thinks it owns the environment and thus ask the operating system for threads every time they feel like it. Seriously?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;use the STL (and Boost) naming convention. Why? Every C++ programmer knows it.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want to write code like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;auto buffer = some_t{no_ranks};&#xA;auto future = gather(comm, root(comm), my_offsets, buffer)&#xA;              .then([&amp;amp;](){&#xA;                /* when the gather is finished, this lambda will &#xA;                   execute at the root node, and perform an expensive operation&#xA;                   there asynchronously (compute data required for load &#xA;                   redistribution) whose result is broadcasted to the rest &#xA;                   of the communicator */&#xA;                return broadcast(comm, root(comm), buffer);&#xA;              }).then([&amp;amp;]() {&#xA;                /* when broadcast is finished, this lambda executes &#xA;                   on all processes in the communicator, performing an expensive&#xA;                   operation asynchronously (redistribute the load, &#xA;                   maybe using non-blocking point-to-point communication) */&#xA;                 return do_something_with(buffer);&#xA;              }).then([&amp;amp;](auto result) {&#xA;                 /* finally perform a reduction on the result to check&#xA;                    everything went fine */&#xA;                 return all_reduce(comm, root(comm), result, &#xA;                                  [](auto acc, auto v) { return acc &amp;amp;&amp;amp; v; }); &#xA;              }).then([&amp;amp;](auto result) {&#xA;                  /* check the result at every process */&#xA;                  if (result) { return; /* we are done */ }&#xA;                  else {&#xA;                    root_only([](){ write_some_error_log(); });&#xA;                    throw some_exception;&#xA;                  }&#xA;              });&#xA;&#xA;/* Here nothing has happened yet! */&#xA;&#xA;/* ... lots and lots of unrelated code that can execute concurrently &#xA;   and overlaps with communication ... */&#xA;&#xA;/* When we now call future.get() we will block &#xA;   on the whole chain (which might have finished by then!).&#xA;*/&#xA;&#xA;future.get();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Think how one could chain all this operations using MPI_C's &lt;code&gt;request&lt;/code&gt;s. You would have to test at multiple (or every single) intermediate step through a whole lot of unrelated code to see if you can advance your chain &lt;em&gt;without blocking&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1831" LastEditorUserId="1831" LastEditDate="2014-09-16T16:09:05.480" LastActivityDate="2014-09-16T16:09:05.480" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="14641" PostTypeId="1" CreationDate="2014-09-16T14:53:11.557" Score="11" ViewCount="171" Body="&lt;h2&gt;Introduction&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Let me first state some conflicting assertions of the matter to illustrate what are the issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Personally I&lt;/strong&gt; would like to have my code open at every stage of development, since &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;others shall see and take advantage of what I am doing&lt;/li&gt;&#xA;  &lt;li&gt;I also like to reuse existing code&lt;/li&gt;&#xA;  &lt;li&gt;third parties can contribute&lt;/li&gt;&#xA;  &lt;li&gt;the public is funding me, so the public has the right to see&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But my &lt;strong&gt;boss says&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;he needs to approve what becomes public under his name (or the institute's) and he cannot approve every single step&lt;/li&gt;&#xA;  &lt;li&gt;There are guidelines referring to intellectual right properties of the institute&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and my &lt;strong&gt;colleagues say&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;others will come and steal my unpublished ideas&lt;/li&gt;&#xA;  &lt;li&gt;my experimental code is of little use for others&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Questions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;To come up with a blueprint for code publication and open source development in my lab, we want to raise the following questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there already such a guideline, covering the important issues of open software in academia?&lt;/li&gt;&#xA;&lt;li&gt;Which issues have to included into such a guideline?&lt;/li&gt;&#xA;&lt;li&gt;What do you think is the right way to implement and use such a guideline?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Remarks&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The issues of crediting, reproducability, code documentation, and where to publish, we want to address in a separate guideline.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-08-03T07:43:04.017" Title="Policies Relating to Publication and Open Source Development of Code in Academia" Tags="|open-source|" AnswerCount="1" CommentCount="7" CommunityOwnedDate="2014-09-16T18:56:10.037" ContentLicense="CC BY-SA 3.0" />
  <row Id="14643" PostTypeId="1" AcceptedAnswerId="14655" CreationDate="2014-09-16T17:22:36.803" Score="6" ViewCount="3812" Body="&lt;p&gt;I created this Python function to generate a sigmoid function where I can modify position and width:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;def sigmoid(x,a,b):&#xA;# sigmoid function with parameters a = center; b = width&#xA;return 1/(1+np.exp(-(x-a)/b))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example changing the parameter b I can make it wider or narrower:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# testing changing sigmoid width (and slope at the same time) - parameter b&#xA; x = np.linspace(0,10,256)&#xA; y = sigmoid(x,5,1) # default&#xA; ymax = sigmoid(x,5,1.75)&#xA; ymin = sigmoid(x,5,0.25)&#xA; # Create the plot&#xA; plt.plot(x,y,lw=2,color='black')&#xA; plt.plot(x,ymax,lw=2,color='green')&#xA; plt.plot(x,ymin,lw=2,color='orange')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above is fairly straightforward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if I wanted a rotated sigmoid function?  I simulated my intended result &quot;graphically&quot; in the figure below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/iQwvN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which I plotted by just creating a new independent variable x ('test' variable below) and swapping x and y:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;test=np.linspace(0,1,128)&#xA;plt.plot(y2*255,test,lw=2,color='yellow')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a way to achieve this shape with a new sigmoid-like function, or to rotate the original?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has occurred to me that I could use something like &lt;code&gt;-np.sinh(x)&lt;/code&gt; but what I'd really like is to have a similar exponential expression with parameters a and b to control the curve's shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question is also posted on Stack Overflow&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/25851287/how-to-generate-a-rotated-by-90-degrees-logistic-sigmoid-function-in-python&quot;&gt;https://stackoverflow.com/questions/25851287/how-to-generate-a-rotated-by-90-degrees-logistic-sigmoid-function-in-python&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9541" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-03-23T15:17:45.610" Title="How to generate a rotated (by 90 degrees) logistic sigmoid function in Python" Tags="|python|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14644" PostTypeId="2" ParentId="14635" CreationDate="2014-09-16T19:33:10.770" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So, if I framed this question properly, is it possible at some point during the RKF calculation to construct a Jacobian to pass out to Scipy along with the objective function evaluation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta%E2%80%93Fehlberg_method&quot; rel=&quot;nofollow&quot;&gt;Runge-Kutta-Fehlberg 4(5) method&lt;/a&gt; you describe is an explicit numerical method for solving an ODE. As such, it does not require the solution of a nonlinear algebraic equation; Jacobian matrices are not required for that algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to construct a Jacobian? Sure; the RKF4(5) method has nothing to do with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you want are &lt;strong&gt;parameter sensitivities&lt;/strong&gt;, namely $\partial{c_{r}}/\partial{k_{a}}$, $\partial{c_{r}}/\partial{k_{m}}$, etc., so that you can calculate the derivatives you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These can be calculated numerically in a few ways, to my knowledge; all of them involve solving a set of auxiliary differential equations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the adjoint method; see &lt;a href=&quot;http://math.mit.edu/~stevenj/18.336/adjoint.pdf&quot; rel=&quot;nofollow&quot;&gt;Steven G. Johnson's notes&lt;/a&gt;, and &lt;a href=&quot;http://www.cs.ucsb.edu/~cse/Files/CCEpaper.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper by Petzold, et al.&lt;/a&gt;, among others by Petzold. Adjoint methods work best when you have fewer state variables than parameters. (That is, the dimension of your $c_{r}$ is smaller than the number of parameters you have.)&lt;/li&gt;&#xA;&lt;li&gt;the simultaneous direct method, where you solve the sensitivity ODE system and your original state ODE system at the same time as a larger coupled ODE system&lt;/li&gt;&#xA;&lt;li&gt;the staggered direct method (see M. Caracotsios, W.E. Stewart, Sensitivity analysis of initial value problems with mixed ODEs and algebraic equations, Comput. Chem. Engrg. 9 (4) (1985) 359–365)&lt;/li&gt;&#xA;&lt;li&gt;the simultaneous corrector method, where you solve for your state variables and sensitivities at the same time (see T. Maly, L.R. Petzold, Numerical methods and software for sensitivity analysis of differential–algebraic&#xA;systems, Appl. Numer. Math. 20 (1997) 57–79)&lt;/li&gt;&#xA;&lt;li&gt;the staggered corrector method (by Barton and Feehery; disclaimer: Barton was one of my advisors); see &lt;a href=&quot;https://yoric.mit.edu/sites/default/files/documents/FeeheryThesis.pdf&quot; rel=&quot;nofollow&quot;&gt;Feehery's thesis, Chapter 4&lt;/a&gt;, also see W.F. Feehery, J.E. Tolsma, P.I. Barton, Efficient sensitivity analysis of large-scale differential–algebraic systems, Appl. Numer. Math. 25 (1997) 41–54&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In chemistry and chemical engineering, there are a few other types of sensitivity analysis available, but I don't think of them will give you the Jacobian matrix information you need for parameter fitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One final warning: ODE-constrained optimization problems are notoriously tricky, and generally nonconvex. It is possible to apply a standard nonlinear programming solver to these problems and obtain local minima that are globally suboptimal. See the &lt;a href=&quot;https://yoric.mit.edu/sites/default/files/documents/ScottThesis.pdf&quot; rel=&quot;nofollow&quot;&gt;thesis of Joseph K. Scott&lt;/a&gt; for examples. (Disclaimer: Dr. Scott was a former colleague.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-09-17T17:59:47.517" LastActivityDate="2014-09-17T17:59:47.517" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14645" PostTypeId="1" AcceptedAnswerId="14646" CreationDate="2014-09-16T19:57:40.480" Score="0" ViewCount="123" Body="&lt;p&gt;I am trying to write an algebraic multi-grid solver (in c++). At a given level I determine which nodes are c-points and which nodes are f-points (where the total number of c and f points equals the matrix dimension on that level). Therefore I need two arrays: one array to hold the indices of the c-points, and one array to hold the indices of the f-points. The problem is I do not know how many c-points (or f-points) there will be before hand and so I don't know how large to make these arrays. One option is to just make both arrays to have size the same as the number of rows in the matrix that way ensuring no overflow. This is what I am doing right now, but this entails significant extra storage wasted. I could also essentially run my function that determines the c-points and f-points twice, where the first time I just record the final sizes, but this is obviously a lot of extra work. Does anyone know what is the best strategy for dealing with this? There does not seem to be any way of determining the final number of c-points without actually computing them one after another.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2014-09-18T01:54:17.587" Title="How to determine the number of c points in algebraic multi grid" Tags="|linear-algebra|c++|multigrid|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14646" PostTypeId="2" ParentId="14645" CreationDate="2014-09-17T01:27:55.363" Score="2" Body="&lt;p&gt;Maybe you could make one array that's sized for the total number of points, then fill it with coarse points from the front and fine points from the back. They'll meet up somewhere in the middle (but not overlap).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2014-09-17T01:27:55.363" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14647" PostTypeId="2" ParentId="14643" CreationDate="2014-09-17T04:23:48.947" Score="3" Body="&lt;p&gt;I am turning my comments into an answer as a petition of Mycarta. One easy option that I would use is to rotate the $(x,y)$. To rotate a vector with coordinates $(x,y)$ by an angle $\theta$ you can use a &lt;a href=&quot;http://en.wikipedia.org/wiki/Rotation_matrix&quot; rel=&quot;nofollow&quot;&gt;rotation matrix&lt;/a&gt;&#xA;$$\begin{pmatrix}&#xA;x' \\ &#xA;y'&#xA;\end{pmatrix} =&#xA;\begin{pmatrix}&#xA;\cos\theta &amp;amp;-\sin\theta\\&#xA;\sin\theta &amp;amp; \cos\theta&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;x \\ &#xA;y&#xA;\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Python this can be easily done as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;R = np.array([[np.cos(t),-np.sin(t)],[np.sin(t),np.cos(t)]])&#xA;X_new = np.dot(R,X_old)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;X_old&lt;/code&gt; is a $2\times n$ array, being $n$ the number of data you have.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2014-09-17T14:51:20.510" LastActivityDate="2014-09-17T14:51:20.510" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14648" PostTypeId="2" ParentId="14458" CreationDate="2014-09-17T09:48:49.643" Score="0" Body="&lt;p&gt;I think it might be helpful to perform a tensor product spline interpolation or rather, approximation by least squares as the data is noisy, e.g., by using Matlab's &lt;a href=&quot;http://www.mathworks.com.au/help/curvefit/approximation-by-tensor-product-splines.html&quot; rel=&quot;nofollow&quot;&gt;Curve fitting toolbox&lt;/a&gt;, and see whether the coefficients along certain dimensions tend to zero, meaning it's a low (hopefully 0-) order polynomial along that dimension.&lt;/p&gt;&#xA;" OwnerUserId="10287" LastActivityDate="2014-09-17T09:48:49.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14649" PostTypeId="1" AcceptedAnswerId="14650" CreationDate="2014-09-17T10:42:58.163" Score="1" ViewCount="512" Body="&lt;p&gt;I am currently trying to solve the 3D Poisson equation with a Chebyshev discretisation in the $z$ direction (from -1 to 1) and Fourier in the $x$ and $y$ (from $-\pi$ to $\pi$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have taken the code into a point where I think it is working but am not totally sure it is outputting the correct result. Does anyone know of any tests that I could use to either confirm it is working or reveal a mistake in it?, e.g., any initial conditions I could insert to make sure the correct result is being obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help!&lt;/p&gt;&#xA;" OwnerUserId="10270" LastEditorUserId="9667" LastEditDate="2014-09-17T18:26:16.587" LastActivityDate="2014-09-17T18:26:16.587" Title="3D Poisson equation, Fourier and Chebyshev" Tags="|fourier-analysis|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14650" PostTypeId="2" ParentId="14649" CreationDate="2014-09-17T11:12:46.903" Score="3" Body="&lt;p&gt;It is so trivial to pick a solution on a box domain for the Laplace equation. Just pick a function, say $\bar u(x,y,z)=x^2y^2\sin(z)$ (chosen in a way so that it isn't in your ansatz space), then compute $f=-\Delta \bar u$ and solve the Laplace equation&#xA;$$&#xA;  -\Delta u = f.&#xA;$$&#xA;Of course, your numerical solution should converge to $\bar u$. If you can't deal with nonzero boundary conditions, choose some other $\bar u$ that satisfies the boundary conditions you &lt;em&gt;can&lt;/em&gt; deal with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This technique is called the &quot;method of manufactured solutions&quot;. &lt;a href=&quot;https://code.google.com/p/dealii/wiki/FrequentlyAskedQuestions#How_do_I_know_whether_my_finite_element_solution_is_correct?_%28Or&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is another description of it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-17T11:12:46.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14653" PostTypeId="2" ParentId="14645" CreationDate="2014-09-17T19:26:18.403" Score="0" Body="&lt;p&gt;In C the command &lt;code&gt;calloc&lt;/code&gt; creates an array at runtime and &lt;code&gt;realloc&lt;/code&gt; makes a previous array bigger or smaller. In reality you may need a linked list, there's an open source library called &lt;code&gt;Gobject&lt;/code&gt;. It'll require further programming skill to use it but may lead to massive memory savings and will allow for a stupidly large number of nodes per layer.&lt;/p&gt;&#xA;" OwnerUserId="10291" LastEditorUserId="7733" LastEditDate="2014-09-18T01:54:17.587" LastActivityDate="2014-09-18T01:54:17.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14654" PostTypeId="2" ParentId="14637" CreationDate="2014-09-17T19:34:03.397" Score="2" Body="&lt;p&gt;I'd definitely pass them, but i'd pass the pointers like this MPI_init(&amp;amp;argc,&amp;amp;argv), allowing for perfectly valid call MPI_init(NULL,NULL) in your function.&lt;/p&gt;&#xA;" OwnerUserId="10291" LastActivityDate="2014-09-17T19:34:03.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14655" PostTypeId="2" ParentId="14643" CreationDate="2014-09-17T23:00:32.967" Score="7" Body="&lt;p&gt;Are you sure you don't want the &lt;a href=&quot;http://en.wikipedia.org/wiki/Logit&quot; rel=&quot;noreferrer&quot;&gt;logit&lt;/a&gt; function?&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-09-17T23:00:32.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14656" PostTypeId="1" AcceptedAnswerId="14658" CreationDate="2014-09-17T23:50:29.297" Score="3" ViewCount="402" Body="&lt;p&gt;How can I test whether a given 1D root-finding procedure is robust? I know that there are data sets and resources online for different kinds of optimization, but I have yet to find anything with regard to 1D root-finding. Do you happen to know of any data sets or functions that are commonly used to test the robustness of 1D root-finding procedures?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, I would like to see some functions that will break software that has not been written carefully to account for finite-precision arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your input!&lt;/p&gt;&#xA;" OwnerUserId="10294" LastActivityDate="2014-09-18T08:57:02.960" Title="Testing 1D root-finding procedures for robustness" Tags="|precision|testing|data-sets|roots|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14657" PostTypeId="1" CreationDate="2014-09-18T00:00:32.567" Score="2" ViewCount="348" Body="&lt;p&gt;I am looking to solve the incompressible Navier-Stokes equations in 3D, using an inflow boundary condition specifying a velocity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{u} = \mathbf{g}_0 \,\, \forall \,\, \mathbf{x} \in \Gamma_u$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I want to use a Neumann type boundary condition where I specify pressure at the outlet &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-p\mathbf{I} + \tau = p_0\mathbf{I} \,\, \forall \,\, \mathbf{x} \in \Gamma_h$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to use a projection method to solve for the combined velocity and pressure fields, however I am not certain as to how to incorporate the Neumann term above into such a scheme. Most of the literature talks about deriving a velocity based boundary condition for the discrete poisson equation - but that is all for problems where the Neumann boundary $\Gamma_h = \phi$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering whether anyone here had any insights on this. I can give more details of the mathematics that I am looking at if needed. Any references, or web-pages, or some brief pointers on this will be very helpful.&lt;/p&gt;&#xA;" OwnerUserId="10295" LastActivityDate="2014-09-18T00:00:32.567" Title="Treatment of Neumann (Traction) boundary conditions using projection methods" Tags="|finite-element|boundary-conditions|navier-stokes|projection|incompressible|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14658" PostTypeId="2" ParentId="14656" CreationDate="2014-09-18T08:57:02.960" Score="8" Body="&lt;p&gt;There are many good open-source implementations of root-finding methods out there already. One example is boost, whose implementation of Newton-Raphson and related methods you can find &lt;a href=&quot;http://www.boost.org/doc/libs/1_56_0/libs/math/doc/html/math_toolkit/internals1/roots.html&quot;&gt;here&lt;/a&gt;. If you read its source code, you will be able to see what issues the authors wanted to address, and how they dealt with issues like convergence, user-specified tolerance parameters, and round-off errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the algorithms boost implements is TOMS748. If you look at the original paper that introduced this algorithm, by Alefeld, Potra, and Shi, you will see what functions they used to test their algorithm (Table I on p. 341), and you can also compare your implementation with all the ones they ran. This is perhaps the simplest and most robust way to do this, and the one I'd personally prefer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way is to look at the testing code for open-source root-finding libraries, and see what functions the libraries' authors use to test them. For example, GSL's source code is &lt;a href=&quot;http://git.savannah.gnu.org/cgit/gsl.git/tree/&quot;&gt;here&lt;/a&gt;, and roots/test_funcs.c contains its testing functions.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-09-18T08:57:02.960" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14659" PostTypeId="1" CreationDate="2014-09-18T17:14:07.440" Score="3" ViewCount="350" Body="&lt;p&gt;My question is related, but not a duplicate of &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5099/asymptotic-convergence-of-the-solution-to-a-parabolic-pde-to-the-solution-of-an&quot;&gt;Asymptotic convergence of the solution to a parabolic pde to the solution of an elliptic pde&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I solve the parabolic PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_t = \Delta u + f(x,t)$ on $\Omega \times(0,T]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla u(x,t) \cdot n = 0 $ on $\partial\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x,0) = g(x)$ on $\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and obtain the (unique) steady state solution $u^*(x)$. &#xA;Assume that the source term in the parabolic PDE satisfies the compatibility condition for a pure neumann problem $\int_{\Omega}f(x,t)= \int_{\Omega}u_t$&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;From the compatibility above, can I claim that it will  also hold as $t \rightarrow \infty$ ? I suppose that the functions $f(x,t)$ and $u_t$ will need to be nice enough for me to take the limit inside the integral ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now consider the elliptic PDE:&#xA;$0 = \Delta u + \tilde{f}(x)$ on $\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla u(x) \cdot n = 0 $ on $\partial\Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is now a pure Neumann problem.&#xA;Assume that the steady state value of the source  is used, i.e., $\tilde{f}(x) = \lim_{t \rightarrow \infty} f(x,t)$ is used and that it satisfies the compatibility condition for this pure Neumann problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I know that if $\hat{u}(x)$ be a solution, then $\hat{u}(x) +c$ for all real numbers $c$, also solves the PDE. I am aware that we fix the constant by demanding  zero mean solution or prescribing the value at a point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible to prescribe a condition for the elliptic problem  &lt;em&gt;a priori&lt;/em&gt;  so that the solution to the elliptic PDE matches the steady state solution $u^*(x)$ of the parabolic PDE ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can we say anything else about the relation between these two solutions ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7728" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-09-19T11:05:28.013" Title="steady state solution from parabolic problem vs solution of elliptic problem" Tags="|pde|boundary-conditions|convergence|parabolic-pde|elliptic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14660" PostTypeId="2" ParentId="2313" CreationDate="2014-09-18T19:50:49.723" Score="0" Body="&lt;p&gt;You can still use R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Revolution R&lt;/code&gt; is a build of R that handles data sets that are larger than RAM. Use the function &lt;code&gt;princomp&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also has a full range of stats functions especially designed for big data style problems that don't fit into RAM, e.g. linear regression, logistic regression, quantiles, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can download fully-featured Academic version free of charge, by ticking the &quot;I am an academic&quot; box.  &lt;/p&gt;&#xA;" OwnerUserId="10305" LastActivityDate="2014-09-18T19:50:49.723" CommentCount="0" CommunityOwnedDate="2014-09-18T19:50:49.723" ContentLicense="CC BY-SA 3.0" />
  <row Id="14661" PostTypeId="2" ParentId="2313" CreationDate="2014-09-18T23:57:42.630" Score="1" Body="&lt;p&gt;I'd suggest using an incremental singular value decomposition, of which there are many in the literature. For instance: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the technical reports of Matthew Brand &lt;a href=&quot;http://link.springer.com/chapter/10.1007/3-540-47969-4_47&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt; and &lt;a href=&quot;http://linkinghub.elsevier.com/retrieve/pii/S0024379505003812&quot; rel=&quot;nofollow&quot;&gt;2&lt;/a&gt; are fairly easy to follow&lt;/li&gt;&#xA;&lt;li&gt;Chris Baker's &lt;a href=&quot;http://diginole.lib.fsu.edu/etd/961/&quot; rel=&quot;nofollow&quot;&gt;master's thesis&lt;/a&gt;, his software &lt;a href=&quot;http://www.math.fsu.edu/~cbaker/IncPACK/&quot; rel=&quot;nofollow&quot;&gt;IncPACK&lt;/a&gt;, and his later &lt;a href=&quot;http://linkinghub.elsevier.com/retrieve/pii/S0024379511005301&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; on incremental SVD method&lt;/li&gt;&#xA;&lt;li&gt;Bunch and Nielsen published the earliest known &lt;a href=&quot;http://link.springer.com/article/10.1007/BF01397471&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Papers by Hall on updating eigenvalue problems &lt;a href=&quot;http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=877525&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt; and &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0262885602001142&quot; rel=&quot;nofollow&quot;&gt;2&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Sequential Karhunen-Loeve analysis by Levy, et al., which is &lt;a href=&quot;http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=723422&quot; rel=&quot;nofollow&quot;&gt;basically the same thing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of these approaches reduce to the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;start with a small data set&lt;/li&gt;&#xA;&lt;li&gt;calculate an SVD somehow (this step is trivial for a single column matrix)&lt;/li&gt;&#xA;&lt;li&gt;repeat until finished:&#xA;&lt;ul&gt;&#xA;&lt;li&gt;add new data set&lt;/li&gt;&#xA;&lt;li&gt;use existing SVD &amp;amp; update rules to calculate SVD of new data set&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In your application, if you have an idea of where your singular value threshold for the top $N$ values will be, you can use that value to calculate a truncated SVD; if the threshold value is small enough, then the matrix you have to keep in memory will also be small (only the singular values above the threshold value are retained, along with their singular vectors; it's not even necessary to keep both left and right singular vectors, in Brand's algorithm).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-09-18T23:57:42.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14662" PostTypeId="1" CreationDate="2014-09-19T10:17:48.397" Score="3" ViewCount="1762" Body="&lt;p&gt;I am currently working on a project of building a simulator. Up till now I have mostly played around with very simple examples, but with regards to the future I have been thinking that it might be a good idea to use an external library for numerical computations, especially since I might also want to look into the possibility for parallelization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have used the Eigen library, which is very simple since you only need to include the appropriate header files and you are good to go. However, I am a bit uncertain about whether it is good enough in the long run? (versatility, speed etc.) Please correct me if you have used Eigen extensively and know I'm wrong! ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But anyway, my idea was to maybe learn to use one of the &quot;big guns&quot;, Trilinos or Petsc, as I understand they are the most well regarded libraries. However, it seems like it might be a bit problematic to get everything to work on Windows? (using Visual Studio)&#xA;From what I've found by googling and reading here, it seems there is an option compiling for Visual Studio using CMAKE, but I've found most of the user guides a bit confusing with regards to how well it will work, what possible ugly issues might turn up etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess my main question is: &lt;strong&gt;How easy will it be to set-up e.g. Trilinos for my current project?&lt;/strong&gt; I mostly need code for doing linear algebra computations (vectors, sparse matrices etc.), possibly in parallel. There is also the issue of choosing the &quot;right&quot; mesh data structure, but maybe I should get those from elsewhere (or build my own, as I have so far).&lt;/p&gt;&#xA;" OwnerUserId="7235" LastActivityDate="2014-09-19T15:39:34.773" Title="Using Trilinos and/or Petsc in Windows with Visual Studio?" Tags="|parallel-computing|software|simulation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14663" PostTypeId="2" ParentId="14659" CreationDate="2014-09-19T11:05:28.013" Score="3" Body="&lt;p&gt;I think that one key point to understand the answers is that, with the parabolic PDE that you wrote, we have some control on the &quot;quantity of $u$&quot;, i.e. $\int_{\Omega} u $: If you integrate the PDE, you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Omega} u_t = \int_{\Omega} \Delta u + \int_{\Omega} f(x,t) = \int_{\Omega} f(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because $\int_{\Omega} \Delta u = \int_{\partial \Omega} \nabla u \cdot n = 0$ thanks to the boundary conditions. Now integrating in time and supposing that everything is nice enough to exchange the integrals, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{t_1}^{t_2} \int_{\Omega} f(x,t)  = \int_{t_1}^{t_2} \int_{\Omega} u_t = \int_{\Omega} \int_{t_1}^{t_2} u_t = \int_{\Omega} u(t_2) - \int_{\Omega} u(t_1)$$&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes. If you want the solution $u^{*}$ to exist, you need $u_t \rightarrow 0$ (the solution converges in time). So you will need also $\int_{\Omega} u_t \rightarrow 0$ and for that, you need $\int_{\Omega} f(x,t) \rightarrow 0$. So you have $\int_{\Omega} \tilde{f} = 0$ for the stationnary solution to exist.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes. Use the relation I wrote before with $t_1 =0$, you will get:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega} u(t_2) = \int_{\Omega} u_0 + \int_{0}^{t_2} \int_{\Omega} f(x,t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you let $t_2 \rightarrow \infty$ (and everything is sufficiently smooth), you obtain if the limits exist&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega} u^{*} = \int_{\Omega} u_0 + \int_{0}^{\infty} \int_{\Omega} f(x,t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This looks like the condition that you are looking for. And it looks rather intuitive: you want your stationary solution to contain as much &quot;u&quot; as would have been created through the whole evolution of the problem (remark that the condition from the first point is necessary for the integral to converge).&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;I don't see what I could add.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2014-09-19T11:05:28.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14664" PostTypeId="2" ParentId="14662" CreationDate="2014-09-19T14:53:23.440" Score="1" Body="&lt;p&gt;Trilinos (and PETSc) will definitely work on windows, but it does have quite a few dependencies. How easy that will be depends on you. I'd suggest just starting up the CMake visual studio build and let it tell you what you're missing and just pick those up one at a time. In e.g. ubuntu it's a lot easier, just apt-get install trilinos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for if it's appropriate, it depends on the particulars of your program. If you definitely need parallelism, then it's a good idea to use trilinos or PETSc. But if you don't, I do find working with Eigen to be much more pleasant. I personally actually use a mix, where I have PETSc vectors for global, parallel data and use its linear/nonlinear solvers, but locally I use a lot of small matrix manipulation where it makes more sense to use Eigen.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-09-19T14:53:23.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14665" PostTypeId="2" ParentId="14662" CreationDate="2014-09-19T15:39:34.773" Score="3" Body="&lt;p&gt;Here is something that works for me to build Trilinos.  Put it in a batch file on a single line aftery modifying it for your file locations. Also install the third party software.  Not all options below are strictly necessary.  You can install clapack using cmake for (a slow) blas and lapack.  See more here &lt;a href=&quot;http://trilinos.sandia.gov/Trilinos10CMakeQuickstart.txt&quot; rel=&quot;nofollow&quot;&gt;http://trilinos.sandia.gov/Trilinos10CMakeQuickstart.txt&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;c:\Program Files (x86)\CMake 2.8\bin\cmake&quot;&#xA;-G &quot;Visual Studio 11 Win64&quot;&#xA;-DTrilinos_ENABLE_TESTS:BOOL=ON&#xA;-DTrilinos_ENABLE_EXAMPLES:BOOL=ON&#xA;-DTrilinos_ENABLE_Fortran=OFF&#xA;-DPYTHON_EXECUTABLE:FILEPATH=C:/Python26/python&#xA;-DTrilinos_ENABLE_Teuchos:BOOL=ON&#xA;-DTrilinos_ENABLE_Epetra:BOOL=ON&#xA;-DTrilinos_ENABLE_Triutils:BOOL=ON&#xA;-DTrilinos_ENABLE_Galeri:BOOL=ON&#xA;-DTrilinos_ENABLE_Amesos:BOOL=ON&#xA;-DTrilinos_ENABLE_Ifpack:BOOL=ON&#xA;-DTrilinos_ENABLE_AztecOO:BOOL=ON&#xA;-DTrilinos_ENABLE_ML:BOOL=ON&#xA;-DTPL_ENABLE_MPI:BOOL=ON&#xA;-DMPI_BASE_DIR:PATH=&quot;c:/Program Files/Microsoft HPC Pack 2008 SDK&quot;&#xA;-DLAPACK_LIBRARY_DIRS:PATH=&quot;C:\blas_lapack\x64\Debug&quot;&#xA;-DLAPACK_LIBRARY_NAMES:STRING=&quot;lapack&quot;&#xA;-DBLAS_LIBRARY_DIRS:PATH=&quot;C:\blas_lapack\x64\Debug&quot;&#xA;-DBLAS_LIBRARY_NAMES:STRING=&quot;blas;f2c&quot;&#xA;C:/Trilinos_master/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user335" LastActivityDate="2014-09-19T15:39:34.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14666" PostTypeId="1" CreationDate="2014-09-19T21:26:55.200" Score="8" ViewCount="257" Body="&lt;p&gt;given a symmetric matrix $Y \in \mathbb{R}^{n \times n}$, and an arbitrary matrix $X \in \mathbb{R}^{n \times n}$, and a vector $v \in \mathbb{R}^{n \times 1}$, is it possible to compute the following expression in $O(n^2)$ time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$diag(X^TYX) \cdot v$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $diag(X)$ returns a $n \times n$ matrix with its main diagonal elements equal to those of $X$ and off-diagonal elements equal to 0, and $X^T$ is the transpose of $X$.&lt;/p&gt;&#xA;" OwnerUserId="2131" LastActivityDate="2016-05-10T12:17:09.237" Title="Are there improved method of computing the following expression?" Tags="|optimization|algorithms|performance|matrix|complexity|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14667" PostTypeId="1" CreationDate="2014-09-20T23:33:21.433" Score="6" ViewCount="5088" Body="&lt;p&gt;I am reading the book &quot;Numerical Recipes in Fortran 77: The Art of Scientific Computing&quot; (Second Edition) and I came across some methods for numerical integration of 1D functions. More specifically the Gauss-Laguerre, Gauss Hermite, and Gauss Jacobi weights and abscissas appealed to me. I need them because I have some ill-behaved integrands that I need to integrate numerically. As far as I can tell there is no python implementation of these routines, the closest to these methods was this &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.hermite.hermgauss.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.hermite.hermgauss.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I feel that documentation is a bit incomplete. I'm not sure what to do with this method and slightly puzzled why is there no standard method for these types of quadrature in python.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;EDIT:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I have implemented and compared scipy quadrature with Gauss-Hermite Quadrature on the example problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;I = \int_{-\infty}^{\infty} \left( i\zeta(t) + \overline{i\zeta(t)} \right) \exp(i\omega t) \, \mathrm{d}t&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here $i=\sqrt{-1}$ and $\zeta(t)$ is a complex function of $t$. The overbear denotes complex conjugate. $\zeta(t)$ is not a standard function, it is something I calculated and the samples can be found at &lt;a href=&quot;http://speedy.sh/SX6JU/seData.npy&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://speedy.sh/SX6JU/seData.npy&lt;/a&gt; as a numpy array. Each row in the array is of form [t real(zeta) imag(zeta)] i.e. the sampled real and imaginary values of the function at time t. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy import integrate&#xA;&#xA;T1 =  np.load('seData.npy') #: [Time, x1, x2] from downloaded file&#xA;&#xA;&#xA;def integrand(t, omega):&#xA;    x1 = np.interp(t, T1[:,0], T1[:,1])  &#xA;    x2 = np.interp(t, T1[:,0], T1[:,2])    &#xA;    z0 = x1+x2*1j&#xA;    zeta = z0/np.sqrt(z0**2+1)&#xA;    return (1j*zeta+np.conjugate(1j*zeta))*np.exp(1j*omega*t) &#xA;&#xA;&#xA;&#xA;def integral(omega):&#xA;    def real_func(x,omega):&#xA;        return np.real(integrand(x,omega))&#xA;    def imag_func(x,t):&#xA;        return np.imag(integrand(x,omega))&#xA;    real_integral = integrate.quad(real_func, -80, 80, args=(omega))&#xA;    imag_integral = integrate.quad(imag_func, -80, 80, args=(omega))   &#xA;    return real_integral[0] + 1j*imag_integral[0]&#xA;&#xA;vintegral = np.vectorize(integral)&#xA;&#xA;&#xA;omega = np.linspace(-30, 30, 601)  &#xA;I = vintegral(omega)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ajAHm.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im not sure what the correct result should be but the &quot;noisiness&quot; present in the result should not be there I believe. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead I tried a Gauss-Hermite routine: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ti_her, wi_her, = np.polynomial.hermite.hermgauss(90)&#xA;&#xA;vintegrand = np.vectorize(integrand)&#xA;&#xA;def integral_hermite(omega):&#xA;    real_integral = np.dot(vintegrand(ti_her, omega).real, wi_her)&#xA;    imag_integral = np.dot(vintegrand(ti_her, omega).imag, wi_her)&#xA;    return real_integral + 1j*imag_integral&#xA;&#xA;vintegral_hermite = np.vectorize(integral_hermite)&#xA;&#xA;I2 = vintegral_hermite(omega)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wXU0s.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4XXBG.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe this is more in line with what I should be getting , however I am not sure I have done the correct thing. It seems the order of Gauss-hermite is very high and at low orders (90) it looks like my result shows some type of periodicity/aliasing. &lt;/p&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="7864" LastEditDate="2014-09-22T14:55:10.563" LastActivityDate="2017-03-09T11:42:03.670" Title="Is there a Gauss-Laguerre integration routine in Python?" Tags="|python|interpolation|quadrature|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14668" PostTypeId="2" ParentId="14667" CreationDate="2014-09-21T01:14:30.513" Score="8" Body="&lt;p&gt;There are standard methods for these types of quadrature in Python, in NumPy and SciPy:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.laguerre.laggauss.html#numpy.polynomial.laguerre.laggauss&quot;&gt;Gauss-Laguerre quadrature&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.legendre.leggauss.html#numpy.polynomial.legendre.leggauss&quot;&gt;Gauss-Legendre quadrature&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.hermite.hermgauss.html#numpy.polynomial.hermite.hermgauss&quot;&gt;Gauss-Hermite quadrature&lt;/a&gt; (as noted in your post)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.chebyshev.chebgauss.html#numpy.polynomial.chebyshev.chebgauss&quot;&gt;Gauss-Chebyshev quadrature&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.integrate.quad.html#scipy.integrate.quad&quot;&gt;QUADPACK adaptive quadrature&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.integrate.quadrature.html#scipy.integrate.quadrature&quot;&gt;adaptive Gaussian quadrature&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and other routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a degree of quadrature (essentially, the degree of the interpolant used to approximate the integrand), these Gauss-type quadrature functions return quadrature points $x_{i}$ and weight functions $w_{i}$ so that you can approximate the integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\int_{a}^{b}f(x)\,\mathrm{d}x&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for appropriate $a$ and $b$ by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\sum_{i=0}^{N}f(x_{i})w_{i};&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it's up to you to use the quadrature points and weights in a summation to calculate an approximation to your integral.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-09-21T01:14:30.513" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14670" PostTypeId="1" CreationDate="2014-09-21T12:04:00.937" Score="1" ViewCount="717" Body="&lt;p&gt;I have just started learning FEniCS and have used: &lt;a href=&quot;http://www.scientificpython.net/pyblog/fenics-linear-two-point-bvp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.scientificpython.net/pyblog/fenics-linear-two-point-bvp&lt;/a&gt; to write a script for solving:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;           u'' + u = 1&#xA;&#xA;           u(0) = 1, u'(1) = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with exact solution,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;           u(x) = exp(-x)/ exp(-1) + x + ( -1+exp(-1) )/ exp(-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Clearly the weak formulation of the above problem is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;           -(u', v') + (u,v) = (g,v) ; with g = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the edited code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    from dolfin import *&#xA;&#xA;    # definig mesh&#xA;    mesh = IntervalMesh(20, 0, 1)&#xA;&#xA;    # definig Function space on this mesh using Lagrange polynoimals of degree 2.&#xA;    V = FunctionSpace(mesh, &quot;CG&quot;, 2)&#xA;&#xA;&#xA;    # definign boundary values&#xA;    #u0 = Constant(0)&#xA;    u0 = Expression(&quot;x[0]&quot;)&#xA;&#xA;    # this functions checks whether the input x is on the boundary or not.&#xA;    def DirichletBoundary(x, on_boundary):&#xA;         tol = 1e-14&#xA;         return on_boundary and abs(x &amp;lt; tol)&#xA;&#xA;     # Enforcing u = u0 at x = 0&#xA;     bc = DirichletBC(V, u0, DirichletBoundary)&#xA;&#xA;     # Setting up the variational problem&#xA;     u = TrialFunction(V)&#xA;     v = TestFunction(V)&#xA;     f = Constant(1)&#xA;     g = Constant(1)&#xA;     a = -inner(grad(u), grad(v))*dx + inner(u,v)*dx&#xA;     L = f*v*dx&#xA;&#xA;     # solving the variational problem.&#xA;     u = Function(V)&#xA;     solve( a == L, u, bc)&#xA;&#xA;     # plotting solution&#xA;     plot(u, interactive = True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Clearly the solution plot does not incorporate the boundary condition u(0) = 1.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/IhxnN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would any body please help?&lt;/p&gt;&#xA;" OwnerUserId="5353" LastEditorUserId="-1" LastEditDate="2014-09-22T05:26:48.190" LastActivityDate="2014-09-22T05:26:48.190" Title="FEniCs: help in implementing the boundary condition for 1D problem" Tags="|fenics|" AnswerCount="0" CommentCount="5" ClosedDate="2014-09-22T15:46:26.123" ContentLicense="CC BY-SA 3.0" />
  <row Id="14671" PostTypeId="1" CreationDate="2014-09-22T05:44:56.063" Score="7" ViewCount="5536" Body="&lt;p&gt;Comsol Multiphysics (a popular FEM package) includes two time-stepping algorithms (&lt;em&gt;IDA&lt;/em&gt; aka &lt;em&gt;BDF&lt;/em&gt;, and &lt;em&gt;Generalized-alpha&lt;/em&gt;), described in their documentation as follows (quoted here under &lt;em&gt;Fair Use&lt;/em&gt;; emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;• IDA was created at the Lawrence Livermore National Laboratory (Ref. 4)&#xA;  and is a modernized implementation of the DAE solver DASPK,&#xA;  which uses variable-order variable-step-size backward differentiation&#xA;  formulas (BDF).  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;• Generalized-α is an implicit, second-order accurate method with a&#xA;  parameter α or ρ∞ (0 ≤ ρ∞ ≤ 1) to control the damping of high&#xA;  frequencies. With ρ∞ = 1, the method has no numerical damping. For&#xA;  linear problems this corresponds to the midpoint rule. ρ∞ = 0 gives&#xA;  the maximal numerical damping; for linear problems the highest&#xA;  frequency is then annihilated in one step. The method was first&#xA;  developed for second-order equations in structural mechanics &#xA;  and later extended to first-order systems (Ref. 7).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;BDF methods have been used for a long time and are known for their stability. However, they can have severe damping effects, especially the lower-order methods. &lt;strong&gt;Backward Euler severely damps any high frequencies.&lt;/strong&gt; Even if you are expecting a solution with sharp gradients, you might get a very smooth solution due to the damping in backward Euler.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Compared to BDF (with maximum order two), &lt;strong&gt;generalized-α causes much less damping&lt;/strong&gt; and is thereby more accurate. For the same reason it is also less stable.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;References:&lt;/p&gt;&#xA;  &#xA;  &lt;ol start=&quot;4&quot;&gt;&#xA;  &lt;li&gt;A.C. Hindmarsh, P.N. Brown, K.E. Grant, S.L. Lee, R. Serban, D.E. Shumaker, and C.S. Woodward, “SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers,” ACM T. Math. Software, vol. 31, p. 363, 2005.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;  &#xA;  &lt;p&gt;[...]&lt;/p&gt;&#xA;  &#xA;  &lt;ol start=&quot;7&quot;&gt;&#xA;  &lt;li&gt;K.E. Jansen, C.H. Whiting, G.M. Hulbert, “A Generalized-α Method for Integrating the Filtered Navier–Stokes Equations with a Stabilized Finite Element Method,” Comput. Methods Appl. Mech. Engrg., vol. 190, pp. 305–319, 2000.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This makes good sense, but I'd like to understand the essence of this numerical damping phenomenon. Is there a minimal example somewhere (1D is best, if possible) that shows numerical damping clearly, and allows to see where it comes from?&lt;/p&gt;&#xA;" OwnerUserId="4402" LastEditorUserId="20688" LastEditDate="2021-12-31T03:22:24.567" LastActivityDate="2021-12-31T03:23:55.690" Title="What is numerical damping in the context of time-dependent FEM solvers?" Tags="|finite-element|numerics|discretization|comsol|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14672" PostTypeId="1" CreationDate="2014-09-22T11:48:11.467" Score="1" ViewCount="253" Body="&lt;p&gt;I wrote a Fortran 2003 wrapper for the ARPACK routine znaupd, basically translating the the example driver routine zndrv1 into modern Fortran 2003 language with automatic arrays. I initialize every variable passed to the znaupd routine. Everything works perfect as long as i do not call the method zndrv1 more than once during the same program run. The first time it computes the correct solution but the second time it causes an &quot;not converged&quot;-error and exits with an zero eigenpair. Compiling ARPACK with the compiler flag -frecursive does not solve the problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my fortran 2003 subroutine is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subroutine zndrv1(mat,gs_vec,gs_val)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;complex(R_P), dimension(:,:), intent(in) :: mat&#xA;complex(R_P), dimension(:), intent(inout) :: gs_vec&#xA;real(R_P), intent(out) :: gs_val&#xA;&#xA;integer, parameter :: nev = 1, ncv = 10&#xA;integer :: iparam(11), ipntr(14)&#xA;logical :: select(ncv)&#xA;complex(R_P) :: d(ncv), v(size(mat,1),ncv), workd(3*size(mat,1)), workev(3*ncv), resid(size(mat,1))&#xA;complex(R_P) :: workl(3*ncv*ncv+5*ncv)&#xA;real(R_P) :: rwork(ncv)&#xA;&#xA;character(len=1) :: bmat = 'I'&#xA;character(len=2) :: which = 'SR'&#xA;integer, parameter :: ishfts = 1, maxitr = 300, mode = 1&#xA;integer :: ido = 0, lworkl = 3 * ncv**2+5*ncv, info = 1, n, ldv, ierr = 0&#xA;complex(R_P) :: sigma = (0.0,0.0_R_P)&#xA;real(R_P) :: tol = 0.0_R_P&#xA;logical :: rvec = .true.&#xA;&#xA;iparam = 0&#xA;ipntr = 0&#xA;select  = .true.&#xA;d = (0.0,0.0_R_P)&#xA;v = (0.0,0.0_R_P)&#xA;workd = (0.0,0.0_R_P)&#xA;workev = (0.0,0.0_R_P)&#xA;workl = (0.0,0.0_R_P)&#xA;rwork = 0.0_R_P&#xA;resid = gs_vec&#xA;n = size(mat,1)&#xA;ldv = n&#xA;iparam(1) = ishfts&#xA;iparam(3) = maxitr&#xA;iparam(7) = mode&#xA;&#xA;do while (ido == 0 .or. ido == 1 .or. ido == -1)&#xA;&#xA;     call znaupd( ido, bmat, n, which, nev, tol, resid, ncv,&amp;amp;&#xA;            v, ldv, iparam, ipntr, workd, workl, lworkl,&amp;amp;&#xA;            rwork,info )&#xA;&#xA;     if (ido .eq. -1 .or. ido .eq. 1) then&#xA;&#xA;        call av(workd(ipntr(1):(ipntr(1)+n-1)), workd(ipntr(2):(ipntr(2)+n-1)),mat)&#xA;&#xA;     end if&#xA;end do&#xA;&#xA;if ( info .lt. 0 ) then&#xA;&#xA;    print *, ' '&#xA;    print *, ' Error with _naupd, info = ', info&#xA;    print *, ' Check the documentation of _naupd'&#xA;    print *, ' '&#xA;&#xA;else&#xA;&#xA;    rvec = .true.&#xA;&#xA;    call zneupd (rvec, 'A', select, d, v, ldv, sigma,&amp;amp;&#xA;         workev, bmat, n, which, nev, tol, resid, ncv,&amp;amp;&#xA;         v, ldv, iparam, ipntr, workd, workl, lworkl, &amp;amp;&#xA;         rwork, ierr)&#xA;&#xA;    if ( ierr .ne. 0) then&#xA;&#xA;        print *, ' '&#xA;        print *, ' Error with _neupd, info = ', ierr&#xA;        print *, ' Check the documentation of _neupd. '&#xA;        print *, ' '&#xA;&#xA;    end if&#xA;&#xA;    gs_vec = v(:,1)&#xA;    gs_val = d(1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;!c        %-------------------------------------------%&#xA;!c        | Print additional convergence information. |&#xA;!c        %-------------------------------------------%&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     if ( info .eq. 1) then&#xA;         print *, ' '&#xA;         print *, ' Maximum number of iterations reached.'&#xA;         print *, ' '&#xA;     else if ( info .eq. 3) then&#xA;         print *, ' '&#xA;         print *, ' No shifts could be applied during implicit Arnoldi update, try increasing NCV.'&#xA;         print *, ' '&#xA;     end if&#xA;&#xA;     print *, ' '&#xA;     print *, '_NDRV1'&#xA;     print *, '====== '&#xA;     print *, ' '&#xA;     print *, ' Size of the matrix is ', n&#xA;     print *, ' The number of Ritz values requested is ', nev&#xA;     print *, ' The number of Arnoldi vectors generated (NCV) is ', ncv&#xA;     print *, ' What portion of the spectrum: ', which&#xA;     print *, ' The number of converged Ritz values is ', iparam(5)&#xA;     print *, ' The number of Implicit Arnoldi update iterations taken is ', iparam(3)&#xA;     print *, ' The number of OP*x is ', iparam(9)&#xA;     print *, ' The convergence criterion is ', tol&#xA;     print *, ' '&#xA;&#xA;  end if&#xA;&#xA;&#xA;&#xA;contains&#xA;&#xA;    subroutine av ( v, w, mat)&#xA;&#xA;        complex(R_P), dimension(:), intent(in) :: v&#xA;        complex(R_P), dimension(:), intent(out) :: w&#xA;        complex(R_P), dimension(:,:), intent(in) :: mat&#xA;&#xA;        w = matmul(mat,v)&#xA;&#xA;    end subroutine av&#xA;&#xA;end subroutine zndrv1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10330" LastActivityDate="2014-09-22T16:15:23.797" Title="Fortran 2003 ARPACK wrapper" Tags="|eigensystem|fortran|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14673" PostTypeId="1" AcceptedAnswerId="14675" CreationDate="2014-09-22T14:46:38.850" Score="3" ViewCount="545" Body="&lt;p&gt;I'm wondering if anyone can help me understand energy conservation when using conservation law methods (i.e. Riemann solver, High-Resolution Wave-Propagation Methods) with the addition of source terms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial \vec{Q}}{\partial t} + \nabla \cdot \vec{F} = \vec{\psi} $,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, $\vec{Q}$, and $\vec{F}$ are the conserved quantities and their corresponding fluxes.  $\vec{\psi}$ are the non-hyperbolic source terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 1D example of my Euler equation setup with an external gravitational field is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial}{\partial t}\left[ \rho \right] + \frac{\partial}{\partial z}\left[ \rho w \right] = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial}{\partial t}\left[ \rho w \right] + \frac{\partial}{\partial z}\left[ \rho w^2 + p \right] = -\rho g$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial}{\partial t}\left[ e \right] + \frac{\partial}{\partial z}\left[ (e+p)w\right] = -\rho g w$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, $t$ is time, $z$ is the spatial coordinate, $\rho$ is density, $w$ is fluid velocity, $e$ is the energy (internal + kinetic), $g$ is the acceleration due to gravity in the $z$ direction, and $p$ is the pressure.  The system is closed by defining an equation of state:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p = (\gamma -1)e_{i} $,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e_{i} = e - \frac{1}{2}\rho w^2$ is the internal energy, and $\gamma$ is the ratio of specific heats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system of equations is solved using a wave-propagation, Riemann solver with an F-Wave approach using the PyClaw software (&lt;a href=&quot;http://www.clawpack.org/doc/pyclaw/&quot; rel=&quot;nofollow&quot;&gt;http://www.clawpack.org/doc/pyclaw/&lt;/a&gt;).  This means the non-hyperbolic gravity source terms are included in the flux differencing (rather than solved using some fractional step source splitting technique).  The advantage of including the gravity source terms in the flux differencing is that in an equilibrium case, the source terms will exactly cancel the pressure gradient and maintain numerical equilibrium.  This has proven to work very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is regarding the conservation of energy, particularly the total energy that also includes gravitational energy,&#xA;$ e_{total} = e_i + e_{KE} + e_{gravitational}$.  Is there any expectation that the total energy $e_{total}$ should be conserved as precisely as the &quot;conserved&quot; quantity energy $e$ when the hyperbolic terms are modified with source terms?  In a test problem where an equilibrium case is perturbed, I see excellent conservation of $e$, but not great conservation of the total energy $e_{total}$.  I'm not sure what to make of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can provide a more detailed explanation of the equilibrium setup, and the perturbed case if that is helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&#xA;Wes&lt;/p&gt;&#xA;" OwnerUserId="10331" LastEditorUserId="10331" LastEditDate="2014-09-22T15:01:41.570" LastActivityDate="2016-07-07T03:17:26.477" Title="Energy Conservation in Conservation Laws with Source Terms" Tags="|fluid-dynamics|finite-volume|hyperbolic-pde|wave-propagation|conservation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14674" PostTypeId="2" ParentId="14672" CreationDate="2014-09-22T16:15:23.797" Score="3" Body="&lt;p&gt;You may want to have a look at &lt;a href=&quot;http://www.cs.rpi.edu/~szymansk/OOF90/bugs.html#4&quot; rel=&quot;nofollow&quot;&gt;surprising Fortran bugs&lt;/a&gt;, in particular what happens when you do things like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;complex(R_P) :: sigma = (0.0,0.0_R_P)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you were accustomed to C programming, you might think that, every time you call this procedure, the variable &lt;code&gt;sigma&lt;/code&gt; is initialized to 0.0. However, this isn't the case in Fortran -- when you initialize a variable at the same time that you declare it, it implicitly has the &lt;code&gt;save&lt;/code&gt; attribute. The next time you call your subroutine, &lt;code&gt;sigma&lt;/code&gt; will have the value that it had at the end of the last call. Try changing this to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;complex(R_P) :: sigma&#xA;&#xA;sigma = (0.0, 0.0_R_P)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as well as any other instances where you declare and initialize a variable in one statement and see if that fixes your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When all else fails, try the Kernighan &amp;amp; Ritchie debugger: a mess of print statements.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-09-22T16:15:23.797" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14675" PostTypeId="2" ParentId="14673" CreationDate="2014-09-22T16:56:42.813" Score="3" Body="&lt;p&gt;I have not studied this particular system before, and I'm sure that someone who has could say much more than I will.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think there is any reason to expect that discretizing the equations in the form you have them will lead to highly accurate conservation of total energy.  In this form, conservation of total energy depends on cancellation of certain flux terms and source terms, but since they are discretized in different ways they will only cancel up to truncation errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to get conservation of total energy, up to roundoff, is to rewrite the equations in conservative form.  I worked this out quickly but I think it's correct.  Since&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial e}{\partial t} = \frac{\partial e_{total}}{\partial t} + \frac{\partial}{\partial z}(\rho g w z) - \rho g w,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial e_{total}}{\partial t} + \frac{\partial}{\partial z}((e+p+\rho g z)w) = 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you discretize this conservation law in place of your current energy equation, you will get numerical conservation of total energy, up to roundoff.  In Clawpack this means that you will need to write a new Riemann solver; I suspect some Riemann solver for this form has been worked out in the literature.  Also, you should use interface values (not cell-center values) of $z$ for the flux term that depends on it (so that the fluxes at interfaces cancel).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-10-24T06:34:28.367" LastActivityDate="2014-10-24T06:34:28.367" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14677" PostTypeId="1" AcceptedAnswerId="14679" CreationDate="2014-09-22T22:33:52.477" Score="4" ViewCount="11337" Body="&lt;p&gt;To get the hang of Gauss-Laguerre integration I have decided to calculate the following integral numerically, which can be compared to the known analytical solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\int_0^{\infty}  s^3 \exp(-s^2 t) \, \mathrm{d}t = s&#xA;\end{align}&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result can be seen in the graph below. The result matches the analytical solution only on a limited subrange of the independent variable &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Evidently more care is needed to ensure convergence of the numerical solution for all &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Perhaps the integration routine must be used on smaller subranges? My question is, how can I make Gauss-Laguerre (or Gaussian Quadrature in general) applicable to problems of the kind shown above, where I need the solution to be accurate for all &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/9S2Z5.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;def integrand(t, s):&#xA;    return s ** 3 * np.exp(-(s ** 2) * t) * np.exp(t)&#xA;&#xA;&#xA;vintegrand = np.vectorize(integrand)&#xA;&#xA;&#xA;def integral(omega):&#xA;    I = np.dot(vintegrand(ti, omega), wi)&#xA;    return I&#xA;&#xA;&#xA;vintegral = np.vectorize(integral)&#xA;&#xA;&#xA;ti, wi, = np.polynomial.laguerre.laggauss(175)&#xA;&#xA;&#xA;s = np.linspace(-50, 50, 100)&#xA;&#xA;Is = vintegral(s)&#xA;&#xA;plt.plot(s, Is, &quot;b&quot;, label=&quot;&lt;span class=&quot;math-container&quot;&gt;$I(s)$&lt;/span&gt; numerical solution&quot;)&#xA;plt.plot(s, s, &quot;r&quot;, label=&quot;&lt;span class=&quot;math-container&quot;&gt;$I(s) = s$&lt;/span&gt; analytical solution&quot;)&#xA;plt.xlabel(&quot;&lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;&quot;)&#xA;plt.legend()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="3980" LastEditDate="2019-10-30T11:25:23.603" LastActivityDate="2019-10-30T11:36:27.737" Title="How to implement Gauss-Laguerre Quadrature in Python?" Tags="|python|quadrature|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14678" PostTypeId="2" ParentId="14677" CreationDate="2014-09-22T23:54:45.077" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;http://en.wikipedia.org/wiki/QUADPACK&quot; rel=&quot;nofollow&quot;&gt;QUADPACK&lt;/a&gt; through SciPy by modifying your code like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# ...&#xA;&#xA;import scipy.integrate&#xA;&#xA;def unweighted(s, t):&#xA;    exponent = -s**2*t&#xA;    return s**3*np.exp(exponent)&#xA;&#xA;def integral(omega):&#xA;    f = functools.partial(unweighted, omega)&#xA;    u, v = scipy.integrate.quad(f, 0, np.inf)&#xA;    return u&#xA;&#xA;# ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-09-22T23:54:45.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14679" PostTypeId="2" ParentId="14677" CreationDate="2014-09-23T02:57:15.487" Score="6" Body="&lt;p&gt;The issue here is that the error term increases very quickly as $s$&#xA;increases. The error term is, according to&#xA;&lt;a href=&quot;http://mathworld.wolfram.com/Laguerre-GaussQuadrature.html&quot; rel=&quot;noreferrer&quot;&gt;this article&lt;/a&gt;,&#xA;is bounded by&#xA;$$ |E| &amp;lt; \frac{n!^2}{(2n)!}\max |f^{(2n)}(t)|. $$&#xA;Now, asymptotically&#xA;$$ \frac{n!^2}{(2n)!} \approx \frac{\sqrt{\pi n}}{4^n}, $$&#xA;and&#xA;$$ |f^{(2n)}(t)| = s^3(1-s^2)^{2n} e^{-s^2t} &amp;lt; s^3(1-s^2)^{2n}, $$&#xA;meaning that the magnitude of the error term bound behaves, approximately, as&#xA;$$ \propto \left(\frac{1-s^2}{2}\right)^{2n}. $$&#xA;This is quite large, and $s$ must be close to $1$ when $n$ is large. This is too pessimistic because these are upper bounds, especially the pessimistic upper bound on $\max|f^{(2n)}|$. In your graph it looks like $s$ can be as large as $10$ or so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can avoid this issue by introducing a change of&#xA;variables $s^2t = u$, that changes the integrand from the form&#xA;$(\cdots)e^{-s^2t}\,dt$ to the form $(\cdots)e^{-u}\,du$, for which&#xA;the quadrature rule should be more accurate, because the function&#xA;multiplying the exponential will not have such fast-growing&#xA;derivatives. It does make your integral trivial, but it would work on other integrands as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is not an issue with how you invoke the quadrature rule, or that the rule is inapplicable. The issue is that the integrand has very large derivatives, so choosing an equivalent easier integrand should be enough.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-09-23T02:57:15.487" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14680" PostTypeId="1" CreationDate="2014-09-23T16:55:28.403" Score="1" ViewCount="15127" Body="&lt;p&gt;When evaluating the integral below in python using scipy.quad I get the following warning:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UserWarning: The maximum number of subdivisions (50) has been achieved.&#xA;  If increasing the limit yields no improvement it is advised to analyze &#xA;  the integrand in order to determine the difficulties.  If the position of a &#xA;  local difficulty can be determined (singularity, discontinuity) one will &#xA;  probably gain from splitting up the interval and calling the integrator &#xA;  on the subranges.  Perhaps a special-purpose integrator should be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore the real integration bounds should be zero to infinity (see below) but when I change the bounds to this my result looks very different from the finite values used in my example below. How should I go about calculating this integral?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following complex integral to compute numerically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\int_0^{\infty} Re \left( \frac{t_1(s)-it_2(s)}{\sqrt{\zeta^2(s)-1}} \right) \exp(i\omega s/V) \, \mathrm{d}s&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here $i=\sqrt{-1}$, $t_1$ and $t_2$ are real parametric functions of $s$, $V=\frac{\pi}{2(\pi+2)}$ is a constant, and $\zeta(s)$ is a complex function of $s$. The functions $t_1$, $t_2$, and $\zeta$ are not available in closed form, rather I have computed them numerically. A numpy array containing the sampled function values at various vales of s can be found to download at &lt;a href=&quot;http://speedy.sh/prG9e/stackexchange.npy&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://speedy.sh/prG9e/stackexchange.npy&lt;/a&gt;. The integrand is problematic because $\zeta(0) = 1$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have plotted the integrands for various values of omega:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/jlnDQ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The singularity in the real integrand is apparent. My work so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy import integrate&#xA;&#xA;T2 = np.load('stackexchange.npy') #Data: [s, x1, x2, t1, t2]&#xA;&#xA;def pyfunc(z):&#xA;    return np.sqrt(z**2-1)&#xA;&#xA;def integrand(s, omega):&#xA;    x1 = np.interp(s, T2[:,0], T2[:,1] )&#xA;    x2 = np.interp(s, T2[:,0], T2[:,2] )&#xA;    t1 = np.interp(s, T2[:,0], T2[:,3] )&#xA;    t2 = np.interp(s, T2[:,0], T2[:,4] )&#xA;    zeta = x2+x1*1j&#xA;    sigma = np.pi/(np.pi+2) &#xA;    V = 1/(2*sigma) &#xA;    return (-t2*np.real(1j/pyfunc(zeta))+t1*np.real(1/pyfunc(zeta)))*np.exp(1j*omega*s/V)&#xA;&#xA;&#xA;def integral(omega):&#xA;    def real_func(x,omega):&#xA;        return np.real(integrand(x,omega))&#xA;    def imag_func(x,omega):&#xA;        return np.imag(integrand(x,omega))&#xA;    a = 0.05 #Lower bound&#xA;    b = 20.0 #Upper bound&#xA;    real_integral = integrate.quad(real_func, a, b, args=(omega))&#xA;    imag_integral = integrate.quad(imag_func, a, b, args=(omega))   &#xA;    return real_integral[0] + 1j*imag_integral[0]&#xA;&#xA;vintegral = np.vectorize(integral)&#xA;&#xA;omega = np.linspace(-30, 30, 601)&#xA;I = integral(omega)&#xA;&#xA;plt.plot(omega, I.real, omega, I.imag)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Edit&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I found an analytical representation of the integrand in question, defined below. It still seems that there are some numerical difficulties with it though as NaNs are returned when I try to integrate.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def pyfunc(z):&#xA;        return np.sqrt(z**2-1)  &#xA;&#xA;    def func(theta):&#xA;        t1 = 1/np.sqrt(1+np.tan(theta)**2)    &#xA;        t2 = -1/np.sqrt(1+1/np.tan(theta)**2)&#xA;        return t1, t2&#xA;&#xA;&#xA;    def integrand(s, omega):&#xA;            sigma = np.pi/(np.pi+2) &#xA;            xs = np.exp(-np.pi*s/(2*sigma))&#xA;            x1 = -2*sigma/np.pi*(np.log(xs/(1+np.sqrt(1-xs**2)))+ np.sqrt(1-xs**2))&#xA;            x2 = 1-2*sigma/np.pi*(1-xs)&#xA;            zeta = x2+x1*1j&#xA;            Vc = 1/(2*sigma)&#xA;            theta =  -1*np.arcsin(np.exp(-np.pi/(2.0*sigma)*s))&#xA;            t1, t2 = func(theta)&#xA;            return np.real((t1-1j*t2)/pyfunc(zeta))*np.exp(1j*omega*s/Vc)&#xA;&#xA;    def integral(omega):&#xA;        def real_func(x,omega):&#xA;            return np.real(integrand(x,omega))&#xA;        def imag_func(x,omega):&#xA;            return np.imag(integrand(x,omega))&#xA;        a = 0&#xA;        b = np.inf&#xA;        real_integral = integrate.quad(real_func, a, b, args=(omega))&#xA;        imag_integral = integrate.quad(imag_func, a, b, args=(omega))   &#xA;        return real_integral[0] + 1j*imag_integral[0]&#xA;&#xA;    vintegral = np.vectorize(integral)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="7864" LastEditDate="2014-10-09T10:44:23.627" LastActivityDate="2014-10-09T10:44:23.627" Title="Using scipy.quad to calculate difficult integral" Tags="|python|quadrature|" AnswerCount="0" CommentCount="22" ContentLicense="CC BY-SA 3.0" />
  <row Id="14681" PostTypeId="2" ParentId="7894" CreationDate="2014-09-23T18:23:51.077" Score="3" Body="&lt;p&gt;The FDM paper (which I personally wrote, BTW) is intended more for basic education than it is for large-scale implementations.  As you may have noticed, the major bottleneck for large domains is setting up the system matrix and inverting it.  In my experience, Matlab simply runs out of memory when trying to run the inversion outright, so you have to be clever how this step is implemented.  I did my best to try and streamline this step in the code, but getting a solid matrix inversion for this kind of computation is a numerical challenge unto itself.  If anyone has some clever updates for allowing my codes to process larger matrices, I'd love to hear about it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;... or of course you can always just use FEM instead, and simply sample the grid more efficiently.  :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:  I'm not sure about the differences between Matlab and Octave when dealing with sparse matrices.  In Matlab, setting up the matrix has always been the easy part for me.  It has always been the inversion step that kills it by running out of memory.&lt;/p&gt;&#xA;" OwnerUserId="10337" LastActivityDate="2014-09-23T18:23:51.077" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14683" PostTypeId="1" AcceptedAnswerId="14684" CreationDate="2014-09-23T21:00:02.267" Score="0" ViewCount="65" Body="&lt;p&gt;Ok, It was hard to explain my problem in the title. I hope it made enough sense to peak the curiosity of some of you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I've been trying to do is find the relationship between the amount of random numbers generated (n) and the probability that an equal amount of odd and even numbers are generated. (n/2 odd and n/2 even). It should be clear that n must be an even number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously the numbers needn't be odd and even, as long as there is a 50% chance of generating one or the other. If this is posted in wrong location can someone navigate me in the right direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, basically I calculated mathematically that for every n amount of numbers generated, it should take on average approximately (4/5)sqrt(n) trials before the outcome is a perfect 50/50 (odd and even).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got this result in terms of sqrt(n) using Stirling's approximation of factorials because I thought the final result would be much more comprehensible and easier to compare to my computed results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my computed averages give me the result of approximately (6/5)sqrt(n) and I can't figure out why. I used two different pseudo-random generators (Math.random() &amp;amp; java.util.Random()) and both yielded the same results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the Maths involved in calculating the probability and I will also provide the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please zoom in on the image if you can't read it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/13RP5.png&quot; alt=&quot;MAths calculations&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computer program for calculating computationally:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;public static void main(String[] args) {&#xA;&#xA;    int run = 0;&#xA;    int total = 0;&#xA;    int amount = 0;&#xA;    int count = 0;&#xA;    int randint;&#xA;    int odd;&#xA;    int even;&#xA;&#xA;    int n =100;                               // Amount of numbers generated randomly &#xA;    int precision = 10000;                    // Amount of averages taken into account &#xA;                                              // (Reduce with increase in &quot;n&quot; as time increases)&#xA;&#xA;    Random generate = new Random(100);        // Seed 100 at beginning of program&#xA;&#xA;    run = 1;&#xA;&#xA;    while(run==1){&#xA;&#xA;        odd = 0;&#xA;        even = 0;&#xA;&#xA;        for(int i=0; i&amp;lt;n; i++){&#xA;&#xA;            randint = generate.nextInt(2);&#xA;&#xA;            if(randint%2==0){&#xA;&#xA;                even++;  &#xA;&#xA;            }else{&#xA;&#xA;                odd++;&#xA;&#xA;            }&#xA;&#xA;        }&#xA;&#xA;        count++;                            // increment count after every check&#xA;&#xA;        if((odd==(n/2)) &amp;amp;&amp;amp; (even==(n/2))){&#xA;&#xA;            total+=count;                   // add count to total ( addition before addition - average calculation)&#xA;            amount++;                       // increment amount, this will be the division later&#xA;            count=0;&#xA;&#xA;        }&#xA;&#xA;        if(amount==precision){&#xA;&#xA;            System.out.println(total/amount);  //print average&#xA;            run=0;&#xA;&#xA;&#xA;&#xA;        }&#xA;&#xA;&#xA;    }&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10338" LastActivityDate="2014-09-23T23:46:37.047" Title="Maths calculations for randomly generated numbers don't match computed averages" Tags="|random-number-generation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14684" PostTypeId="2" ParentId="14683" CreationDate="2014-09-23T22:35:50.533" Score="0" Body="&lt;p&gt;If $p = \binom{n}{n/2} \left( \frac{1}{2} \right)^n$ then your java code is approximating $\frac{1}{p}$ not $n p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a wolfram alpha code for the two exact functions and your two sqrt approximations, plus a better approximation (5/4)*sqrt(n) for the function that your code is actually approximating (this follows from the sterling approximation of 1/p).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot [ 1 /  ( choose[n, (n/2)] * (1/2)^n), n *  ( Choose[n, (n/2)] * (1/2)^n), (4/5)*sqrt(n), (6/5)*sqrt(n), (5/4)*sqrt(n) ] for n=50 to 150&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The explanation is that your java code is approximating the mean of the geometric distribution whose success probability parameter is the probability that &lt;code&gt;(odd==(n/2)) &amp;amp;&amp;amp; (even==(n/2))&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Anyway, basically I calculated mathematically that for every n amount of numbers generated, it should take on average approximately (4/5)sqrt(n) trials before the outcome is a perfect 50/50 (odd and even).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The (4/5)sqrt(n) value is the expected number of perfect (n/2):(n/2) results you get when you try n times, not the expected number of trials it takes to get a perfect (n/2):(n/2).&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2014-09-23T23:11:08.287" LastActivityDate="2014-09-23T23:11:08.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14685" PostTypeId="1" AcceptedAnswerId="14687" CreationDate="2014-09-23T22:44:26.413" Score="2" ViewCount="736" Body="&lt;p&gt;I have some function $F(k_x,k_y,k_z)$ that I wish to numerically integrate over a polygon domain - physically, I am integrating over the first Brillouin Zone (BZ) of the FCC lattice (a truncated octahedron). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to tell Fortran what region of $k$-space and which points to integrate over, given this shape. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can write plane equations and inequalities for this region and perhaps stick them into a logical structure, then integrate over the whole BZ, but that seems inefficient to me. It seems that I should be able to use any available symmetries to pick out a unique bit of the BZ, then reflect/rotate/etc my answer using those symmetries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was told that there already exist routines which can do something like this, but I am unsure if what I googled are appropriate (VASP, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anyone be kind enough to suggest an appropriate package(s) if one exists? If not, are there perhaps multidimensional integration methods I should investigate? If so, is there an efficient way to restrict my integration domain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another avenue is to simply sum $F(k)$ over a uniform grid within the zone - would it then be more efficient to pre-calculate a rank-3 array with 1's and 0's in my desired shape, call, calculate and sum $F(k)$ only for nonzero elements? Just trying to get a feel for which direction would be most efficient and accurate.&lt;/p&gt;&#xA;" OwnerUserId="10279" LastEditorUserId="549" LastEditDate="2014-09-24T21:35:21.933" LastActivityDate="2014-09-24T21:35:21.933" Title="(Fortran) Integrating/summing over complicated 3D domain" Tags="|fortran|quadrature|mesh-generation|computational-physics|symmetry|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14686" PostTypeId="2" ParentId="14683" CreationDate="2014-09-23T23:46:37.047" Score="0" Body="&lt;p&gt;I see that my other answer may be confusing so I'll try again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are two things that are not the same:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The expected number of perfect (n/2):(n/2) results when you try n&#xA;times.&lt;/li&gt;&#xA;&lt;li&gt;The expected number of tries it takes to get a perfect&#xA;    (n/2):(n/2) result.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In your question and in your code you seem to confuse these two things.  At the beginning of your question when you say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Anyway, basically I calculated mathematically that for every n amount of numbers generated, it should take on average approximately (4/5)sqrt(n) trials before the outcome is a perfect 50/50 (odd and even).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;your text seems to mean (2) but your formula is for (1).  Later in your copy pasted tex image, you say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the average number of 50/50 outcomes is given by multiplying the result by n&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That text and its following formula agree, and they are both for (1).  Your java code implements an approximation for the quantity you described in your text introduction; this implementation and the description are both for (2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expectations (1) and (2) are different, and they have sterling approximations (4/5)sqrt(n) and (5/4)sqrt(n) respectively.  If you are using (1) and (2) interchangeably then this explains why you are getting inconsistent numbers.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2014-09-23T23:46:37.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14687" PostTypeId="2" ParentId="14685" CreationDate="2014-09-24T02:00:30.230" Score="2" Body="&lt;p&gt;It would seem to me that you can subdivide your domain into a relatively small set of tetrahedra. Then, it is trivial to do the integration because there are many good (and pre-tabulated) quadrature rules on tetrahedra that will yield reasonably high accuracy -- this is what one does in the finite element method all the time, so there is a lot of information out there. The integral over your Brillouin zone is then simply the sum of the integrals over the tetrahedra you have decomposed your zone into.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the function you want to integrate, $F(k_x,k_y,k_z)$, happens to be highly oscillatory or otherwise not very smooth, then the solution is to split each of your tetrahedra into four smaller tetrahedra, and if necessary repeat the process, and integrate over each of the now smaller cells.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-24T02:00:30.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14688" PostTypeId="2" ParentId="14685" CreationDate="2014-09-24T03:11:44.993" Score="1" Body="&lt;p&gt;Many plane-wave codes use a very simple k-point weighting scheme:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generate a uniform mesh in k-space, $K$&lt;/li&gt;&#xA;&lt;li&gt;Assign each k-point in the irredicuble wedge a weight $w(k) = \#\{q \in K : \text{k and q are symmetric}\}$&lt;/li&gt;&#xA;&lt;li&gt;Write integrals as $\int_{BZ} F(k) dk = \sum_{k \in K} w(k) F(k)$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is quick and dirty, but it usually isn't the leading error (for PWDFT, at least).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find an example of such an integral being used to sum the electron number given a chemical potential (to find the Fermi energy) in Quantum Espresso's &lt;a href=&quot;https://github.com/maxhutch/quantum-espresso/blob/master/PW/src/sumkg.f90&quot; rel=&quot;nofollow&quot;&gt;sumkg&lt;/a&gt;.  I believe the code that finds the irreducible wedge and generates the k-point weights, $w(k)$, is &lt;a href=&quot;https://github.com/maxhutch/quantum-espresso/blob/master/PW/src/symm_base.f90&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.  Quantum Espresso can also use tetrahedra; the equivalent integral of sumkg is called &lt;a href=&quot;https://github.com/maxhutch/quantum-espresso/blob/master/PW/src/sumkt.f90&quot; rel=&quot;nofollow&quot;&gt;sumkt&lt;/a&gt;.  Routines to generate the tetrahedra and uniform meshes in k-space are &lt;a href=&quot;https://github.com/maxhutch/quantum-espresso/blob/master/PW/src/kpoint_grid.f90&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2014-09-24T03:11:44.993" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14689" PostTypeId="1" AcceptedAnswerId="14708" CreationDate="2014-09-24T07:29:13.647" Score="4" ViewCount="367" Body="&lt;p&gt;I was playing with N-body simulations of a game called &lt;a href=&quot;https://kerbalspaceprogram.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kerbal Space Program&lt;/a&gt;, which itself uses the &lt;a href=&quot;http://en.wikipedia.org/wiki/Patched_conic_approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;patched conics approximation&lt;/a&gt;. I have read that for long term stability it is best to use symplectic integrators and one of the simplest would be &lt;a href=&quot;http://en.wikipedia.org/wiki/Verlet_integration&quot; rel=&quot;nofollow noreferrer&quot;&gt;Störmer-Verlet method&lt;/a&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\ddot{\vec{x}}_n = F(\vec{x}_n)&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{x}_{n+1} = 2\vec{x}_n - \vec{x}_{n-1} + F(\vec{x}_n){\Delta t}^2&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;t_{n+1} = t_n + \Delta t&#xA;$$&lt;/span&gt;&#xA;I thought that I might add to this method using a similar approach as how this method can be derived, starting with the Taylor expansions:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{x}(t+\Delta t) = \sum^N_{n=0} \frac{\Delta t^n}{n!}\frac{\partial^n}{\partial t^n}\vec{x}(t) + O(\Delta t^{N+1})&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{x}(t+\Delta t) + \vec{x}(t-\Delta t) = \sum^5_{n=0} \frac{\Delta t^n}{n!}\frac{\partial^n}{\partial t^n}\vec{x}(t) + \sum^5_{n=0} \frac{(-\Delta t)^n}{n!}\frac{\partial^n}{\partial t^n}\vec{x}(t) + O(\Delta t^6)&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{x}(t+\Delta t) + \vec{x}(t-\Delta t) = 2\vec{x}(t) + \Delta t^2 \frac{\partial^2}{\partial t^2}\vec{x}(t) + \frac{\Delta t^4}{12} \frac{\partial^4}{\partial t^2}\vec{x}(t) + O(\Delta t^6)&#xA;$$&lt;/span&gt;&#xA;Since the accelerations can be calculated with &lt;span class=&quot;math-container&quot;&gt;$F(\vec{x}(t))$&lt;/span&gt; I only had to find an expression for the forth derivative (&lt;a href=&quot;http://en.wikipedia.org/wiki/Jounce&quot; rel=&quot;nofollow noreferrer&quot;&gt;jounce or snap&lt;/a&gt;). For this I used &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference#Higher-order_differences&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite difference&lt;/a&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial^4}{\partial t^4}\vec{x}(t) = \frac{\ddot{\vec{x}}(t+\Delta t) - 2\ddot{\vec{x}}(t) + \ddot{\vec{x}}(t-\Delta t)}{\Delta t^2} + O(\Delta t^2)&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{x}(t+\Delta t) = 2\vec{x}(t) - \vec{x}(t-\Delta t) + \Delta t^2 \frac{\ddot{\vec{x}}(t+\Delta t) + 10\ddot{\vec{x}}(t) + \ddot{\vec{x}}(t-\Delta t)}{12} + O(\Delta t^6)&#xA;$$&lt;/span&gt;&#xA;This method would be implicit, since for the next position you would need to know the acceleration at that point, however I thought that higher order precision might give it some advantage over Störmer-Verlet. Similar to Störmer-Verlet I assumed that the global error would only drop two orders, such that this method would be a forth order integrator.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Kepler Orbit&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I did a simple benchmark of an eccentric &lt;a href=&quot;http://en.wikipedia.org/wiki/Kepler_orbit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kepler orbit&lt;/a&gt; (eccentricity of 0.75 and 20 orbits) I found that this method seems to be a fifth order method and Verlet, as suspected, a second order method. I derived this from the slope in this log-log plot of the time-step size versus the error (distance between numerical and analytical endpoint):&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Uo6v2.png&quot; alt=&quot;Error of integrator versus time-step size&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual slopes are &lt;span class=&quot;math-container&quot;&gt;$4.98\pm 0.01$&lt;/span&gt; for my method and &lt;span class=&quot;math-container&quot;&gt;$1.98\pm 0.01$&lt;/span&gt; for Verlet, where the errors are a 95% confidence bound.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Simple Harmonic Oscillator&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I also tested this method with a one dimensional simple harmonic oscillator, &lt;span class=&quot;math-container&quot;&gt;$\ddot{x}=-x$&lt;/span&gt;, with the initial conditions &lt;span class=&quot;math-container&quot;&gt;$x(t_0)=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\dot{x}(t_0)=0$&lt;/span&gt;, which has the solution &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x(t-t_0) = \cos(t).&#xA;$$&lt;/span&gt;&#xA;To avoid errors in the second required step I used &lt;span class=&quot;math-container&quot;&gt;$x(t_0+h)=\cos(h)$&lt;/span&gt; . Even with a time step of only one-forth the period of the oscillation, does the amplitude, after 5000 periods, only drop to 0.973. Would this be enough to say that this method is symplectic, since I have only read about the properties of symplectic integrator, but do not know how to tell whether a method would be symplectic or not. I also looked at the error as a function of time-step size and found that this method seemed to be a forth order method and Verlet a second order method when the end position would be at &lt;span class=&quot;math-container&quot;&gt;$x_N=x(t_{end})=0$&lt;/span&gt; (maximum speed), but it seemed to be a eighth order method and Verlet a forth order method when the end position would be at &lt;span class=&quot;math-container&quot;&gt;$x_N=x(t_{end})=\pm 1$&lt;/span&gt; (minimal speed), see figure below.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/YPCa8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual slopes are &lt;span class=&quot;math-container&quot;&gt;$8.00\pm 0.01$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$3.998\pm 0.003$&lt;/span&gt; for my method and &lt;span class=&quot;math-container&quot;&gt;$3.992\pm 0.005$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1.997\pm 0.002$&lt;/span&gt; for Verlet, where the errors are again a 95% confidence bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this difference in the order of the error is dependent on the local dynamics, but some how for this ODE this error reduces again when speed drops to zero. Therefore it seems that my initial guess was correct and this method is only a forth order accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the performance of my method depends on where and which ODE is used. Is this a common among other methods? I have had one course about scientific computing, in which a few integration methods where covert, but none of them where symplectic, so I wonder if this integrator is symplectic, since Verlet is said to be symplectic? And does this method have a known name, since it was fairly easy to derive?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;PS: In a similar way an eighth order method can be found, by increasing the Taylor expansion by two orders and by using a forth order &lt;a href=&quot;http://mathfaculty.fullerton.edu/mathews/n2003/differentiation/numericaldiffproof.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;approximation&lt;/a&gt; of the second derivative of the acceleration and a second order approximation of the forth derivative of the acceleration:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{x}_{n+1} = 2\vec{x}_{n} - \vec{x}_{n-1} + \Delta t^2 \frac{3\ddot{\vec{x}}_{n+1} + 48\ddot{\vec{x}}_{n+1/2} + 78\ddot{\vec{x}}_{n} + 48\ddot{\vec{x}}_{n-1/2} + 3\ddot{\vec{x}}_{n-1}}{180} + O(\Delta t^8)&#xA;$$&lt;/span&gt;&#xA;However I have not yet figured out a good way to find &lt;span class=&quot;math-container&quot;&gt;$\ddot{\vec{x}}_{n+1/2}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7031" LastEditorUserId="7031" LastEditDate="2020-04-29T07:33:54.957" LastActivityDate="2020-04-29T07:33:54.957" Title="Behavior of integration method" Tags="|time-integration|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14690" PostTypeId="2" ParentId="11006" CreationDate="2014-09-24T12:45:06.010" Score="0" Body="&lt;p&gt;short: Take into account that scientific computing is complicated itself. Do you really want the programming language to came into play?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematics uses abstraction to solve problems that cannot be solved by intuition. Therefore concepts have the tendency to be abstract. This why it is not trivial to understand what concepts to encapsulate. In scientific computing the usual examples for classes like &quot;Animal&quot; &quot;Vehicle&quot; are rather useless. This is true for Object oriented programming, but I believe that reproducing abstract concepts on a computer is not trivial in Imperative programming as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why I believe that here we are dealing with two different efforts: programming, on one side, and scientific computing on the other. At undergraduate level, where students come from heterogeneous backgrounds, you may end up teaching two different things at the same time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your goal is teaching scientific computing I think that is difficult enough.  Having an additional barrier as the programming language (we all agree that C++ requires training) would demotivate a good chunk of students, this is why I suggest to go with python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your course is &quot;Introduction to SC&quot; I believe that python has the best result/effort ratio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: now we have rather good computers, we do not really need to look for efficency at an undergraduate level. &lt;/p&gt;&#xA;" OwnerUserId="6574" LastActivityDate="2014-09-24T12:45:06.010" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14691" PostTypeId="1" CreationDate="2014-09-24T15:50:01.810" Score="4" ViewCount="677" Body="&lt;p&gt;I bought a book where the exercises are given in Maple worksheet format (.mw). Is there an open-source software in Linux able to interpret or to convert this specific file format ? &lt;/p&gt;&#xA;" OwnerUserId="7087" LastEditorUserId="276" LastEditDate="2014-09-25T01:28:38.720" LastActivityDate="2014-10-27T09:09:08.340" Title="Open source linux Maple Worksheet Interpreter/Converter?" Tags="|maple|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14692" PostTypeId="2" ParentId="14691" CreationDate="2014-09-24T17:47:35.880" Score="1" Body="&lt;p&gt;I don't think such luxury is available. &#xA;As a possible solution to your problem, contact some one with maple license, so that he can copy the contents of .mw file to some text editor and mail back to you.&lt;/p&gt;&#xA;" OwnerUserId="5353" LastEditorUserId="276" LastEditDate="2014-09-25T01:15:49.013" LastActivityDate="2014-09-25T01:15:49.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14693" PostTypeId="1" AcceptedAnswerId="14694" CreationDate="2014-09-24T18:47:40.087" Score="5" ViewCount="277" Body="&lt;p&gt;Suppose we have two matrices $A$ and $B$ (we can assume they're symmetric; if absolutely necessary I think they may be positive definite).  Then, is there any technique for solving $$(A\circ B)x=b,$$ where $\circ$ denotes the Hadamard (elementwise) product?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am seeking a direct solver if possible rather than iterative methods like CG.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2014-09-25T07:30:30.990" Title="Solving &quot;Hadamard systems&quot;" Tags="|linear-algebra|linear-solver|linear-system|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14694" PostTypeId="2" ParentId="14693" CreationDate="2014-09-24T19:07:59.273" Score="3" Body="&lt;p&gt;Ultimately, it depends on the sparsity of $A$ and $B$ and the symmetry of the resulting hadamard product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hadamard product will aggregate the sparsity structure of $A$ and $B$.  So if one or the other is sparse, the product is also sparse (and may be more so if $A$ and $B$ have difference sparsity structures).  So any sparse direct solver would be appropriate.  However, if the matrix is extremely large, memory becomes an issue and sparse iterative methods are just about the only choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if $A$ and $B$ are dense, the hadamard product is an $O(n^2)$ operation, which is insignificant compared to a direct linear solve of $O(n^3)$ operations.  So even if A and B are extremely large dense matrices, simply computing the hadamard product before solving the resulting system is not that much more work compare to solving the system.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the dense case, the key question becomes whether the resulting matrix is hermitian or not.  In the worst case (non-hermitian hadamard product), gaussian elimination is pretty much the only thing you can do. If it is hermitian, a Cholesky (definite ) or Bunch-Kaufman (indefinite) algorithm would be appropriate.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-09-24T19:13:38.087" LastActivityDate="2014-09-24T19:13:38.087" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14697" PostTypeId="2" ParentId="14693" CreationDate="2014-09-25T02:01:04.223" Score="2" Body="&lt;p&gt;The nonzero elements of the product are exactly those locations that are nonzero in &lt;em&gt;both&lt;/em&gt; matrices. In other words, the sparsity pattern of the product is a &lt;em&gt;subset&lt;/em&gt; of the sparsity patterns of the two matrices, and a cheap way to form the product is to take the sparser of the two matrices and for each one of its elements replace it by the product of the two matrices. This can be done in at most ${\cal O}(nnz_A + nnz_B)$ operations ($nnz$ = number of nonzero entries) which is at the very worst ${\cal O}(N^2)$ but in practice for sparse matrices is of course much less and typically only ${\cal O}(N)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'd then use your usual solver to solve the linear problem. This will in general cost much more than forming the product in place.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-25T02:01:04.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14699" PostTypeId="1" AcceptedAnswerId="14703" CreationDate="2014-09-25T13:02:12.953" Score="4" ViewCount="4330" Body="&lt;p&gt;The most up-to-date performance benchmarks comparison between R and Matlab that I could find are several years out of date: &lt;a href=&quot;http://mlg.eng.cam.ac.uk/dave/rmbenchmark.html&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt; &lt;a href=&quot;http://www.sciviews.org/benchmark/&quot; rel=&quot;nofollow&quot;&gt;2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is anyone aware of a more up-to-date comparison?&lt;/p&gt;&#xA;" OwnerUserId="9810" LastActivityDate="2014-09-25T15:03:00.513" Title="Is R or Matlab currently faster?" Tags="|matlab|performance|r|benchmarking|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14700" PostTypeId="1" CreationDate="2014-09-25T14:03:54.560" Score="5" ViewCount="1412" Body="&lt;p&gt;I'm trying to analyze a circuit card assembly (CCA). The biggest problem is always trying to mesh the thin copper layers along with the thicker epoxy layers between. I'm making the approximation that because the copper layers are so thin, and copper's thermal conductivity is so high, the normal-direction temperature change is negligible. Thus, they can be modeled as 2D or shell elements. I know Gmsh can handle this part, but can I (and if so, how?) also add in 3D elements for the epoxy layers and electrical components? TIA.&lt;/p&gt;&#xA;" OwnerUserId="10353" LastActivityDate="2014-10-07T22:22:27.007" Title="Mesh generator that can do 2D &amp; 3D elements combined?" Tags="|finite-element|mesh-generation|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14701" PostTypeId="2" ParentId="14700" CreationDate="2014-09-25T14:14:33.837" Score="1" Body="&lt;p&gt;Most static mesh generators create a mesh in stages.  First it meshes all the edges (1D spaces), then the faces of the geometry (2D spaces), and finally the volumes (3D spaces).  As such, gmsh is already doing what you want.  Place a face where the copper layer is located with volumes on each side for the epoxy.  You can mark the face as a different material group to differentiate it, then in your solver you handle that case in 2D, while handling the rest in 3D.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't used worked with GMSH outputs in a while, so I don't remember exactly how the outputs are formatted.  Likely the copper layers would be formatted very similarly to how external boundary faces would be formatted.  The hard part of mixing 2D and 3D geometry is in the solver, as the mesh generators already handle it in most cases.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2014-09-25T14:14:33.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14702" PostTypeId="1" AcceptedAnswerId="14707" CreationDate="2014-09-25T14:19:54.323" Score="3" ViewCount="1106" Body="&lt;p&gt;I'm looking for a method to solve a large overdetermined system of linear equations in a least squares sense. The matrix is dense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to use a method that works even with limited memory (we can't load the full matrix in RAM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix dimensions are something like 10,000,000 by 10,000, where I have a very large number of rows and a constant number of columns.&lt;/p&gt;&#xA;" OwnerUserId="3290" LastEditorUserId="276" LastEditDate="2014-09-25T21:01:48.850" LastActivityDate="2014-09-30T00:15:58.177" Title="Large overdetermined system of linear equations" Tags="|linear-algebra|linear-solver|matrix|dense-matrix|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14703" PostTypeId="2" ParentId="14699" CreationDate="2014-09-25T15:03:00.513" Score="8" Body="&lt;p&gt;The benchmarks on the Julia website &lt;a href=&quot;http://julialang.org/#High-Performance.JIT.Compiler&quot;&gt;1&lt;/a&gt; &lt;a href=&quot;http://julialang.org/benchmarks/&quot;&gt;2&lt;/a&gt; include R and Matlab as competitors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that these are benchmarks focusing on testing the pure speed of the language, not the quality of the underlying linear algebra or FFT libraries. The speed for operations that are outsourced to these libraries (such as a large matrix multiplication) can vary a lot depending on the actual libraries used, especially for open source programs: R linked against &lt;code&gt;libblas&lt;/code&gt; is surely much slower than R linked against &lt;code&gt;libopenblas&lt;/code&gt; or MKL).&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-09-25T15:03:00.513" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14704" PostTypeId="1" AcceptedAnswerId="14794" CreationDate="2014-09-25T18:46:37.470" Score="3" ViewCount="850" Body="&lt;p&gt;I've been reading up on the concept of entropy as a useful way of developing a natural residual for a system of equations. For example, the entropy function for compressible Navier-Stokes is &#xA;$$H(U)=-\rho \ln(p\rho^{-\gamma})$$ &#xA;with entropy flux &#xA;$$F^i(U)=-\rho \ln(p\rho^{-\gamma})u_i$$ &#xA;and for isentropic flow the entropy function is&#xA;$$H(U)=\frac{p}{\gamma-1}+\frac{1}{2}\rho u_i u_i$$&#xA;with entropy flux&#xA;$$F^i(U)=(e+p)u_i\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am unclear on however, is the process by which to derive such an entropy function for a given system of equations. I'm aware it is probably not possible to generate one in general, but I'm interested in systems which are generally well behaved by other criteria, so I'm optimistic an entropy function can be derived for them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Google searches have not yielded constructive help on the topic, so could someone point me to a source on how to derive these entropy functions?&lt;/p&gt;&#xA;" OwnerUserId="3309" LastActivityDate="2014-10-07T15:10:50.157" Title="Deriving entropy function and entropy flux for a system of equations" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14705" PostTypeId="2" ParentId="14700" CreationDate="2014-09-25T21:26:23.693" Score="4" Body="&lt;p&gt;If I understand what you're saying, you have something like a sandwich of epoxy layers that have appreciable thickness (modeled in 3D) and a thin copper layer between them that has negligible thickness (modeled in 2D). In that case, you probably want to use some sort of 2D cohesive element for the negligibly thick copper layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you opt for that approach, there are two basic methods to try:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;intrinsic cohesive elements: you can insert the cohesive elements implicitly, so you restructure your equations to solve for a jump across the copper layer&lt;/li&gt;&#xA;&lt;li&gt;extrinsic cohesive elements: you insert the cohesive elements explicitly by duplicating nodes along the copper layer interface, and tweak the mesh connectivity so that the copper layer is actually represented as a gap in the mesh (a crack); you solve for the temperature on either side of the layer (I'm assuming you're doing thermal modeling)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the extrinsic approach, Gmsh has a plug-in called Crack that's supposed to do all of the mesh arrangement for you, where you label the interface as a Physical Group (see the Gmsh documentation), label the endpoints with another Physical Group, generate your mesh without the crack, and then postprocess the mesh with the crack plugin. &lt;a href=&quot;http://www.geuz.org/pipermail/gmsh/2014/009001.html&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is an example of how to do that with a really simple mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with Crack is that it will insert duplicate nodes correctly, but it will louse up the connectivity of edges and elements. In theory, if you have say, just two epoxy layers separated by a thin copper layer, you should get two connected components in your mesh, and no edges, faces, or 3D elements should cross the copper interface. In practice, you can get edges and 3D elements (well, I tried this with a 2D test case, so I had 2D elements that spanned the cracked interface in my mesh). I &lt;a href=&quot;http://www.geuz.org/pipermail/gmsh/2014/009224.html&quot; rel=&quot;nofollow&quot;&gt;described this behavior&lt;/a&gt; to the developers and I haven't heard back from them as to whether it is a bug, or intentional. Regardless, for my application, I couldn't use the resulting mesh, and I suspect it won't work for your problem either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can, however, use Gmsh or some other generator anyway to generate a 3D mesh, and then manually modify it within your code to duplicate nodes along a crack and connect those duplicate nodes to the mesh in the proper fashion. The best paper to describe this process I could find is &lt;a href=&quot;https://e-reports-ext.llnl.gov/pdf/339953.pdf&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt; by Mota, Knap, and Ortiz; most of the other papers I found weren't terribly useful (Paulino's work didn't go into the algorithms very much; an earlier algorithm by Ortiz didn't yield the correct result, as mentioned in the linked paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar algorithms for inserting cohesive elements and changing the mesh connectivity are in PETSc within &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/index.html&quot; rel=&quot;nofollow&quot;&gt;DMPlex&lt;/a&gt;, such as &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexConstructCohesiveCells.html&quot; rel=&quot;nofollow&quot;&gt;DMPlexConstructCohesiveCells&lt;/a&gt;, although I don't think there's very much documentation on them (there are a couple algorithms papers that describe it, but I found the math in the Ortiz paper easier to understand, despite knowing nothing about algebraic topology), so if you use PETSc, there's no need to reinvent the wheel, as long as you can figure out how to use the cohesive element insertion functions; the PETSc mailing list should be able to help you with that if you decide to go that route.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you understand the gist of the Mota-Knap-Ortiz paper, you could also write your own routines for manipulating the mesh, which is what I did in a private fork of &lt;a href=&quot;https://github.com/inducer/meshpy&quot; rel=&quot;nofollow&quot;&gt;MeshPy&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The difference between the intrinsic and extrinsic approaches, from what I understand, is that in the extrinsic approach, you modify your mesh explicitly, and then mesh nodes correspond to degrees of freedom in your problem. In the intrinsic approach, you don't modify your mesh, you just modify the system of equations you get from your FEM discretization to get a system of equations equivalent to what you would get if you took the extrinsic approach. I don't think the intrinsic approach is any easier; if you can construct the equations in the intrinsic approach, you can probably back out what your mesh would have to look like in the extrinsic approach, and vice versa. Then again, I've only implemented the extrinsic approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you can get the mesh you need with a break in it, then setting up the equations is relatively easy. The interfacial contributions look very similar to a discontinuous Galerkin approach, so you can calculate the terms in your weak form by analogy. Any really basic treatment of discontinuous Galerkin, like Chapter 14 in &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3642332862&quot; rel=&quot;nofollow&quot;&gt;The Finite Element Method: Theory, Implementation, and Applications (Texts in Computational Science and Engineering) by Larson and Bengzon&lt;/a&gt; will work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, Cubit is supposed to generate cohesive elements, but I could never get that functionality to work. Gmsh's Crack Plugin works sometimes, but not reliably. If your geometry is relatively simple, you might try to construct the mesh yourself by hand, or fix up a mesh generated by Gmsh, but for more complicated geometries, you definitely want to automate the procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also use an XFEM approach. I can't help you with how to implement it, but you could look at &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0927025610001084&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;, which is also useful in providing a linear heat transfer problem similar to the one you are describing. The linear heat transfer problem is useful as a test case for debugging your implementation.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-09-26T21:49:09.463" LastActivityDate="2014-09-26T21:49:09.463" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14707" PostTypeId="2" ParentId="14702" CreationDate="2014-09-26T00:41:53.700" Score="8" Body="&lt;p&gt;One option here would be to form the normal equations $A^{T}Ax=A^{T}b$ and solve them by Cholesky factorization of the resulting $n$ by $n$ matrix.  This squares the condition number of the problem which could potentially be a significant problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forming $B=A^{T}A$ doesn't require more than $O(n^2)$ memory, assuming that you can access the rows of $A$ one at a time.  Basically, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;B=zeros(n,n);&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for i=1:m&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;B=B+A(i,:)'*A(i,:);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;end&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this very large number of rows, it might be more appropriate to simply randomly sample from the rows of $A$ rather than using the entire matrix.  This will of course depend on your problem data.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If ill conditioning of $A^{T}A$ is a significant problem, then you might also consider &quot;Q-less&quot; QR factorization methods in which you perform orthogonal transformations on $A$ to compute $R$ from the QR factorization without computing or storing $Q$ and you simultaneously perform the appropriate orthogonal transformations to the right hand side of your least squares problem.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-09-26T00:41:53.700" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14708" PostTypeId="2" ParentId="14689" CreationDate="2014-09-26T05:50:46.167" Score="2" Body="&lt;p&gt;Your method is an example of a multistep linear method for a&#xA;second-order ODE. Have a look at Hairer, Wanner, Norsett, &quot;Solving&#xA;Ordinary Differential Equations I&quot;, they discuss this method on&#xA;p. 464, and call it the Implicit Stormer method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multistep linear methods for a first-order ODE are those methods that&#xA;have the form&#xA;$$ \alpha_0 y_n + \alpha_1 y_{n+1} + \cdots + \alpha_s y_{n+s} =&#xA;\beta_0 f_n + \beta_1 f_{n+1} + \cdots + \beta_s f_{n+s}, $$&#xA;where $\alpha$'s and $\beta$'s are chosen for consistency and numerical&#xA;stability. In your case, one way to generalize is to use&#xA;$$ y_{n+1}-2y_{n}+y_{n-1} = \Delta t^2 \big(\beta_{-1}f_{n+1} +&#xA;\beta_0 f_n + \beta_1 f_{n-1} + \beta_2 f_{n-2} + \cdots), $$&#xA;and choose the coefficients appropriately. Hairer, Wanner, Norsett&#xA;list other related methods as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Order.&lt;/strong&gt;&#xA;I tested your method on&#xA;$$\ddot x = x, \qquad x(0) = 1, \quad x(-h) = e^{-h}, \quad h = 1/n $$&#xA;with $n=2^{\{3,\ldots,13\}}$, integrating to $t=1$. The error of $x(1)$ has order $4$ as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a method for a second-order ODE with local error of order $p+2$ the global error has order $p$. (So some of my comments are dead wrong.) The easy way to see this is to note that the inhomogeneous recurrence equation&#xA;$$ y_{n+1}-2y_n+y_{n-1}=d $$&#xA;has the particular solution $y_n=\frac12 dn^2$, which is $O(h^p)$ when $d=O(h^{p+2})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Symplecticity.&lt;/strong&gt;&#xA;When a system of ODEs comes from a Hamiltonian, i.e., when it is&#xA;$p'=-H_q$, $q'=H_p$, it is &lt;em&gt;symplectic&lt;/em&gt;: the area (or volume) $dp\,dq$&#xA;in $\mathbb{R}^2$ is preserved by the flow defined by the&#xA;equation. Here $p,q$ are generalized momentum and generalized&#xA;coordinate; the space of all $\{(p,q)\}$ is called the phase space and&#xA;all this is standard in physics textbooks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Geometric Numerical Integration&quot; by Hairer, Wanner, and Lubich, mentioned by Geoff Oxberry in a comment above, is a good reference for symplectic methods.&#xA;The definition of symplecticity for an ODE method is the same as in&#xA;physics: a method that defines a map $(x_n,v_n)\mapsto (x_{n+1},&#xA;v_{n+1})$ between consecutive timesteps is &lt;em&gt;symplectic&lt;/em&gt; if it&#xA;preserves area in $\mathbb{R}^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Geoff Oxberry said, to establish symplecticity it is not enough to numerically establish that it conserves energy of a pendulum for a reasonably large number of periods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you have written it, I would say your method is not symplectic&#xA;because it does not define a &lt;em&gt;flow&lt;/em&gt;, let alone a symplectic flow. This&#xA;says nothing about whether it conserves or exponentially decays&#xA;energy, just that it cannot be said to be symplectic in this form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the Stormer-Verlet method&#xA;$$ y_{n+1}-2y_n+y_{n-1} = h^2 f_n $$&#xA;does not define a symplectic flow, but it can be rewritten as&#xA;$$ v_{n+\frac12} = v_n+\frac h2f_n, \quad y_{n+1} = y_n+h&#xA;v_{n+\frac12}, \quad v_{n+1} = v_{n+\frac12} + \frac h2f_{n+1}, $$&#xA;which does define a map $(y_n,v_n)\mapsto (y_{n+1},v_{n+1})$, and it&#xA;is symplectic because $dy_{n+1}\wedge dv_{n+1} = dy_n\wedge dv_n$&#xA;(&quot;$\wedge$&quot; is the anti-symmetric product defined on differential&#xA;forms; it is the product preserved by symplectic maps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I tried to write your method in a form applied to $(y_n,&#xA;v_n)$, I got (for an arbitrary parameter $\theta$):&#xA;$$ y_{n+1} = y_n + hv_n + \tfrac1{12}h^2f_{n+1} +&#xA;h^2(\theta-\tfrac1{12})f_{n}, \\&#xA;v_{n+1} = v_n + h\theta f_{n} + h(1-\theta)f_{n+1}, $$&#xA;for which I got&#xA;$$ dy_{n+1}\wedge dv_{n+1} =&#xA;\frac{1-\frac{h^2}{12}f'_n}{1-\frac{h^2}{12}f'_{n+1}} dy_{n}\wedge&#xA;dv_n. $$&#xA;So unless I made a mistake here, your method is only symplectic when&#xA;the r.h.s. of $y''=f(y)$ is linear in $y$. When it's not linear, it&#xA;will scale volume in phase space by a factor $1+O(h^2)$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-09-26T22:31:09.200" LastActivityDate="2014-09-26T22:31:09.200" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14709" PostTypeId="1" AcceptedAnswerId="14712" CreationDate="2014-09-26T10:24:42.503" Score="3" ViewCount="310" Body="&lt;p&gt;Helo, every one. May I ask for help about how to solve this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{align}&#xA; &amp;amp;  \text{max}_{x_i} \quad |\sum_{i=1}^{4} a_i x_i | \\&#xA;&amp;amp;  s.t. \quad \sum_{i=1}^4 x_i^2=1&#xA;\end{align}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to find the optimal $x_i$, the $a_i$ is known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much. Anyone can give me some tips?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best&lt;/p&gt;&#xA;" OwnerUserId="10363" LastEditorUserId="10363" LastEditDate="2014-09-26T10:58:22.943" LastActivityDate="2014-09-26T15:03:02.850" Title="How to solve this optimization problem with abs object function?" Tags="|optimization|nonlinear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14710" PostTypeId="2" ParentId="14702" CreationDate="2014-09-26T13:29:41.270" Score="2" Body="&lt;p&gt;I think the answer to your problem is called Recursive Least Squares. Basically you treat each row one after the other, and the algorithm is based on the Woodbury matrix identity to update the inverse matrix.&#xA;Enjoy ;)&lt;/p&gt;&#xA;" OwnerUserId="1990" LastActivityDate="2014-09-26T13:29:41.270" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14712" PostTypeId="2" ParentId="14709" CreationDate="2014-09-26T14:51:09.093" Score="6" Body="&lt;p&gt;It looks like you are optimizing over the surface of a ball. That constraint is non-convex, but I think with your objective the solution will always be on the surface, so you can relax it to be $ \leq 1 $ and now its convex - thats a good start. So lets make that change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you ignore the absolute sign, you are maximizing a linear objective over a ball. Thats easy. So solve two optimization problems&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{align}&#xA; &amp;amp;  \text{max}_{x_i} \quad \sum_{i=1}^{4} a_i x_i  \\&#xA;&amp;amp;  s.t. \quad \sum_{i=1}^4 x_i^2 \leq 1&#xA;\end{align}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{align}&#xA; &amp;amp;  \text{max}_{x_i} \quad -\sum_{i=1}^{4} a_i x_i  \\&#xA;&amp;amp;  s.t. \quad \sum_{i=1}^4 x_i^2 \leq 1&#xA;\end{align}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and take the best of the two. Both these problems have a closed form solution - you can use the Karush-Kuhn-Tucker conditions to find it, or just throw it into a solver like Gurobi/CPLEX/Mosek/IPOPT/whatever you have a available that can handle quadratic constraints.&lt;/p&gt;&#xA;" OwnerUserId="9783" LastEditorUserId="9783" LastEditDate="2014-09-26T15:03:02.850" LastActivityDate="2014-09-26T15:03:02.850" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14713" PostTypeId="1" AcceptedAnswerId="16126" CreationDate="2014-09-26T18:34:33.197" Score="9" ViewCount="532" Body="&lt;p&gt;This is a follow-up to a &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14693/solving-hadamard-systems/&quot;&gt;different question&lt;/a&gt; I asked with more detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $v\in\mathbb{R}^n$, denote $D_v\in\mathbb{R}^n$ as the diagonal matrix with elements in $v$.  Given a &quot;tall&quot; matrix $B\in\mathbb{R}^{n\times m}$, I would like to solve the following optimization problem:&#xA;$$\min_{v\in\mathbb{R}^n} \|X-B^\top D_v B\|_{\mathrm{Fro}}^2$$&#xA;Assuming I calculcated it properly, first-order optimality gives the linear system $(BB^\top\circ BB^\top)v=(BX\circ B)\mathbb{1}$, where $\circ$ denotes the elementwise (Hadamard) product and $\mathbb{1}\in\mathbb{R}^n$ is the vector of all ones.  I have checked that this system is invertible for my application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, the matrix $BB^\top\circ BB^\top$ is very large relative to the size of $B$.  I can afford to take the SVD of $B$ (and that of $X$) but not to construct this large, dense matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything I can do to solve this system directly without resorting to an iterative solver?  If I have to do it iteratively, what is the fastest iteration for systems that come from this least-squares problem?&lt;/p&gt;&#xA;" OwnerUserId="5129" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-05-27T15:54:33.800" Title="Least-squares for a diagonal matrix" Tags="|linear-algebra|linear-solver|matrix|linear-system|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14714" PostTypeId="1" CreationDate="2014-09-26T20:41:20.550" Score="6" ViewCount="220" Body="&lt;p&gt;I want to solve a non linear system of equations of a particular kind. I find it hard to formulate clearly so I directly give a simple example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;f''=A(f,g)\\&#xA;g''=B(f,g)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the boundary conditions as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f'(0)=g'(0)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f'(\infty)=g'(\infty)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At $x=1$, f is continuous but there is a discontinuity for $g$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;f'(1^-)=f'(1^+), ~~~ f(1^-)=f(1^+) \\&#xA;g(1^-)=g_{left}, ~~~g(1^+)=g_{right} \\&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can see this as two problems, one for $x&amp;lt;1$, and one for $x&amp;gt;1$, but the boundaries conditions are coupled in $x=1$.&#xA;When I have an analytical solution, there is no big deal. Solve the problem for each side, then match the solution at $x=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not used with numerical methods and I don't even find how to call this problem. I have some ideas to tackle this but I wish to read a bit about that before.&lt;/p&gt;&#xA;" OwnerUserId="10366" LastEditorUserId="10366" LastEditDate="2014-09-29T15:39:53.457" LastActivityDate="2014-09-29T15:39:53.457" Title="Numerical methods for boundary-value ODEs with a jump condition" Tags="|ode|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14715" PostTypeId="2" ParentId="14714" CreationDate="2014-09-26T22:04:06.153" Score="4" Body="&lt;p&gt;These problems are usually called interface problems with jump conditions; an example would be the paper &lt;a href=&quot;http://www.math.ucsd.edu/~bli/publications/GongLiLi_SINUM08.pdf&quot; rel=&quot;nofollow&quot;&gt;Immersed-interface finite-element methods for elliptic interface problems with nonhomogeneous jump conditions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, they come up in PDEs in fracture mechanics to model discontinuities in stress due to cracks, but they can also come up in problems like thermal modeling or electromagnetic modeling to depict something called a &quot;contact resistance&quot; or basically an appreciable resistance due to a thin film.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know the location of the discontinuity, and it does not move or change in shape, you would typically fit a discretization to the discontinuity, so that the discontinuity coincides with nodes on your mesh. If you duplicate mesh nodes along the discontinuity -- in your case, it's 1-D, so you would have two nodes at $x = 1$, so that $x = 1^{-}$ is one node, and $x = 1^{+}$ is another node -- then you can use that discretization to form numerical methods. This approach is one usually taken for PDEs with spatial discontinuities. You could also track the interface implicitly using cut-cell discretizations, with level sets, or some sort of front tracking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the discontinuity is in 1-D, adaptive ODE integration methods are also an option, and in that case, one viable strategy is to do what you propose and integrate from your initial condition to $x = 1$, stop the integrator, reinitialize it with new initial conditions at $x = 1$ that reflect the jump condition, and restart the integrator.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-09-26T22:04:06.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14716" PostTypeId="2" ParentId="14700" CreationDate="2014-09-27T04:06:59.593" Score="2" Body="&lt;p&gt;Another approach is to tie together using constraint equations. You need to have the same number of DOFs at each node participating in a constraint equation. So, if this is structural problem (is it?) then shell elements have 6 DOFs at each node and 3D mesh nodes need to have the 6 DOFs also, namely 3 displacement and 3 rotatoinal DOFs. If it a scalar problem (heat equation or equivalently Poisson equation) then obviously nothing to do other than impose the constraint equations. in any case, if one has a conforming mesh, then the constraint equations are of simpler form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most FEA software do genertate constraint equations and coupling equations and pass them on to the solver as side constraints. The solver has to find a solution to the linear system subject to the condition that the solution vector lies in the subspace satisfying the constraint equations.&lt;/p&gt;&#xA;" OwnerUserId="9394" LastEditorUserId="9394" LastEditDate="2014-10-07T22:22:27.007" LastActivityDate="2014-10-07T22:22:27.007" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14717" PostTypeId="2" ParentId="14700" CreationDate="2014-09-27T17:58:03.653" Score="1" Body="&lt;p&gt;Based on my imperfect understanding of the geometry, an&#xA;idea occurred to me that you might want to consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm assuming the copper circuit traces are laid out on&#xA;a rectangular grid even though the circuits may be quite&#xA;different on either side of the epoxy insulating layer.&#xA;I suspect that a 1D representation of the copper circuit elements&#xA;is not a bad approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may not be very difficult to write a small function that&#xA;creates a conforming, rectangular, but not necessarily uniform&#xA;hexahedral mesh in the epoxy layers. Here are some details of&#xA;what I have in mind:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You start by constructing three lists of x, y, and z coordinate&#xA;values. The list of z-values contains (at least) the z-coordinates&#xA;of all circuit layers. The x and y lists contain the values of&#xA;the appropriate center-line locations of each 1D circuit trace at&#xA;all z-levels (I'm ignoring the case of 2D circuit regions for now).&#xA;With these 3 lists, constructing a hexahedral mesh in the epoxy&#xA;layers with a triply-nested loop is straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step is constructing the mesh of 1D circuit elements (again,&#xA;I'm ignoring the case of 2D circuit elements for now). For each&#xA;circuit, you need to be able to construct a data structure that I'm&#xA;loosely calling a &quot;netlist&quot;. By that I mean a list of the starting and&#xA;ending points of each trace and whether it runs parallel to the x or&#xA;y axis. The nodes corresponding to those end points were&#xA;already defined in the first step. Finding them (their ids) is simply&#xA;a matter of indexing into the x, y, and z lists and computing a global&#xA;offset into the node list. Given the node ids, you can define one or&#xA;more 1D elements for each trace. The process is repeated for every&#xA;trace at every circuit level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My proposed solution is based on the fact that creating good meshes directly&#xA;from the geometry, when there are very thin layers, is quite difficult.&#xA;Most general 3D meshers create tetrahedral elements which are undesirable not &#xA;just in the very thin copper layers (as you've pointed out) but also in the&#xA;fairly thin epoxy layers. Hexahedral elements are much better suited to thin layers but I don't know of any free/open-source hexahedral meshers&#xA;for this class of problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to look for existing meshing software, I think that meshing the&#xA;copper conductor layers with one hex element through the thickness is a more&#xA;straightforward approach. I think you are more likely to find meshers that&#xA;can do this and that the computational disadvantages, compared with creating a 2D circuit mesh&#xA;between the 3D regions, are minor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to consider commercial software, CoventorWare (&lt;a href=&quot;http://www.coventor.com/&quot; rel=&quot;nofollow&quot;&gt;http://www.coventor.com/&lt;/a&gt;)&#xA;has a &quot;Manhattan&quot; hexahedral mesher that I think would be suitable for this&#xA;application. (The term Manhattan comes from the fact that the element mesh is laid out on&#xA;a rectangular grid.)&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2014-09-29T16:29:22.540" LastActivityDate="2014-09-29T16:29:22.540" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14718" PostTypeId="1" CreationDate="2014-09-27T18:49:03.253" Score="2" ViewCount="115" Body="&lt;p&gt;I am a Java programmer who has reached the limits of brute computer power. My relational database (and non relational databases) is not producing results quick enough and I have hit a bottleneck in the software so I am turning to math possibly improve my algorithm to my problem. I have data, lots of data! I am collecting weather data for 30 days and each day contains just over 100,000 variables. The variables can reoccur on the other days or not. In total there are 12 billion different variables which can occur. So on each day only 100,000 of these 12 billion occur, but again these can reoccur on any other day or not. I am trying to see if say varA occurs varH occured 20 times in a 30 day period, or when varB occurs varZ occured 12 times in those 30 days. So currently my approach is running a nested loop through each day and counting each entry in a master count table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ex Master count&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;varA-varA : count&#xA;varA-varB : count&#xA;varA-varC : count&#xA;.&#xA;.&#xA;.&#xA;varZ-VarX : count&#xA;varZ-VarY : count&#xA;varZ-VarZ : count &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have been thinking of different ways to solve this. One approach I thought of is to break each day into a set, then extract the similarities between them somehow. Is there a method to count combinations in sets quickly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry, I use the word 'variable' not in the right context, weather event might be more accurate. The events do not have any values associated with them, they are descriptive. For example an event can be &quot;HotWeatherAtLong22Lat48&quot; or &quot;LowPressureAtLong65Lat78&quot;. I have 12 billion events and only 10^5 of these events can on a single day. Some of these events might not occur at all in the 30 day period, some might occur on all 30 days, some might occur only on some of days, it is what seems quite random, but I would like to find the number of times a pair of events occur together within the 30 days –&lt;/p&gt;&#xA;" OwnerUserId="10370" LastEditorUserId="9667" LastEditDate="2014-10-01T20:38:05.703" LastActivityDate="2014-10-03T01:30:54.007" Title="fastest and most efficent way to count all combinations in many sets and sum them together" Tags="|optimization|algorithms|performance|statistics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14719" PostTypeId="2" ParentId="14718" CreationDate="2014-09-28T16:21:16.917" Score="1" Body="&lt;p&gt;If I understand your description correctly, then your problem falls into a class where there are a very large number of queries you could make, each of which is relatively cheap, but taken together they are too expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sensible strategy in that case would be not to precompute the answers to all possible queries and store them, but to only run a query when you actually need it. The question is what you are going to learn if you know the answers to all of the billions of queries you could possibly do. Presumably, there are questions you want to ask, and these would be associated with specific queries that you could run when you need to.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-28T16:21:16.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14720" PostTypeId="1" AcceptedAnswerId="14721" CreationDate="2014-09-28T21:14:17.440" Score="1" ViewCount="371" Body="&lt;p&gt;How do I ensure that my function below is well conditioned as $s$ approaches $\infty$? The problem I get is that for large $s$ the function returns an indeterminate form $\frac{0}{0}$. I would otherwise have expected the function to increase monotonically from 0 to 1 as $s$  approaches $\infty$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;f(s) = \frac{x_s^2+\sqrt{1-x_s^2}-1}{\sqrt{x_s^2(x_s(\sqrt{1-x_s^2}-1))^2+(x_s^2+\sqrt{1-x_s^2}-1)^2}}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;x_s = e^{-s}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;def func(s):&#xA;    xs = np.exp(-1*s)&#xA;    num = xs**2+np.sqrt(1-xs**2)-1&#xA;    den = np.sqrt(xs**2*(xs*(np.sqrt(1-xs**2)-1))**2+(xs**2+np.sqrt(1-xs**2)-1)**2) &#xA;    return num/den&#xA;&#xA;s = np.linspace(0,20, 1000)&#xA;plot(s, func(s))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/HG49G.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="7864" LastEditDate="2014-09-29T08:07:37.490" LastActivityDate="2014-09-29T08:07:37.490" Title="How to deal with indeterminate function limit?" Tags="|python|numerical-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14721" PostTypeId="2" ParentId="14720" CreationDate="2014-09-28T21:40:07.577" Score="2" Body="&lt;p&gt;The function is really smooth as $s \rightarrow \infty$. This allows you to do the following strategy:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For $s\le s_0$, do the usual evaluation, i.e., return $f(s)$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For $s&amp;gt;s_0$, define the function $g(h)=f(1/h)$, do a Taylor expansion around $h=0$ (i.e., around $s=\infty$) and return $\bar g(h)|_{h=1/s}$ where $\bar g$ contains the first few Taylor terms of the expansion of $g(h)$. In the evaluation of these Taylor terms, you need to compute the $k$th derivatives $g^{(k)}(0)$ which will involve factors $\frac 00$ for which you will have to apply l'Hopital's rule.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-09-28T21:40:07.577" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14723" PostTypeId="1" AcceptedAnswerId="14728" CreationDate="2014-09-29T16:26:33.513" Score="2" ViewCount="2161" Body="&lt;p&gt;The solution to the ode below looks like it is monotonically increasing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/He1kK.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However on closer inspection we see that it is not: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xKJBC.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I ensure that the numerical solution is monotonically increasing?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.integrate import odeint&#xA;&#xA;def f1(theta):&#xA;    return -1*np.cos(theta)/np.tan(theta)&#xA;def f2(theta):&#xA;    return -1*np.cos(theta)&#xA;&#xA;def f3(y, t) :&#xA;    return 1/np.sqrt(f1(y)**2+f2(y)**2)&#xA;&#xA;s = np.linspace(0,30,100)&#xA;theta = odeint(f3, -np.pi/2, s)&#xA;&#xA;plt.plot(s, theta)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Edit 1&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;As Geoff Oxberry mentions in his answer the RHS of the ODE is infinite at the initial condition. So I thought of finding another point to start integrating from which should be on the same trajectory as in the original case above. I now solve $\frac{ds}{d\theta} = f_3(\theta)$ instead of $\frac{d\theta}{ds} = \frac{1}{f_3(\theta)}$. The former does not have the singularity at the initial condition $s=0$ at $\theta = -\pi/2$. I then integrate to a arbitrary $\theta = -\pi/4$, and get the value of $s$ there. I then use this point as my initial condition in the original problem.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f4(theta):&#xA;    return np.sqrt(f1(theta)**2+f2(theta)**2)&#xA;&#xA;# Get another initial condition&#xA;&#xA;theta0 = -np.pi/4&#xA;s0 =integrate.quad(f4, -np.pi/2, theta0) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However I still get the problems of oscillations for this case, so I'm not sure what to make of this method. &lt;/p&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="7864" LastEditDate="2014-10-04T15:22:32.817" LastActivityDate="2014-10-04T15:22:32.817" Title="Why does scipy's odeint function give a non-monotonic solution for a problem whose solution should be monotone?" Tags="|python|numerical-analysis|ode|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14725" PostTypeId="1" AcceptedAnswerId="14727" CreationDate="2014-09-29T19:57:15.387" Score="2" ViewCount="177" Body="&lt;p&gt;I want to analyze buckling in a composite using the FEM. So far I have studied this references&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Zdenek P Bazant, Luigi Cedolin. Stability of Structures: Elastic, Inelastic, Fracture and Damage Theories. World Scientific Publishing Company, 2010.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Wsben Byskov. Elementary Continuum Mechanics for Everyone.  Springer, 2013.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Is there a better reference that tackle the problem of buckling from a numerical point of view?&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2016-09-09T18:33:34.447" LastActivityDate="2016-09-09T18:33:34.447" Title="Buckling reference using the FEM" Tags="|finite-element|reference-request|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14726" PostTypeId="2" ParentId="14723" CreationDate="2014-09-29T20:07:16.820" Score="4" Body="&lt;p&gt;I suggest reducing the tolerance of odeint. Set the &quot;rtol&quot; or &quot;atol&quot; parameters to something smaller than their default which is 1.49012e-8. See the scipy documentation for odeint &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5516" LastActivityDate="2014-09-29T20:07:16.820" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14727" PostTypeId="2" ParentId="14725" CreationDate="2014-09-30T00:02:35.647" Score="4" Body="&lt;p&gt;I suggest you take a look at this site:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://shellbuckling.com/index.php&quot; rel=&quot;nofollow&quot;&gt;http://shellbuckling.com/index.php&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dave Bushnell has pretty much written the book on computational techniques&#xA;for buckling analysis. There is a wealth of information on both classical and&#xA;computational approaches to buckling analysis here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would start with his survey paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://shellbuckling.com/papers/bosor4/1981.pitfalls.pdf&quot; rel=&quot;nofollow&quot;&gt;http://shellbuckling.com/papers/bosor4/1981.pitfalls.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has remarkable insights into what buckling means both physically&#xA;and numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;His software, which you can download from this site, is based on finite difference&#xA;methods but the concepts are equally applicable to finite elements.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-09-30T00:02:35.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14728" PostTypeId="2" ParentId="14723" CreationDate="2014-09-30T00:04:08.483" Score="6" Body="&lt;p&gt;If you want a numerical method to satisfy certain properties, you have to prove (or construct the method in such a way) that these properties hold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;conservative finite-volume formulations for conservation laws&lt;/li&gt;&#xA;&lt;li&gt;Runge-Kutta methods preserve linear invariants&lt;/li&gt;&#xA;&lt;li&gt;symplectic integrators preserve symplecticity, which is important for Hamiltonian systems&lt;/li&gt;&#xA;&lt;li&gt;symplectic Runge-Kutta methods preserve quadratic invariants&lt;/li&gt;&#xA;&lt;li&gt;total variation diminishing discretizations (TVD) are monotonicity-preserving (typically, these are spatial discretizations for hyperbolic problems)&lt;/li&gt;&#xA;&lt;li&gt;strong-stability preserving discretizations in time are those that preserve the TVD property, if a spatial discretization is TVD when integrated in time using forward Euler&lt;/li&gt;&#xA;&lt;li&gt;you can force certain integration schemes (example: BDF implementation in SUNDIALS) to respect bounds in properties of ODE solutions (example: you're integrating an ODE tracking the mass of an object; that object's mass must be nonnegative)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Tightening the tolerances, as DavidKetcheson correctly points out, will not necessarily make the numerical solution monotonic. There's no way to guarantee that. To summarize the following discussion, in a practical sense, if you are approaching a steady-state monotonically, tightening tolerances will limit spurious oscillations (specifically, it will limit their amplitude, but not necessarily their total variation), but it will not eliminate them for certain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your particular ODE is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot{\theta} = \frac{1}{\sqrt{\frac{\cos^{4}{\theta}}{\sin^{2}{\theta}} + \cos^{2}}{\theta}}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some simplification and consulting trigonometric identities, it becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot{\theta} = \pm\tan{\theta},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and since you start at $\theta(0) = -\pi/2$, your right-hand side is singular anyway, and your &quot;solution&quot; is dubious. (The reason for the ambiguity in sign via $\pm$ involves a trigonometric identity for sine in terms of cotangent, and the choice of sign depends on which quadrant $\theta$ is in.) The only reason you see a solution at all is due to quirks in floating-point arithmetic; when I evaluate your right-hand side at zero, I get 16331239353195370.0, probably a system- and hardware-dependent result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $\theta \in (-\pi/2, 0]$, you should have $\dot{\theta} = -\tan{\theta}$, and since $-\tan{\theta}$ is nonnegative on $(-\pi/2, 0]$ and $\tan{0} = 0$, any initial condition in $(-\pi/2, 0]$ should yield a monotonically increasing solution. Again, this property isn't guaranteed by the numerical methods you are using. Furthermore, if $\theta \in (0, \pi/2)$, your equation becomes $\dot{\theta} = \tan{\theta}$ to preserve positivity of the right-hand side (the original form of the equation ensures positivity), and $\tan{\pi/2}$ is of course, again, singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main function of your tolerance here is to ensure that you get the right steady-state, and because floating-point arithmetic is mysterious, you somehow get what looks like a &quot;correct&quot; answer (for $\theta(0) = -\pi/2 + \varepsilon$ for some small $\varepsilon &amp;gt; 0$) despite the initial condition being at a singularity, and you see those deviations from the true steady state at $\theta = 0$ is because tolerances correspond to &quot;roughly zero&quot; in terms of estimating error in the computed solution. Due to quirks in floating-point arithmetic, your right-hand side won't necessarily evaluate to exactly zero, but because it will be sufficiently small relative to your tolerances, you should see no oscillations of amplitude larger than your tolerances (approximately, because this reasoning is heuristic, and not rigorous, but it should hold).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aside from reasons of stability or necessary precision, I am skeptical of extremely small tolerances unless they are required due to physical or numerical reasons. Undue demands on accuracy will unnecessarily restrict time step size and increase simulation execution times without yielding any appreciable improvement in results. As a counterexample, combustion problems are notorious for requiring small tolerances for certain species to get approximately correct numerical results; if certain species are never present, certain reactions cannot occur, even if those species concentrations are calculated in quantities unlikely to be measured, so in that case, small absolute and relative tolerances are warranted. For your particular problem, I suspect that with sufficiently large tolerances, you will see wild oscillations because you will overshoot appreciably your steady state and reach the singularity at $\theta = \pi/2$, so the default tolerance is probably fine, but I see no good reason to decrease the tolerances here.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-10-01T01:12:34.397" LastActivityDate="2014-10-01T01:12:34.397" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14729" PostTypeId="2" ParentId="14702" CreationDate="2014-09-30T00:15:58.177" Score="0" Body="&lt;p&gt;For problems of the sizes that you are interested in, parallel computing seems like a good option. There are algorithms for computing QR (a key step in solving overdetermined least squares problem) in parallel computing. Mapreduce architecture is another architecture that can handle the data size that you are interested in. See this paper &lt;a href=&quot;http://arxiv.org/pdf/1301.1071v1.pdf&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/pdf/1301.1071v1.pdf&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are limited to a single processor, you could  implement the Tall skinny QR algorithm. Essentially, it tries to compute the 'R' in QR recursively and can be applied by sequentially loading parts of the matrix from disk. Details are provided in this paper &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn204.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lapack/lawnspdf/lawn204.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4558" LastActivityDate="2014-09-30T00:15:58.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14731" PostTypeId="1" CreationDate="2014-09-30T05:10:17.000" Score="3" ViewCount="740" Body="&lt;p&gt;I would like to model the 2D diffusion equation with Neumann BC's inside the following egg-shaped domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/6PBaS.jpg&quot; alt=&quot;IMG&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to use the finite difference method with the discretization implied by the image above, but while I can handle the interior points, I don't know how to handle the boundary values. I know how to search for and identify the boundary points, but I'm not sure what finite-difference approximation to the no-flux condition I should use. Currently I'm thinking of estimating the direction of the normal at each boundary point by looking at it's 8 nearest neighbors, and then, if the normal is approximately&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\textbf{n} = [n_1, n_2]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would add an equation like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{\nabla} c \cdot \textbf{n} \approx \frac{c_{i,j}-c_{i-1,j}}{\Delta x} \times n_1 + \frac{c_{i,j}-c_{i,j-1}}{\Delta y} \times n_2 = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though of course I'll need to be careful which side I take the derivative from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any suggestions or resources for this type of problem? Examples in 3D would be particuarly useful, as that's where I'm working towards.&lt;/p&gt;&#xA;" OwnerUserId="6799" LastEditorUserId="22284" LastEditDate="2019-01-19T14:35:02.907" LastActivityDate="2022-11-08T18:01:15.130" Title="2D Neumann Conditions on Irregular Domain" Tags="|finite-difference|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14732" PostTypeId="1" CreationDate="2014-09-30T05:52:51.633" Score="3" ViewCount="272" Body="&lt;p&gt;I am working on the flow stability problem. In this work the main complication is solving generalized eigenvalue problem for a large scale Non-Hermitian matrix. I need only one eigenvalue (most left on real axis), so I used shift-invert transformation. I used SLEPc library to solve this problem, especially Krylov-Shur method with MUMPS LU decomposition. It works fine for small problem (matrix size up to 500k). But real cases give me matrix size about 15M, so I just do not have such amount of memory to solve these cases with MUMPS. I tried to find an iterative method for my problem (method with small memory requirement), but they do not work for me. Generalized Davidson and Jacobi-Davidson are not converged. Also I tried iterative linear solver instead of direct MUMPS for Krylov-Shur method, but they also are not converged. I concentrated on one relatively small problem (about 70k matrix size) that solved fine using MUMPS and cannot find any iterative method to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible that iterative eigenvalue solver (or solver with small memory requirement) does not exist for my matrix? I will very appreciate if someone gives me an advice what I can do more to solve my problem.&lt;/p&gt;&#xA;" OwnerUserId="10388" LastActivityDate="2014-09-30T05:52:51.633" Title="Method with low memory requirement for large-scale eigenvalue problem" Tags="|petsc|eigenvalues|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14733" PostTypeId="1" CreationDate="2014-09-30T14:59:45.583" Score="3" ViewCount="1094" Body="&lt;p&gt;I would like to solve a system of differential equations $u+\nabla(\nabla\cdot u)=f$ or in more detail&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a+\partial_t^2a+\partial_t\partial_xb+\partial_t\partial_yc=f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b+\partial_x^2b+\partial_x\partial_ta+\partial_x\partial_yc=g$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c+\partial_y^2c+\partial_y\partial_tb+\partial_y\partial_xc=h$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with periodic boundary conditions on a regular grid by the method of Finite Differences. $a=a(t,x)$, $b=b(t,x)$ and $c=c(t,x)$ are my unknowns and $f=f(t,x)$, $g=g(t,x)$ and $h=h(t,x)$ are known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I discretized my domain in time ($t$) and space ($x,y$). What I already have now is a big equation system. The matrix has full rank. For a grid with only 3 nodes (=27 nodes overall, =81 equations) in each direction it looks like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/emvtE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When increasing the number of nodes the system gets to big very soon for my computers memory. That's why I actually want the system matrix to be symmetric in order to use Cholesky factorization (hopefully) or something else. But I am not sure if there exists a numbering of nodes which would make the system matrix symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In literature the commonly used example is the Laplace equation, where they just use a symmetric stencil in order to approximate $\Delta$. In contrast to that I have to use one dimensional finite difference for the approximation of e.g. $\partial_x^2$ in a 3 dimensional space. Furthermore I am using one-sided differences for the mixed derivatives at boundaries, which might destroy symmetric. However I am not sure if this really destroys symmetry. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only because I found &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7128/are-there-any-numerical-advantages-in-solving-symmetric-matrix-compared-to-matri&quot;&gt;this thread&lt;/a&gt; I've got the hope that there is a numbering of my nodes, which would make my system matrix symmetric. But actually I cannot imagine that, because a non-symmetric matrix cannot become a symmetric one by only permuting rows and columns in the same way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is there a way to get a symmetric system matrix for my problem? If not, what would you do instead in order to solve the system?&lt;/p&gt;&#xA;" OwnerUserId="8299" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-05-28T10:38:29.207" Title="Get symmetric Finite Difference matrix in non Laplacian settings" Tags="|linear-algebra|finite-difference|boundary-conditions|matrix|symmetry|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14734" PostTypeId="1" CreationDate="2014-09-30T22:31:38.367" Score="1" ViewCount="696" Body="&lt;p&gt;I am trying to solve Euler equations on unstructured grids. Consequently, the problem reduces to solving &lt;code&gt;Ax=b&lt;/code&gt; where &lt;code&gt;A&lt;/code&gt; is a sparse matrix. Right know I am using Gauss-Seidel (GS) in a serial manner however, I need a parallel code. Since the grid is unstructured I cannot use red-black GS. Is there a fast parallel solver for this kind of situation?&lt;/p&gt;&#xA;" OwnerUserId="8103" LastActivityDate="2014-11-04T01:52:18.893" Title="Parallel solver for sparse matrices on unstructured grids" Tags="|parallel-computing|sparse-matrix|unstructured-mesh|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14735" PostTypeId="1" AcceptedAnswerId="14738" CreationDate="2014-10-01T00:53:32.143" Score="2" ViewCount="2363" Body="&lt;p&gt;I have a large system of ODEs.  For various reasons, it is natural to store the values of the dependent variables in a multidimensional array.  For example, these values might represent the solution of a PDE at a point in a 2D or 3D grid, after semi-discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; takes a vector (i.e., a one-dimensional array) of initial values, and returns back vectors of the same size. If also requires a right-hand-side function $f$ that takes and returns vectors.  So one must flatten the state vector in order to use &lt;code&gt;scipy.integrate.odeint&lt;/code&gt;.  Meanwhile, inside $f$ it might be convenient to transform the state back to a multidimensional array, compute the derivative, and flatten the derivative array to a vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is this flattening(for passing to &lt;code&gt;f&lt;/code&gt;)-&gt;unflattening(for working within &lt;code&gt;f&lt;/code&gt;)-&gt;flattening(for returning something from &lt;code&gt;f&lt;/code&gt;) cycle the correct way to do things? Is there some other, better way to set things up (that might also happen to be efficient by avoiding the flattening/unflattening cycle?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="123" LastEditDate="2014-10-01T08:28:19.757" LastActivityDate="2014-10-01T08:29:46.517" Title="How best to use scipy.integrate's ODE solvers when state is not naturally stored as a vector?" Tags="|scipy|memory-management|data-storage|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14736" PostTypeId="2" ParentId="14733" CreationDate="2014-10-01T01:30:51.483" Score="1" Body="&lt;p&gt;Can you use timestepping instead of discretizing in time? You have a second order time derivative, but you can rewrite that as a first order system, i.e.&#xA;$\frac{\partial^2 u}{\partial t^2} = f$ can be rewritten by adding another equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial w}{\partial t} = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} - w = 0,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then use time-stepping methods for first order systems ODEs. This way, you'd end up with only a 2D discretization and not a 3D one (which saves a ton of memory). &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2014-10-01T01:30:51.483" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14737" PostTypeId="1" CreationDate="2014-10-01T02:47:29.803" Score="2" ViewCount="550" Body="&lt;p&gt;This is an attempt to reproduce &lt;a href=&quot;http://journals.aps.org/pre/abstract/10.1103/PhysRevE.52.2318&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moon &lt;em&gt;et al&lt;/em&gt;. 1995&lt;/a&gt;, and the author's copy can be obtained through &lt;a href=&quot;http://bechtel.colorado.edu/~balajir/my-papers/physical-review-E.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a benchmark, we estimate the time-lagged mutual information of a simple sine signal $\sin(0.02\pi t)$ using Gaussian kernels:&#xA;$$K(y)=\dfrac{\exp\left(-\dfrac{(y-y_i)^TS^{-1}(y-y_i)}{2h^2}\right)}{(2\pi)^{d/2}h^d\det(S)^{1/2}},$$&#xA;where $h$ is chosen to be &#xA;$$h=\left\{\dfrac{4}{(d+2)}\right\}^{1/d+4}n^{-1/(d+4)},$$&#xA;where $d$ is the dimension of the multivariate kernel, and $n$ is the number of data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a Matlab function (the code is at the end of the post).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose $x_t$ is the original time series $\sin(0.02\pi t)$ with 400 data points $t=1,2,\cdots,400$, and $x_{t-\tau}$ is $\sin[0.02\pi (t-\tau)]$. The author plotted the estimated mutual information $I_{x_t,x_{t-\tau}}$ as a function of $\tau$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/hxBwO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what I got from my code is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/yIR8y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While the qualitative features (like the diverging mutual information at lag 0, 50 and 100), but the magnitude is far off and the overall shape is not right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My 1st suspicion is the fact that I used different window widths for joint pdf and marginal pdf, and they may not be consistent. &lt;strong&gt;So my question is: how to consistently construct joint and marginal pdf estimate in the context of mutual information estimation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second question is: is there anything wrong in the code besides the possibly inconsistent pdf KDEs?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why I choose KDE over other methods is that I think KDE is simple and intuitive in concept and possibly also simple in implementation. Besides MI, I also want to estimate transfer entropy and possibly other information-theoretic measures as well. I feel that estimating joint probability is general enough to apply on various information-theoretic measures. Please let me know if you have comment on my argument above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't use anything that can be found only in Matlab. Code in R, Python, etc. is welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2: the Matlab code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [MI] = get_MI(xt, xt_lag)&#xA;% xt is original time series, xt_lag is the lagged one, both are column&#xA;% vectors&#xA;x_pair = [xt' xt_lag']; n=length(xt);&#xA;&#xA;d=1; h_d1=(4/(d+2))^(1/(d+4)) * n^(-1/(d+4));&#xA;&#xA;d=2; h_d2=(4/(d+2))^(1/(d+4)) * n^(-1/(d+4));&#xA;&#xA;MI = 0.;&#xA;&#xA;Sxy_pair = cov(x_pair); invS_pair = Sxy_pair\eye(2); detS_pair = det(Sxy_pair);&#xA;&#xA;Sxy_xt = cov(xt'); invS_xt = Sxy_xt\eye(1); detS_xt = det(Sxy_xt);&#xA;&#xA;Sxy_lag = cov(xt_lag'); invS_lag = Sxy_lag\eye(1); detS_lag = det(Sxy_lag);&#xA;&#xA;for ix=1:n&#xA;    Psq = p_mkde(x_pair(ix,:)', x_pair', h_d2, invS_pair, detS_pair);&#xA;    Ps = p_mkde(x_pair(ix,1), xt, h_d1, invS_xt, detS_xt);&#xA;    Pq = p_mkde(x_pair(ix,2), xt_lag, h_d1, invS_lag, detS_lag);&#xA;&#xA;    MI = MI + (log2(Psq) - log2(Ps) - log2(Pq));&#xA;end&#xA;&#xA;MI = MI/n;&#xA;&#xA;end&#xA;&#xA;function [pxy]=p_mkde(x,X,h,invS,detS)&#xA;&#xA;s1=size(X);&#xA;d=s1(1);&#xA;N=s1(2);&#xA;&#xA;pxy_sum=0;&#xA;for ix=1:N&#xA;    p2=(x-X(:,ix))'*invS*(x-X(:,ix));&#xA;    pxy_sum=pxy_sum+1/sqrt((2*pi)^d*detS)*exp(-p2/(2*h^2));&#xA;end&#xA;pxy=1/(N*h^d)*pxy_sum;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3512" LastEditorUserId="3512" LastEditDate="2014-10-05T11:02:48.367" LastActivityDate="2014-10-06T13:37:23.730" Title="Estimating mutual information of sine time series using 2D kernel density estimators" Tags="|matlab|information-theory|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14738" PostTypeId="2" ParentId="14735" CreationDate="2014-10-01T03:02:24.620" Score="1" Body="&lt;p&gt;The &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; function does not take a vector as input. It takes a function &lt;code&gt;f&lt;/code&gt;, an initial state &lt;code&gt;y0&lt;/code&gt; and a vector of points in time &lt;code&gt;t&lt;/code&gt; (and possibly a bunch of other optional arguments to control the integration strategy). So you will need to provide it with a function that calculates the right-hand-side of your ODE system, which will boil down to flattening (i.e. looping over all your 'bodies' and building a vector &lt;code&gt;ft&lt;/code&gt; which is the RHS evaluated at time &lt;code&gt;t&lt;/code&gt;. There is no way around this.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2014-10-01T03:02:24.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14739" PostTypeId="1" AcceptedAnswerId="14740" CreationDate="2014-10-01T04:07:56.393" Score="9" ViewCount="1899" Body="&lt;p&gt;The field of computational fluid dynamics (CFD) is dedicated to solving the Navier-Stokes equations (or some simplification of them). A subset of CFD, ocean and atmospheric models numerically solve the same equations for realistic applications. What are the differences and trade-offs between the general CFD approaches and the applied realistic cases?&lt;/p&gt;&#xA;" OwnerUserId="7726" LastEditorUserId="7726" LastEditDate="2014-10-01T17:52:32.803" LastActivityDate="2015-01-20T19:46:13.310" Title="What are the differences between CFD simulations and realistic ocean/atmosphere model simulations?" Tags="|fluid-dynamics|simulation|computational-physics|numerical-modelling|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14740" PostTypeId="2" ParentId="14739" CreationDate="2014-10-01T05:45:14.970" Score="13" Body="&lt;p&gt;Atmosphere and ocean have highly-stratified flows in which the Coriolis force is a major source of dynamics.  Maintaining geostrophic balance is extremely important and many numerical schemes are intended to be exactly compatible (at least in the absence of topography) to avoid radiating energy in gravity waves.  Due to the stratification, limiting vertical numerical diffusion is extremely important and special grids are often used (especially in the ocean) for that purpose.  Many methods are effectively 2.5-dimensional formulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For climate simulation over long time periods, conservation of energy and other fluxes (like salt) are often considered to be critical for statistically meaningful results.  Methods that are less accurate and have certain numerical artifacts may be chosen in order to avoid damping out the dynamics.  Note that long-term dynamics may not homogenize at continental scales averaged over multiple decades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Industrial CFD solvers tend to be used for flows that are more isotropic (genuinely 3D) and often neglect Coriolis.  They often have stronger forcing and thus less critical energy conservation requirements.  It is common to deal with strong shocks, in which case nonlinear spatial discretizations must be used, despite being more dissipative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because lab experiments can actually be performed for most industrial applications, the software experiences more validation.  Weather models also have constant validation, but climate models are almost impossible to validate due to the time scales involved and unavoidable over-fitting.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-10-01T05:45:14.970" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14741" PostTypeId="2" ParentId="14735" CreationDate="2014-10-01T08:29:46.517" Score="3" Body="&lt;p&gt;Presumably you are using numpy arrays for your data.  The key is to avoid making a copy when you reshape them.  For that, you want to ensure that you are just generating new &quot;views&quot; of your arrays.  Take a look at &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.reshape.html&quot; rel=&quot;nofollow&quot;&gt;the help page for numpy.reshape&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-10-01T08:29:46.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14744" PostTypeId="2" ParentId="14734" CreationDate="2014-10-01T13:04:20.107" Score="1" Body="&lt;p&gt;I used two very good parallel libraries for sparse matrix: &lt;a href=&quot;http://acts.nersc.gov/hypre/&quot; rel=&quot;nofollow&quot;&gt;Hypre&lt;/a&gt; and &lt;a href=&quot;http://mumps.enseeiht.fr/&quot; rel=&quot;nofollow&quot;&gt;MUMPS&lt;/a&gt;.&#xA;The first has very good AMG solver and several iterative methods, it can be run in parallel with either shared (OpenMP) or distributed memory (MPI). The second one implements direct LU factorization.&lt;/p&gt;&#xA;" OwnerUserId="10388" LastActivityDate="2014-10-01T13:04:20.107" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14745" PostTypeId="1" AcceptedAnswerId="16417" CreationDate="2014-10-01T14:02:23.477" Score="8" ViewCount="690" Body="&lt;p&gt;I am looking for a method or algorithm to produce a value that describes how different two meshes are geometrically but that have different topologies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example would be some CAD data that has had an FE mesh applied and the part after it has been manufactured and then scanned into a 3d triangulated mesh.  The manufactured part will be slightly different geometrically from the CAD part, but in addition, the meshes that have been applied are totally different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a good method to produce a measure of how similar or dissimilar they are geometrically.&lt;/p&gt;&#xA;" OwnerUserId="10397" LastActivityDate="2014-12-06T19:06:03.980" Title="Method to quantify geometric difference of two dissimilar meshes" Tags="|optimization|finite-element|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14746" PostTypeId="2" ParentId="14734" CreationDate="2014-10-01T14:17:46.237" Score="0" Body="&lt;p&gt;The question of how to solve sparse linear systems is ubiquitous in computational sciences. As a general rule the better the method, the harder and more complicated it is to implement. I suspect AMG (Algebraic Multi-grid) will be the best method for the problem you describe as multi-grid methods are among the fastest for solving linear systems currently. If you are implementing this yourself though, it will be much harder than your current GS (especially in parallel). If you are just looking for a package then I would use Kirill's answer. At the very least I would look into GMRES or BiCGStab as suggested above or even SOR. You can definitely find a faster method than GS.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2014-10-01T14:17:46.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14749" PostTypeId="2" ParentId="577" CreationDate="2014-10-02T10:34:09.027" Score="2" Body="&lt;p&gt;from a practical point of view it is relative easy to measure $\alpha$ and $\gamma$, the critical exponents of the heat capacity and the susceptibility,&#xA;to test your implementation.&#xA;To get a reasonable result you should use histogram reweighting to locate the maxima of the heat capacity and the the susceptibility.&#xA;Two practical introduction can be found &lt;a href=&quot;http://www.helsinki.fi/~rummukai/lectures/montecarlo_oulu/lectures/mc_notes4.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;&#xA;or in &quot;Histograms and All That&quot; which you can find &lt;a href=&quot;http://www.physik.uni-leipzig.de/~janke/Paper/histograms.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally the Binder cumulant $B$ may be worth looking at as a standard test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA; B = \frac{ 1- &amp;lt; M^4 &amp;gt;}{3 &amp;lt; M^2 &amp;gt; ^2 }&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The crossing point of $B(L)$ where $L$ is the linear size of your system gives a very good approximation of $T_c$. &lt;/p&gt;&#xA;" OwnerUserId="10406" LastActivityDate="2014-10-02T10:34:09.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14750" PostTypeId="2" ParentId="12948" CreationDate="2014-10-02T17:26:34.083" Score="2" Body="&lt;p&gt;With ParaView 4.2, your original code will work as expected. The following is now acceptable.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;output.PointData.append(numpyarray, &quot;name&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10411" LastActivityDate="2014-10-02T17:26:34.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14751" PostTypeId="1" AcceptedAnswerId="14762" CreationDate="2014-10-02T22:58:35.457" Score="4" ViewCount="466" Body="&lt;p&gt;I am using nonlinear optimizers such as BOBYQA to train a model with 10-20 parameters. It so happens that some of the parameters have high correlation. &#xA;Roughly speaking, imagine that you are fitting parameters $a,b$ in a function like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x)=a\frac{x}{1+|x|}-b\frac{x^2}{1+|x|^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The response to the change in parameters $a,b$ is much larger than the response in to the change of $a-b$. Would performance of Powell's nonlinear optimization algorithms such as BOBYQA, NEWUOA, etc., improve if I would &quot;rotate parameters&quot; to isolate gross responses from fine ones? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, would nonlinear optimizers perform better if instead of optimizing $a,b$ in the above function I would optimize for parameters $\alpha,\beta$ a function like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g(x)=\alpha(\frac{x}{1+|x|}+\frac{x^2}{1+|x|^2})+\beta(\frac{x}{1+|x|}-\frac{x^2}{1+|x|^2})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real model is more complicated, of course, and I'd like to find out whether changing parameters to minimize correlations has a chance to improve anything.&lt;/p&gt;&#xA;" OwnerUserId="5281" LastActivityDate="2014-10-04T00:34:50.083" Title="Does the amount of correlation of model parameters matter for nonlinear optimizers?" Tags="|optimization|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14752" PostTypeId="2" ParentId="46" CreationDate="2014-10-03T00:21:44.013" Score="-3" Body="&lt;p&gt;Data is easy; the API to get it can be tough.  I recommend &lt;a href=&quot;http://quandl.com&quot; rel=&quot;nofollow&quot;&gt;Quandl&lt;/a&gt;.  This site has over 10 million publicly available data sets accessible through one, easy, REST-ful API.  All data is returned in either CSV or JSON.  Or, if programming isn't your strong suit, there are easy ways to get the data into Excel.  R, Python and Ruby programmers will be right at home with native libraries.&lt;/p&gt;&#xA;" OwnerUserId="10417" LastActivityDate="2014-10-03T00:21:44.013" CommentCount="2" CommunityOwnedDate="2014-10-03T00:21:44.013" ContentLicense="CC BY-SA 3.0" />
  <row Id="14753" PostTypeId="2" ParentId="14718" CreationDate="2014-10-03T01:30:54.007" Score="0" Body="&lt;p&gt;Just a few thoughts/suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Could you &lt;em&gt;aggregate&lt;/em&gt; groups/sets of similar data points (by day, weather type, location, etc.) and store those in separate/temporary database tables?  Even if it takes a few days to do this massive conversion, you then should be able to query those resulting summary tables very quickly in a variety of different ways.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are the tables in your relational database &lt;em&gt;indexed&lt;/em&gt; in any way?  Could you create more indexes?  Perhaps this could improve performance of your queries.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You mention that the weather event data isn't numeric/scalar (for example, a weather event may be &quot;HotWeatherAtLong22Lat48&quot;).  Is there any you for you to convert this into numeric/scalar data?  For example, create new database columns for &quot;Longitude&quot;, &quot;Latitude&quot;, &quot;Temperature&quot;, etc.  This also might take a few days of crunching to convert all the data.  But once it's converted, you should be able to run all kinds of blazingly-fast queries.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10418" LastActivityDate="2014-10-03T01:30:54.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14754" PostTypeId="1" CreationDate="2014-10-03T10:36:52.940" Score="2" ViewCount="106" Body="&lt;p&gt;There's a vast amount of literature about the numerical solution of partial differential equations. But in none of my books or lectures did I find anything about coupled systems. Internet and library research also did not lead to any introductory material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone recommend a textbook or a paper with an introduction on how to treat coupled PDEs numerically? Particularly for concentrations and potentials in electrochemistry.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastEditorUserId="549" LastEditDate="2014-10-03T14:54:50.400" LastActivityDate="2014-10-03T14:54:50.400" Title="Looking for some literature about numerical solution of _coupled_ PDEs" Tags="|pde|numerical-analysis|multiphysics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14755" PostTypeId="2" ParentId="14401" CreationDate="2014-10-03T10:47:46.953" Score="3" Body="&lt;p&gt;To pass gradients from Python to NLopt, you have to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grad[:] = gradient()  # or np.copyto&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;NOT&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grad = gradient()  # wrong, a new pointer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Python_Reference#Assigning_results_in-place&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLopt Python doc&lt;/a&gt;&#xA;has a big warning on this; unfortunately it's not in the tutorial example you used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, on random startpoints, see NLopt &#xA;&lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms#MLSL_.28Multi-Level_Single-Linkage.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;MLSL&lt;/a&gt;&lt;br&gt;&#xA;&quot;a sequence of local optimizations (using some other local optimization algorithm)&#xA;from random startpoints ... clustering heuristic ...&quot;.&#xA;This theme has many many variations.&lt;br&gt;&#xA;Also, fwiw, this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/3029/testing-numerical-optimization-methods-rosenbrock-vs-real-test-functions&quot;&gt;plot&lt;/a&gt;&#xA;shows 3 NLopt methods from 10 random startpoints on a dozen academic test functions in 8d.&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-10-03T10:47:46.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14756" PostTypeId="2" ParentId="14754" CreationDate="2014-10-03T11:49:52.443" Score="4" Body="&lt;p&gt;Finite element methods for coupled systems are treated for example in Chapter 18 of &lt;a href=&quot;http://books.google.com/books?id=YocoaH8lnx8C&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;The Finite Element Method: Its Basis and Fundamentals&lt;/em&gt; by Zienkiewicz, Taylor and Zhu&lt;/a&gt;. It might also be useful to search for literature on particular coupled systems (presumably there's a specific one you are interested in) such as those modeling fluid-structure interaction or electromechanical systems (e.g., in piezoelectricity or cardiophysics).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2014-10-03T11:49:52.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14757" PostTypeId="1" AcceptedAnswerId="14759" CreationDate="2014-10-03T12:10:21.400" Score="1" ViewCount="174" Body="&lt;p&gt;I have the following set of linear equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_{m+1,n}+a_{m-1,n}+m(a_{m,n+1}+a_{m,n-1})+(m^{2}+n^{2})a_{m,n}=f_{m,n}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $m$ and $n$ run from 1 to $N$, so there are $N^2$ equations for the unknowns $a_{m,n}$. Fixed boundary conditions are used such that $a_{N+1,n}=0$ etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to solve this system numerically, and for that I need to first write it in matrix form $M a = f$. How can I programatically generate the coefficient matrix M?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT&#xA;Trying to follow @Wolfgang's suggestion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, so lets take the simplest case of N=2. Writing the equations by hand and extracting the coefficients results in the matrix equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[\begin{array}{cccc}&#xA;2 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0\\&#xA;1 &amp;amp; 5 &amp;amp; 0 &amp;amp; 1\\&#xA;1 &amp;amp; 0 &amp;amp; 5 &amp;amp; 2\\&#xA;0 &amp;amp; 1 &amp;amp; 2 &amp;amp; 8&#xA;\end{array}\right]\left[\begin{array}{c}&#xA;a_{1,1}\\&#xA;a_{1,2}\\&#xA;a_{2,1}\\&#xA;a_{2,2}&#xA;\end{array}\right]=\left[\begin{array}{c}&#xA;f_{1,1}\\&#xA;f_{1,2}\\&#xA;f_{2,1}\\&#xA;f_{2,2}&#xA;\end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if I follow @Wolfgang's suggestion and replace $(m,n)$ by $i$ I get a different matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[\begin{array}{cccc}&#xA;5 &amp;amp; 1 &amp;amp; 2 &amp;amp; 0\\&#xA;1 &amp;amp; 5 &amp;amp; 1 &amp;amp; 1\\&#xA;2 &amp;amp; 1 &amp;amp; 8 &amp;amp; 1\\&#xA;0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 10&#xA;\end{array}\right]\left[\begin{array}{c}&#xA;a_{1}\\&#xA;a_{2}\\&#xA;a_{3}\\&#xA;a_{4}&#xA;\end{array}\right]=\left[\begin{array}{c}&#xA;f_{1}\\&#xA;f_{2}\\&#xA;f_{3}\\&#xA;f_{4}&#xA;\end{array}\right]$$&lt;/p&gt;&#xA;" OwnerUserId="4656" LastEditorUserId="4656" LastEditDate="2014-10-04T12:25:42.393" LastActivityDate="2014-10-06T15:26:48.823" Title="writing linear system in matrix form" Tags="|linear-algebra|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14758" PostTypeId="2" ParentId="14754" CreationDate="2014-10-03T12:53:37.693" Score="4" Body="&lt;p&gt;There are of course many aspects to solving coupled problems (discretizations, solvers, ...). Assuming you are interested in solvers and preconditioners, then lectures 37 and 38 at &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; may have something for you. (Disclaimer: these are my own lectures.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-03T12:53:37.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14759" PostTypeId="2" ParentId="14757" CreationDate="2014-10-03T12:59:10.463" Score="2" Body="&lt;p&gt;You need to re-enumerate your degrees of freedom. Right now, you have two indices $1\le m,n\le N$. These span a rectangular array. Enumerate them from $i=0$ to $N^2-1$ (it turns out that zero-based indexing is so much simpler), for example by defining $i(m,n)=(m-1)+N*(n-1)$. In your case, this would corresponding to row-wise numbering. You can recover the original indices by observing that $m=(i\%N)+1$ and $n=(i/N)+1$ where the two operations are done in integer arithmetic. Then you can substitute these definitions in your formula everywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, if you write your linear system in terms of $a_i$, not $a_{m,n}$, then you'll get this set of equations:&#xA;$$&#xA;  a_{i+1}+a_{i-1}+(i\%N+1)(a_{i+N}+a_{i-N})+((i\%N+1)^2+(i/N+1)^2)a_i = f_i&#xA;$$&#xA;where I've again assumed that you do the operations $i\%N$ and $i/N$ in integer arithmetic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the equation you have for $i=0,\ldots,N^2$. Now write each of these $N^2$ equations down one by one and you get the corresponding matrix entries.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2014-10-06T15:26:48.823" LastActivityDate="2014-10-06T15:26:48.823" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="14760" PostTypeId="1" AcceptedAnswerId="14761" CreationDate="2014-10-03T15:19:17.873" Score="3" ViewCount="386" Body="&lt;p&gt;I have a system of coupled ODEs that I want to solve. The functions are A(x), B(x), C(x). It is a boundary values problem. I am using Matlab bvp4c.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I am not satisfied with my solutions. For the boundaries that are of interest for me, the solver fail (Matlab return &quot;a singular Jacobian encountered&quot;). For some other boundaries, the result depend of the initial guess. So I think that my system is ill defined. I am thinking of adding one constraint but I don't find how to implement it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to enforce $\int_{a}^{b}d x(A(x)+B(x)+C(x))=N$ ?&lt;/p&gt;&#xA;" OwnerUserId="10366" LastActivityDate="2014-10-03T18:23:01.403" Title="How to impose an integral conservation in solving ODEs boundary value problems (BVP)?" Tags="|matlab|ode|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14761" PostTypeId="2" ParentId="14760" CreationDate="2014-10-03T18:23:01.403" Score="6" Body="&lt;p&gt;You could augment your system of ODEs to include one more equation. If you let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;I(x) = \int_{a}^{x}(A(s) + B(s) + C(s))\,\mathrm{d}s,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then $I(a) = 0$, $I(b) = N$, $\dot{I}(x) = A(x) + B(x) + C(x)$, and you have another boundary value problem that you can solve in MATLAB using &lt;code&gt;bvp4c&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-03T18:23:01.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14762" PostTypeId="2" ParentId="14751" CreationDate="2014-10-03T22:20:39.790" Score="4" Body="&lt;p&gt;If possible, you want to choose a well-scaled objective for your problem, which would mean de-correlating your parameters as much as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a quadratic approximation to an objective function, correlated parameters correspond to objective function isosurfaces that are ellipsoidal in shape; the ideal shape for isosurfaces would be roughly spherical. In algebraic terms, ellipsoidal isosurfaces correspond to ill-conditioned Hessian matrices (the relationship being that the magnitude of the eigenvalues of the Hessian corresponds to the lengths of the semiaxes of the ellipsoids), and for second-order methods, an ill-conditioned Hessian definitely degrades convergence. Empirically, this behavior would be seen by taking small &quot;zig-zag&quot; steps as one approaches a local optimum; from a numerical analysis standpoint, it follows by noting that second-order methods take a form that looks roughly like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;H(x^{n})(x^{n+1} - x^{n}) = -\nabla{f}(x^{n}),&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H$ is the Hessian, $f$ is the objective function, and $\nabla{f}$ is the gradient of the objective function, yielding an ill-conditioned linear system that will be solved inaccurately with numerical methods. (Here, I've assumed a Newton method, but quasi-Newton methods operate similarly.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-10-04T00:34:50.083" LastActivityDate="2014-10-04T00:34:50.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14764" PostTypeId="2" ParentId="14737" CreationDate="2014-10-04T17:34:22.040" Score="3" Body="&lt;p&gt;Let me start by restating that this is a terrible MI estimation procedure. Please don't use this in any serious business, unless you are absolutely sure you know what you are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;You are making two critical mistakes in your code&lt;/strike&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strike&gt;Your scale doesn't match because you didn't do the integration properly. Mutual information is defined as in the paper is for &lt;strong&gt;probability&lt;/strong&gt;, &lt;em&gt;not probability density&lt;/em&gt;. But your kernel density estimator gives you back probability density which integrates to one, but does not sum to one on the points on a grid. If you want to use the sum formula as in the paper, you need to convert the density back to probability.&lt;/strike&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strike&gt;You are adding the probabilities only at the points where your data live, and counting multiple times on those regions. You should be integrating over the support of the entire distribution! An easy way to do this would be to make a uniform grid to integrate over. I think you should carefully review the definition of mutual information. (You need to add the probabilities, not the point wise MI estimates over the data.)&lt;/strike&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is not a code-review site, but a couple of small things on your code.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;never use reserved matlab function names as variables: &lt;code&gt;sum&lt;/code&gt; is one of them&lt;/li&gt;&#xA;&lt;li&gt;You don't need &lt;code&gt;inv&lt;/code&gt;, so don't use it. Use &lt;code&gt;\&lt;/code&gt; and &lt;code&gt;/&lt;/code&gt; instead&lt;/li&gt;&#xA;&lt;li&gt;try not to use &lt;code&gt;i&lt;/code&gt; as a looping variable. &lt;code&gt;i&lt;/code&gt; is for imaginary numbers in MATLAB (so is &lt;code&gt;j&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;hint: MATLAB has a KDE for 1D which you can verify your code against: &lt;code&gt;ksdensity&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: I misinterpreted the code. I was wrong. OP is replacing expectation with sum over samples, so there's no problem using density nor having a grid.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastEditorUserId="1128" LastEditDate="2014-10-06T13:37:23.730" LastActivityDate="2014-10-06T13:37:23.730" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14765" PostTypeId="1" AcceptedAnswerId="14766" CreationDate="2014-10-05T02:52:07.057" Score="1" ViewCount="4305" Body="&lt;p&gt;I might have some non-linear ODEs that are being solved by &lt;code&gt;scipy.integrate.odeint&lt;/code&gt;. However, a parameter at each time step might have to be updated by using a non-DE rule, which uses the results of the ODE solver at each timestep, together with the variable along which integration is being performed, in order to update the parameter. For example, the non-DE rule might be a coarse discretization of a &quot;background&quot; PDE system.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here's a toy example, related to what I am working on right now: We might have a particle, which has some properties ($x$, $y$ and $z$) associated with it that evolve with time -- let's say that $x$ and $y$ represent the 2D position of the particle, and $z$ its &quot;colour&quot;. The particle moves over a 2D space. The 2D space is discretized, and each discretized point has a property $p$ associated with it - let's say that $p$ evolves via the following rules: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;certain pre-selected spatial points have a $p$ source&lt;/li&gt;&#xA;&lt;li&gt;if a particle has recently visited a spatial point, $p$ is set to zero&lt;/li&gt;&#xA;&lt;li&gt;at every time step $t$, 10 random sets (of size 4) of neighbouring spatial points are selected, and their $p$ is set to be the average $p$ of the points in the set&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;When calculating the RHS ($dx/dt$, $dy/dt$ and $dz/dt$), we need to know what the average of value of $p$ is on the spatial points closest to the particle's current location. However, it's not straightforward to embed the evolution of $p$ on the spatial points as a bunch of DE rules...otherwise we could have simply passed an ODE system like $dx/dt$, $dy/dt$, $dp_1/dt$, ... , $dp_n/dt$) to &lt;code&gt;odeint&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let's say I want ODE results at &lt;code&gt;t = [t1, ..., t2]&lt;/code&gt;. Then, I could setup the program so that each time I call &lt;code&gt;odeint&lt;/code&gt;, I only make it give me results between one timestep, then use those results to update the non-DE rule, before calling &lt;code&gt;odeint&lt;/code&gt; again. However, this would be quite inefficient, since I'd be breaking &lt;code&gt;odeint&lt;/code&gt;'s internal flow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other way to get around this issue? Could it be that I can inherit from the object-oriented version of 'odeint', and then have my class execute the non-DE step in between each step? Would this be any more efficient than the naive solution suggested above? &lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="9466" LastEditDate="2014-10-05T19:02:45.890" LastActivityDate="2019-11-18T05:21:38.157" Title="scipy.integrate.odeint: how can odeint access a parameter set that is evolving independently of it?" Tags="|ode|scipy|modeling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14766" PostTypeId="2" ParentId="14765" CreationDate="2014-10-05T04:22:52.767" Score="3" Body="&lt;p&gt;EDIT: No, that process will not be compatible with any adaptive time-stepper. It's not deterministic, and even for non-adaptive ODE solvers, your proposed process will degrade the accuracy of your solution.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;You'll have to describe what you want to do in more detail.&lt;/strike&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, changing a parameter changes the equations you want to solve. If your parameter changes in a smooth (infinitely continuously differentiable, all derivatives bounded over your interval of integration) fashion, depending on this &quot;non-differential equation rule&quot;, it's probably better to figure out how to query an update as part of your right-hand side function evaluation (and Jacobian function evaluation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it does not change in a smooth fashion, it depends on how many bounded, continuous derivatives you have. In general, differential equation solvers use interpolating polynomials to solve approximately differential equations, and the error term for a method of order $k$ is usually related to the derivative of order $k+1$. If you have bounded, continuous derivatives up to order $k + 1$, then you can use a method of up to order $k$. If you have no continuous derivatives, you have to use special methods that have low orders of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What will probably happen if you try to kludge in something like you're proposing is that the integrator will restart itself repeatedly, which will be inefficient, and probably inaccurate. I don't think that subclassing the object-oriented version of &lt;code&gt;odeint&lt;/code&gt; will rectify the problem because as I've described above, the problem is a fundamental numerical analysis issue.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-10-05T22:02:31.493" LastActivityDate="2014-10-05T22:02:31.493" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14768" PostTypeId="1" AcceptedAnswerId="14769" CreationDate="2014-10-05T20:48:58.997" Score="3" ViewCount="986" Body="&lt;p&gt;I am using bvp4c from Matlab to solve a boundary values ODEs problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the ODEs and boundary conditions, is there any way to have more information on the solutions? How many do I expect? 0, 1, several?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I supposed to understand from the bvp4c error message &lt;em&gt;&quot;singular Jacobian encountered&quot;&lt;/em&gt;? No solutions at all? Initial guess doesn't fit?&lt;/p&gt;&#xA;" OwnerUserId="10366" LastEditorUserId="10366" LastEditDate="2014-10-06T09:42:30.120" LastActivityDate="2014-10-06T10:06:28.547" Title="How to know whether a boundary-value ODE problem is well defined?" Tags="|matlab|ode|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14769" PostTypeId="2" ParentId="14768" CreationDate="2014-10-05T22:33:40.197" Score="4" Body="&lt;p&gt;A singular Jacobian means that there is a linear system in there somewhere (probably within a Newton solve) that has multiple (or zero) solutions. This issue may correspond to different solutions having different amounts of material; you might be able to resolve it by adding an equation that governs the amount of material present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/bvp4c.html&quot; rel=&quot;nofollow&quot;&gt;MATLAB's &lt;code&gt;bvp4c&lt;/code&gt;&lt;/a&gt; uses a &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods#Lobatto_IIIA_methods&quot; rel=&quot;nofollow&quot;&gt;three-stage Lobatto IIIa formula&lt;/a&gt; that is fourth-order accurate. Essentially, there's some default discretization that defines the collocation points of the method, and then the Lobatto formula defines the fourth-order interpolating polynomial on that discretization. Since Lobatto IIIa methods are implicit methods, they necessarily involve solving a nonlinear equation, and this nonlinear equation solve is usually done through a variant of Newton's method, which involves solving a linear system at each iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing the details of the BVP solver algorithm at the time of my original response, the existence of the linear solve in the algorithm could be inferred because:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;many ODE discretizations used involve implicit methods, which require solving nonlinear equations, and it's common to do that through a sequence of linear systems&lt;/li&gt;&#xA;&lt;li&gt;attempting to solve a singular linear system is a common point of failure in numerical routines&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;EDIT 2: There's no general theory for existence and uniqueness of two-point boundary value problems (TPBVP). There are results for specific cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of intuition, if you have a TPBVP expressed in first-order form with $n$ equations, you need $n$ linearly independent boundary conditions. A common case involving multiple solutions occurs when you have all Neumann boundary conditions (i.e., all boundary conditions involve derivatives only). Usually, if you have at least one Dirichlet condition, it suggests there might be a solution to your problem (although it may not be unique).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I wrote above, if you get an error message &quot;singular Jacobian encountered&quot;, one common situation is that there should be an additional constraint on your system in the form of another equation for your boundary value problem; adding additional constraint equations is a common way of determining a unique solution to a problem with only derivative boundary conditions, and these constraints could be things like conservation of mass (so the integral of the solution over the domain must be a constant).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could be a problem with your initial guess, too, but without knowing more about intermediate outputs from your problem or the specific problem you are solving, it is difficult to diagnose the precise cause. Generally, if there is a problem with your initial guess, when you look at the iterates for the nonlinear solves in the algorithm, they fail to converge and the residual function values stagnate; it's not clear to me from looking at the documentation if there are MATLAB options to output this information. You could also have a singular TPBVP, and you may have to resort to special methods to solve your problem. I find other software gives you more control over the numerical methods and access to solver information, but it generally comes at the cost of having to implement more of the numerical methods yourself. You could use PETSc, for instance, which would give you access to initial-value problem ODE solvers, nonlinear solvers, and linear solvers, along with tons of solver information that would help you better diagnose these problems, but then you would have to write your own boundary value problem solver (I'd suggest a multiple shooting method).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-10-06T10:06:28.547" LastActivityDate="2014-10-06T10:06:28.547" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14770" PostTypeId="2" ParentId="4883" CreationDate="2014-10-06T06:48:26.453" Score="2" Body="&lt;p&gt;Determinant of matrix A can be computed as&#xA;$$&#xA;det(A) = \prod_{i = 1}^n \alpha_k^{-1}, &#xA;$$&#xA;where $\alpha_k = \cfrac{r_k^Tr_k}{p_k^TAp_k}$ - are steps computed during CG iteration. But this works only if $r_k \ne 0 \;\,\forall k=1, \ldots, n$.&#xA;The proof is the following. Let $R$ be a matrix consisting of vectors $r_k$ and let $P$ be a matrix consisting of $p_k$.&#xA;$$&#xA;p_k = -r_k + \sum_{i=1}^{k-1}\gamma_i r_i.&#xA;$$&#xA;So, by the property of determinant $det(P) = (-1)^n det(R)$. Now notice that vectors $r_k$ are orthgonal and vectors $p_k$ are conjugate with respect to $A$.&#xA;Therefore&#xA;$$&#xA;\prod_{k = 1}^n \alpha_k = \prod_{k = 1}^n \frac{r_k^T r_k}{p_k^TAp_k} = \frac{det(R^TR)}{det(P^TAP)} = \frac{det(R^TR)}{det(A)det(P^TP)}= (det(A))^{-1}.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="10437" LastActivityDate="2014-10-06T06:48:26.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14771" PostTypeId="1" CreationDate="2014-10-06T12:34:25.780" Score="1" ViewCount="269" Body="&lt;p&gt;Please recommend me a well documented C++ library that I could wrap with a mex-interface in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The library should have the capability to read meshes (optimal in different formats, at least in AVS-UCD-format). To these meshes some operations should be applied:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Smoothing, subdivision, reduction, translations, scaling, and most important make a double layer shell from a one layer mesh for printing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a total naive in dealing with mesh, and I'm trying to learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="10439" LastEditorUserId="10439" LastEditDate="2014-10-23T09:37:17.873" LastActivityDate="2015-07-20T13:27:21.363" Title="Windows C++ library for operations on mesh with a mex-interface in Matlab?" Tags="|matlab|c++|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14772" PostTypeId="1" CreationDate="2014-10-06T14:47:21.323" Score="5" ViewCount="776" Body="&lt;p&gt;I have to choose between the PETSc and Trilinos libraries for a project that involves the solution of non-linear systems of equations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From their web sites they both mention support for GPUs, without giving many details (e.g. &lt;a href=&quot;http://www.mcs.anl.gov/petsc/features/gpus.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://trilinos.org/?s=gpu&amp;amp;submit=Search&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know which of these libraries is better for GPGPU. In particular, I am looking for a library that, on the basis of the same calling source code, can use either the CPU or the GPU on the basis of a pre-compiler switch or, better, dynamically, on the basis of the time taken to run the computations and to move the data in and out the graphics memory.&lt;/p&gt;&#xA;" OwnerUserId="10441" LastEditorUserId="10441" LastEditDate="2014-10-06T15:05:09.347" LastActivityDate="2014-10-06T15:44:00.267" Title="PETSc or Trilinos for GPU?" Tags="|petsc|nonlinear-equations|gpu|trilinos|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14773" PostTypeId="1" AcceptedAnswerId="14774" CreationDate="2014-10-06T15:23:44.080" Score="8" ViewCount="826" Body="&lt;p&gt;I know the method of constructing solutions. For example I have a BVP:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_{xx} + u = 0 $$&#xA;subjected to:&#xA;$$ u_{x}(0) = f_1, $$&#xA;$$ u_{x}(1) = f_2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to check the correctness of my numerical implementation, just plug in $u(x) = \sin(x)$ and then $f_1 = \cos(0), f_2 = \cos(1)$ and then run the script for implementation. If the graphs of the numerical solution and that of $u(x) = \sin(x)$ approximately matches then I have the correct implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may present some problems when constructing solutions with not-so-simple differential equations. Currently I am having trouble finding one for:&#xA;$u_t = ku_{xx}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other methods &lt;strong&gt;&lt;em&gt;or&lt;/em&gt;&lt;/strong&gt; is there a simple function for this one?&lt;/p&gt;&#xA;" OwnerUserId="5353" LastEditorUserId="1523" LastEditDate="2014-10-09T08:31:59.770" LastActivityDate="2014-10-09T08:31:59.770" Title="How to check the correctness of my implementation of a numerical scheme for differential equation" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14774" PostTypeId="2" ParentId="14773" CreationDate="2014-10-06T15:34:30.077" Score="14" Body="&lt;p&gt;The &lt;a href=&quot;http://infoserve.sandia.gov/sand_doc/2000/001444.pdf&quot;&gt;Method of Manufactured Solutions (PDF link)&lt;/a&gt; goes more as less as follows. You pick a solution that excites enough modes of your operator and satisfies the boundary and initial conditions, and you then turn the crank to get a right-hand-side function that you can implement. Though, just bare sines and cosines are a poor choice since they might be eigenfunctions of your operator and lead to a solution that has false accuracy. A product of transcendental functions is usually used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you have implemented that RHS function, you should generate a range of numerical solutions on a sequence of successively finer meshes. Then you should compute the difference between your computed values to the exact value on each mesh in some norm or other metric. The choice of that norm should be based on the convergence theory that you have for your numerical method. At that point, the goal is to check to see if your method achieves the order of accuracy that the theory predicts. This is typically $O(h^{(p+k)})$ where $h$ is the mesh size, $p$ is the order of the method, and $k$ is dependent on the norm you chose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, for small enough $h$, your method behaves like the theory predicts, then you've made one measure of the accuracy/correctness of your implementation. &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2014-10-06T18:55:29.077" LastActivityDate="2014-10-06T18:55:29.077" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="14775" PostTypeId="2" ParentId="14772" CreationDate="2014-10-06T15:44:00.267" Score="3" Body="&lt;p&gt;As of writing this answer (October 2014), Trilinos does not have this capability throughout its code base. There are two packages (Kokkos and Tpetra) that will provide this functionality to Trilinos, but neither are used widely throughout the remainder of the library. The Trilinos plan (as far as I understand it) foresees new packages (and some revamped old packages) to either switch from Epetra to Tpetra for their basic linear algebra needs, or to switch to the Xpetra wrappers that abstract away the differences between Epetra and Tpetra. The majority of packages have not been converted yet.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-06T15:44:00.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14777" PostTypeId="1" AcceptedAnswerId="14781" CreationDate="2014-10-06T19:23:20.617" Score="1" ViewCount="124" Body="&lt;p&gt;So far, I have been using &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; as my &quot;workhorse&quot; ODE solver. My current problem requires that I solve a large system (up to ~5000) ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the general workflow of the &lt;code&gt;odeint&lt;/code&gt; program, that I have noticed:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;get initial values and function to calculate Jacobian (or RHS) of the ODE system at $t = t_0$&lt;/li&gt;&#xA;&lt;li&gt;make estimate of $t_n = t_{n-1} + \delta t$, using standard numerical quadrature methods (requires 1 call to Jacobian function)&lt;/li&gt;&#xA;&lt;li&gt;tackle non-linearity by &quot;converging&quot; solution at $t_1$ (might require many calls, say $N_n$, to Jacobian function&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, to proceed at each step $\delta t$, we need at least $N_n + 1$ calls to the Jacobian. If calculating the Jacobian requires some effort, then this can be computationally expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to reduce this cost that I have been thinking about is by considering some coefficients/parameters as staying constant (almost like a &quot;quasi-equilibrium&quot; assumption over some small time scale?) over $\delta t$. That way, while the ODE system is solved over $\delta t$, the repeated calls to the Jacobian function will not be as expensive, since it won't have to calculate certain things, until the next time step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My naive implementation of such a simplification used a method similar to the one described here: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14765/scipy-integrate-odeint-how-can-odeint-access-a-parameter-set-that-is-evolving-i&quot;&gt;scipy.integrate.odeint: how can odeint access a parameter set that is evolving independently of it?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, in order to solve the ODE system over a the interval $[0, 10]$ (as an example), I would:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;consider first the interval $[0, 1]$, and pre-calculate the coefficients/parameters to be used for this time step&lt;/li&gt;&#xA;&lt;li&gt;pass the pre-calculated values from step 1. as constant arguments to the Jacobian function for solving the ODE system over $[0, 1]$,&lt;/li&gt;&#xA;&lt;li&gt;solve the ODE system over $[0, 1]$ but discard all results except for the data obtained for $t = 1$&lt;/li&gt;&#xA;&lt;li&gt;use the data obtained for $t = 1$ to pre-calculate the coefficients/parameters to be used over the time-step $[1, 2]$&lt;/li&gt;&#xA;&lt;li&gt;similar to steps 2-3, call &lt;code&gt;odeint&lt;/code&gt; again &lt;/li&gt;&#xA;&lt;li&gt;rinse and repeat&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;When my supervisor found out what I was doing, similar to the answer for the question I linked above, he warned that this implementation might be both inefficient (since I am breaking the internal flow of &lt;code&gt;odeint&lt;/code&gt;) and inaccurate (since I am sort of artificially introducing discontinuities constantly). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I go about implementing the &quot;quasi-equilibrium&quot; simplification correctly, if it is possible?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-10-06T20:28:17.420" Title="Possible to reduce effort needed to solve non-linear ODEs by taking some coefficients/parameters as constant over small time intervals?" Tags="|ode|nonlinear-equations|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14778" PostTypeId="1" AcceptedAnswerId="14783" CreationDate="2014-10-06T19:31:33.220" Score="0" ViewCount="54" Body="&lt;p&gt;Currently, my problem requires me to solve a system a large system of non-linear ODEs (up to ~5000). So far, I have been using &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; as my workhorse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simpler subset of my problem only involves ~50 non-linear ODEs. Solving the system over an interval $[0, 10]$ takes about 3 times longer than solving the system over $[0, 1]$. Not bad! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, solving the system over $[0, 1000]$ takes about 1500 times longer than solving over $[0, 1]$. Not so nice!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An obvious reason why this might be the case is because at some points between $[0, 1000]$, the solution takes far longer to converge than usual -- so, the average time taken to solve the system at each time step is greater over $[0, 1000]$ then over $[0, 1]$. Is my intuition correct over here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other things might be issues?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2014-10-06T20:40:43.300" Title="Why might the time taken to compute the solution of an ODE system over some interval increase non-linearly with increasing size of interval?" Tags="|ode|nonlinear-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14779" PostTypeId="1" CreationDate="2014-10-06T19:31:40.553" Score="1" ViewCount="264" Body="&lt;p&gt;Suppose $A=uv^T$ where $u$ and $v$ are non-zero column vectors in $\mathbb{R}^n, n\geq 3$. $\lambda=0$ is an eigenvalue of $A$ since $A$ is not of full rank. $\lambda=v^Tu$ is also an eigenvalue of $A$ since $Au=(uv^T)u=u(v^Tu)=(v^Tu)u$. Here is my questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Are there any other eigenvalues of $A$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) If I have another matrix $B_n$ such that $B_n = F^{*}_n\Lambda_nF_n$, and $\Lambda_n$ are diagonal matrices holding the eigenvalues of $B_n$, $F_n$ is the unitary matrix (such as the discrete Fourier-matrix). Then I want to ask:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can sum matrix $C = B_n + A$ be diagonalized by the specified unitary matrix (In fact, $B_n$ can be the $n\times n$ circulant matrix)?&lt;/p&gt;&#xA;" OwnerUserId="10290" LastEditorUserId="4405" LastEditDate="2014-10-07T07:08:45.540" LastActivityDate="2014-12-06T08:22:51.523" Title="Diagonalize a circulant-plus-rank-one matrix" Tags="|linear-algebra|eigenvalues|fourier-transform|rank|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14780" PostTypeId="1" AcceptedAnswerId="14782" CreationDate="2014-10-06T19:46:06.137" Score="4" ViewCount="1580" Body="&lt;p&gt;I would prefer recommendations that don't require the use of proprietary tools (such as Matlab). I know of two ODE solving options for the Python ecosystem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ni.gsu.edu/~rclewley/PyDSTool/FrontPage.html&quot; rel=&quot;nofollow&quot;&gt;PyDSTool&lt;/a&gt; (Dopri, Radau, other Runge-Kutta methods, and whatever &lt;code&gt;scipy.integrate.ode&lt;/code&gt; has access to)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.docs.scipy.org/doc/scipy/reference/integrate.html&quot; rel=&quot;nofollow&quot;&gt;scipy.integrate&lt;/a&gt; (&lt;code&gt;scipy.integrate.odeint&lt;/code&gt; uses &lt;code&gt;lsoda&lt;/code&gt; from the Fortran library &lt;code&gt;odepack&lt;/code&gt; -- solver decides whether to use a Adams method or a BDF method depending on the stiffness of the problem; dopri5, and dopri853 are also available, along with some solvers for complex ODE systems)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I like Python because I can now write C-speed code, combined with Python flexibility using Python-to-C compilers provided by packages such as &lt;a href=&quot;http://cython.org/&quot; rel=&quot;nofollow&quot;&gt;Cython&lt;/a&gt;. Plus, everything is pretty open-source!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my early conversations with the developer of PyDSTool, I know he brought up that Radau might be particularly good for non-linear, stiff ODE problems -- and certainly the pure C implementation included with PyDSTool would be much faster than &lt;code&gt;scipy.integrate&lt;/code&gt;'s standard ODE solvers. I wasn't really able to understand him well at the time simply due to my lack of mathematical background (I am a math undergrad, recently transferred from engineering). Could you comment on concerns like that -- in particular, &lt;strong&gt;what features of my problem do I need to identify in order to figure out which solver is best suited for my system of non-linear ODEs?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2014-10-06T20:34:31.793" Title="Solving large, non-linear systems of ODEs numerically: what do I need to consider in order to figure out which solver to use?" Tags="|ode|nonlinear-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14781" PostTypeId="2" ParentId="14777" CreationDate="2014-10-06T20:28:17.420" Score="3" Body="&lt;p&gt;I recommend that you check out Rosenbrock or Rosenbrock-W methods.  These are multi-stage (like Runge-Kutta) methods that compute the Jacobian only once per step.  The methods have more order conditions than diagonally-implicit Runge-Kutta methods (due to the lagged and/or inexact Jacobian), but are otherwise similar.  There is a chapter in Hairer and Wanner's second volume that explains the order conditions and discusses some classes of methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can access the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSROSW.html&quot; rel=&quot;nofollow&quot;&gt;PETSc implementation&lt;/a&gt;, which implements many Rosenbrock and Rosenbrock-W methods, via petsc4py.  See the &lt;a href=&quot;https://bitbucket.org/petsc/petsc4py/src/master/demo/ode/orego.py&quot; rel=&quot;nofollow&quot;&gt;Oregonator example&lt;/a&gt; for usage.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-10-06T20:28:17.420" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14782" PostTypeId="2" ParentId="14780" CreationDate="2014-10-06T20:34:31.793" Score="3" Body="&lt;p&gt;Consider how stiff the problem is, how strong the nonlinearity is, any model non-smoothness, and the desired accuracy.  The standard way to compare methods is with &quot;work-precision&quot; diagrams (a log-log plot of CPU time versus error with a line for each method).  Hairer and Wanner's books includes comprehensive comparisons of methods across a range of problems.  You can gain some insight by looking at which methods work well for which problems in their book (volume 2 for stiff problems) or find papers that perform such comparisons for a problem similar to yours.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-10-06T20:34:31.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14783" PostTypeId="2" ParentId="14778" CreationDate="2014-10-06T20:40:43.300" Score="4" Body="&lt;p&gt;Adaptive time integrators adjust the step length in an attempt to maximize efficiency while controlling truncation error (usually locally) to a prescribed tolerance.  If the solution undergoes rapid changes, the local truncation error estimates will force the time step to decrease.  Implicit integrators have the additional complication that algebraic systems (especially nonlinear systems) may require vastly different amounts of work, and that amount of work may depend on the step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Production ODE integrators should include diagnostics so that you can see where the time is spent (e.g., very short time steps or many nonlinear iterations).  You can then decide whether to change the method parameters, choose a different method, or live with the observed performance.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-10-06T20:40:43.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14784" PostTypeId="2" ParentId="14779" CreationDate="2014-10-06T20:48:59.993" Score="2" Body="&lt;p&gt;On your first question: Of course, $A$ has other eigenvalues. It has $n$ eigenvalues, to be specific, counting multiplicity. You  have identified one. For a rank-one matrix, the other eigenvalues are all zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On your second question: I don't know whether it is possible in general, but it is possible if $u$ is also an eigenvector of $B$ since then the eigenvectors of $C$ are the same as those of $B$ and only the eigenvalues change. If $u$ lies in a subspace spanned by only a few of the columns of the matrix $F$ (i.e., of only a few of the eigenvectors), then the diagonalization procedure will be fairly simple since you only need to re-diagonalize those eigenvectors in this subspace, not all of them.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-06T20:48:59.993" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14785" PostTypeId="1" AcceptedAnswerId="14807" CreationDate="2014-10-07T00:59:31.860" Score="3" ViewCount="1999" Body="&lt;p&gt;I am aware of a few:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) &lt;a href=&quot;http://journal.r-project.org/archive/2010-2/RJournal_2010-2_Soetaert~et~al.pdf&quot; rel=&quot;nofollow&quot;&gt;R project&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) &lt;a href=&quot;https://software.intel.com/en-us/articles/intel-ordinary-differential-equations-solver-library&quot; rel=&quot;nofollow&quot;&gt;Intel® Ordinary Differential Equations Solver Library&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, I'd like to know which are the best tools that have been developed by professional physicists and mathematicians. The reason I ask is that I'd like to build an ODE solver myself from scratch but I'd like to be aware of the state of the art before getting started.&lt;/p&gt;&#xA;" OwnerUserId="10445" LastEditorUserId="10445" LastEditDate="2014-10-07T01:06:44.077" LastActivityDate="2015-02-03T15:25:18.850" Title="Soft question: What are good libraries for solving ordinary differential equations?" Tags="|software|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14787" PostTypeId="1" CreationDate="2014-10-07T07:56:08.643" Score="3" ViewCount="2873" Body="&lt;p&gt;I'm looking for a library or individual package which provides convenient optimization subroutine for Fortran, any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm testing the codes from Numerical Recipe, downhill simplex method and direction-set method, but the input-output interface is not convenient for chi-squared function, because we generally need to find the minimum of chi-squared function within a cube-like parameter space. Thus, I need a more convenient utility especially for least chi-squared problem.&lt;/p&gt;&#xA;" OwnerUserId="10442" LastActivityDate="2014-10-07T07:56:08.643" Title="Fortran library for minimization or maximization of functions (optimization problem)" Tags="|fortran|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14789" PostTypeId="2" ParentId="14771" CreationDate="2014-10-07T09:03:58.450" Score="1" Body="&lt;p&gt;I don't know if it satisfies all requirements, but TriMesh is pretty easy to use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://gfx.cs.princeton.edu/proj/trimesh2/&quot; rel=&quot;nofollow&quot;&gt;http://gfx.cs.princeton.edu/proj/trimesh2/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;JMeshlib was also a common choice, but an old one:&#xA;&lt;a href=&quot;http://jmeshlib.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;http://jmeshlib.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Instead of pointer based arithmetic, MATLAB in general, uses a [Faces, Vertices] pair data structure to represent meshes. So it might create compatibility issues with efficient mesh libraries. But there are also some MATLAB implementations for mesh tools:&#xA;&lt;a href=&quot;http://www.dgp.toronto.edu/~rms/software/matlabmesh/&quot; rel=&quot;nofollow&quot;&gt;http://www.dgp.toronto.edu/~rms/software/matlabmesh/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2014-10-07T09:35:57.770" LastActivityDate="2014-10-07T09:35:57.770" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14792" PostTypeId="1" CreationDate="2014-10-07T13:40:23.710" Score="1" ViewCount="84" Body="&lt;p&gt;For a linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;M \dot{u} = Au \qquad \textrm{or} \qquad \dot{u} = L u&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;generalized&lt;/strong&gt; eigenvalue problem is&#xA;$$&#xA;A e = \lambda M e&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can use the &lt;em&gt;time-stepper&lt;/em&gt; approach which essentially computes the eigenvalues of the operator&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E = \exp(L T)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\mu$ is an eigenvalue of $E$ then it is related to the original eigenvalue by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mu = \exp(\lambda T)&#xA;$$&#xA;and $T$ has to be chosen appropriately. If I have a code $E_{\Delta} \approx E$ which solves the initial value problem, then I can couple this to ARPACK to compute the eigenvalues. I would use the &lt;strong&gt;standard&lt;/strong&gt; eigenvalue solver of ARPACK on the operator $E_\Delta$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would also like to use the inner product induced by $M$ in the Arnoldi process so that the eigenvectors have unit norm in the inner product of $M$. How can I achieve this with ARPACK ?&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2014-10-07T13:40:23.710" Title="Time-stepper approach to eigenvalue problem" Tags="|eigenvalues|arpack|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14793" PostTypeId="2" ParentId="14745" CreationDate="2014-10-07T14:50:13.527" Score="0" Body="&lt;p&gt;I'm not sure it really helps, but you may have a look to temperature distribution on both mesh and compute a similarity measure between the two distributions. This is done in the following paper:&#xA;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.385.5918&quot; rel=&quot;nofollow&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.385.5918&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers !&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2014-10-07T14:50:13.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14794" PostTypeId="2" ParentId="14704" CreationDate="2014-10-07T15:10:50.157" Score="2" Body="&lt;p&gt;I'm most familiar with entropy in the analysis of non-linear hyperbolic conservation laws. From what I understand the entropy/entropy flux pair $(\Phi,\Psi)$ provided a tool to examine properties of the PDE like stability and irreversibility in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivation of the entropy pair $(\Phi,\Psi)$ for a given conservation law is quite involved and from what I've seen the only detailed outline of the procedure (involving the compressible Euler equations) is in Chap. 5 of Lecture notes for &lt;a href=&quot;http://math.berkeley.edu/~evans/entropy.and.PDE.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;Entropy and PDEs&quot; by Evans&lt;/a&gt;. Evans gives a lot of background information and a mathematically thorough treatment of the topic. Once you're used to his notation and style, the computation of the entropy/entropy flux pair begins on page 138.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastActivityDate="2014-10-07T15:10:50.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14795" PostTypeId="1" AcceptedAnswerId="14796" CreationDate="2014-10-07T15:46:01.013" Score="15" ViewCount="6128" Body="&lt;p&gt;I am having trouble implementing a function numerically. It suffers from the fact that at large input values the result is a very large number times a very small number. I am not sure if catastrophic cancellation is the correct term so please correct me if it is. Evidence of something going awry: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/vswGb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I avoid the oscillations and the assigment of 0.0 for larger inputs of 6?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my function: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;def func(x):&#xA;    t = np.exp(-np.pi*x)&#xA;    return 1/t*(1-np.sqrt(1-t**2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastActivityDate="2014-10-07T16:32:48.673" Title="How to avoid catastrophic cancellation in python function?" Tags="|python|numerical-analysis|floating-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14796" PostTypeId="2" ParentId="14795" CreationDate="2014-10-07T16:32:48.673" Score="32" Body="&lt;p&gt;This is indeed called catastrophic cancellation. In fact, this particular case is very easy: rewrite the function using the equivalent, numerically stable expression $$ \frac{t}{1+\sqrt{1-t^2}}. $$&#xA;Since you probably need a reference, this is discussed in most numerical methods textbooks in relation to the formula for solving quadratic equations (that formula in its standard form is numerically unstable for some parameter values). The way to get rid of $1-\sqrt{1-t^2}$ is to multiply and divide by $1+\sqrt{1-t^2}$. Here is a comparison:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EiGa6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-10-07T16:32:48.673" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14798" PostTypeId="2" ParentId="6758" CreationDate="2014-10-07T17:12:56.250" Score="-2" Body="&lt;p&gt;you can use &quot;return&quot; and it will &quot;return control to the invoking function before it reaches the end of the function&quot;, which for the main program means exiting the program.&lt;/p&gt;&#xA;" OwnerUserId="10452" LastActivityDate="2014-10-07T17:12:56.250" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14799" PostTypeId="1" CreationDate="2014-10-07T18:31:08.790" Score="1" ViewCount="90" Body="&lt;p&gt;I have been playing with an implementation of Visscher's explicit method for solving the time dependent Schrodinger equation (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/10876/are-there-simple-ways-to-numerically-solve-the-time-dependent-sch%C3%B6dinger-equatio/10880#10880&quot;&gt;Are there simple ways to numerically solve the time-dependent Sch&amp;#246;dinger equation?&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presumably if I want to simulate two identical, non-interacting particles I need to duplicate the simulation array for each particle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I then modify the computation to take into account Fermion/Boson effects, i.e. that Fermions can't occupy the same space (ignoring spin), while Bosons are &quot;encouraged&quot; to do so?&lt;/p&gt;&#xA;" OwnerUserId="10453" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-10-07T19:24:31.533" Title="Modifying finite difference solution to Schrodinger eqn to account for fermion/boson effects" Tags="|finite-element|computational-physics|quantum-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14801" PostTypeId="2" ParentId="14799" CreationDate="2014-10-07T19:24:31.533" Score="3" Body="&lt;p&gt;It is not that bosons are &quot;encouraged&quot; to occupy the same place, that is quite wrong. Here is an article on &lt;a href=&quot;http://en.wikipedia.org/wiki/Spin%E2%80%93statistics_theorem&quot; rel=&quot;nofollow&quot;&gt;the spin-statistics theorem&lt;/a&gt; which describes the relationship between particle statistics and the transformation of the wave function under permutations of identical particles. This kind of thing requires more knowledge of quantum mechanics, so I recommend reading an introductory QM textbook.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, if you have two identical particles, and your wave function is $\psi(x_1,x_2)$, then&#xA;$$ \psi(x_1,x_2) = \pm \psi(x_2,x_1), $$&#xA;and this is the relation you should impose on the discretized wave function. You will get a vector of values $\psi_{jk}$ (where $x_j$ are the points in discretized space), and so when you write down the numerical method applied to $\psi_{jk}$, you also require that $\psi_{jk}=\pm \psi_{jk}$, and adjust the numerical method to take account of this (anti-)symmetry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, bear in mind that for non-interacting particles, the PDE separates, and the solution &lt;a href=&quot;http://en.wikipedia.org/wiki/Identical_particles&quot; rel=&quot;nofollow&quot;&gt;will be of the form&lt;/a&gt; $$\frac12\big(\psi_1(x_1)\psi_2(x_2)\pm \psi_1(x_2)\psi_2(x_1)\big),$$&#xA;where each $\psi_{i}$ solves the one-particle Schrodinger equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are asking is more a question of physics than scientific computing.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-10-07T19:24:31.533" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14802" PostTypeId="1" CreationDate="2014-10-08T03:00:19.893" Score="5" ViewCount="208" Body="&lt;p&gt;This question is a search for further answers from a &lt;a href=&quot;https://math.stackexchange.com/questions/925622/when-is-2d-sparse-numerical-integration-by-voronoi-regions-better-than-using-tri&quot;&gt;question&lt;/a&gt; on maths.stackexchange.com.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've inherited some numerical quadrature code that is designed to integrate sparse 2D data. The quadrature is not being done in any of what I would consider the &quot;normal&quot; ways, but seems to be using an &lt;em&gt;ad-hoc&lt;/em&gt; method based on Voronoi cell areas. However the data has some unusual properties that may make this sensible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The samples fall roughly on a grid such that there is either one or zero sample points inside each grid cell. (But randomly positioned inside the cell, not at the cell center).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integration then proceeds by adding &quot;ghost&quot; points to the exterior of the region (to bound the Voronoi regions at the non-ghost points), generating a Delaunay triangulation of the domain and then assigning a weight for each sample point equal to the area of the corresponding Voronoi region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can see that this scheme tries to minimises the error for C1 functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However having gone to the trouble of triangulating the domain, I would have expected the integration to be based upon the neighbouring linear elements, which should give exact answers for linear functions, and thus I'd expect an error estimate (at least for C2 functions) of one order of $h$ higher.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However some numerical experiments I performed suggested that for smooth data both methods had similar convergence rates. (Details can be found in my &lt;a href=&quot;https://math.stackexchange.com/a/934474/11560&quot;&gt;answer&lt;/a&gt; to the original math.SE question)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was noted in &lt;a href=&quot;https://math.stackexchange.com/a/934274/11560&quot;&gt;one of the answer&lt;/a&gt; to a original question that one possible reason for preferring the Voronoi based scheme is continuity of the weights with changes in the sample locations, (which fails for the Delaunay triangular weights as the mesh can change discontinuously with sample locations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there known advantages to using the Voronoi method over the Delaunay style method, or vice-versa? Can these be quantified?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why am I seeing roughly the same convergence rates from the two methods?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10125" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-10-10T03:26:09.783" Title="Comparison between Voronoi and Delaunay 2D quadrature methods" Tags="|quadrature|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14803" PostTypeId="1" CreationDate="2014-10-08T04:40:21.557" Score="1" ViewCount="188" Body="&lt;p&gt;I need to fit a model using MLE with Frank copula by linking two discrete univate distribution function $u = F(x)$ and $v = F(y)$ together, and the joint distribution function is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Phi(x,y) = C(F(x),G(y))&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C(u,v) = \frac{1}{\theta} \log\!\left( 1+\frac{(\exp(-\theta u)-1)(\exp(-\theta v)-1)}{\exp(-\theta)-1} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and end up with the copula density as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;c(F(x),G(y)) = &amp;amp;~C(F(x),G(y)) - C(F(x-1),G(y)) - C(F(x),G(y-1)) \\&#xA;&amp;amp; + C(F(x-1),G(y-1))&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I have to evaluate $\log c(u,v)$ and ultimately I have to evaluate $C$ and its first derivatives $C_u$, $C_v$ and $C_{\theta}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I couldn't get it work with my optimization algorithm (I am using &lt;code&gt;fmin_bfgs&lt;/code&gt; and &lt;code&gt;fmin_l_bfgs_b&lt;/code&gt; from &lt;code&gt;scipy&lt;/code&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the time, &lt;code&gt;fmin_bfgs&lt;/code&gt; try to evaluate some very big point and make my code overflow (get &lt;code&gt;-inf&lt;/code&gt;) and end with incorrect result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use &lt;code&gt;fmin_l_bfgs_b&lt;/code&gt; so that I can put bounds on the variable to prevent overflow, but I end up to only make it works (not overflow) in a too small interval, $[-2,2]$ or $[-1,1]$ in some cases. The resulting point will be on the bound and it must not be the correct solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess the problem is that $\log c$, $C_u$, $C_v$ and $C_{\theta}$ involve too much &lt;code&gt;exp&lt;/code&gt; and &lt;code&gt;log&lt;/code&gt; and I lost too much precision similar to the case like $\exp(x)-1$ and $\log(1+x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any better method to compute $C$, $C_u$, $C_v$ and $C_{\theta}$ accurately? (or compute $\log c$ and its derivatives directly?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am computing them using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def frank_copula(u,v,t):&#xA;    if t == 0.:&#xA;        return u*v&#xA;    else:&#xA;        x1 = expm1(-u*t)&#xA;        y1 = expm1(-v*t)&#xA;        z1 = expm1(-t)&#xA;        return log1p(x1*y1/z1)/(-t)&#xA;&#xA;def frank_copula_u(u,v,t):&#xA;    if t == 0.:&#xA;        return v&#xA;    else:&#xA;        x = exp(t*u)&#xA;        y = exp(t*v)&#xA;        y1 = expm1(t*v)&#xA;        z = exp(t)&#xA;        return z*y1/(z*(x+y1) - x*y)&#xA;&#xA;def frank_copula_v(u,v,t):&#xA;    if t == 0.:&#xA;        return u&#xA;    else:&#xA;        x = exp(t*u)&#xA;        x1 = expm1(t*u)&#xA;        y = exp(t*v)&#xA;        z = exp(t)&#xA;        return z*x1/(z*(x1+y) - x*y)&#xA;&#xA;def frank_copula_t(u,v,t):&#xA;    if t == 0.: # central difference here&#xA;        h = 1e-8&#xA;        return (frank_copula(u,v,t+h) - frank_copula(u,v,t-h))/(2.*h)&#xA;    else:&#xA;        x = exp(-t*u)&#xA;        y = exp(-t*v)&#xA;        z = exp(-t)&#xA;        x1 = expm1(-t*u)&#xA;        y1 = expm1(-t*v)&#xA;        z1 = expm1(-t)&#xA;        tmp1 = -z1*(x1*y*v + y1*x*u) + x1*y1*z&#xA;        tmp2 = z1*z1 + x1*y1*z1&#xA;        tmp3 = log1p(x1*y1/z1)/(-t)&#xA;        return (tmp1/tmp2 + tmp3)/(-t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fixed the dependency parameter $\theta$ first, and then find out the parameter of $F(x)$ and $G(y)$ together by maximizing $\log L(p) = \sum \log c_{\theta}(F(x|p),G(y|p))$. They involve the parameter $p$ of dimension 2 to 5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the output of how &lt;code&gt;fmin_l_bfgs_b&lt;/code&gt; algorithm evaluate my (minus) log-likelihood function using my workaround, set the bounds to $[-0.2,1.2]$ and $[-1,1]$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;L: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; 769.446206148&#xA;dL: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; array([-338.93276474, -114.42079214, -216.55003298, -137.12475499,&#xA;        -52.57668841])&#xA;L: array([ 1.2,  1.2,  1.2,  1. ,  1. ]) =&amp;gt; 8019.20848421&#xA;dL: array([ 1.2,  1.2,  1.2,  1. ,  1. ]) =&amp;gt; array([ 10020.77504911,   6095.3498353 ,   8824.13266013,   6763.34035373,&#xA;         2021.60815303])&#xA;L: array([ 0.31272257,  0.31272257,  0.31272257,  0.26060214,  0.26060214]) =&amp;gt; 638.623452278&#xA;dL: array([ 0.31272257,  0.31272257,  0.31272257,  0.26060214,  0.26060214]) =&amp;gt; array([-77.88688381,  54.50079579,  11.01938969,  98.73509615,  78.99317652])&#xA;L: array([ 0.41609651,  0.21822845,  0.28865935,  0.12751238,  0.13439021]) =&amp;gt; 612.115679161&#xA;dL: array([ 0.41609651,  0.21822845,  0.28865935,  0.12751238,  0.13439021]) =&amp;gt; array([-92.53682091,  12.87793807, -22.45693087,  26.74828638,  25.65511475])&#xA;L: array([  7.05680743e-01,   1.28761254e-01,   3.21734086e-01,&#xA;        -3.25757720e-02,   4.93345936e-04]) =&amp;gt; 590.768526126&#xA;dL: array([  7.05680743e-01,   1.28761254e-01,   3.21734086e-01,&#xA;        -3.25757720e-02,   4.93345936e-04]) =&amp;gt; array([-15.61044704,  16.3196277 ,  16.76613757,  10.17553022,   1.73461386])&#xA;L: array([ 0.81933183,  0.05230577,  0.26232909, -0.07694669, -0.01632839]) =&amp;gt; 587.63001378&#xA;dL: array([ 0.81933183,  0.05230577,  0.26232909, -0.07694669, -0.01632839]) =&amp;gt; array([-10.36049322,   5.82433346,  10.12673749,  -7.60549293,  -3.70091331])&#xA;L: array([ 0.93612938, -0.03119036,  0.15964292, -0.05604783, -0.02092306]) =&amp;gt; 585.655142497&#xA;dL: array([ 0.93612938, -0.03119036,  0.15964292, -0.05604783, -0.02092306]) =&amp;gt; array([-1.60415082,  1.31457884,  5.95207277, -9.31825269, -5.19784526])&#xA;L: array([ 0.99490642, -0.09547364,  0.05431211,  0.01089551,  0.01278564]) =&amp;gt; 584.829795091&#xA;dL: array([ 0.99490642, -0.09547364,  0.05431211,  0.01089551,  0.01278564]) =&amp;gt; array([-0.44946283, -0.58748774, -1.01623299, -2.83092327,  1.18001099])&#xA;L: array([ 0.99380363, -0.10055615,  0.0527463 ,  0.03080186, -0.00557434]) =&amp;gt; 584.840769319&#xA;dL: array([ 0.99380363, -0.10055615,  0.0527463 ,  0.03080186, -0.00557434]) =&amp;gt; array([ 0.29741289,  0.67052685, -0.11044264,  1.88188906, -3.31504582])&#xA;L: array([ 0.99442739, -0.09768137,  0.05363196,  0.01954239,  0.00481047]) =&amp;gt; 584.813957801&#xA;dL: array([ 0.99442739, -0.09768137,  0.05363196,  0.01954239,  0.00481047]) =&amp;gt; array([-0.13879393, -0.04846326, -0.63399355, -0.79472897, -0.7858597 ])&#xA;L: array([ 0.99245896, -0.09809096,  0.05491007,  0.02184427,  0.00701989]) =&amp;gt; 584.811245584&#xA;dL: array([ 0.99245896, -0.09809096,  0.05491007,  0.02184427,  0.00701989]) =&amp;gt; array([ 0.16175504,  0.3075338 , -0.23277139, -0.09032161, -0.17666573])&#xA;L: array([ 0.99043781, -0.09901399,  0.056742  ,  0.02308671,  0.00811308]) =&amp;gt; 584.810330203&#xA;dL: array([ 0.99043781, -0.09901399,  0.056742  ,  0.02308671,  0.00811308]) =&amp;gt; array([ 0.08694667,  0.38532343, -0.11389679,  0.14966267,  0.06054983])&#xA;L: array([ 0.98934696, -0.10209532,  0.05898625,  0.02414812,  0.00863789]) =&amp;gt; 584.809284488&#xA;dL: array([ 0.98934696, -0.10209532,  0.05898625,  0.02414812,  0.00863789]) =&amp;gt; array([ 0.04661414,  0.30695628, -0.01908606,  0.20837613,  0.14123964])&#xA;L: array([ 0.9893386 , -0.10784629,  0.06164861,  0.02504006,  0.00849419]) =&amp;gt; 584.808428091&#xA;dL: array([ 0.9893386 , -0.10784629,  0.06164861,  0.02504006,  0.00849419]) =&amp;gt; array([ 0.00282033,  0.01810351,  0.00812252,  0.0266529 ,  0.02348458])&#xA;L: array([ 0.98941539, -0.10801042,  0.06168044,  0.02499464,  0.00841819]) =&amp;gt; 584.80842517&#xA;dL: array([ 0.98941539, -0.10801042,  0.06168044,  0.02499464,  0.00841819]) =&amp;gt; array([-0.00429622, -0.00085152, -0.00194202,  0.00036292,  0.0010925 ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and here is the output if I use my original bounds $[-5,5]$ (this bound works if I treat $F(x)$ and $G(y)$ as independent):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(Remarks: If I treat them as independent, I just need to evaluate the density function $f$,$g$ and they are much more stable and therefore I guess my problem is due to inaccuracy)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;L: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; 769.446206148&#xA;dL: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; array([-338.93276474, -114.42079214, -216.55003298, -137.12475499,&#xA;        -52.57668841])&#xA;L: array([ 5.,  5.,  5.,  5.,  5.]) =&amp;gt; inf&#xA;dL: array([ 5.,  5.,  5.,  5.,  5.]) =&amp;gt; array([ nan,  nan,  nan,  nan,  nan])&#xA;L: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; 769.446206148&#xA;dL: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; array([-338.93276474, -114.42079214, -216.55003298, -137.12475499,&#xA;        -52.57668841])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;and sorry I just found that &lt;code&gt;fmin_bfgs&lt;/code&gt; &lt;strong&gt;works&lt;/strong&gt; and will converge to the same point got by using my workaround method. But it will evaluate some big point and get &lt;code&gt;-inf&lt;/code&gt; and &lt;code&gt;nan&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dL: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; array([-338.93276474, -114.42079214, -216.55003298, -137.12475499,&#xA;        -52.57668841])&#xA;L: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; 769.446206148&#xA;L: array([ 338.93276474,  114.42079214,  216.55003298,  137.12475499,&#xA;         52.57668841]) =&amp;gt; inf&#xA;dL: array([ 338.93276474,  114.42079214,  216.55003298,  137.12475499,&#xA;         52.57668841]) =&amp;gt; array([ nan,  nan,  nan,  nan,  nan])&#xA;L: array([  3.38932765e-06,   1.14420792e-06,   2.16550033e-06,&#xA;         1.37124755e-06,   5.25766884e-07]) =&amp;gt; 769.444241862&#xA;dL: array([  3.38932765e-06,   1.14420792e-06,   2.16550033e-06,&#xA;         1.37124755e-06,   5.25766884e-07]) =&amp;gt; array([-338.93116156, -114.42016384, -216.54896297, -137.12398571,&#xA;        -52.57656892])&#xA;L: array([ 0.00238973,  0.00080675,  0.00152684,  0.00096683,  0.00037071]) =&amp;gt; 768.063616272&#xA;dL: array([ 0.00238973,  0.00080675,  0.00152684,  0.00096683,  0.00037071]) =&amp;gt; array([-337.80294205, -113.97709305, -215.79496551, -136.58082878,&#xA;        -52.49109471])&#xA;L: array([ 169.46757724,   57.21079945,  108.27577991,   68.56286091,&#xA;         26.28852956]) =&amp;gt; inf&#xA;dL: array([ 169.46757724,   57.21079945,  108.27577991,   68.56286091,&#xA;         26.28852956]) =&amp;gt; array([ nan,  nan,  nan,  nan,  nan])&#xA;L: array([ 0.00238973,  0.00080675,  0.00152684,  0.00096683,  0.00037071]) =&amp;gt; 768.063616272&#xA;dL: array([ 0.00238973,  0.00080675,  0.00152684,  0.00096683,  0.00037071]) =&amp;gt; array([-337.80294205, -113.97709305, -215.79496551, -136.58082878,&#xA;        -52.49109471])&#xA;L: array([ 338.93276474,  114.42079214,  216.55003298,  137.12475499,&#xA;         52.57668841]) =&amp;gt; inf&#xA;L: array([ 169.46638237,   57.21039607,  108.27501649,   68.56237749,&#xA;         26.28834421]) =&amp;gt; inf&#xA;L: array([ 84.73319119,  28.60519803,  54.13750824,  34.28118875,  13.1441721 ]) =&amp;gt; inf&#xA;L: array([ 42.36659559,  14.30259902,  27.06875412,  17.14059437,   6.57208605]) =&amp;gt; inf&#xA;L: array([ 21.1832978 ,   7.15129951,  13.53437706,   8.57029719,   3.28604303]) =&amp;gt; inf&#xA;L: array([ 10.5916489 ,   3.57564975,   6.76718853,   4.28514859,   1.64302151]) =&amp;gt; inf&#xA;L: array([ 5.29582445,  1.78782488,  3.38359427,  2.1425743 ,  0.82151076]) =&amp;gt; inf&#xA;L: array([ 2.64791222,  0.89391244,  1.69179713,  1.07128715,  0.41075538]) =&amp;gt; inf&#xA;L: array([ 1.32395611,  0.44695622,  0.84589857,  0.53564357,  0.20537769]) =&amp;gt; 2010.68946691&#xA;L: array([ 0.25288777,  0.08537274,  0.16157439,  0.10231284,  0.03922902]) =&amp;gt; 651.514679659&#xA;dL: array([ 0.25288777,  0.08537274,  0.16157439,  0.10231284,  0.03922902]) =&amp;gt; array([-214.23525183,  -56.37608777, -123.86215026,  -59.8381013 ,&#xA;        -29.35530006])&#xA;L: array([ 2.45189744, -1.10216675, -0.00860584, -2.25562141, -0.0889141 ]) =&amp;gt; 1140.89767258&#xA;dL: array([ 2.45189744, -1.10216675, -0.00860584, -2.25562141, -0.0889141 ]) =&amp;gt; array([ 200.97116275, -107.57429828,   77.9175913 , -284.63382774,&#xA;        222.78613298])&#xA;L: array([ 0.60625088, -0.10545527,  0.1342278 , -0.27658817,  0.01863746]) =&amp;gt; 641.856514926&#xA;dL: array([ 0.60625088, -0.10545527,  0.1342278 , -0.27658817,  0.01863746]) =&amp;gt; array([-179.74804571,  -91.13312507, -130.64600443, -147.53451709,&#xA;        -47.29257281])&#xA;L: array([ 0.94280274, -0.44136287, -0.34555214, -0.04312928,  0.3963997 ]) =&amp;gt; 650.978041317&#xA;dL: array([ 0.94280274, -0.44136287, -0.34555214, -0.04312928,  0.3963997 ]) =&amp;gt; array([-147.33106574,  -91.10716372, -144.5277363 , -121.87325508,&#xA;         59.49497911])&#xA;L: array([ 0.7272913 , -0.22626399, -0.03832441, -0.19262499,  0.15449914]) =&amp;gt; 638.059222532&#xA;dL: array([ 0.7272913 , -0.22626399, -0.03832441, -0.19262499,  0.15449914]) =&amp;gt; array([-170.21539216,  -91.09157378, -134.59864658, -140.28045749,&#xA;        -16.11667578])&#xA;...&#xA;L: array([ 0.98943719, -0.10800294,  0.06167076,  0.02498195,  0.00840848]) =&amp;gt; 584.808425121&#xA;dL: array([ 0.98943719, -0.10800294,  0.06167076,  0.02498195,  0.00840848]) =&amp;gt; array([ -6.05248630e-08,   2.30028962e-08,   3.00105580e-08,&#xA;        -1.12310441e-07,   1.26875060e-08])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There may be a possibility that my gradient function is computed wrongly, therefore, I check the my gradient function againist finite difference:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def approx_gradient(f):&#xA;    def wrapped_f(x0):&#xA;        n = len(x0)&#xA;        G = np.empty((n,),dtype=np.float64)&#xA;        h = np.sqrt(np.finfo(np.float64).eps)&#xA;&#xA;        g1 = f(x0)&#xA;        for i in xrange(n):&#xA;            x1 = np.array(x0,dtype=np.float64)&#xA;            x1[i] += h&#xA;            g2 = f(x1)&#xA;            G[i] = (g2 - g1) / h&#xA;        return G&#xA;    return wrapped_f&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Print the error as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# r1 is gradient computed by code, r2 is gradient computed by above finite-difference&#xA;d = np.log10(np.mean(np.abs(r1-r2)))&#xA;print(&quot;dL: %s =&amp;gt; %s, error: %s\n&quot; % (repr(x0),repr(r1),str(d)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dL: array([ 0.,  0.,  0.,  0.,  0.]) =&amp;gt; array([-338.93276474, -114.42079214, -216.55003298, -137.12475499,&#xA;        -52.57668841]), error: -1.12866497845&#xA;dL: array([ 338.93276474,  114.42079214,  216.55003298,  137.12475499,&#xA;         52.57668841]) =&amp;gt; array([ nan,  nan,  nan,  nan,  nan]), error: nan&#xA;dL: array([  3.38932765e-06,   1.14420792e-06,   2.16550033e-06,&#xA;         1.37124755e-06,   5.25766884e-07]) =&amp;gt; array([-338.93116156, -114.42016384, -216.54896297, -137.12398571,&#xA;        -52.57656892]), error: -1.3444523876&#xA;dL: array([ 0.00238973,  0.00080675,  0.00152684,  0.00096683,  0.00037071]) =&amp;gt; array([-337.80294205, -113.97709305, -215.79496551, -136.58082878,&#xA;        -52.49109471]), error: -1.76425541273&#xA;dL: array([ 169.46757724,   57.21079945,  108.27577991,   68.56286091,&#xA;         26.28852956]) =&amp;gt; array([ nan,  nan,  nan,  nan,  nan]), error: nan&#xA;dL: array([ 0.00238973,  0.00080675,  0.00152684,  0.00096683,  0.00037071]) =&amp;gt; array([-337.80294205, -113.97709305, -215.79496551, -136.58082878,&#xA;        -52.49109471]), error: -1.76425541273&#xA;dL: array([ 0.25288777,  0.08537274,  0.16157439,  0.10231284,  0.03922902]) =&amp;gt; array([-214.23525183,  -56.37608777, -123.86215026,  -59.8381013 ,&#xA;        -29.35530006]), error: -4.13744558124&#xA;dL: array([ 2.45189744, -1.10216675, -0.00860584, -2.25562141, -0.0889141 ]) =&amp;gt; array([ 200.97116275, -107.57429828,   77.9175913 , -284.63382774,&#xA;        222.78613298]), error: 4.55716763394&#xA;dL: array([ 0.60625088, -0.10545527,  0.1342278 , -0.27658817,  0.01863746]) =&amp;gt; array([-179.74804571,  -91.13312507, -130.64600443, -147.53451709,&#xA;        -47.29257281]), error: -3.6510702955&#xA;dL: array([ 0.94280274, -0.44136287, -0.34555214, -0.04312928,  0.3963997 ]) =&amp;gt; array([-147.33106574,  -91.10716372, -144.5277363 , -121.87325508,&#xA;         59.49497911]), error: -2.92020205735&#xA;dL: array([ 0.7272913 , -0.22626399, -0.03832441, -0.19262499,  0.15449914]) =&amp;gt; array([-170.21539216,  -91.09157378, -134.59864658, -140.28045749,&#xA;        -16.11667578]), error: -3.52122115041&#xA;dL: array([ 0.87382717, -0.37020575, -0.21805711, -0.03796395,  0.05122782]) =&amp;gt; array([-164.76068544,  -94.42547784, -141.97757919, -126.19087232,&#xA;        -39.8567946 ]), error: -3.19928906949&#xA;dL: array([ 0.87817125, -0.41141175, -0.15890429, -0.01557913,  0.05239629]) =&amp;gt; array([-153.02101444,  -89.44357123, -130.9150735 , -117.22771105,&#xA;        -36.90142624]), error: -3.0131329785&#xA;dL: array([ 0.89554758, -0.57623572,  0.07770701,  0.07396015,  0.05707019]) =&amp;gt; array([-98.94618778, -66.39110534, -79.97680149, -75.88852736, -21.65497929]), error: -3.5340970988&#xA;dL: array([ 1.31009037,  1.3391422 , -0.80615157, -0.3413817 , -0.09094827]) =&amp;gt; array([  88.75622027,  103.66551146,   28.52931607,   59.92216422,&#xA;         34.82483184]), error: -3.86721832568&#xA;dL: array([ 1.03325948,  0.06005647, -0.2159125 , -0.0640172 ,  0.00789817]) =&amp;gt; array([-57.66593306, -27.77908949, -59.01722568, -47.64785975, -13.70147329]), error: -3.21832537521&#xA;dL: array([  9.95932857e-01,  -1.40930257e-01,   1.39440628e-01,&#xA;         4.05132617e-02,   5.47288338e-04]) =&amp;gt; array([ 27.98975787,  12.98469404,  25.68770109,  19.28943637,   5.48862252]), error: -4.19602531575&#xA;dL: array([ 0.98974304, -0.10584977,  0.05390259,  0.02224462,  0.00674051]) =&amp;gt; array([-3.07471251, -1.62491769, -2.83340373, -2.41904387, -1.23502546]), error: -3.84209274725&#xA;dL: array([ 0.98982872, -0.10776008,  0.0609875 ,  0.0246589 ,  0.00848128]) =&amp;gt; array([-0.07129277, -0.05980753, -0.10964847, -0.12157649,  0.00033618]), error: -3.57935797704&#xA;dL: array([ 0.98941444, -0.10804747,  0.06173506,  0.02499305,  0.00836133]) =&amp;gt; array([-0.00140251, -0.00150752,  0.00292727,  0.00098813, -0.01230324]), error: -4.19996237842&#xA;dL: array([ 0.98944092, -0.10800269,  0.06166992,  0.0249806 ,  0.00840967]) =&amp;gt; array([ 0.00141414,  0.000485  ,  0.00088848,  0.00042683,  0.00068224]), error: -4.05494671838&#xA;dL: array([ 0.98943748, -0.10800296,  0.06167061,  0.02498174,  0.00840839]) =&amp;gt; array([  2.32462795e-05,  -1.47758964e-05,  -5.04953544e-06,&#xA;        -4.29179409e-05,  -1.69174238e-05]), error: -4.30601047846&#xA;dL: array([ 0.9894372 , -0.10800295,  0.06167076,  0.02498195,  0.00840848]) =&amp;gt; array([  3.89230032e-07,  -8.59830965e-07,  -1.92379457e-07,&#xA;        -1.03506122e-06,  -1.88641715e-06]), error: -4.04379091415&#xA;dL: array([ 0.98943719, -0.10800294,  0.06167076,  0.02498195,  0.00840848]) =&amp;gt; array([ -6.05248630e-08,   2.30028962e-08,   3.00105580e-08,&#xA;        -1.12310441e-07,   1.26875060e-08]), error: -3.67971485116&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;OK. I think my updated question would be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I still think my gradient does not compute accurately in the box $[-5,5]^n$. I will approximate fisher information using my gradient function, I couldn't get a semi-positive definite information matrix.&lt;/li&gt;&#xA;&lt;li&gt;Are there any way to avoid BFGS to evaluate some &quot;big&quot; points?&lt;/li&gt;&#xA;&lt;li&gt;Are there any way to compute the copula density by computing $f$ and $g$ only for discrete margin?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4211" LastEditorUserId="4211" LastEditDate="2014-10-09T04:24:20.690" LastActivityDate="2014-10-09T04:24:20.690" Title="How to compute frank copula and its derivative accurately?" Tags="|numerical-analysis|reference-request|scipy|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14804" PostTypeId="2" ParentId="14785" CreationDate="2014-10-08T07:00:23.117" Score="-1" Body="&lt;p&gt;Matlab has decent ODE solvers. &lt;code&gt;ode45&lt;/code&gt; is popular (4th order Runge-Kutta) but there are obviously more. See &lt;a href=&quot;http://www.mathworks.com/help/matlab/ordinary-differential-equations.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for example.&lt;/p&gt;&#xA;" OwnerUserId="10457" LastActivityDate="2014-10-08T07:00:23.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14805" PostTypeId="2" ParentId="14785" CreationDate="2014-10-08T10:22:15.267" Score="3" Body="&lt;p&gt;If you're using C++, the &lt;code&gt;odeint&lt;/code&gt; library of boost seems to be fast and easy to use: &lt;a href=&quot;http://www.boost.org/doc/libs/1_56_0/libs/numeric/odeint/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://www.boost.org/doc/libs/1_56_0/libs/numeric/odeint/doc/html/index.html&lt;/a&gt;.&#xA;The boost libraries are known as high-quality software packages and are widely used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As shown in &lt;a href=&quot;http://www.boost.org/doc/libs/1_56_0/libs/numeric/odeint/doc/html/boost_numeric_odeint/getting_started/short_example.html&quot; rel=&quot;nofollow&quot;&gt;this example&lt;/a&gt; they also have an &lt;code&gt;ode45&lt;/code&gt;-like integrator.&lt;/p&gt;&#xA;" OwnerUserId="7094" LastActivityDate="2014-10-08T10:22:15.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14806" PostTypeId="2" ParentId="12913" CreationDate="2014-10-08T13:06:01.287" Score="5" Body="&lt;p&gt;I think this happened because $L$ is a finite-difference operator that approximates your equation with some error (it is $\mathcal O(dx^2)$). To get small error your should increase &lt;code&gt;nx&lt;/code&gt; and &lt;code&gt;ny&lt;/code&gt;. If you set in your example &lt;code&gt;nx = ny = 101&lt;/code&gt; ($dx = 0.01$) you will get error about 1e-4.&lt;/p&gt;&#xA;" OwnerUserId="10388" LastEditorUserId="20688" LastEditDate="2018-03-24T16:39:37.753" LastActivityDate="2018-03-24T16:39:37.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14807" PostTypeId="2" ParentId="14785" CreationDate="2014-10-08T19:10:35.557" Score="9" Body="&lt;p&gt;In general, I'd say the following open source tools tend to be (roughly) best-of-breed, in the following order:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;PETSc has implemented a number of ODE solvers as part of TS, its time-stepping routines. There are a number of integrators implemented, including ARKIMEX, EIMEX, Rosenbrock-W, Crank-Nicolson, backward Euler, several Runge-Kutta methods (including the classic 4-5 variant, and SSP schemes), general linear methods, and an interface to CVODE in SUNDIALS, which is a BDF method. It's also a highly scalable parallel library with facilities for both message passing and GPU programming, although I haven't used the GPU side of things. They're currently working out trying to use threads. PETSc is written in C, and has Fortran and Python interfaces; for the implicit methods, a large variety of nonlinear solvers, linear solvers, nonlinear preconditioners, and linear preconditioners are available. PETSc has won an R&amp;amp;D 100 award, and has the best development practices for a scientific code that I am aware of. However, it is very complicated. It includes a huge number of examples and a large manual, along with good mailing lists. A significant number of PDE tools and frameworks build on PETSc, including many finite element codes.&lt;/li&gt;&#xA;&lt;li&gt;Trilinos has implemented in its Rythmos package forward Euler, backward Euler, explicit Runge-Kutta, implicit Runge-Kutta, and BDF methods. It is written in C++, and is also implemented in parallel. Like PETSc, Trilinos also contains linear and nonlinear solvers, along with some preconditioners. Trilinos tends to require more &quot;buy-in&quot; to its ecosystem, and is much more of a framework than PETSc.&lt;/li&gt;&#xA;&lt;li&gt;SUNDIALS currently implements a BDF method and an Adams-Bashforth method. It's written in very readable C code, and includes some examples. It contains serial and parallel implementations, and the internal development version has added threading and ARKIMEX methods. I'm very familiar with this code, and if you want to use an integrator without having to learn a huge framework, SUNDIALS is a reasonable choice. However, I find their development practices frustrating. They release infrequently, and if you find a bug in the code they release (of which there are many), they may have already fixed it internally without letting their mailing list know. There's no formal public repository for the code, although if you dig around enough, you can find informal Git repositories. If they post patches, the patch format is rather informal, instead of using a standard format readable by &lt;code&gt;patch&lt;/code&gt;. Their mailing list also went dark for a few months this past year (disclosure: I work at the same institution as the authors of SUNDIALS, so I prodded them to fix this oversight). Use this code with some caution. There's also an official MATLAB interface, an official Fortran interface, and some unofficial interfaces in other languages, like Python.&lt;/li&gt;&#xA;&lt;li&gt;There are a number of Fortran codes out there, like Ernst Hairer's implementations (DOPRI5 - explicit Runge-Kutta 4(5) method, DOP853 - explicit Runge-Kutta 8(5,3) method, RADAU - Radau collocation, RODAS - Rosenbrock method), old versions of SUNDIALS-related codes (VODE, DASPK), and really old stiff solvers (DASAC, DASSL). Some of these have been wrapped in Python (via SciPy), some of them have been ported to other languages. Most of these are strictly serial codes, but they're well-trusted implementations.  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I hesitate to recommend MATLAB because although the MATLAB solvers are generally good, they can only be used with MATLAB, which is proprietary. If you have MATLAB available, they're good to learn from in terms of using serial codes. Some of them are based on the Fortran codes I mentioned. If you don't have access to MATLAB, it's tough to play with the solver code (which I'm sure you could still find, but you wouldn't be able to execute it).  You should be able to experiment with all of the open source code I mention above without worrying about licensing issues.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-08T19:10:35.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14808" PostTypeId="2" ParentId="6743" CreationDate="2014-10-08T20:39:34.703" Score="3" Body="&lt;p&gt;Note that CPLEX 12.6 and later includes functionality to solve general nonconvex QPs and MIQPs.   However, for the special case of the product of a binary and continuous variables, the reformulation in the previous answer is likely to run faster.   But, for nonconvex QPs with products of continuous variables in the objective, this type of reformulation no longer applies.&lt;/p&gt;&#xA;" OwnerUserId="10466" LastActivityDate="2014-10-08T20:39:34.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14809" PostTypeId="1" AcceptedAnswerId="14810" CreationDate="2014-10-08T21:32:23.907" Score="7" ViewCount="311" Body="&lt;p&gt;I am working with legacy physical codes and I develop new ones based on the output of them.&#xA;They all use their own internal normalization of variables (for example all distances are divided by the physical size of the simulation box) which leads to a mess... The goal, I think, is to have all variables near ~1. On the contrary, my code uses the international system of units to clarify this mess. Since I use floating point double precision, I do not have any problem of denormalization... But my colleagues continue to repeat to me that one should normalize its variables ... without being able to explain me the reasons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it a legacy practice, or there are still excellent reasons to do this ?&lt;/p&gt;&#xA;" OwnerUserId="6627" LastEditorUserId="549" LastEditDate="2014-10-09T17:04:13.927" LastActivityDate="2014-10-09T17:04:13.927" Title="Why should I renormalize physical variables?" Tags="|floating-point|computational-physics|preconditioning|precision|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14810" PostTypeId="2" ParentId="14809" CreationDate="2014-10-08T22:20:44.147" Score="15" Body="&lt;p&gt;This question is very closely related to (and possibly a duplicate of) &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7390/is-variable-scaling-essential-when-solving-some-pde-problems-numerically/7392#7392&quot;&gt;Is variable scaling essential when solving some PDE problems numerically?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are still good practical reasons to nondimensionalize equations, if possible:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It reduces the number of independent parameters for parametric studies (which was one of the original reasons for nondimensionalization in the first place), which is important for uncertainty quantification. The fewer parameters you have, the less expensive uncertainty quantification will be.&lt;/li&gt;&#xA;&lt;li&gt;It's equivalent to preconditioning your equations. For affine transformations of variables (only one variable per transformation, so there is no coupling), it amounts to diagonal preconditioning, which helps with convergence of iterative methods, and it's usually cheap to do it yourself analytically while manipulating the equations on pen and paper (or using your favorite computer algebra system).&lt;/li&gt;&#xA;&lt;li&gt;It gives you insight into the equations. If you know which terms are dominant, and which terms aren't, you can sometimes develop reasonable right preconditioners by dropping non-dominant coupling terms. For an example that could have been deduced due to appropriate scaling (instead, they appeal directly to physical arguments), see &lt;a href=&quot;http://www.cs.odu.edu/~keyes/papers/jfnk.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jacobian-free Newton-Krylov methods: a survey of approaches and applications&lt;/a&gt;, Section 3.4.1.&lt;/li&gt;&#xA;&lt;li&gt;If you know all of your dependent variables are of order 1, it gives you a natural point of reference for error tolerances of numerical methods. Otherwise, if you know characteristic scales for your variables, you'd need to adjust your error tolerances accordingly.&lt;/li&gt;&#xA;&lt;li&gt;For independent variables, it can be helpful when generating meshes. For instance, I have been modeling microscale devices, and some mesh generators (e.g., the built-in mesh generator in FEniCS/DOLFIN 1.3) will return errors if I use standard metric dimensions, like $10^{-6}$ for the edges of a square domain. If I nondimensionalize, I am meshing a unit square, and those numerical issues go away.&lt;/li&gt;&#xA;&lt;li&gt;It's sometimes necessary to avoid singularities in simulations. (See &lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/a270533.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Preconditioning and the Limit to the Incompressible Flow equations&lt;/a&gt;.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-10-09T00:07:24.063" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14811" PostTypeId="1" CreationDate="2014-10-09T00:24:39.483" Score="1" ViewCount="361" Body="&lt;p&gt;I am a complete beginner in computational fluid dynamics. I have a 3D mesh in Blender representing the object I would like to do my tests on. I can export it in (I think) any useful format for 3D. I would like to know where I can start to make this kind of simulation with SU2 CFD:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;External flow&lt;/li&gt;&#xA;&lt;li&gt;Air at standard conditions&lt;/li&gt;&#xA;&lt;li&gt;Air flows against my mesh at some speed (say, 10 m/s)&lt;/li&gt;&#xA;&lt;li&gt;I would like to know what the drag force on my object will be&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is it possible in some easy way? I tried to have a look around but couldn't find any tutorial or example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="10467" LastActivityDate="2014-10-09T00:24:39.483" Title="Compute drag force on a mesh in SU2 CFD (beginner)" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="1" ClosedDate="2014-10-10T03:52:45.247" ContentLicense="CC BY-SA 3.0" />
  <row Id="14812" PostTypeId="1" AcceptedAnswerId="14815" CreationDate="2014-10-09T08:40:56.580" Score="5" ViewCount="1708" Body="&lt;p&gt;In the framework of Finite Element Method, when the Newton method is used, we solve $J(x^k) \delta x = -f(x^k)$, and the increment $\delta x$ would not change some entries from $x^k$ related to Dirichlet boundary conditions. So, I was wondering about the needs to impose homogenous dirichlet conditions in the Jacobian matrix $J(x^k)$, such that $\delta x$ would have some zero entries. &#xA;My question are: is this really need? Should I leave the Jacobian untouched and post-process $x^{k+1}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="10470" LastEditorUserId="10470" LastEditDate="2014-10-09T09:49:25.020" LastActivityDate="2016-04-10T18:20:09.717" Title="Do I need to impose boundary conditions in the Jacobian matrix?" Tags="|finite-element|boundary-conditions|newton-method|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14813" PostTypeId="2" ParentId="14812" CreationDate="2014-10-09T15:44:42.893" Score="6" Body="&lt;p&gt;There are multiple ways of implementing Dirichlet boundary conditions when using the finite element method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each unknown $i$ of the system belonging to the Dirichlet boundary, you can zero out row $i$ of the Jacobian and entry $i$ of the right-hand side. At that rate, you can reformulate the problem for just the unknowns in the interior, which reduces its size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can enforce the Dirichlet boundary conditions at every step of the nonlinear solve. However, this assumes that you're the one responsible for writing the solver; if you want to use someone else's software to solve the nonlinear system for you, you may have to resort to the first approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://books.google.com/books?isbn=0898716144&quot;&gt;This book&lt;/a&gt; has a whole section on Dirichlet boundary conditions which covers both of these options in more detail and a few others besides, e.g. penalty methods.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-10-09T15:44:42.893" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14814" PostTypeId="1" AcceptedAnswerId="14816" CreationDate="2014-10-09T16:13:34.810" Score="1" ViewCount="247" Body="&lt;p&gt;I am doing a nondimensional Navier-Stokes simulation, but it's not exactly clear what values to use for the various gas constants: $R$, $C_v$, and $C_p$. Obviously, once I know one of these and $\gamma$, I can define the others. A colleague of mine is using the formula&#xA;$$C_v=\frac{1}{\gamma*(\gamma-1)M_\infty^2}$$&#xA;but it's not clear where this comes from and what value to use for $M_\infty$, especially in the context of shock-tube problems. For lack of a better option, I've just been setting $M_\infty=1$, but I'm not convinced this is defensible. What do other people do in this context?&lt;/p&gt;&#xA;" OwnerUserId="3309" LastActivityDate="2014-10-10T22:12:51.317" Title="Nondimensional gas constants" Tags="|fluid-dynamics|navier-stokes|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14815" PostTypeId="2" ParentId="14812" CreationDate="2014-10-09T18:43:26.097" Score="4" Body="&lt;p&gt;This remark is a bit too long for a comment, but regarding DanielShapero's remark with respect to changing the Jacobian matrix, for every degree of freedom $i$ corresponding to a Dirichlet boundary condition (BC), a common approach is to set $i$th row of the Jacobian matrix equal to the $i$th row of the identity matrix, meaning your (quasi-)Newton iterates should be linear systems that can be permuted into the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\left[\begin{array}{cc}A &amp;amp; B \\ 0 &amp;amp; I\end{array}\right]\left[\begin{array}{c}\delta x_{interior} \\ \delta x_{BC}\end{array}\right] = \left[\begin{array}{c}-f_{interior}(x^{k}) \\ -f_{BC}(x^{k})\end{array}\right].&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As long as the initial guess to the nonlinear solve sets the Dirichlet BC degrees of freedom to the values specified in those boundary conditions, then at every iteration of the nonlinear solver, the Dirichlet boundary conditions should be satisfied, and $\delta{x}_{BC} = 0$ because $-f_{BC}(x^{k}) = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do something similar for functional iteration-type nonlinear solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: If you zero out the rows of the permuted Jacobian matrix to get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\left[\begin{array}{cc}A &amp;amp; B \\ 0 &amp;amp; 0\end{array}\right]\left[\begin{array}{c}\delta x_{interior} \\ \delta x_{BC}\end{array}\right] = \left[\begin{array}{c}-f_{interior}(x^{k}) \\ -f_{BC}(x^{k})\end{array}\right],&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then the resulting rank-deficient system can be solved using least-squares direct methods (QR, SVD) or Krylov methods (which will calculate a Drazin inverse and converge to the correct solution). LU-based methods should return an error due to a zero pivot, and can't be used without reformulating the system in terms of the interior degrees of freedom only, which is a minor drawback of zeroing out the rows. Solving the reformulation $A \delta{x}_{interior} = -f_{interior}(x^{k})$ is probably preferable at that point.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-10-09T21:22:48.517" LastActivityDate="2014-10-09T21:22:48.517" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="14816" PostTypeId="2" ParentId="14814" CreationDate="2014-10-09T23:00:49.193" Score="4" Body="&lt;p&gt;If you are using an ideal gas equation of state, only the ratio of specific heats should appear in the nondimensional governing compressible  Navier-Stokes equations. Knowing that you want to simulate (say) air, which provides gamma, is sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is your setting and my answer unclear, please let me know and I can point you to a write up.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2014-10-09T23:00:49.193" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14817" PostTypeId="1" AcceptedAnswerId="14827" CreationDate="2014-10-10T00:52:06.187" Score="9" ViewCount="411" Body="&lt;p&gt;I am looking for a fast (dare I say optimal?) explicit solution the 3x3 linear real problem, $\mathbf{A}\mathbf{x} = \mathbf{b}$, $\mathbf{A} \in \mathbf{R}^{3 \times 3}, \mathbf{b} \in \mathbf{R}^{3}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrix $\mathbf{A}$ is general, but close to the identity matrix with a condition number close to 1.  Because $\mathbf{b}$ are actually sensor measurements with about 5 digits of precision, I do not mind losing several digits due to numerical issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, it is not hard to come up with an explicit solution based on any number of methods, but if there is something that has been shown to be optimal in terms of FLOPS count, that would be ideal (after all, the whole problem will likely fit in the FP registers!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Yes, this routine is called &lt;em&gt;often&lt;/em&gt;.  I've already gotten rid of low-hanging fruit and this is next in my profiling list...)&lt;/p&gt;&#xA;" OwnerUserId="761" LastActivityDate="2014-10-11T04:41:22.133" Title="Fast explicit solution for $\mathbf{A}\mathbf{x} = \mathbf{b}$, $ \mathbf{b} \in \mathbf{R}^3$, low condition number" Tags="|linear-solver|reference-request|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14818" PostTypeId="1" CreationDate="2014-10-10T00:54:45.310" Score="13" ViewCount="4885" Body="&lt;p&gt;Now that Formulize / Eureqa started charging $2500 a year for using it and having crippled the trial version, does anyone know of any replacements that can do similar things like find an equation given data points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was free, but is now out of the price range of anyone that's not a company or school.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(it's a &lt;a href=&quot;http://en.wikipedia.org/wiki/Symbolic_regression&quot;&gt;symbolic regression&lt;/a&gt; software)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="10475" LastEditorUserId="549" LastEditDate="2014-10-10T01:22:39.087" LastActivityDate="2021-06-23T20:22:49.333" Title="Seeking a free symbolic regression software" Tags="|machine-learning|regression|symbolic-computation|" AnswerCount="6" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14819" PostTypeId="2" ParentId="14818" CreationDate="2014-10-10T01:41:51.810" Score="4" Body="&lt;p&gt;After a cursory google search on the subject, it appears that &quot;symbolic regression&quot; is a problem that lends itself greatly to stochastic optimization algorithms like genetic programming (GP).  It is conceivable that you should look for an open source genetic programming library with modules specifically for symbolic regression, such as &lt;a href=&quot;https://code.google.com/p/deap/wiki/GPExample&quot; rel=&quot;nofollow&quot;&gt;DEAP (Distributed Evolutionary Algorithms in Python)&lt;/a&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-10-11T01:01:06.150" LastActivityDate="2014-10-11T01:01:06.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14820" PostTypeId="2" ParentId="14817" CreationDate="2014-10-10T02:20:38.680" Score="14" Body="&lt;p&gt;You can't beat an explicit formula. You can write down the formulas for the solution $x=A^{-1}b$ on a piece of paper. Let the compiler optimize things for you. Any other method will almost inevitably have &lt;code&gt;if&lt;/code&gt; statements or &lt;code&gt;for&lt;/code&gt; loops (e.g., for iterative methods) that will make your code slower than any straight line code.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-10T02:20:38.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14821" PostTypeId="2" ParentId="14814" CreationDate="2014-10-10T02:24:19.130" Score="2" Body="&lt;p&gt;It would probably be instructional if you went through the process of non-dimensionalization yourself, starting from the dimensional equations for which it is always clear which constant to use where. This way, you will discover which combinations of constants appear where.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2014-10-10T22:12:51.317" LastActivityDate="2014-10-10T22:12:51.317" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14822" PostTypeId="2" ParentId="14817" CreationDate="2014-10-10T02:24:27.950" Score="4" Body="&lt;p&gt;Probably Cramer's Rule. If you can avoid pivoting, maybe LU factorization; it's a 3x3 matrix, so unrolling the loops manually would be easy. Anything else will probably involve branching, and I doubt that a Krylov subspace method would converge often enough in 1 or 2 iterates for it to be worth it.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-10T02:24:27.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14823" PostTypeId="2" ParentId="14817" CreationDate="2014-10-10T06:11:49.977" Score="9" Body="&lt;p&gt;Since the matrix is so close to the identity, the following Neumann series will converge very rapidly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A^{-1} = \sum_{k=0}^\infty (I-A)^k$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the accuracy required it might even be good enough to truncate after 2 terms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A^{-1} \approx I + (I - A) = 2I - A.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might be slightly faster than a direct formula (as suggested in Wolfgang Bangerth's answer), though with much less accuracy. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You could get more accuracy with 3 terms:&#xA;$$A^{-1} \approx I + (I - A) + (I-A)^2 = 3I - 3A + A^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but if you write out the entry-by-entry formula for $(3I - 3A + A^2)b$, you are looking at a comparable amount of floating point operations as the &lt;a href=&quot;http://en.wikipedia.org/wiki/Invertible_matrix#Inversion_of_3.C3.973_matrices&quot;&gt;direct 3x3 matrix inverse formula&lt;/a&gt; (you don't have to do a division though, which helps a little). &lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2014-10-10T06:26:32.317" LastActivityDate="2014-10-10T06:26:32.317" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="14824" PostTypeId="1" CreationDate="2014-10-10T07:41:23.893" Score="0" ViewCount="87" Body="&lt;p&gt;I have tones of geometrical objects (spheres, meshes) I want to render: One way would be to first write a .rib file which could be certainly done and then push that file to a renderman compliant renderer such as the opensource Aqsis render.&#xA;The problem is this .rib file is soooo massive (1-10gb) that I dont think this is the preferred idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a possibility to access the aqsis renderer via a library and then somehow build the rib objects together in c++ and then call the renderer with this in the c++ code?&#xA;That would be awsome :-).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is there another preferred rendering method to render these huge amount of data? &lt;/p&gt;&#xA;" OwnerUserId="10478" LastActivityDate="2014-12-10T22:15:47.433" Title="Rendering Huge Data with Renderman Interface Bytestream" Tags="|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14825" PostTypeId="1" AcceptedAnswerId="14826" CreationDate="2014-10-10T09:00:57.707" Score="12" ViewCount="438" Body="&lt;p&gt;Just to be more precise, I'll put a worthy example of my typical use case.&lt;/p&gt;&#xA;&lt;p&gt;Let's say I'm developing a FEM software that produces several temporal solutions and inserts them in an HDF5 file, along with a bunch of correlated data.&lt;/p&gt;&#xA;&lt;p&gt;If I use a naive approach to writing integration tests, I could do like this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write the code&lt;/li&gt;&#xA;&lt;li&gt;Test it by hand on a simple case&lt;/li&gt;&#xA;&lt;li&gt;Decide it works and save the right solution&lt;/li&gt;&#xA;&lt;li&gt;Write a custom test that compares the solution of a new version of the code with the right solution I already found for the simple case.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This obviously won't work, as even changing my BLAS implementation could slightly change every single DOF of my solution. Imposing a test of similarity between solutions within a certain norm could be more effective, but it is not trivial and it feels like overshooting for just writing a test.&lt;/p&gt;&#xA;&lt;p&gt;How can I write integration tests for this software to be run automatically on Jenkins? Is there some tool that helps me deal with the normal uncertainty of the solutions?&lt;/p&gt;&#xA;" OwnerUserId="10479" LastEditorUserId="23791" LastEditDate="2021-12-20T21:20:06.163" LastActivityDate="2021-12-20T21:20:06.163" Title="How to write integration tests for numeric simulation software?" Tags="|finite-element|numerics|testing|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14826" PostTypeId="2" ParentId="14825" CreationDate="2014-10-10T18:29:09.343" Score="13" Body="&lt;p&gt;I don't think you can avoid using a tolerance for floating-point comparisons. Error due to round-off, discretization, and so on using floating-point numbers is unavoidable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I typically do to test FEM code I write is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;test the stiffness &amp;amp; mass matrices on a single element to make sure I get local element assembly right, compare against a known result&lt;/li&gt;&#xA;&lt;li&gt;test again on a simple mesh to make sure I get global assembly right, compare against a known result&lt;/li&gt;&#xA;&lt;li&gt;test assembling the load vector on a function with known result&lt;/li&gt;&#xA;&lt;li&gt;test the whole solver on a manufactured solution that should be &quot;exact&quot; (to working precision, with enough refinement; an example would be a Poisson problem with a linear solution on a $P_{1}$ discretization)&lt;/li&gt;&#xA;&lt;li&gt;test the solver on a manufactured solution that will be inexact, refine the solution repeatedly, calculate the error, and regress the error against the mesh size to determine the order of convergence of the solver&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For a Poisson problem, these tests exercise most of the solver. For more complicated problems, your mileage may vary; it's easier to write test cases for some problems than it is for others, and it's easier to write tests for some programs (e.g., a serial code you write yourself) than it is for others (e.g., a parallel code, where processor ordering can affect the results of reductions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these tests require some judgment call when it comes to selecting tolerances, but with manufactured solutions, you'll have a good idea of when something is grossly wrong versus when it's correct. Frequently, I start with something like $10^{-7}$ as an absolute tolerance for a solution with characteristic values around 1, and adjust up or down accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Testing the order of convergence requires the most judgment, because not only do you need to select tolerances, but you need to select the mesh spacings you will use for the regression. It's better to experiment with that first and graph the error versus the mesh size (really, since it's a power law, log of the norm of the error versus log of the mesh size; you will also regress the log of the norm of the error versus the log of the mesh size). Once you have data that makes sense, then convert the code that plots the convergence behavior to a test that compares the order of convergence obtained from regression to the theoretical order of convergence. For example, if I expect an order of convergence of 3, and I get anywhere between, say 2.7 and 3.3, or something to that effect, then I'll say that the test &quot;passed&quot;. (Logging the convergence plots somewhere wouldn't hurt either, if you can do that, so if you get a grossly wrong value, you have a sanity check to look at.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-10T18:29:09.343" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14827" PostTypeId="2" ParentId="14817" CreationDate="2014-10-10T22:33:45.290" Score="5" Body="&lt;p&gt;FLOPS count based on the suggestions above:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;LU, no pivoting:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mul = 11, Div/Recip = 6, Add/Sub = 11, Total = 28; or&lt;/li&gt;&#xA;&lt;li&gt;Mul = 16, Div/Recip = 3, Add/Sub = 11, Total = 30&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Gaussian Elimination with back-substitution, no pivoting:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mul = 11, Div/Recip = 6, Add/Sub = 11, Total = 28; or&lt;/li&gt;&#xA;&lt;li&gt;Mul = 16, Div/Recip = 3, Add/Sub = 11, Total = 30&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Cramer's rule via cofactor expansion&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mul = 24, Div = 3, Add/Sub = 15, Total = 42; or&lt;/li&gt;&#xA;&lt;li&gt;Mul = 27, Div = 1, Add/Sub = 15, Total = 43&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Explicit Inverse then multiply:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mul = 30, Div = 3, Add/Sub = 17, Total = 50; or&lt;/li&gt;&#xA;&lt;li&gt;Mul = 33, Div = 1, Add/Sub = 17, Total = 51&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;MATLAB proof-of-concepts:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cramer's Rule via Cofactor Expansion&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function k = CramersRule(A, m)&#xA;%&#xA;% FLOPS:&#xA;%&#xA;% Multiplications:        24&#xA;% Subtractions/Additions: 15&#xA;% Divisions:               3&#xA;%&#xA;% Total:                  42&#xA;&#xA;a = A(1,1);&#xA;b = A(1,2);&#xA;c = A(1,3);&#xA;&#xA;d = A(2,1);&#xA;e = A(2,2);&#xA;f = A(2,3);&#xA;&#xA;g = A(3,1);&#xA;h = A(3,2);&#xA;i = A(3,3);&#xA;&#xA;x = m(1);&#xA;y = m(2);&#xA;z = m(3);&#xA;&#xA;ei = e*i;&#xA;fh = f*h;&#xA;&#xA;di = d*i;&#xA;fg = f*g;&#xA;&#xA;dh = d*h;&#xA;eg = e*g;&#xA;&#xA;ei_m_fh = ei - fh;&#xA;di_m_fg = di - fg;&#xA;dh_m_eg = dh - eg;&#xA;&#xA;yi = y*i;&#xA;fz = f*z;&#xA;&#xA;yh = y*h;&#xA;ez = e*z;&#xA;&#xA;yi_m_fz = yi - fz;&#xA;yh_m_ez = yh - ez;&#xA;&#xA;dz = d*z;&#xA;yg = y*g;&#xA;&#xA;dz_m_yg = dz - yg;&#xA;ez_m_yh = ez - yh;&#xA;&#xA;&#xA;det_a = a*ei_m_fh - b*di_m_fg + c*dh_m_eg;&#xA;det_1 = x*ei_m_fh - b*yi_m_fz + c*yh_m_ez;&#xA;det_2 = a*yi_m_fz - x*di_m_fg + c*dz_m_yg;&#xA;det_3 = a*ez_m_yh - b*dz_m_yg + x*dh_m_eg;&#xA;&#xA;&#xA;p = det_1 / det_a;&#xA;q = det_2 / det_a;&#xA;r = det_3 / det_a;&#xA;&#xA;k = [p;q;r];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;LU (no pivoting) and back-substitution:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [x, y, L, U] = LUSolve(A, b)&#xA;% Total FLOPS count:     (w/ Mods)&#xA;%&#xA;% Multiplications:  11    16&#xA;% Divisions/Recip:   6     3&#xA;% Add/Subtractions: 11    11&#xA;% Total =           28    30&#xA;%&#xA;&#xA;A11 = A(1,1);&#xA;A12 = A(1,2);&#xA;A13 = A(1,3);&#xA;&#xA;A21 = A(2,1);&#xA;A22 = A(2,2);&#xA;A23 = A(2,3);&#xA;&#xA;A31 = A(3,1);&#xA;A32 = A(3,2);&#xA;A33 = A(3,3);&#xA;&#xA;b1 = b(1);&#xA;b2 = b(2);&#xA;b3 = b(3);&#xA;&#xA;L11 = 1;&#xA;L22 = 1;&#xA;L33 = 1;&#xA;&#xA;U11 = A11;&#xA;U12 = A12;&#xA;U13 = A13;&#xA;&#xA;L21 = A21 / U11;&#xA;L31 = A31 / U11;&#xA;&#xA;U22 = (A22 - L21*U12);&#xA;L32 = (A32 - L31*U12) / U22;&#xA;&#xA;U23 = (A23 - L21*U13);&#xA;&#xA;U33 = (A33 - L31*U13 - L32*U23);&#xA;&#xA;y1 = b1;&#xA;y2 = b2 - L21*y1;&#xA;y3 = b3 - L31*y1 - L32*y2;&#xA;&#xA;x3 = (y3                  ) / U33;&#xA;x2 = (y2 -          U23*x3) / U22;&#xA;x1 = (y1 - U12*x2 - U13*x3) / U11;&#xA;&#xA;L = [ ...&#xA;    L11,   0,   0;&#xA;    L21, L22,   0;&#xA;    L31, L32, L33];&#xA;&#xA;U = [ ...&#xA;    U11, U12, U13;&#xA;      0, U22, U23;&#xA;      0,   0, U33];&#xA;&#xA;x = [x1;x2;x3];&#xA;y = [y1;y2;y3];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Explicit Inverse then Multiply:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function x = ExplicitInverseMultiply(A, m)&#xA;%&#xA;% FLOPS count:                  Alternative&#xA;%&#xA;% Multiplications:        30            33&#xA;% Divisions:               3             1&#xA;% Additions/Subtractions: 17            17&#xA;% Total:                  50            51&#xA;&#xA;&#xA;a = A(1,1);&#xA;b = A(1,2);&#xA;c = A(1,3);&#xA;&#xA;d = A(2,1);&#xA;e = A(2,2);&#xA;f = A(2,3);&#xA;&#xA;g = A(3,1);&#xA;h = A(3,2);&#xA;i = A(3,3);&#xA;&#xA;ae = a*e;&#xA;af = a*f;&#xA;ah = a*h;&#xA;ai = a*i;&#xA;&#xA;bd = b*d;&#xA;bf = b*f;&#xA;bg = b*g;&#xA;bi = b*i;&#xA;&#xA;cd = c*d;&#xA;ce = c*e;&#xA;cg = c*g;&#xA;ch = c*h;&#xA;&#xA;dh = d*h;&#xA;di = d*i;&#xA;&#xA;eg = e*g;&#xA;ei = e*i;&#xA;&#xA;fg = f*g;&#xA;fh = f*h;&#xA;&#xA;dh_m_eg = (dh - eg);&#xA;ei_m_fh = (ei - fh);&#xA;fg_m_di = (fg - di);&#xA;&#xA;A = ei_m_fh;&#xA;B = fg_m_di;&#xA;C = dh_m_eg;&#xA;D = (ch - bi);&#xA;E = (ai - cg);&#xA;F = (bg - ah);&#xA;G = (bf - ce);&#xA;H = (cd - af);&#xA;I = (ae - bd);&#xA;&#xA;det_A = a*ei_m_fh + b*fg_m_di + c*dh_m_eg;&#xA;&#xA;x1 =  (A*m(1) + D*m(2) + G*m(3)) / det_A;&#xA;x2 =  (B*m(1) + E*m(2) + H*m(3)) / det_A;&#xA;x3 =  (C*m(1) + F*m(2) + I*m(3)) / det_A;&#xA;&#xA;x = [x1;x2;x3];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Gaussian Elimination:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function x = GaussianEliminationSolve(A, m)&#xA;%&#xA;% FLOPS Count:      Min   Alternate&#xA;%&#xA;% Multiplications:  11    16&#xA;% Divisions:         6     3&#xA;% Add/Subtractions: 11    11&#xA;% Total:            28    30&#xA;%&#xA;&#xA;a = A(1,1);&#xA;b = A(1,2);&#xA;c = A(1,3);&#xA;&#xA;d = A(2,1);&#xA;e = A(2,2);&#xA;f = A(2,3);&#xA;&#xA;g = A(3,1);&#xA;h = A(3,2);&#xA;i = A(3,3);&#xA;&#xA;b1 = m(1);&#xA;b2 = m(2);&#xA;b3 = m(3);&#xA;&#xA;% Get to echelon form&#xA;&#xA;op1 = d/a;&#xA;&#xA;e_dash  = e  - op1*b;&#xA;f_dash  = f  - op1*c;&#xA;b2_dash = b2 - op1*b1;&#xA;&#xA;op2 = g/a;&#xA;&#xA;h_dash  = h  - op2*b;&#xA;i_dash  = i  - op2*c;&#xA;b3_dash = b3 - op2*b1; &#xA;&#xA;op3 = h_dash / e_dash;&#xA;&#xA;i_dash2  = i_dash  - op3*f_dash;&#xA;b3_dash2 = b3_dash - op3*b2_dash;&#xA;&#xA;% Back substitution&#xA;&#xA;x3 = (b3_dash2                  ) / i_dash2;&#xA;x2 = (b2_dash        - f_dash*x3) / e_dash;&#xA;x1 = (b1      - b*x2 -      c*x3) / a;&#xA;&#xA;x = [x1 ; x2 ; x3];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Please feel free to add your own methods and counts to this post.&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="761" LastEditDate="2014-10-11T04:41:22.133" LastActivityDate="2014-10-11T04:41:22.133" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="14828" PostTypeId="1" CreationDate="2014-10-10T23:27:49.983" Score="4" ViewCount="678" Body="&lt;p&gt;I have a bounded non-convex function in 10-dimensional space. The function is quasi-smooth, you can imagine a histogram, here is an illustration, it just shows the idea and not related to my particular function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/noBFY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function value is obtained by time consuming simulation (it takes about 10 seconds). Obviously if I want compute gradients, I need to approximate them by difference quotients. If you need more details about the function, I might be able to say more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read the article of&#xA;L.M. Rios and N.V. Sahinidis, &lt;a href=&quot;http://link.springer.com/content/pdf/10.1007%2Fs10898-012-9951-y.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Derivative-free optimization: A review of algorithms and comparison of software implementations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I've tried all of TOMLAB solvers, proposed in article and also &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/software/mcs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MCS&lt;/a&gt; method, also mentioned in the article as one of the best.&#xA;But neither of them could not overperform simple Brent method accompanied by hand picked initial guess (well, I hardly believe I can produce such great guesses).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also heard about surrogate modeling, is it worth trying?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So which other global optimization methods should I consider?&lt;/p&gt;&#xA;" OwnerUserId="10484" LastEditorUserId="10484" LastEditDate="2014-10-11T09:38:48.573" LastActivityDate="2017-03-20T00:28:55.153" Title="Finding a global minimum of non-convex quasi-smooth function that is costly to evaluate" Tags="|optimization|constrained-optimization|nonconvex|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="14829" PostTypeId="2" ParentId="14117" CreationDate="2014-10-11T07:28:40.383" Score="1" Body="&lt;p&gt;I would first look into the thin plate splines (TPS), which are widely available in MATLAB. They are simple to use, and were found to perform fairly good in many cases. Here is the documentation:&#xA;&lt;a href=&quot;http://www.mathworks.de/de/help/curvefit/tpaps.html?refresh=true&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.de/de/help/curvefit/tpaps.html?refresh=true&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More detailed information is here: &lt;a href=&quot;http://user.engineering.uiowa.edu/~aip/papers/bookstein-89.pdf&quot; rel=&quot;nofollow&quot;&gt;http://user.engineering.uiowa.edu/~aip/papers/bookstein-89.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-10-11T07:28:40.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14830" PostTypeId="2" ParentId="10656" CreationDate="2014-10-11T08:28:46.393" Score="0" Body="&lt;p&gt;No, your numerical answer is correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main point is that your problem is essentially advective, and particles will move to the left as time goes on, so if you start with the initial condition $g(t=0, x) = 0$, the right section (i.e. $x&amp;lt;0$) cannot acquire non-zero values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed your equation can be solved analytically. Consider the characteristic curves $(t(s),x(s)) = (s,-s+a)$, where $a$ is a constant. Then $dg/dt - dg/dx = (dg/dt)(dt/ds)+(dg/dx)(dx/ds) = dg(t(s),x(s))/ds$. Thus, $dg/dt-dg/dx - x g = dg(t(s),x(s))/ds - (a-s)g(t(s),x(s)) = 0$. In other words, along curves  $(s,-s+a)$, the value of $g(t,x) = g(s)$ changes according to $dg(s)/ds = (a-s)g(s)$. The boundary condition at $x=0$ gives rise to &quot;new materials&quot;, which then &quot;travel to the left&quot; and the amount f materials changes according to $dg(s)/ds = (a-s)g(s)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If at a given $s_0$, $g(s_0) = 0$, then $g(s) = 0$ for $s&amp;gt;s_0$. This is why. If $g(s_0) = 0$, then $dg(s_0)/ds = (a-s_0)g(s_0) = 0 $. Similarly $d^2g(s_0)/ds^2 = -g(s_0) + (a-s_0)g'(s_0) = 0$ and so on. This means the initially zero values on the right section are &quot;transported&quot; to the left as essentially zero values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus how can the right section $x&amp;gt;0$ acquire non-zero values?&lt;/p&gt;&#xA;" OwnerUserId="10487" LastEditorUserId="10487" LastEditDate="2014-10-11T08:38:31.640" LastActivityDate="2014-10-11T08:38:31.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14831" PostTypeId="2" ParentId="14825" CreationDate="2014-10-11T11:59:29.193" Score="9" Body="&lt;p&gt;Geoff has already given an excellent overview, but I wanted to provide another real world look on it. In the deal.II project (&lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;noreferrer&quot;&gt;http://www.dealii.org/&lt;/a&gt;) we run some 7,000 tests with every change in the code base, on multiple platforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tests Geoff describes are mostly &quot;integration tests&quot;, i.e., they run through a significant part of the code base. You need relatively few of these, but it turns out that they are very difficult to debug: if they fail, and you don't have the immediate commit that broke them, there is a great deal of code to look through to find out which piece changed functionality. To avoid this, we have a few integration tests, but a very large number (thousands) of small unit tests that simply test one very small part of the library. The typical case where a change breaks functionality is that several of the integration tests fail plus several of the small unit tests, and the simplest of the unit tests is the one one uses for debugging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for how these tests work: yes, we store the output on a reference platform and then compare the output on platforms that run the tests. We use &lt;code&gt;numdiff&lt;/code&gt; to avoid the issue of small changes in roundoff, with appropriately set tolerances. It took us a very long time of tweaking tolerances to ensure that all 7,000 or so tests run successfully on every platform we use. Before that, we had simply designated one particular machine as the &quot;reference platform&quot;, and the tests were saved there and tests run there.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-11T11:59:29.193" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="14832" PostTypeId="1" CreationDate="2014-10-11T19:44:34.340" Score="1" ViewCount="109" Body="&lt;p&gt;I want to compute the expected value of a multivariate function &lt;code&gt;f(x)&lt;/code&gt; wrt to &lt;a href=&quot;http://en.wikipedia.org/wiki/Dirichlet_distribution&quot; rel=&quot;nofollow&quot;&gt;dirichlet distribution&lt;/a&gt;. My problem is &quot;penta-nomial&quot; (i.e 5 variables) so calculating the explicit form of the expected value seems unreasonable. Is there a way to numerically integrate it? Is there some package (e.g in R) that can let me do that? I realize I can solve it using first principle but I want the computation to be really fast.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(x) = \sum_{0,4}(q_i*log(n/q_i))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10491" LastActivityDate="2014-10-11T19:44:34.340" Title="Package for integration over non-rectangular region" Tags="|r|integral-equations|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15832" PostTypeId="2" ParentId="6853" CreationDate="2014-10-11T22:30:54.213" Score="0" Body="&lt;p&gt;According to &lt;a href=&quot;https://devtalk.nvidia.com/default/topic/624598/double-precision-and-cuda/&quot; rel=&quot;nofollow&quot;&gt;this discussion&lt;/a&gt;, Tesla and Titan GPUs are most the suitable (of the Nvidia GPUs) for double precision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking for example at a &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units&quot; rel=&quot;nofollow&quot;&gt;list of Nvidia GPUs on Wikipedia&lt;/a&gt; (a similar table for AMD GPUs is &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;) and comparing the single and double precision processing power (in terms of GFLOPs) one sees that double precision performance is much smaller than half the single precision performance for most other GPUs. For example for the GTX 900 series, the article mentions that the double precision performance is &lt;strong&gt;1/32&lt;/strong&gt; of the single precision performance, while &lt;a href=&quot;https://en.wikipedia.org/wiki/GeForce_700_series#GeForce_700_.287xx.29_series&quot; rel=&quot;nofollow&quot;&gt;this Wikipedia article&lt;/a&gt; says that for the GTX 700 series, the double precision performance is &lt;strong&gt;1/24&lt;/strong&gt; of the single precision performance (except for Titan where it can be as high as &lt;strong&gt;1/3&lt;/strong&gt; of the single precision performance).&lt;/p&gt;&#xA;" OwnerUserId="273" LastActivityDate="2014-10-11T22:30:54.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15833" PostTypeId="1" AcceptedAnswerId="15840" CreationDate="2014-10-12T05:01:56.073" Score="-1" ViewCount="6004" Body="&lt;p&gt;I have to solve 4 coupled first order differential equations for $f(t)$ ,$g(t)$, $h(t)$ and $w(t)$ witch are only functions of $t$ , but for every reference link a function of 3 variables is assumed (i.e &lt;a href=&quot;http://www.phy.davidson.edu/FacHome/dmb/py200/RungeKuttaMethod.htm&quot; rel=&quot;nofollow&quot;&gt;http://www.phy.davidson.edu/FacHome/dmb/py200/RungeKuttaMethod.htm&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if anyone could enlighten me on this problem,the equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{df}{dt} = \alpha f -\beta f + \theta g - (f+h)f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dg}{dt} = \psi f- \phi g$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dh}{dt} = \xi f+ \mu h -\tau h + \epsilon w- (f+h)h$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dw}{dt} = \nu h - \chi h$&lt;/p&gt;&#xA;" OwnerUserId="10468" LastEditorUserId="10468" LastEditDate="2014-10-12T16:59:25.690" LastActivityDate="2014-10-12T21:38:13.783" Title="Runge-Kutta 4th order for 4 coupled first order differential equation" Tags="|ode|runge-kutta|" AnswerCount="1" CommentCount="2" ClosedDate="2014-10-13T04:13:50.373" ContentLicense="CC BY-SA 3.0" />
  <row Id="15834" PostTypeId="1" AcceptedAnswerId="15837" CreationDate="2014-10-12T08:17:52.720" Score="2" ViewCount="163" Body="&lt;p&gt;Given a symmetric positive semi-definite matrix $Q\in\mathbb{R}^{n\times n}$, a vector $v\in\mathbb{R}^n$, a matrix $A\in\mathbb{R}^{m\times n}$ and a vector $b\in \mathbb{R}^m$ I'd like to solve the following optimization problem:&#xA;$\min_x x^TQx\ \ \text{s.t.}\ \ Qx=(x^TQx)\, v\ ,\ Ax\ge b$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, we have a scaled equality constraint $Qx=av$ where $a=x^TQx$.&#xA;Is there a name to such optimization problem? Are there solvers which can handle it?&lt;/p&gt;&#xA;" OwnerUserId="10450" LastEditorUserId="276" LastEditDate="2014-10-13T05:18:56.643" LastActivityDate="2014-10-13T09:38:46.347" Title="Solving a minimization problem with &quot;scaled&quot; equality constraint" Tags="|optimization|constrained-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15835" PostTypeId="1" AcceptedAnswerId="15839" CreationDate="2014-10-12T08:46:38.287" Score="4" ViewCount="26980" Body="&lt;p&gt;I'm newbie both in calculus and Python/Scipy so I apologize if this&#xA;question is too dumb.&#xA;I'm trying to model flow between two pressure vessels. Let's say we have&#xA;two points and a link between them like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[$Vc_1$, $P_1$]----($A$)----[$Vc_2$, $P_2$]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Vc_1$, $Vc_2$ are constants volumes of the nodes(vessels) and $P_1$, $P_2$ are varying pressures of the nodes respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I end up writing differential questions below. Never mind physical meaning&#xA;I just want to get math correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\mathrm{dP_1} }{\mathrm{d} t} = \frac{\mathrm{dVa} }{\mathrm{d} t} \cdot \frac{1}{Vc_1 \cdot B}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\mathrm{dP_2} }{\mathrm{d} t} = \frac{\mathrm{dVa} }{\mathrm{d} t} \cdot \frac{1}{Vc_2 \cdot B}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $B$ is compressibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\mathrm{dVa} }{\mathrm{d} t} = A \cdot K \cdot \sqrt {P_1 - P_2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\mathrm{dVa} }{\mathrm{d} t}$ is amount of &quot;flow&quot; or change of additional volume between nodes. $K$ is some constant coefficient and $A$ is link &quot;throughput&quot;. ($P_1-P_2$) can change sign so I've adjusted for this in the software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is Python program that I wrote to evaluate this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;#!/usr/bin/env python&#xA;&#xA;import math&#xA;from scipy.integrate import odeint&#xA;from time import time&#xA;import numpy&#xA;&#xA;B_compressibility = 0.0000033 # water compressibility&#xA;K = 0.747871759938 # coefficient&#xA;&#xA;Vc_1 = 20&#xA;Vc_2 = 50&#xA;A = 0.01&#xA;P_1 = 4000&#xA;P_2 = 2000&#xA;&#xA;def deriv(state, t):    &#xA;    _P_1 = state[0]&#xA;    _P_2 = state[2]   &#xA;    diff_P = _P_1 - _P_2&#xA;    flow_direction = math.copysign(1, diff_P)&#xA;    dVa = flow_direction * A * K * math.sqrt(abs(diff_P))&#xA;    dP_1 = -(dVa/Vc_1)/B_compressibility&#xA;    dP_2 =  (dVa/Vc_2)/B_compressibility&#xA;    #print 'IN  ', state&#xA;    #print 'OUT ', [dP_1, -dVa, dP_2, dVa]&#xA;    return [dP_1, -dVa, dP_2, dVa]&#xA;&#xA;if __name__ == '__main__':&#xA;    Va_1 = Vc_1 * P_1 * B_compressibility&#xA;    Va_2 = Vc_2 * P_2 * B_compressibility&#xA;    odeIterations = 10&#xA;    timeperiod = numpy.linspace(0.0,1.0, odeIterations)    &#xA;    initial_state = [P_1, Va_1, P_2, Va_2]&#xA;    t0 = time()&#xA;    state_array = odeint(deriv, initial_state, timeperiod)&#xA;    t1 = time()&#xA;    print 'runtime %fs' %(t1-t0)        &#xA;    print state_array&#xA;    P_1, Va_1, P_2, Va_2 = state_array[odeIterations-1]&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Below is output from program&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;Excess work done on this call (perhaps wrong Dfun type).&#xA;Run with full_output = 1 to get quantitative information.&#xA;runtime 0.041000s&#xA;[[  4.00000000e+003   2.64000000e-001   2.00000000e+003   3.30000000e-001]&#xA; [  3.49242034e+003   2.30499743e-001   2.20303186e+003   3.63500257e-001]&#xA; [  3.09580400e+003   2.04323064e-001   2.36167840e+003   3.89676936e-001]&#xA; [  2.81015098e+003   1.85469965e-001   2.47593961e+003   4.08530035e-001]&#xA; [  2.63546127e+003   1.73940444e-001   2.54581549e+003   4.20059556e-001]&#xA; [  2.57173487e+003   1.69734501e-001   2.57130605e+003   4.24265499e-001]&#xA; [  2.57142857e+003   1.69714286e-001   2.57142857e+003   4.24285714e-001]&#xA; [  1.83357137e-299   1.80790662e-299   1.83145695e-299   1.87152166e-299]&#xA; [  1.83276935e-299   1.80681296e-299   1.83182150e-299   1.87141230e-299]&#xA; [  1.83379011e-299   1.80746916e-299   1.83320682e-299   1.83229543e-299]]&#xA; lsoda--  at current t (=r1), mxstep (=i1) steps   &#xA;       taken on this call before reaching tout     &#xA;      In above message,  I1 =       500&#xA;      In above message,  R1 =  0.6110315150411E+00&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;odeint&lt;/code&gt; gives correct results up to 7th line and then something goes&#xA;seriously wrong. I have searched in google and it looks like I'm not the&#xA;only one who struggles with scipy. Everybody suggests increasing mxstep but&#xA;that doesn't solve my problem. In addition it slows down the method&#xA;significantly. Somebody suggested reducing accuracy but I don't know how to&#xA;do that. Decreasing accuracy is OK if that help as I don't need super accuracy from the &lt;code&gt;odeint&lt;/code&gt;. Couple of digits after dot is more than enough for me. Also I just need final values so in fact I want to decrease number of steps. Any help is greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="11490" LastEditorUserId="9667" LastEditDate="2014-10-12T19:44:38.910" LastActivityDate="2023-11-29T10:16:33.037" Title="scipy odeint - Excess work done on this call" Tags="|ode|scipy|model|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15837" PostTypeId="2" ParentId="15834" CreationDate="2014-10-12T16:34:43.660" Score="3" Body="&lt;p&gt;Write it as the nonconvex quadratically constrained program $\min x^TQx$ s.t. $Qx = av, Ax \geq b, x^TQx = a$. When $Qx=av$ the constraint $a = x^TQx$ simplifies to $a = x^Tav$, i.e, $1 = x^Tv$ or $a=0$ Hence, you can solve the convex quadratic program $\min x^TQx$ s.t. $Qx = av, Ax \geq b, 1=x^Tv$, and another linear programming feasibility problem where you constrain $x$ to the nullspace of $Q$ (which would lead to $a=0$ and optimal objective $0$) $\min 0$ s.t. $Qx = 0, Ax \geq b$ and then you pick the best out of those two solutions.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2014-10-12T18:47:03.093" LastActivityDate="2014-10-12T18:47:03.093" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15839" PostTypeId="2" ParentId="15835" CreationDate="2014-10-12T21:16:24.217" Score="7" Body="&lt;p&gt;Typically, this situation is handled by:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Using an &quot;event function&quot; or something to that effect to stop the integration when $P_{1} = P_{2}$. LSODA (the backend for &lt;code&gt;scipy.integrate.odeint&lt;/code&gt;) does not have this capability, but other integrators, such as CVODE (part of SUNDIALS) do.&lt;/li&gt;&#xA;&lt;li&gt;Reformulating the right-hand side so it's continuously differentiable when $P_{1} = P_{2}$. It's a bit of a hack -- I remember an engineer explaining it to me when talking about process simulators -- but it gets the physics approximately right.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Typical existence theorems for solutions of differential equations assume that the right-hand side is at least Lipschitz continuous. This particular solver fails because you need at least continuous differentiability in your right-hand side in order for the integration formulas to make sense, and at $P_{1} = P_{2}$, your right-hand side is not continuously differentiable (implied because it is not Lipschitz).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Physically, flow should stop when the pressures equilibrate. When modeling the flow as 1-D, there can be pressure waves that flow back and forth between the vessels, but since you're modeling this situation as 0-D, there's no driving force for pressure waves, so when you reach $P_{1} = P_{2}$, stopping integration at that point should give you the correct result.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-12T21:16:24.217" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15840" PostTypeId="2" ParentId="15833" CreationDate="2014-10-12T21:38:13.783" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://wiki.scipy.org/Cookbook/Zombie_Apocalypse_ODEINT&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is a Python example using &lt;code&gt;scipy.integrate.odeint&lt;/code&gt;. It doesn't use a Runge-Kutta method, but by changing the &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; to &lt;code&gt;scipy.integrate.ode&lt;/code&gt; and using the &lt;code&gt;set_integrator&lt;/code&gt; method to set the integrator to “dopri5”, you can use a Runge-Kutta 4(5) method to solve your equations.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-12T21:38:13.783" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15841" PostTypeId="2" ParentId="14828" CreationDate="2014-10-13T04:34:06.010" Score="2" Body="&lt;p&gt;As the review article points out, there's no one best derivative-free optimizer for all problems, much like there's no one best nonlinear solver for finding roots of algebraic equations, or one best linear solver for solving linear equations, etc. Also, like linear solvers and nonlinear solvers, a certain amount of experimentation is required if you want a highly performant solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along that line of argument, surrogate modeling is something that's worth trying. You might first try the algorithms in &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt&quot; rel=&quot;nofollow&quot;&gt;NLOPT&lt;/a&gt;, written by Steven G. Johnson at MIT, which &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms#Local_derivative-free_optimization&quot; rel=&quot;nofollow&quot;&gt;contains a few derivative-free algorithms&lt;/a&gt; that use surrogate models (polynomial interpolants, basically). It sounds like you're using something similar to PRAXIS right now, so you could also try COBYLA, BOBYQA, and NEWUOA, and see how they work.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-13T04:34:06.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15842" PostTypeId="2" ParentId="15834" CreationDate="2014-10-13T09:38:46.347" Score="0" Body="&lt;p&gt;I thought of the following approach: Denoting $a=x^TQx$ we may define $y=x/a$ and thus $a=x^TQx$ becomes $a=1/y^TQy$ and we have a constraint of $Qy=v$ so that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_x x^TQx\ \ \text{s.t.}\ \ Qx=(x^TQx)\, v\ ,\ Ax\ge b$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;will become now:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max_y y^TQy\ \ \text{s.t.}\ \ Qy=v\ ,\ (A-bv^T)y\ge \vec{0}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where in the last them we use the fact that $1/a=y^TQy=v^Ty$ and thus we have inequality constraint $Ay\ge bv^Ty$ which can be written $(A-bv^T)y\ge \vec{0}$.&lt;/p&gt;&#xA;" OwnerUserId="10450" LastActivityDate="2014-10-13T09:38:46.347" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15843" PostTypeId="1" AcceptedAnswerId="15844" CreationDate="2014-10-13T10:25:36.003" Score="2" ViewCount="1174" Body="&lt;p&gt;The interested equation is advection-diffusion equation. One of the canonical example is Navier-Stokes equations. However, I would like to let the coefficient of diffusion constant goes to zero, $\epsilon \rightarrow 0$, while mesh sizes, $ \Delta x, \Delta dy \rightarrow 0 $, and time steps,$\Delta t \rightarrow 0$, decreasing towards zero. Then, I can obtain inviscid compressible equation which can be an example for the Euler equation of gas dynamics. The following equation is scalar advection-diffusion equation in 2D cartesian coordinates.&lt;br&gt;&#xA;$$ \frac{df}{dt} + U\frac{df}{dx} + V\frac{df}{dy} = \epsilon \left( \frac{d^2f}{dx^2} + \frac{d^2f}{dy^2} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main interest is to find exact formulation for the Euler equations of gas dynamics in cylindrical coordinates $(r, \theta)$ with artificial diffusion part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit :&lt;/strong&gt; The main equation that I am interested in the Euler equations of gas dynamics in 2D. The cartesian vector form can be stated as &#xA;$$\begin{pmatrix}&#xA;\rho \\&#xA;\rho u \\&#xA;\rho v \\&#xA;E&#xA;\end{pmatrix}_t + \begin{pmatrix}&#xA;\rho u \\&#xA;\rho u^2 + p \\&#xA;\rho u v \\&#xA;u(E + p)&#xA;\end{pmatrix}_x + \begin{pmatrix}&#xA;\rho v\\&#xA;\rho u v \\&#xA;\rho v^2 + p \\&#xA;v(E + p)&#xA;\end{pmatrix}_y = \textbf{0}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, conservation of mass, momentums in -x and -y directions and energy equations can be seen in a matrix form. However, I would like to add artificial viscosity with small constant factor as in Navier-Stokes equations to smoothen the possible oscillations that will occur during numerical simulation. For this reason, I need to know the vector Laplacian form in cylindrical coordinates for Navier-Stokes equations. Then I can apply the above theory that was stated.      &lt;/p&gt;&#xA;" OwnerUserId="11495" LastEditorUserId="11495" LastEditDate="2014-10-16T16:17:33.680" LastActivityDate="2014-10-16T16:17:33.680" Title="What is the exact formulation of compressible Euler equation of gas dynamics in polar coordinates with artificial diffusion in 2D?" Tags="|pde|computational-geometry|hyperbolic-pde|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="15844" PostTypeId="2" ParentId="15843" CreationDate="2014-10-13T10:42:19.680" Score="1" Body="&lt;p&gt;Have a look at Chapter 3 of the bachelor thesis &lt;a href=&quot;http://ta.twi.tudelft.nl/nw/users/mmbaumann/projects/Mathe_BA/matheba.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Modellierung und Simulation von&#xA;Dispersionen in turbulenten Strömungen&quot; by Manuel Baumann&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is in German but the equations and pictures can be understood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Chapter 3.1.1, the coordinate transformation that in particular affects the convection is explained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Chapter 3.2.1, the transformed &lt;em&gt;RANS&lt;/em&gt; equations are formulated. Set the right terms to zero and you get the Euler equations (equation (19) on page 29).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Euler equation, the main work will be the convective term that in polar coordinates writes as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4hp0v.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in polar coordinates, the coordinate unit vectors $e_\theta$, $e_r$ depend on the &lt;em&gt;angular coordinate&lt;/em&gt; $\theta$. This means, e.g.,&#xA;$$&#xA;\partial _\theta (U_\theta \cdot e_\theta) =  \partial _\theta (U_\theta) \cdot e_\theta - U_\theta \cdot e_r&#xA;$$&#xA;what makes the additional terms appear in the last equality of the equation above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a reference, I can recommend &quot;Mechanics of Fluid Flow&quot; by Longwell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I was supervising the thesis mentioned above.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2014-10-15T07:08:10.433" LastActivityDate="2014-10-15T07:08:10.433" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15845" PostTypeId="1" AcceptedAnswerId="15846" CreationDate="2014-10-13T12:52:51.007" Score="1" ViewCount="2734" Body="&lt;p&gt;I want to minimize function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x) = x^T \cdot A \cdot x + b \cdot x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;given constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B \cdot x = 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here: $x$ is a vector ($x \in \mathbb{R}^n$), $A$ is a matrix of size $n \times n$, $b$ is a vector of size $n$ and $B$ is a matrix of size $k \times n$ (we have $k$ constraints). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without constraint this problem is standard optimization problem. I also know that I can reduce this problem to quadratic programming (because $B \cdot x = 0$ is equivalent of  $B \cdot x \geq 0$ and $-B \cdot x \geq 0$). But I hope that this problem can be solved in a easier way.&lt;/p&gt;&#xA;" OwnerUserId="11496" LastActivityDate="2014-10-14T08:05:19.283" Title="Minimize quadratic form with equality constraints" Tags="|optimization|matrix|quadratic-programming|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15846" PostTypeId="2" ParentId="15845" CreationDate="2014-10-13T14:03:16.520" Score="4" Body="&lt;p&gt;Writing the Lagrangian gives rise to the following optimality conditions &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B x^* =0 $  and   $A x^* + b +A^T\lambda^*=0 $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rewritten in matrix-form we have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\underbrace{\begin{bmatrix} A&amp;amp;B^T \\B &amp;amp;0 \end{bmatrix}}_{=:K}\begin{bmatrix} x \\ \lambda\end{bmatrix} =\begin{bmatrix} -b \\ 0\end{bmatrix}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $K$ is non singular then you have a unique, optimal pair $(x^*,\lambda^*)$. Otherwise the quadratic optimization problem is unbounded below or infeasible.&lt;/p&gt;&#xA;" OwnerUserId="9378" LastActivityDate="2014-10-13T14:03:16.520" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15847" PostTypeId="1" CreationDate="2014-10-13T14:49:16.103" Score="3" ViewCount="3450" Body="&lt;p&gt;I am looking for library written in Python which will enable me to solve the coupled nonlinear equations which looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nUDC4.gif&quot; alt=&quot;formula&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need the library which will enable me to couple this solver to other models. For instance, I would like to impose different boundary conditions and change them in time (at any time step) as well as access to vector of solution at each time step and have possibility to change it (for instance, to implement non-local transport). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other &quot;wishes&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;fast&lt;/li&gt;&#xA;&lt;li&gt;To have possibility of parallel solving.&lt;/li&gt;&#xA;&lt;li&gt;and if possible easy to use.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="11497" LastActivityDate="2014-10-13T21:45:24.333" Title="Solving Advection (Convection) - Diffusion - Reaction Partial Differential Equation in Python" Tags="|pde|python|parallel-computing|libraries|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15848" PostTypeId="1" CreationDate="2014-10-13T19:09:19.533" Score="6" ViewCount="206" Body="&lt;p&gt;I am interested in knowing if there is a generalization of the Uzawa iteration for the linear problems of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[ \begin{array}{ccc}A&amp;amp; B^T&amp;amp;0\\&#xA;B&amp;amp;0&amp;amp;C^T\\&#xA;0&amp;amp;C&amp;amp;0 \end{array}&#xA; \right]\left[\begin{array}{c}x\\ y\\ z\end{array} \right] = \left[\begin{array}{c} f\\ g\\ h&#xA;\end{array} \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $A$, is symmetric and positive definite, and in my case it is diagonal so trivial to invert.  $A\in \mathbb{R}^{n\times n}$, $B\in \mathbb{R}^{m\times n}$, and $C\in \mathbb{R}^{l\times m}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $n \approx 3m $ and $l \ll m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some background,  I need to solve this system for every step in a time simulation.  I hope to use the previous time step's solution as a good guess for the new solution, perhaps after extrapolating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a way to solve this using a triple nested iteration, but I am interested in something like the Uzawa iteration because I feel I have a good starting guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated.  Do these matrices have a name?&lt;/p&gt;&#xA;" OwnerUserId="11501" LastActivityDate="2014-10-14T09:53:42.227" Title="Solving Generalization of Saddle point problem" Tags="|linear-algebra|linear-solver|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="15849" PostTypeId="2" ParentId="15845" CreationDate="2014-10-13T21:09:03.373" Score="1" Body="&lt;p&gt;Quadratic optimization problems with linear equality constraints are pretty simple to solve and a standard problem. Nicolas already gave the correct answer but I'd like to point out that you will find this problem discussed in every textbook on optimization. My favorite is the one by Nocedal and Wright, &quot;Numerical Optimization&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-13T21:09:03.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15850" PostTypeId="2" ParentId="15847" CreationDate="2014-10-13T21:45:24.333" Score="3" Body="&lt;p&gt;I suggest taking a look at FiPy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ctcms.nist.gov/fipy/&quot; rel=&quot;nofollow&quot;&gt;http://www.ctcms.nist.gov/fipy/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It uses the finite-volume method, is written in Python,&#xA;has certainly been used to solve the class of problems you describe, and was&#xA;designed with flexibility in mind. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation seems relatively good to me&#xA;and the authors are responsive to requests for help via a mailing list. &lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-10-13T21:45:24.333" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="15851" PostTypeId="1" AcceptedAnswerId="15858" CreationDate="2014-10-14T06:19:03.660" Score="6" ViewCount="485" Body="&lt;p&gt;I am attempting to solve the following ODE problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-u''+ u = x$$&#xA;$$u(0) = 0$$&#xA;$$u'(1) = -u(1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exact solution is: $u(x) = e^{-x-1} - e^{x-1} + x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a Dirichlet at $x = 0$ and a Robin condition at $x = 1$. I refer to the exact solution with &quot;little&quot; $u$, and the approximate solution with &quot;big&quot; $U_i$, where the subscript $i$ refers to &quot;point number&quot;: if I have $m$ sub-intervals, then I will have $m+1$ points indexed from $0$ to $m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to implement the Dirichlet condition, I eliminated $u_0$ from the finite difference matrix, and modified the right hand side, but in this case I didn't &lt;em&gt;really&lt;/em&gt; modify the RHS since $u(0) = 0$, thus $U_0 = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u''(0+h) \approx \frac{U_0 - 2U_1 + U_2}{h^2} = \frac{- 2U_1 + U_2}{h^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to implement the Robin condition, I decided to use the &quot;ghost point&quot; method along with a centred difference approximation of the first derivative ($U_m = u(1)$, if there are $m$ points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u'(1) = -u(1),\;\therefore u'(1) \approx \frac{U_{m+1} - U_{m-1}}{2h} = -U_m$$&#xA;$$U_{m+1} = U_{m-1} - 2hU_{m}$$&#xA;$$\therefore u''(1) \approx \frac{U_{m-1} - 2U_m + U_{m+1}}{h^2} = \frac{2U_{m-1} + (-2 - 2h)U_m}{h^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if I have 4 sub-intervals (5 points), where $U_0$ and $U_4$ are the correspond to $u(0)$ and $u(1)$ respectively, I should have the following matrix equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(\frac{-1}{h^2}\begin{bmatrix}-2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\ 1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0\\ 0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1\\ 0 &amp;amp; 0 &amp;amp; 2 &amp;amp; -2 - 2h \end{bmatrix} + \mathbf{I}\right)\begin{bmatrix}U_1 \\ U_2 \\ U_3 \\ U_4\end{bmatrix} = \begin{bmatrix}F_1 \\ F_2 \\ F_3 \\ F_4\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then implemented the above approximation using some simple Python code (you should be able to run it too), and I have included comments liberally to aid in readability:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;#Want to solve ODE: -u'' + u = x, u(0) = 0, u'(1) = -u(1)&#xA;#Actual solution: exp(-X-1)&#xA;# a and b mark the interval over which we want to solve our ODE&#xA;a = 0&#xA;b = 1&#xA;num_subintervals = 4&#xA;num_points = num_subintervals+1&#xA;&#xA;#size of each subinterval&#xA;h = (b - a)/num_subintervals&#xA;&#xA;X = np.linspace(a, b, num_points)&#xA;&#xA;def f(x):&#xA;    return x&#xA;vectorized_f = np.vectorize(f)&#xA;&#xA;# actual solution: -u'' + u = x, u(0) = 0, u'(1) = -u(1)&#xA;u = np.exp(-X-1) - np.exp(X-1) + X&#xA;&#xA;#Implementing Dirichlet condition at u(0)&#xA;F = vectorized_f(X)    &#xA;u_0 = 0&#xA;F[1] = F[1] - (u_0/(h**2))&#xA;#We have eliminated U_0 from our equations due to implementation of BCs&#xA;F = F[1:]&#xA;&#xA;#eliminated U_0, but U_n is still in the equation, thus num_points-1 to solve for&#xA;N = num_points-1&#xA;# -2 along main diagonal, 1 along +1 and -1 diagonal&#xA;D2 = np.diag((-2/(h**2))*np.ones(N), 0) + np.diag((1/(h**2))*np.ones(N-1), 1) + np.diag((1/(h**2))*np.ones(N-1), -1)&#xA;&#xA;#D1(U_n) = (U_n+1 - U_n-1)/2h = -U_1&#xA;# -2h*U_n + U_n-1 = U_n+1&#xA;# D2(U_n) = (U_n-1 - 2*U_n + U_n+1)/h**2 = (U_n-1 - 2*U_n - 2*h*U_n + U_n-1)/h**2&#xA;# D2(U_n) = (-2*U_n - 2*h*U_n + 2*U_n-1)/h**2&#xA;D2[N-1, N-2] = 2/(h**2) #Modifying U_n-1 coefficient (index N-2) in equation for U''_n&#xA;D2[N-1, N-1] = (-2 - 2*h)/(h**2) #Modifying U_n (index N-1) in equation for U''_n&#xA;&#xA;I = np.ones((num_points-1, num_points-1))&#xA;&#xA;#LHS&#xA;A = -1*D2 + I&#xA;&#xA;U = np.concatenate(([u_0], np.linalg.solve(A, F)))&#xA;&#xA;error = np.linalg.norm(U - u, ord=np.inf)&#xA;&#xA;print &quot;error: &quot;, error&#xA;plt.plot(X, U, label=&quot;approximate&quot;)&#xA;plt.plot(X, u, label=&quot;exact&quot;)&#xA;plt.legend(loc=&quot;best&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the following image: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fzcq2.png&quot; alt=&quot;approximate vs. exact&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the error does not decrease as I increase number of sub-intervals. So, clearly, I have an error, but I am not sure where. Could you help?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="9466" LastEditDate="2014-10-14T15:59:17.847" LastActivityDate="2014-10-14T15:59:46.027" Title="Error implementing Robin boundary conditions in toy ODE problem" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15852" PostTypeId="1" CreationDate="2014-10-14T07:23:17.170" Score="6" ViewCount="1252" Body="&lt;p&gt;Is there a test case for 3D incompressible Navier Stokes Equations like the &lt;a href=&quot;http://en.wikipedia.org/wiki/Taylor%E2%80%93Green_vortex&quot; rel=&quot;nofollow noreferrer&quot;&gt;Taylor vortex&lt;/a&gt; in two dimensions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know, I can easily construct 3D &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6943/manufactured-solutions-for-incompressible-navier-stokes-how-to-find-divergenc&quot;&gt;manufactured solutions&lt;/a&gt; but I would like to have something more physical.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-10-16T11:44:58.530" Title="Test Case with Known Solution for 3D Navier Stokes Equations" Tags="|fluid-dynamics|numerical-analysis|testing|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="15853" PostTypeId="2" ParentId="15845" CreationDate="2014-10-14T08:05:19.283" Score="5" Body="&lt;p&gt;At first, I thought that Nicolas' answer was right, and then I looked at the question again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a quadratic program (QP), $A$ is symmetric by convention, and it's possible to re-express it as such, so without loss of generality, suppose that $A$ is symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is also positive semidefinite, then a local optimum is a global optimum, and the KKT conditions that Nicolas cites are both necessary and sufficient for finding a global optimum because the constraints in this particular case are linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A counterexample to Nicolas' answer in the case where $A$ is positive semidefinite can be constructed by setting $b = 0$, and setting $B$ and $A$ such that the nullspace of $B$ corresponds to the eigenspace of $A$ with eigenvalue zero. For instance, set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A = \left[\begin{array}{cc}1 &amp;amp; 0 \\ 0 &amp;amp; 0\end{array}\right], \\&#xA;B = \left[\begin{array}{cc}1 &amp;amp; 0\end{array}\right],&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the problem becomes minimizing $x_{2}^{2}$ such that $x_{1} = 0$. The problem then has an infinite number of optimal solutions (any ordered pair such that $x_{2} = 0$), and the KKT matrix constructed by Nicolas has the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;K = \left[\begin{array}{ccc} 1 &amp;amp; 0 &amp;amp; 1 \\ 0 &amp;amp; 0 &amp;amp; 0 \\ 1 &amp;amp; 0 &amp;amp; 0\end{array}\right],&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K$ is obviously rank-deficient. (Trivial counterexamples also arise when $B$ is not full rank.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is not positive semidefinite, then the problem is no longer convex. The KKT conditions are still necessary, but not sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, QP solvers for the convex case are efficient and robust, so I would just use a library for that case. The nonconvex case is NP-hard, but there are still really good solvers available, such as CPLEX, which has free licenses for academics.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-14T08:05:19.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15855" PostTypeId="2" ParentId="15848" CreationDate="2014-10-14T09:39:10.370" Score="4" Body="&lt;p&gt;Looking at your equations, you see that $x=A^{-1}(f-B^Ty)$. Since $A$ is diagonal and invertible, you can easily transform your linear system by a block-Gauss elimination:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[ \begin{array}{cc}-BA^{-1}B^T&amp;amp; C^T\\&#xA;C&amp;amp;0 \end{array}&#xA; \right]\left[\begin{array}{c}y\\ z\end{array} \right] = \left[\begin{array}{c} g - A^{-1}f\\ h&#xA;\end{array} \right].$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is now only a problem in $y,z$, but $x$ can always be recovered via $x=A^{-1}(f-B^Ty)$ which is again cheap to compute. Now the problem is in saddle-point form, which allows the use of Uzawa-type techniques. Let us denote $S_1=BA^{-1}B^T$. Since $B$ has full column rank, the matrix $S_1$ is symmetric and positive definite. Using that $y = S_1^{-1}(C^Tz+A^{-1}f-g)$, we can now form a second Schur complement as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[ \begin{array}{cc}-S_1&amp;amp; C^T\\&#xA;0 &amp;amp; S_2 \end{array}&#xA; \right]\left[\begin{array}{c}y\\ z\end{array} \right] = \left[\begin{array}{c} g - A^{-1}f\\ h + S_1^{-1}(g-A^{-1}f)&#xA;\end{array} \right],$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $S_2 = CS_1^{-1}C^T$ which is as well symmetric and positive definite, since $C$ has full column rank. Now you can use your favorite solver to solve for $z$, then recover $y$ and finally $x$. Note that while $S_1$ can still be explicitly computed for practical purposes, $S_1^{-1}$ and therefore $S_2$ will probably be dense. Therefore it might be good to use a nested Krylov subspace method (e.g. conjugate gradients). In this way, both, the inner and outer iterations only require multiplications with the respective Schur-complements, thereby avoid their direct assembly. For instance an application of $S_2$ would be decomposed into applying $C^T$ then an (inner) solve with $S_1^{-1}$ and another application of $C$. The same can be done for $S_1$ if you don't want to explicitly compute it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me finally remark that in a practical implementation of Uzawa-type iterations it is possible to efficiently update $y$ alongside $z$ and thereby obtain an efficient algorithm. Also inexact Uzawa approaches and suitable preconditioners for your subproblems can possibly be considered.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2014-10-14T09:53:42.227" LastActivityDate="2014-10-14T09:53:42.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15856" PostTypeId="2" ParentId="15852" CreationDate="2014-10-14T11:47:41.860" Score="3" Body="&lt;p&gt;There are several benchmarks, like the flow around a cylinder, which is described in details at the FEATFLOW web page &lt;a href=&quot;http://www.featflow.de/en/benchmarks/cfdbenchmarking/flow.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. It is a well defined configuration of a flow passing by a cylinder obstacle and values such as the drag and lift can be compared with values obtained with different softwares (a file containing the data is provided).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise there are &quot;less physical&quot; flows, like the Ethier-Steinman problem described &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/fld.1650190502/abstract&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, which however has a known exact solution.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="6574" LastEditDate="2014-10-16T11:29:25.527" LastActivityDate="2014-10-16T11:29:25.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15857" PostTypeId="1" AcceptedAnswerId="15859" CreationDate="2014-10-14T14:12:24.990" Score="4" ViewCount="618" Body="&lt;p&gt;I am trying to optimize a constrained-problem with a discrete, non-linear objective function. Evaluating this function is also fairly expensive. Nevertheless, despite the above two factors, I hope, that it can still be solved  efficiently, since the structure of the constrained parameter space should be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more exact: The parameter space will in general be of dimension 4-150. The parameters lie on a n-simplex, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\sum_{i=1}^n p_i =1 \\&#xA;p_i \geq 0 \; \forall \; i =1,\dotsc,n&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is: Which algorithms could work best for solving such types of problems? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have tried variants of the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Constrain the space by $1-\varepsilon \leq \sum_{i=1}^n p_i \leq 1, \varepsilon &amp;gt;0$ and then apply an adaptive barrier method combined with the Nelder-Mead algorithm &lt;a href=&quot;http://stat.ethz.ch/R-manual/R-devel/library/stats/html/constrOptim.html&quot; rel=&quot;nofollow&quot;&gt;(R constrOptim function)&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apply unconstrained optimization in $\mathbb R^n$ by modifying the objective function, so that in the first step it normalizes the parameters appropriately.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Map the simplex to the unit sphere. Then perform unconstrained optimization using Nelder-Mead,the Subplex algorithm or the Covariance Matrix Adaptation Evolution Strategy (CMAES) algorithm based on the spherical coordinates.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So far spherical coordinates followed by CMAES shows the best results, but it is too slow. What else could I try?&lt;/p&gt;&#xA;" OwnerUserId="11508" LastActivityDate="2014-10-15T19:04:08.690" Title="Derivative-free nonlinear optimization of discrete objective function with linear constraints (simplex)" Tags="|constrained-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15858" PostTypeId="2" ParentId="15851" CreationDate="2014-10-14T15:59:46.027" Score="6" Body="&lt;p&gt;This is actually a very frustrating kind of bug. Change the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I = np.ones((num_points-1, num_points-1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I = np.eye(num_points-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you look at your matrix $A=-D_2+I$, it is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([[ 129.,  -63.,    1.,    1.,    1.,    1.,    1.,    1.],&#xA;       [ -63.,  129.,  -63.,    1.,    1.,    1.,    1.,    1.],&#xA;       [   1.,  -63.,  129.,  -63.,    1.,    1.,    1.,    1.],&#xA;       [   1.,    1.,  -63.,  129.,  -63.,    1.,    1.,    1.],&#xA;       [   1.,    1.,    1.,  -63.,  129.,  -63.,    1.,    1.],&#xA;       [   1.,    1.,    1.,    1.,  -63.,  129.,  -63.,    1.],&#xA;       [   1.,    1.,    1.,    1.,    1.,  -63.,  129.,  -63.],&#xA;       [   1.,    1.,    1.,    1.,    1.,    1., -127.,  145.]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is not what you want. You can catch these kinds of bugs by computing $AU-F$ and $Au-F$ and checking the residual.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-10-14T15:59:46.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15859" PostTypeId="2" ParentId="15857" CreationDate="2014-10-14T18:45:45.753" Score="3" Body="&lt;p&gt;Generally speaking, for derivative-free optimization, there's no one algorithm that works best for all problems, although there are some algorithms that tend to work better than others. The classic recent review reference for derivative-free optimization is by &lt;a href=&quot;http://thales.cheme.cmu.edu/dfo/comparison/dfo.pdf&quot; rel=&quot;nofollow&quot;&gt;Rios and Sahinidis&lt;/a&gt; (see also &lt;a href=&quot;http://egon.cheme.cmu.edu/ewocp/docs/SahinidisEWO_DFO2010.pdf&quot; rel=&quot;nofollow&quot;&gt;this related presentation by Sahinidis&lt;/a&gt;, and they suggest the &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/software/mcs/&quot; rel=&quot;nofollow&quot;&gt;MCS algorithm&lt;/a&gt;, the &lt;a href=&quot;http://www.pinterconsulting.com/&quot; rel=&quot;nofollow&quot;&gt;LGO algorithm (proprietary implementation)&lt;/a&gt;, and BOBYQA/NEWUOA (both of which are implemented by &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt&quot; rel=&quot;nofollow&quot;&gt;NLOPT&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Derivative-free methods tend not to incorporate general constraints. If you're using a method that does not permit your linear constraint, you can either normalize the parameters as you are doing right now, or you can use more of a barrier-type approach and penalize violation of the constraint with a penalty term and increase the penalty parameter over a number of solves.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-14T18:45:45.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15860" PostTypeId="1" CreationDate="2014-10-15T05:12:11.640" Score="2" ViewCount="194" Body="&lt;p&gt;I hope to get one smooth curve on one triangle mesh. I get one path on the mesh at first. The path consists of vertices of the mesh. I can see the path from the image below.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Jybbx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each one green dot indicates one vertex on the mesh. The method that I am using is to fit one smooth curve using the vertices and project the smooth curve on the mesh. To project the smooth curve, I find the closest point on the mesh for each vertex of the smooth curve.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/YL0OC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the curve is not on the mesh because it intersects with the mesh. I realize that the vertices need to move along the edge of the triangle shared with the previous vertex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my view, there are two kinds of methods to produce the smooth curve. The first kind is to move the points of the curve along the mesh to make the curve smooth. The second kind is to fit the points to obtain one smooth curve and project the smooth curve on the mesh,but the projection should be complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you give me some advices or links to websites or papers? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="7846" LastActivityDate="2014-10-15T05:12:11.640" Title="Produce one smooth curve on one triangle mesh" Tags="|mesh|smoothing|curve-fitting|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="15861" PostTypeId="1" AcceptedAnswerId="15877" CreationDate="2014-10-15T07:16:04.677" Score="15" ViewCount="663" Body="&lt;p&gt;I came across a puzzling remark in the paper&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://people.cs.vt.edu/~asandu/Public/Qual2011/DiffEqn/vanderHouwen_1986_RK_for_PDE.pdf&quot;&gt;P. J. van der Houwen, The development of Runge-Kutta methods for partial differential equations, Appl. Num. Math. 20:261, 1996&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On lines 8ff on page 264, van der Houwen writes:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;For the Taylor polynomials this implies that the imaginary stability interval is empty for $p = 1, 2, 5, 6, 9, 10, \cdots$&quot; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where &lt;em&gt;Taylor polynomial&lt;/em&gt; refers to the stability polynomial (truncated expansion of $\exp(x)$ around $x=0$) of the Runge-Kutta method and p is the order (see page 263). I assume that I misunderstand something because the fifth-order Runge-Kutta method does not have an empty imaginary stability interval as far as I know. From what I remember, the imaginary limit is about 3.4 or so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's my misunderstanding?  &lt;/p&gt;&#xA;" OwnerUserId="1792" LastEditorUserId="1792" LastEditDate="2014-10-15T09:50:46.133" LastActivityDate="2014-10-16T10:13:19.123" Title="Puzzling remark about stability region of fifth-order Runge-Kutta method" Tags="|numerical-analysis|stability|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15862" PostTypeId="2" ParentId="14739" CreationDate="2014-10-15T11:24:09.540" Score="4" Body="&lt;p&gt;Jed Brown described the traditional approach used in the mesoscale and larger scale models. Actually, in microscale many atmospheric models are very close to to traditional CFD codes, use similar finite volume discretizations, similar 3D grids where vertical is treated similarly as horizontal, and so on. Depending on the resolutions even features like buildings are resolved with the same approaches known from engineering CFD, like the immersed boundary methods or body fitted grids. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can encounter all the discretization techniques you know from the engineering CFD, like finite differences, finite volumes, pseudo-spectral and even finite elements. The same pressure correction (fractional-step) methods are often used to solve the incompressible Navier-Stokes equations (with the Boussinesq or anelastic terms for buoyancy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, different parametrization for the heat and momentum fluxes near the surface are commonly used, taking into account the specifics of the land-surface interactions like the Monin-Obukhov similarity or other semi-empirical relations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole method of large-eddy simulation (LES), now very popular in engineering, actually originates in boundary layer meteorology. I would even say that many atmospheric modelers at this scale wouldn't hesitate at all to call their work CFD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many (but not all) applications you also have to add the Coriolis force. The schemes do not have to be well-balanced however, it is just one additional volume force. If you also compute the processes like cloud formation, precipitation and radiation, things get more complicated, but the same holds for engineering models which solve reaction kinetics, combustion and similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This class of models also includes those accounting for the ocean-atmosphere interactions you asked for, see for example &lt;a href=&quot;https://ams.confex.com/ams/pdfpapers/172658.pdf&quot; rel=&quot;nofollow&quot;&gt;https://ams.confex.com/ams/pdfpapers/172658.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1244" LastEditorUserId="1244" LastEditDate="2014-10-15T12:18:31.110" LastActivityDate="2014-10-15T12:18:31.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15863" PostTypeId="2" ParentId="15857" CreationDate="2014-10-15T12:38:34.577" Score="2" Body="&lt;p&gt;In addition to Geoff's answer, you can also just eliminate the linear constraint. The &quot;mollification&quot; $1-\varepsilon \le \sum_i p_i \le 1$ really creates more problems than it solves because it leaves you optimizing in an $n$ dimensional domain that is, however, very narrow in one direction. A better option is to just replace $p_n= 1 - \sum_{i=1}^{n-1} p_i$ everywhere and add the constraints $\sum_{i=1}^{n-1} p_i \le 1$ to your list of inequality constraints. This way, you can simply express everything in terms of just $n-1$ variables. In essence, you are now optimizing over the &lt;em&gt;interior&lt;/em&gt; of an $n-1$ simplex.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="276" LastEditDate="2014-10-15T19:04:08.690" LastActivityDate="2014-10-15T19:04:08.690" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15864" PostTypeId="1" AcceptedAnswerId="15871" CreationDate="2014-10-15T13:44:30.610" Score="0" ViewCount="1451" Body="&lt;p&gt;I am assuming the pardiso solver calculates (or estimates) the condition number before proceeding to the solution phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to make pardiso output the condition number? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, minimum and maximum eigenvalue are of course also okay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can also output the matrix in matrix-market format and do the calculation with e.g. scipy, but that operation is rather unpleasant for large matrices.&lt;/p&gt;&#xA;" OwnerUserId="86" LastEditorUserId="86" LastEditDate="2014-10-15T14:11:30.883" LastActivityDate="2014-10-15T20:06:51.617" Title="Is it possible to output the matrix condition number from pardiso (MKL)?" Tags="|linear-solver|intel-mkl|" AnswerCount="1" CommentCount="7" ClosedDate="2014-10-15T20:15:41.973" ContentLicense="CC BY-SA 3.0" />
  <row Id="15866" PostTypeId="1" CreationDate="2014-10-15T15:27:15.187" Score="1" ViewCount="172" Body="&lt;p&gt;In my (admittedly brief) experience, there have been instances in which the code got uglier because []*[] == [], and it would be possible to write more elegant code if it was defined to be 0 instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example of such a piece of code, this part is supposed to compute x in L*x=b (L being a lower triangular matrix):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x(1) = b(1)/A(1,1);&#xA;for i=2:n&#xA;   x(i) = (b(i) - A(i,1:i-1)*x(1:i-1))/A(i,i);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is that I have to make it by two separate cases. I do not think I am obsessed with code beauty, but this is not pleasant for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is there a way to make this work in a single for loop (not using isempty)?&#xA;Also, I'd love to know if there is a reason I am unaware of for this particular choice the developers made.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="11512" LastActivityDate="2014-10-15T20:48:41.053" Title="matlab: []*[] == [] makes vectorized code difficult" Tags="|matlab|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15867" PostTypeId="1" CreationDate="2014-10-15T17:19:08.050" Score="5" ViewCount="1116" Body="&lt;p&gt;I'm having difficulties patching together from what I read about obtaining the condition number of a real, symmetric, positive definite sparse matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my code, I found that there is &lt;em&gt;incomplete Cholesky factorization&lt;/em&gt; routine (used as preconditioner I suppose). I also have a &lt;em&gt;ILU&lt;/em&gt; preconditioner which I could use as &lt;em&gt;Wolfgang Bangerth&lt;/em&gt; recommended in an &lt;a href=&quot;https://scicomp.stackexchange.com/questions/15864/is-it-possible-to-output-the-matrix-condition-number-from-pardiso-mkl&quot;&gt;ill-posed version&lt;/a&gt; of this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I can use the Cholesky factorization to &lt;em&gt;estimate&lt;/em&gt; the condition number, but I'm not sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Result of the preconditioner is $U = L^T$ of my CRS matrix. To estimate the condition number, I need the ratio of maximum and minimum eigenvalues. Using this would be very quick to code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how can I use $U$ to get the eigenvalues?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second best approach I've see so far is writing my matrix to a file and using &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.sparse.linalg.lsmr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.sparse.linalg.lsmr&lt;/a&gt; to solve the  least-squares problems. This also outputs the condition number, but is a little overkill and much more work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please understand that matrices and linear algebra are not my daily occupation :)&lt;/p&gt;&#xA;" OwnerUserId="86" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-10-17T08:19:39.777" Title="Condition number from incomplete Cholesky factorization" Tags="|sparse-matrix|eigenvalues|condition-number|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="15868" PostTypeId="1" AcceptedAnswerId="15869" CreationDate="2014-10-15T18:15:14.570" Score="1" ViewCount="241" Body="&lt;p&gt;I am optimizing a function of 10-20 variables by running algorithm such as BOBYQA and a few other derivative-free algorithms. The bad news is that each function evaluation is very expensive, approx 30 min of computation. The good news is that I start in the ballpark of the global minimum, and just a few moves in the &quot;right direction&quot; should suffice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is: I don't know how many function evaluation is required to start moving in the &quot;right direction&quot;. My naive estimate is something like $n^2+n+1$: you've got to evaluate at initial point plus $2n$ points for 1st and 2nd derivatives in each of the $n$ directions plus $n(n-1)$ to evaluate mixed 2nd derivatives, and then you can make the 1st fairly confident step in the &quot;right direction&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the above estimate, which is quadratic in the number of variables, seems excessive. I doubt one really needs that much info to start going toward local minimum. Thus the question: Given a well behaved problem in a general vicinity of the global minimum, after how many iterations would derivative-free optimization methods (such as Powell's ones) are expected to start moving toward local minimum for a function of $n$ variables?&lt;/p&gt;&#xA;" OwnerUserId="5281" LastActivityDate="2014-10-15T18:43:54.210" Title="How does the number of iteration until optimization begins depends on the dimension of the problem?" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15869" PostTypeId="2" ParentId="15868" CreationDate="2014-10-15T18:43:54.210" Score="3" Body="&lt;p&gt;Your estimate refers more to the number of function evaluations required to build a quadratic surrogate model for BOBYQA. Derivative-free methods that build surrogate models have to take a certain number of function evaluations to construct the surrogate model in order to do optimization. This number of function evaluations is the minimum number needed to take one step. In some surrogate modeling methods, it is possible to reuse some function evaluations and augment those with a number of new function evaluations to construct a new surrogate model. No such reuse is possible when initializing the algorithm, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.optimization-online.org/DB_FILE/2008/05/1989.pdf&quot; rel=&quot;nofollow&quot;&gt;ORBIT&lt;/a&gt;, which is based on using radial basis function surrogate models, requires fewer evaluations to construct a surrogate model (it's linear in the number of state variables). I don't know of a publicly available implementation. Part of the reason Powell's methods tend to be recommended is because library-quality implementations are available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to optimization iterations -- not function evaluations -- it is possible under certain assumptions to bound the number of iterations required to reach $\varepsilon$-optimality. There's a famous proof by Kantorovich that does so for Newton's method under some assumptions. I don't know of a result specifically for derivative-free optimization. We do know, for instance, that nonconvex optimization is NP-hard, so I would expect that any such result would require convexity, probably along with a number of technical conditions (like Kantorovich's proof, and Nesterov and Nemirovskii's result for interior point methods for convex nonlinear programs).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-15T18:43:54.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15870" PostTypeId="2" ParentId="15866" CreationDate="2014-10-15T19:08:37.733" Score="6" Body="&lt;p&gt;One option would be to initialize &lt;code&gt;x = zeros(n,1)&lt;/code&gt; before you start your loop which should then go from &lt;code&gt;1&lt;/code&gt; to &lt;code&gt;n&lt;/code&gt;. In that case, the first time through the loop evaluates the term &lt;code&gt;A(1,1:0)*x(1:0)&lt;/code&gt; would mean multiplying a &lt;code&gt;1x0&lt;/code&gt; matrix by a &lt;code&gt;0x1&lt;/code&gt; which MATLAB evaluates as &lt;code&gt;0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is relying on (in my opinion) a very non-intuitive behavior and I'm not sure the perceived benefit of &quot;beauty&quot; is worth the increased level of opacity in what's happening. I would file this under ' &quot;optimizations&quot; that are likely to bite you later'. On another note, you should probably be initializing &lt;code&gt;x&lt;/code&gt; anyway so MATLAB doesn't have to keep growing the array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following code should work as expected:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = zeros(n,1);&#xA;for i=1:n&#xA;    x(i) = (b(i) - L(i,1:i-1)*x(1:i-1))/L(i,i);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;br&gt;&#xA;Upon thinking about this a bit more. This behavior likely translates to many similar situations in MATLAB which is probably why the designers chose to make matrix multiplication a valid operation for &lt;em&gt;any&lt;/em&gt; conformable matrices, even if the some of the array dimensions are zero. By defining the result of this matrix multiplication to be an appropriately sized array of zeros when the inner dimensions are both zero it is likely that the code just works as expected, even if the programmer doesn't think about the details of this corner case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm still not sure this a great thing to rely on. It will likely confuse people and it will almost certainly fail if ported to other languages.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-10-15T20:48:41.053" LastActivityDate="2014-10-15T20:48:41.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15871" PostTypeId="2" ParentId="15864" CreationDate="2014-10-15T20:06:51.617" Score="1" Body="&lt;p&gt;Computing the condition number of a matrix is an expensive operation (for example, involving computing the largest and smallest magnitude eigenvalues, or solving several(!) linear systems), so doing this by default in a call to a linear solver would be very wasteful. It makes much more sense to just try to solve the system, and in case the solver breaks down due to ill-conditioning, catch that error and terminate the routine gracefully (with a helpful error message warning of ill-conditioning). In addition, some iterative solvers (most notably GMRES) produce an estimate of the condition number as a by-product. Since PARDISO is a parallel direct solver based on LU or Cholesky decomposition, this is not available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the rare cases when one wants to know the condition number without solving a linear system, dedicated routines exist. For example, LAPACK (which is bundled in Intel's MKL) provides the &lt;code&gt;???con&lt;/code&gt; routines (see &lt;a href=&quot;https://software.intel.com/en-us/node/520910&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.intel.com/en-us/node/520910&lt;/a&gt;) for dense matrices. Routines for estimating (computing would be infeasible) the condition number of large sparse matrices (based on Krylov methods for computing approximate eigenvalues) also exist, but are not bundled in the MKL. If this is what you need to do, you have to look at one of the dedicated packages for sparse linear algebra such as PETSc (see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/34&quot;&gt;How can I estimate the condition number of a large sparse matrix using PETSc?&lt;/a&gt;) or Trilinos (look for &lt;a href=&quot;http://trilinos.sandia.gov/packages/ifpack/IfpackUserGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;IFPACK&lt;/a&gt; in the documentation).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-10-15T20:06:51.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15872" PostTypeId="2" ParentId="15867" CreationDate="2014-10-15T20:10:41.283" Score="3" Body="&lt;p&gt;Estimates can be obtained from Krylov subspace methods, such as GMRES, and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/34/how-can-i-estimate-the-condition-number-of-a-large-sparse-matrix-using-petsc&quot;&gt;PETSc has functionality for that&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a symmetric matrix, the 2-norm condition number is the spectral condition number, and there exist algorithms to estimate the 2-norm condition number, so if you really do want the condition number for your unpreconditioned matrix, you can use methods for estimating the 2-norm condition number as well. The same methods will work for your preconditioned matrix, although if it is not symmetric, then the 2-norm and spectral condition numbers will not coincide. Note also that condition numbers are norm-dependent, so you should really ascertain the condition number in the norm you're using to assess convergence.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-10-16T18:18:52.520" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15874" PostTypeId="1" CreationDate="2014-10-16T00:59:52.700" Score="3" ViewCount="1177" Body="&lt;p&gt;I am trying to use the MPI share memory feature. I have several SMP nodes, and each of them has four cores. I need an array of size N for each node that should be accessed by all four cores in each node. My plan is to construct a shared window of size N/4 using MPI_Win_allocate_shared, and I expect that memory usage of each node would be N. In the example below, N is 4X10^9 bytes, but the memory usage of each node is not 4GB but 16GB. Am I missing something?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;mpi.h&amp;gt;&#xA;&#xA;int main(int argc, char** argv) {&#xA;   MPI_Init(&amp;amp;argc, &amp;amp;argv);&#xA;&#xA;   int rank_all;&#xA;   int rank_sm;&#xA;   int size_sm;&#xA;&#xA;   // all communicator&#xA;   MPI_Comm comm_sm;&#xA;   MPI_Comm_rank(MPI_COMM_WORLD, &amp;amp;rank_all);&#xA;&#xA;   // shared memory communicator&#xA;   MPI_Comm_split_type(MPI_COMM_WORLD, MPI_COMM_TYPE_SHARED, 0, MPI_INFO_NULL, &amp;amp;comm_sm);&#xA;   MPI_Comm_rank(comm_sm, &amp;amp;rank_sm);&#xA;   MPI_Comm_size(comm_sm, &amp;amp;size_sm);&#xA;&#xA;   std::size_t local_window_count(1000000000);&#xA;&#xA;   char* base_ptr;&#xA;   MPI_Win win_sm;&#xA;   int disp_unit(sizeof(char));&#xA;   MPI_Win_allocate_shared(local_window_count * disp_unit, disp_unit, MPI_INFO_NULL, comm_sm, &amp;amp;base_ptr, &amp;amp;win_sm);&#xA;&#xA;   // write&#xA;   char buffer;&#xA;   if (rank_sm == 0) {&#xA;      buffer = 'A';&#xA;   }&#xA;   else if (rank_sm == 1) {&#xA;      buffer = 'C';&#xA;   }&#xA;   else if (rank_sm == 2) {&#xA;      buffer = 'G';&#xA;   }&#xA;   else {&#xA;      buffer = 'T';&#xA;   }&#xA;&#xA;   MPI_Win_fence(0, win_sm);&#xA;&#xA;   for (std::size_t it = 0; it &amp;lt; local_window_count; it++) {&#xA;      base_ptr[it] = buffer;&#xA;   }&#xA;&#xA;   MPI_Win_fence(0, win_sm);&#xA;&#xA;   // read&#xA;   long long int index_start(-1 * rank_sm * local_window_count);&#xA;   long long int index_end((size_sm - rank_sm) * local_window_count - 1);&#xA;&#xA;   for (long long int it_rel = index_start; it_rel &amp;lt; index_end; it_rel++) {&#xA;      buffer = base_ptr[it_rel];&#xA;&#xA;      if (it_rel == index_start) {&#xA;         std::cout &amp;lt;&amp;lt; rank_sm &amp;lt;&amp;lt; &quot; start: &quot; &amp;lt;&amp;lt; buffer &amp;lt;&amp;lt; std::endl;&#xA;      }&#xA;      else if (it_rel == (index_end - 1)) {&#xA;         std::cout &amp;lt;&amp;lt; rank_sm &amp;lt;&amp;lt; &quot; end: &quot; &amp;lt;&amp;lt; buffer &amp;lt;&amp;lt; std::endl;&#xA;      }&#xA;   }&#xA;&#xA;   MPI_Finalize();&#xA;&#xA;   return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11515" LastActivityDate="2018-05-15T04:08:34.350" Title="total memory usage of MPI shared memory" Tags="|c++|mpi|memory-management|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="15876" PostTypeId="1" AcceptedAnswerId="15889" CreationDate="2014-10-16T02:51:17.020" Score="1" ViewCount="155" Body="&lt;p&gt;For a personal project, I am attempting to write a fairly realistic collision simulator (for relatively large objects, not quantum stuff). As I was consulting my physics textbook and various online resources such as &lt;a href=&quot;http://hyperphysics.phy-astr.gsu.edu/hbase/impcal.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this hyperphysics article&lt;/a&gt;, &lt;a href=&quot;http://hyperphysics.phy-astr.gsu.edu/hbase/flobi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;or this one&lt;/a&gt;, it appears that all methods for determining collision force (or force/area) that I've found rely to some degree on conservation principles, which require some knowledge of post-collision parameters (such as distance traveled after impact, post-collision vectors, time of collision, etc.).&lt;/p&gt;&#xA;&lt;p&gt;If I have absolutely no knowledge of any post-collision parameters, but I have theoretical access to any and all physical, material and motion properties of the colliding objects:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a way I can determine (or at least estimate) the force exerted on each body as a result of the collision?&lt;/li&gt;&#xA;&lt;li&gt;What properties of the objects do I need to know?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In case anyone has any additional insights/criticisms/suggestions about my methods, I plan to use the collision force to determine whether yield strengths of either material are exceeded, which can help me determine what kind of collision results (elastic/inelastic), whether penetration occurs, breaking, shattering, etc.&lt;/p&gt;&#xA;&lt;h2&gt;EDIT:&lt;/h2&gt;&#xA;&lt;p&gt;What If I add the assumption that &lt;em&gt;all&lt;/em&gt; colliding objects consist of a single material? i.e. no compound objects (such as cars) with multiple sub-components of various compositions.&lt;/p&gt;&#xA;" OwnerUserId="11517" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-11-15T22:20:27.417" Title="How to calculate collision force with no future knowledge" Tags="|computational-physics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15877" PostTypeId="2" ParentId="15861" CreationDate="2014-10-16T05:19:45.897" Score="22" Body="&lt;p&gt;van der Houwen's statement is correct, but it is not a statement about all fifth-order Runge-Kutta methods.  The &quot;Taylor polynomials&quot; he is referring to are (as you seem to know) just the polynomials of degree $p$ that approximate $\exp(z)$ to order $p$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P_p(z) = \sum_{j=1}^p \frac{z^j}{j!}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the fifth-order polynomial, it turns out that $|P_5(i\epsilon)|&amp;gt;1$ for small $\epsilon$, so &lt;strong&gt;the stability region of a method having $P_5(z)$ as its stability polynomial does not include any neighborhood of the origin on the imaginary axis&lt;/strong&gt;.  That is, in precise terms, what van der Houwen says.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most likely source of your confusion is what is meant by &quot;the fifth-order Runge-Kutta method&quot;.  There are (infinitely) many fifth-order Runge-Kutta methods, but the most well-known ones do not have $P_5(z)$ as their stability polynomial.  Why?  &lt;strong&gt;As &lt;a href=&quot;http://journals.cambridge.org/action/displayAbstract?fromPage=online&amp;amp;aid=4910144&amp;amp;fileId=S1446788700023387&quot; rel=&quot;noreferrer&quot;&gt;John Butcher famously proved&lt;/a&gt;, a fifth-order Runge-Kutta method must have at least six stages&lt;/strong&gt;.  Usually, the stability polynomial of a method with six (or more) stages would have degree six (or more).  For instance, each of the fifth-order methods listed &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods&quot; rel=&quot;noreferrer&quot;&gt;on this Wikipedia page&lt;/a&gt; uses exactly six stages and has a stability polynomial of degree six.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible for a fifth-order method to have $P_5(z)$ as its stability polynomial?  Yes; a fifth-order explicit extrapolation method (like the well-known ones reviewed in &lt;a href=&quot;http://msp.org/camcos/2014/9-2/p01.xhtml&quot; rel=&quot;noreferrer&quot;&gt;this paper of mine&lt;/a&gt;) would do so.  Note also that a $p$-stage Runge-Kutta method with stability polynomial $P_5(z)$ will be accurate to order 5 for linear ODEs, though not for nonlinear ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, it is easy to make mistakes when determining the extent of the imaginary stability interval for high-order Runge-Kutta methods.  That is because &lt;strong&gt;the boundary of the stability region for such methods lies extremely close to the imaginary axis&lt;/strong&gt;.  Therefore, roundoff errors can lead to incorrect conclusions; only exact calculations should be used (of course, the relevance of the stability region boundary for practical purposes under these circumstances could certainly be debated).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, here is a plot of the stability region of the fifth-order method from the Fehlberg 5(4) pair:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/UiFTk.png&quot; alt=&quot;Fehlberg stability region&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The imaginary stability interval is empty, but you can't tell from the picture at this resolution!  Note that the region clearly includes part of the imaginary axis, but no interval about the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, here is the plot for the fifth-order method from the Dormand-Prince 5(4) pair:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/phXG1.png&quot; alt=&quot;DP5 stability region&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has imaginary stability interval approximately $[-1,1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a precise characterization of the stability region boundary near the imaginary axis for $P_p(z)$ (which is rather fascinating!), see &lt;a href=&quot;http://imajna.oxfordjournals.org/content/early/2014/09/15/imanum.dru039&quot; rel=&quot;noreferrer&quot;&gt;my recent paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also be interested in the &lt;a href=&quot;https://github.com/ketch/nodepy&quot; rel=&quot;noreferrer&quot;&gt;NodePy package&lt;/a&gt;, which produced the plots above and which can be used to accurately determine things like the imaginary stability interval of a method (disclaimer: I created NodePy).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-10-16T10:13:19.123" LastActivityDate="2014-10-16T10:13:19.123" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15878" PostTypeId="2" ParentId="15852" CreationDate="2014-10-16T08:45:15.877" Score="1" Body="&lt;p&gt;Let me add, the cavity flow in three dimensions. A very good reference is R. Glowinski, Handbook of Numerical Analysis, Numerical Methods for Fluids, Chapter 9, Section 44.3. Here the test is very well detailed.&lt;/p&gt;&#xA;" OwnerUserId="6574" LastActivityDate="2014-10-16T08:45:15.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15881" PostTypeId="2" ParentId="15852" CreationDate="2014-10-16T11:44:58.530" Score="4" Body="&lt;p&gt;Chapter 7 of the (free in PDF form) book &quot;&lt;a href=&quot;http://cfdbooks.com/cfdbooks.html&quot; rel=&quot;nofollow&quot;&gt;I do like CFD, Vol. 1&lt;/a&gt;&quot; collects many different known solutions for multidimensional flows that can be used for benchmarking.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastActivityDate="2014-10-16T11:44:58.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15882" PostTypeId="1" CreationDate="2014-10-16T14:06:40.963" Score="3" ViewCount="4569" Body="&lt;p&gt;I'm trying to study with a finite difference method the 2D advection equation with a space-dependant flow.&#xA;Taking a function $f(x,y,t)$ solution of the equation :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\partial_t\,f+\nabla(\textbf{v}\,f)=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\textbf{v}$ is known to be divergence free : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textbf{v}=a&#xA;\begin{bmatrix}&#xA;\,y \\ x&#xA;\end{bmatrix}&#xA;$$&#xA;where $a$ is a given constant. Finally, the equation to study take the form : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_t\,f+a\left[y\,\partial_x\,f+x\,\partial_y\,f\right]=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a first try, I tried to implement a very naive centred explicite scheme with the discretization $x=q\,\Delta_x$, $y=p\,\Delta_y$, $t=n\,\Delta t$ and $\; f(x,y,t)=f^n_{qp}\;$ where $\;(q,p)\in\left[2,K-1\right]^2\;$ and $\;n\in\left[1,N\right]$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f^{n+1}_{qp}=f^n_{qp}+\frac{a \Delta t}{2}\left[p\,(f^n_{q+1\,p}-f^n_{q-1\,p})+q\,(f^n_{q\,p+1}-f^n_{q\,p-1})\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we took $\Delta_x=\Delta_y=h$ for simplicity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fixed boundary conditions as zero-flux condition, I don't know if it's the right thing to do since the problem I'm studying should not have boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to perform a Von Neumann stability analysis for this scheme but it's seems to be simply unstable in any condition. But the fact that the scheme does not depend on $h$ puzzles me, I think it's going nowhere...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computational sciences are quite new for me, and I was wondering if I was going right in my reflections :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Should I rather try an implicite method or a Crank–Nicolson method? Should it be better in terms of stability?&lt;/li&gt;&#xA;&lt;li&gt;Concerning boundary conditions, since my system shouldn't have any boundary, should I try to implement absorbing boundary conditions?&lt;/li&gt;&#xA;&lt;li&gt;Is there any general method(s) to numerically study this kind of equations, for instance in higher dimensions, or with additional terms (like diffusion terms)?&lt;/li&gt;&#xA;&lt;li&gt;Is there any litterature that could help me in this study?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10480" LastActivityDate="2014-10-17T03:44:35.687" Title="Finite differences scheme for 2D advection equation" Tags="|finite-difference|advection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15883" PostTypeId="1" CreationDate="2014-10-16T14:32:42.107" Score="3" ViewCount="1415" Body="&lt;p&gt;I'ld like to calculate the PV of an integral with the form&#xA;$$ \tilde{G}_l(\omega) = -\frac{2\omega}{\pi} PV\int_0^\infty \frac{\tilde{G}_d(\omega^\prime)}{\omega^2 - {\omega^\prime}^2}d\omega^\prime$$&#xA;in MATLAB.  It is not obvious to me how to do this.  I've tried  using factorization to change the power in the denominator to allow for use with the Hilbert function in MATLAB, but that hasn't worked.  Any insight would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{G}_d(\omega)$ is an even function so I feel like I could do a contour integration and pick up the residue in the right-half plane, but I would rather do it numerically, as $\tilde{G}_d(\omega)$ exists more completely as a numerical function than an analytic one.&lt;/p&gt;&#xA;" OwnerUserId="7257" LastEditorUserId="7257" LastEditDate="2014-10-16T14:43:30.720" LastActivityDate="2014-10-16T18:47:30.613" Title="Numerical Principal Value Integration - Hilbert like" Tags="|quadrature|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15884" PostTypeId="1" CreationDate="2014-10-16T16:04:11.603" Score="4" ViewCount="141" Body="&lt;p&gt;I don't have access to a computer cluster in my university. Is there website that accepts applications for free access to a computer cluster for scientific computing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further information: I am in Cuba. I need something on the order of 30 processors, about 50 GB of hard drive, for a week or so. I need to run programs written in C++ and Python.&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2014-10-17T12:03:51.237" LastActivityDate="2014-10-17T12:03:51.237" Title="Apply for a cluster for scientific computing from a developing country?" Tags="|hpc|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15885" PostTypeId="2" ParentId="15884" CreationDate="2014-10-16T16:22:04.277" Score="1" Body="&lt;p&gt;For access to machines in the US, your best bet is to work with a colleague at a US institution. NSF's XSEDE machines accept users from all over the world (persons physically in Iran, Cuba, Syria, Sudan, and North Korea, excepted for sure), but Principal Investigators for allocations must be at a US institution. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-10-16T16:22:04.277" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15886" PostTypeId="1" CreationDate="2014-10-16T16:23:58.770" Score="0" ViewCount="254" Body="&lt;p&gt;I am a newbie in using LAPACK library. I know that LAPACK's internal rountines break the large problem into smaller problems recursively (I am considering level-3 BLAS). If we consider matrix multiplication C= AB+C  example, until what size ( maximum and minimum) we can divide the bigger matrices? &#xA;(128 x 128) would be the smallest size? &lt;/p&gt;&#xA;" OwnerUserId="11524" LastActivityDate="2014-10-16T22:51:26.510" Title="Matrix size LAPACK can support with level-3 BLAS" Tags="|lapack|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15887" PostTypeId="2" ParentId="15886" CreationDate="2014-10-16T18:31:12.700" Score="0" Body="&lt;p&gt;Matrix multiplication in LAPACK (DGEMM) isn't recursive, at least, not in the &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d7/d2b/dgemm_8f_source.html&quot; rel=&quot;nofollow&quot;&gt;Netlib implementation&lt;/a&gt;. It implements matrix multiplication as a triply-nested loop.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-16T18:31:12.700" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15888" PostTypeId="2" ParentId="15883" CreationDate="2014-10-16T18:42:03.633" Score="2" Body="&lt;p&gt;Away from the singular point the standard quadrature methods should be fine, so the problem is really about computing an integral of the form&#xA;$$ \mathrm{P.V.}\int_{-h}^{h} \frac{f(x)}{x}\,dx. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest trying the following things:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know a closed-form Taylor series for $f(x)=G(\omega+x)$, the you can choose $h$ to small enough for the series approximation to be accurate, and calculate the principal value in closed form for each series term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can try to interpolate $f$ numerically, using something like &lt;code&gt;polyfit&lt;/code&gt;, which will give you the Taylor series numerically. Depending on how well-behaved your function is, it is difficult to say from the start whether any of this will definitely work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still the simplest way (although numerically unstable) is to write the integral as&#xA;$$ \int_0^h \frac{f(x)-f(-x)}{x}\,dx, $$&#xA;and apply the standard quadrature algorithm to this. The issue is that the calculation $f(x)-f(-x)$ will lead to inaccuracies on the order of $\int_0^h (\epsilon_{\mathrm{mach}}/x)\,dx$, which may or may not be tolerable. If you have a way to simplify the expression $f(x)-f(-x)$ to make it numerically stable, this is probably the best option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are ways to derive quadrature nodes and weights for computing principal values of such integrals (just search for them, even GSL has them), but I couldn't find this in matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are numerical methods for principal value integrals on the real line. You can also try to compute the integral as&#xA;$$ \mathrm{P.V.}\int_{-h}^{h}\frac{f(x)}{x}\,dx = \int_\gamma \frac{f(x)}{x}\,dx + \pi \mathrm{i}f(0), $$&#xA;where $\gamma$ is a contour going from $-h$ to $h$ passing above the real line. If you can evaluate $G$ for complex $\omega$, then even standard quadrature routines will be able to compute the integral over $\gamma$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-10-16T18:47:30.613" LastActivityDate="2014-10-16T18:47:30.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15889" PostTypeId="2" ParentId="15876" CreationDate="2014-10-16T22:10:51.947" Score="1" Body="&lt;p&gt;For the past few weeks I have been busy with a similar question myself. By no means I am an expert, but I will share some of my 'major' insights.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The contact response is composed of two component: a normal component (which you will use for collision) and a friction component. The former acts normal to the collision plane, the latter in the contact plane. This gives you the direction of the contact forces in which you are interested: those in the normal direction.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The momentum equations have to be satisfied, I will only give linear momentum to keep the answer simple:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$ m{\bf v}(t+\Delta t) - m{\bf v}(t) = \int_t^{t + \Delta t} {\bf F} dt $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\bf{v}$ is the velocity at a certain time, and $F$ the force over time. If the collision takes place between $t$ and $t + \Delta t$ then the integral on the right hand side represents the collision response, which should be equal and opposite for both bodies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you have enough equations to solve the system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which leaves your question about material properties. I think this makes the problem a combined Discrete Element Method (typically dealing with collisions) - Finite Element Method (typically dealing with deformations) problem. I am not yet familiar with this topic, but Prof. Munjiza seems to have written some books about it.&lt;/p&gt;&#xA;" OwnerUserId="10193" LastActivityDate="2014-10-16T22:10:51.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15890" PostTypeId="2" ParentId="15886" CreationDate="2014-10-16T22:51:26.510" Score="1" Body="&lt;p&gt;Bill stated this correctly in his comment.  The reference BLAS implementation uses triply nested loops, but any fast implementation will use small panel matrices.  The minimum size will be architecture and implementation dependent.  You should refer to the &lt;a href=&quot;http://www.cs.utexas.edu/users/pingali/CS378/2008sp/papers/gotoPaper.pdf&quot; rel=&quot;nofollow&quot;&gt;Goto paper&lt;/a&gt; for the gory details.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2014-10-16T22:51:26.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15891" PostTypeId="1" AcceptedAnswerId="15893" CreationDate="2014-10-17T00:21:23.203" Score="17" ViewCount="5528" Body="&lt;p&gt;I am optimizing a function of 10-20 variables. The bad news is that each function evaluation is expensive, approx 30 min of serial computation. The good news is that I have a cluster with a few dozen computational nodes at my disposal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the question: are there optimization algorithms available that would allow me to use all that computational power efficiently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On one side of the spectrum would be an exhaustive search: subdivide the whole search space into a fine grid and compute the function at each gridpoint independently. This is certainly a very parallel computation, but the algorithm is horribly inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other side of the spectrum would be quasi-Newton algorithms: intelligently update the next estimate of the parameters based on the prior history. This is an efficient algorithm, but I don't know how to make it parallel: the concept of the &quot;estimate of the parameters based on the prior history&quot; sounds like a serial computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quadratic algorithms seem to be somewhere in the middle: one can build the initial &quot;surrogate model&quot; by computing a bunch of values in parallel, but I don't know whether the remaining iterations are parallelizable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on what kind of gradient-free optimization methods would perform well on a cluster? Also, are there any parallel implementations of optimization algorithms currently available? &lt;/p&gt;&#xA;" OwnerUserId="5281" LastEditorUserId="123" LastEditDate="2014-10-17T03:33:35.713" LastActivityDate="2017-01-10T04:16:07.210" Title="Parallel optimization algorithms for a problem with very expensive objective function" Tags="|optimization|parallel-computing|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="15892" PostTypeId="2" ParentId="15891" CreationDate="2014-10-17T01:30:24.477" Score="0" Body="&lt;p&gt;The choice of optimization algorithm (and thus its parallelization) highly depends on the properties of the objective function and constraints.  Without knowing more about the problem, it's hard to give any kind of meaningful advice.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But from your considerations of newton methods, I infer that your objective function is differentiable.  If possible, your problem would benefit greatly from parallelizing the function evaluation.  If not possible, then you may also consider an inexact newton method which replaces the exact gradients/hessians by finite difference approximations.  Then, you can use all those processors at your disposal to compute each non-zero element of the jacobian, as @stali suggests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information, read &lt;a href=&quot;http://www.bioinfo.org.cn/~wangchao/maa/Numerical_Optimization.pdf&quot; rel=&quot;nofollow&quot;&gt;Nocedal &amp;amp; Wright's Numerical Optimization, Chapter 7&lt;/a&gt;.  There are many optimization software packages that implement this in parallel.  Among the most widely used freeware is the &lt;a href=&quot;http://dakota.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;DAKOTA software package (Sandia National Labs)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-10-17T02:06:17.360" LastActivityDate="2014-10-17T02:06:17.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15893" PostTypeId="2" ParentId="15891" CreationDate="2014-10-17T02:33:24.503" Score="16" Body="&lt;p&gt;As Paul states, without more information, it is hard to give advice without assumptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With 10-20 variables and expensive function evaluations, the tendency is to recommend derivative-free optimization algorithms. I am going to disagree strongly with Paul's advice: you generally need a machine-precision gradient unless you're using some sort of special method (for instance, stochastic gradient descent in machine learning will exploit the form of the objective to come up with reasonable gradient estimates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each quasi-Newton step is going to be of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\tilde{H}(x_{k})d_{k} = -\nabla{f}(x_{k}),&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\tilde{H}$ is some approximation of the Hessian matrix, $d_{k}$ is the search direction, $x_{k}$ is the value of the decision variables at the current iterate, $f$ is your objective function, and $\nabla{f}$ is the gradient of your objective, and the decision variables are updated like $x_{k+1} = x_{k} + \alpha_{k}d_{k}$, where $\alpha_{k}$ is a step size determined in some fashion (like a line search). You can get away with approximating the Hessian in certain ways and your iterations will converge, although if you use something like finite difference approximations of the Hessian via exact gradients, you might suffer from issues due to ill-conditioning. Typically, the Hessian is approximated using the gradient (for instance, BFGS type methods with rank-1 updates to the Hessian).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Approximating the Hessian and the gradient both via finite differences is a bad idea for a number of reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you're going to have error in the gradient, so the quasi-Newton method you're applying is akin to finding the root of a noisy function&lt;/li&gt;&#xA;&lt;li&gt;if function evaluations are expensive and you're trying to evaluate a gradient with respect to $N$ variables, it's going to cost you $N$ function evaluations &lt;em&gt;per iteration&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;if you have error in the gradient, you're going to have more error in your Hessian, which is a big problem in terms of the conditioning of the linear system&lt;/li&gt;&#xA;&lt;li&gt;...and it's going to cost you $N^{2}$ function evaluations &lt;em&gt;per iteration&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So to get one bad iteration of quasi-Newton, you're doing something like up to 420 function evaluations at 30 minutes per evaluation, which means you're either going to be waiting a while for each iteration, or you're going to need a big cluster just for the function evaluations. The actual linear solves are going to be of 20 by 20 matrices (at most!), so there's no reason to parallelize those. If you can get gradient information by, for instance, solving an adjoint problem, then it might be more worthwhile, in which case, it might be worth looking at a book like Nocedal &amp;amp; Wright.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're going to do a lot of function evaluations in parallel, you should look instead at surrogate modeling approaches or at generating set search methods before considering quasi-Newton approaches. The classic review articles are the one by &lt;a href=&quot;http://thales.cheme.cmu.edu/dfo/comparison/dfo.pdf&quot;&gt;Rios and Sahinidis on derivative-free methods&lt;/a&gt;, which was published in 2012 and provides a really good, broad comparison; the benchmarking article by &lt;a href=&quot;http://www.mcs.anl.gov/papers/P1471A.pdf&quot;&gt;More and Wild&lt;/a&gt; from 2009; the 2009 textbook &quot;Introduction to Derivative-Free Optimization&quot; by Conn, Scheinberg, and Vicente; and the &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.96.8672&amp;amp;rep=rep1&amp;amp;type=pdf&quot;&gt;review article on generating set search methods by Kolda, Lewis, and Torczon&lt;/a&gt; from 2003.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As linked above, the &lt;a href=&quot;http://dakota.sandia.gov/&quot;&gt;DAKOTA software package&lt;/a&gt; will implement some of those methods, and so will &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt&quot;&gt;NLOPT&lt;/a&gt;, which implements DIRECT, and a few of Powell's surrogate modeling methods. You might also take a look at &lt;a href=&quot;http://www.mat.univie.ac.at/~neum/software/mcs/&quot;&gt;MCS&lt;/a&gt;; it's written in MATLAB, but maybe you can port the MATLAB implementation to the language of your choice. DAKOTA's essentially a collection of scripts you can use to run your expensive simulation and collect data for optimization algorithms, and NLOPT has interfaces in a large number of languages, so choice of programming language shouldn't be a huge issue in using either software package; DAKOTA does take a while to learn, though, and has a massive amount of documentation to sift through.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-17T02:33:24.503" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15894" PostTypeId="2" ParentId="15882" CreationDate="2014-10-17T03:44:35.687" Score="3" Body="&lt;p&gt;You have discretized an advection equation using a forward difference in time and centered differences in space.  You have correctly deduced that this is an unstable discretization; in fact it is unstable even for constant-coefficient advection in one dimension.  There are many stable discretizations you could use; the most common (and simplest) is to switch to a centered difference in time.  To learn about other discretizations, I recommend Chapter 10 of &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow&quot;&gt;LeVeque's finite difference text&lt;/a&gt;.  There are many other good references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not recommend using an implicit method for this or most other hyperbolic problems.  Implicit methods are more expensive (per timestep) and more difficult to implement, and for this kind of problem an implicit method will not allow you to take much larger timesteps without badly diffusing (smearing) the solution or introducing oscillations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is really no such thing as &quot;absorbing boundary conditions&quot; for advection, since there is no reflection.  All you can (and should) do is specify the solution values along incoming characteristics at the boundary.  Those depend on the initial data, which you haven't specified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should add that it's also easy to solve this kind of problem with &lt;a href=&quot;http://www.clawpack.org&quot; rel=&quot;nofollow&quot;&gt;Clawpack&lt;/a&gt;, which (however) uses finite volume methods.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-10-17T03:44:35.687" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15895" PostTypeId="2" ParentId="15867" CreationDate="2014-10-17T04:45:14.630" Score="3" Body="&lt;p&gt;You can get a rough estimation of the condition number using the Gershgorin circle theorem. This article in Wikipedia has a nice explanation: &lt;a href=&quot;http://en.wikipedia.org/wiki/Gershgorin_circle_theorem&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Gershgorin_circle_theorem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a complex matrix $A$ of size $n\times n$, its n Gershgorin circles are drawn in the complex plane, the center of the circles are $a_{ii}$, the radius of the ith circle is $\sum_{i\neq j} |a_{ij}|$. The theorem says that every eigenvalue of $A$ lies inside a Gershgorin circle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a SPD matrix the eigenvalues are on the real axis, so you can estimate the highest possible and the lowest possible. Using this you get a rough estimation of the condition number.&lt;/p&gt;&#xA;" OwnerUserId="7020" LastEditorUserId="1804" LastEditDate="2014-10-17T08:19:39.777" LastActivityDate="2014-10-17T08:19:39.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15896" PostTypeId="1" CreationDate="2014-10-17T06:06:30.750" Score="2" ViewCount="2162" Body="&lt;p&gt;Could you please advise some literature about the numerical method of lines (MOL) for parabolic PDEs? It is a method of solving PDEs with discretizing only by space but not by time. A system of ODEs is obtained and it can be solved with, for example, Runge-Kutta method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know some solvers (programs, libraries) for applying the method of lines? Is this method applicable to multi-dimensional (2D, 3D) problems?&lt;/p&gt;&#xA;" OwnerUserId="11531" LastEditorUserId="123" LastEditDate="2015-04-22T17:31:35.710" LastActivityDate="2015-11-30T18:42:05.757" Title="Applying the method of lines to parabolic PDEs: references and software" Tags="|pde|numerical-analysis|reference-request|parabolic-pde|method-of-lines|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15897" PostTypeId="2" ParentId="15896" CreationDate="2014-10-17T06:40:40.727" Score="5" Body="&lt;p&gt;A nice reference for this is Chapter 10 of &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot;&gt;LeVeque's finite difference book&lt;/a&gt;.  Of course, it only covers basic finite difference approaches, and there are plenty of others (all within the method of lines framework).  The method of lines is indeed applicable in multiple dimensions, and two-dimensional problems are discussed in the reference just given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the methods and libraries that exist are based on the method of lines, so if you want meaningful recommendations you'll need to ask a more specific question.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-10-17T06:40:40.727" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15898" PostTypeId="2" ParentId="15896" CreationDate="2014-10-17T06:44:17.670" Score="2" Body="&lt;p&gt;This is a fairly big question. The basic idea is that in solving $\partial_t u = \mathcal{L}u$, you approximate $\mathcal{L}$ with a matrix $A$, and solve&#xA;$$ u' = Au. $$&#xA;In this formulation you have two main issues:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What method to use to solve the ODE. The method has to be implicit because of the nature of parabolic PDEs, and the stiff systems of ODEs that they produce.&lt;/li&gt;&#xA;&lt;li&gt;How to solve the systems of linear equations $A^{-1}x$ that appear in the ODE solver. Here $A$ is sparse.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These issues are not at all unique to the method of lines, or to parabolic equations. Books like &quot;Solving ordinary differential equations&quot; by Hairer and Wanner will cover this; &quot;Finite difference methods&quot; by LeVeque; there's an unfinished textbook available for free online &lt;a href=&quot;http://people.maths.ox.ac.uk/trefethen/pdetext.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; by Trefethen; other PDE textbooks cover this as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that just an explicit Runge-Kutta method to unlikely to work directly: the system of ODEs is typically stiff, sparse, and very large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question of how to solve the resulting systems of linear equations, roughly, can be addressed in different ways. If you can solve $A^{-1}x$ directly and efficiently, there is no problem. There are (diverse!) iterative methods, which work well if you can find a good preconditioner for the matrix. There are also different splitting methods, where you can write $A=A_1+A_2+\cdots$, and define a numerical solution to the ODE that depends only on solutions of systems $A_k^{-1}x$. You can pick $A_k$ to be convenient to solve, such as the individual parts of the differential operator going along individual dimensions. Alternating direction implicit methods are an example of this approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For software, most PDE packages do something like this, something like CVODE (in SUNDIALS) might be appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;For nonlinear systems, the procedure is roughly the same, only instead of needing to solve systems like $A^{-1}x$ you need to solve systems of nonlinear equations with a Newton-like method, where the solution of $J^{-1}x$ with the Jacobian matrix needs to be efficient; again the Jacobian is likely to be sparse and very large. In this context, &lt;a href=&quot;http://computation.llnl.gov/casc/sundials/description/description.html#descr_cvode&quot; rel=&quot;nofollow&quot;&gt;CVODE&lt;/a&gt; looks appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; As &lt;strong&gt;David Ketcheson&lt;/strong&gt; correctly points out, my answer assumes many things that are not explicitly specified in the question, which would make this answer quite misleading if those assumptions don't hold.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-10-17T19:30:27.177" LastActivityDate="2014-10-17T19:30:27.177" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="15899" PostTypeId="1" AcceptedAnswerId="15905" CreationDate="2014-10-17T14:06:38.377" Score="4" ViewCount="124" Body="&lt;p&gt;I often find in the literature some numerical tests use CAD and mesh models. I also want to reproduce their results or test my algorithms on those models. But I am not trained to use CAD or mesh tools. I tried to get cad models from CAD website such as `grabcad'. But I often found it difficult to generate a mesh for a CAD model: sometimes gmsh says the model is incorrect.  Do you know some good website providing such models ready for FEM simulation?&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2014-10-17T22:46:52.527" Title="Where to find CAD and mesh models for tests?" Tags="|mesh|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15900" PostTypeId="1" AcceptedAnswerId="15906" CreationDate="2014-10-17T16:06:44.113" Score="6" ViewCount="3584" Body="&lt;p&gt;How to estimate the local error and the global error for Runge-Kutta method used for solve a system of differential equations in practice? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use Richardson extrapolation for select a adaptive step &lt;a href=&quot;http://s014.radikal.ru/i326/1410/ed/6eedbdf5f72b.png&quot;&gt;[Solving Ordinary Differential Equations I - Nonstiff Problems 167-168p]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I seen the following formula for estimate a global error for one differential equation: $\delta_{i+1} = \delta_{i}\cdot e^{\int^{t_{i+1}}_{t_i} \mu ds} + r_k$, where $r_k$ local error at step, $\mu$ - maximal eigen value of $\frac{J + J^T}{2}$ ($J$ is jacobian of system). But I didn&amp;#39;t find its explanation and application to system of ODE.&lt;/p&gt;&#xA;" OwnerUserId="11536" LastActivityDate="2014-10-18T02:25:12.647" Title="How to estimate the local error and the global error for Runge-Kutta method" Tags="|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15901" PostTypeId="1" CreationDate="2014-10-17T18:37:55.170" Score="2" ViewCount="1011" Body="&lt;p&gt;In python I have used the &lt;code&gt;ode&lt;/code&gt; method from &lt;code&gt;scipy.integrate&lt;/code&gt;. There I used the &lt;code&gt;vode&lt;/code&gt;integrator and the &lt;code&gt;zvode&lt;/code&gt; integrator to solve some numerical odes. I am happy with the solution provided by these methods and would now like to know what the equivalent methods are in matlab. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im not familiar with the intricacies of odes (stiffness and which solvers are suitable in each case) I simply want to know if there is an equivalent matlab routine to the above two integrator methods which I can readily use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of how I have defined a function which should march forward in time &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def time_march2(y0, x1_max):&#xA;&#xA;   y0 = np.asarray(y0)&#xA;   t0 = 0&#xA;   tlimit = 100.0 &#xA;&#xA;   backend = 'vode'  &#xA;   solver = ode(f2).set_integrator(backend, nsteps=1)&#xA;   solver.set_initial_value(y0, t0)&#xA;   # suppress Fortran-printed warning&#xA;   solver._integrator.iwork[2] = -1&#xA;&#xA;   solution_y = [y0]&#xA;   solution_t = [t0]&#xA;   warnings.filterwarnings(&quot;ignore&quot;, category=UserWarning)&#xA;&#xA;   while solver.successful() and solver.y[0] &amp;lt; x1_max and solver.t &amp;lt; tlimit:&#xA;      solver.integrate(tlimit, step=True)          &#xA;      solution_y.append(solver.y)&#xA;      solution_t.append(solver.t)&#xA;&#xA;&#xA;   warnings.resetwarnings()&#xA;   solution_y = np.array(solution_y)&#xA;   solution_t = np.array(solution_t)&#xA;&#xA;&#xA;   return solution_t, solution_y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="7864" LastEditDate="2014-10-17T18:48:23.070" LastActivityDate="2014-10-17T18:53:27.313" Title="Matlab equivalent of scipy's 'vode' and 'zvode' ode routines" Tags="|matlab|python|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15902" PostTypeId="2" ParentId="15901" CreationDate="2014-10-17T18:53:27.313" Score="4" Body="&lt;p&gt;If you want to use base MATLAB routines, &lt;code&gt;ode15s&lt;/code&gt; is the closest analogue (see &lt;a href=&quot;https://www.mathworks.com/help/pdf_doc/otherdocs/ode_suite.pdf&quot; rel=&quot;nofollow&quot;&gt;Shampine's paper&lt;/a&gt;), although you have to set options to ensure that it uses a BDF method instead of the default NDF method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you just want a MATLAB interface, &lt;a href=&quot;http://computation.llnl.gov/casc/sundials/main.html&quot; rel=&quot;nofollow&quot;&gt;SUNDIALS&lt;/a&gt; has a MATLAB interface to CVODE; CVODE is an updated version of VODE.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-17T18:53:27.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15905" PostTypeId="2" ParentId="15899" CreationDate="2014-10-17T22:46:52.527" Score="4" Body="&lt;p&gt;I've used examples from:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.rocq.inria.fr/gamma/download/&quot; rel=&quot;nofollow&quot;&gt;INRIA&lt;/a&gt;: huge number of meshes in many file formats&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cc.gatech.edu/projects/large_models/&quot; rel=&quot;nofollow&quot;&gt;Large Geometric Models archive&lt;/a&gt;: handful of very big meshes in .ply format&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cs.berkeley.edu/~jrs/stellar/&quot; rel=&quot;nofollow&quot;&gt;Stellar&lt;/a&gt;: a program for improving tetrahedral meshes; has a few examples on its website&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Purely a matter of preference, but I like &lt;a href=&quot;http://wias-berlin.de/software/tetgen/&quot; rel=&quot;nofollow&quot;&gt;Tetgen&lt;/a&gt; for 3D tetrahedral mesh generation more than gmsh. While gmsh has more features, I find that I don't need most of them and I'd rather use a file format that's easier to parse / write.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-10-17T22:46:52.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15906" PostTypeId="2" ParentId="15900" CreationDate="2014-10-18T02:25:12.647" Score="4" Body="&lt;p&gt;The local error is typically estimated using something like an embedded pair of Runge-Kutta methods. The classic example would be the 4(5) pair (for instance, Dormand-Prince), where the error over a single step would be estimated by comparing the fourth- and fifth-order solutions. Other methods include approaches like halving the time step, or Richardson extrapolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The global estimate you're referring to is one by Dahlquist, using the logarithmic norm, which is a bound on the growth of the norm of the solution to an ODE. (See reference 3 of this &lt;a href=&quot;http://www.maths.lth.se/na/staff/gustaf/soderlindDqREV.pdf&quot; rel=&quot;nofollow&quot;&gt;review paper on the logarithmic norm by Soderlind&lt;/a&gt;. A similar, weaker result was proven by Gronwall using Lipschitz constants; see &lt;a href=&quot;http://en.wikipedia.org/wiki/Gronwall&amp;#39;s_inequality&quot; rel=&quot;nofollow&quot;&gt;Gronwall's inequality&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a variety of methods for bounding and estimating the global error in solutions of ODEs, and I'm sure I'll fail to list all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For estimates, the most recent paper I'm aware of is by &lt;a href=&quot;http://www.mcs.anl.gov/~emconsta/Repository/Papers/glb-err-preprint-20140207.pdf&quot; rel=&quot;nofollow&quot;&gt;Constantinescu&lt;/a&gt;, which has a large number of references worth looking at; other notable papers include the &lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF01389440#page-1&quot; rel=&quot;nofollow&quot;&gt;now-dated review by Skeel&lt;/a&gt;, the &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0732001&quot; rel=&quot;nofollow&quot;&gt;adjoint approach by Estep&lt;/a&gt; (he has a whole series of adjoint-based a posteriori error estimation papers), and the &lt;a href=&quot;http://www.cs.ucsb.edu/~cse/Files/condition_ode032.pdf&quot; rel=&quot;nofollow&quot;&gt;adjoint approach by Cao &amp;amp; Petzold&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bounds could also be computed using interval arithmetic and Taylor methods. The &lt;a href=&quot;https://yoric.mit.edu/sites/default/files/documents/ScottThesis.pdf&quot; rel=&quot;nofollow&quot;&gt;thesis by Joseph Scott&lt;/a&gt; should have a number of references discussing this topic. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-18T02:25:12.647" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15907" PostTypeId="1" CreationDate="2014-10-18T10:14:14.230" Score="13" ViewCount="7327" Body="&lt;p&gt;I am experiencing some frustration over the way matlab handles numerical integration vs. Scipy. I observe the following differences in my test code below:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Matlab's version runs on average &lt;strong&gt;24 times faster&lt;/strong&gt; than my python equivalent!&lt;/li&gt;&#xA;&lt;li&gt;Matlab's version is able to calculate the integral without warnings, while python returns &lt;code&gt;nan+nanj&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What can I do to ensure I get the same performance in python with respect to the two points mentioned? According to documentation both methods should be using a &amp;quot;global adaptive quadrature&amp;quot; to approximate the integral.&lt;/p&gt;&#xA;&lt;p&gt;Below is the code in the two versions (fairly similar although python requires that an integral function is created so that it can handle complex integrands.)&lt;/p&gt;&#xA;&lt;h1&gt;Python&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy import integrate&#xA;import time&#xA;&#xA;def integral(integrand, a, b,  arg):&#xA;    def real_func(x,arg):&#xA;        return np.real(integrand(x,arg))&#xA;    def imag_func(x,arg):&#xA;        return np.imag(integrand(x,arg))&#xA;    real_integral = integrate.quad(real_func, a, b, args=(arg))&#xA;    imag_integral = integrate.quad(imag_func, a, b, args=(arg))   &#xA;    return real_integral[0] + 1j*imag_integral[0]&#xA;&#xA;vintegral = np.vectorize(integral)&#xA;&#xA;&#xA;def f_integrand(s, omega):&#xA;    sigma = np.pi/(np.pi+2)&#xA;    xs = np.exp(-np.pi*s/(2*sigma))&#xA;    x1 = -2*sigma/np.pi*(np.log(xs/(1+np.sqrt(1-xs**2)))+np.sqrt(1-xs**2))&#xA;    x2 = 1-2*sigma/np.pi*(1-xs)&#xA;    zeta = x2+x1*1j&#xA;    Vc = 1/(2*sigma)&#xA;    theta =  -1*np.arcsin(np.exp(-np.pi/(2.0*sigma)*s))&#xA;    t1 = 1/np.sqrt(1+np.tan(theta)**2)&#xA;    t2 = -1/np.sqrt(1+1/np.tan(theta)**2)&#xA;    return np.real((t1-1j*t2)/np.sqrt(zeta**2-1))*np.exp(1j*omega*s/Vc);&#xA;&#xA;t0 = time.time()&#xA;omega = 10&#xA;result = integral(f_integrand, 0, np.inf, omega)&#xA;print time.time()-t0&#xA;print result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Matlab&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;function [ out ] = f_integrand( s, omega )&#xA;    sigma = pi/(pi+2); &#xA;    xs = exp(-pi.*s./(2*sigma));&#xA;    x1 = -2*sigma./pi.*(log(xs./(1+sqrt(1-xs.^2)))+sqrt(1-xs.^2));&#xA;    x2 = 1-2*sigma./pi.*(1-xs);&#xA;    zeta = x2+x1*1j;&#xA;    Vc = 1/(2*sigma);&#xA;    theta =  -1*asin(exp(-pi./(2.0.*sigma).*s));&#xA;    t1 = 1./sqrt(1+tan(theta).^2);&#xA;    t2 = -1./sqrt(1+1./tan(theta).^2);&#xA;    out = real((t1-1j.*t2)./sqrt(zeta.^2-1)).*exp(1j.*omega.*s./Vc);&#xA;end&#xA;&#xA;t=cputime;&#xA;omega = 10;&#xA;result = integral(@(s) f_integrand(s,omega),0,Inf)&#xA;time_taken = cputime-t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-08-01T12:12:07.457" Title="Why does Matlab's integral outperform integrate.quad in Scipy?" Tags="|matlab|python|quadrature|numba|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="15909" PostTypeId="1" AcceptedAnswerId="15910" CreationDate="2014-10-18T11:08:58.923" Score="4" ViewCount="263" Body="&lt;p&gt;I would like to solve an equation that looks like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E[(R^{1-\gamma})(r_k+\theta-r_z)]=0$ , where $R=\phi r_z+(1-\phi)(r_k+\theta)$ and $\phi\in[0,1]$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\theta$, is a random variable normaly distributed with zero mean, and some variance $\sigma^2$. Thus, the $E[.]$ stands for the expectation operator over this random variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative version of the problem, solves a similar equation, but now $\theta$ is &lt;em&gt;lognormally&lt;/em&gt; distributed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computational task, is to find $\phi$, such as the condition/equation above holds, for some given numbers/values for $r_z,r_k$ and $\gamma$ all of which are positive real numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help on how I can calculate this, ideally in MATLAB ? &lt;/p&gt;&#xA;" OwnerUserId="9826" LastEditorUserId="20688" LastEditDate="2019-07-05T06:11:27.757" LastActivityDate="2019-07-05T06:11:27.757" Title="Solving a nonlinear equation with random variable" Tags="|matlab|nonlinear-equations|stochastic|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15910" PostTypeId="2" ParentId="15909" CreationDate="2014-10-18T14:29:40.810" Score="5" Body="&lt;p&gt;Since $\phi$ is a scalar between $0$ and $1$, the easiest method for finding a root is &lt;a href=&quot;http://en.wikipedia.org/wiki/Bisection_method&quot; rel=&quot;nofollow&quot;&gt;bisection&lt;/a&gt;. If you cannot calculate the expectation of the nonlinear function &#xA;$$f_\phi(\theta) = \left(\phi r_z +(1-\phi)(r_k+\theta)\right)^{1-\gamma}(r_k+\theta-r_z)$$ in terms of $\phi$ analytically, you can use quadrature to approximate. For the first variant, this amounts to&#xA;$$E[f_\phi(\theta)] = \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{-\frac{t^2}{2\sigma^2}} f_\phi(t)\,dt \approx \sum_{i=1}^N \frac{w_i}{\sqrt{\pi}} f_\phi(\sqrt2 \sigma t_i),$$&#xA;where $t_i$ and $w_i$ are the &lt;a href=&quot;http://dlmf.nist.gov/3.5#P6&quot; rel=&quot;nofollow&quot;&gt;Gauss-Hermite quadrature&lt;/a&gt; nodes and weights of order $N$. If $\theta$ is log-normally distributed, then $\theta=e^\beta$, where $\beta$ is normally distributed, so you can just replace $t_i$ by $s_i:=e^{t_i}$ in the above to get a quadrature rule (with the same weights) for the log-normal variant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This procedure is straightforward to implement in Matlab; you can download functions to calculate the quadrature weights and nodes &lt;a href=&quot;http://hips.seas.harvard.edu/content/gauss-hermite-quadrature-matlab&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/8066-gauss-hermite&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-10-19T14:49:04.403" LastActivityDate="2014-10-19T14:49:04.403" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15911" PostTypeId="1" AcceptedAnswerId="15922" CreationDate="2014-10-18T14:31:15.920" Score="3" ViewCount="1212" Body="&lt;p&gt;One of the famous and convenient test cases for shock wave modeling is the 1D Sod's shock tube. This is a Riemann problem for the compressible Euler equations of gas dynamics.  The initial set up has been done by dividing domain half with diaphragm. The left and right sides of the domain have initial values chosen to generate a shock wave. The left region has initial pressure and density values higher then those in the right region. Velocities are zero on both sides of the diaphragm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, I know the initial Mach number of the shock wave and the right hand side initial values. I wish to know how to determine the initial values for the left side when only the Mach number and initial right state are given.&lt;/p&gt;&#xA;" OwnerUserId="11495" LastEditorUserId="123" LastEditDate="2014-10-23T10:00:58.370" LastActivityDate="2014-10-23T10:00:58.370" Title="How to set up a shock tube problem such that the solution includes a shock with a specified Mach number" Tags="|pde|fluid-dynamics|hyperbolic-pde|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15912" PostTypeId="2" ParentId="15911" CreationDate="2014-10-18T14:38:49.843" Score="0" Body="&lt;p&gt;If I recall my compressibles, you want to use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Normal_shock_tables&quot; rel=&quot;nofollow&quot;&gt;normal shock equations&lt;/a&gt;. You'll also need to pick a $\gamma$. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-10-18T14:38:49.843" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="15913" PostTypeId="1" CreationDate="2014-10-19T06:42:59.500" Score="3" ViewCount="761" Body="&lt;p&gt;For &lt;a href=&quot;http://en.wikipedia.org/wiki/Differential_evolution&quot; rel=&quot;nofollow&quot;&gt;differential evolution methods in optimization&lt;/a&gt;, how many generations does it typically  take to reach a global optimum? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we know if the values are never going to converge?&lt;/p&gt;&#xA;" OwnerUserId="11545" LastEditorUserId="1804" LastEditDate="2014-10-20T08:32:45.007" LastActivityDate="2014-10-22T11:04:44.593" Title="How many generations does it typically take for a differential evolution method to reach a global optimum?" Tags="|optimization|iterative-method|convergence|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15914" PostTypeId="2" ParentId="15913" CreationDate="2014-10-19T09:52:17.757" Score="6" Body="&lt;p&gt;I don't think there's enough information to give a heuristic like, &quot;in general, it takes $k$ iterations for differential evolution to reach a global optimum&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we don't know how many iterations deterministic global optimizers will take to converge to global $\varepsilon$-optimality in the general case. In limited special cases, we know that certain optimization algorithms will converge within a certain number of steps (e.g., simplex algorithms for linear programs, branch-and-bound algorithms for mixed-integer linear programs, polynomial integer programs, Newton methods for unconstrained convex quadratic programs, conjugate gradient methods for unconstrained convex quadratic programs, Newton methods for unconstrained convex nonlinear programs inside the quadratic region of convergence). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, differential evolution is a nondeterministic global optimization algorithm.  Nondeterministic global optimization algorithms have weaker convergence theory than deterministic optimization algorithms. While deterministic optimization algorithms converge with certainty (provided the hypotheses of the algorithms used are satisfied), if a nondeterministic global optimization algorithm has a convergence proof, it is usually a proof that the method converges with probability one. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0167715213001934&quot;&gt;Here&lt;/a&gt; is an example of a paper that does just that for a genetic algorithm. &quot;Convergence with probability one&quot;, while precise, and certainly better than no result at all, is akin to saying, &quot;if we let this algorithm run for long enough, it will eventually search the entire search space, and return the correct answer&quot;. If the search space is infinite, it could take infinite time, which is...not a very helpful result. (To be fair, the deterministic case isn't that much better when it comes to global optimization.) A cursory literature search suggests that the convergence theory for differential evolution is not well-developed, and without one, it's difficult to say if the method will converge at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, for deterministic algorithms on general problems, it's entirely possible for a significant optimality gap (i.e., the difference between the upper bound and lower bound calculated for the objective function value) to persist for tens to hundreds of iterations with little improvement, and for the rate of decrease in that optimality gap to depend on tolerances and algorithmic heuristics provided to the optimizer for a given problem instance. A significant amount of effort has gone into coming up with &quot;best input tolerances and heuristics&quot; for general problems, and for &quot;autotuning algorithms&quot; (supposedly, for instance, CPLEX has one) that will determine best values for these tolerances and heuristics. You may find that over the course of your experience with the algorithm that you get a better feel for how to pick input tolerances and heuristics for differential evolution as you work with it, and what values work best for different types of problems. Experimentation is often needed with numerical algorithms (witness, for instance, the amount of experimentation still necessary for solving systems of linear equations with iterative linear solvers). You may find the thesis &lt;a href=&quot;http://www.hvass-labs.org/people/magnus/thesis/pedersen08thesis.pdf&quot;&gt;Tuning &amp;amp; Simplifying Heuristical Optimization&lt;/a&gt; helpful.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-19T09:52:17.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15920" PostTypeId="2" ParentId="5238" CreationDate="2014-10-20T10:01:39.463" Score="0" Body="&lt;p&gt;Try Tveito, Langtangen, Nielsen, Cai: &lt;em&gt;Elements of Scientific Computing&lt;/em&gt;. This is for beginners and pedagogically excellent.&lt;/p&gt;&#xA;" OwnerUserId="11556" LastActivityDate="2014-10-20T10:01:39.463" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="15921" PostTypeId="2" ParentId="2047" CreationDate="2014-10-20T10:35:16.967" Score="0" Body="&lt;p&gt;An introductory book which explains the basics very well is Gander, Gander, Kwok: &lt;em&gt;Scientific Computing.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="11556" LastActivityDate="2014-10-20T10:35:16.967" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15922" PostTypeId="2" ParentId="15911" CreationDate="2014-10-20T10:39:07.847" Score="5" Body="&lt;p&gt;I have &lt;a href=&quot;http://nbviewer.ipython.org/gist/ketch/d31fd8d2d7739e59b6c6&quot; rel=&quot;nofollow noreferrer&quot;&gt;implemented the solution derived below using PyClaw in an IPython notebook&lt;/a&gt;.  If you download that, you can adjust the initial values and see the computed solutions.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;General setup&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In the solution of a Riemann problem for the 1D Euler equations, there are generally 3 waves.  Two of them are genuinely nonlinear; the other is a contact discontinuity, which only carries a jump in the density.  My understanding is that you'd like to set up initial left and right states such that the velocity is initially zero everywhere, the rightmost wave is a shock moving at Mach number $M$ relative to the right-state sound speed, and the leftmost wave is a rarefaction (you didn't specify the last requirement, but it would make the problem analogous to the Sod shock tube).  This situation is depicted in the figure below, which also establishes the notation I'll use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yeg0Q.png&quot; alt=&quot;Riemann solution states&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Finding the state behind the shock&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Let&#xA;$$\mu = \frac{2(M^2-1)}{M(\gamma+1)}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Rankine-Hugoniot conditions, we find that the state just to the right of the contact is given by the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\rho_r^*&amp;amp; = \rho_r \frac{M}{M-\mu} \\&#xA;u &amp;amp; = \mu c_r \\&#xA;p &amp;amp; = p_r \frac{(2M^2 -1)\gamma+1}{\gamma+1}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you were willing to have a non-zero velocity for the left state, you could stop here and use the values just derived for the left state.  If you want $u_l=0$, read on.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Finding the left state&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Next, we know that the left state and the $[\rho_l^*, u, p]$ state must be connected by a left-going rarefaction.  The relevant Riemann invariants imply&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho_l = \frac{4\gamma p_l}{(\gamma-1)^2 u^2} \left(1- \left(\frac{p}{p_l}\right)^{\frac{\gamma-1}{2\gamma}}\right)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho_l^* = \left(\frac{p}{p_l}\right)^{1/\gamma} \rho_l.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can prescribe any value we like for $p_l$ and compute the remaining values $\rho_l,\rho_l^*$ from the equations above.  The only catch is that the resulting rarefaction should satisfy the entropy condition, which is true if $p_l&amp;gt;p$.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Notes&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I worked this out using Chapter 14 of &lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Randall Leveque's text on finite volume methods&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a 1-parameter family of possible left states (which seems obvious once you check the number of unknowns and constraints).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you didn't restrict the left-state velocity to be zero, it would be possible to set up a problem where only the shock wave appears by taking the left state to be $(\rho_r^*, u, p)^T$.  But since you want zero initial velocities, it's necessary to use the full procedure described above.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-10-20T14:41:10.240" LastActivityDate="2014-10-20T14:41:10.240" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15923" PostTypeId="2" ParentId="8806" CreationDate="2014-10-20T10:40:35.310" Score="1" Body="&lt;p&gt;You might try: Bertil Gustafsson: &lt;em&gt;Fundamentals of Scientific Computing&lt;/em&gt; to get an overview, and for a more encyclopaedic treatment of the mathematical methods Gander, Gander, Kwok: &lt;em&gt;Scientific Computing.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="11556" LastActivityDate="2014-10-20T10:40:35.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15926" PostTypeId="1" CreationDate="2014-10-20T18:52:05.000" Score="11" ViewCount="2246" Body="&lt;p&gt;I can't see any mention of it in job listings. I've seen mentioned integer programming, MIP, mixed-integer nonlinear programming, LP, dynamic programming etc., but no SDP.&#xA;Is it much trendier in the academy than in industry? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my limited exposure to academics and industry participants in electric power systems, I think there's a good chance that SDP will be applied in optimal power flow problems by independent system operators, but that it depends on the extent to which the egg-heads can scale up current approaches to deal with bigger problem instances.&lt;/p&gt;&#xA;" OwnerUserId="9474" LastEditorUserId="9474" LastEditDate="2014-10-20T18:58:48.567" LastActivityDate="2014-10-21T03:38:44.980" Title="Do they use semidefinite programming in industry?" Tags="|convex-optimization|semidefinite-programming|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15927" PostTypeId="2" ParentId="15926" CreationDate="2014-10-20T19:58:44.250" Score="9" Body="&lt;p&gt;From my own limited experience in the power industry, no one is solving SDPs at that sort of scale. I have some limited knowledge of what the New England ISO is doing, and I think they are more interested in incorporating stochasticity into their existing MILP models. From friends who have worked on power systems at governmental research labs in the USA, they are also thinking about stochasticity (stochastic programming, chance constraints, robust optimization...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my experience in the large tech company sector, people are solving MILPs at the most complicated, and usually deterministic models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I gather from the chemical engineering side they seem interested in MINLP, in particular nonconvex quadratically-constrained optimization, which arises naturally in mixing problems. There are also PDE constrained problems and all those other fun things, but thats mostly out of my expertise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I had to speculate, SDP might be used in semiconductor design as a subroutine (e.g. for MAXCUT), but given the lack of quality solvers I'm guessing there isn't a huge demand (yet, at least).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd say in academia, SDP is more interesting as a proof tool, i.e. &quot;look, this problem is polynomial time!&quot; if you can figure out how to wrangle as an SDP. SDP solvers are so touchy (compared to other convex problem classes) that I think people aren't really excited about the idea of having to actually solve them.&lt;/p&gt;&#xA;" OwnerUserId="9783" LastEditorUserId="9783" LastEditDate="2014-10-20T20:04:00.107" LastActivityDate="2014-10-20T20:04:00.107" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15928" PostTypeId="2" ParentId="15926" CreationDate="2014-10-20T20:41:16.210" Score="6" Body="&lt;p&gt;Most of the work I'm aware of at labs for power flow problems is on stochastic optimization also, focusing mostly on MILPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In chemical engineering, they are interested in MINLPs, and the classic example is a mixing problem (specifically, the prototypical Haverly pooling problem), so bilinear terms come up a lot. Trilinear terms occasionally pop up, depending on the thermodynamic mixing models or reaction models used. There's also a limited amount of interest in ODE-constrained or PDE-constrained optimization; none of that work uses SDPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the PDE-constrained optimization work I've seen (I'm specifically thinking of topology optimization) does not use SDPs. The PDE constraints could be linear, and in theory, could admit an SDP formulation depending on what the objective and remaining constraints are. In practice, engineering problems tend to be nonlinear, and yield nonconvex problems that are then solved to local optima (possibly also using multistart). Sometimes, penalty formulations are used to exclude known suboptimal local optima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could see it maybe being used in control theory. The small amount of work I've seen on &quot;linear matrix inequalities&quot; suggests that it could possibly be useful there, but control theory in industry tends to rely on tried-and-true methods rather than bleeding edge mathematical formulations, so I doubt SDPs will be used for a while until they can prove their usefulness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few SDP solvers that are okay, and they've solved problems that are pretty big for academia (last I checked was 3-4 years ago, and they were solving tens to hundreds of thousands of variables), but power flow scenarios involve much larger problems (tens of millions to billions of variables), and I don't think the solvers are there yet. I think they could get there -- there's been a fair amount of recent work on matrix-free interior-point methods that suggests that it'd be feasible to scale up SDP solvers using those techniques -- but no one's done it yet, probably because LPs, MILPs, and convex NLPs come up much more frequently and are established technologies.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-20T20:41:16.210" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15929" PostTypeId="1" CreationDate="2014-10-20T21:27:04.390" Score="-1" ViewCount="117" Body="&lt;p&gt;Could you please advise some programs or libraries for solving parabolic PDEs (or its systems) in 1D, 2D and 3D, for example, with the method of lines? The system of parabolic PDEs can be nonlinear in the right-hand side, for example, $\partial u/\partial t - a\Delta u + u^4 = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or it is recommended to use only ODE solvers and construct the ODE system manually? What sparse ODE solvers can I use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.: Are there some methods of solving parabolic PDEs besides the method of lines? For example, one can discretize first by time and obtain elliptic equations at each time step. Is this method really slower than the method of lines? Does there exist something else?&lt;/p&gt;&#xA;" OwnerUserId="11531" LastEditorUserId="11531" LastEditDate="2014-10-20T21:36:05.903" LastActivityDate="2014-10-21T00:47:06.643" Title="Libraries with the method of lines for parabolic PDEs" Tags="|pde|ode|parabolic-pde|method-of-lines|" AnswerCount="1" CommentCount="7" ClosedDate="2014-10-21T14:29:15.940" ContentLicense="CC BY-SA 3.0" />
  <row Id="15930" PostTypeId="2" ParentId="15929" CreationDate="2014-10-21T00:47:06.643" Score="1" Body="&lt;p&gt;For reference, if you want to learn about the difference between the Rothe method and the method of lines, then maybe lectures 26 and 27 on &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; are of interest to you. If you want to see how to actually implement them, take a look at lecture 29.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: these are my own lectures.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-21T00:47:06.643" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15931" PostTypeId="1" AcceptedAnswerId="15933" CreationDate="2014-10-21T01:23:02.967" Score="3" ViewCount="74" Body="&lt;p&gt;Apologies in advance if this has already been asked before (I suspect it has, but I'm not experienced to know what to call it, or how to classify this problem). Given a set of $m$ points in space,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{X}=\left(&#xA;\begin{array}{cccc}&#xA; x_1 &amp;amp; x_2 &amp;amp; \text{...} &amp;amp; x_m \\&#xA; y_1 &amp;amp; y_2 &amp;amp; \text{...} &amp;amp; y_m \\&#xA; z_1 &amp;amp; z_2 &amp;amp; \text{...} &amp;amp; z_m \\&#xA;\end{array}&#xA;\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a set of $N$ numbers $S=\{f_1,...,f_N\}$ with $N&amp;gt;m$, I would like to find the $m$-element ordered subset $S_\text{opt}=\left\{f_{\sigma (1)},f_{\sigma&#xA;   (2)},\text{...},f_{\sigma (m)}\right\}$ of $S$ (where $\sigma:\{1,...,m\}\rightarrow\{1,...,N\}$ is an injective function) such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{Y}(\sigma)=\mathbf{X}+\left(&#xA;\begin{array}{cccc}&#xA; 0 &amp;amp; 0 &amp;amp; \text{...} &amp;amp; 0 \\&#xA; 0 &amp;amp; 0 &amp;amp; \text{...} &amp;amp; 0 \\&#xA; f_{\sigma (1)} &amp;amp; f_{\sigma (2)} &amp;amp; \text{...} &amp;amp;&#xA;   f_{\sigma (m)} \\&#xA;\end{array}&#xA;\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;most closely&quot; resembles a plane. To be precise, I would like $\sigma$ to minimize the RMS error between the plane of best linear fit and the set of points $\mathbf{Y}(\sigma)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know from linear algebra that given a set of points in 3-space $\mathbf{Y}(\sigma)$, the RMS error of the plane of best fit is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\epsilon(\sigma)=\sqrt{\Sigma_3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Sigma_3$ is the 3rd singular value of $\mathbf{Y}(\sigma)-\mu(\mathbf{Y})$ (where the singular values are ordered from largest to smallest), where $\mu(\mathbf{Y})$ is the centroid of the points $\mathbf{Y}(\sigma)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know that the number of possible choices of $\sigma$ is given by &#xA;$$\frac{N!}{(N-m)!}$$&#xA;which means that brute-force enumeration is out of the question (for the application I am looking to use this in, $N=47$ and $m=19$, giving 848255586288593837691617280000 combinations).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there some method of reducing the size of the computation space for this sort of problem? Initially I briefly thought that it might be a simple linear programming problem, since we're finding an optimal choice of members of $S$ to make $\mathbf{Y}=\mathbf{X}+S_\text{opt}$ as flat as possible, but obviously the problem isn't linear... right?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What general class of optimization does this sort of procedure fall under? I am somewhat inexperienced in scientific computing, and thus have no idea what heading this falls under.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If it is any help, the initial set $\mathbf{X}$ already closely resembles points in an $xy$ plane, and the $f_{\sigma(k)}$ are designed to be tiny $z$-axis height corrections to make the resulting set of points become more closely planar.&lt;/p&gt;&#xA;" OwnerUserId="6928" LastEditorUserId="6928" LastEditDate="2014-10-21T01:28:45.993" LastActivityDate="2014-10-21T02:15:50.703" Title="Combinatorial optimization problem: choose a set of corrective factors to make a set of points most closely resemble a plane" Tags="|optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15932" PostTypeId="1" AcceptedAnswerId="15941" CreationDate="2014-10-21T02:00:41.507" Score="0" ViewCount="1293" Body="&lt;p&gt;I am trying to use B spline curve fitting. The order of B-spline curve is 4. When I have many control points, it works well. &#xA;However if the number of control points is small such as two, my  program will crash. I realize that the number of control points is related to number of knots and the order. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me clarify the relationship or give some links on it?&lt;/p&gt;&#xA;" OwnerUserId="7846" LastEditorUserId="4964" LastEditDate="2014-10-23T07:56:59.310" LastActivityDate="2014-10-23T07:56:59.310" Title="Number of control points for B-spline curve" Tags="|b-spline|curve-fitting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15933" PostTypeId="2" ParentId="15931" CreationDate="2014-10-21T02:09:11.767" Score="3" Body="&lt;p&gt;This looks like a very programming-contest-type problem. Let the OLS-fit plane be $z=\alpha x+\beta y$, so that we would like to minimize&#xA;$$ \sum_k \big(z_k + f_{\sigma(k)} - \alpha x_k - \beta y_k\big)^2, $$&#xA;over $\sigma$. After computing $\alpha,\beta$, and by defining $g_k$ appropriately, this is the same as minimizing&#xA;$$ \sum_k \big(f_{\sigma(k)} - g_k\big)^2 $$&#xA;over $\sigma$, where $g_k$ is a given set of real numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of writing it this way is that this makes it clear that this is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Assignment_problem&quot; rel=&quot;nofollow&quot;&gt;optimal assignment problem&lt;/a&gt;, in a graph where each node $g_j$ is connected to each node $f_i$ with an edge that has weight (or &quot;cost&quot;) $C_{i,j}=|f_i-g_j|^2$. This is a &lt;a href=&quot;http://en.wikipedia.org/wiki/Matching_(graph_theory)#In_weighted_bipartite_graphs&quot; rel=&quot;nofollow&quot;&gt;maximum matching problem in a complete weighted bipartite graph&lt;/a&gt;. It can be solved with the &lt;a href=&quot;http://en.wikipedia.org/wiki/Hungarian_algorithm&quot; rel=&quot;nofollow&quot;&gt;Hungarian algorithm&lt;/a&gt; (see also &lt;a href=&quot;http://community.topcoder.com/tc?module=Static&amp;amp;d1=tutorials&amp;amp;d2=hungarianAlgorithm&quot; rel=&quot;nofollow&quot;&gt;this tutorial on TopCoder&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;P.S.&lt;/em&gt; If I misunderstood the problem description, this exact same approach works so long as the costs are additive, and then $C_{i,j}$ is the cost of assigning $f_i$ to the point $j$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-10-21T02:15:50.703" LastActivityDate="2014-10-21T02:15:50.703" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="15934" PostTypeId="1" AcceptedAnswerId="15937" CreationDate="2014-10-21T03:19:01.720" Score="6" ViewCount="15698" Body="&lt;p&gt;What is the difference between convergence and asymptotic convergence? Why say the convergence is asymptotic?&lt;/p&gt;&#xA;" OwnerUserId="8213" LastActivityDate="2014-10-21T03:41:17.407" Title="The definition of asymptotic convergence?" Tags="|convergence|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15935" PostTypeId="2" ParentId="15926" CreationDate="2014-10-21T03:38:44.980" Score="8" Body="&lt;p&gt;Semidefinite programming and second order cone programming have not been adopted as rapidly in practice as many of us hoped.  I've been involved in this for the last 20 years, and it has been very disappointing to see slow progress.  Let me point out some of the challenges:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Although we have polynomial time algorithms for SDP and SOCP, the widely used primal-dual interior point methods typically require $O(m^{2})$ storage where $m$ is the number of constraints.  This makes solving problems with 50,000 constraints possible but solving problems with 500,000 constraints impractical today.  Of course, memory capacity keep growing exponentially, so many important problems will eventually be solvable, but there are many problems that aren't practically solvable today or in the near future. First order methods that don't have $O(m^{2})$ storage requirements are an active topic of research but in the area of SDP they simply haven't proven to be robust enough for use in a general purpose solver.   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The vendors of LP software haven't seen fit to include support for SDP in their products yet.  Some limited support for SOCP is starting to appear.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Knowledge about semidefinite programming has spread slowly.  The textbook by Boyd and Vandenberghe has been hugely helpful in this respect, but there's a long way to go before this technology will be as widely known as older optimization techniques.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Modelling languages and systems (such as GAMS, AMPL, etc.) don't yet provide good support for SOCP and SDP.  The CVX package is the most interesting work in this direction, but even it requires some sophistication on the part of the user.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;SDP has found applications at the research level in many areas of engineering and science.  It seems likely that these will eventually become important in industry as well.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-10-21T03:38:44.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15936" PostTypeId="1" AcceptedAnswerId="15938" CreationDate="2014-10-21T03:40:50.253" Score="5" ViewCount="4033" Body="&lt;p&gt;Is the Crank-Nicolson method appropriate for solving a system of nonlinear parabolic PDEs like $\partial u/\partial t - a\Delta u + u^4 = 0$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to apply this method for solving such system but the solution was oscillating (maybe because of a small value of the coefficient of the time derivative) and the implicit Euler method calculates a correct solution.&lt;/p&gt;&#xA;" OwnerUserId="11531" LastActivityDate="2014-10-22T16:37:42.360" Title="Crank-Nicolson method for solving nonlinear parabolic PDEs" Tags="|pde|finite-difference|parabolic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15937" PostTypeId="2" ParentId="15934" CreationDate="2014-10-21T03:41:17.407" Score="3" Body="&lt;p&gt;Typically convergence refers to the error decreasing with the fineness of the discretization; i.e. for finite difference/finite elements, this tends do deal with grid spacing $h$ and claims that error goes to $0$ as $h \rightarrow 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually asymptotic convergence refers to a convergence behavior that is observed when $h$ is sufficiently small. One example is with finite differences: writing out a Taylor series &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x+h) = u(x) + u'(x)h + \frac{u''(x)}{2}h^2 + \ldots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can rearrange to get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left|\frac{u(x+h) - u(x)}{h} - u'(x)\right| = \left|\frac{u''(x)}{2}h + \frac{u'''(x)}{3!}h^2\ldots\right|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives us an exact expression for finite difference error. For $h$ small enough, $h^2 \ll h$, and the leading term $\frac{u''(x)}{2}h = O(h)$ will dominate the above expression. However, this is only true if $h$ is small enough, which implies that this is an asymptotic result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An additional issue is that, when solving PDEs, sometimes there may be parameters which may influence the convergence of the method. For example, for the Helmholtz equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa^2 u + \Delta u = f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this parameter is the wavenumber $\kappa$, and for convection-diffusion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla\cdot(\beta u) - \epsilon \Delta u = f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this parameter is $\epsilon$. Most discretizations achieve asymptotically (quasi)optimal behavior, that as $h$ is small enough (or as the discretization becomes fine enough), the solution will behave a certain way.  However, this asymptotic limit for where $h$ is small enough can also depend on these parameters; asymptotic convergence is helpful, but it doesn't tell you everything.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2014-10-21T03:41:17.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15938" PostTypeId="2" ParentId="15936" CreationDate="2014-10-21T04:14:43.733" Score="10" Body="&lt;p&gt;This phenomenon is often called &quot;ringing&quot; and plagues methods that are not $L$-stable.  This can be seen in this motivating example from Hairer &amp;amp; Wanner (1999) &quot;Stiff differential equations solved by Radau methods&quot;.  Consider the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \dot y = -50 (y - \cos t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and apply explicit Euler with time step near the stability limit, implicit midpoint (or equivalently for this problem, trapezoid rule, aka. Crank-Nicolson), and implicit Euler.  The result, shown below, illustrates that the $A$-stable (but not $L$-stable) implicit midpoint method produces a poor-quality solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bUbaO.jpg&quot; alt=&quot;Figure 1 of Hairer and Wanner (1999)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid this problem when solving stiff systems, you should use an $L$-stable method such as BDF-2, a suitable DIRK, or a Radau method.  See Hairer and Wanner's second volume for extensive discussion of this topic.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="119" LastEditDate="2014-10-22T16:37:42.360" LastActivityDate="2014-10-22T16:37:42.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15939" PostTypeId="1" AcceptedAnswerId="15982" CreationDate="2014-10-21T06:13:17.547" Score="1" ViewCount="617" Body="&lt;p&gt;I need to solve the following hyperbolic equation in x and phi co-ordinates&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \left ( -s/f \right )}{\partial \varphi }+\frac{\partial \left ( 1/f \right )}{\partial x}=0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\varphi &amp;gt;0\rightarrow f=\left ( 1+\frac{\left ( 1-s \right )\left ( 1+0.4\varphi e^{-x} \right )}{10s} \right )^{-1} $$   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\varphi &amp;lt;0\rightarrow f=\left ( 1+\frac{1-s}{10s} \right )^{-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\varphi =-0.1x\rightarrow s=0.1,f=0.526$$&#xA;  $$x=0,\varphi &amp;gt;0\rightarrow s=1,f=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in domain $$0&amp;lt;x&amp;lt;1$$ $$-0.1x&amp;lt;\varphi &amp;lt;1         &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am searching for the hyperbolic package that can solve this equation like clawpack, but I don't know how to define my equation there&#xA;thanks &lt;/p&gt;&#xA;" OwnerUserId="11554" LastEditorUserId="123" LastEditDate="2014-10-26T05:19:22.777" LastActivityDate="2014-10-28T13:14:13.067" Title="How can I solve this 1D nonlinear, variable-coefficient hyperbolic PDE?" Tags="|pde|hyperbolic-pde|" AnswerCount="2" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15940" PostTypeId="2" ParentId="14588" CreationDate="2014-10-21T07:57:18.200" Score="5" Body="&lt;p&gt;You haven't specified boundary conditions, but it sounds like you want to solve the Cauchy problem (i.e., the initial value problem on the whole real line $\infty &amp;lt; z &amp;lt; \infty$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case, there is no need to use finite differences.  You can simply use Fourier analysis (separation of variables) to write down the exact answer.  Once you have that, you can evaluate it numerically by truncating the domain and the set of Fourier modes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you insert the ansatz&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(z,t) = \exp(i(\xi x - \omega t))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;into your wave equation, you find there are two frequencies of propagation for each wavenumber $\xi$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega(\xi) = \pm \sqrt{a \xi^2 - k \xi^4}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These correspond to left- and right-going waves.  To write down the full solution you need to determine which part of the initial data goes each way.  That depends on $u_t(z,0)$, which you haven't specified.  Without that information, the problem is not well-posed.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-10-21T07:57:18.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15941" PostTypeId="2" ParentId="15932" CreationDate="2014-10-21T08:12:05.420" Score="3" Body="&lt;p&gt;The formula is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$m = n + p + 1$ &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$m$ number of knots. &lt;/li&gt;&#xA;&lt;li&gt;$n$ number of control points.&lt;/li&gt;&#xA;&lt;li&gt;$p$ degree.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can check the &lt;a href=&quot;http://books.google.it/books/about/The_NURBS_Book.html?id=7dqY5dyAwWkC&amp;amp;redir_esc=y&quot; rel=&quot;nofollow&quot;&gt;nurbs book&lt;/a&gt; chapter 2 for a complete set of definitions. The &lt;a href=&quot;http://www.cambridge.org/it/academic/subjects/mathematics/numerical-analysis/spline-functions-basic-theory-3rd-edition?format=PB&quot; rel=&quot;nofollow&quot;&gt;Shumaker's&lt;/a&gt; book &#xA;is a more readable reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a paragraph on interpolation also on &lt;a href=&quot;http://www.uio.no/studier/emner/matnat/ifi/INF-MAT5340/v09/undervisningsmateriale/book.pdf&quot; rel=&quot;nofollow&quot;&gt;Tom Lyche and Knut Mørgens's lecture notes&lt;/a&gt;. Here you shall find that all the entries in your matrix are positive, so the linear algebra should be ok. Donno what about the rest of your code.&lt;/p&gt;&#xA;" OwnerUserId="6574" LastEditorUserId="6574" LastEditDate="2014-10-21T08:18:09.913" LastActivityDate="2014-10-21T08:18:09.913" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15942" PostTypeId="1" CreationDate="2014-10-21T08:44:18.997" Score="2" ViewCount="156" Body="&lt;p&gt;What ODE solvers are optimized for solving stiff systems with sparse Jacobian? Such systems appear, for instance, when a parabolic PDE is discretized in space using typical finite difference or finite element methods.&lt;/p&gt;&#xA;" OwnerUserId="11531" LastEditorUserId="123" LastEditDate="2014-10-22T05:25:54.007" LastActivityDate="2014-10-22T05:25:54.007" Title="Solvers for stiff initial value ODEs with sparse Jacobian" Tags="|pde|ode|sparse-matrix|parabolic-pde|method-of-lines|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15944" PostTypeId="1" CreationDate="2014-10-02T12:39:10.710" Score="2" ViewCount="33" Body="&lt;p&gt;So, lets say for the family of the explicit Runge-Kutta methods:&#xA;$$y_{n+1} = y_n + \sum_{i=1}^s b_i k_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&#xA;$$k_1 = hf(t_n, y_n)$$&#xA;$$k_2 = hf(t_n+c_2h, y_n+a_{21}k_1)$$&#xA;$$\vdots$$&#xA;$$k_s = hf(t_n+c_sh, y_n+a_{s1}k_1+a_{s2}k_2+\cdots+a_{s,s-1}k_{s-1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any high-order Runge-Kutta scheme (preferably more than 3rd order) where either all the nodes are found on $c_i=1,$ for $i=2,...,s$ OR if $c_m\neq{1},m=2...s$ then the weight on this node should be zero $(b_m=0)$?&lt;/p&gt;&#xA;" OwnerDisplayName="runge_kutta_question" LastActivityDate="2014-10-22T07:55:29.057" Title="Runge-Kutta with all nodes at n+1 or zero weights otherwise" Tags="|reference-request|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15945" PostTypeId="2" ParentId="15944" CreationDate="2014-10-05T18:03:17.307" Score="1" Body="&lt;p&gt;The following are necessary (though not sufficient) conditions for a method to have order $p$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{i=1}^s b_i c_i^{k-1} = \frac{1}{k}, \ \ \ \ \ \ \ \ \ \ k=1,2,\dots,p.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A method with the properties you specify would have, for any $k&amp;gt;1$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{i=1}^s b_i c_i^{k-1} = \sum_{i\notin J}^s b_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the sum on the right is independent of $k$ (here $J$ is the set of indices for which $c_i=0$).  To attain order three, this sum would have to be equal to both $1/2$ and $1/3$.  So such methods can have order at most two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is really a statement about quadrature rules (more general than the statement about Runge-Kutta methods).&lt;/p&gt;&#xA;" OwnerUserId="123" OwnerDisplayName="David Ketcheson" LastEditorUserId="123" LastEditDate="2014-10-22T07:55:29.057" LastActivityDate="2014-10-22T07:55:29.057" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15946" PostTypeId="2" ParentId="15944" CreationDate="2014-10-13T15:03:59.890" Score="-2" Body="&lt;p&gt;could you give me a reference of where you can find these conditions to achieve order p? I am not sure if it's valid though for all the methods. For example, the classic RK4 method that has $$b_1=0,\,b2=0.5,\,b_3=0.5,\,b_4=1,\,and\,c_1=1/6,\,c_2=1/3,\,c_3=1/3,\,c_4=1/6$$&#xA;Then if you evaluate for $k=4$ (since its fourth-order) the above expression that you mentioned gives:&#xA;$$\sum_{i=1}^s b_i c_i^{k-1} = \frac{1}{2} \neq 1/4 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, there is no way for example to take $b_1=0, b_2 = b_3 = b_4 = 1.0$ and find the proper $c_i$ in order to have 4-th order accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerDisplayName="runge_kutta_question" LastActivityDate="2014-10-13T15:03:59.890" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15947" PostTypeId="1" CreationDate="2014-10-21T11:57:42.837" Score="2" ViewCount="71" Body="&lt;p&gt;Assume that we have the following equation and the following assumption. The scope is to solve for some particular variables expressed later. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E_{t}\left[ b(A_{t+1})^{1-\gamma} *R_{t+1}^{-\gamma}*[r_k(A_{t+1},a)-r_h(A_{t+1})+\eta)] \right]=0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$R=(1-\theta_1-\theta_2)*r^f + \theta_1*r_k(A,a)+\theta_2 *[r_h(A)+\eta]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also consider that the number &lt;span class=&quot;math-container&quot;&gt;$b(A)$&lt;/span&gt; is pinned down recursively by the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$b_t = G[E_t(b_{t+1}*R_{t+1}^{1-\gamma})]$$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$G(\cdot)$&lt;/span&gt; is a known function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Main Assumptions&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A_t$&lt;/span&gt; is a random variable that follows a Markov process, and we know its value at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, but not at time &lt;span class=&quot;math-container&quot;&gt;$t+1$&lt;/span&gt;.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$IID$&lt;/span&gt; random variable &lt;em&gt;lognormally&lt;/em&gt; distributed with some mean &lt;span class=&quot;math-container&quot;&gt;$\mu_a$&lt;/span&gt; and variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2_a$&lt;/span&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt; is another &lt;span class=&quot;math-container&quot;&gt;$IID$&lt;/span&gt; random variable &lt;em&gt;normally&lt;/em&gt; distributed with some  mean &lt;span class=&quot;math-container&quot;&gt;$\mu_{\eta}$&lt;/span&gt; and variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2_\eta$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The RVs are independent of each other. That is, &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is independent of &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt;, while these two can be &lt;em&gt;correlated&lt;/em&gt; with &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, thought are still independently drawn from &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$b(A)$&lt;/span&gt; is a number that depends on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;em&gt;I do not know their functional relationship&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r^f$&lt;/span&gt; is a known number.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;by &lt;span class=&quot;math-container&quot;&gt;$r_k(A,a) $&lt;/span&gt; and respectively &lt;span class=&quot;math-container&quot;&gt;$r_h(A)$&lt;/span&gt; I mean a functional relationship (which I know) between these two that depend on &lt;span class=&quot;math-container&quot;&gt;$A, a$&lt;/span&gt;  and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; respectively. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To sum up, the expectations operator is about the unknowns in the next period &lt;span class=&quot;math-container&quot;&gt;$A,a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt;. To fix the ideas. Let's suppose a simple case, where &lt;span class=&quot;math-container&quot;&gt;$a,\eta$&lt;/span&gt; take two values and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; follows a two-state Markov process and also take two values. Let's also suppose that at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; we are in one of these states. In that case, the Expectations operator is defined over the next possible state for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, and the possible values for the &lt;span class=&quot;math-container&quot;&gt;$IID$&lt;/span&gt; RVs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;THE COMPUTATIONAL TASK:&lt;/strong&gt; Solve for &lt;span class=&quot;math-container&quot;&gt;$\theta_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta_2$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently, the solution for each &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; will depend on the possible values of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, so I expect to be a vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any guidance on how to solve this equation will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9826" LastEditorUserId="20688" LastEditDate="2019-08-06T03:04:44.643" LastActivityDate="2019-08-06T03:04:44.643" Title="Solving a nonlinear equation with a Markov process and RVs" Tags="|nonlinear-equations|stochastic|roots|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="15948" PostTypeId="1" AcceptedAnswerId="15949" CreationDate="2014-10-21T13:56:55.723" Score="0" ViewCount="45" Body="&lt;p&gt;So my fellow numerical computational peeps it may be that I am suffering from sleep deprivation but I'm struggling to numerically compute a function $u \rightarrow h(u)$ defined implicitly as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u^{h(u)} + u^{-h(u)} = a u&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some constant $u$. To be clear, that's just saying $u$ raised to $h(u)$ etc, and this should hold for all u. We can just consider $u&amp;gt;0$ and $h(u)&amp;gt;0$ actually. It is clear that for large $u$ we can come close by ignoring the second term on the left. So perhaps the question is best phrased as &quot;how to perturb that case to better solve the above&quot;. My instinct was to shove the $u^{-h(u)}$ on the other side, solve for $h(u)$ and iterate - but perhaps someone has a better idea.   &lt;/p&gt;&#xA;" OwnerUserId="11573" LastEditorUserId="27" LastEditDate="2014-10-21T14:27:13.923" LastActivityDate="2014-10-21T15:03:41.390" Title="Implicitly defined univariate function" Tags="|numerics|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15949" PostTypeId="2" ParentId="15948" CreationDate="2014-10-21T15:03:41.390" Score="2" Body="&lt;p&gt;I can't help you at the moment with a numeric procedure to find h, but&#xA;an analytical solution, at least for $u&amp;gt;1$, reads: &#xA;$$&#xA;h=\frac{\mathrm{arcosh}\left(\frac{1}{2}\cdot a \cdot u\right)}{\ln(u)}&#xA;$$&#xA;For $u\in(0,1)$ you can try $u=1/x$ with $x \in (1,\infty)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you can work with that. I'm curious about what you try to model with this thing.&lt;/p&gt;&#xA;" OwnerUserId="11568" LastActivityDate="2014-10-21T15:03:41.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15950" PostTypeId="1" AcceptedAnswerId="15952" CreationDate="2014-10-21T17:58:07.983" Score="0" ViewCount="1270" Body="&lt;p&gt;Does anyone have any experience on Domain Decomposition using PETSc library? I have used PETSc for creating my vectors and matrix within my C++ code. I also used KSP to solve the linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used following examples to learn about Domain Decomposition in PETSc but they were not helpful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.5/src/snes/examples/tutorials/ex10d/ex10.c.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-3.5/src/snes/examples/tutorials/ex10d/ex10.c.html&lt;/a&gt; &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.2/src/snes/examples/tutorials/ex12.c.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-3.2/src/snes/examples/tutorials/ex12.c.html&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be grateful if you give me some hints on how to do the graph partitioning and domain decomposition with PETSc.&lt;/p&gt;&#xA;" OwnerUserId="11575" LastEditorUserId="9667" LastEditDate="2014-10-21T20:56:30.200" LastActivityDate="2014-10-21T20:56:44.643" Title="Domain Decomposition with PETSc" Tags="|c++|petsc|mpi|domain-decomposition|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15952" PostTypeId="2" ParentId="15950" CreationDate="2014-10-21T20:56:44.643" Score="0" Body="&lt;p&gt;I suggest understanding a structured grid example before getting into unstructured.  So start with something like &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.5/src/snes/examples/tutorials/ex5.c.html&quot; rel=&quot;nofollow&quot;&gt;snes/examples/tutorials/ex5.c&lt;/a&gt;.  For unstructured, you can manage everything, as in the ex10d that was removed because it was a bad example, or you can use DMPlex, the component that manages unstructured meshes in PETSc.  The ex12 you mention is an example of this, but it hides a lot of details and I don't think it's great for beginners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question is not very specific so I don't know what you're having a hard time with.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-10-21T20:56:44.643" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15955" PostTypeId="1" CreationDate="2014-10-22T08:53:05.433" Score="3" ViewCount="107" Body="&lt;p&gt;Does someone know a method to get cheap approximation of harmonic problems  (and possibly local approximations)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me explain: I need to compute the solution of an harmonic problem&#xA;\begin{equation} div(\alpha(x) \nabla u) = 0, \quad x \in \Omega \\ u(x) = x,\quad x \in \partial \Omega\end{equation}&#xA;on a very fine mesh $\mathcal T_f$. However I might only be interested by local approximations, ie on convex subdomains of $\mathcal T_f$ of $u(x)$ without the agonizing pain to solve for $u$ on the whole domain. Does some Green functions exists for such problem ? Is there any method to compute cheap approximations? The diffusion tensor $\alpha(x)$ can be any integrable function (no smoothness property assumed here, the equation must be interpreted in its weak formulation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that respect, &quot;cheap&quot; can mean many thing, but what I mainly want is to have something cheap to compute that still retains an &quot;acceptable&quot; accuracy in my subdomains of interest. Since I will have to compute such approximations in many subdomains (in fact the collection of subdomains covers the domain itself), this is why I want it to be &quot;cheap&quot;. An analytical formulation, or hybrid one would be fine, mesh refinement in the vicinity of my subdomain as well (and this is well known I agree) while keeping a coarse resolution elsewhere. &lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="123" LastEditDate="2014-10-26T05:14:48.703" LastActivityDate="2014-10-26T05:14:48.703" Title="Fast methods to solve an elliptic PDE if high accuracy is needed only in part of the domain" Tags="|adaptive-mesh-refinement|elliptic-pde|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="15956" PostTypeId="2" ParentId="15913" CreationDate="2014-10-22T11:04:44.593" Score="1" Body="&lt;p&gt;This method makes few or no assumptions about the problem being optimized and can search very large spaces of candidate solutions. However, DE does not guarantee that an optimal solution is ever found for certain types of problems. DE is used for multidimensional real-valued functions but does not use the gradient of the problem being optimized, which means DE does not require for the optimization problem to be differentiable, as is required by classic optimization methods, such as gradient descent and quasi-newton methods. DE can, therefore, also be used on optimization problems that are not even continuous, noisy and change over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The generation size heavily depends on the tolerance of error (precision) you want to accept as well as the hardness of the problem. In addition, Crossover Rate (CR) is also an important parameter especially when CR≈0, DE makes very small exploratory moves. The search proceeds in a gradual but consistent fashion as the likelihood of making an improving move is higher when moves are small, even if the change in solution quality is not great. When CR≈0.9, DE makes large exploratory moves that, while being less likely to be improving, can yield large improvements in solution quality. These large moves also reduce population diversity, which is a necessary step so that subsequent moves are scaled appropriately for performing a more ﬁne-grained search of the solution space. When CR≈0.5, DE behaves more similarly to when CR = 0.9 than when CR = 0. Large moves with large improvements also result in a reduction in population diversity, yet it appears plausible that the population is often still too spread for difference vectors to be scaled appropriately to continue the search.&lt;/p&gt;&#xA;" OwnerUserId="9948" LastActivityDate="2014-10-22T11:04:44.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15957" PostTypeId="2" ParentId="15955" CreationDate="2014-10-22T11:12:36.397" Score="4" Body="&lt;p&gt;This would call for adaptive finite element methods, and if you are only interested in specific regions, for the use of goal oriented error estimators to drive the adaptive mesh refinement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem you want to solve is a pretty standard one and you can find my own contribution to this in the step-6 tutorial program of the deal.II library for the general case of solving this equation (&lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_6.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_6.html&lt;/a&gt;) and in step-14 for the goal oriented adaptivity (&lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_14.html&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org/developer/doxygen/deal.II/step_14.html&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-22T11:12:36.397" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15958" PostTypeId="2" ParentId="15936" CreationDate="2014-10-22T11:36:14.857" Score="1" Body="&lt;p&gt;Theoretically, if the implicit Euler method works for this equation,&#xA;Crank--Nicolson scheme should also work. let $\tau$ be the step in time and if we only consider the temporal discretization, the linearized Crank--Nicolson scheme is given by &#xA;$$&#xA;\frac{u^n-u^{n-1}}{\tau} - \frac{a}{2} ( \Delta u^n+ \Delta u^{n-1} )&#xA;= -(\frac{3 u^{n-1}-u^{n-2}}{2})^4 .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above scheme is okay under the condition that the regularity of the equation is good. Moreover, spatial approximation does not affect the stability of the scheme and you can use both finite difference and finite element methods to solve it. &lt;/p&gt;&#xA;" OwnerUserId="4457" LastActivityDate="2014-10-22T11:36:14.857" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="15960" PostTypeId="1" CreationDate="2014-10-22T20:37:32.210" Score="2" ViewCount="279" Body="&lt;p&gt;I want to maximize the sum of Rayleigh quotients:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_x\sum_{i=1}^n\frac{x^\top A_i x}{x^\top B_i x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_i$ and $B_i$ is positive definite. I've found a similar question here:&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/5232/minimization-problem-sum-of-rayleigh-quotients?newreg=ce231f04d6284e5281deee8c7c1db53a&quot;&gt;minimization problem: sum of Rayleigh quotients&lt;/a&gt;. But it only deals with the case $n=2$, and therefore it seems that the methods there are not applicable for my problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any suggestion for this problem? Does gradient descent/ascent work for this problem?   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="11587" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-10-22T23:24:37.650" Title="Maximize sum of Rayleigh quotients" Tags="|optimization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15961" PostTypeId="1" AcceptedAnswerId="15962" CreationDate="2014-10-23T13:55:55.077" Score="8" ViewCount="4492" Body="&lt;p&gt;&lt;strong&gt;Goal:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to evaluate numerically an integral of the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_0^\infty \frac{dx}{(a^2+x)\sqrt{(a^2+x)(b^2+x)(c^2+x)}}&#xA;$$&#xA;where $a,b,c \in \mathbb{R}$ are in the interval $(1,1000)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This needs to be done in Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integration algorithm does not converge, but I know the integral is convergent. To my knowledge there is only one package available that handles improper integrals: &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/tutorial/integrate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.integrate.quad&lt;/a&gt;. This method does not converge for the following values: $a=240,b=110,c=100.$ I receive the warning ``IntegrationWarning: The integral is probably divergent, or slowly convergent.'' The integral evaluates to $1.36 \times 10^{-9}$. Computing the same integral in Matlab and Mathematica results in a value of $1.13 \times 10^{-7}$ in each case, with no warnings or errors. This strongly suggests that the integral converges just fine, and that the problem lies in scipy's quadrature routine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Attempts at a Solution:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper from which I took this integral indicates that it is elliptic. There exist several methods to integrate such functions numerically; however, I cannot find any standard elliptic integrals of this form (checking, for example, the discussion on &lt;a href=&quot;http://mathworld.wolfram.com/EllipticIntegral.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;mathworld&lt;/a&gt; and also posts such as &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11722/numerical-evaluation-of-the-first-and-second-complete-elliptic-integrals&quot;&gt;this&lt;/a&gt; one on these forums). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Others have had problems with this package also; for example, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11086/numerical-integration-with-convergence-factor-with-scipy-problem-with-improper&quot;&gt;this&lt;/a&gt; post. The accepted solution in this case is to convert the the integral into an ODE. I have &lt;a href=&quot;https://mathoverflow.net/questions/47658/why-not-evaluate-integrals-using-ode-solvers&quot;&gt;seen&lt;/a&gt;, however, that this can be quite dangerous.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also considered a change of variables in order to transform this into an elliptic form; I cannot find any. For example, an obvious one, $x \to y^2$, results in an undesirable factor of $y$ in the numerator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I think I need:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe I will need to implement some sort of quadrature routine manually. Are there any that might be well-suited to this (seemingly benign) integral?&lt;/p&gt;&#xA;" OwnerUserId="9599" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2014-10-24T08:57:51.070" Title="Numerical evaluation of an elliptic integral in python" Tags="|python|quadrature|scipy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15962" PostTypeId="2" ParentId="15961" CreationDate="2014-10-23T15:04:34.130" Score="8" Body="&lt;p&gt;The problem is almost definitely with how QUADPACK (&lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.integrate.quad.html#scipy.integrate.quad&quot;&gt;which is the backend used by &lt;code&gt;scipy.integrate.quad&lt;/code&gt;&lt;/a&gt;) handles numerically small integrands. Essentially the integrand is so small (at $x=0$ it is $6.58\times 10^{-12}$), that it is comparable to the absolute error tolerance. I'm not sure quite why it says it's &quot;probably &lt;em&gt;divergent&lt;/em&gt;&quot;, but that doesn't matter. I suspect that when the error estimate is comparable with the integral's estimated value, it concludes that the most common explanation is that the integral diverges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several things to try. First is to try and specify the parameter &lt;code&gt;epsabs=0&lt;/code&gt; (by default it is something like $10^{-8}$) to tell it to ignore absolute error completely, and use only relative error. This seems to fix the problem, and is by far the best approach. By comparing the result with the output from &lt;code&gt;mpmath&lt;/code&gt;, the result of &lt;code&gt;quad&lt;/code&gt; with all other default settings is correct to 13 digits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second is to rewrite the integral as&#xA;$$ \frac{1}{a^3bc} \int \frac{1}{(1+x/a^2)\sqrt{(1+x/a^2)(1+x/b^2)(1+x/c^2)}}. $$&#xA;Then the integrand is $O(1)$, which doesn't confuse the integration routine. This also fixes the problem, giving the same result as with &lt;code&gt;epsabs=0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The third option is to unleash the full machinery of elliptic integrals on this. It is, as far as I know, somewhat tricky to reduce a general-form elliptic integral to the standard elliptic integrals. It can certainly be done, though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral is, in the notation of &lt;a href=&quot;http://dlmf.nist.gov/19.16&quot;&gt;DLMF 19&lt;/a&gt;&#xA;$$ \frac{2}{3} R_D(c^2, b^2, a^2). $$&#xA;The function $R_D(x, y, z)$ satisfies the recurrence relation &lt;a href=&quot;http://dlmf.nist.gov/19.26.E20&quot;&gt;DLMF 19.26.E20&lt;/a&gt;&#xA;$$ R_D(x, y, z) = 2R_D(x+\lambda, y+\lambda, z+\lambda) + \frac{3}{\sqrt{z}(z+\lambda)} $$&#xA;$$ \lambda = \sqrt{x}\sqrt{y} + \sqrt{y}\sqrt{z} + \sqrt{z}\sqrt{x}, $$&#xA;and in the limit $x,y,z\to+\infty$, $R_D(x,y,z)$ approaches 0. It is sufficient to apply this duplication formula about $30$ times, and take the single remaining elliptic integral $R_D$ with very large arguments as approximately zero, for your values of $x=c^2$, $y=b^2$, $z=a^2$, before it converges to machine precision to&#xA;$$ 1.1322299840381455\times10^{-7}. $$&#xA;This will work more slowly for smaller values of $a,b,c$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-10-23T16:01:47.833" LastActivityDate="2014-10-23T16:01:47.833" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15963" PostTypeId="1" CreationDate="2014-10-23T19:26:23.203" Score="3" ViewCount="400" Body="&lt;p&gt;I am modifying a conservative form of the Euler equations with gravity in the energy flux (see previous question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14673/energy-conservation-in-conservation-laws-with-source-terms&quot;&gt;Energy Conservation in Conservation Laws with Source Terms&lt;/a&gt;) for use in a Riemann solver.  I'm seeing some unphysical/unstable results despite total energy conservation to roundoff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conservation law equation with source term (a.k.a balance law)&#xA;$\frac{\partial \vec{Q}}{\partial t} + \nabla \cdot \vec{F} = \vec{S}  $, is solved, where the variables are defined in $\vec{Q}$,the flux, $\vec{F_z}$ in the $z$-direction, and the source $\vec{S}$ of modified equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \vec{Q}= \left[\begin{matrix} \rho \\ \rho u \\ \rho v \\ \rho w \\ E_{total} \end{matrix}\right] \quad \quad \vec{F_z}= \left[\begin{matrix} \rho w \\ \rho uw \\ \rho vw \\ \rho w^2+p \\ (E_{total} + p)w \end{matrix}\right] \quad \quad \vec{S}= \left[\begin{matrix} 0 \\ 0 \\ 0 \\ -\rho g \\ 0 \end{matrix}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\rho$ is the density, $u$,$v$, and $w$ are velocities in the $x$, $y$ and $z$ directions respectively.  The total energy is defined as $E_{total} = \frac{p}{\gamma-1} + \frac{1}{2}\rho q^2 + \rho\psi$, with $q^2=u^2+v^2+w^2$, and $\psi=gz$, with $g$ being the acceleration due to gravity in the $z$-direction, and $\gamma$ is the ratio of specific heats.  The pressure $p$ can be defined from the total energy, $p=(\gamma-1)(E_{total} - \frac{1}{2}\rho q^2 - \rho\psi)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find the flux Jacobian, $A = \frac{\partial \vec{F_z}}{\partial \vec{Q}}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A = \left[\begin{matrix} &#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 \\&#xA;-uw &amp;amp; w &amp;amp; 0 &amp;amp; u &amp;amp; 0 \\&#xA;-vw &amp;amp; 0 &amp;amp; w &amp;amp; v &amp;amp; 0 \\&#xA;(\gamma-1)(\frac{1}{2} q^2 - \psi) - w^2 &amp;amp; -(\gamma-1)u &amp;amp; - (\gamma-1)v &amp;amp; (3-\gamma)w&amp;amp; (\gamma-1) \\&#xA;[(\gamma-1)(\frac{1}{2} q^2 - \psi) - h']w &amp;amp; -(\gamma-1)uw &amp;amp; -(\gamma-1)vw &amp;amp;  h'-(\gamma-1)w^2 &amp;amp; \gamma w&#xA; \end{matrix}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h' = h + \psi$, with $h = \frac{a^2}{\gamma-1} + \frac{1}{2}q^2$, and $a^2 = \frac{\gamma p}{\rho}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This systems yields eigenvalues $\lambda =[u-a, u,u,u,u+a] $, and right eigenvectors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R = \left[ \begin{matrix}&#xA;1    &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 \\&#xA;u    &amp;amp; 1 &amp;amp; 0 &amp;amp; u &amp;amp; u \\&#xA;v    &amp;amp; 0 &amp;amp; 1 &amp;amp; v &amp;amp; v \\&#xA;w-a  &amp;amp; 0 &amp;amp; 0 &amp;amp; w &amp;amp; w+a \\&#xA;h'-ua&amp;amp; u &amp;amp; v &amp;amp; \frac{1}{2}q^2+\psi &amp;amp; h'+ua&#xA;\end{matrix}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and left eigenvectors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L = R^{-1} = \frac{\gamma-1}{2a^2}\left[ \begin{matrix}&#xA;(\frac{1}{2}q^2-\psi)+\frac{wa}{\gamma-1} &amp;amp; -u &amp;amp; -v &amp;amp; -w-\frac{a}{\gamma-1} &amp;amp; 1 \\&#xA;\frac{2ua^2}{\gamma-1} &amp;amp; -\frac{2a^2}{\gamma-1} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;-\frac{2va^2}{\gamma-1} &amp;amp; 0 &amp;amp; \frac{2a^2}{\gamma-1} &amp;amp; 0 &amp;amp; 0 \\&#xA;2h'-2q^2 &amp;amp; 2u &amp;amp; 2v &amp;amp; 2w &amp;amp; -2 \\&#xA;(\frac{1}{2}q^2-\psi) - \frac{wa}{\gamma-1} &amp;amp; -u &amp;amp; -v &amp;amp; -w + \frac{a}{\gamma-1} &amp;amp; 1&#xA;\end{matrix}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone see any problems with this formulation?  Does anybody have experience including gravity as a source term for the momentum equation, and as a flux for the energy equation?  Or does anyone see problems with the calculation of the eigensystem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="10331" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-10-24T16:53:30.307" Title="Euler Equation Eigensystem with Gravity in the Energy Flux" Tags="|fluid-dynamics|finite-volume|hyperbolic-pde|wave-propagation|conservation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="15964" PostTypeId="1" CreationDate="2014-10-24T02:25:01.660" Score="3" ViewCount="138" Body="&lt;p&gt;I understand some basic analysis techniques (local truncation error, global error, zero-stable, absolute stable, etc.) of numerical integration.&#xA;But I find it hard to apply these techniques in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case (a hair/cloth simulation for computer games), I usually do a numerical integration step followed by constraints resolving steps in every frame. I wonder how does constraint resolution affect the numerical integration?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Particularly, I am concerned with the following situation: in most cases the exact solution of constraints cannot be reached due to numerical error, insufficiently many iterative steps, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When an error is introduced during constraint resolution,  what will happen to the numerical integration in the next frame? Does this error act as external sources (inhomogeneous part) to the differential equations system?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: more background&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using &lt;a href=&quot;http://en.wikipedia.org/wiki/Verlet_integration&quot; rel=&quot;nofollow&quot;&gt;Verlet integration&lt;/a&gt; in my simulation. To resolve constraints I directly modify the positions. In the next temporal integration step the error in positions would contribute &quot;ghost&quot; velocity to the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intention is to analyze the impact of this error and find a way to control the error to make my temporal integration stable and accurate to some desired order.&lt;/p&gt;&#xA;" OwnerUserId="7258" LastEditorUserId="713" LastEditDate="2014-10-24T05:41:06.363" LastActivityDate="2014-10-26T07:37:56.433" Title="How does constraint resolution affect the stability/accuracy of numerical integration?" Tags="|finite-difference|convergence|stability|time-integration|constraints|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15965" PostTypeId="2" ParentId="15964" CreationDate="2014-10-24T05:23:19.987" Score="1" Body="&lt;p&gt;One way to look at it is to say that you are interested in solving&#xA;$$ y'(t) = f(y(t)), \qquad g(y(t)) = 0, $$&#xA;and you only approximately enforce the constraints, thus solving&#xA;$$ y'(t) = f(y(t)), \qquad g(y(t)) = O(\epsilon). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the constraints are well-behaved (say differentiable, although that's a bit much maybe),&#xA;then (by the implicit function theorem) $g(y(t)) = O(\epsilon)$ implies that $y(t)$ is &quot;close&quot; to a true solution $ y(t) = \tilde y(t) + \delta y(t)$ with $g(\tilde y(t)) = 0$ and the error being on the order of&#xA;$$ |\delta y(t)| = O(|\nabla g^{-1}|\epsilon). $$&#xA;(The inverse of possibly-singular Jacobian is, say, in the sense of least squares.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus if both approximate $y$ and the true $\tilde y$ satisfy the same ODE, then&#xA;$$ (y-\tilde y)' = f(y) - f(\tilde y) \sim (\nabla f)(y-\tilde y). $$&#xA;Conversely, solving the approximate system is like solving the perturbed system&#xA;$$ y' = f(y) + (\nabla f)\delta y $$&#xA;with $\delta y$ being an unknowable function on the order of $|\nabla g^{-1}|\epsilon$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what matters is not the numerical integrator you use, but the sensitivity of your ODE to small perturbations. In other words, if the integrator solves the right equation correctly, then it is only the properties of the ODE that should matter. What you care about is whether the error $|y-\tilde y|$ stays small over longer time periods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the ODE is stable, i.e., solutions starting with similar initial conditions remain close in some sense, the effect of the error will be on the order of $O(|\nabla f||\nabla g^{-1}|\epsilon)$, so hopefully small enough.&#xA;If the ODE is backward stable, i.e., for any approximate solution you can find a &quot;nearby&quot; initial condition that would have produced the true solution, then in that sense the errors would be small enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases, the sensitivity of the ODE to perturbations is the issue, not the numerical method. So I would say it should be sufficient to check for any approximate solution $y$ whether there is indeed a &quot;nearby&quot; true solution $\tilde y$, which can be done by analyzing the constraint function. You can, of course, come up with very badly behaved constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, perturbation theory of ODEs is a very well known subject.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-10-24T05:23:19.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15966" PostTypeId="1" AcceptedAnswerId="15981" CreationDate="2014-10-24T07:09:46.567" Score="1" ViewCount="622" Body="&lt;p&gt;I want to minimize $f(x) = \mathrm{Tr}(\sqrt{\mathbf{X}^{T}\mathbf{X}}\mathbf{A})$, where $\mathbf{X}$ is an matrix variable of dimension $d \times d$, and $\mathbf{A}$ is a known matrix. I tried the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cvx_begin&#xA;    variable x(d,d)&#xA;    minimize(trace(sqrt(sum_square_abs(x))*A)) &#xA;    subject to&#xA;        x(sp_index) == M(sp_index)&#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, there are still errors as following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error using cvx/sqrt (line 61)&#xA;Disciplined convex programming error:&#xA;Illegal operation: sqrt( {convex} ).&#xA;&#xA;Error in Test_CVX_Iterative_Optimal (line 34)&#xA;    minimize(trace(sqrt(sum_square_abs(x))*A)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So how should I solve this problem by CVX? Looking forward to your reply! &#xA;I also asked the same question in the CVX forum&#xA;&lt;a href=&quot;http://ask.cvxr.com/question/2894&quot; rel=&quot;nofollow&quot;&gt;http://ask.cvxr.com/question/2894&lt;/a&gt;&#xA;but haven't got it solved yet. Wish anyone here is able to offer help!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Update:&#xA;Thanks to @DavidKetcheson , I should use &lt;code&gt;sqrtm()&lt;/code&gt; rather than &lt;code&gt;sqrt&lt;/code&gt;. To represent $\mathrm{Tr}(\sqrt{\mathbf{X}^T\mathbf{X}})$ I should use &lt;code&gt;trace_sqrtm(sum_square_abs(x))&lt;/code&gt;. However, I need to represent $\mathrm{Tr}(\sqrt{\mathbf{X}^T\mathbf{X}}\mathbf{A})$, and I don't know how to represent it by a valid CVX expression.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I tried &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minimize(trace(sqrtm(sum_square_abs(x))*A))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to replace &lt;code&gt;sqrt&lt;/code&gt; in the original, but I got the following error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Undefined function 'schur' for input arguments of type 'cvx'.&#xA;&#xA;Error in sqrtm (line 32)&#xA;[Q, T] = schur(A,'complex');  % T is complex Schur form.&#xA;&#xA;Error in Test_WeightNucNorm (line 35)&#xA;        minimize(trace(sqrtm(sum_square_abs(x))*A))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I understand now that &lt;code&gt;sqrtm()&lt;/code&gt; function is not implemented in CVX, so I have the error ' Undefined function 'schur' for input arguments of type 'cvx'.'. But my problem is still not solved.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I think I should use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minimize(trace_sqrtm(sum_square_abs(x))*A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I still get error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error using cvx/trace_sqrtm (line 9)&#xA;Input must be affine.&#xA;&#xA;Error in Test_WeightNucNorm (line 40)&#xA;        minimize(trace_sqrtm(sum_square_abs(x))*A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;---------- Important update&#xA;Thanks to @k20 , I made a mistake!! This is not the weighted nuclear norm of matrix $\mathbf{X}$, but the weighted trace of $\sqrt{\mathbf{X}^T\mathbf{X}}$!! Now my problem becomes that:&lt;strong&gt;how to represent the weighted nuclear norm of a matrix $\mathbf{X}$, where $\mathbf{X}$ is CVX variable, $\mathbf{A}$ is a diagonal weight matrix.&lt;/strong&gt;  I tried to find a CVX function which give me all the singular values of variable $\mathbf{X}$, but I haven't found such a function.&lt;/p&gt;&#xA;" OwnerUserId="11597" LastEditorUserId="10134" LastEditDate="2014-10-25T11:10:55.037" LastActivityDate="2014-10-25T11:10:55.037" Title="How to represent weighted nuclear norm of matrix variable X and minimize it by CVX function, or solve it by other possible packages" Tags="|cvx|" AnswerCount="2" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="15967" PostTypeId="1" CreationDate="2014-10-24T09:00:05.897" Score="2" ViewCount="414" Body="&lt;p&gt;I've been taught that when simulating a biomolecule in thermal equilibrium, it's best to use the Langevin thermostat - an algorithm which produces a trajectory, which is a realization of a stochastic process given by the Langevin (stochastic differential) equation. I haven't had a doubt about this algorithm's power to create a sample from the Boltzmann distribution, but what about it's &quot;correctness&quot; regarding time-dependent behaviour of the system? In other words: does the movie we get from a simulation reflect how biomolecules &lt;em&gt;actually&lt;/em&gt; behave?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently, I've been studying van Kampen's &lt;em&gt;Stochastic Processes in Physics and Chemistry&lt;/em&gt; (3rd edition), where on page 56 (chapter III, paragraph 2.) there is a rather philosophical discussion of the stochastic description of microscopic systems. Here's the part that I found most confusing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Having accepted that the irregular motion of a system may be reformulated as a stochastic process, one is faced with the task of choosing the appropriate process. For a closed, isolated system that is usually done as follows. The microscopic deterministic motion may be represented by a trajectory in the phase space $\Gamma$. Each point $X\in\Gamma$ is, after a time $t$ mapped by the motion into a point $X^t\in\Gamma$, where $X^t = f(X,t)$ is uniquely determined. If one now chooses at some initial time $t=0$ not a single initial state $X$, but a probability density $P(x)$ in $\Gamma$, then $f(x,t)$ is a stochastic process as defined in the preceding section. The initial $P(x)$ is to be chosen so as to reflect the way in which the system was prepared. Any other physical quantity pertaining to the system is a function $Y(X^t)$ of the phase point $X^t$ and has therefore also become a stochastic process $Y(X,t)$.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This is the usual approach to the stochastic description of nonequilibrium behavior and fluctuations. It is the starting point of the derivation of the so-called &quot;generalized Langevin equation&quot; and of the Kubo relations in linear response theory. It was even advocated in the first edition of this book -- &lt;strong&gt;but it is wrong&lt;/strong&gt;. The irregular motion of a Brownian particle can not be related to a probability distribution of some initial state. Rather it is brought about by the surrounding bath molecules and is a vestige of all the variables of the total system that have been ignored in order to obtain an equation for the Brownian particle alone, see IV.a and VIII.3. The proper way of establishing the stochastic description of Brownian motion is therefore the careful elimination of the bath variables from the complete set of microscopic equations of the total system.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Van Kampen provides a reference: N.G. van Kampen and I Oppenheim, Physica A 138, 231 (1986), to which I haven't got access.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; is it then incorrect to use the Langevin thermostat when we are interested in the actual, time-dependant behaviour of a molecular system? Or, perhaps, the Langevin-equation-based description &quot;isn't that bad&quot; once we immerse the biomolecule in a sufficiently large water basin, thus emulating the complexity of the environment? &lt;/p&gt;&#xA;" OwnerUserId="10052" LastActivityDate="2014-10-24T21:27:45.060" Title="Is Langevin thermostat/equation correct when trying to model time-dependent behaviour of a molecule?" Tags="|molecular-dynamics|stochastic|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15969" PostTypeId="1" AcceptedAnswerId="15975" CreationDate="2014-10-24T12:58:09.793" Score="2" ViewCount="83" Body="&lt;p&gt;I have two algebraic equations I am trying to solve in Maple. They are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$14\,{a}^{26}{b}^{2}-91\,{a}^{24}{b}^{4}-364\,{a}^{22}{b}^{6}-1001\,{a}&#xA;^{20}{b}^{8}-2002\,{a}^{18}{b}^{10}-3003\,{a}^{16}{b}^{12}-3432\,{a}^{&#xA;14}{b}^{14}-3003\,{a}^{12}{b}^{16}-2002\,{a}^{10}{b}^{18}-1001\,{a}^{8&#xA;}{b}^{20}-364\,{a}^{6}{b}^{22}-91\,{a}^{4}{b}^{24}-14\,{a}^{2}{b}^{26}&#xA;-{b}^{28}-36\,{a}^{26}-396\,{a}^{24}{b}^{2}-1944\,{a}^{22}{b}^{4}-5544&#xA;\,{a}^{20}{b}^{6}-9900\,{a}^{18}{b}^{8}-10692\,{a}^{16}{b}^{10}-4752\,&#xA;{a}^{14}{b}^{12}+4752\,{a}^{12}{b}^{14}+10692\,{a}^{10}{b}^{16}+9900\,&#xA;{a}^{8}{b}^{18}+5544\,{a}^{6}{b}^{20}+1944\,{a}^{4}{b}^{22}+396\,{a}^{&#xA;2}{b}^{24}+36\,{b}^{26}+1872\,{a}^{24}+16576\,{a}^{22}{b}^{2}+64672\,{&#xA;a}^{20}{b}^{4}+146880\,{a}^{18}{b}^{6}+220080\,{a}^{16}{b}^{8}+246144&#xA;\,{a}^{14}{b}^{10}+245952\,{a}^{12}{b}^{12}+246144\,{a}^{10}{b}^{14}+&#xA;220080\,{a}^{8}{b}^{16}+146880\,{a}^{6}{b}^{18}+64672\,{a}^{4}{b}^{20}&#xA;+16576\,{a}^{2}{b}^{22}+1872\,{b}^{24}-25792\,{a}^{22}-173248\,{a}^{20&#xA;}{b}^{2}-490560\,{a}^{18}{b}^{4}-756800\,{a}^{16}{b}^{6}-672640\,{a}^{&#xA;14}{b}^{8}-258944\,{a}^{12}{b}^{10}+258944\,{a}^{10}{b}^{12}+672640\,{&#xA;a}^{8}{b}^{14}+756800\,{a}^{6}{b}^{16}+490560\,{a}^{4}{b}^{18}+173248&#xA;\,{a}^{2}{b}^{20}+25792\,{b}^{22}+198144\,{a}^{20}+1956864\,{a}^{18}{b&#xA;}^{2}+6753792\,{a}^{16}{b}^{4}+11292672\,{a}^{14}{b}^{6}+10742784\,{a}&#xA;^{12}{b}^{8}+8890368\,{a}^{10}{b}^{10}+10742784\,{a}^{8}{b}^{12}+&#xA;11292672\,{a}^{6}{b}^{14}+6753792\,{a}^{4}{b}^{16}+1956864\,{a}^{2}{b}&#xA;^{18}+198144\,{b}^{20}-731136\,{a}^{18}-12589056\,{a}^{16}{b}^{2}-&#xA;37822464\,{a}^{14}{b}^{4}-45244416\,{a}^{12}{b}^{6}-19279872\,{a}^{10}&#xA;{b}^{8}+19279872\,{a}^{8}{b}^{10}+45244416\,{a}^{6}{b}^{12}+37822464\,&#xA;{a}^{4}{b}^{14}+12589056\,{a}^{2}{b}^{16}+731136\,{b}^{18}-9461760\,{a&#xA;}^{16}-30867456\,{a}^{14}{b}^{2}+287145984\,{a}^{12}{b}^{4}+369033216&#xA;\,{a}^{10}{b}^{6}+120963072\,{a}^{8}{b}^{8}+369033216\,{a}^{6}{b}^{10}&#xA;+287145984\,{a}^{4}{b}^{12}-30867456\,{a}^{2}{b}^{14}-9461760\,{b}^{16&#xA;}+85622784\,{a}^{14}+350257152\,{a}^{12}{b}^{2}-255688704\,{a}^{10}{b}&#xA;^{4}-3238232064\,{a}^{8}{b}^{6}+3238232064\,{a}^{6}{b}^{8}+255688704\,&#xA;{a}^{4}{b}^{10}-350257152\,{a}^{2}{b}^{12}-85622784\,{b}^{14}+&#xA;137035776\,{a}^{12}-5009965056\,{a}^{10}{b}^{2}-3606773760\,{a}^{8}{b}&#xA;^{4}+24823726080\,{a}^{6}{b}^{6}-3606773760\,{a}^{4}{b}^{8}-5009965056&#xA;\,{a}^{2}{b}^{10}+137035776\,{b}^{12}-1408499712\,{a}^{10}-1054605312&#xA;\,{a}^{8}{b}^{2}+81891164160\,{a}^{6}{b}^{4}-81891164160\,{a}^{4}{b}^{&#xA;6}+1054605312\,{a}^{2}{b}^{8}+1408499712\,{b}^{10}+1443889152\,{a}^{8}&#xA;+56962842624\,{a}^{6}{b}^{2}-171624628224\,{a}^{4}{b}^{4}+56962842624&#xA;\,{a}^{2}{b}^{6}+1443889152\,{b}^{8}+13929283584\,{a}^{6}-97504985088&#xA;\,{a}^{4}{b}^{2}+97504985088\,{a}^{2}{b}^{4}-13929283584\,{b}^{6}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$75\,{a}^{44}+1250\,{a}^{42}{b}^{2}+9325\,{a}^{40}{b}^{4}+39500\,{a}^{&#xA;38}{b}^{6}+92625\,{a}^{36}{b}^{8}+37050\,{a}^{34}{b}^{10}-605625\,{a}^&#xA;{32}{b}^{12}-2713200\,{a}^{30}{b}^{14}-7025250\,{a}^{28}{b}^{16}-&#xA;13081500\,{a}^{26}{b}^{18}-18685550\,{a}^{24}{b}^{20}-20995000\,{a}^{&#xA;22}{b}^{22}-18685550\,{a}^{20}{b}^{24}-13081500\,{a}^{18}{b}^{26}-&#xA;7025250\,{a}^{16}{b}^{28}-2713200\,{a}^{14}{b}^{30}-605625\,{a}^{12}{b&#xA;}^{32}+37050\,{a}^{10}{b}^{34}+92625\,{a}^{8}{b}^{36}+39500\,{a}^{6}{b&#xA;}^{38}+9325\,{a}^{4}{b}^{40}+1250\,{a}^{2}{b}^{42}+75\,{b}^{44}-6360\,&#xA;{a}^{42}-124360\,{a}^{40}{b}^{2}-1141040\,{a}^{38}{b}^{4}-6517200\,{a}&#xA;^{36}{b}^{6}-25897560\,{a}^{34}{b}^{8}-75690120\,{a}^{32}{b}^{10}-&#xA;167296320\,{a}^{30}{b}^{12}-281699520\,{a}^{28}{b}^{14}-354817200\,{a}&#xA;^{26}{b}^{16}-309064080\,{a}^{24}{b}^{18}-123936800\,{a}^{22}{b}^{20}+&#xA;123936800\,{a}^{20}{b}^{22}+309064080\,{a}^{18}{b}^{24}+354817200\,{a}&#xA;^{16}{b}^{26}+281699520\,{a}^{14}{b}^{28}+167296320\,{a}^{12}{b}^{30}+&#xA;75690120\,{a}^{10}{b}^{32}+25897560\,{a}^{8}{b}^{34}+6517200\,{a}^{6}{&#xA;b}^{36}+1141040\,{a}^{4}{b}^{38}+124360\,{a}^{2}{b}^{40}+6360\,{b}^{42&#xA;}-286560\,{a}^{40}+1299840\,{a}^{38}{b}^{2}+50378944\,{a}^{36}{b}^{4}+&#xA;401336704\,{a}^{34}{b}^{6}+1740940320\,{a}^{32}{b}^{8}+4901465600\,{a}&#xA;^{30}{b}^{10}+9580140800\,{a}^{28}{b}^{12}+13378708992\,{a}^{26}{b}^{&#xA;14}+13616978752\,{a}^{24}{b}^{16}+10903809280\,{a}^{22}{b}^{18}+&#xA;9196936320\,{a}^{20}{b}^{20}+10903809280\,{a}^{18}{b}^{22}+13616978752&#xA;\,{a}^{16}{b}^{24}+13378708992\,{a}^{14}{b}^{26}+9580140800\,{a}^{12}{&#xA;b}^{28}+4901465600\,{a}^{10}{b}^{30}+1740940320\,{a}^{8}{b}^{32}+&#xA;401336704\,{a}^{6}{b}^{34}+50378944\,{a}^{4}{b}^{36}+1299840\,{a}^{2}{&#xA;b}^{38}-286560\,{b}^{40}+372661248\,{a}^{36}{b}^{2}-707820544\,{a}^{34&#xA;}{b}^{4}-13983124480\,{a}^{32}{b}^{6}-64604672000\,{a}^{30}{b}^{8}-&#xA;171060842496\,{a}^{28}{b}^{10}-301423489024\,{a}^{26}{b}^{12}-&#xA;369866092544\,{a}^{24}{b}^{14}-305887582208\,{a}^{22}{b}^{16}-&#xA;118022141952\,{a}^{20}{b}^{18}+118022141952\,{a}^{18}{b}^{20}+&#xA;305887582208\,{a}^{16}{b}^{22}+369866092544\,{a}^{14}{b}^{24}+&#xA;301423489024\,{a}^{12}{b}^{26}+171060842496\,{a}^{10}{b}^{28}+&#xA;64604672000\,{a}^{8}{b}^{30}+13983124480\,{a}^{6}{b}^{32}+707820544\,{&#xA;a}^{4}{b}^{34}-372661248\,{a}^{2}{b}^{36}-64023552\,{b}^{38}-&#xA;4020256512\,{a}^{36}-25280404992\,{a}^{34}{b}^{2}-17689454336\,{a}^{32&#xA;}{b}^{4}+327596421120\,{a}^{30}{b}^{6}+1568612611072\,{a}^{28}{b}^{8}+&#xA;3802104150016\,{a}^{26}{b}^{10}+5990533923840\,{a}^{24}{b}^{12}+&#xA;6790271389696\,{a}^{22}{b}^{14}+6232580648448\,{a}^{20}{b}^{16}+&#xA;5750651833344\,{a}^{18}{b}^{18}+6232580648448\,{a}^{16}{b}^{20}+&#xA;6790271389696\,{a}^{14}{b}^{22}+5990533923840\,{a}^{12}{b}^{24}+&#xA;3802104150016\,{a}^{10}{b}^{26}+1568612611072\,{a}^{8}{b}^{28}+&#xA;327596421120\,{a}^{6}{b}^{30}-17689454336\,{a}^{4}{b}^{32}-25280404992&#xA;\,{a}^{2}{b}^{34}-4020256512\,{b}^{36}+141841483776\,{a}^{34}+&#xA;812651771904\,{a}^{32}{b}^{2}+844669190144\,{a}^{30}{b}^{4}-&#xA;5879063412736\,{a}^{28}{b}^{6}-27045676376064\,{a}^{26}{b}^{8}-&#xA;58879853699072\,{a}^{24}{b}^{10}-81000773943296\,{a}^{22}{b}^{12}-&#xA;71903915409408\,{a}^{20}{b}^{14}-28790241103872\,{a}^{18}{b}^{16}+&#xA;28790241103872\,{a}^{16}{b}^{18}+71903915409408\,{a}^{14}{b}^{20}+&#xA;81000773943296\,{a}^{12}{b}^{22}+58879853699072\,{a}^{10}{b}^{24}+&#xA;27045676376064\,{a}^{8}{b}^{26}+5879063412736\,{a}^{6}{b}^{28}-&#xA;844669190144\,{a}^{4}{b}^{30}-812651771904\,{a}^{2}{b}^{32}-&#xA;141841483776\,{b}^{34}-3238367662080\,{a}^{32}-15304336146432\,{a}^{30&#xA;}{b}^{2}-9340373663744\,{a}^{28}{b}^{4}+95140684300288\,{a}^{26}{b}^{6&#xA;}+333410977595392\,{a}^{24}{b}^{8}+614533778046976\,{a}^{22}{b}^{10}+&#xA;845198215839744\,{a}^{20}{b}^{12}+1012450883862528\,{a}^{18}{b}^{14}+&#xA;1081581115908096\,{a}^{16}{b}^{16}+1012450883862528\,{a}^{14}{b}^{18}+&#xA;845198215839744\,{a}^{12}{b}^{20}+614533778046976\,{a}^{10}{b}^{22}+&#xA;333410977595392\,{a}^{8}{b}^{24}+95140684300288\,{a}^{6}{b}^{26}-&#xA;9340373663744\,{a}^{4}{b}^{28}-15304336146432\,{a}^{2}{b}^{30}-&#xA;3238367662080\,{b}^{32}+50490836582400\,{a}^{30}+176086600974336\,{a}^&#xA;{28}{b}^{2}-152306959187968\,{a}^{26}{b}^{4}-1684710750093312\,{a}^{24&#xA;}{b}^{6}-3454899096584192\,{a}^{22}{b}^{8}-3298794461921280\,{a}^{20}{&#xA;b}^{10}-1445416723218432\,{a}^{18}{b}^{12}-194713331367936\,{a}^{16}{b&#xA;}^{14}+194713331367936\,{a}^{14}{b}^{16}+1445416723218432\,{a}^{12}{b}&#xA;^{18}+3298794461921280\,{a}^{10}{b}^{20}+3454899096584192\,{a}^{8}{b}^&#xA;{22}+1684710750093312\,{a}^{6}{b}^{24}+152306959187968\,{a}^{4}{b}^{26&#xA;}-176086600974336\,{a}^{2}{b}^{28}-50490836582400\,{b}^{30}-&#xA;534649494896640\,{a}^{28}-1210295967940608\,{a}^{26}{b}^{2}+&#xA;5724614944423936\,{a}^{24}{b}^{4}+26380886835724288\,{a}^{22}{b}^{6}+&#xA;42048454899204096\,{a}^{20}{b}^{8}+30388730092978176\,{a}^{18}{b}^{10}&#xA;+4724812444139520\,{a}^{16}{b}^{12}-7192176335781888\,{a}^{14}{b}^{14}&#xA;+4724812444139520\,{a}^{12}{b}^{16}+30388730092978176\,{a}^{10}{b}^{18&#xA;}+42048454899204096\,{a}^{8}{b}^{20}+26380886835724288\,{a}^{6}{b}^{22&#xA;}+5724614944423936\,{a}^{4}{b}^{24}-1210295967940608\,{a}^{2}{b}^{26}-&#xA;534649494896640\,{b}^{28}+3286873284280320\,{a}^{26}+2713917651943424&#xA;\,{a}^{24}{b}^{2}-77284074436165632\,{a}^{22}{b}^{4}-&#xA;270542370073739264\,{a}^{20}{b}^{6}-349879051057889280\,{a}^{18}{b}^{8&#xA;}-213063378711085056\,{a}^{16}{b}^{10}-57015578923106304\,{a}^{14}{b}^&#xA;{12}+57015578923106304\,{a}^{12}{b}^{14}+213063378711085056\,{a}^{10}{&#xA;b}^{16}+349879051057889280\,{a}^{8}{b}^{18}+270542370073739264\,{a}^{6&#xA;}{b}^{20}+77284074436165632\,{a}^{4}{b}^{22}-2713917651943424\,{a}^{2}&#xA;{b}^{24}-3286873284280320\,{b}^{26}+213111700193280\,{a}^{24}+&#xA;70908012205703168\,{a}^{22}{b}^{2}+694283115155161088\,{a}^{20}{b}^{4}&#xA;+2132555668861747200\,{a}^{18}{b}^{6}+1859501969611161600\,{a}^{16}{b}&#xA;^{8}+84575734147842048\,{a}^{14}{b}^{10}-75928100231184384\,{a}^{12}{b&#xA;}^{12}+84575734147842048\,{a}^{10}{b}^{14}+1859501969611161600\,{a}^{8&#xA;}{b}^{16}+2132555668861747200\,{a}^{6}{b}^{18}+694283115155161088\,{a}&#xA;^{4}{b}^{20}+70908012205703168\,{a}^{2}{b}^{22}+213111700193280\,{b}^{&#xA;24}-197112346768834560\,{a}^{22}-1313568123575599104\,{a}^{20}{b}^{2}-&#xA;4826743947927748608\,{a}^{18}{b}^{4}-15507264587736023040\,{a}^{16}{b}&#xA;^{6}-13320253101815365632\,{a}^{14}{b}^{8}+6228544173411139584\,{a}^{&#xA;12}{b}^{10}-6228544173411139584\,{a}^{10}{b}^{12}+13320253101815365632&#xA;\,{a}^{8}{b}^{14}+15507264587736023040\,{a}^{6}{b}^{16}+&#xA;4826743947927748608\,{a}^{4}{b}^{18}+1313568123575599104\,{a}^{2}{b}^{&#xA;20}+197112346768834560\,{b}^{22}+1396772423168163840\,{a}^{20}+&#xA;11608333269149417472\,{a}^{18}{b}^{2}+24257517869881884672\,{a}^{16}{b&#xA;}^{4}+70758399650983575552\,{a}^{14}{b}^{6}+96421712044744507392\,{a}^&#xA;{12}{b}^{8}-128968645422643937280\,{a}^{10}{b}^{10}+&#xA;96421712044744507392\,{a}^{8}{b}^{12}+70758399650983575552\,{a}^{6}{b}&#xA;^{14}+24257517869881884672\,{a}^{4}{b}^{16}+11608333269149417472\,{a}^&#xA;{2}{b}^{18}+1396772423168163840\,{b}^{20}-433185960612593664\,{a}^{18}&#xA;-55461219976548974592\,{a}^{16}{b}^{2}-157656713576520351744\,{a}^{14}&#xA;{b}^{4}-153912151266903982080\,{a}^{12}{b}^{6}-760803075000404803584\,&#xA;{a}^{10}{b}^{8}+760803075000404803584\,{a}^{8}{b}^{10}+&#xA;153912151266903982080\,{a}^{6}{b}^{12}+157656713576520351744\,{a}^{4}{&#xA;b}^{14}+55461219976548974592\,{a}^{2}{b}^{16}+433185960612593664\,{b}^&#xA;{18}-37561374368266715136\,{a}^{16}-32837068847629467648\,{a}^{14}{b}^&#xA;{2}+1163976553479951876096\,{a}^{12}{b}^{4}+42543947772724248576\,{a}^&#xA;{10}{b}^{6}+1705420374725986615296\,{a}^{8}{b}^{8}+&#xA;42543947772724248576\,{a}^{6}{b}^{10}+1163976553479951876096\,{a}^{4}{&#xA;b}^{12}-32837068847629467648\,{a}^{2}{b}^{14}-37561374368266715136\,{b&#xA;}^{16}+145123568542084497408\,{a}^{14}+2214786761572522917888\,{a}^{12&#xA;}{b}^{2}-2941686581914050232320\,{a}^{10}{b}^{4}-&#xA;10669449418591224987648\,{a}^{8}{b}^{6}+10669449418591224987648\,{a}^{&#xA;6}{b}^{8}+2941686581914050232320\,{a}^{4}{b}^{10}-&#xA;2214786761572522917888\,{a}^{2}{b}^{12}-145123568542084497408\,{b}^{14&#xA;}+249314329329842257920\,{a}^{12}-7902762540911007105024\,{a}^{10}{b}^&#xA;{2}-25076225098692949966848\,{a}^{8}{b}^{4}+83375468150158025293824\,{&#xA;a}^{6}{b}^{6}-25076225098692949966848\,{a}^{4}{b}^{8}-&#xA;7902762540911007105024\,{a}^{2}{b}^{10}+249314329329842257920\,{b}^{12&#xA;}-2801419437596265676800\,{a}^{10}-4512413307974543474688\,{a}^{8}{b}^&#xA;{2}+160715735714773856157696\,{a}^{6}{b}^{4}-160715735714773856157696&#xA;\,{a}^{4}{b}^{6}+4512413307974543474688\,{a}^{2}{b}^{8}+&#xA;2801419437596265676800\,{b}^{10}+157002242738903580672\,{a}^{8}+&#xA;99495970498069735145472\,{a}^{6}{b}^{2}-282658247208303920676864\,{a}^&#xA;{4}{b}^{4}+99495970498069735145472\,{a}^{2}{b}^{6}+&#xA;157002242738903580672\,{b}^{8}+15810272187369010495488\,{a}^{6}-&#xA;123591451896180620918784\,{a}^{4}{b}^{2}+123591451896180620918784\,{a}&#xA;^{2}{b}^{4}-15810272187369010495488\,{b}^{6}-3275378007362758508544\,{&#xA;a}^{4}+10917926691209195028480\,{a}^{2}{b}^{2}-3275378007362758508544&#xA;\,{b}^{4}=0&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plots of a vs b for both these equations on the same graph is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/GVuRt.jpg&quot; alt=&quot;Both equations on one plot&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the graph we can see that one intersection point is near (4,2). Thus, I tried using fsolve in MAPLE to get the exact point to which MAPLE returns {a = -3.984746496, b = 2.012958923}. But, apparently, when I put back this point into the two equations I get values nowhere close to zero. I also tried using fsolve in MATLAB but no success. Somebody please help me out in solving the two equtions simulataneously.&lt;/p&gt;&#xA;" OwnerUserId="9756" LastEditorUserId="123" LastEditDate="2014-10-24T18:20:10.683" LastActivityDate="2014-10-25T15:01:14.070" Title="Solving a pair of high-degree polynomials in two variables with Maple" Tags="|matlab|nonlinear-equations|maple|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15970" PostTypeId="2" ParentId="15967" CreationDate="2014-10-24T13:17:14.503" Score="2" Body="&lt;p&gt;If $H = H(q, p)$ is the Hamiltonian of your mechanical system, $T$ is the temperature and $\gamma$ is the friction, then Langevin dynamics (LD),&#xA;\begin{equation*}&#xA;  \left\{&#xA;    \begin{aligned}&#xA;      \mathrm{d} q &amp;amp;= \nabla_p H(q, p) \, \mathrm{d} t, \\&#xA;      \mathrm{d} p &amp;amp;= -\nabla_q H(q, p) \, \mathrm{d} t - \gamma \, p \, \mathrm{d} t + \sqrt{2 \gamma k_B T} \, \mathrm{d} B,&#xA;    \end{aligned}&#xA;  \right.&#xA;\end{equation*}&#xA;will only coincide with Hamiltonian dynamics (HD) when $\gamma = 0$.&#xA;This will be the case even in the thermodynamic limit, when the canonical and the microcanonical ensembles are equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question, then, is how much does LD diverge from HD.&#xA;The paper [LNP11] addresses this issue by defining two quantities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The rate at which the system, starting from an initial condition sampled from the canonical distribution at temperature $T_0$, converges to the target temperature $T$.&lt;/li&gt;&#xA;&lt;li&gt;The error in the velocity auto-correlation (VAF) function, defined as the difference between the VAF from LD and that of HD (considered as the reference one).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A quantity known as the &lt;em&gt;efficiency&lt;/em&gt; of the thermostat (LD in this case) is then defined as the ratio between the rate of convergence and the error per unit time.&#xA;One of the conclusions is that LD is not the most efficient thermostatting scheme by this measure or, in other words, there exist thermostats that interfere less with the true dynamics than Langevin dynamics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[LNP11]  Benedict Leimkuhler, Emad Noorizadeh, and Oliver Penrose.  Comparing the Efficiencies of Stochastic Isothermal Molecular Dynamics Methods.  &lt;em&gt;Journal of Statistical Physics&lt;/em&gt;, 143(5):921--942, May 2011.  &lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs10955-011-0210-2&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/article/10.1007%2Fs10955-011-0210-2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2014-10-24T21:27:45.060" LastActivityDate="2014-10-24T21:27:45.060" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15971" PostTypeId="2" ParentId="15966" CreationDate="2014-10-24T14:15:45.007" Score="1" Body="&lt;p&gt;minimizing $\sqrt{f(x)}$ is equivalent to minimizing $f(x)$ as long as $f(x) \geq 0$ because taking the square root is a monotone transformation.  Just remove the square root from your objective function. &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-10-24T14:15:45.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15973" PostTypeId="1" AcceptedAnswerId="21246" CreationDate="2014-10-24T19:15:03.873" Score="6" ViewCount="484" Body="&lt;p&gt;I have implemented a version of Visscher's method for numerically solving the TDSE (&lt;a href=&quot;http://www.hunter.cuny.edu/physics/courses/physics485/repository/files/reference-papers/Visscher%20NumSol%20Schrodinger%20Eqt.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A fast explicit algorithm for the time-dependent Schrödinger equation&lt;/a&gt;) (also described in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10876/are-there-simple-ways-to-numerically-solve-the-time-dependent-sch%C3%B6dinger-equatio/10880#10880&quot;&gt;Are there simple ways to numerically solve the time-dependent Sch&amp;#246;dinger equation?&lt;/a&gt; ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R(t+\frac{1}{2} \Delta t)=R(t-\frac{1}{2} \Delta t)+\Delta t HI(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I(t+\frac{1}{2} \Delta t)=I(t-\frac{1}{2} \Delta t)-\Delta t HR(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering how to implement transparent boundary conditions for such a scheme?  If you leave the boundaries uncomputed and set to zero, it produces perfect reflection.  I have also tried &quot;pretending&quot; boundary elements are surrounded by zero on all sides other than the non-boundary side - to mimic a wavefront propagating into nothing - this also reflects.&lt;/p&gt;&#xA;" OwnerUserId="10453" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-11-06T23:26:07.153" Title="Transparent boundary conditions for finite element simulation of TDSE" Tags="|finite-element|boundary-conditions|quantum-mechanics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15974" PostTypeId="1" AcceptedAnswerId="15976" CreationDate="2014-10-24T20:02:34.727" Score="1" ViewCount="1803" Body="&lt;p&gt;I'm trying verify that a 2nd order finite difference in space and time approximation of the 1D wave equation is really 2nd order.  My Matlab implementation tells me otherwise - I'm not sure of what I've done incorrectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity I've decided to use the smooth solution $u(x,t)=\cos{(ct)}\cos{x}$.  The PDE has periodic boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_{tt} = c^2u_{xx},~~~u(x,0)=\cos{x},~~~u_t(x,0)=0,~~~u(-2\pi,t)=u(2\pi,t).&#xA;$$&#xA;I've taken $c=1$, and the domain to be $[-2\pi,2\pi]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite difference scheme I'm using is centered second order in space and time:&#xA;$$&#xA;\frac{u(x_j,t_{i+1})-2u(x_j,t_{i})+u(x_j,t_{i-1})}{(\Delta t)^2}&#xA;=&#xA;c^2\frac{u(x_{j+1},t_{i})-2u(x_{j},t_{i})+u(x_{j-1},t_{i})}{(\Delta x)^2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To approximate the initial condition $u_t(x,0)=0$, I tried both the first order forward approximation&#xA;$$&#xA;\frac{u(x_j,t_{j+1})-u(x_j,t_{j})}{\Delta t}=0,&#xA;$$&#xA;and the second order centered approximation&#xA;$$&#xA;\frac{u(x_j,t_{j+1})-u(x_j,t_{j-1})}{2\Delta t}=0.&#xA;$$&#xA;When I fix $\Delta t$ and refine $\Delta x$, both of these approximations to $u_t(x,0)=0$ seem to give an order of approximately 1.5 according to my Matlab code.  The largest error looks like it occurs near the boundaries $x=\pm 2\pi$.  Can anyone see what I'm doing incorrectly?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = -2*pi;&#xA;b =  2*pi;&#xA;&#xA;NN = 2.^(3:9);&#xA;for jj = 1 : length(NN)&#xA;    n = NN(jj);               %Number of grid points&#xA;    dx = (b-a)/(n-1);         %Spatial mesh spacing&#xA;    x = a : dx : b;           %Mesh in x direction&#xA;    c = 1;&#xA;    u = @(x,t) cos(c*t).*cos(x);&#xA;    u0 = u(x,0.0)';&#xA;&#xA;    e = ones(n,1);&#xA;    A = spdiags([e -2*e e], -1:1, n, n);   %Laplacian matrix&#xA;    A(1,end) = 1;  A(end,1) = 1;           %Periodic boundary conditions&#xA;&#xA;    T = 1;                                 %Final simulation time&#xA;    dt = 1e-3;                             %Temporal mesh spacing&#xA;    nsteps = (floor(T/dt));                %Number if time steps&#xA;&#xA;&#xA;    %u_old = u0;  %First order approxiation of u_t(x,0)=0.&#xA;    %u_now = u0; &#xA;&#xA;                  %Second order approxiation of u_t(x,0)=0.&#xA;    u_old = u0;&#xA;    u_now = u0 + 0.5*(dt*c/dx)^2*(A* u0);  %(approximates u(x,dt))&#xA;&#xA;    for ii = 2 : nsteps  %First time step was directly above&#xA;       u_new = 2*u_now - u_old + (dt*c/dx)^2*(A* u_now);&#xA;       u_old = u_now;&#xA;       u_now = u_new;&#xA;    end&#xA;&#xA;    LI(jj) =  norm(u_new - u(x,T)',inf);&#xA;    L2(jj) = dx*norm(u_new - u(x,T)',2);&#xA;    if ( jj &amp;gt; 1 )  %Look at order of convergence&#xA;       log ( L2(jj-1)/L2(jj) ) / log ( 2 ) &#xA;    end&#xA;end&#xA;&#xA;figure(1)&#xA;semilogy( L2,'-')&#xA;figure(2)&#xA;plot(x,u_new,'ro-',x,u(x,T),'--')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11604" LastActivityDate="2014-10-24T20:30:45.410" Title="2nd Order finite difference for 1D wave equation matlab issue" Tags="|matlab|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15975" PostTypeId="2" ParentId="15969" CreationDate="2014-10-24T20:15:07.560" Score="1" Body="&lt;p&gt;I don't have Maple, but here is what Mathematica and sage can do with more precision (you can translate to Maple easily enough, whatever the equivalent of &lt;code&gt;ContourPlot&lt;/code&gt;$+$&lt;code&gt;findroot&lt;/code&gt; is in Maple). I am assuming you only want real solutions as that's the plot you show.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seem to be using double precision ($\epsilon\sim 10^{-16}$), which is really not enough given the magnitude of the coefficients. In general, if you want $|f(x)| &amp;lt; \epsilon$, then $x$ needs to be calculated to absolute tolerance of $\epsilon/f'(x)$ or so, which in your case is quite a lot smaller than $10^{-16}$. Here is a plot I got (showing only the upper-right quadrant because of the symmetries $a\leftrightarrow\pm a$, $b\leftrightarrow\pm b$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mNrhk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the solutions are (correct to precision shown, rounded down):&#xA;$$\begin{array}{ll}&#xA;\qquad\qquad a&amp;amp;\qquad\qquad b\\&#xA;\\ \phantom{0}0 &amp;amp; 0&#xA;\\ \phantom{0}0 &amp;amp; 2&#xA;\\ \phantom{0}3.98474\ 64962\ 20780\ 16409 &amp;amp; 2.01295\ 89233\ 29253\ 09700&#xA;\\ \phantom{0}2.62879\ 31899\ 96391\ 17312 &amp;amp; 1.04246\ 44796\ 14921\ 50205&#xA;\\ \phantom{0}2.71903\ 38802\ 96411\ 81749 &amp;amp; 0.48706\ 36127\ 79289\ 48764&#xA;\\ 10.32488\ 52886\ 24114\ 42639 &amp;amp; 2.60867\ 53905\ 24137\ 19344&#xA;\\ 11.07895\ 38928\ 95562\ 05933 &amp;amp; 1.51714\ 09752\ 61487\ 55244&#xA;\\ \phantom{0}2.50539\ 67917\ 01949\ 47742 &amp;amp; 7.10235\ 60004\ 90840\ 74450&#xA;\\ \phantom{0}0.63002\ 82486\ 26931\ 58184 &amp;amp; 0.83043\ 91257\ 12434\ 04168&#xA;\\ \phantom{0}0.97964\ 68570\ 87606\ 69608 &amp;amp; 0.53405\ 74656\ 53849\ 71719&#xA;\\ \phantom{0}0.29529\ 36308\ 25062\ 10538 &amp;amp; 0.92298\ 29979\ 75984\ 57228&#xA;\end{array}$$&#xA;I calculated these to about 240 digits, checked their accuracy, then truncated them for display.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-10-25T15:01:14.070" LastActivityDate="2014-10-25T15:01:14.070" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="15976" PostTypeId="2" ParentId="15974" CreationDate="2014-10-24T20:30:45.410" Score="1" Body="&lt;p&gt;This is only a guess, I haven't run your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary condition $y(-2\pi)=y(2\pi)$ means the points $\pm2\pi$ are identified. So the points to the left and right of $2\pi$ are $2\pi-h$ and $-2\pi+h$. However, your mesh has both (non-identical) points $\pm2\pi$ included in it, as separate points, which looks completely suspicious to me. Then the point to the right of $2\pi$ is $-2\pi$, which can't be right as it's the same point.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-10-24T20:30:45.410" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15977" PostTypeId="1" CreationDate="2014-10-24T21:05:35.170" Score="3" ViewCount="164" Body="&lt;p&gt;I am trying to find a good reference on a proper weak formulation for mixed DG (Raviart Thomas and DG) formulation for a Poisson equation with mixed boundary conditions. Can anyone suggest a good reference?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since what follows is lengthy, I am happy to offer a modicum of recompense (of your choice) for good answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I am trying to solve is:&#xA;$$&#xA;\begin{align}&#xA;\mathbf{u} + \mathbf{k} \cdot \nabla p &amp;amp;= 0 \quad in \quad\Omega \\&#xA;\nabla \cdot \mathbf{u} &amp;amp;= q  \quad in \quad \Omega \\&#xA;p &amp;amp;= g  \quad on \quad \partial  \Omega_D\\&#xA;\mathbf{u} \cdot \mathbf{n} &amp;amp;= f \quad on \quad \partial \Omega_N&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I cleaned things up a bit&#xA;$$&#xA;\begin{align}&#xA;\mathbf{k}^{-1} \cdot \mathbf{u} +  \nabla p &amp;amp;= 0 \quad in \quad\Omega \\&#xA;-\nabla \cdot \mathbf{u} &amp;amp;= -q  \quad in \quad \Omega \\&#xA;p &amp;amp;= g  \quad on \quad \partial  \Omega_D\\&#xA;\mathbf{u} \cdot \mathbf{n} &amp;amp;= f \quad on \quad \partial \Omega_N&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;multiplied the equations with the test functions $\mathbf{v}$ and $w$ and integrating by parts:&#xA;$$&#xA;\begin{align}&#xA;\int_{\Omega} (\mathbf{k}^{-1} \cdot \mathbf{u}) \mathbf{v}\mathrm{d}x + \int_{\Omega_i} \{p\} \mathbf{v} \cdot \mathbf{n} \mathrm{d}s + \int_{\Omega_D} g \mathbf{v} \cdot \mathbf{n} \mathrm{d}s + \int_{\Omega_N} p \mathbf{v} \cdot \mathbf{n} \mathrm{d}s - \int_{\Omega} p \nabla \cdot \mathbf{v} \mathrm{d}x= 0&#xA;\end{align}&#xA;$$&#xA;and&#xA;$$&#xA;- \int_{\Omega_D} (\mathbf{u} \cdot \mathbf{n}) w \mathrm{d}s - \int_{\Omega_N} f w \mathrm{d}s + \int_{\Omega} ( \mathbf{u} \cdot \nabla w) \mathrm{d}x = - \int_{\Omega} qw \mathrm{d}x&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\{p\}=0.5(p^++p^-)$ . Since $p$ is discontinuous, I need a trace term (I think). What about $\mathbf{u}$ ? since it is a Raviart Thomas element, do I need to do anything for it? Should I change $\mathbf{u}$ to be DG (of order degree($p$)+1) as well? Does it matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the following weak formulations:&#xA;$$&#xA;\begin{align}&#xA;A(\{\mathbf{u},p\},\{\mathbf{v},w\}) &amp;amp;= (\mathbf{k}^{-1} \cdot \mathbf{u},\mathbf{v})_{\Omega} + (\{p\},\mathbf{v}\cdot \mathbf{n})_{\Omega_i}+ (p,\mathbf{v}\cdot \mathbf{n})_{\Omega_N} - (p, \nabla \cdot \mathbf{v})_{\Omega} \\&#xA; &amp;amp;- (\mathbf{u} \cdot \mathbf{n}, w)_{\Omega_D} + (\mathbf{u}, \nabla w)_{\Omega} \\&#xA;F(\{\mathbf{v},w \}) &amp;amp;= -(g,\mathbf{v} \cdot \mathbf{n})_{\Omega_D}+(f,w)_{\Omega_N}- (q,w)_{\Omega}&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I perform surface integrals where necessary, and all boundary conditions are imposed weakly. Would this be a stable formulation? If I do &lt;em&gt;not&lt;/em&gt; do integration by parts for the second equation $-{\textrm{div}}\ {\mathbf u} = -q$, would imposing $\mathbf{u}\cdot \mathbf{n} = f$ on $\partial \Omega_N$ strongly be better/worse or is it irrelevant? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much in advance.&lt;/p&gt;&#xA;" OwnerUserId="3758" LastActivityDate="2014-10-24T21:05:35.170" Title="Mixed DG for Poisson with mixed BC's" Tags="|finite-element|discontinuous-galerkin|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15978" PostTypeId="2" ParentId="15973" CreationDate="2014-10-24T21:18:46.560" Score="5" Body="&lt;p&gt;In the literature these boundary conditions go by the name of absorbing boundary conditions (or nonreflecting, open, radiation, invisible, far-field), and this is a well-known topic. One clear description I think is &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/S1064827594277053&quot; rel=&quot;noreferrer&quot;&gt;Absorbing Boundary Conditions for the Schrödinger Equation&lt;/a&gt; by Fevens and Jiang.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is one approach (described in the above paper; I haven't implemented and checked it directly). If you had the wavefunction $e^{i(-k^2 t + k x)}$, satisfying $\partial_t\psi=i\partial_{xx}\psi$, corresponding to a free plane wave, with a known fixed momentum $k$, then this wavefunction also satisfies the first-order equation&#xA;$$ (i\partial_x + k)\psi = 0. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you can guess that a certain wavenumber $k$ is common (because, say, you construct the initial condition to be a wave packet around this wavenumber), the boundary condition&#xA;$$ i\partial_x + k = 0$$&#xA;will be nonreflecting for the part of your solution that has the wavenumber $k$. It will be reflecting for other wavenumbers, but it will be more weakly reflecting for the wavenumbers close to $k$. The sign of $k$ is not arbitrary, and determines the direction in which the boundary is nonreflecting. The paper I linked to gives other, more accurate, boundary conditions as well.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-10-24T21:18:46.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15979" PostTypeId="1" AcceptedAnswerId="15992" CreationDate="2014-10-24T22:34:55.677" Score="1" ViewCount="575" Body="&lt;p&gt;I used Gurobi with a MIQP with 26 binary variables and 26*4 interaction term without any other constraint. The speed is very slow already.... I want to ask what is the state of art of MIQP solvers. Generally how many variable can it handle or do we need to manually input linearization techniques to enhance the speed. thank you:)&lt;/p&gt;&#xA;" OwnerUserId="11580" LastEditorUserId="276" LastEditDate="2014-10-26T07:12:21.873" LastActivityDate="2014-10-26T07:12:21.873" Title="Mixed-integer quadratic programming, state of art" Tags="|quadratic-programming|mixed-integer-programming|" AnswerCount="1" CommentCount="2" ClosedDate="2014-10-26T08:04:06.847" ContentLicense="CC BY-SA 3.0" />
  <row Id="15981" PostTypeId="2" ParentId="15966" CreationDate="2014-10-25T00:46:53.930" Score="2" Body="&lt;p&gt;I'm going to boldly say that you are doing it wrong, because your goal is to minimize a weighted nuclear norm but your equations don't agree.  The nuclear norm is the sum of singular values.  The weighted nuclear norm is the weighted sum of singular values.  If your matrix A is intended to be the diagonal matrix of singular value weights, then I don't think your equation represents the correct weighted sum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The response to the question &lt;a href=&quot;http://ask.cvxr.com/question/1708/maximize-the-minimum-singular-value/&quot; rel=&quot;nofollow&quot;&gt;http://ask.cvxr.com/question/1708/maximize-the-minimum-singular-value/&lt;/a&gt; suggests that the minimum singular value is not a convex or concave function so cvx can't deal with it.  Therefore it seems unlikely that cvx can deal with an arbitrary weighted sum of singular values, as would be required by a weighted nuclear norm.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2014-10-25T00:53:12.303" LastActivityDate="2014-10-25T00:53:12.303" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="15982" PostTypeId="2" ParentId="15939" CreationDate="2014-10-25T10:22:24.113" Score="2" Body="&lt;p&gt;Since you have a scalar problem in one space dimension, it shouldn't be too difficult to come up with a semi-analytic solution using the method of characteristics.  Here's something to get you started.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem can be written in a nicer form if you define $g=1/f$.  Then you can solve for $s$ to find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$s(g,x,\phi) = \frac{\alpha(x,\phi)}{\alpha(x,\phi) + g - 1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha(x,\phi) = \begin{cases} \frac{1+\frac{4}{10}\phi e^{-x}}{10} &amp;amp; \phi&amp;gt;0 \\ \frac{1}{10} &amp;amp; \phi \le 0.\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your conservation law is simply&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g_x - \left( s(g,x,\phi) g \right)_\phi = 0,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which looks like a traditional scalar hyperbolic PDE.  The method of characteristics will allow you to find solutions up to the &quot;time&quot; when characteristics cross.  You will probably need a Newton solver and an ODE solver to implement the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case you want to compute an approximate solution by numerical discretization, I recommend the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;If you know your solutions do not contain shocks&lt;/strong&gt;, then I would just code up a simple finite difference discretization of the equation above (for instance, using Python or MATLAB and centered differences).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If you need to compute solutions with shocks&lt;/strong&gt;, then more sophisticated methods are called for.  You might consider using &lt;a href=&quot;http://www.clawpack.org/pyclaw&quot; rel=&quot;nofollow&quot;&gt;PyClaw&lt;/a&gt; (disclaimer: I'm a PyClaw developer).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-10-28T05:34:52.340" LastActivityDate="2014-10-28T05:34:52.340" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15984" PostTypeId="1" CreationDate="2014-10-25T15:31:56.983" Score="1" ViewCount="146" Body="&lt;p&gt;In &lt;strong&gt;combinatorial optimization&lt;/strong&gt;, there are many problems that can be formulated as either &lt;strong&gt;&lt;em&gt;Network Flow&lt;/em&gt;&lt;/strong&gt; model or &lt;strong&gt;&lt;em&gt;Mixed Integer Programming&lt;/em&gt;&lt;/strong&gt; (MIP), e.g. supply chains, transportation, and graph-base problems. Some solvers make use of logical and/or graph-base syntax to efficiently solve network problems. And then the Network Simplex method is applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, As stated in &lt;em&gt;Bazaraa, M.S., Jarvis, J.J. , and Sherali, H.D.; Linear programming and network flows, 4th Edition, Hoboken, New Jersey: Wiley &amp;amp; Sons, Inc., 2010&lt;/em&gt;; &lt;strong&gt;page 453&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We discuss appropriate data structures that facilitate the implementation of such a graph-theoretic procedure on the computer. The overall efficiency with which such a procedure operates enables one to solve problems &lt;strong&gt;200-300 times faster&lt;/strong&gt; than with a standard simplex approach that ignores any inherent special structures other than &lt;strong&gt;&lt;em&gt;sparsity&lt;/em&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Practically speaking, from the aspect of &lt;em&gt;time efficiency&lt;/em&gt;, are there any significant differences between modelling as a mixed integer programming and modeling as a network problem? And why (other than sparsity)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which &lt;em&gt;optimization solvers&lt;/em&gt; are computationally fast at solving Network problems?&lt;/p&gt;&#xA;" OwnerUserId="11547" LastActivityDate="2014-12-25T08:11:14.453" Title="Comparison of the time efficiency of an optimization problem formulated as a Network Flow model and Mixed Integer Programming" Tags="|optimization|efficiency|mixed-integer-programming|combinatorics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="15986" PostTypeId="1" CreationDate="2014-10-25T16:22:40.463" Score="0" ViewCount="548" Body="&lt;p&gt;I'm trying to plot this function &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;func=exp(x-sqrt(2))-cos(x-sqrt(2))-(x-sqrt(2)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in Matlab using 101 evenly spaced x values. If I just say&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fplot(fun, [-1 3], 'o') &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the graph is the correct shape but the actual values of the points are determined by Matlab to give the best shape of the graph. I want to tell it exactly where to put the points, 101 evenly spaced between -1 and 3.&lt;/p&gt;&#xA;" OwnerUserId="11609" LastEditorDisplayName="user10435" LastEditDate="2015-03-14T04:12:09.800" LastActivityDate="2015-03-14T04:12:09.800" Title="How to plot function with evenly spaced x values in Matlab" Tags="|matlab|plotting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15988" PostTypeId="2" ParentId="15986" CreationDate="2014-10-25T16:53:10.107" Score="3" Body="&lt;p&gt;You can use &lt;code&gt;linspace&lt;/code&gt; function,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=linspace(-1,3,101);&#xA;func=exp(x-sqrt(2))-cos(x-sqrt(2))-(x-sqrt(2));&#xA;plot(x,func,'or');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2014-10-25T16:53:10.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="15992" PostTypeId="2" ParentId="15979" CreationDate="2014-10-26T07:01:25.327" Score="2" Body="&lt;p&gt;For LPs, MILPs, and QPs, Gurobi and CPLEX are considered best-of-breed. They beat any open-source general-purpose solver by at least an order of magnitude. I see no reason why that would be different for QPs. Both companies have dedicated teams that comb the literature for various cuts to use in constructing a branch-and-bound tree, and employ heuristics that perform well in terms of trying out these various cuts at different points in the algorithm. Both solvers also have sophisticated presolve routines that preprocess the input optimization problem to speed up the solver algorithms employed after preprocessing, and these routines often make a big difference in solution time compared to an equivalent solver that does not use those presolve routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way you can &quot;beat CPLEX and Gurobi&quot; is to exploit structure, which has been done successfully in stochastic programming. For your problem, without more information, it is difficult to say whether you can exploit structure to accelerate time to solution. Formulation is important for MILPs and MINLPs, so I wouldn't be surprised if it is also important for MIQPs. You might also try alternate formulations of your problem to see if you can achieve better results, or try looking at the verbose output of Gurobi's solver and see if you can change solver input parameters to speed up time to solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd also say that number of variables is a very coarse indicator of problem difficulty. What matters more is how much of the branch-and-bound tree needs to be explored. There are large problem instances out there that do not require much of the branch-and-bound tree to be enumerated. I don't work with MIQPs much, but for MILPs, I've successfully solved problem instances with several thousand integer variables because only a small portion of the branch-and-bound tree must be enumerated. Obviously, if your problem is prohibitively large (say, hundreds of thousands of integer variables with no special structure), then solving the MILP directly is out of the question, and you'll have to content yourself with solving a relaxation, or finding a feasible point with large optimality gap. In your case, a problem with 26 variables is probably on the small side for MIQP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of providing integrality cuts to the solver, I don't know if you could do that, but in theory, if you knew how to provide good cuts, that would be valuable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an alternate approach, you could try MINLP solvers instead. The state-of-the-art code there is BARON, but there are others, like Couenne, and a few more. BARON generally does best on MINLPs, but isn't universally the fastest code for every problem. You could also take a look at something specialized for quadratic problems. GLOMIQO is really geared towards MIQCQPs, but could work for your problem. Both BARON and GLOMIQO are bundled with GAMS, so you could probably try submitting your problem to the NEOS server if there are no IP-type issues, and see if you get a solution more quickly.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-10-26T07:11:40.773" LastActivityDate="2014-10-26T07:11:40.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15993" PostTypeId="2" ParentId="15984" CreationDate="2014-10-26T07:23:05.337" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Practically speaking, from the aspect of time efficiency, are there any significant differences between modelling as a mixed integer programming and modeling as a network problem? And why (other than sparsity)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. The reason network simplex is faster primarily has to do with exploiting the &lt;a href=&quot;http://en.wikipedia.org/wiki/Unimodular_matrix#Total_unimodularity&quot; rel=&quot;nofollow&quot;&gt;total unimodularity of network matrices&lt;/a&gt; -- basically, network matrices are always structured such that their submatrices have integral inverses. This property means that for network problems, instead of having to enumerate a branch-and-bound tree to obtain an integral solution, we can invert the columns of the coefficient matrix corresponding to variables in the simplex basis and obtain an integral solution. These inversion operations are cheaper than solving the branch-and-bound tree, in that they run in worst-case polynomial time (even better than ordinary simplex, which runs in worst-case exponential time, but typically in linear time); branch-and-bound runs in worst-case exponential time, and also typically runs in exponential time for general mixed-integer programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure, sparsity is nice too, and a typical property of network problems, but the main speed gains come from being able to use regular linear algebra operations to obtain integral solutions instead of branch-and-bound, which is a glorified guess-and-check scheme.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-26T07:23:05.337" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15994" PostTypeId="2" ParentId="15964" CreationDate="2014-10-26T07:37:56.433" Score="1" Body="&lt;p&gt;Most of the DAE solvers I'm aware of don't take advantage of symplectic structure, but it's entirely possible that one of them does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, if you use a DAE solver, you are using an implicit method, or maybe a semi-implicit method; the algebraic variables are determined through a nonlinear solve, and I don't see how you get around that. I don't know how you enforce constraints. If you modify the positions via a nonlinear solve, then a semi-implicit DAE solver might be competitive; if modifying positions is cheaper than a nonlinear solve, then the DAE solver is probably more expensive. A fully implicit DAE solver is more likely to be slower than your current approach; however, DAE solvers have solid convergence theory behind them (see Hairer and Wanner's classic text; Brenan and Petzold's book, and the book by Roswitha Marz), which would put the numerical error analysis on more solid footing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other approaches to constraint satisfaction also exist. You could try looking at what the molecular dynamics community does to enforce constraints in their mechanical systems, such as via Lagrange multiplies, or specialized algorithms geared towards enforcing things like bond lengths and bond angles of atoms in molecules. It sounds like what you're doing is solving a projected dynamical system, and you want to be careful with that projection operation. I remember one of my advisors discussing how projection can affect stability, such that perturbations off of the constraint manifold due to, well, floating-point arithmetic, can yield unstable solutions because the underlying ODE system (before projection) is unstable in the off-manifold directions. You could also look at differential-variational inequalities. Both variational inequality solvers and DAE solvers are available in PETSc; you can find an implicit DAE solver in SUNDIALS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you want to watch out for with a mechanical system is the index of your DAE. (There are many definitions of index; I'm just going to assume &quot;index&quot; means &quot;differential index&quot;, even though I'm vaguely aware there are pitfalls in using that definition.) Mechanical systems may have multiple DAE formulations, and some of them are high-index (in practice, this means the index is greater than 2). Most DAE solvers tend to solve index-1 or semi-explicit Hessenberg index-2 DAEs; if you have a system not in that form, you'll need a specialized solver (there are a few index-3 solvers specialized for mechanical systems) or you'll need to reformulate your problem via something like a change in coordinates.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-26T07:37:56.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15996" PostTypeId="1" AcceptedAnswerId="15997" CreationDate="2014-10-26T07:47:58.040" Score="5" ViewCount="116" Body="&lt;p&gt;Sometimes it is meaningful to remove the influence of some old data from a SVD-based model, so as to reflect the most updated trends and provide more accurate results. I've seen there're incremental SVD approaches adding new data, e.g, a simple one is called folding-in. I am wondering is there any inverse operations which remove the old data, or some selected data specified by me, from a common SVD model?&lt;/p&gt;&#xA;" OwnerUserId="7969" LastActivityDate="2014-10-26T08:02:48.390" Title="Are there any algorithms &quot;incrementally remove part of data (esp., old data)&quot; from the existing SVD model of a data?" Tags="|matrix|svd|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15997" PostTypeId="2" ParentId="15996" CreationDate="2014-10-26T08:02:48.390" Score="5" Body="&lt;p&gt;Yes. Take a look at the 2002 incremental SVD paper by Matthew Brand. In it, he discusses how to re-weight the SVD to dilute the influence of old data. He also has a 2006 paper on rank-1 updates of SVDs, and I believe downdates are also discussed. The best review-type paper on incremental SVDs I've seen was by CG Baker, Van Dooren, and Gallivan in 2012, which might have some other good references, but doesn't directly discuss SVD downdates. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-26T08:02:48.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15998" PostTypeId="1" CreationDate="2014-10-26T07:25:50.903" Score="2" ViewCount="194" Body="&lt;p&gt;My 2D-Poisson solver is build for simulation of semiconductor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm is Gauss-Seidel iterative method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use simple PN junction for simulation testing, it diverged while I applied high voltage on electrode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it is fine while there is no applied voltage or very low voltage applied on electrodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following are example for 1D discretized equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/eqkjJ.png&quot; alt=&quot;1D discretized Poisson&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The right hand side have linear and exponential parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A small change of potential cause large difference of charge density, and the large difference of charge density cause larger change of potential in next iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the Possible reason that cause Poisson solver diverged?&#xA;How to avoid it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If the description is still unclear I can upload a part of my code.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is the DD model tutorial I followed which is written by Dragica Vasileska.&#xA;&lt;a href=&quot;https://nanohub.org/resources/1545/download/ddmodel_introductory_part_word.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nanohub.org/resources/1545/download/ddmodel_introductory_part_word.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11614" OwnerDisplayName="Zhean Lee" LastActivityDate="2014-10-29T08:16:03.147" Title="Poisson solver diverged" Tags="|simulation|software|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="15999" PostTypeId="2" ParentId="10167" CreationDate="2014-10-26T14:55:24.370" Score="2" Body="&lt;p&gt;This is the classical harmonic mean interface conductivity expression - See &quot;Numerical Heat Transfer and Fluid Flow&quot; by S.V. Patankar&lt;/p&gt;&#xA;" OwnerUserId="11617" LastActivityDate="2014-10-26T14:55:24.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16000" PostTypeId="1" AcceptedAnswerId="16001" CreationDate="2014-10-26T21:44:10.107" Score="1" ViewCount="1166" Body="&lt;p&gt;I would like to get the result of Fresnel-Kirchhoff integral in paraxial approximation to the cylindrical diffracted wave and here is a part of my code and if it is solved gonna be carrying the other parts out. It is a Matlab code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vpa(int(exp(-1i*2*k*(n_cl-n_s)*sqrt(b^2-x_p^2))*exp(1i*k*((x-x_p)^2)/2z),-b,-a)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with respect to x_p. where &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;b = 62.5*e-6; % cladding radius&#xA;a = 4*e-6; % core radius&#xA;n_co = 1.473; % core ref index&#xA;n_cl = n_co - 0.016; % cladding ref index&#xA;n_s  = 1.37868e-5; % surrounding ref index&#xA;L = 457.9e-9; % wavelength&#xA;k = 2*pi/L; % wavenumber&#xA;z = 0.4 ; &#xA;x =-0.0002;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which command should be used as an integrator? I've been trying to use &lt;code&gt;int()&lt;/code&gt; instead of &lt;code&gt;quad&lt;/code&gt; since &lt;code&gt;quad&lt;/code&gt; doesn't fulfill the requirement to obtain complex results. In addition to that I've been getting the well-known error of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Explicit integral could not be found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Don't know what to do. Please help me. &lt;/p&gt;&#xA;" OwnerUserId="11616" LastEditorUserId="9667" LastEditDate="2017-03-17T01:45:52.603" LastActivityDate="2017-03-17T01:45:52.603" Title="numerical integration of exponential which contains complex function" Tags="|matlab|numerical-modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16001" PostTypeId="2" ParentId="16000" CreationDate="2014-10-27T01:08:05.413" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;http://www.mathworks.com/help/symbolic/int.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;int&lt;/code&gt;&lt;/a&gt; function (more precisely &lt;code&gt;sym/int&lt;/code&gt;) is mainly designed for symbolic integration. When it cannot obtain a symbolic result it will attempt a numeric solution using MuPAD's &lt;a href=&quot;http://www.mathworks.com/help/symbolic/mupad_ref/numeric-int.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;numeric::int&lt;/code&gt;&lt;/a&gt;. I don't know why this won't evaluate to a numeric result, but you can use MuPAD's &lt;a href=&quot;http://www.mathworks.com/help/symbolic/mupad_ref/numeric-quadrature.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;numeric::quadrature&lt;/code&gt;&lt;/a&gt;, which underlies &lt;code&gt;numeric::int&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;evalin(symengine,'numeric::quadrature(exp(-(pi*(4722366482869645/1208925819614629174706176 - x_p^2)^(1/2)*55043382873750776053760*i)/4324743225012021)*exp(((x_p + 1/5000)^2*7366803725989309*i)/2684354560), x_p == -1/16000..-4722366482869645/1180591620717411303424)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, based on all of your floating-point parameter values, I'd imagine that a numeric &lt;a href=&quot;http://en.wikipedia.org/wiki/Quadrature_(mathematics)&quot; rel=&quot;nofollow&quot;&gt;quadrature&lt;/a&gt;-based solution is what you're actually looking for. You should be using &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/integral.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;integral&lt;/code&gt;&lt;/a&gt; (or possibly &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/quadgk.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;quadgk&lt;/code&gt;&lt;/a&gt; in older versions of Matlab):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integral(@(x_p)exp(-1i*2*k*(n_cl-n_s)*sqrt(b^2-x_p.^2)).*exp(1i*k*((x-x_p).^2)/2*z),-b,-a)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results returned by both methods agree with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&#xA;NOTE: there's a typo in your equation. I've assumed that you meant &lt;code&gt;...)/2*z&lt;/code&gt; as opposed to &lt;code&gt;...)/(2*z)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastActivityDate="2014-10-27T01:08:05.413" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16002" PostTypeId="1" CreationDate="2014-10-27T02:57:46.260" Score="2" ViewCount="297" Body="&lt;p&gt;I use B-spline curve fitting to obtain one smooth curve. If I obtain two smooth B-spline , how can I connect then smoothly. For example, I have 59 points((x0,y0,z0),...,(x58, y58, z58)) and I have two fitted B spline. One B-spline is for the first 30 points, another is for the next 30 points and the two point set share one common point((x29,y29,z29)). The point (x29,y29,z29) will be modified twice due to curve fitting and will have two new positions. If I just connect the two new positions, the final curve will not be smooth at the point (x29,y29,z29). Currently I perform curve fitting for all data together but that will modify the smooth curve for the first 30 points entirely. I hope to only modify the connecting part of the first smooth curve. I know I need to impose derivatives need to be equal at the joint. I don't know how to do that.&lt;/p&gt;&#xA;" OwnerUserId="7846" LastActivityDate="2017-12-03T18:20:52.750" Title="How to connect two fitted B-spline curve?" Tags="|b-spline|smoothing|curve-fitting|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16003" PostTypeId="2" ParentId="14691" CreationDate="2014-10-27T09:09:08.340" Score="2" Body="&lt;p&gt;Maple has a built-in function for converting the code into other languages. It is called &lt;a href=&quot;http://www.maplesoft.com/products/maple/features/codegen.aspx&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Code Generation&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can reach a person who has Maple and ask her/him to do it for you.&lt;/p&gt;&#xA;" OwnerUserId="9627" LastActivityDate="2014-10-27T09:09:08.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16004" PostTypeId="1" AcceptedAnswerId="16005" CreationDate="2014-10-27T13:20:13.450" Score="1" ViewCount="88" Body="&lt;p&gt;I have a a function $f(k)$ (calculated in Maple) which is huge and stored in a variable called 'sum' on my drive (with the help of 'save' command in Maple). Since the function is huge, Maple is unable to plot it and is taking endless time. Thus, I want to read the variable 'sum' into Matlab and plot it. I am unable to also just copy paste as the function is really big. I have searched around but am unable to find a solution. Can somebody help me out?&lt;/p&gt;&#xA;" OwnerUserId="9756" LastEditorUserId="9756" LastEditDate="2014-10-27T13:27:42.113" LastActivityDate="2014-10-27T15:05:31.430" Title="Maple stored variable to be read into Matlab" Tags="|matlab|data-storage|maple|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16005" PostTypeId="2" ParentId="16004" CreationDate="2014-10-27T15:05:31.430" Score="2" Body="&lt;p&gt;You can use the &lt;a href=&quot;http://www.maplesoft.com/support/help/maple/view.aspx?path=CodeGeneration&quot; rel=&quot;nofollow&quot;&gt;CodeGeneration&lt;/a&gt; package to do this. It allows to translate to different languages, being &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=CodeGeneration/Matlab&quot; rel=&quot;nofollow&quot;&gt;Matlab&lt;/a&gt; one of those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple example here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with(CodeGeneration);&#xA;suma := sum(sin(n*x)/factorial(n), n = 0 .. 10);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Matlab(suma)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with answer&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cg3 = sin(x) + sin(0.2e1 * x) / 0.2e1 + sin(0.3e1 * x) / 0.6e1 + sin(0.4e1 * x) / 0.24e2 + sin(0.5e1 * x) / 0.120e3 + sin(0.6e1 * x) / 0.720e3 + sin(0.7e1 * x) / 0.5040e4 + sin(0.8e1 * x) / 0.40320e5 + sin(0.9e1 * x) / 0.362880e6 + sin(0.10e2 * x) / 0.3628800e7;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Off course, you can store this in a string and then write it to a text file, or translate a Maple procedure to a Matlab function.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2014-10-27T15:05:31.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16011" PostTypeId="1" CreationDate="2014-10-28T12:32:27.790" Score="2" ViewCount="858" Body="&lt;p&gt;I have a lengthy expression which is calculated using symbolic differentiation of a rather complicated function. I now intend to call this resulting (symbolic function) in a for loop many times, and so I would like to avoid having to perform the symbolic differentiation step every single iteration, rather I would like to &quot;save&quot; the result of the symbolic differentiation, so that on each loop I'm just evaluating a function. Maybe an example will help: Here we try to evaluate the cosine of zero using two ways.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms x&#xA;f = sin(x);&#xA;fp = diff(f,x);&#xA;fp = @(x) subs(fp, 'x', x);&#xA;&#xA;tic &#xA;for i=1:1000&#xA;    fp(0);&#xA;end&#xA;toc&#xA;&#xA;tic &#xA;for i=1:1000&#xA;    cos(0);&#xA;end&#xA;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Elapsed time is 3.654296 seconds.&#xA;Elapsed time is 0.007301 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hence using my symbolic function is super expensive! Is there anyway to mitigate this expensive function call?&lt;/p&gt;&#xA;" OwnerUserId="7864" LastActivityDate="2014-10-28T13:59:26.630" Title="Saving symbolic function to increase efficiency in Matlab" Tags="|matlab|numerical-analysis|symbolic-computation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16012" PostTypeId="2" ParentId="16011" CreationDate="2014-10-28T12:45:38.777" Score="4" Body="&lt;p&gt;Using the function &lt;code&gt;matlabFunction&lt;/code&gt; increases the efficiency by a lot!   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms x&#xA;f = sin(x);&#xA;fp = diff(f,x);&#xA;F = matlabFunction(fp);&#xA;tic &#xA;for i=1:1000&#xA;    F(0);&#xA;end&#xA;toc&#xA;&#xA;tic &#xA;for i=1:1000&#xA;    cos(0);&#xA;end&#xA;toc&#xA;&#xA;Elapsed time is 0.037475 seconds.&#xA;Elapsed time is 0.004203 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Furthermore if the result of the symbolic calculations needs to be saved as a conventional function we can simply use &lt;code&gt;matlabFunction(fp, 'file', 'myfunction')&lt;/code&gt; which will save the results to a function file. &lt;/p&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="7864" LastEditDate="2014-10-28T13:59:26.630" LastActivityDate="2014-10-28T13:59:26.630" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16013" PostTypeId="1" AcceptedAnswerId="16019" CreationDate="2014-10-28T13:10:14.970" Score="4" ViewCount="2111" Body="&lt;p&gt;I am trying to find the minimum of the so called Beale’s function given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x_1,x_2) = (1.5-x_1+x_1x_2)^2 + (2.25-x_1+x_1x_2^2)^2 + (2.625-x_1+x_1x_2^3)^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Newton iteration &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x^{(k+1)} = x^{(k)} - \alpha^{(k)} \eta^{(k)} $ with $ \eta^{(k)} = \left( \frac{\partial ^2f}{\partial x^2}(x^{(k)}) \right)^{-1}\frac{\partial f}{\partial x}(x^{(k)})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;at point  $x = [4 \ \  1]^T$ with $\alpha = 0.5$. The result is $x_{num} = [0 \ \ 1]^T$, while the real minimum is at $x_{min} = [3 \ \ 0.5]^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found out the problem is, that $\eta$ is pointing in the wrong direction. It is not showing “downhill” like the gradient would. My question is, how is this possible? Why does the newton iteration is failing here? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: By the way, I get the some result if I start at $x = [4 \ \ 0.9]^T$. First happens some weird stuff and $x^{(k)}$ is jumping around but then it ends again in $x= [0 \ \ 1]^T$.&lt;/p&gt;&#xA;" OwnerUserId="10329" LastEditorUserId="1804" LastEditDate="2014-10-29T17:09:34.967" LastActivityDate="2014-10-29T17:09:34.967" Title="Beale's function and newton iteration" Tags="|optimization|numerical-analysis|convergence|newton-method|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16014" PostTypeId="2" ParentId="15939" CreationDate="2014-10-28T13:14:13.067" Score="3" Body="&lt;p&gt;This is just a small addition to David Ketcheson's comments about solving numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to obtain a numerical solution using MATLAB, Lawrence Shampine has written&#xA;a MATLAB solver for hyperbolic PDEs that is designed to be (relatively) easy to use.&#xA;If you look about half-way down this page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://faculty.smu.edu/shampine/current.html&quot; rel=&quot;nofollow&quot;&gt;http://faculty.smu.edu/shampine/current.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you will see a section titled &quot;Hyperbolic PDEs&quot; that has links to his paper and MATLAB code for this capability. Because it is written entirely in MATLAB code, the &quot;installation&quot; is trivial.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-10-28T13:14:13.067" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16015" PostTypeId="2" ParentId="16013" CreationDate="2014-10-28T13:48:57.253" Score="3" Body="&lt;p&gt;Newton's method is a fixed point iteration for solving a nonlinear equation. In the context of optimizing a function $f(x)$, you apply this method to find a solution of the necessary optimality conditions $f'(x) = 0$.&#xA; However, this condition characterizes all stationary points such as maximizers or saddle points, so the method is just as happy to give you one of those if they exist. Furthermore, Newton's method is only locally convergent, so it will give you the closest local minimizer (or maximizer) to your starting point (if it converges at all).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beale's function has indeed a saddle point at $(0,1)$, since $\partial_x f(0,1) = \partial_y f(0,1) = 0$, but the Hessian&#xA;$$&#xA;\begin{pmatrix} \partial_{xx} f(0,1) &amp;amp; \partial_{xy} f(0,1) \\ \partial_{xy} f(0,1) &amp;amp; \partial_{yy} f(0,1)\end{pmatrix} = \frac{111}{4}\begin{pmatrix} 0 &amp;amp; 1 \\ 1 &amp;amp; 0 \end{pmatrix}&#xA;$$&#xA;has the eigenvalues $\pm 111/4$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, Beale's method is a popular torture test to illustrate why global minimizers are difficult to compute...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Gradient methods with an appropriate line search have an additional mechanism that tries to enforce (sufficient) function value decrease and thus will avoid maximizers in most cases. They also tend to avoid saddle points (at least compared to Newton's method), but as Geoff Oxberry points out, there's no guarantee.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2014-10-28T22:33:02.473" LastActivityDate="2014-10-28T22:33:02.473" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16016" PostTypeId="1" AcceptedAnswerId="16018" CreationDate="2014-10-28T14:38:05.197" Score="6" ViewCount="1212" Body="&lt;p&gt;I have been working on a particularly challenging problem and was hoping for some guidance. Here is my problem. I have a point cloud containing millions of points. For each point in the set, I need to determine whether there exists an enclosing circle of radius &lt;em&gt;r&lt;/em&gt; that can be drawn around the point (the point can lie anywhere within the circle) but that does not also contain any neighbouring point. Here is a diagram to help illustrate my scenario.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8dNon.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The blue point in the diagram above is the point of interest. The red points all lie within a &lt;em&gt;2r&lt;/em&gt; radius of the point of interest (any neighbouring points beyond this distance couldn't affect the result). In this case it is possible to draw a &lt;em&gt;r&lt;/em&gt; radius circle that contains the point of interest and no other points. In fact there are clearly many such candidate circles. I'm not interested in getting all of the candidates, but rather simply knowing whether this condition occurs at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps there is an obvious way of performing this analysis that I am overlooking. I have thought about using Delaunay triangulation (does the point of interest lie inside a triangular facet?) but I'm not convinced that approach will work because of the specified circle size. I know that if there are two or fewer neighbours, then it is always possible to fit a circle that meets the requirements. I also know that if the convex hull of the neighbours doesn't enclose the point of interest, my condition is met. But none of these things help me with the general solution. Does anyone know of an approach or algorithm that could be used to accomplish this task? &lt;/p&gt;&#xA;" OwnerUserId="11636" LastEditorUserId="11636" LastEditDate="2014-10-28T18:38:31.157" LastActivityDate="2014-10-28T19:23:52.510" Title="A method to determine whether a point can be contained within a circle with no neighbouring points" Tags="|algorithms|computational-geometry|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16017" PostTypeId="2" ParentId="16016" CreationDate="2014-10-28T16:16:16.447" Score="2" Body="&lt;p&gt;I don't think this is optimal, but this doesn't fit into a comment. A maximal enclosing circle around a point would have to pass through at least three other points, as otherwise it can be enlarged or is infinite in some directions. So for a given point, it might be possible to look at all triples of &quot;nearby&quot; points (in the sense of being within a few hops in the Delaunay triangulation graph, for example) and check whether the unique circle passing through that triple has radius $\geq r$ and encloses one and only one other point. Then that enclosed point satisfies the conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that the centre of the enclosing circle isn't necessarily the point it encloses, the Delaunay graph wouldn't easily produce the optimal sets of triples to check, I think. So checking lots of nearby points might be a start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that it's not at all clear how to pick &quot;nearby&quot; triples so that the result is optimal, so maybe this only makes sense as an approximation. The accuracy might depend on how uniformly distributed the points are.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-10-28T16:16:16.447" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16018" PostTypeId="2" ParentId="16016" CreationDate="2014-10-28T19:23:52.510" Score="6" Body="&lt;p&gt;Let the circle centered at your blue point with radius $r$ be denoted as $C_r$ and with radius $2r$ (the solid circle in your diagram) as $C_{2r}$.  For each red point $p_i \in C_{2r}$ construct a circle of radius $r$ centered at $p_i$ and call it $C_i$.  Any circle centered inside of $C_i$ contains $p_i$ and is therefore invalid.  Any circle centered inside of $C_r$ can possibly be valid.  Therefore you need to find a point that is inside of $C_r$ but outside of all $C_i$.  Your solution is then $C_r\setminus\cup C_i $.  Of course finding these points is easier said than done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The union of the circles can be found using generalized voronoi diagrams as discussed in this mathoverflow &lt;a href=&quot;https://mathoverflow.net/questions/24859/finding-the-union-of-n-random-circles-arbitrarily-or-conspiratorially-placed-o&quot;&gt;question&lt;/a&gt;.  This is likely overkill for you question though since you don't need the entire space, only a single point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm I think you need is this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Construct a voronoi diagram of your red points.&lt;/li&gt;&#xA;&lt;li&gt;Find the intersection of $C_r$ (a circle centered at the blue point with radius $r$) with all the edges of the voronoi diagram&lt;/li&gt;&#xA;&lt;li&gt;Check the distance from each vertex of the voronoi diagram inside $C_r$ and the points constructed in step 2. to one of the red points whose region the point borders.  If it is atleast $r$, then you are  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;By construction they are all within $r$ of the blue point.  The vertices around a region of a voronoi diagram are as far as you can possible get from the points from which the diagram was constructed.  For example if a vertex is at the border of 3 regions, by definition it must be equidistant from all 3 red points used to construct those three regions of the diagram.  Therefore you only need to check its distance from 1 of them.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example I have taken the image from the Wikipedia article on &lt;a href=&quot;http://en.wikipedia.org/wiki/Voronoi_diagram&quot; rel=&quot;nofollow noreferrer&quot;&gt;Voronoi Diagrams&lt;/a&gt; and edited it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FL3T7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red and blue points are the same as in your diagram.  The yellow circle is a circle of radius $r$ centered at the blue point.  All you need to do is check each of the Cyan points distance to a single red point in a neighboring region.  If a circle can't be placed centering at one of the cyan points, then no circle can be placed that fits your requirements.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2014-10-28T19:23:52.510" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="16019" PostTypeId="2" ParentId="16013" CreationDate="2014-10-28T20:37:57.437" Score="2" Body="&lt;p&gt;For unconstrained optimization problems, deterministic optimization methods converge to stationary points. Stationary points are points at which the gradient of the objective function is zero. These points are not necessarily optima, unless other conditions are met. For minimization problems, if the Hessian evaluated at a stationary point is positive semidefinite, then that stationary point is a local minimum. If the objective function is convex (equivalently, for twice continuously differentiable objective functions, the Hessian is positive semidefinite everywhere) then a stationary point is a global minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem you're running into is that the objective function you are trying to minimize is nonconvex. As ChristianClason points out, you converge to a stationary point, but that stationary point is a saddle point, implying that your objective function is not convex, and thus, convergence to a global minimum is not assured.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-28T20:37:57.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16020" PostTypeId="2" ParentId="15998" CreationDate="2014-10-29T08:16:03.147" Score="1" Body="&lt;p&gt;I just found that I had a big misunderstanding about the problem I am facing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gauss-Seidel and Jaccobi method is for linear system. And the Poisson equation I just used in my Poisson solver is called &quot;nonlinear Poisson&quot; due to the exponential parts at RHS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the numerical instability do cause by the exponential parts due to the wrong method I used to solve the nonlinear Poisson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I use Newton method in my Poisson solver and it works well, thank you everyone :)&lt;/p&gt;&#xA;" OwnerUserId="11614" LastActivityDate="2014-10-29T08:16:03.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16021" PostTypeId="1" AcceptedAnswerId="16025" CreationDate="2014-10-29T09:40:21.633" Score="8" ViewCount="9661" Body="&lt;p&gt;I am trying to develop a library for finite element methods in C++ and for that I am looking at the data structures for meshes. Based on what I've read up on fenics and deal.ii, the general understanding seems to be that a basic mesh class should have the following members: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Cells, Faces, vertices. &lt;/li&gt;&#xA;&lt;li&gt;Connectivity relationships (What are the neighbors of a given cell, common faces between cells, vertices of a given cell, boundary indicators for cells, faces and vertices)&lt;/li&gt;&#xA;&lt;li&gt;Cell iterators&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These are my questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is the best way to store cells of a mesh. I think an STL vector would suffice. Is this the popular option? I have also read that linked lists have been used for cells but I cannot see the advantage of linked lists over vectors in this case. &lt;/li&gt;&#xA;&lt;li&gt;Again, for the connectivity relationships, STL maps seem to be the best method for me. Any comments on this?&lt;/li&gt;&#xA;&lt;li&gt;Does the same data structure work for both triangular and quadrilateral meshes. I think it should but I am not sure if I am missing anything basic here?&lt;/li&gt;&#xA;&lt;li&gt;Are there any more connectivity relationships to be kept in mind?&lt;/li&gt;&#xA;&lt;li&gt;Are there any good references for programming a finite element library? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thank you. &lt;/p&gt;&#xA;" OwnerUserId="11644" LastActivityDate="2014-10-29T14:29:02.713" Title="Programming Finite Element Methods in C++" Tags="|finite-element|c++|software|data-structures|mesh|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16022" PostTypeId="1" CreationDate="2014-10-29T12:41:34.043" Score="0" ViewCount="319" Body="&lt;p&gt;If we optimize some parameter using 4 optimization algorithms, 2 of which are population based (say A and B) and 2 trajectory methods (single point search)(say C and D); what statistical test can be used for comparing 4 of these. The initial population of A and B is the same, so is for C and D. However, population size of A and B is $n$ and the population size of C and D is 1. So, I think, we should use paired t-test for comparing (A and B) and (C and D). And should use unpaired t-test for all other combinations with Bonferroni correction. But what should actually be done?&lt;/p&gt;&#xA;" OwnerUserId="10379" LastEditorUserId="1804" LastEditDate="2014-10-29T17:11:29.653" LastActivityDate="2014-11-13T08:52:45.397" Title="Statistical analysis of optimization algorithms" Tags="|optimization|statistics|heuristics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16023" PostTypeId="2" ParentId="16021" CreationDate="2014-10-29T13:41:01.150" Score="8" Body="&lt;p&gt;In deal.II, we basically only use vectors. Maps are too slow and scatter data all around memory, so we typically don't use them if the keys are integers and within a given range. For example, for the connectivity between cells, you can do arrays (STL vectors) in which you store neighbor indices and so that neighbor indices $4i\ldots 4i+3$ correspond to cell $i$ (in 2d, because there are 4 neighbors per cell). This way you can do direct array lookup, rather than walking through the tree in STL maps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably realize this already, but we have put a lot of research into the data structures we use in this library, and the scheme we use has led to exceptionally low numbers of cache misses. You will spend a lot of time coming up with the same kinds of solutions we use if you want to be efficient.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-29T13:41:01.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16024" PostTypeId="2" ParentId="16013" CreationDate="2014-10-29T13:47:13.227" Score="1" Body="&lt;p&gt;Others have already suggested solutions. My recommendation is to take a look at the book of Nocedal and Wright, &quot;Numerical Optimization&quot;. The book has a whole section on &quot;Hessian modification&quot; for exactly the sort of problem you are observing. It is really very readable and lays out the problem and its solution in easy steps.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-29T13:47:13.227" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16025" PostTypeId="2" ParentId="16021" CreationDate="2014-10-29T14:29:02.713" Score="9" Body="&lt;p&gt;One of the authors of fenics, A. Logg,  have written a very good paper on datastructures of storing meshes. The paper is &#xA;A. Logg (2009). Efficient Representation of Computational Meshes&#xA;&lt;a href=&quot;http://arxiv.org/abs/1205.3081&quot;&gt;http://arxiv.org/abs/1205.3081&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact it's always a tradeoff between storing all the topological informations (nodes around nodes, faces around nodes, etc...) OR having to recompute them before using them. If you store them then you will probably end up with a very massive amount of data to store, if you recompute them using iterators will take some (too much) time. Depending upon the algorithm (typically for FEM, with linear elements you just want nodes of polyhedrons to assemble your local stiffness matrices), you may want to use some specific traversals and then you can generate what you need on the fly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, the proposed solution in fenics is based on many arrays of integer handles and is probably what's closest to the most efficient method (both in terms of memory and access). &lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2014-10-29T14:29:02.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16027" PostTypeId="1" CreationDate="2014-10-29T20:14:32.010" Score="4" ViewCount="86" Body="&lt;p&gt;Consider the polynomial&#xA;$$ p(x) = -514-462 x+359 x^2+1129 x^3+165 x^4+490 x^5-418 x^6+497 x^7-227 x^8+60 x^9-10 x^{10}, $$&#xA;whose root $A\approx 3.14$ is very close to $\pi$: &#xA;$$|A-\pi|=2.0746\times 10^{-33}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I am given as input the eleven integer coefficients above, and nothing else.&#xA;Is there a way to do calculate $\sin A$ that &lt;em&gt;doesn't&lt;/em&gt; use multiple-precision floating-point arithmetic, just the regular double-precision arithmetic? (With multiple-precision arithmetic this is trivial, so I would like to know if there are any methods or ideas that could apply here with this restriction in place.)&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-10-29T20:50:25.317" LastActivityDate="2014-10-29T22:26:58.907" Title="Evaluate sine of a polynomial root close to $\pi$" Tags="|numerical-analysis|roots|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16028" PostTypeId="2" ParentId="16027" CreationDate="2014-10-29T20:41:21.363" Score="1" Body="&lt;p&gt;It probably depends on how much accuracy you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Call $\pi - A = d$. If you know $d$, then you could use $\sin(\pi - x) = \sin(x)$, and evaluate $\sin(d)$. It's very close to zero, and a first-order Taylor series yields $\sin(d) \approx d$. (Python's &lt;code&gt;math.sin(2e-33)&lt;/code&gt; yields  2 \times 10^{-33} to at least 10 significant digits.) Additional terms in the Taylor series will be negligible, but arbitrary precision arithmetic will be more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate $d$, you can replace $x$ in your polynomial with $\pi - d$ and solve for it using a standard root finder; I used Newton's method and got approximately -2.075e-33. Of course, this method makes use of the knowledge that the root of interest of the original polynomial is close to $\pi$, but it's a standard type of trick in numerical analysis. &lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-10-29T22:26:58.907" LastActivityDate="2014-10-29T22:26:58.907" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16030" PostTypeId="1" AcceptedAnswerId="16031" CreationDate="2014-10-30T07:41:37.210" Score="4" ViewCount="289" Body="&lt;p&gt;I'm working on my thesis and a part of it has to do with adaptive mesh refinement. As a computer science major, I'm not too familiar with this field. The best way I can put my knowledge of AMR is: I understand the purpose and programming part, but I don't understand what it is. Sorry, I'm having difficulty wording it correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found numerous papers with titles similar to &quot;_____ Through Adaptive Mesh Refinement&quot; or &quot;An Adaptive Mesh Refinement Method for _____&quot; and it seems like AMR isn't a field with set algorithms. Instead, there are some templates that are extended based on the functions that describe whatever is being studied. As a followup conclusion, the functions that describe the behavior of the mesh are closely tied to the functions of whatever is being studied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7545/what-are-the-basic-principles-behind-generating-a-moving-mesh&quot;&gt;this&lt;/a&gt; question which led me to moving mesh studies. Is it safe to assume the main difference between AMR and MM is that AMR adds and removes points to alter the fineness of the mesh, while MM moves the existing points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, basically, my questions are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is AMR open-ended in terms of algorithms? &lt;/li&gt;&#xA;&lt;li&gt;Are the calculations on the mesh done with a separate set of functions than the numerical ones that are being studied? &lt;/li&gt;&#xA;&lt;li&gt;Are there any other difference between AMR and MM that I haven't listed?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5415" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2014-11-02T04:27:51.480" Title="Adaptive mesh refinement algorithms and the difference between AMR and moving mesh" Tags="|pde|discretization|adaptive-mesh-refinement|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16031" PostTypeId="2" ParentId="16030" CreationDate="2014-10-30T08:20:39.507" Score="6" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Every major class of discretization is &quot;open-ended&quot; in the sense that there are decisions with no obviously/provably correct answer in the general case, so some decisions are made based on how they perform for the target problem.  Additionally, each major class has active research on new extensions.  AMR has more choices than static-grid discretizations, so if anything, there is more variation between implementations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm not sure what this is asking.  Usually error indicators (AMR) or &quot;monitor functions&quot; (moving mesh, aka. &quot;$r$-adaptivity&quot;) are defined in order to drive refinement.  In case of moving mesh methods, usually there is an auxiliary equation that is solved to find the position of the mesh.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;AMR typically doesn't come with an a priori estimate of the maximum memory consumption.  When implemented in parallel, it needs repartitioning to maintain load balance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-10-30T08:20:39.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16032" PostTypeId="1" CreationDate="2014-10-30T08:46:33.910" Score="1" ViewCount="115" Body="&lt;p&gt;I read the whole list of this question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/46/where-can-one-obtain-good-data-sets-test-problems-for-testing-algorithms-routine&quot;&gt;Where can one obtain good data sets/test problems for testing algorithms/routines?&lt;/a&gt; But the answers are in different areas and I want to ask a specific area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for resting problems for &lt;strong&gt;wave equation&lt;/strong&gt;, &lt;strong&gt;telegraph equation&lt;/strong&gt; or &lt;strong&gt;Klein-Gordon equation&lt;/strong&gt;, etc. What they have in common is that they are &lt;code&gt;second-order linear hyperbolic partial differential equations&lt;/code&gt;. By testing problem, I mean those can show how well my numerical algorithms behaved, which is exactly the meaning in the similar question I listed above.&lt;/p&gt;&#xA;" OwnerUserId="4562" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-10-30T15:21:21.320" Title="What are the good testing problems for hyperbolic equation?" Tags="|pde|numerical-analysis|hyperbolic-pde|testing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16033" PostTypeId="1" CreationDate="2014-10-30T10:23:54.087" Score="5" ViewCount="213" Body="&lt;p&gt;I want to maximise the score of the following table, choosing one item from each column/row, so no two items are on the same row or column. Score to maximise is just adding all the choices together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$*\quad A\quad B\quad C\quad D\quad E$$&#xA;$$\alpha\quad 16\quad 16\quad 18\quad 18\quad 18$$&#xA;$$\beta\quad 20\quad 18\quad 16\quad 12\quad 10$$&#xA;$$\gamma\quad 20\quad 18\quad 18\quad 16\quad 16$$&#xA;$$\delta\quad 18\quad 18\quad 16\quad 16\quad 8$$&#xA;$$\epsilon\quad 10\quad 12\quad 14\quad 14\quad 14$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: ($^{C}$ means chosen)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$*\quad A\quad B\quad C\quad D\quad E$$&#xA;$$\alpha\quad 16^{C}\quad 16\quad 18\quad 18\quad 18$$&#xA;$$\beta\quad 20\quad 18\quad 16\quad 12\quad 10^{C}$$&#xA;$$\gamma\quad 20\quad 18^{C}\quad 18\quad 16\quad 16$$&#xA;$$\delta\quad 18\quad 18\quad 16^{C}\quad 16\quad 8$$&#xA;$$\epsilon\quad 10\quad 12\quad 14\quad 14^{C}\quad 14$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gives a score of $16+10+18+16+14=74$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now there are a few ways to do this, but can firstly, someone actually tell me if $88$ really is the best result and how can this be done computational. Note: I am a Mathematics student and would normally approach such a problem with graph theory, but would like to see computation methods.&lt;/p&gt;&#xA;" OwnerUserId="11651" LastActivityDate="2018-05-31T06:04:38.717" Title="Optimal solution to a table of numbers" Tags="|optimization|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16034" PostTypeId="1" CreationDate="2014-10-30T11:02:29.637" Score="5" ViewCount="89" Body="&lt;p&gt;can any one recommend numerical solver (c/c++ library preferred) for stochastic optimal control problems? For deterministic optimal control I found something like that: &lt;a href=&quot;http://abs-5.me.washington.edu/ocp/index.html&quot; rel=&quot;noreferrer&quot;&gt;http://abs-5.me.washington.edu/ocp/index.html&lt;/a&gt; Now looking for something similar for stochastic OCP.&lt;/p&gt;&#xA;" OwnerUserId="11654" LastActivityDate="2014-10-30T11:02:29.637" Title="numerical solver for stochastic optimal control problems" Tags="|constrained-optimization|optimal-control|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16035" PostTypeId="1" CreationDate="2014-10-30T11:22:53.380" Score="5" ViewCount="2237" Body="&lt;p&gt;Bathe's &lt;em&gt;Finite Element Procedures&lt;/em&gt; shows the &quot;nonlinear strain stiffness matrix&quot; for a 2D truss element as&#xA;$$&#xA;\frac {^tP} {L_0 + \Delta L}&#xA;\left[ \begin{array}{ccc}&#xA;1 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; -1 \\&#xA;-1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 \\&#xA;0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 1 \\&#xA; \end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But other sources, such as this &lt;a href=&quot;http://people.duke.edu/~hpgavin/cee421/truss-finite-def.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://people.duke.edu/~hpgavin/cee421/truss-finite-def.pdf&lt;/a&gt; and the ANSYS theory manual, omit rows 1 and 3 (axial direction):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac {^tP} {L_0}&#xA;\left[ \begin{array}{ccc}&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; -1 \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 1 \\&#xA; \end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The extra 1's don't seem to be physically correct according to my intuition. They say the element becomes stiffer when it's under tension. That suggests the force/displacement relationship is not linear in a seemingly arbitrary way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bathe does say &quot;Note that if the material stress-strain relationship is such that $^tP$ is constant with changes in $\Delta L$, [then something that seems to lead to the extra 1's being omitted]&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps it's using a generalization of Hooke's law which is nonlinear but doesn't have any extra parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My experiments with nonlinear solid elements in Calculix show the same behavior as these truss elements with the extra 1's present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Using the full diagonal matrix with pre-stressed frequency analysis leads to what seems like an incorrect result - a 1D spring-mass system's natural frequency increases with tension on the spring. In other words, this common physics concept of gravity not influencing the motion described here is violated. &lt;a href=&quot;https://share.ehs.uen.org/sites/default/files/Unit02Lesson2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://share.ehs.uen.org/sites/default/files/Unit02Lesson2.pdf&lt;/a&gt; The same problem occurs with solid elements in Calculix. Are they really wrong? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2:&lt;/strong&gt; Example showing that the full-diagonal matrix does not work for pre-stressed frequency analysis. It's a 1D spring with length 1, spring constant k, mass m concentrated at each node and no constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/eKu9u.png&quot; alt=&quot;spring with 2 masses&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The natural frequency should be independent of P. We can see this is true when using the 2nd geometric stiffness matrix above by solving the eigenvalue equation&#xA;$$&#xA;(\textbf K_e + \textbf K_g - \omega^2 \textbf M) \textbf u = 0&#xA;$$&#xA;I form the 1D matrix by taking on the 1st and 3rd rows and columns from the 2D matrix above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\textbf K_e = &#xA;\left[ \begin{array}{rr}&#xA;k &amp;amp; -k \\&#xA;-k &amp;amp; k \\&#xA; \end{array} \right]&#xA;$&#xA;$&#xA;\textbf K_g = &#xA;\left[ \begin{array}{rr}&#xA;0 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 \\&#xA; \end{array} \right]&#xA;$&#xA;$&#xA;\textbf M = &#xA;\left[ \begin{array}{rr}&#xA;m &amp;amp; 0 \\&#xA;0 &amp;amp; m \\&#xA; \end{array} \right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which has the non-zero eigenvalue $ \omega^2=\frac{2k}{m} $. This is consistent with a simple spring-mass oscillator. The factor of 2 appears because it's really two 1-dof oscillators with half the spring each.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if we use the first geometric stiffness matrix above, with $ \Delta L $ approximately 0, we get the wrong answer:&#xA;$$&#xA;(\textbf K_e + \textbf K_g - \omega^2 \textbf M) \textbf u = 0&#xA;$$&#xA;$&#xA;\textbf K_e = &#xA;\left[ \begin{array}{rr}&#xA;k &amp;amp; -k \\&#xA;-k &amp;amp; k \\&#xA; \end{array} \right]&#xA;$&#xA;$&#xA;\textbf K_g = &#xA;\left[ \begin{array}{rr}&#xA;P &amp;amp; -P \\&#xA;-P &amp;amp; P \\&#xA; \end{array} \right]&#xA;$&#xA;$&#xA;\textbf M = &#xA;\left[ \begin{array}{rr}&#xA;m &amp;amp; 0 \\&#xA;0 &amp;amp; m \\&#xA; \end{array} \right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has a non-zero eigenvalue of $ \omega^2=\frac{2(k+P)}{m} $. It's apparently wrong because it's a function of P but it should be independent of P. It shows that a spring is stiffened by increased tension. Conversely, if P is negative, it also shows that compression softens it and it even becomes unstable when P=-k. This indicates axial buckling (not Euler column buckling) under compression, which seems to be unphysical. Even if $\Delta L$ is not zero, it can't make $\textbf K_g$ zero, so there will always be some discrepancy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm wondering if the matrix with the full diagonal above is really correct. At least it appears to be wrong when used for frequency or linear buckling analysis. Solid elements in Calculix follow this &quot;incorrect&quot; behavior for frequency analysis, which adds to the confusion.&lt;/p&gt;&#xA;" OwnerUserId="11650" LastEditorUserId="11650" LastEditDate="2014-11-01T04:01:36.557" LastActivityDate="2014-11-02T22:19:01.163" Title="When should a geometric stiffness matrix for truss elements include axial terms?" Tags="|finite-element|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16036" PostTypeId="2" ParentId="16033" CreationDate="2014-10-30T11:39:35.573" Score="0" Body="&lt;p&gt;I agree 88 looks like the best score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might use a tree. The root node contains 5 subnodes (16 16 18 18 18), each of them contains 4 subnodes (excludes the column of the parent node), etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Traverse the tree recursively and search for the highest score (breadth first or depth first). I wouldn't actually build the tree, just pretend it's there and look up the cell values when you need to compare them. The recursive function would be passed a row index and current score then call itself for each of the 1-5 subnodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has O(N!) complexity so it doesn't scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It sounds tricky to find the highest score without an exhaustive search like this. If you only want a reasonably good guess, then you could ignore the branches with the lowest scores at each level.&lt;/p&gt;&#xA;" OwnerUserId="11650" LastEditorUserId="11650" LastEditDate="2014-10-30T11:47:35.597" LastActivityDate="2014-10-30T11:47:35.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16037" PostTypeId="2" ParentId="16033" CreationDate="2014-10-30T14:03:54.067" Score="7" Body="&lt;p&gt;This is called an &lt;a href=&quot;https://en.wikipedia.org/wiki/Assignment_problem&quot; rel=&quot;noreferrer&quot;&gt;assignment problem&lt;/a&gt;. The most common algorithm to solve it is known as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hungarian_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Hungarian algorithm&lt;/a&gt;, and runs in $O(n^3)$ time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting observation is that it corresponds to computing the &lt;a href=&quot;https://en.wikipedia.org/wiki/Permanent&quot; rel=&quot;noreferrer&quot;&gt;permanent&lt;/a&gt; of a matrix in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Tropical_geometry&quot; rel=&quot;noreferrer&quot;&gt;tropical&lt;/a&gt; (max-plus) semiring. Unfortunately, though, a Google search for &quot;tropical permanent&quot; is complicated by the non-mathematical meanings of these words.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-10-30T14:03:54.067" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16038" PostTypeId="1" AcceptedAnswerId="16075" CreationDate="2014-10-30T14:27:57.397" Score="0" ViewCount="192" Body="&lt;p&gt;I have come across (state of the art) softwares on MAX-SAT PBO or WBO: examples are WBO, open-WBO, SAT4J, scip. They all seem to work in a way that only command line instruction is involved. I want to construct a C++ programme that involves solving some PBO system (equivalently MAX-SAT or WBO) then do a huge bunch of other things. Just like how cplex or Gurobi. What is the state of art solvers for this process? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="7891" LastActivityDate="2014-11-03T02:51:45.363" Title="MAX-SAT pseudo-boolean optimization(PBO) weighted boolean optimization(WBO) solver" Tags="|software|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16039" PostTypeId="2" ParentId="16032" CreationDate="2014-10-30T15:21:21.320" Score="4" Body="&lt;p&gt;The first two can be decomposed into a system of first-order advection equations of which there are a number of standard test problems often involving both smooth and discontinuous initial conditions and all of which can be solved analytically.  For instance &lt;a href=&quot;http://www.clawpack.org/doc/pyclaw/gallery/gallery_all.html#dimensional-acoustics&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; solution is to the linearized acoustics equations.  The choice between one or the other is mainly based on what you want to do, a spectral methods based code for instance would not behave well for discontinuous initial conditions while codes that tend to be able to capture discontinuous solutions do not tend to do smooth solutions as well, hence using both initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Klein-Gordon equation &lt;a href=&quot;http://eqworld.ipmnet.ru/en/solutions/lpde/lpde203.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; link will give you a way to construct analytical solutions and test them.  I am not certain about a particular solution that would test a code to its limits though.&lt;/p&gt;&#xA;" OwnerUserId="567" LastActivityDate="2014-10-30T15:21:21.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16040" PostTypeId="2" ParentId="10356" CreationDate="2014-10-30T18:15:42.830" Score="2" Body="&lt;p&gt;You can optimize the convolution, using separable theorem. Let me give an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image ($M\times N$): 1024, 768 (Grayscale)&#xA;Convolution mask ($k\times k$): 7x7&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computational complexity: Convolution -&gt; $O(MNkk)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computational complexity: Separable convolution -&gt; $O(2*MNk)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;being k = kernel size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using normal convolution you got O(1024*768*7*7) = 38535168 operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using separable convolution you got(2*1024*768*7) = 11010048 operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;a href=&quot;https://github.com/DiegoCatalano/Catalano-Framework/issues/3&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; you got a real example.&lt;/p&gt;&#xA;" OwnerUserId="11660" LastEditorUserId="-1" LastEditDate="2016-08-01T14:23:46.477" LastActivityDate="2016-08-01T14:23:46.477" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16041" PostTypeId="1" CreationDate="2014-10-30T18:22:57.613" Score="1" ViewCount="218" Body="&lt;p&gt;I'm new with FEniCS and Python and I'm stuck with this issue:&#xA;is there a way to write a 4th order tensor in an easy way to implement?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to compute the following stiffnes tensor:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{ijkl}= \lambda \delta_{ij}\delta_{kl} + \mu (\delta_{ik}\delta_{jl} + \delta_{il}\delta_{jk})$&lt;/p&gt;&#xA;" OwnerUserId="11661" LastActivityDate="2014-10-30T18:22:57.613" Title="4th order tensor" Tags="|pde|python|fenics|tensor|" AnswerCount="0" CommentCount="1" ClosedDate="2014-10-31T22:54:46.370" ContentLicense="CC BY-SA 3.0" />
  <row Id="16042" PostTypeId="2" ParentId="16035" CreationDate="2014-10-30T19:09:22.250" Score="6" Body="&lt;p&gt;If your objective is to perform a geometrically nonlinear&#xA;analysis of truss structures where the elements are allowed&#xA;to undergo arbitrarily large rotations, then your first form&#xA;of the geometric stiffness matrix with the &quot;extra&quot; ones is the correct one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see this, it is useful to step back to the internal force vector for this&#xA;truss element. There are several ways to derive this vector but a simple way&#xA;is to just write the equilibrium equations for the member in its deformed&#xA;position. That gives &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F = {^tP} \{-cos \theta, -sin \theta, cos \theta, sin \theta\}^T$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\theta$ is the angle of rotation between the undeformed and deformed&#xA;truss element.&#xA;This equation is on page 547 of Bathe. In a nonlinear analysis, the need for&#xA;a &quot;stiffness matrix&quot; is the result of solving the nonlinear equations by&#xA;the Newton-Raphson method; the derivative of the residual vector with respect to &#xA;to the&#xA;unknowns is needed. Simple differentiation of the internal force vector&#xA;above with respect to the nodal displacements yields a tangent-K that includes the &#xA;first form of the $k_g$ matrix.&#xA;That is what Bathe does on page 547. The nonlinear mechanics is all contained&#xA;in the internal force vector and the tangent-K is just a straightforward &#xA;differentiation. The first form of $k_g$ is correct in that sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the notes you refer to by Gavin, he describes the approximations he&#xA;makes in deriving the simpler matrix. These approximations yield a $k_g$&#xA;matrix that is fine for most engineering linear&#xA;buckling and pre-stressed vibration analysis but not large-rotation nonlinear analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to see a step-by-step, detailed derivation of the first form,&#xA;take a look at these notes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/NFEM.d/NFEM.Ch08.d/NFEM.Ch08.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.colorado.edu/engineering/CAS/courses.d/NFEM.d/NFEM.Ch08.d/NFEM.Ch08.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This note by Felippa has an interesting comment on exactly the question posed in this post so I'm including part of that here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Most of the early work, as well as the confusion alluded to by Martin, pertains to what is now called the&#xA;geometric stiffness matrix. The early name for it was initial stress matrix. A bar geometric stiffness was&#xA;first presented in [767]. If this is compared to (8.26) one may observe that half of the nonzero entries are&#xA;missing. Consequently that inaugural KG is not invariant with respect to the choice of coordinate frames,&#xA;and has the wrong rank.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Felippa's equation 8.26 is the first version of $k_g$ shown above and the version presented in reference 767 is the second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both the Bathe and Felippa derivations, it is important to note the&#xA;distinction between Cauchy and second Piola-Kirchoff stress tensors.&#xA;That is the reason for the $\Delta L$ in the denominator of the first&#xA;$k_g$ matrix. That distinction is&#xA;ignored in the classical (i.e. Gavin) derivation. Specifically, the stress-strain law is assumed to be $S_{11}=E \epsilon_{11}$&#xA;where $S_{11}$ is the axial component of the second Piola-Kirchoff stress&#xA;tensor and $\epsilon_{11}$ is the axial component of the Green-Lagrange strain&#xA;tensor. This stress-strain law agrees with the engineering stress and strain&#xA;version of Hooke's law only for infinitesimal strains and this difference affects&#xA;both the material and geometric element matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few comments on your example problems. First, the classical&#xA;linear spring is a very simplistic model of an elastic solid so it is&#xA;hardly surprising that a more sophisticated continuum mechanics model&#xA;gives different results. A similar comment can be made about the&#xA;&quot;classical&quot; $k_g$ matrix derivation. It is based on an analysis of&#xA;thin, pre-stressed members where &lt;strong&gt;only&lt;/strong&gt; transverse deflections are considered;&#xA;axial deflections are explicitly ignored. So it is hardly surprising that&#xA;the two $k_g$ matrices give different results in problems where &lt;strong&gt;&lt;em&gt;only&lt;/em&gt;&lt;/strong&gt;&#xA;axial effects are considered.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2014-11-02T22:19:01.163" LastActivityDate="2014-11-02T22:19:01.163" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="16043" PostTypeId="1" AcceptedAnswerId="16061" CreationDate="2014-10-31T04:34:09.373" Score="6" ViewCount="598" Body="&lt;p&gt;I am working with the simple example of an oscillator:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(1) \; \; \ddot{u} + u = 0, \; \; u(0) = u_0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that Forward Euler does not preserve an invariant of the above system: $$(2) \; \; \dot{u}^2 + u^2 = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can get the eigenvalues of a matrix that approximately solves for the solution of $(1)$ using Forward Euler if I were to write it as a first order system of equations. However, I don't understand what I can deduce about the ability of Forward Euler to preserve $(2)$, given these eigenvalues and the stability properties of Forward Euler?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to be able to understand this in general, for any scheme, not necessarily FE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for instance, I know that the Trapezoidal Rule scheme, unlike Forward Euler, is good at preserving $(2)$, but its difficult for me to see what is qualitatively different about the eigenvalues of the matrix for the problem based on the Trapezoidal Rule...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Trapezoidal Rule is L-stable (that is, $Re(G(k\lambda)) &amp;lt; 0$ for $\dot{u} = \lambda u$ with $Re(\lambda) &amp;lt; 0$), but so is Backwards Euler, and it does not preserve $(2)$. What's the pattern here?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="123" LastEditDate="2014-11-02T07:20:46.143" LastActivityDate="2014-11-02T08:51:26.393" Title="How does the L-stability or A-stability of a scheme relate to its ability to preserve a quadratic invariant?" Tags="|numerical-analysis|ode|time-integration|conservation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16044" PostTypeId="2" ParentId="16043" CreationDate="2014-10-31T05:40:43.150" Score="5" Body="&lt;p&gt;Stability and preservation of invariants are unrelated. Hairer does a nice job of proving how various methods preserve different kinds of invariants in his book &lt;a href=&quot;http://www.dmae.upct.es/~amat/simplecticos2.pdf&quot;&gt;Geometric Integration&lt;/a&gt;. Chapter III is the most relevant to your question, where he demonstrates, for example, that RK methods and linear multistep methods preserve linear first invariants. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-31T05:40:43.150" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="16045" PostTypeId="2" ParentId="16030" CreationDate="2014-10-31T13:46:18.723" Score="4" Body="&lt;p&gt;Since you are a computer science major, let me posit the following analogy: &quot;adaptive mesh refinement&quot; is a set of techniques for solving partial differential equations in mathematics; this is in the same spirit as &quot;image processing&quot; is a set of techniques to transform and improve images. Both fields have many different aspects, so there are no fixed algorithms and no memory estimates for the whole field, but there are of course for individual methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're still confused, I would suggest you find Graham Carey's book &quot;Computational Grids&quot; as a reference.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-10-31T13:46:18.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16046" PostTypeId="2" ParentId="10356" CreationDate="2014-10-31T13:49:29.160" Score="1" Body="&lt;p&gt;For some 2D convolution operations (e.g. mean filters) an &lt;a href=&quot;http://computersciencesource.wordpress.com/2010/09/03/computer-vision-the-integral-image/&quot; rel=&quot;nofollow&quot;&gt;integral image&lt;/a&gt; (a.k.a. &lt;a href=&quot;http://en.wikipedia.org/wiki/Summed_area_table&quot; rel=&quot;nofollow&quot;&gt;summed area table&lt;/a&gt;) can be used to speed up the calculation considerably. In particular, applying the filter on the integral image rather than on the original image can allow for convolution using very large kernel sizes since the performance becomes independent of the kernel size, i.e. it takes as many calculations to perform a 100 x 100 convolution as a 3 x 3 convolution. It's simple to implement and can be extremely effective.&lt;/p&gt;&#xA;" OwnerUserId="11636" LastActivityDate="2014-10-31T13:49:29.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16049" PostTypeId="1" CreationDate="2014-10-31T17:07:42.860" Score="2" ViewCount="127" Body="&lt;p&gt;I have been working on a multigrid solution to a non-homogeneous Dirichlet boundary value problem.  However, the function goes to infinity on the boundary.  This causes numerical overflow errors to be raised in calculating the discretized derivatives.  Does anyone have a suggestion on how to overcome this problem?  I am not a mathemetician, but the Galerkin Method seems to be a way to transform the problem, but I am not sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the pde's I'm solving (I've not included the formulae for the drift and diffusion coefficients as they are also long):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{2} \omega_i^2 \sigma_{\omega_i}^2 \frac{\partial^2 f^i}{\partial \omega_i^2} + \frac{1}{2} \delta^2 \sigma_\delta^2 \frac{\partial^2 f^i}{\partial \delta^2} + \delta \omega_i \sigma_{\omega_i} \sigma_\delta \frac{\partial^2 f^i}{\partial \delta \partial \omega_i} + \delta \mu_\delta \frac{\partial f^i}{\partial \delta}$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$+ \omega_i \mu_{\omega_i} \frac{\partial f^i}{\omega_i}  - \rho f^i + (\omega^i)^{-\gamma_i} \left ( tan \left [ \pi(\delta - \frac{1}{2}) \right ] \right )^{1-\gamma_i}= 0$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{2} \omega_i^2 \sigma_{\omega_i}^2 \frac{\partial^2 h^i}{\partial \omega_i^2} + \frac{1}{2} \delta^2 \sigma_\delta^2 \frac{\partial^2 h^i}{\partial \delta^2} + \delta \omega_i \sigma_{\omega_i} \sigma_\delta \frac{\partial^2 h^i}{\partial \delta \partial \omega_i} + \delta \mu_\delta \frac{\partial h^i}{\partial \delta}$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$+ \omega_i \mu_{\omega_i} \frac{\partial h^i}{\omega_i}  - \rho h^i + \omega^i(t)^{-\gamma_i} \left ( tan \left [ \pi(\delta(t) - \frac{1}{2}) \right ] \right )^{-\gamma_i}  = 0$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;br&gt;&#xA;$H_0(t) = exp \left ( -\int_0^t r(u) du \right) exp \left (- \int_0^t \frac{\sigma_D}{\xi(u)} du - \frac{1}{2} \int_0^t \left ( \frac{\sigma_D}{\xi(u)} \right)^2 du \right )$  &lt;/p&gt;&#xA;" OwnerUserId="11672" LastEditorUserId="11672" LastEditDate="2014-11-01T14:43:52.070" LastActivityDate="2014-11-04T04:35:38.103" Title="Infinite Function Value on Dirichlet Boundary" Tags="|pde|finite-difference|numerical-analysis|multigrid|elliptic-pde|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="16050" PostTypeId="2" ParentId="16022" CreationDate="2014-10-31T18:34:26.760" Score="2" Body="&lt;p&gt;Established methodologies for benchmarking optimization software can be found in publications such as &lt;a href=&quot;http://arxiv.org/pdf/cs/0102001.pdf&quot; rel=&quot;nofollow&quot;&gt;Benchmarking Optimization Software with Performance Profiles&lt;/a&gt;, &lt;a href=&quot;http://www.mcs.anl.gov/papers/P1471A.pdf&quot; rel=&quot;nofollow&quot;&gt;Benchmarking Derivative-Free Optimization Algorithms&lt;/a&gt;, and &lt;a href=&quot;http://rd.springer.com/article/10.1007/s10898-012-9951-y&quot; rel=&quot;nofollow&quot;&gt;Derivative-free optimization: a review of algorithms and comparison of software implementations&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, algorithms are benchmarked against a suite of problems, and then performance profiles are constructed based on whether or not the algorithm successfully solved each problem to within a given tolerance and time limit. $t$-tests are not typically used.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-10-31T18:34:26.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16051" PostTypeId="1" AcceptedAnswerId="25986" CreationDate="2014-10-31T19:18:01.007" Score="0" ViewCount="497" Body="&lt;p&gt;I'm trying to find some finite temperature properties using ab-initio simulations; and for this, I'm looking for a tool to be used as an auxiliary to DFT simulations (in VASP) to implement Quasiharmonic approximation (QHA). I'm facing some problems using Phonopy, so, could anyone tell me what's a popular package for QHA?&lt;/p&gt;&#xA;" OwnerUserId="11674" LastEditorUserId="11674" LastEditDate="2014-10-31T20:28:59.027" LastActivityDate="2017-01-22T15:13:56.973" Title="Code/package for Quasiharmonic approximation" Tags="|density-functional-theory|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16052" PostTypeId="1" CreationDate="2014-10-31T22:47:00.783" Score="9" ViewCount="226" Body="&lt;p&gt;Most methods for oscillatory integrals I know about deal with integrals of the form&#xA;$$ \int f(x)e^{i\omega x}\,dx $$&#xA;where $\omega$ is large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have an integral of the form&#xA;$$ \int f(x)g_1(x)\cdots g_n(x)\,dx, $$&#xA;where $g_k$ are oscillatory functions whose roots are only known approximately, but some kind of asymptotic form&#xA;$$ g_k(x) \sim e^{i\omega_k x} $$&#xA;is known, with the frequencies $\omega_k$ all different (and $\mathbb{Q}$-linearly independent), then how can I evaluate this integral?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike in the case of $e^{i\omega x}$, the polynomial integrals $\int x^a \prod g_k(x)$ are not known, so I can't construct a set of polynomial interpolants for $f(x)$ and integrate the interpolants exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my exact problem, $g_k$'s are Bessel functions $J_0(\omega_k x)$, and $f(x)=x^\alpha$, and the region of integration is $[0,\infty)$. The method I am using now is to sum up integral contributions over intervals $[x_{k-1},x_k]$ between roots up to some cutoff $M$, then use the asymptotic expansion for $g_k(x)$ for large $x$. This algorithm's time complexity is exponential in $n$ because it involves expanding the product $g_1\ldots g_n$, each of which has a number $r$ of asymptotic terms, giving $r^n$ total terms; pruning terms that are too small doesn't reduce the run time enough to make this feasible for large $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Heuristic non-rigorous answers, suggestions and references are all welcome.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-10-31T22:57:45.467" LastActivityDate="2016-02-23T10:58:39.450" Title="Evaluating oscillatory integrals with many independent periods and no closed forms" Tags="|quadrature|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16053" PostTypeId="1" CreationDate="2014-11-01T00:33:06.413" Score="1" ViewCount="150" Body="&lt;p&gt;What is the best MAX-SAT solver problems for Ising spin glass? I tried Scip-Max-sat and open-wbo. While open-wbo cannot solve the instance with only 27 variable Scip-max-Sat fail to solve the one with 200+ variables (while it can solve model with 100 variables in 30 secs).... I am not sure whether I have missed something or is it the state of art solution space for ising spin glass is 100+ variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the test case I use:&#xA;&lt;a href=&quot;https://www.dropbox.com/sh/qg9u20w040e7hpn/AACg428cWgNfqF6pQG36URCxa?dl=0&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/sh/qg9u20w040e7hpn/AACg428cWgNfqF6pQG36URCxa?dl=0&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="11580" LastActivityDate="2014-11-01T11:47:01.750" Title="state of art MAX-SAT solver for ising spin glass" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16054" PostTypeId="2" ParentId="16051" CreationDate="2014-11-01T00:55:18.097" Score="0" Body="&lt;p&gt;Quantum-Espresso has some QHA tools I believe. And many of the equations are pretty simple, if you are up to coding some of them yourself. See this useful &lt;a href=&quot;http://arxiv.org/abs/1112.4977&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5196" LastActivityDate="2014-11-01T00:55:18.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16055" PostTypeId="2" ParentId="16053" CreationDate="2014-11-01T11:47:01.750" Score="1" Body="&lt;p&gt;I don't know if you're really interested in MAX-SAT, or in finding the ground state of an Ising spin glass (I suppose the two are equivalent if the spin coupling values are $\pm 1$, for example). In any case, if you're interested in algorithms that can find approximate solutions efficiently, you should take a look at the papers by Helmut Katzgraber at Texas A&amp;amp;M University.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-01T11:47:01.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16056" PostTypeId="1" CreationDate="2014-11-01T15:27:55.667" Score="-5" ViewCount="242" Body="&lt;p&gt;recently, I try to write a Matlab codes to implement a sparse approximation inverse factorization method proposed by M. Benzi in his paper &lt;a href=&quot;http://www.mathcs.emory.edu/~benzi/Web_papers/ainv.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathcs.emory.edu/~benzi/Web_papers/ainv.pdf&lt;/a&gt;&#xA;this codes to generate the incomplete factorization as &#xA;$A^{-1} = ZD^{-1}W$ or $A^{-1} = \hat{Z}W$.&#xA;I had written the following codes:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/CDZwp.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bYz5H.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SMXlV.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;then this codes can run with any bugs. However, this codes seems to be very expensive, when I use it to handle a 2961*2961 test matrix, it need one night to product the matrices $W$ and $Z$. So I think that the codes should be optimized to improve the efficiency. So I want to ask someone to give me some suggestions to improve this codes. I also note there are many &quot;for-loop&quot; in my codes, maybe these &quot;for-loop&quot; can be replaced by means of some strategies. Up to now, I have no ideas to improve this codes, so I am turning to everyone. &lt;/p&gt;&#xA;" OwnerUserId="10290" LastActivityDate="2014-11-01T17:47:33.007" Title="Can anyone give me some suggestions about optimize my Matlab codes?" Tags="|matlab|matrix|matrix-factorization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16057" PostTypeId="1" CreationDate="2014-11-01T15:32:49.333" Score="3" ViewCount="2943" Body="&lt;p&gt;I have a vector containing acceleration measurements and the corresponding vector of times in which measurements are taken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain velocity and displacement I used the cumtrapz() function already implemented in MATLAB. This should be fine, as I read here &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11733/numerical-integration-of-non-uniform-acceleration-samples&quot;&gt;Numerical integration of non-uniform acceleration samples&lt;/a&gt; and here &lt;a href=&quot;https://stackoverflow.com/questions/9881430/numerical-integration-using-simpsons-rule-on-discrete-data&quot;&gt;https://stackoverflow.com/questions/9881430/numerical-integration-using-simpsons-rule-on-discrete-data&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My doubts are related to the disturbances introduced by the integration algorithm as I found in a book a chapter in which the author says that the trapezium rule introduces low-frequency disturbances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to remove these disturbances and unwanted 0-frequency components (the acceleration was not 0-mean therefore I had a linear trend in the velocity) applying a 2nd order, highpass, butterworht filter with cut-off frequency at 5 Hz (in both directions to null the phase shift) to the velocity and the displacement but I don't know if this is enough or too much because I need very accurate values for velocity and displacement since I have to use them to plot a surface in the state-space (EDIT: phase-space).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I choose the correct cut-off frequency in order not to loose any important information knowing that the experimental structure is excited in a range of frequencies from 5 to 100 Hz?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;Since I have to integrate noisy data, is it advisable to use an higher order method if my goal is accuracy?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2018-09-06T10:15:43.730" Title="Numerical integration and filtering of acceleration experimental data" Tags="|matlab|time-integration|numerics|" AnswerCount="0" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="16058" PostTypeId="2" ParentId="11272" CreationDate="2014-11-01T15:55:05.610" Score="1" Body="&lt;p&gt;As @NickAlger alludes, the incremental delaunay approach can scale exponentially with the dimension of the space, even if the final tesselation has few facets.  Even if some computable solutions exist for special cases, it's unlikely that any practical algorithms exist for general tesselations, which seems to be what you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2014-11-01T15:55:05.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16059" PostTypeId="2" ParentId="16056" CreationDate="2014-11-01T17:47:33.007" Score="2" Body="&lt;p&gt;It is not slow because you've coded the algorithm poorly, or because Matlab is slow at executing it. The algorithm itself is too slow. I'm not quite sure what it does, but I can explain why it's so slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one part of your function you have the following code structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:n&#xA;  for j=i+1:n&#xA;    for k=1:i&#xA;      &amp;lt;body&amp;gt;&#xA;    end&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This triple nested loop executes its body&#xA;$$ \frac16n(n^2-1) $$&#xA;times, so the time complexity of the algorithm is $O(n^3)$. &#xA;For a matrix of size $n=3000$, this is $\approx 5\times 10^{9}$ times that the body of that loop &lt;em&gt;alone&lt;/em&gt; is executed. Thus this algorithm is prohibitively expensive for a matrix of that size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for why this algorithm has complexity $O(n^3)$, I don't know. I don't really see where it uses sparsity, because it accesses each element $w_{k,j}$ in that loop, although $w$ starts as &lt;code&gt;speye&lt;/code&gt;, so I suspect it's meant to be sparse. Typically an algorithm for a sparse matrix would try to keep the number of operations at something like $O(n)$, not $O(n^3)$. Does the original paper definitely specify an $O(n^3)$ algorithm, or does it have a lower complexity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you replace the for-loops with some other looping construct, the number of iterations will stay the same, and this is unlikely to help very much, unless there is nothing else that can be done. The right thing to do is to figure out how to bring the time complexity down (to $O(n^2)$ or lower), but that is a question of algorithm design, and is also a completely different issue from optimizing Matlab code. It is also important to actually take advantage of sparsity, and I don't see how this algorithm does that now.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-11-01T17:47:33.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16060" PostTypeId="2" ParentId="16049" CreationDate="2014-11-01T18:24:56.603" Score="1" Body="&lt;p&gt;Most of the standard methods for solving PDE numerically are not suited to handle infinite values, since they originate from physical problems. Infinite values of the solved variables may occur in physical problems solved numerically, for example, if one tries to solve the problem of a hairline crack in a fully elastic medium using the stress-displacement method (infinite stresses develop at the crack tip, which is physically wrong but does not affect the solution far from the tip).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To deal with this situation through the Finite Element method, one option would be to enrich the basis of the FE discretization, for example as done in the XFEM method, &lt;a href=&quot;http://www.xfem.rwth-aachen.de/Background/Introduction/XFEM_Introduction.php&quot; rel=&quot;nofollow&quot;&gt;see this site&lt;/a&gt;, where in the present case the global enrichment function will be an exponent tending to infinity as one approaches your boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should note that it is not exactly clear what a Dirichlet boundary on which the solution is infinite actually means. It seems to be a badly-posed problem. Perhaps the rate at which the solution approaches infinity on that boundary would be a better candidate for this boundary condition? That would be ideal, as it would dictate the exponent of the enrichment function.&lt;/p&gt;&#xA;" OwnerUserId="6775" LastEditorUserId="6775" LastEditDate="2014-11-01T19:07:24.653" LastActivityDate="2014-11-01T19:07:24.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16061" PostTypeId="2" ParentId="16043" CreationDate="2014-11-02T04:49:26.033" Score="6" Body="&lt;p&gt;In a certain sense, @Geoff Oxberry is correct in saying that stability and preservation of quadratic invariants are not directly related.  For instance, there exist explicit methods that will preserve energy for your problem (and they are certainly not $A$-stable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in another sense there is a relation between the two.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Well-posedness&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;First, note that your problem as described is not well posed, and the invariant you give does not follow from the problem description.  I believe you should have instead&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot{u} + u = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(0) = u_0$$&#xA;and&#xA;$$\dot{u}(0) = v_0$$&#xA;where $u_0^2 + v_0^2 = 1$.  Then the problem is well-posed and the invariant condition you have written follows.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Conservation of energy&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Now, if you rewrite this second-order equation as a first-order system&#xA;$$\begin{pmatrix} \dot{u} \\ \dot{v} \end{pmatrix} = &#xA;\begin{pmatrix} 0 &amp;amp; 1 \\ -1 &amp;amp; 0 \end{pmatrix}&#xA;\begin{pmatrix} u \\ v\end{pmatrix},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you find that the eigenvalues are $\pm i$.  If we write the system above as $\dot{w} = A w$, the exact solution is given by&#xA;$$w(t) = \exp(At)w(0)$$&#xA;and the reason you have energy conservation is that $|e^{\lambda t}|=1$ for both eigenvalues of $A$.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Discrete conservation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;After discretizing with a one-step method, you will have instead an equation of the form&#xA;$$w_{n+1} = R(\Delta t A)w_n,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Delta t$ is the numerical time step and $R(z)$ is the stability function of the method (generally a polynomial or rational function).  Since $A$ is normal, to have discrete conservation you need that $$|R(\pm i \Delta t)|=1.$$.  &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;The connection&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;One way to interpret the condition just stated is to say that &lt;strong&gt;the eigenvalues of the ODE system, scaled by the numerical timestep, must lie on the boundary of the absolute stability region of the numerical integrator&lt;/strong&gt;.  That's because the stability region is defined precisely as the set of complex numbers $z$ for which $|R(z)| \le 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, &lt;em&gt;with just the right time step&lt;/em&gt;, even the 4th-order Runge-Kutta method can be used to preserve energy for this problem, since its stability region boundary intersects the imaginary axis.  Of course nobody recommends this, because it only works for one special value of $\Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trapezoidal rule conserves energy for any time step, because its region of absolute stability is exactly the left half-plane; i.e., the entire imaginary axis is on its stability region boundary, so the condition above is fulfilled for any $\Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, it's impossible to conserve energy with forward or backward Euler, since those methods have a stability region whose boundary only touches the imaginary axis at the origin.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-11-02T08:51:26.393" LastActivityDate="2014-11-02T08:51:26.393" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16063" PostTypeId="1" CreationDate="2014-11-02T08:41:02.660" Score="1" ViewCount="8267" Body="&lt;p&gt;I am studying fluid-structure interaction across integrated circuits packaging by using Ansys Fluent (I'm a beginner). Currently I had an issue about the time step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I set time step size of 1 s it takes about more than 100 s for a complete fluid flow. While for time steps of 0.1 s and 0.01 s takes about 50 s and 10 s for a complete flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how does time step size affect fluid flow? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let say for the 10 s case,&#xA;$$(\text{time step size})\times(\text{no. of time step})$$&#xA;$$1.0 \text{ s} \times 10 = 10 \text{ s}$$&#xA;$$0.1 \text{ s} \times 100 = 10 \text{ s}$$&#xA;$$0.01 \text{ s} \times 1000 = 10 \text{ s}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose, at specific time (let say at $t=10\text{ s}$) for whatever time step size I'm using, I'll be getting same (or similar) position of flow. Am I wrong? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case details:&#xA;- VOF multiphase &#xA;- Implicit solver&#xA;- Transient&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="11679" LastEditorUserId="9667" LastEditDate="2014-11-03T16:58:04.050" LastActivityDate="2020-06-25T21:37:20.167" Title="Fluent time step size (transient implicit solver)" Tags="|fluid-dynamics|implicit-methods|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16064" PostTypeId="2" ParentId="1449" CreationDate="2014-11-02T09:57:10.283" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://www.allinea.com/products/map&quot; rel=&quot;nofollow&quot;&gt;Allinea MAP&lt;/a&gt; is a commercially developed and supported sampling profiler and hence - like HPC Toolkit suggested in an earlier answer - can run on production sized jobs if you wish. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This kind of tool points to CPU bottlenecks or poor MPI communication, but also the whole oversight of profiling the entire job can be priceless in finding surprise issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are often low-hanging performance fruits to be had that are outside of the core kernel of a CFD code, in areas that were not expected. Randomized stack sampling is - whether done manually with GDB, or with tools like HPC Toolkit and Allinea MAP - the best way to find them. If something is important to performance it will show up.&lt;/p&gt;&#xA;" OwnerUserId="5451" LastActivityDate="2014-11-02T09:57:10.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16065" PostTypeId="2" ParentId="83" CreationDate="2014-11-02T12:06:24.350" Score="3" Body="&lt;p&gt;How about calling Matlab from Python, using &#xA;&lt;a href=&quot;https://github.com/arokem/python-matlab-bridge&quot; rel=&quot;nofollow noreferrer&quot;&gt;python-matlab-bridge&lt;/a&gt;&#xA;or the like ?&lt;br&gt;&#xA;That looks much easier than porting yards of code, not to mention test cases and doc.&lt;br&gt;&#xA;And general:&#xA;&lt;em&gt;any&lt;/em&gt; Python $\to$ &lt;em&gt;any&lt;/em&gt; Matlab, or most $\to$ most.&lt;br&gt;&#xA;(&lt;a href=&quot;https://stackoverflow.com/questions/9845292/a-tool-to-convert-matlab-code-to-python&quot;&gt;A-tool-to-convert-matlab-code-to-python&lt;/a&gt;&#xA;on SO lists source-to-source translators too: difficult.)&lt;br&gt;&#xA;Comments from people who have used such bridges would be welcome.&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-11-02T12:18:56.463" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16066" PostTypeId="2" ParentId="83" CreationDate="2014-11-02T12:31:26.950" Score="1" Body="&lt;p&gt;It is good to mention here that Google Ceres solver is actually a very powerful non-linear optimiser, used in many projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a python wrapper available here: &lt;a href=&quot;https://github.com/rll/cyres&quot; rel=&quot;nofollow&quot;&gt;https://github.com/rll/cyres&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-11-02T12:31:26.950" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16067" PostTypeId="2" ParentId="14117" CreationDate="2014-11-02T13:59:44.383" Score="1" Body="&lt;p&gt;I guess you already tried the matlab &quot;fit&quot; function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = fit( [x, y], z, 'poly23' )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;More infos on this function here: &lt;a href=&quot;http://www.mathworks.it/help/curvefit/command-line-curve-and-surface-fitting.html&quot; rel=&quot;nofollow&quot;&gt;matlab fit&lt;/a&gt; . &lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="11676" LastEditDate="2014-11-02T15:33:19.667" LastActivityDate="2014-11-02T15:33:19.667" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16068" PostTypeId="1" AcceptedAnswerId="16072" CreationDate="2014-11-02T15:14:16.557" Score="3" ViewCount="4906" Body="&lt;p&gt;I tried to plot a surface in MATLAB but, since it is the first time I had to do something like this, I need a confirmation on the process I followed because it is important for my project to plot the right surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 3 vectors obtained from measurement data: $x$ (displacement), $v$ (velocity) and $p$ (force). The goal is to plot a surface where $x$ and $v$ represent a point in the plane and $p$ is the &quot;height&quot; above that plane. The idea is to have a quite smooth surface, because then I have to fit this surface created from measurement data using a mathematical model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I have done:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;first I create the grid: (since my measurements vectors have more than 400000 samples, do I have to use more than 1000 for &lt;code&gt;x_plot&lt;/code&gt; and &lt;code&gt;y_plot&lt;/code&gt; to have a good plot? I have also a problem when I increase too much the length of &lt;code&gt;x_plot&lt;/code&gt; and &lt;code&gt;y_plot&lt;/code&gt;, infact I get an &lt;code&gt;&quot;out of memory&quot;&lt;/code&gt; error message)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_plot = linspace(min(x),max(x),1000);&#xA;y_plot = linspace(min(v),max(v),1000);&#xA;[XI,YI] = ndgrid(x_plot,y_plot);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I create the scattered data interpolant:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F = TriScatteredInterp(x,v,p,'natural');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And now I evaluate the interpolant on the grid:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ZI = F(XI,YI);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally I plot the surface:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;figure()&#xA;mesh(XI,YI,ZI)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this the right method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it better to use ndgrid or meshgrid to create the grid?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="10134" LastEditDate="2014-11-02T19:28:03.293" LastActivityDate="2016-06-22T22:16:13.230" Title="Plot a surface from data sets in MATLAB" Tags="|matlab|interpolation|visualization|plotting|3d|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16069" PostTypeId="2" ParentId="7498" CreationDate="2014-11-02T15:32:26.750" Score="0" Body="&lt;p&gt;Another way could be: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Call your 3 data vectors aa,bb,cc and assume that aa and bb represent a point on the plane and cc is the height above that plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a grid:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_plot = linspace(min(aa),max(aa),100);&#xA;y_plot = linspace(min(bb),max(bb),100);&#xA;[XI,YI] = meshgrid(x_plot,y_plot);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Create the interpolant:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ZI = griddata(aa,bb,cc,XI,YI);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And plot&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;figure()&#xA;mesh(XI,YI,ZI)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11676" LastActivityDate="2014-11-02T15:32:26.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16070" PostTypeId="1" CreationDate="2014-11-02T15:43:20.530" Score="1" ViewCount="98" Body="&lt;p&gt;today, I encountered a confused problem by accident, but I have no ideas to deal with it fully. The question can be described as follows: for example, when we need to use FDM/FEM to discrete the partial differential operator with variable coefficients, &#xA;$\mathcal{L} = d_1(x,y)\frac{\partial^2 u(x,y)}{\partial x^2} + d_2(x,y)\frac{\partial^2 u(x,y)}{\partial y^2} + v(x,y)u(x,y)$, where the coefficients $d_1(x,y),d_2(x,y),v(x,y)\geq 0$. For simplicity, we use the following notations to explain the discretization by FDM:&#xA;\begin{equation}&#xA;d_1(x,y),\ d_2(x,y),\ v(x,y) \rightharpoonup D_1, D_2, V\ \ (\mathrm{three\ diagonal\ matrices,\ whose\ entries\ are\ all\ nonnegative})\\&#xA;\frac{\partial^2 u(x,y)}{\partial x^2}\rightharpoonup L_1 (\mathrm{discretized\ matrix})\\&#xA;\frac{\partial^2 u(x,y)}{\partial y^2}\rightharpoonup L_2 (\mathrm{discretized\ matrix})\\&#xA;\end{equation}&#xA;so we can obtain the discretized matrix of $\mathcal{L}$ as follows,&#xA;$A:= D_1L_1 + D_2L_2 + VI$ ($I$ is the identity matrix). Maybe, we want to construct a preconditioner as $P:=d_1L_1 + d_2L_2+\hat{v}I$, where $d_1,d_2,\hat{v}$ are all the mean value of diagonal entries (for example, if $D_1 = diag(s_1,s_2,\ldots,s_n)$, then $d_1 = \frac{s_1 + s_2+\ldots + s_n}{n}$). in fact, this preconditioner can be regard as the discretized matrix of $\mathcal{L} = d_1\frac{\partial^2 u(x,y)}{\partial x^2} + d_2\frac{\partial^2 u(x,y)}{\partial y^2} + \hat{v}u(x,y)$, so I want to know that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Is this idea reasonable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) if so, how to analyze the spectrum of preconditioned matrix $P^{-1}A$, for the sake of simplicity, take $v(x,y) = 0,$, I can write&#xA;\begin{equation}&#xA;(d_1L_1 + d_2L_2)^{-1}(D_1L_1 + D_2 L_2) = I + (d_1L_1 + d_2L_2)^{-1}[(D_1 - d_1I)L_1 + (D_2-d_2 I) L_2],&#xA;\end{equation} &#xA;it means that there are some eigenvalue closed to 1 and this can translate a rapid convergence of Krylov subspace methods? However, this analysis seems to be very rough. So are there some more detailed spectral analysis of preconditioned matrix ? Maybe, some one can provide some references to explain this idea ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Maybe, some other boundary condition can be also considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have do a small test problem: the spectral distributions of original and preconditioned matrix, please see the following figures:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/T9tmo.jpg&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/OpZeN.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="10290" LastEditorUserId="10290" LastEditDate="2014-11-02T18:01:18.553" LastActivityDate="2014-11-02T18:01:18.553" Title="Eigenvalue analysis of preconditioned partial differential operator" Tags="|pde|eigenvalues|preconditioning|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16071" PostTypeId="1" CreationDate="2014-11-02T21:02:11.363" Score="2" ViewCount="158" Body="&lt;p&gt;I have an equation:&#xA;  $\ddot{x}+(\delta+\epsilon\cos{t})x=0$ known as the Mathieu equation.The $\delta-\epsilon$ parameter space of this equation looks something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rIlRC.png&quot; alt=&quot;transition curves&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red lines in this diagram indicate that if a point on the red lines is chosen, and the corresponding value of $(\delta,\epsilon)$ is plugged into the Mathieu equation, then it is guaranteed that one would get a periodic solution. I have managed to generate these curves by something known as the arc-length continuation method, thus I have the exact data to plot these curves with me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I have an analytic relation approximating these red lines which looks something like:&#xA;$\delta=\frac{1}{4}+f_1(k)\epsilon+f_2(k)\epsilon^2+f_3(k)\epsilon^3+...$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the functions $f_1(k),f_2(k)...$ are known and are highly nonlinear. This equation is for just one of these red lines. This way I have equations for all the red lines. Also, it is only possible to approximate these curves till an upper bound on $\epsilon$, as I have only one degree of freedom (the parameter k) in analytic $\delta-\epsilon$ relation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First let me choose one particular red line and the corresponding $\delta-\epsilon$ relation for it.Now, what I want to do is two things:&#xA;1) I want to find the optimum $\epsilon$ by which I can fit $\delta-\epsilon$ curve.&#xA;2) I want to find the optimum $k$ for that optimum $\epsilon$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find both the above things by some numerical algorithms which converge. This is necessary as my functions $f_1(k),f_2(k)...$ are highly non-linear in nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will tell you where I have reached so far. I have no idea how to even approach point 1). For point 2), I have managed to implement the mean-square errors approach, by which I tried minimizing the error, but that leaves me to solve a polynomial with powers of k like $k^{3500}$. This polynomial is going to have a lot of local minimas and I have no clue of any numerical algorithms which help me in finding a global minima. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if I manage to implement 1) and 2), I want to do some analysis (like R-squared coefficient) to see how well I have fit my curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me out if you know of any numerical procedures that can achieve the above.&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9756" LastActivityDate="2014-11-03T02:30:09.800" Title="fitting a non-linear curve" Tags="|optimization|numerical-analysis|nonlinear-programming|regression|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16072" PostTypeId="2" ParentId="16068" CreationDate="2014-11-02T21:36:13.503" Score="3" Body="&lt;p&gt;You may be better served with a slightly different method of surface plotting, especially if you're having memory issues. What you're doing right now is actually two separate things:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Interpolating your data onto a grid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Plotting the surface corresponding to the interpolated data on the grid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since you have some raw data values which presumably do not line up with your grid this adds the additional errors associated with the interpolation. It also gives the appearance of having uniform data whether this is true or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the approach I would take:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Generate a triangulation of your data points using &lt;code&gt;delaunay&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use &lt;code&gt;trisurf&lt;/code&gt; (or &lt;code&gt;trimesh&lt;/code&gt; if you prefer) to plot the data using the triangulation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This has the advantage of directly plotting your data without requiring interpolation to a grid. It will also minimize the amount of memory used and eliminate the need to set a grid resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a minimal example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all&#xA;&#xA;% Number of points:&#xA;N = 10000;&#xA;&#xA;% Generate some points:&#xA;x = 15*(rand(N,1)-.5);&#xA;y = 15*(rand(N,1)-.5);&#xA;z = cos(sqrt(x.^2+y.^2)).*exp(-0.025*(x.^2+y.^2));&#xA;&#xA;% Delaunay triangulation:&#xA;tri = delaunay(x,y);&#xA;&#xA;% Plot:&#xA;figure(1)&#xA;clf&#xA;trisurf(tri,x,y,z,'linestyle','none')&#xA;shading interp&#xA;lighting p&#xA;camlight&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you prefer could use &lt;code&gt;patch&lt;/code&gt; instead of &lt;code&gt;trisurf&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;figure(1)&#xA;clf&#xA;p.Faces=tri;&#xA;p.Vertices=[x, y, z];&#xA;p.FaceVertexCData=z;&#xA;patch(p)&#xA;view(3)&#xA;shading interp&#xA;lighting p&#xA;camlight&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pKthx.png&quot; alt=&quot;trisurf or patch plot of triangular surface&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this will result in a piecewise planar surface composed of a set of triangles. Obviously the more points you have, the smoother the surface will look. If you want to smooth this surface I think that's really a separate question.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-11-02T21:36:13.503" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16073" PostTypeId="2" ParentId="16071" CreationDate="2014-11-03T02:30:09.800" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;First let me choose one particular red line and the corresponding $\delta$−$\epsilon$ relation for it.Now, what I want to do is two things: 1) I want to find the optimum $\epsilon$ by which I can fit $\delta$−$\epsilon$ curve. 2) I want to find the optimum $k$ for that optimum $\epsilon$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Regarding point 1): you have to define what you mean by &quot;an optimal $\epsilon$&quot;. Usually, this definition suggests an objective function (and potentially a feasible set for that objective function) you can then use as a starting point for a problem formulation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For point 2), I have managed to implement the mean-square errors approach, by which I tried minimizing the error, but that leaves me to solve a polynomial with powers of $k$ like $k^{3500}$. This polynomial is going to have a lot of local minimas [sic] and I have no clue of any numerical algorithms which help me in finding a global minima [sic].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are two major approaches to global optimization: deterministic, and nondeterministic. Assuming your feasible set is compact (highly likely), a properly formulated least-squares approach will yield a twice continuously differentiable objective function, so your formulation will be amenable to deterministic methods. The advantage of these methods is that they return a provably good solution, in that the optimal objective function value can be no more than a certain tolerance (user-specified) less than the returned optimal objective function value. The disadvantage is that these methods have a run time that scales exponentially with problem size, so these methods are likely to take a long time if you are fitting more than a handful of parameters. An example of a solver you can try is BARON; if you can somehow pose your optimization using the GAMS or AMPL modeling languages, you can then submit a job on the NEOS server to solve your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nondeterministic algorithms (e.g., genetic algorithms, particle swarm) do not generate provably good solutions, and merely explore the feasible set of a problem in a random fashion. If you have more than say, 30-300 parameters, you may consider using an algorithm of this type simply to explore your search space in the hopes of calculating a feasible point with a reasonably good objective function value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try to multistart a local optimization algorithm: use the same approach you've been using, but seed the local optimizer with lots of different initial guesses spread out over your feasible set. You may have to content yourself with selecting a provably good local optimum, even if it might be globally suboptimal.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finally, if I manage to implement 1) and 2), I want to do some analysis (like R-squared coefficient) to see how well I have fit my curve.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You're likely to get a better answer re: statistical analysis on CrossValidated, a sister Stack Exchange site. As a non-statistician who has taken basic statistics a few times, I can only provide basic advice: at first glance, I would look at a plot of the residual values for your fit to see if there is any systematic bias in the residual (error) values, to see if you are violating any assumptions about normality in the error, which will give you an idea if the model and fit criteria you've selected are appropriate. If you have multiple candidate models and you are looking to select among them, you might also look at model selection statistics, such as the Akaike Information Criterion.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-03T02:30:09.800" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16074" PostTypeId="2" ParentId="16063" CreationDate="2014-11-03T02:41:53.287" Score="1" Body="&lt;p&gt;You've given scant details for your problem, so I can only quote general trends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Time step size affects numerical error and stability. If you pick a really large step size, your results may be meaningless because the resulting numerical method is unstable. You'll typically see large oscillations in the fluid flow field, and cutting the time step will be necessary to obtain a stable numerical method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that your time step is smaller than the time step governed by the stability limit for your problem and choice of ODE solver, cutting the time step further will generally decrease the numerical error in your solution, so there is a tradeoff between the magnitude of the error you are willing to accept in your calculated solution and the computational effort you will need to expend to calculate a solution such that the error is of that acceptable magnitude (or less).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, at a specific time for whatever time step size you're using, you may or may not get similar flow fields, depending on stability and accuracy of the numerical method for your particular choice of step size and your problem. A good thing to do would be to run a convergence study of your problem for multiple choices of time step and compare the results, using the calculated solution at the smallest choice of time step as the &quot;true solution&quot;. If you see large differences as you approach your smallest time step solution, something is wrong, and you will need to diagnose the error; cutting the time step might help, but there could be other problems.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-03T02:41:53.287" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16075" PostTypeId="2" ParentId="16038" CreationDate="2014-11-03T02:51:45.363" Score="1" Body="&lt;p&gt;I'm not an expert of PBO or WBO (I do mixed-integer programming instead). The best information I could find on solvers was the &lt;a href=&quot;http://www.cril.univ-artois.fr/PB12/&quot; rel=&quot;nofollow&quot;&gt;Pseudo-Boolean Competition 2012&lt;/a&gt; (I could not find a later version), which should give you some idea of the solvers out there. SCIP has an API, and is written in C, so you should be able to the SCIP API for solving problems within source code. If not, it is open source, so at worst, you could write an API yourself and contribute it upstream.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-03T02:51:45.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16076" PostTypeId="1" CreationDate="2014-11-03T05:23:36.013" Score="2" ViewCount="202" Body="&lt;p&gt;The &lt;a href=&quot;http://www.researchgate.net/publication/220411914_A_Fast_Iterative_Method_for_Eikonal_Equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast Iterative Method&lt;/a&gt; is a way of solving the Eikonal Equation on a discrete grid, similar to the Fast Marching Method discussed in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11702/solving-the-quadratic-in-the-fast-marching-method&quot;&gt;another question here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper describes the overall algorithm, as well as providing a method for solving the quadratic, as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;SolveQuadratic(a, b, c, f)&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;em&gt;Returns the value $u = Ux$ that solves $g(U, x) = 0$, where $a ≤ b ≤ c$&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u ← c + 1/f&#xA;if u ≤ b return (u)&#xA;u ← (b + c + sqrt(−b^2 − c^2 + 2bc + 2/f^2))/2&#xA;if u ≤ a return (u)&#xA;u ← (2(a + b + c) + sqrt(4(a + b + c)^2 − 12(a^2 + b^2 + c^2 − 1/f^2)))/6&#xA;return (u)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's easy enough to implement, as is the rest of the FIM algorithm. However, I'm unclear what the inputs $a, b, c$ are in the context of the problem (I do understand how to get $f$, though, I think). They are descried as &quot;values $a, b, c$ of upwind&#xA;neighbors in the cardinal directions of the grid&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only need to solve in a 2-dimensional grid, so I believe I only need $f$ plus two inputs, and would drop the second test-and-recalculate step above, but I still need $b, c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should note that I'm not a strong maths person; I have a BSc in Comp. Sci. but my maths skills are first-year / freshman level at best, and rusty...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="11681" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-04-26T18:34:07.683" Title="Solving the quadratic in the Fast Iterative Method" Tags="|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16077" PostTypeId="1" AcceptedAnswerId="16079" CreationDate="2014-11-03T05:58:03.090" Score="2" ViewCount="153" Body="&lt;p&gt;I'm writing a mod for a game that models orbital physics (Kerbal Space Program, or KSP). I'm attempting to model the effects of thrust on spacecraft in certain states where the game only models them as a Keplerian orbit (The ship is 'on rails'), and I'm having some difficulty with numerical stability - even if I don't have any thrust applied I end up constantly increasing eccentricity for the orbit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 'on rails' state, a spacecraft in the game follows a Keplerian orbit exactly. The orbit is calculated when the spacecraft is put on rails, and then when it goes off rails the game calculates the expected position and velocity at the current time and sets up the spacecraft to do normal Newtonian things from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, all my code does is take the calculated position and velocity of the spacecraft at the current time (using the game's own code) and then call a function already present in the game to calculate a Keplerian orbit for an object with that position and velocity around the currently-dominant body - that is, it should be a no-op. I'm actually seeing the orbit's eccentricity drift rapidly, certainly quickly enough that this won't work for gameplay purposes. This code is being called every physics timestep in the game, which is rather often - something like 50 times a second - so I suspect what I'm seeing is that the code in the game for calculating position and velocity from orbit and vice-versa isn't stable. I've verified that in actual gameplay taking a vessel off rails and then putting it back on rails can cause observable orbital differences, and I've confirmed that the calculated position/velocity of the craft change after I recalculate the orbit. Relative error in position is about $10^{-8}$, relative error in velocity is about $10^{-2}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what the best solution is here, and I only have very limited experience with numerical computation. I'd like to know what a good approach to this problem might look like. As far as I can see there are essentially three classes of solution:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt; Calculate a new orbit directly from the old orbital parameters and the thrust. I don't know how plausible calculating a delta on a Keplerian orbit is, though, and I can't find much online about doing that.  &#xA;&lt;li&gt; Write code with better precision or better numerical stability for doing orbit -&gt; pos/vee -&gt; orbit conversion. I don't know how plausible any stability is when you're doing that fifty times a second, though. I've found some code for calculating that, but I don't know where it sits on the fast/stable/good spectrum.&#xA;&lt;li&gt; Keep my own position/velocity values for the ship, update them continually, recalculate the orbit at intervals just so the player gets some feedback. That would require implementing a general two-body problem solution, which sounds like a lot of work.&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What's a good option here?&lt;/p&gt;&#xA;" OwnerUserId="11683" LastActivityDate="2014-11-03T15:48:44.143" Title="Adjusting Keplerian orbits for thrust with numerical stability" Tags="|algorithms|stability|computational-physics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16078" PostTypeId="1" AcceptedAnswerId="16082" CreationDate="2014-11-03T13:16:43.400" Score="3" ViewCount="1673" Body="&lt;p&gt;Say we have $x_i, \ldots, x_n \in R ^ D$ with positive, real components and use Jaccard distance $d(x_i, x_j) = 1 - \frac{\sum_{d = 1}^D\min(x_i^d, x_j^d)}{\sum_{d = 1}^D\max(x_i^d, x_j^d)}$ to find $k$ nearest neighbors for every point. I wonder, is it possible to get &lt;strong&gt;exact&lt;/strong&gt; solution (all $k$ neighbors are found) without computing all pairwise distances?&lt;/p&gt;&#xA;" OwnerUserId="1517" LastEditorUserId="549" LastEditDate="2014-11-03T20:06:09.750" LastActivityDate="2014-11-12T18:44:23.020" Title="Finding nearest neighbors using Jaccard distance for positive, real-valued vectors" Tags="|computational-geometry|nearest-neighbors|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16079" PostTypeId="2" ParentId="16077" CreationDate="2014-11-03T15:48:44.143" Score="0" Body="&lt;p&gt;Relative error of $10^{-8}$ for what should be an identity operation seems okay (but not great), but $10^{-2}$ is absolutely terrible. With a relative error per time step of $\epsilon$, the relative error after time $t$ at $50$ time steps per second will be&#xA;$$ (1+\epsilon)^{50t} \approx e^{50\epsilon t}, $$&#xA;so the time-scale for accumulation of errors is $0.02/\epsilon$. For $\epsilon=10^{-2}$, this is $2$ seconds. From your drift in eccentricity it sound like that relative error is not only large, but also biased in one direction, and that bias is what would cause the drift. It's not completely clear whether this is specifically numerical stability, or if this is just an inaccurate computation method being used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;0.&lt;/strong&gt;&#xA;Implementing your own functions to compute position-velocty $\Leftrightarrow$ orbit parameters transformations is certainly feasible. A good textbook on orbital dynamics would explain the necessary formulas; there is also &lt;a href=&quot;http://www.braeunig.us/space/&quot; rel=&quot;nofollow&quot;&gt;this helpful site&lt;/a&gt;. A book such as &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; by Higham will explain what numerical stability is and how to design a stable algorithm for these computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difficulty is covering all the corner cases correctly, since the equations are a little more complicated for inclined orbits. If I remember correctly it's all just coordinate transformations, with just one non-linear Kepler equation that is quite easy to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that adding a small amount of acceleration at the beginning of every time step may or may not be accurate enough, this would need to be experimentally checked. This is only first-order accurate (or second-order if the thrust vanishes at end-points of integration), so if it's not enough, then writing your own integrator is the way to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; Implementing your own ODE integrator is quite feasible, and easier than it sounds. The integrator needs to be symplectic, so something like the Stormer-Verlet method is a good start:&#xA;$$ \dot x = v, \qquad \dot v = f(x), $$&#xA;$$ v_{\frac12} = v_0 + \tfrac{h}{2} f(x_0), \qquad x_1 = x_0 + h v_{\frac12}, \qquad v_1 = v_{\frac12} + \tfrac{h}{2} f(x_1). $$&#xA;However, you probably need much more accuracy that just one Stormet-Verlet step per in-game time-step, so it's probably worth looking at a higher-order method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relevant references are &lt;em&gt;Geometric Numerical Integration&lt;/em&gt; by Hairer, Lubich, Wanner, and &lt;em&gt;Solving Ordinary Differential Equations&lt;/em&gt; by Hairer, Norsett, Wanner. According to that, implicit Gauss methods (Butcher-Kuntzmann methods) are symplectic, so for example if you don't mind solving a system of 6 nonlinear equations, then the 6th-order Butcher-Kuntzmann method might be worth trying; it's $\nu=3$ on p.56 of Butcher &lt;em&gt;Implicit Runge-Kutta Processes&lt;/em&gt; (1964).&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-11-03T15:48:44.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16080" PostTypeId="1" CreationDate="2014-11-03T16:10:28.080" Score="9" ViewCount="1903" Body="&lt;p&gt;I've been reading a lot of papers on ADMM lately, and also tried to solve several problems using it, in all of which it was very effective. In contrast to other optimization methods, I can't get a good intuition as to how and why this method is so effective (of course, I've seen convergence analysis for a few cases, but nothing that gave me too much insight). Is there some intuition behind ADMM? How did the first scientists to use it come up with this idea? Some geometrical intuition would be best, but any insight anyone has will help.&lt;/p&gt;&#xA;" OwnerUserId="898" LastEditorUserId="549" LastEditDate="2014-11-04T05:12:33.547" LastActivityDate="2014-11-05T09:49:18.590" Title="Intuition behind Alternating Direction Method of Multipliers" Tags="|optimization|admm|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="16082" PostTypeId="2" ParentId="16078" CreationDate="2014-11-03T17:06:17.913" Score="3" Body="&lt;p&gt;For the Euclidean distance, it's typical to use data structures like a &lt;a href=&quot;http://en.wikipedia.org/wiki/K-d_tree&quot; rel=&quot;nofollow&quot;&gt;k-d tree&lt;/a&gt; for nearest-neighbor and range search problems. With an ordinary binary search tree, you know that all nodes to the left of the root have keys less than that of the root, and likewise all nodes to the right have keys greater than the root. A k-d tree uses a similar idea only alternating dimensions at each level. The wiki article has a more in-depth explanation and some examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nearest-neighbor searches can be done efficiently (e.g. $O(k \log n)$) provided you have more data points than $2^D$. High-dimensional data is hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good implementation of a k-d tree can be found in &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.KDTree.html&quot; rel=&quot;nofollow&quot;&gt;scipy&lt;/a&gt;; there's a version written in C which is pretty efficient. You can do nearest-neighbor queries with arbitrary $p$-norms in scipy, but they have not included the Jaccard distance; you may have to adapt their code accordingly. There are alternatives, like the &lt;a href=&quot;http://en.wikipedia.org/wiki/M-tree&quot; rel=&quot;nofollow&quot;&gt;M-tree&lt;/a&gt;, which are specifically geared toward data in a metric space; someone appears to have an &lt;a href=&quot;https://github.com/erdavila/M-Tree&quot; rel=&quot;nofollow&quot;&gt;implementation&lt;/a&gt; but I haven't tried it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other similar data structures are &lt;a href=&quot;http://en.wikipedia.org/wiki/Quadtree&quot; rel=&quot;nofollow&quot;&gt;quadtrees&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/R-tree&quot; rel=&quot;nofollow&quot;&gt;R-trees&lt;/a&gt;. I find quadtrees the best ratio of ease-of-implementation to efficiency, but your mileage may vary.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-11-03T17:06:17.913" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16083" PostTypeId="2" ParentId="16063" CreationDate="2014-11-03T17:08:46.747" Score="0" Body="&lt;p&gt;What do you mean by &quot;complete flow&quot;? If you're saying that after that amount of physical time, the flow solution in your transient solver becomes steady state then it's because you're not actually resolving the unsteady transients in the flow with your timestep selection. If it has a steady state, altering the time-step of an implicit method like you are is actually more akin to a relaxation parameter on a Newton method than it is actual time-stepping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're only interested in the steady-state solution, then don't use the transient solver, use the steady one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are actually interested in the transient / start-up solution, then do a time-step convergence study with continually decreasing time steps until you reach a point where the flow is the same for any smaller timestep.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-11-03T17:08:46.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16084" PostTypeId="2" ParentId="14734" CreationDate="2014-11-03T17:29:08.687" Score="5" Body="&lt;p&gt;I suggest using PETSc, which has parallel solvers like GMRES, bi Conjugate Gradient, etc. Multiple preconditioners are also available. Furthermore, there might also be the option of the SNES nonlinear solvers. As to which set of solver+preconditioner combination works best, it's likely problem-specific. PETSc will allow you to switch between them easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also wish to consider using FEniCS+PETSc.  With FEniCS, you specify the weak form of the equations and it will figure out how to assemble the matrix for you.  FEniCS interfaces with the PETSc solvers to do the calculation.  In fact, a simple Google search for &quot;petsc euler equations&quot; yields the following link: &lt;a href=&quot;http://www.csc.kth.se/~jjan/transfer/reportMHAF.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.csc.kth.se/~jjan/transfer/reportMHAF.pdf&lt;/a&gt;.  In the appendix, you can see the  FEniCS source code to do the linearised Euler equations (as well as the incompressible Navier Stokes equations).&lt;/p&gt;&#xA;" OwnerUserId="11689" LastEditorUserId="-1" LastEditDate="2014-11-04T01:52:18.893" LastActivityDate="2014-11-04T01:52:18.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16085" PostTypeId="1" AcceptedAnswerId="16086" CreationDate="2014-11-03T19:14:09.647" Score="4" ViewCount="124" Body="&lt;p&gt;Are there any instances of scientific numerical problems where the choice of rounding mode matters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are usually a number of different rounding modes available: to $0$, away from $0$, to $\pm\infty$, nearest ties to even, nearest ties down. Apart from various currency-manipulation problems where rounding is mandated by problem details, does anyone know an example of where choice of rounding mode might actually matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A numerically stable algorithm would be insensitive to round-off errors and hence to choice of rounding mode, but are there any other issues that might be relevant?&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-11-03T19:26:43.647" LastActivityDate="2014-11-04T05:55:24.483" Title="Difference between rounding modes in computational science?" Tags="|numerical-analysis|floating-point|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16086" PostTypeId="2" ParentId="16085" CreationDate="2014-11-03T19:48:25.197" Score="4" Body="&lt;p&gt;Using the rounding modes, it's possible to implement &lt;a href=&quot;http://en.wikipedia.org/wiki/Interval_arithmetic&quot; rel=&quot;nofollow&quot;&gt;interval arithmetic&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that two positive quantities $x$ and $y$ are represented by the intervals $[x_l,x_u]$ and $[y_l,y_u]$. The product is represented in interval arithmetic by $[x_l \otimes y_l, x_u \otimes y_u]$, where $x_l\otimes y_l$ should be rounded down and $x_u\otimes y_u$ should be rounded up. Control over rounding modes makes it easy to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Varying the rounding mode and rerunning your code is also a quick test of your algorithm- if the answers change a lot then you know that the algorithm is quite sensitive to round-off errors (and that you probably need to rewrite it!)&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="713" LastEditDate="2014-11-04T03:06:48.557" LastActivityDate="2014-11-04T03:06:48.557" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16088" PostTypeId="1" AcceptedAnswerId="19150" CreationDate="2014-11-04T03:07:06.800" Score="2" ViewCount="261" Body="&lt;p&gt;I'm wondering if it's possible to create functional intensity plots in matlab. Examples of what I'm looking for are below found in &lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs10618-014-0371-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gg89B.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2aS28.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8VsuZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These figures were (guessing) probably made in R, but is this possible for matlab? I can't seem to find anything related to intensity besides an &lt;em&gt;image&lt;/em&gt; intensity plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;THanks&lt;/p&gt;&#xA;" OwnerUserId="9564" LastActivityDate="2015-03-12T19:41:01.663" Title="single-variate function variational Intensity Plot for Matlab" Tags="|matlab|plotting|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16089" PostTypeId="2" ParentId="2173" CreationDate="2014-11-04T03:12:58.020" Score="6" Body="&lt;p&gt;Optimising your code has to be done carefully.  Let's also assume you have already debugged the code already.  You can save a lot of time if you follow certain priorities, namely:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use highly optimised (or professionally optimised) libraries where possible.  Some examples might include FFTW, OpenBlas, Intel MKL, NAG libraries, etc. Unless you are highly talented (like the developer of GotoBLAS), you probably can't beat the professionals.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a profiler (quite a few in the following list have already been named in this thread--Intel Tune, valgrind, gprof, gcov, etc.) to find out what portions of your code take the most time.  No point wasting time optimising portions of code that are rarely called.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;From the profiler results, look at the portion of your code that took the most time.  Determine what the nature of your algorithm is--is it CPU bound or memory bound?  Each requires a different set of optimisation techniques.  If you are getting a lot of cache misses, memory might be the bottleneck--the CPU is wasting clock cycles waiting for memory to become available. Think about whether the loop fits into the L1/L2/L3 cache of your system. If you have &quot;if&quot; statements in your loop, check if the profiler says anything about branch misprediction?  What is the branch misprediction penalty of your system? By the way, you can get branch misprediction data from the Intel Optimisation Reference Manuals [1].  Note that the branch misprediction penalty is processor-specific, as you'll see in the Intel manual. The manual will also tell you things like how many clock cycles various instructions take (again, this is processor-specific).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lastly, address the problems identified by the profiler. A number of techniques have already been discussed here. A number of good, reliable, comprehensive resources on optimization are also available. To name just two, there is the Intel Optimization Reference Manual [1], and the five optimization manuals by Agner Fog [2]. Note that there are some things that you may not need to do, if the compiler does it already--for example, loop unrolling, aligning memory, etc. Read your compiler documentation carefully.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Intel 64 and IA-32 Architectures Optimization Reference Manual: &lt;a href=&quot;http://www.intel.sg/content/dam/doc/manual/64-ia-32-architectures-optimization-manual.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.intel.sg/content/dam/doc/manual/64-ia-32-architectures-optimization-manual.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Agner Fog, &quot;Software Optimisation Resources&quot;: &lt;a href=&quot;http://www.agner.org/optimize/&quot; rel=&quot;noreferrer&quot;&gt;http://www.agner.org/optimize/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;Optimizing software in C++: An optimization guide for Windows, Linux and Mac platforms&quot; &lt;/li&gt;&#xA;&lt;li&gt;&quot;Optimizing subroutines in assembly language: An optimization guide for x86 platforms&quot;&lt;/li&gt;&#xA;&lt;li&gt;&quot;The microarchitecture of Intel, AMD and VIA CPUs: An optimization guide for assembly programmers and compiler makers&quot;&lt;/li&gt;&#xA;&lt;li&gt;&quot;Instruction tables: Lists of instruction latencies, throughputs and micro-operation breakdowns for Intel, AMD and VIA CPUs&quot;&lt;/li&gt;&#xA;&lt;li&gt;&quot;Calling conventions for different C++ compilers and operating systems&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11689" LastEditorUserId="11689" LastEditDate="2014-11-04T03:24:01.260" LastActivityDate="2014-11-04T03:24:01.260" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16090" PostTypeId="2" ParentId="16085" CreationDate="2014-11-04T03:47:25.950" Score="0" Body="&lt;p&gt;The choice of rounding mode is governed by IEEE standards--Ref [1] mentions &lt;a href=&quot;http://en.wikipedia.org/wiki/IEEE_754-1985&quot; rel=&quot;nofollow&quot;&gt;IEEE 754&lt;/a&gt; and IEEE 854. Without the standards, one could find different results when porting code from one machine to another (see Goldberg [1]). Portability would be important if you want to be check if the different answer you get on another machine is due to a bug versus due to differences in implementation.  The article also gives examples such as $x^2-y^2$ being more accurate than $(x+y)(x-y)$ when $x&amp;gt;&amp;gt;y$ or $y&amp;lt;&amp;lt;x$.  In such cases, rounding error may become more important. As for compiler support, an example is the &lt;a href=&quot;http://en.wikipedia.org/wiki/C99#IEEE.C2.A0754_floating_point_support&quot; rel=&quot;nofollow&quot;&gt;C99 standard which supports IEEE floating point arithmetic&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: some situations call for something more rigorous than rounding modes--these are problems involving the use of &quot;arbitrary precision arithmetic. In the Wikipedia article on the topic, a number of applications are given, as well as appropriate libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] David Goldberg, &quot;What every computer scientist should know about floating-point arithmetic&quot;, ACM Computing Surveys (CSUR), 23(1), 5-48, 1991. See &lt;a href=&quot;http://dl.acm.org/ft_gateway.cfm?id=103163&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/ft_gateway.cfm?id=103163&amp;amp;type=pdf&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11689" LastEditorUserId="11689" LastEditDate="2014-11-04T05:55:24.483" LastActivityDate="2014-11-04T05:55:24.483" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16091" PostTypeId="2" ParentId="16049" CreationDate="2014-11-04T04:35:38.103" Score="1" Body="&lt;p&gt;I don't know off the top of my head how to deal with the situation you describe, but let me outline the general approach one would take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's pretend for a minute that you were solving the following, simpler problem:&#xA;$$&#xA; -\Delta u = f \qquad\qquad \text{in $\Omega$}&#xA;  \\&#xA;  \qquad u = 0 \qquad\qquad \text{on $\partial\Omega$}&#xA;$$&#xA;and assume that $f$ has a singularity at a point $x_0$, i.e., it goes to infinity at this point. Now imagine that you also consider the following, perturbed problem&#xA;$$&#xA; -\Delta \tilde u = \tilde f \qquad\qquad \text{in $\Omega$}&#xA;  \\&#xA;  \qquad \tilde u = 0 \qquad\qquad \text{on $\partial\Omega$}&#xA;$$&#xA;where $\tilde f$ is a perturbed version of $f$. Then&#xA;$$&#xA;  \| u -\tilde u \|_{H^1} \le \| f-\tilde f\|_{H^{-1}}&#xA;$$&#xA;base on elementary a priori stability analysis. So if you discretize the second version using finite elements of finite differences (using for simplicity a lowest order method), you get an approximate solution $\tilde u_h$ that satisfies the estimate&#xA;$$&#xA;\| u -\tilde u_h \|_{H^1} \le&#xA;\| u -\tilde u \|_{H^1} + \| \tilde u - \tilde u_h \|_{H^1}&#xA; \le&#xA; \| f-\tilde f\|_{H^{-1}} +&#xA; C h \| \tilde u \|_{H^2}.&#xA;$$&#xA;Here, $h$ is the me size.&#xA;In other words, if you solve a version of the equation perturbed in such a way that &#xA;$$&#xA;  \| f-\tilde f\|_{H^{-1}} \le Ch&#xA;$$&#xA;then you get a method that has the correct convergence order. It's too late in the night right now to prove this with any certainty, but my best guess is that if you choose $\tilde f=f \ast G_h$ where $G_h$ is a Gaussian with width $h$, then you get at least this order. Note that this smoothed version of $f$ is always finite and so can serve as the basis for a discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you'll have to do is to go through a similar argument for boundary values instead of right hand side values. It shouldn't be too hard to do this. In the end, what you'll have to do is just convince yourself that if you smooth the offending function a bit, that you still converge to the correct solution and at the same convergence order as always.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-04T04:35:38.103" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16092" PostTypeId="1" CreationDate="2014-11-04T10:33:09.407" Score="1" ViewCount="214" Body="&lt;p&gt;I want to write a code that generates a function I(t) that satisfies the following condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\big&amp;lt;I(t)^2\big&amp;gt;}{\big&amp;lt;I(t)\big&amp;gt;^2} &amp;gt; 2$ &lt;br&gt;&#xA;&lt;br&gt;&#xA;In other words,&#xA;&lt;br&gt;&#xA;&lt;br&gt;&#xA;$\frac{\lim_{T \to \infty}\int_0^T I(t)^2dt}{(\lim_{T \to \infty}\int_0^T I(t)dt)^2} &amp;gt; 2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it even possible to do that? If not, please tell me what am I missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.- The function should have a non-Gaussian distribution.&lt;/p&gt;&#xA;" OwnerUserId="11693" LastEditorUserId="11693" LastEditDate="2014-11-27T02:48:06.580" LastActivityDate="2014-11-27T02:48:06.580" Title="How to write a function-generating code in Python/MATLAB?" Tags="|matlab|python|time-integration|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16093" PostTypeId="1" AcceptedAnswerId="16095" CreationDate="2014-11-04T12:11:27.917" Score="7" ViewCount="185" Body="&lt;p&gt;The simplest way to add 8 numbers would be something like this,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sum = one + two + three + four + five + six + seven + eight;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This (in C) would add &lt;code&gt;one&lt;/code&gt; and &lt;code&gt;two&lt;/code&gt;, and add the result to &lt;code&gt;three&lt;/code&gt; and do the same till &lt;code&gt;eight&lt;/code&gt;. Instructions for adding &lt;code&gt;three&lt;/code&gt;,&lt;code&gt;four&lt;/code&gt; and so on till &lt;code&gt;eight&lt;/code&gt; will have wait the first instruction to complete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we rewrite the same addition as,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sum = (   (  (one+two) + (three+four) )  + (  (five+six) + (seven+eight)  )   );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the variables enclosed inside the brackets should be added before the total addition can take place. This presents an obvious chance for the compiler to optimise the program by pipelining the instructions for &lt;code&gt;one+two&lt;/code&gt;,&lt;code&gt;three+four&lt;/code&gt;,&lt;code&gt;five+six&lt;/code&gt; and &lt;code&gt;seven+eight&lt;/code&gt;, since they don't have any data dependence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can compilers like GCC identify such a situation, or are there any practical difficulties? If yes, which would be faster?&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="713" LastEditDate="2014-11-04T17:08:45.310" LastActivityDate="2014-11-04T19:28:39.240" Title="Performance of adding eight numbers sequentially vs. in a tree" Tags="|optimization|performance|c|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16094" PostTypeId="1" AcceptedAnswerId="16096" CreationDate="2014-11-04T13:17:08.447" Score="5" ViewCount="581" Body="&lt;p&gt;I have the following problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial w}{\partial t} = \frac{1}{r^2} \frac{\partial}{\partial r}\left( r^2 D_1 \frac{\partial w}{\partial r} \right) \\&#xA;\\&#xA;\frac{dr_d}{dt} = C_1\frac{\left(C_2 + C_3 \left( \frac{r_d}{r_{d,0}}\right)^{0.5} \right) D_1}{2r_d}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &#xA;\begin{align}&#xA;D_1 &amp;amp;= A  e^{B w} \\&#xA;A &amp;amp;=\operatorname{const} \\&#xA;B &amp;amp;=\operatorname{const} \\&#xA;C_1 &amp;amp;=\operatorname{const} \\&#xA;C_2 &amp;amp;=\operatorname{const} \\&#xA;C_3 &amp;amp;=\operatorname{const} \\&#xA;r_{d, 0} &amp;amp;=\operatorname{const} \\&#xA;t &amp;amp;\ge 0 \\&#xA;r &amp;amp;\in [0, r_d]&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initial and boundary conditions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;w(0, r) &amp;amp;= w_0 \\&#xA;\frac{\partial w(t, 0)}{\partial r} &amp;amp;= 0 \\&#xA;D_1\frac{\partial w(t, r_d)}{\partial r} &amp;amp;= (1-w)_{r_d} \frac{dr_d}{dt}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is what numerical method is suitable to solve this equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I tried is to solve this equation using Finite difference method but there is non-linear boundary condition, that does not allow to use that method.&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="3171" LastEditDate="2014-11-04T16:00:30.977" LastActivityDate="2014-11-04T21:25:58.237" Title="Numerical method for solving PDE with non-linear boundary conditions" Tags="|pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16095" PostTypeId="2" ParentId="16093" CreationDate="2014-11-04T17:06:56.147" Score="6" Body="&lt;p&gt;I think your analysis is basically right. Some notes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt;&#xA;Pipelining is the wrong word here; what you're looking at here is data dependency. A CPU pipeline splits an individual instruction into multiple steps, and different steps of consecutive instructions can then be executed concurrently. A data dependency, on the other hand, is a situation where an instruction cannot begin execution until the result of a previous instruction is available. This instruction will still be pipelined, and the individual pipeline stages will be executed right up until the stage for which the unavailable previous result is necessary: this is a pipeline stall due to data dependency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your analysis of the data dependency here is right; the tree-version should be the same or faster.&#xA;However, your question about what the compiler will do is complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; If the numbers are floating-point numbers, the two versions are &lt;em&gt;not identical&lt;/em&gt;, so the compiler would not be permitted to change the sequential version into the tree version (that would be a bug if it did). Typically, you would have to specify some option like &lt;code&gt;-ffast-math&lt;/code&gt; to indicate that non-equivalent algebraic floating-point expression transformations are acceptable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The internal details of the processor matter too. In the first version, even if there is a dependency between successive instructions, if there is only one arithmetic unit, there would still not be a stall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; My version of clang (5.1) on my laptop (OSX) does the following to this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;using namespace std;&#xA;&#xA;int main(int argc, char** argv) {&#xA;  typedef double T; // Also try with long&#xA;  T M[8] = {1,2,3,4,5,6,7,8};&#xA;  // Don't let the compiler compute the sum of M's at compile time.&#xA;  for (int i = 1; i &amp;lt;= 8; ++i)&#xA;    if (argc &amp;gt; i) M[i-1] = atof(argv[i]);&#xA;&#xA;  // T S1 = M[0] + M[1] + M[2] + M[3] + M[4] + M[5] + M[6] + M[7];&#xA;  // cout &amp;lt;&amp;lt; S1 &amp;lt;&amp;lt; endl;&#xA;  T S2 = (((M[0] + M[1]) + (M[2] + M[3])) + ((M[4] + M[5]) + (M[6] + M[7])));&#xA;  cout &amp;lt;&amp;lt; S2 &amp;lt;&amp;lt; endl;&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The tree version becomes this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-&amp;gt; 0x100000de0:  addsd  %xmm6, %xmm7&#xA;   0x100000de4:  addsd  %xmm5, %xmm1&#xA;   0x100000de8:  addsd  %xmm7, %xmm1&#xA;   0x100000dec:  addsd  %xmm4, %xmm3&#xA;   0x100000df0:  addsd  %xmm2, %xmm0&#xA;   0x100000df4:  addsd  %xmm3, %xmm0&#xA;   0x100000df8:  addsd  %xmm1, %xmm0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(the top two level of additions actually got sequentialized) and the sequential version becomes this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-&amp;gt; 0x100000de8:  addsd  %xmm7, %xmm2&#xA;   0x100000dec:  addsd  %xmm6, %xmm2&#xA;   0x100000df0:  addsd  %xmm4, %xmm2&#xA;   0x100000df4:  addsd  %xmm5, %xmm2&#xA;   0x100000df8:  addsd  %xmm1, %xmm2&#xA;   0x100000dfc:  addsd  %xmm3, %xmm2&#xA;   0x100000e00:  addsd  %xmm0, %xmm2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is with flags &lt;code&gt;-O3 -ffast-math -std=c++11 -g&lt;/code&gt;. To get the assembly output, I ran the program under &lt;code&gt;lldb&lt;/code&gt;, set a breakpoint at the line of interest with &lt;code&gt;break&lt;/code&gt; and used the command &lt;code&gt;dis&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So clearly my version of clang doesn't rewrite the expression like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4.&lt;/strong&gt; As for performance, obviously the second version should be at least as fast. However, you have to actually try this on a function you actually have. And if you find that the performance improvement is insignificant, it's just not worth thinking about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;5.&lt;/strong&gt; As an example of how it can be insignificant, if you store the vector of values to be added in memory, the time required to fetch all the values from memory is much much greater than the time required to actually add them up. In hierarchical memory, accesses to main memory (rather than registers, or the L1 cache) are just so much slower that performance gains from these kinds of micro-optimizations are just rounding errors. Most of the time the CPU will not be executing anything, and will just be sitting there waiting for data to arrive.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-11-04T19:28:39.240" LastActivityDate="2014-11-04T19:28:39.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16096" PostTypeId="2" ParentId="16094" CreationDate="2014-11-04T21:25:58.237" Score="2" Body="&lt;p&gt;All right, so $dr_{d}/dt$ is coupled to $w$ because your diffusion coefficient is a function of $w$. The boundary condition being nonlinear in $w$ isn't a big deal, because after discretization in space, you'll have a collection of differential and algebraic equations in space, and you can use an implicit method to handle it. The trickier thing is that $r_{d}$ changes with time, because that affects how you handle the spatial discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a first attempt, I'd rescale the spatial coordinate $r$, and re-express your PDE for $w$ in terms of $\xi = r/r_{d}$. In $\xi$, your BCs are now at $0$ and $1$, and it shouldn't really impact the radial symmetry BC or the Dirichlet initial condition. It will affect the flux BC at $r = r_{d}$ (now $\xi = 1$). Rescaling the radial coordinate makes it possible to discretize $\xi$ on $[0,1]$ with fixed node locations (in $r$, the node locations will change with time, except for the node at 0). You should be able to use a finite-difference discretization, and it will be fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other approaches are possible, like FEM with cut cells, but I wouldn't investigate that sort of approach right away unless you're familiar with it.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-04T21:25:58.237" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16097" PostTypeId="1" CreationDate="2014-11-04T23:09:22.670" Score="2" ViewCount="139" Body="&lt;p&gt;I am concerned with numerical solution to the following problem on $[0,1]\times[0,1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial\theta}{\partial t}+u(x,t).\nabla \theta(x,t)=\kappa \nabla^2\theta(t,x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with Dirichlet boundary condition $\theta(t,x)=0$ on the boundary of rectangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question in general situation, and in particular for the problem above is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I use trignometric (Fourier) spectral Galerkin approach here ? &#xA;In other words, would using $\phi_{i,j}=sin(2i\pi x)sin(2j\pi y)$ as basis for galerkin give rise to Gibbs phenomenon or not ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial condition I am interested in is smooth and zero at boundaries as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that for non-periodic domains, we ought to be using Chebyshev polynomials instead. But here, can we assume that solution is &quot;periodic&quot; since it is zero at the boundaries ?&lt;/p&gt;&#xA;" OwnerUserId="11698" LastEditorUserId="11698" LastEditDate="2014-11-05T04:42:38.020" LastActivityDate="2014-11-05T04:42:38.020" Title="Trignometric/Fourier spectral collocation with zero Dirichlet BC in 2D" Tags="|fluid-dynamics|advection-diffusion|spectral-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16098" PostTypeId="1" CreationDate="2014-11-05T01:30:15.783" Score="3" ViewCount="422" Body="&lt;p&gt;I have a set of paths between two fixed points (marked in red below). Each of these paths consists of an ordered series of $\{x, y\}$ points (marked in blue).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fnbkz.jpg&quot; alt=&quot;Simple Scenario&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to find the ordered set of points which comprise the boundary of these paths. This boundary is shown in gray. Note that this boundary would contain the intersection of these lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've implemented convex hull algorithms to try and solve this, but, by definition, they discard the concave part of the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a deterministic way to solve this without brute-force calculation? If not, are there any approximation algorithms which give better performance?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Based on the comment by @k20, I realized that there was a case which I had not considered. $x$ and $y$ can both increase and decrease along a path. There can even be loops in the path. Here is another example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/SiFWp.jpg&quot; alt=&quot;Loop Scenario&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="8176" LastEditorUserId="8176" LastEditDate="2014-11-07T03:20:16.657" LastActivityDate="2014-11-07T03:20:16.657" Title="Computing the (non-convex) boundary of a set of paths between two points" Tags="|algorithms|c++|computational-geometry|approximation|convex-hull|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16099" PostTypeId="2" ParentId="16098" CreationDate="2014-11-05T02:41:32.543" Score="1" Body="&lt;p&gt;If you represent the set of paths as a set of individual line segments $S=\{L_k\}$ between consecutive points, then the intersection of the set you want with the line $x=x_0$ is between the lower-most and the upper-most intervals that contain the $x$-coordinate $x_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, your set is bounded by a set of line segments whose endpoints lie only on endpoints of $L_k$ or on their intersections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it is enough to get all the $x$-coordinates of the line segments and their mutual intersections, and at each such coordinate find the bottom and top points that lie on any line segment. The lines between these bottom and top points form the boundary of your set.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-11-05T02:41:32.543" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16100" PostTypeId="2" ParentId="11017" CreationDate="2014-11-05T02:48:53.383" Score="1" Body="&lt;p&gt;Matlab R2014b has this 'histogram' function which has the 'normalization' property, it may give what you want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/histogram.html#namevaluepairarguments&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/matlab/ref/histogram.html#namevaluepairarguments&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8191" LastActivityDate="2014-11-05T02:48:53.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16101" PostTypeId="2" ParentId="16097" CreationDate="2014-11-05T02:51:15.017" Score="1" Body="&lt;p&gt;Yes, you can use Fourier expansions. It will lead to Gibb's phenomenon, but since you're on the unit square you can likely use enough Fourier modes that that doesn't matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the boundary values are equal on opposing boundaries, you can use periodic boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-05T02:51:15.017" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16103" PostTypeId="2" ParentId="16080" CreationDate="2014-11-05T09:49:18.590" Score="10" Body="&lt;p&gt;If I remember correctly, the ADMM ist often stated as an algorithm to solve&#xA;$$\min_{x,y}\ F(x) + G(y),\quad\text{s.t}\quad Ax+By = c$$&#xA;for two convex, lower-semicontinuous functionals $F$ and $G$ and linear, bounded operators $A$ and $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find the following special case of $A=I$, $B=-I$ and $c=0$ illustrative. In this case the constraint says $x - y = 0$, i.e. we can substitute to get the problem&#xA;$$\min_x F(x) + G(x).$$&#xA;Now solving this can be hard, while solving problems of the form&#xA;$$\min_x \rho F(x) + \tfrac12\|x-z\|^2$$&#xA;can be easy. (You can make up examples for this yourself, a popular one is $F(x) = \lambda\|x\|^1$ and $G(x) = \tfrac12\|Ax-b\|^2$).&#xA;In ADMM you start from the &quot;splitted form&quot;&#xA;$$\min_{x,y}\ F(x) + G(y),\quad\text{s.t}\quad x-y=0$$&#xA;and build the &quot;augmented Lagragian&quot;&#xA;$$L_\rho(x,y,z) = F(x) + G(y) + z^T(x-y) + \tfrac\rho2\|x-y\|^2$$&#xA;with the Lagrange &lt;strong&gt;multiplier&lt;/strong&gt; $z$. Now you &lt;strong&gt;alternatingly&lt;/strong&gt; minimize the augemented Lagragian in the different &lt;strong&gt;directions&lt;/strong&gt; $x$ and $y$, i.e. iterate&#xA;$$x^{k+1} = \mathrm{argmin}_x\ L_\rho(x,y^k,z^k)$$&#xA;$$y^{k+1} = \mathrm{argmin}_y\ L_\rho(x^{k+1},y,z)$$&#xA;and update the multiplier according to&#xA;$$z^{k+1} = z^k + \rho(x^{k+1} - y^{k+1}).$$&#xA;This should explain the name &lt;strong&gt;alternating directions method of multipliers&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analyzing these minimization problems for $x$ and $y$ closer, you observe that for each update only needs to solve a problem of the &quot;simpler form&quot;, e.g. for the $x$ update&#xA;$$x^{k+1} = \mathrm{argmin}_x\ F(x) + \tfrac\rho2\|x - y^k + \rho z^k\|^2$$&#xA;(neglecting terms that do not depend on $x$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADMM for the problem&#xA;$$\min_{x,y}\ F(x) + G(y),\quad\text{s.t}\quad Ax+By = c$$&#xA;is derived similar but then the intermediate problems for the updates are still a bit difficult but may be comparably simple in comparison to the original one. Especially in the case of $F(x) = \lambda\|x\|_1$ and $G(x) = \tfrac12\|Ax-b\|^2$ (or equivalently $F(x) = \lambda\|x\|_1$, $G(y) = \tfrac12\|y\|^2$ and the constraint $Ax - y = b$) the updates are more or less straightforward to implement.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2014-11-05T09:49:18.590" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16105" PostTypeId="1" CreationDate="2014-11-05T14:24:59.377" Score="2" ViewCount="1967" Body="&lt;p&gt;In Gram-Schmidt process, &#xA;is it better to do normalization after orthogonalization of all the vectors in a basis, or to normalize each new vector immediately after it is created, from computational point of view?&lt;/p&gt;&#xA;" OwnerUserId="258" LastEditorUserId="713" LastEditDate="2014-11-06T19:37:36.050" LastActivityDate="2014-11-07T08:38:15.320" Title="Is it better to do normalization after all orthogonalization in Gram-Schmidt process?" Tags="|linear-algebra|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16106" PostTypeId="1" CreationDate="2014-11-05T14:36:28.740" Score="1" ViewCount="339" Body="&lt;p&gt;I've been looking at the book &quot;Computational Fluid Dynamics - The Basics With Applications&quot;, by John D. Anderson, and his solution for the one-dimensional subsonic-supersonic nozzle flow using the MacCormack scheme. I was wondering if there are any book chapters or academic papers out there explaining, in the level of detail given by Anderson, how to solve two dimensional nozzle flow. I found the following AIAA paper, but cannot access it since I'm unfortunately not a member: &lt;a href=&quot;http://arc.aiaa.org/doi/abs/10.2514/3.61365&quot; rel=&quot;nofollow&quot;&gt;http://arc.aiaa.org/doi/abs/10.2514/3.61365&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="11673" LastActivityDate="2014-11-05T14:36:28.740" Title="Two Dimensional Nozzle Flow" Tags="|fluid-dynamics|computational-physics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16107" PostTypeId="1" CreationDate="2014-11-05T15:50:32.613" Score="2" ViewCount="1382" Body="&lt;p&gt;I have found that MATLAB solves 1D parabolic-elliptic system incorrectly by using pdepe function. Here is a system:&#xA;$$&#xA;u_t = u_{xx} + 2,&#xA;$$&#xA;$$&#xA;0 = v_{xx} + u.&#xA;$$&#xA;Boundary conditions:&#xA;$$&#xA;(-u_x+u)|_{x=0} = 2t, \;\; (u_x+u)|_{x=1} = 2t,&#xA;$$&#xA;$$&#xA;(-v_x+v)|_{x=0} = 0, \;\; (v_x+v)|_{x=1} = 0.&#xA;$$&#xA;Initial conditions:&#xA;$$&#xA;u|_{t=0} = 0,\;\;v|_{t=0} = 0.&#xA;$$&#xA;MATLAB documentation informs that pdepe can solve such systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function a&#xA;&#xA;tt = 1;&#xA;&#xA;m = 0;&#xA;x = linspace(0,1,100);&#xA;t = linspace(0,tt,100);&#xA;&#xA;sol = pdepe(m,@apde,@aic,@abc,x,t);&#xA;u1 = sol(:,:,1);&#xA;u2 = sol(:,:,2);&#xA;&#xA;figure;&#xA;surf(x,t,u1);&#xA;title('u1(x,t)');&#xA;xlabel('Distance x');&#xA;ylabel('Time t');&#xA;shading interp&#xA;colorbar&#xA;set(gca,'YDir','reverse');&#xA;&#xA;figure;&#xA;surf(x,t,u2);&#xA;title('u2(x,t)');&#xA;xlabel('Distance x');&#xA;ylabel('Time t');&#xA;shading interp&#xA;colorbar&#xA;set(gca,'YDir','reverse');&#xA;&#xA;figure&#xA;plot(x,u1(end,:))&#xA;title('u1 at t = tt')&#xA;xlabel('Distance x')&#xA;ylabel('u1(x,tt)')&#xA;&#xA;figure&#xA;plot(x,u2(end,:))&#xA;title('u2 at t = tt')&#xA;xlabel('Distance x')&#xA;ylabel('u2(x,tt)')&#xA;&#xA;figure&#xA;plot(x,tt*(1+x-x.^2))&#xA;title('TRUE u2 at t = tt')&#xA;xlabel('Distance x')&#xA;ylabel('u2(x,tt)')&#xA;&#xA;% --------------------------------------------------------------&#xA;&#xA;function [c,f,s] = apde(x,t,u,DuDx)&#xA;c = [1; 0];                                 &#xA;f = [1; 1] .* DuDx;                  &#xA;s = [2; u(1)];&#xA;&#xA;% --------------------------------------------------------------&#xA;&#xA;function u0 = aic(x)&#xA;u0 = [0; 0];                                &#xA;&#xA;% --------------------------------------------------------------&#xA;&#xA;function [pl,ql,pr,qr] = abc(xl,ul,xr,ur,t)&#xA;pl = [ul(1)-2*t; ul(2)];                              &#xA;ql = [-1; -1];                                 &#xA;pr = [ur(1)-2*t; ur(2)];                           &#xA;qr = [1; 1];              &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Exact solution:&#xA;$$&#xA;u(x,t) = 2t,\;\;v(x,t) = t(1+x-x^2).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB computes $u$ correctly.&#xA;Exact solution for $v$ at $t = 1$:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/PLiU3.jpg&quot; alt=&quot;Exact solution for $v$ at $t = 1$&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Approximate solution computed by MATLAB:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/0Uisi.jpg&quot; alt=&quot;Approximate solution computed by MATLAB&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This function doesn't even satisfy the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v(x,t)$ graph:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gOYOw.jpg&quot; alt=&quot;$v(x,t)$ graph&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us replace our elliptic equation with a parabolic one, that is use the vector&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c = [1; 1e-100];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;instead of&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c = [1; 0];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It means that we use the equation&#xA;$$&#xA;10^{-100}v_t = v_{xx} + u&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now MATLAB computed a correct solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/BKtjY.jpg&quot; alt=&quot;solution&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, this example demonstrates that MATLAB solves parabolic-elliptic systems with Robin boundary conditions incorrectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we use Dirichlet boundary conditions, the solution is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my reasoning correct?&lt;/p&gt;&#xA;" OwnerUserId="11531" LastActivityDate="2018-02-28T12:18:16.170" Title="MATLAB: incorrect computation with pdepe for parabolic-elliptic system" Tags="|pde|matlab|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16109" PostTypeId="1" CreationDate="2014-11-05T23:31:10.373" Score="0" ViewCount="197" Body="&lt;p&gt;Question: Consider the iterative improvement algorithm below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/6oEJa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with $Az_i = r_i$ and $(A + E)\hat{z}_i = r_i$ derive a formula showing how the absolute error in the $(i + 1)^{st}$ iterate $\Vert\hat{x}_{i+1} - x\Vert$ is bound by a multiple of the absolute error in the $i$-th iterate $\Vert\hat{x}_{i} - x\Vert$. Argue that the magnitude of this multiple is dependent on the condition of A, and is less than one (hence convergence) if $A$ is well-conditioned. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{\kappa(A)} \frac{\Vert r_i\Vert}{\Vert b\Vert} \le \frac{\Vert\hat{x}_{i} - x\Vert}{\Vert x\Vert} \le \kappa(A) \frac{\Vert r_i\Vert}{\Vert b\Vert}$$&#xA;being $\kappa(A)$ the condition number of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now do:&#xA;$$&#xA;\Vert\hat{x}_{i+1} - x\Vert = \Vert(\hat{x}_{i} + z_i) - x\Vert \le \Vert\hat{x}_{i} - x\Vert + \Vert z_i\Vert \le \frac{\Vert\hat{x}_{i} - x\Vert}{\Vert x\Vert} \le \kappa(A) \frac{\Vert r_i\Vert\, \Vert x\Vert}{\Vert b\Vert} + \Vert A^{-1}r_i\Vert$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$= \Vert A\Vert\, \Vert A^{-1}\Vert \frac{\Vert r_i\Vert\, \Vert x\Vert}{\Vert b\Vert} + \Vert A^{-1}r_i\Vert \le \Vert A\Vert \Vert A^{-1}\Vert \frac{\Vert r_i\Vert\, \Vert x\Vert}{\Vert b\Vert} + \Vert A^{-1}\Vert\, \Vert r_i\Vert$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$= \Vert A^{-1}\Vert\, \Vert r_i\Vert \left(\Vert A\Vert \frac{\Vert x\Vert}{\Vert b\Vert}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I did is not correct because I need to bound the absolute error of $||\hat{x}_{i+1} - x||$ by a multiple of  $||\hat{x}_{i} - x||$ which I am not sure how to do. &lt;/p&gt;&#xA;" OwnerUserId="11708" LastEditorUserId="123" LastEditDate="2014-11-08T05:33:23.830" LastActivityDate="2014-11-08T05:33:23.830" Title="Iterative algorithm prove precise conditions for convergence" Tags="|numerical-analysis|convergence|error-estimation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16110" PostTypeId="1" CreationDate="2014-11-06T02:17:14.217" Score="1" ViewCount="351" Body="&lt;p&gt;The B-spline basis function is given by one recursive formula. However，I hope to obtain popular expression: a0x^n + a1x^(n-1) + ... + an.&#xA;From the answer &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2466/how-to-build-a-recursive-spline-function-in-c&quot;&gt;How to build a recursive spline function in C++&lt;/a&gt;, it seems that Piegl's algorithm can do that. Can anyone give more information about the algorithm?&lt;/p&gt;&#xA;" OwnerUserId="7846" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-11-07T18:59:36.510" Title="Piegl's algorithm in calculating B-spline basis function" Tags="|b-spline|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16111" PostTypeId="1" CreationDate="2014-11-06T06:47:42.830" Score="1" ViewCount="213" Body="&lt;p&gt;In Matlab Tensor Toolbox there is a tensor type called &quot;Kruskal tensors&quot;, I found its form is similar to the CP decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia mentioned:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;&lt;a href=&quot;http://en.wikipedia.org/wiki/Higher-order_singular_value_decomposition&quot; rel=&quot;nofollow&quot;&gt;As such, many of the methods have been independently invented&#xA;  several times, often with subtle variations, leading to a confusing&#xA;  literature. Abstract and general mathematical theorems are rare&#xA;  (though see Kruskal[1] with regard to the CP decomposition)&lt;/a&gt;&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In a &lt;a href=&quot;http://arxiv.org/pdf/0901.1796.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; on [&lt;em&gt;A concise proof of Kruskal's theorem on tensor decomposition&lt;/em&gt;]， it mentioned: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;...In these fields, the expression of a tensor as a sum of rank-1&#xA;  tensors is often referred to as the Candecomp or Parafac&#xA;  decomposition.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So, what is the relation or difference between Kruskal tensor decomposition and &lt;a href=&quot;http://en.wikipedia.org/wiki/CP_decomposition&quot; rel=&quot;nofollow&quot;&gt;CP decomposition&lt;/a&gt;?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are they actually the same thing but Kruskal decomposition is devised by a different person in different field for third-order tensor?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="7969" LastEditorUserId="7969" LastEditDate="2014-11-06T10:30:54.360" LastActivityDate="2014-11-06T10:30:54.360" Title="What is the relation between Kruskal tensor and CP decomposition?" Tags="|tensor|tensor-decomposition|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16113" PostTypeId="2" ParentId="16110" CreationDate="2014-11-06T14:27:24.613" Score="2" Body="&lt;p&gt;If you do not have access to the NURBS book by Piegl, then you might look at An Introduction to Polar Forms by Hans-Peter Seidel on pp 38-46 of the 1993 IEEE Computer Graphics &amp;amp; Applications. He diagrams the B-spline computations for the polar form, from which the polynomial coefficient form is a routine conversion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me offer one more reference to an excellent expositor, Phillip J. Schneider, in his paragraph on defining the basis functions. Www.mactech.com/articles/develop/issue_25/schneider.html&lt;/p&gt;&#xA;" OwnerUserId="9683" LastEditorUserId="9683" LastEditDate="2014-11-07T18:59:36.510" LastActivityDate="2014-11-07T18:59:36.510" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16114" PostTypeId="1" AcceptedAnswerId="16125" CreationDate="2014-11-06T18:10:02.240" Score="1" ViewCount="66" Body="&lt;p&gt;Imagine you are given 20 random numbers in a row. The original order must be maintained. From this set you can choose 2 groups of 3 numbers each. The position of these groups must be separated by at least 4 numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to maximise the sum of values in both groups. How can you ensure that optimal groups are taken?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main issue is that the rule to choose the first group may worsen the choice of the second group due to the constraint. The optimal solution may exclude the 'best' single group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What method is there to solve this generally for any size of original set, size of groups, number of groups and length of constraint?&lt;/p&gt;&#xA;" OwnerUserId="11715" LastEditorUserId="11715" LastEditDate="2014-11-08T20:35:54.347" LastActivityDate="2014-11-08T20:35:54.347" Title="Picking highest value non-adjacent groups within a set" Tags="|constrained-optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16115" PostTypeId="2" ParentId="16105" CreationDate="2014-11-06T18:42:35.260" Score="-1" Body="&lt;p&gt;It is best to use Housholder Transofrmations.&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Householder_transformation&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Householder_transformation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11717" LastActivityDate="2014-11-06T18:42:35.260" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16116" PostTypeId="1" CreationDate="2014-11-06T18:56:34.997" Score="0" ViewCount="549" Body="&lt;p&gt;I wish to study the 2D heat transfer of the soda can using Fluent. For my geometry I took a circle. &#xA;My problem while meshing the circle; Because i need to mesh it from inside.&#xA;I used the mapped faced meshing then I used the edge sizing for the edge of the circle with no bias and then I sized the line that I create.&#xA;[I tried to create 1 vertical line (Edge to Edge) and size it using this bias&#xA; ----- --- - --- ----- ]&#xA;But the mesh was not very symmetric and didn't have circular shape.&#xA;I tried put a second horizontal line and size using the correct bias type and factor but also didn't get what I want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I want a good mesh inside a circle in 2D (Not pipe in 3D)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: please as a beginner in fluent I wish to have a step by step procedure to follow &lt;/p&gt;&#xA;" OwnerUserId="11720" LastEditorUserId="4964" LastEditDate="2014-11-07T09:13:16.097" LastActivityDate="2014-11-07T09:13:16.097" Title="CFD: Mesh problem" Tags="|fluid-dynamics|mesh|ansys-fluent|" AnswerCount="1" CommentCount="2" ClosedDate="2014-11-07T17:02:19.860" ContentLicense="CC BY-SA 3.0" />
  <row Id="16118" PostTypeId="2" ParentId="16105" CreationDate="2014-11-06T19:37:18.490" Score="4" Body="&lt;p&gt;When you consider the orthogonalization procedure, if $u$ and $v$ are vectors, then the orthogonal component of $u$ is either&#xA;$$ u - (u,v)v $$&#xA;if $v$ has length $1$, or&#xA;$$ u - \frac{(u,v)v}{\|v\|^2} = u - \left(u,\frac{v}{\|v\|}\right)\frac{v}{\|v\|} $$&#xA;if $v$ has some other length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if $v$ has not been normalized somewhere earlier in the procedure, its every appearance will be normalized anyway. So it shouldn't matter: either you normalize it immediately or you normalize it every time it appears later.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-11-06T19:37:18.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16119" PostTypeId="1" CreationDate="2014-11-06T22:48:28.630" Score="5" ViewCount="1370" Body="&lt;p&gt;I am trying to derive the correct variational form for the Poisson equation with pure Neumann boundary conditions, and an additional contraint $\int_{\Omega} u \, {\rm d} x = 0$, as described in &lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/dev/python/demo/documented/neumann-poisson/python/documentation.html&quot; rel=&quot;nofollow&quot;&gt;this link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, the problem can be defined as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Find $(u, c) \in V \times \mathbb{R}$ such that&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$F(u, c) = \frac1{2} \int \nabla u \cdot \nabla u - \int f u -&#xA; \int_{\partial \Omega}g u + c \int u$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;is minimized wrt $u$ and $c$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To minimize with respect to $u$, I started writing &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left. \frac{\partial}{\partial \epsilon}F(u + \epsilon v, c)\right|_{\epsilon=0} = 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which led to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int \nabla u \cdot \nabla v + \int c v = \int f v + \int_{\partial \Omega} g v $. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to $c$ this gives $\int u = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just adding these two equalities leads to the variational form mentioned in the link (where $d \int u = 0$ for all $d \in \mathbb{R}$ ensures that the two equations remain true separatly). Is this correct so far?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in the finite element description, we have one more unknown, $c$, i.e. the Lagrange multiplier, no? To go to a set of equations, the variational form of the equation must be true for all $N$ testfunctions (shapefunctions) $v$. Am I right that this will lead to a $N \times (N + 1)$ matrix equation? Am I also correct that, since the expression should be valid for all $d \in \mathbb{R}$ we can pick a random value in the actual evaluation of these forms?&lt;/p&gt;&#xA;" OwnerUserId="11723" LastEditorUserId="11723" LastEditDate="2014-11-07T13:30:45.433" LastActivityDate="2014-11-07T13:30:45.433" Title="Poisson equation with pure Neumann boundary conditions (using FEM)" Tags="|finite-element|fenics|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="16120" PostTypeId="1" CreationDate="2014-11-07T00:02:01.583" Score="1" ViewCount="1035" Body="&lt;p&gt;How do I plot a flow diagram (given certain initial conditions, the trajectory in y-x space) for a RG (renormalization group) flow given by the following recursion relations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;x^{\prime}&amp;amp;=&amp;amp;\frac{x(1+y)^2}{(x+y)(1+xy)} \\&#xA; y^{\prime}&amp;amp;=&amp;amp;\frac{y(x+y)}{1+xy}  \\&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have explored several options such as RSolve, Equation trekker package on Mathematica, but none of these can be used to plot a flow diagram for a continuous transformation.&lt;/p&gt;&#xA;" OwnerDisplayName="user1717" LastEditorUserId="713" LastEditDate="2014-11-07T02:48:58.177" LastActivityDate="2014-11-07T02:48:58.177" Title="Plotting renormalization group flow diagram from recursion relation" Tags="|plotting|" AnswerCount="0" CommentCount="4" ClosedDate="2019-06-07T23:16:03.857" ContentLicense="CC BY-SA 3.0" />
  <row Id="16121" PostTypeId="2" ParentId="16114" CreationDate="2014-11-07T04:49:48.620" Score="1" Body="&lt;p&gt;You can write this as the following integer optimization problem, if I understand your question correctly: Let $i,j\in [1,20]$ be the starting indices of the two groups of three numbers, then you want to solve&#xA;$$&#xA;  \max_{i,j} x_i+x_{i+1}+x_{i+2}+x_j+x_{j+1}+x_{j+2}&#xA;  \\&#xA;  i \ge 1&#xA;  \\&#xA;  j-i \ge 7&#xA;  \\&#xA;  j \le 18.&#xA;$$&#xA;Like all integer optimization problems, it is likely difficult to find an exact answer. On the other hand, a greedy algorithms with complexity $O(N)$ is likely going to find the correct solution if you allow the size of your array of numbers $N$ to become large.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-07T04:49:48.620" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16122" PostTypeId="2" ParentId="16116" CreationDate="2014-11-07T07:58:28.107" Score="3" Body="&lt;p&gt;Go for a 5 block structured O-type mesh:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/cg5Ml.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2014-11-07T07:58:28.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16123" PostTypeId="2" ParentId="16105" CreationDate="2014-11-07T08:38:15.320" Score="2" Body="&lt;p&gt;The short answer is yes, you should normalize the orthogonal projection of $u$ immediately after it is computed so that the orthogonalization algorithm may continue to generate the basis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it deserves mention that when one computes an orthogonal basis with the Gram-Schmidt process in finite precision (i.e. numerically) it is possible to create a basis that is no longer orthogonal. As a maliciously chosen example, consider the vectors&#xA;$$&#xA;u_1 = (1,\epsilon,0,0)^T,\quad u_2 = (1,0,\epsilon,0)^T, \quad u_3=(1,0,0,\epsilon)^T,&#xA;$$&#xA;where we assume that $1 + \epsilon^2\approx 1$. Then the &lt;a href=&quot;http://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process&quot; rel=&quot;nofollow&quot;&gt;classical Gram-Schmidt process&lt;/a&gt; creates the &quot;orthogonal&quot; basis vectors&#xA;$$&#xA;q_1 = (1,\epsilon,0,0)^T,\quad q_2=\frac{1}{\sqrt{2}}(0,-1,1,0)^T,\quad q_3 = \frac{1}{\sqrt{2}}(0,-1,0,1)^T.&#xA;$$&#xA;But a quick check of orthogonality yields, for example,&#xA;$$&#xA;q_2^Tq_3 = \frac{1}{2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This loss of orthogonality comes from the accumulation of roundoff errors in the classical Gram-Schmidt process. That is, in classical Gram-Schmidt we compute (signed) lengths of the orthogonal projections of $u_i$ onto the previous basis vectors $q_1, q_2,\ldots,q_{i-1}$, subtract these projections (and the rounding errors) from $u_i$ to obtain the new projection $w$, normalize, and obtain the next piece of the basis $q_i$. This projection (in exact arithmetic) is compactly written&#xA;$$&#xA;w = \left(I-Q_{i-1}Q_{i-1}^T\right)u_i,&#xA;$$&#xA;where the columns of $Q_{i-1}$ are the previously computed basis vectors $q_k$, $k=1,\ldots,i-1$. But numerically, because of rounding errors, the matrix $Q_{i-1}$ &lt;strong&gt;&lt;em&gt;does not&lt;/em&gt;&lt;/strong&gt; have truly orthogonal columns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To stabilize the approximation and help guarantee that the numerical procedure will create an orthonormal basis in finite precision we use the modified Gram-Schmidt process. The difference is subtle but stabilizes the computation such that the vectors created will be &quot;much more&quot; orthogonal than those from classical Gram-Schmidt. The key is to ensure that the computation projects and orthogonalizes with respect to the &lt;strong&gt;&lt;em&gt;computed version&lt;/em&gt;&lt;/strong&gt; of the vector $w$. The spirit of the algorithm is the same, this is just a reordering of the computation to look like&#xA;$$&#xA;w = \left(I - q_{i-1}q_{i-1}^T\right)\ldots\left(I-q_1q_1^T\right)u_i.&#xA;$$&#xA;Notice that in exact arithmetic the two formulations will generate identical output. If we consider the same example as before, modified Gram-Schmidt will compute the orthonormal basis&#xA;$$&#xA;q_1 = (1,\epsilon,0,0)^T,\quad q_2=\frac{1}{\sqrt{2}}(0,-1,1,0)^T,\quad q_3 = \frac{1}{\sqrt{6}}(0,-1,-1,2)^T,&#xA;$$&#xA;for which we have discrete orthogonality (in finite precision as $\epsilon$ is assumed on the order of unit roundoff)&#xA;$$&#xA;q_1^Tq_2 = -\frac{\epsilon}{\sqrt{2}},\quad q_1^Tq_3 = -\frac{\epsilon}{\sqrt{6}},\quad q_2^Tq_3 = 0.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="1057" LastActivityDate="2014-11-07T08:38:15.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16124" PostTypeId="2" ParentId="16092" CreationDate="2014-11-07T16:39:05.933" Score="1" Body="&lt;p&gt;If the function is known to have a Gaussian form then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int\limits^{\infty}_0 \left( A \exp{\left(-\alpha\, t^2\right)} \right)^2 dt &amp;gt; 2 \int\limits^{\infty}_0 A \exp{\left(-\alpha\, t^2\right)}  dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is true for any choice of the pair ${ \alpha, A } \in \Re$ where $\alpha &amp;gt; 0$. If &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A &amp;gt; 2 \sqrt{2}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;then your equality is satisfied for any $\alpha$. You could generate arbitrary pairs of $A$ and $\alpha$ in Python quite easily with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;A_vec = np.linspace(A1,A2,ALength)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and similarly for $\alpha$&lt;/p&gt;&#xA;" OwnerUserId="9580" LastActivityDate="2014-11-07T16:39:05.933" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="16125" PostTypeId="2" ParentId="16114" CreationDate="2014-11-07T17:41:46.430" Score="1" Body="&lt;p&gt;Here's a brute force quadratic solution to the original problem, a linear solution to the original problem, and a brute force and generic linear solution for three groups.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;&lt;code&gt;&#xA;import numpy as np&#xA;&#xA;N = 20&#xA;G = 3&#xA;MINSEP = 4&#xA;&#xA;def solve_cubic(a):&#xA;    r = range(N - G + 1)&#xA;    return max(&#xA;            (&#xA;                (a[i:i+G]+a[j:j+G]+a[k:k+G]).sum(), i, j, k&#xA;                ) for i in r for j in r for k in r if (&#xA;                    i + G + MINSEP &amp;lt= j and j + G + MINSEP &amp;lt= k))&#xA;&#xA;def solve_quadratic(a):&#xA;    r = range(N - G + 1)&#xA;    return max(&#xA;            (a[i:i+G].sum() + a[j:j+G].sum(), i, j) for i in r for j in r if (&#xA;                j-i &gt; G+MINSEP-1))&#xA;&#xA;def solve_linear(a):&#xA;    sumc = np.cumsum(a)&#xA;    sumc = np.array([0] + sumc.tolist())&#xA;    sumg = sumc[G:] - sumc[:-G]&#xA;    rmax = np.maximum.accumulate(sumg[::-1])[::-1]&#xA;    scores = sumg[:-(MINSEP+G)] + rmax[(MINSEP+G):]&#xA;    i = np.argmax(scores)&#xA;    j = i + MINSEP + G + np.argmax(sumg[i + MINSEP + G:])&#xA;    return scores[i], i, j&#xA;&#xA;def solve_linear_generic(numbers, groups):&#xA;    sumc = np.cumsum([0] + list(numbers))&#xA;    sumg = sumc[G:] - sumc[:-G]&#xA;    table = {}&#xA;    trace = {}&#xA;    for a in range(N - G + 2):&#xA;        a_prime = a - MINSEP - G&#xA;        for b in range(groups + 1):&#xA;            if a == 0:&#xA;                table[a, b] = 0 if b == 0 else -1&#xA;                trace[a, b] = ()&#xA;            else:&#xA;                c0 = table[a-1, b]&#xA;                tracec0 = trace[a-1, b]&#xA;                if b == 0:&#xA;                    table[a, b] = c0&#xA;                    trace[a, b] = tracec0&#xA;                else:&#xA;                    c1 = sumg[-a]&#xA;                    tracec1 = (N - G + 1 - a,)&#xA;                    if a_prime &gt;= 0:&#xA;                        c1 += table[a_prime, b-1]&#xA;                        tracec1 = tracec1 + trace[a_prime, b-1]&#xA;                    if c0 &amp;lt c1:&#xA;                        table[a, b] = c1&#xA;                        trace[a, b] = tracec1&#xA;                    else:&#xA;                        table[a, b] = c0&#xA;                        trace[a, b] = tracec0&#xA;    return (table[N - G + 1, groups],)  + trace[N - G + 1, groups]&#xA;&#xA;def main():&#xA;    a = np.random.randint(100, size=N)&#xA;    print a&#xA;    print 'two groups:'&#xA;    print solve_quadratic(a)&#xA;    print solve_linear(a)&#xA;    print solve_linear_generic(a, 2)&#xA;    print 'three groups:'&#xA;    print solve_cubic(a)&#xA;    print solve_linear_generic(a, 3)&#xA;&#xA;main()&#xA;&lt;/code&gt;&#xA;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2014-11-07T19:53:56.590" LastActivityDate="2014-11-07T19:53:56.590" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16126" PostTypeId="2" ParentId="14713" CreationDate="2014-11-08T01:03:08.530" Score="4" Body="&lt;p&gt;The closed-form solution is available by projecting the problem into the space spanned by B. To see this, note that we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{v\in\mathbb{R}^{n}}\|X-BD_{v}B^{T}\|_{F},$$&#xA; but if we introduce $\tilde{X}$ such that $X=B\tilde{X}B^{T}$, then the optimization is reduced&#xA;$$\arg\min_{v\in\mathbb{R}^{n}}\|B\tilde{X}B^{T}-BD_{v}B^{T}\|_{F}\equiv\arg\min_{v\in\mathbb{R}^{n}}\|\tilde{X}-D_{v}\|_{F},$$&#xA; and clearly the best we can do is to set $v=\mathrm{diag}(\tilde{X})$, where the “diag” operator isolates the diagonal of its input matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to obtain $\tilde{X}$, assuming that $B$ has full column rank, is a simple matter of performing a projection onto the range space of $B$, $$\tilde{X}=(B^{T}B)^{-1}B^{T}XB(B^{T}B)^{-1}.$$&#xA; Proof that $X=B\tilde{X}B^{T}$ is obtained by substitution (check this). If $B^TB$ is singular then use the pseudo-inverse. Combined, we have proved the following result.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Theorem 1.&lt;/strong&gt; The optimization has the following closed form solution&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$v=\mathrm{diag}[(B^{T}B)^{-1}B^{T}XB(B^{T}B)^{-1}].$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Finally, it's worth noting that you shouldn't be forming the matrix explicitly if computational load is a concern. Instead, you should take a cholesky factorization of $X$, perform one size of the matrix-matrix product, and then implicitly compute the diagonal. &lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2014-11-08T02:28:44.213" LastActivityDate="2014-11-08T02:28:44.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16127" PostTypeId="1" AcceptedAnswerId="16131" CreationDate="2014-11-08T11:42:40.120" Score="0" ViewCount="1037" Body="&lt;p&gt;I want to pass additional parameters to the ode45 function in MATLAB. There are many questions already on this, but I also have a 'flag' argument in my function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [xdot,isterminal,dircn]=spp(t,x,flag,f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Usually, while calling ode45, something like this would work if spp did not have a flag argument.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options=odeset('RelTol',1e-13,'AbsTol',1e-14,'events','on');&#xA;[t,x,te,xe,ie]=ode45(@(t,x)spp(t,x,f),[0 2*pi/OMEG],[xin;xdin],options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have searched around but cannot find the solution. Please help me out.&lt;/p&gt;&#xA;" OwnerUserId="9756" LastActivityDate="2014-11-08T16:03:42.610" Title="passing arguments to ode45 with 'flag'" Tags="|matlab|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16128" PostTypeId="1" CreationDate="2014-11-08T14:43:53.133" Score="1" ViewCount="2353" Body="&lt;p&gt;I need to fit a curve, obtained from experimental data, with a piecewise linear model (4 knots and therefore 3 lines).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using the MATLAB function &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coeff = lsqnonlin(@fun,X0,[],[],[],x,y);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $fun$ is a function defined as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[f] = fun (X,x,y)&#xA;A = X(1);&#xA;.&#xA;.&#xA;.&#xA;F = X(6);&#xA;&#xA;if x(i) &amp;lt;= kn1 &#xA;    f(i) = A+B*x(i)-y(i);&#xA;elseif x(i) &amp;gt; kn1 &amp;amp;&amp;amp; x(i) &amp;lt;= kn2&#xA;    f(i) = C+D*x(i)-y(i);&#xA;else &#xA;    f(i) = E+F*x(i)-y(i);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this way I should find the coefficients $A,B,C,D,E,F$ which minimize the difference between the model and the real curve (represented by $y$). $X$ is the coefficients vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problem I have is that the knots $kn1$ and $kn2$ cannot be placed anywhere: usually MATLAB places the knots at the same distance one from the other, but in my case I need that the knots are placed in the exact point where the real curve changes its slope (of course the change in the slope does not occur suddenly, it is gradual since it is a curved line but I need to approximate it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do that I thought about some constrained optimization to be run before the fitting, in order to find those 2 points before computing the function $fun$. The MATLAB function I could use to do that is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_opt = fmincon ( fun2, x02, A2, b, Aeq, beq, lb, ub ) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I do not know how to express the constraints related to the slope changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other way I could to this?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="11676" LastEditDate="2014-11-09T14:26:39.670" LastActivityDate="2014-11-09T14:26:39.670" Title="Piecewise linear approximation of an experimental data curve. Knots position constrained" Tags="|matlab|constrained-optimization|regression|curve-fitting|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16130" PostTypeId="1" AcceptedAnswerId="16134" CreationDate="2014-11-08T15:18:28.657" Score="6" ViewCount="5722" Body="&lt;p&gt;I was just implementing the upwind scheme for a linear transport equation $u_t + cu_x = 0$ where $c=0.5$ and I saw that the solution was indeed advected but over time it starts to diffuse. Can anyone provide some references to understand where does it comes from?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2014-11-12T06:15:50.687" Title="How can I prove numerical diffusion in upwind scheme for transport equation" Tags="|finite-difference|advection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16131" PostTypeId="2" ParentId="16127" CreationDate="2014-11-08T15:29:13.847" Score="0" Body="&lt;p&gt;I do not exactly know how to treat 'flags' but usually when I need to pass extra variables to the ode I define them as 'global' in the main .m file and also in the file where I define the function I give to the ode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you might use the varargin function .&lt;/p&gt;&#xA;" OwnerUserId="11676" LastActivityDate="2014-11-08T15:29:13.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16132" PostTypeId="2" ParentId="16127" CreationDate="2014-11-08T16:03:42.610" Score="1" Body="&lt;p&gt;Why don't you simply pass flag the way you are passing f?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[t,x,te,xe,ie]=ode45(@(t,x)spp(t,x,flag,f),[0 2*pi/OMEG],[xin;xdin],options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Global variables in MATLAB often cause problems and it is best to avoid them, if possible.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe I'm missing something here?&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-11-08T16:03:42.610" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16134" PostTypeId="2" ParentId="16130" CreationDate="2014-11-08T18:41:13.630" Score="6" Body="&lt;p&gt;Numerical diffusion arises from a first-order finite difference approximation to the spatial derivative $\partial u/\partial x$. To see how this is the case, examine the Taylor series expansion for $u_{i+1}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u(x_{i+1}) = u(x_{i}) + \left.\frac{\partial u}{\partial x}\right|_{x_i} (\Delta x) + \frac{1}{2} \left. \frac{\partial^2 u}{\partial x^2}\right|_{x_i} (\Delta x)^2 + (\text{Higher order terms})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the first derivative can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial x} =\frac{u(x_{i+1}) - u(x_{i})}{\Delta x}&#xA;- \frac{1}{2}\frac{\partial^2u}{\partial x^2}(\Delta x)&#xA;- \frac{1}{\Delta x}(\text{Higher order terms})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, everything following the first term on the right hand side is truncated when approximating the derivative, but that doesn't mean that it doesn't exist. We can now re-write the original PDE, replacing the exact spatial derivative with the new upwind approximation and moving the truncated terms over to the right hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} + c\left( \frac{u(x_{i+1}) - u(x_{i})}{\Delta x}\right) =&#xA;c\left(\frac{\Delta x}{2}\frac{\partial^2 u}{\partial x^2} + \frac{1}{\Delta x}(\text{Higher order terms})\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, note that the &quot;Higher order terms&quot; are much smaller than the second derivative on the right hand side, so they can be ignored. Therefore, your original diffusion-free PDE has been transformed into a convection-diffusion PDE due to truncation error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} + c\left( \frac{u(x_{i+1}) - u(x_{i})}{\Delta x}\right) =&#xA;D\frac{\partial^2 u}{\partial x^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ is the diffusion coefficient due to numerical diffusion.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2014-11-08T18:41:13.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16135" PostTypeId="1" CreationDate="2014-11-08T19:02:56.407" Score="1" ViewCount="347" Body="&lt;p&gt;Consider Euler equation of gas dynamics in polar coordinates as&lt;br&gt;&#xA;$$&#xA;\left( \begin{array}{ccc}&#xA;\rho \\&#xA;\rho u_{r}  \\&#xA;\rho u_{\theta} \\&#xA;E \end{array} \right)_{t} + &#xA;\frac{1}{r} \left( \begin{array}{ccc}&#xA;r \rho u_{r} \\&#xA;r (\rho u_{r}u_{r} + p)\\&#xA;r \rho u_{r} u_{\theta}\\&#xA;r u_{r}(E + p)\end{array} \right)_{r} + &#xA;\frac{1}{r} \left( \begin{array}{ccc}&#xA;\rho u_{\theta} \\&#xA;\rho u_{r}u_{\theta} \\&#xA;\rho u_{\theta} u_{\theta} + p\\&#xA;u_{\theta}(E + p)\end{array} \right)_{\theta} =  \left( \begin{array}{ccc}&#xA;0 \\&#xA;\frac{p}{r} + \frac{\rho u_{\theta} u_{\theta}}{r} \\&#xA;-\frac{\rho u_{r} u_{\theta}}{r}\\&#xA;0\end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Velocity of shock waves are my interest for this problem. &lt;/li&gt;&#xA;&lt;li&gt;The finite volume numerical method is picked to obtain approximate solutions for this system.   &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Before I run my code for my problem ,which does not have an analytical solution so I can't be certain if my code is correct or not, is there any test cases that I can use to test my code ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my problem boundary conditions are quite challenging, what kind should I pick and how to apply it to my FVM scheme ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab is the preferable language for this calculation. &#xA;Any help is appreciated. &lt;/p&gt;&#xA;" OwnerUserId="11495" LastEditorUserId="11495" LastEditDate="2015-01-18T20:10:51.420" LastActivityDate="2015-01-18T20:10:51.420" Title="What kind of test cases are convenient to use for testing the code for Euler equations of gas dynamics in polar coordinates?" Tags="|pde|matlab|fluid-dynamics|boundary-conditions|finite-volume|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16138" PostTypeId="1" CreationDate="2014-11-10T04:49:34.317" Score="1" ViewCount="1195" Body="&lt;p&gt;We know that principle component analysis (PCA) is a eigenvalue problem. Let $A$ be the covariance matrix of $X$, PCA aims to find the eigenvalue of $A$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max v'Av$, subject to $v'v=1$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiple PCs can be found by deflation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\hat{X}=X-\sum_{i=1}^k v_iv_i'X$ (or equivalently $\hat{A}=(I-\sum_{i=1}^k v_iv_i')A(I-\sum_{i=1}^k v_iv_i')$) where $v_i, i = 1,\dots,k$ is the first $k$ PCs and then the $k+1$st PC can be computed by solving the $1$st PC of $\hat{A}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there is a similar deflation method to find the first $k$ eigenvectors for the generalized eigenvalue problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \frac{v'Av}{v'Bv}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ and $B$ are some covariance matrices: $A=Cov(X), B=Cov(Y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="11587" LastEditorUserId="11587" LastEditDate="2014-11-10T05:37:54.787" LastActivityDate="2016-08-03T05:05:48.713" Title="Deflation for generalized eigenvalue problem" Tags="|optimization|eigenvalues|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16139" PostTypeId="1" CreationDate="2014-11-10T08:18:50.150" Score="2" ViewCount="3611" Body="&lt;p&gt;I want to  minimize &#xA;$$W\, \text{tr}\left([A-Y_{pie}][A-Y_{pie}]^T\right) + \lambda\Vert A\Vert\, \enspace ,$$&#xA;however, I encounter the following problem:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Disciplined convex programming error: Only scalar quadratic forms can be specified in CVX.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Would you like to solve with this, please? Thank you! &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ypie=T*Y; % Y is k*n matrix,T is k*k matrix  ,W is a n*k matrix &#xA;&#xA;lamda=0.01;&#xA;&#xA;cvx_begin&#xA;&#xA;      variable A(K,n)&#xA;&#xA;      minimize (W.*trace((A-Ypie)*(A-Ypie)')+lamda*norm_nuc(A))&#xA;&#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11744" LastEditorUserId="9667" LastEditDate="2014-11-10T21:08:23.137" LastActivityDate="2014-11-13T13:37:58.333" Title="Disciplined convex programming error: Only scalar quadratic forms can be specified in MATLAB's CVX" Tags="|optimization|cvx|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16140" PostTypeId="1" AcceptedAnswerId="16141" CreationDate="2014-11-10T08:57:00.773" Score="0" ViewCount="1182" Body="&lt;p&gt;I have a certain 2-D function. More properly, I have not the function itself, but the matrices $[X,Y,Z]$, where $X,Y$ are $1\times n$, and $Z$ is $n \times n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to calculate a a new matrix, $\hat Z$, whose value at each point is the average of $Z$-values from a small region around that point. In other words,&#xA;$\hat Z(p) = \frac{\int_{B(p,r)} Z(x,y)dxdy}{\pi r^2}$ for some small fixed $r$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I have decide to do this is via convolution with a $\delta$ -like function, and thus via multiplication of the respected FFT transforms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find the FFT of $Z$, as well as the FFT of a cylinder with base $B(0,r)$, and volume $1$. Then I take the product of the FFT transforms, and then take the absolute value of the inverse transform to be $\hat Z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Visually, the result looks correct, i.e. when plotting $\hat Z$ it does look like the original, only more &quot;smoothed-out&quot;. When I increase $r$, the result becomes further flatter and flatter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the are some scaling issues, by which I mean that the scaling of the end result is completely off (orders of magnitude), whereas for small $r$ I expect the range to be roughly the same. I suspect that to be a known issue with the FFT transform, but I do not manage to find the proper scaling factor.&lt;/p&gt;&#xA;" OwnerUserId="9362" LastEditorUserId="713" LastEditDate="2014-11-10T21:14:23.113" LastActivityDate="2014-11-10T21:14:23.113" Title="Scaling factor of the inverse Fourier Transform (for convolution purposes)" Tags="|matlab|fourier-analysis|fourier-transform|smoothing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16141" PostTypeId="2" ParentId="16140" CreationDate="2014-11-10T14:35:53.137" Score="1" Body="&lt;p&gt;You have to normalize by the number of elements in the FFT.  That is if the size of your matrix is $(NxM)$ you must normalize your FFT by $(NxM)$.  You can check this is valid by looking at the energy content of the two matrices.&lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2014-11-10T14:35:53.137" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16142" PostTypeId="1" CreationDate="2014-11-10T19:36:59.177" Score="5" ViewCount="412" Body="&lt;p&gt;Earlier it was considered that frozen coefficients method for Neumann stability analysis for finite difference scheme is more heuristic than rigorous. But I have read some information in a book by Gustafsson,Kreiss,Oliger that there are some conditions, which give us possibility to use frozen coefficients+Neumann analysis at the level of full rigor. As a book didn't give any reference, could you help me to find these conditions?&lt;/p&gt;&#xA;" OwnerUserId="4676" LastActivityDate="2016-12-30T22:19:34.543" Title="Frozen coefficient method (von Neumann stability analysis)" Tags="|pde|numerical-analysis|stability|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16143" PostTypeId="2" ParentId="16138" CreationDate="2014-11-11T01:22:44.320" Score="2" Body="&lt;p&gt;The thing to realize is that solving the eigenvalue problem&#xA;$$&#xA;  Ax = \lambda B x,&#xA;$$&#xA;for which the largest eigenvalue is given by&#xA;$$&#xA;  \lambda_\text{max} = \max_x \frac{x^T A x}{x^T B x},&#xA;$$&#xA;is entirely equivalent to solving the regular eigenvalue problem&#xA;$$&#xA;  B^{-T/2} A B^{-1/2} x = \lambda x.&#xA;$$&#xA;You can see that by writing&#xA;$$&#xA;  \lambda_\text{max}&#xA;  = \max_x \frac{x^T A x}{x^T B x}&#xA;  = \max_x \frac{x^T A x}{(B^{1/2}x)^T (B^{1/2} x)}&#xA;  = \max_{y=B^{1/2}x} \frac{y^T B^{-T/2} A B^{-1/2} y}{y^T y}.&#xA;$$&#xA;What this means is that if you know how to do the deflation for the matrix $A$, then you just need to apply it now for the matrix $B^{-T/2} A B^{-1/2}$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-11T01:22:44.320" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16145" PostTypeId="1" AcceptedAnswerId="16218" CreationDate="2014-11-11T10:51:40.943" Score="1" ViewCount="2174" Body="&lt;p&gt;I have a tree data structure (rooted, unbalanced, with unbounded branching factor), where each individual node has an associated 'weight'. For every node $n$ in the tree, I'd like to compute the cumulative weights of all the nodes in the subtree rooted at node $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the tree below I'd want to compute the numbers in brackets:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A: 4 (27)&#xA;|&#xA;+--B: 1 (12)&#xA;|  |&#xA;|  +--D: 5 (9)&#xA;|  |  |&#xA;|  |  +--I: 3 (3)&#xA;|  |  |&#xA;|  |  `--J: 1 (1)&#xA;|  |&#xA;|  `--E: 2 (2)&#xA;|&#xA;`--C: 2 (11)&#xA;   |&#xA;   +--F: 4 (4)&#xA;   |&#xA;   +--G: 2 (2)&#xA;   |&#xA;   `--H: 3 (3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Starting at the root node, how can I compute these whilst visiting each node a minimum number of times?&lt;/p&gt;&#xA;" OwnerUserId="7993" LastEditorUserId="7993" LastEditDate="2014-11-11T10:57:32.533" LastActivityDate="2014-11-18T15:55:49.540" Title="Efficient way to compute the cumulative weights of all subtrees rooted at each node in a tree?" Tags="|algorithms|graph-theory|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16146" PostTypeId="2" ParentId="16135" CreationDate="2014-11-11T11:13:58.427" Score="2" Body="&lt;p&gt;As a starting point, &quot;method of manufactured solutions&quot; can be used to check the validity of your code. For information regarding this: &lt;a href=&quot;http://www.innovative-cfd.com/manufactured-solutions.html&quot; rel=&quot;nofollow&quot;&gt;http://www.innovative-cfd.com/manufactured-solutions.html&lt;/a&gt;, or you can just google about manufactured solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Roughly, CFL condition scales as $h^2$ instead of $h$ if you use  artificial diffusion terms. For DG methods, it is typically given by:&#xA;$$&#xA;\Delta t \sim \frac{1}{\lambda_{max}\frac{N^2}{h} + ||\nu||_{L^\infty}\frac{N^4}{h^2}} &#xA;$$&#xA;where N is the approximation polynomial degree. &#xA;For reference:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-0-387-72065-4&quot; rel=&quot;nofollow&quot;&gt;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-0-387-72065-4&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.researchgate.net/publication/48208658_Viscous_Shock_Capturing_in_a_Time-Explicit_Discontinuous_Galerkin_Method&quot; rel=&quot;nofollow&quot;&gt;http://www.researchgate.net/publication/48208658_Viscous_Shock_Capturing_in_a_Time-Explicit_Discontinuous_Galerkin_Method&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Boundary conditions question needs to be more concrete I think for the community to help. &lt;/p&gt;&#xA;" OwnerUserId="11644" LastActivityDate="2014-11-11T11:13:58.427" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16147" PostTypeId="1" CreationDate="2014-11-11T13:43:44.810" Score="1" ViewCount="108" Body="&lt;p&gt;The PETSC function &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecSwap.html&quot; rel=&quot;nofollow&quot;&gt;VecSwap&lt;/a&gt; after a few layers of abstraction &lt;a href=&quot;https://bitbucket.org/petsc/petsc/src/f896727a0c6fbe98ba9ae82833ab3817fcf5697d/src/vec/vec/impls/seq/bvec2.c?at=master#cl-491&quot; rel=&quot;nofollow&quot;&gt;calls&lt;/a&gt; the BLAS primitive &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/db/dd4/dswap_8f_source.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;swap&lt;/code&gt;&lt;/a&gt;, which  essentially copies data around (unless I am missing something).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that there should be a O(1) implementation that can be obtained just by exchanging pointers, at least if the vectors are identical in all their characteristics (underlying storage and distribution between the nodes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, &lt;strong&gt;is there a way to swap vectors without a deep copy in PETSC?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-11-11T18:53:08.417" Title="Efficiently swap vectors in PETSC" Tags="|parallel-computing|petsc|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16149" PostTypeId="2" ParentId="16147" CreationDate="2014-11-11T18:53:08.417" Score="2" Body="&lt;p&gt;What if one Vec was created by &lt;code&gt;VecCreateSeqWithArray&lt;/code&gt; and the other not?  How would the user know which Vec holds the memory they allocated?  There is a dirty backdoor in &lt;code&gt;VecPlaceArray&lt;/code&gt;, but using it is typically bad design.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2014-11-11T18:53:08.417" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16150" PostTypeId="1" CreationDate="2014-11-11T19:38:42.853" Score="17" ViewCount="11626" Body="&lt;p&gt;I'm a Mechanical Engineering student interested in the field of aerospace engineering where, I'm told, Fortran is still commonly used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which version of Fortran should I invest my time to learn?&lt;/p&gt;&#xA;" OwnerUserId="11759" LastActivityDate="2015-07-29T23:27:03.263" Title="Which version of Fortran should I learn?" Tags="|fortran|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16151" PostTypeId="2" ParentId="16150" CreationDate="2014-11-11T20:19:14.807" Score="27" Body="&lt;p&gt;It's a bit of a popular misnomer that there is a &quot;version&quot; of Fortran to know. With rare exception, the latest Fortran standards (and compilers) retain excellent backwards compatibility with older standards. This is with good reason: not many people would use Fortran today if it weren't for the large amounts of legacy code still in use. That is to say, a standards-compliant Fortran77 code will almost invariably still work with the latest compilers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should learn to use the most modern version of the language available. F2008 might not have added anything terrible useful to a beginner, but F2003's introduction of object-oriented concepts is definitely a big deal. &quot;Modern Fortran Explained&quot; is a decent place to start, and &quot;Scientific Software Design: The Object Oriented Way&quot; is also pretty good for actually learning to put it into practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pragmatically, if you're going to be working on legacy code (say F77 or earlier), at some point you'll have to learn some of the things they did because of a lack of F90+ features. For example, if you're writing Fortran code for something, you should never use what's called a &quot;COMMON block&quot;, but you will invariably see them come up in F77 code you might have to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would never recommend someone start working in Fortran without at the very least the F90/95 feature sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, if you're not well versed in programming in some other language, you would probably find a consensus that Fortran isn't the best language to learn as your first, even if you're going after aerospace. (And I say that as someone in aerospace that uses Fortran regularly and had it as my first language).&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2014-11-11T20:31:51.600" LastActivityDate="2014-11-11T20:31:51.600" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16152" PostTypeId="1" CreationDate="2014-11-11T20:48:05.107" Score="3" ViewCount="168" Body="&lt;p&gt;I am trying to validate that the &lt;a href=&quot;https://en.wikipedia.org/wiki/Schwarz%E2%80%93Christoffel_mapping&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schwartz-Christoffel mapping&lt;/a&gt; does indeed take the upper-half plane $\mathbb{H} = \{ z: \mathrm{Re}(z) &amp;gt; 0\}$  to a polygon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This involves integrals of functions of the type and computing at every point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(z) = \int_0^z  \frac{dx}{\sqrt{(x-a)(x-b)}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a &amp;lt; b \in \mathbb{R}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like the case $a = 1,b=-1$.  I know this integral can be computed exactly, but not if the denominator $P(z)$ were a cubic or quartic like $x^4 - 1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a numpy script I wrote... sorry if it's wrong.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;impott matplotlib.pyplot as plt&#xA;&#xA;f = lambda z: np.cumsum(z, axis=1) + 1j*np.cumsum(z, axis=0)    &#xA;n = 200&#xA;dt = 1.0/n&#xA;y = (np.ones((n,2*n)))/n&#xA;y = f(y)-1&#xA;w = y[(y.imag &amp;gt; 0.001)] # semicircle of radius 1&#xA;&#xA;g = lambda t: np.sqrt(1 - t**2)**-0.5 # function to be integrated&#xA;&#xA;plt.plot((g(w)).real, g(w).imag, 'b.')&#xA;&#xA;plt.axis(&quot;Equal&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the picture I came up with... defintely not a polygon.  Maybe I have to change the original mesh?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/uwz5L.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I don't want to compute the antiderivative exactly, since I would like it to work for more complex examples like $\int_0^z \frac{dx}{\sqrt{(x-1)(x-2)(x-3)}}$.  Instead I proposed two strategy:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;build a mesh representing values of $\frac{1}{\sqrt{P(z)}}$ &lt;/li&gt;&#xA;&lt;li&gt;transform the mesh into the values of $\int_0^z$ at each point&lt;/li&gt;&#xA;&lt;li&gt;if the roots of $P(z)$ are real and we stick to $\mathrm{Re}(z)&amp;gt; 0$ the square-root is still single-valued at least&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="4006" LastEditDate="2014-11-12T11:10:16.783" LastActivityDate="2014-11-12T11:10:16.783" Title="Riemann surfaces: computing $f(z) = \int_0^z \frac{dx}{\sqrt{P(x)}}$" Tags="|numerical-analysis|computational-geometry|numpy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16153" PostTypeId="1" AcceptedAnswerId="16156" CreationDate="2014-11-11T22:57:49.220" Score="1" ViewCount="239" Body="&lt;p&gt;I was reading through &lt;a href=&quot;http://caminos.udc.es/info/asignaturas/201/Finite%20Volumes%20IMWE.pdf&quot; rel=&quot;nofollow&quot;&gt;this document&lt;/a&gt; about FVM. I understood all up to the point where we have on page 15 the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\bar u^{n+1}_i - \bar u^n_i) \Delta x + \int^{t^{n+1}}_{t^n} f(u(x_{i+1/2},t))-f(u(x_{i-1/2},t)) dt = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How am I suppose to compute the integral from $t_n$ to $t_{n+1}$ of the flux?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2020-11-19T13:20:50.430" Title="Finite Volume Method flux integration" Tags="|fluid-dynamics|finite-volume|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16154" PostTypeId="2" ParentId="12901" CreationDate="2014-11-11T23:38:22.817" Score="2" Body="&lt;p&gt;Yes, I've used tetgen to do precisely this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're filling a surface mesh, you'll have to generate facet triangulations first. Then you can decide whether or not you want a constrained or conforming tetrahedral mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result of either choice is described in better detail within &lt;a href=&quot;http://wias-berlin.de/software/tetgen/1.5/doc/manual/manual002.html&quot; rel=&quot;nofollow&quot;&gt;the manual&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11765" LastActivityDate="2014-11-11T23:38:22.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16155" PostTypeId="1" AcceptedAnswerId="16176" CreationDate="2014-11-11T23:54:48.353" Score="2" ViewCount="778" Body="&lt;p&gt;I am new to numerical methods and I cannot think of a way to start solving the following problem. I know the mantissa, base and the boundaries are related when defining a number but I cannot really put it in together. The question is as follows and I need to know how these terms are related and how to approach a question regarding these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the floating point number set $\mathbb F \subset \mathbb R $ such that $\mathbb F( \beta,t,L,U)$. Here $\beta$ is the base, $t$ is the number of digits in the mantissa and $(L,U)$ is the range of variation of the exponent. Show that the set $\mathbb F$ contains precisely $2( \beta -1 )$ $\beta ^{t-1} (U-L+1)$ elements.&lt;/p&gt;&#xA;" OwnerUserId="11767" LastEditorUserId="549" LastEditDate="2014-11-12T16:09:14.897" LastActivityDate="2014-11-21T02:52:23.697" Title="On the cardinality of the set of numbers representable by a floating point system" Tags="|numerical-analysis|floating-point|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16156" PostTypeId="2" ParentId="16153" CreationDate="2014-11-12T03:14:38.327" Score="6" Body="&lt;p&gt;You can't. You need to approximate the integral via quadrature and or other approximations. See the following section in the document you cite.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-12T03:14:38.327" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16157" PostTypeId="2" ParentId="16152" CreationDate="2014-11-12T05:42:23.763" Score="2" Body="&lt;p&gt;Note that the integral is&#xA;$$ \int^z \frac{dx}{\sqrt{x-a}\sqrt{x-b}}, $$&#xA;and in general for complex variables&#xA;$$ \sqrt{x-a}\sqrt{x-b} \not\equiv \sqrt{(x-a)(x-b)}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what you were trying to do with your code, so I can't help you there. I don't see where in your code you do the integration, it seems to plot some other function entirely: the issue is not with the mesh. When I did your example in Mathematica, $a=-b=1$, and I got this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-((I \[Pi])/2) + 2 ArcSinh[Sqrt[-1 + z]/Sqrt[2]] /. z -&amp;gt; x + I y;&#xA;ParametricPlot[{Re@%, Im@%}, {x, -8, 8}, {y, 0, 8}, PlotRange -&amp;gt; {{-1, 4}, {-2, 2}}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TROc9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-11-12T05:42:23.763" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16158" PostTypeId="2" ParentId="16130" CreationDate="2014-11-12T06:15:50.687" Score="1" Body="&lt;p&gt;Although this equation does not have diffusion, the numerical algorithm for solving the advection equation inherently has a certain diffusion. In fact, there is no numerical method without numerical diffusion. Some algorithms have more numerical diffusion, some less, and there are methods to constrain the spread of discontinuities. But in principle, the numerical diffusion is inevitable.&#xA;One way to see this, is to do the following exercise:&lt;img src=&quot;https://i.stack.imgur.com/H5d7x.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11772" LastActivityDate="2014-11-12T06:15:50.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16161" PostTypeId="1" AcceptedAnswerId="16163" CreationDate="2014-11-12T15:54:25.583" Score="2" ViewCount="351" Body="&lt;p&gt;I am trying to write a simple finite volume method code but there are some concepts I'm still not really getting right (perhaps I'm overcomplicating things)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a uniform grid, the idea is to approximate the solution by computing cell averages as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar u(x_j,t^n) = \bar u_j^n = \frac{1}{h}\int^{x_{j+1/2}}_{x_{j-1/2}} u(x,t^n)dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When implementing the cell average computation in my code, I suppose I can use any quadrature, is this right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, at $t=0$, I know the values for every $x_{j+1/2}$ and compute can compute $\bar u_j^0$, $\bar u_{j+1}^0$, $\bar u_{j-1}^0$ which in turn I can use to compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar u_j^{n+1} = \bar u_j^{n} - \frac{\Delta t}{\Delta x}\Big[ F(\bar u_j^{n},\bar u_{j+1}^{n})-F(\bar u_{j-1}^{n},\bar u_j^{n}) \Big]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;given the numerical flux $F(\cdot,\cdot )$If I understood correctly, everything should be fine up to this point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for the next iteration do I need to recompute the averages for each cell? If so I don't see where to get the values for each $x_{j+1/2}$.&#xA;Or should I just use the new computed values, i.e. $\bar u_{j-1}^{n+1}$,$\bar u_j^{n+1}$,$\bar u_{j+1}^{n+1}$ , to iterate over the scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if I missing something else in here, any thoughts?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2014-11-12T16:37:21.313" Title="Finite volume method implementation issues" Tags="|finite-volume|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16162" PostTypeId="1" AcceptedAnswerId="16165" CreationDate="2014-11-12T16:03:23.753" Score="2" ViewCount="101" Body="&lt;p&gt;I'm analyzing experimental data, and I've produced graphs that look similar to the attached sketch. When I look at the graph, I see structure in the connections (connections tend to be local, there is one central collection of nodes and three branches).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if anyone can point me toward areas of graph theory that might help me to quantify this structure. Note that there are sometimes spurious edges connecting distant parts of the graph (such edges are rare and no examples are shown in the sketch), so I'm hoping to find methods that can tolerate such things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Oyl2v.jpg&quot; alt=&quot;example graph&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7491" LastActivityDate="2014-11-13T16:52:58.637" Title="graph theory operations to explore structure of a graph" Tags="|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16163" PostTypeId="2" ParentId="16161" CreationDate="2014-11-12T16:37:21.313" Score="1" Body="&lt;p&gt;There is a small mistake in your finite volume formulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\bar u_j^{n+1} = \bar u_j^{n} - \frac{\Delta t}{\Delta x}\Big[ F( u_{j+\frac{1}{2}}^{n-},u_{j+\frac{1}{2}}^{n+})-F( u_{j-\frac{1}{2}}^{n-},u_{j-\frac{1}{2}}^{n+}) \Big]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can evolve the cell averages using the above equations. But if you notice, the flux terms requires the values for the cell boundaries and for this you need to use a reconstruction procedure. So, in essence, you are recomputing the cell boundary values at every time step and not the cell averages. The cell averages are computed using the iteration procedure you just mentioned. The cell boundary values are usually computed using the averages of the cells and it's neighbors. Different reconstruction procedures result in different schemes. &lt;/p&gt;&#xA;" OwnerUserId="11644" LastActivityDate="2014-11-12T16:37:21.313" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="16164" PostTypeId="2" ParentId="16078" CreationDate="2014-11-12T18:44:23.020" Score="3" Body="&lt;p&gt;The C++ library mlpack (&lt;a href=&quot;http://www.mlpack.org&quot; rel=&quot;nofollow&quot;&gt;http://www.mlpack.org&lt;/a&gt;) currently contains an implementation of a handful of tree types; of the most interest is probably the cover tree (see &quot;Cover trees for nearest neighbor&quot;, Beygelzimer, Kakade, and Langford, ICML 2006).  The cover tree will work with any metric that satisfies the triangle inequality.  This is not true of the $kd$-tree, the octree (and variants), or the M tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the Jaccard distance does satisfy the triangle inequality, you can easily write a JaccardDistance class with an Evaluate() function that evaluates the Jaccard distance between two points.  Then you can build a cover tree on the points using the Jaccard distance and run exact $k$ nearest neighbors easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this tutorial: &lt;a href=&quot;http://www.mlpack.org/doxygen.php?doc=nstutorial.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mlpack.org/doxygen.php?doc=nstutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given some JaccardDistance class, then, you can write code that looks a bit like this...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using namespace mlpack::neighbor;&#xA;using namespace mlpack::tree;&#xA;&#xA;extern arma::mat dataset; // The dataset containing the points.&#xA;extern size_t k; // The number of neighbors being searched for.&#xA;NeighborSearch&amp;lt;NearestNeighborSort, JaccardDistance, CoverTree&amp;lt;JaccardDistance&amp;gt; &amp;gt; ns(dataset);&#xA;&#xA;arma::Col&amp;lt;size_t&amp;gt; neighbors; // Will store the neighbors of each point.&#xA;arma::mat distances; // Will store the distances to those neighbors.&#xA;&#xA;ns.Search(k, neighbors, distances);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I haven't tested that or anything; that's just a sketch.  But it should be enough to get you started on a fast algorithm to do nearest neighbor search with the Jaccard distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be possible to adapt the mlpack $kd$-tree implementation to work with Jaccard distances too, but it would not be as straightforward, which is why I recommend the cover tree for the sake of simplicity.&lt;/p&gt;&#xA;" OwnerUserId="10415" LastActivityDate="2014-11-12T18:44:23.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16165" PostTypeId="2" ParentId="16162" CreationDate="2014-11-13T00:06:14.800" Score="3" Body="&lt;p&gt;There are a handful of common metrics when analyzing graphs and networks.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Clustering_coefficient&quot; rel=&quot;nofollow&quot;&gt;Clustering coefficient&lt;/a&gt;: given a vertex $i$ in the graph and two neighbors $j$, $k$ of $i$, what are the odds that $j$ and $k$ are also connected? Analogy: knowing that Gertrude Stein and Ernest Hemingway are both friends of mine, what are the chances that they're friends with each other?&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Degree_distribution&quot; rel=&quot;nofollow&quot;&gt;Degree distribution&lt;/a&gt;: plot a histogram of the degrees of all the nodes in the graph. What does it look like? For example, internet sites with connectivity defined by links have a degree distribution such that $P[\textrm{degree}(i) = k] \propto k^{-\gamma}$ for some constant $\gamma$; this is called a &lt;em&gt;scale-free&lt;/em&gt; network. By contrast, Erdos-Renyi graphs' degrees are Poisson-distributed.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Average_path_length&quot; rel=&quot;nofollow&quot;&gt;Breadth&lt;/a&gt;: on average, how long does it take to get from one arbitrary vertex to another? In many networks (social, neural, ...), the distance is proportional to $\log n$ for a network of size $n$.&lt;/li&gt;&#xA;&lt;li&gt;Robustness: on average, how many edges do you have to remove at random for your network to become disconnected? On average, how many connected components can your graph be split into if you get to choose $k$ edges to remove?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This list is, of course, not exhaustive. I also highly recommend using software like &lt;a href=&quot;http://www.graphviz.org/&quot; rel=&quot;nofollow&quot;&gt;Graphviz&lt;/a&gt; to make graph drawings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things to google: network theory, small world network, Watts-Strogatz model, Barabasi-Albert model, scale-free network, Erdos-Renyi model, block two-level Erdos-Renyi model.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2014-11-13T16:52:58.637" LastActivityDate="2014-11-13T16:52:58.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16166" PostTypeId="1" CreationDate="2014-11-13T04:18:52.080" Score="0" ViewCount="457" Body="&lt;p&gt;I originally asked this question at StackOverflow, and was suggested to bring it here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen &lt;a href=&quot;https://stackoverflow.com/questions/3550114/how-to-store-molecules-in-memory&quot;&gt;this question about the representation of molecules in memory&lt;/a&gt;, and it makes sense to me (tl;dr represent it as a graph with atoms as nodes and bonds as edges). But now my question is this: how do we check and see if two molecules are equal? This could be generalized as &lt;strong&gt;how can we check equality of (acyclic) graphs?&lt;/strong&gt; For now we'll ignore &lt;a href=&quot;http://en.wikipedia.org/wiki/Stereoisomerism&quot; rel=&quot;nofollow noreferrer&quot;&gt;stereoisomers&lt;/a&gt; and cyclical structures, such as the carbon ring in the example given in the first link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a more detailed description of my problem: For my &lt;code&gt;Molecule&lt;/code&gt; class (as of now), I intend to have an array of &lt;code&gt;Atom&lt;/code&gt;s and an array of &lt;code&gt;Bond&lt;/code&gt;s. Each &lt;code&gt;Bond&lt;/code&gt; will point to the two &lt;code&gt;Atom&lt;/code&gt;s at either end, and will have a weight (i.e., the number of chemical bonds in that edge). In other words, this will most closely resemble an edge list graph. My first guess is to iterate over the &lt;code&gt;Atom&lt;/code&gt;s in one molecule and try to find corresponding &lt;code&gt;Atom&lt;/code&gt;s in the other molecule based on the &lt;code&gt;Bond&lt;/code&gt;s that contain that &lt;code&gt;Atom&lt;/code&gt;, but this is a rather naive approach, and the complexity seems pretty large (best guess is close to &lt;em&gt;O(n!)&lt;/em&gt;. Yikes.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of complexity, this approach seems like it would work in most cases, however it seems to break down for some molecules. Take these for example (notice the different location of the OH group):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    H   H   H   OH  H&#xA;    |   |   |   |   |&#xA;H - C - C - C - C - C - H (2-Pentanol)&#xA;    |   |   |   |   |&#xA;    H   H   H   H   H&#xA;&#xA;    H   H   OH  H   H&#xA;    |   |   |   |   |&#xA;H - C - C - C - C - C - H (3-Pentanol)&#xA;    |   |   |   |   |&#xA;    H   H   H   H   H&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we examine these molecules, for each atom in one molecule there is a unique same-element atom in the other molecule that has the same number and types of bonds, but these two molecules are clearly not the same, nor are they stereoisomers (which I'm not considering now). Instead they are &lt;a href=&quot;http://en.wikipedia.org/wiki/Structural_isomer&quot; rel=&quot;nofollow noreferrer&quot;&gt;structural isomers&lt;/a&gt;. Is there a way that we can check this relative structure as well? Would this be easier with an adjacency list instead of an edge list? Are there any graph equality algorithms out there that I should look into (ideally in Java)? I've looked a bit into &lt;a href=&quot;http://en.wikipedia.org/wiki/Graph_canonization&quot; rel=&quot;nofollow noreferrer&quot;&gt;graph canonization&lt;/a&gt;, but this seems like it could be NP-hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the &lt;a href=&quot;http://en.wikipedia.org/wiki/Graph_isomorphism&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graph Isomorphism Problem Wikipedia Article&lt;/a&gt;, it seems as if graphs with bounded degree have polynomial time solutions to this problem. Furthermore, planar graphs also have polynomial solutions (i.e., the edges only intersect at their endpoints). It seems to me that (at least most) molecules satisfy both of these conditions, so what is this polynomial-time solution to this problem, or where can I find it? My Google searches are letting me down this time.&lt;/p&gt;&#xA;" OwnerUserId="11785" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-11-21T20:49:32.167" Title="How to represent molecules and compare equality" Tags="|graph-theory|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16167" PostTypeId="2" ParentId="16166" CreationDate="2014-11-13T05:27:03.693" Score="1" Body="&lt;p&gt;I had to work on graph isomorphisms at one point for a parallel computing project involving a computational chemistry package, and from what I remember, I basically just used off-the-shelf software. What you're looking for is a graph isomorphism package that respects node/vertex labeling (i.e., nodes are labeled as types of atoms, e.g., carbon, hydrogen, oxygen) and edge/arc labeling (i.e., edges are labeled as a type of bond, e.g., single, double, triple). I forget exactly which software package I used, but I think &lt;a href=&quot;http://pallini.di.uniroma1.it/&quot; rel=&quot;nofollow&quot;&gt;nauty&lt;/a&gt; was one of the packages I looked at, and there are a couple others out there.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-13T05:27:03.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16168" PostTypeId="2" ParentId="16139" CreationDate="2014-11-13T08:17:11.047" Score="5" Body="&lt;p&gt;CVX is not a symbolic analysis engine. Every single subexpression that you construct must satisfy the &lt;a href=&quot;http://web.cvxr.com/cvx/doc/dcp.html&quot; rel=&quot;noreferrer&quot;&gt;DCP ruleset&lt;/a&gt;. It does not even &lt;em&gt;attempt&lt;/em&gt; to consider the entire expression globally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in your case, once CVX sees &lt;code&gt;(A-Ypie)*(A-Ypie)'&lt;/code&gt;, it declares a DCP error. This is proper behavior even though the &lt;em&gt;trace&lt;/em&gt; of this quantity is indeed convex. After all, only the diagonal elements of this expression are convex; the rest are non-convex. You're never allowed to construct a non-convex (or non-concave) expression in CVX, even if you end up throwing it away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fix this, you have to recognize that this trace quantity is actually the square of the Frobenius norm of &lt;code&gt;A-Ypie&lt;/code&gt;. So &lt;code&gt;sum_square(A(:)-Ypie(:))&lt;/code&gt; will do, as will &lt;code&gt;square_pos(norm(A-Ypie,'fro'))&lt;/code&gt;, or (in CVX 3.0 beta only) &lt;code&gt;norm(A-Ypie,'fro')^2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, Johan's comment is entirely correct as well, and it's quite possible that none of this advice will apply once you have fixed your model to have a scalar objective.&lt;/p&gt;&#xA;" OwnerUserId="3907" LastEditorUserId="3907" LastEditDate="2014-11-13T13:37:58.333" LastActivityDate="2014-11-13T13:37:58.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16170" PostTypeId="2" ParentId="16022" CreationDate="2014-11-13T08:52:45.397" Score="0" Body="&lt;p&gt;An interesting method to find &quot;best optimization algorithms&quot; for a class of test instances is the iRace (resp. F-Race) approach, for which an &lt;a href=&quot;http://iridia.ulb.ac.be/irace/&quot; rel=&quot;nofollow&quot;&gt;R-package exists&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11687" LastActivityDate="2014-11-13T08:52:45.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16173" PostTypeId="1" CreationDate="2014-11-13T18:48:46.160" Score="1" ViewCount="827" Body="&lt;p&gt;I want to use it in Matlab or Java. Will these two languages be much slower for computing the algorithm compared to C, C++, in case efficiency is an important factor?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of that there's a PROPACK and SVDPACK, etc. Seems quite old. What are some popular choices? Are there any &lt;strong&gt;paralleled versions&lt;/strong&gt; existing? If there's no paralleled versions, I plan to make an appropriate sequential version to be parallelized (multiple threads). Parallelization is important for my project since it targets on big datasets. So I suppose I need to modify a chosen codes directly (multithreaded parallelization on the level of vector computation).&lt;/p&gt;&#xA;" OwnerUserId="7969" LastEditorUserId="7969" LastEditDate="2014-11-14T02:57:15.947" LastActivityDate="2015-01-06T08:55:25.967" Title="Are there any popular (paralleled) implementations of Lanczos methods for SVD/eigendecomposion?" Tags="|parallel-computing|matrix|svd|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16174" PostTypeId="1" CreationDate="2014-11-13T22:04:40.660" Score="1" ViewCount="114" Body="&lt;p&gt;I want to compute a few tens of the smallest eigenvalues of a linear system which is a discretization of a linear elasticity. In the presence of additional constraints like Dirichlet boundary conditions, does it help to use the rigid body displacements to deflate the linear system? In that case they might only be approximations of zero eigenmodes, right? Thank you&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="549" LastEditDate="2014-11-13T23:24:19.530" LastActivityDate="2014-11-13T23:24:19.530" Title="smallest eigenvalues for linear elasticity" Tags="|matrix|eigenvalues|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16175" PostTypeId="2" ParentId="16173" CreationDate="2014-11-13T22:10:16.000" Score="2" Body="&lt;p&gt;So, in theory, you could use &lt;a href=&quot;http://www.grycap.upv.es/slepc/&quot; rel=&quot;nofollow&quot;&gt;SLEPc&lt;/a&gt;'s MATLAB interface. Doing so means you can't use any of the MPI-based parallelism, if it's anything like PETSc's MATLAB interface. There's a fair amount to install (PETSc, whatever packages you want to configure with PETSc, plus whatever eigensolvers you want to bundle with SLEPc), but it does give you a single interface from which you can use a variety of methods (e.g., thick restart Lanczos via TRLAN, implicitly restarted Arnoldi via ARPACK, Krylov-Schur), plus a large number of preconditioners. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-13T22:10:16.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16176" PostTypeId="2" ParentId="16155" CreationDate="2014-11-13T22:11:06.200" Score="1" Body="&lt;p&gt;I found the solution to my own problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of values the exponent can take = (U-L+1) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sign can take 2 values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each cell in mantissa can take $\beta$ values except for the first cell which takes only ( $\beta$ -1) values (since it cannot hold '0' in it)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\therefore$ the number of elements in the set $\mathbb F$ = (U-L+1)&lt;em&gt;2&lt;/em&gt;($\beta-1)\beta_{(1)} \beta_{(2)}\beta_{(3)}....\beta_{(t-1)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$=2(\beta-1) \beta^{t-1}(U-L+1)$$&lt;/p&gt;&#xA;" OwnerUserId="11767" LastActivityDate="2014-11-13T22:11:06.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16178" PostTypeId="1" CreationDate="2014-11-14T14:15:50.767" Score="4" ViewCount="2403" Body="&lt;p&gt;I am trying to apply the &quot;restoring force surface&quot; method to a dynamic linear system.&#xA;The idea behind this method is that, knowing acceleration, displacement, velocity and input force it is possible to establish if the system under study is linear or not by plotting a surface in the phase-space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To verify my code I applied the method to a system I already know, therefore I generated the needed data ($\ddot{y},\dot{y},y$) solving the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \ddot{y} + 40 \dot{y} + 10^{4} y = f(t) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $f(t) = 30 \sin(10 t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code I wrote to do that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[t,u] = ode45('odefun',[0 60],[0 0]);&#xA;&#xA;vel = u(:,2);&#xA;spost = u(:,1);&#xA;f_input = 30*sin(10*t);&#xA;&#xA;acc = zeros(size(vel)); % obtain the acceleration from the velocity&#xA;acc(1) = 0;&#xA;for i = 2:length(vel)&#xA;    h = t(i)-t(i-1);&#xA;    acc(i) = (vel(i)-vel(i-1))/h; &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the $odefun$ is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [F] = odefun(t,y)&#xA;&#xA;F = zeros(size(y));&#xA;&#xA;F(1) = y(2);&#xA;F(2) = -10^4*y(1)-40*y(2)+30*sin(10*t);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the application of the method I have to compute the so called restoring force as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ z(y,\dot{y}) = f(t) - \ddot{y} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;therefore I can plot the triplet $(y,\dot{y},z)$ as a surface where $y$ and $\dot{y}$ represent a point in the phase-space and $z$ the height above that point.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Surface&#xA;x_plot = linspace(min(spost),max(spost),200);&#xA;y_plot = linspace(min(vel),max(vel),200);&#xA;&#xA;zgrid = gridfit(sort(spost),sort(vel),sort(z), x_plot,y_plot); &#xA;% trisurf should work as well instead of gridfit&#xA;% sort() used not to have oscillatory data&#xA;&#xA;figure()&#xA;surf(x_plot,y_plot,zgrid)&#xA;title('Restoring force surface')&#xA;xlabel('displacement')&#xA;ylabel('velocity')&#xA;zlabel('force')    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once I have obtained the surface I need to extract from it the cross sections: one for $y = 0$ and another one for $\dot{y} = 0$. The slope of the cross section at $\dot{y} = 0$ should represent the stiffness of the system (therefore it should be $k = 10^4$), whereas the slope of the cross section obatined at $y = 0$ should represent the damping ($ = 40$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But here I have problems, infact, since the system is known to be linear (the coefficients in the equation are constants), I expect to have straight lines as cross sections whereas the plot of $velocity-force$ is not a straight line as it should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To exctract a cross-section I saved the values of $y$ and $z$ corresponding to $|\dot{y}| &amp;lt; \delta$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Surface section to plot force-displacement&#xA;delta_v = 0.0001;&#xA;xx = zeros(length(spost),1);&#xA;fx = zeros(length(z),1);&#xA;for i = 1:length(vel)&#xA;    if abs(vel(i)) &amp;lt;= delta_v&#xA;        xx(i,1) = spost(i);&#xA;        fx(i,1) = z(i);&#xA;    else&#xA;        xx(i,1) = 999; % Just to give some value to be discarded later on&#xA;        fx(i,1) = 999;&#xA;    end&#xA;end&#xA;xx(xx == 999) = []; % Elimination of points outside the range [-delta_v +delta_v]]&#xA;fx(fx == 999) = [];&#xA;&#xA;x_sort = sort(xx);&#xA;fx_sort = sort(fx);&#xA;&#xA;figure()&#xA;plot(x_sort,fx_sort)&#xA;xlabel('displacement')&#xA;ylabel('force')&#xA;&#xA;[b,bint] = polyfit(x_sort,fx_sort,1);&#xA;&#xA;err_K = 10^4-b(1); % It should be 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The same for the $y = 0$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;delta_s = 0.01;&#xA;xx_dot = zeros(length(vel),1);&#xA;fv = zeros(length(z),1);&#xA;for i = 1:length(spost)&#xA;    if abs(spost(i)) &amp;lt;= delta_s&#xA;        xx_dot(i,1) = vel(i);&#xA;        fv(i,1) = z(i);        &#xA;    else&#xA;        xx_dot(i,1) = 999;&#xA;        fv(i,1) = 999;&#xA;    end&#xA;end&#xA;&#xA;xx_dot(xx_dot == 999) = [];&#xA;fv(fv == 999) = [];&#xA;&#xA;x_dot_sort = sort(xx_dot);&#xA;fv_sort = sort(fv);&#xA;&#xA;figure()&#xA;plot(x_dot_sort,fv_sort)&#xA;xlabel('velocity')&#xA;ylabel('force')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This last plot is not even a straight line and this means the damping is not linear but I know it must be linear since I started from a linear equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="11676" LastEditDate="2014-11-15T12:01:55.113" LastActivityDate="2024-04-03T07:08:00.977" Title="2D cross section from 3D surface" Tags="|matlab|ode|plotting|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16179" PostTypeId="1" AcceptedAnswerId="16194" CreationDate="2014-11-14T15:26:07.837" Score="1" ViewCount="112" Body="&lt;p&gt;The set-up is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have $K$ finite sets of real numbers, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. sets $G_i, i=1 \dotsc, K$ and $|G_i| = n_i &amp;lt; \infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, assume that we have a function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;h: \mathbb R^K \to \mathbb R&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is monotonically increasing in each component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, there is another function $$g: \mathbb R^K \to \mathbb R$$,which does not necessarily fullfill the monotonicity condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimization problem I want to solve is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find $$\max_{(x_1,\dotsc,x_K) \in G_1 \times \dotsc \times G_K} h(x_1,\dotsc, x_K)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to $$g(x_1,\dotsc, x_K) \leq c$$, where $c$ is a pre-specified constant. The naive approach takes $2\prod_{i=1}^{K}n_i$ function evaluations (evaluate $h$, check condition defined by $g$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By how much can we improve this by using the componentwise monotonicity of $h$? What if we also assume that $g$ is also increasing in each component?&lt;/p&gt;&#xA;" OwnerUserId="11508" LastActivityDate="2014-11-21T02:37:48.100" Title="Discrete optimization on a cartesian product with component-wise increasing objective function" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16181" PostTypeId="1" CreationDate="2014-11-14T19:13:05.113" Score="1" ViewCount="99" Body="&lt;p&gt;I need to sample from the generalized beta of the second kind (GB2) distribution (McDonald, 1984, Econometrica). &lt;a href=&quot;http://en.wikipedia.org/wiki/Generalized_beta_distribution#Generalized_beta_of_the_second_kind_.28GB2.29&quot; rel=&quot;nofollow&quot;&gt;Its PDF has the form&lt;/a&gt;&#xA;$$ f_{a,b,p,q}(x) = \frac{|a|x^{ap-1}}{b^{ap}B(p,q)(1+(x/b)^a)^{p+q}}. $$&#xA;Its CDF does not have a closed form, so I cannot use an inverse transform method. Does anyone know how to sample from this distribution or know of a Fortran random number generator library that contains this distribution? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of the packages in the netlib.org/random library contain the GB2.&lt;/p&gt;&#xA;" OwnerUserId="11797" LastEditorUserId="713" LastEditDate="2014-11-15T11:45:47.487" LastActivityDate="2014-11-15T19:01:52.550" Title="Sampling from the Generalized Beta of the Second Kind in Fortran" Tags="|fortran|random-number-generation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16182" PostTypeId="2" ParentId="16088" CreationDate="2014-11-15T00:59:08.210" Score="1" Body="&lt;p&gt;I believe that what you are looking for can be found in this package, patchline.m, available from Matlab Central. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/36953-patchline/content/patchline.m&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/36953-patchline/content/patchline.m&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note the basic usage as is outlined in&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;help patchline&#xA;&#xA;%Example &#xA;t = 0:pi/64:4*pi;&#xA;p = patchline(t,sin(t),'edgecolor','b','linewidth',2,'edgealpha',.05);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;'edgecolor' and 'linewidth' correspond to their obvious graphical properties, but what you may be unfamiliar with is the 'edgealpha' (valued between 0 and 1) which will determine the transparency of the lines which you are plotting.&lt;/p&gt;&#xA;" OwnerUserId="11799" LastEditorUserId="11799" LastEditDate="2014-11-15T02:35:16.320" LastActivityDate="2014-11-15T02:35:16.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16183" PostTypeId="1" CreationDate="2014-11-15T04:10:47.773" Score="8" ViewCount="806" Body="&lt;p&gt;In a recent &lt;a href=&quot;http://sinews.siam.org/DetailsPage/tabid/607/ArticleID/251/Periodic-Table-of-the-Finite-Elements.aspx&quot;&gt;SIAM News article&lt;/a&gt;, there is a long article describing a systematic organization of the finite elements, aptly dubbed the &lt;a href=&quot;http://femtable.org/&quot;&gt;Periodic Table of Finite Elements&lt;/a&gt;.  Its really quite fascinating to see how classification can be accomplished via finite element exterior calculus.  As the authors indicate: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Just as the arrangement of the chemical elements in a periodic table&#xA;  led to the discovery of new elements, the periodic table of finite&#xA;  elements has not only clarified existing elements but also highlighted&#xA;  holes in our knowledge and led to new families of finite elements&#xA;  suited for certain purposes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The analogy fascinates me, and makes me wonder if it is possible to fill all the possible &quot;holes&quot; in the same way that missing material elements have been found.  Perhaps this may be stretching the analogy too far, but I'm curious if all the possible &quot;gaps&quot; in finite elements have been fully explored and developed according to this finite element exterior calculus classification approach.  If not, what are the more important &quot;missing methods&quot; that research is currently focused on developing and why?  Furthermore, are there any finite element methods that cannot be classified by this approach (aside from the obvious omission of arbitrary shaped simplices...)?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-11-15T04:37:59.037" LastActivityDate="2014-11-15T16:40:24.637" Title="On the completeness of the Periodic Table of Finite Elements" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16184" PostTypeId="2" ParentId="16183" CreationDate="2014-11-15T06:02:08.787" Score="8" Body="&lt;p&gt;Disclaimer: I don't actually work in this field (just find it interesting), so I could be misunderstanding some of the ideas. Apologies if this happens, and please correct me if you see a mistake. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One side note - finite elements aren't quite equivalent to finite element methods. These are elements defined by Ciarlet - finite dimensional approximation spaces with degrees of freedom defined as linear functionals for space. Finite element methods can be a much broader set of discretizations (for example, stabilizations of the weak form, discrete tricks, etc). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ima.umn.edu/~arnold//publications.html&quot;&gt;Doug Arnold&lt;/a&gt; has a good sampling of current work in the vein of the SIAM table. One neat bit was the extension of this idea to the serendipity group of finite elements, which allowed him to generate a new family of 3D serendipity finite elements. Annalisa Buffa has also fit &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=4&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=0CC0QFjAD&amp;amp;url=http%3A%2F%2Fwww.imati.cnr.it%2Fannalisa%2FARTICOLI%2FBuffa-Sangalli-Rivas-Vazquez.pdf&amp;amp;ei=e99mVMuPFsTxoAT23oLIDA&amp;amp;usg=AFQjCNF_gGYoLJKi6NY4l4LUqXzyR7H6HA&amp;amp;sig2=63W94bS4uBhqaR1O_TbWVQ&amp;amp;bvm=bv.79908130,d.cGU&quot;&gt;B-spline discretizations&lt;/a&gt; into this framework of differential forms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of the above ideas involve a finite-dimensional reproduction of a De Rham complex to form &quot;compatible discretizations&quot; (the &lt;a href=&quot;https://www.ima.umn.edu/~arnold/papers/icm2002.pdf&quot;&gt;stability of mixed finite elements&lt;/a&gt; is tied to the general idea of compatibility in discretizations). Compatibility is also present in &lt;a href=&quot;http://link.springer.com/chapter/10.1007/978-3-540-78319-0_3#page-1&quot;&gt;Maxwells&lt;/a&gt; and &lt;a href=&quot;https://www.ima.umn.edu/~arnold/papers/icm2002.pdf&quot;&gt;curl-curl&lt;/a&gt; problems, where this gives stability of the method and an accurate reproduction of the operator spectrum. Outside of FEM, mimetic finite difference methods seem to be related to these concepts as well (though they're closely related to mixed FEM methods, so I'm not sure how special this is). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More recently, Arnold came up with finite elements for &lt;a href=&quot;https://www.ima.umn.edu/~arnold/papers/icm2002.pdf&quot;&gt;elasticity&lt;/a&gt; based on a separate &quot;elasticity&quot; complex, and John Evans reproduced this idea for Stokes, defining a basis for incompressible flow problems based on a &lt;a href=&quot;http://repositories.lib.utexas.edu/handle/2152/ETD-UT-2011-12-4506&quot;&gt;&quot;Stokes complex&quot;&lt;/a&gt;. If the full problem including the divergence free condition is discretized, the resulting discretization can be shown to be pointwise (not weakly) divergence free. Gerritsma and Hiemstra argue you can use the &lt;a href=&quot;http://link.springer.com/chapter/10.1007/978-3-319-01601-6_2&quot;&gt;same geometric ideas&lt;/a&gt; to construct &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999113006414&quot;&gt;high order discretizations satisfying exact conservation properties&lt;/a&gt; for a variety of conservation laws.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR - for the periodic table of FEM: exotic and nontraditional elements? For the idea of grouping FEM into families: compatible discretizations and geometric modeling of physics?&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2014-11-15T16:40:24.637" LastActivityDate="2014-11-15T16:40:24.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16185" PostTypeId="2" ParentId="11160" CreationDate="2014-11-15T09:10:08.083" Score="0" Body="&lt;p&gt;If you want to obtain the cross-section at a specific location you can use the MATLAB function interp2 in this way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = interp2(xgrid,ygrid,zgrid,x,y0);&#xA;figure()&#xA;plot(x,w)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where $xgrid$, $ygrid$ and $zgrid$ are the matrices you used to plot your surface, $x$ is the coordinate of one of the axis and $y0$ is the coordinate on the other axesat which you want to take the cross-section.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you want to plot the cross-section for $y = 0$ then you have to write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = interp2(xgrid,ygrid,zgrid,x,0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11676" LastActivityDate="2014-11-15T09:10:08.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16186" PostTypeId="1" AcceptedAnswerId="16187" CreationDate="2014-11-15T09:41:10.583" Score="0" ViewCount="341" Body="&lt;p&gt;I was checking the unity condition on a smoothing kernel for SPH, however I don't understand why the integral is not giving 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kernel function is $\displaystyle W(||\vec r||,h) = \frac{315}{64\pi h^8}(h^2-r^2)^3$ if $0 \leq ||\vec r|| \leq h$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to approximate the integral with the following matlab code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;W = @(r) (r &amp;gt;= 0) .* (r &amp;lt;= 1) * 315/(64*pi*1^4) .* (1^2-r.^2).^3;&#xA;integral(W,0,1,'AbsTol',1e-12)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $h=1$ and the result is $0.7162$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong and what is the purpose of having a kernel whose integral is unity, i.e. normalized?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="713" LastEditDate="2014-11-15T12:06:01.513" LastActivityDate="2014-11-15T12:06:01.513" Title="Why doesn't the integral of a smoothing kernel equal 1?" Tags="|matlab|sph|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16187" PostTypeId="2" ParentId="16186" CreationDate="2014-11-15T11:28:22.123" Score="2" Body="&lt;p&gt;The purpose of having a smoothing kernel with integral 1 is typically that when you have a point particle at position $x_0$ with the density function $\delta(x-x_0)$, the smoothing kernel replaces that density with $W(\|x-x_0\|,h)$. In other words, it smoothes out the Dirac delta function. Since the integral of the Dirac delta function is 1, the integral of the smoothed function also better be one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral $\int_{R^3}\delta(x-x_0)\,dx = 1$ counts the number of particles you have (just one). Choosing a kernel with the wrong integral &quot;creates&quot; or &quot;destroys&quot; particles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral is taken over the whole space, &#xA;$$ 1 = \int_{\mathbb{R}^3} W(\|x\|,h)\,dx = 4\pi \int_0^h W(r,h)\,r^2dr, $$&#xA;using spherical coordinates. In your case, the integral is&#xA;$$ 4\pi\int_0^h \frac{315}{64\pi h^8}(h^2-r^2)^3r^2\,dr = h, $$&#xA;while this should be $1$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-11-15T11:59:48.147" LastActivityDate="2014-11-15T11:59:48.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16188" PostTypeId="1" CreationDate="2014-11-15T11:28:42.173" Score="-3" ViewCount="434" Body="&lt;p&gt;I recently stumbled upon interesting site that has interactive 3D representation of radial electron distribution (atomic orbital).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is the url: &lt;a href=&quot;http://winter.group.shef.ac.uk/orbitron/AOs/1s/e-density-dots.html&quot; rel=&quot;nofollow&quot;&gt;http://winter.group.shef.ac.uk/orbitron/AOs/1s/e-density-dots.html&lt;/a&gt; (java is needed)&#xA;clicking on the &quot;equations&quot; or this link &lt;a href=&quot;http://winter.group.shef.ac.uk/orbitron/AOs/1s/equations.html&quot; rel=&quot;nofollow&quot;&gt;http://winter.group.shef.ac.uk/orbitron/AOs/1s/equations.html&lt;/a&gt; gives equations needed for these calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to somehow get the coordinates in 3D to plot the electron cloud but couldn't manage to do it and I need some help over this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea how I would use this functions to get the 3d electron cloud? Any help appreciated!&lt;/p&gt;&#xA;" OwnerUserId="11805" LastEditorUserId="9667" LastEditDate="2014-11-16T09:28:23.213" LastActivityDate="2014-11-21T02:34:56.693" Title="Solving electron density function for Hydrogen and drawing in 3D" Tags="|c++|ode|c|numerical-modelling|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16189" PostTypeId="1" AcceptedAnswerId="16191" CreationDate="2014-11-15T14:32:31.613" Score="6" ViewCount="625" Body="&lt;p&gt;I am investigating a physical process where I believe the 1-D advection-diffusion equation:&#xA;\begin{equation}&#xA;\frac{\partial u}{\partial t} = -\frac{\partial}{\partial x}[\mu(x,t) u(x,t)] + \frac{\partial}{\partial x}[D(x,t)\frac{\partial}{\partial x}u(x,t)]&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;may be applicable. I have a set of experimental data of the density, $u_{exp}(x,t)$, across $x$ for each time $t$. &#xA;To begin my investigation of this I want to assume that $D(x,t)$ is a constant,&#xA;$D(x,t)$=$D_0$, and $\mu(x,t) = \mu_0 + \tilde{\mu}(x,t) $ such that I &#xA;can approach the problem numerically as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial u}{\partial t} =&#xA;-\frac{\partial}{\partial x}[\mu(x,t)] u(x,t) &#xA;-\mu(x,t)\frac{\partial}{\partial x}[u(x,t)] &#xA;+ D_0\frac{\partial^2}&#xA;{\partial x^2}[u(x,t)] &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial u}{\partial t} =&#xA;-\frac{\partial}{\partial x}[\mu_0 + \tilde{\mu}(x,t)] u(x,t) &#xA;-[\mu_0 + \tilde{\mu}(x,t)]\frac{\partial}{\partial x}[u(x,t)] &#xA;+ D_0\frac{\partial^2}&#xA;{\partial x^2}[u(x,t)]  &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rearranging and expanding, I can pick out the terms such that this resembles a typical advection-diffusion equation (with constant diffusion and drift coefficients) with an additional term like so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial u}{\partial t} =&#xA;\bigg[&#xA;-\mu_0\frac{\partial}{\partial x}[u(x,t)]&#xA;+ D_0\frac{\partial^2}{\partial x^2}[u(x,t)]&#xA;\bigg]_{linear-adv.-diff. \;equation}&#xA;\;&#xA;-\bigg[&#xA;\frac{\partial}{\partial x}[\tilde{\mu}(x,t)] u(x,t) &#xA;+\tilde{\mu}(x,t)\frac{\partial}{\partial x}[u(x,t)]&#xA;\bigg]_{additional\;term}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'll define:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;f_0(x,t) = &#xA;-\mu_0\frac{\partial}{\partial x}[u(x,t)]&#xA;+ D_0\frac{\partial^2}{\partial x^2}[u(x,t)]  &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To rewrite as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial u}{\partial t} =&#xA;f_0(x,t)&#xA;-\frac{\partial}{\partial x}[\tilde{\mu}(x,t)] u(x,t) &#xA;-\tilde{\mu}(x,t)\frac{\partial}{\partial x}[u(x,t)]  &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial u}{\partial t} =&#xA;f_0(x,t)&#xA;-\tilde{\mu}_x(x,t) u(x,t) &#xA;-\tilde{\mu}(x,t)u_x(x,t) &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I now want to begin supplementing my equation with my experimental data, and predictions from a numerical simulation of the linear-adv.-diff. equation, to get this show on the road. I substitute in a numerical approximation for $f_0(x,t')$, $f_{num}(x,t')$, at time $t'$ coming from a realization with the initial profile $u_{exp}(x,0)$, and coefficients $u_0$ and $D_0$. I thus write the error for my simulation, using this numerical prediction relative to the experimental $\mu_{exp}$ as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\bigg[&#xA;\frac{\partial u_{exp}(x,t)}{\partial t}&#xA;- f_{num}(x,t) =&#xA;-\tilde{\mu}_x(x,t) u_{exp}(x,t) &#xA;-\tilde{\mu}(x,t)u_{x,exp}(x,t)&#xA;\bigg]\bigg|_{t=t'}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\bigg[&#xA;- \epsilon(x,t) =&#xA;\tilde{\mu}_x(x,t) u_{exp}(x,t) &#xA;+\tilde{\mu}(x,t)u_{x,exp}(x,t)&#xA;\bigg]\bigg|_{t=t'}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\epsilon$ denotes the error in the prediction across $x$ at time $t'$. Next we recast this into the form of a familiar first-order-differential equation at time $t'$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\bigg[&#xA;\tilde{\mu}_x(x,t)&#xA;+\tilde{\mu}(x,t) \frac{u_{x,exp}(x,t)}{u_{exp}(x,t)} =&#xA;- \frac{\epsilon(x,t)}{u_{exp}(x,t)}&#xA;\bigg]\bigg|_{t=t'}&#xA; \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, where $p(x,t) = \frac{u_{x,exp}(x,t)}{u_{exp}(x,t)}$, and $q(x,t) = - \frac{\epsilon(x,t)}{u_{exp}(x,t)}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\bigg[&#xA;\tilde{\mu}_x(x,t)&#xA;+ p(x,t) \tilde{\mu}(x,t) =&#xA;q(x,t)&#xA;\bigg]\bigg|_{t=t'} &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can then be solved by the use of an integrating factor, as detailed:&#xA;&lt;a href=&quot;http://tutorial.math.lamar.edu/Classes/DE/Linear.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://tutorial.math.lamar.edu/Classes/DE/Linear.aspx&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose my question is: is there a more straight-forward method to solving for the expression $\mu(x,t)$ than first assuming a linear portion of the equation and then rederiving the $\mu(x,t)$ profile as I have done here? Also, is what I've done here coherent? I mean, is it permissible even to try to solve for $\mu(x,t)$ via the error in the linear-adv.-diff. approximation? A motivating image for this is shown below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Top) Experimental data. (Bottom) Linear-Adv.-Diff equation. Time is colored, with red being $t=0$ and purple being $t=t_{final}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wEict.jpg&quot; alt=&quot;(Top) Experimental data. (Bottom) Linear-Adv.-Diff equation. Time is colored, with red being $t=0$ and purple being $t=t_{final}$&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11799" LastEditorUserId="14772" LastEditDate="2015-03-10T18:23:45.413" LastActivityDate="2019-07-04T11:09:01.157" Title="Inverse advection-diffusion problem, solving for a drift coefficient with experimental data?" Tags="|pde|advection-diffusion|numerics|inverse-problem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16190" PostTypeId="1" AcceptedAnswerId="16196" CreationDate="2014-11-15T16:07:21.490" Score="2" ViewCount="6432" Body="&lt;p&gt;I am trying to solve the Duffing's equation in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ m\ddot{y}+c\dot{y}+ky+k_{3}y^{3} = f(t) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $ f(t) = A \sin{\omega t}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do that I wrote a function to be given to the ode45.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [F] = ode_fun(t,y)&#xA;&#xA;F = zeros(size(y));&#xA;&#xA;F(1) = y(2);&#xA;F(2) = -k*y(1)-c*y(2)-k3*y(1)^3+A*sin(omega*t);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then I call the ode function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[t,u] = ode45('ode_fun',[0 T],[0 0]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this the right way to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I am expecting to obtain a cubic function when plotting the &quot;force vs displacement&quot; plot but I actually obtain a straight line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I'll give you also the numerical values I used for the simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically I started from a linear equation I found in an example and then I added the nonlinear term to check if my code was able to distinguish between linear and nonlinear dynamic systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values of the linear case are: $m = 1$ , $k = 10^4$, $ c = 40$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I added the nonlinear term as $ k_3 = 11*10^4$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As forcing term I decided to use a sine wave having $A = 30$ and $\omega = 10$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final simulation time is $T = 60$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: the complete code I am testing is the one I talked about in this other question I posted &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16178/2d-cross-section-from-3d-surface&quot;&gt;2D cross section from 3D surface&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to understand if this procedure works for nonlinear parameter identification &lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2018-09-18T13:10:17.993" Title="Nonlinear ODE to solve Duffing's equation" Tags="|matlab|ode|nonlinear-equations|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16191" PostTypeId="2" ParentId="16189" CreationDate="2014-11-15T18:13:39.603" Score="7" Body="&lt;p&gt;There is another approach called the adjoint method, which is commonly used in inverse problems for PDE and which is quite easy to generalize to other problems. This is going to be long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your observations give you a field $u_{\text{exp}}$; you'd like to find a value of $\mu$ for which&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{2}\iint(u - u_\text{exp})^2dx\hspace{2pt}dt$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is a minimum, where $u$ satisfies the advection-diffusion equation with $\mu$ as the drift coefficient. That's a constraint, which we can enforce by the introduction of a Lagrange multiplier $\lambda$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\iint\lambda\left\{\frac{\partial u}{\partial t} + \frac{\partial}{\partial x}(\mu u) - \frac{\partial}{\partial x}\left(D\frac{\partial u}{\partial x}\right)\right\}dx\hspace{2pt}dt$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, altogether now, we seek 3 fields $u$, $\mu$, $\lambda$ which are an extremum of the functional&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J[u, \mu, \lambda] = \frac{1}{2}\iint(u - u_\text{exp})^2dx\hspace{2pt}dt + \iint\lambda\left\{\frac{\partial u}{\partial t} + \frac{\partial}{\partial x}(\mu u) - \frac{\partial}{\partial x}\left(D\frac{\partial u}{\partial x}\right)\right\}dx\hspace{2pt}dt$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's a constrained optimization problem; the first term is what we're optimizing, and the second is the constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the usual technique of the calculus of variations, you can compute the gradient $\nabla J$ of the functional $J$ to a small perturbation $\delta u$, $\delta \mu$, $\delta\lambda$, which gives you a set of PDEs to solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} + \frac{\partial}{\partial x}(\mu u) - \frac{\partial}{\partial x}\left(D\frac{\partial u}{\partial x}\right) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial\lambda}{\partial t} + \mu\frac{\partial\lambda}{\partial x} + \frac{\partial}{\partial x}\left(D\frac{\partial\lambda}{\partial x}\right) = u - u_{\text{exp}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one is (unsurprisingly) your original PDE. The second PDE looks odd; the diffusion coefficient is negative. However, no one said you had to solve it from $t = 0$ to $t = T$; you can go backwards and then everything's fine. This second system is called the &lt;em&gt;adjoint equation&lt;/em&gt;. Note that if you had a perfect guess for $\mu$ which made $u$ exactly equal to the experimental data, the right-hand side of the adjoint equation would be 0 and $\lambda$ would also be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having solved both of these PDE, the change in $J$ with respect to a small perturbation $\delta\mu$ is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla_\mu J\delta\mu = -\iint u\frac{\partial\lambda}{\partial x}\delta\mu\hspace{3pt} dx\hspace{2pt}dt$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, we know that $J$ decreases fastest in the direction opposite its gradient, so if you wanted to improve $J$, you could set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mu' = \mu + \alpha u\frac{\partial\lambda}{\partial x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and search for a value of the scalar $\alpha$ that works best. Or, you could use something more sophisticated like a quasi-Newton method, but that's just a detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pick some guess for $\mu$&lt;/li&gt;&#xA;&lt;li&gt;Solve the forward PDE for $u$&lt;/li&gt;&#xA;&lt;li&gt;Solve the adjoint PDE for $\lambda$&lt;/li&gt;&#xA;&lt;li&gt;Compute the gradient $\nabla J$&lt;/li&gt;&#xA;&lt;li&gt;Update $\mu$ and repeat if the error isn't good enough&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I can offer more detail if necessary, there's quite a bit left out. I don't know of a good introductory reference for the subject that isn't too general to be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, you want to avoid numerically differentiating experimental data. If it has some random red noise errors on top of the actual signal and you differentiate those, you get a bunch of nonsense out; you could apply a low-pass filter to it first to avoid that. In the adjoint method, you can take advantage of all the smoothing properties of parabolic PDE to do that for you.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2014-11-18T22:57:05.803" LastActivityDate="2014-11-18T22:57:05.803" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="16192" PostTypeId="2" ParentId="16188" CreationDate="2014-11-15T18:52:04.960" Score="1" Body="&lt;p&gt;I took a crack at this this morning, but I didn't get as far I as I would have liked, if you want you can look into using MathGL$^{[1]}$ in C++ to try to implement my following suggestion (which my be difficult due to there not being much support for MathGL), but this same thing could very easily be done in either python, octave, or matlab. The general treatment of this problem would be to make a 3D grid over x, y, and z, (each of x, y, and z would be an N by N by N matrix) and then solve $\Psi(x,y,z)^2$ at each point, which is explicitly written for each quantum number on the website you provided. The next thing to do is normalize $\Psi(x,y,z)^2$ to make the &lt;em&gt;color&lt;/em&gt; of each point correspond to the value of $\Psi(x,y,z)^2$ there (for example, 0 to 1, representing white to grey to black), so $\Psi(x,y,z)^2$ close to 0 is white, $\Psi(x,y,z)^2$ ~ .5 is grey, and $\Psi(x,y,z)^2$ ~ .9 is black. In other languages (such as python, octave, or matlab) this can easily be accomplished with the 'scatter'/'scatter3' functions and setting the point transparency with the 'alpha' value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$^{[1]}$ &lt;a href=&quot;http://uni-smr.ac.ru/archive/science/visualization/MathGL/mathgl-2.0.eng.pdf&quot; rel=&quot;nofollow&quot;&gt;http://uni-smr.ac.ru/archive/science/visualization/MathGL/mathgl-2.0.eng.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11799" LastActivityDate="2014-11-15T18:52:04.960" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16193" PostTypeId="2" ParentId="16181" CreationDate="2014-11-15T19:01:52.550" Score="2" Body="&lt;p&gt;If you're interested in the distribution for fixed values of $a,b,p,q$, you can compute and tabulate the cumulative distribution by quadrature. Since it's only a 1d function for fixed parameters, you can compute the cumulative function for fixed values of $x$ with pretty high accuracy, connect these points by a polynomial approximation, and then use this surrogate function for sampling.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-15T19:01:52.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16194" PostTypeId="2" ParentId="16179" CreationDate="2014-11-15T19:06:32.283" Score="2" Body="&lt;p&gt;The monotonicity of $h$ doesn't help you much if you can't say anything about the shape of the feasible set defined by the constraint. Intuitively, for a monotonic objective function, you'd like to &quot;go as far to the right as possible&quot; in each coordinate, but if the constraint function has no properties, in each coordinate direction the feasible set may be disconnected intervals. On the other hand, if $g$ is also increasing in each component, then you know that the feasible set is connected and, I believe, in fact convex. That's a much easier problem to describe.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-15T19:06:32.283" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16195" PostTypeId="1" AcceptedAnswerId="16208" CreationDate="2014-11-15T22:25:28.803" Score="1" ViewCount="140" Body="&lt;p&gt;Is there any solver for convex optimization in C++ (or some dedicated scheme while no solver is yet available) that could solve a convex optimization problem with objective function value given by an oracle? Thank you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My specific problem is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\[\mathop {\max }\limits_\lambda  \mathop {\min }\limits_{\sigma  \in {{\{ 0,1\} }^N}} {E_{\sigma ,\,\lambda }}&#xA;\]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;wherer lambda is a vector, and for each &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\[{\sigma  \in {{\{ 0,1\} }^N}}&#xA;\]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E is a linear function of lambda&#xA;\[{E_{\sigma ,\,\lambda }}&#xA;\]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In words:&#xA;It is actually maximize over lambda the piece-wise linear function defined by the minimum of exponential number of linear functions. Given lambda I have an effective scheme to obtain sigma and thus calculate \[\mathop {\min }\limits_{\sigma  \in {{\{ 0,1\} }^N}} {E_{\sigma ,\,\lambda }}&#xA;\] . so my problem is effectively a convex optimization with objective function given by my oracles (maximize over a concave function) and I am wondering whether there would be some solvers suitable to this type of problem. Or if there is any dedicated procedure for this while no solvers available. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you:D&lt;/p&gt;&#xA;" OwnerUserId="7891" LastActivityDate="2014-11-26T11:39:59.777" Title="convex optimization with objective function given by oracles" Tags="|software|convex-optimization|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16196" PostTypeId="2" ParentId="16190" CreationDate="2014-11-15T23:17:47.083" Score="3" Body="&lt;p&gt;Cubics can look quite linear in certain regimes. Your linear stiffness parameters, $k$ and $k_3$, are rather high. If you look at the the range of the position state variable with your specified values, you'll notice that the system is only deviating from zero by about $\pm 3\times10^{-3}$. Physically, this is effectively in the linear regime of the system so any nonlinear components will be very small. Reducing $k$ by an order of magnitude, the oscillations are larger and the plot of force (acceleration) vs. position starts to look cubic after the system settles into steady-state:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/MrQt2.png&quot; alt=&quot;acceleration vs. position plot&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Some suggestions.&lt;/em&gt;&lt;br&gt;&#xA;1) I may be a good idea to start using a &lt;a href=&quot;http://en.wikipedia.org/wiki/Stiff_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stiff&lt;/a&gt; solver for a this system, depending on what your parameters are. In Matlab, &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ode15s.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ode15s&lt;/code&gt;&lt;/a&gt; is a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Looking at your code in your &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16178/2d-cross-section-from-3d-surface&quot;&gt;other question&lt;/a&gt;, you really should be passing your parameter values as variable rather than hard coding them in multiple places. See &lt;a href=&quot;http://www.mathworks.com/help/matlab/math/parameterizing-functions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article on parametrizing functions&lt;/a&gt; in Matlab for details. For your code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tspan = [0 60];&#xA;k = 1e3;&#xA;c = 40;&#xA;k3 = 1.1e5;&#xA;A = 30;&#xA;omega = 10;&#xA;&#xA;ode_fun = @(t,u,k,c,k3,A,omega)[u(2);-k*u(1)-c*u(2)-k3*u(1).^3+A*sin(omega*t)];&#xA;&#xA;[t,u] = ode15s(@(t,u)ode_fun(t,u,k,c,k3,A,omega),tspan,[0 0]);&#xA;&#xA;figure;&#xA;plot(t,u);&#xA;xlabel('Time');&#xA;ylabel('State');&#xA;&#xA;acc = -k*u(:,1)-c*u(:,2)-k3*u(:,1).^3+A*sin(omega*t);&#xA;figure;&#xA;plot(u(:,1),acc);&#xA;xlabel('Position');&#xA;ylabel('Acceleration');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, I've used the the ODE function to directly obtain the acceleration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) You may want to specify your integration times (&lt;code&gt;tspan&lt;/code&gt;) as a vector with more than two elements that you can ensure a particular output step size. (N.B., this not the actual step size used internally by the integrator – &lt;a href=&quot;https://stackoverflow.com/questions/21859870/absolute-error-of-ode45-and-runge-kutta-methods-compared-with-analytical-solutio&quot;&gt;see here&lt;/a&gt;.) This will help if you use a finite difference scheme for calculating accelerations and it can produce smoother plots in some cases.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-11-16T03:09:45.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16199" PostTypeId="1" CreationDate="2014-11-17T01:10:33.693" Score="5" ViewCount="123" Body="&lt;p&gt;In going from double precision to quad (or higher) precision, &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;roughly how much of a performance hit is taken on various architectures and frameworks. Do floating point operations take twice as much time? Ten times as much time? One hundred times as much time?&lt;/li&gt;&#xA;&lt;li&gt;What frameworks are currently available that support quad precision for large-scale computation?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm interested in answers both about software simulation of quad precision with doubles, as well as quad precision that is done natively in hardware.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: Obviously given a problem requiring quad precision, the first step is to try to reformulate or restructure it so that it doesn't need quad precision anymore. Suppose for the sake of answering this question that such a reformulation of the problem is not possible and quad precision is truly needed.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2014-11-17T07:20:16.413" Title="Additional cost associated with quad (or higher) precision" Tags="|software|hpc|floating-point|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16200" PostTypeId="1" CreationDate="2014-11-17T02:29:00.580" Score="1" ViewCount="371" Body="&lt;p&gt;I have a sparse matrix such as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A =&#xA;&#xA;   (1,1)        1&#xA;   (3,1)        1&#xA;   (1,2)        1&#xA;   (2,2)        1&#xA;   (1,3)        1&#xA;   (3,3)        1&#xA;   (4,3)        1&#xA;   (4,4)        1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The full matrix of &lt;code&gt;A&lt;/code&gt; can see look like as following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;full(A) =&#xA;&#xA;     1     1     1     0&#xA;     0     1     0     0&#xA;     1     0     1     0&#xA;     0     0     1     1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to find the rank of matrix &lt;code&gt;A&lt;/code&gt; by fast way (because my matrix can extend to $10000 \times 20000$). I try to do it by two ways but it give the different result&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Convert to full matrix and find rank using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rank(full(A)) = 3&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find the rank using sprank&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sprank(A) = 4&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The true answer must be 3 that means using first way. However, it take long time to find the rank,especially matrix with large size. I know the reason why the second way give 4 because &lt;code&gt;sprank&lt;/code&gt; only tells you how many rows/columns of your matrix have non-zero elements, while rank is reporting the actual rank of the matrix which indicates how many rows of your matrix are linearly independent. &lt;code&gt;sprank(A)&lt;/code&gt; is 4 but &lt;code&gt;rank(A)&lt;/code&gt; is only 3 because you can write the third row as a linear combination of the other rows, specifically &lt;code&gt;A(2,:) - A(1,:)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My problem is that how to find the rank of a sparse matrix with lowest time consumption&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I tried to use some way. However, it reported larger time consumption comparison with rank function. Could you suggest to me other way?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Create random matrix&#xA; G = sparse(randi(2,1000,1000))-1;&#xA; A=sparse(G) %% Because my input matrix is sparse matrix&#xA;%% Measure performance&#xA;&amp;gt;&amp;gt; tic; rank(full(A)); toc&#xA;Elapsed time is 0.710750 seconds.&#xA;&amp;gt;&amp;gt; tic; svds(A); toc&#xA;Elapsed time is 1.130674 seconds.&#xA;&amp;gt;&amp;gt; tic; eigs(A); toc&#xA;Warning: Only 3 of the 6 requested eigenvalues converged. &#xA;&amp;gt; In eigs&amp;gt;processEUPDinfo at 1472&#xA;  In eigs at 365&#xA;Elapsed time is 4.894653 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10325" LastEditorUserId="9667" LastEditDate="2014-11-17T18:54:03.737" LastActivityDate="2014-11-21T02:15:54.693" Title="Optimizing rank computation for very large sparse matrices" Tags="|optimization|matlab|matrix|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16201" PostTypeId="2" ParentId="16199" CreationDate="2014-11-17T07:20:16.413" Score="5" Body="&lt;p&gt;It depends on how quad precision is implemented. If you want to implement it as &quot;traditional&quot; floating point numbers with sign, mantissa, and exponent (the latter two just having more than the normal 53 and 10 bits of double precision), then doing this on a processor that doesn't natively support it, is going to be pretty expensive because it will involve a lot of bit fiddling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, some ABIs implement quad precision by representing quad precision numbers $x$ as $x=a+\varepsilon b$ where $\epsilon$ is roughly equal to machine precision for double precision. In that case, additions and multiplications for quad precision can be implemented with a cost of 2-4 that of double precision because adding or multiplying two such numbers $x,y$ only involves adding or multiplying their respective components.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-17T07:20:16.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16203" PostTypeId="1" AcceptedAnswerId="16215" CreationDate="2014-11-17T11:30:12.323" Score="2" ViewCount="425" Body="&lt;p&gt;I have been using Fenics to model some simple electrostatic problems, but now I need to simulate more complicated geometries. Assuming&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have created a mesh of several objects, and know their positions in space&lt;/li&gt;&#xA;&lt;li&gt;Each node in those objects carries properties of those objects (dielectric constant/etc)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which software and approach should I use to subdivide the free space between these objects (and some bounding box style boundary), so that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mesh density depends on distance from objects / geometric complexity&lt;/li&gt;&#xA;&lt;li&gt;Fenics will know where the boundary is, and the boundary conditions (zero potential at &quot;infinity&quot;)&lt;/li&gt;&#xA;&lt;li&gt;Fenics will know the values of charge / dielectric in this free space (set to some default for example)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In my simple problems I have used hand coded functions for determining boundary, calculating boundary conditions, and for objects (such as a capacitor's plates) I have used their geometry to assign charges/etc (for example &lt;code&gt;if (distance from sphere center &amp;lt; radius) charge = 1.0 else 0&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="11822" LastActivityDate="2014-11-18T08:07:20.650" Title="Filling out a mesh (for Fenics FEM)" Tags="|fenics|mesh-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16204" PostTypeId="1" AcceptedAnswerId="16205" CreationDate="2014-11-17T11:59:06.493" Score="3" ViewCount="936" Body="&lt;p&gt;For an equation of the kind $u_t=Au$, time step is usually defined by ensuring that the eigenvalues are within the stability region of the time-stepping scheme employed. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the eigenvalues are on positive real axis, then does that mean that the system is unconditionally unstable for all time stepping methods?&lt;/li&gt;&#xA;&lt;li&gt;Is it possible to get a scheme that is stable in one time-stepping scheme but unstable in a different time stepping scheme(no matter how small the time step for the other scheme?). My feeling is that if the scheme is proven unstable for one time-stepping scheme, no matter how small the time-step (say, Forward Euler), then it is unstable for all time-stepping schemes because other TVD RK methods and multi-step methods are just convex combinations of the forward Euler method. Am I correct in assuming that? On the other hand, if it is stable in one time-stepping scheme, does it imply that it is stable in all other schemes by suitably adjusting the time-step?&lt;/li&gt;&#xA;&lt;li&gt;If I use, for example, Discontinuous Galerkin methods for equations like shallow water, then the matrix A changes for every time step, does one calculate the eigenvalues at every time-step to ensure stability? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11644" LastEditorUserId="27" LastEditDate="2014-11-17T12:27:55.510" LastActivityDate="2014-11-17T12:33:14.493" Title="Eigenvalues and Timestep restriction" Tags="|numerical-analysis|eigenvalues|time-integration|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16205" PostTypeId="2" ParentId="16204" CreationDate="2014-11-17T12:33:14.493" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If the eigenvalues are on positive real axis, then does that mean that the system is unconditionally unstable for all time stepping methods?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. If you take a large enough time step with backward Euler, then the scheme will be stable, but probably inaccurate. The underlying continuous problem is unstable (in the sense of dynamical systems).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to get a scheme that is stable in one time-stepping scheme but unstable in a different time stepping scheme(no matter how small the time step for the other scheme?). My feeling is that if the scheme is proven unstable for one time-stepping scheme, no matter how small the time-step (say, Forward Euler), then it is unstable for all time-stepping schemes because other TVD RK methods and multi-step methods are just convex combinations of the forward Euler method. Am I correct in assuming that? On the other hand, if it is stable in one time-stepping scheme, does it imply that it is stable in all other schemes by suitably adjusting the time-step?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Take your system with positive eigenvalues. It's stable for backward Euler for sufficiently large time steps, and unstable when using forward Euler or the trapezoidal rule with any time step. Again, accuracy might be an issue when using backward Euler with large time steps.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If I use, for example, Discontinuous Galerkin methods for equations like shallow water, then the matrix A changes for every time step, does one calculate the eigenvalues at every time-step to ensure stability?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not usually, no. Adaptive time steppers (for instance, SUNDIALS, which implements BDF methods and Adams-Bashforth methods) will use heuristics to detect both stability and accuracy issues to adjust time steps accordingly. When using a fixed time step, instabilities can sometimes be detected by eye (preferably via plotting; potentially via the raw numerical results in obvious cases, or if you have good intuition). Extremal eigenvalues could be estimated, for instance, via GMRES when doing each linear solve, which would be used to confirm the suspected diagnosis of instability. (PETSc is a good choice of package for such diagnostics.) Another test would be to decrease the time step and compare results via a convergence study; such a strategy isn't foolproof, but often works well as a diagnostic.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-17T12:33:14.493" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16206" PostTypeId="2" ParentId="10068" CreationDate="2014-11-17T13:05:30.527" Score="0" Body="&lt;p&gt;If you can't afford 30 runs, each varying one parameter, vary them in groups:&lt;br&gt;&#xA;for example, 8 runs each varying 4 parameters together, then refine the best 2 runs / 8 parameters ...&lt;br&gt;&#xA;(I have no idea how to tradeoff info gain vs. total runtime;&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Multi-armed_bandit&quot; rel=&quot;nofollow&quot;&gt;multi-armed bandit&lt;/a&gt; ?)&lt;/p&gt;&#xA;" OwnerUserId="1841" LastActivityDate="2014-11-17T13:05:30.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16208" PostTypeId="2" ParentId="16195" CreationDate="2014-11-17T21:45:34.230" Score="4" Body="&lt;p&gt;OK- so you're trying to maximize a concave function which is piecewise linear, and you can evaluate the function and get a subgradient at any desired point.  This is equivalent to minimizing a convex non-differentiable function using only function and subgradient evaluations (just minimize minus the objective function.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should read the papers by Yuri Nesterov on these problems (e.g. &lt;a href=&quot;http://link.springer.com/article/10.1007/s10107-004-0552-5&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/article/10.1007/s10107-004-0552-5&lt;/a&gt;) Basically, he establishes bounds for the optimal performance of an algorithm and shows that his algorithm (which smooths the non-smooth problem and then apply an optimal algorithm for the smoothed problem) is optimal in its order of convergence.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Nesterov's 2005 paper there has been a lot of research on fast first order methods for nonsmooth convex optimization, particularly for applications related to image processing and compressive sensing.  Although higher performance can be achieved if you assume strong convexity (which may not be possible in your case), there are algorithms that do not require this assumption.   See for example Arnold Neumaier's OSGA algorithm (&lt;a href=&quot;http://arxiv.org/pdf/1402.1125.pdf&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/pdf/1402.1125.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-11-17T21:45:34.230" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16209" PostTypeId="1" CreationDate="2014-11-18T00:45:23.353" Score="4" ViewCount="3850" Body="&lt;p&gt;I wrote the code below to invert an upper triangular matrix, avoiding any possible multiplication/subtraction by zero. It just uses $\frac{1}{6}n^3+\ldots$ flops instead of $n^3+\ldots$ flops.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function B = InvTrMat(A,n,type)&#xA;  % This code inverts an upper triangular matrix of order n&#xA;  % without doing any multiplication or subtraction by zero.&#xA;  B = zeros(n);&#xA;&#xA;  for j = 1:n&#xA;    B(j,j) = 1 / A(j,j);&#xA;    for i = j-1:-1:1&#xA;      for k = i+1:j&#xA;        B(i,j) = B(i,j) - A(i,k) * B(k,j);&#xA;      end&#xA;      B(i,j) = B(i,j) * B(i,i);&#xA;    end&#xA;  end&#xA;&#xA;  return&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is it is slower than the backslash command of Matlab. I used the following code to test and compare both:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 128;&#xA;t = zeros(1,100);&#xA;u = zeros(1,100);&#xA;&#xA;for i = 1:100&#xA;  A = rand(n,n);&#xA;  A = A+A';&#xA;  A = A + n*eye(n);&#xA;&#xA;  R = chol(A);&#xA;&#xA;  tic;&#xA;  B = R \ eye(n);&#xA;  t(i) = toc;&#xA;&#xA;  tic;&#xA;  B = InvTrMat(R,n,'u');&#xA;  u(i) = toc;&#xA;end&#xA;&#xA;sum(t)/100&#xA;sum(u)/100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to make my code faster than Matlab's code? It must be possible since I'm using just a fraction of flops from the other code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Matlab version is 7.10.0 (R2010a), installed on a Windows 8.0 64-bits PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you, everyone.&lt;/p&gt;&#xA;" OwnerUserId="11831" LastEditorUserId="123" LastEditDate="2014-11-18T09:24:52.513" LastActivityDate="2014-11-18T15:08:58.467" Title="Why is my MATLAB code for back-substitution slower than the backslash operator?" Tags="|linear-algebra|matlab|matrix|performance|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16212" PostTypeId="2" ParentId="16209" CreationDate="2014-11-18T01:26:06.673" Score="11" Body="&lt;p&gt;MATLAB's &lt;code&gt;\&lt;/code&gt; (aka &lt;code&gt;mldivide&lt;/code&gt;) command does not blindly compute the inverse of the matrix. Instead, it uses one of several algorithms based on the type of matrix (see the &quot;Algorithms&quot; section of &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/mldivide.html&quot;&gt;http://www.mathworks.com/help/matlab/ref/mldivide.html&lt;/a&gt;). In the case of a triangular matrix, MATLAB will use a triangular solver which is at least as good as yours in terms of operation count (I haven't looked at your code too closely but they're probably the same algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real difference is that the linear algebra routines in MATLAB are actually calls to highly optimized Fortran routines under the hood (i.e. LAPACK, see &lt;a href=&quot;http://www.mathworks.com/company/newsletters/articles/matlab-incorporates-lapack.html&quot;&gt;http://www.mathworks.com/company/newsletters/articles/matlab-incorporates-lapack.html&lt;/a&gt;). For any case that matches the intended use of these routines you will not be able to match this performance in native MATLAB code. This is due to the nature of the MATLAB language which receives only limited optimization on the fly by the just in time compiler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For best performance in MATLAB, vectorize things and use the built in routines whenever possible.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-11-18T01:34:37.097" LastActivityDate="2014-11-18T01:34:37.097" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16213" PostTypeId="1" AcceptedAnswerId="16214" CreationDate="2014-11-18T02:06:29.203" Score="1" ViewCount="1346" Body="&lt;p&gt;I would like to use Boost C++ odeint Runge-Kutta integrator on a system that looks like this :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot x = - \frac A{||x||^3} *  x   $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ x $ is a vector in 3D space, so basicaly $ x(i, j, k) $ &lt;br&gt;&#xA;$ \ddot x $ is its second derivative &lt;br&gt;&#xA;$ {||x||^3} $ is magnitude cubed of $ x $ &lt;br&gt;&#xA;$ A $ is a constant &lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the initial conditions of the problem, namely $ \dot x(t=0) $ and $ x(t=0) $ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have checked &lt;a href=&quot;http://www.boost.org/doc/libs/1_57_0/libs/numeric/odeint/doc/html/boost_numeric_odeint/tutorial/harmonic_oscillator.html&quot; rel=&quot;nofollow&quot;&gt;this example&lt;/a&gt; in the odeint documentation, as well as the &lt;a href=&quot;https://github.com/headmyshoulder/odeint-v2/blob/master/examples/harmonic_oscillator.cpp&quot; rel=&quot;nofollow&quot;&gt;full code here&lt;/a&gt;. Examples show use of odeint with a single ODE. But my problem would have to be split into 6 ODE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can I use odeint Runge Kutta method for such a system ( 6 ODEs ) and if yes, is there any example that I can follow to help me implement my problem ?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="10011" LastEditorUserId="123" LastEditDate="2014-11-18T09:08:22.360" LastActivityDate="2014-11-18T09:08:22.360" Title="How can I call the Boost C++ odeint Runge-Kutta integrator for a system of ODEs?" Tags="|numerical-analysis|ode|c++|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16214" PostTypeId="2" ParentId="16213" CreationDate="2014-11-18T03:02:44.353" Score="4" Body="&lt;p&gt;In theory, sure, although I'd be wary of your particular ODE system for a couple reasons: (1) the right-hand side is undefined when $x = 0$, which will trip up any integrator in the Boost C++ suite, (2) based on its similarity to functions known to be non-Lipschitz in neighborhoods of the origin, even existence of solutions is unclear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementing your system should be straightforward. Transform your system into first-order form, and then model your implementation after the &lt;a href=&quot;http://headmyshoulder.github.io/odeint-v2/examples.html&quot; rel=&quot;nofollow&quot;&gt;Lorenz system attractor example on the odeint website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-11-18T03:13:34.683" LastActivityDate="2014-11-18T03:13:34.683" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16215" PostTypeId="2" ParentId="16203" CreationDate="2014-11-18T08:07:20.650" Score="2" Body="&lt;p&gt;Refer to &lt;code&gt;dolfin-convert&lt;/code&gt; script installed within FEniCS, and its manual page,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ man dolfin-convert&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for supported mesh formats. Then pick some software generating those formats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that cell functions (useful for specifying permitivities in your case) and/or facet functions (useful for boundary and interfacial conditions) are not fully supported by &lt;code&gt;dolfin-convert&lt;/code&gt; for all formats. To get those details one can dive into &lt;code&gt;&amp;lt;PYTHONPATH&amp;gt;/dolfin_utils/meshconvert/meshconvert.py&lt;/code&gt; or simply try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AFAIK, at least &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;Triangle&lt;/code&gt;&lt;/a&gt; (2D) and &lt;a href=&quot;http://www.geuz.org/gmsh/&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;Gmsh&lt;/code&gt;&lt;/a&gt; (2D and 3D) formats are quite well supported by &lt;code&gt;dolfin-convert&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4254" LastActivityDate="2014-11-18T08:07:20.650" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16216" PostTypeId="1" CreationDate="2014-11-18T10:07:55.893" Score="2" ViewCount="53" Body="&lt;p&gt;This is a follow-up question to the previous questions I had on eigenvalues. Please let me know if I should edit the previous question itself for asking this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the eigenvalues of a matrix generated from a lower order coarse mesh discretization are negative, can we comment anything on the eigenvalues of the matrix that we get by refining the mesh or using a higher order polynomial for discretization of the same pde?&lt;/p&gt;&#xA;" OwnerUserId="11644" LastActivityDate="2014-11-18T10:07:55.893" Title="Eigenvalues and Timestep restriction Follow up" Tags="|numerical-analysis|eigenvalues|time-integration|runge-kutta|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16218" PostTypeId="2" ParentId="16145" CreationDate="2014-11-18T15:55:49.540" Score="5" Body="&lt;p&gt;No need for any advanced books, the easiest to implement answer is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use a DFS (&lt;a href=&quot;http://en.wikipedia.org/wiki/Depth-first_search&quot;&gt;http://en.wikipedia.org/wiki/Depth-first_search&lt;/a&gt;), and store the cumulative sum of each subtree in the stack. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a possible DFS traversal in your example is: A-&gt;B-&gt;D-&gt;I-&gt;J-&gt;E-&gt;C-&gt;F-&gt;G-&gt;H.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After computing the cumulative sum of the child of a node, add this value to that of the parent. When done computing all the sums of the children nodes, have the parent return its value to its own parent. (e.g. After computing all of B's children, return the value of B to A). Continue until done with tree. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pseudocode:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int sum(int n)&#xA;{&#xA;    int cumulative=0;&#xA;    for all children i of n&#xA;        cumulative+=sum(i)&#xA;    cumulative+=value[n]&#xA;    return cumulative&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I apologize for the similarity of the Pseudocode to C++ ;).&#xA;The complexity is Θ(n), and you can't get any faster, since you must access each node at least once to find the sum/get input (if it exists).&lt;/p&gt;&#xA;" OwnerUserId="11839" LastActivityDate="2014-11-18T15:55:49.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16219" PostTypeId="1" AcceptedAnswerId="16226" CreationDate="2014-11-18T19:11:56.570" Score="5" ViewCount="1103" Body="&lt;p&gt;I am working on a problem that involves taking fractional powers of particular matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the matrix A with 2 on the main diagonal and -1 on the sub and super diagonal (the finite difference matrix for the Dirichlet Laplacian in 1D), there was basically no difference between using MATLAB's &lt;code&gt;sqrtm(A)&lt;/code&gt; function and diagonalizing the matrix (&lt;code&gt;[V,D]=eig(A)&lt;/code&gt;, &lt;code&gt;A = V\D*V&lt;/code&gt;) and computing &lt;code&gt;V\D^(1/2)*V&lt;/code&gt; even if the matrix was as large as 1000x1000. This made me think I could compute other fractional powers &lt;code&gt;V\D^(1/n)*V&lt;/code&gt; that MATLAB does not have built in functions for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when I consider the matrix A for the Dirichlet Laplacian in 2D, there is a significant difference between MATLAB's sqrtm function and the diagonalization procedure I used above even if the matrix was as small as 100x100. In particular, MATLAB's sqrtm function is better and my results don't make any sense if I do it the other way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if anyone can explain why this happens and if there is a way to fix it so that I can compute fractional powers of a large matrix in MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="11843" LastActivityDate="2014-11-19T20:44:35.483" Title="What is a good way to take fractional powers of a matrix in MATLAB?" Tags="|matlab|matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16220" PostTypeId="1" AcceptedAnswerId="16240" CreationDate="2014-11-18T21:30:42.413" Score="4" ViewCount="3301" Body="&lt;p&gt;If we call LAPACK &lt;code&gt;DGGEV&lt;/code&gt; or &lt;code&gt;DGGEVX&lt;/code&gt; routines for two badly-conditioned matrices in a C++ code, will we get the same eigen-values &amp;amp; vectors as MATLAB's &lt;code&gt;eig&lt;/code&gt; command with &lt;code&gt;'qz'&lt;/code&gt; algorithm like: &lt;code&gt;eig(a,b,'qz')&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any possibility that eigenvectors of MATLAB are different from LAPACK &lt;code&gt;DGGEV&lt;/code&gt; or &lt;code&gt;DGGEVX&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="11726" LastEditorUserId="386" LastEditDate="2014-11-27T13:47:53.420" LastActivityDate="2014-11-27T13:47:53.420" Title="Eigenvectors: MATLAB vs LAPACK DGGEV or DGGEVX" Tags="|linear-algebra|matlab|eigensystem|eigenvalues|lapack|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16221" PostTypeId="1" AcceptedAnswerId="16223" CreationDate="2014-11-19T03:02:50.463" Score="5" ViewCount="386" Body="&lt;p&gt;After a cursory glance at several references to discontinuous galerkin finite element methods for the elliptic poisson PDE, i notice that all of them emphasize using penalty methods where an additional term in the weak form (involving the average and jump operators of the test function and trial functions) is penalized. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My colleagues seem to indicate that this is the way it is usually done, but can't really explain why.  For hyperbolic PDE's involving conservation laws, no such additional penalty term is needed.  So, i wonder if they are absolutely necessary.  And if so, why?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, what exactly goes wrong if i use DG-FEM for the poisson problem and omit the penalty term?  Does the method lose coercivity or boundedness, for example?  How does the penalty term fix this problem?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-11-19T04:13:59.477" LastActivityDate="2014-11-20T14:07:37.617" Title="Are additional penalty terms necessary to solve elliptic PDE's with DG-FEM?" Tags="|finite-element|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16222" PostTypeId="1" CreationDate="2014-11-19T03:36:29.443" Score="2" ViewCount="179" Body="&lt;p&gt;This question follows my earlier question &lt;a href=&quot;https://scicomp.stackexchange.com/q/16213/10011&quot;&gt;How can I call the Boost C++ odeint Runge-Kutta integrator for a system of ODEs?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm still considering the system:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \ddot x = - \frac{Ax}{||x||^3} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;My intention was to use a classical Runge-Kutta 45 method to solve ODEs of my system. However, I have seen recommendations for using &lt;a href=&quot;https://en.wikipedia.org/wiki/Symplectic_integrator&quot; rel=&quot;nofollow noreferrer&quot;&gt;symplectic integrators&lt;/a&gt;, because they do a better job at conserving the total energy of the system.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How do I know if a symplectic integrator is a better choice for my system?&lt;/strong&gt; And either way, what are the downsides of using symplectic integrators over, say, RK45 ?&lt;/p&gt;&#xA;" OwnerUserId="10011" LastEditorUserId="23791" LastEditDate="2021-12-20T21:20:12.320" LastActivityDate="2021-12-20T21:20:12.320" Title="How to tell if symplectic integrator is suitable for my problem?" Tags="|ode|numerics|runge-kutta|" AnswerCount="0" CommentCount="4" ClosedDate="2022-01-15T00:24:01.947" ContentLicense="CC BY-SA 4.0" />
  <row Id="16223" PostTypeId="2" ParentId="16221" CreationDate="2014-11-19T05:52:43.610" Score="4" Body="&lt;p&gt;The penalty term does indeed act as a stabilization - coercivity is usually shown using norms which depend on this penalty, and you typically need a &quot;sufficiently large penalty&quot; &lt;a href=&quot;http://maths.dur.ac.uk/~dma0mpj/summer_school/IPHO.pdf&quot; rel=&quot;nofollow&quot;&gt;to show convergence of symmetric interior penalty DG&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe one exception is NIPG (non-symmetric interior penalty DG, they flip the sign of the numerical flux terms involving the average of gradient of $v$) you can use any positive value for the penalty. At the very least, the &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/10081784X&quot; rel=&quot;nofollow&quot;&gt;Nitsche's method&lt;/a&gt; analogue to NIPG for boundary conditions has this feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A difference with hyperbolic PDEs is that there's a clear directionality to the problem - you have characteristic directions in which information is transmitted, and in which you can choose upwind directions. Contrast this to an elliptic PDE, in which information is spread isotropically - there's no clear &quot;upwind&quot; direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One note - the idea behind LDG for Poisson is to write the heat equation in first order form, and then use upwind fluxes there (though the two fluxes are upwinded in opposite directions to mimic this isotropic spread of information).  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2014-11-20T14:07:37.617" LastActivityDate="2014-11-20T14:07:37.617" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16224" PostTypeId="1" CreationDate="2014-11-19T16:21:54.193" Score="2" ViewCount="338" Body="&lt;p&gt;I'm looking for a current and comprehensive overview (like a review article) of global optimization methods and their application in computational chemistry. Mostly I'm interested in geometry optimization.&#xA;I am also grateful for single recommendations including &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;literature (articles, textbooks)&lt;/li&gt;&#xA;&lt;li&gt;programs, capable to perform global optimizations&lt;/li&gt;&#xA;&lt;li&gt;presentations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this case it would be very nice, if you could assign the methods to certain areas like solid state, molecules on surfaces, isolated molecules, ensembles, etc. .&lt;/p&gt;&#xA;" OwnerUserId="3486" LastEditorUserId="3486" LastEditDate="2014-11-20T09:55:18.947" LastActivityDate="2014-11-20T09:55:18.947" Title="Global optimization methods in computational chemistry" Tags="|optimization|computational-chemistry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16225" PostTypeId="2" ParentId="16224" CreationDate="2014-11-19T17:52:59.183" Score="5" Body="&lt;p&gt;You say you're mostly interested in geometry optimization. From a software point of view, here are a couple to get you started.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://uspex.stonybrook.edu&quot; rel=&quot;nofollow&quot;&gt;USPEX&lt;/a&gt;, which works with crystals,&#xA;isolated molecules and nanoparticles, using a genetic algorithm and&#xA;linking with many DFT and empirical codes from MATLAB (e.g. VASP, CASTEP, Siesta, GULP...). The link also has&#xA;good articles summarising the technique and comparing it to&#xA;some alternatives method for particular test cases, including a comparison for both  crystal systems and Lennard-Jones clusters.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://wiki.fysik.dtu.dk/ase/tutorials/minimahopping/minimahopping.html&quot; rel=&quot;nofollow&quot;&gt;Constrained Minima&#xA;Hopping&lt;/a&gt;,&#xA;implemented here in the Atomistic Simulation Environment, so again&#xA;can link with your DFT/MD code of choice. This is aimed at&#xA;adsorbates on a surface, and you can find a &lt;a href=&quot;http://dx.doi.org/10.1007/s11244-013-0161-8&quot; rel=&quot;nofollow&quot;&gt;good description in&#xA;this paper by A.&#xA;Peterson&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As I mentioned, there are links to review papers from both of the above methods, which should help you on your way to a more comprehensive literature search (e.g. more review articles). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth noting that there are also these options:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cmmp.ucl.ac.uk/~ajm/airss/airss.html&quot; rel=&quot;nofollow&quot;&gt;Ab-Initio Random Structure&#xA;Searching&lt;/a&gt;, which I&#xA;don't know much about. &lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://dx.doi.org/10.1039/B305686D&quot; rel=&quot;nofollow&quot;&gt;Birmingham Cluster Genetic Algorithm&#xA;(BCGA)&lt;/a&gt;, which I guess is&#xA;similar to the USPEX approach above, only aimed specifically at&#xA;clusters.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8094" LastEditorUserId="8094" LastEditDate="2014-11-20T09:31:51.293" LastActivityDate="2014-11-20T09:31:51.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16226" PostTypeId="2" ParentId="16219" CreationDate="2014-11-19T20:44:35.483" Score="4" Body="&lt;p&gt;MATLAB has the ability to compute fractional powers of matrices by using the normal &lt;code&gt;^&lt;/code&gt; operator which calls the &lt;code&gt;mpower&lt;/code&gt; function when either of the arguments is a matrix. You can read a discussion of when fractional powers of matrices can be computed &lt;a href=&quot;https://math.stackexchange.com/questions/615614/non-integral-powers-of-a-matrix&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your original issues may stem from the fact that you have mixed up the formula to compute matrices raised to a fractional power. The matrix should be diagonalized as $A = PDP^{-1}$. Then $A^\alpha = PD^\alpha P^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB this can be done as follows (your code is a bit off):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[V,D]=eig(A);&#xA;Aalph = V*(D.^alph)/V;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Based on a quick test, it seems that this is exactly the method used by MATLAB inside  &lt;code&gt;mpower&lt;/code&gt;. The following code returns exactly 0 (no differences due to finite precision) on my machine.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = rand(1000);&#xA;alph = 2/3;&#xA;[V,D]=eig(A);&#xA;Aalph = V*(D.^alph)/V;&#xA;B = A^(alph);&#xA;max(abs(Aalph(:)-B(:)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-11-19T20:44:35.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16227" PostTypeId="1" AcceptedAnswerId="16231" CreationDate="2014-11-19T21:57:24.713" Score="0" ViewCount="406" Body="&lt;p&gt;For $i=1, \dotsc, K$, we have $n_i$ ordered real numbers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_i(1) \leq x_i(2) \leq \dotsc \leq x_i(n_i) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve the following optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mathrm{maximize} \; \sum_{i=1}^Kr_i \;\; s.t. \\&#xA;r_i \in \{1,\dotsc, n_i\}\\&#xA;\sum_{i=1}^Kr_i \leq c\sum_{i=1}^K x_i(r_i)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I had the idea of expressing this problem as a MILP as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each $i$, define binary variables which satisfy the  linear constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;z_i(1) \geq z_i(2) \geq \dotsc \geq z_i(n_i) \in \{0,1\}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also define $y_i(k) = x_i(k)-x_i(k-1), k = 1,\dotsc, n_i$, where $x_i(0)=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we can solve the MILP:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mathrm{maximize} \; \sum_{i=1}^K \sum_{j=1}^{n_i} z_i(j) \;\; s.t. \\&#xA;z_i(1) \geq z_i(2) \geq \dotsc \geq z_i(n_i) \in \{0,1\} \\&#xA;\sum_{i=1}^K \sum_{j=1}^{n_i} z_i(j) \leq c\sum_{i=1}^K \sum_{j=1}^{n_i} y_i(j)z_i(j)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any obvious easier way of modelling this problem? e.g. as a MILP with less integer variables or using another optimization methodology.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any simple way to intervene with the solvers in order to improve the solution speed of this problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In most of my problems, the number of binary variables is between 20000-100000. For this reason, I was actually very surprised that using the &lt;a href=&quot;https://projects.coin-or.org/SYMPHONY&quot; rel=&quot;nofollow&quot;&gt;Symphony&lt;/a&gt; solver, those problems get solved in less than a minute! Should these problem sizes not be prohibitive? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Based on prior information, I also tried to remove the highest $x_i(j)$ values, thus reducing the total number of binary variables. E.g. reducing the number of variables by 50% often lead to substantial speed gains. On the other hand, it was often the case that when I reduced the number of variables to 10%, the solver would actually take longer than with all the variables (even though it would eventually return the same solution). How can this be explained?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11508" LastActivityDate="2014-11-20T23:42:27.253" Title="MILP formulation and optimization" Tags="|optimization|linear-programming|mixed-integer-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16229" PostTypeId="2" ParentId="16220" CreationDate="2014-11-20T02:46:20.067" Score="1" Body="&lt;p&gt;The best way to find the answer to this question was to experiment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We calculated the Eigen values &amp;amp; vectors of two badly-conditioned matrices with MATLAB R2013b by using &lt;code&gt;eig&lt;/code&gt; command like &lt;code&gt;eig(a,b,'qz')&lt;/code&gt;. The ill-conditioned matrices were 342x342 in size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the explanations on the following link, MATLAB R2009a implemented LAPACK DGGEV for doing &lt;code&gt;eig(a,b,'qz')&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/26808510/which-algorithm-do-dggev-or-dsygv-eigen-solvers-in-lapack-implement-is-it-qz&quot;&gt;https://stackoverflow.com/questions/26808510/which-algorithm-do-dggev-or-dsygv-eigen-solvers-in-lapack-implement-is-it-qz&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, we implemented DGGEV (and also DGGEVX) routines from LAPACK in our C++ code by using LAPACK&lt;strong&gt;E&lt;/strong&gt; which is a &quot;C&quot; language wrapper for LAPACK developed by INTEL-MKL team. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conclusion was that we got exactly the same Eigen values &amp;amp; vectors as MATLAB &lt;code&gt;eig(a,b,'qz')&lt;/code&gt; precisely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that if you use DGGEVX (expert version of DGGEV), you should turn the balancing off by using &lt;code&gt;'N'&lt;/code&gt; for BALANC argument, to get exactly the same Eigen vectors as MATLAB &lt;code&gt;'qz'&lt;/code&gt; algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also checked the correctness of Eigen values &amp;amp; vectors by checking if they satisfy &lt;code&gt;A*V-B*V*D=0&lt;/code&gt;, and the coclusion was that Eigen values &amp;amp; vectors were correct, as the residual error was close enough to zero. &lt;/p&gt;&#xA;" OwnerUserId="11726" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-11-21T13:27:01.187" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16230" PostTypeId="2" ParentId="16195" CreationDate="2014-11-20T03:05:03.637" Score="1" Body="&lt;p&gt;If your objective function is piecewise linear and concave, then it is the minimum of a bunch of globally linear functions. Let me put this in the more usual minimization framework (instead of maximization -- just flip the sign):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you're trying to solve&#xA;$$&#xA;  \min_\lambda f(\lambda)&#xA;$$&#xA;where $f(\lambda)$ is piecewise linear and convex. Then I can write&#xA;$$&#xA;  f(\lambda) = \min_i f_i(\lambda)&#xA;$$&#xA;where each of the $f_i$ are linear. There is a canonical way of solving this, namely by introducing a scalar slack variable $\mu$ and writing the problem as&#xA;$$&#xA;  \min_{\lambda,\mu} \mu&#xA;\\&#xA;  \text{so that }\quad&#xA;  \mu \ge f_i(\lambda), \qquad i=1...N.&#xA;$$&#xA;This is now a linear program: the objective function and all constraints are linear. Even if large, there are very efficient ways of solving this problem if only you can characterize the $f_i$. In your case, I imagine these are exactly the $E_\sigma(\lambda)$ for each of the possible $\sigma$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2014-11-26T11:39:59.777" LastActivityDate="2014-11-26T11:39:59.777" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16231" PostTypeId="2" ParentId="16227" CreationDate="2014-11-20T08:44:44.093" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Is there any obvious easier way of modelling this problem? e.g. as a MILP with less integer variables or using another optimization methodology.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The first tactic that immediately comes to mind is, for each $i$, modeling the set $\{z_{i,1}, \ldots, z_{i,n_{i}}\}$ as a &lt;a href=&quot;http://en.wikipedia.org/wiki/Special_ordered_set&quot; rel=&quot;nofollow&quot;&gt;SOS1 set&lt;/a&gt;. Your current formulation essentially does the same thing.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Is there any simple way to intervene with the solvers in order to improve the solution speed of this problem?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Aside from reformulating the problem to use SOS1 sets, you could use a better solver. For instance, you could use either Gurobi or CPLEX, both of which regularly compete for best performing solver over test beds of MILP and LP instances. Both of these solvers are roughly 10x faster than any open source MILP solver. Changing your solver is probably the fastest way to achieve some sort of speed-up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, you could look to see if you could reformulate your problem so that the LP relaxations are tighter, or introduce more and better cuts that exclude nonintegral regions of your feasible region. A senior colleague of mine has pointed out that adding custom cuts in CPLEX will disable its very robust presolve routines, along with its cut heuristics; I have not seen this behavior in practice, but the mere possibility is worth keeping an eye on if you choose to go this route.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also be able to come up with a polynomial time approximation scheme that is either &quot;good enough&quot;, or maybe it even generates feasible solutions for your formulation. These feasible solutions would provide upper bounds on the optimal objective function value, which would then be useful in fathoming nodes in a branch-and-bound tree. Your formulation looks vaguely like a knapsack problem, but it doesn't have the right structure to be a knapsack problem. Nevertheless, you might be able to impose some structure like that to obtain a decent polynomial time approximation algorithm (or even a solution algorithm).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;In most of my problems, the number of binary variables is between 20000-100000. For this reason, I was actually very surprised that using the Symphony solver, those problems get solved in less than a minute! Should these problem sizes not be prohibitive?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Structure is everything. Since each group of $z_{i,1}, \ldots, z_{i,n}$ is a SOS1 set, a branch-and-bound implementation only needs to examine $\prod_{i = 1}^{K}n_{i}$ possible combinations of binary variables instead of $2^{\sum_{i=1}^{K}n_{i}}$ variables. The former expression grows more slowly than the latter. I suspect CPLEX or Gurobi would preprocess this problem, deduce this structure, and employ effective cuts to solve this problem quickly (relative, to say, a random MILP instance with 20000-100000 binary variables and no continuous variables).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-20T08:44:44.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16232" PostTypeId="1" CreationDate="2014-11-20T09:30:09.710" Score="1" ViewCount="1793" Body="&lt;p&gt;I am trying to repeat an example I found in a paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to solve this ODE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$25 \ddot{x} + 15 \dot{x} + 330000 x = p(t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $p(t)$ is a white noise sequence band-limited into the 10-25 Hz range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then it says the system is simulated using a Runge-Kutta procedure. The sampling frequency is set to 1000 Hz and a Gaussian white noise is added to the data in such a way that the noise contributes to 5% of the signal r.m.s value (how do I use this last info?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problem is related to the band-limited white noise. I followed the instructions I found here &lt;a href=&quot;https://dsp.stackexchange.com/questions/9654/how-to-generate-band-limited-gaussian-white-noise&quot;&gt;https://dsp.stackexchange.com/questions/9654/how-to-generate-band-limited-gaussian-white-noise&lt;/a&gt; and I wrote the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% White noise excitation&#xA;% design FIR filter to filter noise in the range [10-25] Hz&#xA;Fs = 1000; % sampling frequency&#xA;&#xA;% Time infos (for white noise)&#xA;T = 1/Fs;&#xA;tspan = 0:T:4; % I saw from the plots in the paper that the simulation last 4 seconds&#xA;tstart = tspan(1);&#xA;tend = tspan (end);&#xA;&#xA;b = fir1(64, [10/(Fs/2) 25/(Fs/2)]);&#xA;% generate Gaussian (normally-distributed) white noise&#xA;noise = randn(length(tspan), 1);&#xA;% apply filter to yield bandlimited noise&#xA;p = filter(b,1,noise);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I have to define the function for the ode, but I do not know how to give it the $p$ (white noise):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried coding this way&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;odefun = @(t,u,m,k1,c,p)[u(2); 1/m*(-k1*u(1)-c*u(2)+p)];&#xA;q0 = [0;0]; % Initial conditions&#xA;[t,q,te,qe,ie] = ode45(@(t,u)odefun(t,u,m,k1,c,p),tspan,q0(:,1),options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I get this error message:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@(T,U)ODEFUN(T,U,M,K1,C,NB) returns a vector of length 4002, but the length of initial&#xA;conditions vector is 2. The vector returned by @(T,U)ODEFUN(T,U,M,K1,C,NB) and the&#xA;initial conditions vector must have the same number of elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already had a look at the following questions but I do not understand where the problem could be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/14343563/how-solve-a-system-of-ordinary-differntial-equation-with-time-dependent-paramete?lq=1&quot;&gt;https://stackoverflow.com/questions/14343563/how-solve-a-system-of-ordinary-differntial-equation-with-time-dependent-paramete?lq=1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/19732442/solving-an-ode-when-the-function-is-given-as-discrete-values-matlab?lq=1&quot;&gt;https://stackoverflow.com/questions/19732442/solving-an-ode-when-the-function-is-given-as-discrete-values-matlab?lq=1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/19732442/solving-an-ode-when-the-function-is-given-as-discrete-values-matlab&quot;&gt;https://stackoverflow.com/questions/19732442/solving-an-ode-when-the-function-is-given-as-discrete-values-matlab&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might have found a way to do it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [y] = p(t)&#xA;    b = fir1(64, [10/(Fs/2) 25/(Fs/2)]);&#xA;    % generate Gaussian (normally-distributed) white noise&#xA;    noise = randn(length(t), 1);&#xA;    % apply filter to yield bandlimited noise&#xA;    y = filter(b,1,noise);&#xA;end&#xA;&#xA;odefun = @(t,u,m,k1,c,p)[u(2); 1/m*(-k1*u(1)-c*u(2)+p(t))];&#xA;&#xA;[t,q,te,qe,ie] = ode45(@(t,u)odefun2(t,u,m,k2,c,@p),tspan,q0(:,1),options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does it look fine to you?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-12-13T13:04:41.837" Title="How to solve an ode with stochastic time-dependent input" Tags="|matlab|ode|time-integration|stochastic|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16233" PostTypeId="1" CreationDate="2014-11-20T12:01:49.520" Score="0" ViewCount="173" Body="&lt;p&gt;everyone, I have a question about computational costs for a algorithm. That is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two vectors $u_n,\ v_n\in \mathbb{C}^N$, a matrix $A\in \mathbb{C}^{N\times N}$ (can be both sparse and dense) and scalar coefficient $\alpha_n$, than I need to compute (run by Matlab):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) $y_n = u_n + \alpha_nv_n$ (its computational cost (mainly CPU time) is referred to as $\mathcal{K}$);&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) matrix-vector product: $z_n = Au_n$ (its computational cost (mainly CPU time) is referred to as $\mathcal{L}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I want to know that we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) $5\mathcal{K}  &amp;gt;\mathcal{L}$ ??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) $5\mathcal{K}  &amp;lt;\mathcal{L}$ ??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the assumptions of 1)-2) rational ? I want to receive some comments about this problem.&lt;/p&gt;&#xA;" OwnerUserId="10290" LastEditorUserId="10290" LastEditDate="2014-11-20T12:12:52.087" LastActivityDate="2014-11-21T01:51:46.527" Title="Help me analyze the computational cost of two kinds of operations" Tags="|matlab|algorithms|complexity|dense-matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16234" PostTypeId="1" AcceptedAnswerId="16239" CreationDate="2014-11-20T12:03:59.487" Score="4" ViewCount="2138" Body="&lt;p&gt;I am trying to simulate a system with bilinear stiffness described by the following equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$25\ddot{x}+15\dot{x}+330000x = p(t)$ if $x &amp;lt; 0.00072$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$25\ddot{x}+15\dot{x}+930000x = p(t)$ if $x &amp;gt;= 0.00072$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p(t)$ is the white noise I already talked about in another question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16232/how-to-solve-an-ode-with-time-dependent-input&quot;&gt;How to solve an ode with stochastic time-dependent input&lt;/a&gt; . The transition point is called $d$ in the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I define the input and the equations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% design FIR filter to filter noise in the range [10-25] Hz&#xA;Fs = 1000; % sampling frequency&#xA;&#xA;function [y] = nb(t)&#xA;    b = fir1(64, [10/(Fs/2) 25/(Fs/2)]);&#xA;    % generate Gaussian (normally-distributed) white noise&#xA;    noise = randn(length(t), 1);&#xA;    % apply filter to yield bandlimited noise&#xA;    y = filter(b,1,noise);&#xA;end&#xA;&#xA;% State space functions ===================================================&#xA;odefun1 = @(t,u,m,k1,c,nb)[u(2); 1/m*(-k1*u(1)-c*u(2)+nb(t))];&#xA;odefun2 = @(t,u,m,k2,c,nb)[u(2); 1/m*(-k2*u(1)-(k1-k2)*d-c*u(2)+nb(t))];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I select the starting equation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Time infos&#xA;T = 1/Fs;&#xA;tspan = 0:T:7.7;&#xA;tstart = tspan(1);&#xA;tend = tspan(end);&#xA;&#xA;% Initial conditions&#xA;q0 = [0;0];&#xA;&#xA;% Choose starting equation&#xA;if q0(1,1) &amp;gt; d&#xA;    flag1 = 1;&#xA;else&#xA;    flag1 = 0;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And start the loop for integration:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Options for ODE solver&#xA;options = odeset('RelTol',1e-6,'AbsTol',1e-8,'Events',@events);&#xA;&#xA;% Accumulators for ODE solve output&#xA;tout = tstart;  % global solution time&#xA;qout = q0.';    % global solution position&#xA;teout = [];     % time at which events occur&#xA;qeout = [];     % position at which events occur&#xA;ieout = [];     % flags which event trigger the switch&#xA;&#xA;% Main loop&#xA;while tout(end) &amp;lt; tend&#xA;    if flag1 == 1&#xA;        [t,q,te,qe,ie] = ode45(@(t,u)odefun2(t,u,m,k2,c,@nb),tspan,q0(:,1),options);&#xA;        flag1 = 0;&#xA;    else&#xA;        [t,q,te,qe,ie] = ode45(@(t,u)odefun1(t,u,m,k1,c,@nb),tspan,q0(:,1),options);&#xA;        flag1 = 1;&#xA;    end&#xA;% accumulate data into accumulators vectors&#xA;nt = length(t);&#xA;tout = [tout; t(2:nt)];&#xA;qout = [qout; q(2:nt,:)];&#xA;teout = [teout; te]; % Events at tstart are never reported.&#xA;qeout = [qeout; qe];&#xA;ieout = [ieout; ie];&#xA;tspan = [t(end), tend]; % reset time span,'where you just ended to tend'&#xA;q0 = [q(end,1);q(end,2)]; % reset IC as where you just ended&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And finally I define the events:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [value,isterminal,direction] = events(t,q)&#xA;    value = [q(1)-d,q(1)-d]; % Detect zero of event functions&#xA;    isterminal = [1,1]; % Stop the integration&#xA;    direction = [-1,1]; % Direction&#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this the right way to do it?&#xA;Because I do not get a stiffness change when I plot the force vs displacement vectors in this way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;q2dot = zeros(size(qout(:,2))); % integrate veloocity to get acceleration&#xA;q2dot(1) = 0;&#xA;for i = 2:length(qout(:,2))&#xA;    h = tout(i)-tout(i-1);&#xA;    q2dot(i) = (qout(i,2)-qout(i-1,2))/h; &#xA;end&#xA;&#xA;RF = nb(t)-m*q2dot; % restoring force&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start thinking the problem is in the way I defined the white noise and I used it in the equations, because if I use a forcing function of the form $A \sin(\omega t)$ (with A high enough) I get the change in slope&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2020-12-10T00:25:42.320" Title="Switch between 2 equations to be given to ODE using events" Tags="|matlab|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16235" PostTypeId="2" ParentId="16233" CreationDate="2014-11-20T12:53:47.060" Score="1" Body="&lt;p&gt;Let the average number of sparse entries in a row for a sparse matrix be $N_s$. Then the complexity of the matrix-vector product ($z_n = Au_n$) is approximately $O(NN_s)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complexity of $y_n = u_n + \alpha_nv_n$ is $O(2N)$. For the second to hold true&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$5\mathcal{K}  &amp;lt;\mathcal{L}$, we need&#xA;$5(2N)$ &amp;lt; $NN_s$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if $N_s &amp;gt; 10$, i.e.,  the second condition holds true. &lt;/p&gt;&#xA;" OwnerUserId="11644" LastActivityDate="2014-11-20T12:53:47.060" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16236" PostTypeId="1" CreationDate="2014-11-20T15:12:03.790" Score="0" ViewCount="287" Body="&lt;p&gt;I have been using MAX-SAT solver to obtain the exact ground state of ising spin glass model: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 1D periodic model, for systems with 50 binary variables and interaction range of 15th nearest neighbor. The best MAX-sat solver requires 20 minutes to solve it. For 100 binary, it then becomes impossible... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it seems when people use graph theory max cut methods to solve this problem, they could obtain ground state of much larger scale. But I am wondering if I transform my ising spin glass problem into the format of graph cuts, would graph cut solvers perform in any way better? Is that the case that people who construct theorems for maxsat do not consider its graph cut equivalent method? For your interest, the case of my 50/100 variables problem is as followed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/23lo7lwgd4otse9/50_softer.wcnf?dl=0&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/s/23lo7lwgd4otse9/50_softer.wcnf?dl=0&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.dropbox.com/s/r2r1yfpb2uzvcrn/100_softer.wcnf?dl=0&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/s/r2r1yfpb2uzvcrn/100_softer.wcnf?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(you may see there are 64, 114 variables, but the hard constraints force the equality of the first 15th clauses with the last 15 clauses )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your information, in most cases, the solver finds the exact ground state fast but the proving process is very slow (e.g. it takes 1 second to get the true solution while using 20 minutes proving that)...&lt;/p&gt;&#xA;" OwnerUserId="7891" LastActivityDate="2014-11-21T03:49:45.187" Title="MAX-SAT and MAX-cut" Tags="|algorithms|linear-programming|computational-physics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16238" PostTypeId="2" ParentId="16227" CreationDate="2014-11-20T23:42:27.253" Score="0" Body="&lt;p&gt;I will try to give one quick answer to my 2. question, based on Geoff's accepted answer. Geoff suggested to use a polynomial approximation scheme to get feasible solutions (which in turn can speed-up the branch-and-bound process). One such scheme which will produce a quite high quality solution in $O(nlogn)$ operations ($n$ being the number of binary variables in the original problem)  is the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Merge the groups, i.e. discard all the group information and pool all $x_i(j)$'s together.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sort the above list.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now the problem has been reduced to the case $K=1$. Just one run through the list is necessary in order to determine the best solution of this problem which has objective value $r$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We now look at the $r$-lowest $x_i(j)$'s. By counting how many of these correspond to each of the initial groups, we can get the $r_i$ values. This is a feasible solution and in most cases of quite high quality.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11508" LastActivityDate="2014-11-20T23:42:27.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16239" PostTypeId="2" ParentId="16234" CreationDate="2014-11-20T23:53:28.570" Score="6" Body="&lt;p&gt;You're on the right track and are correct to be using events for this purpose, but you're doing several things wrong that will result in an incorrect output.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You alway use the same initial conditions. I don't know your system, but it looks like you're trying to continuously integrate over time. When you switch from one set of ODEs to the other you need to specify the new initial conditions as the last state of of the previous equations.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You're doing something very similar with the &lt;code&gt;tspan&lt;/code&gt; vector. This is particularly important as it looks like your ODEs are time dependent.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You're overwriting the values of &lt;code&gt;t&lt;/code&gt;, &lt;code&gt;q&lt;/code&gt;, etc. output by &lt;code&gt;ode45&lt;/code&gt; when you switch. Or is that not shown (&lt;code&gt;% accumulate data into accumulators vectors&lt;/code&gt; comment)? It hard to help based on incomplete non-runnable code.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Based on the large stiffness values in your second order equations, you may want to look into using a &lt;a href=&quot;http://en.wikipedia.org/wiki/Stiff_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stiff&lt;/a&gt; solver. &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ode15s.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ode15s&lt;/code&gt;&lt;/a&gt; is usually a good starting point to compare to what you got with a general solver like &lt;code&gt;ode45&lt;/code&gt;. Cleve Moler recently wrote &lt;a href=&quot;http://blogs.mathworks.com/cleve/2014/06/09/ordinary-differential-equations-stiffness/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an excellent blog post&lt;/a&gt; detailing aspects of stiffness and the methods available in Matlab.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you care about detailed statistical properties of your results, you probably shouldn't be using any adaptive scheme or solver designed for ODEs if you're you're working with noise. Using such methods may change the color of your noise and/or the resultant probability distributions of your outputs. You have a &lt;a href=&quot;http://en.wikipedia.org/wiki/Stochastic_differential_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic differential equation&lt;/a&gt; (SDE). I refer you to &lt;a href=&quot;http://journals.aps.org/pre/abstract/10.1103/PhysRevE.74.068701&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which warns agains using ODE methods for SDEs. There are many cases where one can get away with what you're doing, but others that will lead incorrect results.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Though the example doesn't use events (the switching is based on time rather than state), see &lt;a href=&quot;https://stackoverflow.com/a/17370733/2278029&quot;&gt;my answer here on StackOverflow.com/Matlab&lt;/a&gt; for how you might do 1–3 above.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="4474" LastEditDate="2020-12-10T00:25:42.320" LastActivityDate="2020-12-10T00:25:42.320" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16240" PostTypeId="2" ParentId="16220" CreationDate="2014-11-21T00:15:43.487" Score="3" Body="&lt;p&gt;A few things to note:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;By definition &lt;code&gt;A·v = λ·v&lt;/code&gt;, eigenvectors are &lt;em&gt;&lt;a href=&quot;https://stackoverflow.com/a/13041293/97160&quot;&gt;not unique&lt;/a&gt;&lt;/em&gt;. You can multiply by any constant and still get another valid eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/a/18152804/97160&quot;&gt;The convention in MATLAB&lt;/a&gt; is that for &lt;code&gt;eig(A)&lt;/code&gt;, the eigenvectors are scaled so that the norm of each is 1.0, and for &lt;code&gt;eig(A,B)&lt;/code&gt;, the eigenvectors are not normalized (&lt;a href=&quot;https://stackoverflow.com/a/6748385/97160&quot;&gt;see here&lt;/a&gt; for an example). Here is the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/eig.html#outputarg_V&quot; rel=&quot;nofollow noreferrer&quot;&gt;relevant part in the documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;V&lt;/code&gt;: right eigenvectors, returned as a square matrix whose columns are&#xA;  the right eigenvectors of &lt;code&gt;A&lt;/code&gt; or generalized right eigenvectors of the&#xA;  pair &lt;code&gt;(A,B)&lt;/code&gt;. The form and normalization of &lt;code&gt;V&lt;/code&gt; depends on the&#xA;  combination of input arguments:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;code&gt;[V,D] = eig(A)&lt;/code&gt; returns matrix &lt;code&gt;V&lt;/code&gt;, whose columns are the right eigenvectors of &lt;code&gt;A&lt;/code&gt; such that &lt;code&gt;A*V = V*D&lt;/code&gt;. The eigenvectors in &lt;code&gt;V&lt;/code&gt; are&#xA;  normalized so that the 2-norm of each is 1.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;code&gt;[V,D] = eig(A,'nobalance')&lt;/code&gt; also returns matrix &lt;code&gt;V&lt;/code&gt;. However, the 2-norm of each eigenvector is not necessarily 1.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;code&gt;[V,D] = eig(A,B)&lt;/code&gt; and &lt;code&gt;[V,D] = eig(A,B,algorithm)&lt;/code&gt; returns &lt;code&gt;V&lt;/code&gt; as a matrix whose columns are the generalized right eigenvectors that&#xA;  satisfy &lt;code&gt;A*V = B*V*D&lt;/code&gt;. The 2-norm of each eigenvector is not&#xA;  necessarily 1. In this case, &lt;code&gt;D&lt;/code&gt; contains the generalized eigenvalues&#xA;  of the pair &lt;code&gt;(A,B)&lt;/code&gt;, along the main diagonal.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If &lt;code&gt;A&lt;/code&gt; is symmetric and &lt;code&gt;B&lt;/code&gt; is symmetric positive definite, then the eigenvectors in &lt;code&gt;V&lt;/code&gt; are normalized so that the 2-norm of each is&#xA;  1.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In addition, eigenvalues are &lt;em&gt;&lt;a href=&quot;https://stackoverflow.com/a/13704582/97160&quot;&gt;not&lt;/a&gt; &lt;a href=&quot;https://stackoverflow.com/a/3293855/97160&quot;&gt;sorted&lt;/a&gt;&lt;/em&gt;. You are only guaranteed that the columns of &lt;code&gt;V&lt;/code&gt; are the corresponding right eigenvectors to the eigenvalues in &lt;code&gt;D&lt;/code&gt;. That's not the same as &lt;code&gt;svd&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, &lt;a href=&quot;https://math.stackexchange.com/q/492890/133&quot;&gt;there is no total ordering of complex numbers&lt;/a&gt;. The convention in MATLAB is that the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/sort.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sort&lt;/code&gt; function&lt;/a&gt; sorts complex elements first by magnitude (i.e. &lt;code&gt;abs(x)&lt;/code&gt;), then by phase angle on the &lt;code&gt;[-pi,pi]&lt;/code&gt; interval (i.e. &lt;code&gt;angle(x)&lt;/code&gt;) if magnitudes are equal.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Having considered the above, when solving the generalized eigenvalue problem, you should be getting similar results to MATLAB by using DGGEV routine in LAPACK.&lt;/p&gt;&#xA;" OwnerUserId="386" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-11-21T00:15:43.487" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16241" PostTypeId="2" ParentId="14825" CreationDate="2014-11-21T00:57:07.397" Score="4" Body="&lt;p&gt;A few additional points I would like to add to other answers.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Corner test cases should be part of the regression test suite - ill conditioned problems, ill conditioned - bad aspect ratios, orthotropic and  un-isotropic material properties, improperly constrained models.&lt;/li&gt;&#xA;&lt;li&gt;Make sure reaction forces match.&lt;/li&gt;&#xA;&lt;li&gt;Sturm checks for eigenvalue problems especially when there are multiple eigenvalues.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In case of iterative solvers, one might get the correct answer but numerical bugs, or incorrect pre-conditioner construction/factoringmight make the solver slow to converge. So performance needs to be compared with the old code for the same stopping tolerance setting.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stopping tolerance needs to be tightened progressively and number of iterations noted.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Paralle and GPU: Performance scaling for number of threads, cache sizes, etc. need to be cross-checked and should be as expected.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;PS:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;7&quot;&gt;&#xA;&lt;li&gt;Write a short and simple matrix-vector multiply and check the norm ||Ax*-b|| where x* is the answer returned by the solver. This has to be simple and short so that its correctness can be verified in an hour or two by doing a code walk-through with the help of a couple of co-workers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9394" LastEditorUserId="9394" LastEditDate="2014-11-21T01:02:48.990" LastActivityDate="2014-11-21T01:02:48.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16252" PostTypeId="2" ParentId="16236" CreationDate="2014-11-21T03:49:45.187" Score="2" Body="&lt;p&gt;In most cases, when computing graph cuts you are not interested in finding the optimal graph cut. This is true, for example, in domain decomposition of finite element meshes. In such situations, a &quot;good&quot; cut is often good enough. You can then use approximation algorithms and heuristics that will in general not yield the exact answer but one that is close enough. This is what, for example, the METIS package does when computing graph cuts, as well as practically all other similar packages.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-21T03:49:45.187" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16253" PostTypeId="1" CreationDate="2014-11-21T09:36:57.147" Score="0" ViewCount="436" Body="&lt;p&gt;I have a large sparse matrix A and i want to pick the submatrix of it to do block jacobi iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the blocks, i have get a matrix Q that contains the index of nonzero entries in its Jth column define the nodes of Jth block.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:J&#xA;lg=find(Q(level).agg(:,i));&#xA;D(i).matrix=inv((sparse(A(lg,lg))));&#xA;end&#xA;D_inverse=sparse(blkdiag(D(1:J).matrix));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I found that the operations A(lg,lg) cost much time, because A is sparse format,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other ways to do this purpose (building D_inverse)quickly and efficently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="11856" LastActivityDate="2014-11-21T09:36:57.147" Title="how to pick a submatrix of a sparse matrix quickly(matlab)" Tags="|matlab|sparse-matrix|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16254" PostTypeId="1" CreationDate="2014-11-21T20:35:39.510" Score="1" ViewCount="186" Body="&lt;p&gt;I am currently running a linear program with MATLAB to determine, by the exitflag of linprog, if two rotated and shifted hypercubes have nonempty intersection. I wondered if this is a waste of time, or if linprog does nothing more than checking feasibility before starting the optimization.&lt;/p&gt;&#xA;" OwnerUserId="7789" LastEditorUserId="7789" LastEditDate="2014-11-21T23:27:10.220" LastActivityDate="2014-11-21T23:27:10.220" Title="Checking if convex polytope is nonempty" Tags="|computational-geometry|convex-optimization|linear-programming|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16255" PostTypeId="2" ParentId="7248" CreationDate="2014-11-21T20:38:55.223" Score="0" Body="&lt;p&gt;In the title you ask about solving flow past a cylinder on a cartesian grid, but then in the question you talk about what seem to be unstructured grids.  If you want to do finite differences on unstructured grids you should take a look at Discrete Exterior Calculus, there is a Python library called &lt;a href=&quot;https://code.google.com/p/pydec/&quot; rel=&quot;nofollow&quot;&gt;PyDEC&lt;/a&gt; that may be of use to you.&lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2014-11-21T20:38:55.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16256" PostTypeId="2" ParentId="16166" CreationDate="2014-11-21T20:49:32.167" Score="1" Body="&lt;p&gt;There are some links related to the bounded-degree graph isomorphism algorithm in the thread &quot;&lt;a href=&quot;https://cstheory.stackexchange.com/a/12388&quot;&gt;Gentle introduction to graph isomorphism for bounded valance graphs&lt;/a&gt;&quot; on the cstheory site. There, Joshua Grochow points out that the algorithm is closely tied to permutation groups and can be hard to understand if one doesn't have a strong background in group theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since you're willing to ignore graphs with cycles, then you're just working with (unrooted) trees, and testing for isomorphism between those should be much easier. Wikipedia cites P. J. Kelly's &quot;&lt;a href=&quot;http://projecteuclid.org/euclid.pjm/1103043674&quot; rel=&quot;nofollow noreferrer&quot;&gt;A congruence theorem for trees&lt;/a&gt;&quot; (1957), which I don't understand, and another algorithm is given in Marthe Bonamy's &quot;&lt;a href=&quot;http://perso.ens-lyon.fr/eric.thierry/Graphes2010/marthe-bonamy.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A small report on graph and tree isomorphism&lt;/a&gt;&quot; (2010).&lt;/p&gt;&#xA;" OwnerDisplayName="user3883" LastEditorUserId="-1" LastEditDate="2017-04-13T12:32:32.353" LastActivityDate="2014-11-21T20:49:32.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16257" PostTypeId="2" ParentId="7616" CreationDate="2014-11-22T06:07:20.403" Score="4" Body="&lt;p&gt;Unfortunately Wolfgang Bangerth's solution isn't guaranteed to work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q8ytn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what you can do instead: Pick a direction, and move along it until you hit a hyperplane. Pick a direction along the hyperplane and move along it until you hit another. Pick a direction along the intersection of the two hyperplanes... After $i$ steps, you will have $i$ active constraints, and you will pick a direction along the $(n-i)$-dimensional null space. After $n$ steps, you will arrive at a vertex (assuming the feasible set is bounded).&lt;/p&gt;&#xA;" OwnerDisplayName="user3883" LastActivityDate="2014-11-22T06:07:20.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16258" PostTypeId="1" CreationDate="2014-11-22T11:18:48.367" Score="0" ViewCount="115" Body="&lt;p&gt;Currently I am working on my research report where I've done a lot of measurements. For now I have uploaded everything into Matlab and visualise each on chart. Each one consists of 4 lines drawn with different color. As you may see in the picture below they are quite similar to each other:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Figmb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just a part of my measurements and I can't add all off them into my work. Moreover the whole bunch of them don't show explicitly the similarity between them. What I want to achieve is to have one chart that is somehow the average of all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem that I see now is different number of samples in each chart (x-axis) and different scale (y-axis). So I would be grateful for the advice how to cope with this. I can't do simple mean of all of the results because of those two problems. What would be the best method to display all the results as a one chart?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;I totally forget to explain what are the values. All colors beside black represent readings from different axis of accelerometer. So red is X, blue is Y and green i Z. The black one is vector sum of all 3 axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once again, to clarify I want to have all such readings on all chart. So I will have all of those charts combined and displayed as only one chart, where there will be 4 colors still. I expect to have some of readings flatten, some smoothed and so on.&lt;/p&gt;&#xA;" OwnerUserId="11866" LastEditorUserId="11866" LastEditDate="2014-11-23T11:11:40.630" LastActivityDate="2014-11-23T11:11:40.630" Title="What is the best method to combine multiple vectored results into one chart?" Tags="|matlab|vector|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16259" PostTypeId="2" ParentId="16258" CreationDate="2014-11-22T17:29:54.583" Score="1" Body="&lt;p&gt;If you know how the x-axis relates from one dataset to the next, the best solution I can think of would be to use the command 'interp1' in matlab. I've compiled a small script with output to show you some example usage. Essentially what you want to do is remap your data from each dataset so that they share some common axis, which you can do with 'interp1' in the same way I have done in my example.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;close all&#xA;&#xA;x1 = 1:10; %Some data size 10&#xA;y1 = x1.^2; %Some data size 10, y1(x1)&#xA;&#xA;x2 = linspace(1,10,20); %Some data size 20, with different x-axis&#xA;y2 = x2.^2; %Some data size 20, with different x-axis, y2(x2)&#xA;&#xA;y2interp=interp1(x2,y2,x1); %Interpolate y2 data onto x1 axis&#xA;&#xA;subplot(1,2,1);plot(x1,y1,'bo-',x2,y2,'r+-');&#xA;legend('y1','y2','Location','North')&#xA;title('Original data for y1 and y2')&#xA;subplot(1,2,2);plot(x1,y1,'bo-',x1,y2interp,'r+-');&#xA;legend('y1','y2','Location','North')&#xA;title('Interpolated data for y2 plotted over y1')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FXyhZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11799" LastActivityDate="2014-11-22T17:29:54.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16260" PostTypeId="1" AcceptedAnswerId="16261" CreationDate="2014-11-22T18:29:04.257" Score="5" ViewCount="3089" Body="&lt;p&gt;I want to write a 2D FEA code for my course project and I need to import a mesh (2d, simple quad/tri) on a simple geometry such as a L shaped plate or with a square/circular hole in it, something like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I want to write my own code, I need to read the mesh file in my code hence looking for a simple format which specifies element connectivity and help identify boundary nodes (to apply boundary conditions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will be really great if you can point out some easy to use/understand tool for this purpose with good documentation on mesh format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you,&#xA;Pranav&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GMSH is perfectly suited for the use I intended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Older Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this tentative workaround for the problem but is certainly not a solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important part of the file is the information of nodes on corresponding boundaries. I'm using ANSYS DesignModeler and Static structural module in ANSYS Workbench. I'm creating the geometry in DesignModeler and meshing it in ANSYS Meshing which is inside static structural module. You can specify boundaries using Named Selection  so that you can apply loads later. After generating the mesh, you can export the input file for an older ANSYS APDL from Tools menu which export a ASCII &quot;.dat&quot; file which has the mesh information and all the problem setup. I removed all unnecessary details to get following results. Now I can easily write a function to read this file. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Nodes&#xA;        1    0.000000000E+000    1.000000000E+001    0.000000000E+000&#xA;        2    0.000000000E+000    0.000000000E+000    0.000000000E+000&#xA;        3    0.000000000E+000    8.888888889E+000    0.000000000E+000&#xA;        4    0.000000000E+000    7.777777778E+000    0.000000000E+000&#xA;        5    0.000000000E+000    6.666666667E+000    0.000000000E+000&#xA;        6    0.000000000E+000    5.555555556E+000    0.000000000E+000&#xA;        7    0.000000000E+000    4.444444444E+000    0.000000000E+000&#xA;        8    0.000000000E+000    3.333333333E+000    0.000000000E+000&#xA;        9    0.000000000E+000    2.222222222E+000    0.000000000E+000&#xA;       10    0.000000000E+000    1.111111111E+000    0.000000000E+000&#xA;       11    5.000000000E+000    0.000000000E+000    0.000000000E+000&#xA;       12    1.000000000E+000    0.000000000E+000    0.000000000E+000&#xA;       13    2.000000000E+000    0.000000000E+000    0.000000000E+000&#xA;       14    3.000000000E+000    0.000000000E+000    0.000000000E+000&#xA;       15    4.000000000E+000    0.000000000E+000    0.000000000E+000&#xA;       16    5.000000000E+000    5.000000000E+000    0.000000000E+000&#xA;       17    5.000000000E+000    1.000000000E+000    0.000000000E+000&#xA;       18    5.000000000E+000    2.000000000E+000    0.000000000E+000&#xA;       19    5.000000000E+000    3.000000000E+000    0.000000000E+000&#xA;       20    5.000000000E+000    4.000000000E+000    0.000000000E+000&#xA;       21    1.000000000E+001    5.000000000E+000    0.000000000E+000&#xA;       22    6.000000000E+000    5.000000000E+000    0.000000000E+000&#xA;       23    7.000000000E+000    5.000000000E+000    0.000000000E+000&#xA;       24    8.000000000E+000    5.000000000E+000    0.000000000E+000&#xA;       25    9.000000000E+000    5.000000000E+000    0.000000000E+000&#xA;       26    1.000000000E+001    1.000000000E+001    0.000000000E+000&#xA;       27    1.000000000E+001    6.000000000E+000    0.000000000E+000&#xA;       28    1.000000000E+001    7.000000000E+000    0.000000000E+000&#xA;       29    1.000000000E+001    8.000000000E+000    0.000000000E+000&#xA;       30    1.000000000E+001    9.000000000E+000    0.000000000E+000&#xA;       31    8.888888889E+000    1.000000000E+001    0.000000000E+000&#xA;       32    7.777777778E+000    1.000000000E+001    0.000000000E+000&#xA;       33    6.666666667E+000    1.000000000E+001    0.000000000E+000&#xA;       34    5.555555556E+000    1.000000000E+001    0.000000000E+000&#xA;       35    4.444444444E+000    1.000000000E+001    0.000000000E+000&#xA;       36    3.333333333E+000    1.000000000E+001    0.000000000E+000&#xA;       37    2.222222222E+000    1.000000000E+001    0.000000000E+000&#xA;       38    1.111111111E+000    1.000000000E+001    0.000000000E+000&#xA;       39    3.100885795E+000    6.940821089E+000    0.000000000E+000&#xA;       40    3.883906237E+000    7.027860768E+000    0.000000000E+000&#xA;       41    3.017869238E+000    6.161626033E+000    0.000000000E+000&#xA;       42    4.850074589E+000    7.040770750E+000    0.000000000E+000&#xA;       43    5.864808320E+000    7.043474312E+000    0.000000000E+000&#xA;       44    6.897973722E+000    7.037079226E+000    0.000000000E+000&#xA;       45    4.014156390E+000    6.025411191E+000    0.000000000E+000&#xA;       46    2.984856810E+000    5.187710894E+000    0.000000000E+000&#xA;       47    2.963924128E+000    4.159491948E+000    0.000000000E+000&#xA;       48    2.954305198E+000    3.113634563E+000    0.000000000E+000&#xA;       49    2.966833201E+000    2.070582688E+000    0.000000000E+000&#xA;       50    1.954872141E+000    3.206756202E+000    0.000000000E+000&#xA;       51    1.964133590E+000    4.280955357E+000    0.000000000E+000&#xA;       52    1.987442885E+000    5.363330639E+000    0.000000000E+000&#xA;       53    2.037966141E+000    6.460608169E+000    0.000000000E+000&#xA;       54    2.369722555E+000    7.667980533E+000    0.000000000E+000&#xA;       55    3.585407332E+000    7.997051341E+000    0.000000000E+000&#xA;       56    4.672581651E+000    8.037841747E+000    0.000000000E+000&#xA;       57    5.733421173E+000    8.044844080E+000    0.000000000E+000&#xA;       58    6.801325531E+000    8.036846126E+000    0.000000000E+000&#xA;       59    7.868898928E+000    8.028595372E+000    0.000000000E+000&#xA;       60    7.925684523E+000    7.023684703E+000    0.000000000E+000&#xA;       61    1.963418650E+000    2.132912786E+000    0.000000000E+000&#xA;       62    4.968222423E+000    6.024805920E+000    0.000000000E+000&#xA;       63    5.950275027E+000    6.023627968E+000    0.000000000E+000&#xA;       64    6.960337601E+000    6.022422539E+000    0.000000000E+000&#xA;       65    7.971188476E+000    6.011697947E+000    0.000000000E+000&#xA;       66    8.927457869E+000    8.014515675E+000    0.000000000E+000&#xA;       67    8.961487394E+000    7.010908471E+000    0.000000000E+000&#xA;       68    2.983632029E+000    1.037919977E+000    0.000000000E+000&#xA;       69    1.984671847E+000    1.070964476E+000    0.000000000E+000&#xA;       70    9.864873052E-001    1.087871232E+000    0.000000000E+000&#xA;       71    9.783216106E-001    2.183930277E+000    0.000000000E+000&#xA;       72    9.782477164E-001    3.284728024E+000    0.000000000E+000&#xA;       73    9.781132470E-001    4.382433521E+000    0.000000000E+000&#xA;       74    9.974305756E-001    5.503140644E+000    0.000000000E+000&#xA;       75    1.072256462E+000    6.644678490E+000    0.000000000E+000&#xA;       76    1.150918303E+000    7.757754861E+000    0.000000000E+000&#xA;       77    1.153760333E+000    8.870140771E+000    0.000000000E+000&#xA;       78    2.272715412E+000    8.874785514E+000    0.000000000E+000&#xA;       79    3.378616377E+000    8.955660205E+000    0.000000000E+000&#xA;       80    4.521905029E+000    9.017703001E+000    0.000000000E+000&#xA;       81    5.627472506E+000    9.027073068E+000    0.000000000E+000&#xA;       82    6.720590452E+000    9.025172424E+000    0.000000000E+000&#xA;       83    7.808554241E+000    9.018864371E+000    0.000000000E+000&#xA;       84    8.908424832E+000    9.011734730E+000    0.000000000E+000&#xA;       85    8.986541286E+000    6.005180504E+000    0.000000000E+000&#xA;       86    3.993993695E+000    5.054644401E+000    0.000000000E+000&#xA;       87    3.975665129E+000    4.060554879E+000    0.000000000E+000&#xA;       88    3.974355784E+000    3.051025159E+000    0.000000000E+000&#xA;       89    3.980336212E+000    2.026422574E+000    0.000000000E+000&#xA;       90    3.990475831E+000    1.010320637E+000    0.000000000E+000&#xA;-1&#xA;&#xA;#Elements&#xA;        1       39       54       53       41&#xA;        2       39       40       55       54&#xA;        3       41       45       40       39&#xA;        4       45       62       42       40&#xA;        5       41       46       86       45&#xA;        6       40       42       56       55&#xA;        7       53       52       46       41&#xA;        8       42       43       57       56&#xA;        9       42       62       63       43&#xA;       10       43       44       58       57&#xA;       11       43       63       64       44&#xA;       12       44       60       59       58&#xA;       13       44       64       65       60&#xA;       14       86       16       62       45&#xA;       15       52       51       47       46&#xA;       16       47       87       86       46&#xA;       17       51       50       48       47&#xA;       18       48       88       87       47&#xA;       19       50       61       49       48&#xA;       20       49       89       88       48&#xA;       21       49       68       90       89&#xA;       22       61       69       68       49&#xA;       23       50       72       71       61&#xA;       24       51       73       72       50&#xA;       25       52       74       73       51&#xA;       26       53       75       74       52&#xA;       27       54       76       75       53&#xA;       28       54       78       77       76&#xA;       29       55       79       78       54&#xA;       30       56       80       79       55&#xA;       31       57       81       80       56&#xA;       32       58       82       81       57&#xA;       33       59       83       82       58&#xA;       34       59       66       84       83&#xA;       35       60       67       66       59&#xA;       36       65       85       67       60&#xA;       37       71       70       69       61&#xA;       38       62       16       22       63&#xA;       39       63       22       23       64&#xA;       40       64       23       24       65&#xA;       41       65       24       25       85&#xA;       42       66       29       30       84&#xA;       43       67       28       29       66&#xA;       44       85       27       28       67&#xA;       45       68       14       15       90&#xA;       46       69       13       14       68&#xA;       47       70       12       13       69&#xA;       48       70       10        2       12&#xA;       49       71        9       10       70&#xA;       50       72        8        9       71&#xA;       51       73        7        8       72&#xA;       52       74        6        7       73&#xA;       53       75        5        6       74&#xA;       54       76        4        5       75&#xA;       55       77        3        4       76&#xA;       56       77       38        1        3&#xA;       57       78       37       38       77&#xA;       58       79       36       37       78&#xA;       59       80       35       36       79&#xA;       60       81       34       35       80&#xA;       61       82       33       34       81&#xA;       62       83       32       33       82&#xA;       63       84       31       32       83&#xA;       64       84       30       26       31&#xA;       65       85       25       21       27&#xA;       66       87       20       16       86&#xA;       67       88       19       20       87&#xA;       68       89       18       19       88&#xA;       69       90       17       18       89&#xA;       70       90       15       11       17&#xA;-1&#xA;&#xA;#NamedSelection&#xA;BOTTOM  NODE     6&#xA;         2        11        12        13        14        15&#xA;&#xA;&#xA;#NamedSelection&#xA;TOP    NODE       10&#xA;         1        26        31        32        33        34        35        36&#xA;        37        38&#xA;&#xA;#NamedSelection&#xA;RIGHT_SIDE  NODE      6&#xA;        21        26        27        28        29        30&#xA;&#xA;&#xA;#End&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9950" LastEditorUserId="9950" LastEditDate="2015-02-25T18:08:15.220" LastActivityDate="2015-02-26T09:51:56.427" Title="Need a simple mesh format (for FEA) and a tool to generate the mesh" Tags="|finite-element|mesh-generation|mesh|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16261" PostTypeId="2" ParentId="16260" CreationDate="2014-11-22T18:38:52.387" Score="10" Body="&lt;p&gt;I would recommend gmsh. I have just started working with this program actually only a few days ago. But it is straight-forward to use. You can create various 2D and even 3D-geometries and it offers a ton of information, boundary nodes, etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a link to the website: &lt;a href=&quot;http://geuz.org/gmsh/&quot;&gt;http://geuz.org/gmsh/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have many useful references, there is a manual of course and also a mailing list archive that I have to be somewhat useful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically as I understand, you can use the program in a CAD-Like way, i.e. drawing points, lines. Or you can program the information in by hand. I have taking the programming route. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is another good reference to get started:&#xA;&lt;a href=&quot;https://github.com/FluidityProject/fluidity/wiki/Gmsh-tutorial&quot;&gt;https://github.com/FluidityProject/fluidity/wiki/Gmsh-tutorial&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&#xA;Alright, well the node locations and element information can be found in a .txt file by doing the following. Once gmsh is up and running, with a .msh file open(I'm using 2.8.4) go to the &lt;strong&gt;Geometry&lt;/strong&gt; tab and then &lt;strong&gt;Edit file&lt;/strong&gt;. This opens a .txt with the information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have yet to figure out how to interpret the element information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="10225" LastEditorUserId="10225" LastEditDate="2014-11-28T16:57:35.820" LastActivityDate="2014-11-28T16:57:35.820" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="16262" PostTypeId="2" ParentId="16260" CreationDate="2014-11-22T18:51:35.267" Score="4" Body="&lt;p&gt;If you have access to MATLAB, you might consider using PDE Toolbox to generate your geometry and mesh:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/index.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/pde/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is very easy to generate simple geometries like the ones you describe by doing&#xA;boolean operations on primitive shapes. The output mesh is described by three MATLAB arrays: node locations, element connectivity, and a matrix of boundary edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/pde/ug/initmesh.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/pde/ug/initmesh.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-11-22T18:51:35.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16264" PostTypeId="1" AcceptedAnswerId="16296" CreationDate="2014-11-22T21:03:19.137" Score="3" ViewCount="523" Body="&lt;p&gt;I am a physicist with limited numerical methods knowledge and I am trying to speed up the inversion of a very ill-conditioned problem ($rcond&amp;gt;10^{30}$). The same sparse square matrix is used repeatedly, so our current setup relies on a one time LU factorisation and a triangular solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have at my disposal a very powerful GPU and would like to use it (using CUDA with Cusparse, Magma...). My current understanding is that triangular solver are very sequential by nature and do not work well on parallel SIMD architecture, but iterative solver on the other hand do work well on GPU, sadly my problem is far to ill-condtionned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the matrix is used a large number of time, taking some time to compute a preconditioner seems like the way to go. I am therefore looking for ways to design a preconditioner suitable for parallel solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you guys want to take look at it, i have included a link to the kind of matrix I am working with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/file/d/0B7i4KTiB3ryaMng1SnFNZ1R3NzA/view?usp=sharing&quot; rel=&quot;nofollow&quot;&gt;Big Matrix&lt;/a&gt; &lt;br&gt;&#xA;size=114708*114708&lt;br&gt;&#xA;One-based_numbering&lt;br&gt;&#xA;34Mb&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/file/d/0B7i4KTiB3ryaMDJyLW1lbFpBcEE/view?usp=sharing&quot; rel=&quot;nofollow&quot;&gt;Small Matrix&lt;/a&gt;&lt;br&gt;&#xA;size=12890*12890&lt;br&gt;&#xA;One-based_numbering&lt;br&gt;&#xA;3Mb&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also a script to load the matrix with Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n=12890;&#xA;%n=114708;&#xA;fileID = fopen('SmallMatrix.txt','r');&#xA;%fileID = fopen('BigMatrix.txt','r');&#xA;dataArray = textscan(fileID,'%f%f%f%[^\n\r]', 'Delimiter',' ', 'MultipleDelimsAsOne', true);&#xA;fclose(fileID);&#xA;U1 = dataArray{:, 1};&#xA;U2 = dataArray{:, 2};&#xA;V = dataArray{:, 3};&#xA;s=sparse(U1,U2,V,n,n);&#xA;figure;&#xA;spy(s);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: Despite the very high condition number, the direct solvers I have used (matlab, mumps,umfpack and superlu) have no problem solving it. In matlab,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cond(s)&#xA;x=rand(n,1);&#xA;y=s\x;&#xA;norm(s*y-x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Returns,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.4996e+34&#xA;2.7318e-10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit2: Here is the kind of structure my matrix might have (Very big value are always on the diagonal): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{pmatrix}&#xA;0.1 &amp;amp; 2.1 &amp;amp; 3.2 \\&#xA;4.1 &amp;amp; -1.1 &amp;amp; 21\\&#xA;8 &amp;amp;-0.2 &amp;amp; 10^{30} &#xA;\end{pmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the the RHS:&#xA;\begin{pmatrix}&#xA;   3.1 \\&#xA;   4 \\&#xA;   0.3 &#xA;\end{pmatrix}&#xA;And the solution is:&#xA;\begin{pmatrix}&#xA;    1.3544 \\&#xA;    1.4117 \\&#xA;    0&#xA;\end{pmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is very Ill-conditionned but every direct solver is completely okay with it. It is just a easy way to impose boundary conditions (not very graceful, i have to admit)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit3: Following Brian Borchers and Tobias advice, I have removed from my matrix and my RHS all the rows and columns where my very large values appear. It works great in the sense that the solution to a given RHS is the same and the condition number is a more reasonable $10^{5}$. &#xA;Here is the script used to do that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%Creating the new matrix&#xA;sCopy=s;&#xA;nRemoved=nnz(abs(sCopy)&amp;gt;10^29);&#xA;for index=1:n-nRemoved&#xA;    while abs(sCopy(index,index))&amp;gt;10^29&#xA;        sCopy(index,:)=[];   &#xA;        sCopy(:,index)=[];  &#xA;    end&#xA;end&#xA;nnz(abs(sCopy)&amp;gt;10^29)&#xA;&#xA;%Updating the  RHS&#xA;[i,j,]=find(abs(s)&amp;gt;10^29);&#xA;NewRHS=RHS;&#xA;NewRHS(i)=[];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With that out of the way, my main interrogation remains, the condition number is still too high for a iterative method to converge.&lt;/p&gt;&#xA;" OwnerUserId="11869" LastEditorUserId="11869" LastEditDate="2014-11-23T13:21:07.663" LastActivityDate="2014-11-25T14:34:20.633" Title="Designing a preconditioner for a very Ill-conditionned matrix" Tags="|parallel-computing|iterative-method|gpu|preconditioning|condition-number|" AnswerCount="1" CommentCount="16" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16265" PostTypeId="1" AcceptedAnswerId="16280" CreationDate="2014-11-23T04:17:43.337" Score="3" ViewCount="314" Body="&lt;p&gt;I happened to know that there are advanced established techniques for non-smooth convex optimization in research. For example, these two papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Nesterov, &quot;&lt;a href=&quot;http://dx.doi.org/10.1007/s10107-004-0552-5&quot; rel=&quot;nofollow&quot;&gt;Smooth minimization of non-smooth functions&lt;/a&gt;&quot; (2004)&lt;/li&gt;&#xA;&lt;li&gt;Neumaier, &quot;&lt;a href=&quot;http://arxiv.org/abs/1402.1125&quot; rel=&quot;nofollow&quot;&gt;OSGA: A fast subgradient algorithm with optimal complexity&lt;/a&gt;&quot; (2014)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But I want to know is there any solver (better with c++ compatibility) that has already built in algorithms based on those methods? thank you:)&lt;/p&gt;&#xA;" OwnerUserId="7891" LastEditorDisplayName="user3883" LastEditDate="2014-11-23T11:00:56.820" LastActivityDate="2014-11-24T21:46:46.603" Title="non-smooth convex c++ solver" Tags="|linear-solver|c++|software|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16268" PostTypeId="2" ParentId="7616" CreationDate="2014-11-23T17:12:01.850" Score="3" Body="&lt;p&gt;There are actually many different approaches to phase I in the simplex method.  In particular, there are phase I algorithms that use primal simplex simplex iterations and other phase I algorithms that use dual simplex iterations.  Here's a very general approach that could easily be adapted to make use of a known feasible solution.  This version uses the dual simplex method in phase I and the primal simplex method in phase II, but there's a variant that uses primal simplex iterations in phase I and dual simplex iterations in phase II that I'll mention at the end.  The approach that I'm going to describe here is discussed in many textbooks on linear programming.  For example, see &lt;a href=&quot;http://www.princeton.edu/~rvdb/LPbook/&quot; rel=&quot;nofollow&quot;&gt;Robert Vanderbei's text&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that we're solving &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \max cx $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ Ax=b $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$l \leq x \leq u$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ of size $m$ by $n$.  For simplicity, assume that the rows of $A$ are linearly independent (this can be accomplished by a rank revealing factorization.)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pick an initial basis.  This is a collection of $m$ variables so that the corresponding columns of $A$ form a non-singular matrix $B$.  The remaining nonbasic variables can be set to either their upper or lower bounds (or zero if a variable has no bounds at all.)  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;An easy way to do this from your initial solution is to select as basic variables those variables that are furthest from their bounds in the known feasible solution and then verify that $B$ is non-singular.  You may have to modify the basis to make $B$ non-singular.  The point here is that there are many possible bases, but this one has as basic variables that variables that seem to be right from your feasible solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve the equations $Ax=b$ to obtain the values of the basic variables.  &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The basic solution that you obtain is likely to be primal infeasible in the sense that some of the primal variables are outside of their bounds.  It is also likely to be dual infeasible in the sense that some of the reduced costs of the non-basic variables have the wrong signs (e.g. nonbasic variables at lower bounds with positive reduced costs or nonbasic variables at upper bounds with negative reduced costs.)  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We will overcome this problem by changing the objective function to one that is dual feasible.  For each nonbasic variable at its lower bound, subtract a large positive quantity $M$ from the objective function coefficient.  For each nonbasic variable at its upper bound, add a large positive quantity $M$ to the coefficient.  This ensures that the dictionary is dual feasible.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of this modification of the objective function is to try to work towards &#xA;primal feasibility but also to move towards optimality with respect to the original objective function.  You want $M$ to be large enough that you have dual feasibility, but you want to keep as much influence as you can from the original objective function.  &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Perform dual simplex methods to obtain a basic solution which is both primal feasible (all basic variables within boudns) and dual feasible (all reduced costs have the desired sign.)  This solution is optimal for the phase I problem.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Replace the modified phase I objective function with the original objective function.  Now you'll have a basic solution that is primal feasible (changing the objective function doesn't affect this) but dual infeasible.  Perform primal simplex iterations to get back to optimality.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;An obvious alternative to this approach would be to modify the right hand side b at the start of phase I, use primal simplex iterations in phase I to get to optimality, then put the original right hand side back for phase II and use dual simplex iterations in phase II.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-11-23T17:12:01.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16270" PostTypeId="1" CreationDate="2014-11-23T19:07:45.420" Score="2" ViewCount="794" Body="&lt;p&gt;Suppose I have a 1D advection equation in conservation (divergence) form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t u(x,t) = -\partial_x [v(x)u(x,t)],$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is a conserved quantity in space, and $v$ gives the velocity of the flow of mass in space. I want to know if it is possible to re-write this in a way that describes the rate of change of $u(x,t)$ as a function of incoming mass from all other points in space (let us call these points $y$). In other words, can I write an equation of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t u(x,t) = \int_y u(y,t) h(y,x) dy,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the function $h(y,x)$ gives the proportion of the mass $u(y,t)$ at position $y$ that moves to position $x$ in space in a small unit of time? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I arrived at the advection equation via the divergence theorem, which is why I can write $\partial_t u(x,t)$ in terms of the rate of change in the space at the same point $x$, but intuitively it seems to me there should be another possible formulation like I have described. I suppose the mass is only entering point $x$ from the $y$ points that are already 'very close' to $x$, which is why it is an advective equation rather than a diffusive equation, but I should still be able to sum over all the $y$ points from where the mass is coming from, shouldn't I!?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I'm a novice in terms of the mathematics and the applications of the advection equation, I would like to know if the question I have asked is well-posed but also whether or not this is a typical problem people think about in applications?  Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9741" LastActivityDate="2014-11-24T17:25:08.533" Title="Can the conservative form of the advection equation be re-written by replacing the velocity term with an integral over all other points in space?" Tags="|pde|fluid-dynamics|advection|integral-equations|conservation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16273" PostTypeId="2" ParentId="16270" CreationDate="2014-11-24T00:13:33.510" Score="3" Body="&lt;p&gt;Your description &quot;the function $h(y,x)$ gives the proportion of the mass $u(y,t)$ at position $y$ that moves to position $x$ in space in a small unit of time&quot; seems to indicate a slight misunderstanding of derivatives. Since the  term on the left of your equation ($\partial_t u$) is a time derivative, the &quot;small unit of time&quot; must be infinitesimally small. Therefore only points infinitesimally far from $x$ need to be considered and you would end up right back at the advection equation with a spatial derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a related note, differential equations coming from conservation laws can also be written in integral form, which is typically the way they are derived. Any decent book on fluid dynamics will include a derivation of the Euler and Navier-Stokes equations starting from the conservation laws to write the integral form and then derive the differential form. For the advection equation this goes as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming no sources or sinks of $u$, conservation of $u$ says that the amount of $u$ between $x_1$ and $x_2$ at time $t_2$ is given by the amount at $t_1$ plus the flux into and out of the domain at the boundaries during the time from $t_1$ to $t_2$:&#xA;$$&#xA;\int_{x_1}^{x_2} u(x,t_2) dx = \int_{x_1}^{x_2} u(x,t_1) dx + \int_{t_1}^{t_2} \left[ v(x_1,t)u(x_1,t) - v(x_2,t)u(x_2,t) \right] dt.&#xA;$$&#xA;Rearranging and dividing both sides by $(t_2-t_1)$ gives&#xA;$$&#xA;\int_{x_1}^{x_2} \dfrac{u(x,t_2) - u(x,t_1)}{t_2-t_1} dx = \dfrac{1}{t_2-t_1} \int_{t_1}^{t_2} \left[ v(x_1,t)u(x_1,t) - v(x_2,t)u(x_2,t) \right] dt.&#xA;$$&#xA;Then, taking the limit as  $t_1\to t_2$ (assuming $u$ is continuous in time) gives&#xA;$$&#xA;\int_{x_1}^{x_2}\partial_t u(x,t) dx = v(x_1,t)u(x_1,t) - v(x_2,t)u(x_2,t).&#xA;$$&#xA;Finally, dividing by $(x_2-x_1)$ and taking the limit as $x_1\to x_2$ gives the advection equation&#xA;$$&#xA;\partial_t u(x,t) = -\partial_x(v(x,t)u(x,t)).&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-11-24T00:52:19.807" LastActivityDate="2014-11-24T00:52:19.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16274" PostTypeId="1" CreationDate="2014-11-24T04:26:17.543" Score="0" ViewCount="77" Body="&lt;p&gt;The commonly used $3\times 3$ Laplacian convolution kernel&#xA;$\frac{1}{6}\begin{bmatrix}1 &amp;amp; 4 &amp;amp; 1\\4 &amp;amp; -20 &amp;amp; 4\\1 &amp;amp; 4 &amp;amp; 1\end{bmatrix}$&#xA;is only an approximation of rotational symmetry, as mentioned in the Wikipedia article on the &lt;a href=&quot;http://en.wikipedia.org/wiki/Discrete_Laplace_operator&quot; rel=&quot;nofollow&quot;&gt;Discrete Laplace operator&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I found the first order Sobel operator to have &lt;a href=&quot;https://en.wikipedia.org/wiki/Talk:Sobel_operator#Scharr_not_the_ultimate_solution&quot; rel=&quot;nofollow&quot;&gt;an exactly rotational symmetric form&lt;/a&gt; without any proof.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions: How can we compute the exact value of a rotational symmetric $n\times n$ convolution kernel for&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the $\nabla$ operator?&lt;/li&gt;&#xA;&lt;li&gt;the Laplacian operator?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="862" LastEditorUserId="4912" LastEditDate="2014-11-24T20:33:21.707" LastActivityDate="2015-01-24T02:50:19.990" Title="How can we compute the exact value of some rotational symmetric n-by-n convolution kernel" Tags="|pde|image-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16278" PostTypeId="2" ParentId="16270" CreationDate="2014-11-24T17:25:08.533" Score="0" Body="&lt;p&gt;Whilst Doug's answer seems to be strictly correct, I've been informed that it is possible to re-write the spatial derivative as an integral if one uses a &lt;a href=&quot;http://en.wikipedia.org/wiki/Dirac_delta_function&quot; rel=&quot;nofollow&quot;&gt;Dirac delta function&lt;/a&gt; with $\delta(x-y)$. I'm not sure how useful this is for any practical applications of the advection equation, but I find it interesting nevertheless. The 'trick' is to use $h(y,x)=v(y)\partial_{y}[\delta(x-y)]$. Integration by parts then gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_y u(y,t) h(y,x) dy$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$= \int_y \partial_{y}[\delta(x-y)] v(y) u(y,t) dy $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$= \delta(x-y) v(y) u(y,t) - \int_y \delta(x-y) \partial_{y} [v(y) u(y,t)] dy $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$= - \partial_{y} [v(y) u(y,t)]. $&lt;/p&gt;&#xA;" OwnerUserId="9741" LastActivityDate="2014-11-24T17:25:08.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16280" PostTypeId="2" ParentId="16265" CreationDate="2014-11-24T21:46:46.603" Score="2" Body="&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://homepage.univie.ac.at/masoud.ahookhosh/&quot; rel=&quot;nofollow&quot;&gt;Masoud Ahookhosh&lt;/a&gt; has posted a MATLAB implementation of OSGA.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/software_l.html#nonsm&quot; rel=&quot;nofollow&quot;&gt;Arnold Neumaier has posted a list of non-smooth optimization solvers&lt;/a&gt; on his web site. The list is a little dated, but useful.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://napsu.karmitsa.fi/nsosoftware/&quot; rel=&quot;nofollow&quot;&gt;Napsu Karmitsa has also posted a list of non-smooth optimization solvers&lt;/a&gt;. She's recently written a book on non-smooth optimization, and her list of software is more up-to-date. From her list, you might check out &lt;a href=&quot;http://www.kfunigraz.ac.at/imawww/kuntsevich/solvopt/&quot; rel=&quot;nofollow&quot;&gt;SolvOpt&lt;/a&gt;, &lt;a href=&quot;http://guerin.ballarat.edu.au/ard/itms/CIAO/ganso/&quot; rel=&quot;nofollow&quot;&gt;GANSO&lt;/a&gt;, and &lt;a href=&quot;https://projects.coin-or.org/OBOE&quot; rel=&quot;nofollow&quot;&gt;OBOE&lt;/a&gt;; all of these packages are either C- or C++-based. Provided the theory underpinning these methods is amenable to your problem structure, you might find these solvers useful.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-11-24T21:46:46.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16281" PostTypeId="1" CreationDate="2014-11-24T22:45:21.680" Score="0" ViewCount="1471" Body="&lt;p&gt;I'm trying to make an animation of a travelling sine wave (amplitude vs. position) would anyon here happen to know how to do so?&lt;/p&gt;&#xA;" OwnerUserId="11877" LastActivityDate="2015-02-03T05:52:37.557" Title="Simulating a traveling sine wave" Tags="|c++|wave-propagation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16282" PostTypeId="1" CreationDate="2014-11-24T22:47:50.107" Score="1" ViewCount="1265" Body="&lt;p&gt;What Is The Intersection Between CFD and Computer Science...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How hard would it be to undertake a PHD in CFDs after an undergrad and masters in CompSci&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many Thanks....&lt;/p&gt;&#xA;" OwnerUserId="11885" LastActivityDate="2014-11-25T22:43:25.283" Title="What Is The Intersection Between CFD and Computer Science" Tags="|fluid-dynamics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16283" PostTypeId="2" ParentId="16282" CreationDate="2014-11-24T22:50:16.720" Score="1" Body="&lt;p&gt;An aspect where computer science and computational fluid mechanics intersect is the optimization of code and design of algorithms. From  personal experience, the most complicated aspect of CFD for me was numerical analysis (programming) but if you already have a master's in computer science then it shouldn't be too bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;good luck!&lt;/p&gt;&#xA;" OwnerUserId="11877" LastActivityDate="2014-11-24T22:50:16.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16284" PostTypeId="1" AcceptedAnswerId="16288" CreationDate="2014-11-24T22:58:12.933" Score="0" ViewCount="148" Body="&lt;p&gt;When solving a system of nonlinear equations using iterative methods, the rate of convergence usually is defined by the following formula:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt; &lt;img src=&quot;https://i.stack.imgur.com/AuYa2.gif&quot; alt=&quot;rate&quot;&gt; (1)&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;where x*  is the exact solution. However usually we don't know x*, so actually we use the following formula to calculate the rate of convergence of the method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xtmt0.gif&quot; alt=&quot;enter image description here&quot;&gt; &lt;strong&gt;(2)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how to validate that the alapha in (2) equals to that in (1)?&lt;/p&gt;&#xA;" OwnerUserId="8213" LastEditorUserId="8213" LastEditDate="2014-11-25T15:55:21.087" LastActivityDate="2014-11-25T15:55:21.087" Title="Actually calculating the rate of convergence of iteratvie methods when exact solution is unknown" Tags="|convergence|newton-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16286" PostTypeId="2" ParentId="16282" CreationDate="2014-11-24T23:17:08.650" Score="2" Body="&lt;p&gt;Physics and Mathematics. It's hard to do well in CFD if you don't understand something of the underlying partial differential equations that you are trying to make the computer simulate. You can pick this up, but you could easily spend the first two years of your PhD getting up to speed. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-11-24T23:17:08.650" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16287" PostTypeId="2" ParentId="16281" CreationDate="2014-11-25T00:07:18.777" Score="2" Body="&lt;p&gt;A very simple example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt; // sin&#xA;#include &amp;lt;fstream&amp;gt; //ofstream&#xA;#include &amp;lt;sstream&amp;gt; //stringstream&#xA;#include &amp;lt;string&amp;gt; //string&#xA;#include &amp;lt;cstring&amp;gt; //c_str&#xA;&#xA;#define PI 3.14159265&#xA;&#xA;// Modifies the elements in an array to five an evenly spaced number of points &#xA;// Along the specified interval (from b to a)&#xA;void linspace(double a, double b, double c, double * array){&#xA;        double delta =(b-a)/(c-1);&#xA;        for (int i=0; i&amp;lt;c; ++i){&#xA;                array[i]= i*delta;&#xA;       }&#xA;}&#xA;&#xA;&#xA;int main(int argc, char * argv[]){&#xA;       double c=1.;&#xA;       double x=0;&#xA;&#xA;       double u[90];&#xA;       for (int i=0; i&amp;lt;90; ++i){ u[i] = 0.; } // Initalize array&#xA;       double a=0., b=10.;&#xA;       linspace(a, b, 90, u);&#xA;&#xA;std::string file =&quot;datsin&quot;;  // Output file name&#xA;        for (int i=0; i&amp;lt;90; ++i){&#xA;               std::ostringstream oss;&#xA;               oss &amp;lt;&amp;lt; &quot;datsin&quot; &amp;lt;&amp;lt; i;&#xA;               std::ofstream out(oss.str().c_str());&#xA;               for (int t=0; t&amp;lt;=360; ++t){&#xA;                       out &amp;lt;&amp;lt; t&amp;lt;&amp;lt;'\t'&amp;lt;&amp;lt; sin(u[i]+(t*PI/180)) &amp;lt;&amp;lt;'\n';&#xA;               }&#xA;               out.close();&#xA;       }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will output the spatial coordiates of the wave at each moment in a series of files starting with &quot;datsinX&quot; where X will be a number from 0 to 89 in this case.&#xA;In order to now make an animation one can use the following gnuplot script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set terminal gif animate delay 10&#xA;set output &quot;animate.gif&quot;&#xA;set yrange[-1:1]&#xA;set xrange[0:400]&#xA;n=90  &#xA;i=0&#xA;load &quot;plot.gnuplot&quot;&#xA;set output &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which in this case, will in turn call the script &quot;plot.gnuplot&quot;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot 'datsin'.i w p ps 2 title sprintf(&quot;t=%i&quot;,i) &#xA;i=i+1&#xA;if (i &amp;lt; n) reread&#xA;pause 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and will produce a gif called &quot;animation.gif&quot;&lt;/p&gt;&#xA;" OwnerUserId="11877" LastActivityDate="2014-11-25T00:07:18.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16288" PostTypeId="2" ParentId="16284" CreationDate="2014-11-25T01:24:07.857" Score="2" Body="&lt;p&gt;What you have is not an exact equivalence. It uses the fact that &#xA;$$&#xA;  \| x_{k+2}-x_{k+1} \| \approx   \| x^\ast-x_{k+1} \|&#xA;\\&#xA;  \| x_{k+1}-x_{k} \| \approx   \| x^\ast-x_{k} \|&#xA;$$&#xA;because the next iterate is always much closer to the solution $x^\ast$ than the previous one.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-25T01:24:07.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16289" PostTypeId="2" ParentId="16274" CreationDate="2014-11-25T01:35:03.057" Score="1" Body="&lt;p&gt;I don't think you can. The problem is that you represent a function $f(\mathbf x)$ on a Cartesian mesh and you apply the operator to that. If you rotate the function, i.e., you try $f(A\mathbf x)$ (where $A$ is a $2\times 2$ rotation matrix), then you will get different function values at the vertices of the mesh and different values if you apply the discrete Laplace to it. In other words, even though the Laplacian operator is rotationally invariant, the Laplace operator on a mesh is not and can not be.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-25T01:35:03.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16291" PostTypeId="1" CreationDate="2014-11-25T04:01:53.680" Score="-2" ViewCount="1332" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/3BeKB.png&quot; alt=&quot;enter image description here&quot;&gt;I compared 2 methods for 1 dimensional Poisson equation solution.&#xA;One is finite-difference method, &quot;Successive Overrelaxation&quot; from &lt;a href=&quot;http://www.cs.berkeley.edu/~demmel/cs267/lecture24/lecture24.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cs.berkeley.edu/~demmel/cs267/lecture24/lecture24.html&lt;/a&gt;; Another method is spectral method by FFT which is much faster. The following is my matlab/octave code to generate the solutions in the picture. Obviously, the traditional finite-difference method(middle panel) is more acceptable!  WHY FFT method look like this? &#xA;Is may FFT-solution implement correct? Thanks!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is my matlab code:::&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;&#xA;&#xA;x=1:1000;&#xA;&#xA;vort=exp(-(x-600).^2/50^2);% right hand side of poisson eq&#xA;&#xA;% construct the wave number series, dx = 1 in this case&#xA;kx=(x-1);&#xA;jj=find(kx&amp;gt;500);&#xA;kx(jj)=kx(jj)-1000;&#xA;kx=2*pi/1000*kx;&#xA;&#xA;subplot(311);plot(x,vort);title('rhs of Poisson eq','fontsize',20)&#xA;&#xA;% ------------S.O.R solver----------&#xA;&#xA;psi=zeros(1,1000);&#xA;&#xA;for n=1:100&#xA;    for i=2:999&#xA;&#xA;        r=(psi(i-1)+psi(i+1)-vort(i))*0.5-psi(i);&#xA;        psi(i)=psi(i)+1.2*r;&#xA;&#xA;    end&#xA;end&#xA;&#xA;subplot(312);plot(x,psi);title('Successive Overrelaxation','fontsize',20)&#xA;&#xA;% ------ FFT solver -----%&#xA;&#xA;slice=-fft(vort)./(kx.^2);&#xA;&#xA;slice(1)=0;&#xA;&#xA;subplot(313);plot(real(ifft(slice)));;title('FFT method','fontsize',20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11889" LastEditorUserId="11869" LastEditDate="2014-11-26T14:34:44.443" LastActivityDate="2018-09-10T06:00:06.077" Title="explain the difference between 1D Poisson solvers" Tags="|finite-difference|poisson|spectral-method|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16292" PostTypeId="2" ParentId="16260" CreationDate="2014-11-25T11:13:48.473" Score="0" Body="&lt;p&gt;If available to you, Mathematica version 10 offers a PDE solving framework based on the finite element method which is build in. There are tutorials about &lt;a href=&quot;http://reference.wolfram.com/language/FEMDocumentation/tutorial/ElementMeshCreation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;mesh generation&lt;/a&gt; and &lt;a href=&quot;http://reference.wolfram.com/language/FEMDocumentation/tutorial/ElementMeshVisualization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;visualization&lt;/a&gt;. The concept is based on two functions &lt;a href=&quot;http://reference.wolfram.com/language/FEMDocumentation/ref/ToBoundaryMesh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ToBoundaryMesh&lt;/a&gt; and &lt;a href=&quot;http://reference.wolfram.com/language/FEMDocumentation/ref/ToElementMesh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ToElementMesh&lt;/a&gt; both generating the &lt;a href=&quot;http://reference.wolfram.com/language/FEMDocumentation/ref/ElementMesh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ElementMesh&lt;/a&gt; data structure. This data structure can then be queried for connectivity, boundary elements. Here is a simple manual example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Needs[&quot;NDSolve`FEM`&quot;]&#xA;mesh = ToElementMesh[&#xA;  &quot;Coordinates&quot; -&amp;gt; {{1.293, 0.228}, {1., 0.}, {0.94, 0.342}, {1.293, &#xA;     0.}, {1.215, 0.442}, {2., 0.}, {1.879, 0.684}}, &#xA;  &quot;MeshElements&quot; -&amp;gt; {TriangleElement[{{1, 3, 2}, {1, 2, 4}, {1, 4, &#xA;       6}, {1, 6, 7}, {1, 7, 5}, {1, 5, 3}}]}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Visualize a wireframe:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh[&quot;Wireframe&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/JMfju.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compute the element connectivity:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh[&quot;ElementConnectivity&quot;]&#xA;{{{0, 2, 6}, {0, 3, 1}, {0, 4, 2}, {0, 5, 3}, {0, 6, 4}, {0, 1, 5}}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Get the points on the boundary:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh[&quot;PointElements&quot;]&#xA;{PointElement[{{2}, {3}, {4}, {5}, {6}, {7}}]}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Boolean operations are available as well as are many other things. For example you may tag elements with markers, for example to group different material or boundaries together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you like to work with meshes on an even lower level, then there are interfaces to the mesh generators &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Triangle&lt;/a&gt; via TriangleLink and &lt;a href=&quot;http://wias-berlin.de/software/tetgen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TetGen&lt;/a&gt; via &lt;a href=&quot;http://reference.wolfram.com/language/TetGenLink/guide/TetGenLink.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TetGenLink&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to double check your implementation you could make use of the &lt;a href=&quot;http://reference.wolfram.com/language/FEMDocumentation/guide/FiniteElementMethodGuide.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite Element Programming&lt;/a&gt; functionality.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2014-11-25T11:13:48.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16293" PostTypeId="1" CreationDate="2014-11-25T11:21:39.487" Score="3" ViewCount="119" Body="&lt;p&gt;I am simulating a damped harmonic oscillator using the RK4 method of numerical integration. I am comparing the simulated results with the analytical ones (for the free evolution case) and obtaining the error as a function of step-size, for different values of the quality factor Q. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot{X} + \frac{1}{Q}\dot{X} + X = 0$$&#xA;The analytical solution is given by:&#xA;$$x(\tau) = e^{-\tau / 2Q} \left[c_1 \cos\left(\tau \sqrt{1 - \frac{1}{4Q^2}}\right) + c_2\sin\left(\tau \sqrt{1 - \frac{1}{4Q^2}}\right)\right]$$&#xA;where $c_2$ and $c_2$ can be found by using the boundary conditions: $x(0) = 1$ and $\dot{x}(0) = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For RK4 method, we obtain two first order equations which we can use in the standard RK4 formula:&#xA;&lt;strong&gt;Eqn1:&lt;/strong&gt;&#xA;$$\dot{X} = P \Rightarrow \dot{P} = \ddot{X}  $$&#xA;Substitute into original equation to obtain:&#xA;$$\Rightarrow \dot{P} ~+ \frac{1}{Q}P + X = 0$$&#xA;Which leads to &lt;strong&gt;Eqn2:&lt;/strong&gt;&#xA;$$\dot{P} = -\left(\frac{P}{Q}+X\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So these equations allow us to obtain the next point in $X$ and $dX$ for a particular step-size, $h$, using $$y_{n+1} = y_n + h\left(\frac{k_1}{6} + \frac{k_2}{3} + \frac{k_3}{3} + \frac{k_4}{6}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Error Investigation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am interested in the relative error for different step-sizes and Quality factors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Err_{relative} = \frac{\Sigma_i (x_i - X_i)^2}{\Sigma_i x_i^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can logically expect the error to increase as the step-size increases, which is what my graphs show. I run this simulation for different values of Q.&lt;br&gt;&#xA;&lt;strong&gt;I would expect the error to decrease as Q increases, but instead the trajectory is just shifted upwards, i.e: the error is increased but the behaviour with respect to the step size is what we should expect.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/R0FlA.png&quot; alt=&quot;enter image description here&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be hard to see on the graph but those trajectories are for Q = 5 and Q = 95. The trajectory for Q = 5 is lower than the other one. I would have expected it to be the other way round.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Assuming I haven't goofed up my code, are my assumptions correct or is this normal behaviour?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6744" LastEditorUserId="7733" LastEditDate="2014-11-25T19:15:25.220" LastActivityDate="2014-11-25T19:15:25.220" Title="Is it normal to expect the error of simulation of a damped harmonic oscillator to decrease as the damping factor decreases?" Tags="|ode|simulation|runge-kutta|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16296" PostTypeId="2" ParentId="16264" CreationDate="2014-11-25T14:34:20.633" Score="0" Body="&lt;p&gt;I found a solver that seems to be suitable for my problem. My very ill-conditioned matrix is actually the FEM discretisation of a Poisson equation on a bidimentionnal unstructured mesh. &#xA;First results with a geometrical multi-grid approach (multiple V-cycle) show very good weak-scaling on CPU. And since most of the time is spent multiplying big sparse matrices with vectors (on the fine mesh), I expect the GPU performance to be okay. &lt;/p&gt;&#xA;" OwnerUserId="11869" LastActivityDate="2014-11-25T14:34:20.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16299" PostTypeId="2" ParentId="16282" CreationDate="2014-11-25T22:43:25.283" Score="3" Body="&lt;p&gt;Re: The Intersection...  There's a ton.  But a great CFD guy might not offhandedly know basic DAG traversal.  And a great CS guy might not offhandedly know about the practical tradeoffs inherent in various classical numerical schemes.  Each side can pretty easily learn the other whenever it matters, however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Re: Difficulty...  Depends on the program.  You might have trouble in a mechanical engineering department depending on the Ph.D. qualifier requirements as they sometimes expect much more breadth than you would strictly speaking need to complete a dissertation in a well-scoped topic.  There are quite a few good computational science programs out there where you could focus on CFD but get a more gradual intro to the required engineering/mathematics/physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Re: You Didn't Ask But....  If you decide to go into CFD from a programming background, be sure that there is a straight line from the algorithms/tools you're developing to novel results in an application that a non-CFD fluid dynamicist would care about.  It's wayyyyyyyy too easy to spend a lot of time on algorithms/programming/optimization/etc (because it's wicked fun) only to realize you've not got enough physics-related punch to your research results.  Also, for the love of all that is holy, don't casually decide to write your own flow solver because the legacy ones around you don't cut the CS mustard.  It's an excellent way to fall into the aforementioned trap.  Moreso than for most folks, I'd say your advisor choice is crucial.&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2014-11-25T22:43:25.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16301" PostTypeId="1" CreationDate="2014-11-26T11:36:07.050" Score="0" ViewCount="206" Body="&lt;p&gt;I'm writing a solver for a differential equation with two neumann boundaries (u'(0)=u'(1)=0)&#xA;and I can't figure out how to determine how to solve the problem. What will my boundaries be and how do I figure this out?&#xA;The differential equation is &#xA;$$u''(x)+Ku(x)=0.$$&lt;/p&gt;&#xA;" OwnerUserId="11909" LastEditorUserId="549" LastEditDate="2014-12-26T17:12:53.850" LastActivityDate="2014-12-26T17:12:53.850" Title="Neumann boundary problem" Tags="|finite-difference|numerical-analysis|boundary-conditions|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16302" PostTypeId="2" ParentId="16301" CreationDate="2014-11-26T14:00:13.243" Score="1" Body="&lt;p&gt;You should consider what needs to happen between a given pair of u(i) and u(i+1) for a forward difference [or u(i) and u(i-1) for a backwards difference, or whatever scheme you are using] such that u'(i) equals zero. If you just consider the definition of derivative for a forward difference scheme as (u(i+1) - u(i)/dx) where dx is a positive constant, the answer is obvious, u(x) must satisfy the condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{u(i+1)-u(i)}{dx}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding this, I am certain that you will be able to answer your own question. I can comment further if you need more information.&lt;/p&gt;&#xA;" OwnerUserId="11799" LastActivityDate="2014-11-26T14:00:13.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16303" PostTypeId="1" CreationDate="2014-11-26T14:39:35.870" Score="1" ViewCount="163" Body="&lt;p&gt;I am new to CVX and am trying to simulate this convex problem I found in a paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{\gamma,\mathbf{mu},\mathbf{G},\mathbf{\Omega},t} \text{Tr}(\mathbf{G}\mathbf{C}\mathbf{G}^H)+t \\ s.t. -t-\frac{\gamma}{2}\sum\limits_{i=1}^n\log(1-\frac{2\mu_i}{\gamma})-\gamma \log(\delta)\leq0\\\mu_1\geq\mu_2\geq\ldots\geq\mu_n\geq0\\2\mu_1&amp;lt;\gamma\\ \text{S}_k(C^{1/2} \Lambda C^{1/2})\leq \sum\limits_{i=1}^{k} \mu_i \quad \forall k = 0,1,\dots,n-1\\&#xA;\text{Tr}(\Omega C)\leq \sum\limits_{i=1}^{n}\mu_i\\\Omega\geq \Lambda$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $$\Lambda = (\mathbf{G}\mathbf{H}-\mathbf{I})^H(\mathbf{G}\mathbf{H}-\mathbf{I})$$ and $S_k(\mathbf{A})$ is the sum of the largest $k$ eigenvalues of $\mathbf{A}$. In the problem, the scalar $\delta$, matrices $\mathbf{H}$ and $\mathbf{C}$ are assumed known. $\mathbf{C}$ is symmetric positive definite. I am finding it difficult to reformulate the first constraint with the log summation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="11910" LastEditorUserId="11910" LastEditDate="2014-11-30T23:47:06.650" LastActivityDate="2014-11-30T23:47:06.650" Title="Coding a convex problem in CVX" Tags="|optimization|convex-optimization|cvx|" AnswerCount="0" CommentCount="18" ContentLicense="CC BY-SA 3.0" />
  <row Id="16304" PostTypeId="1" AcceptedAnswerId="16305" CreationDate="2014-11-26T15:16:58.590" Score="2" ViewCount="109" Body="&lt;p&gt;Overlapping technique can make each subdomain contain more nodes, and the overlapped subdomains are nonlonger disjoint, is it taking the average value of the multiple nodes as the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After overlapping, we will solve larger sundomins, and it will cost more time and storage, how can the benefit from taking average value of multiple nodes outperform the increse cost per iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is it necessary and usually beneficial.&lt;/p&gt;&#xA;" OwnerUserId="11856" LastActivityDate="2014-11-26T15:37:31.903" Title="why overlapping technique can accelerate the additive/multiplictive Schwarz" Tags="|domain-decomposition|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16305" PostTypeId="2" ParentId="16304" CreationDate="2014-11-26T15:37:31.903" Score="1" Body="&lt;p&gt;The problem with elliptic operators is that the solution at any specific point depends on all the domain. The overlap allows to 'mix' the approximate solution between subdomains and therefore accelerates the solution of the problem. &lt;a href=&quot;http://en.wikipedia.org/wiki/Domain_decomposition_methods&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Domain_decomposition_methods&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2014-11-26T15:37:31.903" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16307" PostTypeId="2" ParentId="16291" CreationDate="2014-11-26T17:19:12.423" Score="4" Body="&lt;p&gt;First, some notation for clarity. We're talking about solving $u_{xx}=f(x)$. So $u$ is the solution and $f$ is the right hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What makes you think either of your solutions is or is not correct?&lt;br&gt;&#xA;To check your results, just compute two derivatives of your solution (using finite differences or FFT, either one) and compare with $f$. In your example, $f$ is always positive so the solution should be concave up everywhere. In this case they don't match for either of your solutions. Now you need to figure out why. You should also use consistent boundary conditions if you want to compare two solutions. You are using boundary conditions $u(1)=u(1000)=0$ for the SOR solution, but the spectral (FFT) solution implies periodic boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The SOR solution&lt;/strong&gt; is clearly concave down on a couple intervals since it is not converged. Convergence is extremely slow for this problem, you will likely need a couple hundred thousand SOR iterations instead of just 100. Consider that in SOR information can move only 1 gridpoint to the left at each iteration and your domain is ~1000 points. Clearly 100 iterations will not be enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The FFT solution&lt;/strong&gt; implies smooth, periodic boundary conditions, but a function cannot be smooth, periodic, and always concave up. You may also have slight issues near the boundaries since your right hand side is not strictly periodic. I believe your FFT method is correct (although you should use &lt;code&gt;ifft(...,'symmetric')&lt;/code&gt; rather than &lt;code&gt;real()&lt;/code&gt; to ensure real results), but the problem is ill posed. Instead of the expected solution, you get a solution where the error is distributed throughout the domain. That's why the solution is concave down where it is expected to be nearly linear. In fact, the error in the second derivative is uniform throughout the domain and is a constant difference of $\int_1^{1000}f(x)dx/1000\approx0.0886$ in the second derivative of your solution. This shift (the mean of $f$) is lost when you set &lt;code&gt;slice(1)=0;&lt;/code&gt; and reflects the fact that the net effect (i.e. the mean) of the $u_{xx}$ must be zero to have a smooth, periodic solution. Put another way, adding any constant value to you $f$ has no effect on the solution in the spectral method since the method enforces smooth periodicity which implies zero mean for the derivatives of $u$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For reference, the solution (with second order centered finite differences and zero boundary conditions) can be found using a direct solver with the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Generate sparse matrix for the linear system:&#xA;i = [2:999, 2:999, 2:999, 1, 1000];&#xA;j = [1:998, 2:999, 3:1000, 1, 1000]; &#xA;s = [ones(1,998), -2*ones(1,998), ones(1,998), 1, 1];&#xA;M = sparse(i,j,s);&#xA;&#xA;% Generate right hand side:&#xA;x = (1:1000)';&#xA;rhs = [0; exp(-(x(2:999)-600).^2/50^2); 0];&#xA;&#xA;% Solve:&#xA;y = M\rhs;&#xA;&#xA;% Plot:&#xA;figure, plot(x,y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and should look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/3O2BS.png&quot; alt=&quot;solution to Poisson equation&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the solution is approximately linear except in the interval [500, 700] where $f$ has the largest values so $u$ is clearly concave up.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-11-26T23:13:45.620" LastActivityDate="2014-11-26T23:13:45.620" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16308" PostTypeId="1" CreationDate="2014-11-26T22:04:23.630" Score="3" ViewCount="365" Body="&lt;p&gt;While I was studying discontinuous finite element methods I found an integration of a Navier Stokes equation using tensorial notation. The equation is the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{\bar {u}}_{t} + (\mathbf{\bar {u}}.\triangledown )\mathbf{\bar {u}} = -\triangledown p + &#xA;\triangledown.\mathbf{\bar{\bar{\tau}}} + \mathbf{\bar {f}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;after integration the weak form is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{\Omega_{j}} \mathbf{\bar {u}}_{t}.\mathbf{\bar {v}}dV - \int_{\Omega_{j}} \mathbf{\bar {u}}_{}.\triangledown.(\mathbf{\bar {v}}\otimes \mathbf{\bar {u}})dV + &#xA;\int_{\partial \Omega_{j}} \mathbf{\bar {u}}.(\mathbf{\bar {n}}\otimes\mathbf{\bar {u}}).&#xA;\mathbf{\bar {v}}ds - \int_{\Omega_{j}}p.\triangledown \mathbf{\bar {v}}dV + &#xA;\int_{\partial \Omega_{j}}p\mathbf{\bar {v}}.\mathbf{\bar {n}}ds + &#xA;\int_{ \Omega_{j}}\mathbf{\bar{\bar{\tau}}}:\triangledown \mathbf{\bar {v}}dV-&#xA;\int_{\partial \Omega_{j}} \mathbf{\bar{\bar{\tau}}}:(\mathbf{\bar {v}}\otimes \mathbf{\bar {n}})ds&#xA;= \int_{ \Omega_{j}} \mathbf{\bar {f}}.\mathbf{\bar {v}}dV$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where a single and a double overbar means first and second order tensors respectively and the $\mathbf{\bar {v}}$ vector is the basis used in the integration.&#xA;I didn't understand the integration of the second LHS and RHS terms. I tried to use the divergence theorem but I wasn't able to obtain the dyadic products of the surface integrals.&#xA;I would appreciate your help.&lt;/p&gt;&#xA;" OwnerUserId="9713" LastEditorUserId="9713" LastEditDate="2014-11-27T08:28:03.773" LastActivityDate="2015-06-13T16:01:41.967" Title="Finite Element integration with tensor notation" Tags="|finite-element|fluid-dynamics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16309" PostTypeId="2" ParentId="12855" CreationDate="2014-11-26T22:54:11.150" Score="4" Body="&lt;p&gt;I think the basic ideas behind this question are extremely valuable (the question should be made a bit more precise, though), and it'd be nice to see an answer in a community wiki-style format, with a list of software and a short explanation of each item. Below, however, are my ramblings and personal experiences.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;2D animations&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;As you are already aware, LaTeX with TikZ/PGF in particular, can produce very high quality plots. I typically use PGFPlots to graph the final figures for academic papers and such, where anything but static graphics is not possible, anyway. Note, though, that you can in fact produce animations in LaTeX (PGFPlots), as attested to by the many questions in TeX.SE (yes, it's quite bug-prone). You can also plot simple 3D images in PGFPlots. Slightly more interactive, and 3D, graphics can be produced with Asymptote, a language that interfaces very well with LaTeX. There is a downside: the only reader that can display these with all of their properties is, to my knowledge, Adobe Acrobat. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there are the different programming languages and their different plotting backends: Python, Julia, R, and Octave, to name a few of the ones scientists commonly use for their graphical features. Many of these probably if not default, at least can use, gnuplot and should be able to graph in 3D. Most explicitly support animations, and the ones that do not, support printing out static images which you can then ffmpeg/avconv/mencoder into a video (this is what the libraries that provide a frontend for animations usually do behind the scenes, anyway). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now one of the reasons that the output of LaTeX is of such high quality is that its default output format is PDF, which can store vector graphics. It should be noted that most other programs made for plotting, too, can output vector graphics, but png and other lossless raster graphics are usually the default. With animations in their most common file formats, .gif, .avi etc., though, you lose the ability to produce vector graphics. Well, not quite, I already did mention that PDF does support some simple animations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Better yet, SVG, the most common vector graphics format, can work wonders: You can even program whole games just using the features of SVG, so animations are easy-peasy. I have in fact largely switched from Beamer to HTML5 (SVG being a part of HTML5, in a way) when preparing presentation slides mainly because of its support for animations (both ones interactively generated by the document itself, but also because with HTML5, embedding .avis is hassle-free). What I have found particularly useful is D3.js, a JavaScript library for displaying data (in SVG). It can do much more than just simple plots so its usage can get a bit unwieldy at times. That said, the library is extremely well documented, and already has books written about it. C3.js is a simpler interface for D3.js for plotting charts, but I found it to have been designed with business charts in mind, and seem to remember that setting the x-axis, y-axis scalings and data was not particularly easy.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;3D animations&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;If you really want quality graphics with lens flare like you're J.J. Abrams, and this applies for 2D as well, vector graphics won't cut it: You'll most probably want to use some image filters and very few are available in vector format. Now in 2D this usually is not an issue (unless you want to include a photograph), but for 3D perspectives such filters help a lot. There are a few ways of producing vector graphics that look 3D-ish; for example Sketch for LaTeX or the PostScript output of MolScript to name a couple, but these are often application specific and don't come even close to emulating different sorts of materials etc. What you'd really want, then, for presenting any 3D data which you want to look artistic or realistic rather than just a sketch, is ray tracing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I like POVRay, it's open source, well documented, and easy to use. I say easy to use, but it might take some figuring out how to use it with scientific data as it was not built with that audience in mind, but it can be done. A popular one in molecular dynamics (MD) simulations is Tachyon as it was built of this particular purpose. It comes with VMD, one of the most popular viewers of MD data.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Simulations&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;What I wrote above is really only about static animations: Ones you make but later cannot affect, like an .avi file (the part about SVG/D3.js, though, works interactively). Why? Because the simulation details will depend on the platform: Do you want a website or a downloadable demonstration of some physical phenomena? Or do you perhaps have a simulation API and want to interface your own interactive graphics to it so that you might affect the simulation and see what's happening?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to deploy online, yes, nowadays you'd probably want to use HTML5 (building the simulation in JavaScript and interfacing it with some SVG or canvas based solutions, for example through D3.js). For 3D you'd probably use a WebGL library. If you're happy with offline, then there are thousands of libraries out there, many built on OpenGL. &lt;/p&gt;&#xA;" OwnerUserId="8143" LastActivityDate="2014-11-26T22:54:11.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16310" PostTypeId="1" AcceptedAnswerId="16311" CreationDate="2014-11-27T00:26:57.523" Score="5" ViewCount="110" Body="&lt;p&gt;In George, Liu, and Ng's book &lt;a href=&quot;http://web.engr.illinois.edu/~heath/courses/cs598mh/george_liu.pdf&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Computer Solutions of Sparse Linear Systems&lt;/em&gt;&lt;/a&gt;, it has been shown that bandwidth of $A$ is equal to bandwidth of its factors in $LL^T$.(section 4.3) However, I guess this is proven only for $LL^T$ factorization for positive definite matrices. Is there a similar proof for indefinite matrices for factors in $LDL^T$, where $D$ is a block-diagonal matrix with at most $2\times 2$ blocks and assuming that such a factorization exists?&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="4244" LastEditDate="2014-11-27T13:10:11.310" LastActivityDate="2014-12-05T18:32:21.900" Title="Is the bandwidth of indefinite A equal to its factor L in LDL^T?" Tags="|sparse-matrix|banded-matrix|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16311" PostTypeId="2" ParentId="16310" CreationDate="2014-11-27T07:32:43.850" Score="1" Body="&lt;p&gt;It is generally true for the $LU$ factorization of any matrix that the bandwidth of the factors equals the bandwidth of the original matrix. In fact, the result is stronger: that the skyline of the factors is contained in the skyline of the original matrix. This is actually not all that difficult to see if you consider the steps one has to take to compute the $LU$ factorization, which are equivalent to the first half of Gaussian elimination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you ask for an $LDL^T$ factorization of a symmetric and possibly indefinite matrix $A$: Your $LU$ factors are $LD^{1/2}$ and $D^{1/2}L^T$ for which the bandwidth is at most that of the original matrix $A$. Consequently, the same can be said about the matrix $L$ as well.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-27T07:32:43.850" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16312" PostTypeId="1" AcceptedAnswerId="16323" CreationDate="2014-11-27T08:23:52.017" Score="2" ViewCount="513" Body="&lt;p&gt;I have questions about the use of &quot;derivative&quot; annotation I wanted to ask you. Lets put a sample. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z_level = LevelZ(time);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where LevelZ(time) is an external C function, and there are also LevelZ_der(time) and LevelZ_der2(time) that are the first and second derivative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It the code is like this, it works well... &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function LevelZ_der2 &#xA;    input Real t; &#xA;    input Real dt; &#xA;    input Real ddt; &#xA;    output Real res; &#xA;external &#xA;    res = LevelZ_der2(t); &#xA;annotation(Library=&quot;...&quot;, Include=&quot;...&quot;);  &#xA;end LevelZ_der2; &#xA;&#xA;function LevelZ_der &#xA;    input Real t &#xA;    input Real dt &#xA;    output Real res; &#xA;external &#xA;    res = LevelZ_der(t); &#xA;annotation(derivative=LevelZ_der2, &#xA;           Library=&quot;...&quot;, Include=&quot;...&quot;);  &#xA;end LevelZ_der; &#xA;&#xA;function LevelZ &#xA;    input Real t &#xA;    output Real res; &#xA;external &#xA;    res = LevelZ(t) &#xA;annotation(derivative=LevelZ_der, &#xA;           derivative(order=2)=LevelZ_der2, &#xA;           Library=&quot;...&quot;, Include=&quot;...&quot;);  &#xA;end LevelZ;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;... although these errors appear: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[C:/Users/Model/Model.mo:52:2-52:28:writable] Warning: Unused input variable dt in function .LevelZ_der2. &#xA;[C:/Users/Model/Model.mo:52:2-52:28:writable] Warning: Unused input variable ddt in function .LevelZ_der2. &#xA;[C:/Users/Model/Model.mo:52:2-52:28:writable] Warning: Unused input variable dt in function .LevelZ_der. &#xA;&#xA;Error: Derivative of expression &#xA;Differentiate.differentiateExpSolve failed for z_level - LevelZ(time) &#xA;is non-existent.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However if the code is like this, OpenModelica simulation ignores the derivatives... &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function LevelZ_der2 &#xA;    input Real t; &#xA;    output Real res; &#xA;external &#xA;    res = LevelZ_der2(t); &#xA;annotation(Library=&quot;...&quot;, Include=&quot;...&quot;);  &#xA;end LevelZ_der2; &#xA;&#xA;function LevelZ_der &#xA;    input Real t &#xA;    output Real res; &#xA;external &#xA;    res = LevelZ_der(t); &#xA;annotation(derivative(zeroDerivative=t)=LevelZ_der2, &#xA;           Library=&quot;...&quot;, Include=&quot;...&quot;);  &#xA;end LevelZ_der; &#xA;&#xA;function LevelZ &#xA;    input Real t &#xA;    output Real res; &#xA;external &#xA;    res = LevelZ(t) &#xA;annotation(derivative(zeroDerivative=t)=LevelZ_der,  &#xA;           derivative(order=2)=LevelZ_der2, &#xA;           Library=&quot;...&quot;, Include=&quot;...&quot;);  &#xA;end LevelZ;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;... and this error appear: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: Derivative of expression &#xA;Differentiate.differentiateExpSolve failed for z_level - LevelZ(time) &#xA;is non-existent.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the right way to do it and why? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards&#xA;Koldo&lt;/p&gt;&#xA;" OwnerUserId="4607" LastActivityDate="2014-11-28T17:58:32.973" Title="Modelica &quot;derivative&quot; annotation problem" Tags="|openmodelica|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16313" PostTypeId="1" AcceptedAnswerId="16324" CreationDate="2014-11-27T09:15:06.080" Score="2" ViewCount="2616" Body="&lt;p&gt;I am looking for information on the time complexity for solving sparse system &lt;em&gt;Ax=b&lt;/em&gt; with direct solver. This system results from a finite-element discretization of an elliptic problem. The matrix &lt;em&gt;A&lt;/em&gt; considered is symmetric definite positive, has &lt;em&gt;n&lt;/em&gt; equations, &lt;em&gt;m&lt;/em&gt; non-zeros and a constant bandwidth &lt;em&gt;b&lt;/em&gt;. How the time complexity &lt;em&gt;T&lt;/em&gt; for solving this system depends on &lt;em&gt;n&lt;/em&gt;, &lt;em&gt;m&lt;/em&gt; and &lt;em&gt;b&lt;/em&gt; (&lt;em&gt;T=O(??)&lt;/em&gt;) ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have some references about that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks!&lt;/p&gt;&#xA;" OwnerUserId="11916" LastEditorUserId="11916" LastEditDate="2014-11-28T08:19:07.063" LastActivityDate="2014-11-28T13:33:54.377" Title="Time complexity for sparse direct solver for SPD system with respect to number of equations, bandwidth, number of nonzeros?" Tags="|linear-solver|sparse-matrix|complexity|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16317" PostTypeId="1" CreationDate="2014-11-27T14:12:48.477" Score="1" ViewCount="265" Body="&lt;p&gt;I was asked to prove that the time complexity of merge sort is $ O(log_2n)$ but I cannot find a way to continue my method. Any help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T(n)=2T(\frac{n}{2} )+n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T(n)= 2[2T(\frac{n}{4})+n] +n = 4T(\frac{n}{4})+3n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T(n)=8T(\frac{n}{8})+7n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$...$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$...$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$...$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$...$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T(n)= 2^kT(\frac{n}{2^k})+(2^k-1)n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally $\frac{n}{2^k}=1$ and $\therefore n=2^k$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know how to continue from here to prove that it is $O(log_2n)$&lt;/p&gt;&#xA;" OwnerUserId="11767" LastActivityDate="2014-12-12T22:57:03.060" Title="How to prove time complexity of merge sort" Tags="|algorithms|complexity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16318" PostTypeId="1" CreationDate="2014-11-27T15:44:22.643" Score="3" ViewCount="157" Body="&lt;p&gt;I want to compute the following conditional expectation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E_{t}[\phi(A_{t+1}, \eta_{t+1})| A_t]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\log A_{t}=\rho \log A_{t-1} + e_{t}$ and $e_{t}$ is IID $N~(0,\sigma_e)$ and $\eta_{t}$ is another normally distributed variable with $(0,\sigma_{\eta})$. The function $\phi(.)$ is known. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unsure if someone wants to compute this through numerical methods how should use the quadrature techniques or any suitable alternatives. &lt;/p&gt;&#xA;" OwnerUserId="9826" LastEditorUserId="20688" LastEditDate="2019-07-05T06:04:24.543" LastActivityDate="2019-07-05T06:04:24.543" Title="Computing expectations" Tags="|numerics|quadrature|probability|integral-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16319" PostTypeId="1" AcceptedAnswerId="16321" CreationDate="2014-11-27T16:26:00.197" Score="2" ViewCount="245" Body="&lt;p&gt;In LeVeque's Finite Volume Methods for Hyperbolic Problems, p. 320-321, one may find the derivation of the Roe matrix to the 1D Shallow Water Equations (SWEs). It is&#xA;$$&#xA;\hat{A}_{i-1/2}=\begin{pmatrix}0&amp;amp;1\\-\hat{u}^2+g\bar{h}&amp;amp;2\hat{u}\end{pmatrix}&#xA;$$&#xA;where $\bar{h}=(h_i+h_{i-1})/2$ and&#xA;$$&#xA;\hat{u}=\frac{\sqrt{h_{i-1}}u_{i-1}+\sqrt{h_i}u_i}{\sqrt{h_{i-1}}+\sqrt{h_i}}&#xA;$$&#xA;is the Roe average. I just want to validate that this in fact &lt;em&gt;is&lt;/em&gt; a Roe matrix to the SWEs by checking the &lt;a href=&quot;http://en.wikipedia.org/wiki/Roe_solver&quot; rel=&quot;nofollow&quot;&gt;three conditions&lt;/a&gt;. I can without problem check the first two, i.e. that the matrix is diagonalizable with real eigenvalues and that the consistency with the exact Jacobian; however, I am struggling with the third condition. In LeVeque, p. 318, it says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If $Q_{i-1}$ and $Q_i$ are connected by a single wave $\mathcal{W}_p=Q_i-Q_{i-1}$ in the true Riemann solution, then $\mathcal{W}_p$ should also be an eigenvector of $\hat{A}_{i-1/2}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(This statement is equivalent with the one on Wikipedia).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempt:&lt;/strong&gt; Assume that $Q_{i-1}$ and $Q_i$ are connected by a single wave $\mathcal{W}_p=Q_i-Q_{i-1}$ in the true Riemann solution. Using the Rankine-Hugoniot condition this means&#xA;$$&#xA;f(Q_i)-f(Q_{i-1})=s(Q_i-Q_{i-1})&#xA;$$&#xA;where $s$ is the shock speed. So if $Q_i=(q^1_i,q^2_i)$ and $Q_{i-1}=(q^1_{i-1},q^2_{i-1})$ then&#xA;$$&#xA;f(Q_i)-f(Q_{i-1})=\begin{pmatrix}q^2_i\\(q^2_i)^2/q^1_i+\frac{1}{2}g(q^1_i)^2\end{pmatrix}-\begin{pmatrix}q^2_{i-1}\\(q^2_{i-1})^2/q^1_{i-1}+\frac{1}{2}g(q^1_{i-1})^2\end{pmatrix}\\&#xA;=\begin{pmatrix}q^2_i-q^2_{i-1}\\(q^2_i)^2/q^1_i-(q^2_{i-1})^2/q^1_{i-1}+\frac{1}{2}g((q^1_i)^2-(q^1_{i-1})^2)\end{pmatrix}\\&#xA;=s\begin{pmatrix}q_i^1-q_{i-1}^1\\q_i^2-q_{i-1}^2\end{pmatrix}&#xA;$$&#xA;From this we get a condition, namely&#xA;$$&#xA;\frac{q^2_i-q^2_{i-1}}{q^1_i-q^1_{i-1}}=\frac{(q^2_i)^2/q^1_i-(q^2_{i-1})^2/q^1_{i-1}+\frac{1}{2}g((q^1_i)^2-(q^1_{i-1})^2)}{q^2_i-q^2_{i-1}}&#xA;$$&#xA;By letting $\overline{q^1}=q^1_i-q^1_{i-1}$ and $\overline{q^2}=q^2_i-q^2_{i-1}$, this can be written&#xA;$$&#xA;\alpha=\frac{\overline{q^2}}{\overline{q^1}}=\frac{(q^2_i)^2/q^1_i-(q^2_{i-1})^2/q^1_{i-1}+\frac{1}{2}g((q^1_i)^2-(q^1_{i-1})^2)}{\overline{q^2}}&#xA;$$&#xA;and we get a relation between the components of the eigenvector $\mathcal{W}_p$. Note that&#xA;$$&#xA;\mathcal{W}_p=Q_{i}-Q_{i-1}=\begin{pmatrix}\overline{q^1}\\\overline{q^2}\end{pmatrix}=\begin{pmatrix}\overline{q^1}\\\alpha\overline{q^1}\end{pmatrix}=\overline{q^1}\begin{pmatrix}1\\\alpha\end{pmatrix}&#xA;$$&#xA;Now, the eigenvalues to the Roe matrix is given by $\hat{u}\pm\sqrt{g\bar{h}}$ with corresponding eigenvectors $(1,\hat{u}\pm\sqrt{g\bar{h}})$, so I really just have to show that $\alpha=\hat{u}\pm\sqrt{g\bar{h}}$; however, plugging the expressions in to Maple gives me that they are not. I have also tried in the other direction, namely to take the eigenvectors and plug it into $f(Q_i)-f(Q)$ but yet again it does not yield the desired result. Is there something else I have to assume – some relation that I am missing? It is funny, the derivation of the matrix is quite simple. &lt;/p&gt;&#xA;" OwnerUserId="11873" LastActivityDate="2014-11-28T09:06:56.743" Title="Unable to validate the Roe matrix for the Shallow Water Equations" Tags="|numerical-analysis|finite-volume|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16320" PostTypeId="2" ParentId="16317" CreationDate="2014-11-28T04:42:01.703" Score="2" Body="&lt;p&gt;You can do the algebra yourself or you can just apply the master theorem &lt;a href=&quot;http://en.wikipedia.org/wiki/Master_theorem&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Master_theorem&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11923" LastActivityDate="2014-11-28T04:42:01.703" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16321" PostTypeId="2" ParentId="16319" CreationDate="2014-11-28T07:00:02.063" Score="1" Body="&lt;p&gt;You do not need to fiddle around with simple waves to show that the Roe-mean satisfies the third condition. Just plug&#xA;\begin{align}&#xA;\hat{u}&amp;amp;=\frac{\sqrt{h_{1}}u_{1}+\sqrt{h_2}u_2}{\sqrt{h_{1}}+\sqrt{h_2}}\\&#xA;\bar{h}&amp;amp;=(h_1+h_2)/2&#xA;\end{align}&#xA;into &#xA;$$&#xA;\hat{A}_{i-1/2}=\begin{pmatrix}0&amp;amp;1\\-\hat{u}^2+g\bar{h}&amp;amp;2\hat{u}\end{pmatrix}&#xA;$$&#xA;and verify $\hat A(q_2-q_1) = F(q_2) - F(q_1)$ being&#xA;$$&#xA;\begin{pmatrix}0&amp;amp;1\\-\hat{u}^2+g\bar{h}&amp;amp;2\hat{u}\end{pmatrix}&#xA;\begin{pmatrix}&#xA;h_2-h_1\\&#xA;(hu)_2-(hu)_1&#xA;\end{pmatrix}&#xA;=&#xA;\begin{pmatrix}&#xA;(hu)_2 - (hu)_1\\&#xA;\frac g2\left(h_2^2 - h_1^2\right)+h_2 u_2^2- h_1 u_1^2&#xA;\end{pmatrix}.&#xA;$$&#xA;(That turns out to be true.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the second equation component you get already the interesting component with $g=0$. The left-hand side is:&#xA;\begin{align}&#xA;&amp;amp;=-\frac{(\sqrt{h_1}u_1 + \sqrt{h_2}u_2}{(\sqrt{h_1}+\sqrt{h_2})^2}\cdot&#xA;(\sqrt{h_1}+\sqrt{h_2})(\sqrt{h_2}-\sqrt{h_1})&#xA;+\frac{\sqrt{h_1}u_1+\sqrt{h_2}u_2}{\sqrt{h_1}+\sqrt{h_2}}((hu)_2-(hu)_1)&#xA;\end{align}&#xA;In the first term $\sqrt{h_1}+\sqrt{h_2}$ cancels out and you get a common denominator $\sqrt{h_1}+\sqrt{h_2}$.&#xA;\begin{align}&#xA;&amp;amp;=\frac{-(\sqrt{h_1}u_1+\sqrt{h_2}u_2)((hu)_2-(hu)_1+\sqrt{h_1h_2}(u_1-u_2))&#xA;+2(\sqrt{h_1}u_1+\sqrt{h_2}u_2)((hu)_2-(hu)_1)&#xA;}{\sqrt{h_1}+\sqrt{h_2}}\\&#xA;&amp;amp;=\frac{(\sqrt{h_1}u_1+\sqrt{h_2}u_2)((hu)_2-(hu)_1-\sqrt{h_1h_2}(u_1-u_2))}{\sqrt{h_1}+\sqrt{h_2}}\\&#xA;&amp;amp;=\frac{(\sqrt{h_1}u_1+\sqrt{h_2}u_2)(\sqrt{h_2}u_2-\sqrt{h_1}u_1)(\sqrt{h_1}+\sqrt{h_2})}{\sqrt{h_1}+\sqrt{h_2}}\\&#xA;&amp;amp;=h_2 u_2^2 - h_1 u_1^2&#xA;\end{align}&#xA;Avoid formula manipulation software for such things -- use your own brain (as far as possible).&lt;/p&gt;&#xA;" OwnerUserId="7652" LastEditorUserId="7652" LastEditDate="2014-11-28T09:06:56.743" LastActivityDate="2014-11-28T09:06:56.743" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16322" PostTypeId="2" ParentId="16313" CreationDate="2014-11-28T08:47:09.567" Score="5" Body="&lt;p&gt;The complexity of direct solver for SPD problems depends essentially of the properties of the underlying graph. If your matrix comes from the discretization of an elliptic equation then it depends essentially on the mesh/ structure which supports the discretization. This is due to the fact that direct methods are based on elimination of unknowns which can generate fill-in (ie denser subgraphs) and therefore more work in the forecoming eliminations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance if the graph of your matrix is a tree you can eliminate unknowns one by one and the complexity of a direct solver is $O(n)$ where is the number of nodes in the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if the graph is a planar graph (a graph that you can draw on a 2d plane), a well known result by Tarjan and Rose (1979) shows  that the complexity is $O(n^{3/2})$.&#xA;Thanks to their structures, those graphs provide fast elimination and are good candidate for direct methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, at the other side of the spectrum,  there are some graphs for which the complexity is maximal like complete graph for which we get $O(n^3$), their corresponding matrices are dense matrices.&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="2116" LastEditDate="2014-11-28T09:17:45.533" LastActivityDate="2014-11-28T09:17:45.533" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16323" PostTypeId="2" ParentId="16312" CreationDate="2014-11-28T11:30:08.887" Score="3" Body="&lt;p&gt;Remove &lt;code&gt;derivative(order=2)=LevelZ_der2,&lt;/code&gt; from the derivative-annotation of &lt;code&gt;LevelZ&lt;/code&gt; and replace the derivative annotation of &lt;code&gt;LevelZ_der&lt;/code&gt; with &lt;code&gt;derivative(order=2)=LevelZ_der2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Section 12.7.1 &quot;Using the Derivative Annotation&quot; of the &lt;a href=&quot;https://www.modelica.org/documents/ModelicaSpec33.pdf&quot; rel=&quot;nofollow&quot; title=&quot;Modelica Specification 3.3&quot;&gt;Modelica specification Version 3.3&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a problem of the tool if the external functions are not accepted after this correction.&lt;/p&gt;&#xA;" OwnerUserId="7652" LastEditorUserId="7652" LastEditDate="2014-11-28T17:58:32.973" LastActivityDate="2014-11-28T17:58:32.973" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16324" PostTypeId="2" ParentId="16313" CreationDate="2014-11-28T13:33:54.377" Score="2" Body="&lt;p&gt;The simple answer is that Gaussian elimination (without pivoting) for a matrix of bandwidth $b$ takes $O(nb^2)$ operations, without exploiting further sparsity properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general it is difficult to exploit further sparsity inside the bandwidth with a direct algorithm; classical Gaussian elimination will fill up the bandwidth quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the matrix is SPD, Gaussian elimination will be stable without the need for pivoting.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-11-28T13:33:54.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16325" PostTypeId="1" AcceptedAnswerId="18907" CreationDate="2014-11-28T14:35:42.473" Score="12" ViewCount="12599" Body="&lt;p&gt;I have a light ray moving through space-time, i.e. a curve in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^4$&lt;/span&gt;, parametrized by some variable λ. The exact trajectory, i.e. the coordinate functions &lt;span class=&quot;math-container&quot;&gt;$x^μ(λ)$&lt;/span&gt; of the curve are given by some ODE &lt;span class=&quot;math-container&quot;&gt;$\frac{dx^μ}{dλ} = f(λ)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I've got two requirements for my code to satisfy:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The results &lt;span class=&quot;math-container&quot;&gt;$x^μ(λ)$&lt;/span&gt; must be given on a regular grid, i.e. the step size dλ must be the same everywhere.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't know the interval λ runs over beforehand, i.e. integration must end dynamically as soon as a certain condition &lt;span class=&quot;math-container&quot;&gt;$g(λ, x^μ(λ))$&lt;/span&gt; is met (basically I'm testing whether the coordinates &lt;span class=&quot;math-container&quot;&gt;$x^μ(λ)$&lt;/span&gt; are outside of some coordinate interval where I don't know the space-time's properties anymore).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Requirement 2 obviously excludes &lt;code&gt;scipy.odeint()&lt;/code&gt;. This leaves me with the &lt;code&gt;scipy.ode&lt;/code&gt; class which allows me to pass the break condition to the integrator via a &lt;code&gt;solout&lt;/code&gt; callback if I choose the &lt;code&gt;dopri5&lt;/code&gt; or the &lt;code&gt;dop853&lt;/code&gt; algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a bit more complicated, though: Since these (and all other) integrators dynamically adjust their internal step size (which is kind of the point of not doing the integration manually) I cannot simply store the values being passed to the callback if I want to fulfill requirement 1. (Update: In fact, it doesn't make sense to do so, anyway. See my remark (*) below.) Instead, I still need to loop over λ and call the &lt;code&gt;scipy.ode.integrate()&lt;/code&gt; method for each step &lt;span class=&quot;math-container&quot;&gt;$λ + dλ$&lt;/span&gt;. This, however, means that I must somehow stop the loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.integrate import ode&#xA;&#xA;def deriv(λ, x, params):&#xA;    # …&#xA;    return dx_dλ&#xA;&#xA;end_reached = False&#xA;&#xA;def solout(λ, x):&#xA;    if g(λ, x):&#xA;        # Stop integration&#xA;        # Returning -1 will NOT make solver.successful() return False, so&#xA;        # we need to inform the integration loop (see below) separately.&#xA;        nonlocal end_reached&#xA;        end_reached = True&#xA;        return -1&#xA;&#xA;    # Everything's ok. Move on.&#xA;    return 0&#xA;&#xA;λ0 = 0&#xA;x0 = …&#xA;params = …&#xA;dλ = 0.1&#xA;&#xA;solver = ode(deriv)&#xA;solver.set_integrator('dopri5', nsteps=500)&#xA;solver.set_solout(solout)&#xA;solver.set_initial_value(x0, λ0)&#xA;solver.set_f_params(params)&#xA;&#xA;λ = [λ0]&#xA;x = [x0]&#xA;&#xA;while True:&#xA;    solver.integrate(solver.t + dλ)&#xA;&#xA;    if not solver.successful() or end_reached:&#xA;        break&#xA;&#xA;    λ.append(solver.t)&#xA;    x.append(solver.y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All in all, this is rather ugly. Not only is the logic basically all over the place (for instance I cannot test the condition in the loop where it would belong). But due to the &lt;code&gt;solout&lt;/code&gt; callback (I suppose) it's also at least one order of magnitude slower than without one. &lt;strong&gt;Update&lt;/strong&gt;: In fact, even without the callback, the &lt;code&gt;dopri5&lt;/code&gt; integrator is orders of magnitude slower than &lt;code&gt;vode&lt;/code&gt; (at least for another ODE where I wanted to use this approach).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something or is this really the only way to meet the above requirements? I also tried raising an exception in &lt;code&gt;deriv()&lt;/code&gt; (and subsequently catching it in the loop) in order to stop the integrator. While this works for all integrators, it will still lead to a bunch of errors being printed to stdout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(*) As far as I understand, the values passed to &lt;code&gt;deriv()&lt;/code&gt; are not meant to be a reliable approximation but only the final value returned by the solver is. For instance, the solver might do overshooting to choose a sensible step size in view of (rapidly) changing slopes and to thereby arrive at a best estimate.&lt;/p&gt;&#xA;" OwnerUserId="11927" LastEditorUserId="11927" LastEditDate="2020-03-09T10:45:25.840" LastActivityDate="2020-03-09T10:45:25.840" Title="Dynamically ending ODE integration in SciPy" Tags="|python|ode|scipy|grid|" AnswerCount="5" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="16326" PostTypeId="1" AcceptedAnswerId="29057" CreationDate="2014-11-28T17:57:46.287" Score="0" ViewCount="463" Body="&lt;p&gt;I want to debug a c++ code with Netbeans. I can use the debugger in sequential mode but I dont know how to debug it when I want to run it in parallel. I would be grateful if anyone help me on this problem. &lt;/p&gt;&#xA;" OwnerUserId="11575" LastActivityDate="2018-03-15T23:21:38.857" Title="Debugging in parallel mode using Netbeans" Tags="|c++|mpi|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16327" PostTypeId="1" AcceptedAnswerId="16330" CreationDate="2014-11-28T19:05:43.050" Score="5" ViewCount="1614" Body="&lt;p&gt;I have a bunch of 3x3 linear systems of the form $Ax=b$. In general, would it be faster to solve each individual system, or to formulate it as a giant block diagonal system and solve that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect to have a million or so systems to solve. I also was planning on using the Eigen C++ library, if that makes any differences, speed-wise.&lt;/p&gt;&#xA;" OwnerUserId="8196" LastActivityDate="2015-02-04T10:23:11.247" Title="Speed of solving linear system with block diagonal matrix" Tags="|c++|linear-system|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16328" PostTypeId="1" CreationDate="2014-11-28T20:40:16.063" Score="2" ViewCount="180" Body="&lt;p&gt;Using an established criterion involving capturing eddies down to the Kolmogorov length scale it can be reasoned that the order of  grid points in the computational mesh needs to be $N^3 \ge Re^{9/4}$ where Re is the Reynolds number of the flow. If $Re \approx 50,000$ then I get about $N^3 \approx 3.7\times 10^{10}$. I know very little about CFD but I am wondering if this is a outrageous requirement given today's high performance computers? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can get access to nodes of a supercomputer cluster. Im not sure of the exact specifications of the cluster, nor which specs are relevant for our purposes of calculating the total time, so if someone could make an assumption and base their calculations on this spec, I could modify my expected computing time once I know what im dealing with.&lt;/p&gt;&#xA;" OwnerUserId="7864" LastActivityDate="2014-11-29T16:47:02.097" Title="Expected computational time for DNS computation of fluid flow" Tags="|fluid-dynamics|performance|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16329" PostTypeId="2" ParentId="16327" CreationDate="2014-11-29T01:49:05.683" Score="1" Body="&lt;p&gt;If there's only a million of them, you should be able to easily solve all of them in about 1 second on a decent laptop or desktop, so performance shouldn't even be a concern (unless 1 second is still too slow for your intended application).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate this, consider the following Mathematica code, which solves 1 million random $3\times 3$ systems:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;First@Timing@Table[LinearSolve[RandomReal[{0, 1}, {3, 3}]], {1000000}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On my ancient 7-year-old budget laptop using a single CPU core, this takes 30 seconds. With 4-6 parallel CPU cores on a modern desktop computer, this should take about 1 second. In CUDA with a GPU exploiting the parallelism, you could probably do it in milliseconds.&lt;/p&gt;&#xA;" OwnerUserId="6928" LastEditorUserId="6928" LastEditDate="2014-11-29T01:55:29.507" LastActivityDate="2014-11-29T01:55:29.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16330" PostTypeId="2" ParentId="16327" CreationDate="2014-11-29T06:33:58.517" Score="1" Body="&lt;p&gt;The numerical costs of the problem itself are linearly dependent on the number of rows. You reach these minimal costs if you solve the blocks separately. The usual matrix solvers that do not exploit the internal structure of the system cause numerical costs that grow with the third power of the number of rows. So, they are more expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, there are general solvers that detect the matrix structure and then solve the system with linear costs. The detection of the matrix structure has costs that are quadratic in the number of rows but have a very low impact since the required time for testing one matrix element is very low. We are using such a solver for &lt;a href=&quot;http://simulationx.com&quot; rel=&quot;nofollow&quot;&gt;SimulationX&lt;/a&gt;. With this solver the time for solving the linear system is rarely the limiting factor in the simulation.&lt;/p&gt;&#xA;" OwnerUserId="7652" LastActivityDate="2014-11-29T06:33:58.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16331" PostTypeId="1" CreationDate="2014-11-29T09:41:38.263" Score="11" ViewCount="3009" Body="&lt;p&gt;I have a question about Galerkin method. I don't understand why the Galerkin method weights the residual by the shape functions and sets it equal to zero. I want to know what is reason of this. Why we must set weights residual functions equal to zero?&lt;/p&gt;&#xA;" OwnerUserId="11936" LastEditorUserId="1804" LastEditDate="2014-12-01T07:02:56.130" LastActivityDate="2016-06-22T20:38:00.697" Title="Motivation behind Galerkin method" Tags="|finite-element|projection|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16332" PostTypeId="2" ParentId="16331" CreationDate="2014-11-29T11:59:51.307" Score="4" Body="&lt;p&gt;Let's say you want to solve the Laplace equation, $-\Delta u = f$. Ideally, of course, you'd like to find a function $u$ so that the residual is zero: $r(u) = -\Delta u - f = 0$. But $u$ is an infinite dimensional object which in general we cannot represent on computers, so we have to find finite dimensional approximations $u_h$. Since $u_h$ is not the exact solution, we cannot expect that $r(u_h)=0$. The question is which set of equations we want $u_h$ to satisfy instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Galerkin method chooses $u_h=\sum_i U_i \varphi_i$ to be a linear combination of $N$ shape functions $\varphi_i$ and then determines the coefficients $U_i$ by requiring the residual to satisfy the set of $N$ equations $\left&amp;lt;\varphi_i,r(u_h)\right&amp;gt;=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are other choices that are possible. For example, the Petrov-Galerkin method requires that $\left&amp;lt;\psi_i,r(u_h)\right&amp;gt;=0$ where the test functions $\psi_i$ are a set of $N$ weighting functions separate from the trial functions $\varphi_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a bit more material on this issue in lecture 4 at &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-29T11:59:51.307" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16333" PostTypeId="2" ParentId="16328" CreationDate="2014-11-29T12:02:38.260" Score="3" Body="&lt;p&gt;$3.7\cdot 10^{10}$ is feasible today on the very largest of computers, but barely so. Recall that you also have to do a lot of time steps (with shorter time step size the larger the Reynolds number is), so it is important that the local problems on every processor remain small. A reasonable number of unknowns per processor may be 100,000 which would translate to needing 370,000 processors -- possible today, but definitely not easily.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-29T12:02:38.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16334" PostTypeId="2" ParentId="16327" CreationDate="2014-11-29T12:06:30.610" Score="1" Body="&lt;p&gt;If you know something about a matrix, it almost always pays off to use it. In your case, use the fact that you know that it is composed of many $3\times 3$ blocks and solve them individually -- which you can do in parallel, for example. If you just give a large block diagonal matrix to an iterative or sparse direct solver, the best they could do is to analyze the matrix and do exactly what you could do yourself, but this analysis step is (i) not usually done, at least in iterative solvers, (ii) expensive.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-11-29T12:06:30.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16335" PostTypeId="2" ParentId="16318" CreationDate="2014-11-29T12:18:33.693" Score="3" Body="&lt;p&gt;Let's first rewrite this. From your formulas, you have that &#xA;$$&#xA;  A_{t+1} = A_t^\rho \exp(e_t)&#xA;$$&#xA;where $A_t$ is just the previous value and, consequently, just a fixed parameter.&#xA;So what you need to compute is then&#xA;$$&#xA;  E_t[\phi(A_{t+1},\eta_{t+1})|A_t]&#xA;  =&#xA;  \frac 1C&#xA;  \int_{-\infty}^\infty&#xA;  \int_{-\infty}^\infty&#xA;  \phi(A_t^\rho e^a,b)&#xA;  e^{-a^2/(2\sigma_e^2)}&#xA;  e^{-b^2/(2\sigma_\eta^2)}&#xA;  \; da \; db.&#xA;$$&#xA;How this is best computed numerically very much depends on the form of $\varphi$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2014-12-01T03:17:25.920" LastActivityDate="2014-12-01T03:17:25.920" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16336" PostTypeId="2" ParentId="16331" CreationDate="2014-11-29T13:32:58.023" Score="0" Body="&lt;p&gt;Remember that when you multiply the strong-form equation by the shape function, the shape function is &lt;strong&gt;arbitrary&lt;/strong&gt;. Therefore, by requiring that the residual be orthogonal to any such shape function, such a residual is in fact very close to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not the same as requiring that the residual be zero exactly, but rather a somewhat weakened requirement that the discrete solution can satisfy.&lt;/p&gt;&#xA;" OwnerUserId="6775" LastEditorUserId="6775" LastEditDate="2014-11-29T13:43:07.577" LastActivityDate="2014-11-29T13:43:07.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16337" PostTypeId="1" AcceptedAnswerId="16353" CreationDate="2014-11-29T14:00:18.897" Score="2" ViewCount="276" Body="&lt;p&gt;I am interested in studying a problem of the form $\min F(\Omega)$ where $\Omega$ varies in the class of convex, open sets in the plane. An idea is to deform $\Omega$ at each step using a steepest descent algorithm, and if the deformed shape $\Omega_d$ is not convex, project it in some way on the space of convex sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess there are different ways of doing this. I've tried replacing $\Omega_d$ by its convex hull. This is not satisfactory, since at some moment we might get into a cyclic situation: for example if $\Omega$ has a segment in its boundary, the algorithm pulls the segment inwards, but convexification brings it back where it was.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid this, we could consider a form $\Omega'$ which convex, and is closest to $\Omega$ in the sense of least squares. For examples, if $\rho_0$ is a radial parametrization of $\Omega$, and $(x_i)$ is a discretization of $[0,2\pi)$ then we could solve&#xA;$$ \min_{\rho} \sum_{i = 1}^n (\rho(x_i)-\rho_0(x_i))^2$$&#xA;where $\rho$ is the radial function of a convex shape. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any simple way to impose the convexity constraint on $\rho$ in the discrete setting? In the end, the above optimization problem is equivalent to fitting the closest convex polygon to a set of points in the plane, in the sense of least squares. Is there any known algorithm to find such a best fit convex polygon.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If I write the analytic condition (positive curvature in 2D), it will need the derivatives $\rho',\rho''$, which is not pleasant: if I parametrize the radial function by Fourier coefficients, it comes to solving a quadratic optimization problem under quadratic constraints.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2014-12-01T03:52:32.353" Title="Projecting onto convex shapes - best fit convex polygon" Tags="|convex-optimization|least-squares|convex-hull|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16338" PostTypeId="1" AcceptedAnswerId="16352" CreationDate="2014-11-29T15:44:52.080" Score="2" ViewCount="1491" Body="&lt;p&gt;I was reading through &lt;a href=&quot;http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_query?1992ARA%26A..30..543M&amp;amp;defaultprint=YES&amp;amp;filetype=.pdf&quot; rel=&quot;nofollow&quot;&gt;Monaghan, SPH, 1992&lt;/a&gt; paper and he states that any quantity $A$ at $r_b$ can be approximated by $A(r) = \sum_b m_b \frac{A_b}{\rho_b}W(r-r_b,h)$ and so the gradient of that quantity can be computed as $\nabla A(r) = \sum_b m_b \frac{A_b}{\rho_b} \nabla W(r-r_b,h)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if we want to approximate the density we have $\rho (r) = \sum_b m_b W(r-r_b,h)$, so far so good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there is the equation to approximate the divergence of the velocity as $\nabla \cdot v = \sum_b m_b v_b \cdot \nabla W(r-r_b,h)$ which I don't understand exactly where it comes from. Is the velocity $v$ playing the role of the approximated quantity, $A_b = v$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, then the&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;second golden rule of SPH is to rewrite formulae with density placed&#xA;  inside operators&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which gives $\nabla \cdot v = [\nabla \cdot (\rho v) - v \cdot \nabla \rho] / \rho$. How is this obtained? why is the density appearing in the formula and how is related to the above formula for the divergence of the velocity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess something similar happens when deriving the estimator for the curl of the velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain a little more in details how are the formulas derived?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="549" LastEditDate="2014-11-29T18:16:27.323" LastActivityDate="2020-02-27T09:21:46.773" Title="How is the divergence and curl approximated in Smoothed Particle Hydrodynamics?" Tags="|sph|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16340" PostTypeId="2" ParentId="16327" CreationDate="2014-11-29T15:53:20.393" Score="0" Body="&lt;p&gt;I agree with Mr. DumpsterDoofus's comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a fairly modest 5-year-old CPU (AMD Phenom II X4 830), this Eigen code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigen::PartialPivLU&amp;lt; Eigen::Matrix3d &amp;gt; lu(a);&#xA;Eigen::Vector3d u = lu.solve(b);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;requires about .37 seconds to execute one million times.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-11-29T15:53:20.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16341" PostTypeId="2" ParentId="16338" CreationDate="2014-11-29T16:02:42.223" Score="4" Body="&lt;p&gt;Your second question is answered by expanding the divergence of $\rho v$ and rearranging terms:&#xA;$$&#xA;\nabla\cdot({\rho v}) = \nabla\rho\cdot v + \rho \nabla \cdot v$$&#xA;so&#xA;$$&#xA;\nabla \cdot v = \frac{\nabla\cdot({\rho v}) - \nabla\rho\cdot v}{\rho}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-11-29T16:02:42.223" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16342" PostTypeId="2" ParentId="16328" CreationDate="2014-11-29T16:47:02.097" Score="3" Body="&lt;p&gt;We have people doing $4096^3$ and $8192^3$ mesh points 3D domains on our systems at TACC using 16384 cores (1024 nodes). These simulations take months of running time to achieve the results the scientists want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your Reynolds number is equivalent to a $\sim 3332^3$ domain, which is smaller than what we have running on our systems, but not by much. You're going to need a large computer, but not, by a fair amount, the largest in the world. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-11-29T16:47:02.097" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16343" PostTypeId="1" AcceptedAnswerId="16344" CreationDate="2014-11-29T19:11:00.897" Score="5" ViewCount="5105" Body="&lt;p&gt;Let curve $C_1$ be defined parametrically&#xA;$$\begin{aligned}&#xA;&amp;amp;x(t)=0.5\cos(t)-0.3\cos(3t),\\&#xA;&amp;amp;y(t)=1.2+0.6\sin(t)-0.07\sin(3t)+0.2\sin(7t)&#xA;\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I find if an arbitrary point $P(x,y)$ is inside or outside of this curve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for another curve $C_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;&amp;amp;x(t)=0.4+\big(0.19+0.07\sin(5t)\big)\cos(t)\\&#xA;&amp;amp;y(t)=0.4+\big(0.19+0.07\sin(5t)\big)\sin(t)&#xA;\end{aligned}&#xA;$$&#xA;I tried with the set $r=0.19+0.07\sin(5t)$ and $t=\tan^{-1}(y/x)$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;if $(x-0.4)^2+(y-0.4)^2\leq r^2$  I supposed that point is inside, but this is not correct for some points.&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="20688" LastEditDate="2017-04-14T16:51:24.703" LastActivityDate="2017-04-14T16:51:24.703" Title="How to determine if a point is outside or inside a curve" Tags="|computational-geometry|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16344" PostTypeId="2" ParentId="16343" CreationDate="2014-11-29T20:13:06.123" Score="11" Body="&lt;p&gt;There's a &lt;a href=&quot;http://en.wikipedia.org/wiki/Point_in_polygon&quot;&gt;simple test&lt;/a&gt; to see if a point $(x, y)$ is enclosed within a curve. Draw a ray from $(x, y)$ to infinity, and count how many times it crosses the curve; if the count is odd, then $(x, y)$ is inside the enclosed region; otherwise, it's outside.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To turn this into a practical algorithm, you could first build polygonal approximation of the curve and look for all segments that the ray from $(x, y)$ intersects. This operation is linear in the number of segments of the polygon. Once you have all the candidate segments, you can use these as the basis for a Newton search to find a more exact point of intersection of the ray with the curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last step might seem overly cautious, but if the point is closer to the curve than the accuracy of the piecewise-linear approximation, you can easily get a wrong answer. Since you have an analytic expression for the curve, you can also analytically compute its derivative; if the ray is really close to being tangent to the curve, expect trouble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even for curves that are exactly polygonal, the even/odd rule can fail in finite precision arithmetic. Computational geometry is tricky because there are often bizarre, unintuitive edge cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://books.google.com/books?isbn=3540779736&quot;&gt;book by de Berg and Cheong&lt;/a&gt; is a great reference on the subject; if you want a Python library to experiment with, I've had good luck with &lt;a href=&quot;https://pypi.python.org/pypi/Shapely/&quot;&gt;Shapely&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2014-11-29T20:13:06.123" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16345" PostTypeId="2" ParentId="16343" CreationDate="2014-11-30T10:27:07.107" Score="8" Body="&lt;p&gt;There's also the &lt;a href=&quot;http://en.wikipedia.org/wiki/Point_in_polygon#Winding_number_algorithm&quot; rel=&quot;noreferrer&quot;&gt;winding number test&lt;/a&gt;, described in the same Wikipedia article that Daniel linked to. The two algorithms give equivalent results for simple curves like yours, but they can differ for self-intersecting curves like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/k21Pd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ray casting method considers both points to lie outside the curve. On the other hand, the winding numbers of $p$ and $q$ are $2$ and $0$ respectively, so if you want to consider points like $p$ to lie inside the curve you can use the &lt;a href=&quot;http://en.wikipedia.org/wiki/Nonzero-rule&quot; rel=&quot;noreferrer&quot;&gt;nonzero rule&lt;/a&gt; instead of the even-odd rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the caveats Daniel mentioned about sufficient sampling apply to both methods.&lt;/p&gt;&#xA;" OwnerDisplayName="user3883" LastActivityDate="2014-11-30T10:27:07.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16346" PostTypeId="2" ParentId="7608" CreationDate="2014-11-30T18:56:37.923" Score="1" Body="&lt;p&gt;This theorem has either been mis-interpreted by the question author or there is a mistake in the referenced book. Consider the following counter example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = 100+x$$ $$h=0.01$$ $$R = 0.01$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At $x=0$ the absolute error in each function evaluation is $100*0.01=1$, so we have&#xA;$$f'^-(0) = \frac{f^-(h) - f^- (-h)}{2h} = \frac{(100+0.01)\pm 1 - ((100-0.01) \pm 1)}{0.02} $$&#xA;$$ \implies f'^-(0) = \frac{0.02}{0.02} \pm \frac{1}{0.02}\pm \frac{1}{0.02}$$&#xA;In the worst case scenario, the two error terms have the same sign and do not cancel out. The relative error of the derivative approximation can therefore be as much as $100$, which is much greater than $R/h = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell there is no bound on the relative error for a general $f$ since by choosing a function of the form $f(x) = n+x$, the relative error in the derivative approximation can always be increased simply by increasing $n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand we can calculate a bound dependent on $f$. The bound on the absolute error for sufficiently small $h$ and $R$ is:&#xA;$$ f'^-(x) = \frac{f(x+h)-f(x-h)}{2h} \pm \frac{f(x)R}{h} $$&#xA;Proof:&#xA;$$f^-(x+h) = (f(x)+hf'(x))(1 \pm R)$$&#xA;$$ \implies f^-(x+h) =f(x)+hf'(x)\pm Rf(x)$$&#xA;where we Taylor expand $f$ around $x$ and neglect terms of order $hR$ or $h^2$ or higher since both $h$ and $R$ are small. Similarly&#xA;$$ f^-(x-h) = f(x) - hf'(x) \pm Rf(x)$$&#xA;Therefore&#xA;$$f'^-(x) = \frac{2hf'(x) \pm Rf(x) \pm Rf(x)}{2h} $$&#xA;$$ \implies f'^-(x) =f'(x) \pm \frac{Rf(x)}{h}$$&#xA;where we once again consider the worst case scenario, in which the errors add up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bound on the relative error is therefore dependent on $f(x)$ and can be expressed as&#xA;$$ f'^-(x) = f'(x)\left(1\pm\frac{Rf(x)}{hf'(x)}\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly we have&#xA;$$f''^-(x) = f''(x)\left(1 \pm \frac{4Rf(x)}{h^2f''(x)}\right)$$&lt;/p&gt;&#xA;" OwnerUserId="9448" LastEditorUserId="9448" LastEditDate="2014-11-30T20:10:44.863" LastActivityDate="2014-11-30T20:10:44.863" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16347" PostTypeId="1" CreationDate="2014-11-30T22:26:51.163" Score="1" ViewCount="267" Body="&lt;p&gt;Suppose I have an unstructured polygonal mesh system like so: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zKAjB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each node $x$ has Cartesian coordinates $(x_1,x_2)$, so for a given node can form matrices like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;J(x,y,z) = \left(\begin{array}{cc}&#xA;y_1-x_1 &amp;amp; z_1-x_1 \\ &#xA;y_2-x_2 &amp;amp; z_2-x_2 &#xA;\end{array}\right) = (y-x,z-x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If my mesh were structured, i.e. a mapped Cartesian grid so that $(x_1,x_2) = (x_1(\xi,\eta),x_2(\xi,\eta))$, then these matrices would be approximations to the Jacobian matrix of the map $(\xi,\eta)\mapsto x$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, for an unstructured mesh like the one above, what are my $J(x,y,z)$ approximations of?  Is there a well-defined, underlying &quot;continuum&quot; mapping for an unstructured grid?  &lt;/p&gt;&#xA;" OwnerUserId="3303" LastActivityDate="2014-11-30T23:16:13.690" Title="Jacobian matrices on unstructured grids: underlying map?" Tags="|unstructured-mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16348" PostTypeId="2" ParentId="16347" CreationDate="2014-11-30T23:16:13.690" Score="3" Body="&lt;p&gt;For quadrilaterals and triangles, one standard thing to do is to use the isoparametric map. That is, you use the finite element basis you are about to use to approximate the solution to map your physical-space element to a &quot;reference element&quot; from the $\xi, \eta$ to $x, y$. You obviously have to add interior and edge nodes to do this for higher-order elements. Another common approach is to use the piecewise (bi-)linears for everything regardless of the actual finite-element order. If your boundaries are straight, the two approaches are equivalent for triangles. The typical reference elements are $[-1,1]\times[-1,1]$ and a right unit isosceles triangle in the upper right quadrant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I presume you could do something similar for higher-degree polygons, but I've never done that. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-11-30T23:16:13.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16349" PostTypeId="1" AcceptedAnswerId="16351" CreationDate="2014-11-30T23:42:53.770" Score="0" ViewCount="207" Body="&lt;p&gt;In CVX, how do we return the value of the parameter over which the problem is minimized at the optimal value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By this, I mean, how do we obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^* = \arg\min_x f(x)$$ when solving the problem with CVX?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have gone through the user's guide and did some google searching. The only thing I could find was &lt;code&gt;cvx_optval&lt;/code&gt; which only gives the optimal value of $f(x)$&lt;/p&gt;&#xA;" OwnerUserId="11910" LastActivityDate="2014-12-01T03:36:10.687" Title="CVX : Obtaining the minimizing parameter at the optimum" Tags="|convex-optimization|cvx|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16350" PostTypeId="1" CreationDate="2014-12-01T00:09:15.163" Score="4" ViewCount="216" Body="&lt;p&gt;Suppose I want to solve the PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\nabla\cdot\left( a_\epsilon\left(x\right)\nabla u\right)=0 \text{ in } \Omega$$&#xA;$$u=g(x) \text{ on }\partial\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, I assume that $a_\epsilon(x)=a(x,\frac{x}{\epsilon})$ for some $a(x,y)$ such that $a$ is 1-periodic in $y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that if I apply the standard finite difference scheme on this PDE with a uniform grid of stepsize $h$, I can obtain a numerical solution $u_h$ an error estimate of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$||u-u_h||\leq Ch^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C$ is a constant independent of $h$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found a paper which suggests that the error estimate can be further refined so that the constant $C$ is also independent of $\epsilon$ as well.  It is stated (without proof or reference) that for the above problem in 2D, the error estimate can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$||u-u_h||\leq C*\frac{h^2}{\epsilon^3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c^*$ only depends on the properties of the function $a$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the life of me, I can't seem to derive this estimate.  The above PDE is a textbook example of an &lt;strong&gt;&lt;em&gt;asymptotic homogenization problem&lt;/em&gt;&lt;/strong&gt;, but all the sources I've found seem to take this as a given without any proof.  Any insight into how this factor of $1/\epsilon^3$ is obtained would be greatly appreciated.  I highly suspect that this formula may only apply in the 2D case as stated above.  I wonder if there is a generalized formula for arbitrary dimensions.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-12-01T01:03:37.827" LastActivityDate="2015-02-12T05:52:58.297" Title="Finite Difference Error Estimate for an Elliptic PDE with an Oscillatory Coefficient" Tags="|finite-difference|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16351" PostTypeId="2" ParentId="16349" CreationDate="2014-12-01T03:36:10.687" Score="2" Body="&lt;p&gt;It's just stored in the regular MATLAB variable x (or whatever you were using as the variable in your problem.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-12-01T03:36:10.687" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16352" PostTypeId="2" ParentId="16338" CreationDate="2014-12-01T03:48:46.623" Score="6" Body="&lt;p&gt;When you approximate a quantity, for example $A(r) = \sum_b m_b \frac{A_b}{\rho_b}W(r-r_b,h)$, then what you are saying is that you are assigning a scalar value $A_b$ to every nodal point $b$. At any point $r$, this formula then simply provides a way to interpolate between the points $r_b$ in the neighborhood. Because the masses and nodal values $A_b$ are just fixed scalars, the gradient of your approximation is simply $\nabla A(r) = \sum_b m_b \frac{A_b}{\rho_b}\nabla W(r-r_b,h)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do the same construction with vector fields. If you assign a vector $v_b$ to every nodal point $b$, then your approximation becomes $\mathbf v(r) = \sum_b m_b \frac{\mathbf v_b}{\rho_b}W(r-r_b,h)$ and its divergence -- because the $\mathbf v_b$ are just fixed vectors, independent of $r$ -- is&#xA;$\nabla \cdot \mathbf v(r) = \sum_b m_b \frac{\mathbf v_b}{\rho_b} \cdot \nabla W(r-r_b,h)$. The same can of course be done to the curl:&#xA;$\nabla \times \mathbf v(r) = \sum_b m_b \nabla \times \left[\frac{\mathbf v_b}{\rho_b}  W(r-r_b,h)\right] = \sum_b m_b \left[\frac{\mathbf v_b}{\rho_b} \times \nabla W(r-r_b,h)\right]$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-01T03:48:46.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16353" PostTypeId="2" ParentId="16337" CreationDate="2014-12-01T03:52:32.353" Score="3" Body="&lt;p&gt;The set of convex sets is infinite dimensional, which is not amenable to computation. I would discretize this by replacing your solution set (consisting of all convex sets) by a finite dimensional set -- for example, polygons with $N$ vertices. This finite dimensional set can conveniently be represented as the intersection of $N$ half-spaces and you would then write your optimization over the parameters that determine these half spaces.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-01T03:52:32.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16354" PostTypeId="2" ParentId="7611" CreationDate="2014-12-01T07:01:23.253" Score="0" Body="&lt;p&gt;I highly recommend on &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt;. You can also look at &lt;a href=&quot;http://www.tau.ac.il/~stoledo/taucs/&quot; rel=&quot;nofollow&quot;&gt;taucs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10226" LastActivityDate="2014-12-01T07:01:23.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16355" PostTypeId="1" CreationDate="2014-12-01T13:21:06.653" Score="1" ViewCount="706" Body="&lt;p&gt;I want to study the Density matrix renormalization Group(DMRG). Can some one provide me with the souce code which is writen by &lt;strong&gt;C++&lt;/strong&gt; for study purpose only? The code is not necessiary very professional, but works.&lt;/p&gt;&#xA;" OwnerUserId="11941" LastActivityDate="2014-12-01T13:21:06.653" Title="Can you provide me with the source code of DMRG writen by C++?" Tags="|c++|computational-physics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16357" PostTypeId="2" ParentId="16331" CreationDate="2014-12-01T18:06:32.507" Score="29" Body="&lt;p&gt;When I studied the finite element method in graduate school, this notion of multiplying by a weight function was also very alien to me.  Eventually, I did find a nice (albeit non-rigorous) analogy that helped me understand it.  This analogy is based on 3D vector geometry and an understanding of projections and dot-product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3D Geometry&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine a 2D plane lying somewhere in 3D euclidean space.  This plane can be though of as the span of two vectors $v_1$ and $v_2$.  Thus, any vector $w$ in the plane can be written as a linear combination of these vectors; i.e. $w=c_1v_1+c_2v_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/VdSaV.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine a point $Q$ in 3D space that is not on the plane. Consider the question:  Of all the points on the plane, which point is &lt;em&gt;closest&lt;/em&gt; to $Q$?  It is the one point $w$ (not shown in the image above) that lies on the line passing through the point $Q$ and perpendicular to the plane.   The point $w$ is also known as the &lt;em&gt;orthogonal projection&lt;/em&gt; of $Q$ onto the plane.  Even though we don't know the coordinates of this point $w$, we do know that the vector between $Q$ and $w$ is perpendicular to all vectors that define the plane, i.e. $v_1$ and $v_2$.  Perpendicular also means that the dot product is zero.  If we denote the vector between $Q$ and $w$ as $\vec{Q}-\vec{w}$, then forcing $\vec{Q}-\vec{w}$ to be perpendicular to the plane also implies&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\vec{Q}-\vec{w})\cdot v_1=0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\vec{Q}-\vec{w})\cdot v_2=0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This results in a system of equations which we can solve Also notice that to construct $\vec{Q}-\vec{w}$, we must know the coordinate of $Q$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analogy for the Galerkin Method&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that the solution $u_h$ is a finite linear combination of functions $N_1$,..,$N_k$; thus, $u_h=C_1N_1+...+C_kN_k$.  This linear combination acts like the &lt;em&gt;plane&lt;/em&gt; in the discussion above.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's assume that there exists some &lt;em&gt;exact solution&lt;/em&gt; $u$, which we don't know.  This solution $u$ is like the point in 3D space which is not on the plane. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the galerkin method, we're looking for the solution in a space (plane) that is closest to the true solution (point not in the plane).  In this sense, the &quot;best solution&quot; is the choice of $u_h$ that the difference $u-u_h$ is perpendicular to the space $u_h$.  Note that for function spaces, the &quot;dot-product&quot; is defined by the integral of their product, i.e. $&amp;lt;u,v&amp;gt;=\int_\Omega u v$.  So, perpedicular implies that the dot product between the $u-u_h$ and all those basis functions $N_1$, ..., $N_k$ must be zero, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega (u-u_h)N_1 = 0$$&#xA;$$...$$&#xA;$$\int_\Omega (u-u_h)N_k = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you may be saying to yourself, &lt;strong&gt;&quot;This whole setup rests critically on the assumption that we know the exact solution $u$ ahead of time.  But the truth is that we generally don't know $u$ a priori.  In which case, how can we possibly compute $u-u_h$ in all these equations?&lt;/strong&gt;&quot;  I'm so glad you asked!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The truth is that we can't and don't evaluate $u-u_h$ directly. But we do know what $u$ and $u_h$ are supposed to satisfy; i.e. the PDE.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose your original PDE problem is&#xA;$$-\nabla\cdot\left(k(x)\nabla u\right)=f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could also rewrite this as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Au=f$$&#xA;where the operator $A$ is defined by the expression $Au=-\nabla\cdot\left(k(x) \nabla u \right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So instead of considering the &lt;strong&gt;&lt;em&gt;absolute difference&lt;/em&gt;&lt;/strong&gt; between the solutions $u-u_h$, we instead consider the &lt;strong&gt;&lt;em&gt;residual difference&lt;/em&gt;&lt;/strong&gt; $Au - Au_h$ in all of the (perpendicular) equations.  That is, consider not what $u$ and $u_h$ are, but rather what $u$ and $u_h$ satisfy instead.  By replacing the absolute difference with the residual difference in the (perpendicular) equations above, we can write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega (Au-Au_h)N_1 = 0$$&#xA;$$...$$&#xA;$$\int_\Omega (Au-Au_h)N_k = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, we still don't know what $u$ is, so this may not seem very helpful.  But in fact,  we can replace $Au$ with the known source term $f$ (since $Au=f$).  Thus, we obtain the equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega (f-Au_h)N_1 = 0$$&#xA;$$...$$&#xA;$$\int_\Omega (f-Au_h)N_k = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, enforcing the residual to be orthogonal to the given space results in a system of equations that one can solve for the coefficients $C_1,...,C_k$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explanation above is a rough &quot;analogy&quot;.  I haven't really derived anything or given a reasonable proof that $u-u_h$ can be replaced by $Au-Au_h$ and still produce a close approximation.  I've also not explained anything about obtaining a weak form of the PDE or how to choose the spaces where $N_1...N_k$ lie in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the whole idea behind galerkin method as a projection is that for all possible linear combinations of functions in a given (finite dimensional) space (span of $N_1$,...,$N_k$), we are looking for the one that is closest to a solution (point) which generally lies outside of the given space.  Closest means that we're looking for the orthogonal projection from the true solution to the given space.  If we don't know what the exact solution $u$ is, then we cannot use the &lt;em&gt;absolute difference&lt;/em&gt; as a metric in our projection.  Thus, we are forced to use the &lt;em&gt;residual difference&lt;/em&gt; as the metric of our projection; In other words:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The galerkin projection is not about what $u$ &lt;em&gt;is&lt;/em&gt;... it's about what $u$ &lt;em&gt;satisfies&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-12-02T16:05:55.227" LastActivityDate="2014-12-02T16:05:55.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16359" PostTypeId="1" CreationDate="2014-12-01T22:58:34.713" Score="2" ViewCount="2500" Body="&lt;p&gt;I am currently writing a code that solves a large tridiagonal matrix every iteration and runs for 1,000's of iterations. I am currently using a Thomas algorithm to solve the matrix serially. I found a parallel version of the Thomas algorithm in the book called &quot;Parallel Scientific Computing in C++ and MPI&quot; (which you can find a version pretty easily if you google it). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is when I run the the parallel Thomas taken directly from the book, it actually has a slower run time than the serial algorithm. This seems baffling to me as the whole point of creating a parallel algorithm is to speed it up. I tested both by running the serial and parallel algorithms verbatim from the book and sending the same tridiagonal into the function for each. I am running MPI on a school cluster to run these and submitting a job for the parallel runs to ensure I am using the same processor type for each node. Could it just be the communication time from the layout of the cluster that is causing this? I feel like I must be doing something wrong so any help would be much appreciated. If posting each algorithm would be of use, let me know.&lt;/p&gt;&#xA;" OwnerUserId="11605" LastActivityDate="2014-12-10T03:45:32.340" Title="Parallel vs Serial Thomas Algorithm" Tags="|parallel-computing|c++|mpi|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16360" PostTypeId="1" AcceptedAnswerId="16365" CreationDate="2014-12-01T23:31:28.900" Score="8" ViewCount="2423" Body="&lt;p&gt;For a silly screen saver I'm trying to develop, I'd like to randomly generate a divergence-free 2D array of 2D vectors, and then use it to generate a line integral convolution plot. I've heard$^1$ that one way to do this is to generate random noise, and then project out the solenoidal component of its Helmholtz-Hodge decomposition. To do that, I tried using the following reasoning:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A function $\mathbf{f}:\mathbb{R}^2\rightarrow\mathbb{R}^2$ has Helmholtz-Hodge decomposition$^2$&#xA;$$\mathbf{f}=\mathbf{h}+\nabla\phi+J\nabla\psi$$&#xA;where $$J=\pmatrix{0&amp;amp;-1\\1&amp;amp;0}$$&#xA;and where $\phi,\psi$ are scalar functions. For now, assume the harmonic component $\mathbf{h}$ vanishes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Fourier space, this becomes&#xA;$$\mathcal{F}\mathbf{f}=-i\mathbf{k}\hat{\phi}-iJ\mathbf{k}\hat{\psi}$$&#xA;and we can define a solenoidal projection operator on Fourier space as&#xA;$$P=I-\frac{\mathbf{k}\otimes\mathbf{k}}{\mathbf{k}\cdot\mathbf{k}}$$&#xA;which projects a function on its solenoidal component, via&#xA;$$\mathcal{F}^{-1}P\mathcal{F}\mathbf{f}=J\nabla\psi.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then tried to implement this in Mathematica, applying it to a $21\times 21\times 2$ random array. First I generate the random array, and take apply the FFT to each of its two components:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arr = RandomReal[{-1, 1}, {2, 21, 21}];&#xA;fArr = Fourier /@ arr;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then define $\mathbf{k}$ as a function of array index:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k[k1_, k2_] := Mod[{k1 - 1, k2 - 1}, 21, -10]/21;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I perform the projection on the Fourier components (the singularity at $\mathbf{k}=0$ is left alone using an &lt;code&gt;If&lt;/code&gt; statement):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dat = Transpose[&#xA;   Table[If[k1 == 1 &amp;amp;&amp;amp; k2 == 1, fArr[[;; , k1, k2]], &#xA;     fArr[[;; , k1, k2]] - &#xA;      k[k1, k2] (k[k1, k2].fArr[[;; , k1, k2]])/(k[k1, k2].k[k1, &#xA;           k2])], {k1, 21}, {k2, 21}], {2, 3, 1}];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I iFFT the two components:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;projArr = InverseFourier /@ dat;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives a purely real array, and I would naively expect the result to be an approximation of $J\nabla\psi$. My question is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In what sense does the result approximate $J\nabla\psi$?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Supposedly the Helmholtz-Hodge decomposition of 2D data is a nontrivial task, as Chris Beaumont's &lt;a href=&quot;https://github.com/ChrisBeaumont/beaumont-idl-library/blob/master/hh_decomp.pro&quot;&gt;HH_DECOMP&lt;/a&gt; routine is supposed to use FFTs to perform Helmholtz-Hodge decomposition, but he also says (in the comments at the top of the code) that the method seems inaccurate. Likewise, there are more complicated variational methods to perform Helmholtz-Hodge decompositions$^3$ of 2D data, which would seem to suggest that the simpler FFT method is somehow inadequate. Why? What does the FFT method get wrong? And is it wrong to assume the harmonic component vanishes for my random noise?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1): &lt;a href=&quot;http://www.autodeskresearch.com/pdf/ns.pdf&quot;&gt;&lt;em&gt;Stable Fluids&lt;/em&gt;&lt;/a&gt;, Jos Stam.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2): &lt;a href=&quot;http://www.informatik.uni-leipzig.de/bsv/PDF_public/Publications/Wiebel/diplomArbeit.pdf&quot;&gt;&lt;em&gt;Feature Detection in Vector Fields Using the Helmholtz-Hodge Decomposition&lt;/em&gt;&lt;/a&gt;, Alexander Wiebel, page 12.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3): &lt;a href=&quot;http://geometry.caltech.edu/pubs/TLHD03.pdf&quot;&gt;&lt;em&gt;Discrete Multiscale Vector Field Decomposition&lt;/em&gt;&lt;/a&gt;, Yiying Tong.&lt;/p&gt;&#xA;" OwnerUserId="6928" LastEditorUserId="6928" LastEditDate="2014-12-02T00:06:18.680" LastActivityDate="2014-12-03T05:49:03.567" Title="Quick and simple discrete 2D Helmholtz-Hodge Decomposition using FFTs?" Tags="|fourier-analysis|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16363" PostTypeId="1" CreationDate="2014-12-02T09:21:33.160" Score="1" ViewCount="344" Body="&lt;p&gt;I have a 2D implementation of smoothed particle hydrodynamics up and running, however when I tried to move it to 3D, using the appropriate 3D kernels, particles always tend to go apart from each other. Since my 2D implementation works fine and the only thing I've changed are the kernels, the only parameter that affect them is the length of the support.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good way to calculate a good value for $h$ rather than just randomly checking?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="549" LastEditDate="2014-12-02T16:03:12.707" LastActivityDate="2017-08-29T11:51:34.170" Title="How to choose $h$ in SPH?" Tags="|sph|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16364" PostTypeId="2" ParentId="16363" CreationDate="2014-12-02T12:57:42.990" Score="2" Body="&lt;p&gt;I'm not an expert in SPH and so others should chime in as well, but here's one consideration: $h$ needs to be chosen so that within a neighborhood of radius $h$ around a point, there are other points. So $h$ is related to the average distance between points. The thing that's different between 2d and 3d is that with the same number of points, their average distance is much larger in 3d. For example, if you scatter $10^6$ points in a 2d domain that is the unit square, then on average they are 0.001 apart ($10^6$ particles in 2d form, if placed purposely, a $1000\times 1000$ lattice) where in 3d they are 0.01, i.e., 10 times farther apart (because in 3d, they form a $100\times 100\times 100$ lattice). You will have to adjust your $h$ correspondingly.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-02T12:57:42.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16365" PostTypeId="2" ParentId="16360" CreationDate="2014-12-02T14:11:53.150" Score="5" Body="&lt;p&gt;The easiest way to generate a divergence-free 2D function is to use a streamfunction, $\psi$. Where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u=\frac{\partial\psi}{\partial y}\quad \rm{and}\quad v=-\frac{\partial \psi}{\partial x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you generate a random $\psi$, you can then use your Fourier representation (or finite differences) to compute the two derivatives. I haven't tried this kind of thing with truly random inputs, though. You might benefit by starting with a smoother but randomized input (like a sum of sine functions with random frequencies and coefficients). &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-12-02T14:11:53.150" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16366" PostTypeId="2" ParentId="16363" CreationDate="2014-12-02T17:07:21.957" Score="1" Body="&lt;p&gt;Two things. Ensure the summation of kernel values which is $\sum (m/\rho) W(r,h)$ is amounting to 1.0 to a desired accuracy. Check if you are using correct value for mass of each particle, if computed from reference density. If you use the same value of h as in 2d, it is supposed to give a stable simulation.&lt;/p&gt;&#xA;" OwnerUserId="11965" LastActivityDate="2014-12-02T17:07:21.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16367" PostTypeId="1" AcceptedAnswerId="16372" CreationDate="2014-12-02T17:16:27.053" Score="0" ViewCount="89" Body="&lt;p&gt;Imagine you have a  piece of wood and from that piece you want to get the largest cylinder possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do you determine the position and orientation of the cylinders axis, to maximize its radius?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been looking at this problem for over a year now and I can't find an efficient (or rather &lt;em&gt;any&lt;/em&gt;) algorithm that can handle 200000 vertices in a second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current idea is a simple:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;slicing up the piece, &lt;/li&gt;&#xA;&lt;li&gt;finding the centroid of the slice &lt;/li&gt;&#xA;&lt;li&gt;and using a best fit line through the slices' centroids&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;...but that's rather considering weight distribution of the polyhedron than in-cylinder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been looking at voronoi diagrams and medial axis, as well as straight skeletons. With these I could possibly get something from which I could generate an axis, but I can't quite wrap my head around it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thankful for any help or hint or full description of an algorithm.&lt;/p&gt;&#xA;" OwnerUserId="11966" LastEditorUserId="11966" LastEditDate="2014-12-03T08:41:29.893" LastActivityDate="2014-12-03T08:41:29.893" Title="Largest Cylinder inside Polyhedron" Tags="|optimization|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="16368" PostTypeId="2" ParentId="16331" CreationDate="2014-12-02T18:15:56.603" Score="3" Body="&lt;p&gt;Boris Grigoryevich wants you not to be able to create residuals with the same functions you used to create the solution.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2014-12-02T18:15:56.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16370" PostTypeId="1" CreationDate="2014-12-02T20:48:02.423" Score="2" ViewCount="516" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to do an update to a &quot;thin&quot; QR decomposition ($A = QR$, where $Q$ is $\mathbf{R}^{m,n}$, the first few columns (up to the matrix rank) of an orthogonal matrix and $R$ is upper triangular and square and $\mathbf{R}^{n,n}$).  The matrix itself is a full column rank &quot;tall&quot; matrix ($m \ge n$), and I'm periodically adding and removing columns from $A$ as part of a larger pivoting optimization algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing the decomposition in-place inside $A$ using Householder transformations, with the Householder vectors stored in the lower triangular portion of the matrix and $R$ stored in the upper triangular portion of $A$.  I need to store the diagonals for $R$ in another extra array.  So I only need $\mathcal{O}(n)$ more memory beyond the space of the matrix being decomposed itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Requirements&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each step of the pivoting algorithm, I need to calculate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R^T(Rx - Q^T f)$$&#xA;$$Q Q^T f $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $Q^T f$ term in both equations can be calculated without needing to store $Q$ directly just by applying each Householder transformation to $f$ as they come in from newly added columns.  That is, at each step $f_{n} := H_n f_{n-1}$ for some Householder transformation $H_n$.  Unfortunately I don't see a way to update $Q Q^T f$ in the same way without explicitly storing $Q$ somewhere, so my guess is I still need to hold on to $Q$. ($Q Q^T f$ is basically the projection of $f$ on to the column space of $A$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Updating&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When deleting a column, all the literature I've found recommends using a series of Givens rotations for the &quot;downstream&quot; columns to push them from Hessenberg form back to triangular form.  However, I don't have anywhere to store these so I can calculate $Q Q^T f$ when I need it.  For the column being removed I need to keep it available to be re-added in the future, so I can't use it for scratch space.  Ideally I'd like to do the decomposition using only $\mathcal{O}(n)$ additional memory beyond the space provided by $A$ in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some other way to push the downstream columns back to triangular format that wouldn't use more memory?  I've tried to think of a way to update the Householder transformations, but I haven't been able to figure out a way that isn't $\mathcal{O}(n^3)$.  I'd like the update itself to be $\mathcal{O}(n^2)$ as it is with the Givens rotations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is there a way of updating $Q Q^T f$ at each stage in the pivoting algorithm when a column is added or removed from $A$ without explicitly storing $Q$?&lt;/p&gt;&#xA;" OwnerUserId="1365" LastActivityDate="2014-12-02T20:48:02.423" Title="In-place QR update: deleting a column" Tags="|linear-algebra|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16371" PostTypeId="1" AcceptedAnswerId="16374" CreationDate="2014-12-02T23:56:50.637" Score="1" ViewCount="896" Body="&lt;p&gt;I have a (cyclic undirected) graph stored in a suitable structure (which depends on the language chosen). Nodes represent places in the real world and edges represent connection between them, so I have a sort of initial guess of the drawing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to draw this graph with some constraints and requirements, and from what I have read (scientific literature and books on graph drawing), currently none of the algorithm specifically addresses that requirements, although Sugiyama's magnetized springs are quite close to what I want to achieve. Simulated Annealing, however, could solve that problem, if a suitable cost function is provided, which is not a simple task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is: could you help me make a program which solve graph drawing problem with the constraints enumerated below?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constraints and requirements&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Edges must be aligned at 45 &lt;em&gt;k&lt;/em&gt; degrees, with &lt;em&gt;k&lt;/em&gt; between 0 and 7 (or at least integer if this requirement is too strong). However, during the execution of the algorithm, I think there's no reason to prohibite violation of that constraint.&lt;/li&gt;&#xA;&lt;li&gt;Since the graph represent physical places, the algorithm &lt;em&gt;must&lt;/em&gt; keep relative positions (e.g. if node A is north-north east of B in the initial guess, A must be north or north east of B, but exceptions can be made to meet the next requirement).&lt;/li&gt;&#xA;&lt;li&gt;Node distance should tend to a &quot;natural&quot; predefined length and alignment (if the graph is an open chain, the algorithm should output equally spaced nodes aligned on a line)&lt;/li&gt;&#xA;&lt;li&gt;Bends &lt;em&gt;should&lt;/em&gt; take place at nodes and the angle between two adjacent edges should be 135 degrees, except for forks.&lt;/li&gt;&#xA;&lt;li&gt;Edge crossings &lt;em&gt;must&lt;/em&gt; be avoided.&lt;/li&gt;&#xA;&lt;li&gt;Labels &lt;em&gt;must not&lt;/em&gt; overlap any part of the drawing and should be positioned by the following specs  (0 degree = horizontal axis, counterclockwise rotation of the text box):&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Orthogonal to the super-edge (aggregation of aligned edges) if this one is not horizontal&lt;/li&gt;&#xA;&lt;li&gt;Bend: prefer [convex side, orthogonal to the longest adjacent super-edge]&lt;/li&gt;&#xA;&lt;li&gt;Horizontal edge&#xA;&lt;ol&gt;&#xA;&lt;li&gt;45 degrees&lt;/li&gt;&#xA;&lt;li&gt;-45 degrees&lt;/li&gt;&#xA;&lt;li&gt;-135 degrees&lt;/li&gt;&#xA;&lt;li&gt;Horizontal below&lt;/li&gt;&#xA;&lt;li&gt;Horizontal above&lt;/li&gt;&#xA;&lt;li&gt;135 degrees&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have not chosen the programming language yet, but I think I'll stick to C/C++, Python, Perl or Lua. The input will be the &quot;object&quot; I talked about in the beginning, and the output will be the same &quot;object&quot; with node positions changed by the optimization algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typical problems will have about 500 nodes, with small areas densely populated and connected and less dense (but not unique) connections between these areas. I'm not interested in efficiency, at least for the moment.&lt;/p&gt;&#xA;" OwnerUserId="11969" LastActivityDate="2014-12-03T05:08:29.867" Title="Graph drawing with constraints" Tags="|constrained-optimization|graph-theory|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16372" PostTypeId="2" ParentId="16367" CreationDate="2014-12-03T01:54:08.570" Score="2" Body="&lt;p&gt;It sounds like you want to solve a variant of the diamond cutting problem; you might try searching for this term and seeing what you come up with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of decision variables should be relatively small -- you need the radius of the cylinder, its center of mass, its height, and the orientation of the axis in 3-D (a vector of unit length, in essence). From there, you should be able to formulate equations that describe the locus of points that make up the cylinder, and all of these should be within your feasible region. If you can describe your feasible region with equations, this information should be enough to formulate your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upon formulating the problem, I believe it will be nonconvex (if memory serves, an example came up in a global optimization class I TAed), in which case, you need to use a global optimization algorithm to solve the resulting optimization problem. BARON is usually the solver of choice, although it is closed-source and only has a GAMS interface, I believe. You could try using Couenne or Bonmin as solvers also, although these codes are not regarded as highly as BARON.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-03T01:54:08.570" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16373" PostTypeId="1" AcceptedAnswerId="16379" CreationDate="2014-12-03T03:43:24.110" Score="1" ViewCount="403" Body="&lt;p&gt;I'm trying to solve the following non-linear diffusion equation:&#xA;$$&#xA;\frac{\partial}{\partial t} u(x,t)= \frac{\partial^{2}}{\partial x^{2}}u(x,t)^{3}$$ $$ -1\leq x \leq1, t \geq 0&#xA;$$ with the boundary conditions: $$u(1,t)=-u(-1,t)=1$$ and the initial conditions: $$u(x,0)=x$$ I'm trying to implement a Crank-Nicolson scheme but I'm a little stuck. Here's what I have so far: &#xA;$$\frac{u_{i}^{n+1} - u_{i}^{n}}{\Delta t} = \frac{(u^{3})_{i-1}^{n+1} - 2(u^{3})_{i}^{n+1} + (u^{3})_{i+1}^{n+1} + (u^{3})_{i-1}^{n} - 2(u^{3})_{i}^{n} + (u^{3})_{i+1}^{n}}{2\Delta x^2}$$ I'm not sure how to evaluate $$(u^3)_i^{n+1}$$ How should I proceed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Forgot to mention that the analytical solution is: $$u(x,\infty) = x^{\frac{1}{3}}$$&lt;/p&gt;&#xA;" OwnerUserId="11974" LastEditorUserId="11974" LastEditDate="2014-12-03T03:50:42.160" LastActivityDate="2014-12-03T13:24:50.437" Title="Numerical Solution of non-linear diffusion equation using Finite Differencing" Tags="|pde|finite-difference|numerical-analysis|diffusion|crank-nicolson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16374" PostTypeId="2" ParentId="16371" CreationDate="2014-12-03T05:08:29.867" Score="1" Body="&lt;p&gt;It sounds like you are trying to draw a subway map. Sure enough, there are several papers on &lt;a href=&quot;http://scholar.google.com/scholar?q=metro+OR+subway+graph+drawing&quot; rel=&quot;nofollow&quot;&gt;metro/subway graph drawing&lt;/a&gt; in the literature. You could start with Alexander Wolff's &quot;&lt;a href=&quot;http://dx.doi.org/10.1007/s00450-007-0036-y&quot; rel=&quot;nofollow&quot;&gt;Drawing Subway Maps: A Survey&lt;/a&gt;&quot; (2007), and then take a look at the more recent &quot;&lt;a href=&quot;http://dx.doi.org/10.1109/TVCG.2010.81&quot; rel=&quot;nofollow&quot;&gt;Drawing and Labeling High-Quality Metro Maps by Mixed-Integer Programming&lt;/a&gt;&quot; by Nöllenburg and Wolff (2011).&lt;/p&gt;&#xA;" OwnerDisplayName="user3883" LastActivityDate="2014-12-03T05:08:29.867" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16376" PostTypeId="2" ParentId="16360" CreationDate="2014-12-03T05:49:03.567" Score="4" Body="&lt;p&gt;You asked several questions in your post, so while Bill has provided a solution to the underlying problem, I feel someone should say some things about your stated questions too.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In what sense does the result approximate $J\nabla\psi$?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you interpret the input array as the alias-free sampling of a periodic, bandlimited vector field $\mathbf f=\mathbf h+\nabla\phi+J\nabla\psi$, then the result is precisely the sampling of $\mathbf h+J\nabla\psi$. After all, the FFT of the sampled array is essentially the Fourier series of the underlying periodic function (assuming sufficient sampling).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that since $\mathbf f$ is periodic, $\mathbf h$ is merely a constant vector field.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Chris Beaumont's HH_DECOMP routine is supposed to use FFTs to perform Helmholtz-Hodge decomposition, but he also says (in the comments at the top of the code) that the method seems inaccurate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what's going on there because I can't read IDL, but it looks like the test vector fields aren't periodic?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Likewise, there are more complicated variational methods to perform Helmholtz-Hodge decompositions of 2D data, which would seem to suggest that the simpler FFT method is somehow inadequate. Why?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think you're worrying too much. On the second page of the paper you cite, they state, &quot;Discrete Helmholtz-Hodge decomposition on regular grids has already been used in graphics (see [25, 10] for instance) and is relatively straightforward to implement with a finite-difference approach. It is, however, much harder to design a practical and accurate method for arbitrary grids.&quot;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What does the FFT method get wrong?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you have periodic boundary conditions, nothing. In fact, under those conditions, &lt;a href=&quot;http://en.wikipedia.org/wiki/Spectral_method&quot; rel=&quot;nofollow&quot;&gt;spectral methods&lt;/a&gt; like this one give you exponential convergence as compared to the polynomial convergence of finite difference methods. They're harder to apply to arbitrary geometries, though.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;And is it wrong to assume the harmonic component vanishes for my random noise?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Technically, yes, but in the periodic case the only possible harmonic vector field is a constant. So you're doing fine by leaving the DC component alone.&lt;/p&gt;&#xA;" OwnerDisplayName="user3883" LastActivityDate="2014-12-03T05:49:03.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16377" PostTypeId="1" AcceptedAnswerId="16386" CreationDate="2014-12-03T07:53:43.980" Score="1" ViewCount="863" Body="&lt;p&gt;I have two sets of 300 VTK files and I want to get a graph comparing the average difference of the two sets. All files have a structured grid containing a scalar value named phi. At the end, I need a graph showing the average difference of each grid for every grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have computed the difference at each cell using a Programmable Filter with the following code, from a previous question(&lt;a href=&quot;https://scicomp.stackexchange.com/questions/8989/how-do-i-calculate-the-numerical-difference-between-two-fields-stored-in-two-dif&quot;&gt;here&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;phi0 = inputs[0].PointData['phi']&#xA;phi1 = inputs[1].PointData['phi']&#xA;output.PointData.append(phi1-phi0, 'difference')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I do not understand how can I compute the average value of the whole grid(sum all differences and divide by the number of cells) and then plot that for every grid I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="11978" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-12-03T22:26:40.687" Title="Visualizing the difference of two sets of VTK files" Tags="|visualization|paraview|vtk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16379" PostTypeId="2" ParentId="16373" CreationDate="2014-12-03T13:24:50.437" Score="3" Body="&lt;p&gt;You should rearrange the terms so that all of the $n+1$ terms are together on one side of the equals sign and all of the $n$ terms are on them other. Then you will have a system of non-linear equations like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A(u^{n+1}_i) u^{n+1}_i=b&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $A$ is a non-linear matrix. Then you can use something like Newton-Raphson to linearize the system and your favorite linear solver to solve the resulting linear systems at each Newton step.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-12-03T13:24:50.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16381" PostTypeId="1" CreationDate="2014-12-03T19:08:28.067" Score="3" ViewCount="1508" Body="&lt;p&gt;Typically, the function that one wants to integrate numerically, $f$, is given, i.e. its values for various points $\{x_i\}$ are known precisely. The resulting error is due to the fact that we chose a finite number of points. Let's call this error $numerical$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a different situation. I know the values of $f$, $\{f(x_i)\}_{i=1}^N$, but with corresponding variances, $\{var(f(x_i))\}$. If I use, for example, the trapezoidal rule:&#xA;$$F(b)-F(a)=\int_a^b f(x) dx \approx \frac{\Delta x}{2} [f(x_1) + 2 f(x_2) + \ldots 2f(x_{N-1}) + f(x_N) ] $$&#xA;(assuming that $x_i$ are equally separated by $\Delta x$),&#xA;I calculate the variance of this estimate as:&#xA;$$\left(\frac{\Delta x}{2}\right)^2 \left[ var(f(x_1)) + 4\cdot var(f(x_2)) + \ldots + 4\cdot var(f(x_{N-1})) + var(f(x_N)) \right] $$&#xA;but what about the $numerical$ error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I combine these two errors?&#xA;What is the &quot;final&quot; error of my estimate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've calculated the variance the same way I would do it for a sum of $N$ independent random variables, $X_i$, with coefficients $a_i$. That is, if:&#xA;$$&#xA;S :=\sum_{i=1}^{N}a_i X_i,\qquad \forall_{i\neq j}Cov(X_i,X_j)=0&#xA;$$&#xA;then&#xA;$$&#xA;Var(S)=\sum_{i=1}^{N} a_i^2 Var(X_i).&#xA;$$&#xA;If this error analysis is not correct, please let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I noticed a problem with the variance of partial sums, $S_k:=\sum_{i=1}^k a_i X_i$. It seems that the error propagates, and therefore $S_N$ has the largest variance, while $S_2$ -- the lowest. But if I were to &lt;em&gt;reverse&lt;/em&gt; the order in the sum (which would correspond to estimating a reversed integral, $\int_b^a f dx$), the error would again propagate, but in the opposite direction. I would get two different estimates of &quot;partial integrals&quot;, depending on the direction of integration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way of combining information from numerical integration in both ways?&lt;/p&gt;&#xA;" OwnerUserId="10052" LastEditorUserId="10052" LastEditDate="2014-12-04T08:03:01.407" LastActivityDate="2014-12-04T08:03:01.407" Title="How to integrate numerically a function with error bars?" Tags="|quadrature|stochastic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16382" PostTypeId="1" AcceptedAnswerId="16387" CreationDate="2014-12-03T19:40:59.887" Score="0" ViewCount="195" Body="&lt;p&gt;This question is a follow up of &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16213/how-can-i-call-the-boost-c-odeint-runge-kutta-integrator-for-a-system-of-odes&quot;&gt;another one&lt;/a&gt; I have asked a while ago. I have successfully implemented my problem using &lt;em&gt;odeint&lt;/em&gt; library and I get the results I expect. &lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However &lt;strong&gt;I would like to give an estimation of the error for the results&lt;/strong&gt;, obtained by Runge-Kutta method of 4th order. In addition to results obtained by the RK4 method I also have an analytical method to provide solutions.&lt;p&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen &lt;a href=&quot;https://scicomp.stackexchange.com/questions/15900/how-to-estimate-the-local-error-and-the-global-error-for-runge-kutta-method/15906#15906&quot;&gt;another similar question&lt;/a&gt; before asking this one. But the treaty of errors in that question is rather rigorous and I don't have expertise in numerical methods. So for me a simple expression based on theory would work fine, and estimation doesn't have to be a very good one. &lt;p&gt;&lt;/p&gt;&#xA;" OwnerUserId="10011" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-12-04T00:46:42.117" Title="How to give a simple estimation of errors for results obtained from 4th order Runge-Kutta" Tags="|c++|error-estimation|runge-kutta|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16383" PostTypeId="1" CreationDate="2014-12-03T20:37:23.847" Score="1" ViewCount="471" Body="&lt;p&gt;So far the closest I've found is ViennaCL, which has a Lanczos implementation for Eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not clear that EigenVectors are produced by this library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone here know whether ViennaCL solves for EigenVectors? And how to get them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or of any other sparse matrix EigenVector solver for GPGPU?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ETA:From what I have read, Lanczos is only an algorithm for converting from a general matrix to tridiagonal form. The QR algorithm (or other) is needed for extracting EigenValues/Vectors from the tridiagonal matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I can upgrade the ViennaCL library to handle complex matricies, does anyone have a recommendation for a GPU based QR algorithm implementation?&lt;/p&gt;&#xA;" OwnerUserId="11984" LastEditorUserId="11984" LastEditDate="2014-12-04T20:15:21.037" LastActivityDate="2014-12-04T20:15:21.037" Title="I am looking for a complex sparse matrix EigenVector solver for GPGPU; preferably CUDA" Tags="|eigensystem|eigenvalues|gpu|viennacl|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="16384" PostTypeId="2" ParentId="16381" CreationDate="2014-12-03T21:13:01.443" Score="1" Body="&lt;p&gt;Assuming the integration is deterministic (that is, ignoring effects like bit flips and hardware issues that are generally negligible at scale), you have two types of uncertainties:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;the uncertainty in calculating the integral approximately with numerical methods instead of exactly; this error is epistemic, and with enough analysis, could be calculated or estimated&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the uncertainty in each of the function values sampled; unless there's additional relevant information, this uncertainty appears to be aleatoric&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since you have variances, one approach to quantifying the aleatoric uncertainty that is compatible with the information you've provided is polynomial chaos. You could integrate the mean and variance separately to obtain the mean and variance of the integral. This approach is different than the one you propose above; how you arrived at the formula above for the variance is unclear to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach would use interval arithmetic, but this approach is better at capturing worst-case upper and lower limits, and it's not clear that such information would be valuable to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the remaining epistemic uncertainty, you could ignore it, if it's sufficiently small relative to the aleatoric uncertainty in the integral (and in your data points). If you feel you do need to quantify it, you'd need to estimate or bound the error somehow (estimates via embedded methods, arbitrary precision integration, or bounds from numerical analysis textbooks/research). As a first approximation, you could model the epistemic error in the integral of the mean as a uniform random variable at each point in space, calculate the resulting variance, and combine that variance information with the integral of the variance calculated from the aleatoric uncertainties, using a polynomial chaos approach. The error in integrating the variances could then be incorporated into calculations of higher-order moments of the random variable describing the integral you wish to calculate. Although this process could be continued with progressively higher-order moments, at some point, the moment expansion of this variable will need to be truncated in calculations.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-03T21:13:01.443" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16385" PostTypeId="1" AcceptedAnswerId="16394" CreationDate="2014-12-03T21:50:26.780" Score="1" ViewCount="78" Body="&lt;p&gt;I'm using DGGEVX routine from LAPACKE with BALANC option as shown below, but to my surprise changing BALANC option from &lt;code&gt;'N'&lt;/code&gt; to &lt;code&gt;'P'&lt;/code&gt; or &lt;code&gt;'S'&lt;/code&gt; or &lt;code&gt;'B'&lt;/code&gt; has no effect on the output Eigen values &amp;amp; vectors. Does anybody know why?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;info=LAPACKE_dggevx(LAPACK_COL_MAJOR,'B','V','V','N',n,a,n,b,n,alphar,alphai,beta,vl,n,vr,n,&amp;amp;ilo,&amp;amp;ihi,lscale,rscale,&amp;amp;abnrm,&amp;amp;bbnrm,rconde,rcondv);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/O1oUz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11726" LastActivityDate="2014-12-04T20:58:01.713" Title="LAPACK DGGEVX: BALANC option" Tags="|c++|eigensystem|eigenvalues|lapack|eigen|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16386" PostTypeId="2" ParentId="16377" CreationDate="2014-12-03T22:26:40.687" Score="1" Body="&lt;p&gt;The answer to your question is that you can change your Programmable Filter output type to a vtkPolyData and set the Script to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;phi0 = inputs[0].PointData['phi']&#xA;phi1 = inputs[1].PointData['phi']&#xA;dd = abs(phi1-phi0) #note absolute value here&#xA;ss = sum(dd)&#xA;&#xA;points = vtk.vtkPoints()&#xA;points.InsertPoint(0, [0, 0, 0])&#xA;output.SetPoints(points)&#xA;globalsum = vtk.vtkDoubleArray()&#xA;globalsum.SetName(&quot;sum&quot;)&#xA;globalsum.InsertValue(0, ss/inputs[0].GetNumberOfPoints())&#xA;output.GetPointData().AddArray(globalsum)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This probably won't work in parallel though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to consider a mesh size independent option though. For this first use one of the calculator filters to create a point data array with value of 1 everywhere and then use the Integrate Variables filter in ParaView to integrate the difference and &quot;1&quot; array result over the entire data set of difference. Then divide the difference result by the 1 array result to get a mesh size independent value. &lt;/p&gt;&#xA;" OwnerUserId="3469" LastActivityDate="2014-12-03T22:26:40.687" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16387" PostTypeId="2" ParentId="16382" CreationDate="2014-12-04T00:46:42.117" Score="0" Body="&lt;p&gt;The theoretical error estimates tend to be worst-case bounds. Pathological examples can be constructed that attain these worst-case bounds; however, other examples can also be constructed where the error bounds overestimate the error by forty orders of magnitude. Even for formulas that are simple, obtaining or estimating the quantities required for these formulas aren't simple, and the answer that I gave in the question you linked to still applies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention being able to calculate a solution via an analytical method. Assuming that the analytical method calculates the exact solution of the problem you are trying to solve numerically, then yes, taking the difference of the two solutions calculates the error in your RK4 solution. However, if your analytical solution calculates an approximate solution, then the difference of that and the RK4 solution will yield an approximate error, and the quality of this approximation depends on the accuracy of the approximate solution calculated by your analytical method.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-04T00:46:42.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16389" PostTypeId="2" ParentId="2656" CreationDate="2014-12-04T06:25:06.127" Score="2" Body="&lt;p&gt;ND4J.org - it has tools to manipulate n-dimensional arrays on a distributed platform that uses native or GPUs. &lt;a href=&quot;http://nd4j.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nd4j.org/&lt;/a&gt;, &lt;a href=&quot;https://github.com/deeplearning4j/nd4j&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/deeplearning4j/nd4j&lt;/a&gt; [Edit: ND4J, Deeplearning4j and other associated libraries were contributed to the Eclipse Foundation in October 2017. &lt;a href=&quot;https://projects.eclipse.org/proposals/eclipse-deeplearning4j]&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://projects.eclipse.org/proposals/eclipse-deeplearning4j]&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11989" LastEditorUserId="11989" LastEditDate="2017-11-21T01:37:01.207" LastActivityDate="2017-11-21T01:37:01.207" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16390" PostTypeId="1" CreationDate="2014-12-04T12:38:40.143" Score="1" ViewCount="914" Body="&lt;p&gt;I would like to solve numerically the diffusion equation, where the sink term depends linearly on the field, and there is field-independent sink:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u(x)}{\partial x^2}  =f(x)u(x) - \kappa(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I am solving the above using a finite difference scheme, where I modify the standard laplacian matrix, to encompass the $f(x)u(x)$ term in the diagonal components. I then solve a linear system of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax = b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where b is a vector composed of the discretised $\kappa(x)$ term, and $A$ is the standard finite difference laplacian (forgetting about boundary conditions), with modified diagonal components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above works fine, but the issue is that I am solving it for the quasi-steady-state at each time step (basically $f(x)$ and $\kappa(x)$ change over time). Since I am required to modify $A$ according to the linear source term at each time step, this means I am required to solve the system afresh each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would prefer is a method where I modify $b$ each time step, and keep $A$ constant. This would allow me to take advantage of $A$'s constancy, and do the bulk of solving once, and then just apply it at each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried a scheme whereby I use the current time step's value of $u_i$ on the RHS opposed to the future value, leading to a scheme of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{u^{t+1}_{i+1} + u^{t+1}_{i-1} - 2u^{t+1}_i}{2h} = f_i u^t_i - \kappa_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this does not seem to work, and leads to instability/non-convergent behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of any schemes which would allow me to keep $A$ constant, and only modify $b$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, more generally can anyone tell me if there are other methods which might be better than finite difference in this case? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ben&lt;/p&gt;&#xA;" OwnerUserId="11972" LastEditorUserId="11972" LastEditDate="2014-12-04T14:14:20.373" LastActivityDate="2014-12-08T08:25:16.667" Title="Solve diffusion equation with linear source term" Tags="|finite-element|finite-difference|linear-solver|advection-diffusion|spectral-method|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16392" PostTypeId="2" ParentId="16390" CreationDate="2014-12-04T15:41:33.640" Score="1" Body="&lt;p&gt;You can't keep $A$ constant, but you could split it into two parts $A(t)=K+C(t)$ for the diffusion and non-constant &quot;reaction&quot; term and then use the $LU$-factorization of $K$ as a preconditioner for an iterative method for the overall system at future times. If $f(x,t)$ is reasonably nice, that should work well. Also, since there's no time-derivative in the PDE, each time level is independent, and you can solve many of them in parallel.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-12-04T15:41:33.640" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16393" PostTypeId="1" CreationDate="2014-12-04T17:36:28.620" Score="-1" ViewCount="163" Body="&lt;p&gt;I want to learn a numerical method for PDEs other than finite difference method. After some research on internet i have found Rothe method and it looks interesting to me. Unfortunately, i couldn't find a good book/ notes for starting. Can you suggest me a book to study? What is prerequisites/corequisites for the Rothe method? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance... &lt;/p&gt;&#xA;" OwnerUserId="10297" LastActivityDate="2014-12-05T02:48:48.293" Title="Can you give some information for rothe method" Tags="|pde|finite-difference|numerical-analysis|reference-request|method-of-lines|" AnswerCount="1" CommentCount="6" ClosedDate="2014-12-05T19:12:41.167" ContentLicense="CC BY-SA 3.0" />
  <row Id="16394" PostTypeId="2" ParentId="16385" CreationDate="2014-12-04T20:58:01.713" Score="2" Body="&lt;p&gt;Based on your comments, the most likely explanation is as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://phase.hpcc.jp/mirrors/netlib/lapack/double/dggevx.f&quot; rel=&quot;nofollow&quot;&gt;dggevx.f&lt;/a&gt;, there is the following paragraph:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;*  Optionally also, it computes a balancing transformation to improve&#xA;*  the conditioning of the eigenvalues and eigenvectors (ILO, IHI,&#xA;*  LSCALE, RSCALE, ABNRM, and BBNRM), reciprocal condition numbers for&#xA;*  the eigenvalues (RCONDE), and reciprocal condition numbers for the&#xA;*  right eigenvectors (RCONDV).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Such balancing transformations are aimed at improving the conditioning of the computation. If successful, this makes the results more accurate (not &lt;em&gt;completely&lt;/em&gt; accurate, though).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, since you see that some but not all of the outputs are different, and that balancing is actually performed, the most likely interpretation is that  the results without balancing were already pretty accurate, so improving the conditioning of the system did not make them much more accurate than that. The outputs that did change are more accurate now than they used to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check whether this interpretation is right, you'd need to say more about your matrices, their condition numbers, etc. Many of such special options are aimed at calculations involving poorly conditioned matrices, for which they can make a difference. If your matrices have small condition numbers, this won't have much of an effect.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-12-04T20:58:01.713" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16396" PostTypeId="1" CreationDate="2014-12-04T21:34:39.393" Score="0" ViewCount="836" Body="&lt;p&gt;I have seen videos like &lt;a href=&quot;http://www.subblue.com/blog/2011/1/16/surface_detail&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; before in the past of things like &quot;3D Mandlebulbs&quot; and similar fractal sets, but can anyone tell me what sorts of programs are actually used create these visualizations? How computationally expensive are these visualizations to make? Also, what sort of simulations are these? I mean, what sort of coordinate system, or mesh, are these sorts of structures realized on? I have most of my experience with computational simulations in FORTRAN, Python, and Matlab, and I just don't see how you would even begin to undertake creating the data to produce these sorts of visualizations in those languages. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="11799" LastEditorUserId="11799" LastEditDate="2014-12-04T21:42:48.820" LastActivityDate="2014-12-06T12:54:29.033" Title="How-to: Epic visualizations of 3D fractals?" Tags="|visualization|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16397" PostTypeId="1" AcceptedAnswerId="16403" CreationDate="2014-12-04T22:55:23.820" Score="3" ViewCount="300" Body="&lt;p&gt;For non-overlapping domain decomposition methods for elliptic problems there is an associated Steklov-Poincaré definite positive operator defined on the interface, allowing a direct computation of the interface solution from the boundary data. I have never heard about an equivalent kind of operator for overlapping methods, though I expect it is probably possible that it exists. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give me some pointers on materials about that ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My ambition is to have a unified view between overlapping and non-overlapping methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much.&#xA;Tom&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2014-12-05T11:16:24.273" Title="Steklov-Poincaré operator for overlapping domain decomposition" Tags="|domain-decomposition|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16398" PostTypeId="2" ParentId="16397" CreationDate="2014-12-04T23:45:53.527" Score="1" Body="&lt;p&gt;I'm not sure you need a Steklov-Poincare operator for overlapping domain decomposition methods. The Dirichlet-to-Neumann map (i.e. DtN or Steklov-Poincare operator) is useful in non-overlapping domain-decomposition because Neumann (i.e. flux) data is used in the transmission conditions between subdomains. A discrete DtN map just does this by solving the Dirichlet problem and computing the Neumann data from the solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Contrast this to, for example, additive Schwarz DD, where the solutions are simply added together at each iteration (communicating information through solution values and the overlap region). Similarly, for multiplicative Schwarz DD, the boundary condition on one domain is initialized with the value of the solution on the other domain.  Neither one necessarily uses Neumann data at all. (I suppose the equivalent to the DtN map would just be the evaluation of the solution at the boundaries of the overlap region?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: there is some overlap, in that DtN maps have been used to &lt;a href=&quot;https://hal.archives-ouvertes.fr/hal-00586246/document&quot; rel=&quot;nofollow&quot;&gt;define coarse spaces&lt;/a&gt; for DD methods. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2014-12-05T00:35:34.823" LastActivityDate="2014-12-05T00:35:34.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16399" PostTypeId="2" ParentId="16396" CreationDate="2014-12-05T00:25:56.583" Score="0" Body="&lt;p&gt;There is no reason at all for the fractals to be defined by iterative algorithms: fractals are carefully defined mathematical objects. For example, the Mandelbrot set is defined as the &lt;em&gt;set of points&lt;/em&gt;&#xA;$$ M= \{z \in\mathbb{C} \mid \text{the iteration }f_z(f_z(\cdots(f_z(0))\cdots) \text{ does not diverge}\}, \qquad f_z(x) = x^2+z. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have this set of points (and it &lt;em&gt;really&lt;/em&gt; doesn't matter how you define it, except that the final images will be more or less interesting), the next task is to render it. Drawing an object means that you have a rectangular array of pixels in an image, and to each pixel you need to assign a colour; the colour is then chosen based on the set of points you want to draw. The simplest method is to map the centre of each pixel $p$ to a point $z_p$ in the complex plane, and draw it with one colour if $z_p\in M$, and another if $z_p\notin M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 3d the procedure is roughly the same, only the rendering techniques are more complicated. For drawing objects like fractals ray-tracing is often used, using software like PBRT or POV-Ray.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you have a number of misconceptions in your question, but I can't guess what it is that you are thinking. The important thing is to be clear about what mathematical objects you want to draw (a set of points, a set of line segments like in a finite approximation of a Koch snowflake, etc.), and how you produce an image from it (how you map pixels to the object you are drawing, how you choose colours for each pixel, etc.).&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-12-05T00:25:56.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16400" PostTypeId="1" CreationDate="2014-12-05T02:20:47.533" Score="2" ViewCount="349" Body="&lt;p&gt;I'm playing with ARPACK. I looked into the examples they provide, zndrv4.f illustrating the usage of the routine znaupd, in the directory of ARPACK/EXAMPLES/COMPLEX/. I also came cross NAG Fortran Library. In NAG, there are some linear problem solvers F12***. The F12*** routines in NAG are equivalent to the znaupd in ARPACK. So I want to check if they will yield the same results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first looked into the example provided in the user guide of F12ARF at &lt;a href=&quot;http://www.nag.co.uk/numeric/fl/nagdoc_fl22/pdf/F12/f12arf.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.nag.co.uk/numeric/fl/nagdoc_fl22/pdf/F12/f12arf.pdf&lt;/a&gt; for example. In the end, it yields the results of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;509.9390&#xA;380.9092&#xA;659.1558&#xA;271.9412&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;around the shift=500. I solved the same generalized eigenvalue problem in Matlab. Matlab gave me the same results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I used znaupd from ARPACK to solve the same problem, I obtained different answers. The 4 eigenvalues are now&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigenvalue               Residual&#xA;501.65650188259684       2.01604D-01&#xA;480.15153312181440       2.10853D-01 &#xA;526.52596256924164       1.91850D-01&#xA;461.99019999608828       2.19336D-01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The routines in NAG and ARPACK both use SHIFTED INVERSE mode and solve the same generalized problem. &#xA;I'm not sure what was wrong, but you can see the residual is too big (usually it should be around 1e-15). &#xA;I attached my scripts of the ARPACK zndrv4.f (it's basically the same as the example file provided by ARPACK, I just need to change a little bit the matrices around line 174 to be the same as that in NAG.) and the Matlab file zndrv4.m in the compressed file below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/mcq2jkmrgk2839u/zndrv4.zip?dl=0&quot; rel=&quot;nofollow&quot;&gt;https://www.dropbox.com/s/mcq2jkmrgk2839u/zndrv4.zip?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then later I found that actually the M matrix in F12ARF is normalized (entries divided by a six). I followed this and got the above wrong results in ARPACK. Now if I didn't divide the entries by six in zndrv4.f, the ARPACK script gives me the correct answer (the same as Matlab). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigenvalue               Residual&#xA;4.90161D+02              5.20906D-16&#xA;5.50553D+02              8.16401D-16&#xA;4.33527D+02              2.29794D-15&#xA;6.14759D+02              4.93630D-15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm confused by this. It seems to say that the ARPACK routine is not robust. Just scaling the matrix with a constant 6 will lead to such a big change in residual? Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="11995" LastActivityDate="2014-12-05T02:20:47.533" Title="ARPACK gives different answers from Matlab and NAG" Tags="|matlab|eigenvalues|arpack|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16401" PostTypeId="2" ParentId="16393" CreationDate="2014-12-05T02:48:48.293" Score="1" Body="&lt;p&gt;Here's a 30 minute introduction that explains the difference between the method of lines and the Rothe method: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.676.26.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.676.26.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-05T02:48:48.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16403" PostTypeId="2" ParentId="16397" CreationDate="2014-12-05T10:52:39.397" Score="4" Body="&lt;p&gt;The classification of &lt;a href=&quot;http://www.ddm.org&quot; rel=&quot;nofollow&quot;&gt;domain decomposition methods&lt;/a&gt; into non-overlapping and overlapping methods can be refined.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The non-overlapping methods have two groups: single trace and multiple trace methods, where `trace' is sometimes the primal variables (aka Dirichlet traces) and sometimes the dual variables (aka Lagrange multipliers) on the interfaces. In the single-trace methods, the adjacent subdomains share a single trace along their common intersections (multiple dual variables are needed at lower dimensional intersections like cross-points in 2-D).  In the multiple-trace methods, each subdomain &lt;strong&gt;never&lt;/strong&gt; equips the same trace with neighbouring subdomains but owns a trace for itself.  In the non-overlapping case, multiple traces must recover the matching conditions: ui = uj, D ui = D uj for second-order PDEs.  For example, just use Dirichlet and Neumann as in the above conditions; or use equivalent linear or non-linear transform of them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The overlapping subdomains have to own a trace for itself and can not share the trace with others because the overlapping subdomains even do not share an interface as their common boundaries. So the overlapping methods are also, in this sense, multiple-trace methods.   We need only one matching condition on each of the two interfaces $\Gamma_{i,j}:=\partial\Omega_i\cap\Omega_j$ and $\Gamma_{j,i}=\partial\Omega_j\cap\Omega_i$ such that the problem defined in the overlap $\Omega_i\cap\Omega_j$ is uniquely solvable with the two type of conditions.  And the matching conditions constitute a problem defined for the traces on the interfaces.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The Steklov-Poincaré operator is a general name referring to the maps between different types of traces.  So, you are right that all the domain decomposition methods can be formed as solving an interface problem, and you can see the Steklov-Poincaré operators in the overlapping case has a little difference to the non-overlapping case: the former is defined between traces on &lt;strong&gt;different&lt;/strong&gt; interfaces.  For example, in the overlapping case, Dirichlet trace $u_i$ on $\Gamma_{i,j}$ can be used to solve out $u_i$ in $\Omega_i$ and then taking a Neumann trace $D u_i$ on $\Gamma_{j,i}\subset\Omega_j$ which should satisfy $D u_i = D u_j$; similarly, we have also an equation for $D u_j$ on $\Gamma_{j,i}$, which involves the following map: solving out $u_j$ in $\Omega_j$ using $D u_j$ and taking trace $u_j$ on $\Gamma_{i,j}$.  The system looks like 2-by-2 blocks&#xA;$$&#xA;\begin{pmatrix}&#xA; I &amp;amp; -S_{i,j}\\&#xA; -S_{j,i} &amp;amp; I&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA; u_i\\ D u_j&#xA;\end{pmatrix}=..&#xA;$$&#xA;This is the same in appearance to the non-overlapping multiple-trace methods. While in the non-overlapping single-trace case, we have a single trace $d = u_i = u_j$ on $\Gamma_{i,j}=\Gamma_{j,i}$ and the interface system looks like &#xA;$$&#xA;(S_{i,j}+S_{j,i})d = ..&#xA;$$&#xA;where $S_{i,j}$ maps $d$ to another type (usually the dual type) trace through $\Omega_i$. &lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2014-12-05T11:16:24.273" LastActivityDate="2014-12-05T11:16:24.273" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16404" PostTypeId="1" CreationDate="2014-12-05T18:13:05.653" Score="0" ViewCount="914" Body="&lt;p&gt;I wrote a simple script to generate random polynoimals $\displaystyle f(z)= \sum_{k=0}^N a_k \frac{z^k}{\sqrt{k!}} $ of high degree and find their roots.  For more discussion on random polyomials see &lt;a href=&quot;http://arxiv.org/abs/math/9501224&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 500&#xA;&#xA;a = np.random.normal(0,1,N)&#xA;&#xA;n = np.arange(N-1)+1&#xA;n = np.insert(n,0,1)&#xA;n = np.sqrt(n)&#xA;n = np.cumproduct(n)&#xA;&#xA;z = np.roots(a/n[::-1])/np.sqrt(N)&#xA;&#xA;plt.xlim([-1,1])&#xA;plt.ylim([-1,1])&#xA;plt.plot(z.real, z.imag, '.')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For smaller values of (here &lt;code&gt;N=150&lt;/code&gt;) we do get a the picture of the unit disk filled with zeros: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/25c2r.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For larger value of &lt;code&gt;N&lt;/code&gt; there is an error message.  I am sure NumPy can compute eigenvalues of larger matrices than this.  Right?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LinAlgError: Eigenvalues did not converge&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="4006" LastEditDate="2014-12-05T19:05:07.587" LastActivityDate="2015-04-04T21:08:34.933" Title="roots of polynomials of high degree: LinAlgError: Eigenvalues did not converge" Tags="|eigensystem|numpy|polynomials|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16405" PostTypeId="1" AcceptedAnswerId="16411" CreationDate="2014-12-05T18:29:23.500" Score="0" ViewCount="117" Body="&lt;p&gt;Lets say I have a function $f(X) = f(x_1,...,x_N)$ to be integrated. But unlike time discrete methods, my integrator uses quantisation to advance time, that is if $|x - q| &amp;gt; dQ$, with $q$ being the quantised version of $x$ and $dQ$ being the quantum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achieve this, the function needs to send $f(X)$, $f'(X)$, and $f''(x)$ to the integrator, which &#xA;determines the time step, $t = \sqrt[3]{( dQ  \frac{3}{f''(x)} )}$, from the 2nd derivative (of the derivative function $f$) and after $t$ units time sends back the updated (integrated) value ($[v,mv,pv]$ ~ value, 1st &amp;amp; 2nd quantized derivative values of $f$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper, $f'$ is approximated by calculating &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c_j = [ f(X) - f(X^*)]/ (x_j - x^*_j)$ for each updated value $x^*_j$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f' = \sum( c_j ) \cdot mv$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in the reference implementation, they update all values, and then simply calculate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f' = f(X) - f(X^*) / 10^{-16} $(and so on for the 2nd derivative) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm unsure on which way would be preferable, the first approach using $2m$ evaluations, the second one just calls $f$ twice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which way would be better for non-linear function approximation?, given that it needs to be accurate only within the quantum bounds (which are generally small,. e.g $0.0001 \cdot |f(X)|$ )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hint: this is a discrete event method, large jumps might be present.&lt;/p&gt;&#xA;" OwnerUserId="12003" LastEditorUserId="7733" LastEditDate="2014-12-08T14:15:25.827" LastActivityDate="2014-12-08T14:15:25.827" Title="Computing multiple numerical derivatives at once" Tags="|finite-difference|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16406" PostTypeId="1" CreationDate="2014-12-05T18:29:40.463" Score="1" ViewCount="68" Body="&lt;p&gt;For $A=LU$, or $A=LDL^T$ factorization, bandwidth is preserved when there is no pivoting. This is true even for indefinite A, see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16310/is-the-bandwidth-of-indefinite-a-equal-to-its-factor-l-in-ldlt&quot;&gt;question&lt;/a&gt;. However, when there is pivoting band structure is destroyed, so fill-ins can occur outside the band. I am curious if there is any pivoting that can still preserve sparsity for the factors within some bounds.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-12-06T14:00:46.280" Title="Is there an upper bound for fill-ins for indefinite triangular factorization?" Tags="|sparse-matrix|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16408" PostTypeId="2" ParentId="16404" CreationDate="2014-12-05T19:03:59.440" Score="1" Body="&lt;p&gt;Using the inversion operation it is possible to circumvent some of the exponential growth issues of the coefficients:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Writing the coefficients backwards &lt;code&gt;n[::1]&lt;/code&gt; instead of &lt;code&gt;n[::-1]&lt;/code&gt; we get the reciprocals of the roots.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 500&#xA;&#xA;a = np.random.normal(0,1,N)&#xA;&#xA;n = (1.0 + np.arange(N-1))/N&#xA;n = np.insert(n,0,1)&#xA;&#xA;n = np.sqrt(n)&#xA;n = 1.0/n&#xA;n = np.cumproduct(n)&#xA;&#xA;z = np.roots(a*n[::1])&#xA;z = 1/z&#xA;L = 1.5&#xA;plt.xlim([-L,L])&#xA;plt.ylim([-L,L])&#xA;plt.plot(z.real, z.imag, '.')&#xA;&#xA;t = np.arange(0,1,0.01)&#xA;z = 1*np.exp(2j*np.pi*t)&#xA;plt.plot(z.real, z.imag, '-')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/VIQoE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then using the function $\displaystyle z \mapsto \frac{1}{z}$ we can flip the roots back.  Once the degree &lt;code&gt;N&lt;/code&gt; is too large, this trick no longer works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/vwPVQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2014-12-05T19:03:59.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16409" PostTypeId="1" CreationDate="2014-12-05T21:27:37.813" Score="1" ViewCount="59" Body="&lt;p&gt;I am working on generating a (complex) solenoidal vector field $\mathbf{A}$ from a prescribed (complex) vector $\mathbf{a}$ and the gradient of a scalar, $b$, such that&#xA;$$\mathbf{A} = \mathbf{a} + \nabla b.$$&#xA;The condition that I require is then&#xA;$$\nabla\cdot\mathbf{A} = \nabla\cdot\mathbf{a} + \nabla^2 b = 0 \qquad \Longrightarrow \qquad \nabla^2 b = - B,$$&#xA;where $B = \nabla\cdot\mathbf{a}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To complicate this slightly, I want to decompose $\mathbf{A},\mathbf{a}, B$ and $b$ into&#xA;$$\mathbf{A} = \hat{\mathbf{A}} \exp[i(\omega t - \mathbf{k}\cdot\mathbf{x})] = \hat{\mathbf{A}} \exp[i\phi],\\&#xA;\mathbf{a} = \hat{\mathbf{a}} \exp[i\phi],\\&#xA;B = \hat{B} \exp[i\phi],\\&#xA;b = \hat{b} \exp[i\phi].$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this decomposition, $B = (\nabla\cdot\hat{\mathbf{a}} - i\mathbf{k}\cdot\hat{\mathbf{a}}) \exp[i\phi]$, and the Poisson equation above becomes&#xA;$$(\dagger)\qquad\nabla^2 b = (\nabla^2 \hat{b} - 2i\mathbf{k}\cdot\nabla b - \lvert\mathbf{k}\rvert^2 b) \exp[i\phi] = -\hat{B} \exp[i\phi].$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is then solved for $\hat{b}$ iteratively using the following solver scheme (given for 1-dimension, but it is easily extended to higher dimensions):&#xA;$$(\ddagger)\qquad \hat{b}_j = \frac{\Delta x^2 \hat{B}_j + \hat{b}_{j+1} + \hat{b}_{j-1} - ik\Delta x(\hat{b}_{j+1} - \hat{b}_{j-1})}{2 + k^2 \Delta x^2},$$&#xA;where $j \in \{2,\ldots,N-1\}$ indicates the discrete grid location, the position $x_j = j \Delta x$, and $\Delta x$ is the grid spacing. The boundary $j=1,N$ is fixed at zero. $\hat{B}_j$ is evaluated as $\hat{B}_j = (\hat{a}_{j+1} - \hat{a}_{j-1})/2\Delta x -ik\hat{a}_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numerical solution to $\hat{b}$ produced satisfies $$\nabla^2 \hat{b} - 2i\mathbf{k}\cdot\nabla b - \lvert\mathbf{k}\rvert^2 b = -\hat{B}$$ very well, however if I construct &#xA;$$\hat{A}_j = \hat{a}_j + \frac{\hat{b}_{j+1} - \hat{b}_{j-1}}{2\Delta{x}} - ik\hat{b}_j$$&#xA;and evaluate&#xA;$$\nabla \cdot \mathbf{A} \simeq \Big(\frac{\hat{A}_{j+1} - \hat{A}_{j-1}}{2\Delta x} - ik \hat{A}_j\Big) \exp[i\phi],$$&#xA;then this is not only non-zero but also large. Putting the expression for $\hat{A}_j$ in terms of $\hat{a}$ and $\hat{b}$, one finds that the above equation reads&#xA;$$(*)\qquad \nabla \cdot \mathbf{A} \simeq \Big( \frac{\hat{a}_{j+1} - \hat{a}_{j-1}}{2\Delta x} - ik \hat{a}_j + \frac{\hat{b}_{j+2} - 2\hat{b}_{j} + \hat{b}_{j-2}}{4\Delta x^2} - 2ik\frac{\hat{b}_{j+1} - \hat{b}_{j-1}}{2\Delta x} - k^2\hat{b}_j \Big) \exp[i\phi]\\&#xA;= \Big( \hat{B} + \nabla_2^2\hat{b} - 2ik \frac{\partial\hat{b}}{\partial x} - 4k^2 \hat{b} \Big) \exp[i\phi].$$&#xA;This is just the 1-dimensional version of $(\dagger)$, but with the Laplacian operator evaluated as&#xA;$$\nabla_2^2 \hat{b} = \frac{\hat{b}_{j+2} - 2\hat{b}_j + \hat{b}_{j-2}}{4\Delta x^2},$$ suggesting that it is this difference which is responsible for $\nabla \cdot \mathbf{A} \neq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To overcome this, I attempted to invert ($*$) rather than use ($\ddagger$), this time with boundary conditions $\hat{b}_j = 0$ for $j = 1, 2, N-1, N$, but the (iterative) solution to the Poisson equation diverges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering what is responsible for this? Could my setting the two boundary cells to zero cause such a problem? The solver scheme for point $j$ involves $j+2, j+1, j-1, j-2$. Given the non-solenoidal vector $\mathbf{a}$, is there a better way to find a scalar $b$ such that $\nabla\cdot(\mathbf{a} + \nabla b) = 0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help or suggestions!&lt;/p&gt;&#xA;" OwnerUserId="12001" LastEditorUserId="12001" LastEditDate="2014-12-12T14:06:46.020" LastActivityDate="2014-12-12T14:06:46.020" Title="Finding a scalar field in order to generate a solenoidal vector from a given vector" Tags="|algorithms|finite-difference|poisson|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16411" PostTypeId="2" ParentId="16405" CreationDate="2014-12-05T23:51:17.110" Score="0" Body="&lt;p&gt;It's not entirely clear to me what you're asking. Let me see answer what I think you're asking, and you can tell me if I'm on the right track.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you've got a function $f: \mathbb{R}^{m} \rightarrow \mathbb{R}$, which is to say that it takes $m$ variables $x_{1}, \ldots, x_{m}$ as inputs, and outputs a scalar. Also, suppose you want to evaluate derivatives of $f$ at some point $\bar{\mathbf{x}} = (\bar{x}_{1}, \ldots, \bar{x}_{m})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to evaluate $\frac{\partial{f}}{\partial{x_{i}}}(\bar{\mathbf{x}})$, then you could approximate it by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{f(\bar{\mathbf{x}} + h\mathbf{e}_{i}) - f(\bar{\mathbf{x}})}{h},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{e}_{i}$ is a unit vector of all zeros, except for a 1 in the $i$th position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, if you want to evaluate $\nabla{f}(\bar{\mathbf{x}}) \cdot \mathbf{v}$, you could approximate it by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{f(\bar{\mathbf{x}} + h\mathbf{v}) - f(\bar{\mathbf{x}})}{h}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases, you will want to use a small $h$; how small depends on the application. Good guidelines for choosing $h$ can be found in the review paper &lt;a href=&quot;http://www.cs.odu.edu/~keyes/papers/jfnk.pdf&quot; rel=&quot;nofollow&quot;&gt;Jacobian-free Newton-Krylov methods: a survey of approaches and applications&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite difference approximation does degrade somewhat the accuracy of the derivative, so if your calculations require precise derivative information, you are better off trying other methods (depending on application, these could be forward or adjoint sensitivity methods; automatic differentiation; complex number finite differences; using some combination of pencil, paper, and your favorite computer algebra system) that get you exact (or close to exact) derivative information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying this process twice to get a Hessian only further degrades the quality of the approximation, and requires $m^{2}$ operations, and is generally regarded as too expensive and too inaccurate to be worth doing. Applications that do need Hessian information can sometimes get by with approximation Hessian information obtained from an initial guess plus rank-1 updates to this approximate Hessian using gradient information. One example would be BFGS methods.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-05T23:51:17.110" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16412" PostTypeId="1" AcceptedAnswerId="16413" CreationDate="2014-12-06T08:53:16.780" Score="2" ViewCount="412" Body="&lt;p&gt;Before there was CUDA or OpenCL people were using GPUs for computation. I am trying to find out how they did that -- because I want to press my Rasberry Pi's GPU for computing and it does not seem to have OpenCL support. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for notes on how they did this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done cursory Googling but have come up empty handed -- primarily because I think the search indexes and full of CUDA and OpenCL related material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would appreciate any pointers on how to use GPUs for computation in the absence of CUDA and OpenCL.  &lt;/p&gt;&#xA;" OwnerUserId="10124" LastActivityDate="2014-12-06T18:24:34.757" Title="using GPUs before CUDA and OpenCL" Tags="|gpu|cuda|opencl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16413" PostTypeId="2" ParentId="16412" CreationDate="2014-12-06T09:56:18.307" Score="3" Body="&lt;p&gt;I don't know a definitive source, but have a look at &quot;GPU Gems 2&quot;, which is a book published by NVIDIA about ten years ago and &lt;a href=&quot;http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter31.html&quot; rel=&quot;nofollow&quot;&gt;available online&lt;/a&gt;. While much of it is about computer graphics, it has a number of sections devoted to general purpose computing on GPUs from the time before CUDA and OpenCL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not familiar enough with Raspberry Pi to tell if this book will actually help you very much.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-12-06T09:56:18.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16414" PostTypeId="2" ParentId="16396" CreationDate="2014-12-06T12:42:10.930" Score="1" Body="&lt;p&gt;Recently I happened to wonder somewhat similar questions expect I was, in particular, interested in &lt;em&gt;real time&lt;/em&gt; visualization of three-dimensional fractals. Since other answers consider two-dimensional fractals, I decided to include some knowledge I have acquired on three-dimensional rendering and &lt;em&gt;ray marching&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Definition of a surface and ray tracing&lt;/h2&gt;&#xA;&lt;p&gt;One way to define a surface in three-dimensions is through a function &lt;span class=&quot;math-container&quot;&gt;$f:\mathbb{R}^3\rightarrow\mathbb{R}$&lt;/span&gt; which satisfies&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(\boldsymbol{x})=0,$$&lt;/span&gt;&#xA;for every point &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}\in \mathbb{R}^3$&lt;/span&gt; that belongs to the surface. Thus, the surface &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; can be defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$S=\{\boldsymbol{x} \in \mathbb{R}^3\,|\,f(\boldsymbol{x})=0\}.$$&lt;/span&gt;&#xA;For one to visualize the surface &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, the intersection point &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{p} \in \mathbb{R}^3$&lt;/span&gt; of a ray—starting from the camera position &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{c} \in \mathbb{R}^3$&lt;/span&gt; to some direction &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{d} \in \mathbb{R}^3$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\|\boldsymbol{d}\|=1$&lt;/span&gt;—and the surface must be sought. After these intersection points are known for sufficiently many rays, a visualization can be constructed (some examples follow).&lt;/p&gt;&#xA;&lt;p&gt;One way to find the intersection point &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{p}$&lt;/span&gt; is through analytical calculations: the surface defined by &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is sometimes so simple that this calculation can be done by hand. An example that considers the intersection of a ray and a sphere can be found &lt;a href=&quot;http://en.wikipedia.org/wiki/Ray_tracing_%28graphics%29#Example&quot; rel=&quot;nofollow noreferrer&quot;&gt;from Wikipedia&lt;/a&gt;. This is exactly what the traditional ray tracing is all about.&lt;/p&gt;&#xA;&lt;p&gt;In a more general setting, no analytical relationship exists for finding the intersection point &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{p}$&lt;/span&gt;. In such a case, one must rely to numerical methods.&lt;/p&gt;&#xA;&lt;h2&gt;Numerical methods for finding the intersection point&lt;/h2&gt;&#xA;&lt;p&gt;A proficient applied mathematician would immediately attempt to apply, e.g., Secant method to the problem: find &lt;span class=&quot;math-container&quot;&gt;$t \in \mathbb{R}$&lt;/span&gt; such that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(\boldsymbol{c}+t\boldsymbol{d}) \approx 0.$$&lt;/span&gt;&#xA;There is, however, a caveat. For a general &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; the function&#xA;&lt;span class=&quot;math-container&quot;&gt;$$g(t):=f(\boldsymbol{c}+t\boldsymbol{d})$$&lt;/span&gt;&#xA;might have several roots and in Secant method (or similar) nothing guarantees that a root nearest to the camera is recovered. You might end up drawing the wrong side of the object of interest! Thus, the applied numerical method should be guaranteed to find a root nearest to the camera.&lt;/p&gt;&#xA;&lt;h3&gt;Ray marching and distance fields&lt;/h3&gt;&#xA;&lt;p&gt;A somewhat trivial method to find the nearest (or approximately nearest) root is to fix a sufficiently small constant step size &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; and then find the smallest &lt;span class=&quot;math-container&quot;&gt;$n \in \mathbb{N} \cup \{0\}$&lt;/span&gt; such that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(\boldsymbol{c}+n \Delta t \boldsymbol{d})\approx 0.$$&lt;/span&gt;&#xA;This is known as &lt;em&gt;ray marching&lt;/em&gt; and is indeed a valid method for visualizing the surface &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. The problem is that it's &lt;em&gt;&lt;strong&gt;tremendously slow&lt;/strong&gt;&lt;/em&gt; since one must try every possible &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; starting from zero!&lt;/p&gt;&#xA;&lt;p&gt;Fortunately, there exists a clever trick to make the computation faster by ensuring that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; defines a &lt;em&gt;distance field of&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. This means that at every point &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}$&lt;/span&gt; in the space &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^3$&lt;/span&gt; the value &lt;span class=&quot;math-container&quot;&gt;$f(\boldsymbol{x})$&lt;/span&gt; represents the &lt;em&gt;minimum distance from &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x}$&lt;/span&gt; to the surface &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&lt;/em&gt;. This sounds like a hard restriction to satisfy but for many fractals the distance fields (or approximate distance fields) have already been derived by enthusiasts (see &lt;a href=&quot;http://www.fractalforums.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fractal Forums&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The algorithm goes as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{p}:=\boldsymbol{c}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{p}:=\boldsymbol{p}+f(\boldsymbol{p})\boldsymbol{d}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;If &lt;span class=&quot;math-container&quot;&gt;$f(\boldsymbol{p})$&lt;/span&gt; is larger than some tolerance go back to step 2. Otherwise, break.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Output of the prescribed algorithm is the intersection point &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{p}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Visualizing fractals in practice&lt;/h2&gt;&#xA;&lt;p&gt;If you happen to own a sufficiently fast graphics processing unit and know the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; for some fractal then this method can be implemented in real time using, e.g., OpenGL Shading Language. See the following Shadertoy entry I constructed as an example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/1qDA1.png&quot; alt=&quot;Screenshot of the running shader&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.shadertoy.com/view/MtX3zr&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.shadertoy.com/view/MtX3zr&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Getting the coloring and other visuals right requires of course lots of experience (or trial-and-error) but this is the basic idea for generating such animations. I tried to add some comments to the shader code and attempted to use same symbols as in this post.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-12-06T12:54:29.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16415" PostTypeId="2" ParentId="16406" CreationDate="2014-12-06T14:00:46.280" Score="2" Body="&lt;p&gt;The commonly-used partial pivoting strategy&#xA;(selecting the pivot as the maximum coefficient&#xA;in the current column) can be used for band matrices.&#xA;If the lower bandwidth of the original matrix A is defined&#xA;as kl and the upper bandwidth as ku, the factors L and U&#xA;will have bandwidths kl+1 and ku+kl, respectively.&#xA;This result is shown in section 4.3.3 of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0801854148&quot; rel=&quot;nofollow&quot;&gt;Matrix Computations&lt;/a&gt;, by Golub and&#xA;Van Loan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method is implemented in Lapack routine dgbtrf. The details of&#xA;the implementation are discussed here:&#xA;&lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn21.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lapack/lawnspdf/lawn21.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2014-12-06T14:00:46.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16416" PostTypeId="2" ParentId="16412" CreationDate="2014-12-06T16:29:13.113" Score="3" Body="&lt;p&gt;Since your question seems to be specifically about the Raspberry Pi, I would suggest searching for &quot;Raspberry Pi GPGPU&quot; in addition to the book suggested by @Kirill. GPGPU refers to &lt;em&gt;general purpose computing on graphics processing units&lt;/em&gt; and should get you results specific to using the RPi's GPU for non-graphics tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also have a look at the answers to &lt;a href=&quot;https://raspberrypi.stackexchange.com/questions/41/can-i-use-the-gpu-for-calculations&quot;&gt;this question&lt;/a&gt; on &lt;a href=&quot;https://raspberrypi.stackexchange.com/&quot;&gt;RaspberryPi.Stackexchange&lt;/a&gt; which is basically a duplicate of yours. The conclusion seems to be that OpenCL is not likely to be implemented on the RPi so your best bet is to use OpenGL capabilities to perform your computations via shaders. This is more limited than truly general purpose techniques such as OpenCL, but appears to be the best you can do. That question is a couple years old so you might want to ask over there about any updates since then.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:29.503" LastActivityDate="2014-12-06T18:24:34.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16417" PostTypeId="2" ParentId="14745" CreationDate="2014-12-06T19:06:03.980" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Hausdorff_distance&quot; rel=&quot;nofollow&quot;&gt;Hausdorff distance&lt;/a&gt; is a measure of how different two arbitrary sets in a metric space are, independent of topology. The open-source software &lt;a href=&quot;http://meshlab.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;MeshLab&lt;/a&gt; lets you &lt;a href=&quot;http://meshlabstuff.blogspot.com/2010/01/measuring-difference-between-two-meshes.html&quot; rel=&quot;nofollow&quot;&gt;compute the Hausdorff distance between two meshes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beware that the Hausdorff distance does not account for translation or rotation, so in case the meshes are misaligned you will have to register them first. MeshLab can &lt;a href=&quot;http://www.instructables.com/id/Using-Meshlab-to-Clean-and-Assemble-Laser-Scan-Dat/step3/Glueing-the-Meshes/&quot; rel=&quot;nofollow&quot;&gt;help you do that&lt;/a&gt; too.&lt;/p&gt;&#xA;" OwnerDisplayName="user3883" LastActivityDate="2014-12-06T19:06:03.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16418" PostTypeId="1" CreationDate="2014-12-06T19:47:26.690" Score="4" ViewCount="161" Body="&lt;p&gt;I have very large matrices that I would like to row reduce (I need to keep track of the steps and find a basis of the kernel/image, not just find the rank).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The good news is that I work mod 2 and the matrices are very sparse. The problem is their size, right now I have a beast that is roughly 3 millions x 1 million, but I expect much bigger, probably billions. I might have a thousand of those matrices that I would like to reduce eventually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the linbox project that might be helpful, but before I wanted to know if anybody has a feeling whether or not this is realistic ? I could be using the Grid of my university which has &quot;the combined processing power of 4,568 cores: 1,346 Intel cores, 3,222 AMD cores, with over 13.5TB of RAM and 1.2PB of disk space.&quot;. What do you think ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="12007" LastActivityDate="2014-12-06T19:47:26.690" Title="How big a matrix can we row reduce in reasonable time?" Tags="|matrix|linear-solver|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16420" PostTypeId="2" ParentId="13037" CreationDate="2014-12-07T08:51:47.603" Score="1" Body="&lt;p&gt;I only have experience with scipy.sparse, it is limited in functionality but API is good. You may want to check out &lt;a href=&quot;https://bitbucket.org/petsc/petsc4py&quot; rel=&quot;nofollow&quot;&gt;PETSC for Python&lt;/a&gt; too.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2014-12-07T08:51:47.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16421" PostTypeId="1" CreationDate="2014-12-07T11:20:38.580" Score="0" ViewCount="76" Body="&lt;p&gt;I am writing a molecular dynamics program to create an lattice and populate it with atoms/molecules. These then are given random velocities and the system is initialized. Then throughout time the molecules interact with each other and exert forces on each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to make my program as readable as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My issue is when the code is run, the values of everything I'm interested in, mainly the vectors _tempreture, _pressure, _totalEnergy, _interactionEnergy, _kineticEnergy all seem to converge (after 5 runs) to certain values. These values then do not change throughout the rest of the runs and sometimes will randomly spike, then drop back down. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't figure out the unexpected behavior of my code, and I've been looking at it hours upon hours. I hoping one of you bright guys will be able to help me. The code should produce values which fluctuate, they should be similar values but definitely not the same exact values, which is what I get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code that does most the work is initLatice, this creates a lattice, with random positions, and velocities of the atoms inside. Then simulate is used to calculate interactions, change positions and velocities of all particles, then calculate the energies of that timestep.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//in MD.h&#xA;&#xA;#ifndef MD_H&#xA;#define MD_H&#xA;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;random&amp;gt;&#xA;#include &amp;lt;time.h&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;&#xA;class MD{&#xA;public:&#xA;    MD();&#xA;    ~MD();&#xA;    void initLatice();&#xA;    void simulate(int _runs);&#xA;    void wrtieToFile(std::string fileName);&#xA;    double randomValue(double upper, double lower);&#xA;private:&#xA;    double _nMolecules;&#xA;    double _estKineticEnergy;&#xA;    double _dt;&#xA;    double _density;&#xA;    bool _tempscale;&#xA;    double _energyCorrection;&#xA;    double _pressureCorrection;&#xA;    double _nFCC = 4;&#xA;    double _truncationSeperation;&#xA;    double _sidelength;&#xA;    double _pi = 3.142;&#xA;    std::vector&amp;lt;double&amp;gt; x_data, y_data, z_data;&#xA;    std::vector&amp;lt;double&amp;gt; _vX, _vY, _vZ;&#xA;    std::vector&amp;lt;double&amp;gt; _xPos, _yPos, _zPos;&#xA;    std::vector&amp;lt;double&amp;gt; _totalEnergy, _kineticEnergy, _interactionEnergy, _pressure, _tempreture;&#xA;    std::vector&amp;lt;double&amp;gt; _radialDist, _radialDistCoord;&#xA;};&#xA;&#xA;#endif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in case anyone is wondering the next file doesn't contain the definition for writeToFile, I decided not to paste it on here to save space.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//in MD.cpp&#xA;#include &quot;MD.h&quot;&#xA;&#xA;MD::MD() : _radialDist(201, 0.0), _radialDistCoord(201, 0.0) {&#xA;        _dt = 0.005;&#xA;        _density = 1.0;&#xA;        _tempscale = true;&#xA;        _nMolecules = pow(_nFCC, 3) * 4;&#xA;        x_data = { 0.25, 0.75, 0.75, 0.25 };&#xA;        y_data = { 0.25, 0.75, 0.25, 0.75 };&#xA;        z_data = { 0.25, 0.25, 0.75, 0.75 };&#xA;        _sidelength = pow((_nFCC / _density), 1. / 3.);&#xA;        _truncationSeperation = 2.5;&#xA;        if (_truncationSeperation &amp;gt; _sidelength / 2.0)&#xA;            _truncationSeperation = _sidelength / 2.0;&#xA;        _energyCorrection = (8.0 * _pi * _density)*(1.0 / (9.0 * pow(_truncationSeperation, 9)) - 1 / (3.0 * pow(_truncationSeperation, 3)));&#xA;        _pressureCorrection = (16.0 * _pi * pow(_density, 2))*(2.0 / (9.0 * pow(_truncationSeperation, 9)) - 1 / (3.0 * pow(_truncationSeperation, 3)));&#xA;}&#xA;&#xA;MD::~MD(){}&#xA;&#xA;double MD::randomValue(double upper, double lower)&#xA;{&#xA;    double returnValue;&#xA;    do{&#xA;        returnValue = upper + (rand() / (RAND_MAX / (lower - upper)));&#xA;    } while (returnValue &amp;gt; upper || returnValue &amp;lt; lower);&#xA;    return returnValue;&#xA;}&#xA;&#xA;void MD::initLatice(){&#xA;    int count = 0;&#xA;    double x_init = 0, y_init = 0, z_init = 0;&#xA;    double _tempreture = 1.0;&#xA;    _estKineticEnergy = 0.5 * (3.0 * _nMolecules - 4.0) * _tempreture;&#xA;    srand(time(NULL));&#xA;&#xA;    for (int i = 0; i &amp;lt; 4; i++){&#xA;        for (int a = 1; a &amp;lt;= _nFCC; a++){&#xA;            for (int b = 1; b &amp;lt;= _nFCC; b++){&#xA;                for (int c = 1; c &amp;lt;= _nFCC; c++){&#xA;                    _xPos.push_back(_sidelength*(a - 1 + x_data[i]) / _nFCC);&#xA;                    _vX.push_back(randomValue(0.5, -0.5));&#xA;                    x_init += _vX[count] / _nMolecules;&#xA;&#xA;                    _yPos.push_back(_sidelength*(b - 1 + y_data[i]) / _nFCC);&#xA;                    _vY.push_back(randomValue(0.5, -0.5));&#xA;                    y_init += _vY[count] / _nMolecules;&#xA;&#xA;                    _zPos.push_back(_sidelength*(c - 1 + z_data[i]) / _nFCC);&#xA;                    _vZ.push_back(randomValue(0.5, -0.5));&#xA;                    z_init += _vZ[count] / _nMolecules;&#xA;&#xA;                    count++;&#xA;                }&#xA;            }&#xA;        }&#xA;    }&#xA;&#xA;    double velocityScale = 0, kineticEnergy = 0;&#xA;&#xA;    for (int i = 0; i &amp;lt; _nMolecules; i++){&#xA;        _vX[i] -= x_init;&#xA;        _vY[i] -= y_init;&#xA;        _vZ[i] -= z_init;&#xA;        kineticEnergy += 0.5 * (pow(_vX[i], 2) + pow(_vY[i], 2) + pow(_vZ[i], 2));&#xA;    }&#xA;&#xA;    velocityScale = sqrt(_estKineticEnergy / kineticEnergy);&#xA;&#xA;    for (int i = 0; i &amp;lt; _nMolecules; i++){&#xA;        _vX[i] = _vX[i] * velocityScale;&#xA;        _vY[i] = _vY[i] * velocityScale;&#xA;        _vZ[i] = _vZ[i] * velocityScale;&#xA;    }&#xA;}&#xA;&#xA;void MD::simulate(int _runs){&#xA;    double force = 0;&#xA;    double seperationSquared = 0;&#xA;    double interactionEnergy = 0;&#xA;    double kineticEnergy = 0;&#xA;    double pressure = 0;&#xA;&#xA;    for (int run = 0; run &amp;lt;= _runs; run++){&#xA;        interactionEnergy = 0;&#xA;        pressure = 0;&#xA;        std::vector&amp;lt;double&amp;gt; force_x(256, 0.0);&#xA;        std::vector&amp;lt;double&amp;gt; force_y(256, 0.0);&#xA;        std::vector&amp;lt;double&amp;gt; force_z(256, 0.0);&#xA;&#xA;        for (int i = 0; i &amp;lt; _nMolecules - 1; i++){&#xA;            double xI = _xPos[i], yI = _yPos[i], zI = _zPos[i];&#xA;            for (int j = i + 1; j &amp;lt; _nMolecules; j++){&#xA;                double x = xI - _xPos[j];&#xA;                double y = yI - _yPos[j];&#xA;                double z = zI - _zPos[j];&#xA;&#xA;                if (x &amp;gt; _sidelength / 2.0)&#xA;                    x -= _sidelength;&#xA;                if (y &amp;gt; _sidelength / 2.0)&#xA;                    y -= _sidelength;&#xA;                if (z &amp;gt; _sidelength / 2.0)&#xA;                    z -= _sidelength;&#xA;                if (x &amp;lt; -_sidelength / 2.0)&#xA;                    x += _sidelength;&#xA;                if (y &amp;lt; -_sidelength / 2.0)&#xA;                    y += _sidelength;&#xA;                if (z &amp;lt; -_sidelength / 2.0)&#xA;                    z += _sidelength;&#xA;&#xA;                seperationSquared = pow(x, 2) + pow(y, 2) + pow(z, 2);&#xA;&#xA;                if (seperationSquared &amp;lt;= (pow(_truncationSeperation, 2))){&#xA;                    interactionEnergy += 4. * ((1. / pow(seperationSquared, 6)) - (1. / pow(seperationSquared, 3)));&#xA;                    force = 24. * ((2. / pow(seperationSquared, 7)) - (1. / pow(seperationSquared, 4)));&#xA;                    force_x[i] += x * force;&#xA;                    force_y[i] += y * force;&#xA;                    force_z[i] += z * force;&#xA;                    force_x[j] -= x * force;&#xA;                    force_y[j] -= y * force;&#xA;                    force_z[j] -= z * force;&#xA;                    pressure += force * seperationSquared;&#xA;&#xA;                    int histCounter = ceil(sqrt(seperationSquared) * (200.0/_truncationSeperation));&#xA;                    _radialDist[histCounter] += 1.0;&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        //thermostating&#xA;        double velocityScale = 0;&#xA;&#xA;        if (_tempscale == true){&#xA;            kineticEnergy = 0;&#xA;            for (int i = 0; i &amp;lt; _nMolecules; i++)&#xA;                kineticEnergy += 0.5 * (pow(_vX[i], 2) + pow(_vY[i], 2) + pow(_vZ[i], 2));&#xA;            velocityScale = sqrt(_estKineticEnergy / kineticEnergy);&#xA;        } &#xA;        else { velocityScale = 1.0; }&#xA;&#xA;        kineticEnergy = 0;&#xA;&#xA;        //applying verlets leapfrog algorithm&#xA;        for (int i = 0; i &amp;lt; _nMolecules; i++){&#xA;            _vX[i] = _vX[i] * velocityScale + force_x[i] * _dt;&#xA;            _xPos[i] += _vX[i] * _dt;&#xA;&#xA;            _vY[i] = _vY[i] * velocityScale + force_y[i] * _dt;&#xA;            _yPos[i] += _vY[i] * _dt;&#xA;&#xA;            _vZ[i] = _vZ[i] * velocityScale + force_z[i] * _dt;&#xA;            _zPos[i] += _vZ[i] * _dt;&#xA;&#xA;            kineticEnergy += 0.5 * (pow(_vX[i], 2) + pow(_vY[i], 2) + pow(_vZ[i], 2));&#xA;&#xA;            //check if the particle is still within the box&#xA;            if (_xPos[i] &amp;gt; _sidelength)&#xA;                _xPos[i] -= _sidelength;&#xA;            if (_yPos[i] &amp;gt; _sidelength)&#xA;                _yPos[i] -= _sidelength;&#xA;            if (_zPos[i] &amp;gt; _sidelength)&#xA;                _zPos[i] -= _sidelength;&#xA;            if (_xPos[i] &amp;lt; 0.0)&#xA;                _xPos[i] += _sidelength;&#xA;            if (_yPos[i] &amp;lt; 0.0)&#xA;                _yPos[i] += _sidelength;&#xA;            if (_zPos[i] &amp;lt; 0.0)&#xA;                _zPos[i] += _sidelength;&#xA;        }&#xA;&#xA;        _kineticEnergy.push_back(kineticEnergy / _nMolecules);&#xA;        _interactionEnergy.push_back((interactionEnergy / _nMolecules) + _energyCorrection);&#xA;        _totalEnergy.push_back(_kineticEnergy[run] + _interactionEnergy[run]);&#xA;&#xA;        if (_tempscale == true)&#xA;            _tempreture.push_back(_nMolecules * 2. * _kineticEnergy[run] / (3.0 * _nMolecules - 4.0));&#xA;        else&#xA;            _tempreture.push_back(_nMolecules * 2. * _kineticEnergy[run] / (3.0 * _nMolecules - 3.0));&#xA;&#xA;        _pressure.push_back((2.0 * _kineticEnergy[run] * _nMolecules + pressure) / (3.0*(pow(_sidelength, 3) + _pressureCorrection)));&#xA;    }&#xA;&#xA;    for (int i = 1; i &amp;lt;= 200; i++){&#xA;        double r = i * _truncationSeperation / 200.0;&#xA;        _radialDistCoord[i] = _radialDistCoord[i - 1] + 2.0 * _radialDist[i] * 10 / _runs / _nMolecules;&#xA;        _radialDist[i] = _radialDist[i] / (2.0*_pi*r*r*(_truncationSeperation/200.0)*_density*_runs*_nMolecules);&#xA;    }&#xA;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and finally...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//in main.cpp&#xA;#include &quot;MD.h&quot;&#xA;&#xA;int main(){&#xA;    MD myMD;&#xA;    myMD.initLatice();&#xA;    myMD.simulate(400);&#xA;    std::string fileName = &quot;myFile.txt&quot;;&#xA;    myMD.wrtieToFile(fileName);&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Many thanks!&lt;/p&gt;&#xA;" OwnerUserId="12013" LastActivityDate="2015-05-13T11:32:31.870" Title="Unexpected behavior, values tend to converge instead of fluctuate. (MD)" Tags="|c++|computational-physics|molecular-dynamics|oop|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16422" PostTypeId="2" ParentId="16390" CreationDate="2014-12-07T15:40:00.150" Score="2" Body="&lt;p&gt;In the comments you mentioned that $f$ has a very special form. Let $u:[a,b]\rightarrow \mathbb{R}$. Then $f:[a,b]\rightarrow\{0,1\}$ is defined for some set $S \subset [a,b]$ as follows:&#xA;$$f(x) = \begin{cases} 1 &amp;amp; \text{if $x \in S$,} \\ 0 &amp;amp; \text{otherwise.} \end{cases}$$&#xA;As Bill Barth mentioned, you can now split the matrix $A$ into two parts as&#xA;$$A = K+C.$$&#xA;I like to think in terms of finite element methods since I am more acquainted with those. Now your weak formulation is: find $u \in H^1_0(a,b)$ such that&#xA;$$(u_x,v_x)+(fu,v)=(\kappa,v)$$&#xA;hold for very test function $v \in H^1_0(a,b)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discretization of the first term leads to $K$ and the second term to $C$. My intuition now says that since your $f$ is so &quot;simple&quot;, you can just build the mass matrix $M$ corresponding to the term $(u,v)$ and then construct a mapping $F_f$ satisfying&#xA;$$C = F_f(M).$$&#xA;If $F_f$ is simple enough to evaluate (i.e. just setting rows and colums to zero), then you only once construct $M$ and every $C$ is just a slight perturbation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it make any sense?&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2014-12-07T15:40:00.150" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16423" PostTypeId="1" CreationDate="2014-12-07T18:07:32.603" Score="1" ViewCount="787" Body="&lt;p&gt;I'm currently revising my optimization algorithm for a specific part of a problem. I have trouble in wrapping my head around a new approach and my mind is having this tunnel-vision of ideas. I could really use some fresh perspectives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll try my best to simply the explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Please see attached file &quot;Example-1.png&quot; and &quot;Example-2.png&quot;)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Say, we are given 3 distinct persons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of these person have a specific &lt;strong&gt;Supply&lt;/strong&gt; (an item that he/she possess) and a &lt;strong&gt;Need&lt;/strong&gt; (an item that needs to be acquired). Now, if these &lt;strong&gt;Supply-Need&lt;/strong&gt; is reversed, and their reverse can be found in another person, they can be traded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, the pairs have a numerical value called &lt;strong&gt;Gravity&lt;/strong&gt; that specifies the importance of the pair to the person. We can treat it as a &lt;em&gt;weight&lt;/em&gt; on how much a person can be &quot;&lt;em&gt;satisfied&lt;/em&gt;&quot; if the &lt;strong&gt;Supply-Need&lt;/strong&gt; pair is met. Each person can only allocate and distribute 100-points of &lt;strong&gt;Gravity&lt;/strong&gt; among all of his/her &lt;strong&gt;Supply-Need&lt;/strong&gt; pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the &lt;strong&gt;Total Satisfaction&lt;/strong&gt; of this process can be computed by getting the sum of all the Persons' individual satisfactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is to have the group of Persons trade their &lt;strong&gt;Supply-Need&lt;/strong&gt; pair/s in different combinations such that we can acquire the largest &lt;strong&gt;Total Satisfaction&lt;/strong&gt; as possible.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In PSO, each &lt;em&gt;Particle&lt;/em&gt; represents a candidate solution based on its location in the Search Space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the attached examples in this post, we can say that &lt;em&gt;Example-1.PNG&lt;/em&gt; is a distinct candidate solution to this problem, as well as &lt;em&gt;Example-2.PNG&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What's the best approach in how this problem can be represented and evaluated by a Fitness/Objective Function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you characterize this problem in PSO?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any recommendations of published work with the same problem as this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EXAMPLE-1.PNG&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ei2Vd.png&quot; alt=&quot;Example-1.png&quot;&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EXAMPLE-2.PNG&lt;/strong&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/QCv4o.png&quot; alt=&quot;Example-2.png&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="12015" LastEditorUserId="12015" LastEditDate="2014-12-08T19:12:33.520" LastActivityDate="2017-04-21T12:06:43.253" Title="How to solve this problem using Particle Swarm Optimization?" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="16425" PostTypeId="1" CreationDate="2014-12-07T22:38:17.187" Score="2" ViewCount="94" Body="&lt;p&gt;I wrote a Monte Carlo simulation of the 2D Lotka-Volterra model on a discrete lattice (with periodic boundary conditions). A video that I produced (which images the system after some number of monte carlo steps) can be found &lt;a href=&quot;https://www.dropbox.com/s/fah25ro1x6203f4/Pred-Prey-Lattive.avi?dl=0&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. I am wondering, what is the best way to track the spatial &quot;speed&quot; of the oscillations of the two species across the lattice from one video frame to the next?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My model is done with the following rates, where $A$ denotes a predator node on the lattice, $B$ denotes a prey node on the lattice, and $\emptyset$ denotes an empty space on the lattice:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;A + B \xrightarrow{\lambda} A+A,\;(predator\;consuming\; prey)\\&#xA;B + \emptyset \xrightarrow{\sigma} B+B,\;(prey\;growing\;into\;empty\;adjacent\;spaces)\\&#xA;A \xrightarrow{\mu} \emptyset,\;(predators\;dying\;off)\\&#xA;$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each Monte Carlo step I choose one space on the lattice randomly, do a cumsum(each neighbor's rate./sum(neighbors' rates)), and then generate a random number to decide which of the three reactions above will be done. In my video, $\lambda=1.6$, $\sigma=50$, and $\mu=0.1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if I should try to track a certain shape in each frame and measure its deformation, or if I should try to use some other sort of spatial metric. To be clear, the data for each video frame is stored in an N by N matrix (with values 0 for empty, 1 for predator, and 2 for prey), so I am not limited to an analysis of the video I provided, it is only for explanation. My source code for this project can be found &lt;a href=&quot;https://www.dropbox.com/sh/k0y7wmqn3djv2aw/AAAWQU_BvyLUwy1UQssDUMa0a?dl=0&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the typical way of approaching this sort of problem, perhaps someone can provide a reference?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="11799" LastEditorUserId="11799" LastEditDate="2014-12-09T12:55:20.947" LastActivityDate="2014-12-09T12:55:20.947" Title="Tracking the speed of 2D oscillations on a lattice" Tags="|simulation|monte-carlo|computational-physics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16427" PostTypeId="2" ParentId="16390" CreationDate="2014-12-08T08:25:16.667" Score="1" Body="&lt;p&gt;You are basically trying to solve a one-dimensional modified Helmholtz equation with variable coefficients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the locations where the changes in the coefficients occur remain constant in time, I would suggest using a standard Finite Element (FE) approach where you would place element interfaces at these locations. Piecewise constant coefficients do not introduce an error in the solution using the FE method in this case. I must note that I am biased toward a FE solution, as this is the field I come from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the coefficients also change suddenly in time. This may introduce a numerical error if not dealt with properly. You can either reduce the time step drastically after each coefficient change (this may be costly if there are a lot of such changes), or perhaps use space-time finite elements - I am not sure but I think in the space-tame case the FE scheme would not be sensitive to time jumps in the coefficients as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also other approaches to dealing with the accuracy problem due to sudden changes in the domain.&lt;/p&gt;&#xA;" OwnerUserId="6775" LastActivityDate="2014-12-08T08:25:16.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16429" PostTypeId="1" CreationDate="2014-12-08T18:29:11.727" Score="2" ViewCount="81" Body="&lt;p&gt;Is there a mesh data format which is particularly well suited for Cartesian meshes with Dey-Mittra style cut-cells? Ideally, one which does not require converting the cut-cells into a more general polyhedron. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Non-proprietary formats with open-source implementations are also preferable.&lt;/p&gt;&#xA;" OwnerUserId="7016" LastActivityDate="2014-12-08T18:29:11.727" Title="Data format for Cartesian mesh with cut cells" Tags="|mesh|data-structures|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16430" PostTypeId="1" CreationDate="2014-12-08T18:36:56.677" Score="1" ViewCount="363" Body="&lt;p&gt;I am solving a problem of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial u(x,y,t)}{\partial t} = \nabla^2 u(x,y,t) - f(x,y,t)u(x,y,t) - \kappa(x,y,t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, I am solving this at each time step by assuming a quasi-steady-state:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^2 u(x,y,t) = f(x,y,t)u(x,y,t) + \kappa(x,y,t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, I use finite differences, which means solving the following (forgetting boundary conditions for now, but for reference they are Neumann) at each time step:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{u^t_{i, j+1} + u^t_{i+1, j} + u^t_{i, j-1} + u^t_{i-1, j} - 4u^t_{i,j}}{h^2} = f^t_{i,j} u^t_{i,j} + \kappa^t_{i,j}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means I have to solve a linear system of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax = b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $A$ is a matrix made of the laplacian, as well as the $f^t_{i,j} u^t_{i,j}$ term. For my 100x100 system (resulting in a laplacian of size 10000x10000), this takes about 0.4 seconds for each time step using Eigen's C++ library solver:SimplicialLLT. However, I would like to speed up solving by any means possible. The issue is that I am limited in the pre-conditioning that I can do since the matrix $A$ changes each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any ideas? Finite element, spectral methods, non-steady state approximations all welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before anyone asks, the terms $f^t_{i,j}$ and $\kappa^t_{i,j}$ cannot be forecast ahead of time, as they depend on $u^{t-1}_{i,j}$. Hence parallelisation is not going to be possible I think. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ben&lt;/p&gt;&#xA;" OwnerUserId="11972" LastEditorUserId="14772" LastEditDate="2015-03-15T23:13:00.850" LastActivityDate="2015-03-15T23:13:00.850" Title="Solve steady state reaction-diffusion/Helmholtz equation numerically" Tags="|finite-element|finite-difference|linear-solver|advection-diffusion|spectral-method|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16431" PostTypeId="1" AcceptedAnswerId="16437" CreationDate="2014-12-08T20:16:07.470" Score="2" ViewCount="117" Body="&lt;p&gt;I'm looking to solve the Richards' equation. This models water flow in porous media and is a nonlinear, possibly degenerative, parabolic differential equation that takes the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t \Theta(\psi) - \nabla \cdot (K(\Theta)\nabla(\psi+z))=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\psi$ is pressure head, $\Theta$ is the water content, $K$ is the conductivity and $z$ is a vertical height. Pressure is the primary unknown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a 'famous' equation and further information can be found on scholar [1,2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to calculate a 3D solution, and I am looking into possible solvers. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is Clawpack applicable to this type of equation? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible to use Clawpack with an 3D unstructured mesh such as those that are generated from CGAL (currently I have a surface mesh defined using CGAL).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any insight is much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S037704270301001X&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/S037704270301001X&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2]&lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1029/WR026i007p01483/abstract&quot; rel=&quot;nofollow&quot;&gt;http://onlinelibrary.wiley.com/doi/10.1029/WR026i007p01483/abstract&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12027" LastEditorUserId="276" LastEditDate="2014-12-09T05:05:19.797" LastActivityDate="2014-12-09T05:05:19.797" Title="Application of CLAWPACK to Richards' equation" Tags="|pde|finite-volume|parabolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16432" PostTypeId="1" CreationDate="2014-12-08T20:16:38.630" Score="1" ViewCount="217" Body="&lt;p&gt;What are the problems that arise when fitting (2D or 3D) a set of scattered data? (non uniformly distributed)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had some data I had to fit and I solved the problem using the &lt;code&gt;triscatteredinterp()&lt;/code&gt; matlab function which implements different types of methods: natural neighbor interpolations, linear interpolation and nearest-neighbor interpolation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to understand is: what are the issues for this kind of fitting? Why should I choose a method instead of another one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another function is &lt;code&gt;delaunay()&lt;/code&gt; which implements the Delaunay triangulation. What is the difference between this and the other methods?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="11676" LastEditDate="2015-01-04T15:18:38.420" LastActivityDate="2015-07-04T16:33:10.130" Title="Conceptual question about fitting of scattered data" Tags="|mesh|unstructured-mesh|delaunay-triangulation|curve-fitting|information-theory|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16433" PostTypeId="1" CreationDate="2014-12-08T20:44:16.743" Score="1" ViewCount="989" Body="&lt;p&gt;I am encountering a problem that my MATLAB code ran and an error occurs and shows that it is out of memory. My code is using finite element method to solve some complicated kind of integral equation and the number of elements can reach more than 100,000. I use the sparse to store the stiffness matrix and ran the code in parallel on our school cluster. I am not sure whether the data is too big or my code is not well programmed that caused the out-of-memory problem. I also have the c version of the code and same problem occured(the error showed to be segmentaion fault). For small data my code does not have anything wrong. Since it is a 2d FEM code, suppose the number of unknown in each coordinate is n(so that the total is n^2), then the bandwidth of the matrix is approximately less than 2*n. It is symmetric positive definite. When I use MATLAB, I just use \ to solve the linear system. For c version, I use the subroutine for banded matrix in CLAPACK. It is the big data problem and my code does not work, so is there anybody who would like to help me and tell me what I should do to fix it, no matter in c or MATLAB? And for c, is there any package available for solving linear system with big data? Thanks so much!&lt;/p&gt;&#xA;" OwnerUserId="12028" LastActivityDate="2014-12-09T23:43:25.460" Title="Searching solution for solving huge linear system in MATLAB or C" Tags="|matlab|c|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16434" PostTypeId="2" ParentId="16433" CreationDate="2014-12-09T00:20:02.713" Score="2" Body="&lt;p&gt;100k rows for FEM is big enough and likely sparse enough that you should probably try a sparse iterative solver package like &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt; or &lt;a href=&quot;http://trilinos.org/&quot; rel=&quot;nofollow&quot;&gt;Trilinos&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-12-09T00:20:02.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16435" PostTypeId="2" ParentId="16430" CreationDate="2014-12-09T01:19:08.277" Score="3" Body="&lt;p&gt;One approach to acceleration would go as follows: Assume that $A^t=A(u^t)$ is the matrix you try to solve with, i.e., you are looking to solve the linear systems&#xA;$$&#xA;  A^t x^t = b^t.&#xA;$$&#xA;Let me assume for a moment that $f^t \ge 0$, then $A^t$ is a symmetric and positive definite matrix. (If my assumption should be wrong, then it is still symmetric but may no longer be positive definite.) Given that, you can solve the linear system with the Conjugate Gradients (CG) method. The question of course is how to precondition it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach would be to compute a Cholesky decomposition of $A^0$ in the first time step and use this as the preconditioner for the next time steps. You will be able to solve time step zero in exactly one CG iteration since the preconditioner is perfect. In subsequent time steps, $A^t$ will start to deviate from the preconditioner and you will need more iterations. Once the number of iterations becomes too large, you re-compute the Cholesky decomposition in the next time step. This works because the expensive step is to compute the Cholesky decomposition, but you only do that every few time steps.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-09T01:19:08.277" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16436" PostTypeId="1" CreationDate="2014-12-09T02:45:00.903" Score="0" ViewCount="55" Body="&lt;p&gt;What is a reasonable manufactured solution to test the following equation against its finite difference approximation? I want it to look like a Cosine function about $0$, rotated about $Z$ axis (basically not varying about $\theta$) such that $f$  turns out to be a not so bad function in terms of $r,\theta$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{rr} \;+\; \frac{1}{r} u_r \;+\; \frac{1}{r^2} u_{\theta\theta}\;+\; Cu \;=\; f(r,\theta)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it should avoid the discontinuity at $r = 0$&lt;/p&gt;&#xA;" OwnerUserId="11828" LastActivityDate="2014-12-09T03:39:27.070" Title="What is a reasonable manufactured solution to test finite difference method?" Tags="|pde|finite-difference|elliptic-pde|" AnswerCount="0" CommentCount="9" ClosedDate="2014-12-10T02:50:28.300" ContentLicense="CC BY-SA 3.0" />
  <row Id="16437" PostTypeId="2" ParentId="16431" CreationDate="2014-12-09T03:50:45.253" Score="2" Body="&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.clawpack.org/&quot; rel=&quot;nofollow&quot;&gt;Clawpack&lt;/a&gt; is designed to solve systems of first-order hyperbolic PDEs on structured (logically quadrilateral or hexahedral) meshes.  It is possible to incorporate parabolic terms, but the real advantage of using Clawpack is its capability for dealing with nonlinear hyperbolic terms that can lead to discontinuities in the solution.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2014-12-09T03:50:45.253" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16438" PostTypeId="1" CreationDate="2014-12-09T11:19:52.687" Score="1" ViewCount="171" Body="&lt;p&gt;&quot;We consider an unstructured planar&#xA;(undirected) graph and calculate the stationary probability distribution of the random&#xA;walk on the graph. The graph is generated by choosing n random points in the unit&#xA;square and triangulating them using Delaunay triangulation.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sentence is from the description of a test problem in one article, I want to use this test problem, but I don't konw how to construct the matrix corresponding to this graph by Matlab.&lt;/p&gt;&#xA;" OwnerUserId="11856" LastEditorUserId="4474" LastEditDate="2014-12-09T18:22:53.963" LastActivityDate="2015-01-08T23:36:04.903" Title="How to construct the matrix like this" Tags="|matlab|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16439" PostTypeId="1" CreationDate="2014-12-09T11:00:14.340" Score="4" ViewCount="573" Body="&lt;p&gt;I am a computer science student and I came across the Verlet Cloth simulation. An implementation is &lt;a href=&quot;https://github.com/suffick/Tearable-Cloth&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. I was interested in how such computation would lend itself to parallelization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem looks like a variant of N-body computation where we could compute constraints on each point and them move the points based on the effect of the constraint. I tried along those lines and ended up with &quot;logical&quot; errors. I am sure my understanding of the physics is incomplete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I tried.&#xA;Assume a constraint C between A and B. Instead of trying to resolve C once and update both A and B, I tried resolving C twice once along with A's constraints and update just A and once along with B's constraints and just update B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://en.wikipedia.org/wiki/Verlet_integration#Constraints&quot; rel=&quot;nofollow&quot;&gt;this link&lt;/a&gt;, I am wondering whether the constraint resolution for each point can be done independently of the other, when each point is constrained by multiple other points. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dist = Math.abs(A - B)&#xA;diff = (dist - r) / dist&#xA;moveBy = 0.5 * diff * dist&#xA;A = A + moveBy&#xA;B = B - moveBy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Doing this for each constraint works. But the moment, I compute all these &quot;moveBy&quot; values for all constraints together and move the points by the sum of these values, things don't work. This makes me think that the order in which the points are resolved for constraints and the order in which constraints are applied matters.Is that right?&lt;/p&gt;&#xA;" OwnerUserId="12035" OwnerDisplayName="Sruti" LastEditorUserId="12035" LastEditDate="2014-12-09T21:11:37.617" LastActivityDate="2015-06-09T04:59:10.280" Title="Is solving Verlet constraints parallelizable?" Tags="|parallel-computing|computational-physics|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="16440" PostTypeId="2" ParentId="16433" CreationDate="2014-12-09T14:16:45.487" Score="0" Body="&lt;p&gt;Take a look at Intel MKL or ACML. Especially MKL provides efficient and parallel sparse solvers on CPU. Both direct and indirect ones are present. As a second alternative, also look at PARDISO solver &lt;a href=&quot;http://www.pardiso-project.org/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2014-12-09T14:16:45.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16441" PostTypeId="2" ParentId="16438" CreationDate="2014-12-09T23:16:03.190" Score="1" Body="&lt;p&gt;The matrix that corresponds to the &quot;stationary probability distribution of the random walk on the graph&quot; is the &lt;em&gt;graph Laplacian&lt;/em&gt;. Wikipedia has a long article on it here: &lt;a href=&quot;http://en.wikipedia.org/wiki/Laplacian_matrix&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Laplacian_matrix&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-09T23:16:03.190" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16442" PostTypeId="2" ParentId="16433" CreationDate="2014-12-09T23:43:25.460" Score="3" Body="&lt;p&gt;While Bill and tbirdal's suggestions of looking at other sparse solvers are good, I suggest trying Matlab's own sparse iterative methods first, since you already have something working. &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/mldivide.html&quot; rel=&quot;nofollow&quot;&gt;From looking at matlab's documentation on the A\b operator,&lt;/a&gt; it's always going to use a direct decomposition method, which as Bill said, 100k unknowns might be too big for. Since A is symmetric positive definite, try the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/pcg.html&quot; rel=&quot;nofollow&quot;&gt;preconditioned conjugate gradient method&lt;/a&gt;  -- x = pcg(A,b).&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-12-09T23:43:25.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16443" PostTypeId="2" ParentId="16359" CreationDate="2014-12-10T03:45:32.340" Score="11" Body="&lt;p&gt;The Thomas algorithm is very efficient because its operation count is very low and because data accesses are very likely to be cache hits once data is initially read from memory.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two loops.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first loop traverses the data forward.  Each element of the lower, main and upper triangle, along with the right-hand-side vector (which is typically also the solution vector upon completion) will be read only once from memory and used several times as part of the computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second loop traverses the data in reverse order, so data that was just used in the first loop is still present in the cache.  On most modern microprocessors, you will have a large highest level cache of several MB.  This means, even for a system of 300,000 equations, much of the data will still be in cache.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any parallel tridiagonal solver I know requires more operations than the serial algorithm.  If you count the operations in the algorithm you mentioned from Karniadakis and Kirby, you will see it does many more operations, even though they are done in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, a parallel tridiagonal solver should be faster than the simple serial algorithm, but at problem sizes and processors count much larger than you are using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would have a better chance of speeding up your solution if you could solve for your multiple systems simultaneously, assuming some of them are available at the same time.  If that is not possible, but your tridiagonal matrix is the same for multiple solves you could save some serial costs by splitting the Thomas algorithm into two parts, factorization and solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If none of these situations is possible, you will likely not see speedup from a parallel version of the Thomas algorithm.  &lt;/p&gt;&#xA;" OwnerUserId="12040" LastActivityDate="2014-12-10T03:45:32.340" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16444" PostTypeId="1" AcceptedAnswerId="16445" CreationDate="2014-12-10T15:28:00.127" Score="1" ViewCount="4179" Body="&lt;p&gt;It is necessary for me to find the unit outward normal vector for the curve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\gamma=(x(t),y(t)) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &#xA;$$x(t)=\cos(t)−0.5\cos(3t)$$ &#xA;and &#xA;$$y(t)=\sin(t)+\sin(7t)+\sin(3t)$$ &#xA;I know how to find unit outward normal vector for this: using&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T=\frac{\gamma' (t)}{||\gamma(t)||} $$ &#xA;so &#xA;$$N=\frac{T (t)}{||T(t)|| }$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;but my problem is that I do not have $t$ . I just have $x(t)$ and $y(t)$. How could I find $t$ or $N$  without knowing $t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any command in MATLAB or MAPLE to this?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="7733" LastEditDate="2014-12-10T17:15:37.857" LastActivityDate="2014-12-10T21:24:36.360" Title="Determine unit outward normal vector for a curve" Tags="|matlab|computational-geometry|maple|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16445" PostTypeId="2" ParentId="16444" CreationDate="2014-12-10T16:10:45.127" Score="2" Body="&lt;p&gt;All the info you need can be found by reading about the Frenet (or TNB, Tangent-Normal-Binormal) frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the notation on &lt;a href=&quot;http://en.wikipedia.org/wiki/Frenet%E2%80%93Serret_formulas&quot; rel=&quot;nofollow&quot;&gt;the Wikipedia page&lt;/a&gt;, you can denote your curve as&#xA;$$&#xA;\vec{r}(s) = (x(s),y(s),0)&#xA;$$&#xA;and the tangent and normal vectors are given by&#xA;$$&#xA;\vec{T} = \dfrac{d\vec{r}}{ds}&#xA;$$&#xA;and&#xA;$$&#xA;\vec{N} = \dfrac{d\vec{T}}{ds}\left/\left|\left|\dfrac{d\vec{T}}{ds}\right|\right|\right.,&#xA;$$&#xA;where $s$ represents arc length.&#xA;Since you are restricted to 2D instead of 3D, the binormal vector is just $\pm\bf\hat{k}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The normal vector $\vec{N}$ always points to the interior of the curve (the direction of normal acceleration if you're moving along the curve), so the vector you want is actually given by $-\vec{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not quite sure what you mean by the statement &quot;I do not have $t$. I just have $x(t)$ and $y(t)$.&quot; The normal vector will be different at different points along your curve. You can write a general expression for $\vec{N}(t)$ or you can pick a specific value for $t$ and get a unique $\vec{N}$. Also, since your independent variable, $t$, is not arc length you will have to compute $\vec{T}$ using the chain rule as&#xA;$$&#xA;\vec{T} = \dfrac{d\vec{r}}{dt}\left/\dfrac{ds}{dt}\right..&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an &lt;a href=&quot;http://en.wikipedia.org/wiki/Arc_length&quot; rel=&quot;nofollow&quot;&gt;analytical formula for arc length&lt;/a&gt; of a parametrically defined curve&#xA;$$&#xA;s = \int_{t_0}^t \sqrt{x'(t^*)^2+y'(t^*)^2} dt^*&#xA;$$&#xA;Or, if you have $x$ and $y$ defined as a set of discrete points along the curve as $x=\{x_1, x_2, x_3, ...\}$, $y=\{y_1, y_2, y_3, ...\}$ you could estimate the tangent and normal vectors using (for example) finite differences.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Your specific case:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An analytical expression for the parameterization you have given will be complicated. A good alternative might be to pick a bunch of (closely spaced) points on your curve to discretize the problem and then use something like &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/11169-frenet&quot; rel=&quot;nofollow&quot;&gt;this Frenet coordinate routine&lt;/a&gt; from the MATLAB File Exchange.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2014-12-10T21:24:36.360" LastActivityDate="2014-12-10T21:24:36.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16446" PostTypeId="1" AcceptedAnswerId="17531" CreationDate="2014-12-10T17:00:16.340" Score="3" ViewCount="2026" Body="&lt;p&gt;I am trying to write a code to solve a nonlinear BVP using the Finite Difference Method. The BVP is:&lt;br&gt;&#xA;$(T^2)\frac{\partial^2 T}{\partial x^2} + T \left( \frac{\partial T}{\partial x}\right)^2 + Q = 0$&lt;br&gt;&#xA;The boundary conditions are Dirichlet at $x = 0$ and Robin at $x = L$. The parameter $Q$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My (probably naive) approach is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use second-order accurate approximations for the first and second derivatives.&lt;/li&gt;&#xA;&lt;li&gt;Use the discretized form to generate a system of coupled equations for the spatial points. &lt;/li&gt;&#xA;&lt;li&gt;Construct a linear system of the form $\mathbf{A}\mathbf{x} = \mathbf{b}$, where $\mathbf{x}$ is a vector of $T$ at each spatial point $x_0, \ldots, x_L$, $\mathbf{A}$ is a matrix of the coefficients of $T$ arising from the discretization, and $\mathbf{b}$ is a solution vector including the source term $Q$.&lt;/li&gt;&#xA;&lt;li&gt;Solve the linear system for $\mathbf{x}$ somehow, I have not gotten this far.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I really don't know what to do about the nonlinearity. My first problem arises with the squared first derivative. If I apply the finite difference approximation to the first derivative, I get:&lt;br&gt;&#xA;$T_i\left(\frac{T_{i+1} - T_{i-1}}{2\Delta x} \right)^2$&lt;br&gt;&#xA;Developing the above term yields:&lt;br&gt;&#xA;$T_i \left(\frac{T_{i+1}^2-2T_{i+1}T_{i-1}+T_{i-1}^2}{4 \Delta x^2} \right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to handle this term. I need to linearize something at some point (before or after discretization?) but I am not sure how. And say I figure out a way to handle the squared first derivative, I don't know how to deal with the $T_i$ in front of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is:&lt;/strong&gt; how do you discretize and then linearize (or vice-versa) these nonlinear terms?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time. Please let me know if I should elucidate anything. &lt;/p&gt;&#xA;" OwnerUserId="10256" LastEditorUserId="10256" LastEditDate="2014-12-11T04:52:09.103" LastActivityDate="2015-11-25T16:38:33.987" Title="Solving a Nonlinear BVP using Finite Difference Method" Tags="|finite-difference|nonlinear-equations|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="16447" PostTypeId="2" ParentId="16444" CreationDate="2014-12-10T17:26:22.970" Score="1" Body="&lt;p&gt;I implemented this real quick using finite differences as suggested by @DougLipinski, and this is what I come up with, at first glance it looks correct.  Green vectors are the tangent vectors and red are the normals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the differential arc length I used this relationship&#xA;$$(ds)^2 = (dx)^2 + (dy)^2$$&#xA;and applied a simple first order finite difference to $x$ and $y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/88VnL.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code, let me know if you see anything wrong&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all&#xA;close all&#xA;&#xA;t = 0:0.01:(2*pi+0.01);&#xA;t = t(:);&#xA;n = length(t);&#xA;x = cos(t) - 0.5*cos(3*t);&#xA;y = sin(t) + sin(7*t) + sin(3*t);&#xA;&#xA;e = ones(n,1);&#xA;A = 0.5*spdiags([-e e],[-1,1],n,n);%first order FD stencil&#xA;&#xA;dx = A*x; &#xA;dy = A*y;&#xA;&#xA;ds = sqrt(dx.^2 + dy.^2);&#xA;&#xA;T = [dx./ds  dy./ds];&#xA;for rr = 1:size(T,1)&#xA;   T(rr,:) = T(rr,:)/sqrt(T(rr,1)^2 + T(rr,2)^2); &#xA;end&#xA;dT  = [A*T(:,1) A*T(:,2)];&#xA;&#xA;N   = [dT(:,1)./ds dT(:,2)./ds];&#xA;for rr = 1:size(N,1)&#xA;   N(rr,:) = N(rr,:)/sqrt(N(rr,1)^2 + N(rr,2)^2); &#xA;end&#xA;&#xA;figure&#xA;    plot(x,y)&#xA;    hold on&#xA;    quiver(x,y,T(:,1),T(:,2))&#xA;    quiver(x,y,N(:,1),N(:,2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7257" LastEditorUserId="7257" LastEditDate="2014-12-10T19:23:32.570" LastActivityDate="2014-12-10T19:23:32.570" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16448" PostTypeId="1" AcceptedAnswerId="16452" CreationDate="2014-12-10T18:21:46.880" Score="4" ViewCount="1127" Body="&lt;p&gt;I recently implemented Successive Over Relaxation using Cuda as a part of my course project and was curious to know how I can make the code more efficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Red/Black SOR scheme which is popular version of parallel SOR. To update the value at a grid point, I need to access 4 adjacent values which are generally not contiguous in the memory and hence the GPU is not able to use memory coalescing feature of cuda which makes the algorithm memory bound. To overcome this I have reordered the storage of odd and even grid points in a separate vectors as mentioned in the paper here: &lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-3-642-31464-3_60&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/chapter/10.1007%2F978-3-642-31464-3_60&lt;/a&gt; which takes advantage of memory coalescing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementing the technique mentioned in the paper I see a speedup of around ~15 (compared to ~10 with traditional Red/Black SOR) when compared with 1 CPU. How much performance speed up I should be expecting? I feel speedup of ~15 is quite low?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm calling the kernel functions to update odd values and then even values sequentially from host CPU in following way :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(size_t it = 0; it &amp;lt; itmax; it++){&#xA;      cuda_odd_update &amp;lt;&amp;lt;&amp;lt;dimGrid_odd,dimBlock&amp;gt;&amp;gt;&amp;gt; (odd,even);&#xA;      cuda_even_update &amp;lt;&amp;lt;&amp;lt;dimGrid_even,dimBlock&amp;gt;&amp;gt;&amp;gt; (odd,even);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;odd&lt;/code&gt; and &lt;code&gt;even&lt;/code&gt; are the pointers to reordered vectors in device memory. I'm using this approach as I'm not aware of any way to synchronize all the threads in the device. Is there any better way to implement the execution loop?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code below implements the technique mentioned in the paper and not a traditional Red/Black SOR: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__global__ void cuda_odd_update(double* odd,double* even){&#xA;&#xA;  size_t tx = blockIdx.x*blockDim.x + threadIdx.x;&#xA;  size_t ty = blockIdx.y*blockDim.y + threadIdx.y;&#xA;  size_t odd_index = tx*height_odd+ty;&#xA;  size_t even_index = tx*height_even+ty;&#xA;&#xA;  if (( (ty == 0 &amp;amp;&amp;amp; tx%2 == 0) || (ty == height_odd-1 &amp;amp;&amp;amp; tx%2 == 1) || (ty &amp;gt; 0 &amp;amp;&amp;amp; ty &amp;lt; height_odd-1) ) &amp;amp;&amp;amp; (tx &amp;gt; 0 &amp;amp;&amp;amp; tx &amp;lt; width-1 &amp;amp;&amp;amp; ty &amp;lt; height_odd) ){&#xA;&#xA;      odd[odd_index] = (1.0 - omega)*odd[odd_index] + omega/(2*(1+beta))&#xA;                     * ( (tx%2)*even[even_index]+(1-tx%2)*even[even_index+1]&#xA;                       + (tx%2)*even[even_index-1]+(1-tx%2)*even[even_index]&#xA;                       + beta * ( even[even_index-height_even]&#xA;                                + even[even_index+height_even]   )) ;&#xA;&#xA;  }&#xA;&#xA;}&#xA;&#xA;&#xA;__global__ void cuda_even_update(double* odd,double* even){&#xA;&#xA;  size_t tx = blockIdx.x*blockDim.x + threadIdx.x;&#xA;  size_t ty = blockIdx.y*blockDim.y + threadIdx.y;&#xA;  size_t odd_index = tx*height_odd+ty;&#xA;  size_t even_index = tx*height_even+ty;&#xA;&#xA;  if(( (ty == 0 &amp;amp; tx%2 == 1) || (ty == height_even-1 &amp;amp;&amp;amp; tx%2 == 0) || (ty &amp;gt; 0 &amp;amp;&amp;amp; ty &amp;lt; height_even-1) ) &amp;amp;&amp;amp; ( tx &amp;gt; 0 &amp;amp;&amp;amp; tx &amp;lt; width-1 &amp;amp;&amp;amp; ty &amp;lt; height_even)){&#xA;&#xA;&#xA;      even[even_index] = (1.0 - omega)*even[even_index] + omega/(2*(1+beta))&#xA;                       * ( (1-tx%2)*odd[odd_index] + (tx%2)*odd[odd_index+1]&#xA;                         + (1-tx%2)*odd[odd_index-1] + (tx%2)*odd[odd_index]&#xA;                         + beta * ( odd[odd_index-height_odd]&#xA;                                  + odd[odd_index+height_odd]  )) ;&#xA;&#xA;&#xA;  }&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm currently using Intel Xeon CPU E5-2660-2.20GHz (8 cores) and NVIDIA TESLA M2090.&#xA;The &lt;code&gt;nvprof&lt;/code&gt; gives the following results (for 1000 iterations):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CPU time : 62.69&#xA;==27665== NVPROF is profiling process 27665, command: ./sor&#xA;GPU time : 3.73&#xA;End!&#xA;==27665== Profiling application: ./sor&#xA;==27665== Profiling result:&#xA;Time(%)      Time     Calls       Avg       Min       Max  Name&#xA; 49.81%  1.83752s      1000  1.8375ms  1.8333ms  1.8422ms  cuda_even_update(double*, double*)&#xA; 49.67%  1.83226s      1000  1.8323ms  1.8290ms  1.8364ms  cuda_odd_update(double*, double*)&#xA;  0.26%  9.6281ms         2  4.8141ms  4.6306ms  4.9975ms  [CUDA memcpy DtoH]&#xA;  0.26%  9.4569ms         2  4.7284ms  4.7065ms  4.7504ms  [CUDA memcpy HtoD]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9950" LastEditorUserId="9950" LastEditDate="2014-12-10T21:55:53.497" LastActivityDate="2015-01-09T23:01:57.373" Title="Efficient Successive Over Relaxation using Cuda?" Tags="|parallel-computing|gpu|cuda|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="16450" PostTypeId="1" CreationDate="2014-12-10T19:46:59.277" Score="0" ViewCount="998" Body="&lt;p&gt;I am performing analysis on chassis (Static Structural) and for optimization purpose i am asked to generate MESH-INDEPENDENCE REPORT,of which i have no idea. I have tried going through research papers but couldn't find anything important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Need a basic idea about Mesh-independence, any example would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="12046" LastActivityDate="2014-12-10T21:34:21.173" Title="What is Mesh Independence Report?" Tags="|finite-element|mesh|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16452" PostTypeId="2" ParentId="16448" CreationDate="2014-12-10T20:16:50.977" Score="3" Body="&lt;p&gt;&lt;strong&gt;Synchronization.&lt;/strong&gt;&#xA;You can't synchronize threads across different blocks in the same grid. On the other hand, if you execute kernels the way you do, they are guaranteed to execute in sequence, so all the threads in &lt;code&gt;even_update&lt;/code&gt; will see all updates made by &lt;code&gt;odd_update&lt;/code&gt; in the same iteration. (See &lt;a href=&quot;http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#asynchronous-concurrent-execution&quot; rel=&quot;nofollow&quot;&gt;NVIDIA's own documentation&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no good way that I know of to synchronize threads across a whole kernel, e.g. see &lt;a href=&quot;https://devtalk.nvidia.com/default/topic/526338/cuda-programming-and-performance/synchronize-all-blocks-in-cuda/post/3724201/#3724201&quot; rel=&quot;nofollow&quot;&gt;this discussion&lt;/a&gt;), so having a main loop execute kernels in sequence seems fine. You can measure &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Speedup.&lt;/strong&gt;&#xA;Speedup relative to the CPU is not &lt;em&gt;such&lt;/em&gt; a useful performance measure. I know it is always reported, and so on, but it can be a little bit misleading. For example, someone using the same GPU and the same memory, but a faster CPU will get a lower speedup, but that says nothing about the performance of their code. &lt;a href=&quot;http://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html#performance-metrics&quot; rel=&quot;nofollow&quot;&gt;NVIDIA's best practices guide&lt;/a&gt; has some things to say about how to measure performance of your code. Since your algorithm is memory-bandwidth-limited (it performs a relatively small number of operations per memory access), the quantity I'd say is more important than the speedup is memory throughput, the number of bytes per second your program actually reads/writes (effective bandwidth), especially as a proportion of theoretical peak bandwidth of your GPU. Also helpful is to know how much more memory your kernel reads/writes due to coalescing requirements than it strictly requires.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have you tried measuring these things? Fraction of theoretical peak bandwidth is a more useful guidance for optimization than speedup relative to CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Performance.&lt;/strong&gt;&#xA;As &lt;strong&gt;@GodricSeer&lt;/strong&gt; points out, executing lots of small kernels might be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One immediate thing to try is to assign more work to each thread. So instead of each thread doing work for one index $j$, make each thread do the work for $m$ indices $j_{0:m-1}$, suitably spread out so that at every step contiguous threads to the work of contiguous indices. This reduces the amount of work the scheduler has to do by a factor of $m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue might be with things like $\mathit{even}[\mathit{even\_index}\pm1]$, where the current threads might be reading memory that was already read by a thread in the same block. I haven't run your code, but one thing to try is to load parts of $\mathit{even}$ into shared memory so that threads with $\mathrm{threadIdx}.y$ differing by $1$ can use each other's global memory reads. There might be an issue with how much shared memory this uses is the size of a block is too large, so this needs careful explicit testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you run NVIDIA's profiler on your code, does it highlight any interesting issues?&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-12-10T20:29:35.310" LastActivityDate="2014-12-10T20:29:35.310" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="16453" PostTypeId="2" ParentId="16450" CreationDate="2014-12-10T20:20:38.487" Score="2" Body="&lt;p&gt;You should solve your problem on multiple, successively-refined meshes to show that the answer is not changing substantially. Many journals require this for any numerical results to demonstrate that the solution is converged. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edited to add: See, for example, &lt;a href=&quot;http://journaltool.asme.org/templates/JFENumAccuracy.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; PDF from the ASME Journal of Fluids Engineering which expresses their editorial policy on the control of numerical accuracy. Item 4 on page 1 speaks to mesh independence. This journal, and most ASME numerical journals, have a similar policy for submitted numerical results. Most of the points are excellent, though I don't necessarily agree with the requirement for second-order methods (#2).&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2014-12-10T21:34:21.173" LastActivityDate="2014-12-10T21:34:21.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16455" PostTypeId="2" ParentId="16350" CreationDate="2014-12-10T21:17:11.057" Score="0" Body="&lt;p&gt;May be a look to the following article of Efendiev and al. may help you (eq 3.8).&#xA;&lt;a href=&quot;http://users.cms.caltech.edu/~hou/papers/MsFEM_nonlinear.pdf&quot; rel=&quot;nofollow&quot;&gt;http://users.cms.caltech.edu/~hou/papers/MsFEM_nonlinear.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers, &#xA;Thomas&lt;/p&gt;&#xA;" OwnerUserId="2116" LastActivityDate="2014-12-10T21:17:11.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16456" PostTypeId="2" ParentId="16439" CreationDate="2014-12-10T21:23:11.110" Score="1" Body="&lt;p&gt;I'm not quite sure if this answers your question, or whether its more of a length comment. If I am completely off-base here, do let me know. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So suppose that &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; are of some type &lt;code&gt;Point&lt;/code&gt;, and you call a method of &lt;code&gt;Point&lt;/code&gt;, &lt;code&gt;resolve(Point that)&lt;/code&gt;, which is defined as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dist   = Math.abs(this.x - that.x)&#xA;diff   = (dist - r) / dist&#xA;moveBy = 0.5 * diff * dist&#xA;this.x = this.x + moveBy&#xA;that.x = that.x - moveBy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not quite sure if I understood you correctly, but are you saying that you're calling this method for both A and B, &lt;em&gt;i.e.&lt;/em&gt; &lt;code&gt;A.resolve(B)&lt;/code&gt; and &lt;code&gt;B.resolve(A)&lt;/code&gt;? Don't. &lt;code&gt;moveBy&lt;/code&gt; is the same in both calls, but &lt;code&gt;this&lt;/code&gt; and &lt;code&gt;that&lt;/code&gt; apply it with different signs, so it would cancel out. This is to say that the order matters. Note that just calling &lt;code&gt;A.resolve(B)&lt;/code&gt; &lt;strong&gt;would&lt;/strong&gt; work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right, I wanted to cover all bases, and I don't think you actually did what I proposed above. Perhaps, instead, what you have is the following for &lt;code&gt;resolve(Point that)&lt;/code&gt;, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dist = Math.abs(this.x - that.x)&#xA;diff = (dist - r) / dist&#xA;this.moveBy = 0.5 * diff * dist&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it is only later in another function that you actually apply the &lt;code&gt;this.x += this.moveBy&lt;/code&gt;. This is almost ok. Not quite, though. The reason is quite simple: &lt;code&gt;Math.abs&lt;/code&gt;. Let's look at the equations of harmonic motion. Here's the energy of the bond between particles A and B:&#xA;$$E = \frac{k}{2}(\|\mathbf{r}_\text{A} - \mathbf{r}_\text{B}\| - r_0)^2$$&#xA;Using Newton's equations of motion, &#xA;$$m_\text{A}\ddot{\mathbf{r}}_\text{A} = - \frac{\partial E}{\partial \mathbf{r}_\text{A}} = -k(\|\mathbf{r}_\text{A} - \mathbf{r}_\text{B}\|-r_0)\frac{\mathbf{r}_\text{A} - \mathbf{r}_\text{B}}{\|\mathbf{r}_\text{A} - \mathbf{r}_\text{B}\|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does this compare with the code above? The code does not preserve directionality (because of the absolute value in the code), and thus all the forces are to one specific direction (positive $x$, $y$ axes). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are more particles, the equations don't really get any more complicated: on the left hand side we have the same thing as before, and on the right more force terms. This is to say that in the Eulerian integration scheme that you are most probably using, you can first compute all the forces, and from those, all the velocities, and from those, all the displacements. Computing the force for one particle ought to be completely independent of the computations done at some other particle (well, independent is a poor choice of words, as half the forces are the same, but with the opposite direction as per Newton's third law, but in the sense of parallel computation they do not affect one another).&lt;/p&gt;&#xA;" OwnerUserId="8143" LastActivityDate="2014-12-10T21:23:11.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="16457" PostTypeId="2" ParentId="14824" CreationDate="2014-12-10T22:15:47.433" Score="1" Body="&lt;p&gt;The approach is &lt;strong&gt;not&lt;/strong&gt; to generate the RIB on disk or generate all the RIB in&#xA;one step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For such large amount of data, the first step is to organize it in some hierarchy fashion so that you have an Octree like bounding box structures. This is important from a memory foot print perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the above, you would use the Procedural Geometry procedural approach&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;lt;https://renderman.pixar.com/resources/RPS_17/proceduralPrimitives.html&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to emit small amount of geometry when the rendering bucket hits a bounding box containing your geometry (hence the importance of hierarchy bounding boxes). Once a rendering bucket is done, the geometries are discard keeping the memory footprint in check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the above is just the documentation from Pixar's PRMan (commercial) but is implemented in open source RenderMan renderer like Aqsis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the nature of the geometry data you have and the rendering requirements, the complexity of implementation can range from a day to a couple of weeks depending on your experience with RenderMan's API.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;" OwnerUserId="7596" LastActivityDate="2014-12-10T22:15:47.433" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="16458" PostTypeId="1" AcceptedAnswerId="17461" CreationDate="2014-12-11T05:20:11.330" Score="0" ViewCount="3847" Body="&lt;p&gt;This is a homework question, I would like to know if someone can shed some light on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $x_n = \frac{a_n + b_n}{2} ,  r=\lim_{n \to \infty}x_n$ and  $e_n =r-x_n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $[a_n,b_n]$ with $n\geq0$ denotes that successive intervals that arise in the bisection method when it is applied to a continuous function $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Show that $|e_n| \leq 2^{-(n+1)}(b_0 - a_0)$.&lt;/p&gt;&#xA;" OwnerUserId="11767" LastEditorUserId="7733" LastEditDate="2014-12-11T19:27:54.433" LastActivityDate="2014-12-11T19:27:54.433" Title="Deriving the error bound for Bisection Method" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17459" PostTypeId="1" AcceptedAnswerId="17460" CreationDate="2014-12-11T05:59:20.640" Score="1" ViewCount="106" Body="&lt;p&gt;In Golub and Van Loan's book, &lt;a href=&quot;http://www.cs.cornell.edu/cv/GVL4/golubandvanloan.htm&quot; rel=&quot;nofollow&quot;&gt;Matrix Computations&lt;/a&gt;, page 606, it is stated that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With standard discretizations, 2-dimensional problems can be solved with $O(n^{3/2})$ work and $O(n \log{n})$ fill-in. For 3-dimensional problems, the typical costs are $O(n^2)$ work and $O(n^{4/3})$ fill-in.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here, they refer to elliptic partial differential equation problems solved on a grid and $n$ refers to the number of unknowns. I guess it is trivial to calculate the cost (probably linear)  for one-dimensional problems, but I don't know how to do that. I would appreciate any help for that.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2014-12-11T07:53:25.247" LastActivityDate="2014-12-11T13:54:05.817" Title="What is the cost of factorization for one-dimensional sparse problems?" Tags="|sparse-matrix|matrix-factorization|scaling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17460" PostTypeId="2" ParentId="17459" CreationDate="2014-12-11T06:29:07.203" Score="6" Body="&lt;p&gt;It may help to define $N$, the number of discretization points along a 1D edge, and relate it to $n$, the number of unknowns in the system. In 2D on a square grid of points, $n = O(N^2)$. &lt;a href=&quot;http://en.wikipedia.org/wiki/Nested_dissection&quot; rel=&quot;nofollow&quot;&gt;Nested dissection&lt;/a&gt; efficiently reduces the sparse system you usually get from discretizations by eliminating levels of &quot;interior&quot; points.  The result is a more dense system the size of a few lines of &quot;separator&quot; points, which is of size $O(n^{1/2})$ and can be solved directly in $O(n^{3/2})$ time. The fill-in costs come from the fact that eliminating interior points at each level couples more points together (creating new connections in the matrix, hence fill-in).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 3D costs come about similarly, except now $n = N^3$, and the separators are now planes with $O(N^2) = O(n^{2/3})$ points. Solving systems involving these planes then gives you the $O(n^2)$ cost).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for 1D problems, $n=N$, and you can order unknowns from left to right to get a tridiagonal system, which can be solved in $O(n)$ time with no fill-in with the &lt;a href=&quot;http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;nofollow&quot;&gt;Thomas algorithm&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Thanks to Bill for pointing out that this is for standard low-order finite difference/element schemes - for example, higher order FD lead to increased bandwidth, high order FEM leads to a block structure. The Thomas algo can be modified to give a similar orders of work/storage, though the constants involved increase.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2014-12-11T13:54:05.817" LastActivityDate="2014-12-11T13:54:05.817" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17461" PostTypeId="2" ParentId="16458" CreationDate="2014-12-11T08:06:11.487" Score="1" Body="&lt;p&gt;The bisection method for finding the zeros of a continuous function $f$ begins with a selection of points $a_0 &amp;lt; b_0$ that bracket a zero.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $f(a_0)f(b_0) &amp;lt; 0$, then $f(a_0)$ and $f(b_0)$ have opposite sign. By the intermediate value property of continuous functions, there must be a zero at a point $r$ such that $a_0 &amp;lt; r &amp;lt; b_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step is to calculate the midpoint $x_0 = (a_0 + b_0)/2$. There are three possible cases:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(a_0)f(x_0) &amp;lt; 0 \implies r \text{ is between} \,\,a_0 \,\,\text{and}\,\, x_0,\\f(a_0)f(x_0) &amp;gt; 0 \implies r \text{ is between} \,\,x_0 \,\,\text{and}\,\, b_0,\\f(a_0)f(x_0) = 0 \implies r = x_0. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first case, set $a_1 = a_0 $ and $b_1 = x_0$. In the second case, set $a_1 = x_0 $ and $b_1 = b_0$. In the third case, the zero is found to be $r = x_0$ to within machine precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this stage, the true zero $r$ must lie in either $[a_0,x_0]$ or $[x_0,b_0]$. These intervals have identical lengths. The error of approximation is bounded by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|e_0| = |x_0 - r| \leqslant x_0 - a_0 = b_0 - x_0 = (b_0 - a_0)/2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Repeat the procedure with the interval $[a_1, b_1]$. The new approximation is $x_1 = (a_1 + b_1)/2$ with error bound&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|e_1| \leqslant (b_1 - a_1)/2 = (b_0 - a_0)/2^2 = 2^{-2}(b_0-a_0)$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuing, iteratively, we find a sequence of approximations $x_n = (a_n + b_n)/2$ for $n = 1, 2, 3, \ldots$   with error bound&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|e_n| \leqslant (b_n - a_n)/2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|e_n| \leqslant |x_n - a_n| = |b_n - x_n| = 2^{-1}(b_n - a_n) = 2^{-2}(b_{n-1} - a_{n-1}) ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it follows by induction that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|e_n| \leqslant 2^{-(n+1)}(b_0 - a_0).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also proves that the bisection method always converges to a zero of a continuous function when the initial interval is selected appropriately. &lt;/p&gt;&#xA;" OwnerUserId="9707" LastEditorUserId="9707" LastEditDate="2014-12-11T09:05:21.197" LastActivityDate="2014-12-11T09:05:21.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17462" PostTypeId="5" CreationDate="2014-12-11T16:51:36.177" Score="0" Body="&lt;p&gt;&lt;strong&gt;Wiki Quote:&lt;/strong&gt;&#xA;In physics, the Navier–Stokes equations [navˈjeː stəʊks], named after Claude-Louis Navier and George Gabriel Stokes, describe the motion of fluid substances. These equations arise from applying Newton's second law to fluid motion, together with the assumption that the stress in the fluid is the sum of a diffusing viscous term (proportional to the gradient of velocity) and a pressure term—hence describing viscous flow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Equation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Non-dimensional Navier-Stokes Equation, also known as conservation of momentum (in quite general form)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial\mathbf{u}}{\partial t}+\mathbf{\left(u\centerdot\nabla\right)u}=-\mathbf{\nabla}p+\frac{1}{Re}\nabla^{2}\mathbf{u}+f&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \mathbf{u} $ is velocity vector&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \mathbf{p} $ is pressure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f $ external forces (gravity, magnetic field, etc.)&lt;/p&gt;&#xA;" OwnerUserId="10303" LastEditorUserId="10303" LastEditDate="2014-12-11T21:53:18.713" LastActivityDate="2014-12-11T21:53:18.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17463" PostTypeId="4" CreationDate="2014-12-11T16:51:36.177" Score="0" Body="Questions about solution methods of the Navier-Stokes equations, related physical constants and non-dimensional number. Also special methods to solve the equations including the assumptions and their implementation in order to simplify them. Also, questions regarding modelling of the non-linear term, coefficients of these model can be subjective of this title. " OwnerUserId="10303" LastEditorUserId="10303" LastEditDate="2014-12-11T21:53:05.447" LastActivityDate="2014-12-11T21:53:05.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17464" PostTypeId="1" CreationDate="2014-12-11T20:47:40.773" Score="1" ViewCount="392" Body="&lt;p&gt;I have a problem with implementing proper boundary conditions on sides of my simulation (in RF or WaveOptics module of Comsol). I want to obtain an infinite but not periodic space (beacuse then I will put something inside). For example, it can be a glass-vacuum interface with incident plain waves. For such a simulation one can check if Fresnel equations work as they should and if not - the simulation is not properly set. I was trying with PMLs around the simulation area and in/out ports from the bottom/top side. But it doesn't work at all. Should I set another type of boundaries on the sides except PMLs? &lt;/p&gt;&#xA;" OwnerUserId="13055" LastActivityDate="2014-12-11T20:47:40.773" Title="infinite but non-periodic space with PMLs in Comsol" Tags="|boundary-conditions|comsol|" AnswerCount="0" CommentCount="0" ClosedDate="2017-08-15T22:02:50.977" ContentLicense="CC BY-SA 3.0" />
  <row Id="17465" PostTypeId="1" CreationDate="2014-12-11T21:44:23.157" Score="8" ViewCount="313" Body="&lt;p&gt;I have some vector $V$ which can be decomposed into the eigenspace of the hermitian sparse operator $M$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V = \sum_i v_i \hat{m}_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to find the $\hat{m}_i$ (the eigenvector itself) that correspond to the largest $v_i$ (in magnitude)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I essentially want the largest few terms of the sum, including the eigenvectors of $M$, which I don't know ahead of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I want to simultaneously find the eigenvectors of $M$ that correspond to the largest $|v_i|$, along with finding the largest $v_i$.  Preferably without finding the entire spectra of $M$ first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some possibilities that I have been thinking about:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can &quot;inflate&quot; the matrix using the opposite of &quot;Wieldant's Deflation&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M_1 = M + \sigma \left[ \Sigma_i v_i \hat{m}_i \right] V^H  = M + \sigma V V^H$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigenvalues for different $\hat{m}_i$ are shifted $\lambda_i + \sigma |v_i|^2$.  I believe we can then extract $\sigma$ and $v_i$ because the eigenvectors don't change.  The problem is that the outer product of $V$ is dense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;another possibility:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The power method (keep multiplying $M$ by our vector $V$ until the convergence) finds the component of $V$ with the largest eigenvalue.  The downside of this method is that we don't control for the magnitude of $v_i$, so we would end up finding ALL the components, and then finding the largest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some way to control this so that we only converge on the largest component?&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2014-12-15T21:19:37.943" LastActivityDate="2015-04-09T00:02:09.760" Title="calculating eigenvector components of a given vector" Tags="|sparse-matrix|eigensystem|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17466" PostTypeId="1" CreationDate="2014-12-12T01:38:09.770" Score="1" ViewCount="1444" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that I am duplicating the question on scicomp. I have already asked this in math.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to come up with a scheme in Polar Co-ordinates for the following PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PDE I am trying to solve&lt;/strong&gt;&#xA;$$  u_t - (u_{rr} + \frac{1}{r} * u_r + \frac{1}{\theta} * u_{\theta\theta} + bu) = f(r,\theta, t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am extending the ideas from the book [1] which is already given for the PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PDE in [1]&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{rr} + \frac{1}{r} * u_r + \frac{1}{\theta} * u_{\theta\theta} = f(r,\theta, t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book discusses a particular scheme in pages 333-334 to solve the above PDE. I am listing the scheme below for convenience and its notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scheme proposed for the above PDE in [1]&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{r_i} (r_{i+1/2} \frac{u_{i+1j} - u_{ij}}{\Delta r} - r_{i-1/2}\frac{u_{ij} - u_{i-1j}}{\Delta r}) \frac{1}{\Delta r} + \frac{1}{r_i^2} \frac{u_{ij+1} - 2u_{ij} + u_{ij-1}}{\Delta \theta^2} = f_{ij}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{ij}$ and $f_{ij}$ are the grid functions at $(r_i, \theta_j) = (i\Delta r, j\Delta \theta)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, to avoid difficulties arising at the origin because of the $\frac{1}{r}$ terms in the scheme, the book goes on to integrate the PDE over a small disc (D) of radius $\epsilon$ near the origin $r=0$, applies the Gauss-Divergence Theorem to get the following result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\iint_D f \;r\;dr\;d\theta =  \iint_D (u_{rr} + \frac{1}{r} * u_r + \frac{1}{\theta} * u_{\theta\theta}) \; \;r\;dr\;d\theta = \iint_D \nabla. (\nabla u) \; \;r\;dr\;d\theta  = \int_{0}^{2\pi} \frac{\partial u}{\partial r} \;\epsilon\;d\theta$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are later approximated as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(0) * \epsilon^2 * \pi = \sum_{j=1}^{J} \frac{u_{1j}-u_0}{\Delta r}  \epsilon\; \Delta \theta$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From which it solves for $u_0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: $J\Delta \theta = 2 \pi$ and $I\Delta r = R$ where R: radius of the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i) How do I extend the idea for the terms &lt;em&gt;$u_t$&lt;/em&gt;, &lt;em&gt;$bu$&lt;/em&gt; terms above when computing an approximation for &lt;em&gt;$u_0$&lt;/em&gt; for my PDE?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I propose to treat &lt;em&gt;$bu$&lt;/em&gt; term where &lt;em&gt;$b$&lt;/em&gt; is a constant the same way as &lt;em&gt;$f$&lt;/em&gt; is treated in the approximation above. But it is unclear to me as to what I need to do for &lt;em&gt;$u_t$&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated. Even pointers to read a particular book/topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;:&#xA;[1] Elliptic Partial Differential Equations and Difference Schemes&#xA;Finite Difference Schemes and Partial Differential Equations, Second Edition. 2004, 311-338 &lt;/p&gt;&#xA;" OwnerUserId="11828" LastActivityDate="2014-12-12T01:38:09.770" Title="Finite Difference in Polar Co-ordinates" Tags="|pde|finite-difference|boundary-conditions|parabolic-pde|elliptic-pde|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="17467" PostTypeId="2" ParentId="17465" CreationDate="2014-12-12T03:32:12.610" Score="0" Body="&lt;p&gt;Sure. Let $\hat{M}$ be a matrix such that each column is a distinct eigenvector of the Hermitian operator $M$; it follows from the Hermiticity of $M$ that $\hat{M}$ is invertible. To obtain the $v_{i}$, solve the linear system $\hat{M}v = V$. Then, select the index $i$ such that $\max_{j}|v_{j}| = |v_{i}|$; the $i$th column of $\hat{M}$ will the eigenvector corresponding to the largest $v_{i}$ in magnitude.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-12T03:32:12.610" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17469" PostTypeId="1" CreationDate="2014-12-12T16:33:09.070" Score="2" ViewCount="336" Body="&lt;p&gt;I am trying to implement a second order DG-method using a monomial basis and explicit Euler in time. I have written down some of the theory, which I present below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theory&lt;/strong&gt;&#xA;Consider the linear advection function over $\Omega\subset\mathbb{R}$ and a class of testfunctions $v\in C_0^1(\Omega)$&#xA;$$&#xA;	\int_\Omega\partial_tuvd x+a\int_\Omega\partial_xuvd x=0\;.\hspace{70pt}(1)&#xA;$$&#xA;Now, divide the computational domain $\Omega=\cup_{i=1}^N\Omega_i$ where $\Omega_i$ are disjoint, and let $h_i=x_{i+1}-x_{i}$ be the cell length. Integration by parts yield of $(1)$ yields&#xA;$$&#xA;	\int_{\Omega_i}\partial_tuv d x-a\int_{\Omega_i}uv' d x+\left[auv\right]_{x_i}^{x_{i+1}}=0\;.\hspace{30pt}(2)&#xA;$$&#xA;The exact solution $u$ is now be approximated with $U_i$ over the cell $\Omega_i$ in a finite dimensional subset of $C_0^1(\Omega_i)$. Then the approximation of $u$ in cell $\Omega_i$ can be written&#xA;$$&#xA;	U_i=\sum_{j=0}^pc_{ij}\phi_j&#xA;$$&#xA;where $c_{ij}$ are time-depentent coefficients. In order to get a Galerkin method the basis functions and the testfunctions should be in the same space. For simplicity let this be the space of polynomials of degree $p$, i.e. $U_i,v\in\Pi_p$. For $v=\phi_k$, with $k=0,\ldots,p$, inserting into $(2)$ yields&#xA;$$&#xA;	\int_{\Omega_i}\partial_t\sum_{j=0}^pc_{ij}\phi_j\phi_k dx-a\int_{\Omega_i}\sum_{j=0}^pc_{ij}\phi_j\phi_k' d x+\left[a\sum_{j=0}^pc_{ij}\phi_j\phi_k\right]_{x_i}^{x_{i+1}}=0\;.\hspace{10pt}(3)&#xA;$$&#xA;Let us define a suitable inner product on $\Omega_i$ following the $L^2$-space inner product definition&#xA;$$&#xA;	(u,v)=\int_{x_L}^{x_R}uvd x\;.&#xA;$$&#xA;Due to the smoothness properties of the testfunctions, the derivatives  and the sums can be moved out from under the integral sign, and we can rewrite $(3)$ as&#xA;$$&#xA;	\sum_{j=0}^p\left[\partial_tc_{ij}(\phi_j,\phi_k)-ac_{ij}(\phi_j,\phi_k')\right]+\left[a\sum_{j=0}^pc_{ij}\phi_j\phi_k\right]_{x_i}^{x_{i+1}}=0\;.\hspace{30pt}(4)&#xA;$$&#xA;Let us now make some simplifications. First let $h=h_i$ for all cells, i.e. an equidistant grid. Secondly, restrict $p=1$ with the monomial basis&#xA;$$&#xA;	\phi_0=1,\qquad\phi_1=\frac{x-x_L}{x_R-x_L}&#xA;$$&#xA;giving&#xA;$$&#xA;	(\phi_0,\phi_0)=h,\quad(\phi_0,\phi_1)=\frac{h}{2},\quad(\phi_1,\phi_1)=\frac{h}{3},\\\quad(\phi_0,\phi_0')=0,\quad(\phi_0,\phi_1')=1,&#xA;\quad(\phi_1,\phi_1')=\frac{1}{2}\;.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are now ready to write this as a system&#xA;$$&#xA;	\partial_tMc_i-aM^fc_i+\left[M^s\overline{f}^N\right]_{x_i}^{x_{i+1}}=0&#xA;$$&#xA;where $M=(m_{jk})$ where $m_{jk}=(\phi_j,\phi_k)$ and $M^f=(m^f_{jk})$ where $m^f_{jk}=(\phi_j,\phi_k')$ and&#xA;$$&#xA;	M^s=\begin{pmatrix}\phi_0(x)&amp;amp;0\\0&amp;amp;\phi_1(x)\end{pmatrix},\qquad\overline{f}^N=\begin{pmatrix}f^N(x)\\f^N(x)\end{pmatrix},\qquad c_i=\begin{pmatrix}c_{i,0}(t)\\c_{i,1}(t)\end{pmatrix}&#xA;$$&#xA;for some numerical flux function $f^N$. We will use the upwind flux, i.e.&#xA;$$&#xA;	f^N(U_{i-1},U_i)=aU_{i-1}(x_{i-1/2})&#xA;$$&#xA;The last step to complete the scheme is to apply the explicit Euler method in time. First rewrite  as&#xA;$$&#xA;	\partial_t c_i=aM^{-1}M^fc_i-M^{-1}\left[M^s\overline{f}^N\right]_{x_i}^{x_{i+1}}&#xA;$$&#xA;We get&#xA;$$&#xA;	c_i^{n+1}=c_i^n+\Delta t\left(aM^{-1}M^fc_i^n-M^{-1}\left[M^s\overline{f}^N\right]_{x_i}^{x_{i+1}}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Implementaiton&lt;/strong&gt; The following is in Matlab&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we have an initial pulse $f(x)=e^{-x^2}$ over $[-4,4]$. Then I extract the initial coefficient using&#xA;$$&#xA;u(x_L)=c_0\qquad u(x_R)=c_0+c_1&#xA;$$&#xA;i.e. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;V =[1 0;-1 1];       &#xA;u=zeros(2, Ncells);&#xA;u(:)=f(x(:));&#xA;cini = V*u;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $x$ is a matrix containing the the end points of each cell. Then I run the above, but the solution diverge quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&#xA;Does the theory seem okay, or are there any flaws? I can't figure out where it goes wrong. I can live with it if the implementation is a little off, but as long as I am not trying to program something that won't work it would be a relief. &lt;/p&gt;&#xA;" OwnerUserId="11873" LastActivityDate="2014-12-13T10:02:34.793" Title="Problem with implementing linear advection using DG-method" Tags="|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17470" PostTypeId="2" ParentId="16317" CreationDate="2014-12-12T22:57:03.060" Score="2" Body="&lt;p&gt;It's actually n(log2n), not log2n (comparison sorts are all constrained to nlogn performance). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looks like you're missing a division by two. Starting from the top (sorry for not knowing laTeX),&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T(n) =  2 T(n/2) + n&#xA;&#xA;     =  2 [2 T(n/4) + n/2] + n&#xA;&#xA;     =  4 T(n/4) + 2n&#xA;&#xA;     = 4 [2 T(n/8) + n/4] + 2n&#xA;&#xA;     = 8 T(n/8) + 3n&#xA;&#xA;     = 2^k T(n/2^k) + kn &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From your observation that n/2^k = 1 we can multiply each side by 2^k, take the log, and find log2 n = k. Inserting this into where we were before,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T(n) = 2^(log2 n)*T(1) + (log2n)*n&#xA;&#xA;     = n*T(1) + (log2n)*n&#xA;&#xA;     = n + (log2n)*n&#xA;&#xA;     = (log2n)*n&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="13066" LastActivityDate="2014-12-12T22:57:03.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17473" PostTypeId="2" ParentId="17469" CreationDate="2014-12-13T04:23:17.070" Score="1" Body="&lt;p&gt;Just glancing through it quickly, it seems that you are using a second-order discretization in space with forward Euler in time.  That is unstable for any fixed CFL number (on fine enough grids).  You should use, e.g., a higher-order Runge-Kutta method.  It is possible to  check the stability by computing the eigenvalues of your semi-discretization and using method of lines stability analysis.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2014-12-13T10:02:34.793" LastActivityDate="2014-12-13T10:02:34.793" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17474" PostTypeId="1" AcceptedAnswerId="17479" CreationDate="2014-12-13T12:18:15.267" Score="0" ViewCount="317" Body="&lt;p&gt;I am interested in solving the problem &lt;span class=&quot;math-container&quot;&gt;$-A u = \lambda u$&lt;/span&gt; on a finite differences grid on a square. In my case, the operator &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is of the type &lt;span class=&quot;math-container&quot;&gt;$-\Delta + \mu I$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is there to impose some constraints. The method I used until now was to consider a finite differences grid and get&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ 4u_{i,j} - u_{i,j+1}-u_{i,j-1}-u_{i+1,j}-u_{i-1,j} + \mu_{i,j} u_{i,j}=\lambda u_{i,j}$$&lt;/span&gt;&#xA;Then we transform &lt;span class=&quot;math-container&quot;&gt;$u_{i,j}$&lt;/span&gt; in a vector and write the above problem in matrix form &lt;span class=&quot;math-container&quot;&gt;$Au = \lambda u$&lt;/span&gt;. The problem I have with this is that the size of the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of points in the grid. Therefore, for a grid with &lt;span class=&quot;math-container&quot;&gt;$N^2$&lt;/span&gt; points the dimension of the matrix is &lt;span class=&quot;math-container&quot;&gt;$N^2 \times N^2$&lt;/span&gt;. Therefore, the size of the matrices grows very fast with respect to the discretization.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is it possible to solve this kind of problem without vectorization, i.e. using only matrices of size &lt;span class=&quot;math-container&quot;&gt;$N \times N$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;My idea is the following: If we denote &lt;span class=&quot;math-container&quot;&gt;$U=(u_{i,j})$&lt;/span&gt; then the discrete PDE can be written as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ (4I -S_{0,1}-S_{0,-1}-S_{1,0}-S_{-1,0})U+\mu \odot U=\lambda U,$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$S_{i,j}$&lt;/span&gt; is the shift of the matrix with &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; on horizontal and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; on vertical directions. The &lt;span class=&quot;math-container&quot;&gt;$\odot$&lt;/span&gt; is &amp;quot;pointwise&amp;quot; product. Can this lead to a solution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-05-31T21:33:00.943" Title="Solve eigenvalue problem using finite differences without vectorization" Tags="|finite-difference|matrix|eigenvalues|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17477" PostTypeId="1" AcceptedAnswerId="17526" CreationDate="2014-12-13T15:22:13.343" Score="6" ViewCount="935" Body="&lt;p&gt;Can someone give some references to understand what's the differences between a component-wise and a characteristic-wise ENO scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm right, the characteristic variables come from the diagonalization of flux matrix but I don't see how it comes to play when doing ENO and what's the advantage of it.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="1567" LastEditDate="2014-12-22T21:02:00.173" LastActivityDate="2014-12-22T21:02:00.173" Title="ENO/WENO component-wise vs characteristic-wise" Tags="|fluid-dynamics|reference-request|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17478" PostTypeId="1" AcceptedAnswerId="17482" CreationDate="2014-12-13T15:50:45.623" Score="4" ViewCount="3551" Body="&lt;p&gt;I have asked a question in this regard earlier. I am trying to solve the following equation in Polar Co-ordinates:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  u_t - (u_{rr} + \frac{1}{r} * u_r + \frac{1}{\theta} * u_{\theta\theta} + bu) = f(r,\theta, t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am building a finite difference method to tackle the above pde. I have assumed the solution to be of the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(r,\theta, t) = e^{-t} r^2 * Sin(2\theta)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, $\frac{\partial u}{\partial r}|_{r=0} = 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how do I impose this? I have seen some documents that assume that the solution is axially symmetric at $r=0$ and hence only solve along the radial line while assuming a phantom node to the left of the origin, $u_{-1}$ and finally eliminates that using the FD scheme. For me, even though  $\frac{\partial u}{\partial r}|_{r=0}$, my assumed solution is not axially symmetric (about z-axis).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone use this image as an example to explain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PMhxv.png&quot; alt=&quot;Polar Discretization&quot; title=&quot;tooltip&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="11828" LastActivityDate="2014-12-14T13:40:01.420" Title="Neumann Boundary Condition at r=0 in Polar Coordinates (Numerical BCs)" Tags="|pde|finite-difference|boundary-conditions|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17479" PostTypeId="2" ParentId="17474" CreationDate="2014-12-13T16:52:33.283" Score="4" Body="&lt;p&gt;For sure it is possible with iterative methods, because you can apply your operator $u\mapsto Au$ cheaply while keeping $u$ in &quot;matrix form&quot;. It is simply a double nested for cycle. Check out &lt;a href=&quot;https://en.wikipedia.org/wiki/ARPACK&quot; rel=&quot;nofollow&quot;&gt;ARPACK&lt;/a&gt;, or more in general Arnoldi-(or Krylov-)type methods: in most implementations, you can use as input a 'black-box' routine that computes $Au$ from $u$, rather than an explicit matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods are best suited to computing only a small section of the spectrum (for instance, the largest or smallest eigenvalues and their corresponding eigenvectors), and nowadays they are the standard method for solving large and sparse eigenproblems numerically.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-12-13T16:52:33.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17480" PostTypeId="2" ParentId="17478" CreationDate="2014-12-13T22:21:42.750" Score="4" Body="&lt;p&gt;An idea is not to discretise at the centre (r=0). You put points in pairs near the centre such that each pair forms a line through the centre.  Then the finite difference stencils at these points just go through the centre; exactly like in your figure except that $u_0$ should not be put.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way, you do not need a boundary condition at $r=0$; the latter is hardly physical but only due to the polar coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, FEM in polar coordinates encounters a difficulty; see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/580/how-to-do-fem-in-sector-elements&quot;&gt;How to do FEM in sector elements?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2014-12-13T22:21:42.750" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17482" PostTypeId="2" ParentId="17478" CreationDate="2014-12-14T13:40:01.420" Score="4" Body="&lt;p&gt;There are no boundary conditions at $r=0$. Boundary conditions are things you can impose at (a part of) the boundary. In other words, there is freedom: you could choose different values for the boundary values and get different solutions. But in the case of cylindrical or spherical coordinate systems, you cannot impose $u(r=0,\theta)=\text{something}$ or $\frac{\partial u(r=0,\theta)}{\partial r}=\text{something}$. In the first case, this is because for elliptic partial differential equations (and many other equations as well) the solution is only in $H^1$ and taking point values is not well defined. In the latter case it is because you cannot impose anything other than a zero value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the condition $\frac{\partial u(r=0,\theta)}{\partial r}=0$ is not a boundary condition. It is a &lt;em&gt;compatibility condition&lt;/em&gt; that the solution has to satisfy, but not something you can impose. To be correct, you better choose a numerical scheme that also satisfies this compatibility condition.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-14T13:40:01.420" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="17483" PostTypeId="1" CreationDate="2014-12-14T15:20:28.543" Score="2" ViewCount="295" Body="&lt;p&gt;I have implemented a FVM code and now I need to plot the accuracy of the method as the mesh gets refined. Having a very fine mesh, my idea is to compare what is the error between the coarser and fine mesh in the L1-norm. My problem is that having a finer mesh the solution vector has more entries than the solution with the coarse grid, therefore, how to compare the two?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2015-03-13T02:03:58.680" Title="Compute accuracy order as mesh gets refined?" Tags="|numerical-analysis|error-estimation|accuracy|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="17484" PostTypeId="2" ParentId="8899" CreationDate="2014-12-14T22:02:33.963" Score="2" Body="&lt;p&gt;For my personal need, I tried to isolate the minimum computation for a 2x2 svd. I guess it is probably one of the simplest and fastest solution. You can find details on my personal blog : &lt;a href=&quot;http://lucidarme.me/?p=4624&quot; rel=&quot;nofollow&quot;&gt;http://lucidarme.me/?p=4624&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Advantages :&lt;/strong&gt; simple, fast and you can only calculate one or two of the three matrices (S, U or D) if you don't need the three matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Drawback&lt;/strong&gt; it uses atan2, which may be inacurate and may require an external library (typ. math.h).&lt;/p&gt;&#xA;" OwnerUserId="13079" LastEditorUserId="-1" LastEditDate="2014-12-16T21:36:06.397" LastActivityDate="2014-12-16T21:36:06.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17485" PostTypeId="2" ParentId="17483" CreationDate="2014-12-15T06:45:40.907" Score="1" Body="&lt;p&gt;If you interpret the $L^{1}$ norm as a function norm (i.e., an integral), then increasing the number of solution values will be offset by a decrease in the mesh width. The result should be comparable values in norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, if your solution is $f$, taking on values $f(x_{i})$ for $i = 1, \ldots, N$, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\|f\|_{1} = \int_{0}^{L}|f(x)|\,\mathrm{d}x \approx h\sum_{i = 1}^{N}|f(x_{i})|,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;assuming:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;your mesh is uniform (otherwise, put $h$ inside the summation and replace it with $x_{i+1/2} - x_{i-1/2}$, assuming the discretization points are at cell centers)&lt;/li&gt;&#xA;&lt;li&gt;the domain of the PDE is $[0,L]$ (without loss of generality).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-15T06:45:40.907" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17487" PostTypeId="2" ParentId="16308" CreationDate="2014-12-15T12:53:29.180" Score="1" Body="&lt;p&gt;Consider the product rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$[\mathbf{\bar {u}}.\triangledown \mathbf{\bar {v}}]_{k}= u_{m}\dfrac{\partial v_{k}}{\partial x_{m}}=&#xA;\dfrac{\partial u_{m} v_{k}}{\partial x_{m}} - v_{k} \dfrac{\partial u_{m}}{\partial x_{m}} =&#xA;[\triangledown.(\mathbf{\bar {u}} \mathbf{\bar {v}}) - \mathbf{\bar {v}}(\triangledown.\mathbf{\bar {u}}]_{k}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;writing the inner product of the basis vector $\mathbf{\bar {v}}$ with the second term in LHS of the Navier Stokes equation in indicial notation and using the product rule we obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{\bar {v}}.(\mathbf{\bar {u}}. \triangledown\mathbf{\bar {u}}) = v_{k} u_{m} \dfrac{\partial u_{k}}{\partial x_{m}}&#xA;= \dfrac{\partial v_{k} u_{m} u_{k}}{\partial x_{m}} - u_{k} \dfrac{\partial v_{k} u_{m}}{\partial x_{m}} = &#xA; \triangledown.(\mathbf{\bar {v}}.\mathbf{\bar {u}} \mathbf{\bar {u}}) - &#xA;  \mathbf{\bar {u}}.\triangledown.(\mathbf{\bar {v}} \mathbf{\bar {u}})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{ \Omega_{j}} \mathbf{\bar {v}}.(\mathbf{\bar {u}}. \triangledown\mathbf{\bar {u}})dV = &#xA;   \int_{ \Omega_{j}} \triangledown.(\mathbf{\bar {v}}.\mathbf{\bar {u}} \mathbf{\bar {u}})dV -&#xA;   \int_{ \Omega_{j}}  \mathbf{\bar {u}}.\triangledown.(\mathbf{\bar {v}} \mathbf{\bar {u}})dV&#xA; $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using Gauss's theorem,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{ \Omega_{j}} \mathbf{\bar {v}}.(\mathbf{\bar {u}}. \triangledown\mathbf{\bar {u}})dV = &#xA;  \int_{ \partial \Omega_{j}}  \mathbf{\bar {n}}. \mathbf{\bar {v}}. \mathbf{\bar {u}}  \mathbf{\bar {u}} dS -&#xA; \int_{ \Omega_{j}}  \mathbf{\bar {u}}.\triangledown.(\mathbf{\bar {v}} \mathbf{\bar {u}})dV$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{\bar {n}}. \mathbf{\bar {v}}. \mathbf{\bar {u}}  \mathbf{\bar {u}} =&#xA;  \mathbf{\bar {u}}.\mathbf{\bar {n}} \mathbf{\bar {u}}. \mathbf{\bar {v}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be proved using  basis notation (not shown here because is tedious)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;proceeding in the same way to the second term in RHS of the NS equation we obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{\bar {v}}.\triangledown. \mathbf{\bar{\bar{\tau}}} = v_{k}\dfrac{\tau_{mk}}{\partial x_{m}} =&#xA;\dfrac{\partial v_{k} \tau_{mk}}{\partial x_{m}} - \tau_{mk}\dfrac{\partial v_{k}}{\partial xm} =&#xA;\triangledown. (\mathbf{\bar {v}}.\mathbf{\bar{\bar{\tau}}}) - \mathbf{\bar{\bar{\tau}}}:\triangledown \mathbf{\bar {v}}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \int_{ \Omega_{j}}  \mathbf{\bar {v}}. \mathbf{\bar{\bar{\tau}}}dV = -  \int_{ \Omega_{j}} \mathbf{\bar{\bar{\tau}}}:&#xA;\triangledown  \mathbf{\bar {v}}dV + \int_{ \partial \Omega_{j}}  \mathbf{\bar {n}}. \mathbf{\bar {v}}.&#xA;\mathbf{\bar{\bar{\tau}}}dS$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\mathbf{\bar {n}}. \mathbf{\bar {v}}.\mathbf{\bar{\bar{\tau}}} =  \mathbf{\bar {v}} \mathbf{\bar {n}}:&#xA;\mathbf{\bar{\bar{\tau}}}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{\bar {n}}. \mathbf{\bar {v}}.\mathbf{\bar{\bar{\tau}}} = n_{i} \mathbf{\bar {e}_{i}}.&#xA;  v_{j} \mathbf{\bar {e}_{j}}. \tau_{mk} \mathbf{\bar {e}_{m}} \mathbf{\bar {e}_{k}} =&#xA;  n_{i} \mathbf{\bar {e}_{i}}.v_{j}\tau_{mk}\mathbf{\bar {e}_{j}}.\mathbf{\bar {e}_{m}} \mathbf{\bar {e}_{k}}=&#xA;   n_{i} \mathbf{\bar {e}_{i}}.v_{j}\tau_{mk}\delta_{jm}\mathbf{\bar {e}_{k}}=&#xA;   n_{i} \mathbf{\bar {e}_{i}}.v_{m}\tau_{mk}\mathbf{\bar {e}_{k}} =&#xA;   n_{i}v_{m}\tau_{mk}\mathbf{\bar {e}_{i}}.\mathbf{\bar {e}_{k}}=&#xA;   n_{i}v_{m}\tau_{mk}\delta_{ik}= n_{k}v_{m}\tau_{mk}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \mathbf{\bar {v}} \mathbf{\bar {n}}:\mathbf{\bar{\bar{\tau}}} = v_{m}n_{k}\tau_{mk}$&lt;/p&gt;&#xA;" OwnerUserId="9713" LastActivityDate="2014-12-15T12:53:29.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17488" PostTypeId="1" AcceptedAnswerId="17489" CreationDate="2014-12-15T14:26:30.883" Score="6" ViewCount="1061" Body="&lt;p&gt;I want to solve a nonlinear optimization problem of the following form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\min\left(\sum_i d^{x_i}c_{i}\right)\\&#xA;0 \leq x_{i} \leq a\\&#xA;\sum_{i} x_{i} \leq b &#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0.95 &amp;lt; d &amp;lt; 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c_i &amp;gt; 1$&lt;/span&gt; are constant. The problem has a very specific structure, and my question is whether using a very general nonlinear optimization software (e.g. NLopt, Ipopt) is the best way to solve it numerically. The second issue is that the number of variables (&lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;) is very large (around one million). Does the standard algorithms for nonlinear optimization can handle this large number of variables? &lt;/p&gt;&#xA;" OwnerUserId="1090" LastEditorUserId="23061" LastEditDate="2020-04-13T09:52:06.547" LastActivityDate="2020-04-13T09:52:06.547" Title="Large-scale nonlinear optimization problem" Tags="|nonlinear-equations|nonlinear-programming|" AnswerCount="4" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17489" PostTypeId="2" ParentId="17488" CreationDate="2014-12-15T19:47:23.073" Score="4" Body="&lt;p&gt;At 1 million variables, you're sort of on the cusp of what a code like (the publicly released version of) IPOPT can do. IPOPT will solve the KKT system derived from an interior point method using direct methods (usually, it's an $LDL^{T}$ factorization using something like Bunch-Parlett). Memory is typically the limiting resource in these algorithms for large-scale problems. I doubt NLopt would be better; interior-point methods are supposed to be good for large-scale problems, and SQP algorithms (such as SLSQP in NLopt) are not supposed to perform as well. Method-of-moving-asymptote methods have been used in topology optimization and are supposed to be geared towards solving large-scale problems, but I don't have much experience with those methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of greater concern might be scaling; even $(1.05)^{200} \approx 10^{10}$. It's something to look out for, and not necessarily a show-stopper.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2014-12-15T22:13:28.283" LastActivityDate="2014-12-15T22:13:28.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17490" PostTypeId="2" ParentId="17488" CreationDate="2014-12-16T02:20:10.740" Score="1" Body="&lt;p&gt;An important issue here is whether this is a convex optimization problem or not.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your objective function is a weighted sum-exp function (since $d$ and the $c_{i}$ are fixed, you can use the change of base formula for logs to write this as $\sum \hat{c}_{i} \exp(x_{i})$.  Assuming that the $\hat{c}_{i}$ are nonnegative, you've got a convex $f(x)$.  Unfortunately, you're trying to maximize, which is the wrong direction.  If by some chance all of your $c_{i}$ coefficients were negative, then you'd be maximizing a concave function which is a good case.  If the $c_{i}$ coefficients have mixed signs you're generally going to have a nonconvex objective function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Research has been done on the related problem of minimizing convex log-sum-exp functions (note that $\log$ is a monotone transformation of your objective function) which might prove helpful if your optimization problem is indeed convex.  Even simpler would be to use a projected gradient method.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if your problem is non-convex, then you're in a world of hurt.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-12-16T02:20:10.740" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17491" PostTypeId="2" ParentId="11768" CreationDate="2014-12-16T04:47:35.773" Score="1" Body="&lt;p&gt;I would just add a comment to the answer by AJK, but I don't have enough rep.  There is a classic paper by Liron and Mochon (&lt;a href=&quot;http://link.springer.com/article/10.1007/BF01535565&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/article/10.1007/BF01535565&lt;/a&gt;) from the late 1970's which gives a series solution for the Green's function for Stokes flow between two parallel plates.&lt;/p&gt;&#xA;" OwnerUserId="13082" LastActivityDate="2014-12-16T04:47:35.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17492" PostTypeId="1" AcceptedAnswerId="17495" CreationDate="2014-12-16T05:08:44.210" Score="4" ViewCount="119" Body="&lt;p&gt;I'm trying to solve a Stokes flow problem with a pseudo-spectral method in periodic boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations of interest are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-\nabla^2 \bf{v} + \nabla p = \bf{f} \\&#xA;\nabla \cdot \bf{v} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\bf{f}$ is a force vector coming from the gradient of a stress tensor, $\bf{f} = \nabla \cdot \sigma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do so, I use a transverse projection operator (i.e. the Green's function/Oseen tensor) in Fourier space as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bf{\tilde{v}} = \bf{\tilde{T}} \cdot \bf{\tilde{f}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\bf{\tilde{T}}$ is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bf{\tilde{T}} = \frac{1}{k^2} \left [ \bf{I} - \frac{\bf{k}\bf{k}}{k^2} \right ]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: What is the appropriate way to treat the zero k-mode? In the statement above the solution is undefined for $k = 0$. I think I should be able to obtain the volume averaged velocity from this mode, and I can't think off-hand why it shouldn't be well-defined. (Note this problem is similar to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11768/solving-stokes-flow-with-walls-using-oseen-tensor/17491#17491&quot;&gt;this related question&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="13082" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-12-16T18:00:07.330" Title="Zero-k mode in Pseudo-spectral solution of Stokes Flow" Tags="|fluid-dynamics|fourier-analysis|spectral-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17493" PostTypeId="1" CreationDate="2014-12-16T11:50:16.450" Score="2" ViewCount="328" Body="&lt;p&gt;There has been quite a flutter recently in the iterative world about an algorithm that speeds up the classical Jacobi method by as much as 200 times using a scheduled relaxation method where a combination of over and under relaxation is employed. The results as shown in &lt;a href=&quot;http://engineering.jhu.edu/fsag/wp-content/uploads/sites/23/2013/10/JCP_revised_WebPost.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; are remarkable and are indicative of a resuscitation of Jacobi method as a method for practical purposes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I implemented it I cannot get it to work. I was wondering if anybody else in this community has tried implementing this method and if so was the attempt successful. It would be interesting to hear if there is anything special that needs to be taken care of when implementing this algorithm. &lt;/p&gt;&#xA;" OwnerUserId="5357" LastEditorUserId="5357" LastEditDate="2014-12-16T12:06:50.230" LastActivityDate="2014-12-16T12:06:50.230" Title="Speeding up the classical Jacobi method using Scheduled Relaxation method?" Tags="|performance|iterative-method|" AnswerCount="0" CommentCount="8" ClosedDate="2014-12-16T20:33:45.117" ContentLicense="CC BY-SA 3.0" />
  <row Id="17494" PostTypeId="1" CreationDate="2014-12-16T13:52:14.350" Score="1" ViewCount="1127" Body="&lt;p&gt;&lt;code&gt;Ply&lt;/code&gt; file formats is used to represent mesh structures in the TETGEN freeware package. All the examples I've seen of 'ply' meshes were used for triangular meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we use ply formats to represent &lt;strong&gt;&lt;code&gt;tetrahedral meshes&lt;/code&gt;&lt;/strong&gt; as well ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any examples would be helpful ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PS :&lt;/strong&gt; I am new in this field  so any pointers for further understanding of the concepts would be really helpful. I am trying to simulate tissue deformation using FEM.&lt;/p&gt;&#xA;" OwnerUserId="13073" LastEditorUserId="549" LastEditDate="2014-12-25T15:50:41.463" LastActivityDate="2015-08-01T11:57:39.490" Title="Tetrahedral Mesh file format in TETGEN" Tags="|mesh-generation|mesh|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="17495" PostTypeId="2" ParentId="17492" CreationDate="2014-12-16T18:00:07.330" Score="2" Body="&lt;p&gt;You are correct that the $k=0$ mode corresponds to the mean (volume averaged) velocity in the domain. Because the equations for Stokes flow are Galilean invariant they are undetermined up to a constant (the mean flow). Usually this ambiguity is resolved by the boundary conditions, but purely periodic boundary conditions offer no such resolution. Instead, you must prescribe the desired mean velocity. Most often this is zero. In fact, all choices of mean velocity are equivalent since they are related by a simple change from one inertial reference frame to another.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2014-12-16T18:00:07.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17496" PostTypeId="2" ParentId="17483" CreationDate="2014-12-16T18:56:10.680" Score="1" Body="&lt;p&gt;If you believe that your mesh size is in the asymptotic regime, you might try Richardson error estimation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, the idea is that you model your error as $O(h^p)$ for some unknown $p$ and assume that your other terms beyond that order are negligibly small.  Then, for three different grid sizes you have the three approximations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{h}\approx f_* + ch^p$$&#xA;$$f_{2h}\approx f_* + c(2h)^p$$&#xA;$$f_{4h}\approx f_* + c(4h)^p$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where here $f_h$ and $f_{2h}$ $f_{4h}$ are the solutions on the fine, medium, and coarser grids.  With some algebra, you get $f_{h} - f_{2h} \approx c(h^p-(2h)^p)$ and similarly for the other difference so you can solve for $p$ and $c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To take the difference between functions of grids on difference sizes, just use restriction -- $f_h$ is sampled on $0,1,2,3,4,5,6$, $f_{2h}$ is sampled on $0,2,4,6$, so evaluate at every other point.  Choose whatever norm you want to estimate the error for.&lt;/p&gt;&#xA;" OwnerUserId="10109" LastActivityDate="2014-12-16T18:56:10.680" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17497" PostTypeId="1" CreationDate="2014-12-16T19:52:18.487" Score="2" ViewCount="427" Body="&lt;p&gt;I'm looking to numerically compute the solution to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ k(x,u) \partial_t u - \Delta u = f \quad\quad\text{ in } \Omega \times [0,T]$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k$ is a continuous but nonlinear (in $u$) real-valued function and $\Omega \subseteq \mathbb{R}^2$. I want to use the Finite Element Method, but I cannot seem to manage the nonlinear $k$. I wrote the code myself and it already works for $k = const$, i.e. in the &quot;standard case&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is, that I cannot get rid of the $k(u)$ factor and therefore don't reach a formula like &#xA;$$ M \dot{u} - A u  =  F $$&#xA;like in &lt;a href=&quot;http://www.ima.umn.edu/~arnold//8445.f11/notes.pdf&quot; rel=&quot;nofollow&quot;&gt;page 80 here&lt;/a&gt; (mass matrix $M$, stiffness matrix $A$). I also can't seem to  reach any formulation which would allow me to apply the Newton method or something similar. Is it even possible to use a FEM approach in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or reference is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="13087" LastEditorUserId="13087" LastEditDate="2014-12-17T08:19:07.740" LastActivityDate="2014-12-18T08:00:01.567" Title="FEM for a nonlinear parabolic PDE" Tags="|pde|finite-element|parabolic-pde|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17498" PostTypeId="1" CreationDate="2014-12-16T22:47:47.547" Score="2" ViewCount="173" Body="&lt;p&gt;I am learning Monte Carlo simulation by C++. I begin with reading codes (from the internet and text books) of the 2D Ising model and the XY model. I find some people define spins simply by a two dimensional array &lt;code&gt;int s[i][j]&lt;/code&gt;, however, some people use vectors like &lt;code&gt;int *s&lt;/code&gt; or &lt;code&gt;int **s&lt;/code&gt; together with &lt;code&gt;new int&lt;/code&gt;. The body of the code are also a bit different when allocating the value to spins, and I think the case only works with a two dimensional array is more readable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be a quite naive to ask, but can anyone help me with what are advantages to use vector and &lt;code&gt;new int&lt;/code&gt; type variable? References are also warmly welcome.&lt;/p&gt;&#xA;" OwnerUserId="10350" LastEditorUserId="7733" LastEditDate="2014-12-17T03:35:52.053" LastActivityDate="2014-12-17T03:35:52.053" Title="What are the tradeoffs of using statically allocated arrays vs. pointers and dynamic allocation?" Tags="|c++|monte-carlo|floating-point|" AnswerCount="1" CommentCount="3" ClosedDate="2014-12-17T03:56:07.170" ContentLicense="CC BY-SA 3.0" />
  <row Id="17499" PostTypeId="2" ParentId="17498" CreationDate="2014-12-16T23:32:57.880" Score="3" Body="&lt;p&gt;Arrays allocated with brackets such as int s[i][j] are fixed to be of size i x j, whereas arrays allocated with new and int *s / int **s are not necessarily uniformly deep and can be resized with subsequent calls to new. If you're solving on a fixed grid, there isn't any need for being able to resize the array, so the two options above will have similar performance. You can find more details here: &lt;a href=&quot;https://stackoverflow.com/questions/17775066/c-c-performance-of-static-arrays-vs-dynamic-arrays&quot;&gt;https://stackoverflow.com/questions/17775066/c-c-performance-of-static-arrays-vs-dynamic-arrays&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third option is to use std::vector. Std:vectors have a few nice features, like throwing out-of-bounds exceptions when you use at to try to access a non-existent index. C-style arrays, on the other hand, will just try to make sense of whatever happens to be at the memory address they're given. That said, working with nested vectors for multiple dimensions can get ugly.&lt;/p&gt;&#xA;" OwnerUserId="13066" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-12-16T23:32:57.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17501" PostTypeId="2" ParentId="17497" CreationDate="2014-12-17T08:33:58.697" Score="2" Body="&lt;p&gt;As a first shot, I suggest you use &lt;em&gt;Rothe&lt;/em&gt;'s method (time discretization first), rather than the method of lines (space discretization first).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you employ &lt;em&gt;implicit Euler&lt;/em&gt; for that, at every time step $l$, you will have to solve the system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-\tau \Delta u^l+k(u^l)u^l = k(u^{l-1})u^{l-1} + \tau [ \Delta u^{l-1} + f^l],&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is a semilinear elliptic problem. Then you can apply, e.g., Newton or fixed point iteration.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-12-17T08:33:58.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17504" PostTypeId="1" AcceptedAnswerId="17506" CreationDate="2014-12-17T20:44:04.600" Score="14" ViewCount="18785" Body="&lt;p&gt;This might be a question better suited for the Software Recommendations side of S.E., however I do believe that people who frequent this part of S.E. are more likely to be able to answer this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a free (not only as in freedom) &lt;strong&gt;alternative to Comsol Multiphysics&lt;/strong&gt;. Here is the tricky bit: I am not simply looking for a modelling and simulation package, of which there are loads, rather I am looking for a free solution that has as similar syntax to Comsol as possible. Maybe there is some package you can run together with Octave? If so, I haven't found it. Any help will be greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit]&#xA;I would need the software for numerical modelling and simulation. Fluids flowing between various containers, heat conduction, ect. Simulating solutions to various PDE's in short. The main functionality I am looking for the other software to parrot is Comsols Model Wizard.   &lt;/p&gt;&#xA;" OwnerUserId="13098" LastEditorUserId="13098" LastEditDate="2014-12-18T08:39:58.833" LastActivityDate="2021-05-17T12:39:23.133" Title="Alternatives to Comsol Multiphysics" Tags="|finite-element|comsol|octave|openmodelica|modeling|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17505" PostTypeId="2" ParentId="17497" CreationDate="2014-12-18T08:00:01.567" Score="2" Body="&lt;p&gt;You will have to iterate out the problem. After discretization, you get a problem of the form &#xA;$$&#xA;  M(U^n) U^n + \Delta t \; A U^n = F^n(U^{n-1})&#xA;$$&#xA;where the mass matrix depends on the solution $U^n$ of the n-th time step. This nonlinear system of equations has to be solved by an iteration, e.g., the Newton method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example for a different question, but using the same idea, is given here:&#xA;  &lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/step_15.html&quot; rel=&quot;nofollow&quot;&gt;https://www.dealii.org/developer/doxygen/deal.II/step_15.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-18T08:00:01.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17506" PostTypeId="2" ParentId="17504" CreationDate="2014-12-18T10:28:14.150" Score="13" Body="&lt;p&gt;Not sure if you find the COMSOL Model Wizard somewhere else, maybe other commercial Multi-physics software but not in the open-source community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had the same question a couple of years ago and I listed all Finite-element, Multi-physics framework. As you may know there are many of them. The one that I found really useful and close, &lt;em&gt;at least in the way that they represent PDE equation&lt;/em&gt; was &lt;a href=&quot;http://fenicsproject.org&quot;&gt;FeniCS&lt;/a&gt;. I even partially implemented the part of the model that I had in COMSOL to see if I can replace COMSOL with it. They have nice Python interface which makes things easier.&lt;/p&gt;&#xA;" OwnerUserId="1622" LastActivityDate="2014-12-18T10:28:14.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17507" PostTypeId="1" CreationDate="2014-12-18T14:43:19.057" Score="1" ViewCount="434" Body="&lt;p&gt;Do you know what numerical software computes an eigenvector basis for a unitary matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I have a unitary matrix $U$. If its eigenvalues are simple (no multiplicities), then for instance Matlab computes an eigenbasis for $U$. However, if there are some eigenvalues with multiplicities, in the subspace for the eigenvalues with multiplicity the software does not find independent eigenvectors. If a matrix is symmetric or Hermitian, Matlab is programmed to output an eigenbasis (even if there are eigenvalues with multiplicities). No such thing for unitary matrices - as fas as I know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a way to avoid this: if $λ$ is an eigenvalue with multiplicity, then I can form the matrix $B=A−λ⋅1$ and find the nullity of $B$. The only problem is that doing this for each possible eigenvalue is slow. I wonder if there is a better solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that makes a difference, I can assume that my unitary matrix is real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question was also asked here &lt;a href=&quot;https://cstheory.stackexchange.com/q/27874/&quot;&gt;https://cstheory.stackexchange.com/q/27874/&lt;/a&gt; and here &lt;a href=&quot;https://stackoverflow.com/q/27533637/&quot;&gt;https://stackoverflow.com/q/27533637/&lt;/a&gt; (no answer).&lt;/p&gt;&#xA;" OwnerUserId="13104" OwnerDisplayName="costelus" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2014-12-18T23:46:03.857" Title="Numerical eigenbasis for a unitary matrix" Tags="|linear-algebra|eigensystem|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="17510" PostTypeId="2" ParentId="17507" CreationDate="2014-12-18T23:46:03.857" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Say I have a unitary matrix $U$. If its eigenvalues are simple (no multiplicities), then for instance Matlab computes an eigenbasis for $U$. However, if there are some eigenvalues with multiplicities, in the subspace for the eigenvalues with multiplicity the software does not find independent eigenvectors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Unitary_matrix#Properties&quot; rel=&quot;nofollow&quot;&gt;Unitary matrices are diagonalizable over the complex numbers&lt;/a&gt;, so you should be getting an eigenbasis from &lt;code&gt;eig&lt;/code&gt;. Furthermore, the eigenbasis will be a unitary matrix, and all eigenvalues will be on the unit circle.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If a matrix is symmetric or Hermitian, Matlab is programmed to output an eigenbasis (even if there are eigenvalues with multiplicities).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Many algorithms take advantage of symmetry (or in the complex case, Hermiticity). Symmetric and Hermitian matrices are also diagonalizable by unitary matrices; Hermitian matrices also have real eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-18T23:46:03.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17512" PostTypeId="1" AcceptedAnswerId="17513" CreationDate="2014-12-19T02:17:35.727" Score="2" ViewCount="3017" Body="&lt;p&gt;This is a follow up to this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16446/solving-a-nonlinear-bvp-using-finite-difference-method&quot;&gt;question&lt;/a&gt;. &#xA;I have a nonlinear BVP on $x=0$ to $L$:&lt;br&gt;&#xA;$$&#xA;(T^2)\frac{\partial^2 T}{\partial x^2} + T \left( \frac{\partial T}{\partial x}\right)^2 + Q = 0&#xA;$$&lt;br&gt;&#xA;to which I apply a &lt;strong&gt;finite difference&lt;/strong&gt; discretization. The parameter $Q$ is a constant.There is a Dirichlet boundary condition at $x = 0$ and Robin at $x = L$: &#xA;$$&#xA;(T^2)\frac{\partial T_L}{\partial x} = h(T-T_{\infty})&#xA;$$&#xA;Where $h$ is a physical constant and $T_{\infty}$ is the ambient temperature. My approach is similar to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/15851/error-implementing-robin-boundary-conditions-in-toy-ode-problem&quot;&gt;this&lt;/a&gt; using the ghost cell method. I sub-divide the grid into $N$ grid points with subscript indices $0,\ldots,N$ and discretize the Robin bc as follows:&#xA;$$&#xA;(T_N^2)\frac{T_{N+1} - T_{N-1}}{2\Delta x} = h(T_N - T_{\infty})&#xA;$$&#xA;where $T_{N+1}$ extends beyond the grid to the &quot;ghost cell.&quot; Re-arranging:&#xA;$$&#xA;T_{N+1} = \frac{2\Delta xh}{T_N^2}(T_N-T_{\infty}) + T_{N-1}&#xA;$$&#xA;Discretization of the BVP at grid point $N$ (right boundary, $x = L$) yields a nonlinear algebraic equation in function of $T_{N-1},\, T_N,\,T_{N+1}$. I can eliminate $T_{N+1}$ using the discretized Robin bc.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can use Newton iteration to solve this system and it will work. The equation, however, is complicated and likewise is the Jacobian. Making a mistake when computing the derivatives or even programming seems likely. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;My Question:&lt;/strong&gt; Is there a better or more commonly used way of handling this Robin boundary condition than the ghost cell method (like some sort or linearization of the bc)? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help. &lt;/p&gt;&#xA;" OwnerUserId="10256" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-12-19T08:14:09.910" Title="Implement Robin Boundary Condition" Tags="|finite-difference|boundary-conditions|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17513" PostTypeId="2" ParentId="17512" CreationDate="2014-12-19T08:14:09.910" Score="2" Body="&lt;p&gt;I don't know that it's &quot;better&quot;, but you could use one-sided boundary conditions instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Variants of Newton's method (e.g., Newton's method, quasi-Newton methods) will take the nonlinear boundary condition and solve a linearized version at each (Newton nonlinear solve) iteration. In that sense, the boundary condition will be linearized; cruder linearizations about a nominal point will probably be less accurate unless the nonlinearity is extremely weak. For example, in your problem, if $|h| \ll 1$ is very small, then you could effectively treat your Robin condition as a Neumann condition.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The equation, however, is complicated and likewise is the Jacobian. Making a mistake when computing the derivatives or even programming seems likely.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Working with indices is definitely challenging, and it is easy to make mistakes. Consider using a computer algebra system (e.g., Sage, SymPy, Mathematica, Maple) to calculate derivatives and limit math errors. In interpreted languages, take advantage of language features that allow you to minimize using indices (for instance, in MATLAB, using arrays and slicing to vectorize your code); writing your code this way sometimes has the added benefit of making it faster (in MATLAB and Python with NumPy, for instance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I feel that learning to work with indices is an important skill you have to develop. If you have to program in a low-level language, working with indices is unavoidable. In further studies of computational mathematics, indicial notation is not only common, but expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is a single PDE in 1-D with cubic nonlinearities. Computational science is rife with more complicated PDEs (e.g., multi-dimensional, coupled systems of PDEs, higher-order or transcendental nonlinearities). It's worth practicing on simpler problems like this one, and learning methods to manage the bookkeeping. It's also worth learning about the method of manufactured solutions so that you can test your discretization and be more confident that you obtain the proper solution and the proper order of convergence of your discretization method. If you test your methods, you can detect programming and derivation errors more easily, and tackle more complicated problems with confidence.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-19T08:14:09.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17514" PostTypeId="1" AcceptedAnswerId="17516" CreationDate="2014-12-19T18:07:39.883" Score="4" ViewCount="4563" Body="&lt;p&gt;I am solving the following PDE;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^2 u = \frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} = \rho,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\rho(0.5,0.5) = 2$ (zero elsewhere), $0\leq x,y\leq1$ and the boundaries will satisfy the condition $u = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving by finite differences using both a 5 point second order stencil and a 9 point fourth order stencil. It is my understanding that the solution to this PDE is the zero function and this is what I am using to calculate rates of convergence. I have used various error measures including the average difference, max difference and the value at the point $(0.5,0.5)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each error measure I am getting convergence rates of approximately order 2 for both stencils, is there a reason why the fourth order stencil would not converge with order 4? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a repeat of a question I asked on MSE - &lt;a href=&quot;https://math.stackexchange.com/questions/1073945/the-rate-of-convergence-for-finite-difference-methods-for-poissons-equation-wit/1074255#1074255&quot;&gt;https://math.stackexchange.com/questions/1073945/the-rate-of-convergence-for-finite-difference-methods-for-poissons-equation-wit/1074255#1074255&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the answers there have helped a lot I would appreciate anyone who could elaborate a little further? &lt;/p&gt;&#xA;" OwnerUserId="13110" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2014-12-19T20:45:21.657" Title="The rate of convergence for finite difference methods for Poisson's equation with piecewise constant data" Tags="|pde|convergence|poisson|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17515" PostTypeId="2" ParentId="17514" CreationDate="2014-12-19T18:15:12.147" Score="0" Body="&lt;p&gt;Think about what &quot;second order&quot; and &quot;fourth order&quot; approximations actually mean. A first order method can exactly reproduce a first order (linear) function, a second order can reproduce a quadratic, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So say if the exact solution to a given problem is linear - if so, a 100th order method won't give you any benefit over a 1st order method, because both can exactly represent the solution. I suspect this might be a similar case with your zero function solution. Try something with a sinusoidal forcing term or boundary condition.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2014-12-19T18:15:12.147" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="17516" PostTypeId="2" ParentId="17514" CreationDate="2014-12-19T20:16:09.587" Score="8" Body="&lt;p&gt;Let's examine the one-dimensional three-point stencil case in detail,&#xA;because I think it's important to be clear just how this behaviour&#xA;arises, and what it means to set a point to a certain value in a&#xA;finite-difference grid when the underlying function is&#xA;discontinuous. The equation will be&#xA;$$ u''(x) = \rho(x). $$&#xA;Instead of using the interval $[-1,1]$ with discontinuity at&#xA;$\frac12$, I will use the interval $[-\frac12,\frac12]$ with the&#xA;discontinuity placed at $0$. The grid size will be $h$, and I will only have to consider the interval $[-\frac h2,\frac h2]$ around the grid point $0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, in a finite-difference approximation, we approximate&#xA;$$ u''(0) = \frac{\hat u(-h) - 2\hat u(0) + \hat u(h)}{h^2} $$&#xA;and solve&#xA;$$ \frac{\hat u(-h) - 2\hat u(0) + \hat u(h)}{h^2} = \hat \rho(0). $$&#xA;(Here the variables with hats are the numerical approximations on the&#xA;grid to the variables without hats.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this is a very bad specification of the problem, because your&#xA;function $\rho(0) = 2$ at $x=0$, and $\rho(x) = 0$ everywhere else, is&#xA;discontinuous. In particular, if we shift the grid point $0$ to either&#xA;side by some tiny amount $\epsilon$, the numerical solution changes&#xA;entirely and becomes exactly zero.&#xA;This means that this is a terribly misspecified problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can make sense of it by&#xA;converting it to an equivalent finite-volume formulation, where it&#xA;will make much more sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a finite-volume method, we solve&#xA;$$ \int_{-h/2}^{h/2} u''(x)\,dx = \int_{-h/2}^{h/2}\rho(x)\,dx, $$&#xA;by picking $u(x)$ to be a suitable approximation to the unknown&#xA;function. Let's pick, on the interval $[-\frac h2,\frac h2]$ the approximation&#xA;$$ u(x) =  \hat u(-h) \phi(x+h) + \hat u(0) \phi(x) + \hat u(h) \phi(x-h), $$&#xA;where $\phi(x)$ is the basis function&#xA;$$ \phi(x) = \max\left(1-\frac{|x|}{h}, 0\right), $$&#xA;(it's a piecewise linear function that goes from $0$ at $-h$ to $1$ at&#xA;$0$ to $0$ at $+h$, thus interpolating between grid points.). The approximation $u(x)$ is a weighted sum of three basis functions that look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OvcSx.png&quot; alt=&quot;Basis functions&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can then compute&#xA;$$ \phi''(x) = \frac1h\delta(h-|x|) - \frac2h \delta(x), $$&#xA;so that the approximation to the integral is&#xA;$$ \int_{-h/2}^{h/2} u''(x) = \frac{\hat u(-h) -2\hat u(0)+\hat&#xA;  u(h)}{h}, $$&#xA;and the finite-volume approximation to our equation becomes&#xA;$$ \frac{\hat u(-h) -2\hat u(0) + \hat u(h)}{h} = \int_{-h/2}^{h/2}&#xA;\rho(x)\,dx = h \hat \rho(0). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two important things here. First, this is &lt;em&gt;equivalent&lt;/em&gt; to&#xA;the finite-difference formulation in that we end up solving the same&#xA;equations. Second, the discontinuity in $\rho$ is given a very precise&#xA;meaning: when we use the value $2$ for $\hat \rho(0)$, we are saying&#xA;that this is the average value of $\rho$ on the interval $[-h/2,h/2]$:&#xA;$$2 = \hat\rho(0) = \frac1h  \int_{-h/2}^{h/2} \rho(x)\,dx.$$&#xA;This interpretation is not available in the finite-difference&#xA;formulation. It is also not so sensitive to the location of the discontinuity: if the discontinuity were at some small distance $\epsilon$ away from $0$, the average value would be almost the same, but the value at $0$ might be completely different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if we say that $2$ is the average value of $\rho$ near the grid&#xA;point, we can then go back and compute the exact solution of the&#xA;equation with the right-hand side given by&#xA;$$ \tilde \rho(x) = 2[-h/2 &amp;lt; x &amp;lt; h/2]. $$&#xA;(We pick a function of our own choice that gives the right average.)&#xA;In this case, the exact solution will be&#xA;$$ \tilde u(x) = 2 \int_{-h/2}^{h/2} G(x; u)\,du \approx 2h G(x;&#xA;0), $$&#xA;in terms of the Green function for the Poisson equation. In the two-dimensional case it will be&#xA;$$ \approx 2h^2 G(x, y; 0, 0), $$&#xA;as in the other (correct) answer on MSE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the outcome of all this is that when you say that you compare&#xA;your numerical approximate solution with the exact solution $u(x)=0$,&#xA;this is wrong. The exact solution should not be zero, it should be&#xA;$$ \approx 2h^2 G(x, y; 0,0). $$&#xA;It therefore should make perfect sense that the fourth-order solution does not converge to zero with order $4$: it &lt;em&gt;should&lt;/em&gt; converge to the correct solution, which is not zero but has magnitude of order $O(h^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you &lt;em&gt;do&lt;/em&gt; want to get zero as the numerical solution and compare with the mathematical solution, you should set $\hat\rho(0)$ to be the average value of $\rho$, which is $0$, not $2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that the exact solution depends on the chosen grid size indicates that this is not a good way to check whether you implemented the method correctly. A very straightforward technique known as the method of manufactured solutions is better for this.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-12-19T20:45:21.657" LastActivityDate="2014-12-19T20:45:21.657" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17517" PostTypeId="1" AcceptedAnswerId="17519" CreationDate="2014-12-19T23:00:42.520" Score="8" ViewCount="224" Body="&lt;p&gt;I was wondering whether there is some theorem that allows me to put an upper bound on the error introduced by omitting small matrix elements from a matrix before diagonalization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume we have a large matrix, whose matrix elements range from $1$ to $10^{-15}$. If I were to set all matrix elements smaller than $10^{-10}$ to $0$ before diagonalizing the matrix, how large would the error be in the eigenvalues and eigenvectors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this implementation dependent?&lt;/p&gt;&#xA;" OwnerUserId="1544" LastActivityDate="2014-12-21T19:59:58.270" Title="diagonalization of matrix - omitting small matrix elements" Tags="|eigensystem|error-estimation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17518" PostTypeId="2" ParentId="17517" CreationDate="2014-12-19T23:24:29.947" Score="10" Body="&lt;p&gt;It's not implementation-dependent in the sense that this is a&#xA;mathematical operation performed on your matrix. However, it is very&#xA;much &lt;em&gt;matrix&lt;/em&gt;-dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix is diagonalizable and $A=XDX^{-1}$, then zeroing out&#xA;some element adds a small perturbation matrix $E$, so the new&#xA;eigenvalues will be (assuming the matrix $X$ does not change much)&#xA;$$ X^{-1}(A+E)X = D + X^{-1}EX. $$&#xA;The difference in $D$ has norm bounded by&#xA;$$ \|E\| \kappa(X), $$&#xA;where $\|E\|$ depends on the magnitude of the elements you drop, and&#xA;$\kappa(X)$ is the condition number of the matrix of eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix $A$ is&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Normal_matrix&quot; rel=&quot;nofollow&quot;&gt;normal&lt;/a&gt; (i.e.,&#xA;$AA^t=A^tA$), then $X$ is unitary and $\kappa(X)=1$, so this is fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrix is not normal, you need the concept of its&#xA;pseudospectrum, defined as the set&#xA;$$ \Lambda_\epsilon(A) = \{z \mid z \text{ is an eigenvalue of $A+E$&#xA;  with $\|E\| \leq \epsilon$} \}. $$&#xA;An equivalent definition, which is easier to compute with, is&#xA;$$ \Lambda_\epsilon(A) = \{z \mid \|(zI-A)^{-1}\|\geq \epsilon^{-1} \}. $$&#xA;There is a good survey of pseudospectra and their properties in&#xA;&lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/publication/PDF/1992_54.pdf&quot; rel=&quot;nofollow&quot;&gt;Pseudospectra of Matrices by Trefethen&lt;/a&gt; (it includes a nice gallery of matrices whose pseudospectra are much larger than their spectra: one might think that the pseudospectrum should be a collection of small $\epsilon$-sized disks around the eigenvalues, but that is really not right at all). In general, you cannot just&#xA;assume that the pseudospectrum is well-behaved and that the&#xA;perturbations are negligible. You can, however, compute $\kappa(X)$ and estimate the resulting error; you can also compute the pseudospectrum directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some sense, dropping small matrix elements is fine: either they don't matter, and the new results are just as accurate as the original, or they do matter, and your original results are just as inaccurate as the new results.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2014-12-21T19:59:58.270" LastActivityDate="2014-12-21T19:59:58.270" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17519" PostTypeId="2" ParentId="17517" CreationDate="2014-12-20T10:06:59.943" Score="6" Body="&lt;p&gt;There is a field of study known as &lt;em&gt;eigenvalue sensitivity analysis&lt;/em&gt; or &lt;em&gt;eigenvalue perturbation analysis&lt;/em&gt; that allows you to estimate the effect of small matrix perturbations on the eigenvalues and eigenvectors. The basic technique used for this is differentiating the eigenvalue matrix equation,&#xA;$$AX = X\Lambda.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For situations where the eigenvalues of the original matrix are all distinct, the following document has a very clear derivation and results:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Mike Giles. &quot;An extended collection of matrix derivative results for forward and reverse mode algorithmic differentiation&quot;. &lt;a href=&quot;https://people.maths.ox.ac.uk/gilesm/files/NA-08-01.pdf&quot;&gt;https://people.maths.ox.ac.uk/gilesm/files/NA-08-01.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When the eigenvalues are not distinct, some more care must be taken. See the following &lt;a href=&quot;http://www.win.tue.nl/casa/meetings/seminar/previous/_abstract051019_files/Presentation.pdf&quot;&gt;presentation&lt;/a&gt; and &lt;a href=&quot;http://alexandria.tue.nl/repository/books/616489.pdf&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the special case of symmetric matrices $A=U \Lambda U^T$ with distinct eigenvalues, subject to a small perturbation $A \rightarrow A + dA$, the results are simple enough that I reproduce them here. The derivative of the eigenvalue matrix is,&#xA;$$d\Lambda = \text{diag} (U^T dA U),$$&#xA;and the derivative of the eigenvector matrix is,&#xA;$$dU = UC(dA),$$&#xA;where the coefficient matrix $C$ is defined as,&#xA;$$C = \begin{cases}&#xA;\frac{u_i^T dA u_j}{\lambda_j - \lambda_i}, &amp;amp; i=j \\&#xA;0, &amp;amp;i=j&#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following paper by Overton and Womersley has a great sensitivity analysis for the symmetric case, including second derivatives.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Overton, Michael L., and Robert S. Womersley. &quot;Second derivatives for optimizing eigenvalues of symmetric matrices.&quot; SIAM Journal on Matrix Analysis and Applications 16.3 (1995): 697-718. &lt;a href=&quot;http://ftp.cs.nyu.edu/cs/faculty/overton/papers/pdffiles/eighess.pdf&quot;&gt;http://ftp.cs.nyu.edu/cs/faculty/overton/papers/pdffiles/eighess.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2014-12-20T10:06:59.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17522" PostTypeId="1" CreationDate="2014-12-21T03:44:44.003" Score="2" ViewCount="72" Body="&lt;p&gt;Consider the diffusion of some unknown $R$ on a &lt;em&gt;physically&lt;/em&gt; discrete, 1-D ring:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    x---------x---------x---------x    &#xA;n = 0         1         2         3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We let grid point $n=0$ be the same as grid point $n=3$, giving us our ring. We are given a diffusion coefficient $D$, which is still multiplied by some to some notion of the &quot;Laplacian&quot; (which in 1-D, is just the second order difference) of $R$ in order to tell us how $R$ moves diffuses between neighbouring nodes &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can use techniques such as 2nd order central differences to calculate some notion of second order difference of $R$ on these physically discrete points, if we consider the numerical discretization of the grid to be the same as the physical discretization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if the physical discretization is not uniform? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    x----------------x--x-------x    &#xA;n = 0                1  2       3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What notion of &quot;second order difference&quot; in $R$ might I now consider given that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I can't simply consider the physical discretization to be the same as the numerical discretization, without complicating things (non-uniform numerical grid spacing);&lt;/li&gt;&#xA;&lt;li&gt;but more importantly, I don't have any continuous physical situation to go back in order to derive a different second order differencing scheme using Taylor series, so I can't use some finer uniform numerical grid, which also happens to coincide with the physical points -- it would make no sense to have a numerical grid point that doesn't coincide with a physical grid point. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is my only solution to make some sort of assumption regarding the &quot;continuous&quot; nature of $R$ between the grid points? Maybe I assume that $R$ varies linearly between the physically discrete points? Then the problem I outline in point 2. isn't there, and I can define a finer numerical grid? This is the solution advocated in a similar question here: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/480/how-can-i-numerically-differentiate-an-unevenly-sampled-function?rq=1&quot;&gt;How can I numerically differentiate an unevenly sampled function?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other ideas might I consider? &lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2014-12-21T04:10:05.557" Title="2nd order differences for modelling a physically discrete phenomenon?" Tags="|finite-difference|discretization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17523" PostTypeId="1" CreationDate="2014-12-21T08:47:19.787" Score="3" ViewCount="82" Body="&lt;p&gt;The wave equation,&#xA;$$\ddot{u} = c^2 \Delta u,$$&#xA; can be generalized to abstract graphs by using the negative &lt;a href=&quot;http://en.wikipedia.org/wiki/Laplacian_matrix&quot; rel=&quot;nofollow&quot;&gt;graph Laplacian&lt;/a&gt; in place of the physical Laplacian. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a graph-theoretic analog of Perfectly Matched Layers (PML), where &quot;outward traveling waves&quot; exit the graph without reflection? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not clear to me what the definition of &quot;outward traveling waves&quot; should be in the graph context, but it does seem intuitively reasonable to consider nonresonant waves propagating through a small region of interest (subgraph) embedded in a much larger sparse graph.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2014-12-21T08:59:51.333" LastActivityDate="2014-12-21T08:59:51.333" Title="Absorbing BC's / PML on a graph" Tags="|boundary-conditions|graph-theory|wave-propagation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17525" PostTypeId="2" ParentId="7837" CreationDate="2014-12-22T05:14:52.143" Score="2" Body="&lt;p&gt;Trilinos provides an incomplete Cholesky preconditioner in two packages, AztecOO and Ifpack.  AztecOO provides a pattern-based incomplete factorization using the concept of level-fill.  Ifpack provides both level-fill and threshold-based drop tolerance approaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These implementations are intended as subdomain solvers as part of an additive Schwarz algebraic domain decomposition preconditioner, or as a local smoother for an algebraic multigrid preconditioner.  They can be executed in parallel on a multicore processor, but only if you enable the use of MPI.  In this case, each MPI process is assigned a portion of the matrix equations on which an incomplete Cholesky preconditioner will be used.  If you compile Trilinos with MPI support disabled, you can still execute the incomplete Cholesky preconditioner, but only on a single core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Details for each are here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://trilinos.org/oldsite/packages/aztecoo/AztecOOUserGuide.pdf&quot; rel=&quot;nofollow&quot;&gt;http://trilinos.org/oldsite/packages/aztecoo/AztecOOUserGuide.pdf&lt;/a&gt; (starting about p 16)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://trilinos.org/oldsite/packages/ifpack/IfpackUserGuide.pdf&quot; rel=&quot;nofollow&quot;&gt;http://trilinos.org/oldsite/packages/ifpack/IfpackUserGuide.pdf&lt;/a&gt; (starting about p 15)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using MPI to obtain parallelism on a multicore processor is quite effective, but it does typically require a fairly substantial refactoring of your code if you are running a sequential or OpenMP based code at this time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obtaining good threaded parallel performance, e.g., with OpenMP, from incomplete Cholesky on a typical sparse matrix is challenging.  The factorization phase can obtain reasonable speedups, say 5x on 8 cores (based on my own experience) for large enough problems.  But the solve phase (which you will call for each iteration) is difficult to improve substantially for general sparse matrices.  I have seldom seen more than a factor of 2 improvement, no matter how many cores are used.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Complete sparse Cholesky algorithms have a rich graph theory framework that enables organization of the factorization and solve into multifrontal (task) and supernodal (data) parallelism, and can benefit from using optimized dense BLAS.  Also, most of the computational time for complete sparse Cholesky is spent in the factorization (again easier to parallelize) and the solve is typically only called once (since the factorization is complete).  So threaded complete Cholesky is typically quite effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Incomplete Cholesky does not have these same favorable properties, which is why threaded parallelism typically doesn't translate over to the incomplete factorization situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are already using PETSc, I am fairly certain it has a similar capability.&lt;/p&gt;&#xA;" OwnerUserId="12040" LastActivityDate="2014-12-22T05:14:52.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17526" PostTypeId="2" ParentId="17477" CreationDate="2014-12-22T09:25:20.733" Score="4" Body="&lt;p&gt;In both components-wise and characteristic variable methods, the basic ENO formulation remains the same. &#xA;In this answer, Only finite volume formulation is considered. however, FD is similar in philosophy and can can be studied from the references. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;note: $i$ is the cell under consideration. $j$ is the generic index for cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Implementation Details&lt;/strong&gt; [1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Component-wise ENO&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the most straightforward procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The procedure is similar as that for a scalar 1d equation. We have a vector of conserved variables $\bar u_{n \times 1} = [u_1, u_2, ... ,u_n]^T$ and the flux vector $f_{n \times 1} = f(\bar u)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We carry out ENO reconstruction for each component of $\bar u$ (lets call it simply $u$) &lt;em&gt;separately&lt;/em&gt;. This is not a &lt;em&gt;true decoupling&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives us left and right values for $u$ ($u_l $ and $u_r$) at $x_{i+\frac{1}{2}}$. Doing this for all components gives us $\bar u_l$ and $\bar u_r$ at $x_{i+\frac{1}{2}}$. Then flux $f$ at $x_{i+\frac{1}{2}}$ is found out by solving the Riemann problem at $x_{i+\frac{1}{2}}$. Either exact or approximate Riemann solver can be used for this. Some of the hyperbolic systems, including Euler's equations, have exact Riemann solutions. A &lt;em&gt;low order&lt;/em&gt; solution is more sensitive towards choice of Riemann solver. Hence we can use more cost effective approximate solvers at high orders (however, we don't use component method for high orders!).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the fluxes are formed, we can advance in time using appropriate time integrator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar approach is taken for WENO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Characteristic-wise ENO&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;a. Linear system with constant coefficient matrix $f'$:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the same system of equations.&#xA;Considering it to be a (strictly) hyperbolic system, the Jacobian $f'$ (or coefficient matrix $A$ as in some cases), has $n$ distinct eigenvalues ($\lambda_j(\bar u); j=1,...,n$) and corresponding right ($r_j(\bar u); j=1,...,n$) and left eigenvectors ($l_j(\bar u); j=1,...,n$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;let $R = [r_1, r_2, ... , r_n]$ and $\Lambda = diag(\lambda_1, \lambda_2,...,\lambda_n)$ be constant everywhere. Then we can diagonalize the Jacobian matrix using $R, R^{-1}$ and $\Lambda$.&lt;br&gt;&#xA;i.e. we get, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bar v_t + \Lambda \bar v_x = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, $\bar v$ is the characteristic variable vector. &#xA;Now each component of $\bar v$ is &lt;em&gt;truely&lt;/em&gt; decoupled and this is nothing but $n$ separate hyperbolic equations. We can then do the ENO procedure on each one to get $v$ at a particular $x_{i+\frac{1}{2}}$. After all the components are treated in this fashion, we get $\bar v$ at $i+\frac{1}{2}$. After this, we can recover $\bar u$ from $\bar v$ by using $\bar u_{i+\frac{1}{2}} = R \bar v_{i+\frac{1}{2}}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;b. Non linear system or variable coefficient matrix $f'$:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problem with this type is that, $R, R^{-1}$ and $\Lambda$ change with u. So we need to freeze them in space at $x_{j+\frac{1}{2}}$. This is done by taking arithmatic or Roe or $somefancy$ average or $\bar u_j$ and $\bar u_{j+1}$ at $j+\frac{1}{2}$. Then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R_{j+\frac{1}{2}} = R(\bar u_{j+\frac{1}{2}}) ; \forall j$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can get characteric  variable $v_{j+\frac{1}{2}}$ from $u_{j+\frac{1}{2}}$ using this $R$. In other words, the transformation into the characteristic variable is &lt;em&gt;local&lt;/em&gt;. Then perform scalar ENO reconstruction on all $v$ and obtain values at $i+\frac{1}{2}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rest of the procedure remains the same as discussed earlier,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transform back to $u$ at $i+\frac{1}{2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve Riemann problem $\rightarrow$ Find flux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Time integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why and when to use component method:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Straightforward and Really simple to use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We need to perform less number of operations. Works well for many problems especially if order of accuracy is small (2 or maybe 3 in some cases). Suitable for simple test cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Characteristic method:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More robust.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As &lt;em&gt;Kyle Mandli&lt;/em&gt; very rightly pointed out in his comment,&lt;br&gt;&#xA; since we are truly decomposing the variables, the upwind fluxes will be more accurate. We are taking into consideration the wave direction and speed. For more demanding test problems and higher accuracies, we should use the characteristic decomposition. Also for highly nonlinear equations, one should use the characteristic decomposition for obtaining a robust solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, you can study the &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999199962071&quot; rel=&quot;nofollow&quot;&gt;FD implementation of WENO to equations of ideal magnetohydrodynamics&lt;/a&gt; [2] (because, it is a very good example to demonstrate the use of the characteristic method. Also you will get to see the FD implementation, which is not discussed here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along with this, the text on Computational Gasdynamics by Culbert Laney discusses about ENO implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Shu, C. W. (1997). Essentially Non-Oscillatory and Weighted Essentially Non-Oscillatory Schemes for Hyperbolic Conservation Laws. ICASE Report, (97 - 65)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Very detailed explanation. In fact I have repeated here most of what is given in this report. &lt;a href=&quot;http://ccam.xmu.edu.cn/teacher/jxqiu/undergraduate/icase-1997-65.pdf&quot; rel=&quot;nofollow&quot;&gt;Here is the link&lt;/a&gt; for the report. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;[2] Jiang, G.-S., &amp;amp; Wu, C. (1999). A High-Order WENO Finite Difference Scheme for the Equations of Ideal Magnetohydrodynamics. Journal of Computational Physics, 150(2), 561–594.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://pyweno.readthedocs.org/en/latest/&quot; rel=&quot;nofollow&quot;&gt;PyWENO&lt;/a&gt; can be used for obtaining the coefficients easily. &lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="1567" LastEditDate="2014-12-22T09:37:22.853" LastActivityDate="2014-12-22T09:37:22.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17527" PostTypeId="2" ParentId="2267" CreationDate="2014-12-22T10:15:25.447" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://the-foam-house5.webnode.es/&quot; rel=&quot;nofollow&quot;&gt;The Foam House&lt;/a&gt; is an excellent website for beginners. Several problems of varying degree of complexity are introduced in step by step manner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;cheers!&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2014-12-22T10:15:25.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17528" PostTypeId="1" AcceptedAnswerId="17529" CreationDate="2014-12-22T15:10:36.437" Score="1" ViewCount="403" Body="&lt;p&gt;I want to minimize the functional of the Blind Deconvolution model as given in: &lt;a href=&quot;https://ieeexplore.ieee.org/document/661187&quot; rel=&quot;nofollow noreferrer&quot;&gt;Total Variation Blind Deconvolution by Chan and Wong&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Their model is given by:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ z = h \ast u + \eta $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$ \ast $&lt;/span&gt; is the convolution operator, &lt;span class=&quot;math-container&quot;&gt;$ h $&lt;/span&gt; is the blurring kernel, &lt;span class=&quot;math-container&quot;&gt;$ u $&lt;/span&gt; is the sharp noiseless image and &lt;span class=&quot;math-container&quot;&gt;$ \eta $&lt;/span&gt; is additive white gaussian noise (AWGN).&lt;/p&gt;&#xA;&lt;p&gt;The functional to be minimized is given by (Assuming the Blurring Kernel is known):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \underset{u}{\min} f \left( u \right ) = \underset{u}{\min} \left \{ \frac{1}{2} {\left \| h \ast u - z \right \|}^{2}_{L_{2}} + \alpha \int \left | \nabla u \right | dx dy \right \} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$ \alpha $&lt;/span&gt; is the smoothing term.&lt;/p&gt;&#xA;&lt;p&gt;Yet, in Blind Deconvolution the Kernel isn't known and the minimization functional is given by:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \underset{u}{\min} f \left( u \right ) = \underset{u}{\min} \left \{ \frac{1}{2} {\left \| h \ast u - z \right \|}^{2}_{L_{2}} + {\alpha}_{1} \int \left | \nabla u \right | dx dy + {\alpha}_{2} \int \left | \nabla h \right | dx dy \right \} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, The Euler Lagrange equations I calculated and given in the article are:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align} \frac{\delta L}{\delta h} &amp;amp; = \left ( u \ast h - z \right ) \ast u \left( -x, -y \right ) - {\alpha}_{2} \nabla \cdot \left( \frac{\nabla h}{\left | \nabla h \right |} \right) \\ \frac{\delta L}{\delta u} &amp;amp; = \left ( u \ast h - z \right ) \ast h \left( -x, -y \right ) - {\alpha}_{1} \nabla \cdot \left( \frac{\nabla u}{\left | \nabla u \right |} \right) \end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I'm not sure about is how can I use it to solve the problem.&lt;br /&gt;&#xA;At the article they suggest the alternating method, namely once solve for &lt;span class=&quot;math-container&quot;&gt;$ h $&lt;/span&gt; and then for &lt;span class=&quot;math-container&quot;&gt;$ u $&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Yet I don't see how to write in in MATLAB code (Or any other pseudo code).&lt;br /&gt;&#xA;It should be some kind of a Gradient Descent step.&lt;/p&gt;&#xA;" OwnerUserId="7951" LastEditorUserId="22284" LastEditDate="2022-05-03T10:17:58.367" LastActivityDate="2022-05-03T10:17:58.367" Title="Minimization of The Blind Deconvolution Functional" Tags="|optimization|pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17529" PostTypeId="2" ParentId="17528" CreationDate="2014-12-22T19:16:43.197" Score="3" Body="&lt;p&gt;The blind deconvolution should be a minimization on both $u$ and $h$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \underset{u,h}{\min} f \left( u, h \right ) = \underset{u,h}{\min} \left \{ \frac{1}{2} {\left \| h \ast u - z \right \|}^{2}_{L_{2}} + {\alpha}_{1} \int \left | \nabla u \right | dx dy + {\alpha}_{2} \int \left | \nabla h \right | dx dy \right \} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard way to do this is to alternately hold one variable fixed and minimize on the other, ie.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{cases} &#xA;      u^{k+1} = \underset{u}{\text{argmin}} f(u,h^k) \\&#xA;      h^{k+1} = \underset{h}{\text{argmin}} f(u^k,h) \\&#xA;\end{cases}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Back when Chan and Wong wrote their paper the way to minimize total variation was to write out the Euler-Lagrange equations which leads to those complicated and slow to solve PDEs. Now there are faster and easier ways to solve each of those subproblems (eg Split-Bregman) which will simplify things for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bit of searching led to me to this 2014 paper: &lt;a href=&quot;http://www.cvg.unibe.ch/dperrone/tvdb/&quot; rel=&quot;nofollow&quot;&gt;http://www.cvg.unibe.ch/dperrone/tvdb/&lt;/a&gt; which has more information as well as publicly-available MATLAB code. I'm not familiar with their work but it's not easy to publish in CVPR so it's probably worth a look...&lt;/p&gt;&#xA;" OwnerUserId="979" LastActivityDate="2014-12-22T19:16:43.197" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17530" PostTypeId="1" AcceptedAnswerId="17536" CreationDate="2014-12-23T06:31:21.827" Score="5" ViewCount="133" Body="&lt;p&gt;It may be that we have a model where the following equation holds for some phenomenon:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(1)\quad x + y + z = T$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Importantly, $T$ is a constant, i.e.: $$(2) \quad \frac{\mathrm{d}T(t)}{\mathrm{d}t} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We may be interested in the evolution of $x$, $y$ and $z$ as governed by their differential equations. If we want to make sure that we maintain condition $(2)$ (not guaranteed for any numerical integration scheme), then we may represent the problem as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (3a)\quad \frac{\mathrm{d}x}{\mathrm{d}t} = f(x, y, z, t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (3b)\quad \frac{\mathrm{d}y}{\mathrm{d}t} = g(x, y, z, t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (3c)\quad \frac{\mathrm{d}z}{\mathrm{d}t} =  \frac{\mathrm{d}T(t)}{\mathrm{d}t}- f(x,y,z,t) - g(x, y, z, t) = - f(x,y,z,t) - g(x, y, z, t)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, some program might calculate the evolution of $x$, $y$ and $z$ by numerical integrating $x$ and $y$, &lt;em&gt;while&lt;/em&gt; using condition $(2)$ to determine $z$ (generally, abd during the calculations for $x$ and $y$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One takeaway I have had from a numerical analysis course I was a part of recently is that &quot;if you're not violating some conservation rule in the phenomenon being modelled, then you have the exact solution&quot; (and numerical solutions are not exact solutions, usually). Thus, even though I might feel nice about saving $(2)$, I know I am losing &lt;em&gt;something&lt;/em&gt; somewhere else -- what is it that I am losing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An idea I have have is that $(3)$ essentially allows a numerical integration scheme to assume that &lt;em&gt;any difference in $T$ is solely made up by $z$, rather than by some combination of $x$, $y$ and $z$&lt;/em&gt;? Or, put differently, any error in the numerical integration scheme in violating $(2)$ is chalked up to $z$, so we may not be getting the &lt;em&gt;actual&lt;/em&gt; dynamics of $z$, but rather, the dynamics of $z$ along with some error based on how $(2)$ is being violated by the numerical scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How might one more precisely explain what the downside of using a DAE system is in terms of &quot;what is lost in approximation&quot;?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="9466" LastEditDate="2014-12-24T17:17:42.277" LastActivityDate="2014-12-24T18:45:00.107" Title="What is lost in terms of approximation, when writing a problem in terms of a Differential Algebraic Equation (DAE) system rather than an ODE system?" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="17531" PostTypeId="2" ParentId="16446" CreationDate="2014-12-23T13:19:08.113" Score="1" Body="&lt;p&gt;Thanks to the help in the comments I solved the problem. I divided the spatial domain into $N$ grid points and discretized the PDE (without any sort of linearization), which resulted in $N-1$ nonlinear algebraic equations (as Geoff suggested) for the temperature at each grid point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I employed Newton's method (again, as suggested by Geoff):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Implement boundary conditions on the $N-1$ equations&lt;/li&gt;&#xA;&lt;li&gt;Develop a physically reasonable initial guess $F(x_0)$&lt;/li&gt;&#xA;&lt;li&gt;Compute the Jacobian of the $N-1$ equations, resulting in an $N-1 \times N-1$ tri-diagonal matrix&lt;/li&gt;&#xA;&lt;li&gt;Perform Newton iteration to solve for $\Delta T$  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The method converged quickly and I am left with the correct temperature distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help with this problem. &lt;/p&gt;&#xA;" OwnerUserId="10256" LastActivityDate="2014-12-23T13:19:08.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17532" PostTypeId="1" AcceptedAnswerId="17533" CreationDate="2014-12-23T20:35:53.763" Score="4" ViewCount="569" Body="&lt;p&gt;Suppose I have a triangulated mesh and I have solved a differential equation problem and have a solution phi. From phi I can construct a vector field, how would I go about computing the gradient of this vector field?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mesh looks something like this. phi is solved for at the center of all of the cells in the picture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xb1Gt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how to handle the spatial discretization for a triangulated lattice. Any references would be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="10225" LastActivityDate="2014-12-28T19:21:52.933" Title="Gradient of a vector on a triangulated lattice" Tags="|numerical-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17533" PostTypeId="2" ParentId="17532" CreationDate="2014-12-23T21:13:37.113" Score="5" Body="&lt;p&gt;You can use a least squares method to calculate the gradient on unstructured meshes.  Here's a presentation on the method. &lt;a href=&quot;https://engineering.purdue.edu/ME608/webpage/l16.pdf&quot; rel=&quot;noreferrer&quot;&gt;least squares gradient&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2014-12-23T21:13:37.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17534" PostTypeId="1" CreationDate="2014-12-23T23:01:57.043" Score="2" ViewCount="376" Body="&lt;p&gt;Trying to solve stationary Navier-Stokes problem for incompressible laminar Newtonian fluid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found a couple solutions for instationary Navier-Stokes equations (like FeniCS examples or CFD Python), but nothing on stationary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand methods are totally so this code couldn't be reused properly. Does anybody know any software which may help?&lt;/p&gt;&#xA;" OwnerUserId="12014" LastActivityDate="2014-12-23T23:01:57.043" Title="Stationary 2D/3D Navier-Stokes source code" Tags="|pde|python|fenics|navier-stokes|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17535" PostTypeId="1" CreationDate="2014-12-24T14:47:56.953" Score="1" ViewCount="373" Body="&lt;p&gt;I decided to solve using finite differences one of the simplest differential equations which has an analytic solution, notably $\frac{du(x)}{dx}=2x$. The equation is first-order ODE with quadratic equation $u(x)=x^{2}$ as its solution. The problem can be categorized as a boundary problem because we know the value of the solution at arbitrary two boundary points (Dirichlet b.c.). I'm looking for the plot of the function between the two boundary points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned, I decided to use finite differences, notably central difference scheme, to solve the equation. However, I quickly discovered that the problem isn't as straightforward as it appears. The coefficient matrix that governs the system of equations appears to be singular Toeplitz matrix of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A = \left(\begin{array}{ccccc} 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ -1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\ 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 1 \\ 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 \end{array}\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for a system of 5 grid points in the domain. In the case of periodic boundary conditions it's also circulant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently this is a well-known problem when CDS scheme is used for first derivatives. Nevertheless, I haven't found a single textbook on finite differences mentioning it. On the contrary, one gets an impression that the CDS scheme for first derivatives is something used on a daily basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know answers to the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What's the most common approach to solving first-order ODE of boundary problems?&lt;/li&gt;&#xA;&lt;li&gt;Is there a way-around the problem of the singularity of a coefficient matrix?&lt;/li&gt;&#xA;&lt;li&gt;Is there a reason why this problem hasn't gotten more attention? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;EDIT: The first and the last question can be probably explained by the fact that this type of equations can be normally solved analytically.&lt;/p&gt;&#xA;" OwnerUserId="13145" LastEditorUserId="13145" LastEditDate="2014-12-25T15:14:41.560" LastActivityDate="2014-12-25T17:21:25.270" Title="Solving first-order ODE with Dirichlet b.c. using CDS -&gt; singular matrix" Tags="|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17536" PostTypeId="2" ParentId="17530" CreationDate="2014-12-24T18:45:00.107" Score="3" Body="&lt;p&gt;Here's a start to answering your question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One takeaway I have had from a numerical analysis course I was a part of recently is that &quot;if you're not violating some conservation rule in the phenomenon being modelled, then you have the exact solution&quot; (and numerical solutions are not exact solutions, usually).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Kirill and JedBrown pointed out situations where conservation doesn't hold; open systems in thermodynamics are another situation. Even in those cases, you don't have the exact solution. Here's a stupidly trivial example that can't be solved exactly on a computer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot{x} = 0, \quad x(0) = 1/3&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One-third is not exactly representable in floating-point arithmetic because it can't be expressed as a finite decimal in base-2. So generally speaking, you should expect any numerical solution to have some amount of error in it (even if that error is small); in this case, the error is going to be in the machine representation of one-third.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An idea I have have is that (3) essentially allows a numerical integration scheme to assume that any difference in $T$ is solely made up by $z$, rather than by some combination of $x$, $y$ and $z$? Or, put differently, any error in the numerical integration scheme in violating (2) is chalked up to $z$, so we may not be getting the actual dynamics of $z$, but rather, the dynamics of $z$ along with some error based on how (2) is being violated by the numerical scheme?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't see why this situation would be true in general, and numerical schemes normally have tolerances so that you could control the error in the numerical solution. In many cases, you can control the error on a per-variable basis, so conceivably, you could place tight tolerances on $y$ and $z$, plus a loose tolerance on $x$, and expect that you &lt;em&gt;might&lt;/em&gt; have more error in $x$ than in $y$ or $z$.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How might one more precisely explain what the downside of using a DAE system is in terms of &quot;what is lost in approximation&quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you pointed out, you can express some physical systems in multiple ways. Any ODE can be expressed as a DAE; you can also take advantage of conservation laws to rewrite your system of differential equations. In general, rewriting an ODE as a DAE (in a trivial way, re-expressing $\dot{y} = f(y)$ as $\dot{y} - f(y) = 0$) and using numerical methods is more expensive, but not always. For instance, if when using sparse direct methods to solve the linear systems for the numerical methods, the Jacobian matrix of the DAE is much more sparse than the Jacobian matrix of the right-hand-side of the ODE, then solving the DAE may be faster. For a while, this approach was advocated in combustion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some ODE methods don't preserve certain types of invariants (e.g., the conservation laws in your example), so the numerical solution might be mostly accurate, but fail to satisfy the invariant in a noticeable way. An example would be something like using a Runge-Kutta method for some Hamiltonian system like orbital dynamics; over short times, your numerical solution will be fine, but over long times, failure to conserve energy will start to introduce significant error in the numerical solution. This shortcoming can be corrected by using symplectic methods (e.g., Verlet), or you could solve the DAE, assuming the DAE is stable.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2014-12-24T18:45:00.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17537" PostTypeId="1" AcceptedAnswerId="17540" CreationDate="2014-12-24T20:43:01.407" Score="0" ViewCount="178" Body="&lt;p&gt;I've never run into a singular matrix before, so bear with me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a complex non-symmetric matrix (about 1000 x 1000) that I know has a couple zero eigenvalues. It isn't guaranteed to be diagonalizable, but when I pass it to ZGEEV, I see no apparent problems. I use all the eigenpairs for later analysis, but I am unsure if I am 'allowed' to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I trust the results of ZGEEV for computing the eigenvalues and eigenvectors when my matrix is singular, or is there a better way to compute these eigenpairs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell, the eigenvectors are not the trivial solutions. Is ZGEEV actually stable for computing Ax = 0?&lt;/p&gt;&#xA;" OwnerUserId="6699" LastEditorUserId="6699" LastEditDate="2014-12-24T21:03:17.850" LastActivityDate="2014-12-25T10:14:55.437" Title="Computing eigenpairs of singular matrix with ZGEEV?" Tags="|eigensystem|eigenvalues|lapack|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17538" PostTypeId="2" ParentId="17537" CreationDate="2014-12-25T01:31:32.343" Score="0" Body="&lt;p&gt;Singularity of the matrix $A$ is required to obtain nontrivial solutions of $Ax=0$. Using a singular matrix with zgeev should be safe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve the homogeneous system, you can use zgeev to compute the right eigenvectors $v$ which correspond to the zero eigenvalues of $A$, since&#xA;$$Av_j=\lambda_j v_j\text{ .}$$&lt;/p&gt;&#xA;" OwnerUserId="13130" LastActivityDate="2014-12-25T01:31:32.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17540" PostTypeId="2" ParentId="17537" CreationDate="2014-12-25T10:14:55.437" Score="2" Body="&lt;p&gt;Not exactly an answer to your question, but if you are trying to compute reliably a basis for the kernel of a matrix (or just its rank), then the preferred method is using an &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow&quot;&gt;SVD&lt;/a&gt;, rather than an eigenvalue decomposition, because the former is &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_stability&quot; rel=&quot;nofollow&quot;&gt;backward stable&lt;/a&gt; and the latter is not for a nonsymmetric matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, one of the issues you will encounter is that eigenvalues that are $0$ in theory are perturbed in practice to small nonzero values such as $10^{-15}$. You will need to &lt;strong&gt;set a threshold&lt;/strong&gt; to tell what is &quot;zero&quot; and what is not. There are no silver-bullet solutions to choose it; in many cases it depends on the matrix size and on the amount of noise that you are expecting to find in your data. This is an intrinsic difficulty, because computing ranks and kernels is a &quot;is this value zero or not&quot; problem, and this kind of problem is inherently non-continuous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from that, using singular matrices in ZGEEV is no problem at all. Maybe you are confused because a zero eigenvalue means trouble when you are solving linear systems. This is not the case for eigenproblems though; zero is a number like all others.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2014-12-25T10:14:55.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17541" PostTypeId="1" CreationDate="2014-12-25T16:01:45.703" Score="1" ViewCount="288" Body="&lt;p&gt;EDIT (26-12-14):In the Belytschko's EFG code, the domain of influence for uniform distributed node can be calculated using the code below; my question is how to calculate xspac and yspac when the nodes are irregularly distributed?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% DETERMINE DOMAINS OF INFLUENCE - UNIFORM NODAL SPACING&#xA;dmax=3.5;&#xA;xspac = Lb/ndivl;&#xA;yspac = D/ndivw;&#xA;dm = zeros(2,numnod);&#xA;dm(1,1:numnod)=dmax*xspac*ones(1,numnod);&#xA;dm(2,1:numnod)=dmax*yspac*ones(1,numnod);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;kindly help with idea on how /or source code to determine the influence domain for a domain in 2D or 3D with irregularly distributed nodes in EFG or any other meshfree method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please refer to the concept proposed by G.R Liu in his book:- Meshfree Methods: Moving beyond the Finite Element Method, Page 24 - 26 (Preview page via Google book at &lt;a href=&quot;http://goo.gl/fd6wyw&quot; rel=&quot;nofollow&quot;&gt;http://goo.gl/fd6wyw&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance. &lt;/p&gt;&#xA;" OwnerUserId="11920" LastEditorUserId="549" LastEditDate="2014-12-27T18:38:02.340" LastActivityDate="2015-01-08T06:50:12.937" Title="How to determine the support/influence domain for irregularly distributed nodes in the Element-Free Galekin Method?" Tags="|finite-element|computational-geometry|meshless-methods|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17542" PostTypeId="2" ParentId="17535" CreationDate="2014-12-25T17:21:25.270" Score="2" Body="&lt;p&gt;One may well state the general fact that if the problem is well posed and the discretization scheme is reasonable, then the 'coefficient matrix' is invertible. So there is no common way around singular 'coefficient matrices' as they do not commonly appear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your particular case, your matrix is ill-posed because the central differences decouple the even and uneven nodes. That is why it is not commonly used for first order differential equations. Btw., this is called &lt;em&gt;numerical dispersion&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One more comment - the problem you state is overdetermined. For a first order differential equation in 1D you only need one boundary condition. And then, you rather use methods for initial value problems for the numerical solution. That is maybe the reason, it is rarely mentioned as an boundary value problem.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2014-12-25T17:21:25.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17543" PostTypeId="2" ParentId="2493" CreationDate="2014-12-26T05:05:19.977" Score="2" Body="&lt;p&gt;Some of the links above are outdated, hence look here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://wiki.scipy.org/PerformanceTips&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://wiki.scipy.org/PerformanceTips&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://wiki.scipy.org/PerformancePython&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://wiki.scipy.org/PerformancePython&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some ideas:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numpy,&#xA;Numba,&#xA;Cython,&#xA;Numexpr,&#xA;Theano,&#xA;Tensorflow,&#xA;f2py,&#xA;CPython C API,&#xA;pypy, &#xA;cffi,&#xA;Pythran,&#xA;Nuitka,&#xA;swig,&#xA;boost.python&lt;/p&gt;&#xA;" OwnerUserId="5401" LastEditorUserId="5401" LastEditDate="2017-06-04T05:53:05.723" LastActivityDate="2017-06-04T05:53:05.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17544" PostTypeId="1" CreationDate="2014-12-26T11:27:40.210" Score="0" ViewCount="94" Body="&lt;p&gt;I have a simple question concerning the management of the output after running a Fortran code, having a &lt;strong&gt;NESTED LOOP&lt;/strong&gt;..&lt;br/&gt;&#xA;I want to calculate for different &lt;strong&gt;k=1,5,9&lt;/strong&gt; the value $\mathbf{Sin(kx)}$ in the range $\mathbf{x=0:2\pi}$ with the step $\mathbf{h=0.01}$.&#xA;As the range of $\mathbf{x}$ for all $\mathbf{k}$ is the same, I would like to do an output in a &lt;strong&gt;cal.dat&lt;/strong&gt; file as the following (of course without the functions and just number):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x$ $\quad\qquad\sin(x)$   $\qquad\sin(5x)$    $\qquad\sin(9x)$ &lt;br/&gt;&#xA;$0.0$ $\quad\qquad0$ $\qquad\qquad0$ $\quad\qquad\qquad0$&lt;br/&gt;&#xA;$...$ $\quad\qquad...$ $\qquad\qquad...$ $\quad\qquad\qquad...$&lt;br/&gt;&#xA;&lt;br/&gt;&#xA;&lt;br/&gt;&#xA;My code is:&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;         program test3&#xA;         real dx,f,x&#xA;         k=1&#xA;         pi=4.0*atan(1.0)&#xA;         open(11,file='sinkx.dat',status='unknown',form='formatted')&#xA;         do  while(k&amp;lt;10)&#xA;         x=0.0&#xA;         dx=0.01&#xA;         do while(x&amp;lt;2*pi)&#xA;         f=sin(k*x) &#xA;         write(11,1001) x,f&#xA;1001     format(' ',f9.3,' ',e12.3)&#xA;         x=x+dx&#xA;         enddo&#xA;         k=k+4&#xA;         enddo&#xA;         end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;&lt;br/&gt;&#xA;  &lt;br/&gt;&#xA;  The questions is that when you run this code, you get the output not in a multi-column out put as desired, but in a two column out put in which first come numbers for the $\mathbf{x}$ range and in the second column the values of $\mathbf{Sin(kx)}$ for different $\mathbf{k=1}$, then $\mathbf{k=5}$ and then $\mathbf{k=9}$.&lt;br/&gt;&#xA;  How to write results in a multi-column out put, as I have indicated in the above?&lt;br/&gt;&#xA;  I think that the question is in &lt;strong&gt;write&lt;/strong&gt; operator rather than in the calculations?&lt;br/&gt;&#xA;  Of course, I could write the results in thre different &lt;strong&gt;cal.dat&lt;/strong&gt;, but I prefer to write it in a one &lt;strong&gt;dat&lt;/strong&gt; file, since the range of $\mathbf{x}$ is the same for all $\mathbf{k}$. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="13155" LastEditorUserId="13155" LastEditDate="2014-12-26T11:32:51.337" LastActivityDate="2015-02-05T05:04:36.137" Title="How in Fortran 77 or in the next standards to do a multi-column out put in a simple calculation, having a Nested loop?" Tags="|fortran|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17545" PostTypeId="2" ParentId="17544" CreationDate="2014-12-26T13:22:00.843" Score="1" Body="&lt;p&gt;There are multiple issues with your program. I am not sure why you are expecting it to produce multiple column output in the way that you want. You are looping over &lt;em&gt;x&lt;/em&gt; for each &lt;em&gt;k&lt;/em&gt;. You need to switch this nesting first. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you rightly guessed there is a specifier for the write statement, &lt;strong&gt;advance='no'&lt;/strong&gt;, which you can use to not advance to the next line when using formatted output. I have made the changes that would give you the desired formatting. It is slightly &lt;em&gt;hacky&lt;/em&gt; but it works.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program test3&#xA;  real dx,f,x&#xA;  pi=4.0*atan(1.0)&#xA;  open(11,file='sinkx.dat',status='unknown',form='formatted')&#xA;  x=0.0&#xA;  dx=0.01&#xA;  do while(x&amp;lt;2*pi)&#xA;    write(11,'(f9.3)',advance='no') x&#xA;    k=1&#xA;    do  while(k&amp;lt;10)&#xA;      f=sin(k*x) &#xA;      write(11,1001,advance='no') f&#xA;      k=k+4&#xA;    enddo&#xA;    x=x+dx&#xA;    write(11,*) ' '&#xA;  enddo&#xA;  1001  format(' ',e12.3)&#xA;endprogram&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="13144" LastActivityDate="2014-12-26T13:22:00.843" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17548" PostTypeId="1" AcceptedAnswerId="17556" CreationDate="2014-12-27T04:22:57.937" Score="1" ViewCount="129" Body="&lt;p&gt;I have an algorithm (in R) that maximizes a convex function on a compact convex set in every iteration. Based on the maximum principle, I know that the maximima are only attained on the boundary. But my current implementation searches in the whole space which is not necessary and time-consumer. To increase the time efficiency, I would like to narrow the search space, but I couldn't find any algorithm that only searchs on the boundary.&#xA;Any help is appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;The dimension can very from 2 to 6.&#xA;For case 2 the compact set is a rectangle [a, b]*[c, d] and the boundary is the perimeter of the rectangle. The function is nonlinear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I have the R code, but if I publish it here then my question would be transfered to Stack Overflow, which I don't think is a right place for my problem. I am looking for an algorithm or optimization method only the boundary.&lt;/p&gt;&#xA;" OwnerUserId="13163" LastEditorUserId="13163" LastEditDate="2014-12-28T23:10:03.830" LastActivityDate="2014-12-28T23:10:03.830" Title="How we can implement the result of maximum principle in our numerical optimization algorithm?" Tags="|convex-optimization|r|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="17549" PostTypeId="1" CreationDate="2014-12-27T10:37:35.120" Score="0" ViewCount="80" Body="&lt;p&gt;I am trying to Draw shear force diagram and bending moment diagram of beams. In this, I need to integrate second order differential.&#xA;So, Anybody can suggest me, which numerical method should I use?  &lt;/p&gt;&#xA;" OwnerUserId="13160" LastActivityDate="2014-12-29T15:46:51.190" Title="How to integrate Euler Bending Equation in C++?" Tags="|numerical-analysis|integral-equations|" AnswerCount="1" CommentCount="2" ClosedDate="2017-06-22T18:31:17.407" ContentLicense="CC BY-SA 3.0" />
  <row Id="17550" PostTypeId="2" ParentId="16076" CreationDate="2014-12-27T15:26:12.847" Score="1" Body="&lt;p&gt;you have a 6-connected neighbourhood. Say a,b,c,d,e,f are the neighbour values. And (a,e) are in x direction, (b,e) are in y direction and (c,f) are in z direction. As your wavefront never changes the sign of its speed, the upwind-neighbourhood are those points with smaller values, that is a&#xA;" OwnerUserId="13167" LastEditorUserId="13167" LastEditDate="2014-12-27T15:46:15.500" LastActivityDate="2014-12-27T15:46:15.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17552" PostTypeId="2" ParentId="17541" CreationDate="2014-12-27T19:19:11.753" Score="1" Body="&lt;p&gt;Theoretically, you can choose this distance arbitrarily.  In practice, you need to be more careful.  If the distance is too small, the support regions around each node will never intersect with each other. Thus, the equations effectively decouple from eachother and no meaningful answer can result from this.  In the other extreme, choosing the distance too large means that a lot of nodes couple to each other, making the linear system dense and more costly to solve.  Balancing the need to guarantee sufficient coupling while promoting sparsity is non-trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, you want to choose the distance large enough to have the support around each node overlap with the neighbors' support regions (as we usually observe with the traditional galerkin finite element method).  This really depends on how your points are distributed.  If you have points that roughly evenly throughout the domain (e.g. poisson disk sampling), you may want to use the &lt;strong&gt;&lt;em&gt;average nearest neighbor distance&lt;/em&gt;&lt;/strong&gt; for this purpose.  If you have the internodal distances stored in a matrix, it is very easy to calculate.  For each node $x_i$, find the nearest neighbor $x_j$ whose distance $d^*_i=d_{ij}$ from $x_i$ is smallest compared to all the other nodes.  If you have n nodes, then simply take the average of all the minimum distances $\frac{\sum_i d^*_i}{n}$.  In practice, you may want use a distance slightly larger than this to overcome any slight biases in the point distribution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you need to be cautious about using this approach for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you don't already have the internodal distances, you'll need to compute them and that can be costly!  ($O(n^2)$ operations).&lt;/li&gt;&#xA;&lt;li&gt;If your points are clustered into groups, then the average nearest neighbor distance can also lead to decoupled systems.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In general, you'll need to analyze the statistics of your spatial distribution to make the wisest choice of your support size.  No method works well for all possible distributions.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2014-12-27T19:26:51.903" LastActivityDate="2014-12-27T19:26:51.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17554" PostTypeId="1" CreationDate="2014-12-28T14:32:10.287" Score="4" ViewCount="24682" Body="&lt;p&gt;How to find all roots of an equation in Matlab? I tried and it gave me just one of the roots. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: my equation is $F(x)=0$ where &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; F(x) = (cos(7*x)).*exp(-2*x.^2).*(1-2*(x.^2)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="13172" LastEditorDisplayName="user10435" LastEditDate="2015-03-08T08:50:52.247" LastActivityDate="2015-03-09T00:49:17.950" Title="How to find all roots of equation in Matlab?" Tags="|matlab|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17555" PostTypeId="2" ParentId="17532" CreationDate="2014-12-28T19:21:52.933" Score="1" Body="&lt;p&gt;The gradient of a vector field is simply a matrix where each row is the gradient of one of the (scalar) components of your vector field. In other words, if you know how to compute the gradient of a scalar solution, you also know how to compute the gradient of a vector field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, on a triangular mesh with piecewise linear elements, the gradient is then simply a constant on every cell.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-28T19:21:52.933" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17556" PostTypeId="2" ParentId="17548" CreationDate="2014-12-28T22:12:31.237" Score="1" Body="&lt;p&gt;Check Rockafellars &quot;Convex Analysis&quot;  Theorem 32.2: If a convex function is defined on a set which is the convex hull of a set of points than the function attains its maximum at one of the points.  If I got your problem right, you just need to check your function on the corners. This seems doable for the dimensions you indicated. &lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2014-12-28T22:12:31.237" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17558" PostTypeId="2" ParentId="17554" CreationDate="2014-12-29T14:11:10.490" Score="9" Body="&lt;p&gt;Before trying to find &lt;em&gt;all&lt;/em&gt; of the roots of this function in MATLAB I think it's worth understanding that &lt;strong&gt;it has infinitely many roots&lt;/strong&gt; due to the inclusion of the $\cos()$ term. Additionally, it is easy to find the roots of the function analytically in this case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The roots are defined by&#xA;$$&#xA;\cos(7x)\cdot \exp(-2x^2)\cdot (1-2x^2) = 0.&#xA;$$&#xA;So we have&#xA;$$&#xA;\cos(7x)=0\text{ or }\exp(-2x^2)=0\text{ or }(1-2x^2) = 0,&#xA;$$&#xA;which gives&#xA;$$&#xA;x = \left\{\left(n+\frac{1}{2}\right)\dfrac{\pi}{7}: n\in\mathbb{Z}\right\} \cup \emptyset \cup \left\{\pm\sqrt{1/2}\right\}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question in a more general sense, a simple way to look for more than one root in MATLAB would be to use the &lt;code&gt;fzero&lt;/code&gt; function with many different starting guesses over some pre-defined range. This is not guaranteed to find all zeros, but by passing an interval to &lt;code&gt;fzero&lt;/code&gt; you can at least guarantee that you will find zeros where the function changes sign on that interval. By choosing small enough intervals you can obtain very good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the following code will find all the roots of your function on the interval [-10,10]. If any roots were missed you could increase &lt;code&gt;N&lt;/code&gt; to use more (smaller) starting intervals for &lt;code&gt;fzero&lt;/code&gt;. Note that this will only find roots where the sign changes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F = @(x) cos(7*x).*exp(-2*x.^2).*(1-2*x.^2);&#xA;interval = [-10, 10];&#xA;N = 500;&#xA;start_pts = linspace(interval(1),interval(2),N);&#xA;found_roots = [];&#xA;for i=1:numel(start_pts)-1&#xA;    try&#xA;        found_roots(end+1) = fzero(F,[start_pts(i),start_pts(i+1)]);&#xA;    end&#xA;end&#xA;&#xA;% Plot results:&#xA;figure, hold on&#xA;fplot(F,interval,1e-4)&#xA;plot(found_roots,zeros(size(found_roots)),'rx')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;*&lt;em&gt;Note that this method (and most numerical methods) will not work for $|x|$ greater than ~19.3 for this function. This is because $\exp(-2(19.4)^2)$ equals 0 with double precision numbers.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-03-09T00:49:17.950" LastActivityDate="2015-03-09T00:49:17.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17559" PostTypeId="1" AcceptedAnswerId="17561" CreationDate="2014-12-29T14:25:04.023" Score="1" ViewCount="1326" Body="&lt;p&gt;Determining the intersection of two convex polygons is one of the fundamental problems in computational geometry .&#xA;I'm asking for an algorithm having:&lt;/p&gt;&#xA;&lt;h1&gt;INPUT:&lt;/h1&gt;&#xA;&lt;p&gt;Given two convex polygons P and Q in 2D (note that the two convex polygons are represented with p &amp;amp; q points)&lt;/p&gt;&#xA;&lt;h1&gt;OUTPUT :&lt;/h1&gt;&#xA;&lt;p&gt;compute their intersection?&lt;/p&gt;&#xA;&lt;p&gt;Can any one help me !!!&lt;/p&gt;&#xA;" OwnerUserId="8289" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2014-12-29T16:47:15.037" Title="Convex Polygon Intersection" Tags="|computational-geometry|convex-optimization|geometry|constraints|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17560" PostTypeId="2" ParentId="17549" CreationDate="2014-12-29T15:46:51.190" Score="1" Body="&lt;p&gt;You can use odeint, which is part of the Boost library, here's a long list of examples &lt;a href=&quot;http://www.boost.org/doc/libs/1_53_0/libs/numeric/odeint/doc/html/boost_numeric_odeint/tutorial/all_examples.html&quot; rel=&quot;nofollow&quot;&gt;odeint&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2014-12-29T15:46:51.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17561" PostTypeId="2" ParentId="17559" CreationDate="2014-12-29T15:55:07.523" Score="3" Body="&lt;p&gt;An algorithm for this computation is described in &lt;em&gt;&lt;a href=&quot;http://cs.smith.edu/~orourke/books/compgeom.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Geometry in C&lt;/a&gt;&lt;/em&gt;, Chapter 7, Section 6. Code is available at that link. Much of the tricky code is concerned with &quot;degenerate&quot; cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a nice Univ Montreal web page (Eric Plante) that describes a different algorithm:&#xA;&lt;a href=&quot;http://www.iro.umontreal.ca/~plante/compGeom/algorithm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;.&#xA;&lt;hr /&gt;&#xA;&amp;nbsp;&#xA;&amp;nbsp;&#xA;&amp;nbsp;&#xA;&amp;nbsp;&#xA;&amp;nbsp;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/xgSYM.gif&quot; alt=&quot;Plante fig&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr /&gt;&#xA;" OwnerUserId="673" LastEditorUserId="673" LastEditDate="2014-12-29T16:47:15.037" LastActivityDate="2014-12-29T16:47:15.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17562" PostTypeId="1" AcceptedAnswerId="17567" CreationDate="2014-12-29T17:47:31.100" Score="3" ViewCount="7995" Body="&lt;p&gt;Is there any reason to always solve the &lt;a href=&quot;http://en.wikipedia.org/wiki/Kepler%27s_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kepler equation&lt;/a&gt; for the &lt;a href=&quot;http://en.wikipedia.org/wiki/Eccentric_anomaly&quot; rel=&quot;nofollow noreferrer&quot;&gt;eccentric anomaly&lt;/a&gt;, $E$, instead of the more meaningful (at least to me) &lt;a href=&quot;http://en.wikipedia.org/wiki/True_anomaly&quot; rel=&quot;nofollow noreferrer&quot;&gt;true anomaly&lt;/a&gt;, $\theta$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving for the eccentric anomaly usually would mean calculating the true anomaly from its solution and maybe even the initial guess of the eccentric anomaly from an initial guess of the true anomaly. So this would require more calculations, however the Kepler equation expressed in the eccentric anomaly is simpler and thus less expensive to calculate. When I search for solving Kepler's equation I only have some approaches which use the eccentric anomaly, however I can imagine that when the error tolerance of the solution is not very tight or the initial guess is good it might be cheaper to solve for the true anomaly. Or are there other reasons for solving for the eccentric anomaly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relevant equations, using Newton's method, are:&#xA;$$&#xA;E = 2 \tan^{-1}\left(\sqrt{\frac{1-e}{1+e}}\tan\frac{\theta}{2}\right),&#xA;$$&#xA;$$&#xA;M = E - e \sin{E},&#xA;$$&#xA;where $M$ is the mean anomaly and $e$ is the eccentricity. Such that using &lt;a href=&quot;http://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Newton's method&lt;/a&gt; results in&#xA;$$&#xA;E_{n+1} = E_n - \frac{E_n - e \sin{E_n} - M}{1 - e \cos{E_n}}.&#xA;$$&#xA;And for the &lt;a href=&quot;https://physics.stackexchange.com/questions/69380/what-are-common-methods-for-calculating-the-time-dependency-of-elliptical-orbit&quot;&gt;true anomaly&lt;/a&gt;,&#xA;$$&#xA;M = 2\tan^{-1}\left(\sqrt{\frac{1-e}{1+e}}\tan{\frac{\theta}{2}}\right)-\frac{e\sqrt{1-e^2}\sin{\theta}}{1+e\cos{\theta}},&#xA;$$&#xA;$$&#xA;\theta_{n+1} = \theta_n - \left(1 + e \cos{\theta_n}\right)^2 \frac{2\tan^{-1}\left(\sqrt{\frac{1-e}{1+e}}\tan{\frac{\theta_n}{2}}\right)-\frac{e\sqrt{1-e^2}\sin{\theta_n}}{1+e\cos{\theta_n}} - M}{\sqrt{(1 - e^2)^3}}.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="7031" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2023-10-08T15:03:14.050" Title="Solving Kepler equation for true or eccentric anomaly" Tags="|newton-method|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17563" PostTypeId="1" CreationDate="2014-12-29T19:44:47.563" Score="0" ViewCount="206" Body="&lt;p&gt;I need to generate a sequence of increasingly denser high-quality triangular meshes for some pentagons in MATLAB. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to supply as my input &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The coordinates of the 5 vertices of the pentagon &lt;/li&gt;&#xA;&lt;li&gt;The desired (approximate) mesh-node spacing or the number of mesh-faces in the output mesh. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The output should be &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A high-quality mesh&lt;/li&gt;&#xA;&lt;li&gt;The list of points on the mesh-boundary (ie the boundary of the pentagon)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are there any softwares to do this, preferably in MATLAB or elsewhere? &lt;/p&gt;&#xA;" OwnerUserId="3352" LastEditorUserId="549" LastEditDate="2014-12-29T23:06:36.267" LastActivityDate="2014-12-30T02:27:09.653" Title="Meshing of polygons" Tags="|matlab|computational-geometry|mesh|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17564" PostTypeId="2" ParentId="17563" CreationDate="2014-12-29T20:14:15.223" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;nofollow&quot;&gt;http://geuz.org/gmsh/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This program is called gmsh it should be able to do exactly what your looking for. I recently posted an answer similar to this which can be found here (&lt;a href=&quot;http://tinyurl.com/n6avzsz&quot; rel=&quot;nofollow&quot;&gt;http://tinyurl.com/n6avzsz&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have the program downloaded the way it works is by coding the .geo file. You can do this by hand, or code it into the .geo file. See this post for a sample of that (&lt;a href=&quot;http://tinyurl.com/lme57dk&quot; rel=&quot;nofollow&quot;&gt;http://tinyurl.com/lme57dk&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="10225" LastActivityDate="2014-12-29T20:14:15.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17565" PostTypeId="2" ParentId="17563" CreationDate="2014-12-30T01:00:56.713" Score="1" Body="&lt;p&gt;There is also Jonathan Shewchuck's &lt;em&gt;triangle&lt;/em&gt; package that should be able to do the same for you.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-30T01:00:56.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17566" PostTypeId="2" ParentId="17563" CreationDate="2014-12-30T02:27:09.653" Score="0" Body="&lt;p&gt;If you have the MATLAB &lt;a href=&quot;http://www.mathworks.com/products/pde/?refresh=true&quot; rel=&quot;nofollow&quot;&gt;PDE Toolbox&lt;/a&gt; it has the necessary tools built in. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-12-30T02:27:09.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17567" PostTypeId="2" ParentId="17562" CreationDate="2014-12-30T06:20:06.683" Score="7" Body="&lt;p&gt;The most robust way of answering this is to benchmark it. Failing&#xA;that, there are several things to note (roughly in order of&#xA;importance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, the most cheap floating-point operations on a modern CPU are&#xA;addition and multiplication (both are equally fast; same as fused&#xA;multiply-adds when available). Division is much slower (by a factor of&#xA;~20), trigonometric functions are also slower (~200), square roots and&#xA;logs similar. In fact, sometimes (depends on architecture) trig&#xA;functions, roots, logs are implemented in a library. The best&#xA;reference for this is usually the optimization guide published by the&#xA;whichever company made your CPU.  So as a rule of thumb you want to&#xA;minimize special functions first, then divisions, then multiplications&#xA;and additions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So by this measure the second formula is much worse: the there is one&#xA;inverse trig function, one square root (it would be wasteful to&#xA;implement your formula as written, with three square roots), and one&#xA;sincos (since they are expensive it is better to evaluate sincos once&#xA;and then write things like $\tan\frac\theta2$ in terms of those). Your&#xA;first formula involves the other special functions, apart from sincos,&#xA;only once at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, you should not stop at Newton's method. For example, if you&#xA;compare Newton's method with Halley's method,&#xA;$$ \frac{e \sin (x)-e x \cos (x)+M}{1-e \cos (x)}, \qquad&#xA;x-\frac{(e \cos (x)-1) (e \sin (x)+M-x)}{e^2+e (M-x) \sin (x)-2 e \cos&#xA;  (x)+1}, $$&#xA;both evaluate the expensive trigonometric functions at the same&#xA;arguments, but with a little more algebra Halley's method can converge&#xA;faster; the extra algebra might not outweigh the savings of&#xA;evaluating trigonometric fewer times. So Halley's method (and other&#xA;iterative methods) also need to be checked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, you can precompute some things. For example, if you start by&#xA;reducing the argument to the range $0&amp;lt;E&amp;lt;2\pi$, you can experimentally,&#xA;in advance, find the maximum number of iterations taken by the&#xA;method. Since the number of iterations is fairly small (depending on&#xA;precision) and mispredicted branches can be expensive, it may also&#xA;make sense to unroll the iteration loop by hand to a fixed sufficient&#xA;number of iterations. Unrolled code can also more easily benefit from&#xA;vectorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fourth, it is difficult to intuitively predict which optimizations&#xA;will do the best. And since this is just one fairly simple equation,&#xA;it is probably best to benchmark many different approaches and find&#xA;the best. When doing this, also consider what optimization flags your&#xA;compiler supports for floating-point arithmetic (some of them are&#xA;interesting and important, many people know about &lt;code&gt;-ffast-math&lt;/code&gt;, but&#xA;it actually decomposes into different helpful or harmful optimization&#xA;flags; &lt;a href=&quot;https://gcc.gnu.org/wiki/FloatingPointMath&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; is gcc's&#xA;list for example). Another thing to do is to look at the assembly&#xA;output of your compiler to see which CPU instructions it actually ends&#xA;up using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fifth, if you need to solve this equation many times for the same&#xA;eccentricity $e$, it is possible to rewrite the problem. If you&#xA;consider the function $E = E(M)$, on the range $[0,2\pi]$, given by&#xA;the solutions of the equation for fixed known $e$, you can approximate&#xA;the function $E(M)$ using, for example, Chebyshev series, which takes&#xA;only a small number of evaluations of $E(M)$ (which can be done with&#xA;any root-finding method). Once you have a sufficiently close&#xA;approximation, which might be a 20-term Chebyshev series or something&#xA;like that, you can evaluate that later without needing to solve the&#xA;equation again.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2014-12-30T06:20:06.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17569" PostTypeId="1" CreationDate="2014-12-24T13:29:58.943" Score="6" ViewCount="193" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Xbh1.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;Let given&lt;br&gt;&#xA;$M =$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1     0     1&#xA; 0     1     1&#xA; 1     1     1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and&#xA;$b =$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 1&#xA; 0&#xA; 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How to find the solution $x_3$ where $x=${$x_1,x_2,x_3$}?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution&lt;/strong&gt;: Based on Wiedemann algorithm we has $u_i=[0; 0; 1]$. &#xA;I am getting confused in the Step2. How to find $M_s$? &#xA;&lt;strong&gt;Is it just the $3^th$ columns of $M_t$ as $M_s=[1 1 1]$ or it is created from Krylov sequence?&lt;/strong&gt; &lt;strong&gt;Please help me find M_s and minimal polynomial?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Input Generation Matrix M, b and index ith&#xA;%% Output single symbol x&#xA;M=[1 0 1;0 1 1;1 1 1];&#xA;b=[1; 0; 1];&#xA;index=2;&#xA;[K N]=size(M);&#xA;Mt=M';&#xA;Imatrix=eye(K,K);&#xA;x=[];&#xA;for index=1:K&#xA;u_i=Imatrix(:,index);&#xA;se_Krylov=[];&#xA;se_Krylov(:,1)=u_i;&#xA;    for i=2:(2*K) % due to i=1 is u_i&#xA;       se_Krylov(:,i)=mod((Mt^i)*u_i,2);&#xA;    end&#xA;%% M_s is the operator Mt retricted to S    &#xA;M_s=se_Krylov(index,:);&#xA;&#xA;%% Compute the polynominal using Berlekmap Massey&#xA;[f, LCP] = Berlekamp_Massey2(M_s);&#xA;d=size(f,2)-1; %deg of f: x^d+x^(d-1)+....&#xA;&#xA;x_comma=zeros(K,1);&#xA;    for i=d:-1:1&#xA;        x_comma=mod(x_comma+f(d).*(Mt)^(i-1)*u_i,2);&#xA;    end&#xA;x_single=mod(x_comma'*b,2) ;%Inner product &#xA;x(index)=x_single;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user8264" LastActivityDate="2014-12-30T13:28:41.570" Title="Find the solution of linear equation using Wiedemann/ Krylov method" Tags="|matlab|linear-algebra|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17570" PostTypeId="1" AcceptedAnswerId="17571" CreationDate="2014-12-30T18:28:54.907" Score="0" ViewCount="82" Body="&lt;p&gt;I am going to solve an linear iterative inverse problem. I have two functions in &lt;code&gt;matlab&lt;/code&gt; which one of them play the &lt;code&gt;forward&lt;/code&gt; and the other play the &lt;code&gt;adjoint&lt;/code&gt; role. I am not familiar with inverse problems very much and I am going to test some algorithms to check which one can help me to obtain a sparse model. Since I have to use my functions to do the forward and inverse operator roles in &lt;code&gt;Ax=b&lt;/code&gt;, so I think I am not able to use some algorithms which needs &lt;code&gt;A&lt;/code&gt; as a matrix. As an example, I could use the &lt;code&gt;linearized bregman&lt;/code&gt;as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;v^{k+1} = v^k + A^T(f-Au^k).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u^{k+1}= \delta * shrink(v^{k+1},1/\mu).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in which, for &lt;code&gt;A&lt;/code&gt; I use that function which transform the model &lt;code&gt;u&lt;/code&gt; to the data space and for &lt;code&gt;A^T&lt;/code&gt; the function which transform the data to the model space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know that is there any other iterative algorithm which can be used in this manner for sparsity?  &lt;/p&gt;&#xA;" OwnerUserId="9754" LastActivityDate="2014-12-30T21:52:00.623" Title="Iterative algorithms for sparsity using a function for operator A in Ax=b" Tags="|sparse-matrix|iterative-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17571" PostTypeId="2" ParentId="17570" CreationDate="2014-12-30T21:52:00.623" Score="0" Body="&lt;p&gt;Yes, most solvers for sparsity regularized least squares problems have been designed to use matrix-vector multiplications rather than accessing the matrix directly.  See for example SPGL1 at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.ucdavis.edu/~mpf/spgl1/&quot; rel=&quot;nofollow&quot;&gt;http://www.math.ucdavis.edu/~mpf/spgl1/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good list of software for these kinds of problems can be found (at the very bottom of the page) at the Rice web site on resources for compressive sensing:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dsp.rice.edu/cs&quot; rel=&quot;nofollow&quot;&gt;http://dsp.rice.edu/cs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2014-12-30T21:52:00.623" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17572" PostTypeId="2" ParentId="2905" CreationDate="2014-12-30T23:01:17.337" Score="3" Body="&lt;p&gt;There are some useful results for Newton's method applied to complex polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Joel Friedman in &lt;a href=&quot;http://www.math.ubc.ca/~jf/pubs/web_stuff/convergence.html&quot; rel=&quot;nofollow&quot;&gt;On the Convergence of Newton's Method&lt;/a&gt; (Theorem 2.2) gives an explicit radius for a disk contained in the immediate basin of attraction of each root of a polynomial $f$:&#xA;$$&#xA;r=\frac{\eta}{2d}&#xA;$$&#xA;where $\eta$ is the minimum distance between two roots of $f$ and $d$ is the degree of $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other explicit bounds are given by Anthony Manning in &lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF01232940&quot; rel=&quot;nofollow&quot;&gt;How to be sure of finding a root of a complex polynomial using Newton's method&lt;/a&gt; (Theorem 1.2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;http://dx.doi.org/10.1007/s002220100149&quot; rel=&quot;nofollow&quot;&gt;How to find all roots of complex polynomials by Newton's method&lt;/a&gt;  by Hubbard et al.&lt;br&gt;&#xA;&lt;em&gt;Invent. Math.&lt;/em&gt; 146 (2001), no. 1, 1–33. &lt;a href=&quot;http://www.math.cornell.edu/~hubbard/NewtonInventiones.pdf&quot; rel=&quot;nofollow&quot;&gt;pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2014-12-30T23:01:17.337" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17573" PostTypeId="1" CreationDate="2014-12-31T00:54:39.600" Score="3" ViewCount="398" Body="&lt;p&gt;To those who are familiar with the projected Newton's method or projected gradient method...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We consider a constrained optimization problem with simple bounds. Particularly, minimize f(x) subject to L &amp;lt;= x &amp;lt;= U, where f maps R^n to R. x, L, and U are vectors in R^n. In the projected Newton's method that is used to solve this problem, the search direction is obtained by solving the linear system (reduced Hessian)*(search direction) = - gradient. Based on this, the active elements of an iterate are moved in the direction of the negative gradients at those elements. By definition of active sets, the gradients at the active elements have to be negative (for upper bound) and positive (for lower bound). Hence, after projection, the active elements are moved back to the boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first question is if x_i is an active element in iteration 1, will it be an active element until convergence? In other words, does the active set only get bigger, not smaller, and members of the active set are only added, not removed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My second question is what if we use epsilon active set instead of just active set? Will it change anything?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need the definition of active set or any other clarification/background, please feel free to let me know. Thank you so much for your ideas and discussions. This is very important to me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;We consider the problem: &#xA;\begin{equation}&#xA;\min f(x) \quad \text{subject to} \quad a \leq x \leq b&#xA;\end{equation}&#xA;where $f: \mathbb{R}^n \rightarrow \mathbb{R}$ is continuously differentiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An active set is defined as&#xA;\begin{equation}&#xA;\mathcal{A}:= \{ i | (x_i = a_i \quad \text{and} \quad (\nabla f(x))_i &amp;gt; 0) \quad \text{or} \quad (x_i = b_i \quad \text{and} \quad (\nabla f(x))_i &amp;lt; 0) \}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose our initial iterate $x_0$ is on the boundary, e.g. $x_0 = a$, then it is possible to have some inactive elements where $(\nabla f(x))_i &amp;lt; 0$, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the projected Newton's method, first with step length $\alpha = 1$, the active elements move in the directions of the negative gradients, which will point outside of the feasible region. Then, those elements are projected back to the boundaries. The inactive elements move in the Newton directions. Then we test for sufficient decrease...but even when we decrease $\alpha$, the active elements still move back to the boundaries and those elements still meet the &quot;active&quot; criteria in the next Newton iteration?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the active set can shrink...there must be something wrong in my thoughts...&lt;/p&gt;&#xA;" OwnerUserId="13191" LastEditorUserId="13191" LastEditDate="2015-01-03T02:26:25.687" LastActivityDate="2015-01-03T02:26:25.687" Title="Active Elements in Projected Newton's Method?" Tags="|algorithms|numerical-analysis|constrained-optimization|newton-method|projection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17574" PostTypeId="1" CreationDate="2014-12-31T04:55:29.100" Score="2" ViewCount="418" Body="&lt;p&gt;I'm with the background of computer engineering and generally use FEM for graphics simulation. As far as I know, FEM formulation is usually expressed with respect to the reference configuration, i.e., the volume integration is over the initial domain and the stress-based force $f$ is computed using the first Piola-Kirchhoff stress $P$ integrated over the initial element $\Omega_0$ as:&#xA;$$&#xA;f = \int_{\Omega_0}P\nabla_X Nd\Omega&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any good reason why initial configuration is preferred over current configuration? Theoretically the force can be expressed with the cauchy stress $\sigma$ integrated over current volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integration is usually approximated with Gauss quadrature. One reason I assume is that accuracy and robustness of the numerically approximated volume integration maybe at stake if the shape of current domain is close to degenerated (with nearly not invertible jacobian). Integrating over initial volume is better as we assume the initial mesh quality is good.&lt;/p&gt;&#xA;" OwnerUserId="1976" LastActivityDate="2015-01-03T13:20:44.550" Title="Why does FEM usually formulate the problems in reference configuration?" Tags="|finite-element|" AnswerCount="2" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17575" PostTypeId="2" ParentId="17574" CreationDate="2014-12-31T05:05:23.827" Score="3" Body="&lt;p&gt;To compute the quadrature points you either need an accurate formula for them once on the reference element, or you have to project them accurately onto the actual element. The former is easier. Also, I'd have to count operations more carefully, but I think it's less operations to do everything on the reference element.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-12-31T05:05:23.827" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17576" PostTypeId="2" ParentId="17573" CreationDate="2014-12-31T13:51:22.217" Score="3" Body="&lt;p&gt;The answer to your first question is no -- the active set does not only grow, it can also shrink, even if your objective function is convex. An example is if you start at some point on the boundary, then the first step moves along one active constraint to find the minimum of the objective function along this segment; but then, the second step will likely move away from the boundary because at the previous point the gradient will have pointed (in general, unless you are at the solution) &lt;em&gt;into&lt;/em&gt; the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer to your second question is not clear to me. I would say that it depends on the size of epsilon. If epsilon is small enough, my best guess is that it doesn't matter, but small enough will be relatively to the curvature of the objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to learn more about the active set method, my favorite book on optimization is &quot;Numerical optimization&quot; by Nocedal and Wright -- which has an extensive discussion of active set methods.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-31T13:51:22.217" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="17577" PostTypeId="1" CreationDate="2014-12-31T15:02:11.303" Score="3" ViewCount="3196" Body="&lt;p&gt;I was looking to brush up and/or pick up some languages for a hobby project on my free Amazon EC2, and I was going through the benchmark results for &lt;a href=&quot;http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=java&amp;amp;lang2=gcc&quot; rel=&quot;nofollow&quot;&gt;Java vs C&lt;/a&gt; (or &lt;a href=&quot;http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=scala&amp;amp;lang2=gpp&quot; rel=&quot;nofollow&quot;&gt;Scala vs C&lt;/a&gt;)and &lt;a href=&quot;http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=ifc&amp;amp;lang2=gcc&quot; rel=&quot;nofollow&quot;&gt;Fortran vs C&lt;/a&gt;. It seems that the margin in Java vs C is nearly the same in Fortran vs C, which made me wonder if JVM (and its JIT optimizations) advances are catching up fast enough to occupy the same status as C &amp;amp; Fortran in High Performance (Scientific) Computing in the near future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Factors, other than performance, to be taken into consideration could be: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;readability, maintainability and expressivity of codes&lt;/li&gt;&#xA;&lt;li&gt;cost of scalable hardware infrastructure&lt;/li&gt;&#xA;&lt;li&gt;Availability of libraries&lt;/li&gt;&#xA;&lt;li&gt;Parallel and/or Concurrent programming support&lt;/li&gt;&#xA;&lt;li&gt;Talent pool&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;edited question: Among all high performance computational fields, what are the fields that might see the rise of Java/Scala JVM ecosystem? For example, it seems to me that complex system analysis, economic/finance, machine learning etc might be a few of those fields. What else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edited on 24 Sept 2015: After I had posted this question, I came across this old (2007) &lt;a href=&quot;http://www.stefankrause.net/wp/?p=4&quot; rel=&quot;nofollow&quot;&gt;blog post&lt;/a&gt; that re-did some of those same benchmarks-game for Java and C, but after excluding the initial warmup/optimization run (although, its effect is debated) for a few different Java compilers, and the results are quite surprising. Although, as the comments from Isaac Gouy at that blog say, running 4-5 trials for each benchmark may not be enough to average out processor usage by other processes running on that system. But still, an interesting blog. Since that 2007 blog, I believe JRockit and HotSpot compilers have been merged by Oracle, but not sure how their combined performance stacks against C compilers'.&lt;/p&gt;&#xA;" OwnerUserId="13193" LastEditorUserId="13193" LastEditDate="2015-09-24T20:48:15.107" LastActivityDate="2015-09-24T20:48:15.107" Title="Use of Java or Scala in HPC?" Tags="|fortran|c|" AnswerCount="4" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="17578" PostTypeId="2" ParentId="17577" CreationDate="2014-12-31T15:09:38.537" Score="6" Body="&lt;p&gt;It seems unlikely to me. The Java MPI APIs haven't been worked on in years (so you're wrong about #4), and the JVM's floating-point performance is notoriously poor. Java may out perform C/C++ or Fortran in some areas due to rapid thread creation and easy memory management, but these aren't the bottlenecks in typical scientific programs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to your #5, the availability of lots of Java programmers isn't really all that helpful. I find that good scientific programmers need to understand the science and software engineering. Both aspects are trainable, so you have to spend the time training on either the science or the software engineering to be a good scientific programmer. There are lots of good scientists, too, so the latent talent pools are big enough to eventually supply good scientific programmers from either side. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2014-12-31T15:09:38.537" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17579" PostTypeId="2" ParentId="17577" CreationDate="2014-12-31T15:37:05.833" Score="9" Body="&lt;p&gt;Java has been around for almost 20 years now as a major programming language, but it hasn't caught on in scientific computing so far. I think that's a good indicator for what's going to happen in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My take is that the issue isn't speed. Most people are probably willing to give up 20% of performance (or even a factor of 2) if they would be vastly faster in developing their programs. But that doesn't seem to be the case with Java vs C++ (though it is almost certainly true for Java vs Fortran). I would claim that in fact the opposite is the case: because practically all HPC libraries (for linear algebra and finite element methods, in particular) are written in C or C++, using Java would be a major pain since one would have to somehow translate the existing interfaces. For libraries the size of PETSc, Trilinos (or my own project deal.II), this is simply not feasible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, using Java (or Scala, or whatever other language you have in mind), the burden compared to C++ is huge: you'd either have to write wrappers and interfaces to existing libraries, or you'd have to reimplement the hundreds of thousands of lines of code of existing libraries. Neither sounds like an attractive option.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2014-12-31T15:37:05.833" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17580" PostTypeId="2" ParentId="17562" CreationDate="2015-01-01T20:29:33.623" Score="0" Body="&lt;p&gt;This may be of interest to you: a paper recently (December 2014) published presents a method for computing solutions to Kepler's Equation analytically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.scirp.org/journal/PaperInformation.aspx?PaperID=52772&quot; rel=&quot;nofollow&quot;&gt;http://www.scirp.org/journal/PaperInformation.aspx?PaperID=52772&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't worked through the paper myself (it involves a two-dimensional Laplace Transform), but it may be relevant to your work.&lt;/p&gt;&#xA;" OwnerUserId="13202" LastActivityDate="2015-01-01T20:29:33.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17582" PostTypeId="2" ParentId="17574" CreationDate="2015-01-02T10:30:59.370" Score="2" Body="&lt;p&gt;Current configuration at time t+1 is unknown.  If you know what is current configuration problem is already solved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Answer for you question is in fact about total and updated formulations, look at Eulerian, Total Lagrangian, Updated L, UALE, TALE, Convective. It is matter of convenience which one to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formulation is &quot;total&quot; in the sense that you integrate over initial configuration. Formulation is &quot;updated&quot; if you integrate over last iterated configuration (not necessary in equilibrium). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at &quot;total&quot; and &quot;updated&quot; formulations, both will give exactly the same result. Only difference is only how you expressing your physical equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Lagrangian, &quot;updated&quot; or &quot;total&quot;, it not issue of accuracy. For both matrix and right hand vector should look the same. It will not help also with the distorted elements (look to ALE as a solution, using meshes method will not help as well). In both formulations you will have exactly the same integration quadrature to integrate exactly (otherwise you will do variational crime).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion only difference is when you look at the force vector, and when you do linearisation to get tangent matrix. In case of total formulations equations will be longer, in case of updated equations will be shorter (easier to implement), but in expense that you need to store additional variables (this story looks a little bit different for fluids, fluids forger about displacements).&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2015-01-03T13:20:44.550" LastActivityDate="2015-01-03T13:20:44.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17583" PostTypeId="2" ParentId="17577" CreationDate="2015-01-02T10:47:31.543" Score="6" Body="&lt;p&gt;I would argue that Java will in fact REDUCE productivity when compared with modern c++, or even with modern Fortran for the purpose of scientific computing.&#xA;Writing &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = B*C+2*D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is just so much more readable than&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = B.mult(C).add(D.mult(2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Assuming the code above deals with arrays, both C++ and fortran will also produce significantly more efficient code for that example, depending on your library of choice.&#xA;Scala is a much nicer language, as it will allow a decent syntax, although you will still take the performance hit from creating a a lot of temporary objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither Java nor Scala have complex data as a primitive type, which means there is a significant overhead when doing computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others have stated, you need a decent interface to C/Fortran libraries for scientific computation.This can be done in Java and Scala, but it is likely to be a massive amount of work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to learn a &quot;commercial&quot; programming language, and you're using scientific computing as training, I would consider C#, which has a saner syntax for these things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you just want to do scientific programming, I would consider learning Julia, as a &quot;get things done fast&quot; language. This will get you faster code than the JVM languages, has excellent plotting support (via python) and is actively developed for scientific computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wish to learn a new programming paradigm, try F#, as you would still have easy access to the various .NET libraries for scientific computing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you want high performance, C++ is currently far more expressive than Java and has decent parallel support with OMP. Arguably, it also has the best library support of any of the mentioned languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, Java is heavily used in finance. Keep in mind that the same is also true of COBOL, which tells you all you need to know about programming in the financial sector.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="7522" LastEditDate="2015-01-02T17:06:40.157" LastActivityDate="2015-01-02T17:06:40.157" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17586" PostTypeId="1" CreationDate="2015-01-03T00:31:29.520" Score="4" ViewCount="413" Body="&lt;p&gt;Consider this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{array}{ll}&#xA;       \text{minimize }   &amp;amp; f(x)  \\&#xA;       \text{subject to } &amp;amp; a \leq x \leq b&#xA;\end{array}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a,b,x \in \mathbb{R}^n$ and $f: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is continuously differentiable. By this definition, the feasible set is closed and convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would greatly appreciate it if you could give me a big picture of the classes of methods that solve this problem. I know there are some methods such as projected gradient/Newton methods or primal-dual active set/semismooth Newton methods and some variations of these methods. However, these are specific. &#xA;The book by Nocedal and Wright seems to discuss the following types of methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;active set&lt;/li&gt;&#xA;&lt;li&gt;interior-point&lt;/li&gt;&#xA;&lt;li&gt;gradient projection&lt;/li&gt;&#xA;&lt;li&gt;penalty and augmented Lagrangian &lt;/li&gt;&#xA;&lt;li&gt;sequential quadratic programming&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The book &quot;Optimization with PDE Constraints&quot; by Hinze, Pinnau, and Ulbrich discusses:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Newton's methods&lt;/li&gt;&#xA;&lt;li&gt;Semismooth Newton's methods (which Google Scholar says is equivalent to the primal-dual active set methods)&lt;/li&gt;&#xA;&lt;li&gt;Sequential quadratic programming methods&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am doing a literature review of the methods for this problem and hope that through my literature review I can discover something that needs improving. I have only 2 months (January and February) to do this. I am very new to this field and do not consider myself mathematically mature. I have very little experiences. Hence, it would be great if some of you could kindly share your experiences/visions. It would help me a lot if I could have a &lt;strong&gt;big picture&lt;/strong&gt; of the methods. I would forever be grateful.&lt;/p&gt;&#xA;" OwnerUserId="13191" LastEditorUserId="13191" LastEditDate="2015-01-04T00:22:18.470" LastActivityDate="2015-01-04T00:22:18.470" Title="Methods for Constrained Optimization Problems with Box Constraints" Tags="|numerical-analysis|iterative-method|nonlinear-programming|constrained-optimization|newton-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17587" PostTypeId="2" ParentId="17586" CreationDate="2015-01-03T01:57:36.843" Score="1" Body="&lt;p&gt;The book by Nocedal and Wright spends a lot of time developing your intuition. It is an excellent resource to get an overview. It is also not very mathematical, so I would suggest reading through the relevant chapters.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-01-03T01:57:36.843" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="17588" PostTypeId="1" CreationDate="2015-01-04T13:13:11.233" Score="5" ViewCount="364" Body="&lt;p&gt;I am trying to solve an almost incompressible problem with heterogeneous properties by domain decomposition. Solution with CG converges slowly or divergerces completely. My problem becomes ill-conditioned (condition number in the order of 10^5) due to the poissons ratio approaching to 0.5 and stiffness differences in between materials. I detected that in element stiffness matrix, several additional zero energy modes (hourglass modes etc) are formed in addition to standard rigid body modes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am planning to prevent these modes by MatNullSpace object in Petsc. Since the computation of nullspaces with SVD or QR is expensive, I am trying to find a cheaper way of predicting the nullspace of floating subdomains. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to predict nullspaces for subdomain from contributing finite element's null spaces? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read a broad sentence in a publication says that &quot;The rigid body modes of a collection of elements is equal to the assembly of the rigid body modes of the individual elements taking into account the multiplicity of those degrees of freedom that lie in multiple neighboring elements.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;does that really make sense?&lt;/p&gt;&#xA;" OwnerUserId="1347" LastActivityDate="2015-04-21T21:42:30.220" Title="Is it possible to predict the null space of a structure from contributing elements null spaces?" Tags="|finite-element|petsc|conjugate-gradient|domain-decomposition|nullspace|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17589" PostTypeId="2" ParentId="16432" CreationDate="2015-01-05T00:41:11.793" Score="1" Body="&lt;p&gt;Your comment mentions that the data shows a piecewise linear trend. In that case it would be most accurate to use linear interpolation. The other options you asked about (nearest neighbor and natural neighbor) result in piecewise constant (generally discontinuous) interpolants. The trade off is that it is more computationally expensive to compute the linear interpolation than the others.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-01-05T13:53:25.230" LastActivityDate="2015-01-05T13:53:25.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17592" PostTypeId="1" CreationDate="2015-01-05T11:52:17.847" Score="-1" ViewCount="2491" Body="&lt;p&gt;Is there some packages in java that can be included to eclipse in order to solve a non linear problem? I have used ipopt but when running, there is a notification which say &quot;NOTE: You are using Ipopt by default with the MUMPS linear solver.&#xA;      Other linear solvers might be more efficient (see Ipopt documentation).&quot; while I search to use a non linear solver for my problem. Please help me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="13222" LastActivityDate="2015-02-06T15:47:08.270" Title="optimization non linear problem with java code" Tags="|optimization|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17593" PostTypeId="1" CreationDate="2015-01-05T14:56:52.003" Score="3" ViewCount="163" Body="&lt;p&gt;Consider &#xA;$$F(\theta)=\sin \theta \int_{-L}^{+L}h(z)e^{-ikz\cos \theta} \,dz$$&#xA;$$|z|\le L$$&#xA;$$0 \le \theta \le \pi$$&#xA;By having knowledge of $F(\theta)$, how can one approximate $h(z)$? In addition, I know that $F$ is differentiable with respect to $\theta$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I model and solve such problem with unknown function $h(z)$ in Mathematica or any other programming language?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an idea in which we write Fourier series for both $F$ and h then by solving or manipulation of the integral we approximate the coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;** $k$ is the wave number, don't confuse it with anything else.&lt;br/&gt;&#xA;** $z$ might be a complex number, but in this case you can consider it as real.&lt;br/&gt;&#xA;**Any idea on this problem would be extremely appreciated.&lt;/p&gt;&#xA;" OwnerUserId="13225" LastActivityDate="2015-11-04T14:47:29.627" Title="Approximate $h$ in $F(\theta)=\sin \theta \int_{-L}^{+L}h(z)e^{-ikz\cos \theta} \,dz$" Tags="|matlab|quadrature|fourier-analysis|mathematica|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17594" PostTypeId="2" ParentId="17593" CreationDate="2015-01-05T16:26:08.587" Score="9" Body="&lt;p&gt;This is a linear integral equation (because the right hand side is linear in $h$) and presumably the problem is ill-posed (for one, because of the multiplication by $\sin(\theta)$ but also because the &quot;integral kernel&quot; is smooth).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Methods to solve this (approximately) are general &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Galerkin_method&quot; rel=&quot;nofollow&quot;&gt;Galerkin methods&lt;/a&gt;&lt;/strong&gt; or &lt;strong&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Collocation_method&quot; rel=&quot;nofollow&quot;&gt;collocation methods&lt;/a&gt;&lt;/strong&gt;. For Galerkin methods you take a finite linear independent set (i.e. a basis for a finite dimensional subspace) of functions to model $h$ and $F$ and derive the linear system for the coefficients of $h$ and $F$ for this basis. For collocation methods you discretize the integral (e.g. by trapezoidal rule) and write down the (linear) equations that you get if you evaluate $F$ at some points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By warned that the respective linear systems that you will get may be ill-conditioned and that you should use some regularization method (check any book on linear inverse problems or look up &quot;Tikhonov regularization&quot;, &quot;truncated SVD&quot;, &quot;Landweber iteration&quot; or &quot;regularization by CG).&quot;&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="302" LastEditDate="2015-11-04T14:47:29.627" LastActivityDate="2015-11-04T14:47:29.627" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17595" PostTypeId="1" AcceptedAnswerId="18844" CreationDate="2015-01-05T16:36:01.393" Score="1" ViewCount="832" Body="&lt;p&gt;I am running 2 simulations on a cantilever plate in Nastran: one is a transient analysis (time domain) and the other one is a frequency response analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transient analysis computes the response to time-varying excitations explicitly defined in the time domain. The equation it has to solve is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M \ddot{u}(t) + C \dot{u}(t) + K u(t) = P(t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $M,C,K$ are matrices and $u,P$ are vectors. The equations is solved by numerical integration (central difference method). $P(t)$ is a sinusoid at frequency equal to the $1^{st}$ natural frequency of the plate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The frequency response compute the response to oscilatory excitation explicitly defined in the frequency domain. The computed responses are given as magnitude/phase (with respect to the forcing) or as real/imaginary components. The equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$[-\omega^2 M +i \omega C+K] u(\omega) = P(\omega)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The forcing function is defined in the range $20-150 \ Hz$ to excite the first mode only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running the 2 analysis I would expect to obtain the same results in terms of maximum displacement (since I am considering only the first mode in both cases), but it is not so. The amplitudes in the time plot and in the magnitude plot do not match at all. Where the problem could be? Am I missing something?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i64.photobucket.com/albums/h182/Olgola/modal_transient.jpg&quot; width=&quot;250&quot; height=&quot;250&quot;&gt; &lt;img src=&quot;https://i64.photobucket.com/albums/h182/Olgola/modal_frequency.jpg&quot; width=&quot;250&quot; height=&quot;250&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I plot the real/imaginary components, the amplitude of the real part (picture on the left) matches the time domain amplitude. How should I interpreted it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i64.photobucket.com/albums/h182/Olgola/modal_frequency_real.jpg&quot; width=&quot;250&quot; height=&quot;250&quot;&gt; &lt;img src=&quot;https://i64.photobucket.com/albums/h182/Olgola/modal_frequency_immag.jpg&quot; width=&quot;250&quot; height=&quot;250&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="-1" LastEditDate="2017-03-10T09:42:37.937" LastActivityDate="2015-02-04T22:48:31.033" Title="Structural FEM analysis: transiet response vs frequency response" Tags="|finite-element|time-integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17596" PostTypeId="2" ParentId="17592" CreationDate="2015-01-05T18:39:19.280" Score="1" Body="&lt;p&gt;You're not very precise regarding what problem you want to solve, but I know of a few Java code libraries that offer mathematical programming functionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.joptimizer.com/&quot; rel=&quot;nofollow&quot;&gt;JOptimizer&lt;/a&gt; Should be able to handle a variety of convex problems. I have no idea about performance, accuracy or bugs - I just know it exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ojalgo.org/&quot; rel=&quot;nofollow&quot;&gt;ojAlgo&lt;/a&gt; (I wrote this) Can handle LP, QP and MIP. It's designed for use cases when you have smaller simpler problems and want a pure Java implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apache Commons Math also has an LP (simplex) solver.&lt;/p&gt;&#xA;" OwnerUserId="13228" LastActivityDate="2015-01-05T18:39:19.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17598" PostTypeId="2" ParentId="17595" CreationDate="2015-01-05T21:52:30.613" Score="2" Body="&lt;p&gt;If $P(t)=\sin(\omega_1 t)$ where $\omega_1$ is the first eigenfrequency, then $P(\omega)$ is really only nonzero for that particular frequency, $\omega=\omega_1$. There is no need to consider any other frequency in the frequency domain.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-01-05T21:52:30.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17599" PostTypeId="2" ParentId="17592" CreationDate="2015-01-06T00:53:05.637" Score="2" Body="&lt;p&gt;IPOPT provides its own nonlinear solvers that are part of the interior point methods it implements. As such, these solvers (really, internal class methods) will construct linear systems that must be solved, similar to a quasi-Newton method. The warning merely points out that you are using the default solver instead of something like Pardiso, or the HSL libraries, and as WolfgangBangerth points out, using a different solver should not affect the solution you get (but will affect the time it takes to get that solution). If you have installed IPOPT correctly, then that installation is probably sufficient to get you started.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-01-06T00:53:05.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17601" PostTypeId="2" ParentId="16173" CreationDate="2015-01-06T08:38:22.957" Score="0" Body="&lt;p&gt;There is a Java linear algebra benchmark that will give you some insight into how much slower Java is than native code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://code.google.com/p/java-matrix-benchmark/wiki/RuntimeCorei7v2600_2013_10&quot; rel=&quot;nofollow&quot;&gt;Older results that compare Java and native code&lt;/a&gt; (Native code called from Java.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ojalgo.org/performance_ejml.html&quot; rel=&quot;nofollow&quot;&gt;Newer results that only compare the Java alternatives&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This benchmark shows that it can be more than 10x faster to call native code from Java rather than implementing directly in Java, but this is With larger matrices and the more complex algorithms. Java performs well with simpler operations, and for small matrices the overhead when calling native code is too big. &lt;/p&gt;&#xA;" OwnerUserId="13228" LastEditorUserId="13228" LastEditDate="2015-01-06T08:55:25.967" LastActivityDate="2015-01-06T08:55:25.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17602" PostTypeId="1" AcceptedAnswerId="17605" CreationDate="2015-01-06T10:02:14.750" Score="4" ViewCount="348" Body="&lt;p&gt;I am currently developing a project that uses finite elements for multi-dimensional PDEs and I'm still wondering if I will use P elements (triangles in 2D and tetra in 3D) or Q elements (squares in 2D and cubes in 3D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I have read on the web, the crucial point seems to be meshing which is supposed to be easier to handle with simplicies (so P elements). On the other side, Q elements are easier to implement in multidimensional cases (in my opinion).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there places in the theory where P elements would work and Q elements not? Or conversely?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that the answer is &quot;no&quot; or &quot;in rare weird cases&quot;, otherwise there would be no big libraries using Q elements, but I'd like to be sure.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2015-01-06T16:33:21.800" Title="P versus Q elements" Tags="|finite-element|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17603" PostTypeId="2" ParentId="17602" CreationDate="2015-01-06T11:49:40.507" Score="3" Body="&lt;p&gt;One fundamental difference, in 3D, is that second order tetrahedrons only have degrees of freedom on the edges, while Q2 elements have dofs on faces as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One macroscopic consequence affects the Stokes problem solution. Sub optimal elements like P2-P0 are not stable, while Q2-Q0 are.&lt;/p&gt;&#xA;" OwnerUserId="6574" LastActivityDate="2015-01-06T11:49:40.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="17604" PostTypeId="1" CreationDate="2015-01-06T14:09:07.460" Score="4" ViewCount="615" Body="&lt;p&gt;In the Nelder-Mead algorithm, the simplex looks for the minimum of the function. If I multiply all the function values times -1, would I trick the simplex into searching for the maximum?&lt;/p&gt;&#xA;" OwnerUserId="13235" LastActivityDate="2019-06-10T08:26:55.743" Title="Using the Nelder-Mead algorithm to find a maximum" Tags="|optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17605" PostTypeId="2" ParentId="17602" CreationDate="2015-01-06T16:17:15.607" Score="4" Body="&lt;p&gt;If you're working w/high order elements, the two differ in approximation spaces - for P elements of order $p$, you have polynomials of total degree $p$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x^iy^jz^k$ where $i+j+k\leq p$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;versus Q elements, which give polynomials of individual degree $p$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x^iy^jz^k$ where $i,j,k\leq p$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At low order, the two are roughly equivalent, and in practice, both spaces usually end up giving the same convergence rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some differences concerning efficient implementations, especially for explicit timestepping, which involves inverting a mass matrix at each step. For example, usually you map a reference element to physical elements of your mesh. On simplical straight-sided elements, the change of variables factor associated with this mapping is constant. As a result, the mass matrix for each element is a scaling of the reference mass matrix, which can be exploited for low-memory time-marching &lt;a href=&quot;http://books.google.com/books?hl=en&amp;amp;lr=&amp;amp;id=0fNte_5nQNwC&amp;amp;oi=fnd&amp;amp;pg=PA1&amp;amp;dq=info:xRpRM9vsJCIJ:scholar.google.com&amp;amp;ots=V7WHT12aR1&amp;amp;sig=gHIvdlPxnzi7qiLYPOy--ret8Jo#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;implementations of Discontinuous Galerkin methods&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Efficiency for &quot;Q&quot; elements comes in another manner - their tensor product structure means that you can typically define basis functions through products of 1D functions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi_{ijk}(x,y,z) = \phi_i(x)\phi_j(y)\phi_k(z)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This carries through to mass and stiffness matrices in that they are Kronecker products of 1D mass/stiffness matrices under the appropriate indexing. If you store only a single 1D matrix, this allows for efficient and low-memory matrix-free solvers at &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521453097&quot; rel=&quot;nofollow&quot;&gt;high order&lt;/a&gt; (even for &lt;a href=&quot;http://gji.oxfordjournals.org/content/192/3/889.full.pdf+html&quot; rel=&quot;nofollow&quot;&gt;at-scale parallel adaptive meshes&lt;/a&gt;). Additionally, under a special quadrature rule, the Spectral Element Method (SEM) allows for an easy-to-invert diagonal mass matrix, making time-marching for quads/hexes very efficient as well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q elements are usually considered to be more efficient than P elements in practice, since they produce a larger approximation space for a fewer number of unknowns. Apart from that, the two are roughly equivalent for most stable problems (as Nicola mentioned, if the problem is not discretely stable, the two can have differing effects - see also locking in elasticity).  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-01-06T16:33:21.800" LastActivityDate="2015-01-06T16:33:21.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17606" PostTypeId="1" AcceptedAnswerId="17609" CreationDate="2015-01-06T16:22:56.927" Score="3" ViewCount="7487" Body="&lt;p&gt;I am into computational fluid dynamics and so far I've found that the most common approaches to solve for the governing equations are Eulerian and Lagrangian. The former samples the domain at fixed locations while the latter samples the domain using the particles moving freely across the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading through some pdf's over the net someone mentioned about mesh-based methods but as far as I could see they were very close to what the Eulerian approach is (if not the same).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone clarify is there is indeed a difference between what is called Grid-Based methods (like Finite differences) and Mesh-Based methods (like ???) ?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2023-01-11T04:51:46.890" Title="What's the difference between grid-based and mesh-based methods for PDEs?" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17607" PostTypeId="1" AcceptedAnswerId="18607" CreationDate="2015-01-06T18:14:19.097" Score="7" ViewCount="1500" Body="&lt;p&gt;This is a continuation of a problem I asked over at &lt;a href=&quot;https://physics.stackexchange.com/questions/154570/can-i-use-runge-kutta-to-solve-these-equations&quot;&gt;physics exchange&lt;/a&gt; and &lt;a href=&quot;https://math.stackexchange.com/questions/1078136/can-i-use-runge-kutta-to-solve-these-two-equations&quot;&gt;math exchange&lt;/a&gt;.  Basically I have two ODEs that I am solving in order to calculate the radial and tangential velocity of liquid dispensed on the center of a disk rotating at rate of $\omega$. $u_m$ is the average flow velocity in the $r$-direction, and the $v_m$ is the average flow velocity in the $\theta$-direction.  They are defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_m(r)=\frac{1}{h(r)}\int_0^{h(r)}u(r,z)\,dz$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$v_m(r)=\frac{1}{h(r)}\int_0^{h(r)}v(r,z)\,dz$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $h(r)$ is the height of the liquid film on the spinning disk.  Using the continuity equation, the height of the film is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$h(r)=\frac{Q}{2\pi\,r\,u_m}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $Q$ is the flow rate of liquid onto the spinning disk.  Using appropriate boundary conditions of no-slip at $z=0$ and no shear at $z=h(r)$, the velocities $u$ and $v$ are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(r,z)=3u_m\left[ \frac{z}{h}-\frac{1}{2}\left(\frac{z}{h} \right)^2 \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$v(r,z)=r\,\omega+\frac{3}{2}(r\,\omega- v_m)\left[\left(\frac{z}{h} \right)^2 -2 \frac{z}{h}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then taking the Navier Stokes equations in cylindrical coordinates and assuming that $\partial P/\partial r=0$, the remaining relevant terms are then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u\frac{\partial u}{\partial r}-\frac{v^2}{r}=\nu \left(\frac{\partial^2 u}{\partial z^2} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u\frac{\partial v}{\partial r}+\frac{vu}{r}=\nu \left(\frac{\partial^2 v}{\partial z^2} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I substitute $u(r,z)$ and $v(r,z)$ into the N-S equations, integrate them from $0$ to $h(r)$ in the $z$-direction, and then substitute $h(r)$ in terms of $Q$ using the equation above.  This then gives me two coupled ODEs where $u_m$ and $v_m$ are the dependent variables and $r$ is the independent variable.  The ODEs are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$69ru_m\frac{du_m}{dr}=8r^2\omega^2-16r\omega v-21{u_m}^2+48{v_m}^2-\frac{480\pi^2\nu r^3{u_m}^3}{Q^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$21r(v_m-r\omega)\frac{du_m}{dr}+48ru\frac{dv_m}{dr}=-69u_m v_m+37r\omega u_m+\frac{480\pi^2\nu r^3(v_m-r\omega){u_m}^2}{Q^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve these equations I rearranged them in matrix form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textbf{A}=\begin{bmatrix}69\,r\,u_m &amp;amp; 0\\ 21(r\,v_m-r^2\omega) &amp;amp; 48\,r\,u_m \end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textbf{u}=\begin{bmatrix}u_m \\ v_m \end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textbf{f}=\begin{bmatrix} 8\,r^2\omega^2-16\,r\,\omega\,v_m-21{u_m}^2+48{v_m}^2-\frac{480\pi^2\nu}{Q^2}r^3{u_m}^3 \\ -69\,u_m\,v_m+37\,r\,\omega\,u_m-\frac{480\pi^2\nu}{Q^2}r^3{u_m}^2(v_m-r\,\omega) \end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the equations are of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textbf{A} \frac{d \textbf{u}}{dr}=\textbf{f}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can rearrange this as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d \textbf{u}}{dr}=\textbf{A}^{-1}\textbf{f}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And solve it using a Runge-Kutta method.  I have tried solving it using the following code in MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function output = improvedPigford(flow,spin)&#xA;if nargin&amp;lt;2&#xA;    spin = 75;          % [RPM]&#xA;end&#xA;if nargin&amp;lt;1&#xA;    flow = 935;         % [mL/min]&#xA;end&#xA;&#xA;w = spin*pi*2/60;       % rotation of wafer [rad/s]&#xA;d = 0.002;              %diameter of injection nozzle [m]&#xA;&#xA;r0 = 0.5*d;             % begin solving here [m]&#xA;q = flow/(100^3*60);    % volumetric flow rate of liquid [m^3/s]&#xA;kv = 8.926780E-07;      % kinematic viscosity of water @ 25C [m^2/s]&#xA;u0 = q/(pi*r0^2);       % initial r-velocity (assume same as nozzle velocity) [m/s]&#xA;v0 = 0;                 % initial theta-velocity [m/s]&#xA;&#xA;%% solve improved Pigford model&#xA;rRange = [r0 0.1];&#xA;initCond = [u0 v0];&#xA;&#xA;options = odeset('Stats','on','RelTol',1e-6,'MStateDependence','strong');&#xA;dudr = @(r,u) improvedModel(r,u,q,kv,w);&#xA;sol = ode15s(dudr,rRange,initCond,options);&#xA;r = sol.x;&#xA;u = sol.y(1,:);&#xA;v = sol.y(2,:);&#xA;h = q./(2*pi*r.*u);&#xA;&#xA;%% plot section&#xA;figure(1);clf;&#xA;plot(r*1000,u,'r-',r*1000,v,'b-');&#xA;xlabel('Radius (mm)');&#xA;ylabel('Fluid Velocity (m/s)');&#xA;&#xA;%% output section&#xA;output=[r,u,v,h];&#xA;&#xA;end&#xA;&#xA;% --------------------------------------------------------------------------&#xA;function dudr = improvedModel(r,u,q,kv,w)&#xA;%{&#xA;Function is of the form A*u=f, so u=inv(A)*f&#xA;%}&#xA;&#xA;A = [69*r*u(1), 0; 21*r*(u(2)-r*w), 48*r*u(1)];&#xA;f = [8*r^2*w^2 - 16*r*w*u(2) - 21*u(1)^2 + 48*u(2)^2 - (480*pi^2*kv*r^3*u(1)^3)/q^2;...&#xA;    -69*u(1)*u(2) + 37*r*w*u(1) + (480*pi^2*kv*r^3*u(1)^2*(u(2)-r*w))/q^2];&#xA;dudr = A\f;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However when I run this code the solution always blows up at about %r=0.0441%, no matter which RK function I use: ode45, ode23, ode113, ode15s, ode23s, ode23t, or ode23tb.  The place where the solution blows up does change when I change the parameters like flow rate, spin speed, viscosity, etc., but it does always blow up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I perhaps doing something wrong in my implementation of the Runge-Kutta method in MATLAB, or is it possible that the equations are simply ill-posed?  Maybe there is simply a singularity at the location where it blows up or something?  I've checked the integration of the N-S equations several times using symbolic mathematics software, and I'm fairly confident that I did it correctly.  Otherwise though I'm kind of out of ideas of what to look at next.  Any suggestions or ideas would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Following the suggestion of @Geoff Oxberry I changed the MATLAB formulation to include the possibility that the mass matrix $\textbf{A}$ may be singular.  Doing that the new code is this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function output = improvedPigford(flow,spin)&#xA;if nargin&amp;lt;2&#xA;    spin = 200;         % [RPM]&#xA;end&#xA;if nargin&amp;lt;1&#xA;    flow = 1000;        % [mL/min]&#xA;end&#xA;&#xA;w = spin*pi*2/60;       % rotation of wafer [rad/s]&#xA;d = 0.002;              %diameter of injection nozzle [m]&#xA;&#xA;r0 = 0.5*d;             % begin solving here [m]&#xA;q = flow/(100^3*60);    % volumetric flow rate of liquid [m^3/s]&#xA;kv = 8.926780E-07;      % kinematic viscosity of water @ 25C [m^2/s]&#xA;u0 = q/(pi*r0^2);       % initial r-velocity (assume same as nozzle velocity) [m/s]&#xA;v0 = 0;                 % initial theta-velocity [m/s]&#xA;&#xA;%% solve improved Pigford model&#xA;rRange = [r0 0.1];&#xA;initCond = [u0 v0];&#xA;&#xA;dudr = @(r,u) improvedModel(r,u,q,kv,w);&#xA;M = @(r,u) massMatrix(r,u,w);&#xA;options = odeset('Stats','on','RelTol',1e-6,'Mass',M);&#xA;sol = ode15s(dudr,rRange,initCond,options);&#xA;r = sol.x;&#xA;u = sol.y(1,:);&#xA;v = sol.y(2,:);&#xA;h = q./(2*pi*r.*u);&#xA;&#xA;%% plot section&#xA;figure(1);clf;&#xA;plot(r*1000,u,'r-',r*1000,v,'b-');&#xA;xlabel('Radius (mm)');&#xA;ylabel('Fluid Velocity (m/s)');&#xA;&#xA;%% output section&#xA;output=[r,u,v,h];&#xA;&#xA;end&#xA;&#xA;function f = improvedModel(r,u,q,kv,w)&#xA;%{&#xA;Function is of the form M*u'=f&#xA;%}&#xA;f = [8*r^2*w^2 - 16*r*w*u(2) - 21*u(1)^2 + 48*u(2)^2 - (480*pi^2*kv*r^3*u(1)^3)/q^2;...&#xA;    -69*u(1)*u(2) + 37*r*w*u(1) + (480*pi^2*kv*r^3*u(1)^2*(u(2)-r*w))/q^2];&#xA;end&#xA;&#xA;function M = massMatrix(r,u,w)&#xA;%{&#xA;Function calculates the mass matrix M for M*u'=f&#xA;%}&#xA;M = [69*r*u(1), 0; 21*r*(u(2)-r*w), 48*r*u(1)];&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the result is the same: the solution still blows up at the same location.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update 2&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As @Kirill asked, I am showing a plot of what the solution looks like when it blows up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mZ0VD.jpg&quot; alt=&quot;Image of blown-up solution&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_m$ goes to $-\infty$, while $u_m$ goes to $+\infty$.  What makes me suspicious that something may be wrong is the fact that due to boundary conditions on $v(r,z)$ [$v=r\,\omega$ @ $z=0$ and $\frac{\partial v}{\partial z}=0$ @ $z=h(r)$], $v_m$ should almost certainly be positive.  @Kirill found a sign discrepancy between my derivation and his derivation done using Mathematica, so I will be checking my derivation again, then getting back with my results.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update 3&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I went over the calculations &lt;em&gt;again&lt;/em&gt;, and I think I did have a sign error as @Kirill postulated.  After working through derivation one more time, I ended up with the following equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$69r\,u_m\frac{du_m}{dr}=8r^2\omega^2-16r\omega v_m-21{u_m}^2+48{v_m}^2-\frac{480\pi^2\nu r^3{u_m}^3}{Q^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$21r(v_m-r\omega)\frac{du_m}{dr}+48r\,u_m\frac{dv_m}{dr}=-69u_m v_m+37r\omega u_m-\frac{480\pi^2\nu r^3(v_m-r\omega){u_m}^2}{Q^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is equivalent to the equations that @Kirill obtained doing the derivation in Mathematica.  When solving this system of equations (using the above code but with the corrected term) I get the following plots for $u_m$ and $v_m$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/i0y2L.jpg&quot; alt=&quot;improved model&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This appears to be identical to the plot that @Kirill showed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a further comparison, this derivation is an improvement of the Pigford model, which can be found here:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;R.M. Wood and B.E. Watts, &quot;The Flow, Heat, and Mass Transfer&#xA;  Characteristics of Liquid Films on Rotating Disks&quot;, &lt;em&gt;Trans. Instn&#xA;  Chem. Engrs.&lt;/em&gt;, Vol 51, 1973.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The derivation of the Pigford model is identical to mine, except that the final integration $\int_0^h(r) dz$ is simply skipped over, with the $u(r,z)$ and $v(r,z)$ simply being replaced by $u_m(r)$ and $v_m(r)$, with an $O(1)$ correction factor added to each equation.  From this the Pigford model equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r\,u_m \frac{du_m}{dr}={v_m}^2-\frac{12 K_1 \nu\, \pi^2\,r^3\,{u_m}^3}{Q^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r\,u_m \frac{dv_m}{dr}=-u_m v_m-\frac{12 K_2 \nu\, \pi^2\,r^3\,(v_m-r\,\omega){u_m}^2}{Q^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I solve those two equations simultaneously, I get the following solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/X80Uj.jpg&quot; alt=&quot;Pigford model&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is qualitatively close to the equations that I derived.  For a final comparison, I'll look at the height $h(r)$ calculated by both of the models:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/cUiI5.jpg&quot; alt=&quot;film height comparison&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The shapes of the two curves are a little different, but mostly only differ by a $O(1)$ constant, as one might expect because of the addition of the fitting constants in the Pigford model.&lt;/p&gt;&#xA;" OwnerUserId="13237" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2015-01-09T16:13:27.067" Title="Solution blows up when using Runge-Kutta to solve simultaneous ODEs for liquid film equations" Tags="|matlab|fluid-dynamics|runge-kutta|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="17608" PostTypeId="2" ParentId="17607" CreationDate="2015-01-06T18:37:55.937" Score="6" Body="&lt;p&gt;The first thing I can think of with your derivation is the $\mathbf{A}$ term. If at any point that term fails to be invertible, then your derivation is no longer valid. You should see the the value of $\mathbf{A}$ is at your singularity, and if it is still invertible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to diagnose this problem would be to use a DAE solver on $\textbf{A} \frac{d \textbf{u}}{dr}=\textbf{f}$ directly, instead of inverting $\mathbf{A}$; you could use &lt;code&gt;ode15i&lt;/code&gt; for this purpose. If the DAE solver works, then there's a good chance singularity of $\mathbf{A}$ is to blame; if not, then you'll have to look for other alternative solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Here are some other possible causes of the behavior you're seeing that you may want to investigate:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Is there a numerical stability issue with the integration methods you're using?&lt;/em&gt; Based on what you've said, it seems a little unlikely; you've tried methods that are generally geared toward stiff problems (&lt;code&gt;ode15s&lt;/code&gt;: NDF/BDF methods, &lt;code&gt;ode23s&lt;/code&gt;: Rosenbrock methods, &lt;code&gt;ode23tb&lt;/code&gt;: TR-BDF2) and nonstiff problems (&lt;code&gt;ode45&lt;/code&gt; &amp;amp; &lt;code&gt;ode23&lt;/code&gt;: explicit Runge-Kutta). Neither seems to help. Sometimes, there are telltale instabilities that can be seen when trying to solve a stiff problem with explicit methods (e.g., sometimes growing oscillations precede the solution blowing up).&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Is there a scaling issue with units?&lt;/em&gt; One way to diagnose this sort of error would be to look at the conditioning of the linear systems being solved within the implicit ODE (or DAE) solvers (so, &lt;code&gt;ode15s&lt;/code&gt;, &lt;code&gt;ode23s&lt;/code&gt;, &lt;code&gt;ode23tb&lt;/code&gt;, etc.). Poorly conditioned systems also tend to be stiff (even if nondimensionalizing them would yield a nonstiff system), which would explain why nonstiff integrators also blow up. Nondimensionalization usually improves the conditioning in these cases, so you may also try to solve a nondimensionalized version of your problem instead. I'd check scaling next.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Is your right-hand side regular enough over the interval of integration?&lt;/em&gt; By that, I mean, is it $k$ times continuously differentiable, where $k$ is greater than or equal to the order of time discretizations you're using? I also doubt that the problems you're seeing are due to derivatives failing to exist, because your right-hand side and mass matrix consist of polynomials.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Is there something about the system analytically that causes it to blow up?&lt;/em&gt; I sort of doubt it, but exploring solved test problems in the literature and asymptotics can be helpful here. If there's a test problem in the literature that maps closely to what you're modeling here, you should use it as a benchmark for validation, because it can be helpful in debugging. You could find out that there's a &quot;blow up time&quot; beyond which something about your approximation breaks down.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-01-07T06:42:20.953" LastActivityDate="2015-01-07T06:42:20.953" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="17609" PostTypeId="2" ParentId="17606" CreationDate="2015-01-06T20:35:59.360" Score="11" Body="&lt;p&gt;There is no firm distinction between grids and meshes. However, they are often used in slightly different ways. The following definitions are more guidelines of common usage than actual rules and you may hear people use them interchangeably in many cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Grids&lt;/strong&gt; are typically a set of simulation elements that have a well defined structure to their alignment with square or rectangular grids being the most prototypical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Meshes&lt;/strong&gt; are often more general. They may be unstructured and use various shapes of elements, sometimes even mixing elements of different types in the same mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, both Eulerian and Lagrangian methods may use meshes and grids. There is even a third class of method called Arbitrary Lagrangian Eulerian Methods which uses a grid that may be either or neither Eulerian or Lagrangian. &lt;strong&gt;Sometimes people use the term mesh-based methods or grid-based methods to refer to the class of all methods that use meshes or grids. In this case I don't believe there is any intention to distinguish between meshes and grids, rather the intention is to distinguish methods that use a mesh &lt;em&gt;or&lt;/em&gt; a grid as a key part of the method from those that require neither.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-01-07T02:04:34.340" LastActivityDate="2015-01-07T02:04:34.340" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="17610" PostTypeId="1" AcceptedAnswerId="17611" CreationDate="2015-01-06T20:54:05.090" Score="1" ViewCount="182" Body="&lt;p&gt;Is there some company providing HPC cluster online? So that I can connect to their cluster by Internet and run my program?&lt;/p&gt;&#xA;" OwnerUserId="7417" LastActivityDate="2016-07-05T14:12:39.040" Title="Are there HPC clould services?" Tags="|cloud-computing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="17611" PostTypeId="2" ParentId="17610" CreationDate="2015-01-06T21:21:28.987" Score="8" Body="&lt;p&gt;Amazon Web Services has &lt;a href=&quot;http://aws.amazon.com/hpc/&quot;&gt;HPC instances&lt;/a&gt;. Penguin Computing provides &lt;a href=&quot;https://pod.penguincomputing.com/&quot;&gt;Penguin On Demand&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Almost all academic and non-commercial HPC services are available over the Internet as well. For example, the site I work for, &lt;a href=&quot;http://www.tacc.utexas.edu&quot;&gt;TACC&lt;/a&gt;, provides its services through the Internet. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-01-06T21:21:28.987" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18605" PostTypeId="1" CreationDate="2015-01-07T11:55:25.967" Score="1" ViewCount="1684" Body="&lt;p&gt;I'm trying to reconstruct an image given its Laplacian, which results in a Poisson equation and I'm using Neumann boundary conditions (derivative at boundary = 0). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have is the laplacian (f, left-hand side), and the image borders (boundaries).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I solve this in Matlab or Octave? I found the &lt;code&gt;poisolv()&lt;/code&gt; and &lt;code&gt;assempde()&lt;/code&gt; functions but I don't quite understand how they work.&lt;/p&gt;&#xA;" OwnerUserId="14238" LastActivityDate="2015-07-06T15:56:42.757" Title="Solve poisson equation with Neumann b.c. (matlab or octave)" Tags="|matlab|poisson|octave|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18606" PostTypeId="2" ParentId="18605" CreationDate="2015-01-07T12:21:37.793" Score="1" Body="&lt;p&gt;In MATLAB I suggest using the pdetool GUI which provides an easier-to-use interface&#xA;to the assempde function.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-01-07T12:21:37.793" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18607" PostTypeId="2" ParentId="17607" CreationDate="2015-01-07T20:10:19.780" Score="6" Body="&lt;p&gt;Bearing in mind that I have just a vague idea of what results you are&#xA;expecting, I want to point out that I do not get the same ODE system&#xA;as you do. I'm using Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using your equations for $h(r)$, $u(r,z)$, $v(r,z)$ and the two NS&#xA;equations, the following code generates the ODE system:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Module[{uu, vv, h},&#xA; h = q/(2 Pi r u[r]);&#xA; uu = 3 u[r] (z/h - (z/h)^2/2);&#xA; vv = r w + 3/2 (r w - v[r]) ((z/h)^2 - 2 z/h);&#xA; Integrate[&#xA;  {uu D[uu, r] - vv^2/r - kv D[uu, z, z],&#xA;   uu D[vv, r] + vv uu/r - kv D[vv, z, z]}&#xA;  , {z, 0, h}]&#xA; ]&#xA;Thread[{0, 0} == (Numerator[Together[#/q]] &amp;amp; /@ %)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;0=480 \pi ^2 \nu r^3 u(r)^3-8 q^2 r^2 w^2+69 q^2 r u(r) u'(r)+21 q^2 u(r)^2+16 q^2 r w v(r)-48 q^2&#xA;   v(r)^2\\&#xA;0=-480 \pi ^2 \nu r^4 w u(r)^2+480 \pi ^2 \nu r^3 u(r)^2 v(r)-21 q^2 r^2 w u'(r)+21 q^2 r v(r)&#xA;   u'(r)+48 q^2 r u(r) v'(r)+69 q^2 u(r) v(r)-37 q^2 r w u(r)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only difference seems to be the sign on the &quot;480&quot;-term in the second equation, I get it as &quot;$-$&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if I solve the system of ODEs that you have in the question using Mathematica's &lt;code&gt;NDSolve&lt;/code&gt;, I also get a blow-up. But if I solve the system I generated above from the same equations, I get the following plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nKyCS.png&quot; alt=&quot;Plot&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This looks okay to me, there is no blow-up, although I can't be sure this is right. So&#xA;having said all this, are you &lt;em&gt;absolutely&lt;/em&gt; sure your ODE system is&#xA;actually correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the ode solver invocation (the commented out sections &lt;code&gt;(*...*)&lt;/code&gt; are two versions of your ODE system):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Module[{&#xA;  spin = 75,&#xA;  flow = 935,&#xA;  w, d, r0, rRange, q, kv, u0, v0, r, u, v&#xA;  , slv&#xA;  },&#xA; w = spin Pi*2/60;&#xA; d = 0.002;&#xA; r0 = 0.5 d;&#xA; q = flow/(100^3*60);&#xA; kv = 8.926780*^-7;&#xA; u0 = q/(Pi r0^2);&#xA; v0 = 0;&#xA; rRange = {r0, 0.1};&#xA; (*slv=NDSolve[&#xA; Join[{u[r0]==u0,v[r0]==v0},&#xA; Thread[{u'[r],v'[r]}== Inverse[{{69r u[r],0},{21r (v[r]-r w),48r \&#xA;u[r]}}]&#xA; .{8*r^2*w^2-16*r*w*v[r]-21*u[r]^2+48*v[r]^2-(480*Pi^2*kv*r^3*u[r]^3)/\&#xA;q^2,-69*u[r]*v[r]+37*r*w*u[r]+(480*Pi^2*kv*r^3*u[r]^2*(v[r]-r*w))/q^2}\&#xA;]]&#xA; ,{u,v},{r,r0,rRange[[2]]}];*)&#xA; (*slv=NDSolve[{&#xA; 69r u[r]u'[r]==8r^2w^2-16r w v[r]-21u[r]^2+48 v[r]^2-480Pi^2 kv \&#xA;r^3u[r]^3/q^2,&#xA; 21r (v[r]-r w)u'[r]+48 r u[r] v'[r]==-69 u[r]v[r]+37r w u[r]+480Pi^2 \&#xA;kv r^3(v[r]-r w)u[r]^2/q^2,&#xA; u[r0]==u0,&#xA; v[r0]==v0&#xA; },{u,v},{r,r0,rRange[[2]]}];*)&#xA; slv = NDSolve[{&#xA;    0 == -8 q^2 r^2 w^2 + 21 q^2 u[r]^2 + 480 kv \[Pi]^2 r^3 u[r]^3 + &#xA;      16 q^2 r w v[r] - 48 q^2 v[r]^2 + 69 q^2 r u[r] &#xA;\!\(\*SuperscriptBox[&quot;u&quot;, &quot;\[Prime]&quot;,&#xA;MultilineFunction-&amp;gt;None]\)[r], &#xA;    0 == -37 q^2 r w u[r] - 480 kv \[Pi]^2 r^4 w u[r]^2 + &#xA;      69 q^2 u[r] v[r] + 480 kv \[Pi]^2 r^3 u[r]^2 v[r] - 21 q^2 r^2 w &#xA;\!\(\*SuperscriptBox[&quot;u&quot;, &quot;\[Prime]&quot;,&#xA;MultilineFunction-&amp;gt;None]\)[r] + 21 q^2 r v[r] &#xA;\!\(\*SuperscriptBox[&quot;u&quot;, &quot;\[Prime]&quot;,&#xA;MultilineFunction-&amp;gt;None]\)[r] + 48 q^2 r u[r] &#xA;\!\(\*SuperscriptBox[&quot;v&quot;, &quot;\[Prime]&quot;,&#xA;MultilineFunction-&amp;gt;None]\)[r],&#xA;    u[r0] == u0,&#xA;    v[r0] == v0&#xA;    }, {u, v}, {r, r0, rRange[[2]]}];&#xA; If[Head[slv] =!= NDSolve, slv = slv[[1]], Abort[]];&#xA; Print@Plot[Evaluate[{u[r], v[r]} /. slv], {r, r0, rRange[[2]]}]&#xA; ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-01-07T20:10:19.780" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18609" PostTypeId="2" ParentId="17541" CreationDate="2015-01-08T06:50:12.937" Score="0" Body="&lt;p&gt;Thank you all for your kind attention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have solved the above challenge using the following approach.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Firstly, I used a nearest neighbor search algorithm to located the closest node to each of the field nodes;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The approach is more convenient for circular support domain - though it can be used for rectangular domain as well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I then obtain the distance between the field node and its nearest neighbor; this distance was then taken/used as the radius of the support domain.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I subsequently obtain the size of the influence domain by using the formula: dm(i) = dmax*delta (i) where i is the field node and delta is the radius of the support domain.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that instead of using a for loop over all the nodes, I simply perform the step 1 above for all the field nodes hence size of delta is (numnod*1) and dm is also (numnod*1) since dmax is a scalar which ranges from 2.0 - 4.0 for static analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;N.B: Please feel free to comment on my approach.&lt;/p&gt;&#xA;" OwnerUserId="11920" LastActivityDate="2015-01-08T06:50:12.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18610" PostTypeId="1" CreationDate="2015-01-08T07:01:57.667" Score="1" ViewCount="93" Body="&lt;p&gt;Consider the following system defined on the open interval (-1, 1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y_1' = c y_3 \\&#xA;     y_2' = c y_4 \\&#xA;     y_3' = -f(y_1, y_2)y_2 \\&#xA;     y_4' = f(y_1, y_2)y_1 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;given  $ y_3(-1) = 0 = y_3(1) \\&#xA;         y_4(-1) = 1, y_4(1) = -1 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the system &lt;strong&gt;cannot&lt;/strong&gt; be reduced to the standard form of two point BVP in $y_3, y_4$ by eliminating $y_1, y_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What methods/packages can I use to solve this system ?&lt;/p&gt;&#xA;" OwnerUserId="7728" LastActivityDate="2015-01-09T06:07:35.000" Title="methods for a peculiar BVP system" Tags="|finite-difference|ode|boundary-conditions|nonlinear-equations|collocation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18611" PostTypeId="1" CreationDate="2015-01-08T09:19:33.050" Score="9" ViewCount="213" Body="&lt;p&gt;Consider the situation where you want to solve a linear system using a preconditioned Krylov method, but applying the preconditioner itself involves solving an auxiliary system, which is done with another preconditioned Krylov method.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;On one extreme, you could run the inner solve to convergence within each step of the outer solve. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On the other extreme, you could not do the inner solve at all, but instead replace it with the inner preconditioner. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Somewhere in the middle, you could truncate the inner Krylov loop after some fixed number of iterations, or after a certain tolerance is achieved.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Empirically, I have come across situations where the first extreme is better, and different situations where the second extreme is better (in terms of total cost). However, I can find no clear reason why certain situations favor one strategy over another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any guidance or theory about when these different strategies are preferable? &lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2016-08-24T20:52:08.510" Title="Guidelines for nested preconditioners" Tags="|preconditioning|krylov-method|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18612" PostTypeId="2" ParentId="17465" CreationDate="2015-01-08T09:29:00.990" Score="1" Body="&lt;p&gt;Since the matrix is hermitian, you could use it as a hamiltonian to propagate it in imaginary time. That is, solve the following system of differential equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ i\frac{d \vec{V}}{dt}=M\vec{V}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general solution to this is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ V(t)=V_0e^{iMt}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you take your $\vec{V(t)} \cdot \vec{V(0)}$, fourier transform it, and the height and placement of the peaks will tell you the components along various eigenvectors and their associated eigenvalues. This is sometimes called &quot;the spectral method&quot; in ultrafast atomic physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have the eigenvalues, find the eigenvectors with whatever specific-eigenvalue solver you prefer. &lt;/p&gt;&#xA;" OwnerUserId="81" LastEditorUserId="81" LastEditDate="2015-01-08T22:50:56.410" LastActivityDate="2015-01-08T22:50:56.410" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18614" PostTypeId="2" ParentId="18610" CreationDate="2015-01-09T06:07:35.000" Score="2" Body="&lt;p&gt;It's true that you can't eliminate $y_1,y_2$ to obtain something where every variable you have has appropriate boundary values. However, you can&#xA;take derivatives on both sides of the first two equations to get&#xA;  $$y_1'' = c y_3' \\&#xA;     y_2'' = c y_4' \\&#xA;     y_3' = -f(y_1, y_2)y_2 \\&#xA;     y_4' = f(y_1, y_2)y_1 $$&#xA;and then eliminate $y_3',y_4'$:&#xA;$$&#xA;  y_1'' = -c f(y_1, y_2)y_2 \\&#xA;     y_2'' = c f(y_1, y_2)y_1 &#xA;$$&#xA;Now you just need boundary values. If you recall that $y_1'=cy_3$ and similarly $y_2'=cy_v$, then they read&#xA;$$ &#xA;  y_1'(-1) = 0 = y_1'(1) \\&#xA;         y_2'(-1) = 1, \\&#xA;         y_2'(1) = -1.&#xA;$$&#xA;In other words, you have a coupled nonlinear second-order system in $y_1,y_2$ with Neumann boundary values. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-01-09T06:07:35.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18615" PostTypeId="1" AcceptedAnswerId="18616" CreationDate="2015-01-09T10:54:09.370" Score="1" ViewCount="229" Body="&lt;p&gt;I am studying DG for conservation laws from &lt;a href=&quot;http://link.springer.com/book/10.1007/978-0-387-72067-8&quot; rel=&quot;nofollow&quot;&gt;this book&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Local inner product is defined like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(u,v)_{D^k} = \int_{D^k} uv dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the $L^2(D^k)$-norm as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;   (u,u)_{D^k} = ||u||^2_{D^k}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, consider the following problem where the flux $f(u) = au$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t} + \frac{\partial (au)}{\partial x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a little confused in example 2.3 they have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d}{dt}||u_h||^2_{\Omega,h} = -a(u^2(R)-u^2(L))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and they say that it is derived from multiplying the equation of the problem by $u(x,t)$, integration over the domain $D^k$, followed by integration by parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I take the equation of the problem and multiply it by $u(x,t)$ is it correct to have the following?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u^2}{\partial t} + \frac{\partial (au^2)}{\partial x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the square exponent in $u$. Or should it be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u \frac{\partial u}{\partial t} + u\frac{\partial (au)}{\partial x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps they are equivalent but don't have arguments to convince myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following with the integration over the domain $D^k$ I have&#xA;$$\int_{D^k} \frac{\partial u^2}{\partial t} dx + \int_{D^k} \frac{\partial (au^2)}{\partial x} dx = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now assuming I can change the derivate and integral (is there some theorem or condition the function must satisfy so that I am allowed to do this?) of the first term and using fundamental theorem of calculus in the second I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial}{\partial t} \int_{D^k} u^2 dx + a[u^2]^R_L = \frac{d}{dt} ||u||^2_{D^k} + a[u^2(R)-u^2(L)]^R_L$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which gives the result they have in the book (the notation is a little bit off don't know if I make a mistake on it but it seems to me that there are some typos in the book)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my last question is, where does the third step comes into play? I don't see where to apply integration by parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe my confusion comes from the notation and some theorems/conditions I am missing to do certain stuff with the integrals/derivatives. Could someone clarify all the small details I asked?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2015-01-10T16:03:05.557" Title="Discontinuous Galerkin energy method" Tags="|discontinuous-galerkin|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18616" PostTypeId="2" ParentId="18615" CreationDate="2015-01-09T13:03:47.013" Score="2" Body="&lt;p&gt;Don't skip steps. &#xA;$$&#xA;\frac{\partial u^2}{\partial t} = 2 u \frac{\partial u}{\partial t}&#xA;$$&#xA;by the Product Rule. The same applies to the flux term as well assuming $a$ is constant. Integration by parts is another name for the Divergence Theorem or the Fundamental Theorem of Calculus in 1D, so converting the second term from an interior integral to a boundary integral, which is point evaluation in 1D, &lt;em&gt;is&lt;/em&gt; an application of Integration by Parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Gibbs notation, the Divergence Theorem says for some vector function $w$:&#xA;$$&#xA;\int_\Omega \nabla \cdot w \; dx = \int_{\partial \Omega} w\cdot\hat n \; ds&#xA;$$&#xA;If you have $w=au$, where $a$ is a scalar function, then &#xA;$$&#xA;\nabla\cdot(au)=\nabla a \cdot u + a\nabla \cdot u&#xA;$$&#xA;Putting the two together gives:&#xA;$$&#xA;\int_\Omega \nabla\cdot(au) \; dx = \int_{\partial\Omega} \hat n \cdot au \; ds = \int_\Omega \nabla a \cdot u + a\nabla \cdot u \; dx&#xA;$$&#xA;which allows you to freely convert between either of the two terms on the right (which you may have derived from the physics of your problem) and boundary term. In 1D the surface integration term is integration around the zero-dimensional end points of your domain, which is just evaluation of the integrand at those endpoints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, the application of integration by parts is done in weak/finite element method derivations to move one derivative from  a second order term from the unknown solution onto the test function (which leads to symmetry for that term and lower-order derivatives). I don't see the need for it here, yet, but I can't access that book, so I don't know if it appears in latter parts of the derivation. &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-01-09T16:19:58.230" LastActivityDate="2015-01-09T16:19:58.230" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18617" PostTypeId="2" ParentId="18615" CreationDate="2015-01-09T16:02:10.023" Score="3" Body="&lt;p&gt;When you apply an energy method, multiply by the solution $u$ and integrate over the domain $D^k$ you obtain $$ \displaystyle\int_{D^k} \left(\frac{\partial u}{\partial t} + \frac{\partial (au)}{\partial x}\right)u \;dx = (u_t,u)_{D^k} + ((au)_x,u)_{D^k} = 0.$$ As was noted previously by Bill Barth we know $$ u\frac{\partial u}{\partial t} = \frac {1}{2}\frac{\partial (u^2)}{\partial t}.$$ If we assume that the value of $a$ is constant then it is also true that $$ u\frac{\partial (au)}{\partial x} = \frac {1}{2}\frac{\partial (au^2)}{\partial x}.$$ So we can rewrite the first equation (as you have previously) $$ \displaystyle\int_{D^k} \frac{\partial (u^2)}{\partial t} + \frac{\partial (au^2)}{\partial x}\;dx = 0. $$ From previous answers your concerns about integration by parts have been addressed (though if you assume $x$ dependence on the wavespeed $a$ the problem becomes slightly more complicated/subtle). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You final concern was can you exchange the time derivative and integration. In general, as long as a given function $f$ and $\frac{\partial f}{\partial t}$ are continuous in the space-time rectangle $[a,b]\times[c,d]$, then $$ \frac{\partial}{\partial t}\int_a^b f(x,t)\,dx = \int_a^b \frac{\partial f}{\partial t}(x,t)\,dx \quad\textrm{for }t\in[c,d]. $$ The idea of this result is simply that $$ \frac{1}{\Delta t}\left[\int_a^b f(x,t + \Delta t)\,dx - \int_a^b f(x,t)\,dx \right]= \int_a^b \frac{f(x,t + \Delta t) - f(x,t)}{\Delta t}\,dx, $$ from which one passes to the limit as $\Delta t \rightarrow 0$.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastEditorUserId="1057" LastEditDate="2015-01-10T16:03:05.557" LastActivityDate="2015-01-10T16:03:05.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18619" PostTypeId="2" ParentId="135" CreationDate="2015-01-09T19:37:56.293" Score="2" Body="&lt;p&gt;If you're still interested in this, I have a python script that will give you the (Abelian) point group (and symmetrically non redundant) atoms of any molecule within a specified tolerance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between my routine and a lot of others I've seen available is that the initial orientation is not important, making it useful to run in the results of geometry optimisations where you haven't specified an initial point group (as often, making an assumption like this can limit the geometry, forcing it to be symmetric and giving non equilibrium ground state.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're still interested, let me know, I'll share it here. &lt;/p&gt;&#xA;" OwnerUserId="8323" LastActivityDate="2015-01-09T19:37:56.293" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18621" PostTypeId="1" AcceptedAnswerId="18654" CreationDate="2015-01-10T07:19:04.973" Score="5" ViewCount="585" Body="&lt;p&gt;One very efficient way to solve Stokes equation with periodic boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;-\eta \nabla^{2} \bf{v} + \nabla p = f \\&#xA;\nabla \cdot \bf{v} = 0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is using the transverse projection operator (in Fourier space)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\tilde{\bf{v}} =  \frac{1}{\eta \bf{k}^2} \left (\bf{I} - \frac{k k}{\left | k \right |^2} \right ) \cdot \tilde{\bf{f}}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe a similar principle is behind the fractional step (i.e. splitting or projection operator) techniques for solving the Navier-Stokes equation (e.g. doi: 10.1016/0021-9991(89)90151-4).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we use a similar trick to help us out when the viscosity is not a constant function of space? In other words, what if our Stokes equation looks like this,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;- \nabla \cdot \left (\eta(\bf{r}) \nabla \bf{v} \right ) + \nabla p = \bf{f} \\&#xA;\nabla \cdot \bf{v} = 0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an efficient (non-iterative) method to solve such a system of equations?&lt;/p&gt;&#xA;" OwnerUserId="13082" LastEditorUserId="13082" LastEditDate="2015-01-13T22:05:35.240" LastActivityDate="2018-10-08T20:49:09.597" Title="Variable viscosity Stokes equation" Tags="|pde|fluid-dynamics|fourier-analysis|spectral-method|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18622" PostTypeId="1" AcceptedAnswerId="18675" CreationDate="2015-01-10T13:34:37.277" Score="0" ViewCount="730" Body="&lt;p&gt;I need to import in Matlab the results (like time histories of diplacement or frequency response at a specific point) obtained from a FEM analysis in Nastran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I ask Nastran to save the results on a file .dat and then I upload this file into Matlab. The fact is that Nastran organizes the data in a way that is not easy to work with, therefore I have to manually re-organize the file before loading it in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more fast and efficient way to do this operation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there toolboxes already implemented in Matlab that are compatible with FEM software outputs?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastActivityDate="2015-01-16T21:55:58.913" Title="Importing results of FEM analysis into Matlab" Tags="|matlab|finite-element|data-management|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18624" PostTypeId="2" ParentId="18621" CreationDate="2015-01-10T18:14:07.163" Score="0" Body="&lt;p&gt;Chapter five of the book &lt;em&gt;Boundary integral and singularity methods for linearized viscous flows&lt;/em&gt; (C. Pozrikidis) might be useful. He writes boundary integral equations for stokes flow in terms of the greens functions for homogeneous medium, for the case where one fluid region lies within another. This allows to handle a piece-wise constant viscosity. I don't know if this is useful for the method you are using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should mention, the &lt;em&gt;projection&lt;/em&gt; in the projection method/fractional-step method in solving the Navier-Stokes equation is completely unrelated to this.&lt;/p&gt;&#xA;" OwnerUserId="13144" LastActivityDate="2015-01-10T18:14:07.163" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18626" PostTypeId="1" CreationDate="2015-01-11T14:28:49.543" Score="2" ViewCount="143" Body="&lt;p&gt;I want to verify a finite-volume solver (SIMPLE-Algorithm) for the incompressible Navier-Stokes equations by using a manufactured solution. I use Dirichlet boundary conditions for the velocity at all boundaries. The manufactured solution for the velocities I use, is constructed as the curl of a vector field and thereby fulfills the continuity equation at any point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on how I choose the domain on which to solve the equations I run into convergence problems: The residual of the pressure correction equation stagnates after an initial decrease. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am approximating the mass fluxes through boundary faces using the midpoint rule and apparently this leads to a net loss/gain of mass considering the entire problem domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the normal behavior? Are there any remedies other than choosing a manufactured solution that identically vanishes at the boundaries? Would it be an option to provide the exact mass flux instead of the approximate (not sure if this is allowed, since the Dirichlet boundary condition only fixes the velocity)?&lt;/p&gt;&#xA;" OwnerUserId="11874" LastActivityDate="2015-01-11T14:28:49.543" Title="Arbitrary Choosing of the Solution Domain - Navier Stokes and Manufactured Solutions" Tags="|fluid-dynamics|boundary-conditions|finite-volume|computational-physics|navier-stokes|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18627" PostTypeId="2" ParentId="10678" CreationDate="2015-01-11T16:22:45.993" Score="-2" Body="&lt;p&gt;as per my understanding.. shape functions are nothing but the relation between field variables and the nodal points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that our earth is being pressurised with external loads and our earth is goin to crack. by analytical method, we use many formulas and find out that at some part (like assume Asia Continent) the earth is goin to crack.&#xA;Using FEM method, we divide earth into different countries, states and cities, we mesh each city, and finally join all the cities to form one globe called earth. &#xA;shape functions is the key which provides a bridge between the meshed cities to form a state and country and finally globe. it is the link that connects the mesh.&#xA;Once this is done load is applied and exact place can be found wher crack starts and that can be strengthened.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;hope this helped you.&lt;/p&gt;&#xA;" OwnerUserId="14261" LastActivityDate="2015-01-11T16:22:45.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18628" PostTypeId="1" AcceptedAnswerId="18631" CreationDate="2015-01-11T18:31:34.817" Score="2" ViewCount="590" Body="&lt;p&gt;I've been trying to figure out how to prove that the following equation is conservative&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{D^k} \partial_t(u^k) v_j^k + \partial_x(f) v_j^k dx = [(f -f^*) v^k_j]_{x^k}^{x^{k+1}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $v_j^k$ is a test function for element $k$, $f$ is the flux and $f^*$ is the numerical flux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone point me out to some reference?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="1553" LastEditDate="2015-01-12T15:18:14.433" LastActivityDate="2015-01-12T15:18:14.433" Title="How to prove the strong formulation for Discontinuous Galerkin is conservative?" Tags="|fluid-dynamics|discontinuous-galerkin|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18629" PostTypeId="1" CreationDate="2015-01-11T18:41:53.130" Score="0" ViewCount="334" Body="&lt;p&gt;I am working on a resource allocation problem, which is convex and has several constraints, and I want to compare the computational complexity of the following algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) The algorithm that uses iterative subgradient method through dual decomposition: It starts with initial values for dual variables and then, in each iteration, it finds the primal variables based on the dual variables and the Lagrangian function. Then it updates the dual variables using the subgradient of dual function (which is obtained using the primal variables in each iteration) and continues until the error between dual variables of two consecutive iterations is less than an epsilon. This algorithm takes a lot of iterations and has low convergence speed, which I think is mostly due to the fact that there are several dual variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) A heuristic algorithm in which I divide the problem into convex subproblems with one constraint and use Golden Section search method for each of the subproblems. This algorithm has clear steps, e.g. O(NM), for dividing the main problem into subproblems and also the Golden section search for each of the subproblems converges very fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to show the difference of these algorithms in terms of complexity through simulations as well as theory. In theory, how can I compare the computational complexity of these algorithms? In other words, is there a systematic way as in the case of algorithms with fixed number of iterations (in which we say that for example one algorithm is O(XY) and the other is O(Ylog(X))) to show the difference in computational complexity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in Advance.&lt;/p&gt;&#xA;" OwnerUserId="14265" LastActivityDate="2015-02-10T22:12:43.427" Title="Comparing computational complexity of convex optimization and a heuristic algorithm" Tags="|convex-optimization|complexity|heuristics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18630" PostTypeId="2" ParentId="18629" CreationDate="2015-01-11T21:18:52.180" Score="1" Body="&lt;p&gt;The running time of the algorithm is the sum of its subcomponents. Thus, the complexity is the asymptotically worst case complexity of the subcomponents. Experimentally, you just run the algorithms at different problem sizes (or numbers of constraints, or both) and plot/fit the resulting data. You need to make sure that you run the problem out to a large enough size that you demonstrate that you're really in the asymptotic regime and not dominated by lower-order but bigger-constant terms.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-01-11T21:18:52.180" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18631" PostTypeId="2" ParentId="18628" CreationDate="2015-01-11T21:19:53.727" Score="2" Body="&lt;p&gt;What you've written is considered the &quot;strong form&quot; of DG. If you integrate by parts the spatial term and rearrange you get&#xA;$$&#xA;\int_{D^k} \left(\partial_t(u^k)v^k - f \partial_x(v^k)\right) dx = -[f^* v^k]_{x^k}^{x^{k+1}} = 0.&#xA;$$&#xA;Taking $v = 1$ then gives&#xA;$$&#xA;\int_{D^k} \partial_t(u^k) dx = f^*(x^{k})-f^*(x^{k+1}). &#xA;$$&#xA;This is a restatement of conservation - that change in the amount of $u^k$ over the element $D^k$ is equal to flux in minus flux out.  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-01-11T21:19:53.727" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18633" PostTypeId="2" ParentId="18628" CreationDate="2015-01-12T08:18:40.190" Score="1" Body="&lt;p&gt;The strong and weak form of the DG approximation are equivalent approximations (full details and analysis are given in &lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs10915-010-9372-3&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;). As was pointed out by JLC the conservation of the DG approximation can be explicitly seen from the weak form and then, by equivalence, conservation of the strong form follows. The one-dimensional case is relatively straightforward, but in higher dimensions one must be concerned with the mapping of the approximation.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastEditorUserId="1057" LastEditDate="2015-01-12T08:33:48.777" LastActivityDate="2015-01-12T08:33:48.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18635" PostTypeId="2" ParentId="5600" CreationDate="2015-01-12T10:05:55.790" Score="2" Body="&lt;p&gt;Matlab '\' operator is highly efficient due to top notch programming. You may see some of the idea and how they used all possible short cuts in Timothy Davis's book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In matlab you can use gmres, which is well developed and stable. Probably minres, which theoretically should be ideal for your case, may not be that reliable (at least my experience says so). You should be getting equal or higher efficiency from matlab gmres if&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Your system is large enough (at least few thousand by few thousand).&lt;/li&gt;&#xA;&lt;li&gt;If you are using right kind of parameters(RESTART, MAXIT, X0). No clear guidelines are available for this. Use your experience.&lt;/li&gt;&#xA;&lt;li&gt;Use a good preconditioner. You may use ILU or even cheaper, a block Jacobi. This will cut the effort considerably.&lt;/li&gt;&#xA;&lt;li&gt;MOST IMPORTANT: If your matrix is sparse, use matlab sparse format. Matlab gmres is ideally built for that. It will cut cost to a large extent.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For even bigger systems, use a tool like PETSc.&lt;/p&gt;&#xA;" OwnerUserId="2019" LastActivityDate="2015-01-12T10:05:55.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18636" PostTypeId="1" CreationDate="2015-01-12T10:41:08.703" Score="1" ViewCount="144" Body="&lt;p&gt;I am trying to do Monte Carlo simulation for a large problem which requires eigensolution of a matrix for each sample. The matrix itself is quite large so much so that I want the eigensolution itself to be parallel as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will probably require dividing the processors in subgroups and then within each group this eigensolution will be parallelized. I am trying to use PETSc and SLEPC for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple parallelization was possible for me but I am unable make use of such subgrouping. Does anybody know, how this can be done?&lt;/p&gt;&#xA;" OwnerUserId="2019" LastActivityDate="2015-01-13T20:38:06.413" Title="Parallel Monte Carlo simulation using PETSc" Tags="|parallel-computing|petsc|eigensystem|eigenvalues|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18638" PostTypeId="1" AcceptedAnswerId="18644" CreationDate="2015-01-12T15:44:01.403" Score="3" ViewCount="467" Body="&lt;p&gt;$f(x)= \frac{1}{1+x^2}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and when I computed the interpolating polynomial of 5 equally spaced points in [-5,5] I got &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ p(x)= 0.0053x^4 -0.1711x^2 +1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I need to estimate the error in the interpolant using the error formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|f(x)-p(x)|=\left|\frac{f^{(5)}(\xi)}{(5)!}\prod_{i=0}^4(x-x_i)\right|$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to self learn this topic and I want to know if there is a way I can use f(x)-p(x) to solve this instead of using the right side of the formula. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the right side is to be used, how is it possible? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The latter part of the question asks to do the sane for 17 equally spaced points so I don't think I am asked to find the 17th derivative of $ f$ .&lt;/p&gt;&#xA;" OwnerUserId="11767" LastActivityDate="2015-01-13T09:39:38.267" Title="Error of interpolating polynomial" Tags="|interpolation|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18639" PostTypeId="1" CreationDate="2015-01-12T16:35:05.793" Score="3" ViewCount="547" Body="&lt;p&gt;I could find the common Moore-Penrose algorithm, but I couldn't find the Drazin or the Bott-Duffin generalised inverse, except for some very specific cases, useless for my studying purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a Matlab or C code for at least one of the two algorithms?&lt;/p&gt;&#xA;" OwnerUserId="14273" LastActivityDate="2018-03-15T06:54:41.427" Title="Algorithm code for Drazin and Bott-Duffin inverse (Matlab or C)" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18640" PostTypeId="2" ParentId="17610" CreationDate="2015-01-12T19:19:07.267" Score="5" Body="&lt;p&gt;You could also take a look at &lt;a href=&quot;https://www.rescale.com&quot;&gt;Rescale&lt;/a&gt;--a web-based, on-demand HPC computing platform.&lt;/p&gt;&#xA;" OwnerUserId="14274" LastActivityDate="2015-01-12T19:19:07.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18642" PostTypeId="1" AcceptedAnswerId="18716" CreationDate="2015-01-13T01:47:22.687" Score="3" ViewCount="252" Body="&lt;p&gt;We have the Schrodinger eqn&#xA;\begin{equation}(−\Delta+V(r))R(r)=E R(r)\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we can take $V(r)=-k/r$ for the beginning and we impose on the reduced radial function $u(r)=r R(r)$ the boundary conditions $u(0)=u(\infty)=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is to:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Solve numerically inwards and outwards the radial equation, and determine the eigenenergy by imposing the vanishing of the wronskian at some intermediate point.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How to proceed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What is meant by solving &quot;inwards and outwards&quot; and what does &quot;imposing the vanishing of the wronskian at some intermediate point&quot; mean for us? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) What numerical method shall I use? The Numerov method is probably not applicable, right?&lt;/p&gt;&#xA;" OwnerUserId="12005" LastEditorUserId="12005" LastEditDate="2015-01-13T02:11:06.433" LastActivityDate="2015-01-22T20:07:25.847" Title="Solve numerically inwards and outwards the radial equation?" Tags="|quantum-mechanics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18643" PostTypeId="1" CreationDate="2015-01-13T03:14:48.113" Score="6" ViewCount="107" Body="&lt;p&gt;I have a variational problem where the unknown function is a periodic path $\gamma:[0,1)\to\mathbb{R}^2$, and the functional is&#xA;$$ \int_0^1\left( \tfrac12\|\dot\gamma(s)\|^2 + \mathcal{F}[\gamma]\right)\,ds. $$&#xA;Here $\mathcal F$ depends on values of $\gamma$ at different values of $s$, so it's not just a function of $\gamma(s)$, like so:&#xA;$$ \mathcal{F}[\gamma](s) = \sum_{0&amp;lt;j&amp;lt;n}\frac{1}{\|\gamma(s+j/n)-\gamma(s)\|}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current approach is to expand $\gamma$ into its Fourier components $e^{2\pi \mathrm{i} k s}$ to ensure periodicity, and use a Newton-type method to minimize the functional as a function of the Fourier coefficients. I can calculate the functional and its gradient efficiently for any given set of Fourier coefficients, in part because the integral can be done with the trapezoid rule, which converges quickly for periodic functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the problems I have is that it is difficult to explore all the different minima the functional might have, because apart from some linear constraints I can impose on the Fourier coefficients, I have very little control over which local minimum I get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good graduate/research-level book on numerical variational methods that would explain the different things I should try? Survey articles?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit:&lt;/em&gt; the exact functional I'm using comes from the n-body choreography problem, but I'm interested more in the general ideas here than in this specific application.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-01-13T21:14:04.077" LastActivityDate="2015-01-13T21:14:04.077" Title="Reference request for numerical variational method" Tags="|optimization|reference-request|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18644" PostTypeId="2" ParentId="18638" CreationDate="2015-01-13T09:39:38.267" Score="7" Body="&lt;p&gt;For a polynomial interpolant we have the formula for the pointwise error $$ E_n(x) = f(x) - p_n(x) = \frac{f^{(n+1)}}{(n+1)!}\prod_{i=0}^n(x-x_i),$$ where the $x_i$ are the interpolation knots. In general, we want to work with the right hand side of the error formula (a derivation appears in the book by &lt;a href=&quot;http://www.springer.com/mathematics/applications/book/978-3-540-34658-6&quot; rel=&quot;noreferrer&quot;&gt;Quarteroni et. al.&lt;/a&gt;) as the function we interpolate is typically not a polynomial or rational function. Some important remarks for the error $E_n$ are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$E_x(x)$ tends to be very oscillatory.&lt;/li&gt;&#xA;&lt;li&gt;If $f^{(n+1)}(x)$ is nicely bounded on $[a,b]$, then the error can be estimated.&lt;/li&gt;&#xA;&lt;li&gt;Problems if $f^{(n+1)}(x)$ grows faster than $(n+1)!$ or if $\prod_{i=0}^n(x-x_i)$ is large.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You are correct to assume that for a large number of interpolation knots it becomes unwieldy to compute the derivative in the error formula, e.g for $n=17$ we would need $f^{(18)}(x)$. Instead we could compute a short sequence of the derivative and attempt to notice a pattern and determine if the value for $E_n(x)$ will be &quot;well-behaved&quot; (i.e. bounded). We'll start at $n=4$ since that was a previous problem $$ \vdots\\E_4(x) = \frac{1}{5!}\left(-\frac{240x(3x^4 - 10x^2+3)}{(1+x^2)^6}\right)\prod_{i=0}^4(x-x_i), \\ E_5(x) = \frac{1}{6!}\left(\frac{720(7x^2 - 35x^4 + 21x^2-1)}{(1+x^2)^7}\right)\prod_{i=0}^5(x-x_i), \\ E_6(x) = \frac{1}{7!}\left(-\frac{40320x(x^6 - 7x^4+7x^2-1)}{(1+x^2)^8}\right)\prod_{i=0}^6(x-x_i), \\ \vdots$$&#xA;This small slice of error calculations reveals that the growth of the terms $f^{(n+1)}(x)$ and $(n+1)!$ balance nicely. But this means that the value of $E_n$ will become large near the endpoints because the value of $f^{(n+1)}(x)/(n+1)!$ will be small but $\prod_{i=0}^n(x-x_i)$ will be large. Such a result for this particular f(x) on $[-5,5]$ has been noted previously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem you are asked to consider is known as &lt;em&gt;Runge's counterexample&lt;/em&gt; which a maliciously chosen problem that highlights the drawbacks of polynomial interpolation on equally spaced knots. In fact, it can be checked that some points $x$ exists within the interpolation interval such that $$ \lim_{n\rightarrow\infty} \left|f(x) - p_n(x)\right|\ne 0.$$ Even more alarming is it is possible to show that Lagrange interpolation diverges (see &lt;a href=&quot;http://www.google.de/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CCEQFjAA&amp;amp;url=http%3A%2F%2Falvand.basu.ac.ir%2F~dezfoulian%2Ffiles%2FNumericals%2FAnalysis.Of.Numerical.Methods_Isaacson.Keller_0486680290.pdf&amp;amp;ei=5-G0VMW2PILtO96IgfAE&amp;amp;usg=AFQjCNFAhg0binmn00XIkQig0yYnmhoj4Q&amp;amp;sig2=r770jIJdkQELEHhk5qxnIg&amp;amp;bvm=bv.83339334,d.ZWU&quot; rel=&quot;noreferrer&quot;&gt;Isaacson and Keller&lt;/a&gt; for a complete proof). If we plot the polynomial interpolate of the Runge function for uniformly spaced knots we'll see significant, spurious oscillations (particularly near the endpoints of the interval). These oscillations are often referred to as Runge phenomena.&lt;img src=&quot;https://i.stack.imgur.com/di2ud.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we remedy this problem for the function $f(x)$? It turns out yes we can, if we choose a set of interpolation knots which are not uniformly spaced. For example, let's choose the Tschebyscheff (Chebyshev in English translations) points of the second kind $$ x_j = \cos\left(\frac{j\pi}{n}\right),\quad j=0,\ldots,n.$$ Then we can visually see that the Runge phenomena has been removed (and one could show that convergence restored using analysis tools from Isaacson and Keller).&lt;img src=&quot;https://i.stack.imgur.com/VWwqI.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1057" LastActivityDate="2015-01-13T09:39:38.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18646" PostTypeId="1" AcceptedAnswerId="18650" CreationDate="2015-01-13T16:47:02.407" Score="3" ViewCount="572" Body="&lt;p&gt;I am trying to solve the Eigenvalue problem&#xA;$$&#xA;x^2 y''+ x y' + x^2 y = \lambda^2 y\,,\quad x\in(0,1)\,,\quad&#xA;y(0)=0\,,\quad&#xA;y'(1)=y(1)&#xA;$$&#xA;The differential equation is the Bessel equation. The solution is given by the Bessel function of the 1st kind $y(x)=J_\lambda(x)$. Bessel functions of the 2nd kind is omitted because of their singularity at $x=0$ . The Eigenvalue $\lambda$ has to be determined using the boundary condition at $x=1$. Using the identity&#xA;$$J'_\lambda(x)=\frac{1}{2}\left(J_{\lambda-1}\left(x\right)-J_{\lambda+1}\left(x\right)\right)$$&#xA;the boundary condition at $x=1$ can be rewritten as&#xA;$$J_{\lambda-1}\left(1\right)-2J_{\lambda}\left(1\right)-J_{\lambda+1}\left(1\right)=0\,.$$&#xA;Now I would like to determine the $\lambda$ with the equation above using the fzero-routine of MATLAB/ GNU Octave. Code below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% initial guess&#xA;xguess=1;&#xA;%% function handle&#xA;f=@(l) besselj(l-1,1)-2*besselj(l,1)+besselj(l+1,1)&#xA;%% set tolerance&#xA;opts=optimset('TolX',1e-12);&#xA;%% determine zero&#xA;xzero=fzero(f,xguess,opts);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you plot the function f, &lt;a href=&quot;http://www.wolframalpha.com/input/?i=BesselJ%28x-1%2C1%29-2*BesselJ%28x%2C1%29-BesselJ%28x%2B1%2C1%29&quot; rel=&quot;nofollow&quot; title=&quot;wolfram alpha&quot;&gt;click here&lt;/a&gt;, you observe that most of the zeros are in $-\infty&amp;lt;x&amp;lt;0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get a sorted sequence $\lambda_{k+1}&amp;lt;\lambda_k$, I would like to search from the &quot;last&quot; zero only in negative $x$-direction to the &quot;next&quot; one. Is there any way to implement this in the fzero-routine?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank in advance!&lt;/p&gt;&#xA;" OwnerUserId="14284" LastEditorUserId="1804" LastEditDate="2015-01-13T22:18:19.313" LastActivityDate="2015-01-14T10:21:24.947" Title="Bessel EVP and fzero" Tags="|matlab|nonlinear-equations|special-functions|roots|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="18648" PostTypeId="2" ParentId="18642" CreationDate="2015-01-13T17:11:11.327" Score="1" Body="&lt;p&gt;To solve numerically you can use finite differences in MATLAB or GNU Octave on a finite interval. The Laplacian $\Delta$ in your equation is just the 2nd derivative $R''$. If you use 2nd order symmetric differences you get the stencil&#xA;$$y''_i=\frac{-y_{i-1}+2y_{i}-y_{i+1}}{h^2}\,.$$&#xA;If you discretize the other term on the left-hand side (diagonal matrix) you will get a matrix Eigenvalue problem if the form&#xA;$$Ay_h=\lambda y_h\,.$$&#xA;Don't forget to think about the boundary conditions as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry but I don't how to do it on infinite intervals. You may use a transformation on a finite intervall e.g. $(0,1)$ such as&#xA;$$z=\frac{x}{1-x}\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hint: In theory of ODEs the Wronskian is the determinate of the fundamental matrix $\Phi$.&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2015-01-13T17:11:11.327" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18649" PostTypeId="2" ParentId="18636" CreationDate="2015-01-13T20:38:06.413" Score="2" Body="&lt;p&gt;If you have an idea about how to statically partition your problem into subgroups, you could try partitioning the MPI processes into these subgroups, and then creating an MPI communicator for each subgroup. Most PETSc &amp;amp; SLEPc object creation routines take a communicator as an argument, so instead of using &lt;code&gt;MPI_COMM_WORLD&lt;/code&gt;, you could use the subcommunicators you defined via your partitioning.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-01-13T20:38:06.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18650" PostTypeId="2" ParentId="18646" CreationDate="2015-01-13T22:35:23.323" Score="5" Body="&lt;p&gt;As you can see in this plot of $\log|f(\lambda)|$, &#xA;$$ f(\lambda) = J_{\lambda-1}(1) - 2J_\lambda(1) -J_{\lambda+1}(1), $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XNvPR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the roots $\lambda_k$ are really regular, and are approximately equal to $-k$ (starting from $k\geq0$, $\lambda_0=1.23219$ is an exception).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the way to get the $k$-th eigenvalue ($k\geq1$), is to bracket the root in $[-k-\frac12,-k+\frac12]$ and use &lt;code&gt;fzero&lt;/code&gt; solver in that interval (it accepts a bracketing interval as the initial guess and stays within that interval).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the magnitude of the function grows very quickly (something like $(2\lambda/e)^\lambda\sin\pi\lambda$), so once the function becomes quite large (for large $k$), it may be better to use $\lambda_k=-k$ as an approximation.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-01-14T07:54:49.000" LastActivityDate="2015-01-14T07:54:49.000" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18651" PostTypeId="2" ParentId="18646" CreationDate="2015-01-14T10:21:24.947" Score="0" Body="&lt;p&gt;Here is the code I used now.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kmax=5;&#xA;lambda=zeros(kmax,1);&#xA;%% function handle&#xA;f=@(l) besselj(l-1,1)-2*besselj(l,1)-besselj(l+1,1);&#xA;%% set tolerance&#xA;opts=optimset('TolX',1e-12);&#xA;%% compute first positive eigenvalue&#xA;lambda(1)=fzero(f,[0.5,1.5],opts);&#xA;%% compute negative eigenvalues&#xA;for k=1:(kmax-1)&#xA;  [lambda(k+1),~,fl]=fzero(f,[-k-0.5,-k+0.5],opts);&#xA;  if not(fl==1)&#xA;    error('iteration    did     not     convert.')&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By the way, I would like to use this to verify my finite difference method.&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2015-01-14T10:21:24.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18654" PostTypeId="2" ParentId="18621" CreationDate="2015-01-14T22:49:46.020" Score="1" Body="&lt;p&gt;My suspicion is that there isn't a good non-iterative way to do it since the linear operator isn't diagonal in Fourier space.  However, it is possible to solve equations like these without total disaster by iterating, since you can choose fairly large iterative steps if you use semi-implicit methods.  To do this, you have to break up the variable viscosity term into two pieces.  Here's a paper that does this for a Cahn-Hilliard problem with a variable mobility - the theory is the same: &lt;a href=&quot;http://journals.aps.org/pre/abstract/10.1103/PhysRevE.60.3564&quot; rel=&quot;nofollow&quot;&gt;http://journals.aps.org/pre/abstract/10.1103/PhysRevE.60.3564&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4494" LastActivityDate="2015-01-14T22:49:46.020" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18655" PostTypeId="2" ParentId="18621" CreationDate="2015-01-14T23:58:52.453" Score="1" Body="&lt;p&gt;I would think that multigrid-preconditioned iterative methods would be most competitive. See, for instance, &lt;a href=&quot;http://users.ices.utexas.edu/~carsten/papers/BursteddeGhattasStadlerEtAl09.pdf&quot; rel=&quot;nofollow&quot;&gt;Parallel scalable adjoint-based adaptive solution of variable-viscosity&#xA;Stokes flow problems, Burstedde, et al., CMAME, 2009&lt;/a&gt; and &lt;a href=&quot;http://arxiv.org/pdf/1308.4605.pdf&quot; rel=&quot;nofollow&quot;&gt;Efficient variable-coefficient finite-volume Stokes solvers, Cai, et al., Communications in Computational Physics, 2014&lt;/a&gt; for a few examples, along with some references. Both approaches are highly scalable, but FEM approaches seem to have more literature associated with effectively preconditioned solvers than FVM approaches.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-01-14T23:58:52.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18656" PostTypeId="1" CreationDate="2015-01-15T04:33:01.400" Score="6" ViewCount="4879" Body="&lt;p&gt;What method of numerical solving ODEs is better? BDF2 or TR-BDF2?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Namely, what advantages has TR-BDF2 over BDF2?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The BDF2 method requires the values of $y_{n-1}$ and $y_n$ for computing $y_{n+1}$ but we can use, for example, the trapezoidal method for $n = 0$ and BDF2 on next steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The TR-BDF2 method computes an auxiliary value $y_{n+1/2}$ with the trapezoidal method and applies the BDF2 for computing $y_{n+1}$ by using $y_n$ and $y_{n+1/2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TR-BDF2 for solving $y' = f(y)$ represents the following scheme:&#xA;$$&#xA;y_{n+1/2} = y_n + \frac{\tau}{4}(f(y_n) + f(y_{n+1/2})),&#xA;$$&#xA;$$&#xA;y_{n+1} = \frac{1}{3}(4y_{n+1/2} - y_n + \tau f(y_{n+1})).&#xA;$$&#xA;Here $\tau$ is a step size. The both stages are implicit. The first stage is the trapezoidal method with the step size $\tau/2$ and the second stage is the BDF2 with the step size $\tau/2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPD&lt;/strong&gt; Edwards et al. in the paper &lt;a href=&quot;http://libgen.in/scimag/get.php?doi=10.1016%2Fj.jcp.2010.10.035&quot; rel=&quot;nofollow&quot;&gt;Nonlinear variants of the TR/BDF2 method for thermal radiative diffusion&lt;/a&gt; point out that BDF2 has undesirable conservation properties. Could you explain please how can this influence the computation accuracy?&lt;/p&gt;&#xA;" OwnerUserId="11531" LastEditorUserId="11531" LastEditDate="2015-01-15T09:24:35.453" LastActivityDate="2017-07-28T22:42:36.377" Title="BDF2 and TR-BDF2: what is better?" Tags="|ode|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18657" PostTypeId="1" AcceptedAnswerId="18662" CreationDate="2015-01-15T05:20:16.180" Score="8" ViewCount="1666" Body="&lt;p&gt;I am implementing a finite element solver in MATLAB and I have the following problem. Let's say I have a mesh $\mathcal{T}_1$ with triangular elements on a rectangular domain $\Omega\subset\mathbb{R}^2$. I know the positions of all the nodes on the mesh, and I have an approximate solution $\hat{u}_1(x,y)$ to my PDE on this mesh (this approximation is stored as a vector of real numbers for each node, and then interpolated between nodes). To be exact, I use piecewise affine basis functions, like seen in this &lt;a href=&quot;http://en.wikipedia.org/wiki/Piecewise_linear_function#Notation&quot;&gt;illustration&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I now create a new mesh $\mathcal{T}_2$ (which may or may not be more coarse/fine, and positions of nodes may be changed), is there a way I can &quot;project&quot; the data of $\hat{u}_1(x,y)$ onto the new mesh $\mathcal{T}_2$, call it $\hat{u}_2(x,y)$, so that $\hat{u}_1(x,y)\approx \hat{u}_2(x,y)$? In short I want $u_2(x,y)$ to be the &quot;best representation&quot; of $u_1(x,y)$ at the new mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, let's say there is a disk of radius $1$ around the origin, $B_1(0,0)\subset\Omega$, and perhaps in the initial mesh $\mathcal{T}_1$  the nodes $v_3^{(1)},v_5^{(1)},v_6^{(1)},$ and $v_8^{(1)}$ are contained in this disk. However in the new mesh $\mathcal{T}_2$, perhaps a coarser mesh, there are only $2$ nodes contained in the disk, say $v_2^{(2)}$ and $v_3^{(2)}$. I now want a solution that interpolates in the nodes, and is approximately equal to the solution on the old mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that if the new mesh is much more coarse, of course the solutions can't be that similar, but I just want a decent approximation. If anyone knows any existing MATLAB code to do this, or anything to help me implement it myself, I'll be happy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a bit difficult to explain exactly what I mean in just words, so if it is not clear at all what I'm asking, please say so, and I'll try to clarify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks &lt;/p&gt;&#xA;" OwnerUserId="10398" LastEditorUserId="10398" LastEditDate="2015-01-15T06:14:52.907" LastActivityDate="2015-01-15T14:43:10.983" Title="Projecting Finite Element solution onto new mesh" Tags="|matlab|finite-element|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="18658" PostTypeId="1" CreationDate="2015-01-15T05:26:16.813" Score="5" ViewCount="161" Body="&lt;p&gt;Is there a reference on the effect of subdomain topology on performance of the overlapping additive Schwarz method for (high order) finite elements?  For example, taking subdomains to be vertex patches (elements directly connected through a vertex) differs from taking a subdomain to be an element + all its neighbors connected through faces (the former I believe gave me iteration counts independent of subdomain size, the former did not for high order). &lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-01-16T20:11:20.190" LastActivityDate="2015-02-15T20:53:33.367" Title="Effect of subdomain topologies on overlapping additive Schwarz?" Tags="|finite-element|preconditioning|domain-decomposition|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18659" PostTypeId="2" ParentId="18656" CreationDate="2015-01-15T07:21:32.433" Score="3" Body="&lt;p&gt;The main advantage appears to be that you can combine the trapezoidal rule and BDF2 steps to create a one-step method. Using BDF2 alone requires bootstrapping to get the first step. Edwards, et al's argument is that the two-step nature of BDF2 means that the conservation relationship within a time step has a memory effect: it depends on previous time steps in a nontrivial way. The conservation relationship within a time step for TRBDF2 is a simple three-point average. Apparently, according to Edwards, these conservation relationships are important for calculating derived quantities (&quot;source and sink terms&quot;), and doing that with TRBDF2 is simpler than BDF2.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-01-15T09:47:03.947" LastActivityDate="2015-01-15T09:47:03.947" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18660" PostTypeId="2" ParentId="14566" CreationDate="2015-01-15T09:09:08.613" Score="0" Body="&lt;p&gt;I have used matplotlib's &lt;em&gt;plot_streamlines&lt;/em&gt; function in the past to create streamline plots. There is a nice introduction to the functionality &lt;a href=&quot;http://tonysyu.github.io/plotting-streamlines-with-matplotlib-and-sympy.html#.VLd9gie9A7Y&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. If you want more control over the creation of the plot &lt;a href=&quot;https://stackoverflow.com/questions/8296617/how-to-plot-a-streamlines-when-i-know-u-and-v-components-of-velocitynumpy-2d&quot;&gt;this thread&lt;/a&gt; at StackOverflow offers more flexible techniques (and source code) for creating streamline plots in Python.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-01-15T09:09:08.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18662" PostTypeId="2" ParentId="18657" CreationDate="2015-01-15T12:51:13.900" Score="8" Body="&lt;p&gt;When you want to say that you want $u_2$ to be the &lt;em&gt;best&lt;/em&gt; approximation of $u_1$ on mesh ${\cal T}_2$, then you have to define what you mean by &quot;best&quot;. Let's assume you define it as that function that has the least $L_2$ error, i.e.,&#xA;$$&#xA;  u_2 = \arg\min_{\phi_h \in V_2} \|\phi_h-u_1\|_{L_2(\Omega)}.&#xA;$$&#xA;Then indeed $u_2$ is the $L_2$ projection of $u_1$ onto the finite element space $V_2$ defined on ${\cal T}_2$, and it satisfies the condition&#xA;$$&#xA;  (\phi_i,u_2) = (\phi_i,u_1), \qquad i=1,\ldots,\textrm{dim}(V_2)&#xA;$$&#xA;where $\phi_i$ are the shape functions defined on mesh ${\cal T}_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to compute this, you expand $u_2(x,y)=\sum_j U_{2,j} \phi_j(x,y)$:&#xA;$$&#xA;  \sum_j (\phi_i,\phi_j) U_{2,j} = (\phi_i,u_1), \qquad i=1,\ldots,\textrm{dim}(V_2),&#xA;$$&#xA;so you have to solve an equation of the form $MU_2 = F$ where $M$ is the mass matrix on the ${\cal T}_2$ and &#xA;$$&#xA;  F_i = (\phi_i,u_1) = \int_\Omega \phi_i(x,y)u_1(x,y) \; dx \; dy.&#xA;$$&#xA;The problematic part is the evaluation of this integral. Of course, you will do it using quadrature, but that will require you to evaluate $u_1$ at the quadrature points defined on mesh ${\cal T}_2$. If ${\cal T}_1$ is a uniform mesh, then evaluating $u_1$ at arbitrary points is not difficult, but if it is an unstructured mesh, then this is in general a difficult and expensive problem.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-01-15T12:51:13.900" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="18663" PostTypeId="1" AcceptedAnswerId="18700" CreationDate="2015-01-15T14:31:17.297" Score="0" ViewCount="85" Body="&lt;p&gt;Suppose $f'(x_1),\ f'(x_2),\ f'(x_3)$ are given, how to give a polynomial interpolation $p(x)$ such that $p'(x_1)= f'(x_1),\ p'(x_2)=f'(x_2),\ p'(x_3)=f'(x_3)$? And how to give an error analysis?&lt;/p&gt;&#xA;" OwnerUserId="14304" LastEditorUserId="9667" LastEditDate="2015-01-20T21:49:29.567" LastActivityDate="2015-01-21T05:59:50.587" Title="How to present a polynomial interpolation if the first order information of function f(x) is given?" Tags="|interpolation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18664" PostTypeId="2" ParentId="18657" CreationDate="2015-01-15T14:43:10.983" Score="2" Body="&lt;p&gt;Since you are using MATLAB, I suggest you take a look at the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/scatteredinterpolant-class.html&quot; rel=&quot;nofollow&quot;&gt;scatteredInterpolant&lt;/a&gt; class. You construct the interpolant by giving it the x-y locations and solution values of points on the first mesh. Then you ask for interpolated values of the solution at points in the new mesh (e.g. new element integration points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will &lt;strong&gt;not&lt;/strong&gt; provide a &quot;best&quot; interpolation in the minumum-L2-error sense as Wolfgang Bangerth has described. It simply does a local linear interpolation based on points in the original mesh. However it is easy to use, quite fast, and is often satisfactory.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-01-15T14:43:10.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18665" PostTypeId="1" CreationDate="2015-01-15T15:09:17.623" Score="1" ViewCount="484" Body="&lt;p&gt;I'm programming a beam finite element model by following a book (Nonlinear Finite Element Analysis of Solids and Structures Volume 2, in case you're wondering!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've come across the following statement, which I've paraphrased slightly (unless mentioned matrices, denoted by $[A]$, are of dimension $3 \times 3$ and vectors, denoted by $\{a\}$ are of size $3 \times 1$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${\{a\}}^T[A]\{\delta\beta\}=\{b\}^T\{c\}   \tag{1}\label{1}$$&#xA;$${\{d\}}^T[E]\{\delta\beta\}=\{f\}^T\{g\}   \tag{2}\label{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equations \eqref{1} and \eqref{2} provide two equations in the three components of $\{\delta\beta\}$. If we use the procedure of Rankin and Brogan [R1.16] which was discussed at the beginning of Section 17.1.6, we can then differentiate equation (17.59a) to obtain equation \eqref{3}:&#xA;$${\{h\}}^T[L]\{\delta\beta\}=\{m\}^T\{n\}   \tag{3}\label{3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using equations \eqref{1}-\eqref{3} one can obtain $[V]^T$ using \eqref{4}, where $[V]^T$ is a $3 \times 12$ matrix and $\{\delta p\}$ is a $12 \times 1$ column vector (both $\{\delta\beta\}$ and $\{\delta p\}$ are known at this stage):&#xA;$$\{\delta\beta\}=[V]^T\{\delta p\} \tag{4}\label{4}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to find $[V]^T$. I think I've correctly managed to solve for the components of $\{\delta\beta\}$, but I'm at a loss as to how to get a $3 \times 12$ matrix from a $3 \times 1$ vector and a $12 \times 1$ vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MatLab you can solve for the pseudo inverse using either $\{\delta\beta\}*\text{pinv}(\{\delta p\})$ or $\{\delta\beta\}/\{\delta p\}$ but although these both give $3 \times 12$ matrices they are not correct (based on numerical tests I've done).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="7637" LastEditorUserId="20688" LastEditDate="2017-04-02T01:01:23.773" LastActivityDate="2017-04-02T01:01:23.773" Title="Solve for a matrix given two vectors" Tags="|linear-algebra|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18667" PostTypeId="1" CreationDate="2015-01-15T16:56:29.417" Score="8" ViewCount="516" Body="&lt;p&gt;I tried to go to the primary sources in order to understand how to use Butcher tables to simplify the algebra I need to do when using Taylor series to find the order of accuracy of a scheme, for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, maybe because of a lack of relevant background, I found it particularly tough to understand how to utilize Butcher tables from Butcher's book. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there good, relatively self-contained (i.e. minimum prerequisites) books or tutorials that cover the necessary mathematics I need to utilize Butcher tables?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="123" LastEditDate="2015-06-07T10:34:59.930" LastActivityDate="2015-06-07T10:34:59.930" Title="Good tutorials on how to use Butcher tables?" Tags="|numerical-analysis|reference-request|runge-kutta|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18668" PostTypeId="2" ParentId="14458" CreationDate="2015-01-15T20:54:02.807" Score="5" Body="&lt;p&gt;Take a look at active subspaces, e.g., Active Subspace Methods in Theory and Practice: &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/130916138&quot; rel=&quot;nofollow&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/130916138&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And a PDF here:&#xA;&lt;a href=&quot;http://inside.mines.edu/~pconstan/docs/constantine-asm.pdf&quot; rel=&quot;nofollow&quot;&gt;http://inside.mines.edu/~pconstan/docs/constantine-asm.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a SIAM book (Active Subspaces: Emerging Ideas for Dimension Reduction in Parameter Studies) coming out in March. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose $f$ maps $\mathbb{R}^m$ to $\mathbb{R}$. One model that addresses what you're looking for is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(x) \approx g(W^T x),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $W$ is $m\times n$, and $g$ is a function that maps $\mathbb{R}^n$ to $\mathbb{R}$. In active subspaces, $W$ is the first $n$ eigenvectors of the symmetric, positive semidefinite matrix,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C = \int (\nabla f)(\nabla f)^T\rho\,dx,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\rho=\rho(x)$ is a given density function on the domain. The paper I linked to discusses how to estimate $W$ and how to construct $g$---along with error estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a video of the process of exploiting active subspaces for constructing response surfaces (i.e., surrogate models or metamodels): &lt;a href=&quot;https://www.youtube.com/watch?v=mJvKzjT6lmY&quot; rel=&quot;nofollow&quot;&gt;https://www.youtube.com/watch?v=mJvKzjT6lmY&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the figure you show, the function is noisy, so the exact gradient might not be the best choice; those eigenvectors will pick up directions associated with the high frequency noise. In the statistics literature, your problem goes by the name &lt;a href=&quot;http://en.wikipedia.org/wiki/Sufficient_dimension_reduction&quot; rel=&quot;nofollow&quot;&gt;sufficient dimension reduction&lt;/a&gt;, which is set in the context of regression (i.e., supervised learning) with one set of variables designated as &lt;em&gt;predictors&lt;/em&gt; (the independent variables) and the other set designated &lt;em&gt;responses&lt;/em&gt; (the dependent variable). Take a look at Dennis Cook's book &lt;a href=&quot;http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471193658.html&quot; rel=&quot;nofollow&quot;&gt;Regression Graphics&lt;/a&gt;, which is my favorite overview of the field. And yes, sufficient dimension reduction is an entire subfield of statistics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I updated the video link, the book is now published by &lt;a href=&quot;http://bookstore.siam.org/sl02/&quot; rel=&quot;nofollow&quot;&gt;SIAM&lt;/a&gt;, and we have a new active subspaces &lt;a href=&quot;http://www.activesubspaces.org&quot; rel=&quot;nofollow&quot;&gt;website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3652" LastEditorUserId="3652" LastEditDate="2015-05-07T02:14:30.140" LastActivityDate="2015-05-07T02:14:30.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18669" PostTypeId="1" CreationDate="2015-01-15T22:58:52.200" Score="1" ViewCount="503" Body="&lt;p&gt;I don't understand the concept with local and global vec's in petsc when it come to DMDAVecGetArray and VecGetValues. I'm creating some DMDA Vec in parallel which works well. Now I want to access certain non local values of my Vec without receiving the whole global array (will not fit into my main memory) on each processor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something I'm getting wrong? In the documentation of VecGetValues it is written &quot;&lt;em&gt;Gets values from certain locations of a vector. Currently can only get values on the same processor&lt;/em&gt;&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;thx&lt;/p&gt;&#xA;" OwnerUserId="14313" LastActivityDate="2015-01-15T22:58:52.200" Title="Get a single value from a global Vec in petsc" Tags="|c++|petsc|vector|" AnswerCount="0" CommentCount="1" ClosedDate="2015-01-16T15:10:05.963" ContentLicense="CC BY-SA 3.0" />
  <row Id="18670" PostTypeId="1" CreationDate="2015-01-16T10:28:30.950" Score="0" ViewCount="1402" Body="&lt;p&gt;Let consider the following Hermitian matrix&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T =[   1.0000            -0.1000 - 0.0600i   0.3000 - 0.0300i&#xA;      -0.1000 + 0.0600i   1.0000            -0.0400 + 0.0800i&#xA;       0.3000 + 0.0300i  -0.0400 - 0.0800i   1.5000            ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The SVD decomposition using matlab is &#xA;&lt;code&gt;[U,S,V] = svd(T)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;U =&#xA;&#xA;  -0.4311 + 0.0000i   0.3253 - 0.0000i   0.8416 + 0.0000i&#xA;   0.1302 - 0.1375i  -0.8697 + 0.1658i   0.4028 - 0.1345i&#xA;  -0.8757 - 0.1074i  -0.2973 - 0.1484i  -0.3336 + 0.0024i&#xA;&#xA;S =&#xA;&#xA;  1.0e+003 *&#xA;&#xA;    1.6662         0         0&#xA;         0    1.0101         0&#xA;         0         0    0.8237&#xA;&#xA;V =&#xA;&#xA;  -0.4311             0.3253             0.8416          &#xA;   0.1302 - 0.1375i  -0.8697 + 0.1658i   0.4028 - 0.1345i&#xA;  -0.8757 - 0.1074i  -0.2973 - 0.1484i  -0.3336 + 0.0024i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The eigendecomposition is &#xA;&lt;code&gt;[F,D] = eig(T_t)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F =&#xA;&#xA;  -0.8416 - 0.0059i  -0.2911 + 0.1453i   0.4279 - 0.0525i&#xA;  -0.4037 + 0.1317i   0.7040 - 0.5368i  -0.1125 + 0.1524i&#xA;   0.3336             0.3323             0.8822          &#xA;&#xA;D =&#xA;&#xA;  1.0e+003 *&#xA;&#xA;    0.8237         0         0&#xA;         0    1.0101         0&#xA;         0         0    1.6662&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Theoretically, for Hermitian matrices both decompositions are equivalent. However, we observe that both decompositions give the same eigenvalues but the eigenvectors are different. &#xA;Which decomposition is accurate?&lt;/p&gt;&#xA;" OwnerUserId="14317" LastEditorUserId="1128" LastEditDate="2016-06-02T11:54:40.317" LastActivityDate="2016-06-02T11:54:40.317" Title="Difference between eigendecomposition and singular value decomposition for Hermitian matrices" Tags="|matlab|matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18672" PostTypeId="2" ParentId="18670" CreationDate="2015-01-16T14:15:04.647" Score="7" Body="&lt;p&gt;If you entrywise divide the third column of $F$ by the first column of either $U$ or $V$, you will find that the two are just scaled versions of each other. Matlab returns eigenvectors to be unit normed, but they can be scaled arbitrarily by a complex number of modulus 1. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-01-16T14:15:04.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18673" PostTypeId="1" CreationDate="2015-01-16T16:58:33.790" Score="2" ViewCount="2003" Body="&lt;p&gt;I got confused when trying to implement a scheme using Lax-Friedrichs numerical flux for a system of equations in 1D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to my notes Lax-Friedrichs numerical flux is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{LF}(u_l,u_r) = \frac{1}{2}[f(u_l)+f(u_r)] - \frac{\alpha}{2}(u_r-u_l)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha$ is chosen as the fastest speed in the system. The formula I found is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha \geq max_w \Big( max_l \Big| \mu_l\Big( \frac{\partial f}{\partial u}(w) \Big) \Big| \Big)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does $w$ represents here? do I have to evaluate the above expression with $u_r$ and $u_l$ and then take the max? It is the only thing that occurs to me but don't know if that is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the CFL condition I have that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a\frac{\Delta t}{\Delta x} \leq 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a$ represents the wave speed but in scalar case, how do I check for CFL in a system? Moreover, &lt;a href=&quot;http://www.ifu.ethz.ch/EFM/education/NHY_Kapitel_8_EN.pdf&quot; rel=&quot;nofollow&quot;&gt;in this document&lt;/a&gt; I found that the CFL condition is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|\lambda|\frac{\Delta t}{\Delta x} = (|u| + c)\frac{\Delta t}{\Delta x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose $u$ is the velocity of the system, but what does $c$ means?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2015-01-16T17:11:09.527" LastActivityDate="2023-01-31T22:07:05.877" Title="CFL condition and Lax-Friedrich numerical flux" Tags="|finite-difference|discontinuous-galerkin|cfl|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18674" PostTypeId="2" ParentId="18658" CreationDate="2015-01-16T18:12:51.037" Score="1" Body="&lt;p&gt;For your example, it is the ratio of the overlap size and the subdomain size that matters. With coarse grid, the condition number scales like H/d with H the subdomain size and d the overlap size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does depend on the geometry of the subdomains but it seems unaffected from high-order finite elements because this is a property holding already at the continuous level and h or p refinement just makes it closer to the continuous case. You can see &lt;a href=&quot;http://www.unige.ch/~gander/Preprints/paperAS.pdf&quot; rel=&quot;nofollow&quot;&gt;Continuous Analysis of the Additive Schwarz Method: a Stable Decomposition in H^1&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2015-01-16T19:09:54.197" LastActivityDate="2015-01-16T19:09:54.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18675" PostTypeId="2" ParentId="18622" CreationDate="2015-01-16T21:55:58.913" Score="0" Body="&lt;p&gt;I finally found a way to import the data I need from Nastran to Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do that I used Patran as post-processor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all you need to plot the results you are interested in using the &quot;Graph&quot; option in the &quot;Results&quot; tab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have created all the plots you need you have to click on &quot;Utilities&quot; --&gt; &quot;Results&quot; --&gt; &quot;Write XY curves to file...&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The select the plot you need to import and click on &quot;Apply&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same folder where you saved your model you will find the files which contain your data. Convert it to .txt or to .dat to import them in Matlab.&lt;/p&gt;&#xA;" OwnerUserId="11676" LastActivityDate="2015-01-16T21:55:58.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18677" PostTypeId="1" CreationDate="2015-01-17T19:51:34.510" Score="1" ViewCount="3072" Body="&lt;p&gt;I have an irregular grid of points describing &lt;a href=&quot;http://earthquake.usgs.gov/data/slab/images/sam_slab1.jpg&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; surface (a large subduction fault in South America). The color is depth. Anyway I have 3D coordinates (lon,lat,depth) at irregular intervals. I'm trying to generate a triangular mesh using gmsh but I'm struggling with how. I can make every irregular grid point be a &quot;Point&quot; in gmsh with a little python function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def xyz2gmsh(fout,x,y,z):&#xA;    f=open(fout,'w')&#xA;    for k in range(len(x)):&#xA;        line='Point('+str(k+1)+') = {%.6f, %.6f, %.6f, 0.01};\n' %(x[k],y[k],z[k])&#xA;        f.write(line)&#xA;    f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this file (fout) gnerated loads into gmsh but nothing shows! As I understand it I need to somehow tell gmsh that these points collectively represent a surface to be meshed. How? And can I tell gmsh to shoot for making elements of a certain size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="14326" LastEditorUserId="14326" LastEditDate="2015-01-17T20:09:52.337" LastActivityDate="2015-01-17T22:02:34.923" Title="Triangular mesh of a 3D surface" Tags="|mesh-generation|gmsh|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18678" PostTypeId="1" CreationDate="2015-01-17T21:49:08.757" Score="2" ViewCount="333" Body="&lt;p&gt;I am trying to solve this problem using advection-diffusion model and finite element method for the solution, due to the complex geometry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically the problem i'm trying to solve using OpenFOAM is see how heat is distributed when fan is placed on a heated power source, and how much speed or how many fans should be placed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, i'm thinking first about the advection-diffusion model for this, is it the proper model ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, how about stokes equations (they are hyperbolic) , how would heat be expressed ? (They're not suited for the heat problem, if i'm not mistaken)&lt;/p&gt;&#xA;" OwnerUserId="13121" LastActivityDate="2015-02-18T14:08:49.820" Title="Model of heat sink problem with fan" Tags="|finite-element|openfoam|diffusion|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18679" PostTypeId="1" CreationDate="2015-01-17T21:49:53.487" Score="1" ViewCount="167" Body="&lt;p&gt;Could you help me with a Conjugate Gradient question? In using CG to solve Ax=b, why is the search direction $p_{k+1}$ in CG chosen as a linear combination of the residual $r_k$ and previous direction $p_k$? I know the set of search directions are supposed to be conjugate with respect to A. One way to generate such sets is using the eigenvectors of A, but this is very computationally expensive. Another way is to use modified Gram Schmidt, but this requires so much storage. Hence, we choose $p_{k+1}=-r_k+\beta p_k$... but why this choice?&lt;/p&gt;&#xA;" OwnerUserId="13191" LastActivityDate="2015-01-17T21:49:53.487" Title="Search Direction in Conjugate Gradient" Tags="|linear-algebra|optimization|iterative-method|conjugate-gradient|krylov-method|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="18680" PostTypeId="2" ParentId="18677" CreationDate="2015-01-17T21:51:32.530" Score="4" Body="&lt;p&gt;I'll expand my comment to an answer. Since your surface is fairly smooth, rather than generating a surface mesh, you can generate a 2D mesh of just the $(x, y)$-points that have been sampled, and then create a surface mesh by adding in the $z$-values later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might suffice or it might not. Triangulation algorithms, like the one Tyler Olsen linked, are optimized for certain criteria (maximize the minimum angle). When you lift the 2D triangulation to a surface mesh, the outcome may be less than ideal depending on the slopes of the surface. Chapter 8 in &lt;a href=&quot;https://books.google.com/books?isbn=3540332618&quot;&gt;Hjelle's book&lt;/a&gt; on triangulations covers scattered data interpolation and may be of some use to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You've mentioned gmsh, but I actually prefer using the program &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot;&gt;Triangle&lt;/a&gt; for most meshing tasks. Its input/output file formats are much simpler, so you can quite easily write scripts to either create or parse them. I have code for this in C, Fortran and Python if you'd rather be spared the trouble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In gmsh, you'd have to specify a line loop that parameterizes the convex hull of your input points. With Triangle, you can just give it a point cloud; it will compute the convex hull for you and then triangulate the interior. You only have to describe the boundary if the domain isn't convex. To get triangles of given area, you can either give it a command line switch, or you can write a special &lt;code&gt;.area&lt;/code&gt; file if the desired sizes are depend on where you are. I have example code for doing just that in Python for a case where I needed a mesh that was finer in some areas than others.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2015-01-17T22:02:34.923" LastActivityDate="2015-01-17T22:02:34.923" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18682" PostTypeId="2" ParentId="18665" CreationDate="2015-01-18T00:02:44.617" Score="1" Body="&lt;p&gt;Your system of equations is under-specified (3 equations, 36 unknowns!). You probably will not be able to solve it (i.e. find all components of $[V]$) by purely algebraic means.&#xA;I suggest instead that you look at the structure of V and more generally of your problem. For instance, if V is a transformation matrix applied to some deformation of a 4-node element, then the deformations at each node are not independent, so the coefficients of V are related in some way. As another example, in continuum mechanics, the spin matrix, although its dimensions are $3\times3$, only has 3 independent components. I suspect the situation here is similar.&lt;/p&gt;&#xA;" OwnerUserId="14327" LastActivityDate="2015-01-18T00:02:44.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18683" PostTypeId="1" CreationDate="2015-01-18T02:54:28.663" Score="0" ViewCount="835" Body="&lt;p&gt;Good Day &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to heat transfer technique please give me some suggestion on solving energy balance equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a \frac{\partial T_p}{\partial t}=\frac{\partial}{\partial x}\left(b\frac{\partial T_p}{\partial x}\right)+\frac{\partial}{\partial z}\left(c\frac{\partial T_p}{\partial z}\right)+b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is discretized as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a\left(\frac{T_p-T_{p_0}}{\Delta t}\right)=c\left(\frac{T_u-T_p}{\Delta X_u \Delta X}\right)+d\left(\frac{T_D-T_p}{\Delta X_d \Delta X}\right)+e\left(\frac{T_w-T_p}{\Delta Z_w \Delta Z}\right)+f\left(\frac{T_E-T_p}{\Delta Z_E \Delta Z}\right)+b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b=S_c+S_pT_p$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $T_p$ is the temperature of solar panel&#xA;$a,c,d,e,f$ are the constant value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not getting how to proceed with this equation. Please suggest some hints at solutions.  &lt;/p&gt;&#xA;" OwnerUserId="14328" LastActivityDate="2015-02-20T01:21:35.220" Title="How to solve Energy Balance equation by numerical method" Tags="|finite-element|fluid-dynamics|boundary-conditions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18685" PostTypeId="2" ParentId="18678" CreationDate="2015-01-19T12:38:11.120" Score="1" Body="&lt;p&gt;You will probably need two &quot;part&quot; to model your problem.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The first one will deal with the motion of the air. You can start with the Stokes equations, but this will probably not be suitable for your problem (Stokes flow are usually used for very viscous fluids... and air is not particularly viscous). The next steps would be to consider the Navier-Stokes equations, turbulence models,... but all these should already be present in OpenFOAM.&lt;/li&gt;&#xA;&lt;li&gt;The second part deals with the distribution of the temperature in the fluid. And for that, the advection diffusion equation that you mention should be fine.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You need of course to link both parts.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The main coupling will probably consider that the air velocity transports the temperature, i.e. the velocity of the air is the advection field of the advection-diffusion equation for the temperature.&lt;/li&gt;&#xA;&lt;li&gt;A second coupling would consider that when the air gets warmer, it becomes lighter, so it goes up. However, since you consider the action of the fans, you can probably neglect this effect is the fans are strong enough.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are two pages in the tutorials of the dealII library that explain very well the modeling and resolution of such a problem. You can find them &lt;a href=&quot;http://dealii.org/8.2.1/doxygen/deal.II/step_31.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://dealii.org/8.2.1/doxygen/deal.II/step_32.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. For the CFD part (i.e. motion of the air), some works using OpenFOAM for the modelling of wind turbine have already been done. You should have a look at them, you will probably face similar problematics (look at &lt;a href=&quot;http://eawephdseminar.sciencesconf.org/conference/eawephdseminar/pages/hamid_phd_seminar2014.pdf&quot; rel=&quot;nofollow&quot;&gt;this presentation&lt;/a&gt; for example).&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="3684" LastEditDate="2015-01-19T13:22:02.310" LastActivityDate="2015-01-19T13:22:02.310" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18686" PostTypeId="2" ParentId="18678" CreationDate="2015-01-19T13:13:17.390" Score="1" Body="&lt;p&gt;Maybe you should ask yourself some general questions considering the physics&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How large is the Reynolds number $Re$? (Stokes flow is only applicable to $Re\ll 1$.)&lt;/li&gt;&#xA;&lt;li&gt;Are effects of compressibility relevant? So what is your Mach number $Ma$? A rule of thumb: $Ma&amp;lt;0.3$ incompressible flow, $Ma&amp;gt;0.3$ compressible flow. You can also use the Eckert number for characterization. Be careful if $Ma\to 1$ you have discontinuities in your field variables.&lt;/li&gt;&#xA;&lt;li&gt;There are also similar numbers for heat transfer like Nusselt number.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can select the appropriate OpenFOAM-solvers from the list at &lt;a href=&quot;http://www.openfoam.org/archive/2.3.0/docs/user/standard-solvers.php#dx13-89118&quot; rel=&quot;nofollow&quot;&gt;http://www.openfoam.org/archive/2.3.0/docs/user/standard-solvers.php#dx13-89118&lt;/a&gt; . Remember to choose steady-state or transient. (Often a steady-state doesn't exist because of turbulence, hence no convergence of steady-state solvers.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convection-diffusion equation is in my opinion only applicable to frozen velocity fields. I guess &lt;em&gt;thermoFoam&lt;/em&gt; solves this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenFOAM is a finite volume solver not a finite element one.&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2015-01-19T13:13:17.390" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18688" PostTypeId="1" CreationDate="2015-01-19T19:45:08.860" Score="2" ViewCount="767" Body="&lt;p&gt;I am using octave and observed a problem with the eigs-routine for non symmetric matrices. Using GNU octave version 3.8.1 the code below gives significant difference of eigenvalues although same commands are executed. The difference of eigenvalues is computed by subracting the &lt;code&gt;eig&lt;/code&gt;- and &lt;code&gt;eigs&lt;/code&gt;-eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% solves the Eigenvalue Problem &#xA;%   -y''= lambda^2 y on the interval (a,b)&#xA;%&#xA;% with mixed boundary conditions&#xA;%   -- dirichlet boundary condition y(a)=0&#xA;%   -- robin boundary condition   y'(b)=y(b)&#xA;clear all;&#xA;close all;&#xA;p=6;&#xA;%% number of points&#xA;n=2^p;&#xA;%% grid space&#xA;h=1/2^p;&#xA;%% numbers of eigenvalues to find&#xA;kmax=8;&#xA;%% assemble symmetric 2nd finite difference matrix&#xA;Lh=-gallery('tridiag',n,1,-2,1)./(h^2);&#xA;%% apply robin boundary in last line&#xA;Lh(n,:)=0.0;&#xA;Lh(n,n-1)=-2/h^2;&#xA;Lh(n,n)=-1/h^2*(2*h-2);&#xA;%% options for eigs&#xA;% opts.tol=1e-10;&#xA;opts.p=max(floor(n/2),2*kmax);&#xA;comp=0;&#xA;while comp&amp;lt;10 % do the same thing 10 times&#xA;    %% use eig to get all eigenvalues&#xA;    [~,LambdaEig]=eig(Lh);&#xA;    if isreal(diag(LambdaEig))&#xA;      LambdaEig=sort(diag(LambdaEig));&#xA;      lambdaEig=LambdaEig(1:kmax);&#xA;    else&#xA;      warning('eigenvalues of eig are not real')&#xA;    end&#xA;    %% use eigs routine to get 1st kmax eigenvalues of smallest magnitude&#xA;    [~,LambdaEigs,fl]=eigs(Lh,kmax,'sr');&#xA;    if isreal(diag(LambdaEigs))&#xA;      LambdaEig=sort(diag(LambdaEig));&#xA;      lambdaEigs=sort(diag(LambdaEigs));&#xA;    else&#xA;      warning('eigenvalues of eigs are not real')&#xA;    end&#xA;    errmax=max(abs(lambdaEig-lambdaEigs));&#xA;    disp(['eigs flag: ' num2str(fl) ', maximum error of eigenvalues: '...&#xA;            num2str(errmax)])&#xA;    comp=comp+1;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do you have an explanation for this? GNU octave's manual reports that &lt;code&gt;eigs&lt;/code&gt; is based on the ARPACK package. I guess ARPACK is widely used for large eigenvalue problems and tested well. But GNU octave's &lt;code&gt;eigs&lt;/code&gt;-routine seems not to be reliable for this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to use the &lt;code&gt;eigs&lt;/code&gt; routine for eigenvalue computations of larger general eigenvalue problems $A x=\lambda B x$ with $A\in\mathbb{R}^{2000\times2000}$ in GNU octave. Because of the matrix size Krylow subspace methods are reasonably to me and as far as I know &lt;code&gt;eigs&lt;/code&gt; is a modified Arnoldi iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any recommendations how to continue?&lt;/p&gt;&#xA;" OwnerUserId="14284" LastEditorUserId="14284" LastEditDate="2015-01-20T17:58:39.310" LastActivityDate="2018-04-06T07:32:51.210" Title="eigs routine in octave" Tags="|linear-algebra|matlab|finite-difference|eigenvalues|octave|" AnswerCount="1" CommentCount="7" ClosedDate="2018-04-06T23:48:39.243" ContentLicense="CC BY-SA 3.0" />
  <row Id="18689" PostTypeId="1" AcceptedAnswerId="18690" CreationDate="2015-01-19T22:40:23.653" Score="4" ViewCount="211" Body="&lt;p&gt;I have this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\min_{\mathbf{w}} &amp;amp; \sum_{i=1}^N c_i P_i(w_i)\\&#xA;s.t &amp;amp; \notag\\&#xA; &amp;amp; \sum_{i =1 }^N w_i = w \\&#xA; &amp;amp; 0 \leq w_i \leq w_{max},~~\forall i \in 1, ..., N &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $P_i(w)$ is a piecewise linear convex function. Is there a closed formula to solve this? If not, what is the most relevant algorithm? &lt;/p&gt;&#xA;" OwnerUserId="14339" LastActivityDate="2015-08-03T23:25:23.203" Title="Piecewise linear optimization with resource allocation constraints" Tags="|optimization|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18690" PostTypeId="2" ParentId="18689" CreationDate="2015-01-20T00:45:35.840" Score="7" Body="&lt;p&gt;If $P_i(w)$ is a piecewise linear, convex function, then it can be written as the maximum of a number of linear functions, $P_i(w)=\max \{L_{i1}(w),\ldots,L_{iJ_i}(w)\}$. Then, the optimization problem allows for the reformulation&#xA;$$&#xA;  \min_{\mathbf w,\mathbf x} \sum_{i=1}^N c_ix_i = \mathbf c^T \mathbf x, \\&#xA;  \sum_{i=1}^N w_i = w \\&#xA;  0\le w_i \le w_{max}, \qquad i=1,\ldots,N \\&#xA;  x_i \ge L_{ij}(w) \qquad j=1,\ldots,J_i.&#xA;$$&#xA;This is a linear program with linear constraints that can be solved using, for example, the simplex algorithm. There is, in general, no closed-form solution, however.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2015-08-03T23:25:23.203" LastActivityDate="2015-08-03T23:25:23.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18691" PostTypeId="1" CreationDate="2015-01-20T03:23:21.273" Score="1" ViewCount="1334" Body="&lt;p&gt;Doing research project in material involving (mixed integer nonlinear programming) MINLP problem, we want to implement MINLP based on C++, because the majority of our project is based on C++. What would be our option?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We noted that GAMS is a good modeling system, but it requires (is it true?) us to abandon a major part of our original material science code in C++. &lt;/p&gt;&#xA;" OwnerUserId="7891" LastEditorUserId="20688" LastEditDate="2018-06-12T10:05:14.977" LastActivityDate="2018-06-12T10:07:51.893" Title="C++ mixed integer nonlinear programming (MINLP) solver" Tags="|c++|software|mixed-integer-programming|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18692" PostTypeId="2" ParentId="18691" CreationDate="2015-01-20T04:34:39.133" Score="2" Body="&lt;p&gt;A list of MINLP solvers can be found at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.minlplib.org/instances.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MINLPLib Instances&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of these solvers are written in C++ and could be called directly from your code.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="20688" LastEditDate="2018-06-12T10:07:51.893" LastActivityDate="2018-06-12T10:07:51.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18694" PostTypeId="1" CreationDate="2015-01-20T13:36:05.940" Score="1" ViewCount="220" Body="&lt;p&gt;I tried to compute analitically the stiffness of a cantilever thin plate (shown in picture). The plate is also homogeneous and isotropic. The aim is to compare the result I obtain with the result I get when running the FEM analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i64.photobucket.com/albums/h182/Olgola/plate.jpg&quot; alt=&quot;plate&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a = 0.127; b = 0.0508; h = 0.0012; E = 7*10^{10}; \nu = 0.3$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my theoretical derivation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D \nabla^4 w(x,y,t) + \rho h \ddot{w}(x,y,t) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $w$ is the coordinate in the $z$ direction and $D = \dfrac{E h^3}{12(1-\nu^2)}$ is the flexural rigidity, $\rho$ is the density and $h$ the thickness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w(x,y,t) = \vec{N}(x,y) \vec{u}(t)$ approximation with a trial set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in the first bending mode only, therefore the trial set $\vec{N}(x,y)$ has only one element:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N_i(x,y) = X_r(x)Y_s(y) = X_1(x)Y_1(y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions $\vec{N}(x,y)$ has to satisfy are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w(0,y) = 0$ constraint on the translation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w_{/x}(0,y) = 0$ constraint on the rotation ($_{/x}$ means derivation with respect to $x$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore $X_1(x) = \biggl(\dfrac{x}{a} \biggr)^2$ and $Y_1(y) = \biggl(\dfrac{y}{b} \biggr)^0 = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stiffnes $K$ can be found solving:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K = \int\int_{A} D [N_{/xx}(N_{/xx}+\nu N_{/yy})+N_{/yy}(\nu N_{/xx}+N_{/yy})+2 (1-\nu)N_{/xy}N_{/xy}]dA $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only terms different from $0$ are $N_{/x}$ and $N_{/xx}$. After the integration on the area $A$ of the plate I obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K = \dfrac{4Db}{a^3} = 1098.8$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I run the FEM analysis and I post-process the results: knowing the time history of the acceleration and the input force applied to the plate ($p = \sin(\omega t)$ applied at the bottom tip corner of the plate, $\omega = 1^{st}$ natural frequency of the plate) I can compute:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_i(x,\dot{x}) = p_i-m\ddot{x}_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I can extract the curve $f(x)$ setting $\dot{x} = 0$. Fitting this curve with a line and computing the slope of this line I get $K = 3618.0$ which is different from the analytical value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing/doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="-1" LastEditDate="2017-03-10T09:42:37.977" LastActivityDate="2015-01-20T13:36:05.940" Title="Thin plate stiffness: analytical formula to validate FEM model" Tags="|finite-element|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18697" PostTypeId="2" ParentId="14739" CreationDate="2015-01-20T19:46:13.310" Score="0" Body="&lt;p&gt;Difference between weather prediction software and &quot;casual CFD solver&quot; is how weather prediction works with transition of water. Water is being treated as second component, so model becomes 3 dimensional with 2 components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second (not the least important) is that weather (and other convection) models work directly with $\omega$ from $d \omega/dt = (\omega \nabla) u + \nu \nabla^2 \omega$.&lt;/p&gt;&#xA;" OwnerUserId="10464" LastActivityDate="2015-01-20T19:46:13.310" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18700" PostTypeId="2" ParentId="18663" CreationDate="2015-01-20T23:15:59.793" Score="2" Body="&lt;p&gt;The problem boils down to the solution of a linear system a equations. You get one equation for the function value at $f(x_1)$, and three more equations from your derivative equations. Then, a polynomial of degree three is suitable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more brute-force, not so clever way is to use computer algebra such as SymPy for a symbolical solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sympy import *&#xA;&#xA;N = 3 # degree of polynomial - for N + 1 'pieces of information', use a polynomial of degree N&#xA;&#xA;known_x = symbols([&quot;x_%i&quot;%i for i in range(N)])  # create symbols for supporting points&#xA;known_derivs = symbols([&quot;d_%i&quot;%i for i in range(N)])  # create symbols for supporting points&#xA;y1 = Symbol(&quot;y_1&quot;)  # the single zeroth derivative we know&#xA;&#xA;# create symbols for the independent variable and the coefficients:&#xA;x = Symbol(&quot;x&quot;)&#xA;a = [ Symbol(&quot;a_%i&quot;%i) for i in range(N+1) ]&#xA;&#xA;# generate polynomial and the derivative&#xA;p = sum([ coeff*x**i for i, coeff in enumerate(a) ])&#xA;p_prime = diff(p, x, 1)&#xA;&#xA;# define the equations to solve:&#xA;eqns = [ p_prime.subs(x, xi) - d for xi, d in zip(known_x, known_derivs) ]  # equation for the first derivative&#xA;eqns.append(p.subs(x, known_x[0]) - y1)&#xA;&#xA;# solve for the coefficients and print solution:&#xA;solution = solve(eqns, a)&#xA;pretty_sol = [ str(coeff) + &quot; = &quot; + str(value) + &quot;\n&quot;&#xA;               for coeff, value in zip(solution.keys(), solution.values()) ]&#xA;print(&quot;The coefficients are:\n&quot; + &quot;&quot;.join(pretty_sol))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This not very elegant, but seems to work even you added another equation for the derivative (so that could use a polynomial of degree 'N = 4' for the five equations).&lt;/p&gt;&#xA;" OwnerUserId="1553" LastEditorUserId="1553" LastEditDate="2015-01-21T05:59:50.587" LastActivityDate="2015-01-21T05:59:50.587" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18701" PostTypeId="2" ParentId="18691" CreationDate="2015-01-20T23:26:23.237" Score="1" Body="&lt;p&gt;Of that list by BrianBorchers, the main MINLP C++ codes that I'm familiar with are &lt;a href=&quot;https://projects.coin-or.org/Bonmin&quot; rel=&quot;nofollow&quot;&gt;Bonmin&lt;/a&gt; and &lt;a href=&quot;https://projects.coin-or.org/Couenne&quot; rel=&quot;nofollow&quot;&gt;Couenne&lt;/a&gt;. Both are part of the &lt;a href=&quot;http://www.coin-or.org/&quot; rel=&quot;nofollow&quot;&gt;COIN-OR project&lt;/a&gt;, and are open source. The remainder of the libraries in that list are closed-source, or do not have C++ interfaces. Some of the codes are in Fortran; if you can get copies, you might consider writing C++ interfaces, assuming you are willing to put in that effort.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We noted that GAMS is a good modeling system, but it requires (is it true?) us to abandon major part of our original material science code in c++.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you really wanted to, you could write a problem formulation to a GAMS file, call GAMS, and then read the GAMS output, but doing so is clumsy. GAMS isn't really designed to do anything other than specify optimization problems, so trying to rewrite your code in GAMS isn't an option.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-01-20T23:26:23.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18702" PostTypeId="2" ParentId="18683" CreationDate="2015-01-21T01:12:56.013" Score="2" Body="&lt;p&gt;The problem with giving you the next step in this solution is that it is very easy to naively create a Finite Difference Scheme and have it become unstable. The things (without going into the math too much) which matter in this regard are your time step, grid size and how quick your variables are changing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to have your scheme be stable, it is often preferred to use an implicit time integration. This means that each iteration in your loop (looping over the time of interest), this Tp0 value will be your b vector in the prototypical Ax=b linear system. To form your A matrix you will need to read more about difference schemes. I would defer you to googling about &quot;5 point stencils&quot; for 2D FD Approximations. You'll find a full list of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Honestly, I do not understand why the discretized equation below the balance law looks as it does. To me, this looks like a slightly retooled Unsteady Heat Conduction equation in 2D. If that is the case, that is usually not the typical way to discretize a second order differential equation with differences. (Though I do not know what Tu, Td, Tw, or Te are. &lt;/p&gt;&#xA;" OwnerUserId="11658" LastActivityDate="2015-01-21T01:12:56.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18703" PostTypeId="1" AcceptedAnswerId="18704" CreationDate="2015-01-21T03:35:19.327" Score="4" ViewCount="1495" Body="&lt;p&gt;I have the quadratic programming problem in $x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\text{Minimize}\;\; x^T\Sigma x$$&#xA;$$\hspace{15mm}\text{Subject to}\;\; p^Tx = \frac{1}{n}p^T\boldsymbol{1}$$&#xA;$$\hspace{25mm}\boldsymbol{1}^Tx=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Sigma$ is positive semidefinite. I can convert this to an epigraph problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\text{Minimize}\;\; t$$&#xA;$$\hspace{16mm}\text{Subject to}\;\; x^T\Sigma x \leq t$$&#xA;$$\hspace{43mm} p^Tx = \frac{1}{n}p^T\boldsymbol{1}$$&#xA;$$\hspace{32mm}\boldsymbol{1}^Tx=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where one of the constraints is now quadratic. I would like to use the Schur complement in order to convert the first constraint to the requirement that the following matrix is positive semidefinite&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{bmatrix}&#xA;\Sigma^{-1} &amp;amp; x\\&#xA;x^T &amp;amp; t&#xA;\end{bmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, $\Sigma$ is only positive semidefinite, and not strictly positive definite, and thus I cannot invert it. Is there a way to circumvent this?&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="20688" LastEditDate="2018-05-11T14:04:51.287" LastActivityDate="2018-05-11T14:04:51.287" Title="Converting convex quadratic constraint to linear matrix inequality (LMI)" Tags="|optimization|convex-optimization|constrained-optimization|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18704" PostTypeId="2" ParentId="18703" CreationDate="2015-01-21T04:27:47.223" Score="5" Body="&lt;p&gt;One option here is to use the pseudoinverse of $\Sigma$ rather than the actual inverse.  Appendix A of Boyd and Vandenberghe discusses a version of the Schur complement that includes this case.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative is to find a factorization of $\Sigma$ as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma=M^{T}M$ (e.g. by eigenvalue decomposition), and then use the Schur theorem on  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[&#xA;\begin{array}{cc}&#xA;I  &amp;amp; Mx \\&#xA;x^{T}M^{T} &amp;amp; t &#xA;\end{array}&#xA;\right].&#xA;$&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-01-21T04:27:47.223" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18706" PostTypeId="1" AcceptedAnswerId="18715" CreationDate="2015-01-21T12:15:31.997" Score="1" ViewCount="1734" Body="&lt;p&gt;I'm looking for a way to compute the coordinates of the intersection of two lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each lines are defined with a point and a normal vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/9FDHl.png&quot; alt=&quot;Graphic representation&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can assume than the normal vectors are not zero and that the lines are not parallel (this has been easily checked by computing the scaral product of u and v), so this intersection exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coordinates of A and B are known (lets call them $(A_x,A_y)$ and $(B_x,B_y)$) as well as the vectors : $(u_x,u_y)$ and $(v_x,v_y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you point me to a simple computer algorithm that could give me $(M_x,M_y)$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that by hand I could do this by resolving a simple equation system, but I wonder if there is some direct way to do it on a computer.&lt;/p&gt;&#xA;" OwnerUserId="14353" LastActivityDate="2019-07-19T13:49:45.657" Title="Algorithm to compute the intersection of two lines given their cartesian equations" Tags="|algorithms|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18707" PostTypeId="2" ParentId="18706" CreationDate="2015-01-21T18:27:56.713" Score="0" Body="&lt;p&gt;Hi Orabig and welcome to scicomp!  The answer to your question seems pretty straightforward...  You could setup the matrix system $Ax=b$ from the equations defining the lines and solve the matrix system on a computer directly (e.g. gaussian elimination).  There's nothing really tricky about it..  As long as the slopes of the two lines are not too close to each other (i.e. an ill-conditioned system), it's really that simple.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2015-01-21T18:27:56.713" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18708" PostTypeId="2" ParentId="7894" CreationDate="2015-01-21T21:36:44.030" Score="1" Body="&lt;p&gt;FYI, I've recently published a more comprehensive tutorial with the IEEE APS magazine.  Here is the &lt;a href=&quot;http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&amp;amp;tp&amp;amp;arnumber=6931698&amp;amp;url=http%3A%2F%2Fieeexplore.ieee.org%2Fstamp%2Fstamp.jsp%3Ftp%3D%26arnumber%3D6931698&quot; rel=&quot;nofollow&quot;&gt;direct link.&lt;/a&gt;  This one includes more material and has been formally peer-reviewed, so publication references finally have something solid to point to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a great starter paper to get the basic idea of FDM working.  It also seems to work very well with domain sizes on the order of 300x300 or so, depending on your memory.  As we've all seen, the only real bottleneck for larger domains is setting up that pesky system matrix and inverting it.&lt;/p&gt;&#xA;" OwnerUserId="10337" LastActivityDate="2015-01-21T21:36:44.030" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18709" PostTypeId="2" ParentId="10876" CreationDate="2015-01-21T22:33:49.307" Score="5" Body="&lt;p&gt;I can recommend using the finite-difference time-domain (FDTD) method.  I even wrote a tutorial some time back that should answer most of your questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;J. R. Nagel, &quot;A review and application of the finite-difference time-domain algorithm applied to the Schrödinger equation,&quot; ACES Journal, Vol. 24, No. 1, February 2009&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some Matlab codes that run nicely for 1D systems.  If you have experience with FDTD doing electromagnetics, it works great for quantum mechanics as well.  I can post my codes if you're interested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, it just operates on the wavefunctions directly by splitting the derivatives up into finite differences.  It is kind of similar to the Crank-Nicholson scheme, but not exactly.  If you are familiar with FDTD from electromagnetic wave theory, then FDTD will be very intuitive when solving the Schrodinger equation.&lt;/p&gt;&#xA;" OwnerUserId="10337" LastActivityDate="2015-01-21T22:33:49.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18710" PostTypeId="1" CreationDate="2015-01-22T00:33:05.673" Score="4" ViewCount="640" Body="&lt;p&gt;I have a set of left and right eigenvectors from an nonsymmetric eigenproblem, and I'd like to biorthogonalize them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried Gram-Schmidt, but this fails for most cases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then read that the SVD is the best way to get an orthonormal basis for a matrix, where U would be my basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I extend the SVD to the case of two sets of eigenvectors? Or is there a better way to biorthogonalize my left and right eigenvectors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; More details&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a complex, non-Hermitian matrix,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\qquad\mathbf{M} = \left[\begin{array}{cc} \mathbf{A} &amp;amp; \mathbf{B} \\ -\mathbf{B^*} &amp;amp; -\mathbf{A^*}\end{array}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $*$ indicates conjugate transpose. I solve via ZGEEV for the left and right eigenvectors, so I have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\qquad\mathbf{MX_R} = \mathbf{X_RE}\qquad$ and $\qquad \mathbf{X_LM} = \mathbf{EX_L}\qquad$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathbf{E}$ is a diagonal matrix of eigenvalues, and $\mathbf{X_L}$ and $\mathbf{X_R}$ are matrices containing my left and right eigenvectors, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can easily orthogonalize the right and left eigenvectors among themselves, i.e. $\mathbf{X_R^*}\cdot\mathbf{X_R} = \mathbf{I}$ or $\mathbf{X_L^*}\cdot\mathbf{X_L} = \mathbf{I}$, but what I want is to biorthonormalize them to each other, e.g. obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\qquad\mathbf{X_L}\cdot\mathbf{X_R} = \mathbf{I}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written my own modified Gram-Schmidt to accomplish this, but like I said, it does not work for many cases. I cannot see any way of using LAPACK routines (i.e. ZGEQRF followed by ZUNGQR) to accomplish this, nor does using the SVD seem apparent to me. &lt;/p&gt;&#xA;" OwnerUserId="6699" LastEditorUserId="6699" LastEditDate="2015-01-23T21:02:35.017" LastActivityDate="2018-06-03T06:22:22.727" Title="Using SVD to biorthogonalize left and right eigenvectors?" Tags="|linear-algebra|algorithms|svd|basis-set|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18711" PostTypeId="1" CreationDate="2015-01-22T10:09:32.167" Score="1" ViewCount="155" Body="&lt;p&gt;I have a system of non-linear DAE and I noticed that the system does not converge if some of the equations are not differentiated. For example, if the control volume equation is represented as this:&#xA;$V_{total}  = V_1 + V_2$&#xA;The solver won't convege.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if I differentiate the equation like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ 0 = m_1(\partial v_1/\partial p)(dp/dt)+\dot{m}_2v_2+ m_2(\partial v_2/\partial p)(dp/dt)+(\dot{m}_2v_2) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then it does converge.&#xA;I'm using DASPK on octave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Background: &#xA;The equation is part of another five equations that are used to solve the dynamics of a steam accumulator. In such a system, a metal vessel is almost half filled with liquid water and steam is occupying the top other half of the vessel. The system operates under varying pressures and the two phases (1: liquid and 2: steam) are not always in equilibrium. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full system of equations is as follows:&#xA;the Unknowns are &#xA;$M_{1},M_{2},M_{1r},M_{2r},p,\dot{m}_r$&#xA;But notice that $\dot{m}_r$ is actually calculated readily from 4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{d}{dt}[M_1+M_2] = \dot{m}_f - \dot{m}_s$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{d}{dt}[M_1h_1+M_2h_2-pV_t] = \dot{m}_fh_f - \dot{m}_sh_s$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_{t}  = V_1 + V_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{d}{dt}[M_{1r}+M_{2r}] = \dot{m}_{dc} - \dot{m}_r$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{d}{dt}[M_{1r}h_1+M_{1r}h_2-pV_r] = \dot{m}_{dc}h_1 - \dot{m}_sh_s\alpha_r$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_{r}  = V_{1r} + V_{2r}$&lt;/p&gt;&#xA;" OwnerUserId="6701" LastEditorUserId="6701" LastEditDate="2015-01-23T15:22:51.520" LastActivityDate="2015-01-23T15:22:51.520" Title="Convergence of a DASPK depending on DAE formulation" Tags="|convergence|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18713" PostTypeId="2" ParentId="18706" CreationDate="2015-01-22T15:47:14.723" Score="3" Body="&lt;p&gt;To elaborate a bit, start with the ray/parametric form of the first line (0):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec r_0(t) = \vec p_0 + t \cdot \vec q_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;.. where $\vec p_0$ is some origin point known to be on the line (like your point $\vec A$), and $\vec q_0$ is the known unit vector along the line (the tangent, orthogonal to your normal $\vec u$). You can write a similar statement for the other line (1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec r_1(t) = \vec p_1 + s \cdot \vec q_1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intersection point is where $\vec r_0(t) = \vec r_1(s)$, write that statement down:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec p_0 + t \cdot \vec q_0 = \vec p_1 + s \cdot \vec q_1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dot that equation with $\vec q_0$, and dot it again with $\vec q_1$ to get two scalar equations in terms of the two unknowns t and s:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left&amp;lt;\vec q_0,\vec p_0\right&amp;gt; + t \cdot \left&amp;lt;\vec q_0,\vec q_0\right&amp;gt; = \left&amp;lt;\vec q_0,\vec p_1\right&amp;gt; + s\cdot\left&amp;lt;\vec q_0,\vec q_1\right&amp;gt;$&#xA;$\left&amp;lt;\vec q_1,\vec p_0\right&amp;gt; + t \cdot \left&amp;lt;\vec q_1,\vec q_0\right&amp;gt; = \left&amp;lt;\vec q_1,\vec p_1\right&amp;gt; + s\cdot\left&amp;lt;\vec q_1,\vec q_1\right&amp;gt;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $\left&amp;lt;\vec q_0,\vec q_0\right&amp;gt;$ = $\left&amp;lt;\vec q_1,\vec q_1\right&amp;gt;$ = 1, since these are unit vectors. Denote $c = \left&amp;lt;\vec q_0,\vec q_1\right&amp;gt;$, it's the cosine of the angle between the two lines. Then rearrange into a 2x2 matrix equation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[  1  -c ] [ t ] = [ &amp;lt;q0,p1-p0&amp;gt; ]&#xA;[ -c   1 ] [ s ]   [ &amp;lt;q1,p0-p1&amp;gt; ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This matrix is nonsingular (symmetric positive definite, in fact) as long as $|c|$ is bounded away from 1, which is the parallel/antiparallel case (where there is either no intersection or infinitely many). Solve for s and t, then evaluate either $\vec r_0(t)$  or $\vec r_1(s)$ to find the intersection point explicitly.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2015-01-22T15:47:14.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18714" PostTypeId="2" ParentId="8516" CreationDate="2015-01-22T16:22:23.573" Score="3" Body="&lt;p&gt;The Teuchos Unit test harness in Trilinos natively supports unit tests that use MPI.  Things like controlling output from multiple processes and aggregating pass/fail over all processes is automatic.  Take a look:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://trilinos.org/docs/dev/packages/teuchos/doc/html/group__Teuchos__UnitTest__grp.html&quot; rel=&quot;nofollow&quot;&gt;http://trilinos.org/docs/dev/packages/teuchos/doc/html/group__Teuchos__UnitTest__grp.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14362" LastEditorUserId="1804" LastEditDate="2015-01-22T20:27:10.187" LastActivityDate="2015-01-22T20:27:10.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18715" PostTypeId="2" ParentId="18706" CreationDate="2015-01-22T17:25:29.210" Score="5" Body="&lt;p&gt;I might as well add this since you haven't gotten an answer that deals with the format you have for the lines (i.e. point and normal vector). In your case, you can write the equation of each line as&#xA;$$&#xA;(\vec{x}_0-\vec{x})\cdot\vec{n}=0&#xA;$$&#xA;where $\vec{x}_0$ is a point on the line. This gives&#xA;$$&#xA;(A_x-x)u_x+(A_y-y)u_y=0&#xA;$$&#xA;$$&#xA;(B_x-x)v_x+(B_y-y)v_y=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear system can then be written in matrix form as&#xA;$$&#xA;\left[ \begin{array}{cc}&#xA;u_x &amp;amp; u_y \\&#xA;v_x &amp;amp; v_y&#xA;\end{array} \right]&#xA;\left[ \begin{array}{c}&#xA;x \\&#xA;y&#xA;\end{array} \right] = &#xA;\left[ \begin{array}{c}&#xA;A_x u_x + A_y u_y \\&#xA;B_x v_x + B_y v_y&#xA;\end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This linear system can be solved easily via Gaussian elimination with pivoting for any case where the lines are not parallel (i.e. $u_x v_y - u_y v_x \ne 0$). As mentioned in the other solutions, if the lines are very nearly parallel there may be some issues with accuracy due to rounding errors in the computer.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-01-22T21:34:07.883" LastActivityDate="2015-01-22T21:34:07.883" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18716" PostTypeId="2" ParentId="18642" CreationDate="2015-01-22T20:07:25.847" Score="0" Body="&lt;p&gt;This is what brought me meanwhile to the solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &quot;Solving inwards and outwards&quot; simply means numerically integrating the mentioned differential equation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;from some small $r_1$ outwards to some $r_2&amp;gt;r_1$ &#xA;(suitable BC in our case: $u_{out}(r_1)=r_1, u_{out}'(r_1)=1$ for example) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;from some larger $r_3&amp;gt;r_2$ inwards back to $r_2$ (suitable BC for example: $u_{in}(r_3)= e^{-\sqrt{E} r_3}, u_{in}'(r_3) = -\sqrt{E} u(r_3)$, as we expect exponential decay)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;2) If the wronskian $$W(E):=u_{out}'(r_2) u_{in}(r_2) - u_{out}(r_2) u_{in}'(r_2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;vanishes at point $r_2$, it means that the two solutions are linerly dependent and we get the energy eigenvalue(s) $E$ we were looking for (we have to find the roots of the function W(E)=0).we expect exponential decay, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) In Mathematica, for instance, the &quot;ImplicitRungeKutta&quot; method for solving the equation for $u_{out}$ and $u_{in}$, i.e. $$-u''(r) - u(r)/r= -E u(r)$$ seems to be quite suitable.&lt;/p&gt;&#xA;" OwnerUserId="12005" LastActivityDate="2015-01-22T20:07:25.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18717" PostTypeId="1" CreationDate="2015-01-22T22:01:18.073" Score="3" ViewCount="876" Body="&lt;p&gt;I am trying to implement a FEM solver for the following initial value problem&#xA;\begin{align}&#xA;\frac{\partial u}{\partial t} - \nabla^2 u &amp;amp;= f\quad \text{ in } \Omega\times (0,T)\\&#xA;u &amp;amp;= g\quad \text{ on }\partial\Omega\times(0,T)\\&#xA;u(x,y,0) &amp;amp;= u_0(x,y)&#xA;\end{align}&#xA;where $\Omega\subset\mathbb{R}^2$ is just a rectangular domain. The functions $f$ and $g$ can vary with time. I have discretized this IVP into&#xA;$$\mathbf{B}\frac{\partial}{\partial t}\mathbf{u} + \mathbf{A}\mathbf{u} = \mathbf{l} $$&#xA;where&#xA;\begin{align}&#xA;\text{B}_{ij} &amp;amp;= \int_\Omega \phi_i\phi_j\\&#xA;\text{A}_{ij} &amp;amp;= \int_\Omega \nabla\phi_i\nabla\phi_j\\&#xA;l_i &amp;amp;= \int_\Omega \phi_i f&#xA;\end{align}&#xA;and I can assemble these matrices and vector. I have also discretized time to get the time-stepping scheme&#xA;$$(\mathbf{B}+k\theta\mathbf{A})\mathbf{u}^{n+1} = (\mathbf{B}-k(1-\theta)\mathbf{A})\mathbf{u}^n + k(\theta \mathbf{l}^n + (1-\theta)\mathbf{l}^{n+1}), $$&#xA;where $\theta\in[0,1]$ and $k$ is the time-step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here comes my problem. How exactly do I impose the boundary and initial condition, implementation-wise? What changes should I make to the matrices $\mathbf{A},\mathbf{B}$, etc? Should I do anything at each time-step? I feel like it shouldn't be too hard, however when the time is introduced it suddenly confuses me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="10398" LastActivityDate="2022-02-20T09:46:03.240" Title="Initial Value Problem using Finite Element" Tags="|finite-element|time-integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18718" PostTypeId="2" ParentId="18717" CreationDate="2015-01-22T22:43:45.803" Score="5" Body="&lt;p&gt;There are several equivalent ways of implementing Dirichlet boundary conditions with the finite element method. I'll give a brief overview, but you'll probably need more details, which can be found in the book &lt;a href=&quot;https://books.google.com/books?isbn=0898716144&quot; rel=&quot;nofollow&quot;&gt;Understanding and Implementing the Finite Element Method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be easier to analyze this for the Poisson problem $-\nabla^2u = f$ than for the heat equation; the techniques are the same in both cases, and if you understand them for the steady-state problem then it shouldn't be a big leap to adapt them to the time-dependent one. For the initial value, that amounts to setting $u^0_i = \mathbf{u}_0(x_i)$. You'll then use this as part of the right-hand side in the linear system you've written down when you go to solve for the next time-step $\mathbf{u}^1$. The boundary conditions on $\partial\Omega$ are trickier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to implement Dirichlet boundary conditions is to alter the matrices $A$ and $B$ and the load vector in order to guarantee that the boundary conditions are satisfied exactly. Suppose that $i$ is some unknown of the linear system corresponding to a Dirichlet boundary point; the modification you have to make is to zero out row $i$ of $B$, set row $i$ of $A$ to be the corresponding row in the identity matrix and set $\mathbf{l}_i$ equal to $g(x_i)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate this approach, suppose you had a $3 \times 3$ chunk of the linear system that looked like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[\begin{matrix} a_{11} &amp;amp; a_{12} &amp;amp; a_{13} \\&#xA;                      a_{21} &amp;amp; a_{22} &amp;amp; a_{23} \\&#xA;                      a_{31} &amp;amp; a_{32} &amp;amp; a_{33} \end{matrix}\right]&#xA;\left[\begin{matrix} u_1 \\ u_2 \\ u_3\end{matrix}\right]&#xA;= \left[\begin{matrix} l_1 \\ l_2 \\ l_3\end{matrix}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and that $u_2$ is a degree of freedom corresponding to a boundary node. The modified system would look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[\begin{matrix} a_{11} &amp;amp; a_{12} &amp;amp; a_{13} \\&#xA;                      0      &amp;amp; 1      &amp;amp; 0      \\&#xA;                      a_{31} &amp;amp; a_{32} &amp;amp; a_{33} \end{matrix}\right]&#xA;\left[\begin{matrix} u_1 \\ u_2 \\ u_3\end{matrix}\right]&#xA;= \left[\begin{matrix} l_1 \\ g_2 \\ l_3\end{matrix}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see how the modified system isn't symmetric anymore. Nonetheless, $u_1$ and $u_3$ are still coupled to $u_2$. The asymmetry of the system isn't usually an issue in practice provided that your initial guess for $u$ exactly satisfies the Dirichlet boundary conditions; try throwing the problem into a naive CG solver and see what happens. (Can you come up with a reason why this might be so?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is to reduce the system size. Instead of solving for all of the coefficients $u_i$ in the expansion $\mathbf{u} = \sum_iu_i\phi_i$, you'll only solve for the coefficients corresponding to interior nodes. This means extracting sub-matrices out of $A$ and $B$ for those unknowns, and adding an extra component to the load vector $\mathbf{l}$ consisting of the effective forcing on the interior from boundary conditions. Referring back to the example matrices from before, we're left with the following reduced system for $u_1$ and $u_3$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[\begin{matrix} a_{11} &amp;amp; a_{13} \\&#xA;                      a_{31} &amp;amp; a_{33} \end{matrix}\right]&#xA;\left[\begin{matrix} u_1 \\ u_3\end{matrix}\right] =&#xA;\left[\begin{matrix} l_1 - a_{12}g_2 \\ l_3 - a_{32}g_2\end{matrix}\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing the linear system or deflating it are, from what I've observed, the most common approaches. You can also add on boundary conditions later as a constraint, or you can use penalty methods, but these are less common. Of the two, I tend to favor the first approach.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2015-01-23T00:34:57.250" LastActivityDate="2015-01-23T00:34:57.250" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18720" PostTypeId="1" AcceptedAnswerId="19115" CreationDate="2015-01-22T23:11:36.247" Score="1" ViewCount="412" Body="&lt;p&gt;I want to solve a family of MILP problems (indexed by $k \geq 0$) of the following type:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;\max \; c^Tx \;\; s.t. \\&#xA;Ax \leq b \\&#xA;d^Tx \leq k&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the problems are the same with the exception of the additional constraint $d^Tx \leq k$. In particular, I want to solve this problem for many values of $k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any tricks I could use in order to make this fast?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible way would be to add an additional bound for the objective function (based on the previous maximum) as we move from lower $k$ to higher $k$, as these problems are nested. Is there anything else I could do?&lt;/p&gt;&#xA;" OwnerUserId="11508" LastActivityDate="2015-04-08T13:58:55.927" Title="Solving nested MILP problems" Tags="|linear-programming|mixed-integer-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18723" PostTypeId="1" AcceptedAnswerId="18724" CreationDate="2015-01-23T16:13:39.753" Score="0" ViewCount="448" Body="&lt;p&gt;I have a set of independent meshes whose results I would like to project onto another non-hierachally related mesh. Until now, I've been accomplishing this by finding the nearest-distance node in the input meshes (by minimizing the $L_2$ error) and assigning its values to the target mesh. This seems to be the most often used way of approaching the problem (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/18657/projecting-finite-element-solution-onto-new-mesh&quot;&gt;ref1&lt;/a&gt;, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1473/sort-a-cloud-of-points-with-respect-to-an-unstructured-mesh-of-hexahedral-cells&quot;&gt;ref2&lt;/a&gt;) but due to the requirements of this project I would like to implement something quicker if possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, I was wondering if there were other, faster, methods for projecting mesh results onto another mesh. Unfortunately, I can't enforce that the input meshes and the target mesh will be related, since the results of the input mesh will be transformed (rotated / translated) during runtime. This takes out the majority of mesh projection methods that I'm at least aware of, which is why I'd like to go to the community for help. &lt;strike&gt;I understand that my current method likely retains the most accuracy as far as grid projection methods go, but&lt;/strike&gt; I'm okay with trading accuracy for speed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9952" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-01-23T19:16:04.757" Title="Faster methods for projecting a mesh onto a hierachally unrelated mesh?" Tags="|algorithms|mesh|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18724" PostTypeId="2" ParentId="18723" CreationDate="2015-01-23T17:29:35.987" Score="1" Body="&lt;p&gt;The MATLAB scatteredInterpolant class I mentioned in one of the&#xA;posts you refer to works as follows: given a new point where you&#xA;want interpolated values, scatteredInterpolant first tries to find&#xA;the triangular cell in the old mesh containing this point and then&#xA;does a linear interpolation within the cell.&#xA;The most time-consuming part of the algorithm is finding the cell containing&#xA;the point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand correctly, you are interpolating to the new point simply&#xA;by finding the point in the old mesh closest to the new one. If you are&#xA;satisfied with this level of accuracy, the performance of the search for&#xA;the closest point can be dramatically improved. Entire books have been&#xA;written about algorithms for spatial searching. One common approach is to&#xA;use a kd-tree. First you add all the points from the old mesh to the tree&#xA;and then you can quickly query the tree to find the closest points to a set&#xA;of new points. There are several open-source implementations of kd-trees&#xA;that you can find on the web, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://code.google.com/p/kdtree/&quot; rel=&quot;nofollow&quot;&gt;https://code.google.com/p/kdtree/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.umd.edu/~mount/ANN/&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.umd.edu/~mount/ANN/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-01-23T17:29:35.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18725" PostTypeId="2" ParentId="18723" CreationDate="2015-01-23T18:05:52.950" Score="1" Body="&lt;p&gt;With the approach you describe (using the value of the nearest point), you're essentially projecting not the original function $u_1$ defined on the mesh ${\cal T}_1$ onto mesh ${\cal T}_2$, but a function $Iu_1$ that is piecewise constant. Specifically, $Iu_1$ has the same value at point $x$ as the nearest node of the mesh it is defined on, which corresponds to a piecewise constant interpolation on the &lt;em&gt;dual mesh&lt;/em&gt; of ${\cal T}_1$ (the Voronoi mesh corresponding to the nodes). In other words, you're computing $u_2=P_2Iu_i$. Because the interpolation is only ${\cal O}(h)$ accurate, there isn't really any reason to use a ${\cal O}(h^2)$ accurate projection for which you have to compute and invert a mass matrix. Rather, you may as well use an interpolation onto the second mesh. In other words, if an error of size ${\cal O}(h)$ is sufficient for you, then the value $u_2(x_j^{(2)})$ at a node $x_j^{(2)}$ of mesh ${\cal T}_2$ should simply be the value of $u_1(x_k^{(1)})$ where $x_k^{(1)}$ is the &lt;em&gt;nearest node on mesh 1&lt;/em&gt; for $x_j^{(2)}$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="27" LastEditDate="2015-01-23T19:16:04.757" LastActivityDate="2015-01-23T19:16:04.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18730" PostTypeId="2" ParentId="18717" CreationDate="2015-01-24T11:45:54.643" Score="0" Body="&lt;p&gt;I guess you are also missing a discretized weak form for the boundary condition $u = g$, i.e.&#xA;$\mathbf{G} \mathbf{u} = \mathbf{m}$, where $\mathbf{G}$ is assembled from the boundary elements (the basis function set for elements of $\partial \Omega$). I found the treatment in Volakis, &lt;em&gt;et al.&lt;/em&gt;, &quot;Finite Element Method for Electromagnetics&quot; to be quite helpful regarding this basis set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, if both $f$ and $g$ change in time, you need to recompute the respective right-hand sides before the time-stepping iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a guess: what about unifying the volume balance equation and the boundary flux equation in one? As in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{B}_{\Omega} \, \partial_t \mathbf{u}_{\Omega} + \mathbf{A}_\Omega \mathbf{u}_\Omega  + \mathbf{G}_{\partial \Omega} \mathbf{u}_{\partial \Omega} - \mathbf{l}_{\Omega} -\mathbf{m}_{\partial \Omega} = 0$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $\partial \Omega$ and $\Omega$ subscripts should indicate the restrictions to the respective subdomains (the first method in Daniel's comment above) &lt;/p&gt;&#xA;" OwnerUserId="9461" LastActivityDate="2015-01-24T11:45:54.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18731" PostTypeId="2" ParentId="12957" CreationDate="2015-01-24T20:37:00.607" Score="1" Body="&lt;p&gt;You can use fluctuations of the energy, or the energy derivative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The energy as function of temperature shows an inflexion point at the transition temperature, so you would expect the energy derivative to have a maximum at this temperature. But as the energy derivative is not really easy to compute, you may use the fluctuations (as the standard deviation) which is proportional to the energy derivative. This is a nice feature of the equilibrium distribution of the energy.&lt;/p&gt;&#xA;" OwnerUserId="14379" LastActivityDate="2015-01-24T20:37:00.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18732" PostTypeId="2" ParentId="7646" CreationDate="2015-01-24T20:40:16.980" Score="1" Body="&lt;p&gt;I am surprised no one told about &lt;a href=&quot;http://lammps.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;LAMMPS&lt;/a&gt;. It is pretty nice, and do have GPU capabilities.&lt;/p&gt;&#xA;" OwnerUserId="14379" LastActivityDate="2015-01-24T20:40:16.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18733" PostTypeId="2" ParentId="943" CreationDate="2015-01-24T22:25:18.347" Score="2" Body="&lt;p&gt;GPUs have long latency I/O, so lots of threads need to be used to saturate the memory.  To keep a warp busy requires lots of threads.  If the code path is 10 clocks and I/O latency 320 clocks, 32 threads should come close to saturating the warp.  If the code path is 5 clocks, then double the threads.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a thousand cores, look for thousands of threads to fully utilize the GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Memory access is by cache line, usually 32 bytes.  Loading one byte has comparable cost to 32 bytes.  So, coalesce the storage to increase locality of usage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of registers and local RAM to each warp, allowing for neighbor sharing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proximity simulations of large sets should optimize well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Random I/O and single threading is a kill joy...&lt;/p&gt;&#xA;" OwnerUserId="14381" LastActivityDate="2015-01-24T22:25:18.347" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18738" PostTypeId="1" AcceptedAnswerId="18754" CreationDate="2015-01-25T15:16:57.007" Score="0" ViewCount="1945" Body="&lt;p&gt;I need to solve this integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M = \rho h \int_0^a \int_0^b N^2 \ dx dy$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N$ is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N = sin\left(\dfrac{\pi}{4}\eta +\dfrac{3\pi}{4}\right)sin\left(\dfrac{\pi}{4}\eta +\dfrac{3\pi}{4}\right)sin\left(\dfrac{\pi}{4}\psi +\dfrac{3\pi}{4}\right)sin\left(\dfrac{\pi}{4}\psi +\dfrac{3\pi}{4}\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\eta = y/b$ and $\psi = x/a$ (change of variable)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore the integral becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M = \rho h a b \int_0^1 \int_0^1 \biggl[sin^2\left(\dfrac{\pi}{4}\eta +\dfrac{3\pi}{4}\right)sin^2\left(\dfrac{\pi}{4}\psi +\dfrac{3\pi}{4}\right) \biggr]^2 \ d\eta d\psi$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve it I used the Matlab function &lt;code&gt;trapz()&lt;/code&gt; in this way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eta = 0:0.01:1; &#xA;psi = 0:0.01:1;&#xA;rho = 2770; h = 0.0012; a = 0.127; b = 0.0508;&#xA;fun = rho*h*a*b.*(sin(pi/4.*psi+3*pi/4).*sin(pi/4.*psi+3*pi/4) ...&#xA;      .*sin(pi/4.*eta+3*pi/4).*sin(pi/4.*eta+3*pi/4)).^2;&#xA;M = trapz(fun);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$M = 0.0175$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I set  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eta = 0:0.001:1; &#xA;psi = 0:0.001:1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then $M = 0.1754$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried using the &lt;code&gt;quad2d()&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fun = @(eta,psi)  rho*h*a*b.*(sin(pi/4.*psi+3*pi/4).*sin(pi/4.*psi+3*pi/4) ...&#xA;      .*sin(pi/4.*eta+3*pi/4).*sin(pi/4.*eta+3*pi/4)).^2;&#xA;M = quad2d(fun,0,1,0,1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this time $M = 6.8920\times10^{-5}$&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="7733" LastEditDate="2015-01-27T00:10:51.160" LastActivityDate="2015-02-03T05:33:54.670" Title="Solving a double integral in Matlab" Tags="|matlab|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="18739" PostTypeId="1" AcceptedAnswerId="18741" CreationDate="2015-01-25T18:55:12.710" Score="2" ViewCount="586" Body="&lt;p&gt;From the order condition $$\sigma(w)=\frac{\rho(w)}{\ln w}+O(|w-1|^p)$$&#xA;I get &#xA;$$\sigma(w)=1+\frac{3}{2}(w-1)+\frac{5}{12}(w-1)^2=-\frac{1}{12}+\frac{2}{3}w+\frac{5}{12}w^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those coefficients are for Adams-Moulton method&#xA;$$y_{n+2}=y_{n+1}+h(\frac{5}{12}f(t_{n+2},y_{n+2})+\frac{2}{3}f(t_{n+1},y_{n+1})-\frac{1}{12}f(t_{n},y_{n}))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how can I get the coefficients for Adams-Bashforth?&#xA;$$y_{n+2}=y_{n+1}+h(\frac{3}{2}f(t_{n+1},y_{n+1})-\frac{1}{2}f(t_{n},y_{n}))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My text says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Thus, for quadratic $\sigma$ and order 3 we truncate, obtaining&#xA;  $$\sigma(w)=1+\frac{3}{2}(w-1)+\frac{5}{12}(w-1)^2=-\frac{1}{12}+\frac{2}{3}w+\frac{5}{12}w^2$$&#xA;  whereas in the explicit case where $\sigma$ is linear we have $p=2$, and so recover, unsurprisingly, the Adams-Basforth scheme&#xA;  $$y_{n+2}=y_{n+1}+h(\frac{3}{2}f(t_{n+1},y_{n+1})-\frac{1}{2}f(t_{n},y_{n})).$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14387" LastEditorUserId="713" LastEditDate="2015-01-25T20:15:15.823" LastActivityDate="2015-01-25T22:17:20.370" Title="Derivation of Adams-Bashforth coefficients" Tags="|numerical-analysis|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18740" PostTypeId="1" AcceptedAnswerId="18743" CreationDate="2015-01-25T18:58:14.343" Score="5" ViewCount="565" Body="&lt;p&gt;According to &lt;a href=&quot;http://math.nyu.edu/faculty/greengar/shortcourse_fmm.pdf&quot; rel=&quot;nofollow&quot;&gt;Beatson and Greengard's short course on FMM&lt;/a&gt;:&#xA;( Eq. 5.15 &amp;amp; 5.16 setting k=1, q=1 )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can approximate a potential $\phi = 1/(r-R)$ using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 	{1\over |\vec{r}-\vec{R}|} = \sum_{n=0}^{\infty}{r^n\over R^{n+1}} {4\pi\over(2n+1)}&#xA;			  \sum_{m=-n}^{n} Y_n^{-m}(\theta, \phi) Y_l^m(\theta', \phi') $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried this in Python, the error is such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left|  \phi(P)-\sum_{n=0}^p\sum_{m=-n}^{n} ( \cdots ) \right| \le {1 \over R-r}\left( r\over R \right)^{p+1} $$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def potential_expansion( p,&#xA;                         r1, theta1, phi1, &#xA;                         r2, theta2, phi2 ):&#xA;    &quot;&quot;&quot;&#xA;    Return inverse r potential expansion upto the pth term.&#xA;&#xA;    Input&#xA;    ======&#xA;    p - terms in the expansion to return&#xA;    r1, theta1, phi1 - position vector components for r&#xA;    r2, theta2, phi2 - position vector components for R&#xA;&#xA;    &quot;&quot;&quot;&#xA;    coefficients = np.zeros( (p+1, p+1), complex )&#xA;    for n in xrange(p+1):&#xA;        for m in xrange(-n, n+1):&#xA;            coefficient = sph_harm( m, n, theta1, phi1 )*sph_harm( -m, n, theta2, phi2)&#xA;            coefficients[n][m] = 4*pi/(2*n+1)*(r2/r1)**n/r1*coefficient&#xA;    return coefficients.sum().real&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using this method I am getting wrong results, taking a simple example &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p = 5&#xA;&#xA;r1 = 100.&#xA;theta1 = 0.&#xA;phi1 = 0.&#xA;&#xA;r2 = 1. &#xA;theta2 = pi/2.&#xA;phi2 = 0.&#xA;&#xA;cosgamma = cos(theta1)*cos(theta2)+sin(theta1)*sin(theta2)*cos(phi1-phi2)&#xA;potential = 1/root( r1**2 + r2**2 - 2*r1*r2*cosgamma)&#xA;print &quot;Direct calculation: %s&quot; % potential&#xA;&#xA;approx_potential = potential_expansion(p, r1, theta1, phi1, r2, theta2, phi2)&#xA;print &quot;Approximation: %s&quot; % approx_potential&#xA;print&#xA;print &quot;Error: %s&quot; % np.abs((approx_potential - potential))&#xA;print &quot;Upper bound on error: %s&quot; % (1/(r1-r2)*(r2/r1)**(p+1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This outputs completely wrong results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Approximation:0.010101010101&#xA;Direct calculation:0.0099995000375&#xA;Error:0.000101510063503&#xA;Upper bound on error:1.0101010101e-14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Should I investigate whether or not this is a floating point error? If so, how can I go about this?&lt;/p&gt;&#xA;" OwnerUserId="7311" LastEditorUserId="7311" LastEditDate="2015-01-26T21:58:11.920" LastActivityDate="2015-01-26T21:58:11.920" Title="Computation of multipole expansion of potential not converging" Tags="|python|floating-point|approximation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18741" PostTypeId="2" ParentId="18739" CreationDate="2015-01-25T20:18:31.057" Score="2" Body="&lt;p&gt;The polynomial $\sigma(w)$ corresponds to a series expansion around $w=1$, not $w=0$. So when you truncate it to degree 1, you eliminate the term $\frac5{12}(w-1)^2$, not the term $\frac5{12}w^2$, so that it becomes $\sigma(w) = 1+\frac32(w-1)$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-01-25T22:17:20.370" LastActivityDate="2015-01-25T22:17:20.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18742" PostTypeId="1" CreationDate="2015-01-25T22:51:25.663" Score="1" ViewCount="63" Body="&lt;p&gt;Any hints on how to simplify the following double sum to be able to find the sum at least numerically? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{n=2}^{\infty}\frac1{n(n^2-1)} \sum_{k=1}^\infty \frac{(k-1/n)^{2n-2}}{(k+1/n)^{2n+2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Series comes from quantum mechanics (2nd order perturbation theory).&lt;/p&gt;&#xA;" OwnerUserId="12005" LastActivityDate="2015-01-27T01:45:41.713" Title="Is it possible to eliminate the inner sum to evaluate numerically?" Tags="|quantum-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18743" PostTypeId="2" ParentId="18740" CreationDate="2015-01-25T23:00:45.780" Score="5" Body="&lt;p&gt;In order to inconvenience as many people as possible, long ago, mathematicians and physicists decided to use two different conventions on whether $\theta$ or $\phi$ is the latitude angle. Greengard's notes use the physicists' convention that $\theta$ is latitude and $\phi$ is longitude, whereas Scipy uses $\theta$ for longitude and $\phi$ for latitude, so switching the order of the arguments &lt;code&gt;theta&lt;/code&gt; and &lt;code&gt;phi&lt;/code&gt; fixed your code.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-01-25T23:00:45.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18744" PostTypeId="1" AcceptedAnswerId="18747" CreationDate="2015-01-26T00:43:16.247" Score="2" ViewCount="162" Body="&lt;p&gt;I am trying to simulate the phase separation of a binary mixture. If the free energy F is known as a function of the concentration $c$, the dynamical equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\partial c(x,t)}{\partial t}=\frac{d^2}{dx^2} \frac{\delta F[c]}{\delta c}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Flory-Huggins free energy we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\frac{\delta F[c]}{\delta c}=\log\left(\frac{c}{1-c}\right) + \chi c^2 + \gamma (c')^2&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First term is entropy, second term is attraction between particles ($\chi&amp;lt;0$) and third term is similar as a surface tension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use time forward and space centered differentiation. Even for very small $dt$ I get numerical instabilities. I first thought the term $\gamma(c'^2)$ was responsible, but instabilities remain even without.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my Matlab code for no-flux boundary conditions, clear and simplified as much as I can. I am aware the boundary condition implementation may not be correct but I don't think the problem comes from this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function phaseSep ()&#xA;clear all;&#xA;clf;&#xA;&#xA;N = 101;   %number of grid points in x&#xA;dx = 1/(N - 1);&#xA;x = 0:dx:1;   %vector of x values&#xA;T = 1e3;  %number of time steps&#xA;dt = 1e-8;&#xA;&#xA;% Second derivative&#xA;function y=mylaplace(f,i)&#xA;    y = f(i+1)-2*f(i)+f(i-1);&#xA;end&#xA;&#xA;&#xA;function y=dfdc(C)&#xA;&#xA;    p=-0.01;&#xA;    g=0.01;&#xA;&#xA;    for i=2:N-1&#xA;    y(i) = log(C(i)/(1-C(i)))+p*C(i)+g*mylaplace(C,i)/dx^2;&#xA;    end&#xA;&#xA;    % No-flux Boundary conditions&#xA;    y(1)=y(2);&#xA;    y(N)=y(N-1);&#xA;end&#xA;&#xA;%Initial concentration  &#xA;for i=1:N&#xA;    C(i)=0.2+0.1*tanh(10*(x(i)-0.5));&#xA;end&#xA;&#xA;% Plot initial concentration&#xA;hold;&#xA;plot(x,C);&#xA;&#xA;% iterate&#xA;for t=1:T&#xA;    Z=dfdc(C);&#xA;   for i=2:N-1&#xA;      C(i) = C(i) + (mylaplace(Z,i)/dx^2)*dt; &#xA;   end&#xA;&#xA;   %No flux boundary conditions&#xA;   C(1) = C(2);&#xA;   C(N) = C(N-1);&#xA;end&#xA;&#xA;% Plot final concentration&#xA;plot(x,C);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10366" LastEditorUserId="10366" LastEditDate="2015-01-26T12:52:51.487" LastActivityDate="2015-01-26T12:52:51.487" Title="Modified diffusion equation and unstabilities" Tags="|finite-difference|diffusion|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18745" PostTypeId="2" ParentId="18742" CreationDate="2015-01-26T01:20:46.573" Score="4" Body="&lt;p&gt;Numerically the sum isn't terribly easy: it behaves as&#xA;$$ \sum_{n\geq 2, k\geq 1} \frac{1}{n(n^2-1)} \frac{e^{-4/k}}{(k^2-n^{-2})^2}, $$&#xA;so it converges linearly. Most numerical methods that can be expected to do well only really work for either alternating, or quickly converging sums. With a sum that converges linearly, you can only really get anywhere if you can guess what the remainder looks like. In your case, the remainder behaves as $k^{-3}$ or $n^{-2}$, so it's quite regular and there are Richardson, Shanks and Levin methods to try first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried it in mpmath (part of sage) in python, and I got&#xA;$$\begin{array}{rr}&#xA;&amp;amp;0.00920\ 70258\ 69569\ 42996\ 92855\ 67863\ 59226\ 74177\\&#xA;&amp;amp;52180\ 49145\ 07649\ 14972\ 00277\ 81544\ 60503\ 04938\\&#xA;&amp;amp;73843\ 97518\ 00620\ 22842\ 37461\ 13380\ 72993\ 00261\\&#xA;&amp;amp;71196\ 37097\ 25227\ 29413\ 59023\ 59030\ 36624\ 51643\\&#xA;&amp;amp;60770\ 76310\ 65123\ 95919\ 68631\ 77487\ 86050\ 33349&#xA;\end{array}$$&#xA;using the levin method, using this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nsum((lambda n, k: (k-1/n)**(2*n-2)/((k+1/n)**(2*n+2)*n*(n*n-1))), [2,inf], [1,inf], method='levin', verbose=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; In Mathematica, this seems to work, but doesn't give nearly as many digits, only about six (four correct):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NSum[1/(n (n^2 - 1))*(k - 1/n)^(2 n - 2)/(k + 1/n)^(2 n + 2), {k, 1, \[Infinity]}, {n, 2, \[Infinity]}, Method -&amp;gt; &quot;WynnEpsilon&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;References that I used myself:&lt;/em&gt; Numerical Recipes book, &lt;a href=&quot;http://dlmf.nist.gov/3.9&quot; rel=&quot;nofollow&quot;&gt;http://dlmf.nist.gov/3.9&lt;/a&gt; (clearest, shortest explanation), original papers for the methods (see references in the DLMF link). Also, just general mathematical considerations: acceleration methods make lots of assumptions about the series' asymptotic behaviour, so those assumptions need to be correct.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-01-27T01:45:41.713" LastActivityDate="2015-01-27T01:45:41.713" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18746" PostTypeId="1" CreationDate="2015-01-26T08:39:22.223" Score="1" ViewCount="313" Body="&lt;p&gt;I am a little confused about where does the mass and stiff matrix come from. In Discontinuous Galerkin we divide the domain in elements, $\Omega = \cup^K_{k=1} D^k$. Then assume the solution $u$ can be locally approximated by a polynomial as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_h=\sum_{i=1}^{Np}\hat u_i^k \Phi_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then introduce the approximated solution into the conservation law to get the residual&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{R} = \frac{\partial u_h}{\partial t} + \frac{\partial au_h}{\partial x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We define now a space of test functions $V_h = \cup^K_{k=1}V_h^k$ where a locally defined member of $V_h^k$ is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\phi_h^k=\sum_{j=1}^{N_p} \hat \phi_j^k \Phi_j$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally we require that the residual to be orthogonal to all the test functions in $V_h$ yielding&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{D^k}\mathcal{R}(x,t)\Phi_j dx= 0$$&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why do we say that the residual must be orthogonal to &lt;strong&gt;all&lt;/strong&gt; test functions? Shouldn't we need it to be just orthogonal to the functions locally?&lt;/li&gt;&#xA;&lt;li&gt;In the inner product between the residual and the test functions, why are we using $\Phi_j$ instead of $\phi_h^k$? Is it because we are asking to be orthogonal to all the test functions, i.e. the basis of the test functions? So we don't care about the coefficients $\hat \phi_j^k$ ?&lt;/li&gt;&#xA;&lt;li&gt;How does the mass/stiffness matrix appear? This is just notation probably, but I don't see how from the integral a matrix is built.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I didn't explain myself correctly. So, given the PDE we multiply by a test function and integrate by parts over spatial domain, gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{D^k} \frac{\partial u_h}{\partial t} \phi_j+ \frac{\partial au_h}{\partial x} \phi_j dx$$&#xA;$$\int_{D^k} \frac{\partial u_h}{\partial t} \phi_j+ \frac{\partial \phi_j}{\partial x} au_h = -[(au_h)\phi_j]^{k+1}_{k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose $u_h = \sum_{i=1}^{N_p}\hat u_h \phi_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{D^k} \frac{\partial}{\partial t}(\sum_{i=1}^{N_p}\hat u_h \phi_i) \phi_j+ \frac{\partial \phi_j}{\partial x} (\sum_{i=1}^{N_p}\hat u_h \phi_i) = -[(au_h)\phi_j]^{k+1}_{k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where in this last step, I'm confused about two things&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;should I replace $u_h$ in the RHS?&lt;/li&gt;&#xA;&lt;li&gt;From there how can a matrix arise?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm sure I am missing something in my development but I cannot see what it is.&lt;/p&gt;&#xA;" OwnerUserId="3304" LastEditorUserId="3304" LastEditDate="2015-01-27T09:43:28.987" LastActivityDate="2015-01-27T09:43:28.987" Title="Discontinuous Galerkin, residual orthogonal to test functions?" Tags="|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18747" PostTypeId="2" ParentId="18744" CreationDate="2015-01-26T08:51:56.890" Score="2" Body="&lt;p&gt;At first glance, it looks like you are using the method of lines with forward Euler time steps. What WolfgangBangerth is getting at with his example is that even for a simple heat equation, the stability limit of forward Euler (namely, that $|\lambda\Delta{t}| &amp;lt; 1$) combined with the eigenvalues induced by a finite difference approximation of the diffusion operator result in a very stringent limit on the time step relative to the spatial discretization. For a second-order centered finite difference approximation to the second-order derivative term, this limit is of the form $\Delta{t} &amp;lt; h^{2}/(2a)$, where $a$ is the diffusivity and $h$ is the grid spacing of the spatial discretization (assumed uniform). You should check out a text such as &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898716292&quot; rel=&quot;nofollow&quot;&gt;LeVeque's Finite Difference Methods for Ordinary and Partial Differential Equations&lt;/a&gt; for a basic overview of the theory. In terms of practical advice, you probably want to replace the time discretization you are currently using with an L-stable implicit method. To start, you might try using &lt;code&gt;ode23tb&lt;/code&gt; in MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-01-26T08:51:56.890" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18748" PostTypeId="2" ParentId="18746" CreationDate="2015-01-26T10:07:34.557" Score="3" Body="&lt;p&gt;Some answers to your questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;since the test functions are discontinuous, all test functions contain those with only element support.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;in matrix-vector form, you would write the discrete variational problem (with boundary conditions) as $v^T A u = v^T f$ for all $v \in \mathbb{R}^n$ (I skipped the temporal derivative here for simplicity). Now, if $A$ is invertible, you could as well just solve the problem $Au=f$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if you insert the expansions of your trial and test functions, you can simply compute the entries of the mass/stiffness-matrix. For instance for the mass part you obtain $M_{ij}^k = \int_{D_k}\Phi_j\Phi_i\,{\rm d}x$. This is explained in any finite element primer (which I suggest to read before the more advanced topics like DG).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That being said, the DG-formulation which you propose will most likely not produce any meaningful results if you don't introduce any sort of coupling between the cell-degrees of freedom. For continuous finite elements this coupling is introduced in the function space, for DG, you have to introduce it in the derivation via fluxes (which are usually suitably chosen for consistency/conservativeness/stability).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure about your mathematical background, but this and much more is covered in Di Pietro's and Ern's &lt;a href=&quot;http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-3-642-22979-4&quot; rel=&quot;nofollow&quot;&gt;book&lt;/a&gt;. However, I would recommend to read about the basics of FEM/FV methods first.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastActivityDate="2015-01-26T10:07:34.557" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18749" PostTypeId="1" CreationDate="2015-01-26T13:51:29.620" Score="2" ViewCount="85" Body="&lt;p&gt;Is there any algorithm to determine the minimum number of multiplication(division) of a specific expression? and the optimal expression form for implementation?&lt;/p&gt;&#xA;&lt;p&gt;For example, given values of &lt;span class=&quot;math-container&quot;&gt;$\cos\alpha, \sin\alpha$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\cos\beta, \sin\beta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\cos\theta,\sin\theta$&lt;/span&gt;, what is the minimum number of multiplication in order to obtain numerical matrix:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XabiS.gif&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;How to obtain the final optimal expression (codes) for implementation?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-01-26T14:23:42.767" Title="How to determine the minimum number of multiplication needed for a specific expression?" Tags="|complexity|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="18750" PostTypeId="1" AcceptedAnswerId="18755" CreationDate="2015-01-26T16:53:34.293" Score="6" ViewCount="1738" Body="&lt;p&gt;I am trying to solve for the eigenfunctions of a (1D) differential operator using finite differences:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A \, f(x) = \lambda f(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example in Python where $A = \partial_x^4$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np &#xA;import matplotlib.pyplot as plt&#xA;from scipy.sparse import spdiags&#xA;from scipy import linalg&#xA;&#xA;def derivative_matrix(n):&#xA;    stencil = np.array((1,-4,6,-4,1)) &#xA;    diags = range(-2,3)&#xA;    bands = np.tile(stencil,(n,1)).T&#xA;    A = spdiags(bands,diags,n,n).todense()&#xA;    A = np.array(A,dtype=float)&#xA;    return A&#xA;&#xA;if __name__==&quot;__main__&quot;:&#xA;    n = 100&#xA;    A = derivative_matrix(n)&#xA;    vals,vecs = linalg.eig(A)&#xA;    plt.plot(np.real(vecs[:,-3:None]))&#xA;    plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives me really nice eigenfunctions:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/I4NcT.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions is: what do I have to change in the implementation to change the boundary condition of $f$? Currently, the method seems to implicitely assume I want $f(x)=f'(x)=0$. But what if I instead wanted $f'(x)=f''(x)=0$? How would I implement that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be much appreciated. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="14394" LastActivityDate="2015-01-27T00:25:33.943" Title="How to impose boundary conditions on eigenfunction problems?" Tags="|finite-difference|numerical-analysis|ode|eigensystem|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18751" PostTypeId="2" ParentId="18750" CreationDate="2015-01-26T17:44:36.673" Score="2" Body="&lt;p&gt;I solved a similar problem a few days ago. I am going to present the implementation for a simpler case. Consider the eigenvalue problem equation&#xA;$$&#xA;-y''=\lambda y\,,\quad x\in\left(a,b\right)\\&#xA;y\left(a\right)=0\,\quad y'\left(b\right)=y\left(b\right)&#xA;$$&#xA;The boundary condition at $x=a$ is a homogenous Dirichlet boundary condition. In finite differences we don't have to worry too much about this one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Boundary condition at $x=b$ is called Robin boundary condition. The value $y\left(b\right)$ is unknown. Let $y_i$ be the discrete solution at the $i$-th grid point i.e. $y_i=y\left(x_i\right)\,,\, x_i=ih\,,\,i&amp;gt;0\,.$ Using symmetric 2nd order finite differences, we may write the derivative at $x=b$, i.e. $x=x_N$, as&#xA;$$&#xA;y'_N=\frac{y_{N+1}-y_{N-1}}{2h}\,.&#xA;$$&#xA;The value $y_{N+1}$ is outside of the domain. Using the boundary Robin condition we obtain&#xA;$$&#xA;y'_N=y_N\Leftrightarrow y_{N+1}=y_{N-1}+2h y_{N}\,.&#xA;$$&#xA;Substitute in formulas for 2nd derivative yields&#xA;$$&#xA;y''_{N}=\frac{y_{N+1}-2y_N+y_{N-1}}{h^2}&#xA;=\frac{-2\left(1-h\right)y_{N}+2y_{N-1}}{h^2}\,.&#xA;$$&#xA;In conclusion just modify the last line of your finite difference matrix for this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hoped I could show how to do it. I think the scheme is straight forward for symmetric 4th order finite differences.&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2015-01-26T17:44:36.673" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="18753" PostTypeId="2" ParentId="18740" CreationDate="2015-01-26T17:50:59.870" Score="0" Body="&lt;p&gt;Just a small comment: The sequence of summation may cause divergence as well. I learned in school always to sum up the coefficients from small to large magnitudes. How is the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;return coefficients.sum().real&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;sums up, is not specified. You may consider that.&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2015-01-26T17:50:59.870" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18754" PostTypeId="2" ParentId="18738" CreationDate="2015-01-26T20:16:44.507" Score="0" Body="&lt;p&gt;Since the function to be integrated is dependent on 2 parameters: $\eta$ and $\psi$, it has to be integrated as a 2D function in this way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = 0:0.001:1;&#xA;y = x;&#xA;[eta,csi] = meshgrid(x,y);&#xA;fun = rho*h*a*b.*(sin(pi/4.*csi+3*pi/4).*...&#xA;...sin(pi/4.*csi+3*pi/4).*sin(pi/4.*eta+3*pi/4).*sin(pi/4.*eta+3*pi/4)).^2;&#xA;M = trapz(y,trapz(x,fun,2));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this approach gives the same result obtained using the &lt;code&gt;quad2d&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fun = @(eta,psi)  rho*h*a*b.*(sin(pi/4.*psi+3*pi/4).*sin(pi/4.*psi+3*pi/4).*...&#xA;...sin(pi/4.*eta+3*pi/4).*sin(pi/4.*eta+3*pi/4)).^2;&#xA;M = quad2d(fun,0,1,0,1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The main difference between the 2 methods is that in the &lt;code&gt;quad2d&lt;/code&gt; the function to be integrated has to be defined as an function of the 2 parameters (in this case it has been defined as a &lt;code&gt;function_handle&lt;/code&gt;). &lt;/p&gt;&#xA;" OwnerUserId="11676" LastActivityDate="2015-01-26T20:16:44.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18755" PostTypeId="2" ParentId="18750" CreationDate="2015-01-26T22:12:41.067" Score="4" Body="&lt;p&gt;Consider what happens when you approximate a function's derivative&#xA;using finite differences near a boundary. If the boundary is at the&#xA;point $x_0$, the point $x_1$ is just outside the boundary, then the&#xA;approximation at the point just before the boundary, $x_{-1}$ is:&#xA;$$ h^{-4}(f_{-3} -4f_{-2}+6f_{-1}-4f_0+f_1), $$&#xA;where $f_0$ and $f_1$ must be given by boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary condition $f=f'=0$ can be approximated to order $O(h^2)$&#xA;with the conditions&#xA;$$ f_0 = 0, \qquad \frac{f_1 - f_{-1}}{2h} = 0, $$&#xA;using the usual finite difference formulas. Therefore, to get the&#xA;boundary condition $f=f'=0$, you adjust the matrix coefficients at the&#xA;point $x_{-1}$ by substituting the $f_0$ and $f_1$ that you get from BCs:&#xA;$$ f_{-3}-4f_{-2}+7f_{-1}. $$&#xA;Note that this is different from your code, where you use $6$ instead&#xA;of $7$. If you work it out backwards, that's a bit like using $2f'+hf''=0$ instead of $f'=0$. Here is what I get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/V0OAB.png&quot; alt=&quot;zero-one&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the boundary condition $f'=f''=0$, the procedure is the same. The&#xA;approximate boundary conditions are:&#xA;$$ \frac{f_1-f_{-1}}{2h} = 0, \qquad \frac{f_1-2f_0+f_{-1}}{h^2} = 0,&#xA;$$&#xA;so that $f_1 = f_0 = f_{-1}$, and therefore the stencil coefficients&#xA;are&#xA;$$ x_{-2}:\ [1, -4, 6, -3], \qquad x_{-1}:\ [1, -4,3]. $$&#xA;Here is the outcome:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/0jEXx.png&quot; alt=&quot;one-two&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Matrix corners for case 1&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 7 -4  1  0  0  0  0]    [[ 6 -4  1  0  0  0  0] &#xA; [-4  6 -4  1  0  0  0]     [-4  6 -4  1  0  0  0] &#xA; [ 1 -4  6 -4  1  0  0]     [ 1 -4  6 -4  1  0  0] &#xA; [ 0  1 -4  6 -4  1  0]     [ 0  1 -4  6 -4  1  0] &#xA; [ 0  0  1 -4  6 -4  1]     [ 0  0  1 -4  6 -4  1] &#xA; [ 0  0  0  1 -4  6 -4]     [ 0  0  0  1 -4  6 -4] &#xA; [ 0  0  0  0  1 -4  6]]    [ 0  0  0  0  1 -4  7]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Matrix corners for case 2&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ 3 -4  1  0  0  0  0]    [[ 6 -4  1  0  0  0  0] &#xA; [-3  6 -4  1  0  0  0]     [-4  6 -4  1  0  0  0] &#xA; [ 1 -4  6 -4  1  0  0]     [ 1 -4  6 -4  1  0  0] &#xA; [ 0  1 -4  6 -4  1  0]     [ 0  1 -4  6 -4  1  0] &#xA; [ 0  0  1 -4  6 -4  1]     [ 0  0  1 -4  6 -4  1] &#xA; [ 0  0  0  1 -4  6 -4]     [ 0  0  0  1 -4  6 -3] &#xA; [ 0  0  0  0  1 -4  6]]    [ 0  0  0  0  1 -4  3]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-01-27T00:25:33.943" LastActivityDate="2015-01-27T00:25:33.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18756" PostTypeId="2" ParentId="18738" CreationDate="2015-01-27T00:07:12.583" Score="2" Body="&lt;p&gt;Your function $N$ is separable and the limits of integration are constants so this can easily be computed as the product of two one dimensional integrals. In fact those integrals are identical so it is just the square of a 1d integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;M &amp;amp;= \rho h a b \int_0^1 \int_0^1 \left[sin^2\left(\dfrac{\pi}{4}\eta +\dfrac{3\pi}{4}\right)sin^2\left(\dfrac{\pi}{4}\psi +\dfrac{3\pi}{4}\right) \right]^2 \ d\eta d\psi\\&#xA;&amp;amp;= \rho h a b \int_0^1 sin^4\left(\dfrac{\pi}{4}\eta +\dfrac{3\pi}{4}\right) d\eta&#xA;\int_0^1 sin^4\left(\dfrac{\pi}{4}\psi +\dfrac{3\pi}{4}\right) d\psi\\&#xA;&amp;amp;= \rho h a b \left[\int_0^1 sin^4\left(\dfrac{\pi}{4}\eta +\dfrac{3\pi}{4}\right) d\eta\right]^2&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This integral can be done analytically (I used &lt;a href=&quot;http://www.wolframalpha.com/input/?i=integral+of+sin%5E4%28+pi%2F4*%28x%2B3%29+%29+from+0+to+1&quot; rel=&quot;nofollow&quot;&gt;WolframAlpha&lt;/a&gt;), giving&#xA;$$&#xA;M = \rho h a b \left( \dfrac{3}{8}-\dfrac{1}{\pi} \right)^2\approx 6.8920\times10^{-5}&#xA;$$&#xA;if you use $\rho = 2770$, $h = 0.0012$, $a = 0.127$, $b = 0.0508$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if the function is separable but the integral can't be solved analytically it will be much more efficient to compute the product of two one-dimensional integrals than to directly compute the two-dimensional integral.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-01-27T03:39:54.103" LastActivityDate="2015-01-27T03:39:54.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18757" PostTypeId="1" AcceptedAnswerId="18758" CreationDate="2015-01-27T00:09:06.197" Score="3" ViewCount="164" Body="&lt;p&gt;In a program I'm writing, I have a sub-routine that does some vectorized linear operations (specifically differentiation).  Say for convenience I have defined the following inline function, which approximates a partial derivative from a 3D array:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F = @(X)(X(3:\text{end},:,:)-X(1:\text{end}-2,:,:))/(2h)&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a linear transformation which would clearly have a matrix representation.  Is there a simple way to derive the index form of its transpose, $F^{T}$?  I would like to avoid building matrices if possible because the array formulation is much cleaner (and I've already implemented it...).  The &quot;interior&quot; grid points seem obvious - the transpose is also a difference operator.  But is there an easy way to see what the boundary terms will be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3303" LastEditorUserId="3303" LastEditDate="2015-01-27T15:09:09.313" LastActivityDate="2015-01-27T15:21:47.867" Title="Simple way to derive transpose of a vectorized operation" Tags="|matlab|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18758" PostTypeId="2" ParentId="18757" CreationDate="2015-01-27T02:08:31.267" Score="5" Body="&lt;p&gt;I don't know of a general way to do this, but it is certainly not generally possible to write the transpose of a given linear operator with such a simple vectorized expression. Consider your example, but applied to a simple 1d vector &lt;code&gt;x&lt;/code&gt; instead of a 3d array, I will call this function $G(x)$:&lt;br&gt;&#xA;&lt;code&gt;G = @(x) ( x(3:end)-x(1:end-2) )/(2*h)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$G$ operates on vectors of length $n$ and returns vectors of length $n-2$&lt;/li&gt;&#xA;&lt;li&gt;If I write $G$ as a matrix it is&#xA;$$&#xA;G_{(n-2)\times n} = \dfrac{1}{2h}\left[\begin{array}{c}&#xA;-1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \dots &amp;amp; 0\\&#xA;0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 1 &amp;amp; \dots &amp;amp; 0\\&#xA; &amp;amp;  &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \\&#xA;0 &amp;amp; \dots &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 1\\&#xA;\end{array}\right]&#xA;$$&lt;/li&gt;&#xA;&lt;li&gt;The transpose of $G$ is&#xA;$$&#xA;G^\intercal_{n\times(n-2)} = \dfrac{1}{2h}\left[\begin{array}{c}&#xA;-1 &amp;amp; 0 &amp;amp; &amp;amp; 0\\&#xA; 0 &amp;amp; -1 &amp;amp; &amp;amp; \vdots\\&#xA; 1 &amp;amp; 0 &amp;amp; \ddots &amp;amp; 0 \\&#xA; 0 &amp;amp; 1 &amp;amp; \ddots &amp;amp; -1 \\&#xA; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; 0\\&#xA; 0 &amp;amp; 0 &amp;amp; &amp;amp; 1&#xA;\end{array}\right]&#xA;$$&lt;/li&gt;&#xA;&lt;li&gt;There are a couple obvious ways to write a function for $G^\intercal x$, but none of them quite as simple as the expression for $G$. For example:&lt;br&gt;&#xA;&lt;code&gt;Gtranspose = @(x) [-x(1:2); x(1:end-2)-x(3:end); x(end-1:end)]/(2*h);&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A primary reason for the difference is the fact that &lt;code&gt;G&lt;/code&gt; is non-square. What you requested can be done very easily if the operator is square and symmetric or anti-symmetric. As you probably know, a symmetric operator is its own transpose and the negative of an anti-symmetric operator is its transpose. For example, your derivative operator $F$ is very nearly anti-symmetric. If you consider periodic boundary conditions then the operator is symmetric and can be written as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;H = @(X) [ X(end,:,:) - X(2,:,:); ...&#xA;           X(3:end,:,:) - X(1:end-2,:,:); ...&#xA;           X(end-1,:,:)-X(1,:,:) ]/(2*h);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the transpose is then given by $H^\intercal = -H$ so&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Htranspose = @(X) [ X(2,:,:) - X(end,:,:); ...&#xA;                    X(1:end-2,:,:) - X(3:end,:,:); ...&#xA;                    X(1,:,:) - X(end-1,:,:) ]/(2*h);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This approach should work for finite differences based on central differences which are always either symmetric (even derivatives) or anti-symmetric (odd derivatives).&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-01-27T15:21:47.867" LastActivityDate="2015-01-27T15:21:47.867" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18760" PostTypeId="1" AcceptedAnswerId="27250" CreationDate="2015-01-27T16:32:00.987" Score="13" ViewCount="25034" Body="&lt;p&gt;What is the best way to get a visual appealing plot of a singular vector field (if you want to visualize also the field strength). As an example I am playing with the electric fields of two point charges as in the following example: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from pylab import *&#xA;from scipy.integrate import odeint&#xA;from matplotlib import animation&#xA;from matplotlib import cm&#xA;import numpy as np&#xA;&#xA;rc('font', **{'family': 'serif', 'serif': ['Computer Modern']})&#xA;rc('text', usetex=True)&#xA;&#xA;## Set up charges&#xA;&#xA;class charge:&#xA;    def __init__(self, q, pos):&#xA;        self.q=q&#xA;        self.pos=pos&#xA;&#xA;&#xA;chargesPlus=[]&#xA;chargesMinus=[]&#xA;&#xA;#for i in arange(0,1,1):&#xA;chargesPlus.append(charge(1,[3,0]))&#xA;chargesMinus.append(charge(-1,[-3,0]))&#xA;charges = chargesPlus + chargesMinus&#xA;&#xA;&#xA;def E_point_charge(q, a, x, y,r):&#xA;    return q*(x-a[0])/((x-a[0])**2+(y-a[1])**2)**(1.5), \&#xA;        q*(y-a[1])/((x-a[0])**2+(y-a[1])**2)**(1.5)&#xA;&#xA;&#xA;def E_total(x, y, charges):&#xA;    Ex, Ey=0, 0&#xA;    for C in charges:&#xA;        E=E_point_charge(C.q, C.pos, x, y,1)&#xA;        Ex=Ex+E[0]&#xA;        Ey=Ey+E[1]&#xA;    return [Ex, Ey]&#xA;&#xA;&#xA;domain =2&#xA;&#xA;## Cut Quiver plot&#xA;def cut(r):&#xA;    if r &amp;lt; domain:&#xA;        return 0&#xA;    else:&#xA;        return 1&#xA;&#xA;cutv = np.vectorize(cut)&#xA;&#xA;&#xA;def cut_total(charges,x): &#xA;    c = 1&#xA;    for C in charges: &#xA;        r = sqrt((C.pos[0] - x[0])**2 + (C.pos[1] - x[1])**2)&#xA;        c = c*cutv(r)&#xA;        print c&#xA;        print C.pos[0],C.pos[1]&#xA;    return c&#xA;&#xA;fig = figure()&#xA;&#xA;ax = fig.add_subplot(1,1,1)&#xA;&#xA;&#xA;xMin,xMax=-15,15&#xA;yMin,yMax=-10,10&#xA;&#xA;#ax.plot(x,y)&#xA;ax.axis('tight')&#xA;xlim([xMin,xMax])&#xA;ylim([yMin,yMax])&#xA;&#xA;&#xA;# plot point charges&#xA;for C in charges:&#xA;    if C.q&amp;gt;0:&#xA;        plot(C.pos[0], C.pos[1], 'bo', ms=8*sqrt(C.q))&#xA;    if C.q&amp;lt;0:&#xA;        plot(C.pos[0], C.pos[1], 'ro', ms=8*sqrt(-C.q))&#xA;&#xA;&#xA;xG,yG = meshgrid(linspace(xMin,xMax,25),linspace(yMin,yMax,25))&#xA;&#xA;# plot vector field&#xA;E_totalX,E_totalY = E_total(xG,yG,charges)&#xA;&#xA;EAbs = (E_totalX**2 + E_totalY**2)**(0.5)&#xA;E_XX = E_totalX/EAbs&#xA;E_YY = E_totalY/EAbs&#xA;#EAbs = np.nan_to_num(EAbs) &#xA;&#xA;#ax.streamplot(xG,yG,E_XX,E_YY,color=EAbs,cmap=cm.autumn)&#xA;ax.quiver(xG,yG,E_XX,E_YY,EAbs,cmap=cm.GnBu)&#xA;&#xA;xlabel('x')&#xA;ylabel('y')&#xA;ax.set_aspect(1)&#xA;&#xA;&#xA;plt.savefig('fig1.png')&#xA;&#xA;E_totalX = E_totalX*cut_total(charges,[xG,yG])&#xA;E_totalY = E_totalY*cut_total(charges,[xG,yG])&#xA;&#xA;ax.cla()&#xA;&#xA;# plot point charges&#xA;for C in charges:&#xA;    if C.q&amp;gt;0:&#xA;        plot(C.pos[0], C.pos[1], 'bo', ms=8*sqrt(C.q))&#xA;    if C.q&amp;lt;0:&#xA;        plot(C.pos[0], C.pos[1], 'ro', ms=8*sqrt(-C.q))&#xA;&#xA;&#xA;ax.quiver(xG,yG,E_totalX,E_totalY,pivot='middle',minshaft=0.1,minlength=0.3,headlength=2,headaxislength=2,headwidth=3,scale=4,alpha=0.4,width=0.002,linestyle='solid')&#xA;&#xA;xlabel('x')&#xA;ylabel('y')&#xA;ax.set_aspect(1)&#xA;&#xA;plt.savefig('fig2.png')&#xA;#show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output looks like follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/YWXhe.png&quot; alt=&quot;fig1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pZT1Q.png&quot; alt=&quot;fig2&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with the first example is how to choose colormap. I played with different maps from &lt;a href=&quot;http://matplotlib.org/examples/color/colormaps_reference.html&quot; rel=&quot;noreferrer&quot;&gt;http://matplotlib.org/examples/color/colormaps_reference.html&lt;/a&gt; but none gave me a really satisfying result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with the first example is the cut function which doesn't work automatically. You need to cut out a specific region manually. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what would be your suggestions to improve those plots? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have alternative FOSS tools, which can do those plots better or easier please share it. Please don't share LaTeX solutions since I asked a corresponding question on TeX.sx of how to attack this problem with PsTricks or something like that: &lt;a href=&quot;https://tex.stackexchange.com/questions/225176/visualize-singular-vector-fields-with-tikz-or-pstricks-and-friends&quot;&gt;https://tex.stackexchange.com/questions/225176/visualize-singular-vector-fields-with-tikz-or-pstricks-and-friends&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the question should be community wiki. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; &#xA;Since especially the colormap part of the question is not answered yet, I add another simple example where the colormap doesn't look nice:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%matplotlib inline&#xA;from pylab import *&#xA;&#xA;X=linspace(-2,2,40)&#xA;Y=linspace(-2,2,24)&#xA;X,Y=meshgrid(X, Y)&#xA;&#xA;&#xA;def E(x,y):&#xA;    r = sqrt(x**6 + y**6)&#xA;    return (x/r,y/r)&#xA;&#xA;def E_dir(x,y):&#xA;    #direction field&#xA;    Ex,Ey=E(x,y)&#xA;    n=sqrt(Ex**2+Ey**2)&#xA;    return [Ex/n, Ey/n]&#xA;&#xA;Ex,Ey = E(X,Y)&#xA;Exdir,Eydir = E_dir(X,Y)&#xA;EE=sqrt(Ex**2+Ex**2)&#xA;E&#xA;Q  = quiver(X,Y,Exdir,Eydir,EE,cmap='autumn')&#xA;show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Almost everything of the picture just looks red. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7ZdZv.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7ZdZv.png&quot; alt=&quot;output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9417" LastEditorUserId="9417" LastEditDate="2017-06-24T09:24:13.747" LastActivityDate="2017-06-28T20:03:50.140" Title="Visually appealing ways to plot singular vector fields with matplotlib or other foss tools" Tags="|python|visualization|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18761" PostTypeId="1" AcceptedAnswerId="18804" CreationDate="2015-01-27T17:16:19.157" Score="2" ViewCount="666" Body="&lt;p&gt;I know how to compute the discrete Laplacian of a graph and of a mesh (the Laplace-Beltrami operator). Is there an analogous definition for the computation of the &lt;strong&gt;Laplacian of a parametric curve&lt;/strong&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance the Laplacian matrix of a graph is defined as $L=D−A$ where $D$ is the diagonal degree matrix and $A$ is the adjacency matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case the parametric curve is defined as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \phi(t) : \mathbb{R} \to  \mathbb{R} ^2 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \phi(t) = (x(t), y(t))  $&lt;/p&gt;&#xA;" OwnerUserId="14404" LastActivityDate="2015-02-06T17:02:57.140" Title="Laplacian discretization for parametric curves" Tags="|finite-difference|numerical-analysis|eigenvalues|discretization|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="18765" PostTypeId="1" CreationDate="2015-01-28T00:12:03.947" Score="1" ViewCount="58" Body="&lt;p&gt;I need to calculate a determinant consisting of vectors, using the Kronecker product as product. As an example I would need to be able to calculate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left| \begin{array}{cc}&#xA;\left(\begin{array}{c}1\\0\end{array}\right) &amp;amp; 0 \\&#xA;0 &amp;amp; \left(\begin{array}{c}0\\1\end{array}\right) \end{array} \right|=\left(\begin{array}{c}0\\1\\0\\0\end{array}\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know of libraries containing routines for tensor network calculations, but those libraries to my knowledge do not implement determinants. Doing this right will require a good amount of work, so I wanted to ask beforehand whether anybody knows of any code that implements this already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is pretty specialized and therefore a long shot, but I wanted to ask nonetheless before spending days implementing it myself. Any language would be acceptable (be it Mathematica or Fortran77 or anything else).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="1544" LastActivityDate="2015-01-28T00:12:03.947" Title="Library for calculating determinants with Kronecker products" Tags="|linear-algebra|matrix|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18766" PostTypeId="2" ParentId="8899" CreationDate="2015-01-28T01:58:05.433" Score="5" Body="&lt;p&gt;This code is based on &lt;a href=&quot;http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=486688&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blinn's paper&lt;/a&gt;, &lt;a href=&quot;http://www.ualberta.ca/~mlipsett/ENGM541/Readings/svd_ellis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ellis paper&lt;/a&gt;, &lt;a href=&quot;https://www.youtube.com/watch?v=cOUTpqlX-Xs&amp;amp;index=1&amp;amp;list=LLQjU6L2plCWo9t3X-_k4qLg&quot; rel=&quot;nofollow noreferrer&quot;&gt;SVD lecture&lt;/a&gt;, and &lt;a href=&quot;https://math.stackexchange.com/questions/326334/a-question-about-the-arctangent-addition-formula&quot;&gt;additional&lt;/a&gt;&#xA; calculations.&#xA;An algorithm is suitable for regular and singular real matrices.&#xA;All previous versions works 100% as well as this one.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;&#xA;void svd22(const double a[4], double u[4], double s[2], double v[4]) {&#xA;    s[0] = (sqrt(pow(a[0] - a[3], 2) + pow(a[1] + a[2], 2)) + sqrt(pow(a[0] + a[3], 2) + pow(a[1] - a[2], 2))) / 2;&#xA;    s[1] = fabs(s[0] - sqrt(pow(a[0] - a[3], 2) + pow(a[1] + a[2], 2)));&#xA;    v[2] = (s[0] &amp;gt; s[1]) ? sin((atan2(2 * (a[0] * a[1] + a[2] * a[3]), a[0] * a[0] - a[1] * a[1] + a[2] * a[2] - a[3] * a[3])) / 2) : 0;&#xA;    v[0] = sqrt(1 - v[2] * v[2]);&#xA;    v[1] = -v[2];&#xA;    v[3] = v[0];&#xA;    u[0] = (s[0] != 0) ? (a[0] * v[0] + a[1] * v[2]) / s[0] : 1;&#xA;    u[2] = (s[0] != 0) ? (a[2] * v[0] + a[3] * v[2]) / s[0] : 0;&#xA;    u[1] = (s[1] != 0) ? (a[0] * v[1] + a[1] * v[3]) / s[1] : -u[2];&#xA;    u[3] = (s[1] != 0) ? (a[2] * v[1] + a[3] * v[3]) / s[1] : u[0];&#xA;}&#xA;&#xA;int main() {&#xA;    double a[4] = {1, 2, 3, 6}, u[4], s[2], v[4];&#xA;    svd22(a, u, s, v);&#xA;    printf(&quot;Matrix A:\n%f %f\n%f %f\n\n&quot;, a[0], a[1], a[2], a[3]);&#xA;    printf(&quot;Matrix U:\n%f %f\n%f %f\n\n&quot;, u[0], u[1], u[2], u[3]);&#xA;    printf(&quot;Matrix S:\n%f %f\n%f %f\n\n&quot;, s[0], 0, 0, s[1]);&#xA;    printf(&quot;Matrix V:\n%f %f\n%f %f\n\n&quot;, v[0], v[1], v[2], v[3]);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14413" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-01-29T15:39:35.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18768" PostTypeId="1" AcceptedAnswerId="18769" CreationDate="2015-01-28T14:24:24.023" Score="2" ViewCount="117" Body="&lt;p&gt;As you know, the multi-objective optimization methods are developing so fast, i.e.,epsilon constraints. I have a problem that I want to apply the most recent MOOP method for it.&lt;br&gt;&#xA;Please tell me the recent methods for solving multi-objective optimisation(MOOP).&lt;br&gt;&#xA;Thanks you.&lt;/p&gt;&#xA;" OwnerUserId="14420" LastActivityDate="2015-01-28T14:34:56.957" Title="What are the novel MOOP method?" Tags="|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18769" PostTypeId="2" ParentId="18768" CreationDate="2015-01-28T14:34:56.957" Score="1" Body="&lt;p&gt;&lt;em&gt;The ones that I know are:&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Physical Programming (PP) method was suggested by Messac (1996).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Normal Boundary Intersection (NBI) method was developed by Das and Dennis (1998) and Das (1999).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Normal Constraint (NC) method by Messac et al. (2003) and Messac and Mattson (2004) represents a modification of the NBI approach&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enhanced Normalized Normal constriant method is also devolved (2008)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Utyuzhnikov et al. (2005, 2009) suggested modifications to the PP method to make it possibly simpler and more flexible for practical applications&lt;/p&gt;&#xA;" OwnerUserId="14366" LastActivityDate="2015-01-28T14:34:56.957" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18770" PostTypeId="2" ParentId="5314" CreationDate="2015-01-28T17:45:02.970" Score="1" Body="&lt;p&gt;It seems that both are correct; here's why.  The $n/2$ mode is &lt;em&gt;aliased&lt;/em&gt; to the zero mode on an equispaced grid with $n$ points (because that mode passes through zero precisely at all of the grid points).  Therefore the two are indistinguishable.  I'm not sure why in the code (and in Trefethen's book) they bother to replace $n/2$ with $0$; perhaps because a few flops can be saved in that way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a short explanation and demo of aliasing, see &lt;a href=&quot;http://nbviewer.ipython.org/gist/ketch/74cf44877c706325e524&quot; rel=&quot;nofollow&quot;&gt;my IPython notebook here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-01-28T17:45:02.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18771" PostTypeId="1" AcceptedAnswerId="18778" CreationDate="2015-01-28T19:21:00.993" Score="1" ViewCount="150" Body="&lt;p&gt;As of 2015, what shared memory parallel computing solutions are available?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the advantages and disadvantages of each for various use cases arising in high performance scientific computing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am primarily interested in things that work with C or C++, similar to OpenMP or Threading Building Blocks; or perhaps other low-level languages that can be a reasonable alternative to C++ for scientific computing (e.g. Fortran or some of the new languages designed for parallel scientific computing, like &lt;a href=&quot;http://en.wikipedia.org/wiki/Chapel_(programming_language)&quot; rel=&quot;nofollow&quot;&gt;Chapel&lt;/a&gt;, though I'm not sure if any of these have reached a usable quality already).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a brief high level overview that could be useful for a beginner to decide which direction to follow for his/her particular use case.&lt;/p&gt;&#xA;" OwnerUserId="605" LastActivityDate="2015-01-29T01:49:36.993" Title="Shared memory parallel computing solutions compared" Tags="|parallel-computing|hpc|libraries|languages|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18772" PostTypeId="1" CreationDate="2015-01-28T20:04:53.277" Score="0" ViewCount="86" Body="&lt;p&gt;I am looking at a heat initial value problem&#xA;\begin{align}&#xA;\frac{\partial u}{\partial t}-\nabla^2u = f\quad&amp;amp;\text{in}\quad \Omega\times(0,T)\\&#xA;u = g \quad&amp;amp;\text{on}\quad \partial\Omega\times(0,T)\\&#xA;u = u_0\quad&amp;amp;\text{in}\quad \Omega\times\{0\}&#xA;\end{align}&#xA;which I have succesfully solved with the finite element method along with a time-stepping scheme, so that is not my question. My question is however, it seems to work to just solve the boundary value problem&#xA;\begin{align}&#xA;-\nabla^2u = f\quad&amp;amp;\text{in}\quad \Omega\\&#xA;u = g \quad&amp;amp;\text{on}\quad \partial\Omega\\&#xA;\end{align}&#xA;for various fixed times, e.g. $f(x,y,0),f(x,y,0.1),$ etc. This seems to give a good approximation of the solution to the initial value problem, and I don't understand why. This simply ignores the time-derivative it seems to me (of course, the functions $f$ are the same for both problems so the &quot;information&quot; of the time-derivative is not lost, but it still confuses me). Is this simply the method of lines?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="10398" LastActivityDate="2015-01-29T05:25:43.077" Title="Solving Initial Value problem ignoring the time-derivative" Tags="|finite-element|time-integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18774" PostTypeId="2" ParentId="18760" CreationDate="2015-01-28T20:26:25.257" Score="3" Body="&lt;p&gt;I have some simple Matlab codes that plot little arrows for the vectors.  I then use a color map to indicate intensity underneath.  I think they look pretty cool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/04ugq.png&quot; alt=&quot;Wires&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of the quiver plotter function.  It generates the vertices for the quiver/arrow, which you can then draw using the fill() function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%   INPUTS:&#xA;%       U,V = Quiver x- and y-components.&#xA;%       X,Y = x- and y-position of the center of the quiver stem.&#xA;%&#xA;%   OUTPUTS:&#xA;%       Xv,Yv = Quiver vertices.&#xA;&#xA;function [Xv, Yv] = quiver_vertices(U,V,X,Y)&#xA;&#xA;%  Magnitude of vector.&#xA;MAG = sqrt(U.^2 + V.^2);&#xA;&#xA;%  Quiver stem half-thickness.&#xA;T = MAG/40;&#xA;&#xA;%  Arrowhead thickness.&#xA;AH  = MAG/3;            %  Height&#xA;AHx = AH/3;             %  Width&#xA;&#xA;%  Solve for angle of quiver relative to x-axis.&#xA;THETA = atan2(V,U);&#xA;sinT  = sin(THETA);&#xA;cosT  = cos(THETA);&#xA;&#xA;%  Solve for tips of quiver stem.&#xA;Xa = X + U/2 - AH*cosT;&#xA;Xb = X - U/2;&#xA;Ya = Y + V/2 - AH*sinT;&#xA;Yb = Y - V/2;&#xA;&#xA;%  Solve for quiver arrowhead vertices.&#xA;PHI = pi/2 - THETA;&#xA;Xt1 = X + U/2;&#xA;Yt1 = Y + V/2;&#xA;Xt2 = Xa + AHx*cos(PHI);&#xA;Yt2 = Ya - AHx*sin(PHI);&#xA;Xt3 = Xa - AHx*cos(PHI);&#xA;Yt3 = Ya + AHx*sin(PHI);&#xA;&#xA;%  Solve for vertices of quiver stem.&#xA;Xa1 = Xa - T*sinT;&#xA;Xa2 = Xa + T*sinT;&#xA;Xb1 = Xb - T*sinT;&#xA;Xb2 = Xb + T*sinT;&#xA;Ya1 = Ya + T*cosT;&#xA;Ya2 = Ya - T*cosT;&#xA;Yb1 = Yb + T*cosT;&#xA;Yb2 = Yb - T*cosT;&#xA;&#xA;%  Fill the quiver vertices.&#xA;Xv = [Xt1 Xt2 Xa2 Xb2 Xb1 Xa1 Xt3 Xt1];&#xA;Yv = [Yt1 Yt2 Ya2 Yb2 Yb1 Ya1 Yt3 Yt1];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10337" LastEditorUserId="23592" LastEditDate="2017-03-14T12:22:19.020" LastActivityDate="2017-03-14T12:22:19.020" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18775" PostTypeId="1" AcceptedAnswerId="18784" CreationDate="2015-01-28T23:33:43.127" Score="3" ViewCount="470" Body="&lt;p&gt;In the following problem, $A$ is a given $\mathbb{R}^{m\times n}$ matrix: &#xA;\begin{align}&#xA;\mbox{minimize}\quad &amp;amp; x^TAy \\&#xA;\mbox{subject to}\quad &amp;amp; 1^Tx=1^Ty=1, \\&#xA;&amp;amp; x\ge 0,y\ge 0.&#xA;\end{align}&#xA;Despite the simple formulation, it is not easy at all for me. Hope that somebody can help. Many thanks!&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2015-01-29T13:14:33.773" Title="Find $\min x^TAy$ subject to $1^Tx=1^Ty=1,x\ge 0,y\ge 0$" Tags="|optimization|nonlinear-programming|constrained-optimization|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18776" PostTypeId="1" CreationDate="2015-01-28T23:59:07.193" Score="4" ViewCount="644" Body="&lt;p&gt;I already successfully verified my solver (SIMPLE-type FVM-method) with the following manufactured solution (3d Taylor-Green vortex) on the solution domain $[-1,1]^3$ with Dirichlet boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*}&#xA;u &amp;amp;= \frac{1}{2}\sin\left(\pi x\right)\cos\left(\pi y\right)\cos\left(\pi z\right) \\[0.5em]&#xA;v &amp;amp;= \frac{1}{2}\cos\left(\pi x\right)\sin\left(\pi y\right)\cos\left(\pi z\right) \\[0.5em]&#xA;w &amp;amp;= -1\cos\left(\pi x\right)\cos\left(\pi y\right)\sin\left(\pi z\right).&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use this manufactured solution also for wall boundary conditions on all boundaries. I think this should be possible because wall boundary conditions are essentially Dirichlet boundary conditions with a vanishing normal velocity $\vec{u}\cdot\vec{n}_w$ and as you will note, my manufactured solution exhibits this behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect the same order for the decrease of the discretization error, but slightly different absolute values since my code doesn't discretize Dirichlet and wall boundary conditions in the same way. This is due to the fact, that on wall boundaries the transfer of momentum only is owed to shear stress. This implies that the wall normal stress vanishes. That additional condition is not automatically fulfilled if one would only apply a Dirichlet boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am not able to verify my application code using this manufactured solution with wall boundary conditions. Is this because, despite fulfilling the Dirichlet part (vanishing normal velocity), the additional requirement demanding a vanishing normal stress is not met?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you point me to a working manufactured solution for wall boundary conditions that I could use in my pressure based 3d incompressible Navier-Stokes solver (SIMPLE with FVM)?&lt;/p&gt;&#xA;" OwnerUserId="11874" LastActivityDate="2016-01-29T14:59:04.430" Title="Manufactured solution for pressure based 3d incompressible Navier-Stokes solver with wall boundaries" Tags="|fluid-dynamics|boundary-conditions|finite-volume|computational-physics|navier-stokes|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18777" PostTypeId="2" ParentId="18775" CreationDate="2015-01-29T01:26:45.873" Score="2" Body="&lt;p&gt;Suppose that $m=n$ and $A = I$. Then your problem reduces to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mbox{minimize}\quad &amp;amp; \sum_{i=1}^{n}x_{i}y_{i} \\&#xA;\mbox{subject to}\quad &amp;amp; \sum_{i=1}^{n}x_{i}=\sum_{i=1}^{n}y_{i}=1, \\&#xA;&amp;amp; x\ge 0,y\ge 0.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The isolated bilinear terms are typically nonconvex, so solving the problem is challenging and typically requires special, expensive methods in order to calculate a globally $\varepsilon$-optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For general $A$, I would expect similar behavior, because I don't see any way to group the bilinear terms to achieve convexity (say, for instance, via a sum-of-squares type formulation).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-01-29T01:26:45.873" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18778" PostTypeId="2" ParentId="18771" CreationDate="2015-01-29T01:49:36.993" Score="3" Body="&lt;p&gt;You have two basic choices with a distant third. Everything else is not really advisable in my opinion right now. You can use Pthreads (and Boost::Thread) if you want to do 100% of the work yourself. You will have to write your own barriers and reductions, generate thread-local storage, and generally do all the heavy lifting. With all that work, comes all of the control you could want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenMP gets rid of most of these problems by hiding most of the details. If you need to do both task parallelism and worksharing, you'll need a more modern implementation. GCC is coming along, but I haven't compared its completeness or performance to the Intel compiler in a long time. The Intel compiler's OpenMP implementation is traditionally pretty good. With OpenMP you get parallel loops, tasks, atomic/master/single regions for free which seem to cover 99% of scientific computing needs (where that means PDE and ODE simulation and linear algebra). There are lots of libraries (like the Intel MKL) that use/cooperate with OpenMP, so that's an advantage, too. With OpenMP 4.0, you get offload to accelerators, though I think that Intel is the only one supporting that and only for Xeon Phi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TBB, while now open sourced, is a much less used and not standardized. That being said, it is very powerful, has many OpenMP-like constructs, and is very C++y. However, if you try to share your TBB code with anyone, they may not be able to help you since it's much less familiar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the other things I can think of have less adoption, are not robust, have poor programmability, or have poorer performance. My list for that includes OpenCL (on the CPU not GPU), Haskell, Julia, Chapel, X10, UPC, Co-Array Fortran, Python Multiprocessing, etc., etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you're a language nerd who likes explore new languages for fun, I wouldn't recommend writing a scientific app that needs to see the light of day, have performance, and use a library in anything but C/C++/Fortran with OpenMP for threading. When it comes to traditional scientific computing, there are too many libraries out there that expect a C/threads interface to waste your time reinventing the wheel. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-01-29T01:49:36.993" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18779" PostTypeId="2" ParentId="18776" CreationDate="2015-01-29T05:19:09.860" Score="1" Body="&lt;p&gt;There is an analytic solution by Kovasznay, published in 1947, for 2d Navier-Stokes flow. It has nonzero (and non-trivial!) boundary values.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-01-29T05:19:09.860" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18780" PostTypeId="2" ParentId="18772" CreationDate="2015-01-29T05:25:43.077" Score="1" Body="&lt;p&gt;If $f$ and $g$ are functions that depend on the spatial variable, but not on time, then $u(x,t)$ converges to a function $\bar u(x)$ that satisfies the Poisson equation&#xA;$$&#xA;  -\Delta u=f, &#xA;$$&#xA;together with $u|_{\partial\Omega}=g$. It does so regardless of the initial conditions. In other words, the solution of the heat equation is, in a sense, not very interesting if you consider the &quot;long-term&quot; behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What constitutes &quot;long-term&quot; depends on the eigenvalues of the Laplace operator on the domain you're on, as well as the decomposition of the initial condition and the right hand side term with regard to the eigenvectors of the Laplace operator. It may simply be that in the situation you describe, you are &lt;em&gt;always&lt;/em&gt; in the long-term case and so you do not observe behavior that is fundamentally different from just the Laplace equation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-01-29T05:25:43.077" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18781" PostTypeId="2" ParentId="18776" CreationDate="2015-01-29T08:08:06.527" Score="1" Body="&lt;p&gt;Here's a recent paper that focuses on 2D manufactured solutions including wall boundary conditions &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999114000278&quot; rel=&quot;nofollow&quot;&gt;Eca &amp;amp; Hoekstra&lt;/a&gt;. There are 3D examples in &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045793012001697&quot; rel=&quot;nofollow&quot;&gt;Veluri e.a.&lt;/a&gt;. If you have a laminar code, then simply replace the viscosity by the manufactured effective viscosity (viscosity + eddy viscosity from manufactured turbulence). There are many other references, try searching for &quot;code verification&quot;. In &lt;a href=&quot;http://user.uni-frankfurt.de/~shahraki/index_htm_files/Ferziger%20Peric%20-%20Computational%20Methods%20for%20Fluid%20Dynamics,%203rd%20Ed%20-%202002.pdf&quot; rel=&quot;nofollow&quot;&gt;Ferziger &amp;amp; Peric&lt;/a&gt; Section 8.10.3 you will find how to ensure zero normal stress in the context of FVM and SIMPLE. &lt;/p&gt;&#xA;" OwnerUserId="1973" LastEditorUserId="1973" LastEditDate="2015-02-03T07:50:29.443" LastActivityDate="2015-02-03T07:50:29.443" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18782" PostTypeId="1" CreationDate="2015-01-29T09:40:12.360" Score="2" ViewCount="347" Body="&lt;p&gt;I am trying to implement SPH using the directions shown in &lt;a href=&quot;http://nccastaff.bournemouth.ac.uk/jmacey/MastersProjects/MSc2010/08ChrisPriscott/ChrisPriscott_THESIS.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The density needs to be updated using the formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\rho(\mathbf{x}_i)=\sum_j m_jW_\text{default}(\mathbf{x}_i-\mathbf{x}_j,h)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The smoothing kernel is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W_\text{default}(\mathbf{r},h)=\frac{315}{64\pi h^9}\cases{(h^2-||\mathbf{r}||^2)^3,\quad 0\leq\mathbf{r}\leq h\\0,\quad\quad\quad\quad\quad ||\mathbf{r}||&amp;gt;h}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are no particles within the smoothing length(&lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;), won't the density of a particle become zero based on this formula?&lt;/p&gt;&#xA;" OwnerUserId="14432" LastEditorUserId="20688" LastEditDate="2020-04-23T03:32:31.603" LastActivityDate="2020-04-23T03:32:31.603" Title="Value of density when there are no or very few neighbours in SPH simulation" Tags="|fluid-dynamics|simulation|sph|smoothing|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18784" PostTypeId="2" ParentId="18775" CreationDate="2015-01-29T13:14:33.773" Score="5" Body="&lt;p&gt;If there's no mistake then it's much easier than I thought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will show that the minimum is equal to the smallest component of $A$, denoted by $a_{i_0j_0}$ where $(i_0,j_0) = \arg\min_{(i,j)} a_{ij}$, and attained when $x_{i_0}=y_{j_0}=1$ and $x_i=y_i=0\quad\forall i\neq i_0,j\neq j_0.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, we have &#xA;\begin{align}&#xA;x^TAy=\sum_{1\le i\le m}\sum_{1\le j\le n} a_{ij}x_iy_j \ge \sum_{1\le i\le m}\sum_{1\le j\le n} a_{i_0j_0}x_iy_j = a_{i_0j_0}\sum_{1\le i\le m}\sum_{1\le j\le n} x_iy_j = a_{i_0j_0}. &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equality is easy to check.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2015-01-29T13:14:33.773" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18785" PostTypeId="1" AcceptedAnswerId="18791" CreationDate="2015-01-29T16:28:04.307" Score="0" ViewCount="1054" Body="&lt;p&gt;The other day I had a discussion with a friend about the GAMS solvers and we were wondering what are the mathematical differences between the solvers. Which one to use for which kind of problem? How to know which solvers to use and what happens if the &quot;wrong&quot; solver is selected?&lt;/p&gt;&#xA;" OwnerUserId="9571" LastActivityDate="2018-04-17T13:34:56.537" Title="GAMS solvers: which one to use" Tags="|linear-programming|nonlinear-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18788" PostTypeId="1" AcceptedAnswerId="23363" CreationDate="2015-01-29T23:44:56.047" Score="4" ViewCount="141" Body="&lt;p&gt;I have a curve in the complex plane given by&#xA;$$ f(t) = \sum_k r_k\exp(2\pi\mathrm{i}(t+\varphi_k)p_k). $$&#xA;Some of the parameters are specially chosen: $r_k&amp;gt;0$, $\sum_k r_k=1$, $p_k\in\mathbb{Z}$, $\mathrm{gcd}(p\ldots)=1$, $\varphi_k\in\mathbb{R}$. These restrictions are probably not very important for this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I plot the curve, it will sometimes have some self-kissing points: points $t_1$ and $t_2$ where $f(t_1)=f(t_2)$ and $f'(t_1)\parallel f'(t_2)$. A self-kissing point doesn't have to be a self-crossing: at the self-intersection with parallel tangent vectors, the normal vectors can be pointing the opposite ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody suggest a good method for finding such points? I can't seem to think of a real-valued function that changes sign at the self-kissing point. I would like to be able to find these points once every frame in an animation, so efficiency can be an issue. If you can suggest a reference for this, that helps too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; Direct root finding doesn't work because such a point would be a multiple root and the curve might come close to touching itself without doing so to numerical accuracy. Optimizing a function like $|f(t_1)-f(t_2)|^2$ doesn't work: it has many spurious critical points. Splitting the interval $[0,1]$ into $N$ parts and treating the curve as approximately quadratic inside each interval also doesn't work because there are two variables in $f(t_1)-f(t_2)$, and so requires $O(N^2)$ work, which is too much. (Although when looking for self-intersection points instead, this can work because of efficient line segment intersection algorithms.)&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-02-02T22:42:58.333" LastActivityDate="2016-03-15T21:57:26.077" Title="Finding self-kissing points on a plane curve?" Tags="|nonlinear-equations|roots|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18790" PostTypeId="2" ParentId="18782" CreationDate="2015-01-30T04:10:20.360" Score="2" Body="&lt;p&gt;Yes. You need to choose the radius $h$ large enough that for each particle, there is always a significant number of other particles within the first particle's radius.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-01-30T04:10:20.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18791" PostTypeId="2" ParentId="18785" CreationDate="2015-01-30T04:19:15.397" Score="3" Body="&lt;p&gt;Somewhere in the GAMS file, after you've declared almost all of your model, you have to write a solve statement of the form &lt;code&gt;solve &amp;lt;your_problem&amp;gt; using &amp;lt;formulation&amp;gt; {minimizing|maximizing} &amp;lt;your_objective_function_variable&amp;gt;;&lt;/code&gt;, where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;&amp;lt;your_problem&amp;gt;&lt;/code&gt; should be replaced with the name of your problem&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;&amp;lt;formulation&amp;gt;&lt;/code&gt; is one of the GAMS formulation types (&lt;code&gt;lp&lt;/code&gt;, &lt;code&gt;mip&lt;/code&gt;, &lt;code&gt;nlp&lt;/code&gt;, etc.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;{minimizing|maximizing}&lt;/code&gt; means you're either solving a minimization problem or a maximization problem, so pick one of the two&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;&amp;lt;your_objective_function_variable&amp;gt;&lt;/code&gt; is whatever gams variable you're using to encode the objective function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given a formulation type, &lt;a href=&quot;http://gams.com/dd/docs/solvers/allsolvers.html&quot; rel=&quot;nofollow&quot;&gt;GAMS provides a list of solvers you can use to solve that type of formulation&lt;/a&gt;. So if you use the wrong combination of formulation and solver, GAMS will return an error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably want to pick the most restrictive formulation type that satisfies the formulation you've declared in the GAMS file. That is, you could write out an LP and then write in your GAMS file &lt;code&gt;solve MyProblem using nlp minimizing z;&lt;/code&gt;, but LP solvers generally exploit additional structure to make solves faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For general solver recommendations, you can look at &lt;a href=&quot;http://plato.la.asu.edu/bench.html&quot; rel=&quot;nofollow&quot;&gt;Hans Mittelmann's benchmark data&lt;/a&gt; for general purpose recommendations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My experience is as follows, with the caveat that solver performance obviously depends on problem instance and input parameters, you should try multiple solvers and parameter tuning for best performance, and you should definitely consult Mittelmann's benchmarks yourself:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LP/QCP/MIP/MIQCP solvers: use CPLEX or Gurobi, if possible. Those two solvers are best of breed, and around 10x faster than anything else. SCIP is also pretty good, and CBC is one of the best free options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NLP solvers (convex): CONOPT and DICOPT were good for small to medium-scale problems; for medium-scale problems, SNOPT works pretty well, too; for large-scale problems, I'd use IPOPT first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MINLP/NLP (nonconvex): BARON's been considered the gold standard for a while. ANTIGONE is relatively new, and worth trying. COUENNE and BONMIN are also good options.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-01-30T04:19:15.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18792" PostTypeId="1" CreationDate="2015-01-30T11:10:41.743" Score="1" ViewCount="527" Body="&lt;p&gt;I have to solve an ill-conditioned sparse matrix. Once I read that iterative solvers are the better tool for such problems. Is that true? If yes, why?&lt;/p&gt;&#xA;" OwnerUserId="14444" LastEditorUserId="20688" LastEditDate="2021-12-21T22:44:09.167" LastActivityDate="2021-12-21T22:45:28.310" Title="Direct or iterative solver for ill-conditioned problems" Tags="|linear-algebra|linear-solver|iterative-method|condition-number|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18793" PostTypeId="2" ParentId="8072" CreationDate="2015-01-30T12:56:56.583" Score="1" Body="&lt;p&gt;I know the methods which are used for prolongation/restriction affects speedup too much. The point is your selected scheme should not introduce any positive or negative source term.&#xA;In the other word it is better to use a conservative operator particularly if you are using it for conservative equations.&#xA;So I recommend to plot your coarse and fine points together and make cells (even in 1D) then for interpolation a data in a cell just integrate upon the grid. for some regular simple 1d and 2d cases you result in some trivial popular operators. &lt;/p&gt;&#xA;" OwnerUserId="14445" LastActivityDate="2015-01-30T12:56:56.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18794" PostTypeId="1" CreationDate="2015-01-30T15:10:25.697" Score="3" ViewCount="84" Body="&lt;p&gt;I would greatly appreciate some help/references on solving the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are in charge of searching through a n-dimensional hyper-cube  $[0,1]^n$  to make sure that it does not contain any defects. (To be precise, suppose we are given a black-box, Boolean function $D:[0,1]^n \to \{ 0,1\}$. A point $p \in [0,1]^n$ is said be defective if $D(p) = 1$. )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any point $ p \in [0,1]^n$, you can find a ball of radius $ \epsilon(p)$ about that point, for which you are 100% confident there are no defects. The radius $\epsilon$ depends continuously on the point $p$ and is expensive to compute. (Alternatively, it may be better to just assume that the radius $\epsilon(p)$ is randomly chosen from an uniform probability distribution.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you plan which points to check for nearby defects, so that you know when the entire hyper-cube has been covered by defect-free regions?&lt;/p&gt;&#xA;" OwnerUserId="14447" LastEditorUserId="14447" LastEditDate="2015-01-30T22:17:56.443" LastActivityDate="2015-01-31T00:51:18.817" Title="How to Check a Hyper-Cube for Defects" Tags="|algorithms|reference-request|computational-geometry|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18795" PostTypeId="1" AcceptedAnswerId="18798" CreationDate="2015-01-30T15:11:10.993" Score="5" ViewCount="189" Body="&lt;p&gt;I have a medical physics problem - I want to maximise the dose absorbed by a brain tumour whilst minimising the dose in the rest of the brain, especially certain organs, such as the pituitary gland, optic nerves and cochlea (organs at risk, OAR). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dose is delivered by syringes which have 3 different entry points, placing spheres of radiation within the tumour along their paths (imagine the syringes create a cylinder in which the spheres are placed, so I will have 3 cylinders containing spheres of radiation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my goal function :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a target absorbed dose and homogeneity for the tumour&lt;/li&gt;&#xA;&lt;li&gt;minimise absorbed dose for OAR (with an absolute upper limit)&lt;/li&gt;&#xA;&lt;li&gt;minimise absorbed dose for normal brain tissue (with upper limit that is lower than for the OAR)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And the variables I can change include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Entry points of syringes &lt;/li&gt;&#xA;&lt;li&gt;Directions of syringes&lt;/li&gt;&#xA;&lt;li&gt;Where to place spheres along the syringe's path (spacing along the cylinder)&lt;/li&gt;&#xA;&lt;li&gt;At some later point, I would like to implement variable sphere size between a minimum and maximum volume, but for now I'll keep it simple...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am completely new to inverse problem solving algorithms and am at a complete loss as to where to start looking for answers. Any help or pointers towards good material to read would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I should have mentioned that the spheres emit almost purely beta radiation.&lt;/p&gt;&#xA;" OwnerUserId="14446" LastEditorUserId="14446" LastEditDate="2015-02-02T10:17:04.897" LastActivityDate="2015-02-02T10:17:04.897" Title="Algorithms for radiation treatment planning" Tags="|pde|algorithms|reference-request|inverse-problem|optimal-control|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18796" PostTypeId="2" ParentId="18795" CreationDate="2015-01-30T15:52:18.097" Score="1" Body="&lt;p&gt;There has been a lot of research in this area over the last 20 years.  It's appropriate to start by using search engines such as Google Scholar and Web of Science to look for survey and review articles.  For example, you might look at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S2211692313000428&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/S2211692313000428&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-01-30T15:52:18.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18797" PostTypeId="2" ParentId="18792" CreationDate="2015-01-30T15:59:14.110" Score="8" Body="&lt;p&gt;Your question really doesn't admit a simple answer—we need to know more specifics about your problem to provide a useful answer.&lt;/p&gt;&#xA;&lt;p&gt;In general, iterative methods can be faster than direct factorization for large sparse systems of equations if the system is reasonably well conditioned or if it is badly conditioned but you have a good preconditioner, or if you will apply regularization to help improve the conditioning of the problem.  Another important issue is whether you need an extremely precise solution or whether you're willing to live with a less accurate solution.&lt;/p&gt;&#xA;&lt;p&gt;In deciding between the approaches it's important to know:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How big are your systems of equations?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do they have any special structure (e.g. symmetric and positive definite.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How badly conditioned are the systems of equations?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there an available pre-conditioner if the system is not well-conditioned?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How accurate an answer do you need?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are you willing to use some kind of regularization to improve any ill-conditioning?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="20688" LastEditDate="2021-12-21T22:45:28.310" LastActivityDate="2021-12-21T22:45:28.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18798" PostTypeId="2" ParentId="18795" CreationDate="2015-01-30T16:08:26.660" Score="5" Body="&lt;p&gt;This problem is actually more of an optimal control problem for a partial differential equation. As a starting point, I would recommend the following books:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ams.org/bookstore-getitem/item=gsm-112&quot; rel=&quot;noreferrer&quot;&gt;F. Tröltzsch, &lt;em&gt;Optimal control of partial differential equations&lt;/em&gt;, AMS, 2010&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/mathematics/book/978-1-4020-8838-4&quot; rel=&quot;noreferrer&quot;&gt;M. Hinze, R. Pinnau, M. Ulbrich, S. Ulbrich, &lt;em&gt;Optimization with PDE constraints&lt;/em&gt;, Springer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first one in particular is an excellent textbook.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your specific problem, however, is far from the standard examples treated in textbooks: First, your equation is the radiative transport equation, which behaves very different from standard second-order equations. There are some groups working on this topic, though: You should look at the papers of &lt;a href=&quot;https://sites.google.com/site/richardcharlesbarnard/&quot; rel=&quot;noreferrer&quot;&gt;Rick Barnard&lt;/a&gt;, &lt;a href=&quot;http://www.mathcces.rwth-aachen.de/5people/frank/start&quot; rel=&quot;noreferrer&quot;&gt;Martin Frank&lt;/a&gt; and &lt;a href=&quot;https://sites.google.com/site/michaelherty/&quot; rel=&quot;noreferrer&quot;&gt;Michael Herty&lt;/a&gt;, e.g., &lt;a href=&quot;http://arxiv.org/abs/1105.5261&quot; rel=&quot;noreferrer&quot;&gt;http://arxiv.org/abs/1105.5261&lt;/a&gt;. There's also a &lt;a href=&quot;http://dx.doi.org/10.1016/j.laa.2007.03.003&quot; rel=&quot;noreferrer&quot;&gt;paper by J. Tervo, M. Vauhkonen, E. Boman&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, your variables involve the geometry of the problem, which leads to a so-called shape optimization problem. Here the standard reference would be&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898719826&quot; rel=&quot;noreferrer&quot;&gt;M.C. Delfour, J.-P. Zolésio, &lt;em&gt;Shapes and Geometries&lt;/em&gt;, SIAM, 2011&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898718690&quot; rel=&quot;noreferrer&quot;&gt;J. Haslinger, R. A. E. Mäkinen, &lt;em&gt;Introduction to Shape Optimization: Theory, Approximation, and Computation&lt;/em&gt;, SIAM, 2003&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There's also recent work by &lt;a href=&quot;http://www.math.uni-trier.de/~schulz/&quot; rel=&quot;noreferrer&quot;&gt;Volker Schulz&lt;/a&gt;. (I'm not an expert in this area, so these are very selective references to give you some keywords to search for.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-01-30T16:08:26.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18800" PostTypeId="2" ParentId="18794" CreationDate="2015-01-31T00:51:18.817" Score="1" Body="&lt;p&gt;Without knowing the actual values of $\varepsilon$ at the chosen points, the best you can hope to do is to &lt;strong&gt;take a probabilistic approach&lt;/strong&gt;. Doing so will not lead to a guarantee that there are no defects, but a probability that you would have detected a defect if one were present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One tactic would be to check a set of points corresponding to the centers of close-packed hyperspheres within your hypercube. The radius of the hyperspheres will depend on the desired confidence level as well as the distribution from which $\varepsilon$ is chosen. Based on this you could determine the expected volume fraction coverage of the hyperspheres and therefore the probability of detecting a defect if one exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an alternative, if you know the probability distribution function (PDF) for $\varepsilon$ over your hypersphere (e.g. you know the variance of the normal distribution as a function of  $p$) then it is probably possible to use random sampling based on a PDF where the probability of choosing a point in a given volume is proportional to one over the variance of the normal distribution corresponding to $\varepsilon$ (or something similar).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not much of a stats/probability guy, but perhaps others can expand on this idea to provide additional quantitative details.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-01-31T00:51:18.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18801" PostTypeId="2" ParentId="18795" CreationDate="2015-01-31T03:45:43.130" Score="3" Body="&lt;p&gt;In addition to Brian's reference, there was also a review article in SIAM Review a few years ago that summarized the state of research at the time:&#xA;  &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/S0036144598342032&quot; rel=&quot;nofollow&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/S0036144598342032&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-01-31T03:45:43.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18803" PostTypeId="2" ParentId="5202" CreationDate="2015-01-31T09:29:27.943" Score="4" Body="&lt;p&gt;it might be a bit late, but we have some Matlab packages for it including 'classical' node elements and even for edge elements, see&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/27826-fast-assembly-of-stiffness-and-matrices-in-finite-element-method-using-nodal-elements&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/27826-fast-assembly-of-stiffness-and-matrices-in-finite-element-method-using-nodal-elements&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/46635-fast-assembly-of-stiffness-and-matrices-in-finite-element-method-using-edge-elements&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/46635-fast-assembly-of-stiffness-and-matrices-in-finite-element-method-using-edge-elements&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also reports enclosed, so implementation details should be clearer. The main message is, that these FEM assemblies can be done very fast in Matlab as well. For instance, you can easily generate a FEM matrix for 1 milion x 1 milion elements in several seconds, depending on your CPU speed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best, Jan&lt;/p&gt;&#xA;" OwnerUserId="14452" LastActivityDate="2015-01-31T09:29:27.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18804" PostTypeId="2" ParentId="18761" CreationDate="2015-01-31T22:11:34.003" Score="2" Body="&lt;p&gt;&lt;strong&gt;There is already a very clear analog of the Laplacian when restricted to a curve: the second derivative along the curve.&lt;/strong&gt; It's not clear to me why and how you believe that the shape of your curve should have any explicit effect on the analogue of the Laplacian you seek. The second derivative along the curve implicitly accounts for the curve's shape since the values (and therefore their derivatives) presumably depend on the curve's shape. Of course this also relies on choosing the correct metric and coordinates (global or curve-local) for the derivatives. Whether the curve is defined continuously or discretely seems to make no difference to the concept, it only impacts the method used to compute this derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you also look at &lt;a href=&quot;https://math.stackexchange.com/questions/138694/laplace-beltrami-operator-for-curves&quot;&gt;this very similar question&lt;/a&gt; over on &lt;a href=&quot;https://math.stackexchange.com/&quot;&gt;Math.Stackexchange&lt;/a&gt; which essentially just derives a finite difference approximation of the second derivative based on discrete points along a curve. In the case where the points lie on some underlying continuous curve and you take the limit as the edge lengths go to zero this is just equal to the second derivative along the curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you still don't agree that this is the logical analogue to the Laplacian on a discrete curve you will need to more explicitly state precisely how you expect the curvature to play a role other than it's implicit influence on the values at the points.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-02-06T17:02:57.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18805" PostTypeId="1" AcceptedAnswerId="18812" CreationDate="2015-02-01T00:15:38.633" Score="2" ViewCount="608" Body="&lt;p&gt;Given the generalized linear-fractional program:&#xA;$$\text{Minimize}\;\; \max_{i}\Big|\frac{c_i^Tx+d_i}{e_i^Tx+f_i}\Big|$$&#xA;$$\hspace{-5mm}\text{Subject to}\;\; e_i^Tx+f_i&amp;gt;0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I convert this into the form:&#xA;$$\text{Minimize}\;\; t$$&#xA;$$\hspace{26mm}\text{Subject to}\;\; 0\leq c_i^Ty_i+d_iz_i\leq t$$&#xA;$$\hspace{37mm}e_i^Ty+f_iz_i=1$$&#xA;$$\hspace{24mm}z_i&amp;gt;0$$&#xA;where $y_i=\frac{x}{e_i^Tx+f_i}$ and $z_i=\frac{1}{e_i^Tx+f_i}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this is for $i=1,...,k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet when I put this into my solver (SCSSolver with the modeling tool Convex.jl), using actual data given to me in the problem, I get warnings thrown at me about the problem being likely degenerate and something about column pointers not being strictly increasing, none of which I understand.  I can probably go to a different forum to find out what these warnings mean, but is my mathematical approach to the problem correct?&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="6849" LastEditDate="2015-02-01T03:42:57.523" LastActivityDate="2015-02-02T07:50:55.350" Title="Generalized linear-fractional program" Tags="|optimization|convex-optimization|" AnswerCount="1" CommentCount="10" ClosedDate="2015-02-02T15:43:29.267" ContentLicense="CC BY-SA 3.0" />
  <row Id="18807" PostTypeId="1" CreationDate="2015-02-01T08:01:05.157" Score="1" ViewCount="234" Body="&lt;p&gt;I have a homework problem that asks me to show that 1st order unwinding or central differencing can give a strongly conservative, consistent scheme for the 1-D Burger's Equation using a finite volume approach. I think I understand unwinding schemes and central differencing, but I'm not sure what &quot;strongly conservative means or how to demonstrate it.&lt;/p&gt;&#xA;" OwnerUserId="14458" LastEditorUserId="20688" LastEditDate="2019-06-22T19:07:21.310" LastActivityDate="2019-06-22T19:07:21.310" Title="What does &quot;strongly conservative&quot; mean in the context of numerical methods?" Tags="|pde|finite-volume|numerics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18808" PostTypeId="2" ParentId="18807" CreationDate="2015-02-01T16:25:27.160" Score="4" Body="&lt;p&gt;Strong conservative in 1D usually means that the change in the solution is equal to the flux in minus flux out.  For example, for Burgers' equation without source term, you can rewrite it as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{,t} + (u^2)_{,x}/2 = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you integrate both sides in $x$ over an interval $[a,b]$, you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_a^b u_{,t} = u^2(a)/2 - u^2(b)/2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, $u^2/2$ is the flux, and the total rate of change in $u$ over an interval $[a,b]$ is equal to the flux in at one end minus flux out at the other. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-02-01T16:25:27.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18809" PostTypeId="1" CreationDate="2015-02-01T16:29:55.743" Score="1" ViewCount="48" Body="&lt;p&gt;I would like to simulate a 2D classical spin system, whose interactions are only nearest neighbor, using Monte Carlo.  I would like to use Metropolis for updating.  I have seen that when updating one can choose a lattice site at random, or update sequentially.  I was wondering if it is possible to update part of the lattice simultaneously by considering only a sub-lattice.  For instance, let the lattice be $L \times L$ with $L$ even. Indexing from zero, call every site whose sum of their $x$ and $y$ coordinates is odd part of sub-lattice $A$, and every other site sub-lattice $B$.  Is it acceptable to compute probabilities of spin acceptances for all of $A$ simultaneously and then implement those spin changes simultaneously?  Then subsequently for $B$?&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="1574" LastEditDate="2015-02-01T16:56:46.867" LastActivityDate="2015-02-01T17:13:09.130" Title="Monte Carlo update based on sub-lattices" Tags="|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18810" PostTypeId="2" ParentId="11733" CreationDate="2015-02-01T16:45:14.147" Score="0" Body="&lt;p&gt;You can use the trapezium rule as said by Doug in this way (in MATLAB):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cumtrapz(time,data) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is the cumulative trapezoidal integration.&#xA;The 2 inputs are vectors having the same lenght.&#xA;In this way you can for example obtain the velocity integrating the acceleration. &#xA;The output is a vector with the same length of the inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you can use the Simpson's rule as follows: use &lt;code&gt;quadl()&lt;/code&gt; to integrate your data but first you need to create a function in which you interpolate them.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function f = int_fun(x,xdata,ydata)&#xA;f = interp1(xdata,ydata,x);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then feed it to the &lt;code&gt;quadl()&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integral = quadl(@int_fun,A,B,[],[],x,y) % syntax to pass extra arguments&#xA;                                         % to the function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11676" LastActivityDate="2015-02-01T16:45:14.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18811" PostTypeId="2" ParentId="18809" CreationDate="2015-02-01T17:13:09.130" Score="1" Body="&lt;p&gt;There is a large amount of literature on the kind of moves and updates you can do in MCMC methods for spin systems. I would start by reading some of the recent papers of Helmut Katzgraber and the references he has.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-01T17:13:09.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18812" PostTypeId="2" ParentId="18805" CreationDate="2015-02-02T07:50:55.350" Score="2" Body="&lt;p&gt;To answer your question, yes, the mathematical approach you've taken is correct, and standard. You've combined two transformations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Linear-fractional_programming&quot; rel=&quot;nofollow&quot;&gt;Transforming a linear fractional program to a linear program, as shown by Wikipedia&lt;/a&gt;; deducing the transformation is also a problem in some linear programming textbooks (for instance, Bertsimas and Tsitsiklis)&lt;/li&gt;&#xA;&lt;li&gt;Converting a $\max$ function objective into a linear program via constraints, which is typically covered explicitly in linear programming textbooks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;About the only thing I would change would be to change the $z_{i} &amp;gt; 0$ constraint to $z_{i} \geq 0$. Although the former is pedantically correct, no LP solver will actually implement such a constraint, so the closest realizable constraint is the latter. In the unlikely event that you obtain an optimal solution for which a $z_{i} = 0$, you could probably insert a cut that would exclude it.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-02-02T07:50:55.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18814" PostTypeId="1" CreationDate="2015-02-02T10:39:56.770" Score="4" ViewCount="174" Body="&lt;p&gt;I need to find roots for the following function: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(\theta) \equiv E[R(\theta;\eta)]=0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some unknown &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; which is deterministic, while the expectation is taken over a normally distributed random variable &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider for example that &lt;span class=&quot;math-container&quot;&gt;$R(\theta;\eta) = (x+\theta(k+\eta))^{-\gamma} (k+\eta)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x,k$&lt;/span&gt; are given numbers. So to make it simple, I essentially need to search for a &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; such as the expectation becomes zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas for some algorithm? &lt;/p&gt;&#xA;" OwnerUserId="9826" LastEditorUserId="20688" LastEditDate="2019-06-22T22:40:45.717" LastActivityDate="2019-06-22T22:40:45.717" Title="solving for unknown inside an expectation" Tags="|nonlinear-equations|probability|roots|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18815" PostTypeId="1" CreationDate="2015-02-02T11:00:47.710" Score="6" ViewCount="634" Body="&lt;p&gt;I want to calculate the geometric series of a matrix $A$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S=I+A+A^2+\dots+A^n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then apply to a vector $v$, $Sv$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done it in Matlab with a loop and I think it's quite efficient applying the matrix $A$ to $v$ at each step instead of calculating $S$ first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, I tried to compute the series using $S=(I-A^n)(I-A)^{-1}$, but when calculating the inverse I get an error because $A$ is nearly singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the an efficient way to &lt;strong&gt;compute the inverse&lt;/strong&gt; in this case? I thought it would be better than a loop since Matlab is quite good manipulating matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that $A$  has dominant diagonal and tridiagonal.&lt;/p&gt;&#xA;" OwnerUserId="13094" LastActivityDate="2015-12-24T23:21:29.160" Title="Is there a faster method to compute the geometric series of a matrix?" Tags="|matlab|matrix|inverse|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18816" PostTypeId="1" AcceptedAnswerId="18818" CreationDate="2015-02-02T14:17:06.010" Score="0" ViewCount="766" Body="&lt;p&gt;I want to find the argument of a function for which it is minimal. The function is expected to be convex but I cannot evaluate it exactly so I have to deal with the fact that there's noise on top. The noise is purely statistical and I roughly know its magnitude. Essentially, I run a Monte Carlo simulation for each evaluation of the function, so I can even control the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't need to know the minimum with too much precision. The higher the value of the function, the larger the yield of the simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus my requirements are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;really costly function evaluation&lt;/li&gt;&#xA;&lt;li&gt;1D function&lt;/li&gt;&#xA;&lt;li&gt;no derivative available&lt;/li&gt;&#xA;&lt;li&gt;convexity up to statistical errors of know magnitude&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, it's rather important that the whole procedure is not terribly complicated. I'm aware of this thread:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2249/finding-a-global-minimum-of-a-smooth-bounded-non-convex-2d-function-that-is-co&quot;&gt;Finding a global minimum of a smooth, bounded, non-convex 2D function that is costly to evaluate&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but that's just a little too much. I need something simple, yet reliable. I tried a simple bisection scheme but that really didn't handle the noise too well.&lt;/p&gt;&#xA;" OwnerUserId="14467" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-02-02T15:38:59.863" Title="Finding the minimum of a convex function with noisy evaluation" Tags="|convex-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18818" PostTypeId="2" ParentId="18816" CreationDate="2015-02-02T15:38:59.863" Score="1" Body="&lt;p&gt;A standard approach to these kinds of problems is to to evaluate the objective function at several points covering a reasonable range and fit a simple (e.g. a quadratic model) to the points, and then minimize this &quot;response surface.&quot;  You can repeat the process by fitting a new response surface using points near the minimum of the first fitted model.  Response surface modeling or surrogate modeling was discussed in the answers to the earlier question you mentioned- it really is the simplest approach to such problems.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your Monte Carlo simulations produce not just a function value but also an approximate subgradient, then there has been some research into methods that use inexact function values and subgradients.  However, these kinds of approaches typically require a lot of function evaluations.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-02-02T15:38:59.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18819" PostTypeId="1" AcceptedAnswerId="18820" CreationDate="2015-02-02T17:03:30.480" Score="0" ViewCount="1318" Body="&lt;p&gt;I need to install Python to do some undergraduate level computational physics (course, not research). What library do I need? Scipy only?&lt;/p&gt;&#xA;" OwnerUserId="14407" LastActivityDate="2015-02-03T08:27:42.870" Title="What Python libraries do I need to do computational physics?" Tags="|python|scipy|" AnswerCount="1" CommentCount="3" ClosedDate="2015-02-02T18:06:29.427" ContentLicense="CC BY-SA 3.0" />
  <row Id="18820" PostTypeId="2" ParentId="18819" CreationDate="2015-02-02T17:35:09.830" Score="4" Body="&lt;p&gt;You will probably need something to visualize your output such as &lt;code&gt;Matplotlib&lt;/code&gt;. I have used it extensively during my undergrad and PhD (on-going) for day-to-day work and reports and it's excellent:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://matplotlib.org/&quot; rel=&quot;nofollow&quot;&gt;http://matplotlib.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also advise familiarizing yourself with the basic &lt;code&gt;NumPy&lt;/code&gt; functions and syntax (&lt;code&gt;NumPy&lt;/code&gt; is a dependency of &lt;code&gt;SciPy&lt;/code&gt;) so that you can test out small solutions via the interactive python prompt (&lt;code&gt;iPython&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of documenting your progress (very important for any assessed work), check out the &lt;code&gt;iPython&lt;/code&gt; notebook. This is web browser based and allows you to store and run your code blocks in the browser, interspersed with comment boxes. You can also insert $\LaTeX$ equations. Here's a fantastic series of examples by Lorena Barba's CFD group:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://nbviewer.ipython.org/github/barbagroup/CFDPython/blob/master/lessons/01_Step_1.ipynb&quot; rel=&quot;nofollow&quot;&gt;http://nbviewer.ipython.org/github/barbagroup/CFDPython/blob/master/lessons/01_Step_1.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To restore the interactivity, you will need to download the .ipynb file and run using the &lt;code&gt;iPython&lt;/code&gt; notebook in a terminal [if on *Nix].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is to install Enthought Canopy - it's a python distribution specifically aimed at scientists and analysts. The free academic version comes with several hundred plugins for tasks such as image analysis, in addition to the basic modules mentioned above. I've used it extensively and particularly like the facility for updating/downloading new packages very easily:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.enthought.com/products/canopy/&quot; rel=&quot;nofollow&quot;&gt;https://www.enthought.com/products/canopy/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9941" LastEditorUserId="9667" LastEditDate="2015-02-03T08:27:42.870" LastActivityDate="2015-02-03T08:27:42.870" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="18821" PostTypeId="2" ParentId="18814" CreationDate="2015-02-02T20:28:32.687" Score="5" Body="&lt;p&gt;Since you are assuming $\eta$ is normal what i would do is try to compute the expectation as fast as possible for every $\theta$. So I would compute the expectation using any numerical integration I might know. That is, if $n$ and $N$ are the Gaussian density and distribution respectively, &#xA;$$ f( \theta ) = \int_{ -\infty }^\infty R(\theta, \eta ) n( \eta ) d\eta = \int_0^1 R( \theta, N^{-1}(w ) ) dw\\&#xA;=\sum_{j=1}^n \int_{w_{j-1} }^{w_j} R( \theta, N^{-1}(w ) ) dw,$$ &#xA;where $0=w_0 &amp;lt; w_1 &amp;lt; ....&amp;lt; w_n = 1 $. You can use any integration tricks on each of the integrals above. For example, take $w_j = N(z_j)$ for some partition $(z_j)_{j&amp;gt;0 }$ of the real line, and approximate &#xA;$$&#xA;\int_{w_{j-1} }^{w_j} R( \theta, N^{-1}(w ) ) dw = R( \theta, N^{-1}(w_j ) )( w_j - w_{j-1} ) = R( \theta, z_j )( w_j - w_{j-1} ).&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have this step, just use any root finding method like brent's algorithm to find $\theta$. The integration needs to be fast because every step in the root finding is going to perform a new integration. Hence in your implementation you should be able to re-use parameters in the integration step, for example $z$, $w$, etc. &lt;/p&gt;&#xA;" OwnerUserId="11923" LastEditorUserId="11923" LastEditDate="2015-02-02T20:36:53.460" LastActivityDate="2015-02-02T20:36:53.460" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18822" PostTypeId="1" AcceptedAnswerId="18824" CreationDate="2015-02-02T21:24:25.403" Score="1" ViewCount="87" Body="&lt;p&gt;Although I already did some work in the intersection of theory and simulation I'm still very new to this field and I need some guidance. If anybody can give some recommendations for introductory literature about computational science and how to plan scientific software I appreciate it. Below are some more concrete questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the work for my thesis I did and still do numerical simulations of a quantum-mechanical system and I encountered the following problem. Basically, the three important steps of these simulations are: solving a system of ODEs, computation of physical observables using this solution, and plotting of the observables. My first question is, when it comes to writing the software what is the best practice for the separation of these steps? Should a program do all of these steps or is it better to divide the steps between different scripts and save the data in between?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My concern is that my current approach is chaotic and not efficient. My first scripts did all the steps in one go. When the ODE solver took a lot of time for the solution, I changed the script to save the raw solution for later purposes. My current approach is to calculate the solution and the quantities of interest in one go, and save the data to text files. The plotting is done by separate scripts. Currently it takes me about 0.5-3h to revise a figure and imho this is very slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there are a lot of parameters, there is a lot of room for choosing which variables are fixed, which are varied, what the x-axis is, and so forth. This led me to write a lot of small scripts each tailored to a specific region of the parameter space and a specific choice of observables which subsequently led to a complicated and fragmented folder and file structure. Is this a good style or should one choose to write programs that cover all possible cases and observables?&lt;/p&gt;&#xA;" OwnerUserId="7649" LastActivityDate="2015-02-03T00:11:44.470" Title="Help on writing sofware: general guidelines, in particular separation of computation and visualization" Tags="|software|visualization|efficiency|plotting|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18823" PostTypeId="2" ParentId="18822" CreationDate="2015-02-02T22:19:01.377" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Should a program do all of these steps or is it better to divide the steps between different scripts and save the data in between?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Keep them separate.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason to consider integrating a simulation (solving ODEs) and analysis is to avoid writing data to disk and then reloading it for the analyses.  If your ODE is hard to solve (more than O(N)) and you change your analysis with any regularity, then loading an old result from disk will be a lot faster than regenerating a new one.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is this a good style or should one choose to write programs that cover all possible cases and observables?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How broad or narrow your tools should be is very situation specific.  In particular, it depends on:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What you're trying to do&lt;/li&gt;&#xA;&lt;li&gt;What 3rd party programs/libraries you are using (gnuplot, matplotlib, etc)&lt;/li&gt;&#xA;&lt;li&gt;How good of a programmer you are&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You should try to strike a balance between investing time in building tools that make your science easier and doing the actual science.  If you're more comfortable writing a relatively large number of specialized tools, there's nothing wrong with that.  On the other hand, if you can spend 5 minutes to generalize a plotting script in a way that eliminates 10 others and saves you time, you should do that.&lt;/p&gt;&#xA;" OwnerUserId="521" LastEditorUserId="521" LastEditDate="2015-02-02T22:30:24.043" LastActivityDate="2015-02-02T22:30:24.043" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18824" PostTypeId="2" ParentId="18822" CreationDate="2015-02-03T00:11:44.470" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My first question is, when it comes to writing the software what is the best practice for the separation of these steps? Should a program do all of these steps or is it better to divide the steps between different scripts and save the data in between?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How you separate the steps depends on many things, like how long it takes to solve the ODEs, what software environment you are using, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, keep things as simple as they can possibly be while still being productive. If you're writing your ODE solver code in MATLAB or Python, it's fairly easy to keep the visualization step in the same script, as long as the ODE solve isn't unacceptably slow. In a compiled language, it's usually easier write to output and then use a third-party plotting tool.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My concern is that my current approach is chaotic and not efficient. My first scripts did all the steps in one go. When the ODE solver took a lot of time for the solution, I changed the script to save the raw solution for later purposes. My current approach is to calculate the solution and the quantities of interest in one go, and save the data to text files. The plotting is done by separate scripts. Currently it takes me about 0.5-3h to revise a figure and imho this is very slow.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This iterative approach is exactly what you should be doing. As a first hack, doing all the steps in one go makes sense. It's like a first draft of a document; putting everything down in code helps you see what needs to be done, and what can be improved. Sometimes, some design work can be done in advance, but in general, whatever is written will need to be revised.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since there are a lot of parameters, there is a lot of room for choosing which variables are fixed, which are varied, what the x-axis is, and so forth. This led me to write a lot of small scripts each tailored to a specific region of the parameter space and a specific choice of observables which subsequently led to a complicated and fragmented folder and file structure. Is this a good style or should one choose to write programs that cover all possible cases and observables?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'd say, in general, pick tools that work for you. If you find that a lot of the scripts you're writing are cut-and-paste, that's probably a sign you can make your scripts more general.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do you know some literature about scicomp or designing scientific software?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.software-carpentry.org/&quot; rel=&quot;nofollow&quot;&gt;Software Carpentry&lt;/a&gt; maintains a &lt;a href=&quot;http://software-carpentry.org/bib/bib.html&quot; rel=&quot;nofollow&quot;&gt;list of suggested materials&lt;/a&gt;. I've read some of the suggested works (for instance, Code Complete) and found them to be helpful. Some sources have more of a general software engineering focus, and are useful because they impart the decades of experience software engineers have in best practices. A more narrowly focused book that might be of use is &lt;a href=&quot;http://software-carpentry.org/bib/bib.html&quot; rel=&quot;nofollow&quot;&gt;Writing Scientific Software: A Guide to Good Style&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I have taught two Software Carpentry workshops.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-02-03T00:11:44.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18825" PostTypeId="1" AcceptedAnswerId="18833" CreationDate="2015-02-03T01:48:17.740" Score="3" ViewCount="1976" Body="&lt;p&gt;I have a fitting routine set up. It works, but pretty slow. I was wondering if there is a better method to use. I checked my (forward) code against some literature data and at least I do have no bugs and mistakes in my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fit the steric mass action model to some some isotherms like the ones shown below.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/wYUfG.png&quot; alt=&quot;my measured data&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, i assume some arbitrary numbers of unknown components and fit just the measured concentrations of my known components (it is a multicomponent system). Additionally, the measured compound itself consists of some charge variants with different adsorption characteristics. Therefore, I am playing around right know with the number of components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model for a three component system takes following form. (the first component (the salt concentration) has to be calculated numerically)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;   qs &amp;amp;=&amp;amp; \frac{\Lambda \, c_1}{c_1 + (\sigma_2 + \nu_2)* \{ k_2 \, ( \frac{qs}{c_1} )\}^{\nu_2 - 1} + (\sigma_3 + \nu_3)* \{ k_3 \, ( \frac{qs}{c_1} )\}^{\nu_3 - 1} }  \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;   q2 &amp;amp;=&amp;amp; \frac{\Lambda \, {k \, ( \frac{qs}{c_1} )^{\nu_2 - 1} } }{c_1 + (\sigma_2 + \nu_2)* \{ k_2 \, ( \frac{qs}{c_1} )\}^{\nu_2 - 1} + (\sigma_3 + \nu_3)* \{ k_3 \, ( \frac{qs}{c_1} )\}^{\nu_3 - 1} }  \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;   q3 &amp;amp;=&amp;amp; \frac{\Lambda \, {k \, ( \frac{qs}{c_1} )^{\nu_3 - 1} } }{c_1 + (\sigma_2 + \nu_2)* \{ k_2 \, ( \frac{qs}{c_1} )\}^{\nu_2 - 1} + (\sigma_3 + \nu_3)* \{ k_3 \, ( \frac{qs}{c_1} )\}^{\nu_3 - 1} }  \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;qs is the stationary concentration of free ligands [mM] typically between 50 and 1500&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Lambda$ is initial concentration of ligands; can be a fit parameter; typically between 100 and 1700&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k$ is a fit parameter typically between 0.1 and 40&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sigma $ is a fit parameter typically between 1 and 50&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nu $ is a fit parameter typically between 1 and 15&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the solver fails on some sets of input parameters and i do not want to fiddle around, i use scipy`s &lt;a href=&quot;http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.optimize.basinhopping.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;basin-hopping&lt;/a&gt; solver together with the method L-BFGS-B (quite slow). The parameters are bound so they make physical sense. Starting points are generated randomly within the bounded domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering how someone used to optimization would set this problem up ? Would you try to provide an analytical gradient ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do minimize the sum of squares. I thought about &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.leastsq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy`s leastsq&lt;/a&gt;, but it does not accept bounds. From the local solvers, &lt;a href=&quot;http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.optimize.fmin_l_bfgs_b.html#scipy.optimize.fmin_l_bfgs_b&quot; rel=&quot;nofollow noreferrer&quot;&gt;fmin&lt;/a&gt; (using L-BFGS-B) was the most robust and fastest one.&lt;/p&gt;&#xA;" OwnerUserId="3957" LastEditorUserId="3957" LastEditDate="2015-02-03T06:08:25.057" LastActivityDate="2015-02-04T01:21:54.200" Title="optimization (using python) how to find the most suitable solver?" Tags="|optimization|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18826" PostTypeId="2" ParentId="18738" CreationDate="2015-02-03T05:33:54.670" Score="0" Body="&lt;p&gt;The symbolic solution is direct in &lt;em&gt;Mathematica&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;\[Rho] h a b Integrate[Sin[\[Eta] \[Pi]/4 + 3 \[Pi]/4]^4  Sin[\[Psi] \[Pi]/4 + 3 \[Pi]/4]^4, {\[Eta], 0, 1}, {\[Psi], 0, 1}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$\frac{(8-3 \pi )^2 a b h \rho }{64 \pi ^2}$&lt;/p&gt;&#xA;" OwnerUserId="14471" LastActivityDate="2015-02-03T05:33:54.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18827" PostTypeId="2" ParentId="16281" CreationDate="2015-02-03T05:52:37.557" Score="0" Body="&lt;p&gt;In &lt;em&gt;Mathematica&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Animate[Plot[Sin[x + t], {x, -5, 5}], {t, 0, 10}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14471" LastActivityDate="2015-02-03T05:52:37.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18828" PostTypeId="1" CreationDate="2015-02-03T14:42:52.623" Score="0" ViewCount="1752" Body="&lt;p&gt;I'm trying to write a Python code that is a numerical solver for 1-d heat conduction (using FVM) with a temperature dependent thermal conductivity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solver has three functions I need to iterate until convergence. This is basically the algorithm from Patankar's Numerical Heat Transfer and Fluid Flow book. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conductivity function: takes an initial guess for temperature field and outputs nodal conductivities, dependent on past value of nodal temperatures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nodal coefficients: calculates nodal coefficients based on current nodal conductivities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nodal temperatures: uses a TDMA algorithm to calculate the nodal temperatures based on current coefficients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the temperatures are calculated, the values are to be cycled through to the conductivity function to continue the loop until convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can make this work just fine when the conductivity is constant, but start getting issues when I try to implement this pseudocode. I get a maximum recursion depth exceeded error. My code is below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import copy&#xA;&#xA;# SOLUTION ALGORITHM&#xA;&#xA;# Step 1: DATA  &#xA;def data(n, hw, he, TinfW, TinfE):    &#xA;    n = 5&#xA;    hw = 10**10&#xA;    he = 10**-10&#xA;    TinfW = 100&#xA;    TinfE = 10**12&#xA;&#xA;    return n, hw, he, TinfW, TinfE    &#xA;&#xA;# Step 2: GRID    &#xA;def grid(x1, x2, data):&#xA;    x1 = 0 &#xA;    x2 = 0.5&#xA;    xe = (x2-x1) / data(n, hw, he, TinfW, TinfE)[0]&#xA;&#xA;    return xe&#xA;&#xA;# Step 3: GAMSOR     &#xA;def gamsor(k, q, TDMA):&#xA;    k0 = 100&#xA;    beta = 20&#xA;    T0 = 25   &#xA;    for i in range(data(n, hw, he, TinfW, TinfE)[0]):&#xA;        k[i] = k0 + beta#*(TDMA(data, coeff)[i]-T0)&#xA;        q[i] = 10**5&#xA;&#xA;    return k, q&#xA;&#xA; #Step 4: COEFF     &#xA;def coeff(Ae, Aw, Ap, data, gamsor, grid):&#xA;    for i in range(data(n, hw, he, TinfW, TinfE)[0]):       &#xA;        Ae[i] = Aw[i] = gamsor(k,q, TDMA(data, coeff))[0][i] /     grid(x1, x2, data)&#xA;        Ap[i] = Ae[i] + Aw[i]  &#xA;&#xA;# Define &quot;b&quot; array - including source term and boundary conditions&#xA;        b = np.zeros((data(n, hw, he, TinfW, TinfE)[0], 1))&#xA;        b.fill(gamsor(k,q, TDMA(data, coeff))[1][i])    &#xA;&#xA;        b[0] = (gamsor(k,q, TDMA(data, coeff))[1][i]*(grid(x1,x2,data)/2) + &#xA;           data(n, hw, he, TinfW, TinfE)[1]*&#xA;           data(n, hw, he, TinfW, TinfE)[3])&#xA;&#xA;        b[-1] = (gamsor(k,q, TDMA(data, coeff))[1][i]*(grid(x1,x2,data)/2)+ &#xA;            data(n, hw, he, TinfW, TinfE)[2]*&#xA;            data(n, hw, he, TinfW, TinfE)[4])&#xA;&#xA;# Change values in first and last coefficients to reflect specified BC's&#xA;        Aw[0] = 0&#xA;        Ap[0] = Ae[0] + Aw[0] + data(n, hw, he, TinfW, TinfE)[1]&#xA;&#xA;        Ae[-1] = 0&#xA;        Ap[-1] = Ae[0] + Aw[0] + data(n, hw, he, TinfW, TinfE)[2]&#xA;&#xA;    return Ae, Aw, Ap, b&#xA;&#xA; #Step 5: SOLVE&#xA;def TDMA(data, coeff):&#xA;# Initialize &quot;T&quot; array - the solution array&#xA;    T = np.zeros(data(n, hw, he, TinfW, TinfE)[0]);&#xA;&#xA;# Initialize recursion functions as arrays&#xA;    P = np.zeros(data(n, hw, he, TinfW, TinfE)[0]);&#xA;    Q = np.zeros(data(n, hw, he, TinfW, TinfE)[0]);&#xA;&#xA;## Step 1: evaluate at node 1&#xA;    P[0] = (coeff(Ae, Aw, Ap, data, gamsor, grid)[0][0] / &#xA;       coeff(Ae, Aw, Ap, data, gamsor, grid)[2][0])&#xA;&#xA;    Q[0] = (coeff(Ae, Aw, Ap, data, gamsor, grid)[3][0] / &#xA;       coeff(Ae, Aw, Ap, data, gamsor, grid)[2][0])&#xA;&#xA;## Step 2: sweep from node 2 to node (n-1) (python&#xA;## cell 1-&amp;gt;(n-2) ) evaluating P and Q&#xA;&#xA;    for i in range(1, data(n, hw, he, TinfW, TinfE)[0]-1):&#xA;        P[i] = ((coeff(Ae, Aw, Ap, data, gamsor, grid)[0][i]) / &#xA;           (coeff(Ae, Aw, Ap, data, gamsor, grid)[2][i] - &#xA;           coeff(Ae, Aw, Ap, data, gamsor, grid)[1][i]*P[i-1]))&#xA;&#xA;        Q[i] = ((coeff(Ae, Aw, Ap, data, gamsor, grid)[3][i] + &#xA;           coeff(Ae, Aw, Ap, data, gamsor, grid)[1][i]*Q[i-1]) / &#xA;           (coeff(Ae, Aw, Ap, data, gamsor, grid)[2][i] - &#xA;           coeff(Ae, Aw, Ap, data, gamsor, grid)[1][i]*P[i-1]))&#xA;&#xA;## Step 3: calculate for node n&#xA;&#xA;    P[data(n, hw, he, TinfW, TinfE)[0]-1] = 0&#xA;&#xA;    Q[data(n, hw, he, TinfW, TinfE)[0]-1] = ( &#xA;(coeff(Ae, Aw, Ap, data, gamsor, grid)[3][data(n, hw, he, &#xA;TinfW, TinfE)[0]-1] + coeff(Ae, Aw, Ap, data, gamsor, grid)&#xA;[1][data(n, hw, he, TinfW, TinfE)[0]-1]*Q[data(n, hw, he, TinfW, &#xA;TinfE)[0]-2]) / (coeff(Ae, Aw, Ap, data, gamsor, grid)[2][data(n, &#xA;hw, he, TinfW, TinfE)[0]-1] - coeff(Ae, Aw, Ap, data, gamsor, &#xA;grid)[1][data(n, hw, he, TinfW, TinfE)[0]-1]*P[data(n, hw, he, &#xA;TinfW, TinfE)[0]-2]))&#xA;&#xA;    T[data(n, hw, he, TinfW, TinfE)[0]-1] = ( &#xA;    Q[data(n, hw, he, TinfW, TinfE)[0]-1] )&#xA;&#xA;## Step 4: back fill, giving the temperatures&#xA;    for i in range(data(n, hw, he, TinfW, TinfE)[0] - 2, -1, -1):&#xA;        T[i] = P[i]*T[i+1] + Q[i]&#xA;&#xA;    return T&#xA;&#xA;def Temps(TDMA):&#xA;    Temps = copy.copy(TDMA(data,coeff)); Temps.fill(100)&#xA;    return Temps&#xA;&#xA;if __name__ == '__main__': &#xA;&#xA;# Initialize variables&#xA;    n = 0&#xA;    hw = 0&#xA;    he = 0&#xA;    TinfW = 0&#xA;    TinfE = 0&#xA;&#xA;    k = np.zeros(data(n, hw, he, TinfW, TinfE)[0])&#xA;    q = np.zeros(data(n, hw, he, TinfW, TinfE)[0])&#xA;    k0 = 0&#xA;    beta = 20&#xA;    T0 = 0&#xA;&#xA;    x1 = 0&#xA;    x2 = 0&#xA;&#xA;    Ae = np.zeros(data(n, hw, he, TinfW, TinfE)[0])&#xA;    Aw = np.zeros(data(n, hw, he, TinfW, TinfE)[0])&#xA;    Ap = np.zeros(data(n, hw, he, TinfW, TinfE)[0])&#xA;    b = 0&#xA;&#xA;    data(n, hw, he, TinfW, TinfE)&#xA;    grid(x1, x2, data)&#xA;&#xA;    Temps(TDMA)&#xA;    for _ in range(1):&#xA;        gamsor(k, q)&#xA;        coeff(Ae, Aw, Ap, data, gamsor, grid)&#xA;        TDMA(data, coeff)&#xA;&#xA;#print Temps(TDMA) &#xA;&#xA;&#xA;&#xA;    print &quot;Nodal Temperature Distribution (C):&quot;, TDMA(data, coeff) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14475" LastActivityDate="2015-02-03T18:38:30.843" Title="Temperature dependent 1-d conduction in Python?" Tags="|python|convergence|finite-volume|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18829" PostTypeId="2" ParentId="14785" CreationDate="2015-02-03T15:25:18.850" Score="-1" Body="&lt;p&gt;I have been using octave for a while now. It is the open-source cousin of Matlab and it has many decent solvers lsode for odes and daspk for DAEs are the ones I use and they are pretty good.&lt;/p&gt;&#xA;" OwnerUserId="6701" LastActivityDate="2015-02-03T15:25:18.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18830" PostTypeId="1" CreationDate="2015-02-03T15:30:55.363" Score="0" ViewCount="422" Body="&lt;p&gt;I am trying to create an animation of a 3d vector field in ParaView.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All my data is in three matrices (201 x 201 x 201) corresponding to vectors components in x,y,z directions. And it's calculated in MATLAB. &#xA;If I do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;B = cat(4,Bx,By,Bz);&#xA;B = permute(B,[4 1 2 3]);&#xA;fid = fopen('Bvec.raw','w'); fwrite(fid,B,'float'); fclose(fid);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can load Bvec.raw into ParaView and visualise the vector field at one instant of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how would I go about making an animation if I had say 100 (Bx, By, Bz) triples at different instants of time? Without creating 100 different raw files and loading them into paraview by hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ALSO, I'm not set on using paraview, it's just the first software I tried, so if you think there's an easier/better/ more competent way of going about it I'd appreciate suggestions.&lt;/p&gt;&#xA;" OwnerUserId="14474" LastActivityDate="2015-02-03T15:30:55.363" Title="3D animation of a vector field (ParaView)" Tags="|matlab|visualization|vector|paraview|" AnswerCount="0" CommentCount="2" ClosedDate="2018-03-16T16:55:50.610" ContentLicense="CC BY-SA 3.0" />
  <row Id="18831" PostTypeId="1" CreationDate="2015-02-03T16:39:57.123" Score="1" ViewCount="53" Body="&lt;p&gt;The following image represents the phase of a wavefunction (in radians) on a square lattice, where $m$ and $n$ label the lattice sites. Computationally speaking, it is the density plot of a 41x41 real matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how can I numerically compute the total change in phase as one goes around a closed loop centered on the central site $m=n=0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XsImy.png&quot; alt=&quot;phase&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4656" LastActivityDate="2015-02-04T11:43:41.937" Title="compute change of phase along closed contour" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18832" PostTypeId="2" ParentId="18828" CreationDate="2015-02-03T18:38:30.843" Score="2" Body="&lt;p&gt;If you are interested in a pre-existing solver you should take a look a FiPy. It's a Finite-Volume method solver implemented in Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ctcms.nist.gov/fipy/&quot; rel=&quot;nofollow&quot;&gt;http://www.ctcms.nist.gov/fipy/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a 1D example: (&lt;a href=&quot;http://www.ctcms.nist.gov/fipy/examples/diffusion/generated/examples.diffusion.mesh1D.html&quot; rel=&quot;nofollow&quot;&gt;http://www.ctcms.nist.gov/fipy/examples/diffusion/generated/examples.diffusion.mesh1D.html&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example shows how to work with time-dependent b.c.'s and varying diffusion coefficients. It might be useful for you. &lt;/p&gt;&#xA;" OwnerUserId="10225" LastActivityDate="2015-02-03T18:38:30.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18833" PostTypeId="2" ParentId="18825" CreationDate="2015-02-04T01:21:54.200" Score="6" Body="&lt;p&gt;Some general thoughts from someone with optimization experience:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you can provide an analytical gradient, do it. In addition, providing an analytical Hessian is even better. Calculating a gradient with finite differences results in a loss of precision for ill-conditioned problems; you'll lose about half your significant figures in the best case, and in worse cases, more than that. Low-rank approximations to the Hessian such as BFGS and TR1 will retard convergence somewhat, but the improvement from a low-rank Hessian approximation to an analytical Hessian is not as great as the improvement in going from a finite difference gradient to an analytical gradient.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For a bound-constrained least-squares problem, methods like Gauss-Newton and L-BFGS-B are good first choices. For nontrivial constraints, you should look at an active set method (for instance, SLSQP in SciPy is an SQP method; OpenOpt interfaces to ALGENCAN, an augmented Lagrangian approach) or an interior point method (pyipopt interfaces to IPOPT). Active set methods tend to be good for small- to medium-scale problems; interior point methods tend to be good for large-scale problems.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your nonlinear least squares problem is nonconvex, you should consider a globalization strategy like multistart to augment your choice of local optimization solver in the previous bullet. An actual nonconvex optimizer would be preferable, but those methods tend to be limited to small scale problems, are intrusive, and generally do not have Python interfaces.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-02-04T01:21:54.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18834" PostTypeId="1" CreationDate="2015-02-04T02:27:17.400" Score="1" ViewCount="71" Body="&lt;p&gt;I'm looking for advice, or references, for a change of basis to my dependent variables that leads to a less computationally expensive scheme when solving a system of coupled polynomial equations. Below is an explanation of the problem, which will be somewhat involved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general idea is this. I have a system of equations I'm solving for the dependent variables $a=(a_1,...,a_N)$, all of them are a function of a parameter $\mu$. That is, my system takes the form ( I will call this method 1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(a; \mu)=0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a fixed (prescribed) value of $\mu\in(0,1)$, I solve this system using a Newton-Raphson method (implemented in SUNDIALS), increasing $N$ until my solutions converge to some given precision. For large values of $\mu$, $N$ grows very large and at a certain point becomes very impractical (&lt;strong&gt;&lt;em&gt;see my note at the bottom for more about this&lt;/em&gt;&lt;/strong&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, these variables $a_i$ correspond to Fourier coefficients of a physical boundary displacement. In particular, they are Fourier coefficients to two parametric functions $(X(x),Y(x))$, with $X= x+\sum_{n=1}^N a_n \sin n x$, $Y=\sum_{n=1}^N a_n \cos n x$, and $x\in (-\pi,\pi)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a simple example, I have plotted $(X,Y)$, $(x,X(x))$, and $(x,Y(x))$ in black in the figure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, so there exists another way to formulate this problem, using a different set of variables that leads to a very different set of equations that returns the same information about the boundary displacement. Using &lt;code&gt;ginput&lt;/code&gt; in Matlab, I can extract the boundary displacements based on results in the literature that employ this other method (call it method 2), to find a set $(U(x),V(x))$, $(x,U(x))$, and $(x,V(x))$, where the independent variable $x$ is exactly the same as before. These are the red plots in the figure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/GosBG.png&quot; alt=&quot;enter image description here&quot;&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we note that $(U,V)$ and $(X,Y)$ give approximately the same plot, but $U$ and $V$ are not nearly as localized in space as $X$ and $Y$. This implies that $U$ and $V$ need much lower spectral resolution than $X$ and $Y$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This motivates me to look for a change of basis in my original problem, to one that needs lower resolution. In particular, this later approach makes me think that I'm working too hard to get at these solutions, and this becomes particularly apparent as I try to describe the system with higher values of $\mu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I realize I should just figure out a mapping between the two methods, but this is very difficult due to the fact that both methods use a series of mappings to get to the final set of equations. So, this information is not available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;My question&lt;/em&gt;&lt;/strong&gt;: Is there any general way, or are there any instances where people have found a way, to do a simple transformation of my variables $a$, to converge for smaller $N$. (I'm not sure if this is the same thing as asking about $preconditioning$ the jacobian of $F$, to allow for more rapid convergence of the Newton-Raphson scheme?). If so, how does one look for this transformation ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I apologize for the vague question, and please feel free to ask any questions of clarification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Note&lt;/em&gt;&lt;/strong&gt;: I'm actually solving an ODE with a nontrivial set of coefficients on the highest order derivative, so I'm really solving a system $F(a, t_i)$ for each time step $t_i$. This makes it very important for me to optimize the time taken to solve $F(a)$, as I will do this many, many times. &lt;/p&gt;&#xA;" OwnerUserId="8021" LastActivityDate="2015-02-04T02:27:17.400" Title="Mapping to a computationally less expensive basis when employing Newton's method" Tags="|newton-method|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18835" PostTypeId="2" ParentId="16327" CreationDate="2015-02-04T10:23:11.247" Score="1" Body="&lt;p&gt;I think it's best to solve the individual 3 x 3 systems. The problems of doing such a large matrix, containing millions of 3 x 3 systems:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;computational cost of assembly&lt;/strong&gt;: assembling such a large matrix could be take a substantial amount of time (even with a sparse solver).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;parallelization overheads&lt;/strong&gt;: should there be a need to parallelize the code, the large matrix would have to be distributed over many processors and there would be communication overheads for passing information between processors. On the other hand, for individual 3 x 3 systems, the coefficients could easily be distributed over the various processors, after which it would be an &quot;embarrassingly parallel&quot; operation (i.e. no communication between processors would be necessary).  Furthermore, parallelization using OpenMP or MPI would not be too difficult.  On the other hand, for the large matrix, it would probably be easier to make use of a library (such as PETSc: e.g. the example &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.4/src/ksp/ksp/examples/tutorials/ex2.c.html&quot; rel=&quot;nofollow&quot;&gt;ex2.c&lt;/a&gt; is a nice start), rather than to write your own code.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Lastly, I would like to point out that for 3 x 3 systems, it's easy to work out an analytical formula using computer algebra systems (or using Wolfram Alpha).  As an illustration, here's how it can be done with the Python package, sympy:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sympy import *&#xA;x1,x2,x3=symbols('x1 x2 x3')&#xA;b1,b2,b3=symbols('b1 b2 b3')&#xA;a11,a12,a13=symbols('a11 a12 a13')&#xA;a21,a22,a23=symbols('a21 a22 a23')&#xA;a31,a32,a33=symbols('a31 a32 a33')&#xA;eq1=Eq(a11*x1+a12*x2+a13*x3,b1)&#xA;eq2=Eq(a21*x1+a22*x2+a23*x3,b2)&#xA;eq3=Eq(a31*x1+a32*x2+a33*x3,b3)&#xA;x=solve([eq1,eq2,eq3], [x1,x2,x3])&#xA;x.get(x1)&#xA;x.get(x2)&#xA;x.get(x3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just replace x1, x2, x3, b1, b2, b3, etc. with the actual variables in your existing code.&#xA;If generation of C code is required:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[(c_name, c_code), (h_name, c_header)] = codegen((&quot;x1&quot;,x.get(x1)), &quot;C&quot;, &quot;test&quot;, header=False, empty=False)&#xA;print(c_code)&#xA;[(c_name, c_code), (h_name, c_header)] = codegen((&quot;x2&quot;,x.get(x2)), &quot;C&quot;, &quot;test&quot;, header=False, empty=False)&#xA;print(c_code)&#xA;[(c_name, c_code), (h_name, c_header)] = codegen((&quot;x3&quot;,x.get(x3)), &quot;C&quot;, &quot;test&quot;, header=False, empty=False)&#xA;print(c_code)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sample output (for just x1):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; print(c_code)&#xA;#include &quot;test.h&quot;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;double x1(double a11, double a12, double a13, double a21, double a22, double a23, double a31, double a32, double a33, double b1, double b2, double b3) {&#xA;   return (a12*a23*b3 - a12*a33*b2 - a13*a22*b3 + a13*a32*b2 + a22*a33*b1 - a23*a32*b1)/(a11*a22*a33 - a11*a23*a32 - a12*a21*a33 + a12*a23*a31 + a13*a21*a32 - a13*a22*a31);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here it may make sense to pre-calculate the determinant to further reduce the computational cost, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;determinant = (a11*a22*a33 - a11*a23*a32 - a12*a21*a33 + a12*a23*a31 + a13*a21*a32 - a13*a22*a31);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11689" LastActivityDate="2015-02-04T10:23:11.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18836" PostTypeId="2" ParentId="18831" CreationDate="2015-02-04T11:43:41.937" Score="1" Body="&lt;p&gt;Let's write the closed loop as a series of lattice sites $ (m_1, n_1), (m_2, n_2), \ldots (m_K, n_K) $ where each site in the series should be a neighbor of the preceding site. Denote the phase at the $i$th site as $\phi_i$. If the phase is well approximated as a continuous function, it's trivial to compute the phase accumulated around the contour: $(\phi_2 - \phi_1) + (\phi_3 - \phi_2) + \ldots (\phi_K - \phi_{K-1}) + (\phi_1 - \phi_K) = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interesting part comes when the phase is discontinuous. When the phase at a site exceeds $2\pi$ or falls below $0$, your numerical algorithm typically &quot;wraps&quot; the phase by subtracting or adding $2 \pi$. This is clearly happening in your plot quite a bit, for instance when you go from $(m,0)$ to $(m,1)$ with $m \gg 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concretely, if you see that $\phi_j = 2 \pi - \epsilon_1$ and $\phi_{j+1} = \epsilon_2$ for $\epsilon_1, \epsilon_2 \ll 1$, you probably have a phase wrapping of $+2\pi$. Similarly you could have $-2\pi$ phase wrappings. Between $i = 1$ and $i = K$, sum up all the phase wrappings. This sum is equal to the phase accumulated around the contour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the phase steps in code, you can just test each successive pair of sites in your contour as in the example above. However, the numerical difficulty comes in choosing $\epsilon_1$ and $\epsilon_2$. If your data is too coarse, you will not be able to choose them small enough for reliable unwrapping.&lt;/p&gt;&#xA;" OwnerUserId="14481" LastActivityDate="2015-02-04T11:43:41.937" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="18837" PostTypeId="1" CreationDate="2015-02-04T13:38:27.380" Score="1" ViewCount="148" Body="&lt;p&gt;What is the difference (in terms of e.g robustness and speed) between proving a gradient obtained by an AD package (like PyAutoDiff) and let the solver (e.g BSGS) calculate the gradient ? It seems so tempting to do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from autodiff import gradient&#xA;&#xA;def my_costfun(x):&#xA;    return f(x)&#xA;&#xA;@gradient&#xA;    def g(x):&#xA;    return f(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is that really that simple ? What about functions with, lets say 10 parameters ?&lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2015-02-04T22:04:18.717" Title="Optimization: Providing gradients by autodifferentiation e.g PyAutoDiff" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18839" PostTypeId="2" ParentId="18837" CreationDate="2015-02-04T16:12:48.043" Score="1" Body="&lt;p&gt;If the cost function is complex or has lots of parameters, calculating the gradient with automatic differentiation can grow in cost rapidly. Often exact gradients aren't needed (especially far from the minimum), and so other strategies may be much, much cheaper.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-02-04T16:12:48.043" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="18841" PostTypeId="1" CreationDate="2015-02-04T17:35:20.967" Score="3" ViewCount="102" Body="&lt;p&gt;My apologies if the question is simple. I need to write a code for straightening algorithm. Which includes defining bracket algebra. I tried to write it in CoCoA-5, but it wasn't possible because CoCoA-5 does not accept brackets like [1,4,6,8] as ring's indeterminates. I did some Googleing and got more confused.&#xA;Can anyone introduce a Computer Algebra System which is capable of defining bracket algebra, computing G-Basis, defining ideals and other abilities which are required in Lie algebra and commutative algebra?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14483" LastActivityDate="2015-02-04T17:35:20.967" Title="Bracket Algebra, Straightening Algorithm" Tags="|matlab|algorithms|c++|computational-geometry|maple|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18842" PostTypeId="1" AcceptedAnswerId="18867" CreationDate="2015-02-04T20:03:26.433" Score="2" ViewCount="484" Body="&lt;p&gt;When I use PETSc to solve my linear systems, I always use the subroutine&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PetscErrorCode  KSPSetOperators(KSP ksp,Mat Amat,Mat Pmat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;Amat&lt;/code&gt; refers to the matrix that defines my linear system and &lt;code&gt;Pmat&lt;/code&gt; is the Matrix used in constructing the preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure in which scenarios it would be advisable to use another matrix than $A$ or &lt;code&gt;Amat&lt;/code&gt; to construct the preconditioner, i.e. to use a preconditioner from another (probably related) linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;Taking a look at the PETSc Manual, one possible occasion in which the matrices &lt;code&gt;Amat&lt;/code&gt; and &lt;code&gt;Pmat&lt;/code&gt; differ is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;when a preconditioning matrix is obtained from a lower order method&#xA;  than that employed to form the linear system matrix.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="11874" LastEditorUserId="11874" LastEditDate="2015-02-06T20:22:54.427" LastActivityDate="2015-02-07T13:13:42.517" Title="Construct a preconditioner for the linear system $Ax = b$ from a different matrix" Tags="|linear-solver|petsc|preconditioning|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18843" PostTypeId="2" ParentId="18837" CreationDate="2015-02-04T22:04:18.717" Score="1" Body="&lt;p&gt;Disclaimer: I'm going to talk in generalities about AD packages, not PyAutoDiff.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Finite Differences&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;Pros:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Easy to implement&lt;/li&gt;&#xA;&lt;li&gt;Useful for testing whether analytical gradient routines or AD give the right result&lt;/li&gt;&#xA;&lt;li&gt;If you can reimplement your function to take complex number inputs, there's a trick that will give you nearly exact gradients&lt;/li&gt;&#xA;&lt;li&gt;Non-intrusive&lt;/li&gt;&#xA;&lt;li&gt;Good enough accuracy in many cases&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Cons:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can be noisy; ill-conditioned functions can lead to inaccurate finite difference approximations&lt;/li&gt;&#xA;&lt;li&gt;For quasi-Newton methods that use Hessian updates derived from gradients (e.g., BFGS-type methods), noise in gradient is propagated into Hessian.&lt;/li&gt;&#xA;&lt;li&gt;Expensive: for a function of $n$ variables, requires $n$ function evaluations&lt;/li&gt;&#xA;&lt;li&gt;There's a bit of an art to choosing the magnitude of the difference used in the denominator of the quotient&lt;/li&gt;&#xA;&lt;li&gt;Even if sufficiently accurate, the noise can slow convergence of optimization methods.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Automatic Differentiation&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;Pros:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Returns nearly exact gradients&lt;/li&gt;&#xA;&lt;li&gt;Reverse mode is cheap (&quot;3-5x&quot; a function evaluation in theory, but in practice, it's an implementation-dependent multiple bounded above by a constant)&lt;/li&gt;&#xA;&lt;li&gt;Can analyze very complicated functions&lt;/li&gt;&#xA;&lt;li&gt;Avoids potentially time-consuming derivations of gradient functions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Cons:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Forward mode is expensive; scales with # of input variables&lt;/li&gt;&#xA;&lt;li&gt;Heinous to implement yourself; you should definitely rely on a library&lt;/li&gt;&#xA;&lt;li&gt;The method is intrusive -- it requires source-code access&lt;/li&gt;&#xA;&lt;li&gt;Source-to-source translation methods can analyze source code automatically, but can return ludicrously obfuscated code; more to the point, these methods frequently cannot differentiate the code they generate, so you're usually limited to derivatives of order 1.&lt;/li&gt;&#xA;&lt;li&gt;Operator overloading approaches require rewriting your code to use the overloaded operators; this process can be time-consuming&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-02-04T22:04:18.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18844" PostTypeId="2" ParentId="17595" CreationDate="2015-02-04T22:48:31.033" Score="2" Body="&lt;p&gt;The frequency domain solution is a steady-state solution to the transient equation. From your plot of displacement as a function of time, it is clear that the solution is not close to steady-state. You need to solve the transient system for a much longer time span-- until the amplitude no longer changes from cycle to cycle.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-02-04T22:48:31.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18845" PostTypeId="1" CreationDate="2015-02-04T23:11:45.527" Score="4" ViewCount="91" Body="&lt;p&gt;I am trying to integrate a 2nd order ODE with a singularity at close to the initial condition.&#xA;Why do I get large residuals when I plug-in the result of my integration back into the ODE?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is Newcomb's Euler-Lagrange equation from the field of plasma physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d}{dr}(f \frac{d\xi}{dr}) - g \xi = 0$$&#xA;or as a set of first order ODE's:&#xA;$$y_0 = \xi $$&#xA;$$y_1 = f \xi' $$&#xA;$$y_0' = \frac{y_1}{f} $$&#xA;$$y_1' = y_0 g $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ and $g$ are complicated expressions of magnetic fields and pressure gradients.&#xA;However, I even have trouble for simple cases, such as:&#xA;$$f=r$$&#xA;$$g=-1 + r + \frac{1}{r} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ODE is always singular at $r=0$&#xA;From Fobenius expansion the solution close to $r=0$ for this case should be $\xi \propto r^1 \approx 0$ and $\xi' \propto r^0 \approx 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code below I set up the problem and integrate with scipy.integrate.ode using the lsoda solver. &#xA;I plug the result back into the 2nd order ODE form:&#xA;$$f \xi'' + f' \xi' -g \xi $$&#xA;I have to use numpy.gradient to calculate $\xi''$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get residuals on the edge order of order of $10^2$. Why are they so large?&#xA;If I use constants for $f$ and $g$ I get residuals on the order of my step size.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.integrate import ode&#xA;&#xA;# Setup ODE system&#xA;def f(r):&#xA;   return r&#xA;&#xA;def g(r):&#xA;    return -1 + r + 1./r&#xA;&#xA;def der(r, y):&#xA;    y_der = np.zeros(2)&#xA;    y_der[0] = y[1]/f(r)&#xA;    y_der[1] = g(r)*y[0]&#xA;    return y_der&#xA;&#xA;#Integrate&#xA;integrator = ode(der)&#xA;integrator.set_integrator('lsoda') &#xA;r_init = 1E-3&#xA;init = [r_init, 1.]&#xA;integrator.set_initial_value(init, t=r_init)&#xA;r = np.linspace(r_init, 1., 100)&#xA;results = np.zeros((2, r.size))&#xA;results[:,0] = init&#xA;for i, position in enumerate(r[1:]):&#xA;    integrator.integrate(position)&#xA;    results[:, i+1] = integrator.y&#xA;&#xA;# Print residual&#xA;dr = r[1] - r[0]&#xA;print(f(r)*np.gradient(results[1]/f(r))/dr +&#xA;      np.gradient(f(r))/dr*results[1]/f(r) - g(r)*results[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output I get looks like the array below. There are large residuals at the edge.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([  5.70419912e+02,  -1.59592700e+02,  -9.00242498e-01,&#xA;        -1.50959245e-01,  -4.59057150e-02,  ...,&#xA;        -1.13773852e-03,  -1.11566938e-03,  -1.07162899e-03,&#xA;        -1.79645207e+00])      &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14391" LastEditorUserId="14391" LastEditDate="2015-02-05T18:10:31.463" LastActivityDate="2015-02-05T18:10:31.463" Title="Large residual when integrating 2nd order ode close to singularity with SciPy ode / ODEPACK" Tags="|ode|scipy|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="18846" PostTypeId="1" AcceptedAnswerId="18849" CreationDate="2015-02-05T02:17:18.530" Score="8" ViewCount="2360" Body="&lt;p&gt;It's a fairly well known trick to avoid division in calculating square-roots to &lt;a href=&quot;https://math.stackexchange.com/questions/550177/using-newtons-method-calculate-frac1-sqrta-without-division&quot;&gt;apply Newton's method to finding $1/\sqrt{x}$&lt;/a&gt;, and probably better known, using Newton's method to find &lt;a href=&quot;http://www.math.niu.edu/~rusin/known-math/98/recip_newt&quot; rel=&quot;nofollow noreferrer&quot;&gt;reciprocals without division&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In rescuing a StackOverflow thread &lt;a href=&quot;https://stackoverflow.com/questions/7463486/seeding-the-newton-iteration-for-cube-root-efficiently&quot;&gt;Seeding the Newton iteration for cube root efficiently&lt;/a&gt; from link rot, the thought came to me that a division-free iteration for cube roots should also be possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we were to solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x^{-3} = a^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then $x = a^{-2/3}$ and $\sqrt[3]{a} = ax$.  The Newton iteration for the above equation is simply:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_{n+1} = x_n - \frac{ x_n^{-3} - a^2 }{-3x_n^{-4}} &#xA;  = \frac{4}{3}x_n - \frac{1}{3}a^2 x_n^4 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again we avoid division operations, at least if the fractional constants are pre-evaluated for FP multiplications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So something of the sort is possible, but I did not find a specific discussion of such methods in my (admittedly shallow) search of the Web.  More to the point, I suspect that a clever person has already discovered a better idea and that one of you treasured colleagues has seen it or thought it through.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-02-05T06:15:43.113" Title="Newton iteration for cube root without division" Tags="|algorithms|numerical-analysis|special-functions|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18847" PostTypeId="2" ParentId="10176" CreationDate="2015-02-05T02:53:50.203" Score="1" Body="&lt;p&gt;The second algorithm does look like a Jacobi iteration. In that case, you should try using a relaxation coefficient of less than one (2/3 is the typically prescribed value according to wikipedia's entry on the subject).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to go with Jacobi iteration, scheduled relaxation may help a little. (Yang, Xiang; Mittal, Rajat (June 27, 2014). &quot;Acceleration of the Jacobi iterative method by factors exceeding 100 using scheduled relaxation&quot;. Journal of Computational Physics. doi:10.1016/j.jcp.2014.06.010)&#xA;Essentially, it involves using multiple dampening coffecients with corresponding prescribed numbers of iterations to form a larger cycle.&#xA;The paper gives several out of the box relaxation schedules, so it should be easy to get running quickly with a minimal amount of overhead... personally, I would go with Conjugate Gradient. It seems to be faster for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tested out the SRJ method (using a clunky csc_matrix based implementation) to solve a finite difference discretization of poisson's equation with periodic boundaries on systems as large as 512x512 (2D) and 64x64x64 (3D). SRJ seems to converge with about 1/5 as many total iterations as the typically suggested singular over-relaxation coefficient of 2/3 requires...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, conjugate gradient got far better convergence properties, even unconditioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Geometric Multigrid and Incomplete Cholesky preconditioned CG are also definitely good choices, particularly for problems in higher dimensions, and seem to give the best performance of the iterative linear solver's I've seen so far (although I'm still working on comparing with lean algebraic and combinatorial multigrid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are simply solving laplace's equation with spatially constant coefficients, fourier transform based methods (via fftw) are blazingly fast.&lt;/p&gt;&#xA;" OwnerUserId="14484" LastActivityDate="2015-02-05T02:53:50.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18848" PostTypeId="2" ParentId="17544" CreationDate="2015-02-05T04:52:38.113" Score="0" Body="&lt;p&gt;I would suggest moving to Fortran 90 and using an implied do loop inside the write statement:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program test3&#xA;    real, parameter :: dx=0.01&#xA;    real :: x&#xA;    integer :: k&#xA;    pi=4.0*atan(1.0)&#xA;    open(unit=11,file='sinkx.dat',status='replace',form='formatted')&#xA;    x=0.0&#xA;    do while(x &amp;lt; 2.0*pi)&#xA;        write(unit=11,fmt='(X,F9.3,10(X,E12.3))') &amp;amp;&#xA;            x, (SIN(REAL(k)*x),k=1,10,4)&#xA;        x=x+dx&#xA;    end do&#xA;    close(unit=11)&#xA;end program test3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: Note that this would also work if we replaced the 10 in &lt;code&gt;fmt='(X,F9.3,10(X,E12.3))'&lt;/code&gt; with any number greater or equal to 3. We could also calculate the number during runtime:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program test3&#xA;    ...&#xA;    integer, parameter :: k_max=10&#xA;    character(len=64) :: my_format&#xA;    write(my_format,'(A,I0,A)') &quot;(X,F9.3,&quot;, 1+(k_max-1)/4, &quot;(X,E12.3))&quot;&#xA;    ...&#xA;    do while(x &amp;lt; 2.0*pi)&#xA;        write(unit=11,fmt=my_format) &amp;amp;&#xA;            x, (SIN(REAL(k)*x),k=1,k_max,4)&#xA;        x=x+dx&#xA;    end do&#xA;    ...&#xA;end program test3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11689" LastEditorUserId="11689" LastEditDate="2015-02-05T05:04:36.137" LastActivityDate="2015-02-05T05:04:36.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18849" PostTypeId="2" ParentId="18846" CreationDate="2015-02-05T05:21:32.080" Score="8" Body="&lt;p&gt;Cube roots are not nearly as important as square roots (e.g., for normalizing vectors), so that might be why they are discussed much less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you apply Newton's method to $x^\alpha-\beta$, you get the iteration&#xA;$$ (1-\alpha^{-1})x+\frac\beta\alpha x^{1-\alpha}, $$&#xA;so you just need to pick the equation so that $\alpha$ is negative integer, it's not just cube and square roots, this works for other rational powers as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your scheme only needs 7 iterations to converge to double precision on the interval $a\in[\frac12,1]$ starting from constant initial guess $1$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/RvSNs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting thing to consider is what existing libraries implement:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MPFR does cube roots using integer cube roots (&lt;a href=&quot;http://www.mpfr.org/algo.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.mpfr.org/algo.html&lt;/a&gt;, around page 36)&lt;/li&gt;&#xA;&lt;li&gt;Boost (&lt;a href=&quot;http://www.boost.org/doc/libs/1_57_0/libs/math/doc/html/math_toolkit/powers/cbrt.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.boost.org/doc/libs/1_57_0/libs/math/doc/html/math_toolkit/powers/cbrt.html&lt;/a&gt;, see source code &lt;code&gt;math/special_functions/cbrt.hpp&lt;/code&gt;) uses Halley iteration with division:&#xA;$$ x_{n+1} = x_n \frac{2b+x_n^3}{b+2x_n^3}. $$&#xA;I tried it, and it seems to be slightly (10-25%, depending on which unsafe floating-point optimizations I enable) faster than division-free Newton's method, when running on my laptop. Given that division is much slower than multiplication/addition (by $\sim20$, see &lt;a href=&quot;https://software.intel.com/sites/landingpage/IntrinsicsGuide/&quot; rel=&quot;noreferrer&quot;&gt;https://software.intel.com/sites/landingpage/IntrinsicsGuide/&lt;/a&gt; and look for latencies of &lt;code&gt;div{s,p}{s,d}&lt;/code&gt;), there must be some other process that allows it to run faster.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; Another way to generate a good guess is to start with a polynomial approximation to $a^{-2/3}$ before doing Newton iterations. Two-term Chebyshev series reduces number of iterations to 4, three-term requires 3 iterations, and 6-term requires two iterations. In my testing, three-term Chebyshev series followed by 3 iterations was about $\frac13$ faster than just Halley. I haven't tested all possible combinations yet, it seems the fastest so far is 6-term Chebyshev series followed by 1 Halley iteration for $x^3-a$; I also only tested the interval $[\frac12,1]$, to which every floating point number can be reduced by separating out the mantissa.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-02-05T06:09:13.843" LastActivityDate="2015-02-05T06:09:13.843" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18850" PostTypeId="1" AcceptedAnswerId="18865" CreationDate="2015-02-05T07:58:57.270" Score="7" ViewCount="1862" Body="&lt;p&gt;So I have a symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and I would like to solve the optimization problem,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hspace{2.5mm}\text{Minimize}\;\; \|A-S\|_2$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\hspace{-5mm}\text{Subject to}\;\; S\geq0.$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is given and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is the variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can put this straight into my solver (SCSSolver in Convex.jl) but it's too slow for my matrix size (n=500).  I'm pretty sure it's the processor and not a lack of RAM, since it taxes my processor to 100% but there is no read/write occurring on my disk while the process is running.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried setting it up as a Schur complement problem but that didn't help.  Since &lt;span class=&quot;math-container&quot;&gt;$A-S$&lt;/span&gt; is symmetric I'm really just looking at the largest magnitude eigenvalue, I understand that the power-iteration algorithm is quite efficient, could I somehow incorporate that?  What is the standard procedure for speeding this up?  Is there a way to make it much faster by restricting to a subset of the positive semidefinite cone?  I'm willing to take a hit on the accuracy if necessary.&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="6849" LastEditDate="2019-12-14T17:32:31.483" LastActivityDate="2019-12-15T00:19:00.020" Title="Nearest positive semidefinite matrix to a symmetric matrix in the spectral norm" Tags="|optimization|convex-optimization|semidefinite-programming|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18851" PostTypeId="2" ParentId="18850" CreationDate="2015-02-05T10:41:01.567" Score="2" Body="&lt;p&gt;Since $A$ is symmetric, you can diagonalize it with $A=XDX^t$, and $S=XWX^t$, to get the equivalent formulation&#xA;$$ \text{minimize}\quad\|D-W\|_2, \qquad \text{s.to }W\geq0. $$&#xA;So for any positive eigenvalue $d_i$ of $A$, you can use $W_{ii}=d_i$ to subtract it off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that the answer might be the absolute value $|\lambda_-|$ of the largest negative eigenvalue of $A$, with $S$ equal to the same matrix as $A$ but with negative eigenvalues zeroed out, and with positive eigenvalues of $A-S$ reduced to be at most as large as $|\lambda_-|$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-02-05T11:06:56.967" LastActivityDate="2015-02-05T11:06:56.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18852" PostTypeId="1" AcceptedAnswerId="18853" CreationDate="2015-02-05T13:05:02.013" Score="10" ViewCount="309" Body="&lt;p&gt;Are there finite element method setups that provide error estimates in the $W^{1,\infty}$ norm (i.e., bounds on $\|u'_h - u'\|_\infty$)? Which families of elements can be used for implementing them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://mathoverflow.net/questions/195284&quot;&gt;Crossposted&lt;/a&gt; from MathOverflow, where it encountered little interest, but probably here I can find more people with a FEM background.)&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2015-02-06T00:30:11.960" Title="Finite elements $W^{1,\infty}$ error estimates" Tags="|finite-element|error-estimation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18853" PostTypeId="2" ParentId="18852" CreationDate="2015-02-05T18:42:52.637" Score="14" Body="&lt;p&gt;Chapter 8 of Brenner and Scott's &lt;a href=&quot;https://books.google.com/books?isbn=0387759336&quot;&gt;Mathematical Theory of Finite Element Methods&lt;/a&gt; is devoted to this subject. In particular, theorem 8.1.11 and the corollary give you that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \|u - u_h\|_{W^1_\infty} \le C h^{k - 1}\|u\|_{W^k_\infty}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for linear elliptic problems with sufficiently smooth coefficients, provided that the finite element space satisfies some other inequalities. They leave it as an exercise (how typical) to verify that this applies to the usual Lagrange, Hermite and Argyris elements.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-02-05T18:42:52.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18854" PostTypeId="1" AcceptedAnswerId="18858" CreationDate="2015-02-06T00:17:09.253" Score="0" ViewCount="255" Body="&lt;p&gt;Basically, I failed to trying to implement individual global MINLP solvers (alphaBB, ANTIGONE, BARON, Couenne, LindoAPI, and SCIP) in C++/Xcode/Mac system. On the other hand, I realize that GAMS already integrate a lot of solvers. So I am thinking of using C++/Xcode/Mac system to construct models and let GAMS solve that. What would be the standard practice in this respect? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="7891" LastActivityDate="2015-02-06T08:11:57.283" Title="Mac/Xcode/C++ connect with GAMS" Tags="|software|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18855" PostTypeId="1" CreationDate="2015-02-06T00:21:01.857" Score="4" ViewCount="315" Body="&lt;p&gt;How can we determine the numerical dispersion relation of a Spectral Element Method which leads to coupled systems of algebraic equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What approaches to do analysis of dispersion relations is available?&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="1804" LastEditDate="2015-02-10T08:13:46.437" LastActivityDate="2015-02-10T08:13:46.437" Title="How to calculate numerical dispersion relations for Spectral Elements?" Tags="|advection|spectral-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18856" PostTypeId="2" ParentId="18855" CreationDate="2015-02-06T00:21:01.857" Score="3" Body="&lt;p&gt;The key steps are to consider the advection equation&#xA;$u_t + au_x = 0$&#xA;where $a=\omega/k$ is the advection speed. Exact solutions to this equation is of the form $u(x,t) = f(x-at)$, where $f(y)$ is an arbitrary function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, discretize using a standard Galerkin method we derive the weak form&#xA;$\int_\Omega v u_t dx + \int_\Omega a u_x = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that the SEM solution is of the form&#xA;$u(x,t) = \sum_j u_j N_j(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and selecting the test functions $v=N_i(x)$, we can derive the algebraic system of equations&#xA;$M \frac{d}{dt} {\bf u} + a S {\bf u} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&#xA;$M_{ij} = \int_\Omega N_i N_j dx, \quad S_{ij} = \Omega N_i N_j' dx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the solution ansatz &#xA;$u(x,t) = A \exp( i (k x - \omega t) )$&#xA;we can insert this in the solution vector to derive a generalised eigenvalue problem of the form&#xA;$-i \omega M {\bf u} + a S {\bf u} = 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can introduce a dimensionless variable&#xA;$-i \Omega M {\bf u} + S {\bf u} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Omega = \frac{\omega h}{a}$, $h$ is the element size and $a$ is the advection speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By solving for the eigenpairs $(v,\lambda)$ of the generalised eigenvalue problem $Av=Bv\lambda$, we can obtain estimates of the angular frequency $\omega$ which can be used to characterise the numerical dispersion. The $\lambda\approx \Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab this can be implemented as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;B = 1i*M;&#xA;    A = S;&#xA;    [V,D] = eig(A,B);&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;where D contains the eigenvalues of the system and V the eigenvectors. The numerical value contained in D that matches the analytical the closest is assumed to be the approximation to the physical eigenmode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it is noted that Both M and S is constructed to be periodic.&lt;/p&gt;&#xA;" OwnerUserId="304" LastEditorUserId="304" LastEditDate="2015-02-07T20:31:33.300" LastActivityDate="2015-02-07T20:31:33.300" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18857" PostTypeId="2" ParentId="18852" CreationDate="2015-02-06T00:30:11.960" Score="3" Body="&lt;p&gt;If you want to go beyond the standard Laplace equation, Alan Demlow (at Texas A&amp;amp;M University) has derived estimates.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-06T00:30:11.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18858" PostTypeId="2" ParentId="18854" CreationDate="2015-02-06T08:11:57.283" Score="1" Body="&lt;p&gt;Standard practice tends to be one of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;formulate the model in GAMS and solve it in GAMS&lt;/li&gt;&#xA;&lt;li&gt;formulate the model in C++ and solve it in C++&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In either case, you can read data from a text file and use it to populate your model instead of hard-coding inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GAMS does have a C++ API. At one point, I've used it, and based on that experience, I wouldn't recommend it, simply because mixed-language programming tends to be challenging to maintain. Nevertheless, if you wanted to generate data in C++, and then use the GAMS API to import that data into GAMS, you could do that as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the solvers you cite are not available from C++. For instance, BARON interfaces with GAMS and MATLAB, but does not have a callable C++ interface. ANTIGONE also has a GAMS interface, but no publicly available C++ interface. (Perhaps you have a version directly from the authors?) alphaBB is an algorithm for solving MINLPs, but I don't know of software called alphaBB that solves MINLPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have any experience using Xcode as an IDE for software development, but at minimum, you should be able to get Couenne and SCIP to work on a Mac by compiling them and linking your software appropriately. If you are having trouble with this step, I'd recommend decoupling data generation and MINLP model formulation. Generate the data using a C++ program, and then read the data in GAMS and formulate your MINLP in GAMS, then call GAMS to solve your model.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-02-06T08:11:57.283" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18859" PostTypeId="1" CreationDate="2015-02-06T10:48:23.027" Score="1" ViewCount="232" Body="&lt;p&gt;I am using Arpack to compute the eigenvalues of the problem $\lambda Mx = Ax$ with reverse shift method with complex shift. $A$ and $M$ are real, $M$ is symmetric. Then, I use znaupd e zneupd. I use superLU to solve the system. Everything seems to be ok. I wrote a function that takes the eigenvalues and the eigenvectors computed with Arpack and put them in the original problem: $\lambda Mx -Ax = \text{residual}$. The norm of the residual is supposed to be almost zero or very small. Anyway, I obtain an error of $10^{-3}$ that is quite high! Do you have any idea where the problem can come from? I hope it is quite clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nora&lt;/p&gt;&#xA;" OwnerUserId="14489" LastEditorUserId="713" LastEditDate="2015-02-06T12:51:55.380" LastActivityDate="2015-02-06T12:51:55.380" Title="Compute eigenvalues with Arpack" Tags="|eigenvalues|arpack|" AnswerCount="0" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="18860" PostTypeId="1" CreationDate="2015-02-06T11:31:13.757" Score="2" ViewCount="155" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/L4uuN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results found are as under &lt;img src=&quot;https://i.stack.imgur.com/ZKKbK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;how do we find the exact solution ref The finite element method using matlab by Kwon and Bang.Page no 280-281 Example no 8.10.1. &lt;/p&gt;&#xA;" OwnerUserId="14490" LastActivityDate="2015-02-06T19:17:26.390" Title="find the exact solution ref The finite element method using matlab by Kwon and Bang" Tags="|matlab|eigenvalues|" AnswerCount="1" CommentCount="3" ClosedDate="2015-02-08T17:07:32.427" ContentLicense="CC BY-SA 3.0" />
  <row Id="18861" PostTypeId="1" CreationDate="2015-02-06T14:17:50.287" Score="2" ViewCount="1124" Body="&lt;p&gt;Density has units of $kg/m^3$. In simulations often scaling is done to deal with non-dimensional values, so the physical density is converted to the non-dimensional density so that it can be used in simulations. How is this done in 2D simulations? One may need to assign a value of mass to certain area.&lt;/p&gt;&#xA;" OwnerUserId="9721" LastActivityDate="2015-02-14T03:10:14.843" Title="How to interpret/convert density in 2D simulations?" Tags="|fluid-dynamics|simulation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18862" PostTypeId="2" ParentId="17592" CreationDate="2015-02-06T15:47:08.270" Score="2" Body="&lt;p&gt;Since your problem only concerns one variable and is unconstrained in this variable (yes?), you should consider to use the &lt;em&gt;Brent&lt;/em&gt; algorithm. You can find an implementation of the &lt;em&gt;Brent&lt;/em&gt; algorithm in &lt;a href=&quot;http://commons.apache.org/proper/commons-math/&quot; rel=&quot;nofollow&quot;&gt;Apache Commons Math&lt;/a&gt;, see API documentation &lt;a href=&quot;http://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/optim/univariate/BrentOptimizer.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you encounter problems with more variables and/or nonlinear constraints, you could also consider to use the &lt;a href=&quot;http://en.wikipedia.org/wiki/COBYLA&quot; rel=&quot;nofollow&quot;&gt;COBYLA&lt;/a&gt; algorithm, developed by Michael Powell. I have ported the algorithm from FORTRAN 77 to Java, see &lt;a href=&quot;https://github.com/cureos/jcobyla&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Powell has also developed more efficient algorithms for nonlinear objective functions without constraints, with bound constraints and with linear constraints for two or more variables. There is a Java implementation of his algorithm for bound constraints, &lt;a href=&quot;http://en.wikipedia.org/wiki/BOBYQA&quot; rel=&quot;nofollow&quot;&gt;BOBYQA&lt;/a&gt;, included in Apache Commons Math, see API documentation &lt;a href=&quot;http://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/optimization/direct/BOBYQAOptimizer.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3301" LastActivityDate="2015-02-06T15:47:08.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18863" PostTypeId="1" CreationDate="2015-02-06T19:04:34.977" Score="0" ViewCount="158" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;TASK:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make a simple animation of an object shot from a catapult. Object is moving through a medium. The problem must be solved in MAPLE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;NARROWING OF THE PROBLEM:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2D problem &lt;/li&gt;&#xA;&lt;li&gt;consider atmospheric density to be constant &lt;/li&gt;&#xA;&lt;li&gt;for calculations use Newton's term of the aerodynamics resistance&lt;/li&gt;&#xA;&lt;li&gt;animation: simple point following a trajectory &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;MY PROGRESS:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fa:=-1/2 * Cd * S * rho * V^2;&lt;/strong&gt; //Newton's term of the aerodynamics resistance - absolute value of the force&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fax:=Fa * cos(alpha); Fay:=Fa * sin(alpha)&lt;/strong&gt;; //Components of the vector of the aerodynamics force&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fax:=subs(cos(alpha)=Vx/V,Fax); Fay:=subs(sin(alpha)=Vy/V,Fay);&lt;/strong&gt; //Substitution of the goniometric functions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Vx:=diff(x(t),t); Vy:=diff(y(t),t); V:=sqrt(Vx^2+Vy^2);&lt;/strong&gt; // V = [Vx(t), Vy(t)]=[d * x(t)/dt, d * y(t)/dt]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ax:=diff(x(t),t,t)=Fax/m; Ay:=diff(y(t),t,t)=Fay/m-g;&lt;/strong&gt; //Aerodynamics deceleration in x and y direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;g:=9.81; v:=500.0; Cd:=1.05; d:=1; L:=1; Rho:=2500; S:=evalf(Pi * (d/2)^2); m:=S * L * Rho; rho:=1.22;&lt;/strong&gt; // &lt;strong&gt;g&lt;/strong&gt; = gravitational acceleration, &lt;strong&gt;v&lt;/strong&gt; = projectile initial velocity, &lt;strong&gt;Cd&lt;/strong&gt; = coefficient of the projectile's aerodynamic resistance, &lt;strong&gt;d&lt;/strong&gt; = projectile's diameter, &lt;strong&gt;Rho&lt;/strong&gt; = projectile's mass density, &lt;strong&gt;S&lt;/strong&gt; = projectile's cross section, &lt;strong&gt;m&lt;/strong&gt; = mass of the projectile, &lt;strong&gt;rho&lt;/strong&gt; = density of the atmosphere&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ini:=x(0)=0,y(0)=0,D(x)(0)=v * cos(alpha),D(y)(0)=v * sin(alpha); alpha:=40;&lt;/strong&gt; //Initial conditions. alpha = elevation angle&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I guess, here should come some nice procedure. The procedure should probably contain some sort of cycle that would solve the ODE and kick out colums of Xs and Ys, that would be used for plotting of the trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the animation itself I have even less of an idea.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So if anyone have any ideas of how should I proceed, please, share them. More explicit the better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks V.&lt;/p&gt;&#xA;" OwnerUserId="14493" LastActivityDate="2015-03-10T02:32:05.420" Title="Solving differential equations in MAPLE (ballistic curve problem)" Tags="|computational-physics|maple|" AnswerCount="1" CommentCount="3" ClosedDate="2017-05-14T13:14:28.627" ContentLicense="CC BY-SA 3.0" />
  <row Id="18864" PostTypeId="2" ParentId="18860" CreationDate="2015-02-06T19:17:26.390" Score="1" Body="&lt;p&gt;I don't have Kwon and Bang's book, so I can't say what it says near by this table, but presumably the &quot;exact&quot; solution comes from 1-D or quasi-2-D beam theory. If you search around, you should be able to find either a reference on the internet or in Kwon and Bang's book near the description of this problem. Presumably if you refine this from 4 to many more elements, you will get closer to the &quot;exact&quot; solution given here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;FWIW, this is a bit of an odd problem description since a beam of unit length and 1 by 1 cross-section is a actually a cube. I usually think of beams as being long and slender. Beam theory tends to make the same assumption. It's possible that the &quot;exact&quot; solution is simply the eigenmodes of the Laplacian on the unit cube. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think we need more information to help you.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-02-06T19:17:26.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18865" PostTypeId="2" ParentId="18850" CreationDate="2015-02-07T05:04:59.680" Score="6" Body="&lt;p&gt;There's no need to use the Schur complement here because &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; are already symmetric. The conventional formulation of this problem as an SDP is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min t \quad\text{ subject to}\\&#xA;A-S+tI \succeq 0 \\&#xA;A-S - tI \preceq 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;results in an SDP of the form (I'm specific here because there are so many different &quot;standard forms&quot; for SDP)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min c^{T}x\\&#xA;F_{0}+x_{1}F_{1}+...+x_{m}F_{m} \succeq 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x=[t\;\; S_{1,1} \;\; S_{1,2} \;\; S_{1,3} \; \ldots \; S_{n,n}]^{T}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;has &lt;span class=&quot;math-container&quot;&gt;$1+n(n-1)/2$&lt;/span&gt; elements, and the matrices &lt;span class=&quot;math-container&quot;&gt;$F_{i}$&lt;/span&gt; are of sizes &lt;span class=&quot;math-container&quot;&gt;$2n \times 2n$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your problems with &lt;span class=&quot;math-container&quot;&gt;$n=500$&lt;/span&gt;, this means that you have about 125,000 variables &lt;span class=&quot;math-container&quot;&gt;$x_{j}$&lt;/span&gt;, and matrices of size &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt;.  Primal-dual interior-point methods (as implemented in SDPA, SDPT3, SeDuMi, CSDP, etc.) would require the solution of a system of 125,000 equations in 125,000 variables in each iteration, and this system would typically be fully dense.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not something you can do on a typical desktop machine but would be within reach on a more powerful server with enough RAM (more than 128 gigabytes.)  Solution times would be fairly long (10+ hours)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;splitting cone solver&quot; that Julia implements appears to me to be an implementation of a first-order ADMM method such as the one described in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://doi.org/10.1007/s10957-016-0892-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;B. O'Donoghue, E. Chu, N. Parikh, and S. Boyd.  Operator Splitting for Conic Optimization via Homogeneous Self-Dual Embedding&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with ADMM for SDP is that it doesn't yield very accurate solutions (you'll typically get two or three digits of accuracy at best) and the method requires lots of tuning to achieve good results.  It's not nearly as robust (in the sense that it can consistently solve every problem that you throw at it) as the primal-dual interior-point methods.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't vouch for the particular implementation of this method in Julia since I've never worked with it.  It might be that some other first-order code for SDP could do a better job than SCSSolver.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is probably some work in the convex optimization literature on this kind of spectral norm minimization that avoids even formulating an SDP, but I'm no expert on that topic.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are you sure that you really need the best approximation in the matrix 2-norm sense rather than in the Frobenius norm?    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note added much later:  It turns out that the approach that computes this projection in the Frobenius norm also works equally well to compute the projection in the spectral norm.  First, perform an orthogonal diagonalization of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; as  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A=\sum_{i=1}^{n} \lambda_{i} u^{(i)}u^{(i)^{T}} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$S=\sum_{i=1}^{n} \lambda_{i}^{+} u^{(i)}u^{(i)^{T}} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I wrote this answer, I had known that this worked for the Frobenius norm projection, but not that it also works for the 2-norm projection. For non-symmetric &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, or if there are additional constraints, this simple formula isn't applicable and the SDP formulation might still be useful.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2019-12-15T00:19:00.020" LastActivityDate="2019-12-15T00:19:00.020" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18867" PostTypeId="2" ParentId="18842" CreationDate="2015-02-07T13:13:42.517" Score="3" Body="&lt;p&gt;Since nobody is stepping up to answer this question, let me point to a paper that contains an example of why one would want to do this: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications.html#publications/reviewed-34&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/publications.html#publications/reviewed-34&lt;/a&gt; (publication 34, Kronbichler et al.). In fact, there are multiple examples in this paper: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;We precondition $\begin{pmatrix}A &amp;amp;B \\ B^T &amp;amp; 0\end{pmatrix}$ by a block triangular matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We precondition $A$ by one V-cycle based not on $A$ but based on an approximation $\tilde A$ that is simpler and emptier.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We approximate the exact $S^{-1}$ in the block preconditioner by the application of a mass matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Complex applications quite frequently play with these things. You just don't ever see them if you just solve simple, scalar problems :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-07T13:13:42.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18870" PostTypeId="2" ParentId="18863" CreationDate="2015-02-08T01:56:40.870" Score="1" Body="&lt;p&gt;Maple has its own function for solving ODEs: &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=dsolve&quot; rel=&quot;nofollow&quot;&gt;dsolve&lt;/a&gt;, which can also solve ODEs numerically (&lt;a href=&quot;http://www.maplesoft.com/support/help/maple/view.aspx?path=dsolve%2fnumeric&quot; rel=&quot;nofollow&quot;&gt;dsolve/numeric&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-02-08T01:56:40.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18871" PostTypeId="2" ParentId="18861" CreationDate="2015-02-08T08:08:14.803" Score="0" Body="&lt;p&gt;In some cases one can integrate properties like the density across the thickness of the 2D area.  The 2D sheet does not have to have a uniform thickness, but it can.  For example, say I wanted to calculate the mass of a thin sheet, but the material is not homogeneous.  Its density is $\rho (x,y,z)$.  Say I also have a function, $f$, that will calculate the mass of a sheet, but $f$ only takes the mass per unit area as an argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I have a thin sheet, I might want to assume the density varies linearly across the thickness, in the $z$-direction.  This allows me to assume the density is the average of the density at the top and the bottom of the sheet, or is the density at the middle of the sheet.  Then the density of thin sheet becomes $\rho (x,y,z_{mid}) \times t(x,y)$, where $t(x,y)$ is the thickness of the sheet and $z_{mid}$ is the coordinate of the middle of the sheet.  I would define a density function $\sigma (x,y) = \rho (x,y,z_{mid}) \times t(x,y)$ which is the mass per unit area of the sheet.  Defining $\sigma$ does not change anything, but it helps me visualize what the terms mean and keep my units straight.  $\sigma(x,y)$ is what I would use in my function $f$.  If the sheet was too thick to assume the density varies linearly, then I would use some other numerical method to integrate across the thickness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this method depends on being able to integrate, at least in principle, across the thickness.  If I had an equation in the $x,y,z$-coordinates, I would want to integrate each expression in the equation across the thickness of the sheet.  The result would be an equation in just the $x,y$-coordinates.  &lt;/p&gt;&#xA;" OwnerUserId="11950" LastActivityDate="2015-02-08T08:08:14.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18872" PostTypeId="1" AcceptedAnswerId="18873" CreationDate="2015-02-08T11:21:02.300" Score="1" ViewCount="176" Body="&lt;p&gt;I have to solve numerically the following initial value problem, which physical origin is discussed in &lt;a href=&quot;http://www3.nd.edu/~powers/ame.90931/taylor.blast.wave.I.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; by G.I. Taylor (1941). Here $\eta$ is the independent variable and ranges beetween $0$ and $1$, $\gamma$ is fixed and $=1.4$ in my simulation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f'=\dfrac{f[-3\eta + (3+\gamma/2)\phi - 2\gamma \dfrac{\phi ^2}{\eta}]}{(\eta-\phi)^2-f/\psi},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\phi'=\dfrac{\frac{1}{\gamma} f'}{\psi(\eta-\phi)}-\dfrac{3}{2}\dfrac{\phi}{\eta-\phi},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi ' = \dfrac{\phi ' + 2 \frac{\phi}{\eta}}{\eta-\phi}\psi.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the equations are in normal form. The initial conditions are: $$\psi (1) = \dfrac{\gamma + 1}{\gamma -1},\quad f(1)=\dfrac{2\gamma}{\gamma + 1},\quad \phi (1)=\dfrac{2}{\gamma + 1}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result I'm expecting is more or less this, where density=$\psi$, velocity=$\phi$, pressure=$f$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PCZIm.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, $\phi$ becomes soon linear, with a slope that can be seen to be $\frac{1}{\gamma}$, while $\psi$ and $f$ respectively approach the costant values $0$ and (according to Taylor) $0.436$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integration scheme I'm using right now is the midpoint method: $$y_{n+1}=y_n + hF(x_n+\frac{h}{2},y_n+\frac{h}{2}F(x_n,y_n)).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything seems to work quite well until $\eta \approx 0.5$, $\phi'$ seems to approach $0.714=1/\gamma$ (however $f$ at that point is about $0.427$, which is a little lower than its limit and this should not occur). About at that point, $\phi'$ starts again to rise and for $\eta \to 0$ $\phi',\phi,\psi'$ and $\psi$ reach astronomical values. This is a plot of $\phi$ vs. $\eta$; here the ihtegration was done with a step $h=0.0001$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/I4bCC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the main problem lies in the approximation errors due to the little numbers that enter the ratios I have to calculate in order to compute $f',\phi ',\psi '$ (that are computed as in the equations above in my program). If it can be of any relevance, I'm writing the program in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you suggest any improvement in the algorithm (or maybe a more robust one) and/or the possible problems which may arise in this computation?  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I have tried the substitution $\xi = \phi - \eta/\gamma$ as suggested by Kirill and effectively the system seems to be more stable. However I'm still getting an explosion near $0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/N89WP.png&quot; alt=&quot;enter image description here&quot;&gt; &lt;/p&gt;&#xA;" OwnerUserId="6988" LastEditorUserId="6988" LastEditDate="2015-02-09T14:30:35.250" LastActivityDate="2015-03-14T23:20:30.443" Title="Numerical integration algorithm for this set of ODEs" Tags="|algorithms|numerical-analysis|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18873" PostTypeId="2" ParentId="18872" CreationDate="2015-02-08T12:29:30.617" Score="1" Body="&lt;p&gt;It is entirely possible, although I cannot tell by looking at it, that your system becomes stiff near $\eta=0$. I say this because Mathematica's NDSolve integrates this up to $\eta=3.4\times 10^{-9}$, then complains about a singularity or stiffness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The midpoint method is not A-stable, so that may be the source of inaccuracy: it is unsuitable for stiff problems. The ODE has some kind of singularity near $0$, so maybe the solution blows up because it hits the singularity due to inaccuracy before the method's instability becomes apparent. What do you get if you use a larger time step, like $0.01$ or $0.001$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try an A-stable method, like the trapezoid rule:&#xA;$$ y_{n+1} = y_n + \frac{h}{2}(F(x_n,y_n) + F(x_{n+1}, y_{n+1})). $$&#xA;You'd need to solve the implicit system of nonlinear equations here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach, since you know the asymptotic behaviour of the ODE solution, is to write the unknown functions as perturbations around the known asymptotic behaviour (e.g., $\frac1\gamma\eta+\eta^2\hat\phi(\eta))$, etc., as in a Taylor series), which might make the resulting ODE in $\hat\phi$ easier to solve by (maybe) avoiding the numerical errors in the denominator.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-02-08T12:29:30.617" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18875" PostTypeId="1" CreationDate="2015-02-08T20:19:31.980" Score="2" ViewCount="237" Body="&lt;p&gt;As you know, a generic Multiobjective optimization problem can be stated as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$min{\space}F(\bf{X})=[f_1(x),...,f_n(x)]$&lt;br&gt;&#xA;$h_k(x)=0{\space\space\space} k=1,...,n_e$&lt;br&gt;&#xA;$g_i(x)\leq0{\space\space\space} i=1,...,n$&lt;br&gt;&#xA;where $\bf{X}=[x_1, x_2, ... ,x_j]$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;definitions : &#xA;&lt;strong&gt;Objective Space&lt;/strong&gt; is a vector space including objective functions,i.e.$[f_1(x),...,f_n(x)]$  , of the Multiobjective Optimization problem as its dimensions. It is different from &lt;strong&gt;solution space&lt;/strong&gt;, which is a vector space with decision variables,i.e.$[x_1, x_2, ... ,x_j]$, of the Multiobjective Optimization problem as the dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is obvious that no one can plot feasible solution space when number of decision variables are more than three, i.e., $j&amp;gt;3$. Also, It is not possible to plot feasible objective space when number of objectives are more than three, i.e., $n&amp;gt;3$.&lt;br&gt;&#xA;I want to pull your attention to the case that we have 5 decision variables so we cannot plot the solution space, and we have three objective functions. Having three objective functions enables us to plot feasible objective space.  Objective space for a MO problem including three objective functions of $f_1(.)$ , $f_2(.)$ and $f_(3)$ is shown in the figure:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/oSOJ7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;br&gt;&#xA;where $\mu_1,\mu_2,\mu_3$ are three objective functions of the Multiobjective Optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Now my question is:&lt;/em&gt;&lt;br&gt;&#xA; How to plot feasible objective space of a Generic Multiobjective Optimization problem?&lt;br&gt;&#xA;For example, imagine the problem bellow with the given constraints and tell me how can I obtain the feasible objective space similar to the one in the figure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_1(X)= norm(x)^2$&lt;br&gt;&#xA;$f_2(X)= 3x_1+2x_2 - x_3/3 + 0.01(x_4 - x_5)^3$ &#xA;$f_3(X)= x_1^2 + 3x_2^2 + 0.2(x_3 - x_5)^3 + log(x_4^2 + x_1^2 + x_2^2 + 1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subject to:&lt;br&gt;&#xA;$h_1(X)  = x_1 + 2x_2 - x_3 - 0.5x_4 + x_5 - 2$&lt;br&gt;&#xA;$h_2(X)  = 4x_1 - 2x_2 + 0.8x_3 + 0.6x_4 + 0.5x_5^2$&lt;br&gt;&#xA;$g_1(X)= norm(x)^2 - 10$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that, I don't expect the solution of the given problem. Please give me some applicable insights about obtaining the graphing of feasible objective space. &lt;/p&gt;&#xA;" OwnerUserId="14366" LastEditorUserId="14366" LastEditDate="2015-02-09T11:06:08.007" LastActivityDate="2015-06-09T13:04:35.537" Title="How to plot feasible design space of a Generic Multiobjective Optimization problem?" Tags="|constrained-optimization|plotting|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18876" PostTypeId="2" ParentId="18875" CreationDate="2015-02-08T20:38:39.583" Score="1" Body="&lt;p&gt;You can't. Our mind is not equipped to visualize things in spaces with more than 3 dimensions. The best you can do is to visualize &lt;em&gt;projections&lt;/em&gt; onto lower-dimensional spaces, or to find analoga (e.g., &quot;the solution space is a polyhedron in 7 dimensions; if it were in 3 dimensions, it would look like this: ...&quot;).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-08T20:38:39.583" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="18877" PostTypeId="2" ParentId="18872" CreationDate="2015-02-08T20:43:11.883" Score="2" Body="&lt;p&gt;Instead of trying to shoehorn this into an ODE solver you write yourself (that probably uses a simplistic time step length selection method, low order, etc), use one of the existing, advanced and very accurate solvers for ODEs that you can find in any software package -- e.g., &lt;code&gt;dsolve&lt;/code&gt; in Maple, &lt;code&gt;NDSolve&lt;/code&gt; in Mathematica, or &lt;code&gt;ode45&lt;/code&gt; in Matlab. If necessary, play with the parameters these functions have and that you can glean from their respective help pages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point being that these are fantastic methods that give you high accuracy you could never get from a method you implement yourself in a finite amount of time. There's no point wasting your time reinventing something others have already done.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-08T20:43:11.883" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18878" PostTypeId="1" CreationDate="2015-02-09T00:17:38.433" Score="0" ViewCount="60" Body="&lt;p&gt;I have a little computational geometry project I'm struggling with for a non-commercial &quot;art&quot; installation.  It is driving me crazy and I'd happily pay for an implementable algorithm/solution (apologies if Stack Exchange doesn't allow this - I looked but didn't find any restriction on offering a cash bounty).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a starting location P1 on the surface of the earth (latitude phi1, longitude L1), compute the exit point P2 (latitude phi2, longitude L2) of a ray that departs from the starting point at local azimuthal angle alpha1 (from true north) with downward altitude angle beta1.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In less formal terms, imagine sitting on a rotating stool. Turn the stool from true north by some angle alpha1.  Point your arm straight out.  Now drop your arm by angle beta1.  Imagine a ray that extends from your pointed finger into the ground. Imagine this ray continuing straight through the earth.  What point P2 does this ray emerge from the earth?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notes and Suggestions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) As a first approach, suggest using a spherical earth model.  Here are some useful resources, &lt;a href=&quot;http://www.movable-type.co.uk/scripts/latlong.html&quot; rel=&quot;nofollow&quot;&gt;http://www.movable-type.co.uk/scripts/latlong.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) The &quot;direct Vincenty method&quot;, &lt;a href=&quot;http://en.wikipedia.org/wiki/Vincenty%27s_formulae&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Vincenty%27s_formulae&lt;/a&gt;, is potentially useful.  It provides an iterative approach to the related problem of determining the azimuthal angles when point P2 is known.  The method uses an oblate spheroid model of the earth (same as WGS84)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Determine how much accuracy is improved by 2 versus 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Determine if incorporating height above sea level of P1 or P2 has significant impact on answer.&lt;/p&gt;&#xA;" OwnerUserId="14507" LastEditorUserId="14507" LastEditDate="2015-02-09T00:26:51.610" LastActivityDate="2015-02-09T01:53:20.647" Title="Exit point of a ray shot through the earth from another location on the earth?" Tags="|computational-geometry|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18879" PostTypeId="2" ParentId="18878" CreationDate="2015-02-09T01:53:20.647" Score="1" Body="&lt;p&gt;In spherical coordinates, this is not much harder than solving a quadratic equation. Let the coordinates be $(r,\theta,\varphi)$, (radial, longitude, latitude). If the starting point is $\mathbf{R}=R\mathbf{e}_r$, then after rotating by $\alpha$, the observer is pointing in the direction of the unit vector&#xA;$$ \mathbf{e}_\alpha = \mathbf{e}_\varphi \cos\alpha + \mathbf{e}_\theta\sin\alpha, $$&#xA;and after rotating by $\beta$, the observer is pointing in the direction of&#xA;$$ \mathbf{e}_\beta = \mathbf{e}_\alpha\cos\beta - \mathbf{e}_r\sin\beta. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So on the line passing through $\mathbf{R}$ with direction $\mathbf{e}_\beta$, if $t$ is a parameter, the point where the line crosses the Earth's sphere in a second point is given by a quadratic equation in $t$:&#xA;$$ \|R\mathbf{e}_r + t\mathbf{e}_\beta\|^2 = R^2. $$&#xA;Expanding the vector norm, solving the equation to get the other root $t$, finding the point corresponding to that $t$, and converting that point to spherical coordinates gives you the coordinates of the other intersection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use an ellipsoid instead, the basis vectors and the equation of intersection will change. If you want to use terrain height (call it $h$), the change in angles will be on the order of $h/R$ where $R$ is Earth's radius, and $h/R$ will be quite tiny.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-02-09T01:53:20.647" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18882" PostTypeId="1" CreationDate="2015-02-09T11:55:36.740" Score="2" ViewCount="219" Body="&lt;p&gt;I have created a large positive definite (thus symmetric) matrix in Matlab. For sake of simplicity, let's assume that the matrix has the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K= \left[ \begin{array}{ccc} k_{11} &amp;amp; \cdots &amp;amp; k_{1n} \\ \vdots &amp;amp; \ddots &amp;amp; \vdots  \\&#xA;k_{n1} &amp;amp; \cdots &amp;amp; k_{nn}\end{array} \right]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to raise it to a power in an elementwise way, let's say to the power of $\lambda$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $K$ is known, while $\lambda$ is unknown and has to be estimated via an optimization routine, I save beforehand the matrix $K$. In the optimization routine I have to compute at each iteration the value of the matrix raised to the power of $\lambda$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is the following: Is there a way to perform this operation in a fast way, by exploiting the properties of the matrix $K$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have done until now:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1st approach: Compute $K^\lambda$ in a straightforward way, that is to say, using command &quot;K.^\lambda&quot; in matlab. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2nd approach: Use two for loops, where the first one iterates from i=1:n and the second one from j=i:n. This will create an upper diagonal matrix and due to symmetry, I can use triu command to get back the full matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, both approaches are rather time consuming, due to the large size of the matrix. Does anyone have any better idea about how I could perform this operation?&lt;/p&gt;&#xA;" OwnerUserId="14510" LastActivityDate="2015-06-09T20:29:18.867" Title="Raising elementwise a large symmetric, positive definite matrix to a power in Matlab" Tags="|matlab|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18884" PostTypeId="1" CreationDate="2015-02-09T14:22:49.823" Score="0" ViewCount="214" Body="&lt;p&gt;I am trying to minimize the edge lengths of a polygon while keeping the angles the same. I can achieve this geometrically (iteratively), however, I am looking for some related papers that can solve the problem algebraically.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/dtHJn.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="8069" LastActivityDate="2015-02-09T16:33:54.000" Title="Minimizing the edge length of a polygon preserving its angles" Tags="|computational-geometry|constrained-optimization|geometry|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18885" PostTypeId="2" ParentId="18882" CreationDate="2015-02-09T14:55:14.020" Score="1" Body="&lt;p&gt;I don't think you'll be able to do better than your second approach, but keep in mind that Matlab is column-major (like fortran):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for j=1:n&#xA;    K_lam(1:j,j) = K(1:j,j).^lambda&#xA;end&#xA;K_lam = K_lam+triu(K_lam,1)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="521" LastActivityDate="2015-02-09T14:55:14.020" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18886" PostTypeId="2" ParentId="18884" CreationDate="2015-02-09T15:42:31.140" Score="0" Body="&lt;p&gt;I think this doesn't solve your problem but may help you.&#xA;I used &lt;a href=&quot;http://www.mapshaper.org/&quot; rel=&quot;nofollow&quot;&gt;mapshaper&lt;/a&gt; to do reduce the number of points in the polygon.&#xA;You drag a polygon file into it (I used geojson) and you can change the resolution of the polygon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an &lt;a href=&quot;https://gist.github.com/yoavram/6f9d3d42861f0ff13665&quot; rel=&quot;nofollow&quot;&gt;example file&lt;/a&gt; that you can play with - a polygon from &lt;a href=&quot;http://nominatim.openstreetmap.org&quot; rel=&quot;nofollow&quot;&gt;Nominatim&lt;/a&gt;. I used mapshaper to reduce it from 1500KB to 15KB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mapshaper code is on &lt;a href=&quot;https://github.com/mbloch/mapshaper&quot; rel=&quot;nofollow&quot;&gt;Github&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14512" LastActivityDate="2015-02-09T15:42:31.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18887" PostTypeId="2" ParentId="18884" CreationDate="2015-02-09T15:43:15.193" Score="3" Body="&lt;p&gt;If you multiply the length of every side by a constant then all the angles will be preserved, this just produces a &lt;a href=&quot;http://en.wikipedia.org/wiki/Similarity_%28geometry%29&quot; rel=&quot;nofollow&quot;&gt;similar polygon&lt;/a&gt;. Multiply by a very small constant and you get a very small length, as small as you want, all the way down to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this easily if you have the $(x,y)$ coordinates of the vertices of the polygon with perimeter $L$ and area $A$. Simply multiply all the vertices by the same scaling factor, say $\alpha$, to get a new set of vertices and a polygon with perimeter length $\alpha L$ and area $\alpha^2 A$. You can avoid also translating the polygon with this rescaling by placing it with its center of area at the origin.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-02-09T16:33:54.000" LastActivityDate="2015-02-09T16:33:54.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18890" PostTypeId="1" CreationDate="2015-02-10T14:04:28.327" Score="0" ViewCount="1543" Body="&lt;p&gt;I have the following 6x6 matrix (taken from &lt;a href=&quot;http://books.google.de/books?id=x5Cd__PDf1kC&amp;amp;printsec=frontcover&amp;amp;hl=de&amp;amp;source=gbs_ge_summary_r&amp;amp;cad=0#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Books&lt;/a&gt; p. 129):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ond0J.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For background info:&lt;/strong&gt; All the entries depend on the momentum $k$. Getting the eigenvalues of this matrix for each $k$ corresponds to the electronic energies of electrons for 6 different &lt;em&gt;electronic bands&lt;/em&gt; and the eigenvectors are coefficients (lets call them $a$) to compute the electrons wave function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could also write this eigenvalue problem as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{j'}H^{LK}_{j,j'}a_{j'} = E_ja_j$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I know which j is which &lt;em&gt;electronic band&lt;/em&gt;. &lt;strong&gt;But:&lt;/strong&gt; when I introduce a numerical grid and solve this eigenvalue problem (using c++ and ComplexEigenSolver from the Eigen library) and I look at the very first eigenvalue, it does not correspond to $j=1$ (whose value is known). It rather corresponds to another j. This is due to that the eigensolver does not give the eigenvalues in a particular order. But I need this order to not confuse eigenvalues or eigenvectors of different $j$. Both eigenvalues and vectors can in principle be complex. Is there a way to get the eigenvalues and vectors so that their order corresponds to my matrix definition?&lt;/p&gt;&#xA;" OwnerUserId="7040" LastEditorUserId="7040" LastEditDate="2015-02-11T14:31:04.140" LastActivityDate="2015-02-11T22:48:59.933" Title="Order of eigenvalue problem using c++ Eigen library" Tags="|c++|eigenvalues|eigen|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="18891" PostTypeId="2" ParentId="15835" CreationDate="2015-02-10T19:42:38.370" Score="3" Body="&lt;p&gt;If you need to terminate the ODE solver at some point (P1==P2) you can use &lt;a href=&quot;https://github.com/hplgit/odespy&quot; rel=&quot;nofollow&quot;&gt;odespy&lt;/a&gt;. The &lt;code&gt;solve&lt;/code&gt; method of &lt;a href=&quot;https://github.com/hplgit/odespy/blob/master/odespy/solvers.py#L1002&quot; rel=&quot;nofollow&quot;&gt;the solvers&lt;/a&gt; accepts a &lt;code&gt;terminate&lt;/code&gt; function that accepts the state &lt;code&gt;u&lt;/code&gt;, the time &lt;code&gt;t&lt;/code&gt; and the integration step &lt;code&gt;step_no&lt;/code&gt; and returns a boolean. I used it today to stop the integration when any of the state elements is too low:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;u,t = solver.solve(linspace(0,100,1000), terminate=lambda u,t,step_no: (u &amp;lt; 1e-6).any())&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="14512" LastActivityDate="2015-02-10T19:42:38.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18892" PostTypeId="2" ParentId="17483" CreationDate="2015-02-11T01:50:59.260" Score="1" Body="&lt;p&gt;Interpolate on the coarser grid, e.g., by tensor product splines of sufficient smoothness. Then, evaluate the interpolant over the finer grid. Now you have two same-length solution vectors: one sampled from the low resolution model, the other -- actual data solved on the higher resolution. Find their difference in a given norm.&lt;/p&gt;&#xA;" OwnerUserId="10287" LastActivityDate="2015-02-11T01:50:59.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18895" PostTypeId="2" ParentId="16325" CreationDate="2015-02-11T10:32:35.423" Score="2" Body="&lt;p&gt;I ended up raising an exception in the &lt;code&gt;deriv()&lt;/code&gt; function (and catching it in the loop) because it turned out to be the only reliable way to end integration in every single case and for every ODE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(While I considered this option already in my original question, it seems there is no other way. If anybody presents a better option I will gladly mark her/his answer as accepted. &lt;strong&gt;Update:&lt;/strong&gt; I accepted Dmitry's answer.)&lt;/p&gt;&#xA;" OwnerUserId="11927" LastEditorUserId="11927" LastEditDate="2015-02-14T18:20:20.463" LastActivityDate="2015-02-14T18:20:20.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18896" PostTypeId="1" CreationDate="2015-02-11T10:45:32.563" Score="1" ViewCount="79" Body="&lt;p&gt;The radius of convergence of any power series can be found by simply using the root test, ratio test etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused as to how to find the radius of convergence for an analytic $f$ such as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(z)=\frac{4}{(z-1)(z+3)}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't imagine that I would have to find the power series representation of this, find the closed form, and then use one of the convergence tests. I am fairly certain that the radius of convergence would have to do with the singularities at $1$ and $-3$, however, I can't find a formula for the radius of convergence..&lt;/p&gt;&#xA;" OwnerUserId="14420" LastActivityDate="2015-02-11T10:45:32.563" Title="What is the Radius of Convergence for analytic functions?" Tags="|convergence|" AnswerCount="0" CommentCount="4" ClosedDate="2015-02-11T19:07:21.393" ContentLicense="CC BY-SA 3.0" />
  <row Id="18897" PostTypeId="2" ParentId="18890" CreationDate="2015-02-11T11:28:15.123" Score="4" Body="&lt;p&gt;This has nothing to do with Eigen: if you cannot, just by looking at a matrix, determine how eigenvalues should be labelled, you cannot expect Eigen to do it for you. Also, it is up to you to define  precisely, mathematically, how the labels should be assigned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Judging from your example plot, you seem to be assuming that labelled eigenvalues form smooth curves. So the obvious thing to try is extrapolation from previous grid points (on a one-dimensional grid): at the grid point $k$ extrapolate from two or three previous grid points where the next term is expected, &#xA;$$ (A_{k-3}, A_{k-2}, A_{k-1})\mapsto A^{extrap}_k, \qquad (B_{k-3},B_{k-2},B_{k-1})\mapsto B^{extrap}_k. $$&#xA;Then calculate your eigenvalues $\lambda_{1,2}$ and assign each to $A$ or $B$ depending on which it is closer to. For example, you can try assigning the labels to minimize the value of $$|\lambda_{a}-A_k^{extrap}|+|\lambda_{b}-B_k^{extrap}|.$$ This would work with your example graph, maybe even just linear extrapolation might work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also need to tweak extrapolation order, or try different kinds of extrapolation to see what works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; &lt;em&gt;(The question changed).&lt;/em&gt; So if you have your eigenvalues $E_j(k)$ as functions of $k$, basically the same approach might work. If you assume that $E_j(k)$ is a smooth function of $k$, for each $j$, then when assigning labels $j$ at the next value of $k$ you can extrapolate each past eigenvalue to $E_j(k)$, then assign labels based on minimizing something like sum of squared extrapolation errors,&#xA;$$ \sum_j \big| \lambda_{\sigma(j)} - E_j \big|^2, $$&#xA;where $\sigma$ is the permutation that assigns labels to unlabelled eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-02-11T22:48:59.933" LastActivityDate="2015-02-11T22:48:59.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18900" PostTypeId="1" AcceptedAnswerId="18901" CreationDate="2015-02-11T19:28:20.407" Score="4" ViewCount="102" Body="&lt;p&gt;Consider the function&#xA;$$f(\mathbf{x}) = \sum_{n=0}^{N} a_n \left( (\mathbf{b}-\mathbf{x})\cdot \nabla \right)^n \frac{1}{r}$$&#xA;where $r = |\mathbf{x}| = \sqrt{(x-x_0)^2 + (y-y_0)^2}$ and $a_n$ and $\mathbf{b}$ are well-known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any good way to evaluate this function numerically (for fairly large N)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I don't have a general expression for&#xA;$$\left( (\mathbf{b}-\mathbf{x})\cdot \nabla \right)^n \frac{1}{r}.$$&#xA;Is there a smarter way than just recursively do numerical derivatives of $1/r$? I am quite scared of this method since multiple derivatives could lead to numerical errors accumulating. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="14394" LastActivityDate="2015-02-11T21:20:30.833" Title="How to evaluate a series of derivatives?" Tags="|algorithms|finite-difference|python|performance|iterative-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18901" PostTypeId="2" ParentId="18900" CreationDate="2015-02-11T21:13:59.620" Score="7" Body="&lt;p&gt;You can convert $\mathbf{b}-\mathbf{x}$ into polar coordinates, and do the dot product in this system. This changes $((\mathbf{b}-\mathbf{x})\cdot\nabla)^n\frac{1}{r}$ to &#xA;$$\left((\mathbf{b}-\mathbf{x})_r\frac{\partial}{\partial r} + (\mathbf{b}-\mathbf{x})_{\theta}\frac{1}{r}\frac{\partial}{\partial \theta}\right)^n\frac{1}{r}$$&#xA;Here, I am using the subscripts $r$ and $\theta$ to indicate the $r$ and $\theta$ components, respectively, of the difference. You can ignore the second term of this entirely, as it involves derivatives with respect to $\theta$ and you only have a dependence on $r$, which further simplifies the term to&#xA;$$(\mathbf{b}-\mathbf{x})_r^n\frac{\partial^n}{\partial r^n}\frac{1}{r}$$&#xA;Now use the fact that &#xA;$$\frac{\partial^n}{\partial r^n} \frac{1}{r} = (-1)^n\frac{n!}{r^{n+1}}$$&#xA;You now have a nice analytical derivative that you can use in your summation. Your final equation will look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(\mathbf{x})=\sum_{n=0}^{N}(-1)^n a_n(\mathbf{b}-\mathbf{x})_r^n\frac{n!}{r^{n+1}}$$&lt;/p&gt;&#xA;" OwnerUserId="13159" LastEditorUserId="13159" LastEditDate="2015-02-11T21:20:30.833" LastActivityDate="2015-02-11T21:20:30.833" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18902" PostTypeId="2" ParentId="16350" CreationDate="2015-02-12T05:12:54.930" Score="3" Body="&lt;p&gt;Just like in&#xA;ordinary analysis of global error, you have a truncation error that&#xA;behaves asymptotically as&#xA;$$ \tau \sim h^2 \partial^4 u + h^2\epsilon^{-1}\partial^3u, $$&#xA;and the global error approximately solves the equation&#xA;$$ \nabla\cdot(a\nabla E) = -\tau. $$&#xA;Ordinarily (I remember only 1d $a=1$ explicitly), this leads to an estimate&#xA;like $\|E\|\sim Ch^2\|\partial^2u\|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When $u$ oscillates on a scale of $\epsilon$, its derivatives become&#xA;large, behaving as $\partial^ku=O(\epsilon^{-k})$. Therefore, keeping&#xA;$h$ constant and $\epsilon$ variable (and small), the truncation error&#xA;for a given $\epsilon$ behaves as&#xA;$$ \tau_\epsilon \sim \epsilon^{-4}\tau. $$&#xA;Now when solving the equation $\nabla\cdot(a\nabla&#xA;E_\epsilon)=-\tau_\epsilon$, the only question is to estimate the&#xA;magnitude of $E_\epsilon$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because $\tau_\epsilon$ oscillates rapidly, one might expect some&#xA;amount of &quot;smoothing&quot; by the elliptic equation. For this analysis I&#xA;found &lt;em&gt;Introduction to Perturbation Methods&lt;/em&gt; by Holmes, Chapter 5 very&#xA;helpful. The relevant result is as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In solving $\nabla\cdot(a\nabla u) = f$ we can write $u=u_0+\epsilon u_1+\cdots$ to&#xA;leading order, and then the solution is given by the equation&#xA;$$ \nabla_x\cdot\big(\langle a \partial_{y_i}w\rangle \cdot&#xA;\nabla_xu_0 + \langle a\rangle\partial_{x_i} u_0\big) =&#xA;\langle f\rangle, $$&#xA;(the expression in the brackets is a vector with components indexed by $i$), where $w$ is a vector of functions satisfying the equations&#xA;$$ \nabla_y\cdot(a \nabla_y w_k) = -\partial_{y_k}a, $$&#xA;and $\langle\cdots\rangle$ denotes averaging over the periodic&#xA;fast-scale variable:&#xA;$$ \langle q(x, y)\rangle = \int_0^1 q(x, y)\,dy, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to estimate global error, we need&#xA;$\langle\tau(x)\rangle$. The local truncation error $\tau(x)$ consists&#xA;of derivatives of the exact solution $v(x, y)$, and scales as&#xA;$\epsilon^{-4}$. If I can use the one-dimensional expression, it is&#xA;$$ \tau(x) \sim C_1 h^2\partial^4v + C_2h^2\epsilon^{-1}\partial^3v,&#xA;\qquad \partial = \epsilon^{-1}\partial_y + \partial_x. $$&#xA;So the leading term is $\epsilon^{-4}h^2(C_1v_{yyyy}+C_2 v_{yyy})$. &#xA;When this is averaged over the period $y$, periodicity causes the&#xA;result to vanish to first order:&#xA;$$ \epsilon^{-4}h^2\big( C_1 v_{yyy} + C_2 v_{yy})|_{y=0}^1. $$&#xA;But periodicity is only valid to first order, so $$v_{yyy}|_{y=0}^{1} \sim&#xA;\epsilon v_{xyyy}.$$ So the averaged local truncation error behaves&#xA;as&#xA;$$ \langle \tau\rangle \sim \epsilon^{-3}h^2(C_1 v_{xyyy}+C_2v_{xyy}), $$&#xA;and the magnitude of the global error scales as $\epsilon^{-3}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This slightly ad-hoc and crufty, and doesn't give a precise error estimate, just the scaling power for $\epsilon$. I don't know how to derive from this an estimate on the global error $E_\epsilon$ in terms of some norm of the exact solution. I understand that this is very&#xA;hand-wavy, but this is the most intuitive I could do, and the order doesn't depend on problem dimensions, so it&#xA;addresses your other question as well.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-02-12T05:52:58.297" LastActivityDate="2015-02-12T05:52:58.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18905" PostTypeId="1" AcceptedAnswerId="18906" CreationDate="2015-02-12T13:31:43.820" Score="0" ViewCount="112" Body="&lt;p&gt;I have the following definitions &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;long  = 1000;&#xA;  init  = 500;      &#xA;  N = init+long;        &#xA;  T = init+1:N-1;   &#xA;  T1    = init+2:N;   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then fill the vectors that I want according to this loop,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   for t=1:N+1&#xA;     Rk(t)=1+A(t)*alpha*kh(t).^(alpha-1)-delta;  &#xA;     Yi(t)=A(t)*(1-alpha)*kh(t).^alpha - A(t)*alpha*kh(t).^(alpha-1)+eta(t);&#xA;     Rap(t)=Rk(t).^(1-gamma)+(1-gamma)*Rk(t).^(-gamma)*theta(t)*Yi(t)-0.5*gamma*(1-gamma)*(theta(t)*Yi(t)).^2;  &#xA;   end   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, I compute some new vectors according to this loop &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for t=T1&#xA;           fi1(t)=(v(t).^(1-gamma))*(Rk(t))*(Yi(t)');  % Expectations 1 &#xA;           fi2(t)=((v(t).^(1-gamma))*(Yi(t)).^2);   &#xA;           fi3(t)=(((v(t).^(1-gamma))*Rap(t)).^(1/(1-gamma)));&#xA;       end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, from the last loop I aim into getting a vector of size T1, but instead I receive a 1x1500 vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My scope is essentially to take all the numbers in vectors, v, Rk, Yi, from exactly the region that T1 specifies (i.e 502:1500), so I should get the 1x999, and not the 1x1500 vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I suppose even if someone stars with an index 502 to count the loop, Matlab still understand this as 1, and loops until N, which is the end point of T1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody tell me how should amend the code in order to get what I described above. &lt;/p&gt;&#xA;" OwnerUserId="9826" LastActivityDate="2015-02-12T13:51:32.350" Title="For loops in matlab with a specific starting point" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18906" PostTypeId="2" ParentId="18905" CreationDate="2015-02-12T13:51:32.350" Score="3" Body="&lt;p&gt;&lt;code&gt;t&lt;/code&gt; starts at 502, but &lt;em&gt;all&lt;/em&gt; MATLAB arrays start at 1. So MATLAB pads &lt;code&gt;fi1&lt;/code&gt;, &lt;code&gt;fi2&lt;/code&gt;, and &lt;code&gt;fi2&lt;/code&gt;, with zeros from index 1 to 501 and then puts your data in starting at 502. You can either live with this or write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for t=T1&#xA;  fi1(t-(init+1))=(v(t).^(1-gamma))*(Rk(t))*(Yi(t)');&#xA;  ......&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-02-12T13:51:32.350" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18907" PostTypeId="2" ParentId="16325" CreationDate="2015-02-12T14:48:16.083" Score="8" Body="&lt;p&gt;The feature that you demand is called &lt;em&gt;event location&lt;/em&gt; in Matlab ODE solvers pack, or &lt;em&gt;rootfinding&lt;/em&gt; in &lt;a href=&quot;http://computation.llnl.gov/casc/sundials/main.html&quot; rel=&quot;noreferrer&quot; title=&quot;SUNDIALS&quot;&gt;SUNDIALS solvers suite&lt;/a&gt; terminology. Essentially this feature allows to stop integration exactly at the point where some vector function of free and dependent variables has a root. Namely, for system&#xA;$$&#xA;\frac{d\boldsymbol y}{dt} = f(t, \boldsymbol y),&#xA;    \quad \boldsymbol y \in \mathbb R^n,&#xA;$$&#xA;an ODE solver with the ability to do event location will be able to find when some function $ g(t, \boldsymbol y)$ crosses zero—and this process of zero-crossing is called an &lt;em&gt;event&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, &lt;code&gt;scipy.integrate&lt;/code&gt; lacks good ODE solver with the support of event location. You can use &lt;code&gt;solout&lt;/code&gt; feature of &lt;code&gt;dopri5&lt;/code&gt; solver, but you will overshoot in this case, because &lt;code&gt;solout&lt;/code&gt; is called only after step is done. In contrast, when ODE solver is able to do event location, it will work hard on finding the exact location of the root of $g(t, \boldsymbol y)$, so that the integration will be stopped much more precisely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also need this feature for my computations. I use the following package &lt;a href=&quot;https://github.com/bmcage/odes&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/bmcage/odes&lt;/a&gt;. This package is quite difficult to compile, because it depends on BLAS, LAPACK and SUNDIALS. However, interface (API) is quite good, and it will work much better than using &lt;code&gt;solout&lt;/code&gt; feature of &lt;code&gt;dopri5&lt;/code&gt; from &lt;code&gt;scipy.integrate&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3254" LastActivityDate="2015-02-12T14:48:16.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18908" PostTypeId="1" CreationDate="2015-02-12T15:58:59.977" Score="0" ViewCount="67" Body="&lt;p&gt;I have two time-dependent coupled equations. One of which is several orders of magnitude more computationally demanding than the other. I am trying to use machine learning to reproduce the behavior of the more expensive equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;equation 1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input: c(t), a(t)&#xA;output a(t+dt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;equation 2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input: a(t)&#xA;output: c(t+dt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So essentially I want to reconstruct the response of equation 2. Keep in mind that internally there are variables in equation 2 which retain 'memory' of the previous states. So the response depends on the history of input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice on where to start or what methods have been developed for this type of system? OR if there is a more appropriate place to post this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: some more details, this is a multiscale simulation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;equation 1 is a simple finite difference equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a(x,t+dt) = 2a(x,t) - a(x,t-dt) + \frac{dt^2}{dx^2} \left[ a(x+dx,t)-2a(x,t) + a(x-dx,t) \right] + dt^2 c(x,t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the second part at each x I have a time-dependent set U(t) to propogate to U(t+dt). This propagation depends on an input a(x,t) and produces c(x,t+dt) to be fed back into the first equation. The details of this part are a bit convoluted/involved, but the essential point is that I want to avoid explicitly storing or propagating U (very very very expensive e.g. 10,000+ cpu cores needed)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2:&#xA;A NARX network seems to be able to do almost what I want. However, I have a number of different 'examples' which I want the network to learn from. Maybe the only way I can do it is to stitch everything together into one big (input, output) set?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/nnet/ug/design-time-series-narx-feedback-neural-networks.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/nnet/ug/design-time-series-narx-feedback-neural-networks.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14534" LastEditorUserId="14534" LastEditDate="2015-02-23T20:58:12.670" LastActivityDate="2015-02-23T20:58:12.670" Title="approximation of nonlinear time-dependent system with history" Tags="|machine-learning|time-integration|approximation|support-vector-machines|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18909" PostTypeId="1" CreationDate="2015-02-12T17:47:17.593" Score="1" ViewCount="189" Body="&lt;p&gt;I am currently dealing with the following problem. I am given a matrix $A$ of order $n \times n$ where $n \leq 20.$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principal $(n-1) \times (n-1)$ matrix of $A$ is symmetric and contains only $\{0,1\}$ as its entries. The $n'$th row and column contains a scaled row sum of this principal sub-matrix. In particular $A$ is not symmetric but its eigenvalues are real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to determine whether the second largest eigenvalue $\lambda_2$ of $A$ is greater than $2.$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its perfectly ok for me if the procedure says this is not so (while in fact it is) as long as I do not get false positives. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I am simply computing the eigenvalues of $A$ and checking the condition but it takes a lot of time given that $A$ is not symmetric. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence my question is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there any better (faster) ways to determine if $\lambda_2 &amp;gt; 2?$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I suppose I could evaluate some determinants of the form $A+\epsilon+2I$ and compare signs, but this approach looks really scary to me.&lt;/p&gt;&#xA;" OwnerUserId="4582" LastEditorUserId="4582" LastEditDate="2015-02-12T17:53:33.320" LastActivityDate="2015-02-12T19:55:23.957" Title="Estimating the second largest eigenvalue" Tags="|linear-algebra|matrix|eigenvalues|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="18911" PostTypeId="1" AcceptedAnswerId="18912" CreationDate="2015-02-12T23:25:36.310" Score="1" ViewCount="94" Body="&lt;p&gt;My question concerns coordinate maps and non-equally spaced fourier transforms.&lt;/p&gt;&#xA;&lt;p&gt;I have dependent variables &lt;span class=&quot;math-container&quot;&gt;$(X(\xi),Y(\xi))$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\xi\in(0,2\pi)$&lt;/span&gt;. In general, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is assumed even and expanded as a Fourier series, ie&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y(\xi) = \sum_{k=-N}^{N} Y_k e^{-ik\xi},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is taken to tend towards infinity and the reality condition implies &lt;span class=&quot;math-container&quot;&gt;$Y_{-k} = Y_k$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, my governing equations are derived from a lagrangian. I won't go into the full details of it, since they are anscillary for the question, but it is illustrative to look at the potential energy term, V, which takes the form&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V=\int_0^{2\pi} Y(\xi)^2 \frac{dX(\xi)}{d\xi} \ d\xi,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$X-\xi$&lt;/span&gt; being the Hilbert transform of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; (because (X,Y) are real and imaginary parts of an analytic function in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{C}$&lt;/span&gt;). That is,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$X = \xi + \sum i \sigma_k Y_k e^{-ik\xi},$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt; being 1 for &lt;span class=&quot;math-container&quot;&gt;$i&amp;gt;0$&lt;/span&gt;, 0 when &lt;span class=&quot;math-container&quot;&gt;$i=0$&lt;/span&gt;, and -1 when &lt;span class=&quot;math-container&quot;&gt;$i&amp;lt;0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Substituting in the Fourier expansion, &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; can be written entirely in terms of the &lt;span class=&quot;math-container&quot;&gt;$Y_k$&lt;/span&gt;s. The Euler-Larange equations then returns a set of governing (algebraic) equations are found and a system of equations is solved for a given &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;. It turns out that for the solutions I desire, the series converges very slowly, and this has to do with the fact that &lt;span class=&quot;math-container&quot;&gt;$Y(\xi)$&lt;/span&gt; is very localized.&lt;/p&gt;&#xA;&lt;p&gt;After going through &lt;a href=&quot;http://www-personal.umich.edu/%7Ejpboyd/aaabook_9500may00.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boyd's book&lt;/a&gt; (in particular, see chapter 16), it seems like one way to deal with this is to map the independent variable into a different, non-uniformly spaced map. One example is to send &lt;span class=&quot;math-container&quot;&gt;$$\xi\to \theta-sin(\theta)$$&lt;/span&gt; (Boyd calls this a Kepler mapping), for &lt;span class=&quot;math-container&quot;&gt;$\theta \in(0,2\pi)$&lt;/span&gt;, which will tend to crowd more points towards the ends of the domain, which is where my functions are localized.&lt;/p&gt;&#xA;&lt;p&gt;This seems promising to me, and I would like to pursue it.&lt;/p&gt;&#xA;&lt;p&gt;To this end, I take a change of coordinates at the level of the integral &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, shown above, to find&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V = \int_0^{2\pi} Y(\theta)^2 \frac{d X(\theta)}{d \theta} \ d\theta$$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From here it is unclear to me how one takes a Fourier type expansion (needed to allow for the easy computation of the Hilbert transform) on this non-uniform grid.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated,&lt;/p&gt;&#xA;&lt;p&gt;Nick&lt;/p&gt;&#xA;" OwnerUserId="8021" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-02-13T02:36:55.870" Title="Coordinate transformations and analytic form of non-uniformly gridded fourier transform" Tags="|fourier-transform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18912" PostTypeId="2" ParentId="18911" CreationDate="2015-02-13T02:36:55.870" Score="1" Body="&lt;p&gt;Unless I'm missing something obvious, if you transform $Y=Y_\xi(\xi)$ to use $Y=Y_\theta(\theta) = Y_\xi(\theta-\sin\theta)$, then the function $w(\xi)=X(\xi)+iY(\xi)$, which, as you say, was analytic in $\xi$, would be analytic in $\theta$ also, with $w(\theta) = w(\xi=\theta-\sin\theta)$, so that whatever Fourier series $Y(\theta)$ has w.r.t. $\theta$, $X$ would have the same expression as before in terms of the new variable $\theta$ and the new Forier coefficients $Y_k^{(\theta)}$ (it's a generic expression, independent of the coordinates being used to write it down).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is assuming that the rest of your equations can be rewritten in terms of $\theta$ as well: I don't know a good way to relate two Fourier series obtained by a change of variable. But this is not the same as doing a discrete Fourier transform on a non-uniform grid, anyway, because you seem to be using a Galerkin-type method, and evaluating all the necessary integrals in closed form by hand. In fact, I don't quite see where in your question a discrete Fourier transform is being used.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-02-13T02:36:55.870" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18913" PostTypeId="1" CreationDate="2015-02-13T08:12:00.623" Score="-1" ViewCount="660" Body="&lt;p&gt;I am trying to solve ODEs  in matlab using ode15s. Instead of specifying ODEs in the format     &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  M * dC/dt = f(C,t)  where C is a function of x and t.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  M * dC/dt = J*C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since the vector C is very large I am expecting considerable reduction in computation time.  I am trying to figure out how to code the Dirichlet boundary condition C(x=0,t) = a. thanks&lt;/p&gt;&#xA;" OwnerUserId="14540" LastActivityDate="2015-02-19T05:19:29.830" Title="Dirichlet boundary condition" Tags="|matlab|ode|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18914" PostTypeId="1" AcceptedAnswerId="18927" CreationDate="2015-02-13T13:33:41.353" Score="6" ViewCount="497" Body="&lt;p&gt;I would like to compute the roots of a polynomial with exponentially small coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum_{n=0}^N a_n \frac{z^n}{\sqrt{n!}} \tag{$\ast$}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a_n$ are Normal random variables with mean $0$ and variance $1$.  For simplicity you can just say $a_n \equiv 1$ always.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the decay is large the coefficients are getting very small.  $\sqrt{n!} \approx (n/e)^{n/2}$ this is causing underflow on my computer.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q&lt;/strong&gt; Calling &lt;code&gt;np.roots&lt;/code&gt; on $\ast$ for $N=2000$ results errors because the coefficients are too small?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are some possible ways to handle numbers that are getting small this quickly, or can the power method simplify in some other way in this case?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;What I Learned&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Numpy's &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.roots.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;np.roots&lt;/code&gt;&lt;/a&gt; appproximates roots as the eigenvalues of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Companion_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;companion matrix&lt;/a&gt; of the polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left[ \begin{array}{cccl} &#xA;0 &amp;amp; 0 &amp;amp; \dots &amp;amp; 0 &amp;amp; -a_n \\&#xA;1 &amp;amp; 0 &amp;amp; \dots &amp;amp;0 &amp;amp; -a_{n-1} \\&#xA;0 &amp;amp; 1 &amp;amp;  \dots &amp;amp; 0  &amp;amp; -a_{n-2} \\&#xA;0 &amp;amp; \vdots &amp;amp; &amp;amp; \vdots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; \dots &amp;amp; 1 &amp;amp; -a_0&#xA;\end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In term, these eigenvalues are computed using something called &lt;a href=&quot;http://www.netlib.org/lapack/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAPACK&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without referring to a library, I could maybe implement the &lt;a href=&quot;https://en.wikipedia.org/wiki/Power_iteration&quot; rel=&quot;nofollow noreferrer&quot;&gt;power method&lt;/a&gt; on my own.  From initial random vector $b_0 \in S^N$, multiply by $A$ and rescale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ b_{k+1} = \frac{A b_k}{|| A b_k ||}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coefficients are getting small very quickly but the ratio of neighbor coefficients is not growing too fast $b_n = a_n/\sqrt{n!}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ b_n/b_{n+1} = \sqrt{n+1} $$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Really Simple Example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;What are the roots of $\epsilon x^2 + ax + b = 0$ ?  The quadratic formula returns two numbers approximately&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x = \frac{ -a + \sqrt{a^2 - 4b \epsilon} }{2\epsilon }&#xA;=  \frac{ -a + a\sqrt{1 - 4(b/a) \epsilon} }{2\epsilon }&#xA;\approx  -\frac{a}{\epsilon} , -\frac{b}{a}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ratio of the coefficients seems to matter a lot.  In our case, they are of the same order of magnitude $\frac{a}{\epsilon} , \frac{b}{a} \approx \sqrt{n}$ &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is some code I wrote incorporating the rescaling suggestion... not many lines.  Certainly it's enough to get the roots, but not sure if I trust it for finer statistics.  Convergence is king here, I think.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 2000&#xA;a = 0.5*(np.random.normal(0,1,N) + 1j*np.random.normal(0,1,N))&#xA;&#xA;fact = np.cumproduct(np.sqrt(np.arange(N)+1)/np.sqrt(N/np.exp(1)))&#xA;&#xA;w = np.roots(a/fact[::-1])&#xA;plt.plot(w.real, w.imag, '.')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EtwPn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Final Version&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The obvious (in hindsight) rescaling leads to much more accurate and convincing pictures, that I believe will hold up to further scrutiny.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 2000&#xA;a = 0.5*(np.random.normal(0,1,N) + 1j*np.random.normal(0,1,N))&#xA;&#xA;A = 1j*np.zeros((N,N))&#xA;A[:,-1] = a&#xA;&#xA;x = np.arange(N-1)&#xA;A[x+1, x] =np.sqrt(x+1)&#xA;&#xA;w = np.linalg.eigvals(A)&#xA;plt.plot(w.real, w.imag, '.')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/oRPeD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="4006" LastEditDate="2015-02-15T15:13:30.940" LastActivityDate="2015-02-15T15:13:30.940" Title="roots of polynomials with small coefficients" Tags="|numerical-analysis|eigenvalues|numpy|polynomials|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="18915" PostTypeId="2" ParentId="7149" CreationDate="2015-02-13T14:35:53.253" Score="8" Body="&lt;p&gt;I know this is an old question,  but the correct command to move a queued job from one queue to another in torque is &quot;qmove&quot; and not qalter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://qcd.phys.cmu.edu/QCDcluster/pbs/PBS_qmove.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;man page&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usage: &lt;code&gt;qmove destination job_identifier ...&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Description:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To move a job is to remove the job from the queue in which&#xA;  it resides and instantiate the job in another queue. The&#xA;  qmove command issues a Move Job batch request to the batch&#xA;  server that currently owns each job specified by&#xA;  job_identifier. A job in the running state cannot be moved by the user.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14541" LastEditorUserId="21201" LastEditDate="2017-07-16T09:49:44.580" LastActivityDate="2017-07-16T09:49:44.580" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18918" PostTypeId="1" CreationDate="2015-02-13T22:26:11.217" Score="4" ViewCount="562" Body="&lt;p&gt;I read a master thesis on a topic I'm interested too. This work concern the solution of the displacement equation of motion for a homogeneous, elastic, isotropic material:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho \ddot{\mathbf{u}} - \mathbf{F} = \mu \nabla^2 \mathbf{u} + (\lambda + \mu) \mathrm{grad} \; \mathrm{div} \; \mathbf{u}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathrm{u}$ is the displacement vector, $\lambda$ and $\mu$ are the Lamé constants (knowns with some uncertainty), and $\mathbf{F}$ an external force per unit volume (known exactly), and particularly it's steady state variant, the so called Navier-Cauchy equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more precise, the domain is not infinite and homogeneus: it's a disk with few different regions and the Lamé constants are separately constants over each sub-region; over one of these sub-region $\mu$ become very very small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author of the thesis uses a finite difference spatial discretization with polar coordinates and compute the coefficient matrix and the data vector of the linear system to be solved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author notice the system is vary badly conditioned and, analyzing the singular value decomposition of this matrix, by some features of the singular values spectrum, apparently conclude it's (or behave as) a discrete ill-posed problem, coming from the discretization of a continuous &lt;em&gt;inverse&lt;/em&gt; &lt;em&gt;ill-posed&lt;/em&gt; problem. So the author apply a Tickhonov regularization to get an apparently meaningful solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading that Navier-Cauchy equation can be &lt;em&gt;ill-posed&lt;/em&gt; or that is an &lt;em&gt;inverse&lt;/em&gt; problem surprised me a bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I know about the well posedness of elasticity problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;under the hypothesis of &quot;small displacements&quot;, and if the elastic tensor is such that the deformation energy is positive-definite, the solution of the elastic problem, if exists, is unique (Kirchhoff theorem);&lt;/li&gt;&#xA;&lt;li&gt;the existence of a solution &quot;is more complex to prove but has been proved in several scenarios useful for applications&quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't know nothing about the third requirement for well-posedness in the Hadamard sense, i.e. if the solution is continuously dependent on data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neglecting now this thesis, at this point I think I need to know something more about the well posedness of the linear eleasticity problem and the Navier-Cauchy equation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What is a good, authoritative, reference about the well posedness of elasticity problem (not too advanced please)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are the Navier-Cauchy equation well-posed? Are there cases where the problem can become ill-posed? How the well posedness is affected by the fact that Lamé constants are non-uniform, possibly near to zero, and/or not continuous?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the case the problem prove to be well-posed, but the linear still system ill-conditioned, what is a good way to handle this specific bad conditioning? A preconditioner for example? What a good one?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14545" LastEditorUserId="14545" LastEditDate="2015-02-14T22:20:37.320" LastActivityDate="2015-02-17T21:15:59.827" Title="Well-posedness of a linear elasticity problem and Navier-Cauchy equation" Tags="|linear-algebra|pde|" AnswerCount="2" CommentCount="12" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18919" PostTypeId="1" CreationDate="2015-02-13T22:36:52.387" Score="0" ViewCount="698" Body="&lt;p&gt;I am working on creating a program for simulations where three variables are parametrized, and we modify one parameter while keeping the other two constant. An example array looks like this when outputted by MatLab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    dim3arr(:,:,1) =&#xA;&#xA;0.8000    0.9000    1.0000    1.1000    1.2000&#xA;     0         0         0         0         0&#xA;0.5000    0.5000    0.5000    0.5000    0.5000&#xA;&#xA;&#xA;    dim3arr(:,:,2) =&#xA;&#xA;0.8000    0.8000    0.8000    0.8000    0.8000&#xA;     0   10.0000   20.0000   35.0000   60.0000&#xA;0.5000    0.5000    0.5000    0.5000    0.5000&#xA;&#xA;&#xA;    dim3arr(:,:,3) =&#xA;&#xA;0.8000    0.8000    0.8000    0.8000    0.8000&#xA;     0         0         0         0         0&#xA;0.5000    0.6000    0.8000    1.1000    1.5000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to figure out how to interpolate between points in the array. MatLab's documentation for interpolating on N-dimensional arrays seems to assume that the data is monotonically increasing. How would I get MatLab to interpolate the data?&lt;/p&gt;&#xA;" OwnerUserId="14546" LastActivityDate="2015-02-14T01:12:08.053" Title="Interpolating 3D Array non-monotonic data in MatLab" Tags="|matlab|interpolation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18920" PostTypeId="2" ParentId="18919" CreationDate="2015-02-14T00:32:15.350" Score="2" Body="&lt;p&gt;It's not the data that needs to be monotonic, it's the coordinates (your three variables) that must be monotonically increasing. As suggested by @nicoguaro in his comment, &lt;code&gt;interp3&lt;/code&gt; should work fine for this as long as you have things ordered properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you mean that your variables are non-monotonic then you will simply have to reorder things. Calling your variables &lt;code&gt;x&lt;/code&gt;, &lt;code&gt;y&lt;/code&gt;, and &lt;code&gt;z&lt;/code&gt; and assuming that they correspond to dimensions 1, 2, and 3 respectively of &lt;code&gt;dim3arr&lt;/code&gt; you could do this via:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[x, Ix]=sort(x);&#xA;[y, Iy]=sort(y);&#xA;[z, Iz]=sort(z);&#xA;dim3arr = dim3arr(Ix,:,:);&#xA;dim3arr = dim3arr(:,Iy,:);&#xA;dim3arr = dim3arr(:,:,Iz);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, just make sure &lt;code&gt;x&lt;/code&gt;, &lt;code&gt;y&lt;/code&gt;, &lt;code&gt;z&lt;/code&gt; are monotonic before using them to generate &lt;code&gt;dim3arr&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-02-14T01:12:08.053" LastActivityDate="2015-02-14T01:12:08.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18921" PostTypeId="2" ParentId="18861" CreationDate="2015-02-14T03:10:14.843" Score="1" Body="&lt;p&gt;When the Navier-Stokes equations are non-dimensionalized, one of the coefficients that comes out of the non-dimensionalization process is the Reynolds number. The Reynolds number parameter is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Re = \frac{\rho VL}{\nu}&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is important is not so much that you are using the same density as what you are trying to simulate, but that you are using the same Reynolds number.  This concept is called &lt;em&gt;similarity&lt;/em&gt;, i.e. two problems that have the same Reynolds numbers (even though the densities may be different) produce a similar flowfield. So, in most CFD codes you wouldn't specify the density as an input, but rather the Reynolds number.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You basically need to compute this for the simulation you are running.  $L$ here is some characteristic length of your object.  For an airplane this might be the wing span, for a river it might be the width of the river, for a submarine it is usually the length of the submarine.  $\nu$ is the kinematic viscosity.  $V$ is the freestream (unperturbed) velocity.&lt;/p&gt;&#xA;" OwnerUserId="4873" LastActivityDate="2015-02-14T03:10:14.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18923" PostTypeId="1" CreationDate="2015-02-14T14:59:06.473" Score="4" ViewCount="641" Body="&lt;p&gt;I want to set up a surface charge boundary condition for the simulation of semiconductor and electrolyte interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 2D-Poisson example and surface charge boundary condition are shown in following picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QZeIC.png&quot; alt=&quot;2D Poisson &amp;amp; surface charge boundary condition&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then I substitute the 2nd part of LHS with boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/9ZEm1.png&quot; alt=&quot;substitution&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final results are shown in figure 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/AvxM4.png&quot; alt=&quot;final result&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After I finished the substitution of boundary condition. The 2nd part of Poisson's equation became a constant value which is the surface charge density divided by y-direction meshing. This result let me feel a little bit strange. The influence of y-direction flux is now disappeared!? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this result correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is not, which kind if boundary condition should I apply for the surface charge simulation?&lt;/p&gt;&#xA;" OwnerUserId="11614" LastActivityDate="2015-02-20T19:35:37.143" Title="Surface charge boundary conditon for Poisson-Boltzmann solver" Tags="|simulation|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18925" PostTypeId="1" CreationDate="2015-02-14T16:17:40.287" Score="2" ViewCount="93" Body="&lt;p&gt;I am using FORTRAN to implement &lt;a href=&quot;http://en.wikipedia.org/wiki/Gillespie_algorithm&quot; rel=&quot;nofollow&quot;&gt;Gillespie's stochastic simulation algorithm&lt;/a&gt;. I would be running many simulations in parallel (both parallel instances with different seed and parallel functions); if I have to store every time interval then a lot of memory would be consumed. So I thought I can have some fixed number of time points where the data would be saved (Allocate space beforehand either dynamically or statically).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I decide the size of the time intervals? Should the intervals have a fixed size or it should be based on local dynamics(some regions would have more dynamics)? Is there any way to predict that based on the deterministic function without actually running a deterministic simulation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have this vague idea that the tau-leap formula can be used to predict time intervals where &lt;code&gt;n&lt;/code&gt; number of a major reaction would occur. But this still doesn't help me predict the number of steps beforehand. &lt;/p&gt;&#xA;" OwnerUserId="14551" LastEditorUserId="14551" LastEditDate="2015-02-14T21:36:43.377" LastActivityDate="2015-02-14T21:36:43.377" Title="How to fix time intervals to store data in a stochastic simulation (continous time markov chain)" Tags="|algorithms|fortran|stochastic|memory-management|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18926" PostTypeId="2" ParentId="14195" CreationDate="2015-02-14T17:12:07.390" Score="0" Body="&lt;p&gt;If ODE methods can contribute to optimization, is there a really simple example problem to show this ?&lt;br&gt;&#xA;A straw man: is there an ODE solver that does a reasonable job on&lt;br&gt;&#xA;$\qquad \dot{ x } = - \nabla f( x ) $&lt;br&gt;&#xA;or $\quad \ddot{ x } = \beta \dot{ x } - \alpha \nabla f( x ) \ \ $ as Christian Clason suggests&lt;br&gt;&#xA;for $f$ say the Rosenbrock function, in 2d or 10d ?&#xA;If that's silly, does anyone have a better straw man ?&lt;br&gt;&#xA;(Note &quot;reasonable&quot;, not &quot;competitive with state-of-the-art optimizers&quot;.&#xA;I imagine one needs decreasing step sizes / tolerance, and maybe a stiff solver.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, &quot;too big&quot; steps are much more problematic than &quot;too small&quot; &amp;mdash;&#xA;oscillations are messy.&lt;br&gt;&#xA;I'd have thought naively that control theory could help.&#xA;Numerical Recipes p. 915 describes&lt;br&gt;&#xA;&lt;a href=&quot;http://apps.nrbook.com/empanel/index.html?pg=915&quot; rel=&quot;nofollow&quot;&gt;PI adaptive stepsize control&lt;/a&gt;&#xA;for ODEs, but I don't know if this is used in practice.&lt;/p&gt;&#xA;" OwnerUserId="1841" LastActivityDate="2015-02-14T17:12:07.390" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18927" PostTypeId="2" ParentId="18914" CreationDate="2015-02-15T01:57:50.670" Score="7" Body="&lt;p&gt;Note that, if $D$ is invertible, the eigenvalues of $A$ and $DAD^{-1}$ are the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can avoid floating-point underflow when forming the matrix by scaling the companion matrix by a diagonal matrix $D$ such that $D_{ii} = 1/\sqrt{(n-i)!}$.  For the polynomial $x^4 + a x^3/\sqrt{1!} + b x^2/\sqrt{2!} + c x/\sqrt{3!} + d/\sqrt{4!}$, this gives a modified companion matrix of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left(\begin{array}{cccc}&#xA;  &amp;amp; &amp;amp; &amp;amp; -d\\&#xA;  1/\sqrt{4} &amp;amp; &amp;amp; &amp;amp; -c\\&#xA;  &amp;amp; 1/\sqrt{3} &amp;amp; &amp;amp; -b\\&#xA;  &amp;amp; &amp;amp; 1/\sqrt{2} &amp;amp; -a\\&#xA;\end{array}\right).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use a trick such as $p(x) \to x^n p(1/x)$ that inverts the roots of $p$ with respect to the circle in order to get $p$ into the form given above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know whether LAPACK reliably finds the eigenvalues of such matrices, but all of the entries here are quite reasonably scaled.&lt;/p&gt;&#xA;" OwnerUserId="3525" LastActivityDate="2015-02-15T01:57:50.670" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18930" PostTypeId="1" CreationDate="2015-02-15T22:38:55.650" Score="0" ViewCount="520" Body="&lt;p&gt;It seems that scipy.minimize can find the minimum of an unbounded function. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import numpy as np&#xA;&amp;gt;&amp;gt;&amp;gt; from scipy.optimize import minimize&#xA;&amp;gt;&amp;gt;&amp;gt; def fun(y):&#xA;...     return y&#xA;... &#xA;&amp;gt;&amp;gt;&amp;gt; res = minimize(fun, np.array([0.0]), method='SLSQP')&#xA;&amp;gt;&amp;gt;&amp;gt; res&#xA;  status: 0&#xA; success: True&#xA;    njev: 14&#xA;    nfev: 42&#xA;     fun: array([ -3.05175781e+08])&#xA;       x: array([ -3.05175781e+08])&#xA; message: 'Optimization terminated successfully.'&#xA;     jac: array([ 0.,  0.])&#xA;     nit: 14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To be clear, this function is unbounded below and does not have a minimum. Anyone know why scipy.minimize with SLSQP terminates successfully? Am I supposed to interpret -3e8 as negative infinity? This doesn't inspire much confidence. &lt;/p&gt;&#xA;" OwnerUserId="3652" LastActivityDate="2015-02-16T00:50:28.567" Title="Unconstrained minimization of unbounded function with SciPy" Tags="|optimization|scipy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18931" PostTypeId="2" ParentId="18930" CreationDate="2015-02-16T00:50:28.567" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;To be clear, this function is unbounded below and does not have a minimum.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That means it does not satisfy the preconditions of &lt;code&gt;minimize&lt;/code&gt;, assumptions that it makes about its input, so you should not pass it to &lt;code&gt;minimize&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Anyone know why scipy.minimize with SLSQP terminates successfully?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It uses some heuristics to determine what's going on and whether it's appropriate to terminate, and in this case of an invalid input the heuristics failed because the author of those heuristics did not take this input into account. All heuristics are, by nature, heuristic, and not guaranteed to be correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One commonly made assumption is that all interesting values of $x$ and $f(x)$ are of reasonably small magnitude (not huge, or infinite, as in this case). Not that this is ideal, but it holds (possibly after rescaling) for most interesting problems.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I supposed to interpret -3e8 as negative infinity?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. Numerical algorithms only aim to be approximately correct, and rarely offer any strong guarantees on the correctness of their outputs. All you can conclude is that $-3\times 10^8$ is the algorithm's best guess at the minimum value. In this case, the result is clearly wrong, but that is because you violated the assumptions the algorithm makes about its input functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either change your function so that it does have a minimum, or consider filing a bug report with scipy, so that the invalid input gets recognized and causes an explicit error.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This doesn't inspire much confidence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any introductory numerical analysis textbook discusses the issues involved in using floating-point arithmetic (which is by nature and by necessity approximate), so I would recommend reading some introductions to numerical analysis to understand the challenges involved here, before blaming the authors of scipy.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-02-16T00:50:28.567" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18933" PostTypeId="2" ParentId="18918" CreationDate="2015-02-16T13:31:03.897" Score="2" Body="&lt;p&gt;The constant $\mu$ is the resistance of the material to shear stresses. If this constant is zero in a region and you are considering the steady-state case, the system of PDE are singular there; i.e. they are no longer elliptic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it is not surprising that if you discretize the PDE using finite differences or finite elements, and define $\mu$ to be very  small, the matrix equations are ill-conditioned.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-02-16T13:31:03.897" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18935" PostTypeId="2" ParentId="18918" CreationDate="2015-02-17T21:15:59.827" Score="1" Body="&lt;p&gt;I think I found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\mu = 0$ the reverse elastic bond &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{E} = \frac{1}{2\mu} \mathbf{S} - \frac{\lambda}{2\mu} \frac{\mathrm{tr }\mathbf{S}}{3\lambda + 2\mu} \mathbf{I}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is not defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover the Kirchoff theorem about the uniqueness of the elastic response rely on the fact the strain energy density is positve-definite. For this to be true it is required&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mu &amp;gt; 0 \quad \text{and} \quad  3\lambda + 2\mu &amp;gt; 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if $\mu = 0$ we cannot prove the uniqueness of the solution.&lt;/p&gt;&#xA;" OwnerUserId="14545" LastActivityDate="2015-02-17T21:15:59.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18936" PostTypeId="1" AcceptedAnswerId="18939" CreationDate="2015-02-18T05:05:34.367" Score="0" ViewCount="1189" Body="&lt;p&gt;I am running some scientific (parallel) code and would like to obtain some performance profiling measurements. I want to obtain the &quot;efficiency&quot; of the code in terms of flops/s over theoretical (peak) performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran a simple matrix multiplication example (PAPI_flops.c which can be found on the web) and got the measurements using PAPI measurement counts. The HPC system (56 Gb/s interconnect) I used has this cpuinfo:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;processor   : 39&#xA;vendor_id   : GenuineIntel&#xA;cpu family  : 6&#xA;model       : 62&#xA;model name  : Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz&#xA;stepping    : 4&#xA;microcode   : 1064&#xA;cpu MHz     : 1200.000&#xA;cache size  : 25600 KB&#xA;physical id : 1&#xA;siblings    : 20&#xA;core id     : 12&#xA;cpu cores   : 10&#xA;apicid      : 57&#xA;initial apicid  : 57&#xA;fpu     : yes&#xA;fpu_exception   : yes&#xA;cpuid level : 13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Let A and B be 1000x1000 matrices with randomized floats. For a serial process, I get the following metrics after multiplying A by B:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Real_time:  3.340896&#xA;Proc_time:  3.344447&#xA;Total flpins:   2837232042&#xA;MFLOPS:     848.341187&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, from what I understand, the theoretical peak performance would be #cores x clock x FLOPs/cycle. I believe the above processor can do 4 FLOPs/cycle, so if I were to assume that I had a single-core 2.8 GHz processor, the theoretical performance would be about 11200 MFLOPS. Which would give me about 7% efficiency...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is an extremely low efficiency rate. 850 MFLOPS seems to be a pretty good and high performance rate to me. Am I doing something wrong here, or is this expected for something as simple as matrix multiplication?&lt;/p&gt;&#xA;" OwnerUserId="14576" OwnerDisplayName="Justin" LastActivityDate="2015-02-18T13:28:53.823" Title="Determining efficiency in MFLOPS/s of a parallel program" Tags="|parallel-computing|efficiency|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18937" PostTypeId="2" ParentId="18936" CreationDate="2015-02-18T08:17:28.233" Score="3" Body="&lt;p&gt;For dense matrices or order $n$, the theoretical number of FLOPS required for matrix product is $2n^3$. In your case, since you are multiplying two matrices of order 1000, the number of FLOPS should be, in MFLOPS, 2000. Therefore, the 848.3 MFLOPS performance of your code is less than 50% of the theoretical performance for matrix multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scaling $n$, you should see (or at least expect) better performances if you are using a good multiplication algorithm (a recursive, cache-aware algorithm fares much better than the ordinary algorithm relying on the matrix multiplication formula from linear algebra).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's worth noting here that FLOP counting is just a crude estimate of performance: when you use it to evaluate a program's efficiency, you are obviously ignoring a lot of overhead factors affecting program's execution. Therefore, FLOP counting can only capture one specific dimension of a program's efficiency.&lt;/p&gt;&#xA;" OwnerUserId="3533" OwnerDisplayName="Massimo Cafaro" LastActivityDate="2015-02-18T08:17:28.233" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18938" PostTypeId="2" ParentId="16178" CreationDate="2015-02-18T12:14:42.893" Score="0" Body="&lt;p&gt;I think you do not need to use &lt;code&gt;sort&lt;/code&gt; function and your &lt;code&gt;delta_s&lt;/code&gt; is too high, I would suggest that reduce the &lt;code&gt;delta_s&lt;/code&gt; to 0.000005 and remove all your sort functions and you will get right slope for damping.&lt;/p&gt;&#xA;" OwnerUserId="14575" LastEditorUserId="9667" LastEditDate="2015-02-18T19:50:04.773" LastActivityDate="2015-02-18T19:50:04.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18939" PostTypeId="2" ParentId="18936" CreationDate="2015-02-18T12:53:18.517" Score="3" Body="&lt;p&gt;The factor you miss is that when you do matrix-matrix multiplications, you have to reload data from memory. Since your matrices are big enough that they won't fit into the cache, you access main memory which is incredibly slow compared to a floating point operation. To give you an idea how slow: a floating point multiplication on a typical processor today takes 1.5ns, while loading a floating point number from main memory takes 60-100ns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the floating point performance of your chip does not matter for matrices of size 1000-by-1000. It is all about the memory bandwidth and latency.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-18T12:53:18.517" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18940" PostTypeId="2" ParentId="18936" CreationDate="2015-02-18T13:28:53.823" Score="2" Body="&lt;p&gt;Since Intel Westmere, you cannot use the hardware performance counters to measure flops like we think of them accurately. The counter counts operations issued not executed or retired. Thus, if you need to load some data from memory and there is a stall waiting for it to come back, the instruction may be re-issued many times before it completes. We've seen overcounts as high as 10x on E5-2650 v1 (i.e. SandyBridge), and the same is true for IvyBridge and Haswell. If you want to count flops, you have to count them by hand in your routines instead of using the hardware counters. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-02-18T13:28:53.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18942" PostTypeId="1" CreationDate="2015-02-18T19:07:48.857" Score="1" ViewCount="619" Body="&lt;p&gt;In work I am currently working on I need to mesh some structure with equilateral triangles to study it using a kind of discrete element method known as spring networks or Lattice model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To mesh the structure, first I generate the equilateral triangular grid then I do the intersection between the structure and the grid.&#xA;My problem lies in the second step, the intersection between the structure to mesh and the grid, I found difficulties to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion will be highly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/NBaam.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="14548" LastEditorUserId="14548" LastEditDate="2015-02-18T21:00:46.823" LastActivityDate="2015-02-18T21:00:46.823" Title="Equilateral triangle based mesh generation by intersection" Tags="|computational-geometry|discretization|mesh|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="18943" PostTypeId="1" CreationDate="2015-02-18T19:30:50.383" Score="5" ViewCount="395" Body="&lt;p&gt;It is necessary for me to solve a Poisson problem with a numerical method on a square domain with two types of triangular mesh: uniform triangular mesh (using uniform distribution nodes on square) and nonuniform triangular mesh using pdetool in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one leads to a banded matrix and the second one leads to a sparse matrix but not banded. This is the only different between two method. It should be noted that the sparsity of two matrices are the same. Also the condition numbers of those are nearlly equal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the result by using uniform triangular mesh is much better than the second one.&#xA;Is there anyone who know about this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I should say the quality of two meshes are same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nWgKf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="123" LastEditDate="2015-04-24T07:48:05.263" LastActivityDate="2021-10-29T11:08:18.240" Title="Why do structured and unstructured discretizations give different errors?" Tags="|linear-algebra|matlab|finite-element|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="18948" PostTypeId="2" ParentId="18913" CreationDate="2015-02-19T05:19:29.830" Score="0" Body="&lt;p&gt;Typically, you would impose the Dirichlet conditions as initial conditions. If $C_{i}$ corresponds to an initial condition for each index $i$ in some set, you would set each of these $C_{i}$s to a fixed value (the Dirichlet initial condition), and then make sure that $f_{i}$ is set to zero, and the $i$th row of $M$ is the $i$th row of the identity matrix.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-02-19T05:19:29.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18949" PostTypeId="1" CreationDate="2015-02-19T17:40:14.553" Score="0" ViewCount="2927" Body="&lt;p&gt;I am working on a problem that requires an iterative procedure to solve a linear system of equations, the system of equations in matrix form is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\underbrace{\begin{bmatrix}&#xA;r_{11} &amp;amp; r_{12} &amp;amp; r_{13} &amp;amp; \cdots &amp;amp; r_{1j} \\&#xA;r_{21} &amp;amp; r_{22} &amp;amp; r_{23} &amp;amp; \cdots &amp;amp; r_{2j} \\&#xA;r_{31} &amp;amp; r_{32} &amp;amp; r_{33} &amp;amp; \cdots &amp;amp; r_{3j} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;r_{i1} &amp;amp; r_{i2} &amp;amp; r_{i3} &amp;amp; \cdots &amp;amp; r_{ij}&#xA;\end{bmatrix}}_\textit{R}&#xA;\underbrace{\begin{bmatrix}&#xA;a_{1} &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; a_{2} &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; a_{3} &amp;amp; \cdots &amp;amp; 0 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; a_{j}&#xA;\end{bmatrix}}_\textit{A}&#xA;+&#xA;\underbrace{\begin{bmatrix}&#xA;b_{1} &amp;amp; b_{2} &amp;amp; b_{3} &amp;amp; \cdots &amp;amp; b_{j} \\&#xA;b_{1} &amp;amp; b_{2} &amp;amp; b_{3} &amp;amp; \cdots &amp;amp; b_{j} \\&#xA;b_{1} &amp;amp; b_{2} &amp;amp; b_{3} &amp;amp; \cdots &amp;amp; b_{j} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;b_{1} &amp;amp; b_{2} &amp;amp; b_{3} &amp;amp; \cdots &amp;amp; b_{j}&#xA;\end{bmatrix}}_\textit{B}&#xA;=&#xA;\underbrace{\begin{bmatrix}&#xA;c_{1} &amp;amp; c_{1} &amp;amp; c_{1} &amp;amp; \cdots &amp;amp; c_{1} \\&#xA;c_{2} &amp;amp; c_{2} &amp;amp; c_{2} &amp;amp; \cdots &amp;amp; c_{2} \\&#xA;c_{3} &amp;amp; c_{3} &amp;amp; c_{3} &amp;amp; \cdots &amp;amp; c_{3} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;c_{i} &amp;amp; c_{i} &amp;amp; c_{i} &amp;amp; \cdots &amp;amp; c_{i}&#xA;\end{bmatrix}}_\textit{C}\\&#xA;R_{i, j}A_{j, j} + B_{i, j} = C_{i, j}&#xA;$$&#xA;Now matrix R is fully known (input), matrices A, B, C are unknown. I am working on an iterative procedure in which I provide initial guesses for A and B, then calculate C. The iteration is to be carried out till I get a converged value for C (output I require). The code is being developed in Python. So far no luck in choosing the initial guess too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EXAMPLE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate the problem I have given an example for a $2 \times 2$ matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix}&#xA;r_{11} &amp;amp; r_{12} \\&#xA;r_{21} &amp;amp; r_{22} \\&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;a_{1} &amp;amp; 0 \\&#xA;0 &amp;amp; a_{2} \\&#xA;\end{bmatrix}&#xA;+&#xA;\begin{bmatrix}&#xA;b_{1} &amp;amp; b_{2} \\&#xA;b_{1} &amp;amp; b_{2} \\&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;c_{1} &amp;amp; c_{1} \\&#xA;c_{2} &amp;amp; c_{2} \\&#xA;\end{bmatrix}\\$$&#xA;this gives us:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_{1}r_{11} + b_{1} = a_{2}r_{12} + b_{2} = c_{1}\\&#xA;a_{1}r_{21} + b_{1} = a_{2}r_{22} + b_{2} = c_{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From these equations I get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_{2} = a_{1}\frac{(r_{11} - r_{21})}{(r_{12} - r_{22})}; \quad b_{2}=b_{1} + a_{1}\frac{(r_{11} - r_{21})(r_{11} - r_{12})}{(r_{12} - r_{22})}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This helps in the iterative method, but I would like to generalize it for a bigger matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have added two reproducible Python codes that should be helpful for a $2 \times 2$ matrix:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Code 1&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;R = np.matrix([[2.5, 2.9], [2.3, 2.7]])&#xA;m = R.shape[0]&#xA;n = R.shape[1]&#xA;A = np.matrix(np.zeros(shape=(n,n)))&#xA;B = np.matrix(np.zeros(shape=(m,n)))&#xA;A[0,0] = 1&#xA;B[0,0] = 1&#xA;k = 0&#xA;for i in range (1, 10000):&#xA;    A[0,0] = A[0, 0] - 0.0000001&#xA;    B[0,0] = B[0, 0] - 0.0000001&#xA;    for j in range(1, n):&#xA;        A[j, j] = A[(j-1), (j-1)] * ((R[0, 0] - R[1, 0]) / (R[0, 1] - R[1, 1]))&#xA;        B[0, j] = B[0, 0] + A[0, 0] * (((R[0, 0] - R[1, 0]) / (R[0, 1] - R[1, 1])) * (R[0, 0] - R[0, 1]))&#xA;        B[1, 0] = B[0, 0]&#xA;        B[1, j] = B[0, 0] + A[0, 0] * (((R[0, 0] - R[1, 0]) / (R[0, 1] - R[1, 1])) * (R[0, 0] - R[0, 1]))&#xA;    C = R * A + B&#xA;    C_convergence = np.all(np.all(np.diff(C, axis = 1) == 0, axis = 1), axis = 0)&#xA;    k = k + 1&#xA;    print k&#xA;print C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Code 2&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;R = np.matrix([[2.5, 2.9], [2.3, 2.7]])&#xA;m = R.shape[0]&#xA;n = R.shape[1]&#xA;A = np.matrix(np.zeros(shape=(n,n)))&#xA;B = np.matrix(np.zeros(shape=(m,n)))&#xA;A[0,0] = 1&#xA;B[0,0] = 1&#xA;k = 0&#xA;C_convergence = False&#xA;k = 0&#xA;while C_convergence == False:&#xA;    A[0,0] = A[0, 0] - 0.0000001&#xA;    B[0,0] = B[0, 0] - 0.0000001&#xA;    for j in range(1, n):&#xA;        A[j, j] = A[(j-1), (j-1)] * ((R[0, 0] - R[1, 0]) / (R[0, 1] - R[1, 1]))&#xA;        B[0, j] = B[0, 0] + A[0, 0] * (((R[0, 0] - R[1, 0]) / (R[0, 1] - R[1, 1])) * (R[0, 0] - R[0, 1]))&#xA;        B[1, 0] = B[0, 0]&#xA;        B[1, j] = B[0, 0] + A[0, 0] * (((R[0, 0] - R[1, 0]) / (R[0, 1] - R[1, 1])) * (R[0, 0] - R[0, 1]))&#xA;    C = R * A + B&#xA;    C_convergence = np.all(np.all(np.diff(C, axis = 1) == 0, axis = 1), axis = 0)&#xA;    k = k + 1&#xA;    print k&#xA;print C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;UPDATE 2&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I have written the matrices in an alternative way which is of the form $AX = B$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix}&#xA;r_{11} &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; r_{12} &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; r_{13} &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; r_{1j} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 1 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;r_{21} &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; r_{22} &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; r_{23} &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; r_{2j} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 1 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp;\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp;\\&#xA;r_{i1} &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; -1 \\&#xA;0 &amp;amp; r_{i2} &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; -1 \\&#xA;0 &amp;amp; 0 &amp;amp; r_{i3} &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; -1 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; r_{ij} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; -1 \\&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;a_{1} \\&#xA;a_{2} \\&#xA;a_{3} \\&#xA;\vdots \\&#xA;a_{j} \\&#xA;b_{1} \\&#xA;b_{2} \\&#xA;b_{3} \\&#xA;\vdots \\&#xA;b_{j} \\&#xA;c_{1} \\&#xA;c_{2} \\&#xA;c_{3} \\&#xA;\vdots \\&#xA;c_{i} \\&#xA;\end{bmatrix} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now how do I solve this using an iterative method?&lt;/p&gt;&#xA;" OwnerUserId="14586" LastEditorUserId="14586" LastEditDate="2015-02-23T10:02:30.107" LastActivityDate="2015-02-23T12:50:58.613" Title="How to obtain a convergent solution iteratively for a linear system of equations?" Tags="|linear-algebra|python|matrix|numerical-analysis|iterative-method|" AnswerCount="2" CommentCount="4" ClosedDate="2015-04-24T16:56:15.603" ContentLicense="CC BY-SA 3.0" />
  <row Id="18950" PostTypeId="2" ParentId="11448" CreationDate="2015-02-19T18:33:38.417" Score="1" Body="&lt;p&gt;The following answer from before the edit interpreted the words &quot;not possible&quot; as &quot;not possible to take 1&quot;.  If we read it as &quot;not possible to both take 1&quot;, then this answer does not apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm reading the question correctly, it seems to me this problem is far easier than maximum weight clique problems.  It appears to be $O(m^2)$.  The given coefficients $b_i$ are all positive, so it is apparent that the maximum weighted sum for a given set of coefficients would be where $x_i$ were all 1.  So we want them to be 1 unless they cannot be.  Therefore, initialize all $x_i$ to 1, iterate over all $m^2$ elements of $A_{ij}$, and if those matrix elements are 0, set the corresponding $x_i$ and $x_j$ to 0.  Then we have all the 1's in the vector $x$ that we are allowed to have by the matrix, and the weighted sum is maximized.&lt;/p&gt;&#xA;" OwnerUserId="14557" LastEditorUserId="14557" LastEditDate="2015-03-26T20:39:13.227" LastActivityDate="2015-03-26T20:39:13.227" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18951" PostTypeId="1" CreationDate="2015-02-19T19:57:55.923" Score="1" ViewCount="127" Body="&lt;p&gt;I have an application where I want to have 3 different machines writing to their own independent HDF5 files at a high rate, but have a way to read data from them externally and combine it. Is HDF5 a good way to do that or should I look into another format?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main purpose is I have 3 independent packet capturing boxes that will be capturing at about 10Gbps each. They all filtered for specific UDP messages and drop all the rest of the traffic. Inside of these UDP messages there are 3 different types of data, which can be thought of as C structs. These UDP messages need to be written to disk (and possibly formatted differently) in a time series format so that I can search them later. My ultimate goal is to be able to load them into MATLAB quickly, which likely means HDF5 since that's the native format there. I was looking in to PyTables and TsTables, but I'm not quite sure if these do what I want. PyTables doesn't look like it's for time series, and TsTables looks fairly immature.&lt;/p&gt;&#xA;" OwnerUserId="14591" LastEditorUserId="14591" LastEditDate="2015-02-20T15:09:32.217" LastActivityDate="2015-02-20T15:09:32.217" Title="HDF5 Many Writers, One External Reader" Tags="|algorithms|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="18954" PostTypeId="2" ParentId="18949" CreationDate="2015-02-20T01:06:07.477" Score="3" Body="&lt;p&gt;It looks like you have &lt;strong&gt;&lt;em&gt;i&lt;/em&gt;&lt;/strong&gt; total equations. If A, B, and C are all unknown then you have &lt;strong&gt;&lt;em&gt;j&lt;/em&gt;&lt;/strong&gt; + &lt;strong&gt;&lt;em&gt;j&lt;/em&gt;&lt;/strong&gt; + &lt;strong&gt;&lt;em&gt;i&lt;/em&gt;&lt;/strong&gt; number of unknowns. You cannot find a unique solution to this problem because you have more unknowns than number of equations. Even if you create an iterative method that converges to a solution, this will be one solution of an infinite number of solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for people interested in iterative solutions to linear systems of equations they can look into the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jacobi Method (&lt;a href=&quot;http://en.wikipedia.org/wiki/Jacobi_method&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Jacobi_method&lt;/a&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gauss-Seidel (&lt;a href=&quot;http://en.wikipedia.org/wiki/Gauss-Seidel_method&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Gauss-Seidel_method&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For nonlinear algebraic equations you can look into the Newton-Raphson method.&lt;/p&gt;&#xA;" OwnerUserId="14594" LastActivityDate="2015-02-20T01:06:07.477" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18956" PostTypeId="1" CreationDate="2015-02-20T18:16:11.230" Score="0" ViewCount="84" Body="&lt;p&gt;I want to solve for $n_{0}$ for a fixed value of $n$, lets say $n=1$&#xA; $$ n= n_{0}+ \dfrac{1}{2}\int_{-1/2}^{1/2}dq\left(\dfrac{e_{q}+Un_{0}}{\hbar\omega}-1\right) $$ &#xA;where $e_{q}=2[1-cos(2\pi q)] $ and $\hbar\omega=(e_{q}^{2}+2Un_{0}e_{q})^{1/2} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I want to plot between  $U$ and $n_{0}$. The following is the code in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; eq = @(q) 2*(1-cos(2*pi*q));&#xA; hq = @(q,U,n0) ((eq(q))^2+2*U*n0*(eq(q))).^(1/2);&#xA; y = @(q,U,n0) (((eq(q))+(U*n0))/hq(q,U,n0))-1;&#xA; a = -0.5;&#xA; b = 0.5;&#xA; v = @(U,n0) quad(@(q) y(q,U,n0),a,b);&#xA; cv =@(U,n0) n0+(0.5*v)-1;&#xA; n0 = 0.1;&#xA; options = optimset('Display', 'iter'); &#xA; n = @(U) fsolve(vv,n0,0.1);&#xA; U = 0:0.1:20;&#xA; plot(U,n(U))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;edit 2 code corrected &#xA;edit 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I still facing the following errors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error using  ^ &#xA;Inputs must be a scalar and a square matrix.&#xA;To compute elementwise POWER, use POWER (.^) instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error in @(q,n0)((eq(q))^2+2*U&lt;em&gt;n0&lt;/em&gt;(eq(q))).^0.5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error in @(q,n0)(((eq(q))+(U*n0))/hq(q,n0))-1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error in @(q)y(q,n0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error in quad (line 72)&#xA;y = f(x, varargin{:});&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error in @(n0)quad(@(q)y(q,n0),a,b)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error in @(n0)n0+(0.5*v(n0))-1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error in simulv1 (line 12)&#xA;n = fsolve(cv(n0),n0,0.1,options);&lt;/p&gt;&#xA;" OwnerUserId="14602" LastEditorUserId="14602" LastEditDate="2015-02-25T15:28:26.140" LastActivityDate="2015-02-25T15:28:26.140" Title="Solving condensate density problem in MATLAB" Tags="|matlab|quantum-mechanics|integral-equations|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18957" PostTypeId="2" ParentId="18923" CreationDate="2015-02-20T18:58:54.493" Score="3" Body="&lt;p&gt;Your equations look correct except that you dropped a minus sign on $\rho_d$ in the last equations. However the mistake (or misinterpretation) is assuming that the finite difference method (FDM) can represent an ideal surface charge. The difference equations that you gave can not tell the difference if the charge computed by $\nabla^2 \phi$ at a given point is due to surface charges or the charge per unit volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion is to fix the charge density per unit volume to the average surface charge density with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho_{d \, i,j} = \frac{\sigma_{s \, i,j}}{\Delta y} \, \, \, \, \, \textrm{ (at the boundary)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this approach is much less confusing than trying to force an ideal surface charge into the difference equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to address the dielectric boundary somehow. Sadiku (Numerical Techniques in Electromagnetics with Matlab, 3rd Ed., page 148-149) gives a nice example on how to modify the difference equation to deal with the dielectric boundary.&lt;/p&gt;&#xA;" OwnerUserId="14594" LastEditorUserId="14594" LastEditDate="2015-02-20T19:35:37.143" LastActivityDate="2015-02-20T19:35:37.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18958" PostTypeId="1" CreationDate="2015-02-20T21:06:10.733" Score="1" ViewCount="303" Body="&lt;p&gt;I have a couple of conceptual questions regarding the calculation of phonon dispersions. I wish to calculate the phonon dispersion of Nickel with my lattice vectors given by [111], [1-10] and [11-2] (though my lattice vectors are non-orthogonal to minimize the number of atoms. My lattice vectors are 2.48901587 0 0, 1.244507935 2.155550974 0, 0 0 40). I also have ~15 A vacuum in my cell. I seem to be getting incorrect phonons for this structure but my results are correct if I choose a FCC unit cell with 4 atoms. My questions are summarized below: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do I have to make any changes to my calculations because I have vacuum in my cell?&lt;/li&gt;&#xA;&lt;li&gt;I see that calculating force constants (using small displacement method), and hence phonons, requires the creation of supercells. What should they ideally be? Would it be wrong if I just use 1x1x1 cell?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have attached the results I'm obtaining. The first figure is for the Ni unit cell and the second figure is for the Ni cell with vacuum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/1UinA.png&quot; alt=&quot;Phonon dispersion for Ni FCC unit cell&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7vNW4.png&quot; alt=&quot;Phonon dispersions for Ni cell with vacuum&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11674" LastActivityDate="2015-02-20T21:06:10.733" Title="Calculation of phonon dispersion" Tags="|density-functional-theory|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18959" PostTypeId="2" ParentId="3540" CreationDate="2015-02-21T21:36:38.273" Score="2" Body="&lt;p&gt;I assume you didn't specify the &lt;code&gt;fprime&lt;/code&gt; parameter. If you don't provide this param &lt;code&gt;fmin_cg&lt;/code&gt; has to figure out its own solution what usually is much slower than which a provided optimal solution. Your code might look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;theta = fmin_cg(compute_cost_reg, fprime=compute_gradient_reg,&#xA;                x0=theta, args=(X, y, lambd), maxiter=50)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14616" LastActivityDate="2015-02-21T21:36:38.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18961" PostTypeId="1" CreationDate="2015-02-22T13:58:14.410" Score="0" ViewCount="294" Body="&lt;p&gt;As we know, in the interior point methods, all the iterates have to be strictly feasible. I implemented an affine scaling interior point for nonlinear objective functions. For small examples (2D), it works as expected. However, for large examples (e.g. optimal control problems), after a finite number of steps, some elements of the iterates start to be on the boundaries. I don't know the exact solutions of those optimal control problems, so I compared the results with the other methods (i.e. same problems, different methods). The numerical solutions obtained from this interior point method have the same number of active elements as the solutions obtained from the other methods. The solutions have the same plots as well as objective function values. The issue here is why the iterates are not strictly feasible. (they are for small examples but not for examples with large numbers of variables). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, the method of interest (i.e. interior point method) produces the same numerical solutions as the other methods but the iterates are not strictly feasible. Could you please help me?&lt;/p&gt;&#xA;" OwnerUserId="13191" LastActivityDate="2015-02-22T17:50:46.670" Title="Strict Feasibility in Interior Point Methods" Tags="|numerical-analysis|iterative-method|constrained-optimization|newton-method|optimal-control|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18962" PostTypeId="2" ParentId="18872" CreationDate="2015-02-22T14:20:07.450" Score="0" Body="&lt;p&gt;My suggestion is to analytically detect the singularities and then to switch form explicit solver to implicit one around the singularity.&#xA;From this equation&#xA;$&#xA;\psi ' = \dfrac{\phi ' + 2 \frac{\phi}{\eta}}{\eta-\phi}\psi.&#xA;$&#xA;it follows that the singularity is around $\eta-\phi$. Therefore, you should require that &#xA;$&#xA;\phi ' = - 2 \frac{\phi}{\eta} &#xA;$&#xA;with a solution $\phi=\frac{c}{{\eta}^{2}}$.&#xA;Therefore &#xA;$&#xA;\eta = \frac{c}{{\eta}^{2}}&#xA;$&#xA;and the critical values is $\eta^*= ^3\sqrt{c}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second equation&#xA;$&#xA;\phi'=\dfrac{\frac{1}{\gamma} f'}{\psi(\eta-\phi)}-\dfrac{3}{2}\dfrac{\phi}{\eta-\phi}&#xA;$&#xA;will be then&#xA;$&#xA;  f' = \dfrac{3 \gamma \psi}{2} \phi  &#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first equation&#xA;$&#xA;f'=\dfrac{f[-3\eta + (3+\gamma/2)\phi - 2\gamma \dfrac{\phi ^2}{\eta}]}{(\eta-\phi)^2-f/\psi}&#xA;$&#xA;will become&#xA;$&#xA;f'=\psi(-3\eta + (3+\gamma/2)\phi - 2\gamma \dfrac{\phi ^2}{\eta}) &#xA;$&lt;/p&gt;&#xA;" OwnerUserId="10345" LastEditorUserId="10345" LastEditDate="2015-03-14T23:20:30.443" LastActivityDate="2015-03-14T23:20:30.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18963" PostTypeId="1" CreationDate="2015-02-22T15:11:57.213" Score="1" ViewCount="135" Body="&lt;p&gt;For fitting a response surface model to a physical process, I have 3-4 relevant &quot;signals&quot;, like a feature density, a signal based on a feature width, or a signal based on a distance to the next feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of the surface model is to capture (=fit) the effect of inter-dependencies between the different signals on the response. I started with a bilinear model, where I have a constant term, a linear term for each signal, and a bilinear term for each product of two signals. This gave a nice fit overall, but I wondered how to add more fitting parameters for better fine tuning and non-physcial overfitting:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I could add terms of higher polynomial degree&lt;/li&gt;&#xA;&lt;li&gt;I could use piecewise defined functions, something like piecewise multilinear interpolation or multilinear spline interpolation&lt;/li&gt;&#xA;&lt;li&gt;I could use rational functions instead of polynomial functions&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I tried to fit the quotient of two bilinear model, and the result looked great. Maybe it's just because the many parameters allowed a non-physical overfitting, but now I'm burning to understand more about rational functions and their approximation properties anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many possible parameterizations for rational functions, in addition to the simple quotient of two polynomial functions which I have used. The barycentric form is one possible parametrization of a univariate rational function. When I tried to better understand it, it turned out that it is &lt;a href=&quot;https://math.stackexchange.com/questions/1158836/can-every-rational-function-be-represented-in-barycentric-form&quot;&gt;closely related to the simple quotient of two polynomial functions&lt;/a&gt;. But other forms like allowing $\frac{1}{x}$ or $\frac{1}{1-x}$ terms (and some of their products with other terms) seem to be independent and closer related to higher polynomial functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, I'm thinking about invariance of the form under transformations like $f(x,y)\to \alpha f(x,y)+\beta$ or $f(x,y)\to f(\alpha x+\beta, y)$. Now the simple quotient form is invariant under Möbious transformations like $f(x,y)\to \frac{a f(x,y)+b}{cf(x,y)+d}$, but that doesn't really seem like an important invariance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More practically relevant seems the fact that any representation for polynomial functions directly translates into a corresponding representation for rational functions as a simple quotient. The number of parameters is at most doubled by this, which also seems like a good thing.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-03-09T00:28:11.213" Title="What are good parametrizations of rational functions for response surface models?" Tags="|interpolation|regression|approximation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18964" PostTypeId="2" ParentId="3417" CreationDate="2015-02-22T15:42:42.803" Score="11" Body="&lt;p&gt;Doing one-off best rational approximations can often be accomplished by &quot;manual&quot; iterations of the &lt;a href=&quot;http://en.wikipedia.org/wiki/Remez_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Remez algorithm&lt;/a&gt;: interpolate a rational approximation with (relative or absolute) alternating sign errors at an initial guess for interpolation points, locate one (or more) points where the actual error exceeds that of the interpolation points and pivot (swapping out one or more of the guessed points for these, in a way that preserves alternating sign errors).  It's a good way to understand the theory from the ground up.  When the required accuracy is modest, I find a spreadsheet implementation helps me feel my way through the calculations and adapt problems through cut-and-paste.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give a simple illustration, consider the non-polynomial function $y=1/x$ on interval $1 \le x \le 2$.  The secant line through the endpoints looks like:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/6LRrV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we lower the straight line by half the distance at the point of &quot;maximum error&quot;, then we will have three &quot;equi-oscillatory&quot; points of absolute error counting the two endpoints and this critical point in the interior.  Thus a best linear polynomial approximation to a smooth convex function can be found almost by inspection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.chebfun.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chebfun software&lt;/a&gt; itself is open source (&lt;a href=&quot;https://raw.githubusercontent.com/chebfun/chebfun/master/LICENSE.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;BSD-style license&lt;/a&gt;) since v4.0.  Not sure why links to their site may have given a 404 error back at that time, but they are now at v5.2.1.  (They also have a &lt;a href=&quot;https://github.com/chebfun/chebfun&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub repository&lt;/a&gt;.) The Chebfun project aims at more than best rational approximations, with the central concept being a Matlab vector-like syntax for (univariate real) functions on an interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To complete the circle would require getting it working under &lt;a href=&quot;http://www.gnu.org/software/octave/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Octave&lt;/a&gt; rather than Matlab.  There was an &lt;a href=&quot;http://octave.1599824.n4.nabble.com/Chebfun-td1651165.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Octave-maintainers thread about this&lt;/a&gt; started in 2010.  A &lt;a href=&quot;http://www.chebfun.org/publications/hale_trefethen_Feb142012.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2012 paper by Chebfun maintainers/authors&lt;/a&gt; suggests some effort has gone into Octave compatibility from their side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the &lt;a href=&quot;http://www.chebfun.org/about/projects.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chebfun-related projects page&lt;/a&gt; suggests that because the Chebfun software is open source, it might be possible to do this port, or to make use of one of the open source packages listed there, such as Olivier Verdier's &lt;code&gt;pychebfun&lt;/code&gt;, a &lt;a href=&quot;https://github.com/pychebfun/pychebfun#readme&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python Chebyshev Functions project&lt;/a&gt; hosted at GitHub.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2015-10-23T12:08:06.717" LastActivityDate="2015-10-23T12:08:06.717" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18965" PostTypeId="2" ParentId="18961" CreationDate="2015-02-22T16:58:25.373" Score="2" Body="&lt;p&gt;Unlike barrier methods, the affine scaling method doesn't use a barrier to push the iterates away from the boundaries of the feasible region.  As a result, the iterates can very quickly approach the boundary of the feasible reason.  Furthermore (and this can be a problem) the method can easily get &quot;stuck&quot; taking very short steps along the boundary of the feasible region.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not surprising at all that you're seeing this behavior.  If you switched to a barrier method you would likely see faster and more robust convergence.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-02-22T17:50:46.670" LastActivityDate="2015-02-22T17:50:46.670" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="18966" PostTypeId="1" CreationDate="2015-02-23T07:19:03.920" Score="0" ViewCount="207" Body="&lt;p&gt;I know of several ways to calculated the eigen states of 1D potentials (i.e. DVR, Crank–Nicolson, etc). However I wonder what is the most efficient way to do the same for a N-Dimensional potential? How would this change if the potential has some kind of symmetry ?&lt;/p&gt;&#xA;" OwnerUserId="5502" LastActivityDate="2015-02-25T00:47:56.417" Title="most efficient way to calculate eigen states of a 2D or 3D potential (Matlab)" Tags="|eigensystem|high-dimensional|efficiency|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18968" PostTypeId="2" ParentId="18949" CreationDate="2015-02-23T12:50:58.613" Score="0" Body="&lt;p&gt;I don't get why you need an iterative solution. Reorder to $RA =C-B$ and solve columnwise: Choose some $a_i$ and $i$th columns of $C$ and $B$ that fit. By the way: You may want to exclude the solution $A=B=C=0$. &lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2015-02-23T12:50:58.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18969" PostTypeId="1" CreationDate="2015-02-23T13:08:08.297" Score="1" ViewCount="42" Body="&lt;p&gt;I am working on a simple explicit-integration lumped-mass elastic FEM code which implements CST+DKT triangles (plate+shell) and constant-strain tetrahedra (&lt;a href=&quot;http://woodem.eu/doc/theory/membrane-element.html&quot; rel=&quot;nofollow&quot;&gt;http://woodem.eu/doc/theory/membrane-element.html&lt;/a&gt;, &lt;a href=&quot;http://woodem.eu/doc/theory/tet4-element.html&quot; rel=&quot;nofollow&quot;&gt;http://woodem.eu/doc/theory/tet4-element.html&lt;/a&gt;). The code focuses on contact dynamics, so FEM is there only to model flexible boundaries. I would like to add some kind of viscous damping to the model, and I am looking for some resource which is not overly complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I independently thought I could use the elastic stiffness matrix $\mathbf{K}$ (as in $f=\mathbf{K}u$), scaled by some viscosity factor $\eta'$ (that would be computed from material's $E$ and $\eta$), to compute viscous resisting force as $f_v=-\eta'\mathbf{K}\dot u$. It this formulation something known in literature? Or is it plain wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for pointers.&lt;/p&gt;&#xA;" OwnerUserId="9988" LastActivityDate="2015-02-23T13:08:08.297" Title="Resources for viscous behavior in simple FEM" Tags="|finite-element|stiffness|explicit-methods|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="18970" PostTypeId="1" CreationDate="2015-02-23T13:59:17.093" Score="1" ViewCount="2051" Body="&lt;p&gt;I am not sure if this is the correct forum for this question but here it goes. If this is not the correct forum please direct me to the appropriate forum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to add some periodic noise to a 1-D signal in Matlab. I have to create the noise myself. Now my issues is of the frequency and the sample size of the noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say that the signal vector in which I wish to add noise has length &lt;code&gt;x&lt;/code&gt; and in Matlab, as we know, when we wish to add two signals, the length of the vectors must be same, so I will have to create a noise with the same length &lt;code&gt;x&lt;/code&gt;. Now first I have to decide the frequency &lt;code&gt;f&lt;/code&gt; of the noise, which is a problem because in Matlab every thing is discrete and I am not sure whether my &lt;code&gt;f&lt;/code&gt; should be a function of &lt;code&gt;x&lt;/code&gt; because the value of &lt;code&gt;f&lt;/code&gt; affects the length of my noise vector signal overall and also the shape/look of my noise vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to ask that what are the best practices to follow when creating and adding periodic noise to a signal of fixed length. For random/Gaussian random it is very easy as it is not periodic but what about periodic noise.&lt;/p&gt;&#xA;" OwnerUserId="9478" LastActivityDate="2015-02-23T14:19:34.807" Title="Creating Periodic Noise in Matlab and then adding it to a signal" Tags="|matlab|signal-processing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18971" PostTypeId="2" ParentId="18966" CreationDate="2015-02-23T14:13:30.317" Score="2" Body="&lt;p&gt;This is really too big a question to ask in a single post, as there are at least two steps that need to be addressed: discretization and eigensolves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step is that you need to discretize the partial differential equation that underlies your problem. There are of course many ways of doing this, the finite element and finite differences method being two popular ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second one being to find the eigenvalues and eigenvectors of this discrete problem. This is much harder than in 1d because the matrices are typically much larger, and methods are consequently more expensive. There are, again, any number of methods available for this (and they are available in high-quality libraries).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to see some of this in action, here is a tutorial program of the deal.II library that shows you some of this: &lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/step_36.html&quot; rel=&quot;nofollow&quot;&gt;http://dealii.org/developer/doxygen/deal.II/step_36.html&lt;/a&gt; (Disclaimer: I'm one of the authors of this library.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-23T14:13:30.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18972" PostTypeId="2" ParentId="18970" CreationDate="2015-02-23T14:19:34.807" Score="2" Body="&lt;p&gt;You start at the wrong end. Your question is about how to do it with a vector of length $N$ in Matlab but your question states that you are not clear about what periodic noise actually means in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't ask how to do something you don't yet understand. You first need to understand what it is you want to do, and then actually doing it will become obvious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in your case, you first need to figure out what the individual entries in your vector correspond to (time instances? locations?) and what periodicity and frequency actually mean in this context (e.g., if the elements of your vector correspond to locations on a 2d grid, do you want to be periodic in $x$ or $y$ direction). You then also have to ask what features your noise should have (e.g., a single frequency with random amplitude, or a particular power spectrum with random phases?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary: You can't correctly implement something for which you don't know what &quot;correct&quot; actually means.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-23T14:19:34.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18974" PostTypeId="1" CreationDate="2015-02-23T15:59:53.757" Score="3" ViewCount="515" Body="&lt;p&gt;I am going to start my first year of a research-oriented master program on inverse problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I know, unlike pure math students, applied math students usually don't spend the first year reading certain textbook (such that Evans' PDE book). Instead, applied math students start by reading certain papers, and if they don't understand some concepts, they refer to certain books/papers, read specific sections which can solve the problem and proceed the original paper. And they spend a lot of time implementing numerical schemes. Hence I got a feeling that applied math students may not have enough time to systematically build up a solid theoretical background (such as functional analysis) by finishing an advanced textbook, taking their own notes and solving exercises.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;May I know whether my understanding is correct? As graduate students/ researchers on numerical analysis, have you spent some time reading a entire (or a large part of) pure math book (especially at the beginning of your graduate program)? May you share your experience about how to balance reading textbooks and doing research with me? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, currently I am interested in reading Brezis book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387709134&quot; rel=&quot;nofollow&quot;&gt;Functional analysis, sobolev space and partial differential equation&lt;/a&gt;. But I am not sure whether it's useful and how to allocate the time. &lt;/p&gt;&#xA;" OwnerUserId="5527" LastEditorUserId="20688" LastEditDate="2019-06-22T20:13:19.077" LastActivityDate="2019-06-22T20:13:19.077" Title="Do you spend time reading pure math books as a graduate student on computational math" Tags="|inverse-problem|education|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="18975" PostTypeId="2" ParentId="18974" CreationDate="2015-02-23T16:21:50.090" Score="7" Body="&lt;p&gt;I think your premise is flawed: If you wish to do research on a topic, you first need to learn the fundamentals -- no matter whether it's &quot;pure&quot; or &quot;applied&quot; math (a distinction which is not really useful in general) -- and the best place is from a good textbook (or, even better, attending a good lecture). Once you get to the cutting edge, you need to switch to current publications (again, both in pure and applied math). If your research bridges several fields (say, PDEs, numerical analysis, optimization in your case, or number theory and abstract algebra in algebraic number theory), you of course need to be more selective to fit the fundamentals of different fields into the same time frame. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for which books to read and how much, this depends on your specific topic. Surely your advisor is a better person to ask than random strangers on the internet -- after all, this is exactly his job?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(But I'd be surprised if this did not involve a solid foundation in functional analysis, and Brezis' is certainly a good book.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-02-23T16:21:50.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18976" PostTypeId="2" ParentId="18966" CreationDate="2015-02-23T17:49:20.230" Score="1" Body="&lt;p&gt;The main difference is in the dimensionality, that is reflected in the matrices. In 1D, e.g, using (FEM) Finite Element Method or (FDM) Finite Difference Methods the Stiffness (Hamiltonian) matrix is tridiagonal. In 2D, the Stiffness matrix is pentadiagonal for the FDM and octadiagonal for the FEM with linear or bilinear elements, using structured meshes. For Mass (overlap) matrices, we have identity matrix for FDM and octadiagonal for FEM. Depending on the solver the storage of the matrix is also a factor to take into account.  Besides these changes in dimensions, the procedure is similar: you look the first $m$ eigenvalues/eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, potentials are defined over non-bounded domains and you will need to do one of two things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;truncate&quot; it, make it's value infinite outside some region. Or, equivalently make the function zero (artificial Dirichlet BCs); or&lt;/li&gt;&#xA;&lt;li&gt;Use some type of BCs that approximate the unboundedness of the domain, like absorbing boundary conditions or infinite elements.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding the symmetries, you can have smaller meshes with these symmetries using the right BCs. For example, a Quantum Harmonic Oscillator have symmetric and anti-symmetric modes. For the symmetric ones we have $f'(x=0)=0$, for the anti- symmetric cases $f(x=0)=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR, I would suggest FDM if you are interested in simple cases, and FEM, since it can handle boundary conditions more naturally.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-02-25T00:47:56.417" LastActivityDate="2015-02-25T00:47:56.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18978" PostTypeId="2" ParentId="18974" CreationDate="2015-02-23T20:38:44.150" Score="3" Body="&lt;p&gt;To add my own experience to ChristianClason's post: it's not uncommon to for graduate students in computational science to read pure mathematics textbooks and take pure mathematics classes. I graduated with an engineering degree and have read large portions of several pure math books. Part of the reason I did so is because my adviser required me to take a certain amount of pure mathematics, since it was a prerequisite for optimization classes at MIT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are tradeoffs. It's certainly possible to read from textbooks about things like optimization, PDE methods, and numerical analysis that don't require much in the way of pure math knowledge. These textbooks are useful for rapid introduction to a subject, and will help you hit the ground running. However, you will be severely limited when it comes to developing novel numerical methods -- you'll be limited to more basic methods presented in those books, and won't have as good an understanding of when certain methods fail, and why. Without any pure math knowledge, you would likely be unable to demonstrate basic facts that would make such methods more compelling, such as convergence proofs or error bounds. Learning pure mathematics will give you the skills to understand and eventually write proofs that demonstrate rigorously mathematical properties of numerical methods that you are investigating. It also enables you to understand a wider selection of papers: after learning pure math, you'll better understand more papers in computational journals like SISC, SINUM, SIMAX, BIT Numerical Mathematics, Numerische Mathematik, etc., and it will be easier to write papers geared towards those journals.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-02-23T20:38:44.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18979" PostTypeId="1" CreationDate="2015-02-23T21:08:53.860" Score="4" ViewCount="1546" Body="&lt;p&gt;Here, SSPRK3 refers to third order strong stability preserving Runge-Kutta and RK3 refres to regular third order Runge-Kutta method. The meaning of the method is obvious from the name. However there is a restriction while one is choosing the time step. It is mentioned in &lt;a href=&quot;https://epubs.siam.org/doi/10.1137/S003614450036757X&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Strong stability preserving high-order time discretization methods&amp;quot; by Gottlieb, Shu , and Tadmor.&lt;/a&gt; Since my problem is nonliner systems of hyperbolic PDEs (partial differential equations), related part about my problem is in the section 4.1. The question is that what is the interaction between CFL condition and strong stability preserving coefficient ?&lt;/p&gt;&#xA;" OwnerUserId="11495" LastEditorUserId="36438" LastEditDate="2022-01-18T17:14:53.433" LastActivityDate="2022-01-18T17:14:53.433" Title="What is the difference between SSPRK3 and RK3 time discretization methods?" Tags="|finite-difference|numerics|finite-volume|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18982" PostTypeId="1" AcceptedAnswerId="19045" CreationDate="2015-02-24T11:42:57.517" Score="4" ViewCount="810" Body="&lt;p&gt;Maybe, this is a dump question. But anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I developed a finite difference code for a class of problems in dynamo theory. I used GNU Octave (MATLAB) which is good for testing. The problem size scales quadratic. Hence, the number of unknown becomes relatively large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am considering to rewrite my program in a different programming language. In my impression the most common languages used in scientific computing are FORTRAN, C++ or Python. The point I would like to make is that I don't want to reinvent the wheel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some aspects to get the idea:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I would like to use existing libraries/ packages such as LAPACK, ARPACK, UMFPACK, PETSc,... for linear algebra applications. I don't want to write Gaussian elimination again.&lt;/li&gt;&#xA;&lt;li&gt;I don't want to program every matrix multiply in for-loop.&lt;/li&gt;&#xA;&lt;li&gt;I would like to export unstructured data to the &lt;code&gt;vtk&lt;/code&gt; file format. I would like to use 2-D plotting tool like in GNU Octave.&lt;/li&gt;&#xA;&lt;li&gt;I would like to use MATLAB like methods such as &lt;code&gt;sparse&lt;/code&gt; or &lt;code&gt;blkdiag&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The possibility of running on multiple processors would be appreciated, i.e. openMPI.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am neither looking for a book such as &lt;em&gt;Numerical Recipes&lt;/em&gt;, a book on numerics of PDEs nor an introductory textbooks on programming. I am looking for a book that describes the general approach. How to structure the program? How to make us of existing packages? An example, such as the standard 2-D Poisson equation would be good.&lt;/p&gt;&#xA;" OwnerUserId="14284" LastEditorUserId="14284" LastEditDate="2015-02-28T08:59:21.093" LastActivityDate="2015-03-02T21:06:36.993" Title="literature on scientific computing" Tags="|python|c++|fortran|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="18984" PostTypeId="1" AcceptedAnswerId="18985" CreationDate="2015-02-24T18:42:55.090" Score="2" ViewCount="366" Body="&lt;p&gt;I am working on computing a challenging integral. I am working with someone else who wrote some code in Mathematica to compute it. I do not have mathematica so I am trying to do the same thing in C++. The problem is, he uses Mathematica's &lt;a href=&quot;http://reference.wolfram.com/language/ref/NIntegrate.html&quot; rel=&quot;nofollow&quot;&gt;NIntegrate function&lt;/a&gt; which I cannot figure out how to re-write.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I re-write this function in C++(or any other language)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Thanks.&lt;/strong&gt; &lt;em&gt;If you vote down, please tell me why so I can improve this question.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="14641" LastActivityDate="2015-02-24T19:57:34.527" Title="Mathematica NIntegrate function in C++" Tags="|c++|integral-equations|mathematica|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18985" PostTypeId="2" ParentId="18984" CreationDate="2015-02-24T19:57:34.527" Score="4" Body="&lt;p&gt;First of all, it is impossible to completely rewrite &lt;code&gt;NIntegrate&lt;/code&gt; because usually, it evaluates (parts of) the integrand symbolically to check for certain properties. This means, you probably have to re-implement &lt;em&gt;Mathematica&lt;/em&gt; completely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, if you are tackling one specific integral, I assume chances are very good that you can get similar numerical results by re-implementing some parts in C++ or using a library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would start by analysing (in &lt;em&gt;Mathematica&lt;/em&gt;) what algorithm is used for your integral. &lt;a href=&quot;http://reference.wolfram.com/language/tutorial/SomeNotesOnInternalImplementation.html#9009&quot; rel=&quot;nofollow&quot;&gt;In this tutorial&lt;/a&gt; you can find details about how &lt;em&gt;Mathematica&lt;/em&gt; chooses the algorithm when &lt;code&gt;NIntegrate&lt;/code&gt; uses &lt;code&gt;Method-&amp;gt;Automatic&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this, you should &lt;a href=&quot;http://reference.wolfram.com/language/ref/NIntegrate.html&quot; rel=&quot;nofollow&quot;&gt;study the help-page&lt;/a&gt; of &lt;code&gt;NIntegrate&lt;/code&gt; carefully to understand how it works. Then you should go on by &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;fixing the &lt;code&gt;Method&lt;/code&gt; option of &lt;code&gt;NIntegrate&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;using &lt;code&gt;EvaluationMonitor&lt;/code&gt; to see where your integrand is sampled&lt;/li&gt;&#xA;&lt;li&gt;using specific &lt;code&gt;MaxPoints&lt;/code&gt;, &lt;code&gt;MaxRecursion&lt;/code&gt;, etc settings&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;until you finally get equally good results. If you have achieved this, then you know what algorithm you have to implement or find in C++ library.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let me say that there is another way: Understand the integral you want to solve! You as human should &lt;em&gt;see&lt;/em&gt; what type of integrand you have and you can check the literature, how those types are solved numerically. This is probably faster than trying to understand what &lt;em&gt;Mathematica&lt;/em&gt; does.&lt;/p&gt;&#xA;" OwnerUserId="1978" LastActivityDate="2015-02-24T19:57:34.527" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="18986" PostTypeId="2" ParentId="14818" CreationDate="2015-02-24T20:04:09.873" Score="4" Body="&lt;p&gt;I once started writing anopen source version of Eureqa in Java. The project has limited capabilities but it implements the fitness function described in [1] and couple optimizations mentioned by the authors in other publications (e.g., searching for solutions in Pareto front).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link: &lt;a href=&quot;https://github.com/pkoperek/hubert&quot; rel=&quot;nofollow&quot;&gt;https://github.com/pkoperek/hubert&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Schmidt, Michael, and Hod Lipson. &quot;Distilling free-form natural laws from experimental data.&quot; Science 324.5923 (2009): 81-85. DOI:&lt;a href=&quot;http://doi.org/10.1126/science.1165893&quot; rel=&quot;nofollow&quot;&gt;10.1126/science.1165893&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14643" LastEditorUserId="9667" LastEditDate="2015-02-25T02:19:56.097" LastActivityDate="2015-02-25T02:19:56.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18987" PostTypeId="1" AcceptedAnswerId="18989" CreationDate="2015-02-24T22:48:55.570" Score="7" ViewCount="9248" Body="&lt;p&gt;How can I generate arbitrarily high quality $1/f$ noise, for use in a model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My model involves a lot of feedback, over a large number of iterations, with a very high bandwidth, so I'd like the $1/f$ noise to be as &quot;ideal&quot; as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming there's not an ideal solution, it would be nice to have an approach that allowed &quot;noise quality&quot; to be a parameter, so I could generate arbitrarily good noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;scipy/numpy&lt;/code&gt; right now for exploring the model, but I'll probably be reimplementing in C.&lt;/p&gt;&#xA;" OwnerUserId="14644" LastEditorUserId="10134" LastEditDate="2015-02-25T11:29:08.130" LastActivityDate="2021-03-18T09:17:27.140" Title="Algorithm for high quality 1/f noise?" Tags="|scipy|stochastic|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="18989" PostTypeId="2" ParentId="18987" CreationDate="2015-02-25T02:50:03.543" Score="9" Body="&lt;p&gt;You can generate a noise sequence with whatever noise spectrum you want (including $1/f$, also known as pink noise) by generating the noise coefficients in spectral space. The magnitudes of the coefficients should be chosen to give the desired spectrum and the phases should be chosen randomly. You then simply perform an inverse Fourier transform to give the sequence values. The following algorithm will generate 1/f noise:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine the number of points, $n$, and the length, $T$, of your sequence. This also determines the spectral space with wave numbers $-k_{\max}$ to $k_{\max}$ which correspond to frequency values $f_k=kT/(2\pi)$.&lt;/li&gt;&#xA;&lt;li&gt;Set the magnitudes of your spectral coefficients: $C_k = 1/\left|f_k\right|$. Set $C_0 = 0$ to give zero mean to the noise sequence.&lt;/li&gt;&#xA;&lt;li&gt;Set the phases of the spectral coefficients to random values, ensure symmetry if you want real valued noise.&lt;br&gt;&#xA;for $k=0..k_{\max}$&lt;br&gt;&#xA;$~~~\phi_k=\left(~\operatorname{rand\in[0,2\pi)}~\right)$,&lt;br&gt;&#xA;$~~~C_k = C_ke^{i\varphi_k}$,&lt;br&gt;&#xA;$~~~C_{-k} = C_{-k}e^{-i\varphi_k}$&lt;/li&gt;&#xA;&lt;li&gt;Take an inverse Fourier transform of the spectral coeficients to get your noise sequence, $\{y_i\} = \operatorname{ifft}(\{C_k\})$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you want higher or lower noise power you can simply multiply the resulting sequence by a constant (or multiply all the $C_k$ values by a constant, it's the same effect).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By definition, noise generated in this way has &lt;em&gt;exactly&lt;/em&gt; the specified spectrum since you are creating the noise by specifying its spectrum.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-02-25T13:46:57.477" LastActivityDate="2015-02-25T13:46:57.477" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="18990" PostTypeId="1" CreationDate="2015-02-25T07:11:23.883" Score="4" ViewCount="214" Body="&lt;p&gt;Consider the integral &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$2\pi T = -\frac{1}{2} \int_0^{2\pi} A\frac{\partial B}{\partial \xi} \ d\xi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $$A= \sum_{-N}^N i \ sign(n) \ B_n e^{-in\xi}; \quad \quad B= \sum_{-N}^N \ B_n e^{-in\xi}.$$ These expansions imply &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$2\pi T = \sum |k|B_kB_{-k}$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore,&#xA; $$B_k = \frac{i}{k}\left(-\dot{Y}_k + \sum_{l+j=k}\dot{Y}_lY_j\ j (sign(l)-sign(j))\right),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with dots indicating temporal derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the canonical variables of the system end up being the complex valued variables $Y_{\pm 1},Y_{\pm 2},..,Y_{\pm N}$, and so the Euler-Lagrange equations involve terms like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (1) \quad \quad \frac{\partial T}{\partial Y_m}; \quad \quad \frac{d}{dt}\left(\frac{\partial T}{\partial \dot{Y}_m}\right).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, to derive the governing equations, I must explicitly expand terms like these and write them as some function $F(Y_{\pm 1},Y_{\pm 2},..., \dot{Y}_{\pm 1},...)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the way I'm currently computing this term is to note&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T = \sum_{k,l} Q_{kl} \dot{Y}_k\dot{Y}_l,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $$Q_{kl} = \frac{1}{4}\sum_{m=1}^N P_{mk}P_{-ml}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $$P_{mn} = (sign(m-n)-sign(n))Y_{m-n} -\frac{1}{2} Y_mY_{-n}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To set up my governing equations via the Euler-Lagrange equations, I need to compute terms like the ones mentioned above in (1). They both lead to the need to calculate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial Q_{kl}}{\partial Y_n} = \sum_{m=1}^N \left(\frac{\partial P_{mk}}{\partial Y_n}P_{-ml}+P_{mk}\frac{\partial P_{-ml}}{\partial Y_n}\right).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to compute this for all n, so I end up needing to perform a series of 2N+1, $((2N+1),N)\times (N,2N+1))$ matrix multiplications. For large $N$, this computation is absolutely slaughtering my run times, and generally take up +80% of the run time for each time step (depending on the size of N). &lt;strong&gt;This is my problem&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I've been trying to overcome this is to focus my attention on using the best optimized BLAS libraries to compute the matrix multiplications, and exploiting symmetries of the matrices to try to reduce computation time. However, I think I've milked this line of reasoning for all it's worth without getting too deep into the weeds, so I would like to atleast think about alternatives. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am very new to numerical methods, and might be completely off base, but it seems like there should be a way to manipulate the FFT here, to avoid doing these matrix multiplications as these Cauchy products are similar to convolutions (ie the pseudo spectral method). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't see how one can explicitly do this, especially as I need to differential these products with respect to the dependent variables of the system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has any insight, it's greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8021" LastEditorUserId="8021" LastEditDate="2015-02-26T05:32:31.403" LastActivityDate="2015-02-26T05:32:31.403" Title="Alternative to (costly) matrix multiplication" Tags="|matrix|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="18991" PostTypeId="1" CreationDate="2015-02-25T11:21:15.247" Score="1" ViewCount="92" Body="&lt;p&gt;Iam basically trying to fit a curve similar to that shown in fig (a) with 7 straight line segments (ie. 6 knots) as shown in fig (b) and I need to get the optimum knot positions. This is a tripartite plot in which there are 4 axes (vertical , one inclined at 45 degrees with horizontal, other inclined at 135 degrees with the horizontal ) and the horizontal axis and all are in log scale. Constraints are 3 rd line should be parallel to 45 degree lines, 4 th line should be parallel to the horizontal, 5th line should be parallel to 135 degree lines as shown in the attachment. –  Aiswarya 11 mins ago   .&lt;img src=&quot;https://i.stack.imgur.com/OUOHT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="14647" LastEditorUserId="14647" LastEditDate="2015-02-26T08:01:26.737" LastActivityDate="2015-02-26T08:01:26.737" Title="Piecewise linear fitting of a curve in loglog plot" Tags="|approximation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18994" PostTypeId="1" CreationDate="2015-02-25T15:24:42.953" Score="3" ViewCount="121" Body="&lt;p&gt;I wish to create a FEM mesh to solve an inverse elasticity problem, for an irregular domain. This domain is given by a medical image, so it is discretised and each square on the grid has one scalar value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the discretized and band-limited nature of the domain can inform my choice of FEM mesh, but have not seen this particular case discussed in my source books.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the 1D case. I know from Lagrange that for a set of regular samples of length n I can construct a polynomial of order n−1. This is one way of expressing the bandwith and information limitations of the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My gut tells me that for this reason, I can mesh with squares at the same resolution as the image, using linear shape functions, and capture all the information available in the image. But I lack the math chops to derive this result. Is it correct?&lt;/p&gt;&#xA;" OwnerUserId="3657" LastActivityDate="2015-02-25T15:24:42.953" Title="Creating FEM mesh for image region — what is the most suitable shape function?" Tags="|finite-element|mesh|image-processing|inverse-problem|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18995" PostTypeId="2" ParentId="135" CreationDate="2015-02-26T04:10:33.117" Score="2" Body="&lt;p&gt;I once wrote a small Python script to detect the point group symmetry for a molecule.  If you're interested, please see &lt;a href=&quot;https://github.com/sunqm/pyscf/blob/master/pyscf/symm/geom.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sunqm/pyscf/blob/master/pyscf/symm/geom.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14655" LastEditorUserId="20688" LastEditDate="2021-09-09T14:47:52.370" LastActivityDate="2021-09-09T14:47:52.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18996" PostTypeId="2" ParentId="16260" CreationDate="2015-02-26T09:51:56.427" Score="2" Body="&lt;p&gt;Another option not mentioned yet is NetGen:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hpfem.jku.at/netgen/&quot; rel=&quot;nofollow&quot;&gt;http://www.hpfem.jku.at/netgen/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://sourceforge.net/projects/netgen-mesher/&quot; rel=&quot;nofollow&quot;&gt;http://sourceforge.net/projects/netgen-mesher/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It supports CSG (constructive solid geometry) and can output 2D and 3D meshes. A number of output formats are supported and described in the documentation PDF.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-02-26T09:51:56.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18997" PostTypeId="2" ParentId="18943" CreationDate="2015-02-26T10:19:06.470" Score="3" Body="&lt;p&gt;This has nothing to do with banded/non-banded matrices: since you use the &lt;code&gt;\&lt;/code&gt; operator to solve your system, you will always obtain an exact solution to your discretized problem (up to roundoff effects). The question is therefore how well the solution of the discrete problem approximates the exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that you use a standard FEM with Courant elements (piecewise linear and continuous). We know, by Céa's Lemma, that the discrete solution will be quasi-optimal in the sense that it is only by a constant factor worse than the best possible approximation of the solution that your discretization space allows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what you really have to ask is how good the approximation properties of your discretization space are. The typical approximation properties derived for the FEM usually assume that the mesh is quasi-uniform and shape-regular. In other words, the elements are roughly the same size and none of them are extremely stretched in one dimension. If this assumption is violated, then we must expect a degradation of the approximation quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend you to read up on these topics in any standard textbook or lecture notes on numerical methods for elliptic PDEs. Please ask if you need pointers on particular sources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A practical concern is also if the two discretizations you use have the same or at least comparable number of degrees of freedom.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastEditorUserId="14598" LastEditDate="2015-04-24T12:15:16.737" LastActivityDate="2015-04-24T12:15:16.737" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="18998" PostTypeId="2" ParentId="11290" CreationDate="2015-02-26T11:10:40.380" Score="1" Body="&lt;p&gt;Your target size of 10,000 degrees of freedom should be easy to meet without embedding an entire FEM library (which may anyway be hard to do, as others have pointed out), and in much less than a second. Assuming that solving the system is your current bottleneck and not assembling it (check this first), I would recommend you to use any of the existing and free direct solvers for large sparse systems. Some pointers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot; rel=&quot;nofollow&quot;&gt;SuperLU&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://developer.nvidia.com/cholmod&quot; rel=&quot;nofollow&quot;&gt;CHOLMOD&lt;/a&gt; (this even has GPU support, though you may not need that)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://mumps.enseeiht.fr/&quot; rel=&quot;nofollow&quot;&gt;MUMPS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, if you use, say, the &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt; linear algebra library, it comes with some sparse direct solvers already built-in, so you wouldn't even need to link to an external library. These may be a bit slower than the specialized libraries linked to above, but easier and faster to get started with.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-02-26T11:10:40.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="18999" PostTypeId="1" AcceptedAnswerId="19010" CreationDate="2015-02-26T13:01:10.513" Score="2" ViewCount="586" Body="&lt;p&gt;I am trying to animate a dampened pendulum using RK4 for a highschool project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations that describe the dampened system are as follows:&#xA;(from &lt;a href=&quot;http://www.maths.tcd.ie/~smurray/Pendulumwriteup.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.maths.tcd.ie/~smurray/Pendulumwriteup.pdf&lt;/a&gt;)&#xA;$$\begin{aligned}&#xA; \frac{d\theta}{dt}&amp;amp;=\omega,\\&#xA; \frac{d\omega}{dt}&amp;amp;=-\beta^2\sin\theta-k\omega+A\cos\Omega.&#xA;\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $B$ is a constant, $k$ is the coefficient of damping, $A$ is the driving amplitude and $\Omega$ is the driving frequency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I am getting an increasing amplitude on each swing, varying the coefficient of damping does not change this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe my problem is either my programming is wrong, or I have the wrong idea of what the driving amplitude and the driving frequency are.&lt;br&gt;&#xA;Ideally, I want the program to be able to showcase simple harmonic motion (when $k = 0$) and also allow the user to investigate dampening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my first time attempting to use the Runge Kutta method and I have most likely incorrectly implemented it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is written in Visual Basic.net 2010 and is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Public Class Form1&#xA;&#xA;Dim l As Decimal = 1 'Length of rod (1m)&#xA;Dim g As Decimal = 9.81 'Gravity&#xA;Dim w As Decimal = 0 ' Angular Velocity&#xA;Dim initheta As Decimal = -Math.PI / 2 'Initial Theta&#xA;Dim theta As Decimal = -Math.PI / 2 'Theta (This one changes for the simulation)&#xA;Dim t As Decimal = 0 'Current time of the simulation&#xA;Dim h As Decimal = 0.01 'Time step&#xA;Dim b As Decimal = Math.Sqrt(g / l) 'Constant used in the function for dw/dt&#xA;Dim k As Decimal = 0 'Coefficient of Damping&#xA;Dim initialx = l * Math.Sin(initheta) 'Initial Amplitude of the pendulum&#xA;&#xA;Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load&#xA;End Sub&#xA;&#xA;'Function for dw/dt&#xA;Public Function f(ByRef the As Decimal, ByRef omega As Decimal, ByRef time As Decimal)&#xA;&#xA;    Return ((-b ^ 2) * Math.Sin(the)) - (k * omega) + (initheta * Math.Cos(omega * time))&#xA;End Function&#xA;&#xA;Public Function y(ByRef the As Decimal, ByRef omega As Decimal, ByRef time As Decimal)&#xA;    Return omega&#xA;End Function&#xA;&#xA;&#xA;Dim k1, k2, k3, k4, l1, l2, l3, l4 As Decimal 'Initialising RK4 variables&#xA;&#xA;&#xA;Public Sub RK4Solve(ByRef The As Decimal, ByRef Ome As Decimal, ByRef h As Decimal)&#xA;&#xA;    l1 = y(The, Ome, t)&#xA;    k1 = f(The, Ome, t)&#xA;    l2 = y(The + (0.5 * h * l1), Ome + (0.5 * h * k1), t + (0.5 * h))&#xA;    k2 = f(The + (0.5 * h * l1), Ome + (0.5 * h * k1), t + (0.5 * h))&#xA;    l3 = y(The + (0.5 * h * l2), Ome + (0.5 * h * k2), t + (0.5 * h))&#xA;    k3 = f(The + (0.5 * h * l2), Ome + (0.5 * h * k2), t + (0.5 * h))&#xA;    l4 = y(The + (h * l3), Ome + (h * k3), t + h)&#xA;    k4 = f(The + (h * l3), Ome + (h * k3), t + h)&#xA;&#xA;    'Setting next step of variables&#xA;    The = The + (h / 6 * (l1 + (2 * l2) + (2 * l3) + l4))&#xA;    Ome = Ome + (h / 6 * (k1 + (2 * k2) + (2 * k3) + k4))&#xA;    t += h&#xA;End Sub&#xA;&#xA;&#xA;'Timer ticking every 0.1s&#xA;'Time step is 0.01s to increase accuracy of results for testing&#xA;Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick&#xA;    ComboBox1.Items.Add(theta) 'Adding theta to a drop down box to test data&#xA;    RK4Solve(theta, w, h)&#xA;End Sub&#xA;&#xA;Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click&#xA;    Timer1.Enabled = False&#xA;End Sub&#xA;End Class&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is a picture of the Dis, Vel and Acc graphs (in that order)&#xA;&lt;img src=&quot;https://i.stack.imgur.com/kEtJ3.png&quot; alt=&quot;Graphs&quot;&gt;&#xA;&lt;br&gt;&#xA;As you can see, the acceleration falls apart during the simulation (forgive my lack of scientific terms)&#xA;Why does this happen? (I will also update my code with my new RK4 implementation)&lt;/p&gt;&#xA;" OwnerUserId="12044" LastEditorUserId="12044" LastEditDate="2015-02-27T09:43:31.667" LastActivityDate="2015-02-27T12:40:04.957" Title="Simulating a Simple Pendulum - Increasing amplitude on each swing?" Tags="|simulation|runge-kutta|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19000" PostTypeId="2" ParentId="18999" CreationDate="2015-02-26T13:31:32.187" Score="1" Body="&lt;p&gt;You are multiplying $l$'s and $k$'s by too many $h$'s. In particular look out for expressions like $l_2 = h l_1/2+\Omega$, where $\Omega$ is $O(1)$ but $l_1$ is $O(h)$ and $l_2$ is $O(h^2)+O(1)$. This is not a correct implementation of RK4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of RK methods is that you have terms like&#xA;$$ k_2 = f(t_0 + c_2h, y_0 + h a_{21}k_1), $$&#xA;but $k_2$ has the magnitude of the rhs of the equation, it is an &lt;em&gt;approximation&lt;/em&gt; to the rhs at some point, not like in your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, consider asking on &lt;a href=&quot;http://codereview.stackexchange.com&quot;&gt;http://codereview.stackexchange.com&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-02-26T13:31:32.187" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19001" PostTypeId="1" CreationDate="2015-02-26T15:31:21.897" Score="1" ViewCount="195" Body="&lt;p&gt;So I implemented the projection method as in Chorin (1969) for the solution of the Navier-Stokes Equations with the Boussinesq Approximation, using a 2nd Order accurate FD scheme. The primitive variables are arranged using a MAC scheme. Basically, what I want is to be able to generate a file (or files) to see the solution on a rendering software (a.k.a, Tecplot, paraView, etc). How can I arrange the data? I mean, from a pseudo-code kind of view.&lt;/p&gt;&#xA;" OwnerUserId="14659" LastActivityDate="2015-02-26T15:31:21.897" Title="Visualization of solution for a MAC grid" Tags="|finite-difference|visualization|navier-stokes|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19002" PostTypeId="2" ParentId="18982" CreationDate="2015-02-26T16:00:15.430" Score="3" Body="&lt;p&gt;I think you are going to have to choose what your primary criteria is: performance, modularity, or portability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took a highly optimized Fortran 90 code and tried to modularize it and make it more object-oriented, and the performance became 3X worse in the object-oriented version. I found the best I could do to have the best of both worlds was to make OO data structures that were all pointers pointing to the original non-OO data structures. Fortran uses highly efficient array strides which makes it very fast for doing linear algebra computations.  The problem is that Object-oriented (OO) approaches destroy the optimal cache coherence you get from loops in non-OO approaches.  There was a pretty good discussion on that &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1731/what-modern-oop-features-should-a-computational-scientist-use&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are primarily concerned about performance, I would use a combined Python / Fortran 90 approach and connect them via the f2py module. Fortran 90 in general is faster than C or C++ especially for linear algebra (although depends on the platform), and has good support for many of the numerical libs such as OpenMPI, Lapack, etc.  I would do all the compute intensive (e.g. linear algebra) stuff in Fortran 90, and do all the coding intensive stuff in Python (e.g. vtk).  There is an example of solving Laplace's equation $\nabla^2u=0$ using a combined Fortran 90 / Python approach via f2py &lt;a href=&quot;http://wiki.scipy.org/PerformancePython#head-6a1aacc7a6b8defdea38dc1f3e664ef0d1683d83&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  Python has one of the largest scientific computing communities for a modern high-level language, and has good support for a number of modules, especially numpy, scipy, and also the vtk module. There is an example of reading unstructured grid via Python's vtk module &lt;a href=&quot;http://www.vtk.org/Wiki/VTK/Examples/Python/vtkUnstructuredGridReader&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you want to be more modular and/or portable, you may want to look into using Java, but you will have to sacrifice performance.  I haven't been too impressed with Java-based simulations especially in terms of performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning reference material, I don't think there are any books covering the software architecture of these kinds of problems. However, you may want to checkout this paper titled &quot;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045794908001387&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using design patterns in object-oriented finite element programming&lt;/a&gt;&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have any experience with it, but there is a rapidly growing interest in coding these kinds of problems in Julia.  So, you might check out this as well: &lt;a href=&quot;http://www.codeproject.com/Articles/579983/Finite-Element-programming-in-Julia&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.codeproject.com/Articles/579983/Finite-Element-programming-in-Julia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4873" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-02-26T16:00:15.430" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19003" PostTypeId="2" ParentId="11290" CreationDate="2015-02-26T18:29:15.447" Score="0" Body="&lt;p&gt;I'll suggest an FE library that is less well known&#xA;than those previously mentioned. It is not as &#xA;sophisticated as those but may satisfy your&#xA;requirements. It is reasonably well documented and&#xA;because it is simple, it is also easier to use, modify, and understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The library/code is FeaTure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.utwente.nl/ctw/tm/research/NSM/software/feature/&quot; rel=&quot;nofollow&quot;&gt;http://www.utwente.nl/ctw/tm/research/NSM/software/feature/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say you are more interested in speed than accuracy.&#xA;However, as a long-time stress analyst, I can say that it&#xA;is not straightforward to represent a solid continuum (like&#xA;a concrete wall) as a collection of truss elements. So&#xA;I think you are on the right track in looking for&#xA;an FE library that includes continuum elasticity elements&#xA;rather than trying to build your own from low-level linear&#xA;algebra components.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-02-26T18:29:15.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19004" PostTypeId="1" CreationDate="2015-02-26T19:16:12.927" Score="3" ViewCount="1068" Body="&lt;p&gt;Recently I tried to compare results for 1D direct convolution and convolution via FFT. I expected to get absolutely the same result, however I faced with a problem that results are different, especially near the left boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y9Jhn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PVYCB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FFT convolution should be normalized, however it doesn't change the difference near the left boundary. As I understand this difference appears due to the fact that FFT provides circular convolution, while the direct convolution is linear. I have found that it can be fixed by Overlap–add method, however as I understand it should be used when curve and kernel have the different length, but in my case both of them have equal length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mat = Table[0, {x, 512}, {y, 512}];&#xA;SetAttributes[FillMatrix, HoldAll]&#xA;FillMatrix[Kep_, Ktrans_, mat_] := &#xA;  Do[Do[mat[[j, i]] = Exp[-Kep (j - i + 1)], {i, 1, j, 1}], {j, 1, &#xA;    Length[mat], 1}];&#xA;&#xA;FillArray[Kep_, Ktrans_, mat_] := &#xA; Module[{curve = {}}, &#xA;  Do[curve = Append[curve, Exp[-Kep*i]], {i, 1, Length[mat], 1}]; &#xA;  curve]&#xA;&#xA;AIF = {};&#xA;SetAttributes[FillAIF, HoldAll]&#xA;FillAIF[AIF_, n_] := &#xA; Do[AIF = Append[AIF, PDF[GammaDistribution[3, 2], i*0.1]], {i, 1, n, &#xA;   1}]&#xA;&#xA;FillAIF[AIF, Length[mat]]&#xA;&#xA;FillMatrix[0.005, 1, mat]&#xA;conv = mat.AIF;&#xA;&#xA;AIFFourier = Fourier[AIF];&#xA;kernelCurve = FillArray[0.005, 1, mat];&#xA;kernelCurveFourier = Fourier[kernelCurve];&#xA;&#xA;resSource = InverseFourier[AIFFourier*kernelCurveFourier];&#xA;ListPlot[resSource, Joined -&amp;gt; True]&#xA;ListPlot[conv, Joined -&amp;gt; True]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Question: Why does this difference appear and how can it be fixed?&lt;/p&gt;&#xA;" OwnerUserId="14663" LastActivityDate="2015-02-26T20:04:15.377" Title="FFT convolution vs direct convolution" Tags="|fourier-transform|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19005" PostTypeId="2" ParentId="19004" CreationDate="2015-02-26T20:04:15.377" Score="4" Body="&lt;p&gt;A brute-force approach is to pad the signal and the kernel with enough zeros that there would be no aliasing when doing circular convolution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AIFFourier = Fourier[PadRight[AIF, 1024]];&#xA;kernelCurve = FillArray[0.005, 1, mat];&#xA;kernelCurveFourier = Fourier[PadRight[kernelCurve, 1024]];&#xA;&#xA;resSource = InverseFourier[AIFFourier*kernelCurveFourier];&#xA;ListPlot[resSource[[1 ;; 512]], Joined -&amp;gt; True]&#xA;ListPlot[conv, Joined -&amp;gt; True]&#xA;StandardDeviation[resSource[[1 ;; 512]]/conv]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$$ \Longrightarrow 2.37134\times10^{-15} $$&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-02-26T20:04:15.377" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19006" PostTypeId="1" AcceptedAnswerId="19068" CreationDate="2015-02-27T00:18:39.377" Score="7" ViewCount="640" Body="&lt;p&gt;I tried asking a similar question in SE.Physics, and I got some information regarding the abstract side of this, but I figured I should post here to get more complete information about the numerical benefits of variational formalisms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I am able derive a functional representation for &lt;em&gt;any&lt;/em&gt; dynamical system (dissipative, nonlinear, fractional, PDE, ODE, discontinuous, etc), why would such a result or capability be useful? What are some practical consequences?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, what are functionals/variational formalisms used for in practice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind, the functionals I'm referring to might be of non-standard form (i.e, non-conservative systems), for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any system with that is a potential or conservative will have a functional representation as:&#xA;$$&#xA;F[\mathbf{x}]=\int^{t}_0\left(\frac{1}{2}m\dot{\mathbf{x}}(\tau)^2-V(\mathbf{x}(\tau))\right)\,\text{d}\tau&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where taking the first variation of this functional yields the dynamics of the system, along with a condition that effectively states that the initial configuration should be similar to the final configuration (variation at the boundaries is zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;But&lt;/em&gt;, you can construct other functionals for non-conservative systems, such as this convolutional functional:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  &#xA;F[\mathbf{x}]=\frac{1}{2}[\mathbf{x}^{\text{T}} * D(\mathbf{x})]-\frac{1}{2}[\mathbf{x}^{\text{T}} * \mathbf{Ax}]-\frac{1}{2}\mathbf{x}'(0)\mathbf{x}(t)&#xA;$$&#xA;With $\mathbf{A}$ symmetric and $\mathbf{x}(0)$ being the initial condition, and:&#xA;$$&#xA;[\mathbf{f}^{\text{T}} * \mathbf{g}]=\int^{t}_0 \mathbf{f}^{\text{T}}(t-\tau)\mathbf{g}(\tau)\,\text{d}\tau&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take the first variation and assume &lt;strong&gt;only&lt;/strong&gt; that the initial variation is zero, the functional is stationary with respect to:&#xA;$$&#xA;\frac{d\mathbf{x}(t)}{dt}= \mathbf{Ax}(t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Point being that one should also consider the implications of functionals which are not inner product based.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that these functionals are useful in the context of FEM (but how?), but I'm also interested in whether or not they may be useful in other contexts (parameter estimation, simulation, data assimilation, etc).&lt;/p&gt;&#xA;" OwnerUserId="4509" LastEditorUserId="4509" LastEditDate="2015-03-05T00:52:43.317" LastActivityDate="2015-07-31T00:39:11.503" Title="Why are functional representations of systems important in numerical applications?" Tags="|pde|finite-element|ode|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19007" PostTypeId="1" AcceptedAnswerId="19008" CreationDate="2015-02-27T01:59:01.533" Score="13" ViewCount="1139" Body="&lt;p&gt;I'd like to solve some PDEs on manifolds, say for example an elliptic equation on a sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where do I start? I'd like to find something that use preexisting code/libraries in 2d , nothing &lt;a href=&quot;http://www.ima.umn.edu/~arnold//papers/acta.pdf&quot;&gt;so&lt;/a&gt; fancy (for the moment) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Added Later :&lt;/strong&gt; Articles and reports are welcome.&lt;/p&gt;&#xA;" OwnerUserId="14566" LastEditorUserId="14566" LastEditDate="2015-02-27T14:28:58.913" LastActivityDate="2015-02-27T14:43:17.120" Title="Finite elements on manifold" Tags="|pde|finite-element|reference-request|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19008" PostTypeId="2" ParentId="19007" CreationDate="2015-02-27T03:11:59.747" Score="8" Body="&lt;p&gt;I think you start by looking at something like &lt;a href=&quot;http://fenicsproject.org/&quot;&gt;FEniCS&lt;/a&gt;. Marie Rognes has a &lt;a href=&quot;http://fenicsproject.org/pub/workshops/fenics13/slides/Rognes.pdf&quot;&gt;presentation with code examples&lt;/a&gt; and a &lt;a href=&quot;http://www.geosci-model-dev.net/6/2099/2013/gmd-6-2099-2013.pdf&quot;&gt;paper discussing the theory and implementation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://libmesh.github.io/&quot;&gt;libMesh&lt;/a&gt; &lt;a href=&quot;http://sourceforge.net/p/libmesh/mailman/message/22824303/&quot;&gt;is supposed to be able to do something similar&lt;/a&gt; for 2-manifolds in 3-space, and so is &lt;a href=&quot;http://www.dealii.org/&quot;&gt;deal.II&lt;/a&gt;, judging from &lt;a href=&quot;http://www.dealii.org/8.0.0/reports/codimension-one/desimone-heltai-manigrasso.pdf&quot;&gt;this manuscript&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Developers of deal.II and FEniCS answer questions on SciComp, and would be able to provide more detailed answers; I'm not sure if libMesh developers also view the site, but I think we have a few libMesh users that answer questions.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-02-27T03:11:59.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19010" PostTypeId="2" ParentId="18999" CreationDate="2015-02-27T12:40:04.957" Score="1" Body="&lt;p&gt;It looks like your revised code still has a problem. What is this term &lt;code&gt;(initheta * Math.Cos(omega * time))&lt;/code&gt; in your function f() supposed to be? It's like a forcing term that has amplitude equal to the initial angle and a time-dependent frequency equal to the instantaneous angular frequency. Shouldn't it be &lt;code&gt;(A * Math.Cos(Omega * time))&lt;/code&gt; and you have to choose &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;Omega&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the (incorrect) term you have now and your initial conditions this forcing will initially act in opposition to the motion, slowing the pendulum. Weird things might happen eventually since the forcing frequency is constantly changing.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-02-27T12:40:04.957" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19011" PostTypeId="2" ParentId="19007" CreationDate="2015-02-27T14:26:28.250" Score="4" Body="&lt;p&gt;As Geoff already points out, deal.II (&lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;nofollow&quot;&gt;http://www.dealii.org&lt;/a&gt;) does support solving equations on surfaces. There is even a tutorial program, &lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/step_34.html&quot; rel=&quot;nofollow&quot;&gt;step-34&lt;/a&gt;, that demonstrates how one does so -- although it shows how to solve an integral equation on the sphere, not a differential equation. The main reason why it shows something more complicated than a differential equation is because solving differential equations on the sphere works exactly the same way as it does on a planar geometry, which is demonstrated in the previous 33 tutorial programs :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-02-27T14:26:28.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19012" PostTypeId="2" ParentId="19007" CreationDate="2015-02-27T14:43:17.120" Score="3" Body="&lt;p&gt;Besides the following survey article &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gerhard Dziuk and Charles M. Elliott (2013). &lt;a href=&quot;https://homepages.warwick.ac.uk/staff/C.M.Elliott/DziEll13a.pdf&quot; rel=&quot;nofollow&quot;&gt;Finite element methods for surface PDEs&lt;/a&gt;. Acta Numerica, 22, pp 289­396 doi:10.1017/S0962492913000056,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;there's &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Michael Holst (2001). &lt;a href=&quot;http://arxiv.org/pdf/1001.1367v1.pdf&quot; rel=&quot;nofollow&quot;&gt;Adaptive numerical treatment of elliptic systems on manifolds&lt;/a&gt;. Advances in Computational Mathematics, 15, pp. 139-191,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which describes a software package for an adaptive finite element method on surfaces. The package itself can be downloaded from &lt;a href=&quot;http://fetk.org/codes/mc/&quot; rel=&quot;nofollow&quot;&gt;http://fetk.org/codes/mc/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-02-27T14:43:17.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19013" PostTypeId="1" CreationDate="2015-02-27T16:47:17.440" Score="1" ViewCount="550" Body="&lt;p&gt;is it legal here to ask this problem?&#xA;I need someone help. &#xA;I run mcnpx. I try to calculate f4:n using nps:1000. But, it stop directly before culculate f4. There is comment: stack trace terminated abnormally. And there is warning: cross-section file xsdir and bertin does not exist. &#xA;Anyone can help what the problem in my work? I appreciated every suggestion. Thanks. Rose&lt;/p&gt;&#xA;" OwnerUserId="14671" LastActivityDate="2015-02-27T16:47:17.440" Title="MCNPX for nuclear simulation" Tags="|algorithms|" AnswerCount="0" CommentCount="4" ClosedDate="2015-02-27T22:28:17.657" ContentLicense="CC BY-SA 3.0" />
  <row Id="19015" PostTypeId="1" AcceptedAnswerId="19017" CreationDate="2015-02-28T15:59:38.470" Score="2" ViewCount="1232" Body="&lt;p&gt;I am solving an unconstrained convex optimization problem, which can easily have a million variables. I am trying to get a working system with a toy problem of around 200 variables. I am noticing that the Newton steps are becoming of very small magnitude, even though the gradient is still not as close to zero as desired. I then tried simple gradient descent with line search and I am getting extremely small step sizes after a few iterations, even though the gradient is not that small. What could be happening?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update 1: Thanks to Prof. Borchers and Prof. Bangerth for sharing your knowledge. I need to investigate further along these lines, to reach a conclusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update 2: Indeed there was a bug in the implementation of the gradient. Thanks for your inputs for verifying the gradient and for a good rule of thumb to terminate optimization problems.&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2015-03-01T21:57:02.370" LastActivityDate="2015-03-01T21:57:02.370" Title="Doubt regarding stopping criterion for Newton method" Tags="|linear-algebra|optimization|numerical-analysis|convex-optimization|newton-method|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19016" PostTypeId="1" CreationDate="2015-03-01T00:23:03.690" Score="0" ViewCount="2605" Body="&lt;p&gt;I'm getting confused about implementing the Gonzalez algorithm for k-center clustering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm is :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;let $S$ be a data set&lt;/li&gt;&#xA;&lt;li&gt;Choose $z_1 \in S$ arbitrarily.  Number $z$ is center. (In general let $Z_i = \{z_1, \dots, z_i\}$).&lt;/li&gt;&#xA;&lt;li&gt;for i = 2 to k do &#xA;&lt;ul&gt;&#xA;&lt;li&gt;Set $z_i = \operatorname{argmax}_{x\in S}  d(x, \phi (Z_{i−1})(x))$ &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The formula of  $\phi(Z)(x)= \operatorname{argmin} _{z\in Z} d(x, z)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should implement this in Python. There used to be functions for clustering in Python, is there any function in Python for this algorithm?&lt;/p&gt;&#xA;" OwnerUserId="14682" LastEditorDisplayName="user10435" LastEditDate="2015-03-14T04:11:54.267" LastActivityDate="2020-11-11T21:55:00.643" Title="Gonzalez algorithm" Tags="|python|data-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19017" PostTypeId="2" ParentId="19015" CreationDate="2015-03-01T01:33:32.720" Score="4" Body="&lt;p&gt;If you converge, you would expect the steps to get small. Ideally, a step $\delta x_k$ in an optimization algorithm would go from the current iterate $x_k$ to the exact solution $x^\ast$, so $\|\delta x_k\| \approx \|x_k-x^\ast\|$ which is, as you approach the solution, going to be small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you say that $\delta x_k$ is getting small even though the gradient is still large. That, of course, depends on how you define &quot;large&quot;. Is $10^3$ small or large? By itself, there is no way to say this. It depends on the units of the thing, as well as on what you compare with. Compared with the typical size of objects we have around us, $10^3$ light years is clearly large. $10^3$ nanometers is pretty small, however. But if you're a cosmologist, then $10^3$ light years is small. And if you're looking at atomic distances, then $10^3$ nanometers is large. In other words, you need to investigate what exactly it means for the gradient to be large in your case, and whether a number that is, for example, not on the order of $10^{-7}$ but rather $10^7$ really means that you're still far from the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of optimization problems, you need to ask &#xA;&quot;what is small?&quot; when you look at the gradient. One way to approach this is to ask &quot;what is a typical size for the gradient?&quot;. To give an example, let us say that you have a spring-mass system for which you'd like to find the minimum energy position. Let's assume that the springs are all around 10cm long, then a typical &lt;em&gt;displacement&lt;/em&gt; of springs might be $\Delta x=1cm$. Choose two positions for the bodies and connecting springs that are approximately $\Delta x$ apart and evaluate the energies for these two positions to get a corresponding &quot;typical energy difference&quot; $\Delta E$. Then a &quot;typical size of the gradient&quot; would be $\Delta E/\Delta x$. If your optimization algorithm has produced a position for which the gradient is $\|g_k\| \le 10^{-3} \left|\Delta E/\Delta x \right|$, then you can say that you're converged.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2015-03-01T15:28:32.893" LastActivityDate="2015-03-01T15:28:32.893" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19018" PostTypeId="1" AcceptedAnswerId="19021" CreationDate="2015-03-01T01:54:01.257" Score="0" ViewCount="238" Body="&lt;p&gt;Has there been an attempt to &quot;explain&quot; Jupiter's Red Spot via &#xA;what is known as the &lt;em&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Standard_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;standard map&lt;/a&gt;&lt;/em&gt;&#xA;in chaos theory?&#xA;I cannot be the first to notice the visual similarities...&#xA;&lt;hr /&gt;&#xA;&amp;nbsp; &amp;nbsp; &amp;nbsp; &#xA;&lt;img src=&quot;https://i.stack.imgur.com/HxWqE.gif&quot; alt=&quot;StandardMap&quot;&gt;&#xA;&lt;br /&gt;&#xA;&amp;nbsp; &amp;nbsp; &amp;nbsp; &#xA;&lt;sup&gt;&#xA;Orbits of the Standard Map&#xA;$(x, y) \to (x + y, y - 0.971635 \sin (2px)/2p)$.&#xA;Image from &lt;a href=&quot;http://ckw.phys.ncku.edu.tw/public/pub/Notes/NonlinearPhysics/Web/Chaos/ChaosHypertext/hypertextbook.com/chaos/43.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;.&#xA;&lt;/sup&gt;&#xA;&lt;hr /&gt;&#xA;&amp;nbsp; &amp;nbsp; &amp;nbsp; &#xA;&lt;img src=&quot;https://i.stack.imgur.com/tkeda.jpg&quot; alt=&quot;RedSpot&quot;&gt;&#xA;&lt;br /&gt;&#xA;&amp;nbsp; &amp;nbsp; &amp;nbsp; &#xA;&lt;sup&gt;&#xA;Jupiter's Red Spot. Image from &lt;a href=&quot;http://www.davidreneke.com/why-jupiters-great-red-spot-hasnt-disappeared/#&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website&lt;/a&gt;.&#xA;&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr /&gt;&#xA;" OwnerUserId="673" LastEditorUserId="673" LastEditDate="2015-03-01T02:40:10.483" LastActivityDate="2015-03-01T03:24:16.257" Title="The &quot;standard map&quot; and Jupiter's red spot" Tags="|fluid-dynamics|chaotic-systems|" AnswerCount="2" CommentCount="1" ClosedDate="2015-03-01T16:39:39.990" ContentLicense="CC BY-SA 3.0" />
  <row Id="19019" PostTypeId="2" ParentId="19018" CreationDate="2015-03-01T02:55:48.890" Score="2" Body="&lt;p&gt;This question is of course slightly off-topic here, but in the interest of general education, it may be worth answering it anyway:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just because two things look similar does not imply that the underlying causes are related. In particular, it is worth pointing out that the picture of the standard map you show actually shows a cross-section of a three-dimensional process (two space and one time dimensions) and only shows points where a line crosses the section. On the other hand, the picture of Jupiter shows certain aspects of fluid flow in a three-dimensional space at a fixed time instant, projected onto a 2d camera observer. In other words, the pictures show entirely different things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you plotted the standard map not as a Poincare section but as lines in 3d space, and Jupiters red spot as a vector plot or flowlines in Jupiter's 3d atmosphere, the two pictures would look completely different. Your observation is, in other words, just another example of &quot;correlation is not causation&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-01T02:55:48.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19020" PostTypeId="1" AcceptedAnswerId="19022" CreationDate="2015-03-01T03:00:08.517" Score="11" ViewCount="10807" Body="&lt;p&gt;How can I replace the Euler method by Runge-Kutta 4th order to determine the free fall motion in not constant gravitional magnitude (eg. free fall from 10 000 km above ground)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I wrote simple integration by Euler method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while()&#xA;{&#xA;    v += getMagnitude(x) * dt;&#xA;    x += v * dt;&#xA;    time += dt;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;x variable means current position,&#xA;v means velocity,&#xA;getMagnitude(x) returns acceleration on x position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried implement RK4:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while()&#xA;{&#xA;    v += rk4(x, dt) * dt; // rk4() instead of getMagintude()&#xA;    x += v * dt;&#xA;    time += dt;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where rk4() function body is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inline double rk4(double tx, double tdt)&#xA;{&#xA;   double k1 = getMagnitude(tx);&#xA;   double k2 = getMagnitude(tx + 0.5 * tdt * k1);&#xA;   double k3 = getMagnitude(tx + 0.5 * tdt * k2);&#xA;   double k4 = getMagnitude(tx + tdt * k3);&#xA;&#xA;   return (k1 + 2*k2 + 2*k3 + k4)/6.0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But something is wrong, because I am integrating only once using RK4 (acceleration). Integrating velocity using RK4 dosen't make sense because it's same as v * dt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could You tell me how to solve second order differential equations using Runge-Kutta integration? Should I implement RK4 by computing k1, l1, k2, l2 ... l4 coefficients? How can I do that?&lt;/p&gt;&#xA;" OwnerUserId="14684" LastEditorUserId="7733" LastEditDate="2015-03-01T04:42:29.343" LastActivityDate="2015-05-30T17:50:24.117" Title="Applying the Runge-Kutta method to second order ODEs" Tags="|numerical-analysis|c++|runge-kutta|integral-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19021" PostTypeId="2" ParentId="19018" CreationDate="2015-03-01T03:17:19.193" Score="3" Body="&lt;p&gt;There are many systems that exhibit (quasi-)periodic orbits surrounded by chaos without much (or any) similarities in the underlying physics. Dynamical systems theory is full of examples. To list a few:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The Standard Map is a 2+1 dimensional (2 space and 1 time) system that originates from the physics of a kicked rotator.&lt;/li&gt;&#xA;&lt;li&gt;Arnold-Beltrami-Childress (ABC) flow is a 3 dimensional system representing a spatially periodic steady-state solution of Euler's equations for inviscid fluid motion.&lt;br&gt;&#xA;&lt;sub&gt;Image from &lt;a href=&quot;http://www.wolfram.com/mathematica/new-in-9/advanced-hybrid-and-differential-algebraic-equations/poincare-sections.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wolfram&lt;/a&gt;&lt;/sub&gt;&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/em1U9.gif&quot; alt=&quot;Poincare Section for ABC Flow&quot;&gt;&lt;/li&gt;&#xA;&lt;li&gt;A periodic double gyre flow. This is a 2+1 dimensional system that is qualitatively similar to some ocean gyres, but is not based on any underlying physical model.&lt;br&gt;&#xA;&lt;sub&gt;Image from &lt;a href=&quot;http://www.birs.ca/workshops/2013/13w5089/files/rowley_banff.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this PDF presentation&lt;/a&gt;&lt;/sub&gt;&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/CVNz2.png&quot; alt=&quot;Double Gyre Poincare Section&quot;&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I doubt there is any real connection here aside from some similarities in the Lagrangian dynamics of the system. Examining the Lagrangian dynamics of a system may give many insights into transport and Lagrangian behavior, but typically gives little insight into the physics that give rise to the system in the first place. I see no reason to expect any correspondance between a 3d time-depended fluid dynamical system in a rotating reference frame (Jupiter's red spot) and a simple time-dependent kicked rotator.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-03-01T03:24:16.257" LastActivityDate="2015-03-01T03:24:16.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19022" PostTypeId="2" ParentId="19020" CreationDate="2015-03-01T04:29:40.167" Score="18" Body="&lt;p&gt;There seems to be quite a bit of confusion about how to apply multi-step (e.g. Runge-Kutta) methods to 2nd or higher order ODEs or systems of ODEs. The process is very simple once you understand it, but perhaps not obvious without a good explanation. The following method is the one I find simplest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the differential equation you would like to solve is $F = m\ddot{x}$. The first step is to write this second-order ODE as a system of first order ODEs. This is done as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\begin{array}{c}&#xA;\dot{x} \\&#xA;\dot{v}&#xA;\end{array}\right]&#xA;=&#xA;\left[\begin{array}{c}&#xA;v \\&#xA;F/m&#xA;\end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All equations in this system must be solved simultaneously, which is to say that you should &lt;em&gt;not&lt;/em&gt; advance $v$ and then advance $x$, they should both be advanced at the same time. In languages that support vector operations without loops this is easily done by making all the necessary terms in your code vectors of length 2. The function that computes the right hand side (the rate of change) of your ODE should return a vector of length 2, &lt;code&gt;k1&lt;/code&gt; to &lt;code&gt;k4&lt;/code&gt; should be vectors of length 2, and your state variable $(x,v)$ should be a vector of length 2. In MATLAB the necessary code for the time stepping can be written as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while (t&amp;lt;TMAX)&#xA;    k1 = RHS( t, X );&#xA;    k2 = RHS( t + dt / 2, X + dt / 2 * k1 );&#xA;    k3 = RHS( t + dt / 2, X + dt / 2 * k2 );&#xA;    k4 = RHS( t + dt, X + dt * k3 );&#xA;    X = X + dt / 6 * ( k1 + 2 * k2 + 2 * k3 + k4 );&#xA;    t = t + dt;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;X&lt;/code&gt;$=(x,v)$ and &lt;code&gt;RHS( t, X )&lt;/code&gt; returns a vector containing $(\dot{x}(t),\dot{v}(t))$. As you can see, by vectorizing things you don't even need to change the syntax of the RK4 code no matter how many equations are in your ODE system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;Unfortunately C++ does not natively support vector operations like this so you need to either use a vector library, use loops, or manually write out the separate parts.&lt;/strike&gt; In C++ you can use &lt;code&gt;std::valarray&lt;/code&gt; to achieve the same effect. Here's a simple working example with constant acceleration.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;valarray&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;const size_t NDIM = 2;&#xA;&#xA;typedef std::valarray&amp;lt;double&amp;gt; Vector;&#xA;&#xA;Vector RHS( const double t, const Vector X )&#xA;{&#xA;  // Right hand side of the ODE to solve, in this case:&#xA;  // d/dt(x) = v;&#xA;  // d/dt(v) = 1;&#xA;  Vector output(NDIM);&#xA;  output[0] = X[1];&#xA;  output[1] = 1;&#xA;  return output;&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;&#xA;  //initialize values&#xA;&#xA;  // State variable X is [position, velocity]&#xA;  double init[] = { 0., 0. };&#xA;  Vector X( init, NDIM );&#xA;&#xA;  double t = 0.;&#xA;  double tMax=5.;&#xA;  double dt = 0.1;&#xA;&#xA;  //time loop&#xA;  int nSteps = round( ( tMax - t ) / dt );&#xA;  for (int stepNumber = 1; stepNumber&amp;lt;=nSteps; ++stepNumber)&#xA;  {&#xA;&#xA;    Vector k1 = RHS( t, X );&#xA;    Vector k2 = RHS( t + dt / 2.0,  X + dt / 2.0 * k1 );&#xA;    Vector k3 = RHS( t + dt / 2.0, X + dt / 2.0 * k2 );&#xA;    Vector k4 = RHS( t + dt, X + dt * k3 );&#xA;&#xA;    X += dt / 6.0 * ( k1 + 2.0 * k2 + 2.0 * k3 + k4 );&#xA;    t += dt;&#xA;  }&#xA;  std::cout&amp;lt;&amp;lt;&quot;Final time: &quot;&amp;lt;&amp;lt;t&amp;lt;&amp;lt;std::endl;&#xA;  std::cout&amp;lt;&amp;lt;&quot;Final position: &quot;&amp;lt;&amp;lt;X[0]&amp;lt;&amp;lt;std::endl;&#xA;  std::cout&amp;lt;&amp;lt;&quot;Final velocity: &quot;&amp;lt;&amp;lt;X[1]&amp;lt;&amp;lt;std::endl;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-03-01T18:54:12.830" LastActivityDate="2015-03-01T18:54:12.830" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="19024" PostTypeId="1" CreationDate="2015-03-01T11:40:55.773" Score="1" ViewCount="66" Body="&lt;p&gt;I need a little help and advice with a project I want to do: the idea is to &quot;couple&quot; (I don't know whether I can call it like this) a conservative Navier-Stokes Solver (Fractional-step, 2nd order FDM) with itself. Roughtly speaking, I would like to split the domain in halves so to be able to solve one fluid on one side and another fluid on the other side. The interface is supposed to be fixed at all times, and for the moment I don't need to have different discretizations for each domain. The problem is that one of the fluids is around 1000 times lighter that the other fluid, so (I think) the transmission conditions for the stresses are not going to be as straightforward. Anyhoo, I would like someone to point me in the right track (state-of-the-art literature, jargon, &quot;big names&quot;,etc) and shoot some ideas if you like. Ciao &amp;amp; grazie&lt;/p&gt;&#xA;" OwnerUserId="14659" LastActivityDate="2015-03-01T11:40:55.773" Title="Help about Fluid-Fluid coupling techniques" Tags="|finite-difference|fluid-dynamics|navier-stokes|coupling|" AnswerCount="0" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19026" PostTypeId="1" AcceptedAnswerId="19034" CreationDate="2015-03-01T08:05:59.293" Score="1" ViewCount="528" Body="&lt;p&gt;I recently have been assigned a project to calculate the roots of a cubic polynomial. However, the issue is that the roots could be very big, but also extremely small. I've been trying to use bisection method to find the first root, and thus I start from +/- &lt;code&gt;realmax&lt;/code&gt;.&#xA;The problem I'm having is using Matlab's &lt;code&gt;realmax&lt;/code&gt;. I've noticed that &lt;code&gt;realmax&lt;/code&gt; does not give us good decimal accuracy. Is there any way to get the wide range that realmax covers with bisection while still giving us good decimal accuracy?&lt;/p&gt;&#xA;" OwnerUserId="14686" OwnerDisplayName="Alex Khodaverdian" LastEditorUserId="9667" LastEditDate="2015-03-01T22:33:02.383" LastActivityDate="2015-03-02T04:17:43.830" Title="Matlab Large Numbers and Small Numbers" Tags="|floating-point|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19027" PostTypeId="2" ParentId="18963" CreationDate="2015-03-01T14:36:13.880" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What are good parametrizations of rational functions for response surface models?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A widely used flexible parametrization of (piecewise) rational functions are &lt;a href=&quot;http://en.wikipedia.org/wiki/Non-uniform_rational_B-spline&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-uniform rational basis spline&lt;/a&gt; (NURBS) models. The advantage of this parametrization is that the effect of the parameters can be understood intuitively. The effect of the denominator here is to assign a weight to each control point. (It allows to represent surfaces of revolution exactly without special treatment, which might be the main reason why &lt;a href=&quot;http://en.wikipedia.org/wiki/B-spline&quot; rel=&quot;nofollow noreferrer&quot;&gt;basis spline&lt;/a&gt; models were generalized to NURBS models.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every piecewise univariate rational function can be represented by a NURBS model. Every (non-piecewise) multivariate rational function can be represented by a tensor product NURBS model. However, while the monomial basis representation of the quotient of two linear functions has $2n+1$ free parameters, the smallest tensor product NURBS model representation has $2\cdot 2^n$ parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the difference between an $n$-simplex and an $n$-cube. If we give up the piecewise part, then we can define an analogous parametrization on an $n$-simplex by observing that the &lt;a href=&quot;http://Bernstein%20polynomial&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bernstein polynomials&lt;/a&gt; arise from $1=(t+(1-t))^n=\sum_{j=0}^n\frac{n!}{j!(n-j)!}t^j(1-t)^{n-j}$, which is easy to generalize, for example the basis polynomials for a triangle arise from $1=(s+t+(1-s-t))^n=\sum_{i+j+k=n}\frac{n!}{i!j!k!}s^it^j(1-t)^k$.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The barycentric form is one possible parametrization of a univariate rational function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is know that any rational function $r(x)=\frac{p(x)}{q(x)}$ with $\max(\operatorname{deg}p,\operatorname{deg}q)\leq n$ for $n+1$ distinct interpolation points $x_j$ with $q(x_j)\neq 0$ can be written in barycentric form&#xA;$$r(x)=\frac{\sum_{j=0}^n\frac{u_j}{x-x_j}r_j}{\sum_{j=0}^n\frac{u_j}{x-x_j}}$$&#xA;for $r_j:=r(x_j)$ and suitable weights $u_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tensor product barycentric form writes a rational function $r(x,y)=\frac{p(x,y)}{q(x,y)}$ with $\max(\operatorname{deg}_x p,\operatorname{deg}_x q)\leq m$ and $\max(\operatorname{deg}_y p,\operatorname{deg}_y q)\leq n$ as&#xA;$$r(x,y)=\frac{\sum_{i=0}^m\sum_{j=0}^n\frac{u_{ij}}{(x-x_i)(y-y_j)}r_{ij}}{\sum_{i=0}^m\sum_{j=0}^n\frac{u_{ij}}{(x-x_i)(y-y_j)}}$$&#xA;for $m+1$ distinct $x$-positions $x_i$ and $n+1$ distinct $y$-positions $y_j$ with $q(x_i,y_j)\neq 0$ for $r_{ij}=r(x_i,y_j)$ and suitable weights $u_{ij}$. The &lt;a href=&quot;https://math.stackexchange.com/a/1160109&quot;&gt;proof&lt;/a&gt; remains essential identical to the univariate case: The barycentric Lagrange formula represents the denominator as $q(x,y)=l^m(x)l^n(y)\sum_{i=0}^m\sum_{j=0}^n\frac{w^m_iw^n_j}{(x-x_i)(y-y_j)}q_{ij}$ and $q_{ij}\neq0$ ensures $p(x,y)=\tilde{p}(x,y)$ for $\tilde{p}(x,y)=l^m(x)l^n(y)\sum_{i=0}^m\sum_{j=0}^n\frac{w^m_iw^n_j}{(x-x_i)(y-y_j)}q_{ij}r_{ij}$. Hence the weights $u_{ij}=w^m_iw^n_jq_{ij}$ lead to the barycentric representation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is an interesting topic, but I'm not sure what the question is. When adding parameters, I always like to remember von Neumann's comment: &quot;With four parameters I can fit an elephant, and with five I can make him wiggle his trunk as well.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One implicit question I have is why rational functions are not considered a worthy topic for scicomp.SE or math.SE? You may object that I self-answered &lt;a href=&quot;https://math.stackexchange.com/questions/1158836/can-every-rational-function-be-represented-in-barycentric-form&quot;&gt;my question&lt;/a&gt; on math.SE. However, the answer is from a reference of a reference (I guess Nick Trefethen intentionally made the passing remark about rational functions, because they have been neglected indeed), and didn't answer the &quot;What about the rational function $r(x,y)=\frac{xy}{x^2+y^2}$?&quot; part, which is only answered above. This scicomp.SE question was more open ended, and also described the context, why I'm interested in parametrizations of rational functions. A reference to NURBS (or projective geometry) in the comments would have been helpful to put me on a track where additional &quot;literature&quot; and &quot;hints&quot; are available. I'm convinced that this answer is still much too focused on the few things about rational functions I could find myself, and misses some important bigger picture.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Overall, I'm thinking about invariance of the form under transformations like ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here are various transformations for which invariance of the form can be desirable:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$r(x,y,z)=f(x)+g(y,z)$&lt;/li&gt;&#xA;&lt;li&gt;$r(x,y,z)=f(x)g(y,z)$&lt;/li&gt;&#xA;&lt;li&gt;$r(x,y,z)=\alpha f(x,y,z)$&lt;/li&gt;&#xA;&lt;li&gt;$r(x,y,z)=f(x,y,z)+\beta$&lt;/li&gt;&#xA;&lt;li&gt;$r(x,y,z)=f(\alpha x,y,z)$&lt;/li&gt;&#xA;&lt;li&gt;$r(x,y,z)=f(x+\beta,y,z)$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These invariances can be achieved by multilinear functions. But which additional invariances can be achieved by rational functions?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$r(x,y,z)=\frac{f(x,y,z)}{1+\gamma f(x,y,z)}$&lt;/li&gt;&#xA;&lt;li&gt;$r(x,y,z)=f(\frac{x}{1+\gamma x},y,z)$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The simple quotient form is invariant under Möbious transformations like (1.), but that doesn't really seem like an important invariance. Let's check (2.) for $f(x,y)=\frac{c+c_x x+c_y y+c_{xy}xy}{1+d_x x+d_y y+d_{xy}xy}$&#xA;$$r(x,y)=\frac{c(1+\gamma x)+c_x x+c_y (1+\gamma x)y+c_{xy}xy}{1+\gamma x+d_x x+d_y(1+\gamma x)y+d_{xy}xy}$$&#xA;$$=\frac{c+(c_x+c\gamma)x+c_y y+(c_{xy}+c_y\gamma)xy}{1+(d_x+\gamma) x+d_y y+(d_{xy}+d_y\gamma)xy}$$&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-03-09T00:28:11.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19028" PostTypeId="2" ParentId="19026" CreationDate="2015-03-01T14:41:11.817" Score="2" Body="&lt;p&gt;MATLAB uses double precision by default which gives about 16 digits. Do you mean that you want some &lt;em&gt;absolute&lt;/em&gt; precision (for example, accurate within 0.01)? If you have numbers near &lt;code&gt;realmax&lt;/code&gt; (~1e308) and you want precision of 0.01 you will need numbers that give about 310 digits of precision which will require about 1 kilobit of memory (2&lt;sup&gt;1024&lt;/sup&gt; ~ 1e309) to store the mantissa of your number. This will obviously be much slower to work with than 64-bit double precision numbers. I'm not sure how feasible this will be, but unless you can come up with another solution you could look into arbitrary precision arithmetic. There is an existing MATLAB toolbox that might work for you: &lt;a href=&quot;http://www.advanpix.com&quot; rel=&quot;nofollow&quot;&gt;http://www.advanpix.com&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-03-01T14:41:11.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19029" PostTypeId="1" CreationDate="2015-03-01T16:18:09.073" Score="1" ViewCount="333" Body="&lt;p&gt;I was told that the fast Chebyshev transform has superior spectral convergence, but I am unable to verify its rumored convergence. I was given plots of its spectral convergence, where the signal's 40th Chebyshev coefficient shrinks to $10^{-16}$ from its largest. The signal was a Gaussian sampled evenly distributed 64 data points. My implementation of the fast Chebyshev transform only yields the 40th Chebyshev coefficient to be $10^{-2}$ smaller than the largest. Unfortunately, I have not been given permission yet to post the plots of the supposed convergence. I wrote code in Python to reproduce the results for myself, to convenience myself of the power of the fast Chebyshev transform. Below is my Python code for implementing the fast Chebyshev transform and sorting the log absolute of the Chebyshev coefficients. Am I using the incorrect implementation of the fast Chebyshev transform?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fct(xp,fp): # fast Chebyshev transform&#xA;    # size of array&#xA;    N = xp.size&#xA;    # define Chebyshev grid and scale/translate to fit inside window&#xA;    x = xp[0]+(xp[n-1]-xp[0])*(1-np.cos(np.pi*(np.arange(N)+1/2)/N))/2&#xA;    # interpolate signal from Chebyshev grid and compute Chebyshev coefficients&#xA;    a = ((-1)**(np.arange(N)))*dct(np.interp(x,xp,fp),type=2)&#xA;    a[0]=a[0]/2&#xA;    return a&#xA;def sortnormlogspec(a,ee=1e-100): # normalize data to log absolute&#xA;    # log10 abs norm with -100 floor&#xA;    b=np.log10(ee+np.abs(a))&#xA;    # sort from largest to smallest coefficient&#xA;    b=sorted(b,reverse=True)&#xA;    return b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Below are my plots of the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/oq94D.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7633" LastEditorUserId="7633" LastEditDate="2015-03-01T16:31:43.610" LastActivityDate="2015-03-01T16:31:43.610" Title="Am I using the incorrect implementation of the fast Chebyshev transform?" Tags="|numerical-analysis|convergence|spectral-method|fourier-transform|signal-processing|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19030" PostTypeId="2" ParentId="19026" CreationDate="2015-03-01T17:17:08.683" Score="4" Body="&lt;p&gt;Unless you have a polynomial in multiple variables, finding the roots of a cubic polynomial is pretty simple: there is an &lt;a href=&quot;http://en.wikipedia.org/wiki/Cubic_function#Roots_of_a_cubic_function&quot; rel=&quot;nofollow&quot;&gt;explicit formula&lt;/a&gt; in terms of the coefficients of your polynomial.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-01T17:17:08.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19031" PostTypeId="1" AcceptedAnswerId="19038" CreationDate="2015-03-01T21:50:47.173" Score="0" ViewCount="98" Body="&lt;p&gt;I am attempting to work through a very simple problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determine the Fourier series expansion for the following heat PDE problem with ICS and BCS:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_{t} = \alpha^2u_{xx}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u(0, t) = u(L, t) = 0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u(x, 0) = \begin{cases} 0 &amp;amp; 0 \leq x \leq L/2 \\ 1 &amp;amp; L/2 &amp;lt; x \leq L\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the form of the boundary conditions, and using separation of variables, I concluded that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa = \frac{n\pi}{L}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x, t) = \sum_{n=1}^{\infty} a_n \exp(-\alpha^2\kappa^2t) \sin(\kappa x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to use the ICs, one uses the orthogonality of sinusoids to obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ a_n = \frac{2}{L}\int_{0}^{L} u(x, 0) \sin(\kappa x)\;\textrm{d}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In our case, with the given $u(x, 0)$, I obtained:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align}&#xA;   a_n &amp;amp;= \frac{2}{L}\int_{0}^{L} u(x, 0) \sin(\kappa x)\;\textrm{d}x \\&#xA;       &amp;amp;= \frac{2}{L}\left[\int_{0}^{L/2} 0\;\textrm{d}x + \int_{L/2}^{L} \sin(\kappa x) \;\textrm{d}x\right] \\&#xA;       &amp;amp;= \frac{2}{L}\int_{L/2}^{L} \sin(\kappa x) \;\textrm{d}x \\&#xA;       &amp;amp;= -\frac{2}{n\pi}\cos{n\pi}&#xA;   \end{align} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I have written some very simple python code to compute the first however many terms I'd like: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;#=========================================&#xA;&#xA;def sum_f_series(start, end_p1, t, x, L, alpha2):&#xA;    n = np.arange(start,end_p1)&#xA;    n2 = n*n&#xA;&#xA;    K = (np.pi/L)&#xA;&#xA;    C1 = -1*alpha2*(K**2)*t&#xA;    t_part = np.exp(C1*n2)&#xA;&#xA;    C2 = K&#xA;    x_part = np.sin(C2*n)&#xA;&#xA;    a_n = -1*(2*np.pi/n)*np.cos(n*np.pi)&#xA;&#xA;    return np.sum(t_part*x_part*a_n)&#xA;&#xA;#======================================&#xA;&#xA;start = 1&#xA;end_p1 = 11 # last term + 1&#xA;&#xA;a = 0&#xA;b = 1.5&#xA;L = b-a&#xA;&#xA;t = 0&#xA;x = 1&#xA;&#xA;alpha2 = 5&#xA;&#xA;print sum_f_series(start, end_p1, t, x, L, alpha2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result is around $6.6$ -- a far cry from $1$, which is what the result should be (consider the initial conditions, and the fact that $1 &amp;gt; 0.75 = L/2$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I am making an error somewhere, but I have been unable to find it. Can you help?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2015-03-02T11:03:15.790" Title="Mathematical error when attempting to represent step function using fourier series" Tags="|python|numpy|education|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19032" PostTypeId="1" AcceptedAnswerId="19035" CreationDate="2015-03-01T22:49:35.103" Score="5" ViewCount="833" Body="&lt;p&gt;I want to compute the derivative of a generalized eigenvalue $\lambda$ which is solution of $A u = \lambda Bu$ ($A,B,u,\lambda$ all depend on $t$; in my case $A,B$ are known explicitly, and the eigenvalue $\lambda$ and its corresponding eigenvector $u$ can be immediately computed using eigs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I write formally the derivation, I arrive to the problem &#xA;$$ (A-\lambda B)u' = -(A'-\lambda B')u+\lambda' Bu \ \ \ (1)$$&#xA;In the above equation I know $A,B,\lambda,u,A',B'$. The unknowns are $\lambda'$ and $u'$. If $A,B$ are symmetric, we can get rid of $u'$ by taking the scalar product with $u$. In my case $A,B$ are not always symmetric, so I need to solve $(1)$ with both unknowns $u'$ and $\lambda'$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has the form $Xu' = v+\lambda ' w$ where $X$ is a known matrix (non-invertible) and $v,w$ are known vectors.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can we compute $u'$ and $\lambda'$?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2015-03-02T06:46:18.453" Title="Derivative of a generalized eigenvalue problem" Tags="|linear-algebra|eigenvalues|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19034" PostTypeId="2" ParentId="19026" CreationDate="2015-03-02T04:17:43.830" Score="2" Body="&lt;p&gt;Correctly implemented bisection method cannot just fail to find a root, so it is probably an implementation issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, it's possible that you stop bisection too early (how exactly have you implemented it?). If you start with an interval $\pm M\approx 10^{300}$, then you'll need $\log_2 (2M/\mathrm{tol})\approx 1000$ iterations before convergence. So it's possible you have accidentally put a limit somewhere on the number of bisections that stops convergence &quot;artificially&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, there is a much better way to pick a bound. According to &lt;a href=&quot;http://en.wikipedia.org/wiki/Properties_of_polynomial_roots#Bounds_on_.28complex.29_polynomial_roots&quot; rel=&quot;nofollow&quot;&gt;wikipedia&lt;/a&gt;, an upper bound on the magnitude of (complex) roots of the polynomial $a_0+a_1x+a_2x^2+a_3x^3$ is&#xA;$$ M = \max\left(1, \frac{|a_0|+|a_1|+|a_2|}{|a_3|}\right), $$&#xA;so a real root of the cubic can be found in $[-M,M]$. There is a number of other bounds there as well. Often one takes a slightly larger bound, like $1.01M$, to avoid corner cases. Since in your case you know the coefficients, this bound is much easier to work with than &lt;code&gt;realmax&lt;/code&gt;, which is a very conservative upper bound.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-03-02T04:17:43.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19035" PostTypeId="2" ParentId="19032" CreationDate="2015-03-02T06:46:18.453" Score="5" Body="&lt;p&gt;The eigenproblem you have doesn't have a unique solution, any multiple&#xA;of $u$ solves it as well. So suppose we impose the condition&#xA;$u^tu=1$. The &lt;em&gt;two&lt;/em&gt; equations are&#xA;$$ Au=\lambda Bu, \qquad u^Tu=1, $$&#xA;giving derivatives&#xA;$$ (A-\lambda B)u' -\lambda' Bu = (\lambda B'-A')u, \qquad 2u^T u' =&#xA;0. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is nothing but a system of $n+1$ equations in $n+1$ unknowns:&#xA;$$ \begin{pmatrix}&#xA;A-\lambda B &amp;amp; -B u\\ u^T &amp;amp; 0&#xA;\end{pmatrix}&#xA;\begin{pmatrix}u'\\ \lambda' \end{pmatrix}&#xA;=&#xA;\begin{pmatrix}&#xA;  (\lambda B'-A')u\\&#xA;  0&#xA;\end{pmatrix}, $$&#xA;which should have a unique solution, provided &lt;em&gt;this&lt;/em&gt; matrix is nonsingular, which should happen whenever $\lambda$ has multiplicity 1.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-03-02T06:46:18.453" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19036" PostTypeId="2" ParentId="7248" CreationDate="2015-03-02T08:46:09.727" Score="1" Body="&lt;p&gt;Is there a reason why you don't want to implement your FDM solver in generalized coordinates? Even if you use the IBM, is still a good idea to have the N-S equations in a generalized coordinate system. Let us say you have a 'S-shaped' geometry, Wouldn't it be better (in terms of grid size) just to make your domain as two curves instead of a box? Documentation on generalized coordinates can be found on the paper of Steger, J.L. (1978) AIAA J. 16, 679-686&lt;/p&gt;&#xA;" OwnerUserId="14659" LastActivityDate="2015-03-02T08:46:09.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19038" PostTypeId="2" ParentId="19031" CreationDate="2015-03-02T11:03:15.790" Score="2" Body="&lt;p&gt;Your errors are on the lines for &lt;code&gt;x_part&lt;/code&gt; and &lt;code&gt;a_n&lt;/code&gt;.  The argument to &lt;code&gt;np.sin()&lt;/code&gt; should include &lt;code&gt;x&lt;/code&gt;, and the coefficient for &lt;code&gt;a_n&lt;/code&gt; by your calculation should be &lt;code&gt;2/(np.pi*n)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14688" LastActivityDate="2015-03-02T11:03:15.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19039" PostTypeId="1" CreationDate="2015-03-02T14:31:50.650" Score="2" ViewCount="136" Body="&lt;p&gt;I want to optimize the vertex positions in a mesh, with a given cost function on the associated triangles. The &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=2631978.2602143&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; gives a cost function, which evaluates to an real number by using a sum over the triangles in the mesh, which connect the vertices to a valid simplical complex. They suggest to use a L-BFGS solver and i want to use PETSc to for the calculations.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/SNES/SNESFunction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;solver interface&lt;/a&gt; for L-BFGS (and some other algorithms) in PETSc gets a vector with the current values and has a pointer to a output vector for the residual values, with the same cardinality.&lt;/p&gt;&#xA;&lt;p&gt;How do i design the cost function and the residual vector based on the cost function to evaluate the cost of vertex positions based on the resulting triangles?&lt;/p&gt;&#xA;&lt;p&gt;i filled the f vector like this: &lt;span class=&quot;math-container&quot;&gt;$[v_1^x, v_1^y, v_1^z, \dots, v_N^x, v_N^y, v_N^z]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What do it put in the vector returned to get a good solution? I tried ...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;all the same: &lt;span class=&quot;math-container&quot;&gt;$\text{cost}(v) \equiv cost \in \mathbb{R}\ \forall v$&lt;/span&gt; (&lt;code&gt;VecSet(r, cost)&lt;/code&gt;.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\text{cost}(v_i^x) = \text{cost}(v_i^y) = \text{cost}(v_i^z) = \sum_{t \in T, v_i\in t} \text{cost}(t)\ \forall i=1\dots N$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; as the set of all triangles and the relation &lt;span class=&quot;math-container&quot;&gt;$v \in t$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is a vertex of a triangle &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Both do not obtain good solutions. Further i guess the residual may need to differ in x,y,z to get useful gradients for moving the vertices.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-03-03T13:47:36.797" Title="optimize vertices using a cost function on triangles" Tags="|optimization|mesh|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19041" PostTypeId="1" AcceptedAnswerId="19044" CreationDate="2015-03-02T18:30:35.277" Score="4" ViewCount="339" Body="&lt;p&gt;I experimented with the PageRank algorithm. When the number of pages is large, I encountered a situation when one formula for re-normalizing a vector (so that sum of its components is equal to 1; elements of the vector are guaranteed to be positive) works fine, while another stops working and using it causes that the iterations start to diverge (difference between old and new r starts growing bigger) after a while.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    r = r./sum(r)                  # this does not work&#xA;    r = r + (1-sum(r))/N * ones(N) # this works&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My PageRank algorighm looks like this (Julia)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M # M is NxN sparse transition matrix;&#xA;  # may contain all-zero columns (sinks); &#xA;beta = 0.8&#xA;epsilon = 1e-6&#xA;&#xA;r = 1/N ./ ones(N)&#xA;rtm1 = ones(N)&#xA;while(sum(abs(r-rtm1)) &amp;gt; epsilon)&#xA;  rtm1 = r&#xA;  r = beta*M*r&#xA;  ######&#xA;  # renormalization&#xA;  # pick one or the other&#xA;  # r = r./sum(r)&#xA;  # r = r + (1-sum(r))/N * ones(N)&#xA;  ######&#xA;end&#xA;# now r holds the result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am guessing that for large number of pages the elements of r can get very small (like 1.0e-7 or smaller) and the first normalization formula then does not work very well. I would like to hear an explanation why is that from somebody who has some experience with numeric computations.&lt;/p&gt;&#xA;" OwnerUserId="6697" LastActivityDate="2015-03-02T20:16:11.337" Title="When and why is `r./sum(r)` not a good way to renormalize a vector in PageRank computation?" Tags="|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19042" PostTypeId="1" CreationDate="2015-03-02T19:11:22.530" Score="3" ViewCount="322" Body="&lt;p&gt;In general, do iterative solvers require more floating point operations than the direct solver counterparts? I have some scientific code (written in both PETSc and FEniCS) for solving a mixed FE poisson equation and the profiling summaries seem to always show the direct solvers (eg, mumps, superlu) utilizing less flops by at least one order. Moreover, it also seems that for problems with over 1 million degrees of freedom, the solver requires more time, thus yielding an even smaller FLOPS/s efficiency than if I were to employ iterative solvers like GMRES (with bjacobi preconditioner).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this expected? Or is there potentially something wrong with my implementation(s)? Thanks&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2015-03-03T09:43:23.527" Title="FLOPs of iterative vs direct solvers" Tags="|iterative-method|floating-point|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19043" PostTypeId="2" ParentId="19042" CreationDate="2015-03-02T19:24:30.313" Score="4" Body="&lt;p&gt;How are you counting FLOPS? If you are using the hardware performance counters on Intel chips starting with Westmere and later, then you may significantly over count them. These counters count instructions &quot;executed&quot; not &quot;retired&quot;, but an executed instruction on these chips may not have actually been executed to completion and may be retried several times if the data is not available from RAM yet. This is very likely to happen with iterative solvers which are more likely to be based on matrix-vector products and therefore much more likely to be memory bandwidth limited. These codes are likely to be spending a lot of time waiting on data to be available from memory and will reissue the floating-point instructions many times. If you count FLOPS with the hardware counters on SandyBridge for the STREAM Triad benchmark, you will over count FLOPS by a factor of 10x or so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't begin to address your other questions, but it could be an explanation of your problem depending on whether or not you're trying to use the hardware counters.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-03-02T20:02:14.563" LastActivityDate="2015-03-02T20:02:14.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19044" PostTypeId="2" ParentId="19041" CreationDate="2015-03-02T20:16:11.337" Score="7" Body="&lt;p&gt;The two normalization formulas result in two different algorithms, that they both &quot;normalize&quot; a vector is not so relevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, consider the following transition matrix:&#xA;$$M = \begin{pmatrix}0&amp;amp;1\\1&amp;amp;0\end{pmatrix}.$$&#xA;Starting with $r=(1,0)$, consecutive normalized vectors $r$ will be $(0,1)$ and $(1,0)$, so the algorithm clearly will not ever converge. In this case the transition matrix is not &lt;a href=&quot;https://www.google.ca/search?q=regular+markov+chain&quot;&gt;&lt;em&gt;regular&lt;/em&gt;&lt;/a&gt;, and this is not a numerical issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the second formula, $r$ will instead be $(\frac{1-\beta}{2},\frac{1+\beta}{2})$ after first iteration, and&#xA;$$ \tfrac12(1+(-\beta)^k, 1-(-\beta)^k) $$&#xA;after $k$ iterations. So this is computing a fixed point of a completely different function, and it clearly converges ($\beta&amp;lt;1$) where the first algorithm does not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This behaviour is much easier to understand if you think of it in terms of a random walk on the underlying graph; this $M$ corresponds to the graph $1\rightleftharpoons2$. The first algorithm's random walk simply walks along the graph: the probability distribution of this walk is &lt;em&gt;not&lt;/em&gt; guaranteed to converge to the stationary distribution. The second algorithm walks along the graph with probability $\beta$, and jumps to a random node with probability $1-\beta$: this walk is guaranteed to converge to a stationary distribution. The key property here is that some power of $M$ must have all positive entries for the first algorithm to converge.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-03-02T20:16:11.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19045" PostTypeId="2" ParentId="18982" CreationDate="2015-03-02T21:06:36.993" Score="5" Body="&lt;p&gt;Honestly, the best ideas have already been said. I'll try to synthesize my thoughts anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, the best way to write a program is whatever way gives you the results you need, for the least development time. For some applications, you NEED bare-metal performance, but a lot of the time you don't. Since you've been working in Octave up until now, you may not fall into the bare-metal category, unless you're suddenly increasing your problem size dramatically. Even if you are, the C vs. Fortran speed debates are mostly immaterial at this scale. Even a factor of 10 doesn't matter much if it's 0.1 second vs 1 second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, my experience has been that most of what a code does is best handled by a high-level language such as Matlab, Python, etc. There are usually only a few small segments of code that really benefit from the extra programming effort to really optimize them. Often, these are things like inner-loop solvers, complicated compiled functions, and so on. It can be very beneficial to write the majority of a program in a high-level language, and then call low-level library functions as-needed for the really performance-intensive stuff. .mex files are Matlab's way of handling this (it looks like Octave has a clone of this, too). Python is especially good at it, too, with things like f2py, Cython, numba, and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, and finally, the most important aspect of this by far is the software ecosystem that will be available to you. Matlab isn't widely used because it's so blazing fast; it's widely used because it can plot data, perform LU-decomposition, numerically integrate, and much more, all with simple commands, and knocking a few months (or years!) off of development time is almost always worth the performance hit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum up, you need to quantify your needs (what's your problem size, and what's the acceptable length of time for a solution), examine what you already have (for re-use and/or porting), and make a list of your ecosystem requirements (vtk, linear algebra, etc.). Depending on the specific answers you come up with for these, there are many possible &quot;best&quot; options. Some of these might include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Keep your Octave code, but refactor it to use external libraries using .mex files where the performance requirements justify it. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rewrite your code in Python, for access to a wide variety of libraries and excellent mixed-language programming tools. This is what I have done, fairly successfully, in my work.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Write a new, completely parallelized HPC code using PETSc, MPI, etc. Unless the problem size dictates that you run this code on a supercomputer, my opinion is that you should stay away from this one. A friend of mine migrated a serial CFD code to PETSc once, and it took him years. YMMV&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14704" LastActivityDate="2015-03-02T21:06:36.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19048" PostTypeId="1" CreationDate="2015-03-03T07:13:40.560" Score="-1" ViewCount="163" Body="&lt;p&gt;I need a solver of stiff Inital-Value Problems (IVP) in python exploiting RK4 preferably explicit. I have been searching for past few days but could not find it. Following are my queries:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does the solver, i.e. any module, exist?&lt;/li&gt;&#xA;&lt;li&gt;If no, will it be reasonable to code one? I am asking this because I can't find any reference for using RK4-explicit for stiff problems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14708" LastActivityDate="2015-03-16T09:43:19.960" Title="RK4 for stiff IVP" Tags="|stiffness|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19049" PostTypeId="2" ParentId="19048" CreationDate="2015-03-03T07:53:45.963" Score="4" Body="&lt;p&gt;You can see the formulas for explicit RK4 at &lt;a href=&quot;http://mathworld.wolfram.com/Runge-KuttaMethod.html&quot; rel=&quot;nofollow&quot;&gt;Mathworld&lt;/a&gt;. Given the simplicity of the expressions it is very reasonable to code one for yourself which is why you may not have found a module.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if as you say you are solving a stiff IVP then an explicit Runge-Kutta method is not appropriate (the regions of absolute stability are bounded). Perhaps you should investigate implicit Runge-Kutta methods since there exist A-stable methods of arbitrarily high order.&lt;/p&gt;&#xA;" OwnerUserId="14706" LastActivityDate="2015-03-03T07:53:45.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19050" PostTypeId="1" AcceptedAnswerId="19063" CreationDate="2015-03-03T09:42:49.987" Score="1" ViewCount="72" Body="&lt;p&gt;Let $n_1,n_2 \in \mathbb{N}$ and $n=n_1n_2$ and $b\in \mathbb{R}^n$. I have a SPD-matrix $A=(a_{i,j})\in \mathbb{R}^{n \times n}$ with $a_{i,j}=0$ if $|i-j| \notin \{0,1,n_1\}$. Can we solve the system $Ax=b$ in time $\mathcal{O}(n)$? I work with Matlab. The Backslash operator does not seem to scale linearly in $n$. What about if $A=(a_{i,j})\in \left(\mathbb{R}^{d \times d}\right)^{n \times n}$ where $d\in \mathbb{N}$ is a fixed integer?&#xA;The problem arises from an image analysis problem with an image of size $n_1 \times n_2$ resp. $n_1 \times n_2 \times d$. The hessian of a certain functional has nonzero entries only for neighboring pixels. That's why my matrix has this special structure.&lt;/p&gt;&#xA;" OwnerUserId="14710" LastActivityDate="2015-03-04T00:29:18.643" Title="Sparse linear system of certain type" Tags="|linear-algebra|matlab|sparse-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19051" PostTypeId="2" ParentId="19042" CreationDate="2015-03-03T09:43:23.527" Score="1" Body="&lt;p&gt;It really depends a lot on the problem. In my experience, direct solvers are very performant for 2D problems with moderate (say, up to approximately a million or so) numbers of degrees of freedom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to deal with very large problems, you typically want to use iterative solvers. You need very good preconditioners to achieve good performance, though; the most common would be multigrid preconditioners, either of the geometric or algebraic variety. The Jacobi preconditioner simply won't give good results.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-03-03T09:43:23.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19052" PostTypeId="2" ParentId="19039" CreationDate="2015-03-03T09:54:13.567" Score="1" Body="&lt;p&gt;If I'm not mistaken, the SNES interface in PETSc deals only with nonlinear equations, not optimization problems. So you may be using the wrong software here. Remember that your cost functional yields only a single scalar value, so you don't want to have a vector-valued cost function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to stay with PETSc, it seems the TAO toolkit may be what you are looking for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mcs.anl.gov/research/projects/tao/&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/research/projects/tao/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-03-03T09:54:13.567" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19054" PostTypeId="1" AcceptedAnswerId="19059" CreationDate="2015-03-03T15:01:55.613" Score="0" ViewCount="7517" Body="&lt;p&gt;It was said in a comment that I should recude integration tolerance to eps. &lt;a href=&quot;https://scicomp.stackexchange.com/questions/18929/what-does-this-matlab-error-message-mean&quot;&gt;https://scicomp.stackexchange.com/questions/18929/what-does-this-matlab-error-message-mean&lt;/a&gt; I read the manual and I still don't know how to do it. Can you please tell me how to reduce integration tolerance in matlab?&lt;/p&gt;&#xA;" OwnerUserId="3566" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-03-03T20:34:14.087" Title="How can I reduce integration tolerance in matlab?" Tags="|matlab|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19055" PostTypeId="1" AcceptedAnswerId="19110" CreationDate="2015-03-03T15:41:35.630" Score="2" ViewCount="739" Body="&lt;p&gt;I am trying to solve shallow water equations using DG methods. Flow over a bump is a common problem that comes up in this context. For example (&lt;a href=&quot;http://loki.udc.es/info/asignaturas/calculo_ii/Finite%20Volumes%20IMWE.pdf&quot; rel=&quot;nofollow&quot;&gt;http://loki.udc.es/info/asignaturas/calculo_ii/Finite%20Volumes%20IMWE.pdf&lt;/a&gt;, page 46). These are my questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The boundary conditions are mentioned only for the upstream discharge and downstream height. How can I get the boundary values for upstream height and downstream discharge that is required for flux calculations? &lt;/li&gt;&#xA;&lt;li&gt;How can I apply slope limiters at the boundaries in this case or even a simpler case of an advection equation where Dirichlet boundary condition is specified at the boundary? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11644" LastEditorUserId="123" LastEditDate="2015-04-24T08:16:21.017" LastActivityDate="2015-04-24T08:16:21.017" Title="Shallow Water Equations Boundary Conditions" Tags="|boundary-conditions|hyperbolic-pde|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19057" PostTypeId="2" ParentId="19054" CreationDate="2015-03-03T17:43:42.057" Score="2" Body="&lt;p&gt;Choose the type of tolerance you want to set (&lt;code&gt;RelTol&lt;/code&gt;, &lt;code&gt;AbsTol&lt;/code&gt;, or &lt;code&gt;NormControl&lt;/code&gt;), use &lt;code&gt;odeset()&lt;/code&gt; to define the options for your ODE solver, call the ODE solver and pass the desired options. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options = odeset('RelTol',eps);&#xA;[T,Y] = ode45(odefun,tspan,y0,options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is extensive documentation about these functions that can be accessed by entering &lt;code&gt;doc odeset&lt;/code&gt; or &lt;code&gt;doc ode45&lt;/code&gt; in MATLAB's command prompt.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-03-03T17:43:42.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19058" PostTypeId="1" CreationDate="2015-03-03T20:25:00.803" Score="1" ViewCount="158" Body="&lt;p&gt;Given the functional:&#xA;$$  &#xA;F[\mathbf{x}]=\frac{1}{2}[\mathbf{x}^{\text{T}} * D(\mathbf{x})]-\frac{1}{2}[\mathbf{x}^{\text{T}} * \mathbf{Ax}]-\frac{1}{2}\mathbf{x}^{\text{T}}(0)\mathbf{x}(t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathbf{A}$ is symmetric, $\mathbf{x}(0)$ being the initial condition, $\mathbf{x}(t)$ is continuous, and:&#xA;$$&#xA;[\mathbf{f}^{\text{T}} * \mathbf{g}]=\int^{t}_0 \mathbf{f}^{\text{T}}(t-\tau)\mathbf{g}(\tau)\,\text{d}\tau&#xA;$$&#xA;and:&#xA;$$&#xA;D\left(\cdot\right)=\frac{\text{d}}{\text{d}t}\left(\cdot\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We take the variation as:&#xA;$$&#xA;\delta F\left[\mathbf{x}\right]=\left.\frac{\partial}{\partial \varepsilon} F\left[\mathbf{x}+\varepsilon\delta\mathbf{x}\right]\right|_{\varepsilon=0}&#xA;$$&#xA;Functional at the varied function is then:&#xA;$$&#xA;F\left[\mathbf{x}+\varepsilon\delta\mathbf{x}\right]=\frac{1}{2}[\mathbf{x}^{\text{T}}+\varepsilon\delta\mathbf{x}^{\text{T}} * D(\mathbf{x}+\varepsilon\delta\mathbf{x})]-\frac{1}{2}[\mathbf{x}^{\text{T}}+\varepsilon\delta\mathbf{x}^{\text{T}} * \mathbf{A}\left(\mathbf{x}+\varepsilon\delta\mathbf{x}\right)]-\frac{1}{2}\left(\mathbf{x}^\text{T}(0)+\varepsilon\delta\mathbf{x}^\text{T}(0)\right)\left(\mathbf{x}(t)+\varepsilon\delta\mathbf{x}(t)\right)&#xA;$$&#xA;The individual terms being:&#xA;$$&#xA;\frac{1}{2}[\mathbf{x}^{\text{T}}+\varepsilon\delta\mathbf{x}^{\text{T}} * \mathbf{A}\left(\mathbf{x}+\varepsilon\delta\mathbf{x}\right)]=\frac{1}{2}[\mathbf{x}^{\text{T}}* D(\mathbf{x})]+\frac{1}{2}\varepsilon[\mathbf{x}^{\text{T}}* D(\delta\mathbf{x})]+\frac{1}{2}\varepsilon[\delta\mathbf{x}^{\text{T}}* D(\mathbf{x})]+\frac{1}{2}\varepsilon^2[\delta\mathbf{x}^{\text{T}}* D(\delta\mathbf{x})]&#xA;$$&#xA;$$&#xA;\frac{1}{2}[\mathbf{x}^{\text{T}}+\varepsilon\delta\mathbf{x}^{\text{T}} * \mathbf{A}\left(\mathbf{x}+\varepsilon\delta\mathbf{x}\right)]=\frac{1}{2}[\mathbf{x}^{\text{T}} * \mathbf{A}\mathbf{x}]+\frac{1}{2}\varepsilon[\mathbf{x}^{\text{T}}* \mathbf{A}\delta\mathbf{x}]+\frac{1}{2}\varepsilon[\delta\mathbf{x}^{\text{T}} * \mathbf{A}\mathbf{x}]+\frac{1}{2}\varepsilon^2[\delta\mathbf{x}^{\text{T}} * \mathbf{A}\delta\mathbf{x}]&#xA;$$&#xA;$$&#xA;-\frac{1}{2}\left(\mathbf{x}^\text{T}(0)+\varepsilon\delta\mathbf{x}^\text{T}(0)\right)\left(\mathbf{x}(t)+\varepsilon\delta\mathbf{x}(t))\right)=-\frac{1}{2}\mathbf{x}^\text{T}(0)\mathbf{x}(t)-\frac{1}{2}\varepsilon\mathbf{x}^\text{T}(0)\delta\mathbf{x}(t)-\frac{1}{2}\varepsilon\delta\mathbf{x}^\text{T}(0)\mathbf{x}(t)-\frac{1}{2}\varepsilon^2\delta\mathbf{x}^\text{T}(0)\delta\mathbf{x}(t)&#xA;$$&#xA;We can differentiate these with respect to $\varepsilon$ and then set it to zero, adding them up leads to:&#xA;$$&#xA;\delta F\left[\mathbf{x}\right]=\frac{1}{2}[\mathbf{x}^{\text{T}}* D(\delta\mathbf{x})]+\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* D(\mathbf{x})]-\frac{1}{2}[\mathbf{x}^{\text{T}}* \mathbf{A}\delta\mathbf{x}]-\frac{1}{2}[\delta\mathbf{x}^{\text{T}} * \mathbf{A}\mathbf{x}]-\frac{1}{2}\mathbf{x}^\text{T}(0)\delta\mathbf{x}(t)-\frac{1}{2}\delta\mathbf{x}^\text{T}(0)\mathbf{x}(t)&#xA;$$&#xA;We also can say:&#xA;$$&#xA;\frac{1}{2}[\mathbf{x}^{\text{T}}* \mathbf{A}\delta\mathbf{x}]=\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* \mathbf{A}^{\text{T}}\mathbf{x}]&#xA;$$&#xA;Since $\mathbf{A}$ is symmetric, this just means that:&#xA;$$&#xA;\frac{1}{2}[\mathbf{x}^{\text{T}}* \mathbf{A}\delta\mathbf{x}]=\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* \mathbf{A}\mathbf{x}]&#xA;$$&#xA;Substituting this back into the varied functional yields:&#xA;$$&#xA;\delta F\left[\mathbf{x}\right]=\frac{1}{2}[\mathbf{x}^{\text{T}}* D(\delta\mathbf{x})]+\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* D(\mathbf{x})]-\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* \mathbf{A}\mathbf{x}]-\frac{1}{2}[\delta\mathbf{x}^{\text{T}} * \mathbf{A}\mathbf{x}]-\frac{1}{2}\mathbf{x}^\text{T}(0)\delta\mathbf{x}(t)-\frac{1}{2}\delta\mathbf{x}^\text{T}(0)\mathbf{x}(t)&#xA;$$&#xA;Notice, now we have two similar terms which can be grouped, doing so yields:&#xA;$$&#xA;\delta F\left[\mathbf{x}\right]=\frac{1}{2}[\mathbf{x}^{\text{T}}* D(\delta\mathbf{x})]+\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* D(\mathbf{x})]-[\delta\mathbf{x}^{\text{T}}* \mathbf{A}\mathbf{x}]-\frac{1}{2}\mathbf{x}^\text{T}(0)\delta\mathbf{x}(t)-\frac{1}{2}\delta\mathbf{x}^\text{T}(0)\mathbf{x}(t)&#xA;$$&#xA;We can lump this into the other term which a variation:&#xA;$$&#xA;\delta F\left[\mathbf{x}\right]=\frac{1}{2}[\mathbf{x}^{\text{T}}* D(\delta\mathbf{x})]+\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* \left(D(\mathbf{x})-2\mathbf{A}\mathbf{x}\right)]-\frac{1}{2}\mathbf{x}^\text{T}(0)\delta\mathbf{x}(t)-\frac{1}{2}\delta\mathbf{x}^\text{T}(0)\mathbf{x}(t)&#xA;$$&#xA;Now, if we say that $\delta\mathbf{x}(0)=0$, we get:&#xA;$$&#xA;\delta F\left[\mathbf{x}\right]=\frac{1}{2}[\mathbf{x}^{\text{T}}* D(\delta\mathbf{x})]+\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* \left(D(\mathbf{x})-2\mathbf{A}\mathbf{x}\right)]-\frac{1}{2}\mathbf{x}^\text{T}(0)\delta\mathbf{x}(t)&#xA;$$&#xA;This is the form I would use for discretizing, but we can show which system it is stationary with respect to. Taking integration by parts on the first term yields:&#xA;$$&#xA;\frac{1}{2}[\mathbf{x}^{\text{T}}* D(\delta\mathbf{x})]=\frac{1}{2}\mathbf{x}^{\text{T}}(0)\delta\mathbf{x}(t)-\frac{1}{2}\mathbf{x}^{\text{T}}(t)\delta\mathbf{x}(0)+\frac{1}{2}[\delta\mathbf{x}^{\text{T}}* D(\mathbf{x})]&#xA;$$&#xA;Substituting this back in yields (with $\delta\mathbf{x}(0)=0$):&#xA;$$&#xA;\delta F\left[\mathbf{x}\right]=[\delta\mathbf{x}^{\text{T}}* \left(D(\mathbf{x})-\mathbf{A}\mathbf{x}\right)]&#xA;$$&#xA;Which implies:&#xA;$$&#xA;D(\mathbf{x})-\mathbf{A}\mathbf{x}=0&#xA;$$&#xA;Or:&#xA;$$&#xA;\frac{d\mathbf{x}(t)}{dt}= \mathbf{Ax}(t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, the first variation of this functional (with no IBP) is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  &#xA;\delta F[\mathbf{x}]=\frac{1}{2}[\mathbf{x}^{\text{T}} * D(\delta\mathbf{x})]+\frac{1}{2}[\delta\mathbf{x}^{\text{T}} * \left(D(\mathbf{x})-2\mathbf{Ax}\right)]-\frac{1}{2}\mathbf{x}^{\text{T}}(0)\delta\mathbf{x}(t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now discretize this over the interval $\left[0,\Delta\right]$, using the approximation:&#xA;$$&#xA;\mathbf{x}(t)=\mathbf{x}_0 N_0(t)+\mathbf{x}_1 N_1(t)&#xA;$$&#xA;$$&#xA;\delta\mathbf{x}(t)=\delta\mathbf{x}_0 N_0(t)+\delta\mathbf{x}_1 N_1(t)&#xA;$$&#xA;Where:&#xA;$$&#xA;N_0(t)=1-\frac{t}{\Delta}&#xA;$$&#xA;And&#xA;$$&#xA;N_1(t)=\frac{t}{\Delta}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is:&#xA;$$&#xA;\delta F[\mathbf{x}] \approx \delta\mathbf{x}_1^{\text{T}}\left(\left(\frac{1}{2}\mathbf{I}-\frac{\Delta}{6}\mathbf{A}\right)\mathbf{x}_1-\frac{\Delta}{3}\mathbf{A}\mathbf{x}_0-\frac{1}{2}\mathbf{x}(0)\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that $\delta\mathbf{x}_1$ is some arbitrary discrete variation, we can say that it's coefficient must be zero:&#xA;$$&#xA;\left(\frac{1}{2}\mathbf{I}-\frac{\Delta}{6}\mathbf{A}\right)\mathbf{x}_1-\frac{\Delta}{3}\mathbf{A}\mathbf{x}_0-\frac{1}{2}\mathbf{x}(0)=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is the following also true?:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left(\frac{1}{2}\mathbf{I}-\frac{\Delta}{6}\mathbf{A}\right)\mathbf{x}_{i+1}-\frac{\Delta}{3}\mathbf{A}\mathbf{x}_i-\frac{1}{2}\mathbf{x}(0)=0&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="4509" LastEditorUserId="1804" LastEditDate="2015-03-05T07:14:01.343" LastActivityDate="2015-03-05T07:16:31.607" Title="Step-wise finite element formulations: can this be done?" Tags="|finite-element|ode|time-integration|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="19059" PostTypeId="2" ParentId="19054" CreationDate="2015-03-03T20:34:14.087" Score="2" Body="&lt;p&gt;The error message&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Warning: Failure at t=3.974486e+03.  Unable to meet integration tolerances&#xA;  without reducing the step size below the smallest value allowed&#xA;  (7.275958e-12) at time t.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;usually means that your error tolerances are too tight.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In brief, variable time-step integrators will adapt the time step based on weighted norm that more or less computes the ratio of the supplied error tolerances to the current local error estimate. Roughly speaking, if the error tolerances are smaller, the next time step will be smaller; if the error tolerances are larger, the next time step will be larger. There is usually a minimum time step size for integration, so what you are seeing is that the outcome of the error test is to shrink the time step, but the time step is already at its minimum value, so it cannot be decreased any more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If looser error tolerances do not yield a reasonable solution, it usually suggests one of the following possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you're using the wrong integration method; this usually happens with stiff solvers, in which case, you should consider using an implicit, stiff ODE solver&lt;/li&gt;&#xA;&lt;li&gt;your system is poorly behaved (the right-hand side might not be Lipschitz, for instance); in that case, you need to consider changing the model you're solving&lt;/li&gt;&#xA;&lt;li&gt;you introduced some sort of accidental bug in your right-hand side; in that case, you need to test your code and fix that bug&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-03-03T20:34:14.087" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19060" PostTypeId="1" CreationDate="2015-03-03T21:25:57.410" Score="1" ViewCount="134" Body="&lt;p&gt;I'm not a huge programmer, but if it'll save me lots of time, that's when I code (given that it's possible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know MATLab has a Computer Vision Toolbox, so my question is: is it possible to program MATLab to find a picture within a video, without actually playing the video?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have hundreds of 2 hour videos of participants viewing pictures. The problem is, the images are randomized in the videos, so finding reactions to certain images is rather tedious because it means searching the video for a while. I wanted to know if I could program MATLab to find a given picture within the video, without playing the video, and if possible export the time at which the image appeared in the video. This would save us quite a great deal of time in searching.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="14714" LastEditorUserId="14716" LastEditDate="2015-03-04T09:27:03.087" LastActivityDate="2015-03-04T09:27:03.087" Title="Is it possible to locate an image in a video using MATLab?" Tags="|matlab|image-processing|computer-vision|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19061" PostTypeId="2" ParentId="19060" CreationDate="2015-03-03T22:50:58.790" Score="1" Body="&lt;p&gt;Take a look at this &lt;a href=&quot;http://www.mathworks.com/help/vision/examples/object-detection-in-a-cluttered-scene-using-point-feature-matching.html&quot; rel=&quot;nofollow&quot;&gt;example showing how to detect objects in cluttered scenes&lt;/a&gt; in matlab. This should work well for finding a picture in a video. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure I understand what you mean by &quot;without playing the video&quot;. This can be done without you having to look at the video yourself, but your program would still have to read the video file frame by frame, and search for the object of interest in each frame.&lt;/p&gt;&#xA;" OwnerUserId="14716" LastActivityDate="2015-03-03T22:50:58.790" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19063" PostTypeId="2" ParentId="19050" CreationDate="2015-03-04T00:29:18.643" Score="3" Body="&lt;p&gt;You have the equivalent of the 5-point stencil in finite differences. The general answer is that you cannot solve this in $O(n)$ without making use of other properties of the matrix and/or its entries. Using the sparsity structure, by itself, is not enough. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if the matrix arises from the Laplace or another symmetric and elliptic equation, then you can use multigrid solvers to obtain $O(n)$ complexity. On the other hand, you get the same sparsity pattern if you discretize the high-frequency Helmholtz equation (with the bad sign) using the five-point stencil, and for that equation multigrid does not help.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-04T00:29:18.643" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19065" PostTypeId="1" AcceptedAnswerId="19066" CreationDate="2015-03-04T11:36:47.070" Score="1" ViewCount="287" Body="&lt;p&gt;Is there any algorithm that can learn/search efficiently the best sequence of 1's and 0's of length $n$ to fulfill certain performance? The search is performed in a high-dimensional binary data space. This means that the search space is of $2^n$ where $n$ usually is in the order of hundreds. I know that Discrete Particle Swarm Optimization, Simulated Annealing or Genetic Algorithms can be used for this purpose, but I'm afraid they tend to get stuck in local optima and/or can not work efficiently in high-dimensional spaces.&lt;/p&gt;&#xA;" OwnerUserId="14719" LastEditorUserId="14719" LastEditDate="2015-03-04T11:45:15.660" LastActivityDate="2016-12-09T04:44:22.987" Title="Algorithms for searching in high-dimensional binary data spaces" Tags="|optimization|machine-learning|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19066" PostTypeId="2" ParentId="19065" CreationDate="2015-03-04T12:02:51.517" Score="0" Body="&lt;p&gt;This question could search for be the analog of a (almost-polynomial) n-SAT algorithm, but no such algorithm is currently available (or may never be if P &amp;lt;&gt; NP), so an optimisation method like &lt;strong&gt;SA&lt;/strong&gt; or &lt;strong&gt;GA&lt;/strong&gt;, should be the best option under your requirements&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionaly, there is an algorithm called &lt;a href=&quot;https://www.google.gr/search?q=deterministic+annealing&amp;amp;ie=utf-8&amp;amp;oe=utf-8&amp;amp;gws_rd=cr&amp;amp;ei=yvP2VKHgDMvyUtfog5gD&quot; rel=&quot;nofollow&quot;&gt;Deterministic Annealing (&lt;strong&gt;DA&lt;/strong&gt;)&lt;/a&gt; which performs quite well on many circumstances, maybe check it out&lt;/p&gt;&#xA;" OwnerUserId="8172" LastActivityDate="2015-03-04T12:02:51.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19068" PostTypeId="2" ParentId="19006" CreationDate="2015-03-04T18:57:56.353" Score="8" Body="&lt;p&gt;Let us start with FEM. We have a differential equation of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{A}u=f\quad \forall x \text{ in } \Omega \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal{A}$ is a linear operator, and this problem is subject to some boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{H}u=g(x)\quad \forall x \text{ in } \partial\Omega \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are interested in finding an approximate solution to the PDE, $\hat{u}$ (let's assume that this approximate solution satisfy BCs), if we plug it in the first equation we do not get an equality. The difference&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R = \mathcal{A}\hat{u} - f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is the residual, and it is the zero function for the solution. We would like to make this $R$ as close to zero as possible, for this we can try to minimize the maximum norm, or the two norm of $R$ (the latter leads to Least Squares Finite Elements).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to go is to compute an &lt;em&gt;average&lt;/em&gt; over the domain, i.e., compute the weighted residual&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int\limits_{\Omega} wR\, d\Omega = \int\limits_{\Omega} w\mathcal{A}\hat{u}\, d\Omega - \int\limits_{\Omega} wf\, d\Omega \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For positive-definite operators we can write $\mathcal{A} = \mathcal{T}^*\mathcal{T}$ ($\mathcal{T}^*$ is the adjoint), thus&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int\limits_{\Omega} wR\, d\Omega = \int\limits_{\Omega} \mathcal{T}^*w\mathcal{T}\hat{u}\, d\Omega - \int\limits_{\Omega} wf\, d\Omega \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this last step is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Green%27s_theorem&quot; rel=&quot;nofollow&quot;&gt;Green's theorem&lt;/a&gt;. Depending on the constraints that you impose on your functions $w$ and $u$ you can have different version of the FEM: Galerkin, Petrov-Galerkin, Bubnov-Galerkin. The most common case is the Galerkin FEM, when the two functions are the same and you end up with a &lt;em&gt;variational&lt;/em&gt; formulation of your differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a (static) membrane, we can compute the &lt;em&gt;Lagrangian&lt;/em&gt; for the system as&#xA;$$\Pi[u] = \int\limits_\Omega (\nabla u)^2\, d\Omega - \int\limits_\Omega uf\, d\Omega$$&#xA;the solution is then given for the $u$ that makes&#xA;$$\delta \Pi[u] = 0$$&#xA;and for a given set of approximate solutions $\hat{u}$ (a base) we can find the best one in the same way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other methods where we can find these functionals are &lt;a href=&quot;http://en.wikipedia.org/wiki/Hartree%E2%80%93Fock_method&quot; rel=&quot;nofollow&quot;&gt;Hartree-Fock Methods&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Density_functional_theory&quot; rel=&quot;nofollow&quot;&gt;Density functional theory&lt;/a&gt; (DFT). In these methods one changes the Schrödinger equation for a variational formulation where the &lt;em&gt;electron-electron&lt;/em&gt; interaction is replaced for the interaction of the electrons with a mean field. Something similar is done here, you propose a set of functions (a base) and then find the coefficients for the combinations of your functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of DFT, the functional looks like&#xA;$$E[n] = T[n] + U[n] + \int\limits_{\mathbb{R}^3} V(\mathbf{r})n(\mathbf{r})\, d^3\mathbf{r}$$&#xA;$n$ is the density function (unknown), $T$ is the kinetic energy functional, $U$ is the potential energy functional and the last term is the one for the interaction with the mean field. In this case you should solve in a self-consistent way because the function $n$ is not known before hand (what makes the problem non-linear).&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-07-31T00:39:11.503" LastActivityDate="2015-07-31T00:39:11.503" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19069" PostTypeId="1" CreationDate="2015-03-04T20:10:59.630" Score="4" ViewCount="1346" Body="&lt;p&gt;I know what vectorizing is, and I know what pipelining is.  I assume &quot;vector pipelining&quot; might mean vectorizing in such a way that vector instructions can be pipelined.  Is this a new or coming feature in modern architectures with a more specific meaning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update on March 5&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I consider this one pretty well-answered.  I indeed was unclear about whether I was looking for &quot;a new thing&quot; or the vector pipelining of yore (Sorry Brian!).  I encountered this phrase in the context of future hardware technology and wanted to know if I was missing something.  Thanks to everyone for their insight/contributions.  I'll be more careful with my questions in the future.&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2015-03-05T16:04:26.963" LastActivityDate="2015-03-05T16:04:26.963" Title="Is &quot;vector pipelining&quot; a thing?" Tags="|architecture|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19070" PostTypeId="2" ParentId="19069" CreationDate="2015-03-04T20:43:44.087" Score="3" Body="&lt;p&gt;I'm barely old enough to understand this, but my understanding is that vector pipelining in the old days allowed for one vector instruction in the pipeline to use the result from the vector instruction ahead of it. I believe that now on the most common vector architectures this is not possible. Vector instructions are pipelined, but only insofar as they are independent.  &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-03-04T20:43:44.087" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19071" PostTypeId="2" ParentId="19069" CreationDate="2015-03-04T21:23:00.543" Score="3" Body="&lt;p&gt;Instruction Level Parallelism (ILP) is complementary to vectorization, but both require sufficient cache and register space.  &lt;a href=&quot;http://dx.doi.org/10.1177/1094342012444795&quot; rel=&quot;nofollow&quot;&gt;Our paper&lt;/a&gt; (it's fun to learn that your own paper answers your question, eh?) analyzes this in detail for stencil operations on Blue Gene/P, combining vectorization and unroll/jam to expose sufficient parallelism while using registers effectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern hardware has a floating point instruction latency of a few cycles (more for GPUs).  For peak flops, you need to be able to issue FP instructions every cycle.  This can be done using ILP or using multiple hardware threads.  Hardware threads typically give you access to more registers, but have to share caches and cross-thread communication is relatively more expensive than data dependencies within a single thread.  Consequently, it is application- and hardware-dependent whether hardware threads are better or worse than ILP.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2015-03-04T21:23:00.543" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19073" PostTypeId="2" ParentId="19069" CreationDate="2015-03-05T03:44:56.857" Score="6" Body="&lt;p&gt;The term &quot;vector pipeline&quot; was used in the 1970's to describe vector processing at a time when a single vector instruction might (for example) compute the sum of two vectors of floating point numbers using a single pipelined floating point arithmetic unit.  Pairs of floating point numbers would be brought from memory into the pipelined arithmetic unit to be added together, and once the pipeline filled, you'd get one sum out per cycle and these sums would be streamed back into memory.  See this 1977 article by Ramamoorthy and Li:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.ece.ucdavis.edu/~vojin/CLASSES/EEC272/S2005/Papers/Ramamoorthy_mar77.pdf&quot; rel=&quot;nofollow&quot;&gt;http://web.ece.ucdavis.edu/~vojin/CLASSES/EEC272/S2005/Papers/Ramamoorthy_mar77.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Early supercomputers like the STAR, ASC, and CRAY-1 implemented vector processing in this way.  The basic approach can be modified by adding &quot;vector registers&quot; to store fixed length sections (say 64 elements long) of a vector.  Multiple pipelined ALU's can also be used to simultaneously perform the same or different operations on their own vectors.  The Cray-1 could &quot;chain&quot; together vector operations so that the output from one vector operation was fed into another operation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is somewhat different from the SSE and AVX instructions on modern x86 processors from Intel and AMD.  These processors include &quot;Single Instruction Multiple Data&quot; instructions.  The processors have multiple parallel floating point arithmetic units that can (for example) take four pairs of floating point numbers from two vector registers,  add them together in a single cycle and then store the result in another vector register.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the modern implementation of this is vastly different from the way things were done in the 1970's and 80's, from a programming point of view the idea of expressing your algorithm in terms of operations like vector addition is similar.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-03-05T05:53:36.500" LastActivityDate="2015-03-05T05:53:36.500" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19074" PostTypeId="2" ParentId="19069" CreationDate="2015-03-05T04:01:14.630" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://books.google.com/books?id=UkCjBQAAQBAJ&amp;amp;pg=PA18&amp;amp;lpg=PA18&amp;amp;dq=%22vector+pipelining%22&amp;amp;source=bl&amp;amp;ots=qH607a0wFS&amp;amp;sig=XPHQJwrQjT95KmYHni4EqYiEZqo&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ei=1c33VPX-HbWBsQSv5IKACg&amp;amp;ved=0CDEQ6AEwBDgK#v=onepage&amp;amp;q=%22vector%20pipelining%22&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;This obscure 1990 technical review from the U of Sydney&lt;/a&gt; refers to &quot;vector pipelining&quot; as a means of implementing SIMD under the hood — if you suppose that the processor doesn't have enough logic to execute a vector instruction in &quot;full width&quot; at one go, but that the instruction can be broken down into sub-operations (at a level below instruction set level), then there's a performance gain to be had by pipelining those sub-operations, so that processing &lt;em&gt;n&lt;/em&gt; times as much data takes less than &lt;em&gt;n&lt;/em&gt; times as long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate a little bit, suppose that the 8x8=8 bit operation &quot;swizzle&quot; takes 4 clocks, and can be conveniently broken down into four 1-clock stages. Our chip also has eight independent swizzle units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can swizzle one pair of values in 4 clocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With SIMD, we can swizzle up to 8 pairs of values (64x64=64 bits) by engaging all of the swizzle units at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without pipelining, a 16-wide (128x128=128) swizzle instruction would take 8 clocks, and a 32-wide (256x256=256) swizzle instruction would take 16 clocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With vector vector pipelining, the chip can introduce the first 8 operands to the first stage of the swizzle pipeline on clock 1 (to emerge on clock 4), the second set on clock 2 (to emerge on clock 5), etc. Assuming nil overhead of breaking up and reassembling the operands, and assuming I've understood this all right, then you can have a 16-wide swizzle in 5 clocks, and a 32-wide swizzle in 7.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether modern architectures have (or need) anything like this, I'm afraid I don't know.&lt;/p&gt;&#xA;" OwnerUserId="11564" LastActivityDate="2015-03-05T04:01:14.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19075" PostTypeId="1" CreationDate="2015-03-05T06:03:08.440" Score="6" ViewCount="1201" Body="&lt;p&gt;The step size is computed by solving&#xA;$$&#xA;(A + \mu I) h = -g&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could find in some literature that one can compute the step size by solving&#xA;$$&#xA;(A + \mu \operatorname{diag}(A) ) h = -g&#xA;$$&#xA;It is said that this is helpful for error valley problems, where the error surface at minima is flat and long. I am not able to decide whether the diagonal of $A$ should be used in general for all cases, or the identity matrix $I$ is more appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some information:&lt;br&gt;&#xA;Levenberg-Marquardt algorithm: An iterative technique that locates the minimum of a function that is expressed as the sum of squares of nonlinear functions.&lt;br&gt;&#xA;$g$: Gradient matrix (Jacobian x Function values)&lt;br&gt;&#xA;$A$: Approximate Hessian matrix (JacobianTransposed x Jacobian)&lt;br&gt;&#xA;$\mu$: Damping factor&lt;br&gt;&#xA;$h$: Step size  &lt;/p&gt;&#xA;" OwnerUserId="14727" LastEditorUserId="14727" LastEditDate="2015-03-17T10:58:30.833" LastActivityDate="2015-06-08T14:39:02.003" Title="Levenberg-Marquardt - What is preferable (A + mu.I) or (A + mu.diag[A])?" Tags="|optimization|numerical-analysis|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19076" PostTypeId="2" ParentId="19058" CreationDate="2015-03-05T07:03:16.400" Score="2" Body="&lt;p&gt;&lt;em&gt;(too many comments, so I'll continue here)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume you're referring to equation (4.40) in &lt;a href=&quot;http://www.dic.units.it/perspage/tonti/DEPOSITO/Initial.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.dic.units.it/perspage/tonti/DEPOSITO/Initial.pdf&lt;/a&gt;. In this case the answer is yes, it is possible: Due to the specific integral formulation based on convolution, the first variation of (4.40) leads to the initial value problem&#xA;$$ x'(t) = Ax(t)$$&#xA;with $x(0)$ given. You can then discretize this equation with a standard Galerkin method; the one you are trying to get is equivalent to a Crank-Nicolson time stepping method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, you can also go the other way around as you are proposing: Discretize the functional, and then take discrete variations. Depending on the discretisation, you end up with different time stepping methods, but often there's one choice that leads to a standard time stepping method such as Crank-Nicolson. Whether that is possible in this case, I'm not sure -- the Crank-Nicolson method is equivalent to a Petrov-Galerkin method (i.e., trial and test functions come from different spaces; continuous piecewise linear and discontinuous piecewise constant, in this case), which means there can be no variational formulation (since the test functions come from variations of the trial functions, which means they have to live in the same space).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As regards your specific question: No, this is unlikely to be true, since the initial condition shouldn't appear there. To get the next time step, you need to explicitly compute variations with respect to $\delta x_2$ (and so forth). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note that in the definition of your functional, $t$ should really be $T$ (the end time), otherwise the functional makes no sense. (This means that $f^T*g$ is not really the convolution of $f$ and $g$ (which would be a function), but a related bilinear form (and hence a real number).) That means you need to be careful when taking variations not to mix up $t$ and $T$. Also, the $x(0)$ in your formulation is a &lt;em&gt;fixed&lt;/em&gt; initial condition, better denoted by $x_0$. Since $x(0)=x_0$ is fixed, there can be no variation at $t=0$, i.e., $\delta x(0) = 0$.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-03-05T07:16:31.607" LastActivityDate="2015-03-05T07:16:31.607" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19077" PostTypeId="2" ParentId="19075" CreationDate="2015-03-05T07:17:12.243" Score="3" Body="&lt;p&gt;Using $\mu\operatorname{diag}\left(A\right)$ instead of $\mu I$ is in general superior. When you use $\mu I$ you have step width $\mu$ in every direction. If you use $\mu\operatorname{diag}\left(A\right)$, the $i$th component is $\mu a_{i,i}$, when $a_{i,i}$ is the main diagonal element of $A$ at position $\left(i,i\right)$. Therefor you have a variable step width in every direction. As Levenberg-Marquardt is essentially a mixture of Gauss–Newton and gradient descent depending on wether the damping parameter $\mu a_{i,i}$ is small or big, you can have this two algorithms combined in one iteration step for different directions. &lt;/p&gt;&#xA;" OwnerUserId="14251" LastActivityDate="2015-03-05T07:17:12.243" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19078" PostTypeId="1" CreationDate="2015-03-05T08:51:23.020" Score="3" ViewCount="1306" Body="&lt;p&gt;Just speaking from my experience in c++ - Its really tricky to take advantage of the SIMD capabilities of modern CPU's for more complex algorithms. If I see an opportunity from a high level OO perspective, I would then have to break the whole thing down to a very non-OO kind of code, babysitting every register individually even if they have no interaction with each other and not only the part that benefits from vectorizing but often the entire flow of data along with data formats have to be redesigned into a more basic and harder to manage form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The automatic vectorization tools(qvec/openmp) offer no benefit either since they can only deal with just the most basic data types and loops with no OO concepts supported at all and imo only succeed in obscuring the code and limiting your options over the manual way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, when coding for GPGPU's, you essentially get to do it in a perfectly OO manner(more/less, depending on the language) while focusing on the journey of one piece of data through the pipeline which just gets duplicated to every other piece of data, brilliant!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that gpu &quot;cores&quot; aren't the same as a SIMD unit on a cpu but from what I've gathered they are much closer to that than actual individual cores since they have to move in a kind of a lockstep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically, why is that? &lt;/p&gt;&#xA;" OwnerUserId="14729" LastActivityDate="2015-03-05T09:51:05.430" Title="Why is SIMD programming on such a low level in CPU's compared to GPU's?" Tags="|c++|vectorization|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="19079" PostTypeId="2" ParentId="19078" CreationDate="2015-03-05T09:44:42.073" Score="4" Body="&lt;p&gt;Before I try to answer your question let me comment on the words &quot;low level&quot; in your question statement.&#xA;In my opinion, I prefer not to say one programming model(one of SIMD and SIMT) is at a low level.&#xA;As an example, during my undergraduage diploma project I had experienced many low level issues about GPU programming, which I have not encountered in CPU programming. Most of these issues involve the memory architecture of GPU, such as memory coalescing access, bank conflict, etc. You can search these topics to get a feeling of low-level-ness from GPU programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And now return to your question, by &quot;low level&quot; I assume you mean that you have to refactor your code or even redesign your algorithm to fit the SIMD model, while on GPU(e.g. using CUDA) it &lt;em&gt;may&lt;/em&gt; not require too much work like that.&#xA;In some cases your source code for CPU can even compiled without any errors using CUDA compiler. But to vectorize a serial program in SIMD you have to translate all branching code(if, while block).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am not quite sure of what you mean by &quot;OOP&quot; in your question statement. Please tell me if I made a wrong assumption.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is that GPU programming(CUDA) uses a different model from the one used in CPU programming. The model for GPU programming is called SIMT. T is for thread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In SIMD model, all arithmetic operations have to performed in a synchronized way. So any branching execution is not allowed.&#xA;While in CUDA, branching is allowed in term of CUDA syntax, and more importantly, it is supported by hardware architecture and CUDA runtime. When a branching execution happens in CUDA/SIMT, the thread manager will coordinate the execution: those threads with identical execution path will be executed, and the diverse threads will be queued for later execution. During this process, other independent threads may be brought in to keep a high throughput.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the SIMT model helps you to handle the branching execution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I would like to let you know that you can programming SIMD in a &quot;high level&quot; by introducing another syntax layer.&#xA;You can try &lt;a href=&quot;https://ispc.github.io/&quot; rel=&quot;nofollow&quot;&gt;https://ispc.github.io/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7258" LastActivityDate="2015-03-05T09:44:42.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19080" PostTypeId="2" ParentId="19078" CreationDate="2015-03-05T09:51:05.430" Score="2" Body="&lt;p&gt;I think you are comparing using high-level libraries built on GPGPU techniques, like C++ AMP, to programming SIMD at the lowest possible assembly language or intrinsics level. That's not a fair comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you specifically mentioned C++ AMP, let me use a basic example to argue that your premise isn't true. The &lt;a href=&quot;https://msdn.microsoft.com/en-us/library/hh265136.aspx&quot; rel=&quot;nofollow&quot;&gt;introductory AMP example&lt;/a&gt; shows how to parallelize the following simple vector addition using AMP:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;void StandardMethod() {&#xA;&#xA;    int aCPP[] = {1, 2, 3, 4, 5};&#xA;    int bCPP[] = {6, 7, 8, 9, 10};&#xA;    int sumCPP[5];&#xA;&#xA;    for (int idx = 0; idx &amp;lt; 5; idx++)&#xA;    {&#xA;        sumCPP[idx] = aCPP[idx] + bCPP[idx];&#xA;    }&#xA;&#xA;    for (int idx = 0; idx &amp;lt; 5; idx++)&#xA;    {&#xA;        std::cout &amp;lt;&amp;lt; sumCPP[idx] &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To do this, you have to use an AMP library function and write your loop in a specific way. What if we instead want to parallelize this loop using SIMD instructions on the CPU? We only have to compile it with a reasonably modern compiler and suitable flags!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compiling the above code with gcc 4.8.3 on Linux and disassembling it, I obtain&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ gcc -O3 -c vectest.cpp &amp;amp;&amp;amp; objdump -M intel -d vectest.o &#xA;&#xA;vectest.o:     file format elf64-x86-64&#xA;&#xA;&#xA;Disassembly of section .text:&#xA;&#xA;0000000000000000 &amp;lt;_Z14StandardMethodv&amp;gt;:&#xA;   0:   55                      push   rbp&#xA;   1:   53                      push   rbx&#xA;   2:   48 83 ec 68             sub    rsp,0x68&#xA;   6:   c7 04 24 01 00 00 00    mov    DWORD PTR [rsp],0x1&#xA;   d:   c7 44 24 04 02 00 00    mov    DWORD PTR [rsp+0x4],0x2&#xA;  14:   00 &#xA;  15:   48 8d 5c 24 40          lea    rbx,[rsp+0x40]&#xA;  1a:   c7 44 24 08 03 00 00    mov    DWORD PTR [rsp+0x8],0x3&#xA;  21:   00 &#xA;  22:   c7 44 24 0c 04 00 00    mov    DWORD PTR [rsp+0xc],0x4&#xA;  29:   00 &#xA;  2a:   48 8d 6c 24 54          lea    rbp,[rsp+0x54]&#xA;  2f:   66 0f 6f 04 24          movdqa xmm0,XMMWORD PTR [rsp]&#xA;  34:   c7 44 24 20 06 00 00    mov    DWORD PTR [rsp+0x20],0x6&#xA;  3b:   00 &#xA;  3c:   c7 44 24 24 07 00 00    mov    DWORD PTR [rsp+0x24],0x7&#xA;  43:   00 &#xA;  44:   c7 44 24 28 08 00 00    mov    DWORD PTR [rsp+0x28],0x8&#xA;  4b:   00 &#xA;  4c:   c7 44 24 2c 09 00 00    mov    DWORD PTR [rsp+0x2c],0x9&#xA;  53:   00 &#xA;  54:   66 0f fe 44 24 20       paddd  xmm0,XMMWORD PTR [rsp+0x20]&#xA;  5a:   66 0f 7f 44 24 40       movdqa XMMWORD PTR [rsp+0x40],xmm0&#xA;  60:   c7 44 24 10 05 00 00    mov    DWORD PTR [rsp+0x10],0x5&#xA;  67:   00 &#xA;  68:   c7 44 24 30 0a 00 00    mov    DWORD PTR [rsp+0x30],0xa&#xA;  6f:   00 &#xA;  70:   c7 44 24 50 0f 00 00    mov    DWORD PTR [rsp+0x50],0xf&#xA;  77:   00 &#xA;  78:   8b 33                   mov    esi,DWORD PTR [rbx]&#xA;  7a:   bf 00 00 00 00          mov    edi,0x0&#xA;  7f:   48 83 c3 04             add    rbx,0x4&#xA;  83:   e8 00 00 00 00          call   88 &amp;lt;_Z14StandardMethodv+0x88&amp;gt;&#xA;  88:   ba 01 00 00 00          mov    edx,0x1&#xA;  8d:   be 00 00 00 00          mov    esi,0x0&#xA;  92:   48 89 c7                mov    rdi,rax&#xA;  95:   e8 00 00 00 00          call   9a &amp;lt;_Z14StandardMethodv+0x9a&amp;gt;&#xA;  9a:   48 39 eb                cmp    rbx,rbp&#xA;  9d:   75 d9                   jne    78 &amp;lt;_Z14StandardMethodv+0x78&amp;gt;&#xA;  9f:   48 83 c4 68             add    rsp,0x68&#xA;  a3:   5b                      pop    rbx&#xA;  a4:   5d                      pop    rbp&#xA;  a5:   c3                      ret    &#xA;&#xA;Disassembly of section .text.startup:&#xA;&#xA;0000000000000000 &amp;lt;_GLOBAL__sub_I__Z14StandardMethodv&amp;gt;:&#xA;   0:   48 83 ec 08             sub    rsp,0x8&#xA;   4:   bf 00 00 00 00          mov    edi,0x0&#xA;   9:   e8 00 00 00 00          call   e &amp;lt;_GLOBAL__sub_I__Z14StandardMethodv+0xe&amp;gt;&#xA;   e:   ba 00 00 00 00          mov    edx,0x0&#xA;  13:   be 00 00 00 00          mov    esi,0x0&#xA;  18:   bf 00 00 00 00          mov    edi,0x0&#xA;  1d:   48 83 c4 08             add    rsp,0x8&#xA;  21:   e9 00 00 00 00          jmp    26 &amp;lt;_GLOBAL__sub_I__Z14StandardMethodv+0x26&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the compiler has automatically used MMX instructions to optimize your loop, and without having to add any CPU- or library-specific annotations. So I'd say that your claim that using SIMD is more difficult than using GPGPU techniques in high-level code is not true -- quite the contrary.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-03-05T09:51:05.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19081" PostTypeId="2" ParentId="19075" CreationDate="2015-03-05T11:23:14.387" Score="7" Body="&lt;p&gt;This is rather a note than an answer, but you can't always use the form with $\mu \text{diag}(A)$. In particular, if you have a case where $A$ has zeros or negative elements on the diagonal, then it is no longer guaranteed that you get a descent direction and, consequently, you may not converge.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-05T11:23:14.387" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19082" PostTypeId="1" CreationDate="2015-03-05T14:16:01.637" Score="7" ViewCount="450" Body="&lt;p&gt;I am trying to solve a multiphysic problem using finite elements and a Newton Raphson solution scheme. I have two non-linear subsystems that are coupled bi-directionally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first subsystem includes an equation that I solve for $p_f$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla u - \nabla a^3 \nabla p_f = \nabla a^3 B$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation becomes very non-linear, due to the parameter $a$ with an exponent of 3!&#xA;$B$ are body forces and $u$ is the solution of the &lt;em&gt;second&lt;/em&gt; subsystem that I am trying to solve. This second subsystem includes the Stokes equation that depends on the gradient of $p_f$. Therefore, both system are coupled bi-directionally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a Newton-Raphson solution scheme for each subsystem. First, I solve for $p_f$. After convergence occurred, I pass this solution to the second subsystem, which I solve using a Newton Raphson scheme, too. Then, the solution $u$ is passed to the first subsystem ... until the solutions of both subsystems do not change anymore. See the flow chart below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TjvE1.png&quot; alt=&quot;Flow Chart&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This procedure works very well so far. However, if the values of the parameter $a$ become very low (thus the gradient of $a$ becomes very, very low), there is no convergence anymore! The reason seems to be that the coupling between both subsystem becomes very strong. A small change in one subsystem creates a significant changes in the other one (especially where the gradient of $a$ is very low). I tried a variety of good initial conditions, however, since the first subsystem reacts very sensitiv, convergence has been impossible so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the figure below, you can see how the solution of this approach (&quot;Segregated Approach&quot;-blue line) is different from the &quot;true&quot; solution (&quot;Fully Coupled&quot;-red line). The &quot;true solution&quot; is the result of a benchmark, which I calculated by solving a single system of equations that includes the equations of both subsystems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ee5OI.png&quot; alt=&quot;Differences in solution&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The greatest differences between both solutions are located where the gradient of $a$ becomes very small. See the figure below (in this figure the values of $p_f$ and $a$ are normalized to range between 0 and 1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/c8vNY.png&quot; alt=&quot;Parameter $a$ and the &amp;#39;wrong&amp;#39; solution for $p_f$&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So: Is a Newton-Raphson solution scheme the wrong method for solving a strongly coupled system, if there are low gradients in a subsystem? Could my method be fixed to solve the problem that I described? If yes, how? Are there alternative solution schemes for this problem? &lt;/p&gt;&#xA;" OwnerUserId="14732" LastActivityDate="2017-02-21T16:07:46.070" Title="Strong coupling of a non-linear multiphysic problem: failure with Newton Raphson method" Tags="|convergence|nonlinear-equations|newton-method|coupling|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19083" PostTypeId="1" AcceptedAnswerId="19093" CreationDate="2015-03-05T19:47:56.580" Score="5" ViewCount="395" Body="&lt;p&gt;I am trying to integrate a 2nd order ODE with potential several singularities using the lsoda solver wrapped in scipy.integrate.ode().&#xA;I would like to put an error bar on the solution or at least estimate of upper bound to the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For understand how to calculate truncation errors for simple finite difference methods. But LSODA seems to be more complicated from the &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy documentation&lt;/a&gt;, the solver uses a adaptive stepsize and automatically switches between a stiff and non-stiff method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Options to set the absolute tolerance, relative tolerance and maximum step size exist but I don't understand exactly how they would relate to the error in my solution from this &lt;a href=&quot;https://computation.llnl.gov/casc/nsde/pubs/u113855.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;document&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few cases where analytic solutions are possible. e.g. a case that I debugged in a previous &lt;a href=&quot;https://scicomp.stackexchange.com/questions/18845/large-residual-when-integrating-2nd-order-ode-close-to-singularity-with-scipy-od&quot;&gt;question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is Newcomb's Euler-Lagrange equation from the field of plasma physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d}{dr}(f \frac{d\xi}{dr}) - g \xi = 0$$&#xA;or as a set of first order ODE's:&#xA;$$y_0 = \xi $$&#xA;$$y_1 = f \xi' $$&#xA;$$y_0' = \frac{y_1}{f} $$&#xA;$$y_1' = y_0 g $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ and $g$ are complicated expressions of magnetic fields and pressure gradients.&#xA;For the more complicated cases, I use splines generated with scipy.interpolate.InterpolateUnivariateSpline() to describe the magnetic fields and pressure gradients.&#xA;$f$ and $g$ can vary rapidly and $f$ can be zero at several locations resulting in singularities. I use Frobenius expansions to find the solutions close to singularties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a short example code for a simple case of $f$ and $g$, where there is only a singularity at $r=0$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.integrate import ode&#xA;&#xA;# Setup ODE system&#xA;def f(r):&#xA;   return r&#xA;&#xA;def g(r):&#xA;    return -1 + r + 1./r&#xA;&#xA;def der(r, y):&#xA;    y_der = np.zeros(2)&#xA;    y_der[0] = y[1]/f(r)&#xA;    y_der[1] = g(r)*y[0]&#xA;    return y_der&#xA;&#xA;#Integrate&#xA;integrator = ode(der)&#xA;integrator.set_integrator('lsoda') &#xA;r_init = 1E-3&#xA;init = [r_init, 1.]&#xA;integrator.set_initial_value(init, t=r_init)&#xA;r = np.linspace(r_init, 1., 100)&#xA;results = np.zeros((2, r.size))&#xA;results[:,0] = init&#xA;for i, position in enumerate(r[1:]):&#xA;    integrator.integrate(position)&#xA;    results[:, i+1] = integrator.y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14391" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-03-06T20:17:06.047" Title="Quantify integration error of scipy ode / ODEPACK" Tags="|ode|error-estimation|scipy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19084" PostTypeId="1" CreationDate="2015-03-05T22:25:55.373" Score="1" ViewCount="29" Body="&lt;p&gt;I am trying to make a speech synthesizer using a Hidden Markov Model. I read &lt;a href=&quot;http://en.wikipedia.org/wiki/Speech_synthesis#HMM-based_synthesis&quot; rel=&quot;nofollow&quot;&gt;the wikipedia page&lt;/a&gt; and several whitepapers and presentations (&lt;a href=&quot;http://homepages.inf.ed.ac.uk/ckiw/rpml/HMM_speech_synthesis.pdf&quot; rel=&quot;nofollow&quot;&gt;such as this one&lt;/a&gt;) but I am still not sure how this algorithm can be used to generate waveforms. Can someone explain to me how the algorithm can be applied to this problem?&lt;/p&gt;&#xA;" OwnerUserId="14736" LastActivityDate="2015-03-05T22:25:55.373" Title="Using HMM for speech synthesis" Tags="|algorithms|python|" AnswerCount="0" CommentCount="2" ClosedDate="2015-03-07T16:06:33.760" ContentLicense="CC BY-SA 3.0" />
  <row Id="19088" PostTypeId="1" AcceptedAnswerId="19094" CreationDate="2015-03-06T16:11:48.360" Score="2" ViewCount="539" Body="&lt;p&gt;The problem that I'm trying to solve is as follows. &#xA;A server has at its disposal a pool of video encoders (each encoder has different settings and causes a different load on the server) that can be enabled or disabled. All clients need to be connected to exactly one of those encoders, but one encoder can accomodate multiple clients. The objective is that the clients are connected to an encoder that suits their bitrate capacity (so an encoder that sends approx. the same bitrate as the client can handle is a 'good' encoder for that client). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a rather easy linear problem and the objective function can be described as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\max{ \sum_{j \in encoders} \sum_{i \in clients} u_{ij} x_{ij}}&#xA;$$&#xA;with $u_{ij}$ being a utility that is high when a client is assigned a 'good' encoder and $x_{ij}$ a binary variable that is 1 if client i is connected to server j, and 0 otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the constraints is of course the constraint that each client can only be connected to 1 encoder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sum_{j \in encoders} x_{ij} = 1,  \forall i \in clients&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem now is that there is also a constraint that says that the sum of the loads of all the encoders that are enabled cannot be bigger than the maximum load of the server.&#xA;The encoders that are enabled are the ones that are connected to at least 1 client. So every encoder that has at least one $x_{ij}=1$ is enabled and brings an extra load $load_{j}$ on the server. The constraint is thus that the sum of all these $load_j$ of the enabled encoders should be smaller than the maximum load on the server.&#xA;The way i translated this into a constraint is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sum_{j \in encoders} load_j *\max_{i \in clients}{x_{ij}} \leq load_{max}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. if encoder $3$ is connected to at least one client, at least one of the $x_{i3}$ of that encoder will be one, and thus the max term will be equal to one, and encoder $3$ contributes $load_3$ to the sum. If no client is connected to the encoder, the max term will be equal to zero and this encoder will not contribute to the sum. This is exactly the behaviour that I need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: is it really necessary to write this last constraint this way (using the max operator, which introduces non-linearity)? Or am i overlooking something and is it possible to write this constraint in a simpler/linear way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is not possible to write this in a linear way, which optimization framework or toolbox could I possibly use to implement and solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="14741" LastEditorUserId="14741" LastEditDate="2015-03-06T19:28:51.667" LastActivityDate="2015-03-06T21:30:21.730" Title="Constraint containing 'max' in linear program unnecessary?" Tags="|optimization|nonlinear-programming|constrained-optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19089" PostTypeId="1" CreationDate="2015-03-06T16:15:28.203" Score="0" ViewCount="1254" Body="&lt;p&gt;I am solving the following matrix equation for $\mathbf{x}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(J^{\mathbf{T}}J)\mathbf{x}=J^{\mathbf{T}}\mathbf{r}$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$J$ is $m\times n$ matrix&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{x}$ is vector of size $n$&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{r}$ is vector of size $m$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Furthermore, $m\gg n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is sometimes the case that $n$ is quite large (e.g. 1000) and $J$ contains values like 500. Standard multiplication is then impossible to do even with double-precision numbers as the values exceed $10^{128}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that scaling $J$ by a nonzero value has predictable effect on the result (it has the same effect as multiplying both sides of the above equation by scalars), so I am thinking about computing a scale factor so that the above equation can be solved without overflows. However, too small scale factor can also endanger accuracy of the matrix multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, $J$ can contain both very small and very large values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is how to choose some scaling factor $s$ so that we can substitute $J$ by $sJ$ and avoid too big numbers and also keep high accuracy of the multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;make average/median of elements in $J$ equal to e.g. $10^3$&lt;/li&gt;&#xA;&lt;li&gt;scale $J$ according to sum of its biggest column&lt;/li&gt;&#xA;&lt;li&gt;find some &quot;preconditioner&quot; matrix $A$ and substitute $J$ for $AJ$ ... the preconditioned solution can be obtained using inverse transform, but $A$ can get very large as it has to be $m\times m$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="4591" LastEditorUserId="1804" LastEditDate="2016-05-31T21:32:34.317" LastActivityDate="2016-05-31T21:32:34.317" Title="Avoid arithmetic overflow in matrix multiplication" Tags="|linear-algebra|linear-solver|stability|preconditioning|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19090" PostTypeId="2" ParentId="19089" CreationDate="2015-03-06T16:37:40.927" Score="4" Body="&lt;p&gt;This isn't a direct answer to your question but rather an alternative approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like you are solving a least squares (LS) problem using the normal equations.&#xA;The normal equations are known to be a poor way to solve least squares problems&#xA;because $J^T J$ is often very ill-conditioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why don't you simply solve the LS problem directly ($J x = r$) using, for example, a QR&#xA;decomposition? The LAPACK routine, DGELS, will do that for you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/lapack/lug/node27.html&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lapack/lug/node27.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are interfaces to this routine in Octave, MATLAB, Python, and other scripting &#xA;languages so you don't need to use Fortran/C/C++ if you'd prefer not to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in reading more about using QR factorizations to solve LS problems, this chapter in Cleve Moler's book provides a very readable introduction:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/moler/leastsquares.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/moler/leastsquares.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-03-06T16:37:40.927" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19091" PostTypeId="1" AcceptedAnswerId="19092" CreationDate="2015-03-06T17:49:16.730" Score="9" ViewCount="2253" Body="&lt;p&gt;I have a rather specific question regarding the condition number. I run FEM simulations which have multiple length scales to them which results in a huge disparity between the largest entries and the smallest entries in my matrix. The condition number can get as high as 10^15 in some circumstances. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In numerical analysis I often see the error bound for the condition number as it applies to error in the computed solution using direct methods. My curiosity is whether this logic applies to error in an iterative-type solver like CG, or GMRES as well. I know that the convergence rate is impacted by the eigenvalues of the matrix, and I do notice huge speed losses when running problems of this type. But, I am uncertain as to the accuracy. Any help would be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="11658" LastEditorUserId="123" LastEditDate="2015-04-24T08:11:06.387" LastActivityDate="2015-04-24T23:09:52.760" Title="Does the matrix condition number affect accuracy of iterative linear solvers?" Tags="|linear-algebra|numerical-analysis|linear-solver|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19092" PostTypeId="2" ParentId="19091" CreationDate="2015-03-06T20:00:27.327" Score="13" Body="&lt;p&gt;Ill conditioning is a feature of the system of the equations, not of the algorithm used to solve the system of equations.  If your systems are that badly conditioned ($10^{15}$), then you can expect the solution to the system to be extremely sensitive to any perturbation of the problem data, even if the solution is done in extremely high precision (e.g. 500 digits) arithmetic using direct factorization.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your iterative method is unlikely to converge to a solution in any reasonable amount of time.  Even if you were willing to wait around for centuries, the solution you got would still be extremely sensitive to any perturbations in the problem data.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="713" LastEditDate="2015-03-06T20:07:31.027" LastActivityDate="2015-03-06T20:07:31.027" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19093" PostTypeId="2" ParentId="19083" CreationDate="2015-03-06T20:17:06.047" Score="2" Body="&lt;p&gt;If I remember correctly, you have some options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You could use an integrator that calculates rigorous bounds via interval arithmetic. For instance, Taylor models are such an approach, and you could look at work by &lt;a href=&quot;http://bt.pa.msu.edu/index.htm&quot; rel=&quot;nofollow&quot;&gt;Berz and Makino&lt;/a&gt;. You'll need to use a different integrator, and probably change your code substantially to enable use of interval arithmetic data types (and operations).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could estimate the error using &lt;em&gt;a posteriori&lt;/em&gt; approaches, of which there are several. &lt;a href=&quot;http://www.mcs.anl.gov/~emconsta/Repository/Papers/glb-err-preprint-20140820.pdf&quot; rel=&quot;nofollow&quot;&gt;This preprint by Emil Constantinescu&lt;/a&gt; has a comprehensive literature review. One of these approaches (a reference by Robert Skeel in Emil's paper reviews thirteen methods, so there are others) is to derive a defect equation that approximates the error, and integrate that in time, so you have to solve your ODE at least twice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Based on global error estimates, you could actually try to control the global error in the integration. &lt;a href=&quot;http://www.cs.ucsb.edu/~cse/Files/condition_ode032.pdf&quot; rel=&quot;nofollow&quot;&gt;Cao and Petzold&lt;/a&gt; describe such an approach using DASPK. In order to use this method, you need to use a solver capable of adjoint sensitivity analysis. Cao and Petzold use DASPK, but you could also use CVODES. This is one instance of the approach DavidKetcheson remarked on.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-03-06T20:17:06.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19094" PostTypeId="2" ParentId="19088" CreationDate="2015-03-06T21:30:21.730" Score="1" Body="&lt;p&gt;It should be possible to introduce an auxiliary variable $p_{j}$ so that $p_{j} \geq x_{ij}$ for all $i$. If $p_{j}$ is binary and the $x_{ij}$ are binary, you should be able to construct, at worst, a relaxation. It might be an equivalent formulation, although that depends on the whole formulation, and you should prove that. You might even be able to relax the binary constraint on the $p_{j}$ so that they are continuous variables on $[0,1]$.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-03-06T21:30:21.730" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19095" PostTypeId="1" CreationDate="2015-03-06T22:48:33.240" Score="0" ViewCount="34" Body="&lt;p&gt;I'm looking at your standard definition of the TSP.... (see wikipedia).... and in the statement leading up to the definition, it states that we must return to the city we began from. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which constraint makes sure that this is the case? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Travelling_salesman_problem#Integer_linear_programming_formulation&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Travelling_salesman_problem#Integer_linear_programming_formulation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14744" LastActivityDate="2015-03-06T23:17:17.263" Title="Travelling Salesman Problem - where's the &quot;return to city&quot;-constraint?" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ClosedDate="2015-03-07T16:06:43.507" ContentLicense="CC BY-SA 3.0" />
  <row Id="19096" PostTypeId="2" ParentId="19095" CreationDate="2015-03-06T23:17:17.263" Score="1" Body="&lt;p&gt;The constraint &#xA;$\sum_{i=1,i \neq j}^n x_{ij}=1$ makes sure that for every city there is a next city and the constraint $\sum_{j=1,j \neq i}^n x_{ij}=1$ guarantee that for each city there exists a previous one. So necessarely there is a &quot;return to the city&quot;.&lt;/p&gt;&#xA;" OwnerUserId="9749" LastActivityDate="2015-03-06T23:17:17.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19097" PostTypeId="1" CreationDate="2015-03-07T02:37:28.920" Score="1" ViewCount="295" Body="&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt;&#xA;I have translated &lt;strong&gt;Jacobian-Free Newton-Krylov&lt;/strong&gt; solver written by&#xA;&lt;a href=&quot;http://www4.ncsu.edu/%7Ectk/newton/SOLVERS/nsoli.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;C. T. Kelley&lt;/em&gt;&lt;/a&gt; to Fortran and now want to parallelize it on a shared-memory system with OpenMP. In addition, I want to precondition the system with ILU0 or ILUT preconditioners.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Considered solution:&lt;/strong&gt;&#xA;I want to use FGMRES from Intel MKL library - I expect that it is highly optimized and threaded. Since it does not support complex numbers, I will follow &lt;a href=&quot;https://software.intel.com/en-us/forums/topic/283225&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel's&lt;/a&gt; solution to split the problem:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left(\array{A_r &amp;amp; -A_i \\A_i &amp;amp;A_r}\right)\left(\array{x_r\\x_i}\right)=\left(\array{f_r\\f_i}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For ILU0 precondition, I have decided to use a piece of code from &lt;em&gt;Yousef Saad&lt;/em&gt;'s book &lt;strong&gt;Iterative Methods for Sparse Linear Systems Second Edition&lt;/strong&gt; and parallelize it.&lt;/p&gt;&#xA;&lt;p&gt;Then I will use &lt;code&gt;?getri&lt;/code&gt; from MKL which &amp;quot;Computes the inverse of an LU-factored general matrix&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Does it sounds reasonable? Any other solver for shared-memory systems?&lt;/p&gt;&#xA;" OwnerUserId="14746" LastEditorUserId="20688" LastEditDate="2021-12-24T05:25:24.837" LastActivityDate="2021-12-24T05:25:24.837" Title="OpenMP threaded nonlinear solver for complex numbers" Tags="|nonlinear-equations|iterative-method|preconditioning|gmres|complex-arithmetic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19098" PostTypeId="2" ParentId="19097" CreationDate="2015-03-07T03:37:41.090" Score="3" Body="&lt;p&gt;What other preconditioners have you considered? In my experience with ILU, it's more difficult to parallelize effectively than other methods like alternating Schwarz or algebraic multigrid. As always, your mileage may vary: experiment and profile before trusting nincompoops on the internet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using alternating Schwarz breaks up the problem into several sub-problems that are solved serially, for which ILU + a Krylov subspace solver or a direct LU factorization can be very effective depending on size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As regards OpenMP, the best-of-breed libraries (PETSc, Trilinos) for this sort of thing use message passing via MPI before shared-memory parallelism. The aforementioned libraries can link to Intel MKL;if you choose to use them, you can still take full advantage of all the optimized linear algebra kernels. However, you'll have to weigh the time sunk into learning how to use the library vs. how long it would take you to write it yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, shared memory parallelism lends itself like gangbusters to bugs that are &lt;em&gt;way&lt;/em&gt; harder to diagnose than message-passing. What happens when library A uses OpenMP and library B uses OpenMP and they short-circuit each other? That only becomes a consideration when you write big applications, but you never know when some script that you wrote because you couldn't sleep on a long flight metamorphoses into like... half of your Ph.D. thesis*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*Speaking from experience.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2015-03-07T04:20:31.487" LastActivityDate="2015-03-07T04:20:31.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19100" PostTypeId="1" CreationDate="2015-03-07T18:03:53.227" Score="5" ViewCount="161" Body="&lt;p&gt;I have to numerically solve a nonlinear partial integro-differential equation. This is my equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial y(x,t)}{\partial t}=\int_{-1/2}^{1/2} \frac{\pi\cos u}{\sin\pi u-\sin\pi x} \frac{\partial y(u,t)}{\partial u}\mathrm du-\sin\,y(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kernel has a singularity at $x=u$ and vanishes at the boundaries ($u=-1/2,1/2$). Answer $y$ is a function of $x,t$.&#xA;At the moment I am not worried about the initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice, the derivative inside the integral is a first derivation in spacial component and the equation has a non-linearity of $\sin(y(x,t))$. No boundary condition is needed to solve this equation (It's all been taken care of by the kerenl).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the possible numerical recipes to solve this equation?&lt;/p&gt;&#xA;" OwnerUserId="14751" LastEditorUserId="14751" LastEditDate="2015-08-28T17:24:05.867" LastActivityDate="2015-08-28T17:24:05.867" Title="numerical analysis of a partial integro-differential equation" Tags="|pde|finite-difference|nonlinear-equations|time-integration|integral-equations|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19102" PostTypeId="1" CreationDate="2015-03-08T04:33:45.760" Score="1" ViewCount="667" Body="&lt;p&gt;I edited my question trying to make it as short and precise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am developing a prototype of a facial recognition system for my Graduation Project. I use Eigenface and my main source is the document Turk and Pentland. It is available here: &lt;a href=&quot;http://www.face-rec.org/algorithms/PCA/jcn.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.face-rec.org/algorithms/PCA/jcn.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My doubts focus on step 4 and 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can not correctly interpret the number of thresholds: If two types of thresholds, or only one (Notice that the text speaks of two types but uses the same symbol). And again, my question is whether this (or these) threshold(s) is unique and global for all person or if each person has their own default.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the steps to be calculated until an matrix O() of classes with weights or weighted. So this matrix O() is of dimension M'x P. Since M' equal to the amount of eigenfaces chosen and P the number of people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What follows and confuses me. He speaks of two distances: the distance of a class against another, and also from a distance of one face to another. I call it D1 and D2 respectively. NOTE: In the training set there are M images in total, with F = M / P the number of images per person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that threshold(s) should be chosen empirically. But there must be a way to approximate. I was initially designing a matrix of distances D1() of dimension PxP. Where the row vector D(i) has the distances from the vector average class O(i) to each O(j), j = 1..P. Ie a &quot;all vs all.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Until I came here, and what follows depends on whether I should actually choose a single global threshold for all. Or if I should be chosen for each individual value. Also not if they are 2 types: one for distance classes, and one for distance faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a theory as could proceed but not so supported by the concepts of Turk:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Stage Pre-Test:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gender two matrices of distances D1 and D2:&#xA;In D1 would be stored distances between classes, and in D2 distances between faces. This basis of the matrices W and A respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, as indeed in the training set are P people, taking the F vectors columns D1 for each person and estimate a threshold T1 was in range [Min, Max]. Thus I will have a T1(i), i = 1..P&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Separately have a T2 based on the range [Min, Max] out of all the matrix D2. This define is a face or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step Test:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Buid a test set of image with a 1 image for each known person&#xA;Itest = {Itest(1) ... Itest(P)}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every image Itest(i) test:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the space face Atest = Itest - Imean&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the weight vector Otest = UT * Atest&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculating distances:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;dist1(j) = distance(Otest, O (j)), j = 1..P&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Af = project(Otest, U)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;dist2 = distance(Atest, Af)&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Evaluate recognition:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;MinDist = Min(dist1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each j = 1..P&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If dist2 &gt; T2 then &quot;not is face&quot; else:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If MinDist &amp;lt;= T1(j) then &quot;Subject identified as j&quot; else &quot;subject unidentified&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I take account of TFA and TFR and repeat the test process with different threshold values until I find the best approach gives to each person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Already defined thresholds can put the system into operation unknown images. The algorithm is similar to the test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I get out of &quot;script&quot; of the official documentation but at least this reasoning is the most logical place my head. I wondered if I could give guidance.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i No more to say that has not already been said and that may help clarify things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone tell me if I'm okay tackled with my &quot;theory&quot;? I'm moving into my project, and if this is not the right way would appreciate some guidance and does not work and you wrong.&lt;/p&gt;&#xA;" OwnerUserId="14754" LastEditorUserId="14754" LastEditDate="2015-03-12T13:54:34.697" LastActivityDate="2015-03-12T13:54:34.697" Title="How to choose (and many) threshold in Eigenface?" Tags="|eigensystem|statistics|nearest-neighbors|pattern-recognition|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19103" PostTypeId="2" ParentId="17554" CreationDate="2015-03-08T11:20:45.433" Score="4" Body="&lt;pre&gt;&lt;code&gt;syms x&#xA;eq=cos(7*x)*exp(-2*x^2)*(1-2*x^2);&#xA;solve(eq==0,x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;ans =&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      pi/14&#xA;  2^(1/2)/2&#xA; -2^(1/2)/2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or if your version of Matlab is recent, you can do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;evalin(symengine,'solve(cos(7*x)*exp(-2*x^2)*(1-2*x^2)=0,x)') &#xA;&#xA;{-2^(1/2)/2, 2^(1/2)/2} union Dom::ImageSet(pi/14 + (pi*k)/7, k, Z_)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="727" LastActivityDate="2015-03-08T11:20:45.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19104" PostTypeId="1" CreationDate="2015-03-08T12:12:25.983" Score="2" ViewCount="136" Body="&lt;p&gt;It is known from Geometry that any 3 points determine a circle.  This is the problem of Appolonius &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/0/02/Apollonius_PPP_black.svg/100px-Apollonius_PPP_black.svg.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programming solutions try to &lt;em&gt;approximate&lt;/em&gt; using Bezier curves.  Can you not draw conics as Bezier curves?  I thought you could get an exact answer using enough control points.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/28229695/draw-2-bezier-curves-through-3-points-to-mimic-a-circle-arc&quot;&gt;https://stackoverflow.com/questions/28229695/draw-2-bezier-curves-through-3-points-to-mimic-a-circle-arc&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/4376749/drawing-a-circle-given-three-points&quot;&gt;https://stackoverflow.com/questions/4376749/drawing-a-circle-given-three-points&lt;/a&gt; [closed]&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/20314306/find-arc-circle-equation-given-three-points-in-space-3d&quot;&gt;https://stackoverflow.com/questions/20314306/find-arc-circle-equation-given-three-points-in-space-3d&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would like to be able to input the points $A,B,C$ and return the list of points on the arc $\overline{AB}$ on the circle.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It is possible to compute this formula exactly, such as using the determinant method (or &lt;a href=&quot;http://www.qc.edu.hk/math/Advanced%20Level/circle%20given%203%20points.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;others&lt;/a&gt;), but here I am interested in a &lt;strong&gt;spline based solution&lt;/strong&gt;.  This will be useful when I have many many measurements, known to fit a circle, but no one circle exactly fits all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left| \begin{array}{llll} x^2 + y^2 &amp;amp; x &amp;amp; y &amp;amp; 1 \\ &#xA;x_1^2 + y_1^2 &amp;amp; x_1 &amp;amp; y_1 &amp;amp; 1 \\&#xA;x_2^2 + y_2^2 &amp;amp; x_2 &amp;amp; y_2 &amp;amp; 1 \\&#xA;x_3^2 + y_3^2 &amp;amp; x_3 &amp;amp; y_3 &amp;amp; 1 \\\end{array}\right| = 0$$&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-03-08T12:12:25.983" Title="interpolate the circle going through three points using splines" Tags="|computational-geometry|b-spline|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19105" PostTypeId="1" AcceptedAnswerId="19121" CreationDate="2015-03-08T14:42:47.087" Score="1" ViewCount="319" Body="&lt;p&gt;Suppose a program was written in two distinct languages, let them be language X and language Y, if their compilers generate the same byte code, why I should use language X instead of the language Y? What would make a language better than the other to resolve a problem? What defines than one language is faster than other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the language speed... I asked that because often you see people say things like: &quot;C is the fastest language, ATS is a language fast as C&quot;. I was seeking to understand the &quot;fast&quot; definition for programming languages.&lt;/p&gt;&#xA;" OwnerUserId="14515" LastEditorUserId="14515" LastEditDate="2015-03-10T20:56:58.360" LastActivityDate="2015-03-10T20:56:58.360" Title="Languages Speed" Tags="|algorithms|compiling|" AnswerCount="3" CommentCount="3" ClosedDate="2015-03-11T02:15:40.803" ContentLicense="CC BY-SA 3.0" />
  <row Id="19108" PostTypeId="1" AcceptedAnswerId="19109" CreationDate="2015-03-08T18:23:47.003" Score="5" ViewCount="249" Body="&lt;p&gt;From the &lt;a href=&quot;https://github.com/dealii/dealii/wiki/Frequently-Asked-Questions&quot; rel=&quot;nofollow&quot;&gt;deal.II FAQ&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...quadrilaterals and hexahedra typically provide a significantly better &#xA;  approximation quality than triangular meshes with the same number of &#xA;  degrees of freedom; you therefore get more accurate solutions for the &#xA;  same amount of work...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How can this point be made more precise?&lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2015-03-18T22:34:56.487" LastActivityDate="2015-03-19T08:45:46.313" Title="How do hexahedral FEM meshes improve approximation quality per degree of freedom, compared to tetrahredal meshes?" Tags="|pde|finite-element|deal.ii|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19109" PostTypeId="2" ParentId="19108" CreationDate="2015-03-08T22:37:18.460" Score="6" Body="&lt;p&gt;Having written this entry, let me also answer you question :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue in essence boils down to the following: given a function $u(x)$ and its interpolation $u_h(x)$ onto either a triangular or quadrilateral mesh with the same number of unknowns, then which of the two is more accurate? In other words, is&#xA;$$&#xA;  \| u - I_h^\text{triangles} u \| &amp;lt; \| u - I_h^\text{quads} u \|,&#xA;$$&#xA;or is it the other way around. We know that they have the same asymptotic order ${\cal O}(h^{p+1})$ when using polynomials of order $p$ (in $L_2$). The question is simply which of the two has the smaller interpolation constant. The answer, in practice, is that the inequality above is wrong in general, and that the following is empirically true:&#xA;$$&#xA;  \| u - I_h^\text{triangles} u \| \approx 10 \| u - I_h^\text{quads} u \|.&#xA;$$&#xA;In other words, the extra polynomial $xy$ on quadrilaterals allows for a 10 times better approximation.&#xA;(I'm sure you can find counterexamples; this is just an empirical observation for &quot;typical&quot; solutions of PDEs.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-08T22:37:18.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19110" PostTypeId="2" ParentId="19055" CreationDate="2015-03-09T00:51:39.030" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;For hyperbolic PDEs often the specification of the boundary conditions tend to only be needed on one boundary.  For a system of equations like you have specified the boundary conditions needed are exactly the ones you mentioned.  When implementing these numerically we often specify the other boundary conditions as extrapolated boundary conditions, in the simplest case just copying the values from the domain into the ghost cells.&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;With specification of ghost cells calculation of the slope should be possible for all the boundaries of cells.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="567" LastActivityDate="2015-03-09T00:51:39.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19112" PostTypeId="1" CreationDate="2015-03-09T04:13:52.153" Score="2" ViewCount="264" Body="&lt;p&gt;For a very large mixed-poisson problem with lowest order Raviart-Thomas elements (RT0), I plan on using an iterative solver. However, this kind of problem is not positive-definite (saddle point problem IIRC), thus standard preconditioners (like jacobi) will not converge well for large-problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was told that the algebraic multigrid preconditioner is the one of the most common methods to achieving fast convergence for Krylov solvers. My question is, what would it look like for a mixed-poisson problem? Or if there are better preconditioning methods, do point them out to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2022-05-03T23:06:15.380" Title="Best preconditioner for mixed-poisson problem (RT0 elements)" Tags="|finite-element|iterative-method|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19113" PostTypeId="2" ParentId="19112" CreationDate="2015-03-09T04:50:15.257" Score="4" Body="&lt;p&gt;Algebraic Multigrid (AMG) methods have the fundamental limitation that, by and large, their current implementations only work reliable and well for elliptic or similar problems. For saddle point problems, you need to reduce things to blocks for which the AMG methods work. This could include the top left (mass matrix) block of the mixed formulation. In the end, for block systems such as the one you have, the best preconditioners work by operating on individual blocks of your matrix.&lt;/p&gt;&#xA;&lt;p&gt;I've collected some thoughts on this in lecture 38 at &lt;a href=&quot;http://www.colostate.tamu.edu/%7Ebangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.colostate.tamu.edu/~bangerth/videos.html&lt;/a&gt; . You can also look at the implementation of a (non-optimal) approach in lecture 21.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2022-05-03T23:06:15.380" LastActivityDate="2022-05-03T23:06:15.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19114" PostTypeId="2" ParentId="19105" CreationDate="2015-03-09T13:48:34.033" Score="2" Body="&lt;p&gt;The questions you ask, in general, are quite difficult to answer because there are just too many languages for so many things. Each language has been built either to satisfy some necessity or for some intended audience. Also, even for general-purpose languages I think these are built in a way it can serve better to a particular subgroup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, I don't know what are your expectations about using a programming language in general, so I will try to answer your question in a intuitive (maybe even wrong), and biased, way. First, lets talk about execution speed: speed and size is not directly related to the semantics of a language in particular, but on how its associated &lt;em&gt;compiler&lt;/em&gt; translates your source code into processor instructions. In the era of 256kb RAM mainframes, code efficiency was a must, thus most languages tried to stay as close to assembler code (faster, processor-dependent programming language) as possible. Two examples of this era are FORTRAN VI and K&amp;amp;R C. Nowadays, the gears are shifting toward  greater levels of abstraction (Java, C++, F2003, etc), and higher performance (F2008, UPC, C++, etc) languages, which intertwine new language semantics with system architectures. I think that we will get into a point in which making one &lt;em&gt;single&lt;/em&gt; processor faster would become very expensive due to the appearance of quantum effects, so the future might be going towards massively parallel systems (mainframes, again!) in order to be fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About which language is better: it depends. Depends on what field are you, on the existing/previous code you have to work on, and personal taste of course. For example, people on mathematics and/or working on FEM would tend to work more in languages that let have higher levels of abstraction (i.e. C++, pascal), whereas people working on the physics and/or engineering sciences tend to work more with &quot;procedural&quot; (not quite true, but...) languages like FORTRAN, UPC, or C. Why the difference? To be honest I don't know, but some people tend to be really opinionated about this. Of the five languages I have mentioned, you can do whatever the mathematician/physicist/engineer does, with relative ease. How do I know? Well, I decided to do a parallel 2-D Navier Stokes solver in FEM both in C++ and FORTRAN and, from the programming perspective, the difficulty was relatively the same (on the c++ code I got 130 lines more of code, but that’s a different story).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ADVICE&lt;/strong&gt;: try to pick a language that you know is widely used in the specific realm you are going to be working. Learn all the available programming paradigms (also parallel) in that language. By corollary, moving to another language would be just a matter of learning the &quot;new words&quot; &lt;/p&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="14659" LastEditDate="2015-03-09T14:30:19.390" LastActivityDate="2015-03-09T14:30:19.390" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19115" PostTypeId="2" ParentId="18720" CreationDate="2015-03-09T13:50:18.990" Score="2" Body="&lt;p&gt;If the problems $P_k$ are not too difficult each, I would suggest to solve the problem for the lowest value of $k$ first and use the optimal solution as initial value for the second lowest value of $k$ and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often, a reasonable starting solution speeds up the solution process of a Branch-And-X a lot. Setting initial values is easy for solvers like CPLEX or Gurobi.&lt;/p&gt;&#xA;" OwnerUserId="11687" LastActivityDate="2015-03-09T13:50:18.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19116" PostTypeId="2" ParentId="19105" CreationDate="2015-03-09T15:30:49.113" Score="0" Body="&lt;p&gt;There are lots of things including clarity of syntax, expressibility (e.g., array slicing, support for whole array operations etc.), availability of of APIs/libs etc. E.g., if you want to use OpenMP then your choice of language is rather limited. Same argument holds for things like MPI, numerical libs etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an ideal world you should be able to use any language that would eventually generate the same machine code. It might be true a few years/decades down the road especially with things like LLVM.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2015-03-09T15:30:49.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19118" PostTypeId="1" AcceptedAnswerId="19122" CreationDate="2015-03-09T16:14:31.837" Score="3" ViewCount="576" Body="&lt;p&gt;I’m trying to evaluate an improper integral of a 0th order Bessel function of the first kind  using Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v = integral(@(x)besselj(0, x), 0, Inf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which returns &lt;code&gt;v = 3.7573e+09&lt;/code&gt;. However, this should be &lt;code&gt;1&lt;/code&gt;, in theory. What should I do to get &lt;code&gt;1&lt;/code&gt; in the Matlab?&lt;/p&gt;&#xA;" OwnerUserId="14764" LastEditorUserId="4474" LastEditDate="2015-03-09T20:55:25.680" LastActivityDate="2015-03-09T20:55:25.680" Title="Integration including bessel function" Tags="|matlab|integral-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19119" PostTypeId="1" CreationDate="2015-03-09T16:38:26.107" Score="2" ViewCount="973" Body="&lt;p&gt;I am trying to figure out why my sparse (CRS) Gauss Seidel solver is so slow. I tried to find an implementation of the Gauss Seidel method in sparse format online but could only find implementations using dense matrices. Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void gs(int r[], int c[], double v[], double x[], int n, double tol)&#xA;{&#xA;  //x is initially b in Ax=b&#xA;  double *b = new double[n];&#xA;  for(int i=0;i&amp;lt;n;i++){b[i] = x[i];}&#xA;&#xA;  int ii = 0, jj = 0;&#xA;  double err = 1.0;&#xA;  while(err&amp;gt;tol &amp;amp;&amp;amp; ii&amp;lt;MAX_ITER){&#xA;    //Gauss-Seidel iteration&#xA;    double sigma;&#xA;    double ajj;&#xA;    for(int j=0;j&amp;lt;n;j++){&#xA;      sigma = 0.0;&#xA;      ajj = 0.0;   //diagonal entry a_jj&#xA;      for(int k=r[j];k&amp;lt;r[j+1];k++){&#xA;        if(c[k]!=j){&#xA;          sigma = sigma + v[k]*x[c[k]];&#xA;        }&#xA;        else{&#xA;          ajj = v[k];&#xA;        }&#xA;      }&#xA;      x[j] = (b[j] - sigma)/ajj;&#xA;    }&#xA;&#xA;    if(jj==4){&#xA;      //err = error(ar,ac,av,x,b,n);&#xA;      err = fast_error(r,c,v,x,b,n,tol);&#xA;      jj = 0;&#xA;      #if(DEBUG)&#xA;        std::cout&amp;lt;&amp;lt;&quot;error: &quot;&amp;lt;&amp;lt;err&amp;lt;&amp;lt;std::endl;&#xA;      #endif&#xA;    }&#xA;    ii++;&#xA;    jj++;&#xA;  }&#xA;&#xA;  delete[] b;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: I use pre-processor flags to choose how to calculate the error, but I have profiled and this is not the issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above code snippet the arrays &lt;code&gt;r&lt;/code&gt;, &lt;code&gt;c&lt;/code&gt;, and &lt;code&gt;v&lt;/code&gt; represent the sparse matrix &lt;code&gt;A&lt;/code&gt; in compressed row storage, b is the right-hand side of &lt;code&gt;Ax=b&lt;/code&gt;, n is the dimension of the matrix &lt;code&gt;A&lt;/code&gt;, and &lt;code&gt;tol&lt;/code&gt; is the error tolerance we want to solve to. My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this how you would implement a sparse CRS Gauss Seidel solver?&lt;/li&gt;&#xA;&lt;li&gt;Is there anything obvious that could be changed to speed up this code?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Why do I think my implementation is slow? I am using this solver as a smoother for my AMG implementation and the Gauss Seidel smoothing at the highest level takes the majority of the time (significantly more than computing the Galerkin triple matrix product) even though I may only use 72 Gauss Seidel iterations at the highest level. Any help is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14765" LastEditorUserId="9667" LastEditDate="2015-03-10T00:58:05.937" LastActivityDate="2015-03-10T00:58:05.937" Title="Efficient compressed row storage Gauss Seidel C/C++" Tags="|linear-solver|c|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19120" PostTypeId="2" ParentId="19119" CreationDate="2015-03-09T17:25:12.797" Score="3" Body="&lt;p&gt;How many multigrid levels are you using? As you go to coarser matrices, they grow progressively more dense, so the cost of doing an individual matrix/vector multiply or Gauss-Seidel smoothing will grow accordingly. Try checking the ratio of the number of non-zero entries to the matrix size at each AMG level. Eventually, you reach a point where any additional gain in the convergence speed of the algorithm is offset by the higher cost of executing the smoother; even though you need fewer outer-level iterations, each one is more expensive. To that end, you can try using fewer multigrid levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also want to consider using the &lt;a href=&quot;http://www.cs.colostate.edu/~mroberts/toolbox/c++/sparseMatrix/sparse_matrix_compression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;modified sparse row format&lt;/a&gt;. It's a slight variation on CRS which stores all the diagonal entries of the matrix at the beginning of your array &lt;code&gt;v&lt;/code&gt;. The advantage of doing so is that you can get rid of the conditional expression&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if(c[k]!=j){&#xA;    sigma += v[k] * x[c[k]];&#xA;} else {&#xA;    ajj = v[k];&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;inside your loop over &lt;code&gt;k&lt;/code&gt; in favor of something more like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sigma = 0.0;&#xA;for (int k = r[j]; k &amp;lt; r[j + 1]; k++) {&#xA;    sigma += v[k] * x[c[k]];&#xA;}&#xA;x[j] = (b[j] - sigma)/v[j];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Getting rid of conditionals in tight loops can speed things up dramatically because &lt;a href=&quot;http://en.wikipedia.org/wiki/Branch_predictor&quot; rel=&quot;nofollow noreferrer&quot;&gt;branch prediction&lt;/a&gt; is expensive. You can read more about that &lt;a href=&quot;https://stackoverflow.com/questions/11227809/why-is-processing-a-sorted-array-faster-than-an-unsorted-array&quot;&gt;here&lt;/a&gt;. Naturally, you should do some experiments to be sure that it's really helping; these things are famously difficult to predict.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-03-09T17:25:12.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19121" PostTypeId="2" ParentId="19105" CreationDate="2015-03-09T19:36:19.273" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Suppose a program was written in two distinct languages, let them be language X and language Y, if their compilers generate the same byte code, why I should use language X instead of the language Y?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Programmer productivity is important. If it's easier to get the job done with language X -- which could be for various reasons, like syntax, libraries, tools -- than language Y, that's a net win. If their compilers generate the same byte code, as far as I know, they'll execute in roughly the same amount of time under comparable conditions (same hardware, same load, etc.).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What would make a language better than the other to resolve a problem?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Performance is usually a limiting consideration; if performance matters, you're almost certainly going to look at a compiled language and exclude interpreted languages. After that, it's usually some combination of personal preference and programmer productivity to maximize useful work done before deadlines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example of a productivity versus performance argument, interpreted languages like Python, MATLAB, and Julia typically lead to shorter programs accomplishing a given task; here's &lt;a href=&quot;http://redmonk.com/dberkholz/2013/03/25/programming-languages-ranked-by-expressiveness/&quot; rel=&quot;nofollow&quot;&gt;one example of an empirical ranking&lt;/a&gt;, but there are others, such as in &lt;em&gt;Code Complete&lt;/em&gt;. This phenomenon typically has a number of consequences, like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it's usually faster to write such a program&lt;/li&gt;&#xA;&lt;li&gt;it's usually faster to debug such a program&lt;/li&gt;&#xA;&lt;li&gt;it usually means there are large standard libraries available to accomplish common tasks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, the cost of this expressiveness is usually performance, which you can see in benchmarks, so the typical pure Python program is usually 2-3 orders of magnitude slower than a typical C program; for Julia, it's supposed to be more like a factor of 2ish, which is why it's popular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you have to know -- and be good at -- these languages to extract performance from them, and for MATLAB and Python, if a pure interpreted language implementation isn't fast enough, you have to start replacing parts with compiled language implementations anyway. Once you start delving into multi-language programs, productivity goes down, and it's harder to debug. Similarly, if you have to learn one of these languages, that takes time, and when you're not a language expert, you program more slowly than if you are a language expert. So there are a number of tradeoffs.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-03-09T19:36:19.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19122" PostTypeId="2" ParentId="19118" CreationDate="2015-03-09T20:21:42.273" Score="3" Body="&lt;p&gt;The problem is that you are integrating an oscillatory function over an infinite interval. The MATLAB website doesn't give specifics on the algorithm behind their &lt;code&gt;integral&lt;/code&gt; function (it just says 'globally adaptive'). The older &lt;code&gt;quad&lt;/code&gt; function used to be adaptive Simpson, so I can assume that &lt;code&gt;integral&lt;/code&gt; is the same. What probably is happening is that the code transforms the interval $[0,\infty]$ to $[0,1]$ or $[-1,1]$ or so and hence it is squeezing all the oscillations in this interval. As a result it is adding a lot of positive contributions and subtracting a lot of negative contributions. Hence a totally wrong answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For oscillatory integrals like these a couple of tricks exist. First of all, if the function is rapidly decaying, you could truncate the interval (and as such the classical routines will not transform the interval). If you have ideas on the zero crossings, you could loop over all these and integrate between the zeros, taking care to sum positive parts and negative parts first and then subtract in the end. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second approach might be the double exponential quadrature formula adapted for oscillatory integrals. Please refer to Ooura and Mori, &quot;The double exponential formula for oscillatory functions over the half infinite interval&quot;, J. Comp. App. Math. , 38, p353-360, 1991. Ooura has FORTRAN and C implementations on his &lt;a href=&quot;http://www.kurims.kyoto-u.ac.jp/~ooura/intde.html&quot; rel=&quot;nofollow&quot;&gt;website&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, since you already know the correct answer to your integral, why do it numerically? Just to understand why &lt;code&gt;integral&lt;/code&gt; fails or is there a bigger picture?&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2015-03-09T20:21:42.273" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19123" PostTypeId="2" ParentId="2255" CreationDate="2015-03-10T04:16:05.720" Score="1" Body="&lt;p&gt;See the thesis of Jerome Bonelle which is available on &lt;a href=&quot;http://cermics.enpc.fr/~bonellej/home.html&quot; rel=&quot;nofollow&quot;&gt;his website&lt;/a&gt; or directly &lt;a href=&quot;http://chercheurs.edf.com/fichiers/fckeditor/Commun/Innovation/theses/TheseBonelle.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. I found his chapters 2 - 4 to be quite easy to read and give a nice introduction. He also talks about two examples, one elliptic PDE and the Stokes equations.&lt;/p&gt;&#xA;" OwnerUserId="1176" LastActivityDate="2015-03-10T04:16:05.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19124" PostTypeId="1" CreationDate="2015-03-10T04:50:47.213" Score="1" ViewCount="42" Body="&lt;p&gt;I have a bunch of comments from my website that I want to use to build a profile of someone's personality based on the topics they chose to write in and the their specific comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that this entails natural language processing but I have no idea where to start. I am  a python enthusiast so prefer the library being python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm open to commercial libraries as well but prefer to learn how to do this successfully.&lt;/p&gt;&#xA;" OwnerUserId="14770" LastActivityDate="2015-03-10T04:50:47.213" Title="What library can you use to detect personalities from comments?" Tags="|machine-learning|" AnswerCount="0" CommentCount="2" ClosedDate="2018-03-12T14:57:54.567" ContentLicense="CC BY-SA 3.0" />
  <row Id="19125" PostTypeId="1" CreationDate="2015-03-10T08:48:31.460" Score="4" ViewCount="794" Body="&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: I asked this on &lt;a href=&quot;https://math.stackexchange.com/questions/1181970/usability-of-upwind-finite-difference-schemes&quot;&gt;Mathematics Stack Exchange&lt;/a&gt; and there were no answers. So, I thought I might try here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upwind schemes like the classic &quot;upstream&quot; scheme, can be used to solve, for example, the advection equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\dfrac{\partial \psi}{\partial t} + \dfrac{\partial }{\partial x}(u\psi) = 0 \qquad (1)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this makes sense, because the scalar $\psi$ is being transported by the wind speed $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what is the extent to which these schemes could be used? Could they be used to solve all partial differential equations? Does it make sense? For example you can find in the literature attempts to solve the diffusion equation with an upwind method.&#xA;And if we were to use them to solve an equation, for example, of the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\dfrac{\partial \psi}{\partial t} + a(x,t)\dfrac{\partial \psi}{\partial x} = 0\qquad (2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;should we use a virtual wind speed, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u=\dfrac{1}{\psi} \int a \dfrac{\partial \psi}{\partial x}dx&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to determine the stencils?&lt;/p&gt;&#xA;" OwnerUserId="14772" LastEditorUserId="14772" LastEditDate="2015-03-11T11:09:17.067" LastActivityDate="2015-03-12T21:24:35.007" Title="Usability of upwind finite difference schemes" Tags="|pde|finite-difference|advection|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19126" PostTypeId="2" ParentId="19125" CreationDate="2015-03-10T13:35:32.050" Score="2" Body="&lt;p&gt;Let me assume that &lt;code&gt;upwind&lt;/code&gt; refers to the upwind discretization of a advection term in &lt;em&gt;Finite Volumes&lt;/em&gt; or &lt;em&gt;Finite Differences&lt;/em&gt; Schemes, see e.g. the book &lt;em&gt;Numerical Treatment of Partial Differential Equations&lt;/em&gt; by Grossmann, Roos, and Stynes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can upwind be used to solve all equations?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Certainly not. If in $(1)$ the wind $a$ is not integrable, then there won't be a classical solution that can be approximated by a numerical scheme. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So lets assume that $a$ is continuous, that the problem is posed on a regular domain, with reasonable data and the discretization scheme is reasonable. Then, upwind applied to $(1)$ will give useful results because it is stable and consistent.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Should we use a virtual windspeed?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any reformulation of your problem that results in a form like $(1)$ with a smooth $a$ can thus be well tackled by an upwind discretization. You can go more general and also have a source term and a linear coercive operator like diffusion.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can we solve equations of type $(2)$?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Probably yes. Doing the reformulation $a\partial_x\psi = \partial_x (a\psi) - \partial_x(a) \psi $ gives you the form $(1)$ plus an additional operator that hopefully does not destroy the stability of the scheme. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways, the source of an equation like $(2)$ is most probably a differential form of a conservation law (like the momentum equation in the Navier-Stokes equation) that takes the form $(2)$ because $a$ is constant in space (in 1D) or divergence free (2D or 3D). In this case, in the proposed reformulation, the additional term is zero and you get Equation $(1)$.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2015-03-10T13:41:29.967" LastActivityDate="2015-03-10T13:41:29.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19127" PostTypeId="1" CreationDate="2015-03-10T14:02:57.603" Score="0" ViewCount="44" Body="&lt;p&gt;I would like to solve 3 differential equations for 3 unknowns. So I wrote a MATLAB code, which solves (using the '\' operator) these equation using a linear system of equations (in which the 3 equations are coupled). So the linear system  has a size of (ny*nx*3,ny*nx*3) [nx/ny are the # of nodes in x/y - direction]. I used a finite-difference method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solved for $F$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla u - \nabla a \nabla F =  \nabla a B_1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is the unknown of a different equation, and $a$, $\alpha$, $t$ and $B$ are parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameter $a$ produces very different results of $F$, as shown in the figure below [dotted-black line-$a$ parameter, which is here normalized to the range of $F$; solid red line - solution of $F$]. These results seem to be correct, since they reflect the solution of a benchmark.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I would like to isolate these equations and pass the solution of one equation to the the other one (as a parameter on the right-hand-side of the equation). So I store the solution of $u$ from the previous ('coupled') approach and pass it as a parameter to the equation, which is written above. Again I use MATLAB, finite differences and the '\' operator. But this time the linear system of equation only has a size of (ny * nx,ny * nx), since I solve only for one unknown ($F$). I expect the solution of this isolated system to be very similar to the one of the coupled system, since I solve the same equation and have the same values for $u$, $a$, $B$, spatial step size ... !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the results are very different from the previous ('coupled') approach, if $a$ is very low (see the solid blue line on the lower figure). Especially where $\nabla a$ becomes very low, there is a great discrepancy.&#xA;Why is such an approach numerical instable? Is it because of floating point inaccuracy? How could I make the problem work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/iiCFn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="14732" LastEditorUserId="14732" LastEditDate="2015-03-10T15:41:51.827" LastActivityDate="2015-03-10T15:41:51.827" Title="Isolated solving of coupled system of PDE" Tags="|pde|coupling|numerical-modelling|" AnswerCount="0" CommentCount="3" ClosedDate="2015-03-10T20:25:01.617" ContentLicense="CC BY-SA 3.0" />
  <row Id="19129" PostTypeId="1" AcceptedAnswerId="19131" CreationDate="2015-03-10T17:44:30.633" Score="5" ViewCount="412" Body="&lt;p&gt;Consider a $2\times 2$ block matrix and a linear system of equations associated to it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; \begin{pmatrix} - A &amp;amp; B \\ B^t &amp;amp; C \end{pmatrix}&#xA; \begin{pmatrix} x \\ y \end{pmatrix}&#xA; =&#xA; \begin{pmatrix} \phi \\ \psi \end{pmatrix}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that $A$ and $C$ are symmetric, and symmetric positive semi-definite, and that $A$ is even invertible. One can construct the Schur complement system&#xA;$(C + B^t A^{-1} B ) y = \psi - B^t A^{-1} \phi$&#xA;where $S = C + B^t A^{-1} B$ is symmetric positive-definite, by assumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you precondition such a system in general? I have noted that preconditioners for $S$ are derived from block matrix preconditioners for the original block matrix. Is there a general consensus how such a block matrix preconditioner looks like?&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2015-03-10T22:46:25.847" Title="Preconditioning symmetric Schur complement" Tags="|preconditioning|block-decomposition|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19131" PostTypeId="2" ParentId="19129" CreationDate="2015-03-10T20:09:29.370" Score="5" Body="&lt;p&gt;You won't find any black-box scalable solutions because $S$ is typically dense and thus cannot be formed. If your problem comes from a mature research area, there might be experience in the literature demonstrating how to approximate Schur complements. One common technique is to use approximate commutator arguments. There are a quite a few papers on this topic by Elman and others for incompressible flow.  You can also see &lt;a href=&quot;http://dx.doi.org/10.1017/S0962492904000212&quot; rel=&quot;noreferrer&quot;&gt;Benzi, Golub, and Liesen, &lt;em&gt;Numerical Solution of Saddle Point Problems&lt;/em&gt; (2005)&lt;/a&gt; for a (slightly dated) broader review.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2015-03-10T20:09:29.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19132" PostTypeId="1" AcceptedAnswerId="19138" CreationDate="2015-03-10T20:26:48.260" Score="0" ViewCount="365" Body="&lt;p&gt;I start a study the best way to mount a HPC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On my datacenter i have to initial tests 3 servers whit this settings.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;DELL PowerEdge R720&lt;/li&gt;&#xA;&lt;li&gt;2 x Intel Xeon E5-2680v2 2.8GHz&lt;/li&gt;&#xA;&lt;li&gt;10 x 16GB RDIMM&lt;/li&gt;&#xA;&lt;li&gt;XenServer 6.5&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have more at ease with CentOS. So the first question is what is the most stable CentOS for HPC?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plane for this HPC using this tools:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Openstack&lt;/li&gt;&#xA;&lt;li&gt;Torque (Resource Manager)&lt;/li&gt;&#xA;&lt;li&gt;Maui (Cluster Scheduler)&lt;/li&gt;&#xA;&lt;li&gt;Gold (Allocation Manager)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://modules.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt; Environment Modules&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So if you have interest in this project please help to find de way&lt;/p&gt;&#xA;" OwnerUserId="14776" LastActivityDate="2015-03-11T10:03:21.380" Title="Planning to build a HPC" Tags="|hpc|" AnswerCount="2" CommentCount="3" ClosedDate="2015-03-13T23:37:05.553" ContentLicense="CC BY-SA 3.0" />
  <row Id="19133" PostTypeId="2" ParentId="19132" CreationDate="2015-03-10T22:09:06.620" Score="0" Body="&lt;p&gt;Stable? Either the latest CentOS 6 or 7 should be fine from a stability perspective. I wouldn't count OpenStack as an HPC, tool, but the cloud stacks are coming along performance-wise from where they started. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-03-10T22:09:06.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19135" PostTypeId="2" ParentId="19129" CreationDate="2015-03-10T22:46:25.847" Score="3" Body="&lt;p&gt;It is most often problem dependent. For example, the matrix $S$ might actually correspond to an operator that could be individually discretized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see an example of what one can do, you could take a look at step-20 of the deal.II tutorials (&lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/step_20.html&quot; rel=&quot;nofollow&quot;&gt;https://www.dealii.org/developer/doxygen/deal.II/step_20.html&lt;/a&gt;). It's not an example intended to show how to build the &lt;em&gt;best performing&lt;/em&gt; preconditioner, but how one can think about preconditioners and stack and nest them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I wrote step-20. I also confess that it's not my best tutorial program.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-10T22:46:25.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19136" PostTypeId="1" CreationDate="2015-03-11T04:40:01.060" Score="1" ViewCount="329" Body="&lt;p&gt;I'm using the Metropolis algorithm for 2D up to 5D for the Ising Model and I want to compute the spin-spin correlation function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$c(r)=&amp;lt;s_is_r&amp;gt;−〈s_i〉〈s_r 〉$$ but I'm not sure how to estimate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&amp;lt;s_is_r&amp;gt; = \frac{1}{N}\sum s_is_r$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I do for $s_is_r$ numerically?&lt;/p&gt;&#xA;" OwnerUserId="8012" LastActivityDate="2015-03-11T04:40:01.060" Title="Spin-spin correlation functions in the Ising Model Monte Carlo" Tags="|computational-physics|monte-carlo|" AnswerCount="0" CommentCount="1" ClosedDate="2019-06-09T18:07:23.893" ContentLicense="CC BY-SA 3.0" />
  <row Id="19137" PostTypeId="1" CreationDate="2015-03-11T05:07:20.043" Score="27" ViewCount="31171" Body="&lt;p&gt;What is the preferred and efficient approach for interpolating multidimensional data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I'm worried about:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;performance and memory for construction, single/batch evaluation&lt;/li&gt;&#xA;&lt;li&gt;handling dimensions from 1 to 6&lt;/li&gt;&#xA;&lt;li&gt;linear or higher-order&lt;/li&gt;&#xA;&lt;li&gt;ability to obtain gradients (if not linear)&lt;/li&gt;&#xA;&lt;li&gt;regular vs scattered grid&lt;/li&gt;&#xA;&lt;li&gt;using as Interpolating Function, e.g. to find roots or to minimize&lt;/li&gt;&#xA;&lt;li&gt;extrapolation capabilities&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there efficient open-source implementation of this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had partial luck with scipy.interpolate and kriging from scikit-learn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did not try splines, Chebyshev polynomials, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I found so far on this topic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/14119892&quot;&gt;Python 4D linear interpolation on a rectangular grid&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/16217995&quot;&gt;Fast interpolation of regularly sampled 3D data with different intervals in x,y, and z&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/16983843&quot;&gt;Fast interpolation of regular grid data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/592026&quot;&gt;What method of multivariate scattered interpolation is the best for practical use?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5401" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-06-23T02:42:14.117" Title="What is the preferred and efficient approach for interpolating multidimensional data?" Tags="|python|performance|interpolation|delaunay-triangulation|extrapolation|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19138" PostTypeId="2" ParentId="19132" CreationDate="2015-03-11T10:03:21.380" Score="1" Body="&lt;p&gt;Maui and Gold are pretty much dead (check the last release date) - I'd be looking at SLURM &lt;a href=&quot;http://slurm.schedmd.com/&quot; rel=&quot;nofollow&quot;&gt;http://slurm.schedmd.com/&lt;/a&gt; if you need something free with advanced scheduling and allocation management (though they all have their pain points). If you only have three nodes though, do you need these at all? Then Torque would do just as well for simple scheduling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are building a prototype for a larger system, then go SLURM and also focus on the node configuration tools, so that you know how to expand and keep things under control (some keywords to look for are Ansible/Puppet/Chef/xCat - OpenStack not yet but it depends on your workload type and if you want to expand to the cloud later)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Centos 6 is pretty stable (wait a bit for Centos 7.1 and plan an upgrade path).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Environment Modules rocks.&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2015-03-11T10:03:21.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19139" PostTypeId="2" ParentId="19125" CreationDate="2015-03-11T10:57:12.617" Score="3" Body="&lt;p&gt;There are multiple questions here, but let's start with the basics.  You have written two hyperbolic PDEs; (1) is the &lt;strong&gt;continuity equation&lt;/strong&gt;, which is conservative and (2) is the &lt;strong&gt;color equation&lt;/strong&gt;, which is not conservative.  &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What are the characteristic speeds for these equations?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For (1), you have stated correctly that the characteristic speed is $u(x,t)$.  For (2), you have made incorrect suggestions in both your answer and your comment.  The characteristic speed for (2) is $a(x,t)$.  Therefore, the &quot;virtual wind speed&quot; you introduce is not useful.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Can an upwind scheme be used to solve the color equation (2)?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Yes.  Furthermore, and contrary to what @Jan has suggested, (2) is well-posed even for typical discontinuous functions $a$.  An upwind discretization for the problem with discontinuous $a$ will be convergent, under the usual restriction on the CFL number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not directly related to your question, but since @Jan brought it up I will add that integrability of $a$ is not the relevant condition.  The simplest situation in which a classical solution fails to exist is when there is a point where $a$ changes sign from positive to negative.  A delta function must form at that point.  Exactly the same thing happens for (1) in the case that $u$ changes sign, so this is nothing special about the color equation.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Could upwind methods be used to solve all PDEs?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It depends on what exactly you mean by &quot;upwind methods&quot;.  For the diffusion equation, there is no directional bias in the transmission of information.  An explicit, purely one-sided stencil cannot be convergent for diffusion (see the CFL paper).  You could use a stencil that is biased toward one direction (but includes some points in both directions) for any PDE.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;To learn more:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For a lengthy and excellent discussion of differences between (1) and (2) and their numerical discretization by upwind methods, see Chapter 9 of &lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot; rel=&quot;nofollow&quot;&gt;LeVeque's book on finite volume methods&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-03-12T21:24:35.007" LastActivityDate="2015-03-12T21:24:35.007" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19141" PostTypeId="1" AcceptedAnswerId="19144" CreationDate="2015-03-11T21:19:58.207" Score="3" ViewCount="228" Body="&lt;p&gt;I have matrices that are extremely easy to compute pointwise, but are too large to store. (they are not sparse)&#xA;On the MATLAB site I was told MATLAB doesnt support computations with non-stored matrices.&#xA;Is their any other software package out there that does?&lt;/p&gt;&#xA;" OwnerUserId="7789" LastActivityDate="2015-03-12T00:18:45.963" Title="Which software packages can solve linear systems that are not stored" Tags="|linear-algebra|linear-solver|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19142" PostTypeId="2" ParentId="19141" CreationDate="2015-03-11T21:57:58.113" Score="3" Body="&lt;p&gt;If the matrices are too large to fit into RAM but reasonably could fit on your hard disk, then you should look at what are called &quot;Out of Core&quot; solvers.  These solvers partition the large matrix into blocks that are small enough to be handled one at a time in memory.  There are many available codes for both dense and sparse problems, but getting good performance out of an out of core solver often requires tuning the code to the particular I/O performance of the computer that you're using.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-03-11T21:57:58.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19144" PostTypeId="2" ParentId="19141" CreationDate="2015-03-12T00:18:45.963" Score="6" Body="&lt;p&gt;If the individual entries of the matrices involved can be calculated on the fly relatively cheaply, as you say, then you could turn to matrix-free methods (also known as black box solvers).  Krylov-subspace methods such as GMRES belong to that class of solvers and would be useful for you because they only require that the user calculates matrix-vector products.  You can do these matrix-vector products by computing the relevant entries of your matrix on the fly, without storing the whole matrix in memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reverse communication routines in the Fortran version of the code available in &lt;a href=&quot;http://www.netlib.org/templates/&quot;&gt;Templates for the Solution of Linear Systems&lt;/a&gt; would match your requirements.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-03-12T00:18:45.963" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19145" PostTypeId="1" CreationDate="2015-03-12T03:03:06.967" Score="3" ViewCount="974" Body="&lt;p&gt;I am new to OpenFoam. And I am trying to add a set (user defined) of differential equations to OpenFoam. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve this user defined set of equations at each time point in addition to standard fluid dynamic equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SET-1 is ODE/PDE's associated with fluid dynamics (equations already there in openfoam)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SET-2 is ODE/PDE's that are defined by user (how to add this set to openfoam is my question)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could this be done in OpenFoam? does anyone have a sample example?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much for your time and attention.&lt;/p&gt;&#xA;" OwnerUserId="14786" LastEditorUserId="14659" LastEditDate="2015-03-12T15:46:37.817" LastActivityDate="2015-03-12T15:46:37.817" Title="Add User-defined/custom differential equations in OpenFoam (CFD)" Tags="|fluid-dynamics|finite-volume|openfoam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19146" PostTypeId="1" AcceptedAnswerId="19148" CreationDate="2015-03-12T05:56:24.920" Score="4" ViewCount="1539" Body="&lt;p&gt;I want to solve the Laplace Equation with pure Neumann B.C. using Finite Element Method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$- \Delta u = f  \ $ in $ \ \Omega $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$- \partial u/\partial n = g  \ $ on $ \ \Gamma = \partial \Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With weak formulation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{\Omega} \nabla v \cdot \nabla u = \int_{\Omega} v \ f + \int_{\partial \Omega} v \ g  $. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain a unique solution, I was following the Lagrange Multiplier method,  enforcing a constraint of the type&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{\Omega} v \ dx= 0  $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following linear system would then be obtained&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{pmatrix}A &amp;amp; B^T \\ B &amp;amp; 0\end{pmatrix} \begin{pmatrix}U \\ \lambda \end{pmatrix} = \begin{pmatrix}F \\ 0\end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $A$ corresponds to the LHS (stiffness matrix), $F$ to the RHS (load vector), $U$ to the solution vector and $\lambda$ to the Lagrange Multiplier, which can be discarded from the resulting solution vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Larson, Bengzon in the book &quot;The Finite Element Method&quot; (p. &#xA;95),&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Lagrangian multiplier $\lambda$ may be thought of as a force acting to&#xA;  enforce  ̋the constraints. Because the zero mean value on $u_h$ is a&#xA;  constraint, which do not alter the solution to the underlying Neumann&#xA;  problem, the force should vanish or, at least, be very small.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would like to know how small this value should be, and how to analyze the simulation results based on this (are the results valid if the multiplier is too large? Should I compare it based on the minimum value obtained from my solution? etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14783" LastActivityDate="2015-03-12T15:02:45.073" Title="Solving pure Neumann problem enforcing B.C. with Lagrange Multiplier" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19147" PostTypeId="1" CreationDate="2015-03-12T08:02:08.727" Score="0" ViewCount="426" Body="&lt;p&gt;What does &quot;&lt;code&gt;Desired error not necessarily achieved due to precision loss&lt;/code&gt;&quot; mean in the context of the &lt;code&gt;scipy_fmin&lt;/code&gt; methods? I can't seem to find an explanation anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;import numpy&#xA;import random&#xA;import scipy.optimize as opt&#xA;import matplotlib.pyplot as plt&#xA;from numpy import array&#xA;from numpy import dot&#xA;from random import randint&#xA;from numpy import matrix&#xA;import sys&#xA;&#xA;ns = []&#xA;st = []&#xA;lam_funtrix = []&#xA;&#xA;time_steps = 1000&#xA;delta_t = 0.1&#xA;&#xA;mu = -0.7&#xA;&#xA;def gen_st():&#xA;    global st&#xA;    st = []&#xA;    for i in range(0, time_steps):&#xA;        st.append(random.normalvariate(0,1) * math.sqrt(delta_t))&#xA;&#xA;def f(val):&#xA;    return math.exp(val)&#xA;&#xA;def get_lam(t):&#xA;    rate = mu&#xA;    return pow(delta_t, -1) * f(rate)&#xA;&#xA;def white_noise():&#xA;    global ns&#xA;    for i in range(0, time_steps):&#xA;        lam = get_lam(i) * delta_t&#xA;        spike_at_bin = numpy.random.poisson(lam)&#xA;        ns.append(spike_at_bin)&#xA;&#xA;def gen_lam_log(i, mu):&#xA;    rate = mu&#xA;    return pow(delta_t, -1) * f(rate)&#xA;&#xA;def gen_lam_fun(mu):&#xA;    global lam_funtrix&#xA;    lam_funtrix = []&#xA;    for i in range(0, time_steps):&#xA;        lam_funtrix.append(gen_lam_log(i, mu))&#xA;&#xA;def log_like(t):&#xA;    mu = t&#xA;    gen_lam_fun(mu)&#xA;    sum = 0&#xA;    for i in range(0,time_steps):&#xA;        val = lam_funtrix[i]&#xA;        sum = sum - ((ns[i] * math.log(val*delta_t)) - (val*delta_t))&#xA;    return sum&#xA;&#xA;def der_mu():&#xA;    sum = 0.0&#xA;    for i in range(0, time_steps):&#xA;        sum -= (ns[i] - lam_funtrix[i] * delta_t)&#xA;    return sum&#xA;&#xA;def first_der(t):&#xA;    mu = t&#xA;    gen_lam_fun(mu)&#xA;    dm = der_mu()&#xA;    return dm&#xA;&#xA;gen_st()&#xA;white_noise()&#xA;init_guess = array([0])&#xA;vals = opt.fmin_cg(log_like, init_guess, fprime=first_der)&#xA;print vals&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code is a  crumby since I pared it down a bit for the question.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Warning: Desired error not necessarily achieved due to precision loss.&#xA;         Current function value: 822.835581&#xA;         Iterations: 1&#xA;         Function evaluations: 18&#xA;         Gradient evaluations: 6&#xA;[-0.7943019]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14788" LastActivityDate="2015-04-15T18:01:11.373" Title="Scipy min warning meanings?" Tags="|python|numpy|scipy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19148" PostTypeId="2" ParentId="19146" CreationDate="2015-03-12T15:02:45.073" Score="2" Body="&lt;p&gt;What is small is relative. I gave a few examples &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19015/doubt-regarding-stopping-criterion-for-newton-method/19017#19017&quot;&gt;here&lt;/a&gt; to explain that &quot;small&quot; must be seen in the context of the other terms in your equation. For example, the total force exerted on the membrane that your equation describes is $\int_\Omega |f(x)| \; dx$, and the total boundary forces are $\int_{\partial\Omega} |g(x)| \; dx$. The Lagrange multiplier $\lambda$ should be small compared to these forces.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-03-12T15:02:45.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19149" PostTypeId="2" ParentId="19145" CreationDate="2015-03-12T15:22:53.043" Score="4" Body="&lt;p&gt;The whole point of the OpenFOAM (OF hereinafter) libraries is exactly what you are inquiring for: PDE mimicking programming for CFD using the FV framework. In order to do what you are asking, you should have a decent foundation in C++ (not necessarily an expert...), of course in FVM and CFD in general. The first step in order to program using the OF framework is to get comfortable using the doxygen browser to dive into the code. Don't get your hopes high if you expect to find a thorough tutorial on OF programming, basically there isn't. You'll have to resort to the OF community or some examples/homeworks/theses that are posted on Chalmers University and foamcfd.com. A good review on the OF framework can be found in the following &lt;a href=&quot;http://www.ams.org/notices/201404/rnoti-p354.pdf&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14659" LastActivityDate="2015-03-12T15:22:53.043" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19150" PostTypeId="2" ParentId="16088" CreationDate="2015-03-12T19:41:01.663" Score="1" Body="&lt;p&gt;So it turns out I think I found the closest thing to what I really wanted: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/71TPb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is from &lt;a href=&quot;http://stanford.edu/~mwaskom/software/seaborn/examples/timeseries_bootstrapped.html#timeseries-bootstrapped&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the &lt;code&gt;seaborn&lt;/code&gt; library in python, which seems to act mostly as an extension to matplotlib and pyplot. My OP seems to be a variation on this library plotting function. &lt;/p&gt;&#xA;" OwnerUserId="9564" LastActivityDate="2015-03-12T19:41:01.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19151" PostTypeId="1" CreationDate="2015-03-12T20:23:13.030" Score="1" ViewCount="50" Body="&lt;p&gt;What I specifically mean is, given some functional $F\left[\mathbf{x}\right]$ which is stationary with respect to $\dot{\mathbf{x}}=f(\mathbf{x})$ and some boundary or initial conditions, can one choose:&#xA;$$&#xA;\mathbf{x}(t)=\mathbf{x}_0+\mathbf{x}_1t+\mathbf{x}_2t^2+\dots&#xA;$$&#xA;And substitute this into the functional (or it's first variation) and some how solve for the coefficients?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would the variations be expressed in this case, would they also be:&#xA;$$&#xA;\delta\mathbf{x}(t)=\delta\mathbf{x}_0+\delta\mathbf{x}_1t+\delta\mathbf{x}_2t^2+\dots&#xA;$$&#xA;And then I would collect with respect to them if I was substituting into the first variation?&lt;/p&gt;&#xA;" OwnerUserId="4509" LastActivityDate="2015-03-12T20:23:13.030" Title="Can variational formulations be solved using series solutions?" Tags="|finite-element|ode|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19152" PostTypeId="1" AcceptedAnswerId="19153" CreationDate="2015-03-12T22:29:20.950" Score="6" ViewCount="453" Body="&lt;p&gt;Hessian free iterative optimization techniques like Newton-CG, do not explicitly compute the Hessian but instead approximate the product of the Hessian with a vector through finite difference. The approximation error is $\mathcal{O}(\varepsilon)$, where $\varepsilon$ is the parameter in the finite difference calculation. It will be good to know how this approximation affects the number of CG iterations compared to having the complete Hessian? Are there theoretical/empirical understandings about this? Can the product of a more sparse Hessian with a vector be better approximated with finite difference?&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="276" LastEditDate="2015-03-12T22:36:18.840" LastActivityDate="2015-03-12T22:50:15.660" Title="Caveats of Hessian free method" Tags="|linear-algebra|numerical-analysis|newton-method|conjugate-gradient|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19153" PostTypeId="2" ParentId="19152" CreationDate="2015-03-12T22:50:15.660" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The approximation error is $\mathcal{O}(\varepsilon)$, where $\varepsilon$ is the parameter in the finite difference calculation. It will be good to know how this approximation affects the number of CG iterations compared to having the complete Hessian? Are there theoretical/empirical understandings about this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. In short, there are problems where CG iterations will fail on a finite-difference approximation of the Hessian, but succeed when given an analytical Hessian. These situations tend to occur when the analytical Hessian is ill-conditioned; then the finite-difference Hessian tends to be even more so. The finite-difference Hessian is derived from the Jacobian of the objective function and the Jacobian of the constraints; attempting to calculate these Jacobians via finite-differences is likely to lead to catastrophic loss of precision, since you're apt to lose at least half of your digits of precision every time you take finite differences. There's a detailed discussion with references in &lt;a href=&quot;http://www.cs.odu.edu/~keyes/papers/jfnk.pdf&quot;&gt;Jacobian-free Newton-Krylov methods: a survey of approaches and applications&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can the product of a sparse Hessian with a vector be better approximated with finite difference?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. If your function is analytic, and you're willing to use intrusive approaches that modify your code base, you could use &lt;a href=&quot;http://blogs.mathworks.com/cleve/2013/10/14/complex-step-differentiation/&quot;&gt;complex-step differentiation&lt;/a&gt;. Basically, you have to rewrite your functions to take complex-valued arguments, and then you can use complex step differentiation to calculate derivatives with much less error.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-03-12T22:50:15.660" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19154" PostTypeId="1" CreationDate="2015-03-13T08:50:07.020" Score="0" ViewCount="73" Body="&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Ewald_summation#Particle_mesh_Ewald_.28PME.29_method&quot; rel=&quot;nofollow&quot;&gt;Particle mesh Ewald&lt;/a&gt;  method for acceleration of solving pairwise interaction by long range forces (electrostatic, gravitational ... ) seem to be very general and easy to implement. The basic principle is well described in many sources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is harder to find is how exactly split the potential kernel ( say $V(r)=1/r = V_{SR}(r) + V_{LR}$ ) into short range and long range part. In other words:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How should I choose short range part $V_{SR}(r)$ so that the long&#xA;range part $V_{LR} = V(r) - V_{SR}(r)$ is easy to express in Fourier&#xA;space as some analytic function fast to evaluate numerically.&lt;/li&gt;&#xA;&lt;li&gt;It would be nice to use similar splitting in all dimension ( 1D, 2D, 3D ). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;e.g. somewhere I read that often is used &lt;a href=&quot;http://en.wikipedia.org/wiki/Yukawa_potential&quot; rel=&quot;nofollow&quot;&gt;Yukawa-like potential&lt;/a&gt; &#xA; $V_{SR} = \exp(-\beta r )/r$ or Gaussian like damping $V_{SR} = \exp(-\beta r^2 )/r$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how to evaluate Fourier transform of long range $V_{LR}(r)$ efficintly in such case? Or is there any better choice of $V_{SR}$.&lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="15" LastEditDate="2015-03-13T13:03:59.093" LastActivityDate="2015-03-13T13:03:59.093" Title="Particle mesh Ewald: recommended splitting into short and long range" Tags="|poisson|fourier-transform|partitioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19155" PostTypeId="2" ParentId="19154" CreationDate="2015-03-13T12:58:46.230" Score="1" Body="&lt;p&gt;Check out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nijboer, B. R. A., &amp;amp; De Wette, F. W. (1957). On the calculation of lattice sums. Physica, 23(1-5), 309–321. &lt;a href=&quot;http://dx.doi.org/10.1016/S0031-8914(57)92124-9&quot; rel=&quot;nofollow&quot;&gt;doi:10.1016/S0031-8914(57)92124-9&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There they make the case for using a splitting based on the &lt;em&gt;incomplete Gamma function&lt;/em&gt;.  That splitting generalizes the choice of the &lt;em&gt;error function&lt;/em&gt; to arbitrary dimensions.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-03-13T12:58:46.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19157" PostTypeId="1" CreationDate="2015-03-14T00:12:46.377" Score="2" ViewCount="581" Body="&lt;p&gt;I was working in my project when I was struck by the question of whether it would be necessary, or at least cautious, prevent overflow and underflow in the calculation of these two distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I remembered that there is an implementation of the calculation of the hypotenuse to prevent this. Most languages ​​implementers, and is known for &lt;a href=&quot;http://en.wikipedia.org/wiki/Hypot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hypot&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The calculation of the Euclidean distance remains the same &quot;pattern&quot; and I thought that if &lt;code&gt;Hypot()&lt;/code&gt; controls the overflow and underflow should also beware of the Euclidean distance.  I've disappointed to note that the language we use, and others, do not control the overflow and underflow for calculating distance. Will not worth spend this &quot;additional effort&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a searchs and came to a question in &lt;a href=&quot;https://math.stackexchange.com/questions/102298/precision-and-performance-of-euclidean-distance&quot;&gt;Math.StackExchange&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no definitive answer to this issue and is somewhat old. The first thing I wondered is: Will okay? I think that yes, seeing that is a generalization of the same procedure that performs &lt;code&gt;Hypot()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I decided to extrapolate this concept to the Mahalanobis distance. The original is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D_M(X,Y,L) = \sqrt{\sum_{i=1}^{n}\left(\frac{X_i-Y_i}{L_i}\right)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $L$ is the vector of eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my proposal is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D_M(X,Y,L) = C\sqrt{\sum_{i=1}^{n}\left(\frac{X_i-Y_i}{L_i }\frac{1}{C}\right)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is the same to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D_M(X,Y,L) = C\sqrt{\sum_{i=1}^{n}\left(\frac{X_i-Y_i}{L_i C}\right)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And $C$ is the max value from the $|(X_i-Y_i)/L_i|$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C = \max_{i}\left(\mid\frac{X_i-Y_i}{L_i}\mid\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it okay?&lt;/p&gt;&#xA;" OwnerUserId="14754" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-06-14T20:06:47.000" Title="How to prevent overflow and underflow in the Euclidean distance and Mahalanobis distance" Tags="|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19158" PostTypeId="1" AcceptedAnswerId="19161" CreationDate="2015-03-14T21:50:23.487" Score="3" ViewCount="1964" Body="&lt;p&gt;I am trying to understand 1D $L^2$-projections using quadratic basis functions. Using 3 data points, and the Lagrange polynomial it is easy enough to see how to write out 3 basis functions. With the hat functions from the linear basis, it is easy to see how to expand a function. The hat functions are like delta-functions. With the quadratic basis I am having trouble writing out an explicit expression for the basis function because there are three functions in the same interval. (Fig 8.36 here : &lt;a href=&quot;https://people.fh-landshut.de/~maurer/femeth/node265.html#SHP3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://people.fh-landshut.de/~maurer/femeth/node265.html#SHP3&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step of what I am looking to do is construct the so called &quot;mass matrix&quot;. Is there a way to visualize the quadratic basis like a hat-function? Once the basis functions are known, the mass matrix elements can be computed by looking at the overlap between the basis functions, L2-inner product. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks ahead, any advice or comments appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mass matrix structure for the linear-hat functions (x's denote a non-zero entry):&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/xW4S0.jpg&quot; alt=&quot;linear&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This information is based off of the accepted answer below, which gives good references for figuring this out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Structure of the mass matrix with quadratic basis functions:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/w9qpr.jpg&quot; alt=&quot;quad&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="10225" LastEditorUserId="10225" LastEditDate="2015-03-19T15:20:50.230" LastActivityDate="2015-03-19T15:20:50.230" Title="L2-Projection using quadratic basis functions" Tags="|interpolation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19159" PostTypeId="2" ParentId="11435" CreationDate="2015-03-15T11:08:18.373" Score="2" Body="&lt;p&gt;First, an important distinction must be done. A Tucker decomposition of a third-order tensor $\mathcal{T}$ is &lt;em&gt;any&lt;/em&gt; decomposition of the form&#xA;$$&#xA;\mathcal{T} = \mathcal{G} \times_1 \mathbf{A} \times_2 \mathbf{B} \times_3 \mathbf{C},&#xA;$$&#xA;where $\mathcal{G}$ is also a third-order tensor called &lt;em&gt;core&lt;/em&gt; and $\mathbf{A}$, $\mathbf{B}$ and $\mathbf{C}$ are the &lt;em&gt;matrix factors&lt;/em&gt; of the decomposition. In scalar form,&#xA;$$&#xA; t_{i_1,i_2,i_3} = \sum_{r_1=1}^{I_1} \sum_{r_2=1}^{I_2} \sum_{r_3=1}^{I_3} g_{r_1,r_2,r_3} a_{i_1,r_1} b_{i_2,r_2} c_{i_3,r_3}.&#xA;$$&#xA;where $I_1, I_2, I_3$ are the dimensions of $\mathcal{T}$. The flat (horizontal) matrix unfoldings of $\mathcal{T}$ are then written as&#xA;$$&#xA; \mathbf{T}_1 = \mathbf{A} \mathbf{G}_1 (\mathbf{C} \otimes \mathbf{B})^T \\&#xA; \mathbf{T}_2 = \mathbf{B} \mathbf{G}_2 (\mathbf{A} \otimes \mathbf{C})^T \\&#xA; \mathbf{T}_3 = \mathbf{C} \mathbf{G}_3 (\mathbf{B} \otimes \mathbf{A})^T,&#xA;$$&#xA;where $\otimes$ denotes the Kronecker product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the factor matrices are the matrices of left singular vectors of the singular value decompositions of each matrix unfolding, then we have a particular Tucker decomposition which is usually called the &lt;em&gt;higher-order singular value decomposition&lt;/em&gt; (HOSVD) (see the seminal paper by De Lathauwer &lt;em&gt;et al.&lt;/em&gt;, &lt;a href=&quot;ftp://ftp.esat.kuleuven.be/pub/SISTA/delathauwer/reports/ldl-94-31.pdf&quot; rel=&quot;nofollow&quot;&gt;ftp://ftp.esat.kuleuven.be/pub/SISTA/delathauwer/reports/ldl-94-31.pdf&lt;/a&gt;). Mathematically, expressing the SVDs of $\mathbf{T}_1$, $\mathbf{T}_2$ and $\mathbf{T}_3$ as&#xA;$$&#xA; \mathbf{T}_n = \mathbf{U}^{(n)} \mathbf{\Sigma}^{(n)} {\mathbf{V}^{(n)}}^H, \quad n = 1,2,3,&#xA;$$&#xA;the HOSVD of $\mathcal{T}$ is given by&#xA;$$&#xA;\mathcal{T} = \mathcal{S} \times_1 \mathbf{U}^{(1)} \times_2 \mathbf{U}^{(2)} \times_3 \mathbf{U}^{(3)},&#xA;$$&#xA;where the core satisfies&#xA;$$&#xA;\mathcal{S} = \mathcal{T} \times_1 {\mathbf{U}^{(1)}}^H \times_2 {\mathbf{U}^{(2)} }^H \times_3 {\mathbf{U}^{(3)}}^H.&#xA;$$&#xA;It is easy to verify the above formula for the core, as &#xA;$$&#xA; \mathbf{S}_1 = {\mathbf{U}^{(1)}}^H \mathbf{T}_1 ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T \\&#xA;             = {\mathbf{U}^{(1)}}^H \mathbf{U}^{(1)} \mathbf{S}_1 ({\mathbf{U}^{(3)}} \otimes {\mathbf{U}^{(2)}})^T ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T = \mathbf{S}_1&#xA;$$&#xA;because each $\mathbf{U}^{(n)}$ is unitary by definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the answer to your question can be immediately taken from the above formulas, since substituting the SVD of $\mathbf{T}_1$ in the expression for $\mathbf{S}_1$ we have&#xA;$$&#xA; \mathbf{S}_1 = {\mathbf{U}^{(1)}}^H \mathbf{T}_1 ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T&#xA;             = {\mathbf{U}^{(1)}}^H \mathbf{U}^{(1)} \mathbf{\Sigma}^{(1)} {\mathbf{V}^{(1)}}^H ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T&#xA;             = \mathbf{\Sigma}^{(1)} {\mathbf{W}^{(1)}}^H&#xA;$$&#xA;where we have defined&#xA;$$&#xA;{\mathbf{W}^{(1)}} = ({\mathbf{U}^{(3)}} \otimes {\mathbf{U}^{(2)}})^T {\mathbf{V}^{(1)}}.&#xA;$$&#xA;Note that &#xA;$$&#xA;{\mathbf{W}^{(1)}}^H {\mathbf{W}^{(1)}} = {\mathbf{V}^{(1)}}^H ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T ({\mathbf{U}^{(3)}} \otimes {\mathbf{U}^{(2)}})^T {\mathbf{V}^{(1)}} \\&#xA;   = {\mathbf{V}^{(1)}}^H [({\mathbf{U}^{(3)}} \otimes {\mathbf{U}^{(2)}}) ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)]^T {\mathbf{V}^{(1)}} \\&#xA;   = {\mathbf{V}^{(1)}}^H {\mathbf{V}^{(1)}} = \mathbf{I},&#xA;$$&#xA;and hence $\mathbf{W}^{(1)}$ is unitary. This tells us that the mode-1 unfolding of the core $\mathcal{S}$ is a product of the diagonal matrix of singular values of $\mathbf{T}_1$ by a unitary matrix. Therefore, the norm of the $i$th &lt;em&gt;row&lt;/em&gt; of $\mathbf{S}_1$ is precisely the $i$th singular value $\sigma_i^{(1)}$ of $\mathbf{T}_1$. An analogous reasoning applies of course to the other modes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, observe also that &#xA;$$ \|\mathcal{S}\|_F^2 = \|\mathbf{S}_n\|_F^2 = \text{Trace}(\mathbf{S}_n\mathbf{S}_n^H) = \sum_i (\sigma_i^{(n)})^2 = \|\mathbf{T}_n\|_F^2 = \|\mathcal{T}\|_F^2,$$ &#xA;which comes as no surprise, since $\mathcal{S}$ is given by a unitary multilinear transformation of $\mathcal{T}$.&lt;/p&gt;&#xA;" OwnerUserId="14803" LastActivityDate="2015-03-15T11:08:18.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19161" PostTypeId="2" ParentId="19158" CreationDate="2015-03-15T16:11:47.457" Score="1" Body="&lt;p&gt;With linear elements, you put the shape functions together into little hat-shaped basis functions and overlap them like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lt5Lw.png&quot; alt=&quot;linear basis&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With quadratic shape functions, you do something similar and overlap them in a similar way, &lt;em&gt;but&lt;/em&gt; you gain an interior point inside the element that doesn't overlap with its neighboring element:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nwAV6.gif&quot; alt=&quot;quadratic basis&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*Images from &lt;a href=&quot;http://hplgit.github.io/INF5620/doc/pub/sphinx-fem/._main_fem003.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hplgit.github.io/INF5620/doc/pub/sphinx-fem/._main_fem003.html&lt;/a&gt; and &lt;a href=&quot;http://nptel.ac.in/courses/112104116/lecture-8/8_3.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nptel.ac.in/courses/112104116/lecture-8/8_3.htm&lt;/a&gt;, respectively.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-03-15T16:11:47.457" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19163" PostTypeId="1" AcceptedAnswerId="19164" CreationDate="2015-03-15T21:36:37.880" Score="1" ViewCount="4196" Body="&lt;p&gt;Is there a free C or C++ library including a routine for the eigenvalues of a general complex matrix? I checked a number of linear algebra packages like Eigen, but there does not seem to be support for complex matrices.&lt;/p&gt;&#xA;" OwnerUserId="14807" LastActivityDate="2016-05-30T09:39:16.943" Title="eigenvalues of a general complex matrix in C++" Tags="|linear-algebra|c++|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19164" PostTypeId="2" ParentId="19163" CreationDate="2015-03-15T21:53:05.383" Score="6" Body="&lt;p&gt;Eigen can compute eigenvalues of a complex matrix. Take a look at this documentation page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://eigen.tuxfamily.org/dox/classEigen_1_1ComplexEigenSolver.html&quot;&gt;http://eigen.tuxfamily.org/dox/classEigen_1_1ComplexEigenSolver.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an Eigen version of a Lapack, zgeev example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;complex&amp;gt;&#xA;#include &amp;lt;Eigen/Eigenvalues&amp;gt; &#xA;&#xA;void complexEigenproblem()&#xA;{&#xA;  const int n = 4;&#xA;  Eigen::MatrixXcd a(n, n);&#xA;  typedef std::complex&amp;lt;double&amp;gt; C;&#xA;  a &amp;lt;&amp;lt;&#xA;    C(-3.97, -5.04), C(-4.11, 3.70), C(-0.34, 1.01), C(1.29, -0.86),&#xA;    C(0.34, -1.50), C(1.52, -0.43), C(1.88, -5.38), C(3.36, 0.65),&#xA;    C(3.31, -3.85), C(2.50, 3.45), C(0.88, -1.08), C(0.64, -1.48),&#xA;    C(-1.10, 0.82), C(1.81, -1.59), C(3.25, 1.33), C(1.57, -3.44);&#xA;  Eigen::ComplexEigenSolver&amp;lt;Eigen::MatrixXcd&amp;gt; ces;&#xA;  ces.compute(a);&#xA;  std::cout &amp;lt;&amp;lt; &quot;The eigenvalues of a are:&quot; &amp;lt;&amp;lt; std::endl &amp;lt;&amp;lt; ces.eigenvalues() &amp;lt;&amp;lt; std::endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The eigenvalues are the same (to five significant digits) as those from zgeev:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.nag.com/lapack-ex/node92.html&quot;&gt;http://www.nag.com/lapack-ex/node92.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note: Eigen and zgeev order the output eigenvalues differently)&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2015-03-16T17:11:32.443" LastActivityDate="2015-03-16T17:11:32.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19165" PostTypeId="2" ParentId="19137" CreationDate="2015-03-16T01:06:10.530" Score="18" Body="&lt;p&gt;For the first part of my question, I found this very useful comparison for performance of different linear interpolation methods using python libraries:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://nbviewer.ipython.org/github/pierre-haessig/stodynprog/blob/master/stodynprog/linear_interp_benchmark.ipynb&quot; rel=&quot;noreferrer&quot;&gt;http://nbviewer.ipython.org/github/pierre-haessig/stodynprog/blob/master/stodynprog/linear_interp_benchmark.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is list of methods collected so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Standart interpolation, structured grid:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.ndimage.interpolation.map_coordinates.html&quot; rel=&quot;noreferrer&quot;&gt;http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.ndimage.interpolation.map_coordinates.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.RegularGridInterpolator.html&quot; rel=&quot;noreferrer&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.RegularGridInterpolator.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/rncarpio/linterp/&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/rncarpio/linterp/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Unstructured (scattered) grid:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.LinearNDInterpolator.html#scipy.interpolate.LinearNDInterpolator&quot; rel=&quot;noreferrer&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.LinearNDInterpolator.html#scipy.interpolate.LinearNDInterpolator&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.griddata.html&quot; rel=&quot;noreferrer&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.griddata.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.Rbf.html&quot; rel=&quot;noreferrer&quot;&gt;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.Rbf.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2 large projects that include interpolation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/sloriot/cgal-bindings&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/sloriot/cgal-bindings&lt;/a&gt; (parts of CGAL, licensed GPL/LGPL)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.earthsystemcog.org/projects/esmp/&quot; rel=&quot;noreferrer&quot;&gt;https://www.earthsystemcog.org/projects/esmp/&lt;/a&gt; (University of Illinois-NCSA License ~= MIT + BSD-3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sparse grids:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/EconForge/Smolyak&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/EconForge/Smolyak&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/EconForge/dolo/tree/master/dolo/numeric/interpolation&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/EconForge/dolo/tree/master/dolo/numeric/interpolation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/py_src/sparse_grid/sparse_grid.html&quot; rel=&quot;noreferrer&quot;&gt;http://people.sc.fsu.edu/~jburkardt/py_src/sparse_grid/sparse_grid.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://aerodynamics.lr.tudelft.nl/~rdwight/work_sparse.html&quot; rel=&quot;noreferrer&quot;&gt;https://aerodynamics.lr.tudelft.nl/~rdwight/work_sparse.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pypi.python.org/pypi/puq&quot; rel=&quot;noreferrer&quot;&gt;https://pypi.python.org/pypi/puq&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Kriging (Gaussian Process):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://scikit-learn.org/stable/modules/generated/sklearn.gaussian_process.GaussianProcess.html&quot; rel=&quot;noreferrer&quot;&gt;http://scikit-learn.org/stable/modules/generated/sklearn.gaussian_process.GaussianProcess.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/SheffieldML/GPy&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/SheffieldML/GPy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://software.sandia.gov/svn/surfpack/trunk/&quot; rel=&quot;noreferrer&quot;&gt;https://software.sandia.gov/svn/surfpack/trunk/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://openmdao.org/dev_docs/_modules/openmdao/lib/surrogatemodels/kriging_surrogate.html&quot; rel=&quot;noreferrer&quot;&gt;http://openmdao.org/dev_docs/_modules/openmdao/lib/surrogatemodels/kriging_surrogate.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;General GPL licensed:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/rncarpio/delaunay_linterp&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/rncarpio/delaunay_linterp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://tasmanian.ornl.gov/about.html&quot; rel=&quot;noreferrer&quot;&gt;Tasmanian&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Toolkit for Adaptive Stochastic Modeling and Non-Intrusive Approximation - is a robust library for high dimensional integration and &#xA;interpolation as well as parameter calibration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python binding for Tasmanian:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/rncarpio/py_tsg&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/rncarpio/py_tsg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/sloriot/cgal-bindings&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/sloriot/cgal-bindings&lt;/a&gt; (parts of CGAL, licensed GPL/LGPL)&lt;/p&gt;&#xA;" OwnerUserId="5401" LastEditorUserId="5401" LastEditDate="2016-06-23T02:42:14.117" LastActivityDate="2016-06-23T02:42:14.117" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19166" PostTypeId="2" ParentId="19157" CreationDate="2015-03-16T05:40:53.213" Score="1" Body="&lt;p&gt;Assuming the $L_i$ are nonzero, such an algorithm will probably suffice in mitigating overflow issues, by analogy to the analysis done for norms; the Mahalanobis distance is like a weighted norm. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-03-16T05:40:53.213" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19167" PostTypeId="1" CreationDate="2015-03-16T06:53:55.403" Score="5" ViewCount="544" Body="&lt;p&gt;I have a complex, non-Hermitian matrix $\mathbf{A}$, for which I need to find a few eigenvalues and eigenvectors in the generalised eigenvalue problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A}\cdot \mathbf{x} = \lambda \mathbf{B} \cdot \mathbf{x},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with real symmetric positive definite matrix $\mathbf{B}$. I am currently using &lt;code&gt;scipy.sparse.linalg.eigs&lt;/code&gt;, which is a Python wrapper around ARPACK routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have successfully found both the eigenvalues and right eigenvectors, however I also need the corresponding left eigenvectors satisfying:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{y}\cdot\mathbf{A} = \lambda \mathbf{y}\cdot \mathbf{B}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(According to my notation there is no complex-conjugation required on these left eigenvectors, but obviously undoing a complex-conjugate is easy enough).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can see, neither scipy nor ARPACK provides a ready-made solution to this problem, not could they be easily modified to give both sets of vectors. The possible solutions I have come across are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Transpose my matrix $\mathbf{A}$ and make another call to &lt;code&gt;eigs&lt;/code&gt;. This is obviously inefficient, and I'm not sure if the returned eigenvalues are guaranteed to have the same values and be in the same order, so matching the left and right eigenvectors may not be reliable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use inverse iteration, extrapolating from Numerical Recipes 2nd ed Section 11.7. Solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\mathbf{A}^T - \lambda\mathbf{B})\cdot \mathbf{y} = \mathbf{b}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;iteratively for $\mathbf{x}$, starting from some random vector $\mathbf{b}$. The problem with this approach is that it may not find a full set of eigenvectors in the case of degenerate eigenvalues, which I expect to see in some cases.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are there any better solutions available, or ways that my proposed solutions could be fixed to overcome their (perceived) limitations? &lt;/p&gt;&#xA;" OwnerUserId="1750" LastActivityDate="2022-02-17T09:50:37.283" Title="Iteratively finding both left and right eigenvectors for non-symmetric complex matrix" Tags="|python|eigensystem|fortran|scipy|arpack|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19168" PostTypeId="2" ParentId="19048" CreationDate="2015-03-16T09:43:19.960" Score="0" Body="&lt;p&gt;You can find lots of RK4 codes with many languages in this page &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rosettacode.org/wiki/Runge-Kutta_method#Python&quot; rel=&quot;nofollow&quot;&gt;http://rosettacode.org/wiki/Runge-Kutta_method#Python&lt;/a&gt;. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="14811" LastActivityDate="2015-03-16T09:43:19.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19170" PostTypeId="2" ParentId="14566" CreationDate="2015-03-16T11:21:10.113" Score="2" Body="&lt;p&gt;You can calculate the stream function yourself, and from it you can draw contours or streamlines of constant $\psi$. Let us assume you are on a two-dimensional incompressible flow&#xA;$\mathbf{u}=(u,v,0)^T$, then you can find an exact differential $d\psi$ that satisfies the mass conservation equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial u}{\partial x  } + \frac{\partial v}{\partial y} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;whence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$d\psi = udy - vdx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u = \frac{\partial \psi}{\partial y  }, \ v = -\frac{\partial \psi}{\partial x }$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over an arbitrary path immersed in the fluid, you can integrate the exact differential equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\psi - \psi_0 = \int_{path}  udy - vdx $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the implementation perspective, if you are working on structured orthogonal grids you can implement a Simpson's rule to integrate the above equation in a given direction (have in mind that if you choose to integrate along one of the coordinates, one of the terms inside the integral vanishes identically). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For three dimensional flows, one can define two stream-functions instead of one. By assuming the velocity in the continuity equation ($\nabla \cdot \mathbf{u} = 0 $) as a vector vector potential  $\mathbf{u}=\nabla \times \mathbf{B}$, where $\mathbf{B} = \phi \nabla \psi$. Thus, one can extend the above definition by noting:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{u} = \nabla \phi \times \nabla \psi $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bulk flow over a volume enclosed by the surfaces of constant $\phi$ and $\psi$ can be calculated, using the Gauss Theorem, as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ M = \int \int_{\partial\Omega} \mathbf{u} \, d\mathbf{\nu} = \oint \phi \nabla \psi \, dl$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, by taking the curl (vorticity) of the velocity potential $\mathbf{B}$ and after a bit of vector algebra you get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal{L}(\phi)\nabla \psi - \mathcal{L}(\psi)\nabla \phi = \mathbf{\omega}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where,&#xA;$\mathcal{L}(\phi) = -\mathbf{I}\nabla\cdot (\nabla \phi) + \underline {\nabla}(\nabla \phi) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: The underline indicates that the operator is a column vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last equation can be reduced to simpler cases (irrotational flow, axisymmetric). You can extend the method further, by decomposing your vector field in a set of  a curl-free and a divergence-free field by means of the Hemholtz's Theorem.&lt;/p&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="14659" LastEditDate="2015-03-30T14:02:36.020" LastActivityDate="2015-03-30T14:02:36.020" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19171" PostTypeId="1" AcceptedAnswerId="19172" CreationDate="2015-03-16T14:17:17.003" Score="1" ViewCount="100" Body="&lt;p&gt;I need to solve a Reaction-Diffusion using Finite Elements, piecewise linear elements. In this problem, a reaction $A \rightarrow B$, with rate law $ r_A = - k_A \cdot u_A $, takes part, where $u_i$ denotes concentration. Initially, $u_A = u_B = 0$. The time dependent formulation for the conservation of $A$ and $B$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \begin{matrix} &#xA;\frac{\partial \ u_A}{\partial \ t} -\Delta u_A - k \cdot u_A = f_A &#xA;\end{matrix} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \begin{matrix}&#xA; \frac{\partial \ u_B}{\partial \ t} -\Delta u_B + k \cdot u_A = 0 &#xA;\end{matrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: How is the best way to solve for $u_B$? Solving only $u_A$ seems trivial, using Crank-Nicholson as time discretization and finding a weak formulation that looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U^n[M+\delta_t\theta (A- kM)] = U^{n-1}[M+\delta_t(1-\theta) (-A+kM)] + \delta_t (F+ N)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $U^n$ denotes the solution at time step $n$; $M, A, F, N$ are the mass matrix, stiffness matrix, load vector and B.C. vector (Neumann); $\delta_t$ the time step and $\theta$ the time discretization parameter ($1/2$ for C-N).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first approach that I thought of is to solve for $u_A$ at each time step and then, using the value of $u_A$ at each time step, solve for $u_B$ using similar weak formulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U^n[M+\delta_t\theta (A + kC)] = U^{n-1}[M+\delta_t(1-\theta) (-A-kC)]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $kC$ takes the place of $kM$ in the first equation, and &#xA;$C = \int u_A \ \phi_i dx$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would imply evaluating the matrix $C$ at each time step, using the $u_A$ calculated on the previous time step, interpolated on the quadrature points, which will certainly delay the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the standard procedure or is there a different approach, more efficient? Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="14783" LastActivityDate="2015-03-16T14:34:30.720" Title="Reaction-Diffusion problem A-&gt;B, solving for B" Tags="|pde|finite-element|diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19172" PostTypeId="2" ParentId="19171" CreationDate="2015-03-16T14:34:30.720" Score="2" Body="&lt;p&gt;I would do it exactly how you describe, with the two following remarks.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I would solve the two evolutions simultaneously. In the way you describe, you will need to store and retrieve all the time steps for the solution $u_A$. Instead, you could solve sequentially for $u_A$ and $u_B$ at each time step, since $u_B$ just requires two solutions for $u_A$.&lt;/li&gt;&#xA;&lt;li&gt;If you assume that the solutions for $u_A$ and $u_B$ are solved on the same grid, then the computation of the vector $C$ is not that much expensive: because you loop on the cells to assemble the vector, you do not need to interpolate the whole solution $u_A$ in the quadrature points, but just its restriction to cell in which the quadrature points are.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2015-03-16T14:34:30.720" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19173" PostTypeId="1" CreationDate="2015-03-16T15:07:56.587" Score="2" ViewCount="259" Body="&lt;p&gt;I'm trying to solve this particular equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} = \frac{\partial}{\partial x} \big[D_{i}(x)\frac{\partial u}{\partial x} \big] + S(x,t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $i$ index denotes different layers. Let's say that $ i \in \{1,2,3\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interface conditions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-D_1(x)\frac{\partial u}{\partial x}\bigg|_{x=x_1^-} = -D_2(x)\frac{\partial u}{\partial x}\bigg|_{x=x_1^+}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-D_2(x)\frac{\partial u}{\partial x}\bigg|_{x=x_2^-} = -D_3(x)\frac{\partial u}{\partial x}\bigg|_{x=x_2^+}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and at the boundaries:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial x}\bigg|_{x=0}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial x}\bigg|_{x=L}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial conditions:&#xA;$u(x,t_0) = u_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $[0;L]$ denotes the spatial domain over which I want to solve my problem. Every single layer has its own length, let's define with $L_1,L_2$ and $L_3$ and $L = \sum_i L_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the MOL approach in order to obtain a continous time-discrete space formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover let's assume that the source term $S(x,t)$ is not zero within the layers 1 and 3, but it's null within the second layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current workflow has been:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 - Divide every signel layer into $N_i$ points with the FDM method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 - Treat the interfaces points as &quot;common&quot; points; therefore the total number of discretized points will be $N_1+N_2+N_3-2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3 - At the interfaces points ($N_1$ and $N_1+N_2-1$) enforce the interface BCs. In $N_1$ I am treating that point as part of the layer 1, while in $N_1+N_2-1$ I'm treating that point as part of the layer 2 (is that correct?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4 - At the end points I'm enforcing the null-flux conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am able to solve the equation, but I'm experiencing an increase of the overall mass in the system. As far as I know this is a common problem when dealing with conservation of mass problems and FDM; theoretically I could adopt the FVM method in order to avoid this behaviour, but I was courious to understand if I was doing right in this way or wrong. Understand if it was up to the numerical approach or up to some kind of mine bug.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover: the term $S(x,t)$ I'm assuming to be non-zero in $[1;N_1]$ and $[N_1+N_2;N_1+N_2+N_3-2]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14817" LastEditorUserId="14772" LastEditDate="2015-03-28T02:17:22.280" LastActivityDate="2015-03-28T02:17:22.280" Title="1 D Diffusion equation FDM with different layers" Tags="|matlab|pde|finite-difference|diffusion|method-of-lines|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19174" PostTypeId="1" CreationDate="2015-03-16T17:17:05.307" Score="1" ViewCount="95" Body="&lt;p&gt;I have a small system of stiff ODEs describing a chemical reaction. The right-hand side is quite complicated, as well as the Jacobian. This equation will be solved many times with different initial conditions. Given that I've already selected a stiffness-oriented method, is it realystic to create more effective code than some existing solver? For example, is it worth trying to invest into optimizing LU-decomposition for my small system by loop unrolling, or whatever?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://scicomp.stackexchange.com/q/2173/720&quot;&gt;this thread&lt;/a&gt; some general ideas are given, but maybe there are some tips about stiff ODE solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: there are only three equations. I'm using Fortran and compared the efficiency of RADAU (implicit RK), RODAS (Rosenbrock) and DLSODE (BDF method) codes. RODAS seems to work a bit faster.&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-03-17T21:18:01.020" Title="What are some tips on developing a problem-specific ODE solver?" Tags="|ode|software|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19175" PostTypeId="2" ParentId="19147" CreationDate="2015-03-16T17:18:07.637" Score="1" Body="&lt;p&gt;It looks to me like the log_like function you're providing optimizes to infinity. That would trigger this specific error. &lt;/p&gt;&#xA;" OwnerUserId="14704" LastActivityDate="2015-03-16T17:18:07.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19176" PostTypeId="1" CreationDate="2015-03-16T17:35:02.283" Score="3" ViewCount="202" Body="&lt;p&gt;One typically computes Julia sets by iterating a complex function, such as a polynomial or rational function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do rounding errors affect the results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for references on this issue, especially but not exclusively for the escape time method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only ones I have found so far are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;The [inverse iteration] method is very insensitive to round-off errors, since $f$ tends to be expanding on its Julia set, so that $f^{-1}$ tends to be contracting.'' [Milnor, &lt;em&gt;Dynamics in one complex variable&lt;/em&gt;, page 49]&lt;/li&gt;&#xA;&lt;li&gt;&quot;Theorem 1 makes it plausible that in almost all cases rounding errors do&#xA;not affect the computer graphics of Julia sets.'' [Steinmetz, &lt;em&gt;Rational iteration&lt;/em&gt;, page 175]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="640" LastActivityDate="2017-12-14T23:06:27.043" Title="Rounding errors in images of Julia sets" Tags="|numerical-analysis|floating-point|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19177" PostTypeId="1" AcceptedAnswerId="19221" CreationDate="2015-03-17T14:43:15.513" Score="10" ViewCount="364" Body="&lt;p&gt;I have found that the method of lines is a very natural way to think about the discretization of PDE's.  Therefore I always default to that mindset when presented with a new set of equations.  I have never seen a PDE where this would not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am wondering is if there are discretization methods (or types of PDEs) which can not be formulated through method of lines.  I expect that any PDE where the time derivative is implicit in the equation and can't be solved for would be one such case (although I know of no actual example of this).  I am looking for reasoning as to why the method of lines is always applicable or a counter example.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="123" LastEditDate="2015-03-24T02:23:25.197" LastActivityDate="2015-03-24T02:23:25.197" Title="Can the method of lines be used to discretize all PDEs?" Tags="|pde|method-of-lines|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19178" PostTypeId="2" ParentId="19174" CreationDate="2015-03-17T21:18:01.020" Score="3" Body="&lt;p&gt;I assume you have already verified that your system is stiff? Otherwise you&#xA;might be paying a significant performance penalty for using an implicit solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next thing I suggest is to gather more data from the ODE solvers you are using.&#xA;Typically this kind of data is available in some additional output variables or&#xA;by turning on some kind of verbose output flag.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Total number of time steps in the solution.&lt;/li&gt;&#xA;&lt;li&gt;Minimum and maximum time step sizes.&lt;/li&gt;&#xA;&lt;li&gt;Number of times in the solution where the time step is changed.&lt;/li&gt;&#xA;&lt;li&gt;Number of times the Jacobian is updated and factored.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You say your RHS is quite complicated so I'm guessing your problem may be&#xA;very nonlinear. The algorithm will change time step size either because the&#xA;problem is very nonlinear and the algorithm is having difficulty obtaining&#xA;a converged solution or because it didn't meet the specified accuracy&#xA;requirements (Have you experimented with different accuracy tolerances?)&#xA;If there are a large number of Jacobian formations and factorizations, those routines&#xA;might be candidates for optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to implement a simple fixed time-step backward Euler &#xA;solver and see how the solution time compares with the off-the-shelf solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you able to get some insights about the characteristics of your&#xA;ODE system from the above experiments, you could try to profile your code.&#xA;If you are running on Linux, you can use gprof or valgrind/callgrind &#xA;(&lt;a href=&quot;http://valgrind.org/docs/manual/cl-manual.html&quot; rel=&quot;nofollow&quot;&gt;http://valgrind.org/docs/manual/cl-manual.html&lt;/a&gt;) to&#xA;obtain data about where the time is being spent.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-03-17T21:18:01.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19180" PostTypeId="1" CreationDate="2015-03-17T22:28:28.487" Score="0" ViewCount="667" Body="&lt;p&gt;I am working now in  Beowulf Cluster and parallel processing, I want code for Fast Fourier transfer functions written in any language, e.g., C/C++. Without using FFTW library based on message passing interface method.&lt;/p&gt;&#xA;" OwnerUserId="14827" LastEditorUserId="9667" LastEditDate="2015-03-20T12:14:24.063" LastActivityDate="2018-04-30T04:36:55.553" Title="FFT parallel processing in MPI" Tags="|parallel-computing|mpi|hpc|fftw|" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19181" PostTypeId="1" CreationDate="2015-03-18T00:59:52.063" Score="0" ViewCount="362" Body="&lt;p&gt;What is the state of art gradient based algorithms in convex optimization solving non-smooth piece-wise linear functions? Thank you.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: It is different from one of my previous post in the sense that here, I am focusing on piece-wise linear functions:D&lt;/p&gt;&#xA;" OwnerUserId="7891" LastEditorUserId="7891" LastEditDate="2015-03-18T20:39:24.477" LastActivityDate="2015-03-20T05:09:21.057" Title="fast gradient method for convex piecewise linear function" Tags="|convex-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19182" PostTypeId="1" CreationDate="2015-03-18T01:17:24.767" Score="0" ViewCount="42" Body="&lt;p&gt;Basically, I am trying to implement non-smooth convex optimization in c++. I am wondering what is the state of art condition of non-smooth convex optimization. For example, what's the best method currently in practice? What is the best solver available? And what is the most recent theoretical progress/ or the most recent review of this topic? Thank you :D&lt;/p&gt;&#xA;" OwnerUserId="7891" LastActivityDate="2015-03-18T01:17:24.767" Title="state of art non smooth convex optimization" Tags="|convex-optimization|" AnswerCount="0" CommentCount="2" ClosedDate="2015-03-20T05:02:52.783" ContentLicense="CC BY-SA 3.0" />
  <row Id="19186" PostTypeId="1" CreationDate="2015-03-18T06:37:51.550" Score="7" ViewCount="1302" Body="&lt;p&gt;The transport equation is actually an advection-diffussion-reaction equation, which has the form as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial C}{\partial t} + v_1 \frac{\partial C}{\partial x} + v_2 \frac{\partial C}{\partial x} = D \left(\frac{\partial^2C}{\partial x^2}+\frac{\partial^2C}{\partial y^2}\right)-R(x,y)\cdot C+S$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C$ is the unknown substrate concentration, $v_1$ and $v_2$ are the fluid velocities in the $x$ and $y$ direction, respectively, $D$ is the diffusion coefficient, $R$ is the reaction term, and $S$ is the source term. I use the traditional FEM scheme with backward Euler for the time advancing. But it seems that there are some negative values appearing in the numerical solution I solved. Is there any method/scheme to avoid negativeness?&lt;/p&gt;&#xA;" OwnerUserId="12028" LastEditorUserId="10256" LastEditDate="2015-03-18T16:39:21.183" LastActivityDate="2015-03-18T16:39:21.183" Title="How to avoid negative values of numerical solution of transport equation using FEM scheme?" Tags="|finite-element|stability|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19187" PostTypeId="1" CreationDate="2015-03-18T12:23:23.887" Score="3" ViewCount="81" Body="&lt;p&gt;I am an undergraduate student majoring in computer science. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently, I am interested in the division operation, which is not directly supported by some architectures. While some architectures support division, the  performance is not very good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a typical application in which division performance is critical?&lt;/p&gt;&#xA;" OwnerUserId="2025" LastEditorUserId="7733" LastEditDate="2015-03-18T13:09:54.283" LastActivityDate="2015-08-07T04:09:28.080" Title="Is there any numerical application whose performance heavily depends on the division operation?" Tags="|computational-physics|computational-chemistry|computational-biology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19188" PostTypeId="2" ParentId="19186" CreationDate="2015-03-18T13:01:52.977" Score="4" Body="&lt;p&gt;Typically you would use a slope limiter (or artificial diffusion and just cross your fingers) which detects where the solution has gone negative and modifies the solution to restore positivity (often by modifying the gradient of the solution in order to maintain conservation, at least in conservative schemes like Discontinuous Galerkin and Finite Volume). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also more general options - &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999113002180&quot; rel=&quot;nofollow&quot;&gt;Ridzal, Bochev and Shashkov&lt;/a&gt; have a nice trick where they solve a bound-constrained optimization problem to minimize difference between the computed solution and a new positive solution.  This turns out to decouple into a cheap iterative method which is computed independently for each solution coefficient. The iteration appears to converge very rapidly. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-03-18T13:01:52.977" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19189" PostTypeId="1" CreationDate="2015-03-18T14:27:12.780" Score="4" ViewCount="442" Body="&lt;p&gt;My task is to simulate quantum evolution. To do that I need to perform this operation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w = e^{-itH}v$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H$ is a sparse matrix and $v$ is the initial column vector. I am wondering if there is a way to calculate vector $w$ without evaluating the MatrixExp? What I mean is: if $t$ is really small (time step) is there some kind of an algorithm (stable and accurate) which uses only matrix-vector multiplication?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w(t) = \left(e^{-i\frac{t}{n}H}\right)^{n}v\\&#xA;\frac{t}{n} = dt$$&#xA;Algorithm would go like this:&#xA;$$w(0) = v \\&#xA;w(dt) = e^{-idtH}w(0)\\&#xA;w(2dt) = e^{-idtH}w(dt)\\&#xA;\ldots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I don't know, replace &#xA;$$e^{-idtH}\approx 1 -idtH$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing should remain unchanged: norm of the vector $||w(t)||=1$&lt;/p&gt;&#xA;" OwnerUserId="14780" LastActivityDate="2015-03-22T20:22:47.947" Title="Propagating Schrodinger equation" Tags="|algorithms|quantum-mechanics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19190" PostTypeId="2" ParentId="19189" CreationDate="2015-03-18T16:31:01.063" Score="1" Body="&lt;p&gt;By definition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e^{idtH} = \sum\limits_{k=0}^\infty \frac{1}{k!}\left( i dt H\right)^k$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is easily approximated by truncating the sum after a number of terms.  Ideally you would want to use only two terms so that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e^{i dt H} \approx 1 + i dt H$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What that means is that $\frac{1}{2}(i dt H)^2$ must have a small effect on the solution.  Assuming $H$ has eigenvalues $\lambda_i$, then you need $dt^2 \lambda_i^2 &amp;lt;&amp;lt; 1$ for every eigenvalue.  As long as you choose a small enough $dt$ that this is true, you can avoid the matrix exponential with only the first two terms of the sum.  &lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2015-03-18T16:31:01.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19191" PostTypeId="2" ParentId="19189" CreationDate="2015-03-18T17:49:42.383" Score="3" Body="&lt;p&gt;In addition to Godric Seer's suggestion, you can also use rational approximations to the exponential, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e^z = \frac{1 + z/2}{1 - z/2} + \mathcal{O}(z^3)$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to devise more accurate approximations to the matrix exponential:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$e^{-itH} \approx (I + itH)^{-1}(I - itH)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approximation has the advantage of being a unitary operator, so that in exact arithmetic $\|w\|$ remains unchanged. Of course you have to watch out in floating point arithmetic if the condition number of $H$ is very large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;a href=&quot;http://www.cs.cornell.edu/cv/researchpdf/19ways+.pdf&quot; rel=&quot;nofollow&quot;&gt;19 Dubious Ways to Compute the Exponential of a Matrix&lt;/a&gt; is worth a read, and if you want to dig deeper, &lt;a href=&quot;https://books.google.com/books?isbn=1611970733&quot; rel=&quot;nofollow&quot;&gt;Saad's book&lt;/a&gt; is both very readable and very comprehensive.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-03-18T17:49:42.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19192" PostTypeId="1" AcceptedAnswerId="19229" CreationDate="2015-03-18T18:06:00.867" Score="2" ViewCount="1430" Body="&lt;p&gt;Does anyone know where could I find a code (in Matlab or Mathematica, for example) for he &lt;a href=&quot;http://en.wikipedia.org/wiki/Stokes_flow#Stokes_equations&quot; rel=&quot;nofollow&quot;&gt;Stokes equation&lt;/a&gt; in 2D? It has been solved numerically by so many people and referenced in so many paper that I guess someone has had the generous (and in science, appropriate) idea to share it somewhere. Thanks a lot. &lt;/p&gt;&#xA;" OwnerUserId="9865" LastActivityDate="2018-01-28T19:31:23.040" Title="2D Stokes equation Code" Tags="|finite-difference|fluid-dynamics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19193" PostTypeId="1" CreationDate="2015-03-18T19:36:21.380" Score="1" ViewCount="308" Body="&lt;p&gt;I'm trying to solve an advection - convection problem using an implicit upwind scheme - you can see here the finite difference discretization used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2V71y.png&quot; alt=&quot;Advection convection implicit upwind&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start the model (built from scratch on Scilab) and I run some convergence tests.&lt;br&gt;&#xA;The big problem is that the velocity v is enormous - about 200m/s.&lt;br&gt;&#xA;The spatial discretization I use is of 2m, and after some tests it works just fine.&lt;br&gt;&#xA;However, the time discretization is what really bugs me. I try to run some scripts by discretizing the time as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time = 0:dt:tend&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I do some tests for different timescales, all fractions of tend. More specifically, the array of different timesteps I use is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dtlist = tend/10^(0:5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, for example, for tend=1s I'll test for dtlist = [0.00001, 0.0001, 0.001, 0.01, 0.1, 1]s.&lt;br&gt;&#xA;I run the test, and what I get is this:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/CO72J.png&quot; alt=&quot;convergence 0.1s&quot;&gt;&lt;br&gt;&#xA;and&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/oFbLK.png&quot; alt=&quot;convergence 1s&quot;&gt;&lt;br&gt;&#xA;(DT is the temperature difference between the fluid between t=0 and t=tend)&lt;br&gt;&#xA;Now... How is it possible that I have such different results, for the same timesteps of 10^-5s and 10^-4s?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, is there a general rule to determine the convergence of an implicit upwind problem? I know that the CFL condition doesn't count here, since the scheme is implicit, but maybe there are other conditions I didn't analize...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks everybody!&lt;/p&gt;&#xA;" OwnerUserId="14841" LastActivityDate="2016-03-22T18:49:28.687" Title="CFD: Doubt with time convergence in advection fully implicit upwind scheme" Tags="|convergence|advection|implicit-methods|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19194" PostTypeId="2" ParentId="19108" CreationDate="2015-03-19T08:45:46.313" Score="3" Body="&lt;p&gt;I fully agree with Wolfgang's answer, but I'd like to add something of my experience. I had recently to choose between tri and quad based FEM and after googling for it, I decided that I would code both from scratch for a fair comparison (I had a previous experience in writing FEM kernel. I used C++ and the Eigen library for matrices and vectors. I considered only cartesian meshes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer by Wolfgang seemed to hold in my cases (simple advection-diffusion problems) as well, so I can confirm this. However, the &quot;same amount of work&quot; is not exactly true. There are other differences that we need take into account for a true &lt;strong&gt;time to error&lt;/strong&gt; comparison.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The number of quadrature points might be different. In my case, the tri quadrature had 1 point while the quad one had 4 points. That makes twice less quadrature nodes in the assembly procedure for the tri based FEM. However, this depends on the problem at hand. &lt;/li&gt;&#xA;&lt;li&gt;The number of non zero entries in the resulting matrix is different for the two approaches. For a cartesian-like mesh with triangles, a internal degree of freedom typically couples with 6 others, while with quad base, it does so with 8 others. That makes the assembly more expensive and might also affect the linear solver depending on which one is used.&lt;/li&gt;&#xA;&lt;li&gt;The linear system for the quad-based FEM was (suprisingly?) easier for the iterative solver than the tri-based (less iterations for a given tolerance). I used an ILUT-preconditioned Krylov-based solver (can't remember which one exactly). I don't know if this is general or specific to the problem, but that is what I saw.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the end, I still found that quad-based FEM was faster for the time-to-error comparison, but the factor was definitly less than 10, around 3 if I remember correctly (This number of course depends on the specific implementations and problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another point for the quad-based FEM is that it is WAY easier to implement if you want to consider high order or &quot;high&quot; dimensional (&gt;2) FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this is useful!&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2015-03-19T08:45:46.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19195" PostTypeId="1" AcceptedAnswerId="19200" CreationDate="2015-03-19T10:55:41.313" Score="0" ViewCount="951" Body="&lt;p&gt;I have an overdetermined system (too many equations), expressed as Ax=b, in MATLAB. When I try to solve it using A\b, I receive the error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Warning: System is inconsistent. Solution does not exist.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I substitute values for the symbolic variables, the system can be solved, and the answer presented matches what I calculated by hand. Is MATLAB unable to handle overdetermined systems involving symbolic variables?&lt;/p&gt;&#xA;" OwnerUserId="14852" LastActivityDate="2015-03-21T00:04:57.133" Title="MATLAB: Backslash operator using symbolic variables with an overdetermined system" Tags="|matlab|linear-solver|symbolic-computation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19199" PostTypeId="1" CreationDate="2015-03-19T21:13:06.163" Score="3" ViewCount="244" Body="&lt;p&gt;I have a ADI finite difference scheme for the 2D Navier-Stokes equations that uses a second order accurate (central) approximation for the advective terms. I am ignoring the diffusive terms for now. I am trying to check the order of accuracy of the scheme by using a test case with analytical solution. For implementation reasons, the grid is non-uniform in the upper half (see footnotes for the reasons for it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is:&lt;/strong&gt; can I still get second order of accuracy? It is known that for non uniform grid the central scheme reduce to first order (see slides 17-18 &lt;a href=&quot;http://cfd.mace.manchester.ac.uk/twiki/pub/Main/TimCraftNotes_All_Access/cfd1-findiffs.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;). Note that I use equation 32 without the two rightmost terms, clearly. However  the mentioned slides say that if grid is slowly expanding, i.e., $\Delta x_i = r \Delta x_{i−1}$ with the grid expansion ratio $r$ not too far from unity, the leading order error can then still be essentially second order. Any experience with that? How small has to be $r$? I guess it depends from the problem and the value of the second derivative, since the first truncated term is proportional to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: The grid is non-uniform in the upper half because it is the only way to have a circular periodic boundary condition without rewriting all the code, i.e., I stretch the grid and I use two lines of cells as the halo for periodicity). The upper half have basically 4 line of cells missing, so I need either to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;stretch the remaining smoothly&lt;/li&gt;&#xA;&lt;li&gt;have a discontinuity on the grid size at 2 single interfaces.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I guess option (1) is the better, since I make the error smaller locally, right?&lt;/p&gt;&#xA;" OwnerUserId="10032" LastEditorUserId="9667" LastEditDate="2015-03-20T05:01:50.957" LastActivityDate="2015-03-20T05:01:50.957" Title="Computing accuracy of my finite difference scheme for uniform grid on a non-uniform grid" Tags="|finite-difference|accuracy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19200" PostTypeId="2" ParentId="19195" CreationDate="2015-03-19T21:58:07.313" Score="3" Body="&lt;p&gt;&lt;strong&gt;tl;dr:&lt;/strong&gt; &lt;em&gt;Use MATLAB primarily for numerical computation, not symbolic computation.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB was primarily designed as a numerical computation package, and its symbolic capabilities were added later. Also, as a general rule, it's easier to compute quantities with concrete values than it is to calculate the same quantities in the abstract. For instance, no closed form solutions exist for general polynomial equations of a single variable of degree 5 or greater. Given both of these trends, the expectation is that, for any given symbolic expression and MATLAB operation, MATLAB is unlikely to return a symbolic solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the specific case of overdetermined linear systems, MATLAB backslash, as noted by DaveP, is doing least-squares, so it's doing a QR factorization, followed by a linear solve, probably by LU decomposition. &lt;code&gt;rref&lt;/code&gt;, on the other hand, is putting the system into reduced row echelon form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Row_echelon_form&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;rref&lt;/code&gt; algorithm is essentially Gaussian elimination&lt;/a&gt;, probably augmented by good choices of pivots for some numerical stability. Divisions are done by pivots, which probably makes symbolic computation relatively straightforward (even if the resulting expressions are beyond heinous).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/QR_decomposition&quot; rel=&quot;nofollow&quot;&gt;QR decompositions&lt;/a&gt; can be computed with many algorithms (modified Gram-Schmidt, Givens rotations, Householder transformations). Modified Gram-Schmidt is probably the simplest, and involves norms and divisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Heuristically, since &lt;code&gt;rref&lt;/code&gt; is likely to yield simpler expressions, that is probably why &lt;code&gt;rref&lt;/code&gt; works, but backslash fails. In either case, trying to use these algorithms for symbolic manipulation is generally not a good strategy because these algorithms were designed primarily for numerical computation, not to grind through symbolic expressions. Could you use them for symbolic manipulation? Sure. Is it a good idea? Probably not.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-03-21T00:04:57.133" LastActivityDate="2015-03-21T00:04:57.133" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19201" PostTypeId="2" ParentId="19082" CreationDate="2015-03-19T22:17:35.383" Score="1" Body="&lt;p&gt;Instead of running Newton-Raphson to convergence of each subsystem, try 1 iteration on the first subsystem followed by 1 iteration on the second subsystem. This may keep the subsystems more coupled, not going to &quot;distant&quot; unrelated sub solutions. Repeat this two step iteration and see if it converges.&lt;/p&gt;&#xA;" OwnerUserId="14862" LastEditorUserId="9667" LastEditDate="2015-03-21T12:17:56.940" LastActivityDate="2015-03-21T12:17:56.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19202" PostTypeId="2" ParentId="19181" CreationDate="2015-03-20T05:09:21.057" Score="2" Body="&lt;p&gt;There are two cases I can think of (assuming minimization):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;your non-smooth piecewise linear function is convex, in which case, standard reformulations result in a convex program (or even a linear program), so you can consult a textbook in convex optimization such as the one by Boyd to find applicable algorithms. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;your non-smooth piecewise linear function is nonconvex; then the reformulation I mentioned above doesn't apply, and you probably need to provide more about the problem structure. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-03-20T05:09:21.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19204" PostTypeId="2" ParentId="19192" CreationDate="2015-03-20T13:19:30.257" Score="0" Body="&lt;p&gt;Well, implementing the Stokes equations, as you said, is rather easy. I would suggest you to do it yourself, just by reading the first 20 pages of chapter 7 on Peyret's (1987) &quot;Computational Methods For fluid flow&quot; you will be able to implement such a simple model (just drop the convective term). &lt;/p&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="14659" LastEditDate="2015-03-20T14:02:33.223" LastActivityDate="2015-03-20T14:02:33.223" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19205" PostTypeId="2" ParentId="19082" CreationDate="2015-03-20T15:29:33.417" Score="7" Body="&lt;p&gt;The issues you're running into now are not a failing of Newton-Raphson, but a question of coupling.  You're doing iterated sequential coupling -- solving each equation sequentially and then iterating until (hopeful) convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No solver choice in place of NR is going to fix this lack of convergence, as long as you are doing iterated sequential coupling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, you want to consider different coupling schemes that use more information about the coupling terms.  What @paul suggests is effectively equivalent to doing a quasi-Newton method using an approximation of the Jacobian of the coupled system that is block diagonal.  Note that this is NOT Newton-Raphson any more, because you do not have the true Jacobian of the coupled system -- instead you form an approximation of the Jacobian using fewer terms than the true Jacobian and use this instead.  This is a good starting point because it requires no additional knowledge of the physics -- you can build this entirely from pieces you already have.  Norms for the coupled system become a combination of the norms for the two systems you have.  Residuals for the coupled system become the vector of both systems' residuals.  And the approximate inverse is block diagonal.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible, but unlikely, that this will help much.  The issue is that the approximate (block diagonal) inverse will be too far from the true inverse.  One potential option is to try Jacobian-Free Newton Krylov to get a better inverse.  This is likely to not work all that much better until you start to add off-diagonal blocks to the approximate inverse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a practical standpoint -- your best bet is to bite the bullet and find a way to start to include at least some information about how the equations couple in your approximate inverse, i.e. including off-diagonal blocks in your approximate Jacobian for use in a fully implicit coupling scheme.&lt;/p&gt;&#xA;" OwnerUserId="508" LastActivityDate="2015-03-20T15:29:33.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19206" PostTypeId="2" ParentId="19192" CreationDate="2015-03-20T18:06:48.090" Score="2" Body="&lt;p&gt;You can of course implement a simple algorithm in Matlab or similar programs. Or you could use a professional implementation using modern numerical methods that is part of one of the existing finite element packages. For example, &lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/step_22.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; is the Stokes tutorial program of deal.II. (Disclaimer: I wrote this program.) I'm sure there are similar programs for libMesh, FEniCS, etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-20T18:06:48.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19207" PostTypeId="1" CreationDate="2015-03-21T09:59:37.683" Score="3" ViewCount="92" Body="&lt;p&gt;When reading about discontinuous Galerkin methods one finds the argument that these methods allow higher-order accuracy while maintaining a compact stencil (a cell only communicates with its direct neighbors) and that this is beneficial for parallel computations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can understand why a wider stencil would be bad for parallelization with domain decomposition: it would require more than one layer of overlap and thereby increase the communication cost. But how big is this penalty in practice? &lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2015-03-22T12:59:50.547" Title="communication penalty when using wide stencils in parallel computations" Tags="|parallel-computing|discretization|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19208" PostTypeId="1" CreationDate="2015-03-21T19:24:56.570" Score="5" ViewCount="877" Body="&lt;p&gt;After applying finite difference method to a Laplace/Poisson problem always arises a diagonal dominant system of equations that can be solved with Gauss-Seidel or SOR methods. If the original PDE does not have a second order derivative (Laplace operator), but  a first order operator (divergence or curl), the arising system of equations is not diagonal dominant, thus we need to solve it with another solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There any way to change it (pivoting, preconditioning, ...) so that it can be solved with Gauss-Seidel or SOR methods?&lt;/p&gt;&#xA;" OwnerUserId="14880" LastEditorUserId="1804" LastEditDate="2015-04-27T11:31:40.730" LastActivityDate="2016-01-22T17:23:13.157" Title="Can Gauss-Seidel/SOR (preconditioned?) be applied to a non-diagonally dominant matrix?" Tags="|finite-difference|linear-solver|iterative-method|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19209" PostTypeId="1" CreationDate="2015-03-21T20:52:10.140" Score="12" ViewCount="657" Body="&lt;p&gt;The complex inner product &lt;span class=&quot;math-container&quot;&gt;$\langle u,v\rangle$&lt;/span&gt; has two different definitions decided by conventions: &lt;span class=&quot;math-container&quot;&gt;$\bar{u}^Tv$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$u^T\bar{v}$&lt;/span&gt;. In BLAS, I found the routines &lt;code&gt;cdotu&lt;/code&gt;, &lt;code&gt;zdotu&lt;/code&gt;, and &lt;code&gt;cdotc&lt;/code&gt;, &lt;code&gt;zdotc&lt;/code&gt;. The former two routines actually compute &lt;span class=&quot;math-container&quot;&gt;$u^Tv$&lt;/span&gt; (a fake inner product!) and the last two routines conjugate the first vector in the inner product. Also, by either definition (conjugate &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;), &lt;span class=&quot;math-container&quot;&gt;$\langle u,v\rangle=\overline{\langle v,u\rangle}$&lt;/span&gt; with conjugation! Moreover, as pointed out in a comment, choosing the principal values for multi-valued complex functions can be convention dependent.&lt;/p&gt;&#xA;&lt;p&gt;My question is: does this complication cause true danger for use of complex arithmetic in scientific computing? This issue is emphasised by the authors of deal.ii who suggest to always split complex numbers into real part and imaginary part and use the real arithmetic only. But I never found the splitting approach is convenient. For example, think about the Perfectly Matched Layer &lt;a href=&quot;http://en.wikipedia.org/wiki/Perfectly_matched_layer&quot; rel=&quot;nofollow noreferrer&quot;&gt;(PML)&lt;/a&gt; for time-harmonic Maxwell's equations.&lt;/p&gt;&#xA;&lt;p&gt;It seems the worry of using complex numbers is prevalent in most open source FEM softwares except FreeFem++ and libmesh. But even for the two exceptions, the complex arithmetic is less tested than the real.&lt;/p&gt;&#xA;&lt;p&gt;My final question is: shall we just always avoid using complex numbers?&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="23791" LastEditDate="2021-12-16T23:11:36.017" LastActivityDate="2021-12-16T23:11:36.017" Title="Danger of complex arithmetic in scientific computing" Tags="|linear-algebra|software|blas|complex-arithmetic|" AnswerCount="2" CommentCount="14" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19210" PostTypeId="1" AcceptedAnswerId="19246" CreationDate="2015-03-21T21:32:12.927" Score="1" ViewCount="766" Body="&lt;p&gt;I'm currently taking a course in computational physics. I'm new to computational physics and programming in general. I'm using numerical recipes to try and integrate the radial Schrodinger equation with a Lennard-jones potential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[ \frac{\hbar^2}{2m}\frac{d^2}{dr^2} + \left( E-V(r)-\frac{\hbar^2 l (l+1)}{2mr^2}\right) \right] u_l(r)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$V(r)= \epsilon \left[ \left( \frac{\rho}{r} \right)^{12}-2\left(\frac{\rho}{r}\right)^6 \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical recipes has a function called odeint which will use a fifth-order Runge-Kutta algorithm to integrate an ordinary differential equation for you. The function has an adjustable step-size which appears to be causing problems in my code. Namely, my step-size is going to zero, which causes numerical recipes to throw an error and exit prematurely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing the integration from $r_{min}=\frac{\rho}{2}$ to $r_{max}=5\rho$ numerically, and up to my minimum value analytically in order to take care of the singularity at zero. I have attached my code below, and more information on odeint and how it works can be found at: &lt;a href=&quot;http://www.itp.uni-hannover.de/Lehre/cip/odeint_c.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.itp.uni-hannover.de/Lehre/cip/odeint_c.pdf&lt;/a&gt;. Can anyone help me understand where I'm going wrong?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#define NRANSI&#xA;#include &quot;nr.h&quot;&#xA;#include &quot;nrutil.h&quot;&#xA;&#xA;#define N 2&#xA;&#xA;float dxsav,*xp,**yp;  /* defining declarations */&#xA;int kmax,kount;&#xA;&#xA;int nrhs;   /* counts function evaluations */&#xA;&#xA;/* Schrodinger equation and L-J parameters */&#xA;double alpha = 6.12; &#xA;double rho = 3.57;&#xA;double epsilon = 5.9;&#xA;double l = 1.0;&#xA;double energy = 3;&#xA;&#xA;double rmin=1.785;&#xA;double rmax=17.85;&#xA;&#xA;&#xA;void derivs(float x,float y[],float dydx[])&#xA;{&#xA;    nrhs++;&#xA;    printf(&quot;xodeint check: x=%f\n&quot;, x);&#xA;    dydx[1] = y[2];&#xA;    dydx[2]=alpha*((l*(l+1)/(alpha*x*x))+epsilon*(pow(rho/x,12)-2*pow(rho/x,6))-energy)*y[1];&#xA;}&#xA;&#xA;int main(void)&#xA;{&#xA;    int i,nbad,nok;&#xA;    float eps=1.0e-4,h1=0.1,hmin=0,x1=rmin,x2=rmax,*ystart;&#xA;&#xA;    ystart=vector(1,N);&#xA;    xp=vector(1,200);&#xA;    yp=matrix(1,10,1,200);&#xA;    ystart[1]=0.93583;&#xA;    ystart[2]=0.17385;&#xA;    nrhs=0;&#xA;    kmax=100;&#xA;    dxsav=(x2-x1)/20.0;&#xA;//  printf(&quot;%f\n&quot;, h1);&#xA;    odeint(ystart,N,x1,x2,eps,h1,hmin,&amp;amp;nok,&amp;amp;nbad,derivs,rkqs);&#xA;    printf(&quot;\n%s %13s %3d\n&quot;,&quot;successful steps:&quot;,&quot; &quot;,nok);&#xA;    printf(&quot;%s %20s %3d\n&quot;,&quot;bad steps:&quot;,&quot; &quot;,nbad);&#xA;    printf(&quot;%s %9s %3d\n&quot;,&quot;function evaluations:&quot;,&quot; &quot;,nrhs);&#xA;    printf(&quot;\n%s %3d\n&quot;,&quot;stored intermediate values:    &quot;,kount);&#xA;    printf(&quot;\n%8s %18s %15s\n&quot;,&quot;r&quot;,&quot;integral&quot;,&quot;x^2&quot;);&#xA;    for (i=1;i&amp;lt;=kount;i++)&#xA;        printf(&quot;%10.4f %16.6f %14.6f\n&quot;,xp[i],yp[1][i],xp[i]*xp[i]);&#xA;    free_matrix(yp,1,10,1,200);&#xA;    free_vector(xp,1,200);&#xA;    free_vector(ystart,1,N);&#xA;    return 0;&#xA;}&#xA;#undef NRANSI&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This outputs &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Numerical Recipes run-time error...&#xA;stepsize underflow in rkqs&#xA;...now exiting to system...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14882" LastEditorUserId="1280" LastEditDate="2015-03-28T02:17:25.433" LastActivityDate="2017-03-04T20:30:49.290" Title="Integrating radial Schrodinger equation with Lennard-Jones potential using Runge-Kutta with adaptive step size ends up with a step-size of zero" Tags="|quantum-mechanics|runge-kutta|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19211" PostTypeId="1" CreationDate="2015-03-21T22:12:07.857" Score="1" ViewCount="2432" Body="&lt;p&gt;I am quite new to FEA but need to determine if I am performing my simulations correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a cube of material with defined elastic properties (Youngs Modulus and poison ratio). I perform a simple stretching experiment ($F=-kx$) to determine the effective spring constant $k$ of the material. I can convert $k$ into Youngs modulus $E=k\frac{A}{L}$ where $A$ is area over which force is applied and $L$ is the original length before stretching. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can stretch it in two ways with the bottom face as an encastre boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i) Apply a pressure to the top face and monitor displacement&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ii) Prescribe a displacement to the top face and monitor the reaction forces&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases I apply low enough pressures/displacements so that I don't accidentally go beyond the elastic regime (&amp;lt;0.05% strain in total).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Whenever I do this, I get a Young's modulus that is wrong by about 10%, and sometimes even higher.Should FEA not give an effectively perfect result?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few notes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-One thing I am aware of is that I use the original $A$ and not the $A$ at each step of the simulation - so there will be a small error associated with that (perhaps not as small as I expect?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-My results do not depend on mesh density&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-My problem is quasi-static and the solutions do not change if my step set to solve as &quot;Static&quot; or &quot;Dynamic&quot;.&lt;/p&gt;&#xA;" OwnerUserId="14883" LastEditorUserId="14883" LastEditDate="2015-03-22T06:01:59.180" LastActivityDate="2015-03-22T06:01:59.180" Title="Determining Youngs Modulus of defined material in FEA (ABAQUS)" Tags="|finite-element|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19212" PostTypeId="1" CreationDate="2015-03-22T04:03:07.683" Score="3" ViewCount="565" Body="&lt;p&gt;I want to triangulate a point-set like the one below.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/FI65g.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like the triangulation of the point-set to have the following properties&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The triangles must have as vertices the black and orange points&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The triangles should be nicely shaped (like in Delaunay triangulations) ie not to skinny.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I would like to enforce the condition that each black segments should be the edge of some triangle in the triangulation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The other edges of the triangulation must lie within the polygonal region indicated by black segments joining orange points to orange points or orange points to the black points.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;CGAL can probably do this but it has been a nightmare to install CGAL on Windows 7. I am looking for any freely available, robust and easily installable MATLAB, Python, C++ (Visual Studio 2012 or Cygwin) solution to this problem on the Windows 7 platform. &lt;/p&gt;&#xA;" OwnerUserId="3352" LastActivityDate="2015-03-27T01:20:13.260" Title="Software for triangulating a point set (with restrictions)" Tags="|matlab|python|c++|delaunay-triangulation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19213" PostTypeId="2" ParentId="19207" CreationDate="2015-03-22T12:59:50.547" Score="3" Body="&lt;p&gt;I think that from a practical perspective, it's not an important point. Sure, you have less data to send around, and to fewer neighbors, but I don't think I've ever seen anyone quantify the impact in any meaningful way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Papers about DG tend to repeat the same arguments in favor of DG methods over and over without attribution to a source and without providing quantitative backup for their claims. This includes the one you cite here. Not all of these claims will stand up to critical review if someone were to actually compare DG codes against the corresponding Continuous Galerkin implementation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-22T12:59:50.547" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19214" PostTypeId="2" ParentId="19189" CreationDate="2015-03-22T20:22:47.947" Score="1" Body="&lt;p&gt;Is H time-dependent? If not, can not you just diagonalize H, and then express your initial vector &quot;v&quot; as a linear combination of the eigenvectors of H, and then propagate those?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If H is time-dependent, another technique (in addition to those already mentioned) would be to use a split operator technique like the Trotter decomposition.&lt;/p&gt;&#xA;" OwnerUserId="14868" LastActivityDate="2015-03-22T20:22:47.947" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19215" PostTypeId="1" CreationDate="2015-03-23T13:24:38.843" Score="3" ViewCount="2183" Body="&lt;p&gt;I am using &lt;code&gt;BVP4C&lt;/code&gt; to solve a system of ODEs which is as follows.&#xA;\begin{equation}&#xA;\left\{&#xA;\begin{aligned}&#xA;\frac{\partial f(x,y)}{\partial x} &amp;amp;- \frac{d}{ds}\big(\dot{x} f(x,y)\big) = \lambda \ddot{x}(s)\\&#xA;\frac{\partial f(x,y)}{\partial y} &amp;amp;- \frac{d}{ds}\big(\dot{y} f(x,y)\big) = \lambda \ddot{y}(s)\\&#xA;\end{aligned}&#xA;\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a constraint which is of the form&#xA;$$\dot{x}^2 + \dot{y}^2 = 1$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions are &#xA;$$x(0) = x_A, \, y(0) = y_A, \, x(l) = x_B, y(l) = y_B$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do to deal with above constraint?&lt;/p&gt;&#xA;" OwnerUserId="14896" LastEditorUserId="14896" LastEditDate="2015-03-24T07:10:09.890" LastActivityDate="2015-03-30T08:25:22.587" Title="How to solve ODEs with constraints using BVP4C?" Tags="|matlab|ode|boundary-conditions|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19216" PostTypeId="2" ParentId="19209" CreationDate="2015-03-23T16:49:06.437" Score="-1" Body="&lt;p&gt;This paper is relevant:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Branch Cuts for Elementary Complex Functions or Much Ado About Nothing's Sign Bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://people.freebsd.org/~das/kahan86branch.pdf&quot; rel=&quot;nofollow&quot;&gt;http://people.freebsd.org/~das/kahan86branch.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14899" LastActivityDate="2015-03-23T16:49:06.437" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19217" PostTypeId="1" CreationDate="2015-03-23T17:28:18.100" Score="1" ViewCount="144" Body="&lt;p&gt;I am seeking recommendations on how to compute the Binder ratio numerically accurate when doing Monte Carlo simulation on spin models. Binder ratio is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ B = \frac{\langle M^4\rangle}{\langle M^2\rangle^2}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a safe method to compute $M$ per metropolis sweep, one can get directly $M^2$ and $M^4$. If we take $N$ samples of these values, we can then get the average ones; $\langle M\rangle$, $\langle M^2\rangle$ and $\langle M^4\rangle$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then one can compute the Binder ratio. But near the critical temp, the results are not so precise. Am I having too much floating point error in $\langle M^2\rangle$ and $\langle M^4\rangle$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of how the Binder ratio looks, it was a very short simulation. A proper simulation generates the very smooth curve, with little standard error, but the spike remains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gIe3g.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: It is a parallel tempering simulation, it might be possible that the cause for the unexpected Binder Ratio could be related to measuring too early when replicas are not thermalized properly.&lt;/p&gt;&#xA;" OwnerUserId="14703" LastEditorUserId="14703" LastEditDate="2015-03-26T13:57:17.253" LastActivityDate="2015-03-26T13:57:17.253" Title="Accurate way for computing a ratio coming from Monte Carlo simulation" Tags="|numerics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19218" PostTypeId="2" ParentId="1122" CreationDate="2015-03-23T18:22:13.837" Score="0" Body="&lt;p&gt;There is an R package which supplies a fast and efficient implementation of the &quot;log-sum-exp trick&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.inside-r.org/packages/cran/matrixStats/docs/logSumExp&quot; rel=&quot;nofollow&quot;&gt;http://www.inside-r.org/packages/cran/matrixStats/docs/logSumExp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The logSumExp function accepts a numeric vector lX and outputs log(sum(exp(lX))) while avoiding underflow and overflow problems using the method you described.&lt;/p&gt;&#xA;" OwnerUserId="14901" LastActivityDate="2015-03-23T18:22:13.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19219" PostTypeId="2" ParentId="19217" CreationDate="2015-03-23T22:21:35.697" Score="2" Body="&lt;p&gt;The solution to get better estimates near the critical temperature is to get better statistics. In other words, you need more samples (or at least more &lt;em&gt;independent&lt;/em&gt; samples, for example by using better sampling techniques) -- that's precisely what makes computations near the critical temperature so difficult.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-23T22:21:35.697" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19220" PostTypeId="2" ParentId="19210" CreationDate="2015-03-24T01:42:02.723" Score="2" Body="&lt;p&gt;The first thing I'd try is an implicit integrator, to check for the possibility that your ODE is stiff. If you're using RK5 -- an explicit method -- to solve your ODE and the step size becomes very small, it could be that stability considerations are severely limiting your step size. If you want to use an ODE solver written in C, SUNDIALS or PETSc are good choices for production code. PETSc is very well supported. SUNDIALS has a mailing list that also gets a fair amount of traffic. The GNU Scientific Library also has some routines that are probably good for academic use, or prototyping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try a different RK implementation. I'm much more likely to trust code that's part of a library that people use and support than someone transcribing code from Numerical Recipes. Opinions on Numerical Recipes vary. &lt;a href=&quot;http://www.nr.com/bug-rebutt.html&quot; rel=&quot;nofollow&quot;&gt;The authors themselves say that criticism of bugs in the library is unfounded&lt;/a&gt; and list &lt;a href=&quot;http://www.nr.com/latest-known-bugs.html&quot; rel=&quot;nofollow&quot;&gt;current bug reports (now obsolete, since they have a &lt;/a&gt;&lt;a href=&quot;http://www.nr.com/forum/&quot; rel=&quot;nofollow&quot;&gt;forum)&lt;/a&gt;. However, most of the stories I've heard about Numerical Recipes are like &lt;a href=&quot;http://www.uwyo.edu/buerkle/misc/wnotnr.html&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt; -- not great. Your mileage may vary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, as a catchall, you could double-check (preferably test) to make sure that there are no bugs in the implementation of the right-hand side of your ODE, but this step would probably be my last step after ruling out the first two sources of problems.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-03-24T01:42:02.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19221" PostTypeId="2" ParentId="19177" CreationDate="2015-03-24T02:22:35.537" Score="8" Body="&lt;p&gt;One situation in which the usual method-of-lines approach cannot be used in a straightforward way is with equations that have mixed space-time derivatives..  By &quot;usual method-of-lines approach&quot;, I mean discretization of spatial derivatives followed by application of a Runge-Kutta or linear multistep method.  This usually applies only to systems of first-order (in time) evolution PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of equations with such mixed derivatives is Eq. (2.1) of &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/060676064&quot;&gt;http://epubs.siam.org/doi/pdf/10.1137/060676064&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In at least some cases, it is possible to rewrite such equations as first-order systems of evolution PDEs, but I don't immediately see a way to do it here.  There may be other tricks to apply the method of lines to such equations, but I don't know of them.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-03-24T02:22:35.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19223" PostTypeId="1" CreationDate="2015-03-24T09:03:29.703" Score="0" ViewCount="90" Body="&lt;p&gt;Parallel computing allow us to compute tasks in parallel. And if there was a system like torrent that unlike sharing files, it share computing power?&#xA;For example I have a gpu and I use only 25% of its power, so the remaing 75% is used for compute tasks of others computers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for my poor english &lt;/p&gt;&#xA;" OwnerUserId="14905" LastActivityDate="2015-03-24T09:46:53.230" Title="Already exist? parallel computing" Tags="|parallel-computing|opencl|" AnswerCount="1" CommentCount="1" ClosedDate="2015-03-24T22:12:52.673" ContentLicense="CC BY-SA 3.0" />
  <row Id="19225" PostTypeId="2" ParentId="19215" CreationDate="2015-03-24T09:43:15.310" Score="3" Body="&lt;p&gt;Introduce a new unknown $\phi(s)$ such that&#xA;$$&#xA;  (\dot x(s), \dot y(s)) = (\cos\phi(s), \sin\phi(s)),&#xA;$$&#xA;then rewrite the problem as a system of coupled first-order ODEs. You now got rid of the constraint and can use any standard ODE integrator.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-03-24T09:43:15.310" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19226" PostTypeId="2" ParentId="19223" CreationDate="2015-03-24T09:46:53.230" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;http://boinc.berkeley.edu/&quot; rel=&quot;nofollow&quot;&gt;BOINC&lt;/a&gt; project does what you describe. Some of the most famous projects using this concept are &lt;a href=&quot;https://folding.stanford.edu/&quot; rel=&quot;nofollow&quot;&gt;Folding@home&lt;/a&gt; and &lt;a href=&quot;http://setiathome.berkeley.edu/&quot; rel=&quot;nofollow&quot;&gt;SETI@home&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-03-24T09:46:53.230" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19227" PostTypeId="2" ParentId="19215" CreationDate="2015-03-25T00:25:17.027" Score="1" Body="&lt;p&gt;You call this system an ODE (&lt;a href=&quot;http://en.wikipedia.org/wiki/Ordinary_differential_equation&quot; rel=&quot;nofollow&quot;&gt;ordinary differential equation&lt;/a&gt;), but this sort of system is actually called an DAE (&lt;a href=&quot;http://en.wikipedia.org/wiki/Differential_algebraic_equation&quot; rel=&quot;nofollow&quot;&gt;differential algebraic equation&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What should I do to deal with above constraint?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The constraint $\dot{x}^2 + \dot{y}^2 = 1$ is a smaller problem than the fact that you need to determine the time evolution of $\lambda$, or more precisely analytically compute the constraint satisfied by $\lambda$. One way to do this is to first write the system in integral form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left\{&#xA;\begin{aligned}&#xA; \frac{d}{ds}\big(\dot{x} (f(x,y)-\lambda)\big) &amp;amp;= \frac{\partial f(x,y)}{\partial x}\\&#xA; \frac{d}{ds}\big(\dot{y} (f(x,y)-\lambda)\big) &amp;amp;= \frac{\partial f(x,y)}{\partial y}\\&#xA;\end{aligned}&#xA;\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then introduce dummy variables for converting it into semi-explicit form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left\{&#xA;\begin{aligned}&#xA; \frac{dx}{ds} &amp;amp;= \dot{x}\\&#xA; \frac{dy}{ds} &amp;amp;= \dot{y}\\&#xA; \frac{du}{ds} &amp;amp;= \frac{\partial f(x,y)}{\partial x}\\&#xA; \frac{dv}{ds} &amp;amp;= \frac{\partial f(x,y)}{\partial y}\\&#xA;\end{aligned}&#xA;\right.&#xA;\end{equation}&#xA;\begin{equation}&#xA;\left\{&#xA;\begin{aligned}&#xA; u &amp;amp;= \dot{x} (f(x,y)-\lambda)\\&#xA; v &amp;amp;= \dot{y} (f(x,y)-\lambda)\\&#xA; 1 &amp;amp;= \dot{x}^2 + \dot{y}^2\\&#xA;\end{aligned}&#xA;\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are lucky that the algebraic equation system can be solved for $\dot{x}$, $\dot{y}$, and $\lambda$. In general, you must use &lt;a href=&quot;http://en.wikipedia.org/wiki/Pantelides_algorithm&quot; rel=&quot;nofollow&quot;&gt;Pantelides algorithm&lt;/a&gt; (for example) to generate more constraints (and dummy variables) until your original dynamic variables are uniquely determined by the constrains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't even need $\lambda$, so let's eliminate it:&#xA;\begin{equation}&#xA;\left\{&#xA;\begin{aligned}&#xA; \dot{y}u &amp;amp;= \dot{x}v\\&#xA; 1 &amp;amp;= \dot{x}^2 + \dot{y}^2\\&#xA;\end{aligned}&#xA;\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This algebraic equation system now allows you to compute $\dot{x}$ and $\dot{y}$ from $u$ and $v$. So &lt;code&gt;BVP4C&lt;/code&gt; will only &lt;em&gt;see&lt;/em&gt; $x$, $y$, $u$, and $v$, and you will solve for $\dot{x}$ and $\dot{y}$ yourself and use it where it is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: Warning, the solution above is wrong!&lt;/strong&gt; Writing the system in integral form is not as straightforward as suggested, because we actually have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left\{&#xA;\begin{aligned}&#xA; \frac{d}{ds}\big(\dot{x} f(x,y)\big)-\lambda\frac{d}{ds}\dot{x} &amp;amp;= \frac{\partial f(x,y)}{\partial x}\\&#xA; \frac{d}{ds}\big(\dot{y} f(x,y)\big)-\lambda\frac{d}{ds}\dot{y} &amp;amp;= \frac{\partial f(x,y)}{\partial y}\\&#xA;\end{aligned}&#xA;\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could introduce dummy variables $\ddot{x}$ and $\ddot{y}$ to get an integral form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left\{&#xA;\begin{aligned}&#xA; \frac{d}{ds}\big(\dot{x} f(x,y)\big) &amp;amp;= \frac{\partial f(x,y)}{\partial x}+\lambda\ddot{x}\\&#xA; \frac{d}{ds}\big(\dot{y} f(x,y)\big) &amp;amp;= \frac{\partial f(x,y)}{\partial y}+\lambda\ddot{y}\\&#xA; \frac{d}{ds}\dot{x} &amp;amp;= \ddot{x}\\&#xA; \frac{d}{ds}\dot{y} &amp;amp;= \ddot{y}\\&#xA;\end{aligned}&#xA;\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And now we really need to apply Pantelides algorithm...&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2015-03-30T08:25:22.587" LastActivityDate="2015-03-30T08:25:22.587" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19228" PostTypeId="1" CreationDate="2015-03-25T08:46:06.173" Score="8" ViewCount="318" Body="&lt;p&gt;If I have already accurately known the eigenvalue spectrum (i.e. all eigenvalues) of a matrix, is there any efficient numerical algorithm to compute all the eigenvectors corresponding to these eigenvalues? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess with the information about eigenvalues, there should be some quicker way to compute eigenvectors of the matrix compared with simply diagonalize it without any information.&lt;/p&gt;&#xA;" OwnerUserId="14913" LastEditorUserId="276" LastEditDate="2015-03-25T18:57:44.663" LastActivityDate="2015-03-25T22:59:33.757" Title="Compute eigenvectors of a matrix with known eigenvalue spectrum" Tags="|algorithms|eigenvalues|matrix|numerics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19229" PostTypeId="2" ParentId="19192" CreationDate="2015-03-25T09:11:04.693" Score="9" Body="&lt;p&gt;In Mathematica you can do this on multiple levels. First, let's just try to solve it. We need a geometry, an equation and boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;reg = ImplicitRegion[0 &amp;lt;= x &amp;lt;= 2 &amp;amp;&amp;amp; 0 &amp;lt;= y &amp;lt;= 0.5 &amp;amp;&amp;amp; ! (x &amp;gt;= 1 &amp;amp;&amp;amp; y &amp;lt;= 0.1) &amp;amp;&amp;amp; ! (x &amp;gt;= 1 &amp;amp;&amp;amp; y &amp;gt;= 0.4), {x,y}];&#xA;rp = RegionPlot[reg, AspectRatio -&amp;gt; Automatic]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/rV91K.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, we need the equation and boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;stokesFlowOperator = {&#xA;Div[{{-1, 0}, {0, -1}}.Grad[u[x, y], {x, y}], {x, y}] + Derivative[1,0][w][x, y], &#xA;Div[{{-1, 0}, {0, -1}}.Grad[v[x, y], {x, y}], {x, y}] + Derivative[0,1][w][x, y], &#xA;Derivative[0, 1][v][x, y] + Derivative[1, 0][u][x, y]};&#xA;&#xA;bc = {DirichletCondition[u[x, y] == 4*0.3*y*(0.5 - y)/(0.41)^2,x == 0.], &#xA;   DirichletCondition[{u[x, y] == 0., v[x, y] == 0.}, 0 &amp;lt; x &amp;lt; 2], &#xA;   DirichletCondition[w[x, y] == 0., x == 2]};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Top-level solution:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK, this was the setup now to solve the equation we use &lt;code&gt;NDSolve&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{xVel, yVel, pressure} = NDSolveValue[{stokesFlowOperator == {0, 0, 0}, bc}, {u, v, w}, {x, y} \[Element] reg, Method -&amp;gt; {&quot;FiniteElement&quot;, &#xA;     &quot;InterpolationOrder&quot; -&amp;gt; {u -&amp;gt; 2, v -&amp;gt; 2, w -&amp;gt; 1}}];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And visualize the velocity vector field over the region:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Show[rp, StreamPlot[{xVel[x, y], yVel[x, y]}, {x, y} \[Element] reg]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pFDOY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An issue raised in the comments, here is an alternative way to visualize the same stream plot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rmf = RegionMember[reg];&#xA;Show[rp, StreamPlot[{xVel[x, y], yVel[x, y]}, {x, 0, 2}, {y, 0, 0.5}, &#xA;  AspectRatio -&amp;gt; Automatic, &#xA;  RegionFunction -&amp;gt; Function[{x, y}, rmf[{x, y}]]]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives the same result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Low-level access:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, say you want to drill in deeper, you certainly can do that and there are many ways you can do that explained &lt;a href=&quot;http://reference.wolfram.com/language/FEMDocumentation/tutorial/FiniteElementProgramming.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I show one way. We use &lt;code&gt;NDSolve&lt;/code&gt; as an equation pre-processor and extract the finite element data from it. The fact that you can do that means that you have access to every stage of the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use &lt;code&gt;NDSolve&lt;/code&gt; as an equation pre-processor:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{ndstate} = &#xA; NDSolve`ProcessEquations[{stokesFlowOperator == {0, 0, 0}, bc}, {u, &#xA;   v, w}, {x, y} \[Element] reg, &#xA;  Method -&amp;gt; {&quot;FiniteElement&quot;, &#xA;    &quot;InterpolationOrder&quot; -&amp;gt; {u -&amp;gt; 2, v -&amp;gt; 2, w -&amp;gt; 1}}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will return you an &lt;code&gt;NDSolve&lt;/code&gt; state object.We load the finite element package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Needs[&quot;NDSolve`FEM`&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And extract and inspect the finite element data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;femd = ndstate[&quot;FiniteElementData&quot;];&#xA;sd = ndstate[&quot;SolutionData&quot;][[1]];&#xA;femd[&quot;Properties&quot;]&#xA;{&quot;BoundaryConditionData&quot;, &quot;FEMMethodData&quot;, &quot;PDECoefficientData&quot;, \&#xA;&quot;Properties&quot;, &quot;Solution&quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the equation pre-procesing created a couple of data structures. Let's extract and use them:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcd = femd[&quot;BoundaryConditionData&quot;];&#xA;methodData = femd[&quot;FEMMethodData&quot;];&#xA;pded = femd[&quot;PDECoefficientData&quot;];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are then the key steps in this low level approach:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dpde = DiscretizePDE[pded, methodData, sd];&#xA;dbc = DiscretizeBoundaryConditions[bcd, methodData, sd];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Get the system matrices:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{load, stiffness, damping, mass} = dpde[&quot;SystemMatrices&quot;];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You could look at them, modify them or what not.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MatrixPlot[stiffness]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/D4LgP.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To apply the boundary conditions use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DeployBoundaryConditions[{load, stiffness, None, None}, dbc]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Solve the equations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;stationarySolution = LinearSolve[stiffness, load];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Create an interpolating function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh = methodData[&quot;ElementMesh&quot;];&#xA;offsets = methodData[&quot;IncidentOffsets&quot;];&#xA;xVel = ElementMeshInterpolation[{mesh}, &#xA;   stationarySolution[[offsets[[1]] + 1 ;; offsets[[2]]]]];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And visualize, say, the x-velocity in a contour plot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ContourPlot[xVel[x, y], {x, y} \[Element] mesh, &#xA; AspectRatio -&amp;gt; Automatic, PlotRange -&amp;gt; All, &#xA; ColorFunction -&amp;gt; ColorData[&quot;TemperatureMap&quot;], Contours -&amp;gt; 10]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/01Rxs.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an overview of the finite element method in &lt;code&gt;NDSolve&lt;/code&gt; have a look &lt;a href=&quot;http://reference.wolfram.com/language/FEMDocumentation/guide/FiniteElementMethodGuide.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. &lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="11893" LastEditDate="2015-03-26T22:26:29.223" LastActivityDate="2015-03-26T22:26:29.223" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19230" PostTypeId="1" CreationDate="2015-03-25T10:00:39.293" Score="3" ViewCount="1056" Body="&lt;p&gt;I am reading &lt;em&gt;Algorithm Design Manual&lt;/em&gt; by Skiena, which says in Chapter 8, Section 8.1.4 when talking about the calculation of binomial coefficients: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Intermediate calculations can easily cause arithmetic overflow, even when the final coefficient fits comfortably within an integer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can’t think of a reason why. Shouldn’t whatever calculation is done in between always be less than the final result?&lt;/p&gt;&#xA;" OwnerDisplayName="Overflow" LastEditorUserId="7574" LastEditDate="2015-03-25T13:55:02.237" LastActivityDate="2015-03-25T14:02:11.627" Title="How can calculations cause an arithmetic overflow even if the final value fits?" Tags="|algorithms|numerical-limitations|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19232" PostTypeId="2" ParentId="19230" CreationDate="2015-03-25T10:33:00.940" Score="11" Body="&lt;p&gt;The problem with binomial coefficients is that they are a quotient of factorials, which may be rather large and in particular much larger than the final binomial coefficent:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\binom{n}{k} = \frac{n!}{k!(n-k)!}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, $\binom{10}{5}=252$ but $10!=3628800$ (the numerator). In another example $\binom{100}{5}=75287520$ and thus can be represented as a regular integer, while $100! \approx 10^{158}$ cannot be represented as an integer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, when calculating the binomial coefficient, it’s usually unwise to just calculate $n!$ and $k!(n-k)!$ and divide the two, because your computer may not be able to handle $n!$. Instead, you need to consider ways that do not contain large numbers as intermediate results such as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \binom{n}{k} = \prod\limits_{i=1}^k \frac{n+1-i}{i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, each factor is a comparibly small integer and no intermediate result is larger than the final result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even then, binomial coefficients may become very large easily (and even be beyond the limits of floats) and may only be intermediate results themselves. You may thus need to resort to calculating everything in the logarithmic domain and making use of an approximation of the factorial such as &lt;a href=&quot;http://en.wikipedia.org/wiki/Stirling&amp;#39;s_approximation&quot;&gt;Stirling’s&lt;/a&gt;. Most math libraries provide a &lt;code&gt;loggamma&lt;/code&gt; function for such purposes, which returns $\log(\Gamma(n))= \log((n-1)!)$. (On the other hand, many math libraries also provide a reasonable implementation of binomial coefficients.)&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2015-03-25T11:58:48.740" LastActivityDate="2015-03-25T11:58:48.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19233" PostTypeId="2" ParentId="19230" CreationDate="2015-03-25T14:02:11.627" Score="7" Body="&lt;p&gt;Intermediate results may overflow even if the final result doesn't. There are ways one can re-arrange an equation so that this doesn't happen but it's up to the programmer (maybe an optimizing compiler that's also a theorem solver can do it too - but I've never seen one) and the CPU won't re-order a sequence of arithmetic instructions on its own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of trying to prove it in theory I'll merely provide a trivial example of at least one instance of it happening:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume an 8 bit machine for simplicity:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int8 a = 10&#xA;int8 b = 30&#xA;int8 c&#xA;&#xA;c = (a * b) / 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The end result is 150. Which fits in an 8 bit register. But the intermediate result, &lt;code&gt;(a * b)&lt;/code&gt; is 30 x 10 which is 300 which doesn't fit in an 8 bit register and thus overflows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the overflow, if one were to run the above calculation on an 8 bit machine, the result would be a surprising 22 instead of 150 (300 mod 256 is 44 and 44/2 = 22).&lt;/p&gt;&#xA;" OwnerUserId="14917" LastActivityDate="2015-03-25T14:02:11.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19234" PostTypeId="5" CreationDate="2015-03-25T14:28:26.180" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2015-03-25T14:28:26.180" LastActivityDate="2015-03-25T14:28:26.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19235" PostTypeId="4" CreationDate="2015-03-25T14:28:26.180" Score="0" Body="Questions regarding the limitations of numerical representations of numbers and similar other than their precision." OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2015-03-26T15:21:55.790" LastActivityDate="2015-03-26T15:21:55.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19237" PostTypeId="1" AcceptedAnswerId="19244" CreationDate="2015-03-25T21:59:02.867" Score="1" ViewCount="126" Body="&lt;p&gt;I am trying to implement Nesterov's first method to solve convex piece-wise linear optimization problem, from this website:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blogs.princeton.edu/imabandit/2013/04/01/acceleratedgradientdescent/&quot; rel=&quot;nofollow&quot;&gt;https://blogs.princeton.edu/imabandit/2013/04/01/acceleratedgradientdescent/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then, such $\beta$ does not exist convex piece-wise linear function. So I am wondering what shall I put into $\beta$ for my implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: LP is not feasible because there are $2^{80}$ such hyper planes.&lt;/p&gt;&#xA;" OwnerUserId="7891" LastEditorUserId="9667" LastEditDate="2015-03-27T04:27:40.430" LastActivityDate="2015-03-27T04:27:40.430" Title="beta in Nesterov's first method for piece wise linear convex optimization problem" Tags="|convex-optimization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19238" PostTypeId="2" ParentId="19228" CreationDate="2015-03-25T22:59:33.757" Score="3" Body="&lt;p&gt;If you can invert the matrix, the simplest choice is shifted inverse iteration, which is just power iteration for $(\mu I - A)^{-1}$, where $\mu$ is some estimate of an eigenvalue whose eigenvector you want. Convergence speed depends on how close you set $\mu$ is to your desired eigenvalue and how close other eigenvalues are to $\mu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Inverse_iteration&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Inverse_iteration&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-03-25T22:59:33.757" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19239" PostTypeId="2" ParentId="480" CreationDate="2015-03-26T02:28:38.463" Score="3" Body="&lt;p&gt;The above answers are great in terms of giving you a code to use, but aren't as good in terms of theory. If you want to delve deeper into interpolating polynomials, take a look at this theoretical treatment with a few concrete examples:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Singh, Ashok K., and B. S. Bhadauria. &quot;Finite difference formulae for&#xA;  unequal sub-intervals using lagrange’s interpolation formula.&quot;&#xA;  International Journal of Mathematics and Analysis 3.17 (2009):&#xA;  815-827.    (&lt;a href=&quot;http://www.m-hikari.com/ijma/ijma-password-2009/ijma-password17-20-2009/bhadauriaIJMA17-20-2009.pdf&quot; rel=&quot;nofollow&quot;&gt;Link to PDF&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The authors use Lagrangian Interpolation (see the &lt;a href=&quot;http://en.wikipedia.org/wiki/Lagrange_polynomial&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt; article) to calculate 3-point, 4-point and 5-point interpolating polynomials, as well as their first, second and third derivatives. They have expressions for the truncation error as well, which is important to consider when using any finite difference scheme. They also have the generic formula for calculating interpolating polynomials using &lt;em&gt;N&lt;/em&gt; points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lagrangian interpolating polynomials are useful because they and their derivatives can be very accurate in the domain you are interpolating, and they do not assume an even grid spacing. Due to the nature of Lagrangian interpolating polynomials, you can never have more orders of derivatives than you have grid points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this answers your question well because the paper I cited has formulae for arbitrarily high-order finite difference schemes, which by nature are for uneven grids and are limited only by the number of grid points you include in your stencil. The paper also has a generic formula for the truncation error, which will help you evaluate the Lagrangian interpolating polynomial scheme against other schemes you might be considering. The author's paper should give the same results as Fornberg's method. Their contribution is really just tallying a few examples and giving an estimate of the error, which you may find useful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found both the paper I cited &lt;em&gt;and&lt;/em&gt; Fornberg's work to be useful for my own research.&lt;/p&gt;&#xA;" OwnerUserId="9451" LastEditorUserId="9451" LastEditDate="2015-04-06T20:17:51.940" LastActivityDate="2015-04-06T20:17:51.940" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19241" PostTypeId="2" ParentId="4658" CreationDate="2015-03-26T03:32:10.900" Score="1" Body="&lt;p&gt;I am also using the Newton's method to solve a system of non linear equations, it's a simple method, in fact it's a generalization of the known method for one variable.&#xA;In the general case, we have the following system:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_1(x_1,x_2,x_3, ...,x_n)=0 \\&#xA;f_2(x_1,x_2,x_3, ...,x_n)=0 \\&#xA;\vdots \\&#xA;f_n(x_1,x_2,x_3, ...,x_n) = 0&#xA; $$&lt;/span&gt;&#xA;Since the Newton's method is iterative, we guess the first vector of the solution then we search for correction to this last one and so on.&#xA;Firstly, let's develop the functions in Taylor series  (we neglect the terms of second and higher orders):&#xA; &lt;span class=&quot;math-container&quot;&gt;$$  &#xA;f_1(x_1^0+\delta x_1^0,x_2^0+\delta x_2^0,...)=f_1(x_1^0,x_2^0,...)+\dfrac{\partial f_1}{\partial x_1}(x_1^0,x_2^0,...)\delta x_1+\dfrac{\partial f_1}{\partial x_2}(x_1^0,x_2^0,...)\delta x_2 + ...=0  \\ f_2(x_1^0+\delta x_1^0,x_2^0+\delta x_2^0,...)=f_2(x_1^0,x_2^0,...)+\dfrac{\partial f_2}{\partial x_1}(x_1^0,x_2^0,...)\delta x_1+\dfrac{\partial f_2}{\partial x_2}(x_1^0,x_2^0,...)\delta x_2 + ...=0  \\ \vdots \\ f_n(x_1^0+\delta x_1^0,x_2^0+\delta x_2^0,...)=f_n(x_1^0,x_2^0,...)+\dfrac{\partial f_n}{\partial x_1}(x_1^0,x_2^0,...)\delta x_1+\dfrac{\partial f_n}{\partial x_2}(x_1^0,x_2^0,...)\delta x_2 + ...=0  \\ &#xA;$$&lt;/span&gt;&#xA;we can write it in matrix form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix}&#xA;\dfrac{\partial f_1}{\partial x_1}(x_1^0,x_2^0,...,x_n^0) &amp;amp; \dfrac{\partial f_1}{\partial x_2}(x_1^0,x_2^0,...,x_n^0) \dots \dfrac{\partial f_1}{\partial x_n}(x_1^0,x_2^0,...,x_n^0) \\ \\ &#xA;\dfrac{\partial f_2}{\partial x_1}(x_1^0,x_2^0,...,x_n^0) &amp;amp; \dfrac{\partial f_2}{\partial x_2}(x_1^0,x_2^0,...,x_n^0) \dots \dfrac{\partial f_2}{\partial x_n}(x_1^0,x_2^0,...,x_n^0) \\ &#xA;~~~~\vdots &amp;amp; \vdots ~~~~~~~~~~\vdots \\ \\&#xA;\dfrac{\partial f_n}{\partial x_1}(x_1^0,x_2^0,...,x_n^0) &amp;amp; \dfrac{\partial f_n}{\partial x_2}(x_1^0,x_2^0,...,x_n^0) \dots \dfrac{\partial f_n}{\partial x_n}(x_1^0,x_2^0,...,x_n^0)&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\delta x_1 \\ \\ \\  \delta x_2 \\ \vdots \\  \\ \\ \delta x_n   &#xA;\end{bmatrix} =&#xA;\begin{bmatrix}&#xA;-f_1(x_1^0,x_2^0,...,x_n^0) \\ \\ \\ -f_2(x_1^0,x_2^0,...,x_n^0) \\ \vdots \\ \\ \\ -f_n(x_1^0,x_2^0,...,x_n^0) &#xA;\end{bmatrix}&#xA;$$&lt;/span&gt; or in a compact form: &lt;span class=&quot;math-container&quot;&gt;$J( \vec{x^i})\vec{\delta}x = -\vec{R}(\vec{x^i})$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is called the jacobian matrix (&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you give the initial guess: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{x^0} = &#xA;\begin{bmatrix}&#xA;x_1^0 \\&#xA;x_2^0 \\&#xA;x_3^0 \\&#xA;\vdots \\&#xA;x_n^0\\&#xA;\end{bmatrix}&#xA; $$&lt;/span&gt;&#xA;Then you solve the system for the correction vector (&lt;span class=&quot;math-container&quot;&gt;$\vec{\delta}x$&lt;/span&gt;)&#xA;when you obtain the values of &lt;span class=&quot;math-container&quot;&gt;$\vec{\delta}x$&lt;/span&gt; you find the new vector &lt;span class=&quot;math-container&quot;&gt;$\vec{x^1}$&lt;/span&gt;&#xA;which is &lt;span class=&quot;math-container&quot;&gt;$\vec{x^1}$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$\vec{x^0}$&lt;/span&gt;+&lt;span class=&quot;math-container&quot;&gt;$\vec{\delta^1}x$&lt;/span&gt; and you repeat this to find &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the new values in every iteration until it converges to the solution (you specify the convergence test), usually the convergence test is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\| \vec{\delta x }\| &amp;lt; \epsilon  $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\|\vec{R}(\vec{x}^i)\| &amp;lt;\epsilon $&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="14930" LastEditorUserId="14930" LastEditDate="2019-07-21T19:10:29.140" LastActivityDate="2019-07-21T19:10:29.140" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="19244" PostTypeId="2" ParentId="19237" CreationDate="2015-03-26T20:02:33.863" Score="6" Body="&lt;p&gt;A piecewise linear function is not differentiable (except in the trivial case), so as you noticed this method cannot be applied - the gradient does not exist, let alone its Lipschitz constant beta.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use a variant of Nesterov's accelerated algorithm, you have two options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You replace your function by a smooth approximation and apply an accelerated gradient descent; this is described in Nesterov's paper &lt;a href=&quot;http://dx.doi.org/10.1007/s10107-004-0552-5&quot;&gt;&lt;em&gt;Smooth minimization of non-smooth functions&lt;/em&gt;, Mathematical Programming&#xA;May 2005, Volume 103, Issue 1, pp 127-152&lt;/a&gt;, or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;you use his accelerated subgradient scheme for nonsmooth convex functions; this is described in his paper &lt;a href=&quot;http://dx.doi.org/10.1007/s10107-007-0149-x&quot;&gt;&lt;em&gt;Primal-dual subgradient methods for convex problems&lt;/em&gt;, Mathematical Programming&#xA;August 2009, Volume 120, Issue 1, pp 221-259&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-03-26T20:02:33.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19245" PostTypeId="2" ParentId="19212" CreationDate="2015-03-27T00:02:54.193" Score="4" Body="&lt;p&gt;Scipy already comes with Delaunay triangulation via Qhull, and is easy to install on Windows. Here's an example. As mentioned by Tyler Olsen in a comment above, &quot;[a] normal Delaunay triangulation will work for any set of points where your enforced edges form the convex hull of the point set (as they do in this case)&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [1]: %pylab&#xA;Using matplotlib backend: TkAgg&#xA;Populating the interactive namespace from numpy and matplotlib&#xA;&#xA;In [2]: X = rand(50,2)&#xA;&#xA;In [3]: from scipy.spatial import Delaunay&#xA;&#xA;In [5]: D = Delaunay(X)&#xA;&#xA;In [6]: triplot(X[:,0], X[:,1], D.simplices)&#xA;&#xA;In [7]: plot(X[:,0], X[:,1], 'o')&#xA;&#xA;In [8]: show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XkN2l.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="14598" LastEditorUserId="14598" LastEditDate="2015-03-27T01:20:13.260" LastActivityDate="2015-03-27T01:20:13.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19246" PostTypeId="2" ParentId="19210" CreationDate="2015-03-27T01:40:20.850" Score="4" Body="&lt;p&gt;I know this problem well from my own research: it is given by the fact that the equation is &lt;em&gt;very&lt;/em&gt; stiff. Thus, it is likely that you're doing nothing wrong (--although I haven't inspected your code). So where does the stiffness come from? &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In order to solve the equation numerically, it is common to pick a grid and use finite-differences to discretize the functions and operators. For simplicity, let's say your grid is equally spaces with a spacing of $\Delta x$. Usually, one wants to choose $\Delta x$ small in order to achieve a good approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for an analysis of the terms in your equation: The kinetic energy term (i.e. the derivative) leads to upper energies of $\mathcal O(\frac{1}{\Delta x^2})$, as the largest frequency which is representable on the grid is $\mathcal O(\frac{1}{\Delta x})$. (I'm lazy so I write $\mathcal O$ here--you surely know the formulas). This term is contained in any Schrödinger equations and does not really pose a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's consider the potential functions. Those are diagonal on the grid, so a function $1/r^k$ will have a largest energy of $1/(\Delta x^k)$. In your equation, you have a maximum $k$ of $12$. It is this term which primarily kills your propagation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Explanation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Why does this term poses a large problem? Consider your complete discretized Hamiltonian matrix $H$ (which is symmetric), and an expansion of your wavefunction in terms of your grid. The explicit solution of the Schrödinger equation is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\psi(t) = \exp(i H t) \psi(t=0)\,.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that there is no time-ordering operator as the Hamiltonian is time-independent. Now make an eigendecomposition of your Hamiltonian, $H = U E U^\dagger$. Then, the exponential may be represented as $\exp(i H t) = U \exp(i E t) U^\dagger$. If you now have a very large eigenenergy $e_{max}$ in your diagonal matrix $E$, then you have a term like $exp(ie_{max} t)$ in your propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here enters the Shannon theorem (or alternatively the Fourier transformation), which states that in order to sample this function well, it should be $\Delta t \sim 1/e_{max}$. Otherwise, you get weird things like aliasing effects, which spoil your solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of convenience, one can approximate the maximal eigenenergies the complete Hamiltonian as the sum of the maximal eigenenergies of it's terms. With this, you see that the time step needs to be as small as $\Delta t \sim 1/\Delta x$. That is, if your $\Delta x$ is $10^{-2}$ (no units here, you know them), you'd need a time step of $\sim 10^{-24}$. Adaptive stepize integrators will thus lower and lower the stepsize, until they arrive their underflow constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a rather un-mathematical treatment so don't quote me on the numbers, but it intuitively explains the occuring effects.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Suggested solution&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There is a simple alternative which I think is suited to your problem, which is often called the &lt;em&gt;spectral method&lt;/em&gt;. It is applicable if your grid has a small to medium size -- say up to $N=10000$ gridpoints -- which I guess is the case here. Then, in fact, it is the method of choice for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is then simply to calculate the matrix exponential $\exp(iHt)$ as sketched above, and apply it to your initial wavefunction vector. The costs are $\mathcal O(N^3)$ for the diagonalization, but you need to execute it only once. Plus, by this you get the &lt;em&gt;exact&lt;/em&gt; numerical solution -- the solution all those ODE methods try to achieve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside is that this approach is only applicable when the Hamiltonian is time-independent. Otherwise, when it changes over time, you'd need to diagonalize it often anew.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2017-03-04T20:30:49.290" LastActivityDate="2017-03-04T20:30:49.290" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19248" PostTypeId="1" CreationDate="2015-03-27T21:24:00.633" Score="6" ViewCount="790" Body="&lt;p&gt;Say I have a simple elliptic PDE:&#xA;$$&#xA;-\nabla\cdot(K\nabla p) = f \;\;\;\text{in}\;\Omega&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the appropriate boundary conditions. I solve for $p$ using a FEM (a discontinuous Galerkin method to be exact). Let's denote this as $p^{DG}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I compute the velocity $\mathbf{u}^{DG} = -K\nabla p^{DG}$. This velocity will be used in the solution of a transport equation. Using this computed velocity is causing oscillations and instability in my solution of the transport equation, and a common technique in the area I'm working in right now is to project the velocity into the $H(div)$ space so that the normal component is continuous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been reading up on this technique and am following the paper &quot;Superconvergence and H(div) projection for discontinuous Galerkin methods.&quot; (Riviere and Bastian).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I understand conceptually how to do the projection, but am getting caught up on some of the details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper states as follows: Say we have a regular triangulation $\mathcal{T}_{h}$ of the domain $\Omega$. For each $E\in\mathcal{T}_{h}$, we seek a reconstructed velocity/flux $\mathbf{u}^{*} \in BDM_{k-1}(E)$ (Brezzi, Douglas, Marini space) s.t.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;$\int_{e}\mathbf{u}^{*}\cdot\mathbf{n}_{e}z = \int_{e}\mathbf{u}^{DG}\cdot\mathbf{n}_{e}z, \;\;\;\;\;\;\;z\in\mathbb{P}_{k-1}(e), \;\forall e\in\partial E $&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$\int_{E}\mathbf{u}^{*}\cdot\nabla w = \int_{e}\mathbf{u}^{DG}\cdot\nabla w, \;\;\;\;\;\;w\in\mathbb{P}_{k-2}(E)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$\int_{E}\mathbf{u}^{*}\cdot\mathbf{S}(\phi) = \int_{E}\mathbf{u}^{DG}\cdot\mathbf{S}(\phi), \;\;\;\phi\in M_{k}(E) = \left\{\phi \in \mathbb{P}_{k}(E): \phi|_{\partial E} = 0\right\} \;\text{and} \;\mathbf{S}(\phi) = (\phi_{y}, -\phi_{x})$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;1-3 should uniquely define $\mathbf{u}^{*}$. So on each element, I need to solve a small linear system. I am familiar with $L^{2}$ projections, which I think are much more straightforward. The main thing I am struggling with here is how to generate all the of the necessary equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example, say I have $k=3$. The $BDM_{2}(E)$ space has dimension 12. So I need to generate a 12x12 system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at (2), then $z \in \mathbb{P}_{1}(E)$. Since $E$ is triangular, I have exactly three basis functions and thus, I have 3 unique equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For (1), I am a bit confused. Since $k=3$, we have $w\in\mathbb{P}_{2}$. For discontinuous Galerkin (and CG), there are typically 6 quadratic polynomials on each element. So does this mean I get 6 equations from (1)? Do I just assign 2 of the polynomials arbitrarily to each edge...?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand (3) mostly. I need to generate some cubic bubble functions, but since I don't know how many equations I get from (1), I'm not sure how many bubble functions I need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone shed some light on this topic? Is the method above a &quot;standard&quot; way of doing it? There is a surprisingly small amount of resources available on this, which is probably the most frustrating part so far and I have worked much with $H(div)$ elements before. If anyone knows of some relevant sources here, I'd also be really grateful.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2015-03-30T09:56:26.333" LastActivityDate="2015-03-30T17:41:09.727" Title="How to project a vector into the H(div) space (in the context of finite elements)?" Tags="|pde|finite-element|discontinuous-galerkin|projection|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19250" PostTypeId="2" ParentId="6745" CreationDate="2015-03-28T19:27:02.730" Score="3" Body="&lt;p&gt;This is the procedure to derive formulations for Compact finite difference schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, assume the stencil and the general structure of the scheme. For the classical Padé scheme you have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\alpha f'_{n+1} + f'_{n} + \alpha f'_{n-1} = a \dfrac{f_{n+1} - f_{n-1}}{2 \Delta x} \qquad(1)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You write the Taylor series for the derivatives and the functions, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f'_{n+1}=f'_{n} + \Delta x f''_n + \dfrac{(\Delta x)^2}{2!} f^{(3)}_n + \cdots&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then replace the derivatives and the functions in Eq. (1) with their Taylor series. The system of equations to solve consists of the following equations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Set $f'_{n}$'s coefficients on both sides to be equal. Here we have,&#xA;$$&#xA;2\alpha + 1 = a&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$f_n$ coefficient could only appear on the right hand side and its coefficient must be set equal to zero. In this case, this does not give us any equations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Start with coefficients of the lowest powers of $\Delta x$ and set them to be equal, until you get enough equations, e.g.,&#xA;$$&#xA;a = 2 \dfrac{3!}{2!} \alpha&#xA;$$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14772" LastEditorUserId="14772" LastEditDate="2015-03-29T14:41:37.587" LastActivityDate="2015-03-29T14:41:37.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19252" PostTypeId="1" CreationDate="2015-03-29T18:29:26.847" Score="2" ViewCount="264" Body="&lt;p&gt;I am trying to solve an optimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;\min f(x)\\&#xA;&amp;amp;\text{subject to } Ax\leq b\\&#xA;&amp;amp;x \in R^{\sim 10000},\ b \in R^{\sim 10000}&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A$ is somewhat sparse (usually less than 5% populated) and I can efficiently evaluate $f(x)$ and $\nabla f(x)$. The Hessian of $f(x)$ comes at prohibitive computation times. $f(x)$ is a convex, non-linear, smooth function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried Matlab's built-in solver &lt;code&gt;fmincon&lt;/code&gt; but I keep receiving memory errors even though I am running on a system with 32GB memory. The exact Matlab settings I use are&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options = optimoptions(@fmincon,'GradObj', 'on','SubproblemAlgorithm', 'cg', 'Display', 'iter','Hessian',{'lbfgs',20}, 'MaxIter', 50, 'Diagnostics', 'on');&#xA;[x,fval] = fmincon(@(x)myObjFunc(x),x0,A,b,[],[],lb,[],[],options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would be very happy if someone could recommend a suited open source solver I could use for this problem -preferably with some Matlab interface- or even better: more elaborate settings for Matlab's &lt;code&gt;fmincon&lt;/code&gt; to circumvent the memory issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already found the &lt;code&gt;tomopt&lt;/code&gt; package. However, this is not open source. In case I cannot find any open source alternative, I will check this out.&lt;/p&gt;&#xA;" OwnerUserId="14956" LastEditorUserId="4405" LastEditDate="2015-04-06T00:44:30.847" LastActivityDate="2015-04-06T00:44:30.847" Title="Looking for open source numerical solver" Tags="|convex-optimization|constrained-optimization|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19253" PostTypeId="1" CreationDate="2015-03-29T20:16:32.567" Score="8" ViewCount="5092" Body="&lt;p&gt;Is there any library/toolbox which has implementation of incremental SVD in MATLAB. I have implemented this &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=339347&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;, it is fast but does not work well. I tried &lt;a href=&quot;http://jblevins.org/research/centroid/updatesvd.m&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; but in this also error propagates fast (within updating 5-10 points error is high). &lt;/p&gt;&#xA;" OwnerUserId="14952" LastActivityDate="2018-03-07T10:29:45.000" Title="Incremental SVD implementation in MATLAB" Tags="|linear-algebra|matlab|svd|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19254" PostTypeId="2" ParentId="19253" CreationDate="2015-03-30T03:13:04.810" Score="4" Body="&lt;p&gt;One method to compute the svd of a matrix &lt;code&gt;X&lt;/code&gt; is to first factor &lt;code&gt;X=QR&lt;/code&gt; using the QR decomposition (for stability, use pivoting, so this is &lt;code&gt;[Q,R,E] = qr(X,0)&lt;/code&gt; in Matlab), and then compute the svd of &lt;code&gt;R&lt;/code&gt;. If the matrix is very rectangular in either, then the most expensive computation is the QR factorization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus if you increment your matrix &lt;code&gt;X&lt;/code&gt; with another row or column (this is what you meant, right?), you can just update the QR factorization with Matlab's &lt;code&gt;qrinsert&lt;/code&gt; function, and then re-do the SVD calculation of &lt;code&gt;R&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a large square matrix, this method would not be as useful, since re-doing the SVD of &lt;code&gt;R&lt;/code&gt; will be time-consuming.&lt;/p&gt;&#xA;" OwnerUserId="14958" LastActivityDate="2015-03-30T03:13:04.810" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19255" PostTypeId="1" CreationDate="2015-03-30T03:27:02.730" Score="6" ViewCount="1613" Body="&lt;p&gt;I noticed something odd today. I have a matrix &lt;code&gt;X&lt;/code&gt; that is very skinny (&lt;code&gt;20800 x 200&lt;/code&gt;), double precision real numbers, not sparse, and I want the SVD of it quickly. Matlab does it rather fast:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; tic; [U,S,V] = svd(X,'econ'); toc&#xA;Elapsed time is 0.280848 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But if I ask for the SVD of its transpose, which is an extremely fat matrix, it is considerably slower.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Xt = X';&#xA;&amp;gt; tic; [UU,SS,VV] = svd(Xt,'econ'); toc&#xA;Elapsed time is 0.722308 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any ideas why this is?  It seems quite odd, since if I wanted the SVD of a fat matrix, this means I can do it faster by taking the transpose, finding the SVD of this skinny matrix, and then swapping the &quot;U&quot;s and &quot;V&quot;s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that it's because Matlab uses column major order, so in the skinny case, the &quot;U&quot; matrix is the large one and whatever routine that operates on it gets to use a stride-length of 1, whereas in the opposite case, the Matlab implementation calls something with a non-unit stride-length which is less efficient in terms of memory calls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even if there is a good reason, it begs the question, why doesn't Matlab check for fat matrices and just take the SVD of the transpose? The transpose operator is blazing fast. e.g., &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; tic; [VV,SS,UU] = svd(Xt','econ'); toc&#xA;Elapsed time is 0.293725 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives me the same &lt;code&gt;VV,UU,SS&lt;/code&gt; as above, but much faster.&lt;/p&gt;&#xA;" OwnerUserId="14958" LastEditorUserId="11747" LastEditDate="2015-03-31T09:38:51.977" LastActivityDate="2016-03-30T20:58:24.203" Title="Why is Matlab's SVD faster on skinny matrices than on fat matrices?" Tags="|matlab|svd|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19256" PostTypeId="2" ParentId="19253" CreationDate="2015-03-30T07:27:38.187" Score="8" Body="&lt;p&gt;Yes. Christopher Baker has implemented his incremental SVD method in a MATLAB package called IncPACK (&lt;a href=&quot;https://github.com/andrewssobral/imtsl/tree/master/libs/IncPACK2&quot; rel=&quot;nofollow&quot;&gt;archived on GitHub, within the imtsl project&lt;/a&gt;). It implements methods that are described in his &lt;a href=&quot;http://purl.flvc.org/fsu/fd/FSU_migr_etd-0961&quot; rel=&quot;nofollow&quot;&gt;master's thesis&lt;/a&gt;. A brief discussion of why Brand's algorithm tends to accumulate error can be found in a 2012 paper by &lt;a href=&quot;http://www.math.fsu.edu/~aluffi/archive/paper389.pdf&quot; rel=&quot;nofollow&quot;&gt;Baker, et al&lt;/a&gt;. A related method by &lt;a href=&quot;http://core.ac.uk/download/pdf/287126.pdf&quot; rel=&quot;nofollow&quot;&gt;Chahlaoui, et al&lt;/a&gt; discusses error bounds on the left singular subspace and the singular values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already mentioned these points in the comments on Stephen's answer, but it bears repeating that the methods by both Baker and by Chahlaoui scale as $O(mnk + nk^{3})$ for a truncated rank-$k$ SVD of an $m$ by $n$ matrix. For low-rank approximations, the $mnk$ term dominates and, depending on the algorithm variant, has a leading constant that is usually between 8 and 12.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like Stephen's answer, Chahlaoui's algorithm starts with a QR factorization. Stephen's answer will work for calculating left singular vectors, but a dense SVD of the $R$ matrix would have superlinear complexity in $m$ and $n$ prior to truncation (it would be $O(mn^{2})$), which would probably reduce efficiency, but be more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, I've implemented Brand's algorithm myself, and it's somewhat sensitive to the inner product tolerance used for rank truncation. I haven't used Baker's package, but I believe it would be better, because error estimates exist for Baker's algorithm (or one closely related) and not Brand's algorithm, and because the rank truncation tolerance for Baker's algorithm is on singular values, not inner products. &lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2016-05-01T03:11:53.903" LastActivityDate="2016-05-01T03:11:53.903" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19257" PostTypeId="1" AcceptedAnswerId="19266" CreationDate="2015-03-30T14:44:10.563" Score="4" ViewCount="256" Body="&lt;p&gt;At the moment I consider the following problem. I have a huge dense banded matrix $A$ which I want to factorize and use to solve linear systems $Ax=b$. $b$ has around more than 100 columns. At the moment I use &lt;code&gt;DGBTRF&lt;/code&gt; from LAPACK to factorize $A$. This routine is a level-3 BLAS accelerated one which works fast and efficiently. The final step, solving $ Ax = LUx = b$ is done using &lt;code&gt;DGBTRS&lt;/code&gt; which is not a level-3 BLAS accelerated one. In the current LAPACK version 3.5 this solver still works nearly sequentially on the columns of $b$. Namely the code looks like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IF( lnoti ) THEN&#xA;  DO 10 j = 1, n - 1&#xA;    lm = min( kl, n-j )&#xA;    l = ipiv( j )&#xA;    IF( l.NE.j ) CALL dswap( nrhs, b( l, 1 ), ldb, b( j, 1 ), ldb )&#xA;    CALL dger( lm, nrhs, -one, ab( kd+1, j ), 1, b( j, 1 ), ldb, b( j+1, 1 ), ldb )&#xA;10 CONTINUE&#xA;END IF&#xA;DO 20 i = 1, nrhs&#xA;   CALL dtbsv( 'Upper', 'No transpose', 'Non-unit', n, kl+ku,$ ab, ldab, b( 1, i ), 1 )&#xA;20 CONTINUE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is does a level 3 BLAS enabled, better performing variant of the forward/backward substitution routine exist?  &lt;/p&gt;&#xA;" OwnerUserId="569" LastActivityDate="2015-03-31T23:53:53.360" Title="Level 3 BLAS accelerated solver for banded linear systems." Tags="|lapack|blas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19258" PostTypeId="2" ParentId="11086" CreationDate="2015-03-30T17:24:05.803" Score="3" Body="&lt;p&gt;SciPy's integration tools are based on the QUADPACK numerical integration library. If you really want to get into the gory details of how quad() treats improper integrals and what specific error messages mean, you can check out the source code and documentation at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/quadpack/&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/quadpack/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specific routine that is called for infinite intervals is QAGI, and helpful documentation is available in the doc file. If Fortran isn't your thing, you could also look at the equivalent C functions in the GSL: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Numerical-Integration.html&quot; rel=&quot;nofollow&quot;&gt;https://www.gnu.org/software/gsl/manual/html_node/Numerical-Integration.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your specific problem, I think your interpretation is correct. The adaptive quadrature works by sampling the function at 22 points, fitting a polynomial, and integrating that. This result is compared to the lower-order polynomial resulting from only 15 of those points. If the two are in close agreement, then integration is halted. Otherwise, the domain is automatically bisected and the pieces evaluated separately. &lt;/p&gt;&#xA;" OwnerUserId="14704" LastActivityDate="2015-03-30T17:24:05.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19259" PostTypeId="2" ParentId="19248" CreationDate="2015-03-30T17:41:09.727" Score="2" Body="&lt;p&gt;The projection you consider is &lt;em&gt;local&lt;/em&gt; on every cell. With the spaces you cite, your condition 1 requires test functions of degree $k-1$ on each edge (constant, linear, and quadrature along the edge); for $k=3$, there are 3 linearly independent such shape functions per edge, for a total of 9. Your condition 2 requires test functions of degree $k-2$ for the cell; for $k=3$, you then need the constants and linears on the cell, of which there are 3. Your condition 3 requires test functions of degree $k$ with zero boundary values (&quot;bubble functions&quot;); for $k=3$, you have 3 such bubble functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in total that would suggest 15 conditions. That's too many for the BDM element you're considering which suggest that you need to modify the conditions you listed above (i.e., either they are stated wrongly in the paper, or you copied them wrongly, or there is something else fishy going on).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-03-30T17:41:09.727" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19260" PostTypeId="1" CreationDate="2015-03-31T01:18:46.087" Score="4" ViewCount="187" Body="&lt;p&gt;I have some non-hermitian matrix $A$, that I have the left and right eigenvectors.  (Calculated using SLEPc, by finding the eigenvectors of $A$ and $A^H$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how to orthogonalize them however.  I know that they must obey the relation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L^HR = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm not sure how to enforce this.  The normalization of the vectors isn't clear to me either, since $\left&amp;lt;l|r\right&amp;gt; = 1$, does $\left&amp;lt;l|l\right&amp;gt; = 1$? (And similarly $\left&amp;lt;r | r\right&amp;gt;$?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grahm-Schmidt (tildes represent non-orthogonalized quantities):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \tilde{q}_i =  \frac{\tilde{r}_i}{\left&amp;lt;\tilde{r}_i | \tilde{r}_i \right&amp;gt;}$$&#xA;$$ r_i = \tilde{q}_i - \sum_{j\neq i}\left&amp;lt;\tilde{l}_j|\tilde{q}_i\right&amp;gt; \tilde{l}_j^H$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;seems like it might work, but unlike grahm-schmidt for self-orthogonalizing, the first step of normalization doesn't feel right.  And what about $l_i$?  Is $l_i$ found by doing the same procedure?  i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ l_i = \tilde{q}_i - \sum_{j\neq i}\left&amp;lt;\tilde{q}_j|r_i\right&amp;gt;^H r_j^H$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some resource that can give me more information about non-hermitian eigenvalue problems?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2015-04-07T06:39:33.627" Title="Correctly orthogonalizing and normalizing eigenvectors of a non-hermitian problem" Tags="|eigenvalues|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19261" PostTypeId="1" CreationDate="2015-03-31T05:30:46.453" Score="3" ViewCount="543" Body="&lt;p&gt;I am setting up to solve a 2D Poisson equation $\nabla^2u = T$ by the finite difference method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the multigrid method. On the coarsest mesh, where the grid size is 2x2, I want to set up boundary conditions so that the right hand side, $T$, is constant. &lt;strike&gt;I do not want a Dirichlet boundary condition, where $u$ is constant, nor a Neumann boundary where the derivative of $u$ is constant.&lt;/strike&gt;&#xA;&lt;strong&gt;I do have Dirichlet boundaries. See edit, below.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the base mesh:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/OI76v.png&quot; alt=&quot;Four nodes of coarse Poisson mesh&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I set up boundary &quot;ghost&quot; nodes as so, which will hold the constant values of $T$ and influence the 4 &quot;actual&quot; nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4ML9L.png&quot; alt=&quot;Four nodes with ghost nodes&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now, using the discretized Poisson equation for $u_{0,0}$,&lt;br&gt;&#xA;$\frac{1}{h^2}(-4u_{0,0} + u_{0,1} + u_{1,0} + u_{-1,0} + u_{0,-1}) = T_{0,0}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is straightforward. The question I have is - what equation(s) to use for the boundary nodes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To further clarify with a physical correlation, I am attempting something similar to convection, where $T$ represents a temperature analogue, and $u$ an analogue of vorticity, which will then be used to calculate velocities, which then calculates temperature changes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I want these boundary nodes to be set at a constant temperature, in essence &quot;heated from below&quot; where $T_{0,2}$ and $T_{1,2}$ are set to $1.0$ and all other boundary nodes to $0.0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously when calculating temperature change for the next time step, it is trivial simply not to update the $T$ values on the boundary and leave them constant. But I'm not sure what to do with the boundaries as part of solving the Poisson equation exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it necessary to make the boundary nodes influenced by each other, i.e.&#xA;$\frac{1}{h^2}(-4u_{0,-1} + u_{1,-1} + u_{0,0}) = T_{0,-1} = 0.0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The left hand side of that equation is probably totally wrong, I could forward difference it I suppose in the y-direction, but then do I need to add a corner boundary node, $u_{-1,-1}$ for the x-direction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or does a boundary node, being outside of the &quot;fluid&quot; (base mesh) so to speak, not need to be influenced by other nodes also outside the &quot;fluid&quot;? Could I simply write the equation as a 1D forward difference:&#xA;$\frac{1}{h^2}(u_{0,-1} - 2u_{0,0} + u_{0,1}) = T_{0,-1} = 0.0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, although there are plenty of resources describing how to set up Dirichlet and Neumann boundaries, I couldn't find much on how to set up a constant right-hand side boundary. I'm sure it's trivial and I'm missing something obvious, but I appreciate the help, especially since I have no formal training in any scientific computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; In reviewing my question, I realized that I do have an implicit Dirichlet boundary of $u=0$ anywhere outside of the boundary nodes, if I choose a 4x4 mesh, and the equation for the outer nodes is, e.g. $\frac{1}{h^2}(-4u_{0,-1} + u_{-1,-1} + u_{1,-1} + u_{0,0}) = T_{0,-1}$, because $u_{0,-2}$ would be 0 in this case (sorry about the negative indexes, hopefully it's not too confusing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which would mean I just stick to central differences. But I still want to define my source term on the outer nodes to get the &quot;heated from below&quot; effect. Should I just redefine $T$ as needed at the beginning of each time step?&lt;/p&gt;&#xA;" OwnerUserId="14973" LastEditorUserId="14973" LastEditDate="2015-03-31T16:34:04.330" LastActivityDate="2015-03-31T16:34:04.330" Title="Boundary equations for constant right hand side in Poisson equation (FD)" Tags="|finite-difference|boundary-conditions|poisson|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19263" PostTypeId="1" CreationDate="2015-03-31T14:46:01.007" Score="2" ViewCount="234" Body="&lt;p&gt;This &lt;a href=&quot;http://jblevins.org/research/centroid/updating.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; says centroid decomposition (CD) is an approximation to singular value decomposition (SVD). First I do not understand CD yet, since code is available I just want to try it out how it works but I cannot figure out how to use it, it looks preety much different than SVD. Given a CD how do I get left/right singular vectors and singular values which I can use in my code?  &lt;/p&gt;&#xA;" OwnerUserId="14952" LastActivityDate="2015-03-31T14:46:01.007" Title="Using Centroid decomposition instead of SVD" Tags="|linear-algebra|matlab|svd|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19264" PostTypeId="1" CreationDate="2015-03-31T17:22:37.160" Score="1" ViewCount="730" Body="&lt;p&gt;I am working in my graduating project about the hypersonic flow, particularly the properties of the air behind a strong shock wave, I try to do a simulation for that process in Fluent but I don't know exactly if that it's possible in Fluent. Anyone who have an idea how to do that?&lt;/p&gt;&#xA;" OwnerUserId="14930" LastActivityDate="2015-04-02T22:55:54.687" Title="Hypersonic flow in Fluent" Tags="|simulation|ansys-fluent|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19265" PostTypeId="2" ParentId="19264" CreationDate="2015-03-31T18:13:06.923" Score="3" Body="&lt;p&gt;i wouldn't call this an answer yet - i assume the question you actually want to ask will require you to specify a lot more details.  is fluent capable of simulating hypersonic flow?  yes.  i'm sure you can find better literature, but &lt;a href=&quot;http://enu.kz/repository/2010/AIAA-2010-7184.pdf&quot; rel=&quot;nofollow&quot;&gt;comparisons&lt;/a&gt; have been done, so maybe that will be useful to you if you have more options than fluent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;personally, i'd advise you pay attention to the type of grid you plan to use, but i don't know what your geometry is like.  is it symmetric in any way?  blunt or sharp geometry/what kind of shock do you expect?  what metric are you most closely tracking?  something like peak magnitude of surface heating, location, or both?  what do you plan to do with chemical nonequilibrium?  'how' to do it depends on a lot of things; the list goes on and on...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;i assume this is an academic exercise, so i'm guessing this problem is going to be in translational/rotational equilibrium and vibrational nonequilibrium...and that you understand you'll see some differences between your finite difference method and fluent's cell centered finite volume method.  i don't have specific experience with your zeldovitch chemical model (zel'dovich? as in 1d znd detonation model?), more so with models by kang &amp;amp; dunn or park.  at any rate, can you ask something more specific than 'how do you set up a hypersonic flow solution in fluent'?&lt;/p&gt;&#xA;" OwnerUserId="1280" LastEditorUserId="1280" LastEditDate="2015-04-02T22:55:54.687" LastActivityDate="2015-04-02T22:55:54.687" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19266" PostTypeId="2" ParentId="19257" CreationDate="2015-03-31T23:46:50.707" Score="2" Body="&lt;p&gt;Unfortunately, it looks like presently the only BLAS routines which take advantage of triangular/band structure are both level-2, &lt;code&gt;_tbmv&lt;/code&gt; and &lt;code&gt;_tbsv&lt;/code&gt;, the latter appearing in the code snippet above. (You are asking for a &quot;&lt;code&gt;_tbsm&lt;/code&gt;&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer to your question depends on the interpretation of 'level-3 BLAS enabled'. There is presently no &quot;&lt;code&gt;_tbsm&lt;/code&gt;&quot;, so No. On the other hand, you could build &quot;&lt;code&gt;_tbmm&lt;/code&gt;&quot; and &quot;&lt;code&gt;_tbsm&lt;/code&gt;&quot; from the existing level-3 routines &lt;code&gt;_trmm&lt;/code&gt;, &lt;code&gt;_trsm&lt;/code&gt;, and &lt;code&gt;_gemm&lt;/code&gt;; so Yes.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;By the way, if this does turn out to be a missing feature, feel free to post a feature request in the &lt;a href=&quot;http://icl.cs.utk.edu/lapack-forum/&quot; rel=&quot;nofollow&quot;&gt;LAPACK forums&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5551" LastEditorUserId="5551" LastEditDate="2015-03-31T23:53:53.360" LastActivityDate="2015-03-31T23:53:53.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19267" PostTypeId="1" CreationDate="2015-04-01T02:17:48.133" Score="1" ViewCount="118" Body="&lt;p&gt;If one has to simulate oscillating plate/solid wall ( sinusoidal function of time) in a domain (a simple piston movement in 'y' direction), the obvious way would be scenario 1. For brevity, assume there is no topological changes involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scenario 1:&#xA;To use dynamic mesh and move the mesh points via the prescribed equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scenario 2:&#xA;The fluid near wall takes up a zero velocity (assuming no slip). Now, With this in mind, using an oscillating velocity boundary condition in 'U' at the plate/solid wall (working out the velocity of the fluid for the plate's displacement amplitude) as the fluid should obey the no slip condition and produce the flow features similar to a moving mesh ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will the result of scenario 2 be different to scenario 1? Is scenario 2 the right way to represent a near wall fluid with no slip? what happens to 'k' and 'pressure' in scenario 2?&lt;/p&gt;&#xA;" OwnerUserId="9956" LastActivityDate="2015-04-01T02:52:33.030" Title="alternatives to moving mesh technique" Tags="|fluid-dynamics|mesh|oscillations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19268" PostTypeId="2" ParentId="19267" CreationDate="2015-04-01T02:52:33.030" Score="1" Body="&lt;p&gt;One and two do not lead to the same flow. One is like a piston that actually compresses the flow, two will behave differently. The details are dependent on the inlet flow characteristics, but you should not expect the same behavior. If the flow is slow ($M \ll 1$), you might be able to get away with a short-time simulation of the starting conditions in a fixed box, but the two cases will rapidly diverge. You might also be able to simulate a speaker/diaphragm this way, but solving the compressible Navier-Stokes equations for very small amplitudes is usually overkill.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-04-01T02:52:33.030" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19269" PostTypeId="2" ParentId="19255" CreationDate="2015-04-01T08:39:38.987" Score="4" Body="&lt;p&gt;As I already mentioned in the comments here is a possible answer which is backed by some experiments from AlexE in a further comment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The SVD in MATLAB uses the DGESVD from LAPACK, which is based on the ideas of Gene Golub. The main thing is that is is implemented on matrices in Fortran, i.e. columwise storage. In this way processing values in the same colunm is cheap in the sense of memory transfers because the prefetch system of the CPU detects a continuous data stream and elements from the same column are most likely available to the CPU before they are requested by the algorithm. Regarding the Fortran storage scheme for matrices and how the algorithm is implemented, accessing elements in a column is more efficient as accessing elements in a row.  Furthermore having a huge number of rows, these operations take advantage out of the threading of the BLAS backend (which is the MKL in the case of MATLAB). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applied to the problem of computing the SVD of tall and skinny matrix many operations take advantages of the above mentioned techniques to accelerate the computation. If one have the transpose of such a matrix the data access scheme is not longer that regular and the number of expensive row accesses increase. Furthermore, the number of rows is too small to accelerate computations by additional parallelization in a column. &lt;/p&gt;&#xA;" OwnerUserId="569" LastActivityDate="2015-04-01T08:39:38.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19273" PostTypeId="2" ParentId="19255" CreationDate="2015-04-01T18:51:50.490" Score="2" Body="&lt;p&gt;In my opinion, and with my limited knowledge of the Matlab internals, your understanding is correct. Matlab uses a suboptimal algorithm, and I don't see other plausible reasons than &quot;they didn't think of that&quot; or &quot;they didn't care optimizing it&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Another related question to which I don't have a good answer is &quot;why doesn't LAPACK's SVD routine &lt;code&gt;DGESVD&lt;/code&gt; accept a &lt;code&gt;TRANS&lt;/code&gt; parameter, unlike several others?&quot;.)&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2015-04-01T18:57:57.183" LastActivityDate="2015-04-01T18:57:57.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19277" PostTypeId="1" CreationDate="2015-04-02T01:23:03.490" Score="2" ViewCount="2054" Body="&lt;p&gt;A bit of a simple question here. Recently, I was evaluating some line integrals using a Gaussian quadrature rule on $[-1,1]$ where the abscissae $x_{i}$ are just the roots of the Legendre polynomial $P_{n}(x)$ and the weights are&#xA;$$&#xA;w_{i} = \frac{-2}{(1-x_{i}^{2})[P_{n}^{'}(x_{i})]^{2}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Embarrassingly, I just spent an entire day trying to debug my code and painstakingly doing line integrals out by hand for comparison before realizing that I wasn't using enough quadrature points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the wiki page (&lt;a href=&quot;http://en.wikipedia.org/wiki/Gaussian_quadrature#Error_estimates&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Gaussian_quadrature#Error_estimates&lt;/a&gt;), I thought a Gaussian quadrature rule of order $n$ was accurate to polynomials of order $2n-1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was using a 12-point rule and my integrands were all polynomials with order of at most 3. It turns out this wasn't enough, but increasing the number of quadrature points to 20 fixed everything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody explain or suggest some readings on what exactly is meant by the order of the quadrature rule?&lt;/p&gt;&#xA;" OwnerUserId="6643" LastActivityDate="2015-04-02T16:14:58.603" Title="How to determine the order of accuracy of a quadrature rule" Tags="|quadrature|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19278" PostTypeId="2" ParentId="19252" CreationDate="2015-04-02T05:19:39.027" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://projects.coin-or.org/Ipopt&quot; rel=&quot;noreferrer&quot;&gt;IPOPT&lt;/a&gt; is a good interior point method solver for convex nonlinear problems, and has a &lt;a href=&quot;http://www.coin-or.org/Ipopt/documentation/node18.html&quot; rel=&quot;noreferrer&quot;&gt;MATLAB interface&lt;/a&gt;, although I haven't used the MATLAB interface. (The solver, called from GAMS, is very good.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cvxr.com/cvx/&quot; rel=&quot;noreferrer&quot;&gt;CVX&lt;/a&gt; is also a good package for convex problems, and &lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.WhatIsYALMIP&quot; rel=&quot;noreferrer&quot;&gt;YALMIP&lt;/a&gt; is slightly more general; both of these packages provide a modeling language for posing nonlinear programs, which may or may not be what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a whole host of open source optimization software under the auspices of the COIN-OR project, and if you're familiar with GAMS or AMPL, you might be able to use the NEOS server to submit your problem as a job and use the closed- and open-source solvers they have available.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-02T05:19:39.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19279" PostTypeId="1" CreationDate="2015-04-02T06:15:43.333" Score="0" ViewCount="38" Body="&lt;p&gt;In the method of weighted residual applied to boundary value problems, is it necessary for the basis function to satisfy all of the boundary conditions? Will it work even if it does not satisfy all of the boundary conditions?&lt;/p&gt;&#xA;" OwnerUserId="8062" LastActivityDate="2015-04-02T11:24:12.660" Title="In the method of weighted residual, is it necessary for the basis function to satisfy the boundary conditions?" Tags="|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19281" PostTypeId="2" ParentId="19279" CreationDate="2015-04-02T11:24:12.660" Score="1" Body="&lt;p&gt;The test functions must come from the space of variations, i.e., if your solution has to satisfy&#xA;$$&#xA;  u|_{\partial \Omega}=g&#xA;$$&#xA;then your test functions have to satisfy&#xA;$$&#xA;  v|_{\partial \Omega}=0.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a question everyone has trouble with at first, which is why I expand on it in quite some detail in lecture 21.5 &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-02T11:24:12.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19283" PostTypeId="2" ParentId="19277" CreationDate="2015-04-02T13:21:14.753" Score="2" Body="&lt;p&gt;If your integrals were all with order of at most 3, then certainly there is an implementation error. Besides, all your weights should be positive. So the minus sign in your definition of weights is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a python code snippet, in which I integrate with 2 quadrature points arbitrary polynoms of degree 3 within machine precision.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy.polynomial.legendre as leg&#xA;import numpy.polynomial&#xA;import numpy as np&#xA;&#xA;def F(x,c):&#xA;    V=numpy.polynomial.polynomial.polyvander(x,c.size)&#xA;    i=1./(np.arange(c.size)+1) # integration factor                          &#xA;    return np.dot(i*V[:,1:],c)                      &#xA;&#xA;def f(x,c):&#xA;    V=numpy.polynomial.polynomial.polyvander(x,c.size-1)&#xA;    return np.dot(V,c)                          &#xA;&#xA;[x,w]=leg.leggauss(2)&#xA;&#xA;c=np.random.randn(4)&#xA;a1=F(1,c)-F(-1,c)&#xA;a2=np.sum(f(x,c)*w)&#xA;print a1&#xA;print a2&#xA;print a1-a2&#xA;&#xA;Out[13]: array([ 0.])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2015-04-02T16:14:58.603" LastActivityDate="2015-04-02T16:14:58.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19286" PostTypeId="1" CreationDate="2015-04-02T13:57:42.533" Score="0" ViewCount="186" Body="&lt;p&gt;i am studying the solver for PageRank problems which drived from the web link graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried using METIS to divided the matrix into subdomains, but METIS can only produce subdomains with nearly equal size. For the web link graph, it usually has blocks with different sizes because the domain or host of pages has different sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, i want to find a method to divide the graph to subdomain with different sizes adaptively. Is there such a method or how can i do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you &lt;/p&gt;&#xA;" OwnerUserId="14615" LastActivityDate="2015-04-03T01:10:15.010" Title="how to partition a graph(matrix) into subdomains with different sizes" Tags="|linear-algebra|graph-theory|domain-decomposition|partitioning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19288" PostTypeId="1" AcceptedAnswerId="19291" CreationDate="2015-04-02T18:41:35.110" Score="7" ViewCount="554" Body="&lt;p&gt;Out of curiosity, I was recently trying to search what skills are required to be successful as developer in scientific computing field (e.g. CFD or similar). And to do so, I was going to through various job postings to understand what industries look for in candidates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most basic requirements are strong programming skills, in depth understanding of physics and mathematics, parallel programming, etc. which is understandable. However I also see requirements of knowledge of &lt;strong&gt;computational geometry&lt;/strong&gt; (such as &lt;strong&gt;spatial search algorithms&lt;/strong&gt; etc.) which I do not have knowledge about as I am not from Computer Science background. Can anyone highlight what are the applications of computational geometry in CFD related applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="9950" LastEditorUserId="1280" LastEditDate="2015-04-04T00:24:51.647" LastActivityDate="2015-05-03T17:07:02.187" Title="applications of computational geometry in fields such as CFD?" Tags="|fluid-dynamics|computational-geometry|terminology|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19289" PostTypeId="1" AcceptedAnswerId="19290" CreationDate="2015-04-02T20:25:57.320" Score="15" ViewCount="1277" Body="&lt;p&gt;If I were to solve a relatively small problem, that is, a problem that can be handled by a direct method like LU, then does the condition number of the linear operator affect the accuracy of the solution?&lt;/p&gt;&#xA;&lt;p&gt;One of the research problems I am working on focuses on the development of optimization techniques to solve linear systems of equations and the &amp;quot;issues&amp;quot; I am running into are that the condition numbers of the matrices can be very high.&lt;/p&gt;&#xA;&lt;p&gt;This would be an important factor to consider if I were to use an iterative method and preconditioner, but right now I am solving small problems (less than 1M degrees of freedom), so a direct solver is appropriate for now.&lt;/p&gt;&#xA;" OwnerUserId="14576" LastEditorUserId="20688" LastEditDate="2021-04-14T00:43:59.360" LastActivityDate="2021-04-14T00:43:59.360" Title="Are direct solvers affected by the condition number of a matrix?" Tags="|linear-solver|condition-number|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19290" PostTypeId="2" ParentId="19289" CreationDate="2015-04-02T22:16:11.080" Score="29" Body="&lt;p&gt;Yes, the condition number &lt;em&gt;always&lt;/em&gt; matters in floating-point arithmetic, whether you choose to solve your system with an iterative or direct method. The relative accuracy of an approximate solution to $Ax = b$ obtained from LU factorization with pivoting is $O(\kappa(A) \cdot \varepsilon)$, where $\varepsilon$ is the smallest floating point number such that $1 + \varepsilon &amp;gt; 1$ on your machine. If you're using 64-bit floats, $\varepsilon \approx 10^{-16}$, so if your matrix has a condition number of $10^{12}$ then you can only guarantee that your solution has 4 digits of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For iterative solvers, the matrix condition number enters the show even in infinite-precision arithmetic because it often dictates the theoretical convergence rate of the algorithm. With direct solvers, this only becomes a consideration once you take into account the fact that your computer operates in finite precision.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-04-02T22:16:11.080" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19291" PostTypeId="2" ParentId="19288" CreationDate="2015-04-02T23:51:37.450" Score="10" Body="&lt;p&gt;if my years in the industry have taught me anything, it's this: everything depends on the grid.  developing a robust solver that efficiently converges to machine zero might be the flashy rock star job, but the unsung heroes are the developers that improve our gridding algorithms.  if you're looking for a really great way to lose the effects of a vortex, try to capture it with a dissipative mesh.  obviously, most of us are limited by computational resources, so you can't just place points haphazardly.  when you're staring at a deadline as you fight other projects for computational resources, and your matrix is composed of hundreds of cases, each with hundreds of thousands of cells...you learn to appreciate efficient grid placement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;getting off my gridding-needs-more-attention soapbox, i would wager those jobs might require you to work with some unstructured meshing.  i'm thinking storage of multi-dimensional data, maybe nearest neighbor queries...i'm sure there are more modern buzzwords.  perhaps they'll involve some kind of grid adaptation.  there are some cases in which the solver and grid generator might be more closely coupled that you'd prefer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;all that being said, i don't write job reqs and i currently don't have funding to work on any grid algorithms.  so this is just the knothole i work in.&lt;/p&gt;&#xA;" OwnerUserId="1280" LastActivityDate="2015-04-02T23:51:37.450" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19292" PostTypeId="2" ParentId="19286" CreationDate="2015-04-03T01:10:15.010" Score="3" Body="&lt;p&gt;If different nodes have different costs, for example because different rows of your matrix have different numbers of nonzero entries, then you need to attach &lt;em&gt;weights&lt;/em&gt; to each node of your graph. Graph partitioning algorithms such as METIS allow you to do this, creating partitions where it is not the number of nodes that are about equal between partitions, but where the sum of weights of nodes are about equal.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-03T01:10:15.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19295" PostTypeId="1" AcceptedAnswerId="19333" CreationDate="2015-04-03T18:43:22.390" Score="1" ViewCount="1652" Body="&lt;p&gt;04/06/15&lt;/p&gt;&#xA;&lt;p&gt;After more research I found a candidate algorithm but it seems to be incomplete. I have also reworked the spider chart to number all of the boxes using an x,y ((origin = 1,1) lower left) coordinate method.&lt;/p&gt;&#xA;&lt;p&gt;The following sample spider chart shows attribute values of A=80 (19,19) B=30 (14,8) C=40 (7,7) and D=70 (4,18).&lt;/p&gt;&#xA;&lt;p&gt;I learned that one can &lt;a href=&quot;http://jwilson.coe.uga.edu/emt668/EMT668.Folders.F97/Patterson/EMT%20669/centroid%20of%20quad/Centroid.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;find the centroid&lt;/a&gt; (modern term for center of gravity) of a quadrilateral by drawing lines to create two triangles, finding the centroid of those, then draw a line between them, the center point of that line is the centroid.&lt;/p&gt;&#xA;&lt;p&gt;Using this method the centroid of these sample scores is (10,13).&lt;/p&gt;&#xA;&lt;p&gt;Based on knowing the x,y coordinates of the four attributes what calculation can duplicate the drawing lines method to find the centroid?&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TOHhN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;04/05/15&lt;/p&gt;&#xA;&lt;p&gt;Here is another thought on finding the center of gravity. Find the mid-point of each axis and draw a perpendicular line, the lines intersect. Is this intersection the center of gravity? If so how can this be calculated as a plot point?&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ba0Co.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;04/04/15 For clarity purposes I am putting new thoughts at the top of this question.&#xA;Geoff suggested using a Radar Graph.&lt;/p&gt;&#xA;&lt;p&gt;Here is sample radar chart. How could the &amp;quot;center of gravity&amp;quot; be calculated as a plot point? Do the intersecting lines indicate the &amp;quot;center of gravity&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/OIUu0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Original question follows:&lt;/p&gt;&#xA;&lt;p&gt;This question involves writing an algorithm (or a table) to plot a point on a square grid based on four attributes each with five levels.&lt;/p&gt;&#xA;&lt;p&gt;Each Attribute, A, B, C and D is assigned one of five Levels, Very High, High, Moderate, Low and Very Low. All four attributes affect the plot point.&lt;/p&gt;&#xA;&lt;p&gt;How many plot points are possible based on these quantities?&lt;/p&gt;&#xA;&lt;h3&gt;Examples&lt;/h3&gt;&#xA;&lt;p&gt;The following are some examples of how the points are to be plotted in the grid shown below based on various combinations of levels for the attributes. Essentially, Very High levels are near the four corners going down to Very Low in the XX-opposite corner-XX middle of the grid.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Plot point 1 - &#xA;A=Very High&#xA;B=Very Low&#xA;C=Very Low&#xA;D=Very Low&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These levels place the plot point as close as possible to box number 1, since it has the highest level of the four attributes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Plot point 625 (old 32) - &#xA;A=Moderate&#xA;B=Moderate&#xA;C=Moderate&#xA;D=Moderate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These levels place the plot point as close as possible to box number 625, since all four levels are moderate (in the middle) of the four attributes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Plot point 469 (old 19) - &#xA;A=Very High&#xA;B=High&#xA;C=Very High&#xA;D=Low&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These levels place the plot point near box number 469, based on Moderate B and Low D.&lt;/p&gt;&#xA;&lt;p&gt;Thus each combination of Attributes and Levels creates a pre-defined number of plot points.&lt;/p&gt;&#xA;&lt;p&gt;Does this make sense?&lt;/p&gt;&#xA;&lt;h3&gt;This section is in reply to Tyler&lt;/h3&gt;&#xA;&lt;p&gt;Yes, I do have a defined order that the plot points need to be in. Attribute A represents point 1, B point 25, C point 49 and D point 73. Thus when the score is Very High, the plot point is 1, 25, 49 or 73.&lt;/p&gt;&#xA;&lt;p&gt;Reiterating my example for Plot point 1/97 (above) the attribute ranks would be: A=Very High B=Very Low C=Very Low D=Very Low. On the opposite corner, plot point 49/141, the attribute ranks are: A=Very Low B=Very Low C=Very High and D=Very Low. The middle plot point, 625, consists of all attributes raking the same (though only one is possible in my scoring algorithm, Moderate)&lt;/p&gt;&#xA;&lt;p&gt;This repeats for all of the attributes and possible combinations of ranks. What I need an algorithm that will plot these points, essentially making a table of all possible points, such that each box has at least one reference to a ranking.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Such an algorithm would be able to tell which exact plot point the following rank is:&#xA;A=High B=Very Low C=High D=Very High&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Point       Sample Rankings                     &#xA;1   =   A=Very High B=Very Low C=Very Low D=Very Low                        &#xA;25  =   A=Very Low B=Very High C=Very Low D=Very Low                        &#xA;49  =   A=Very Low B=Very Low C=Very High D=Very Low                        &#xA;73  =   A=Very Low B=Very Low C=Very Low D=Very High                        &#xA;625 =   All Moderate (other combinations, such as All Very High, not possible)                      &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which point 337, 401,457,505? would be created from: A=Very High B=Very High, C=Very Low D= Very High&lt;/p&gt;&#xA;&lt;p&gt;Here is the sample grid. This is a new(er) grid with 25 boxes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/c030A.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Edit: 04/04/15 6:10 PM EDT&#xA;Here is another way to look at this using scores for the diagonal axis instead of rankings.&lt;/p&gt;&#xA;&lt;p&gt;Let's say the scores for the axis are A=90, B=40, C=50 and D=60. How can the average of these for scores be calculated and plotted on this graph?&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/F1whu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the original grid with 15 boxes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TciYk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="8326" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-04-09T19:26:28.870" Title="How to calculate grid coordinates of the centroid based on four attributes in a spider chart" Tags="|plotting|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="19296" PostTypeId="1" AcceptedAnswerId="19328" CreationDate="2015-04-03T19:14:15.143" Score="1" ViewCount="167" Body="&lt;h2&gt;The description:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I have a number of similar piecewise functions, where $d$ is the translation distance lets assume this is the function (where a and b are known constants):&#xA;$$f(x-d) = \left\{ \begin{array}{l}&#xA;a-b(x-d) &amp;amp; \text{if } 0 \le x-d &amp;lt; \frac{a}{b}\\&#xA;a+b(x-d) &amp;amp; \text{if } -\frac{a}{b} &amp;lt; x-d &amp;lt; 0&#xA;\end{array} \right.$$&#xA;&lt;img src=&quot;https://i.stack.imgur.com/JbKij.png&quot; alt=&quot;Example&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when $d$ is small we will have a number of them overlapping ($d$ is always bigger than zero) and as $d$ gets big fewer will overlap until they are are barley touching and that is the maximum $d$ (since we cannot have any area in the middle that is not covered by at least one function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value that I care about is the sum of these functions, so overlapping areas add up (it is like stacking these graphs on top of each other) and my total function is:&#xA;$$f_\text{total}(x) = \sum_{n=0}^{i} f(x-nd)$$&#xA;&lt;em&gt;$i$ is the number of functions in a system&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The problem:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I want to optimize the system by finding the maximum $d$ that still satisfy the condition:&#xA;$$f_\text{total}(x) \ge z$$&#xA;&lt;em&gt;$z$ is defined based on the system requirement (known value)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The question:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;How can I solve this? what type of mathematical methods will help me accomplish this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did it on Matlab by simply running a loop and incrementing $d$ by some small value until the condition is not satisfied, however, to get an accurate result I need to increment $d$ by a very tiny value and the computation will take a very long time.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Here is a Matlab simulation of the function translation as we increase $d$ until it reaches the maximum value $2\frac{a}{b}$, the thick black line is the sum of these function $f_\text{total}(x)$.&#xA;I realized from this that the lowest point in the sum is always at point $x$ where two adjacent nodes intersect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/1P98X.gif&quot; alt=&quot;Example2&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="14968" LastEditorUserId="14968" LastEditDate="2015-04-08T20:33:13.317" LastActivityDate="2015-04-13T19:40:12.023" Title="Maximum translation distance between piecewise functions that satisfy a condition" Tags="|optimization|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19297" PostTypeId="1" AcceptedAnswerId="19301" CreationDate="2015-04-04T02:38:24.633" Score="0" ViewCount="1744" Body="&lt;p&gt;After wasting 3 days with scalapack, I gave up and moved to Armadillo, considering it uses lapack underneath its beatiful and easy interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to calculate the eigen values and eigen vectors of a non-sparse Hermitian matrix. This is very easily done with Armadillo with like 4 lines of code with the function &lt;code&gt;eig_sym()&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Is this function parallelized? How can I parallelize it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, is there any alternative that is not as crazy as lapackand supports parallelization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW: I'm more than happy to implement an algorithm for splitting the matrix into submatrices with Armadillo to get the eigenvalues. Whatever works and doesn't make me spends 2 weeks to get my first set of eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using debian 7 and Armadillo 3 is available in the repos, just in case I have to modify stuff in it (noticed that could be a necessity from reading other posts).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you require any additional information, please ask. Thanks for your help in advance.&lt;/p&gt;&#xA;" OwnerUserId="5410" LastEditorUserId="5410" LastEditDate="2015-04-04T02:47:22.653" LastActivityDate="2015-04-04T20:13:30.367" Title="Armadillo eig_sym() for extracting eigenvalues. Is it parallel at all?" Tags="|linear-algebra|parallel-computing|eigensystem|eigenvalues|" AnswerCount="1" CommentCount="3" ClosedDate="2015-04-06T20:40:36.530" ContentLicense="CC BY-SA 3.0" />
  <row Id="19300" PostTypeId="1" CreationDate="2015-04-04T15:17:23.080" Score="1" ViewCount="451" Body="&lt;p&gt;I have a scatter plot where I need to shade the Ramachandran allowed regions for aminoacids. I know there is a function that plots them for you, but it is not helpful for my case since I need the regions in my plot and this function opens a new figure.&lt;/p&gt;&#xA;" OwnerUserId="15010" LastActivityDate="2015-05-05T22:07:33.647" Title="How can I plot the Ramachandran regions in MatLab?" Tags="|matlab|computational-biology|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19301" PostTypeId="2" ParentId="19297" CreationDate="2015-04-04T20:13:30.367" Score="0" Body="&lt;p&gt;Based on a quick perusal of the Armadillo library docs and experience using it years ago, the quick and dirty answer is that Armadillo will use a thread-parallel BLAS and LAPACK library, if you link them. I think that's the extent to which shared memory parallelism is implemented in Armadillo. For distributed memory parallelism, you're probably better off using &lt;a href=&quot;http://libelemental.org/&quot; rel=&quot;nofollow&quot;&gt;Elemental&lt;/a&gt; or ScaLAPACK. An acquaintance in the quantum chemistry community raves about how Elemental is easier to use than ScaLAPACK; your mileage may vary.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-04T20:13:30.367" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19302" PostTypeId="1" CreationDate="2015-04-04T20:37:10.340" Score="3" ViewCount="338" Body="&lt;p&gt;Consider points $(x,y)$ in a domain $\mathcal{D}$ that is bounded above by a curve $C$ that is multivalued (i.e. I have points defining it that are of the form $(x_n,y_n)$, $n=1,2,..,N$), and looks like, for instance the figure shown below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7Ofgd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The remaining parts of the boundary $\partial \mathcal{D}$ occur at $x=0, x=L$ and $y=-h$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I numerically compute general surface integrals of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\mathcal{D}} f(x,y) \ dx\ dy$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for a function $f(x,y)$. For example, $f(x,y) = x^2+y^2$. &lt;/p&gt;&#xA;" OwnerUserId="8021" LastEditorUserId="1280" LastEditDate="2015-04-04T22:21:41.520" LastActivityDate="2015-04-08T09:33:15.563" Title="Integration over a complicated domain" Tags="|numerical-analysis|quadrature|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19303" PostTypeId="2" ParentId="19295" CreationDate="2015-04-04T22:40:35.990" Score="2" Body="&lt;p&gt;I'm going to suggest something different than your proposal to see what you think: try a &lt;a href=&quot;http://en.wikipedia.org/wiki/Radar_chart&quot; rel=&quot;nofollow&quot;&gt;radar chart&lt;/a&gt;. The main advantage of that chart is that axis values will correspond to the scores you are talking about, so they're good for ordinal measurements, like the type you are describing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A challenge I see with the graphic you are proposing is that you're flattening (projecting) a 5 by 5 by 5 by 5 4-dimensional hypercube with discrete data onto a square. This necessarily means that some weird things are going to happen. For the sake of argument, I'm going to label Very Low through Very High as 1 through 5. Then at point 1, we have $(A=5, B=1, C=1, D=1)$, and at point 25, we have $(A=1, B=5, C=1, D=1)$. There are 25 points between point 1 and point 25 inclusive, which means you can't do something intuitive like assume $C$ and $D$ are constant, have $A$ decrease monotonically from point 1 to point 25 while $B$ increases monotonically from point 1 to point 25, and make sure that the values of $A$ and $B$ are still whole numbers. Instead, you'll need to assign every possible combination of $(A,B)$ values along that side of your square, giving you some counterintuitive visual effect (to me, anyway).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-04T22:40:35.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19304" PostTypeId="2" ParentId="19302" CreationDate="2015-04-05T13:11:03.223" Score="5" Body="&lt;p&gt;You could triangulate the domain (with say a Delaunay method), and then integrate over the triangles using standard integration techniques. This would be a finite element method, but just for integrating a function rather than solving a differential equation.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-04-05T13:11:03.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19305" PostTypeId="2" ParentId="19300" CreationDate="2015-04-05T20:57:28.713" Score="2" Body="&lt;p&gt;Is there a reason you can't just call that function and then plot your scatter plot on the figure it creates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise you could either open up the existing function that plots the Ramachandran regions and copy/modify the relevant parts to do what you want or you could copy the data out of the plotted objects (using the &lt;code&gt;get&lt;/code&gt; function, see &lt;a href=&quot;http://www.mathworks.com/matlabcentral/answers/100687-how-do-i-extract-data-from-matlab-figures&quot; rel=&quot;nofollow&quot;&gt;this link&lt;/a&gt;) and re-plot it however you want.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-04-05T20:57:28.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19306" PostTypeId="1" AcceptedAnswerId="19307" CreationDate="2015-04-05T23:55:59.337" Score="7" ViewCount="398" Body="&lt;p&gt;A sixth order backward differentiation formula (BDF) need six (five plus initial value) previous solutions to get started. How I can get these previous solutions? I need a method accurate to sixth order capable of handling stiff problems.&lt;/p&gt;&#xA;" OwnerUserId="14880" LastActivityDate="2015-04-16T11:52:04.697" Title="Initializing implicit linear multistep methods" Tags="|ode|implicit-methods|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19307" PostTypeId="2" ParentId="19306" CreationDate="2015-04-06T03:11:42.500" Score="8" Body="&lt;p&gt;The standard approach is to use a self-starting time-marching algorithm with sufficiently small timestep (such that the order of accuracy is not spoiled) and compute the 5 non-initial value previous solutions.  These are then used to &quot;start&quot; the BDF formula.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-04-06T03:11:42.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19310" PostTypeId="1" CreationDate="2015-04-06T22:55:30.993" Score="2" ViewCount="97" Body="&lt;p&gt;I'm trying to find the &quot;looped&quot; route with the lowest value of &lt;code&gt;D/n&lt;/code&gt;, where &lt;code&gt;D=Distance&lt;/code&gt;, and &lt;code&gt;n=Number_of_points&lt;/code&gt;, for a cloud of 3D points. However there are a few conditions.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Conditions:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Each point must have one or more points in the route that are greater than $X$ distance away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not every point needs to be passed through.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will try and explain what I mean:&#xA;Imagine a system of delivery routes, whereby you get paid only if the delivery point is greater than $X$ meters away from the pickup point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you travel point $A \rightarrow B$ where the distance is $X$, you get paid, but you have been paid once in $X$. If you have say 10 equidistant points on a circle with circumference $X$. The distance in a line between them is $X/10$. So you are stopping once every $x/10$ to pick up a new package. Now, once you travel half way around the circle to point 5, you can deliver the package from the starting point and get paid. This means once the route has been completed halfway, you are getting paid every $X/10$ instead. Much more efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now here is the kicker, my data is not in a neat circle, but rather a cloud of 3D points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithm or set of algorithms could be used to determine the most efficient route between the number of points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry if I horribly explained my problem, from what I understand the problem might be NP-complete but if an approximate solution can be calculated, that would work.&lt;/p&gt;&#xA;" OwnerUserId="15028" LastEditorUserId="9667" LastEditDate="2015-04-07T07:51:41.007" LastActivityDate="2015-04-07T07:51:41.007" Title="Finding most efficient route (distance/number of nodes) that uses nodes at least X amount away from another node" Tags="|algorithms|approximation|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19311" PostTypeId="1" CreationDate="2015-04-07T06:14:03.973" Score="0" ViewCount="3055" Body="&lt;p&gt;I need to solve an optimization problem with two nonlinear equality constraints. My function evaluation is very fast (less than a second) and I also provide &lt;code&gt;fmincon&lt;/code&gt; with the gradients of my objective and constraint functions. However, when &lt;code&gt;fmincon&lt;/code&gt; is slow when there are about 2000 variables, and it is very slow when there are 6000 variables. Considering my function evaluation is super fast, I guess the bottle neck is the speed of &lt;code&gt;fmincon&lt;/code&gt;. How can I possibly speed up &lt;code&gt;fmincon&lt;/code&gt;? If not, can anyone suggest any other optimization packages?&lt;/p&gt;&#xA;" OwnerUserId="14909" LastEditorUserId="9667" LastEditDate="2015-04-07T14:46:14.317" LastActivityDate="2015-04-07T21:37:14.963" Title="How to speed up fmincon in MATLAB when there are many variables? Alternatives to MATLAB optimization toolbox?" Tags="|optimization|matlab|constrained-optimization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19312" PostTypeId="2" ParentId="19260" CreationDate="2015-04-07T06:39:33.627" Score="1" Body="&lt;p&gt;Regarding normalisation, $\left&amp;lt;l|r\right&amp;gt; = 1$ is the only normalisation required for the matrix to be decomposed correctly as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = R \Lambda L^H,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Lambda$ has the eigenvalues on the main diagonal. This leaves you with one (complex) degree of freedom in the mutual definition of $l$ and $r$, but as long as only a product of the two vectors is used, then these factors will cancel out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convenience you may wish to remove this degree of freedom e.g. by enforcing $\left&amp;lt;r|r\right&amp;gt; = 1$, but in this case you will almost certainly get $\left&amp;lt;l|l\right&amp;gt; \neq 1$.&lt;/p&gt;&#xA;" OwnerUserId="1750" LastActivityDate="2015-04-07T06:39:33.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19313" PostTypeId="2" ParentId="19302" CreationDate="2015-04-07T07:43:03.007" Score="4" Body="&lt;p&gt;Depending on your accuracy requirements and the effort you are willing to put into the method, you can do different things, especially since you already have an implicit description of your surface. Here's the idea: Using the example of the integral over a circle, you can define a function&#xA;$$\varphi(x,y) = 1 - (x^2 + y^2)$$&#xA;that is positive inside of your domain of integration (in this case, a circle with radius 1) and negative anywhere else. $\varphi(x,y)$ is then called a &lt;em&gt;level set function&lt;/em&gt; and there's tons of papers dealing with the integration over the enclosed area/volume $(\varphi(x,y) &amp;gt; 0)$ and over the corresponding domain boundary $(\varphi(x,y) = 0)$. The corresponding integrals can simply be rewritten as&#xA;$$\int_D H(\varphi(x,y)) \,dV$$&#xA;and&#xA;$$\int_D \delta(\varphi(x,y)) \,dV,$$&#xA;respectively, where $H$ is the Heaviside function and $\delta$ is the Dirac delta distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These integrals cannot be computed directly, but they can be evaluated using different methods which strongly differ in terms of complexity and accuracy. One approach has already been described by Bill, but, if your accuracy requirements are relatively low, you can get away with a few very simple options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use simple adaptive quadrature techniques (e.g., such the ones implemented in Matlab)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Replace $H$ and $\delta$ with smoothed approximations (see &lt;a href=&quot;http://en.wikipedia.org/wiki/Heaviside_step_function&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt; for a few &lt;a href=&quot;http://en.wikipedia.org/wiki/Dirac_delta_function&quot; rel=&quot;nofollow&quot;&gt;examples&lt;/a&gt;) and evaluate them using classical Gauss quadrature&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Monte_Carlo_integration&quot; rel=&quot;nofollow&quot;&gt;Monte-Carlo&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If your accuracy requirements are high, you will have to invest a little more into the topic, since the above-mentioned approaches tend to become exceedingly inefficient when you increase the resolution (they just converge very slowly). In this case, it's not easy giving a simple answer without knowing a little more about your problem.&lt;/p&gt;&#xA;" OwnerUserId="3738" LastActivityDate="2015-04-07T07:43:03.007" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19315" PostTypeId="2" ParentId="19302" CreationDate="2015-04-07T11:38:32.497" Score="5" Body="&lt;p&gt;To give you a better picture, what I mean with my comment, consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Stokes Theorem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_D db^{n-1}=\int_{\partial D}b^{n-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;be $b^{n-1}$ an arbitrary 1-form&#xA;$$b^{1}= a_1 dx + a_2 dy$$&#xA;leads to&#xA;$$\int_D f(x,y)dxdy = \int_D\left(\frac{\partial a_2}{\partial x}-\frac{\partial a_1}{\partial y}\right)dxdy$$&#xA;Without loss of generality, $a_1=0$ one needs to solve $f(x,y)=\frac{\partial a_2}{\partial x}$ for $a_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you need the parametrization for the boundary functions. For the first three bounding functions $$(x=0,y\in[-h,\eta(0)]),(x\in[0,L],y=-h),(x=L,y\in[-h,\eta(L)])$$ this is straight forward. For the bounding function I assume you know or can evaluate the function in $(x(t),y(t)) \wedge t\in[0,1]$. So I suggest using two Gaussian quadratures. First from $[lb,peak]$ and second from $[peak, rb]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given you can obtain information on the peak and integrate $f$, this method will give you very high accuracy and very good convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Supplementing Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As bigge pointed out, knowing $f$ does not guarantee to find an parametrization easily. Of course this is true. But even if one is not able to solve $f(x,y)=\frac{\partial a_2}{\partial x}$ analytically, one can still use this approach. Assuming Gaussian quadrature techniques one can read it as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_2(x,y)=\int_{x_0}^x f(t,y)dt =\int_{-1}^1 f(r,y) \frac{dt}{dr} dr\approx \sum_k w_k f(r_k,y) \frac{dt}{dr}$$ With e.g. $x_0=0$ chosen arbitrarily. Now the scaling factor $\frac{dt}{dr}$ depends on the integration range $x$. Since we still integrate line integrals along the boundary, $x$ is not arbitrary but specific set of quadrature points for evaluating $\int_{\partial D} a_2(x,y) dy$.&lt;br&gt;&#xA;So the final term is a double sum for each boundary integral as expected for an integral over a 2-D domain.&#xA;In that form it is simply a very sophisticated 2D Simpson's rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, it is usually more challenging to find a good parametrization of the bounding curves than evaluation of $f$.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2015-04-08T09:33:15.563" LastActivityDate="2015-04-08T09:33:15.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19317" PostTypeId="2" ParentId="19311" CreationDate="2015-04-07T21:37:14.963" Score="4" Body="&lt;p&gt;Without knowing more about your problem, it's difficult to make specific recommendations. I've made some general recommendations about nonlinear programming solvers in &lt;a href=&quot;https://scicomp.stackexchange.com/a/544/276&quot;&gt;this question&lt;/a&gt;. To summarize:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;IPOPT is a good interior point method (IPM) solver&lt;/li&gt;&#xA;&lt;li&gt;SNOPT is a good sequential quadratic programming (SQP) solver&lt;/li&gt;&#xA;&lt;li&gt;CONOPT is a pretty good generalized reduced gradient (GRG) solver&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In your case, if there are only two active constraints (not counting bound constraints), I'd think that a GRG solver might do reasonably well, and that SQP wouldn't be that helpful, but for a problem with 6000 variables, all of these methods are viable. These are all convex solvers, and since your problem has nonlinear equality constraints, it is likely that your problem is nonconvex, in which case these solvers will return locally optimal solutions. However, &lt;code&gt;fmincon&lt;/code&gt; will also return a locally optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a globally optimal solution, you should instead look at global optimization solvers such as BARON, Couenne, Bonmin, and DICOPT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have access, consider using a modeling environment such as GAMS or CVX, which will give you the option of posing your problem once, and selecting from many different solvers, to see which one works best for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From grepping around the MATLAB source code, it seems likely that &lt;code&gt;fmincon&lt;/code&gt; is mostly implemented in pure MATLAB, which would explain why it is slow. It would be better to use a solver that implements the core mathematical algorithms in the solver in a compiled language, which would be (significantly) faster.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-04-07T21:37:14.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19318" PostTypeId="1" CreationDate="2015-04-07T23:32:58.107" Score="1" ViewCount="70" Body="&lt;p&gt;What is the most computationally efficient way to find the layer on which a ball (i) belongs when arranged in a tetrahedron or 3 dimensional triangle with a triangular base. The ball on the top layer is numbered one. The balls on the second layer are numbered 2 - 4. The third layer 5-10 and so on. &lt;/p&gt;&#xA;" OwnerUserId="15037" LastEditorUserId="7733" LastEditDate="2015-04-09T13:10:34.703" LastActivityDate="2015-07-08T14:16:40.863" Title="Layer on which ball belongs in tetrahedron" Tags="|geometry|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19319" PostTypeId="1" CreationDate="2015-04-08T12:16:19.137" Score="-1" ViewCount="197" Body="&lt;p&gt;I am using 8 ODEs in Matlab to simulate the effect of asymptomatic infections in the epidemiology of a vector borne disease. Searching the parameter space under certain settings produces negative numbers for the human population and the following warning in the command consol:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Warning: Failure at t=3.562559e+03. Unable to meet integration tolerances without reducing the step size below the smallest value allowed (7.275958e-12) at time t. &gt; In ode45 at 308 In Mosquito_Framework_2_human_plots at 79&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some setting I could put the solver on to get rid of the problem. The current solver is ode45 under 'RelTol', 1e-6 option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following code produces the negative human population and warning message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [t,Sh,Ah,Ih,Rh,Se,Sv,Iv] = Mosquito_Framework_2_human_plots(betaI,betaV,c,...&#xA;theta,d,muh,p,muv,omega,epsilon,Sh0,Ah0,Ih0,Rh0,Sv0,Ev0,Iv0,MaxTime)&#xA;&#xA;% Sets up default parameters if necessary.&#xA;if nargin ~= 18&#xA;    disp('defaults')&#xA;    %displays message default to say simulation is using defaults if not&#xA;    %enouth argumants is given&#xA;   betaI = (0.3+1)/2*(0.1+0.75)/2;&#xA;   betaV = (0.3+1)/2*(0.5+1)/2;&#xA;   c=1;&#xA;   theta=8;&#xA;   d=0.25;&#xA;   muh=1/(60*365.0);&#xA;   p=0.9;&#xA;   muv=1/6;&#xA;   omega=1/11;&#xA;   epsilon=1/((8+12)/2);&#xA;   Sh0=(1e6)-1;&#xA;   Ah0=1;&#xA;   Ih0=0;&#xA;   Rh0=0;&#xA;   Sv0=9.5e6;&#xA;   Ev0=0;&#xA;   Iv0=0;&#xA;   MaxTime=10*(365);&#xA;end&#xA;&#xA;gammaI=1/theta;&#xA;gammaA=1/(d*theta);&#xA;betaA=c*betaI;&#xA;Nh0=Sh0+Ah0+Ih0+Rh0;&#xA;Nv0=Sv0+Ev0+Iv0; &#xA;Se0=Nv0*(muv/omega);&#xA;&#xA; % Checks all the parameters are valid&#xA; if Sh0&amp;lt;=0 &#xA;     error('Initial level of susceptibles (%g) is less than or equal to  zero',Sh0);&#xA; end&#xA;&#xA;if Ah0+Ih0&amp;lt;=0 &#xA;     error('Initial level of infecteds (%g) is less than or equal to zero',Ah0+Ih0);&#xA;end&#xA;&#xA;if betaA+betaI&amp;lt;=0  &#xA;    error('Transmission rate betaA+betaI (%g) is less than or equal to zero',betaA+beta);&#xA;end&#xA;&#xA;if betaV&amp;lt;=0  &#xA;    error('Transmission rate betaV (%g) is less than or equal to zero',betaA+beta);&#xA;end&#xA;&#xA;if gammaA+gammaI&amp;lt;=0 &#xA;     error('Recovery rate gammaA+gammaI (%g) is less than or equal to zero',gammaA+gammaI);&#xA;end&#xA;&#xA;if muh&amp;lt;=0 &#xA;    error('Death and birth rate gamma (%g) is less than or equal to zero',muh);&#xA;end&#xA;&#xA;if MaxTime&amp;lt;=0 &#xA;    error('Maximum run time (%g) is less than or equal to zero',MaxTime);&#xA;end&#xA;&#xA;if Sh0+Ah0+Ih0&amp;gt;Nh0&#xA;warning('Initial level of susceptibles+infecteds (%g+%g=%g) is greater than human population size (%g)'...&#xA;,Sh0,Ah0,Ih0,Sh0+Ah0+Ih0,Nh0);&#xA;end&#xA;&#xA;if Sv0+Ev0+Iv0&amp;gt;Nv0&#xA;warning('Initial level of susceptibles+infecteds (%g+%g=%g) is greater than mosquito population size (%g)'...&#xA;,Sv0,Ev0,Iv0,Sv0+Ev0+Iv0,Nv0);&#xA;end&#xA;&#xA;Sh=Sh0; Ah=Ah0; Ih=Ih0; Rh=Rh0; Se=Se0; Sv=Sv0; Ev=Ev0; Iv=Iv0;&#xA;&#xA;% The main iteration &#xA;options = odeset('RelTol', 1e-6);&#xA;[t, pop]=ode45(@Diff_Framework_1,[0 MaxTime],[Sh Ah Ih Rh Se Sv Ev Iv],options,...&#xA;    [betaA betaI betaV gammaA gammaI muh p muv omega epsilon]);&#xA;&#xA;Sh=pop(:,1); Ah=pop(:,2); Ih=pop(:,3); Rh=pop(:,4); Se=pop(:,5); Sv=pop(:,6); Ev=pop(:,7); Iv=pop(:,8);&#xA;&#xA;%plot the graphs with scaled colours&#xA;plot(t,Ah,'Color',[250/255,228/255,32/255],'LineWidth',1.4);&#xA;hold on&#xA;%hold on makes matlab plot to the same plot.&#xA;plot(t,Ih,'-r','LineWidth',1)&#xA;set(gca, 'FontSize', 14)&#xA;xlabel('Time in Days','FontSize',18);&#xA;ylabel('Human Population','FontSize',18);&#xA;% ylim([0,1e5])&#xA;name = strcat('Vector_dep_M2@_c=',num2str(c),'_d=',num2str(d),'_p=',num2str(p));&#xA;Human_pop_plot_name = strcat('Human_pop_',name,'.fig');&#xA;savefig(Human_pop_plot_name);&#xA;hold off&#xA;%hold off stops matlab ploting to the same plot.&#xA;%Saves graph&#xA;%creates string based around parameters used called dataset_name &#xA;&#xA;Both_pops_csv_names = strcat('Both_pops_',name,'.csv');&#xA;&#xA;Both_pops_csv = dataset({t,'Time_in_Days'},{Sh+Ah+Ih+Rh,'Total_Human_Population'},...&#xA;{Sh,'Susceptibles_Humans'},{Ah,'Asymptomatic_Humans'},{Ih,'Symptomatic_Humans'},...&#xA;{Rh,'Recovered_Humans'},{Se+Sv+Ev+Iv,'Total_Mosquito_Population'},...&#xA;{Sv+Ev+Iv,'Total_Adult_Mosquito_Population'},{Se,'Pre_Adult_Mosquito_Population'},...&#xA;    {Sv,'Susceptible_Mosquito_Population'},     {Ev,'Incubating_Mosquito_Population'},...&#xA;    {Iv,'Infectious_Mosquito_Population'});&#xA;%&#xA;export(Both_pops_csv,'File',Both_pops_csv_names,'Delimiter',',')&#xA;%creats csv file of outputs&#xA;&#xA;&#xA;&#xA;% Calculates the differential rates used in the integration.&#xA;function dPop=Diff_Framework_1(t,pop, parameter)&#xA;&#xA;betaA=parameter(1); betaI=parameter(2); betaV=parameter(3); gammaA=parameter(4);... &#xA;    gammaI=parameter(5); muh=parameter(6); p=parameter(7); muv=parameter(8); ...&#xA;    omega=parameter(9); epsilon=parameter(10);&#xA;&#xA;Sh=pop(1); Ah=pop(2); Ih=pop(3); Rh=pop(4); Nh=Sh+Ah+Ih+Rh; ...&#xA;    Se=pop(5); Sv=pop(6); Ev=pop(7); Iv=pop(8); Nv=Sv+Ev+Iv; &#xA;&#xA;dPop=zeros(8,1);&#xA;&#xA;dPop(1)= muh*Nh - Sh/Nh*(betaV*Iv) - muh*Sh;&#xA;dPop(2)= (1-p)*Sh/Nh*(betaV*Iv)-(gammaA+muh)*Ah;&#xA;dPop(3)= p*Sh/Nh*(betaV*Iv) - (gammaI+muh)*Ih;&#xA;dPop(4)= gammaA*Ah+gammaI*Ih - muh*Rh;&#xA;dPop(5)= muv*Nv - omega*Se;&#xA;dPop(6)= omega*Se - (Sv*betaA*Ah/Nh + Sv*betaI*Ih/Nh) - muv*Sv;&#xA;dPop(7)= (Sv*betaA*Ah/Nh + Sv*betaI*Ih/Nh) - epsilon*Ev - muv*Ev;&#xA;dPop(8)= epsilon*Ev - muv*Iv;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15040" LastActivityDate="2015-04-08T13:08:10.200" Title="How do I stop negative numbers and error message: &quot; Failure at t=3.562559e+03. Unable to meet integration tolerances" Tags="|matlab|ode|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19320" PostTypeId="2" ParentId="19319" CreationDate="2015-04-08T13:08:10.200" Score="1" Body="&lt;p&gt;I suggest plotting the eight solutions up to the point where&#xA;&lt;code&gt;ode45&lt;/code&gt; can no longer obtain a converged solution (t=3562).&#xA;In my experience, the error message you site is usually caused by&#xA;one of the solutions going to positive or negative infinity.&#xA;Looking at the behavior of the solutions prior to convergence&#xA;failure may help you identify a problem with the equations&#xA;themselves or an error in coding them with MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A less common problem is the value of a solution becoming large&#xA;but not infinite and the ode solver not able to reach the required&#xA;convergence tolerance. In this case, you can make &lt;code&gt;AbsTol&lt;/code&gt; and/or &lt;code&gt;RelTol&lt;/code&gt; larger (say one or two orders of magnitude) and obtain a converged solution.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-04-08T13:08:10.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19321" PostTypeId="2" ParentId="2267" CreationDate="2015-04-08T16:00:16.240" Score="1" Body="&lt;p&gt;I also started from the basic and have capture all my experiences, problems etc in the two links:&#xA;&lt;a href=&quot;http://www.cfdyna.com/Home/OpenFOAM.html&quot; rel=&quot;nofollow&quot;&gt;http://www.cfdyna.com/Home/OpenFOAM.html&lt;/a&gt;&#xA;&amp;amp;&#xA;&lt;a href=&quot;http://www.cfdyna.com/Home/OpenFOAM_Tut.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cfdyna.com/Home/OpenFOAM_Tut.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps! Please let me know your comments and I will try to make it better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&lt;/p&gt;&#xA;" OwnerUserId="15042" LastActivityDate="2015-04-08T16:00:16.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19322" PostTypeId="1" CreationDate="2015-04-08T16:47:45.733" Score="0" ViewCount="93" Body="&lt;p&gt;I managed to solved a lid-driven cavity flow using LB code. It gave me the velocity field data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have to obtain streamlines too, of course from the obtained velocity field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides, I know the theory of Potential Flow as well as all of the associated relations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I directly draw the streamlines of this flow without solving a Poisson equation. If numerical integration must to be used (as I guess), How should I do handle with partial derivatives?&lt;/p&gt;&#xA;" OwnerUserId="15043" LastActivityDate="2015-04-08T17:40:15.647" Title="How to obtain streamslines from velocity field data points" Tags="|finite-difference|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19323" PostTypeId="2" ParentId="19322" CreationDate="2015-04-08T16:55:24.180" Score="1" Body="&lt;p&gt;Probably the easiest thing to do is to use a visualization package or something with built-in vis capabilities. &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/streamline.html?refresh=true&quot; rel=&quot;nofollow&quot;&gt;MATLAB&lt;/a&gt;, &lt;a href=&quot;https://www.scorec.rpi.edu/wiki/Using_ParaView#Stream_Tracer&quot; rel=&quot;nofollow&quot;&gt;ParaView&lt;/a&gt;, and &lt;a href=&quot;http://www.visitusers.org/index.php?title=Streamline_Plots_in_VisIt&quot; rel=&quot;nofollow&quot;&gt;VisIt&lt;/a&gt; all support plotting streamlines natively, and the latter two are freely available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to plot them yourself, they are usually integrated using a psuedo-time integration method from seed points such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dx(s)}{ds} = u(x(s),t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $t$ fixed and some initial point $x(0)=x_0$. If the flow is steady, then you can drop the $t$. &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-04-08T16:59:04.583" LastActivityDate="2015-04-08T16:59:04.583" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19324" PostTypeId="2" ParentId="19322" CreationDate="2015-04-08T17:40:15.647" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;http://www.cgal.org/&quot; rel=&quot;nofollow&quot;&gt;CGAL library&lt;/a&gt; has a package for this purpose.  See &lt;a href=&quot;http://doc.cgal.org/latest/Stream_lines_2/index.html#Chapter_2D_Placement_of_Streamlines&quot; rel=&quot;nofollow&quot;&gt;their documentation for details and references&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-04-08T17:40:15.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19326" PostTypeId="1" CreationDate="2015-04-08T18:54:44.587" Score="0" ViewCount="997" Body="&lt;p&gt;I am interested in solving the following PDE (heat equation):&#xA;$$\frac{\partial u}{\partial t} = \kappa \frac{\partial ^2 u}{\partial x^2}$$&#xA;In order to solve it, I discretize space uniformly into $N$ segments and convert the PDE into $N+1$ ODEs:&#xA;$$ \frac{du_k}{dt} = \kappa\frac{u_{k+1} + u_{k-1} - 2u_k}{(\Delta x)^2} \quad k=0,1\ldots.N$$&#xA;which I can solve using any ODE routine (such as ode45 or ode15s in MATLAB).&#xA;For the particular case of my problem, the Neumann boundary conditions for the PDE are dependent on the values of $u_k$ itself, i.e &#xA;$$ \frac{du_b}{dx}= \left\{&#xA;\begin{array}{cc}&#xA;\alpha &amp;amp; u_b &amp;lt; u_{critical}, \\&#xA;-\gamma u_b &amp;amp; u_b \geq u_{critical}&#xA;\end{array} \right. $$&#xA;where $\alpha$ and $\gamma$ are constants and $b=\{0,N\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that I am solving the above problem using some numerical ODE solver (such as ode45 in MATLAB), and there are two routines: &lt;em&gt;odefun&lt;/em&gt; (which I provide to the solver) to evaluate the derivatives and &lt;em&gt;odestep&lt;/em&gt; which I define and the solver calls it after every successful integration step, where should I check if the boundary condition needs to be changed, in &lt;em&gt;odefun&lt;/em&gt; or &lt;em&gt;odestep&lt;/em&gt;? I am asking this because the &lt;em&gt;odefun&lt;/em&gt; routine might be used internally by the solver for evaluation of jacobian, errors etc and ideally, we should not change boundary conditions in the middle of an integration step, right?&lt;/p&gt;&#xA;" OwnerUserId="15045" LastActivityDate="2015-06-19T16:21:39.673" Title="Solving PDE with state and time dependent boundary conditions" Tags="|matlab|pde|finite-difference|boundary-conditions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19327" PostTypeId="1" CreationDate="2015-04-08T19:14:01.657" Score="1" ViewCount="471" Body="&lt;p&gt;I need to describe the optical aberrations of my set-up with Zernike polynomials. To do this, I want to fit the first 15 polynomials in the following way on my data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\chi^2 = \sum_k\left(\beta_k^x - \sum_{n=2}^M C_n\frac{\partial P_n}{\partial x}\right)^2 + \sum_k\left(\beta_k^y - \sum_{n=2}^M C_n\frac{\partial P_n}{\partial y}\right)^2$$&#xA;where $\beta_x$ and $\beta_y$ are my data, $P_n$ are the Zernike polynomials, and $C_n$ are the coefficients. Because the coefficients $C_n$ for both the $x$ and $y$ derivative of the polynomial needs to be the same, I do not think I can use a normal polynomial least square. I need to do this in Labwindows/CVI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have an idea how to approach this problem? &lt;/p&gt;&#xA;" OwnerUserId="15046" LastEditorUserId="9667" LastEditDate="2015-04-10T06:43:04.133" LastActivityDate="2015-04-10T08:39:34.737" Title="Least Square fit with Double Zernike Polynomials" Tags="|least-squares|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19328" PostTypeId="2" ParentId="19296" CreationDate="2015-04-08T20:45:21.553" Score="5" Body="&lt;p&gt;As a general rule in developing and testing numerical software, I try to eliminate as many parameters as possible.  I'm sure it has occurred to you that the distance $d$ and &quot;floor&quot; $z$ are scalable for specified values of $a$ and $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular the maximum of the reference &quot;hat&quot; function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) = a \cdot \left(1 - \frac{|x|}{a/b} \right) \;\; \text{ on } \;\;(-a/b,+a/b) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;occurs at $x=0$, where $f(0) = a$.  So we can factor $a$ out of each hat function, and modify the floor $z$ to be instead $z/a$.  Effectively we will take $a=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly the horizontal scale of distances is relative to the size of the interval $(-a/b,+a/b)$, so that if you also set $b=1$ we can treat $d$ as a multiple of the interval half-length $a/b = 1$.  Knowing the solution $d$ for given $z$ with $a=b=1$ allows us to give the solutions for any $a,b,z$ with a simple scaling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make all this concrete, define the special case of the original problem where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(x) = 1 - |x| \;\; \text{ on } \;\; (-1,+1) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and otherwise $F(x) = 0$.  We reduce the original problem to the special case by taking $Z = z/a$, and then asking for the maximum distance $D$ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F_{\text{total}}(x) \equiv \sum_{k\in \mathbb{Z}} F(x - kD) \ge Z $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then an original problem with arbitrary $a,b,z \gt 0$ would have a solution $d = (a/b)D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve the special case, start with $D=2$. The sum of our functions $F_{\text{total}}(x)=\sum_k F(x-kD)$ is a simple sawtooth wave with a peak (maximum) value $1$ at every even integer and a floor (minimum) value zero at every odd integer, i.e. periodic of length $D=2$.  Thus $D=2$ would be our maximum only if $Z \le 0$, which is probably not interesting.  We therefore consider how to reduce the peak-to-peak distance $D$ until everywhere $F_{\text{total}} \ge Z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we cut $D$ by half, so $D=1$, the effect is to make $F_{\text{total}}$ a constant function, $F_{\text{total}}(x)=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Repeatedly &quot;halving&quot; $D$ exactly, e.g. $D = 2^{-n}$, will again produce constant functions (something that can be glimpsed in your animated GIF).&#xA;For $D = 2^{-n}$, $F_{\text{total}}(x) = 2^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore we can quickly &quot;bracket&quot; the maximum value of $D$ by finding nonnegative integer $n$ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 2^n \le Z \lt 2^{n+1} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This cuts down considerably on the effort required to find the maximum distance $D$.  But more can be said, so that eventually a fairly explicit expression for $D$ in terms of $Z$ can be given.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Despite the infinite number of summands appearing in the definition, $F_{\text{total}}(x)$ gets&#xA;nonzero contributions at any particular real $x$ from only finitely many summands.  The regular&#xA;spacing $D$ between hat functions $F(x-kD)$ allows us to bound the number whose supports contain&#xA;a common point.  If $|kD| \gt 2$, then $F(x)$ and $F(x-kD)$ have supports with empty intersection,&#xA;so a particular point is in the support of at most $\lfloor 2/D \rfloor$ hat functions.  From this&#xA;&quot;local&quot; finiteness it follows that $F_{\text{total}}$ is continuous and piecewise linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The regular spacing $D$ makes $F_{\text{total}}$ periodic with period $D$.  Since the reference&#xA;hat function $F(x)$ is &lt;em&gt;even&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(x) = F(-x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F_{\text{total}}$ is also even and symmetric about $x = D/2$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F_{\text{total}}(x + D/2) = F_{\text{total}}(-x + D/2) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By its periodicity it suffices to consider the restriction of $F_{\text{total}}$ to $[0,D]$, and&#xA;its minimum there (resp. its maximum there) will be the global minimum (resp. global maximum).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed because peaks of our hat functions occur only to the left or to the right of $[0,D]$, the&#xA;summands' restrictions to it are convex (concave up).  It follows that $F_{\text{total}}$ is convex&#xA;on $[0,D]$.  Thus by the symmetry already noted, the minimum occurs at the midpoint $x = D/2$ while&#xA;the maximum occurs at endpoints $x=0,D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The supports of hat functions containing the minimizing argument $x = D/2$ lie symmetrically to its&#xA;left and right.  On the left $D/2$ is in the support of $F(x+kD)$ for $k \ge 0$ if and only if:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{D}{2} + kD \le 1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ k \le \frac{1}{D} - \frac{1}{2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It follows that to the left of $D/2$ we have $\lfloor D^{-1} + 1/2 \rfloor$ contributing summands:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$k = 0,\ldots,\lfloor D^{-1} - 1/2 \rfloor $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and an equal number of equal contributions to the right.  Thus:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align*}&#xA;    F_{\text{total}}(D/2) &amp;amp;= 2 \sum_{k=0}^{\lfloor D^{-1} - 1/2 \rfloor} F(kD + D/2) \\&#xA;	                      &amp;amp;= 2 \sum_{k=0}^{\lfloor D^{-1} - 1/2 \rfloor} (1 - (k+1/2)D)&#xA;\end{align*}	$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gauss's trick of summing an arithmetic sequence applies, absorbing the factor $2$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align*}&#xA;    F_{\text{total}}(D/2) &amp;amp;= \lfloor D^{-1} + 1/2 \rfloor \cdot \left[\left(1 - \frac{D}{2}\right)&#xA;    + \left(1 - \left(\lfloor D^{-1} - 1/2 \rfloor + 1/2 \right) D \right) \right] \\&#xA;	                      &amp;amp;= \lfloor D^{-1} + 1/2 \rfloor \cdot [2 - \lfloor D^{-1} + 1/2 \rfloor D]&#xA;\end{align*}	$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite the evident jump in $\lfloor D^{-1} + 1/2 \rfloor$ when $D^{-1}$ is an integer plus one-half, the&#xA;above expression for $F_{\text{total}}(D/2)$ is continuous there and elsewhere.  Confirmation of&#xA;this is left as &quot;an exercise for the Reader&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that when $D^{-1} = n$, a positive integer, $\lfloor D^{-1} + 1/2 \rfloor = n$ and:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F_{\text{total}}(D/2) = n \cdot [2 - 1] = n $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, when $D^{-1} + 1/2 = n$ is a positive integer, then $D = (n - 1/2)^{-1}$ and:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F_{\text{total}}(D/2) = n \left[2 - \frac{n}{n - 1/2} \right] = n \cdot \frac{n-1}{n - 1/2} \lt n $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So $F_{\text{total}}(D/2)$ is strictly increasing as $D$ decreases, and when $D \in [(n-1/2)^{-1},&#xA;(n+1/2)^{-1}]$, we have a linear dependence of $F_{\text{total}}(D/2)$ on $D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining the maximum $D$ for which everywhere $F_{\text{total}}(x) \ge Z$ amounts to solving for $D$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F_{\text{total}}(D/2) = Z $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $n = \lfloor Z \rfloor$. Then $(n+1)^{-1} \lt D \le n^{-1}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It remains only to compute $F_{\text{total}}(D/2)$ at $D = (n + 1/2)^{-1}$ to learn whether $Z = F_{\text{total}}(D/2)$ is attained for $D \in ((n+1)^{-1},(n + 1/2)^{-1}]$ or $D \in ((n+1/2)^{-1}, n^{-1}]$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exact answer may be found, whichever subinterval is correct, by linear interpolation on $D$, because in either case we have reduced the term $\lfloor D^{-1} + 1/2 \rfloor$ to a constant.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2015-04-13T19:40:12.023" LastActivityDate="2015-04-13T19:40:12.023" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19329" PostTypeId="2" ParentId="19318" CreationDate="2015-04-08T23:26:25.323" Score="1" Body="&lt;p&gt;Rootfinding is probably the most computationally efficient way to find the layer in which a ball $i$ resides in a close-packed tetrahedral arrangement. Assume the balls are labeled consecutively such that balls increase as the layer number increases, so that layer 1 corresponding to the topmost ball (i.e., ball 1), layer 2 is beneath layer 1 (balls 2-4), layer 3 is beneath layer 2 (balls 5-10), and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the number of objects in an $n$-layer tetrahedron is $T_{n} = n(n+1)(n+2)/6$, if $m$ satisfies $T_{n-1} &amp;lt; m \leq T_{n}$, ball $m$ is in layer $n$, with the convention that $T_{0} = 0$. This observation suggests the following procedure: solve $n(n+1)(n+2)/6 = m$ for $n$. Then $\lceil{n}\rceil$ (the ceiling of $n$) will be the layer number containing ball $m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In certain applications, you might instead want to store a sequence of tetrahedral numbers (according to the formula above) and then perform a table lookup instead of using rootfinding.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-08T23:26:25.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19330" PostTypeId="1" CreationDate="2015-04-09T03:06:41.930" Score="1" ViewCount="863" Body="&lt;h2&gt;Update 4&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I have almost given up on getting this right. This is the solution to the time-independent Schrodinger's equation, so the analytical solution is:&#xA;$\psi(x,t) = \psi(x,0)e^{\frac{-iE t}{\hbar}}$. That means $|\psi(x,t)|^2 = |\psi(x,0)|^2$ for all t. However, I cannot make that work numerically, and I do not know why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My latest failed attempt:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import *&#xA;from scitools.all import *&#xA;&#xA;V0= 10.0 #eV&#xA;a= 1.0 #nm&#xA;c = 299792.0 # nm per ps&#xA;m= (0.510*10**6)/(c**2) #eV/c**2&#xA;hbar= 6.58*10**(-4) #eV ps&#xA;&#xA;z = 1.456 # fra c&#xA;alpha = 16.2 # fra c&#xA;&#xA;def Psi0(x):&#xA;    psi = x.astype(complex128)&#xA;    ka = (z/a)*sqrt((alpha/z)**2 -1.)&#xA;    A = exp(ka*a)*cos(z)/(a + 1./ka)&#xA;    B = 1./(a+ 1./ka)&#xA;    for i in range(len(x)):&#xA;        if x[i]&amp;lt;-a or x[i]&amp;gt;a:&#xA;            psi[i] = A*exp(-ka*abs(x[i]))&#xA;        else:&#xA;            psi[i] = B*cos((z/a)*x[i])&#xA;    return psi&#xA;&#xA;&#xA;x = linspace(-8.0,8.0,1001).astype(complex128)&#xA;t = linspace(0.0,0.1,2001).astype(complex128)&#xA;&#xA;dx = x[2]-x[1]&#xA;dt = t[2]-t[1]&#xA;k1 = 1.j*hbar/(2.0*m)&#xA;k2 = 1.j/hbar&#xA;&#xA;def V(x,a=1.0):&#xA;    V = zeros(len(x)).astype(complex128)&#xA;    for i in range(len(x)):&#xA;        if x[i] &amp;gt;= -a and x[i] &amp;lt;= a:&#xA;            V[i] = -V0&#xA;    return V&#xA;&#xA;def d2(Psi, dx):&#xA;    D2P = zeros(len(Psi)).astype(complex128)&#xA;    for i in range(len(D2P)-2):&#xA;        D2P[i+1] = (Psi[i+2] - 2*Psi[i+1] + Psi[i])/(dx**2)&#xA;    return D2P&#xA;&#xA;def sch(Psi,x,dx):&#xA;    return k1*d2(Psi,dx) - k2*V(x)*Psi&#xA;&#xA;Psi = []&#xA;Psi.append(Psi0(x).astype(complex128))&#xA;&#xA;a = abs(Psi[0])**2&#xA;plot(x.real, a, xlabel='x', ylabel='f')&#xA;&#xA;''' Euler '''&#xA;'''for i in range(len(t)):&#xA;    b = dx*sum(abs(Psi[-1] + dt*sch(Psi[-1],x,dx))**2)&#xA;    Psi.append((Psi[-1] + dt*sch(Psi[-1],x,dx))/sqrt(b))&#xA;    print b&#xA;'''&#xA;'midpoint'&#xA;for i in range(len(t)):&#xA;    a = Psi[-1] + dt*sch(Psi[-1] + (dt/2.)*sch(Psi[-1],x,dx),x,dx)&#xA;    b = dx*sum(abs(a)**2)&#xA;    Psi.append(a/sqrt(b))&#xA;    print b&#xA;&#xA;counter = 0&#xA;forhver = 10&#xA;&#xA;for i in range(len(t)):&#xA;    if counter == forhver:&#xA;        a = abs(Psi[i])**2&#xA;        plot(x.real, a, axis=[-8, 2, -1, 3], xlabel='x', ylabel='f', legend='t=%4.2f' % i, savefig='obl8%04d.png' % i)&#xA;        counter = 0&#xA;    counter += 1&#xA;&#xA;movie('obl8*.png')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Update 3:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I changed the units from meter per second to nanometers per picoseconds. It made the code work well. However, it just does not look right:&#xA;&lt;a href=&quot;http://sineofmadness.com/wp-content/uploads/2014/11/movie.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://sineofmadness.com/wp-content/uploads/2014/11/movie.gif&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No matter which $\psi(x,0)$ I choose, and I can choose between five, it turns into that double bellcurve. I have used the midpoint method, and Euler's method. It gives more or less the same result. The animation above screams numerical error to me. Yet, I have now seen that result with several different methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this look right to anybody?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Original&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am struggling with a specific type of problem in quantum physics (one dimensional):&#xA;Let's say I have a function $\psi(x,0)$, and I have to show how it evolves over time using the Schrodinger equation (by animating $|\psi(x,t)|^2$):&#xA;$$ -\frac{\hbar^2}{2m}\frac{d^2\psi}{dx^2} + V(x)\psi = i \hbar\frac{d\psi}{dt}$$&#xA;So, what I did was to define functions for the potential and the second derivative, and use Euler's method. At first, the plot function kept throwing away the imaginary parts of the function, so I decided to store the values for $\psi(x,t)$ in a separate array. However, it still does not work. I get problems like every element in the arrays being Not a Number, or the functions remains constant(i.e. $\psi(x,t) = \psi(x,0)$ for all t after some $t_0$).&lt;/p&gt;&#xA;" OwnerUserId="15047" LastEditorUserId="15047" LastEditDate="2015-04-13T18:17:14.273" LastActivityDate="2015-04-13T18:17:14.273" Title="Python, numpy and complex functions (PDE's)" Tags="|pde|python|numpy|quantum-mechanics|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19331" PostTypeId="1" CreationDate="2015-04-09T14:52:55.420" Score="2" ViewCount="121" Body="&lt;p&gt;I am looking for explanations of algorithms to adaptively sample a function of two variables $f(x,y)$, in a given domain $x_0\le x \le x_1$, $y_0\le y \le y_1$. Intuitively, I want to sample more densely regions were $f(x,y)$ has more variation. I know almost nothing &lt;em&gt;a priori&lt;/em&gt; about the function, except that inside the domain $x_0\le x \le x_1$, $y_0\le y \le y_1$, there maybe regions where it is infinite (that I obviously don't want to sample, so the algorithm should detect that too), and that everywhere else, the function is continuous. These singular regions are &quot;smooth sets&quot;.&lt;/p&gt;&#xA;" OwnerUserId="988" LastActivityDate="2017-10-07T01:16:14.367" Title="Adaptive plotting of two-variable functions $z=f(x,y)$ algorithm pseudocode?" Tags="|reference-request|plotting|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19332" PostTypeId="2" ParentId="19330" CreationDate="2015-04-09T18:30:14.070" Score="2" Body="&lt;p&gt;First, you should simplify the problem, so you can test the pieces individually. Get the code working first with V = 0. A simpler Psi0 would also help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, you need to have a better understanding of exactly where things go wrong. Python includes the python debugger pdb. If you put the line: &quot;import pdb;pdb.set_trace()&quot; into your code, it will give you a prompt where you can look at the contents of variables, try out different commands, plot variables, and so on. The pdb documentation can help you with this. I would recommend doing this right before your time loop, to see if everything has been initialized correctly. Then, stick it inside the loop in order to identify exactly where things fail. Debugging one line is much easier than a whole code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, keep at it! There were a bunch of other things I was going to say based on your first code, but the second one solved most of them. &lt;/p&gt;&#xA;" OwnerUserId="14704" LastActivityDate="2015-04-09T18:30:14.070" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19333" PostTypeId="2" ParentId="19295" CreationDate="2015-04-09T19:26:28.870" Score="0" Body="&lt;p&gt;I am answering my own question here, based on research on various sites. Thanks Geoff for helping with the radar chart idea. It is what I am using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I needed to calculate the centroid of four x,y coordinates&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find the centroid of triangle 1&#xA;(x1+x2+x4)/3&#xA;(y1+y2+y4)/3&#xA;The result is a coordinate such as (12,10.6). Fractions are possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find the centroid of triangle 2 &#xA;(x2+x3+x4)/3&#xA;(y2+y3+y4)/3&#xA;The result is a coordinate such as (8.3,7.0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find the midpoint of the line between the centroids of triangle 1 and triangle 2&#xA;(x1+x2)/2&#xA;(y1+y2)/2&#xA;The result is a coordinate that represents the centroid of the four original x,y coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fjZIj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/szf7y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="8326" LastActivityDate="2015-04-09T19:26:28.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19334" PostTypeId="1" CreationDate="2015-04-09T19:35:56.677" Score="0" ViewCount="65" Body="&lt;p&gt;I have a Cartesian grid (100x100) in which some of the points are known (30 out of 10,000) and the rest are unknown. I want to use the known points and estimate the other cells. Is there any philosophies behind the way that I must use the known points to estimate the unknown cells? In other words, does it matter that where I should start the estimation? Generally speaking, I personally prefer to start from the locations where I have the maximum amount of the information rather than a random location that probably I might have one known data. However, there is another school says that I should start from a random location in order to prevent any bias. I am more interested to know your ideas about any differences between these methods.&#xA;Thanks !&lt;/p&gt;&#xA;" OwnerUserId="14583" LastEditorUserId="14583" LastEditDate="2015-04-09T20:36:16.040" LastActivityDate="2015-05-10T13:58:53.107" Title="Best path for estimation" Tags="|linear-algebra|statistics|error-estimation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19336" PostTypeId="2" ParentId="19327" CreationDate="2015-04-10T08:27:02.767" Score="4" Body="&lt;p&gt;If I read this correctly, you can rewrite your problem as an ordinary least square problem $\left\|Ax-b\right\|$, where A is the stacked Vandermonde-Matrix of the derivatives of the Zernike polynomials and $b$ the stacked measured data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\chi^2 =\left \|\begin{pmatrix}&#xA; \frac{\partial P_n}{\partial x}\\ &#xA; \frac{\partial P_n}{\partial y}   &#xA;\end{pmatrix}C-\begin{pmatrix}\beta^x\\\beta^y\end{pmatrix}\right\|^2$$&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2015-04-10T08:39:34.737" LastActivityDate="2015-04-10T08:39:34.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19338" PostTypeId="2" ParentId="19334" CreationDate="2015-04-10T12:11:46.507" Score="1" Body="&lt;p&gt;The important question is &quot;What do you know how the values at points where you did not measure behave, relative to surrounding points?&quot;. In other words, what &lt;em&gt;prior knowledge&lt;/em&gt; do you have about how the values behave?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know that values at each point are completely independent of their neighbors, but that they are typically small, then the best you can do is to set unknown values to a small value. If you know that the value at each point is typically correlated with the values of its neighbors, then you will likely want to choose the value at unknown points to be &quot;similar&quot; to the value at known or inferred neighbors, where &quot;similar&quot; depends on the correlation statistics you know. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you read the book &quot;Statistical Inverse Probles&quot; by Kaipio and Somersalo that explains the process and the thought that goes into modeling your prior knowledge in great detail, with well chosen examples.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-10T12:11:46.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19339" PostTypeId="1" CreationDate="2015-04-10T13:46:56.423" Score="3" ViewCount="449" Body="&lt;p&gt;I need to integrate the following integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;I = \int^z\frac{1-\zeta^2}{(1+\zeta^2)(\zeta-\zeta_l)(1-\zeta_l\zeta)}\prod_{k=2}^{n-1}\left ( \frac{\zeta-z_k}{1-\zeta z_k} \right )^{-\beta_k} d\zeta&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;along any complex path within the upper half of the unit circle in the argand plane, where $z_k$, $\beta_k$ and $\zeta_l$ have prescribed values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am wondering if anyone can suggest a way to robustly calculate this integral with their own code, or comment on if what I am doing below is sensible:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral arises in Kirchoff flow problems and is part of the Schwarz Christoffel Parameter problem. I have modified the following code from Toby Driscoll in an attempt to compute my integral. Note that this assumes that &lt;code&gt;qdat&lt;/code&gt; which is a matrix of the Gauss Jacobi Quadrature weights is available.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As explained in the comments below, the idea is to refine the interval on which quadrature is applied such that any singularities $z_k$ (other than the end point of integration, are farther away than at least half the integration interval. This is coined &quot;adaptive or compound&quot; Gauss Jacobi Quadrature. Because of the form of the integrand (essentially it is of the type $\int f(x) (1+x)^\alpha(1-x)^\beta dx$) it is amenable to Gauss Jacobi Quadrature.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function I = modified_hpquad(z1,z2,sing1,z,beta,qdat)&#xA;%HPQUAD (not intended for calling directly by the user)&#xA;%   Numerical quadrature for the half-plane map.&#xA;&#xA;%   Copyright 1998 by Toby Driscoll.&#xA;%   $Id: hpquad.m 298 2009-09-15 14:36:37Z driscoll $&#xA;&#xA;%   HPQUAD(z1,z2,sing1,z,beta,qdat)&#xA;%   z1,z2 are vectors of left and right endpoints.  sing1 is a vector of&#xA;%   integer indices which label the singularities in z1.  So if sing1(5)&#xA;%   = 3, then z1(5) = z(3).  A zero means no singularity.  z is the&#xA;%   vector of finite singularities; beta is the vector of associated&#xA;%   turning angles.  qdat is quadrature data from SCQDATA.&#xA;%&#xA;%   Make sure z and beta are column vectors.&#xA;%   &#xA;%   The integral is subdivided, if necessary, so that no singularity&#xA;%   lies closer to the left endpoint than 1/2 the length of the&#xA;%   integration (sub)interval.&#xA;&#xA;&#xA;%Some parameters for modified SCT:&#xA;sl = 0;&#xA;&#xA;&#xA;nqpts = size(qdat,1);&#xA;% Note: Here n is the total number of *finite* singularities; i.e., the&#xA;% number of terms in the product appearing in the integrand.&#xA;n = length(z);&#xA;&#xA;bigz = z(:,ones(1,nqpts));&#xA;bigbeta = beta(:,ones(1,nqpts));&#xA;if isempty(sing1)&#xA;  sing1 = zeros(length(z1),1);&#xA;end&#xA;&#xA;%I = zeros(size(z1));&#xA;I = 0;&#xA;%nontriv = find(z1(:)~=z2(:))';&#xA;nontriv = find(z1~=z2);&#xA;&#xA;if nontriv == 1&#xA;&#xA;    za = z1; &#xA;    zb = z2;&#xA;    sng = sing1;&#xA;&#xA;    if sng &amp;gt; n&#xA;        sng = 0; %No singularity &#xA;    end&#xA;&#xA;    % Allowable integration step, based on nearest singularity.&#xA;    dist = min(1,2*min(abs(z([1:sng-1,sng+1:n])-za))/abs(zb-za)); % z([1:sng-1,sng+1:n]) jumps over sing&#xA;    zr = za + dist*(zb-za);&#xA;    ind = rem(sng+n,n+1)+1;&#xA;&#xA;    % Adjust Gauss-Jacobi nodes and weights to interval.&#xA;    nd = ((zr-za)*qdat(:,ind) + zr + za).'/2; % G-J nodes&#xA;    wt = ((zr-za)/2) * qdat(:,ind+n+1);     % G-J weights&#xA;    %terms = nd(ones(n,1),:) - bigz;&#xA;    terms = (nd(ones(n,1),:) - bigz)./(1-bigz.*nd(ones(n,1),:)); %Modified SC&#xA;    if any(terms(:)==0) &#xA;        % Endpoints are practically coincident.&#xA;        I = 0;&#xA;    else&#xA;        % Use Gauss-Jacobi on first subinterval, if necessary.&#xA;        if sng &amp;gt; 0&#xA;        terms(sng,:) = terms(sng,:)./abs(terms(sng,:));&#xA;        wt = wt*(abs(zr-za)/2)^beta(sng);&#xA;        end&#xA;        %I(k) = exp(sum(log(terms).*bigbeta,1))*wt;&#xA;        I = exp(log((1-nd.^2)./((1+nd.^2).*(nd-sl).*(1-sl.*nd)))+sum(log(terms).*bigbeta,1))*wt;&#xA;        while dist &amp;lt; 1              &#xA;            % Do regular Gaussian quad on other subintervals.&#xA;            zl = zr;&#xA;            dist = min(1,2*min(abs(z-zl))/abs(zl-zb));&#xA;            zr = zl + dist*(zb-zl);&#xA;            nd = ((zr-zl)*qdat(:,n+1) + zr + zl).'/2;&#xA;            wt = ((zr-zl)/2) * qdat(:,2*n+2);&#xA;            %terms = nd(ones(n,1),:) - bigz;&#xA;            terms = (nd(ones(n,1),:) - bigz)./(1-bigz.*nd(ones(n,1),:));&#xA;            I = I + exp(log((1-nd.^2)./((1+nd.^2).*(nd-sl).*(1-sl.*nd)))+sum(log(terms).*bigbeta,1))*wt;&#xA;        end&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code above is a modified version of a code which does exactly what I want for the integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;I = \int^z\prod_{k=1}^{n-1}\left ( \zeta-z_k \right )^{-\beta_k} d\zeta&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I have copied that code down for reference here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function I = hpquad(z1,z2,varargin)&#xA;%HPQUAD (not intended for calling directly by the user)&#xA;%   Numerical quadrature for the half-plane map.&#xA;&#xA;%   Copyright 1998 by Toby Driscoll.&#xA;%   $Id: hpquad.m 298 2009-09-15 14:36:37Z driscoll $&#xA;&#xA;%   HPQUAD(z1,z2,sing1,z,beta,qdat)&#xA;%   z1,z2 are vectors of left and right endpoints.  sing1 is a vector of&#xA;%   integer indices which label the singularities in z1.  So if sing1(5)&#xA;%   = 3, then z1(5) = z(3).  A zero means no singularity.  z is the&#xA;%   vector of finite singularities; beta is the vector of associated&#xA;%   turning angles.  qdat is quadrature data from SCQDATA.&#xA;%&#xA;%   Make sure z and beta are column vectors.&#xA;%   &#xA;%   HPQUAD integrates from a possible singularity at the left end to a&#xA;%   regular point at the right.  If both endpoints are singularities,&#xA;%   you must break the integral into two pieces and make two calls, or&#xA;%   call HPQUAD(z1,z2,sing1,sing2,z,beta,qdat) and accept an automatic&#xA;%   choice. &#xA;%   &#xA;%   The integral is subdivided, if necessary, so that no singularity&#xA;%   lies closer to the left endpoint than 1/2 the length of the&#xA;%   integration (sub)interval.&#xA;&#xA;if nargin==7&#xA;  % Break into two pieces with recursive call.&#xA;  [sing1,sing2,z,beta,qdat] = deal(varargin{:});&#xA;  mid = (z1+z2)/2;&#xA;  mid = mid + 1i*abs(mid);&#xA;  I1 = hpquad(z1,mid,sing1,z,beta,qdat);&#xA;  I2 = hpquad(z2,mid,sing2,z,beta,qdat);&#xA;  I = I1-I2;&#xA;  return&#xA;else&#xA;  [sing1,z,beta,qdat] = deal(varargin{:});&#xA;end&#xA;&#xA;&#xA;&#xA;&#xA;nqpts = size(qdat,1);&#xA;% Note: Here n is the total number of *finite* singularities; i.e., the&#xA;% number of terms in the product appearing in the integrand.&#xA;n = length(z);&#xA;bigz = z(:,ones(1,nqpts));&#xA;bigbeta = beta(:,ones(1,nqpts));&#xA;if isempty(sing1)&#xA;  sing1 = zeros(length(z1),1);&#xA;end&#xA;&#xA;I = zeros(size(z1));&#xA;nontriv = find(z1(:)~=z2(:))';&#xA;&#xA;&#xA;&#xA;for k = nontriv&#xA;  za = z1(k);&#xA;  zb = z2(k);&#xA;  sng = sing1(k);&#xA;&#xA;  % Allowable integration step, based on nearest singularity.&#xA;  dist = min(1,2*min(abs(z([1:sng-1,sng+1:n])-za))/abs(zb-za));&#xA;%%  if isempty(dist), dist=1; end&#xA;  zr = za + dist*(zb-za);&#xA;  ind = rem(sng+n,n+1)+1;&#xA;  % Adjust Gauss-Jacobi nodes and weights to interval.&#xA;  nd = ((zr-za)*qdat(:,ind) + zr + za).'/2; % G-J nodes&#xA;  wt = ((zr-za)/2) * qdat(:,ind+n+1);   % G-J weights&#xA;  terms = nd(ones(n,1),:) - bigz;&#xA;  if any(terms(:)==0) &#xA;    % Endpoints are practically coincident.&#xA;    I(k) = 0;&#xA;  else&#xA;    % Use Gauss-Jacobi on first subinterval, if necessary.&#xA;    if sng &amp;gt; 0&#xA;      terms(sng,:) = terms(sng,:)./abs(terms(sng,:));&#xA;      wt = wt*(abs(zr-za)/2)^beta(sng);&#xA;    end&#xA;    I(k) = exp(sum(log(terms).*bigbeta,1))*wt;&#xA;    while dist &amp;lt; 1              &#xA;      % Do regular Gaussian quad on other subintervals.&#xA;      zl = zr;&#xA;      dist = min(1,2*min(abs(z-zl))/abs(zl-zb));&#xA;      zr = zl + dist*(zb-zl);&#xA;      nd = ((zr-zl)*qdat(:,n+1) + zr + zl).'/2;&#xA;      wt = ((zr-zl)/2) * qdat(:,2*n+2);&#xA;      terms = nd(ones(n,1),:) - bigz;&#xA;      I(k) = I(k) + exp(sum(log(terms).*bigbeta,1)) * wt;&#xA;    end&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastActivityDate="2015-04-10T13:46:56.423" Title="Calculate Integral Using Gauss Jacobi Quadrature or otherwise" Tags="|matlab|numerical-analysis|quadrature|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19342" PostTypeId="2" ParentId="19330" CreationDate="2015-04-11T03:44:28.857" Score="5" Body="&lt;p&gt;Caveat: I did not read beyond the statement&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So, what I did was to define functions for the potential and the second derivative, and use Euler's method.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So there may be other issues with your code, but this one is already fundamental.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;You are trying to solve a wave equation with Euler's method.  This is numerically unstable.&lt;/strong&gt;  To see why, note that the second-derivative operator (and its centered finite-difference discretization) is symmetric and has only negative real eigenvalues.  When you combine this with the imaginary unit multiplying the time-derivative, you find that the dynamics of this equation are wavelike -- the eigenvalues of the semi-discretization are purely imaginary.  Meanwhile, the stability region of Euler's method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\{z\in{\mathbb C : |1+z|\le 1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;contains none of the imaginary axis (except for the origin).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Instead of Euler's method, you should use a time discretization that is stable on the imaginary axis&lt;/strong&gt;, like the midpoint method or the trapezoidal method.  Some good methods and references can be found in answers to this question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/10876/are-there-simple-ways-to-numerically-solve-the-time-dependent-sch%C3%B6dinger-equatio/10878#10878&quot;&gt;Are there simple ways to numerically solve the time-dependent Sch&amp;#246;dinger equation?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-04-11T03:44:28.857" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19343" PostTypeId="1" CreationDate="2015-04-11T07:46:20.467" Score="8" ViewCount="137" Body="&lt;p&gt;I'm using VEGAS integration, specifically the GSL implementation, for some QCD calculations, and I've been investigating the behavior of the algorithm for various numbers of iterations in an attempt to get more accurate results. The value produced by the integration routine changes rather drastically with the number of iterations, as shown in this plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sjHmY.png&quot; alt=&quot;plot of value increasing with iterations&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results shown are for one specific random seed, but they are indicative of other results obtained by running many instances of the program with different random seeds and averaging; specifically, the value with a million iterations is significantly larger than that with a hundred thousand iterations, as shown in this plot (which includes the above value, plus a few other terms whose values are nearly constant):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/vIX3G.png&quot; alt=&quot;box plot of statistical results&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best guess as to the cause is that there are peaks in the function being integrated which have a characteristic size around $V/10^6$, with $V$ being the integration volume (in this case, an eight-dimensional hypercube). These peaks would then be detected when sampling with a million points, but sampling with only a hundred thousand points would have a good chance of missing them. That would explain why &lt;em&gt;some&lt;/em&gt; of the values at $10^5$ iterations are larger than the values with $10^6$ iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Preliminary question&lt;/em&gt;: am I on the right track with this explanation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I am, that brings up the question of how to know when I can safely stop increasing the number of iterations. For example, is there some sort of analysis I can apply to the analytic form of the function to pick out the scale of features which will significantly affect the result of the integration?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The integral in question, in case it helps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{multline}&#xA;\int_{\tau}^1\mathrm{d}z\int_{\tau/z}^1\mathrm{d}\xi\iiint\mathrm{d}^2\vec{q}_1\mathrm{d}^2\vec{q}_2\mathrm{d}^2\vec{q}_3 \\&#xA;\frac{1}{1 - \xi}\biggl[\frac{\frac{\tau}{z\xi}g\bigl(\frac{\tau}{z\xi}\bigr)D(z)}{z^2}\frac{(1 - \xi(1-\xi))^2}{\xi^2}F\biggl(\frac{\vec{k}}{\xi} + \vec{q}_3 - \vec{q}_2, z\biggr)&#xA;- \frac{\frac{\tau}{z}g\bigl(\frac{\tau}{z}\bigr)D(z)}{z^2}F\bigl(\vec{k} + \vec{q}_3 - \vec{q}_2, z\bigr)\biggr] \\&#xA;\times F\bigl(\vec{k} + \vec{q}_3 - \vec{q}_1, z\bigr)F(\vec{q}_3, z)\frac{\vec{q}_1\cdot\vec{q}_2}{q_1^2q_2^2}&#xA;\end{multline}$$&#xA;where $F(\vec{q}, z) = \frac{z^{0.288}}{C\pi}\exp\bigl(-q^2 z^{0.288}/C\bigr)$ with $C$ a constant.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2015-08-11T01:12:57.633" Title="Choosing how many iterations to use in VEGAS" Tags="|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19344" PostTypeId="1" AcceptedAnswerId="19347" CreationDate="2015-04-12T02:28:11.283" Score="0" ViewCount="349" Body="&lt;p&gt;I need to permute the degrees of freedom of a system and apply this permutation to a few sparse matrices in CCS (or CRS) format. I could construct a permutation matrix and perform sparse matrix-matrix multiplies, but that seems like overkill. Are there standard algorithms for permuting sparse matrices? How do users of packages like METIS permute sparse matrices? &lt;/p&gt;&#xA;" OwnerUserId="1862" LastActivityDate="2015-04-12T12:46:25.597" Title="Standard Algorithms for Permuting CCS or CRS Sparse Matrices" Tags="|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19345" PostTypeId="1" CreationDate="2015-04-12T09:42:24.820" Score="2" ViewCount="1017" Body="&lt;p&gt;There are examples for fast numerical inversion of the Laplace transforms. For example here:&#xA;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/32824-numerical-inversion-of-laplace-transforms-in-matlab&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/32824-numerical-inversion-of-laplace-transforms-in-matlab&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: are there implementations of Fast Laplace transform?&#xA;For example if one takes and array of values $[a_1 \ldots a_n] $ and transforms then to  $[b_1 \ldots b_n] $.&lt;/p&gt;&#xA;" OwnerUserId="10345" LastEditorUserId="10345" LastEditDate="2015-09-08T19:13:14.250" LastActivityDate="2015-09-08T19:13:14.250" Title="Fast Forward Laplace transform" Tags="|fourier-transform|signal-processing|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19346" PostTypeId="1" CreationDate="2015-04-12T12:13:53.993" Score="1" ViewCount="110" Body="&lt;p&gt;I am investigating stability and convergence of series of approximations for coupled thermoelasticity problem yielded by one-step recurrent time-integration scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've managed to show that the one-step time integration scheme ( OSTIS ) is stable and convergent when the gradient of thermal field is positive (energy of heat of a body at every step is not less than energy on the previous time step of OSTIS). &lt;strong&gt;But now I wonder if this is sufficient result or this is unnatural condition?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generally speaking can someone tell the example of thermo-elastic&#xA;process that occurs in nature (or in machinery) where the gradient of&#xA;temperature of a body is always positive?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;P.S. As I understand this occurs when the system/body is constantly heated by external source, so that more heat comes is than comes out in every time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to know when this is true:&#xA;\begin{align}&#xA; &amp;amp; \tfrac{1}{2}\Delta {{t}^{-1}}s({{z}^{j+1}}-{{z}^{j}},{{u}^{j+1}}+{{u}^{j}})\ge 0 \\ &#xA;\end{align}&#xA;Here I user the following notation:&#xA;\begin{align}&#xA;  &amp;amp; \,\theta =\theta (x,t) \\ &#xA;\end{align} is the temperature field over the body in every time;&#xA;\begin{align}&#xA; &amp;amp; \frac{\partial }{\partial t}{{\left. [{{\theta }^{\Delta t}}(t)] \right|}_{{{t}_{j}}}}={{\left. {{[{{\theta }^{\Delta t}}(t)]}^{\prime }} \right|}_{{{t}_{j}}}}={{z}^{j}} \\ &#xA;\end{align} and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA; &amp;amp; s(\theta ,\xi )=\int_{\Omega }{\theta \xi dx} \\ &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically, my question concerns term&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align} \Delta {{t}^{-1}}[{{z}^{j+1}}-{{z}^{j}}]\ge 0 \end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there physical processes where this occurs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Meaning of indexes on the top of variables can be understood from here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  &amp;amp; {{\theta }^{\Delta t}}(t)={{\theta }^{j+1/2}}+\Delta t[{{\omega }_{j}}(t)-\tfrac{1}{2}]{{z}^{j+1/2}}+\tfrac{1}{2}\Delta {{t}^{2}}{{\omega }_{j}}(t)[{{\omega }_{j}}(t)-1]{{{\dot{z}}}^{j+1/2}}, \\ &#xA; &amp;amp; {\theta }'(t)={{z}^{j+1/2}}+\Delta t[{{\omega }_{j}}(t)-\tfrac{1}{2}]{{{\dot{z}}}^{j+1/2}} \\ &#xA; &amp;amp; {\theta }''(t)={{{\dot{z}}}^{j+1/2}}, \\ &#xA; &amp;amp; \forall t\in [{{t}_{j}},{{t}_{j+1}}]. \\&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerUserId="15074" LastEditorUserId="15074" LastEditDate="2015-04-15T07:42:22.960" LastActivityDate="2016-02-01T14:33:26.770" Title="Conditions for always positive gradient of heat field in evolutionary thermo-elastic system" Tags="|finite-element|hyperbolic-pde|model|modeling|numerical-modelling|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19347" PostTypeId="2" ParentId="19344" CreationDate="2015-04-12T12:46:25.597" Score="1" Body="&lt;p&gt;A function written in C to permute a CCS matrix is shown here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/CSparse/CSparse/Source/cs_permute.c&quot; rel=&quot;nofollow&quot;&gt;http://www.cise.ufl.edu/research/sparse/CSparse/CSparse/Source/cs_permute.c&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm is fairly simple but there aren't many comments. If you want&#xA;more details, take a look at Davis' book where the routine is presented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/CSparse/&quot; rel=&quot;nofollow&quot;&gt;http://www.cise.ufl.edu/research/sparse/CSparse/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-04-12T12:46:25.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19348" PostTypeId="2" ParentId="19343" CreationDate="2015-04-12T22:33:45.333" Score="-1" Body="&lt;p&gt;The integral in question has a singularity at $\xi=1$, and this singularity is at the border of the integration domain. You have to handle singularities analytically or remove them by some other mean, before you can use numerical integration, and VEGAS integration is no exception here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably have some idea why the value of the integral is well defined and finite despite this singularity. Such an idea normally also suggests a method how the singularity could be removed or handled analytically.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2015-04-12T22:33:45.333" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19350" PostTypeId="1" CreationDate="2015-04-13T10:47:15.027" Score="0" ViewCount="844" Body="&lt;p&gt;I have a FE model of a simple plate with hole (tension load) with HEX20 mesh. I need to obtain the Shape Function of one of the elements (the one with highest stress) and plot it (with MATLAB). After that I will find the Stress Gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how can I get the values of the stress between the 3 nodes on one of the edges of the HEX20 element? I'm sure this is done with the shape functions, but I don't know how to do it in practice. I only know how to check the stress values on the nodes...&lt;/p&gt;&#xA;" OwnerUserId="15081" LastActivityDate="2015-04-13T12:34:38.193" Title="FEM - Shape function of a HEX20 - plot in MATLAB" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19352" PostTypeId="2" ParentId="19350" CreationDate="2015-04-13T12:34:38.193" Score="3" Body="&lt;p&gt;The finite element solution is generally represented by a linear combination of basis functions:&#xA;$$&#xA;u_h(x)=\sum_{i=1}^N \alpha_i \phi_i(x)&#xA;$$&#xA;where the $\alpha_i$s are the nodal values that you solve for and the $\phi_i(x)$s are functions you choose in advance. In order to evaluate $u_h$ between some nodes, assuming you have solved for all the $\alpha_i$s already, you need to find the positions $x$ you want to evaluate at and plug into the above equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now generally speaking, the basis functions have compact support, so you don't need to evaluate the above sum over the whole mesh, but only on the supporting elements (i.e. the elements that contain the edge in question). Also, the basis functions are typically broken down into shape functions which are known and the same on each element, making it relatively easy to look name them for future reference (i.e. HEX20).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question becomes, is the stress a primitive variable in your method, or do you have to derive it by taking derivatives of the solution? If you have to take derivatives, does your method have continuous derivatives on the edge in question, or will taking the derivative of the above sum lead to a multi-valued answer? If it's the latter, you may have to use some sort of projection/recovery method to get a single-valued answer for the stress on that edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you know what you want, you can look up the shape functions and evaluate them at the points that you care about to sum up the solution value or its derivatives. If you are using some sort of package like NASTRAN or ABAQUS, you can probably just ask it to tell you the values you are interested in. It will use the best method it knows to work them out.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-04-13T12:34:38.193" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19353" PostTypeId="1" AcceptedAnswerId="19354" CreationDate="2015-04-13T15:07:45.843" Score="0" ViewCount="3167" Body="&lt;p&gt;I have three 32 bit integers a,b,c. I want to make &#xA;10th bit of a=(23 rd bit of b) xor (4th bit of c) without &#xA;disturbing other bits of a. How can I do this in C programming language? &#xA;a can be zero also. In that case I consider &#xA;a= 00...0, 32 zeros.   &lt;/p&gt;&#xA;" OwnerUserId="277" LastActivityDate="2015-04-13T15:23:34.600" Title="How to change one bit of a 32 bit integer in C" Tags="|c|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19354" PostTypeId="2" ParentId="19353" CreationDate="2015-04-13T15:23:34.600" Score="1" Body="&lt;pre&gt;&lt;code&gt;// 10th bit of a=(23 rd bit of b) xor (4th bit of c)&#xA;&#xA;if( a )&#xA;{ // skip if 'a' already all 0's&#xA;    int tempB = (b&amp;gt;&amp;gt;23)&amp;amp;0x01; // extract bit 23 and place on bit 0&#xA;    int tempC = (c&amp;gt;&amp;gt;4)&amp;amp;0x01;  // extract bit 4 and place in bit 0&#xA;    int tempXOR = (tempB^tempC)&amp;amp;0x01;// XOR the bits&#xA;&#xA;    a &amp;amp;= ~(1&amp;lt;&amp;lt;10);        // clear bit 10&#xA;    a |= (tempXOR&amp;lt;&amp;lt;10);   // place result of XOR into bit 10 of a&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15083" LastActivityDate="2015-04-13T15:23:34.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19355" PostTypeId="1" CreationDate="2015-03-26T06:47:30.350" Score="11" ViewCount="1194" Body="&lt;p&gt;In FEM literature, semi-variational methods are typically used in the solution of time-dependent PDEs. I have not seen a fully-variational approach i.e. where space and time are discretised by FEM, perhaps allowing the use of unstructured space-time meshes. Although timestepping methods may be easier to implement, is there a particular reason why space-time meshing is not viable? I imagine one has to tailor meshes to respect the physical properties of a given problem, but I am not certain.&lt;/p&gt;&#xA;" OwnerDisplayName="stephn28" LastEditorUserId="1804" LastEditDate="2015-04-14T12:06:55.457" LastActivityDate="2015-10-12T16:01:58.427" Title="Space-time finite element discretization for time-dependent PDEs" Tags="|pde|finite-element|discretization|time-integration|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19356" PostTypeId="1" CreationDate="2015-04-14T13:37:19.583" Score="7" ViewCount="469" Body="&lt;p&gt;The &lt;a href=&quot;http://en.wikipedia.org/wiki/Fast_marching_method&quot; rel=&quot;noreferrer&quot;&gt;Fast Marching Method&lt;/a&gt;, &lt;a href=&quot;http://www.researchgate.net/publication/220411914_A_Fast_Iterative_Method_for_Eikonal_Equations/file/79e415088285c84739.pdf&quot; rel=&quot;noreferrer&quot;&gt;Fast Iterative Method&lt;/a&gt;, and &lt;a href=&quot;http://www.math.uci.edu/~zhao/homepage/research_files/FS_factored.pdf&quot; rel=&quot;noreferrer&quot;&gt;Fast Sweeping Method&lt;/a&gt; are three ways of solving the &lt;a href=&quot;http://en.wikipedia.org/wiki/Eikonal_equation&quot; rel=&quot;noreferrer&quot;&gt;Eikonal Equation&lt;/a&gt; on a discrete grid, essentially just a wavefront spreading out from initial points, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/qPEIb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that we want to compute the time $T(x, y)$ at which a wavefront, moving across a field with a velocity function $F(x, y)$, reaches each point in a grid, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/HZyYv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where walls and such would be represented with $F(x, y)=\epsilon$ for some very small $\epsilon$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is good and these methods work very well in practice, however they all apply to fixed size grids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many applications, it is better to represent the speed field with some sort of tree, say a quadtree or octree. Areas with a larger tree cell representing them would simply then all have the same speed, though some sort of interpolation may even be possible. This can be interpreted by saying that we desire &quot;higher resolution&quot; results in some areas with a higher tree density, and &quot;lower resolution&quot; results in some areas with a lower tree density. Either way, is it possible to apply these methods (or methods like them) for solving $T(x, y)$ using a speed field $F(x, y)$ represented with some kind of a tree?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trivially, I know you could simply find the &quot;lowest resolution&quot; of your tree - IE the smallest cell - then transform each larger cell into these smaller cells as the algorithm runs over them, but this seems far less than optimal, and kinda defeats the point of a tree in the first place. Thus, is there a better way to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems pretty clear that the goal would be to have a $T(x, y)$ field that is also represented in the same kind of tree that $F(x, y)$ is represented in, to give similar density results, however I don't know if that's the best way to think about it either. More specifically $T(x, y)$ would probably end up being some kind of form where inner points (if we were trying to recreate the fixed-size discrete grid) are interpolations between the corners of the tree grid, but this is also essentially what it's doing anyway in order to map a discrete grid answer to a continuous one, so I don't feel like that's too big of a concern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really, this question was just motivated because current implementations of an Eikonal Equation solver are slow, for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The dependencies each grid cell has on its neighbor's results. Something like the Fast Iterative Method makes them essentially less &quot;tied together&quot; so some level of parallelization is possible, but I'm wondering if there would be some way to further split up the work into independent grid cells that possibly derive results dependent on their neighbor values which will eventually be computed. This is sort of a side-note but I feel like the tree aspect may be tied into it due to how &quot;low density&quot; results would have to be smoothly approximated by &quot;higher density&quot; neighbors. This gets into point two:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The current methods compute the same accuracy for all points in the grid. In many situations good accuracy is only needed for specific portions of the map, so it would be nice if it were possible to &quot;split up&quot; the work into sections of desired high resolution results and sections of desired low resolution results. Doing that using some kind of a tree doesn't seem that straightforward with the current methods of solving this equation though.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8275" LastEditorUserId="1804" LastEditDate="2015-04-14T13:51:04.653" LastActivityDate="2015-04-15T08:20:03.960" Title="Eikonal Equation solver with different grid densities" Tags="|nonlinear-equations|wave-propagation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19357" PostTypeId="1" AcceptedAnswerId="19358" CreationDate="2015-04-14T13:41:20.813" Score="1" ViewCount="166" Body="&lt;p&gt;I would like to know more on choosing hardware to get the maximum price/performance when using the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc library&lt;/a&gt; (and various third-party preconditionners)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently working on a 2 cpu (2*E5-2640) workstation with a total of 24 logical cores. But my system is bounded by RAM banwidth such that I don't get any speed-up with more than six cores (even on a large problem). I get such results both on my apps and with the &lt;a href=&quot;https://i.stack.imgur.com/wl0El.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;included benchmark&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since some of my colleagues are in the process of buying hardware in the order or 100 cores, I would like to know how hardware should be chosen in order to get better memory bandiwidth.  &lt;/p&gt;&#xA;" OwnerUserId="11869" LastActivityDate="2015-04-14T15:15:10.887" Title="Choosing hardware to use with PETSc" Tags="|algorithms|petsc|performance|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19358" PostTypeId="2" ParentId="19357" CreationDate="2015-04-14T15:15:10.887" Score="4" Body="&lt;p&gt;If you're using iterative methods with assembled matrices, just buy DDR channels. Don't pay attention to number of cores when loooking at the spec sheet. Within the same class of processors (e.g., a recent generation of Xeon), the achievable memory bandwidth will be proportional to the quoted peak bandwidth. Note that for very small problem sizes, you might be able to fit within cache and thus see significantly better performance.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2015-04-14T15:15:10.887" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19359" PostTypeId="1" AcceptedAnswerId="19364" CreationDate="2015-04-14T16:13:59.833" Score="1" ViewCount="156" Body="&lt;p&gt;This is an incredibly mundane integration, but for some reason I can't spot the error. I am just performing a simple 3d integration in spherical coordinates which should return the value of 1.0:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Nr = N;&#xA;Nt = round(pi*N);&#xA;Np = round(2*pi*N);&#xA;&#xA;rs = linspace(R0, Rf, Nr);&#xA;ts = linspace(0, pi, Nt);&#xA;ps = linspace(0, 2*pi, Np);&#xA;&#xA;dr = rs(2)-rs(1);&#xA;dt = ts(2)-ts(1);&#xA;dp = ps(2)-ps(1);&#xA;&#xA;%fprintf('Total number of nodes: %i\n', Nr*Nt*Np);&#xA;&#xA;C = 1/((4/3)*pi);&#xA;&#xA;fint = 0.0;&#xA;for ir = 2:Nr&#xA;  r = rs(ir);&#xA;  r2dr = r*r*dr;&#xA;  for it = 1:Nt-1&#xA;    t = ts(it);&#xA;    sintdt = sin(t)*dt;&#xA;    for ip = 1:Np-1&#xA;      p = ps(ip);&#xA;      fint = fint + C*r2dr*sintdt*dp;&#xA;    end &#xA;  end &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the same thing as &#xA;$$ \int_0^{2\pi} \int_0^\pi \int_0^1 C r^2 \sin(\theta) dr\, d\theta\, d\phi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I take&#xA;$$ C = \frac{1}{(4/3)\pi}$$&#xA;since that is the reciprocal of the volume of a sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I find that my returned integral value has a consistent error which is approximately &lt;code&gt;(1/N)/(1.0 - numeric_integral) ~= 0.66&lt;/code&gt;. So I'm making a mistake somewhere but I can't spot where or why.&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="9667" LastEditDate="2015-04-15T06:13:40.870" LastActivityDate="2015-04-15T06:13:40.870" Title="Simple integration in Matlab is off" Tags="|matlab|integration|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19361" PostTypeId="2" ParentId="19355" CreationDate="2015-04-14T16:27:15.307" Score="6" Body="&lt;p&gt;Full space-time discretization of time-dependent partial differential equations is indeed a thing. If you use a structured mesh in time (in the sense that the time discretization does not depend on space) and appropriate choice of trial and test functions, you can fit several standard time-stepping methods (Crank-Nicolson, implicit Euler or some Runge-Kutta schemes) into a Galerkin framework, which gives an elegant approach for analysis. This is described, for example, in Thomée's book &lt;em&gt;Galerkin Finite Element Methods for Parabolic Problems&lt;/em&gt; (Springer, 2nd ed., 2006) or Chrysafinos' and Walkington's paper &lt;em&gt;Error estimates for the discontinuous Galerkin&#xA;methods for parabolic equations&lt;/em&gt;, (SIAM J. Numer. Anal. 44.1, 349–366, 2006).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a fully unstructured mesh is less common, but can make sense for hyperbolic problems where you have a transport of information along characteristics. If you use a discontinuous Galerkin formulation, each space-time element only couples with the neighboring element via face terms (you have no global continuity requirements), and you can use a sweeping process to compute the solution by going from element to element along characteristics -- a sort of &quot;oblique&quot; time-stepping. Of course, this is much more difficult to implement, even if it does not require storing the full space-time mesh (which can be prohibitive). On the other hand, you gain the advantage of unstructured meshes of allowing local (adaptive) refinement and hence locally adaptive time-stepping. One of the earliest references I know is &lt;a href=&quot;http://web.mit.edu/ehliu/Public/ehliu/SpaceTime/Hughes_1988_Space-time_FEM_for_elastodynamics_formulations_and_error_estimates.pdf&quot;&gt;Hughes and Hulbert: &lt;em&gt;Space-time finite element methods for elastodynamics: formulations and error estimates&lt;/em&gt;, Computer Methods in Applied Mechanics and Engineering 66(3):339-363, 1988&lt;/a&gt;. There's also a &lt;a href=&quot;http://arxiv.org/abs/0804.0942&quot;&gt;PhD thesis by Shripat Thite on &lt;em&gt;Spacetime Meshing for Discontinuous Galerkin Methods&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another context where I have seen this idea is in PDE-constrained optimization for parabolic problems. There you can formulate the first-order necessary optimality conditions as a coupled system of forward-backward equations, which you can interpret as the mixed formulation of a 2nd-order in time, 4th-order in space elliptic equation with initial-final (and boundary) conditions. By doing an adaptive space-time discretization of this coupled system, you can have an efficient one-shot approach for computing the solution, see &lt;a href=&quot;http://dx.doi.org/10.1515/jnum-2012-0005&quot;&gt;Gong, Hinze, Zhou: &lt;em&gt;Space-time finite element approximation&#xA;of parabolic optimal control problems&lt;/em&gt;, J Numer. Math. 20(2):111-145 (2012)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-04-14T16:27:15.307" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19362" PostTypeId="1" CreationDate="2015-04-14T16:33:55.413" Score="2" ViewCount="642" Body="&lt;p&gt;For a given discontinuous galerkin (DG) implementation for Navier-Stokes, targeting 10,000 to 1,000,000 4th order cells in 3D, I'm using PETSc's suite of linear/non-linear solvers on the back-end. It seems to me that there are two broad methods of implicitly solving the discretization. One can explicitly form the Jacobian in a sparse format and use whatever tools are available to solve it, or use a Jacobian-Free Newton-Krylov method that approximates the jacobian-vector products by evaluating the right-hand-side by finite differences. Advantages and disadvantages I see of each:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Explicit Jacobian&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Con - Requires a large amount of memory (precisely how much? can I do this with &amp;lt;100GB total memory?)&lt;/li&gt;&#xA;&lt;li&gt;Con - Requires writing code to evaluate the Jacobian and store it&lt;/li&gt;&#xA;&lt;li&gt;Pro - Allows the use of black-box preconditioners already present in PETSc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Jacobian-Free Newton-Krylov&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pro - Minimal memory needs&lt;/li&gt;&#xA;&lt;li&gt;Con - Unknown how often to re-linearize or what finite difference steps to use, how often to restart, etc.&lt;/li&gt;&#xA;&lt;li&gt;Con - must write own preconditioners.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'd like some input from those more familiar with these methods as to the advantages and disadvantages of them. Am I right to assume the explicit jacobian would be much faster?&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2015-04-16T19:44:17.007" Title="Jacobian-Free Newton-Krylov vs explicitly forming jacobian in DG" Tags="|petsc|discontinuous-galerkin|jacobian|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19363" PostTypeId="1" CreationDate="2015-04-14T16:55:01.537" Score="1" ViewCount="365" Body="&lt;p&gt;Consider a thin film with a perpendicular applied magnetic field $H_a$ in $z$-axis. The nonlocal relation between $H_a$, the self-field $H_\text{self}$ (generated by the eddy current $J$) and the local magnetic flux density $B_z$ is given by :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B_z = \mu(H_\text{self} + H_a)\quad (1)\\&#xA;H_\text{self} = \frac{1}{4\pi} \int \frac{J(r')(r-r’)}{(r-r')^2} dr$$&#xA;that is Biot-Savart’s law in 2D with $r=(x,y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One defines the local magnetization $m$ as : $J = \nabla\times m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the Biot-Savart's law becomes highly singular when $r = r'$, one uses the Fourier transform. In fact, by inverting (1), one obtains&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$m = 2\mathcal{F}^{-1}[ k^{-1} \mathcal{F}(\mu B_z – H_a)]\quad (2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal{F}$ and $\mathcal{F}^{-1}$ are the Fourier and inverse Fourier transforms and $k$ is the wave vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By increasing the applied magnetic field $H_a$, one can write the time evolution of the local magnetization as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dm}{dt} = 2\mathcal{F}^{-1}\left[ k^{-1} \mathcal{F} \left(\mu  \frac{dB_z}{dt} – \frac{dH_a}{dt}\right)\right]\\&#xA;\frac{dB_z}{dt} = \nabla\times E \enspace ,$$&#xA;the Farady's law.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think one can solve this diffusion equation using the weak form PDE interface in Comsol but I don’t know how to do it. However, I have no idea how to solve this in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1103/PhysRevB.84.054537&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nonlinear diffusion equation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15093" LastEditorUserId="20688" LastEditDate="2018-03-26T06:14:31.563" LastActivityDate="2020-05-12T13:55:22.380" Title="How to solve a nonlinear diffusion equation?" Tags="|matlab|nonlinear-equations|electromagnetism|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19364" PostTypeId="2" ParentId="19359" CreationDate="2015-04-14T17:12:15.453" Score="7" Body="&lt;p&gt;You're using a first order accurate integration technique (the rectangle rule) and your error is proportional to $1/N\propto\Delta r,\Delta \phi,\Delta \theta$. This is exactly the type of convergence behavior you should expect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use a more accurate integration scheme you will see faster convergence. You can easily implement the trapezoid rule for this integral and I would expect to see second order convergence: $\text{Error}\propto(1/N)^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit to add more details:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rectangle rule you're using is identical to the following (check your answers with this, they should match up to rounding errors):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%create a 3d grid of values&#xA;[PHI,THETA,R] = ndgrid(ps,ts,rs);&#xA;&#xA;%function to be integrated&#xA;F = C*R.^2.*sin(THETA);&#xA;fint = sum(sum(sum(F(1:end-1,1:end-1,2:end))))*dr*dt*dp;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while the trapezoid rule can be implemented as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%create a 3d grid of values&#xA;[PHI,THETA,R] = ndgrid(ps,ts,rs);&#xA;&#xA;%function to be integrated&#xA;F = C*R.^2.*sin(THETA);&#xA;&#xA;% INTEGRATE:&#xA;% r direction&#xA;I = sum( F(:,:,1:end-1)+F(:,:,2:end), 3 );&#xA;% theta direction&#xA;I = sum( I(:,1:end-1)+I(:,2:end), 2 );&#xA;% phi direction&#xA;I = sum( I(1:end-1)+I(2:end) );&#xA;fint = I*dr*dp*dt/8;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Computing the errors for different $N$ and plotting on a log-log plot reveals their order of accuracy. As expected, the trapezoid rule is $\mathcal{O}(1/N^2)$ while the rectangle rule is $\mathcal{O}(1/N)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jqxmf.png&quot; alt=&quot;log-log error plot&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-04-14T19:50:48.960" LastActivityDate="2015-04-14T19:50:48.960" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19365" PostTypeId="1" CreationDate="2015-04-14T23:46:29.173" Score="4" ViewCount="762" Body="&lt;p&gt;Consider a function $X(\xi,\nu)$, $2\pi$ periodic in $\xi$ satisfying &#xA;$$\nabla^2 X = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in a domain $D$ with $\nabla = (\partial_{\xi},\partial_{\nu})$. If I know the values of $X$ on the boundary $\partial D$, what is the simplest (in a numerical sense) way of finding $X$ in the interior? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $\partial D$ is a complex geometry, which will be evolving at each instant of time. It looks like, for example, the graph shown in figure 1 (this is the boundary together with lines at $\xi=0$, $\xi=2\pi$ and $\nu=-h$ for some large constant $h$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nht6i.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="8021" LastEditorUserId="123" LastEditDate="2015-04-15T06:34:03.963" LastActivityDate="2015-04-22T09:38:12.243" Title="Solving Laplace's equation on a domain with moving boundary" Tags="|pde|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19366" PostTypeId="2" ParentId="19365" CreationDate="2015-04-15T00:24:10.327" Score="3" Body="&lt;p&gt;You may wish to look into boundary integral equation (BIE) methods.  These reformulate homogeneous (zero forcing) PDEs into integral equations on the boundary of the geometry, thereby reducing a $d$-dimensional problem to a $d-1$ dimensional problem. These are often the most flexible methods for dealing with complex and moving geometries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you would only have to solve a 1D problem for a density (which is then used to compute your solution in the interior) along your geometry boundary. Systems arising in these methods are smaller and well-conditioned (usually) but are typically fully dense.  However, there are recent developments which also make numerical discretization of such problems more tractable (notably the Fast Multipole method of Greengard/Rokhlin, advanced quadrature rules, and work generalizing these ideas to the H-matrix and algebraic setting by Martinsson, Gillman, Bremer, and some others).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatives are to use volume-based methods like finite elements and mesh the geometry using something like Delaunay triangulation or a more robust mesh generator (which wouldn't be too difficult for your geometry). You could also look into the cut-cell/immersed boundary method, where you use a structured grid to represent the solution, and take a complicated boundary into account only in the cells which it intersects. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-04-15T00:24:10.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19368" PostTypeId="2" ParentId="4707" CreationDate="2015-04-15T06:00:19.763" Score="1" Body="&lt;p&gt;Disclaimer: I'm very familiar with Monte Carlo algorithms in general, but not with the VEGAS algorithm specifically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The VEGAS algorithm is based on the fact that if we knew the exact probability density that minimizes the variance of our Monte Carlo integration, we could get our answer using exactly 1 evaluation of the function f.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(\mathbf{x}) = \frac {|f(\mathbf{x})|} {\int_\Omega |f(\mathbf{x})| &#xA;d\mathbf{x}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't know the probability density, and estimating it exactly is not feasible for a high-dimensional function as it would quickly take up a massive amount of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead the VEGAS algorithm approximates it as an M-step piecewise constant function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm guessing you don't have access to the full article? The original article does NOT use the squared function, but the absolute one (a pre-print version can be found &lt;a href=&quot;http://slac.stanford.edu/pubs/slacpubs/1750/slac-pub-1839.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps answer your question. The original article answers most of this, so it might be worthwhile getting&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2015-04-15T06:00:19.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19369" PostTypeId="2" ParentId="19365" CreationDate="2015-04-15T06:35:54.427" Score="1" Body="&lt;p&gt;Since your equation is time-independent, what you seem to have are a sequence of decoupled problems (one for each instant in time).  If I am wrong and there is some coupling between them, you should clarify your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any method for solving parabolic problems on arbitrary domains will work fine.  The most well-known are standard finite element methods.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-04-16T11:38:44.157" LastActivityDate="2015-04-16T11:38:44.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19370" PostTypeId="2" ParentId="19356" CreationDate="2015-04-15T08:20:03.960" Score="5" Body="&lt;p&gt;This may not be your definitive solution, but what you need in any case is to change the discretization method that replace the continuous Eikonal equation with some discrete numerical scheme on the grid. The classical methods like fast marching method etc. are based mostly on the Rouy-Tourin scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the work of Gibou and Min you can find the discretization methods on quadtree or octree data structures for level set equations, my quick search gave me:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.lsa.umich.edu/~psmereka/LEVELSET/LSPAPERS/min_gibou.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.math.lsa.umich.edu/~psmereka/LEVELSET/LSPAPERS/min_gibou.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that paper they solve the Eikonal equation for so called reinitialization purpose, but they do not use any kind of fast method. May be you can learn how to obtain the numerical scheme on your grid and then the fast sweeping method could be applied. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least this may be a direction where to look for a solution of your problem.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2015-04-15T08:20:03.960" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19371" PostTypeId="1" AcceptedAnswerId="19374" CreationDate="2015-04-15T09:04:23.773" Score="0" ViewCount="108" Body="&lt;p&gt;$$ \frac{\partial|\nabla\phi|^2}{\partial\phi}=-2\nabla\cdot\nabla\phi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would very appreciate that you help me . Please do it in detail, I am quite not good at such problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is something wrong with this equation, and it is the primary condition below :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F = \int\limits_\Omega (f_\text{gr} + f_\text{loc} + f_\text{el})\, dV$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then it gives&#xA;$$\psi = \frac{\delta F}{\delta \phi} = -\nabla\cdot \lambda\nabla\phi + \frac{\lambda}{\epsilon^2}(\phi^2 - 1)\phi + \frac{\partial f_\text{el}}{\partial\phi}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so I cut it short (wrongly ) as given before.&lt;/p&gt;&#xA;" OwnerUserId="15099" LastEditorUserId="9667" LastEditDate="2015-04-15T19:08:06.543" LastActivityDate="2015-04-15T19:08:06.543" Title="Explain this multivariate differential identity" Tags="|pde|numerical-analysis|tensor|tensor-decomposition|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19373" PostTypeId="2" ParentId="19371" CreationDate="2015-04-15T09:49:37.360" Score="-3" Body="&lt;p&gt;I guess is not like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial}{\partial \phi}\left (\frac{\partial  \phi}{\partial x_i}\frac{\partial  \phi}{\partial x_i}\right) =-2 &#xA;\frac{\partial  }{\partial x_i}\frac{\partial \phi }{\partial x_i}$&lt;/p&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="14659" LastEditDate="2015-04-15T12:25:42.283" LastActivityDate="2015-04-15T12:25:42.283" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19374" PostTypeId="2" ParentId="19371" CreationDate="2015-04-15T11:35:50.897" Score="9" Body="&lt;p&gt;The equation is not correct as stated, but needs to be interpreted in the variational sense when multiplied by a test function. To this end, consider the operator&#xA;$$&#xA;  L(\phi) = |\nabla \phi|^2.&#xA;$$&#xA;Here, $\phi=\phi(x)$ is a function. Now, imagine what its derivative would be. Derivatives of operators are most commonly considered &lt;em&gt;in certain directions&lt;/em&gt; $v(x)$. So we could denote &#xA;$$&#xA;  L'(\phi)(v) = \lim_{\varepsilon\rightarrow 0} \frac{L(\phi+\varepsilon v)-L(\phi)}{\varepsilon}.&#xA;$$&#xA;If you go through the motions with the limit, you will find that&#xA;$$&#xA;  L'(\phi)(v) = 2 \nabla \phi \cdot \nabla v.&#xA;$$&#xA;If you wanted to, you could then define the derivative of the operator, $L'(\phi)$ (or, in your notation, $\frac{\partial}{\partial\phi}L(\phi)$ as&#xA;$$&#xA;  L'(\phi) = 2 \nabla \phi \cdot \nabla&#xA;$$&#xA;which is an operator that acts on whatever it is applied to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can extend this. Assume that the functions $\phi$ you consider live on a domain $\Omega\subset {\mathbb R}^d$ and that $\phi|_{\partial\Omega}=0$. (In other words, you assume that $\phi \in H^1_0(\Omega)$.) If you define a &lt;em&gt;functional&lt;/em&gt;&#xA;$$&#xA;  J(\phi) = \int_\Omega L(\phi) = \int_\Omega |\nabla \phi|^2 &#xA;$$&#xA;that associates a &lt;em&gt;number&lt;/em&gt; to every function $\phi$, then you can again ask what the derivative of $J$ is in a direction $v$. You get&#xA;$$&#xA;  J'(\phi)(v) = \int_\Omega L'(\phi)(v) = \int_\Omega 2\nabla \phi \cdot \nabla v.&#xA;$$&#xA;You can integrate the last formula by parts, using that both $\phi$ and $v$ are zero on the boundary, to get&#xA;$$&#xA;  J'(\phi)(v) = \int_\Omega\left[ -2\nabla \cdot \nabla \phi\right] v.&#xA;$$&#xA;It is in this context that the term you cited in the question appears. But the formula you cite is, by itself, wrong.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-15T11:35:50.897" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19375" PostTypeId="1" CreationDate="2015-04-15T12:16:40.887" Score="3" ViewCount="241" Body="&lt;p&gt;I have a nicely working multigrid solver, which I use for solving the Poisson equation from an electrostatic problem. I solve this equation first without any charges, and then many times with a slowly changing charge distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, I do a FMG from scratch each time I solve this equation, ignoring the fact that I already have a previous solution which could be updated instead of starting new from scratch. But if I use a V- or a W-cycle for this, I fear that there might be situations (if the charge distribution changed significantly) where this is worse than just starting from scratch with FMG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could try to apply FMG to the difference between my old solution and the final solution. However, this might be wasteful (in terms of memory), because I would have to store both the old solution, the real right hand side, the right hand side for the difference, and the difference on the finest grid. Hence I wonder whether using the F-cycle instead wouldn't be just as good, or even equivalent to this procedure.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2016-01-17T21:19:15.827" Title="Can F-cycle substitue FMG for update of existent solution?" Tags="|multigrid|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19376" PostTypeId="1" AcceptedAnswerId="19380" CreationDate="2015-04-15T13:56:35.100" Score="2" ViewCount="243" Body="&lt;p&gt;Is there an iterative sparse parallel linear equation solver with &lt;a href=&quot;http://en.wikipedia.org/wiki/Out-of-core_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;out-of-core capabilities&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I need to solve a very large system of equations. I have implemented direct sparse parallel solvers in-core and out-of-core and now I have to try an iterative one.&lt;/p&gt;&#xA;&lt;p&gt;I mainly use FORTRAN for the implementation.&lt;/p&gt;&#xA;" OwnerUserId="15018" LastEditorUserId="20688" LastEditDate="2021-12-23T02:48:51.590" LastActivityDate="2021-12-23T02:48:51.590" Title="Solving sparse linear equations with an iterative, out-of-core algorithm" Tags="|linear-solver|sparse-matrix|iterative-method|fortran|memory-management|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19377" PostTypeId="1" AcceptedAnswerId="19378" CreationDate="2015-04-15T21:46:41.343" Score="4" ViewCount="76" Body="&lt;p&gt;Let's say we have an unorganized point cloud P1 with N points, each with coordinates {x,y,z}. We apply non-rigid transformation to P1 (translation + rotation + warping), to obtain point cloud P2. Given a pair of corresponding points in P1 and P2, how can we efficiently estimate the change in local point density around this point without performing explicit meshing? In other words, how to estimate the local space compression/expansion around each point after the transformation? Consider that the expansion/compression may not be isotropic.&lt;/p&gt;&#xA;" OwnerUserId="15105" LastEditorUserId="7733" LastEditDate="2015-04-16T02:25:45.983" LastActivityDate="2015-04-16T13:46:25.753" Title="Estimating the local compression/expansion ratio for a transformation on a point cloud" Tags="|linear-algebra|finite-element|computational-geometry|finite-volume|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19378" PostTypeId="2" ParentId="19377" CreationDate="2015-04-16T02:07:52.917" Score="5" Body="&lt;p&gt;If you wish to know only the change in density you are interested in the &lt;a href=&quot;http://en.wikipedia.org/wiki/Infinitesimal_strain_theory#Volumetric_strain&quot; rel=&quot;nofollow&quot;&gt;dilatation or volumetric strain&lt;/a&gt; of the stress tensor associated with the transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you also wish to know about the anisotropy of the stretching you should look at computing the &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_strain_theory#The_Right_Cauchy-Green_deformation_tensor&quot; rel=&quot;nofollow&quot;&gt;right Cauchy-Green deformation tensor, $\Delta$,&lt;/a&gt; and it's associated eigenvalues and eigenvectors. The eigenvectors of this tensor define the primary axes of deformation and the eigenvalues are equal to the squares of the stretching ratios along these axes. The density ratio is given by the square root of the determinant of $\Delta$ which equals the square root of the eigenvalues: $\rho_2/\rho_1 = \sqrt{\det(\Delta)}= \sqrt{\lambda_1 \lambda_2 \lambda_3}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do these computations you will need to compute (or at least estimate) the derivatives of your transformation. If you can do this analytically then the problem is easy. If not, the simplest way to compute the relevant tensors for these quantities is to perform the transformation on 6 additional points in a stencil around each point in your cloud and then use those to compute a finite difference approximation of the terms in the stress/deformation tensor. This &lt;a href=&quot;http://mmae.iit.edu/shadden/LCS-tutorial/computation.html&quot; rel=&quot;nofollow&quot;&gt;is often done&lt;/a&gt; to compute the Cauchy-Green deformation tensor for computing &lt;a href=&quot;http://en.wikipedia.org/wiki/Lagrangian_coherent_structure&quot; rel=&quot;nofollow&quot;&gt;Lagrangian coherent structures (LCS)&lt;/a&gt; for fluid flow analysis. If central differences are used this method is second order accurate in the stencil size, but quickly loses accuracy if the deformations are large enough that the transformation is not approximately linear on the scale of the stencil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are not able to perform the transformation on arbitrary points you can use the points you know to estimate the deformation tensor. The simplest method involves choosing N neighbors of a given point and using those to estimate the deformation tensor. This can be formulated as a least-squares problem to determine the components of a 3x3 transformation matrix, $\Phi$, such that $\Phi~(\text{P}1_i-\text{P}1_0) = (\text{P}2_i-\text{P}2_0)$ where subscript $0$ denotes the point of interest and $i$ denotes the neighbors. In 3D you will need a minimum of 3 neighbors to determine $\Phi$ (3 points x 3 components {x,y,z} gives 9 linear equations for the 9 entries in $\Phi$). The deformation tensor is then given by $\Delta = \Phi^\intercal \Phi$. This method is first order accurate in the point spacing and quickly loses accuracy when the points are far enough apart so that the transformation is not locally linear on the neighborhood being used. If my memory is correct, this is the method that is used for computing the deformation tensor to find LCS on unstructured meshes in &lt;a href=&quot;http://scitation.aip.org/content/aip/journal/chaos/20/1/10.1063/1.3278516&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. As a side note, this method is provably equivalent to using centered finite differences (and recovers second order accuracy) if the you use 6 neighbors at the locations of the points in the finite difference stencil.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-04-16T13:46:25.753" LastActivityDate="2015-04-16T13:46:25.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19380" PostTypeId="2" ParentId="19376" CreationDate="2015-04-16T10:31:17.343" Score="5" Body="&lt;p&gt;For a sparse parallel solver, it's your own responsibility to provide a matrix vector product and a suitable preconditioner. The data for the vector itself should fit into main memory in any case. If the matrix has at most a fixed small number of non-zero elements (&amp;lt;20) per column or row, then the same is also true for the matrix itself. In this case, an incomplete LU preconditioner would not need out-of-core capabilities either, and otherwise it would be quite slow anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, I don't think that the iterative solver itself needs to take care of out of core capabilities, even for problems that need such capabilities.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2015-04-16T10:31:17.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19381" PostTypeId="2" ParentId="19306" CreationDate="2015-04-16T11:52:04.697" Score="1" Body="&lt;p&gt;Just to add a bit to @Jesse Chan's answer: you can preserve 6th-order convergence if you use a 5th-order starting method; in general, the starting method can be one order lower than the multistep method without decreasing the global convergence rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common starting methods are, of course, Runge-Kutta methods.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-04-16T11:52:04.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19382" PostTypeId="1" CreationDate="2015-04-16T12:12:57.780" Score="0" ViewCount="1876" Body="&lt;p&gt;I'm aware of a Matlab function which take a set of point as an input and select the points which compose the convex hull.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Exemples from Mathworks website :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xx = -1:.05:1;&#xA;yy = abs(sqrt(xx));&#xA;[x,y] = pol2cart(xx,yy);&#xA;k = convhull(x,y);&#xA;plot(x(k),y(k),'r-',x,y,'b*')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, my question is : does there exists a Matlab function or what could be the method to follow in order to compute the intersection of two convex sets defined by the set of points of their convex hull ?&lt;/p&gt;&#xA;" OwnerUserId="9749" LastActivityDate="2015-04-16T22:55:20.697" Title="Compute the intersection of convex sets in Matlab" Tags="|matlab|convex-hull|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19383" PostTypeId="1" CreationDate="2015-04-16T14:33:22.150" Score="2" ViewCount="131" Body="&lt;p&gt;This is a follow-up question to an answer I read &lt;a href=&quot;https://scicomp.stackexchange.com/a/18612/14251&quot;&gt;here&lt;/a&gt;. &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is some hermitian matrix and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; an vector.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Since the matrix is hermitian, you could use it as a hamiltonian to propagate it in imaginary time. That is, solve the following system of differential equations:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ i\frac{d \vec{V}}{dt}=M\vec{V}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The general solution to this is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ V(t)=V_0e^{iMt}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then you take your &lt;span class=&quot;math-container&quot;&gt;$\vec{V(t)} \cdot \vec{V(0)}$&lt;/span&gt;, fourier transform it, and the height and placement of the peaks will tell you the components along various eigenvectors and their associated eigenvalues. This is sometimes called &amp;quot;the spectral method&amp;quot; in ultrafast atomic physics.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I want to understand this method.&lt;/p&gt;&#xA;&lt;p&gt;My thoughts on an intuitive level so far: If I take the second equation as given, &lt;span class=&quot;math-container&quot;&gt;$V(t)$&lt;/span&gt; is a vector-valued process for &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. The scalar product &lt;span class=&quot;math-container&quot;&gt;$\left&amp;lt;V_{0}e^{iMt},V_{0}\right&amp;gt;$&lt;/span&gt; encodes the matrix &lt;span class=&quot;math-container&quot;&gt;$e^{iMt}$&lt;/span&gt; in a scalar-valued object. The fourier transform for a scalar &lt;span class=&quot;math-container&quot;&gt;$e^{imt}$&lt;/span&gt; is the delta distribution &lt;span class=&quot;math-container&quot;&gt;$\delta_{m}$&lt;/span&gt;, up to possibly rescaling by &lt;span class=&quot;math-container&quot;&gt;$2\pi$&lt;/span&gt;. That is, the fourier transform decodes information of test functions &lt;span class=&quot;math-container&quot;&gt;$m\mapsto e^{imt}$&lt;/span&gt;. So the foruier transform of &lt;span class=&quot;math-container&quot;&gt;$\left&amp;lt;V_{0}e^{iMt},V_{0}\right&amp;gt;$&lt;/span&gt; might somehow decode information about the hermitian matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is this intuition right so far? If so, any help on how to make this rigorous and how to understand what is going on? A small explicit example would be very helpful.&lt;/p&gt;&#xA;&lt;p&gt;Any source such as a textbook, lecture notes etc. is most welcome.&lt;/p&gt;&#xA;" OwnerUserId="14251" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-04-16T14:33:22.150" Title="Help understanding the so-called &quot;spectral method&quot;" Tags="|linear-algebra|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19384" PostTypeId="1" CreationDate="2015-04-16T18:27:37.080" Score="3" ViewCount="6319" Body="&lt;p&gt;I would like to know the algorithm to calculate contour line length.&#xA;Suppose we have numerical data set of an function $f(x,y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I calculate the length of the line from $(x_1,y_1)$ to $(x_2,y_2)$ on the contour $f = f_0$?&#xA;I am working on this with Fortran, but the example with any other language will be OK.&lt;/p&gt;&#xA;" OwnerUserId="15112" LastEditorUserId="9667" LastEditDate="2015-04-16T19:47:33.363" LastActivityDate="2015-04-22T08:10:56.440" Title="Calculate contour line length" Tags="|algorithms|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19385" PostTypeId="2" ParentId="19384" CreationDate="2015-04-16T19:44:25.947" Score="3" Body="&lt;p&gt;As Doug Lipinski mentioned you need to:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1) find the contour&lt;br&gt;&#xA;  2) compute the arc length.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h2&gt;Find the contour&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;To find the contour you can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Marching_squares&quot; rel=&quot;nofollow&quot;&gt;Marching Squares&lt;/a&gt;, where you rewrite&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x,y) = f_0 \Leftrightarrow f(x,y) - f_0 = 0 \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You assign a primitive to each individual cell (each 4 cells can be considered to be the corners of a bilinear interpolation, for example). Thus, you consider the sign of the values at vertices; intersections occur on &lt;strong&gt;edges with sign change&lt;/strong&gt;. There are 16 options for this, although just 4 topologically different&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No crossing: 4 edges with same sign&lt;/li&gt;&#xA;&lt;li&gt;Singlet: 1 edge with different sign&lt;/li&gt;&#xA;&lt;li&gt;Double adjacent&lt;/li&gt;&#xA;&lt;li&gt;Double opposite&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the &lt;em&gt;double opposite&lt;/em&gt; case might be some ambiguities and the sign of the saddle point should be computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a &lt;em&gt;last step&lt;/em&gt; you determine the position of intersection (using interpolation along grid edges).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Compute the arc length&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You want to perform the numeric integration to obtain the &lt;a href=&quot;http://en.wikipedia.org/wiki/Arc_length&quot; rel=&quot;nofollow&quot;&gt;arc length&lt;/a&gt;, i.e.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$s = \int_{a}^{b} \sqrt { [x'(t)]^2 + [y'(t)]^2 }\, dt. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $t$ is a parameter for $x(t)$ and $y(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your input is the numerical data for $x$ and $y$, you can directly proceed with the numerical integration. The simplest case is to &lt;em&gt;connect&lt;/em&gt; your points with lines and add all those segments&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$s \approx \sum_{k=1}^{N} \sqrt{ (x_{k} - x_{k-1})^2 + (y_{k} - y_{k-1})^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example in Python&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;def arc_length(x, y):&#xA;    npts = len(x)&#xA;    arc = np.sqrt((x[1] - x[0])**2 + (y[1] - y[0])**2)&#xA;    for k in range(1, npts):&#xA;        arc = arc + np.sqrt((x[k] - x[k-1])**2 + (y[k] - y[k-1])**2)&#xA;&#xA;    return arc&#xA;&#xA;&#xA;# Parabolic segment&#xA;npts = 1000&#xA;a = 3.&#xA;h = 5.&#xA;x = np.linspace(-a, a, npts)&#xA;y = h*(1 - x**2/a**2)&#xA;analytic = np.sqrt(a**2 + 4*h**2) + a**2/(2*h)*np.arcsinh(2*h/a)&#xA;numeric = arc_length(x, y)&#xA;print analytic, numeric&#xA;&#xA;# Semicircle&#xA;npts = 1000&#xA;x = np.linspace(-1, 1, npts)&#xA;y = np.sqrt(1 - x**2)&#xA;analytic = np.pi&#xA;numeric = arc_length(x, y)&#xA;print analytic, numeric&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;12.1673133338 12.1881924551&#xA;3.14159265359 3.20484351644&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can use higher order &lt;em&gt;derivatives&lt;/em&gt; (interpolations or B-Splines, as suggested by Model_Math) for more accurate results, but the concept is the same.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-04-21T20:31:37.137" LastActivityDate="2015-04-21T20:31:37.137" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19386" PostTypeId="2" ParentId="19382" CreationDate="2015-04-16T20:48:49.800" Score="1" Body="&lt;p&gt;Let the $x$-coordinates for the points in the convex hull of the first convex region be stored in the array &lt;code&gt;x1&lt;/code&gt;, and let the $y$-coordinates of those same points be stored in the array &lt;code&gt;y1&lt;/code&gt;. Let the arrays &lt;code&gt;x2&lt;/code&gt; and &lt;code&gt;y2&lt;/code&gt; store analogous information for the second convex region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have MATLAB's Mapping Toolbox, you can use the &lt;a href=&quot;http://www.mathworks.com/help/map/ref/polybool.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;polybool&lt;/code&gt; function&lt;/a&gt; to return the coordinates of a polygonal approximation of the intersection of your two regions via &lt;code&gt;[x, y] = polybool('intersection', x1, y1, x2, y2)&lt;/code&gt;. I'm not familiar with the algorithm behind the function, so I can't comment on that.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-16T20:48:49.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19387" PostTypeId="2" ParentId="19382" CreationDate="2015-04-16T22:55:20.697" Score="0" Body="&lt;p&gt;There is a &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/18173-polygon-intersection&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;polygon_intersection&lt;/code&gt;&lt;/a&gt; script on the Matlab file exchange which will compute the intersection of one or more polygons. In your case the convex hulls of your two sets are the polygons you wish to intersect.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-04-16T22:55:20.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19388" PostTypeId="2" ParentId="304" CreationDate="2015-04-17T01:53:27.687" Score="4" Body="&lt;p&gt;Sorry for digging up an old thread but it seems that even in 2015, Fortran is being used a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just came across &lt;a href=&quot;http://phys.org/wire-news/190569248/olcf-selects-application-readiness-projects-to-prepare-for-summi.html&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; (alternate &lt;a href=&quot;http://www.theregister.co.uk/2015/04/15/summit_projects/&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;) list which basically is a list of 13 codes approved by DOE's OCLF facility to run on the 300-petaFLOPS Summit machine which will be made available to researchers in 2018. I tried to find the main language used for the code (based on a quick google search) and here is what I found:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;XGC Fortran&#xA;&#xA;SPECFEM Fortran&#xA;&#xA;ACME Fortran (Bunch of climate codes)&#xA;&#xA;DIRAC Fortran (Mostly)&#xA;&#xA;FLASH Fortran&#xA;&#xA;GTC Fortran&#xA;&#xA;HACC C/C++&#xA;&#xA;LS-DALTON Fortran (some C)&#xA;&#xA;NAMD C/C++&#xA;&#xA;NUCCOR Fortran&#xA;&#xA;NWCHEM Fortran&#xA;&#xA;QMCPACK C++&#xA;&#xA;RAPTOR Fortran&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So out of 13 codes, at least 10 (based on my quick search) appear to be written in Fortran. Not bad for a 50 year old language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: I am well aware that language comparisons are useless but given the number of folks (specially C++ users) that bad-mouth Fortran, I thought it might be worthwhile to mention it.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2016-02-07T16:06:53.033" LastActivityDate="2016-02-07T16:06:53.033" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19389" PostTypeId="1" AcceptedAnswerId="19391" CreationDate="2015-04-17T05:48:59.613" Score="4" ViewCount="779" Body="&lt;p&gt;There are so many Runge-Kutta methods, including &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Dormand%E2%80%93Prince_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dormand-Prince&lt;/a&gt; 45, &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Cash%E2%80%93Karp_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cash-Karp&lt;/a&gt; 54, &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Runge%E2%80%93Kutta%E2%80%93Fehlberg_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fehlberge&lt;/a&gt; 78.&lt;/p&gt;&#xA;&lt;p&gt;Is there any comparison between them? E.g.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What is each approach sacrificing?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the general trade-off in different RK methods?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Which approach is suitable for what model?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How does Cash-Karp Method perform when we have a non-smooth solution?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How does Dormand-Prince Method when we have a non-smooth solution?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15113" LastEditorUserId="20688" LastEditDate="2021-12-28T22:28:03.697" LastActivityDate="2021-12-28T22:28:03.697" Title="What is the case of trade-off in different Runge-Kutta methods" Tags="|ode|differential-equations|runge-kutta|stiffness|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19390" PostTypeId="1" CreationDate="2015-04-17T09:08:03.840" Score="0" ViewCount="968" Body="&lt;p&gt;I want to solve a 1-D heat conduction PDE using Matlab which looks like &#xA;$$&#xA;\rho c_p \dfrac{\partial T}{\partial t} = \dfrac{\partial}{\partial z}\left( \lambda \dfrac{\partial T}{\partial z} \right),&#xA;$$&#xA;where the initial condition is T1 (in Kelvin) and the final condition is T2 (also in Kelvin), the difference being T1 takes different values from t=0 hours to t=24 hours (t being time) and T2 being constant (say 200 K)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked the &lt;em&gt;pdepe&lt;/em&gt; toolbox in Matlab and I felt the conditions that I wanted to implement might not be possible through the toolbox.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest a way to create a loop through which T1 can read values from an array so that a diurnal plot can be created ?&lt;/p&gt;&#xA;" OwnerUserId="15114" LastEditorUserId="7733" LastEditDate="2015-04-17T16:10:10.220" LastActivityDate="2015-04-17T16:10:10.220" Title="Solving a PDE using Matlab (with varying initial conditions)" Tags="|matlab|pde|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19391" PostTypeId="2" ParentId="19389" CreationDate="2015-04-17T10:25:25.760" Score="7" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;There are so many Runge Kutta methods, including&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Dormand-Prince 45&lt;/li&gt;&#xA;  &lt;li&gt;Cash-Karp 54&lt;/li&gt;&#xA;  &lt;li&gt;Fehlberge (sic) 78&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;Is there any comparison between them?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Well, sure. Here are some traits to compare:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the method implicit or explicit? (All of your examples are explicit RK methods.)&lt;/li&gt;&#xA;&lt;li&gt;What is the order of convergence? Are there any embedded error estimators? How many, and of what order(s)?&lt;/li&gt;&#xA;&lt;li&gt;How many stages does it have?&lt;/li&gt;&#xA;&lt;li&gt;How many function evaluations are required? For implicit methods, how many linear solves are involved?&lt;/li&gt;&#xA;&lt;li&gt;What is the stability region? Is it A-stable? L-stable?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My relatively simplistic understanding of these methods is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Dormand-Prince 4(5) is a 4th-order method with a 5th-order error estimator designed to minimize the error in the 5th-order solution, which is preferable when using the 5th-order solution to continue the integration&lt;/li&gt;&#xA;&lt;li&gt;Fehlberg methods are supposed to minimize the error in the lower-order solution (in your case, the 7th-order solution); in general, Dormand-Prince methods of the same order should perform better&lt;/li&gt;&#xA;&lt;li&gt;I've heard of the Cash-Karp discretization, but I haven't across a source that claims any sort of advantage or disadvantage compared with the Dormand-Prince methods&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is each approach sacrificing?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;What is the general trade-off in different RK methods?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You should look at (or construct!) work-precision diagrams for the problems and integrators you're interested in investigating. Hairer and Wanner have some examples.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Which approach is suitable for what model?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Again, you should look at work-precision diagrams. To a very coarse approximation, you want explicit Runge-Kutta methods for non-stiff problems (assuming, that is, you want to use a Runge-Kutta method), and for stiff problems, you're likely to want an implicit Runge-Kutta method. Methods with a higher order of accuracy per time step are likely to perform better at stricter error tolerances than methods with a lower order of accuracy. However, the best approach is (very) problem-dependent.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-17T10:25:25.760" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19392" PostTypeId="2" ParentId="19390" CreationDate="2015-04-17T11:27:27.340" Score="5" Body="&lt;p&gt;this problem can be solved using the &lt;a href=&quot;http://fr.mathworks.com/help/matlab/ref/pdepe.html&quot;&gt;&lt;code&gt;pdepe&lt;/code&gt;&lt;/a&gt; solver in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand well, T1 and T2 represent your left and right boundary conditions for the spatial domain :&#xA;$$&#xA;T(z=0,\,t) = T_1( t ) \\&#xA;T(z=L,\,t) = Const. = T_2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Initial' and 'final' terms are usualy related to the time domaine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial (i.e. at time t=0) temperature field could be, for instance, uniform in $z$: $T(z,\, t=0 ) = Const. $ (specified using the &lt;code&gt;icfun&lt;/code&gt; function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And there is no final condition in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Spatial boundary conditions are specified by the &lt;code&gt;bcfun&lt;/code&gt; function, which return values of the $p$ and $q$ coefficients for the left and right boundaries (see eq. 1-6 in the matlab doc). &#xA;These coefficients can be function of time. In your case it will be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [pl, ql, pr, qr] = bcfun(xl, ul, xr, ur, t)&#xA;    pl = T1( t );&#xA;    pr = T2;&#xA;    ql = 0;&#xA;    qr = 0;&#xA;&#xA;function T = T1( t )&#xA;    T = ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15117" LastActivityDate="2015-04-17T11:27:27.340" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19395" PostTypeId="1" AcceptedAnswerId="19399" CreationDate="2015-04-18T15:38:50.283" Score="3" ViewCount="297" Body="&lt;p&gt;I have developed a pseudospectral solver of the Navier-Stokes equations using FFTW. I tested my formulation of right hand sides (RHS) of the NS equations against standard trigonometric functions (sines, cosines and their combinations). For example, I set &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;density = sin 5x&#xA;x_velocity = 5cos 5y + 6sin7z&#xA;y_velocity = 4sin4y + cos x&#xA;z_velocity = 1&#xA;pressure = cos z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Supplying these values to the solver, it computed the RHS of the NS equations. I did the same by hand and compared the results with that obtained by the solver. Results were to good agreement. The maximum error between the exact answer and that computed by the solver was of the order of &lt;code&gt;E-13&lt;/code&gt; for a &lt;code&gt;128*128*128 grid.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next I used a different function of the following form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;density = constant1+constant2*(tanh(x-constant3)-tanh(x-constant4))&#xA;x_velocity = 0&#xA;y_velocity = 0&#xA;z_velocity = 0&#xA;temperature = 1&#xA;pressure -&amp;gt; from ideal gas equation connecting density, temperature and pressure&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The density was adjusted suitably based on the constants, to have a period of &lt;code&gt;2*pi&lt;/code&gt;. On calculating the RHS of the x-momentum Navier Stokes based on these values given and comparing it with my answer (calculated by hand), I obtained a maximum error of the order of &lt;code&gt;E-03&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, using these values as initial values of the variables and moving forward in time by a Runge-Kutta 4 scheme, I get values of the density that seem to diverge very quickly. After about 30 time steps, I get NaNs.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a specific reason why I notice a decrease in precision when non trigonometric periodic functions are used ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is 1. related to why my code seems to produce unstable results when marching forward in time ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I wouldn't mind pasting the code here but it's pretty large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought I would plot the initial density and its variation. But turns out I can't as I do not have enough reputation to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial plot (@t = 0.0s) is a density plot that looks like a rectangular wave with the tanh functions used to smoothen the wave at the various corners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At around t = 0.10s (the time step is 0.01s so, after 10 iterations), it develops spikes and becomes non-differentiable (still continuous). &lt;/p&gt;&#xA;" OwnerUserId="15126" LastActivityDate="2015-04-18T18:51:13.277" Title="precision loss in non-trigonometric, periodic functions using FFTW and NaNs after marching forward in time (Fortran)" Tags="|fortran|navier-stokes|precision|fourier-transform|fftw|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19396" PostTypeId="1" CreationDate="2015-04-18T17:17:56.423" Score="2" ViewCount="126" Body="&lt;p&gt;With the advent of &quot;big data&quot; applications, different algorithms have to be used to efficiently solve optimization problems, even in the convex case (e.g. the recent success of stochastic gradient descent). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I was wondering what happens in cases where you have hard MILP instances with millions ($10^6$-$10^9$) of variables and a similar number of integer constraints (say $10^5$-$10^8$). Also assume that these problems have to be solved really quickly, in a few seconds or less, but a very good approximative answer suffices. In such cases, a standard branch-and-bound algorithm is bound to fail (setting a time limit on standard MILP solvers will usually produce a bad solution or no solution at all).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any resources for dealing with such kind of problems (in the MILP context)? I am looking both for textbooks or for journal articles which describe methods to tackle such problems and especially cases where the large scale of the problem actually increases the power of these methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of articles I am looking for, are the following ones by Robin Vujanic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6961472&quot; rel=&quot;nofollow&quot;&gt;Vujanic, Robin, et al. &quot;Large scale mixed-integer optimization: A solution method with supply chain applications.&quot; Control and Automation (MED), 2014 22nd Mediterranean Conference of. IEEE, 2014.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://control.ee.ethz.ch/~peymanm/drafts/MathProg.pdf&quot; rel=&quot;nofollow&quot;&gt;Vujanic, Robin, et al. &quot;Vanishing duality gap in large scale mixed-integer optimization: a solution method with power system applications.&quot; submitted to Journal of Mathematical Programming (2014).&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11508" LastActivityDate="2015-04-18T17:17:56.423" Title="Resources for large-scale MILP optimization" Tags="|reference-request|linear-programming|mixed-integer-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19397" PostTypeId="1" CreationDate="2015-04-18T17:20:54.833" Score="2" ViewCount="1816" Body="&lt;p&gt;Which method should I choose to solve a large (~20 000 variables) dense symmetric positive-definite, possibly ill-conditioned, system of linear equations?&#xA;The system will be solved for two vectors. I'm interested both in performance and numerical accuracy, so I'm looking for possible solutions to choose the best compromise.&#xA;I want to use it as a part of a special purpose SDP solver under OCTAVE/MATLAB, so most probably it will be implemented as a mex file, with a single thread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Thank you for your responses.&#xA;I mean an optimization solver, with primal-dual interior point method. Dual solution is always feasible in my case, and dense (I mean the dual is not block-diagonal).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2:&#xA;The reason why I'm writing an SDP solver is that I want to try to get a more efficient tool for solving large NPA problems (&lt;a href=&quot;http://arxiv.org/abs/0803.4290&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/abs/0803.4290&lt;/a&gt;). This case allows for some simplifications, the interior is always non-empty, I can find a good initial dual feasible solution (and thus keep this feasibility), and the structure allows for easier creation of the equation for dy. The problem is that this equation is large for these problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use SeDuMi and SDPT3, with the latter being able to run much larger programs. For large I got with SDPT3 the error saying &quot;Schur complement matrix not positive definite&quot; meaning that Cholesky factorization failed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT3:&#xA;I observe that with this warm start it takes about 3-4 iterations less to solve the program. The problem is the failure of Cholesky for the large Schur complement matrix. In this case it is fully dense, but near the last iteration it becomes very bad conditioned. This is why I'm looking for a method which will be able to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT4:&#xA;Thank you for your response. I'll try pivoting with Cholesky. Do you recomend any package which works under OCTAVE for this task?&lt;/p&gt;&#xA;" OwnerUserId="15127" LastEditorUserId="15128" LastEditDate="2015-04-24T16:15:08.513" LastActivityDate="2015-04-24T16:15:08.513" Title="Solve large dense positive-definite linear system" Tags="|linear-solver|dense-matrix|semidefinite-programming|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19399" PostTypeId="2" ParentId="19395" CreationDate="2015-04-18T18:26:17.710" Score="6" Body="&lt;p&gt;There are three issues that are likely to cause such problems in pseudospectral methods:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Gibbs oscillations&lt;/li&gt;&#xA;&lt;li&gt;Aliasing&lt;/li&gt;&#xA;&lt;li&gt;Time step too large&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In any case you likely develop oscillations in the solution until some point ends up with a negative density, resulting in a NaN when computing the pressure or sound speed or some other term. The solution to 3 is obvious, decrease the time step until the time integration is stable. The other two are more nuanced.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Gibbs oscillations&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Gibbs_phenomenon&quot; rel=&quot;noreferrer&quot;&gt;Gibbs oscillations&lt;/a&gt; arise when computing the Fourier series of discontinuous functions. Gibbs oscillations arise in the derivatives if the function is non-smooth. If you have large jumps in your initial conditions then the Fourier series will match the values at grid points exactly, but the derivative will have large oscillations, leading to loss of precision in the derivative (right hand side) computations. See the image below for a demonstration of this, the values match but the derivative does not. As a rule of thumb, jumps must be smoothed out over about 10 grid to prevent this behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7tLX4.png&quot; alt=&quot;Image of Gibbs oscillations&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if your initial conditions are smooth on the scale of the grid, the state variables may quickly steepen. In compressible Navier-Stokes, the viscous terms act to prevent shocks from forming, but if your simulation is not sufficiently resolved you will still develop jumps in your simulation. Sufficiently resolved means having grid spacing small enough to capture the viscous dissipation, which can be estimated by looking at the Kolmogorov scale, see this &lt;a href=&quot;http://www.eng.utah.edu/~mcmurtry/Turbulence/turblt.pdf&quot; rel=&quot;noreferrer&quot;&gt;PDF&lt;/a&gt;. This quickly leads to large, non-physical oscillations and a divergent solution.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Aliasing&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Aliasing occurs in pseudospectral methods due to the presence of nonlinear terms (e.g. $u u_x$) in the evolution equation. Computation of the derivatives in spectral space assumes that you are resolving a certain number of wavelengths. However, nonlinear terms continuously generate higher and higher wavenumbers. In a discrete problem, these higher wavenumbers are &quot;aliased&quot; back to affect the lower wavenumbers that can actually be represented at the chosen resolution. This corrupts the lower wavenumber values and can quickly lead to oscillations, non-physical results, and the simulation blowing up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple demonstration of how nonlinear terms generate higher wavenumbers, and how those wavenumbers are aliased to lower wavenumbers is shown in the following scenario (see image below):&lt;br&gt;&#xA;Take a grid of 7 points on the interval [0,1). Let $a = \cos(6\pi x )$ and $b = \sin(6\pi x)$. These terms both have (angular) frequencies of $6\pi$. The term&#xA;$$ab = \cos(6\pi x )\sin(6\pi x ) = \frac{1}{2}\sin(12\pi x)$$&#xA;has frequency $12\pi$. This frequency cannot be resolved on the chosen grid, but the values of $ab$ on the grid are equal to the values of $-\sin(2\pi x)/2$ on the grid. So instead of a $12\pi$ frequency component (which is not captured in the discretized space), the result of $ab$ appears as a $2\pi$ frequency component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/elASK.png&quot; alt=&quot;Image demonstrating aliasing&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are multiple options available to prevent aliasing from corrupting your results, commonly termed dealiasing. The most common methods are zero padding (3/2-rule, effectively increasing grid resolution before nonlinear multiplications and then discarding the higher frequencies), 2/3-rule truncation (zeroing out the highest wavenumbers before nonlinear multiplications, &lt;a href=&quot;http://journals.ametsoc.org/doi/abs/10.1175/1520-0469(1971)028%3C1074%3AOTEOAI%3E2.0.CO%3B2&quot; rel=&quot;noreferrer&quot;&gt;original paper&lt;/a&gt;), or filtering procedures (e.g. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999107001623&quot; rel=&quot;noreferrer&quot;&gt;this method&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, there is evidence that for well resolved simulations dealiasing is not crucial since the highest wavenumbers components (which may produce aliasing) are small due to the viscous dissipation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://helper.ipam.ucla.edu/publications/mtws1/mtws1_12187.pdf&quot; rel=&quot;noreferrer&quot;&gt;This presentation (PDF)&lt;/a&gt; also provides a good overview of dealiasing, including some of the history.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-04-18T18:51:13.277" LastActivityDate="2015-04-18T18:51:13.277" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19400" PostTypeId="2" ParentId="19397" CreationDate="2015-04-18T18:32:14.253" Score="1" Body="&lt;p&gt;Provided the matrix is symmetric and positive-definite, the &lt;a href=&quot;http://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow&quot;&gt;conjugate gradient method&lt;/a&gt; is the provably fastest iterative method, requiring $O(\sqrt\kappa)$ iterations to converge. Depending on how badly-conditioned your matrix is, that may be more favorable than using a direct method like the &lt;a href=&quot;http://en.wikipedia.org/wiki/Cholesky_decomposition&quot; rel=&quot;nofollow&quot;&gt;Cholesky decomposition&lt;/a&gt; to solve your system. It is likely that you'll need some kind of preconditioner, the choice of which will depend on the nature of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 20,000 x 20,000 dense matrix is getting uncomfortably close to the limit of what can be addressed on a 32-bit machine. Storing that matrix alone will require 3.2 GB of memory. All that you need for many iterative methods is to be able to compute the matrix-vector product $A\cdot x$; for some problems, this can be done without explicitly creating the matrix $A$ entry-by-entry. If you were able to do that for your problem, I think it would speed things up dramatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Failing that, I think you'll need to resort to multiple threads and an optimized BLAS/LAPACK implementation like &lt;a href=&quot;http://math-atlas.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;ATLAS&lt;/a&gt; if you want good performance. You may also want to look into the library &lt;a href=&quot;http://libelemental.org/&quot; rel=&quot;nofollow&quot;&gt;Elemental&lt;/a&gt;, which does distributed dense linear algebra and has a C interface.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-04-18T18:32:14.253" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19401" PostTypeId="2" ParentId="19397" CreationDate="2015-04-18T18:35:13.577" Score="6" Body="&lt;p&gt;In double precision, a system of 20,000 equations in 20,000 variables will require 3.2 gigabytes of RAM to store the matrix.  This isn't &quot;big&quot; by contemporary standards, and is reasonably easy to solve in practice.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make this run fast, you'll want to use of a well tuned linear algebra library.  The standard for this kind of work would be to use LAPACK and BLAS routines, with a well tuned, multithreaded, implementation of the BLAS.  You'd use the Cholesky factorization routine in LAPACK to compute a Cholesky factorization and then use other LAPACK routines to do the forward and backward solves.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the MATLAB environment, your best bet is to piggy-back off of MATLAB.  The Mathworks supplies efficient BLAS/LAPACK routines with MATLAB and handles any licensing issues.  You can call these routines from within your MEX routine.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Octave you should hope that the installation of Octave came with a good multithreaded BLAS.  Although they aren't quite as good as Intel's MKL or AMD's ACML libraries (which are not open source), there are open source alternatives including Openblas and ATLAS.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're implementing a primal-dual interior point method for SDP, you should be aware that some special tricks are needed to make Cholesky factorization work well.  You should also be aware that in many cases, the system of equations actually isn't dense, and it can be worthwhile to use a sparse Cholesky factorization routine.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-04-19T14:48:29.643" LastActivityDate="2015-04-19T14:48:29.643" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19403" PostTypeId="1" CreationDate="2015-04-19T16:05:19.497" Score="6" ViewCount="1788" Body="&lt;p&gt;Suppose we have a triangular mesh of a two dimensional shape $\Omega$, and on this mesh we define a P1 finite element structure. I know that given $u,v$ by their values at the vertices of the triangles, we can compute $\int_\Omega \nabla u \cdot \nabla v$ using the stiffness matrix $A = (\int_\Omega \nabla \varphi_i\cdot \nabla\varphi_j)$ (where $\varphi_i$ is the P1 basis). This makes it possible to find expressions of the form $\int_\Omega |\nabla u|^2$ if $u$ is known at the triangle vertices.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can we compute the approximation of the gradient $\nabla u$, given the values of $u$ on the points?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I know that the gradient of $\varphi_i$ is constant on each triangle, but how do we need to combine these gradients for heighboring triangles, such that the value we obtain in the end is relevant.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The purpose of doing this: I would like to integrate a quantity of the form $\int_\Omega \varphi(\nabla u)^2$, where $\varphi$ is a norm, different from the euclidean one. Now that I think about it, I guess it is necessary to compute the matrix $A = \int_\Omega b(\nabla \varphi_i,\nabla \varphi_j)$, where $b$ is the scalar product associated to $\varphi$. &lt;/p&gt;&#xA;" OwnerUserId="15136" LastEditorUserId="15136" LastEditDate="2015-04-19T18:28:29.507" LastActivityDate="2015-04-19T20:28:08.090" Title="Find the direction of the gradient on a finite element mesh" Tags="|finite-element|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19405" PostTypeId="1" CreationDate="2015-04-19T20:03:38.523" Score="1" ViewCount="142" Body="&lt;p&gt;I have data which can be described by $y=f(x,z)$ where $z$ varies from 170 ~ 154. Now values given by $ks$ are known sample values that equals value given in the table header, $uks$ are unknown samples.&#xA;$$&#xA;\begin{array}&#xA;&amp;amp; z &amp;amp; x_{ks_{1}=0} &amp;amp; x_{ks_{2}=50} &amp;amp; x_{ks_{3} = 60} &amp;amp; \cdots &amp;amp; x_{uks_{1}}&amp;amp; \cdots &amp;amp; x_{uks_{n}}\newline&#xA;170 &amp;amp; 3.5 &amp;amp;  5 &amp;amp; 6 &amp;amp;\cdots &amp;amp; 5.3 &amp;amp; \cdots &amp;amp; 12 \newline&#xA;168 &amp;amp; 3.7 &amp;amp;  5.1 &amp;amp; 6.6 &amp;amp;\cdots &amp;amp; 5.6 &amp;amp; \cdots &amp;amp; 13.2 \newline&#xA;\vdots &amp;amp; \vdots &amp;amp;  \vdots &amp;amp; \vdots &amp;amp;\ddots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \newline&#xA;154 &amp;amp; 6.8 &amp;amp;  9.52 &amp;amp; 10 &amp;amp;\cdots &amp;amp; 11.5 &amp;amp; \cdots &amp;amp; 26.5 \newline&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotting values of $y$ vs. $x$ for various $z$, I get this plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZQtqR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and on a semi-log plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/0kdK9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the values on $y-axis$ are the values given by the subscripts of $x$ in the table headers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;**My objective is to find the value of $y$ for the corresponding $x_{uks_{1}}, x_{uks_{2}}, \cdots, x_{uks_{n}}$ for different $z$.**There might be alternate methods, such methods are most welcome. And my approach is as follow:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I try to fit a linear function of the form:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$y = \frac{A_{i}}{ln[x]^{3}} + B_{i},\quad \text{i for each z}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this gives me plots as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7UD0f.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Since $A_{i}$ and $B_{i}$ is different for $z$, I can  fit a function. Therefore $A_{i} = f(z)$ and $B_{i} = f(z)$. For step 1, I used the following function:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$A = a_1 z^3 + a_2 z^2 + a_3 z + a_4$$&#xA;$$B = b_1 z^2 + b_2 z + b_3$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Using the correlation:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$y = \frac{a_1 z^3 + a_2 z^2 + a_3 z + a_4}{ln[x]^{3}} + b_1 z^2 + b_2 z + b_3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And substituting different values of $x_{uks}$ I get corresponding $y$, but when I plot them vs. $z$, I get plots of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/RYcfq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my questions are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How am I getting $y$ to be a linear function of $z$?&lt;/li&gt;&#xA;&lt;li&gt;Is this approach a valid one?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14586" LastEditorUserId="14586" LastEditDate="2015-04-20T03:51:02.427" LastActivityDate="2015-04-20T03:51:02.427" Title="How to curve fit an unknown function?" Tags="|linear-algebra|statistics|plotting|curve-fitting|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19406" PostTypeId="2" ParentId="19403" CreationDate="2015-04-19T20:28:08.090" Score="5" Body="&lt;p&gt;Your approach with the matrix won't work unless the norm is quadratic. Here is the general approach:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Your solution is defined as $u_h(x) = \sum_j U_j \phi_j(x)$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want to compute $\int_\Omega \varphi(\nabla u_h(x)) dx$ (i.e., a number), then you need to replace the integral with a quadrature formula. If you use piecewise linear elements, then the one-point midpoint formula is sufficient, i.e., &#xA;$$&#xA;  \int_\Omega \varphi(\nabla u_h(x)) dx&#xA;  = \sum_K |K| \; \varphi(\nabla u_h(x_K))&#xA;$$&#xA;where $x_K$ is the midpoint (or, in fact, any point in the interior) of cell $K$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;So you only need to evaluate $\nabla u_h(x_K)$. Using the definition of $u_h$, you get that this equals&#xA;$$&#xA;  \nabla u_h(x_K) = \sum_j U_j \nabla \phi_j(x_K).&#xA;$$&#xA;On the other hand, because all shape functions are zero on cell $K$ except the ones defined at the three vertices of $K$, you have that in fact&#xA;$$&#xA;  \nabla u_h(x_K) = \sum_{j_K=1}^3 U_{j(j_K)} \nabla \phi_{j(j_K)}(x_K)&#xA;   = \sum_{j_K=1}^3 U_{j(j_K)} \nabla \phi^K_{j_K}(x_K).&#xA;$$&#xA;Here, $j(j_K)$ is the global index of the $j_K$-th shape function on cell $K$ where $j_K=1..3$, and $\phi^K_{j_K}$ are the three shape functions defined on cell $K$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, if you can evaluate the gradients of the three shape functions on cell $K$, then you can compute $\nabla u_h(x_K)$ and then $\varphi(\nabla u_h(x_K)$ and from there the whole integral.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-19T20:28:08.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19410" PostTypeId="1" AcceptedAnswerId="19418" CreationDate="2015-04-20T09:48:27.163" Score="0" ViewCount="570" Body="&lt;p&gt;Suppose a cuboid $\mathbb{A}$ has $L$,$M$ and $N$ as its length, width and height respectively, where $L\ge{M}\ge{N}&amp;gt;0$; Now we want to cut $\mathbb{A}$ into smaller cuboids with length $x$, width $y$ and height $z$ respectively, where $x\ge{y}\ge{z}&amp;gt;0$, $L\ge x$, $M\ge{y}$ and $N\ge{z}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to design an algorithm to obtain as many as possible smaller cuboids and give all the feasible cutting methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when $x=y=2,z=1$, and $L=M=N=3$ an optimal solution is: at most 6 small cuboids, and typical cutting mode:&lt;img src=&quot;https://i.stack.imgur.com/VILnP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2015-04-20T20:50:02.203" Title="Is there general algorithms to solve such 3D cutting problems?" Tags="|optimization|algorithms|nonlinear-programming|mixed-integer-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19411" PostTypeId="1" CreationDate="2015-04-20T09:55:03.713" Score="3" ViewCount="391" Body="&lt;p&gt;I am trying to solve simple scalar biharmonic equation using bubnov-galerkin finite element method. I am using $H^2$ conforming basis functions. I was wondering that if anyone can give me some pointers on how can I further debug my algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I proceeded by converting the strong form of equation to weak form:&#xA;\begin{align}&#xA;\Delta^2 u &amp;amp;= f \\&#xA;u &amp;amp;= g \;\;\; \text{on} \;\;\; \Gamma_D \\&#xA;\Gamma_N &amp;amp;= \emptyset&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example problem, I am trying to solve:&#xA;\begin{align}&#xA;u &amp;amp;= \cos(4 \pi x) \cos(4 \pi y)&#xA;\end{align}&#xA;Boundary conditions are implemented using penalty method. The weak form is follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;a(u,v) &amp;amp;= L(v) \\&#xA;a(u,v) &amp;amp;= \sum_{\Omega_K \in \mathcal T} \int_{\Omega_K} \Delta \psi_{i} \Delta \psi_{j} + \sum_{E \in \mathcal E} \gamma \int_{E} \psi_{i} \psi_{h,j} \\&#xA;L(v) &amp;amp;= \sum_{\Omega_K \in \mathcal T} \int_{\Omega_K} \psi_{i} f + \sum_{E \in \mathcal E} \gamma \int_{E} \psi_{i} g\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;: When I solve this equation on a square domain, my L2 error barely converges (convergence rate ~ 0.3). However, if I solve $u = \sin(4 \pi x) \sin(4 \pi y)$, I get correct convergence rates. I have tried the following things to debug my code:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I solves the poisson equation, so replaced the first integrand with the stiffness integrand. I get correct convergence rates. I concluded from this that my penalty method implementation is right&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am using Cartesian grid, so Jacobian lines up with expectations&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since, my shape functions are defined in parametric ($\xi,\eta$) space, I had to work out the transformation for the laplacian. I tested this transformation on polar coordinates.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Many thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apologies. Editted.&lt;/p&gt;&#xA;" OwnerUserId="4460" LastEditorUserId="1804" LastEditDate="2015-04-22T17:38:27.650" LastActivityDate="2015-04-22T17:38:27.650" Title="Boundary conditions in conforming Galerkin method for biharmonic equation" Tags="|pde|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19414" PostTypeId="1" CreationDate="2015-04-20T13:15:34.273" Score="2" ViewCount="1799" Body="&lt;p&gt;What methods are available to integrate a sharply peaked function (position of peak known) on a finite interval (the interval includes the peak)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am getting underflows using some of GSL's adaptive algorithms. I suspect that GSL fails to find the position of the peak, and hence is thinks that the function is mostly zero. Is there a method in GSL so that I can tell where the peak is located? Or maybe I can use an alternative routine (it doesn't have to be GSL)?&lt;/p&gt;&#xA;" OwnerUserId="988" LastActivityDate="2015-04-21T12:42:58.633" Title="Numerical integration of sharp peaked function (position of peak known)?" Tags="|integration|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19417" PostTypeId="2" ParentId="19414" CreationDate="2015-04-20T17:25:45.520" Score="8" Body="&lt;p&gt;If you know where the peak is, then you can always split the interval. For example, if you know that the peak is at $a$ and has a &quot;width&quot; (however you want to define that) of $\sigma$ so that you can say that it is mostly confined within $[a-\sigma,a+\sigma]$, then split the integral as&#xA;$$&#xA;  \int_l^u f(x) \; dx&#xA;=&#xA;  \int_l^{a-\sigma} f(x) \; dx&#xA;  +&#xA;  \int_{a-\sigma}^{a+\sigma} f(x) \; dx&#xA;  +&#xA;  \int_{a+\sigma}^u f(x) \; dx.&#xA;$$&#xA;Each of these three integrals should now be relatively well-behaved on their own, and should be easy enough to integrate.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2015-04-21T12:42:58.633" LastActivityDate="2015-04-21T12:42:58.633" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19418" PostTypeId="2" ParentId="19410" CreationDate="2015-04-20T20:50:02.203" Score="2" Body="&lt;p&gt;This problem looks like a variant of a &lt;a href=&quot;http://en.wikipedia.org/wiki/Cutting_stock_problem&quot; rel=&quot;nofollow&quot;&gt;cutting stock problem&lt;/a&gt;. The general idea is to set up an optimization problem that encodes the geometric constraints (in your case, the cuboid shapes) and the objective (maximizing the number of cuboids). In the paper industry, the classical example is the 1-D cutting stock problem, where you try to cut a roll of paper into various lengths to satisfy demands for various products while minimizing the paper wasted (because the remnants cannot be sold as product). The 1-D cutting stock problem is a mixed-integer linear program, and I would not be surprised if your instance were also mixed-integer.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-20T20:50:02.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19419" PostTypeId="1" AcceptedAnswerId="19426" CreationDate="2015-04-21T10:22:42.360" Score="4" ViewCount="1980" Body="&lt;p&gt;I have an isoparametric mapping $F_{E}: \hat{E} \to E$ where $\hat{E}$ is a reference quadrilateral (square) and $E$ is some quadrilateral in the domain $\Omega$. If $E$ is a parallelogram, then the mapping is affine and computing $F_{E}^{-1}$ is straightforward. But for the general case, we have:&#xA;$$&#xA;F_{E}(\hat{x},\hat{y}) = \left(\begin{array}{c}\sum_{i=1}^{4}x_{i}\hat{\psi}_{i}(\hat{x},\hat{y})\\ \sum_{i=1}^{4}y_{i}\hat{\psi}_{i}(\hat{x},\hat{y})\end{array}\right) = \left(\begin{array}{c}x\\ y\end{array}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\hat{\psi}_{i}$ are the basis functions on $\hat{E}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the discontinuous Galerkin method, I need to compute some integrals over the interfaces (2D) of the elements. A simple example would be the following:&#xA;$$&#xA;\int_{e}\langle\langle\nabla\psi_{i}\cdot\mathbf{n}\rangle\rangle[[\psi_{j}]]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\langle\langle\cdot\rangle\rangle$ and $[[\cdot]]$ denote the average and jump operators, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have that $\psi_{j}(x,y) = \hat{\psi}_{j}(F_{E}^{-1}(x,y)) = \hat{\psi}_{j}(\hat{x},\hat{y})$. Furthermore, on a given edge $e$, the point $(x,y) \in e$ maps to two values due to the discontinuities of the basis functions across the interfaces, i.e. we have $\psi_{j}^{-}$ and $\psi_{j}^{+}$ along $e$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute the above integral, I use a quadrature rule on the interval $\hat{e} = [-1,1]$. For a quadrature point $\hat{\zeta} \in \hat{e}$, I map this 1D point to $e$ and call it $\zeta$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to compute $\hat{\psi}_{j}^{-}(F_{E-}^{-1}(\zeta))$ and $\hat{\psi}_{j}^{+}(F_{E+}^{-1}(\zeta))$, where $F_{E-}$ and $F_{E+}$ are the mappings associated with the neighboring elements to $e$. For triangular elements and parallelograms, this is really straightforward since the mapping is affine and thus, computing $F_{E}^{-1}$ is easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for general quadrilaterals, is this even possible? I found &lt;a href=&quot;http://deepblue.lib.umich.edu/bitstream/handle/2027.42/28323/0000079.pdf?sequence=1&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; paper, but it's quite old and the process described seemed a bit complicated. Is there any other literature anyone can recommend on this? Or some insight into a simpler way to do this?&lt;/p&gt;&#xA;" OwnerUserId="6643" LastEditorUserId="6643" LastEditDate="2015-04-21T12:13:08.227" LastActivityDate="2015-04-21T13:47:24.097" Title="Inverse isoparametric mappings for quadrilateral finite elements" Tags="|finite-element|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19421" PostTypeId="2" ParentId="19384" CreationDate="2015-04-21T12:32:54.140" Score="2" Body="&lt;p&gt;Even if I knew the contour in terms of points where $f_0$ was constant to some useful approximation, I would probably still use &lt;em&gt;B-Splines&lt;/em&gt; to adaptively refine the point sequence to converge to an arc length. (If that's gobblety gook, the reference I'm about to give will cover this among more advanced spline details.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider reading (may require purchase)&#xA;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0921889009000190&quot; rel=&quot;nofollow&quot;&gt;Contour reconstruction using recursive smoothing splines - Algorithms and experimental validation&lt;/a&gt;&#xA;from Robotics and Autonomous Systems&#xA;Volume 57, Issues 6–7, 30 June 2009, Pages 617–628&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That article will either answer your question or point you in more elementary directions if they better suit you.&lt;/p&gt;&#xA;" OwnerUserId="9683" LastEditorUserId="690" LastEditDate="2015-04-22T08:10:56.440" LastActivityDate="2015-04-22T08:10:56.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19422" PostTypeId="1" AcceptedAnswerId="19425" CreationDate="2015-04-21T12:41:05.953" Score="2" ViewCount="545" Body="&lt;p&gt;Is there an efficient way to perform an incomplete Cholesky factorization on a symmetric positive definite sparse matrix (CSR format), in order to use it as a preconditioner for a CG solver? Is there a FORTRAN subroutine that performs such an factorization in parallel?&lt;/p&gt;&#xA;" OwnerUserId="15018" LastActivityDate="2015-04-21T13:35:17.710" Title="Incomplete Cholesky" Tags="|linear-solver|iterative-method|fortran|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19423" PostTypeId="1" CreationDate="2015-04-21T12:59:46.863" Score="1" ViewCount="1714" Body="&lt;p&gt;here is a Matlab beginner banging his head on the wall...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve a system of partial differential equations in Matlab, with both derivatives in time and space domains. I am using the pdepe function for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system is, to be simple, a sort of solar thermal panel, made of three layers: an absorber plate, a fluid layer of running water and a back insulation layer. each of these layers is represented by a differential equation. Of course, many assumptions are implicit in the model, e.g. the model is here considered 1D (no edge effects...), the temperature disuniformity inside each layer is  neglected ecc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The differential equations that describe the system are (at the present level of complexity):&#xA;&lt;img src=&quot;https://i.stack.imgur.com/wCREL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where CXX are the thermal capacitances and hXX are the heat transfer coefficients between the different layers of the thermal model. Vfl is the fluid velocity, Tia is a known ambient temperature and Psolar is a heating power due to solar irradiance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As initial conditions:&#xA;the temperature of the recirculating water varies linearly from an inlet temperature to a first-attempt outlet temperature of 12°C&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the temperature of the solar absorber varies linearly from 16.5 to 19 (respectively, averages between the inlet temperature of the fluid and the ambient temperature and between the outlet temperature of the fluid and the ambient temperature)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;analogously, the temperature of the back insulation layer varies linearly from 16.5 to 19.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The spatial boundary conditions are:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/8iTji.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where Tfl,inlet and Tfl,outlet are the inlet and outlet temperatures of the circulating water.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab this becomes (not all code lines are reported)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;function SolarAbsorber_TimeSpace&#xA;&#xA;&#xA;m = 0;&#xA;&#xA;x = linspace(0,2.5,30);&#xA;&#xA;t = linspace(0,7200,100); &#xA;&#xA;sol = pdepe(m,@pd_SolAbs_pde,@pde_SolAbs_ic,@pde_SolAbs_bc,x,t);&#xA;&#xA;T_met = sol(:,:,1); &#xA;&#xA;T_w = sol(:,:,2);&#xA;&#xA;T_is = sol(:,:,3);&#xA;&#xA;function [c,f,s] = pd_SolAbs_pde(x,t,u,DuDx)&#xA;% I do  not report here the initialization of the coefficients&#xA;&#xA;c = [C_met;C_w;C_is];                         &#xA;&#xA;f = [ 0; 0; 0];&#xA;&#xA;s1 = (h_ai_met)*(T_ai - u(1))+ h_met_w*(u(2) - u(1))+ Gv*tau*abs);&#xA;&#xA;s2 = (h_met_w*(u(1) - u(2))+ h_is_w*(u(3) - u(2)))- DuDx(2)*vx;&#xA;&#xA;s3 = (h_is_w*(u(2) - u(3)) + h_is_ai*(T_ai - u(3)));&#xA;&#xA;s = [s1; s2; s3];                                &#xA;&#xA;% --------------------------------------------------------------------------  &#xA;function u0 = pde_SolAbs_ic(x)&#xA;&#xA;alfa =((12-7)/2.5);&#xA;&#xA;beta = ((26+12)/2 -(26+7)/2)/2.5;&#xA;&#xA;T_met_0 = beta*x+(26+7)/2; %°C&#xA;&#xA;T_w_0 = alfa*x+7;    %°C&#xA;&#xA;T_is_0 = beta*x+(26+7)/2; %°C&#xA;&#xA;u0 = [T_met_0; T_w_0; T_is_0];&#xA;&#xA;&#xA;% --------------------------------------------------------------------------  &#xA;function [pl,ql,pr,qr] = pde_SolAbs_bc(xl,ul,xr,ur,t,u0,u)&#xA;&#xA;pl = [ul(1)- (26+7)/2; ul(2)-7; ul(3)-(26+7)/2];&#xA;&#xA;ql = [0;0;0];  &#xA;&#xA;pr = [ur(1)-(26+12)/2; ur(2)-12; ur(3)-(26+12)/2];&#xA;&#xA;qr = [0;0;0];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd like to describe the profile of the water temperature, giving a constant mass flow rate, a constant inlet temperature, variable forcing boundary conditions such as solar radiation, the initial conditions for the three layers etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first simulation, I just assume that solar radiation is constant and, starting at time t=0, it heats up the absorber while the circulating water removes the heat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here my questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 Matlab_pdepe asks me to define boundary conditions also on the &quot;right side&quot; (outlet of the solar panel), which is actually unknown!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 The graph shows that in correspondence of the boundaries, the profile gets unstable. This does not change significantly with the mesh size nor with the integration time span. Please consider that the left boundary condition of the absorber plate is fixed by the heat balance with the water inlet temperature and the ambient temperature, both supposed constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/VrwhW.jpg&quot; alt=&quot;enter image description here&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea or request for further explanations will be welcome :) :) :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Giulio&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Thank you Bill Greene for the comment about the Peclet number and the issue about diffusion vs convection mechanisms. Actually, the water velocity is fixed (dependent on the fixed water mass flow). I have refined the mesh in order to decrease the Peclet number and the result is that the profile tends to stability, but for the regions in proximity of the boundaries.&#xA;Of course, now the point is understanding more in depth what happened ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the results are not physically consistent: the water passes from 7 to 11.7°C in less than 1cm. In a similar way, close to the outlet, the temperature oscillates to fit with the ooutlet boundary condition :(&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still would like to fix only the inlet temperature and the water velocity, letting the outlet temperature free to vary according to the physics of the system. Any suggestions about the &quot;right&quot; boundary conditions for the three layers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/v8GQG.jpg&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/fM6Cf.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="15151" LastEditorUserId="15151" LastEditDate="2015-04-22T13:44:15.503" LastActivityDate="2015-04-22T13:44:15.503" Title="Instability of pdepe in Matlab... boundary conditions?" Tags="|matlab|pde|boundary-conditions|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19424" PostTypeId="2" ParentId="19411" CreationDate="2015-04-21T13:01:39.893" Score="2" Body="&lt;p&gt;I agree with Christian Clason that your problem is likely with the boundary conditions.&#xA;$\partial_n u$ is another essential boundary condition (like $u$)  so, in general, you&#xA;will also need to use your penalty approach to satisfy it. Apparently, the prescribed&#xA;$\partial_n u$ for your $u=\sin(4\pi x)\sin(4\pi y)$ case is consistent with the zero natural boundary&#xA;conditions that you are getting by default from the weak form.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-04-21T13:01:39.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19425" PostTypeId="2" ParentId="19422" CreationDate="2015-04-21T13:35:17.710" Score="2" Body="&lt;p&gt;There's no need to do it yourself: The good people who bring us PETSc, Trilinos, and a number of other linear algebra libraries have already done it for you. I'm not sure about Fortran interfaces, but I think that PETSc has them. If they don't, it should not be overly difficult to write some if the ILU is all you want to compute and apply.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-21T13:35:17.710" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19426" PostTypeId="2" ParentId="19419" CreationDate="2015-04-21T13:47:24.097" Score="7" Body="&lt;p&gt;You rarely compute the inverse transform (though it's not overly difficult: you just need to do a Newton iteration). The thing is that to compute&#xA;$\hat\Psi_j^\pm(F_{E\pm}^{-1}(\zeta_k))$ at a quadrature point $\zeta_k$ you already have everything you need: the quadrature points $\zeta_k$ are in fact the mapped locations $\zeta_k=F_{E\pm}(\hat\zeta_k)$ of the quadrature points $\hat\zeta_k$ defined on the reference cell!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, $\hat\Psi_j^\pm(F_{E\pm}^{-1}(\zeta_k)) = \hat\Psi_j^\pm(\hat\zeta_k)$. In other words, the evaluation of shape functions happens on the reference cell, not the real cell. It's really that easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In the interest of expanding the discussion: The only times when it gets more interesting is if someone wants to know what the value of the finite elements solution is at an &lt;em&gt;arbitrary&lt;/em&gt; point $\zeta$ that is not the mapped location of a quadrature point. In that case, you really have to find the cell that $\zeta$ is in, and then do the inverse mapping $F_{E\pm}^{-1}(\zeta)$. This is expensive, and such queries are consequently avoided if possible.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-21T13:47:24.097" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19427" PostTypeId="1" AcceptedAnswerId="19436" CreationDate="2015-04-21T14:48:30.607" Score="3" ViewCount="123" Body="&lt;p&gt;Is the following a standards-conformant MPI-3 C program? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;mpi.h&amp;gt;&#xA;int main(int argc, char** argv)&#xA;{&#xA;  int dlocalouter=1, dglobalouter=0, dlocalinner=1, dglobalinner=0;&#xA;  MPI_Request router, rinner;&#xA;  MPI_Init(&amp;amp;argc,&amp;amp;argv);&#xA;  MPI_Iallreduce(&amp;amp;dlocalouter,&amp;amp;dglobalouter,1,MPI_INT,MPI_SUM,MPI_COMM_WORLD,&amp;amp;router);&#xA;  MPI_Iallreduce(&amp;amp;dlocalinner,&amp;amp;dglobalinner,1,MPI_INT,MPI_SUM,MPI_COMM_WORLD,&amp;amp;rinner);&#xA;  MPI_Wait(&amp;amp;rinner,MPI_STATUS_IGNORE);&#xA;  MPI_Wait(&amp;amp;router,MPI_STATUS_IGNORE);&#xA;  MPI_Finalize();&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2015-09-25T13:34:31.623" LastActivityDate="2015-09-25T13:34:31.623" Title="Is it valid MPI-3 to nest non-blocking collectives?" Tags="|c|mpi|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19428" PostTypeId="1" CreationDate="2015-04-21T18:27:11.787" Score="1" ViewCount="50" Body="&lt;p&gt;Is anybody here using kriging for surface interpolation from huge datasets (&gt; 1 million points)?&#xA;How large datasets are your datasets and what do you model?&#xA;Also what software are you using?&lt;/p&gt;&#xA;" OwnerUserId="3622" LastActivityDate="2015-04-21T18:27:11.787" Title="Anybody using kriging for surface interpolation from huge point datasets?" Tags="|interpolation|" AnswerCount="0" CommentCount="1" ClosedDate="2015-04-24T02:13:24.697" ContentLicense="CC BY-SA 3.0" />
  <row Id="19429" PostTypeId="1" CreationDate="2015-04-21T19:55:44.330" Score="4" ViewCount="363" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For a study that is beyond the scope of this question, I applied all of &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Stepping-Functions.html&quot; rel=&quot;nofollow&quot;&gt;GSL’s adaptive Runge–Kutta methods&lt;/a&gt; to a certain problem. This includes a Runge–Kutta method of 2&lt;sup&gt;nd&lt;/sup&gt; and 3&lt;sup&gt;rd&lt;/sup&gt; order, denoted as &lt;code&gt;gsl_odeiv2_step_rk2&lt;/code&gt;. It is arguably a rather bad method, but this makes the method particularly intersting for my application, about which I am writing a paper now. &lt;strong&gt;For this purpose, I am interested in further references about this method, and in particular about its name and origin.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The method&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The method is mainly what Wikpedia calls &lt;a href=&quot;http://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods#Kutta.27s_third-order_method&quot; rel=&quot;nofollow&quot;&gt;Kutta’s third-order method&lt;/a&gt; and uses the &lt;a href=&quot;http://en.wikipedia.org/wiki/Midpoint_method&quot; rel=&quot;nofollow&quot;&gt;midpoint method or modified Euler method&lt;/a&gt; for error estimation. The Butcher tableau is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{c|ccc}&#xA;0 \\&#xA;\frac{1}{2} &amp;amp; \frac{1}{2} \\&#xA;1 &amp;amp; -1 &amp;amp; 2 \\\hline&#xA;&amp;amp; \frac{1}{6} &amp;amp; \frac{2}{3} &amp;amp; \frac{1}{6} \\&#xA;&amp;amp; 0 &amp;amp; 1&#xA;\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;My research so far&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the GSL’s source code, this method is called &lt;em&gt;Euler–Cauchy,&lt;/em&gt; however all other sources I could find apply this name to the &lt;a href=&quot;http://en.wikipedia.org/wiki/Euler_method&quot; rel=&quot;nofollow&quot;&gt;classical Euler method&lt;/a&gt;, so I am skeptical about this.&lt;/li&gt;&#xA;&lt;li&gt;The source code further references &lt;a href=&quot;http://en.wikipedia.org/wiki/Abramowitz_and_Stegun&quot; rel=&quot;nofollow&quot;&gt;Abramowitz and Stegun&lt;/a&gt; for the component methods. The component methods can indeed be found there but is neither equipped with a name nor a reference.&lt;/li&gt;&#xA;&lt;li&gt;No other list of Runge–Kutta methods I could find contains the method as a whole. The 2&lt;sup&gt;nd&lt;/sup&gt;-order method is usually called &lt;em&gt;midpoint method.&lt;/em&gt; The only source I could find that mentions the 3&lt;sup&gt;rd&lt;/sup&gt;-order method is Wikipedia, which calls it &lt;em&gt;Kutta’s third-order method,&lt;/em&gt; but lists no source to which I have easy access.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2023-08-18T08:14:21.327" Title="Looking for references on this adaptive Runge–Kutta method (GSL’s rk2)" Tags="|reference-request|runge-kutta|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19430" PostTypeId="1" AcceptedAnswerId="19448" CreationDate="2015-04-21T20:12:22.840" Score="2" ViewCount="1286" Body="&lt;p&gt;I try to solve the problem $Ax=B$ where $A$ is a large sparse $n\times n$ matrix, and $B$ is a dense $n\times m$ matrix (here $n=754850$ and $m=182$). The backslash operator yields correct solution (&lt;code&gt;x = A\B&lt;/code&gt;), but most of the computational time is done on one thread (the initial step of the process is nicely executed in parallel but not the finale steps). This obviously slows down the process. What is wrong here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried an LU factorization of $A$ and then solve for each column in $B$, but &lt;code&gt;lu(A)&lt;/code&gt; seems not to be parallelized (although I am using version &lt;code&gt;R2014b&lt;/code&gt;). I have seen that people think &lt;code&gt;lu(A)&lt;/code&gt; is parallelized, so is there some update I miss or something?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By executing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;spparms('spumoni',2)&#xA;x = A\b;&#xA;spparms&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got the following output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sp\: bandwidth = 60795+1+60795.&#xA;sp\: is A diagonal? no.&#xA;sp\: is band density (0.00) &amp;gt; bandden (0.50) to try banded solver? no.&#xA;sp\: is A triangular? no.&#xA;sp\: is A morally triangular? no.&#xA;sp\: is A a candidate for Cholesky (symmetric, real positive diagonal)? no.&#xA;sp\: use Unsymmetric MultiFrontal PACKage with automatic reordering.&#xA;UMFPACK V5.4.0 (May 20, 2009), Control:&#xA;    Matrix entry defined as: double complex&#xA;    Int (generic integer) defined as: UF_long&#xA;&#xA;    0: print level: 2&#xA;    1: dense row parameter:    0.2&#xA;        &quot;dense&quot; rows have    &amp;gt; max (16, (0.2)*16*sqrt(n_col) entries)&#xA;    2: dense column parameter: 0.2&#xA;        &quot;dense&quot; columns have &amp;gt; max (16, (0.2)*16*sqrt(n_row) entries)&#xA;    3: pivot tolerance: 0.1&#xA;    4: block size for dense matrix kernels: 32&#xA;    5: strategy: 0 (auto)&#xA;    6: initial allocation ratio: 0.7&#xA;    7: max iterative refinement steps: 2&#xA;    12: 2-by-2 pivot tolerance: 0.01&#xA;    13: Q fixed during numerical factorization: 0 (auto)&#xA;    14: AMD dense row/col parameter:    10&#xA;       &quot;dense&quot; rows/columns have &amp;gt; max (16, (10)*sqrt(n)) entries&#xA;        Only used if the AMD ordering is used.&#xA;    15: diagonal pivot tolerance: 0.001&#xA;        Only used if diagonal pivoting is attempted.&#xA;    16: scaling: 1 (divide each row by sum of abs. values in each row)&#xA;    17: frontal matrix allocation ratio: 0.5&#xA;    18: drop tolerance: 0&#xA;    19: AMD and COLAMD aggressive absorption: 1 (yes)&#xA;&#xA;    The following options can only be changed at compile-time:&#xA;    8: BLAS library used:  Fortran BLAS.  size of BLAS integer: 8&#xA;    9: compiled for MATLAB&#xA;    10: CPU timer is POSIX times ( ) routine.&#xA;    11: compiled for normal operation (debugging disabled)&#xA;    computer/operating system: Linux&#xA;    size of int: 4 UF_long: 8 Int: 8 pointer: 8 double: 8 Entry: 16 (in bytes)&#xA;&#xA;    sp\: UMFPACK's factorization was successful.&#xA;    sp\: UMFPACK's solve was successful.&#xA;    UMFPACK V5.4.0 (May 20, 2009), Info:&#xA;        matrix entry defined as:          double complex&#xA;        Int (generic integer) defined as: UF_long&#xA;        BLAS library used: Fortran BLAS.  size of BLAS integer: 8&#xA;        MATLAB:                           yes.&#xA;        CPU timer:                        POSIX times ( ) routine.&#xA;        number of rows in matrix A:       754850&#xA;        number of columns in matrix A:    754850&#xA;        entries in matrix A:              86456682&#xA;        memory usage reported in:         16-byte Units&#xA;        size of int:                      4 bytes&#xA;        size of UF_long:                  8 bytes&#xA;        size of pointer:                  8 bytes&#xA;        size of numerical entry:          16 bytes&#xA;&#xA;        strategy used:                    symmetric&#xA;        ordering used:                    amd on A+A'&#xA;        modify Q during factorization:    no&#xA;        prefer diagonal pivoting:         yes&#xA;        pivots with zero Markowitz cost:               0&#xA;        submatrix S after removing zero-cost pivots:&#xA;            number of &quot;dense&quot; rows:                    0&#xA;            number of &quot;dense&quot; columns:                 0&#xA;            number of empty rows:                      0&#xA;            number of empty columns                    0&#xA;            submatrix S square and diagonal preserved&#xA;        pattern of square submatrix S:&#xA;            number rows and columns                    754850&#xA;            symmetry of nonzero pattern:               1.000000&#xA;            nz in S+S' (excl. diagonal):               85701832&#xA;            nz on diagonal of matrix S:                754850&#xA;            fraction of nz on diagonal:                1.000000&#xA;        AMD statistics, for strict diagonal pivoting:&#xA;            est. flops for LU factorization:           4.16517e+14&#xA;            est. nz in L+U (incl. diagonal):           7435413184&#xA;            est. largest front (# entries):            896703025&#xA;            est. max nz in any column of L:            29945&#xA;            number of &quot;dense&quot; rows/columns in S+S':    0&#xA;        symbolic factorization defragmentations:       0&#xA;        symbolic memory usage (Units):                 199337590&#xA;        symbolic memory usage (MBytes):                3041.7&#xA;        Symbolic size (Units):                         1961813&#xA;        Symbolic size (MBytes):                        30&#xA;        symbolic factorization CPU time (sec):         15.57&#xA;        symbolic factorization wallclock time(sec):    15.45&#xA;&#xA;        matrix scaled: yes (divided each row by sum of abs values in each row)&#xA;        minimum sum (abs (rows of A)):              3.25924e-02&#xA;        maximum sum (abs (rows of A)):              6.95838e+01&#xA;&#xA;        symbolic/numeric factorization:      upper bound               actual      %&#xA;        variable-sized part of Numeric object:&#xA;            initial size (Units)               226394213            225639362   100%&#xA;            peak size (Units)               101780904057           8946021598     9%&#xA;            final size (Units)               98119112401           7543694062     8%&#xA;        Numeric final size (Units)           98124018962           7548223198     8%&#xA;        Numeric final size (MBytes)            1497253.7             115176.7     8%&#xA;        peak memory usage (Units)           101792135841           8957253382     9%&#xA;        peak memory usage (MBytes)             1553224.7             136676.8     9%&#xA;        numeric factorization flops          2.95433e+16          4.20989e+14     1%&#xA;        nz in L (incl diagonal)              36831301536           3820083550    10%&#xA;        nz in U (incl diagonal)              58996420369           3718115473     6%&#xA;        nz in L+U (incl diagonal)            95826967055           7537444173     8%&#xA;        largest front (# entries)            13417023050            896703025     7%&#xA;        largest # rows in front                    93703                29945    32%&#xA;        largest # columns in front                144673                29945    21%&#xA;&#xA;        initial allocation ratio used:                 0.0942&#xA;        # of forced updates due to frontal growth:     0&#xA;        number of off-diagonal pivots:                 38&#xA;        nz in L (incl diagonal), if none dropped       3820083550&#xA;        nz in U (incl diagonal), if none dropped       3718115473&#xA;        number of small entries dropped                0&#xA;        nonzeros on diagonal of U:                     754850&#xA;        min abs. value on diagonal of U:               8.14e-09&#xA;        max abs. value on diagonal of U:               1.42e+02&#xA;        estimate of reciprocal of condition number:    5.75e-11&#xA;        indices in compressed pattern:                 13860039&#xA;        numerical values stored in Numeric object:     7537500101&#xA;        numeric factorization defragmentations:        3&#xA;        numeric factorization reallocations:           0&#xA;        costly numeric factorization reallocations:    0&#xA;        numeric factorization CPU time (sec):          106790.54&#xA;        numeric factorization wallclock time (sec):    7368.85&#xA;        numeric factorization mflops (CPU time):       3942.19&#xA;        numeric factorization mflops (wallclock):      57130.86&#xA;        symbolic + numeric CPU time (sec):             106806.11&#xA;        symbolic + numeric mflops (CPU time):          3941.62&#xA;        symbolic + numeric wall clock time (sec):      7384.30&#xA;        symbolic + numeric mflops (wall clock):        57011.33&#xA;&#xA;        solve flops:                                   1.86839e+11&#xA;        iterative refinement steps taken:              1&#xA;        iterative refinement steps attempted:          2&#xA;        sparse backward error omega1:                  4.93e-16&#xA;        sparse backward error omega2:                  0.00e+00&#xA;        solve CPU time (sec):                          248.79&#xA;        solve wall clock time (sec):                   246.02&#xA;        solve mflops (CPU time):                       750.99&#xA;        solve mflops (wall clock time):                759.45&#xA;&#xA;        total symbolic + numeric + solve flops:        4.21176e+14&#xA;        total symbolic + numeric + solve CPU time:     107054.90&#xA;        total symbolic + numeric + solve mflops (CPU): 3934.20&#xA;        total symbolic+numeric+solve wall clock time:  7633.09&#xA;        total symbolic+numeric+solve mflops(wallclock) 55177.60&#xA;&#xA;    SParse MONItor output level 2.&#xA;    mmd: threshold = 1.1 * mindegree + 1,&#xA;         using approximate degrees in A'*A,&#xA;         supernode amalgamation every 3 stages,&#xA;         row reduction every 3 stages,&#xA;         withhold rows at least 50% dense in colmmd.&#xA;    Minimum degree orderings used with v4 chol, lu, and qr in \ and /.&#xA;    Approximate minimum degree orderings used with CHOLMOD and UMFPACK in \ and /.&#xA;    Pivot tolerance of 0.1 used by UMFPACK in \ and /.&#xA;    Backslash uses band solver if band density is &amp;gt; 0.5&#xA;    UMFPACK used for lu in \ and /.&#xA;    Symmetric pivot tolerance of 0.001 used by UMFPACK in \ and /.&#xA;    Pivot tolerance of 0.01 used by MA57 in \ and /.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15159" LastEditorUserId="15159" LastEditDate="2015-04-23T17:38:01.933" LastActivityDate="2015-04-24T00:20:50.353" Title="Solve Ax=B where B is a matrix in parallell" Tags="|sparse-matrix|matrix|matrix-factorization|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="19431" PostTypeId="2" ParentId="11704" CreationDate="2015-04-22T00:19:41.847" Score="3" Body="&lt;p&gt;Boost provides all the functions you need, so it should be sufficient. Instead of using the direct integral representation of the incomplete beta function, it's better to use the &lt;em&gt;regularized incomplete beta function&lt;/em&gt; (see &lt;a href=&quot;http://dlmf.nist.gov/8.17&quot; rel=&quot;nofollow&quot;&gt;DLMF 8.17&lt;/a&gt;):&#xA;$$ B(x,y;a,b) = (I_y(a,b)-I_x(a,b))B(a,b). $$&#xA;This way, $I_x(a,b)=B(x;a,b)/B(a,b)$ will have values in $[0,1]$, so that&#xA;$$ \log B(x,y;a,b) = \log(I_y(a,b)-I_x(a,b)) + \log B(a,b). $$&#xA;is much easier to compute. This way the only source of numerical instability, assuming the underlying functions are stable, would be the subtraction when $x$ and $y$ are very close; in that case, if it ever arises, the direct integral representation might work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The regularized incomplete beta function is given by &lt;a href=&quot;http://www.boost.org/doc/libs/1_58_0/libs/math/doc/html/math_toolkit/sf_beta/ibeta_function.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;ibeta&lt;/code&gt; in Boost&lt;/a&gt;, and the log-Beta function can be calculated using the &lt;a href=&quot;http://www.boost.org/doc/libs/1_58_0/libs/math/doc/html/math_toolkit/sf_gamma/lgamma.html&quot; rel=&quot;nofollow&quot;&gt;log-gamma function &lt;code&gt;lgamma&lt;/code&gt;, also in Boost&lt;/a&gt;. Boost's implementation of $I_x$ is based on the paper &lt;a href=&quot;http://dl.acm.org/citation.cfm?doid=131766.131776&quot; rel=&quot;nofollow&quot;&gt;Algorithm 708: Significant digit computation of the incomplete beta function ratios&lt;/a&gt; by Didonato and Morris.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-04-22T00:19:41.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19432" PostTypeId="1" AcceptedAnswerId="19433" CreationDate="2015-04-22T02:52:20.453" Score="1" ViewCount="159" Body="&lt;p&gt;I have to minimize a nonlinear objective function $f(x_0, x_1, x_2, x_3, x_4, x_5)$ with 6 variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraints governing these these variables are a mix of &lt;strong&gt;nonlinear inequality constraints&lt;/strong&gt;, and some &lt;strong&gt;combinatorial constraints&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;strong&gt;nonlinear inequality constraints and the objective function&lt;/strong&gt;, I can evaluate the functional and Jacobian values as long as all the variables ($x_0$ to $x_5$) are specified. I can also obtain the Hessian values if I want, but I afraid that the values might be inaccurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the objective function and the nonlinear inequality constraints are &lt;em&gt;always continuous, &lt;strong&gt;but&lt;/strong&gt; only &lt;a href=&quot;http://en.wikipedia.org/wiki/Piecewise&quot; rel=&quot;nofollow&quot;&gt;piecewise differentiable&lt;/a&gt; over the region of interest&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By &lt;strong&gt;combinatorial constraints&lt;/strong&gt;, I mean that ${x_3, x_4, x_5}$  can only take a list of discrete values together, such as {3,4,5}, {8,7,18} and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any existing scientific package ( or mathematical algorithm, I can program the solver myself) that does what I need?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: the inequality constraints are highly nonlinear and can only be evaluated via numerical means. &lt;/p&gt;&#xA;" OwnerUserId="690" LastEditorUserId="690" LastEditDate="2015-04-24T06:15:18.397" LastActivityDate="2015-04-24T06:15:18.397" Title="Solving nonlinear optimization problem with combinational constraints" Tags="|optimization|nonlinear-programming|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19433" PostTypeId="2" ParentId="19432" CreationDate="2015-04-22T06:27:33.173" Score="1" Body="&lt;p&gt;At best, your problem is a mixed-integer nonlinear program (MINLP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you can decompose your problem into subregions, so that in each region, your objective function and all constraints are twice continuously differentiable, you could probably use BARON, Couenne, or Bonmin. I've assume that a decent locally optimal solution is satisfactory; if a globally optimal solution is required, BARON is probably your best option, but will require analytical expressions for your objective function and constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you cannot do that, then you'll probably have to find a nonsmooth mixed-integer nonlinear programming solver. I could not find one with a cursory Google search, but perhaps one exists. I suppose you could wrap a non-smooth solver in a branch-and-bound algorithm, if you wanted to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless you plan on pursuing the implementation of such solvers as the primary thrust of your research, I strongly recommend that you avoid trying to program the solver yourself. Many of the best optimization solvers are the product of man-decades of work, and simple implementations often achieve only a small fraction of the performance of a state-of-the-art implementation. (I am currently in the process of implementing a mixed-integer programming solver as part of my research, and expect that programming a respectable implementation will be a multi-year effort.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-04-22T06:51:18.683" LastActivityDate="2015-04-22T06:51:18.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19436" PostTypeId="2" ParentId="19427" CreationDate="2015-04-22T08:03:36.477" Score="2" Body="&lt;p&gt;Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I asked was that with a particular implementation (cray-mpich 7.0.4), in a particular environment on a particular system, the second &lt;code&gt;MPI_Wait&lt;/code&gt; call would not return. This was likely due to a bug somewhere in the MPI library or a dependent communication library, and was remedied by upgrading to a newer version of the MPI implementation (cray-mpich 7.1.1).&lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2015-04-22T08:54:39.437" LastActivityDate="2015-04-22T08:54:39.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19437" PostTypeId="2" ParentId="19365" CreationDate="2015-04-22T09:38:12.243" Score="1" Body="&lt;p&gt;If you are interested in numerical (approximative) solution, one of many methods that I would recommend (and I use it) is an &quot;immersed interface&quot; method (or several other similar methods with similar names like ghost fluid methods etc). In such a case you need to specify boundary conditions on your (moving and fixed parts of) boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that your moving boundary is represented as a zero set of some function (the level set function, usually a signed distance function). To solve the Laplace equation numerically, you can use very simple finite difference method on structured (quadrangle) grid that involves always your moving boundary. As you want to solve it only for grid nodes where your level set function is (e.g.) negative and not there where it is positive, you have to apply some &quot;extension&quot; (extrapolation) for the nodes next to your boundary. If your boundary conditions on the moving boundary are Dirichlet type (the values of solution are prescribed) then this extension is straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I am not sure if you are interested in numerical solution, I stop my description here. As a literature I would recommend:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;F. Gibou, R.P. Fedkiw, L.T. Cheng, M. Kang, A second-order-accurate&#xA;symmetric discretization of the Poisson equation on irregular domains,&#xA;J. Comput. Phys. 176 (2002) 205-227&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that is available on the net and you should look only for the stuff dealing with Poisson equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was using such method for moving groundwater table:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P. Frolkovič: Application of level set method for groundwater flow with moving boundary. Advances in Water Resources, Volume 47, October 2012, Pages 56–66&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fancy animation ;-) below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://dl.dropboxusercontent.com/u/386482/dam.gif&quot; alt=&quot;animation of groundwater table&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2015-04-22T09:38:12.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19438" PostTypeId="2" ParentId="19375" CreationDate="2015-04-22T10:02:36.470" Score="-1" Body="&lt;p&gt;I would like to approach this problem from the perspective of the simulation of an incompressible flow. Pressure is governed by a Poisson's equation and as the simulation progresses the source term of this equation changes as it is dependent on the evolving velocity field. In my experience I found that retaining the old guess for the pressure does not slow down the solution process or change the solution. Could you test whether retaining the previous solution slows the convergence on the next round of FMG cycle where the charge distribution is different? I infer from your question that this test has not been performed. &lt;/p&gt;&#xA;" OwnerUserId="5357" LastActivityDate="2015-04-22T10:02:36.470" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19439" PostTypeId="2" ParentId="19429" CreationDate="2015-04-22T11:26:51.180" Score="5" Body="&lt;p&gt;John Butcher's &lt;a href=&quot;http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470723351.html&quot; rel=&quot;nofollow&quot;&gt;Numerical Methods for Ordinary Differential Equations&lt;/a&gt; lists your 3rd-order method as (233f) on page 95, though it gives no attribution.  The 2nd-order method is of course well-known as you say.  I know of no reference that suggests pairing them, but it is a fairly obvious embedded error estimator for this particular 3rd-order method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 3rd-order method is not mentioned in the book of Hairer, Wanner, &amp;amp; Norsett, nor in Butcher's &lt;a href=&quot;http://people.cs.vt.edu/~asandu/Public/Qual2011/DiffEqn/Butcher_1996_RK-history.pdf&quot; rel=&quot;nofollow&quot;&gt;paper on the history of Runge-Kutta methods&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-04-28T04:06:47.663" LastActivityDate="2015-04-28T04:06:47.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19440" PostTypeId="2" ParentId="19423" CreationDate="2015-04-22T12:32:48.370" Score="1" Body="&lt;p&gt;This looks like a classic case of the convective term in your equations being large relative to the diffusive (conductive) term. The pdepe function is basically designed to handle the opposite case-- where diffusion dominates. You can quantify the relative effects of convection and diffusion in your problem by computing the mesh Peclet number. This has been discussed in some detail in another question previously asked on this site: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8649/does-the-finite-element-method-impose-any-restrictions-on-the-peclet-number-for&quot;&gt;Does the finite element method impose any restrictions on the Peclet number for numerical stability?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-04-22T12:32:48.370" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19441" PostTypeId="1" CreationDate="2015-04-22T19:13:59.580" Score="1" ViewCount="69" Body="&lt;p&gt;I'm working with Computational Neuroscience. I have a large Synaptic Matrix (x axis: presynaptic NeuronID, y axis: postsynaptic NeuronID) in a Modular network. This &lt;a href=&quot;https://drive.google.com/open?id=0ByAtIC3JJ5OAYnJpeGFDdEdlS1E&amp;amp;authuser=0&quot; rel=&quot;nofollow&quot;&gt;matrix&lt;/a&gt; is close to a random one and Girko's circular law applies partly, in the sense that the bulk of the eigenvalues lies in a circle when you plot their Imag/Real parts; some eigenvalues will be outside the bulk because of the network's modularity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, my system is nonlinear since it is composed of neurons, whose population activity's transfer function is close to a sigmoid (&lt;a href=&quot;http://neuronaldynamics.epfl.ch/online/x316.png&quot; rel=&quot;nofollow&quot;&gt;2&lt;/a&gt;). That is Frequency/Input in continuous time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I derive some conclusions about my system's stability (locally at least) based on the eigenvalues? For instance, if I have a large real eigenvalue, my system could prove to be unstable, with exponentially increased activity over time..&lt;/p&gt;&#xA;" OwnerUserId="7631" LastEditorUserId="7631" LastEditDate="2015-04-22T19:52:25.247" LastActivityDate="2017-05-19T17:11:58.110" Title="Locally evaluate nonlinear dynamic system's stability using eigenvalues" Tags="|linear-algebra|eigenvalues|stability|neuroscience|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19442" PostTypeId="1" CreationDate="2015-04-22T19:24:38.077" Score="4" ViewCount="1281" Body="&lt;p&gt;I am trying to solve a 2D wave equation &lt;strong&gt;implicitly&lt;/strong&gt; using FD with central approximations with the following boundary conditions &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;u=2\sin\left(\frac{2\pi}{5}t\right)\quad \text{at }x=0\\&#xA;&amp;amp;\frac{\partial u}{\partial y}=0 \quad \text{at }y=0\\&#xA;&amp;amp;\frac{\partial u}{\partial y}=0 \quad \text{at }y=2\\&#xA;&amp;amp;\frac{\partial u}{\partial t}=-\frac{\partial u}{\partial x} \quad \text{at }x=5&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;graphically  represented as&#xA;&lt;img src=&quot;https://i.stack.imgur.com/N3kJ1.png&quot; alt=&quot;schematics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary condition at x=0 generates the wave.&#xA;The boundary condition at x=5 refers to &lt;strong&gt;Mur boundary condition&lt;/strong&gt;, i.e. the wave doesn't bounce back, but simply continues to move outside the domain.&#xA;The general equation is given as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nRMZI.png&quot; alt=&quot;general equation&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Mur boundary condition can be expressed mathematically as &#xA;&lt;img src=&quot;https://i.stack.imgur.com/QibfA.png&quot; alt=&quot;Mur BC&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions were evaluated analytically and these are the 7 equations following the entire domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/KIbdP.png&quot; alt=&quot;equations&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;- I have managed to write some code, including setting up the &lt;code&gt;[A]&lt;/code&gt; matrix (the LHS of the equations), but I have problems with formulating the &lt;code&gt;[b]&lt;/code&gt; matrix (RHS) and solving it through the time-steps. I would appreciate if someone show me how to write it for the first equation - then I will be able to do it dor the remaining 6.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% length, time, height&#xA;L = 5; % [m]&#xA;h = 2; % [m]&#xA;&#xA;d = 0.25; % space spacing&#xA;dt = 0.05; % time increment&#xA;M = 100; &#xA;T_max = M*dt; % [s]&#xA;&#xA;nx = floor(L/d); % number of &amp;lt;x&amp;gt; samples&#xA;ny = floor(h/d); % number of &amp;lt;y&amp;gt; samples&#xA;k = floor(T_max/dt) + 1; % number of time samples&#xA;&#xA;% Constants&#xA;alpha = dt/d;&#xA;r = dt^2/d^2;&#xA;&#xA;% Number of grid points:&#xA;N=nx*ny;&#xA;&#xA;% Initialise matrices&#xA;U = zeros(N,1);&#xA;b = zeros(N,1);&#xA;x = zeros(N,1);&#xA;A = zeros(N,N);&#xA;&#xA;% Set numbering for the [A] matrix&#xA;num = 1;&#xA;for i=1:nx&#xA;    for j=1:ny&#xA;        number(i,j) = num;&#xA;        num = num + 1;&#xA;    end &#xA;end &#xA;&#xA;% [A] matrix - boundary conditions&#xA;    % Case 1 &#xA;    for i=2:nx-1&#xA;        for j=2:ny-1&#xA;            ii=number(i,j);&#xA;            A(ii, number(i, j)) = 1+4*r;&#xA;            A(ii, number(i+1, j)) = -r;&#xA;            A(ii, number(i-1,j)) = -r;&#xA;            A(ii, number(i, j+1)) = -r;&#xA;            A(ii, number(i, j-1)) = -r;&#xA;        end &#xA;    end       &#xA;&#xA;    % Case 2&#xA;    for i=1&#xA;        for j=2:ny-1&#xA;            ii=number(i,j);&#xA;            A(ii, number(1, j)) = 1+4*r;&#xA;            A(ii, number(2, j)) = -r;&#xA;            A(ii, number(1, j+1)) = -r;&#xA;            A(ii, number(1, j-1)) = -r;&#xA;        end &#xA;    end &#xA;&#xA;    % Case 3 &#xA;    for i=nx&#xA;        for j=1:ny&#xA;            ii=number(i,j);&#xA;            A(ii, number(nx, j)) = 1+alpha;&#xA;            A(ii, number(nx-1,j)) = 1-alpha;&#xA;        end &#xA;    end &#xA;&#xA;    % Case 4 &#xA;    for i=2:nx-1&#xA;        for j=1&#xA;            ii=number(i,j);&#xA;            A(ii, number(i, j)) = 1+4*r;&#xA;            A(ii, number(i-1,j)) = -r;&#xA;            A(ii, number(i+1,j)) = -r;&#xA;            A(ii, number(i, j+1)) = -2*r;&#xA;        end &#xA;    end &#xA;&#xA;    % Case 5&#xA;    for i=1&#xA;        for j=1&#xA;            ii=number(i,j);&#xA;            A(ii, number(i,j)) = 1+4*r;&#xA;            A(ii, number(i+1,j)) = -r;&#xA;            A(ii, number(i, j+1)) = -2*r;&#xA;        end &#xA;    end &#xA;&#xA;    % Case 6&#xA;    for i=2:nx-1&#xA;        for j=ny&#xA;            ii=number(i,j);&#xA;            A(ii, number(i,j)) = 1+4*r;&#xA;            A(ii, number(i-1,j)) = -r;&#xA;            A(ii, number(i+1, j)) = -r;&#xA;            A(ii, number(i, j-1)) = -2*r;&#xA;        end &#xA;    end &#xA;&#xA;    % Case 7&#xA;    for i=1&#xA;        for j=ny&#xA;            ii=number(i,j);&#xA;            A(ii, number(i,j)) = 1+4*r;&#xA;            A(ii, number(i+1, j)) = -r;&#xA;            A(ii, number(i, j-1)) = -2*r;&#xA;        end&#xA;    end&#xA;&#xA;% Inverse of [A] matrix&#xA;A_inv = inv(A);&#xA;&#xA;% [b] matrix (RHS)&#xA;time = 0;&#xA;for t=1:M&#xA;    % Computing [b] matrix&#xA;    for i=1:nx&#xA;        for j=1:ny&#xA;            ii = number(i,j);&#xA;            % U(1,1) = 2*sin((2*pi/5)*(t*dt));&#xA;&#xA;            % Case 1&#xA;            for i=2:nx-1&#xA;                for j=2:ny-1&#xA;&#xA;                end &#xA;            end &#xA;            % or&#xA;&#xA;            for i=1:nx&#xA;                for j=1:ny&#xA;&#xA;                end &#xA;            end &#xA;&#xA;        end &#xA;    end &#xA;&#xA;    % The solution vector&#xA;    x = A_inv*b;&#xA;&#xA;    % 2D matrix from the solution vector&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15165" LastEditorUserId="9667" LastEditDate="2015-04-22T19:49:02.503" LastActivityDate="2015-04-22T19:49:02.503" Title="2D wave equation with Mur boundary condition - setting up the RHS and solving (time-steps)" Tags="|matlab|finite-difference|boundary-conditions|wave-propagation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19444" PostTypeId="1" CreationDate="2015-04-23T00:25:51.113" Score="0" ViewCount="806" Body="&lt;p&gt;So I'm constructing a Gibbs sampler for a moderately sized graphical model, and I started encountering underflow errors (the computed relative frequencies were all very small). To get around this, I've started using the LOG-SUM-EXP-LOG trick found here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/648514/preventing-underflow-log-sum-exp-trick&quot;&gt;https://math.stackexchange.com/questions/648514/preventing-underflow-log-sum-exp-trick&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now however I have to compute a large number of logs and exponentials. Does anyone know how to do this efficiently in C/C++? I'm considering making a look-up table, but that's a lot of memory...&lt;/p&gt;&#xA;" OwnerUserId="15065" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-04-23T00:25:51.113" Title="Efficient Implementation of the Log Sum Exp Log trick for Preventing Underflow erros" Tags="|efficiency|numerical-limitations|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19445" PostTypeId="1" AcceptedAnswerId="19446" CreationDate="2015-04-23T02:12:41.430" Score="5" ViewCount="196" Body="&lt;p&gt;I'm a chemical engineering undergraduate and I'm currently starting to work in a theoretical transport phenomena/colloid science group. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While my group has a nice code base for larger scale simulations (by this I mean Monte Carlo/Molecular Dynamics/Brownian Dynamics type methods) it seems that when it comes to smaller problems that can be reduced to numerically solving pde's (with finite difference/elements methods for example) they tend to leave it to each student to learn by themselves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I'm just starting out, I'd like to learn these methods well for future work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone recommend a good book or reference for numerical methods in transport phenomena? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd prefer if it was focused on finite differences since that is the method I currently have the most familiarity with, but I'd be perfectly happy with any method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some extra notes:&#xA;I am aware and have access to LeVeque's book, which I've seen mentioned as the standard introductory textbook. However I'd prefer something that is more focused on the specific needs and methods of transport phenomena. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be awesome if it was available for less than around $120 since that is what I have available right now, but I understand that might be unrealistic. &lt;/p&gt;&#xA;" OwnerUserId="15014" LastActivityDate="2015-04-23T07:36:28.940" Title="Need a good reference for numerical transport phenomena" Tags="|finite-difference|reference-request|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19446" PostTypeId="2" ParentId="19445" CreationDate="2015-04-23T07:36:28.940" Score="4" Body="&lt;p&gt;Speaking as a chemical engineer, the numerical methods courses and textbooks I've seen come out of the field have been poor. There is &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521859719&quot; rel=&quot;nofollow&quot;&gt;the book by Beers&lt;/a&gt;, for instance, which was used as the main text when I studied for my PhD at MIT. The exposition isn't that great, and frankly, if you really get into any sort of numerical methods research, you'll be better served by having more depth in any of the topics covered in that book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A cursory Amazon search pulled up &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0521553784&quot; rel=&quot;nofollow&quot;&gt;Computational Transport Phenomena by Scheisser and Silebi&lt;/a&gt;. Scheisser is at Lehigh, and has long been a proponent of the method of lines, so you likely won't learn anything that isn't in LeVeque's book. The code samples look to be in Fortran 77, with bad practices like relying on &lt;code&gt;IMPLICIT DOUBLE PRECISION&lt;/code&gt;, so you'll also pick up bad coding habits, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend instead sticking with something like LeVeque or Strikwerda for finite difference methods, LeVeque or Toro for finite volume methods, or something like the text by Larson and Bengzon for finite element methods. I'd avoid recommending a text like Brenner and Scott to someone who doesn't have the pure math background, but it's a good book also. What discretization method you should use really depends on the problem; without more knowledge about the research problems you're interested in, I can't give you more specific recommendations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you've got a good handle on transport phenomena at the level of a textbook like Bird, Stewart, and Lightfoot; Incropera and DeWitt; or Deen (which is fine for an advanced undergrad, but probably not my first choice, even though it's a great book), and you understand the numerical methods, you'll be in good shape, because frankly, you'll be better prepared than most of the grad students who come out of chemical engineering programs when it comes to numerical methods.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-23T07:36:28.940" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19448" PostTypeId="2" ParentId="19430" CreationDate="2015-04-23T20:01:14.733" Score="3" Body="&lt;p&gt;The MATLAB \ operator uses UMFPACK when the input matrix is sparse, square,&#xA;and unsymmetric. UMFPACK is not a parallel sparse solver. However, the&#xA;UMFPACK factorization step calls BLAS level-3 routines in its computational&#xA;kernel. Most modern implementations of the BLAS (e.g. Intel MKL in MATLAB)&#xA;support multicore architectures so UMFPACK can obtain fairly significant&#xA;speedup in the factorization step. Since there isn't any high-level&#xA;parallelism in UMFPACK and the parallelized BLAS doesn't help much in&#xA;the solve step, there isn't any significant speedup there on multicore&#xA;machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MATLAB lu function calls an older LU implementation&#xA;(&lt;a href=&quot;https://www.mathworks.com/help/pdf_doc/otherdocs/simax.pdf&quot; rel=&quot;nofollow&quot;&gt;https://www.mathworks.com/help/pdf_doc/otherdocs/simax.pdf&lt;/a&gt;)&#xA;when the following form is used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[L,U,P] = lu(A);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and UMFPACK for this form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[L,U,P,Q] = lu(A);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The older implementation doesn't support BLAS level-3 calls so there&#xA;is no speedup on multicore machines. The UMFPACK version of lu supports&#xA;the parallel BLAS in the same way as backslash.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interesting discussion of the MATLAB backslash operator and the lu function&#xA;is included in this paper by Tim Davis:&#xA;&lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/techreports/factorize.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.cise.ufl.edu/research/sparse/techreports/factorize.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2015-04-24T00:20:50.353" LastActivityDate="2015-04-24T00:20:50.353" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19449" PostTypeId="1" CreationDate="2015-04-23T21:19:10.520" Score="1" ViewCount="51" Body="&lt;p&gt;I have the following problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two sequences of elements $A = [a_1,a_2,\cdots,a_n]$ and $B = [b_1,b_2,\cdots,b_m]$. I can build a matrix $D[n \times m]$ where $d_{ij} = d(a_i,b_j)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My greedy merging algorithm works as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For each iteration it takes the shortest distance pair and merges them.&lt;/li&gt;&#xA;&lt;li&gt;Then all the distances are recomputed (considering the merged pair)&lt;/li&gt;&#xA;&lt;li&gt;The process is repeated until each element of the two sequences is merged.&lt;/li&gt;&#xA;&lt;li&gt;Every time I merge a pair, I sum their distance. The final value of the algorithm is the total sum.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, I have to run it on a very large set of sequences.&#xA;I would like to have a fast, parallel, algorithm that provides an approximation or an upper bound of the sum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked for eigenvector techniques, but I could not find any suitable algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any advice?&lt;/p&gt;&#xA;" OwnerUserId="15170" LastEditorUserId="9667" LastEditDate="2015-04-23T23:38:36.910" LastActivityDate="2015-04-23T23:38:36.910" Title="Minimum effort merging of two sets" Tags="|algorithms|parallel-computing|eigenvalues|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19451" PostTypeId="2" ParentId="18943" CreationDate="2015-04-24T07:59:24.400" Score="2" Body="&lt;p&gt;There are two sources of error, and each of them will be different for your two discretizations.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Truncation error&lt;/strong&gt;, also referred to as discretization error.  This results from the fact that you approximate the Laplacian by a discrete stencil.  This error will remain even if you solve the linear system exactly; the only way to reduce it is to use a finer or more accurate discretization.  Since you haven't specified precisely the discretization you are using in each case, it is impossible to say by how much the discretization errors differ.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Error in the linear solver&lt;/strong&gt;.  Since you are computing in finite precision arithmetic, you will not obtain an exact solution to the linear algebraic system given by your discretization.  As you are aware, the influence of roundoff errors will be amplified by the conditioning of the problem, and also by the stability of the algorithm you use.  MATLAB's backslash operator uses a variety of algorithms, depending on the structure of the matrix in question.  It is possible that it is able to use an algorithm with better stability for the banded matrix, thus leading to a smaller error (even though the matrices have similar condition numbers).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It seems most likely that the difference you see is due to #1, but #2 cannot be ruled out based on what you have written.  Also, it may be helpful to consider a third possibility: that there is a bug in the unstructured grid code (which is necessarily more complicated than the structured-grid code).  It would be useful to quantify what you mean by &quot;much better&quot;.  If both solutions appear to converge to the same thing upon refinement, a bug is less likely.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-04-24T07:59:24.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19452" PostTypeId="5" CreationDate="2015-04-24T08:20:59.723" Score="0" Body="" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-04-24T08:20:59.723" LastActivityDate="2015-04-24T08:20:59.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19453" PostTypeId="4" CreationDate="2015-04-24T08:20:59.723" Score="0" Body="Matrices and operators that are not self-adjoint are termed non-Hermitian." OwnerUserId="123" LastEditorUserId="9667" LastEditDate="2018-04-22T18:41:25.060" LastActivityDate="2018-04-22T18:41:25.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19454" PostTypeId="1" CreationDate="2015-04-24T10:17:32.950" Score="5" ViewCount="1755" Body="&lt;p&gt;I have a model of a tall, slender structure that I am investigating using both shell and $3D$ frame elements.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The shell elements are type MITC4, $4$-node membrane elements.  The frame elements are the basic line ($1D$) elements found in any introductory structural analysis / stiffness method book - they include axial, bending, and torsion, but no shear deformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both analysis are linear-elastic (no geometric or material non-linearties), small-deformations using the direct-stiffness formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a test case, I loaded the structure with a unit load at the top node (or nodes, for the shell element model), once in a direction perpendicular to the long axis of the structure ($X$-axis), and once down the axis of the structure for pure axial loading ($Z$-axis).  I neglected all self-weight for this test case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[I also compared $X$ and $Y$ direction bending, which matched exactly as expected, since the structure is symmetric.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I compared these results with those found by simple hand computations for a linear-elastic beam:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Axial deformation ($Z$-axis) = $PL/(AE)$&#xA;Bending deformation ($X$-axis) = $PL^3 / (3EI)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the frame element model, these results matched exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the shell element model, the axial value was nearly identical (which I take as a good sign that the area is correct).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bending deformation for the shell element model were somewhat smaller (~8%).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to know is, is this expected behavior?  Should I expect the shell element model to be &quot;stiffer&quot; than the frame element in general?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the choice of Poisson's ratio in the shell element has an effect here as well.  I'm using $0.30$ for steel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards, Madeleine.&lt;/p&gt;&#xA;" OwnerUserId="15176" LastEditorUserId="690" LastEditDate="2016-08-30T19:30:08.793" LastActivityDate="2016-08-30T19:30:08.793" Title="Shell vs frame element model stiffness differences" Tags="|finite-element|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19455" PostTypeId="1" CreationDate="2015-04-24T10:53:02.873" Score="2" ViewCount="280" Body="&lt;p&gt;What are theoretical hurdles in applying Galerikin method on, say, first order time dependent ODE?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there no way we can form an inner product??&lt;/p&gt;&#xA;" OwnerUserId="5353" LastEditorUserId="5353" LastEditDate="2015-04-24T11:01:06.853" LastActivityDate="2015-04-24T17:21:12.083" Title="Finite element method for odd order DE" Tags="|finite-element|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19457" PostTypeId="1" AcceptedAnswerId="19460" CreationDate="2015-04-24T14:53:54.640" Score="2" ViewCount="586" Body="&lt;p&gt;I want to compute the value of an integral of a function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This function, however, is not given by a formula, say $f(x) \: \forall x \in [0,1]$, but is only known through its values on some given points, say&#xA;$f \left(\frac{k}{n} \right) \: \forall k \in \{0,\cdots,n\}$. (is the name for that scattered data ?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I do so ? I don't want to implement a quadrature method by hand (I know how to do so in a basic way). Methods refering to a scientific library would be prefered (e.g. GSL).&lt;/p&gt;&#xA;" OwnerUserId="7784" LastActivityDate="2015-04-24T20:18:46.627" Title="Numerical integration of a function whose expression is unknown" Tags="|time-integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19458" PostTypeId="1" CreationDate="2015-04-24T16:15:09.783" Score="4" ViewCount="2346" Body="&lt;p&gt;I do understand the meanning of &quot;conservative discretization&quot; within the FVM/FDM framework, indeed it is well explained in this &lt;a href=&quot;https://physics.stackexchange.com/questions/70496/conservation-vs-non-conservation-forms-of-conservation-equations&quot;&gt;post&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, according to the table in this &lt;a href=&quot;http://persson.berkeley.edu/pub/aachen2016.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slide&lt;/a&gt; (pp.8), it concludes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;FEM suffers from &lt;strong&gt;&quot;Stability for conservation laws&quot;&lt;/strong&gt;, while Discontinuous Galerkin (DG) is just fine&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I interpret this statement as, FEM(Galerkin) is a &lt;strong&gt;non-conservative discretization&lt;/strong&gt;, while&#xA;Discontinuous Galerkin is &lt;strong&gt;conservative&lt;/strong&gt;. Am I right ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anybody elaborate in mathematical sense the meaning of conservative/non-conservative in FEM/DG ?  ( I do have some experience with conventional FEM but not DG :-) )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Perhaps I should ask simply&lt;/strong&gt;: Is it possible to demonstrate whether it's a conservative discretization based upon any weak formulation(from conventional FEM/SUPG/DG)?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="4532" LastEditDate="2018-06-25T14:05:58.963" LastActivityDate="2018-06-25T14:05:58.963" Title="The meaning of conservative discretization in Galerkin FEM and Discontinuous Galerkin" Tags="|finite-element|finite-difference|finite-volume|discontinuous-galerkin|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19459" PostTypeId="2" ParentId="19455" CreationDate="2015-04-24T17:21:12.083" Score="3" Body="&lt;p&gt;I do not think that you can form an inner product there. Nevertheless, there are other methods that do not use the same space for test and trial functions, e.g., &lt;a href=&quot;https://en.wikipedia.org/wiki/Petrov%E2%80%93Galerkin_method&quot; rel=&quot;nofollow&quot;&gt;Petrov-Galerkin&lt;/a&gt; methods. The problem here is that the matrix is not symmetric anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use Least Squares FEM. For this you have&#xA;$$\mathcal{L}u=f\quad \text{in }\Omega\quad \text{and}\quad \mathcal{R}u=g\quad \text{on } \partial\Omega$$&#xA;where, $\mathcal{L}u=g$ and $\mathcal{R}u=g$ is the boundary condition. Then, you form the functional&#xA;$$J(u;f,g) =\Vert \mathcal{L}u-f\Vert^2 + \Vert \mathcal{R}u-g\Vert^2$$&#xA;and solve the minimization problem&#xA;$$\min_{u\in S} J(u;f,g)$$&#xA;and you end up with a system of the form&#xA;$$(\mathcal{L}v,\mathcal{L}u) + (\mathcal{R}v,\mathcal{R}u) = (\mathcal{L}v,f)  + (\mathcal{R}v,\mathcal{L}f) \forall\ v \in S \enspace .$$&#xA;A caveat here is that the norms and inner products are not in the same space. I have seen this method for Quantum Electrodynamics, where the &lt;a href=&quot;https://en.wikipedia.org/wiki/Dirac_equation&quot; rel=&quot;nofollow&quot;&gt;equation&lt;/a&gt; is first order, example &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/080729633&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-04-24T17:21:12.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19460" PostTypeId="2" ParentId="19457" CreationDate="2015-04-24T20:18:46.627" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I want to compute the value of an integral of a function.&lt;/p&gt;&#xA;&lt;p&gt;This function, however, is not given by a formula, say &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$ \forall{x}\in[0,1]$&lt;/span&gt;, but is only known through its values on some given points, say &lt;span class=&quot;math-container&quot;&gt;$f(k/n)$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\forall{k}\in{0,⋯,n}$&lt;/span&gt;. (is the name for that scattered data ?)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In general, if you have data as collections of points &lt;span class=&quot;math-container&quot;&gt;$(x_{k}, f(x_{k})$&lt;/span&gt;, the absolute simplest method of integrating the &amp;quot;curve&amp;quot; through those points (there are many such curves) is to assume a linear interpolant between adjacent points, which gives you a trapezoidal rule. As far as I know, the trapezoidal rule is not in GSL (or at least, not in the &lt;a href=&quot;http://www.gnu.org/software/gsl/manual/html_node/Numerical-Integration.html#Numerical-Integration&quot; rel=&quot;nofollow noreferrer&quot;&gt;numerical integration section&lt;/a&gt;). However, there are subroutines available in some libraries or software packages; &lt;code&gt;trapz&lt;/code&gt; &amp;amp; &lt;code&gt;cumtrapz&lt;/code&gt; in MATLAB come to mind. At worst, a trapezoidal rule wouldn't be onerous to implement by hand.&lt;/p&gt;&#xA;&lt;p&gt;Higher-order methods assume higher order interpolants, and also typically do not equispace the quadrature points due to &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%27s_phenomenon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Runge's phenomenon&lt;/a&gt;, which is one motivation for using methods with non-equispaced points (for instance, quadrature based at Chebyshev points, or adaptive quadrature methods). If you have the ability to query your function anywhere (it need not be given by a formula, but it must be computable for arbitrary inputs within your interval of integration), you should use one of these methods. If not, then I doubt there is much advantage to trying to use higher-order piecewise interpolants (e.g., splines), but I am not an expert on numerical integration.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-04-24T20:18:46.627" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19461" PostTypeId="2" ParentId="19458" CreationDate="2015-04-24T21:45:40.817" Score="4" Body="&lt;p&gt;No. The statement means that the finite element method, without modifications, will lead to numerical solutions for conservation laws that violate many of the physically reasonable constraints (e.g., that solutions have certain monotony properties, that entropy can only grow, of that the density can not become negative -- in mathematical language, the discrete solution at every time step does not satisfy stability estimates with the solution of the previous time step and boundary values as data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reasons are complicated and beyond what can be explained on this forum. But, if you allow me the conflict of interest, you can find more material in lecture 31 at &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-24T21:45:40.817" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19462" PostTypeId="2" ParentId="19091" CreationDate="2015-04-24T23:09:52.760" Score="9" Body="&lt;p&gt;We should be more precise here. The simplest estimate that you can give is that&#xA;$$&#xA;  ||x^* - x|| = ||A^{-1} A (x^* - x)|| \le ||A^{-1}||\,||b - A x||&#xA;$$&#xA;so that if you terminate your iteration using the residual, you can be off by a factor of $||A^{-1}||$, for the relative residual by&#xA;$$&#xA;  \kappa(A) = ||A||\,||A^{-1}||&#xA;$$&#xA;so you have a simple estimate of how many digits you are losing&#xA;$$&#xA;  \log\kappa.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2015-04-24T23:09:52.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19463" PostTypeId="1" AcceptedAnswerId="19465" CreationDate="2015-04-25T07:26:34.643" Score="3" ViewCount="9074" Body="&lt;p&gt;I'm trying to implement nonzero Displacement Boundary Conditions in &lt;a href=&quot;http://run.usc.edu/vega/&quot; rel=&quot;nofollow&quot;&gt;VegaFEM&lt;/a&gt; on a non-linear model, using the method outlined in §3.6.2 of &lt;a href=&quot;http://www.colorado.edu/engineering/cas/courses.d/IFEM.d/IFEM.Ch03.d/IFEM.Ch03.index.html&quot; rel=&quot;nofollow&quot;&gt;University of Colorado's intro to FEM&lt;/a&gt; (modify $f = Ku$: set desired displacements in $u$ and eliminate rows and columns in $K$ by calculating forces and applying to $f$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However it's not working as expected. I'm not sure exactly where in the solver source code I need to make the modifications to $f$, $K$ and $u$ but no matter where I do it the result is that either the model explodes or, if I make the displacements tiny, it seems to be having some influence in the correct direction (like an external force) but massively exaggerated and not anything resembling a 'boundary condition'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if I'm being thrown off because I'm assuming that the &quot;&lt;em&gt;tangent&lt;/em&gt; stiffness matrix&quot; in the VegaFEM integrator is just another name for the &quot;stiffness matrix&quot; in the Introduction to FEM notes. Is this assumption correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so - VegaFEM computes internal forces and then adds external forces before passing the forces on to the solver. Does the $f$ in $f = Ku$ refer to the external forces or the internal forces?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit&lt;/strong&gt; clarified linear vs nonlinear, and what 'not working as expected' means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit&lt;/strong&gt; The goal is effectively that I want to attach a highly elastic object to one or more virtual sticks which I can then drag around the simulation environment with the mouse in order to stretch the object. I'm currently simulating the boundary conditions (attachment to sticks) using PID controllers to apply external forces to the bound nodes, but this introduces another kind of elasticity that I'd rather avoid. Is there some other approach I should be considering?&lt;/p&gt;&#xA;" OwnerUserId="14561" LastEditorUserId="14561" LastEditDate="2015-04-25T14:38:24.690" LastActivityDate="2015-04-25T15:34:46.053" Title="Is &quot;tangent stiffness matrix&quot; the same as &quot;stiffness matrix&quot;?" Tags="|finite-element|boundary-conditions|nonlinear-equations|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19464" PostTypeId="2" ParentId="19463" CreationDate="2015-04-25T12:50:27.663" Score="3" Body="&lt;p&gt;From looking at the website, VegaFEM is a specific solver for nonlinear elasticity. Solving a nonlinear PDE requires an iterative procedure, where for every step you have to solve a linearized PDE, which is sometimes called &quot;tangent equation&quot;. The tangent stiffness matrix would then refer to the stiffness matrix obtained from discretizing the tangent equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the tangent equation always involves &lt;em&gt;homogeneous&lt;/em&gt; boundary conditions even if the nonlinear PDE has nonhomogeneous boundary conditions (conceptually, you want to use the tangent solution $\delta u$ as an update of the current iteration $u^k$, and $u^k+\delta u$ will not satisfy the correct boundary conditions otherwise).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-04-25T12:50:27.663" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19465" PostTypeId="2" ParentId="19463" CreationDate="2015-04-25T13:04:31.697" Score="7" Body="&lt;p&gt;$K u$ equals the internal forces only in the linear case. The tangent stiffness matrix, $K$, in a nonlinear problem is normally used in a Newton-Raphson algorithm to calculate updates to the displacement vector as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$K \Delta u = f - f_{internal}$$&#xA;$$ u_{i+1} = u_i + \Delta u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vector of internal forces, $f_{internal}$ must be calculated from the nonlinear element equations. The iteration continues until $u$ converges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a nonlinear problem, both the tangent $K$ and the $f_{internal}$ vector are functions of the displacements. One consequence of this is that to prescribe displacements at selected nodes, the displacement vector passed into the functions that calculate these must contain the prescribed values. The nonlinear solution algorithm typically begins with a solution vector of all zeros. Instead, selected entries can be set to the prescribed values so that the internal forces will be calculated correctly. If VegaFEM explicitly eliminates global equations for a constrained degree of freedom, it also should do&#xA;that for prescribed displacements. Those might be the only changes to the code needed to prescribe non-zero instead of zero displacements.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2015-04-25T15:34:46.053" LastActivityDate="2015-04-25T15:34:46.053" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19466" PostTypeId="2" ParentId="19454" CreationDate="2015-04-25T14:02:26.107" Score="3" Body="&lt;p&gt;There really aren't enough details about either your structure or your&#xA;shell FE model to be definitive but, from what you have provided, I would&#xA;say it is not all surprising that the results from your beam element and&#xA;shell element models differ significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You didn't say, but if your objective is to be able to accurately analyze&#xA;this structure using a beam model, I think that is definitely possible.&#xA;However, it might require a somewhat different approach than your current one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some things to consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How sure are you that your shell element model is giving accurate&#xA;(converged) results? In particular, are you sure your mesh is sufficiently&#xA;refined? I suggest building three shell FE models with 10, 20, and 30&#xA;shell elements down the length of the tube. These results should give you a good idea&#xA;of the correct displacement from the shell model.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I believe that the formula you are using to calculate the equivalent&#xA;moment of inertia for your polygonal-cross section tube assumes that the&#xA;wall thickness of each segment in the cross section is thin. That is, the&#xA;bending stiffness of the individual segments is ignored. Is this really&#xA;a valid assumption?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the wall thickness of the cross section segments is not small, this&#xA;also has a consequence in your shell model. At each node around the cross&#xA;section where shell elements connect, you have a small bit of overlapping&#xA;material. This seems like it should be possible to ignore this but it can&#xA;become important if you are trying to get close agreement between shell&#xA;and beam models. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For hollow &quot;tubes&quot;, like this one, the transverse shear stiffness is&#xA;low so you might need to include shear deformation in your beam&#xA;equation to compute an accurate deflection. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-04-25T14:02:26.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19467" PostTypeId="2" ParentId="41" CreationDate="2015-04-25T14:49:55.823" Score="3" Body="&lt;p&gt;Many times, the equations to be solved represent a physical conservation law.  For example, the Euler equations for fluid dynamics are representations of conservation of mass, momentum, and energy.  Given that the underlying reality that we are modeling is conservative, it is advantageous to choose methods that are also conservative&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also see something similar with electromagnetic fields.  Maxwell's laws include the divergence-free condition for the magnetic field, but that equation is not always used for the evolution of the fields.  A method that conserves this condition (for example: constrained transport) helps match the physics of reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;@hardmath pointed out that I forgot to address the &quot;what could go wrong&quot; part of the question (Thanks!).  The question specifically refers to engineers, but I'll provide a few examples from my own field (astrophysics) and hope that they help illustrate the ideas enough to generalize to what might go wrong in an engineering application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) When simulating a supernova, you have fluid dynamics linked to a nuclear reaction network (and other physics, but we'll ignore that).  Many nuclear reactions depend strongly on the temperature, which (to a first-order approximation) is some measure of the energy.  If you fail to conserve energy, your temperature will be either too high (in which case your reactions run much too fast and you introduce far more energy and you get a runaway that shouldn't exist) or too low (in which case your reactions run much too slow and you can't power a supernova).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) When simulating binary stars, you need to recast the momentum equation to be conservation of angular momentum.  If you fail to conserve angular momentum, then your stars cannot orbit each other correctly.  If they gain extra angular momentum, they separate and stop interacting correctly.  If the lose angular momentum, they crash into each other.  Similar issues occur when simulating stellar disks.  Conservation of (linear) momentum is desirable, because the laws of physics conserve linear momentum, but sometimes you have to abandon linear momentum and conserve angular momentum because that is more important to the problem at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to admit, despite citing the divergence-free condition of magnetic fields, I'm not as knowledgeable there.  Failure to maintain the divergence-free condition can generate magnetic monopoles (which we have no evidence of currently), but I don't have any good examples offhand of issues that might cause in a simulation.&lt;/p&gt;&#xA;" OwnerUserId="15167" LastEditorUserId="15167" LastEditDate="2015-04-26T13:30:45.503" LastActivityDate="2015-04-26T13:30:45.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19468" PostTypeId="1" AcceptedAnswerId="19490" CreationDate="2015-04-25T17:54:17.817" Score="1" ViewCount="2217" Body="&lt;p&gt;Is it better matplotlib interactive mode or matplotlib animation in terms of performance? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a continous flow of data coming from the serial port. I get 2 measures let's say var1 and var2. I need to plot them in a 3d plot adding the 3th dimension, the time. I've tried different approches based on ion, but i couldn't get what i want. I've seen many examples of animation, but no one is suitable for my purpose. I was arguing which one ion or animate is quicker, since the flow of data has a high frequency update. So i need to update data var1 as x and var2 as y and have a sliding z axis, the time is flowing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is there a something similar to &quot;set_xdata&quot; in a 3d plot?&lt;/p&gt;&#xA;" OwnerUserId="15187" LastEditorUserId="782" LastEditDate="2015-04-28T07:06:41.270" LastActivityDate="2015-04-28T07:42:48.770" Title="interactive mode vs animation - python" Tags="|python|" AnswerCount="1" CommentCount="10" FavoriteCount="0" ClosedDate="2015-04-28T15:13:14.190" ContentLicense="CC BY-SA 3.0" />
  <row Id="19470" PostTypeId="1" CreationDate="2015-04-25T19:06:36.620" Score="0" ViewCount="110" Body="&lt;p&gt;This is something I've been trying to figure out for a long time, and all I have is vague numerical results. I'm trying to answer the following question analytically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have a time dependent system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_t = L(x, u, u_x, u_{xx}) , u(t=0,x) = u_0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I approximate differences with some finite difference scheme, for instance $u_{xx}^j = \frac{1}{dx^2} (u^{j-1} +u^{j+1} -2u^{j} )$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I approximate each time step with an implicit FD method, for instance Runge-Kutta.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Question&lt;/strong&gt;: Given an orthonormal polynomial base $\{ p_j (x) \}_{j=0}^{\infty } $, and given that $ u_0 $ is as smooth as necessary, what can we say about the spectral coefficients  $u(x,t_f) = \sum\limits_j a_j p_j (x)$ for large $t_f$ ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is closely tied, if not determined by, how differentiable is $u(x,t)$ , but I'm not sure about that either.&lt;/p&gt;&#xA;" OwnerUserId="15188" LastEditorUserId="123" LastEditDate="2015-04-26T08:22:40.780" LastActivityDate="2015-05-28T07:29:07.150" Title="Spectral Coefficients of Implicit Finite Difference Solution" Tags="|finite-difference|numerical-analysis|runge-kutta|polynomials|implicit-methods|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19471" PostTypeId="1" CreationDate="2015-04-26T16:07:50.887" Score="1" ViewCount="356" Body="&lt;p&gt;I am currently working on 2D interpolation and I am using cubic splines. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 1D given N points one would use N+2 splines and would have to apply additional constraints in order to achieve a square system. For example requiring that the interpolant, S, second derivative is equal to zero ,Sxx = 0, at the end points (Natural Spline conditions) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering how one would extend this to 2D? I believe it should be something like Sxx = Syy = 0 at the edge of your domain but I am not sure (if so I believe we would still be missing some equations). Any help would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will provide an example in 2D: Consider the points 1:1:10 in the x component and 1:1:10 in the y component, thus N points in each direction. For cubic splines one need N+2 splines in each direction.  In 1D interpolation we can add the constraint that the second derivative should be zero to arrive at (N+2) equations for (N+2) unknowns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus to extend to 2D I will take the tensor product of these points I will have a 10x10 grid (100 pairs of points) and the tensor product of 12 splines in each direction (144 splines). Since I am using these for interpolation then I can retrieve 100 equations by setting up an equation for each point in my grid. I can retrieve an additional 40 equations by asking that S_{xx} = S_{yy} = 0 at the boundaries. 10 at $(x_0,y)$, $(x_N,y)$, $(x,y_0)$, $(x,y_N)$ Thus I would be missing 4 equations. This is where my confusion lies. &lt;/p&gt;&#xA;" OwnerUserId="15192" LastEditorUserId="15192" LastEditDate="2015-04-27T13:00:14.637" LastActivityDate="2015-04-27T13:00:14.637" Title="Natural spline conditions for 2D cubic splines" Tags="|image-processing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19475" PostTypeId="2" ParentId="19471" CreationDate="2015-04-27T00:18:50.443" Score="2" Body="&lt;p&gt;There are two different ways to extent cubic splines to 2D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more official approach is the tensor product approach. On the boundaries where $x$ is constant, you require $S_{xx}=0=S_{xxy}$. On the boundaries where $y$ is constant, you require $S_{yy}=0=S_{xyy}$. The condition $S_{xxy}=0$ can be derived from $S_{xx}(x_0,y)=0\ \forall y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The less official approach is the hierarchical basis or sparse grid approach. Here you just require $S_{xx}=0$ on the boundary grid points where $x$ is constant, and $S_{yy}=0$ on the boundary grid points where $y$ is constant. Then you use 1D splines to determine the partial derivative in $x$- and $y$-direction in all grid points. On a cell, you now have 12 prescribed degrees of freedom: the value and two derivatives in each corner of the cell. Linear interpolation takes care of the values, hence all you need for the derivatives it a form function which is zero on the four corners, and has partial derivative 1 in one direction in one corners, and all other partial derivatives zero in the corners: $x^2(x-1)y$. The resulting function will be continuous everywhere, and smooth along the axis parallel lines through grid points. So you don't have the complete spline smoothness, but the expected convergence order (of the approximation) should already be achieved.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2015-04-27T00:18:50.443" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19476" PostTypeId="1" CreationDate="2015-04-27T03:05:33.463" Score="4" ViewCount="188" Body="&lt;p&gt;I want to solve&#xA;$$\min_x f(x)\qquad \textrm{s.t.}\qquad g_i(x) \geq 0\ \ \textrm{or}\ \ h_i(x) \geq 0$$&#xA;for $i=1,\ldots,m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly if the inequality constraints split the feasible set into $2^m$ disconnected components, there is no hope for an efficient numerical solution. But in my case I know there is only one feasible connected component (consider for example a single pair of constraints, $g(x,y) = x$ and $h(x,y) = y$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some trick for converting these constraints to the standard conjunction of inequality constraints supported by black-box (nonlinear, nonconvex) optimization software? One can try for instance replacing the constraints with $k_i = \max(g_i,h_i)$ but since $k_i$'s gradient is not continuous I have little hope that this idea is effective in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Another idea (thanks to a quick discussion with Bernard Mourrain) is to introduce slack variables $y_i$ and solve&#xA;$$\min_{x,y} f(x)\quad \textrm{s.t.}\quad [g_i(x)-y_i][h_i(x)-y_i]=0,\ y_i \geq 0$$&#xA;which now has only conjunctions of constraints, albeit for increased number of constraints and variables. Is this approach viable?&lt;/p&gt;&#xA;" OwnerUserId="3612" LastEditorUserId="3612" LastEditDate="2015-05-07T19:30:56.573" LastActivityDate="2015-05-07T19:30:56.573" Title="optimization subject to disjunction of inequality constraints" Tags="|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19478" PostTypeId="1" CreationDate="2015-04-27T07:43:39.347" Score="4" ViewCount="212" Body="&lt;p&gt;Cross posted &lt;a href=&quot;https://mathematica.stackexchange.com/q/80828/1871&quot;&gt;in Mathematica.SE&lt;/a&gt;, I'll try to rephrase it in a more general way here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A friend of mine showed me this initial value problem (IVP) for a linear ordinary differential equation (ODE) with variable coefficient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y''(x)=\left(x^2-1\right) y(x)$$$$y(0)=1$$$$y'(0)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seems to be a simple one, right? Actually it can be solved analytically and the solution is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y(x)=e^{-\frac{x^2}{2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I tried to solve it with &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods#The_Runge.E2.80.93Kutta_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;the classical Runge–Kutta method&lt;/a&gt;, the numerical solution blows up:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ByH5k.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The step size used here is 0.001.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why did this happen? If I've chosen an improper method, what's a suitable one? &lt;/p&gt;&#xA;" OwnerUserId="5331" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2015-04-29T15:31:00.480" Title="Numerical solution of IVP for linear ODE with variable coefficient blows up" Tags="|ode|stability|time-integration|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19479" PostTypeId="2" ParentId="19429" CreationDate="2015-04-27T12:55:36.400" Score="3" Body="&lt;p&gt;Only a partial answer:&lt;/p&gt;&#xA;&lt;p&gt;I managed to take a look into Kutta’s &lt;em&gt;&lt;a href=&quot;https://archive.org/details/zeitschriftfrma12runggoog/page/438/mode/2up?view=theater&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beitrag zur näherungweisen Integration totaler Differentialgleichungen&lt;/a&gt;,&lt;/em&gt; which does indeed mention and derive the 3&lt;sup&gt;rd&lt;/sup&gt;-order method, giving no other source than relating it to &lt;a href=&quot;http://en.wikipedia.org/wiki/Simpson%27s_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simpson’s rule&lt;/a&gt;. Thus I deem it appropriate to call it &lt;em&gt;Kutta’s third-order method.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2023-08-18T08:14:21.327" LastActivityDate="2023-08-18T08:14:21.327" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19480" PostTypeId="1" AcceptedAnswerId="19484" CreationDate="2015-04-27T12:59:48.687" Score="1" ViewCount="530" Body="&lt;p&gt;When I run the following (in Matlab) on a sparse matrix $A$, I get larger band width. The &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/symrcm.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;symrcm&lt;/code&gt;&lt;/a&gt; (symmetric reverse Cuthill-McKee permutation) is not guarenteed to find the smallest band width, but it makes no sense for it to increase. What is wrong here?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;reorderingForSmallBand = symrcm(A);&#xA;A = A(reorderingForSmallBand,reorderingForSmallBand);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running &lt;code&gt;spy(A)&lt;/code&gt; on $A$ before and after reordering yields: &lt;a href=&quot;http://folk.ntnu.no/jonvegar/spyAcuttedPermuted.pdf&quot; rel=&quot;nofollow&quot;&gt;Before&lt;/a&gt; (PDF) and &lt;a href=&quot;http://folk.ntnu.no/jonvegar/spyAcuttedPermutedSmallBand.pdf&quot; rel=&quot;nofollow&quot;&gt;After&lt;/a&gt; (PDF)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Originally, my matrix $A$ has &lt;a href=&quot;http://folk.ntnu.no/jonvegar/spyAcutted.pdf&quot; rel=&quot;nofollow&quot;&gt;this form&lt;/a&gt; (PDF). I also wonder if it is a big deal to have large bandwidth as the backslash solver in Matlab says it is above the limit for a solver exploiting the banded structure. Is it any advantage for an iterative solver to have small band width?&lt;/p&gt;&#xA;" OwnerUserId="15159" LastEditorUserId="4474" LastEditDate="2015-04-27T17:47:29.753" LastActivityDate="2015-04-27T19:44:56.133" Title="Why does `symrcm` create larger band width?" Tags="|matlab|sparse-matrix|banded-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19481" PostTypeId="1" CreationDate="2015-04-27T13:05:56.867" Score="4" ViewCount="1072" Body="&lt;p&gt;This question is based more on the theory of neural networks than my particular implementation. Therefore I will leave out my code unless requested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on a project in C# which can create and train feed forward neural networks. It works well and can train networks to perform some simple functions such as squaring the input or other basic math functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My next task was to use this to train a network to predict an outcome of positive or negative based on 39 continuous input variables. I have a dataset comprised of around 7000 training examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a ROC curve analysis on each input variable and found that some are very good predictors and some are almost useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I sorted the variables by AUC, the best was around 78%. I created a network with only this variable as it's input and one hidden layer with one neuron. After training, the network performed better than random but not amazingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I next added the second best variable by AUC (75%) as another input, keeping everything else the same. After training the network's fitness score was about 1.5 times better than with one input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next I added the third best variable and trained. The networks fitness score was only around 85% of that when it had 2 inputs. I have tried adding more neurons to the hidden layer, running the training for more iterations and many different runs of initial random weights and biases and it seems that no matter what I do the network will perform worse with 3 inputs than 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find this confusing as neural networks are theoretically able to replicate any function. So even if the 3rd variable I introduced was a rubbish predictor then the training should just be able to set the weight of that input to zero and it will perform the same as it did with two inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is my assumption correct that adding inputs to a neural network (and given enough training iterations) it should never perform worse than with less inputs? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do I require more neurons in the hidden layer when adding more inputs for assumption 1 to hold? Would a good guideline be one neuron in the hidden layer for every input?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it a fair assumption that, given everything I have tried, my training code has not been written well enough to train networks to their full capability? (Or in other words, it should be easy for the networks to just ignore the third input, so the problem is more likely to be that my training code is not good enough rather than the networks are not able to easily ignore the third input)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it a good method to create networks with successively more inputs to see which variables improve the fitness the most, and then not use those that don't improve the network? Or should I just create a network with all 39 input variables, train it for a long time and then perform some kind of analysis on the resulting weights to see which inputs are being weighted close to zero and remove them? (as they are essentially being ignored by the network).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Extra details: Each neuron in the hidden layer is connected to every input and the output neuron is connected to every neuron in the hidden layer. All neurons use a sigmoid function with weights and a bias. If the output of the output neuron is &gt;0.8 then this is taken as the network predicting positive, otherwise negative.&lt;/p&gt;&#xA;" OwnerUserId="15201" LastActivityDate="2015-04-27T13:05:56.867" Title="Neural network performs worse when using more input variables" Tags="|regression|data-management|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19482" PostTypeId="2" ParentId="19478" CreationDate="2015-04-27T17:45:46.210" Score="3" Body="&lt;p&gt;This is a second order ODE so there are two solutions. The second solution&#xA;is unstable in the sense that it goes either to plus or minus infinity as $x$ becomes large. For $x$ near zero the solution is obviously stable. Even if this solution is the analytic one you show above, in the numerical integration algorithm, due to round off error, a component of the second solution will creep in. Eventually, this component will grow and dominate the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that all standard integration algorithms for solution of initial value ODE will exhibit this behavior.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-04-27T17:45:46.210" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19483" PostTypeId="2" ParentId="19478" CreationDate="2015-04-27T18:18:39.027" Score="1" Body="&lt;p&gt;Your analytical solution assumes that $y&amp;gt;=0$ for $x&amp;gt;0$. However, for initial conditions where $y&amp;lt;0$, $y(x) = -e^{-x^2/2}$. Analyzing the stability of this system in detail is likely off-topic, and would be better suited for a mathematics site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution asymptotically converges to $0$ exponentially. Numerically, most integration methods will have difficulty with this. Numerical noise around small values can cause a solver to take a step that is too large, causing $y(x)$ to go negative. One way to get around this is to use a solver that is able to intelligently constrain solution components to be nonnegative. Doing this correctly isn't trivial – see &lt;a href=&quot;http://dx.doi.org/10.1016/j.amc.2004.12.011&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shampine, et al. 2005&lt;/a&gt;. Matlab's ODE solvers, e.g., &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ode45.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ode45&lt;/code&gt;&lt;/a&gt;, have this feature via the &lt;code&gt;'NonNegative'&lt;/code&gt; &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/odeset.html#ode_solver_output_properties&quot; rel=&quot;nofollow noreferrer&quot;&gt;output property&lt;/a&gt; that can be set with &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/odeset.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;odeset&lt;/code&gt;&lt;/a&gt;. I don't think Mathematics's &lt;a href=&quot;https://reference.wolfram.com/language/ref/NDSolve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;NDSolve&lt;/code&gt;&lt;/a&gt; has such an option. This &lt;a href=&quot;https://mathematica.stackexchange.com/questions/45727/constraining-function-found-by-ndsolve-to-stay-positive&quot;&gt;Mathematica.StackExchange post&lt;/a&gt; offers several ways you could try to add it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a quick example in Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = @(x,y)[y(2);(x^2-1)*y(1)];&#xA;opts = odeset('NonNegative',1); % Keep y(1) &amp;gt;= 0&#xA;[x,y] = ode45(f,[0 10],[1 0],opts);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Plotting, you get something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gyaDg.png&quot; alt=&quot;plot of simulation ouput&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2015-04-27T18:33:02.063" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19484" PostTypeId="2" ParentId="19480" CreationDate="2015-04-27T19:44:56.133" Score="5" Body="&lt;p&gt;Nothing is wrong here. The Cuthill-McKee algorithm is a greedy algorithm, and doesn't depend too much on the order of A on input. The reverse Cuthill-McKee algorithm is often used to produce nice orders for skyline solvers, and the skyline of the reordered matrix looks indeed quite reasonable. (The bandwidths of Cuthill-McKee and reverse Cuthill-McKee is the same, if I remember correctly.) But I don't know whether Matlab has a special skyline solver. It certainly has banded solvers, because LAPACK has banded solvers. LAPACK didn't have skyline solvers in the past, so it is quite likely that Matlab has neither. Matlat however has general sparse direct solvers, but the reverse Cuthill-McKee ordering is not too useful for those.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2015-04-27T19:44:56.133" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19485" PostTypeId="1" AcceptedAnswerId="19508" CreationDate="2015-04-27T21:09:42.647" Score="3" ViewCount="953" Body="&lt;p&gt;I am trying to write a 2D Navier Stokes solver using an RK3 for time advancement in python. For debugging, I have converted the RK3 to an Euler step for simplicity. Checking my divergence for my predictor velocity I find that the divergence is acceptably close to zero. I then go to my pressure solver and the solution is incorrect. It does not even follow the same pattern as the exact solution. Can anyone see where my error is?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code isn't the most efficient, but I'm not worried about that currently just trying to get a physically correct answer. The analytical solutions for the u, v, and p are given by &#xA;$$u = -e^{-2t}\cos x \sin y$$&#xA;$$v = -e^{-2t}\sin x \cos y$$&#xA;$$p = -\frac{e^{-4t}}{4} \left(\cos 2x + \sin 2y \right)$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def Poisson_Solver_Neumann(u, v, Nx, Ny, dt, dx, dy, T, xp, yp):&#xA;&quot;&quot;&quot;Solves the 2D Poisson equation implicitly on a staggered grid&#xA;using Neumann Boundary Conditions&#xA;&#xA;Params:&#xA;------&#xA;u, v           2D array of float, x and y velocities&#xA;Nx, Ny         float, Number of segments in x and y&#xA;dt             float, time step size&#xA;T              float, current time&#xA;X, Y           2D array of float, meshgrid&#xA;&#xA;Returns:&#xA;-------&#xA;ANeum        2D array of float, A matrix with Neumann conditions&#xA;f_RHSn       1D array of float, f(x,y) for Neumann conditions&#xA;&quot;&quot;&quot;&#xA;#Building A&#xA;ANeum = np.zeros((Nx*Ny,Nx*Ny),dtype=float)&#xA;&#xA;a = Nx * Ny&#xA;b = Nx * Ny&#xA;c_int = -4 * (Nx * Ny)&#xA;c_edge = -3 * (Nx * Ny)&#xA;c_corner = -2 * (Nx * Ny)&#xA;d = Nx * Ny&#xA;e = Nx * Ny&#xA;&#xA;#Set corner points&#xA;ANeum[0,0] = c_corner&#xA;ANeum[-1,-1] = c_corner&#xA;ANeum[Nx*Ny-Ny,Nx*Ny-Ny] = c_corner&#xA;ANeum[Ny-1,Ny-1] = c_corner&#xA;&#xA;#Set edges in first block&#xA;for j in range(1,Ny-1):&#xA;    ANeum[j,j] = c_edge&#xA;    j +=j&#xA;&#xA;#Set edges in last block&#xA;for j in range((Nx*Ny)-Ny,(Nx*Ny)-2):&#xA;    ANeum[j+1,j+1] = c_edge&#xA;    j +=j&#xA;&#xA;#Set edges along main diagonal except for first block&#xA;for j in range(Nx+1,Ny*Nx):&#xA;    if j % Nx ==0:&#xA;        ANeum[j-1,j-1] = c_edge&#xA;    j +=j&#xA;&#xA;#Set edges on main diagonal except for last block&#xA;for j in range(Nx,(Ny*Nx)-Nx):&#xA;    if j % Nx ==0:&#xA;        ANeum[j,j] = c_edge&#xA;    j +=j&#xA;&#xA;#Second diagonal above and below diagonal&#xA;for j in range(Ny,Ny*Nx):&#xA;    ANeum[j,j-Ny] = a&#xA;    ANeum[j-Nx,j] = e&#xA;    j +=j&#xA;&#xA;#first diagonal below main diagonal   &#xA;for j in range(1,Ny*Nx):&#xA;    if j % Ny ==0:&#xA;        ANeum[j,j-1] = 0&#xA;    else:&#xA;        ANeum[j,j-1] = b&#xA;    j +=j&#xA;&#xA;#first diagonal above main diagonal&#xA;for j in range(0,Ny*Nx):&#xA;    if j % Nx ==0:&#xA;        ANeum[j-1,j] = 0&#xA;    else:&#xA;        ANeum[j-1,j] = d&#xA;    j +=j&#xA;&#xA;#Main Diagonal&#xA;for j in range(0,Ny*Nx):&#xA;    if ANeum[j,j] ==0:&#xA;        ANeum[j,j] = c_int&#xA;&#xA;#Setting random point to Dirichlet&#xA;ANeum[0,:] = 0&#xA;ANeum[0,0] = 1&#xA;&#xA;return ANeum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My Euler step is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for t in range(0,nt):&#xA;un = np.empty_like(u)&#xA;vn = np.empty_like(v)&#xA;#copy info from last loop, needed for current loop into variable_n&#xA;un = u.copy()&#xA;vn = v.copy()&#xA;#Pn = p.copy()&#xA;T = T_0 + t*dt&#xA;&#xA;####STEP 1&#xA;### t -----&amp;gt; t + (1/3)*dt&#xA;un, vn = BCs(un, vn)&#xA;G1 = Build_G1(un, vn, dx, dy, nu)&#xA;G2 = Build_G2(un, vn, dx, dy, nu)&#xA;u1 = un + (dt*G1) #put 1/3 back in&#xA;v1 = vn + (dt*G2)&#xA;u1, v1 = BCs(u1, v1)&#xA;u_div = (u1[1:-1,2:-1] - u1[1:-1,1:-2])/dx&#xA;v_div = (v1[2:-1,1:-1] - v1[1:-2,1:-1])/dy&#xA;#Solve for Pressure Field&#xA;b1 = np.zeros((Ny+2,Nx+2), dtype=float)&#xA;b1[1:-1,1:-1] = (u1[1:-1,2:-1] - u1[1:-1,1:-2])/dx +\&#xA;    (v1[2:-1,1:-1] - v1[1:-2,1:-1])/dy&#xA;b1 = np.reshape(b1[1:-1,1:-1], Ny*Nx)&#xA;A1 = Poisson_Solver_Neumann(u1, v1, Nx, Ny, dt, dx, dy, T, xp, yp)&#xA;b1 = b1[:]*(1.0/dt)&#xA;temp1 = np.linalg.solve(A1,b1)&#xA;&#xA;p_star = np.reshape(temp1, (Ny,Nx))&#xA;&#xA;&#xA;F1_Pstar, F2_Pstar = Pressure_Terms(p_star, dx, dy)&#xA;&#xA;#calc predictor velocities&#xA;u_star = u1.copy()&#xA;v_star = v1.copy()&#xA;&#xA;u_star[1:-1,2:-2] = u1[1:-1,2:-2] - (dt*F1_Pstar)&#xA;v_star[2:-2,1:-1] = v1[2:-2,1:-1] - (dt*F2_Pstar)&#xA;&#xA;u = u_star.copy()       &#xA;v = v_star.copy()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Update: My code is now working for the first time step but subsequently blows up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Some code changes&lt;/p&gt;&#xA;" OwnerUserId="15205" LastEditorUserId="15205" LastEditDate="2015-04-27T22:18:05.417" LastActivityDate="2015-04-30T00:54:41.053" Title="2D Poisson Solver for Taylor Green Vortex Problem" Tags="|fluid-dynamics|python|poisson|navier-stokes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19486" PostTypeId="1" CreationDate="2015-04-27T23:45:05.150" Score="1" ViewCount="109" Body="&lt;p&gt;I'm working with the Boundary Element Method and want to integrate an expression over a triangular region. I would like to use Gauss Integration to do this, but I'm having trouble since the triangular element is zero order(1 degree lower than a constant stress triangle). Unlike the higher order elements, the zero order triangle has only one node at its centroid. Normally, Gauss Integration is carried out over a triangular element by defining a natural coordinate system and corresponding shape functions/Jacobin. In the case of the zero order element, however, their is one shape function = 1. Therefore the shape functions can not be expressed in terms of the natural coordinates, and by extension, in terms of the Gauss Points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to integrate an expression over a zero order triangular element using Gauss Integration? If not does anyone know a different numerical integration scheme to integrate over a triangular region by just plugging in points and weights. I don't want to write out both integrals and then apply a numerical method to each one. I would also like to avoid defining the limits of integration. &lt;/p&gt;&#xA;" OwnerUserId="15207" LastActivityDate="2015-04-27T23:45:05.150" Title="Gauss Integration over Zero Order Element" Tags="|finite-element|interpolation|numerical-modelling|integration|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19487" PostTypeId="1" CreationDate="2015-04-28T02:53:27.193" Score="1" ViewCount="41" Body="&lt;p&gt;Consider two functions &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X=\sum_{n=-N}^N i\ sign(n) y_n e^{-in\xi}; \quad \quad Y = \sum_{n=-N}^N y_n e^{-in\xi}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $y_n$ are the dependent variables of the system, and are functions of time. sign(n) is $1,0,-1$ for n positive, 0 and negative respectively.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the physics of my system, I find that the Lagrangian contains terms like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{2\pi}\int_0^{2\pi} \frac{\partial X}{\partial \xi} Y^2 \ d\xi = \sum_{j+k+l=0} |j|y_jy_ky_l$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $(j,k,l)\in (-N,N)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then apply the Euler-Lagrange equations, which take the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d}{dt} \frac{\partial L}{\partial \dot{y_k}} - \frac{\partial L}{\partial y_k}=0 \quad \quad k=\pm 1,\pm 2,... ,\pm N,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $L$ my Larangian to get my governing system of couples ODEs in the dependent variables $y_k$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are then solved numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this - does one need to worry about aliasing errors in the sample terms shown above as one normally does for pseudospectral models (even though here my adventure occurs entirely in spectral space)? e.g. should I be cleaving away the highest 1/3 of my modes? &lt;/p&gt;&#xA;" OwnerUserId="8021" LastEditorUserId="8021" LastEditDate="2015-04-28T05:01:25.863" LastActivityDate="2015-04-28T05:01:25.863" Title="Aliasing question for a spectrally derived Lagrangian" Tags="|spectral-method|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19488" PostTypeId="2" ParentId="19476" CreationDate="2015-04-28T02:58:24.633" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there some trick for converting these constraints to the standard conjunction of inequality constraints supported by black-box (nonlinear, nonconvex) optimization software?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You could introduce binary variables for each logical statement. For instance, you add a binary variable $z_{i}$ for each $i = 1, \ldots, m$, and have $\min_{x, z}f(x)$ such that $g_{i}(x)z_{i} + h_{i}(x)(1-z_{i}) \geq 0$, where $z_{i} = 1$ if the $g_{i}$ constraint is satisfied, and $z_{i} = 0$ if the $h_{i}$ constraint is satisfied. The resulting problem will be mixed-integer, so you would need to use an MINLP solver.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One can try for instance replacing the constraints with $k_{i}=\max(g_{i},h_i)$ but since $k_i$'s gradient is not continuous I have little hope that this idea is effective in practice.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I agree. I am not an expert in nonsmooth methods, but it seems as though there are more MILP and MINLP solvers available than nonsmooth optimization solvers. If you elect to use a nonsmooth formulation, Overton has some papers that describe how BFGS methods appear to work fairly well in practice, and he provides a line-search criterion for an appropriate step size. However, the problems he and his coauthors solve tend to be no more than 2000ish variables; the number is impressive for nonsmooth problems, but I am aware of larger problem instances in real applications.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-04-28T02:58:24.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19489" PostTypeId="2" ParentId="19470" CreationDate="2015-04-28T06:16:07.400" Score="1" Body="&lt;p&gt;It depends on how well resolving your mesh is. On some level of coarseness you will first observe $2h$ waves (smallest resolvable waves on your mesh), and soon after, your solution will blow-up. That means that in spectral representation of your solution, coefficients of higher modes start to grow due to aliasing and spectral blockage. You first observe that growth of higher coefficients when you see mentioned waves in solution. What you need to do, in that case, is to refine the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you might be interested in Chapter 11 of J.Boyd- &quot;Fourier and Chebyshev Spectral Methods&quot;, Dover, 2001.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="1050" LastEditDate="2015-04-28T06:34:05.507" LastActivityDate="2015-04-28T06:34:05.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19490" PostTypeId="2" ParentId="19468" CreationDate="2015-04-28T07:42:48.770" Score="1" Body="&lt;p&gt;&lt;code&gt;matplotlib.animation&lt;/code&gt; is a nice approach because you can specify the frame or update rate. Here you have an example of a 3d animation, including a comment about &lt;code&gt;set_data&lt;/code&gt; for 3D plots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://matplotlib.org/1.4.3/examples/animation/simple_3danim.html&quot; rel=&quot;nofollow&quot;&gt;http://matplotlib.org/1.4.3/examples/animation/simple_3danim.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you have another example plotting data coming from an Arduino board:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/electronut/d5e5f68c610821e311b0&quot; rel=&quot;nofollow&quot;&gt;https://gist.github.com/electronut/d5e5f68c610821e311b0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, using your own logic mixing &lt;code&gt;plt.ion&lt;/code&gt;, &lt;code&gt;plt.draw&lt;/code&gt; and such is more complicated than using &lt;code&gt;matplotlib.animation&lt;/code&gt;, so I recommend you to use the latter because it is specifically meant for this kind of task and &lt;em&gt;I guess&lt;/em&gt; the performance would be better.&lt;/p&gt;&#xA;" OwnerUserId="782" LastActivityDate="2015-04-28T07:42:48.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19492" PostTypeId="1" CreationDate="2015-04-28T12:41:21.420" Score="0" ViewCount="84" Body="&lt;p&gt;I am facing a problem solving a mechanical system which is shown in the scheme below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QPPuT.png&quot; alt=&quot;system&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We assume that coupling of the two shafts is done with a clutch&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J1*\frac{d\omega1}{dt} + Td(\omega12) + Ts(\phi12) = T_1$$&#xA;$$ J2*\frac{d\omega2}{dt} - Td(\omega12) - Ts(\phi12) = T_2$$&#xA;    where $$w1 = \frac{d\phi_1}{dt}$$ &#xA;        $$w2 = \frac {d\phi_2}{dt} $$&#xA;        $$\phi12 = \phi1 - \phi2$$&#xA;        $$\omega12 = \omega_1 - \omega2$$&#xA;        $$T_d(\omega12) = c12 * \omega12$$&#xA;        $$T_s(\phi12) = k_s * \phi12$$&#xA;    $c_12$ and $k_s$ are some coefficients,&#xA;So the equations have the form of:&#xA;$$\frac{d\phi12}{dt} = \omega_12$$&#xA;$$\frac{d\omega12}{dt} = \frac{T_1}{J_1} - \frac{T_2}{J_2} - \frac{T_d(\omega12)}{Jeq} - \frac{T_s(phi12)}{Jeq}$$&#xA;    and also i have the dependencies:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ccr = 2*Jeq*\omega_n$$&#xA;$$\omega_n = \sqrt{\frac {k_s}{Jeq}}$$&#xA;    $$Jeq = \frac{J1*J2}{J1+J2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My task is to plot how nonlinear clutch affects the rotational inertia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_1(t) = T_0*1(t), T_0 = 1, T_2(t) = 0,$$&#xA;$$J_1 = J_2 = 5, c12 = 0, \omega_n = 100$$&#xA;$$T_s(\phi12) = k_1*\phi12 + k_2*(\phi12)^3$$&#xA;$$ k20 = 10e14, k2 = [10^-6 ... 10^4]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to plot the $$ \frac{Tsmax}{T_0} = f(log(\frac {k2}{k20}))$$;&#xA;So far my progress is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/spK2A.png&quot; alt=&quot;matlab_plot_1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But i dont think that the system actually behaves that way. I mean, i expected a lilte more stable curve without such a chaotic behaviour.&#xA;Whatsmore i tried to solve it in simscape but got really confused.&#xA;Any help is more than appreciated.&lt;/p&gt;&#xA;" OwnerUserId="15212" OwnerDisplayName="sayid jetzenden" LastEditorUserId="15212" LastEditDate="2015-04-29T09:30:35.700" LastActivityDate="2015-04-30T09:04:45.167" Title="Mechanics - Rotational Inertia" Tags="|computational-physics|software|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19493" PostTypeId="1" CreationDate="2015-04-28T15:19:50.830" Score="3" ViewCount="134" Body="&lt;p&gt;I want a numerical method to evaluate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\log \int_a ^b f(x) \mathrm{d}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when what I have is a numerical routine to evaluate $\log f(x)$. The problem is that if $f(x)$ takes very large or very small values, direct evaluation of $f(x)$ will lead to overflows/underflows. I am wondering if anyone has approached this numerical integration problem in general?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that even though $f(x)$ can be very small in some regions and very large in other regions, you can't just approximate the small values to zero, because if the region where $f(x)$ is very small is large enough, it may contribute significantly to the integral.&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2015-04-30T13:12:08.507" LastActivityDate="2015-04-30T13:12:08.507" Title="Numerical computation of $\log \int_a ^b f(x) \mathrm{d}x$ from $\log f(x)$?" Tags="|reference-request|integration|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19494" PostTypeId="2" ParentId="19493" CreationDate="2015-04-28T16:48:51.060" Score="1" Body="&lt;p&gt;You say that $f(x)$ has both very large and very small (positive) values. Then the integral you have is &quot;large&quot;, and the small values of $f(x)$ don't matter very much: in those areas where $f(x)$ would produce an underflow, you might as well add zeros because the contributions of this to the overall integral do not matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what matters are those areas where $f(x)$ is so large that you get overflows. This must happen because you have some kind of singularity at these locations. If so, it would be worth splitting $f(x)$ into its singular and non-singular parts and integrating the singular parts analytically if possible, and the remainder term with quadrature.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-28T16:48:51.060" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19496" PostTypeId="1" AcceptedAnswerId="19499" CreationDate="2015-04-28T21:29:36.153" Score="8" ViewCount="240" Body="&lt;p&gt;The equations of motion for an elastic solid are given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;\nabla \cdot \boldsymbol{\sigma} + \mathbf{f} = \rho \ddot{\mathbf{u}}\\&#xA;&amp;amp;\boldsymbol{\sigma} = \mathbb{C}\boldsymbol{\varepsilon}\\&#xA;&amp;amp;\boldsymbol{\varepsilon} = \frac{1}{2}\left(\nabla \mathbf{u} + [\nabla\mathbf{u}]^T\right)&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or in index notation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;\sigma_{ij,j} + f_i = \rho \ddot{u_i}\\&#xA;&amp;amp;\sigma_{ij} = C_{ijkl}\varepsilon_{kl}\\&#xA;&amp;amp;\varepsilon = \frac{1}{2}(u_{i,j} + u_{j,i})&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{u}$ is the displacement vector, $\mathbf{f}$ is the body force (source term), $\boldsymbol{\sigma}$ is the stress tensor, $\mathbf{\varepsilon}$ is the strain tensor, and $\mathbb{C}$ is the stiffness tensor. In the case of isotropic solids the stiffness tensor is written in terms of &lt;a href=&quot;https://en.wikipedia.org/wiki/Lam%C3%A9_parameters&quot; rel=&quot;nofollow&quot;&gt;two different constants&lt;/a&gt;, for unbounded domains, the equation admits two type of waves that are uncoupled and the criterion for stability is given by the worst case of the two different cases (i.e., the one with higher speed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;a href=&quot;https://en.wikipedia.org/wiki/Transverse_isotropy&quot; rel=&quot;nofollow&quot;&gt;transverse isotropic&lt;/a&gt; materials there are 5 independent parameters that define the tensor, and 3 types of waves (2 of them are coupled). In the more general case the number of parameters is 21 and the wave are coupled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; How do you find the criterion for stability in a time marching algorithm for the general case?&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2016-09-09T18:37:10.140" LastActivityDate="2017-03-22T19:53:41.167" Title="Stability criterion for waves in anisotropic solids" Tags="|pde|numerical-analysis|finite-difference|cfl|solid-mechanics|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19499" PostTypeId="2" ParentId="19496" CreationDate="2015-04-29T06:28:38.197" Score="6" Body="&lt;p&gt;Wave equations like this can be rewritten as a hyperbolic system of first-order conservation laws:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q_t + \nabla \cdot F(q) = 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stable time step for any explicit numerical discretization depends on the CFL number, which is proportional to the maximum wave speed appearing in the problem.  That speed can be found by computing the eigenvalues of the Jacobian of the flux function, $F$ and taking the largest (in absolute value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In $d$ dimensions, $F$ has $d$ components and the proper analysis requires finding the eigenvalues of arbitrary linear combinations of those components.  But for most systems, including elasticity, it is sufficient to look at the eigenvalues of each of the components of $F$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice reference for this theory is &lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot; rel=&quot;noreferrer&quot;&gt;LeVeque's text on finite volume methods&lt;/a&gt;.  Elasticity is covered in detail in Chapter 22.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the usual caveats regarding the CFL condition apply: it is a necessary but not usually sufficient condition for stability.  But the sufficient condition for stability of a given discretization is generally given by the CFL condition multiplied by some constant.  To find your stability criterion, you need to know both the maximum wave speed (based on the equations you're solving) and that constant (based on the discretization you're using).&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-04-29T06:28:38.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19500" PostTypeId="1" AcceptedAnswerId="19658" CreationDate="2015-04-29T12:35:58.347" Score="1" ViewCount="139" Body="&lt;p&gt;I am currently working with a optimization problem involving a non-linear least square problem. I have chosen to use &lt;code&gt;lsqnonlin&lt;/code&gt; in Matlab. What follows is a simplified short description of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $P_i=(x_i,y_i)$ $i=1,...,n$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_1=(a_1,b_1)$, $A_2=(a_2,b_2)$ be a set of points which are known. Let $\tilde{R}_1$ ans $\tilde{R}_2$ be an unknown set of variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d_i = \Vert P_i-A_1\Vert-\Vert P_i-A_2\Vert-(\Vert P_i-\tilde{R}_1\Vert -\Vert P_i-\tilde{R}_2\Vert)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And non-linear-least square problem is then defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min\sum_{i=1}^{n} d_{i}^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This initial setup works very well hovever I am now in need to introduce random errors on the $P_i$ variables. This makes the system very difficult to solve and my question is if any of you have any advise on how to best adapt for these errors? Is there maybe an alternative better way to pose the problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if anything is unclear.&lt;/p&gt;&#xA;" OwnerUserId="8127" LastEditorUserId="9667" LastEditDate="2015-04-30T01:53:48.830" LastActivityDate="2015-05-16T15:25:13.327" Title="Dealing with errors in non-linear least square problem" Tags="|optimization|matlab|nonlinear-equations|nonlinear-programming|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19501" PostTypeId="1" AcceptedAnswerId="19502" CreationDate="2015-04-29T14:32:49.867" Score="2" ViewCount="138" Body="&lt;p&gt;I am trying to numerically solve a simple second order differential equation $x'' = -x$. I used a new variable $x'=v$, so I have two equations. While it seems simple, it somehow produces a result that's really far of the correct result.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(x):&#xA;    return -x&#xA;&#xA;def rk4(f=f,h=2*pi/100,x0=1,v0=0,t0=0,n=100):&#xA;    '''RK4'''&#xA;    v=[v0]&#xA;    x=[x0]&#xA;    for i in range(n-1):&#xA;        v1= h*f(x[-1])&#xA;        x1= h *(v[-1])&#xA;        v2= h*f(x[-1]+1/2*x1)&#xA;        x2= h *(v[-1]+1/2*v1)&#xA;        v3= h*f(x[-1]+1/2*x2)&#xA;        x3= h *(v[-1]+1/2*v2)&#xA;        v4= h*f(x[-1]+x3)&#xA;        x4= h *f(v[-1]+v3)&#xA;        v.append(v[-1]+1/6 *(v1+2*v2+2*v3+v4))&#xA;        x.append(x[-1]+1/6 *(x1+2*x2+2*x3+x4))&#xA;    return x,v&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The weird thing is that if I use RK45 coefficients, everything works fine. Any idea on what could be wrong?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def rk4(f=f,h=2*pi/100,x0=1,v0=0,t0=0,n=100):&#xA;    '''RK4'''&#xA;    v=[v0]&#xA;    x=[x0]&#xA;&#xA;    for i in range(n-1):&#xA;        v1=h*f(x[-1])&#xA;        x1=h *v[-1]&#xA;        v2=h*f(x[-1]+1/4*x1)&#xA;        x2=h *(v[-1]+1/4*v1)&#xA;        v3=h*f(x[-1]+9/32*x2+3/32*x1)&#xA;        x3=h *(v[-1]+9/32*v2+3/32*v1)&#xA;        v4=h*f(x[-1]+7296/2197*x3 - 7200/2197 * x2 + 1932 / 2197 * x1)&#xA;        x4=h *(v[-1]+7296/2197*v3 - 7200/2197 * v2 + 1932 / 2197 * v1)&#xA;        v5=h*f(x[-1]-845/4104 * x4 +3680/513*x3 - 8 * x2 + 439 / 216 * x1)&#xA;        x5=h *(v[-1]-845/4104 * v4 +3680/513*v3 - 8 * v2 + 439 / 216 * v1)&#xA;        v.append(v[-1]+25/216*v1+1408/2565*v3+2197/4104*v4-1/5*v5)&#xA;        x.append(x[-1]+25/216*x1+1408/2565*x3+2197/4104*x4-1/5*x5)&#xA;    return x,v&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15224" LastEditorUserId="9667" LastEditDate="2015-04-29T23:17:24.980" LastActivityDate="2015-04-29T23:17:24.980" Title="RK4 giving wrong result" Tags="|python|ode|runge-kutta|" AnswerCount="1" CommentCount="3" ClosedDate="2015-05-02T01:21:46.810" ContentLicense="CC BY-SA 3.0" />
  <row Id="19502" PostTypeId="2" ParentId="19501" CreationDate="2015-04-29T14:55:14.763" Score="3" Body="&lt;p&gt;As discovered in &lt;a href=&quot;https://stackoverflow.com/a/29947165/3088138&quot;&gt;https://stackoverflow.com/a/29947165/3088138&lt;/a&gt; , there is a simple typo in the &lt;code&gt;x4=...&lt;/code&gt; line. There should be no &lt;code&gt;f&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2015-04-29T14:55:14.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19504" PostTypeId="2" ParentId="19478" CreationDate="2015-04-29T15:31:00.480" Score="1" Body="&lt;p&gt;The complete solution is&#xA;$$&#xA;y(x)=C·e^{-x^2/2}+D·e^{-x^2/2}·\int_0^x e^{s^2}ds&#xA;$$&#xA;The first is an even function and bounded, the second an odd function, has a not further reducible integral and is exploding like $\sim x·e^{x^2/2}$.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2015-04-29T15:31:00.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19505" PostTypeId="1" CreationDate="2015-04-29T21:15:34.340" Score="3" ViewCount="44" Body="&lt;p&gt;I have a 4D surface in the form of the spin quadric. I would like to display this surface onto the screen. One of the method which I would like to use to present surface onto the screen is projection. I would like to perform 4D-&gt;3D and then 3D-&gt;2D projections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be able to perform a projection I need to have a set of simplexes which belongs to the surface. Unfortunately, I have only implicit form of the surface and I can't find any library which allows me to convert 4D implicit surface to simplexes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know any kind of library which allows for &lt;code&gt;converting&lt;/code&gt; 4D implicit surface to simplexes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="15227" LastActivityDate="2015-04-29T21:15:34.340" Title="Is there a library which allows for converting 4D implicit surfaces to simplexes?" Tags="|visualization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19506" PostTypeId="1" CreationDate="2015-04-29T22:10:57.563" Score="4" ViewCount="341" Body="&lt;p&gt;I need to write a DNS code for simulating reacting flows in turbulent conditions. The code has to be highly scalable, because the computational cost of simulations is expected very large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to use C++ and I was wondering if you have suggestions about a very efficient C++ library to manage vector/matrix operations in distributed memory architectures. As an example, I know that PETSc and Trilinos packages are widely used, but I have no idea about the best choice for my case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional details:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;compressible formulation &lt;/li&gt;&#xA;&lt;li&gt;structured 3D grids &lt;/li&gt;&#xA;&lt;li&gt;finite difference discretization (staggered arrangement) &lt;/li&gt;&#xA;&lt;li&gt;high order FD&lt;/li&gt;&#xA;&lt;li&gt;good scalability on 400-500 cores&lt;/li&gt;&#xA;&lt;li&gt;only for use in my group (no need of complex object oriented design)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="15229" LastEditorUserId="15229" LastEditDate="2015-04-30T14:47:26.090" LastActivityDate="2015-05-06T06:11:10.803" Title="Direct Numerical Simulation of reacting flows" Tags="|fluid-dynamics|hpc|vector|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19508" PostTypeId="2" ParentId="19485" CreationDate="2015-04-30T00:54:41.053" Score="1" Body="&lt;p&gt;I got the code working, the full solution can be found in my github repo &lt;a href=&quot;https://github.com/cbell14/CFD_Class&quot; rel=&quot;nofollow&quot;&gt;https://github.com/cbell14/CFD_Class&lt;/a&gt;. The file is Taylor_Green_Vortices.ipynb. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There were 2 main issues in the code posted above. The first was in my implementation of the Dirichlet Boundary Condition (BC) in the A matrix. Implementing the BC in the A matrix does not work as it effects more than just the point you wish to make Dirichlet. Instead after solving for the pressure, I subtract the value at the middle from the entire field. This is necessary because all the boundary conditions are Neumann which yields an infinite number of solutions so the pressure must be anchored to a particular value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second issue was that I was incorporating the dx and dy terms from the denominator of the numerical scheme in the A matrix. This was proving to be difficult, so I incorporated them into the RHS of the linear system (my b matrix). This solved all my issues. Hope this helps anyone else who has a similar issue.&lt;/p&gt;&#xA;" OwnerUserId="15205" LastActivityDate="2015-04-30T00:54:41.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19510" PostTypeId="2" ParentId="19496" CreationDate="2015-04-30T08:09:26.340" Score="2" Body="&lt;p&gt;In the case of an anisotropic material, the phase velocities of waves traveling through that material are determined by Christoffel's equation:&#xA;\begin{equation}&#xA;[\rho c^2\delta_{ij} - C_{ijkl}n_jn_l][u_k]=0&#xA;\end{equation}&#xA;Here, $\rho$ is the density of the material, $c$ is the propagation velocity of the wave, $\delta$ is Kronecker's delta, $n_j$ is the unit vector in the direction $j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be seen that the solution corresponds to taking the determinant of the brackets on the left, which gives an eigenvalue equation in the eigenvalues $\rho c^2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, in the three-dimensional anisotropic case, we still have three different phase velocities for a particular direction of propagation, the largest of which has to be used for CFL analysis, in a manner similar to the way the longitudinal speed is used in an isotropic problem.&lt;/p&gt;&#xA;" OwnerUserId="6775" LastEditorUserId="6775" LastEditDate="2015-04-30T17:34:53.727" LastActivityDate="2015-04-30T17:34:53.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19511" PostTypeId="2" ParentId="19492" CreationDate="2015-04-30T09:04:45.167" Score="1" Body="&lt;p&gt;I got similar results using the values you gave.  When $\phi\ll1$, the $\phi^3$ term may be negligible, depending on the value of $k_2$.  With $\phi&amp;gt;1$, the $\phi^3$ term would become significant and probably is the reason we got the results that you show in your graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that $\phi&amp;gt;1$ could not happen in a real system, because the material would rupture first.  I don't remember my failure theories well enough to be too specific, but I would choose a shaft diameter that prevents failure due to maximum shear stress, and due to fatigue.  Based on my experience with steel shafting for ship propulsion, $\phi_{max}$ should be on the order of one milliradian or less. In my experience, the shafting was designed so that the alternating component of torque was no more than 4% of the average torque, to prevent fatigue failures.  Of course, your system could be much different than what I'm thinking of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, it would be easier to answer your question if we knew more about the actual system, the value of $k_1$ and the units for your values.  Which component has the nonlinear behavior?&lt;/p&gt;&#xA;" OwnerUserId="11950" LastActivityDate="2015-04-30T09:04:45.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19512" PostTypeId="1" CreationDate="2015-04-30T10:02:32.840" Score="1" ViewCount="187" Body="&lt;p&gt;I am solving a non-linear second order system of PDEs in two variables. The equations are too complicated to write out here, but an essential feature is that there is a propagating wave which then bounces on a boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I have is that the numerics breaks down at the boundary, when the wave reaches this point. I have tried by &quot;trial and error&quot; to just change the way I compute derivatives at this point with different finite difference stenils, but with no luck (pseudospectral methods do not work btw), and I have no idea on how to systematically try to improve the stability and I have no intuition of what can work and what will not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any tips on what to try when one encounters such problems? How can I systematically move forward to try to make my numerical scheme stable at this boundary point? Basically, if someone just has a list of ideas to blindly try, that would also be great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I should add that the &quot;boundary&quot; can also be interpreted as an origin in polar coordinates, so close to the boundary we have a cylindrical wave scattering at the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit2: By request I have added a (reduced version) of the equations I am trying to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot{c}=K'$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot{K}=\frac{K^2f}{5r^2}+25(1-f)+\frac{5r}{4}(fc/r)'$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot{f}=-\frac{f^2K'}{5r}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;polar coordinate origin&quot; is at r=0, and the wave that propagates is in $c$ and $K$. I have removed many terms to obtain the above equations, and only kept the ones that I think are important. It seems that the instability is due to the $1-f$ term in $\dot{K}$ (if I remove it I get stable evolutions). The instability manifests itself by having $f$ and $K$ diverging with opposite signs at the points close to the origin. Around $r\sim 0$ we expect $K\sim r^4$ and $c\sim r^3$ and $f-1\sim r^2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions I put are $K=e^{-t^2}$ at some $r=r_0$, which induces a wave that should bounce at $r=0$. The initial conditions are $f=1$, $c=K=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more details about my numerics: I have K and c on different grids (such that a point in $c$ lies between two points in $K$). This removed other instability issues I had. I use second/third order finite difference methods for computing derivatives and for interpolating between the grids. $c$ has a point at the origin, $K$ does not, and $f$ is on the same grid as $K$. At the origin I use the fact that the functions should be even when computing derivatives (by extending the radial coordinate to negative values), but this does not change much.&lt;/p&gt;&#xA;" OwnerUserId="15232" LastEditorUserId="15232" LastEditDate="2015-05-02T13:04:51.857" LastActivityDate="2015-05-02T13:04:51.857" Title="Tips on improving stability in numerical scheme for non-linear PDE" Tags="|pde|finite-difference|numerical-analysis|stability|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="19513" PostTypeId="1" AcceptedAnswerId="19521" CreationDate="2015-04-30T11:33:25.107" Score="9" ViewCount="13334" Body="&lt;p&gt;I'm using Armadillo to do very intensive matrix multiplications with side lengths $2^n$, where $n$ can be up to 20 or even more. I'm using &lt;a href=&quot;http://arma.sourceforge.net/&quot;&gt;Armadillo&lt;/a&gt; with &lt;a href=&quot;http://www.openblas.net/&quot;&gt;OpenBLAS&lt;/a&gt; for matrix multiplication, which seems to be doing a very good job in parallel cores, except that I have a problem with the formalism of multiplication in Armadillo for super optimization of performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say that I have a loop in the following form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arma::cx_mat stateMatrix, evolutionMatrix; //armadillo complex matrix type&#xA;for(double t = t0; t &amp;lt; t1; t += 1/sampleRate)&#xA;{&#xA;    ...&#xA;    stateMatrix = evolutionMatrix*stateMatrix;&#xA;    ...&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In fundamental C++, I find the problem here is that C++ will allocate a new object of &lt;code&gt;cx_mat&lt;/code&gt; to store &lt;code&gt;evolutionMatrix*stateMatrix&lt;/code&gt;, and then copy the new object to &lt;code&gt;stateMatrix&lt;/code&gt; with &lt;code&gt;operator=()&lt;/code&gt;. This is very, very inefficient. It's well known that returning complex classes of large data types is a bad idea, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I see this going way more efficient is with a function doing the multiplication in the form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void multiply(const cx_mat&amp;amp; mat1, const cx_mat&amp;amp; mat2, cx_mat&amp;amp; output)&#xA;{&#xA;    ... //multiplication of mat1 and mat2 and then store it in output&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This way, One doesn't have to copy huge objects with return value, and the output doesn't have to be reallocated with every multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Question&lt;/strong&gt;: How can I find a compromise, in which I can use Armadillo for multiplication with its nice interface of BLAS, and do this efficiently without having to recreate matrix objects and copy them with each operation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isn't this an implementation problem in Armadillo?&lt;/p&gt;&#xA;" OwnerUserId="5410" LastActivityDate="2015-05-02T01:24:11.580" Title="Super C++ optimization of matrix multiplication with Armadillo" Tags="|c++|matrix|dense-matrix|" AnswerCount="6" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19514" PostTypeId="1" CreationDate="2015-04-30T11:37:38.700" Score="3" ViewCount="1033" Body="&lt;p&gt;In texts I often read that meshless methods, as opposed to the FEM, do not exhibit the Kronecker Delta Property [1]:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N_I(x_J)\neq\delta_{IJ} = \begin{cases}1 &amp;amp;\mbox{if } I=J \\0 &amp;amp;\mbox{otherwise}\end{cases}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $N_I$ is the meshless shape function at position $x_I$. I understand this property, however, I'm not sure of the implications. Say, one wants to solve a problem in elastostatics of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{Kf}=\mathbf{u}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathbf{K}$ is the stiffness Matrix, $\mathbf{f}$ the force vector and $\mathbf{u}$ the displacement. The construction of those matrices would be analogous to FEM, except the for using the meshless shape functions, see e.g. [2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, also in [2] (and in in other texts) it is claimed that, when solving for $\mathbf{u}$, the solution is in fact a fictitious one and the real solution needs to be found by re-approximation using the shape functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_I = \sum\limits_JN(x_J)u_J$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't understand how the use of shape functions that do not satisfy the Kronecker delta property entails the need for this step. Can anybody provide me some insight?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] see e.g. &lt;a href=&quot;http://www.academia.edu/4135937/Numerical_Implementation_of_Solid_Boundary_Conditions_in_Meshless_Methods&quot; rel=&quot;nofollow&quot;&gt;http://www.academia.edu/4135937/Numerical_Implementation_of_Solid_Boundary_Conditions_in_Meshless_Methods&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://www.academia.edu/225241/Meshless_methods_A_review_and_computer_implementation_aspects&quot; rel=&quot;nofollow&quot;&gt;http://www.academia.edu/225241/Meshless_methods_A_review_and_computer_implementation_aspects&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9366" LastActivityDate="2015-04-30T19:33:25.560" Title="Meshless Methods and the Kronecker Delta Property" Tags="|finite-element|meshless-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19515" PostTypeId="2" ParentId="19513" CreationDate="2015-04-30T12:57:59.940" Score="5" Body="&lt;p&gt;Stackoverflow (&lt;a href=&quot;https://stackoverflow.com/&quot;&gt;https://stackoverflow.com/&lt;/a&gt;) is probably a better discussion forum for this question. However, here is a short answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I doubt that the C++ compiler is generating code for this expression like you describe above. All modern C++ compilers implement an optimization called &quot;return value optimization&quot; (&lt;a href=&quot;http://en.wikipedia.org/wiki/Return_value_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://en.wikipedia.org/wiki/Return_value_optimization&lt;/a&gt;). With return value optimization the result of &lt;code&gt;evolutionMatrix*stateMatrix&lt;/code&gt; is stored directly in &lt;code&gt;stateMatrix&lt;/code&gt;; no copy is made. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is obviously considerable confusion on this topic and that&#xA;is one of the reasons I suggested Stackoverflow might be a better&#xA;forum. There are many C++ &quot;language lawyers&quot; there while most of&#xA;us here would rather spend our time on CSE. ;-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created the following simple example based on Professor Bangerth's&#xA;post:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#ifndef NDEBUG&#xA;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;using namespace std;&#xA;#endif&#xA;&#xA;class ExpensiveObject  {&#xA;public:&#xA;  ExpensiveObject () {&#xA;#ifndef NDEBUG&#xA;    cout &amp;lt;&amp;lt; &quot;ExpensiveObject  constructor called.&quot; &amp;lt;&amp;lt; endl;&#xA;#endif&#xA;    v = 0;&#xA;  }&#xA;  ExpensiveObject (int i) { &#xA;#ifndef NDEBUG&#xA;    cout &amp;lt;&amp;lt; &quot;ExpensiveObject  constructor(int) called.&quot; &amp;lt;&amp;lt; endl;&#xA;#endif&#xA;    v = i; &#xA;  }&#xA;  ExpensiveObject (const ExpensiveObject  &amp;amp;a) {&#xA;    v = a.v;&#xA;#ifndef NDEBUG&#xA;    cout &amp;lt;&amp;lt; &quot;ExpensiveObject  copy constructor called.&quot; &amp;lt;&amp;lt; endl;&#xA;#endif&#xA;  }&#xA;  ~ExpensiveObject() {&#xA;#ifndef NDEBUG&#xA;    cout &amp;lt;&amp;lt; &quot;ExpensiveObject  destructor called.&quot; &amp;lt;&amp;lt; endl;&#xA;#endif&#xA;  }&#xA;  ExpensiveObject  operator=(const ExpensiveObject  &amp;amp;a) {&#xA;#ifndef NDEBUG&#xA;    cout &amp;lt;&amp;lt; &quot;ExpensiveObject  assignment operator called.&quot; &amp;lt;&amp;lt; endl;&#xA;#endif&#xA;    if (this != &amp;amp;a) {&#xA;      return ExpensiveObject (a);&#xA;    }&#xA;  }&#xA;  void print() const {&#xA;#ifndef NDEBUG&#xA;    cout &amp;lt;&amp;lt; &quot;v=&quot; &amp;lt;&amp;lt; v &amp;lt;&amp;lt; endl;&#xA;#endif&#xA;  }&#xA;  int getV() const {&#xA;    return v;&#xA;  }&#xA;private:&#xA;  int v;&#xA;};&#xA;&#xA;ExpensiveObject  operator+(const ExpensiveObject  &amp;amp;a1, const ExpensiveObject  &amp;amp;a2) {&#xA;#ifndef NDEBUG&#xA;  cout &amp;lt;&amp;lt; &quot;ExpensiveObject  operator+ called.&quot; &amp;lt;&amp;lt; endl;&#xA;#endif&#xA;  return ExpensiveObject (a1.getV() + a2.getV());&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;  ExpensiveObject  a(2), b(3);&#xA;  ExpensiveObject  c = a + b;&#xA;#ifndef NDEBUG&#xA;  c.print();&#xA;#endif&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It looks more complicated than it actually is because I wanted to completely remove&#xA;all code for printing output when compiling in optimized mode. When I run the version&#xA;compiled with a debug option, I get the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ExpensiveObject  constructor(int) called.&#xA;ExpensiveObject  constructor(int) called.&#xA;ExpensiveObject  operator+ called.&#xA;ExpensiveObject  constructor(int) called.&#xA;v=5&#xA;ExpensiveObject  destructor called.&#xA;ExpensiveObject  destructor called.&#xA;ExpensiveObject  destructor called.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first thing to notice is that &lt;em&gt;no&lt;/em&gt; temporaries are constructed-- only a, b, and c.&#xA;The default constructor and the assignment operator are never called because they aren't&#xA;needed in this example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Professor Bangerth mentioned expression templates. Indeed, this optimization technique is&#xA;very important in obtaining good performance in a matrix class library. But it is important&#xA;only when the object expressions are more complicated than simply  a + b. If, for example, my&#xA;test was instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  ExpensiveObject  a(2), b(3), c(9);&#xA;  ExpensiveObject  d = a + b + c;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would get the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ExpensiveObject  constructor(int) called.&#xA; ExpensiveObject  constructor(int) called.&#xA; ExpensiveObject  constructor(int) called.&#xA; ExpensiveObject  operator+ called.&#xA; ExpensiveObject  constructor(int) called.&#xA; ExpensiveObject  operator+ called.&#xA; ExpensiveObject  constructor(int) called.&#xA; ExpensiveObject  destructor called.&#xA; v=14&#xA; ExpensiveObject  destructor called.&#xA; ExpensiveObject  destructor called.&#xA; ExpensiveObject  destructor called.&#xA; ExpensiveObject  destructor called.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This case shows the undesirable construction of a temporary (5 calls to the constructor and two calls of operator +).&#xA; Proper use of expression templates (a topic well beyond the scope of this forum) would prevent this temporary. (For the highly motivated, a particularly readable discussion of expression templates can be found in chapter 18 of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0201734842&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.amazon.com/C-Templates-The-Complete-Guide/dp/0201734842&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the real &quot;proof&quot; of what the compiler is actually doing comes from examining the assembly code output by&#xA; the compiler. For the first example, when compiled in optimized mode, this code is astonishly simple. All function&#xA; calls have been optimized away and the assembly code essentially loads 2 into one register, 3 into a second, and&#xA; adds them.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-04-30T18:20:39.763" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19516" PostTypeId="2" ParentId="19513" CreationDate="2015-04-30T15:55:28.540" Score="8" Body="&lt;p&gt;@BillGreene points to the &quot;return value optimization&quot; as a way around the fundamental problem, but this actually only helps for one half of it. Assume you have code of this form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;struct ExpensiveObject { ExpensiveObject(); ~ExpensiveObject(); };&#xA;&#xA;ExpensiveObject operator+ (ExpensiveObject &amp;amp;obj1,&#xA;                           ExpensiveObject &amp;amp;obj2)&#xA;{&#xA;   ExpensiveObject tmp;&#xA;   ...compute tmp based on obj1 and obj2...&#xA;   return tmp;&#xA;}&#xA;&#xA;void f() {&#xA;  ExpensiveObject o1, o2, o3;&#xA;  ...initialize o1, o2...;&#xA;  o3 = o1 + o2;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A naive compiler will&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;create a slot to store the result of the plus operation (one temporary),&lt;/li&gt;&#xA;&lt;li&gt;call operator+, &lt;/li&gt;&#xA;&lt;li&gt;create the 'tmp' object inside operator+ (a second temporary), &lt;/li&gt;&#xA;&lt;li&gt;compute tmp, &lt;/li&gt;&#xA;&lt;li&gt;copy tmp into the result slot, &lt;/li&gt;&#xA;&lt;li&gt;destroy tmp,&lt;/li&gt;&#xA;&lt;li&gt;copy the result object into o3&lt;/li&gt;&#xA;&lt;li&gt;destroy the result object&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Return value optimization can only unify the 'tmp' object and the 'result' slot, but not remove the need for a copy. So, you are still left with the creation of a temporary, the copy operation, and the destruction of a temporary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way around this is operator+ does not return an object, but an object of some intermediate class that, when assigned to an &lt;code&gt;ExpensiveObject&lt;/code&gt;, does the addition and copy operation in place. This is the approach typical used in &lt;em&gt;expression template libraries&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-04-30T15:55:28.540" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19517" PostTypeId="1" CreationDate="2015-04-30T17:56:50.997" Score="0" ViewCount="65" Body="&lt;p&gt;I'm trying to integrate an incredibly simple ODE:&#xA;$$&#xA;y'(x) = -f(y),\quad y(0) = y_0 \ ,&#xA;$$&#xA;from $x=0$ to $x=1$. This is a decay type of equation, $f$ is the (variable) decay rate and $y$ is the abundance.  Both $y$ and $f(y)$ are positive everywhere.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm faced with a couple of constraints.  Sometimes $f(y) \gg 1$, which makes this a stiff problem on the timescales I'm interested in.  I can evaluate $f(y)$ exactly, but it can be costly. So I'd like to minimize the number of times I have to call it.  Luckily, the result doesn't have to be super accurate.  This is a tiny routine that will be called often in a much larger code.  The current method is obviously horrible: we just take a single forward Euler step:&#xA;$$&#xA;y(1) \approx y(0) + f(y_0) \ .&#xA;$$&#xA;So literally anything suggested will be an improvement :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, I could use an RK scheme, probably with adaptive step sizing, or better yet an implicit scheme.  However, given the simplicity of the equation I thought there may be something more elegant.  For instance, we could rewrite the problem as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{y_0}^{y_1} \frac{dy}{f(y)} = -1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and attempt to solve for $y_1$, but I'm not sure of the best way to approach this numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions appreciated!&lt;/p&gt;&#xA;" OwnerUserId="14688" LastEditorUserId="14772" LastEditDate="2015-05-07T04:18:52.560" LastActivityDate="2015-05-07T04:18:52.560" Title="Quick scheme for separable first-order ODE" Tags="|finite-difference|ode|integral-equations|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19518" PostTypeId="2" ParentId="19513" CreationDate="2015-04-30T18:30:31.980" Score="5" Body="&lt;p&gt;I'd like to refute your premise that this is &amp;ldquo;very, very inefficient&amp;rdquo;. Performance-wise, it's actually pretty irrelevant, because matrix multiplication is&lt;sup&gt;1&lt;/sup&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Strassen_algorithm&quot; rel=&quot;noreferrer&quot;&gt;$\mathcal{O}(n^{2.8})$&lt;/a&gt;, whereas copying one of those matrices is only $\mathcal{O}(n^2)$. So for $n$ as big as you have, the multiplication will probably completely outweigh the copying time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, unless you incur an enormous constant into the copying &amp;ndash; which is actually not so far-fetched, because the version with copying &lt;em&gt;is&lt;/em&gt; much more expensive in another regard: it needs way more memory. So if you end up having to swap to and from hard disk, the copying might indeed become the bottleneck. However, even if you &lt;em&gt;don't&lt;/em&gt; copy anything yourself, a strongly parallelised algorithm may well do some copies of its own. Really, the only way to make sure not too much memory will be used in each step is to split up the multiplication in &lt;em&gt;columns of &lt;code&gt;stateMatrix&lt;/code&gt;&lt;/em&gt;, so only small multiplications are done at a time. For instance, you can define&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class HChunkMatrix // optimised for destructive left-multiplication updates&#xA;{&#xA;  std::vector&amp;lt;arma::cx_mat&amp;gt; colChunks; // e.g. for an m×n matrix,&#xA;                                      //  use √n chunks, each an m×√n matrix&#xA; public:&#xA;  ...&#xA;&#xA;  HChunkMatrix&amp;amp; operator *= (const arma::cx_mat&amp;amp; lhMult) {&#xA;    for (&amp;amp;colChunk: colChunks) {&#xA;      colChunk = lhMult * colChunk;&#xA;    }&#xA;    return *this;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should also consider whether you even need to evolute that &lt;code&gt;stateMatrix&lt;/code&gt; as one in the first place. If you basically just want independent time-evolution of &lt;code&gt;n&lt;/code&gt; state kets, then you might as well evolute them one by one, which is much less memory-costly. In particular if &lt;code&gt;evolutionMatrix&lt;/code&gt; is &lt;em&gt;sparse&lt;/em&gt;, which you should definitely check out! For this is basically just a Hamiltonian, isn't it? Hamiltonians are often sparse or approximately sparse.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;sub&gt;Or &lt;a href=&quot;https://en.wikipedia.org/wiki/Coppersmith%E2%80%93Winograd_algorithm&quot; rel=&quot;noreferrer&quot;&gt;$\mathcal{O}(n^{2.38})$&lt;/a&gt;, but that one isn't practically useful.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="562" LastEditorUserId="562" LastEditDate="2015-04-30T18:40:59.007" LastActivityDate="2015-04-30T18:40:59.007" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19520" PostTypeId="2" ParentId="19514" CreationDate="2015-04-30T19:33:25.560" Score="3" Body="&lt;p&gt;I would say that this is not just a property of Finite Element Methods, but in general, when you approximate a function using a base that does not satisfy the Kronecker delta relation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the general case, your system of equations is posed for the coefficients of the expansion&#xA;$$u(x) \approx \sum_{i=1}^N c_i f_i(x)$$&#xA;i.e., for $c_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us expose this with an example using the Ritz method. We are trying to solve&#xA;$$\frac{d^2 u}{dx^2} + u + x = 0$$&#xA;with boundary conditions $u(0)=u(1)=0$. Let us use the approximate function&#xA;$$u_2(x) = x(1-x)[c_1 + c_2 x]$$&#xA;The functional for this approximation is&#xA;$$J[u_2] = \frac{13}{210}c_2^2 + \frac{3}{20}c_1 c_2 - \frac{1}{2}c_2 + \frac{3}{20}c_1^2 - \frac{1}{12}c_1 \enspace .$$&#xA;Imposing the stationarity ($\delta J=0$) we obtain the system of equations&#xA;$$\begin{align}&#xA;&amp;amp;126 c_1 + 63 c_2 = 35\\&#xA;&amp;amp;63 c_1 + 52 c_2 = 21&#xA;\end{align}$$&#xA;that has as solution $c_1 = 71/369$ and $c_2=7/41$. Notice that these two coefficients are just numbers, and they do not represent a function over our domain $[0,1]$. They play the role of multiply the functions of our expansion, that give us as solution&#xA;$$u \approx u_2(x) = x(1-x)\sum_{i=1}^2 c_i x^{i-1} = (1 - x)x\left(\frac{7}{41}x + \frac{71}{369}\right) \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;em&gt;common&lt;/em&gt; FEM we use a Lagrange interpolation that satisfy the Kronecker delta condition, and then we can interpret our coefficients to be of the same nature of our function of interest (which is not always the case).&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-04-30T19:33:25.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19521" PostTypeId="2" ParentId="19513" CreationDate="2015-04-30T20:40:54.673" Score="14" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;In fundamental C++, I find the problem here is that C++ will allocate a new object of cx_mat to store evolutionMatrix*stateMatrix, and then copy the new object to stateMatrix with operator=().&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think you're right that it's creating temporaries, which is too slow, but I think the reason for why it's doing that is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Armadillo, like any good C++ linear algebra library,&#xA;uses expression templates to implement delayed evaluation of expressions. When you&#xA;write down a matrix product like &lt;code&gt;A*B&lt;/code&gt;, no temporaries are created,&#xA;instead Armadillo makes a temporary object (&lt;code&gt;x&lt;/code&gt;) that keeps references to&#xA;&lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt;, and then later, given an expression like &lt;code&gt;C = x&lt;/code&gt;,&#xA;computes the matrix product storing the result directly in &lt;code&gt;C&lt;/code&gt;, without creating any temporaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also uses this optimization to handle expressions like &lt;code&gt;A*B*C*D&lt;/code&gt;,&#xA;where, depending on matrix sizes, certain orders of multiplication are&#xA;more efficient than others.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Isn't this an implementation problem in Armadillo? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If Armadillo is &lt;em&gt;not&lt;/em&gt; performing this optimization, that would be a&#xA;bug in Armadillo that should be reported to developers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in your case, there is another problem that is more&#xA;important. In an expression like &lt;code&gt;A=B*C&lt;/code&gt; the storage of &lt;code&gt;A&lt;/code&gt; doesn't&#xA;contain any input data if &lt;code&gt;A&lt;/code&gt; doesn't alias &lt;code&gt;B&lt;/code&gt; or &lt;code&gt;C&lt;/code&gt;. In your case,&#xA;&lt;code&gt;A = A*B&lt;/code&gt;, writing anything to the output matrix would modify one of&#xA;the input matrices as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even given your suggested function &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;multiply(const cx_mat&amp;amp;, const cx_mat&amp;amp;, cx_mat&amp;amp;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;how exactly would that function help in the&#xA;expression &lt;code&gt;multiply(A, B, A)&lt;/code&gt;? For most ordinary implementations of&#xA;that function, that would lead to a bug. It would need to use some temporary storage on its own, to make sure its input data is not corrupted. Your suggestion is pretty&#xA;much how Armadillo implements matrix multiplication already, but I&#xA;think it probably takes care to avoid situtations like&#xA;&lt;code&gt;multiply(A, B, A)&lt;/code&gt; by allocating a temporary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most likely explanation of why Armadillo &lt;em&gt;isn't&lt;/em&gt; doing this&#xA;optimization is that it would be incorrect to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, there is a much simpler way to do what you want, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cx_mat *A, *Atemp, B;&#xA;for (;;) {&#xA;  *Atemp = (*A)*B;&#xA;  swap(A, Atemp);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is identical to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cx_mat A, B;&#xA;for (;;) {&#xA;  A = A*B;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but it allocates one temporary matrix, instead of one temporary matrix&#xA;per iteration.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-04-30T20:40:54.673" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19522" PostTypeId="1" CreationDate="2015-04-30T22:03:48.710" Score="0" ViewCount="3025" Body="&lt;p&gt;I have a question about Matlab and restarted gmres. I would like to use &lt;code&gt;gmres.m&lt;/code&gt; &lt;a href=&quot;http://www.netlib.org/templates/matlab/gmres.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;provided here&lt;/a&gt;. This code seems to be popular for the scientific computation newcomer. I also want to modify this code for my particular case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I have a question about recording the number of matrix-vector products and the&#xA;2-norm of each residual vector. If you use the built-in function &lt;code&gt;gmres.m&lt;/code&gt; (e.g., &lt;code&gt;gmres(30)&lt;/code&gt;) to solve a linear system, one obtains&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;iter = [17,20], % the total number of matrix-vector products is (17 - 1)*30 + 12 (= 492).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;code&gt;resvec&lt;/code&gt;, a $493\times1$ vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, in the &lt;code&gt;gmres.m&lt;/code&gt; function these two outputs are not available. As you know, we always call the gmres.m in MATLAB in the following style:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[x, flag, relres, iter, resvec] = gmres(A, b, restart, tol, maxit);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I want to use these outputs to plot the convergence histories, i.e., &lt;/p&gt;&#xA;&#xA;&lt;p&gt;X-axis: the number of matrix-vector products,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Y-axis: 2-norm of relative residual, i.e., &lt;code&gt;norm(b - Ax_m)/norm(b);&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how can I record this two outputs reasonably in the modified &lt;code&gt;gmres.m&lt;/code&gt;? Despite my many attempts to deal with this problem, I still fail to manage it. It makes me very confused, especially for recording &lt;code&gt;resvec&lt;/code&gt;. Can anyone provide suggestions for handling this problem?&lt;/p&gt;&#xA;" OwnerUserId="10290" LastEditorUserId="20688" LastEditDate="2018-07-09T03:52:17.100" LastActivityDate="2018-07-09T04:26:37.627" Title="MATLAB: code for restarted gmres" Tags="|matlab|linear-solver|iterative-method|gmres|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19523" PostTypeId="2" ParentId="19513" CreationDate="2015-04-30T22:06:33.003" Score="5" Body="&lt;p&gt;Modern C++ has a solution for the problem by using &quot;move constructors&quot; and &quot;rvalue references&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;move constructor&quot; is a constructor for a class, for example a matrix class, which takes another instance of the same class and moves the data from the other instance into the new instance, leaving the original instance empty. Typically, a matrix object will have two numbers for the size and a pointer to the data. Where a normal constructor would duplicate the data, a move constructor will only copy the two numbers and the pointer, so this is really fast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An &quot;rvalue reference&quot;, written for example as &quot;matrix&amp;amp;&amp;amp;&quot; instead of the usual &quot;matrix&amp;amp;&quot; is used for temporary variables. You would declare a matrix multiplication as returning a matrix&amp;amp;&amp;amp;. By doing that, the compiler will make sure that a very cheap move constructor will be used to get the result out of the function that calls it. So an expression like result = (a + b) * (c + d) where a, b, c, d are huge matrix objects, will happen without any copying. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Googling for &quot;rvalue references and move constructors&quot; will find examples and tutorials. &lt;/p&gt;&#xA;" OwnerUserId="15239" LastActivityDate="2015-04-30T22:06:33.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19524" PostTypeId="1" AcceptedAnswerId="19528" CreationDate="2015-05-01T00:39:25.417" Score="1" ViewCount="734" Body="&lt;p&gt;I've been given an assignment: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a console application (using C and Obj C) that will calculate area of a random polygon. &#xA;The application should process input data as a .txt file with a list of Cartesian coordinates pairs in the following format: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(12;34)&#xA;(23;35)&#xA;(85;23)&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From those pairs a polygon should be formed. &#xA;Sides of the polygon (line segments) should not intersect. If they do - return an error. &#xA;The application should divide the polygon into triangles, calculate their area, sum and then return  the area of the polygon. &#xA;Input processor, triangles and the sum operation - all should be implemented as a separate classes. &#xA;The application should support command &quot;help&quot; and return info on how to use the app. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So far I've build the following algorithm. &#xA;It lacks some features (that I need help with) and have several programmatical mistakes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How to check that the sides of a polygon (line segments) - don't intersect? Each line segment should be checked to another. &#xA;At which point of the algorithm this checking should take place? &lt;/li&gt;&#xA;&lt;li&gt;If the polygon has concave sides, my algorithm of building triangles out of the first point fails. What would be the solution for that? &#xA;(the app should work for the concave/complex polygons too)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The app processes the entry data. Form 2 NSMutableArrays one for X, the other for Y coordinates.  &lt;/li&gt;&#xA;&lt;li&gt;Count number of values in any array. If less then 3 - display an error &quot;Entry at least 3 pairs to form a polygon&quot;. &lt;/li&gt;&#xA;&lt;li&gt;Create class &quot;Corner&quot;. Properties of it will be coordinates X and Y. An object of this class will be one corner of our polygon. &lt;/li&gt;&#xA;&lt;li&gt;Create a &lt;code&gt;for&lt;/code&gt; loop that should be repeated less than number of values in one of our NSMutableArrays. &lt;/li&gt;&#xA;&lt;li&gt;At the start of the loop, we create one object from our &quot;Corner&quot; class. We take out from our NSMutableArrays coordinates and give it as properties of our &quot;corner&quot; object. This way, at the end of the loop cycle we will have all corners/vertexes created. &lt;/li&gt;&#xA;&lt;li&gt;Every created &lt;code&gt;Corner&lt;/code&gt; we will be adding to another array. &lt;/li&gt;&#xA;&lt;li&gt;If the for loop already made at least 3 cycles - we already have 3 vertexes from which we can build the first triangle(from item 10 will learn why we care about triangles) &lt;/li&gt;&#xA;&lt;li&gt;Therefore, we create a new class &lt;code&gt;Triangle&lt;/code&gt;. We give it properties: &lt;code&gt;&quot;side1&quot;, &quot;side2&quot;, &quot;side3&quot;, &quot;area&quot;&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;Back to our &quot;for&quot; loop. We create an instance of the &lt;code&gt;Triangle&lt;/code&gt; class. &lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Because we will be calculating area of our ultimate polygon, we can split it on triangles, calculate their area and then add them up. We will be drawing triangles from the first vertex: 1v-&gt;2v-&gt;3v; 1v-&gt;3v-&gt;4v; 1v-&gt;4v-&gt;5v...&#xA;Because we know coordinate points - we can calculate length between points. Therefore, we know length of all sides of a triangle. To find an area of such triangle we can use Heron's formula (more about it in item 13).&#xA;To calculate length between points we will create a new method inside the &lt;code&gt;Triangle&lt;/code&gt; class: &lt;code&gt;sideLengthWithVert:corner1 vert2:corner2 vert3:corner3&lt;/code&gt; . That method takes corners/vertexes of the triangle - objects of &quot;Corner&quot; class and one after another calculates length between corners using formula: d=sqrt((x2-x1)^2+(y2-y1)^2). The length will be assigned to properties of a &quot;Triangle&quot; object - &lt;code&gt;side1 side2 side3&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(here I have an error in my code, I suspect it is a logical mistake)&#xA;Since this method takes &lt;code&gt;corners&lt;/code&gt; as arguments - we create those 3 corners out of our previously created arrays of &lt;code&gt;corners&lt;/code&gt;. &#xA;An error says: &lt;code&gt;[__NSArrayM objectAtIndex:]:index 567 beyond bounds [0 .. 2]&lt;/code&gt; &#xA;Where is index 567 comes from - I have no clue. &#xA;In the method of angle creation I pass variable &lt;code&gt;t&lt;/code&gt; that equals to number of cycles of the &lt;code&gt;for&lt;/code&gt; loop. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;We use newly created method: &lt;code&gt;sideLengthWithVert:corner1 vert2:corner2 vert3:corner3&lt;/code&gt; and give it our corners. &#xA;Now our object &lt;code&gt;Triangle&lt;/code&gt; has properties &lt;code&gt;side1 side2 side3&lt;/code&gt;equaled length between corners. &lt;/li&gt;&#xA;&lt;li&gt;Now we have a complete triangle we can calculate its area. &#xA;For that we will write a new method inside class &lt;code&gt;Triangle&lt;/code&gt; called &lt;code&gt;calcArea&lt;/code&gt;. &#xA;Inside it, we use Heron's formula: &#xA;p=(a+b+c)/2&#xA;then&#xA;A=sqt(p(p-a)(p-b)(p-c))&#xA;Once calculated, we give that to &lt;code&gt;Triangle's&lt;/code&gt; property &lt;code&gt;area&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;We create a new NSMutableArray and add there our triangles that will be created with each cycle of our big &lt;code&gt;for&lt;/code&gt; loop.&lt;/li&gt;&#xA;&lt;li&gt;All what is last is to add up areas of triangles inside that last array. For that we create a condition &quot;if we have at least one &lt;code&gt;Triangle&lt;/code&gt; object created - perform following code: (inside next items 16, 17, 18) &quot;&lt;/li&gt;&#xA;&lt;li&gt;Calculate number of triangles inside the array. &#xA;Create a cycle that will loop less or equal number of triangles inside the array. With each step add up triangle's areas. &lt;/li&gt;&#xA;&lt;li&gt;If user enters &lt;code&gt;calc&lt;/code&gt;in console - show the result of our computations. &lt;/li&gt;&#xA;&lt;li&gt;If a user enters &lt;code&gt;help&lt;/code&gt; in the console - show examples of usage of the application. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15241" LastActivityDate="2015-05-01T16:50:47.983" Title="Find area of a polygon. In C and Obj C." Tags="|computational-geometry|c|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19525" PostTypeId="1" CreationDate="2015-05-01T01:24:27.943" Score="0" ViewCount="907" Body="&lt;p&gt;Consider the following two point BVP:&#xA;$$&#xA;-u''(x)=f(x),~~~u(0)=u(1)=0.&#xA;$$&#xA;An interior penalty DG method for this BVP that weakly imposes homogeneous Dirichlet boundary conditions is of the form:&#xA;$$&#xA;\sum_{i=0}^{N-1} \int_{I_i} u'(x) v'(x)\,dx&#xA;+&#xA;\frac{1}{h}&#xA;\sum_{i=0}^{N }&#xA; [\![u]\!]|_{x_i} &#xA; [\![ v]\!]|_{x_i} &#xA;- \epsilon&#xA; (\!( v')\!)|_{x_i}      [\![ u ]\!]|_{x_i}   - (\!( u')\!)|_{x_i}      [\![ v ]\!]|_{x_i}  &#xA;=&#xA;\int_{0}^{1} f(x)v(x)\,dx,&#xA;$$&#xA;where $[\![ \cdot ]\!]$ is the jump operator, and $(\!( \cdot )\!)$ is the average operator (see for more detail: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19394/help-implementing-1d-ode-discontinuous-galerkin-method&quot;&gt;https://scicomp.stackexchange.com/questions/19394/help-implementing-1d-ode-discontinuous-galerkin-method&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;  How can I modify the above interior penalty DG method so that it can handle Neumann boundary conditions?  For Poisson's equation in 2D and 3D, I've seen Neumann boundary conditions handled through integrals on the boundary of the elements...not sure if there is analogy of this for ODEs...&lt;/p&gt;&#xA;" OwnerUserId="11604" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-05-01T10:07:43.573" Title="How to impose Neumann boundary conditions in interior penalty DG method" Tags="|pde|discontinuous-galerkin|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19526" PostTypeId="2" ParentId="19513" CreationDate="2015-05-01T07:35:41.793" Score="0" Body="&lt;p&gt;You might also want to benchmark the operations in question in Eigen.  It is the only linear algebra library (that I know of) that even attempts to optimize matrix expressions involving more than two arrays.  For example, I'm pretty sure if you do $v'*M*M*M*M*M$ or $M*M*M*M*M*v$ it will do a bunch of vector-matrix multiplies, rather than matrix-matrix multiplies.  And if you do (v1 + v2 - v3) it will do the computation entry-wise in one pass with no intermediate, rather than two with an intermediate.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then again, I gather OpenBLAS has a bigger collection of architecture-specific optimizations, so Eigen may or may not be a win for you.  Unfortunately, there is no linear algebra library so awesome that you don't even have to consider the others when fighting for &quot;the last 10%&quot; of performance.  Wrappers are not a 100% solution; most (all?) of them cannot take advantage of eigen's ability to merge computations this way.&lt;/p&gt;&#xA;" OwnerUserId="15246" LastEditorDisplayName="user15247" LastEditDate="2015-05-02T01:24:11.580" LastActivityDate="2015-05-02T01:24:11.580" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19527" PostTypeId="2" ParentId="19525" CreationDate="2015-05-01T07:54:18.607" Score="1" Body="&lt;p&gt;It works exactly the same as in higher dimensions, except that the integral is replaced by evaluation at the boundary points of the interval. You have&#xA;$$&#xA;\int_{(0,1)} -u'' v \,dx = \int_{(0,1)} u' v' \,dx - (u'(1) v(1) - u'(0) v(0)),&#xA;$$&#xA;so if you have known values $g_0$ or $g_1$ for $u'(0)$ or $u'(1)$, you simply insert them into this equation and get new terms which should go on the right-hand side, $g_1 v(1)$ or $-g_0 v(0)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach works just the same in your DG formulation.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastEditorUserId="14598" LastEditDate="2015-05-01T10:07:43.573" LastActivityDate="2015-05-01T10:07:43.573" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19528" PostTypeId="2" ParentId="19524" CreationDate="2015-05-01T08:05:09.110" Score="2" Body="&lt;p&gt;After checking that the input data forms a valid polygon, you can compute the area without triangulating the polygon. The key is to recognize that the area formula can be transformed by Green's theorem (&lt;a href=&quot;http://en.wikipedia.org/wiki/Green%27s_theorem&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Green%27s_theorem&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A = \int_\Omega 1\,dA = \oint_{\partial\Omega}x\,dy = -\oint_{\partial\Omega} y\,dx= \frac{1}{2}\oint_{\partial\Omega}\left(-y\,dx + x\,dy \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you're given a description of the boundary, it is very natural to compute this line integral over the set of edges. Since you're integrating over linear edges, formulating this integral properly has already been done for you, and it is called the &quot;Shoelace formula.&quot; The Wikipedia page for this has the formula written out explicitly several ways, so you can pick the version that is easiest to deal with for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Shoelace_formula&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Shoelace_formula&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Checking for the intersection of two line segments can be done a lot of different ways, but one conceptually easy way to do it is to write each line n $y=mx + b$ form, compute the intersection of the two lines, then check whether the intersection occurs within the bounds of both line segments. Just be careful to handle perfectly vertical/horizontal lines properly, since that could be a likely opportunity for a divide-by-zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: &lt;strong&gt;Triangulation Method&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry, I didn't see the part in your question about requiring a triangulation. I came up with a pretty simple and effective way to triangulate the interior of a non-convex polygon. The basic idea is that you keep track of the interior, non-triangulated boundary polygon while you iteratively make triangles from this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic algorithm written in pseudocode is this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;While(interiorPolygon.size() &amp;gt;= 3)&#xA;&#xA;    [v1, v2, v3] = threeConsecutiveVertices in interiorPolygon&#xA;    if( isCCW(v1,v2,v3) )&#xA;       tris.add(v1,v2,v3)&#xA;       Drop v2 from interiorPolygon list&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The method assumes that the vertices that you are given are order counterclockwise around the border of the polygon. This is necessary because I used a isCCW(v1,v2,v3) function, which determines if {v1,v2,v3} form a triangle that is inside the polygon or outside the polygon. This can be achieved by computing the cross product of (v2-v1) and (v3-v1). If the cross product is positive, then the vertices form a counterclockwise triangle, which is in the interior of the polygon. Otherwise, the triangle formed by the vertices lies outside the interiorPolygon boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I coded it up in Matlab, and a reasonably robust version can be done in &amp;lt; 20 lines of code (I didn't check for polygon validity). I'll leave you to flesh out the details and bookkeeping that need to happen in the code.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastEditorUserId="7378" LastEditDate="2015-05-01T16:50:47.983" LastActivityDate="2015-05-01T16:50:47.983" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19529" PostTypeId="1" CreationDate="2015-05-01T14:03:17.263" Score="3" ViewCount="88" Body="&lt;p&gt;I am interested in learning numerical methods that specifically have to do with analyzing dynamical systems. In particular:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;drawing phase plane diagrams&lt;/li&gt;&#xA;&lt;li&gt;drawing phase portraits&lt;/li&gt;&#xA;&lt;li&gt;analyzing bifurcations and drawing bifurcation diagrams&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have found the following sources so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.springer.com/gp/book/9783540753919&quot; rel=&quot;nofollow&quot;&gt;Numerical Methods for Nonsmooth Dynamical Systems&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.springer.com/gp/book/9781402063558&quot; rel=&quot;nofollow&quot;&gt;Numerical Continuation Methods for Dynamical Systems&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.springer.com/us/book/9781461270447&quot; rel=&quot;nofollow&quot;&gt;Numerical Methods for Bifurcation Problems and Large-Scale Dynamical Systems&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898719543&quot; rel=&quot;nofollow&quot;&gt;Numerical Methods for Bifurcations of Dynamical Equilibria&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, these are just results I have found from doing searches using keywords like &quot;numerical dynamical systems bifurcation&quot;. I don't know the quality of these resources, and thus their usefulness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you comment if you have experience with a resource I have listed, or another one I might not have listed? By the way, I would use a &lt;code&gt;dynamical-systems&lt;/code&gt; tag, but I realized there isn't one!&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2015-05-01T17:51:41.140" Title="Resource recommendations for numerical methods involved in dynamical systems analysis" Tags="|numerical-analysis|reference-request|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19532" PostTypeId="2" ParentId="19529" CreationDate="2015-05-01T17:51:41.140" Score="5" Body="&lt;p&gt;In addition to the references you listed, you might like &lt;a href=&quot;https://books.google.com/books?isbn=3540306668&quot;&gt;Geometric Numerical Integration&lt;/a&gt; by Hairer, Lubich and Wanner. They go into a lot of detail about &quot;structure-preserving&quot; methods for solving systems of ODE. For example, given a Hamiltonian system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \dot q = \frac{\partial H}{\partial p}, \quad \dot p = -\frac{\partial H}{\partial q}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the trajectories lie along curves of constant $H$ and you would like your numerical scheme to reflect this fact as much as possible. Unfortunately, the usual forward and backward Euler methods don't do this at all, to the point that they can give you the wrong impression about the long-term stability of a dynamical system. However, for &lt;em&gt;symplectic&lt;/em&gt; methods, e.g. Stormer-Verlet, the numerical solution lies on an exact trajectory of a slightly different Hamiltonian $H + \delta H$.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-05-01T17:51:41.140" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19533" PostTypeId="2" ParentId="19016" CreationDate="2015-05-01T19:35:54.957" Score="2" Body="&lt;p&gt;The input for the Gonzalez algorithm is a set of elements to cluster and the distance between every pair of them. The first step of the Gonzalez algorithm is to select an element at random, this will be your first center (lets call it c_1). Since you already have a center, you must compute the distance from every element to c_1. Next, select as a new center the farthest element to c_1. In the remaining steps (k-2) you must keep computing the distance from every element to the set of already selected centers and just select as center the farthest element. The important thing here is that, in this context, the distance from an element e to a set of centers C, distance(e,C), is defined as the distance from e to its nearest center in C. I give you a general sketch for the Gonzalez algorithm, where matrix[i][j] stores the distance from element i to element j. I'm new at python, so my code is not optimal but gives you an idea.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def gon():&#xA;    C = []&#xA;    global distance = []&#xA;    for i in range(0,n):&#xA;        distance.append(float(&quot;inf&quot;))&#xA;    f = random.randint(0, n-1)&#xA;    C.append(f)&#xA;    update_distance(C, 0)&#xA;    for i in range(1, k):&#xA;        f = farthest_element()&#xA;        C.append(f)&#xA;        update_distance(C, i)&#xA;    size = solution_size()&#xA;    out = [size, C]&#xA;    return out&#xA;&#xA;# Update the distance of every element to the set of added centers&#xA;def update_distance(centers, max_index):&#xA;    global distance&#xA;    for i in range(0, n):&#xA;        if matrix[i][centers[max_index]] &amp;lt; distance[i]:&#xA;            distance[i] = matrix[i][centers[max_index]]&#xA;&#xA;# Get the farthest element&#xA;def farthest_element():&#xA;    max_dist = 0&#xA;    max_dist_element = 0&#xA;    global distance&#xA;    for i in range(0,n):&#xA;        if distance[i] &amp;gt; max_dist:&#xA;            max_dist = distance[i]&#xA;            max_dist_element = i&#xA;    return max_dist_element&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15250" LastActivityDate="2015-05-01T19:35:54.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19535" PostTypeId="1" CreationDate="2015-05-02T02:37:27.943" Score="6" ViewCount="141" Body="&lt;p&gt;An unpivoted QR factorization produces a triangular factor $R$. A rank-revealing QR factorization is typically done with column pivoting. My question is, how does an unpivoted QR factorization fail to reveal rank? If a matrix is singular, then a diagonal element of $R$ must be zero or tiny. And this element is almost always (as far as I can tell) the bottom left element.&lt;/p&gt;&#xA;" OwnerUserId="317" LastActivityDate="2015-05-02T07:57:29.890" Title="How does an unpivoted QR fail to reveal rank?" Tags="|linear-algebra|matrix|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19538" PostTypeId="1" AcceptedAnswerId="19540" CreationDate="2015-05-02T20:51:16.200" Score="1" ViewCount="215" Body="&lt;p&gt;I'm learning MPI in order to use MPI with a model written in Fortran. What are some good resources (books, websites, etc.)? Introductory/beginner material, and detailed references would both be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="15220" LastActivityDate="2015-05-06T05:58:03.817" Title="Reference Suggestions for MPI" Tags="|reference-request|mpi|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19539" PostTypeId="1" AcceptedAnswerId="19541" CreationDate="2015-05-03T09:10:43.753" Score="2" ViewCount="133" Body="&lt;p&gt;I am using the Hydra Process Manager from MPICH and I plan on running jobs on multiple compute nodes. Currently working with an Intel Xeon E5-2680v2 with a total of 20 cores with 2 sockets (32 GB of memory and 25 MB L3 cache for each socket). Socket 0 has hwthreads labeled 0-9 and Socket 1 with 10-19.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see within hydra's documentation that you can specify the motherboard option &quot;B&quot;. In the context of HPC does this equate to the compute node? That is, say if I wanted to run a job on 32 processes on 2 of these compute nodes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mpiexec.hydra -n 32 -bind-to hwthread -map-by board ./myprogram &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it divide or bind the processes equally among my two nodes? If not, do you know how I can achieve this?&lt;/p&gt;&#xA;" OwnerUserId="14576" OwnerDisplayName="Justin" LastActivityDate="2015-05-03T14:05:38.963" Title="On an HPC machine, does one compute node equate to one motherboard?" Tags="|parallel-computing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19540" PostTypeId="2" ParentId="19538" CreationDate="2015-05-03T12:46:24.397" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/mpi/usingmpi/&quot; rel=&quot;nofollow&quot;&gt;two books&lt;/a&gt; that come to mind are Using MPI and Using Advanced MPI by Gropp and Lusk. They were also released in a 3rd edition last year, so they are very up to date. As others have said, the &lt;a href=&quot;http://mpi-forum.org/docs/&quot; rel=&quot;nofollow&quot;&gt;reference manuals&lt;/a&gt; are open and available for free. It's nice to have the books to help you get started and the reference manuals to look up certain details.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-03T12:46:24.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19541" PostTypeId="2" ParentId="19539" CreationDate="2015-05-03T14:05:38.963" Score="3" Body="&lt;p&gt;You've asked two separate questions here. The second one is better addressed by the &lt;a href=&quot;https://www.mpich.org/support/&quot; rel=&quot;nofollow&quot;&gt;MPICH Support&lt;/a&gt; options or documentation. The answer to the title question is &quot;yes (typically)&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen designs or proposed designs where more than one node (think of independent Linux OSes with independent memories) is physically mounted on the same PCB board, but this isn't very common. Typically, for manufacturing and engineering reasons, each node is on a separate PCB board or motherboard. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-03T14:05:38.963" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19543" PostTypeId="2" ParentId="19288" CreationDate="2015-05-03T17:07:02.187" Score="2" Body="&lt;p&gt;To partially echo @aeroNotAuto, meshing algorithms are crucial.&#xA;Here is a &lt;a href=&quot;http://www.cs.berkeley.edu/~jrs/mesh/present.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;useful page&lt;/em&gt;&lt;/a&gt; listing important papers on meshing,&#xA;from a Berkeley course taught by Jonathan Shewchuck.&#xA;&lt;hr /&gt;&#xA;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/7tLnX.jpg&quot; alt=&quot;MeshSandals&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr /&gt;&#xA;" OwnerUserId="673" LastActivityDate="2015-05-03T17:07:02.187" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19544" PostTypeId="1" AcceptedAnswerId="19559" CreationDate="2015-05-03T18:48:05.083" Score="3" ViewCount="731" Body="&lt;p&gt;The default option for matplotlib's &lt;code&gt;pcolormesh&lt;/code&gt; (also true for VisIt and I think also true for Matlab) is to allow the maximum and minimum of a colorbar vary. Which in my case looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/dxZVH.png&quot; alt=&quot;before&quot;&gt; &lt;img src=&quot;https://i.stack.imgur.com/hMYM4.png&quot; alt=&quot;after&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that this default behavior makes sense and is also much more aesthetically pleasing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my advisor wants me to keep the colorbar at some fixed maximum throughout the plots. The result is the same for the initial plot and this for the final plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/KMWNA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the tracer concentration has entirely disappeared!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason he gave for keeping the colorbar fixed is that otherwise it is deceiving; at a glance it looks like there is more tracer concentration later than before, but there should be less since it has diffused. Also, he argued that the intense colors distract and make it hard to locate where the vortices have moved to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel like a reader should be assumed to be competent enough to look at the colorbar. Also, the I think that the colors help the reader to see where the vortex is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is:&lt;/strong&gt; for scientific plots is there a default/consensus position either that (i) plots should have variable colorbars or (ii) plots should have fixed colorbars?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, what factors should guide one to producing plots (with colorbars) with the most scientific value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I say &quot;scientific value&quot; as opposed to &quot;aesthetic value&quot; but if you think aesthetics should be a factor of some weight please justify yourself or give a reference to support yourself.&lt;/p&gt;&#xA;" OwnerUserId="15264" LastEditorUserId="9667" LastEditDate="2015-05-03T20:13:13.500" LastActivityDate="2021-05-13T21:12:33.393" Title="Should a colorbar have constant values throughout a simulation?" Tags="|simulation|visualization|plotting|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19545" PostTypeId="1" CreationDate="2015-05-03T20:00:30.870" Score="6" ViewCount="730" Body="&lt;p&gt;I'm interested in implementing the muscle model used in &lt;a href=&quot;http://www.goatstream.com/research/thesis/index.html&quot; rel=&quot;noreferrer&quot;&gt;Geijtenbeek&lt;/a&gt; and &lt;a href=&quot;http://graphics.stanford.edu/projects/bio-locomotion/&quot; rel=&quot;noreferrer&quot;&gt;Wang et al's&lt;/a&gt; work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both papers link to the paper by &lt;a href=&quot;https://www.cs.cmu.edu/~hgeyer/Publications/Geyer&amp;amp;Herr-ReflexModel2Column.pdf&quot; rel=&quot;noreferrer&quot;&gt;Geyer and Herr&lt;/a&gt;, which describes this model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pAHmS.png&quot; alt=&quot;Geyer and Herr muscle model&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the paper on this model assumes you already know how to model a Hill-Type muscle, and are simply augmenting it and placing many copies of it together in a biologically accurate way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in trying to learn how to implement the model itself, I found &lt;a href=&quot;http://www.gribblelab.org/compneuro/&quot; rel=&quot;noreferrer&quot;&gt;Gribble's Tutorial&lt;/a&gt; which was informative, but a very simplified model of what is used in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found &lt;a href=&quot;http://jn.physiology.org/content/77/6/3237&quot; rel=&quot;noreferrer&quot;&gt;Prochazka et al's&lt;/a&gt; paper, which seems to be the best detailed reference on implementation, however it describes it in the context of a visual language for describing dynamic systems (Simulink). So I have been learning simulink to do this, but once I reproduce the models it will require some kind of &lt;a href=&quot;http://www.mathworks.com/products/simulink-coder/&quot; rel=&quot;noreferrer&quot;&gt;Code Generation&lt;/a&gt; tool to convert into c code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can do this, but I was wondering if there was a better, more standard way to go about implementing a Hill-Type muscle model because the models presented there seemed overly complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't mind a description of how to implement one here as an answer, but a simple reading reference will also suffice.&lt;/p&gt;&#xA;" OwnerUserId="8275" LastActivityDate="2018-08-03T08:48:54.437" Title="Implementing a Hill-Type Muscle Model" Tags="|nonlinear-equations|computational-biology|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19546" PostTypeId="1" CreationDate="2015-05-03T20:18:18.987" Score="0" ViewCount="161" Body="&lt;p&gt;I have data of the form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X       Y&#xA;3.53    0&#xA;4.93    50&#xA;5.53    60&#xA;6.21    70&#xA;7.37    80&#xA;9.98    90&#xA;16.56   100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I want to find out $n$ so that this can be fit to a function of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y = \frac{a}{x^n} + b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to determine $n$ by Box-Cox transformation. Can n be determined with Box-Cox transformation?&lt;/p&gt;&#xA;" OwnerUserId="14586" LastEditorUserId="9667" LastEditDate="2015-05-07T15:54:27.683" LastActivityDate="2015-08-05T18:10:33.597" Title="Can Box-Cox transformation be applied for data of this form?" Tags="|statistics|regression|data-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19549" PostTypeId="2" ParentId="19544" CreationDate="2015-05-03T21:36:27.790" Score="4" Body="&lt;p&gt;I agree that using the same color scale is generally good practice. Not doing so is confusing. Now, as you note, there are cases where this doesn't leave very much information in each picture. In such cases, you should at least make it clear in the caption that you are using different color scales for different panels of a figure.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-05-03T21:36:27.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19550" PostTypeId="1" CreationDate="2015-05-04T00:11:23.360" Score="1" ViewCount="90" Body="&lt;p&gt;I am working with an objective function that is convex globally, but the path downward is lined (if you will) with quartz crystals. In this case, the update vector (gradient solution) of partial derivatives of a potential crystal's surface you land on sometimes points upward, so the algorithm throws you backward. Hence, it's a quite bumpy objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thought of an approach similar to genetic algorithms, where I could add a small random value in the range [-0.5,0.5] to each gradient update, and have also thought about using orthogonal vectors for which the question becomes: which dimension and therefore which orthogonal vector?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, a line search assumes a monotonically decreasing function that's continuously differentiable in the limit. Aside from using GA's is there a more recent approach for a bumpy landscape when derivatives are known?&lt;/p&gt;&#xA;" OwnerDisplayName="user15259" LastActivityDate="2015-05-04T03:02:25.013" Title="Gradient Descent" Tags="|optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19551" PostTypeId="2" ParentId="19538" CreationDate="2015-05-04T01:02:21.807" Score="3" Body="&lt;p&gt;First, you should make sure that you understand the fundamentals of parallel processing first.  It's very hard to write good MPI code if you don't understand the principles of distributed computing.  You might find &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0321940784&quot; rel=&quot;nofollow&quot;&gt;Patterns for Parallel Programming&lt;/a&gt; useful, although there are plenty of other books on the topic of parallel programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Full disclosure: I work for one of the authors of &quot;Patterns for Parallel Programming&quot;, but I have confirmed that none of the authors make enough money off of it to justify me recommending for any other reason than technical value.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, read well-written MPI applications (not necessarily Fortran ones - see the next point) for insight.  It is often the case that the examples suitable for textbooks and tutorials gloss over some of the nuances that you will need to understand to be successful in your work.  For example, every book I've seen on MPI covers the 2D heat equation solver using the Jacobi method, but absolutely no one should use Jacobi in the &quot;real world&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, you may find that there is a lot of good MPI material written for C programmers, which means that you should not focus specifically on Fortran in your search for knowledge.  And without trying to be a language bigot, I find that the MPI examples written by C programmers are - on average - better than those written by Fortran programmers.  This is correlation (primarily with the authors' academic training), not causality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fourth, you will inevitably write an incorrect MPI program some day and need to debug it.  OpenMPI's &lt;a href=&quot;http://www.open-mpi.org/faq/?category=debugging&quot; rel=&quot;nofollow&quot;&gt;FAQ: Debugging applications in parallel&lt;/a&gt; is a great starting point for learning about that topic.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2015-05-04T01:02:21.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19552" PostTypeId="2" ParentId="19550" CreationDate="2015-05-04T03:02:25.013" Score="2" Body="&lt;p&gt;A common approach to dealing with noisy objective functions is to use regression techniques to build a smooth and convex surrogate &quot;response surface&quot; that approximates the function and then minimize over the response surface.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-05-04T03:02:25.013" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19554" PostTypeId="1" CreationDate="2015-05-04T14:46:13.353" Score="1" ViewCount="369" Body="&lt;p&gt;This question is related to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19457/numerical-integration-of-a-function-whose-expression-is-unknown&quot;&gt;the previous one&lt;/a&gt;, but I believe the extension to several variables is a problem in itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a collection of points $(x_j,y_k,f(x_j,y_k))$ for $x_j=\frac{j}{n}$ and $y_k=\frac{k}{n}$ a grid over $[0,1]^2$. These values come from a PDE solver for $f$. I need to compute the integral of $f$ on $[0,1]^2$. How could I make it ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, what is the natural extension of the trapezoidal rule for higher dimensions ? As I see it, the problem is that, while there is a unique segment interpolating two points, there can be no plane interpolating four points.&lt;/p&gt;&#xA;" OwnerUserId="7784" LastActivityDate="2015-05-05T07:24:55.037" Title="Numerical integration of a multivariate function whose expression is unknown" Tags="|quadrature|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19555" PostTypeId="1" AcceptedAnswerId="19571" CreationDate="2015-05-04T15:24:33.637" Score="0" ViewCount="982" Body="&lt;p&gt;I have a 2D square ABCD in a 3D space, with side length &lt;code&gt;2s&lt;/code&gt;, that is represented by four vectors (one for each vertex) and a fifth vector &lt;code&gt;v&lt;/code&gt; for the center point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The square lies standing upright on the plane &lt;code&gt;z = -1&lt;/code&gt; (xy plane).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v  = &amp;lt; 0, 0, -1&amp;gt;&#xA;OA = &amp;lt;-s, s, -1&amp;gt;&#xA;OB = &amp;lt; s, s, -1&amp;gt;&#xA;OC = &amp;lt;-s,-s, -1&amp;gt;&#xA;OD = &amp;lt; s,-s, -1&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, consider point &lt;code&gt;P&lt;/code&gt; anywhere in the 3D space. I want to rotate the square around the origin, such that &lt;code&gt;v&lt;/code&gt; aligns with &lt;code&gt;OP&lt;/code&gt;. The result that I want is mainly the resulting rotated &lt;code&gt;OA&lt;/code&gt;, &lt;code&gt;OB&lt;/code&gt;, &lt;code&gt;OC&lt;/code&gt; and &lt;code&gt;OD&lt;/code&gt; vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cldup.com/wfdBaFSseC.png&quot; rel=&quot;nofollow&quot;&gt;Screenshot of scenario&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point &lt;code&gt;P&lt;/code&gt; is arbitrary and can be any point in the 3D space. Any help is greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="15271" LastActivityDate="2015-05-06T03:33:47.750" Title="Rotate 2D shape around origin in a 3D space" Tags="|vector|3d|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19557" PostTypeId="2" ParentId="19554" CreationDate="2015-05-04T15:43:38.213" Score="1" Body="&lt;p&gt;There's no reason to think that you'll find a plane that intersects all four points in general, but you can try various other approximations. A few quick ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Some sort of midpoint rule where you assume that the average value of $f $ across the region $[j/n,(j+1)/n] \times [k/n,(k+1)/n]$ is simply the average of the four corner values.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of assuming a plane through all four corners, assume a linear interpolation in $x$ along the slices $y = k/n$ and $(k+1)/n$, then a linear interpolation in $y$ between those two linear fits; this results in an equation of the form $a + bx + cy + dxy$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some Googling for things like &quot;2D numerical integration&quot; will turn up a variety of methods.  It's all variations on the idea of estimating the value of $f$ between your data points and then integrating between the data points.  Which method you choose will depend on how accurate you want your results to be, and how much work you want to do to compute the integral.&lt;/p&gt;&#xA;" OwnerUserId="15167" LastEditorUserId="9667" LastEditDate="2015-05-04T20:40:24.487" LastActivityDate="2015-05-04T20:40:24.487" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19558" PostTypeId="2" ParentId="19554" CreationDate="2015-05-04T16:04:55.257" Score="0" Body="&lt;p&gt;If you have a fixed grid, the simplest integration you can do would be to sum up the average values multiplied by the area they're on, like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_0^1 \int_0^1 f(x,y) dx dy \approx \sum_{i,j}^{NI-1,NJ-1} 1/4 (f_{i,j}+f_{i+1,j}+f_{i,j+1}+f_{i+1,j+1})\Delta x \Delta y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're just doing some simple post-processing, this may be sufficiently accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general if you can sample $f(x,y)$ at arbitrary locations, you'd want to look at doing multi-dimensional &lt;a href=&quot;http://en.wikipedia.org/wiki/Gaussian_quadrature&quot; rel=&quot;nofollow&quot; title=&quot;quadrature&quot;&gt;quadrature.&lt;/a&gt; There is free code for this in the GNU scientific library and others. Higher-dimensional quadrature is done by tensor products of the 1D case.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As I see it, the problem is that, while there is a unique segment interpolating two points, there can be no plane interpolating four points.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;True, although you can always construct a unique &lt;a href=&quot;http://en.wikipedia.org/wiki/Lagrange_polynomial&quot; rel=&quot;nofollow&quot;&gt;Lagrange polynomial&lt;/a&gt; which you can then integrate analytically. Any of this is available in matlab.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2015-05-04T16:04:55.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19559" PostTypeId="2" ParentId="19544" CreationDate="2015-05-04T17:32:48.843" Score="6" Body="&lt;p&gt;The selection of colormap should be based on your dataset and audience, e.g., you do not want to use a colormap that have some cultural background for a group of people. Also, if your images are going to be printed (in grey scale), you should consider using a colormap that will preserve the ordering after the color transformation.&lt;/p&gt;&#xA;&lt;p&gt;Then, you should take into account that when using/designing a colormap you are doing a mapping from the values in your dataset to a color space. This space can be defined in &lt;span class=&quot;math-container&quot;&gt;$(R,G,B)$&lt;/span&gt; components, but there exist other color spaces that are more intuitive like &lt;a href=&quot;https://en.wikipedia.org/wiki/HSL_and_HSV&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$(H,S,V)$&lt;/span&gt; (Hue, Saturation, Value) and &lt;span class=&quot;math-container&quot;&gt;$(H,S,L)$&lt;/span&gt; (Hue, Saturation, Lightness)&lt;/a&gt;. Based on this, we can think that a (univariate) color scale is equivalent to a path in the color space. The rainbow colormap changes the Hue. From the &lt;a href=&quot;http://matplotlib.org/examples/color/colormaps_reference.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;matplotlib documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sequential:&#xA;    These colormaps are approximately monochromatic colormaps varying smoothly&#xA;    between two color tones---usually from low saturation (e.g. white) to high&#xA;    saturation (e.g. a bright blue). Sequential colormaps are ideal for&#xA;    representing most scientific data since they show a clear progression from&#xA;    low-to-high values.&#xA;&#xA;Diverging:&#xA;    These colormaps have a median value (usually light in color) and vary&#xA;    smoothly to two different color tones at high and low values. Diverging&#xA;    colormaps are ideal when your data has a median value that is significant&#xA;    (e.g.  0, such that positive and negative values are represented by&#xA;    different colors of the colormap).&#xA;&#xA;Qualitative:&#xA;    These colormaps vary rapidly in color. Qualitative colormaps are useful for&#xA;    choosing a set of discrete colors. For example::&#xA;&#xA;        color_list = plt.cm.Set3(np.linspace(0, 1, 12))&#xA;&#xA;    gives a list of RGB colors that are good for plotting a series of lines on&#xA;    a dark background.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can also use redundant colormaps, where you play with more than one parameter at the time, an example for the &lt;a href=&quot;https://mycarta.wordpress.com/2012/12/06/the-rainbow-is-deadlong-live-the-rainbow-part-5-cie-lab-linear-l-rainbow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;rainbow colormap&lt;/a&gt;. This &lt;a href=&quot;https://youtu.be/rkDgBvT-giw&quot; rel=&quot;nofollow noreferrer&quot;&gt;talk in SciPy 2014&lt;/a&gt;, discuss some topics of colormaps in matplotlib and is really enlightening, the slidese can be found &lt;a href=&quot;https://github.com/dmcdougall/scipy14-colormaps&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Why using colormaps&lt;/h2&gt;&#xA;&lt;p&gt;Some reasons to use colormaps are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mimic reality (use the same set of color that appear in the physical world)&lt;/li&gt;&#xA;&lt;li&gt;Show classification (qualitative)&lt;/li&gt;&#xA;&lt;li&gt;Show value (quantitative)&lt;/li&gt;&#xA;&lt;li&gt;Draw attention&lt;/li&gt;&#xA;&lt;li&gt;Show grouping / similarities&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;That is why some scales are better at qualitative judgments [2] (Relative shapes and sizes). And, Other color scales are better for quantitative&#xA;judgments (Looking up values), that is the application that you are interested now.&lt;/p&gt;&#xA;&lt;h2&gt;Trumbos's Principles&lt;/h2&gt;&#xA;&lt;p&gt;Trumbo's Principles present some guidelines to take into account when designing/using colormaps [1]:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Order: ordered values should be represented by&#xA;ordered colors&lt;/li&gt;&#xA;&lt;li&gt;Separation: significantly different levels should be&#xA;represented by distinguishable colors&lt;/li&gt;&#xA;&lt;li&gt;Rows and columns: to preserve univariate&#xA;information, display parameters should not obscure&#xA;one another&lt;/li&gt;&#xA;&lt;li&gt;Diagonal: to show positive association, displayed&#xA;colors should group into three perceptual classes:&#xA;diagonal, above, below&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The rainbow colormap is not good at present the color information ordered, and that is why some people suggest to avoid it [3]. Although there are versions of the raibow colormap that are redundant and present the ordering property. Some examples of &lt;a href=&quot;http://www.ncl.ucar.edu/Document/Graphics/color_table_gallery.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;colormaps&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&lt;p&gt;[1] Bruce E. Trumbo (1981), Theory for Coloring Bivariate Statistical Maps,&#xA;The American Statistician, vol. 35, no. 4, pp. 220-226. &lt;a href=&quot;http://www.tandfonline.com/doi/abs/10.1080/00031305.1981.10479360&quot; rel=&quot;nofollow noreferrer&quot;&gt;url&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;[2] Ware, C., (1988). Color Sequences for Univariate Maps: Theory, Experiments and Principles. IEEE Computer Graphics and Applications. 8(5), 41-49. &lt;a href=&quot;http://ccom.unh.edu/vislab/PDFs/ColorSequences.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;url&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;[3] Borland, David, and Russell M. Taylor II. &amp;quot;Rainbow color map (still) considered harmful.&amp;quot; IEEE computer graphics and applications 27.2 (2007): 14-17. &lt;a href=&quot;http://data3.mprog.nl/course/15%20Readings/40%20Reading%204/Borland_Rainbow_Color_Map.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;url&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[4] Kovesi Peter (2015) Bad Colour Maps Hide Big Features and Create False Anomalies. ASEG Extended Abstracts 2015 , 1–4. &lt;a href=&quot;http://www.publish.csiro.au/?paper=ASEG2015ab107&quot; rel=&quot;nofollow noreferrer&quot;&gt;url&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[5] Rougier, Nicolas P., Michael Droettboom, and Philip E. Bourne. &amp;quot;Ten simple rules for better figures.&amp;quot; PLoS computational biology 10.9 (2014): e1003833. &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003833&quot; rel=&quot;nofollow noreferrer&quot;&gt;url&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2021-05-13T21:12:33.393" LastActivityDate="2021-05-13T21:12:33.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19560" PostTypeId="2" ParentId="19554" CreationDate="2015-05-04T20:39:11.647" Score="2" Body="&lt;p&gt;There is a simple extension of trapezoid rule to 2d. In 1d, if you have the interval $[0,1]$, the rule takes the linear interpolant&#xA;$$ l(x) = f(0)(1-x) + f(1)x, $$&#xA;and integrates it to get&#xA;$$ \int_0^1 l(x) = \tfrac12 f(0) + \tfrac12 f(1). $$&#xA;To integrate over a large interval, you then sum up the contributions of the individual small intervals to get the usual trapezoid formula:&#xA;$$ h \big(\tfrac12 f(x_0) + f(x_1) + \cdots + f(x_{n-1}) + \tfrac12 f(x_n) \big). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2d, on the square $[0,1]^2$, you can interpolate the function linearly on the bottom edge $y=0$, and separately on the top edge $y=1$, and then interpolate linearly between the two linear interpolants:&#xA;$$ l_0(x) = f(0,0)(1-x)+f(1,0)x, \qquad l_1(x) = f(0,1)(1-x)+f(1,1)x, $$&#xA;$$ l(x,y) = l_0(x)(1-y) + l_1(x)y. $$&#xA;The resulting interpolant is a linear combination of elementary interpolants on the square $[0,1]^2$, with coefficients being the function values, and the elementary interpolants look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/255H2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the elementary interpolants are the coefficients of the function values $f(x,y)$ in the interpolant constructed from them:&#xA;$$ l(x,y) = f(0,0)(1-x)(1-y) + f(1,0)x(1-y) + f(0,1) (1-x)y + f(1,1)xy, $$&#xA;so of the four functions $xy$, $x(1-y)$, $(1-x)y$ and $(1-x)(1-y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral of $l(x,y)$ will be&#xA;$$ \int_{[0,1]^2}l(x,y)\,dx\,dy = \tfrac14\big( f(0,0)+f(0,1)+f(1,0)+f(1,1) \big). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summing up these contributions over small intervals would then give&#xA;$$ \frac{h^2}{4}\sum_{j,k} w_{j,k} f(x_j, y_k) $$&#xA;where $w_{j,k}$ is an integer that counts how many squares the point $(x_j,y_k)$ belongs to: one in the corners, two on the edges, four in the middle.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-05-05T07:24:55.037" LastActivityDate="2015-05-05T07:24:55.037" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19561" PostTypeId="2" ParentId="19496" CreationDate="2015-05-05T00:33:00.397" Score="1" Body="&lt;p&gt;I will expand the answer provided by @DavidKetcheson. First the equations are rewritten as a hyperbolic system of first-order conservation laws:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q_t + \nabla \cdot F(q) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q_t + A q_x + B q_y + C q_z = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $q$ is a state vector formed with the components of the stress tensor $(\sigma_{11}, \sigma_{22}, \sigma_{33}, \sigma_{12}, \sigma_{23}, \sigma_{13})$ and components of the velocity vector $(u, v, w)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q = \begin{pmatrix}&#xA;\sigma_{11}\\ \sigma_{22}\\ \sigma_{33}\\ \sigma_{12}\\ \sigma_{23}\\ \sigma_{13}\\ u\\ v\\ w &#xA;\end{pmatrix} \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = \begin{pmatrix}&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{11} &amp;amp; {c}_{16} &amp;amp; {c}_{15}\\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{12} &amp;amp; {c}_{26} &amp;amp; {c}_{25}\\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{13} &amp;amp; {c}_{36} &amp;amp; {c}_{35}\\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{14} &amp;amp; {c}_{46} &amp;amp; {c}_{45}\\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{15} &amp;amp; {c}_{56} &amp;amp; {c}_{55}\\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{16} &amp;amp; {c}_{66} &amp;amp; {c}_{56}\\&#xA; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0&#xA;\end{pmatrix} \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B = \begin{pmatrix}&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{16} &amp;amp; {c}_{12} &amp;amp; {c}_{14}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{26} &amp;amp; {c}_{22} &amp;amp; {c}_{24}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{36} &amp;amp; {c}_{23} &amp;amp; {c}_{34}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{46} &amp;amp; {c}_{24} &amp;amp; {c}_{44}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{56} &amp;amp; {c}_{25} &amp;amp; {c}_{45}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{66} &amp;amp; {c}_{26} &amp;amp; {c}_{46}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0&#xA;\end{pmatrix} \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C = \begin{pmatrix}&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{15} &amp;amp; {c}_{14} &amp;amp; {c}_{13}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{25} &amp;amp; {c}_{24} &amp;amp; {c}_{23}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{35} &amp;amp; {c}_{34} &amp;amp; {c}_{33}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{45} &amp;amp; {c}_{44} &amp;amp; {c}_{34}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{55} &amp;amp; {c}_{45} &amp;amp; {c}_{35}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; {c}_{56} &amp;amp; {c}_{46} &amp;amp; {c}_{36}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; \frac{1}{\rho} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0&#xA;\end{pmatrix} \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to compute the speeds of the problem (as described above) we need to form the matrix $\hat{A}(n_1, n_2, n_3) = n_1 A + n_2 B + n_3 C$, where $\mathbb{n}=(n_1, n_2, n_3)$ is a unit vector and determines the direction of propagation. To find the CFL condition it is necessary to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{(\theta, \phi)} \max_i \gamma_i(\theta, \phi) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $(\theta, \phi)$ are spherical angles and $\gamma_i$ are the eigenvalues of the matrix $\hat{A}(\theta, \phi)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on this, and the answer provided by @DavidKetcheson it is simpler to compute the eigenvalues of the Christoffel equation, and solve the optimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{(\theta, \phi)} \max_i \lambda_i(\theta, \phi) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\lambda_i$ eigenvalues of the Christoffel equation. And the speed is just $c = \sqrt{\lambda_i/\rho}$.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-03-22T19:53:41.167" LastActivityDate="2017-03-22T19:53:41.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19564" PostTypeId="1" CreationDate="2015-05-05T07:52:00.623" Score="3" ViewCount="108" Body="&lt;p&gt;Let us consider the following equation :&#xA;$$\partial_t u = \Delta_x u + K*u$$&#xA;where $K$ is a smooth kernel, $u(x,t)$ is the unknown and $x$ is in $\Omega$ a domain in 1d or 2d.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to numerically solve it by the method of lines : &#xA;$$\dot{u_i} = \frac{u_{i+1}-2u_i+u_{i-1}}{h^2} + (K * u)_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a good way to go ? And is there any trick for the integral part $(K * u)_i$ ? I find it very time consuming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is actually&#xA;$$\partial_t u = \Delta_x u + u (K*u)$$&#xA;The method of lines gives&#xA;$$\dot{u_i} = \frac{u_{i+1}-2u_i+u_{i-1}}{h^2} + u_i (K * u)_i$$&lt;/p&gt;&#xA;" OwnerUserId="7784" LastEditorUserId="7784" LastEditDate="2015-05-06T09:00:15.070" LastActivityDate="2015-05-06T16:24:56.847" Title="Integro-differential PDE and method of lines" Tags="|finite-difference|method-of-lines|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19565" PostTypeId="1" CreationDate="2015-05-05T09:43:30.223" Score="5" ViewCount="981" Body="&lt;p&gt;Ok, I've already asked this question in &lt;a href=&quot;https://math.stackexchange.com/q/1257497/144053&quot;&gt;math.stackexchange&lt;/a&gt;, but I feel it is more appropriate to ask here (hopefully I am not violating any rules by repeating!). So here it is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I wonder if there is a local convergence proof for ADMM applied to biconvex problems? More specifically, my problem is as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min_{x,y} f(x) + g(y) + \| y \circ Ax \|_2^2  $$&lt;/span&gt; ,&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\circ$&lt;/span&gt; denotes Hadamard product and &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g(y)$&lt;/span&gt; are convex functions. This fits into biconvex setting mentioned in Boyd's paper on distributed ADMM, but without theoretical evidence.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15277" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-06-01T12:44:39.183" Title="Is there a convergence proof for ADMM applied to biconvex/bilinear problems?" Tags="|optimization|admm|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19566" PostTypeId="2" ParentId="19506" CreationDate="2015-05-05T12:23:57.627" Score="4" Body="&lt;p&gt;What you suggest is a multi-year research program. Don't even think about doing this yourself, you will not within several years achieve anything that comes close to the current research. Rather, search the web for codes that are available and adjust them to your needs. A good first place would be the Department of Energy's Computational Combustion research center and network.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-05-05T12:23:57.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19567" PostTypeId="1" CreationDate="2015-05-05T14:27:20.243" Score="1" ViewCount="101" Body="&lt;p&gt;When sampling configurations of point particles in statistical mechanics by means of (Markov chain) Monte Carlo, we may work with a potential that only depends on the positions of the particles, and to generate new states we assign probabilities to configurations according to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Boltzmann_distribution&quot; rel=&quot;nofollow&quot;&gt;Boltzmann distribution&lt;/a&gt;, which (to my understanding) arises from the (temperature dependent) distribution of velocities of the particles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can we do when we want to work with a (generalized) potential that also depends on the velocity of the particles? I would assume that in that case we need to consider the velocity of each particle, along with its position, to be part of the state, that statistics alone doesn't suffice anymore. Or does it? How do we assign relative probabilities to different configurations? Can we still use the Boltzmann distribution? &lt;/p&gt;&#xA;" OwnerUserId="5521" LastEditorUserId="5521" LastEditDate="2015-05-06T08:08:14.803" LastActivityDate="2015-05-06T08:08:14.803" Title="Monte Carlo sampling of particle system for velocity dependent potential" Tags="|monte-carlo|random-sampling|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19568" PostTypeId="1" CreationDate="2015-05-05T17:35:40.227" Score="1" ViewCount="53" Body="&lt;p&gt;I am maximizing a linear functional subject to an integrates to one constraint. More explicitly, my problem is&#xA;$$\begin{align}&#xA;&amp;amp;\max_{x \in \mathbb{R}^n}\quad c \cdot x\\&#xA;&amp;amp;\text{subject to} \int_{a}^{b} \prod_{i=1}^{n} e^{x_i \cdot y^i} dy &amp;lt;=d&#xA;\end{align}$$&#xA;where $a, b, d \in \mathbb{R}$, $c \in \mathbb{R}^n$, and $n \in \mathbb{N} $ are constants. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have plugged this into the convex solver &lt;code&gt;SCS&lt;/code&gt;, relaxing my integral constraint to a crude Riemann-sum type estimate. Because the sum of exponential constraints is so large in this expression, it takes a very long time to solve. This is my first experience working numerically with a constraint like this, and I would like to increase the speed and/or accuracy of this solution. Is there any more efficient, or computationally tractable, method to implement such an integration constraint like the one above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose I am hoping for some &quot;convergence in epigraph&quot; type result for approximating this constraint. Is this something that is realistic to hope for?&lt;/p&gt;&#xA;" OwnerUserId="11642" LastEditorUserId="11642" LastEditDate="2015-05-08T00:37:09.200" LastActivityDate="2015-05-08T00:37:09.200" Title="Numerical Implementation of &quot;integrates to some values&quot; type constraint in convex solvers?" Tags="|optimization|convex-optimization|constraints|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19569" PostTypeId="2" ParentId="19564" CreationDate="2015-05-05T17:42:09.200" Score="3" Body="&lt;p&gt;Do a Fourier transform in space and solve the ODEs that describe the first $N$ Fourier coefficient. This is possible because the Laplacian turns into a multiplication with the wave number upon Fourier transform, and the convolution turns into a straight multiplication. This makes all operations reasonably trivial, particularly if all you have is a 1d domain.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-05-05T17:42:09.200" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19570" PostTypeId="1" CreationDate="2015-05-05T23:31:56.090" Score="1" ViewCount="53" Body="&lt;p&gt;I am trying to optimize the execution time for this particular piece of fortran code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Details:&#xA;i_gc is a (ngpts, 3) array of containing (i,j,k) indices for each grid point. This is a subset of the full NX x NY x NZ grid. The subset is connected in the sense that density of sampling varies, but the points aren't scattered randomly throughout the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially I have a non-local operator on the grid which is a tensor product of 3 one dimensional operators. If I had the data on the full 3d grid it would be pretty easy to write this code without all the if statements in the inner loop. Also, I would like to avoid storing a set {i},{j},{k} indices for each grid point, but maybe this is the way to go?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there is a way to do much better than this? I do have some freedom to reorder the points if needed. Maybe I can exploit that somehow? I suppose I could try something like a Hilbert curve, but I'm not sure how well that could apply to my case where I have an oddly shaped subset of a regular grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit:&#xA;mathematically the operation is like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Y_{ijk} = \sum^{N^x}_{q=1} M^x_{qi} X_{qjk} + &#xA;\sum^{N^y}_{q=1} M^y_{qj} X_{iqk} + &#xA;\sum^{N^z}_{q=1} M^z_{qk} X_{ijq} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the sums are only over points in the subset (i_gc)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine apply_lf4(ngpts, i_gc, mx,my,mz,x,y)&#xA;  implicit none&#xA;  integer, intent(in) :: ngpts&#xA;  integer, intent(in), dimension(:,:) :: i_gc&#xA;  real(8), intent(in), dimension(:,:) :: mx,my,mz&#xA;  real(8), intent(in), dimension(ngpts) :: x&#xA;  real(8), intent(out), dimension(ngpts) :: y&#xA;&#xA;  integer :: ig,jg,i1,i2,i3,j1,j2,j3&#xA;&#xA;  y=0.0d0&#xA;&#xA;  do ig=1, ngpts&#xA;     i1=i_gc(ig,1); i2=i_gc(ig,2); i3=i_gc(ig,3)&#xA;    inner: do jg=1, ngpts&#xA;        j1=i_gc(jg,1)&#xA;        if (j1==i1) then&#xA;           j2=i_gc(jg,2)&#xA;           if (j2==i2) then&#xA;              j3 = i_gc(jg,3)&#xA;              y(ig) = y(ig) + mz(j3,i3)*x(jg)&#xA;              cycle inner&#xA;           end if&#xA;           j3=i_gc(jg,3)&#xA;           if (j3==i3) then&#xA;              y(ig) = y(ig) + my(j2,i2)*x(jg)&#xA;              cycle inner&#xA;           end if&#xA;        else&#xA;           j2 = i_gc(jg,2)&#xA;           if (j2 == i2) then&#xA;              j3 = i_gc(jg,3)&#xA;              if (j3 == i3) then&#xA;                 y(ig) = y(ig) + mx(j1,i1)*x(jg)&#xA;                 cycle inner&#xA;              end if&#xA;           end if&#xA;        end if                        &#xA;     end do inner&#xA;  end do             &#xA;&#xA;end subroutine apply_lf4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14534" LastEditorUserId="14534" LastEditDate="2015-05-06T02:21:50.163" LastActivityDate="2015-05-06T02:21:50.163" Title="Optimization of nonlocal stencil-like operator on subset of regular grid" Tags="|optimization|algorithms|fortran|multigrid|tensor|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19571" PostTypeId="2" ParentId="19555" CreationDate="2015-05-06T03:06:05.110" Score="1" Body="&lt;p&gt;You should take a look at the Wikipedia page on rotation matrices, specifically the section on &lt;a href=&quot;http://en.wikipedia.org/wiki/Rotation_matrix#Rotation_matrix_from_axis_and_angle&quot; rel=&quot;nofollow noreferrer&quot;&gt;forming a rotation matrix from an axis and an angle&lt;/a&gt;. In your case, you have a vector $\vec{v}$ that you would like to align with another vector, $\vec{OP}$. The axis of rotation should be the unit vector $\hat{u}$ normal to these two vectors given by&#xA;$$\vec{n}=\vec{v}\times\vec{OP},&#xA;~~\hat{u}=\dfrac{\vec{n}}{||\vec{n}||}.&#xA;$$&#xA;The order of the cross product is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The angle between the two vectors is given by&#xA;$$\theta = \operatorname{acos}\left(\dfrac{\vec{v}\cdot\vec{OP}}{\left|\left|\vec{v}\right|\right|~\left|\left|\vec{OP}\right|\right|}\right)&#xA;$$&#xA;where $\theta\in[0,\pi)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have an angle and an axis of rotation you can form the rotation matrix $R$. This is given as&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/1RV0J.png&quot; alt=&quot;rotation matrix&quot;&gt;&lt;br&gt;&#xA;on the Wikipedia page. To rotate your square (and any other vectors or points of interest) so that $\vec{v}$ is aligned with $\vec{OP}$ simply multiply all the coordinates $\vec{OA}$, $\vec{OB}$, $\vec{OC}$, and $\vec{OD}$ by the rotation matrix $R$. As mentioned in a comment, you could then also rotated the resulting points by any angle you wish about the axis $\vec{v}_{\text{new}}=\vec{OP}$ while still maintaining the same orientation for $\vec{v}_{\text{new}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the only cases where this will fail are if $\vec{OP}=\pm\vec{v}$, in which case the cross product is zero. If $\vec{OP}=\vec{v}$ then there is nothing to do. If $\vec{OP}=-\vec{v}$ then $\theta = \pi$ you can choose $\vec{n} = (a,b,0)$ for any $a$ and $b$ you wish.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-05-06T03:33:47.750" LastActivityDate="2015-05-06T03:33:47.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19572" PostTypeId="2" ParentId="19555" CreationDate="2015-05-06T03:28:38.847" Score="0" Body="&lt;p&gt;The transformation that you are looking for is for a rotation about a fixed axis through a given angle.  This transformation can be described using using matrices and vectors.  A reference for the formula for the &lt;a href=&quot;http://en.wikipedia.org/wiki/Rotation_matrix#Rotation_matrix_from_axis_and_angle&quot; rel=&quot;nofollow&quot;&gt;matrix for rotation about an axis&lt;/a&gt; is given in the Wikipedia. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fixed axis is perpendicular to the vector from the origin to P and perpendicular to the vector from the origin to V.  Since you want to rotate from V up to P, the fixed vector is in the direction of the cross product $\vec{V} \times \vec{P}$.  Since $\vec{V}$ points straight down, the fixed vector will be in the xy-plane. In other words, the variable $u_z$ in the Wikipedia formula will be zero.  Note that the formula in the Wikipedia requires a &lt;em&gt;unit vector&lt;/em&gt;, so you will have to divide the cross product by the distance from the origin to P to obtain $u_x$ and $u_y$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The angle of the rotation can be found from the dot product $\vec{V} \cdot \vec{P}$.  The cosine on the angle will be the dot product divided by the distance from the origin to P.  The sine of the angle can be calculated from the cosine.  The cosine of the angle may be positive (P below the xy-plane) or negative (P above the xy-plane), but the sine of the rotation angle will always be positive.  That should give you all nine of the terms in the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the matrix elements are calculated, multiply the coordinates of each point on the square by the matrix to get the new coordinates.&lt;/p&gt;&#xA;" OwnerUserId="11950" LastActivityDate="2015-05-06T03:28:38.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19575" PostTypeId="1" CreationDate="2015-05-06T05:50:49.090" Score="1" ViewCount="270" Body="&lt;p&gt;I need to use line segments to approximate a function, f = 1/x. The range of x is from 1 to 2048 with an interval of 1. I will pick 10 locations for x and interpolate y between two adjacent x using line segment. The error between interpolated values and real values is expected to be minimum. I am using MATLAB for implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that picked points are in order. If the picked points are modified, the points between two picked points will be changed accordingly. How can I write an objective function for such case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have little experience of optimization using MATLAB. How do I write an objective function that need to be optimized?&lt;/p&gt;&#xA;" OwnerUserId="7846" LastEditorUserId="625" LastEditDate="2015-05-25T16:13:45.117" LastActivityDate="2015-12-21T19:54:19.500" Title="Use line segments to approximate a function" Tags="|optimization|matlab|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19576" PostTypeId="2" ParentId="19538" CreationDate="2015-05-06T05:58:03.817" Score="1" Body="&lt;p&gt;I would also like to recommend &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0262571323&quot; rel=&quot;nofollow&quot;&gt;Using MPI&lt;/a&gt;. I used this book back at school to implement some CFD codes on a cluster. It's a very good reference and discusses using MPI with C, C++, and Fortran. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book provides simple examples to help you get into the mindset of breaking your problem into one that can be distributed across different computers, in the event you're not too familiar with that sort of thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are ever interested in taking your skills further and going into heterogeneous programming (CUDA, OpenCL), you could also check out a &lt;a href=&quot;https://www.coursera.org/course/hetero&quot; rel=&quot;nofollow&quot;&gt;course on Coursera&lt;/a&gt; that goes over some pretty approaches and concepts.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-05-06T05:58:03.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19577" PostTypeId="2" ParentId="19506" CreationDate="2015-05-06T06:11:10.803" Score="9" Body="&lt;p&gt;Like WolfgangBangerth, I strongly recommend that you reconsider your motivation (or your supervisor's motivation) for this goal. First, look at &lt;a href=&quot;http://exactcodesign.org/&quot; rel=&quot;noreferrer&quot;&gt;Center for Exascale Simulation of Combustion in Turbulence (ExaCT) web site&lt;/a&gt;. They have some proxy applications that may be of use to you in testing your mechanisms, and publications by authors such as John Bell, Joe Oefelein, and Jacqueline Chen can give you an idea of what is the current state of the art in combustion simulation (not just DNS, but also LES, low Mach-number approximations, etc.). Most of these efforts are multi-year, decade-plus efforts. If all you want is something to test some algorithms for detailed chemical kinetics, you might want to either: (a) scale down your ambitions by looking at something simpler, e.g., something like laminar flames or shock tubes or (b) come up with a series of research papers you can write at various intermediate steps in your code development. Either option mitigates your risk. You don't want to spend multiple years developing code and get nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did my PhD on a related area of combustion research (well, reduced order modeling of detailed kinetic mechanisms). What you're proposing is probably on the order of 2-3 years, at least. For context: a group-mate of mine wrote a laminar finite-difference reacting flame code (I believe it was incompressible), and it took him at least 6 months. Another grad student I knew worked in Ahmed Ghoniem's group and took around a year to write a compressible flow code that was in parallel, but not highly parallelizable. He also wrote the code with a couple other people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your first 5 design criteria alone are going to require some serious work, because you're going to have to do something like the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;learn PETSc/Trilinos by hacking one or more examples&lt;/li&gt;&#xA;&lt;li&gt;setup your mesh (probably the easiest part)&lt;/li&gt;&#xA;&lt;li&gt;write out your right-hand side on an incompressible test problem&lt;/li&gt;&#xA;&lt;li&gt;probably write out some approximate Jacobian on a incompressible test problem&lt;/li&gt;&#xA;&lt;li&gt;test for correctness, maybe proof-of-concept scalability&lt;/li&gt;&#xA;&lt;li&gt;add compressibility to right-hand side&lt;/li&gt;&#xA;&lt;li&gt;add compressibility to approximate Jacobian&lt;/li&gt;&#xA;&lt;li&gt;test for correctness, maybe proof-of-concept scalability&lt;/li&gt;&#xA;&lt;li&gt;add reactions to right-hand side&lt;/li&gt;&#xA;&lt;li&gt;add reactions to approximate Jacobian&lt;/li&gt;&#xA;&lt;li&gt;test for correctness on simple reaction case (1-step Arrhenius, 4-step methane, maybe 10-step hydrogen)&lt;/li&gt;&#xA;&lt;li&gt;do a more serious test of scalability&lt;/li&gt;&#xA;&lt;li&gt;start tuning your solver for the laminar compressible case&lt;/li&gt;&#xA;&lt;li&gt;add simple turbulence model to right-hand side&lt;/li&gt;&#xA;&lt;li&gt;add simple turbulence model to approximate Jacobian&lt;/li&gt;&#xA;&lt;li&gt;test for correctness&lt;/li&gt;&#xA;&lt;li&gt;test for scalability&lt;/li&gt;&#xA;&lt;li&gt;add more complicated turbulence model + more testing&lt;/li&gt;&#xA;&lt;li&gt;add more complicated reaction models + even more testing&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of the above description assumes you know exactly what you're doing with the discretizations in space and time, and you know exactly what you're going to use for approximate Jacobians (assuming you need them). I haven't included the inevitable debugging that is going to arise, nor managing the complexity of such a code. If you choose to embark on writing such a code, I urge you to reconsider your remark on object-oriented design. You, as a developer, are going to want to use some sort of organizing principle to manage a large codebase, even if you don't release your software outside of your group. I also haven't included deriving any terms you may not have (e.g., the Jacobian of the reaction terms), coping with performance issues, getting time on clusters, managing the output of your simulations, etc. All of these comments are to say: yes, WolfgangBangerth is right, this endeavor is a multi-year effort.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-05-06T06:11:10.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19578" PostTypeId="2" ParentId="19500" CreationDate="2015-05-06T06:26:03.280" Score="0" Body="&lt;p&gt;You could consider trying to solve this problem using some more global optimization approach, such as a particle swarm optimization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that you set this up sufficiently well, this approach may be able to traverse the solution field better than a gradient based approach since you may be able to avoid local minima better.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-05-06T06:26:03.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19580" PostTypeId="2" ParentId="19500" CreationDate="2015-05-06T11:43:49.637" Score="0" Body="&lt;p&gt;A possible way of solving this is with Bayesian inference. Since you have already a model, you need to define the priors to $R_1$ and $R_2$ as well as the distribution for the error in $P_i$.&#xA;For this approach, fhe following steps have to be performed&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Set up a full probability model defining the joint probability distribution for all observable ($P_i$) as well as unobservable quantities ($R_1, R_2$).&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Define the conditioning on the observed data, calculate the posterior distribution given the observed data. $p(R_1,R_2|P_i)\propto p(P_i|R_1,R_2)p(R_1,R_2)$. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Estimate the unobserved quantities $(R_1,R_2)$ of interest given the observed data $(P_i)$ from the posterior distribution. Estimate their confidence intervals.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2015-05-06T11:43:49.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19581" PostTypeId="2" ParentId="19564" CreationDate="2015-05-06T16:24:56.847" Score="3" Body="&lt;p&gt;If your discretization of the reaction part of the problem is explicit, you can still use Prof. Bangerth's suggestion. You can quickly compute $K*u$ using the fast Fourier transform on both $K$ and $u$, multiplying the transforms and then transforming back. Then you can easily compute $u\cdot K*u$ at any grid point. However, you may run into stability issues with an explicit time discretization and a reasonably large time-step; reaction-diffusion equations are often stiff. We can probably give you more suggestions if you write out what the kernel is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the kernel is rapidly decreasing far from the origin, you could also try to approximate the convolution $K * u$ using something like the &lt;a href=&quot;http://math.nyu.edu/faculty/greengar/shortcourse_fmm.pdf&quot; rel=&quot;nofollow&quot;&gt;fast multipole method&lt;/a&gt;. This idea was originally used to turn the $O(N^2)$ computation of gravitational $N$-body potentials into an $O(N\log N)$ computation of an approximate potential, but it's been extended to more general problems. FMM often shows up on lists of the top 10 greatest algorithms, so it's also worth knowing about on principle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fast multipole method and more general &lt;a href=&quot;http://en.wikipedia.org/wiki/Hierarchical_matrix&quot; rel=&quot;nofollow&quot;&gt;hierarchical matrix&lt;/a&gt; methods are often used for the numerical solution of integral equations.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-05-06T16:24:56.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19585" PostTypeId="1" CreationDate="2015-05-06T22:46:32.730" Score="0" ViewCount="141" Body="&lt;p&gt;I am interested in numerically solving the following system of coupled ODEs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(i-\frac{1}{2\Omega}f_{m,n}\right) \frac{d a_{m,n}(t)}{dt} =E_{m,n}^{\text{kin}}(t) + \left(\omega_{0}+V_{m,n}-\frac{i}{2}\gamma+\frac{i}{2}f_{m,n}\right)a_{m,n}(t)&#xA;+\left(g-\frac{i}{2}r\right)|a_{m,n}(t)|^2 a_{m,n}(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $m$ and $n$ are integers, and represent indices on a 2D lattice, as in fact we are dealing with a system of coupled ODEs, one for each lattice point. The lattice is NxN, with N or the order of 30.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The unknown is the complex function $a_{m,n}(t)$. $f_{m,n}$ is also complex, while all the other parameters are real.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kinetic and potential energy terms are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  E_{m,n}^{\text{kin}}(t) = -J [ e^{-i\phi_{m,n}^x}a_{m+1,n}(t)+e^{i\phi_{m,n}^x}a_{m-1,n}(t)&#xA;+e^{-i\phi_{m,n}^y}a_{m,n+1}(t)+e^{i\phi_{m,n}^y}a_{m,n-1}(t) ]$$&#xA;and&#xA;$$V_{m,n} = \frac{1}{2} \kappa b^2 \left[ (m-m_0)^2 + (n-n_0)^2 \right]$$&#xA;while&#xA;$(\phi_{m,n}^{x},\,\phi_{m,n}^{y})=(-\pi\alpha nb,\,\pi\alpha mb)$, $\alpha = \frac{p}{q}$ and $p,q \in \mathbb{Z}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone suggest a suitable numerical scheme for tackling this problem?&lt;/p&gt;&#xA;" OwnerUserId="4656" LastActivityDate="2015-05-06T22:46:32.730" Title="system of coupled nonlinear ODEs with complex coefficients" Tags="|ode|nonlinear-equations|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19586" PostTypeId="1" CreationDate="2015-05-07T00:21:50.350" Score="47" ViewCount="124512" Body="&lt;p&gt;Are there any tools in Python that are like Matlab's parfor? I found &lt;a href=&quot;https://stackoverflow.com/questions/4682429/parfor-for-python&quot;&gt;this thread&lt;/a&gt;, but it's four years old. I thought maybe someone here might have more recent experience. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example of the type of thing I'd like to parallelize:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = np.random.normal(size=(10, 3))&#xA;F = np.zeros((10, ))&#xA;for i in range(10):&#xA;    F[i] = my_function(X[i,:])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;my_function&lt;/code&gt; takes an &lt;code&gt;ndarray&lt;/code&gt; of size &lt;code&gt;(1,3)&lt;/code&gt; and returns a scalar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the least, I'd like to use multiple cores simultaneously---like parfor. In other words, assume a shared memory system with 8-to-16 cores. &lt;/p&gt;&#xA;" OwnerUserId="3652" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2019-09-03T10:34:06.227" Title="Parallelizing a for-loop in Python" Tags="|python|parallel-computing|" AnswerCount="7" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19587" PostTypeId="2" ParentId="19586" CreationDate="2015-05-07T01:26:19.140" Score="6" Body="&lt;p&gt;My impression of parfor is that MATLAB is encapsulating implementation details, so it could be using both shared memory parallelism (which is what you want) and distributed memory parallelism (if you're running a &lt;a href=&quot;http://www.mathworks.com/products/distriben/&quot; rel=&quot;noreferrer&quot;&gt;MATLAB distributed computing server&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want shared memory parallelism, and you're executing some sort of task parallel loop, the &lt;a href=&quot;https://docs.python.org/2/library/multiprocessing.html&quot; rel=&quot;noreferrer&quot;&gt;multiprocessing standard library package&lt;/a&gt; is probably what you want, maybe with a nice front-end, like &lt;a href=&quot;https://pypi.python.org/pypi/joblib&quot; rel=&quot;noreferrer&quot;&gt;joblib&lt;/a&gt;, as mentioned in Doug's post. The standard library isn't going to go away, and it's maintained, so it's low-risk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other options out there, too, like &lt;a href=&quot;http://www.parallelpython.com/&quot; rel=&quot;noreferrer&quot;&gt;Parallel Python&lt;/a&gt; and &lt;a href=&quot;http://ipython.org/ipython-doc/dev/parallel/index.html&quot; rel=&quot;noreferrer&quot;&gt;IPython's parallel capabilities&lt;/a&gt;. A quick glimpse at Parallel Python makes me think that it's closer to the spirit of parfor, in that the library encapsulates details for the distributed case, but the cost of doing so is that you have to adopt their ecosystem. The cost of using IPython is similar; you have to adopt the IPython way of doing things, which may or may not be worth it to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you care about distributed memory, I recommend &lt;a href=&quot;https://bitbucket.org/mpi4py/mpi4py&quot; rel=&quot;noreferrer&quot;&gt;mpi4py&lt;/a&gt;. Lisandro Dalcin does great work, and mpi4py is used in the PETSc Python wrappers, so I don't think it's going away anytime soon. Like multiprocessing, it's a low(er)-level interface to parallelism than parfor, but one that is likely to last for a while.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-05-07T01:26:19.140" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19588" PostTypeId="1" CreationDate="2015-05-06T17:48:29.380" Score="2" ViewCount="243" Body="&lt;p&gt;I'm trying to model adiabatic passage between two levels in a three-level atom interacting with two laser fields using Scipy and Numpy.&lt;img src=&quot;https://i.stack.imgur.com/tIumF.jpg&quot; alt=&quot;three-level atom&quot;&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if my model is wrong due to my incorrectly formulating the equations in python or the Scipy ODE solver picking up errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the coefficients of each of the three states I use&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;i\frac{d}{dt}\left( \begin{array}{c}&#xA;c_1 \\&#xA;c_2 \\&#xA;c_3  \end{array} \right) = \left( \begin{array}{ccc}&#xA;0 &amp;amp; \Omega_1(t) &amp;amp; 0 \\&#xA;\Omega_1(t) &amp;amp; 0 &amp;amp; \Omega_2(t) \\&#xA;0 &amp;amp; \Omega_2(t) &amp;amp; 0 \end{array} \right)\left( \begin{array}{c}&#xA;c_1 \\&#xA;c_2 \\&#xA;c_3  \end{array}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\Omega_{1,2}(t)$ give the (slow) time dependence of the amplitudes of the electric fields generated by the two laser beams. The initial conditions are $c_1(-\infty) = 1, c_2(-\infty) = c_3(-\infty) = 0 $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the Gaussian forms of $\Omega_{1,2}(t)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Omega_{1}(t) = exp(-(t-\tau)^2/T^2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Omega_{2}(t) = exp(-(t+\tau)^2/T^2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\tau$ is the delay time such that a positive value of $\tau$ means that field 2 is switched on before field 1. $T$ sets the duration of the pulses.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy as sp&#xA;from scipy.integrate import odeint&#xA;import matplotlib.pyplot as plt&#xA;from math import exp&#xA;from collections import defaultdict&#xA;import pylab as p&#xA;&#xA;tau = 2&#xA;T = 1&#xA;&#xA;def omega1(t,tau=tau,T=T):&#xA;    return exp(-(t - tau)**2/T**2)&#xA;&#xA;def omega2(t,tau=tau,T=T):&#xA;    return exp(-(t + tau)**2/T**2)&#xA;&#xA;def g(y, t,tau=tau,T=T):&#xA;    c1_i = y[0]&#xA;    c2_i = y[1]&#xA;    c3_i = y[2]&#xA;&#xA;    g1 = omega1(t, tau, T)*c2_i&#xA;    g2 = omega1(t, tau, T)*c1_i + omega2(t)*c3_i&#xA;    g3 = omega2(t, tau, T)*c2_i&#xA;    return [g1, g2, g3]&#xA;&#xA;# initial conditions&#xA;c1_0 = 1&#xA;c2_0 = 0&#xA;c3_0 = 0&#xA;&#xA;y0 = [c1_0, c2_0, c3_0]&#xA;&#xA;step_size = 1000&#xA;t_max = 10.&#xA;t_min = 0.&#xA;delta_t = (t_max - t_min)/step_size&#xA;&#xA;t = np.linspace(t_max, t_min, step_size) # time grid&#xA;soln = odeint(g, y0, t)&#xA;c1 = soln[:, 0]&#xA;c2 = soln[:, 1]&#xA;c3 = soln[:, 2] &#xA;&#xA;# plot results&#xA;plt.figure()&#xA;plt.plot(t, np.abs(c1), label='c1')&#xA;plt.plot(t, np.abs(c2), label='c2')&#xA;plt.plot(t, np.abs(c3), label='c3')&#xA;&#xA;plt.xlabel('time')&#xA;plt.ylabel('energy level population')&#xA;plt.title('initial attempt')&#xA;plt.legend(loc=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/e3zgk.png&quot; alt=&quot;initial result&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result seems wrong since it shows the coefficients having values above one. Furthermore, I can't seem to find values of $\tau$ and $T$ that yield a non-zero final $c_3$ value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One issue that might be contributing is that currently my answer doesn't include the multiplication by $i$. I'm not sure how to include that in my python equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help!&lt;/p&gt;&#xA;" OwnerUserId="16296" OwnerDisplayName="Thunder Shiviah" LastActivityDate="2015-05-07T02:16:22.343" Title="Modeling simple laser induced population transfer via adiabatic passage in python" Tags="|quantum-mechanics|computational-physics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19589" PostTypeId="1" AcceptedAnswerId="19592" CreationDate="2015-05-07T02:48:51.983" Score="2" ViewCount="127" Body="&lt;p&gt;I am running some quantum chemical calculations on a refurbished PowerEdge rack server, with dual quad-core Xeons, 16GB RAM, and a single 500GB SATA HDD.  OS is vanilla Debian Wheezy.  For one of the calculation types I'm running, I think I'm becoming disk-I/O-limited and so I installed a spare 160GB HDD I had on hand to try to ease the bottleneck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make use of the additional drive, though, I need to be able to tell OpenMPI to use multiple working directories.  I know the &lt;code&gt;--wdir&lt;/code&gt; commandline flag exists, but the documentation makes it look like it only accepts a single value.  I unsuccessfully tried a hacky workaround:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Creating a new user &lt;code&gt;scratch&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Mounting the new 160GB drive as &lt;code&gt;/home/scratch/&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Setting up &lt;code&gt;/home/bskinn/.ssh/config&lt;/code&gt; to default to user &lt;code&gt;bskinn&lt;/code&gt; when SSH-ing to &lt;code&gt;localhost&lt;/code&gt; and to default to user &lt;code&gt;scratch&lt;/code&gt; when SSh-ing to the server's internal &lt;code&gt;192.168.x.x&lt;/code&gt; address&lt;/li&gt;&#xA;&lt;li&gt;Creating appropriate RSA SSH keys, adding to &lt;code&gt;/home/bskinn/.ssh/authorized_keys&lt;/code&gt; and &lt;code&gt;/home/scratch/.ssh/authorized_keys&lt;/code&gt;, and SSH-ing into each to get the host information stored for auto-acceptance&lt;/li&gt;&#xA;&lt;li&gt;Adding &lt;code&gt;localhost&lt;/code&gt; and &lt;code&gt;192.168.x.x&lt;/code&gt; to my &lt;code&gt;ompi_hosts&lt;/code&gt; file and passing to OMPI with &lt;code&gt;--hostfile /home/bskinn/ompi_hosts&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If I &lt;code&gt;ssh localhost&lt;/code&gt; I get logged in as &lt;code&gt;bskinn&lt;/code&gt;, and if I &lt;code&gt;ssh 192.168.x.x&lt;/code&gt; I get logged in as &lt;code&gt;scratch&lt;/code&gt;, just like I want.  However, while OpenMPI doesn't throw any errors, it appears to just ignore the &lt;code&gt;ompi_hosts&lt;/code&gt; file and just runs all processes locally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So: Is there a way to get OpenMPI 1.6.5 to use multiple working directories?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ADDENDUM:&lt;/strong&gt; I'm running ORCA computations. &lt;a href=&quot;https://orcaforum.cec.mpg.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ORCA&lt;/a&gt; is distributed as pre-compiled, statically-linked binaries.  In particular, the OMPI calls are hard-coded into said binaries and not accessible to me.  So, I can't intercept what commands are passed to OMPI, except perhaps by some sleight-of-hand wherein I relocate the ORCA binaries and replace them with shell scripts. That seems like a lot more work than it's worth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ORCA does provide the ability to pass through whatever OMPI environment variables I want; I'm only really interested in solutions on this level of complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This is a cross-post from &lt;a href=&quot;https://superuser.com/questions/856914/is-it-possible-to-configure-linux-openmpi-v1-6-5-to-use-multiple-working-directo&quot;&gt;SuperUser&lt;/a&gt; -- I won the Tumbleweed badge for it there (yay?), so I'm guessing it's not a good place for it. I don't have rep to migrate it, so apologies if re-posting in this manner is frowned upon.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="13107" LastEditorUserId="-1" LastEditDate="2017-03-20T10:18:34.427" LastActivityDate="2015-06-01T02:41:20.573" Title="Is it possible to configure Linux OpenMPI v1.6.5 to use multiple working directories on a single node?" Tags="|mpi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19590" PostTypeId="2" ParentId="19589" CreationDate="2015-05-07T03:05:40.750" Score="3" Body="&lt;p&gt;Instead of launching the executable, why don't you launch a script that changes directories into the one you want and then execs your program? OpenMPI sets &lt;code&gt;$OMPI_COMM_WORLD_RANK&lt;/code&gt; in the environment equal to the rank that the task will get within MPI_COMM_WORLD inside the code, so you can do something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;foo.sh:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;total_tasks=100 # Or whatever&#xA;if [ $OMPI_COMM_WORLD -lt $($total_tasks/2) ]; then&#xA;  cd /home/scratch&#xA;fi&#xA;exec /path/to/my/chemistry/code arg1 arg2 arg3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then if you make foo.sh executable (chmod +x foo.sh) and mpirun or mpiexec (or whatever) foo.sh, it should put the first half of the ranks with their working directory in /home/scratch and the latter half of the ranks wherever you were when you did the mpirun/exec.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also just patch this into your chemistry program if you have the source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if these two drives are connected to the same controller, you may or may not get any additional bandwidth. Have you tried profiling your app? What code is it? Are you running it in an out-of-core mode?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-07T03:05:40.750" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19591" PostTypeId="1" CreationDate="2015-05-07T13:35:40.817" Score="5" ViewCount="1045" Body="&lt;p&gt;When using the SIMPLE method on a mesh with a collocated variable arrangement, the following interpolation is used for the advecting velocities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_f = \overline{u}_f - \overline{D}_f\left(\left(\nabla P\right)_f - \overline{\left(\nabla P\right)}_f\right)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the over bar denotes a geometrically interpolated quantity and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;D_P = \frac{V_P}{a_P}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $V_P$ is the volume of cell $P$ and $a_P$ is the central coefficient arising from the discretized momentum equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;a_Pu_P + \sum_{F}a_Fu_F = -V_P\nabla P&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways, my question is, how do you correctly determine $\overline{D}_f$ at boundaries? I recently wrote a SIMPLE solver for unstructured meshes, but have noticed that the largest continuity errors are always occurring at the boundary cells, leading me to think I may have accounted for this term incorrectly (I simply extrapolate it to the boundary face). This term can be very important in the computations as it also shows up in the pressure correction equation. I also have some difficulty getting the solver to converge when using inlet/outlet boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="16303" LastActivityDate="2018-05-24T13:45:19.240" Title="A Question About the Rhie-Chow Interpolation Used for Solving the Incompressible Navier-Stokes Equations on Unstructured Grids" Tags="|fluid-dynamics|computational-physics|numerics|numerical-modelling|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19592" PostTypeId="2" ParentId="19589" CreationDate="2015-05-07T14:59:20.713" Score="1" Body="&lt;p&gt;If you don't have the source and you can't modify the mpirun/mpiexec command line at all, you're probably going to have a hard time doing things the way you tried. You should profile your runs if you can and verify that I/O is, in fact, your problem. If it is, just moving to a single SSD might be enough to make you some progress, but splitting across drives could be more challenging unless you have enough room (controller ports and physical space) to build a 2-3 drive striped RAID array (RAID 0). You are almost certainly better off this way than trying to explicitly stripe across different mount points. If you don't have the space for 3-4 total drives, but you do have room for two, you'll probably need to completely wipe and reinstall the system with a RAID 0 system partition to put everything on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*Given the update and the request, I have added this extra answer.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-07T14:59:20.713" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19593" PostTypeId="2" ParentId="19546" CreationDate="2015-05-07T16:18:01.483" Score="1" Body="&lt;p&gt;As they mentioned (and scolded you), why are you trying to use that method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a non-linear regression is enough you can see the code below, the option &lt;code&gt;curve_fit&lt;/code&gt; did not work. So I used &lt;code&gt;minimize&lt;/code&gt; in this case, the initial guess  is obtained from a linear regression of $y$ and $(1/x)$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.optimize import minimize&#xA;&#xA;def func(coef, x, y):&#xA;    a = coef[0]&#xA;    b = coef[1]&#xA;    n = coef[2]&#xA;    return np.linalg.norm(y - a/x**n - b)&#xA;&#xA;x = np.array([3.53, 4.93, 5.53, 6.21, 7.37, 9.98, 16.56])&#xA;y = np.array([0, 50, 60, 70, 80, 90, 100])&#xA;&#xA;&#xA;# The initial values are obtained from a linear regression between&#xA;# y and (1/x)&#xA;res = minimize(func, [-446, 136, 1], args=(x, y))&#xA;a, b, n = res.x&#xA;print a, b, n&#xA;x_vec = np.linspace(3.5, 17)&#xA;y_vec = a/x_vec**n + b&#xA;&#xA;plt.plot(x, y, 'ko')&#xA;plt.plot(x_vec, y_vec)&#xA;plt.xlabel(&quot;x&quot;)&#xA;plt.xlabel(&quot;y&quot;)&#xA;plt.savefig(&quot;regression.png&quot;, dpi=300)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-1181.72495119 105.101516419 1.91866624841&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the image&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8QPQ5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-05-07T16:58:43.667" LastActivityDate="2015-05-07T16:58:43.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19595" PostTypeId="1" AcceptedAnswerId="19597" CreationDate="2015-05-07T19:30:42.367" Score="2" ViewCount="586" Body="&lt;p&gt;In this &lt;a href=&quot;http://www.icml2010.org/papers/458.pdf&quot; rel=&quot;nofollow&quot;&gt;paper on Deep Learning for Machine Learning&lt;/a&gt;, the approach is referred to as Hessian-free method. That is because the Hessian is never computed explicitly. Instead, the product of the Hessian and a vector is obtained using finite difference approximation. This is used by the Conjugate Gradient step to compute the Newton direction. The number of iterations that Conjugate Gradient runs is truncated, as soon as a &quot;good enough&quot; direction is obtained. The paper states this truncation as an aspect of the Hessian-free method. However, isn't this truncated Newton method? It looks like this paper is combining Hessian-free with truncated Newton method.&lt;/p&gt;&#xA;" OwnerUserId="14478" LastActivityDate="2015-05-07T22:56:03.727" Title="Hessian-free and Truncated Newton methods" Tags="|linear-algebra|optimization|numerical-analysis|newton-method|conjugate-gradient|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19596" PostTypeId="1" AcceptedAnswerId="19600" CreationDate="2015-05-07T19:33:17.480" Score="4" ViewCount="102" Body="&lt;p&gt;Is there a standard approach to testing codes with refined regions? Specifically, I am interested in testing whether the refinement is working correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of simplicity, let's consider a (time-independent) boundary value problem in a domain with only one refined area, and a case with a known exact solution. Let's consider a finite difference method with a uniform grid size. Assume I've already verified that the code obtains the correct convergence rate for a uniform grid with the exact solution I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider a uniform grid of size $h$ in the fine region and $H$ in the coarse region. I can compare the error (however it might be defined) between that case and the case where the grid is uniform with resolution $h$. Hopefully, the accuracy degradation is not &quot;too bad&quot;. I don't know precisely what &quot;too bad&quot; in this case might mean, however, and hence my question. There may be a better approach altogether than what I am thinking here.&lt;/p&gt;&#xA;" OwnerUserId="437" LastEditorUserId="123" LastEditDate="2015-05-08T06:18:07.550" LastActivityDate="2015-05-08T06:18:07.550" Title="How can I compare errors in PDE solvers with non-uniform grids?" Tags="|pde|testing|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19597" PostTypeId="2" ParentId="19595" CreationDate="2015-05-07T22:56:03.727" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;It looks like this paper is combining Hessian-free with truncated Newton method.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, it is.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...the approach is referred to as Hessian-free method. That is because the Hessian is never computed explicitly. Instead, the product of the Hessian and a vector is obtained using finite difference approximation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Right. This step is analogous to the &quot;Jacobian-free&quot; half of Jacobian-free Newton-Krylov methods.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is used by the Conjugate Gradient step to compute the Newton direction. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Correct.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The number of iterations that Conjugate Gradient runs is truncated, as soon as a &quot;good enough&quot; direction is obtained. The paper states this truncation as an aspect of the Hessian-free method. However, isn't this truncated Newton method?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. You're right that the discussion in Section 3 of the paper is confusing. The author is combining two things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;an &quot;inexact&quot; or &quot;truncated&quot; (quasi-)Newton method will solve the linear equations within a (quasi-)Newton method inexactly, using an iterative solver. In this paper, the author is using the conjugate gradient (CG) method. Such a method requires Hessian-vector products (or, for solving nonlinear equations, Jacobian matrix-vector products). One could calculate this matrix-vector product exactly, or approximately.&lt;/li&gt;&#xA;&lt;li&gt;a &quot;Hessian-free&quot; method in optimization (respectively, a &quot;Jacobian-free&quot; method for solving nonlinear equations) approximates the Hessian-vector product (respectively, the Jacobian matrix-vector product) using finite difference evaluations.&lt;/li&gt;&#xA;&lt;li&gt;So, typically, one uses a Hessian-free method within a truncated Newton method, but you could just as easily use a truncated Newton method with exact Hessian-vector products. This paper chooses to combine the Hessian-free method with a truncated Newton method, which is a reasonable choice.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-05-07T22:56:03.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19598" PostTypeId="1" AcceptedAnswerId="19599" CreationDate="2015-05-08T00:15:10.667" Score="0" ViewCount="174" Body="&lt;p&gt;I am simulating a Mixed-Boundary value (Dirichlet-Neumann) problem using Finite Difference Method on a unit 3-D cube such that the left, lower, and front plane have $u=u(x,y,z)=1$ (Dirichlet) and right, upper and back plane have $\frac{\partial u}{\partial n}=0$. I know that the true solution is $1$ everywhere and hence I have used the stopping condition as $err_\max &amp;lt; 0.00001$ where I find error like $err = 1.0 - u_\text{current}$. The problem converges when input is $16\times16\times16$ and $32\times32\times32$ but when input is $64\times64\times64$, the maximum error does not decrease below $0.000019$. Similarly when the problem size is $128\times128\times128$, the error does not decrease below $0.000040$. My question is : Is this an expected behaviour ?   &lt;/p&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="9667" LastEditDate="2015-05-08T09:35:44.393" LastActivityDate="2015-05-08T09:35:44.393" Title="Simulation of Laplace Equation in 3-D with mixed BC of Dirichlet-Neumann" Tags="|boundary-conditions|poisson|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19599" PostTypeId="2" ParentId="19598" CreationDate="2015-05-08T01:45:05.013" Score="1" Body="&lt;p&gt;No, it is not expected. Your iteration needs to converge. Since you don't say what your iteration is, it is difficult to say what me be wrong. But it is easy to say that you must have a bug somewhere: as long as your iteration is any reasonable method, it needs to converge!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-05-08T01:45:05.013" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19600" PostTypeId="2" ParentId="19596" CreationDate="2015-05-08T01:47:28.853" Score="3" Body="&lt;p&gt;It is difficult to assess the correctness of a code for only a single value of the mesh sizes $h$, $H$. Rather, one typically evaluates the accuracy for a &lt;em&gt;sequence&lt;/em&gt; of mesh sizes $h,H \rightarrow 0$ by comparing against the exact solution to which your numerical approximations have to converge.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-05-08T01:47:28.853" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19601" PostTypeId="1" CreationDate="2015-05-08T09:35:41.917" Score="1" ViewCount="104" Body="&lt;p&gt;I want to use one of these two libraries into my C++ project to basically invert a dense matrix (with Cholesky). Of course, I am interested in a distributed environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both libraries seem nice so choosing between them is hard. Does anybody have experience on them? Can anybody provide a comparison for them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Better support when you have a problem from Elemental, rather than&#xA;DPLASMA.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Relevant question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/351/recommendations-for-a-usable-fast-c-matrix-library&quot;&gt;Recommendations for a usable, fast C++ matrix library?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16311" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-05-08T09:35:41.917" Title="Elemental vs DPLASMA" Tags="|linear-algebra|c++|c|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19602" PostTypeId="1" CreationDate="2015-05-08T12:23:01.580" Score="1" ViewCount="110" Body="&lt;p&gt;I'd like to numerically integrate $\frac{1}{\sqrt{f(x)}}$ on an interval between two consecutive zeros of the function $f(x)$, which makes the integrand singular at two endpoints. Standard practice seems to be to split into intervals so that each has one singularity at an endpoint. However, I don't know the locations of these endpoints explicitly; at best, I can solve for them numerically to arbitrary precision. This presents a dilemma because one needs to take into account the size of the singularity (i.e. the size of $f'(x)$ at the endpoints) to know how precisely to specify the endpoint. What's the best way to handle such a situation? Can standard software carry out the whole procedure to arbitrary precision?&lt;/p&gt;&#xA;" OwnerUserId="11628" LastActivityDate="2015-10-07T09:53:51.070" Title="Numerical quadrature when locations of singularities are approximate" Tags="|quadrature|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19603" PostTypeId="2" ParentId="16325" CreationDate="2015-05-08T17:01:09.880" Score="2" Body="&lt;p&gt;A simpler answer, IMHO, is to solve your system using &lt;a href=&quot;http://pydstool.sf.net&quot; rel=&quot;nofollow&quot;&gt;PyDSTool&lt;/a&gt;. It has built-in support for &lt;a href=&quot;http://www.ni.gsu.edu/~rclewley/PyDSTool/Events.html&quot; rel=&quot;nofollow&quot;&gt;accurate event finding&lt;/a&gt;. In fact, it's easier to set up and more efficient (including faster) than Matlab's approach if you use the C-based solvers. I am the author of this package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A tutorial using events is given &lt;a href=&quot;http://www.ni.gsu.edu/~rclewley/PyDSTool/Tutorial/Tutorial_VdP.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1693" LastActivityDate="2015-05-08T17:01:09.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19605" PostTypeId="1" AcceptedAnswerId="19619" CreationDate="2015-05-08T20:24:46.213" Score="4" ViewCount="103" Body="&lt;p&gt;i have an array of boolean variables. The array has a sort of structure, it's formed of many sequences of zeros and ones, sequences of different length. such as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1111111100001111111111000000000000001111111000000111111111111111110000]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;commas omitted between values&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i need to separate it in this way &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[111111110000],[111111111100000000000000],[1111111000000],[111111111111111110000]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone knows a good way or just a way to do this ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithm could be language free, but in case you know how to do it in python you are welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some struggling i found a way to divide the array, but only if it begins with ONES. I realy can't figure out how to make it work with array beginning with 1 or 0.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;x = np.asarray([1,1,1,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,0,0,0,0,0,0,1,1,1,1,0,0,0])&#xA;&#xA;c = 1&#xA;&#xA;for i in range(len(x)):&#xA;    if x[i] == 1:&#xA;        if c == 0:&#xA;            print &quot;\n\tDIVIDE\n&quot;&#xA;            c = 1&#xA;        print &quot;one&quot;&#xA;&#xA;    else:&#xA;        if c == 1:&#xA;            c = 0&#xA;        print &quot;zero&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15187" LastEditorUserId="15187" LastEditDate="2015-05-08T22:10:07.093" LastActivityDate="2015-05-10T10:59:48.187" Title="divide an array using a changing length sequence of boolean var as index" Tags="|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19607" PostTypeId="1" AcceptedAnswerId="19608" CreationDate="2015-05-09T05:28:00.643" Score="6" ViewCount="1274" Body="&lt;p&gt;I would like to solve for the optimum $A$ values for a series of matrix equations $Ax_{1} = b_{1}, Ax_{2} = b_{2} ... Ax_{n} = b_{n}$ where only the $x$ values are known and when I start with an intelligent guess for $A$. I am using QR decomposition to solve the individual matrix equations for $b$, but wondering how to go about optimizing $A$ values as I don't have a real $b$ value to use in defining some error metric to use for convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nature of the problem is that $x$ is a set of sensor values and $A$ represents a set of constant coefficients defining to what degree the value at an adjacent sensor affects the measured values of $x$. The $b$ values should be the actual values without superposition from other sensors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on tackling this would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="16321" LastEditorUserId="16321" LastEditDate="2017-04-04T05:02:49.697" LastActivityDate="2017-04-04T05:02:49.697" Title="Algorithm for optimizing Ax = b with unknown A and known x values" Tags="|linear-algebra|inverse-problem|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="19608" PostTypeId="2" ParentId="19607" CreationDate="2015-05-09T10:21:05.170" Score="9" Body="&lt;p&gt;Your problem sounds like Independent Component Analysis. Where $x_i$ are the measurements in which the source signals have got mixed and $b_i$ are the values emitted by the sources. The $A$ in your equations is called the unmixing matrix. There is an iterative procedure to estimate $A$ and hence, the $b_i$s, based on the maximum likelihood principle. Refer to these notes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cs229.stanford.edu/notes/cs229-notes11.pdf&quot;&gt;Andrew Ng's notes on ICA&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit] FastICA looks like a popular implementation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://research.ics.aalto.fi/ica/fastica/&quot;&gt;http://research.ics.aalto.fi/ica/fastica/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://itpp.sourceforge.net/4.3.0/fastica_8cpp.html&quot;&gt;http://itpp.sourceforge.net/4.3.0/fastica_8cpp.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://tumic.wz.cz/fel/online/libICA/&quot;&gt;http://tumic.wz.cz/fel/online/libICA/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit] The OP (Jason) also found these for future reference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/cgearhart/students-filters/&quot;&gt;https://github.com/cgearhart/students-filters/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://sourceforge.net/projects/fastica/&quot;&gt;http://sourceforge.net/projects/fastica/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://shulgadim.blogspot.ca/2014/02/independent-component-analysis-ica.html&quot;&gt;http://shulgadim.blogspot.ca/2014/02/independent-component-analysis-ica.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2015-05-09T16:19:41.803" LastActivityDate="2015-05-09T16:19:41.803" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19609" PostTypeId="1" AcceptedAnswerId="19610" CreationDate="2015-05-09T10:48:05.673" Score="7" ViewCount="358" Body="&lt;p&gt;I currently use FEniCS and Deal.II to solve various FEM problems. I am also writing my own implementation of these problems by directly implementing the data structures, routines, and solvers within PETSc. What kind of comparisons can I draw between these three various implementations given the same FEM problem/discretization? Particularly in the context of high performance and parallel computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naturally, the first thing to do is check the numerical accuracy - do I get the correct solutions. Then I could check the strong scaling, weak scaling, and associated wall-clock time across multiple processing cores. But what else can I do? For instance, is there a way to determine how &quot;efficient&quot; these frameworks are algorithmically and in terms of machine hardware?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to show people which of these implementations will yield the fastest possible time given a specific problem size and FE discretization, but I think it would also be interesting to show how &quot;credible&quot; these timings are. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Why does implementation X take 45 seconds to solve a problem but implementations Y and Z only take 30 and 25 seconds respectively? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Why can't any of these three take only 4.5 seconds? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) How much of the time within the computational framework is spent doing useful work versus waiting on/accessing memory/cache? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure memory bandwidth is likely the answer to why these implementations don't achieve the ideal or perfect time, and benchmarks like STREAM can tell you what you can expect given your machine. I am wondering if anyone has a methodology, tool, or suggestion to properly determine or quantify these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="14576" LastEditorUserId="14576" LastEditDate="2015-05-09T10:57:50.037" LastActivityDate="2016-11-21T01:50:25.863" Title="Comparing various implementations/software packages for large-scale finite element simulations" Tags="|parallel-computing|petsc|fenics|efficiency|deal.ii|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19610" PostTypeId="2" ParentId="19609" CreationDate="2015-05-09T14:58:47.093" Score="4" Body="&lt;p&gt;Unfortunately there's no tool for this. You can run each on a variety of input sizes to establish the computational complexity they appear to have, i.e. the $f$ in the $O(f(n))$ that characterizes each code. This can point you towards what underlying algorithms each is using and verify or not that something that should be $O(n)$ is actually implemented to achieve this. You can run fixed $n$ and vary the number of processors $p$ in order to try to determine the efficiency of their parallel implementations. And then you run a fixed problem on a variety of CPU frequencies and memory frequencies to try to determine how sensitive they are to hardware characteristics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These studies can help you lay out a handful of mostly orthogonal axes in the design space. This space is actually much bigger and quite vast, and it doesn't even begin to take in the various input parameters to the solvers (max iterations, stopping criteria, algorithm selection, etc). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As best I can tell, basically no one does this. You usually figure out pretty quickly whether you have accidentally implemented something $O(n^2)$ when you meant to be $O(n)$. With practice, you can look at the code to most core algorithms and be sure you got the complexity right, so in the end, when comparing two implementations that are supposed to have the same computational complexity, you're trying to compute whether they have achieved through implementation the multiplicative and additive constants the theory predicts. This can be very tedious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you've done all of this, then you have to spend the time comparing the implementation, the compilation to machine code, and the hardware itself to see why you didn't get anywhere near peak performance. People spend years doing this. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-09T14:58:47.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19611" PostTypeId="1" CreationDate="2015-05-09T17:58:01.620" Score="1" ViewCount="103" Body="&lt;p&gt;I am used to using GPU hardware for large scale matrix operations and vectorizing mathematical operations on a continuous space which has been discretized for numerical computation, but this is a completely different problem and I would like to know if it is suitable for a GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have a string &lt;code&gt;X&lt;/code&gt; of length &lt;code&gt;N&lt;/code&gt; and a string &lt;code&gt;Y&lt;/code&gt; of length &lt;code&gt;M&lt;/code&gt;, such that &lt;code&gt;N &amp;lt; M&lt;/code&gt;. I can use, for example, the Python statement &lt;code&gt;if( X in Y )&lt;/code&gt; to find if &lt;code&gt;X&lt;/code&gt; is a substring of &lt;code&gt;Y&lt;/code&gt;. Now, suppose that I have a string &lt;code&gt;X&lt;/code&gt; and 10 million strings &lt;code&gt;{Y_i}&lt;/code&gt;. Is it possible (worth it) to put this type of problem on a GPU? Rather than having this be 10 million operations, we could do it in one operation on a GPU, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I am asking is that, in reality, I have maybe 10 Billion strings &lt;code&gt;{X_i}&lt;/code&gt; and need to, for each string, compare it to every string in &lt;code&gt;{Y_i}&lt;/code&gt; and am wondering if I can make it so that finding &lt;code&gt;X&lt;/code&gt; in any of &lt;code&gt;{Y_i}&lt;/code&gt; will be a GPU kernel for when these sets get incredibly large.&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2015-05-09T17:58:01.620" Title="Is this problem statement good for a GPU?" Tags="|python|gpu|combinatorics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19612" PostTypeId="1" CreationDate="2015-05-09T20:08:49.307" Score="1" ViewCount="1402" Body="&lt;p&gt;I use Armadillo as an interface to OpenBLAS. In my current program, I have a loop, in which I do multiplications of the form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(long t = t0; t &amp;lt; t1; t+=tStep)&#xA;{&#xA;    stateMatrix %= elementWiseEvolutionMatrix;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The operator &lt;code&gt;%&lt;/code&gt; is an element-wise multiplication operator. The problem here is that for matrices of side length of 500+ (the ones I have at hand), I can see that there is no parallelization whatsoever. Now I would like to point out that normal matrix multiplication is parallelized. But this kind of element-wise multiplication is not parallelized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I know that? Because I go to &lt;code&gt;htop&lt;/code&gt; in my linux system, and I see that only one core is busy, while if I do the same with normal matrix multiplication, I see that all cores go busy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I tried to manually parallelize this with OpenMP, but no luck. I tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(long t = t0; t &amp;lt; t1; t+=tStep)&#xA;{&#xA;    #pragma omp parallel for&#xA;    for(long i = 0; i &amp;lt; static_cast&amp;lt;long&amp;gt;(stateMatrix.n_rows); i++)&#xA;    {&#xA;        stat1eMatrix.row(i) %= elementWiseEvolutionMatrix.row(i);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this got all the cores busy, but the program became about a factor of 10 slower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; How can I get the element-wise multiplication to be as fast as possible with parallelization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I would like to point out that I'm more than happy to use another library for the element-wise multiplication, if necessary.&lt;/p&gt;&#xA;" OwnerUserId="5410" LastEditorUserId="5410" LastEditDate="2015-05-09T23:43:15.817" LastActivityDate="2015-05-10T12:57:49.167" Title="Parallelization of element-wise matrix multiplication" Tags="|parallel-computing|c++|matrix|quantum-mechanics|dense-matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19613" PostTypeId="1" CreationDate="2015-05-09T21:25:42.830" Score="1" ViewCount="764" Body="&lt;p&gt;It has been known that Fourier power spectrum somehow obeys power law therefore the slope of the spectrum can be used to calculate the fractal dimension of an image. Many people have used it for texture analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to calculate the Fourier fractal dimension of a binary vessel image and have produced radially averaged power spectrum, using the matlab code here: &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/6167-calculation-of-fractal-dimension-of-fractal-surfaces-using-fft&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/6167-calculation-of-fractal-dimension-of-fractal-surfaces-using-fft&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the spectrum I have plotted is like this,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vr84f.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I was told that this curve does not look like power law. I want to ask whether the comment is true and why, and is it because I'm using binary image rather than gray-scale?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was also told that power law plot should be like a straight line so I further extended my experiments and added Gaussian noise to my binary image, like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4iSTz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the plot now much more like a power law so the problem is that I did it with binary image, but adding the Gaussian noise destroy the image and the results is not valid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to keep the result I had before but is there anyway to explain my plot to be valid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="16324" LastActivityDate="2015-05-09T21:25:42.830" Title="radially averaged power spectrum of a binary image does not look like power law" Tags="|matlab|computational-physics|fftw|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19614" PostTypeId="2" ParentId="19612" CreationDate="2015-05-09T23:07:06.553" Score="3" Body="&lt;p&gt;There's no element wise multiplication operation in the BLAS library.  Your best approach is probably to just implement the operation yourself using (e.g.) OpenMP threading.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before you do this, you should consider Amdahl's law and whether speeding up this bit of your code is really going to help- chances are that these elementwise multiplications are not where your code is spending most of its time, and as a result you probably won't see much speedup from parallelizing this part of your code.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-05-09T23:07:06.553" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="19616" PostTypeId="2" ParentId="19612" CreationDate="2015-05-10T01:03:52.110" Score="2" Body="&lt;p&gt;First off, I agree with Brian Borchers comments about profiling to make sure these element-wise multiplications are where your performance issue lies. However, since you are convinced that is your problem, here is another suggestion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before trying to exploit multiple CPU, I would make sure that you have an&#xA;implementation that exploits vectorization. The SSE2 instruction set (available in most modern processors) has an&#xA;operation to multiply vectors of double precision floating point numbers. Your code may or may not allow your compiler to exploit this instruction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, Armadillo does not have any direct support for SSE2. But since you indicated a willingness to switch libraries, the Eigen library (&lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;http://eigen.tuxfamily.org/index.php?title=Main_Page&lt;/a&gt;) definitely does generate code using the SSE2 instructions. This could give you a 4x improvement for single precision multiplies and 2x improvement for double precision. If you are fortunate enough to have a CPU that supports the AVX instruction set, the development version of Eigen supports this to provide additional speedup.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2015-05-10T12:57:49.167" LastActivityDate="2015-05-10T12:57:49.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19617" PostTypeId="2" ParentId="19602" CreationDate="2015-05-10T07:05:26.337" Score="2" Body="&lt;p&gt;You can use double exponential quadrature which is very effective in handling end point singularity.&lt;/p&gt;&#xA;" OwnerUserId="1106" LastActivityDate="2015-05-10T07:05:26.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19618" PostTypeId="1" AcceptedAnswerId="19707" CreationDate="2015-05-10T08:22:12.573" Score="2" ViewCount="3537" Body="&lt;p&gt;It is well known that upwind schemes are stable when calculating convection flows with $|\text{Pe}|&amp;gt;2$, $\text{Pe}$ is the Peclet number. Why is that, and why is central difference unstable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any intrinsic reason there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any explanation, references, links will be helpful.&lt;/p&gt;&#xA;" OwnerUserId="16325" LastEditorUserId="9667" LastEditDate="2015-05-12T01:16:51.057" LastActivityDate="2022-01-06T18:22:22.707" Title="Why are upwind schemes stable in convection flow calculation?" Tags="|fluid-dynamics|stability|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19619" PostTypeId="2" ParentId="19605" CreationDate="2015-05-10T10:53:48.550" Score="5" Body="&lt;p&gt;Get the indices, where you want to split:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; np.diff(x)&#xA;array([ 0,  0, -1,  0,  0,  0,  0,  0,  0,  0,  1,  0,  0,  0,  0,  0,  0,&#xA;        0,  0,  0,  0,  0, -1,  0,  1,  0,  0, -1,  0,  0,  0,  0,  0,  1,&#xA;        0,  0,  0, -1,  0,  0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Every &lt;code&gt;1&lt;/code&gt; denotes the change from &lt;code&gt;0&lt;/code&gt; to &lt;code&gt;1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apply it (shifted by one index of course):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; np.split(x, np.nonzero(np.diff(x) == 1)[0]+1)&#xA;[array([1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0]), array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]), array([1, 1, 1, 0, 0, 0, 0, 0, 0]), array([1, 1, 1, 1, 0, 0, 0])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10152" LastEditorUserId="10152" LastEditDate="2015-05-10T10:59:48.187" LastActivityDate="2015-05-10T10:59:48.187" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19621" PostTypeId="2" ParentId="19618" CreationDate="2015-05-10T18:26:52.330" Score="4" Body="&lt;p&gt;An intuitive reason is that upwinding can be viewed as providing additional numerical diffusion, which is typically associated with stability in standard Finite Difference/Finite Element schemes. In finite differences, this means that the first order upwind scheme can be rewritten&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{u_i-u_{i-1}}{h} = \frac{u_{i+1}-u_{i-1}}{2h} - \frac{h}{2} \frac{u_{i+1}-2u_i + u_{i-1}}{h^2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the first order upwind difference can be interpreted as adding additional artificial diffusion relative to the 2nd order central difference scheme. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why central differences is unstable is a little more involved. IMO, it's easier to analyze stability in the Finite Element framework. For a variational problem $b(u,v) = l(v)$, the constant $\gamma_h$ in the discrete coercivity condition (where $\|u\|_h$ is some norm on $u$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$b(u,u)\geq \gamma_h \|u\|_h^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gives a measure of the stability of the problem (sort of like the smallest singular value tells you the stability of a matrix equation). For standard Galerkin methods (which are related to 2nd order central finite differencing) and Discontinuous Galerkin with central fluxes, $\gamma_h$ is inversely proportional to the Peclet number, and grows more unstable as Peclet number increases. For example, take the 1D Galerkin scheme for constant advection diffusion $ u' - \epsilon u = f$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$b(u,v) = \int_0^1 u'v + \epsilon \int_0^1 u'v' = \int f v$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming $0$ Dirichlet boundary conditions, $b(u,u)$ gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_0^1 u'u + \epsilon \int_0^1 (u')^2 = \int_0^1 (u^2/2)' + \epsilon \int_0^1 (u')^2 &#xA;= [u^2/2]_0^1 + \epsilon\int_0^1 (u')^2 = \epsilon\|u\|_{H^1}^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which implies issues with stability as $\epsilon \rightarrow 0$. Note that this implies this issue with stability is usually worse for boundary layer problems (Dirichlet boundary conditions on both sides).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upwinded Galerkin (i.e. SUPG, streamline diffusion, etc) and upwind DG restore discrete coercivity with respect to a specific norm $\|u\|_h$, such that $\gamma_h$ does not approach $0$ as Peclet number increases. This is also a part of why adding numerical diffusion tends to stabilizes advective flows - additional numerical diffusion adds a coercive term to the variational form.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-05-10T21:40:27.460" LastActivityDate="2015-05-10T21:40:27.460" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19622" PostTypeId="2" ParentId="19609" CreationDate="2015-05-10T18:41:34.003" Score="8" Body="&lt;p&gt;As one of the library's authors, I would of course love for deal.II to come out on top with this comparison. But I suspect it may not, and the answer lies in a factor you omit from your comparison: how long it actually takes to implement your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Few people in academia with the skills to implement a FEM code from scratch spend more time &lt;em&gt;solving&lt;/em&gt; PDEs than actually &lt;em&gt;implementing&lt;/em&gt; them. In other words, the time to implement things is &lt;em&gt;more valuable&lt;/em&gt; then the actual run time of your code. This is what brought us libraries such as deal.II, FEniCS, PETSc, etc: Yes, they may not be the very most efficient implementation of whatever it is you want to do, but they are &lt;em&gt;generic&lt;/em&gt; in the sense that they can be used for a large variety of problems, and they are &lt;em&gt;comprehensive&lt;/em&gt; in that they get you 90% there without having to write very much code. For example, deal.II's parallel Laplace solver, step-40, has only around 135 lines of code (semicolons). That's likely 2 orders of magnitude less than you had to implement if you wanted to do it from scratch -- saving you from writing 20,000 lines of code yourself (about a year's worth of work). Furthermore, switching to a higher order element is a one line change. FEniCS may even get you down to 20 lines of code, though the saved 115 lines of course don't quite add up to very much compared to the initial savings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Point being, performance and run time are only one metric. They matter if you intend to burn hundreds of thousands or more CPU hours. For the rest of us, development time is more valuable.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-11-21T01:50:25.863" LastActivityDate="2016-11-21T01:50:25.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19624" PostTypeId="2" ParentId="19607" CreationDate="2015-05-11T12:07:34.400" Score="1" Body="&lt;p&gt;Reformulate your set of equations such that the coefficients are the unknowns and perform least square fitting on A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done by the following trick:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be $x$ a column vector $(n\times 1)$, $A$ the unknown matrix $(n\times n)$,&#xA;you know that $Ax =b \Rightarrow  \sum_j a_{ij} x_j = b_i$ but this can also be seen as $\hat X\, \textrm{Vec}(A)= b$ where $\textrm{Vec}(A)$ is the &lt;a href=&quot;http://en.wikipedia.org/wiki/Vectorization_(mathematics)&quot; rel=&quot;nofollow&quot;&gt;vectorization operator&lt;/a&gt; and $\hat X$ must be n-times repeated and shifted by n $x^T$ or $0^T$ otherwise, leading to $\Rightarrow \hat X=x^T\otimes I_n $, where $I_n$ is the identity matrix of dimension n.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the full system you can rewrite it as following,&#xA;set $X=(x_1,...,x_n)$ and $B=(b_1,...,b_n)$ of your original equations $A X = B$, your least square problem for $A$ reads $(X^T\otimes I_n)\textrm{Vec}(A)=\textrm{Vec}(B)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it all boils down if you have enough matrix equations whether this will be an over or undertemined system and can be directly solved by standard numerical techniques.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2015-05-19T08:55:32.750" LastActivityDate="2015-05-19T08:55:32.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19625" PostTypeId="2" ParentId="19618" CreationDate="2015-05-11T13:14:34.413" Score="0" Body="&lt;p&gt;I hope to answer your question in a general matter without any references to mathematical equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be stable, a numerical scheme needs to capture the physics of the equation you are discretizing. Without going into the details, each term in your equation has eigenvalues which define the direction in which the information of your flow travels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when discretizing the diffusion term in the Navier-Stokes Equations (NSE), you should use a central scheme since diffusion, by definition, happens in all directions. If you were to use a one-sided numerical scheme, you could make your scheme stabilized by satisfying the CFL condition, but your results may become non-physical, since it can't exactly capture expansion waves. Where we should use &lt;a href=&quot;http://math.tifrbng.res.in/~praveen/notes/cm2013/fv_hyp2.pdf&quot; rel=&quot;nofollow&quot;&gt;Godunov's family&lt;/a&gt; schemes. Please note that Godunov's family schemes use upwinding with some constraints. Now, if you are solving the NSE for a supersonic flow, where the flow characteristics travel in &lt;strong&gt;ONLY&lt;/strong&gt; one direction, then you &lt;strong&gt;MUST&lt;/strong&gt; use an upwind scheme for stability. Good reference for upwind schemes in high speed flow is a paper by &lt;a href=&quot;http://arc.aiaa.org/doi/abs/10.2514/6.2003-3559&quot; rel=&quot;nofollow&quot;&gt;van Leer&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="16910" LastEditDate="2016-09-07T14:55:19.153" LastActivityDate="2016-09-07T14:55:19.153" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19626" PostTypeId="1" AcceptedAnswerId="19627" CreationDate="2015-05-11T19:14:25.387" Score="2" ViewCount="1277" Body="&lt;p&gt;I have a collection of sample particles, with (x,y,z) coordinates generated by a simplified Monte Carlo-like code.  I expect that these particles will follow an anisotropic diffusion process, which would give me a 3D Gaussian distribution with different variances along different axes (based on the geometry of the problem, I do expect the Gaussian to be axis-aligned; I suppose future development may generate a rotated 3D Gaussian, but I can deal with that generalization when and if it arises).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm looking for is a way to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Check that my data is consistent with a 3D Gaussian distribution (to confirm that diffusion is a good model);&lt;/li&gt;&#xA;&lt;li&gt;Get the best-fit mean and variance (along each axis), with error if possible (to measure the drift velocity and the diffusion coefficients).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I thought about binning the data and doing a fit, but I'm not sure that's a great method.  I was told to look into the Kolmogorov-Smirnov test, but that appears to be applicable to 1D distributions not 3D.  Is there an accepted method for this sort of thing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I generally work in C++, Fortran, or Python, so I'm hoping to avoid answers along the lines of &quot;Matlab has a function to do that&quot;.  If Python has a method, I can work with that.  Otherwise, I don't mind coding up my own tool, but I don't know what algorithm to use.&lt;/p&gt;&#xA;" OwnerUserId="15167" LastActivityDate="2015-08-03T23:45:31.297" Title="Determining if samples fit a 3D Gaussian distribution" Tags="|algorithms|diffusion|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19627" PostTypeId="2" ParentId="19626" CreationDate="2015-05-11T19:33:48.843" Score="2" Body="&lt;p&gt;Your first test should be to compute the mean value and covariance matrix of your point sample. If these converge to the correct values as you increase the number of samples, you are, from a practical perspective on the safe side that your points indeed come from the correct distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in practice there are many distributions that have the same mean and covariance matrix. If you are still unsure, you could also consider higher order moments of your point set, though I think that in practice mean and covariance are probably sufficient discriminating.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-05-11T19:33:48.843" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19628" PostTypeId="1" AcceptedAnswerId="19637" CreationDate="2015-05-12T03:00:12.937" Score="15" ViewCount="46193" Body="&lt;p&gt;Given a positive definite symmetric matrix, what is the fastest algorithm for computing the inverse matrix and its determinant? For problems I am interested in, the matrix dimension is 30 or less.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;High accuracy and speed is really necessary. (millions matrices are performed)&lt;/li&gt;&#xA;&lt;li&gt;The determinant is necessary.In each calculation, only one element of the iverse matrix is required. &#xA;Thanks!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3911" LastEditorUserId="3911" LastEditDate="2016-06-11T00:45:04.150" LastActivityDate="2020-07-28T23:08:43.233" Title="What is the fastest algorithm for computing the inverse matrix and its determinant for positive definite symmetric matrices?" Tags="|algorithms|matrix|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19630" PostTypeId="1" CreationDate="2015-05-12T04:44:56.350" Score="0" ViewCount="79" Body="&lt;p&gt;I need to include experimental results for lots of PDE (partial differential equation) matrices in my research work. How can I identify PDE matrices? For example, matrices in &lt;a href=&quot;http://yifanhu.net/GALLERY/GRAPHS/search.html&quot; rel=&quot;nofollow&quot;&gt;the UFL Sparse Matrix Collection&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3978" LastEditorUserId="3978" LastEditDate="2015-05-12T05:56:37.660" LastActivityDate="2015-05-12T05:56:37.660" Title="How can PDE matrices be identified?" Tags="|pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19631" PostTypeId="2" ParentId="19630" CreationDate="2015-05-12T05:51:54.113" Score="2" Body="&lt;p&gt;For the UF sparse matrix collection, you can search for &quot;PDE&quot; in the keyword section and 8 results will appear. The notes section for many of the entries will indicate the provenance of the example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another strategy you can employ is to look at examples in packages like PETSc, FEniCS, deal.II, and others that are commonly used to solve PDEs. It should be possible to modify the code in these examples to write the coefficients of linear systems to disk to get additional examples of &quot;PDE matrices&quot;. &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-05-12T05:51:54.113" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19632" PostTypeId="1" AcceptedAnswerId="19639" CreationDate="2015-05-12T05:53:21.960" Score="0" ViewCount="3166" Body="&lt;p&gt;What does the following &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; error mean numerically?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Repeated convergence failures (perhaps bad Jacobian or tolerances).&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, if I have some certainty that the model's programmatic implementation is correct, then what does the error message itself regarding the numerical scheme's issue with the solution process?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="9466" LastEditDate="2015-05-12T13:11:41.200" LastActivityDate="2015-05-13T06:39:04.490" Title="`scipy`: what does `scipy.integrate.odeint`'s Repeated convergence failures (perhaps bad Jacobian or tolerances).&quot; error mean?" Tags="|python|convergence|time-integration|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19634" PostTypeId="1" CreationDate="2015-05-12T15:03:16.463" Score="5" ViewCount="377" Body="&lt;p&gt;There are many well defined measures for many basic geometrical objects such as rectangularity (area coverage of minimum bounding rectangle), triangularity (area coverage of minimum enclosing triangle), circularity ratio (area/perimeter$^2$), etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for relevant measures to classify C-shaped objects (or similarly U-shaped objects). By classify I simply mean to assign a membership to the set &quot;C-shaped objects&quot; if the measure (or aggregate of measures) reach a certain threshold. The measure itself together with the threshold level will thus define what a constitutes a &quot;C-shaped object&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of the articles I have found on shape measurements and feature extraction ever mention C-shapes. I know that defining a C-shape might be difficult and rather subjective, and therefore a single measure might be difficult to construct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If no single measurement that can be used to describe C-shapes exists, maybe there is a good aggregate of measures that can help in classifying C-shapes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that various forms of template matching methods could do the trick, but I am looking for a measure that is not reliant on a model and less computationally expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only idea I could come up with is some kind of radial density distribution (see pic below). But I am unsure on how to utilize the density information to create a relevant measure with good discriminatory properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any input would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Pdsqu.png&quot; alt=&quot;radial_density_plot&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="6601" LastEditorUserId="6601" LastEditDate="2015-05-12T19:43:50.700" LastActivityDate="2019-08-26T05:49:57.923" Title="Shape measure for C-shaped objects" Tags="|image-processing|geometry|computational-geometry|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19636" PostTypeId="2" ParentId="19586" CreationDate="2015-05-12T22:50:41.797" Score="4" Body="&lt;p&gt;Before looking for a &quot;black box&quot; tool, that can be used to execute in parallel &quot;generic&quot; python functions, I would suggest to analyse how &lt;code&gt;my_function()&lt;/code&gt; can be parallelised by hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, compare execution time of &lt;code&gt;my_function(v)&lt;/code&gt; to python &lt;code&gt;for&lt;/code&gt; loop overhead: [C]Python &lt;code&gt;for&lt;/code&gt; loops are pretty slow, so time spent in &lt;code&gt;my_function()&lt;/code&gt; could be negligible.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; timeit.timeit('pass', number=1000000)&#xA;0.01692986488342285&#xA;&amp;gt;&amp;gt;&amp;gt; timeit.timeit('for i in range(10): pass', number=1000000)&#xA;0.47521495819091797&#xA;&amp;gt;&amp;gt;&amp;gt; timeit.timeit('for i in xrange(10): pass', number=1000000)&#xA;0.42337894439697266&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Second check if there is a simple vector implementation of &lt;code&gt;my_function(v)&lt;/code&gt; that does not require loops: &lt;code&gt;F[:] = my_vector_function(X)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(These two first point are pretty trivial, forgive me if I mentioned them here just for completeness.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third and most important point, at least for CPython implementations, is to check whether &lt;code&gt;my_function&lt;/code&gt; spends most of it's time &lt;em&gt;inside&lt;/em&gt; or &lt;em&gt;outside&lt;/em&gt; the &lt;a href=&quot;https://wiki.python.org/moin/GlobalInterpreterLock&quot; rel=&quot;nofollow noreferrer&quot;&gt;global interpreter lock&lt;/a&gt;, or &lt;a href=&quot;https://wiki.python.org/moin/GlobalInterpreterLock&quot; rel=&quot;nofollow noreferrer&quot;&gt;GIL&lt;/a&gt;. If time is spent outside the GIL, then the &lt;a href=&quot;https://docs.python.org/3/library/threading.html#module-threading&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;threading&lt;/code&gt; standard library module&lt;/a&gt; should be used. (&lt;a href=&quot;http://wiki.scipy.org/Cookbook/Multithreading&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; an example). BTW, one could think of writing &lt;code&gt;my_function()&lt;/code&gt; as a C extension just to release the GIL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if &lt;code&gt;my_function()&lt;/code&gt; does not release the GIL, one could use the &lt;a href=&quot;https://docs.python.org/3/library/multiprocessing.html#module-multiprocessing&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;multiprocessing&lt;/code&gt; module&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References: &lt;a href=&quot;https://docs.python.org/3/library/concurrency.html#concurrent-execution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python docs on Concurrent Execution&lt;/a&gt;, and &lt;a href=&quot;http://wiki.scipy.org/ParallelProgramming&quot; rel=&quot;nofollow noreferrer&quot;&gt;numpy/scipy intro on parallel processing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2015-05-12T22:50:41.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19637" PostTypeId="2" ParentId="19628" CreationDate="2015-05-13T00:29:19.480" Score="15" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;For problems I am interested in, the matrix dimension is 30 or less.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As WolfgangBangerth notes, unless you have a large number of these matrices (millions, billions), performance of matrix inversion typically isn't an issue.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given a positive definite symmetric matrix, what is the fastest algorithm for computing the inverse matrix and its determinant?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If speed is an issue, you should answer the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do you really need the whole inverse? (Many applications don't need to form an explicit inverse.)&lt;/li&gt;&#xA;&lt;li&gt;Do you really need the determinant? (Determinants are uncommon, but certainly not unheard of in computational science.)&lt;/li&gt;&#xA;&lt;li&gt;Do you need either to high accuracy? (Low accuracy algorithms tend to be faster.)&lt;/li&gt;&#xA;&lt;li&gt;Would a probabilistic approximation suffice? (Probabilistic algorithms tend to be faster.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The standard response to your problem of inverting a small, positive definite matrix and calculating its determinant would be Cholesky decomposition. If $A = LL^{T}$, then $\det(A) = \prod_{i=1}^{n}l_{ii}^{2}$, and $\det(A^{-1}) = \prod_{i=1}^{n}l_{ii}^{-2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming $A$ is $n$ by $n$, the Cholesky decomposition can be computed in around $n^{3}/3$ flops, which is about half the cost of an LU decomposition. However, such an algorithm would not be considered &quot;fast&quot;. A &lt;a href=&quot;http://arxiv.org/pdf/1310.7202v3.pdf&quot;&gt;randomized LU decomposition&lt;/a&gt; might be a faster algorithm worth considering if (1) you really do have to factor a large number of matrices, (2) the factorization is really the limiting step in your application, and (3) any error incurred in using a randomized algorithm is acceptable. Your matrices are probably too small for sparse algorithms to be worthwhile, so the only other opportunities for faster algorithms would require additional matrix structure (e.g., banded), or exploiting problem structure (e.g., maybe you can cleverly restructure your algorithm so that you no longer need to calculate a matrix inverse or its determinant). Efficient determinant algorithms are roughly the cost of solving a linear system, to within a constant factor, so the same arguments used for linear systems apply to calculating determinants as well.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-05-13T00:29:19.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19638" PostTypeId="1" AcceptedAnswerId="19640" CreationDate="2015-05-13T02:59:47.160" Score="3" ViewCount="641" Body="&lt;p&gt;With &lt;code&gt;scipy&lt;/code&gt;, I have the choice of using &quot;lsoda&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Real-valued Variable-coefficient Ordinary Differential Equation solver, with fixed-leading-coefficient implementation. It provides automatic method switching between implicit Adams method (for non-stiff problems) and a method based on backward differentiation formulas (BDF) (for stiff problems).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or &quot;dopri5&quot;, which Wikipedia tells me is a non-stiff method that is the default choice of MATLAB's &lt;code&gt;ode45&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some general heuristics for using &quot;lsoda&quot; vs. &quot;dopri5&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose an obvious one consideration: &quot;is your problem stiff?&quot;. Well, if the problem I am working with is stiff, then &quot;lsoda&quot; is struggling with &quot;Excess work done.&quot; too...so, the BDF method doesn't seem to help too much with stiffness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you share some insight?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2015-05-13T06:59:58.103" Title="General heuristics for making a choice &quot;dopri5&quot;, and &quot;lsoda&quot;?" Tags="|ode|scipy|integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19639" PostTypeId="2" ParentId="19632" CreationDate="2015-05-13T06:39:04.490" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What does the following scipy.integrate.odeint error mean numerically?&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Repeated convergence failures (perhaps bad Jacobian or tolerances).&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You'll have to tell us what method you're using. I can think of two broad (not necessarily exhaustive) scenarios:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;You're using an explicit method&lt;/strong&gt;, so there are no nonlinear equation solves. Then any adaptive-time-stepping method is going to base adapting the time step on an error criterion involving some tolerances set either by you, the user, or by default in the software. The test in the error criterion will reduce the time step if the error is too large relative to the tolerances, and once the suggested time step becomes too small, the program typically returns an error message saying that the error tolerances cannot be met. (See, for instance, Hairer and Wanner, Volume I, Section II.4, where they discuss a strategy for Runge-Kutta methods; a similar strategy is also briefly discussed in the &lt;a href=&quot;https://computation.llnl.gov/casc/sundials/documentation/cv_guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SUNDIALS documentation&lt;/a&gt; around equation (2.6) in that text.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;You're using an implicit method,&lt;/strong&gt; in which case there are nonlinear equation solves, and more potential points of failure. For instance, there could be an issue with tolerances, again, which usually manifests itself as in the case with explicit methods described above: if tolerances are too tight and the error criterion in the step size selection step cannot be met by the minimum time step defined in the implementation, the implementation will return a convergence failure. There could also be issues where the nonlinear solver fails to converge to a given tolerance (either user set, or hardcoded in the implementation) within a sufficient number of iterations. Depending on the method used for solving the nonlinear equations embedded in the implicit method, there could be embedded linear solves, and these solves may not converge; in many cases, this convergence failure can be due to errors in implementing the Jacobian, or due to an insufficiently accurate approximation to the Jacobian. (For instance, I have seen convergence failures for stiff problems that occur when a finite difference approximation to the Jacobian matrix is used, and I have seen these convergence failures resolve when the finite difference approximation is replaced with an analytical expression for the Jacobian.) Chapter 2 of the &lt;a href=&quot;https://computation.llnl.gov/casc/sundials/documentation/cv_guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SUNDIALS documentation&lt;/a&gt; does a good job of overviewing these general considerations as they apply to the BDF method implemented in CVODE.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;That is, if I have some certainty that the model's programmatic implementation is correct, then what does the error message itself regarding the numerical scheme's issue with the solution process?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It really means you have to do more diagnosis.&lt;/p&gt;&#xA;&lt;p&gt;SciPy is a fine package, and they do a great job of wrapping established numerical software in Python wrappers (dopri5, vode, etc.), but these codes weren't written with ease of numerical diagnostics in mind, and consequently, you'll have to rig up a lot more in the way of &amp;quot;tools&amp;quot; (ways to get output regarding the convergence of any linear or nonlinear solver iterations) than you would if you were using a library like PETSc (where the developers, through experience in having to experiment with different numerical methods, have built in many tools that provide helpful diagnostic output).&lt;/p&gt;&#xA;&lt;p&gt;The tradeoff between these two sets of tools is that SciPy-like tools are simpler to learn, are great when they &amp;quot;just work&amp;quot;, and provide small, simple public APIs, while PETSc-like tools have larger and more complicated public APIs, are more powerful, and require more time and effort to learn.&lt;/p&gt;&#xA;&lt;p&gt;Generally speaking, you need to look at the residual versus iteration output for any equation solves in your ODE solver, and probably also output from the step size error test in your ODE solver in order to distinguish where the point of failure is in your application, and then from there, you need to figure out how to address that point of failure (which could be as simple as loosening tolerances, or as complicated as using a different discretization because there could be fundamental flaws in the numerical methods you are using, which could be inapplicable to your problem).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-05-13T06:39:04.490" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19640" PostTypeId="2" ParentId="19638" CreationDate="2015-05-13T06:59:58.103" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What are some general heuristics for using &amp;quot;lsoda&amp;quot; vs. &amp;quot;dopri5&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;I suppose an obvious one consideration: &amp;quot;is your problem stiff?&amp;quot;. Well, if the problem I am working with is stiff, then &amp;quot;lsoda&amp;quot; is struggling with &amp;quot;Excess work done.&amp;quot; too...so, the BDF method doesn't seem to help too much with stiffness.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If forced to choose between only those two implementations, &amp;quot;Is your problem stiff?&amp;quot; is going to be the most important question, followed by more subtle discussions of work-precision tradeoffs. You really would need to construct a work-precision diagram, or you would need to say something about your application. What follows are some speculative scenarios that might explain what you would see:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If your problem is &amp;quot;moderately&amp;quot; or &amp;quot;mildly&amp;quot; stiff, it could still be worth using LSODE instead of DOPRI5 because you might be able to take significantly larger time steps than DOPRI5, and the cost of the linear solves required could be cheaper than the many function evaluations DOPRI5 would have to employ with shorter time steps.&lt;/li&gt;&#xA;&lt;li&gt;It could also be that linear solves and Jacobian evaluations are relatively cheap for your case (maybe your Jacobian is banded, or tridiagonal), so the LSODE time steps don't have to be that much larger to see an advantage.&lt;/li&gt;&#xA;&lt;li&gt;If LSODE adaptively selects increasingly high orders for your problem, for stringent error tolerances, higher order methods generally win, and LSODE implements an order 12 implicit Adams method for &amp;quot;nonstiff&amp;quot; problems&lt;/li&gt;&#xA;&lt;li&gt;However, for loose error tolerances, if stiffness (i.e., stability limit) issues do not dominate, lower order methods tend to prevail, so DOPRI5 might be more advantageous.&lt;/li&gt;&#xA;&lt;li&gt;Also, if Jacobian matrix evaluations or linear solves are significantly more expensive than function evaluations, and stability/stiffness is not an issue, DOPRI5 is probably more advantageous.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Unless the performance is critical, it's usually easier just to try DOPRI5, which is frequently (but not always) cheaper, and then if it doesn't work, try LSODE. If neither works with the initial settings you give them, then you have to do some diagnostic work to see where problems are occurring.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-05-13T06:59:58.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19641" PostTypeId="2" ParentId="16421" CreationDate="2015-05-13T11:32:31.870" Score="2" Body="&lt;p&gt;Your algorithm is a first-order Euler and not the Verlet algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the actual box size when you run the code? If you made a mistake and your particles end up being well separated (i.e. further apart than your cutoff) then the force is zero and the kinetic energy will remain constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just one suggestion to get started.&lt;/p&gt;&#xA;" OwnerUserId="16340" LastActivityDate="2015-05-13T11:32:31.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19643" PostTypeId="1" AcceptedAnswerId="19705" CreationDate="2015-05-13T17:25:49.557" Score="1" ViewCount="171" Body="&lt;p&gt;I have been stuck at this problem for a while :(&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $\mathbf{A}\in\mathbb{R}^{p\times p}, \mathbf{A}\ge 0,\mathbf{A} \text{ symmetric}, \mathbf{b}\in\mathbb{R}^n,\mathbf{c}_i\in\mathbb{R}^p\forall i,\mathbf{d}\in\mathbb{R}^p$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the function:&#xA;$$f(\mathbf{x},\mathbf{y},\mathbf{z}) = \sum_{1\le i\le n} x_i\mathbf{z}^T\mathbf{A}\mathbf{y}_i +\mathbf{b}^T\mathbf{x} + \sum_{1\le i\le n}\mathbf{c}_i^T\mathbf{y}_i +\mathbf{d}^T\mathbf{z}$$&#xA;where $\mathbf{x}\in \mathbb{R}^n,\mathbf{y}_i\in\mathbb{R}^p \forall i,\mathbf{z}\in \mathbb{R}^p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimize $f(\mathbf{x},\mathbf{y},\mathbf{z})$ subject to $\mathbf{0}\le\mathbf{x},\mathbf{y}_i,\mathbf{z}\le \mathbf{1}$ and $\mathbf{1}^T\mathbf{x}=\mathbf{1}^T\mathbf{y}_i=\mathbf{1}^T\mathbf{x}=1 \quad \forall i.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it can make the problem easier, then remove some (or all) of the three equality constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\mathbf{1}^T\mathbf{x}=1$&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{1}^T\mathbf{y}_i=1\quad\forall i,1\le i\le n$&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{1}^T\mathbf{z}=1$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for any suggestions!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P/s: a related problem: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/18775/find-min-xtay-subject-to-1tx-1ty-1-x-ge-0-y-ge-0&quot;&gt;Find $\min x^TAy$ subject to $1^Tx=1^Ty=1,x\ge 0,y\ge 0$&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-05-28T09:38:41.197" Title="Find $\min \sum_{1\le i\le n} x_i\mathbf{z}^T\mathbf{A}\mathbf{y}_i +\mathbf{b}^T\mathbf{x} +\cdots$" Tags="|optimization|nonlinear-programming|constrained-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19644" PostTypeId="1" CreationDate="2015-05-13T23:42:20.777" Score="2" ViewCount="191" Body="&lt;p&gt;To somewhat follow up on the question asked &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19609/comparing-various-implementations-software-packages-for-large-scale-finite-eleme&quot;&gt;here&lt;/a&gt;, I have been told that the &lt;a href=&quot;http://crd.lbl.gov/assets/pubs_presos/parlab08-roofline-talk.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roofline model&lt;/a&gt; is one way of assessing the performance of any scientific code. Basically I compute the Arithmetic Intensity (ratio of FLOPS over DRAM bytes) and multiply this by the STREAM bandwidth to obtain the ideal FLOPS/s. I could also use the AI to see how close I get to the maximum performance of my given machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, what is the easiest way to obtain this AI for finite element packages like FEniCS or Deal.II? To simply things for now, I am not too concerned with register/cache reuse or quantification of useful bandwidth sustained for some level of cache. If I wrote my own explicit finite element implementation using PETSc, I could simply count by hand the approximate number of FLOPS and approximate the number of load/stores from all vector operations and sparse matrix-vector multiply as outlined &lt;a href=&quot;http://www.cs.odu.edu/~keyes/papers/pcfd99_gkks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. However, does anyone have any suggestions for doing this for any given implementation?&lt;/p&gt;&#xA;" OwnerUserId="14576" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-05-14T00:05:21.423" Title="Performance based on the Roofline model" Tags="|parallel-computing|hpc|efficiency|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19645" PostTypeId="2" ParentId="19644" CreationDate="2015-05-14T00:05:21.423" Score="3" Body="&lt;p&gt;What you absolutely cannot do is measure the flops on any modern Intel architecture using the hardware performance counters. Starting with Nehalem, Intel chips essentially count floating point instructions issued, and if the data hasn't arrived to the processor registers, the instruction may not get retired. Instead it will get reissued some time later, and this keeps happening until the data is available and the instruction can finish. As a result, hardware PMC measurements of STREAM Triad over count flops by a factor of 10. DGEMM is over counted by a few percent. These probably represent the bounds. This got so bad that Intel disabled the flop performance counters on Haswell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, your only real option is to open the code and manually count the operations unless some of these libraries include instrumentation that counts them for you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some sense, why go to this trouble? Why not do two things, since you're going to be setting up and running these codes anyway: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Run exactly the same problem in each code on a given architecture compiled the same way to the best of your ability&lt;/li&gt;&#xA;&lt;li&gt;Run each case for different values of the size parameter (grid points, elements, whatever)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With the run times of each of these cases, you should be able to fit a polynomial or other complexity theory function to run time data thereby confirming that the method has the expected computational complexity and also estimating the various associated constants. I.e. if you expect $O(n^2)$, then you should get a good fit with a quadratic. You may need to ignore the small end of $n$ entirely. You should also be able to see which one is faster. Unless these curves cross multiple times (which they really shouldn't once you get into the asymptotic regime), the one that is fastest is the most efficient. Make sure you run large enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asking whether any one of these codes is as efficient as it theoretically could be is a much harder question.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-14T00:05:21.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19646" PostTypeId="2" ParentId="8899" CreationDate="2015-05-14T05:05:12.080" Score="10" Body="&lt;p&gt;@Pedro Gimeno&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;I doubt it can be any more robust than that.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Challenge accepted.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed the usual approach is to use trig functions like atan2. Intuitively, there shouldn't be a need to use trig functions. Indeed, all the results end up as sines and cosines of arctans--which can be simplified to algebraic functions. It took quite a while, but I managed to simplified Pedro's algorithm to use only algebraic functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following python code does the trick.&#xA;&lt;pre&gt;&#xA;from numpy import asarray, diag&lt;/p&gt;&#xA;&#xA;&lt;p&gt;def svd2(m):&lt;/p&gt;&#xA;&#xA;&lt;code&gt;    y1, x1 = (m[1, 0] + m[0, 1]), (m[0, 0] - m[1, 1])&#xA;    y2, x2 = (m[1, 0] - m[0, 1]), (m[0, 0] + m[1, 1])&#xA;&#xA;    h1 = hypot(y1, x1)&#xA;    h2 = hypot(y2, x2)&#xA;&#xA;    t1 = x1 / h1&#xA;    t2 = x2 / h2&#xA;&#xA;    cc = sqrt((1 + t1) * (1 + t2))&#xA;    ss = sqrt((1 - t1) * (1 - t2))&#xA;    cs = sqrt((1 + t1) * (1 - t2))&#xA;    sc = sqrt((1 - t1) * (1 + t2))&#xA;&#xA;    c1, s1 = (cc - ss) / 2, (sc + cs) / 2,&#xA;    u1 = asarray([[c1, -s1], [s1, c1]])&#xA;&#xA;    d = asarray([(h1 + h2) / 2, (h1 - h2) / 2])&#xA;    sigma = diag(d)&#xA;&#xA;    if h1 != h2:&#xA;        u2 = diag(1 / d).dot(u1.T).dot(m)&#xA;    else:&#xA;        u2 = diag([1 / d[0], 0]).dot(u1.T).dot(m)&#xA;&#xA;    return u1, sigma, u2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;" OwnerUserId="16343" LastActivityDate="2015-05-14T05:05:12.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19647" PostTypeId="1" AcceptedAnswerId="19648" CreationDate="2015-05-14T13:56:48.050" Score="2" ViewCount="158" Body="&lt;p&gt;I obtained a mass matrix through Finite Elements discretization. Now, I want to check if it is symmetric. To do that I subtract to my matrix $M$ its transposed $M^T$. The result is another matrix of which I compute the norm. The norms that I use are max norm defined as $\max(|a_{i,j}|; i,j)$ or infnorm defined as $\max(\sum(|a_{i,j}|,j),i)$. I would expect these norms to be zero but they are in the order of $10^{-19}$ or $10^{-17}$ if I use a coarser mesh. Do you think that it is a numerical problem simply due to finite precision effects?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14489" LastEditorUserId="9667" LastEditDate="2015-05-15T06:44:45.803" LastActivityDate="2015-05-15T06:44:45.803" Title="Is my matrix symmetric?" Tags="|finite-element|matrix|symmetry|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="19648" PostTypeId="2" ParentId="19647" CreationDate="2015-05-14T15:19:11.700" Score="5" Body="&lt;p&gt;As the comments note, it shouldn't be an issue with your eigensolver if the matrix is symmetric to machine precision. That being said, however, if you are really worried about it, just copy the transpose of the upper triangle to lower triangle after you generate the mass matrix, and then it will be exact.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-14T15:19:11.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19650" PostTypeId="1" CreationDate="2015-05-14T19:23:18.207" Score="1" ViewCount="156" Body="&lt;p&gt;I am solving a PDE, and one of the intermediate steps is to numerically integrate a function over a compact interval. The function is represented on a linearly spaced grid. I am using Simpsons rule (so I use the value of the integrand at $x_i$,$x_{i-1}$ and $x_{i-2}$ as well as the value of the integral at $x_{i-2}$ to obtain the integral at $x_i$) , but sometimes I get a zigzag noise after integrating. I know why (in the Simpsons rule the even numered points $x_{2i}$ and the odd $x_{2i+1}$ are sort of obtained separately, and depend a lot on the error of the first point $x_0$ and $x_1$. But the error there is greater since one must use a lower order routine since simpsons rule requires three points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I fix this? What are the state of the art integration routines on linearly spaced grids?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/jZm9V.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="15232" LastActivityDate="2015-05-14T19:23:18.207" Title="Numerical integration when solving PDE: Simpsons rule and high frequency noise" Tags="|pde|numerical-analysis|integration|" AnswerCount="0" CommentCount="2" FavoriteCount="0" ClosedDate="2019-06-09T18:16:30.053" ContentLicense="CC BY-SA 3.0" />
  <row Id="19651" PostTypeId="1" CreationDate="2015-05-15T02:53:23.167" Score="0" ViewCount="583" Body="&lt;p&gt;I have a grid of data on a 3D Cartesian grid and I would like to find a routine that will allow me to input this data and output a spherical harmonic transform for specific values of radial distances. Does anyone know if such a routine exists?? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="16352" LastActivityDate="2015-09-22T00:16:28.500" Title="Discrete Spherical Harmonic Transform from Cartesian grid" Tags="|spherical-harmonics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19652" PostTypeId="1" CreationDate="2015-05-15T13:14:57.313" Score="13" ViewCount="1783" Body="&lt;p&gt;I have implemented a backward-Euler solver in python 3 (using numpy). For my own convenience and as an exercise, I also wrote a small function that computes a finite difference approximation of the gradient so that I don't always have to determine the Jacobian analytically (if it is even possible!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the descriptions provided in &lt;a href=&quot;http://bookstore.siam.org/ot61/&quot;&gt;Ascher and Petzold 1998&lt;/a&gt;, I wrote this function which determines the gradient at a given point x:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def jacobian(f,x,d=4):&#xA;    '''computes the gradient (Jacobian) at a point for a multivariate function.&#xA;&#xA;    f: function for which the gradient is to be computed&#xA;    x: position vector of the point for which the gradient is to be computed&#xA;    d: parameter to determine perturbation value eps, where eps = 10^(-d).&#xA;        See Ascher und Petzold 1998 p.54'''&#xA;&#xA;    x = x.astype(np.float64,copy=False)&#xA;    n = np.size(x)&#xA;    t = 1 # Placeholder for the time step&#xA;    jac = np.zeros([n,n])&#xA;    eps = 10**(-d)&#xA;    for j in np.arange(0,n):&#xA;        yhat = x.copy()&#xA;        ytilde = x.copy()&#xA;        yhat[j] = yhat[j]+eps&#xA;        ytilde[j] = ytilde[j]-eps&#xA;        jac[:,j] = 1/(2*eps)*(f(t,yhat)-f(t,ytilde))&#xA;    return jac&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tested this function by taking a multivariate function for the pendulum and comparing the symbolic Jacobian to the numerically determined gradient for a range of points. I was pleased with the results of the test, the error was around 1e-10. When I solved the ODE for the pendulum using the approximated Jacobian, it also worked very well; I couldn't detect any difference between the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I tried testing it with the following PDE (Fisher's equation in 1D):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \partial_t \mathbf{u} = \partial_x(k\partial_x\mathbf{u}) + \lambda(\mathbf{u}(C-\mathbf{u}))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using a finite difference discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now Newton's method blows up in the first timestep:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/home/sfbosch/Fisher-Equation.py:40: RuntimeWarning: overflow encountered in multiply&#xA;  du = (k/(h**2))*np.dot(K,u) + lmbda*(u*(C-u))&#xA;./newton.py:31: RuntimeWarning: invalid value encountered in subtract&#xA;  jac[:,j] = 1/(2*eps)*(f(t,yhut)-f(t,yschlange))&#xA;Traceback (most recent call last):&#xA;  File &quot;/home/sfbosch/Fisher-Equation.py&quot;, line 104, in &amp;lt;module&amp;gt;&#xA;    fisher1d(ts,dt,h,L,k,C,lmbda)&#xA;  File &quot;/home/sfbosch/Fisher-Equation.py&quot;, line 64, in fisher1d&#xA;    t,xl = euler.implizit(fisherode,ts,u0,dt)&#xA;  File &quot;./euler.py&quot;, line 47, in implizit&#xA;    yi = nt.newton(g,y,maxiter,tol,Jg)&#xA;  File &quot;./newton.py&quot;, line 54, in newton&#xA;    dx = la.solve(A,b)&#xA;  File &quot;/usr/lib64/python3.3/site-packages/scipy/linalg/basic.py&quot;, line 73, in solve&#xA;    a1, b1 = map(np.asarray_chkfinite,(a,b))&#xA;  File &quot;/usr/lib64/python3.3/site-packages/numpy/lib/function_base.py&quot;, line 613, in asarray_chkfinite&#xA;    &quot;array must not contain infs or NaNs&quot;)&#xA;ValueError: array must not contain infs or NaNs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This happens for a variety of eps values, but strangely, only when the PDE spatial step size and time step size are set so that the Courant–Friedrichs–Lewy condition is not met. Otherwise it works. (This is the behaviour you'd expect if solving with forward Euler!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness, here is the function for the Newton Method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def newton(f,x0,maxiter=160,tol=1e-4,jac=jacobian):&#xA;    '''Newton's Method.&#xA;&#xA;    f: function to be evaluated&#xA;    x0: initial value for the iteration&#xA;    maxiter: maximum number of iterations (default 160)&#xA;    tol: error tolerance (default 1e-4)&#xA;    jac: the gradient function (Jacobian) where jac(fun,x)'''&#xA;&#xA;    x = x0&#xA;    err = tol + 1&#xA;    k = 0&#xA;    t = 1 # Placeholder for the time step&#xA;    while err &amp;gt; tol and k &amp;lt; maxiter:&#xA;        A = jac(f,x)&#xA;        b = -f(t,x)&#xA;        dx = la.solve(A,b)&#xA;        x = x + dx&#xA;        k = k + 1&#xA;        err = np.linalg.norm(dx)&#xA;    if k &amp;gt;= maxiter:&#xA;        print(&quot;Maxiter reached. Result may be inaccurate.&quot;)&#xA;        print(&quot;k = %d&quot; % k)&#xA;    return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(The function la.solve is scipy.linalg.solve.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confident that my backward Euler implementation is in order, because I have tested it using a function for the Jacobian and get stable results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can see in the debugger that newton() manages 35 iterations before the error occurs. This number remains the same for every eps I have tried.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An additional observation: when I compute the gradient with FDA and a function using the initial condition as an input and compare the two while varying the size of epsilon, the error grows as epsilon shrinks. I would expect it to be large at first, then get smaller, then larger again as epsilon shrinks. So an error in my implementation of the Jacobian is a reasonable assumption, but if so, it's so subtle that I am unable to see it. EDIT: I modified jacobian() to use forward instead of central differences, and now I observe the expected development of the error. However, newton() still fails to converge. Observing dx in the Newton iteration, I see that it only grows, there is not even a fluctuation: it nearly doubles (factor 1.9) with each step, with the factor getting progressively larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ascher and Petzold do mention that difference approximations for the Jacobian do not always work well. Can an approximated Jacobian with finite differences cause instability in Newton's method? Or is the cause somewhere else? How else might I approach this problem?&lt;/p&gt;&#xA;" OwnerUserId="16358" LastEditorUserId="1280" LastEditDate="2015-05-16T08:04:24.670" LastActivityDate="2015-05-20T19:09:38.100" Title="Can an approximated Jacobian with finite differences cause instability in the Newton method?" Tags="|pde|stability|numpy|scipy|newton-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19654" PostTypeId="1" CreationDate="2015-05-15T18:17:53.283" Score="0" ViewCount="111" Body="&lt;p&gt;Can I solve the integral given below using Matlab?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{C(J)}{C_0} =&#xA;\frac{2e^J}{\pi}\int\limits_{0}^{\infty} \frac{e^{\frac{\gamma}{2}\left[1 - \sqrt{\rho}\cos(\theta/2)\right]}}{a_1^2 + a_2^2} [a_1\cos(ZJ-w) + a_2\sin(ZJ - w)]\, dZ$$&#xA;where&#xA;\begin{align}&#xA;&amp;amp;\theta = \arctan(v/u)\\&#xA;&amp;amp;u = 1 + \frac{4}{\gamma}\left[1 + \frac{ba + a(1 + Z^2)}{(1 + b)^2 + Z^2}\right]\\&#xA;&amp;amp;v = \frac{4Z}{\gamma}\left[1 + \frac{ab}{(1 + b)^2 + Z^2}\right]\\&#xA;&amp;amp;\rho = \sqrt{u^2 + v^2}\\&#xA;&amp;amp;b = \frac{af}{1 - f}\\&#xA;&amp;amp;a_1 = 1 + \sqrt{\rho}\cos(\theta/2) - Z\sqrt{\rho}\sin(\theta/2)\\&#xA;&amp;amp;a_2 = Z\left[1 + \sqrt{\rho}\cos(\theta/2)\right] + \sqrt{\rho}\sin(\theta/2)\\&#xA;&amp;amp;w = \frac{\gamma}{2}\sqrt{\rho}\sin(\theta/2)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relation between $Z$ and $J$ is as follows:&#xA;$$Z = \frac{af}{1 - f} (J-y) \enspace .$$&#xA;$y$ varies in $[0, 1]$; $C_0=1$; $a= 0.065$; $f=0.7$.&#xA;I am new to Matlab. Any help will be highly appreciated. I ran the following code (gamma is not y and gamma=m=60, J=1/0.7, O=theta, C=C(J)/C0), but matlab is busy from last 12-13 hours!!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms z&#xA;m=60;&#xA;a = 0.065;&#xA;f = 0.7;&#xA;b = a*f/(1-f);&#xA;u = 1+((4/m)*(1+(b*a+a*(1+z.^2))/((1+b).^2)+z.^2));&#xA;v = (4*z/m)*(1+(a*b/(1+b).^2+z.^2));&#xA;p = sqrt(u.^2+v.^2);&#xA;O = atan(v/u);&#xA;a1 = 1+sqrt(p)*cos(O/2)-z*sqrt(p)*sin(O/2);&#xA;a2 = z*(1+sqrt(p)*cos(O/2))+sqrt(p)*sin(O/2);&#xA;w = (m/2)*sqrt(p)*sin(O/2);&#xA;fun = (exp(m/2*(1-sqrt(p)*cos(O/2)))/(a1.^2+a2.^2))*(a1*cos((z/0.7)-w)+a2*sin((z/0.7)-w));&#xA;I = int(fun, z, 0, inf);&#xA;C = 2*exp(1/0.7)*I/pi;&#xA;display(C)&#xA;plot(z,C)&#xA;xlabel('Z')&#xA;ylabel('concentration')&#xA;if C&amp;gt;1&#xA;    stop&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16362" LastEditorUserId="16362" LastEditDate="2015-05-16T11:08:08.077" LastActivityDate="2015-05-16T11:08:08.077" Title="Integration in MATLAB: Complicated integrand" Tags="|matlab|integration|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19655" PostTypeId="2" ParentId="943" CreationDate="2015-05-16T07:30:51.950" Score="4" Body="&lt;p&gt;Old question, but I think that &lt;a href=&quot;http://scicomp.stackexchange.com/questions/5372/for-which-statistical-methods-are-gpus-faster-than-cpus/12891#12891&quot;&gt;this answer from 2014&lt;/a&gt; - related to statistical methods, but generalisable for anyone who knows what a loop is - is particularly illustrative and informative.&lt;/p&gt;&#xA;" OwnerUserId="16367" LastEditorUserId="16367" LastEditDate="2021-01-30T01:05:37.720" LastActivityDate="2021-01-30T01:05:37.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19658" PostTypeId="2" ParentId="19500" CreationDate="2015-05-16T15:25:13.327" Score="1" Body="&lt;p&gt;Your problem can be described as a regression problem where not only the function values $y_i$ but also the points $x_i$ where the values are given are noisy. The classical approach to solve such a problem is called &lt;em&gt;total least squares&lt;/em&gt;, which basically amounts to fitting the &lt;em&gt;pairs&lt;/em&gt; $(x_i,y_i)$ using regular least squares (in a higher-dimensional space). The classical reference is &lt;a href=&quot;http://www.jstor.org/stable/2156807&quot; rel=&quot;nofollow&quot;&gt;Golub, van Loan: An analysis of the total least squares problem. SIAM J Numerical Analysis 17 (1980), 883-893&lt;/a&gt; (Charles van Loan's website has a -- rather poorly, but freely available -- &lt;a href=&quot;https://www.cs.cornell.edu/cv/ResearchPDF/Analysis.total.least.squares.prob.pdf&quot; rel=&quot;nofollow&quot;&gt;scanned version&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a simple regression problem, total least squares can be formulated as minimizing&#xA;$$\sum_{i=1}^n\frac{d_i^2}{1+\|x\|_2^2}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, total least squares applies to finding a vector $x\in\mathbb{R^n}$ satisfying $Ax=b$ for a given matrix $A\in\mathbb{R}^{m\times n}$  and vector $b\in \mathbb{R}^m$ such that $Ax=b$. (Here, $x$ could contain the coefficients of a regression polynomial, $A$ the polynomial basis evaluated in the given regression points and $b$ the given values.) If both $A$ and $b$ are contain errors, you cannot have $Ax=b$, but there are some (unknown) $E$ and $e$ such that $(A+E)x=(b+e)$. Total least squares then amounts to solving&#xA;$$\min_{E,e} \|E\|_F^2 +\|b\|_2^2 \quad\text{subject to}\quad (A+E)x=(b+e),$$&#xA;where $\|E\|_F$ is the Frobenius norm. This is a standard least squares problem for the extended matrix $\hat E:=[E|e]\in \mathbb{R}^{(n+1)\times m}$, which can be solved (if it can be solved at all; this is not always the case) using the singular value decomposition of $\hat E$: If $v_{n+1}\in\mathbb{R}^{n+1}$ is a singular vector corresponding to the smallest singular value $\sigma_{n+1}$ with $v_{n+1} = [y^T|\alpha]^T$ for some $y\in\mathbb{R}^n$ and $\alpha\in\mathbb{R}$, then the solution is given by&#xA;$$ x = -\frac{1}{\alpha} y.$$&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-05-16T15:25:13.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19659" PostTypeId="1" AcceptedAnswerId="19661" CreationDate="2015-05-16T16:52:23.483" Score="0" ViewCount="2527" Body="&lt;p&gt;I have been trying for 2-3 days now to get L2 regularized logistric regression to work in Matlab (CVX) and Python(CVXPY) but no success. I am fairly new to convex optimization so I am quite frustrated. Following is the equation that I am trying to solve using CVX/CVXPY. I have taken this equation from the paper &#xA;&lt;a href=&quot;https://intentmedia.github.io/assets/2013-10-09-presenting-at-ieee-big-data/pld_js_ieee_bigdata_2013_admm.pdf&quot; rel=&quot;nofollow&quot;&gt;https://intentmedia.github.io/assets/2013-10-09-presenting-at-ieee-big-data/pld_js_ieee_bigdata_2013_admm.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the case of L2 regularized logistic regression the problem becomes:&#xA;  $$ \text{minimize} \frac{1}{m}\sum_{i=1}^{m}\log[1 + \exp(-b_i\mathbf{A}_i^Tx)] + \lambda\Vert x\Vert_2^2$$&#xA;  where $\lambda$ is the regularization factor.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My Matlab (CVX) code is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function L2&#xA;m = 800; N = 5;&#xA;lambda =0.000001;&#xA;&#xA;A = load('/path/to/training/file'); &#xA;b= A(:,6); //Label Matrix (800x1)&#xA;A = A(:,1:5); //Feature matrix (800x5)&#xA;&#xA;cvx_begin&#xA;    variable x(N)&#xA;    minimize( (1/m * sum( log(1+ exp(-1* A' * (b * x')) ) ) ) + lambda*(norm(x,2)))&#xA;&#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;CVX returns an error saying which makes sense but the paper mentions the above equation. How can I solve it ?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Your objective function is not a scalar.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;After trying on Matlab, I tried on CVXPY. Here is the python code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from cvxopt import solvers, matrix,log, exp,mul&#xA;from cvxopt.modeling import op,variable&#xA;import numpy as np&#xA;&#xA;n = 5&#xA;m=800&#xA;data = np.ndarray(shape=(m,n), dtype=float,)&#xA;bArray = []&#xA;&#xA;file = open('/path/to/training/file')&#xA;&#xA;i = 0;&#xA;j=0;&#xA;for line in file:&#xA;    for num in line.split():&#xA;        if(j==5):&#xA;            bArray.append(float(num))&#xA;        else:&#xA;            data[i][j] = num&#xA;            j = j + 1&#xA;&#xA;    j=0&#xA;    i = i + 1&#xA;&#xA;A = matrix(data)&#xA;b_mat= matrix(bArray)&#xA;m, n = A.size&#xA;&#xA;&#xA;lamb_default = 0.000001&#xA;&#xA;x=variable(n)&#xA;&#xA;b = -1*b_mat&#xA;w = exp(A.T*b*x)&#xA;f = (1/m) + sum(log(1+w)) + lamb_default*mul(x,x)&#xA;&#xA;lp1 = op(f)&#xA;lp1.solve()&#xA;lp1.status&#xA;print(lp1.objective.value())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the error&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TypeError: incompatible dimensions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, my question is:&#xA;What am I doing wrong in the code for calculation of L2 problem in CVX/CVXPY ?&lt;/p&gt;&#xA;" OwnerUserId="15268" LastEditorUserId="123" LastEditDate="2015-05-18T04:43:17.897" LastActivityDate="2015-05-18T11:42:16.463" Title="How do I correctly multiply vectors and matrices in Python and MATLAB?" Tags="|python|regression|numpy|cvx|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19660" PostTypeId="1" CreationDate="2015-05-16T20:19:20.003" Score="3" ViewCount="942" Body="&lt;p&gt;I am a beginner with Unix and MPI. For the serial program, I run the program by typing &lt;code&gt;make&lt;/code&gt; in &lt;code&gt;/directory/buildagain&lt;/code&gt; and then &lt;code&gt;./runPR.sh values.txt&lt;/code&gt;. (&lt;code&gt;values.txt&lt;/code&gt; just contains the line &lt;code&gt;Chain&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a small portion of my code. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; int main( int argc, char *argv[ ] )&#xA; {&#xA;      if( strcmp(argv[1],&quot;-PR&quot;) == 0 )&#xA;           runPR(argc-2, &amp;amp;argv[2]);&#xA;      return 0;&#xA; }&#xA;&#xA; int runPR(int argc, char* argv[])&#xA; { &#xA;      cout&amp;lt;&amp;lt;&quot;run here&quot;&amp;lt;&amp;lt;endl;&#xA;&#xA;      MPI_Status status;&#xA;      PetscMPIInt    rank,size;&#xA;&#xA;      //bunch of code that assigns argv2 to include argv and some new strings and sets &#xA;      //argc2=argc plus the  number of new string&#xA;      PetscInitialize(&amp;amp;argc2, &amp;amp;argv2,0,0);&#xA;&#xA;      MPI_Comm_size(PETSC_COMM_WORLD,&amp;amp;size);&#xA;      MPI_Comm_rank(PETSC_COMM_WORLD,&amp;amp;rank);&#xA;      PetscPrintf(PETSC_COMM_WORLD,&quot;Number of processors = %d, rank = %d\n&quot;,size,rank);&#xA;      PetscSynchronizedPrintf(PETSC_COMM_WORLD,&quot;synchronized rank = %d\n&quot;,rank);&#xA;      PetscSynchronizedFlush(PETSC_COMM_WORLD);//,PETSC_STDOUT);&#xA;&#xA;      MPI_Barrier(PETSC_COMM_WORLD);&#xA;&#xA;      PetscPrintf(PETSC_COMM_SELF,&quot;[%d] Jumbled Hello World\n&quot;,rank);&#xA;&#xA;      PROpt opt;&#xA;      Solve* ps = new Solve();&#xA;&#xA;      PROpt opt;&#xA;      Solve* ps = new Solve();&#xA;      ps-&amp;gt;getFile(&amp;amp;opt,argv[0]);&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By typing &lt;code&gt;mpirun -np 4 ./runPR.sh values.txt&lt;/code&gt;, I see &lt;code&gt;Number of processors = 1, rank = 0&lt;/code&gt; and &#xA;&lt;code&gt;synchronized rank = 0&lt;/code&gt; printed 4 times. But I don't see &lt;code&gt;synchronized rank = 1&lt;/code&gt; or 2 or 3 printed out. Why is this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what &lt;code&gt;run.PR.sh&lt;/code&gt; contains&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; DIR=/directory/buildagain/bin/Project&#xA; FILELIST=$1&#xA;&#xA; while read FILE&#xA; do&#xA;       echo &quot;Processing ${FILE}...&quot;&#xA;          ./makeInp.sh ${FILE} ${FILE} &amp;gt;INP/${FILE}.inp&#xA;      mpiexec -np 4 ${DIR} -PR INP/${FILE}.inp&#xA; done &amp;lt; ${FILELIST}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT: I was able to get the helloworld PETSc example to work properly at &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/sys/examples/tutorials/ex2.c.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-current/src/sys/examples/tutorials/ex2.c.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16372" LastEditorUserId="16372" LastEditDate="2015-05-16T20:26:07.343" LastActivityDate="2015-10-14T03:38:35.997" Title="Why is MPI/PETSc only showing 1 processor even though it works for helloworld?" Tags="|parallel-computing|c++|petsc|mpi|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19661" PostTypeId="2" ParentId="19659" CreationDate="2015-05-16T21:25:09.460" Score="3" Body="&lt;p&gt;David Ketcheson has already indicated the problem in his comment. I will flesh it out here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the form of the argument of the log in the logistic regression objective:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$log[1+exp(−b_iA^T_ix)]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $b_i$ and $x$ are vectors and $A_i$ is a matrix. Only in the above order of matrix-vector multiplications will you get a scalar as an exponent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Matlab code should be (thanks for the correction OP):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;log(1 + exp(-1*b.*(A * x)))      &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will be a vector which will work with the following sum:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sum(log(1 + exp(-1*b.*(A * x))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Python code could be (updated after OP's comment):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;log_vec = [log(1+ exp(b[i]*A[i,:].T*x)) for i in range(n)] &#xA;f = (1/m) + sum(log_vec) + lamb_default*mul(x,x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This list comprehension will work for numpy matrix type. Make sure it works for matrix type of cvxopt.&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2015-05-18T11:42:16.463" LastActivityDate="2015-05-18T11:42:16.463" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19662" PostTypeId="2" ParentId="19660" CreationDate="2015-05-17T01:00:30.807" Score="1" Body="&lt;p&gt;I can't tell from this code directly, but this behavior is typical of code that never called MPI_Init(). I know that PetscInitialize() is supposed to do that for you, but I'm guessing that since you appear to have failed to assign argv2 and argc2 to anything, that something went wrong during initialization. I'm guessing that this code will fail to compile as is, since argc2 and argv2 aren't even defined in this example.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-17T01:00:30.807" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19663" PostTypeId="1" AcceptedAnswerId="19664" CreationDate="2015-05-17T03:17:33.450" Score="0" ViewCount="384" Body="&lt;p&gt;Let W be a graph Laplacian (symmetric diagonally dominant, and thus PSD), and X the matrix variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $&amp;lt;A,B&amp;gt;=Tr(A^TB)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\text{Maximize}\;\; \displaystyle\sum_{i,j} w_{ij}(x^{(i)}\cdot x^{(j)})$$&#xA;$$\text{Subject to}\;\; x^{(i)}\cdot x^{(i)}=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to write this in standard semidefinite programming form.  The best I've been able to do is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\text{Maximize}\;\; &amp;lt;W, X^TX&amp;gt;$$&#xA;$$\text{Subject to}\;\; &amp;lt;A_i, X^TX&amp;gt;=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_i$ is a matrix of zeros except for a $1$ at the $i^{th}$ slot on its diagonal, and where $X^TX$ is automatically PSD.  If I replace $X^TX$ with $S$ in the above SDP and require that it be PSD, then I can clearly optimize with $S$ as my variable, but there may be no $X^{*}$ which corresponds to $S^{*}$, and if there is, it's not clear how I would recover it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I'm curious what about the Laplacian ensures the convexity of $\sum_{i,j} w_{ij}(x^{(i)}\cdot x^{(j)})$, is it just the fact that it's PSD, or does it need to be diagonally dominant?&lt;/p&gt;&#xA;" OwnerUserId="6849" LastActivityDate="2015-05-17T04:42:00.390" Title="How to write Goemans-Williamson MAX-CUT relaxation as SDP" Tags="|convex-optimization|semidefinite-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19664" PostTypeId="2" ParentId="19663" CreationDate="2015-05-17T04:24:33.320" Score="2" Body="&lt;p&gt;A common definition of &quot;positive semidefinite&quot; is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A real symmetric $n$ by $n$ matrix $S$ is positive semidefinite if and only if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z^{T}Sz \geq 0$ for all $z \in R^{n}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another equivalent definition is that a real symmetric $n$ by $n$ matrix $S$ is positive semidefinite if and only if all of its eigenvalues are &#xA;non-negative.  I won't go through the proof that these two definitions are equivalent (it's in lots of textbooks.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third equivalent definition that is not quite so well known is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A real symmetric $n$ by $n$ matrix $S$ is positive semidefinite if and only if there is a real $n$ by $n$ matrix $X$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S=X^{T}X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To show that that this last definition is equivalent, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $S$ is positive semidefinite, then we diagonalize $S$ as  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S=U \Lambda U^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is orthogonal and $\Lambda$ is diagonal with non-negative elements on the diagonal.  By taking the square roots of these non-negative diagonal elements, we can construct a diagonal matrix square root of $\Lambda$, $\Lambda^{1/2}$.   We can then write $S$ as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S=U \Lambda^{1/2} U^{T} U \Lambda^{1/2} U^{T}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $X=U \Lambda^{1/2} U^{T}$.  Note that $X$ is symmetric.  Thus &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S=XX=X^{T}X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computationally, we can get the matrix square root $X$ from $S$ by computing the eigenvalue decomposition of $S$.  In MATLAB, there's a &lt;em&gt;sqrtm()&lt;/em&gt; function that does the work for you.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, suppose that $S$ is a real and symmetric $n$ by $n$ matrix with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S=X^{T}X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then for any $z \in R^{n}$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z^{T}Sz=z^{T}X^{T}Xz=\| Xz \|_{2}^{2} \geq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus $S$ is positive semidefinite.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your last question, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&amp;lt;W, S&amp;gt;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is linear in the elements of $S$ and is thus a convex (and concave) function of $S$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-05-17T04:42:00.390" LastActivityDate="2015-05-17T04:42:00.390" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19665" PostTypeId="1" AcceptedAnswerId="19666" CreationDate="2015-05-17T05:04:23.063" Score="4" ViewCount="1017" Body="&lt;p&gt;I have an equation $Ax=b$ for which I need to solve for numerous $x$ matrices given $b$. Both $x$ and $b$ are nx1 matrices. Unfortunately, $A$ is a 32x32 matrix and inversion gives highly unstable values. I am wondering what the appropriate algorithm/technique is for obtaining a stable $A$ matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My searches for an alternative calculation approach or way to restore stability to obtain a solution have so far led me nowhere so I would appreciate any input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jason&lt;/p&gt;&#xA;" OwnerUserId="16321" LastActivityDate="2015-05-19T11:09:56.843" Title="Numerically stable approach for calculating x in Ax=b" Tags="|matrix|stability|inverse-problem|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19666" PostTypeId="2" ParentId="19665" CreationDate="2015-05-17T15:40:25.277" Score="13" Body="&lt;p&gt;If the solution of $Ax=b$ is unstable, the matrix is very ill-conditioned (i.e., has a very large condition number), and (paraphrasing Lanczos) no amount of mathematical trickery can make it stable. The best you can hope for is to solve a &lt;em&gt;different&lt;/em&gt; problem that is a) stable and b) gives you a solution that is sufficiently close; this is called &lt;em&gt;regularization&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For linear ill-conditioned problems, there are two classical approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Tikhonov regularization&lt;/em&gt; replaces $Ax=b$ by the stabilized least-squares problem &#xA;$$ \min_x \|Ax-b\|^2 + \alpha \|x\|^2$$&#xA;for some regularization parameter $\alpha&amp;gt;0$. The minimizer can of course be computed by solving the normal equations&#xA;$$ (A^TA+\alpha I)x_\alpha = A^Tb.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Truncated singular value decomposition&lt;/em&gt; computes the singular value decomposition&#xA;$$ U \Sigma V^T = A,$$&#xA;where the columns of $U$ and $V$ contain the left and right singular vectors, respectively, and $\Sigma$ is a diagonal matrix containing the singular values (usually in order of descending magnitude). Since $U$ and $V$ are unitary, the inverse of $A$ (if it exists) is given by $A^{-1}=V\Sigma^{-1} U^T$. Ill-conditionedness of $A$ manifests in the existence of very small singular values, such that the corresponding entry in $\Sigma^{-1}$ would be very large, amplifying small perturbations (e.g., due to finite numerical precision). As the name indicates, stability is restored by ignoring these small entries, i.e., replacing $\Sigma^{-1}$ by&#xA;$\Sigma_{\alpha}^{-1}$ where&#xA;$$[\Sigma_{\alpha}^{-1}]_{ii} = \begin{cases} \Sigma_{ii}^{-1} &amp;amp; \text{if } |\Sigma_{ii}| &amp;gt; \alpha \\ 0 &amp;amp;\text{else}&#xA;\end{cases}$$&#xA;and setting &#xA;$$x_\alpha = V\Sigma_{\alpha}^{-1} U^T b.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In both cases, you need to choose $\alpha$ specifically for your problem to get good results. You could start by computing the SVD of a representative $A$ and looking at the singular values to see if there's a clear threshold.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-05-17T15:40:25.277" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19667" PostTypeId="2" ParentId="11285" CreationDate="2015-05-17T16:43:57.937" Score="0" Body="&lt;p&gt;Did you consider implementing the algebraic MSFV instead? The implementation complexity is a lot less. Details can be found in the paper &quot;An operator formulation of the multiscale finite-volume method with correction function&quot; by Lunati, 2009.&lt;/p&gt;&#xA;" OwnerUserId="16379" LastEditorUserId="549" LastEditDate="2015-05-18T03:14:06.990" LastActivityDate="2015-05-18T03:14:06.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19668" PostTypeId="1" AcceptedAnswerId="19703" CreationDate="2015-05-17T16:53:25.257" Score="1" ViewCount="118" Body="&lt;p&gt;I am going to rewrite an algorithm using genetic methods for mutating my solutions, and I am wondering what I should expect and only consider my algorithm &quot;optimized&quot; or &quot;finished&quot; if various runs of the algorithm give the same result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I run the program once and get a good solution, &quot;A&quot;, but the second time I run it and get a good solution, &quot;B&quot;, is this expected behavior for an adequately designed genetic algorithm? Or should I only get a solution with minor variations of the same solution on each run?&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="625" LastEditDate="2015-05-25T15:23:16.240" LastActivityDate="2015-05-25T15:23:16.240" Title="Should a randomly-seeded genetic algorithm give deterministic optimized solutions on each run?" Tags="|algorithms|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19669" PostTypeId="2" ParentId="19668" CreationDate="2015-05-17T20:26:59.103" Score="3" Body="&lt;p&gt;Your optimization problem might well have multiple isolated but globally optimal solutions.  It might even have infinitely many optimal solutions.  You really shouldn't be surprised if you get different optimal solutions from different runs.  If the optimal objective values from some runs are not as good as in other runs, then this is a sign that you can't trust the result of any one run.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-05-17T20:26:59.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19671" PostTypeId="2" ParentId="3540" CreationDate="2015-05-18T04:14:27.687" Score="0" Body="&lt;p&gt;I know this is an old question, but I've just been struggling with a similar issue and thought I'd post my solution incase anyone else comes across this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the problem was that I hadn't properly initialised the $ \Theta $ vector to break the symmetry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After doing this, I ran &lt;code&gt;fmin_cg&lt;/code&gt; with f &lt;strong&gt;and&lt;/strong&gt; fprime, and although it still ran rather slowly, it did so without any of the errors or warnings I had been seeing up to that point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, it actually did a better job of minimising the cost function than the matlab implementation, despite both having &lt;code&gt;max_iters = 100&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16383" LastActivityDate="2015-05-18T04:14:27.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19673" PostTypeId="2" ParentId="11285" CreationDate="2015-05-18T06:33:45.833" Score="1" Body="&lt;p&gt;I'm not familiar with the work by Jenny and Lunati, so this answer might be wrong.  But if I understand correctly, the geometry you want can be generated in a few lines using PyClaw; see &lt;a href=&quot;http://nbviewer.ipython.org/gist/ketch/1a7888d1fcc37209b260&quot; rel=&quot;nofollow noreferrer&quot;&gt;this notebook&lt;/a&gt;.  This figure shows the primary and dual grids:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/b4624.png&quot; alt=&quot;Grid&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use mapped grids, and ghost cells are automatically generated if you need them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/exRPf.png&quot; alt=&quot;mapped_grid&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If either of those is what you're looking for, I can post all the necessary code here inline (it's in the notebook linked above).&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-05-18T06:33:45.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19674" PostTypeId="1" CreationDate="2015-05-18T09:13:48.957" Score="1" ViewCount="389" Body="&lt;p&gt;I am trying to plot spherical particles in a cube of fixed dimension in matlab. I face a problem here where the center of the sphere is too close to the edge of the cube in this case the rest of the sphere needs to be plotted in the opposite side. I hope I am clear. How can I do this&lt;/p&gt;&#xA;" OwnerUserId="16386" LastEditorUserId="1744" LastEditDate="2015-05-18T15:34:45.623" LastActivityDate="2015-05-18T15:34:45.623" Title="Satisfying Periodic Boundary Conditions while plotting spherical particles inside a cube" Tags="|matlab|boundary-conditions|plotting|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19675" PostTypeId="1" CreationDate="2015-05-18T10:25:33.987" Score="0" ViewCount="443" Body="&lt;p&gt;I simulated nanowire deformation using LAMMPS and visualized it using VMD in Centos 6.5. But I'm not able to generate .jpg image file or the simulation movie. How can I proceed??&lt;/p&gt;&#xA;" OwnerUserId="16388" LastActivityDate="2015-05-18T12:26:30.317" Title="How to generate movie of the simulation done in LAMMPS?" Tags="|simulation|molecular-dynamics|" AnswerCount="1" CommentCount="1" ClosedDate="2015-05-20T00:52:07.540" ContentLicense="CC BY-SA 3.0" />
  <row Id="19676" PostTypeId="2" ParentId="19675" CreationDate="2015-05-18T12:26:30.317" Score="1" Body="&lt;p&gt;You provide very little detail as to 'not able' means. I suppose you have error in your lammps output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check the page &lt;a href=&quot;http://lammps.sandia.gov/doc/dump_image.html&quot; rel=&quot;nofollow&quot;&gt;http://lammps.sandia.gov/doc/dump_image.html&lt;/a&gt; that gives instructions. Most importantly, check that you compiled lammps with the appropriate options ( -DLAMMPS_JPEG being the most obvious )&lt;/p&gt;&#xA;" OwnerUserId="16340" LastActivityDate="2015-05-18T12:26:30.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19677" PostTypeId="2" ParentId="19674" CreationDate="2015-05-18T15:28:32.897" Score="3" Body="&lt;p&gt;A quick and dirty solution is to plot a &lt;em&gt;phantom&lt;/em&gt; sphere outside the cube on the opposite side, and then apply clipping to the cube boundary.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2015-05-18T15:28:32.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19678" PostTypeId="1" AcceptedAnswerId="19679" CreationDate="2015-05-18T19:27:21.753" Score="8" ViewCount="5377" Body="&lt;p&gt;I have seen in some papers that the energy levels in some arbitrary potential are specified. How can one find the energy levels in such arbitrary potentials. For example, $V(x)=\sin^2(x/2)$ with $x\in[−3\pi,3\pi]$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that in a &lt;code&gt;triple well potential&lt;/code&gt;, there will be 3 bands comprising of three energy levels. But how do we find the exact value of these energy levels. I would like to know about the numerical methods used for this. It 'd be nice if you prefer Fortran or C, since I am not good in Python.&lt;/p&gt;&#xA;" OwnerUserId="14801" LastEditorUserId="14801" LastEditDate="2015-05-19T08:16:22.777" LastActivityDate="2021-08-23T22:12:43.747" Title="Computational methods for finding the energy eigenvalues of the time-independent Schrodinger equation with arbitrary potential" Tags="|pde|eigenvalues|quantum-mechanics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19679" PostTypeId="2" ParentId="19678" CreationDate="2015-05-18T21:20:20.397" Score="15" Body="&lt;p&gt;Finding the eigenvalues for the Schrödinger equation is really similar to finding the eigenvalues for the wave equation. You start with your differential equation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left[-\frac{1}{2}\nabla'^2 + V(r)\right]\psi(\mathbf{r}) = E' \psi(\mathbf{r})$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where we did the change of variable &lt;span class=&quot;math-container&quot;&gt;$(x,y,z) \rightarrow (a_0 x, a_0 y, a_0 z)$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$a_0 \equiv 1$&lt;/span&gt; Bohr, &lt;span class=&quot;math-container&quot;&gt;$E'= E/E_0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E_0 \equiv \frac{\hbar^2}{m_e a_0^2}$&lt;/span&gt;. The equation is written in a system of units that is more suitable for numerics (&lt;span class=&quot;math-container&quot;&gt;$m_e=1, \hbar=1, e=1$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The more straightforward approach is to discretize the operator with &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite differences&lt;/a&gt; and find the eigenvalues. An example for the &lt;a href=&quot;http://en.wikipedia.org/wiki/Particle_in_a_box&quot; rel=&quot;nofollow noreferrer&quot;&gt;particle in a box&lt;/a&gt;, written in Python is here:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from __future__ import division, print_function&#xA;import numpy as np&#xA;from scipy.sparse import diags&#xA;from scipy.sparse.linalg import eigsh&#xA;&#xA;L = 2&#xA;N = 1000&#xA;dx = L/(N-1)&#xA;&#xA;H = diags([-2., 1., 1.], [0,-1, 1], shape=(N, N))/dx**2&#xA;&#xA;vals, vecs = eigsh(-0.5*H, which='SA')&#xA;&#xA;print(np.round(vals, 6))&#xA;print(np.round([(n*np.pi/L)**2/2 for n in range(1,6)], 6))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where the results are&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[  1.228775   4.915086  11.058899  19.660151  30.71876   44.234615]&#xA;[  1.233701   4.934802  11.103305  19.739209  30.842514  44.41322 ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The analytic solution for this normalized case is &lt;span class=&quot;math-container&quot;&gt;$E' = \frac{n^2 \pi^2}{2L^2}$&lt;/span&gt;. That is printed in the second line, the agreement is &amp;quot;adequate&amp;quot;, and if we discretize more, the error is smaller.&lt;/p&gt;&#xA;&lt;p&gt;In the case of an &amp;quot;arbitrary&amp;quot; potential, we can do the same, just adding the potential energy&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from __future__ import division, print_function&#xA;import numpy as np&#xA;from scipy.sparse import diags&#xA;from scipy.sparse.linalg import eigsh&#xA;import matplotlib.pyplot as plt&#xA;&#xA;L = 2*np.pi&#xA;N = 501&#xA;x = np.linspace(0, L, N)&#xA;dx = x[1] - x[0]&#xA;&#xA;T = -0.5*diags([-2., 1., 1.], [0,-1, 1], shape=(N, N))/dx**2&#xA;U_vec = np.sin(x)**2&#xA;U = diags([U_vec], [0])&#xA;&#xA;H = T + U&#xA;&#xA;vals, vecs = eigsh(H, which='SM')&#xA;&#xA;print(np.round(vals, 6))&#xA;&#xA;for k in range(4):&#xA;    vec = vecs[:, k]&#xA;    mag = np.sqrt(np.dot(vecs[:, k],vecs[:, k]))&#xA;    vec = vec/mag&#xA;    plt.plot(x, vec, label=r&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$n=%i$&lt;/span&gt;&amp;quot;%k)&#xA;&#xA;plt.xlabel(r&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;&amp;quot;)&#xA;plt.ylabel(r&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$\psi(x)$&lt;/span&gt;&amp;quot;)&#xA;plt.savefig(&amp;quot;eigenvectors.png&amp;quot;, dpi=600)&#xA;plt.legend()    &#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with results&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[ 0.545762  1.227567  1.657063  2.470791  3.610032  4.970446]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That I have no idea if are good or not. The eigenvectors make sense, though&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/JKQ3F.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Comparing to the Ritz method (with 10 sine functions)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[0.4111, 0.8299, 1.6696, 2.83472, 4.3342, 6.16726]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it makes sense&lt;/p&gt;&#xA;&lt;p&gt;Another option is to base your formulation on a variational approach. In this case, you start from the variational formulation of the problem, i.e.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \Pi[u] = \int_\Omega\left[\frac{1}{2}\nabla u^*(\mathbf{r}) \cdot\nabla u(\mathbf{r})  + u^*(\mathbf{r})(V(\mathbf{r})-E)u(\mathbf{r})\right] d\Omega$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;That has as components, the Hamiltonian&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H[u] = \int_\Omega\left[\frac{1}{2}\nabla u^*(\mathbf{r}) \cdot\nabla u(\mathbf{r})  + u^*(\mathbf{r})V(\mathbf{r})u(\mathbf{r})\right] d\Omega$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the overlap operator&#xA;&lt;span class=&quot;math-container&quot;&gt;$$S[u] = \int_\Omega\left[u^*(\mathbf{r})u(\mathbf{r})\right] d\Omega \enspace .$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The solution for this system is at a stationary &lt;em&gt;point&lt;/em&gt; of the functional. For an approximate solution of the form &lt;span class=&quot;math-container&quot;&gt;$\hat{u} = \sum_i^N c_i f_i(x)$&lt;/span&gt;, we obtain an approximation of the functionals. And we can form the eigenvalue problem&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$[\hat{H}]\lbrace c\rbrace = E [\hat{S}]\lbrace c\rbrace$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$\hat{H}_{ij} = \frac{H}{\partial c_i \partial c_j}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat{S}_{ij}=\frac{S}{\partial c_i \partial c_j} \enspace.$&lt;/span&gt; This approach is a Ritz method and it is similar to the one used in &lt;a href=&quot;http://en.wikipedia.org/wiki/Hartree%E2%80%93Fock_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hartree-Fock Methods&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Density_functional_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Density Functional Theory (DFT)&lt;/a&gt;, and also to &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_element_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite Element Methods&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I solved for a potential &lt;span class=&quot;math-container&quot;&gt;$V(x) = x(2\pi - x)$&lt;/span&gt; analytically using &lt;code&gt;Sympy&lt;/code&gt; &lt;a href=&quot;http://nbviewer.ipython.org/github/nicoguaro/notebooks_examples/blob/master/quantum_mechanics_simple.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, with eigenvalues&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[  5.72324464,   9.04413044,  11.55005606,  14.95945642,  19.85359933]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using 20 functions I found these values (not in Sympy though)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; [5.717086647332907,9.033168327518542,11.52268082000659,14.84436075114633,19.25485456061611]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;that are really close.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2021-08-23T22:12:43.747" LastActivityDate="2021-08-23T22:12:43.747" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19680" PostTypeId="1" AcceptedAnswerId="19684" CreationDate="2015-05-19T11:03:40.903" Score="0" ViewCount="103" Body="&lt;p&gt;I am currently working on a CFD code over a cubic grid. Now, the number of elements used in the simulation is decomposed among the number of processors. Each of those processors (a section of the cube) is then divided up in a number of elements that corresponds to the polynomial order. I would like to understand the mathematical reason of doing so.&lt;img src=&quot;https://i.stack.imgur.com/PWZZV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3607" LastEditorUserId="3607" LastEditDate="2015-05-19T11:09:04.573" LastActivityDate="2015-05-19T13:17:31.140" Title="Representation of polynomial order in CFD codes" Tags="|fluid-dynamics|polynomials|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19681" PostTypeId="1" AcceptedAnswerId="19706" CreationDate="2015-05-19T11:05:39.320" Score="0" ViewCount="300" Body="&lt;p&gt;I am a beginner in adaptive mesh refinement (AMR). After I am done with the first two papers by Dr. Marsha Berger, I was trying to write my own code for a problem which has a parabolic partial differential equation. In this regard, I wish to know if I could use the same trick as described by Dr. Berger to this problem? Because she dealt with hyperbolic PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to simulate a two dimensional heat conduction problem of which governing equation is just a Laplace equation and temperature at a nodal point average of neighboring four nodal points.&lt;/p&gt;&#xA;" OwnerUserId="16397" LastEditorUserId="9667" LastEditDate="2015-05-19T19:09:28.780" LastActivityDate="2015-05-21T04:39:11.470" Title="Adaptive mesh refinement basic conceptual problem" Tags="|mesh|parabolic-pde|adaptive-mesh-refinement|fluid-dynamics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19684" PostTypeId="2" ParentId="19680" CreationDate="2015-05-19T13:17:31.140" Score="3" Body="&lt;p&gt;What you have is most likely a mesh for a &lt;a href=&quot;http://en.wikipedia.org/wiki/Spectral_element_method&quot; rel=&quot;nofollow&quot;&gt;Spectral Element Method&lt;/a&gt; or something similar). The solution is represented using a high order polynomial each section of the cube (referred to as an element). For convenience, these polynomials are represented as Lagrange nodal functions, which are associated with a set of points on each element (which are denoted by the vertices of the sub-mesh on each cube).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;High order methods tend to be more accurate than a low order method, at least when measured in terms of error as a function of the element size, and are fairly simple to make efficient on modern hardwares. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-05-19T13:17:31.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19685" PostTypeId="1" CreationDate="2015-05-19T13:33:05.980" Score="10" ViewCount="1360" Body="&lt;p&gt;Hyperbolic space in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Poincar%C3%A9_half-plane_model&quot; rel=&quot;noreferrer&quot;&gt;Poincaré upper half space model&lt;/a&gt; looks like ordinary &lt;span class=&quot;math-container&quot;&gt;$\Bbb R^n$&lt;/span&gt; but with the notion of angle and distance distorted in a relatively simple way. In Euclidean space I can sample a random point uniformly in a ball in several ways, e.g. by generating &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; independent Gaussian samples to obtain a direction, and separately sample a radial coordinate &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; by uniformly sampling &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\left[0, \frac1{n+1}R^{n+1}\right]$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the radius, and setting &lt;span class=&quot;math-container&quot;&gt;$r = \left((n+1)s\right)^{\frac1{n+1}}$&lt;/span&gt;. In the hyperbolic upper half plane a sphere happens to still be a sphere, only its centre will not be the centre in the Euclidean metric, so we could do the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we want to sample according to a non-uniform distribution, but still in an isotropic way, e.g. a Gaussian distribution, this doesn't seem so easy. In Euclidean space we could just generate a Gaussian sample for each coordinate (this only works for the Gaussian distribution), or equivalently generate a multidimensional Gaussian sample. Is there a direct way to convert this sample to a sample in hyperbolic space? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative approach could be to first generate a direction uniformly distributed direction (e.g. from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; Gaussian samples) then a Gaussian sample for the radial component, and finally generate the image under the &lt;a href=&quot;https://en.wikipedia.org/wiki/Exponential_map_(Riemannian_geometry)&quot; rel=&quot;noreferrer&quot;&gt;exponential map&lt;/a&gt; in the specified direction for the specified length. A variation would be to just take the Euclidean Gaussian sample and map it under the exponential map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what would be a good and efficient way to obtain a Gaussian sample with given mean and standard deviation in hyperbolic space?&lt;/li&gt;&#xA;&lt;li&gt;do the ways I describe above provide the desired sampling?&lt;/li&gt;&#xA;&lt;li&gt;did anyone work out the formula's already?&lt;/li&gt;&#xA;&lt;li&gt;how does this generalize to other metrics and other probability distributions?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just realized that even in the case of uniform sampling these questions remain; even though a sphere is a sphere, a uniform distribution would not be described by a constant function on a ball.&lt;/p&gt;&#xA;" OwnerUserId="5521" LastEditorUserId="5521" LastEditDate="2019-06-05T21:58:19.513" LastActivityDate="2020-03-07T21:49:09.970" Title="How to sample points in hyperbolic space?" Tags="|computational-geometry|monte-carlo|geometry|random-sampling|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19686" PostTypeId="2" ParentId="19458" CreationDate="2015-05-19T15:16:30.173" Score="1" Body="&lt;p&gt;Long story short.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example the simplest diffusion equation $-\nabla\cdot(A\nabla u) = f$, with certain boundary condition to form up a BVP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose a finite dimensional approximation $u_h$ to $u$, is obtained by a discretization scheme, continuous Galerkin (CG), non-conforming (Crouzeix-Raviart), DG (IPDG, LDG, HDG, DPG, etc), or mixed formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we call this scheme conservative if the numerical flux $\boldsymbol{\sigma}_h:= -A\nabla u_h$ on each element satisfy the equilibrium equation $\nabla\cdot \boldsymbol{\sigma}_h = f_h$ &quot;in some sense&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CG and non-conforming are non-conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the DG schemes are conservative if the vector flux $\boldsymbol{\sigma}_h$ is formulated in an appropriate way. The conservativity means the equilibrium equation is satisfied in the integral sense on each element. But for DG, there are more things to consider like the consistency of $u_h$ and its &quot;scalar flux&quot; $\hat{u}_h$ defined on the skeleton of the mesh. For the summary, please refer to the paper by DN Arnold, F Brezzi, B Cockburn, LD Marini on SINUM, &lt;strong&gt;&lt;em&gt;Unified analysis of discontinuous Galerkin methods for elliptic problems&lt;/em&gt;&lt;/strong&gt;, page 1761-1762.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mixed methods are &quot;naturally&quot; conservative because the equilibrium equation is discretized at the same time with the constitutive equation.&lt;/p&gt;&#xA;" OwnerUserId="350" LastActivityDate="2015-05-19T15:16:30.173" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19688" PostTypeId="1" CreationDate="2015-05-19T17:42:01.163" Score="1" ViewCount="553" Body="&lt;p&gt;I am trying to solve the 1-dimensional diffusion problem numerically using method of lines:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial c}{\partial t} =D \frac{\partial^2 c}{\partial z^2},$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the right hand side is discretised and the left hand side is handled with a standard ode solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two boundary conditions. One boundary condition dynamically changes with every timestep (&lt;code&gt;dcdt[0]&lt;/code&gt;), while the other is always fixed at a constant value (&lt;code&gt;dcdt[-1]&lt;/code&gt;). There is also one initial condition (concentration profile) at time zero, which is the line in dark blue in the picture shown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, what happens is that &lt;code&gt;scipy.integrate.ode&lt;/code&gt; ignores both boundary conditions!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/7ycjV.png&quot; alt=&quot;Output&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def diffusion(t,c, h):&#xA;    dcdt = zeros(Nz)&#xA;    dcdt[0] = funcA(h)&#xA;    dcdt[-1] = B # a constant    &#xA;    for i in range(1,Nz-1):&#xA;        dcdt[i] = D * (c[i+1] - 2*c[i] + c[i-1]) / (dz*dz) &#xA;&#xA;    return dcdt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I use &lt;code&gt;scipy.integrate.ode&lt;/code&gt; to integrate the discretisation equation. I have defined the &lt;code&gt;diffusion&lt;/code&gt; function to set the boundary condition every time the &lt;code&gt;ode(diffusion)&lt;/code&gt; is called. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ode = integrate.ode(diffusion).set_integrator('vode')&#xA;ode.set_initial_value(c_initial, 0).set_f_params(h0)&#xA;k = 0&#xA;&#xA;while ode.successful() and k &amp;lt; Nt:&#xA;    dcdz = (y[1] - y[0]) / dz&#xA;    h[k] = h[k-1] + Q*(h[k-1]**2) / func(h[k-1]) * dt * dcdz&#xA;    c_out[:,k] = ode.y&#xA;    t[k] = ode.t&#xA;&#xA;    ode.set_f_params(h[k]) # Change the b.c.&#xA;    ode.integrate(ode.t + dt)&#xA;    k += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have checked the following things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;By inserting &lt;code&gt;print h&lt;/code&gt; into the &lt;code&gt;diffusion&lt;/code&gt; function I have checked that &lt;code&gt;set_f_params&lt;/code&gt; has successfully passed &lt;code&gt;h&lt;/code&gt; into the solver. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are no numerical instabilities. I have also checked other solvers &lt;code&gt;dopri5&lt;/code&gt;, stiff/non-stiff options, etc. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have switched the order of functions in the while loop around. It seems that the problem is internal, i.e. &lt;code&gt;ode.integrate(ode.t + dt)&lt;/code&gt; solves the equation and ignores the boundary conditions I want to impose before returning the output to me. I'm puzzled why it would do that. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;One thing I have thought about is that I could use &lt;code&gt;set_initial_value&lt;/code&gt; to set the boundary condition. But as far as I know, I have to provide a full solution vector $c(z,t)$ at a certain time, and &lt;code&gt;ode.integrate&lt;/code&gt; is already ignoring my boundary condition at time zero. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16399" LastEditorUserId="1804" LastEditDate="2015-05-19T18:05:24.787" LastActivityDate="2015-05-19T18:05:24.787" Title="scipy.integrate.ode ignores boundary conditions" Tags="|pde|python|time-integration|scipy|" AnswerCount="0" CommentCount="4" ClosedDate="2019-06-09T18:17:28.037" ContentLicense="CC BY-SA 3.0" />
  <row Id="19689" PostTypeId="1" AcceptedAnswerId="19690" CreationDate="2015-05-19T20:21:25.287" Score="4" ViewCount="4136" Body="&lt;p&gt;I am trying to solve the Navier Stokes equations using the finite element method. I plan on using the pressure correction method to deal with the pressure and an implicit time stepping scheme for diffusion and an explicit time stepping scheme for convection. My question is: How does one deal with the non-linear term using the finite element method? I have investigated a couple approaches. For example, I understand for steady problems it is common to use a Newton iteration. Since I am interested in unsteady problems I don't think this method is suitable. I have also seen an approach where part of the non-linear term is &quot;frozen&quot; at the last time step so that at each time step the problem appears as a linear advection problem, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{i}\frac{\partial{u_{i}}}{\partial{x_{j}}} \approx{} u_{i}^{n-1}\frac{\partial{u_{i}^{n}}}{\partial{x_{j}}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words the velocity at the previous time step ($u_{i}^{n-1}$) is treated as constant and no shape function expansion is used in approximating this term. Is this method any good? What methods should I use to deal with the non-linear term in the unsteady Navier Stokes equations where accuracy, ease of implementation, stability, and robustness to different Reynolds numbers are all considerations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="-1" LastEditDate="2015-05-20T06:08:01.943" LastActivityDate="2020-10-27T20:10:29.583" Title="How to deal with nonlinear term in Navier Stokes equations (finite element code)" Tags="|finite-element|fluid-dynamics|navier-stokes|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19690" PostTypeId="2" ParentId="19689" CreationDate="2015-05-20T01:07:56.307" Score="4" Body="&lt;p&gt;You can absolutely use the Newton method to linearize the system of equations that results after you have discretized in time. It's a pretty common approach. It might be overkill in lots of cases, but it's not inappropriate. Freezing like you suggest and not iterating at all within a timestep can lead to some pretty awful solutions unless your timestep is miniscule. But you don't want to do that since you are interested in implicit approaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Write the split equations, and then discretize in time to give a non-linear PDE in space. Then apply Newton's method. Then discretize in space with FEM. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-20T01:07:56.307" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19691" PostTypeId="2" ParentId="19689" CreationDate="2015-05-20T02:51:32.337" Score="2" Body="&lt;p&gt;Addendum to Bill's answer: a Newton iteration combined with implicit timestepping usually regularizes the problem enough to allow just one Newton iteration per timestep (for one example - &lt;a href=&quot;http://users.ices.utexas.edu/~benkirk/dissertation.pdf&quot; rel=&quot;nofollow&quot;&gt;http://users.ices.utexas.edu/~benkirk/dissertation.pdf&lt;/a&gt;).  With a decent preconditioner, this can be competitive with pressure projection if the timescale of your physics isn't too small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your question: if you're doing unsteady flows, the nonlinear term is usually dealt with in an explicit manner, as it's roughly equivalent to a convective term.  So long as you treat the nonlinear properly as a linear convection equation &lt;em&gt;with non-constant convection&lt;/em&gt; (i.e. aliasing, higher order quadratures, etc), this is usually fine. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-05-20T02:51:32.337" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19692" PostTypeId="2" ParentId="1207" CreationDate="2015-05-20T04:06:43.303" Score="0" Body="&lt;h3&gt;list of XSEDE software&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The XSEDE network of NSF-funded supercomputers has a &lt;a href=&quot;https://portal.xsede.org/software#/&quot; rel=&quot;nofollow&quot;&gt;list of available software&lt;/a&gt;, and you can search by scientific domain or application. However, you will have to google the software names to learn more.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Software Carpentry Foundation member projects&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The Software Carpentry Foundation maintains a &lt;a href=&quot;https://software-carpentry.org/pages/projects.html&quot; rel=&quot;nofollow&quot;&gt;list of projects&lt;/a&gt; led by its members. Many of these are hosted on GitHub or similar and are open to contributors. A few are tools for HPC simulation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Disclosure&lt;/em&gt;: I am a volunteer Software Carpentry Foundation instructor, and my project is listed on the projects page. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Plug / Public Service Announcement:&lt;/em&gt; if you are interested in contributing to open source scientific software, I'd suggest that you read &lt;a href=&quot;http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001745&quot; rel=&quot;nofollow&quot;&gt;&quot;Best Practices in Scientific Computing&quot; by Wilson et al 2014&lt;/a&gt; and consider attending a workshop run by the Software Carpentry Foundation - and if/ when you are ready, instructor training. &lt;/p&gt;&#xA;" OwnerUserId="452" LastEditorUserId="452" LastEditDate="2015-05-20T18:04:47.810" LastActivityDate="2015-05-20T18:04:47.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19693" PostTypeId="1" CreationDate="2015-05-20T04:38:37.803" Score="2" ViewCount="2699" Body="&lt;p&gt;I am trying to fit a restricted cubic spline (natural cubic spline) with 4 knots to toy data, attempting to follow&#xA;Hastie, Tibshirani, Friedman 2nd ed. 5.2.1 p.144-146, Eqs 5.4 and 5.5. &#xA;Data: Is basically a transposed ‘S’ shape. R-code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n &amp;lt;© 100&#xA;x &amp;lt;- (1:n)/n&#xA;true &amp;lt;- ((exp(1.2*x)+1.5*sin(7*x))-1)/3&#xA;noise &amp;lt;- rnorm(n, 0, 0.15)&#xA;y &amp;lt;- true + noise&#xA;plot(x,y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I set knots at:   {.2, .4, .6, .8} and am fitting using the non-linear NLS() function in R, but I can’t get the S-shape of the data no matter what I try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My equations is wrong ? Or I am completely off-base in my approach? Any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Book-excerpt, my equation, and data-plot posted below)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/NvHAO.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A natural cubic spline with $K$ knots is represented by $K$ basis functions. One can start from a basis for cubic splines, and derive the reduced basis by imposing the boundary constraints. For example, starting from the truncated power series basis described in section 5.2, we arrive at&#xA;  where&#xA;      $$d_k=\frac{(X-\xi_k)^3_+ - (X-\xi_K)^3_+}{\xi_K - \xi_k} \enspace .$$&#xA;  Each of these basis functions can be seend to have zero second and third derivative for $X\geq \xi_K$.&#xA;  $$y = \beta_0 + \beta_1 x + \beta_2\left(\left[\frac{(x-k_1)^3_+ - (x-k_4)^3_+}{k_4 - k_1}\right] - \left[\frac{(x-k_3)^3_+ - (x-k_4)^3_+}{k_4 - k_3}\right] \right) + \beta_3\left(\left[\frac{(x-k_2)^3_+ - (x-k_4)^3_+}{k_4 - k_2}\right] - \left[\frac{(x-k_3)^3_+ - (x-k_4)^3_+}{k_4 - k_3}\right] \right)$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Can I ask a simpler question: Sites/books say: for my natural cubic spline approach (i.e. restricted cubic spline) w/ 4 knots, I need 4 basis functions.  Is it Beta_0, Beta_1*x, and '4 more' ?     Or  is indeed just 4 betas (conceptually, as I have above) ?&#xA;Thank you.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Thank you for your guidance.&#xA;I am fitting the spline, along with many other modeling co-variates, i.e. explanatory variables.  So, a simple canned package for the spline alone is not sufficient.&lt;br&gt;&#xA;The actual shape I expect, when I fit to my modeling data, is a very positive skewed distributional form (I was trying to fit the fancy shaped example data, with an inflection, thinking it would be a good test of my coding of the spline functional-form). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about snitching the functional form and calibrated-parameterization (from your Python above or from R) - but its a cubic-spline, not a natural cubic spline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get how my ftn is linear to the LHS of first knot.  I guess next step is for me to see that various terms cancel, and indeed I'd be linear to the RHS of the right-most knot too.  Also look for a 4-term basis that is said to be 'stable'.&lt;/p&gt;&#xA;" OwnerUserId="16401" LastEditorUserId="276" LastEditDate="2015-05-29T03:04:01.567" LastActivityDate="2017-01-11T08:55:17.927" Title="Is my Restricted (Natural) Cubic Spline equation wrong?" Tags="|interpolation|r|b-spline|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19694" PostTypeId="1" CreationDate="2015-05-20T13:37:50.367" Score="1" ViewCount="488" Body="&lt;p&gt;I have the &lt;strong&gt;underdetermined&lt;/strong&gt; outer optimization problem&#xA;$$\text{min}_{x\geq 0}\quad \|Ax-b_1\|_2^2+\|AT(x)-b_2\|_2^2$$&#xA;with $A\in\mathbb{R}^{m\times n}$ and $m&amp;lt;&amp;lt;n=64^2$ or in corresponding CVX Matlab code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;variable x(n) nonnegative&#xA;minimize(sum( (A*x-b1).^2 + (A*T(x,p1,p2,p3,p4,p5)-b2).^2 ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;p1,...,p5&lt;/code&gt; are fixed parameters required by the function &lt;code&gt;T&lt;/code&gt;. Inside &lt;code&gt;T&lt;/code&gt; there is another linear minimization problem&#xA;$$\begin{aligned}\text{min}_{c,q} \quad &amp;amp;1^\top q&amp;amp;\\\text{s.t.}\quad&amp;amp;Dc-x \leq q\\-&amp;amp;Dc-x \leq -q\end{aligned}$$&#xA;with $D\in\mathbb{R}^{n\times n}$ or in Matlab syntax&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function value = T(x,p1,p2,p3,p4,p5)&#xA;... something happens ...&#xA;cvx_begin&#xA;    variable c(n)&#xA;    variable q(n)&#xA;    minimize sum(q)&#xA;    subject to&#xA;        D*c-x &amp;lt;= q&#xA;        -D*c-x &amp;gt;= -q&#xA;cvx_end&#xA;... something else happens and calculates return value ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I get the error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Undefined function 'newcnstr' for input arguments of type 'cvx'.&#xA;&#xA;Error in cvx/lt (line 22)&#xA;b = newcnstr( evalin( 'caller', 'cvx_problem', '[]' ), x, y, '&amp;lt;' );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;after the &lt;code&gt;cvx_end&lt;/code&gt; line in the function &lt;code&gt;T&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I encountered this error several times before when dealing with other problems, but this time I cannot replace the outer optimization with a build in Matlab function, since &lt;code&gt;fmincon&lt;/code&gt; is far to slow. Replacing the inner problem by &lt;code&gt;linprog&lt;/code&gt; is too slow as well, unfortunately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is it possible to nest CVX optimization problems like that? If yes, how?&lt;/strong&gt;&#xA;Are there any other ideas what I could try?&lt;/p&gt;&#xA;" OwnerUserId="8299" LastActivityDate="2016-04-25T18:52:11.967" Title="How to nest 2 simple CVX problems? Is it possible at all?" Tags="|optimization|matlab|convex-optimization|constrained-optimization|cvx|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19695" PostTypeId="1" AcceptedAnswerId="19697" CreationDate="2015-05-20T14:16:47.817" Score="0" ViewCount="217" Body="&lt;p&gt;Is there an analytic solution or series approach to the 2D time-dependent heat equation in a square with a harmonic source and Neumann boundary conditions $\nabla u.n=0$?&lt;/p&gt;&#xA;" OwnerUserId="9378" LastEditorUserId="625" LastEditDate="2015-05-25T15:24:12.670" LastActivityDate="2015-05-25T15:24:12.670" Title="Heat equation with harmonic source and Neumann boundary" Tags="|parabolic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19696" PostTypeId="2" ParentId="19326" CreationDate="2015-05-20T16:03:13.340" Score="1" Body="&lt;p&gt;You shouldn't have any problem coding in a conditional boundary condition into the &lt;em&gt;odefun&lt;/em&gt;, like the one you defined in your problem statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just use the last timestep's value of $u_b$ to do the conditional statement and grab the correct boundary condition value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the reality is, if your problem needs to change the boundary condition in the middle of integrating, then it has to do it. That's just how the physics are being modeled in your case. Given that these dynamics are okay, you should ideally &lt;em&gt;incorporate&lt;/em&gt; that boundary condition change &lt;strong&gt;whenever&lt;/strong&gt; you need to.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-05-20T16:03:13.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19697" PostTypeId="2" ParentId="19695" CreationDate="2015-05-20T16:08:33.010" Score="1" Body="&lt;p&gt;This is a good reference on approaching this problem analytically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ocw.mit.edu/courses/mathematics/18-303-linear-partial-differential-equations-fall-2006/lecture-notes/pde3d.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ocw.mit.edu/courses/mathematics/18-303-linear-partial-differential-equations-fall-2006/lecture-notes/pde3d.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-05-20T16:08:33.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19698" PostTypeId="1" CreationDate="2015-05-20T17:15:37.043" Score="2" ViewCount="170" Body="&lt;p&gt;Wikipedia tells me that the equations for &lt;a href=&quot;http://en.wikipedia.org/wiki/Linear_elasticity#The_biharmonic_equation&quot; rel=&quot;nofollow&quot;&gt;linear elasticity and biharmonic equations&lt;/a&gt; have the same solution for Dirichlet boundary condition. How do you show the equivalence in the variational formulation?&lt;/p&gt;&#xA;" OwnerUserId="16404" LastEditorUserId="9667" LastEditDate="2015-05-20T19:31:02.207" LastActivityDate="2015-05-21T18:15:15.247" Title="Equivalence of linear elasticity and biharmonic equations: variational formulation" Tags="|elliptic-pde|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19699" PostTypeId="2" ParentId="19694" CreationDate="2015-05-20T18:48:48.417" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to nest CVX optimization problems like that? If yes, how?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think you can, no. CVX is designed to model convex programs; as such, it cannot model nonconvex programs (but could model their convex relaxations). In general, the bilevel linear programming case can yield a nonconvex feasible set (see &lt;a href=&quot;http://www.or.ms.unimelb.edu.au/handouts/Bilevel.pdf&quot; rel=&quot;nofollow&quot;&gt;this presentation by Chris Fricke&lt;/a&gt;, slides 40-44), which would make it impossible for CVX to solve these correctly, so this construction is probably not allowable for CVX.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there any other ideas what I could try?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In order to try other ideas, it would help to know if $T$ is a smooth function.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-05-20T18:48:48.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19700" PostTypeId="2" ParentId="19652" CreationDate="2015-05-20T19:09:38.100" Score="3" Body="&lt;p&gt;More of a long comment than anything else:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using the descriptions provided in Ascher and Petzold 1998, I wrote this function which determines the gradient at a given point x:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Look at the code for SUNDIALS' difference quotient approximation to get a better idea of what you should do in an implementation. Ascher and Petzold is a good book to get started, but SUNDIALS is actually used in production work, and thus has been better tested. (Also, SUNDIALS is related to DASPK, which Petzold worked on.)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Ascher and Petzold do mention that difference approximations for the Jacobian do not always work well. Can an approximated Jacobian with finite differences cause instability in Newton's method? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Empirically, approximate Jacobians can cause convergence failures in Newton's method. I don't know that I'd characterize them as &quot;instability&quot;; in some cases, it's just not possible to achieve the desired error tolerances in the termination criteria. In other cases, it could manifest as an instability. I'm almost certain there's a more quantitative result on this phenomenon in Higham's numerical methods book, or Hairer and Wanner's discussion of W-methods.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or is the cause somewhere else? How else might I approach this problem?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It depends where you think the error might be. If you're extremely confident in your implementation of backward Euler, I wouldn't start there. Experience has made me paranoid in my implementations of numerical methods, so if it were me, I'd start by coding up a few really basic test problems (a couple nonstiff and stiff linear problems, the heat equation with a centered finite difference approximation, things like that) and I'd use the method of manufactured solutions to assure myself that I know what the solution will be, and what I should be comparing against.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you've already done some of that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am confident that my backward Euler implementation is in order, because I have tested it using a function for the Jacobian and get stable results.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That would be the next thing I would test: use an analytical Jacobian. After that, you might also look at the extremal eigenvalues of your finite difference Jacobian on the off chance you're in the unstable region of backward Euler. Looking at extremal eigenvalues of your analytical Jacobian as a basis for comparison might give you some insight. Assuming those all check out, the problem is probably in the Newton solve.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-05-20T19:09:38.100" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19702" PostTypeId="2" ParentId="19575" CreationDate="2015-05-20T20:06:09.423" Score="1" Body="&lt;p&gt;First, I would like to mention that what you define as 'error' is pretty important. You could obtain error estimates based on an assortment of different norms or error measures, each most likely resulting in a slightly different solution. Optimality will always be depending on the error/cost function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I personally formulated it as a &lt;strong&gt;Weighted Nonlinear Least Squares&lt;/strong&gt; problem, where the location of each x point used for interpolation is a part of a vector seeking to be optimized using the standard Weighted Least Squares styled cost function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a first attempt trying to use gradient descent with this formulation, trying out the results with a few different guesses. A guess I formulated before I even attempted this numerically has found to be roughly optimal when I plugged it into the numerical algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sub-optimal analytical solution is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x = e^{(-as)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a = -ln(2048)$, and $s \in [0, 1]$. For this problem, choose 10 equally spaced apart values for $s$ starting at $s$ = 0 and ending at $s$ = 1. This generates a set of x values increasingly spreading apart as x increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I got some pretty worthwhile results with the approach I did, but I am not sure if it's totally optimal or not with respect to the cost function I made. I would recommend tackling this problem using a more global optimization approach, such as &lt;strong&gt;Particle Swarm Optimization&lt;/strong&gt;, &lt;strong&gt;Genetic Optimization&lt;/strong&gt;, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here's some possibly helpful references&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Lp_space&quot; rel=&quot;nofollow&quot;&gt;Basic $L^p$ Norm Info&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Least_squares&quot; rel=&quot;nofollow&quot;&gt;Least Squares Info&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-05-20T20:06:09.423" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19703" PostTypeId="2" ParentId="19668" CreationDate="2015-05-20T21:18:22.397" Score="1" Body="&lt;p&gt;As mentioned, you could have various global optimal solutions, so this behavior could be possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is also possible you could have a cost function surface that has a lot of variation based on the selected hypercube you are using for your search space. If there is a lot of variation and you aren't using enough points per generation in the genetic algorithm, you may not be adequately sampling the surface to end up at a given optimal location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So while this behavior isn't uncommon, you might want to experiment with having more points per generation to see if that helps you end up at a consistent global optimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I use methods such as this, I usually try to find a list of the top N best locations and then I 'zoom' in on those locations and do some more optimization procedures to better sort out which spots are most optimal.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-05-20T21:18:22.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19704" PostTypeId="1" AcceptedAnswerId="19714" CreationDate="2015-05-20T21:46:54.143" Score="16" ViewCount="23178" Body="&lt;p&gt;When solving time dependent PDE's using the finite element method, for example say the heat equation, if we use explicit time stepping then we have to solve a linear system because of the mass matrix. For example if we stick with the heat equation example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial{u}}{\partial{t}} = c\nabla{}^{2}u$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then using forward Euler we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M(\frac{u^{n+1}-u^{n}}{dt}) = -cKu^{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and thus even though we are using an explicit time stepping scheme we still have to solve a linear system. This is obviously a major problem since the primary advantage of using explicit schemes is to NOT have to solve a linear system. I have read that a common way to get around this problem is to instead use a &quot;lumped&quot; mass matrix which transforms the regular (consistent?) mass matrix into a diagonal matrix and thus makes inversion trivial. Upon doing a google search however I am still not entirely sure how this lumped mass matrix is created. For example looking at he paper &lt;em&gt;NUMERICAL EXPERIMENTS ON MASS LUMPING FOR THE ADVECTION-DIFFUSION EQUATION&lt;/em&gt; by Edson Wendland&#xA;Harry and Edmar Schulz they create their lumped mass matrix by simply summing all coefficients onto the diagonal. So for example if our original consistent mass matrix was:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{pmatrix}&#xA;    4 &amp;amp; 2 &amp;amp; 1 &amp;amp; 2 \\&#xA;    2 &amp;amp; 4 &amp;amp; 2 &amp;amp; 1 \\&#xA;    1 &amp;amp; 2 &amp;amp; 4 &amp;amp; 2 \\&#xA;    2 &amp;amp; 1 &amp;amp; 2 &amp;amp; 4\end{pmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then the lumped mass matrix would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{pmatrix}&#xA;    9 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0 &amp;amp; 9 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0 &amp;amp; 0 &amp;amp; 9 &amp;amp; 0 \\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 9\end{pmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question then is: Is this the correct way to form the lumped mass matrix? What disadvantages exist when using the lumped mass matrix instead of the full consistent mass matrix in terms of accuracy? The authors of the paper I mentioned actually suggested not using the lumped mass matrix, although it seemed they were only using an implicit time stepping scheme which I thought was odd given that the primary reason to use such matrices is for explicit methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I would never use forward Euler to solve the heat equation, that was just an example. Also if it matters my problem is solving the Navier Stokes equations where the nonlinear term is treated explicitly and the diffusion term is treated implicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="9667" LastEditDate="2015-05-21T16:51:24.363" LastActivityDate="2017-04-06T20:05:30.770" Title="How to formulate lumped mass matrix in FEM" Tags="|finite-element|time-integration|navier-stokes|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19705" PostTypeId="2" ParentId="19643" CreationDate="2015-05-20T22:03:39.393" Score="3" Body="&lt;p&gt;I'm looking forward to your comments to the following solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will solve the problem for the constraints $\mathbf{1}^T\mathbf{x}=\mathbf{1}^T\mathbf{y}_i=\mathbf{1}^T\mathbf{x}=1$ and for a general matrix $\mathbf{A}$ (not necessarily non-negative or symmetric).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, since $f$ is linear with respect to $\mathbf{x}$, an optimal solution is attained at an extreme point, thus &lt;strong&gt;we can replace the constraint $\mathbf{x}\in [0,1]^n$ by $\mathbf{x}\in\left\{0,1\right\}^n$&lt;/strong&gt;. (Of course we have the same statement for $\mathbf{y}$ and $\mathbf{z}$ but we do not need that).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\mathbf{x}\in\left\{0,1\right\}^n$ and $\mathbf{1}^T\mathbf{x}=1$, $\mathbf{x}$ has the form: one component of $\mathbf{x}$ is $1$ and all the others are $0$. Thus, $\mathbf{x}$ can only take $n$ different values. The idea is that we iterate over these $n$ values of $\mathbf{x}$ and minimize the function with respect to $(\mathbf{y},\mathbf{z})$ (which is efficient because we have an analytical solution), then we compare all the $n$ obtained optimal values and pick the best one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose that $\mathbf{x} = \mathbf{x}_0$ where $x_{i_0} = 1$ and $x_i=0$ for any $i\neq i_0$. We have &#xA;\begin{align*}f(\mathbf{x}_0,\mathbf{y},\mathbf{z}) &amp;amp;=\mathbf{z}^T\mathbf{A}\mathbf{y}_{i_0} + b_{i_0} + \sum_{1\le i\le n}\mathbf{c}_i^T\mathbf{y}_i +\mathbf{d}^T\mathbf{z} \\&#xA;&amp;amp;= \left( \mathbf{z}^T\mathbf{A}\mathbf{y}_{i_0} + \mathbf{c}_{i_0}^T\mathbf{y}_{i_0} +\mathbf{d}^T\mathbf{z} \right) +  \sum_{1\le i\le n,i\neq i_0}\mathbf{c}_i^T\mathbf{y}_i + b_{i_0}.&#xA;\end{align*}&#xA;Denote $\mathbf{y}_i = (y^i_1,y^i_2,\ldots,y^i_p)$. Clearly, the minimum value of $\mathbf{c}_i^T\mathbf{y}_i$ is the smallest component of $\mathbf{c}_i$, and if $j_0(i)$ is the position of this component, then the minimum is attained when $y^i_{j_0(i)} = 1$ and $y^i_j = 0 \forall j\neq j_0(i).$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It remains to minimize $g(\mathbf{y}_{i_0},\mathbf{z}) = \mathbf{z}^T\mathbf{A}\mathbf{y}_{i_0} + \mathbf{c}_{i_0}^T\mathbf{y}_{i_0} +\mathbf{d}^T\mathbf{z}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\mathbf{1}^T\mathbf{y}_{i_0} = \mathbf{1}^T\mathbf{z}=1$ we have &#xA;$$g(\mathbf{y}_{i_0},\mathbf{z}) = \mathbf{z}^T\mathbf{A}\mathbf{y}_{i_0} + \mathbf{z}^T\mathbf{1}\cdot\mathbf{c}_{i_0}^T\mathbf{y}_{i_0} +\mathbf{z}^T\mathbf{d}\cdot  \mathbf{1}^T\mathbf{y}_{i_0} = \mathbf{z}^T \left( \mathbf{A} + \mathbf{1}\cdot\mathbf{c}_{i_0}^T + \mathbf{d}\cdot  \mathbf{1}^T  \right) \mathbf{y}_{i_0}.$$&#xA;According to this lemma: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/18775/find-min-xtay-subject-to-1tx-1ty-1-x-ge-0-y-ge-0&quot;&gt;Find $\min x^TAy$ subject to $1^Tx=1^Ty=1,x\ge 0,y\ge 0$&lt;/a&gt;, the minimum value of $g$ is the smallest component of the matrix $\mathbf{A} + \mathbf{1}\cdot\mathbf{c}_{i_0}^T + \mathbf{d}\cdot  \mathbf{1}^T$, and if $(k_0,j_0)$ is the position of this component, then the minimum is attained when $z_{k_0} =1$ and $y^{i_0}_{j_0}=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we have the following algorithm to solve the problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Initialize the best-so-far minimum: $f_0=+\infty$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pre-compute the position $j_0(i)$ of the smallest components of the vectors $\mathbf{c}_i$. Compute the sum of these components: $S_c = \sum_{1\le i\le n}c^i_{j_0(i)}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For $i=1,2,\ldots,n$:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;+) Find the smallest component $m_{k_0,j_0}$ of the matrix $\mathbf{M}=\mathbf{A} + \mathbf{1}\cdot\mathbf{c}_{i}^T + \mathbf{d}\cdot  \mathbf{1}^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;+) Compute the current minimum $f=m_{k_0,j_0} + (S_c - c^i_{j_0(i)}) + b_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;+) Compare with the best-so-far minimum: if $f \le f_0$ then $f_0 = f$ and set the corresponding optimal solution.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-05-28T09:38:41.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19706" PostTypeId="2" ParentId="19681" CreationDate="2015-05-21T04:39:11.470" Score="0" Body="&lt;p&gt;Yes, adaptivity is usable for parabolic problems. We put together an example here: &lt;a href=&quot;https://www.dealii.org/8.2.1/doxygen/deal.II/step_26.html&quot; rel=&quot;nofollow&quot;&gt;https://www.dealii.org/8.2.1/doxygen/deal.II/step_26.html&lt;/a&gt; . Take a look at the results section in particular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I discuss this program in lectures 29 and 30 here: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-05-21T04:39:11.470" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19707" PostTypeId="2" ParentId="19618" CreationDate="2015-05-21T05:11:02.957" Score="3" Body="&lt;p&gt;The reasoning for the stability of upwind schemes based on an understanding of the characteristics of the hyperbolic equation(s). Characteristics are essentially the finite speeds at which information in a hyperbolic system travel, and are found via decomposing a hyperbolic system into independent hyperbolic PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now characteristics are essentially just pushing along the initial conditions of a given hyperbolic equation (though nonlinear equations can distort them). The fact this speed is finite results in a need to be careful with your numerical stencil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical example to illustrate the need to sample carefully is to imagine an initial condition of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_0(x) = \begin{cases} 1 &amp;amp; x &amp;lt; 2\\ 0 &amp;amp; 2 \leq x \end{cases} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the characteristic just pushes this initial condition along the space-time domain, its general shape and the center discontinuity remains. Now imagine some time later, you are aiming to evaluate the derivative at some x* location. There's a chance that the x* location is at the center of the discontinuity, where the derivative is technically undefined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/72Aq8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To counter this, we sample on one side. Now how do we decide which side to sample on? It should be the side that would change value first.. Which is the side the characteristic would touch first. This means that if the characteristic is moving in the positive x direction, we should sample to the left to ensure we capture any possible changes in the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sampling of data that is going in the opposite direction of the characteristic is known as upwinded sampling. This helps ensure numerical stability by only sampling data we know we have information for.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-05-21T05:11:02.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19709" PostTypeId="1" CreationDate="2015-05-21T09:28:43.880" Score="1" ViewCount="3489" Body="&lt;p&gt;I try to use sparse matrix operations in GPU in Python and now try to use PyCUDA with theano. But I can't find how to do sparse matrix and vector multiplication. I only got an example showing how to solve $A\cdot x=b$ using CG from the library document. Is there any document or example detailing how to do some basic sparse matrix operations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, is there another python library that can do sparse matrix operations in GPU?&lt;/p&gt;&#xA;" OwnerUserId="16412" LastEditorUserId="10152" LastEditDate="2015-05-22T22:46:14.970" LastActivityDate="2018-03-02T18:59:57.707" Title="How to use the basic Sparse matrix operations (multiplication, .etc) in PyCUDA" Tags="|python|matrix|parallel-computing|sparse-matrix|gpu|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19710" PostTypeId="1" AcceptedAnswerId="19711" CreationDate="2015-05-21T11:50:05.873" Score="3" ViewCount="224" Body="&lt;p&gt;If you want to use a computational library on a high-performance computing center where you have no root access to install new software you can use header-only template libraries such as MTL4, Eigen3, GMM++ easily by including the library folder in the linking path. How about other libraries such as PetSc, Trilinos, Armadillo, Blaze, GSL,... that require compilation? Is there a way to use them on a machine where you have no root access if they are not preinstalled?&lt;/p&gt;&#xA;" OwnerUserId="1863" LastEditorUserId="20688" LastEditDate="2019-06-22T20:28:52.883" LastActivityDate="2019-06-22T20:28:52.883" Title="Using computational libraries on a machine without root access" Tags="|software|libraries|hpc|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19711" PostTypeId="2" ParentId="19710" CreationDate="2015-05-21T12:27:09.087" Score="7" Body="&lt;p&gt;Almost everything you can build and install in your own space. With GNU autotools, you can do something like &lt;code&gt;./configure --prefix=/path/to/your/work/space ...&lt;/code&gt; and then follow the usual compilation instructions. Things based on CMake and Scons have similar facilities. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-21T12:27:09.087" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19712" PostTypeId="2" ParentId="14552" CreationDate="2015-05-21T12:50:23.480" Score="0" Body="&lt;p&gt;If you still need help, you can refer to this Github&#xA;&lt;a href=&quot;https://github.com/xuaoxiqi/Computational-Fluid-Dynamics&quot; rel=&quot;nofollow&quot;&gt;https://github.com/xuaoxiqi/Computational-Fluid-Dynamics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has lid driven cavity flow solved using SIMPLE and is quite explanatory.&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2015-05-21T12:50:23.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19714" PostTypeId="2" ParentId="19704" CreationDate="2015-05-21T13:53:44.110" Score="18" Body="&lt;p&gt;I do not think that there is a definite answer to this, because it might change from one topic to other (and also depends on the type of elements you are using). There are some recent papers talking about that, as well [2]. So, it is not a closed discussion. Furthermore, you can have different inertial components (at least in mechanics), when you have elements with kinematic constraints as beams or shells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Zienkiewicz (See [1], section 16.2.4) discuss three methods for lumping the mass matrix&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The row sum method&#xA;$$M^{(\text{lumped})}_{ii} = \sum_j M_{ij}$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Diagonal scaling&#xA;$$M^{(\text{lumped})}_{ii} = c M_{ii}$$&#xA;with $c$ adjusted to satisfy $\sum_j M^{(\text{lumped})}_{jj} = \int_\Omega \rho d\Omega$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Evaluation of $M$ using a quadrature involving only the nodal points and thus automatically yielding a diagonal matrix for standard element shape function in wich $N_i=0$ for $x=x_j$, $i\neq j$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Not all the methods work in all the cases, for example, the row sum method does not work for 8-node serendipity elements since it would lead to negative masses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used method 2 with the factor being the total mass of the element ($M_{\text{tot}}$) divided by the trace of the matrix ($\mathrm{Tr}(M)$), i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M^{(\text{lumped})}_{ii} = \frac{M_{\text{tot}}}{\mathrm{Tr}(M)} M_{ii} \quad \text{(no summation on $i$)} \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also used method 3 with the so-called Spectral Element Methods with Lobatto nodes (using these locations as nodes and integration points), that automatically lead to diagonal matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From [1], you can see this figure describing some of the methods for some element types&#xA;&lt;img src=&quot;https://i.stack.imgur.com/mCC2a.png&quot; alt=&quot;Mass lumping for some two dimensional finite elements&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;[1] Zhu, J., Z. R. L. Taylor, and O. C. Zienkiewicz. &quot;The finite element method: its basis and fundamentals.&quot; (2005): 54-102.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Felippa, Carlos A., Qiong Guo, and K. C. Park. &quot;Mass matrix templates: General description and 1d examples.&quot; Archives of Computational Methods in Engineering 22.1 (2015): 1-65.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-04-06T20:05:30.770" LastActivityDate="2017-04-06T20:05:30.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19716" PostTypeId="2" ParentId="19698" CreationDate="2015-05-21T18:15:15.247" Score="3" Body="&lt;p&gt;The article on Wikipedia does not mention boundary conditions for the biharmonic equation. It is therefore impossible to establish a variational formulation, cfr. this &lt;a href=&quot;https://scicomp.stackexchange.com/a/7863/1744&quot;&gt;answer&lt;/a&gt;. Please also note that Dirichlet b.c. for the biharmonic equation are written in terms of $u_i$ and $\frac{\partial u_i}{\partial n}$ (or some other first order derivative, depending on the problem), while Dirichlet b.c. for linear elasticity are written in terms of $u_i$ alone. It is therefore dubious to claim that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;linear elasticity and biharmonic equations have the same solution for Dirichlet boundary condition&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;if one does not provides some practical mean for defining a well posed problem for the biharmonic equation equivalent to the elastic problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover I had no time to check if the derivation of the biharmonic equation given on Wikipedia is correct (and I do not have access to the classical literature right now) but a lot of &lt;em&gt;ad hoc&lt;/em&gt; assumptions are made (e.g. $\lambda = -\beta^2$) so I suspect that the supposed &lt;em&gt;equivalence&lt;/em&gt; is far from being general.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-05-21T18:15:15.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19717" PostTypeId="1" AcceptedAnswerId="19722" CreationDate="2015-05-21T22:11:10.950" Score="8" ViewCount="150" Body="&lt;p&gt;Suppose we have an initial value problem of the form&#xA;$$ \frac{\mathrm{d} \mathbf{x}}{\mathrm{d} t} = f(\mathbf{x}) \qquad \mathbf{x}(0) = \mathbf{x}_0 $$&#xA;where $\mathbf{x}_0 \in \mathbb{R}^n$ is known exactly (i.e. to unlimited precision) and we can efficiently evaluate $f: \mathbb{R}^n \to \mathbb{R}^n$ to any precision. That is to say, we have a black box which, given a vector $\mathbf{x} \in \mathbb{R}^n$ and an integer $M$, returns an approximation to $f(\mathbf{x})$ guaranteed to be correct to $M$ digits in time polynomial in $M$. I would like to know if there are any practical methods for obtaining an approximation to $\mathbf{x}(t_f)$ (where $t_f \in \mathbb{R}$ is some given final time) which is &lt;strong&gt;provably correct to $N$ digits&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, this will not be possible for just any function $f: \mathbb{R}^n \to \mathbb{R}^n$, since $f$ might have some crazy behavior that drastically alters the true solution but isn't picked up in any reasonable number of evaluations. Thus, I am also interested in knowing what kind of well-behavior conditions on $f$ (e.g. all partial derivatives exist and are bounded, small Lipschitz constant, etc.) would be necessary to do this.&lt;/p&gt;&#xA;" OwnerUserId="11775" LastEditorUserId="11775" LastEditDate="2015-05-21T22:17:53.620" LastActivityDate="2015-05-22T06:11:11.033" Title="How can I numerically solve an ODE to $N$ provably correct digits?" Tags="|numerical-analysis|ode|precision|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19718" PostTypeId="1" AcceptedAnswerId="19735" CreationDate="2015-05-21T23:50:23.913" Score="1" ViewCount="56" Body="&lt;p&gt;I have a quantity which is estimated from a number of noisy measurements. I know that the real underlying value must be some integer multiple of two quantities, e.g. $M = I_1C_1 + I_2C_2$ where $C_1$ and $C_2$ are some known (from theory) values and $I_1,I_2$ are the unknown integer multiples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm casting this as a least squares minimisation problem, minimising over $I_1,I_2$ but I'm unsure of what minimisation technique is appropriate here? Is there some standard approach to this type of problem? I can get extreme values for the unknown integers by doing $ceil(M/C_i)$, so could then check every possible integer combination within that range. There are typically many such combinations though, so this would not be very efficient.   &lt;/p&gt;&#xA;" OwnerUserId="1314" LastActivityDate="2015-05-23T22:26:56.813" Title="Approach to determining most likely integer factors of a noisy measurement?" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19719" PostTypeId="2" ParentId="19591" CreationDate="2015-05-22T04:44:53.977" Score="0" Body="&lt;p&gt;Thanks for the replies everyone, I simply just use an extrapolation  ($D_f$ was needed for the pressure correction equation, but not the Rhie-Chow interpolation since $u_b$ is specified) and it seems to be working okay as long as the extrapolation scheme is consistent with the one used for $u_b$, $p_b$ and $p'_b$.  &lt;/p&gt;&#xA;" OwnerUserId="16303" LastActivityDate="2015-05-22T04:44:53.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19720" PostTypeId="2" ParentId="19717" CreationDate="2015-05-22T04:53:57.520" Score="2" Body="&lt;p&gt;The first of your questions is something you can actually get from most of the canned ODE integrators since they all, in one way or the other, keep track of estimates of the error. I don't think that you can get a &lt;em&gt;certificate&lt;/em&gt; that the error is below a certain number, but you will get that the &lt;em&gt;estimate&lt;/em&gt; is below your tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second of your questions is harder to answer: What is the relationship between the accuracy with which you can evaluate $f(\mathbf x)$ and how that affects the accuracy of your solution (assuming that you could integrate the ODE exactly). The ratio of these two accuracies is just the stability constant of your ODE. It will depend on things like the norm of $f$, the norm of $\nabla f$, and the length of the time period over which you integrate (and it will typically depend exponentially on this interval length). Most textbooks on numerical ODEs will have a section on stability.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-05-22T04:53:57.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19721" PostTypeId="1" CreationDate="2015-05-22T05:01:42.063" Score="2" ViewCount="68" Body="&lt;p&gt;Are there guidelines to follow in selecting the right FE discretization. Specifically, for subsurface flow models like Darcy's equation (both single and multi-phase), Richard's equation, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know one of the most important issues in the fluid mechanics community is the issue of local mass conservation, which the standard classical Galerkin method does not satisfy. That said, there are so many discretizations available out there (e.g., DG, RT0, P2/(P1-P0), least squares FEM, etc). What factors should be taken into account when deciding on a FE formulation to use for my problem? (assuming that they are compatible with the governing equations) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had these in mind:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) User implementation - probably the most obvious one. How easy it is to implement in your specific application? Why spend weeks or months attempting to implement a certain formulation when one can achieve the same solution using another formulation that's easier to use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Numerical accuracy - what is the mathematical rate of convergence for each method? This may be important for problems in which there is no analytical solution to compare to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Problem size - that is, how many degrees of freedom each formulation ends up having? If one does not have access to super computers, this may lead to memory issues when running on a laptop or PC&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Solvers/preconditioners - certain formulations (namely the classical mixed ones) result in saddle-point structures and render standard iterative solvers/preconditioners useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) Parallel performance - How do they scale across multiple processing cores? I think this would be a factor to consider particularly during the assembly, because certain formulations like DG are completely local whereas nodal degrees of freedom may be shared by multiple processors, hence communication overhead. Or would this be more of a user/software implementation issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's all the main ones I could think of for now. Are any of the above valid points to consider? And/or are there other important factors that I missed?&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2015-05-22T05:01:42.063" Title="Selecting a Finite Element discretization for subsurface flows" Tags="|finite-element|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19722" PostTypeId="2" ParentId="19717" CreationDate="2015-05-22T06:11:11.033" Score="9" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I would like to know if there are any practical methods for obtaining an approximation to $\mathbf{x}(t_f)$ (where $t_f \in \mathbb{R}$ is some given final time) which is provably correct to $N$ [sic] digits.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That all depends on your opinion of the practicality of interval arithmetic. There are validated integrators available, such as the &lt;a href=&quot;http://www.bt.pa.msu.edu/index_cosy.htm&quot;&gt;COSY&lt;/a&gt; code out of Martin Berz's group. You'd probably want to look at papers by Neumaier, Nedialkov, Berz &amp;amp; Makino, Chachuat, Stadtherr, and maybe a few other groups. Their papers tend to use the phrases &quot;Taylor model&quot;, &quot;validated integrator&quot;, and &quot;interval arithmetic&quot; among others.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Clearly, this will not be possible for just any function $f: \mathbb{R}^{n} \rightarrow \mathbb{R}^{n}$, since $f$ might have some crazy behavior that drastically alters the true solution but isn't picked up in any reasonable number of evaluations. Thus, I am also interested in knowing what kind of well-behavior conditions on $f$ (e.g. all partial derivatives exist and are bounded, small Lipschitz constant, etc.) would be necessary to do this.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The standard proofs (here, I am thinking of Dahlquist's proof, for instance) typically use Gronwall-type inequalities for error bounds, so in theory, you should just need a bounded Lipschitz constant over the domain in $\mathbb{R}^{n}$ you care about, which speaks to some of what WolfgangBangerth is talking about, although I don't know that a graduate level textbook like Hairer and Wanner would discuss specifically about the accuracy which which you can evaluate $f(\mathbf{x})$; Higham's book on accuracy and stability of numerical methods might discuss this point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, for the Taylor model methods I mentioned above, you typically need a combination of theoretical and practical conditions. The theory part is straightforward: if you want a $k$th order Taylor model, you need a function in $C^{k}$. The implementation part, which speaks to your issue about practicality, is harder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a user perspective, these integrators boil down to a two(-ish) things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do I have the source code to evaluate $f$? (An object file isn't enough.)&lt;/li&gt;&#xA;&lt;li&gt;Can I augment this source code for compatibility with an automatic differentiation library, as well as an interval arithmetic library?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Assuming the function can be parsed or operator-overloaded (depending on the automatic differentiation technology used), and assuming you can generate a function that will calculate the interval extension of $f$ and its first $k$ derivatives, then you can implement a validated integration method with $k$th-order Taylor models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for typical ODE solving methods, to comment on Wolfgang's answer:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I don't think that you can get a certificate that the error is below a certain number, but you will get that the estimate is below your tolerance.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any method that has an embedded error estimator has the information Wolfgang refers to. Usually, this means that the integration method really calculates two (or more solutions; e.g., DOP853 calculates 3 solutions) solutions and compares them via some norm. The assumption is that the higher order solution is more accurate, which may not actually be true, depending on the given problem, time step, initial conditions, etc. The solution returned by an implementation could be any of the candidate solutions computed. Taking the common Runge-Kutta 4(5) case as an example, one could return the 4th order solution or the 5th order solution; typical approaches use the Dormand-Prince formulas, which minimize the error in the 5th order solution, and return that, rather than the 4th order solution, because the 5th order solution is more likely to be more accurate. In addition to looking at stability issues, I think you should look at error control (Section II.4 of Hairer and Wanner); stability is necessary, but not sufficient for accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, a validated integrator will compute an interval containing the true solution. If the upper and lower bound on this interval agree to $M$ digits, then those are the first $M$ digits in the true solution, which is the certificate of accuracy, and sounds like what you want.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-05-22T06:11:11.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19723" PostTypeId="1" AcceptedAnswerId="19729" CreationDate="2015-05-22T08:03:30.150" Score="3" ViewCount="293" Body="&lt;p&gt;I am reading the work &quot;Blessing of Dimensionality High dimensional feature and its efficient compression for the face verification&quot; CVPR 2013. &#xA;One of the key contributes is the authors propose a new so called &quot;rotated sparse regression&quot; method for the dimension reduction. However, I don't know how to get the closed form solution for following optimization problem. The details are not given in the paper. Could someone give me a hint to  derive the following closed form in the following &#xA;&lt;img src=&quot;https://i.stack.imgur.com/adE6T.jpg&quot; alt=&quot;enter image description here&quot;&gt; &lt;/p&gt;&#xA;" OwnerUserId="16425" LastActivityDate="2015-06-08T23:09:12.183" Title="Solution to the optimization problem in &quot;Blessing of Dimensionality High ... the face verification&quot;" Tags="|optimization|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19726" PostTypeId="2" ParentId="19668" CreationDate="2015-05-22T22:44:11.787" Score="0" Body="&lt;p&gt;In general, you cannot say with certainty that a solution is &quot;optimized enough&quot; if you get it several times. The reason is that genetic algorithms are incomplete methods that do not consider the entire design space systematically. You can only make statements with certainty once you have complete information, i.e. the design space has been explored exhaustively. This is not what genetic algorithms are designed to do; instead, they are designed to quickly lead you to a good solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two scenarios to consider if you are getting solutions of the same quality several times:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The solutions are actually the same.&lt;/li&gt;&#xA;&lt;li&gt;The solutions are different, but evaluate to the same quality.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first case is stronger because at the very least it tells you that &lt;em&gt;the particular implementation you're using&lt;/em&gt; is unlikely to find something better. The second case only tells you that there are a lot of good solutions to be found. In particular, it may be reasonable to terminate the process after a sufficient number of runs in the first case, but in the second case you have no indication that a subsequent run may not give you a better solution (assuming of course no knowledge of the design space).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out in the other answers, there could be many globally optimal solutions, and these are the ones that you find. However, &lt;em&gt;you have no way of knowing that&lt;/em&gt;. It could simply be the case that there are a lot of locally optimal solutions and you are finding those, but in the next run of your genetic algorithm you find the much better global optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider a case where the global optimum can only be found by mutation of a specific gene with a very low probability. It will never be discovered by combination, because the gene is very weak and will be dominated by anything combined with it. The performance of all individuals without the mutation is the same. A genetic algorithm would give you solutions with exactly the same quality for lots of runs and never find the optimum solution, even though it may be much better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, the behaviour you get depends more on the search space than on the implementation of the genetic algorithm. In particular finding many good solutions is not necessarily a good measure of the quality of implementation of the genetic algorithm -- it may be optimized for different kinds of search spaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for a termination condition, I would consider terminating if a sufficient number of separate runs give you &lt;em&gt;exactly&lt;/em&gt; the same answer reasonable -- but not because you can now be reasonably sure that something close to the global optimum has been found, but because it is unlikely that your particular genetic algorithm will improve on what it found before.&lt;/p&gt;&#xA;" OwnerUserId="16430" LastActivityDate="2015-05-22T22:44:11.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19727" PostTypeId="2" ParentId="19709" CreationDate="2015-05-23T06:17:13.397" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://wiki.tiker.net/PyCuda/Examples/SparseSolve&quot; rel=&quot;nofollow&quot;&gt;This example from the PyCUDA&lt;/a&gt; demonstrates how to do sparse matrix vector multiplication.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2015-05-23T06:17:13.397" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19728" PostTypeId="2" ParentId="19709" CreationDate="2015-05-23T15:01:01.270" Score="2" Body="&lt;p&gt;I think the most used libraries for sparse matrix operations using CUDA is &lt;a href=&quot;https://developer.nvidia.com/cuSPARSE&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuSPARSE&lt;/a&gt;, which already comes included in the CUDA toolkit and supports all common sparse matrix formats. There is a Python wrapper for it &lt;a href=&quot;https://github.com/lebedov/scikit-cuda/blob/master/skcuda/cusparse.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, the Python/CUDA ecosystem still seems weirdly fractured, with no obvious choice existing for many common tasks. That said, most of my Python/CUDA experience comes from writing custom kernels using PyCUDA, which worked well.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastEditorUserId="14598" LastEditDate="2017-09-21T10:17:08.763" LastActivityDate="2017-09-21T10:17:08.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19729" PostTypeId="2" ParentId="19723" CreationDate="2015-05-23T15:23:06.917" Score="5" Body="&lt;p&gt;There's a well known problem called the orthogonal Procrustes problem,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \min_{Q} \| AQ-B \|_{F} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to the constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ Q^{T}Q=I. $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the norm in the objective function is the Frobenius norm rather than the matrix 2-norm.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this problem, the solution is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q=UV^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A^{T}B$ has the SVD&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A^{T}B=U\Sigma V^{T}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solution to the Procrustes problem is derived in lots of places.  See for example the Wikipedia page here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Orthogonal_Procrustes_problem&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Orthogonal_Procrustes_problem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem had the objective $\min \| R^{T}Y-B^{T}X \|_{F} $ rather than $\| R^{T}Y-B^{T}X \|_{2}^{2}$, then the problem could be rewritten as a Procrustes problem by transposing inside the norm,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\min \| Y^{T}R - X^{T}B \|_{F}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R^{T}R=I$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution would be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;R=UV^{T}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $YX^{T}B$ has the singular value decomposition &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$YX^{T}B=U\Sigma V^{T}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This matches up exactly with the solution given in the paper that you've quoted.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't believe that minimizing the 2-norm is equivalent to minimizing the Frobenius norm.  It seems to me that the authors of the paper you cite may have confused the Frobenius norm and the 2-norm.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added after I finally got my hands on Watson's 1993 paper...  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See&lt;/p&gt;&#xA;&#xA;&lt;p&gt;G. A. Watson.  Solving Generalizations of Orthogonal Procrustes Problems. In Contributions in Numerical Mathematics, World Scientific Series in Applicable Analysis, World Scientifc Publishers, 1993.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classical solution using the SVD is optimal with respect to the 2-norm only if the matrices are square ($m=n$), and the $A$ matrix is the identity- that is, we're minimizing $\min \| Q-B \|_{2}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, Watson offers an iterative algorithm that requires the solution of a quadratic programming problem at each iteration.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-06-08T23:09:12.183" LastActivityDate="2015-06-08T23:09:12.183" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19730" PostTypeId="1" CreationDate="2015-05-23T16:28:17.873" Score="4" ViewCount="1134" Body="&lt;p&gt;The answer to this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10329/stable-time-step-limits-for-velocity-verlet-integration&quot;&gt;question&lt;/a&gt; implies that reducing the time step would make it more stable. However I have tried reducing the time step but the system is still unstable(the total energy increases to a very large value). What I understand is that if two balls are very close and the end of some step, in the next step they will have very large velocity, thus making the system unstable. How can I choose the time step so that the system remains stable? I would also appreciate if someone could look into the code I provide to figure out if there is some error in implementation of the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program mol_dyn_ref&#xA;  implicit none&#xA;  double precision,allocatable,dimension(:) :: posx,posy,posz,velx,vely,velz,ax,ay,az,tempx,tempy,tempz&#xA;  integer,allocatable,dimension(:) :: seed&#xA;  double precision :: x,y,z,m,rad,eps,kin,pot,k,p&#xA;  double precision :: xinit,yinit,zinit,xlen,ylen,zlen,mindist&#xA;  integer :: i,j,st,seedsize,n,iter,t&#xA;  double precision :: tot_t,dt,dist,r,Fx,Fy,Fz&#xA;&#xA;  interface&#xA;     subroutine acc(n,posx,posy,posz,m,eps,tempx,tempy,tempz)&#xA;       implicit none&#xA;       integer :: i,j,n&#xA;       double precision,dimension(n) :: posx,posy,posz,ax,ay,az,tempx,tempy,tempz&#xA;       double precision :: r,dist,Fx,Fy,Fz,eps,m&#xA;     end subroutine acc&#xA;&#xA;     subroutine pos_upd(n,posx,posy,posz,velx,vely,velz,ax,ay,az,dt,xlen,ylen,zlen,rad)&#xA;       implicit none&#xA;       double precision,dimension(n) :: posx,posy,posz,velx,vely,velz,ax,ay,az&#xA;       integer :: n&#xA;       double precision :: tot_t,dt,xlen,ylen,zlen,rad&#xA;     end subroutine pos_upd&#xA;&#xA;     subroutine vel_upd(n,velx,vely,velz,ax,ay,az,dt,tempx,tempy,tempz)&#xA;       implicit none&#xA;       integer n&#xA;       double precision,dimension(n) :: velx,vely,velz,ax,ay,az,tempx,tempy,tempz&#xA;       double precision :: dt&#xA;     end subroutine vel_upd&#xA;  end interface&#xA;&#xA;&#xA;&#xA;  n=200          !Number of particles&#xA;  eps=1.d0&#xA;  m=1.d0&#xA;  xlen=20.d0&#xA;  ylen=20.d0&#xA;  zlen=20.d0&#xA;  xinit=0.d0&#xA;  yinit=0.d0&#xA;  zinit=0.d0&#xA;  mindist=1.5d0&#xA;  rad=mindist/2.d0&#xA;  tot_t=10.d0&#xA;  dt=0.001d0&#xA;&#xA;  iter=int(tot_t/dt)&#xA;&#xA;  allocate(posx(n),posy(n),posz(n),velx(n),vely(n),velz(n),ax(n),ay(n),az(n),tempx(n),tempy(n),tempz(n))&#xA;&#xA;  open(100,file=&quot;pos.dat&quot;,status=&quot;replace&quot;)&#xA;  open(200,file=&quot;vel.dat&quot;,status=&quot;replace&quot;)&#xA;  open(300,file=&quot;acc.dat&quot;,status=&quot;replace&quot;)&#xA;  open(400,file=&quot;energy.dat&quot;,status=&quot;replace&quot;)&#xA;&#xA;  call random_seed(size=seedsize)&#xA;  allocate(seed(seedsize))&#xA;  do i=1,seedsize&#xA;     call system_clock(st)&#xA;     seed(i)=st&#xA;  enddo&#xA;  call random_seed(put=seed)&#xA;&#xA;  !Assigning initial position to first particle&#xA;10 call random_number(x)&#xA;  posx(1)=xinit+x*xlen&#xA;  call random_number(y)&#xA;  posy(1)=yinit+y*ylen&#xA;  call random_number(z)&#xA;  posz(1)=zinit+z*zlen&#xA;&#xA;  if(posx(1)&amp;lt;rad .OR. posx(1)&amp;gt;xlen-rad) goto 10&#xA;  if(posy(1)&amp;lt;rad .OR. posy(1)&amp;gt;ylen-rad) goto 10&#xA;  if(posz(1)&amp;lt;rad .OR. posz(1)&amp;gt;zlen-rad) goto 10&#xA;&#xA;  !Assigning initial position&#xA;  do i=2,n&#xA;20   call random_number(x)&#xA;     posx(i)=xinit+x*xlen&#xA;     call random_number(y)&#xA;     posy(i)=yinit+y*ylen&#xA;     call random_number(z)&#xA;     posz(i)=zinit+z*zlen&#xA;     if(posx(i)&amp;lt;rad .OR. posx(i)&amp;gt;xlen-rad) goto 20&#xA;     if(posy(i)&amp;lt;rad .OR. posy(i)&amp;gt;ylen-rad) goto 20&#xA;     if(posz(i)&amp;lt;rad .OR. posz(i)&amp;gt;zlen-rad) goto 20&#xA;     do j=1,i-1&#xA;        if (dist(posx(i),posy(i),posz(i),posx(j),posy(j),posz(j))&amp;lt;mindist) goto 20&#xA;     enddo&#xA;  enddo&#xA;  print*, &quot;Position initialisation finished&quot;&#xA;&#xA;  !Assigning initial velocities&#xA;  do i=1,n&#xA;     call random_number(x)&#xA;     velx(i)=02.1d0*(2.d0*x-1.d0)&#xA;     call random_number(y)&#xA;     vely(i)=02.1d0*(2.d0*y-1.d0)&#xA;     call random_number(z)&#xA;     velz(i)=02.1d0*(2.d0*z-1.d0)&#xA;   enddo&#xA;  print*, &quot;Velocity initialisation finished&quot;&#xA;&#xA;  !Assigning initial acceleration&#xA; do i=1,n&#xA;     ax(i)=0&#xA;     ay(i)=0&#xA;     az(i)=0&#xA;     do j=1,n&#xA;        if (j==i) cycle&#xA;        r=dist(posx(i),posy(i),posz(i),posx(j),posy(j),posz(j))&#xA;        ax(i)=ax(i)+ Fx(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;        ay(i)=ay(i)+ Fy(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;        az(i)=az(i)+ Fz(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;enddo&#xA;  enddo&#xA;  print*, &quot;Acceleration initialisation finished.&quot;&#xA;&#xA;!Molecular Dynamics Simulation&#xA;  do t=1,iter&#xA;     print*, t&#xA;     do i=1,n&#xA;     write(100,*) posx(i),posy(i),posz(i)&#xA;     write(200,*) velx(i),vely(i),velz(i)&#xA;     write(300,*) ax(i),ay(i),az(i)&#xA;     enddo&#xA;     k=kin(velx,vely,velz,m,n)&#xA;     p=pot(posx,posy,posz,eps,n)&#xA;     write(400,*) t,k,p,k+p&#xA;&#xA;     call pos_upd(n,posx,posy,posz,velx,vely,velz,ax,ay,az,dt,xlen,ylen,zlen,rad)&#xA;     call acc(n,posx,posy,posz,m,eps,tempx,tempy,tempz)&#xA;     call vel_upd(n,velx,vely,velz,ax,ay,az,dt,tempx,tempy,tempz)&#xA;  enddo&#xA;  call system(&quot;gnuplot --persist plot.gp&quot;)&#xA;endprogram mol_dyn_ref&#xA;&#xA;&#xA;!Updating Acceleration&#xA;subroutine acc(n,posx,posy,posz,m,eps,tempx,tempy,tempz)&#xA;  implicit none&#xA;&#xA;  integer :: i,j,n&#xA;  double precision,dimension(n) :: posx,posy,posz,ax,ay,az,tempx,tempy,tempz&#xA;  double precision :: r,dist,Fx,Fy,Fz,eps,m&#xA;  do i=1,n&#xA;     tempx(i)=ax(i)&#xA;     tempy(i)=ay(i)&#xA;     tempz(i)=az(i)&#xA;     ax(i)=0.d0&#xA;     ay(i)=0.d0&#xA;     az(i)=0.d0&#xA;     do j=1,n&#xA;        if (j==i) cycle&#xA;        r=dist(posx(i),posy(i),posz(i),posx(j),posy(j),posz(j))&#xA;        ax(i)=ax(i)+ Fx(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;        ay(i)=ay(i)+ Fy(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;        az(i)=az(i)+ Fz(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;     enddo&#xA;  enddo&#xA;end subroutine acc&#xA;&#xA;!Updating Position&#xA;subroutine pos_upd(n,posx,posy,posz,velx,vely,velz,ax,ay,az,dt,xlen,ylen,zlen,rad)&#xA;  implicit none&#xA;&#xA;  integer n,i&#xA;  double precision,dimension(n) :: posx,posy,posz,velx,vely,velz,ax,ay,az&#xA;  double precision :: dt,xlen,ylen,zlen,rad&#xA;     do i=1,n&#xA;        posx(i)=posx(i) + velx(i)*dt + (ax(i)*(dt**2))/2&#xA;        posy(i)=posy(i) + vely(i)*dt + (ay(i)*(dt**2))/2&#xA;        posz(i)=posz(i) + velz(i)*dt + (az(i)*(dt**2))/2&#xA;&#xA;        if(posx(i)&amp;lt;rad .OR. posx(i)&amp;gt;xlen-rad) velx(i)=-velx(i)&#xA;        if(posy(i)&amp;lt;rad .OR. posy(i)&amp;gt;ylen-rad) vely(i)=-vely(i)&#xA;        if(posz(i)&amp;lt;rad .OR. posz(i)&amp;gt;zlen-rad) velz(i)=-velz(i)&#xA;     enddo&#xA;end subroutine pos_upd&#xA;&#xA;!Updating Velocity&#xA;subroutine vel_upd(n,velx,vely,velz,ax,ay,az,dt,tempx,tempy,tempz)&#xA;&#xA;  implicit none&#xA;  integer n,i&#xA;  double precision,dimension(n) :: velx,vely,velz,ax,ay,az,tempx,tempy,tempz&#xA;  double precision :: dt&#xA;  do i=1,n&#xA;    velx(i)=velx(i) + 0.5d0*(ax(i)+tempx(i))*dt&#xA;    vely(i)=vely(i) + 0.5d0*(ay(i)+tempy(i))*dt&#xA;    velz(i)=velz(i) + 0.5d0*(az(i)+tempz(i))*dt&#xA;  enddo&#xA;end subroutine vel_upd&#xA;&#xA;function pot(posx,posy,posz,eps,n)&#xA;  implicit none&#xA;  double precision pot,r,dist,eps&#xA;  integer i,j,n&#xA;  double precision, dimension(n) :: posx,posy,posz&#xA;  pot=0.d0&#xA;  do i=1,n&#xA;  do j=1,n&#xA;  if(i==j) cycle&#xA;  r=dist(posx(i),posy(i),posz(i),posx(j),posy(j),posz(j))&#xA;  pot= pot + (4.d0*eps*((1.d0/r)**12 - (1.d0/r)**6))                 !r is relative distance. x,y,z are components of r.&#xA;  enddo&#xA;  enddo&#xA;end function pot&#xA;&#xA; function kin(velx,vely,velz,m,n)&#xA; implicit none&#xA;  double precision :: kin,m&#xA;  integer :: i,n&#xA;  double precision, dimension(n) :: velx,vely,velz&#xA;  kin=0.d0&#xA;  do i=1,n&#xA;  kin = kin + ((velx(i)**2 + vely(i)**2 + velz(i)**2)/(2*m))&#xA;  enddo&#xA;end function kin&#xA;&#xA;function dist(x1,y1,z1,x2,y2,z2)&#xA;  implicit none&#xA;  double precision :: dist,x1,y1,z1,x2,y2,z2&#xA;  dist = sqrt((x1-x2)**2 + (y1-y2)**2 + (z1-z2)**2)&#xA;end function dist&#xA;&#xA;function Fx(r,x,y,z,eps)&#xA;  implicit none&#xA;  double precision :: Fx,r,x,y,z,eps&#xA;  Fx = 4.d0*eps*((12.d0/r**14) - (6.d0/r**8))*x&#xA;end function Fx&#xA;&#xA;function Fy(r,x,y,z,eps)&#xA;  implicit none&#xA;  double precision :: Fy,r,x,y,z,eps&#xA;  Fy = 4.d0*eps*((12.d0/r**14) - (6.d0/r**8))*y&#xA;end function Fy&#xA;&#xA;function Fz(r,x,y,z,eps)&#xA;  implicit none&#xA;  double precision :: Fz,r,x,y,z,eps&#xA;  Fz = 4.d0*eps*((12.d0/r**14) - (6.d0/r**8))*z&#xA;end function Fz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16433" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-05-23T19:06:59.737" Title="How can I make velocity verlet algorithm more stable?" Tags="|computational-physics|molecular-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19731" PostTypeId="2" ParentId="19730" CreationDate="2015-05-23T17:25:04.463" Score="6" Body="&lt;p&gt;You can try using an adaptive timestep, where &lt;code&gt;dt&lt;/code&gt; is adjusted at each step if the velocities get too large. However, the Verlet method is symplectic, so if the total energy is not roughly conserved then something is wrong. If two particles get too close, no reasonable timestep will make the method stable, and accurately computing the forces between particles with double-precision floats will be impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it looks like you're randomly initializing the particle positions so that they are not too close to each other, you don't appear to have any guards on the particle velocities to keep the particles from approaching too close at the first update. The Lenard-Jones potential that you're using is very stiff at close distances, so if you're not careful in how you initialize the system you can have one particle fly off to infinity right away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, try initializing the system in a &quot;cold&quot; state: set all the velocities to zero and put the particles in a hexagonally close-packed arrangement, with the distance equal to the equilibrium distance for the potential you've chosen. You'll see some amount of oscillation about the true equilibrium configuration. This will be a much easier system to solve so that you can verify the correctness and stability of your algorithm. From there, you can slowly increase the velocities of the initial arrangement until the system reaches the liquid or gaseous state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You didn't ask about coding style, but all the same, I recommend organizing your code into modules and declaring the &lt;code&gt;intent&lt;/code&gt; of every procedure argument. &lt;a href=&quot;http://www.fortran90.org/src/best-practices.html&quot; rel=&quot;noreferrer&quot;&gt;Fortran Best Practices&lt;/a&gt; has more good information on this subject.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2015-05-23T19:06:59.737" LastActivityDate="2015-05-23T19:06:59.737" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19732" PostTypeId="2" ParentId="19723" CreationDate="2015-05-23T20:16:28.493" Score="2" Body="&lt;p&gt;I couldn't find anything on the Procrustes problem in the spectral norm online, but I had a feeling that the same result as in the Frobenius norm should hold, so I worked something out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we're given a matrix $M$ and want to find an orthogonal matrix $Q$ which minimizes&#xA;$$&#xA;  \min_{Q^T Q=I} \| M - Q \|_2.&#xA;$$&#xA;We take the SVD of $M$, i.e.,&#xA;$$&#xA;  M = U \Sigma V^T.&#xA;$$&#xA;In particular, we have normed singular vectors $u_j,v_j$ with $M v_j = \sigma_j u_j$ and hence $\|M v_j\| = \sigma_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any such orthogonal matrix $Q$ and any vector $v$ with $\|v\|=1$, we have&#xA;$$&#xA; \| M - Q \|_2 \ge \| Mv - Qv \| \ge \Big| \| Mv \| - \| Qv\|\Big| = \Big| \| Mv \| - 1 \Big|,&#xA;$$&#xA;and by inserting the singular vectors $v_j$, we obtain&#xA;$$&#xA;  \| M - Q \|_2 \ge \max_j | \sigma_j - 1 |.&#xA;$$&#xA;On the other hand, the special choice $Q = UV^T$ leads to&#xA;$$&#xA;  \| M - Q \|_2 = \| U (\Sigma - I) V^T \|_2 = \| \Sigma - I \|_2 = \max_j |\sigma_j - 1|&#xA;$$&#xA;and is therefore the best approximation of $M$ in the spectral norm as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this does not hold for the generalization to the more general problem $\| A Q - B\|_2 \to \min$ according to the paper cited by Brian Borcher above: Watson, G. A. &quot;Solving generalizations of orthogonal Procrustes problems.&quot; World Scientific Series in Applicable Analysis 2 (1993): 413-426&lt;/p&gt;&#xA;" OwnerUserId="14598" LastEditorUserId="14598" LastEditDate="2015-05-23T21:22:55.087" LastActivityDate="2015-05-23T21:22:55.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19733" PostTypeId="2" ParentId="19693" CreationDate="2015-05-23T21:58:31.323" Score="1" Body="&lt;p&gt;Answering your second question first: for &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; knots, natural cubic splines really have only &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; degrees of freedom/basis functions.&lt;/p&gt;&#xA;&lt;p&gt;This is quite easy to see if you count degrees of freedom: &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; knots split the real line into &lt;span class=&quot;math-container&quot;&gt;$K+1$&lt;/span&gt; intervals, and a piecewise cubic function in these intervals has &lt;span class=&quot;math-container&quot;&gt;$4(K+1)$&lt;/span&gt; degrees of freedom. By matching function value, first and second derivative in each knot, we get &lt;span class=&quot;math-container&quot;&gt;$3K$&lt;/span&gt; conditions, hence &lt;span class=&quot;math-container&quot;&gt;$K+4$&lt;/span&gt; degrees of freedom are left. Finally, the &amp;quot;natural&amp;quot; spline condition says that we want the spline to be linear in the first and last interval, which kills another &lt;span class=&quot;math-container&quot;&gt;$2+2$&lt;/span&gt; degrees of freedom (the quadratic and cubic coefficients in these intervals). All in all, we are left with &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; degrees of freedom.&lt;/p&gt;&#xA;&lt;p&gt;For your first question, it's really hard to tell what's going wrong without seeing your code. In general, I wouldn't recommend implementing spline routines yourself unless you have to; the basis you have used is very numerically unstable, and in general the code is a bit finicky to get right.&lt;/p&gt;&#xA;&lt;p&gt;However, any serious scientific computing environment has these things built in, so generally you don't need to write it yourself. Here is your example in Python/scipy:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;In [1]: %pylab&#xA;Using matplotlib backend: Qt4Agg&#xA;Populating the interactive namespace from numpy and matplotlib&#xA;&#xA;In [2]: n = 100&#xA;&#xA;In [3]: x = linspace(0, 1, n)&#xA;&#xA;In [10]: y = (((exp(1.2*x) + 1.5*sin(7*x))-1)/3) + normal(0, 0.15, size=n)&#xA;&#xA;In [13]: from scipy.interpolate import LSQUnivariateSpline&#xA;&#xA;In [15]: t = [0.2, 0.4, 0.6, 0.8]&#xA;&#xA;In [16]: spl = LSQUnivariateSpline(x, y, t)&#xA;&#xA;In [21]: plt.style.use('bmh')&#xA;&#xA;In [22]: plot(x, spl(x), '-', x, y, 'o')&#xA;&#xA;In [23]: plot(x, (((exp(1.2*x) + 1.5*sin(7*x))-1)/3), '--')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/com5Y.png&quot; alt=&quot;smoothing B-spline&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;The blue line is the smoothing B-spline with four knots, the dashed line the original function without noise.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-05-23T21:58:31.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19734" PostTypeId="2" ParentId="19710" CreationDate="2015-05-23T22:19:17.103" Score="2" Body="&lt;p&gt;If you want to use Python libraries, you can even use the &lt;code&gt;pip&lt;/code&gt; package manager to install a package for your user account only with this invocation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install --user &amp;lt;package&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another possibility for Python is using &lt;a href=&quot;https://virtualenv.pypa.io/en/latest/&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;virtualenv&lt;/code&gt;&lt;/a&gt;, which allows you to set up isolated Python environments without the need for root access.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-05-23T22:19:17.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19735" PostTypeId="2" ParentId="19718" CreationDate="2015-05-23T22:26:56.813" Score="3" Body="&lt;p&gt;I think this paper should be relevant to your problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Arash  Hassibi  and  Stephen  Boyd: &lt;a href=&quot;https://web.stanford.edu/~boyd/papers/pdf/int_est.pdf&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Integer Parameter Estimation in Linear Models with Applications to GPS&lt;/em&gt;&lt;/a&gt;. IEEE Transactions on Signal Processing, Vol. 46, No. 11, November 1998&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors state that the integer least-squares problem is NP-hard, but can be solved efficiently in many practical cases by the &lt;a href=&quot;http://en.wikipedia.org/wiki/Lenstra%E2%80%93Lenstra%E2%80%93Lov%C3%A1sz_lattice_basis_reduction_algorithm&quot; rel=&quot;nofollow&quot;&gt;LLL algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2015-05-23T22:26:56.813" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19736" PostTypeId="1" CreationDate="2015-05-24T03:06:47.980" Score="2" ViewCount="85" Body="&lt;p&gt;I am doing some statistical signal analysis and was wondering if there are any C/Java packages that do L-moment calculations, specifically L-Kurtosis as I am wanting to do things such as differentiating delta trains and sine waves with similar standard deviations, as one example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are other more appropriate shape statistics to use, I would appreciate any information there as well.&lt;/p&gt;&#xA;" OwnerUserId="16321" LastEditorUserId="20688" LastEditDate="2019-07-02T16:38:34.160" LastActivityDate="2019-07-02T16:43:09.197" Title="Efficient calculation for L-Kurtosis?" Tags="|software|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19738" PostTypeId="1" CreationDate="2015-05-24T13:44:13.540" Score="0" ViewCount="121" Body="&lt;p&gt;I am trying to make a very simple molecular dynamics simulator with Reflective Boundary Conditions. I am assigning the initial positions in a cube randomly while making sure they are not too close to each other and keeping the initial velocities zero. I am using the Lennard-Jones potential and Velocity-Verlet algorithm. But as soon as I start the simulation, Velocities of some of the particles reach very high value and total energy also goes to very high value(not consistent with the initial total energy). Can anyone point out what are the possible reasons for such a behaviour?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code (in Fortran).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program mol_dyn_ref&#xA;  implicit none&#xA;  double precision,allocatable,dimension(:) :: posx,posy,posz,velx,vely,velz,ax,ay,az,tempx,tempy,tempz&#xA;  integer,allocatable,dimension(:) :: seed&#xA;  double precision :: x,y,z,m,rad,eps,kin,pot,k,p,avgvx,avgvy,avgvz&#xA;  double precision :: xinit,yinit,zinit,xlen,ylen,zlen,mindist&#xA;  integer :: i,j,st,seedsize,n,iter,t&#xA;  double precision :: tot_t,dt,dist,r,Fx,Fy,Fz&#xA;&#xA;  interface&#xA;     subroutine acc(n,posx,posy,posz,m,eps,tempx,tempy,tempz)&#xA;       implicit none&#xA;       integer :: i,j,n&#xA;       double precision,dimension(n) :: posx,posy,posz,ax,ay,az,tempx,tempy,tempz&#xA;       double precision :: r,dist,Fx,Fy,Fz,eps,m&#xA;     end subroutine acc&#xA;&#xA;     subroutine pos_upd(n,posx,posy,posz,velx,vely,velz,ax,ay,az,dt,xlen,ylen,zlen,rad)&#xA;       implicit none&#xA;       double precision,dimension(n) :: posx,posy,posz,velx,vely,velz,ax,ay,az&#xA;       integer :: n&#xA;       double precision :: tot_t,dt,xlen,ylen,zlen,rad&#xA;     end subroutine pos_upd&#xA;&#xA;     subroutine vel_upd(n,velx,vely,velz,ax,ay,az,dt,tempx,tempy,tempz)&#xA;       implicit none&#xA;       integer n&#xA;       double precision,dimension(n) :: velx,vely,velz,ax,ay,az,tempx,tempy,tempz&#xA;       double precision :: dt&#xA;     end subroutine vel_upd&#xA;  end interface&#xA;&#xA;&#xA;&#xA;  n=50         !Number of particles&#xA;  eps=1.0d0&#xA;  m=1.0d0&#xA;  xlen=20.0d0&#xA;  ylen=20.0d0&#xA;  zlen=20.0d0&#xA;  xinit=0.0d0&#xA;  yinit=0.0d0&#xA;  zinit=0.0d0&#xA;  mindist=1.5d0&#xA;  rad=mindist/2.0d0&#xA;  tot_t=10.0d0&#xA;  dt=0.0001d0&#xA;&#xA;  iter=int(tot_t/dt)&#xA;&#xA;  allocate(posx(n),posy(n),posz(n),velx(n),vely(n),velz(n),ax(n),ay(n),az(n),tempx(n),tempy(n),tempz(n))&#xA;&#xA;  open(100,file=&quot;pos.dat&quot;,status=&quot;replace&quot;)&#xA;  open(200,file=&quot;vel.dat&quot;,status=&quot;replace&quot;)&#xA;  open(300,file=&quot;acc.dat&quot;,status=&quot;replace&quot;)&#xA;  open(400,file=&quot;energy.dat&quot;,status=&quot;replace&quot;)&#xA;&#xA;  call random_seed(size=seedsize)&#xA;  allocate(seed(seedsize))&#xA;  do i=1,seedsize&#xA;     call system_clock(st)&#xA;     seed(i)=st&#xA;  enddo&#xA;  call random_seed(put=seed)&#xA;&#xA;  !Assigning initial position to first particle&#xA;10 call random_number(x)&#xA;  posx(1)=xinit+x*xlen&#xA;  call random_number(y)&#xA;  posy(1)=yinit+y*ylen&#xA;  call random_number(z)&#xA;  posz(1)=zinit+z*zlen&#xA;&#xA;  if(posx(1)&amp;lt;rad .OR. posx(1)&amp;gt;xlen-rad) goto 10&#xA;  if(posy(1)&amp;lt;rad .OR. posy(1)&amp;gt;ylen-rad) goto 10&#xA;  if(posz(1)&amp;lt;rad .OR. posz(1)&amp;gt;zlen-rad) goto 10&#xA;&#xA;  !Assigning initial position&#xA;  do i=2,n&#xA;20   call random_number(x)&#xA;     posx(i)=xinit+x*xlen&#xA;     call random_number(y)&#xA;     posy(i)=yinit+y*ylen&#xA;     call random_number(z)&#xA;     posz(i)=zinit+z*zlen&#xA;     if(posx(i)&amp;lt;rad .OR. posx(i)&amp;gt;xlen-rad) goto 20&#xA;     if(posy(i)&amp;lt;rad .OR. posy(i)&amp;gt;ylen-rad) goto 20&#xA;     if(posz(i)&amp;lt;rad .OR. posz(i)&amp;gt;zlen-rad) goto 20&#xA;     do j=1,i-1&#xA;        if (dist(posx(i),posy(i),posz(i),posx(j),posy(j),posz(j))&amp;lt;mindist) goto 20&#xA;     enddo&#xA;  enddo&#xA;  print*, &quot;Position initialisation finished&quot;&#xA;&#xA;  !Assigning initial velocities&#xA;  do i=1,n&#xA;     velx(i)=0.d0&#xA;     vely(i)=0.d0&#xA;     velz(i)=0.d0&#xA;  enddo     &#xA;  print*, &quot;Velocity initialisation finished&quot;&#xA;&#xA;  !Calculating initial acceleration&#xA;  do i=1,n&#xA;     ax(i)=0&#xA;     ay(i)=0&#xA;     az(i)=0&#xA;     do j=1,n&#xA;        if (j==i) cycle&#xA;        r=dist(posx(i),posy(i),posz(i),posx(j),posy(j),posz(j))&#xA;        ax(i)=ax(i)+ Fx(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;        ay(i)=ay(i)+ Fy(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;        az(i)=az(i)+ Fz(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;     enddo&#xA;  enddo&#xA;  print*, &quot;Acceleration initialisation finished.&quot;&#xA;&#xA;  !Molecular Dynamics Simulation&#xA;  do t=1,iter&#xA;     print*, t&#xA;     do i=1,n&#xA;        write(100,*) posx(i),posy(i),posz(i)&#xA;        write(200,*) velx(i),vely(i),velz(i)&#xA;        write(300,*) ax(i),ay(i),az(i)&#xA;     enddo&#xA;&#xA;     k=kin(velx,vely,velz,m,n)&#xA;     p=pot(posx,posy,posz,eps,n)&#xA;     write(400,*) t,k,p,k+p&#xA;&#xA;     call pos_upd(n,posx,posy,posz,velx,vely,velz,ax,ay,az,dt,xlen,ylen,zlen,rad)&#xA;     call acc(n,posx,posy,posz,m,eps,tempx,tempy,tempz)&#xA;     call vel_upd(n,velx,vely,velz,ax,ay,az,dt,tempx,tempy,tempz)&#xA;  enddo&#xA;  call system(&quot;gnuplot --persist plot.gp&quot;)&#xA;endprogram mol_dyn_ref&#xA;&#xA;&#xA;!Updating Acceleration&#xA;subroutine acc(n,posx,posy,posz,m,eps,tempx,tempy,tempz)&#xA;  implicit none&#xA;  integer :: i,j,n&#xA;  double precision,dimension(n) :: posx,posy,posz,ax,ay,az,tempx,tempy,tempz&#xA;  double precision :: r,dist,Fx,Fy,Fz,eps,m&#xA;  do i=1,n&#xA;     tempx(i)=ax(i)&#xA;     tempy(i)=ay(i)&#xA;     tempz(i)=az(i)&#xA;     ax(i)=0.d0&#xA;     ay(i)=0.d0&#xA;     az(i)=0.d0&#xA;     do j=1,n&#xA;        if (j==i) cycle&#xA;        r=dist(posx(i),posy(i),posz(i),posx(j),posy(j),posz(j))&#xA;        ax(i)=ax(i)+ Fx(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;        ay(i)=ay(i)+ Fy(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;        az(i)=az(i)+ Fz(r,posx(i)-posx(j),posy(i)-posy(j),posz(i)-posz(j),eps)/m&#xA;     enddo&#xA;  enddo&#xA;end subroutine acc&#xA;&#xA;!Updating Position&#xA;subroutine pos_upd(n,posx,posy,posz,velx,vely,velz,ax,ay,az,dt,xlen,ylen,zlen,rad)&#xA;  implicit none&#xA;&#xA;  integer n,i&#xA;  double precision,dimension(n) :: posx,posy,posz,velx,vely,velz,ax,ay,az&#xA;  double precision :: dt,xlen,ylen,zlen,rad&#xA;  do i=1,n&#xA;     posx(i)=posx(i) + velx(i)*dt + (ax(i)*(dt**2))/2&#xA;     posy(i)=posy(i) + vely(i)*dt + (ay(i)*(dt**2))/2&#xA;     posz(i)=posz(i) + velz(i)*dt + (az(i)*(dt**2))/2&#xA;&#xA;     if(posx(i)&amp;lt;rad .OR. posx(i)&amp;gt;xlen-rad) velx(i)=-velx(i)&#xA;     if(posy(i)&amp;lt;rad .OR. posy(i)&amp;gt;ylen-rad) vely(i)=-vely(i)&#xA;     if(posz(i)&amp;lt;rad .OR. posz(i)&amp;gt;zlen-rad) velz(i)=-velz(i)&#xA;  enddo&#xA;end subroutine pos_upd&#xA;&#xA;!Updating Velocity&#xA;subroutine vel_upd(n,velx,vely,velz,ax,ay,az,dt,tempx,tempy,tempz)&#xA;&#xA;  implicit none&#xA;  integer n,i&#xA;  double precision,dimension(n) :: velx,vely,velz,ax,ay,az,tempx,tempy,tempz&#xA;  double precision :: dt&#xA;  do i=1,n&#xA;     velx(i)=velx(i) + 0.5d0*(ax(i)+tempx(i))*dt&#xA;     vely(i)=vely(i) + 0.5d0*(ay(i)+tempy(i))*dt&#xA;     velz(i)=velz(i) + 0.5d0*(az(i)+tempz(i))*dt&#xA;  enddo&#xA;end subroutine vel_upd&#xA;&#xA;function pot(posx,posy,posz,eps,n)&#xA;  implicit none&#xA;  double precision pot,r,dist,eps&#xA;  integer i,j,n&#xA;  double precision, dimension(n) :: posx,posy,posz&#xA;  pot=0.d0&#xA;  do i=1,n&#xA;     do j=1,n&#xA;        if(i==j) cycle&#xA;        r=dist(posx(i),posy(i),posz(i),posx(j),posy(j),posz(j))&#xA;        pot= pot + (4.d0*eps*((1.d0/r)**12 - (1.d0/r)**6))                 !r is relative distance. x,y,z are components of r.&#xA;     enddo&#xA;  enddo&#xA;end function pot&#xA;&#xA;function kin(velx,vely,velz,m,n)&#xA;  implicit none&#xA;  double precision :: kin,m&#xA;  integer :: i,n&#xA;  double precision, dimension(n) :: velx,vely,velz&#xA;  kin=0.d0&#xA;  do i=1,n&#xA;     kin = kin + ((velx(i)**2.d0 + vely(i)**2.d0 + velz(i)**2.d0)/(2.d0*m))&#xA;  enddo&#xA;end function kin&#xA;&#xA;function dist(x1,y1,z1,x2,y2,z2)&#xA;  implicit none&#xA;  double precision :: dist,x1,y1,z1,x2,y2,z2&#xA;  dist = sqrt((x1-x2)**2.d0 + (y1-y2)**2.d0 + (z1-z2)**2)&#xA;end function dist&#xA;&#xA;function Fx(r,x,y,z,eps)&#xA;  implicit none&#xA;  double precision :: Fx,r,x,y,z,eps&#xA;  Fx = 4.d0*eps*((12.d0/r**14.d0) - (6.d0/r**8.d0))*x&#xA;end function Fx&#xA;&#xA;function Fy(r,x,y,z,eps)&#xA;  implicit none&#xA;  double precision :: Fy,r,x,y,z,eps&#xA;  Fy = 4.d0*eps*((12.d0/r**14.d0) - (6.d0/r**8.d0))*y&#xA;end function Fy&#xA;&#xA;function Fz(r,x,y,z,eps)&#xA;  implicit none&#xA;  double precision :: Fz,r,x,y,z,eps&#xA;  Fz = 4.d0*eps*((12.d0/r**14) - (6.d0/r**8))*z&#xA;end function Fz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16433" LastEditorUserId="9667" LastEditDate="2015-05-25T06:14:00.360" LastActivityDate="2015-05-25T06:14:00.360" Title="How to keep velocities in check in molecular dynamics simulation?" Tags="|computational-physics|time-integration|molecular-dynamics|integration|" AnswerCount="0" CommentCount="7" ClosedDate="2022-01-09T01:15:03.730" ContentLicense="CC BY-SA 3.0" />
  <row Id="19740" PostTypeId="1" AcceptedAnswerId="21652" CreationDate="2015-05-25T07:30:23.777" Score="3" ViewCount="1921" Body="&lt;p&gt;Is anybody aware of any GPU-enabled Lattice Boltzmann solvers (preferably on C++/OpenCL and open-source) that would be recommended?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found &lt;a href=&quot;http://asl.org.il/&quot; rel=&quot;nofollow&quot;&gt;Advanced Simulation Library&lt;/a&gt;, but it seems to be very young. Did anybody try to work with it and verify &lt;a href=&quot;http://asl.org.il/benchmarks/&quot; rel=&quot;nofollow&quot;&gt;their benchmarks&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="16442" LastEditorUserId="16442" LastEditDate="2015-12-24T20:24:40.693" LastActivityDate="2015-12-29T19:38:18.693" Title="GPU-enabled Lattice Boltzmann solvers?" Tags="|fluid-dynamics|parallel-computing|simulation|computational-physics|gpu|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19741" PostTypeId="1" CreationDate="2015-05-25T09:52:37.767" Score="1" ViewCount="934" Body="&lt;p&gt;I want to do LES of an oscillating flow i.e. a sinusoidal flow without a mean component in a channel.&#xA;In order to have a fully developed flow at lower mesh count without using a long channel I want to  map it back at a location from downstream or recycle from outlet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sinusoidal profile I used codedFixedValue in RANS.&#xA;For mapping I can use something like mappedpatch as in PitzDailyMapped tutorial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to combine the mapped inlet with codedFixedValue in some way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other suggestions which reduce computational effort are also welcome .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SM &lt;/p&gt;&#xA;" OwnerUserId="16444" LastActivityDate="2015-06-16T08:13:58.157" Title="oscillating flow inlet BC" Tags="|boundary-conditions|openfoam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19742" PostTypeId="1" AcceptedAnswerId="26605" CreationDate="2015-05-25T11:40:46.093" Score="10" ViewCount="442" Body="&lt;p&gt;I have read that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(1) Ill conditioned operations should be performed before well conditioned ones.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As an example, one should calculate $xz-yz$ as $(x-y)z$ since subtraction is ill conditioned while multiplication isn't. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a first-order error analysis of both algorithms reveals that they differ only by a factor of three (*), and I don't see why one can generalize this to statement (1), nor do I intuitively grasp the significance of order of operations. Do you think statement is (1) is an accepted rule, and do you have other explanations for it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*: more specifically, the first version has a relative error bounded by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\text{eps}+3\frac{|x|+|y|}{|x|-|y|}\text{eps}$$ while the relative error of the second version is bounded by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$3\text{eps}+\frac{|x|+|y|}{|x|-|y|}\text{eps}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\text{eps}$ is the machine precision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This analysis is based on the assumption that the $i$-th intermediate result is multiplied with $(1+\varepsilon_i)$ (due to rounding errors), where $\varepsilon_i$ are i.i.d. random variables bounded by $\text{eps}$. &quot;First-order&quot; means that higher-order terms, like $\epsilon_i\epsilon_jx$, are neglected.&lt;/p&gt;&#xA;" OwnerUserId="7789" LastEditorUserId="20688" LastEditDate="2017-04-11T19:20:25.137" LastActivityDate="2017-04-11T19:20:25.137" Title="Order of operations, numerical algorithms" Tags="|stability|floating-point|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19743" PostTypeId="2" ParentId="6822" CreationDate="2015-05-25T12:18:38.780" Score="1" Body="&lt;p&gt;In general hyperthreading is good if you have a of operations depending on results of the previous operations, because then, the pipeline can be filled with operations from the other thread until the required informations to continue are there. On the other hand, you half the size of available cache size per thread, so depending on your memory usage this can give a huge performance hit.&lt;/p&gt;&#xA;" OwnerUserId="16446" LastActivityDate="2015-05-25T12:18:38.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19745" PostTypeId="2" ParentId="8396" CreationDate="2015-05-25T17:54:33.307" Score="2" Body="&lt;p&gt;$$dx_{t}=2x_{t} dt +x_{t} dw_{t}\\x_{0}=1$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% EM Euler-Maruyama method on linear SDE  &#xA;%  &#xA;% SDE is dX = lambda*X dt + mu*X dW, X(0) = Xzero,  &#xA;% where lambda = 2, mu = 1 and Xzero = 1.  &#xA;%  &#xA;% Discretized Brownian path over [0,1] has dt = 2^(-8).  &#xA;% Euler-Maruyama uses timestep R*dt.  &#xA;&#xA;state=randn(100)  &#xA;lambda = 2;  &#xA;mu = 1;  &#xA;Xzero = 1;   % problem parameters  &#xA;T = 1;  &#xA;N = 2^8;  &#xA;dt = 1/N;  &#xA;dW = sqrt(dt)*randn(1,N); % Brownian increments  &#xA;W = cumsum(dW); % discretized Brownian path  &#xA;Xtrue = Xzero*exp((lambda-0.5*mu^2)*([dt:dt:T])+mu*W);  &#xA;plot([0:dt:T],[Xzero,Xtrue]), hold on  &#xA;R = 4;  &#xA;Dt = R*dt;  &#xA;L = N/R; % L EM steps of size Dt = R*dt  &#xA;Xem = zeros(1,L); % preallocate for efficiency  &#xA;Xtemp = Xzero;  &#xA;for j = 1:L  &#xA;Winc = sum(dW(R*(j-1)+1:R*j));  &#xA;Xtemp = Xtemp + Dt*lambda*Xtemp + mu*Xtemp*Winc;  &#xA;Xem(j) = Xtemp;  &#xA;end  &#xA;plot([0:Dt:T],[Xzero,Xem]), hold off &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16448" LastEditorUserId="9667" LastEditDate="2015-05-26T16:43:14.753" LastActivityDate="2015-05-26T16:43:14.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19746" PostTypeId="1" AcceptedAnswerId="19751" CreationDate="2015-05-25T18:50:41.723" Score="3" ViewCount="5791" Body="&lt;p&gt;My finite difference scheme for the 2D Euler equations is second order accurate in theory, since all the terms are second order accurate, with the advective terms being third order. So I expect a rate of convergence of about 2 when using successive refinements. However, I know that the discretization at the boundary reduces to first order (I am working on that). These are the values I compute (Dx=4 m, and coarser grid is 40X40 cells):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|------|-----------------------------------------------------------------&#xA;| grid | Norm L1  |  Rate L1 |  Norm L2  |  Rate L2 |  Norm Linf | Rate Linf  &#xA;|------|----------------------------------------------------------------&#xA;|Dx    | 0.0264   |    /     |0.0017799  |   /      |   0.149    |   /&#xA;|Dx/2  | 0.0088   |   1.58   |0.0003712  |  2.26    |   0.075    |  1.00 &#xA;|Dx/4  | 0.0032   |   1.40   |0.0000858  |  2.09    |   0.043    |  0.78 &#xA;|Dx/8  | 0.0009   |   1.70   |0.0000159  |  2.38    |   0.024    |  0.90 &#xA;|Dx/16 | 0.0003   |   1.45   |0.0000038  |  2.10    |   0.013    |  0.74 &#xA;|Dx/32 | 0.0001   |   1.51   |0.0000007  |  2.19    |   0.009    |  0.82 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Therefore, I do understand that my Linf norm is first order, since the max errors are at the boundary where the scheme is only first order accurate. Also, from my results it looks like the L2 norm averages the boundary error through the domain and blurs the information, therefore still giving a global second order of accuracy (or slightly above 2). What I do not understand is: why is the L1 norm converging with a rate of about 1.5? Or better why does the norm L2 blur the error at the boundary why the L1 does not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I compute the norms with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  L_1     &amp;amp;= \frac{1}{N}\sum_{j=1}^N|u^{numerical}_j-u^{exact}_j| \\&#xA;  L_2     &amp;amp;= \frac{1}{N}\sqrt{\sum_{j=1}^N(u^{numerical}_j-u^{exact}_j)^2} \;\;\;\;\;\;\;\;\;\;\;\; (1)\\&#xA;  L_{inf} &amp;amp;= \max|u^{numerical}_j-u^{exact}_j|&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u^{numerical}$ is the computed along-x velocity at velocity point $j$, with $j=1..N$, and $u^{exact}$ is the analytical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I am thinking that the way I am computiong $L_2$ norm might be wrong. See my related question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19749/correct-way-of-computing-norm-l-2-for-a-finite-difference-scheme&quot;&gt;Correct way of computing norm $L_2$ for a finite difference scheme&lt;/a&gt; . If I compute $L_2$ as &#xA;\begin{align}&#xA;  L_2     &amp;amp;= \sqrt{\frac{1}{N}\sum_{j=1}^N(u^{numerical}_j-u^{exact}_j)^2} \;\;\;\;\;\;\;\;\;\;\;\; (2)\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I obtain this convergence rates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|------|----------------------| &#xA;| grid |   Norm L2 |  Rate L2 |  &#xA;|------|----------------------| &#xA;|Dx    | 0.0308232 |   /      |  &#xA;|Dx/2  | 0.0133451 |  1.17    |    &#xA;|Dx/4  | 0.0065001 |  1.01    |   &#xA;|Dx/8  | 0.0024768 |  1.36    |    &#xA;|Dx/16 | 0.0011984 |  1.08    |     &#xA;|Dx/32 | 0.0005142 |  1.20    |   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Question is: either if I use equation (1) or (2)  the rate of convergence in $L_2$ is pretty different (respectively higher or lower) than the one $L_1$, which clashes with Leveque's statement &quot;Except in fairly rare cases, the 1-norm and 2-norm will give similar results&quot; (&lt;a href=&quot;https://books.google.com/books/about/Finite_Volume_Methods_for_Hyperbolic_Pro.html?id=QazcnD7GUoUC&amp;amp;hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://books.google.com/books/about/Finite_Volume_Methods_for_Hyperbolic_Pro.html?id=QazcnD7GUoUC&amp;amp;hl=en&lt;/a&gt;). Is this a rare case, in which the accuracy is locally smaller at boundary?&lt;/p&gt;&#xA;" OwnerUserId="10032" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-05-26T01:17:01.047" Title="Why a finite difference scheme would give second order of accuracy in norm L2 but 1.5 with L1 (while 1 with Linf)?" Tags="|finite-difference|error-estimation|accuracy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19747" PostTypeId="1" AcceptedAnswerId="19748" CreationDate="2015-05-25T19:59:55.987" Score="0" ViewCount="2046" Body="&lt;p&gt;I am working on a modified interior point algori thm for semidefinite for my special problem. I don't have enough skills and knowledge about interior point for semidefinite to code it from scratch. Also, I don't have enough time for that. I need a simple open source implementation of semidefinite interior point algorith to change. There are some open source codes available but I want a simple yet efficient code. &#xA;What is the best code to modify it's interior point algorithm easily to demonstrate usefulness of the proposed algorithm? I highly prefer matlab codes or codes in C++ with matlab interface, easily attached to matlab. &#xA;Any answer, comments or suggestion is highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8223" LastActivityDate="2018-03-26T16:51:29.320" Title="What are open source codes for interior point optimization to modify?" Tags="|optimization|convex-optimization|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19748" PostTypeId="2" ParentId="19747" CreationDate="2015-05-25T21:01:54.417" Score="3" Body="&lt;p&gt;A lot depends on what features you need to have in the code and whether it will be reasonable to make your modification to the code.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you need block diagonal structured variables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;second-order cones?  Rotated second-order cones?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Free variables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Special support for low-rank constraint matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Support for log det optimization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need a lot of these features then you will probably want to start with one of the well developed primal-dual codes.  The most well know primal-dual interior point codes for SDP are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;SeDuMi.   This code uses a self-dual embedding approach that may or not be reasonable to modify a for your purposes.  It is written in MATLAB with C mex subroutines.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SDPA.  Uses an infeasible interior point method.  Written in C++.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;SDPT3. Infeasible interior point method written in MATLAB plus C.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;CSDP.  Infeasible interior point method, written in C.  Disclaimer: I'm the author of CSDP.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Of these, perhaps SDPT3 would be of the most interest to you,&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2018-03-26T16:51:29.320" LastActivityDate="2018-03-26T16:51:29.320" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19749" PostTypeId="1" AcceptedAnswerId="19750" CreationDate="2015-05-25T21:48:03.310" Score="2" ViewCount="6692" Body="&lt;p&gt;I am computing the rate of convergence of my finite difference scheme in norm $L_2$. Which is the correct way to compute it? This:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  L_2     &amp;amp;= \frac{1}{N}\sqrt{\sum_{j=1}^N(u^{numerical}_j-u^{exact}_j)^2}        \;\;\;\;\;\;\;\;\;\;\;\; (1)\\ &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  L_2     &amp;amp;= \sqrt{\frac{1}{N}\sum_{j=1}^N(u^{numerical}_j-u^{exact}_j)^2} \;\;\;\;\;\;\;\;\;\;\;\; (2)\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u^{numerical}$ is the computed along-x velocity at velocity point $j$, with $j=1..N$, and $u^{exact}$ is the analytical solution. In some publications I saw the second option, but that clashes against the fact that norm $L_2$ should be always smaller than norm $L_1$ (see &lt;a href=&quot;https://math.stackexchange.com/questions/245052/showing-that-l2-norm-is-smaller-than-l1&quot;&gt;https://math.stackexchange.com/questions/245052/showing-that-l2-norm-is-smaller-than-l1&lt;/a&gt; ). However here norms are &quot;scaled&quot; so I guess that inequality should hold only for the unscaled version. Note that if one uses (1) the inequality still holds. &#xA;In Leveque's book a pag 140 the error is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  L_2     &amp;amp;= \sqrt{\Delta{x} \sum_{j=1}^N(u^{numerical}_j-u^{exact}_j)^2} \;\;\;\;\;\;\;\;\;\;\;\; (3)\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this would be strictly true for a finite volume method, if we extend it to a finite difference method and to two dimension, by division by the total active area of the domain one gets equation (2). This question is related to question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19746/why-a-finite-difference-scheme-would-give-second-order-of-accuracy-in-norm-l2-bu&quot;&gt;Why a finite difference scheme would give second order of accuracy in norm L2 but 1.5 with L1 (while 1 with Linf)?&lt;/a&gt; , since I see that the order of accuracy is larger than the thoretical value of 2 if I use equation (1), while it is smaller if I use equation (2). Which one is the best formulation? &lt;/p&gt;&#xA;" OwnerUserId="10032" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-05-26T01:03:08.953" Title="Correct way of computing norm $L_2$ for a finite difference scheme" Tags="|finite-difference|error-estimation|accuracy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19750" PostTypeId="2" ParentId="19749" CreationDate="2015-05-26T00:57:25.433" Score="7" Body="&lt;h2&gt;Use equation (2), equation (1) is wrong.&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Technically the $L^2$ norm (upper case &quot;L&quot;) is an integral norm of a function &lt;a href=&quot;http://mathworld.wolfram.com/L2-Norm.html&quot;&gt;defined as&lt;/a&gt;&#xA;$$&#xA;\left|\left|f(x)\right|\right|_2 = \sqrt{ \int_\Omega |f(x)|^2 dx}.&#xA;$$&#xA;I'm sure you meant $l^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you typically want to compute in the context of convergence of numerical methods is the finite dimensional analog of the $L^2$ norm which is an area-normalized version of the $l^2$ norm, meaning&#xA;$$&#xA;\left|\left|f\right|\right| = \sqrt{\sum_i \left|f_i\right|^2 \Delta x_i}&#xA;$$&#xA;Note that $\Delta x$ is inside the square root.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the area of the domain is just a constant that you don't really care about and the step size ($\Delta x$) is often a constant so you can just use your equation (2). Equation (1) is off by a factor of $1/\sqrt{N}$ so you will underestimate your error for large $N$.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-05-26T01:03:08.953" LastActivityDate="2015-05-26T01:03:08.953" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19751" PostTypeId="2" ParentId="19746" CreationDate="2015-05-26T01:17:01.047" Score="3" Body="&lt;p&gt;&lt;strong&gt;Your initial method for computing the norms was incorrect&lt;/strong&gt;&lt;br&gt;&#xA;(see &lt;a href=&quot;https://scicomp.stackexchange.com/a/19750/7733&quot;&gt;my answer to your related question&lt;/a&gt;)&lt;br&gt;&#xA;The norm was incorrect by a factor of $1/\sqrt{N}$, leading to underestimating the error as $N$ increases and giving the appearance of a higher-than-expected convergence rate. I would expect that both L1 and L2 norms give somewhere between orders 1 and 2 for convergence which appears to be the case in your update.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-05-26T01:17:01.047" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19753" PostTypeId="1" CreationDate="2015-05-26T11:14:51.750" Score="2" ViewCount="505" Body="&lt;p&gt;I am solving a generalized eigenvalues problem with inversed complex shift:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(M-\sigma J)^{-1}J \boldsymbol{x} = \boldsymbol{x} \nu \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My matrices are obtained from a finite element discretization with Getfem++. The jacobian $J$ depends on some parameters. I computed the eigenvalues both with Matlab and directly with Arpack in the same C++ code where I compute the matrices $M$ and $J$. &#xA;When I use Matlab to compute the eigenvalues I load the matrices $M$ and $J$ previously saved in Market Matrix format with Getfem++ and I use the function &lt;code&gt;eigs&lt;/code&gt;.&#xA;Now, in some cases the difference between Arpack eigenvalues and Matlab eigenvalues is really big. In particular, when the values of my parameters are such that I have an eigenvalue with almost zero real part. &#xA;With Arpack the real part is of the order of $10^{-10}$ with Matlab is of the order of $10^{-4}$. In addition, in these cases Matlab gives me some warning about the matrix is ill-conditioned. &#xA;Have you ever experienced such a problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. : I also tried to change default options of eigs but I always obtain the same result!&lt;/p&gt;&#xA;" OwnerUserId="14489" LastEditorUserId="9667" LastEditDate="2015-05-26T14:27:26.180" LastActivityDate="2015-05-26T14:27:26.180" Title="Arpack and Matlab give different values for eigenvalues" Tags="|matlab|finite-element|eigenvalues|complex-arithmetic|arpack|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="19754" PostTypeId="1" CreationDate="2015-05-26T12:27:23.837" Score="2" ViewCount="780" Body="&lt;p&gt;I am looking for a method to automatically solve custom PDEs on a custom control volume. Specifically I would like to solve equations similar (but not exactly alike) to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial y}{\partial t} = \frac{\partial^2}{\partial x} y^n + A \frac{\partial y}{\partial x}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to what I read this is what OpenFOAM is for. The problem is that I cannot find a comprehensive guide on how to program things like this is OpenFOAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the only answer for &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19145/add-user-defined-custom-differential-equations-in-openfoam-cfd&quot;&gt;this question&lt;/a&gt; there wasn't, indeed, such a guide by the time that answer was posted. (apart from a small review that doesn't aim to teach)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, however, it is three years later and I'm thinking there might be more documentation about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: is OpenFOAM really the best approach for what I want or are there better ways of doing this? If it is the best approach, is a there now a guide on how to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="16453" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-05-28T02:22:31.423" Title="openfoam - Programming customized PDEs" Tags="|finite-element|finite-difference|fluid-dynamics|openfoam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19755" PostTypeId="1" CreationDate="2015-05-26T15:21:26.030" Score="-1" ViewCount="364" Body="&lt;p&gt;I have a problem where I am trying to solve many systems of equations, that have very few variables per equation, but a lot of equations.  For example potentially 10 variables max in a single equation, but a million equations.  A lot of them are trivial in finding a solution, but some of the larger equations are difficult to solve.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a naive algorithm in mind to simply solve the trivial equations first, and hope those solutions can then just translate into solving more and more etc, until the more complicated problems are completely solved.  Is this a solid initial approach?  Is there a different way of thinking about this type of problem that I do not know about?  I have tried looking through this SE and similar but I tend to get confused once people start mentioning sparse matrices verse dense ones and things such as that.  If someone could point me in the right direction that would be awesome.&lt;/p&gt;&#xA;" OwnerUserId="16457" LastEditorUserId="1804" LastEditDate="2016-05-27T15:47:07.237" LastActivityDate="2016-05-27T15:47:07.237" Title="Solving large system of equations, is linear programming best option?" Tags="|matrix|linear-system|" AnswerCount="1" CommentCount="4" ClosedDate="2016-05-29T03:49:59.840" ContentLicense="CC BY-SA 3.0" />
  <row Id="19756" PostTypeId="1" CreationDate="2015-05-26T15:32:18.750" Score="2" ViewCount="157" Body="&lt;p&gt;I'm trying to numerically find periodic solutions to different systems of autonomous nonlinear ordinary differential equations. I decided to use a finite difference scheme and solve the resulting system of equations iteratively, which works reasonably well using a sufficiently good initial guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now most of the systems considered depend on an additional parameter $\lambda$, with a solution in the limit $\lambda\to 0$ easily found, so I would like to use some sort of numerical continuation method to generate a family of solutions for different values of $\lambda$. All literature I found on the subject, e.g. &lt;a href=&quot;http://dx.doi.org/10.1137/1.9780898719154&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1137/1.9780898719154&lt;/a&gt;, assumes the Jacobian to have full rank, which it does not in my case: There is a zero eigenvalue corresponding to a shift in the parametrization of the solution to the original system of ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ODEs typically look like&#xA;$$&#xA; \ddot{x} = - \sqrt{\dot{x}^2+\dot{y}^2}\ f(x, y, \lambda)\ \dot{y},\\&#xA; \ddot{y} = \sqrt{\dot{x}^2+\dot{y}^2}\ f(x, y,\lambda)\ \dot{x},&#xA;$$&#xA;with the constraints $x(0) = x(1)$, $y(0) = y(1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there either&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a good way to somehow fix the parametrization of the solution to remove this degree of freedom or&lt;/li&gt;&#xA;&lt;li&gt;a numerical continuation method that works with a rank deficient Jacobian?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16456" LastEditorUserId="1280" LastEditDate="2015-05-27T09:28:42.610" LastActivityDate="2015-05-27T09:28:42.610" Title="Rank deficient Jacobian in discretized periodic solutions to autonomous ODE" Tags="|linear-algebra|finite-difference|ode|nonlinear-equations|jacobian|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19757" PostTypeId="1" CreationDate="2015-05-26T18:41:26.440" Score="4" ViewCount="539" Body="&lt;p&gt;So I have a point cloud that I am creating a 3D flat rectangular surface from.  I'm then turning it into a hollow box and connecting the corners by just dropping this surface mesh down.  I need it to be consistently a meter thick throughout so it is important to drop the surface mesh down so it matches the 1 meter delta z from the top.  So I end up with something that looks like this: &lt;img src=&quot;https://i.stack.imgur.com/1MUBP.png&quot; alt=&quot;surface mesh dropped for time with side connectivites&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that tetgen even when I'm allowing it to doesn't want to seem to throw away points.  So I would like a very refined mesh at top and a coarse one at the bottom.  It wants to keep all of the bottom points, so if I drop the mesh one time 1 meter down and give it a high volume constraint I get something like this: &#xA;&lt;img src=&quot;https://i.stack.imgur.com/cdtSO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see it is refined at the top but it needs to go back to being refined at the bottom to match the surface mesh.  I don't want tetgen to keep these facets.  I'm wanting it to get coarser as it goes down.  Does anyone know how to fix this problem?  Or maybe some other software that is better suited to my needs?  Thanks for your help and for taking a look.  I'm most proficient in python, so I have used meshpy in the past as well. &lt;/p&gt;&#xA;" OwnerUserId="16459" LastEditorUserId="1280" LastEditDate="2015-05-26T19:06:12.210" LastActivityDate="2015-05-26T19:06:12.210" Title="Large meshing with tetgen" Tags="|python|mesh-generation|unstructured-mesh|" AnswerCount="0" CommentCount="2" ClosedDate="2021-07-09T15:20:19.613" ContentLicense="CC BY-SA 3.0" />
  <row Id="19759" PostTypeId="2" ParentId="19740" CreationDate="2015-05-27T04:53:45.183" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://code.google.com/p/sailfish-cfd/&quot; rel=&quot;nofollow&quot;&gt;Sailfish&lt;/a&gt; is a great OS LBM solver that meets your needs.  Sailfish is developer in the Python language in order to provide a rapid development environment; however, it compiles optimized OpenCL or CUDA via the pyopencl and pycuda libraries.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found it to be highly flexible with remarkable computational speeds.  Inherently, no computation is performed with Python.  The down-side to Sailfish is the very limited user community; however, if you're down the LBM path anyways, you're likely used to that.&lt;/p&gt;&#xA;" OwnerUserId="16463" LastEditorUserId="16463" LastEditDate="2015-05-28T03:14:00.423" LastActivityDate="2015-05-28T03:14:00.423" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19760" PostTypeId="2" ParentId="19754" CreationDate="2015-05-27T05:24:01.977" Score="3" Body="&lt;p&gt;I think OpenFOAM is pretty well suited for these problems; however, the best place to send your questions are to the CFD forum &lt;a href=&quot;http://www.cfd-online.com/Forums/openfoam/&quot; rel=&quot;nofollow&quot;&gt;where much of the OPF community resides&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also find some more guidance &lt;a href=&quot;http://cfd.direct/openfoam/user-guide/programming-language-openfoam/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend starting with something simple, like &lt;a href=&quot;https://github.com/OpenFOAM/OpenFOAM-2.2.x/blob/master/applications/solvers/basic/potentialFoam/potentialFoam.C&quot; rel=&quot;nofollow&quot;&gt;this example&lt;/a&gt;, to learn how to program your PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, I would comment that there is no, strictly speaking, best approach to this kind of problem.  OpenFOAM has a number of numerical techniques available to you as problems become stiffer and more difficult.  In addition, you have a number of pre-processing tools at your disposal.  The disadvantage is that it has a modest learning curve and it is developed by the CFD community, which has different needs. On this note, you might consider a tools such as &lt;a href=&quot;https://code.google.com/p/futureye/&quot; rel=&quot;nofollow&quot;&gt;futureye&lt;/a&gt; or &lt;a href=&quot;http://mooseframework.org/&quot; rel=&quot;nofollow&quot;&gt;MOOSE&lt;/a&gt;.  For your application, MOOSE would be my recommended choice. &lt;/p&gt;&#xA;" OwnerUserId="16463" LastEditorUserId="16463" LastEditDate="2015-05-28T02:22:31.423" LastActivityDate="2015-05-28T02:22:31.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19761" PostTypeId="2" ParentId="19618" CreationDate="2015-05-27T05:27:33.590" Score="0" Body="&lt;p&gt;Also, here is a pretty extensive discussion with results of various numerical schemes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.holzmann-cfd.de/index.php/en/numerical-schemes&quot; rel=&quot;nofollow&quot;&gt;http://www.holzmann-cfd.de/index.php/en/numerical-schemes&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16463" LastActivityDate="2015-05-27T05:27:33.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19763" PostTypeId="1" AcceptedAnswerId="19766" CreationDate="2015-05-27T22:29:58.597" Score="6" ViewCount="1121" Body="&lt;p&gt;I am using Heun's method with a third order upwind spatial scheme, which is suggested by Shao (2008) to be used for solving the horizontal advection part of the advection-diffusion equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I got:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA; &amp;amp;C^{\ast} = C^{n} - A^{n} \delta t\\&#xA; &amp;amp;C^{n+1} = C^{n} - \dfrac{1}{2}\left( A^{n} + A^{\ast} \right) \delta t&#xA; \end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming $u=cte&amp;gt;0$, we have,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; A = \dfrac{u}{\delta x} ( \dfrac{1}{6} C_{i-2} - C_{i-1} + \dfrac{1}{2} C_{i} + \dfrac{1}{3} C_{i+1} )&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; \hat{C}_{\ast} = \hat{C}_{n} - \dfrac{u \delta t}{\delta x} ( \dfrac{1}{6} e^{-2imh} - e^{-imh} + \dfrac{1}{2} + \dfrac{1}{3} e^{imh} ) \hat{C}_{n}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; \hat{C}_{\ast} =  \left[ 1 - r F(h) \right] \hat{C}_{n}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In which,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; r = \dfrac{u \delta t}{\delta x}&#xA;\end{equation}&#xA;\begin{equation}&#xA; F(h) = \dfrac{1}{2}+\dfrac{1}{6} e^{-2imh} - e^{-imh} + \dfrac{1}{3} e^{imh}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; \hat{C}_{n+1} = \hat{C}_{n} - \dfrac{r}{2} F(h) \hat{C}_{n} - \dfrac{r}{2} F(h) \hat{C}_{\ast}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; g = \dfrac{\hat{C}_{n+1}}{\hat{C}_{n}} = 1 - r F(h) - \dfrac{r^2}{2} F^2(h) &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enforcing&#xA;\begin{equation}&#xA; |g| \leq 1&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;should give the stability conditions. I have drawn $g$'s graph&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sps4z.png&quot; alt=&quot;g&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and it seems the scheme is always unstable. But I know it is not. What am I doing wrong? Do you have any suggestions? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Shao, Yaping. Physics and modelling of wind erosion. Springer, 2008.&lt;/p&gt;&#xA;" OwnerUserId="14772" LastEditorUserId="14772" LastEditDate="2015-05-28T13:18:58.410" LastActivityDate="2015-05-30T00:03:11.480" Title="Stability analysis of Heun's method" Tags="|finite-difference|numerical-analysis|stability|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19765" PostTypeId="1" AcceptedAnswerId="19770" CreationDate="2015-05-28T03:35:18.263" Score="0" ViewCount="1134" Body="&lt;p&gt;I am approximating the expression $\left\|Ax-b\right\|_1$ by the expression&#xA;$$\text{minimize}\;\;\sum_i\sqrt{(a_i^Tx-b_i)^2+\varepsilon}$$&#xA;where $a_i$ is the $i^{th}$ row of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This function is convex, and I was wondering if there was a way to convert it into an SOCP, or if it's hopelessly nonlinear.&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="6849" LastEditDate="2015-05-28T14:40:34.650" LastActivityDate="2015-05-28T14:40:34.650" Title="Converting smooth $L1$ norm approximation into SOCP" Tags="|optimization|convex-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19766" PostTypeId="2" ParentId="19763" CreationDate="2015-05-28T05:39:21.243" Score="8" Body="&lt;p&gt;Notice that $\hat C_*=(1-r F(h))\hat C_n$, but the sign in front of $r$ is lost when you use $\hat C_*$ inside $\hat C_{n+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking at p.149 of Numerical Solution of Time-Dependent Advection-Diffusion-Reaction equations by Hundsdorfer and Verwer on Google books. (I'm going to use their signs.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stability region of Heun's method is &#xA;$$|g(z)| = \big|1+z+\frac12z^2\big|&amp;lt; 1, $$&#xA; and if I substitute &#xA;$$ z = \nu h(\omega) = \nu\big(-\tfrac16 e^{-2i\omega} + e^{-i\omega}-\tfrac12-\tfrac13e^{i\omega} \big), $$&#xA;and ask Mathematica to plot the values of $g(\nu h(\omega))$ for $|\omega|&amp;lt;\pi$, $0&amp;lt;\nu&amp;lt;1$, it gives me&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/oHnXI.png&quot; alt=&quot;Stability plot&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which I think is what you should get instead of your plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, to find the maximal value of $\nu$, I solved $\partial_\omega^4\big |_{\omega=0} |g(\nu h(\omega))| = 0$ (which isn't quite rigorous as this uses a Taylor series approximation), and got &#xA;$$ 0.87358046473629887, $$&#xA;matching Hundsdorfer and Verwer's number of 0.87 in Table 1.2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since flipping the signs in your expressions appears to solve the issue and matches Hundsdorfer &amp;amp; Verwer, I think that's the error here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[&lt;em&gt;Edit to explain 0.87&lt;/em&gt;] For any fixed $\nu$, it you plot the function $\omega\mapsto |g(\nu h(\omega))|$, it will have its maximum either at the origin (value 1), or it will be convex at the origin and the maximum will be slightly to the side (plot it and see). The first three derivatives of $|g(\nu h(\omega))|$ w.r.t. $\omega$ at $\omega=0$ vanish identically for all $\nu$, so the function can be approximated as $1+\frac1{24}\omega^4 \partial_\omega^4$. The condition that its maximum value be $1$ can then be approximated (although I think without proof that it's also exactly correct) by the condition that the first non-constant non-zero Taylor term $\partial_\omega^4\big|_{\omega=0}$ has a maximum at $\omega=0$, so to find the maximal value of $\nu$ it is enough to solve $0 = \partial_\omega^4\big|_{\omega=0}$ because it's negative for smaller values of $\nu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sage: from mpmath import *&#xA;sage: h = lambda w: -(mpf(&quot;1/6&quot;)*expj(-2*w)-expj(-w)+mpf(&quot;1/2&quot;)+expj(w)/3)&#xA;sage: g = lambda z: 1+z+z**2/2&#xA;sage: findroot(lambda t: diff(lambda w: abs(g(t * h(w))), 0, 4), 0.87)&#xA;mpf('0.87358046473629887')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-05-30T00:03:11.480" LastActivityDate="2015-05-30T00:03:11.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19767" PostTypeId="1" CreationDate="2015-05-28T06:39:04.223" Score="0" ViewCount="37" Body="&lt;p&gt;I'm looking for an algorithm to generate discrete-time pseudorandom sequences based on other sequences I already have that would be used to train the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, I have some time-based discrete-time measurements of some real-live event (for example: sunspot number, stock market, sea height at a beach...). I have long sequences of those measurements and I would like to generate other sequences having the same long-term and short-term properties that the sequences I already have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the case of the sunspot number, it shows some random short-term noise and a clear long-term periodicity. I would like to produce more like those sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been looking at Markov chains, but they would be valid just for the short-term, and just for a discrete set of values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't look like a new problem so I suppose it has been solved elsewhere and has a name :) but I can't find it.&lt;/p&gt;&#xA;" OwnerUserId="7405" LastActivityDate="2015-08-26T08:23:00.133" Title="Generating pseudorandom sequences based on measured sequences" Tags="|random-number-generation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19768" PostTypeId="2" ParentId="19767" CreationDate="2015-05-28T07:09:06.877" Score="-1" Body="&lt;p&gt;You can model your time series with Gaussian processes. Once you have decided on the correct covaraince functions and estimated their hyperparameters you can draw new series from this process. But I agree with Kirill, this is only one of many possible ways to do this.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2015-05-28T07:09:06.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19769" PostTypeId="1" CreationDate="2015-05-28T08:05:34.647" Score="1" ViewCount="428" Body="&lt;p&gt;I have a collection of time intervals with integer valued endpoints , e.g. (1,2), (2, 6) (5,6), and  the number of events falling in each time interval, and I would like to determine if from them I can extract a mutually exclusive set of intervals covering the same period, in this case (1,2), (2, 5), (5,6), where the number of events falling in (2,5) is calculated by taking the difference of the number in (2,6) and (5,6).&#xA;There is no limit on the number of possible overlaps, e.g they could all be of the form (n, 24).&#xA;I would like to know if this is a standard problem with a standard solution.&lt;/p&gt;&#xA;" OwnerUserId="16469" LastActivityDate="2015-07-29T23:30:40.010" Title="extract exhaustive mutually exclusive intervals from larger set of intervals" Tags="|algorithms|sorting|" AnswerCount="1" CommentCount="0" ClosedDate="2015-07-30T00:32:53.720" ContentLicense="CC BY-SA 3.0" />
  <row Id="19770" PostTypeId="2" ParentId="19765" CreationDate="2015-05-28T12:06:59.460" Score="2" Body="&lt;p&gt;The original 1-norm minimization problem can easily be converted to an LP (SOCP) using standard techniques.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is quite odd to convert the smoothed problem into an SOCP, but you could do this as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{i}=[a_{i}^{T}x-b_{i} \;\; \sqrt{\epsilon}]^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_{i} \geq \| u_{i} \|_{2} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \sum_{i} t_{i}$&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-05-28T12:06:59.460" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19771" PostTypeId="1" CreationDate="2015-05-28T12:23:58.627" Score="0" ViewCount="149" Body="&lt;p&gt;I'm following the article at &lt;a href=&quot;http://www.paykin.info/irina/project_2.jsp&quot; rel=&quot;nofollow&quot;&gt;http://www.paykin.info/irina/project_2.jsp&lt;/a&gt;&#xA;Finite Difference method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to interpret this one? How to convert this to pseudo code?&#xA;$$u(i,j+1) = 2u(i,j) + \left(\frac{k}{h}\right)^2[u(i+1,j) - 2u(i,j) + u(i-1,j)] - u(i,j-1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the ff given values (Im not sure if values are possible).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Initial Condition:&#xA;u(i,0)=sin(i)&#xA;du(i,0)/dt=0&#xA;&#xA;Boundary Condition:&#xA;u(0,j)=0&#xA;u(95,j)=0&#xA;&#xA;dt=1&#xA;dx=5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the result when t=5.&lt;/p&gt;&#xA;" OwnerUserId="16470" LastEditorUserId="16470" LastEditDate="2015-05-29T01:30:36.690" LastActivityDate="2015-05-29T01:30:36.690" Title="Finite Difference method" Tags="|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19773" PostTypeId="2" ParentId="19771" CreationDate="2015-05-28T15:30:32.110" Score="3" Body="&lt;p&gt;The term on LHS is the variable's value at next time step i.e. (j+1) which depends on the values of variable u at previous time steps j and (j-1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pseudo code could be like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initialize u(i,j) and let it equal to u(i,j-1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For (all grid points except those on boundary), use the discretized equation to advance in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For boundary nodes, use the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have to write in C with i varying from 0 to N, I can write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i = 0; i &amp;lt;= N; i++)&#xA;{&#xA;     if (i &amp;gt; 0 &amp;amp;&amp;amp; i &amp;lt; N)&#xA;     {&#xA;          unew[i] = 2*u[i] + (dt/dx)*(dt/dx)*(u[i+1] - 2*u[i] + u[i-1]) - uold[i]&#xA;     }&#xA;     else&#xA;     {&#xA;         unew[i] = u[i]&#xA;     }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. If still got some query, feel free to ask.&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2015-05-28T15:30:32.110" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19774" PostTypeId="1" CreationDate="2015-05-28T15:35:15.510" Score="1" ViewCount="134" Body="&lt;p&gt;I am using adaptive mesh refinement to solve one dimensional inviscid Burgers equation. However I am facing some difficulty to handle grid interfaces which are not uniform (coarse-fine grid interface). There has to be conservation of variable at the grid interfaces which I am not able to incorporate in my code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has any idea about it or has done the similar work based preferably on finite difference schemes, it could be a great help to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0724063&quot; rel=&quot;nofollow&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/0724063&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2015-05-28T15:35:15.510" Title="Conservation at grid interface in adaptive mesh refinement" Tags="|fluid-dynamics|simulation|computational-physics|mesh|adaptive-mesh-refinement|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19775" PostTypeId="1" AcceptedAnswerId="19776" CreationDate="2015-05-28T15:55:26.077" Score="0" ViewCount="177" Body="&lt;p&gt;I have used free vibration analysis in FEM. After analysis, we can usually use animation to see the motion of each eigenmode (In Abaqus or Comsol, I would  choose either half harmonic or full harmonic). Does anyone know the principle behind it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my understanding, eigenmode analysis already removes the time factor, i.e., $\exp(i\omega t)$. If so, how can FEM show animation?&lt;/p&gt;&#xA;" OwnerUserId="16474" LastEditorUserId="9667" LastEditDate="2015-05-28T18:03:50.473" LastActivityDate="2015-05-28T18:49:54.117" Title="How does the animation work in eigenvalue problem of FEM" Tags="|finite-element|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19776" PostTypeId="2" ParentId="19775" CreationDate="2015-05-28T18:49:54.117" Score="3" Body="&lt;p&gt;The &lt;em&gt;solution&lt;/em&gt; for a particular mode is of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_i(x,t) = w_i(x)\sin(\omega_i t + \phi)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $w_i(x)$ is the $i$th eigenfunction and $\omega_i$ the $i$th eigenvalue, $\phi$ is a phase and, we can assume $\phi=0$. Then, the &lt;em&gt;solution&lt;/em&gt; for the mode is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_i(x,t) = w_i(x)\sin(\omega_i t) \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, you can present your mode as an animation changing the amplitude proportionally to the sine function presented. In commercial software it is common to have the option to vary $t \in [0,\pi]$ and $t \in [0,2\pi]$, and they are termed &lt;em&gt;half cycle&lt;/em&gt; or &lt;em&gt;full cycle&lt;/em&gt; (or &lt;em&gt;half harmonic&lt;/em&gt; and &lt;em&gt;full harmonic&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correct thing to do is to animate over the full range $[0,2\pi]$, but since the amplitude will be the same (but opposite sign) in the other half some people do not consider it necessary.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-05-28T18:49:54.117" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19777" PostTypeId="1" CreationDate="2015-05-28T21:18:06.983" Score="2" ViewCount="296" Body="&lt;p&gt;I probably miss something very basic. I don't see how to use Metropolis–Hastings algorithm for computation of integrals, if I &lt;strong&gt;don't know the volume of accessible phase space&lt;/strong&gt; (i.e. proper normalization by partition function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider this two problems ( which are very much connected):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) Number of possible states under some constrains&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some phase-space with some areas &lt;em&gt;possible&lt;/em&gt; ( probability $p(X) = 1$ ) and some &lt;em&gt;impossible&lt;/em&gt; ( probability $p(X) = 0$ ). I don't know the volume of neither areas, but I can easily check if a point is in &lt;em&gt;possible&lt;/em&gt; or &lt;em&gt;inpossible&lt;/em&gt; area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I know that area of &lt;em&gt;impossible&lt;/em&gt; part of phase-space is much bigger than area of &lt;em&gt;possible&lt;/em&gt; part, I wan't to use Metropolis–Hastings algorithm to preferentially explore the &lt;em&gt;possible&lt;/em&gt; configurations. I can easily implement the updates according to Metropolis–Hastings to produce Markov Chain which does explore the &lt;em&gt;possible&lt;/em&gt; part of phase-space, &lt;strong&gt;however, I don't see how to use it to actually compute the &lt;em&gt;possible&lt;/em&gt; area&lt;/strong&gt; by summing over such Markov chain. I still don't know proper normalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a javascript visualization of the problem in 2D&#xA;&lt;a href=&quot;http://www.openprocessing.org/sketch/200710&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.openprocessing.org/sketch/200710&lt;/a&gt;&#xA;the most relevant part of code is there:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// in this example probability of configuration is p=0 &#xA;// if the x,y point overlap with some sphere, otherwise it is p=1&#xA;float getUnnormalizedProbability( float x, float y ){&#xA;  float prob = 1.0;&#xA;  for(int i=0; i&amp;lt;nsph; i++){  &#xA;    float dx = x - sphx[i];&#xA;    float dy = y - sphy[i];&#xA;    float r2 = dx*dx + dy*dy;&#xA;    if( r2 &amp;lt; sphR2[i] ){&#xA;      prob = 0.0;&#xA;      break;&#xA;    }&#xA;  }&#xA;  return prob;&#xA;}&#xA;&#xA;void MMCstep(){&#xA;  float newx = x + random(-stepsz,stepsz);&#xA;  float newy = y + random(-stepsz,stepsz);&#xA;  // Periodic boundary condition&#xA;  if ( newx&amp;lt;0      ) newx = boxsz + newx;&#xA;  if ( newx&amp;gt;boxsz  ) newx = newx  - boxsz;&#xA;  if ( newy&amp;lt;0      ) newy = boxsz + newy;&#xA;  if ( newy&amp;gt;boxsz  ) newy = newy  - boxsz;&#xA;  float newp = getUnnormalizedProbability( newx, newy );&#xA;  // Metropolis-Hastings condition  alfa =  min( 1, pnew/p );&#xA;  if( newp &amp;gt;= p ){&#xA;    move( newx, newy, newp );&#xA;  }else{&#xA;    if( (newp/p) &amp;gt; random(1.0) ){&#xA;      move( newx, newy, newp );&#xA;    }&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) Neutron scattering on finite plates&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have source of radiation $S$ and detector $D$ (of some size) and several thin plates of neutron scattering material (of some size and with some normalized angular scattering distribution). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to compute percentage of neutrons emitted by $S$ which impact on $D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be computed by path-integration over paths by randomly choosing points $i$ and $j$ on plates. This however require renormalization of the sum by probability that a neutron originating in point $i$ impact second plate at all, because I don't sample situations where neutron does not hit the plate, I don't know this renormalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explicity analytical computation of this probability renormalization (i.e. solid angle under which plat_2 is visible from point $i$ ) is quite laboriously. I don't want to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if Metropolis–Hastings algorithm can help with this somehow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/NVZCR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4696" LastActivityDate="2015-07-28T19:15:08.260" Title="Metropolis Monte Carlo integration of Area with unknown normalization" Tags="|monte-carlo|probability|integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19778" PostTypeId="1" CreationDate="2015-05-29T01:13:42.550" Score="2" ViewCount="959" Body="&lt;p&gt;I was trying to simulate the precession of Mercury based on the perturbed solution: &#xA;$$\frac{1}{r}=\frac{m}{B^{2}}(1+e\cos\phi+3\frac{m^{2}}{B^{2}}(1+e\phi \sin\phi +e^{2}(\frac{1}{2}-\frac{1}{6}\cos2\phi))),$$ where $e$ is the eccentricity of mercury, B is a constant which is the angular momentum per mass. &#xA;here is my code in Python: &lt;a href=&quot;http://sprunge.us/TFhU&quot; rel=&quot;nofollow&quot;&gt;http://sprunge.us/TFhU&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code I have to multiply a large number(SF) so that the precession is slightly visible(or you can make a bigger SF value then the precession would be much easier to see but it would distort the orbits). &#xA;I guess it is okay if I use this as a illustration for the mercury precession, however, I want to be able to obtain the value of the precession rate which is known as 43 arcseconds per century, does anyone know how to do that in the code or anyone has python code that does a better job? (I also came through a model based on these two equations: $$r_{n+1} = 2r_{n}-r_{n-1}+\Delta \tau^{2}(-\frac{GM}{{r_{n}}^{2}}+\frac{B^{2}}{{r_{n}}^{3}}-\frac{3GMB^{2}}{{r_{n}}^{4}}),\phi_{n+1}=\phi_{n}+\Delta\tau\frac{B}{[\frac{1}{2}(r_{n+1}+r_{n})]^{2}},$$ where B is same as above, but I couldn't get it to work.)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(Update: I used Vpython and Leapfrog method to do it and the simulation worked out all right. here is the code and modify it as however you want to,&#xA;  &lt;a href=&quot;http://sprunge.us/ehgS&quot; rel=&quot;nofollow&quot;&gt;http://sprunge.us/ehgS&lt;/a&gt; (in linux the sphere doesn't leave trail probably because the vpython in linux is very out of date). &#xA;  However, I still need to find a way to output the precession rate 43&quot; out of the data.)   &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16476" LastEditorUserId="16476" LastEditDate="2015-06-05T22:43:40.663" LastActivityDate="2015-06-05T22:43:40.663" Title="Precession of Mercury Python simulation" Tags="|python|computational-physics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19779" PostTypeId="1" CreationDate="2015-05-29T05:04:49.667" Score="1" ViewCount="326" Body="&lt;p&gt;I am facing a problem as follows:&#xA;I have created a Voronoi diagram and now I want to assign to each region, a value, i.e., I want to extend this Voronoi diagram to form a 2 variable function in Matlab. For example, let one of the cells be named $R_1$, I want that the 2 variable function I am going to assign a constant value (e.g., $f(x,y)=4$) to all the points in $R_1$. Similarly for the other cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I am including the information about how Voronoi diagram is being stored herein. What I am using is &lt;code&gt;voronoin&lt;/code&gt; function that returns the vertices of polygons and cells with corresponding vertices of which they are made up of. Also, since the edges partitioning in Voronoi diagrams are straight lines, it is possible to trace out the cells.&lt;/p&gt;&#xA;" OwnerUserId="16478" LastEditorUserId="9667" LastEditDate="2015-05-29T19:38:10.067" LastActivityDate="2015-05-29T19:38:10.067" Title="Assigning values to different regions in matlab" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19780" PostTypeId="1" AcceptedAnswerId="19784" CreationDate="2015-05-27T00:48:53.220" Score="2" ViewCount="6941" Body="&lt;p&gt;I am &lt;strong&gt;self teaching&lt;/strong&gt; myself python and computational physics via Mark Newmans book Computational Physics the exercise is 2.9 of Computational Physics I have to compute the &lt;a href=&quot;http://en.wikipedia.org/wiki/Madelung_constant&quot; rel=&quot;nofollow noreferrer&quot;&gt;Madelung&lt;/a&gt; constant. &lt;img src=&quot;https://i.stack.imgur.com/QjT51.png&quot; alt=&quot;enter image description here&quot;&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have &lt;strong&gt;two different solutions&lt;/strong&gt; worked out I wanted to know which code is correct and is there a way to exclude &lt;strong&gt;0&lt;/strong&gt; in my range so I don't have to write two separate for loops. I think my second answer is correct can somebody verify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My first answer:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M = 0&#xA;&#xA;for i in range(-200,0):&#xA;    for j in range(-200,0):&#xA;        for k in range(-200,0):&#xA;&#xA;            M += ((-1)**(i+j+k))/((i**2 + j**2 + k**2)**(1/2))&#xA;&#xA;print(M)           &#xA;for i in range(1,200):&#xA;    for j in range(1,200):&#xA;        for k in range(1,200):&#xA;&#xA;            M += ((-1)**(i+j+k))/((i**2 + j**2 + k**2)**(1/2)) &#xA;&#xA;print(M)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My second answer:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M = 0&#xA;i = 0&#xA;j = 0&#xA;k = 0&#xA;for l in range(-200,0):&#xA;    i = l&#xA;    j = l&#xA;    k = l&#xA;    M += ((-1)**(i+j+k))/((i**2 + j**2 + k**2)**(1/2))&#xA;&#xA;print(M)&#xA;&#xA;for l in range(1,200):&#xA;    i = l&#xA;    j = l&#xA;    k = l&#xA;    M += ((-1)**(i+j+k))/((i**2 + j**2 + k**2)**(1/2))&#xA;&#xA;print(M)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9607" OwnerDisplayName="FireFistAce" LastActivityDate="2022-05-15T22:52:48.990" Title="Computing the Madelung constant" Tags="|python|" AnswerCount="5" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19781" PostTypeId="2" ParentId="19780" CreationDate="2015-05-27T01:10:38.500" Score="3" Body="&lt;p&gt;There &lt;em&gt;is&lt;/em&gt; a huge, obvious difference between the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one loops over every combination of negative &lt;code&gt;i, j, k&lt;/code&gt; values, and every combination of positive &lt;code&gt;i, j, k&lt;/code&gt; values. The second only loops over &lt;code&gt;i == j == k&lt;/code&gt; values. So, for example, the first one has &lt;code&gt;-200, -200, -200&lt;/code&gt;, then &lt;code&gt;-200, -200, -199&lt;/code&gt;, and so on to &lt;code&gt;-200, -200, -1&lt;/code&gt;, then &lt;code&gt;-200, -199, -200&lt;/code&gt;, and so on; the second just goes right from &lt;code&gt;-200, -200, -200&lt;/code&gt; to &lt;code&gt;-199, -199, -199&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I have no idea which one is correct or how to verify it, because that's a physics question, not a Python question.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, there is a Python question hidden in there:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;is there a way to exclude 0 in my range so I don't have to write two separate for loops. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sure. You want an iterable over all the values in &lt;code&gt;range(-200, 0)&lt;/code&gt; and all the values in &lt;code&gt;range(1, 200)&lt;/code&gt;. You can do that by just &lt;a href=&quot;https://docs.python.org/3/library/itertools.html#itertools.chain&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;chain&lt;/code&gt;&lt;/a&gt;ing the two iterables together:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import itertools&#xA;&#xA;for i in itertools.chain(range(-200, 0), range(1, 200)):&#xA;    print(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, if you prefer, just merge them manually and explicitly skip the 0:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(-200, 200):&#xA;    if i == 0:&#xA;        continue&#xA;    print(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;By the way, are you sure you wanted &lt;code&gt;range(1, 200)&lt;/code&gt;, not &lt;code&gt;range(1, 201)&lt;/code&gt;? It seems strange that &lt;code&gt;-200&lt;/code&gt; is inside the set of values, but &lt;code&gt;200&lt;/code&gt; isn't. But maybe that's correct.&lt;/p&gt;&#xA;" OwnerDisplayName="abarnert" LastActivityDate="2015-05-27T01:10:38.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19782" PostTypeId="2" ParentId="19779" CreationDate="2015-05-29T11:31:30.693" Score="2" Body="&lt;p&gt;&lt;strong&gt;You can use the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/griddata.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;griddata&lt;/code&gt;&lt;/a&gt; function with the &lt;code&gt;'nearest'&lt;/code&gt; option to accomplish this. For better performance with multiple calls to your function use a kd-tree nearest neighbor search.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use the &lt;code&gt;griddata&lt;/code&gt; method, simply define a set of values, &lt;code&gt;F&lt;/code&gt; for the cells and use the points that determined the Voronoi diagram &lt;code&gt;X, Y&lt;/code&gt; for nearest neighbor interpolation. To do this you will obviously need to know the points that defined the Voronoi diagram. I'm guessing you have those points since you said you're using the &lt;code&gt;voronoin&lt;/code&gt; function to produce the Voronoi diagram. You can define your function as &lt;code&gt;f = @(x_in,y_in) griddata(X,Y,F,x_in,y_in,'nearest');&lt;/code&gt; and then call it as &lt;code&gt;f(x,y)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to do this for many separate calls to your function it will be much faster to define a search tree and then re-use it. If you have the statistics toolbox you can &lt;a href=&quot;http://www.mathworks.com/help/stats/kdtreesearcher-object.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;do this with KD Trees as follows&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;X=rand(15,1);&#xA;Y=rand(15,1);&#xA;kdtree = KDTreeSearcher([X(:),Y(:)]);&#xA;F(:,1)=(1:15);&#xA;f = @(x_in,y_in) F( kdtree.knnsearch([x_in(:),y_in(:)]) );&#xA;x=rand(1000,1);&#xA;y=rand(1000,1);&#xA;fvals = f(x,y);&#xA;figure(1)&#xA;clf&#xA;hold on&#xA;voronoi(X,Y,'kx')&#xA;scatter(x,y,100,fvals,'.')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/IuFks.png&quot; alt=&quot;Scatter plot with Voronoi cells&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;Note:&lt;br&gt;&#xA;I think any solution that explicitly searches for which polygonal Voronoi cell contains a point will be much slower than this method. If you do not know the points that determined the Voronoi diagram it might be possible to determine them from the cells.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-05-29T14:09:45.933" LastActivityDate="2015-05-29T14:09:45.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19784" PostTypeId="2" ParentId="19780" CreationDate="2015-05-29T16:56:57.040" Score="4" Body="&lt;p&gt;Below is the code (in C, though) for the kind of loop you can write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  for (int n = 1; n &amp;lt;= L; n++) {&#xA;    for (int i = -n; i &amp;lt;= n; i++) {&#xA;      for (int j = -n; j &amp;lt;= n; j++) {&#xA;        for (int k = -n; k &amp;lt;= n; k++) {&#xA;          if (abs(i) != n &amp;amp;&amp;amp; abs(j) != n &amp;amp;&amp;amp; abs(k) != n)&#xA;            continue;&#xA;&#xA;          // Your update to the potential energy goes here.&#xA;        }&#xA;      }&#xA;    }&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Observe that this loop iterates over cubic shells of radius $n$.  This is the way to properly compute the Madelung constant (see Section III of [1] for details).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Borwein, D., Borwein, J. M., &amp;amp; Taylor, K. F. (1985). Convergence of lattice sums and Madelung’s constant. Journal of Mathematical Physics, 26(11), 2999. doi:10.1063/1.526675&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-05-29T16:56:57.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19785" PostTypeId="2" ParentId="19777" CreationDate="2015-05-29T18:02:50.167" Score="2" Body="&lt;p&gt;I don't think you're missing anything, MCMC is used to sample points&#xA;from a given distribution, known up to a normalization constant, and&#xA;to evaluate expectations w.r.t. that distribution (not integrals over&#xA;the volume).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's often used when the normalization constant of the distribution is&#xA;not known, and doesn't require you to know it. On the other hand, if&#xA;you had two distributions, you could estimate the ratio of their&#xA;normalizing constants. Since you can pick any MC, you can pick one&#xA;with a different stationary distribution $q$ with a known&#xA;normalization constant, then use that to estimate $\mathbb{E}^Q[p/q]$&#xA;which would compute the integral $\int p(x)\,dx$ over the domain of&#xA;$q$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to your comment, I can think of a few things to try, but I find it&#xA;difficult to be definitive here. I think at some point you might just&#xA;have to compute the probability that a neutron hits plate 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simulating paths that don't hit plate 1 or hit plate 1 but don't hit&#xA;plate 2 should work: depending on how big the plates are, this might&#xA;work just fine. This should let you know analytically the appropriate normalization constant. I don't see too much reason to avoid this at all&#xA;costs: for one thing, whatever your final method is, you could also&#xA;compute this probability by letting $D$ have infinite size and letting&#xA;$S$ and plate 1 have point sizes and maybe close to each other, then&#xA;the probability is mostly the probability of plate 2 being hit from&#xA;the point plate 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do sample points that don't lie on plate 2, you don't have to sample them uniformly from all of the plane in which it lies: e.g., you can pick some Gaussian distribution approximately centered on top of plate 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if computing the solid angle is really the main thing stopping&#xA;you from normalizing your functions, then it should be possible to do&#xA;it numerically. Looking at&#xA;&lt;a href=&quot;http://en.wikipedia.org/wiki/Solid_angle#Tetrahedron&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;,&#xA;there is a formula for solid angle subtended by a triangle with given&#xA;vertices, so to calculate the solid angle subtended by plate 2 from&#xA;point $i$, you can just split plate 2 into a suitable number of&#xA;triangles and sum up individual solid angles for each triangle. This&#xA;might be the most straightforward solution here.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-05-29T18:02:50.167" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19786" PostTypeId="1" AcceptedAnswerId="19946" CreationDate="2015-05-29T20:21:44.520" Score="14" ViewCount="1468" Body="&lt;p&gt;Multigrid (MG) may be used to solve a linear system &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; by constructing an initial guess &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; and repeating the following for &lt;span class=&quot;math-container&quot;&gt;$i=0,1..$&lt;/span&gt; until convergence:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute the residual &lt;span class=&quot;math-container&quot;&gt;$r_i = b-Ax_i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Apply a multigrid cycle to obtain an approximation &lt;span class=&quot;math-container&quot;&gt;$\Delta x_i \approx e_i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Ae_i = r_i$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Update &lt;span class=&quot;math-container&quot;&gt;$x_{i+1} \gets x_i + \Delta x_i$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The multigrid cycle is some sequence of smoothing, interpolation, restriction, and exact coarse grid solve operations applied to &lt;span class=&quot;math-container&quot;&gt;$r_i$&lt;/span&gt; to produce &lt;span class=&quot;math-container&quot;&gt;$\Delta x_i$&lt;/span&gt;. This is typically a V-cycle or a W-cycle. This is a linear operation so we write &lt;span class=&quot;math-container&quot;&gt;$\Delta x_i = B r_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;One can interpret this process as preconditioned Richardson iteration. That is, we update &lt;span class=&quot;math-container&quot;&gt;$x_{i+1} \gets x_i + B r_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Richardson iteration is a prototypical Krylov subspace method, which suggests the use of multigrid cycles to precondition other Krylov subspace methods. This is sometimes called &amp;quot;accelerating&amp;quot; multigrid with a Krylov method, or alternately can be seen as a choice of a preconditioner for a Krylov method.&lt;/p&gt;&#xA;&lt;p&gt;Another way to extend the algorithm above is to employ Full Multigrid (FMG). See &lt;a href=&quot;https://scicomp.stackexchange.com/a/4935/3824%20this%20answer&quot;&gt;this answer&lt;/a&gt; for a concise description.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;In what situations is Krylov-accelerated MG preferable to MG or FMG?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2021-03-14T18:28:05.470" LastActivityDate="2021-03-14T18:28:05.470" Title="How is Krylov-accelerated Multigrid (using MG as a preconditioner) motivated?" Tags="|linear-algebra|linear-solver|multigrid|krylov-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19789" PostTypeId="2" ParentId="14513" CreationDate="2015-05-30T15:11:59.777" Score="0" Body="&lt;p&gt;I suppose your problem (log-likelihood from a Potts model) is convex. For very large scale problems, using Hessian (second order) information may not be scalable and/or efficient. There is a growing body of work on first order methods for large scale optimization problems. They are designed to be scalable and efficient. Most first order methods are parallelizable. Refer to these surveys as a starting point:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/pdf/1411.0972.pdf&quot; rel=&quot;nofollow&quot;&gt;Convex optimization for big data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/pdf/1406.5429.pdf&quot; rel=&quot;nofollow&quot;&gt;Playing with duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14478" LastActivityDate="2015-05-30T15:11:59.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19791" PostTypeId="1" CreationDate="2015-05-31T02:58:10.667" Score="0" ViewCount="581" Body="&lt;p&gt;How to discretize the following 4th order PDE using finite difference method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial^{2} y}{\partial t^{2}}+\frac{\partial^{4} y}{\partial x^{4}}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks&lt;/p&gt;&#xA;" OwnerUserId="16470" LastEditorUserId="16397" LastEditDate="2015-05-31T07:03:20.640" LastActivityDate="2015-05-31T07:08:16.747" Title="Finite Difference for Fourth-Order PDE" Tags="|finite-difference|numerical-analysis|approximation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19792" PostTypeId="2" ParentId="19791" CreationDate="2015-05-31T04:16:28.117" Score="3" Body="&lt;p&gt;You can use central difference scheme for both of the time and space derivatives. For time derivative, three time levels would come in to picture i.e. n, n-1 and n+1 where n is the existing time level and n+1 is the one you want to get for the next time step. Use second order accurate space derivative and that would include five grid points ranging from i-2 to i+2. This would be a second order accurate method. Use your boundary conditions and observe that the second and second last point will also need special attention as they can not be calculated with this method. Use B.C. wisely. This surely will do it! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: &lt;a href=&quot;http://en.wikipedia.org/wiki/Finite_difference_coefficient&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Finite_difference_coefficient&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16397" LastEditorUserId="16397" LastEditDate="2015-05-31T07:08:16.747" LastActivityDate="2015-05-31T07:08:16.747" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19793" PostTypeId="1" AcceptedAnswerId="19794" CreationDate="2015-05-31T04:34:25.543" Score="15" ViewCount="3974" Body="&lt;p&gt;Since this question is related to computation, I decided to post here. Hopefully it will be seen as appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've just started running atmospheric and oceanic models, and I realize that I need more cores, memory, and disk space than my current desktop has. My institution has a high performance computing service, where researchers can rent cores at a fixed price per core per month. No one in my research group has used this service, but instead run jobs their beefed up machines. However, several other research groups in the department pool funds and take advantage of the computing service. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I need to make a decision as to whether to purchase a new machine with 60 GB of RAM, or rent cores from the computing service. What factors should I consider in making this decision? What are the advantages of using a computing service than to purchase a dedicated machine?&lt;/p&gt;&#xA;" OwnerUserId="15220" LastActivityDate="2015-06-20T13:38:17.110" Title="Should I rent computing resources, or buy my own computers" Tags="|hpc|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19794" PostTypeId="2" ParentId="19793" CreationDate="2015-05-31T07:08:33.093" Score="19" Body="&lt;p&gt;Ultimately, the answer to this question depends on the prices being charged for the services that you need.  At some very low price, this would almost certainly be better than buying your own computer, while at some higher price you would be better off buying your own computer.  The case for using a shared resource is pretty strong though and these factors might overwhelm the amortized cost of the computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;No system administration hassle.  Even if you have the skills (and you probably don't unless you've recently had a full time job doing this), administering and securing the computer takes time.  Since it doesn't take much more time to manage 1,000 computers than it takes to manage 1, there is a huge economy of scale.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Busy/idle efficiency.  Unless your research is very unusual, you probably don't have a constant 24/7/365 workload for your computer.  You might go on vacation for a few weeks and not have any jobs to run during that time, or you might be preparing for a conference and have lots of extra work.  A shared resource handles these situations well.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Access to expert advice.  Your shared resource might come with free help from an expert at the computing center.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Cons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You don't own the resource, so you might lose access or have to pay a higher price in the future.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-05-31T07:08:33.093" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19795" PostTypeId="2" ParentId="19793" CreationDate="2015-05-31T18:06:40.697" Score="12" Body="&lt;p&gt;$6.60/core-month is less than a penny a core-hour. This is a good deal, and it's a better deal than you can get if you buy identical hardware yourself and pay your own power and sysadmin bill. If all you are going to do is buy one probably less powerful workstation node with sufficient RAM, then you may do better than this, but you may also complete your work faster on the better hardware offered by your central service organization. They presumably only charge you when you are actually running, so you should compare the running cost of a single computation, to the cost of what you can buy on your own. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will you admin this box, or will you pay someone? If you will pay a grad student to do this, then will this lengthen their stay in grad school? Is that fair? Who pays your power bill? Can you effectively cool your purchase, or will you have to upgrade the A/C in your office? Who works on the machine if it breaks, and can you afford to be down while it's down? How many core-months can you get for the cost of the machine you might buy? Etc.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-05-31T18:06:40.697" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19796" PostTypeId="2" ParentId="19793" CreationDate="2015-05-31T18:59:55.777" Score="5" Body="&lt;h2&gt;Will you be using it all the time, for a long time?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In general, the economics for this are simple - if you need a valuable resource for short intermittent bursts, it will generally be cheaper to rent than own; and if you expect to use most of it for a prolonged time then it will be cheaper to own it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple rule of thumb actually is about the terms used - is a core-month a meaningful metric for you, one where you expect to be using a lot of cores for a lot of months in a stable manner? It may be reasonable to own the resource in this case; however if you would describe your load in core-hours (even if the total amount would be the same) then it would be efficient to share it with others by, for example, renting it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, there is a scaling advantage - if for your work amount the cost of buying and renting come out similar, then there is still a big difference between scheduling a 1200 core-hour job to be run on the 12 cores of a server you bought and getting a response in 4 days and being able to run it on rented 1200 cores in a single hour, if your tasks parallelize well.&lt;/p&gt;&#xA;" OwnerUserId="9999" LastActivityDate="2015-05-31T18:59:55.777" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19798" PostTypeId="1" AcceptedAnswerId="19829" CreationDate="2015-05-31T20:36:39.213" Score="4" ViewCount="231" Body="&lt;p&gt;I have a non-convex optimization problem in the form:&#xA;\begin{align}&#xA;\min_{b,\xi,\eta} \sum_{i=1}^{n} b_i \xi_i + \gamma \Vert \eta \Vert \cr&#xA;\text{s.t.} b\geq 0, b^\mathsf{T} 1 = 1,b_i \leq \frac{1}{n-n_m}, \xi \geq 0 , \xi_i \geq 1-a_i^\mathsf{T} \eta\cr&#xA;\end{align}&#xA;In &lt;a href=&quot;http://stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf&quot; rel=&quot;nofollow&quot;&gt;Boyd and Vandenberghe's Convex Optimization&lt;/a&gt;, Appendix B (p. 653), it is said that non-convex quadratic with only one quadratic constraint has strong duality, i.e. you can find it's dual and then find the optimal of the dual and it is equal to non-convex quadratic global optimal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : Can I do this procedure for the above problem and how? Is it the best method for solving the above problem with respect to $b,\xi$? &lt;/p&gt;&#xA;" OwnerUserId="8223" LastEditorUserId="276" LastEditDate="2015-06-03T00:03:54.220" LastActivityDate="2015-06-03T10:02:32.540" Title="non-convex quadratic with only one quadratic constraint?" Tags="|optimization|nonconvex|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19799" PostTypeId="1" CreationDate="2015-05-31T22:12:13.053" Score="2" ViewCount="181" Body="&lt;p&gt;I want to find the frequencies of vibration of a circular and square drum. To do this, I need to solve a 2-dimensional wave equation (PDE) with boundary conditions. Every method that I have researched to solve this uses separation of variables to generate two ODE's, and this is referred to as the &quot;eigenvalue problem&quot;. My question is, is there any other way to do this that does not use separation of variables? I am looking for a numerical method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find approximate eigenvalues using a numerical method (like finite differences in 2-dimensions) but not sure if it is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any information would be appreciated&lt;/p&gt;&#xA;" OwnerUserId="16504" LastActivityDate="2015-06-01T03:14:01.107" Title="Finding the frequencies of vibration of a circular and square drum" Tags="|pde|numerical-analysis|boundary-conditions|elliptic-pde|wave-propagation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19800" PostTypeId="1" AcceptedAnswerId="19802" CreationDate="2015-06-01T00:14:12.113" Score="3" ViewCount="1600" Body="&lt;p&gt;I am currently a .NET software developer(SQL Server, ASP.NET, C#, MVC &amp;amp; Web Forms). In my spare time I'm researching different areas of environmental science. E.g.(Hydrology, Ecology, Atmospheric Science...) I'd eventually like to be working in a scientific field contributing in some way to the preservation and healthy, efficient function of our home. Clean water is my biggest interest. I wanted to say all that so you could get a feel of what type of sci-computing I could be doing; I might be working with Weather models or hydrology models. I've also stumbled upon BOINC(Berkley Open Infrastructure for Network Computing) and wouldn't mind have my machine work with some Distributed Scientific Computing. E.g. &lt;a href=&quot;http://www.climateprediction.net/&quot; rel=&quot;nofollow&quot;&gt;http://www.climateprediction.net/&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of processing power do I need? Could anyone recommend a sample build?&lt;/p&gt;&#xA;" OwnerUserId="16505" LastActivityDate="2015-06-01T03:12:13.797" Title="Computer Build for Scientific Computing" Tags="|computational-physics|discretization|computational-chemistry|hpc|modeling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19801" PostTypeId="2" ParentId="19793" CreationDate="2015-06-01T01:55:59.900" Score="2" Body="&lt;p&gt;Renting the service is the better choice. Computer hardware loses its value faster than cars (when it comes to value measure by its computing power), its value decreases exponentially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That means over time your initial investment of buying the cores will actually near 0. So the money you spend in renting the hardware isn't that much worse than buying the hardware. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, in your case, it seems like what you gain from investment is not the computing power but on the results from using the computing power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your decision is an economic decision, look at it this way: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Money spent = M&lt;/li&gt;&#xA;&lt;li&gt;Value from results = R&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Value over time of computing Power = P.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Renting has an expected value (EV) of: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EV(M) = R&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing to consider in case of renting, is that the same investment M will give you more computer power over time, because the same technology becomes exponentially cheaper over time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the case of (b) buying the cores your EV is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EV(M) = R + P&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that the results you are getting are valuable, and that the value of those results is higher than the value of the cores (i.e. R &gt;&gt; P), then the value of P becomes irrelevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem is that P over time tends to become 0 because technology becomes obsolete at an exponential rate, having its value near 0 as time passes, so if you integrate EV over Time t you get: EV over T(M) = R&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is easy to see that buying your own cores has very little expected value added to your investment (M).&lt;/p&gt;&#xA;" OwnerUserId="16506" LastEditorUserId="16506" LastEditDate="2015-06-20T13:38:17.110" LastActivityDate="2015-06-20T13:38:17.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19802" PostTypeId="2" ParentId="19800" CreationDate="2015-06-01T03:12:13.797" Score="10" Body="&lt;p&gt;Many of us in scientific computing simply have well-equipped laptops for regular software development tasks, some multicore workstations for smaller-scale testing, and access to clusters for larger runs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you an idea:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;My laptop is a Dell M3800 (4-core Intel i7, hyperthreading, 16GB of RAM). This is good enough to regularly compile my software and do small-scale tests with a few hundred thousand unknowns.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;My lab has a few 64-core AMD Opteron processors and 128 or 256 GB of RAM. For myself, one of these would be good enough, but they are shared with colleagues and my students and postdocs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We have a ~1200 core cluster and a ~16,000 core cluster on campus to which I have access. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I believe that many others in my field have similar setups.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-01T03:12:13.797" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19803" PostTypeId="2" ParentId="19799" CreationDate="2015-06-01T03:14:01.107" Score="1" Body="&lt;p&gt;You can of course solve the partial differential eigenvalue problem that describes the shape of the eigenmode numerically, using finite differences of finite elements. For the square and the circle that is pointless since the solution is well known, but it allows you to generalize the approach to other domains.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-01T03:14:01.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19805" PostTypeId="1" AcceptedAnswerId="19806" CreationDate="2015-06-01T12:20:27.597" Score="1" ViewCount="59" Body="&lt;p&gt;In essence we can create a cluster based on a number of machines(e.g., Beouwulf style). Here the question is, do all the cluster nodes require the exact same cpu(s) specification (e.g., core, cache, etc.)? If not how the companies can scale their cluster units, they just by hundreds or thousands units at the same time?&lt;/p&gt;&#xA;" OwnerUserId="16510" LastActivityDate="2015-06-01T14:29:29.750" Title="CPU Specification Within Each Node of a Cluster" Tags="|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19806" PostTypeId="2" ParentId="19805" CreationDate="2015-06-01T12:34:52.620" Score="2" Body="&lt;p&gt;Nothing requires the nodes of a cluster to be uniform. Most batch scheduling software (SLURM, LSF, SGE, PBS, etc.) allows the end user to specify requirements for the nodes that will be used to run their jobs. So if the user needs a certain amount of RAM or a certain number of cores per node, then the scheduler can select nodes to satisfy those requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the end, most cluster builders make them fairly uniform. Clusters are often bought all at once, up front, with a small number of node configurations. Our biggest systems at TACC tend to have one main compute node variety that makes up the bulk of the system, and 2-3 other compute node configurations to support special needs. E.g., we have 16 1TB large-memory nodes with more 32 cores on &lt;a href=&quot;https://www.tacc.utexas.edu/systems/stampede&quot; rel=&quot;nofollow&quot;&gt;Stampede&lt;/a&gt; that are available in addition to the 6400 compute nodes with 32GB of RAM and 16 cores per node. We also have 128 nodes on the system that have an NVIDIA GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Companies may buy all their nodes upfront or buy them in increments depending on their needs. Software can help them manage the complexity. However, most people buy in large chunks. The pricing just works out better this way. It may cost less to acquire the nodes in a big bunch from the vendor, and it certainly saves installation time to do it all at once. These systems are usually run for 3-5 years and then retired. Other systems may be purchased and installed during this time.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-06-01T14:29:29.750" LastActivityDate="2015-06-01T14:29:29.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19807" PostTypeId="1" CreationDate="2015-06-01T12:34:59.687" Score="6" ViewCount="3217" Body="&lt;p&gt;I am trying to solve a linear equation system $\textbf{A}\textbf{x}=\textbf{b}$, e.g. a Poisson equation discretized in strong form, using &lt;em&gt;biCGstab&lt;/em&gt; method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since there are only natural Neumann boundary conditions, at least one Dirichlet condition must be given. I simply add condition $\sum_{j} x=0$ to row $i$ in $\textbf{A}$. The result converges, but the error in the gradient field $\nabla x$ at $i$ is very large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the reason for this, and how should I imposition this condition?&lt;/p&gt;&#xA;" OwnerUserId="16325" LastEditorUserId="16325" LastEditDate="2015-06-01T17:00:24.363" LastActivityDate="2020-04-26T06:50:58.460" Title="Solving a linear equation system with pure Neumann condition" Tags="|boundary-conditions|nonlinear-equations|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19808" PostTypeId="1" CreationDate="2015-06-01T14:07:57.140" Score="-2" ViewCount="455" Body="&lt;p&gt;I have been given the following homework question to solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2AMVi.png&quot; alt=&quot;image of equations&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having trouble writing a RK4 solver in C++ for this ODE. I am also not sure how to plot my solution. Here is my code so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt; //Header file for cin &amp;amp; cout&#xA;#include &amp;lt;cmath&amp;gt;    //Header file for mathematical operartions&#xA;#include &amp;lt;iomanip&amp;gt;  //Header file for precession&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;using namespace std; //calling the standard directory&#xA;float F(float y, float x)&#xA;{ return 0; }&#xA;&#xA;&#xA;int main()&#xA;{&#xA;  float y0,x0,z0,y1, z1,n,h,f,k1,k2,k3,k4, j1, j2, j3, j4;&#xA;  cout&amp;lt;&amp;lt;&quot;\nEnter the value of x0 (initial value): &quot;; //Entering the initial values of x &amp;amp; y&#xA;  cin&amp;gt;&amp;gt;x0;&#xA;  cout&amp;lt;&amp;lt;&quot;\nEnter the value of y0 (initial value): &quot;;&#xA;  cin&amp;gt;&amp;gt;y0;&#xA;  cout&amp;lt;&amp;lt;&quot;\nEnter the value of z : &quot;;&#xA;  cin&amp;gt;&amp;gt;z0;&#xA;  cout&amp;lt;&amp;lt;&quot;\nEnter the value of h (step size): &quot;;//Entering the width of step size&#xA;  cin&amp;gt;&amp;gt;h;&#xA;  cout&amp;lt;&amp;lt;&quot;\nEnter the value of last point of x: &quot;;&#xA;  cin&amp;gt;&amp;gt;n;&#xA;  cout&amp;lt;&amp;lt;&quot;\nStart point (x,y) : &quot;&amp;lt;&amp;lt;x0&amp;lt;&amp;lt;&quot;, &quot;&amp;lt;&amp;lt;y0&amp;lt;&amp;lt;endl;&#xA;&#xA;  for( ; x0&amp;lt;n; x0=x0+h){&#xA;    f=F(x0,y0);&#xA;    k1 = h * f;&#xA;    f = F(x0+h/2,y0+k1/2);&#xA;    k2 = h * f;&#xA;    f = F(x0+h/2,y0+k2/2);&#xA;    k3 = h * f;&#xA;    f = F(x0+h/2,y0+k2/2);&#xA;    k4 = h * f;&#xA;&#xA;    k1=z0;&#xA;    j1=-x0*y0;&#xA;    k2=z0+(0.5)*j1;&#xA;    j2=-(x0+(0.5)*h)*(y0+(0.5)*k1);&#xA;    k3=z0+(0.5)*j2;&#xA;    j3=-(x0+(0.5)*h)*(y0+(0.5)*k2);&#xA;    k4=z0+j3;&#xA;    j4=-(x0+h)*(y0+k3);&#xA;&#xA;    y1=y0+(h/6)*(k1+2*(k2)+2*(k3)+k4);&#xA;    z1=z0+(h/6)*(j1+2*(j2)+2*(j3)+j4);&#xA;&#xA;    cout&amp;lt;&amp;lt;&quot;\n k1 = &quot;&amp;lt;&amp;lt;k1;&#xA;    cout&amp;lt;&amp;lt;&quot;\n k2 = &quot;&amp;lt;&amp;lt;k2;&#xA;    cout&amp;lt;&amp;lt;&quot;\n k3 = &quot;&amp;lt;&amp;lt;k3;&#xA;    cout&amp;lt;&amp;lt;&quot;\n k4 = &quot;&amp;lt;&amp;lt;k4;&#xA;    cout&amp;lt;&amp;lt;&quot;\n j1 = &quot;&amp;lt;&amp;lt;j1;&#xA;    cout&amp;lt;&amp;lt;&quot;\n j2 = &quot;&amp;lt;&amp;lt;j2;&#xA;    cout&amp;lt;&amp;lt;&quot;\n j3 = &quot;&amp;lt;&amp;lt;j3;&#xA;    cout&amp;lt;&amp;lt;&quot;\n j4 = &quot;&amp;lt;&amp;lt;j4&amp;lt;&amp;lt;endl;&#xA;    cout&amp;lt;&amp;lt;&quot;\n x = &quot;&amp;lt;&amp;lt;x0+h&amp;lt;&amp;lt;&quot; and y = &quot;&amp;lt;&amp;lt;y1&amp;lt;&amp;lt;endl;&#xA;    y0=y1; //take new point for next calculation&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16489" LastEditorUserId="20688" LastEditDate="2022-12-13T23:01:09.503" LastActivityDate="2022-12-13T23:02:05.177" Title="How to solve this differential equation using RK4 in C++?" Tags="|ode|c++|numerical-modelling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19809" PostTypeId="1" CreationDate="2015-06-01T17:56:35.563" Score="1" ViewCount="33" Body="&lt;p&gt;I have a real symmetric function $C(x,y)$ defined on $x,y\in[0,\infty)$, i.e. $C(x,y)=C(y,x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve the eigenvalues problem, i.e. find eigen values and eigen functions:&#xA;$$\lambda \psi(x)=\int_0^\infty C(x,y)\psi(y)dy$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This operator corresponds to the covariance matrix such as $C_{i,j}=Cov[x_i,x_j]$, i.e. it's positive definite, real and symmetric. In the functional case, I'm not sure what is the analog of positive definite. I'm trying to extend the covariance matrix into the infinite dimensional (continuous) case from the matrix version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for references on eigenvalues problem solutions, software packages or papers.&lt;/p&gt;&#xA;" OwnerUserId="7470" LastActivityDate="2015-06-01T17:56:35.563" Title="Eigenvalue problem of the symmetric real operator which corresponds to the symmetric positive definite matrix" Tags="|eigensystem|integral-equations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19810" PostTypeId="2" ParentId="19807" CreationDate="2015-06-01T18:13:48.583" Score="0" Body="&lt;p&gt;I remember that the matrices from the Poisson equations with pure Neumann boundary conditions is singular. Is it right for your case ? If so, you cannot use the BiCGSTAB to solve the resulting linear systems.&lt;/p&gt;&#xA;" OwnerUserId="10290" LastEditorUserId="1804" LastEditDate="2015-11-28T21:50:41.943" LastActivityDate="2015-11-28T21:50:41.943" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19812" PostTypeId="2" ParentId="8481" CreationDate="2015-06-01T20:05:29.237" Score="3" Body="&lt;p&gt;I recently found this thesis on TDD in Computational Science. I haven't read it yet so I have no idea if it is any good, but hopefully it can be of some help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cyber.ua.edu/files/2014/12/u0015_0000001_0001551.pdf&quot; rel=&quot;nofollow&quot;&gt;http://cyber.ua.edu/files/2014/12/u0015_0000001_0001551.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6601" LastActivityDate="2015-06-01T20:05:29.237" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19813" PostTypeId="1" AcceptedAnswerId="19814" CreationDate="2015-06-01T23:33:06.123" Score="1" ViewCount="989" Body="&lt;p&gt;It has been a while since I have done linear least squares, so forgive the simple question, but here goes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am attempting to find the best fit coefficients, $\{c_i\}$, of a linear combination of basis functions: $f(x) = \Sigma_j^{M} c_j g_j(x)$ for data $\{x_i,y_i\}$. Where there are $M$ basis functions and $N$ points to be fit. I know that if all I care about is the best fit, I can solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(A^T \cdot A)\cdot X = (A^T \cdot B)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $ A_{i,j} = g_j(x_i)$ and $X = \{c_j\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, however, I want to make sure that $\Sigma c_i = 1$, where do I account for this in the matrix equation specified above?&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2015-06-02T02:49:32.650" Title="Constrained linear least squares matrix equation" Tags="|linear-algebra|regression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19814" PostTypeId="2" ParentId="19813" CreationDate="2015-06-02T02:49:32.650" Score="3" Body="&lt;p&gt;This is a constrained minimization problem (with linear constraints). You can write it like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_X \frac{1}{2}\|AX-B\|^2_2 \quad \text{ s.t. } e^TX = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e$ is the vector of all ones.  This problem is typically solved by forming the Lagrangian $L(X,\lambda)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L(X,\lambda) = \frac{1}{2}\|AX-B\|^2_2 + \lambda(1-e^TX)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for which the constrained solution is a stationary point of the Lagrangian. Finding this involves equating the derivatives in $X$ of the first term with derivatives in $X$ of the second.  In your particular problem, the derivative of the first term with respect to $X$ leads to the normal equations, and since the constraint is linear, derivatives of the second term with respect to the entries $X_i$ are just $\lambda e_i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long story short, setting these derivatives equal to each other leads to solving a saddle point system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[\begin{array}{cc}A^TA &amp;amp; e\\e^T &amp;amp;\end{array}\right]\left[\begin{array}{c}X\\ \lambda\end{array}\right] = \left[\begin{array}{c}A^TB\\ 1\end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to add a constraint, just augment your system with one extra unknown and one extra equation per constraint.  The extra unknowns $\lambda$ are referred to as Lagrange multipliers - these may have some meaning, but you can usually just discard them.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-06-02T02:49:32.650" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19816" PostTypeId="1" AcceptedAnswerId="19824" CreationDate="2015-06-02T05:19:33.417" Score="0" ViewCount="164" Body="&lt;p&gt;I use FEM to simulate deformations of elastic objects for animation applications in computer graphics. The governing equation is generally with the form:&#xA;$$&#xA;\mathbf{M}\ddot{\mathbf{u}} +\mathbf{C}\dot{\mathbf{u}}+\mathbf{K}\mathbf{u}=\mathbf{f}&#xA;$$&#xA;where $\mathbf{M}$ is the mass matrix, $\mathbf{C}$ is the damping matrix, $\mathbf{K}$ is the stiffness matrix, and $\mathbf{u}$ is the displacement to be solved for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to solve this time-dependent equation using FEM with very coarse spatial resolutions, mainly to save computational time. As far as I know, the FEM solution converges to the exact solution as spatial resolution goes to infinity. What I'm trying to figure out is: &lt;strong&gt;what is the relationship between FEM solutions using different spatial resolutions besides that they converge to the same exact solution?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm hoping to achieve is to get coarse solution as the downsampling of fine solution, since fine solution is closer to exact solution. Is this even possible? &lt;/p&gt;&#xA;" OwnerUserId="1976" LastActivityDate="2015-06-05T15:44:10.983" Title="Relationship between FEM solutions of PDE with different spatial resolutions" Tags="|pde|finite-element|simulation|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19817" PostTypeId="1" AcceptedAnswerId="19819" CreationDate="2015-06-02T08:27:46.633" Score="0" ViewCount="106" Body="&lt;p&gt;I have a set of data that I need to visualise under Tecplot, the file format is for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X  U1   U2        &#xA;1  0    0         &#xA;2  0.1  0.01     &#xA;3  0.15 0.01     &#xA;4  0.12 0.015  &#xA;.  .    .    &#xA;.  .    .    &#xA;.  .    .    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to display X and U1, and ideally U2 on the same graphic. &lt;/p&gt;&#xA;" OwnerUserId="16516" LastEditorUserId="16516" LastEditDate="2015-06-02T12:51:20.933" LastActivityDate="2015-06-02T12:51:20.933" Title="Visualise data under Tecplot" Tags="|fluid-dynamics|plotting|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19818" PostTypeId="2" ParentId="19808" CreationDate="2015-06-02T08:30:43.540" Score="0" Body="&lt;p&gt;This looks like you are asked to plot current with respect to time. I will recommend you to use MATLAB because it has an inbuilt solver for the 4th order RK method and therefore you won't need to write your own code. Plotting in MATLAB is pretty easy and clean as well.&lt;/p&gt;&#xA;&lt;p&gt;In one of my assignments, I solved an equation called Falkner Skan Flow Equation in MATLAB though &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/50460-falkner-skan-solution--without-using-ode45--fundamental-program&quot; rel=&quot;nofollow noreferrer&quot;&gt;I wrote my own code&lt;/a&gt; as you have done in C++.&lt;/p&gt;&#xA;&lt;p&gt;On the same website, called MATLAB File Exchange, you can find other codes which can help you a lot in understanding RK method and its application to differential equations. If you are insisted to use C++, you can see algorithm in my code and implement the same in your program. Also, you can check your C++ results with MATLAB RK solver's results&lt;/p&gt;&#xA;" OwnerUserId="16397" LastEditorUserId="20688" LastEditDate="2022-12-13T23:02:05.177" LastActivityDate="2022-12-13T23:02:05.177" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19819" PostTypeId="2" ParentId="19817" CreationDate="2015-06-02T08:37:38.373" Score="2" Body="&lt;p&gt;This is the content of a sample Tecplot data file. Data is one dimension i.e. U vs. X for 20 points. I was using a C program to solve my equations and at the end of my program, I had a small subroutine which writes this information into a .PLT or .DAT file which is then used in Tecplot. I think this should suffice for very basic data plotting. If there is anything specific, I would like you to post that in your question specifically.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VARIABLES=&quot;X&quot;,&quot;U&quot;&#xA;&#xA;ZONE  F=POINT&#xA;&#xA;I=20&#xA;&#xA;0.00000000  0.00000000&#xA;&#xA;5.00000000  0.28632834&#xA;&#xA;10.00000000 0.52683960&#xA;&#xA;15.00000000 0.68127329&#xA;&#xA;20.00000000 0.72938137&#xA;&#xA;25.00000000 0.67444474&#xA;&#xA;30.00000000 0.54886950&#xA;&#xA;35.00000000 0.40325212&#xA;&#xA;40.00000000 0.28895415&#xA;&#xA;45.00000000 0.24889311&#xA;&#xA;50.00000000 0.30863131&#xA;&#xA;55.00000000 0.46400776&#xA;&#xA;60.00000000 0.67566198&#xA;&#xA;65.00000000 0.93921596&#xA;&#xA;70.00000000 1.29542423&#xA;&#xA;75.00000000 1.51374208&#xA;&#xA;80.00000000 1.35904516&#xA;&#xA;85.00000000 1.01646862&#xA;&#xA;90.00000000 0.54498717&#xA;&#xA;95.00000000 0.00000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16397" LastEditorUserId="1804" LastEditDate="2015-06-02T09:03:11.130" LastActivityDate="2015-06-02T09:03:11.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19821" PostTypeId="2" ParentId="19798" CreationDate="2015-06-02T10:25:49.773" Score="1" Body="&lt;p&gt;$b$ is bang-bang, ie $b$ puts 1 on where $\xi$ is smallest. So for any fixed $\eta$, the $b, \xi$ part is known&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$b^\top\xi=\min(\max(1-a_i^\top\eta,0))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem reduces to an unconstrained problem on $\eta$&lt;/p&gt;&#xA;" OwnerUserId="14305" LastActivityDate="2015-06-02T10:25:49.773" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19823" PostTypeId="1" CreationDate="2015-06-02T12:19:04.163" Score="1" ViewCount="62" Body="&lt;p&gt;I usually debug a python script by putting the following line into the source code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import ipdb; ipdb.set_trace()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then when I run the script, ipdb starts. Very often I need to plot some arrays in an interactive graph. I use the following commands to make it possible inside the ipdb:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;plt.ion()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is whether it is possible to run these two commands automatically when ipdb starts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P. S. I put this question here, in Computational Science, as I believe that ipdb is mostly used by people who do scientific computing.&lt;/p&gt;&#xA;" OwnerUserId="3254" LastActivityDate="2015-06-02T12:19:04.163" Title="Execute commands when starting ipdb" Tags="|python|" AnswerCount="0" CommentCount="1" ClosedDate="2015-06-03T02:46:11.690" ContentLicense="CC BY-SA 3.0" />
  <row Id="19824" PostTypeId="2" ParentId="19816" CreationDate="2015-06-02T12:50:55.443" Score="4" Body="&lt;p&gt;It doesn't generally work this way. You want to compare $u_H$ and $u_h$ computed on coarse and fine meshes $T_H$ and $T_h$. It is generally not true that $u_H=I_H u_h$ where $I_H$ is the interpolation to the coarse mesh. Nor is it generally true that $u_H=P_H u_h$ for some projection operator (though that is indeed the case for elliptic problems if you could integrate coefficients exactly), but even if it were, it would not yield an easily understandable relationship. In general, you don't even know that $u_h$ is more accurate than $u_H$ (in the sense that $\|u-u_h\| &amp;lt; \|u-u_H\|$) on coarse meshes, as convergence only exists &lt;em&gt;in the limit&lt;/em&gt; and not for a particular choice of $h,H$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in essence, there is not very much you can say about the solutions on different meshes.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-02T12:50:55.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19825" PostTypeId="2" ParentId="10721" CreationDate="2015-06-02T14:48:04.707" Score="1" Body="&lt;p&gt;In the mean time, I've found an answer on pages 24--25 of this &lt;a href=&quot;https://noppa.aalto.fi/noppa/kurssi/kul-24.z/luennot/Kul-24_Z_lecture6.pdf&quot; rel=&quot;nofollow&quot;&gt;lecture&lt;/a&gt;. Translated to the notation above, Eq.(15) from page 24 states&#xA;\begin{equation}&#xA;\tilde{\phi} = \frac{\phi-\phi_U}{\phi_A - \phi_U} = 1 - \frac{\phi_A-\phi}{\phi_A-\phi_U}&#xA;\end{equation}&#xA;This expression is then overruled by Eq.(16) from page 25 such that&#xA;\begin{equation}&#xA;\tilde{\phi}=0.5 \quad \text{ if $|\phi_A-\phi| \leq 10^{-6}$ or $|\phi_A - \phi_U| \leq 10^{-6}$}&#xA;\end{equation} &#xA;This seems to imply that the central scheme is being used whenever the variation in $\phi$ is small, because $\tilde{\phi}=0.5 \Leftrightarrow \phi=\frac{1}{2}(\phi_U+\phi_A)$. &lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2015-06-02T14:48:04.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19826" PostTypeId="1" CreationDate="2015-06-02T21:17:15.380" Score="7" ViewCount="861" Body="&lt;p&gt;Are there any high quality &lt;a href=&quot;http://en.wikipedia.org/wiki/Trust_region&quot;&gt;trust region&lt;/a&gt; optimization implementations that &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;allow nonspherical ellipsoid trust regions, and&lt;/li&gt;&#xA;&lt;li&gt;are written in Python, or are easy to call from python?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;By nonspherical ellipsoid trust regions, I mean that each Newton step solves (or approximately solves) a problem of the following form:&#xA;\begin{align}\min_x \quad&amp;amp; \frac{1}{2}x^T H x + g^T p \\&#xA;\text{such that} \quad &amp;amp; x^T C x \le \Delta^2,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some symmetric positive definite matrix $C$. The matrix $C$ acts like a Hessian preconditioner and may vary from Newton iteration to Newton iteration - for example, see &lt;a href=&quot;http://www.ii.uib.no/~trond/publications/papers/trust.pdf&quot;&gt;the classic paper by Steihaug&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked around a little and haven't found much yet.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;scipy.optimize only allows spherical trust regions. That is, it can only solve the case $C=I$&lt;/li&gt;&#xA;&lt;li&gt;Tao in PETSc allows you to choose from a fixed set of premade preconditioners for the CG-Steihaug step, but not use your own custom preconditioner (as far as I can tell).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2015-06-03T00:17:03.460" LastActivityDate="2017-02-19T21:57:09.397" Title="Python trust region optimization code that allows ellipsoid-shaped trust regions" Tags="|optimization|python|software|trust-regions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19827" PostTypeId="1" AcceptedAnswerId="19832" CreationDate="2015-06-02T21:54:14.103" Score="1" ViewCount="211" Body="&lt;p&gt;I am simulating $\nabla^{2}u=0$ with mixed Dirichlet-Neumann boundary conditions on &lt;code&gt;2-D&lt;/code&gt; using &lt;code&gt;2-Grid&lt;/code&gt; method. Dirichlet conditions are applied to left and bottom boundary with $u=1+x+y$ and Neuman conditions are applied to top and right boundary with $\frac{\partial u}{\partial n}=1$. The solution converges for &lt;code&gt;4x4&lt;/code&gt; grid without using &lt;code&gt;2-grid&lt;/code&gt;. With 2-grid, please let me know if the following are correct (as I am unable to converge with 2-grid).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I do &lt;code&gt;2&lt;/code&gt; iterations on the fine grid (using Pure Jacobi), calculate residual, use injection to restrict to Coarse grid (&lt;code&gt;2x2&lt;/code&gt;). Is it okay to use &lt;strong&gt;Pure Jacobi&lt;/strong&gt; or do we always use &lt;strong&gt;Damped (weighted) Jacobi&lt;/strong&gt; ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With error matrix $e=0$ as initial guess, I solve the residual equation $Ae =r$ till full convergence, then interpolate/prolongate the error back to fine grid. Should the residual equation be solved till convergence or would 2-3 iterations be sufficient ? ( The error equation &lt;em&gt;does&lt;/em&gt; converge fully.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Where exactly do I have to use the &lt;strong&gt;norm&lt;/strong&gt; of an error/residual/whatever ? Since I know the true solution on the Fine Grid is $u = 1 + i*h + j*h$ where &lt;code&gt;h=1/4&lt;/code&gt;for &lt;code&gt;i,j&lt;/code&gt; vary from &lt;code&gt;0&lt;/code&gt; to&lt;code&gt;4&lt;/code&gt;, I am using it as convergence criterion on the fine grid i.e. till $error = u_{new} - (1 + i*h + j*h)$ is greater than the tolerance of $10^{-5}$ keep iterating. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="14303" LastEditDate="2015-06-04T09:24:47.110" LastActivityDate="2015-06-09T10:32:49.627" Title="Specific questions for 2-D Multigrid" Tags="|multigrid|elliptic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19828" PostTypeId="1" CreationDate="2015-06-02T22:05:05.087" Score="2" ViewCount="55" Body="&lt;p&gt;Not sure if this is the right place to post this, but some time ago i stumbled upon a website which had algorithms submitted by users in different languages... I am unalbe to find that website, anyone know aobut it here?&lt;/p&gt;&#xA;" OwnerUserId="3842" LastActivityDate="2015-06-03T00:31:47.763" Title="website for algorithm analysis in various languages" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19829" PostTypeId="2" ParentId="19798" CreationDate="2015-06-03T00:09:44.427" Score="3" Body="&lt;p&gt;Following the derivation in Section B.1 of Boyd and Vandenberghe, it seems plausible that you could solve the SDP dual of your primal to obtain the optimal objective function value. You could also solve the dual of the dual, which is an SDP relaxation of your primal problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following Boyd's notation, suppose the primal variables of your problem are denoted by the vector $x$, and the decision variables of the SDP relaxation (i.e., the dual of the dual) are denoted $X$. There are two cases:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;An optimal value of $X$ has the form $vv^{T}$; then $v$ is an optimal solution of the nonconvex primal.&lt;/li&gt;&#xA;&lt;li&gt;All optimal values of $X$ cannot be expressed in the form $vv^{T}$ for any $v$, in which case it is not possible to recover a primal optimal solution from the SDP relaxation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the second case, all is not lost: by strong duality, the optimal objective function value of the SDP relaxation equals the optimal objective function value of the primal. This observation is useful in the following strategy:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Solve the SDP relaxation, which gives you an optimal objective function value. This operation can be done using an SDP solver, which is theoretically efficient, and practical with current solvers if your instances are not too large.&lt;/li&gt;&#xA;&lt;li&gt;Supply this optimal objective function value to a global optimization solver (e.g., GLOMIQO, BARON, Couenne, Bonmin, etc.). These solvers operate on the principle of generating sequences of subproblems that generate lower and upper bounds on the optimal objective function value; in the process, they also yield feasible solutions (which are used for upper bounds).&lt;/li&gt;&#xA;&lt;li&gt;Since you already know the optimal objective function value from the SDP relaxation, you can calculate the optimality gap of any upper bound exactly. This calculation is a significant improvement over the estimates of the optimality gap given by these solvers, which are usually calculated by taking the difference of the best upper and lower bounds.&lt;/li&gt;&#xA;&lt;li&gt;Also, since you already know the optimal objective function value from the SDP relaxation, you could potentially save some work in the lower bounding problems, because you do not need lower bounds.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The situation you describe, where you can determine efficiently the optimal solution value, but not necessarily where it is attained, isn't a case I normally see treated explicitly in a solver, because it's rare. Normally, the optimal objective function value isn't known in advance, and can't be determined easily. However, one way to incorporate this information in a formulation would be to include it in the constraints as a lower bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, regarding linearization techniques, you could look at work by &lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs10957-009-9626-0&quot; rel=&quot;nofollow&quot;&gt;Misener and Floudas&lt;/a&gt; in the context of global optimization, which would potentially reduce your problem to solving a bunch of mixed-integer linear programs.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-06-03T10:02:32.540" LastActivityDate="2015-06-03T10:02:32.540" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19830" PostTypeId="2" ParentId="19828" CreationDate="2015-06-03T00:31:47.763" Score="2" Body="&lt;p&gt;I believe you're referring to the &lt;a href=&quot;http://rosettacode.org/&quot; rel=&quot;nofollow&quot;&gt;Rosetta Code&lt;/a&gt; website. Their wiki hosts source code for many algorithms and tasks implemented in various programming languages. For example, here is the page for the &lt;a href=&quot;http://rosettacode.org/wiki/Euler_method&quot; rel=&quot;nofollow&quot;&gt;forward Euler method&lt;/a&gt; which currently includes code snippets in 46 different programming languages ranging from COBOL and Fortran to Python or Javascript.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-06-03T00:31:47.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19831" PostTypeId="2" ParentId="3177" CreationDate="2015-06-03T03:57:59.993" Score="1" Body="&lt;p&gt;I'm not sure it's possible to answer the question without a detailed reading of the paper.  But in regards to the first question, you have $r/R = (y+1)/2$. And this factor cannot be divided out since it does not multiply all terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For question 2: since this equation is to be used to apply the boundary condition at $r=0$, I think the term you mention should vanish.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-06-03T03:57:59.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19832" PostTypeId="2" ParentId="19827" CreationDate="2015-06-03T12:53:46.713" Score="2" Body="&lt;p&gt;Jacobi is usually not a great smoother, Gauss-Seidel is a better choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also want to use a larger initial grid so you see whether the defect is being generated at the boundary or in the interior. Your 2x2 coarse grid is essentially all boundary condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mixed boundaries are tricky to implement. I'd get only Dirichlet boundaries working, and then only Neumann working, and then try to mix them. You have to think carefully about how to handle restriction at the corners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This discussion might also be helpful:&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/10209/multigrid-stops-converging-when-more-grid-levels-are-used&quot;&gt;Multigrid stops converging when more grid levels are used&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6702" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-06-03T12:53:46.713" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19833" PostTypeId="1" CreationDate="2015-06-03T13:19:02.043" Score="10" ViewCount="185" Body="&lt;p&gt;I have a linear system with matrix which eigenvalues are uniformly distributed on the unit circle like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/kx2vt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to solve this kind of system effectively by iterative method, maybe with some preconditioner?&lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2015-06-04T15:49:03.040" Title="What iterative method can effectively solve a linear system with this kind of spectrum" Tags="|linear-algebra|iterative-method|preconditioning|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19834" PostTypeId="1" AcceptedAnswerId="19839" CreationDate="2015-06-03T18:21:39.820" Score="-1" ViewCount="91" Body="&lt;p&gt;My task in numerical analysis is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We are interested in finding values of β0 for which z(x) = 2500. Use&#xA;  an efficient algorithm to determine the rays which pass through the&#xA;  receiver.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now I'm just looping through and printing and plotting it and seeing from inspection that the values between 8 and 9 and between -8 and -9 are candidates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=0:1:6076*25;&#xA;hold on&#xA;&#xA;for w = -10:14&#xA;&#xA;[X,Z] = ode45(@(t,Z) sys(t,Z,w),x,[2000 tand(7.8)]);&#xA;&#xA;w&#xA;Z(end,[1,end])&#xA;&#xA;&#xA;plot(X,Z(:,1),'r')  %Z(:,1) is z(x) and Z(:,2) is z'(x).&#xA;&#xA;end&#xA;&#xA;hold off&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can it be done more efficient? β0 is the angle of the source of the sound and z(x) is the depth of the sound. My system is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dZ=sys(t,Z,w)&#xA;&#xA;% inline function for c(z)&#xA;c=@(z)4800 - 20.2090 + (17.3368)*z/1000+ (272.9057)*exp(-z*0.7528/1000); &#xA;% inline function for c'(z)&#xA;&#xA;dc=@(z) 17.3368/1000 - (272.9057)*(0.7528/1000)*exp(-z*0.7528/1000); &#xA;q=c(2000);&#xA;dZ=zeros(2,1);    % a column vector&#xA;dZ(1)=Z(2);&#xA;dZ(2)=-(q/cosd(w))^2* dc(Z(1))/c(Z(1))^3; &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3566" LastEditorUserId="3566" LastEditDate="2015-06-03T18:56:21.697" LastActivityDate="2015-06-04T06:32:54.043" Title="Is iteration an efficient algorithm in this case?" Tags="|matlab|algorithms|numerical-analysis|ode|" AnswerCount="1" CommentCount="3" ClosedDate="2015-06-08T14:36:31.353" ContentLicense="CC BY-SA 3.0" />
  <row Id="19838" PostTypeId="1" AcceptedAnswerId="20010" CreationDate="2015-06-04T02:54:54.713" Score="0" ViewCount="240" Body="&lt;p&gt;I am pretty sure that the following variance objective function should be a convex quadratic problem. My objective function is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \text{argmin } \text{var }(X*L) \xi \geq 1, \text{ where } \xi \text{ are integers} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\xi$  are random variables, $1 \leq i \leq 1000$ and elements of vector $X$. $L$ is a $1000\times20$ known matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to solve this problem with &lt;code&gt;fmincon&lt;/code&gt; and by thresholding the final variables. However each time I run the algorithm with a different initial point, I get a different answer. The step size and &lt;code&gt;tolFun&lt;/code&gt; are &lt;code&gt;1e-100&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you shed light on the problem for me?&lt;/p&gt;&#xA;" OwnerUserId="16535" LastEditorUserId="9667" LastEditDate="2015-08-26T03:33:45.307" LastActivityDate="2015-08-26T03:33:45.307" Title="Convex quadratic problem solver gives different answers?" Tags="|convex-optimization|quadratic-programming|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19839" PostTypeId="2" ParentId="19834" CreationDate="2015-06-04T06:32:54.043" Score="1" Body="&lt;p&gt;I have these two things in mind which can help you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Since I see that you are using ode45 i.e. runge kutta method, you probably can use adaptive runge kutta methods if you want to make your algorithm more efficient. This might remove the restriction of using every β value between your specified interval. You can find a video here which I saw quite sometime ago: &lt;a href=&quot;http://nptel.ac.in/courses/122106033/33&quot; rel=&quot;nofollow&quot;&gt;http://nptel.ac.in/courses/122106033/33&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) And since you want to have a β for which z equals 2500. You can use additional array,say Y to store the value you get for that value of β. And having one more array, say A such that A=|z-Y| i.e. the difference between the z for current β value and z that is required. The minimum element of this array corresponds to the β you are looking for. You can look about this at: &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/min.html?refresh=true&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/matlab/ref/min.html?refresh=true&lt;/a&gt; This removes the &lt;em&gt;inspection&lt;/em&gt; aspect from your side and the program itself will tell which β is most suitable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps! &lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2015-06-04T06:32:54.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19840" PostTypeId="1" CreationDate="2015-06-04T08:34:11.643" Score="4" ViewCount="2962" Body="&lt;p&gt;I'm solving a constrained optimization for matrix $\mathbf{A}$ with dimension 6x6, where one of the constraints is $\mathrm{det}(\mathbf{A})&amp;gt;0$. I use the &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt&quot; rel=&quot;nofollow&quot;&gt;NLopt&lt;/a&gt; package to solve my problem and provide value&amp;amp;&amp;amp;gradient routines of the objective and constraints. As described in &lt;a href=&quot;http://www.eattardo.com/wp-content/uploads/2013/10/MatrixCookbook2012.pdf&quot; rel=&quot;nofollow&quot;&gt;the matrix cookbook&lt;/a&gt;, the gradient of matrix determinant is computed as $\frac{\partial \mathrm{det}(\mathbf{A})}{\partial \mathbf{A}} = \mathrm{det}(\mathbf{A})(\mathbf{A}^{-1})^T$ and involves matrix inverse. During the optimization iterations, one intermediate solution might violates the constraint and leads to singular matrix $\mathbf{A}$. The singular $\mathbf{A}$ will halt the optimization because evaluation of constraint gradient throws a &quot;not invertible&quot; exception.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible solution is to use optimization algorithms that do not need the gradient of constraints. But I wonder if there's any workaround if I use gradient-based algorithms. As far as I know, methods using gradient information would converge faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;=============== Edit =================&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function is of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\min_{\mathbf{A}}\quad\sum_i\frac{1}{2}(\mathbf{E}_i^T\mathbf{A}\mathbf{E}_i-t_i)^2&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{E}_i$ are known 6x1 vectors and $t_i$ are knwon scalars. Other constraints enforced on $\mathbf{A}$ are: (1) $\mathbf{A}$ is symmetric matrix (2) entries of $\mathbf{A}$ is non-negative&lt;/p&gt;&#xA;" OwnerUserId="1976" LastEditorUserId="1976" LastEditDate="2015-06-27T09:11:14.773" LastActivityDate="2018-03-19T21:03:41.257" Title="Optimization with matrix determinant as constraint" Tags="|matrix|nonlinear-programming|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19841" PostTypeId="2" ParentId="19827" CreationDate="2015-06-04T09:55:01.453" Score="0" Body="&lt;p&gt;I used weighted Jacobi (doesn't work with pure Jacobi). In this problem the Neumann Boundary condition becomes $\frac{\partial e}{\partial n} = 0$ for the error matrix i.e. while solving $Ae = r$ if we take the central difference at the boundary of the coarse grid then $\frac{e_{i+1,j} - e_{i-1,j}}{2h_{c}} = 0$, where $e_{i,j}$ is the point on the Neumann boundary. Similarly $\frac{e_{i,j+1} - e_{i,j-1}}{2h_{c}} = 0$, where $e_{i,j}$ is a boundary point (and not the fictitious boundary point, of course !) on the coarse grid. &lt;/p&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="14303" LastEditDate="2015-06-09T10:32:49.627" LastActivityDate="2015-06-09T10:32:49.627" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19842" PostTypeId="1" AcceptedAnswerId="19845" CreationDate="2015-06-04T09:59:04.973" Score="6" ViewCount="927" Body="&lt;p&gt;I'm attempting derivative-free minimization of, essentially, a black-box function in one dimension. Up to now I've been using BOBYQA as &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms#Local_derivative-free_optimization&quot; rel=&quot;noreferrer&quot;&gt;implemented in NLopt&lt;/a&gt;. The shape of the function looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/E1sVs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, giving a good initial guess will help immensely here, as I can avoid the nasty flat region to the right of the graph. However, sometimes a &lt;strong&gt;good&lt;/strong&gt; initial guess is unavailable, and it can end up in the flat region and hence never find the minima.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are there any tips/tricks for getting out/avoiding the flat region at&#xA;all?&lt;/li&gt;&#xA;&lt;li&gt;Would a simple Fibonacci / Golden Section search be any better?&lt;/li&gt;&#xA;&lt;li&gt;Or would my time be better spent on working out good initial guesses?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8094" LastActivityDate="2015-06-05T12:36:24.400" Title="Derivative-free optimization of function with a flat region" Tags="|optimization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19844" PostTypeId="2" ParentId="19075" CreationDate="2015-06-04T13:23:09.873" Score="0" Body="&lt;p&gt;The form $(A+μdiag(A))h=−g$ can be used always and it gives better results than $(A+μI)h=−g$ in practice. The reason given by the OP is right. It moves faster along flatter directions in error valleys because it considers curvature information better. However, Levenberg-Marquardt is a heuristic method and doesn't possess well defined guarantees on speed and convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, Levenberg-Marquardt deals with the situation when $(A+μdiag(A))h=−g$ does not give a descent direction. Note that this method is one of the ancestors of trust region methods. So, if an iteration does not give a descent direction, then do not take the step, increase $\mu$ and proceed to the next iteration.&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2015-06-07T12:47:30.190" LastActivityDate="2015-06-07T12:47:30.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19845" PostTypeId="2" ParentId="19842" CreationDate="2015-06-04T15:41:34.610" Score="7" Body="&lt;p&gt;The following works in any dimension:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a poor initial guess $x_0$ only (but outside the completely flat region), minimize $f(x)+c_k||x-x_k||^2$ for $k=0$ with some small $c_0$, call the result $x_1$, and iterate with a strongly decreasing sequence of $c_k$'s. In many cases $c_1=0$ will already work.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2015-06-05T12:36:24.400" LastActivityDate="2015-06-05T12:36:24.400" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="19846" PostTypeId="2" ParentId="19840" CreationDate="2015-06-04T15:46:40.463" Score="4" Body="&lt;p&gt;Replace $A$ by a factored form from which the determinant can be computed stably and cheaply. E.g., add new triangular variables $L$ and $R$ and the constraint $A=PLR$ for some fixed permutation matrix $P$; often the identity permutation $P=I$ is enough. This means that you treat the nontrivial entries of $L$ and $R$ as additional variables. Then determinants and their gradients are trivially to compute. If $A$ is known to be symmetric, you may use instead $A=LDL^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you may eliminate $A$ by substituting $PLR$ for it everywhere it occurs. Thus in place of minimizing $f(A,\det A)$ subject to $F(A)=0$, say, you would minimize $f(LR,\det R)$ subject to $F(LR)=0$, (assuming for simplicity that $L$ is unit lower triangular and no permutation was applied), and use $\det R=\prod R_{ii}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case the solution has no such factorization, you need to change $P$ after a preliminary round of minimization with $P=I$.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2015-06-05T09:47:31.770" LastActivityDate="2015-06-05T09:47:31.770" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19847" PostTypeId="2" ParentId="19833" CreationDate="2015-06-04T15:49:03.040" Score="1" Body="&lt;p&gt;The matrix is very well-conditioned, hence GMRES(k) should work fine without preconditioner.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2015-06-04T15:49:03.040" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19848" PostTypeId="2" ParentId="19755" CreationDate="2015-06-04T15:55:04.247" Score="1" Body="&lt;p&gt;If your equations are linear you have a sparse linear system of equations, and you'd read some of the standard material about these (e.g., the book by Duff, Erisman, Reid, or the book Matrix computations by Golub and van Loan).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your equations are nonlinear you have a sparse nonlinear system of equations, and you'd use a damped Newton method to solve it iteratively.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2015-06-04T15:55:04.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19849" PostTypeId="1" AcceptedAnswerId="19867" CreationDate="2015-06-04T19:42:42.447" Score="5" ViewCount="530" Body="&lt;p&gt;Geometrically, scaling and preconditioning seem to address similar challenges in optimization. However, these two concepts are implemented very differently. Take trust region Newton method, as an example. When a problem is poorly scaled, an elliptical trust region is recommended. Is it possible to formulate an equivalent preconditioner based approach such that one works with spherical trust regions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;update: Section 7.5 in Practical Optimization by Gill , Murray &amp;amp; Wright gives a clear connection between variable scaling and preconditioning the hessian.&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2015-08-26T10:05:50.403" LastActivityDate="2015-08-26T10:05:50.403" Title="scaling and preconditioning for trust region Newton methods" Tags="|optimization|numerical-analysis|newton-method|trust-regions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19850" PostTypeId="2" ParentId="19842" CreationDate="2015-06-04T19:46:40.173" Score="4" Body="&lt;p&gt;In your particular case, you could consider transforming the $x$ variable by replacing the problem by the following: find $y$ so that $f(e^y)$ is minimal. In essence, you're compressing the long tail to the right and thereby make its gradient steeper.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-04T19:46:40.173" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19851" PostTypeId="2" ParentId="19816" CreationDate="2015-06-04T20:41:23.177" Score="2" Body="&lt;p&gt;The governing equation&#xA;\begin{equation*}&#xA;\mathbf{M}\ddot{\mathbf{u}} +\mathbf{C}\dot{\mathbf{u}}+\mathbf{K}\mathbf{u}=\mathbf{f}&#xA;\end{equation*}&#xA;is linear assuming that $\mathbf{M}$, $\mathbf{C}$, $\mathbf{K}$ are constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the equations governing an elasticity problem are linear only under the assumption of &lt;em&gt;small&lt;/em&gt; displacements and &lt;em&gt;small&lt;/em&gt; strains, which makes them absolutely inadequate for computer graphics applications. (Strange things occur if one uses equations valid at the small displacement limit, when finite displacements are analysed, like objects changing their size when subjected to rigid rotations.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This said I think that what you are interested in is actually a &lt;strong&gt;Model Reduction&lt;/strong&gt; technique, i.e. given a fine mesh which gives accurate results for your problem, to define an equivalent reduced model, which has much less degrees of freedom but still correctly describes the dynamics of the system under study. Typically this is done not by solving on a coarse mesh, but by suitably defining a projection on a lower dimensional space with the desired properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Literature is vast on the argument, but a quick check on google let me find a link to this ACM SIGGRAPH 2012 Course:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;FEM Simulation of 3D Deformable Solids: A practitioner's guide to&#xA;  theory, discretization and model reduction&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;please see &lt;a href=&quot;http://www.femdefo.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.femdefo.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT to address a question in a comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by Wolfgang in his &lt;a href=&quot;https://scicomp.stackexchange.com/a/19824/1744&quot;&gt;answer&lt;/a&gt;, a frame-by-frame comparison between two different meshes does not make much sense: $u_H \neq I_H u_h$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Model reduction tries to project some of the information gained from the solution on the fine mesh onto a coarser mesh to speed up dynamic computations. But of course there is no silver bullet here, and every speed-up comes at the cost of some inaccuracy on the reduced model. Think it as the spectrum of $\mathbf{M}^{-1} \mathbf{K}$: if you do model reduction correctly, then in a given frequency range this matrix will have almost the same eigenvalues/eigenvectors for both the fine mesh and the reduced model. But of course the complete model has much more eigenvalues/eigenvectors than the reduced one, so its dynamics &lt;em&gt;has&lt;/em&gt; to be different from the one of the reduced one.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-06-05T15:44:10.983" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19852" PostTypeId="1" AcceptedAnswerId="19853" CreationDate="2015-06-04T22:37:47.230" Score="4" ViewCount="1044" Body="&lt;p&gt;I am working on some CFD codes - generating adjoint of a CFD model, for example. My background however is in computer science. I have absolutely no clue what the model represents or what it is doing. I would like to gain some domain knowledge of CFD. However, I notice that as I start, my physics foundations are a bit shaky and then follows my math background. Eventually, it seems to me like I am getting into a rabbit hole. What steps in terms of books/topics/lecture series would be a good start for me to self-orient later?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="11828" LastActivityDate="2015-06-09T03:54:31.473" Title="Where do I begin learning CFD?" Tags="|fluid-dynamics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19853" PostTypeId="2" ParentId="19852" CreationDate="2015-06-05T02:42:10.557" Score="4" Body="&lt;p&gt;I'll try to give a list of the key areas that a typical university student might study to prepare for CFD research. Perhaps someone else can give some advice on where to focus for most efficient use of your time or point out specific resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a lot of background info involved in fully understanding fluid dynamics and CFD. And, as with many advanced topics, there's always more to learn. If I had to make a list of topics to study to better understand things I would divide it into two areas: &lt;em&gt;Mathematical fundamentals&lt;/em&gt; and &lt;em&gt;Physics&lt;/em&gt;. In rough order of sequence for learning I would propose the following lists for these topics:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Mathematical fundamentals&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Linear algebra&lt;/li&gt;&#xA;&lt;li&gt;Ordinary differential equations&lt;/li&gt;&#xA;&lt;li&gt;Partial differential equations&lt;/li&gt;&#xA;&lt;li&gt;Numerical methods&lt;/li&gt;&#xA;&lt;li&gt;Specific CFD methods (e.g. finite difference/finite volume/finite element implementations)&lt;/li&gt;&#xA;&lt;li&gt;Asymptotics (maybe)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;Physics&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Introductory physics&lt;/li&gt;&#xA;&lt;li&gt;Classical mechanics/Newtonian physics&lt;/li&gt;&#xA;&lt;li&gt;Transport phenomena&lt;/li&gt;&#xA;&lt;li&gt;Thermodynamics and/or statistical mechanics (maybe)&lt;/li&gt;&#xA;&lt;li&gt;Aerodynamics or other application relevant topic (e.g. ocean dynamics)&lt;/li&gt;&#xA;&lt;li&gt;Turbulence&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As I'm sure you know, CFD methods rely on (sometimes complex) numerical methods to solve nonlinear partial differential equations derived from physical situations. Math and physics are two subjects that build in complexity quite significantly so there is simply a lot of background material which is needed to fully understand the end processes. You could possibly skip some of these topics and still greatly improve your understanding, but I think this is a good starting list to really understand many CFD methods. In my opinion, numerical methods for scientific computing, partial differential equations, and transport phenomena are the most essential topics.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-06-05T02:42:10.557" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19855" PostTypeId="2" ParentId="18667" CreationDate="2015-06-05T08:54:15.140" Score="2" Body="&lt;p&gt;It sounds like there are two things you might want to &quot;use&quot; them for:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;To implement a method.  Any reference will give you a clear algorithmic description that should make this easy.&lt;/li&gt;&#xA;&lt;li&gt;To check the order of a method.  I'm not sure why you need to do this, but it's just a matter of looking up the order conditions (equations) and plugging in numbers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can find both in almost any reference, though the order conditions for very high order methods are only in more specialized sources (like Butcher's book).  Just to give you something concrete, I recommend chapter 7 of LeVeque's book on finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a Python implementation of the algorithm and the order conditions (up to order 14) see my &lt;a href=&quot;http://nodepy.readthedocs.org/en/latest/&quot; rel=&quot;nofollow&quot;&gt;Nodepy package&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-06-05T08:54:15.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19857" PostTypeId="1" AcceptedAnswerId="19861" CreationDate="2015-06-05T10:58:27.800" Score="4" ViewCount="2890" Body="&lt;p&gt;Normally when searching in sorted sets, binary searches are a fast, nice and easy way to locate data. It does however break down in the hypothetical scenario of having a set of arbitrarily large but unknown size as it cannot have its upper limit set to infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to solve this, one could imagine doing a, for lack of a better term, &quot;reverse binary search&quot; to find rough lower/upper limits, then perform a regular binary search:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;min = 0&#xA;max = 1&#xA;while( set[max] exists &amp;amp;&amp;amp; set[max] &amp;lt; target ) {&#xA;    min = max + 1&#xA;    max *= 2&#xA;}&#xA;binarySearch(min, max)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since this seems to be a somewhat trivial algorithm, I cannot possibly be the first one to think of it, so my question is simply whether this algorithm has a name (and if so, what it is).&lt;/p&gt;&#xA;" OwnerUserId="16545" LastActivityDate="2015-06-05T20:15:38.090" Title="Does this &quot;reverse binary search&quot; have a name?" Tags="|algorithms|terminology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19858" PostTypeId="1" AcceptedAnswerId="19874" CreationDate="2015-06-05T11:19:23.150" Score="9" ViewCount="404" Body="&lt;p&gt;Consider, you have a problem in an infinite dimensional Hilbert or Banach space (think of a PDE or an optimization problem in such a space) and you have an algorithm that converges weakly to a solution. If you discretize the problem and apply the corresponding discretized algorithm to the problem, then weak convergence is convergence in every coordinate and hence also strong. My question is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does this kind of strong convergence feel or look any different from convergence obtained from good old plain strong convergence of the original infinite algorithm?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Or, more concrete:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What kind of bad behaviour can happen with a &quot;discretized weakly convergence method&quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I myself are usually not quite happy when I can only prove weak convergence but up to now I could not observe some problem with the outcome of the methods even if I scale the problem discretized problems to higher dimensions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I am not interested in the &quot;first discretize than optimize&quot; vs. &quot;first optimize than discretize&quot; problem and I am aware of problems that can occur if you apply an algorithm to a discretized problem that does not share all properties with the problem for which the algorithm was designed for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; As a concrete example consider an optimization problem with a variable in $L^2$ and solving it with something like (an inertial) forward-backward splitting or some other method for which only weak convergence in $L^2$ is known. For the discretized problem you can use the same method and with the correct discretization you get the same algorithm is if you discretized the algorithm directly. What can go wrong when you increase the discretization accuracy? &lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="302" LastEditDate="2015-06-07T09:46:45.443" LastActivityDate="2015-06-07T10:26:23.663" Title="How does weak convergence feel, numerically?" Tags="|algorithms|convergence|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19860" PostTypeId="1" AcceptedAnswerId="19873" CreationDate="2015-06-05T18:06:25.173" Score="1" ViewCount="2060" Body="&lt;p&gt;I obtained 6 eigenpairs of a matrix using &lt;code&gt;eigs&lt;/code&gt; of Matlab.&#xA;How can I demonstrate that these eigenvectors are orthogonal to each other? I am almost sure that I normalized in the right way modulus and phase but they do not seem to be orthogonal. The matrix should be normal.&#xA;The matrix comes from the discretization of the Euler-Bernoulli beam problem for a beam of length 1 with hinged free boundary conditions:&#xA;$$ \frac{\partial u }{\partial t} + \gamma \frac{\partial^4 y}{\partial x^4} = 0,\\ &#xA;\frac{\partial y}{\partial t} - u = 0.\\$$&#xA;Then, the eigenproblem can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \lambda \left[ \begin{matrix} I &amp;amp; 0 \\ 0 &amp;amp; I \end{matrix} \right] \left\{ \begin{matrix} y \\ u \end{matrix} \right\} = \left[ \begin{matrix} 0 &amp;amp; I \\ -\gamma B &amp;amp; 0 \end{matrix} \right] \left\{ \begin{matrix} y \\ u \end{matrix} \right\},$$&#xA;where $I$ is the identity matrix and $B$ is the bilaplacian operator discretized using finite difference.&lt;/p&gt;&#xA;" OwnerUserId="14489" LastEditorUserId="1804" LastEditDate="2015-06-07T09:15:55.897" LastActivityDate="2015-06-07T09:27:44.407" Title="How can I prove that two eigenvectors are orthogonal?" Tags="|matlab|finite-difference|eigenvalues|matrix|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="19861" PostTypeId="2" ParentId="19857" CreationDate="2015-06-05T19:58:00.417" Score="6" Body="&lt;p&gt;This seems to be called an &lt;a href=&quot;http://en.wikipedia.org/wiki/Exponential_search&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;exponential search&lt;/strong&gt;, &lt;strong&gt;doubling search&lt;/strong&gt;, or &lt;strong&gt;galloping search&lt;/strong&gt;&lt;/a&gt;. I've also heard it called a &lt;strong&gt;geometric expansion search&lt;/strong&gt; or something similar. In principal, it is similar to &lt;a href=&quot;http://en.wikipedia.org/wiki/Dynamic_array#Geometric_expansion_and_amortized_cost&quot; rel=&quot;noreferrer&quot;&gt;the geometric expansion strategy&lt;/a&gt; that is often used for resizing dynamic arrays in computer programming. Resizing dynamic arrays in this way ensures that adding $n$ elements takes $\mathcal{O}(n)$ time. This does not necessarily rely on doubling the storage each time, you could use a factor of 3 or 3/2 or any other number greater than 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use this method to find an upper bound for an interval and then use a binary search to find the exact element you would need $\mathcal{O}(\log_2(N))$ for each part of the algorithm where N is the index of the entry that is eventually found. Averaged over many searches, I think this would be an optimal search strategy if the probability that you will need to find entry $N$ is proportional to $1/N$. In other words, this would be a good strategy to use if you expect to preferentially choose points that are near the beginning of your array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a continuous setting this would be useful if you were trying to find where a function reaches a certain value and all you know is that the function grows approximately exponentially. You could use the binary expansion search to find a range that encompasses the value you want and then use a binary search to narrow down on the precise value. Of course there are faster search methods in this case (taking better advantage of your knowledge of the function or using higher order search methods), especially for well-behaved functions, but this would allow you reasonable efficiency with a very simple method.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-06-05T20:15:38.090" LastActivityDate="2015-06-05T20:15:38.090" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19862" PostTypeId="1" CreationDate="2015-06-05T20:38:59.140" Score="7" ViewCount="1009" Body="&lt;p&gt;I would like to start running FSI simulations for cardiovascular applications.&#xA;More precisely, I'm interested in the behaviour of aortic valve under&#xA;physiological flow conditions in which the interaction of fluid and structure&#xA;must taken into account and the valve undergoes very large deformations.&#xA; I read that there are two approaches to couple fluid and solid solvers:&#xA;segregated or monolithic. Apparently seregated solver has drawbacks in terms&#xA;of accuracy and stability. Is any of the 2 coupling ways more advised for&#xA;certain type of setups?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While searching online I have find several opensource codes such as Fenics,&#xA;Oomphlib, ElmerFEM and LifeV (I have used LifeV in the past). Besides this, I&#xA;have access to commercial software, but would like to have an open source&#xA;alternative as well. This is for two reasons: self-education (I can look at&#xA;the source code and learn more about the numerical method) and flexilibility -&#xA;if I have some special requirement, I could eventually implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I don't have the time to develop an FSI solver from scratch and&#xA;I need to start running simulations fairly quickly before I dig into the&#xA;source code. Which solver and method would you recommend so that the initial&#xA;learning curve is not too steep?&lt;/p&gt;&#xA;" OwnerUserId="16550" LastEditorUserId="123" LastEditDate="2015-06-06T02:49:03.260" LastActivityDate="2022-03-12T12:26:14.263" Title="Fluid-structure interaction solver for cardiovascular applications" Tags="|software|navier-stokes|fluid-dynamics|open-source|multiphysics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19864" PostTypeId="2" ParentId="19838" CreationDate="2015-06-06T02:41:25.577" Score="0" Body="&lt;p&gt;In addition to the problems mentioned by the commenters above, let me comment on a few peculiar aspects of your problem:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;arg min var(sum(L) + X*L) xi&amp;gt;=0&lt;/p&gt;&#xA;&lt;p&gt;where xi s are random variables and elements of vector X and sum(L) is a vector which is constituted of sum of the elements in each column of L separately.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As hardmath points out, adding to &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; any matrix whose columns are in the left nullspace of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; will result in an equivalent objective function value. Bilinear products of variables can indicate nonconvexity as well. Like the other commenters, I can't understand your formulation. However, if it is nonconvex, that would explain why different initial points yield different solutions.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;vector X is constituted of 1000 variables and they all should be positive integers. L is a 1000*20 known matrix.&lt;/p&gt;&#xA;&lt;p&gt;I tried to solve this problem with fmincon and round the final variables.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Do you want integer-feasible solutions or integer-optimal solutions? The latter requires mixed-integer programming, and will be more expensive. Also, &lt;code&gt;fmincon&lt;/code&gt; won't work. In either case, rounding could also account for not obtaining the same solution every time, in pathological cases.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I tried to solve this problem with fmincon and round the final variables. However each time I run the algorithm with a different initial point,I get a different answer the step size and tolFun are 1e-100.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Setting the tolerance on function evaluations to &lt;span class=&quot;math-container&quot;&gt;$10^{-100}$&lt;/span&gt; is probably overkill unless you expect your function evaluations to be somewhere around &lt;span class=&quot;math-container&quot;&gt;$10^{-90}$&lt;/span&gt; or so. Convex optimization algorithms return (local) &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt;-optimal solutions for some tolerance &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt;. These algorithms will return the first iterate satisfying the termination tolerance criteria, so it could be that different initial guesses result in sequences of iterates that both satisfy the termination tolerances, just at different points in the feasible set. For instance, it could also be that the problem has multiple (local) minima, which can occur even in the convex case when the function is &amp;quot;flat&amp;quot; at the bottom. Again, hardmath's observation suggest that something like this situation might occur.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-06-06T02:41:25.577" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19865" PostTypeId="2" ParentId="5586" CreationDate="2015-06-06T02:45:53.400" Score="1" Body="&lt;p&gt;Of course, all methods in this area conserve mass.  For scalar conservation laws, conservation of mass plus $L^1$ contractivity implies positivity.  But for systems of conservation laws, it does not.  In practice, schemes have been designed to preserve positivity for Euler and shallow water flows by many means, most of which involve special Riemann solvers.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-06-06T02:45:53.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19867" PostTypeId="2" ParentId="19849" CreationDate="2015-06-06T20:37:46.103" Score="2" Body="&lt;p&gt;The ideas are certainly related, at the very least if your preconditioner corresponds to a symmetric and positive definite matrix. This is because in that case, preconditioning simply means using a different inner product, i.e., a different metric. This can be interpreted  as saying that what is a very elongated ellipsoid in the usual $l_2$ metric may turn out to be much closer to a sphere in the preconditioner metric. This picture is valid because the application of an SPD preconditioner can be interpreted as a rotation, axis-parallel scaling, and inverse rotation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The picture becomes much less clear to interpretation if you use preconditioners that correspond to indefinite or non-symmetric matrices since these can then no longer be interpreted as a simple change in metric.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-06T20:37:46.103" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19868" PostTypeId="2" ParentId="19862" CreationDate="2015-06-06T20:40:47.837" Score="2" Body="&lt;p&gt;People typically agree that the monolithic approach is the conceptually better one, but of course more difficult to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use a code that already does most of what you need, I would suggest to look at this paper and its accompanying code: &lt;a href=&quot;http://journals.ub.uni-heidelberg.de/index.php/ans/issue/view/1244&quot; rel=&quot;nofollow&quot;&gt;http://journals.ub.uni-heidelberg.de/index.php/ans/issue/view/1244&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-06T20:40:47.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19869" PostTypeId="2" ParentId="19858" CreationDate="2015-06-06T20:58:46.913" Score="2" Body="&lt;p&gt;The question you ask is often of not much practical concern because weak convergence in one norm may imply strong convergence in another one, for the same sequence of solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you one example, let's assume we solve the Laplace equation with a sufficiently smooth right hand side on a convex polygonal domain with standard finite elements. Then the solution $u$ is in $H^2$, but of course the finite element solution $u_h$ is only in $H^1$. We do know that $u_h \rightarrow u$ strongly in both the $L_2$ and $H^1$ norms as the maximal mesh size $h\rightarrow 0$ because we have the a priori error estimates $\|u-u_h\|_{L_2} \le Ch^2$ and $\|u-u_h\|_{H^1} \le Ch$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But clearly we cannot expect $u_h \rightarrow u$ strongly in $H^2$ because the $u_h$ are only in $H^1$. But we might have $u_h \rightharpoonup u$ weakly in $H^2$ (in fact, I think that that holds). This would then probably imply a statement such as&#xA;$$&#xA;  \left&amp;lt; \nabla^2 (u-u_h), \nabla^2 v\right&amp;gt; \le {\cal o}(1) \qquad \forall v \in H^2.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that the question of weak vs strong convergence is typically a question of what norm you look at, and &lt;em&gt;not&lt;/em&gt; a property of the sequence of solutions you get from your method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-06T20:58:46.913" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19870" PostTypeId="1" AcceptedAnswerId="19956" CreationDate="2015-06-06T21:32:26.540" Score="11" ViewCount="17490" Body="&lt;p&gt;I want to minimize a black box function $f(x)$, which takes a 8$\times$3 matrix of non-negative integers as input. Each row specifies a variable, whereas each column specifies a certain time period so that $x_{ij}$ is the $i$th variable in the $j$th time period.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x)$ is the activation function for a convolutional filter in the 2. convolutional layer of a convolutional neural network. It is therefore expected to be nonlinear and nonconvex. The input is subject to constraints as seen below, where $\text{sgn}(x)$ is the sign function as defined at &lt;a href=&quot;http://en.wikipedia.org/wiki/Sign_function&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{aligned}&#xA;  \text{Objective:} \hspace{1cm} &amp;amp; \text{minimize} \hspace{0.2cm} f(x)\\&#xA;  \text{Constraints:} \hspace{1cm} &amp;amp; \sum_{i=3}^{6}\sum_{j=1}^{3} x_{ij} = 10 &amp;amp; \\&#xA;  &amp;amp; x_{3j} + x_{5j} \geq x_{1j} &amp;amp;\forall j = 1,2,3\\&#xA;  &amp;amp; x_{4j} + x_{6j} \geq x_{2j} &amp;amp;\forall j = 1,2,3\\&#xA;  &amp;amp; x_{7j} \leq 15 &amp;amp;\forall j = 1,2,3\\&#xA;  &amp;amp; x_{8j} \leq 15 &amp;amp;\forall j = 1,2,3\\&#xA;  &amp;amp; \text{sgn}(x_{7j}) = \text{sgn}(x_{3j}) &amp;amp;\forall j = 1,2,3 \\&#xA;  &amp;amp; \text{sgn}(x_{8j}) = \text{sgn}(x_{4j}) &amp;amp;\forall j = 1,2,3 \\&#xA;  &amp;amp; x_{ij} \in \mathbb{N_0} &amp;amp;\forall i = 1,2,\cdots,8 \hspace{0.2cm} \forall j = 1,2,3&#xA;\end{aligned}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone recommend any Python packages that would be able to solve this problem? Any commercial software with an interface to Python and a free academic license/evaluation period would also be great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; It should be noted that the optimization does not have to find a global minimum (although that is, of course, preferred).&lt;/p&gt;&#xA;" OwnerUserId="16558" LastEditorUserId="123" LastEditDate="2015-06-17T03:42:44.770" LastActivityDate="2019-04-03T00:21:35.483" Title="Python solvers for mixed-integer nonlinear constrained optimization" Tags="|optimization|python|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="19871" PostTypeId="2" ParentId="19852" CreationDate="2015-06-06T22:03:14.890" Score="2" Body="&lt;p&gt;Try learning some CFD via this course in MIT OpenCourseware:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ocw.mit.edu/courses/mechanical-engineering/2-29-numerical-fluid-mechanics-fall-2011/lecture-notes/&quot; rel=&quot;nofollow&quot;&gt;http://ocw.mit.edu/courses/mechanical-engineering/2-29-numerical-fluid-mechanics-fall-2011/lecture-notes/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-06-06T22:03:14.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19872" PostTypeId="2" ParentId="19860" CreationDate="2015-06-07T08:35:17.007" Score="0" Body="&lt;p&gt;When you are dealing with complex valued vectors, the inner product is probably defined as $(u,v)=u_1^*v_1+...+u_n^*v_n$, where * indicates the complex conjugate.  For example, the vector $u=(1,i)$ is &lt;em&gt;not&lt;/em&gt; orthogonal to $v=(-i,1)$, because $(u,v)=1(-i)+(i)^*(1)=-2i$.  On the other hand, $u$ is orthogonal to $w=(i,1)$.  You cannot just use the ordinary &quot;dot product&quot; to show complex vectors are orthogonal.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the test matrix $\left(\begin{matrix}1&amp;amp; -i \\ i&amp;amp; 1\end{matrix}\right)$.  This matrix is Hermitian and it has distinct eigenvalues 2 and 0 corresponding to the eigenvectors $u$ and $w$ respectively.  You can use this test matrix and its eigenvectors in Matlab to verify you calculating an inner product rather than what I call the ordinary &quot;dot product.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note, the inner product is defined as above in physics.  That's why I said &quot;probably defined as&quot;.  Mathematicians are more likely to define the inner product on complex vector spaces as $(u,v)=u_1v_1^*+...+u_nv_n^*$, which is just the complex conjugate of the one I defined above.&lt;/p&gt;&#xA;" OwnerUserId="11950" LastActivityDate="2015-06-07T08:35:17.007" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19873" PostTypeId="2" ParentId="19860" CreationDate="2015-06-07T09:27:44.407" Score="4" Body="&lt;p&gt;If the matrix is normal (i.e., $A^HA=AA^H$), you should indeed get orthonormal eigenvectors both theoretically or numerically. You can check this by numerically by taking the matrix &lt;code&gt;V&lt;/code&gt; built from columns of eigenvectors obtained from &lt;code&gt;[V,D] = eigs(A)&lt;/code&gt; and computing &lt;code&gt;V'*V&lt;/code&gt;, which should give you (very close to) the identity matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that you are not observing orthogonality most likely is due to the matrix not being normal (which you can also check numerically, e.g., by &lt;code&gt;norm(A'*A-A*A','fro')&lt;/code&gt;). This is not unsurprising: Although your differential operator (in particular, the bilaplacian) is self-adjoint, this need not be the case for its discretization. In particular, most ways of modifying the stiffness matrix (in your case, $B$) to incorporate Dirichlet boundary conditions destroys the symmetry.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-06-07T09:27:44.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19874" PostTypeId="2" ParentId="19858" CreationDate="2015-06-07T10:02:20.870" Score="3" Body="&lt;p&gt;It is true that weak convergence is most crucial in the continuum limit as $h\to 0$ (e.g., by not being able to observe &lt;em&gt;any&lt;/em&gt; convergence rate). At least in Hilbert spaces, it is also closely tied to non-uniqueness of the limit and hence only subsequential convergence (e.g., where you can alternate between approaching different limit points, again destroying rates), and it is difficult to separate the influence of the two on the convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically for weak convergence in $L^2$, you also have the fact that the convergence need not be pointwise, and this you can actually observe in a (sufficiently fine) discretization. Here's an example from a sequence of minimizers $\{u_\varepsilon\}_{\varepsilon&amp;gt;0}$ that converges as $\varepsilon\to 0$ to&#xA;$$ u(x) = \begin{cases} -1 &amp;amp; x&amp;lt;\frac13 \\ 0  &amp;amp;x\in [\frac13,\frac23] \\ 1 &amp;amp;x&amp;gt;\frac23\end{cases}$$&#xA;where the convergence is weak but not pointwise on $[\frac13,\frac23]$ (but pointwise almost everywhere else). The following figures show three representative elements from the sequence (for $\varepsilon$ already quite small).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gO7JB.png&quot; alt=&quot;weak convergence 1&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/LOdSA.png&quot; alt=&quot;weak convergence 2&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/m6W04.png&quot; alt=&quot;weak convergence 3&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This phenomenon is known as &quot;chittering&quot; in the approximation of bang-bang control problems for differential equations (i.e., problems with box constraints where the solution almost everywhere attains either the lower or upper bound).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This specific example is taken from our paper on &lt;a href=&quot;https://www.uni-due.de/~adf040p/preprints/MultibangControl.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-bang control of elliptic systems&lt;/a&gt;, Ann. Henri Poincaré (C) 2014, 1109-1130, Remark 4.2.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-06-07T10:26:23.663" LastActivityDate="2015-06-07T10:26:23.663" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19875" PostTypeId="2" ParentId="19075" CreationDate="2015-06-07T12:13:36.870" Score="2" Body="&lt;p&gt;The book &quot;Nonlinear regression&quot; by Seber and Wild references quite some paper on that matter. I think most implementations use $\mu\text{diag}(A)$ instead of $I$ in general, although I can not proof that it does perform better in all cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer Umesh Tiwari's comment (I could not comment myself): &#xA;$[A + \mu\text{diag}(A)]h=−g$ is indeed equivalent to what Marquardt did propose. He formulated: &#xA;$(A^*+\mu I)h^*=−g^*$&#xA;which can be modified as follows:&#xA;\begin{align}&#xA;(A^*+\mu I)h^* =&amp;amp; −g^* \\&#xA;A^*Dh +\mu Dh =&amp;amp; −D^{-1}g \\&#xA;DA^*Dh +\mu  D^2h =&amp;amp; −g \\&#xA;Ah +\mu  D^2h =&amp;amp; −g \\&#xA;(A +\mu  D^2)h =&amp;amp; −g \\&#xA;\end{align}&#xA;where $D^2$ is the diagonal of A.&lt;/p&gt;&#xA;" OwnerUserId="16557" LastEditorUserId="9667" LastEditDate="2015-06-08T14:39:02.003" LastActivityDate="2015-06-08T14:39:02.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19877" PostTypeId="2" ParentId="19870" CreationDate="2015-06-07T16:53:23.323" Score="4" Body="&lt;p&gt;The feasible region can be broken up into several &quot;convex&quot; integer lattice subdomains by setting out the specific combinations of sign-agreement pairs.  [That is, each sign agreement between $x_{7,j}$ and $x_{3,j}$, resp. between $x_{8,j}$ and $x_{4,j}$ can occur in one of two ways: both positive or both zero.  Given the three &quot;time&quot; indices $j=1,2,3$, there are $2^6 = 64$ &quot;components&quot; which are the integer lattice solutions of linear constraints.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the region described by linear constraints is too complicated or large for direct generation of all feasible points, a popular technique is to generate a random sample of the points, e.g. by a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo&quot; rel=&quot;nofollow&quot;&gt;Monte Carlo Markov Chain (MCMC) algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a Python implementation of MCMC samplers as &lt;a href=&quot;https://pypi.python.org/pypi/pymc&quot; rel=&quot;nofollow&quot;&gt;pymc&lt;/a&gt; by Christopher Fonnesbeck, Anand Patil and David Huard.  Although targeted to Bayesian estimation, the sampling portion of the toolkit could be used to generate &quot;random&quot; feasible points from each subdomain described above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimum would then be approximated by the least value of $f(x)$ attained over all the sampled points.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is a sketch of how one might generate all the feasible points for these particular constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the &quot;odd&quot; row and &quot;even&quot; row indices $i$ are linked only by the sum-to-ten constraint at top.  We can rearrange the unknowns from the $8\times 3$ matrix $x$ into a $4\times 6$ matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{pmatrix} x_{1,1} &amp;amp; x_{1,2} &amp;amp; x_{1,3} &amp;amp; x_{2,1} &amp;amp; x_{2,2} &amp;amp; x_{2,3} &#xA;   \\              x_{3,1} &amp;amp; x_{3,2} &amp;amp; x_{3,3} &amp;amp; x_{4,1} &amp;amp; x_{4,2} &amp;amp; x_{4,3} &#xA;   \\              x_{5,1} &amp;amp; x_{5,2} &amp;amp; x_{5,3} &amp;amp; x_{6,1} &amp;amp; x_{6,2} &amp;amp; x_{6,3} &#xA;   \\              x_{7,1} &amp;amp; x_{7,2} &amp;amp; x_{7,3} &amp;amp; x_{8,1} &amp;amp; x_{8,2} &amp;amp; x_{8,3} &#xA;   \end{pmatrix}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each column of non-negative integers $(a,b,c,d)^T$ in this form obeys the same constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ a \le b + c $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  {b = 0} \Leftrightarrow {d = 0} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ d \le 15 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can add the constraint $b+c \le 10$ deduced from the combined middle two row sums in the new form being &lt;em&gt;exactly&lt;/em&gt; ten, and we get a finite set of possibilities  $S \subset \mathbb{N}_0^4$ from which all six columns are chosen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore the sum-to-ten constraint for the two middle rows allows us to assign parts that add to $10$ for the six columns, thereby generating all the feasible points for this problem.  There are $35$ partitions of $10$ in not more than six parts, and $\binom{15}{5} = 3003$ weak compositions of $10$ in exactly six summands, so this appears to be a tractable task computationally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Added:&lt;/strong&gt; A program written to carry out these calculations found some 8 trillion feasible solutions.  More precisely there are 8,054,848,375,116 black box function evaluations needed to exhaust the search space.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2015-06-17T04:10:31.320" LastActivityDate="2015-06-17T04:10:31.320" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19878" PostTypeId="1" AcceptedAnswerId="19879" CreationDate="2015-06-07T18:35:06.220" Score="4" ViewCount="628" Body="&lt;p&gt;Given some equally spaced data for a function on an interval, we can use line segments (pictured below), quadratic functions, various cubic schemes, and increasingly higher order polynomials to approximate the function using this equally spaced data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/h7cBM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the linear scheme I show as an example in the figure above, as $h \rightarrow 0$, the approximation error also approaches zero, so we say the scheme is &quot;convergent&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general though, using increasingly higher order polynomials as approximations on the equally spaced intervals &lt;em&gt;does not&lt;/em&gt; guarantee convergence. &lt;a href=&quot;http://en.wikipedia.org/wiki/Polynomial_interpolation#Interpolation_error&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; has an explanation that says that $h^{n+1}f^{(n+1)}(\xi)$ may not approach zero as $h$ approaches zero, if $f^{(n+1)}(\xi)$ dominates. Can you show this in a problematic case using pictures?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="9466" LastEditDate="2015-06-07T18:43:35.163" LastActivityDate="2015-06-07T18:52:45.067" Title="Function approximation: why does using increasingly higher order approximating polynomials not lead to convergent approximation schemes in general?" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19879" PostTypeId="2" ParentId="19878" CreationDate="2015-06-07T18:52:45.067" Score="10" Body="&lt;p&gt;This problem is commonly called &quot;Runge's phenomenon&quot;, and is similar to Gibbs' phenomenon, which is encountered when using Fourier series to approximate a function. The Wikipedia pages &lt;a href=&quot;http://en.wikipedia.org/wiki/Polynomial_interpolation#Convergence_properties&quot; rel=&quot;nofollow noreferrer&quot;&gt;convergence of polynomial approximation schemes&lt;/a&gt;, and on &lt;a href=&quot;http://en.wikipedia.org/wiki/Runge%27s_phenomenon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Runge's phenomenon&lt;/a&gt; have a clear discussion, along with a picture to show the phenomenon in action:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FnPAD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Caption from Wikipedia&lt;/strong&gt; (emphasis mine): The &lt;strong&gt;red&lt;/strong&gt; curve is the Runge function.&#xA;  The &lt;strong&gt;blue&lt;/strong&gt; curve is a 5th-order interpolating polynomial (using six equally spaced interpolating points).&#xA;  The &lt;strong&gt;green&lt;/strong&gt; curve is a 9th-order interpolating polynomial (using ten equally spaced interpolating points).&#xA;  At the interpolating points, the error between the function and the interpolating polynomial is (by definition) zero. Between the interpolating points (especially in the region close to the endpoints 1 and −1), the error between the function and the interpolating polynomial gets worse for higher-order polynomials.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2015-06-07T18:52:45.067" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19882" PostTypeId="1" CreationDate="2015-06-08T20:21:51.457" Score="0" ViewCount="679" Body="&lt;p&gt;I wanted to know the procedure of starting up a water vapor calculation in LAMMPS. I want to calculate properties of water (i.e. in vapor state) at a higher temperature than 100 Celsius. Currently I am setting up the simulation same as liquid water but with the density of $0.001\, \text{g/cm}^3$.  Initially my box has 1000 times less molecules than that of liquid water and it has 800 atoms in $200\times200\times200\ \mathring{\text{A}}^3$ box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am running this simulation in LAMMPS 2014. I have tried all ensembles (NVT, NPT and NVE) with various time intervals. The simulation starts running for only NVE but for a very small time-step of $0.001\, \text{fs}$. If I increase the time interval, the simulation stops with an error &quot;Out of range atoms - cannot compute PPPM&quot;. The error also comes if I run a NPT simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if anybody can help me with this problem. I can also attach my input scripts and data file if needed.&lt;/p&gt;&#xA;" OwnerUserId="16576" LastEditorUserId="9667" LastEditDate="2015-06-09T20:48:05.537" LastActivityDate="2017-05-31T16:59:55.937" Title="MD simulation - Water Vapour" Tags="|molecular-dynamics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19883" PostTypeId="2" ParentId="7597" CreationDate="2015-06-08T23:55:45.750" Score="2" Body="&lt;p&gt;A very good start is the book:&#xA;Low-Speed Aerodynamics by Joseph Katz and Allen Plotkin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It describes a variety of panel methods for airfoils and wings,&#xA;and, best of all, it includes source code.&lt;/p&gt;&#xA;" OwnerUserId="16571" LastActivityDate="2015-06-08T23:55:45.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19884" PostTypeId="2" ParentId="19852" CreationDate="2015-06-09T03:54:31.473" Score="1" Body="&lt;p&gt;I am in the same boat and collected some newbee &quot;feral learning&quot; information here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.vespalabs.org/projects/vespa-cfd-3d-model&quot; rel=&quot;nofollow&quot;&gt;http://www.vespalabs.org/projects/vespa-cfd-3d-model&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you state what type of CFD you are working with then you should be able to narrow down your learning curve. For example, My interest is in CFD for scooter/motorcycle streamlining (as well as a HPC learning exercise) and I found the following books great value for money (due to age) and are written in a more accessible language (i.e. assumed prior knowledge is low)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aerodynamics of Road Vehicles (4th Edition): From Fluid Mechanics to Vehicle Engineering ([Proceedings] / SAE)&quot;  - Hucho, Wolf-Heinrich.'s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computational Fluid Dynamics - The Basics with Applications. John Anderson. 1995&lt;/p&gt;&#xA;" OwnerUserId="564" LastActivityDate="2015-06-09T03:54:31.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19885" PostTypeId="1" CreationDate="2015-06-09T05:55:29.917" Score="3" ViewCount="145" Body="&lt;p&gt;I am curious about how much one loses in terms of computational cost when complex numbers are used instead of real numbers? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess the number of floating-point operations and memory doubles, but I am not sure if the wall time doubles necessarily. I am especially asking this for matrix factorizations, &lt;span class=&quot;math-container&quot;&gt;$LU$&lt;/span&gt;, or &lt;span class=&quot;math-container&quot;&gt;$LDL^T$&lt;/span&gt; factorizations of complex symmetric (non-Hermitian) vs real-symmetric matrices. I am not sure if the answer changes based on using sparse or dense storage, but I'd appreciate if you comment on both situations. Obviously symmetric factorization cost should not change, but numerical factorizations should have a performance penalty. &lt;/p&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="20688" LastEditDate="2020-05-01T06:46:53.840" LastActivityDate="2020-05-01T06:46:53.840" Title="What is the computational cost of using complex numbers in contrast to real numbers in matrix operations, e.g. $LU$ or $LDL^T$ factorizations?" Tags="|complexity|matrix-factorization|complex-arithmetic|" AnswerCount="0" CommentCount="2" ClosedDate="2018-09-23T20:51:23.583" ContentLicense="CC BY-SA 4.0" />
  <row Id="19886" PostTypeId="1" CreationDate="2015-06-09T15:17:06.627" Score="4" ViewCount="225" Body="&lt;p&gt;I would like to use the  &lt;a href=&quot;http://www.cs.ox.ac.uk/pseudospectra/eigtool/download/&quot; rel=&quot;nofollow&quot;&gt;Eigtool of professor Trefethen&lt;/a&gt; for pseudospectra but I have a generalized eigenvalue problem to solve:&#xA;$$ \lambda M x = K x. $$ It seems that Eigtool takes only one matrix as input. Is it possible to use the Eigtool my case too? I.e. having two matrices as input? &#xA;In addition, my matrices are really big, but sparse, so I do not think that it is possible to invert them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case $M$ is singular and not invertible.&lt;/p&gt;&#xA;" OwnerUserId="14489" LastEditorUserId="14489" LastEditDate="2015-06-12T15:35:11.077" LastActivityDate="2015-06-13T13:41:40.720" Title="Is it possible to use Eigtool for generalized problem pseudospectra?" Tags="|eigenvalues|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19888" PostTypeId="2" ParentId="3029" CreationDate="2015-06-09T18:36:26.517" Score="1" Body="&lt;p&gt;(I hope there is no objection to my tacking onto the end of this discussion. I'm new here, so please let me know if I have transgressed!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Test functions for evolutionary algorithms are now much more complicated than they were even 2 or 3 years ago, as can be seen by the suites used in competitions at conferences like the (very recent) 2015 Congress on Evolutionary Computation. See:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cec2015.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.cec2015.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These test suites now include functions with several non-linear interactions between variables. The number of variables can be as large as 1000, and I would guess that might increase in the near future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another very recent innovation is a &quot;Black Box Optimization Competition&quot;.&#xA;See: &lt;a href=&quot;http://bbcomp.ini.rub.de/&quot; rel=&quot;nofollow&quot;&gt;http://bbcomp.ini.rub.de/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An algorithm can query the value f(x) for a point x, but it does not obtain gradient information, and in particular it cannot make any assumptions on the analytic form of the objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a sense, this might be closer to what you referred to as a &quot;real problem&quot; but in an organised, objective setting.&lt;/p&gt;&#xA;" OwnerUserId="16571" LastActivityDate="2015-06-09T18:36:26.517" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19889" PostTypeId="2" ParentId="19862" CreationDate="2015-06-09T18:38:40.333" Score="0" Body="&lt;p&gt;Fenics has support for FSI via Unicorn, that is an extension for the flexible handling of PDE systems problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should check out the freely available Fenics book that contains a detailed exposition of the topic [1]. You should be able to find example code for FSI problems as well, that come with Unicorn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Logg, Anders, Kent-Andre Mardal, and Garth Wells, eds. Automated solution of differential equations by the finite element method: The FEniCS book. Vol. 84. Springer Science &amp;amp; Business Media, 2012.; &lt;strong&gt;&lt;em&gt;Chapter 28 Turbulent flow and fluid–structure interaction&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;URL: &lt;a href=&quot;https://launchpadlibrarian.net/83776282/fenics-book-2011-10-27-final.pdf&quot; rel=&quot;nofollow&quot;&gt;https://launchpadlibrarian.net/83776282/fenics-book-2011-10-27-final.pdf&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="3229" LastActivityDate="2015-06-09T18:38:40.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19890" PostTypeId="2" ParentId="7452" CreationDate="2015-06-09T18:49:50.400" Score="0" Body="&lt;p&gt;Autodesk has multiple CFD offerings (Simulation CFD, NASTRAN) that would integrate well with your Inventor based workflow. If you are a student you can have free access to these tools following a simple registration process (&lt;a href=&quot;http://www.autodesk.com/education/free-software/simulation-cfd&quot; rel=&quot;nofollow&quot;&gt;http://www.autodesk.com/education/free-software/simulation-cfd&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aside from the selection of the appropriate fluid model, would you like to model the interaction with the cells as well? For this purpose a Lagrangian (particle based) model for the E.coli might be suitable.&lt;/p&gt;&#xA;" OwnerUserId="3229" LastActivityDate="2015-06-09T18:49:50.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19891" PostTypeId="1" CreationDate="2015-06-09T20:17:07.820" Score="1" ViewCount="163" Body="&lt;p&gt;Can I start parallel run of LAMMPS from inside a C++ code. Is it possible to compile my code and then do mpirun with my compiled code. I also have one system command inside the code, which starts parallel instances of LAMMPS with mpirun. &lt;/p&gt;&#xA;" OwnerUserId="16576" LastActivityDate="2020-05-24T21:09:56.543" Title="C++ - Starting paraller instances of an executable from a C++ code" Tags="|parallel-computing|c++|mpi|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19892" PostTypeId="2" ParentId="18882" CreationDate="2015-06-09T20:29:18.867" Score="1" Body="&lt;p&gt;I see three possible storage strategies:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; n = 1000;&#xA;&amp;gt;&amp;gt; l = 2.34;&#xA;&amp;gt;&amp;gt; K = randn(n,n); K = K*K';&#xA;&amp;gt;&amp;gt; Ku = triu(K);&#xA;&amp;gt;&amp;gt; p = 0; for i = 1:n, kvu(p+(1:i)) = K(1:i,i); p = p + i; end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;K&lt;/code&gt; is the full matrix&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Ku&lt;/code&gt; is the upper triangular part stored as a full matrix&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;kvu&lt;/code&gt; is a vectorization of the non zero elements of &lt;code&gt;Ku&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and these are my timings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; tic, r = K.^l; toc&#xA;Elapsed time is 0.033273 seconds.&#xA;&amp;gt;&amp;gt; tic, r = Ku.^l; toc&#xA;Elapsed time is 0.029381 seconds.&#xA;&amp;gt;&amp;gt; tic, r = kvu.^l; toc&#xA;Elapsed time is 0.019505 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;Ku&lt;/code&gt; approach is interesting, since a lot of matrix functions for symmetric positive definite matrices access only the upper triangular part: e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; isequal(chol(K), chol(Ku))&#xA;&#xA;ans =&#xA;&#xA;     1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course the ideal situation is if the optimisation algorithm could be rewritten in terms of &lt;code&gt;kvu&lt;/code&gt; instead of &lt;code&gt;K&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2015-06-09T20:29:18.867" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19894" PostTypeId="1" CreationDate="2015-06-09T21:30:32.523" Score="2" ViewCount="948" Body="&lt;p&gt;I am looking at a very simple problem, but can't quite find the best solution. I need to accept a lat/lon coordinate and based on that coordinate find all the points within roughly ~1km (accuracy is not too important to me). It will always be ~1km searches as well (fixed). I now face how to store these coordinates in my database and how to retrieve the results very quickly. I am open to using any database or language to complete this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am using MongoDB and there 2D spatial index (&lt;a href=&quot;http://docs.mongodb.org/manual/applications/geospatial-indexes/&quot; rel=&quot;nofollow&quot;&gt;http://docs.mongodb.org/manual/applications/geospatial-indexes/&lt;/a&gt;) to store my locations as lat/long on a flat surface. I am then creating a bounding box (accuracy is not super important to me, so I accept with a box the distance is not the same in all directions) and using a bounding box search (&lt;a href=&quot;http://docs.mongodb.org/manual/reference/operator/query/box/&quot; rel=&quot;nofollow&quot;&gt;http://docs.mongodb.org/manual/reference/operator/query/box/&lt;/a&gt;) for to get all the points. This approach brings decent performance, but I am looking for faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know databases really love integer based indexes. They perform the quickest. I was looking for a way to maybe convert coordinates into integers or something along those lines?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know some databases such as MySQL 5.7 have spatial index which utilize r-trees which is great for vast geospatial operations, but I have what I believe is a simple use case which can avoid these indexes and utilize faster structures such as native integers, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some thoughts on algorithms which could be utilized: z-order, hilbert, x-tree, geohash, kd-tree, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;To summarize my ultimate goal:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use accept a lat/lon coordinate and transform this coordinate which can then be best stored in the database for very fast nearby searches on the database. I am open to any methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;" OwnerUserId="10370" LastEditorUserId="10370" LastEditDate="2015-06-22T19:52:27.993" LastActivityDate="2015-06-26T17:01:46.997" Title="Ideas on how to search nearby geospatial data fast" Tags="|optimization|algorithms|computational-geometry|geometry|spatial-data|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19895" PostTypeId="1" AcceptedAnswerId="19977" CreationDate="2015-06-09T22:39:15.583" Score="1" ViewCount="1353" Body="&lt;p&gt;I am self teaching myself python and computational physics via Mark Newmans book Computational Physics the exercise is 5.17 of Computational Physics. I have to shift the limits of integration for an improper integral after a change of variables so that Gaussian quadrature method gives a more accurate result. I am suppose to approximate the Gamma function $\Gamma(a)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The relevant equation is here:&lt;/strong&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Lb4WA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This is the question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/yEAjy.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/zAAjy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;When I isolate for $x$ I get $x = \frac{zc}{1-x}$.  But when $z = \frac{1}{2}$ I get $x = c$ but this makes no sense because this implies z is a constant which it's not. I know in order to shift a function $f(x)$ to the left you $f(x + c)$ but how would I do this with an integrand?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9607" LastActivityDate="2015-06-19T10:03:43.417" Title="Improper Numerical integral" Tags="|python|numerical-analysis|numerics|integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19896" PostTypeId="1" AcceptedAnswerId="19903" CreationDate="2015-06-09T23:04:55.820" Score="3" ViewCount="887" Body="&lt;p&gt;Typically I have seen P2/P1 elements used for Stokes equation, but I want to use P2/(P1+P0) and P2/P0 elements because I want to ensure local mass conservation. When I say P2/(P1+P0), I simply mean the concatenation of the basis for P1 and P0 separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran simple problems in FEniCS with both element pairs and while the velocity works great in both cases (i.e., element-wise conservative), the pressure does get a little screwed up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been told mixed opinions about using anything with piecewise constants for pressure. I have heard that P2/P0 is not LBB stable for 3D elements (however, I fail to see this happening because I do not get node-to-node spurious oscilliations) although this could potentially explain why my pressure looks screwed up. I have also heard that P2/P0 should be doable in 3D (hence P2/(P1+P0) should work as well).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I guess my question is, theoretically are P2/P0 and/or P2(P1+P0) elements stable and usable for 3D Stokes and Darcy problems?&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2015-06-10T15:26:47.933" Title="P2/P0 and P2/(P1+P0) finite elements for stokes and darcy equations" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19899" PostTypeId="1" CreationDate="2015-06-10T06:21:10.237" Score="1" ViewCount="79" Body="&lt;p&gt;I have a set of four non-linear ODEs representing a negative feedback. I have done parameter variation by random sampling to study the sensitivity of steady state and other dynamic properties to parameter fluctuations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the analytical expressions of steady state I have calculated jacobian and its eigenvalues for each parameter set and based on that I could observe the fraction of oscillating cases (complex eigenvalues). I wish to calculate something like damping factor for these cases. I am not sure what to use as a metric. I think of three choices:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Least eigenvalue (absolute value of the real part). This would be rate limiting.&lt;/li&gt;&#xA;&lt;li&gt;Biggest eigenvalue (absolute value of the real part). This might dictate the initial damping (which is what I am basically interested in).&lt;/li&gt;&#xA;&lt;li&gt;Determinant of the jacobian (?)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Please suggest me what is the right metric or let me know if there are better alternatives.&lt;/p&gt;&#xA;" OwnerUserId="14551" LastActivityDate="2015-06-10T06:21:10.237" Title="&quot;Damping factor&quot; for a set of non-linear ODEs" Tags="|jacobian|oscillations|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19900" PostTypeId="1" CreationDate="2015-06-10T10:10:45.487" Score="5" ViewCount="160" Body="&lt;p&gt;I have an inverse problem in which the optimal positions for a variable number of injections needs to be determined. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the number of injections was fixed, I could easily imagine implementing Simplex or Genetic Algorithm (or really any optimisation algorithm). With a changing number of variables, however, it seems that reflection in Simplex and crossover in GA become impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could use a meta-optimiser to find the optimal positions for a given number of injections and repeat this process for every number of injections, but this is computationally expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any thoughts on what I could do? Or any sources you can point me to (searching &quot;optimising a variable number of variables&quot; unsurprisingly isn't very fruitful)?&lt;/p&gt;&#xA;" OwnerUserId="16592" LastActivityDate="2015-07-17T02:38:18.427" Title="Inverse problem with changing number of variables" Tags="|inverse-problem|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19901" PostTypeId="2" ParentId="19900" CreationDate="2015-06-10T12:37:47.773" Score="6" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I have an inverse problem in which the optimal positions for a variable number of injections needs to be determined.&lt;/p&gt;&#xA;&lt;p&gt;Are there any thoughts on what I could do? Or any sources you can point me to (searching &amp;quot;optimising a variable number of variables&amp;quot; unsurprisingly isn't very fruitful)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;One potential strategy is using a mixed-integer formulation, which is pretty common in applications such as heat exchange network synthesis, or chemical process design, where the number of pieces of equipment used in the design is unknown &lt;em&gt;a priori&lt;/em&gt;, but usually bounded above by some number (e.g., due to cost constraints, or some application-specific intuition).&lt;/p&gt;&#xA;&lt;p&gt;The basic idea is that you assume you have at most, say, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; injectors, and you introduce integer variables that model how many injectors you have; usually, formulations use binary variables, which can make it easier to &amp;quot;inactivate&amp;quot; constraints corresponding to injectors that don't exist, and &amp;quot;activate&amp;quot; constraints corresponding to the number of injectors currently under consideration.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If the number of injections was fixed, I could easily imagine implementing Simplex or Genetic Algorithm (or really any optimisation algorithm). With a changing number of variables, however, it seems that reflection in Simplex and crossover in GA become impossible.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I presume you mean a Nelder-Mead Simplex algorithm? In theory, it is possible to change the number of decision variables mid-iteration; this sort of strategy is done, for instance, when reduced order models are used within optimization problems (for instance, SQP solvers), although it requires some modifications to the underlying numerical methods to account for the changes in the number of variables; you need (pseudo)invertible mappings between the different dimensional spaces under consideration, and you need an implementation that enables you to change the number of variables mid-method. Most implementations assume the number of variables is fixed once initial problem data is specified, which poses an implementation problem from the outset.&lt;/p&gt;&#xA;&lt;p&gt;I think the basic problem with this sort of approach for your application is that unlike something like model reduction or adaptive mesh refinement, it's less clear how to decide cheaply when to change the number of decision variables, and by how many.&lt;/p&gt;&#xA;&lt;p&gt;The first strategy that comes to mind is one you suggest:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I could use a meta-optimiser to find the optimal positions for a given number of injections and repeat this process for every number of injections, but this is computationally expensive.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;...and this sort of strategy is going to be expensive, as you point out. A mixed-integer formulation would also be expensive, but with a good solver and good choices of heuristics, you might get lucky.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-06-10T12:37:47.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19902" PostTypeId="1" AcceptedAnswerId="19937" CreationDate="2015-06-10T14:28:51.943" Score="9" ViewCount="4427" Body="&lt;p&gt;I'm looking for an open-source (to use and learn from) software which computes Bessel functions of integer order of real argument to &lt;a href=&quot;https://en.wikipedia.org/wiki/Double-precision_floating-point_format&quot;&gt;double precision&lt;/a&gt; the fastest among all such implementations. Currently I've tried &lt;a href=&quot;http://www.boost.org/doc/libs/1_55_0/libs/math/doc/html/math_toolkit/bessel/bessel_first.html&quot;&gt;Boost.Math&lt;/a&gt; and &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Bessel-Functions.html&quot;&gt;GSL&lt;/a&gt;. From these GSL appeared to be faster for smaller arguments and much slower for larger ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any implementations specially designed to be very fast for the whole working ranges of arguments and orders, but still not neglecting precision?&lt;/p&gt;&#xA;" OwnerUserId="7873" LastActivityDate="2015-07-30T01:36:03.713" Title="What is the fastest opensource implementation of Bessel functions computation?" Tags="|performance|special-functions|open-source|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19903" PostTypeId="2" ParentId="19896" CreationDate="2015-06-10T14:59:46.537" Score="4" Body="&lt;p&gt;The tables starting on page 462 of Gresho and Sani's book &quot;Incompressible Flow and the Finite Element Method&quot; say that $P_2 P_0$ is stable but first-order only and $P_2(P_1+P_0)$ is also stable but has two hydrostatic modes that you should account for. This is all for Stokes, so you should look for other references for Darcy flow.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-06-10T15:26:47.933" LastActivityDate="2015-06-10T15:26:47.933" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19904" PostTypeId="2" ParentId="16052" CreationDate="2015-06-10T16:17:39.993" Score="1" Body="&lt;p&gt;I have worked on simpler integrals where there are points of stationary phase.&#xA;I found two methods that work quite well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One is to introduce an exponential damping factor that depends on the phase &#xA;function, a kind of artificial viscosity if you like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another technique (where there are multiple points of stat. phase) was described in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tuck, E.O, Collins, J.L. and Wells, W.H., &quot;On ship waves and their spectra&quot;, Journal of Ship Research, pp. 11–21, 1971.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That method applies exponential decay factors to the integrand where it&#xA;becomes rapidly-oscillating away from the stat. phase points, but leaves the integrand intact where it is&#xA;not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's me out of ideas!&lt;/p&gt;&#xA;" OwnerUserId="16571" LastEditorUserId="16571" LastEditDate="2015-06-10T19:10:22.780" LastActivityDate="2015-06-10T19:10:22.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19905" PostTypeId="2" ParentId="19886" CreationDate="2015-06-10T17:13:05.457" Score="1" Body="&lt;p&gt;Maybe, you set $\lambda x = M^{-1}Kx$?, then if we need to implement the matrix-vector product in Eigtool:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y = M^{-1}K v$, $v$ is any given vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can do it by two steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Compute the matrix-vector product: $u = Kv$; (2) Solve linear systems $My = u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that you can modified some lines in Eigtool.&lt;/p&gt;&#xA;" OwnerUserId="10290" LastActivityDate="2015-06-10T17:13:05.457" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19906" PostTypeId="2" ParentId="19902" CreationDate="2015-06-10T20:53:16.530" Score="2" Body="&lt;p&gt;This is really going to require some testing with regards to speed, but here are some examples I remember coming across that aren't Boost or GSL: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A wrapper of the Fortran routines by DE Amos is &lt;a href=&quot;https://code.google.com/p/complex-bessel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and on &lt;a href=&quot;https://github.com/valandil/complex_bessel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Integer order in &lt;a href=&quot;http://www.alglib.net/specialfunctions/bessel.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;ALGLIB here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, a related SO question &lt;a href=&quot;https://stackoverflow.com/questions/29145922/integration-of-bessel-functions-in-c-fortran&quot;&gt;here with links to other C implementations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8094" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-06-10T20:53:16.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19908" PostTypeId="1" CreationDate="2015-06-11T05:00:20.843" Score="0" ViewCount="117" Body="&lt;p&gt;I have a set of first 25 &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.123.2644&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;Zernike polynomials&lt;/a&gt;. Below are shown few in Cartesin co-ordinate system.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z2 = 2*x&#xA;&#xA;z3 = 2*y&#xA;&#xA;z4 = sqrt(3)*(2*x^2+2*y^2-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z24 = sqrt(14)*(15*(x^2+y^2)^2-20*(x^2+y^2)+6)*(x^2-y^2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not using 1st since it is piston; so I have these 24 two-dim ANALYTICAL functions expressed in X-Y Cartesian co-ordinate system. All are defined over unit circle, as they are orthogonal over unit circle. The problem which I am describing here is relevant to other 2D surfaces also apart from Zernike Polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that origin (0,0) of the XY co-ordinate system and the centre of the unit circle are same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I take linear combination of these 24 polynomials to build a 2D wavefront shape. I use 24 random input coefficients in this combination.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w(x,y) = sum_over_i   a_i*z_i         (i=2,3,4,....24)&#xA;&#xA;a_i = random coefficients&#xA;z_i = zernike polynomials&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Upto this point, everything is analytical part which can be done on paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now comes the discretization!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that when you want to re-construct a signal (1Dim/2Dim), your sampling frequency should be at least twice the maximum frequency present in the signal (Nyquist-Shanon principle).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here signal is w(x,y) as mentioned above which is nothing but a simple 2Dim&#xA;function of x &amp;amp; y. I want to represent it on computer now. Obviously I can not take all infinite points from -1 to +1 along x axis and same for y axis.&#xA;I have to take finite no. of data points (which are called sample points or just samples) on this analytical 2Dim surface w(x,y) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am measuring x &amp;amp; y in metres, and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; -1 &amp;lt;= x &amp;lt;= +1; -1 &amp;lt;= y &amp;lt;= +1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;e.g. If I divide my x-axis from -1 to 1, in 50 sample points then dx = 2/50= 0.04 metre. Same for y axis. Now my sampling frequency is 1/dx i.e. 25 samples per metre. Same for y axis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I took 50 samples arbitrarily; I could have taken 10 samples or 1000 samples. That is the crux of the matter here: how many samples points?How will I determine this number?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one theorem (Nyquist-Shanon theorem) as mentioned above which says that if I want to re-construct w(x,y) faithfully, I must sample it on both axes so that my sampling frequency (i.e. no. of samples per metre) is at least twice the maximum frequency present in the w(x,y). This is nothing but finding power spectrum of w(x,y). Idea is that any function in space domain can be represented in spatial-frequency domain also, which is nothing but taking Fourier transform of the function! This tells us how many (spatial) frequencies are present in your function w(x,y) and what is the maximum frequency out of these many frequencies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is first how to find out this maximum sampling frequency in my case. I can not use MATLAB fft2() or any other tool since it means already I have samples taken across the wavefront!! Obviously remaining option is find it analytically ! But that is time consuming and difficult since I have 24 polynomials &amp;amp; I will have to use then continuous Fourier transform i.e. I will have to go for pen and paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="16599" LastEditorUserId="16599" LastEditDate="2015-06-12T05:50:50.483" LastActivityDate="2015-06-12T08:35:43.743" Title="How many sample points are needed when re-constructing linear combination of 2D polynomials defined over unit circle?" Tags="|numerical-analysis|computational-physics|numerical-modelling|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="19909" PostTypeId="2" ParentId="19908" CreationDate="2015-06-11T11:40:00.200" Score="2" Body="&lt;p&gt;Without additional assumptions on the structure of the set if sampling points there can't be an upper bound. The reason is that a 2D polynomial can have a 1 dimensional set of zeros. If your sampling point all lie in this set, the polynomial looks like the zero polynomial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the edit it becomes a bit more clear what you are after. As I understand it, you want to sample linear combinations of Zernicke polynomials on a (uniformly spaces?) rectangular grid and want to know how fine your grid needs to be such that you can reconstruct the coefficients of the linear combination from the functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, Shannon's sampling theorem will not be helpful here.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You could view Zernike polynomials as functions on the unit disk, but there is no Shannon theorem for functions on the unit disk.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could extend the Zernicke polynomials to the whole plane by zero extension, making them square integrable functions. Unfortunately, these functions will have infinite bandwidth (as they are not continuous).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could extend the Zernike polynomials to the whole plane but using their formula. This would give smooth functions on the plane but unfortunately, they will not be square integrable (not even bounded), and hence, Shannon does not apply.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What you can do is the following: Let $z_1,\dots,z_k$ be your Zernike polynomials (i.e. $k=24$) and let $(x_i,y_i)_{i=1,\dots,n}$ be your sampling points (I am ignoring the grid structure here). Then the mapping that takes coefficients to sampling values is&#xA;$$&#xA;M: \mathbb{R}^k\to\mathbb{R}^n,\qquad (a_1,\dots,a_k) \mapsto \sum_{j=1}^k a_j z_j(x_i,y_i)&#xA;$$&#xA;and is &lt;strong&gt;linear&lt;/strong&gt; in the coefficients. I.e. you can express that mapping as a matrix and need the see if it has full rank.&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="302" LastEditDate="2015-06-12T08:35:43.743" LastActivityDate="2015-06-12T08:35:43.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19910" PostTypeId="1" AcceptedAnswerId="19913" CreationDate="2015-06-11T17:10:45.857" Score="24" ViewCount="11211" Body="&lt;p&gt;I know that the Nitsche's method is a very attractive methods since it allows to take into account Dirichlet type boundary conditions or contact with friction boundary conditions in a weak way without the use of Lagrange multipliers. And its advantage, which is to transform a Dirichlet boundary condition into weak terms similarly as a Neumann boundary condition, is paid by the fact that the implementation is model dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it seems to be too general for me. Can you give me more specific idea of this method? A simple example would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4670" LastEditorUserId="6775" LastEditDate="2017-06-10T21:48:39.147" LastActivityDate="2017-06-10T21:48:39.147" Title="What is the general idea of Nitsche's method in numerical analysis?" Tags="|finite-element|boundary-conditions|elliptic-pde|nitsche-method|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19911" PostTypeId="1" CreationDate="2015-06-11T18:46:49.970" Score="1" ViewCount="1464" Body="&lt;p&gt;would you please help me to solve following problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^*= \text{argmin}\ xLx^T+ |P^Tx|$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x$  is binary &lt;/li&gt;&#xA;&lt;li&gt;$P$ is a known vector (with positive and negative values)&lt;/li&gt;&#xA;&lt;li&gt;$L$ is Laplacian matrix&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have limited knowledge on optimization. As I read on the web, I can solve the problem with quadratic MILP if there was no absolute  value for the second element of the objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Since $P$ is not a constant positive value, how can get rid of the absolute value?&lt;/p&gt;&#xA;" OwnerUserId="16605" LastEditorUserId="9667" LastEditDate="2015-06-11T19:05:38.757" LastActivityDate="2015-06-26T15:06:45.550" Title="Solving absolute value quadratic optimization problem" Tags="|optimization|quadratic-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19912" PostTypeId="2" ParentId="19911" CreationDate="2015-06-11T19:06:26.767" Score="4" Body="&lt;p&gt;You can reformulate this as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x^{*}=\arg \min x^{T}Lx+ t $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t \geq P^{T}x $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t \geq -P^{T}x $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x \in \left\{ 0, 1 \right\}^{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a 0-1 mixed integer quadratic programming problem (MIQP) The case where $L$ is positive definite is well handled by solvers like CPLEX.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-06-11T19:06:26.767" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19913" PostTypeId="2" ParentId="19910" CreationDate="2015-06-12T12:31:33.940" Score="33" Body="&lt;p&gt;Nitsche's method is related to discontinuous Galerkin methods (indeed, as Wolfgang points out, it is a precursor to these methods), and can be derived in a similar fashion. Let's consider the simplest problem, Poisson's equation:&#xA;$$&#xA;\left\{\begin{aligned}&#xA;-\Delta u &amp;amp;= f \qquad\text{on }\Omega,\\&#xA; u &amp;amp;= g \qquad\text{on }\partial\Omega.&#xA;\end{aligned}\right.&#xA;\tag{1}&#xA;$$&#xA;We are now looking for a variational formulation that&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;is satisfied by the (weak) solution $u\in H^1(\Omega)$ (i.e., consistent),&lt;/li&gt;&#xA;&lt;li&gt;is symmetric in $u$ and $v$,&lt;/li&gt;&#xA;&lt;li&gt;admits a unique solution (which means that the bilinear form is coercive).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We start as usual by taking the strong form of the differential equation, multiplying by a test function $v\in H^1(\Omega)$ and integrating by parts. Starting with the right-hand side, we obtain&#xA;$$&#xA;\begin{aligned}&#xA;(f ,v) = (-\Delta u,v)&amp;amp;=(\nabla u,\nabla v) - \int_{\partial\Omega} \partial_\nu u v\,ds \\&#xA;&amp;amp;= (\nabla u,\nabla v) - \int_{\partial\Omega} \partial_\nu u v\,ds - \int_{\partial\Omega} (u-g)\partial_\nu v\,ds&#xA;\end{aligned}&#xA;$$&#xA;where in the last equation we have added the productive zero $0=u-g$ on the boundary. Rearranging the terms to separate linear and bilinear forms now gives a variational equation for a symmetric bilinear form that is satisfied for the solution $u\in H^1(\Omega)$ of $(1)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bilinear form is however not coercive, since you cannot bound it from below for $u=v$ by $c\|v\|_{H^1}^2$ (as we don't have any boundary conditions for arbitrary $v\in H^1(\Omega)$, we cannot use Poincaré's inequality as usual -- this means we can make the $L^2$ part of the norm arbitrarily large without changing the bilinear form). So we need to add another (symmetric) term that vanishes for the true solution: $\eta\int_{\partial\Omega} (u-g)v\,ds$ for some $\eta&amp;gt;0$ large enough. This leads to the (symmetric, consistent, coercive) weak formulation: Find $u\in H^1(\Omega)$ such that&#xA;$$&#xA;(\nabla u,\nabla v) - \int_{\partial\Omega} \partial_\nu u v\,ds - \int_{\partial\Omega} u\partial_\nu v\,ds +\eta\int_{\partial\Omega} u v\,ds = -\int_{\partial\Omega} g\partial_\nu v\,ds + \eta\int_{\partial\Omega} g v\,ds + \int_\Omega f v\,dx&#xA;\qquad\text{for all }v\in H^1(\Omega).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking instead of $u,v\in H^1(\Omega)$ discrete approximations $u_h,v_h\in V_h\subset H^1(\Omega)$ yields the usual Galerkin approximation. Note that since it's non-conforming due to the boundary conditions (we are looking for the discrete solution in a space that is &lt;em&gt;larger&lt;/em&gt; than the one we sought the continuous solution in), one cannot deduce well-posedness of the discrete problem from that of the continuous problem. Nitsche now showed that if $\eta$ is chosen as $ch^{-1}$ for $c&amp;gt;0$ sufficiently large, the discrete problem is in fact stable (with respect to a suitable mesh-dependent norm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is not Nitsche's original derivation, which predates discontinuous Galerkin methods and starts from an equivalent minimization problem. In fact, &lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF02995904&quot;&gt;his original paper&lt;/a&gt; does not mention the corresponding bilinear form at all, but you can find it in, e.g., &lt;a href=&quot;http://math.aalto.fi/~rstenber/Publications/Venice95.pdf&quot;&gt;Freund and Stenberg, &lt;em&gt;On weakly imposed boundary conditions for second-order problems&lt;/em&gt;, Proceedings of the Ninth Int. Conf. Finite Elements in Fluids, Venice 1995. M. Morandi Cecchi et al., Eds. pp. 327-336&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-09-22T11:14:40.780" LastActivityDate="2015-09-22T11:14:40.780" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="19914" PostTypeId="1" AcceptedAnswerId="19915" CreationDate="2015-06-12T13:37:07.697" Score="1" ViewCount="141" Body="&lt;p&gt;Assume I have 3 levels of grids. Finest Grid = level 2, Coarser Grid = level 1, Coarsest Grid = level 0.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Relax $u$ on $Au = b$ at level 2 for 3 times.&lt;/li&gt;&#xA;&lt;li&gt;Find residual $r2$ at level 2, then restrict to level 1&lt;/li&gt;&#xA;&lt;li&gt;Relax $e$ on $Ae=r$ at level 1 for 3 times.&lt;/li&gt;&#xA;&lt;li&gt;Find residual $r1$ at level 1, then restrict to level 0.&lt;/li&gt;&#xA;&lt;li&gt;Solve $Ae=r$ on level 0 till convergence.&lt;/li&gt;&#xA;&lt;li&gt;Interpolate $e$ on level 0 to level 1.&lt;/li&gt;&#xA;&lt;li&gt;Correct $e$ on level 1 and relax $e$ 3 more times.&lt;/li&gt;&#xA;&lt;li&gt;Interpolate $e$ on level 1 to level 2.&lt;/li&gt;&#xA;&lt;li&gt;Correct $u$ on level 2 and relax $u$ 3 more times. &lt;/li&gt;&#xA;&lt;li&gt;Check if current error is $&amp;gt;$ tolerance (assume yes)&lt;/li&gt;&#xA;&lt;li&gt;Relax $u$ again 3 times on level 2, calculate residual, then restrict.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Clearly at this stage at level 1 matrix $r$ i.e. $Ae=r$ has changed. So before we relax on level 1, do we set $e=0$ ? Or do we continue with the previous value of $e$ ? I think $e$ should be reset to 0 as the previous calculations were according to a different $r$ (restricted residual) matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.  &lt;/p&gt;&#xA;" OwnerUserId="14303" LastActivityDate="2015-06-12T15:47:55.810" Title="A doubt in Multigrid V-cycle" Tags="|pde|finite-difference|multigrid|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19915" PostTypeId="2" ParentId="19914" CreationDate="2015-06-12T15:47:55.810" Score="1" Body="&lt;p&gt;Set $e=0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The previous $e$ there has already been &quot;accounted for,&quot; as you have used it to correct your solution.&lt;/p&gt;&#xA;" OwnerUserId="3824" LastActivityDate="2015-06-12T15:47:55.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19916" PostTypeId="1" CreationDate="2015-06-12T16:11:41.523" Score="3" ViewCount="351" Body="&lt;p&gt;I am solving a linear stability problem using finite element discretization. Then, I have a generalised eigenvalue problem:&#xA;$$ \lambda M x = J x.$$&#xA;I obtain complex eigenvalue and eigenvectors from Arpack using inverse shift method. I would like to compare the eigenvectors obtained with different meshes. To do this it is necessary to normalize norm and phase in a proper way. As concerns the norm of the eigenvector I normalize it imposing: &#xA;$$ x^HMx = 1. $$&#xA;Now, I would need an integral norm for the phase too. I tried one but it doesn't seem to work. I cannot obtain similar eigenvectors with two different meshes.&lt;/p&gt;&#xA;" OwnerUserId="14489" LastEditorUserId="14489" LastEditDate="2015-06-13T09:02:46.577" LastActivityDate="2015-07-14T09:05:32.237" Title="How to correctly normalize modulus and phase of an eigenvector?" Tags="|finite-element|eigensystem|eigenvalues|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19917" PostTypeId="2" ParentId="19916" CreationDate="2015-06-12T17:20:51.007" Score="2" Body="&lt;p&gt;What do you mean by &quot;compare eigenvectors&quot;? If you just want to measure a concept of distance between them, I would use the &lt;a href=&quot;https://it.mathworks.com/help/matlab/ref/subspace.html&quot; rel=&quot;nofollow&quot;&gt;angle&lt;/a&gt; between subspaces, which is independent of the normalization.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2015-06-12T17:20:51.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19918" PostTypeId="1" AcceptedAnswerId="19963" CreationDate="2015-06-12T20:24:24.840" Score="5" ViewCount="258" Body="&lt;p&gt;Consider the approximation of a function by a truncated (finite) Fourier series, using complex notation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;f(x) \approx F_M(x) = \sum_{\alpha = -M}^{M} \hat{f}_{\alpha}e^{-i\alpha x} \quad (1)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\hat{f}_{\alpha}$ are the Fourier coefficients, which would be computed as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\hat{f}_{\alpha} = \frac{1}{2\pi}\int_{0}^{2\pi} f(x)e^{-i\alpha x} \;\textrm{d}x \quad (2)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider $(1)$ evaluated on a grid of $2M$ points ${x_j}$ where:&#xA;\begin{align}&#xA;x_j &amp;amp;= jh \\&#xA;h &amp;amp;= \frac{2\pi}{2M} = \frac{\pi}{M}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we would have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;F_M(x_j) = F_M(\frac{j\pi}{M}) = \sum_{\alpha = -M}^{M} \hat{f}_{\alpha}e^{-i\alpha j\pi/M}  \quad (3)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now things begin to become fuzzy, so this is where I would appreciate completions and corrections of my explanation.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some $j$, consider the summation in $(3)$ when $\alpha = M$ or $\alpha = -M$. In the summation, $e^{-i\alpha j\pi/M} = e^{\pm i j \pi} = (-1)^j$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Thus, on the grid, contributions from $\hat{f}_{M}$ and $\hat{f}_{-M}$ cannot be distinguished.&lt;/em&gt; &lt;strong&gt;Why?&lt;/strong&gt; In general, $\hat{f}_{M} \neq \hat{f}_{-M}$? So, they could always be distinguished?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In general, $\hat{f}_{\alpha}$ and $\hat{f}_{\alpha + 2M}$ cannot be distinguished on the grid.&lt;/em&gt; I am guessing the reason for this is similar to the reason for $\hat{f}_{M}$ and $\hat{f}_{-M}$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If $f(x)$ is a real function, then $\hat{f}_{\alpha}$ and $\hat{f}_{-\alpha}$ will be complex conjugates, so there is only $2M$ &quot;real information&quot; on the grid.&lt;/em&gt; &lt;strong&gt;This point I am very confused on.&lt;/strong&gt; Perhaps I should be writing &quot;there is only $M$ real information, rather than $2M$? Or am I interpreting the statement incorrectly?  &lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2015-06-18T16:13:13.123" Title="Complete (an incomplete) explanation of the phenomenon of &quot;aliasing&quot;, when using Fourier series to approximate functions?" Tags="|fourier-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19919" PostTypeId="1" AcceptedAnswerId="19920" CreationDate="2015-06-12T20:56:06.810" Score="2" ViewCount="121" Body="&lt;p&gt;I have tried it with &lt;code&gt;Lis&lt;/code&gt;, &lt;code&gt;Intel mkl&lt;/code&gt; and &lt;code&gt;PETSc&lt;/code&gt;. Everywhere you need to pass an  actual matrix to &lt;code&gt;GMRES&lt;/code&gt; or other similar iterative solver, while I want  to pass just function which multiplies my matrix on vector (my matrix is  having special structure and forming it completely is inefficient). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no problem for me writing simple solver which would accept my custom matrix-vector product myself, but I think this kind of problem  should be probably already figured out and implemented in many  libraries. &lt;/p&gt;&#xA;" OwnerUserId="16620" LastEditorUserId="9667" LastEditDate="2015-06-12T21:28:23.477" LastActivityDate="2015-06-13T20:33:35.770" Title="How do I make sparse solvers to accept custom matvec function insted of matrix?" Tags="|linear-algebra|linear-solver|matrix|libraries|mpi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19920" PostTypeId="2" ParentId="19919" CreationDate="2015-06-12T21:38:42.707" Score="4" Body="&lt;p&gt;PETSc does this. See &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MATSHELL.html&quot; rel=&quot;nofollow&quot;&gt;the documentation for &lt;code&gt;MatShell&lt;/code&gt;&lt;/a&gt; and Section 3.3 in &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf&quot; rel=&quot;nofollow&quot;&gt;the manual&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2015-06-13T20:33:35.770" LastActivityDate="2015-06-13T20:33:35.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19922" PostTypeId="2" ParentId="19886" CreationDate="2015-06-13T13:41:40.720" Score="3" Body="&lt;p&gt;Professor Mark Embree kindly answered to my emails. I report what he suggested to do:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;For large scale problems, this is what I generally recommend (note that this doesn't make special provisions for the energy norm):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use an iterative method (like &quot;eigs&quot; in MATLAB) to compute a d-dimensional dominant invariant subspace for inv(K)*M.  Suppose the columns of V span this subspace (generally, the columns of V will be eigenvectors associated with the largest eigenvalues of inv(K)*M, which hopefully correspond to the rightmost eigenvalues of the pencil (K,M)).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Let the columns of U form an orthonormal basis for V, e.g., in MATLAB, U = orth(V).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Define Ginv = inv(U'*inv(K)&lt;em&gt;M&lt;/em&gt;U).  This will be a d-by-d matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the pseudospectra of Ginv.  These pseudospectra will be contained within the pseudospectra of the full-size problem that would come if you took V to span the invariant subspace associated with all the finite eigenvalues of the pencil.  Experiment with enough values of the dimension &quot;d&quot; to make sure that the pseudospectra have converged in the rightmost part of the spectrum.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;[...] Generally I have K and M stored as sparse matrices, then use K\M to compute inv(K)*M.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case it seems to work. I hope it will be useful for someone else too.&lt;/p&gt;&#xA;" OwnerUserId="14489" LastActivityDate="2015-06-13T13:41:40.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19924" PostTypeId="1" AcceptedAnswerId="19925" CreationDate="2015-06-13T15:33:45.283" Score="1" ViewCount="682" Body="&lt;p&gt;I would like to write the most optimized program for array manipulation in Fortran (e.g. matrix multiplication of large matrices, squaring large matrices, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, using the fortran compilers on the market today (e.g. gfortran), how does F77 measure up against F90/95 (or F03)? Is there a particular compiler I should use to optimize speed with a certain version of Fortran? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, is there a difference in how I code these Fortran programs to optimize speed that is based distinctly in the version of Fortran used? Or is this simply changing the extension name (e.g. .f90 to .f70) and the program will run faster?  &lt;/p&gt;&#xA;" OwnerUserId="16628" LastActivityDate="2015-06-13T15:46:01.060" Title="How do Fotran77 and Fortran90/95 differ in performance via present-day compilers with array manipulation?" Tags="|fortran|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19925" PostTypeId="2" ParentId="19924" CreationDate="2015-06-13T15:46:01.060" Score="7" Body="&lt;p&gt;There is almost no good reason to write your own dense matrix manipulation routines until you have compared to fast libraries (MKL, FLAME, MAGMA, etc). Writing these libraries is challenging work that depends highly on the target architecture and requires deeper concepts than naive nested do loops. There's a whole literature on the subject that you should engage with, and you should note that basically none of the best-performing libraries in this space are written in Fortran (of any flavor) these days. Most of them are C/C++ plus targeted intrinsics or assembly code. Compilers just aren't magic enough to achieve the full performance of any chips.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-06-13T15:46:01.060" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19926" PostTypeId="2" ParentId="19916" CreationDate="2015-06-14T08:16:38.087" Score="0" Body="&lt;p&gt;Please refer to some work of Peter Arbenz: &#xA;&lt;a href=&quot;http://people.inf.ethz.ch/arbenz/publications.html&quot; rel=&quot;nofollow&quot;&gt;http://people.inf.ethz.ch/arbenz/publications.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He had worked out some generalized eigenvalue problems with constraint condition (I remembered that he used the Jacobi–Davidson Algorithm), his model problems came from the FEM for Maxwell equations.&lt;/p&gt;&#xA;" OwnerUserId="10290" LastActivityDate="2015-06-14T08:16:38.087" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19927" PostTypeId="1" CreationDate="2015-06-14T16:04:33.017" Score="1" ViewCount="457" Body="&lt;p&gt;Currently I am using the EIGEN C++ library to try to solve $x$ from the equation $Ax$ = $b$. One problem I encountered is that the matrix $A$ is a correlation matrix with size  &gt; 5000 and can sometimes be rank deficient and have small, even negative eigenvalues. As a result of that, the matrix tends to have huge condition numbers 100~1000+.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Special properties of the matrix $A$ include: symmetric (because it is a correlation matrix) and the correlation decreases as the two entries get further away (e.g. $R_{1,2}$ &gt; $R_{1,6}$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current way I used to solve the equation is to use the &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=FAQ#Is_there_a_method_to_compute_the_.28Moore-Penrose.29_pseudo_inverse_.3F&quot; rel=&quot;nofollow&quot;&gt;truncated SVD method&lt;/a&gt; to obtain a pseudoinverse of $A$. I also tried to exploit the fact that the matrix is symmetric by using the &lt;a href=&quot;http://eigen.tuxfamily.org/dox/classEigen_1_1SelfAdjointEigenSolver.html&quot; rel=&quot;nofollow&quot;&gt;self-adjoint eigen solver&lt;/a&gt; instead of SVD which speed up the process considerably. However, it still remains rather slow and from what I read, people always said one should &lt;strong&gt;NEVER EVER&lt;/strong&gt; try to solve the inverse whenever possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result of that, I wonder if there is anyway for one to speed up this process even further. I have read that one can use the QR method to obtain the pseudoinverse. My understanding is that for $A$ = $QR$, I can replace $R$ with $R_k$ the when the diagonal at $k$ is the last element bigger than a threshold and any subsequent row will be replaced by 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is: is this QR method correct? Or is there any method that can better solve this kind of equation?&lt;/p&gt;&#xA;" OwnerUserId="16636" LastEditorUserId="6706" LastEditDate="2015-06-14T17:59:26.147" LastActivityDate="2015-06-14T17:59:26.147" Title="How can I efficiently solve $Ax$=$b$ given $A$ is symmetric and contains very small (even negative) eigenvalues using EIGEN" Tags="|matrix|eigen|inverse-problem|" AnswerCount="0" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="19928" PostTypeId="1" CreationDate="2015-06-15T05:39:39.313" Score="0" ViewCount="2131" Body="&lt;p&gt;&lt;strong&gt;Issue&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hello everyone, I have got a program (from the net) that I intend to speed up by converting it into its parallel version thru the use of &lt;code&gt;pthreads&lt;/code&gt; surprisingly though, it runs &lt;strong&gt;slower&lt;/strong&gt; than the serial version. Below is the program:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# include &amp;lt;stdio.h&amp;gt;&#xA;&#xA;//fast square root algorithm&#xA;double asmSqrt(double x) &#xA;{&#xA;  __asm__ (&quot;fsqrt&quot; : &quot;+t&quot; (x));&#xA;  return x;&#xA;}&#xA;&#xA;//test if a number is prime&#xA;bool isPrime(int n)&#xA;{   &#xA;    if (n &amp;lt;= 1) return false;&#xA;    if (n == 2) return true;&#xA;    if (n%2 == 0) return false;&#xA;&#xA;    int sqrtn,i;&#xA;    sqrtn = asmSqrt(n);&#xA;&#xA;    for (i = 3; i &amp;lt;= sqrtn; i+=2) if (n%i == 0) return false;&#xA;    return true;&#xA;}&#xA;&#xA;//number generator iterated from 0 to n&#xA;int main()&#xA;{&#xA;    n = 1000000; //maximum number&#xA;    int k,j;&#xA;&#xA;    for (j = 0; j&amp;lt;= n; j++)&#xA;    {&#xA;        if(isPrime(j) == 1) k++;&#xA;        if(j == n) printf(&quot;Count: %d\n&quot;,k);&#xA;    }&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First attempt for parallelization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I let the &lt;code&gt;pthread&lt;/code&gt; manage the &lt;code&gt;for loop&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# include &amp;lt;stdio.h&amp;gt;&#xA;.&#xA;.&#xA;&#xA;int main()&#xA;{&#xA;    .&#xA;    .&#xA;    //-----&amp;gt;pthread code here&amp;lt;----&#xA;    for (j = 0; j&amp;lt;= n; j++)&#xA;    {&#xA;        if(isPrime(j) == 1) k++;&#xA;        if(j == n) printf(&quot;Count: %d\n&quot;,k);&#xA;    }&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Well, it runs slower than the serial one&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second attempt&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I divided the &lt;code&gt;for loop&lt;/code&gt; into &lt;strong&gt;two threads&lt;/strong&gt; and run them in parallel using &lt;code&gt;pthreads&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it still runs slower, I am intending that it may run about twice as fast or well faster. But its not!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: These is my parallel code by the way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# include &amp;lt;stdio.h&amp;gt;&#xA;# include &amp;lt;pthread.h&amp;gt;&#xA;# include &amp;lt;cmath&amp;gt;&#xA;&#xA;# define NTHREADS 2&#xA;&#xA;pthread_mutex_t mutex1 = PTHREAD_MUTEX_INITIALIZER;&#xA;int k = 0;&#xA;&#xA;double asmSqrt(double x) &#xA;{&#xA;  __asm__ (&quot;fsqrt&quot; : &quot;+t&quot; (x));&#xA;  return x;&#xA;}&#xA;&#xA;struct arg_struct&#xA;{&#xA;    int initialPrime;&#xA;    int nextPrime;&#xA;};&#xA;&#xA;bool isPrime(int n)&#xA;{   &#xA;    if (n &amp;lt;= 1) return false;&#xA;&#xA;    if (n == 2) return true;&#xA;&#xA;    if (n%2 == 0) return false;&#xA;&#xA;    int sqrtn,i;&#xA;    sqrtn = asmSqrt(n);&#xA;&#xA;    for (i = 3; i &amp;lt;= sqrtn; i+=2) if (n%i == 0) return false;&#xA;&#xA;    return true;&#xA;}&#xA;&#xA;void *parallel_launcher(void *arguments)&#xA;{&#xA;    struct arg_struct *args = (struct arg_struct *)arguments;&#xA;&#xA;    int j = args -&amp;gt; initialPrime;&#xA;    int n = args -&amp;gt; nextPrime - 1;&#xA;&#xA;    for (j = 0; j&amp;lt;= n; j++)&#xA;    {&#xA;        if(isPrime(j) == 1)&#xA;        {&#xA;            printf(&quot;This is prime: %d\n&quot;,j);&#xA;pthread_mutex_lock( &amp;amp;mutex1 );&#xA;            k++;&#xA;pthread_mutex_unlock( &amp;amp;mutex1 );&#xA;        }&#xA;&#xA;        if(j == n) printf(&quot;Count: %d\n&quot;,k);&#xA;    }&#xA;pthread_exit(NULL);&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    int f = 100000000;&#xA;    int m;&#xA;&#xA;    pthread_t thread_id[NTHREADS];&#xA;    struct arg_struct args;&#xA;&#xA;    int rem = (f+1)%NTHREADS;&#xA;    int n = floor((f+1)/NTHREADS);&#xA;&#xA;    for(int h = 0; h &amp;lt; NTHREADS; h++)&#xA;    {&#xA;        if(rem &amp;gt; 0)&#xA;        {&#xA;            m = n + 1;&#xA;            rem-= 1;&#xA;        }&#xA;        else if(rem == 0)&#xA;        {&#xA;            m = n;&#xA;        }&#xA;&#xA;        args.initialPrime = args.nextPrime;&#xA;        args.nextPrime = args.initialPrime + m;&#xA;&#xA;        pthread_create(&amp;amp;thread_id[h], NULL, &amp;amp;parallel_launcher, (void *)&amp;amp;args);&#xA;        pthread_join(thread_id[h], NULL);&#xA;    }&#xA;   // printf(&quot;Count: %d\n&quot;,k);&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;&#xA;OS: Fedora 21 x86_64,&#xA;Compiler: gcc-4.4,&#xA;Processor: Intel Core i5 (2 physical core, 4 logical),&#xA;Mem: 6 Gb,&#xA;HDD: 340 Gb,&lt;/p&gt;&#xA;" OwnerUserId="16639" LastEditorUserId="16639" LastEditDate="2015-06-15T06:14:07.363" LastActivityDate="2015-06-15T12:51:48.027" Title="Parallel slower than serial" Tags="|parallel-computing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19929" PostTypeId="1" CreationDate="2015-06-15T08:45:49.360" Score="-2" ViewCount="223" Body="&lt;p&gt;I have to solve numerically a differential equation in C. The equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/FGTQ9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I write some code to solve it? Are there some numerical methods (Runge-Kutta maybe?) to solve it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A colleague proposed to me to use the exact discretization method, but I don't think it's the smartest way to do, especially in programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="6703" LastActivityDate="2015-06-15T12:20:20.093" Title="Ways to solve numerically differential equations in C" Tags="|numerical-analysis|ode|c|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ClosedDate="2015-06-15T15:34:20.673" ContentLicense="CC BY-SA 3.0" />
  <row Id="19930" PostTypeId="2" ParentId="19929" CreationDate="2015-06-15T12:11:01.207" Score="1" Body="&lt;p&gt;I think that the easiest way is to use the Forward Euler method. You should take a look at &lt;a href=&quot;http://rosettacode.org/wiki/Euler_method#C.2B.2B&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; page in which the code is explained.&lt;/p&gt;&#xA;" OwnerUserId="16640" LastActivityDate="2015-06-15T12:11:01.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19931" PostTypeId="1" AcceptedAnswerId="19980" CreationDate="2015-06-15T12:15:24.473" Score="3" ViewCount="1115" Body="&lt;p&gt;i need to get a Givens-Rotation, which zeros a matrix entry when multiplied from the right side. I did already look at this Topic &lt;a href=&quot;https://math.stackexchange.com/questions/945736/givens-rotation-from-right-side&quot;&gt;https://math.stackexchange.com/questions/945736/givens-rotation-from-right-side&lt;/a&gt; but i could not really understand the process of getting there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more details: I have to matrices, A and B and i Need to get their Eigenvalues using the QZ-Algorithm. I get B to triagonal form using Givens-Rotations from left. These transformations are applied to A from the left side, too. After B is in triagonal form, i want to get A in triagonal form, too. Therefore i need Givens-Rotations from Right so that it does not destroy the zeros of Matrix B (Or is there another possibility to do this?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix}a&amp;amp;b\\c&amp;amp;d\end{bmatrix}\cdot\begin{bmatrix}e&amp;amp;f\\g&amp;amp;h\end{bmatrix}=\begin{bmatrix}ae+bg&amp;amp;af+bh\\ce+dg&amp;amp;cf+dh\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ce+dg=0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to find appropriate e,f,g,h ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody provide me some hints how to achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Im pretty sure it works like this:&#xA;$$\begin{bmatrix} a&amp;amp;b \end{bmatrix}\cdot\begin{bmatrix} c&amp;amp;-s\\ s&amp;amp;c \end{bmatrix}=\begin{bmatrix} 0&amp;amp;r \end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r = \sqrt{a^2+b^2}$$&#xA;$$c = \frac{b}{r}$$&#xA;$$s = \frac{-a}{r}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone approve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2:&#xA;Consider a simple 3x3-Matrix which i am trying to triangularize using These givens-rotation from the right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can Zero out elements (3,1) and (2,1) using the same rotation-Matrix $G$ with different Parameters $c$ and $s$. So i can create a Zero like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{bmatrix}*&amp;amp;*&amp;amp;*\\*&amp;amp;*&amp;amp;*\\a&amp;amp;b&amp;amp;*\end{bmatrix}\cdot\begin{bmatrix}c&amp;amp;-s&amp;amp;0\\s&amp;amp;c&amp;amp;0\\0&amp;amp;0&amp;amp;1\end{bmatrix}=\begin{bmatrix}*&amp;amp;*&amp;amp;*\\*&amp;amp;*&amp;amp;*\\0&amp;amp;*&amp;amp;*\end{bmatrix}$&#xA;BUT also like that&#xA;$\begin{bmatrix}*&amp;amp;*&amp;amp;*\\a&amp;amp;b&amp;amp;*\\*&amp;amp;*&amp;amp;*\end{bmatrix}\cdot\begin{bmatrix}c&amp;amp;-s&amp;amp;0\\s&amp;amp;c&amp;amp;0\\0&amp;amp;0&amp;amp;1\end{bmatrix}=\begin{bmatrix}*&amp;amp;*&amp;amp;*\\0&amp;amp;*&amp;amp;*\\*&amp;amp;*&amp;amp;*\end{bmatrix}$&#xA;for the same Rotation Matrix $G$. This makes it impossible for me to Zero out the elements iteratively, as the second Rotation will destroy the Zero created in the first one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is no Problem to Zero out all elements in the last row except the one in the right bottom edge (as each Zero is created by another Rotation Matrix $G$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how can i achieve to Zero out the elements mentioned in my example, when using the same Rotation Matrix $G$ which destroys already created zero-elements?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, any help is greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="16641" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-07-20T04:01:08.783" Title="Givens-Rotation from the Right Side (used in QZ-Algorithm)" Tags="|linear-algebra|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19932" PostTypeId="2" ParentId="19929" CreationDate="2015-06-15T12:20:20.093" Score="1" Body="&lt;p&gt;If you are willing to use C++, I suggest taking a look at the odeint package that&#xA;is part of the Boost library:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.boost.org/doc/libs/1_58_0/libs/numeric/odeint/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://www.boost.org/doc/libs/1_58_0/libs/numeric/odeint/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, writing a production quality ODE solver is not a simple task. odeint has been available in Boost for several years and is relatively mature and robust. Pre-built Boost libraries are available for many platforms which significantly simplifies installation.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-06-15T12:20:20.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19934" PostTypeId="2" ParentId="19928" CreationDate="2015-06-15T12:51:48.027" Score="3" Body="&lt;p&gt;Part of the problem is that there is one thread that does &lt;em&gt;all&lt;/em&gt; numbers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void *parallel_launcher(void *arguments)&#xA;{&#xA;  struct arg_struct *args = (struct arg_struct *)arguments;&#xA;&#xA;  int j = args -&amp;gt; initialPrime;&#xA;  int n = args -&amp;gt; nextPrime - 1;&#xA;&#xA;  for (j = 0; j&amp;lt;= n; j++)&#xA;  {&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note how you initialize &lt;code&gt;j&lt;/code&gt; correctly and then immediately set it to zero again. In other words, where every thread should do the interval &lt;code&gt;[n,m)&lt;/code&gt;, instead they do &lt;code&gt;[0,m)&lt;/code&gt;. The last one of your threads then does &lt;em&gt;all&lt;/em&gt; primes and will of course not be faster than the sequential program.&#xA;You should of course have seen this because the number of primes you print at the end is different for the sequential and the parallel version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other problem is that your code looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; for(int h = 0; h &amp;lt; NTHREADS; h++)&#xA;  {&#xA;    ...&#xA;    pthread_create(&amp;amp;thread_id[h], NULL, &amp;amp;parallel_launcher, (void *)&amp;amp;args);&#xA;    pthread_join(thread_id[h], NULL);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In other words, you create a thread and then immediately wait for it to finish before you start the next thread. That means that you never have two threads running in parallel -- no speedup can be expected from this.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-15T12:51:48.027" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19935" PostTypeId="1" CreationDate="2015-06-15T13:40:34.193" Score="1" ViewCount="195" Body="&lt;p&gt;I would be really glad to get some specific advise on how to implement a simple MCMC algorithm (in Matlab, if possible). I'm not yet too familiar with optimization methods. My problem goes as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $s(c_1,c_2)$ is a simulation which depends on the two constants $c_1$ and $c_2$, and $f(s(c_1,c_2))$ is the &quot;goodness function&quot; which determines the error of the simulation. This means I want $f$ small in the end. Furthermore, I have a range for both $c_1$ and $c_2$; all values in these ranges are assumed equally probable at the beginning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything more one would need to know? My question is now if anyone can provide a simple algorithm or pseudo-algorithm for this specific problem. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16642" LastActivityDate="2015-06-15T13:40:34.193" Title="Simple MCMC Algorithm in Matlab" Tags="|optimization|matlab|probability|inverse-problem|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19936" PostTypeId="1" CreationDate="2015-06-15T15:05:39.647" Score="7" ViewCount="3587" Body="&lt;p&gt;In thin plate bending problems, it would be very nice if we could model a thin rectangular plate with a mesh of many elements in the transverse directions, but only a single element in the longitudinal direction.  However, when try to model a plate this way and use piecewise linear polynomials, the resulting numerically obtained displacements are much smaller when compared to known analytical solutions.  This phenomenon is called &quot;mesh locking&quot;.  It can easily be resolved by either using higher order polynomials basis functions, mixed formulations, specialized plate/shell elements, or by utilizing more elements in the longitudinal direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that &quot;mesh locking&quot; happens in thin plate bending problems when you use low order elements and few elements in the longitudinal direction, but I'm still confused as to &lt;em&gt;why&lt;/em&gt; it happens.  What's the fundamental principle that prevents the low order basis functions from approximating the analytical solution?  How do specialized formulations/elements and/or more elements in the longitudinal direction remedy this problem?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2015-06-15T15:28:17.293" LastActivityDate="2016-04-17T10:54:58.983" Title="What exactly causes mesh locking in thin plate bending problems?" Tags="|finite-element|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19937" PostTypeId="2" ParentId="19902" CreationDate="2015-06-15T20:12:28.277" Score="8" Body="&lt;p&gt;The appropriate and fastest library depends on several things. Which Bessel functions (only J, Y &amp;amp; Hankel or modified Bessel functions I &amp;amp; K too), for which types of arguments (real or complex, integer, fractional or general order)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Amos's libraries are written in Fortran-77 (there are Fortran-90 coverted versions of TOMS 644 on a mirror of Alan Miller's fortran page: &lt;a href=&quot;http://jblevins.org/mirror/amiller/&quot; rel=&quot;nofollow&quot;&gt;http://jblevins.org/mirror/amiller/&lt;/a&gt;) and are numerically accurate and fast for complex argument of all Bessel functions, but they are difficult to understand the source code. He produced several versions of his library, but only the one included in SLATEC (&lt;a href=&quot;http://www.netlib.org/slatec/&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/slatec/&lt;/a&gt;) is actually open source, the other versions have a copyright associated with the transactions on mathematical software journal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://octave-bug-tracker.gnu.narkive.com/ym1U5WEL/bessel-function-scaling-limited-range&quot; rel=&quot;nofollow&quot;&gt;http://octave-bug-tracker.gnu.narkive.com/ym1U5WEL/bessel-function-scaling-limited-range&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mpmath (&lt;a href=&quot;http://mpmath.org/&quot; rel=&quot;nofollow&quot;&gt;http://mpmath.org/&lt;/a&gt;) uses a totally different and more general approach, by expressing all Bessel functions as special cases of hypergeometric functions. Mpmath is also arbitrary precision, so this is probably overkill if you just need double precision. This is more general, but slower.  The author of mpmath has a C-based library called arb (&lt;a href=&quot;http://fredrikj.net/arb/&quot; rel=&quot;nofollow&quot;&gt;http://fredrikj.net/arb/&lt;/a&gt;) that implements some of the functions in mpmath but much more quickly (since it is C-based rather than Python-based).  Since arb and mpmath are arbitrary precision, they might be good for benchmarking, but probably not for speed testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't personally worked much with the Gnu Scientific Library, but I believe it is fast and accurate for what functions it covers. It may not have all the functions you need. It would probably be a better example of well-written modern code, compared to the Amos libraries.&lt;/p&gt;&#xA;" OwnerUserId="7477" LastEditorUserId="7477" LastEditDate="2015-07-30T01:36:03.713" LastActivityDate="2015-07-30T01:36:03.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19938" PostTypeId="1" CreationDate="2015-06-15T20:50:09.950" Score="2" ViewCount="64" Body="&lt;p&gt;I'm trying to do an integral of the form $\int_C f(u,v) $, where $C$ is a set of contours in $u$ and $v$. In particular, each variable's contour starts at $-\infty+i \epsilon$, goes around a branch cut at $-2g$ (where $g$ is a parameter of the problem), then goes below the real line and out to $-\infty-i \epsilon$ before circling around to $\infty-i \epsilon$, coming back around a branch cut at $2g$ and ending up at $\infty+i \epsilon$. If this description is a bit unclear, there's a drawing of the contour in Figure 5 of &lt;a href=&quot;http://arxiv.org/pdf/1402.3307v1.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(u,v)$ has poles when $u=v-2i$ and when $v=u-2i$. While the paper suggests integrating above the real axis and replacing the integration below the real axis with analytic residues around these poles, that method isn't feasible for the case I'm working on. Instead, I'm trying to integrate along this contour numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I'm having is that when I try to approximate the (infinite) contour with a finite one, I inevitably end up with a contour that crosses the poles, since the poles in each variable depend on the other one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that there's a smarter way to approach all of this, but I can't find much literature on taking residues numerically at all, let alone anything applicable to this particular situation. Any recommendations would be appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: One thing that I should clarify, I need to have fixed contours in $u$ and $v$, neither can depend on the other. If I had a different $v$ contour for each value of $u$ then taking the $v$ residues analytically would be faster, and I'm trying to get a speedup over doing that.&lt;/p&gt;&#xA;" OwnerUserId="16648" LastEditorUserId="16648" LastEditDate="2015-06-17T13:47:14.950" LastActivityDate="2015-06-17T13:47:14.950" Title="Need a smart way to numerically take residues in a multidimensional integral" Tags="|numerical-analysis|complex-analysis|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="19939" PostTypeId="2" ParentId="19936" CreationDate="2015-06-15T22:12:15.240" Score="3" Body="&lt;p&gt;Let me try to expound a very simple argument. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 3D element with piecewise linear shape functions can exactly represent a constant strain solution. (Consider e.g. the patch test.) Loosely speaking this means that when the exact solution exhibits a strain gradient, mesh size has to be adapted so that there is not too much change of strain across one single element. This holds true also for hexahedral elements, where some linear terms arise in the strain expression: however linear hex element cannot represent exactly an arbitrary linear strain field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we admit that at the limit of very thin slabs the 3D solution approaches the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kirchhoff%E2%80%93Love_plate_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kirchhoff–Love plate&lt;/a&gt; solution, then we have a very peculiar situation: mild strain gradient in the in-plane directions $x_1, x_2$, but a very steep gradient in the transverse (thickness) direction: remember that strain (stress) changes sign across $x_3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xk4o0.png&quot; alt=&quot;Image by User:Bbanerje on commons.wikimedia.org&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now it should be clear that this strain/stress state cannot be sensibly approximated by a single linear FE in the $x_3$ direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other terms, for thin plates (Kirchhoff-Love) the most simple straining solution is constant curvature $\kappa$: no gradient in $x_1, x_2$, but a gradient equal to $\kappa$ along $x_3$ of the in plane longitudinal strains $\varepsilon_{11}$, $\varepsilon_{22}$ and vanishing transverse shear strain $\gamma_{13}=\gamma_{23}=0$. A coarse mesh should be able to represent the &quot;simple straining&quot; state, but this is not possible with linear elements: in fact while, as pointed out by Bill Green, it is possible to have $\varepsilon_{11}$ and $\varepsilon_{22}$ linear across $x_3$, it is impossible to meet at the same time the vanishing transverse shear strain condition. These spurious shear stains are sometimes called &quot;parasitic.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result the approximated elastic strain energy is wrong, and the finite element patch (under the assumption of complete integration) exhibits a stiffness which is bigger than expected. This point could be easily developed by computing the elastic strain energy of the exact solution, and comparing with the elastic strain energy of a single-element-across-the-thickness patch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way this is a common situation, with any FE approximation. Actually what makes the bending problem pathological is the fact that for a single linear element in the thickness direction $x_3$, the strain energy is dominated by the parasitic transverse shear deformation, so that the displacement components are heavily underestimated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remedy for this problem are &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;structural elements (plate/shell) in which Kirchhoff-Love or Reissner-Mindlin theory is incorporated as an assumed displacement field. (Gradient in $x_3$ direction is exactly modelled by this assumption.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;more linear element in the thickness (more than 5) or use of higher order elements in order to better catch the linear strain/stress gradient along $x_3$, without the introduction of parasitic shear strains.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;selective reduced integration, which amounts to integrating the transverse shear field with a single Gauss point at the centroid of the element (where the parasitic shear fields vanishes) while still integrating other terms with more gauss points.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Final notes&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Thanks to Bill Green who criticised my first (simplistic) answer and forced me to write a longer (and I hope also better) answer. It was not my intention to give a complete account of mesh locking phenomena, which are indeed well understood and described in the literature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm still convinced that, without going into the full details of shear locking, the starting point for explaining this phenomenon should be the observation that for pure bending the limiting solution for thin plates has very high strain gradients in the thickness directions, compared to the in plane ones. Linear elements are &quot;constantish&quot; in strain. Linear strains for hex elements arise from $x_1\cdot x_3$ (and similar mixed terms): the lack of squared terms (like $x_1^2$) in fact makes it impossible to properly decouple shear and longitudinal strains...&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2015-06-16T20:15:56.093" LastActivityDate="2015-06-16T20:15:56.093" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19940" PostTypeId="1" CreationDate="2015-06-16T01:57:51.007" Score="1" ViewCount="1377" Body="&lt;p&gt;I have this map showing the geography of Europe (below), and I wish to convert it to a matrix in python that would be a 2D approximation of this image where 0's would represent the ocean and 1's would represent landmasses. I plan to subsequently to do a biased random-walk on the resulting grid where walkers are  more likely to travel on land than water. Right now I just need help converting this image to 0's and 1's as mentioned, what would be the most straightforward way to do this in Python?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resolution of the resulting matrix representation of 0's and 1's does not need to astoundingly high, although I would like to distinguish between high and low elevations (presumably I would need to account for higher elevations by maybe noting them as entries of 2's in the matrix representation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/EnhWz.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11799" LastActivityDate="2015-06-16T02:49:32.907" Title="Convert Image of Map to 2D Grid in Python" Tags="|python|matrix|image-processing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19941" PostTypeId="2" ParentId="19940" CreationDate="2015-06-16T02:49:32.907" Score="1" Body="&lt;p&gt;The picture you have is a poor starting point because it shows rivers as blue even though they are typically well above sea level and are, therefore, most likely to be considered part of the &quot;land&quot; region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you'd be better off using the original data set from which maps such as these were generated. Data that has the information you're looking for can, for example, be obtained from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Shuttle_Radar_Topography_Mission&quot; rel=&quot;nofollow&quot;&gt;SRTM data set&lt;/a&gt; for which the wikipedia site also has links to the place where you can get the data set. SRTM data is already rastered and so should serve your purpose much better: every topography point with an elevation greater than zero will get a one in your notation, every topography point with an elevation of zero or below will get a zero. (Unless you take the few depressions such as the Dead Sea or Death Valley into account, which are part of land but lie below sea level.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-16T02:49:32.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19942" PostTypeId="2" ParentId="19741" CreationDate="2015-06-16T08:13:58.157" Score="1" Body="&lt;p&gt;I would go ahead and extend the &lt;code&gt;codedFixedValue&lt;/code&gt; boundary condition, that you have already available with some lines that access the mapped inlet patch directly from its code. Below is a stupid example, that &lt;em&gt;should&lt;/em&gt; illustrate that access. Please mind, that the assignment operator's argument is just made up by myself, which is sole purpose is to illustrate the working principle. The lines above are the important ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You first define the inlet patch name and get a const reference to the mesh. Afterwards a const reference to the velocity field is created, which is looked up from the object registry. The next step looks for the patch name and stores the corresponding patch ID as a lable, which is then reused by the following line to access the boundary field of the inlet patch.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myPatch&#xA;{&#xA;    type            codedFixedValue;&#xA;    value           uniform 0;&#xA;    redirectType    rampedFixedValue;   // name of generated BC&#xA;&#xA;    code&#xA;    #{&#xA;        const word coupledPatchName(&quot;INLET&quot;);&#xA;        const fvMesh&amp;amp; mesh(dimensionedInternalField().mesh());&#xA;        const volVectorField&amp;amp; U(db().lookupObject&amp;lt;volVectorField&amp;gt;(&quot;U&quot;);&#xA;&#xA;        const label coupledPatchID(mesh.boundaryMesh().findPatchID(coupledPatchName));&#xA;&#xA;        const vectorField&amp;amp; UbCoupled(U.boundaryField()[coupledPatchID]);&#xA;&#xA;        operator==(10*vector::one*max(mag(UbCoupled)));&#xA;    #};&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I hope this helps with your problem, otherwise please shout. &lt;/p&gt;&#xA;" OwnerUserId="1960" LastActivityDate="2015-06-16T08:13:58.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19943" PostTypeId="1" CreationDate="2015-06-16T10:40:08.477" Score="3" ViewCount="1028" Body="&lt;p&gt;I saw many examples of application of FFT for computation of convolutions and solving PDEs ( like Poisson equation ). It is very strightforward and efficient if I work with rectangular (orthogonal) lattice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't see how exatly modify the transformation if the lattice is sampled along non-orthogonal lattice vectors &lt;/p&gt;&#xA;&#xA;&lt;p&gt;eg. like this lattice:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tEWM4.gif&quot; alt=&quot;example of non-orthogobnal lattice &quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess, I can still use common algorithms or libraries ( like &lt;a href=&quot;http://www.fftw.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FFTW&lt;/a&gt;) because they care just about the n-dimensional array of numbers (they are agnostic to position of sampling points in real space ) ... but after that I have to transfrom the results by the lattice vectors somehow (or it's reciprocal vectors)( ? am I right? ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially I'm interested in three application:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;solving posisson equation &lt;/li&gt;&#xA;&lt;li&gt;covolution of two functions &lt;/li&gt;&#xA;&lt;li&gt;gradient ofthe function.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="4696" LastEditDate="2015-06-16T11:55:40.560" LastActivityDate="2021-04-22T23:36:09.853" Title="FFT on non-orthogonal lattice ( for computing convolutions and solving PDEs )" Tags="|fourier-transform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19944" PostTypeId="1" AcceptedAnswerId="19954" CreationDate="2015-06-16T10:49:32.807" Score="0" ViewCount="156" Body="&lt;p&gt;Consider the following differential equation&#xA;\begin{equation}&#xA;p(t) = \frac{\partial q(t)}{\partial t}&#xA;\end{equation}&#xA;where $t \in (0,\infty)$. I have a build a code that spits out values of the function $p$ for the variable $t$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'm looking for a scheme that will analyse these values of $p$ in order to determine $q$. Does anybody have any suggestions? The main problem is that if I would calculate $q(t) = \int p(t) \; \mathrm{d}t$, then I'll end up with just a number (i.e. the area under the graph), but I need to know the actual function of $q(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've cross-posted this question here: &lt;a href=&quot;https://math.stackexchange.com/questions/1326854/looking-for-a-particular-algorithm-for-numerical-integration#comment2695909_1326854&quot;&gt;https://math.stackexchange.com/questions/1326854/looking-for-a-particular-algorithm-for-numerical-integration#comment2695909_1326854&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16650" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-06-17T12:15:53.620" Title="Looking for a particular algorithm for numerical integration" Tags="|algorithms|time-integration|approximation|numerical-modelling|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19945" PostTypeId="1" AcceptedAnswerId="19947" CreationDate="2015-06-16T11:10:27.740" Score="3" ViewCount="470" Body="&lt;p&gt;I am trying to do a simple parallel sparse matrix vector multiplications using PETSC. My sparse matrix is a simple tridiagonal laplacian matrix, which is distributed over multiple processors using PETSC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is that if I do the same operation by simply iterating over the vector and updating each value&#xA;A[i] = -1*A[i-1] + 2*A[i] + -1*A[i+1], &#xA; it takes much lesser time than by using the PETSC SpMV. Why is it so? Or am I doing something wrong?&lt;/p&gt;&#xA;" OwnerDisplayName="user16651" LastActivityDate="2015-06-16T18:30:14.353" Title="Sparse matrix vector product using PETSC" Tags="|sparse-matrix|petsc|matrix|vector|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19946" PostTypeId="2" ParentId="19786" CreationDate="2015-06-16T11:39:44.290" Score="12" Body="&lt;p&gt;In some cases, (F)MG provides an algorithm with optimal properties. For instance, properly tuned FMG can solve some elliptic problems in a small number of &quot;work units&quot;, where a work unit is defined to be the computational effort required to express the problem itself - in this case the operations to form the residual $b-Ax$ on the finest grid. This is such an efficient (hence hard-to-beat) algorithm that it is the basis for an HPC benchmark designed to measure the maximum capacity of a supercomputer to solve realistic physics problems (&lt;a href=&quot;https://hpgmg.org&quot; rel=&quot;noreferrer&quot;&gt;HPGMG&lt;/a&gt;). If such a method is available, it is of course advisable to use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in many practical cases, an optimal or effective multigrid method is not used. This may be because &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;such a method is unknown or unavailable for the given problem&lt;/li&gt;&#xA;&lt;li&gt;smoothers and intergrid operators are not sufficient to give textbook convergence&lt;/li&gt;&#xA;&lt;li&gt;the coarse grid solver is inexact&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In these cases, the solution may be degraded by error which is not reduced as it should be by the multigrid cycle. If this error is contained in a low-dimensional subspace, however, a Krylov method can can solve for this error in a small number of iterations, and hence &quot;clean up&quot; after an imperfect multigrid solve. That is, if $BA$ has a few outlying eigenvalues, a Krylov method should be able to capture the corresponding eigenspaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the choice to use a suboptimal method might result in a much &quot;cheaper&quot; multigrid cycle, to the point that Krylov acceleration pays off. That is, there could be problems (and computing systems) where Krylov-accelerated MG can outperform MG. I would be interested in finding a concrete example of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Thanks to @chris above and &lt;a href=&quot;https://scicomp.stackexchange.com/users/21/matt-knepley&quot;&gt;Matt Knepley&lt;/a&gt; who mentioned some of the above in a tutorial)&lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-09-20T11:39:53.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19947" PostTypeId="2" ParentId="19945" CreationDate="2015-06-16T11:47:55.370" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;I assume that you are comparing multiplication with an &lt;em&gt;assembled&lt;/em&gt; PETSc matrix with your hand-coded &lt;em&gt;matrix-free&lt;/em&gt; method. The latter may indeed be faster, but this could be because no entries of &lt;code&gt;A&lt;/code&gt; need be loaded from memory. A more meaningful comparison might be to a matrix-free operator in PETSc (see Section 3.3 in the manual).&lt;/li&gt;&#xA;&lt;li&gt;Once you have an apples-to-apples comparison, you can try to determine if any remaining speed differences are due to PETSc overhead or not by using an optimized build (&lt;code&gt;configure --with-debugging=0&lt;/code&gt;) and running your code with the &lt;code&gt;-log_summary&lt;/code&gt; option to see timings of various operations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2015-06-16T18:30:14.353" LastActivityDate="2015-06-16T18:30:14.353" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19949" PostTypeId="2" ParentId="19943" CreationDate="2015-06-16T13:29:14.407" Score="5" Body="&lt;p&gt;That is not very difficult actually. Let &lt;span class=&quot;math-container&quot;&gt;$\Lambda\subset\Bbb R^n$&lt;/span&gt; be the lattice on which you sampled your points, and let &lt;span class=&quot;math-container&quot;&gt;$\Phi:\Bbb R^n\to\Bbb R^n$&lt;/span&gt; be a linear transformation under which &lt;span class=&quot;math-container&quot;&gt;$\Bbb Z^n\to\Lambda$&lt;/span&gt; isomorphically (i.e. the columns of the matrix of &lt;span class=&quot;math-container&quot;&gt;$\Phi$&lt;/span&gt; generate &lt;span class=&quot;math-container&quot;&gt;$\Lambda$&lt;/span&gt;). Let &lt;span class=&quot;math-container&quot;&gt;$f:\Bbb R^n\to\Bbb C$&lt;/span&gt; be the function you are interested in, and of which you are interested in its values in &lt;span class=&quot;math-container&quot;&gt;$\Lambda$&lt;/span&gt;. From now on, we will alternatingly identify &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with a function on &lt;span class=&quot;math-container&quot;&gt;$\Lambda$&lt;/span&gt; taking the value &lt;span class=&quot;math-container&quot;&gt;$f(\lambda)$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$\lambda\in\Lambda$&lt;/span&gt;, and with a function (or rather distribution)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{\lambda\in\Lambda}f(\lambda)\delta(\lambda - x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;on &lt;span class=&quot;math-container&quot;&gt;$\Bbb R^n$&lt;/span&gt;, i.e. zero outside &lt;span class=&quot;math-container&quot;&gt;$\Lambda$&lt;/span&gt;, and a delta function at lattice points.&lt;/p&gt;&#xA;&lt;p&gt;We can obtain the properties of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; in terms of its Fourier transform from the Fourier transform of &lt;span class=&quot;math-container&quot;&gt;$f\circ\Phi$&lt;/span&gt;, which is a function on &lt;span class=&quot;math-container&quot;&gt;$\Bbb Z^n$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathscr F(f\circ\Phi)(s) \equiv \int f\circ\Phi(x)e^{-2\pi i\langle s,x\rangle}dx = \frac1{|\det\Phi|}\int f(y)e^{-2\pi i\langle\Phi^{-T}s,y\rangle}dy = \frac1{|\det\Phi|}\mathscr F(f)(\Phi^{-T}s)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here a substitution &lt;span class=&quot;math-container&quot;&gt;$x = \Phi^{-1}y$&lt;/span&gt; was made, and it was used that &lt;span class=&quot;math-container&quot;&gt;$\langle s,\Phi^{-1}y\rangle = \langle\Phi^{-T}s,y\rangle$&lt;/span&gt;. Note that your conventions w.r.t. the sign of the exponent and factors of &lt;span class=&quot;math-container&quot;&gt;$\sqrt{2\pi}$&lt;/span&gt; may differ.&lt;/p&gt;&#xA;&lt;p&gt;Now what does the formula mean that we just wrote down? &lt;span class=&quot;math-container&quot;&gt;$f\circ\Phi$&lt;/span&gt; is just our set of samples of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; arranged in an &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional array, whose values have to be interpreted as lying on a standard grid. Its Fourier transform gives an array of the same shape, but the interpretation is different: the value at index &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; in reality is the value at &lt;span class=&quot;math-container&quot;&gt;$\Phi^{-T}s$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\Bbb R^n$&lt;/span&gt;, multiplied by &lt;span class=&quot;math-container&quot;&gt;$|\det\Phi|$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In particular, if the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is supported on &lt;span class=&quot;math-container&quot;&gt;$\Lambda\subset\Bbb R^n$&lt;/span&gt;, then its Fourier transform is supported on the image of &lt;span class=&quot;math-container&quot;&gt;$\Phi^{-T}$&lt;/span&gt;, the span of the columns of the inverse transposed matrix of &lt;span class=&quot;math-container&quot;&gt;$\Phi$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5521" LastEditorUserId="5521" LastEditDate="2021-04-22T23:36:09.853" LastActivityDate="2021-04-22T23:36:09.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19950" PostTypeId="1" CreationDate="2015-06-16T15:36:28.417" Score="1" ViewCount="437" Body="&lt;p&gt;Currently, I am working on an unconstrained energy minimization function, but I need to add some constraints. My system is a 2D lattice with a force applied to it, and I want the sides to be able to move, but the length of each side to be constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that using a Lagrange multiplier could make the terms I want to stay constant, constant by making the energy cost so huge that it won't move anything. But I was wondering if there was a different way to add constraints to the system without changing the fact that the energy minimization is unconstrained.&lt;/p&gt;&#xA;" OwnerUserId="16656" LastEditorUserId="9667" LastEditDate="2015-06-16T20:28:03.760" LastActivityDate="2015-06-17T00:19:55.087" Title="Minimization constraints without using Lagrange Multiplier" Tags="|constrained-optimization|constraints|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19951" PostTypeId="1" AcceptedAnswerId="19953" CreationDate="2015-06-16T16:32:43.370" Score="4" ViewCount="131" Body="&lt;p&gt;I don't have an high-end machine on which run my simulation (Python + C extensions, based on numpy for calculations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I intend to run the program on my PC, for several hours. I won't use the computer during the simulation, but many other processes run in background anyway &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will those other processes running on my Windows () computer harm the performance, generally speaking? Or is it a marginal factor?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(sorry if it's the wrong site, but I don't know whom to ask) &lt;/p&gt;&#xA;" OwnerUserId="8115" LastActivityDate="2015-06-16T19:28:15.877" Title="Can other running processes harm the performance of my simulation?" Tags="|software|simulation|performance|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19952" PostTypeId="2" ParentId="19951" CreationDate="2015-06-16T17:49:15.000" Score="5" Body="&lt;p&gt;If you just want the computational results and aren't running benchmarking tests then this isn't a serious problem.  If you're trying to benchmark the performance of the code and get repeatable run times for comparisons with a an alternate version of the code, then this can be an issue.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-06-16T17:49:15.000" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19953" PostTypeId="2" ParentId="19951" CreationDate="2015-06-16T19:28:15.877" Score="3" Body="&lt;p&gt;I am assuming that your computer is multi-core. In that case your operating system will probably delegate those background processes to one of the cores not being used by your simulation. If your simulation is parallel however (e.g. uses openMP) then background processes could potentially cause some slow down. In my experience minor background processes don't seem to cause serious slow down though. You could also do some tests by running with and without background processes and report back to let us know.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2015-06-16T19:28:15.877" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19954" PostTypeId="2" ParentId="19944" CreationDate="2015-06-16T19:51:05.530" Score="3" Body="&lt;p&gt;I will answer for the simplest case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dy}{dt} = -\lambda{y}, \hspace{4mm} y(0)=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: You need an initial condition which you did not specify in your original question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this simple problem $q(t) = y(t)$ and $p(t) = -\lambda{q(t)}$. If you use a basic forward Euler finite difference we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{y^{n+1}-y^{n}}{dt} = -\lambda{y^{n}}, \hspace{4mm} y^{0} = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which gives the recurrence relation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y^{n+1} = (1-dt\lambda{})y^{n}, \hspace{4mm} y^{0}=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You now just march forward using a specific dt to get y at time t. Obviously there is much more to the story. Here we have used the simplest time-stepping scheme (forward Euler). We have also used a very simple and specific example, but hopefully this will get you started. The basic procedure is to first approximate the derivative using a discrete formula (i.e. forward Euler) and then march forward in time starting from the initial condition. &lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2015-06-16T19:51:05.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19955" PostTypeId="2" ParentId="19950" CreationDate="2015-06-17T00:19:55.087" Score="4" Body="&lt;p&gt;You want to convert a constrained problem into an unconstrained problem. There are two general techniques for this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have a constraint of the form $g(x_1,...,x_N)=0$, then you may be able to write it as $x_N=f(x_1,...,x_{N-1})$ and thereby simply &lt;em&gt;eliminate&lt;/em&gt; one of the variables from your problem. You then don't need the constraint any more. (Of course, it may be that your constraint doesn't allow you to eliminate $x_N$ but another variable, with the same result.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can &lt;em&gt;augment&lt;/em&gt; your objective function by a penalty term that corresponds to your constraint. For equality constraints, this is most often done using a quadratic penalty, whereas for inequality constraints one most often uses a logarithmic penalty term. I would advise you to read through the book by Nocedal and Wright on &quot;Numerical Optimization&quot; to see how this works.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-17T00:19:55.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19956" PostTypeId="2" ParentId="19870" CreationDate="2015-06-17T01:39:52.380" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to minimize a black box function $f(x)$, which takes a 8×3 matrix of non-negative integers as input.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general, this sort of problem should be solved with a derivative-free MINLP (or if the function is linear, an MILP) solver. A very cursory glance at the literature suggests that the algorithm DFL, presented in a &lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs10957-014-0617-4&quot; rel=&quot;nofollow&quot;&gt;JOTA paper&lt;/a&gt; (also see &lt;a href=&quot;http://www.optimization-online.org/DB_HTML/2011/10/3193.html&quot; rel=&quot;nofollow&quot;&gt;preprint&lt;/a&gt;) could work; there might be other algorithms that also solve derivative-free MINLPs. The &lt;a href=&quot;http://www.dis.uniroma1.it/~lucidi/DFL/&quot; rel=&quot;nofollow&quot;&gt;authors' implementation&lt;/a&gt; is in Fortran 90, so you would need to write some wrappers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, though, you need some solver that can solve:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;black-box/derivative-free problems (I assume here that derivatives are not available, otherwise it would not be &quot;black box&quot;)&lt;/li&gt;&#xA;&lt;li&gt;that are also mixed-integer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since your problem contains no continuous decision variables, exhaustive sampling, as proposed by @hardmath, is another option that is probably easier to implement if you'd rather not write Python wrappers to a Fortran package (I wouldn't blame you).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, IPOPT would &lt;em&gt;not&lt;/em&gt; work, because it requires at least gradient information, and solves continuous problems, not mixed-integer problems. Even ignoring that estimating both gradient and Hessian information with finite-difference-type approaches is likely to yield poor outcomes due to numerical errors in these approximations, IPOPT will only solve the continuous relaxation of the problem. IPOPT (like any other continuous optimization solver) would have to be augmented with branch-and-cut or branch-and-bound-type methods, which is a great deal of work. If you're going to forge ahead and use a solver that assumes you have gradient information (even though you don't), you would be much better served using an MINLP solver (e.g., BARON, Couenne, Bonmin) that would at least give you integer-feasible solutions.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-06-17T01:39:52.380" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19958" PostTypeId="2" ParentId="19944" CreationDate="2015-06-17T12:15:53.620" Score="3" Body="&lt;p&gt;Your confusion mainly seems to be with calculus, not with numerical methods. Specifically, &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;make sure you understand the difference between an indefinite integral and a definite integral.&lt;/li&gt;&#xA;&lt;li&gt;consider what the initial conditions or boundary conditions are for your problem. Perhaps you know $q(t_0) = q_0$ for a given time $t_0$. &lt;/li&gt;&#xA;&lt;li&gt;Write the integral formulation again, explicitly involving the boundary conditions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3824" LastActivityDate="2015-06-17T12:15:53.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19961" PostTypeId="1" CreationDate="2015-06-17T18:01:50.357" Score="3" ViewCount="260" Body="&lt;p&gt;I have been looking for stability analysis of general reaction-diffusion problems, of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t}=\nabla\cdot D\nabla u-k\,u$ , &lt;/p&gt;&#xA;&#xA;&lt;p&gt;to be solved using the standard Finite Element in 2D with an explicit Euler time stepping. I haven't found any estimate or proof for the stability criteria for $\Delta t$ for this case, most of the literature seems to point to the solution of the Heat equation by Finite Differences in 1D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if someone can point me how to find the stability criterion in this case, especially for quadrilateral elements.&lt;/p&gt;&#xA;" OwnerUserId="16667" LastActivityDate="2015-06-17T18:01:50.357" Title="Stability analysis for explicit time discretization in the Finite Element Method" Tags="|finite-element|numerical-analysis|stability|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19963" PostTypeId="2" ParentId="19918" CreationDate="2015-06-18T01:13:34.637" Score="6" Body="&lt;p&gt;&lt;sub&gt;This turned out much longer than I planned so I hope it is useful and can be a resource for others with the same questions.&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I believe you mean to ask primarily about the discrete Fourier transform (in which case your definition (2) for $\hat{f}_\alpha$ is not right, for the DFT it should be $\hat{f}_\alpha = \sum_{j=0}^{2M-1} x_j \exp(- i\alpha\frac{\pi j}{M})$) so I will concentrate on that. I think it is instructive to consider this graphically. I'll try to break things down into a few steps and you should feel free to ask for clarification.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Phasors&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;A complex number, $z = r \cdot \operatorname{e}^{i\theta}$, can be drawn in the complex plane as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/LKemV.png&quot; alt=&quot;plot of r*exp(i*theta)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This type of plot is sometimes called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Phasor&quot; rel=&quot;nofollow noreferrer&quot;&gt;phasor diagram&lt;/a&gt; and is commonly used by electrical engineers and others who deal with complex valued signal processing applications. It can be a very useful way to visualize complex signals.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Phasor diagrams of complex exponentials&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Using phasors, the function $z_1 = \operatorname{e}^{ix}$ can be plotted on three dimensional axes as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xTaBP.png&quot; alt=&quot;3d plot of exp(i*x)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what happens if we plot at $z_2 = \operatorname{e}^{-ix}$? The spiral rotates in the opposite direction, but with the same frequency! This causes the real parts of $z_1$ and $z_2$ to remain in phase while the imaginary parts are 180 degrees out of phase. This is why there is a distinction between positive and negative frequencies. Unfortunately this is a bit hard to see with a 3D plot displayed on a 2D screen so I'll also plot the real and imaginary parts of these functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/BScki.png&quot; alt=&quot;exp(i*x) and exp(-i*x)&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/SGpcw.png&quot; alt=&quot;real part of exp(i*x) and exp(-i*x)&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/sj7vC.png&quot; alt=&quot;imaginary part of exp(i*x) and exp(-i*x)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple things to note:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Real valued functions have Fourier coefficients which satisfy $\hat{f}_{\alpha} = \operatorname{conj}\left(\hat{f}_{-\alpha}\right)$, this &lt;em&gt;must&lt;/em&gt; be the case so that the imaginary parts sum to zero.&lt;/li&gt;&#xA;&lt;li&gt;The complex exponentials are periodic and intersect at regular intervals.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h1&gt;Uniqueness of Fourier coefficients&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Since the complex exponentials intersect at regular intervals, one must take care when representing them on a grid. Making things more confusing, the example you have picked out highlights a somewhat special case, the Nyquist frequency. Let's take $M=10$ and plot the complex exponentials $z_1 = e^{iMx}$ and $z_2 = e^{-iMx}$ and also mark the grid points $x_j = \frac{\pi j}{M}$ for $j\in\{0..19\}$. Just as you noted, both functions are equal to $(-1)^j$ on this grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/vzVkK.png&quot; alt=&quot;plot at Nyquist frequency&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, these two functions are identical on the grid despite having different imaginary parts between grid points. &lt;strong&gt;If the only points you sample are on this grid, these functions cannot be distinguished.&lt;/strong&gt; In fact, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nyquist-Shannon Sampling Theorem&lt;/a&gt; states that it is &lt;em&gt;sufficient&lt;/em&gt; to sample at a rate at least twice the highest frequency present in a signal. Under some additional restrictions (uniform sampling, Fourier reconstruction) this condition is also &lt;em&gt;necessary&lt;/em&gt;. This means that since we have taken $2M$ samples, we should not be able to reconstruct (or uniquely determine) any signal components with wavelengths shorter than $2\pi/M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The frequency you have chosen to ask about has wavelength exactly equal to $2\pi/M$ and is called the Nyquist frequency. This is right on the boundary of meeting the Nyquist-Shannon sampling criteria and leads to much confusion. Notice that we have chosen $M=10$, which leads to 20 sampling points (the point at $x=2\pi$ is the same as $x=0$ due to periodicity and is therefore not included) and thus we expect 20 Fourier coefficients (including zero). Whether we have $\alpha \in\{-10..9\}$ or $\alpha \in\{-9..10\}$ or some combination of $\alpha=-10$ and $\alpha=10$ is ambiguous. &lt;strong&gt;For even numbers of sample points this is always the case, we can tell that the $M$ frequency component is present, but we cannot tell if it is positive or negative or some combination of the two&lt;/strong&gt;. For odd numbers of sample points this ambiguity does not exist, adding another sample point would push us past the sampling limit and give us $\alpha \in\{-10..10\}$ and the ability to distinguish these two frequency components.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Can you discern $\hat{f}_{\alpha}$ from $\hat{f}_{\alpha + 2M}$?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;This question again comes back to the Shannon Sampling Theorem. $\alpha + 2M$ (for $\alpha&amp;lt;M$) is always greater than your sampling rate and therefore exceeds tthe frequency you can distinguish at that sampling rate. Furthermore, the algebra works out that&#xA;\begin{align}&#xA;\exp\left(i~(\alpha+2M)\dfrac{j\pi}{M}\right)&#xA; &amp;amp; = \exp\left(i~\alpha\dfrac{j\pi}{M}\right)\exp\left(i~2M\dfrac{j\pi}{M}\right)\\&#xA; &amp;amp; = \exp\left(i~\alpha\dfrac{j\pi}{M}\right)\exp\left(i~2\pi j\right)\\&#xA; &amp;amp; = \exp\left(i~\alpha\dfrac{j\pi}{M}\right)\exp\left(i~0\right)\\&#xA; &amp;amp; = \exp\left(i~\alpha\dfrac{j\pi}{M}\right)&#xA;\end{align}&#xA;so the two functions are always equal on your sampling grid.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Complex versus real discrete Fourier transforms&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;You are correct that weights for the positive and negative frequency components are complex conjugates in the case of real-valued functions. As I mentioned, this must be the case so that the imaginary parts cancel out and give you a real-valued function. There are two ways to think of this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You have only half as much information in the function values since there is no imaginary part ($2M$ &quot;pieces&quot; of real-valued information), and that leads to half as much information in the Fourier coefficients ($M$ complex values that each hold a real and imaginary part for $2M$ pieces of information). If you know the positive frequency coefficients you know the negative frequency coefficients are just their conjugates (or vice versa).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You have the same amount of information in the function values since knowing that they are all real &lt;em&gt;is&lt;/em&gt; information. Knowing that the $2M$ imaginary parts are zero is just as much information as knowing that they had some other value. If even one value was complex it would change all your frequency coefficients. That leads to the same amount of information in the frequency components since knowing that the are all conjugates &lt;em&gt;is&lt;/em&gt; information (2 pieces for each: real parts equal and imaginary parts opposite).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Both of these views are equivalent and lead to the same conclusions. Personally, I prefer to think of real-valued samples as $2M$ pieces of information which leads to $M$ complex-valued coefficients, each consisting of a magnitude and phase, giving the same amount of information.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-06-18T16:13:13.123" LastActivityDate="2015-06-18T16:13:13.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19964" PostTypeId="1" AcceptedAnswerId="19965" CreationDate="2015-06-18T02:49:02.543" Score="1" ViewCount="366" Body="&lt;p&gt;Given a linear BIP&#xA;$$\text{Minimize}\;\;\;c^Tx$$&#xA;$$\hspace{6.5mm}\text{Subject to}\;\;\;Ax\leq b$$&#xA;$$\hspace{38mm}x\in\{0,1\}^n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can in theory convert the constraints to the convex hull constraint $Hx\leq d$ which allows us to perform a linear relaxation and find the solution to the linear BIP that way.  Is there an efficient way to find this convex hull and put it in the aforementioned form?&lt;/p&gt;&#xA;" OwnerUserId="6849" LastActivityDate="2015-06-19T21:46:43.160" Title="Converting linear BIP constraints into convex hull" Tags="|optimization|convex-optimization|linear-programming|mixed-integer-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19965" PostTypeId="2" ParentId="19964" CreationDate="2015-06-18T03:22:03.523" Score="2" Body="&lt;p&gt;I don't think such a method exists. 3-SAT is polynomial-time reducible to integer programming.. If you could find the integer hull (i.e., convex hull of the integer feasible set) in polynomial time in the general case, you could solve the LP efficiently using a polynomial time algorithms (for instance, an interior point method) and obtain an optimal solution to the ILP, which in turn would give you a polynomial time solution to 3-SAT. We know that 3-SAT is in NP, so unless P = NP (which is unlikely), no such efficient algorithm exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only algorithm I'm vaguely aware of involves solving an exponential number of auxiliary LPs to determine the extreme points of the set. The exponential number arises from noting that an $n$-cube has $2^{n}$ vertices.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-06-19T21:46:43.160" LastActivityDate="2015-06-19T21:46:43.160" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19966" PostTypeId="1" CreationDate="2015-06-18T10:27:12.707" Score="1" ViewCount="127" Body="&lt;p&gt;Originally posted on &lt;a href=&quot;https://stats.stackexchange.com/questions/157483/back-prop-question-can-this-gradient-be-decomposed&quot;&gt;stats.stackexchange&lt;/a&gt;, I'll pair the post down to something a bit more general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have vectors $\{\mathbf{\delta}, \mathbf{x}_1, \ldots, \mathbf{x}_J\}$, where $\delta \in \mathbb{R}^{J}$ and $\mathbf{x}_i \in \mathbb{R}^{I}$. Furthermore, let $\mathbf{A} \in \mathbb{R}^{I\times J}$ be such that&#xA;$$&#xA;\mathbf{A} =&#xA;\begin{pmatrix} &#xA;\delta_1 \mathbf{x}_1 \\&#xA;\delta_2 \mathbf{x}_2 \\&#xA;\vdots \\&#xA;\delta_J \mathbf{x}_J&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any sequence of BLAS-optimized operations that can make $\mathbf{A}$ from my set of vectors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be concrete, I understand that if I have the matrix&#xA;$$&#xA;\mathbf{B} =&#xA;\begin{pmatrix} &#xA; \mathbf{x}_1 \\&#xA; \mathbf{x}_2 \\&#xA;\vdots \\&#xA; \mathbf{x}_J&#xA;\end{pmatrix}&#xA;$$&#xA;then&#xA;$$&#xA;\text{diag}(\delta) \,  \mathbf{B} :=&#xA;\begin{pmatrix}&#xA;\delta_1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; \delta_2 &amp;amp; \cdots &amp;amp; 0 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \delta_j&#xA;\end{pmatrix}&#xA;\mathbf{B}&#xA;= \mathbf{A} &#xA;$$&#xA;But it's not clear to me if making $\text{diag}(\delta)$ is an efficient operation, or even if the multiplication between $\text{diag}(\delta)$ and $\mathbf{B}$ is efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit:&lt;/em&gt; I noticed that a similar, fortran, specific question was &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10667/what-is-the-best-way-to-multiply-a-diagonal-matrix-in-fortran&quot;&gt;asked and answered&lt;/a&gt;. However that question addresses efficient ways to multiply diagonal matrices (in fortran, no less). Whereas, my question references the use of diagonal matrices as &lt;em&gt;one&lt;/em&gt; way in which the solution can be approached. My thoughts are that this question is a bit more general than the aforementioned one.&lt;/p&gt;&#xA;" OwnerUserId="16675" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-06-19T06:22:06.707" Title="How to make this matrix efficiently?" Tags="|linear-algebra|blas|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19968" PostTypeId="2" ParentId="10045" CreationDate="2015-06-18T13:06:18.830" Score="3" Body="&lt;p&gt;This &lt;a href=&quot;http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf&quot; rel=&quot;nofollow&quot;&gt;tutorial&lt;/a&gt; talks about recalculating the residual every 50 iterations to mitigate round-off errors.&lt;/p&gt;&#xA;" OwnerUserId="14478" LastActivityDate="2015-06-18T13:06:18.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19969" PostTypeId="1" AcceptedAnswerId="19971" CreationDate="2015-06-18T13:52:32.943" Score="6" ViewCount="11070" Body="&lt;p&gt;I am doing a molecular dynamics simulation. I need to assign initial velocities to the atoms. I want to assign the initial velocities which follow the Maxwell-Boltzmann distribution. How do I calculate such initial velocities using a uniform random number generator with range [0,1)?&lt;/p&gt;&#xA;" OwnerUserId="16433" LastEditorUserId="20688" LastEditDate="2019-05-26T05:45:17.667" LastActivityDate="2019-05-26T05:45:17.667" Title="How do I generate Maxwell-Boltzmann variates using a uniform distribution random number generator?" Tags="|computational-physics|molecular-dynamics|random-number-generation|probability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19970" PostTypeId="1" AcceptedAnswerId="19975" CreationDate="2015-06-18T13:54:42.913" Score="3" ViewCount="270" Body="&lt;p&gt;UPDATE: The problem turned out to be the step (refer penultimate paragraph below) where I was factoring out a small value from the vectors of the numerator and denominator and then computed dot products/norm-square. Computing these terms in straightforward manner resulted in CG steps converging with almost same number of steps as Eigen Solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am implementing Steihaug method in C++, for large scale unconstrained convex optimization problems. The current instance, involves around 33000 variables; ideally, the problem sizes will be half-a-million to a million variables. The Hessian matrix is symmetric and quite sparse (~30% non-zero entries) and has a block structure about the main diagonal. The Hessian is PSD (never strictly PD) and has a huge (~$10^8$) condition number.  I am using a diagonal preconditioner. The scientific problem is about applying log-sum-exp based smoothing to LP relaxation of combinatorial optimization problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the method approaches the optimum it takes increasingly more internal CG iterations per outer iteration: ~7700 CG iterations in the last outer iteration which reaches the global optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I plug in the data of the last outer iteration to Eigen::BiCGSTAB solver, with diagonal preconditioner, it converges in 100 iterations to the desired optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate some pointers on making the implementation numerically robust.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: I suppose an important issue is that all non-zero numbers in the gradient and hessian are quite small in magnitude. Since, we are nearing the global optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the calculation of $\alpha$ (steepest descent step) and $\beta$ (coefficient in direction update), I am factoring out the smallest non-zero magnitude (and cancel it between the numerator and denominator) and computing the dot products. This is not helping. (UPDATE: this turned out to be an unnecessary step that caused the error in the implementation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am computing the residual (as $Ax - b$) every fifty iterations to compensate for drift.&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2016-05-27T18:35:04.773" LastActivityDate="2016-05-27T18:35:04.773" Title="Trust-region Newton: implementation issue with Conjugate Gradient calculations" Tags="|linear-algebra|optimization|numerical-analysis|conjugate-gradient|krylov-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19971" PostTypeId="2" ParentId="19969" CreationDate="2015-06-18T14:21:40.907" Score="6" Body="&lt;p&gt;The initial velocities are drawn from a Gaussian distribution with variance&#xA;$$\sigma_i^2=\frac{k_{\textrm{B}}T}{m_i},$$&#xA;where $k_{\textrm{B}}$ denotes Boltzmann's constant, $T$ is the temperature and $m_i$ is the mass of the $i^{\textrm{th}}$ particle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the problem boils down to generate random numbers from a gaussian distribution using uniformly distributed random numbers. This is fortunately quite simple: the Wikipedia article https://en.wikipedia.org/wiki/Box–Muller_transform shows some very common algorithms how to transform uniform random numbers into gaussian random numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's put everything together: every component of the velocity of the $i^{\textrm{th}}$ particle is computed via&#xA;$$v_{i,\alpha}=\sqrt{\frac{k_{\textrm{B}}T}{m_i}}\,\mathcal{N}(0,1)\,,\quad\alpha\in\{x,y,z\}\,,$$&#xA;where $\mathcal{N}(0,1)$ is a gaussian random number with variance 1 and mean 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this definition, each velocity component follows a gaussian distribution&#xA;$$\pi(v_\alpha)\textrm{d}v_\alpha\propto\exp\left(-\frac{v_\alpha^2}{2\sigma^2}\right)\textrm{d}v_\alpha\,,$$&#xA;but when you write the distribution of the velocity vector in spherical coordinates and integrate the angular components, you obtain&#xA;$$\pi(v)\textrm{d}v \propto v^2\exp\left(-\frac{v^2}{2\sigma^2}\right)\textrm{d}v\,,$$&#xA;which is the desired Maxwell-Boltzmann distribution.&lt;/p&gt;&#xA;" OwnerUserId="16655" LastEditorUserId="16655" LastEditDate="2015-06-18T16:18:08.027" LastActivityDate="2015-06-18T16:18:08.027" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19972" PostTypeId="1" AcceptedAnswerId="19973" CreationDate="2015-06-18T16:00:05.757" Score="4" ViewCount="81" Body="&lt;p&gt;A canonical form of primal linear program is&#xA;$$&#xA;    \text{minimize } c^T \cdot x \\&#xA;    \text{subject to } Ax = b, x \geq 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dual is&#xA;$$&#xA;    \text{maximize } b^T \cdot y \\&#xA;    \text{subject to } z \equiv c - A^T y \geq 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know $y$ are called Lagrange multipliers. How do you refer to the other terms, &lt;em&gt;i.e.&lt;/em&gt; $A,b,c,x$ and $z$ in LP (and SDP)?&lt;/p&gt;&#xA;" OwnerUserId="15128" LastActivityDate="2015-06-18T16:22:18.937" Title="LP and SDP nomenclature" Tags="|convex-optimization|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19973" PostTypeId="2" ParentId="19972" CreationDate="2015-06-18T16:22:18.937" Score="4" Body="&lt;p&gt;$x$ is primal variable, $y$ is dual variable ($y$ is usually not referred to as Lagrange multiplier unless you form Lagrangian explicitly). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Others are usually referred by symbol directly rather than name, the linear coefficient ($c$), linear constraint matrix ($A$), right-hand-side of linear constraint ($b$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have quadratic term $x^\top Hx$, then $H$ is the quadratic matrix or the Hessian&lt;/p&gt;&#xA;" OwnerUserId="14305" LastActivityDate="2015-06-18T16:22:18.937" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19974" PostTypeId="1" AcceptedAnswerId="19978" CreationDate="2015-06-18T22:17:02.183" Score="1" ViewCount="2101" Body="&lt;p&gt;I have multiple CSV files with point-coordinates and, possibly, multiple data values attached to them, i.e., the rows look like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x y z data1 data2 ... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any possibility to display say 10 of those CSV files in a &quot;time&quot; series in Paraview? &lt;/p&gt;&#xA;" OwnerUserId="5337" LastEditorUserId="9667" LastEditDate="2015-06-19T14:21:36.247" LastActivityDate="2015-06-19T18:15:59.180" Title="Paraview: Multiple csv files to time series" Tags="|visualization|paraview|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19975" PostTypeId="2" ParentId="19970" CreationDate="2015-06-19T05:15:54.233" Score="1" Body="&lt;p&gt;33000 decision variables is still within the realm of direct solvers, so for at least that problem instance, you could try a sparse LU or sparse Cholesky factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For larger problem instances, where direct solvers are not an option, you could try an incomplete Cholesky conditioner. Separate papers by Gondzio and by Waechter, Curtis, et al use left-ILU-preconditioned GMRES to solve the KKT system in their interior point methods. I would think that you could try something similar with CG.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-06-19T05:15:54.233" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="19976" PostTypeId="2" ParentId="19966" CreationDate="2015-06-19T06:22:06.707" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Suppose I have vectors $\{\mathbf{\delta}, \mathbf{x}_1, \ldots, \mathbf{x}_J\}$, where $\delta \in \mathbb{R}^{J}$ and $\mathbf{x}_i \in \mathbb{R}^{I}$. Furthermore, let $\mathbf{A} \in \mathbb{R}^{I\times J}$ be such that&#xA;  $$&#xA;\mathbf{A} =&#xA;\begin{pmatrix} &#xA;\delta_1 \mathbf{x}_1 \\&#xA;\delta_2 \mathbf{x}_2 \\&#xA;\vdots \\&#xA;\delta_J \mathbf{x}_J&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There's some confusion about notation here. As written, $\mathbf{A} \in \mathbb{R}^{IJ}$; that is, is has one column and $IJ$ rows, instead of $I$ rows and $J$ columns.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any sequence of BLAS-optimized operations that can make $\mathbf{A}$ from my set of vectors?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What I think you're proposing essentially reduces to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\begin{pmatrix}&#xA;\mathbf{x}_{1} &amp;amp; \ldots &amp;amp; \mathbf{x}_{J}&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;\delta_1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; \delta_2 &amp;amp; \cdots &amp;amp; 0 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \delta_j&#xA;\end{pmatrix},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;followed by reshaping the result. I would not do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, &lt;code&gt;dscal&lt;/code&gt; each of the $\mathbf{x}_{i}$ by their respective $\delta_{i}$ in-place. As DougLipinski points out, this is basically just a &lt;code&gt;for&lt;/code&gt; loop (or &lt;code&gt;do&lt;/code&gt; loop in Fortran). A tuned BLAS implementation might do some loop unrolling.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To be concrete, I understand that if I have the matrix&#xA;  $$&#xA;\mathbf{B} =&#xA;\begin{pmatrix} &#xA; \mathbf{x}_1 \\&#xA; \mathbf{x}_2 \\&#xA;\vdots \\&#xA; \mathbf{x}_J&#xA;\end{pmatrix}&#xA;$$&#xA;  then&#xA;  $$&#xA;\text{diag}(\delta) \,  \mathbf{B} :=&#xA;\begin{pmatrix}&#xA;\delta_1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;0 &amp;amp; \delta_2 &amp;amp; \cdots &amp;amp; 0 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \delta_j&#xA;\end{pmatrix}&#xA;\mathbf{B}&#xA;= \mathbf{A} &#xA;$$&#xA;  But it's not clear to me if making $\text{diag}(\delta)$ is an efficient operation, or even if the multiplication between $\text{diag}(\delta)$ and $\mathbf{B}$ is efficient.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The memory movement isn't worth it; also, the multiplication you've proposed isn't conformal, so it would not work. Likely, you intended to do what I show above.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-06-19T06:22:06.707" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19977" PostTypeId="2" ParentId="19895" CreationDate="2015-06-19T10:03:43.417" Score="1" Body="&lt;p&gt;The task in this problem is to apply the variable transformation&#xA;$$z=\frac{x}{c+x} \Leftrightarrow x=\frac{zc}{1-z}\,,\quad \textrm{d}x=\frac{c}{(1-z)^2}\textrm{d}z$$&#xA;to the integral&#xA;$$\int\limits_0^\infty f(x)\textrm{d}x\,,\quad f(x)=x^{a-1}e^{-x}\,,$$&#xA;which yields&#xA;$$\int\limits_0^1 \tilde{f}(z)\textrm{d}z\,,\quad \tilde{f}(z)=\left(\frac{zc}{1-z}\right)^{a-1}\exp\left(-\frac{zc}{1-z}\right)\frac{c}{(1-z)^2}\,,$$&#xA;and chose the parameter $c$ such that $\tilde{f}(z)$ takes its maximum at $z=\frac{1}{2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have already completed the first part of the question: $z$ takes the value $\frac{1}{2}$ when $x=c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part asks which is the correct $c$. We tackle this by computing the derivative of $\tilde{f}(z=\frac{1}{2})$ with respect to the unknown parameter $c$ and set this to zero:&#xA;$$\left.\frac{\partial \tilde{f}}{\partial c}\right\vert_{z=\frac{1}{2}} = \frac{1}{4}c^{a-1}e^{-c}\left(a-c\right)=0\,.$$&#xA;Thus, we find $c=a$.&lt;/p&gt;&#xA;" OwnerUserId="16655" LastActivityDate="2015-06-19T10:03:43.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19978" PostTypeId="2" ParentId="19974" CreationDate="2015-06-19T18:15:59.180" Score="1" Body="&lt;p&gt;If you managed to of load a single file and plot what you want from it you can  try to rename the files. In Paraview, when you have multiple files with the same base name but consecutive enumeration you can plot them over time.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-06-19T18:15:59.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19979" PostTypeId="1" CreationDate="2015-06-19T23:43:34.620" Score="3" ViewCount="164" Body="&lt;p&gt;I just implemented an Algebraic Multigrid solver for a Mixed Dirichlet-Neumann Boundary Value problem and was surprised to see the speed-up as compared to a simple iterative solver for a large problem like 2048x2048 and 4096x4096. My next step is to implement it in parallel. If Multigrid is so useful, I am sure people would like to (and have I think) implemented it for millions of processors. My question is : &lt;strong&gt;What are the challenges/bottlenecks in implementing Algebraic Multigrid on a large scale&lt;/strong&gt; ? It will be extremely interesting to gain an insight into this to be able to produce a moderately optimised code at this stage. &lt;/p&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="14303" LastEditDate="2015-06-20T02:32:58.557" LastActivityDate="2015-06-28T01:07:24.740" Title="Challenges in implementing Algebraic Multigrid on millions of processors" Tags="|pde|parallel-computing|multigrid|exascale|multicore|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19980" PostTypeId="2" ParentId="19931" CreationDate="2015-06-20T23:00:02.400" Score="4" Body="&lt;p&gt;You are correct. With the choice&#xA;$$\begin{eqnarray*}r&amp;amp;=&amp;amp;\sqrt{a^2+b^2}\\[.6em]c&amp;amp;=&amp;amp;\frac{b}{r}\\[.6em]s&amp;amp;=&amp;amp;\frac{-a}{r}\end{eqnarray*}$$&#xA;a Givens rotation from the right yields&#xA;$$\begin{pmatrix}a&amp;amp;b\end{pmatrix} \begin{pmatrix}c&amp;amp;-s\\s&amp;amp;c\end{pmatrix} = \begin{pmatrix}ac+bs&amp;amp;-as+bc\end{pmatrix} = \begin{pmatrix}0&amp;amp;r\end{pmatrix}\,,$$&#xA;because&#xA;$$ac+bs=a\frac{b}{r}+b\frac{-a}{r}=\frac{ab}{r}-\frac{ab}{r}=0$$&#xA;and&#xA;$$-as+bc=-a\frac{-a}{r}+b\frac{b}{r}=\frac{a^2+b^2}{r}=\frac{r^2}{r}=r\,.$$&#xA;Just take care that $r\neq0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Concerning EDIT2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can only zero out both elements $(3,1)$ and $(2,1)$ in your example with the same Givens rotation matrix if the matrix looks like this:&#xA;$$\begin{pmatrix}*&amp;amp;*&amp;amp;*\\a&amp;amp;b&amp;amp;*\\a&amp;amp;b&amp;amp;*\end{pmatrix}$$&#xA;And in that case, multiplying your Givens rotation matrix from the right yields&#xA;$$\begin{pmatrix}*&amp;amp;*&amp;amp;*\\a&amp;amp;b&amp;amp;*\\a&amp;amp;b&amp;amp;*\end{pmatrix} \begin{pmatrix}c&amp;amp;-s&amp;amp;0\\s&amp;amp;c&amp;amp;0\\0&amp;amp;0&amp;amp;1\end{pmatrix} = \begin{pmatrix}*&amp;amp;*&amp;amp;*\\0&amp;amp;r&amp;amp;*\\0&amp;amp;r&amp;amp;*\end{pmatrix}\,,$$&#xA;and you have brought both elements to zero simultaneously. If the rows 2 and 3 are not both $\begin{pmatrix}a&amp;amp;b&amp;amp;*\end{pmatrix}$, you need different rotations, i.e., different $c$, $s$, and $r$ values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you do in practice to bring a matrix on upper triangle shape, is to start with the lowest row and bring all elements but the last to zero:&#xA;$$\begin{pmatrix}*&amp;amp;*&amp;amp;*\\*&amp;amp;*&amp;amp;*\\a_1&amp;amp;b_1&amp;amp;*\end{pmatrix} \begin{pmatrix}c_1&amp;amp;-s_1&amp;amp;0\\s_1&amp;amp;c_1&amp;amp;0\\0&amp;amp;0&amp;amp;1\end{pmatrix} = \begin{pmatrix}*&amp;amp;*&amp;amp;*\\*&amp;amp;*&amp;amp;*\\0&amp;amp;r_1&amp;amp;*\end{pmatrix} = \begin{pmatrix}*&amp;amp;*&amp;amp;*\\*&amp;amp;*&amp;amp;*\\0&amp;amp;a_2&amp;amp;b_2\end{pmatrix}$$&#xA;and&#xA;$$\begin{pmatrix}*&amp;amp;*&amp;amp;*\\*&amp;amp;*&amp;amp;*\\0&amp;amp;a_2&amp;amp;b_2\end{pmatrix} \begin{pmatrix}1&amp;amp;0&amp;amp;0\\0&amp;amp;c_2&amp;amp;-s_2\\0&amp;amp;s_2&amp;amp;c_2\end{pmatrix} = \begin{pmatrix}*&amp;amp;*&amp;amp;*\\*&amp;amp;*&amp;amp;*\\0&amp;amp;0&amp;amp;r_2\end{pmatrix}\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we shall see that a row vector $\begin{pmatrix}0&amp;amp;0&amp;amp;*\end{pmatrix}$ is invariant under Givens rotations from the right which targets the first column:&#xA;$$\begin{pmatrix}0&amp;amp;0&amp;amp;*\end{pmatrix} \begin{pmatrix}c&amp;amp;-s&amp;amp;0\\s&amp;amp;c&amp;amp;0\\0&amp;amp;0&amp;amp;1\end{pmatrix} = \begin{pmatrix}0 \cdot (c+s)&amp;amp;0 \cdot (c-s)&amp;amp;1 \cdot *\end{pmatrix}\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This allows us to bring the element $(2,1)$ to zero without changing the last row:&#xA;$$\begin{pmatrix}*&amp;amp;*&amp;amp;*\\a_3&amp;amp;b_3&amp;amp;*\\0&amp;amp;0&amp;amp;r_2\end{pmatrix} \begin{pmatrix}c_3&amp;amp;-s_3&amp;amp;0\\s_3&amp;amp;c_3&amp;amp;0\\0&amp;amp;0&amp;amp;1\end{pmatrix} = \begin{pmatrix}*&amp;amp;*&amp;amp;*\\0&amp;amp;r_3&amp;amp;*\\0&amp;amp;0&amp;amp;r_2\end{pmatrix}\,,$$&#xA;and we have reached the upper triangle shape by applying three (different) Givens rotations from the right.&lt;/p&gt;&#xA;" OwnerUserId="16655" LastEditorUserId="16655" LastEditDate="2015-07-19T18:30:13.900" LastActivityDate="2015-07-19T18:30:13.900" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19981" PostTypeId="1" CreationDate="2015-06-21T07:04:50.790" Score="1" ViewCount="131" Body="&lt;p&gt;Given a triangulation (geometry), are there known algorithm in finding common side of triangles, that is O(N) or better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/9nHKJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="16470" LastActivityDate="2015-06-21T13:51:36.843" Title="Finding Common Side of Triangle" Tags="|graph-theory|delaunay-triangulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19982" PostTypeId="1" AcceptedAnswerId="19985" CreationDate="2015-06-21T13:02:40.960" Score="8" ViewCount="6428" Body="&lt;p&gt;I need to solve a non-linear set of three equations using scipy.&lt;/p&gt;&#xA;&lt;p&gt;However, I do not have any clue on which algorithm is suitable for my problem from a mathematical point of view (stability, convergence behaviour), since scipy provides a huge variety of different algorithms in the &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.15.1/reference/optimize.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scipy.optimize&lt;/code&gt;&lt;/a&gt; module such as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.optimize.root.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.optimize.root&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.fsolve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.optimize.fsolve&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/optimize.nonlin.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;nonlinear solvers&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here is my non-linear set (two linear, and a non-linear) of equations with 3 unknown variables:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q(t_k)=m_Sc_S\left(\vartheta_S(t_k)-\vartheta_S(t_{k-1})\right)\\&#xA;Q(t_k)=\dot{m}_Gc_{pG}\big(\vartheta_{GE}-\vartheta_{GA}(t_{k})\big)\Delta t\\&#xA;Q(t_k)=A\alpha\frac{\big(\vartheta_S(t_k)-\vartheta_{GE}\big)-\big(\vartheta_S(t_k)-\vartheta_{GA}(t_k)\big)}{\ln{\left(\frac{\vartheta_S(t_k)-\vartheta_{GE}}{\vartheta_S(t_k)-\vartheta_GA(t_k)}\right)}}\Delta t&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where unknown variables are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vartheta_S(t_k)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vartheta_{GA}(t_k)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$Q(t_k)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$m_S = 6.868\,\mathrm{kg}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\dot{m}_G = 0.007\,\mathrm{kg/s}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$c_S = 500\,\mathrm{J/kg K}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$c_{pG} = 1005\,\mathrm{J/kg K}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vartheta_S(0) = 273\,\mathrm{K}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vartheta_{GA}(0) = 293\,\mathrm{K}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vartheta_{GE} = 353\,\mathrm{K}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A = 3.733\,\mathrm{m}^2$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\alpha = 99\,\mathrm{W/m}^2\mathrm{K}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\Delta t = t_k-t_{k-1} = 1\,\mathrm{s}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Which algorithm is probably the best one for my problem?&lt;/p&gt;&#xA;" OwnerUserId="16692" LastEditorUserId="20688" LastEditDate="2022-01-04T20:48:03.807" LastActivityDate="2022-01-04T20:48:03.807" Title="Solve non-linear set of three equations using scipy" Tags="|python|nonlinear-equations|scipy|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19983" PostTypeId="2" ParentId="19981" CreationDate="2015-06-21T13:51:36.843" Score="2" Body="&lt;p&gt;If you know the number of edges and triangles, you could raster through a list of your triangles and create a triangle-to-edge map $(\mathcal{O}(N_{tri}))$ and simultaneously create the inverse map, which will give you the two triangles that share an edge. A naive search of that list for the right triangle-pair will be $\mathcal{O}(N_{edge})$, where $N_{edge}$ will at worst be $3 \cdot N_{tri}$. Of course, this all assumes you have a method for computing a unique ID for each edge in the graph.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2015-06-21T13:51:36.843" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19984" PostTypeId="2" ParentId="19982" CreationDate="2015-06-22T07:10:53.740" Score="2" Body="&lt;p&gt;&lt;code&gt;scipy.optimize.root&lt;/code&gt; is a unified interface for the nonlinear solvers that you list in your third point, which are implemented in Python. On the other hand, &lt;code&gt;scipy.optimize.fsolve&lt;/code&gt; is a wrapper around some Fortran functions (see docs). The interface is the same for both and I haven't seen any benchmarks (quick Google search), so just implement the function of the system and give them a go. Focus on correct implementation of this function and on choosing a sensible initial condition and then just try both &lt;code&gt;root&lt;/code&gt; with its different methods and &lt;code&gt;fsolve&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="782" LastActivityDate="2015-06-22T07:10:53.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19985" PostTypeId="2" ParentId="19982" CreationDate="2015-06-22T07:23:10.600" Score="7" Body="&lt;p&gt;Since your problem is small, you're probably best off trying &lt;code&gt;fsolve&lt;/code&gt; or &lt;code&gt;root&lt;/code&gt;. Both of these are interfaces to MINPACK and call &lt;code&gt;HYBRD&lt;/code&gt; or &lt;code&gt;HYBRJ&lt;/code&gt;. Since calculating a Jacobian matrix for your system shouldn't be hard (either do it by hand, or use your favorite computer algebra system, like SymPy, Sage, Maple, or Mathematica), you should supply a Jacobian matrix. The &lt;code&gt;HYBR&lt;/code&gt; functions in MINPACK use &quot;Powell's hybrid method&quot;, which uses Newton's method and checks if Newton steps will be descent steps by comparing against least-squares minimization (specifically, does the Newton step also decrease the sum-of-squared-residuals). If Newton steps are not descent steps, then it falls back to the gradient of the sum-of-squared-residuals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you're probably better off ignoring all of the alternatives for now. All of these other alternatives, broadly speaking, try to find low-dimensional structure in high-dimensional problems. Your problem has dimension 3, so the comparative savings of finding lower-dimensional structure aren't that great compared to, say, PDE discretizations that might have tens of thousands of variables.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-06-22T07:23:10.600" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19987" PostTypeId="1" AcceptedAnswerId="19990" CreationDate="2015-06-22T10:46:55.623" Score="3" ViewCount="1158" Body="&lt;p&gt;I have a 2D mesh of triangles used in Finite Element method to discretize the domain. I want to calculate the midpoints of all the edges because I want to use $\mathbb{P}^2$ elements. I am using MatLab. The data that I have is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Matrix containing nodes coordinates, $x$ and $y$. The size is $N\times 2$, where $N$ is the number of nodes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Matrix containing each triangle nodes. The size is $NT\times 3$, where $NT$ is the number of triangles in the mesh and 3 columns because each triangle has three nodes. For example, a row of this matrix could have $[10, 4 ,22]$ so the triangle is formed by these nodes and in order to know their coordinates I just take the rows $10, 4$ and $22$ from the node coordinate matrix (the previous one).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Array with boundary nodes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I know in advance that the number of nodes (including midpoints) is going to be $2N\times N_b -3$ where $N_b$ is the number of nodes in the boundary of the mesh, so I can preallocate the matrix. The new node matrix will contain the previous nodes coordinates and the new ones and the element matrix will now have 6 columns because one node will be added per edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What my code is doing is to run across all triangles and calculate their edges midpoints and if it is not yet in the coordinate node matrix include it (here I am losing many time checking if it is yet inside or not). The main problem is that as interior edges are common to two triangles many times the midpoint of an edge is already calculated so I am wasting time calculating the midpoint and after it checking that is already in the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you give me any advice about how to calculate more efficiently the midpoints of edges and include them in the matrix. Note: I do not have a matrix with the edges of the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16698" LastActivityDate="2015-06-23T06:29:35.510" Title="How to calculate efficiently mesh edges midpoints?" Tags="|finite-element|mesh-generation|graph-theory|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19988" PostTypeId="1" CreationDate="2015-06-22T11:12:19.517" Score="1" ViewCount="141" Body="&lt;p&gt;I want to solve linear hyperbolic system using Chebyshev collocation method. As this method puts severe constraint on the time step for the explicit time integration, I decided to switch to implicit time integration (backward Euler). I use time integrator &lt;code&gt;vode&lt;/code&gt; with options &lt;code&gt;method='bdf'&lt;/code&gt; and &lt;code&gt;order=1&lt;/code&gt; from Python package &lt;code&gt;scipy.integrate&lt;/code&gt;. I do not supply Jacobian to the time integrator, only the vector of right-hand side. In the beginning it was great: the computation with resolution $N=100$ gave me solution 108 times faster with implicit time integration, then with explicit (Runge-Kutta 4th order).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, It turned out that if I increase the grid resolution, then computations become unstable—in initial time period solution is stable, but then it starts to &quot;tremble&quot; and develops instability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This figure shows the numerically stable solution obtained with resolution $N=200$:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/AeNyt.png&quot; alt=&quot;Stable solution with resolution N=200&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This figure shows the numerically unstable solution obtained with resolution $N=400$:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/D8ckX.png&quot; alt=&quot;Unstable solution with resolution N=400&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please recommend to me implicit time integrators that are good to use with Chebyshev collocation method?&lt;/p&gt;&#xA;" OwnerUserId="3254" LastEditorUserId="3254" LastEditDate="2015-06-22T14:02:06.757" LastActivityDate="2015-06-22T14:02:06.757" Title="Implicit time integrator for Chebyshev collocation method for linear hyperbolic system" Tags="|hyperbolic-pde|implicit-methods|spectral-method|collocation|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="19989" PostTypeId="1" CreationDate="2015-06-22T13:01:39.527" Score="0" ViewCount="82" Body="&lt;p&gt;I am trying to replicate the MATLAB simulation presented in this &lt;a href=&quot;http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6399743&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;.  More specifically, I have to code the solution to this equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{a}{2R_i}\frac{\partial^2 V_m}{\partial x^2} - \left[g_\text{Na}m^3 h(V_m - E_\text{Na}) + g_\text{K} n^4(V_m -  E_\text{K}) + g_\text{S}(V_m - E_\text{S})\right] = C_m \frac{\partial V_m}{\partial t} + \frac{a}{2R_i}\frac{\partial E_x}{\partial x}(x,t) \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $m$, $n$, and $h$ are gating variables that are ordinary differential equations and are functions of time, and I have solved for them already. I have also written the difference equation for this PDE.  I would like to set $\frac{\partial E}{\partial x}(x,t)$ equal to 1 (denoted as $g$ in the script) for now (and tackle that next).  I am using a finite differences method (I think)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;&#xA;T=1; % Max time (s)&#xA;L=1; % Max length of neuron&#xA;dt=0.04; %time step&#xA;dx=0.01; %space step&#xA;t=[0:dt:T]; %time vector&#xA;x=[-L:dx:L]; %Space vector&#xA;&#xA;M=length(x);&#xA;g=1; %The dE/dx bit&#xA;a=5; %Neuron radius&#xA;R_i=1; %Intracellular resistance&#xA;A=a./2*R_i;&#xA;I=0.1; %External Current Applied&#xA;ENa=55.17; % mv Na reversal potential&#xA;EK=-72.14; % mv K reversal potential&#xA;El=-49.42; % mv Leakage reversal potential&#xA;g_Na=1.2; % mS/cm^2 Na conductance&#xA;g_K=0.36; % mS/cm^2 K conductance&#xA;g_l=0.003; % mS/cm^2 Leakage conductance&#xA;b=4;&#xA;Cm=0.01; %Membrane capacitance&#xA;C=Cm;&#xA;%Solve for m,n,h&#xA;&#xA;%Solve for m, n, h   &#xA;V=-60; % Initial Membrane voltage&#xA;m=am(V)/(am(V)+bm(V)); % Initial m-value&#xA;n=an(V)/(an(V)+bn(V)); % Initial n-value&#xA;h=ah(V)/(ah(V)+bh(V)); % Initial h-value&#xA;y0=[V;n;m;h];&#xA;tspan = [0,max(t)];&#xA;%Matlab's ode45 function&#xA;[time,V] = ode45(@HH,tspan,y0);&#xA;V_hh=V(:,1);&#xA;n=V(:,2);&#xA;m=V(:,3);&#xA;h=V(:,4);&#xA;N=length(time);&#xA;plot(time,m,time,h,time,n);&#xA;V_initialgrid=zeros(M,N);&#xA;figure();&#xA;plot(time,V_hh)&#xA;V_initialgrid(1,:)=V_hh';&#xA;&#xA;xlabel('time');&#xA;ylabel('length');&#xA;for j=2:N-1 %Stepping through time&#xA;    B=g_Na*m(j).^3*h(j)+g_K*n(j).^4+g_l;&#xA;    for i=2; %stepping thru space&#xA;        V_initialgrid(i,j)=(A* V_initialgrid(i-1,j)+B* V_initialgrid(i,j)+C* V_initialgrid(i-1,j)+g);&#xA;&#xA;        for i=3:M-1 %stepping thru space&#xA;            V_initialgrid(i+1,j)=(-dt*(g-(A.*(( V_initialgrid(i,j-1)-2.* V_initialgrid(i,j)+ V_initialgrid(i,j+1))./(dx.^2))-B.*-60))./C)+ V_initialgrid(i,j);&#xA;        end&#xA;    end&#xA;end&#xA;    imagesc(log(abs(V_initialgrid)))&#xA;&#xA;    xlabel('Time');&#xA;    ylabel('L');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial V}{\partial x}=\frac{\partial m}{\partial x}=\frac{\partial h}{\partial x}=\frac{\partial n}{\partial x}=0\quad \text{for } x=\pm L$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial V}{\partial t}=\frac{\partial m}{\partial t}=\frac{\partial h}{\partial t}=\frac{\partial n}{\partial t}=0 \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know my error lies within the indexing of the solution of the difference equation.&lt;/p&gt;&#xA;" OwnerUserId="16703" LastEditorUserId="9667" LastEditDate="2015-06-22T19:40:38.763" LastActivityDate="2015-06-22T19:40:38.763" Title="Nonlinear 2D modeling of Neural Electromagnetic field in Matlab" Tags="|matlab|pde|finite-difference|computational-biology|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="19990" PostTypeId="2" ParentId="19987" CreationDate="2015-06-22T15:36:02.123" Score="4" Body="&lt;p&gt;If you have an array that stores the indices of the 3 neighbors of each cell, then you would only compute the midpoint of an edge of the neighbor cell has a higher index than the current cell, or if there is no neighbor at all. This way you have an easy tie breaker to decide which of the two cells is responsible for computing the edge midpoint.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-22T15:36:02.123" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19991" PostTypeId="1" AcceptedAnswerId="19992" CreationDate="2015-06-22T16:18:01.823" Score="1" ViewCount="244" Body="&lt;p&gt;I am trying to learn Gmsh and Fenics and was looking at an example which shows the application of Boundary conditions on a simple Poisson problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/1.3.0/python/demo/documented/bcs/python/documentation.html&quot; rel=&quot;nofollow&quot;&gt;http://fenicsproject.org/documentation/dolfin/1.3.0/python/demo/documented/bcs/python/documentation.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example defines the domain Ω to be a model of blood vessels with an aneurysm. It has one inlet vessel and two outlet vessels. They define no slip boundary conditions on the walls of the vessels and aneurysm, that is u=u0=0.0. We let u=u1=1.0 be the Dirichlet condition on the inlet, the outlets will have the prescribed values u=u2=2.0 and u=u3=3.0. In summary, we have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u=u0=0.0 on ΓD,0 (noslip boundary);&#xA;u=u1=1.0 on ΓD,1 (inlet);&#xA;u=u2=2.0 on ΓD,2 (outlet 1) ;&#xA;u=u3=3.0 on ΓD,3 (outlet 2) ;&#xA;f=0.0 (source term)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can understand the code overall, but I am unable to understand the meaning of these Boundary conditions, i.e., unable to visualize them in a physical sense, intuitively. I' ll appreciate if you can post any other link or better example too which explains the BC phenomena better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="549" LastEditDate="2015-06-22T18:50:47.677" LastActivityDate="2015-06-22T20:51:50.600" Title="Please explain the meaning of these Boundary conditions" Tags="|boundary-conditions|fenics|" AnswerCount="1" CommentCount="2" ClosedDate="2015-06-23T21:22:50.893" ContentLicense="CC BY-SA 3.0" />
  <row Id="19992" PostTypeId="2" ParentId="19991" CreationDate="2015-06-22T20:51:50.600" Score="2" Body="&lt;p&gt;If you look at the documentation,&lt;sup&gt;1&lt;/sup&gt; $u$ is described as solving the Poisson equation $(\nabla u,\nabla v) = (f,v)$ for $v\in H^1(\Omega)$. This means that it is unlikely to be a velocity field (it's scalar, after all). I would interpret the documentation insofar as only the &lt;em&gt;geometric&lt;/em&gt; domain is meant to model a blood vessel, while the PDE is only a very academic example that isn't supposed to model anything. Hence, there is no further interpretation than that the value of $u$ is prescribed (as different values at different parts of the boundary).&lt;/p&gt;&#xA;&#xA;&lt;hr/&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;1. which is outdated, by the way -- current is 1.5, and 1.6 is coming up soon&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-06-22T20:51:50.600" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19993" PostTypeId="2" ParentId="19987" CreationDate="2015-06-23T06:29:35.510" Score="2" Body="&lt;p&gt;If the main bottleneck is computing a list (well, in MATLAB, I guess it's a matrix) of unique edges, as long as you can find a hash table implementation, you should be able to find a(n average-case) linear time algorithm in the number of vertices (or elements, or edges, by Euler's formula, since your 2D mesh is a planar graph).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On way to do this is to iterate over the matrix containing the triangle nodes. Each row contains three node indices, and thus three two-element vectors of potential edges. Insert these vectors into your hash table, using the &lt;em&gt;sorted&lt;/em&gt; version of the vector as a key. (An example: If my edges are $[1, 3]$, $[3, 2]$, $[2, 1]$, my hash table should look like &lt;code&gt;{[1, 2]: [[2, 1]], [1, 3]: [[1, 3]], [2, 3]:[[3, 2]]}&lt;/code&gt;, where I'm using Python dictionary notation to represent the hash table because Python dictionaries are implemented as hash tables in the standard library.) Then reading off the keys of the hash table will give you unique edges, and you can use this information to insert midpoints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are almost certainly more efficient data structures for manipulating finite element information. &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/sieve.pdf&quot; rel=&quot;nofollow&quot;&gt;Sieves&lt;/a&gt;, which I think is DMPlex in PETSc, come to mind; other DAG-like representations exist in the literature that are similar in spirit, if not practice. However, I don't think there's a MATLAB library that implements sieves, whereas there are hash table implementations available for most languages as user-facing libraries, making it an easier solution to kludge together in any language. (It also depends on less in the way of data structures and algorithms knowledge.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-06-23T06:29:35.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="19994" PostTypeId="2" ParentId="19651" CreationDate="2015-06-23T19:01:17.333" Score="1" Body="&lt;p&gt;To do precisely what you are describing, you likely want to interpolate your grid data to a set of discrete spheres with prescribed radii, then perform the spherical transform on each set of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility, and a generalization of the above, would be to define a spherical basis set with the spherical harmonics as the $(\theta, \phi)$-varying functions, and some sort of radial basis function defined on $r \in \left[0,a\right]$, where $a$ is the radius of the largest sphere that will fit inside your grid box. You can then solve for the coefficients of these functions via a point-matching testing procedure and inverting the coefficient matrix. This will give you a continuous function whose value coincides with your grid data at the grid points and interpolates the grid data between grid points, and which you can evaluate at each radius of interest to give you the data you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if there is an extant piece of software that suits your exact needs, but both approaches I described above are easily implemented in a few lines of Matlab or Python code.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2015-06-23T19:01:17.333" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="19997" PostTypeId="1" AcceptedAnswerId="20000" CreationDate="2015-06-24T00:02:54.887" Score="3" ViewCount="502" Body="&lt;p&gt;For an assignment I have to implement a 1D Poisson PDE with inhomogeneous Dirichlet BC's &#xA;$$\Delta_1 u = f,  \quad u(a)=g(a), \: u(b) = g(b) $$&#xA;I have managed to make it work, but I am not seeing the expected convergence behaviour. I used the standard (?) discretisation on the unit interval [0,1], &#xA;$$-u_{j+1} + 2u_j - u_{j-1} = h^2f_j$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following MATLAB code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [sol, xVals, L, fVals] = poissonSolveFD2(f, g, N)&#xA;&#xA;h = 1/(N+2);&#xA;h2= h*h;&#xA;&#xA;sol = zeros(N+2,1);&#xA;sol(1) = g(0);&#xA;sol(N+2) = g(1);&#xA;&#xA;xVals = linspace(0,1,N+2)';&#xA;&#xA;L = makeLaplace(N);&#xA;&#xA;fVals = f(xVals);&#xA;fVals = h2*fVals;&#xA;&#xA;fVals(2) = fVals(2) + g(0);&#xA;fVals(N+1) = fVals(N+1) + g(1);&#xA;&#xA;sol(2:N+1) = L\fVals(2:N+1);&#xA;&#xA;&#xA;plot(xVals,sol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;L=makeLaplace(N) returns the tridiagonal matrix $L=\operatorname{tridiag}(-1,2,-1) \in \mathbb{R}^{n \times n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code yields me a solution, but only shows an error reduction of $\mathcal{O}(h)$ instead of $\mathcal{O}(h^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have studied the answers at &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5520/testing-1d-poisson-solver&quot;&gt;Testing 1D Poisson Solver&lt;/a&gt;, but that didn't get me any closer. That question also only considers homogeneous BC's, so I think this question has a right of existence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is probably something small and stupid, but for some reason I can't figure it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16722" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-07-01T11:11:16.543" Title="1D inhomogeneous Poisson PDE with Dirichlet BCs, slow convergence" Tags="|matlab|pde|poisson|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="19998" PostTypeId="2" ParentId="19894" CreationDate="2015-06-24T17:32:54.580" Score="1" Body="&lt;p&gt;I don't know much about using databases but it seems that a k-d tree would be a good way to do it. Look at the following link &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.stanford.edu/class/cs106l/handouts/assignment-3-kdtree.pdf&quot; rel=&quot;nofollow&quot;&gt;http://web.stanford.edu/class/cs106l/handouts/assignment-3-kdtree.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2015-06-24T17:32:54.580" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20000" PostTypeId="2" ParentId="19997" CreationDate="2015-06-24T20:19:12.407" Score="2" Body="&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: the advice below is necessary, but was written just to expand on the comments above and isn't the actual issue with the code above. See the other answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have a discretized system $Ax=b$, and certain entries of $x$ are specified by your boundary conditions. Let $x_1$ denote the vector of &quot;free&quot; (unknown) values, and let $x_2$ denote the vector of &quot;fixed&quot; (known) values. Similarly partition $A$ and $b$, obtaining the block system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left[ \begin{array}{cc}  A_{11} &amp;amp; A_{12} \\ A_{21} &amp;amp; A_{22} \end{array} \right]\left[ \begin{array}{c}  x_1 \\ x_2 \end{array} \right] = \left[ \begin{array}{c}  b_1 \\ b_2 \end{array}  \right] $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values of $x_2$ are specified by your problem, so your task is to compute $x_1 = A_{11}^{-1}(b_1 - A_{12}x_2$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values of $b_2$ should be consistent (if they are specified). &lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2015-07-01T11:11:16.543" LastActivityDate="2015-07-01T11:11:16.543" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20001" PostTypeId="2" ParentId="19997" CreationDate="2015-06-25T00:16:58.143" Score="5" Body="&lt;p&gt;The answers so far seem to me to be completely off target. The boundary conditions you have are consistent with the r.h.s. of the equation and the matrix $L$, so they are not the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, there are two things wrong here. First, the usual finite second-order difference approximation to $u''$ is&#xA;$$ u''(x_0) = h^{-2}\big(u(x_{-1}) - 2u(x_0) + u(x_1)\big), $$&#xA;which has the opposite sign to what you have. In effect, if you use $\mathrm{diag}(-1,2,-1)$ you end up solving $\Delta u=-f$, which is a bug your testing doesn't seem to have caught.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, when you have $N$ interior points and $N+2$ points in total, with $x_1 = 0$ and $x_{N+2} = 1$, the $j$-th point is&#xA;$$ x_j = \frac{j-1}{N+1}, $$&#xA;and so&#xA;$$ h = \frac{1}{N+1}, $$&#xA;whereas you have $h=1/(N+2)$. This is the reason for incorrect order of convergence. The easiest way here to test for this kind of mistake is to use the exact solution $u=4x(x-1)$ to the equation $\Delta u=8$ with $g(x)=0$, so that $u(\frac12)=-1$. Because the error of the FD formula behaves as $u^{(4)}$, the scheme is exact in this case, so gettting any error other than round-off error is a bug.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, for how to set boundary conditions see the answer by &lt;strong&gt;Patrick&#xA;Sanan&lt;/strong&gt;, but that doesn't seem to be a problem here as far as I can&#xA;see. The matrix $A_{12}$ has first row $(-1,0)$ and last row $(0,-1)$, with zeros everywhere else,&#xA;so you are correct to add the boundary conditions to the rhs at $x_2$&#xA;and $x_{N+1}$ (although the sign would still be wrong as above). The units of $h^2f$ and $g$ are also correct ($f$ has units $u/x^2$ (like $u''$), $g$ has units $u$, $h$ has units $x$ and $h^2f$ has units $u$, same as $g$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fourth, for testing, it is much easier to pick a known exact solution, say $v(x)=e^x$, go backwards to work out $f(x)=e^x$ and $g(x)=e^x$, and then apply the scheme and compare $u$ with $v$. This is commonly known as the method of manufactured solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a modified version of your code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [sol, xVals, L, fVals] = poissonSolveFD2(f, g, N)&#xA;&#xA;h = 1/(N+1);&#xA;h2= h*h;&#xA;&#xA;sol = zeros(N+2,1);&#xA;sol(1) = g(0);&#xA;sol(N+2) = g(1);&#xA;&#xA;xVals = linspace(0,1,N+2)';&#xA;&#xA;L = spdiags(repmat([1,-2,1], N, 1), -1:1, N, N);&#xA;&#xA;fVals = f(xVals);&#xA;fVals = h2*fVals;&#xA;&#xA;fVals(2) = fVals(2) - g(0);&#xA;fVals(N+1) = fVals(N+1) - g(1);&#xA;&#xA;sol(2:N+1) = L\fVals(2:N+1);&#xA;&#xA;plot(xVals,sol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Simple check that it works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; u = @(x) exp(x);&#xA;&amp;gt;&amp;gt; e1 = norm(poissonSolveFD2(u, u, 100) - u(linspace(0,1,102)'), 'inf')&#xA;e1 =&#xA;   1.7307e-06&#xA;&amp;gt;&amp;gt; e2 = norm(poissonSolveFD2(u, u, 201) - u(linspace(0,1,203)'), 'inf')&#xA;e2 =&#xA;   4.3269e-07&#xA;&amp;gt;&amp;gt; e1 / e2&#xA;ans =&#xA;    3.9999&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-06-25T02:06:47.077" LastActivityDate="2015-06-25T02:06:47.077" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20003" PostTypeId="1" AcceptedAnswerId="20004" CreationDate="2015-06-25T17:00:31.007" Score="3" ViewCount="762" Body="&lt;p&gt;I am trying to understand the DG FEM methods, but I got lost in their definitions. In some papers I read that the &quot;C0 penalty method&quot; is one example of the DG method, but sometimes they are separated and handled as two different methods. Can someone explain what's the difference between them? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;:&#xA;There is a &lt;a href=&quot;https://dl.dropboxusercontent.com/u/48847074/Publications/motivationv2.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; dealing with three different finite element methods for solving the Monge-Ampere equation: C1 finite element method, C0 penalty method, discontinuous galerkin method. After I familiarized myself with the basics of the discontinuous galerkin methods I don't understand why the C0 penalty method is not a DG method (the C0 penalty method for the Monge-Ampere equations was developed in 2011, and this paper from 2014 considers itself as a first try to solve the Monge-Ampere equations with a DG method). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested only about elliptic PDEs (second order and fourth order as well) and I would like to clarify the meanings and differences behind the names related to the topic. If I got it right, in case of a fourth order equation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;continuous galerkin method: we need C1 continuity and seek the solution in the H2 space&lt;/li&gt;&#xA;&lt;li&gt;interior penalty method: we use C0 continuous elements and seek the solution in the H1 space&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Does this mean that in the interior penalty method the elements are not fully discontinuous? Therefore we can define a truly discontinuous method, where we seek the solution in the L2 space. &lt;/p&gt;&#xA;" OwnerUserId="14907" LastEditorUserId="14907" LastEditDate="2015-06-28T22:28:14.217" LastActivityDate="2015-06-28T22:28:14.217" Title="What's the difference between C0 penalty methods and Discontinuous Galerkin methods?" Tags="|finite-element|discontinuous-galerkin|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20004" PostTypeId="2" ParentId="20003" CreationDate="2015-06-25T19:01:47.343" Score="4" Body="&lt;p&gt;DG methods are a more general description, referring to finite element methods which represent the solution in a discontinuous manner (usually with coupling between elements through a numerical flux). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specific numerical flux tends to differ between type of equation (hyperbolic vs parabolic/elliptic) and formulation (2nd order equation vs first order system). The interior penalty DG method is one common choice of flux for elliptic equations (like diffusion or Laplace) in primal (2nd order) form.  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-06-25T19:01:47.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20005" PostTypeId="1" AcceptedAnswerId="20006" CreationDate="2015-06-25T20:29:02.833" Score="2" ViewCount="149" Body="&lt;p&gt;Is there a natural way to find the solution to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$AX = B, X^TX = C \enspace \text{?}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X$ is a matrix and has a small number of rows, and $A$ is sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approximate solution would be fine.&lt;/p&gt;&#xA;" OwnerUserId="16749" LastEditorUserId="9667" LastEditDate="2015-06-25T21:38:32.290" LastActivityDate="2015-06-26T01:45:10.273" Title="Solve $AX = B$ where $X^T X = C$" Tags="|linear-algebra|optimization|convex-optimization|quadratic-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20006" PostTypeId="2" ParentId="20005" CreationDate="2015-06-26T01:19:36.570" Score="5" Body="&lt;p&gt;Your problem is related to low rank approximation problems, about which there has been a lot of research in recent years.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are you looking for a solution in which $X$ has a specified number of rows, or are you trying to find the smallest number of rows possible? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases, users are interested in finding $X$ such that $X^{T}X$ is close to $C$ (in e.g. the Frobenius norm.)  Are you sure that you need to have $X^{T}X$ equal to $C$ to some fairly tight tolerance, or would you be willing to accept a solution with $\| X^{T}X-C \|_{F}$ reasonably small? Could you trade off that objective against the rank of $X$?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The occurrence of linear constraints on the elements of $X$ rather than on the elements of $X^{T}X$ is also unusual.  Is it possible to write your constraints in terms of $X^{T}X$ rather than $X$?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, these low rank optimization problems are non-convex and NP-Hard.  However, heuristic approaches using surrogate objective functions and semidefinite programming can be very effective in finding solutions with relatively low rank.  Another alternative is to apply a more general nonlinear optimization method with $X$ as the variable (but the problems are non-convex so a local optimum might not be globally optimal.)  See for example the SDPLR method of Burer and Monteiro.      &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-06-26T01:45:10.273" LastActivityDate="2015-06-26T01:45:10.273" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20008" PostTypeId="2" ParentId="19840" CreationDate="2015-06-26T14:10:21.973" Score="3" Body="&lt;p&gt;The constraint $\det(A) &amp;gt; 0$ is &lt;strong&gt;very unstable&lt;/strong&gt; and &lt;strong&gt;absolutely horrible&lt;/strong&gt;.  Would you believe that on practical, non-contrived 6 by 6 symmetric matrices, changing one element of the matrix by 1e-15 can make the determinant computed in double precision swing from 1e20 to -1e20?  Good luck in trying to have a meaningful determination of whether $\det(A)$ is &gt; or &amp;lt; 0 at any point.  The gradient is the least of your worries. The constraint itself just will not work unless you compute in ultra-high precision, or perhaps you are only dealing with a 2 or 3 dimensional problem. Prof Neumaier was giving you a more numerically stable workaround. But I can say from my experience, imposing a Cholesky factorization type constraint does not seem to work well, and makes the problem non-convex, even if the original problem is convex; you may succeed, but you may wind up adding a bunch of spurious local optima (and even saddlepoints?) to your problem, and that is not a good thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the matrix $A$ (constrained to be) symmetric, and do you really wish to impose that $\det(A) &amp;gt; 0$ and that all leading principal submatrices have $\det &amp;gt; 0$, i.e., that $A$ be positive definite?  If so, instead of using the constraint $\det(A) &amp;gt; 0$ (which apart from its terrible numerical properties, is not sufficient to ensure positive definiteness of $A$, unless it is imposed (as perhaps you do?) on all leading principal submatrices),  use a semidefinite constraint, $A &amp;gt; 0$ in Linear Matrix Inequality (LMI) sense. I.e., a constraint that A is positive definite.  However, practical optimization software either does not accept strict inequality constraints, or if it does, treats them as non-strict inequality constraints.  If positive semidefinite is o.k., your constraint would be $A \ge 0$ (in LMI sense). If you really need A to be strictly positive definite (det(A) strictly positive), then you need to determine a minimum acceptable eigenvalue, &lt;code&gt;min_eig&lt;/code&gt;, and then impose the constraint &lt;code&gt;A - min_eig * (Identity matrix of correct size) &amp;gt;= 0&lt;/code&gt; (in LMI sense). You do need to make some allowance for solver tolerance used in constraint feasibility assessment (i.e., solver will allow constraint to be violated by a little, for example, by 1e-6 or 1e-8).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, it would help to know what your objective function and the rest of the constraints are. If the objective function and the rest of the constraints are convex (presuming a minimization problem), you could use CVX or YALMIP (both are free) to solve your problem.  You said in a comment (belongs in the question, so you should edit) that the objective is quadratic, but is it convex?  If non-convex, you may still be able to use YALMIP, perhaps in conjunction with having YALMIP call the free solver PENLAB. If you use CVX or YALMIP, they take care of any differentiation for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit in response to edit of Question&lt;/strong&gt;: As shown in the edited version of your question, $A$ is constrained to be symmetric nonnegative and have $\det(A) &amp;gt; 0$, but $A$ is not constrained to be positive definite. It is in fact possible for an nonnegative symmetric matrix of dimension &gt;= 3 to have positive determinant yet not be positive definite. Do you wish to allow that situation?  If so, my answer above the edit above does not apply.  If in fact you require $A$ be positive definite, then my answer above the edit does apply, and you should further edit your question to reflect the constraint that $A$ be positive definite. &lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="27208" LastEditDate="2018-03-19T21:03:41.257" LastActivityDate="2018-03-19T21:03:41.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20009" PostTypeId="2" ParentId="19911" CreationDate="2015-06-26T15:00:10.953" Score="1" Body="&lt;p&gt;You can download YALMIP for free &lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.Download&quot; rel=&quot;nofollow&quot;&gt;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.Download&lt;/a&gt; . YALMIP will do the dirty work for you. Install YALMIP, run yalmiptest to test installation, read &lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Tutorials.Basics&quot; rel=&quot;nofollow&quot;&gt;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Tutorials.Basics&lt;/a&gt; .  If you have CPLEX available and installed under MATYLAB,  it will/can use that, otherwise something else.  The L need not be psd in order to solve under YALMIP using CPLEX or scip as the solver. However, you stated it is a Laplacian matrix, therefore is must be psd.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = length(P); % this is not really YALMIP code&#xA;x = binvar(n,1) % declare x as binary; would use sdpvar instead for continuous variable&#xA;sol = optimize([Ax&amp;lt;=b,Ax==b,lb&amp;lt;=x&amp;lt;=ub],x'*L*x+abs(P'*x))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is complete YALMIP code to solve this. After being solved, value(x) provides the optimal value of x.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to specify a particular solver, for instance scip, rather than letting YALMIP pick the solver, use the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sol = optimize([Ax&amp;lt;=b,Ax==b,lb&amp;lt;=x&amp;lt;=ub],x'&lt;em&gt;L&lt;/em&gt;x+abs(P'*x), sdpsettings('solver','scip'))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't need to put into generic format. YALMIP handles that for you &quot;under the hood&quot; to put in form suitable for solvers. Just put in the constraints you have. If you don't have any constraints other than x being binary, you can use [] instead of [Ax&amp;lt;=b,Ax==b,lb&amp;lt;=x&amp;lt;=ub] . Also note that sdpvar doesn't mean that x is being declared to be senidefinite. The declaration is a holdover from when YALMIP only addressed SDPs.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2015-06-26T15:06:45.550" LastActivityDate="2015-06-26T15:06:45.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20010" PostTypeId="2" ParentId="19838" CreationDate="2015-06-26T15:32:52.523" Score="0" Body="&lt;p&gt;You can formulate and solve this using YALMIP under MATLAB. You can download YALMIP for free &lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.Download&quot; rel=&quot;nofollow&quot;&gt;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Main.Download&lt;/a&gt; . YALMIP will do the dirty work for you. Install YALMIP, run yalmiptest to test installation, read &lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Tutorials.Basics&quot; rel=&quot;nofollow&quot;&gt;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Tutorials.Basics&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I generated L= rand(1000,20) and solved the following problem in under 3 sec when YALMIP selected cplex by default as the solver. It also solved it in under 3 sec using scip.  These times are if I don't impose a constraint X &gt;= 1 (see next paragraph, because I am confused as to what your optimization problem actually is).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, and this is an important point, the problem as you have stated it always has an optimal objective value = 0, which is achieved with X = all zeros. So you need some other constraint on X or a different formulation in order to have a non-trivial problem. If you do that, it is possible that the run time could go up considerably.  I have been ignoring ξ &gt;= 1; I don't know what your problem is. I have assumed that X is a vector of the optimization (decision) variables. So you need to fix up the problem. Clearly state the objective and what the decision variables are, and what the problem inputs are.  I don't know what ξ is or how it enters the objective function. O.k., here's my guess: you mean that all elements of X are &gt;= 1.  is that correct? If so, this will take a long time to run.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = size(L,1);&#xA;X = intvar(n,1) % declares X to be an n by 1 vector of integer variables&#xA;sol = optimize([X &amp;gt;= 1],var(X'*L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to specify a particular solver, such as scip, rather than letting YALMIP pick what it thinks is best from among the installed solvers, use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sol = optimize([X &amp;gt;= 1],var(X'*L),sdpsettings('solver','scip'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If there are any other constraints, put them inside the [] , separated by , or ; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the problem has been solved, the optimal argument value (i.e., argmin) is obtained with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;value(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is being solved as a Mixed Integer QP. YALMIP is putting it into the form needed by the solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, I believe FMINCON will ignore tolerance of 1e-100; it just can't handle that, so it will use what &quot;it wants to&quot;.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2015-06-26T15:57:24.170" LastActivityDate="2015-06-26T15:57:24.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20011" PostTypeId="1" AcceptedAnswerId="20016" CreationDate="2015-06-26T15:52:16.243" Score="2" ViewCount="623" Body="&lt;p&gt;I am attempting to simulate fluid flow through a porous foam. I would like to have no-slip boundary conditions on part of the boundary and free flow conditions on the inlet and outlet. Right now I am using the Darcy-Brinkman equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-\mu\Delta\mathbf{u} + \mathbf{K}\mathbf{u} + \nabla p = \mathbf{0}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla\cdot\mathbf{u} = 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{K}$ is a permeability tensor. Mathematically the boundary conditions I have here make sense, however Whitaker (&lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF01036523&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/article/10.1007%2FBF01036523&lt;/a&gt;) uses a length scale argument to say that the Darcy-Brinkman equations cannot be used to impose no slip boundary conditions. In literature this is still a common technique however. Is there a justification for using the Darcy-Brinkman equations as I wish to do? Ideally I'd like a mathematical justification, but that would apparently directly contradict Whitaker, so an experimental justification is acceptable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is no justification, can I reformulate my problem somehow using regular Darcy flow? &lt;/p&gt;&#xA;" OwnerUserId="9728" LastActivityDate="2015-06-26T23:23:04.840" Title="What are acceptable boundary conditions for porous media flow?" Tags="|pde|fluid-dynamics|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20012" PostTypeId="2" ParentId="19894" CreationDate="2015-06-26T17:01:46.997" Score="1" Body="&lt;p&gt;If your data is indexed, I'm not sure if a 3rd party algorithm would perform much faster than the ones already included in mongoDB. According to your link,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Changed in version 2.2.3: Applications can use $box without having a geospatial index. However, geospatial indexes support much faster queries than the unindexed equivalents. Before 2.2.3, a geospatial index must exist on a field holding coordinates before using any of the geospatial query operators.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, if I were to do this from scratch, I would convert my geodetic data into Cartesian coordinates (using e.g. &lt;a href=&quot;http://www.movable-type.co.uk/scripts/latlong.html&quot; rel=&quot;nofollow&quot;&gt;Haversine&lt;/a&gt;), then sort them into a balanced &lt;a href=&quot;http://www.autonlab.org/autonweb/14665/version/2/part/5/data/moore-tutorial.pdf&quot; rel=&quot;nofollow&quot;&gt;kd-tree&lt;/a&gt;. Kd-trees index &lt;em&gt;k&lt;/em&gt;-dimensional points and sort them relative to a root node (usually the mean or median of the data set) using a divide-and-conquer methodology, making them especially adept for nearest-neighbor and range search algorithms. A range search will find points in $O(log\,n)$ time, which should be sufficiently fast. Of course, you would have to convert your target query into Cartesian coordinates before conducting a search. I suspect, however, that mongoDB implements search algorithms with the same level of performance, assuming your data is indexed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's important to first convert to Cartesian coordinates because most kd-tree searches rely on the $L_2$-norm as the metric for determining distance between points, which is incorrect for geospatial coordinates. Furthermore, using lat/long for the kd-tree's splitting plane may also result in difficulties, since longitudes are not equidistant from each other (i.e. distance between longitudes is a function of latitude, which might make it a poor choice for splitting the kd-tree). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kd-trees are really only efficient for bulk-loading situations, where you know your dataset in advance and can sort them all in one go. It's possible to add new points to a kd-tree, but it's not ideal. If you expect your data set to change often, I would suggest an R-tree.&lt;/p&gt;&#xA;" OwnerUserId="9952" LastActivityDate="2015-06-26T17:01:46.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20013" PostTypeId="1" CreationDate="2015-06-26T18:27:38.513" Score="0" ViewCount="525" Body="&lt;p&gt;I'm Looking for an implementation of the IAPWS-IF97 water properties on C/C++.&#xA;I'm aware of the library &lt;a href=&quot;http://freesteam.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;freesteam&lt;/a&gt;. However freesteam does not include all the properties I'm looking for. Particularly I need differential properties like dh/dp, dv/dp and in certain regions. I use a similar implementation for Octave/Matlab and I would also like to have it in C/C++.&lt;/p&gt;&#xA;" OwnerUserId="6701" LastActivityDate="2015-08-25T12:54:49.823" Title="Water Properties IAPWS-IF97 implementation on C/C++" Tags="|c|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20014" PostTypeId="2" ParentId="20003" CreationDate="2015-06-26T18:36:37.460" Score="3" Body="&lt;p&gt;From a mathematical perspective, the term 'DG method' by itself refers only to the class of methods that use a piecewise discontinuous basis. This choice of basis necessitates the introduction of some additional parameters enforcing the continuity requirements on the solution as desired (penalty parameters) as well as to control the numerical properties of the resulting linear system (s-form)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put it simply, for the case of elliptic and parabolic equations, the choice of parameters dictates the &lt;em&gt;type&lt;/em&gt; of flux introduced into the problem (as mentioned by Jesse). You mention the C0/IIPG method, which is what you get when one of the parameters is chosen to be zero. For elliptic equations, the choice of an appropriate &lt;code&gt;sform&lt;/code&gt; and penalty parameter gives rise to SIPG, IIPG, NIPG methods. So ALL these methods are DG-methods, but their solutions differ from one another in that some would exhibit more numerical dispersion and/or smearing of solution fronts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you read through Ch 1 of &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898717440&quot; rel=&quot;nofollow&quot;&gt;Rivière's Book&lt;/a&gt; if you can get a hold of it. &lt;/p&gt;&#xA;" OwnerUserId="15103" LastActivityDate="2015-06-26T18:36:37.460" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20015" PostTypeId="2" ParentId="10678" CreationDate="2015-06-26T20:59:43.253" Score="-2" Body="&lt;p&gt;As per what I understand about shape functions is that, it is the about connecting the geometrical nodal coordinates with Element displacement  with a same shape function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a 1D case. A bar with 2 nodes at it ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I connect this element with its nodal coordinates, I can find out displacement at any point in this element with the help of interpolation function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, basically shape functions are the approximations that we do in order to find deformation at any point in space in a commendable way.&lt;/p&gt;&#xA;" OwnerUserId="16763" LastActivityDate="2015-06-26T20:59:43.253" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20016" PostTypeId="2" ParentId="20011" CreationDate="2015-06-26T23:23:04.840" Score="5" Body="&lt;p&gt;The velocity $\mathbf u$ is in $H^1$, so imposing all or some components of the velocity as boundary conditions is allowed. This includes no-slip or tangential flow boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from a physical perspective, this only makes sense if the ratio of the Stokes to the Brinkman terms is sufficient large, i.e., in particular if $\frac{\mu}{|\mathbf K| L^2} \gg 1$ where $L$ is a length scale that compensates that the Stokes term has two derivatives whereas the Brinkman term does not. On the other hand, if $\frac{\mu}{|\mathbf K| L^2} \ll 1$, then the porous media aspect dominates the Stokes flow aspect, and the solution degenerates to one that is only a slightly regularized function in $H_\text{div}$ where the solution of the mixed Laplace lives. In that case, you can no longer impose all components of the velocity at the boundary, but only the normal component.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-06-26T23:23:04.840" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20017" PostTypeId="1" CreationDate="2015-06-26T23:23:31.777" Score="2" ViewCount="51" Body="&lt;p&gt;I'm currently conducting research into subway paths, but stand in front of a problem, with no programming knowledge and an extremely short time-limit. &#xA;Probably the question I am asking is trivial, but I would be very happy if you could point me in the right direction or to the right forum!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having generated all possible routes a passenger can take underground for a specific city subway system with a starting, transfer and exit station, I now have to find the on-vehicle travel time for each route. I have excel-style data for each single link of the route, but these have to be ordered and added up for each vector of (starting, transfer, exit) to give me the total travel time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take for example the following route: A -&gt; B -&gt; C and I have information on the traveltime for each line, e.g. A4 -&gt; A5, A5 -&gt; A6, A6 -&gt; A, A -&gt; ... -&gt; B and B3 -&gt; B4, B4 -&gt; B5, B5 -&gt; B6, B6 -&gt; ... -&gt;  C, where each k = {An, Bm, C} is a station and A is the starting point, B the transfer point and C the exit point for the route.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I write a program, which goes through the excel observations and adds up all the relevant travel time for each of the routes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With about 50000 routes considered, this can be hugely time-intensive to be done manually.&#xA;Any help on this would be greatly appreciated! If anything is unclear, I will be glad to try and clarify what I mean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best&lt;/p&gt;&#xA;" OwnerUserId="16764" LastActivityDate="2015-06-28T02:36:14.157" Title="Simple Path to Route Algorithm" Tags="|algorithms|data-analysis|data-sets|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20018" PostTypeId="1" AcceptedAnswerId="20019" CreationDate="2015-06-27T01:48:26.510" Score="1" ViewCount="186" Body="&lt;p&gt;I am in the process of writing a finite element code to solve the Navier-Stokes equations using the theta method for time stepping (basically Crank-Nicholson for diffusion and forward Euler for nonlinear terms). To find the pressure I use the pressure correction method. Once I find the pressure I need to update the velocities to include contributions from the pressure gradient. For example I need to compute:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial{u}}{\partial{t}} = -\frac{\partial{p}}{\partial{x}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is the the velocities are defined on quadratic triangular elements (6 points) while the pressure is defined on linear triangular elements (3 points). I think this means I need to interpolate the pressure gradient to the extra three points in the quadratic velocity element when updating the velocity to include the pressure gradient. My question is how do I do this? I was thinking of using the barycentric interpolation method but I am not sure if this is the right thing to do. Also I think this would only be linear interpolation so I am not what you would do if you trying to get a second order solution.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2015-06-27T16:01:27.540" Title="How to update velocity to include pressure when using P2/P1 elements" Tags="|finite-element|interpolation|navier-stokes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20019" PostTypeId="2" ParentId="20018" CreationDate="2015-06-27T14:22:33.770" Score="1" Body="&lt;p&gt;The pressure is represented over the entire element, say element $e$, by its shape functions:&#xA;$$&#xA;p_h^e(x)=\sum_i P_i^e \phi_i^e(x)&#xA;$$&#xA;So, you can simply evaluate it where you need it. If $x_k$ is a point on the velocity element, you simply compute:&#xA;$$&#xA;p_h^e(x_k)=\sum_i P_i^e \phi_i^e(x_k)&#xA;$$&#xA;Now, you've probably used a master element formulation for your integration, so you may need to map the points $x_k$ onto the master element in order to do this, but presumably you've already done this since you needed those points on the master element for your work on the velocity.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-06-27T16:01:27.540" LastActivityDate="2015-06-27T16:01:27.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20020" PostTypeId="1" AcceptedAnswerId="20036" CreationDate="2015-06-27T14:42:27.193" Score="0" ViewCount="682" Body="&lt;p&gt;I wrote a program that calculates the best fit in VBA excel for the following model&#xA;$$&#xA;y_k=c_1x_k+c_0+c_{-1}(x_k)^{-1}&#xA;$$&#xA;solving for the best fit parameters $c_1$, $c_0$, and $c_{-1}$. However I wrote several custom subroutines to accomplish this. One subroutine calculates the moments of $x$ and $y$, another builds the augment matrix from the moments and calculates the matrix inverse solution using a custom row reduced echelon form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there functions I can use in VBA excel that will handle these calculations without passing any data to the excel spreadsheet and can handle negative powers in the regression model?&lt;/p&gt;&#xA;" OwnerUserId="7633" LastActivityDate="2019-08-29T17:12:58.220" Title="Power series regression linear fit in VBA excel" Tags="|performance|regression|data-analysis|efficiency|curve-fitting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20021" PostTypeId="1" CreationDate="2015-06-27T21:03:06.960" Score="3" ViewCount="5050" Body="&lt;p&gt;I am trying to find the most optimized way to perform Matrix Multiplication of very large sizes in C language and under Windows 7 or Ubuntu 14.04. And searching led me to BLAS, LAPACK and ATLAS. However, I couldn't tell which one I can use? I mean BLAS Level 3 DGEMM  seems like the function for my case but I couldn't find out how to use it in C. Can anyone help me with this issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="16771" LastActivityDate="2015-08-20T14:50:38.737" Title="BLAS, LAPACK or ATLAS for Matrix Multiplication in C" Tags="|linear-algebra|matrix|c|lapack|blas|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20022" PostTypeId="2" ParentId="20021" CreationDate="2015-06-27T21:35:51.270" Score="5" Body="&lt;p&gt;Yes, you want to call the BLAS routine DGEMM. The place to start for how to call it from C is to look at the documentation for DGEMM, which you can find online. Then you want to understand how to call FORTRAN routines from C (DGEMM, like all the&#xA;standard BLAS routines has a FORTRAN calling convention). For example, this&#xA;document &lt;a href=&quot;https://computing.llnl.gov/tutorials/bgq/mixedProgramming2.pdf&quot;&gt;https://computing.llnl.gov/tutorials/bgq/mixedProgramming2.pdf&lt;/a&gt; might be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, LAPACK provides higher-level linear algebra operations which call the BLAS routines for their low-level functionality. ATLAS is a high-performance implementation of BLAS. Another high performance open-source implementation of the BLAS is &lt;a href=&quot;http://www.openblas.net/&quot;&gt;http://www.openblas.net/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-06-27T21:35:51.270" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20023" PostTypeId="2" ParentId="19979" CreationDate="2015-06-28T01:07:24.740" Score="1" Body="&lt;p&gt;I am by no means an expert but I will share some challenges that I have heard of. One challenge is (and this may depend on how you chose your c-points) in order to avoid extensive communication between nodes you want your matrix to have a nice structure. For example lets say you are solving a finite element problem. Then you want your nodes to be ordered in such a way so that nodes that are geometrically close together are on the same processor. This will result in a matrix that has some kind of banded structure. This way when you are choosing your c-points every processor does not have to compare with every other processor (to determine which points should be cpoints and which should be f-points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another important issue is what happens as you coarsen your mesh. As you get to coarser and coarser meshes you will have to deal with load imbalancing. For example some processors might end up only containing fpoints by the time you get to very coarse grids. At these grid levels those nodes wont be doing anything. Meanwhile other nodes that might contain many cpoints and will be busy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of this has to do with how you choose your c-point/f-point grids but hopefully this helps. I actually just wrote my own AMG solver and came across the same issue when trying to parallelise. Good luck.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2015-06-28T01:07:24.740" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20024" PostTypeId="2" ParentId="20017" CreationDate="2015-06-28T02:36:14.157" Score="1" Body="&lt;p&gt;A practical complication of such travel time estimates is the time spent transferring from one line/train to another.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all you aim for is the minimum travel time, then there are simple relatively fast algorithms to find one or all of the &quot;optimal&quot; routes.  See &lt;a href=&quot;https://en.wikipedia.org/wiki/Dijkstra&amp;#39;s_algorithm&quot; rel=&quot;nofollow&quot;&gt;Dijkstra's shortest path algorithm&lt;/a&gt; for a single start and destination, or &lt;a href=&quot;https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm&quot; rel=&quot;nofollow&quot;&gt;Floyd-Warshall algorithm&lt;/a&gt; for all pairs of starts and destinations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the mention of &quot;no programming knowledge&quot; and &quot;excel observations&quot; it sounds as if you'd prefer some sort of spreadsheet implementation.  I don't think that is going to prove satisfactory, but perhaps you could find some supporting resources to help you out.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2015-06-28T02:36:14.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20027" PostTypeId="1" CreationDate="2015-06-28T11:02:23.207" Score="0" ViewCount="91" Body="&lt;p&gt;I am trying to predict y(t+1) of a function by using a feedforward neural network with Matlab. The inputs are the previous 3 previous values (y(t-2), y(t-1), y(t)) and the training output is the actual value I want to predict y(t+1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;prediction&quot; seems perfect! The output seems like a perfect copy of the original graph. BUT, the output is shifted. So it doesn't look like it's predicting, but actually taking y(t) (one of my inputs) as the output everytime and it's not predicting but shifted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the graph (the red one is the 'prediction'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/fwE2l.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What causes this behavior?? How can I fix this? I have no idea what's going on, I even tried with NARX and the same thing happens..... any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="16776" OwnerDisplayName="Luis Cruz" LastActivityDate="2016-05-10T13:45:58.897" Title="Feedforward net lagged prediction" Tags="|machine-learning|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20028" PostTypeId="1" AcceptedAnswerId="20040" CreationDate="2015-06-28T20:01:27.003" Score="1" ViewCount="253" Body="&lt;p&gt;I have an infinitely long cylinder defined using&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;radius&lt;/li&gt;&#xA;&lt;li&gt;a point in 3d &lt;/li&gt;&#xA;&lt;li&gt;Axis defined using a 3d vector&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have a set of points with 3d coordinates placed in a grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to wrap this grid of points around the curvature of my cylinder. How to do it &lt;/p&gt;&#xA;" OwnerUserId="16780" LastActivityDate="2015-07-01T15:17:28.327" Title="Wrapping grid of points around curvature of an infinitely long cylinder" Tags="|computational-geometry|" AnswerCount="2" CommentCount="2" ClosedDate="2015-07-02T14:18:21.997" ContentLicense="CC BY-SA 3.0" />
  <row Id="20030" PostTypeId="1" AcceptedAnswerId="20033" CreationDate="2015-06-29T00:56:43.143" Score="19" ViewCount="6234" Body="&lt;p&gt;Is there any (efficient) algorithm to select subset of $M$ points from a set of $N$ points ($M &amp;lt; N$) such that they &quot;cover&quot; most area (over all possible subsets of size $M$)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the points are in 2D plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The naive algorithm is simple, but prohibitive in terms of time complexity:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each subset of N points&#xA;    sum distance between each pair of points in the subset&#xA;    remember subset with the maximum sum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am looking for a more efficient or even approximate method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example, here is a plane with some random points in it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/5E9I4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $M=5$, I expect selecting points like these:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/d7Gnb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the selected points (red) are scattered all over the plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found an article &quot;&lt;a href=&quot;http://lucafoschini.com/papers/Efficiently_ICIP11.pdf&quot; rel=&quot;noreferrer&quot;&gt;EFFICIENTLY SELECTING SPATIALLY DISTRIBUTED KEYPOINTS FOR VISUAL TRACKING&lt;/a&gt;&quot; which is related to this problem. However, this assumes the points are weighted.&lt;/p&gt;&#xA;" OwnerUserId="4591" LastEditorUserId="4591" LastEditDate="2015-06-30T12:10:00.810" LastActivityDate="2018-06-03T23:46:52.937" Title="Selecting most scattered points from a set of points" Tags="|optimization|computational-geometry|" AnswerCount="4" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="20031" PostTypeId="1" AcceptedAnswerId="20032" CreationDate="2015-06-29T04:02:29.317" Score="1" ViewCount="134" Body="&lt;p&gt;I am not sure if my question is on topic but I have a piece of Fortran code that is used to perform successive over relaxation. Prior to performing successive over relaxation the author is calculating the spectrum of the matrix.There is very little documentation so I am just trying to figure out all the details for documentation. &#xA;It appears from the code comments that  the following code calculates matrix spectrum. Would it be possible to get a mathematical background for this code ?&#xA;Here is the equation for the successive over relaxation as requested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linearized second order partial differential equation(Laplacian) that I am trying to solve is this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta_x(A.\Delta_x\psi) + \Delta_y(B.\Delta_y\psi) + \Delta_z(C.\Delta_z\psi) + S = 0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &#xA;$$\Delta_x(A.\Delta_x\psi) = A_{i+1/2,j,k} .(\psi_{i+1,j,k}-\psi_{i,j,k}) -  A_{i-1/2,j,k} .(\psi_{i,j,k}-\psi_{i-1,j,k})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta_y(B.\Delta_y\psi) = B_{i,j+1/2,k} .(\psi_{i,j+1,k}-\psi_{i,j,k}) -  B_{i,j-1/2,k} .(\psi_{i,j,k}-\psi_{i,j-1,k})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta_z(C.\Delta_z\psi) = C_{i,j,k+1/2} .(\psi_{i,j,k+1}-\psi_{i,j,k}) -  A_{i,j,k-1/2} .(\psi_{i,j,k}-\psi_{i,j,k-1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The additive constant  is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ S_{i,j,k} =  \Delta_x .\Delta_y . \Delta_z * \sigma_{i,j,k} $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \psi_i^{(n+1)} = \omega\left( b_i - \sum_{j=1}^{i-1} A_{ij} \psi_j^{(n+1)} - \sum_{j=i}^{m}A_{ij}\psi_j^{(n)}\right) + (1-\omega)\psi_i^{(n)}. $$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  i=nx/2&#xA;  specx=4.*a(i,0,0)/&#xA; &amp;gt;      (2.*a(i,0,0)+b(i,0,0)+b(i,1,0)+c(i,0,0)+c(i,0,1))&#xA;  specy=2.*(b(i,0,0)+b(i,1,0))/&#xA; &amp;gt;      (2.*a(i,0,0)+b(i,0,0)+b(i,1,0)+c(i,0,0)+c(i,0,1))&#xA;  specz=2.*(c(i,0,0)+c(i,0,1))/&#xA; &amp;gt;      (2.*a(i,0,0)+b(i,0,0)+b(i,1,0)+c(i,0,0)+c(i,0,1))&#xA;  do k=1,nz-2&#xA;     do j=1,ny-2&#xA;&#xA;        helpx=4.*a(i,j,k)/&#xA; &amp;gt;            (2.*a(i,j,k)+b(i,j,k)+b(i,j-1,k)+c(i,j,k)+c(i,j,k-1))&#xA;        if (helpx.gt.specx) specx=helpx&#xA;&#xA;        helpy=2.*(b(i,j,k)+b(i,j+1,k))/&#xA; &amp;gt;            (2.*a(i,j,k)+b(i,j,k)+b(i,j-1,k)+c(i,j,k)+c(i,j,k-1))&#xA;        if (helpy.gt.specy) specy=helpy&#xA;&#xA;        helpz=2.*(c(i,j,k)+c(i,j,k+1))/&#xA; &amp;gt;            (2.*a(i,j,k)+b(i,j,k)+b(i,j-1,k)+c(i,j,k)+c(i,j,k-1))&#xA;        if (helpz.gt.specz) specz=helpz&#xA;&#xA;     enddo&#xA;  enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The following code fragments calculate the a,b,c matrices &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  do i=0,nx&#xA;     do j=0,ny&#xA;        do k=0,nz&#xA;           a(i,j,k)=dy*dz*rhoref(2*k)/(dx*coriol(i,j))&#xA;           if (j.lt.ny) then&#xA;              b(i,j,k)=dx*dz*rhoref(2*k)/&#xA; &amp;gt;                     (dy*0.5*(coriol(i,j)+coriol(i,j+1)))&#xA;           else&#xA;              b(i,j,k)=0.&#xA;           endif&#xA;           if (k.lt.nz) then&#xA;              c(i,j,k)=dx*dy*rhoref(2*k+1)*coriol(i,j)/&#xA; &amp;gt;                     (dz*nsq(2*k+1))&#xA;           else&#xA;              c(i,j,k)=0.&#xA;           endif&#xA;        enddo&#xA;     enddo&#xA;  enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user16555" LastEditorDisplayName="user16555" LastEditDate="2016-09-07T15:17:56.293" LastActivityDate="2016-09-07T15:17:56.293" Title="Is this the correct procedure for calculating matrix spectrum?" Tags="|algorithms|numerical-analysis|matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20032" PostTypeId="2" ParentId="20031" CreationDate="2015-06-29T05:23:39.383" Score="3" Body="&lt;p&gt;The spectrum of a matrix is a set of numbers, so since there isn't a set of numbers being computed here, it's almost certainly not the spectrum. Also relevant is that you have three tensors ($a$, $b$, and $c$), so which spectrum do you think it would calculate? Calculating the spectrum of a matrix numerically also cannot be done in a finite number of rational operations and requires an iterative algorithm, unless something very special is known about the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do you think it calculates a spectrum of any kind?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention it does this before performing successive over-relaxation. SOR requires a guess as to the best value of the relaxation parameter, which is related to the &lt;em&gt;spectral norm&lt;/em&gt; of a matrix (see, e.g., &lt;a href=&quot;https://scicomp.stackexchange.com/q/851/713&quot;&gt;Are there any heuristics for optimizing the successive over-relaxation (SOR) method?&lt;/a&gt;). So I guess it's possible it's trying to compute a heuristic estimate of &lt;em&gt;something&lt;/em&gt; related to the spectral norm of the matrix, and hence to the optimal relaxation parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But since there are three different matrices present here, and since we don't know the form of the linear system it's trying to solve, or how it uses the computed values in the solution, it's just impossible to say what it's doing.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-06-29T05:23:39.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20033" PostTypeId="2" ParentId="20030" CreationDate="2015-06-29T05:41:49.597" Score="11" Body="&lt;p&gt;Here is an approximate solution. Since &lt;strong&gt;&lt;em&gt;N&lt;/em&gt;&lt;/strong&gt; is so large and &lt;strong&gt;&lt;em&gt;M&lt;/em&gt;&lt;/strong&gt; is so small, how about the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute the convex hull of &lt;strong&gt;&lt;em&gt;N&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Select up to &lt;strong&gt;&lt;em&gt;M&lt;/em&gt;&lt;/strong&gt; points from the hull that satisfy your maximum distance criteria.&lt;/li&gt;&#xA;&lt;li&gt;If Step 2 leaves you with fewer than &lt;strong&gt;&lt;em&gt;M&lt;/em&gt;&lt;/strong&gt; points then select 1 point from the interior that maximizes its distance from the previously selected points.&lt;/li&gt;&#xA;&lt;li&gt;Repeat Step 3 until the number of selected points is &lt;strong&gt;&lt;em&gt;M&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The intuition behind it is that since &lt;strong&gt;&lt;em&gt;N&lt;/em&gt;&lt;/strong&gt; &gt;&gt; &lt;strong&gt;&lt;em&gt;M&lt;/em&gt;&lt;/strong&gt;, and you want points as a far away from each other as possible, they will likely be close to the edges of the data, so you might as well start with the hull and then iteratively work your way in from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, by starting with the hull, you reduce your initial search from N to N&lt;sup&gt;1/2&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If steps 3 and 4 above are taking too long (since you're iteratively testing the interior of your dataset) two more ideas occurred to me to speed up your problem.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Randomized Search&lt;/strong&gt;: Say you found &lt;strong&gt;&lt;em&gt;P&lt;/em&gt;&lt;/strong&gt; points on the hull in Step 2. Then randomly draw &lt;strong&gt;&lt;em&gt;M&lt;/em&gt;&lt;/strong&gt;-&lt;strong&gt;&lt;em&gt;P&lt;/em&gt;&lt;/strong&gt; points from the interior. Select the best set after X trials.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Simulated Annealing&lt;/strong&gt;: Compute the smallest bounding box that covers your dataset (doesn't have to be aligned with the axes, could be tilted). Then define a set of &lt;strong&gt;&lt;em&gt;M&lt;/em&gt;&lt;/strong&gt; uniformly distributed grid points on that bounding box. Note, these points do not necessarily coincide with any of your dataset points. Then for each grid point find the &lt;strong&gt;&lt;em&gt;k&lt;/em&gt;&lt;/strong&gt;-nearest neighbors in your dataset. Run through every &lt;strong&gt;&lt;em&gt;M&lt;/em&gt;&lt;/strong&gt; x &lt;strong&gt;&lt;em&gt;k&lt;/em&gt;&lt;/strong&gt; combination and select the one that satisfies your maximum distance criteria. In other words, you are using the initial grid as a bootstrap to find a &lt;em&gt;good&lt;/em&gt; initial solution.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9813" LastEditorUserId="27569" LastEditDate="2018-06-03T23:46:52.937" LastActivityDate="2018-06-03T23:46:52.937" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20036" PostTypeId="2" ParentId="20020" CreationDate="2015-06-29T18:01:50.060" Score="1" Body="&lt;p&gt;I doubt that such a built-in function exists in MS Excel. Nevertheless, this problem is a linear regression that is simple enough to solve analytically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us start with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Pi = \sum_{i=1}^{n} \left[y_i - \left(a x_i + b + \frac{c}{x_i}\right)\right]^2 \enspace $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; the number of data pairs and &lt;span class=&quot;math-container&quot;&gt;$(a,b,c)\equiv (c_1, c_0, c_{-1})$&lt;/span&gt;. To find the minimum of this approximation we compute the partial derivatives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial \Pi}{\partial a} = 0\\&#xA;\frac{\partial \Pi}{\partial b} = 0\\&#xA;\frac{\partial \Pi}{\partial c} = 0 \enspace ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;obtaining the system of equations&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sum x_i y_i = a\sum x_i^2 + b\sum x_i + c\sum 1\\&#xA;\sum y_i = a\sum x_i + b\sum 1 + c\sum \frac{1}{x_i}\\&#xA;\sum \frac{y_i}{x_i} = a\sum 1 + b\sum \frac{1}{x_i} + c\sum \frac{1}{x_i^2} \enspace .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we simplify the notation with&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;&amp;amp;m_1 = \sum x_i y_i, &amp;amp;m_2=\sum y_i,\quad &amp;amp;m_3 = \sum\frac{y_i}{x_i},\\&#xA;&amp;amp;\gamma_1 = \sum x_i^2, &amp;amp;\gamma_2 = \sum x_i,\quad &amp;amp;\gamma_3 = \sum \frac{1}{x_i},\\&#xA;&amp;amp;\gamma_4 = \sum \frac{1}{x_i^2},  &amp;amp; &amp;amp;&#xA;\end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the system of equations can be written as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{pmatrix}{\gamma}_{1} &amp;amp; {\gamma}_{2} &amp;amp; n\\&#xA; {\gamma}_{2} &amp;amp; n &amp;amp; {\gamma}_{3}\\&#xA; n &amp;amp; {\gamma}_{3} &amp;amp; {\gamma}_{4}\end{pmatrix}&#xA;\begin{pmatrix} a\\ b\\ c\end{pmatrix} = &#xA;\begin{pmatrix} m_1\\ m_2\\ m_3\end{pmatrix} \enspace .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the solution is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;a &amp;amp;= \frac{1}{\Delta}[m_3 n^2 + \left(-m_1\gamma_4 - m_2\gamma_3\right)n + \gamma_2\left(m_2\gamma_4 - m_3\gamma_3\right) + m_1\gamma_3^2]\\&#xA;b &amp;amp;= \frac{1}{\Delta}[m_2 n^2 + \left(-m_1\gamma_3 - \gamma_2 m_3\right)n + \gamma_1\left(m_3\gamma_3 - m_2\gamma_4\right) + m_1\gamma_2\gamma_4]\\&#xA;c &amp;amp;= \frac{1}{\Delta}[m_1 n^2 + \left(-\gamma_1 m_3 - m_2\gamma_2\right)n - m_1\gamma_2\gamma_3 + \gamma_1 m_2\gamma_3 + \gamma_2^2 m_3]&#xA;\end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$\Delta = n^3+\left(-\gamma_1\gamma_4 - 2\gamma_2 \gamma_3\right)n + \gamma_2^2 \gamma_4 +\gamma_1 \gamma_3^2$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2019-08-29T17:12:58.220" LastActivityDate="2019-08-29T17:12:58.220" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20038" PostTypeId="2" ParentId="8998" CreationDate="2015-06-29T18:53:17.700" Score="1" Body="&lt;p&gt;R is a functional language &amp;amp; also a statistics (&amp;amp; now Machine learning) language and actually the number 1 language for statistics. It's not a HPC language though: it's not used for traditional &quot;number crunching&quot;  like physics simulations etc. But it can be made to run on massive clusters (e.g. via MPI) for massive statistics simulations (MCMC) of machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica is also a functional language but it's core domain is symbolic computing rather than numerical computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Julia you can also program in a functional style (next to procedural &amp;amp; their flavor of OO (multi-dispatch)) but it's not pure (the base data structures are all mutable (except for tuples), although there are some libraries with immutable functional data structures. More importantly, it's much slower than the procedural style so it's not used much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't call Scala a functional language but rather a object-functional hybrid. You can use many functional concepts in Scala. Scala is important for Cloud computing because of Spark (&lt;a href=&quot;https://spark.apache.org/&quot; rel=&quot;nofollow&quot;&gt;https://spark.apache.org/&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that modern Fortran has actually some elements of functional programming: it has strict pointer semantics (unlike C), you can have pure (no side effect) functions (&amp;amp; mark it as such) and you can have immutability. It even has smart indexing where you can specify conditions for matrix indices. This is query like and normally only found in high level language like R of LINQ in C# or via higher order filter functions in functional languages. So Fortran isn't so bad at all, it even has some pretty modern features (e.g. co-arrays) not found in many languages. In fact in future versions of Fortran I'd rather see more functional features added rather than OO-features (which is now usually the case) because OO in Fortran is really awkward and ugly.&lt;/p&gt;&#xA;" OwnerUserId="16794" LastEditorUserId="16794" LastEditDate="2015-06-29T19:30:15.413" LastActivityDate="2015-06-29T19:30:15.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20040" PostTypeId="2" ParentId="20028" CreationDate="2015-06-29T22:34:41.427" Score="2" Body="&lt;p&gt;It seems like you are trying to &lt;a href=&quot;https://en.wikipedia.org/wiki/Vector_projection&quot; rel=&quot;nofollow&quot;&gt;project&lt;/a&gt; the grid points onto the cylindrical surface. You can do this via a few vector projections.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Let &lt;strong&gt;&lt;em&gt;r&lt;/em&gt;&lt;/strong&gt; be the cylinder radius&lt;/li&gt;&#xA;&lt;li&gt;Let &lt;strong&gt;&lt;em&gt;P&lt;/em&gt;&lt;/strong&gt; be a grid point.&lt;/li&gt;&#xA;&lt;li&gt;Let &lt;strong&gt;&lt;em&gt;a&lt;/em&gt;&lt;/strong&gt; be the cylinder axis unit vector.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Project &lt;strong&gt;&lt;em&gt;P&lt;/em&gt;&lt;/strong&gt; along &lt;strong&gt;&lt;em&gt;a&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA; &lt;b&gt;&lt;i&gt;P&lt;sub&gt;a&lt;/sub&gt; = (P &amp;middot; a) * a &lt;/i&gt;&lt;/b&gt;&#xA;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute projection of grid point perpendicular to axis&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA; &lt;b&gt;&lt;i&gt;P&lt;sub&gt;&amp;perp;&lt;/sub&gt; = P - P&lt;sub&gt;a&lt;/sub&gt; &lt;/i&gt;&lt;/b&gt;&#xA;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute unit vector of perpendicular projection&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA; &lt;b&gt;&lt;i&gt;p&lt;sub&gt;&amp;perp;&lt;/sub&gt; = P&lt;sub&gt;&amp;perp;&lt;/sub&gt; / ||P&lt;sub&gt;&amp;perp;&lt;/sub&gt;|| &lt;/i&gt;&lt;/b&gt;&#xA;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the projection of the grid point on the cylinder&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA; &lt;b&gt;&lt;i&gt;P&lt;sub&gt;cyl&lt;/sub&gt; = P&lt;sub&gt;a&lt;/sub&gt; + r * p&lt;sub&gt;&amp;perp;&lt;/sub&gt; &lt;/i&gt;&lt;/b&gt;&#xA;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;P&lt;sub&gt;cyl&lt;/sub&gt;&lt;/em&gt;&lt;/strong&gt; is the value you want. Compute it for each grid point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you are moving up the cylinder axis until you reach the grid point in that direction, then you move toward the grid point until you hit the cylinder, then you stop.&lt;/p&gt;&#xA;" OwnerUserId="9813" LastEditorUserId="9813" LastEditDate="2015-07-01T15:17:28.327" LastActivityDate="2015-07-01T15:17:28.327" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20041" PostTypeId="2" ParentId="19769" CreationDate="2015-06-29T23:06:56.703" Score="0" Body="&lt;p&gt;I don't know of a standard name for the problem as you describe it but you can use a standard data structure to solve the problem: &lt;a href=&quot;https://en.wikipedia.org/wiki/Interval_tree&quot; rel=&quot;nofollow&quot;&gt;interval tree&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can sequentially insert each interval into the tree, but only insert those parts of the interval that are disjoint from any intervals already inserted in the tree. That way as you build the tree you are guaranteed that only mutually exclusive intervals are in the tree. Interval trees are designed to quickly search for overlapping intervals so all this searching should not be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an algorithm to do it:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Let &lt;strong&gt;T&lt;/strong&gt; be an interval tree (initially empty)&lt;/li&gt;&#xA;&lt;li&gt;Let &lt;strong&gt;L&lt;/strong&gt; be the list of intervals&lt;/li&gt;&#xA;&lt;li&gt;Pop an interval &lt;strong&gt;I&lt;/strong&gt; from &lt;strong&gt;L&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Query &lt;strong&gt;T&lt;/strong&gt; for a list of intervals &lt;strong&gt;O&lt;/strong&gt; overlapping &lt;strong&gt;I&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Intersect &lt;strong&gt;I&lt;/strong&gt; with each interval in &lt;strong&gt;O&lt;/strong&gt;. For each non-empty intersection remove that intersection from &lt;strong&gt;I&lt;/strong&gt; (this will make &lt;strong&gt;I&lt;/strong&gt; very disjoint) and update the event count of the interval in &lt;strong&gt;O&lt;/strong&gt; with the event count from &lt;strong&gt;I&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Add any remaining non-overlapping intervals from &lt;strong&gt;I&lt;/strong&gt; to &lt;strong&gt;T&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;If &lt;strong&gt;L&lt;/strong&gt; is non-empty go to Step 3 else go to Step 8&lt;/li&gt;&#xA;&lt;li&gt;Query &lt;strong&gt;T&lt;/strong&gt; for all the intervals.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The final step gives you your list of mutually exclusive intervals with the appropriate event counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, the way you update the event count will require some experimenting. For example, you may only want to update the event count in the overlapping intervals of &lt;strong&gt;O&lt;/strong&gt; with a proportion of the event count in &lt;strong&gt;I&lt;/strong&gt; that overlapped and not the raw count.&lt;/p&gt;&#xA;" OwnerUserId="9813" LastActivityDate="2015-06-29T23:06:56.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20042" PostTypeId="2" ParentId="20028" CreationDate="2015-06-30T00:38:39.227" Score="0" Body="&lt;p&gt;It might be easiest to first place the cylinder in some standard position,&#xA;e.g., base on the $xy$-plane and axis along $z$, generate the grid of points&#xA;as $(r\cos \theta, r\sin \theta, z)$, and then translate &amp;amp; rotate &#xA;both cylinder and points out&#xA;to wherever it should reside:&#xA;&lt;hr /&gt;&#xA;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &#xA;&lt;img src=&quot;https://i.stack.imgur.com/33y3L.jpg&quot; alt=&quot;CylinderGrid&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr /&gt;&#xA;" OwnerUserId="673" LastActivityDate="2015-06-30T00:38:39.227" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20043" PostTypeId="1" CreationDate="2015-06-30T01:13:14.660" Score="2" ViewCount="353" Body="&lt;p&gt;I am interested in a reference in the literature that discusses the performance of Dense Linear Algebra (blas routines) and dense linear algebra (sparse blas routines).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in knowing for what combinations of size and density sparse routines outperform dense routines.  I am mainly interested in a shared memory computer.  Does anyone know of a reference?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="16799" LastActivityDate="2015-06-30T01:13:14.660" Title="Sparse Linear Algebra vs Dense Linear Algebra" Tags="|linear-algebra|sparse-matrix|blas|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20044" PostTypeId="2" ParentId="20030" CreationDate="2015-06-30T05:35:36.403" Score="6" Body="&lt;p&gt;With a very large number $N$ of points and a small subset $M$ to be chosen, it may be helpful to consider what is known about continuous versions of the problem in two-dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L. Fejes Tóth (&quot;On the sum of distances determined by a pointset&quot;, Acta Math.&#xA;Acad. Sci. Hungar., 7:397–401, 1956) showed that the set of $M$ points on a circle which maximizes the sum of pairwise distances is achieved by vertices of a regular $M$-gon inscribed on the circle.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;He subsequently (L. Fejes Tóth, &quot;Über eine Punktverteilung auf der Kugel&quot;, Acta Math. Acad. Sci. Hungar., 10:13-19, 1959) posed the more difficult problem of maximizing the sum of pairwise distances for $M$ points in the plane whose diameter (maximum pairwise distance) is $1$.  This problem remains open in general, although Friedrich Pillichshammer has given an upper bound and shown it to be sharp for $M=3,4,5$ (&lt;a href=&quot;http://link.springer.com/article/10.1023%2FA%3A1022838328562&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;On extremal point distributions in the Euclidean plane&quot;&lt;/a&gt;, Acta Mathematica Hungarica,  98(4):311-321, 2003).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These few cases suggest that the points of such extremal distributions will tend to occur on the periphery of a region.  For $M=3$ the solution is an equilateral triangle with edge length $1$.  For $M=4$ three of the points again form an equilateral triangle and the fourth point is located at the midpoint of a circular arc through two of the points, centered on the third point.  For $M=5$ the solution is a regular pentagon of diameter $1$.  None of these present a &quot;scattering&quot; of points through the interior of a figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we wish to avoid predominating selection of points at the periphery, a different objective is apt to prove useful.  The maximization of the minimum distance between points is such a criterion.  Related problems have been broached at &lt;a href=&quot;https://stackoverflow.com/questions/11000271/find-the-set-s-which-maximize-the-minimum-distance-between-points-in-s-union-a&quot;&gt;StackOverflow&lt;/a&gt;, at &lt;a href=&quot;https://cs.stackexchange.com/questions/22767/choosing-a-subset-to-maximize-the-minimum-distance-between-points&quot;&gt;Computer Science SE&lt;/a&gt;, at &lt;a href=&quot;https://math.stackexchange.com/questions/718733/maximizing-minimum-distance-between-points-placed-in-a-polygon&quot;&gt;Math.SE&lt;/a&gt;, and at &lt;a href=&quot;https://mathoverflow.net/questions/166042/finding-a-point-maximizing-the-minimal-distance-to-a-set-of-points&quot;&gt;MathOverflow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some insight into why this approach yields points interior to a figure, consider its rough equivalence to &lt;a href=&quot;https://en.wikipedia.org/wiki/Circle_packing&quot; rel=&quot;nofollow noreferrer&quot;&gt;packing $M$ circles&lt;/a&gt; of diameter $D$ inside a figure.  The $M$ centers are then points no two of which are closer than distance $D$.  The picture in &lt;a href=&quot;https://math.stackexchange.com/a/790324/3111&quot;&gt;this Math.SE Answer&lt;/a&gt; will likely be worth a glance, showing how to best arrange ten points in a square.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-06-30T05:35:36.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20045" PostTypeId="1" AcceptedAnswerId="20050" CreationDate="2015-06-30T08:42:15.630" Score="0" ViewCount="196" Body="&lt;p&gt;Given 2 vectors $\mathbf{u}$ and $\mathbf{v}$ the following are equivalent:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt; $\mathbf{u}\cdot\mathbf{v}$&#xA;&lt;li&gt; $\mathbf{u}^T \mathbf{v}$&#xA;&lt;li&gt; $u_i v_i$&#xA;&lt;li&gt; $v_i u_i$&#xA;&lt;li&gt; $\mathbf{v}\cdot\mathbf{u}$&#xA;&lt;li&gt; $\mathbf{v}^T\mathbf{u}$&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given a second order tensor (matrix) $\mathbf{A}$ and a vector $\mathbf{b}$, are the following also equivalent?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt; $\mathbf{A}\cdot\mathbf{b}$&#xA;&lt;li&gt; $\mathbf{A} \mathbf{b}$&#xA;&lt;li&gt; $A_{ij} b_j$&#xA;&lt;li&gt; $b_j A_{ij}$&#xA;&lt;li&gt; $\mathbf{b}\cdot\mathbf{A}$&#xA;&lt;li&gt; $\mathbf{b}^T\mathbf{A}^T$&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I know that the last one (and possibly the second last one) gives a row vector instead of a column vector. Is it possible to tell in index notation whether a vector is a row or column vector, or is that supposed to be clear based on context?&lt;/p&gt;&#xA;" OwnerUserId="9728" LastEditorUserId="9728" LastEditDate="2015-06-30T08:48:24.957" LastActivityDate="2015-06-30T23:52:58.317" Title="Vector and index notation equivalence" Tags="|vector|tensor|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20046" PostTypeId="2" ParentId="20045" CreationDate="2015-06-30T15:16:43.913" Score="0" Body="&lt;p&gt;No.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Non-standard notation as noted by @WolfgangBangerth&lt;/li&gt;&#xA;&lt;li&gt;Column vector&lt;/li&gt;&#xA;&lt;li&gt;Same as 2&lt;/li&gt;&#xA;&lt;li&gt;Same as 3&lt;/li&gt;&#xA;&lt;li&gt;Non-standard notation&lt;/li&gt;&#xA;&lt;li&gt;Row vector&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9813" LastActivityDate="2015-06-30T15:16:43.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20050" PostTypeId="2" ParentId="20045" CreationDate="2015-06-30T23:52:58.317" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to tell in index notation whether a vector is a row or column vector, or is that supposed to be clear based on context?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems the answer is actually lurking in your question itself here: &#xA;if $u \cdot v \equiv u^T v$ then that doesn't leave much wiggle room. $u^T$ will have to be a row vector in order for the resulting product to be a scalar quantity - and as we all know, the dot product of two vectors is a scalar. The other scenario would give you a $n$ x $n$ system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extending this logic to the case of $v^T u$ we see that both $v$ and $u$ are indeed column vectors - this is something I have observed is usually true regarding vector representation in literature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now coming to the second part of your question, as Wolfgang has already mentioned, the &lt;strong&gt;dot&lt;/strong&gt; notation between a matrix and a vector is not standard notation. However, if you must use it I think you will have to follow the guidelines of indicial notation in which a dot product essentially refers to a contraction, i.e. $A \cdot b \equiv A_{ij} b_j$. Which is once again a column vector (say $\mathbf{X}$). So there are some equivalencies in your second list, yes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the last case, if you consider $(A b)^T = \mathbf{X}^T$, and apply the standard identity to the LHS you get $b^T A^T = \mathbf{X}^T$. So really, you are getting a row vector when you go out of your way to take a transpose of your entire system, which - if you think about it - is quite intuitive. Strictly speaking, your vectors are still column vectors, it is just that you have &lt;em&gt;chosen&lt;/em&gt; to take their transpose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Non standard notation, but equals $A_{ij} b_j$&lt;/li&gt;&#xA;&lt;li&gt;Equivalent to (1) as per convention&lt;/li&gt;&#xA;&lt;li&gt;Equivalent to (1)&lt;/li&gt;&#xA;&lt;li&gt;Equivalent to (3) as at this point, $A_{ij}$ and $b_j$ are merely scalars and their order is interchangeable. Also, note that $i$ is a &lt;em&gt;dummy&lt;/em&gt; index here. &lt;/li&gt;&#xA;&lt;li&gt;I'm not sure what this yields but my best guess is $b_i A_{ij}$ which is NOT equivalent to the above. &lt;/li&gt;&#xA;&lt;li&gt;Transpose of solution to (1-4)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="15103" LastActivityDate="2015-06-30T23:52:58.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20052" PostTypeId="1" AcceptedAnswerId="20097" CreationDate="2015-07-01T07:29:23.157" Score="-1" ViewCount="392" Body="&lt;p&gt;I am trying to reproduce &lt;a href=&quot;http://www.math.umn.edu/~othmer/papers/Tang:1995:EOW.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tang &amp;amp; Othmer&lt;/a&gt; paper which is related to excitations and oscillations in G-protein model in &lt;em&gt;Dictyostelium discoideum&lt;/em&gt;, an amoeba species. The mathematical model in the paper is like below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XD8AH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $w_1,w_2,w_3,w_4,w_5,u_1,u_2,u_4$ are biochemicals and other parameters are constants associated with &lt;em&gt;Dictyostelium discoideum&lt;/em&gt;, $sr(.)$ is secretion rate function. &#xA;I am supposed to simulate a situation in a square mesh where cells are uniformly distributed and interact each other. $w_5$ is the extracellular cAMP (cyclic AMP) which diffuses out from the amoeba cells and creates spiral wave forms in space under some conditions. So I basically simulated this using Euler method in MATLAB. But I am unable to generate spirals in simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can someone give me some idea to generate $w_5$ spirals in the mesh?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the results (for 5 minutes) considering a single cell and no diffusion content. (Just to show that I have reproduced some results from paper)&#xA;&lt;img src=&quot;https://i.stack.imgur.com/9YkKm.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One 2D solution looks like below (from the paper) : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/bDbzK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my attempt to this problem is unsuccessful. What I did to initiate output is first define every cells with an initial value set for variables like $w_1, w_2 ...$ etc except a patch where those values are $180°$ shifted (&lt;code&gt;imagesc&lt;/code&gt; plot of $w_5$ shown in the figure) . I selected these initial values from the ODE plots. But this arrangement is proving to be unsuccessful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/W6mF5.png&quot; alt=&quot;enter image description here&quot;&gt;         &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt; : Now I improved my code so that output is bounded between 2 values. Its not blowing up any more. I can provide my code if necessary. Here is the &lt;a href=&quot;https://youtu.be/JIS9Y5s6scQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; to a video output I created.&lt;/p&gt;&#xA;" OwnerUserId="5003" LastEditorUserId="5003" LastEditDate="2015-07-06T06:59:38.193" LastActivityDate="2018-05-29T01:35:43.767" Title="How to initiate spirals in this model?" Tags="|matlab|finite-difference|simulation|computational-biology|" AnswerCount="1" CommentCount="19" ContentLicense="CC BY-SA 3.0" />
  <row Id="20054" PostTypeId="1" AcceptedAnswerId="20072" CreationDate="2015-07-01T16:04:22.113" Score="2" ViewCount="3800" Body="&lt;p&gt;I'm trying to implement 1D advection solver using WENO and ENO schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial u}{\partial t} &#xA;+ \frac{\partial f(u)}{\partial x}   =0 &#xA;\end{equation} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&#xA;\begin{equation}&#xA;f(u) = C\,u&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discretized as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{u^{n+1}_j-u^{n}_j}{\Delta t} &#xA;+ \frac{F_{j+1/2}-F_{j-1/2}}{\Delta x}  =0&#xA;\end{equation} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've translated &lt;a href=&quot;http://www.cfm.brown.edu/people/jansh/page5/page51/page52/page52.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jan S Hesthaven&lt;/a&gt; Matlab implementations into Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how we call the &lt;code&gt;ENO&lt;/code&gt; and &lt;code&gt;WENO&lt;/code&gt; routines &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import *&#xA;nx = 81&#xA;dx = 2./(nx-1)&#xA;x = linspace(0,2,nx)&#xA;nt = 25    &#xA;dt = .02  &#xA;c = 1.      #assume wavespeed of c = 1&#xA;u = zeros(nx)      #numpy function ones()&#xA;u[.5/dx : 1/dx+1]=2  #setting u = 2 between 0.5 and 1 as per our I.C.s&#xA;k = 3 # number of weights Order= 2*k-1&#xA;gc = k-1 #number of ghost cells &#xA;#adding ghost cells &#xA;gcr=x[-1]+linspace(1,gc,gc)*dx&#xA;gcl=x[0]+linspace(-gc,-1,gc)*dx&#xA;xc = append(x,gcr)&#xA;xc = append(gcl,xc)&#xA;uc = append(u,u[-gc:])&#xA;uc = append(u[0:gc],uc)&#xA;&#xA;for n in range(1,nt):  &#xA;    un = uc.copy() &#xA;    for i in range(1,nx): &#xA;        xloc = xc[i-(k-1):i+k]&#xA;        floc = c*uc[i-(k-1):i+k]&#xA;        #f_left,f_right = ENO(xloc,floc,k)&#xA;        f_left,f_right = WENO(xloc,floc,k)        &#xA;        uc[i] = un[i]-dt/dx*(f_right-f_left)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally either for WENO or ENO I get the same initial condition &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/g4e1U.png&quot; alt=&quot;IC&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what we should get using a naive Euler integration. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for n in range(1,nt):  &#xA;    un = u.copy() &#xA;    for i in range(1,nx): &#xA;        u[i] = un[i]-c*dt/dx*(un[i]-un[i-1]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since $\Delta x = (n_t * \Delta t) C = (25*0.02)*1 = 0.5$ the wave displaces 0.5 units to the right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4oDXM.png&quot; alt=&quot;Naive Euler&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I use the fluxes gotten from WENO and ENO for a simple 1st order Euler integration of 1D advection equation? or Do you have an implementation of 1D Advection using WENO or ENO schemes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the way I'm doing it, but the fluxes ($F_{j+1/2}=$ &lt;code&gt;f_right&lt;/code&gt; and $F_{j-1/2}=$&lt;code&gt;f_left&lt;/code&gt;) are too small to produce any update. It seems I need to use upwinding for the fluxes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{j}^{n+1} = u_j^{n}-\frac{\Delta t}{\Delta x}(F_{j+1/2}-F_{j-1/2})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s.: For reference only, these are the ENO and WENO routines I'm using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def ENOweights(k,r):&#xA;    #Purpose: compute weights c_rk in ENO expansion &#xA;    # v_[i+1/2] = \sum_[j=0]^[k-1] c_[rj] v_[i-r+j]&#xA;    #where k = order and r = shift &#xA;&#xA;    c = zeros(k)&#xA;&#xA;    for j in range(0,k):&#xA;            de3 = 0.&#xA;            for m in range(j+1,k+1):&#xA;                #compute denominator &#xA;                de2 = 0.&#xA;                for l in range(0,k+1):&#xA;                    #print 'de2:',de2&#xA;                    if l is not m:&#xA;                        de1 = 1.&#xA;                        for q in range(0,k+1):&#xA;                            #print 'de1:',de1&#xA;                            if (q is not m) and (q is not l):&#xA;                                de1 = de1*(r-q+1)&#xA;&#xA;&#xA;                        de2 = de2 + de1&#xA;&#xA;&#xA;                #compute numerator &#xA;                de1 = 1.&#xA;                for l in range(0,k+1):&#xA;                    if (l is not m):&#xA;                        de1 = de1*(m-l)&#xA;&#xA;                de3 = de3 + de2/de1&#xA;&#xA;&#xA;            c[j] = de3&#xA;&#xA;&#xA;    return c&#xA;&#xA;def nddp(X,Y):&#xA;    #Newton's divided difference table &#xA;    #the input are two vectors X and Y that represent points &#xA;&#xA;    n = len(X)&#xA;&#xA;    DD = zeros((n,n+1))&#xA;&#xA;    #inserting x into 1st column of DD-table &#xA;    DD[:,0]=X&#xA;&#xA;    #inserting y into 2nd column of DD-table&#xA;    DD[:,1]=Y&#xA;&#xA;    #creates divided difference coefficients &#xA;    #e.g: D[0,0] = (Y[1]-Y[0])/(X[1]-X[0])&#xA;&#xA;    for j in range(0,n-1):&#xA;        for k in range(0,n-j-1): #j goes from 0 to n-2&#xA;            DD[k,j+2]= (DD[k+1,j+1]-DD[k,j+1])/(DD[k+j+1,0]-DD[k,0])&#xA;&#xA;    return DD&#xA;&#xA;def ENO(xloc, uloc, k):&#xA;    #Purpose: compute the left and right cell interface values using an ENO &#xA;    #Approach based on 2k-1 long vectors uloc with cell k &#xA;&#xA;    #treat special case of k=1 - no stencil to select &#xA;    if (k==1):&#xA;        ul = uloc[0]&#xA;        ur = uloc[0]&#xA;&#xA;    #Apply ENO procedure &#xA;    S = zeros(k,dtype=int)&#xA;    S[0] = k&#xA;    for kk in range (0,k-1):&#xA;        #print 'S:',S&#xA;        #left stencil&#xA;        xvec = zeros(k)&#xA;        uvec = zeros(k)&#xA;        Sindxl = append(S[0]-1, S[0:kk+1])-1&#xA;        xvec = xloc[Sindxl]&#xA;        uvec = uloc[Sindxl]&#xA;        DDl = nddp(xvec,uvec)&#xA;        Vl = abs(DDl[0,kk+2])&#xA;&#xA;        #right stencil &#xA;        xvec = zeros(k)&#xA;        uvec = zeros(k)&#xA;        Sindxr = append(S[0:kk+1], S[kk]+1)-1&#xA;        xvec = xloc[Sindxr]&#xA;        uvec = uloc[Sindxr]&#xA;        DDr = nddp(xvec,uvec)&#xA;        Vr = abs(DDr[0,kk+2])&#xA;&#xA;        #choose stencil through divided differences &#xA;        if (Vr&amp;gt;Vl):&#xA;            #print 'Vr&amp;gt;Vl'&#xA;            S[0:kk+2] = Sindxl+1&#xA;        else:&#xA;            S[0:kk+2] = Sindxr+1&#xA;&#xA;    #Compute stencil shift 'r'&#xA;    r = k - S[0]&#xA;&#xA;    #Compute weights for stencil &#xA;    cr = ENOweights(k,r)&#xA;    cl = ENOweights(k,r-1)&#xA;&#xA;    #Compute cell interface values &#xA;    ur = 0 &#xA;    ul = 0 &#xA;    for i in range(0,k):&#xA;        ur = ur + cr[i]*uloc[S[i]-1]&#xA;        ul = ul + cl[i]*uloc[S[i]-1]&#xA;&#xA;    return (ul,ur)&#xA;&#xA;def WENO(xloc, uloc, k):&#xA;    #Purpose: compute the left and right cell interface values using ENO &#xA;    #approach based on 2k-1 long vectors uloc with cell k &#xA;&#xA;    #treat special case of k = 1 no stencil to select &#xA;    if (k==1):&#xA;        ul = uloc[0]&#xA;        ur = uloc[1]&#xA;&#xA;    #Apply WENO procedure &#xA;    alphal = zeros(k)&#xA;    alphar = zeros(k)&#xA;    omegal = zeros(k)&#xA;    omegar = zeros(k)&#xA;    beta = zeros(k)&#xA;    d = zeros(k)&#xA;    vareps= 1e-6&#xA;&#xA;    #Compute k values of xl and xr based on different stencils &#xA;    ulr = zeros(k)&#xA;    urr = zeros(k)&#xA;&#xA;    for r in  range(0,k):&#xA;        cr = ENOweights(k,r)&#xA;        cl = ENOweights(k,r-1)&#xA;&#xA;        for i in range(0,k):&#xA;            urr[r] = urr[r] + cr[i]*uloc[k-r+i-1] &#xA;            ulr[r] = ulr[r] + cl[i]*uloc[k-r+i-1] &#xA;&#xA;&#xA;    #setup WENO coefficients for different orders -2k-1&#xA;    if (k==2):&#xA;        d[0]=2/3.&#xA;        d[1]=1/3.&#xA;        beta[0] = (uloc[2]-uloc[1])**2&#xA;        beta[1] = (uloc[1]-uloc[0])**2&#xA;&#xA;&#xA;    if(k==3):&#xA;        d[0] = 3/10. &#xA;        d[1] = 3/5.&#xA;        d[2] = 1/10.&#xA;        beta[0] = 13/12.*(uloc[2]-2*uloc[3]+uloc[4])**2 + 1/4.*(3*uloc[2]-4*uloc[3]+uloc[4])**2&#xA;        beta[1] = 13/12.*(uloc[1]-2*uloc[2]+uloc[3])**2 + 1/4.*(uloc[1]-uloc[3])**2&#xA;        beta[2] = 13/12.*(uloc[0]-2*uloc[1]+uloc[2])**2 + 1/4.*(3*uloc[2]-4*uloc[1]+uloc[0])**2&#xA;&#xA;    #compute alpha parameters&#xA;    for r in range(0,k):&#xA;        alphar[r] = d[r]/(vareps+beta[r])**2&#xA;        alphal[r] = d[k-r-1]/(vareps+beta[r])**2&#xA;&#xA;    #Compute WENO weights parameters&#xA;    for r in range(0,k):&#xA;        omegal[r] = alphal[r]/alphal.sum()&#xA;        omegar[r] = alphar[r]/alphar.sum()&#xA;&#xA;    #Compute cell interface values&#xA;    ul = 0 &#xA;    ur = 0 &#xA;    for r in range(0,k):&#xA;        ul = ul + omegal[r]*ulr[r]&#xA;        ur = ur + omegar[r]*urr[r]&#xA;&#xA;    return (ul,ur)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11656" LastEditorUserId="11656" LastEditDate="2015-07-03T15:47:49.253" LastActivityDate="2015-07-03T15:48:51.327" Title="Implementation of 1D Advection in Python using WENO and ENO schemes" Tags="|pde|python|parabolic-pde|advection|numerical-modelling|" AnswerCount="1" CommentCount="8" ClosedDate="2015-07-03T22:21:47.397" ContentLicense="CC BY-SA 3.0" />
  <row Id="20056" PostTypeId="1" CreationDate="2015-07-01T17:08:53.457" Score="4" ViewCount="269" Body="&lt;p&gt;Given a set $Q$ of $n$ points, we want to find the subset $S_\max \subset Q$ of $k$ elements that maximize the total distance between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S_\max = \max_S \sum_{\substack{&#xA;i,j\in S\\&#xA;i \neq j}} d(x_i,x_j)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where in my case $x_i$ is a boolean vector and the distance considered is the Manhattan/Hamming distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any efficient way to solve this problem? Is it possible to rewrite it in another simpler way?&lt;/p&gt;&#xA;" OwnerDisplayName="Barbus" LastEditorUserId="9667" LastEditDate="2015-07-01T18:37:19.753" LastActivityDate="2017-02-21T03:18:09.877" Title="Find the set of K elements between n that maximize the total distance" Tags="|optimization|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="20057" PostTypeId="1" CreationDate="2015-07-01T19:40:17.580" Score="1" ViewCount="283" Body="&lt;p&gt;I have a system of differential equations with some unknown parameters and I need to find the optimal parameter set that fits best with the data I have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each parameter set choice (using for loops), my code solves the system of DE numerically, then extracts from the time vector a time subvector that agrees with data time values. The next step requires to extract a new output subvector for each of the variables that match with the time subvector (in order to compute the error).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how my code is looking:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;OO=[];&#xA;y2=(y(:,2))';&#xA;t1=t';&#xA;FF=[];&#xA;y1=(y(:,1))';&#xA;&#xA;global &quot;parameters and other global variables&quot;&#xA;&#xA;for parameter_1&#xA;    for parameter_2&#xA;        [t,y]=ode45('system',[time interval], [initial conditions]);&#xA;        time; % &quot;time&quot; is a code that extracts the time subvector&#xA;&#xA;        for i=1:length(time)&#xA;            for j=1:length(t1)&#xA;                if time(i)==t1(j);&#xA;                    FF(end+1) = y1(j);&#xA;                end&#xA;            end&#xA;        end&#xA;&#xA;        for k=1:length(time)&#xA;            for m=1:length(t1)&#xA;                if time(k)==t1(m);&#xA;                    OO(end+1) = y2(m);&#xA;                end&#xA;            end&#xA;        end&#xA;&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At this point the code is not over, but I would be expecting it to return the &lt;code&gt;OO&lt;/code&gt; vector and the &lt;code&gt;FF&lt;/code&gt; vector for the last parameter set choice of the loop. However when the code runs (without any errors) it produces  vectors &lt;code&gt;OO&lt;/code&gt; and &lt;code&gt;FF&lt;/code&gt; that are longer than vector time: &lt;code&gt;length(time) &amp;lt; {length(OO),length(FF)}&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16815" LastEditorUserId="9667" LastEditDate="2015-07-01T21:18:52.970" LastActivityDate="2015-09-30T14:53:27.307" Title="Curve Fitting problem MATLAB" Tags="|matlab|curve-fitting|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20058" PostTypeId="1" AcceptedAnswerId="20059" CreationDate="2015-07-01T23:41:49.627" Score="3" ViewCount="840" Body="&lt;p&gt;I need to invert a matrix. Of course, I'm not the first person in this situation, and I know that there's a wealth of powerful libraries out there, of which I only know a couple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, there is a twist: I know that matrix inversion is (in general) numerically unstable, and I need a solution that is absolutely failsafe, if need be at the cost of large amounts of computing time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrices in question will be relatively moderate in size (not exceeding 100x100 entries), and it's really important to get it right. If the computation takes a couple of minutes on a regular desktop computer, that is completely acceptable. The reason for this is that (a) the matrix inversion will only happen once in the execution of the program, and (b) the analytic solution will only be used as an optional replacement for the already implemented numeric one (mainly for running cross-checks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not think that it matters for the issue at hand, but the matrices will be relatively sparse (10-50% of entries different from zero), and relatively well-behaved (entries in the range 0.001-1000 expected, but nearly identical rows are highly probable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution will be used in C++ code that will be distributed freely, so it is necessary to use either an open (BSD, LGPL, ...) library that can be distributed alongside what I'm doing to a small audience of scientific users (or is ideally already installed on most linux systems), or to implement something from scratch. In that case, a clean, short and less convoluted implementation is to be given preference over an optimized one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im happy about:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pointers to existing libraries that I could use for this&lt;/li&gt;&#xA;&lt;li&gt;names or outlines of algorithms that could be useful&lt;/li&gt;&#xA;&lt;li&gt;additional ideas or considerations that will help me pursue this endeavour (i.e. will I need to define a custom arbitrary-precision-float type?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Edit&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;For illustration purposes, I have added a 15x15 matrix and it's inverse using unsatisfactory numeric inversion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the input matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{matrix}&#xA;              0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;    0.01513 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;   0.003782 \\&#xA;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     0.0625 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     0.0625 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;    0.01538 &amp;amp;          0 \\&#xA;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     0.0625 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     0.01537 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;   0.003782 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;           0 &amp;amp;          0 &amp;amp;   0.003782 \\&#xA;     0.0625 &amp;amp;          0 &amp;amp;          0 &amp;amp;     0.0625 &amp;amp;          0 &amp;amp;          0 &amp;amp;     0.0625 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;     0.0625 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;     0.0625 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;     0.0625 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;    0.01537 &amp;amp;          0 &amp;amp;          0 &amp;amp;    0.01537 &amp;amp;          0 &amp;amp;           0 &amp;amp;    0.01538 &amp;amp;          0 \\&#xA;     0.0625 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;     0.0625 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;     0.0625 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;    0.01537 &amp;amp;          0 &amp;amp;          0 &amp;amp;    0.01537 &amp;amp;          0 &amp;amp;          0 &amp;amp;     0.01537 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;    0.01513 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;   0.003782 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;       0.25 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;     0.0625 &amp;amp;          0 &amp;amp;   0.003782 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;    0.01537 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;       0.25 &amp;amp;    0.01513 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     0.0615 &amp;amp;          0 &amp;amp;          0 &amp;amp;           0 &amp;amp;          0 &amp;amp;          0 \\&#xA;\end{matrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the numeric inverse:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{matrix}&#xA;         -0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          4 &amp;amp;          0 &amp;amp;          0 \\&#xA;         -0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;       66.1 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;         -0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;      264.4 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;         16 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;  2.935e^{-17} &amp;amp;         -0 &amp;amp; -1.805e^{-15} &amp;amp;          0 &amp;amp;          0 &amp;amp; -2.037e^{-31} &amp;amp; -7.338e^{-18} &amp;amp; -1.907e^{-31} &amp;amp;  4.513e^{-16} &amp;amp;  3.944e^{-31} &amp;amp;          0 \\&#xA;         -0 &amp;amp;      264.4 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;       66.1 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;        -16 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;         16 &amp;amp;          0 &amp;amp;          0 &amp;amp;  1.835e^{-15} &amp;amp; -4.441e^{-16} &amp;amp;  1.718e^{-15} &amp;amp;         -4 &amp;amp; -3.553e^{-15} &amp;amp;          0 \\&#xA;          0 &amp;amp;     -264.4 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;      264.4 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;      -66.1 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;      -66.1 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;      264.4 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     -264.4 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;      65.04 &amp;amp;         -0 &amp;amp;      65.04 &amp;amp;         -0 &amp;amp;     -65.04 &amp;amp;          0 &amp;amp;     -65.04 &amp;amp;         -0 &amp;amp;     -65.04 &amp;amp;          0 &amp;amp;      65.04 &amp;amp; -7.338e^{-15} &amp;amp;      16.26 &amp;amp; -6.872e^{-15} &amp;amp;      16.26 &amp;amp;  1.421e^{-14} &amp;amp;     -16.26 \\&#xA;      16.26 &amp;amp;          0 &amp;amp;         -0 &amp;amp;      65.04 &amp;amp;     -65.04 &amp;amp;          0 &amp;amp;     -65.04 &amp;amp;          0 &amp;amp;         -0 &amp;amp;     -65.04 &amp;amp;      65.04 &amp;amp;          0 &amp;amp;      65.04 &amp;amp;          0 &amp;amp;          0 &amp;amp;      16.26 &amp;amp;     -65.04 &amp;amp;          0 \\&#xA;       66.1 &amp;amp;      264.4 &amp;amp;     -264.4 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;     -264.4 &amp;amp;     -264.4 &amp;amp;      264.4 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;      264.4 &amp;amp;       66.1 &amp;amp;     -264.4 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     -16.26 &amp;amp;          0 &amp;amp;     -16.26 &amp;amp;          0 &amp;amp;      16.26 \\&#xA;          0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     -65.04 &amp;amp; -7.105e^{-15} &amp;amp;  1.421e^{-14} &amp;amp;     -16.26 &amp;amp;      65.04 &amp;amp;  3.553e^{-15} \\&#xA;          0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;         -0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;     -264.4 &amp;amp;      -66.1 &amp;amp;      264.4 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;          0 &amp;amp;     -65.04 &amp;amp;         -0 &amp;amp;     -65.04 &amp;amp;         -0 &amp;amp;      65.04 &amp;amp;          0 &amp;amp; -1.193e^{-16} &amp;amp;          0 &amp;amp;  7.338e^{-15} &amp;amp;          0 &amp;amp;          0 &amp;amp;   8.28e^{-31} &amp;amp;  2.983e^{-17} &amp;amp;  7.498e^{-31} &amp;amp; -1.835e^{-15} &amp;amp; -1.578e^{-30} &amp;amp;          0 \\&#xA;     -16.26 &amp;amp;          0 &amp;amp;         -0 &amp;amp;     -65.04 &amp;amp;      65.04 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;      -66.1 &amp;amp;     -264.4 &amp;amp;      264.4 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 &amp;amp;          0 \\&#xA;\end{matrix}$&lt;/p&gt;&#xA;" OwnerUserId="16818" LastEditorUserId="123" LastEditDate="2015-07-04T04:44:35.520" LastActivityDate="2015-07-04T04:44:35.520" Title="Exact analytical matrix inversion of sparse 100x100 matrices in C++" Tags="|matrix|computer-arithmetic|inverse|" AnswerCount="3" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="20059" PostTypeId="2" ParentId="20058" CreationDate="2015-07-02T04:28:36.333" Score="2" Body="&lt;p&gt;This may be more of a comment than an answer, but I can't comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes on arbitrary precision. No on symbolic inverse, because at some point, you have to numerically evaluate it, and it is likely to be extremely numerically unstable. Even for 6 by 6 matrix, the symbolic inverse is starting to get pretty ungainly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which leaves me the main item. Why do you &quot;need&quot; the matrix inverse? Quite frequently, there is a numerically stable and accurate (and possibly faster)  calculation which avoids the need to invert a matrix when a matrix inverse id &quot;needed&quot;.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2015-07-02T04:28:36.333" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20060" PostTypeId="1" CreationDate="2015-07-02T07:47:24.620" Score="1" ViewCount="53" Body="&lt;p&gt;I am setting up a simple model that uses recursion to iterate between known constraints - relaxation, in other words. I need a spreadsheet that allows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A0 50&#xA;&#xA;A1 =(A0+A2)/2&#xA;&#xA;A2 =(A1+A3)/2&#xA;&#xA;A3 =(A2+A4)/2&#xA;&#xA;A4 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It should be simple, but OpenOffice and Apple Numbers specifically forbid this. If you know of configuration setting to override this, that would be great too.&lt;/p&gt;&#xA;" OwnerUserId="16820" LastEditorUserId="9667" LastEditDate="2015-07-02T14:07:26.073" LastActivityDate="2015-07-02T14:07:26.073" Title="Relaxation - spreadsheet solution to recursive algorithm" Tags="|numerical-modelling|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20061" PostTypeId="2" ParentId="20058" CreationDate="2015-07-02T08:26:48.587" Score="1" Body="&lt;p&gt;You may want to check &lt;em&gt;division-free Gaussian elimination&lt;/em&gt; as described, e.g., in this &lt;a href=&quot;https://www.google.com/url?q=http://www.dtic.mil/cgi-bin/GetTRDoc%3FAD%3DADA313755&amp;amp;sa=U&amp;amp;ei=2fKUVcffDMKqsAHg35ugBA&amp;amp;ved=0CAUQFjAA&amp;amp;client=internal-uds-cse&amp;amp;usg=AFQjCNG2BKpQH17G8pWT0VI69siTZnjDCg&quot; rel=&quot;nofollow&quot;&gt;document&lt;/a&gt; titled &quot;A simplified fraction-free integer Gauss elimination algorithm&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been told that these approaches are precise until you eventually evaluate the solution (up to loss of significance, which, however, you can monitor). They rather fail because of overflow, which you can handle using appropriate data types.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2015-07-02T08:26:48.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20063" PostTypeId="2" ParentId="20057" CreationDate="2015-07-02T13:57:19.147" Score="1" Body="&lt;p&gt;This is a more general nonlinear optimization problem than would usually be described as simply &quot;curve fitting&quot;. I suggest you approach it as such. You will need to define a few things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The parameters you are optimizing.&lt;/li&gt;&#xA;&lt;li&gt;The quantity to be minimized.&lt;/li&gt;&#xA;&lt;li&gt;Helper functions to simplify things.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Matlab has tools that can help and it will help you to break the problem into a couple parts as follows:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Main script&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;&#xA;% load the curve and time values you want the ODE to fit&#xA;[ tReference, yReference ] = loadKnownValues();&#xA;&#xA;% load the initial conditions for the ODE:&#xA;initialConditions = loadInitialConditions();&#xA;&#xA;% initial guess for parameter values (a column vector)&#xA;parametersGuess = [ 1; 6.83; ... ];&#xA;&#xA;% define an objective function for the optimization&#xA;% I will assume you want to minimize the sum of the errors squared&#xA;% lsqnonlin is in the Optimization toolbox&#xA;%&#xA;% You could also use fminsearch. In that case you would want to use the&#xA;% objective function F = @(param) sum( objectiveFunction(param).^2 );&#xA;objectiveFunction = @(parameters) yReference ...&#xA;    - solveODE(tReference,initialConditions,parameters);&#xA;&#xA;optimizedParameters = lsqnonlin(objectiveFunction,parametersGuess);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This main script relies on a few other functions that do specific things. Obviously you will need to load in the curve you're trying to fit and the initial conditions for the ODE (&lt;code&gt;loadKnownValues()&lt;/code&gt; and &lt;code&gt;loadInitialConditions()&lt;/code&gt;). You will also need a function that solves the ODE given a set of initial conditions and parameters and returns the values at the desired times. Those first two should be easy, the last one can be done as follows:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;solveODE.m&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function yVals = solveODE(tReference,initialConditions,parameters)&#xA;% yVals = solveODE(tReference,initialConditions,parameters)&#xA;% Solves the ODE for your problem and with the given initialConditions&#xA;% and parameters and returns the values at the specified times in&#xA;% tReference. The first time in tReference must correspond to the&#xA;% initial time.&#xA;&#xA;% function that returns the right hand side of the ODE,&#xA;% use the parameters in the input vector &quot;parameters&quot;&#xA;rightHandSide = @(t,y) ... ;&#xA;&#xA;[~, yVals] = ode45(rightHandSide, tReference, initialConditions);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obviously I can't comment on the convergence or performance of the optimization without knowing a lot more about the system, but this should get you started.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-07-02T13:57:19.147" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20064" PostTypeId="1" AcceptedAnswerId="20082" CreationDate="2015-07-02T14:01:12.053" Score="1" ViewCount="223" Body="&lt;p&gt;Lets say in Graph $G$ we have two vertices $v$ and $u$, each vertex is connected to several neighbors by edges describing the distances $d_{ij}$ from these neighbors. The neighbors themselves are also connected to more vertices by such edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a vertex has relatively short edges and is connected to neighbors that also have relatively short edges we could say it is in a dense area, the opposite is also true; if a vertex has mostly long edges to neighbors that also have mostly long edges, it's in a relatively sparse area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a way to measure this sparsity / density and be able to compares this attribute between different vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence, we have as input, two lists of scalars (vertexs' edges' lengths) and possibly direct neighbors edges' lengths. We need to output the density measure and be able to compare it between vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm doing now is using a trimmed mean and simply computing the ratio between densities but perhaps there is a more robust approach.&lt;/p&gt;&#xA;" OwnerUserId="16822" LastActivityDate="2015-07-06T01:30:18.460" Title="Measure the differences in vertices density in a graph?" Tags="|graph-theory|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20065" PostTypeId="1" CreationDate="2015-07-02T14:28:31.363" Score="0" ViewCount="22" Body="&lt;p&gt;Lets say we have two things as input, $N$ scalars (measurements) that we know are erroneous to some degree (i.e. the correct values are &lt;em&gt;somewhat&lt;/em&gt; similar). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, we also have a roughly more reliable order $O$ among the measurements. Using the ordering:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$O = &amp;lt;u_1, u_2, u_3 ... u_n&amp;gt;$ we can deduct that with some known probability, $p &amp;gt; 0.5$ the ordering is correct, $[i \geq j \rightarrow u_i \geq u_j]$ is true. Now, how would you then adjust the scalar values accordingly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Creating a new set of values, cloning the original scalars.&lt;/li&gt;&#xA;&lt;li&gt;Making the minimal needed changes in order to satisfy the order. i.e. slightly increasing or decreasing the original values until we get a corrctly ordered $\{v_1, v_2, v_3 ... v_n\}$ such that $\sum\limits_{i=1}^n (u_i - v_i)^2$ is minimal.&lt;/li&gt;&#xA;&lt;li&gt;Doing a weighted average between the newly generated values and the original ones, using $p$ as the weight for the new value.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;While $p$ is unknown, we can guesstimate it from data that is manually inspected.&lt;/p&gt;&#xA;" OwnerUserId="16822" LastEditorUserId="16822" LastEditDate="2015-07-04T13:26:56.883" LastActivityDate="2015-07-04T13:26:56.883" Title="Assigning new values based on original guesstimates and ranking / ordering?" Tags="|error-estimation|probability|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20066" PostTypeId="1" AcceptedAnswerId="20068" CreationDate="2015-07-02T17:48:58.870" Score="1" ViewCount="390" Body="&lt;p&gt;I am interested in computing only the generalized singular values, and was wondering if this was faster (and by how much?) than computing the full GSVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, I was wondering what the fastest algorithms are for computing the generalized &lt;strong&gt;&lt;em&gt;singular values&lt;/em&gt;&lt;/strong&gt; (ideally the pairs $(\alpha_i, \beta_i)$ instead of just $\sigma_i = \alpha_i/\beta_i$), given two matrices $A$ and $B$, where $A$ is of size $m\times k$ and $B$ is of size $n\times k$? I was also wondering what the run-times of these algorithms are in terms of $n, m$, and $k$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, are there any implementations of these algorithms out there? I know Matlab has a &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/gsvd.html&quot; rel=&quot;nofollow&quot;&gt;GSVD function&lt;/a&gt;, which can compute only the singular values, using the command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sigma = gsvd(A,B)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I doubt from the way it is implemented that it is faster than doing the full GSVD using the command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[U,V,X,C,S] = gsvd(A,B)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But, maybe I am wrong. Would there potentially be a way in Matlab (or in another language) to just compute $C$ and $S$, and not $U$, $V$, and $X$? If so, how much faster would that be than computing the full GSVD?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An answer to any/all of these questions would be appreciated, or even a relevant reference (it has been hard to find any since most of the algorithms I have seen compute the full GSVD, which is not what I am looking for).&lt;/p&gt;&#xA;" OwnerUserId="16826" LastEditorUserId="16826" LastEditDate="2015-07-02T19:57:43.337" LastActivityDate="2015-07-02T22:49:50.020" Title="Fast algorithms for computing only the generalized singular values (but not the vectors)" Tags="|linear-algebra|matlab|algorithms|matrix|svd|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20067" PostTypeId="1" CreationDate="2015-07-02T19:07:35.970" Score="3" ViewCount="1206" Body="&lt;p&gt;I am currently writing a small Python script to estimate the pulse duration from the optical spectrum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end, the idea is to observe the effects of the spectral phase on the pulse duration and shape. But for now, I cannot retrieve a well-known result without taking into account spectral phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, a time-bandwidth limited Gaussian pulse with an optical spectrum of 30 nm width centered at 800 nm has a duration of around 31 fs. This is a basic result for a rule of thumb. You can check the result &lt;a href=&quot;http://www.rp-photonics.com/gaussian_pulses.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://www.ape-berlin.de/de/page/calculator&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sadly, my Python script, using &lt;code&gt;numpy/matplotlib&lt;/code&gt;, gives me a pulse duration which is twice too large:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import power, linspace, sqrt, log, exp, abs, max, conj&#xA;from numpy.fft import fft, fftshift, fftfreq&#xA;from scipy.constants import c&#xA;&#xA;from matplotlib import pyplot as plt&#xA;&#xA;# Number of points for the computations&#xA;n_points = 2**9&#xA;&#xA;# Spectral width and center in wavelength (nm)&#xA;wl_center = 800e-9&#xA;wl_width = 30e-9&#xA;&#xA;# Conversion of spectral parameters in frequency (s-1)&#xA;freq_center = c/wl_center&#xA;freq_width = wl_width*c/power(wl_center, 2)&#xA;&#xA;# Building of the x axes&#xA;frequencies = linspace(freq_center - freq_width*8, freq_center + freq_width*8, n_points)&#xA;wavelengths = c/frequencies&#xA;&#xA;# Building the Gaussian spectrum according to the parameters&#xA;diff_freq = frequencies - freq_center&#xA;sig_freq = freq_width/(sqrt(8*log(2)))&#xA;spectrum_freq_abs = exp(-(power(diff_freq, 2))/(2*power(sig_freq, 2)))&#xA;&#xA;# FFT&#xA;step = frequencies[1] - frequencies[0]&#xA;time = fftshift(fftfreq(n_points, d=step))&#xA;field = fftshift(fft(spectrum_freq_abs))&#xA;&#xA;# Plotting&#xA;# The commented line plot the spectrum vs the wavelength. The second line plot the pulse.&#xA;# The second line plot the optical pulse envelope vs the time in fs. It is twice too large (60 fs fwhm instead of 30 fs.)&#xA;# plt.plot(wavelengths*1e9, spectrum_freq_abs)&#xA;plt.plot(time*1e15, abs(field*conj(field)))&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I checked the &lt;em&gt;time&lt;/em&gt; array created thanks to the &lt;code&gt;fftfreq&lt;/code&gt; function of &lt;code&gt;numpy&lt;/code&gt;. It seems to be good according to the &lt;em&gt;frequency&lt;/em&gt; array size and step. So I don't think it's an x axis problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot find where I am wrong in this small and easy Python script, which is very frustrating. Maybe the community could give me an hint somewhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: This is a repost from &lt;a href=&quot;https://physics.stackexchange.com/questions/192291/my-calculated-laser-pulse-duration-is-twice-too-large-where-am-i-wrong&quot;&gt;physics community&lt;/a&gt;, where I've been kindly redirected to scicomp.&lt;/p&gt;&#xA;" OwnerUserId="16828" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2019-06-30T17:21:10.923" Title="My calculated laser pulse duration is too large. Where am I wrong?" Tags="|python|numpy|fourier-transform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20068" PostTypeId="2" ParentId="20066" CreationDate="2015-07-02T21:18:13.623" Score="2" Body="&lt;p&gt;Per &lt;a href=&quot;http://www.nag.com/numeric/fl/manual/pdf/F08/f08msf.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.nag.com/numeric/fl/manual/pdf/F08/f08msf.pdf&lt;/a&gt; , regarding the LAPACK routine in the NAG Library for complex singular values/singular vectors. (Not generalized, but I think the idea is basically the same for generalized. See &lt;a href=&quot;http://www.nag.com/numeric/fl/manual/pdf/F08/f08vaf.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.nag.com/numeric/fl/manual/pdf/F08/f08vaf.pdf&lt;/a&gt; for real GSVD and &lt;a href=&quot;http://www.nag.com/numeric/fl/manual/pdf/F08/f08vnf.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.nag.com/numeric/fl/manual/pdf/F08/f08vnf.pdf&lt;/a&gt; for complex GSVD, but they don't have discussion of computational effort/)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;08MSF (CBDSQ/ZBDSQR) computes the singular value decomposition of a&#xA;  complex general matrix which has been reduced to bidiagonal form.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;F08MSF (CBDSQR&#xA;  / ZBDSQR) uses two different algorithms. If any singular vectors are required (that is, if NCVT &gt; 0 or NRU &gt; 0 or NCC &gt; 0), the bidiagonal QR algorithm is used, switching between zero- shift and implicitly shifted forms to preserve the accuracy of small&#xA;  singular values, and switching between QR and QL variants in order to&#xA;  handle graded matrices effectively (see Demmel and Kahan (1990)).  If&#xA;  only singular values are required (that is, if NCVT = NRU = NCC = 0),&#xA;  they are computed by the differential qd algorithm (see Fernando and&#xA;  Parlett (1994)), which is faster and can achieve even greater&#xA;  accuracy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The total number of real floating-point operations is roughly&#xA;  proportional to n^2 if only the singular values are computed.  About&#xA;  12 n^2 * nru additional operations are required to compute the left&#xA;  singular vectors and about 12 n^^2 * ncvt to compute the right singular&#xA;  vectors.  The operations to compute the singular values must all be&#xA;  performed in scalar mode; the additional operations to compute the&#xA;  singular vectors can be vectorized and on some machines may be&#xA;  performed much faster. The real analogue of this routine is F08MEF(SBDSQR/DBDSQR).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See Demmel and Kahan 1990 &quot;Accurate Singular Values of Bidiagonal Matrices&quot; &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn03.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lapack/lawnspdf/lawn03.pdf&lt;/a&gt; for a discussion of why singular values plus vectors takes longer than singular values only. It is  because singular vectors converge more slowly than singular values. Table 2 shows some timing results comparing with and without singular vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some quick timing results I ran using MATLAB R2014A WIN64 on an 8 core machine&quot;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; n=4000;A=randn(n);B=randn(n);tic,sigma = gsvd(A,B);toc,tic,[U,V,X,C,S] = gsvd(A,B);toc&#xA;Elapsed time is 31.649874 seconds.&#xA;Elapsed time is 33.202460 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A few other runs give similar results, and the no singular vector version was still faster by about the same amount when it was done second (checked in case the processor was getting overheated and had to decrease turbo level for second calculation).  Multiple cores were used by MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With n = 1000; the no singular vector version averaged about 8% longer than with singular vectors. I have no idea why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not saying that randn of a square matrix for both A and B is representative of problems you care about, but you can try yourself on your own A and B.  &lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2015-07-02T22:49:50.020" LastActivityDate="2015-07-02T22:49:50.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20069" PostTypeId="2" ParentId="20058" CreationDate="2015-07-03T05:03:22.403" Score="3" Body="&lt;p&gt;There are a number of symbolic algebra packages that will compute the exact matrix inverse; e.g. Sage, Mathematica, Maple, or Maxima.  This becomes computationally expensive for large matrices, but there is a chance that a sparse $100 \times 100$ matrix would be within reach.  I wouldn't rule it out without trying, since it's so easy to try.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-07-03T05:03:22.403" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20070" PostTypeId="1" CreationDate="2015-07-03T08:56:09.250" Score="8" ViewCount="3235" Body="&lt;p&gt;To give you some context,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently implementing a simple finite element solver in Julia. I am getting run-times that are 70% of a Matlab code. (Both codes are essential equivalent in structure.) I've run some profiles on my Julia code and I find that most of the run-time is taken up by the linear system solver (essentially the &quot;\&quot; operator in $K \backslash F$ where $K$ is a sparse symmetric positive definite matrix and $F$ is a vector.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried looking up efficient solvers in Julia. I've come across the MUMPS package but I haven't used it yet. (I'm planning on giving it a go soon and will update this question when I do.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More interestingly I came across this thread on the julia-users &lt;a href=&quot;https://groups.google.com/forum/#!topic/julia-users/FagaqWVkBGU&quot; rel=&quot;nofollow&quot;&gt;google-group&lt;/a&gt; which has a lot of content on how Matlab implements its (very efficient) solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what &lt;em&gt;is&lt;/em&gt; the most efficient way of implementing a linear solver in Julia? Does the backslash operator choose the most efficient solution algorithm?&lt;/p&gt;&#xA;" OwnerUserId="14657" LastEditorUserId="14657" LastEditDate="2015-07-14T04:32:41.643" LastActivityDate="2016-01-31T08:28:00.210" Title="Solving Linear Systems in Julia" Tags="|linear-algebra|finite-element|linear-solver|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20071" PostTypeId="1" CreationDate="2015-07-03T14:40:07.350" Score="7" ViewCount="240" Body="&lt;p&gt;I have a PDE optimization problem, and a scalar field (which I am optimizing over) is supposed to be nonnegative everywhere in the domain. Since I am working in Fourier space for solving this problem numerically, I need to convert this non-negative constraint into Fourier domain (i.e. constraint on Fourier coefficients).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the solution is smooth, what is best way to go about it?&#xA;I really can't think of anything more complicated than just choosing a (large?) grid of points in space, say $\mathbf{x_l}, l=1,2...,M$, and adding M linear constraints of the type:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(\mathbf{x_l})=\sum_{j=1}^N\hat f_\mathbf{j}e^{i2\pi \mathbf{j}.\mathbf{x_l}}\geq 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the smoothness of $f$ will ensure that this suffices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any other ideas? Thoughts on this?&lt;/p&gt;&#xA;" OwnerUserId="16835" LastEditorUserId="713" LastEditDate="2015-07-08T00:37:24.677" LastActivityDate="2015-07-08T02:57:47.967" Title="Enforcing non-negative constraint in fourier-spectral method" Tags="|pde|constrained-optimization|fourier-analysis|spectral-method|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20072" PostTypeId="2" ParentId="20054" CreationDate="2015-07-03T15:40:35.867" Score="3" Body="&lt;p&gt;The confusion was the misleading variables $F_{j-1/2}$ and $F_{j+1/2}$ with &lt;code&gt;f_left&lt;/code&gt; and &lt;code&gt;f_right&lt;/code&gt;, which are completely different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;f_left&lt;/code&gt; and &lt;code&gt;f_right&lt;/code&gt; are the interpolated fluxes at a one single face. They must be then upwinded using the advection speed to compute the Flux at a specific cell face. Which means if $C&amp;gt;0$ we take &lt;code&gt;f_left&lt;/code&gt;, otherwise we take &lt;code&gt;f_right&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gs = zeros((nx+2*gc,nt))&#xA;flux = zeros(nx+2*gc)&#xA;&#xA;for n in range(1,nt):  &#xA;    un = uc.copy() &#xA;    for i in range(gc,nx-1+gc): #i=2&#xA;        xloc = xc[i-(k-1):i+k] #i+k-1-(i-(k-1)-1) = 2k -1 &#xA;        uloc = uc[i-(k-1):i+k]&#xA;        f_left,f_right = ENO(xloc,uloc,k)&#xA;        #f_left,f_right = WENO(xloc,uloc,k)&#xA;        #upwind flux&#xA;        flux[i]=0.5*(c+fabs(c))*f_left + 0.5*(c-fabs(c))*f_right&#xA;&#xA;    for i in range(gc,nx-gc):&#xA;        if c&amp;gt;0:&#xA;            uc[i] = un[i]-dt/dx*(flux[i]-flux[i-1])&#xA;        else:&#xA;            uc[i] = un[i]-dt/dx*(flux[i+1]-flux[i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally this is the advected scalar&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/aYl0G.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11656" LastEditorUserId="11656" LastEditDate="2015-07-03T15:48:51.327" LastActivityDate="2015-07-03T15:48:51.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20073" PostTypeId="1" CreationDate="2015-07-03T17:16:13.090" Score="3" ViewCount="327" Body="&lt;p&gt;I tried using Python's matplotlib on the logarithm and here is what I got, a kind of starburst pattern.  Since the angle jumps between $\theta = 0$ and $\theta = 2\pi$, contour assumes there is a contour line between those those two points.  Here in in fact $\bbox[#EEEDAA,1pt]{0=2\pi}$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps instead of &lt;code&gt;matplotlib.pyplot.contour&lt;/code&gt; the contour lines can be connected with &lt;code&gt;meshgrid&lt;/code&gt; and &lt;code&gt;numpy.where&lt;/code&gt; statements.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Contour Plot of $\log z$ on $z \in [-1,1] + i[-1,1]$&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;%matplotlib inline&#xA;plt.rcParams['figure.figsize']=4,4&#xA;&#xA;plt.contour(&#xA;    np.arange(-1,1,0.01),&#xA;    np.arange(-1,1,0.01), &#xA;    np.angle(t[...,None] + 1j*t[None,...]) , &#xA;    levels = 2*np.pi*np.arange(-1,1,0.025)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/f4rGn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Contour Lines Accumulate at the Negative Imaginary Axis $-i \, \mathbb{R}_{\geq 0}$&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t = np.arange(-1,1,0.025)&#xA;plt.contour(t,t, np.angle(t[...,None] + 1j*t[None,...]) , levels = 2*np.pi*np.arange(-1,1,0.025))&#xA;&#xA;L = 0.25&#xA;plt.xlim([-L,L])&#xA;plt.ylim([-L,L])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/J4VqT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Code Inspection&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import inspect&#xA;print inspect.getsource(plt.contour)&#xA;&#xA;@_autogen_docstring(Axes.contour)&#xA;def contour(*args, **kwargs):&#xA;    ax = gca()&#xA;    # allow callers to override the hold state by passing hold=True|False&#xA;    washold = ax.ishold()&#xA;    hold = kwargs.pop('hold', None)&#xA;    if hold is not None:&#xA;        ax.hold(hold)&#xA;    try:&#xA;        ret = ax.contour(*args, **kwargs)&#xA;        draw_if_interactive()&#xA;    finally:&#xA;        ax.hold(washold)&#xA;    if ret._A is not None: sci(ret)&#xA;    return ret&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print inspect.getsource(plt.Axes.contour)&#xA;&#xA;def contour(self, *args, **kwargs):&#xA;    if not self._hold:&#xA;        self.cla()&#xA;    kwargs['filled'] = False&#xA;    return mcontour.QuadContourSet(self, *args, **kwargs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally one can call on the function containing the actual source code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib&#xA;print inspect.getsource(matplotlib.contour.QuadContourSet)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2015-07-03T17:16:13.090" Title="matplotlib contourplot for $\log z$ in the Complex Plane $\mathbb{C}$" Tags="|finite-difference|numerical-analysis|numpy|mesh-generation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20074" PostTypeId="1" CreationDate="2015-07-03T21:36:02.253" Score="2" ViewCount="153" Body="&lt;p&gt;I am interested in proving or obtaining a counterexample to the following conjecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\Omega\in \mathbb{R}^d$ be a bounded open domain.  Let $u_d\in H^{1/2}(\partial\Omega) \times \mathbb{R}^+$. Suppose $u\in L^2((0,\infty); H^1(\Omega))$ is the weak solution of &#xA;$$&#xA;\left\{&#xA;\begin{align*}&#xA;  u_t- \nabla \cdot \nabla u &amp;amp;= 0&amp;amp;\,  \mathrm{in}\,&amp;amp; (\Omega\times \mathbb{R}^+) \\&#xA;  u &amp;amp;= u_d&amp;amp;\, \mathrm{on}\,&amp;amp; (\partial\Omega \times \mathbb{R}^+)\\&#xA;  u &amp;amp;= 0&amp;amp;\, \mathrm{at}&amp;amp;\, (\Omega \times \{0\})&#xA;\end{align*}&#xA;\right.&#xA;$$&#xA;Then there exists a constant $C$ dependent only on the domain $\Omega$ such that&#xA;$&#xA;\left\| u \right\|_{L^\infty((0,\infty); L^2(\Omega))} \le C \left\| u_d \right\|_{L^{\infty}((0,\infty); H^{1/2}(\partial\Omega))}.&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't been able to find many texts that treat the heat equation with time varying boundary conditions.  Thanks so much for any direction you can give me.  Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addendum:  Following @WolfgangBangerth's advice we have the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step One, convert non-homogeneous boundary conditions to a right hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $u_{BC}\in H^1(\Omega) \times \mathbb{R}^+$ be the weak solution to the following elliptic problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \left\{&#xA;\begin{align*}&#xA;-\nabla \cdot \nabla u_{BC} &amp;amp;= 0 &amp;amp; \mathrm{in}&amp;amp; (\Omega \times \mathbb{R}^+)\\&#xA; u_{BC} &amp;amp;=u_d &amp;amp;\mathrm{on} &amp;amp;(\partial\Omega \times \mathbb{R}^+) &#xA;\end{align*}&#xA; \right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So $u_{BC}$ is the weak solution of an elliptic equation at each instant in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now define $w = u-u_{BC}$.  Then we have that $w$ is the weak solution of the following homogeneous initial boundary value problem.&#xA;$$&#xA;\left\{&#xA;\begin{align*}&#xA; w_t - \nabla \cdot \nabla w &amp;amp;= -(u_{BC})_t&amp;amp; \mathrm{in}&amp;amp; (\Omega\times \mathbb{R}^+)\\&#xA; w &amp;amp;= 0&amp;amp; \mathrm{on}&amp;amp; (\partial\Omega \times \mathbb{R}^+)\\ &#xA;w &amp;amp;= -u_{BC}\big|_{t=0} &amp;amp;\mathrm{at} &amp;amp;(\Omega\times \{0\})&#xA;\end{align*}&#xA;\right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step Two Provide bounds for each eigenfunction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\left\{(\lambda_i^2, V_i)\right\}_{i=1}^\infty\in (\mathbb{R}^+ \times H_0^1)$ be the  the eigenvalue and eigenfunction pairs ordered by increasing eigenvalue. Expand $w$ as $w = \sum_{i=1}^\infty \omega_i(t) V_i$ where $\omega_i(t)$ is the weight of the $i$'th eigenfunction at time $t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then after testing the PDE with a test function $V_i$ we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d}{dt} \omega_i(t) + \lambda_i^2 \omega_i(t) = -(V_i, \frac{d}{dt} u_{BC})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This ODE has solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega_i(t) = \exp(-\lambda^2_i t) \omega_i(0) - \int_{\tau =0 }^t \exp(-\lambda_i^2(t-\tau))(V_i, \frac{\mathrm{d}}{\mathrm{d}\tau}u_{BC})\, \mathrm{d}\tau&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we integrate by parts on the integral term to get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*}\omega_i(t) =&amp;amp; \exp(-\lambda^2_i t) \omega_i(0) - (V_i, u_{BC}(t)) + (V_i, u_{BC}(0)) \exp(-\lambda^2_i t) \\&#xA;&amp;amp;+ \int_{\tau=0}^t \lambda_i^2 \exp(-\lambda_i^2 (t-\tau)) ( V_i, u_{BC}(\tau))\, \mathrm{d}\tau&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the sum of the first and third term vanishes because of the intial conditions for $u$ in the original problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we are left with&#xA;$$&#xA;\begin{align*}\omega_i(t) =&amp;amp;  - (V_i, u_{BC}(t)) + \int_{\tau=0}^t \lambda_i^2 \exp(-\lambda_i^2 (t-\tau)) ( V_i, u_{BC}(\tau))\, \mathrm{d}\tau&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now expanding $u$ as $u = \sum_{i=1}^\infty \mathcal{u}_{i}(t) V_i$ and using the previous equation we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{u}_i(t) = \int_{\tau=0}^t \lambda_i^2 \exp(-\lambda_i^2 (t-\tau)) ( V_i, u_{BC}(\tau))\, \mathrm{d}\tau$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step Three Estimate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now&#xA;$$&#xA;\begin{align*}&#xA;\|u\|_{L^\infty(\mathbb{R}^+, L^2(\Omega)}^2\\&#xA;&amp;amp;= \sup_{t\in\mathbb{R}^+} \sum_{i=1}^{\infty} \left(\mathcal{u}_i(t)\right)^2\\&#xA;&amp;amp;= \sup_{t\in\mathbb{R}^&#xA;+} \sum_{i=1}^{\infty} &#xA;\left(\int_{\tau=0}^t \lambda_i^2\exp(-\lambda_i^2 (t-\tau)) (V_i,u_{BC}(\tau))\, \mathrm{d} \tau \right)^2\\&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where I am stuck.  I have attempted estimates like the following but I run into problems because of the countably infinite quantity of eigenpairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*}&#xA;&amp;amp; \sup_{t\in\mathbb{R}} \sum_{i=1}^{\infty} &#xA;\left(\int_{\tau=0}^t \lambda_i^2\exp(-\lambda_i^2 (t-\tau)) (V_i,u_{BC}(\tau))\, \mathrm{d} \tau \right)^2\\&#xA;&amp;amp;\le \sup_{t\in\mathbb{R}} \sum_{i=1}^{\infty} \left(\sup_{\tau\in (0,t)} (V_i, u_{BC}(\tau)) \right)^2 \left(\int_{\tau=0}^t \lambda_i^2 \exp(-\lambda_i^2 \tau)\, \mathrm{d}\tau \right)^2\\&#xA;&amp;amp;\le \sup_{t\in\mathbb{R}} \sum_{i=1}^\infty \left(\sup_{\tau\in(0,t)}(V_i, u_{BC}(\tau) ) \right)^2&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any further estimation I perform I lose the boundedness.  I am guessing I just need a different inequality I am not thinking of.  Any ideas?  Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another Addendum:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think it is legal to expand $u$ as $u= \sum_{i=1}^\infty \mathcal{u}_i V_i$ as all the $V_i$'s are zero on the boundary condition.  So a new strategy would be to bound $w$ and use Lax-Milgram and to $u_{BC}$ and then use the triangle inequality to bound $u$. &lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2015-07-06T13:58:04.387" LastActivityDate="2015-07-06T13:58:04.387" Title="Bounded Input Boundaed Output stability for heat equation. Proof or Counter example?" Tags="|pde|fourier-analysis|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20075" PostTypeId="2" ParentId="20074" CreationDate="2015-07-03T22:57:42.407" Score="3" Body="&lt;p&gt;You immediately get $L^\infty$ stability by using the eigenfunction decomposition of the solution. The coefficient of each mode is strictly decreasing exponentially if you don't have a right hand side (or boundary) values, and consequently the $L_2$ norm of the solution decays. In your case, you need to write down the differential equation for each mode and prove stability for it; the stability for the entire solution the follows immediately by summation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-07-03T22:57:42.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20076" PostTypeId="1" CreationDate="2015-07-04T02:30:58.747" Score="2" ViewCount="831" Body="&lt;p&gt;I have four partial differential equations representing mass conservation of two compressible fluid phases (marked by subscripts $p1$ and $p2$) in two different continuum media (marked by subscripts $c1$ and $c2$) and they are coupled together by the term $T_{c1c2}$ as shown below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Continuum 1:&lt;/strong&gt;&#xA;$$\begin{align}&#xA;\text{Phase 1: } \frac{\partial }{\partial t}\left(\frac{\phi_{c1}S_{p1c1}}{B_{p1c1}} \right) = \nabla\left(\lambda_{p1c1}\nabla{P_{p1c1}} \right) - \frac{T_{c1c2}}{V} - \frac{q_{p1c1}}{V}, \\&#xA;\text{Phase-2:} \frac{\partial }{\partial t}\left(\frac{\phi_{c1}S_{p2c1}}{B_{p2c1}} \right) = \nabla\left(\lambda_{p2c1}\nabla{P_{p2c1}} \right) - \frac{T_{c1c2}}{V} - \frac{q_{p2c1}}{V},&#xA;\end{align}&#xA;$$&#xA;&lt;strong&gt;Continuum 2:&lt;/strong&gt;&#xA;$$\begin{align}&#xA;\text{Phase 1: }\frac{\partial }{\partial t}\left(\frac{\phi_{c2}S_{p1c2}}{B_{p1c2}} \right) = \nabla\left(\lambda_{p1c2}\nabla{P_{p1c2}} \right) - \frac{T_{c1c2}}{V} - \frac{q_{p1c2}}{V}, \\&#xA;\text{Phase 2: } \frac{\partial }{\partial t}\left(\frac{\phi_{c2}S_{p2c2}}{B_{p2c2}} \right) = \nabla\left(\lambda_{p2c2}\nabla{P_{p2c2}} \right) - \frac{T_{c1c2}}{V} - \frac{q_{p2c2}}{V},&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, there are 4 more linear equations which are:&#xA;$$S_{p1c1} + S_{p2c1} = 1$$&#xA;$$S_{p1c2} + S_{p2c2} = 1$$&#xA;$$P_{p2c1} - P_{p1c1} = P_{cap1}$$&#xA;$$P_{p2c2} - P_{p1c2} = P_{cap2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Variables:&lt;/strong&gt; In these 4 linear equations, $P_{cap1}$ and $P_{cap2}$ are known.&#xA;Except for $\phi_{c1}$, $\phi_{c2}$ and $V$, which are constants, all other variables vary with time $t$ and distance $(x,y)$. Also, &#xA;$$\frac{1}{B_{p1c1}} = 1+c_{p1}\left(P_{p1c1} - P^{STC}\right) \text{and} \frac{1}{B_{p2c1}} = 1+c_{p2}\left(P_{p2c1} - P^{STC}\right)$$ where $c_{p1}$, $c_{p2}$ and $P^{STC}$ are constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Initial Condition:&lt;/strong&gt; Dirichlet constant pressure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Boundary Condition:&lt;/strong&gt; Neumann no-flow boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Objective:&lt;/strong&gt; To solve for saturations and pressures as a function of space and time, i.e. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S_{p1c1}$, $S_{p2c1}$, $S_{p1c2}$, $S_{p2c2}$ and &#xA;$P_{p1c1}$, $P_{p2c1}$, $P_{p1c2}$, $P_{p2c2}$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This system of equations could be discretized into linear system of equations and then solved for each time step, however, that is a complex task and would take a lot of time. I would, however, like to use some form of &quot;tool&quot; which can solve this coupled system of equations in a mesh grid format. I saw an answer in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/15847/solving-advection-convection-diffusion-reaction-partial-differential-equat&quot;&gt;other post&lt;/a&gt; suggested to use FiPy, but the feedback from the OP was that it's way too slow. Please suggest what could be a good tool to solve this problem. &lt;/p&gt;&#xA;" OwnerDisplayName="user5510" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-07-05T01:02:24.393" Title="Solving a system of 4 coupled PDEs representing variable diffusivity" Tags="|pde|python|linear-solver|parabolic-pde|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20077" PostTypeId="1" AcceptedAnswerId="20080" CreationDate="2015-07-04T05:45:40.013" Score="1" ViewCount="50" Body="&lt;p&gt;Consider the following situation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/6MQK2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two boundaries -- one is denoted using grey lines, and the other is denoted using black lines. The boundaries are numerically represented using &quot;vertices&quot;, with edges assumed to be connecting these vertices, but otherwise not necessary for the calculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a given vertex (say, vertex $i$), we can write ODEs of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\textrm{d}x_i}{\textrm{d}t} &amp;amp;= A_i(...) \\&#xA;\frac{\textrm{d}y_i}{\textrm{d}t} &amp;amp;= B_i(...)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the $...$ are placeholders for various variables that could be, for example: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the position of other vertices on the same boundary (relevant for instance, if we are modelling &quot;spring-like&quot; connections between vertices)&lt;/li&gt;&#xA;&lt;li&gt;the position of vertices on the other boundary (relevant for instance, if we are modelling repulsive forces between the boundaries, &lt;strong&gt;as shown in the situation diagram above as an example&lt;/strong&gt;)&lt;/li&gt;&#xA;&lt;li&gt;various other variables (for example, maybe a mystery chemical signal which determines the strength of the forces acting on the vertex in some particular direction)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In order to model the dynamics of the boundaries, one can integrate the vertex ODEs. For a simple example as shown in the diagram above, we can simply consider the ODEs for all the vertices together in one large system, and then integrate this system of ODEs numerically using integrator.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;A possibly invalid integration simplification?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If the boundaries rarely ever interact (i.e. they are rarely so close as to actually affect each other, depending on how the model is set up), then one could attempt another approximation, in order to lessen the computational burden on the integrator:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Say we know the state of the system (containing both grey and black boundaries) at some time $t$.&lt;/li&gt;&#xA;&lt;li&gt;In order to compute the state of the system at $t + \Delta t$, pick one boundary at random, call it boundary $P$ for &quot;picked&quot;. Evolve the chosen boundary from time $t$ to $t + \Delta t$ using numerical integration, but consider the other state of the other boundary (call it $Q$ for &quot;not picked&quot;, or &quot;not $P$&quot;) to be held fixed during this calculation. So, when we need to use information regarding $Q$ to evolve the state of $P$, we only use information about the state of $Q$ at $t$.&lt;/li&gt;&#xA;&lt;li&gt;Evolve state of boundary $Q$ via integration from time $t$ to $\Delta t$, and if any information regarding $P$ is needed for the calculation, only use information regarding $P$ from time $t$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Summarizing, this method aims to reduce the &quot;burden&quot; on the integrator by integrating small pieces of the global system at a time, while assuming that the rest of the system has not evolved.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;The problem of &quot;volume&quot; exclusion&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Maybe the boundaries denote special areas that never intersect: for instance, both boundaries might represent separate membranes which can come very close to one another, but do not intersect. I call this &quot;volume exclusion&quot; (two &quot;volumes&quot; cannot intersect), but perhaps there is better terminology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, one might implement volume exclusion by assigning repulsive forces to the vertices representing each boundary, where these repulsive forces depend on the distance between a particular vertex and the closest point away from it on the other boundary (see the situation diagrammed above for an example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several issues with implementing such a rule however, with one in particular being how &quot;intersections&quot; (due to numerical error) are handled. If one vertex does end up in the volume contained by another boundary, how do we correct for it without breaking the integration process?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any resources which explore such problems?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2015-07-05T02:13:21.010" Title="Resources exploring the problem of &quot;volume exclusion&quot;?" Tags="|numerical-analysis|reference-request|time-integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20078" PostTypeId="1" CreationDate="2015-07-04T15:23:42.293" Score="2" ViewCount="1335" Body="&lt;p&gt;Basically, I am trying to solve the 2D poisson problem in order to learn implementation of IPDG methods. The problem states $-\nabla a(x)(\nabla u)=0\  \text{in} \  \Omega$ with $U=0$ on Dirichlet boundary $\Gamma_D$. The weak DG formulation gives find u such that&#xA;$$a_{\epsilon}(u,v)=l(v)$$ where  $$a_{\epsilon}(u,v)=\sum_{K\in T_h}\int_K a\nabla u\cdot \nabla v-\sum_{e\in \Gamma_h\cup \Gamma_D}\int_e\{a\nabla u\cdot n_e\}[v]+ \sum_{e\in \Gamma_h\cup \Gamma_D}\int_e\epsilon\{a\nabla v\cdot n_e\}[u] + \sum_{e\in \Gamma_h\cup \Gamma_D}\frac{\sigma_0}{|e|}\int_e[u][v]$$ Note that $\Gamma_h$ represents the collection of interior edges and $\epsilon$ chooses the method from SIPG,NIPG or IIPG. I understand very well how to compute and assemble the integral over volume in matlab but the skeleton terms (2nd and third terms and fourth ) are my problems. I need  help with a simple matlab code for these three terms. I need to know how to code them  and assemble them.&lt;/p&gt;&#xA;" OwnerUserId="16842" LastEditorUserId="27" LastEditDate="2015-07-04T17:19:21.960" LastActivityDate="2017-06-29T14:23:28.700" Title="Matlab implementation of 2D Interior penalty discontinuous Galerkin poisson problem" Tags="|discontinuous-galerkin|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20079" PostTypeId="1" AcceptedAnswerId="20102" CreationDate="2015-07-04T17:11:28.550" Score="5" ViewCount="356" Body="&lt;p&gt;I tried to write an analysis program in python for the Millikan experiment based on a paper from Jones &lt;a href=&quot;http://scitation.aip.org/content/aapt/journal/ajp/63/11/10.1119/1.18001&quot; rel=&quot;nofollow&quot;&gt;&quot;The Millikan oil‐drop experiment: Making it worthwhile&quot;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I cannot reproduce the calculated values of this paper for the radius of the oil drops and the charges when using an iterative Cunningham correction approach. If I iterate 6 times as indicated in the paper I get completely different results and if I take enough iteration steps, $r$ and $q$ seem to converge to zero, which doesn't make sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a quote from the paper (with some minor changes in notation to match my program).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From Stokes' law, the radius, $r$, of the drop is given by  $$ r =&#xA;\sqrt{\frac{9\eta v_\text{Fall}}{2g\rho}} $$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where $\eta$ is the viscosity of the air, $v_{\text{Fall}}$ is the&#xA;  terminal speed of the drop falling under gravity, $g$ is the&#xA;  acceleration of gravity and $\rho$ is the mass density of the oil&#xA;  measured in air. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Stoke's law must be corrected for spheres that are comparable in size&#xA;  to the mean-free path of the air molecules by multiplying the&#xA;  viscosity by a correction factor: &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$ \eta_{\text{eff}} = \eta \left (1 + \frac{b}{pr} \right)^{-1} $$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where $\eta_{\text{eff}}$ is the effective viscosity of air $p$ is&#xA;  the absolute barometric pressure in $\text{cm Hg}$ and $b =&#xA;6.17\cdot 10^{-6}\,\text{m cm Hg}$ is a constant. This correction factor depends on the drop radius, and the calculated drop radius&#xA;  depends on this factor via the equation above. It seems to be widely&#xA;  assumed that that iteration between those equations is not necessary.&#xA;  In his original work Millikan iterated twice so that this correction&#xA;  converged sufficiently. Teaching versions of this experiment use lower&#xA;  voltages and thus often work with smaller drops than did Millikan. [...] In&#xA;  this study the computer iterated six times for each timing of a drop.&#xA;  This produced complete convergence in single-precision computer&#xA;  arithmetics.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Do you have any idea what may be wrong in my program:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there an error in the implementation?&lt;/li&gt;&#xA;&lt;li&gt;Did I understand wrongly what is meant be &quot;iteration&quot; in the paper? (Just see my source code, of how I interpreted it)&lt;/li&gt;&#xA;&lt;li&gt;Why does this iteration even make sense? I don't see how to justify it mathematically. However it seems to be used by Millikan in his work which lead to a Nobel prize and also by lot's of other works about Millikan's oil drop experiment. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;%pylab inline&#xA;# ipython3 notebook&#xA;&#xA;def mil(v_Fall,v_Rise,U,d,rho=859.9,g=9.81,eta=1.81e-5,N=0,verbose=False):&#xA;    pi = math.pi&#xA;    b = 6.17e-6 # m cm Hg&#xA;    p = 76 # cm Hg&#xA;&#xA;    # Analytic approach&#xA;    r_ana = (9*eta*v_Fall/(2*rho*g) + (b/(2*p))**2)**(0.5) - b/(2*p)&#xA;    q_ana = (6*pi*d/U)*(9*eta**3/(2*rho*g))**(0.5)* \&#xA;            (1+(b/(p*r_ana)))**(-3/2)*(v_Rise+v_Fall)*v_Fall**(0.5)&#xA;&#xA;    # Iterative Approach&#xA;    r= sqrt(9*eta*v_Fall/(2*g*rho))&#xA;    for i in range(0,N):&#xA;        eta = eta*(1 + b/(p*r))**(-1)&#xA;        if verbose==True:&#xA;            print(r,eta) # it doesn't seem to converge to a value &amp;gt; 0&#xA;        r= sqrt(9*eta*v_Fall/(2*g*rho))&#xA;    q = 4*pi*d/(3*U)*(g*rho*r**3 + 9/2*eta*r*v_Rise)&#xA;    return array([r*1e6,r_ana*1e6,q/1.5793e-19,q_ana/1.5793e-19])&#xA;&#xA;def evalMil(s_Fall,s_Rise,t_Fall,t_Rise,U,d,**kwargs):&#xA;    v_Fall = s_Fall/t_Fall&#xA;    v_Rise = s_Rise/t_Rise&#xA;    return mil(v_Fall,v_Rise,U,d,**kwargs)&#xA;&#xA;# Read data&#xA;M = np.loadtxt('Millikan_Jones_test.dat', skiprows=1)&#xA;t_Fall = M[:,1]&#xA;t_Rise= M[:,0]&#xA;d = 0.0044012 # m&#xA;U = 405 # V&#xA;s = 0.0015945 # m&#xA;&#xA;# eta value was not given in the paper and a bit different for each value&#xA;# (depending on the temperature)&#xA;A = evalMil(s,s,t_Fall,t_Rise,U,d,g=9.79637,N=6,eta=1.79237e-5) &#xA;# Set N = 100 and you get q = 0 everywhere&#xA;&#xA;# Analytic&#xA;rA = A[1]&#xA;rI = A[0]&#xA;&#xA;# Iterative&#xA;qA = A[3]&#xA;qI = A[2]&#xA;&#xA;&#xA;figsize(15,10)&#xA;xlim(0,1.5)&#xA;ylim(0,10)&#xA;plot(rA,qA,'x')&#xA;plot(rI,qI,'x',color='red')&#xA;for k in range(0,35):&#xA;    axhline(y=k,color='gray')&#xA;&#xA;&#xA;&#xA;# Output as table: &#xA;class ListTable(list):&#xA;&quot;&quot;&quot;&#xA;Overridden list class which takes a 2-dimensional list of &#xA;the form [[1,2,3],[4,5,6]], and renders an HTML Table in &#xA;IPython Notebook.&#xA;&quot;&quot;&quot;&#xA;&#xA;    def _repr_html_(self):&#xA;        html = [&quot;&amp;lt;table&amp;gt;&quot;]&#xA;        for row in self:&#xA;            html.append(&quot;&amp;lt;tr&amp;gt;&quot;)&#xA;&#xA;            for col in row:&#xA;                html.append(&quot;&amp;lt;td&amp;gt;{0}&amp;lt;/td&amp;gt;&quot;.format(col))&#xA;&#xA;            html.append(&quot;&amp;lt;/tr&amp;gt;&quot;)&#xA;        html.append(&quot;&amp;lt;/table&amp;gt;&quot;)&#xA;        return ''.join(html)&#xA;&#xA;tabular = ListTable()&#xA;tabular.append(['$r$ in $10^{-6}\mathrm{m}$ (iterative)',&#xA;                '$r$ in $10^{-6}\mathrm{m}$ (analytic)',&#xA;                '$q$ in $1.5793\cdot 10^{-19}\,\mathrm{C}$ (iterative)',&#xA;                '$q$ in $1.5793\cdot 10^{-19}\,\mathrm{C}$ (analytic)'])&#xA;for i in range(0,len(A[1])):&#xA;    tabular.append(A[:,i].tolist())&#xA;tabular&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are my data as taken from the paper:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t_Rise t_Fall&#xA;9.181   17.490&#xA;22.436  52.182&#xA;35.473  11.766&#xA;30.967  12.875&#xA;29.823  12.955&#xA;9.454   7.260&#xA;4.148   5.554&#xA;23.698  17.446&#xA;28.967  7.602&#xA;7.269   12.229&#xA;11.285  7.252&#xA;7.417   15.982&#xA;7.198   17.095&#xA;15.170  34.503&#xA;28.977  45.775&#xA;26.561  46.392&#xA;10.375  47.318&#xA;10.588  2.939&#xA;6.675   10.481&#xA;14.269  11.695&#xA;24.748  50.816&#xA;23.838  51.498&#xA;4.848   7.251&#xA;8.753   15.073&#xA;18.803  28.796&#xA;88.926  30.198&#xA;36.984  12.453  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16837" LastEditorUserId="9667" LastEditDate="2015-07-06T19:20:13.993" LastActivityDate="2015-07-15T21:03:13.017" Title="Iterative Cunnigham correction in Millikans oil-drop experiment" Tags="|python|data-analysis|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="20080" PostTypeId="2" ParentId="20077" CreationDate="2015-07-05T02:13:21.010" Score="2" Body="&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; I believe the usual name for these things is non-penetration constraints. As you say, there is already quite some literature on this, especially in robotics and computer graphics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; Your method looked like explicit Euler to me because I didn't notice that you didn't say how you integrated the non-fixed half of the system. So you seem to know what you're doing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the question of whether it's valid, here is a simplified analysis of it. Take the coupled ODE&#xA;$$ \dot u = f(u,v), \qquad \dot v = g(u,v). $$&#xA;With a time step $\delta t$, the exact solution at $t+\delta t$ is&#xA;$$ u(t+\delta t) = u(t) + \int_t^{t+\delta t} f\big(u(s),v(s)\big)\,\mathrm{d}s. $$&#xA;The solution with &quot;fixed&quot; $v$ is&#xA;$$ \tilde u(t+\delta t) = u(t) + \int_t^{t+\delta t} f\big(u(s), v(t)\big)\,\mathrm{d}s, $$&#xA;which is approximated by the corresponding step of the ODE solver of order $s$:&#xA;$$ \tilde u(t+\delta t) = u^{n+1} + O(\delta t^{s+1}). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between the two schemes is:&#xA;$$ u(t+\delta t) - u^{n+1} = O(\delta t^{s+1}) + \int_t^{t+\delta t}\Big( f(u(s),v(s)) - f(u(s),v(t))\Big)\,\mathrm{d}s, $$&#xA;and the second error term here can be approximately bounded using Taylor series:&#xA;$$ O(\delta t^{s+1}) + |f_v||\dot v|\delta t^2. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that the scheme is first-order accurate, and, depending on how small $|f_v|$ is (i.e., whether the interaction between the two system parts is the dominant interaction), it might be sufficiently accurate for your purposes. It might break down for a sufficiently stiff ODE, but that's a separate question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; For the purposes of looking up more references, I think your method is &lt;em&gt;very&lt;/em&gt; close to a first-order split-step method. In a typical split step method, if you have a linear system&#xA;$$ \dot y = \mathbf{A}y, \qquad y(t) = e^{t\mathbf{A}}y(0), $$&#xA;and you define a splitting $\mathbf{A} = \mathbf{A}_1 + \mathbf{A}_2$, you can find appropriate constants so that&#xA;$$ y(t) \approx e^{c_1 t\mathbf{A}_1}e^{c_2 t\mathbf{A}_2}\cdots y(0). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the linearized system &#xA;$$ \mathbf{A} = \begin{pmatrix} \mathbf{A}_{11}&amp;amp;\mathbf{A}_{12}\\ \mathbf{A}_{21}&amp;amp;\mathbf{A}_{22} \end{pmatrix}, $$&#xA;your method is close to being the split-step method with the splitting&#xA;$$ \mathbf{A} = \begin{pmatrix} \mathbf{A}_{11}&amp;amp;\mathbf{A}_{12}\\0&amp;amp;0\end{pmatrix} + \begin{pmatrix}0&amp;amp;0\\ \mathbf{A}_{21}&amp;amp;\mathbf{A}_{22}\end{pmatrix}. $$&#xA;Notice that both $e^{t\mathbf{A}_1}$ and $e^{t\mathbf{A}_2}$ evolve only part of the variables, and keep the rest fixed. The splitting operators are singular here, which is a bit unusual. This isn't quite the same as your method because $e^{h \mathbf{A}_2}e^{h \mathbf{A}_1}$ would use the &lt;em&gt;new&lt;/em&gt; value of the part of the system that gets evolved first, rather than the old value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formulating it this way would make it easier to analyze the method's order and stability properties. For your purposes the first-order split-step method might be sufficient, in which case formulating it in the generic terms of split-step methods might be a bit overkill.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-07-05T02:13:21.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20081" PostTypeId="1" CreationDate="2015-07-05T04:38:37.030" Score="0" ViewCount="153" Body="&lt;p&gt;I have created a 4-dimensional array (cell) of arrays called &quot;a&quot; where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a{i,j,k,l} = [ v_1 v_2 v_3 v_4] for some vector v. each of the index variables runs from 1 to 10 in steps of 1. I have a total of 10000 entries in the cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to reshape this cell into another cell whose dimensions are 1x10000.&#xA;I want the first element of the new cell to be the first element of a (i.e. a{1,1,1,1}).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using &quot;cat&quot; command but it didn't work. It only produced a single output instead of an array output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help.&lt;/p&gt;&#xA;" OwnerUserId="16815" LastActivityDate="2017-01-05T11:26:03.017" Title="Multi-dimensional array reshaping Matlab" Tags="|matlab|" AnswerCount="1" CommentCount="3" ClosedDate="2017-01-12T04:28:25.660" ContentLicense="CC BY-SA 3.0" />
  <row Id="20082" PostTypeId="2" ParentId="20064" CreationDate="2015-07-05T05:31:27.693" Score="1" Body="&lt;p&gt;If you want to compute a physical density for a graph then why don't you try and estimate the &lt;a href=&quot;https://en.wikipedia.org/wiki/Number_density&quot; rel=&quot;nofollow&quot;&gt;Number Density&lt;/a&gt;. That is, the &lt;strong&gt;number of vertices per unit area&lt;/strong&gt; (or volume if that is more appropriate for your problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is what area to use to perform the computation? You may need to sample the local area around a vertex to see how the vertex density behaves as it approaches the vertex. For example, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Select a vertex $u$&lt;/li&gt;&#xA;&lt;li&gt;Find the distance $d$ to its nearest neighbor&lt;/li&gt;&#xA;&lt;li&gt;Count the number of vertices $|V_{r}|$ within a radii $r$=$d$, $2d$, $3d$, ..., $Nd$&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the density at each radii: $\rho_{d}$, $\rho_{2d}$, $\rho_{3d}$, ..., $\rho_{Nd}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho_{r}=|V_{r}| / \pi r^{2}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fit a line between the estimates of $\rho$ and the radii $r$: $\rho(r) = m * r + \rho_{0}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Use the offset $\rho_{0}$ (i.e., $r \rightarrow 0$ ) as your estimate of the density at the vertex $u$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A few things to keep in mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You may run into problems of negative density, in which case either this method won't work or you need enforce the constraint $\rho &amp;gt; 0$ during the line fitting optimization.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You'll also need to pick a neighborhood size $N$. I would start with $N=10$. That is, start with an order of magnitude from the nearest neighbor.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You may also need to verify that a line model is a good local model. You may need to try other functions to get a good prediction.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9813" LastEditorUserId="9813" LastEditDate="2015-07-06T01:30:18.460" LastActivityDate="2015-07-06T01:30:18.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20083" PostTypeId="1" CreationDate="2015-07-05T10:38:06.423" Score="-1" ViewCount="434" Body="&lt;p&gt;I am trying to find the minimum of this function. But I receive the following error when I run the script. What am I doing wrong:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            % pre allocating&#xA;            x=zeros(10,4);&#xA;            mu=zeros(10,1);% process parameter&#xA;            T=zeros(10,1);% spike times&#xA;            y=zeros(10,1);%number of spikes&#xA;            eta=zeros(10,1);% linear predictor&#xA;            for i=1:10 % on trials&#xA;                for j=1:4 % on pixels&#xA;                    x(i,j)=randi(2,1)-1;&#xA;                end&#xA;                mu(i)=randi(20,1)/100;&#xA;                [T,y(i)]=ppsample(mu(i),20);&#xA;                eta(i)=log(mu(i));&#xA;                syms b0;syms b1;syms b2;syms b3;syms b4;syms objfun;&#xA;                o(i)=((b0+b1*x(i,1)+b2*x(i,2)+b3*x(i,3)+b4*x(i,4))*y(i))-exp(b0+b1*x(i,1)+b2*x(i,2)+b3*x(i,3)+b4*x(i,4));&#xA;            end&#xA;            %%finding optimum weights&#xA;&#xA;            objfun=sum(o);&#xA;            ofun= matlabFunction(objfun);&#xA;            x0 = ones(1,5);&#xA;            [x,fval] = fminunc(ofun,x0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error using makeFhandle/@(b0,b1,b2,b3,b4)b0.*1.9e1+b1.*9.0+b2.*1.2e1+b3.*7.0+b4.*7.0-exp(b0+b1+b2+b3)-exp(b0+b2+b3+b4)-exp(b0+b1).*2.0-exp(b0+b2)-exp(b0)-exp(b0+b1+b2)-exp(b0+b1+b3)-exp(b0+b2+b4)-exp(b0+b3+b4)&#xA;Not enough input arguments.&#xA;&#xA;Error in fminunc (line 254)&#xA;    f = feval(funfcn{3},x,varargin{:});&#xA;&#xA;Error in GLM (line 30)&#xA;[x,fval] = fminunc(ofun,x0);&#xA;&#xA;Caused by:&#xA;    Failure in initial user-supplied objective function evaluation.&#xA;    FMINUNC cannot continue.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16845" LastEditorUserId="7733" LastEditDate="2015-07-05T18:20:21.040" LastActivityDate="2015-07-06T03:48:27.630" Title="Problem using MATLAB `fminunc`" Tags="|optimization|statistics|" AnswerCount="1" CommentCount="1" ClosedDate="2015-07-06T00:03:27.687" ContentLicense="CC BY-SA 3.0" />
  <row Id="20085" PostTypeId="2" ParentId="20081" CreationDate="2015-07-05T15:56:25.957" Score="1" Body="&lt;p&gt;I figured it out!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c=cell(4,4)&#xA;for i&#xA;for j&#xA;for k&#xA;for l&#xA;c{i,j,k,l} = [v_i v_j v_k v_l];&#xA;end&#xA;end&#xA;end&#xA;end&#xA;&#xA;e=reshape(c,1,i*j*k*l);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to call the mth element: e{1,m}&lt;/p&gt;&#xA;" OwnerUserId="16815" LastActivityDate="2015-07-05T15:56:25.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20086" PostTypeId="2" ParentId="20083" CreationDate="2015-07-05T21:00:33.117" Score="1" Body="&lt;p&gt;Your syntax for the objective function evaluation is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code below should work. The 1 by 5 vector b is your optimization variable. It inherits its dimensions from the dimensions of x0 in the fminunc call. I got rid of all the syms stuff.  Note that you need @ before your objective function name in the call to fminunc so that it becomes a function handle.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                % Put this in the file ofun.m somewhere in your MATLAB path                &#xA;                function [objective] = ofun(b)&#xA;                % The line below is so that your expression for o(i) is left unchanged&#xA;                b0 = b(1); b1 = b(2); b2 = b(3); b3 = b(4); b4 = b(5);&#xA;                % pre allocating&#xA;                x=zeros(10,4);&#xA;                mu=zeros(10,1);% process parameter&#xA;                T=zeros(10,1);% spike times&#xA;                y=zeros(10,1);%number of spikes&#xA;                eta=zeros(10,1);% linear predictor&#xA;                for i=1:10 % on trials&#xA;                    for j=1:4 % on pixels&#xA;                        x(i,j)=randi(2,1)-1;&#xA;                    end&#xA;                    mu(i)=randi(20,1)/100;&#xA;                    [T,y(i)]=ppsample(mu(i),20);&#xA;                    eta(i)=log(mu(i));&#xA;                    o(i)=((b0+b1*x(i,1)+b2*x(i,2)+b3*x(i,3)+b4*x(i,4))*y(i))-exp(b0+b1*x(i,1)+b2*x(i,2)+b3*x(i,3)+b4*x(i,4));&#xA;                end&#xA;                objective = sum(o);&#xA;                end&#xA;                % end of what goes in ofun.m&#xA;&#xA;                %%finding optimum weights&#xA;&#xA;                x0 = ones(1,5);&#xA;                [x,fval] = fminunc(@ofun,x0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2015-07-06T03:48:27.630" LastActivityDate="2015-07-06T03:48:27.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20087" PostTypeId="1" CreationDate="2015-07-05T22:07:09.893" Score="2" ViewCount="134" Body="&lt;p&gt;I have been experimenting with using Autodesk CFD to investigate facade/ cladding pressures on a (rectangular) building, comparing results with cladding/ facades pressures pressures from design codes and wind tunnel results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the complexities and difficulties in what I am doing generally and am really exploring at the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my introductory understanding of CFD from uni and reading around in the literature I understand that the pressures the various turbulence model produce are 'mean' pressures. Papers etc. are always very explicit in referring to 'mean' whenever talking about velocities/ pressures using these techniques. In cladding design one is interested in peak pressures - this is what the design codes and wind tunnel testing give you to design with.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder whether anyone might be able to provide insight, or might have a good reference for further reading, on the possible relationship between peaks and means in these theories? I am not too strong on statistics but is it something that can be derived? It feels like it ought to be. &lt;/p&gt;&#xA;" OwnerUserId="16843" LastActivityDate="2015-07-06T08:54:17.283" Title="CFD turbulence modelling mean pressures vs peak pressures" Tags="|fluid-dynamics|simulation|statistics|architecture|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20088" PostTypeId="2" ParentId="20087" CreationDate="2015-07-05T23:24:00.260" Score="1" Body="&lt;p&gt;First, you can model turbulence using averages; e.g.: &lt;a href=&quot;https://en.wikipedia.org/wiki/Reynolds-averaged_Navier%E2%80%93Stokes_equations&quot; rel=&quot;nofollow&quot;&gt;RANS&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you will see, higher order moments appear and you need to find a way to close your equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that's about mean flows since you want to find solution for the mean values of your variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, you can derive a stochastic form of the Navier Stokes Equation and solve the probability density function (pdf). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have some background on Ito Calculus and a great place to start would be &lt;a href=&quot;http://www.springer.com/us/book/9781461462613&quot; rel=&quot;nofollow&quot;&gt;The Kolmogorov-Obukhov Theory of Turbulence&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that you compute averages using RANS (in statistics you will see they call them moments, and the mean is just an example of a first order moment). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this leaves out some information of the system. In reality we have a fluctuating variable, but we only predict the mean. The mean gives you very little information about the system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we can improve our prediction by predicting the variance, which gives you more information about variations about the mean. If you go further, you can predict the probability density function (pdf), and the values of the mean, the variance, and high order moments just fall on your lap. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stochastic methods give you almost all the information you need, but there is a catch. You must have some basic assumptions; for example you could say that your pdf follows a gaussian distribution, which is not necessarily correct.&lt;/p&gt;&#xA;" OwnerUserId="11656" LastEditorUserId="11656" LastEditDate="2015-07-06T08:54:17.283" LastActivityDate="2015-07-06T08:54:17.283" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20089" PostTypeId="1" CreationDate="2015-07-06T13:21:06.170" Score="5" ViewCount="911" Body="&lt;p&gt;&lt;strong&gt;Problem Summary&lt;/strong&gt;&lt;br&gt;&#xA;I am trying to estimate the &lt;code&gt;(x,y)&lt;/code&gt; coordinates of each node in a graph, where I know the distances between connected nodes. For example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PL03Q.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this graph, I can define an equation for the length of each link simply using the Pythagorean theorem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L_{AB}=\sqrt{(X_B-X_A)^2+(Y_B-Y_A)^2}$$&#xA;$$L_{AD}=\sqrt{(X_D-X_A)^2+(Y_D-Y_A)^2}$$&#xA;$$L_{BC}=\sqrt{(X_C-X_B)^2+(Y_C-Y_B)^2}$$&#xA;$$L_{BD}=\sqrt{(X_D-X_B)^2+(Y_D-Y_B)^2}$$&#xA;$$L_{CD}=\sqrt{(X_D-X_C)^2+(Y_D-Y_C)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this problem I know all of the &lt;code&gt;L&lt;/code&gt; values, and I need to solve for all the &lt;code&gt;X&lt;/code&gt; and &lt;code&gt;Y&lt;/code&gt; values. I can arbitrarily pick a node, and call it the origin, so I can say &lt;code&gt;A = (0,0)&lt;/code&gt;, then I have 6 unknowns $X_{B-D}$ and $Y_{B-D}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I figure I can pose this question as minimization of the sum of the square of the errors, mathematically&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f=(\sqrt{(X_B-X_A)^2+(Y_B-Y_A)^2}-L_{AB})^2\\+(\sqrt{(X_D-X_A)^2+(Y_D-Y_A)^2}-L_{AD})^2\\+(\sqrt{(X_C-X_B)^2+(Y_C-Y_B)^2}-L_{BC})^2\\+(\sqrt{(X_D-X_B)^2+(Y_D-Y_B)^2}-L_{BD})^2\\+(\sqrt{(X_D-X_C)^2+(Y_D-Y_C)^2}-L_{CD})^2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;br&gt;&#xA;I'm not sure how to perform a minimization of a function in C++ (I'm not too familiar with this area, so I apologize if that is a broad question). If I were in &lt;code&gt;MATLAB&lt;/code&gt;, I could make some initial guesses&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = [0 1 2 3];&#xA;Y = [0 2 1 3];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then call &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/fminunc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;fminunc&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fminunc(@f, [X,Y])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This would return an array that I could pull apart into the &lt;code&gt;X&lt;/code&gt; and &lt;code&gt;Y&lt;/code&gt; coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically in C++ I can define some function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double f(double* A)&#xA;{&#xA;    // Let A be an array of all X and Y coordinates&#xA;    // This function is the error function described above&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to write (or hopefully find!) a function that does the same thing as &lt;code&gt;fminunc&lt;/code&gt; but in C++, so I guess it would look like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double* fminunc(double(*f)(double*), A);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;f&lt;/code&gt; is the pointer to my function to minimize, and &lt;code&gt;A&lt;/code&gt; is my array of initial guesses, and it would return an array of the actual solution values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does such a function exist already? Otherwise, is there a name of a technique I can search for, again I'm not too familiar with this field, so something like &quot;Jones' Method&quot; could get me started in the right direction, I just need to know what I'm looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;br&gt;&#xA;As @TylerOlsen indicated, depending on the number of nodes and number of links, it is possible that this system may be under-constrained. Are there techniques that can provide &lt;strong&gt;a&lt;/strong&gt; valid solution, even if there is no &lt;strong&gt;unique&lt;/strong&gt; solution? The background to this problem is I'm working on a certain flavor of shortest-path algorithm, and as long as the coordinates respect the edge lengths, I think &lt;em&gt;any&lt;/em&gt; valid solution should work.&lt;/p&gt;&#xA;" OwnerUserId="16853" LastEditorUserId="16853" LastEditDate="2015-07-06T15:20:39.073" LastActivityDate="2015-07-08T00:25:14.990" Title="Minimization of non-linear function" Tags="|optimization|c++|graph-theory|" AnswerCount="3" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20090" PostTypeId="2" ParentId="20089" CreationDate="2015-07-06T20:27:47.777" Score="2" Body="&lt;p&gt;I think better approach is to use 'fzero()' function like this:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/Q7AxV.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;then simply use (1) and (2) to find Y_B and Y_D&lt;/p&gt;&#xA;&#xA;&lt;p&gt;use same trick for other triangles&lt;/p&gt;&#xA;" OwnerUserId="16584" LastActivityDate="2015-07-06T20:27:47.777" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20091" PostTypeId="1" AcceptedAnswerId="20095" CreationDate="2015-07-06T20:51:25.990" Score="5" ViewCount="316" Body="&lt;p&gt;The Method of manufactured is commonly used for verification of computational science codes. I want to use the method for verification of Navier-Cauchy (elasticity) equations with &lt;a href=&quot;https://en.wikipedia.org/wiki/Periodic_boundary_conditions&quot; rel=&quot;noreferrer&quot;&gt;periodic&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Bloch_wave&quot; rel=&quot;noreferrer&quot;&gt;Bloch-periodic&lt;/a&gt; boundary conditions. In this case, the coefficients of the PDE (material propertise) are periodic, and the form of the boundary value problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla\cdot \sigma + \mathbf{f} = -\omega^2 \rho \mathbf{u}\quad \forall \mathbf{x} \in \Omega\\&#xA;\mathbf{u}(\mathbf{x} + \mathbf{a}) = \mathbf{u}(\mathbf{x})e^{\mathbf{k}\cdot\mathbf{a}}\quad \forall \mathbf{x} \in \Gamma_u\\&#xA;\mathbf{t}(\mathbf{x} + \mathbf{a}) = -\mathbf{t}(\mathbf{x})e^{\mathbf{k}\cdot\mathbf{a}}\quad \forall \mathbf{x} \in \Gamma_t&#xA;$$&#xA;with&#xA;$$\begin{align}&#xA;&amp;amp;\sigma = \underline{C} \epsilon\\&#xA;&amp;amp;\epsilon = \frac{1}{2}\left[\nabla\mathbf{u} + \nabla\mathbf{u}^T\right] \enspace ,&#xA;\end{align}$$&#xA;and $\underline{C}$ presents the desired periodicity, i.e., $\underline{C}(\mathbf{x} + \mathbf{a}) = \underline{C}(\mathbf{x})$, being $\mathbf{a}$ the vector periodicity of the material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When $\omega=0$ the Bloch-periodic conditions turn into &lt;em&gt;common&lt;/em&gt; periodic conditions and $\mathbf{u}(\mathbf{x} + \mathbf{a}) = \mathbf{u}(\mathbf{x})$ (similar for the tractions $\mathbf{t}(\mathbf{x})$). In that case one can propose a solution that presents the same periodicity of the material, e.g&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{u} = \left[e^{\sin \left(2\,x\right)\,\sin y}-\cos \left(3\,x\right)\,\sin ^2&#xA; \left(2\,y\right)\right] (u_1, u_2, 0)$$&#xA;and the periodicity for the material being&#xA;$$\underline{C} = \left[\sin ^2x\,\sin \cos y^3+\sin \cos x^2\,\sin ^4y\right] \underline{C}_0 \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then, plug these terms in the equations and obtain the expression for $\mathbf{f}$ (this can be done symbolically using Maxima or SymPy and then generate Fortran, C++ or Python code).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of Bloch-periodicity it is a little bit more convoluted, since the functions that solve the equation are not periodic. They are the product of two periodic functions, and just periodic in the particular case of commensurate wavelengths. Thus, in this case the solution would depend on the wavevector $\mathbf{k}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; It is customary to assume that the material properties are composites where the properties change abruptly, e.g., a bilayer material, fiber reinforced composites. In these cases the constitutive tensor is not a differentiable function. If the code is verified with smooth functions what can be said about the case of non-differentiable material properties? Is the test still valid?&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-07-07T01:35:09.040" Title="Method of Manufactured Solutions for non-differentiable coefficients" Tags="|pde|finite-element|verification|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20092" PostTypeId="1" AcceptedAnswerId="20093" CreationDate="2015-07-06T22:37:20.583" Score="4" ViewCount="1859" Body="&lt;p&gt;I am attempting to write a matrix multiplication routine because I need to do some analysis in &lt;code&gt;CUDA&lt;/code&gt; and I want to validate it with CPU code. I am trying to use &lt;code&gt;dgemm&lt;/code&gt; to do this, but I seem to be doing it wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My matrices are constructed as &lt;code&gt;1D&lt;/code&gt; arrays since the same thing is done in &lt;code&gt;CUDA&lt;/code&gt; kernels, so I am attempting to call &lt;code&gt;dgemm&lt;/code&gt; with &lt;code&gt;1D&lt;/code&gt; matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I use random numbers as my input matrices I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = [ ...&#xA;        0.840187717154710        0.394382926819093        0.783099223758606&#xA;        0.798440033476073        0.911647357936784        0.197551369293384&#xA;        0.335222755714889        0.768229594811904        0.277774710803188&#xA;];&#xA;B = [ ...&#xA;        0.553969955795431        0.477397051862160        0.628870924761924&#xA;        0.364784472791843        0.513400910195616        0.952229725174713&#xA;        0.916195068003701        0.635711727959901        0.717296929432683&#xA;];&#xA;myC = [ ...&#xA;        1.057423514989923        1.136811309272234        0.702808322919262&#xA;        1.035616345918396        1.343436407318801        0.651590826816703&#xA;        1.517807789358121        1.491925339118482        1.042304316032681&#xA;];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as my output, which, as I found out, is actually &lt;code&gt;B*A&lt;/code&gt;. This happens consistently - when I attempt to multiply square matrices, &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt;, I actually get&lt;code&gt;B*A&lt;/code&gt;. I am testing this because when I do non-square matrices, it's just a mess - I can't even trace what is happening!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My &lt;code&gt;C&lt;/code&gt; code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void test_host_matmult() {&#xA;    fprintf(stderr, &quot;Inside test_host_matmult\n&quot;);&#xA;    int nrA = 3;&#xA;    int ncA = 3;&#xA;&#xA;    int nrB = 3;&#xA;    int ncB = 3;&#xA;&#xA;    double* A;&#xA;    double* B;&#xA;    double* C;&#xA;&#xA;    A = (double*)malloc(nrA*ncA*sizeof(double));&#xA;    B = (double*)malloc(nrB*ncB*sizeof(double));&#xA;    C = (double*)malloc(nrA*ncB*sizeof(double));&#xA;&#xA;    //int icount = 0;&#xA;    for(int ir = 0; ir &amp;lt; nrA; ir++) {&#xA;        for(int ic = 0; ic &amp;lt; ncA; ic++) {&#xA;            //A[ir*ncA + ic] = 10*(ir+1) + (ic+1);&#xA;            A[ir*ncA + ic] = rand_double(0.0, 1.0);&#xA;            //fprintf(stderr, &quot;%i ... %i\n&quot;, icount++, 10*(ir+1) + (ic+1));&#xA;        }&#xA;    }&#xA;    for(int ir = 0; ir &amp;lt; nrB; ir++) {&#xA;        for(int ic = 0; ic &amp;lt; ncB; ic++) {&#xA;            //B[ir*ncB + ic] = 10*(ir+1) + (ic+1);&#xA;            B[ir*ncB + ic] = rand_double(0.0, 1.0);&#xA;        }&#xA;    }&#xA;&#xA;    matmult(A, nrA, ncA, B, nrB, ncB, C, nrA, ncB);&#xA;&#xA;    fprintf(stderr, &quot;A = [ ...\n&quot;);&#xA;    for(int ir = 0; ir &amp;lt; nrA; ir++) {&#xA;        for(int ic = 0; ic &amp;lt; ncA; ic++) {&#xA;            fprintf(stderr, &quot;%25.15f&quot;, A[ir*ncA + ic]);&#xA;        }&#xA;        fprintf(stderr, &quot;\n&quot;);&#xA;    }&#xA;    fprintf(stderr, &quot;];\n&quot;);&#xA;&#xA;    fprintf(stderr, &quot;B = [ ...\n&quot;);&#xA;    for(int ir = 0; ir &amp;lt; nrB; ir++) {&#xA;        for(int ic = 0; ic &amp;lt; ncB; ic++) {&#xA;            fprintf(stderr, &quot;%25.15f&quot;, B[ir*ncB + ic]);&#xA;        }&#xA;        fprintf(stderr, &quot;\n&quot;);&#xA;    }&#xA;    fprintf(stderr, &quot;];\n&quot;);&#xA;&#xA;    fprintf(stderr, &quot;myC = [ ...\n&quot;);&#xA;    for(int ir = 0; ir &amp;lt; nrA; ir++) {&#xA;        for(int ic = 0; ic &amp;lt; ncB; ic++) {&#xA;            fprintf(stderr, &quot;%25.15f&quot;, C[ir*ncB + ic]);&#xA;        }&#xA;        fprintf(stderr, &quot;\n&quot;);&#xA;    }&#xA;    fprintf(stderr, &quot;];\n&quot;);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and here is the &lt;code&gt;matmult&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void matmult( double* A, int nrA, int ncA, double* B, int nrB, int ncB, double* C, int nrC, int ncC) {&#xA;    // (nrA x ncA) . (nrB x ncB ) = (nrA x ncB)&#xA;    if( !(nrA == nrC &amp;amp;&amp;amp; ncB == ncC) ) { &#xA;        fprintf(stderr, &quot;ERROR: incorrect matrix sizes!!\n&quot;);&#xA;    }   &#xA;    char TRANS = 'N';&#xA;    int M = nrA;&#xA;    int N = ncB;&#xA;    int K = ncA;&#xA;    double alpha = 1.0;&#xA;    double beta = 0.0;&#xA;    int lda = nrA;&#xA;    int ldb = nrB;&#xA;    int ldc = nrC;&#xA;    dgemm_(&amp;amp;TRANS, &amp;amp;TRANS, &amp;amp;M, &amp;amp;N, &amp;amp;K, &amp;amp;alpha, A, &amp;amp;lda, B, &amp;amp;ldb, &amp;amp;beta, C, &amp;amp;ldc);&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="164" LastActivityDate="2015-07-06T23:06:40.507" Title="What is wrong with this matrix multiplication?" Tags="|matrix|c|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20093" PostTypeId="2" ParentId="20092" CreationDate="2015-07-06T22:59:37.570" Score="4" Body="&lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; I haven't run your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps this is a problem with the row-major/column-major conventions at play here: &lt;a href=&quot;http://docs.nvidia.com/cuda/cublas/index.html#data-layout&quot; rel=&quot;nofollow&quot;&gt;http://docs.nvidia.com/cuda/cublas/index.html#data-layout&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seem to use row-major matrices, but BLAS uses the column-major convention. When you pass row-major $A$ and $B$ to &lt;code&gt;dgemm&lt;/code&gt;, it implicitly interprets them as $A^t$ and $B^t$, giving the output matrix $A^tB^t$. When you read it back, you treat it as row-major, thus implicitly applying another transpose: $(A^tB^t)^t = BA$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect you also might have a problem in your code with non-square matrices, because then the leading dimension parameters and matrix sizes that you pass to &lt;code&gt;dgemm&lt;/code&gt; would be wrong.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-07-06T23:06:40.507" LastActivityDate="2015-07-06T23:06:40.507" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20094" PostTypeId="2" ParentId="20089" CreationDate="2015-07-06T23:52:18.173" Score="2" Body="&lt;p&gt;There are many methods to minimize a multi-dimensional function. &lt;a href=&quot;https://en.wikipedia.org/wiki/Mathematical_optimization#Computational_optimization_techniques&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is a list of some standard ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a good description along with some sample code you can check out &lt;a href=&quot;http://apps.nrbook.com/c/index.html&quot; rel=&quot;nofollow&quot;&gt;Numerical Recipes&lt;/a&gt;. In particular checkout chapter 10 which covers minimization and maximization of functions. I've implemented their simplex method to solve 2D problems like yours with good success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although as with most non-linear solvers, it requires an initial estimate. You'll find the same issue in your problem as you have probably already noticed that there are multiple solutions since you can flip triangles and get the same solution.&lt;/p&gt;&#xA;" OwnerUserId="9813" LastActivityDate="2015-07-06T23:52:18.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20095" PostTypeId="2" ParentId="20091" CreationDate="2015-07-07T01:35:09.040" Score="4" Body="&lt;p&gt;Let me see if I can turn my comment into a fully-fledged answer. Suppose you are solving the Laplacian with a variable $k$ which has a step function across the domain. Suppose you have homogeneous boundary conditions so that we don't have to deal with that in my answer. Now your weak form will look something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_\Omega k\nabla u \cdot \nabla v \; dx = \int_\Omega f v \;dx \;\;\forall v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you want to have $u=\tanh(x)\sin(x)$, in the strong form, you can't differentiate $k$, because it's discontinuous, but you can compute &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_\Omega k\nabla (\tanh(x)\sin(x)) \cdot \nabla v \; dx = \int_\Omega f v \;dx \;\;\forall v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, given that, instead of trying to compute $f$ from this, you can just implement the above as your right-hand side. I.e., solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_\Omega k\nabla u_h \cdot \nabla v_h \; dx = \int_\Omega k\nabla (\tanh(x)\sin(x)) \cdot \nabla v_h \; dx \;\;\forall v_h&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using your favorite $k$. As long as your integration makes sense, and as long as you can take that single derivative of your chosen solution, you should be good to go.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-07-07T01:35:09.040" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20096" PostTypeId="1" AcceptedAnswerId="20098" CreationDate="2015-07-07T06:49:22.070" Score="2" ViewCount="2190" Body="&lt;p&gt;I'm interested in solving an electrostatics problem in 2d case in some domain with a conductor placed inside the domain. &#xA;From a mathematical point of view, I have to solve a Poisson equation &lt;img src=&quot;https://i.stack.imgur.com/l5seM.png&quot; alt=&quot;poisson&quot;&gt;  with user-defined boundary conditions (let us consider a rectangular domain for simplicity) and a certain region in the domain with a constant user-defined potential (see the figure).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/5acR0.png&quot; alt=&quot;scheme&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a finite difference method to solve the Poisson equation numerically, which means I have to construct a grid in the domain and then attempt to find values of the potential at the nodes of the grid. &#xA;Recall that in the finite difference method, we write an equation for the unknown potential at each node of the grid.&#xA;Combined together, these equations form a system of linear equations. &#xA;Solution of this system is an approximate solution to the Poisson equation in the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to construct and solve the system of linear equations for the unknow potential in a rectangular domain without any &quot;inner objects&quot;.&#xA;My question is: how should I approach the problem, when there are some &quot;inner objects&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current idea is to record each node the object occupies (orange on the figure) and then cross out the corresponding equations from the linear system. &#xA;I would also have to modify the equations corresponding to the nodes that happen to lie near the boundary of the object.&#xA;Having done that, it should be possible solve the system. &#xA;Then, using this solution and the user-defined value of the potential on the conductor, it is possible to restore the potential in the full domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it the only possible strategy or are there any simpler ways to do this?&#xA;I'm interested in the approach, most reasonable from a programming point of view. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW: I use PETSc to solve linear system. Could it possibly have a certain feature for such case?&lt;/p&gt;&#xA;" OwnerUserId="7730" LastActivityDate="2015-07-15T20:37:53.767" Title="How to solve a Poisson equation using the finite difference method when there is an object inside a domain?" Tags="|finite-difference|poisson|electromagnetism|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20097" PostTypeId="2" ParentId="20052" CreationDate="2015-07-07T08:55:02.750" Score="1" Body="&lt;p&gt;So I managed to create a spiral finally. The method I used to initiate spiral is by initializing the mesh with a set of initial values expect a small square patch which is initialized with $180°$ phase shift with others and some having $90°$ phase shift. The link to spiral video I created is &lt;a href=&quot;https://www.youtube.com/watch?v=Jj20OUnY_m4&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5003" LastEditorUserId="5003" LastEditDate="2015-07-10T06:36:52.957" LastActivityDate="2015-07-10T06:36:52.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20098" PostTypeId="2" ParentId="20096" CreationDate="2015-07-07T12:59:50.030" Score="5" Body="&lt;p&gt;&lt;strong&gt;If you have simple, grid-aligned interior objects or accuracy is not crucial then stick with the method you have described. If you need to accurately represent arbitrary boundary shapes then you're probably best off moving to a (more complicated) finite element approach on an unstructured grid.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have described the simplest approach to solving this problem. If the interior objects can only be exactly aligned with your grid (e.g. rectangles in a rectangular grid) this approach will work very well. However, you are discretizing the boundary onto the chosen grid which results in low (first?) order accuracy at the interior boundary regardless of the order of accuracy of the finite difference discretization you use. Depending on your goals, this might be ok. As is often the case in solving PDEs, more complex boundary shapes (interior or exterior) create significant difficulties. The most common solution for the Poisson equation would be the use of a body-fitted grid or unstructured mesh. Most typical would be using finite elements on an unstructured triangular mesh (e.g. &lt;a href=&quot;http://persson.berkeley.edu/228A/Fall10/doc/lec33-2x3.pdf&quot;&gt;pdf link&lt;/a&gt;). This requires dealing with grid generation and typically requires more complex data structures for managing the relationships between elements and the other challenges of dealing with the unstructured mesh.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-07-07T12:59:50.030" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20099" PostTypeId="2" ParentId="20071" CreationDate="2015-07-07T16:45:00.497" Score="1" Body="&lt;p&gt;This is really a comment, but I am one point short of being able to comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you show us the rest of your optimization problem, to include clearly identifying all optimization (a.k.a decision) variables, as well as other constraints?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are your f(x1) actually linear in the optimization variables? Can you evaluate the gradient of the objective function by PDE solve as opposed to requiring finite differences?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much (magnitude and locations) of nonnegativity violation can you tolerate?  Can your objective function be evaluated if nonnegativity is violated?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why are you assuming the solution is smooth, and even if it is, why does that mean there will not be &quot;smooth&quot; violations between grid points?  Perhaps you want an adaptive or multi-grid approach in which the grid is made finer as the overall algorithm progress to a solution, or for instance, after initial convergence of the optimization algorithm, put in a finer grid and re-optimize, using the final solution from the coarser grid optimization as your starting value for a new optimization using a finer grid.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2015-07-07T17:03:39.813" LastActivityDate="2015-07-07T17:03:39.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20100" PostTypeId="2" ParentId="20089" CreationDate="2015-07-08T00:25:14.990" Score="1" Body="&lt;p&gt;After looking through various non-linear solver libraries, I've found one that adequately solves my problem, &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLopt&lt;/a&gt;. They have implemented a &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;variety of optimization algorithms&lt;/a&gt; that are fairly well documented. The library itself has a C++ API (among other available languages)&lt;/p&gt;&#xA;&lt;p&gt;For the problem I described, the algorithm I selected was &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms#PRAXIS_.28PRincipal_AXIS.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;nlopt::algorithm::LN_PRAXIS&lt;/code&gt;&lt;/a&gt;. This is described as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;PRAXIS&amp;quot; &lt;strong&gt;gradient-free local optimization&lt;/strong&gt; via the &amp;quot;principal-axis method&amp;quot; of Richard Brent, based on a C translation of Fortran code downloaded from &lt;a href=&quot;http://en.wikipedia.org/wiki/Netlib&quot; rel=&quot;nofollow noreferrer&quot;&gt;Netlib&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This algorithm was originally designed for &lt;strong&gt;unconstrained optimization&lt;/strong&gt;. In NLopt, bound constraints are &amp;quot;implemented&amp;quot; in PRAXIS by the simple expedient of returning infinity (Inf) when the constraints are violated (this is done automatically—you don't have to do this in your own function). This seems to work, more-or-less, but appears to slow convergence significantly. If you have bound constraints, you are probably better off using COBYLA or BOBYQA.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Therefore, a gradient function (Jacobian) was not required, and the algorithm applied to unconstrained systems, which applied to my problem since the solution space for the unknown variables is all reals.&lt;/p&gt;&#xA;&lt;p&gt;To use this library, here is an example of what I had to do&lt;/p&gt;&#xA;&lt;p&gt;Set up an &lt;code&gt;nlopt::opt&lt;/code&gt; which is the optimizer object.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nlopt::opt optimizer{ nlopt::algorithm::LN_PRAXIS, xy.size()};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I passed it my algorithm choice, and the number of unknown variables (which were in a vector &lt;code&gt;xy&lt;/code&gt;). Then I called &lt;code&gt;set_min_objective&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;optimizer.set_min_objective(coord_estim::ObjectiveFunctionCallback, static_cast&amp;lt;void*&amp;gt;(&amp;amp;coord_est));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;ObjectiveFunctionCallback&lt;/code&gt; was a wrapper around my function &lt;code&gt;f&lt;/code&gt; that I described in my post, and the requirement for the callback function was that it has the signature&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;double ObjectiveFunctionCallback(const std::vector&amp;lt;double&amp;gt; &amp;amp;x, std::vector&amp;lt;double&amp;gt; &amp;amp;grad, void* f_data);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then set the stop condition, in my case I specified that the total error should be below some threshold, though &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Reference#Stopping_criteria&quot; rel=&quot;nofollow noreferrer&quot;&gt;there are other stop criteria&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;optimizer.set_stopval(0.01);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally, you call &lt;code&gt;optimize&lt;/code&gt; to perform the optimization&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;optimizer.optimize(xy, total_error);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first argument of &lt;code&gt;optimize&lt;/code&gt; will contain the optimized values, and the second argument contains the final value of your objective function corresponding to these values. &lt;code&gt;optimize&lt;/code&gt; also provides a &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Reference#Performing_the_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;return code&lt;/a&gt; so you know what caused the optimization to terminate.&lt;/p&gt;&#xA;" OwnerUserId="16853" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-07-08T00:25:14.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20101" PostTypeId="2" ParentId="20071" CreationDate="2015-07-08T00:59:52.620" Score="2" Body="&lt;p&gt;This is more of a comment, but I believe the more common name for this is &quot;positive trigonometric polynomial&quot;, so &lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-1-4020-5125-8&quot; rel=&quot;nofollow&quot;&gt;this book&lt;/a&gt; might be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach (&lt;a href=&quot;http://www.mit.edu/~parrilo/cdc03_workshop/Vandenberghe.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mit.edu/~parrilo/cdc03_workshop/Vandenberghe.pdf&lt;/a&gt;) is to use the result that the polynomial&#xA;$$ x(t) = r_0 + 2r_1\cos t + \cdots + 2r_n \cos nt $$&#xA;is nonnegative if and only if there is an $(n+1)\times(n+1)$ positive semidefinite matrix $Y\succeq 0$ such that&#xA;$$ r_k = \mathrm{tr}(E^k Y) = \sum_{0\leq j\leq n-k} Y_{j+k,j}, \qquad E = \begin{pmatrix}0&amp;amp;I_n\\0&amp;amp;0\end{pmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not really too familiar with this, unfortunately, and there exist other characterizations as well.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-07-08T02:57:47.967" LastActivityDate="2015-07-08T02:57:47.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20102" PostTypeId="2" ParentId="20079" CreationDate="2015-07-08T01:10:52.710" Score="5" Body="&lt;p&gt;When solving a nonlinear equation&#xA;$$ x = f(x), \qquad x\in\mathbb{R}^m, $$&#xA;the iteration method consists of generating successive guesses&#xA;$$ x_1 = f(x_0), \quad x_2 = f(x_1), \qquad x_k = f(x_{k-1}). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this converges to a limit $x_*$, that limit will satisfy the equation $x_* = f(x_*)$. Convergence is guaranteed by Taylor's theorem so long as $|f'|&amp;lt;1$ in a neighbourhood of $x_*$ and the initial guess is sufficiently close. This is discussed in every introductory numerical analysis textbook.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is easy to explain why your solution converges to an incorrect value: iterating the equation&#xA;$$ \eta = \eta \times (\cdots) $$&#xA;has only one fixed point, $\eta=0$, as the multiplier is less than 1. So this is solving the wrong equation, not the equations you wrote down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality you have just one equation in one unknown $r$, but you write it as two equations. So write the two equations as&#xA;$$ r = g(r, \eta_e), \qquad \eta_e = h(\eta, r). $$&#xA;Note that the two $\eta$'s are distinct: $\eta_e$ is an unknown, $\eta$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fixed point method is to iterate&#xA;$$ r_{k+1} = g(r_k, \eta_{e,k}), \qquad \eta_{e,k+1} = h(\eta, r_k). $$&#xA;Your code instead iterates&#xA;$$ r_{k+1} = g(r_k, \eta_{e,k}), \qquad \eta_{e,k+1} = h(\eta_{e,k}, r_k), $$&#xA;which replaces the original equation for $\eta_e$ with &#xA;$$ \eta_e = \eta_e / (1 + p/br). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; I haven't run your code and haven't checked for any other errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your loop should read:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## Iterative Approach&#xA;r =  sqrt(9*eta*v_Fall/(2*g*rho))&#xA;for i in range(0,N):&#xA;    eta_e = eta*(1 + b/(p*r))**(-1)&#xA;    if verbose==True:&#xA;        print(r,eta_e) # now it works&#xA;    r = sqrt(9*eta_e*v_Fall/(2*g*rho))&#xA;q = 4*pi*d/(3*U)*(g*rho*r**3 + 9/2*eta_e*r*v_Rise)&#xA;return array([r*1e6, r_ana*1e6, q/1.5793e-19, q_ana/1.5793e-19])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="9667" LastEditDate="2015-07-15T21:03:13.017" LastActivityDate="2015-07-15T21:03:13.017" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20103" PostTypeId="1" AcceptedAnswerId="20184" CreationDate="2015-07-08T02:50:20.083" Score="3" ViewCount="965" Body="&lt;p&gt;When using the pressure projection method to solve the incompressible Navier-Stokes equations do we apply Neumann boundary conditions for pressure only where there are associated no-slip velocity boundary conditions? For example suppose we were trying to solve the vortex shedding problem. The velocity boundary conditions are shown in the figure &lt;img src=&quot;https://i.stack.imgur.com/3gOJD.png&quot; alt=&quot;velocity boundary conditions&quot;&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then would I use a Neumann condition on pressure for the top, bottom, and circle? If so what about the left and right sides where the velocity is not no-slip? Would I have to specify a Dirichlet pressure here and if so then what would that be? I know that in order to solve the Poisson type problem for pressure that at least one node (I am using finite elements) must be specified in order for the system to have a unique solution, but should we be specifying pressure at all nodes where there is not a no-slip boundary condition on velocity? The related question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6932/flow-past-a-cylinder-projection-method-boundary-conditions?rq=1&quot;&gt;Flow past a cylinder - Projection Method - Boundary Conditions&lt;/a&gt; seemed to indicate that Neumann conditions on pressure are only used when that boundary also has a no-slip velocity boundary condition. Is this correct?&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-07-15T18:48:57.837" Title="Pressure projection method boundary conditions" Tags="|simulation|navier-stokes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20104" PostTypeId="1" CreationDate="2015-07-08T05:29:52.893" Score="1" ViewCount="580" Body="&lt;p&gt;&lt;strong&gt;Objective:&lt;/strong&gt; I am trying to simulate the following advection-diffusion-reaction equation in 2D space (x,y) and time. &#xA;$$\begin{align}&#xA;\text{ADR Equation: }\frac{\partial C}{\partial t} + \nabla\left(v.C - D\nabla{C} \right)=  \alpha.C&#xA;\end{align}$$&#xA;I discretized the above ADR equation in 2D using finite-difference implicit scheme and as a result I get the following discretized equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;p_1C^{n+1}_{i,j-1}+p_2C^{n+1}_{i-1,j}+p_3C^{n+1}_{i,j}+p_4C^{n+1}_{i+1,j}+p_5C^{n+1}_{i,j+1} = C^{n}_{i,j}&#xA;\end{align}$$&#xA;where, $p_1, p_2, p_3, p_4, p_5$ are constants in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve this as a system of equations using $A^{n+1}.C^{n+1}=C^{n}$, with no-flow i.e. $C=0$ outside the boundary domain. Here, $A^{n+1}$ would be a penta-diagonal, symmetric (not sure about this) and a diagonally dominant matrix. I have derived matrix $A$ for $2\times2$, $3\times3$ and $4\times4$ systems. For example, below you can see matrix $A$ for $3\times3$ and $4\times4$ systems, respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Issue:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I am not sure if the form of matrix $A$ I have derived is correct because as per my understanding it should be symmetric, however, it's not as per my derivations.&lt;/li&gt;&#xA;&lt;li&gt;Owing to an unsymmetric form of the matrix $A$ I need help to efficiently form $A$ for $N\times N$ system. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Would appreciate if someone could use their awesome numerical skills to answer these issues.&lt;/p&gt;&#xA;" OwnerDisplayName="user5510" LastEditorDisplayName="user5510" LastEditDate="2015-07-20T08:04:57.617" LastActivityDate="2015-09-30T20:54:25.273" Title="ADR equation implicit solution: Penta-diagonal matrix for a 2D $N\times N$ system" Tags="|finite-difference|matrix|advection-diffusion|implicit-methods|" AnswerCount="1" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="20105" PostTypeId="2" ParentId="20104" CreationDate="2015-07-08T07:17:39.963" Score="1" Body="&lt;p&gt;This is mostly just a matter of assigning an index to each vertex. If you have $n\times n$ unknowns $C_{i,j}$, $0\leq i,j\leq n-1$, then you would typically map the variable corresponding to the vertex $(i,j)$ to something like an index $I(i,j) = i + n j$. So each variable $C_{i,j}$ that is indexed by a pair of numbers $(i,j)$ is equivalent to the variable $C_{I(i,j)}$ indexed by a single number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this mapping, you rewrite the equations as&#xA;$$ p_1 C_{I(i,j)} + p_{2:3} C_{I(i\pm1,j)} + p_{4:5} C_{I(i,j\pm1)} = \cdots, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the $I(i,j)$-th row (the $(i+jn)$-th row) of the matrix $A^{n+1}$ has the entries $p_{1,2,3,4,5}$ in columns $I(i,j)$, $I(i\pm1,j)$, $I(i,j\pm1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this indexing method the nonzero entries will all fall into the diagonals $\pm n$ (coefficients $p_4$, $p_5$), $\pm1$ ($p_2$, $p_3$), and $0$ ($p_1$). The main thing to bear in mind, when constructing such a matrix knowing its diagonal elements, is that the diagonal elements for non-existent boundary variables such as $I(i,n)$, $I(i,-1)$, $I(-1,j)$, $I(n,j)$ should be zeroed out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for a $3\times 3$ grid, you would get the matrix (I dropped the minus signs here)&#xA;$$&#xA;\begin{pmatrix}&#xA; p_1 &amp;amp; p_2 &amp;amp; 0 &amp;amp; p_4 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA; p_3 &amp;amp; p_1 &amp;amp; p_2 &amp;amp; 0 &amp;amp; p_4 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA; 0 &amp;amp; p_3 &amp;amp; p_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; p_4 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA; p_5 &amp;amp; 0 &amp;amp; 0 &amp;amp; p_1 &amp;amp; p_2 &amp;amp; 0 &amp;amp; p_4 &amp;amp; 0 &amp;amp; 0 \\&#xA; 0 &amp;amp; p_5 &amp;amp; 0 &amp;amp; p_3 &amp;amp; p_1 &amp;amp; p_2 &amp;amp; 0 &amp;amp; p_4 &amp;amp; 0 \\&#xA; 0 &amp;amp; 0 &amp;amp; p_5 &amp;amp; 0 &amp;amp; p_3 &amp;amp; p_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; p_4 \\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; p_5 &amp;amp; 0 &amp;amp; 0 &amp;amp; p_1 &amp;amp; p_2 &amp;amp; 0 \\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; p_5 &amp;amp; 0 &amp;amp; p_3 &amp;amp; p_1 &amp;amp; p_2 \\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; p_5 &amp;amp; 0 &amp;amp; p_3 &amp;amp; p_1&#xA;\end{pmatrix} $$&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-07-08T07:17:39.963" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20106" PostTypeId="1" CreationDate="2015-07-08T13:50:35.897" Score="1" ViewCount="77" Body="&lt;p&gt;Could someone please explain what is meant with e.g &lt;a href=&quot;https://en.wikipedia.org/wiki/Flux_limiter&quot; rel=&quot;nofollow&quot;&gt;high precision low resolution flux&lt;/a&gt;. Does it mean, that I use some higher order method (still first order) but with a more complex stencil and consequently lower resolution ? While low precision and high resolution means essentially a forward or backward difference formula ? As a consequence, it is on me to choose the appropriate combinations of difference methods and flux limiter (e.g by solving an analytical form of my PDE prior to the more complex one ?)&lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2015-07-08T13:50:35.897" Title="Method of lines: explanation for high precision low resolution flux and vice versa" Tags="|method-of-lines|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20107" PostTypeId="1" AcceptedAnswerId="20131" CreationDate="2015-07-08T13:54:07.467" Score="2" ViewCount="538" Body="&lt;p&gt;Am implementing a monte carlo integration routine to compute this double integral in eqn 0.3 of page 2 of this paper 'Mobius energy of knots and unknots', Annals of Mathematics, &lt;a href=&quot;http://www.math.ucsb.edu/~zhenghwa/data/research/pub/Mobiusenergy-94.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.math.ucsb.edu/~zhenghwa/data/research/pub/Mobiusenergy-94.pdf&lt;/a&gt; in a case where both the integrals run over a circle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E(\gamma)=\int\int&#xA;\left(&#xA;\frac{1}{|\gamma(v)-\gamma(u)|^2}-\frac{1}{D(\gamma(v),\gamma(u))^2} &#xA;\right)&#xA;|\dot{\gamma}(u)||\dot{\gamma}(v)|\text{d}u\text{d}v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of trivia, the first author of the paper in the above link, Michael H. Freedman is a Fields Medalist. $\gamma(.)$ denotes the parametric form of the circle and returns a 2d vector. $D(\gamma(u),\gamma(v))$ denotes the shortest arc-length on the circle between two points on it. The following is the code I wrote. If that works, then I'd be able to generalize it to other parametric curves. Can you help me do the below integration, properly? In the below code I've computed total length of the curve(circle) using the differential geometric notion of $\int_0^{2\pi}||\gamma'(t)||dt$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q1: In the case of a double-integral when u &gt; v, R produces a NA or a negative result going by the rules of integration. In that case, how can a double integral be applied in a monte carlo routine, when both the integrals have the same limits of $0$ to $2\pi$ as there is always a chance of NA's occurring during the sample phase. Can you correct the implementation, so as to do the monte carlo double integration correctly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q2: Why is the monte carlo integration not producing a theoretical result of approximately 4, as suggested in the paper for the case of the circle?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(cubature)&#xA;I.2d &amp;lt;- function(z) {&#xA;flag=0&#xA;FUN = function(x){&#xA;    sqrt( (dxdt(x))^2 + (dydt(x))^2 + (dzdt(x))^2 )&#xA;&#xA;}&#xA;dxdt &amp;lt;- function(x) {return(-1*sin(x))}&#xA;dydt &amp;lt;- function(x) {return(cos(x))}&#xA;dzdt &amp;lt;- function(x) {return(0)}&#xA; x = z[1]&#xA; y = z[2]&#xA;x1Coord = cos(x);y1Coord = sin(x);z1Coord = 0&#xA;x2Coord = cos(y);y2Coord = sin(y);z2Coord = 0&#xA;EuclideanDist = c(x1Coord,y1Coord,z1Coord) - c(x2Coord,y2Coord,z2Coord) &#xA;EuclideanDist = 1 / sqrt(sum(EuclideanDist^2 ))&#xA;totalLength = integrate(FUN, 0,2*pi)$value&#xA;    arcLength = integrate(FUN, x,y)$value&#xA;             if(arcLength &amp;gt; (totalLength/2)){&#xA;                GeodesicDistance = totalLength - arcLength&#xA;             } &#xA;                         if(arcLength &amp;lt;= (totalLength/2)){&#xA;                         GeodesicDistance = arcLength&#xA;                        }&#xA;GeodesicDistance = 1 / (GeodesicDistance)^2 &#xA;if(is.na(EuclideanDist-GeodesicDistance))&#xA;{&#xA;show(&quot;NA Found&quot;)&#xA;return(0)&#xA;}&#xA;    if(!is.na(EuclideanDist-GeodesicDistance)){&#xA;    return(EuclideanDist-GeodesicDistance)&#xA;    }&#xA;}&#xA;&#xA;adaptIntegrate(I.2d, c(0, 0), c(2*pi, 2*pi), maxEval=10000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1934" LastEditorUserId="1934" LastEditDate="2015-07-10T22:09:17.347" LastActivityDate="2015-07-10T22:09:17.347" Title="Monte Carlo Double Integration Implementation" Tags="|numerical-analysis|computational-geometry|quadrature|monte-carlo|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20108" PostTypeId="1" CreationDate="2015-07-08T15:17:13.157" Score="3" ViewCount="761" Body="&lt;p&gt;I am new to this community as well as to scientific programming. I programmed a simple 4th order Runge-Kutta for the 1-D Cahn-Hilliard Equation for some first simple calculations on pattern forming systems. It turned out to be extremely sensitive to the change of the size of the spatial grid. With 100 grid elements the time step needs to be like 100 times smaller in order to converge ...&#xA;This is the code i used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;# Parameters&#xA;&#xA;k_c = 1.&#xA;eps = 0.5&#xA;L=12. # Domainsize&#xA;&#xA;size = 50 # Gridsize&#xA;dx = L/size&#xA;&#xA;T = 100 # total time&#xA;dt = .001 # time step&#xA;n = int(T/dt)&#xA;&#xA;# Random Initial Conditions&#xA;&#xA;U = np.random.normal(0,0.01,size) # Normalverteilte Zufallswerte um 0&#xA;&#xA;# Laplace-Operator&#xA;&#xA;def laplacian(Z):&#xA;    Zleft = np.roll(Z,1)&#xA;    Zright = np.roll(Z,-1)&#xA;    Zcenter = Z&#xA;    return (Zright + Zleft -2*Zcenter)/dx**2&#xA;&#xA;def DoubleLaplacian(Z):&#xA;    Z2left = np.roll(Z,2)&#xA;    Z2right = np.roll(Z,-2)&#xA;    Zleft = np.roll(Z,1)&#xA;    Zright = np.roll(Z,-1)&#xA;    Zcenter = Z&#xA;    return (Z2left - 4* Zleft + 6*Zcenter - 4*Zright + Z2right)/dx**4&#xA;&#xA;# Equation Body&#xA;&#xA;def equation(U):&#xA;     return laplacian(U**3)-2*eps/(k_c**2)*laplacian(U)-eps*(k_c**4)*DoubleLaplacian(U)&#xA;&#xA;# Runge-Kutta-Step&#xA;&#xA;def RK4Step(U, dt, equation):&#xA;    k1 = dt*equation(U)&#xA;    U_temp = U + k1/2; k2 = dt*equation(U_temp);&#xA;    U_temp = U + k2/2; k3 = dt*equation(U_temp);&#xA;    U_temp = U + k3; k4 = dt*equation(U_temp);&#xA;    U += (k1 + 2*k2 + 2*k3 + k4)/6&#xA;&#xA;# Integration&#xA;for i in range(n):&#xA;    RK4Step(U, dt, equation)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since I am new to this kind of stuff I wanted to ask whether there is a conceptual fault in my thinking or coding or it is simply due to the nonlinearity. I would be as well pretty happy to get some hints how to improve the performance of the code.&lt;/p&gt;&#xA;" OwnerUserId="16872" LastEditorUserId="9667" LastEditDate="2015-07-08T18:33:09.733" LastActivityDate="2015-07-09T14:58:41.147" Title="Simple Runge-Kutta Scheme for nonlinear PDE" Tags="|python|performance|runge-kutta|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20109" PostTypeId="2" ParentId="4946" CreationDate="2015-07-08T16:27:13.413" Score="2" Body="&lt;p&gt;do you happen to have any non-comment lines in your batch script before the #PBS -o line?  TORQUE will ignore any #PBS lines that occur after the first executable statement in the script.  The qsub command will scan your batch script for #PBS lines until it encounters a line that is not a comment or whitespace. &lt;/p&gt;&#xA;" OwnerUserId="14541" LastActivityDate="2015-07-08T16:27:13.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20110" PostTypeId="2" ParentId="112" CreationDate="2015-07-08T16:45:08.643" Score="2" Body="&lt;p&gt;old question, but I'm adding some additional information for anyone that lands here from Google&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that once a job is running in TORQUE, any request to lengthen the walltime will be ignored unless the qalter command is executed by a TORQUE manager or operator.  To extend the walltime of a running job, you must contact an administrator.  You can shorten the walltime of your own running jobs, and if your job is not running (queued or held)  you can increase the walltime with qalter.&lt;/p&gt;&#xA;" OwnerUserId="14541" LastActivityDate="2015-07-08T16:45:08.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20111" PostTypeId="1" CreationDate="2015-07-08T17:09:37.800" Score="0" ViewCount="148" Body="&lt;p&gt;I have a problem fitting a $N\times N$ ($32\times 32$ in this case) matrix in Matlab. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the deal &lt;img src=&quot;https://i.stack.imgur.com/1ll3b.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have this image, and I want to create 1 dimensional fit between the points and then be able to interpolate it in order to make it smoother. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any idea if that can be done?&lt;/p&gt;&#xA;" OwnerUserId="16873" LastEditorUserId="9667" LastEditDate="2015-07-08T18:27:30.993" LastActivityDate="2015-07-08T18:27:30.993" Title="1D fit on a surface in matlab" Tags="|matlab|image-processing|curve-fitting|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20112" PostTypeId="1" CreationDate="2015-07-08T19:39:00.140" Score="2" ViewCount="641" Body="&lt;p&gt;I have the following function that I want to implement in scipy.integrate.odeint &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def numericSolver(rv_,t,alpha=1.,gamma=1.):&#xA;    BVal = B_f(alpha,gamma,t) #this function returns a vector&#xA;    BdotVal = Bdot_f(alpha,gamma,t) #this function returns a vector&#xA;    matDynamics = np.array([[0.,1.,float(BdotVal[2]),0.,0.,0.],&#xA;            [0.,0.,0.,float(BVal[2]),0,-float(BVal[1])],&#xA;            [0.,0.,0.,1,0.,0.],&#xA;            [0.,-float(BVal[2]),0.,0.,0.,float(BVal[0])],&#xA;            [0.,0.,0.,0.,0.,1.],&#xA;            [0.,float(BVal[1]),0.,-float(BVal[0]),0.,0.]])&#xA;    derivatives = matDynamics.dot(rv_) &#xA;    return derivatives&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In order to implement this function I call odeint as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solution = odeint(numericSolver,IC,t=t,args=(alpha,gamma))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;IC&lt;/code&gt; is the initial condition vector, &lt;code&gt;t&lt;/code&gt; is the time vector of interest, and &lt;code&gt;alpha&lt;/code&gt; and &lt;code&gt;gamma&lt;/code&gt; are inputs for &lt;code&gt;numericSolver&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more clear explanation, &lt;code&gt;derivatives&lt;/code&gt; is the solution, $\dot{\vec{X}}$, to the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\dot{\vec{X}} = M \vec{X},&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $M$ is &lt;code&gt;matDynamics&lt;/code&gt; and &lt;code&gt;rv_&lt;/code&gt; is $\vec{X}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way that I can get odeint to return the values of &lt;code&gt;derivatives&lt;/code&gt; at each value in &lt;code&gt;t&lt;/code&gt;? Or is it best to calculate the derivatives values manually using the values returned in &lt;code&gt;solution&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="11799" LastActivityDate="2015-07-08T19:39:00.140" Title="Return derivative vector from odeint scipy function" Tags="|python|numerical-modelling|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20113" PostTypeId="1" CreationDate="2015-07-08T21:15:11.027" Score="14" ViewCount="17511" Body="&lt;p&gt;Let us consider a smooth initial condition and the heat equation in one dimension :&#xA;$$ \partial_t u = \partial_{xx} u$$&#xA;in the open interval $]0,1[$, and let us assume that we want to solve it numerically with finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that for my problem to be well-posed, I need to endow it with boundary conditions at $x=0$ and $x=1$. I know that Dirichlet or Neumann work well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have in the first case $N$ interior points $x_k=\frac{k}{N+1}$ for $k=1,\cdots,N$, then I have $N$ unknowns: $u_k=u(x_k)$ for $k=1,\cdots,N$, because $u$ is prescribed at the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second case I really have $N+2$ unknowns $u_0,\cdots,u_{N+1}$,&#xA;and I know how to use the (homogeneous) Neumann BC to discretize the laplacian at the border, for example with the adjunction of two fictious points $x_{-1}$ and $x_{N+2}$ and the equalities :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u_1-u_{-1}}{2 h} = 0 = \frac{u_{N+2}-u_N}{2 h}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about periodic BC. I have the feeling that I could use one equation, namely&#xA;$$u(0) = u(1)$$&#xA;but maybe two, and then I would use&#xA;$$\partial_x u(0) = \partial_x  u(1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I am not sure. I don't know either how many unknowns I should have. Is it $N+1$ ?&lt;/p&gt;&#xA;" OwnerUserId="7784" LastEditorUserId="7784" LastEditDate="2015-07-09T08:14:13.640" LastActivityDate="2015-07-10T12:35:39.263" Title="Periodic boundary condition for the heat equation in ]0,1[" Tags="|pde|boundary-conditions|parabolic-pde|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20114" PostTypeId="1" AcceptedAnswerId="20913" CreationDate="2015-07-08T21:24:43.583" Score="7" ViewCount="361" Body="&lt;p&gt;Let us say that I have a function like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(x):&#xA;    return g(x)*h(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, &lt;code&gt;g(x)&lt;/code&gt; and &lt;code&gt;h(x)&lt;/code&gt; are such that when &lt;code&gt;g(x)&lt;/code&gt; produces a result large enough to cause overflow errors, &lt;code&gt;h(x)&lt;/code&gt; produces a number that is approximately zero. So, the result provided by &lt;code&gt;f(x)&lt;/code&gt; should be $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be enough to write &lt;code&gt;f(x)&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(x):&#xA;    h_x = h(x)&#xA;    if abs(h_x) &amp;lt; 1e-9:&#xA;        return 0&#xA;    else:&#xA;        return g(x)*h_x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, is there a neater, more general way to write such a function? If one wants an example of functions &lt;code&gt;f&lt;/code&gt;, &lt;code&gt;g&lt;/code&gt; and &lt;code&gt;h&lt;/code&gt; that behave as described, consider:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;    h(x) &amp;amp;= \textrm{erf}(a) - \textrm{erf}(x) \\&#xA;    g(x) &amp;amp;= e^{x^2} \\&#xA;    f(x) &amp;amp;= g(x)h(x) \\&#xA;    x &amp;amp;= 30 \\&#xA;    a &amp;amp;= 30&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2015-10-04T19:22:55.773" Title="How to avoid overflow error in program that computes product of two numbers, such that when one is big enough to cause overflow, other is $0$?" Tags="|numerical-analysis|programming-paradigms|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="20115" PostTypeId="2" ParentId="20113" CreationDate="2015-07-09T00:57:37.340" Score="-1" Body="&lt;p&gt;According to &lt;a href=&quot;https://web.stanford.edu/class/math220b/handouts/heateqn.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; you should impose periodic boundary conditions as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u(0, t) = u(1, t) \\&#xA;u_x(0, t) = u_x(1, t)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way of discretising the Heat Equation implicitly using backward Euler is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{u^{n+1}-u_{n}}{\Delta t}= \frac{u^{n+1}_{i+1}-2u^{n+1}_{i}+u^{n+1}_{i+1}}{\Delta x^2}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the system of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \begin{bmatrix}&#xA;  I-\frac{\Delta t}{\Delta x^2}A &#xA;    \end{bmatrix}&#xA;  \begin{bmatrix}&#xA;    u^{n+1}_1 \\&#xA;    u^{n+1}_1 \\&#xA;     \vdots&#xA;     \\&#xA;     \\&#xA;     \\&#xA;    u^{n+1}_{N}&#xA;  \end{bmatrix}  &#xA; = &#xA;  \begin{bmatrix}&#xA;    u^n_1 \\&#xA;    u^n_2 \\&#xA;     \vdots&#xA;     \\&#xA;     \\&#xA;     \\&#xA;    u^n_{N}&#xA;  \end{bmatrix}  &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &#xA;\begin{equation}&#xA; A=&#xA; \begin{bmatrix}&#xA;    -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0 \\&#xA;    1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp;\ldots &amp;amp; 0 \\&#xA;    0 &amp;amp; 1  &amp;amp; -2&amp;amp; 1 &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0 \\&#xA;    0 &amp;amp; 0  &amp;amp; 1&amp;amp; -2 &amp;amp; 1 &amp;amp; \ldots &amp;amp; 0 \\&#xA;    0 &amp;amp; 0  &amp;amp; 0&amp;amp; 1 &amp;amp; -2 &amp;amp; \ldots &amp;amp; 0 \\&#xA;    \vdots  &amp;amp;\vdots &amp;amp;\vdots &amp;amp;\vdots &amp;amp; \ddots &amp;amp;\ddots &amp;amp;\vdots \\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \ldots &amp;amp;0  &amp;amp;1 &amp;amp;-2&#xA;  \end{bmatrix}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your periodic boundary conditions can be included by adding two more equations and two additional(ghost) cells $u_{0}$ and $u_{N+1}$ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_1 - u_{N} = 0 \\&#xA;\frac{u_{2}-u_{0}}{2 \Delta x} - \frac{u_{N+1}-u_{N-1}}{2 \Delta x} = 0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;http://marian.fsik.cvut.cz/~halama/nopr/amath58X.pdf&quot; rel=&quot;nofollow&quot;&gt;Section 2.11 LeVeque&lt;/a&gt; this gives you a 2nd order accuracy for $u_x$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally your system of equations will look like:&#xA;\begin{equation}&#xA;  \begin{bmatrix}&#xA;    0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 \\&#xA;    -1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \ldots &amp;amp; 1 &amp;amp; 0 &amp;amp; -1 \\&#xA;    0 &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; 0\\&#xA;    0 &amp;amp; &amp;amp; &amp;amp;  &amp;amp; &amp;amp; &amp;amp; &amp;amp;  0\\&#xA;    0 &amp;amp; &amp;amp; &amp;amp; I-\frac{\Delta t}{\Delta x^2}A &amp;amp; &amp;amp; &amp;amp; &amp;amp;  0\\&#xA;    0 &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp;  0\\&#xA;    0 &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp;  0\\&#xA;    0 &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp; 0 &#xA;  \end{bmatrix}&#xA;  \begin{bmatrix}&#xA;    u^{n+1}_{0}\\&#xA;    u^{n+1}_1 \\&#xA;    u^{n+1}_2 \\&#xA;     \\&#xA;     \\&#xA;     \\&#xA;     \\&#xA;    u^{n+1}_{N} \\&#xA;    u^{n+1}_{N+1} &#xA;  \end{bmatrix}  &#xA; = &#xA;  \begin{bmatrix}&#xA;    0 \\&#xA;    0 \\&#xA;    u^n_1 \\&#xA;    u^n_2 \\&#xA;     \\&#xA;     \\&#xA;     \\&#xA;     \\&#xA;    u^n_{N}&#xA;  \end{bmatrix}  &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which gives you N+2 equations and N+2 unknowns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also get rid of the first to equations and the ghost cells and arrive at a system of N equations and N unknowns. &lt;/p&gt;&#xA;" OwnerUserId="11656" LastEditorUserId="7733" LastEditDate="2015-07-09T16:25:40.390" LastActivityDate="2015-07-09T16:25:40.390" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="20116" PostTypeId="2" ParentId="20108" CreationDate="2015-07-09T03:24:04.840" Score="3" Body="&lt;p&gt;Profiling your code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import cProfile, pstats, StringIO&#xA;pr = cProfile.Profile()&#xA;pr.enable()&#xA;# Integration&#xA;for i in range(n):&#xA;    RK4Step(U, dt, equation)&#xA;pr.disable()&#xA;s = StringIO.StringIO()&#xA;sortby = 'cumulative'&#xA;ps = pstats.Stats(pr, stream=s).sort_stats(sortby)&#xA;ps.print_stats()&#xA;print s.getvalue()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives the following : &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;         27300018 function calls in 103.787 seconds&#xA;&#xA;   Ordered by: cumulative time&#xA;&#xA;   ncalls  tottime  percall  cumtime  percall filename:lineno(function)&#xA;        2    0.000    0.000  103.787   51.894 /Users/Leo/anaconda/lib/python2.7/site-packages/IPython/core/interactiveshell.py:3005(run_code)&#xA;        1    0.466    0.466  103.787  103.787 &amp;lt;ipython-input-2-3cf9531a6a7d&amp;gt;:5(&amp;lt;module&amp;gt;)&#xA;   100000    4.605    0.000  103.317    0.001 &amp;lt;ipython-input-1-3a5916441c4c&amp;gt;:44(RK4Step)&#xA;   400000    9.648    0.000   98.712    0.000 &amp;lt;ipython-input-1-3a5916441c4c&amp;gt;:39(equation)&#xA;  3200000   16.343    0.000   63.233    0.000 /Users/Leo/anaconda/lib/python2.7/site-packages/numpy/core/numeric.py:1279(roll)&#xA;   800000   13.794    0.000   45.859    0.000 &amp;lt;ipython-input-1-3a5916441c4c&amp;gt;:23(laplacian)&#xA;   400000   12.037    0.000   43.205    0.000 &amp;lt;ipython-input-1-3a5916441c4c&amp;gt;:29(DoubleLaplacian)&#xA;  3200000   12.589    0.000   12.589    0.000 {numpy.core.multiarray.concatenate}&#xA;  6400000   12.371    0.000   12.371    0.000 {numpy.core.multiarray.arange}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can see how Laplacian and Double laplacian take about 88% of the computing time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can either rewrite your code in Fortran or C and wrap it with &lt;code&gt;f2py&lt;/code&gt; or rewrite those functions in &lt;code&gt;cython&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way is compiling your code using &lt;code&gt;Nuitka&lt;/code&gt;, this decreases runtime to 64%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also see that roll is what takes most of the time. So you can start working on an optimised version of that.   &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;         25600000 function calls in 64.852 seconds&#xA;&#xA;   Ordered by: cumulative time&#xA;&#xA;   ncalls  tottime  percall  cumtime  percall filename:lineno(function)&#xA;  3200000   17.494    0.000   64.852    0.000 /usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/numeric.py:1279(roll)&#xA;  6400000   12.644    0.000   12.644    0.000 {numpy.core.multiarray.arange}&#xA;  3200000   12.635    0.000   12.635    0.000 {numpy.core.multiarray.concatenate}&#xA;  3200000    4.499    0.000   10.167    0.000 /usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/numeric.py:464(asanyarray)&#xA;  3200000    6.472    0.000    6.472    0.000 {method 'take' of 'numpy.ndarray' objects}&#xA;  3200000    5.668    0.000    5.668    0.000 {numpy.core.multiarray.array}&#xA;  3200000    5.440    0.000    5.440    0.000 {method 'reshape' of 'numpy.ndarray' objects}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Finally, find a way to get your CFL number to define your $\Delta t$&lt;/p&gt;&#xA;" OwnerUserId="11656" LastEditorUserId="11656" LastEditDate="2015-07-09T14:58:41.147" LastActivityDate="2015-07-09T14:58:41.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20117" PostTypeId="1" CreationDate="2015-07-09T06:12:25.180" Score="1" ViewCount="653" Body="&lt;p&gt;I have a 3-D structured grid that I am trying to convert over to an unstructured grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a picture of the 2-D structured grid with the nodes shown. It is axisymmetric about the x axis (y = 0). I will rotate this to make it 3-D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main concern in this conversion is that I need to output the volume of every cell from the structured grid. But as you can see, my cells are irregular hexa/penta-hedras, with non-parallel opposing faces (correct me if I'm wrong and I really hope I'm wrong, but I don't think I am). Does anyone have any suggestions on how to compute the volume of each cell?&#xA;I have some ideas, but am open to other ideas. &#xA;A colleague of mine mentioned that I could break up the volumes into tetrahedras and compute the volume that way. That is my #1 option so far. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My ideas are not so simple and the algorithm can get pretty involved.&#xA;Does any one have a suggestion on how to compute the volume? &lt;/p&gt;&#xA;" OwnerUserId="16882" LastActivityDate="2015-07-13T08:13:10.277" Title="structured to unstructured grid - calculating volume of cell" Tags="|mesh-generation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20118" PostTypeId="2" ParentId="20117" CreationDate="2015-07-09T13:59:31.617" Score="2" Body="&lt;p&gt;Note that for any element shape, you can write (here dimension dim = 2, but 3D is similar)&#xA;$$&#xA;V = \frac{1}{dim}\int\limits_K \nabla \cdot (x,y)^T\,dx = &#xA;\frac{1}{2}\int\limits_{\partial K} (x,y)^T \cdot \mathbf{n}\,ds&#xA;$$&#xA;Using the second integral, you can loop over the faces of each element and integrate on its boundary to obtain volume $V$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a list of (unique) mesh faces, you can iterate over all faces and contribute to the volume of left/right element adjacent to each face with a $\pm$ sign depending on the orientation of the normal with respect to the element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does not completely solve your problem in the sense that even the cell faces might have other shapes that triangles and quadrilaterals (for which you can find quadrature formulas easily) and hence you might be anyway forced to do some element subdivision. This is perhaps still easier for 2D faces rather than 3D cells.&lt;/p&gt;&#xA;" OwnerUserId="4688" LastActivityDate="2015-07-09T13:59:31.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20119" PostTypeId="1" CreationDate="2015-07-09T15:47:50.863" Score="2" ViewCount="102" Body="&lt;p&gt;I have been wrestling with how to properly deal with bonded (1-2 1-3 and 1-4) particle neighbor interactions, or whether they have to be dealt with, in the long range electrostatic solver codes (PME, SPME, etc). For instance, in the short range calculation, these interactions are typically excluded by omitting them from the neighbor list, or are simply scaled by the appropriate factor (0.0, 0.5, 0.833 etc) in the short range calculation. However, I am confused as to where this would happen, or how it could happen, in the long range solver. In the long range calculation, we, to my understanding, perform the following steps.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;interpolate charges on the mesh, and obtain total charge density&lt;/li&gt;&#xA;&lt;li&gt;FFT and solve for the electrostatic field&lt;/li&gt;&#xA;&lt;li&gt;FFT back and obtain electrostatic potential&lt;/li&gt;&#xA;&lt;li&gt;interpolate forces back to particles&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am not seeing where in these steps we could exclude bonded neighbor interactions. The electric field is obtained via the total charge density which contains information from all particles, whether or not they are bonded. Is it even necessary to exclude these interactions here?&lt;/p&gt;&#xA;" OwnerUserId="16889" LastActivityDate="2015-07-09T17:32:22.460" Title="Proper handling of 1-2 1-3 1-4 bonded neighbors in long range electrostatic solver" Tags="|molecular-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20120" PostTypeId="2" ParentId="20119" CreationDate="2015-07-09T16:22:26.810" Score="2" Body="&lt;p&gt;You have to include the bonded interactions in the list of so-called masked pairs.  If you consult [1], this is the first term in equation (2.5).  That will subtract the contribution from the long-range (FFT-based) computation.  The third paragraph in section II of that paper explains it in detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the same correction term used in SPME is also valid in PME, since this term does not depend on the choice of interpolating functions used to compute the forces.  This is so because $E_{\text{rec}}$ can be written as (following the same notation as [1]):&#xA;$$E_{\text{rec}} = \frac{1}{2} \sum_{n}^{\star} \sum_{i,j=1}^N \frac{q_i q_j\text{erf}(\beta \, | r_j - r_i + n |)}{|r_j - r_i + n|} +\frac{\beta}{\sqrt{\pi}} \sum_{i=1}^N q_i^2,$$&#xA;so the first term in $E_{\text{corr}}$, namely&#xA;$$-\frac{1}{2} \sum_{(i,j)\in M} \frac{q_i q_j\text{erf}(\beta \, | r_j - r_i|)}{|r_j - r_i|}$$&#xA;takes care of removing the contribution of the nearest neighbors to the $E_{\text{rec}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Essmann, U., Perera, L., Berkowitz, M. L., Darden, T., Lee, H., &amp;amp; Pedersen, L. G. (1995). A smooth particle mesh Ewald method. The Journal of Chemical Physics, 103(19), 8577. doi:10.1063/1.470117&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2015-07-09T17:32:22.460" LastActivityDate="2015-07-09T17:32:22.460" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20122" PostTypeId="1" AcceptedAnswerId="20123" CreationDate="2015-07-09T19:24:10.300" Score="5" ViewCount="626" Body="&lt;p&gt;I have a recurrence relation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P_{n} = A_{n} P_{n-1} - B_{n}P_{n-2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with given $P_{0}$ and $P_{1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically, each $A_{n}$ and $B_{n}$ is calculated with some precision. The same applies to the $P_{0}$ and $P_{1}$. How can one estimates the error for the $n$-th term?&lt;/p&gt;&#xA;" OwnerUserId="14780" LastEditorUserId="9667" LastEditDate="2015-07-09T21:44:14.090" LastActivityDate="2015-07-10T01:48:47.730" Title="Error propagation in recurrence relation" Tags="|numerical-analysis|precision|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20123" PostTypeId="2" ParentId="20122" CreationDate="2015-07-09T23:57:20.710" Score="4" Body="&lt;p&gt;I think a good reference for this is &lt;em&gt;Error Bounds for Linear Recurrence Relations&lt;/em&gt; by F. W. J. Olver (Mathematics of Computation, 1988).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the most basic level, you can examine the sensitivity of $P_n$ to changes in $A_m,B_m$. So&#xA;$$ P_n+\delta P_n = (A_n+\delta A_n)(P_{n-1}+\delta P_{n-1}) - (B_n + \delta B_n)(P_{n-2} + \delta P_{n-2}), $$&#xA;which to first order leads to the recurrences&#xA;$$ P_n = A_nP_{n-1} - B_n P_{n-2}, $$&#xA;$$ \delta P_n = A_n \delta P_{n-1} - B_n \delta P_{n-2} + (\delta A_n P_{n-1} - \delta B_n P_{n-2}). $$&#xA;As you can see, the error terms satisfy the same recurrence relation, but with an extra inhomogeneous term that depends on the errors in $A_n,B_n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can find a general solution to the inhomogeneous recurrence relation, and also put a bound on the inhomogeneous term, that would give you a bound on the errors in $P_n$. This is analogous to how truncation error of a finite difference method for ODEs is converted into a bound on the method's global error. This also usually goes by the name of forward-mode sensitivity analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One issue here is that often the errors in $A_n,B_n$ are not independent, and this analysis can be overly conservative; a similar issue can arise with using interval arithmetic to analyse roundoff errors in the recurrence relation. Olver's paper shows how to get rigorous bounds.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-07-10T01:48:47.730" LastActivityDate="2015-07-10T01:48:47.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20124" PostTypeId="2" ParentId="20113" CreationDate="2015-07-10T00:53:38.763" Score="11" Body="&lt;p&gt;The best way to do this is (as you said) to just use the definition of periodic boundary conditions and set up your equations correctly from the start using the fact that $u(0)=u(1)$. In fact, even more strongly, &lt;strong&gt;periodic boundary conditions identify $x=0$ with $x=1$&lt;/strong&gt;. For this reason, you should only have one of these points in your solution domain. An open interval does not make sense when using periodic boundary conditions since &lt;em&gt;there is no boundary&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This fact means that you should not place a point at $x=1$ since it is the same as $x=0$. Discretizing with $N+1$ points, you then use the fact that &lt;strong&gt;by definition, the point to the left of $x_0$ &lt;em&gt;is&lt;/em&gt; $x_N$ and the point to the right of $x_N$ &lt;em&gt;is&lt;/em&gt; $x_0$&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/j9IXp.png&quot; alt=&quot;schematic of a periodic grid&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your PDE can then be discretized in space as&#xA;$$&#xA;\frac{\partial}{\partial t}&#xA;\left[\begin{array}{c}&#xA;x_0 \\&#xA;x_1 \\&#xA;\vdots \\&#xA;x_N&#xA;\end{array}\right]&#xA;=&#xA;\frac{1}{\Delta x^2}&#xA;\left[\begin{array}{c}&#xA;x_N - 2x_0 + x_1 \\&#xA;x_0 - 2x_1 + x_2 \\&#xA;\vdots \\&#xA;x_{N-1} - 2x_N + x_0&#xA;\end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be written in matrix form as&#xA;$$&#xA;\frac{\partial}{\partial t}\vec{x}&#xA;=&#xA;\frac{1}{\Delta x^2} \mathbf{A} \vec{x}&#xA;$$&#xA;where&#xA;$$&#xA;\mathbf{A} = &#xA;\left[\begin{array}{c}&#xA;-2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 \\&#xA;1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA; &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots \\&#xA;  &amp;amp;&amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots \\&#xA;0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1 \\&#xA;1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 &amp;amp; -2&#xA;\end{array}\right].&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course there is no need to actually create or store this matrix. The finite differences should be computed on the fly, taking care to handle the first and last points specially as needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a simple example, the following MATLAB script solves &#xA;$$&#xA;\partial_t u = \partial_{xx}u + b(t,x)&#xA;$$&#xA;with periodic boundary conditions on the domain $x\in[-1,1)$. The manufactured solution $u_\text{Ref}(t,x) = \exp(-t)\cos(5\pi x)$ is used, meaning $b(t,x) = (25\pi^2-1)\exp(-t)\cos(5\pi x)$. I used forward Euler time discretization for simplicity and computed the solution both with and without forming the matrix. The results are shown below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;&#xA;% Solve: u_t = u_xx + b&#xA;% with periodic boundary conditions&#xA;&#xA;% analytical solution:&#xA;uRef = @(t,x) exp(-t)*cos(5*pi*x);&#xA;b = @(t,x) (25*pi^2-1)*exp(-t)*cos(5*pi*x);&#xA;&#xA;% grid&#xA;N = 30;&#xA;x(:,1) = linspace(-1,1,N+1);&#xA;&#xA;% leave off 1 point so initial condition is periodic&#xA;% (doesn't have a duplicate point)&#xA;x(end) = [];&#xA;uWithMatrix = uRef(0,x);&#xA;uNoMatrix = uRef(0,x);&#xA;&#xA;dx = diff(x(1:2));&#xA;dt = dx.^2/2;&#xA;&#xA;%Iteration matrix:&#xA;e = ones(N,1);&#xA;A = spdiags([e -2*e e], -1:1, N, N);&#xA;A(N,1) = 1;&#xA;A(1,N) = 1;&#xA;A = A/dx^2;&#xA;&#xA;%indices (left, center, right) for second order centered difference&#xA;iLeft = [numel(x), 1:numel(x)-1]';&#xA;iCenter = (1:numel(x))';&#xA;iRight = [2:numel(x), 1]';&#xA;&#xA;%plot&#xA;figure(1)&#xA;clf&#xA;hold on&#xA;h0=plot(x,uRef(0,x),'k--','linewidth',2);&#xA;h1=plot(x,uWithMatrix);&#xA;h2=plot(x,uNoMatrix,'o');&#xA;ylim([-1.2, 1.2])&#xA;legend('Analytical solution','Matrix solution','Matrix-free solution')&#xA;ht = title(sprintf('Time t = %0.2f',0));&#xA;xlabel('x')&#xA;ylabel('u')&#xA;drawnow&#xA;&#xA;for t = 0:dt:1&#xA;    uWithMatrix = uWithMatrix + dt*( A*uWithMatrix + b(t,x) );&#xA;    uNoMatrix = uNoMatrix + dt*(  ( uNoMatrix(iLeft) ...&#xA;                                - 2*uNoMatrix(iCenter) ...&#xA;                                  + uNoMatrix(iRight) )/dx^2 ...&#xA;                                + b(t,x) );&#xA;    set(h0,'ydata',uRef(t,x))&#xA;    set(h1,'ydata',uWithMatrix)&#xA;    set(h2,'ydata',uNoMatrix)&#xA;    set(ht,'String',sprintf('Time t = %0.2f',t))&#xA;    drawnow&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/HRWFG.png&quot; alt=&quot;Plot of initial condition&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/RMkbO.png&quot; alt=&quot;Plot of solution at t = 0.5&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZoVwG.png&quot; alt=&quot;Plot of solution at t = 1.0&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/K0bgF.png&quot; alt=&quot;Plot of solution at t = 2.0&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-07-10T12:35:39.263" LastActivityDate="2015-07-10T12:35:39.263" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20125" PostTypeId="1" CreationDate="2015-07-10T09:50:13.090" Score="1" ViewCount="241" Body="&lt;p&gt;&lt;strong&gt;Objective:&lt;/strong&gt; I am trying to solve for $C$ in 2D space (x,y) and time from following PDE. &#xA;$$&#xA;\text{PDE: }\frac{\partial C}{\partial t} + \nabla\left(v.C - D\nabla{C} \right)=  \alpha.C&#xA;$$&#xA;&lt;strong&gt;Method:&lt;/strong&gt; I discretized the above equation in 2D using finite-difference implicit scheme and as a result I get the following discretized equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p_1C^{n+1}_{i,j-1}+p_2C^{n+1}_{i-1,j}+p_3C^{n+1}_{i,j}+p_4C^{n+1}_{i+1,j}+p_5C^{n+1}_{i,j+1} = C^{n}_{i,j}$$&#xA;where, $p_1, p_2, p_3, p_4, p_5$ are constant matrices in time. They are matrices instead of scalars because I am assuming $v$ and $D$ vary in space $(x,y)$ but constant in time i.e. $v(x,y)$ and $D(x,y)$. The variables  $p_1, p_2, p_3, p_4, p_5$ are shown in the snapshot below:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/xgp23.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can solve for $C$ at each time by solving the system of equations as $A^{n+1}.C^{n+1}=C^{n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Issue:&lt;/strong&gt; Since $p_1, p_2, p_3, p_4, p_5$ are matrices, I am not sure how to develop the matrix $A$. As an example for a system of $3\times3$ grid size, matrix $A$ will have the form as shown below. In that case how can I form matrix $A$ from $p_1, p_2, p_3, p_4, p_5$, each of which is a matrix of size $3\times3$? I would really appreciate if someone can guide how to solve this. Thanks.&#xA;&lt;img src=&quot;https://i.stack.imgur.com/4FgNP.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user5510" LastEditorUserId="1804" LastEditDate="2016-05-31T21:30:51.787" LastActivityDate="2016-05-31T21:30:51.787" Title="$AX=B$: How to solve for $X$ if elements of matrix A are matrices" Tags="|pde|finite-difference|matrix|advection-diffusion|" AnswerCount="1" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="20126" PostTypeId="1" AcceptedAnswerId="20155" CreationDate="2015-07-10T11:29:58.747" Score="3" ViewCount="223" Body="&lt;p&gt;How can I use Fourier Transform to solve Fisher-Kolmogorov Equation in 1D?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_t(x,t) = u_{xx}(t) + u(1-u)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u(0,x) = \phi(x)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with Dirichlet &#xA;\begin{equation}&#xA;u(0,t)=0 \\&#xA;u(1,t)=0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and Neumann boundary conditions&#xA;\begin{equation}&#xA;u_x(0,t)=0\\&#xA;u_x(1,t)=0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I just do the following?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;F\{u(x,t)\}=\hat{u}(k,t) = \int_{-\infty}^{+\infty}u(x,t)e^{-ikx}dx&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\hat{u}_t(k,t) = (ik)^2\hat{u}(k) + F\{u(x,y)(1-u(x,t))\} \\&#xA;u_t(x,t)=F^{-1} \{ (ik)^2\hat{u}(k) \} + u(x,y)(1-u(x,t))&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/ilciavo/ffea558fd290a2a99b0b&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is a simple Matlab implementation in 1D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After 250 iterations using forward Euler with $\Delta t = 0.01$ , the solution looks something like &lt;img src=&quot;https://i.stack.imgur.com/8zjM1.png&quot; alt=&quot;Fisher-Kolmogorov&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="11656" LastEditorUserId="11656" LastEditDate="2015-07-11T19:25:37.063" LastActivityDate="2015-07-13T23:12:26.427" Title="How to get Fourier transform of Fisher-Kolmogorov?" Tags="|pde|fourier-transform|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20127" PostTypeId="2" ParentId="20125" CreationDate="2015-07-10T13:35:34.993" Score="4" Body="&lt;p&gt;What is usually done when you have a matrix-like variable due to the dimensions of your problem and the indexing is to linearize the index. I.e., if you have $C_{i,j}$ you would replace that with $\hat{C}_k=C_{i,j}$ where $k=i\times N+j$ and $N$ is the number of points on one side of your $N\times N$ grid of points that you are doing finite differences on. This makes it easy to convert your kind of strange linear system into the standard form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that you might end up representing $C$ as a single-indexed array in your program instead of treating it with double-indexing. There are other ways to accomplish this once you've written down the right linear system, but this method is compatible with most linear solver libraries which expect vectors of unknowns and right-hand sides.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-07-10T13:35:34.993" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="20128" PostTypeId="1" AcceptedAnswerId="20129" CreationDate="2015-07-10T18:07:47.903" Score="3" ViewCount="1832" Body="&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/jxwTB.png&quot; alt=&quot;enter image description here&quot;&gt;I am trying to do SVD of a large block-hankel matrix for model order reduction (Low rank approximation). However, I quickly run into memory issues in forming the large Block-Hankel matrix and CPU issues due to running SVD itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, it seems like we don't have to form the Hankel matrix itself. As long as we have the elements of either the first row/column of the matrix, we have complete information of all the entries of the Hankel matrix, and we should be able to perform the SVD without even forming this large matrix in memory ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any algorithm (in netlib or any of the other numerical libraries) or any other user-submitted MATLAB package that takes advantage of the special structure of the block-Hankel matrix ?&lt;/p&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2015-07-25T01:02:52.330" Title="SVD of large block-hankel matrix" Tags="|linear-algebra|matlab|matrix|svd|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20129" PostTypeId="2" ParentId="20128" CreationDate="2015-07-10T18:36:41.753" Score="4" Body="&lt;p&gt;There are matrix free algorithms (algorithms that use only matrix-vector multiplications rather than working directly with the entries of the matrix) that can compute approximate values of a few singular values/vectors of a large matrix.  Since you want a low rank approximation to this matrix, you could use such an algorithm to find the $k$ largest singular values and associated singular vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make this work you will need to develop routines that can compute matrix-vector productions $Ax$ and $A^{T}y$ for your structured matrix $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow&quot;&gt;ARPACK&lt;/a&gt; is perhaps the most widely used package in this area.  In MATLAB if you have $A$ as a sparse matrix you can use the svds() function which calls ARPACK routines.  Unfortunately, MATLAB doesn't seem to provide an interface to these functions that allows you to just provide matrix-vector multiplication routines instead of a sparse matrix.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a &lt;a href=&quot;http://slepc.upv.es/documentation/reports/str6.pdf&quot; rel=&quot;nofollow&quot;&gt;survey&lt;/a&gt; of available software package for sparse eigenvalue and singular value problems that might be helpful to you.  I assume that someone has put together MATLAB interfaces to some of these libraries that do have the ability to work with user supplied matrix-vector multiplication routines.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I recall, &lt;a href=&quot;http://sun.stanford.edu/~rmunk/PROPACK/&quot; rel=&quot;nofollow&quot;&gt;PROPACK&lt;/a&gt; includes a MATLAB interface that allows for user supplied matrix-vector multiplication routines.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-07-10T18:36:41.753" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20131" PostTypeId="2" ParentId="20107" CreationDate="2015-07-10T20:53:04.560" Score="1" Body="&lt;p&gt;In this line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EuclideanDist = 1 / sqrt(sum(EuclideanDist^2 ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the &lt;code&gt;sqrt&lt;/code&gt; should not be present: the integrand is one over the squared norm, not one over norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arcLength = integrate(FUN, x,y)$value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this will give you a negative (incorrect) value for arc length whenever $x&amp;gt;y$, which confuses the rest of the code. The correct evaluation would be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arcLength = abs(integrate(FUN, x,y)$value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, the reason for &lt;code&gt;NA&lt;/code&gt;s is that the integration method will sometimes try to evaluate your functions at diagonal points with $x=y$, for which the integrand is undefined. One simple way to fix this is to replace the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt; y = z[2]&#xA;&amp;gt; y = z[2] + 1e-12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to break exact equality, or simply return zero when $x$ and $y$ are exactly equal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integrand &lt;code&gt;FUN&lt;/code&gt; is $ \sqrt{\cos^2x+\sin^2x} = 1 $, so I'm not sure why you are integrating it numerically.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-07-10T21:57:46.967" LastActivityDate="2015-07-10T21:57:46.967" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20134" PostTypeId="1" CreationDate="2015-07-11T07:22:25.663" Score="5" ViewCount="117" Body="&lt;p&gt;I am trying to solve $Ax=b$ a system of linear equation. The matrix $A=(a_{ij})$ is constructed as $$a_{ij}=\int_0^1\int_0^1 K(x,y)h_i(x)h_j(y) dxdy.$$ The evaluation of integral takes a lot of time, and overall computation is very slow. Can I evaluate these integrals in parallel using GPU cores? Please suggest how can I compute these integrals simultaneously??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks&lt;/p&gt;&#xA;" OwnerUserId="16905" LastActivityDate="2015-07-11T07:22:25.663" Title="Computing entries of a matrix in parallel using GPU" Tags="|parallel-computing|gpu|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20135" PostTypeId="1" CreationDate="2015-07-11T10:44:22.603" Score="1" ViewCount="32" Body="&lt;p&gt;How many CPU cycle is required to simulate a complete human body from it's very initial stem cell using classical algorithms and also is it possible to use similar algorithms for simulating stem cells themselves from their very initial atoms? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much computing these models will cost based on available hardwares today?&lt;/p&gt;&#xA;" OwnerUserId="16908" LastActivityDate="2015-07-11T12:13:45.527" Title="Computational complexity of chemical dynamics for biological studies" Tags="|complexity|computational-chemistry|" AnswerCount="1" CommentCount="0" ClosedDate="2015-07-11T18:48:01.137" ContentLicense="CC BY-SA 3.0" />
  <row Id="20136" PostTypeId="2" ParentId="20135" CreationDate="2015-07-11T12:13:45.527" Score="3" Body="&lt;p&gt;The estimated number of cells in the human body is $3.72 \times 10^{13}$;&lt;sup&gt;1&lt;/sup&gt; the estimated number of molecules in a single cell is about $10^{14}$.&lt;sup&gt;2&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that the current peta-scale supercomputers need a full day to simulate the motion of $1.2\times 10^{7}$ particles over $49$ nanoseconds&lt;sup&gt;3&lt;/sup&gt; -- and a simulation of the complexity you propose needs to keep track of much more than just the position! -- what you propose is completely infeasible with the technology that can realistically be expected to be available in the near and mid future.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&#xA;  1. &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/23829164&quot; rel=&quot;nofollow&quot;&gt;Bianconi E1, Piovesan A, Facchin F, Beraudi A, Casadei R, Frabetti F, Vitale L, Pelleri MC, Tassani S, Piva F, Perez-Amodio S, Strippoli P, Canaider S: An estimation of the number of cells in the human body. Ann Hum Biol. 2013 Nov-Dec;40(6):463-71&lt;/a&gt;&#xA;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&#xA;  2. This number appears as a factoid on a number of websites and is often attributed to &quot;a group of engineers from Washington University&quot;. No reference is given, and my brief search didn't turn up anything, either. As a ballpark estimate, it seems about right. &#xA;&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&#xA;  3. &lt;a href=&quot;https://www.mpibpc.mpg.de/14613164/Kutzner_2014_ParCo-conf2013.pdf&quot; rel=&quot;nofollow&quot;&gt;Kutzner, C., Apostolov, R., Hess, B., &amp;amp; Grubmüller, H. (2014). Scaling of the GROMACS 4.6 molecular dynamics code on SuperMUC. In M. Bader, A. Bode, &amp;amp; H. J. Bungartz (Eds.), Parallel Computing: Accelerating Computational Science and Engineering (CSE) (pp. 722-730). Amsterdam: IOS Press&lt;/a&gt;&#xA;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-07-11T12:13:45.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20137" PostTypeId="2" ParentId="2026" CreationDate="2015-07-11T12:22:06.400" Score="1" Body="&lt;p&gt;You can try the geogram software that I'm developping:&#xA;&lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has a parallel algorithm that computes the DT of&#xA;14 million vertices in less than 19 seconds on an Intel Core I7 &#xA;(for 1 million vertices it takes around 0.8 s)&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2015-07-11T12:27:02.657" LastActivityDate="2015-07-11T12:27:02.657" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20138" PostTypeId="1" CreationDate="2015-07-11T12:24:10.937" Score="1" ViewCount="160" Body="&lt;p&gt;I have the following quadratic form I need to evaluate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x^T A^{-1} y$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a sparse positive definite matrix, &lt;span class=&quot;math-container&quot;&gt;$x, y$&lt;/span&gt; are sparse vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now assume that I am given &lt;strong&gt;for free&lt;/strong&gt; both &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt; and the Cholesky decomposition &lt;span class=&quot;math-container&quot;&gt;$A = L D L^T$&lt;/span&gt;. I understand that it still will be faster to evaluate &lt;span class=&quot;math-container&quot;&gt;$x^T A^{-1} y$&lt;/span&gt; using a Cholesky decomposition, rather than directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you relate to the computational complexity (+ references) of each alternative?&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="20688" LastEditDate="2020-05-18T07:59:50.760" LastActivityDate="2020-05-18T07:59:50.760" Title="Evaluating a quadratic form with an inverse of a sparse PD matrix, comparison between using the inverse vs using a Cholseky decomposition" Tags="|linear-solver|sparse-matrix|matrix-factorization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20140" PostTypeId="2" ParentId="20138" CreationDate="2015-07-11T13:44:10.187" Score="4" Body="&lt;p&gt;It depends on the matrix. If $A=\alpha I$, you need one less operation with $A^{-1}$ than with the Cholesky factor. Usually, the reason why the Cholesky factor version is faster is that it is more sparse, but this is highly dependent on the matrix. It will be difficult to get a formal result, I imagine.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2015-07-11T13:44:10.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20141" PostTypeId="1" AcceptedAnswerId="20142" CreationDate="2015-07-11T15:11:21.860" Score="3" ViewCount="239" Body="&lt;p&gt;I am wondering how can I solve following problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Arrange randomly $n$ points inside a square of side $a$ under the condition that the distance between any two points may not be smaller than 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to see how can it be solved.&lt;/p&gt;&#xA;" OwnerUserId="16846" LastEditorUserId="1744" LastEditDate="2015-07-13T19:42:34.217" LastActivityDate="2015-09-10T21:35:14.430" Title="Distance between points" Tags="|numerical-analysis|computational-geometry|geometry|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20142" PostTypeId="2" ParentId="20141" CreationDate="2015-07-11T15:26:21.110" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;Choose n random points in the unit square.  &lt;/li&gt;&#xA;&lt;li&gt;Compute the minimum point-to-point distance, $D$.  &lt;/li&gt;&#xA;&lt;li&gt;Re-scale the space by $1/D$. The minimum point-to-point distance is now equal to 1.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;br&gt;&#xA;The best method to do step 2. depends on many things.&#xA;If don't have too many points you can just compute (using matlab syntax):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dmin=1;&#xA;for i=1:n&#xA;  for j=i+1:n&#xA;    dmin = min( dmin,norm( p(i,:)-p(j,:) ) );&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This brute force method will take $\mathcal{O}(n^2)$ time. If you have many, many points it may be worth using an $\mathcal{O}(n\log(n))$ algorithm using a divide and conquer approach. See &lt;a href=&quot;https://en.wikipedia.org/wiki/Closest_pair_of_points_problem&quot; rel=&quot;nofollow&quot;&gt;the wikipedia page&lt;/a&gt; for more information.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-07-11T15:51:22.413" LastActivityDate="2015-07-11T15:51:22.413" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20143" PostTypeId="1" CreationDate="2015-07-11T17:26:01.543" Score="2" ViewCount="102" Body="&lt;p&gt;I need to know what library, given a vector of coefficients and modulus N return the roots of polynomial modulus N. The library should support big integer. I'm coding in C++. &lt;/p&gt;&#xA;" OwnerUserId="14612" LastActivityDate="2015-07-11T19:03:55.350" Title="C++: How to find the roots of polynomial modulus N" Tags="|linear-algebra|c++|polynomials|" AnswerCount="0" CommentCount="12" ClosedDate="2015-07-13T20:40:50.410" ContentLicense="CC BY-SA 3.0" />
  <row Id="20144" PostTypeId="2" ParentId="20141" CreationDate="2015-07-11T18:16:14.893" Score="3" Body="&lt;p&gt;If $n$ is too large and the side length of your squarer $a$ is fixed then there may not be a solution to your problem. If you are willing to increase the size of the square to fit the points then the solution by @DougLipinski will work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the square size if fixed (which is why I assume you asked the question in the first place), then the re-scale step in his solution may push points outside the square. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can actually estimate an upper bound on the number of points that will fit inside your square by computing the number of points that make up a &lt;a href=&quot;https://en.wikipedia.org/wiki/Tessellation&quot; rel=&quot;nofollow&quot;&gt;tessellation&lt;/a&gt; of the square.&lt;/p&gt;&#xA;" OwnerUserId="9813" LastEditorUserId="9813" LastEditDate="2015-07-12T00:05:57.360" LastActivityDate="2015-07-12T00:05:57.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20146" PostTypeId="2" ParentId="1426" CreationDate="2015-07-12T13:16:16.210" Score="-4" Body="&lt;p&gt;please see my fdm paper which you can locate in researchgate under my name david Edwards jr.  if you have questions I would be glad to help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;david&lt;/p&gt;&#xA;" OwnerUserId="6880" LastActivityDate="2015-07-12T13:16:16.210" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20147" PostTypeId="1" CreationDate="2015-07-13T05:53:19.987" Score="1" ViewCount="104" Body="&lt;p&gt;I’m interested in solving nonlinear elliptic boundary value problems of the type&#xA;$$&#xA;-a\Delta u + f(u) = 0,&#xA;$$&#xA;$$&#xA;u|_\Gamma = u_0&#xA;$$&#xA;by Newton’s method when its convergence is global and monotonic.&#xA;Could you advice some references concerning this problem, containing proofs of global convergence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Newton's method takes the form&#xA;$$&#xA;-a\Delta u + f(\widetilde u) + f'(\widetilde u)(u - \widetilde u) = 0&#xA;$$&#xA;where $\widetilde u$ is the previous approximation for the solution.&lt;/p&gt;&#xA;" OwnerUserId="11531" LastActivityDate="2015-07-13T05:53:19.987" Title="Monotonic convergence of Newton's method for boundary value problems" Tags="|pde|reference-request|nonlinear-equations|newton-method|elliptic-pde|" AnswerCount="0" CommentCount="2" FavoriteCount="0" ClosedDate="2018-03-01T22:34:02.367" ContentLicense="CC BY-SA 3.0" />
  <row Id="20148" PostTypeId="2" ParentId="7714" CreationDate="2015-07-13T06:11:15.820" Score="16" Body="&lt;p&gt;Although this post is now two years old, in case someone stumbles across it, let me give a brief update:&lt;/p&gt;&#xA;&lt;p&gt;Martin Gander recently wrote a nice review article, that gives a historical perspective on the field and discusses many different PINT methods: &lt;a href=&quot;http://www.unige.ch/%7Egander/Preprints/50YearsTimeParallel.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.unige.ch/~gander/Preprints/50YearsTimeParallel.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is now also a community website which lists very many references and gives descriptions of different methods: &lt;a href=&quot;http://www.parallel-in-time.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.parallel-in-time.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;A discussion of the Parareal parallel-in-time algorithm in particular can be found here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Parareal&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Parareal&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; There is now a newer 2020 review paper by &lt;a href=&quot;https://doi.org/10.1007/s00791-020-00331-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ong and Schroder&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="16921" LastEditDate="2020-11-30T07:40:58.633" LastActivityDate="2020-11-30T07:40:58.633" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20149" PostTypeId="1" CreationDate="2015-07-13T07:00:01.940" Score="1" ViewCount="276" Body="&lt;p&gt;I am working on a transient heat transfer problem. The problem that I have is as the time goes the range of temperature is changing but Tecplot only use one sort of levels for contour for all of time steps. Therefore the change in temperature is not very sensible at the end of the time which transient diminishes. I am wondering if there is anyway to change the levels of contours as time goes forward? (Could I specify a contour for any specific time step?)&lt;/p&gt;&#xA;" OwnerUserId="16922" LastActivityDate="2015-07-14T11:29:36.887" Title="How to change level of contours in animation for Tecplot360" Tags="|plotting|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20150" PostTypeId="1" CreationDate="2015-07-13T07:31:37.750" Score="1" ViewCount="505" Body="&lt;p&gt;I am using ANSYS Fluent 15 to simulate a case of fluid flow. In my case, I have three cell zones in my simulation, let's call them top, middle and bottom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My requirement is to put a vertical constant force in the &quot;middle&quot; cell zone. I can not use gravity because it is then acted in all other cell zones as well. &lt;strong&gt;My purpose is to define this force exclusively in the middle cell zone only.&lt;/strong&gt;&#xA;Apart from it, I tried to use the &quot;Y-momentum&quot; source term in cell zone conditions panel but it a did not seem to work either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be great. Thanks&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2015-07-13T07:31:37.750" Title="ANSYS Fluent: Defining External Force on a Cell Zone" Tags="|simulation|fluid-dynamics|ansys-fluent|" AnswerCount="0" CommentCount="1" ClosedDate="2015-07-14T07:17:40.370" ContentLicense="CC BY-SA 3.0" />
  <row Id="20151" PostTypeId="2" ParentId="20117" CreationDate="2015-07-13T08:13:10.277" Score="0" Body="&lt;p&gt;Take a look at this &lt;a href=&quot;http://www.osti.gov/scitech/servlets/purl/632793/&quot; rel=&quot;nofollow&quot;&gt;Efficient Computation of Volume of&#xA;Hexahedral Cells&lt;/a&gt;. It explains several methods to calculating the volume using algebraic equation (instead of integration), and the first method is very straightforward and does not need a complicated algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if you have problem for finding the outward normal based on order of points, choose an arbitrary point inside of the hexahedra and use dot product to find the direction of normal vector.  &lt;/p&gt;&#xA;" OwnerUserId="16922" LastActivityDate="2015-07-13T08:13:10.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20152" PostTypeId="1" AcceptedAnswerId="20164" CreationDate="2015-07-13T09:27:19.943" Score="10" ViewCount="1422" Body="&lt;p&gt;I had written a code a while ago which attempted to calculate $log(x)$ without using library functions. Yesterday, I was reviewing the old code, and I tried to make it as fast as possible, (and correct). Here's my attempt so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const double ee = exp(1);&#xA;&#xA;double series_ln_taylor(double n){ /* n = e^a * b, where a is an non-negative integer */&#xA;    double lgVal = 0, term, now;&#xA;    int i, flag = 1;&#xA;&#xA;    if ( n &amp;lt;= 0 ) return 1e-300;&#xA;    if ( n * ee &amp;lt; 1 )&#xA;        n = 1.0 / n, flag = -1; /* for extremely small n, use e^-x = 1/n */&#xA;&#xA;    for ( term = 1; term &amp;lt; n ; term *= ee, lgVal++ );&#xA;    n /= term;&#xA;&#xA;    /* log(1 - x) = -x - x**2/2 - x**3/3... */&#xA;    n = 1 - n;&#xA;    now = term = n;&#xA;    for ( i = 1 ; ; ){&#xA;        lgVal -= now;&#xA;        term *= n;&#xA;        now = term / ++i;&#xA;        if ( now &amp;lt; 1e-17 ) break;&#xA;    }&#xA;&#xA;    if ( flag == -1 ) lgVal = -lgVal;&#xA;&#xA;    return lgVal;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here I am trying to find $a$ so that $e^{a}$ is just over n, and then I add the logarithm value of $\frac{n}{e^{a}}$, which is less than 1. At this point, the Taylor expansion of $log(1\ - \ x)$ can be used without worrying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have recently grown an interest in numerical analysis, and that's why I can't help asking the question, how much faster this code segment can be run in practice, while being correct enough? Do I need to switch to some other methods , for example, using continued fraction, like &lt;a href=&quot;https://math.stackexchange.com/questions/75074/an-alternative-way-to-calculate-logx&quot; title=&quot;this&quot;&gt;this&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $log(x)$ function provided with C standard library is almost 5.1 times faster than this implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 1&lt;/strong&gt;: Using the hyperbolic arctan series mentioned &lt;a href=&quot;https://en.wikipedia.org/wiki/Logarithm#Calculation&quot; rel=&quot;nofollow noreferrer&quot;&gt;in Wikipedia&lt;/a&gt;, the computation seems to be almost 2.2 times slower than C standard library log function. Though, I have not extensively checked the performance, and for larger numbers, my current implementation seems to be REALLY slow.&#xA;I want to check both of my implementation for error bound and average time for a wide range of numbers if I can manage.&#xA;Here is my second effort.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double series_ln_arctanh(double n){ /* n = e^a * b, where a is an non-negative integer */&#xA;    double lgVal = 0, term, now, sm;&#xA;    int i, flag = 1;&#xA;&#xA;    if ( n &amp;lt;= 0 ) return 1e-300;&#xA;    if ( n * ee &amp;lt; 1 ) n = 1.0 / n, flag = -1; /* for extremely small n, use e^-x = 1/n */&#xA;&#xA;    for ( term = 1; term &amp;lt; n ; term *= ee, lgVal++ );&#xA;    n /= term;&#xA;&#xA;    /* log(x) = 2 arctanh((x-1)/(x+1)) */&#xA;    n = (1 - n)/(n + 1);&#xA;&#xA;    now = term = n;&#xA;    n *= n;&#xA;    sm = 0;&#xA;    for ( i = 3 ; ; i += 2 ){&#xA;        sm += now;&#xA;        term *= n;&#xA;        now = term / i;&#xA;       if ( now &amp;lt; 1e-17 ) break;&#xA;    }&#xA;&#xA;    lgVal -= 2*sm;&#xA;&#xA;    if ( flag == -1 ) lgVal = -lgVal;&#xA;    return lgVal;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or criticism is appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 2:&lt;/strong&gt; Based on suggestions made below, I have added some incremental changes here, which is around 2.5 times slower than standard library implementation. However, I have tested it only for integers $\leq1e^{8}$ this time, for larger numbers the runtime would increase. For now. I do not yet know of techniques to generate random double numbers $\leq1e^{308}$, so it's not yet fully benchmarked. To make the code more robust, I have added corrections for corner cases. The average error for the tests I made is around $4e^{-15}$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double series_ln_better(double n){ /* n = e^a * b, where a is an non-negative integer */&#xA;    double lgVal = 0, term, now, sm;&#xA;    int i, flag = 1;&#xA;&#xA;    if ( n == 0 ) return -1./0.; /* -inf */&#xA;    if ( n &amp;lt; 0 ) return 0./0.;   /* NaN*/&#xA;    if ( n &amp;lt; 1 ) n = 1.0 / n, flag = -1; /* for extremely small n, use e^-x = 1/n */&#xA;&#xA;    /* the cutoff iteration is 650, as over e**650, term multiplication would&#xA;       overflow. For larger numbers, the loop dominates the arctanh approximation&#xA;       loop (with having 13-15 iterations on average for tested numbers so far */&#xA;&#xA;    for ( term = 1; term &amp;lt; n &amp;amp;&amp;amp; lgVal &amp;lt; 650 ; term *= ee, lgVal++ );&#xA;    if ( lgVal == 650 ){&#xA;        n /= term;&#xA;        for ( term = 1 ; term &amp;lt; n ; term *= ee, lgVal++ );&#xA;    }&#xA;    n /= term;&#xA;&#xA;    /* log(x) = 2 arctanh((x-1)/(x+1)) */&#xA;    n = (1 - n)/(n + 1);&#xA;&#xA;    now = term = n;&#xA;    n *= n;&#xA;    sm = 0;&#xA;&#xA;    /* limiting the iteration for worst case scenario, maximum 24 iteration */&#xA;    for ( i = 3 ; i &amp;lt; 50 ; i += 2 ){&#xA;        sm += now;&#xA;        term *= n;&#xA;        now = term / i;&#xA;        if ( now &amp;lt; 1e-17 ) break;&#xA;    }&#xA;&#xA;    lgVal -= 2*sm;&#xA;&#xA;    if ( flag == -1 ) lgVal = -lgVal;&#xA;&#xA;    return lgVal;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16638" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2020-05-24T01:28:37.600" Title="About faster approximation of log(x)" Tags="|numerical-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20153" PostTypeId="2" ParentId="20070" CreationDate="2015-07-13T10:21:29.220" Score="9" Body="&lt;p&gt;In the context of finite element methods (and, especially, symmetric problems) the most common direct solution method is Cholesky factorization (plus following substitutions). MATLAB uses &lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot;&gt;Tim Davis' CHOLMOD package&lt;/a&gt; to compute Cholesky factorization whenever the heuristics of backslash operator encounter a symmetric positive definite matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, Julia also interfaces Davis' CHOLMOD through its &lt;a href=&quot;http://julia.readthedocs.org/en/latest/stdlib/linalg/&quot;&gt;&lt;em&gt;cholfact&lt;/em&gt; command&lt;/a&gt;. I have found that it is sufficient to call&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u=cholfact(K)\F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;em&gt;K&lt;/em&gt; is a sparse matrix.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastEditorUserId="7733" LastEditDate="2015-07-13T11:22:54.103" LastActivityDate="2015-07-13T11:22:54.103" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20154" PostTypeId="1" CreationDate="2015-07-13T12:18:14.133" Score="3" ViewCount="4347" Body="&lt;p&gt;I need to know what libraries (in C++) support polynomial arithmetic specially over a field. So I can give to it an array of coefficients of polynomial over a field and it returns the roots of polynomial over the field. It also should support big integers, as the coefficients and the roots are big integers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For security reason my polynomial is defined over a finite field of prime order. So it's always possible to convert it to a polynomial whose leading coefficient is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried NTL, but it apparently does not support coefficients of type big integers!&lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="14612" LastEditDate="2015-07-13T19:55:41.977" LastActivityDate="2015-07-13T21:04:52.623" Title="C++ Library: What is the common libraries that do polynomial arithmetic?" Tags="|linear-algebra|algorithms|c++|libraries|polynomials|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20155" PostTypeId="2" ParentId="20126" CreationDate="2015-07-13T17:58:01.570" Score="2" Body="&lt;p&gt;The problem can be partially solved following &lt;a href=&quot;http://www.wikiwaves.org/Reaction-Diffusion_Systems&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; tutorial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given Fisher-Kolmogorov equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_t=u_{xx}+u(1-u)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can also be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_t = u_{xx} + u\,v\\&#xA;v_t = v_{xx} - u\,v  &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $v = (1-u)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the equation &#xA;\begin{equation}&#xA;u_t = u_{xx} \\&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from $t$ to $\Delta t$ gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\hat{u}_t = (ik)^2 \hat{u} \\&#xA;\tilde{u}(x,t+\Delta t) = F^{-1} \left\{e^{-k^2 \Delta t} \hat{u} \right\} &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now solving the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_t = u \, v \\&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the splitting operator method we can write &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u(x,t+\Delta t) = e^{-v\Delta t} \tilde{u}(x,t+\Delta t)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing that likewise for $v$ gives the solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4yVQj.gif&quot; alt=&quot;Solution&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/ilciavo/8ea235906a5e48ed11f3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Diffusion_Reaction.m&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Boundary conditions still must be imposed.&lt;/p&gt;&#xA;" OwnerUserId="11656" LastEditorUserId="11656" LastEditDate="2015-07-13T23:12:26.427" LastActivityDate="2015-07-13T23:12:26.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20156" PostTypeId="1" CreationDate="2015-07-13T18:21:09.060" Score="1" ViewCount="38" Body="&lt;p&gt;W==a(x)=2*(W* x)/41;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm attempting to make a new type of compression algorithm. A rubber band ball type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to make a software that extracts, from that formula, the secret volumes of number sets within. By taking the the opposite formula, b(x) i should get the answer. But unfortunately, i dont. I have to search 0-255(+1). Through the first generation the numbers would be like as follows&#xA;(W starts at 1)&#xA;    a(82*1)==2&#xA;    b(2/1)==82&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need the numbers reappear for me. How can I do this without cheating?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q: What is the best &lt;code&gt;if&lt;/code&gt; statement to get my numbers to reappear?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ideone.com/mZKttO&quot; rel=&quot;nofollow&quot;&gt;My code as is right now.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16929" LastEditorUserId="16929" LastEditDate="2015-07-30T15:32:35.400" LastActivityDate="2015-07-30T15:32:35.400" Title="How can i get this number set to be recooperated from a product sum?" Tags="|mathematica|" AnswerCount="0" CommentCount="6" ClosedDate="2015-07-13T20:33:11.773" ContentLicense="CC BY-SA 3.0" />
  <row Id="20157" PostTypeId="1" CreationDate="2015-07-13T19:25:17.990" Score="5" ViewCount="115" Body="&lt;p&gt;Consider the following example structure of overlapping images marked A,B,C,D. The possible overlaps are marked by gray color:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/BrqDV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The structure can be represented by a weighted undirected graph (images being the nodes, edges representing overlaps and the weights representing overlap areas or time needed to process them):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/C02gK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The images are to be rendered on surface while overlaps require extra processing (e.g. avergaing the pixels for transparency).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we can decide to merge A,B first. The situation becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Prd70.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After each merge, the graph has to be updated.&#xA;In this case, only images C,D are to be merged and the algorithm finishes with single composite image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can observe that if the merging algorithm runs in parallel, it can now process C and D simultaneously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if we start merging A,C first, then B and D are blocked and the algorithm can run only sequentially (considering that individual image can be used by single thread / cannot be shared).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the graph, I would like to schedule parallel computation for $N$ computing units (e.g. CPU cores).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The schedule for $N=2$ can look like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;iteration #1&#xA;    core 1: A-B&#xA;    core 2: waiting&#xA;iteration #2&#xA;    core 1: (A-B)C&#xA;    core 2: (A-B)D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think the scheduler should aim for two objectives:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Minimize the number of iterations (i.e. utilize maximum number of cores at any given time)&lt;/li&gt;&#xA;&lt;li&gt;Balance load to all cores at any given iteration (i.e. assign them approximately same areas of overlap so that they will finish approximately at the same time; unnecessary waiting would spoil the benefits of parallel computation).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The first objective is important while the second is &quot;nice to have&quot; (the time needed to process an overlap is proportional to its area).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am stuck with two major problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What algorithm to use? Backtracking with storing current best solution?&lt;/li&gt;&#xA;&lt;li&gt;Would it be easier to schedule tasks for $N=\infty$ (maximize parallelism) rather that $N$ being corresponding to actual number of processing units available?&lt;/li&gt;&#xA;&lt;li&gt;Only the overlap areas between pairs of images are known, but since the image are merged in the process, how to compute new overlaps (e.g. C(A-B))?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The greedy version of the algorithm would look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Grab as much overlapping image pairs as possible such that no two pairs themselves are overlapping.&#xA;Sort the pairs in N bins such that each bin contains roughly same total area.&#xA;Merge images in parallel.&#xA;Update graph and repeat until single node remains.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know this question is maybe too broad. I believe this problem has been certainly solved many times in parallel computation projects so many people from parallel computing areas should have some immediate ideas and hints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once solved, I plan to write an elaborate answer to benefit you as well. But first I need something to start with. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4591" LastActivityDate="2015-07-13T19:25:17.990" Title="Graph optimization for parallel processing" Tags="|optimization|graph-theory|combinatorics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20158" PostTypeId="2" ParentId="20154" CreationDate="2015-07-13T19:32:25.313" Score="1" Body="&lt;p&gt;Root finding of an arbitrary polynomial with degree $\gt 4$ over any field is supposed to be a trial and error process, as far as I know. &lt;strike&gt;If you are not dealing with any special type of polynomial, I can't see how any library is going to help you.&lt;/strike&gt; Fast algorithms of finding roots for polynomial modulo p are there, for example &lt;a href=&quot;https://en.wikipedia.org/wiki/Cantor%E2%80%93Zassenhaus_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. You can also try Pari library written in C, which has implementations of this algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can factorize the polynomial into smaller polynomials of lower degrees so that you can search for roots trivially, you can use Chinese Remainder Theorem to combine the results and get roots for original polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a special property of the polynomials you are using, you can exploit it. I will advise you to be a bit more specific about the problem you are dealing with, rather than posting same problem in different sites again and again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should have commented, but I don't have the necessary reputation yet. I'll try to update the answer as soon as I have a clear understanding of what you are dealing with. And please feel free to share your efforts so far in your post and comments. Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; You can also check out &lt;a href=&quot;https://stackoverflow.com/questions/29001275/roots-of-a-polynomial-mod-a-prime&quot;&gt;here&lt;/a&gt; for algorithms for finding roots of polynomials. There is always an option of using data structures provided by existing packages and mixing them as you see fit.&lt;/p&gt;&#xA;" OwnerUserId="16638" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-07-13T21:04:52.623" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20159" PostTypeId="2" ParentId="20141" CreationDate="2015-07-13T19:39:05.763" Score="2" Body="&lt;p&gt;The problem of finding the smallest $a$-square that can contain $n$ points whose distance is not less than $1$ is equivalent to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Circle_packing_in_a_square&quot; rel=&quot;nofollow noreferrer&quot;&gt;circle packing in a square problem&lt;/a&gt;, see e.g &lt;a href=&quot;http://hydra.nat.uni-magdeburg.de/packing/equiv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Equivalence between circle packing and point packing&lt;/a&gt;. Info about this problem can be found also in &lt;a href=&quot;http://mathworld.wolfram.com/CirclePacking.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Circle Packing&lt;/a&gt; and a list of optimal or best known solution in &lt;a href=&quot;http://hydra.nat.uni-magdeburg.de/packing/csq/csq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The best known packings of equal circles in a square&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is, as far as I know, a still unsolved problem: the answer to the OP question depends on how close is the desired solution to the optimal one, or on how big is $n$ for fixed $a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we look for a far from optimal solution, the &lt;a href=&quot;https://scicomp.stackexchange.com/a/20142/1744&quot;&gt;answer&lt;/a&gt; by &lt;a href=&quot;https://scicomp.stackexchange.com/users/7733/doug-lipinski&quot;&gt;Doug&lt;/a&gt; could be viable. On the contrary if we approach the optimal limit, the random scatter and rescale approach will give unfeasible solutions, as pointed out by &lt;a href=&quot;https://scicomp.stackexchange.com/users/9813/dpmcmlxxvi&quot;&gt;dpmcmlxxvi&lt;/a&gt; in his &lt;a href=&quot;https://scicomp.stackexchange.com/a/20144/1744&quot;&gt;answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is not defined in such a way that a unique or best answer exists, but I would proceed in the following way in $n$ is large enough, but still not too close to the optimal/best known solution.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;arrange the $n$ points on an hexagonal lattice whose envelope is approximately square. (See &lt;a href=&quot;http://oeis.org/A093766&quot; rel=&quot;nofollow noreferrer&quot;&gt;OEIS A093766&lt;/a&gt; for the properties of this lattice.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if the side of this square is less then $a$, rescale it to tightly fit inside the $a$ square.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;after rescaling apply a small perturbation, in order to give a &quot;random&quot; appearance to the solution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If $n$ is close to the optimal/best know solution, than simply do a table lookup from a list of the known solutions.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-07-13T19:39:05.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20160" PostTypeId="2" ParentId="20154" CreationDate="2015-07-13T19:53:09.087" Score="1" Body="&lt;p&gt;A package that can do what you want is &lt;a href=&quot;http://www.singular.uni-kl.de/&quot; rel=&quot;nofollow&quot;&gt;Singular&lt;/a&gt;.  It is written in C++ and it is my understanding that there is a library called libSingular that allows other programs to use its features (among them solving polynomial equations over finite fields and big integer arithmetic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option that may fulfill your requirements is &lt;a href=&quot;http://flintlib.org/&quot; rel=&quot;nofollow&quot;&gt;FLINT&lt;/a&gt;.  It is written in C but I believe they have C++ wrappers for their functionality.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2015-07-13T20:42:02.890" LastActivityDate="2015-07-13T20:42:02.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20162" PostTypeId="1" AcceptedAnswerId="20168" CreationDate="2015-07-13T20:36:54.440" Score="2" ViewCount="728" Body="&lt;p&gt;I have been struggling for two days with the following problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to do a $d$-dimensional interpolation over some data. I tried first&#xA;to use polyharmonic splines, but when the size of data increase, the inversion of matrices is problematic because they are almost singular (large condition&#xA;number). Therefore, I tried to use &lt;a href=&quot;http://www.google.ch/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0CCcQFjAAahUKEwjTj4_39tjGAhUBbhQKHQ7_CwM&amp;amp;url=http%3A%2F%2Fwww.math.iit.edu%2F~fass%2F603_ch4.pdf&amp;amp;ei=OB2kVdOTMIHcUY7-rxg&amp;amp;usg=AFQjCNEChE8IT8me2nV-iwwKs3C2Yizh8g&amp;amp;bvm=bv.97653015,d.d24&amp;amp;cad=rja&quot; rel=&quot;nofollow&quot;&gt;compactly supported radial basis&#xA;function&lt;/a&gt; (CSRBF).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I cannot understand is: why using exactly the same algorithm (see below),&#xA;it works with a Thin-Plate Spline function (or any other polyharmonic basis&#xA;function) but it fails completely with any CSRBF. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I provide a minimal partially working example that implements two working&#xA;functions (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Polyharmonic_spline&quot; rel=&quot;nofollow&quot;&gt;wiki&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$r^2\ln(r)$&lt;/li&gt;&#xA;&lt;li&gt;$r^3$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and two non-working functions (see page 39 of the pdf)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$(1-r)^2$&lt;/li&gt;&#xA;&lt;li&gt;$(1-r)^4_+(4r+1)$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;here is the code, I tried to simplify it as much as I could, so it is not&#xA;efficient but you can play with it and see my comments :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;random&amp;gt;&#xA;&#xA;unsigned int N(10);&#xA;std::vector&amp;lt;double&amp;gt; x;&#xA;std::vector&amp;lt;double&amp;gt; y;&#xA;std::vector&amp;lt;double&amp;gt; z;&#xA;std::vector&amp;lt;double&amp;gt; weights;&#xA;&#xA;extern &quot;C&quot; void dsytrf_(char const&amp;amp; uplo, unsigned int const&amp;amp; n, double *m, unsigned int const&amp;amp; lda, int *ipiv, double *work, int const&amp;amp; lwork, int&amp;amp; info);&#xA;extern &quot;C&quot; void dsytri_(char const&amp;amp; uplo, unsigned int const&amp;amp; n, double *m, unsigned int const&amp;amp; lda, int const *ipiv, double *work, int&amp;amp; info);&#xA;&#xA;class Matrix{&#xA;    public:&#xA;        Matrix(unsigned int L):L_(L),mat_(new double[L_*L_]){}&#xA;        ~Matrix(){ delete[]  mat_; }&#xA;        double const&amp;amp; operator()(unsigned int const&amp;amp; i, unsigned int const&amp;amp; j) const { return mat_[i+j*L_]; }&#xA;        double&amp;amp; operator()(unsigned int const&amp;amp; i, unsigned int const&amp;amp; j) { return mat_[i+j*L_]; }&#xA;        /*method that calls lapack routine to invert a symmetric matrix (only&#xA;         *need the upper triangular part)*/&#xA;        void inv(){&#xA;            int info(1);&#xA;            int* ipiv(new int[L_]);&#xA;            double* work(new double[L_]);&#xA;            dsytrf_('U',L_,mat_,L_, ipiv, work,L_,info);&#xA;            delete[] work;&#xA;            work = new double[L_];&#xA;            dsytri_('U',L_,mat_,L_,ipiv,work,info);&#xA;            delete[] work;&#xA;            delete[] ipiv;&#xA;            for(unsigned int i(0);i&amp;lt;L_;i++)&#xA;                for(unsigned int j(i+1);j&amp;lt;L_;j++)&#xA;                    (*this)(j,i) = (*this)(i,j);&#xA;        }&#xA;&#xA;    private:&#xA;        unsigned int L_;&#xA;        double* mat_;&#xA;};&#xA;/*everything above this line is irrelevent to my question*/&#xA;&#xA;/*with the first or second f, I have the expected behaviour, but if I use the&#xA; *third or fourth definition of f, it fails*/&#xA;double f(double r){ return r*r*r; }&#xA;//double f(double r){ return r*(r&amp;lt;1?log(pow(r,r)):r*log(r)); }&#xA;//double f(double r){ return (r&amp;lt;1.0?(1-r)*(1-r):0.0); }&#xA;//double f(double r){ return (r&amp;lt;1.0?(1-r)*(1-r)*(1-r)*(1-r)*(4*r+1):0.0); }&#xA;&#xA;/*once the weights have been computed, this function returns the interpolated&#xA; * z-value. if (a,b) is one of the known points (x,y), is should exactly the&#xA; * same value z*/&#xA;double extrapolate(double a, double b){&#xA;    double tmp(0);&#xA;    for(unsigned int i(0);i&amp;lt;N;i++)&#xA;        tmp += weights[i]*f(sqrt( (a-x[i])*(a-x[i]) + (b-y[i])*(b-y[i]) ));&#xA;    return tmp;&#xA;}&#xA;&#xA;int main(){&#xA;    /*create a random number generator withing the range [-1,1]*/&#xA;    std::random_device rd;&#xA;    std::mt19937_64 mt(rd());&#xA;    std::uniform_real_distribution&amp;lt;double&amp;gt; dist(-1.0,1.0);&#xA;    for(unsigned int i(0);i&amp;lt;N;i++){&#xA;        x.push_back(dist(mt));//generate random x&#xA;        y.push_back(dist(mt));//generate random y&#xA;        /*evaluate x,y for a given function*/&#xA;        z.push_back(x.back()*y.back()+cos(x.back())*exp(y.back()*y.back()));&#xA;    }&#xA;    /*computes the matrix such that M.weights=z*/&#xA;    Matrix M(N);&#xA;    for(unsigned int i=0;i&amp;lt;N;i++)&#xA;        for(unsigned int j(i+1);j&amp;lt;N;j++)&#xA;            M(i,j) = f(sqrt( (x[i]-x[j])*(x[i]-x[j]) + (y[i]-y[j])*(y[i]-y[j]) ));&#xA;    /*compute the invert of M. warning ! it might give wrong result if one&#xA;    line of m is 0 (all neighbours are too far) but that not where my&#xA;    problem is*/&#xA;    M.inv();&#xA;    /*using the invert of M, compute the weights*/&#xA;    for(unsigned int i(0);i&amp;lt;N;i++){&#xA;        weights.push_back(0.0);&#xA;        for(unsigned int j(0);j&amp;lt;N;j++)&#xA;            weights[i] += M(i,j)*z[j];&#xA;    }&#xA;    /*if it works, this should always return 0*/&#xA;    for(unsigned int i(0);i&amp;lt;N;i++)&#xA;        std::cout&amp;lt;&amp;lt;z[i]-extrapolate(x[i],y[i])&amp;lt;&amp;lt;std::endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code is in C++ and to compile you will need the &lt;code&gt;-std=c++11 -llapack&lt;/code&gt; flags&lt;/p&gt;&#xA;" OwnerUserId="4626" LastEditorUserId="9667" LastEditDate="2015-07-13T23:08:11.193" LastActivityDate="2021-01-17T12:49:47.883" Title="Interpolation using compactly supported radial basis function" Tags="|c++|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20163" PostTypeId="1" AcceptedAnswerId="20180" CreationDate="2015-07-13T23:31:17.363" Score="1" ViewCount="88" Body="&lt;p&gt;Hello I am trying to shift points which have been previously generated in the square area. I am having a trouble with some additional conditions how they should have been shifted. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xt=x(j)+2*rand()*shift - shift&#xA;yt=y(j)+2*rand()*shift - shift&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also I have more conditions to apply:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a) xt&amp;lt;0&#xA;b)xt&amp;gt;a&#xA;c)yt&amp;lt;0&#xA;d)yt&amp;gt;a&#xA;d) the distance (x(i)-xt)**2+(y(i)-yt)**2 &amp;lt;1 for i=1,2,...n; i/=j&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;x(i)&lt;/code&gt; is a randomly generated number and &lt;code&gt;shift&lt;/code&gt; is a parameter in the interval [0.1,1].&lt;code&gt;x(j)&lt;/code&gt; is the randomly generated number again from the &lt;code&gt;x(i)&lt;/code&gt; array. &lt;code&gt;y(j)&lt;/code&gt; and &lt;code&gt;y(i)&lt;/code&gt; is the same as &lt;code&gt;x&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if the conditions are not being executed should I go back to the generating &lt;code&gt;xt&lt;/code&gt; and &lt;code&gt;yt&lt;/code&gt; positions or what would be other way to carry out with this problem?&lt;/p&gt;&#xA;" OwnerUserId="16846" LastActivityDate="2015-07-15T12:24:27.650" Title="Shifting points in fortran" Tags="|numerical-analysis|fortran|computational-chemistry|random-number-generation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20164" PostTypeId="2" ParentId="20152" CreationDate="2015-07-14T00:39:59.513" Score="21" Body="&lt;p&gt;This is not really an &lt;em&gt;authoritative&lt;/em&gt; answer, more a list of issues I think you should consider, and I haven't tested your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;0.&lt;/strong&gt; How did you test your code for correctness and speed? Both are&#xA;  important, somewhat tricky to do well, and you don't give&#xA;  details. In other words, if &lt;em&gt;I&lt;/em&gt; compare your function with the &lt;code&gt;log&lt;/code&gt;&#xA;  on &lt;em&gt;my&lt;/em&gt; machine, will I also get the same numbers, $2.1$, $5.1$? In&#xA;  my experience with reading other people's timing benchmarks in&#xA;  academic literature, it requires a lot of care and precision to get&#xA;  &lt;em&gt;reproducible&lt;/em&gt; timings, which are the only timings anyone is ever&#xA;  going to care about. Microbenchmarks especially are notoriously unreliable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; The common problem with evaluating a function $f(x)$ directly with its (unmodified)&#xA;  Taylor series is the number of terms necessary for convergence. There are 52 bits in the mantissa of a&#xA;  &lt;code&gt;double&lt;/code&gt;, so when $n\approx\frac12$ at the beginning of the Taylor&#xA;  series loop, you can expect the loop to take about 50-ish&#xA;  iterations. That's pretty expensive and should be optimized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.5.&lt;/strong&gt; Have you checked your code for large $n$? I tried &lt;code&gt;1.7976e+308&lt;/code&gt;, which leads to &lt;code&gt;term=inf&lt;/code&gt;, and then to &lt;code&gt;n=1&lt;/code&gt; in the Taylor series loop, leading to &lt;em&gt;extremely&lt;/em&gt; slow convergence: it converges like the Harmonic series, i.e. it doesn't but there will be at most $10^{17}$ terms. As a rule of thumb, you should have some kind of a &quot;max-iteration-count&quot; bound for the loop. In this case it behaves like this because $n$ is finite, but &lt;code&gt;term *= e&lt;/code&gt; overflows to $\infty$ in the argument reduction loop. The correct answer is $709.78266108405500745\ldots$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; Functions implemented in standard libraries are expected to be&#xA;  extremely robust. Returning $10^{-300}$ ($\approx0$) as the logarithm of a&#xA;  negative (or zero) number is not correct. The logarithm of $0$&#xA;  should be $-\infty$, logarithm of a negative number should be NaN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that with a little effort you can sacrifice some of that robustness for performance, e.g., by restricting the argument range or returning slightly less accurate results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; Performance of this kind of code can depend a lot on the CPU&#xA;  architecture it's running on. It's a deep and involved topic, but&#xA;  CPU manufacturers like Intel publish optimization guides that&#xA;  explain the different interactions between your code and the CPU&#xA;  it's running on. Caching can be relatively straightforward, but things like branch prediction, instruction-level parallelism, and pipeline stalls due to data dependencies are difficult to see precisely in high-level code, but matter a lot for performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4.&lt;/strong&gt; Implementing a function like this correctly usually means that&#xA;  you &lt;em&gt;guarantee&lt;/em&gt; that for the input floating-point number $\tilde x$, the&#xA;  output $\tilde y = \tilde f(\tilde x)$ is within a certain distance&#xA;  of the nearest floating-point number to the true value $y = f(\tilde&#xA;  x)$. Verifying this is not totally trivial, there isn't any evidence&#xA;  in your code that you've done this, so I don't know if your function&#xA;  is correct (I'm sure it's quite accurate, but &lt;em&gt;how&lt;/em&gt; accurate?). This is not the same as showing that the Taylor series converges, due to the presence of floating-point roundoff errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4.5.&lt;/strong&gt; A good way to test an untested function for accuracy, would&#xA;  be to evaluate it at each one of the four billion (fewer if you are&#xA;  doing argument reduction correctly, as here) single-precision floats,&#xA;  and compare the errors with the standard log from libm. Takes a bit&#xA;  of time, but at least it's thorough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;5.&lt;/strong&gt; Because you know from the start the precision of doubles, you&#xA;  don't have to have an unbounded loop: the number of iterations can&#xA;  be figured out up front (it's probably about 50). Use this to remove&#xA;  branches from your code, or at least set the number of iterations in&#xA;  advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the usual ideas about loop unrolling apply too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;6.&lt;/strong&gt; It is possible to use approximation techniques other than&#xA;  Taylor series. There are also Chebyshev series (with the Clenshaw&#xA;  recurrence), Pade approximants, and sometimes root-finding methods like Newton's method whenever your function can be recast as the root of a simpler function (e.g., the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fast_inverse_square_root&quot;&gt;famous sqrt trick&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continued fractions are probably not going to be too great, because&#xA;they involve division, which is much more expensive than&#xA;multiplies/adds. If you look at &lt;code&gt;_mm_div_ss&lt;/code&gt; at&#xA;&lt;a href=&quot;https://software.intel.com/sites/landingpage/IntrinsicsGuide/&quot;&gt;https://software.intel.com/sites/landingpage/IntrinsicsGuide/&lt;/a&gt;,&#xA;division has latency of 13-14 cycles and throughput of 5-14, depending on architecture, compared with 3-5/0.5-1 for multiply/add/madd. So in&#xA;general (not always) it makes sense to try to eliminate divisions as&#xA;much as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, mathematics isn't &lt;em&gt;such&lt;/em&gt; a great guide here, because expressions with &lt;em&gt;short&lt;/em&gt; formulas are not necessarily the fastest ones. Mathematics doesn't penalize divisions, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;7.&lt;/strong&gt; Floating-point numbers are stored internally in the form $x =&#xA;  m\times 2^e$ (mantissa $m$, $\frac12&amp;lt;m\leq1$, exponent&#xA;  $e$). The natural log of $x$ is much less natural than the base-2&#xA;  log, for which the first part of your code can be replaced with one&#xA;  call to &lt;code&gt;frexp&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;8.&lt;/strong&gt; Compare your &lt;code&gt;log&lt;/code&gt; with the &lt;code&gt;log&lt;/code&gt; in &lt;code&gt;libm&lt;/code&gt; or&#xA;  &lt;code&gt;openlibm&lt;/code&gt;(e.g.: &lt;a href=&quot;https://github.com/JuliaLang/openlibm/blob/master/src/e_log.c&quot;&gt;https://github.com/JuliaLang/openlibm/blob/master/src/e_log.c&lt;/a&gt;). This&#xA;  is by far the easiest way to find out what other people have already&#xA;  figured out. There are also specially-optimized versions of &lt;code&gt;libm&lt;/code&gt;&#xA;  specific to CPU manufacturers, but those usually don't have their&#xA;  source code published.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Boost::sf has some special functions, but not the basic ones. It might be instructive to look at the source of log1p, though: &lt;a href=&quot;http://www.boost.org/doc/libs/1_58_0/libs/math/doc/html/math_toolkit/powers/log1p.html&quot;&gt;http://www.boost.org/doc/libs/1_58_0/libs/math/doc/html/math_toolkit/powers/log1p.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also open-source arbitrary-precision arithmetic libraries&#xA;like mpfr, which might use different algorithms than libm due to the higher precision required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;9.&lt;/strong&gt; Higham's Accuracy and Stability of Numerical Algorithms is a&#xA;  good upper-level introduction to analyzing errors of numerical&#xA;  algorithms. For approximation algorithms themselves, Approximation&#xA;  Theory Approximation Practice by Trefethen is a good reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;10.&lt;/strong&gt; I know this is said a bit too often, but reasonably large software projects rarely depend on runtime of one small function being called over and over again. It's not &lt;em&gt;so&lt;/em&gt; common to have to worry about performance of log, unless you've profiled your program and made sure it's important.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-07-14T00:46:31.977" LastActivityDate="2015-07-14T00:46:31.977" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20165" PostTypeId="1" CreationDate="2015-07-14T01:47:54.997" Score="6" ViewCount="3169" Body="&lt;p&gt;I am trying to do analysis on a data set of atomic coordinates generated form lammps. I simulated an alpha glycine crystal in a triclinic box. The box vectors look like the following, where xy,xz,and yz, are the tilt factors. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    xlo = -0.34; xhigh = 40.466&#xA;    ylo = -1.6283; yhigh = 49.50861&#xA;    zlo = 0.77; zhigh = 39.79&#xA;    xy  = -8.62; xz = 0.0; yz = 0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the case of a cubic box, the way to implement periodic boundary conditions is trivial.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if(x.gt.xhigh)x=x-xbox&#xA;if(x.lt.xlo)x=x+box&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I cannot seem to figure out how to handle the tilt factors, for the triclinic box, to implement the PBC's correctly . Could someone please provide a code snippet outlining this for me?&lt;/p&gt;&#xA;" OwnerUserId="16889" LastEditorUserId="9667" LastEditDate="2015-08-13T21:45:18.657" LastActivityDate="2015-10-13T17:15:13.497" Title="periodic boundary conditions for triclinic box" Tags="|boundary-conditions|molecular-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20166" PostTypeId="1" AcceptedAnswerId="20171" CreationDate="2015-07-14T07:48:34.807" Score="6" ViewCount="591" Body="&lt;p&gt;Ok so I am nearing the completion of my finite element Navier-Stokes solver that uses the $\theta$-method for time stepping and the pressure correction method for the pressure. I am following the notes &lt;a href=&quot;http://ta.twi.tudelft.nl/users/vuik/burgers/fem_notes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ta.twi.tudelft.nl/users/vuik/burgers/fem_notes.pdf&lt;/a&gt; by Ir. A. Segal (page 74-75). So far everything seems to work &lt;strong&gt;except&lt;/strong&gt; there appears to be a bug in how I implement the pressure projection. Allow me to explain using some pictures...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider flow going over a step. When I run my simulation I get: &lt;img src=&quot;https://i.stack.imgur.com/PLdyb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first glance everything appears to be ok. However you may notice that in the bottom left corner the pressure is slightly larger (I know its hard to see). This bottom corner is exactly where I specified my one Dirichlet boundary condition on the Poisson equation necessary for the system to be non-singular. If we look at the corresponding u-velocity the error becomes much more clear: &lt;img src=&quot;https://i.stack.imgur.com/wXQLk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly the velocity should not be peaking at the bottom left-hand corner. I have tried moving the Dirichlet condition away from the corner, but this does not fix the problem. For example moving it up the left-hand side boundary results in the pressure and u-velocity looking like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Oc4bE.png&quot; alt=&quot;enter image description here&quot;&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/wXjRX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the issue is with how I am solving the Poisson equation that arises from the pressure correction method. In particular with how I am implementing the boundary conditions. Ok lets look at the math now (hopefully someone can shed some light on what I am doing wrong). From the above cited paper, we need to solve the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^{2}(p^{n+1}-p^{n}) = \frac{\nabla{}\cdot{\vec{u}^{*}}}{\theta{}dt}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this with finite elements we multiply by a test function $\phi_{j}$ and integrate over the domain (I replace $\frac{1}{\theta{}dt}$ with the constant c):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int\int_{\Omega}{\nabla^{2}(p^{n+1}-p^{n})\phi_{j}}dA = c\int\int_{\Omega}{\nabla{}\cdot{\vec{u}^{*}}\phi_{j}}dA$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Greens theorem,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int\int_{\Omega}{\nabla{}\cdot{\vec{F}}}dA = \int_{\partial\Omega}{\vec{F}\cdot{\vec{n}}}dS$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int\int_{\Omega}{\nabla^{2}(p^{n+1}-p^{n})\phi_{j}}dA = -\int\int_{\Omega}{\nabla{(p^{n+1}-p^{n})}\cdot{\nabla{\phi_{j}}}}dA + \int_{\partial\Omega}{(\nabla{(p^{n+1}-p^{n})\phi_{j}})\cdot{\vec{n}}}dS$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we prescribe Neumann boundary conditions for pressure on all boundaries &lt;strong&gt;except&lt;/strong&gt; for &lt;strong&gt;one&lt;/strong&gt; point on the boundary for which we prescribe a Dirichlet condition. This is necessary since otherwise the resulting Poisson matrix system would be singular. Now my theory as to what is going wrong is currently in the above equation I just get rid of the last term (because I assume a Neumann boundary condition on &lt;strong&gt;all&lt;/strong&gt; boundaries for pressure, i.e. $\nabla{(p^{n+1}-p^{n})}\cdot{n} = 0$. However I am now beginning to think that I must account for that one point on the boundary. So for that one Dirichlet point I need to retain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{\partial\Omega}{(\nabla{(p^{n+1}-p^{n})\phi_{j}})\cdot{\vec{n}}}dS$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if this is right? Has anyone come accross similar problems when implementing the pressure correction method? I have been racking my brain trying to figure out what could be going wrong. In the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^{2}(p^{n+1}-p^{n}) = \frac{\nabla{}\cdot{\vec{u}^{*}}}{\theta{}dt}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what, if any, boundary conditions do you apply to $\frac{\nabla{}\cdot{\vec{u}^{*}}}{\theta{}dt}$? Should it be zero on the boundary? How does one enforce the Dirichlet condition at one point for pressure in the global stiffness matrix? Any help is &lt;strong&gt;really&lt;/strong&gt; appreciated. I have run out of ideas!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok so after reading the comments and running so more tests I am more convinced my problem is in how I implement the one Dirichlet boundary condition on the pressure. So far I have been following &lt;a href=&quot;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&lt;/a&gt; (pg 12-13) in prescribing the one Dirichlet condition on pressure. Basically if Kp is my global stiffness matrix for the Poisson equation then I simply add a 1 to the first entry, i.e.&#xA;Kp[0][0] = Kp[0][0] + 1. Is this correct? I dont do anything special on the r.h.s of the Poisson equation, i.e. $\frac{\nabla{}\cdot{\vec{u}^{*}}}{\theta{}dt}$.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-07-14T20:00:22.653" LastActivityDate="2018-04-30T10:23:53.150" Title="Implementing the pressure correction method using finite elements" Tags="|finite-element|simulation|navier-stokes|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20167" PostTypeId="1" AcceptedAnswerId="20169" CreationDate="2015-07-14T09:22:34.293" Score="7" ViewCount="598" Body="&lt;p&gt;I am doing research in electromagnetics (a branch of physics) and I deal with complex numbers in my computing tasks (I use MATLAB). Let's say $z$ is a complex number. To calculate the square of the magnitude of $z$, which is $|z|^2$, I can use two different ways that are mathematically equivalent:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$|z|^2 = \operatorname{abs}(z)^2$&lt;/li&gt;&#xA;&lt;li&gt;$|z|^2 = zz^*$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, the outcomes of the two operations above are slightly different (numerically). The difference is usually in the last decimal point of the result, so practically this does not affect my computations. But I'm curious why this difference occurs. Now, the interesting thing is the following. I can also calculate $|z|^2$ as&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$|z|^2 = \operatorname{real}(z)^2+\operatorname{imag}(z)^2$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and this computation equals the result of $|z|^2 = zz^*$. So, how is the absolute value function (abs) implemented (in MATLAB or elsewhere)? Why does it give a different result? Which one of the results is more accurate?&lt;/p&gt;&#xA;" OwnerUserId="16932" LastEditorUserId="-1" LastEditDate="2015-07-14T15:00:22.010" LastActivityDate="2015-07-14T15:00:22.010" Title="What is the numerical difference between abs(z)^2 and z x z*, where z is a complex number" Tags="|matlab|precision|accuracy|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20168" PostTypeId="2" ParentId="20162" CreationDate="2015-07-14T09:44:52.470" Score="3" Body="&lt;p&gt;I feel so stupid... I know what is wrong...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for(unsigned int i=0;i&amp;lt;N;i++)&#xA;    for(unsigned int j(i);j&amp;lt;N;j++)&#xA;        M(i,j) = f(sqrt( (x[i]-x[j])*(x[i]-x[j]) + (y[i]-y[j])*(y[i]-y[j])));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where I just changed the &lt;code&gt;i+1&lt;/code&gt; into &lt;code&gt;i&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4626" LastEditorUserId="4626" LastEditDate="2021-01-17T12:49:47.883" LastActivityDate="2021-01-17T12:49:47.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20169" PostTypeId="2" ParentId="20167" CreationDate="2015-07-14T10:50:04.750" Score="7" Body="&lt;p&gt;At least in MATLAB, I believe &lt;code&gt;abs(z)&lt;/code&gt; is implemented as &lt;code&gt;sqrt(z*z')&lt;/code&gt;. The extra square-root and squaring operation reduces numerical precision.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; z = randn + randn * i&#xA;z =&#xA;   0.5377 + 1.8339i&#xA;&#xA;&amp;gt;&amp;gt; abs(z)^2 - z*z'&#xA;ans =&#xA;   4.4409e-16&#xA;&#xA;&amp;gt;&amp;gt; abs(z)^2 - sqrt(z*z')^2&#xA;ans =&#xA;     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2015-07-14T10:50:04.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20170" PostTypeId="2" ParentId="20149" CreationDate="2015-07-14T11:29:36.887" Score="1" Body="&lt;p&gt;In Tecplot you can go option-&gt;color map, where you can find lot of build in color bands available. You can play a lot in &quot;Base color map&quot; option to get desire color contour you want.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Click a small icon next to &quot;Contour&quot; in &quot;Zone surface&quot; pallet then click levels where you can change color bands etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using user defined macros you can control that but this is slightly tedious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last but not least you shall post your problem here: &lt;a href=&quot;http://tecplottalk.com/forums/&quot; rel=&quot;nofollow&quot;&gt;http://tecplottalk.com/forums/&lt;/a&gt;. You may get good answer.&lt;/p&gt;&#xA;" OwnerUserId="16930" LastActivityDate="2015-07-14T11:29:36.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20171" PostTypeId="2" ParentId="20166" CreationDate="2015-07-14T13:18:45.450" Score="3" Body="&lt;p&gt;It depends on how you set your Dirichlet conditions, but they are often equivalent to setting $\phi_j=0$ on those nodes, so you shouldn't have to worry about that equation as long as you set the Dirichlet condition correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, that being said, I've never been a huge fan of pinning one pressure node in traditional mixed formulations for Navier-Stokes using Taylor-Hood elements. You have other choices. If your linear solver is a Krylov method, you can usually get away with doing nothing. That is, form the matrix with the Neumann/natural boundary condition everywhere, and just feed it to the solver. As long as your preconditioner is non-singular, you will usually iterate to a good solution with the Krylov method &quot;picking&quot; your pressure level for you. Your other choice is to impose an average pressure zero condition across the whole domain&#xA;$$&#xA;\int_\Omega p \; dx =0&#xA;$$&#xA;which, when you plug in your pressure representation will give you a constraint equation for your system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edited to add based on the question edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you have the $j^{\rm th}$ node where you want to apply the boundary condition $p_j=c$, then you can replace row $j$ of your matrix with all zeros and a 1 on the diagonal. Then you put $c$ in the RHS vector on row $j$ as well (don't add, replace). Now on whatever other rows $p_j$ appeared in your LHS, i.e. non-zeros row entries in original column $j$, you need to subtract $c \times K_p[i][j]$ from the RHS vector on row $i$. You then set $K_p[i][j] = 0$ in the original matrix. That will enforce the Dirichlet condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get away with not modifying anything other than row $j$, but you will destroy the symmetry of your matrix. If your solver required symmetry (like Conjugate Gradient or Cholesky), then you have to pick a non-symmetric solver. If you can get the bookkeeping right, it's probably better to modify the matrix.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-07-14T20:45:02.517" LastActivityDate="2015-07-14T20:45:02.517" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20172" PostTypeId="1" AcceptedAnswerId="20179" CreationDate="2015-07-14T18:08:41.090" Score="14" ViewCount="64794" Body="&lt;p&gt;I am solving a system of linear equations, $\underline {\dot x}=\underline A\cdot \underline x$, numerically. I have done this using the popular of methods of Euler and &lt;a href=&quot;https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods&quot; rel=&quot;noreferrer&quot;&gt;Runge-Kutta&lt;/a&gt; (RK). I have noticed quite a difference between the two in accuracy to the analytic solution.&#xA;What is the reason for this?&lt;/p&gt;&#xA;" OwnerDisplayName="user16937" LastEditorUserId="625" LastEditDate="2015-12-08T09:41:11.960" LastActivityDate="2015-12-08T15:34:15.227" Title="Why are Runge-Kutta and Euler's method so different?" Tags="|time-integration|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20173" PostTypeId="2" ParentId="20172" CreationDate="2015-07-14T18:40:49.453" Score="4" Body="&lt;p&gt;Basically the reason they have different levels of accuracy is in the way they are derived. The (I'm assuming forward)Euler method is first order and can be derived a number of ways. Most easily from Taylor's theroem. In that method, only the second order term is retained thereby making the error first order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Runge kutta methods on the other hand are staged and have very specifically chosen coefficients to &quot;cancel&quot; out several orders of error terms. Rk4 is fourth order since the zeroth, first, second, third order error terms cancel out by the choice of stage coefficients. The derivation of these coefficients is not very difficult and can probably be found on wikipedia if you're interested in seeing it.&lt;/p&gt;&#xA;" OwnerUserId="16936" LastActivityDate="2015-07-14T18:40:49.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20174" PostTypeId="2" ParentId="20172" CreationDate="2015-07-14T18:59:11.827" Score="9" Body="&lt;p&gt;The Euler method does not take into account the curvature of the solution, so it tends to give different results depending on the step size. RK, depending on the order, takes into account the curvature. And this makes the estimated &quot;next step&quot; more accurate. Basically, if you are pretending a straight line is a good approximation of a curve (Euler) you will always be overshooting your solution. But if you take account of the curvature (RK) you can follow the curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare with the famous hockey quote (Gretzky): Euler skates to where the puck is; Runge-Kutta skates to where the puck is going to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend you read about these algorithms. You can start with Wikipedia articles &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler_method&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16935" LastEditorUserId="9667" LastEditDate="2015-07-14T20:16:24.607" LastActivityDate="2015-07-14T20:16:24.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20175" PostTypeId="2" ParentId="1014" CreationDate="2015-07-14T20:04:43.907" Score="2" Body="&lt;p&gt;Interval arithmetics is very useful for geometric algorithms. Such geometric algorithms take as input a set of geometric objects (e.g. a set of points) and construct a combinatorial data structure (e.g. a triangulation) based on spatial relations between the points. These algorithms depend on a small number of functions, called 'predicates', that take as input a fixed number of geometric objects and return a discrete value (typically one of 'above,aligned,below'). Such predicates typically correspond to the sign of a determinant of the point's coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using standard floating-point numbers is not sufficient, since it may fail accurately compute the sign of the determinant, and even worse, return incoherent results (i.e., saying that A is above B AND B is above A, thus making the algorithm create a mess instead of a mesh !).&#xA;Systematically using multi-precision (such as in the Gnu Multi-Precision library and its MPFR extension to multi-precision floating point numbers) works but causes a significant performance penalty. &#xA;When the geometric predicate is the sign of something (as in most cases), using interval arithmetics allows one to do a faster computation, and then only launch the more expansive multi-precision computation if zero is in the interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such an approach is used in several large computational geometry codes (e.g. CGAL).&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2015-07-14T20:04:43.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20176" PostTypeId="2" ParentId="20165" CreationDate="2015-07-14T20:43:02.957" Score="2" Body="&lt;p&gt;I have never implemented periodicity for this symmetry class, although I have done it for hexagonal crystals before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us start with a (2D) box like the one depicted in the image&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/l1YrB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, we have one axis aligned with $x$ and the other one with the vector $u_2$. The sides of the box are given by $y_\text{low}$, $y_\text{high}$, and the oblique line $ax + by=c_1$, $ax + by=c_2$. And make the sides $A$ and $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, when a particle goes above $y_\text{high}$ you reassign it with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(x,y) = (x,y) - B u_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in the other case, when $y &amp;lt; y_\text{low}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(x,y) = (x,y) + B u_2 \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the $x$ direction the new position is easier is just&#xA;$$(x,y) = (x,y) -  A u_1$$&#xA;and&#xA;$$(x,y) = (x,y) +  A u_1 \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difficulty in this case is knowing when the particle goes out of the boundary. In the example above, the particle goes out to the right if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ax + by &amp;lt; c_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and goes out to the left if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ax + by \geq c_1 \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the outside lies below the line $ax + by = c_2$ if the shear of the box is to the right and above if it is to the left. This reasoning can be extended to more dimensions (although in your particular example there is just one shear in the box).&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-07-14T20:43:02.957" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20177" PostTypeId="2" ParentId="10533" CreationDate="2015-07-14T22:08:26.843" Score="2" Body="&lt;p&gt;You can rephrase your question as an optimization problem and solve numerically with gradient descent. You would probably converge quite fast, because your initial guess will be the solution you already have for $\mu^2=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More formally, we like to find $x$ that minimizes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x) = ||(LL^T+μ^2I)x -y||^2$, where $L, μ^2, y$ are given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve it, start with $f(x_0) = f(A^{-1}y)$ and update $f(x)$ with gradient descent steps until convergence. Each evaluation of the gradient should cost you $O(d^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your initial guess is good, other iterative approaches will probably work well. See for example chapter 2, &lt;a href=&quot;http://ocw.mit.edu/courses/mathematics/18-409-topics-in-theoretical-computer-science-an-algorithmists-toolkit-fall-2009/lecture-notes/MIT18_409F09_scribe21.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2015-07-14T22:49:47.270" LastActivityDate="2015-07-14T22:49:47.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20178" PostTypeId="1" CreationDate="2015-07-14T22:27:57.297" Score="12" ViewCount="624" Body="&lt;p&gt;Let us start with a problem of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\mathcal{L} + k^2) u=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with a set of given boundary conditions (&lt;a href=&quot;https://en.wikipedia.org/wiki/Dirichlet_boundary_condition&quot;&gt;Dirichlet&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Neumann_boundary_condition&quot;&gt;Neumann&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Robin_boundary_condition&quot;&gt;Robin&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Periodic_boundary_conditions&quot;&gt;Periodic&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Bloch_wave&quot;&gt;Bloch-Periodic&lt;/a&gt;). This corresponds with finding the eigenvalues and eigenvectors for some operator $\mathcal{L}$, under some geometry, and boundary conditions. One can obtain a problem like this in acoustics, electromagnetism, elastodynamics, quantum mechanics, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that one can discretize the operator using different methods, e.g, Finite Difference Methods to obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[A]\{U\} = k^2 \{U\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or using, Finite Element Methods to obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[K]\{U\} = k^2 [M]\{U\} \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one case getting an &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors&quot;&gt;eigenvalue problem&lt;/a&gt; and a &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigendecomposition_of_a_matrix#Generalized_eigenvalue_problem&quot;&gt;generalized eigenvalue problem&lt;/a&gt; in the other. After obtaining the discrete version of the problem one uses a solver for the eigenvalue problem.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Some thoughts&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The method of Manufactured Solutions is not useful in this case since there is no source term to balance the equation.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;One can verify that the matrices $[K]$ and $[M]$ are well captured using a frequency domain problem with source term, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[\nabla^2 + \omega^2/c^2] u(\omega) = f(\omega) \enspace ,\quad \forall \omega \in [\omega_\min, \omega_\max]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;instead of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[\nabla^2 + k^2] u = 0 \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this will not check the solver issues.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Maybe, one can compare solutions for different methods, like FEM and FDM.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;What is the way to verify the solutions (eigenvalue-eigenvector pairs) for discretization schemes due to numerical methods like FEM and FDM for eigenvalue problems?&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-08-08T01:21:31.417" LastActivityDate="2019-03-03T05:22:56.770" Title="Verification in Eigenvalue problems" Tags="|pde|finite-element|eigensystem|verification|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="20179" PostTypeId="2" ParentId="20172" CreationDate="2015-07-15T02:31:26.343" Score="22" Body="&lt;p&gt;First thing, you could have mentioned, what RK method you have used. Here is a brief introduction to RK methods and Euler method, working, there merits and demerits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Euler method&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Euler's method is first order method. It is a straight-forward method that estimates the next point based on the rate of change at the current point and it is easy to code. It is a single step method. Notably, Forward Euler's method is unconditionally unstable for un-damped oscillating systems (such as a spring-mass system or wave equations) in space desctretization. For complex problems and/or boundary condition it may fail. It can be used for basic numerical analysis. This method is not commonly used for spatial discretization but some times used in time discretization. This scheme is not recommended for hyperbolic differential equation because this is more diffusive. Order of convergence of this scheme with grid refinement is very poor. Extending Euler method to higher order method is easy and straight forward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;RK methods:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Runge-Kutta methods are actually a family of schemes derived in a specific style. You can refer this link to get a basic idea of RK methods:&#xA;&lt;a href=&quot;http://web.mit.edu/10.001/Web/Course_Notes/Differential_Equations_Notes/node5.html&quot; rel=&quot;noreferrer&quot;&gt;http://web.mit.edu/10.001/Web/Course_Notes/Differential_Equations_Notes/node5.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The forward Euler method is actually the simplest RK method (1 stage, first order). Higher order accurate RK methods are multi-stage because they involve slope calculations at multiple steps at or between the current and next discrete time values. The next value of the dependent variable is calculated by taking a weighted average of these multiple stages based on a Taylor series approximation of the solution. The weights in this weighted average are derived by solving non-linear algebraic equations which are formed by requiring cancellation of error terms in the Taylor series. Developing higher order RK methods is tedious and difficult without using symbolic tools for computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most popular RK method is RK4 since it offers a good balance between order of accuracy and cost of computation. RK4 is the highest order explicit Runge-Kutta method that requires the same number of steps as the order of accuracy (i.e. RK1=1 stage, RK2=2 stages, RK3=3 stages, RK4=4 stages, RK5=&lt;em&gt;6 stages&lt;/em&gt;, ...). Beyond fourth order the RK methods become relatively more expensive to compute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Usually error in Euler method is higher than higher order RK method (RK2, RK3, etc.), because truncation error in higher order methods is less compared to Euler method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In some of the beginner level literature in numerical methods, it is loosely  mentioned that higher order methods (say, RK4) give less error than lower order method (say, Euler method). Most of the time this is true, but not all the time. This property depends on the mesh and initial condition and differential equations you have considered.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the exact solution to the differential equation is a polynomial of order $n$, it will be solved exactly by an $n$-th Runge-Kutta method. For example, forward Euler will be exact if the solution is a line. RK4 will be exact if the solution is a polynomial of degree 4 or less.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Initial &quot;absolute maximum difference error&quot; in RK4 method is equal (or) higher than Euler method for coarse grid  and reduces with refining grid for problems with shorter waves relative &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782509004216&quot; rel=&quot;noreferrer&quot;&gt;to&lt;/a&gt; &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0021999185901883&quot; rel=&quot;noreferrer&quot;&gt;grid&lt;/a&gt;. Because convergence rate of RK4 method is more than Euler.  Please note that coarseness or fineness of grid is completely based on differential equation, initial condition and numerical scheme. Please refer following &lt;a href=&quot;https://scicomp.stackexchange.com/questions/20197/error-in-result-of-finite-difference-approximation-when-refining&quot;&gt;link&lt;/a&gt; for more details. Though that is based on differentiation,  we can make a relative comparison between time numerical integration  and differentiation as long as the &quot;numerical integration&quot; is stable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can try this experiment in your code with different differential equations, different number of grids with Euler and RK4 if you have enough time. Order of convergence of scheme can be find out by calculating slope of plot - $\log(\mathrm{error})$ vs $\log(\mathrm{grid\ size})$.&lt;/p&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-12-08T15:34:15.227" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20180" PostTypeId="2" ParentId="20163" CreationDate="2015-07-15T12:24:27.650" Score="1" Body="&lt;p&gt;In order to apply the last condition, I suggest you displace the points like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;REAL, PARAMETER :: PI = 3.1415927&#xA;...&#xA;rt=shift*sqrt(rand())&#xA;tt=2.d0*PI*rand()&#xA;xt=x(j)+rt*cos(tt)&#xA;yt=y(j)+rt*sin(tt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;see &lt;a href=&quot;http://mathworld.wolfram.com/DiskPointPicking.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest and quickest solution to constrain the new points to the interval $[0,a]^2$ is to impose some form of boundary condition. For example, periodic boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if(xt&amp;lt;0) xt=xt+a&#xA;else if(xt&amp;gt;a) xt=xt-a&#xA;same for yt...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or you could reflect the points back across the boundary:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if(xt&amp;lt;0) xt=-xt&#xA;else if(xt&amp;gt;a) xt=2*a-xt&#xA;same for yt...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11739" LastActivityDate="2015-07-15T12:24:27.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20181" PostTypeId="1" AcceptedAnswerId="20185" CreationDate="2015-07-15T15:44:08.533" Score="9" ViewCount="448" Body="&lt;p&gt;I have an ODE of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{dy}{dt} = -i H y \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $y$ is a complex vector and $H$ is a time dependent Hermitian matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The norm of the solution $y(t)$ at any point in time should be 1, but due to accumulation of small numerical errors it ends up being substantially off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A solution for this problem that is used by the &lt;code&gt;qutip&lt;/code&gt; library is to reset the ode solver every so often by normalizing $y$ at some time $t'$ and resuming from that point. Is there a rigorous explanation why this is a good idea (it seems to work in practice)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way? Even better if it does not require a modification of the solver code?&lt;/p&gt;&#xA;" OwnerUserId="1028" LastEditorUserId="9667" LastEditDate="2015-07-15T16:43:08.473" LastActivityDate="2015-07-15T19:12:42.677" Title="Forcing an ODE solver to preserve the norm" Tags="|ode|stability|scipy|quantum-mechanics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20182" PostTypeId="1" CreationDate="2015-07-15T16:35:27.793" Score="3" ViewCount="161" Body="&lt;p&gt;I was reading about Petrov-Galerkin Enrichment Method for Darcy equations. Here are a couple papers that discuss this in detail:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782507000059&quot; rel=&quot;nofollow&quot;&gt;A Petrov–Galerkin enriched method: A mass conservative finite element method for the Darcy equation&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/080724381&quot; rel=&quot;nofollow&quot;&gt;A Symmetric Nodal Conservative Finite Element Method for the Darcy Equation&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, I am not sure I understand what they are doing completely and would like some help understanding how they are constructing these &quot;multiscale&quot; or enrichment functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand, it seems they are &quot;enriching&quot; the $P_1$ velocity by an $RT_0$, but what exactly does this mean? Specifically, what is meant when they &quot;correct edge residuals&quot;? Because I would like to implement this in something like FEniCS (if possible) but in order to do that I need to understand this method completely.&lt;/p&gt;&#xA;" OwnerUserId="14576" LastEditorUserId="9667" LastEditDate="2015-07-15T16:50:06.377" LastActivityDate="2015-07-15T16:50:06.377" Title="Petrov-Galerkin enrichment method for Darcy equation" Tags="|finite-element|fenics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20183" PostTypeId="2" ParentId="20172" CreationDate="2015-07-15T17:29:01.260" Score="7" Body="&lt;p&gt;Forgive me for adding a me-too answer, but I just couldn't resist including this page from Press, et al: &quot;Numerical Recipes in C&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/B3FkE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="1262" LastActivityDate="2015-07-15T17:29:01.260" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20184" PostTypeId="2" ParentId="20103" CreationDate="2015-07-15T18:48:57.837" Score="3" Body="&lt;p&gt;For the incompressible Navier-Stokes equations, in practice, where ever there is a Dirichlet boundary condition on the velocity, Neumann boundary condition is applied for the pressure. It does not matter if it no-slip or &quot;velocity-inlet&quot; or &quot;velocity-outlet&quot;. You should &lt;strong&gt;not&lt;/strong&gt; specify the pressure where the velocity is specified. Your figure shows velocity specified at all the boundaries, so you would need to apply Neumann BCs for pressure at all the boundaries including the left and right faces of the domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with a &quot;velocity-outlet&quot; is that this velocity is not known &lt;em&gt;apriori&lt;/em&gt;. and if it chosen poorly it will affect the solution upstream.&#xA;The other alternative is to use a &quot;do nothing&quot; boundary condition at the outflow, for example by setting derivative of the velocity to be zero at the outflow and specfying the pressure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For concrete example using finite difference you could look at: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Van Kan, &quot;A second-order accurate pressure-correction scheme for viscous incompressible flow.&quot; SIAM Journal on Scientific and Statistical Computing 7.3 (1986): 870-891.&#xA;&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0907059&quot; rel=&quot;nofollow&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/0907059&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2015-07-15T18:48:57.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20185" PostTypeId="2" ParentId="20181" CreationDate="2015-07-15T19:12:42.677" Score="14" Body="&lt;p&gt;The best approach is to use an ODE solver that is guaranteed to conserve the norm of the initial condition, i.e., for which $\|y_n\| = \|y_0\|$ for all $n\in\mathbb{N}$. Such solvers exist, and are called &lt;em&gt;geometric integrators&lt;/em&gt;, since they preserve geometric properties of the exact solution (in this case, that energy is conserved, i.e., $\frac{d}{dt}\|y(t)\| = 0$). (A special class are &lt;em&gt;symplectic integrators&lt;/em&gt; for ODE systems arising in Hamiltonian mechanics.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest such integrator (that is not specific to Hamiltonian systems) is actually a quite standard method, variously referred to as &lt;em&gt;trapezoidal&lt;/em&gt;, &lt;em&gt;implicit mid-point&lt;/em&gt;, or &lt;em&gt;Crank-Nicolson&lt;/em&gt; method. In your case, it amounts to solving in every step the linear system&#xA;$$\left(I+\frac{i\,h_n}2 H(t_{n+1})\right) y_{n+1} =  \left(I-\frac{i\,h_n}2 H(t_{n})\right) y_n,$$&#xA;where $I$ is the identity matrix and $h_n = t_{n+1}-t_n$ is the current time step length. This method is implicit, but second-order accurate and A-stable. (I don't think there's a first order geometric integrator.) I don't know if it's included in ODEPACK (the basis of scipy's &lt;code&gt;odeint&lt;/code&gt; routines), but it shouldn't be too hard to implement yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know more about geometrical integration:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Probably the most accessible discussion of geometric integration is Chapter 5 in &lt;a href=&quot;http://www.cambridge.org/us/academic/subjects/mathematics/numerical-analysis/first-course-numerical-analysis-differential-equations-2nd-edition&quot;&gt;A. Iserles, &lt;em&gt;A First Course in the Numerical Analysis of Differential Equations&lt;/em&gt;, 2nd edition, Cambridge University Press, 2008&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &quot;bible&quot; is of course &lt;a href=&quot;http://link.springer.com/book/10.1007%2F3-540-30666-8&quot;&gt;E. Hairer, G. Wanner, C. Lubich, &lt;em&gt;Geometric Numerical Integration&lt;/em&gt;, 2nd edition, Springer, 2006&lt;/a&gt; (see Chapter VII.6 for applications in quantum mechanics). There's also a survey article in &lt;a href=&quot;http://journals.cambridge.org/action/displayAbstract?fromPage=online&amp;amp;aid=165633&amp;amp;fileId=S0962492902000144&quot;&gt;Acta Numerica 12, 2003, p. 399-450&lt;/a&gt; (here's a &lt;a href=&quot;http://www.math.kit.edu/ianm3/lehre/geonumint2009s/media/gni_by_stoermer-verlet.pdf&quot;&gt;preprint&lt;/a&gt;), although that seems to focus on second-order ODEs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Specifically the application to the Schrödinger equation is discussed in &lt;a href=&quot;http://www.ems-ph.org/books/book.php?proj_nr=143&quot;&gt;E. Faou, &lt;em&gt;Geometric Numerical Integration and Schrödinger Equations&lt;/em&gt;, Zurich Lectures in Advanced Mathematics, EMS, 2012&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-07-15T19:12:42.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20186" PostTypeId="2" ParentId="20096" CreationDate="2015-07-15T20:37:53.767" Score="1" Body="&lt;p&gt;I found a specific approach for dirichlet BCs &lt;a href=&quot;http://www.mathematik.uni-dortmund.de/~kuzmin/cfdintro/cfd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; in lecture 4. It shows one possible way to solve problems with complex boundaries. Because of the linear interpolation, I think up to second order accuracy is possible. Relevant page shown below:&lt;br&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/E0pvk.png&quot; alt=&quot;Lecture 4, slide 16&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2015-07-15T20:37:53.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20189" PostTypeId="1" AcceptedAnswerId="20193" CreationDate="2015-07-16T11:23:06.170" Score="2" ViewCount="599" Body="&lt;p&gt;I need to find C/C++ libraries which offer function for computing Schur complement. I know about MUMPS and Pastix, but I need more of them to compare them in my research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any experience with other libraries?&lt;/p&gt;&#xA;" OwnerUserId="13211" LastActivityDate="2015-07-17T09:40:07.313" Title="Recommendation for C/C++ library which offers Schur complement functions?" Tags="|linear-algebra|matrix|c++|c|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20190" PostTypeId="1" CreationDate="2015-07-16T11:34:12.270" Score="2" ViewCount="261" Body="&lt;p&gt;I built a sparse matrix &lt;code&gt;A&lt;/code&gt; at each step as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% 1 &amp;lt; DX &amp;lt; 120000 &#xA;&#xA;A = sparse(i,j,s,DX,DX,6*DX)&#xA;&#xA;b = (1, DX)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem that I am dealing with is a discretization problem. I have maximum 120000 nodes. Each of these nodes have special characters and I choose only the ones that meet a (previously) defined criteria. The number of these chosen ones is &lt;code&gt;DX&lt;/code&gt; and is completely dependent on the physical process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using backslash in &lt;code&gt;x = A\b&lt;/code&gt;. But as the size of A could become quite big, the computational time rises drastically (more than &lt;code&gt;10e5&lt;/code&gt; time steps have &lt;code&gt;DX &amp;gt; 6e4&lt;/code&gt;). As far as I know, backslash operation is already well optimized in MATLAB but I would like to know:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Would it make sense to use &lt;code&gt;codegen&lt;/code&gt; and convert the code to C? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does any one know an alternative method instead of backslash, so that the computational time decreases (maybe an iterative method?)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16956" LastEditorUserId="9667" LastEditDate="2015-07-16T19:28:10.940" LastActivityDate="2015-11-18T02:15:15.930" Title="High computational time in using backslash for soving sparse matrix" Tags="|matlab|sparse-matrix|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="20191" PostTypeId="1" CreationDate="2015-07-16T11:37:11.233" Score="2" ViewCount="531" Body="&lt;p&gt;I managed to implement the Nearest-Neighboor Ising Model with periodic boundary conditions, it was doable. I also made a modified version of it, where the interaction would go further than the nearest neighbors. In this case the implementation of periodic boundary conditions got more complicated (matrices instead of single variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example would be simulation software which does not have an implementation of periodic boundaries. The implementation of them on my own would be possible (for sure if it is open source). It would maybe even be possible to &lt;em&gt;hack&lt;/em&gt; together some scripts that do this, depending on the scripting possibilities of the tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is therefore:&lt;/strong&gt; &#xA;What would be a the general route for implementing periodic boundary conditions? What is considered quick and dirty, and what the proper way? What could people who try to implement them be missing to do/notice? (I know it depends on the problem)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there creative enumeration techniques or other smart tricks that can help to create a periodic boundary. What &lt;em&gt;general principles&lt;/em&gt; should be adhered to when doing it for FEM, FDM or others?&lt;/p&gt;&#xA;" OwnerUserId="16957" LastEditorUserId="9667" LastEditDate="2015-07-16T19:32:42.843" LastActivityDate="2015-07-18T10:25:01.207" Title="Periodic boundaries - implementation strategies" Tags="|boundary-conditions|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20192" PostTypeId="1" AcceptedAnswerId="20194" CreationDate="2015-07-16T12:41:24.150" Score="3" ViewCount="5927" Body="&lt;p&gt;After my first course in the Finite element method I understand that elemental and  global stiffness matrices are symmetric. What is the reason behind symmetry of stiffness matrices? Is it because symmetric property of the bilinear term in weak form?&lt;/p&gt;&#xA;" OwnerUserId="7663" LastActivityDate="2017-09-01T06:27:25.603" Title="FEM: Symmetry of stiffness matrix" Tags="|finite-element|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20193" PostTypeId="2" ParentId="20189" CreationDate="2015-07-16T13:01:03.130" Score="2" Body="&lt;p&gt;My first suggestion would be to take a look at &lt;a href=&quot;http://www.netlib.org/lapack/&quot; rel=&quot;nofollow&quot;&gt;LAPACK&lt;/a&gt; (Linear Algebra Package). It defines a set of interfaces and there are several optimized implementations available for different machines. Personally, I had good experience with &lt;a href=&quot;http://math-atlas.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;ATLAS&lt;/a&gt;, a LAPACK implementation that provides lots of auto tuning. This was some time ago though, there might be better alternatives available now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that on the LAPACK website there are links to other projects, e.g. MAGMA which is similar to LAPACK but for systems with multi-core nodes and GPUs (never tried that one myself, though).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LAPACK itself is based on &lt;a href=&quot;http://www.netlib.org/blas/&quot; rel=&quot;nofollow&quot;&gt;BLAS&lt;/a&gt; (basical linear algebra), which is in itself a set of interfaces for, well, basic linear algebra routines (matrix-vector product, vector-vector product, matrix-matrix product etc). The advantage here again is that BLAS by itself is a set of interfaces and different implementations are available, some of them heavily optimized (e.g. &lt;a href=&quot;https://www.tacc.utexas.edu/research-development/tacc-software/gotoblas2&quot; rel=&quot;nofollow&quot;&gt;GOTO-BLAS&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume one could also directly use BLAS to compute a Schur complement $S = A - B*D^{-1}*C$ using routines for matrix products and inversion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Addendum:&lt;/em&gt; Jack Dongarra has compiled long list of different LA libraries, see &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. The list also shows what features the libraries have (Languages, Dense/Sparse, Real/Complex etc.).&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="16921" LastEditDate="2015-07-17T09:40:07.313" LastActivityDate="2015-07-17T09:40:07.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20194" PostTypeId="2" ParentId="20192" CreationDate="2015-07-16T13:19:16.940" Score="6" Body="&lt;p&gt;If the bilinear form is symmetric (including however you choose to impose boundary conditions), and you choose to use the same kinds of trial and test functions, then the matrix will be symmetric. You can see that by writing out the form of your problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ K_{ij}=b(\phi_i,\phi_j) = b(\phi_j,\phi_i) = K_{ji}$$&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-07-16T13:19:16.940" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20195" PostTypeId="2" ParentId="20192" CreationDate="2015-07-16T13:22:03.863" Score="4" Body="&lt;p&gt;The stiffness matrix is symmetric if the operator $L$ of the PDE is self-adjoint, i.e. if you have $\langle Lf,g\rangle = \langle f,Lg\rangle$ for any pair of functions $(f,g)$ in the suitable function space, where $\langle u,v\rangle$ denotes the inner product between two functions $u,v$, for instance $\int_\Omega uv dx$ ($L_2$ inner product).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stiffness matrix is obtained by projecting the PDE $Lf = g$ onto the finite element function basis $(\Phi_i)$, in the following form: $\forall i, \langle Lf,\Phi_i\rangle = \langle g, \Phi_i\rangle$ (Galerkin formulation), then looking for the solutions in the form $f = \sum \alpha_j \Phi_j$, by injecting the second equation into the first one, you obtain the discretized equation, in the form $K \alpha = B \beta$, where $\alpha$ is the vector of unknowns (coordinates of the solution in the function basis $(\Phi_i)$), $\beta$ is the RHS projected onto the function basis ($\beta_i = \langle g, \Phi_i\rangle$), $B$ is the mass matrix ($b_{i,j} = \langle \Phi_i,\Phi_j\rangle$) and the coefficients of the matrix $k_{i,j} = \langle L \Phi_i, \Phi_j\rangle$. Therefore, if the operator $L$ is self-adjoint, you have $ \langle L \Phi_i, \Phi_j\rangle = \langle L \Phi_j, \Phi_i\rangle$ and the matrix $K$ is symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the case of most commonly encountered PDEs, but there are exceptions when the physics gets more complicated...&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="127" LastEditDate="2017-09-01T06:27:25.603" LastActivityDate="2017-09-01T06:27:25.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20196" PostTypeId="2" ParentId="20190" CreationDate="2015-07-16T14:35:44.783" Score="0" Body="&lt;p&gt;Since (I assume) your matrix size and possibly its structure changes from time step to time step you can't, for example, compute an efficient ordering once and then reuse that with e.g. the chol() function to compute a solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The backslash operator does incur some slight overhead determining if your matrix is dense/sparse, symmetric, etc. If, for example, your matrix is symmetric, you might improve performance slightly by computing an ordering (e.g. symamd()) and then using that with chol().&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand right, you are computing for 1e6 time steps? I don't suppose there is an alternate time integration algorithm you can use that would require fewer time steps?&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-07-16T14:35:44.783" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20197" PostTypeId="1" AcceptedAnswerId="20199" CreationDate="2015-07-16T17:03:27.540" Score="4" ViewCount="10966" Body="&lt;p&gt;I have calculated the first derivative of following equation using Euler method (first order), Three point Finite Difference method (second order) and Four point Finite Difference method (third order). &#xA;$$f(x)=e^{-32*(x-5)^2}$$ in the domain  $0 \leq x \leq 10$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I discretized the domain by 50, 500, 5000 and 50000 number of points and calculated the solution. Then I compared the solution with analytical solution and calculated the maximum absolute difference in analytical and numerical result in that region and plotted the graph shown below (sorry for bad quality of graph).&lt;img src=&quot;https://i.stack.imgur.com/qwCfL.jpg&quot; alt=&quot;enter image description here&quot;&gt;  here y-axis is log(error) and x-axis is log(number of grid points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Higher order methods are supposed to be more accurate than lower order methods. In the graph error in third order method is more than second and first order method for course grid (50 and 500 points) and third order method is more accurate than second and first order method in fine grid ( 5000 and 50000) points). Please explain me why this is happening? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;thank you!&lt;/p&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="16930" LastEditDate="2015-07-17T11:43:16.920" LastActivityDate="2016-08-01T13:02:40.027" Title="Error in result of finite-difference approximation when refining" Tags="|finite-difference|numerical-analysis|convergence|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20198" PostTypeId="1" CreationDate="2015-07-15T20:22:17.240" Score="3" ViewCount="386" Body="&lt;p&gt;I am currently trying to reproduce &lt;a href=&quot;http://core.ac.uk/download/pdf/97962.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the results published by Hands et al.&lt;/a&gt;&lt;sup&gt;1&lt;/sup&gt; using MATLAB. They calculated the bases of the C&lt;sub&gt;60&lt;/sub&gt; wave functions of HOMO, LUMO and LUMO+1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I implemented these functions into MATLAB syntax, e.g.&#xA;&lt;code&gt;&#xA;Tp1uxL5 = @(x,y,z) 35.*z.*(x.^4-6*x.^2.*y.^2+y.^4)-5*x.*(1-14*z.^2+21.*z.^4);&#xA;Tp1uyL5 = @(x,y,z) 5*y.*(28.*z.^4-28*x.*z.*(x.^2-y.^2)-(1-7*z.^2).^2);&#xA;Tp1uzL5 = @(x,y,z) 2*z.*(15-70*z.^2+63*z.^4)+7*x.*(5*(x.^2-y.^2).^2-4*x.^2);&#xA;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;for the pentagon prone $T^p_{1ui} (L=5)$ (i=x,y,z) functions, multiplied by an exponential function $\exp(-k R)$ utilizing $k = \frac{Z_\text{eff}}{2} a_0$ with the effective charge $Z_\text{eff}=3.14$, the Bohr radius $a_0 = 52 \text{ pm}$, and $R = \sqrt{x^2+y^2+z^2}$ as proposed by the authors. &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;I normalized the functions by integrating in three dimensions using MATLAB's &lt;code&gt;integral3&lt;/code&gt; function, i.e. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;A = integral3( FKT , -B,B , -B,B , -B,B )&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the boundary $B$ which i varied between $0.5 \text{ nm} \le B \le 10 \text{ nm}$ with no change in result. I also tried just generating the 3D &quot;wavefunction matrix&quot; in those boundaries and taking the sum over all values.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;I equated each base $T$ times exponential times normalization factor $\frac{1}{\sqrt{A}}$, squared for the propability amplitude, added the corresponding functions for the probability density $\Psi^2$ of HOMO, LUMO and LUMO+1 as well as pentagon, hexagon and double-bond prone configuration.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I generated plots:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A) &quot;Constant height&quot; STM images at $z=z_0$ by equating $\Psi^2 (x,y,z_0)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;B) &quot;Constant current&quot; STM images equating $\Psi^2 (x,y,z) = const.$ and visualizing the $z$ map.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;C) 3D plots of the electron density (comparable to Fig. 4,6,8,11 in &lt;a href=&quot;http://core.ac.uk/download/pdf/97962.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;) by MATLABs &lt;code&gt;isosurface(X,Y,Z,DOS,DOSvalue)&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I fail to reproduce the pictures of &lt;a href=&quot;http://core.ac.uk/download/pdf/97962.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the report&lt;/a&gt;. Below you see a calculated constant current image; constant height and the 3D probability density fronts show the same structure, so the visualization part of the code I guess is fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked and checked the functions again, I am starting to get mad over this. I am not reproducing the results of the paper. What am I doing wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gJAk8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fig. 1.&lt;/strong&gt; Simulated constant current images. First row: LUMO, second row: HOMO, third row: LUMO+1. First column: pentagone prone, second column: hexagone prone, third column double-bond prone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;View image &lt;a href=&quot;http://postimg.org/image/5ty7fu5rj/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt; &quot;Calculation of images of oriented C&lt;sub&gt;60&lt;/sub&gt; molecules using molecular orbital theory&quot; Phys. Rev. B &lt;strong&gt;81&lt;/strong&gt;, 205440 (2010), &lt;a href=&quot;http://core.ac.uk/download/pdf/97962.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://core.ac.uk/download/pdf/97962.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16975" OwnerDisplayName="t0xic" LastEditorUserId="9667" LastEditDate="2015-07-22T20:26:20.723" LastActivityDate="2015-07-22T20:26:20.723" Title="C60 orbital calculation" Tags="|matlab|visualization|quantum-mechanics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20199" PostTypeId="2" ParentId="20197" CreationDate="2015-07-16T17:26:36.827" Score="7" Body="&lt;p&gt;High order methods are not necessarily always more accurate than low order methods; they simply have a more rapid convergence rate.  For example, if you take the Taylor expansion of a function at a point $x$ and evaluate it at $x+h$, you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x+h) = f(x) + f'(x)h + \frac{f''(x)}{2}h^2 + \ldots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The order of a finite difference method is determined by the power of $h$ that is eliminated using finite difference coefficients; for example, rearranging gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left|\frac{f(x+h)-f(x)}{h} - f'(x)\right| = \left| \frac{f''(x)}{2}h + \frac{f'''(x)}{6}h^2 + \ldots\right|$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The left term is error ,and you can make the asymptotic argument that if $h \ll 1$, then $h \gg h^2$, so that for $h$ small enough, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left|\frac{f(x+h)-f(x)}{h} - f'(x)\right| \approx \left| \frac{f''(x)}{2}\right| h.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This truncation error term implies that error decreases linearly with grid size ($h$), or that the error is $O(h)$.  High order methods give that the error is $O(h^r)$ for some order $r$; again, this doesn't imply that high order method errors are smaller, just that the error decreases faster with $h$ than for a low order method.The magnitude of the truncation error depends &lt;strong&gt;specifically on magnitude of $ f'', f'''$ etc. we truncated, and power of $h$.&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem you have considered here has complex behavior in derivatives. If your grid size is not small enough to nullify the effect of magnitude of derivative term  you truncated,  will give high error. Because of high rate of convergence of higher order methods, your 4th order method error decreases with a faster slope than the lower order methods.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="16930" LastEditDate="2015-07-20T04:00:37.970" LastActivityDate="2015-07-20T04:00:37.970" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20200" PostTypeId="1" CreationDate="2015-07-16T17:55:21.227" Score="3" ViewCount="83" Body="&lt;p&gt;Since evaluating a matrix condition number usually takes $O(n^3)$, I wonder whether there is an efficient $O(n^2)$ way to estimate in MATLAB a matrix condition number given its LDL decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2015-07-17T01:36:20.877" Title="Is there an efficient $O(n^2)$ way to estimate in MATLAB a matrix condition number given its LDL decomposition?" Tags="|matlab|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20201" PostTypeId="2" ParentId="20197" CreationDate="2015-07-16T18:28:00.097" Score="4" Body="&lt;p&gt;You've chosen a very specific example where the function in question has a single extremely narrow peak and is almost zero everywhere else in the domain. This is an example of a stiff problem where you must take a very tight discretization to resolve the peak. Finite difference approximations are based on polynomial approximations to a curve. In the case where the curve is not locally well-approximated by a polynomial (such as the several grid points that straddle the peak in the Gaussian function here) this can lead to oscillations in the approximating polynomial (see &lt;a href=&quot;https://scicomp.stackexchange.com/a/11158/7733&quot;&gt;this answer&lt;/a&gt;) resulting in a poor estimate of the derivative. Refining the grid eventually leads to a situation where the function is well approximated by a polynomial on the scale of the finite difference stencil and accuracy improves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As discussed in Jesse Chan's answer, low order methods may actually have lower magnitude errors in some cases. However, I would be skeptical of the accuracy of either method in that case and suggest that you simply need higher resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to replicate your results, but I found that higher order finite differences were actually more accurate (based on RMS error) even in this example. Were you using one sided differences? Here's my matlab script if you want to mess with it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;&#xA;y = @(x) exp(-32*(x-5).^2);&#xA;dydx = @(x) -(64*(x-5)).*y(x);&#xA;&#xA;x=linspace(0,10,50);&#xA;&#xA;%first order (one sided) difference&#xA;x1 = x(1:end-1);&#xA;dy1 = (y(x(2:end))-y(x(1:end-1)))/diff(x(1:2));&#xA;&#xA;%second order centered difference&#xA;x2 = x(2:end-1);&#xA;dy2 = (y(x(3:end))-y(x(1:end-2)))/(2*diff(x(1:2)));&#xA;&#xA;%fourth order centered difference&#xA;x3 = x(3:end-2);&#xA;dy3 = (-y(x(5:end))+8*y(x(4:end-1))-8*y(x(2:end-3))+y(x(1:end-4)))/(12*diff(x(1:2)));&#xA;&#xA;xRef = linspace(0,10,1e5);&#xA;dyRef = dydx(xRef);&#xA;&#xA;figure(1)&#xA;clf&#xA;hold on&#xA;plot( xRef, dyRef, 'k' )&#xA;plot( x1, dy1,'o')&#xA;plot( x2, dy2,'x' )&#xA;plot( x3, dy3,'^' )&#xA;legend('Analytical','First order','Second order','Fourth order')&#xA;title('Computed derivative values')&#xA;&#xA;figure(2)&#xA;clf&#xA;hold on&#xA;plot( x1, abs(dy1-dydx(x1)),'o')&#xA;plot( x2, abs(dy2-dydx(x2)),'x' )&#xA;plot( x2, abs(dy2-dydx(x2)),'^' )&#xA;legend('First order','Second order','Fourth order')&#xA;title('Error in the derivative computation')&#xA;&#xA;% rms errors:&#xA;sqrt( sum((dy1-dydx(x1)).^2 ) / numel(x1) ) %first order&#xA;sqrt( sum((dy2-dydx(x2)).^2 ) / numel(x2) ) %second order&#xA;sqrt( sum((dy3-dydx(x3)).^2 ) / numel(x3) ) %fourth order&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please note that this is a slightly different situation than that in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/20172/why-are-runge-kutta-and-eulers-method-are-so-different&quot;&gt;the related question&lt;/a&gt;. That question was about ODE solvers (Euler and RK). The reason I edited your answer there was to highlight the fact that this case is not typical and that in almost all  cases where RK4 is stable it will outperform Euler for a given step size. The ODE analogue to your example here is a stiff ODE where you are not in the stability region of explicit solvers. As I said there, if your ODE solver is unstable all bets are off as far as accuracy (just as all the finite difference methods give poor results on a coarse grid here). Of course a stable low-order implicit method will typically have less error than an unstable explicit method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though you may have &quot;less error&quot; for course grids with the lower order finite differences, the actual errors in your example are huge. In that case, neither solution should be used and you have no choice but to refine the grid or use a different method entirely.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-08-01T13:02:40.027" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="20202" PostTypeId="1" AcceptedAnswerId="20207" CreationDate="2015-07-16T20:37:10.170" Score="4" ViewCount="290" Body="&lt;p&gt;I am trying to solve a large scale inverse problem using the Bayesian formulation. To estimate the &lt;a href=&quot;https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation&quot; rel=&quot;noreferrer&quot;&gt;Maximum a Posteriori Estimation (MAP)&lt;/a&gt; solution I will have to minimize the following objective function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F(m) = F_d + F_\text{prior} = \frac{(f(m) - d)^T(f(m) - d)}{\sigma^2} + \frac{(m - m_\text{prior})^T(m - m_\text{prior})}{\alpha^2} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where d is the observed data, and $\sigma$ is the uncertainty in $d$. $m$ is the optimization parameter, $\alpha$ represents the confidence in prior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the current problem setup, the number of data points, $d$, are $O(n)$ whereas the number of parameters, $m$, are $O(n^2)$. Everything else is dimensionless, therefore order of magnitude of f(m) is same as m. This results to an objective function which is inherently biased towards the prior term($F_\text{prior}$), unless $\sigma &amp;lt;&amp;lt; \alpha$, further resulting in poor convergence of $F_d$ in the optimization process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In such a case can I scale $F_d$ and $F_\text{prior}$ by the number of terms they contain? From what I understand such scaling will change the interpretation of $\sigma$ and $\alpha$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I get better &quot;match&quot; between $f(m)$ and $d$ without the prior term. But I need to include the prior in order to get the bounds on the posterior solution.&lt;/p&gt;&#xA;" OwnerUserId="3640" LastEditorUserId="9667" LastEditDate="2015-07-16T21:32:03.667" LastActivityDate="2015-07-17T22:53:00.157" Title="Objective function scaling in an Inverse Problem" Tags="|optimization|inverse-problem|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20203" PostTypeId="2" ParentId="20202" CreationDate="2015-07-16T22:03:47.743" Score="4" Body="&lt;p&gt;Changing the relative weight of the two terms is equivalent to changing your prior.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fundamental issue here is that you've got a problem with far more parameters to estimate than data points.  You will need to use a fairly strong prior to get tight bounds on the fitted parameters, and the choice of that prior will have a strong influence on your solution.  .&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-07-16T22:03:47.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20204" PostTypeId="2" ParentId="20200" CreationDate="2015-07-17T01:36:20.877" Score="1" Body="&lt;p&gt;There should be. Given a factorized matrix (usually LU), there are iterative methods to estimate the norm of the matrix and its inverse, and thus estimate the condition number in $\mathcal{O}(n^{2})$ time. Since $DL^{T}$ is upper triangular, you could alter methods that usually use an iterative method with an LU factorization by supplying $D$ and $L^{T}$ separately (this might require some hacking of source code if the routine isn't available in, for instance, BLAS). Basically, since $U$ is only going to be used in matrix-vector products, you would instead take matrix-vector products of $L^{T}$ and $D$, which would still be $\mathcal{O}(n^{2})$. &lt;a href=&quot;http://www.alglib.net/matrixops/rcond.php&quot; rel=&quot;nofollow&quot;&gt;ALGLIB&lt;/a&gt; provides some helpful documentation to get you started.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-07-17T01:36:20.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20205" PostTypeId="2" ParentId="19900" CreationDate="2015-07-17T02:38:18.427" Score="0" Body="&lt;p&gt;There's a large literature on &quot;trans-dimensional&quot; Markov Chain Monte Carlo methods in geophysics where the geophysical model consists of a number of layers and the inversion process adjusts the number of layers in addition to the properties of the layers.  This might be very relevant to your problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's not clear that your problem is an inverse problem at all- if you're deciding where you should perform a series of injections, that's an optimization problem rather than a parameter estimation problem, even if the optimization has to be done in a stochastic sense relative to some random parameters in the system.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-07-17T02:38:18.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20206" PostTypeId="2" ParentId="14095" CreationDate="2015-07-17T13:51:15.313" Score="4" Body="&lt;p&gt;You can use a 2D anisotropic mesh generator (see e.g. H. Borouchaki, George, P. L. , F. Hecht , P. Laug and E. Saltel, Delaunay mesh generation governed by metric specifications. Part 1: Algorithms, Finite Elements in Analysis and Design, Vol. 25, pp. 61-83, 1997).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a 2D anisotropic mesh generator takes as input:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a 2D domain to be meshed&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a metric $G(x,y)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The metric is a function that associates a $2\times 2$&#xA;symmetric positive definite matrix with each point of the domain. This metric specifies the desired edge length as follows: suppose you have an edge $(x_1, y_1) - (x_2, y_2)$, then what you want is $e^t G(x,y) e = 1$, where e = $(x_2, y_2) - (x_1, y_1)$ and where $x=(x_1 + x_2)/2, y = (y_1 + y_2)/2)$. Intuitively, the squared norm of $e = \| e \|^2 = e^t e$ is replaced with $e^t G e$. This provides a means of &quot;shearing&quot; and &quot;scaling&quot; the definition of distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the metric can be stored at the vertices of an input background mesh and linearly interpolated over the triangles of the background mesh. Then the anisotropic mesh generator will create a new triangulation of the input geometry where all the edges are (mostly) of length 1 &lt;em&gt;with respect to the metric&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you can see your 4D input domain as a 2D domain transformed into 4D with a (vector-valued) function $\Phi(x,y)$. Now if you generate a 2D mesh deformed in such a way that it will exactly counterbalance the deformation created by the mapping $\Phi$, then you will have mostly equilateral triangles in 4D. This can be achieved by using for $G(x,y)$ the pullback metric of $\Phi$, given by $G(x,y) = J^t J$, where $J$ denotes the Jacobian matrix of $\Phi$. This technique is used for tesselating parametric surfaces (Splines, Nurbs ...) using a 2D anisotropic mesh generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a mapping $\Phi$ and its Jacobian cannot be computed, there is an alternative method that we used (in 6D in our case), that directly optimizes and computes the mesh in high dimensional space:&#xA;&lt;a href=&quot;http://www.imr.sandia.gov/papers/abstracts/Le621.html&quot; rel=&quot;nofollow&quot;&gt;http://www.imr.sandia.gov/papers/abstracts/Le621.html&lt;/a&gt;&#xA;(but it is much more costly, the anisotropic meshing steered by $J^tJ$ is probably much better in most cases).&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2015-07-17T13:51:15.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20207" PostTypeId="2" ParentId="20202" CreationDate="2015-07-17T13:56:11.873" Score="5" Body="&lt;p&gt;First, a disclaimer: I'll answer specifically within the context of Bayesian inverse problems, not the wider statistical theory of Bayesian inference (which tends to devolve into philosophy at some point...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, a general point: If you are only computing a MAP estimate and are not trying to extract higher order moments from the posterior distribution, the only (meaningful) difference between Bayesian and classical inverse problems is in the modeling, i.e., the choice of discrepancy and regularization terms (including their scaling). To put it bluntly: If you're computing a MAP estimate and you're not doing Bayesian modeling (i.e., based on objective statistical considerations), all you're doing is exchanging the labels &quot;discrepancy term&quot; and &quot;regularization term&quot; with &quot;likelihood&quot; and &quot;prior&quot;, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you didn't give any details on where your objective comes from, I see three possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your modeling &lt;em&gt;is&lt;/em&gt; based on proper statistical considerations, i.e., you &lt;em&gt;know&lt;/em&gt; (from somewhere) that the data $d_i$ are independently and identically normally distributed with mean $f(m)_i$ and variance $\sigma^2$, and in the absence of any further information, you'd expect the $m_j$ to also be independently and identically normally distributed with mean $m_{\text{prior},j}$ and variance $\alpha^2$. In this case the scaling you are currently observing and are uncertain about is in fact the &lt;em&gt;correct&lt;/em&gt; scaling: Bayesian theory tells you that in this case, you need to give a very strong weight to the prior. (Similar to what Brian Borchers was telling you.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your modeling is based on &lt;em&gt;improper&lt;/em&gt; statistical considerations. The most common instance is when your problem comes from a discretization of a parameter identification problem in a partial differential equation. In this case, your model tells you, e.g., that the unknown parameter &lt;em&gt;function&lt;/em&gt; is normally distributed with variance $\alpha^2$, but that doesn't necessarily mean that the entries of the discrete parameter &lt;em&gt;vector&lt;/em&gt; are. If you consider the vector $m$ as the unknown expansion coefficients of a given basis $\{\phi_i\}$ of piecewise linear functions (as in a standard finite element discretization) and the corresponding discrete parameter &lt;em&gt;function&lt;/em&gt; $m_h := \sum_i m_i\phi_i$ as normally distributed with variance $\alpha^2$, your likelihood term would in fact be $\alpha^{-2}(m_h-m_p)^T M_h(m_h-m_p)$, where $M_h$ is the usual mass matrix. (And similarly for the likelihood.) Now, the mass matrices already account for the different scalings of both terms due to the discretization, and you can use the same parameters as given by your model. &#xA;You can find discussions on the discretization of Bayesian inverse problems in &lt;a href=&quot;http://www.siltanen-research.net/LassasSaksmanSiltanen.pdf&quot;&gt;http://www.siltanen-research.net/LassasSaksmanSiltanen.pdf&lt;/a&gt; or the works of &lt;a href=&quot;http://cims.nyu.edu/~stadler/publications/index.html&quot;&gt;Georg Stadler&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your choice of likelihood and prior (including $\sigma$ and $\alpha$) is in fact &lt;em&gt;not&lt;/em&gt; based on firm a priori knowledge, but chosen ad hoc. (Not that there's anything wrong with that -- people in inverse problems have been doing this for years without being ashamed of it.) Then all that's left of the Bayesian framework is the interpretation of the terms, but since there is no external objective information to relate this to, it is (from the mathematical point of view) meaningless. But this means that you are free to scale the parameters (or rather, the ratio $\sigma/\alpha$, since that is all that counts here) to your liking, without anybody being able to tell you you're wrong. Still, it is always good practice make sure your discrepancy and regularization terms are discretization-independent, so you can change the discretization without having to fiddle with the parameters again. You can then also treat the parameter as an additional unknown variable and include it in the Bayesian formulation with the help of a so-called &lt;em&gt;hyper-prior&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Assuming it's either 2. or 3., the answer is therefore &lt;strong&gt;Yes, you can, and you should.&lt;/strong&gt; (This will also &lt;em&gt;help&lt;/em&gt; with the interpretation, because otherwise you mix confidence/noise covariance with a purely numerical scaling.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-07-17T17:58:58.193" LastActivityDate="2015-07-17T17:58:58.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20208" PostTypeId="2" ParentId="14134" CreationDate="2015-07-17T14:20:56.653" Score="2" Body="&lt;p&gt;To do that, I am using two different sparse matrix data structures. One is dynamic and new coefficients can be inserted into it during assembly. Once assembly is finished, I convert it into a CRS matrix. It is not as optimal as precomputing the sparsity pattern with two passes, but easier to use, and works reasonably well (I used it for problems of dimension 20 million with hundreds of millions non-zero entries).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementation is available in my OpenNL and Geogram libraries:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://alice.loria.fr/index.php/software/4-library/23-opennl.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/index.php/software/4-library/23-opennl.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dynamic matrix structure is an array of SparseRow objects. The idea is to have objects that have both a size (number of used coefficients) and a capacity (number of allocated coefficients), and to double capacity each time more space is needed (to avoid reallocating too often). Same strategy is used in the std::vector C++ class:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/**&#xA; * \brief Represents a coefficient in a sparse matrix&#xA; * \relates NLSparseMatrix&#xA; */&#xA;typedef struct  {&#xA;    /**&#xA;     * \brief index of the coefficient.&#xA;     */    &#xA;    NLuint index ;&#xA;&#xA;    /**&#xA;     * \brief value of the coefficient. &#xA;     */    &#xA;    NLdouble value ; &#xA;} NLCoeff ;&#xA;&#xA;/**&#xA; * \brief Represents a row or a column of a sparse matrix&#xA; * \relates NLSparseMatrix&#xA; */&#xA;typedef struct {&#xA;    /**&#xA;     * \brief number of coefficients. &#xA;     */    &#xA;    NLuint size ;&#xA;&#xA;    /** &#xA;     * \brief number of coefficients that can be &#xA;     * stored without reallocating memory.&#xA;     */    &#xA;    NLuint capacity ;&#xA;&#xA;    /**&#xA;     * \brief the array of coefficients, with enough&#xA;     * space to store capacity coefficients.&#xA;     */&#xA;    NLCoeff* coeff ;  &#xA;} NLRow ;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are functions to construct a row, to add a new coefficient to it and to delete it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void nlRowConstruct(NLRow* c) {&#xA;    c-&amp;gt;size     = 0 ;&#xA;    c-&amp;gt;capacity = 0 ;&#xA;    c-&amp;gt;coeff    = NULL ;&#xA;}&#xA;&#xA;void nlRowDestroy(NLRow* c) {&#xA;    free(c-&amp;gt;coeff) ;&#xA;}&#xA;&#xA;void nlRowGrow(NLRow* c) {&#xA;    if(c-&amp;gt;capacity != 0) {&#xA;        c-&amp;gt;capacity = 2 * c-&amp;gt;capacity ;&#xA;        c-&amp;gt;coeff = (NLCoeff*)realloc(c-&amp;gt;coeff, c-&amp;gt;capacity*sizeof(NLCoeff)) ;&#xA;    } else {&#xA;        c-&amp;gt;capacity = 4 ;&#xA;        c-&amp;gt;coeff = (NLCoeff*)malloc(c-&amp;gt;capacity*sizeof(NLCoeff)) ;&#xA;    }&#xA;}&#xA;&#xA;void nlRowAdd(NLRow* c, NLuint index, NLdouble value) {&#xA;    NLuint i ;&#xA;    for(i=0; i&amp;lt;c-&amp;gt;size; i++) {&#xA;        if(c-&amp;gt;coeff[i].index == index) {&#xA;            c-&amp;gt;coeff[i].value += value ;&#xA;            return ;&#xA;        }&#xA;    }&#xA;    if(c-&amp;gt;size == c-&amp;gt;capacity) {&#xA;        nlRowColumnGrow(c) ;&#xA;    }&#xA;    c-&amp;gt;coeff[c-&amp;gt;size].index = index ;&#xA;    c-&amp;gt;coeff[c-&amp;gt;size].value = value ;&#xA;    c-&amp;gt;size++ ;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A SparseMatrix is like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;typedef struct {&#xA;   /**&#xA;     * \brief number of rows &#xA;     */    &#xA;    NLuint m ;&#xA;&#xA;    /**&#xA;     * \brief number of columns &#xA;     */    &#xA;    NLuint n ;&#xA;&#xA;    /**&#xA;      * \brief the rows, array of size m&#xA;      */&#xA;        NLRow* rows;&#xA;    } SparseMatrix;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Inserting a coefficient in the matrix is done as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void nlSparseMatrixAdd(NLSparseMatrix* M, NLuint i, NLuint j, NLdouble value) {&#xA;    nl_parano_range_assert(i, 0, M-&amp;gt;m - 1) ;&#xA;    nl_parano_range_assert(j, 0, M-&amp;gt;n - 1) ; &#xA;    nlRowColumnAdd(&amp;amp;(M-&amp;gt;row[i]), j, value) ;  &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2015-07-17T14:20:56.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20209" PostTypeId="1" CreationDate="2015-07-17T14:52:21.687" Score="4" ViewCount="132" Body="&lt;p&gt;Can you advise me any kind of existing software that can help to solve the &lt;strong&gt;disjunctive programming problem&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have unit 3D planes $\Pi_{1}, \ldots, \Pi_{N}$ (they are known). Let us write them as follows: $U_{j} x = H_{j}, j = 1, \ldots, N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having the fixed numbers $n$ and $m$, we want to find 3D planes $\pi_{1}, \ldots, \pi_{n}$ and 3D points $x_{1}, \ldots, x_{m}$ such that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Planes and points satisfy the incidence structure:&#xA;$x_{i} \in \pi_{j}\;\;\forall(i, j) \in I$, where $I$ is fixed known set $I \subseteq \{1, \ldots, m\} \times \{1, \ldots, n\}$.&lt;/li&gt;&#xA;&lt;li&gt;The polyhedron bounded by planes $\pi_{1}, \ldots, \pi_{n}$ is convex.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;and minimize the function $\sum \limits_{j = 1}^{N}(H_{j} - \max \limits_{i = 1, \ldots, m} (x_{i}, U_{j}))^{2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we have a $\max$ in the functional, the above problem can be formulated as a disjunctive programming program as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;\text{minimize} \sum \limits_{j = 1}^{N}(H_{j} - L_{j})^{2}\\&#xA;&amp;amp;\text{subject to } \, (1-2) \text{ and }&#xA;\forall j = 1, \ldots, N\; \exists i \text{ such that } (x_{i}, U_{j}) = H_{j} \enspace .&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read from Internet that such problems are called disjunctive programming. Can you give any hint which software can be used to solve it?&lt;/p&gt;&#xA;" OwnerUserId="7716" LastEditorUserId="20688" LastEditDate="2022-01-01T00:24:17.720" LastActivityDate="2022-01-01T19:41:57.053" Title="Disjunctive programming software" Tags="|quadratic-programming|software-recommendation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20210" PostTypeId="1" CreationDate="2015-07-17T20:20:27.997" Score="1" ViewCount="37" Body="&lt;p&gt;When we compute the complexity of an image processing algorithm, we get an $O(N)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;does the $N$ refers to the number of pixels in the image &lt;strong&gt;or&lt;/strong&gt; to the height/width of the image, I mean it is computed related to $O(nm)$ with $n\times m$ the size of the image ?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;please, cite some references.&lt;/p&gt;&#xA;" OwnerDisplayName="user16973" LastEditorUserId="27" LastEditDate="2015-07-17T20:27:36.873" LastActivityDate="2015-07-17T22:49:37.990" Title="When we compute the complexity of a given algorithm related to image processing does the N refers to the number of Pixels in the image?" Tags="|algorithms|complexity|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20211" PostTypeId="1" CreationDate="2015-07-17T22:10:48.183" Score="1" ViewCount="516" Body="&lt;p&gt;I'm trying to install Pastix library, but I'm still having problems with missing libraries and I'm really helpless now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started with copying &lt;code&gt;LINUX-GNU.in&lt;/code&gt; as &lt;code&gt;config.in&lt;/code&gt;, because I'm working on Lubuntu 15.04. I've changed paths to Scotch and Blas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now, when I try to &lt;code&gt;make examples_s&lt;/code&gt;, it appears ok, but no compiled files appear in the &lt;code&gt;bin&lt;/code&gt; folder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try to make any other example, e.g. &lt;code&gt;make examples_z&lt;/code&gt;, eventually I'm getting the error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/bin/ld: cannot find -lblas&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that Blas library couldn't be found, but I don't see the mistake I made. Blas libraries are located in my &lt;code&gt;/usr/lib&lt;/code&gt; folder like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;├── libblas&#xA;|     ├── libblas.so.3 -&amp;gt; libblas.so.3.0&#xA;|     └── libblas.so.3.0&#xA;├── libblas.so.3 -&amp;gt; /etc/alternatives/libblas.so.3&#xA;├── libblas.so.3gf -&amp;gt; /etc/alternatives/libblas.so.3gf&#xA;├── libgslcblas.so.0 -&amp;gt; libgslcblas.so.0.0.0&#xA;└── libgslcblas.so.0.0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;config.in&lt;/code&gt; I've set &lt;code&gt;BLAS_HOME&lt;/code&gt; variable like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BLAS_HOME=/usr/lib/libblas&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, do you know, what I'm doing wrong? I have no experience with Pastix and so I'm very confused right now.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Solution&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I've finally found out, that Pastix needs specifically &lt;code&gt;libblas.a&lt;/code&gt; library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the solution is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find the package which contains &lt;code&gt;libblas.a&lt;/code&gt; - &lt;code&gt;apt-file search libblas.a&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Choose one of the packages - e.g. &lt;code&gt;libblas-dev&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Install the chosen package - &lt;code&gt;sudo apt-get install libblas-dev&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;config.in&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;HOSTARCH    = i686_pc_linux&#xA;VERSIONBIT  = _32bit&#xA;EXEEXT      =&#xA;OBJEXT      = .o&#xA;LIBEXT      = .a&#xA;CCPROG      = gcc -Wall&#xA;CFPROG      = gfortran&#xA;CF90PROG    = gfortran -ffree-form&#xA;MCFPROG     = mpif90&#xA;CF90CCPOPT  = -ffree-form -x f95-cpp-input&#xA;# Compilation options for optimization (make expor)&#xA;CCFOPT      = -O3&#xA;# Compilation options for debug (make | make debug)&#xA;CCFDEB      = -g3&#xA;CXXOPT      = -O3&#xA;NVCCOPT     = -O3&#xA;&#xA;LKFOPT      =&#xA;MKPROG      = make&#xA;MPCCPROG    = mpicc -Wall&#xA;MPCXXPROG   = mpic++ -Wall&#xA;CPP         = cpp&#xA;ARFLAGS     = ruv&#xA;ARPROG      = ar&#xA;EXTRALIB    = -lgfortran -lm -lrt&#xA;CTAGSPROG   = ctags&#xA;&#xA;VERSIONMPI  = _mpi&#xA;VERSIONSMP  = _smp&#xA;VERSIONSCH  = _static&#xA;VERSIONINT  = _int&#xA;VERSIONPRC  = _simple&#xA;VERSIONFLT  = _real&#xA;VERSIONORD  = _scotch&#xA;&#xA;###################################################################&#xA;#                  SETTING INSTALL DIRECTORIES                    #&#xA;###################################################################&#xA;# ROOT          = /home/osboxes/pastix&#xA;# INCLUDEDIR    = ${ROOT}/include&#xA;    # LIBDIR        = ${ROOT}/lib&#xA;# BINDIR        = ${ROOT}/bin&#xA;    # PYTHON_PREFIX = ${ROOT}&#xA;&#xA;###################################################################&#xA;#                  SHARED LIBRARY GENERATION                      #&#xA;###################################################################&#xA;#SHARED=1&#xA;#SOEXT=.so&#xA;#SHARED_FLAGS =  -shared -Wl,-soname,__SO_NAME__&#xA;#CCFDEB       := ${CCFDEB} -fPIC&#xA;    #CCFOPT       := ${CCFOPT} -fPIC&#xA;#CFPROG       := ${CFPROG} -fPIC&#xA;&#xA;###################################################################&#xA;#                          INTEGER TYPE                           #&#xA;###################################################################&#xA;# Uncomment the following lines for integer type support (Only 1)&#xA;&#xA;#VERSIONINT  = _long&#xA;#CCTYPES     = -DFORCE_LONG -DINTSIZELONG&#xA;#---------------------------&#xA;#VERSIONINT  = _int32&#xA;#CCTYPES     = -DINTSIZE32&#xA;#---------------------------&#xA;#VERSIONINT  = _int64&#xA;#CCTYPES     = -DINTSSIZE64&#xA;&#xA;###################################################################&#xA;#                           FLOAT TYPE                            #&#xA;###################################################################&#xA;CCTYPESFLT  =&#xA;# Uncomment the following lines for double precision support&#xA;VERSIONPRC  = _double&#xA;CCTYPESFLT := $(CCTYPESFLT) -DPREC_DOUBLE&#xA;&#xA;# Uncomment the following lines for float=complex support&#xA;#VERSIONFLT  = _complex&#xA;#CCTYPESFLT := $(CCTYPESFLT) -DTYPE_COMPLEX&#xA;&#xA;&#xA;###################################################################&#xA;#                          MPI/THREADS                            #&#xA;###################################################################&#xA;&#xA;# Uncomment the following lines for sequential (NOMPI) version&#xA;#VERSIONMPI  = _nompi&#xA;#CCTYPES    := $(CCTYPES) -DFORCE_NOMPI&#xA;    #MPCCPROG    = $(CCPROG)&#xA;#MCFPROG     = $(CFPROG)&#xA;&#xA;# Uncomment the following lines for non-threaded (NOSMP) version&#xA;#VERSIONSMP  = _nosmp&#xA;#CCTYPES    := $(CCTYPES) -DFORCE_NOSMP&#xA;&#xA;# Uncomment the following line to enable a progression thread,&#xA;#  then use IPARM_THREAD_COMM_MODE&#xA;#CCPASTIX   := $(CCPASTIX) -DPASTIX_THREAD_COMM&#xA;&#xA;# Uncomment the following line if your MPI doesn't support MPI_THREAD_MULTIPLE level,&#xA;#  then use IPARM_THREAD_COMM_MODE&#xA;#CCPASTIX   := $(CCPASTIX) -DPASTIX_FUNNELED&#xA;&#xA;# Uncomment the following line if your MPI doesn't support MPI_Datatype correctly&#xA;#CCPASTIX   := $(CCPASTIX) -DNO_MPI_TYPE&#xA;&#xA;# Uncomment the following line if you want to use semaphore barrier&#xA;# instead of MPI barrier (with IPARM_AUTOSPLIT_COMM)&#xA;#CCPASTIX    := $(CCPASTIX) -DWITH_SEM_BARRIER&#xA;&#xA;# Uncomment the following lines to enable StarPU.&#xA;#CCPASTIX   := $(CCPASTIX) `pkg-config libstarpu --cflags` -DWITH_STARPU&#xA;    #EXTRALIB   := $(EXTRALIB) `pkg-config libstarpu --libs`&#xA;# Uncomment the correct 2 lines&#xA;#CCPASTIX   := $(CCPASTIX) -DCUDA_SM_VERSION=11&#xA;    #NVCCOPT    := $(NVCCOPT) -maxrregcount 32 -arch sm_11&#xA;#CCPASTIX   := $(CCPASTIX) -DCUDA_SM_VERSION=13&#xA;    #NVCCOPT    := $(NVCCOPT) -maxrregcount 32 -arch sm_13&#xA;CCPASTIX   := $(CCPASTIX) -DCUDA_SM_VERSION=20&#xA;    NVCCOPT    := $(NVCCOPT) -arch sm_20&#xA;&#xA;# Uncomment the following line to enable StarPU profiling&#xA;# ( IPARM_VERBOSE &amp;gt; API_VERBOSE_NO ).&#xA;#CCPASTIX   := $(CCPASTIX) -DSTARPU_PROFILING&#xA;&#xA;# Uncomment the following line to disable CUDA (StarPU)&#xA;#CCPASTIX   := $(CCPASTIX) -DFORCE_NO_CUDA&#xA;&#xA;###################################################################&#xA;#                          Options                                #&#xA;###################################################################&#xA;&#xA;# Show memory usage statistics&#xA;#CCPASTIX   := $(CCPASTIX) -DMEMORY_USAGE&#xA;&#xA;# Show memory usage statistics in solver&#xA;#CCPASTIX   := $(CCPASTIX) -DSTATS_SOPALIN&#xA;&#xA;# Uncomment following line for dynamic thread scheduling support&#xA;#CCPASTIX   := $(CCPASTIX) -DPASTIX_DYNSCHED&#xA;&#xA;# Uncomment the following lines for Out-of-core&#xA;#CCPASTIX   := $(CCPASTIX) -DOOC -DOOC_NOCOEFINIT -DOOC_DETECT_DEADLOCKS&#xA;&#xA;###################################################################&#xA;#                      GRAPH PARTITIONING                         #&#xA;###################################################################&#xA;&#xA;# Uncomment the following lines for using metis ordering&#xA;#VERSIONORD  = _metis&#xA;#METIS_HOME  = ${HOME}/metis-4.0&#xA;    #CCPASTIX   := $(CCPASTIX) -DMETIS -I$(METIS_HOME)/Lib&#xA;    #EXTRALIB   := $(EXTRALIB) -L$(METIS_HOME) -lmetis&#xA;&#xA;# Scotch always needed to compile&#xA;#SCOTCH_HOME ?= ${HOME}/scotch_5.1/&#xA;    SCOTCH_INC ?= /usr/include/scotch-int64  #$(SCOTCH_HOME)/include&#xA;SCOTCH_LIB ?= /usr/lib  #$(SCOTCH_HOME)/lib&#xA;    # Uncomment on of this blocks&#xA;    #scotch&#xA;    #CCPASTIX   := $(CCPASTIX) -I$(SCOTCH_INC) -DWITH_SCOTCH&#xA;    #EXTRALIB   := $(EXTRALIB) -L$(SCOTCH_LIB) -lscotch -lscotcherrexit&#xA;    #ptscotch&#xA;    CCPASTIX   := $(CCPASTIX) -I$(SCOTCH_INC) -DDISTRIBUTED -DWITH_SCOTCH&#xA;    #if scotch &amp;gt;= 6.0&#xA;    EXTRALIB   := $(EXTRALIB) -L$(SCOTCH_LIB) -lptscotch -lscotch -lptscotcherrexit&#xA;    #else&#xA;    #EXTRALIB   := $(EXTRALIB) -L$(SCOTCH_LIB) -lptscotch -lptscotcherrexit&#xA;&#xA;###################################################################&#xA;#                Portable Hardware Locality                       #&#xA;###################################################################&#xA;# By default PaStiX uses hwloc to bind threads,&#xA;# comment this lines if you don't want it (not recommended)&#xA;HWLOC_HOME ?= /opt/hwloc/&#xA;HWLOC_INC  ?= $(HWLOC_HOME)/include&#xA;    HWLOC_LIB  ?= $(HWLOC_HOME)/lib&#xA;CCPASTIX   := $(CCPASTIX) -I$(HWLOC_INC) -DWITH_HWLOC&#xA;EXTRALIB   := $(EXTRALIB) -L$(HWLOC_LIB) -lhwloc&#xA;&#xA;###################################################################&#xA;#                             MARCEL                              #&#xA;###################################################################&#xA;&#xA;# Uncomment following lines for marcel thread support&#xA;#VERSIONSMP := $(VERSIONSMP)_marcel&#xA;    #CCPASTIX   := $(CCPASTIX) `pm2-config --cflags` -I${PM2_ROOT}/marcel/include/pthread&#xA;    #EXTRALIB   := $(EXTRALIB) `pm2-config --libs`&#xA;# ---- Thread Posix ------&#xA;EXTRALIB   := $(EXTRALIB) -lpthread&#xA;&#xA;# Uncomment following line for bubblesched framework support (need marcel support)&#xA;#VERSIONSCH  = _dyn&#xA;#CCPASTIX   := $(CCPASTIX) -DPASTIX_BUBBLESCHED&#xA;&#xA;###################################################################&#xA;#                              BLAS                               #&#xA;###################################################################&#xA;&#xA;# Choose Blas library (Only 1)&#xA;# Do not forget to set BLAS_HOME if it is not in your environnement&#xA;BLAS_HOME=/usr/lib/libblas&#xA;#----  Blas    ----&#xA;BLASLIB  = -lblas&#xA;#---- Gotoblas ----&#xA;#BLASLIB  = -L${BLAS_HOME} -lgoto&#xA;    #----  MKL     ----&#xA;    #Uncomment the correct line&#xA;    #BLASLIB  = -L$(BLAS_HOME) -lmkl_intel_lp64 -lmkl_sequential -lmkl_core&#xA;#BLASLIB  = -L$(BLAS_HOME) -lmkl_intel -lmkl_sequential -lmkl_core&#xA;    #----  Acml    ----&#xA;    #BLASLIB  = -L$(BLAS_HOME) -lacml&#xA;&#xA;###################################################################&#xA;#                             MURGE                               #&#xA;###################################################################&#xA;# Uncomment if you need MURGE interface to be thread safe&#xA;# CCPASTIX   := $(CCPASTIX) -DMURGE_THREADSAFE&#xA;    # Uncomment this to have more timings inside MURGE&#xA;    # CCPASTIX   := $(CCPASTIX) -DMURGE_TIME&#xA;&#xA;###################################################################&#xA;#                          DO NOT TOUCH                           #&#xA;###################################################################&#xA;&#xA;FOPT      := $(CCFOPT)&#xA;    FDEB      := $(CCFDEB)&#xA;CCHEAD    := $(CCPROG) $(CCTYPES) $(CCFOPT)&#xA;    CCFOPT    := $(CCFOPT) $(CCTYPES) $(CCPASTIX)&#xA;CCFDEB    := $(CCFDEB) $(CCTYPES) $(CCPASTIX)&#xA;    NVCCOPT   := $(NVCCOPT) $(CCTYPES) $(CCPASTIX)&#xA;&#xA;###################################################################&#xA;#                        MURGE COMPATIBILITY                      #&#xA;###################################################################&#xA;&#xA;MAKE     = $(MKPROG)&#xA;    CC       = $(MPCCPROG)&#xA;CFLAGS   = $(CCFOPT) $(CCTYPESFLT)&#xA;FC       = $(MCFPROG)&#xA;    FFLAGS   = $(CCFOPT)&#xA;LDFLAGS  = $(EXTRALIB) $(BLASLIB)&#xA;CTAGS    = $(CTAGSPROG)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="13211" LastEditorUserId="20688" LastEditDate="2019-06-22T20:29:20.067" LastActivityDate="2019-06-22T20:29:20.067" Title="Pastix installation" Tags="|libraries|" AnswerCount="0" CommentCount="4" ClosedDate="2015-07-19T17:59:07.010" ContentLicense="CC BY-SA 3.0" />
  <row Id="20212" PostTypeId="1" CreationDate="2015-07-17T22:27:55.283" Score="5" ViewCount="105" Body="&lt;p&gt;I am interested in approximating the time evolution of 2D curves. Here's an illustration:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/YvwMd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An issue that arises when naively making this approximation as illustrated above, is that as one increases the number of &quot;line segments&quot; used to approximate the curve, then one is also introducing extra &quot;flexibility&quot;. That is, curves with less approximating line segments are stiffer than curves with many approximating line segments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in knowing what the theory that handles analytical solutions of simple continuous cases is called. I realize it would be a topic under solid mechanics/elasticity, but are there any more specific subject headings I might use to look up in textbooks? (an idea I have from my novice intuition is that this is going to be some sort of a PDE problem -- the elasticity is very simple: so I don't think it needs a tensor description?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also interested in knowing if there are any textbooks that explore the sort of numerical approximation I illustrated? (an idea I have from my novice intuition is that it is &lt;em&gt;not&lt;/em&gt; finite element method)&lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="9466" LastEditDate="2015-07-19T02:37:07.690" LastActivityDate="2015-07-19T02:37:07.690" Title="Reference request: theory regarding time evolution of closed loop 2D elastic shapes?" Tags="|numerical-analysis|reference-request|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20213" PostTypeId="1" CreationDate="2015-07-17T22:48:54.463" Score="1" ViewCount="56" Body="&lt;p&gt;I have the following simple code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v=[];&#xA;&#xA;for i=10:-1:1&#xA;for j=0:0.2:5&#xA;    v(end+1)=i+j;&#xA;end&#xA;end&#xA;min(v) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It returns the minimum of vector $v$ whose length in this case is 260. I would like the code to return the values of $i$ and $j$ that produce the minimum value of $v$. I defined &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a=cell(2,2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I included the statement in the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a{end+1,end+1}=[i j];&#xA;b=reshape(a,1,length(i)*length(j))&#xA;for k=1:length(v)&#xA;if v(k)==min(v)&#xA;disp(b{1,k})&#xA;end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it didn't work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;" OwnerUserId="16815" LastActivityDate="2015-07-21T00:20:15.873" Title="Arranging variable ranges into a cell MATLAB" Tags="|matlab|vector|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20214" PostTypeId="2" ParentId="20210" CreationDate="2015-07-17T22:49:37.990" Score="2" Body="&lt;p&gt;The only interesting quantity of a picture -- from a complexity perspective -- is the number of pixels, not its size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether any given paper follows this observation is of course a different matter, but if nothing different is specified, I would assume that $N$ is the number of pixels indeed.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-07-17T22:49:37.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20215" PostTypeId="2" ParentId="20202" CreationDate="2015-07-17T22:53:00.157" Score="3" Body="&lt;p&gt;As a sidenote, the physical dimensions of your scaling parameters $\sigma,\alpha$ are different. So a statement of the form $\sigma\ll\alpha$ as given in your question doesn't make any sense: are 3kg of apples much less or much more than 5 micrometers of yarn?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to the references others have given before, let me also recommend the book by Somersalo and Kaipio on &quot;Statistical Inverse Problems&quot;. It also contains many discussions of the practicalities of inverse problems.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-07-17T22:53:00.157" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20216" PostTypeId="2" ParentId="20212" CreationDate="2015-07-18T06:23:05.157" Score="1" Body="&lt;p&gt;This sounds like something that could be handled with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Level_set_method&quot; rel=&quot;nofollow&quot;&gt;level set method&lt;/a&gt;, although I've never done enough with them to know how to apply the elasticity. Stan Osher's group at UCLA used to be doing a lot of work in this area, but I'm not sure who the experts are nowadays. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This book by Stan Osher and Ron Fedkiw might be a good start: &lt;a href=&quot;http://www.springer.com/us/book/9780387954820&quot; rel=&quot;nofollow&quot;&gt;Level Set Methods and Dynamic Implicit Surfaces&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="12" LastActivityDate="2015-07-18T06:23:05.157" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20217" PostTypeId="1" AcceptedAnswerId="20219" CreationDate="2015-07-18T06:45:59.307" Score="0" ViewCount="1126" Body="&lt;p&gt;I am in the process of writing a finite element solver for the Navier-Stokes equations and am having trouble computing things like the divergence and vorticity correctly. Currently to compute the vorticity,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega = \frac{\partial{v}}{\partial{x}} - \frac{\partial{u}}{\partial{y}}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would multiply the r.h.s by a test function and integrate over the element. This gives element matrices of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma_{i}\int\int_{\Omega}v_{i}\frac{d\phi_{i}}{dx}\phi_{j}dA \hspace{5mm}\text{and}\hspace{5mm} \Sigma_{i}\int\int_{\Omega}u_{i}\frac{d\phi_{i}}{dy}\phi_{j}dA$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;after plugging in our expansions for u and v. This however appears to be the wrong way to do this since my results do not look right. For example after computing the u and v velocity fields:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/SlqzW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/06cQ0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my vorticity looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/gd74S.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular the vorticity appears very non-smooth. Here is another example showing the u-velocity and vorticity (zoomed in for clarity):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/W9JaT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/IptsA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a similar problem with calculating the divergence. It seems that my computed velocity field is not smooth enough and thus when computing the vorticity and divergence I get many positive and negative gradients resulting in choppy fields. So my question is, how do you calculate the vorticity (or divergence) of a computed velocity field using finite elements?&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2015-07-18T10:34:05.580" Title="How to calculate divergence and vorticity from a velocity field using finite elements" Tags="|finite-element|simulation|navier-stokes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20218" PostTypeId="2" ParentId="20191" CreationDate="2015-07-18T10:25:01.207" Score="1" Body="&lt;p&gt;In the specific case of the Ising model and its generalizations, imagine you enumerate the spins, and then you create a matrix $H$ where the $ij$th entry $H_{ij}$ indicates the coupling strength between spins $i$ and $j$. You can obviously implement the dynamics of the Ising system using only this matrix. In particular, the energy is simply the product $\mathbf s^T H \mathbf s$ where $\mathbf s$ is the vector that contains plus or minus ones, depending on whether a spin is up or down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The one-dimensional Ising model then corresponds to a matrix $H$ where only the diagonal and the immediate off-diagonals are nonzero. If you have interactions with spins further away, then there are also nonzero entries further away. In the case of the one-dimensional, nearest-neighbor, periodic Ising model, you would get a matrix $H$ where the diagonal, the immediate upper and lower off-diagonal entries, as well as the bottom left and the top right entry of the matrix are nonzero, and all other entries are zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this connection between matrix and system, you can pretty easily generalize the original Ising model: just write your algorithm in terms of $H$, and then choose the $H$ that corresponds to the situation you care about.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-07-18T10:25:01.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20219" PostTypeId="2" ParentId="20217" CreationDate="2015-07-18T10:34:05.580" Score="3" Body="&lt;p&gt;If you have a vector of coefficients $U$ for the velocity approximation, then your velocity field is given by $\vec u_h = \sum_j U_j \vec\varphi_j$. The divergence is then clearly&#xA;$$&#xA;  \text{div} \;\vec u_h = \sum_j U_j\;\text{div}\,\vec\varphi_j&#xA;$$&#xA;and similarly for the curl.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing to realize, though, is that even if you are using continuous finite elements, i.e., where the $\vec\varphi_j$ are continuous functions, the divergence and curl are not continuous functions. The same then holds true for the divergence and curl of the error. Consequently, you must expect these errors to be oscillatory. This may be what you observe.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-07-18T10:34:05.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20220" PostTypeId="2" ParentId="20213" CreationDate="2015-07-18T10:46:54.457" Score="0" Body="&lt;p&gt;here I'm making some correction in that code and logic.&#xA;$i$ and $j$ in matlab are reserved for complex numbers so please avoid that as much as possible in your variable name.&#xA;My first code uses the advantage of matlab built in command to find the values.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc&#xA;clear all&#xA;i1=10:-1:1;&#xA;j1=0:0.2:5;&#xA;v=zeros(1,length(i1)*length(j1));&#xA;k=1;&#xA;for ii=1:1:length(i1)&#xA;for jj=1:1:length(j1)&#xA;    v(k)=i1(ii)-j1(jj);&#xA;      k=k+1;&#xA;end&#xA;end&#xA;&#xA;v1=[reshape(v,length(j1),length(i1))]';&#xA;[M,I] = min(v1(:));&#xA;[I_row, I_col] = ind2sub(size(v1),I);&#xA;disp('requied i_value is')&#xA;disp(i1(I_row))&#xA;display('requied j_value is')&#xA;display(j1(I_col))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Second code will give all the values of $i$ and $j$ corresponding to minimum value of $v$. This logic can be used in other programming languages.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc&#xA;clear all&#xA;i1=10:-1:1;&#xA;j1=0:0.2:5;&#xA;v=zeros(1,length(i1)*length(j1));&#xA;k=1;&#xA;for ii=1:1:length(i1)&#xA;for jj=1:1:length(j1)&#xA;    v(k)=i1(ii)-j1(jj);&#xA;    k=k+1;&#xA;end&#xA;end&#xA;k=1;&#xA;for ii=1:1:length(i1)&#xA;for jj=1:1:length(j1)&#xA;    if v(k)==min(v)&#xA;        disp('requied i_value is')&#xA;        display(i1(ii))&#xA;        display('requied j_value is')&#xA;        display(j1(jj))&#xA;&#xA;&#xA;    end&#xA;    k=k+1;&#xA;end&#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16930" LastActivityDate="2015-07-18T10:46:54.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20221" PostTypeId="1" AcceptedAnswerId="20222" CreationDate="2015-07-18T16:00:30.623" Score="3" ViewCount="671" Body="&lt;p&gt;The Darcy equations for porous media flow are given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\mu}{\kappa}\mathbf{u} - \nabla p = \mathbf{0}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla\cdot\mathbf{u} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\kappa$ is the permeability and can in general be spatially dependent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be put into a weak formulation that looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a(\mathbf{u},\mathbf{v}) + b(p,\mathbf{v}) = -\int_\Gamma p\mathbf{v}\cdot\mathbf{n}d\Gamma$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b(q,\mathbf{u}) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a(\mathbf{u},\mathbf{v}) = \int_\Omega \frac{\mu}{\kappa} \mathbf{u}\cdot\mathbf{v}d\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b(p, \mathbf{v}) = -\int_\Omega p\nabla\cdot\mathbf{v}d\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'd like to have no through boundary conditions on part of the domain $\Gamma$, say $\Gamma_1$. Then I can say that $\mathbf{u}$ and $\mathbf{v}$ belong to the space:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\hat{\mathbf{H}} = \{\mathbf{v}\in\mathbf{H}(\text{div}) : \mathbf{v}\cdot\mathbf{n} = 0 \text{ on } \Gamma_1\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to prescribe $p$ on the remaining part of the boundary $\Gamma_2$. Then I have $p$ belonging to the space:&#xA;$ L^2_a = \{p \in L^2 : p = a \text{ on } \Gamma_2\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $q$ belonging to $L^2_0$. If there's anything wrong with what I've done so far please let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question now is, what basis for my discrete space should I choose? For Stokes flow the common choice is Taylor-Hood; I think that should work here too. The only possible complication I see is that now $\mathbf{u}$ has to be in $\mathbf{H}(\text{div})$ instead of $\mathbf{H}^1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that $a(\mathbf{u}, \mathbf{v})$ is still bounded and coercive (for smooth $\kappa$), and the Taylor-Hood element guarantees that $b(p,\mathbf{v})$ is also bounded and coercive. In literature however I'm finding it difficult to find many references to support this choice. Is there a better choice for Darcy flow? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: If I expect my solution $\mathbf{u}$ to be in $\mathbf{H}^1$, due to a smooth domain and smooth $\kappa$, would it be acceptable to use the Taylor-Hood element pair? If so, would this choice of spaces be better if I am interested in the viscous stresses, which depend on $\nabla\mathbf{u}$ (and might not exist if $\mathbf{u}\in\mathbf{H}(\text{div})$)?&lt;/p&gt;&#xA;" OwnerUserId="9728" LastEditorUserId="9728" LastEditDate="2015-07-29T10:22:17.660" LastActivityDate="2015-07-29T10:22:17.660" Title="Darcy flow finite elements" Tags="|pde|finite-element|fluid-dynamics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20222" PostTypeId="2" ParentId="20221" CreationDate="2015-07-18T19:47:44.703" Score="4" Body="&lt;p&gt;I think this is more the mixed finite element method for Poisson's equation with Dirichlet boundary data.  In that case, a(.,.) is not coercive over the whole velocity space, but rather only coercive over the kernel of b(p,.), i.e. coercive over the space of all divergence free velocities (I believe). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shows up in the approximation spaces as you need the div(velocity space) = pressure space.  Taylor-Hood elements do not have this property and you need to use the Raviart-Thomas elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prescribing discontinuous pressure boundary conditions shouldn't be a problem as long as they are in $L^2$.  However, just becareful with your notation, I once used the notation $L^{2}_{0}$ and was repremanded that $L^{2}$ functions dont enough regularity to have a trace! :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two good references are: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0387759336&quot; rel=&quot;nofollow&quot;&gt;http://www.amazon.com/Mathematical-Element-Methods-Applied-Mathematics/dp/0387759336&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as well as Arnold's notes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ima.umn.edu/~arnold/8445-8446.14-15/notes.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.ima.umn.edu/~arnold/8445-8446.14-15/notes.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16983" LastActivityDate="2015-07-18T19:47:44.703" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20223" PostTypeId="1" AcceptedAnswerId="20227" CreationDate="2015-07-19T05:41:36.703" Score="1" ViewCount="1133" Body="&lt;p&gt;Please, I am currently implementing direct multiple shooting methods* and I need one simple but fundamental concept answered:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I want to provide not only objective function value (the result of ODE integrator) but as well derivatives (for the NLP solver), for the control parameters it is just sensitivity analysis of said ODE.&#xA;&lt;strong&gt;But what about the parameters, which arise in direct multiple shooting from the discretization points of the path (state and cost trajectory)? What should I tell the solver about derivation of the cost by these parameters?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, I managed to run the multiple shooting algorithms in Matlab, but now I want to add the derivatives of the ODE concerning control parameters.&#xA;(Note to say, that this question does not depend on the implementation or on the problem itself.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it sufficient to set them to zero? These parameters are not free, they are just numerical-clever-thingies, and their values are just bound using equality constraints.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;*Brief description of direct multiple shooting, if you know the method by another name:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is numeric method of discretization for optimal control problems.&#xA;We have an ODE and control function. By translating this to NonLinear Programming problem, we can solve the original problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Direct Single shooting does just this, just discretizes (at certain timepoints) the control function and calls a routine for NLP using the parameters of the discretized function as parameters to optimize against. (And the Cost function is just solved using ODE solver that gets these parameters.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Direct multiple shooting is somewhat clever. When computing the ODE (for calculating the cost function), it gets not only the parameters of the discretized control function but also the beginning points, where to begin the integration of the ODE. AND at the same time, the method says to the NLP optimizer - use equivality constraints to keep these parameters (beginnings of the trajectories) equal to endings of just the trajectory, that was computed at the previous time slot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Exactly as described, for example, in &lt;a href=&quot;https://workspace.imperial.ac.uk/people/Public/chemicalengineering/b.chachuat/ic-32_secured.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://workspace.imperial.ac.uk/people/Public/chemicalengineering/b.chachuat/ic-32_secured.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;By the way, if I get this working, I will definitely upload my solution in python + theano somewhere :)&lt;/p&gt;&#xA;" OwnerUserId="16985" LastEditorUserId="21193" LastEditDate="2017-12-23T19:14:57.947" LastActivityDate="2017-12-23T19:14:57.947" Title="Direct multiple shooting (numerical optimal control)" Tags="|optimization|nonlinear-programming|optimal-control|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20224" PostTypeId="1" CreationDate="2015-07-19T11:16:14.923" Score="2" ViewCount="281" Body="&lt;p&gt;&lt;strong&gt;Objective:&lt;/strong&gt; I am trying to numerically solve $C(x,y,t)$ from the following advection-diffusion-reaction equation in 2D space (x,y) and time. I will be testing my numerical solution with an approximate 1D analytical solution (given on 2nd page in &lt;a href=&quot;https://drive.google.com/open?id=0B0g3l3PoZi9lNUtaUEd1amwydWc&quot; rel=&quot;nofollow noreferrer&quot;&gt;this file&lt;/a&gt;). However, the issue is that I am not able to get correct numerical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\text{ADR Equation: }\frac{\partial C}{\partial t} + \nabla\left(v  C - D\nabla{C} \right)=  \alpha C&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method:&lt;/strong&gt; I discretized the above ADR equation in 2D using finite-difference implicit scheme and I get the following equation in difference form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;p_1C^{n+1}_{i,j-1}+p_2C^{n+1}_{i-1,j}+p_3C^{n+1}_{i,j}+p_4C^{n+1}_{i+1,j}+p_5C^{n+1}_{i,j+1} = C^{n}_{i,j}&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve this as a system of equations using $A^{n+1} C^{n+1}=C^{n}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Variables:&lt;/strong&gt; $A$ is a matrix formed from variables $p_1, p_2, p_3, p_4, p_5$, which are constants in time but vary along $(x,y)$ grid. Superscript $n$ represents time index and subscripts $i,j$ represent space along $x,y$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I.C:&lt;/strong&gt; $$\textrm{$C(x,y,0) = C_i$, assume $C_i$ as 1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;B.C:&lt;/strong&gt; $$\frac{\partial C}{\partial x} = \frac{\partial C}{\partial y} = 0 \text{ at boundaries}$$. &#xA;$$C(0,y=1,t) = \begin{cases} C_0\quad &amp;amp; \text{if $0&amp;lt;t\leq t_0$} \\ &#xA; 0 \quad &amp;amp; \text{if $t&amp;gt;t_0$} \end{cases}$$ &#xA;$\textrm{assume $C_0$ and $t_0$ as 5000 and 2 days, respectively}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Numerical Solution:&lt;/strong&gt; I estimated $C(x,y,t)$ for $t=0$ to $t=100$ days, where dimensions of $x$ and $y$ are shown below in results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/n20Vz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textrm{Fig-1: Snapshots of $C(x,y)$ at various $t$}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Fig-2, I am showing $C(x,t)$ for $0&amp;lt;t\leq t_0$ and $t=0$ to $t=100$ days:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/iwsuJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textrm{Fig-2: Snapshots of $C(x,t)$ obtained after averaging $C(x,y,t)$ over Y-axis}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analytical Solution:&lt;/strong&gt; Here I am showing $C(x,t)$ for $0&amp;lt;t\leq t_0$ and $t=0$ to $t=100$ days. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/BHcRO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textrm{Fig-3: Snapshots of $C(x,t)$. I am trying to match this with Fig-2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Issue:&lt;/strong&gt; I have been trying to figure out if there is a problem in my numerical solution developed as per the implicit scheme discussed above; however, I have spent several hours and I couldn't figure it out. Please let me know if I missed providing anything.&lt;/p&gt;&#xA;" OwnerDisplayName="user5510" LastEditorUserId="9667" LastEditDate="2015-07-20T13:19:05.803" LastActivityDate="2015-07-20T13:19:05.803" Title="Not getting correct numerical solution for Advection-Diffusion-Reaction eqn" Tags="|finite-difference|advection-diffusion|implicit-methods|diffusion|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20225" PostTypeId="1" CreationDate="2015-07-19T15:26:29.613" Score="1" ViewCount="500" Body="&lt;p&gt;I am trying to implement a 1D k-$\varepsilon$ turbulent model for a turbidity current, hence the conservation equation for $c$. I'm solving for the variables $u,c,k$ and $\varepsilon$. The remaining terms are parameters.&#xA;$$&#xA;\newcommand{\cmu}{c_{\mu}}&#xA;\newcommand{\cmuc}{c_{\mu c}}&#xA;\newcommand{\dervz}[1]{\frac{\partial #1}{\partial z}}&#xA;\newcommand{\eddyvis}{\cmu \frac{k^2}{\varepsilon}}&#xA;\newcommand{\eddyvisC}{\cmuc \frac{\hat{k}^2}{\hat{\varepsilon}}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u }{\partial t} = \frac{\partial}{\partial z} \left[ \left( \nu + \cmu \frac{k^2}{\varepsilon} \right) \frac{\partial u }{\partial z} \right]&#xA;+ \frac{\rho_s - \rho_w}{\rho_w} c g_x&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA; \frac{\partial c}{\partial t} =&#xA; v_s \dervz{c} + \frac{\partial}{\partial z} \left[ \left( \nu + \cmuc \frac{k^2}{\varepsilon} \right) \dervz{c} \right] &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA; \frac{\partial k}{\partial t} =&#xA;  \frac{\partial}{\partial z}  \left[ \left( \nu + \frac{\cmu}{\sigma_k} \frac{k^2}{\varepsilon} \right) \dervz{k} \right] + \eddyvis \left(\dervz{U}\right)^2 - \varepsilon &#xA;  + \frac{\rho - \rho_w}{\rho_w} g_z \frac{1}{\sigma} \eddyvisC \dervz{c}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA; \frac{\partial \varepsilon}{\partial t} =&#xA;  \frac{\partial}{\partial z}  \left[ \left( \nu + \frac{\cmu}{\sigma_{\varepsilon}} \frac{k^2}{\varepsilon} \right) \dervz{\varepsilon} \right] &#xA;  + c_{\varepsilon 1} \frac{\varepsilon}{k} &#xA;  \left( \eddyvis \left(\dervz{u}\right)^2  + c_{\varepsilon 3} &#xA;  \frac{\rho - \rho_w}{\rho_w} g_z \frac{1}{\sigma} \eddyvisC \dervz{c} \right) - C_{\varepsilon 2} \frac{ \varepsilon^2}{k}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I am using finite difference. Upwind scheme for the first order derivatives. Because the paramter $v_s$ is positive, concentration travels downwards at a speed $v_s$, I'm implementing the upwind scheme as:&#xA;$$&#xA;\frac{c^{i+1}_n - c^{i}_n}{\Delta z}&#xA;$$&#xA;for the node $i$. I also use the same upwind scheme for all the first-order derivative terms, i.e. also the terms inside the diffusion terms, which are nonlinear. For the linear diffusion terms, I apply second order central difference. For the time integration, I use a Crank-Nicholson method treating all terms implicitly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions are complex. I'm using a standard wall function, using the log law at a node far from the wall. Using the log law, I also impose a gradient boundary condition because the log law adds the shear velocity as a unknown, hence the additional equation/condition. For &#xA;$k$ and $\varepsilon$ I use conditions that depend on the shear velocity. That's for one side of the domain, at the other side, sufficiently far, I use zero value conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not seeing much success with this implementation, should I use different discretization schemes for the diffusion and the advective terms? Should I use a different method altogether such as Finite Volume? What confuses me are the diffusion terms. Which discretization should I use there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to the time discretization. Should I treat some terms explicitely? The source terms are nonlinear as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Response to the first comment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry I was not clear there. The model blows up for certain initial values of the concentration (too much acceleration), although that might have to do with the wall function implementation I think. Another issue that I had  were the initial conditions for $k$ and $\varepsilon$. I'll explain. The initial conditions for the velocity and concentration do not span the entire domain. They have positive values only for one tenth of the lower part of the domain, where I implement the wall function boundary condition, eventually, they will diffuse upwards. However, they will not do so if the values for $k$ and $\varepsilon$ are zero above where $u$ and $c$ are. This is because they are in the diffusion term and I think because of the upwind scheme, which assumes the wave travels downwards.&lt;/p&gt;&#xA;" OwnerUserId="7023" LastEditorUserId="7023" LastEditDate="2015-07-19T15:58:38.987" LastActivityDate="2017-11-06T10:06:55.730" Title="1D k-epsilon turbulence model in a turbidity current" Tags="|finite-difference|advection-diffusion|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20226" PostTypeId="2" ParentId="20224" CreationDate="2015-07-20T07:28:41.427" Score="2" Body="&lt;p&gt;One of the best ways to test a PDE solver is to use the method of manufactured solutions. Essentially, you modify the PDE (and discretization) by adding a source term that yields an exact solution known in advance. You can then compare your numerical solution against the exact solution for debugging purposes. You should test your numerical solution against a known exact solution, and you should also test to make sure that the order of accuracy of your discretization is roughly what you expect (e.g., if you have a second-order accurate discretization in space, you should expect the error to be proportional to the square of the mesh spacing when performing a refinement study). A good reference is &lt;a href=&quot;http://prod.sandia.gov/techlib/access-control.cgi/2000/001444.pdf&quot; rel=&quot;nofollow&quot;&gt;this Sandia National Laboratory technical report&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to be more thorough, you could also set up and execute unit tests on various terms in your discretization (e.g., the diffusion term), comparing the numerical value on a known input to known (exact) output.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-07-20T07:28:41.427" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20227" PostTypeId="2" ParentId="20223" CreationDate="2015-07-20T07:37:17.793" Score="2" Body="&lt;p&gt;In the direct multiple shooting method, using the notation from the linked notes by Chachuat, the &quot;extra decision variables&quot; $\mathbf{\xi}_{0}^{k}$ are not present in the objective function, therefore the sensitivities of the objective function (i.e., the derivatives of the objective function) with respect to these parameters are all zero. You are correct that these parameters are not free; however, their values will change according to the KKT-type conditions used in nonlinear programming algorithms.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-07-20T07:37:17.793" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20229" PostTypeId="1" CreationDate="2015-07-20T08:30:00.870" Score="1" ViewCount="131" Body="&lt;p&gt;I am wondering if I am thinking correctly about the following problem :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Define the box of the dimensions &lt;span class=&quot;math-container&quot;&gt;$(a,a,H)$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; directions, respectively.&lt;/li&gt;&#xA;&lt;li&gt;Insert &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; particles into the box using PBC in &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; directions&lt;/li&gt;&#xA;&lt;li&gt;Move the particles using a quite analogous condition as in the previous pgms (of course, PBC in &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; directions should be taken into account)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To test the pgm use &lt;span class=&quot;math-container&quot;&gt;$a=10$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$H=6$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; of the order of &lt;span class=&quot;math-container&quot;&gt;$100-500$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I have generated random numbers for the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; axis from the interval &lt;span class=&quot;math-container&quot;&gt;$[0,a]$&lt;/span&gt; and for &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$[0,h]$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then, I have applied the PBC to the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; axis and shifted them with the following rule: &lt;code&gt;xt=x(j)+shift*2*rand() - shift&lt;/code&gt; and also applied PBC to the shifted position.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Xt&lt;/code&gt; is the new position of the point. For the y axis the operation is similar. &#xA;&lt;code&gt;shift&lt;/code&gt; is the parameter in the interval &lt;span class=&quot;math-container&quot;&gt;$[0.1,1]$&lt;/span&gt; but in this case, it is &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I thinking correctly about this problem, or there is some other way that I should take into account?&lt;/p&gt;&#xA;" OwnerUserId="16846" LastEditorUserId="20688" LastEditDate="2019-06-22T22:33:08.220" LastActivityDate="2019-06-22T22:33:08.220" Title="Monte Carlo simulation" Tags="|monte-carlo|" AnswerCount="0" CommentCount="7" ClosedDate="2019-06-22T22:33:16.963" ContentLicense="CC BY-SA 4.0" />
  <row Id="20230" PostTypeId="1" AcceptedAnswerId="23701" CreationDate="2015-07-20T09:31:16.830" Score="1" ViewCount="230" Body="&lt;p&gt;I want to invert large matrices ($10^4 \times 10^4$ to $10^6 \times 10^6$) but sparse (less than $100$ non-zero entries per line) on clusters with $16$ to $48$ processors per node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for an efficient method to do so. I've tried a few different&#xA;solutions but I face some problems.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As the matrix is big, my first attempt to use lapack failed because it&#xA;doesn't handle sparse matrix&lt;/li&gt;&#xA;&lt;li&gt;The condition number can be quite large&lt;/li&gt;&#xA;&lt;li&gt;It seems that with that kind of matrix, the exact solvers are not efficient&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;On the positive side, I don't need an exact inverse or an exact solution to&#xA;the equation $Ax=b$, therefore iterative methods may be more efficient. I&#xA;think I'm going to use the conjugate gradient method because I could&#xA;efficiently make use of the many processors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question comes now : do you think that this method could be efficient ? if&#xA;so, what preconditioner should I use ? (I can't really use the Jacobi&#xA;preconditioner because $A_{ii}=1$ and $\sum_{j\neq i}|A_{ij}|\approx 10$)&lt;/p&gt;&#xA;" OwnerUserId="4626" LastEditorUserId="20417" LastEditDate="2016-06-01T04:48:26.200" LastActivityDate="2016-06-01T04:48:26.200" Title="solve linear system of equation of a large sparse symetric positive definite matrix" Tags="|linear-solver|sparse-matrix|matrix|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="20231" PostTypeId="1" CreationDate="2015-07-20T12:20:16.393" Score="1" ViewCount="68" Body="&lt;p&gt;Assume I have a data set &lt;span class=&quot;math-container&quot;&gt;$ { \left\{ {x}_{i} \right\} }_{i = 1}^{N} $&lt;/span&gt; which represents the value of each data point.&lt;br&gt;&#xA;For each data we have its features &lt;span class=&quot;math-container&quot;&gt;$ {f}_{i} \in {\mathbb{R}}^{d} $&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model I came for the data is &lt;span class=&quot;math-container&quot;&gt;$ \arg \min_x {\left( x - \sum_{j \neq i} {w}_{ij} {x}_{j} \right)}^{2} $&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$ {w}_{i, j} = \exp \left( -\frac{1}{2} {\left( {f}_{i} - {f}_{j} \right)}^{T} C \left( {f}_{i} - {f}_{j} \right) \right) $&lt;/span&gt;.&lt;br&gt;&#xA;This model is idealization of the data, namely ideally the data would have obeyed this law yet it doesn't due to perturbations in data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet the model is correct only if the the correct &lt;a href=&quot;https://en.wikipedia.org/wiki/Mahalanobis_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mahalanobis Distance&lt;/a&gt; Matrix &lt;span class=&quot;math-container&quot;&gt;$ C $&lt;/span&gt; is chosen (We don't need its inverse, so lets stick with &lt;span class=&quot;math-container&quot;&gt;$ C $&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if I have a set of data and a set of features of the data, how can I find the matrix &lt;span class=&quot;math-container&quot;&gt;$ C $&lt;/span&gt; which minimizes the model?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formally, what I'm looking for given the data set &lt;span class=&quot;math-container&quot;&gt;$ { \left\{ \left( {x}_{i}, {f}_{i} \right) \right\}}_{i = 1}^{N} $&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \arg \min_{C} {\sum_{i = 1}^{N} \left( {x}_{i} - \sum_{j \neq i} exp \left( -\frac{1}{2} {\left( {f}_{i} - {f}_{j} \right)}^{T} C \left( {f}_{i} - {f}_{j} \right) \right) {x}_{j} \right)}^{2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&#xA;Is it any other defined problem type I can find and read about?&lt;br&gt;&#xA;How would you solve it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="7951" LastEditorUserId="7951" LastEditDate="2020-01-11T21:11:11.313" LastActivityDate="2020-01-11T21:11:11.313" Title="Interpolation of Data Value using Optimized Weighting of Its Features" Tags="|optimization|matlab|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20232" PostTypeId="1" AcceptedAnswerId="20233" CreationDate="2015-07-20T15:17:20.417" Score="4" ViewCount="411" Body="&lt;p&gt;I like to create interpolation functions for second order finite element meshes. For elements with straight edges all is good, but some of my elements may have curved edges as shown in the figure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/6uGMu.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for references on how to test efficiently and robustly if a query point is inside a curved element. The interpolation itself is not a problem. One general way to do it is to find the mapping of the global element in $x$, $y$-coordinates to its mother element in $r$, $s$-coordinates, then map the query point and see if that mapped query point is within the mother element which has straight edges. This seems quite excessive and I am wondering if there are alternatives that you could point me to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&#xA;This is for a general interpolation, to be used in plotting functions and the like. The query points can be 'random'. Even if one has a very good algorithm to find the closest element to a query point one needs to check if it is in the element. If that's not the case then a new potential element needs to be found and tested. This means that at a minimum at least one query needs to be done per query point. The expensive testing can be reduced if information if the element is curved or not can be used. In case the element is not curved then a cheap linear test can be used.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="11893" LastEditDate="2015-11-18T21:33:36.037" LastActivityDate="2015-11-18T21:33:36.037" Title="Point inside curved finite element" Tags="|reference-request|computational-geometry|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20233" PostTypeId="2" ParentId="20232" CreationDate="2015-07-20T16:43:36.230" Score="6" Body="&lt;p&gt;I don't think you can do better in general than mapping to the reference element and testing there. If your mapping is somehow special, you might be able to develop a test that's more efficient than the usual method, but I haven't seen one in practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One question I have for you is what are you doing that requires lots of checks for which element a point is in?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-07-20T16:43:36.230" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20234" PostTypeId="2" ParentId="20232" CreationDate="2015-07-20T18:43:07.810" Score="3" Body="&lt;p&gt;You could adapt one of the traditional Point-in-Polygon algorithms to suit your needs. In particular, you could generalize the &quot;Ray Casting&quot; method. The idea is this: Cast a ray in any direction from your point. Compute the number of times that ray intersects the boundary of your element (note, this could be &gt;1 per side for your curved edges). If the number of intersections is &lt;em&gt;odd&lt;/em&gt;, then the point is inside the element. If the nubmer of intersections is even (or zero), the point is outside the element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cound is a bit sensitive to corner cases (literally corners: you have to be careful not to double-count an intersection at an element corner), but as long as you take care of that, it's not hard to write a reasonably efficient implementation of this. For the quadratic elements that you have pictured, you will be solving for the intersection between a line and a parabola, which is very easy to do (solve a quadratic equation). Once you have your 0 or 2 solutions to the equation, you check to see if the intersections occur on the segment of the parabola that comprises your element edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've never done this for curved elements before, but I can't see any reason why the method shouldn't work. Generalizing this to 3D should work, in principle, but computing the intersections would probably be more difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a link to the wikipedia page that I used as a reference when I encountered this problem.&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_algorithm&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2015-07-20T18:43:07.810" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="20236" PostTypeId="2" ParentId="20213" CreationDate="2015-07-21T00:20:15.873" Score="2" Body="&lt;p&gt;you just need to add output to your &lt;code&gt;min()&lt;/code&gt; function, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[value, ind] = min(v)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;ind&lt;/code&gt; is then the index values of your min locations (the i,j that you are looking for) and &lt;code&gt;value&lt;/code&gt; has the corresponding min values&lt;/p&gt;&#xA;" OwnerUserId="16952" LastActivityDate="2015-07-21T00:20:15.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20237" PostTypeId="2" ParentId="20190" CreationDate="2015-07-21T00:25:02.250" Score="1" Body="&lt;p&gt;You can try using one of the iterative solvers that MATLAB provides, as x = A\b works well for many systems, you can run into memory issues causing slowdowns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately, MATLAB has a number of built in functions for iteratively solving Ax=b for sparse matrices, such as &lt;code&gt;pcq(), bigcg(), cgs()&lt;/code&gt;, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the &lt;a href=&quot;http://www.mathworks.com/help/matlab/linear-equations-iterative-methods.html?searchHighlight=iterative%20methods%20linear%20equations&quot; rel=&quot;nofollow&quot;&gt;MATLAB Documentation on interative methods for solving systems of linear equations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16952" LastActivityDate="2015-07-21T00:25:02.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20238" PostTypeId="5" CreationDate="2015-07-21T01:30:36.783" Score="0" Body="&lt;p&gt;A nonlinear system is a system which does not satisfy the superposition principle, i.e., meaning that the output of a nonlinear system is not directly proportional to the input.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-08-24T16:38:23.060" LastActivityDate="2015-08-24T16:38:23.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20239" PostTypeId="4" CreationDate="2015-07-21T01:30:36.783" Score="0" Body="Solution of nonlinear systems of equations. The equations might be algebraic or differential equations." OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-08-24T16:38:28.830" LastActivityDate="2015-08-24T16:38:28.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20241" PostTypeId="1" CreationDate="2015-07-21T08:03:54.137" Score="5" ViewCount="630" Body="&lt;p&gt;Most authors are very clear that it's very dangerous to just use $\mathrm{H}(curl)$ conforming edge elements, which are divergence free, to satisfy $\mathrm{div}(\mathbf{B})=0$ and implement this condition in no other way.&#xA;I just don't understand why. Could anyone explain in which case this could fail or point me to some good source explaining in which case $\mathrm{H}(curl)$ conforming Nedelec Elements would fail to satisfy $\mathrm{div}(\mathbf{B})=0$ and why?&lt;/p&gt;&#xA;" OwnerUserId="16990" LastActivityDate="2015-07-27T12:10:44.063" Title="H(curl) conforming Nédélec-Elements to satisfy div(B)=0" Tags="|finite-element|computational-physics|electromagnetism|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20242" PostTypeId="1" CreationDate="2015-07-21T11:25:41.840" Score="1" ViewCount="182" Body="&lt;p&gt;I have a numerical scheme to solve an hyperbolic system of equations of this type (this a more simple version just for clarity):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t}+u\frac{\partial u}{\partial x}+v\frac{\partial u}{\partial y}=\frac{\partial H}{\partial x}$$&#xA;$$\frac{\partial v}{\partial t}+u\frac{\partial v}{\partial x}+v\frac{\partial v}{\partial x}=\frac{\partial H}{\partial y}$$&#xA;$$\frac{\partial H}{\partial t}+ \frac{\partial u}{\partial x}+\frac{\partial v}{\partial x}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have  a steady state analytical solution, and I am checking the order of accuracy of the scheme. It is a direction splitting scheme, and all the terms should be second order accurate. I used 4 grids, with successive refinements $i=1\ldots 4$ (cartesian uniform grids with grid size  $\Delta x_i=\Delta x_{i-1}/2$ and $\Delta x=\Delta y$). I saw that the order of accuracy is somehow around 1.5, therefore I decided to replace each derivative with the correspondent analytical term  until I found the term responsible for the loss of accuracy. The culprits were: $\frac{\partial H}{\partial x}$ and $\frac{\partial H}{\partial y}$.  Now my questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Am I doing right by using a fixed CFL number, in the sense that every time I half the grid size I also half the time step? What if I use a fixed dt for all the grids? Note that the scheme is implicit and unconditionally stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Im using a staggered grid, i.e. H is cell centered and u and v are on the center of the cell edges. The terms $\frac{\partial H}{\partial x}$ and $\frac{\partial H}{\partial y}$ are discretized by second order central differences. When I print the error of $\frac{\partial H}{\partial x}$ after a single time step starting from the exact solution I see that it converges perfectly with second order accuracy (it is 4 times smaller at each refinement). However when I start from a second-order accurate approximation of H, $\frac{\partial H}{\partial x}$ has first order rate of convergence. I am trying to figure this out and it might make sense since  $\frac{\partial H}{\partial x}= \frac{H_{m+1}-H_m}{\Delta x} + O(\Delta x^2)$ that is a second order accurate expression at the edge $m+1/2$ if $H_{m+1}$ and $H_m$ are exact. However, if I start from a second order accurate approximation of $H_{m+1}$ and $H_m$: &#xA;$$H_{m+1} = H^{exact}_{m+1}+ O(\Delta x^2)$$&#xA;$$H_{m} = H^{exact}_{m}+ O(\Delta x^2)$$&#xA; (as it would be the second order accurate solution of the system) I get  $$\frac{\partial H}{\partial x}= \frac{H^{exact}_{m+1}-H^{exact}_m}{\Delta x} + \frac{O(\Delta x^2)}{\Delta x} + O(\Delta x^2)$$ that is first order, and thats what I get (first order) if I print the numeric value of $\frac{\partial H}{\partial x}$ for the 4 different grids. However, if I print $H$ for the first half time step it is second order (it is the second half of the splitting that cause the order of accuracy). Am I doing something wrong here?&lt;/p&gt;&#xA;" OwnerUserId="10032" LastEditorUserId="17294" LastEditDate="2017-03-29T07:04:59.653" LastActivityDate="2017-05-28T08:18:50.677" Title="Debugging an implemented numerical method: which term gives the drop in accuracy?" Tags="|hyperbolic-pde|accuracy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20243" PostTypeId="1" AcceptedAnswerId="20255" CreationDate="2015-07-21T11:41:05.420" Score="2" ViewCount="756" Body="&lt;p&gt;I would like to solve a generalized eigenproblem of real sparse symmetric matrices. Is there an efficient library which utilizes OpenCL in order to find a limited amount of the smallest eigenvalues in magnitude?&lt;/p&gt;&#xA;" OwnerUserId="15018" LastActivityDate="2015-07-27T15:43:39.337" Title="Sparse generalized eigensolver using OpenCL" Tags="|eigenvalues|opencl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20244" PostTypeId="2" ParentId="10541" CreationDate="2015-07-21T13:54:15.210" Score="5" Body="&lt;p&gt;With the reverse communication protocol of ARPACK, you do not need to store the $3n \times 3n$ matrix explicitly: you just need to provide two functions that compute:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \left[ \begin{array}{c} x \\ y \\ z \end{array}\right] \rightarrow \left[ \begin{array}{c} -A_0 x \\ y \\ z \end{array}\right]$ and &#xA;$ \left[ \begin{array}{c} x \\ y \\ z \end{array}\right] \rightarrow \left[ \begin{array}{c} A_1 x + A_2 y + A_3 z \\ y \\ z \end{array}\right]$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(you still pay the price of storing the $3\times n$-dimensional vectors but you do not pay anything for the matrices).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the invert transform, you can do the same, i.e. implement it yourself by using a callback that computes $x \mapsto M^{-1} x$ instead of $x \mapsto M x$ and replace the computed $\lambda's$ with $\lambda^{-1}$. To compute $M^{-1}x$, you can pre-factor your matrix $M$, which means only pre-factoring $A_0$ (using LU, Cholesky, or sparse versions of them depending on the structure of the matrix). For the full shift-invert transform, I think that something similar can be done.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2015-07-21T13:54:15.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20245" PostTypeId="1" AcceptedAnswerId="20246" CreationDate="2015-07-21T17:19:34.623" Score="11" ViewCount="4077" Body="&lt;p&gt;I'd like to learn how the Raviart-Thomas (RT) element works. To that end I'd like to analytically describe how the basis functions look on the reference square. The goal here is not to implement it myself, but rather just to get an intuitive understanding of the element. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm largely basing this work off of the triangular elements discussed &lt;a href=&quot;http://www.math.chalmers.se/~logg/pub/papers/KirbyLoggEtAl2010a.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, perhaps extending it to quadrilaterals is a mistake in itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I can define the basis functions for first RK element RK0:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{\phi}_i(\mathbf{x}) = \mathbf{a} + \mathbf{b}\mathbf{x} = \begin{pmatrix} a_1 + b_1 x\\a_2 + b_2 y\end{pmatrix}$$ for $i = 1,\dots,4.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conditions on $\mathbf{\phi}_i$ are that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{\phi}_i(\mathbf{x}_j)\cdot\mathbf{n}_j = \delta_{ij}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{n}_j$ is the unit normal shown below, and $\mathbf{x}_j$ is its coordinate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y8y6y.png&quot; alt=&quot;RT0&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the reference square $[-1,1]\times[1,1]$, so this leads to a system of equations for each basis function. For $\mathbf{\phi}_1$ this is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{pmatrix} 1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\ 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 1\\ -1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\ 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 1\end{pmatrix}\begin{pmatrix}a_1 \\ a_2\\ b_1\\ b_3\end{pmatrix} = \begin{pmatrix} 1\\0\\0\\0\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be solved to give:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{\phi}_1(\mathbf{x}) = \frac{1}{2}\begin{pmatrix} 1 + x\\ 0\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other basis functions can be found similarly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming this is correct, the next step is to find the basis functions for RK1. This is where I'm getting a little unsure of myself. According to the link above, the space we are interested in is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P_1(K) + \mathbf{x}P_1(K)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A basis for $P_1$ would be $\{ 1, x, y\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this means the RK1 basis functions should take the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{\phi}_i(\mathbf{x}) = \begin{pmatrix} a_1 + b_1 x + c_1 y + d_1 x^2 + e_1 xy\\ a_2 + b_2 x + c_2 y + d_2 xy + e_2 y^2\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leaves 10 unknowns for each basis function. If we apply the same conditions as in the RK0 case, namely:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{\phi}_i(\mathbf{x}_j)\cdot\mathbf{n}_j = \delta_{ij}$$, where $\mathbf{n}_j$ is the unit normal as shown below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tzNGG.png&quot; alt=&quot;RK1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this gives us 8 equations. The other 2 I think can be found from some moments. I'm not really sure how exactly. The link above talks about integrating against a basis for $[P_1]^2$, but I'm having trouble figuring out what that means. Am I on the right track, or have I completely missed something here?&lt;/p&gt;&#xA;" OwnerUserId="9728" LastEditorUserId="9728" LastEditDate="2015-07-21T17:45:06.230" LastActivityDate="2016-06-11T15:19:21.220" Title="Raviart-Thomas elements on reference square" Tags="|pde|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20246" PostTypeId="2" ParentId="20245" CreationDate="2015-07-21T17:58:42.140" Score="12" Body="&lt;p&gt;In general, you cannot just transfer the same polynomial basis from tetrahedral to quadrilateral elements.&lt;sup&gt;1&lt;/sup&gt;  In particular, the whole point of quadrilateral elements is to work with tensor products of one-dimensional polynomials, which is not possible for tetrahedral elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are in fact quadrilateral Raviart-Thomas elements, but their definition is different. In two dimensions, the polynomial space for $RT_k$ is given by&#xA;$$P_{k+1,k} \times P_{k,k+1},$$&#xA;where&#xA;$$P_{k,l} = \left\{\sum_{i=0}^k\sum_{j=0}^l a_{ij} x^iy^j: a_{ij}\in\mathbb{R}\right\}.$$&#xA;A typical polynomial for $k=0$ would thus be as you wrote, but for $k=1$ it would be&#xA;$$\begin{pmatrix} a_1 + b_1 x + c_1 x^2 + d_1 y + e_1 xy + f_2 x^2y\\&#xA;a_2 + b_2 y + c_2 y^2 + d_2 x + e_2 xy + f_2 xy^2&#xA;\end{pmatrix}.$$&#xA;Hence, $\dim RT_1 = 12$, and in general, $\dim RT_k = 2(k+1)(k+2)$. This means you need two additional degrees of freedom, which should be located on the interior of the element. (In general, for $RT_k$ you take $k+1$ normal derivatives on each facet, and the remaining degrees of freedom from the interior.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your actual question: For Raviart-Thomas elements, you usually take moments rather than point evaluations, i.e., the remaining conditions come from the conditions&#xA;$$ \int_{-1}^1 \int_{-1}^1 \phi_i(x,y) q_j(x,y) dx\,dx=\delta_{ij},$$&#xA;where the $\{q_j\}$ are a basis of $P_{k-1,k}\times P_{k,k-1}$ (e.g., $\{1,x,y\}$ for $k=1$).&#xA;To make it easier to get a full nodal basis, the facet degrees of freedom are usually not taken as point evaluation, but also as moment conditions:&#xA;$$ \int_{e_m} \phi_i(s)^T\nu_{e_m} \, q_{m,j}(s)\,ds,$$&#xA;where $e_m$ is one of the four edges, $\nu_{e_m}$ is the corresponding outer normal, and for each $m$, the $q_{m,j}$ form a basis of $P_k(e_m)$ (e.g., $\{1,x\}$ or $\{1,y\}$ for $k=1$ depending on the edge orientation).&#xA;Together, these degrees of freedom are unisolvent (i.e., the corresponding system of basis functions is always invertible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a discussion on quadrilateral Raviart-Thomas-Elements in Chapter 2.4.1 of &lt;a href=&quot;http://www.springer.com/us/book/9783642365188&quot; rel=&quot;nofollow&quot;&gt;Boffi, Brezzi, Fortin: &lt;em&gt;Mixed Finite Element Methods and Applications&lt;/em&gt;, Springer 2013&lt;/a&gt;, &lt;a href=&quot;http://www.ima.umn.edu/~arnold//papers/vecquad.pdf&quot; rel=&quot;nofollow&quot;&gt;Arnold, Boffi, Falk: &lt;em&gt;Quadrilateral $H(div)$ finite elements&lt;/em&gt;, SINUM 42(5), 2005, pp. 2429-2451&lt;/a&gt;, and Chapter 3.2.3 of &lt;a href=&quot;http://www.math.uh.edu/~rohop/fall_03/Downloads/Skript/Chapter3b.pdf&quot; rel=&quot;nofollow&quot;&gt;Ronald Hoppe's lecture notes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;1. As a rule of thumb, a polynomial space of order $k$ on tetrahedral elements contains monomials whose powers &lt;em&gt;sum&lt;/em&gt; to $k$, while a space of order $k$ on quadrilateral elements contains monomials whose &lt;em&gt;maximal&lt;/em&gt; power is $k$. For example, $x^2y$ would be of order $3$ on tetrahedrals, but only of order $2$ on quadrilaterals.&#xA;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-06-11T15:19:21.220" LastActivityDate="2016-06-11T15:19:21.220" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="20248" PostTypeId="1" AcceptedAnswerId="20249" CreationDate="2015-07-22T00:07:05.753" Score="3" ViewCount="133" Body="&lt;p&gt;I'm trying to plot the orbit of a compact binary star system where general relativistic effects become important. I'm using post-Newtonian approximation and I want to solve the orbit numerically based on the equation below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;\ddot{\vec{r}}_{i}&#xA;&amp;amp;=&amp;amp; \sum_{j\neq i}^{ } \frac{\mu_{j}(\vec{r}_{j}-\vec{r}_{i})}{r_{ij}^{3}}\left[1-\frac{4}{c^{2}}\sum_{k\neq i}^{ }\frac{\mu_{k}}{r_{ik}}-\frac{1}{c^{2}}\sum_{k\neq j}^{ }\frac{\mu_{k}}{r_{jk}}+\frac{\vec{v}_{i}^{2}}{c^2}+2\frac{\vec{v}_{j}^{2}}{c^2}-\frac{4}{c^2}\vec{v}_{i} \cdot \vec{v}_{j} - \frac{3}{2c^{2}}\left(\frac{(\vec{r}_{i}-\vec{v}_{j})\cdot\vec{v}_{j}}{r_{ij}}\right)^{2}+\frac{1}{2c^{2}}(\vec{r}_{j}-\vec{r}_{i})\cdot\ddot{\vec{r}}_{j}\right]&#xA;\\&amp;amp;&amp;amp; + \frac{1}{c^2}\sum_{j\neq i}^{ }\frac{\mu_{j}}{r_{ij}^{3}}\left[(\vec{r}_{i}-\vec{r}_{j})\cdot(3\vec{v}_{i}-3\vec{v}_{j})\right](\vec{v}_{i}-\vec{v}_{j}) + \frac{7}{2c^{2}}\sum_{j\neq i}^{ }\frac{\mu_{j}\ddot{\vec{r}}_{j}}{r_{ij}},&#xA;\end{aligned} $$&#xA; where $\mu_{i}=Gm_{i}$ and $G$ is gravitational constant, $\vec{r}_{i}$,$\vec{v}_{i}$ and $\ddot{\vec{r}}_{i}$ are the position vector, velocity vector and acceleration vector of star $i$, and $r_{ij} = |\vec{r}_{i}-\vec{r}_{j}|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this is listed as Eq. 1 in this paper: &lt;a href=&quot;http://arxiv.org/pdf/0709.1160v2.pdf&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/pdf/0709.1160v2.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see on the right hand side of this equation there is $\ddot{\vec{r}}_{j}$ term which makes finding $\ddot{\vec{r}}_{i}$ not straightforward. There are two ways I could think of: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In the code, just plug in the value of $\ddot{\vec{r}}_{j}$ that was obtained at one time-step ago, but I assume this would give large errors after a long simulation time.&lt;/li&gt;&#xA;&lt;li&gt;Use only Newtonian part as the value of $\ddot{\vec{r}}_{j}$ to compute $\ddot{\vec{r}}_{i}$ but this gave me a very messy plot, and to improve it I used the value of $\ddot{\vec{r}}_{i}$ that I just calculated to be the new value of $\ddot{\vec{r}}_{j}$ and use this to obtain a new $\ddot{\vec{r}}_{i}$, although the plot I got is indeed better now but it's still far from accurate. Maybe I could do the same thing recursively but I don't know how much accurate it can get and if it's worth the computation time.&lt;/li&gt;&#xA;&lt;li&gt;Convert this to explicit expression of $\ddot{\vec{r}}_{i}$ but this expression is too complicated to do so. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to numerically solve this $\ddot{\vec{r}}_{i}$ correctly? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I posted a similar question in Astronomy but here might be a more suitable place to ask this question)&lt;/p&gt;&#xA;" OwnerUserId="16476" LastEditorUserId="713" LastEditDate="2015-07-22T00:54:45.687" LastActivityDate="2015-07-22T00:54:45.687" Title="Solve implicit ODE numerically in orbit simulation" Tags="|ode|simulation|computational-physics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20249" PostTypeId="2" ParentId="20248" CreationDate="2015-07-22T00:46:26.730" Score="6" Body="&lt;p&gt;&lt;em&gt;(1)&lt;/em&gt; Using the previous value of $\ddot{r}_j$ is like adding an error term to the r.h.s. of your equation of magnitude $\mathit{const}\times(\ddot{r}_j(t+\delta t) - \ddot{r}_j(t))$, meaning your scheme will only be first-order correct, regardless of whether the integration method you use has a higher order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(2)&lt;/em&gt; I believe what you are describing is similar to applying the iteration method to solve the equation in the unknowns $\ddot{r}_i$. But what you're describing differs from how the iteration method would normally be applied because normally you would iterate until convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a very conventional idea described in numerical analysis textbooks in relation to nonlinear root finding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(3)&lt;/em&gt; This is the best approach here. The system of equations is linear in the unknowns $\ddot{r}_i$. If I denote by $\ddot R$ the vector whose components are the components of all $\ddot{r}_1,\ldots,\ddot{r}_n$, what you can do is rewrite the system of equations in the form&#xA;$$ \ddot R = f(r, v) + A(r, v)\ddot R,$$&#xA;where $A(r,v)$ is a simple, but not diagonal (because of the $(r_j-r_i)\cdot \ddot{r}_j$ term), matrix, and then just solve this as a system of linear equations. It only needs to be linear in the unknowns, after all, and you do not need to solve the linear system by paper and pencil, only construct it and hand it to a linear solver (which would use LU/GE, for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the system is linear, the usual methods for solving nonlinear equations like iteration method or Newton's method aren't really necessary. If you do end up using an iterative method, then what you're describing is close to how it's usually done: you iterate until convergence, using either direct iteration, Newton's method, or something similar, and the guess used for the first iteration is something like the value from previous time step. There is already a lot of research literature on this in the context of differential-algebraic equations (DAE).&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-07-22T00:46:26.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20250" PostTypeId="1" AcceptedAnswerId="20317" CreationDate="2015-07-22T02:01:31.603" Score="4" ViewCount="384" Body="&lt;p&gt;I am looking for a relatively simple book/paper that explains the basic Streamline Upwind Petrov Galerkin (SUPG) method for solving the incompressible unsteady Navier-Stokes equations. Most of the references I have managed to find online have been papers that more-or-less assume you already know the method well. I plan on actually implementing my own solver that uses this method, so references that provide detail on how to do this would be helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: Preference given to references that are freely available online.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2015-07-30T08:12:39.783" Title="Looking for reference on Streamline Upwind Petrov Galerkin finite elements for incompressible unsteady Navier-Stokes" Tags="|finite-element|reference-request|navier-stokes|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20252" PostTypeId="1" CreationDate="2015-07-22T03:57:18.623" Score="7" ViewCount="1631" Body="&lt;p&gt;I hope the following question will not be perceived as to vague. I am trying to ask the question directly, without going into too much information regarding my code. I am currently developing a molecular dynamics code for a custom application, as part of my PhD thesis. I have checked over and over again the validity of the force subroutines (both non bonded and bonded) by comparing numerical values directly from lammps. The integration routines were the same as I used previously to reproduce LJ equation of state data, so I am operating on the assumption they are a correct form of the verlet algorithm. However, I still have a sneaking suspicion I am experiencing more energy drift than I should be. For instance, a recent test I just ran is 1 glycine molecule in an 92.1*92.1*92.1 angstrom box. I am using OPLS force field with wolf summation for the electrostatics. I defined the drift as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;drift = (total potential + kinetic - total initial energy)/total initial energy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After 50,000 steps (1.0 fs time step, no shake, just brute force MD), the drift value is upwards of 0.07 (7%). To test whether or not my assumption about the integrator being correct was true, I ran a simulation for 100k iterations with a tilmestep of 0.5 fs and one with 1.0 fs. I have provided the energy drift plots. For these simulations, the optimization flag was set to -O0. The forces are converted internally from kcal/mol/A to amu*A/fs^2 by multiplying dthalf by 0.0004184.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XqtIh.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XqtIh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1OnBT.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1OnBT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that I am running in full double precision, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) How is it possible to determine possible sources of energy drift in the code base? For instance, if I were comparing the same simulation ran with lammps and my own, saw that my drift was greater, how can I go about mechanistically to determine what possible sources of this error are in my lines of code? What tests should I be running? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) What is the rule of thumb for acceptable energy drift? For instance, how can I quantify whether a simulation is following &quot;good&quot; energy drift or &quot;poor&quot; energy drift. Is this metric system size dependent?&lt;/p&gt;&#xA;" OwnerUserId="16889" LastEditorUserId="16889" LastEditDate="2015-07-22T20:07:31.347" LastActivityDate="2015-08-27T06:47:07.977" Title="energy drift in molecular dynamics" Tags="|molecular-dynamics|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20255" PostTypeId="2" ParentId="20243" CreationDate="2015-07-22T08:18:26.080" Score="6" Body="&lt;p&gt;One possibility is to use a combination of ARPACK and ViennaCL:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ARPACK is an eigensolver. It works with a callback interface (you supply a function that computes $Ax$ for a given $x$ and it computes the eigenvalues by invoking this function multiple times). If you want to solve a generalized eigenvalue problem ($ Ax = \lambda Bx$) you need to provide a callback for computing $x \rightarrow Bx$ as well.&lt;/li&gt;&#xA;&lt;li&gt;These callback functions can be implemented in the GPU (and it will work with ARPACK), now the way to implement it will depend on the structure of your matrices $A$ and $B$. You will use a different implementation if your matrix is full, diagonal or sparse. ViennaCL proposes several functions for different matrix structures.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow&quot;&gt;http://www.caam.rice.edu/software/ARPACK/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://viennacl.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;http://viennacl.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides ARPACK, the same method can be applied any eigensolver that has a reverse communication API (i.e. with a callback that computes matrix-vector products), for instance SLEPc (see comments by Daniel Shapero below).&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="3481" LastEditDate="2015-07-27T15:43:39.337" LastActivityDate="2015-07-27T15:43:39.337" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20256" PostTypeId="2" ParentId="20232" CreationDate="2015-07-22T08:30:57.540" Score="5" Body="&lt;p&gt;If your element is quadratic,  you can find an implicit equation of the three quadratic edges of the triangle in function of $x,y$, in the form $E_1(x,y)=0$, $E_2(x,y)=0$, $E_3(x,y)=0$ (more references below about how to do that). The three function $E_1$, $E_2$ and $E_3$ measure an (oriented) distance to the curved edges, and the points inside the quadratic triangle are characterized by $E_1(x,y) \ge 0$ and $E_2(x,y) \ge 0$ and $E_3(x,y) \ge 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to find the implicit form from the Bézier control mesh is explained in Section 4.4 of [1] (it is used in computer graphics to display quadratic curves on graphic processor). Now if you have a large number of quadratic triangles (a mesh) and want to find which one contains a particular point, the same paper proposes a space decomposition data structure that dramatically speeds-up things (Section 5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another reference on the &quot;implicitization&quot; procedure [2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this technique is interesting if you have many point queries to do (else the preprocessing phase / transform may be prohibitive)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://research.microsoft.com/en-us/um/people/hoppe/proj/ravg/&quot; rel=&quot;nofollow&quot;&gt;http://research.microsoft.com/en-us/um/people/hoppe/proj/ravg/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] GOLDMAN R., SEDERBERG T., AND ANDERSON D. 1984. Vector&#xA;elimination: A technique for the implicitization, inversion, and&#xA;intersection of planar parametric rational polynomial curves.&#xA;CAGD 1, 327-356.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2015-11-18T15:50:45.637" LastActivityDate="2015-11-18T15:50:45.637" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20257" PostTypeId="1" AcceptedAnswerId="20264" CreationDate="2015-07-22T15:52:22.593" Score="2" ViewCount="588" Body="&lt;p&gt;\begin{equation}&#xA;\frac{\partial C_i}{\partial t} = D_i \nabla^2 C_i - \frac{I \cdot \nabla t_i}{z_i F} - \sum_{i'} \frac{z_{i'}}{z_i} D_{i'}\nabla \cdot (t_i\nabla C_{i'})&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\nabla \eta = \frac{I}{\kappa} + \frac{F}{\kappa} \sum_i z_i D_i \nabla C_i&#xA;\end{equation}&#xA;The dependent variables are $C_i$ and $\eta$ ($t_i$ is a function of $C_i$). In the 1D case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some thought on difference scheme (TRBDF2) to try on the first equation. The second equation looks simple, but I'm afraid a simple centered difference will not work. I've try something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left. D_i\frac{\partial C_i}{\partial x}\right \vert_{k-1/2} = D_{i, k-1/2}\frac{C_{i,k}-C_{i,k-1}}{\Delta x} \approx \frac{D_{i, k}+D_{i,k-1}}{2}\frac{C_{i,k}-C_{i,k-1}}{\Delta x}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but with no luck when implementing in the nonlinear solver MINPACK. What will be a good scheme on differencing this two equations? Also, should I run analysis on stability issue every time I came up with a difference scheme before I compute them? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another set of equation look like this, where the dependent variables are $C_i$, $\eta$, $i_2$, $\epsilon$ and $\epsilon_k$. I've try difference scheme and put on some efforts but in vain. Is there any general guideline as to the way of discretization? I'm using the method of line by the way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial \epsilon C_i}{\partial t} = \nabla \cdot \biggl( \epsilon D_i \nabla C_i \biggr) t-\frac{i_2\cdot \nabla t_i}{z_i F} - \sum_j ai_j\biggl( \frac{t_i}{z_i F}+\frac{s_{i,j}}{n_j F} \biggr) - \sum_{i'} \frac{z_{i'}}{z_i}\epsilon D_{i'}\nabla \cdot (t_i\nabla C_{i'}) - R_i&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\nabla \eta = -\biggl( \frac{I-i_2}{\sigma} \biggr) + \frac{i_2}{\epsilon \kappa} + \frac{F}{\kappa} \sum_i z_i D_i \nabla C_i&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\nabla \cdot i_2 = a \sum_j i_j&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial \epsilon}{\partial t} = -\sum_k \overset{\sim}{V_k} k_k\epsilon_k\biggl(\prod_i C_i^{\gamma_i,k}-K_{sp,k}\biggr)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\partial \epsilon_k}{\partial t} = \overset{\sim}{V_k} k_k\epsilon_k\biggl(\prod_i C_i^{\gamma_i,k}-K_{sp,k}\biggr)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;To be more specific I'm now trying to solve the initial condition for $\eta$ and $i_2$ which is not given. Since there is no concentration gradient, in the separator (of the battery)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\nabla \eta = \frac{I}{\kappa}&#xA;\end{equation}&#xA;\begin{equation}&#xA;i_2=I&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the porous cathode&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\nabla \eta = -\biggl( \frac{I-i_2}{\sigma} \biggr) + \frac{i_2}{\epsilon \kappa}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\nabla \cdot i_2 = a \sum_j i_j&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$i_j$ is Butler-Volmer equation, which in the Newton's method tends to explode when initial guess is bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;i_j = i_{o,j}\biggl[ \prod_i \biggl(\frac{C_i}{C_{i,o}}\biggr)^{p_{i,j}}\exp\biggl(\frac{\alpha_{aj} F}{RT}(\eta_j-U_j)\biggr) - \prod_i \biggl(\frac{C_i}{C_{i,o}}\biggr)^{q_{i,j}}\exp\biggl(-\frac{\alpha_{cj} F}{RT}(\eta_j-U_j)\biggr) \biggr]&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I discretize the spatial derivative with second order difference, and the output shows a jump every other mesh point (I did not encounter this when I wrote my own code with Newton's method and analytic Jacobian, but I am afraid later on I will make another error when writing down analytic Jacobian since there are so many variables. So I try the MINPACK with numerical Jacobian but with no luck). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I felt like something is wrong in the boundary. Since $\eta$ is not continuous in the separator/porous cathode interface, so I imposed $\nabla \eta_s = \nabla \eta_c$. This is correct because $\eta=\phi_1-\phi_2$, and I assumed $\phi_2$ is continuous in the interface, $\phi_1$ does not exist in the separator but $\nabla \phi_1=0$ in the interface. Particularly in the interface mesh points I wrote &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{-3\eta_{s,n}+4\eta_{s,n-1}-\eta_{s,n-2}}{2\Delta x} = \frac{3\eta_{c,1}-4\eta_{c,2}+\eta_{c,3}}{2\Delta x}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16998" LastEditorUserId="16998" LastEditDate="2015-07-23T03:58:09.547" LastActivityDate="2015-07-23T13:48:40.510" Title="Finite difference for nonlinear system of equation" Tags="|pde|finite-difference|discretization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20258" PostTypeId="1" AcceptedAnswerId="20259" CreationDate="2015-07-22T17:06:34.370" Score="1" ViewCount="570" Body="&lt;p&gt;As per the suggestion by &lt;a href=&quot;https://scicomp.stackexchange.com/users/1804/christian-clason&quot;&gt;Christian&lt;/a&gt; in the comments &lt;a href=&quot;https://scicomp.stackexchange.com/questions/20245/raviart-thomas-elements-on-reference-square/20246#20246&quot;&gt;here&lt;/a&gt;, as part of my continuing quest to understand the Raviart-Thomas (RT) elements I'd like to know how exactly the RT elements are defined globally, and in particular how they have compact support. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For RT0 on the reference square, one of the basis functions is $\mathbf{\phi}(\mathbf{x}) = \frac{1}{4}\langle 1 + x, 0\rangle^T$. This function is only dependent on $x$ so it is non-zero over all elements above and below the reference square. Since the RT are $H$(div) conforming, I suppose there is no need to enforce continuity in the solution, or the basis functions. As I understand it, this means that we could simply set $\mathbf{\phi}$ to be $\mathbf{0}$ outside some domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a concrete example, given the edge numbering below, (I assume there is one basis functions per edge for RK0, but this may be wrong) what basis functions are non-zero over the middle element (A)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7Z4Hm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7Z4Hm.png&quot; alt=&quot;RT_global&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a separate question, for Langrangian elements of order $k$ we choose the finite dimensional subset of $H^1$ to be the set of all piece-wise continuous polynomials of order $k$. For the RT elements of order $k$ we take the subspace of $H$(div):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P_{k+1,k} \times P_{k,k+1}$$&#xA;as defined in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/20245/raviart-thomas-elements-on-reference-square/20246#20246&quot;&gt;the answer to my last question&lt;/a&gt;. Does this space have a name?&lt;/p&gt;&#xA;" OwnerUserId="9728" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-07-22T18:22:43.863" Title="Raviart-Thomas elements global definition and compact support" Tags="|pde|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20259" PostTypeId="2" ParentId="20258" CreationDate="2015-07-22T18:22:43.863" Score="1" Body="&lt;p&gt;In your example, the basis functions corresponding to the four edges 9,12,13,16. (For $k=1$, it would be &lt;em&gt;two&lt;/em&gt; basis functions per edge, plus the four basis functions corresponding to the interior degrees of freedom (DOF).)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is that the normal trace of an interpolant from the &lt;em&gt;local&lt;/em&gt; polynomial space over one of the edges is uniquely determined by the corresponding edge DOFs (this is part of the reason the local polynomial space is defined the way it is). Since the &lt;em&gt;global&lt;/em&gt; Raviart-Thomas interpolant should be $H(div)$ conforming, it must have continuous normal trace across this edge, and hence you cannot choose these DOFs differently for the adjoining elements (otherwise, you'd introduce a discontinuity). This means that the corresponding basis functions cannot have different coefficients, so they cannot be linearly independent and hence not really separate basis functions. You therefore have to &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;identify &lt;em&gt;local&lt;/em&gt; DOFs for the same edge with one &lt;em&gt;global&lt;/em&gt; degree of freedom (for $k&amp;gt;0$, you of course still have to keep the moments separate, i.e., identify the first DOF of element $K_1$ with the first DOF of element $K_2$ etc.) and&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;glue together&quot; all corresponding &lt;em&gt;local&lt;/em&gt; basis functions to a single &lt;em&gt;global&lt;/em&gt; basis function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(By the way, this is exactly the same way you'd work with $H^1$ conforming elements; in particular, for the implementation you only need to do 1.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To your separate question: The &lt;em&gt;local&lt;/em&gt; Raviart-Thomas space of degree $k$ is usually just called $RT_k(K)$ (with $K$ usually denoting the reference element). The &lt;em&gt;global&lt;/em&gt; Raviart-Thomas space&#xA;$$\{v\in H(div):v|_K \in RT_k(K) \text{ for all }K\in \mathcal{T}_h\}$$&#xA; over a tesselation $\cal{T}_h$ of your domain is usually just denoted by $RT_k$ (or $RT_k(\mathcal{T}_h)$, if this should be emphasized).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-07-22T18:22:43.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20260" PostTypeId="1" CreationDate="2015-07-22T22:05:44.957" Score="5" ViewCount="105" Body="&lt;p&gt;I run many PBS scripts that take advantage of the job array structure for similar jobs (e.g. 12345[0] through 12345[9]). With the way that the code is written, job [0] needs to run first, and then any other job can start. Preferably, the jobs would run in numerically increasing order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I submit an array of jobs, they all have the same priority and queue time, but are listed in a random order. I don't have the permissions to change the priority once a job is submitted (AFAIK), so I am looking for a solution during the job submission.&lt;/p&gt;&#xA;" OwnerUserId="17015" LastActivityDate="2015-07-22T22:07:23.227" Title="Is there a way to ensure that PBS array jobs will be run in order?" Tags="|hpc|pbs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20261" PostTypeId="2" ParentId="20260" CreationDate="2015-07-22T22:07:23.227" Score="4" Body="&lt;p&gt;If you need a guarantee, submit element zero as a separate job and make 1-9 dependent on its completion.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-07-22T22:07:23.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20263" PostTypeId="2" ParentId="12862" CreationDate="2015-07-23T00:50:30.500" Score="0" Body="&lt;p&gt;Depending on how much flexibility you need, a high level kernel crafting tool, such as Blurate, might work too. That way you won't have to deal with SDKs&lt;/p&gt;&#xA;" OwnerUserId="17017" LastActivityDate="2015-07-23T00:50:30.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20264" PostTypeId="2" ParentId="20257" CreationDate="2015-07-23T02:49:05.017" Score="2" Body="&lt;p&gt;I'm guessing based on the notation that the application is electrolyte-related (batteries?), because the equations look like Nernst-Planck. Regarding the last system of equations: you're probably going to have to clarify your notation here. The variable $\eta$ looks like a potential, but I don't see how to close the system of equations you have. Usually, you're looking to solve for concentrations and electric potential. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have some thought on difference scheme (TRBDF2) to try on the first equation...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;TRBDF is usually used for time-stepping discretization. What you have depicted is a spatial discretization; don't use TRBDF for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, these equations have only diffusive terms (that is, things that look like scaled second-order derivatives), in which case, any stencil that typically works for second-order derivatives is likely to work here. A collaborator of mine uses second-order centered differences for your spatial derivatives, which I would recommend to start unless you need high accuracy.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...but with no luck when implementing in the nonlinear solver MINPACK.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's not clear to me what system of equations you are solving in MINPACK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would do here is semi-discretize your PDE system by choosing a discretization in space. The result will be a set of ODEs, which can then be solved using standard software. Since your system of equations is diffusive in nature, choice of time-step and scheme is crucial for avoiding instabilities and achieving accurate results. You will want to choose an L-stable scheme to avoid ringing effects. (See Hairer and Wanner, Vol. 2 for details.) For example, TRBDF2 is an L-stable scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice for your problem, an almost L-stable scheme might be good enough, and an A-stable scheme that lacks L-stability might also work if the spatial discretization is not too fine. (L-stability is really best here.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of software, I would use an ODE solver such as the schemes built into PETSc or Trilinos, or maybe use the implicit BDF methods in CVODE in SUNDIALS (or something else implicit, maybe an implicit Rosenbrock method like RODAS3). Using MINPACK will require you to implement your own time-stepping discretization and error control, which I think is usually a waste of effort, given the availability of good libraries that implement state-of-the-art versions of these features (and are well-tested!).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...is [sic] Butler-Volmer equation, which in the Newton's method tends to explode when initial guess is bad.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, the Butler-Volmer equation has a phenomenally ill-conditioned Jacobian matrix.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I discretize the spatial derivative with second order difference, and the output shows a jump every other mesh point (I did not encounter this when I wrote my own code with Newton's method and analytic Jacobian, but I am afraid later on I will make another error when writing down analytic Jacobian since there are so many variables. So I try the MINPACK with numerical Jacobian but with no luck).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Don't use the numerical Jacobian. The numerical error in finite differencing is going to give you garbage. Use an analytical Jacobian. If you're afraid of making an error, derive it using a computer algebra system (e.g., Maple, Mathematica, SymPy, SageMath...).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since $\eta$ is not continuous in the separator/porous cathode interface, so I imposed $\nabla\eta_{s} = \nabla_{c}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This statement is incorrect, because: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;these quantities are both vectors (except in the 1-D case, in which case it's just misleading)&lt;/li&gt;&#xA;&lt;li&gt;these should be current fluxes, and thus have the wrong units (so they should probably look something like $-\kappa_{s}\nabla\eta_{s} \cdot n$ instead of $\nabla\eta_{s}$, where $\kappa_{s}$ is the electrical conductivity of the separator, and $n$ is the interface outward normal)&lt;/li&gt;&#xA;&lt;li&gt;you really need to explain your notation because most readers aren't going to be familiar with it, and even though I am, I'm largely just guessing based on experience -- people use different conventions all the time&lt;/li&gt;&#xA;&lt;li&gt;assuming phase 1 is the solid phase and phase 2 is the liquid phase, and their electrical potentials are $\phi_{1}$ and $\phi_{2}$, then $\phi_{2}$ is going to be continuous across the separator-electrode interface (doesn't matter which one), but $\phi_{1}$ is not, and experiences a zero flux condition across the interface ($\nabla\phi_{1} \cdot n = 0$, where $n$ is the interface outward normal).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This combination of incorrect boundary conditions and general confusion probably contributes to your weird results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your comments:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think PETSc is too much for me in the meantime. I am wondering which Fortran ode solver (subroutine) is the best alternative. Besides RODAS (Rosenbrock method), there are also RADAU5 (IRK) and the ones in netlib.org/ode (mostly BDF), and also DVODE which you don't recommend. Which is the most stable and efficient one?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I still recommend using PETSc. You are not going to know which method is most efficient without experimenting; PETSc makes it easy to experiment. Provided you use the library according to the documentation, changing numerical methods should be as easy as changing flags at the command line, plus PETSc includes features that make it easier to diagnose numerical problems (command-line switches that output residuals and performance information). Assuming you implement everything correctly, you should also be able to get decent parallel performance, since PETSc is portable and includes data structures you can use to write your program to take advantage of distributed-memory parallelism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use any of those other libraries, most of them are written without any parallelism features at all (certainly RODAS and RADAU, probably also DVODE and LSODE; the only one I can think of that includes any parallel features would be FCVODE, which you are also avoiding). Thus, you will be limited in the size and dimension of problem you can solve, which will likely limit your science as well. Other developer efficiencies are also an issue. Both PETSc and Trilinos make it easy to experiment with different numerical methods through use of things like command-line switches. If, on the other hand, you want to switch time-steppers from RODAS to DVODE, you are likely to have to write whole new drivers, which takes time, and is more difficult to manage. Many of those old Fortran codes do not include any options for iterative linear solvers or preconditioners; again, lack of such features in these old codes will limit the size and dimension of problem you can solve, which again, limits your science.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are problems for which RODAS, or RADAU, or other old Fortran codes are certainly appropriate (even if these libraries are dated); &lt;strong&gt;however, for this particular problem, I strongly recommend you use a framework like PETSc (or Trilinos), since it will not limit your work in the long run in a way that those other solvers will&lt;/strong&gt;. In the case of PETSc, again: using Fortran (or Trilinos) is not an issue. C (or in the case of Trilinos, C++) is probably a better choice -- I would argue that knowing how to write C (respectively, C++) programs will make you more employable -- but you can use PETSc's (respectively, Trilinos') Fortran wrappers without knowing C (respectively, C++) , and PETSc (respectively, Trilinos) includes Fortran examples.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-07-23T13:48:40.510" LastActivityDate="2015-07-23T13:48:40.510" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="20265" PostTypeId="1" CreationDate="2015-07-23T09:29:01.020" Score="3" ViewCount="2556" Body="&lt;p&gt;I'm using &lt;code&gt;scipy.odeint&lt;/code&gt; to solve Fisher-Kolmogorov equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_t = u_{xx}+u(1-u)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code can be found &lt;a href=&quot;https://gist.github.com/ilciavo/b112168ff3af009156db&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From Ablowitz and Zeppetella we know that the analytical solution reads:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u(x,t)=\frac{1}{1+e^{{(-\frac{5}{6} t+ \frac{\sqrt{6}x}{6}})^2}}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if we compare the analytical solution to the numerical solution obtained with ODE integrator we get the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E2gfN.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E2gfN.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Red: Analytical, Blue: OdeInt&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default &lt;code&gt;scipy.odeint&lt;/code&gt; uses Dirichlet boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What kind of boundary conditions should I use for odeInt to compare to the analytical solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) How should I include these boundary conditions using scipy.odeint?&lt;/p&gt;&#xA;" OwnerUserId="11656" LastActivityDate="2015-07-24T00:02:39.710" Title="Scipy OdeInt solver with Neumann boundary conditions" Tags="|pde|python|boundary-conditions|scipy|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20266" PostTypeId="2" ParentId="20265" CreationDate="2015-07-23T09:41:00.090" Score="2" Body="&lt;p&gt;As for 1), if you want to verify convergence order etc., I'd start with prescribing the value of the exact solution at the boundary as a time-dependent Dirichlet BC. That is, at boundary $x_{\rm left}$, use $u(x_{\rm left},t)$ as Dirichlet value. Be careful to select $t$ correctly to match the time step you are computing. This way, you can verify the convergence order of your interior discretisation without having to worry about the BC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, as for 2), I do not know how to do this in scipy.odeint so I can't help you there.&lt;/p&gt;&#xA;" OwnerUserId="16921" LastActivityDate="2015-07-23T09:41:00.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20267" PostTypeId="1" CreationDate="2015-07-23T14:47:50.980" Score="5" ViewCount="814" Body="&lt;p&gt;Note: I'm trying to implement a Discontinuous Galerkin method, as kind of a way to learn about these things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of now, I've taken the acoustic wave equation $c^2 \nabla \cdot \nabla u(x,t) - \frac{\partial^2 u(x,t)}{\partial t^2} = f(x,t)$, rewritten it in weak form with the test function $v(x,t)$, and used Green's first identity to obtain a flux integral from the laplace operator:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Gamma_K} c^2 v(x,t) \nabla u(x,t) \cdot dS - \int_K c^2 \nabla u(x,t) \cdot \nabla v(x,t) dV - \int_K \frac{\partial^2 u(x,t)}{\partial t^2} v(x,t) dV = \int_K f(x,t) v(x,t) dV$$&#xA; where $K$ is an element of the mesh and $\Gamma_K$ is its edges / faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not super sure how ABC's work, but I believe the reason for using Green's first identity to get the flux integral on $\Gamma_K$ is so that we can replace existing boundary conditions with absorbing boundary conditions on the boundary faces of $\partial \Omega$? If I understand correctly, these absorbing boundary conditions would also be a differential equation that are only imposed on the boundary $\partial \Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: What are some good absorbing boundary conditions (differential equations) I can use for the acoustic wave equation in a 2-dimensional or 3-dimensional rectangular domain? &lt;/p&gt;&#xA;" OwnerUserId="17029" LastEditorUserId="17029" LastEditDate="2015-07-24T14:19:51.950" LastActivityDate="2016-06-29T20:06:49.330" Title="Absorbing boundary conditions for acoustics in Discontinuous Galerkin" Tags="|finite-element|boundary-conditions|discontinuous-galerkin|wave-propagation|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20268" PostTypeId="1" CreationDate="2015-07-23T15:56:36.123" Score="1" ViewCount="259" Body="&lt;p&gt;I am translating a piece of Fortran code into Python and am testing my code with a certain test case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the results differ with 0.04% compared to the Fortran results. This is a very small difference and I am not sure if I can almost see this as machine zero or if there are mistakes in my Python implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is odd is that all the values differ with exactly 0.04...%  from the Fortran results. &lt;/p&gt;&#xA;" OwnerUserId="9930" LastEditorUserId="9930" LastEditDate="2015-07-23T16:02:01.470" LastActivityDate="2015-07-29T23:29:03.640" Title="Differences in answers between Python and Fortran" Tags="|python|fortran|" AnswerCount="0" CommentCount="12" ClosedDate="2015-12-02T15:57:05.983" ContentLicense="CC BY-SA 3.0" />
  <row Id="20269" PostTypeId="2" ParentId="20265" CreationDate="2015-07-23T21:01:05.860" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;From Ablowitz and Zeppetella we know that the analytical solution reads:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;\begin{equation}&#xA;u(x,t)=\frac{1}{1+e^{{(-\frac{5}{6} t+ \frac{\sqrt{6}x}{6}})^2}}&#xA;\end{equation}&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Usually, analytical solutions require the imposition of boundary conditions. Are there any used in the derivation of this expression? If so, you must use the same boundary conditions when implementing the numerical solution intended to replicate this analytical solution.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By default &lt;code&gt;scipy.odeint&lt;/code&gt; uses Dirichlet boundary conditions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think there might be a misunderstanding here. After semi-discretizing a PDE in space, boundary conditions are enforced by discretization, not by the ODE solver. The ODE solver requires initial conditions, not boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I would like to know the following:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1) What kind of boundary conditions should I use for &lt;code&gt;odeint&lt;/code&gt; to compare to the analytical solution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You should use the same boundary conditions that are used to derive the analytical solution. (See above.)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2) How should I include these boundary conditions using &lt;code&gt;scipy.odeint&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You should include these boundary conditions via your discretization. For examples of how to do so, see LeVeque's finite difference book (&lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;book web site&lt;/a&gt;), Strikwerda's finite difference book, etc. A &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1426/how-to-impose-boundary-conditions-in-finite-difference-methods&quot;&gt;previous SciComp.StackExchange question&lt;/a&gt; covers the basics: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Dirichlet boundary conditions amount to setting the right-hand side of equations denoting boundary points to zero, and setting initial conditions on these equations to match a discrete version of the boundary conditions in the continuum problem.&lt;/li&gt;&#xA;&lt;li&gt;Neumann boundary conditions amount to replacing the ODEs governing degrees of freedom on those boundaries with (possibly) modified finite difference stencils to approximate the derivative. These modified stencils could be one-sided, or they could be the same stencil as on the interior, but using ghost points. You should consult a textbook (such as LeVeque or Strikwerda) for details.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-07-24T00:02:39.710" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20270" PostTypeId="1" AcceptedAnswerId="20273" CreationDate="2015-07-23T23:59:36.713" Score="5" ViewCount="238" Body="&lt;p&gt;Let's say I have a LDL factorization of a matrix A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an efficient $O(n^2)$ way to get the eigen decomposition of A given it's LDL factorization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more efficient way, in case L and A are sparse and we have a fill-reducing permutation of A?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2015-07-24T09:39:40.157" Title="Is there an efficient $O(n^2)$ way to get the eigen decomposition given a LDL factorization?" Tags="|matrix|eigensystem|matrix-factorization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20271" PostTypeId="1" AcceptedAnswerId="20272" CreationDate="2015-07-24T02:23:56.440" Score="0" ViewCount="991" Body="&lt;p&gt;This is a fairly simple question but my Matlab knowledge is still very limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to take a given binary number (or rather, a bistring) of length $mn$ and generate an $m \times n$ matrix whose entries correspond to the bit values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if the given bitstring is $b = 1001110101$ (which is the binary value of the base-10 number 629), and since $b$ has length 10 which can be represented as 2*5, I am looking for a sequence of Matlab commands to convert $b$ to this matrix $A$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A = \begin{pmatrix} &#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 \\ &#xA;1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 1 &#xA;\end{pmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have come close in the past using cell and string commands, but I never actually seem to get the &lt;em&gt;matrix&lt;/em&gt; in the proper sense that I can compute things like $AA'$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it helps, my application program will always generate bitstrings that are of a given length $b = mn$. That is, $m$ and $n$ are known so there is no guesswork when it comes to the size of the desired matrix.&lt;/p&gt;&#xA;" OwnerUserId="10378" LastActivityDate="2015-07-24T07:27:17.893" Title="How to represent a binary number in a matrix in Matlab?" Tags="|matlab|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20272" PostTypeId="2" ParentId="20271" CreationDate="2015-07-24T07:27:17.893" Score="1" Body="&lt;p&gt;There is a MATLAB function in the Communications Toolbox called &lt;a href=&quot;http://www.mathworks.com/help/comm/ref/de2bi.html?searchHighlight=de2bi&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;de2bi&lt;/code&gt;&lt;/a&gt; that will convert a decimal number to a binary vector. Once you get the binary vector, you can &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/reshape.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;reshape&lt;/code&gt;&lt;/a&gt; it. The &lt;code&gt;reshape&lt;/code&gt; command preserves columnwise ordering, so you might need to transpose the dimensions (i.e., generate an $n$ by $m$ matrix) and then transpose the resulting reshaped matrix (via &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/transpose.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;transpose&lt;/code&gt;&lt;/a&gt;) operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have the Communications Toolbox, you could probably emulate &lt;code&gt;de2bi&lt;/code&gt; using built-in commands. If &lt;code&gt;x&lt;/code&gt; is your decimal number, then &lt;code&gt;b = num2str(dec2bin(x.))-'0'&lt;/code&gt; will probably work, following this &lt;a href=&quot;http://www.mathworks.com/matlabcentral/newsreader/view_thread/246172&quot; rel=&quot;nofollow&quot;&gt;MathWorks Central post&lt;/a&gt;. (I haven't tested this function, however, because I don't have a personal MATLAB license.)&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-07-24T07:27:17.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20273" PostTypeId="2" ParentId="20270" CreationDate="2015-07-24T09:39:40.157" Score="4" Body="&lt;p&gt;It's unlikely. Demmel and co-authors have a paper called &lt;a href=&quot;http://arxiv.org/pdf/math/0612264v3.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;Fast Linear Algebra is Stable&quot;&lt;/a&gt; that shows that numerically stable algorithms for eigendecompositions (up to finite precision) cost at least as much as matrix multiplication, which is $O(n^{\omega})$, where $2 \leq \omega \leq 2.376$ (or so). Methods for transforming an LDL or Cholesky decomposition into an eigendecomposition typically use a polar decomposition (Newton methods for calculating this would be $O(n^{\omega})$) or other means of computing similar information (e.g., SVD, $O(n^{3})$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think sparsity helps that much here either. Iterative methods (Krylov subspace methods like Lanczos or Arnoldi) will be $O(n^{3})$ (that is, $O(n^{2})$ per eigenvalue). I don't know of any sparse direct methods for the polar decomposition. Sparse SVD exists, which will offer some savings over the dense algorithm, but I don't know that a result exists that states it will be $O(n^{2})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optimization algorithms will use sparse Cholesky, LDL, or LU. The complexity of sparse Cholesky is $O(nnz(L))$, where $nnz$ is the number of nonzeros function, and for sparse LU, it is $\Omega(nnz(L) + nnz(U))$ and $O(n^{\omega})$. However, these algorithms will perform a factorization for each linear solve, which is at least once per iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That exhausts all I can think of regarding algorithmic complexity on this topic. Quadratic complexity algorithms are unlikely in the dense case, and also probably unlikely in the sparse case.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-07-24T09:39:40.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20274" PostTypeId="2" ParentId="5101" CreationDate="2015-07-24T09:42:50.470" Score="3" Body="&lt;p&gt;Latest version of CHOLMOD SuiteSparse package (beta 4.4.5) supports modifying a symmetric row/column (rank2 update) for &lt;span class=&quot;math-container&quot;&gt;$LDL^T$&lt;/span&gt; decomposition, using a matlab (and C) API. I used it successfully in one of my projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use it to make &lt;span class=&quot;math-container&quot;&gt;$nnz(G)$&lt;/span&gt; updates to the factorization. It is based on &lt;a href=&quot;http://users.clas.ufl.edu/hager/papers/LPDASA/rowmod.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the complexity will be &lt;span class=&quot;math-container&quot;&gt;$O(nnz(G)*nnz(L))$&lt;/span&gt;. Where &lt;span class=&quot;math-container&quot;&gt;$nnz(L)$&lt;/span&gt; can be significantly reduced when using a &lt;em&gt;fill reducing permutation&lt;/em&gt; for a sparse &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The package can be downloaded from &lt;a href=&quot;http://faculty.cse.tamu.edu/davis/SuiteSparse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are some notes the package owner gave (Prof. Tim Davis):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;API:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;LD = ldlrowmod (LD,k) deletes row/column k, by setting A(:,k) and&#xA;  A(k,:) to the kth row/col of identity. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;LD = ldlrowmod (LD,k,C)&#xA;  replaces the kth row/col of A (which must be the kth row/col of&#xA;  identity) with the sparse column C.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Complexity:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The row add/delete takes at most &lt;span class=&quot;math-container&quot;&gt;$O(nnz(L))$&lt;/span&gt; time, so if &lt;span class=&quot;math-container&quot;&gt;$nnz(L)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;,&#xA;  then the time is at most &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Fill reducing permutation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Rarely is it a good idea to factorize a user's matrix, as in &lt;span class=&quot;math-container&quot;&gt;$LDL^T$&lt;/span&gt; =&#xA;  A. Rather, we permute to &lt;span class=&quot;math-container&quot;&gt;$LDL^T$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$PAP^T$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; has vastly fewer&#xA;  nonzeros.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16909" LastEditorUserId="16909" LastEditDate="2019-12-04T11:16:00.450" LastActivityDate="2019-12-04T11:16:00.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20275" PostTypeId="1" CreationDate="2015-07-24T12:31:31.583" Score="2" ViewCount="85" Body="&lt;p&gt;I was wondering if there is a Fortran library that contains a solver for the Sparse LSE(linear equality-constrained least squares) problem&#xA;$$&#xA;min_{x}\|Cx-d\|^2  \text{ subject to }   Ax=b&#xA;$$&#xA;where $A$ and $C$ are large and sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For dense linear systems, LAPACK has the subroutine &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d0/d85/dgglse_8f.html#a1f03c8580ac1aaafb0f092fad4377d16&quot; rel=&quot;nofollow&quot;&gt;DGGLSE&lt;/a&gt;, but I am looking for a solver for sparse systems.&lt;/p&gt;&#xA;" OwnerUserId="17039" LastActivityDate="2015-07-24T12:31:31.583" Title="Fortran solver for the Sparse LSE problem" Tags="|sparse-matrix|fortran|constrained-optimization|least-squares|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20277" PostTypeId="1" CreationDate="2015-07-24T13:34:19.477" Score="28" ViewCount="4619" Body="&lt;p&gt;I am looking for books or articles, or blog-posts, or any published material in general, that address specifically the uses of C++ modern features (move semantics, the STL, iterators, lazy evaluation, etc.) in scientific computing. Can you suggest any?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that these new features will make it easier to write efficient code, but I haven't found real examples. Most references I've read are about generic uses of C++, and do not contain examples of scientific computing. So I am looking for examples (do not have to be production code examples, just pedagogical examples, at the level of, say, Numerical Recipes) of scientific computing code using C++ modern features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I am not asking about libraries that use these features. I am asking about articles/books/etc explaining how I can exploit these features in scientific computing.&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2015-07-24T19:24:44.187" LastActivityDate="2020-05-13T07:29:14.863" Title="Modern C++ in scientific computing?" Tags="|c++|reference-request|" AnswerCount="9" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20278" PostTypeId="2" ParentId="20277" CreationDate="2015-07-24T13:57:36.480" Score="10" Body="&lt;p&gt;I would suggest taking a look at Deal.II.  It uses the STL, it's own iterators, shared pointers, etc.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The various linear solvers can use the various matrices because of how it was designed.  I haven't come across any use of move semantics, but that doesn't mean they aren't there.  Here is a &lt;a href=&quot;http://dealii.org&quot;&gt;link.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16799" LastActivityDate="2015-07-24T13:57:36.480" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20279" PostTypeId="1" AcceptedAnswerId="20354" CreationDate="2015-07-24T16:51:45.787" Score="5" ViewCount="2638" Body="&lt;p&gt;The original equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{c^2} \frac{\partial^2 u}{\partial t^2} = \frac{\partial^2 u}{\partial r^2} + \frac{1}{r}\frac{\partial u}{\partial r} + \frac{1}{r^2}\frac{\partial^2 u}{\partial \phi^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the finite difference version used is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U^t_{i,j} = {c^2 \Delta t^2}\left[\frac{1}{2r_i\Delta r }(U^{t-1}_{i+1,j} - U^{t-1}_{i-1,j}) + \frac{1}{\Delta r^2}(U^{t-1}_{i-1,j} + U^{t-1}_{i+1,j} - 2U^{t-1}_{i,j}) + \frac{1}{r_i^2 \Delta \phi^2}{}(U^{t-1}_{i,j-1} + U^{t-1}_{i,j+1} - 2U^{t-1}_{i,j})\right] - U^{t-2}_{i,j} + 2U^{t-1}_{i,j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $c^2=0.25$, $\Delta r=0.1$, $\Delta \phi=0.1256$, $\Delta t=0.05$. The total radius is 5.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R = np.linespace(0, radius, 50)&#xA;phi = np.linspace(0, 2*np.pi, 50)&#xA;R, phi = np.meshgrid(R, phi)&#xA;x = R*np.cos(phi)&#xA;y = R*np.sin(phi)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;thus &lt;code&gt;dr=5./50=0.1&lt;/code&gt;, and &lt;code&gt;dPhi=2*np.pi/50=0.1256&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial conditions looks like&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0E1eP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0E1eP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I create &lt;code&gt;r[50][50]&lt;/code&gt;, &lt;code&gt;phi[50][50]&lt;/code&gt;, and the same space for &lt;code&gt;x&lt;/code&gt;, &lt;code&gt;y&lt;/code&gt; ($x=r\cos(\phi)$, $y=r\sin(\phi)$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know why, but I get strange result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I made a model for a Cartesian system, and &lt;a href=&quot;https://youtu.be/Okb1oDcL1Lc&quot; rel=&quot;nofollow noreferrer&quot;&gt;it works&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17041" LastEditorUserId="9667" LastEditDate="2016-01-28T22:01:37.437" LastActivityDate="2017-03-04T17:27:42.050" Title="How can I solve wave equation for circular membrane in polar coordinates?" Tags="|finite-difference|python|numerical-analysis|wave-propagation|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="20280" PostTypeId="2" ParentId="20277" CreationDate="2015-07-24T19:17:12.270" Score="7" Body="&lt;p&gt;The &lt;a href=&quot;http://stellar-group.org/libraries/hpx/&quot;&gt;HPX&lt;/a&gt; library makes heavy use of a range of C++11 features such as move constructors and is also aiming to be a complete implementation of &lt;a href=&quot;http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4409.pdf&quot;&gt;N4409&lt;/a&gt; (Working Draft, Technical Specification for C++ Extensions for Parallelism).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have a list of &lt;a href=&quot;http://stellar-group.org/publications/&quot;&gt;publications&lt;/a&gt; on their site which includes a number of example of using the library to speed up scientific computation. There is also so interesting discussion of the library and it's use of modern C++ in this &lt;a href=&quot;http://cppcast.com/2015/04/hartmut-kaiser/&quot;&gt;CppCast episode&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17043" LastEditorUserId="17043" LastEditDate="2015-07-25T06:32:06.307" LastActivityDate="2015-07-25T06:32:06.307" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20281" PostTypeId="1" CreationDate="2015-07-24T22:36:40.613" Score="2" ViewCount="362" Body="&lt;p&gt;I am dealing with following problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\min_{u,\gamma}\Bigg\{ \frac{1}{1000}  \iint_{S_2} {\gamma (x,y)^2 dxdy} + \iint_{S_2} {[u(x,y)  - u_0 (x,y)]^2 dxdy} + \iint_{S_2} {[\Delta u(x,y) - \gamma (x,y) u(x,y)]^2 dxdy}\Bigg\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the unknown functions $u(x, y)$ and $\gamma(x, y)$ are defined on the unit square $S_2$ and the function $u_0(x, y)$ is defined on $S_2$ by $u_0(x, y) = \sin(6\pi x) \sin(2\pi y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem should be discretized using 5-point finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no experience discretizing such problems. Could somebody help me out, or point me to right literature to look for some examples? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: this problem comes from the paper &lt;a href=&quot;http://www.cerfacs.fr/algor/reports/2006/TR_PA_06_32.pdf&quot; rel=&quot;nofollow&quot;&gt;Recursive trust-region methods for multiscale nonlinear optimization&lt;/a&gt; (page 10)&lt;/p&gt;&#xA;" OwnerUserId="17045" LastEditorUserId="17045" LastEditDate="2015-08-07T02:34:30.987" LastActivityDate="2015-08-07T02:34:30.987" Title="Finite difference scheme for solving nonlinear least-squares problem" Tags="|finite-difference|nonlinear-equations|discretization|nonconvex|" AnswerCount="0" CommentCount="17" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20282" PostTypeId="2" ParentId="20277" CreationDate="2015-07-24T23:10:56.540" Score="4" Body="&lt;p&gt;Besides deal.ii (which was already suggested here) you can also have a look at the &lt;a href=&quot;http://dune-project.org&quot; rel=&quot;nofollow&quot;&gt;Dune&lt;/a&gt; library which makes extensive use of some advanced C++ features like template metaprogramming, iterator ranges, smart pointers, and so on. There is also a recent &lt;a href=&quot;http://www.asc.tuwien.ac.at/~schoeberl/wiki/publications/ngs-cpp11.pdf&quot; rel=&quot;nofollow&quot;&gt;preprint&lt;/a&gt; by Joachim Schöberl, which comments on the use of C++ 11 features, like for instance lambda functions, for simplifying the implementation of the finite element methods in NGSolve. &lt;a href=&quot;http://boost.org&quot; rel=&quot;nofollow&quot;&gt;Boost&lt;/a&gt; also has some libraries related to scientific programming like uBLAS, Graph, etc. I guess in most of these libraries you will find good  examples of modern C++ usage. However, be aware that you may encounter also bad examples of using advanced/modern C++. In some cases, while reading the code/documentation, I got the feeling that occasionally things are vastly over-generalized for the sake of showing off advanced skills like TMP, where for 99% of all potential applications a more straightforward implementation would also do the job.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2015-07-24T23:17:32.833" LastActivityDate="2015-07-24T23:17:32.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20284" PostTypeId="2" ParentId="20128" CreationDate="2015-07-25T01:02:52.330" Score="0" Body="&lt;p&gt;I suggest to avoid the  SVD, use Lanczos with Fast Block matrix-vector multiplication via FFTs. You don't need to save the matrix, it is all on the flight. The theory is explained here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A fast reduced-rank interpolation method for prestack seismic volumes that depend on four spatial dimensions,Jianjun Gao1, Mauricio D. Sacchi and Xiaohong Chen, GEOPHYSICS,Volume 78, Issue 1.&#xA;&lt;a href=&quot;http://library.seg.org/doi/abs/10.1190/geo2012-0038.1&quot; rel=&quot;nofollow&quot;&gt;http://library.seg.org/doi/abs/10.1190/geo2012-0038.1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers, Mauricio D Sacchi&lt;/p&gt;&#xA;" OwnerUserId="17047" LastActivityDate="2015-07-25T01:02:52.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20285" PostTypeId="2" ParentId="20267" CreationDate="2015-07-25T07:20:42.420" Score="6" Body="&lt;p&gt;The problem with ABC in the form of derivative operators at the boundary is that the exact ABCs are non-local in space and often in time, which makes them hard to use in numerical simulations. You can derive approximate ABCS based on series expansions, but this can be tedious and it may be difficult to get stable schemes. This approach was made famous by a very widely cited paper by &lt;a href=&quot;http://www.pnas.org/content/74/5/1765.short&quot; rel=&quot;nofollow&quot;&gt;Enquist and Majda&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An easier and widely used approach are &lt;em&gt;perfectly matched layers&lt;/em&gt;. You extend your computational domain $\Omega$ by an additional layer in which you modify your derivatives by a complex scaling factor&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_x \rightarrow \left(1 + \frac{i \sigma(x)}{\omega} \right) \partial_x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This modifies the dispersion relation of your system such that traveling wave modes in the exterior layer are damped.&#xA;Here, $\sigma$ is chosen such that it is zero at the boundary of your physical domain $\partial \Omega$ and increases towards the end of your layer, thus slowly increasing the damping. A function like $\textrm{erf}(x) = \frac{1}{\sqrt{\pi}}\int_{0}^{x} \exp(-s^2) ds$ might work well for $\sigma$, but you will need to rescale it so that it starts at zero at the physical boundary. You may have to play a little with your parameters $\sigma$ and $\omega$ to get good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that PML provides an approximate ABC, unless the layer would be infinitely long, which is of course impossible in a numerical simulation. However, the amplitude of the waves usually decays exponentially, so that small layers can already damp your reflections to below the discretization error of your DG method in the interior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Originally, PML was introduced by &lt;a href=&quot;http://dx.doi.org/10.1006/jcph.1994.1159&quot; rel=&quot;nofollow&quot;&gt;Berenger&lt;/a&gt; although this paper might not be the best way to start. There is also a well written &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfectly_matched_layer&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt; entry and searching for perfectly matched layer certainly will provide you with good material - as said, the method is very widely used.&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="16799" LastEditDate="2016-06-29T20:06:49.330" LastActivityDate="2016-06-29T20:06:49.330" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20287" PostTypeId="2" ParentId="20267" CreationDate="2015-07-25T11:50:17.283" Score="3" Body="&lt;p&gt;There exist Absorbing Boundary Conditions for the wave equation that are stable and that go up to any order of accuracy (limited only by the accuracy of discretization of your model), so that they are good competition for PMLs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A commonly used ABC of the Enquist-Majda type mentioned above, which is of the second order, is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{tt}-u_{xt}-\frac{1}{2}u_{yy}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Derivatives are implied). This should be plugged into the boundary term in your weak form, but you may need to define $u_x$ on the boundary as an additional variable in your model to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need higher accuracy I suggest the &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S016521250300129X&quot; rel=&quot;nofollow&quot;&gt;Hagstrom-Warburton&lt;/a&gt; boundary condition: this condition is based on the definition of auxiliary variables on the boundary, which also satisfy the wave equation there. Corners require special treatment.&lt;/p&gt;&#xA;" OwnerUserId="6775" LastEditorUserId="6775" LastEditDate="2015-07-26T15:15:32.663" LastActivityDate="2015-07-26T15:15:32.663" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20288" PostTypeId="1" CreationDate="2015-07-25T16:19:37.523" Score="3" ViewCount="424" Body="&lt;p&gt;I am facing a simple (at first glance) problem. I need to implement a numerical scheme for the solution of the first order wave propagation equation with chromatic dispersion included. My original problem is (for a forward propagating wave):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{1}{c} \frac{\partial u(x,t)}{\partial t} = -\frac{ \partial u}{ \partial x} - \frac{i \beta_2}{2} \frac{ \partial^2 u}{ \partial t^2},&#xA;\end{equation} &#xA;where $c$ is the velocity of light, $u$ is the (complex) envelope of the field, $\beta_2$ is the 2nd order dispersion coefficient. Assume also that the wave is propagating inside a ring cavity of length , say, L where I take periodic boundary conditions: $u(x+L,t) = u(x,t)$ and also that at $t=0$ we know $u(x,0) $ and $u_t(x,0)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to implement a time-stepping numerical scheme and in the process I tried the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) MOL approach, where I do semidiscretization along $x$, reduce the set of equations to a system of first order ODEs (by setting $v = \dot{u}$) and I establish a system: &#xA;\begin{equation}&#xA;\begin{bmatrix} \dot{v} \\ \dot{u} \end{bmatrix} = A \begin{bmatrix} v \\ u \end{bmatrix} . &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I solve the corresponding ODEs via 4th Runge-Kutta, Crank-Nicholson , or simply precomputing the matrix exponential, unfortunatelly, all my solutions eventually blow up to Inf. I implemented the periodic boundary conditions by modifying the matrix $A$ as $A \leftarrow PA$ , where $P$ is the identity matrix with the first row identical copy of the last row. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried a simple finite differences approach where the spatial derivative is approximated via an upwind FD (first order) but to no avail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly I tried a strang splitting approach based on the two equations: &#xA;\begin{align}&#xA;\frac{1}{c}\dot{u} &amp;amp;= -\frac{\delta u }{\delta x} \\&#xA;\frac{1}{c}\dot{u} &amp;amp;= -\frac{i\beta_2}{2}\frac{\delta^2 u }{\delta t^2} ,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where this time the solution does not blow up but it looks somehow unphysical. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does someone here know a stable and possibly higher than first order time-stepping scheme for this equation? Please, note that a solution based on a Fourier transform in $x$ is also not a good option for me because I would like to have the flexibility to implement different non-periodic boundary conditions. I would also dislike substituting the second order derivative in time with one in space due to the fact that this complicates the implementation of boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.  &lt;/p&gt;&#xA;" OwnerUserId="17049" LastActivityDate="2015-12-09T05:39:58.650" Title="Numerical Solution of the Advection Dispersion equation" Tags="|numerical-analysis|time-integration|advection-diffusion|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20289" PostTypeId="1" CreationDate="2015-07-25T19:56:22.383" Score="2" ViewCount="829" Body="&lt;p&gt;$X_{1} = (A, b)$, where $X_{1}$ is a $n\times p$ matrix, $A$ is a $n\times (p-1)$ and $b$ is $n\times1$. update $b$ with $c$,Is there any update method to compute more efficiently?&lt;/p&gt;&#xA;" OwnerUserId="17052" LastEditorUserId="17052" LastEditDate="2015-09-17T18:15:03.997" LastActivityDate="2015-09-17T18:15:03.997" Title="How to calculate $det(X^TX)$ efficiently, update one column of X each time" Tags="|linear-algebra|matrix|numerics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20290" PostTypeId="2" ParentId="20289" CreationDate="2015-07-25T21:07:13.820" Score="2" Body="&lt;p&gt;Consider the matrix&#xA;$$ \begin{pmatrix}A^TA &amp;amp; x_{1:p-1} \\ w_{1:p-1} &amp;amp; z \end{pmatrix}. $$&#xA;In terms of &lt;a href=&quot;https://en.wikipedia.org/wiki/Minor_(linear_algebra)&quot; rel=&quot;nofollow&quot;&gt;minors&lt;/a&gt; $M_{ij}$ of the matrix $A^TA$ ($M_{ij}$ is the determinant of $A^TA$ with row $i$ and column $j$ removed), its determinant is&#xA;$$ \sum_{i=1}^{p-1} (-1)^{p+i} x_i \sum_{j=1}^{p-1} (-1)^{p-1+j} w_j M_{ij} + z \,\mathrm{det} (A^TA)&#xA;\\ = -\sum_{1\leq i,j\leq p-1} x_i (-1)^{i+j}M_{ij} w_j + z\,\mathrm{det}(A^TA). $$&#xA;(Using the cofactor expansion, applied twice: to the last column and to the last row.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, $x=A^Tb$, $z=b^Tb$ and $w=b^TA$ are known, so the determinant is&#xA;$$ -(b^TA)C(A^Tb) + b^Tb\, \mathrm{det}(A^TA), $$&#xA;where $C = \big( (-1)^{i+j}M_{ij} \big)$ is the cofactor matrix of $A^TA$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To evaluate the cofactor matrix, one can use the matrix inversion formula&#xA;$$ (A^TA)^{-1} = \frac{1}{\mathrm{det}(A^TA)} C^T, $$&#xA;from which it follows that the determinant of $X^TX$ is&#xA;$$ \big(b^Tb - b^TA(A^TA)^{-1}A^Tb\big)\,\mathrm{det}(A^TA). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, it is sufficient to precompute the matrix $(A^TA)^{-1}$ in time $O(p^3 + p^2n)$ using Cholesky factorization of $A^TA$, which also provides the determinant of $A^TA$, and then on each update evaluate one quadratic form in $A^Tb$ in time $O(p^2+pn)$. This is about $p$ times better than $O(p^3+p^2n)$ for computing the determinant directly. I think after this the efficiency would probably have to come from exploiting special structure of $A^T$ and $A^TA$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-07-27T22:21:50.443" LastActivityDate="2015-07-27T22:21:50.443" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20291" PostTypeId="1" CreationDate="2015-07-26T10:10:13.790" Score="2" ViewCount="267" Body="&lt;p&gt;I want so visualize the spatial density for my simulation. The  result of my simulation is a (time-dependent) system of large number (~100k) of moving particles in a confined space. Each particle has its own Cartesian coordinates as well as a weight ($w$) in the form of $(x_i,y_i,z_i;w_i)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a software/app/package to create a snapshot of the 3D spatial density map of these particles (&lt;a href=&quot;http://www.nature.com/ncomms/2013/130906/ncomms3414/fig_tab/ncomms3414_F3.html&quot; rel=&quot;nofollow&quot;&gt;something like this&lt;/a&gt;). Like you see in this figure, the particles are not going to be displayed individually, but only a transparent cloud will be drawn whose local intensity is proportional with the local number of particles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final goal is to create a movie of the changing 3D spatial density with time.  So far, I have tried R, Matlab, Origin, and ImageJ. But no success!&lt;/p&gt;&#xA;" OwnerUserId="15135" LastEditorUserId="9667" LastEditDate="2015-07-28T17:45:47.533" LastActivityDate="2016-01-24T21:48:50.427" Title="Creating a 3D spatial density map from simulation results" Tags="|visualization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20292" PostTypeId="1" CreationDate="2015-07-26T10:29:36.437" Score="3" ViewCount="396" Body="&lt;p&gt;I'm a computer science student. Please I need a help in solving a constrained normalized quadratic function. I'm familiar with solving quadratic constrained optimization function with matlab by providing a symmetric matrix and a vector as inputs for quadprog matlab function. Now, I encountered an other form of quadratic function described as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \min_{0 \leq \alpha \leq C} \left( \frac{1}{2} \frac{\alpha^tB\alpha}{\alpha1_N}+ b^t\alpha \right)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $ b^t=\left[b_1,...,b_N\right] $ is a parameter vector. $B=\left[b_ij\right] $ is a symmetric positive matrix of parameters.&#xA;$ \alpha 1_N = \sum_{i=1}^{N} \alpha_i $. $0 \leq \alpha  \leq C $ mean that  $ \forall i \in {1,...,n}, 0 \leq \alpha_i \leq C $. I don't know how I can deal with such function. Can I rewrite the normalized quadratic function as a quadratic function? Is there any tutorial, any link describing the different mathematical steps for solving such problem in order to make the necessary implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks.&lt;/p&gt;&#xA;" OwnerUserId="17053" LastActivityDate="2015-07-26T17:37:54.197" Title="minimization of normalized constrained quadratic function" Tags="|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20293" PostTypeId="2" ParentId="20292" CreationDate="2015-07-26T17:37:54.197" Score="5" Body="&lt;p&gt;A good starting point for this is Boyd and Vandenberghe's textbook, Convex Optimization.  I'd strongly encourage you to get a copy (the authors have posted a free .pdf online, so it won't cost you anything, and the hardcover edition from Cambridge University Press is quite reasonably priced.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Begin by Cholesky factoring your symmetric and positive definite matrix $B$ as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B=R^{T}R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then write your problem as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \frac{z^{T}z}{y} + b^{T}\alpha$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z=R\alpha$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y=2\alpha^{T}1_{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introduce an auxilliary variable $t$, and write the problem as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min t + b^{T}\alpha$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{z^{T}z}{y} \leq t $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z=R\alpha$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y=2\alpha^{T}1_{N}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0 \leq \alpha \leq C$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, you've got a linear objective function and a bunch of linear equality and inequality constraints in the variables $\alpha$, $z$, $y$, and $t$.  Also, the constraints ensure that $y&amp;gt;0$ and $t \geq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only difficult part is the hyperbolic constraint &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{z^{T}z}{y} \leq t$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $y&amp;gt;0$, this can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z^{T}z \leq ty$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A clever trick from convex optimization (it's an exercise in the Boyd and Vandenberghe book) is that this hyperbolic constraint can be written as a &#xA;second order cone constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left\|&#xA;\left[&#xA;\begin{array}{c}&#xA;2z \\&#xA;y-t \\&#xA;\end{array}&#xA;\right]&#xA;\right\|_{2}&#xA;\leq&#xA;y+t $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, you've got a standard second order cone programming problem that can be solved by a variety of solvers.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CVX package for convex optimization in MATLAB can do this reformulation for you automatically using its &quot;quad_over_lin&quot; function.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-07-26T17:37:54.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20294" PostTypeId="2" ParentId="19902" CreationDate="2015-07-26T19:58:22.977" Score="-1" Body="&lt;p&gt;You can try the Colt library for Java:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dst.lbl.gov/ACSSoftware/colt/&quot; rel=&quot;nofollow&quot;&gt;http://dst.lbl.gov/ACSSoftware/colt/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In spite of having had trouble setting them up, I have been able to run the GNU GMP and MPFR libraries. MPFR has some Bessel functions as well. See the documentation below in the link:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mpfr.org/mpfr-current/#doc&quot; rel=&quot;nofollow&quot;&gt;http://www.mpfr.org/mpfr-current/#doc&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One nice thing about GSL is that you can use functions to get the error of your calculation. If you're calculating functions in Fortran, you might want to look at FGSL, the Fortran version of GSL.&lt;/p&gt;&#xA;" OwnerUserId="17058" LastActivityDate="2015-07-26T19:58:22.977" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20295" PostTypeId="2" ParentId="1007" CreationDate="2015-07-26T20:26:58.247" Score="0" Body="&lt;p&gt;I am told that MPIR is a good library to use if you are working with Visual Studio:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mpir.org/&quot; rel=&quot;nofollow&quot;&gt;http://mpir.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17058" LastActivityDate="2015-07-26T20:26:58.247" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20296" PostTypeId="2" ParentId="19891" CreationDate="2015-07-26T20:59:07.793" Score="1" Body="&lt;p&gt;Look at &lt;a href=&quot;https://github.com/jeffhammond/HPCInfo/blob/master/docs/LAMMPS.mediawiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jeffhammond/HPCInfo/blob/master/docs/LAMMPS.mediawiki&lt;/a&gt; and &lt;a href=&quot;https://github.com/awlange/lammps_ensembles&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/awlange/lammps_ensembles&lt;/a&gt;. We did exactly this for the purpose of umbrella sampling etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interface to PLUMED worked, among other things.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2020-05-24T21:09:56.543" LastActivityDate="2020-05-24T21:09:56.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20298" PostTypeId="2" ParentId="20241" CreationDate="2015-07-27T12:10:44.063" Score="3" Body="&lt;p&gt;One issue (and this is mentioned by Mur in the first paper you linked in the comments above) is the fact that, while these edge functions provide tangential field continuity across interfaces and zero divergence within each element, they also allow for discontinuities in normal fields across interfaces. This behavior is non-physical, giving rise to artificial surface charges which produce errors in finite element solutions.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2015-07-27T12:10:44.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20300" PostTypeId="2" ParentId="20277" CreationDate="2015-07-27T13:56:27.040" Score="13" Body="&lt;p&gt;&lt;strong&gt;Two examples of libraries that use modern C++ constructs:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Both the eigen and armadillo libraries (linear algebra) use several modern C++ constructs. For instance, they use both expression templates to simplify arithmetic expressions and can sometimes eliminate some temporaries: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://eigen.tuxfamily.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eigen.tuxfamily.org&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arma.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://arma.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hpac.rwth-aachen.de/teaching/sem-accg-14/Armadillo.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hpac.rwth-aachen.de/teaching/sem-accg-14/Armadillo.pdf&lt;/a&gt; (presentation on expression templates in Armadillo)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The CGAL library (computational geometry) uses many modern C++ features (it heavily uses templates and specializations):&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cgal.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cgal.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;modern C++ constructs are very elegant and can be very fun to use. It is both a strong point and a weakness: when using them, it is so tempting to add several layers of templates / specializations / lambdas that in the end you sometimes get more &quot;administration&quot; than effective code in the program (in other words, your program &quot;talks&quot; more about the problem than describing the solution). Finding the right balance is very subtle. Conclusion: one needs to track the evolution of the &quot;signal/noise&quot; ratio in the code by &lt;em&gt;measuring&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how many lines of code in the program ?&lt;/li&gt;&#xA;&lt;li&gt;how many classes/templates ?&lt;/li&gt;&#xA;&lt;li&gt;running time ?&lt;/li&gt;&#xA;&lt;li&gt;memory consumption ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Everything that increases the first two ones may be considered as a &lt;em&gt;cost&lt;/em&gt; (because it may make the program harder to understand and to maintain), everything that decreases the last two ones is a &lt;em&gt;gain&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, introducing an abstraction (a virtual class or a template) can factor code and make the program simpler (&lt;em&gt;gain&lt;/em&gt;), but if it is never derivated / instanced once only, then it introduces a &lt;em&gt;cost&lt;/em&gt; for no associated &lt;em&gt;gain&lt;/em&gt; (again it is subtle because the gain may come later in the future evolution of the program, therefore there is no &quot;golden rule&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programmer's comfort is also an important factor to be taken into account in the cost/gain balance: with too many templates, compilation time may increase significantly, and error messages become difficult to parse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;See also&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/1745/to-what-extent-is-generic-and-meta-programming-using-c-templates-useful-in-com?rq=1&quot;&gt;To what extent is generic and meta-programming using C++ templates useful in computational science?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-08-31T07:14:12.807" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20302" PostTypeId="1" CreationDate="2015-07-28T01:44:10.890" Score="0" ViewCount="469" Body="&lt;p&gt;I have a matrix &lt;code&gt;A&lt;/code&gt; which is of size &lt;code&gt;(n2, n1)&lt;/code&gt; and I am multiplying it by a matrix, &lt;code&gt;B&lt;/code&gt;, of size &lt;code&gt;(n1, n0)&lt;/code&gt;. I have identified this single matrix multiplication as the bottleneck in my &lt;code&gt;Fortran&lt;/code&gt; code. Out of ~2000 lines of code, this single line takes about 77% of the runtime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;A&lt;/code&gt; is a double precision matrix with floating point values. &lt;code&gt;B&lt;/code&gt; is, currently, a double precision matrix containing only values &lt;code&gt;1.0&lt;/code&gt; and &lt;code&gt;0.0&lt;/code&gt;. I can easily make this integer, or even binary, but I was using it as &lt;code&gt;real&lt;/code&gt; so that I could preserver precision in &lt;code&gt;matmul(A,B)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a better way to perform this matrix multiplication to cut down on runtime?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before anyone suggests it, I am using &lt;code&gt;DGEMM&lt;/code&gt; and compiling with &lt;code&gt;-O3&lt;/code&gt; and &lt;code&gt;-mavx&lt;/code&gt; for &lt;code&gt;gfortran&lt;/code&gt;, and &lt;code&gt;-O3&lt;/code&gt; with &lt;code&gt;-xhost&lt;/code&gt; on &lt;code&gt;ifort&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The largest data I have implemented this program on so far, &lt;code&gt;N = 5000&lt;/code&gt;, results in &lt;code&gt;n2 = 1668&lt;/code&gt;, &lt;code&gt;n1 = 1701&lt;/code&gt;, and &lt;code&gt;n0 = 1631&lt;/code&gt;. This algorithm was implemented in &lt;code&gt;Matlab&lt;/code&gt; and has shorter runtime. Matlab version is about 2.5 seconds, while this fortran program is about 7 seconds. Since this single matrix multiplication is so large, I'm thinking that Matlab is doing something interesting with the variable types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have compiled this with &lt;code&gt;ifort&lt;/code&gt; using &lt;code&gt;MKL&lt;/code&gt; and am current linking against &lt;code&gt;-lblas&lt;/code&gt; and using &lt;code&gt;-fexternal-blas&lt;/code&gt;, relying on &lt;code&gt;matmul&lt;/code&gt; to perform the underlying &lt;code&gt;BLAS&lt;/code&gt; routines. The result of &lt;code&gt;ldd&lt;/code&gt; on my binary executable is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;linux-vdso.so.1 =&amp;gt;  (0x00002aaaaaacb000)&#xA;liblapack.so.3 =&amp;gt; /usr/lib64/atlas/liblapack.so.3 (0x00002aaaaaccd000)&#xA;libblas.so.3 =&amp;gt; /usr/lib64/libblas.so.3 (0x00002aaaab4f0000)&#xA;libgfortran.so.3 =&amp;gt; /usr/lib64/libgfortran.so.3 (0x00002aaaab747000)&#xA;libm.so.6 =&amp;gt; /lib64/libm.so.6 (0x00002aaaaba39000)&#xA;libgcc_s.so.1 =&amp;gt; /lib64/libgcc_s.so.1 (0x0000003f78c00000)&#xA;libc.so.6 =&amp;gt; /lib64/libc.so.6 (0x00002aaaabcbe000)&#xA;libf77blas.so.3 =&amp;gt; /usr/lib64/atlas/libf77blas.so.3 (0x00002aaaac052000)&#xA;libcblas.so.3 =&amp;gt; /usr/lib64/atlas/libcblas.so.3 (0x00002aaaac272000)&#xA;/lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)&#xA;libatlas.so.3 =&amp;gt; /usr/lib64/atlas/libatlas.so.3 (0x00002aaaac492000)&#xA;libpthread.so.0 =&amp;gt; /lib64/libpthread.so.0 (0x00002aaaacaee000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;B&lt;/code&gt; is structured in the way that it has zeros and ones. The lower left portion (not truly lower triangular) has ones and the upper right portion (not triangular) is zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears that the Matlab code is treating the &lt;code&gt;B&lt;/code&gt; matrix as &lt;code&gt;logical&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="164" LastEditorUserId="164" LastEditDate="2015-07-28T12:52:54.297" LastActivityDate="2015-07-28T12:52:54.297" Title="Efficient method to multiply floating point matrix with binary matrix and get double precision results" Tags="|matrix|fortran|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20303" PostTypeId="2" ParentId="20302" CreationDate="2015-07-28T01:56:13.627" Score="1" Body="&lt;p&gt;What is the percentage of nonzero entries in $B$?  If a high percentage of the entries are 0's, then you might well be better off treating $B$ as a sparse matrix in the multiplication.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't told us the size of the matrices.  What are n0 and n1?  Or at least their order of magnitude?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't said what implementation of the BLAS you're currently using.  Your specific comments about compilation flags suggest that you might be using the reference BLAS implementation, which would be a very poor choice in comparison with optimized cache-aware and multithreaded implementations of the BLAS routines such as ATLAS, OpenBLAS, ACML, and MKL.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-07-28T01:56:13.627" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20304" PostTypeId="1" CreationDate="2015-07-28T02:57:55.587" Score="1" ViewCount="91" Body="&lt;p&gt;I am working on some model analysis, getting two diff equations and&#xA;after I convert them into matrix form, I have equations looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ [A][X]=C\times\big(\exp([B][X])-1\big), $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C$ is a constant&#xA;and Both $[A]$ and $[B]$ are found out from two diff equations and boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is how am I able to find the value of $[X]$, I am thinking using fixed point method, but this is in matrix form, could any one give me any clue or hint?&lt;/p&gt;&#xA;" OwnerUserId="17074" LastEditorUserId="713" LastEditDate="2015-07-28T03:58:18.867" LastActivityDate="2015-07-28T03:58:18.867" Title="fixed point iteration to find out second order non-linear diff equations" Tags="|matlab|numerical-analysis|matrix|computational-physics|mathematica|" AnswerCount="0" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20305" PostTypeId="1" CreationDate="2015-07-28T04:15:56.717" Score="4" ViewCount="148" Body="&lt;p&gt;So I inherited from some people a code that solves the advection-diffusion-reaction equation for a particular system. The original code was first implemented in 1D which worked fine in cartesian coordinates. However, now that I'm trying to implement it in 2d I'm having a few problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t} + \nabla \cdot \left( \boldsymbol{v} u - D\nabla u \right) = f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ are the primitive variables being evolved, $D$ is a diffusion coefficient, and $f$ is source+sink.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By separating the spatial discretization from the temporal through a method of lines, integrating time with a Runge-Kutta 4th order scheme, and using a cartesian coordinate discretation it becomes relatively straightforward to extend a model from 1D -&gt; 2D because the spatial components are additive. However, the problem is that the genuine solutions to the equations I'm solving tends be somewhat spherically symmetric (but not completely), but the grid is cartesian so the solution tends to show grid orientation errors in the cartesian grid, i.e. the solution looks kinda &quot;square&quot; instead of looking like a &quot;circle&quot;. A CPU costly solution to this grid orientation error is to make the grid finer, but it's getting to the point the computations become very slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was able to solve the 2D grid orientation errors for the diffusion part of the advection-difussion-reaction equations by discretizing the laplacian operator into a 9-point stencil instead of the original 5-point stencil, the latter 5-point stencil is acquired by just naively adding the 1D laplacian discretations of the x and y components. The 9 point stencil adds spatially &quot;transverse&quot; components to the discretization. However, the advection part still has some pretty bad grid orientation problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is a &quot;9-point stencil&quot; analog to the 2D advection problem. Maybe a stencil that takes into account the &quot;transverse&quot; cross-term spatial components somehow. I know of Corner Transport Upwind (CTU) which has a transverse component: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;Q_{ij}^{n+1}&amp;amp;=Q_{ij}^n-\frac{u\Delta t}{\Delta x}\left[Q_{ij}^n-Q_{i-1j}^n\right] -\frac{v\Delta t}{\Delta y}\left[Q_{ij}^n-Q_{ij-1}^n\right] \\&#xA;&amp;amp;\qquad+uv\frac{2\Delta t^2}{\Delta x\Delta y}\left[\left(Q_{ij}^n-Q_{ij-1}^n\right)+\left(Q_{i-1j}^n-Q_{i-1j-1}^n\right)\right.\\&#xA;&amp;amp;\qquad\qquad\qquad\qquad\left.+\left(Q_{ij}^n-Q_{i-1j}^n\right)+\left(Q_{ij-1}^n-Q_{i-1j-1}^n\right)\right]&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I'm not sure if it can be used with MOL and Runge-Kutta 4th order because it depends explicitly on time e.g. $\Delta t$, while it seems that cell discretization schemes used for MOL only depend on the spatial components so that time can be integrated separately. However I might be wrong and CTU could be perfectly appropriate. I would be really grateful for some guidance here because I'm a noob!&lt;/p&gt;&#xA;" OwnerUserId="17075" LastEditorUserId="17075" LastEditDate="2015-07-28T05:19:14.087" LastActivityDate="2015-09-30T10:44:21.563" Title="transverse component for multidimensional advection in method of lines" Tags="|finite-difference|fluid-dynamics|finite-volume|method-of-lines|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20306" PostTypeId="1" CreationDate="2015-07-28T04:29:44.400" Score="12" ViewCount="138" Body="&lt;p&gt;Character tables are an important tool for symmetry analysis in many computational chemistry software packages. Are there any standardized file formats for point group character tables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may seem to be an odd question as I could (and I do) just retrieve this information from other sources. But for code interoperability/sharability purpose, code documentation and just for laziness (as it is a very common task), it would be nice to have a standard file format for point group character tables.&lt;/p&gt;&#xA;" OwnerUserId="14867" LastEditorUserId="3887" LastEditDate="2015-07-30T15:51:03.913" LastActivityDate="2015-07-30T15:51:03.913" Title="Are there any standardized file formats for point group character tables?" Tags="|computational-chemistry|symmetry|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20307" PostTypeId="1" CreationDate="2015-07-28T08:00:09.590" Score="2" ViewCount="124" Body="&lt;p&gt;I'm familiar with finding the order of accuracy using von Neumann analysis for finite difference schemes formulated using Taylor series expansions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But is there a similar technique for finding the order of accuracy for schemes that use a least squares fit for approximating gradients?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am solving a 1D advection equation in advective form.  I'm using Forward Euler timestepping and an upwind quadratic least squares fit with a three-point stencil.  My experiments suggest an order of accuracy of ~0.7 which suggests that I've either got bugs in my implementation or the scheme is inherently slow to converge.&lt;/p&gt;&#xA;" OwnerUserId="7192" LastActivityDate="2015-07-28T08:00:09.590" Title="Calculating theoretical order of accuracy of least squares fit advection scheme" Tags="|finite-difference|numerical-analysis|least-squares|advection|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20308" PostTypeId="1" CreationDate="2015-07-28T11:18:19.197" Score="1" ViewCount="130" Body="&lt;p&gt;Is there a relation between say the heat equation $u_t -\Delta u = f.$ and advection-diffusion equation $-\Delta u + c \cdot \nabla u = f$? I have heard several people use this argument in many talks and articles but this is particularly not clear to me. &lt;/p&gt;&#xA;" OwnerUserId="7984" LastEditorUserId="7984" LastEditDate="2015-07-28T17:53:43.750" LastActivityDate="2015-07-29T09:06:53.663" Title="Relation between Time dependent problem and advection diffusion" Tags="|parabolic-pde|advection-diffusion|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20309" PostTypeId="2" ParentId="20291" CreationDate="2015-07-28T18:04:09.133" Score="1" Body="&lt;p&gt;The way I understand this question is that you want to project a 3D scatter plot into a 2D density map and then plot it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if there is a specific software for that but it should be relatively simple, you could always make a routine that computes the sum of all $w_i$ at all  $z$ values at a particular $(x,y)$ coordinate and then you would have the density at that particular $(x,y)$, for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x,y)=\sum_k w(x,y,k)$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k$ is an index summed across all z coordinates at a particular constant (x,y), and $f(x,y)$ is your density. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x,y)$ now then is a density function of two dimensions so you could plot it with any plotting software, whether gnuplot, matlab, mathematica etc. &lt;/p&gt;&#xA;" OwnerUserId="17075" LastEditorUserId="17075" LastEditDate="2015-07-28T18:16:28.527" LastActivityDate="2015-07-28T18:16:28.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20310" PostTypeId="2" ParentId="20308" CreationDate="2015-07-28T18:13:54.720" Score="0" Body="&lt;p&gt;Yes there is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The heat equation has the same mathematical form as the diffusion part of the diffusion-advection equation. Both are Parabolic operators and behave very similarly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the heat equation, temperature will &quot;diffuse&quot; from the hottest zone to the cooler zones. This is very similar to how a substance will &quot;diffuse&quot; from the area of highest concentration to the lowest concentration. The behavior is almost identical. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many numerical schemes, you can always treat the diffusion part of the diffusion-advection equation separately from the advection part, so effectively the diffusion part can be treated as like a heat equation or viceversa. &lt;/p&gt;&#xA;" OwnerUserId="17075" LastActivityDate="2015-07-28T18:13:54.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20311" PostTypeId="1" AcceptedAnswerId="20315" CreationDate="2015-07-29T08:16:38.537" Score="6" ViewCount="503" Body="&lt;p&gt;I've written a basic 2D &lt;a href=&quot;https://en.wikipedia.org/wiki/Langevin_dynamics&quot;&gt;Langevin dynamics&lt;/a&gt; simulator in C++, for a particle in a potential, solving the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M\ddot{X} = - \nabla U(X) - \gamma M \dot{X} + \sqrt{2 \gamma k_B T M} R(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This relies on a source of random numbers for $R(t)$, and I've been using &lt;a href=&quot;http://www.cplusplus.com/reference/random/&quot;&gt;&lt;code&gt;std::random&lt;/code&gt;&lt;/a&gt; (in particular, a Mersenne Twister RNG).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current approach is to generate the random numbers all in one go at the beginning of the simulation, rather than at each time step, as it was (for me at least) faster. This is fine for runs up to $10^{8}$ time steps, but eventually I run out of RAM to store the random numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to at least have the ability to run longer simulations (I'm investigating diffusion behaviour of particles in simple potentials), and would like to know what potential pitfalls to look out for when either&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;splitting the simulation into segments, or&lt;/li&gt;&#xA;&lt;li&gt;breaking up the calculation of the random numbers into smaller chunks (say do a block of numbers ever $10^{6}$ steps.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One thing I noticed was the risk of repeating sequences - see &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2632580/&quot;&gt;&quot;Vulnerability in Popular Molecular Dynamics Packages Concerning Langevin and Andersen Dynamics&quot;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8094" LastActivityDate="2015-07-29T11:46:22.850" Title="Generating random numbers for long molecular dynamics simulations" Tags="|molecular-dynamics|random-number-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20312" PostTypeId="1" AcceptedAnswerId="20318" CreationDate="2015-07-29T08:49:48.520" Score="4" ViewCount="151" Body="&lt;p&gt;When solving semi-discrete equations (originating from finite element models, for example), which are second-order in time of the form&#xA;\begin{equation}&#xA;M\ddot d + C\dot d + Kd = F,&#xA;\end{equation}&#xA;where $d$ is the solution vector, $M$, $C$ and $K$ are matrices, and $F$ is a vector, one can make use of methods that damp out spurious high-frequency oscillations such as damped versions of the Newmark method, HHT-$\alpha$, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one wishes to solve instead a system of the form &#xA;\begin{equation}&#xA;M\dot d + Kd = F,&#xA;\end{equation}&#xA;the obvious choice would be to use a generalized trapezoidal method. However, I am looking for a method that exhibits damping out of spurious high-frequency oscillations, as in the second-order case. I don't need a highly accurate method, but it would preferably be an explicit one.&lt;/p&gt;&#xA;" OwnerUserId="6775" LastEditorUserId="6775" LastEditDate="2015-07-29T08:56:22.850" LastActivityDate="2015-07-30T15:44:38.070" Title="Dissipative time-stepping scheme for first order in time system" Tags="|finite-element|time-integration|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20313" PostTypeId="2" ParentId="20308" CreationDate="2015-07-29T09:06:53.663" Score="3" Body="&lt;p&gt;So after browsing the paper a bit, I think that the answer is essentially what  Christian Clason stated in his comment.&lt;/p&gt;&#xA;&lt;p&gt;It seems that the original question refers to the statement just above Equation (3) in the &lt;a href=&quot;http://dx.doi.org/10.1016/j.apnum.2005.11.003&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; linked by kwesi : There, the authors say that the advection-diffusion equation (Equation (1) in the paper)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial c}{\partial t} + \sum_{i=1}^{d} u_i \frac{\partial c}{\partial x_i} - \sum_{i,j=1}^{d} D_{ij} \frac{\partial}{\partial x_{j}}\frac{\partial c}{\partial x_i} = 0$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;can be rewritten as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} \nabla \cdot \left( B  c - A \nabla c \right) = 0 \end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$A = \begin{pmatrix} 0 &amp;amp; 0 \\ 0 &amp;amp; \mathbf{D} \end{pmatrix}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B = (1, \mathbf{u})$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$\nabla$&lt;/span&gt; is defined as the gradient in space &lt;strong&gt;and&lt;/strong&gt; time, that is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla = \left( \frac{\partial}{\partial x_0}, \frac{\partial}{\partial x_1}, \ldots, \frac{\partial}{\partial x_d} \right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$x_0 \equiv t$&lt;/span&gt;. In this sense, the time-dependent heat equation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c_t - \sum_{i,j=1}^{d} \frac{\partial}{\partial x_{j}}\frac{\partial c}{\partial x_i}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;can be written as an advection-diffusion problem in space-time&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla \cdot \left( (1,\mathbf{0}) c - A \nabla c \right) = 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The paper then proceeds to analyse space-time discontinuous Galerkin methods based on this formulation of the problem.&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-07-29T09:06:53.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20315" PostTypeId="2" ParentId="20311" CreationDate="2015-07-29T11:46:22.850" Score="6" Body="&lt;p&gt;I suspect the reason why generating the random numbers on the fly is slower for you is due to the rather large state of the Mersenne Twister.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Switching to something like the &lt;a href=&quot;http://www.pcg-random.org/&quot;&gt;PCG&lt;/a&gt; or &lt;a href=&quot;http://xorshift.di.unimi.it/&quot;&gt;XorShift+&lt;/a&gt; random number generator would have several advantages for you:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Higher quality of randomness (Mersenne Twister fails several tests for randomness)&lt;/li&gt;&#xA;&lt;li&gt;Smaller state, so generating the random number on the fly should be faster than storing in memory, thus solving your problem.&lt;/li&gt;&#xA;&lt;li&gt;General speed - both PCG and XorShift+ are around 2x faster than the Mersenne Twister.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As for repeating sequences, this should not happen if you initialize the generator properly, using std::seed_seq. &#xA;See the &lt;a href=&quot;http://www.pcg-random.org/using-pcg-cpp.html&quot;&gt;PCG C++ documentation&lt;/a&gt; for instance.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2015-07-29T11:46:22.850" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20317" PostTypeId="2" ParentId="20250" CreationDate="2015-07-29T17:03:43.293" Score="0" Body="&lt;p&gt;See &lt;a href=&quot;http://www.ccpo.odu.edu/~tejada/USACM07/BrooksHughes.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.ccpo.odu.edu/~tejada/USACM07/BrooksHughes.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where the technique was first used for convection dominated flows.&lt;/p&gt;&#xA;" OwnerUserId="5353" LastActivityDate="2015-07-29T17:03:43.293" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20318" PostTypeId="2" ParentId="20312" CreationDate="2015-07-29T18:21:53.373" Score="3" Body="&lt;p&gt;There is an excellent family of time integration schemes that fit your description called Generalized Single Step Single Solve (GS4).  The original work on the implicit methods for first order systems can be found in [1]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the implicit algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;C \widetilde{\dot{d}}  + K \widetilde{d}  =  \widetilde{F} &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the variables have been approximated between the $n$th and $(n+1)$th timestep as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp; \widetilde{\dot{d}} = \dot{d}_n + \Lambda_6 W_1 (  \dot{d}  _{n+1}-  \dot{d}  _n)&#xA;\\&#xA;&amp;amp; \widetilde{d}   =  d_n  + \Lambda_4 W_1\Delta t \dot{d}_n + \Lambda_5 W_2\Delta t(  \dot{d}  _{n+1}-  \dot{d}  _n)&#xA;\\&#xA;&amp;amp; \widetilde{F}  =(1-W_1)q_n+W_1 F_{n+1}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now one can solve for $ \Delta\dot{d}  =   \dot{d}  _{n+1}-  \dot{d}  _n$ using&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;( \Lambda_6 W_1 C &amp;amp;+ \Lambda_5 W_2 \Delta t K)   \Delta \dot{d }&#xA;\\&#xA;= &amp;amp; -C   \dot{d}  _n - K (   d  _{n} +\Lambda_4 W_1 \Delta t   \dot{d}  _n) &#xA;\\&#xA; &amp;amp;+ (1-W_1)F_n+W_1 F_{n+1}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the updates&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\dot{d}  _{n+1} =   \dot{d}  _n +  \Delta \dot{d }  &#xA;\end{equation}&#xA;\begin{equation}&#xA;  d  _{n+1} =   d  _{n} + \lambda_4 \Delta t   \dot{d}  _{n} + \lambda_5 \Delta t  \Delta \dot{d }  &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\Lambda_4 W_1 = \frac{1}{1+\rho_{\infty}} \nonumber \\&#xA;&amp;amp;\Lambda_5 W_2 = \frac{1}{(1+\rho_{\infty})(1+\rho_{\infty}^{s})}\nonumber \\&#xA;&amp;amp;\Lambda_6 W_1 = \frac{3+\rho_{\infty}+\rho_{\infty}^{s} - \rho_{\infty}\rho_{\infty}^{s}}{2(1+\rho_{\infty})(1+\rho_{\infty}^{s})}  \\&#xA;&amp;amp;W_1 = \frac{1}{1+\rho_{\infty}} \nonumber \\&#xA;&amp;amp;\lambda_4 = 1, \quad \nonumber &#xA;\lambda_5 = \frac{1}{1+\rho_{\infty}^{s}} \nonumber&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Admittedly, this may seem like a whole load of parameters and nonsense, but all parameters only depend upon two chosen values $(\rho_{\infty}, \rho_{\infty}^{s})$.  The beauty is once you have it programmed you can choose from a whole family of algorithms just by choosing you values for $(\rho_{\infty}, \rho_{\infty}^{s})$.  These parameters correspond to the eigenvalues of amplification matrix of the single DOF problem.  Thus, you can choose the amount of damping (numerical dissipation) simply by choosing them.  Some noteable choices: $(\rho_{\infty}, \rho_{\infty}^{s}) = (1,1)$ gives the Crank-Nicolson method (no damping, not for you) and $(\rho_{\infty}, \rho_{\infty}^{s}) = (0,0)$ gives an algorithm equivalent to the highly dissipative Gear's method (aka two-step backwards difference formula).  Any choice will give you a second-order, unconditionally stable algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the restriction on your choices: $0 \leq \rho_{\infty}^{s} \leq \rho_{\infty} \leq 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you want an explicit algorithm, some algorithms have been developed using the same approach that led to the mess above.  I don't think they have been published any place highly visible yet but early work can be found in a master's thesis here [2].  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest thing to do to get an explicit scheme, with the nice dissipation properties of the above algorithm, is to turn it into a predictor-corrector method.  You, of course, lose the unconditional stability, but you will still have a second-order time integrator.  To do so you can replace the $\widetilde{d}$ above with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\widetilde{d}   =  d_n  + \Lambda_4 W_1\Delta t \dot{d}_n&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and lump the $C$ matrix, then march to your hearts content.  Everything else stays the same but the restrictions to the $\rho$ parameters above are lifted (they can be anything).  The stability of the algorithm and the amount of dissipation still depends upon this choice.  Hover around 0 and you should be okay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/nme.3228/full&quot; rel=&quot;nofollow&quot;&gt;http://onlinelibrary.wiley.com/doi/10.1002/nme.3228/full&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://conservancy.umn.edu/handle/11299/162393&quot; rel=&quot;nofollow&quot;&gt;http://conservancy.umn.edu/handle/11299/162393&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17089" LastActivityDate="2015-07-29T18:21:53.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20319" PostTypeId="2" ParentId="16150" CreationDate="2015-07-29T23:27:03.263" Score="1" Body="&lt;p&gt;I disagree with both of you. C++ and Python are still not as optimized as FORTRAN. I would definitely agree that FORTRAN 90/95 is the minimum. However, it is always useful to use other more modern versions. The challenge with FORTRAN is that if you learn it before you learn object oriented languages, it is easy to understand. It is much harder to learn if you are used to thinking with objects. I teach the supercomputing class here and I force my students to use FORTRAN. It forces them to use a different sort of logic in programming. They don't like it because it doesn't &quot;think&quot; like Python or C++. It all really comes down to what you intend to do and how much parallization and performance matter.&lt;/p&gt;&#xA;" OwnerUserId="17062" LastActivityDate="2015-07-29T23:27:03.263" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20321" PostTypeId="1" AcceptedAnswerId="20322" CreationDate="2015-07-30T02:32:12.993" Score="7" ViewCount="6351" Body="&lt;p&gt;How to solve the optimization problem written below?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;\operatorname{argmax}\limits_{a}\; a^T b - \frac{1}{2}  a^T X a\\&#xA;&amp;amp;\text{subject to } \sum_i |a_i|=4,\;  \sum_i a_i = 0&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a$, $b$ are $n$-vectors and $X$ is a $n\times n$ matrix. Also, $b$ and $X$ are constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main issue is about the absolute values. Without absolute values, there is actually an analytic solution. I guess with absolute values, I have to use iterative approach such as quadratic programming but still not sure how to express the problem to call relevant optimization procedures.&lt;/p&gt;&#xA;" OwnerUserId="17079" LastEditorUserId="2150" LastEditDate="2015-07-30T03:26:56.687" LastActivityDate="2015-07-30T04:49:17.833" Title="Convex Optimization problem with sum of absolute value constraints" Tags="|constrained-optimization|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20322" PostTypeId="2" ParentId="20321" CreationDate="2015-07-30T03:26:37.247" Score="10" Body="&lt;p&gt;Unfortunately, your problem isn't a convex optimization problem because the constraint $\Sigma_{i} | a_{i}|=4$ describes a non-convex feasible region.  If you could change this to $\Sigma_{i} | a_{i} | \leq 4$, you'd have a convex constraint.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the constraint were $\Sigma_{i} | a_{i} | \leq 4$, then you can introduce auxiliary variables $t_{i}$, and add the constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma_{i} t_{i} \leq 4$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_{i} \geq a_{i}$  for all $i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_{i} \geq -a_{i}$ for all $i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a standard reformulation technique in convex optimization.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue with your original problem statement is that $X$ must be positive semidefinite to ensure concavity of the objective function.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that $X$ is positive semidefinite, you've now got a linear constrained convex quadratic programming problem which is solvable by lots of solvers. &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-07-30T04:49:17.833" LastActivityDate="2015-07-30T04:49:17.833" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20323" PostTypeId="2" ParentId="20250" CreationDate="2015-07-30T08:12:39.783" Score="3" Body="&lt;p&gt;The original paper introducing the SUPG was from Brooks and Hughes, 1982 (&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0045782582900718&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/0045782582900718&lt;/a&gt;). This was purposely for the convection dominated flows using the Incompressible Navier-Stokes as an emphasis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can recommend the monograph of Roos, Styne and Tobiska (&lt;a href=&quot;http://www.springer.com/us/book/9783540344667&quot; rel=&quot;nofollow&quot;&gt;http://www.springer.com/us/book/9783540344667&lt;/a&gt;). This book particularly covers this topic into great depth and will easily recommend it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think may be it is important to understand the Incompressible unsteady Navier-Stokes before considering the numerical discretization schemes.&lt;/p&gt;&#xA;" OwnerUserId="7984" LastActivityDate="2015-07-30T08:12:39.783" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20324" PostTypeId="1" AcceptedAnswerId="20385" CreationDate="2015-07-30T11:05:35.370" Score="3" ViewCount="348" Body="&lt;p&gt;I am trying to implement the finite difference beam propagation method to study the propagation of a TE light signal through a waveguide. However, my solutions are exponentially growing, and display no propagation. For more information, see &quot;Introduction to Optical Waveguide Analysis&quot; by Kawano and Kitoh, section 4.3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method involves iteratively solving a complex-valued matrix equation of the form $Ax = b$, with $A$ a square matrix, $b$ a known vector and $x$ a vector to be solved for. I am using the LAPACK routine &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/dd/dfb/zgtsv_8f.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;zgtsv&lt;/code&gt;&lt;/a&gt;  via C++ to solve this equation, but I am getting numerical instabilities. In particular, the vectors are exponentially growing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More explicitly, given the complex function $\phi(x,0)$ we wish to solve for $\phi(x,z)$ subject to the partial differential equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 2i\beta\frac{\partial \phi}{\partial z} = \frac{\partial^2 \phi}{\partial x^2}+(k_0^2 n(x)^2 - \beta^2)\phi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $-3&amp;lt;x&amp;lt;3$ and $k_0=4$. $n(x) = 3.5$ for $-1&amp;lt;x&amp;lt;1$ and $n(x) = 1.5$ otherwise. Suppose further that $0&amp;lt;z&amp;lt;40$. $\beta = 14$ is the propagation constant of the waveguide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We discretize the problem into $N$ steps in the $x$ direction and $M$ steps in the $z$ direction and replace $\phi(x,z)$ by $\phi(p,q)$ where $p$ and $q$ are integers. Let $ dx = \frac{6}{N-1} $ and $ dz = \frac{40}{M-1} $ Using $(p,q+1/2)$ as a center difference for Taylor expansions, we get the equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\frac{1}{dx^2} \phi(p-1,q+1) + \left\{\frac{2}{dx^2} + \frac{4i\beta}{dz} - (k_0^2 n(x)^2 - \beta^2)\right\}\phi(p,q+1)-\frac{1}{dx^2} \phi(p+1,q+1) = \frac{1}{dx^2} \phi(p-1,q) + \left\{-\frac{2}{dx^2} + \frac{4i\beta}{dz} + (k_0^2 n(x)^2 - \beta^2)\right\}\phi(p,q)+\frac{1}{dx^2} \phi(p+1,q)$$ which much be satisfied for all $p$. This allows us to step forward from $\phi(p,q)$ to $\phi(p,q+1)$. This equation may be rephrased as a matrix equation $A \phi(q) = b$, where $A$ is a symmetric tridiagonal matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this method does not appear to be working for me. In fact, $\phi(p,q)$ is growing exponentially with $q$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any known problems with using &lt;code&gt;zgtsv&lt;/code&gt; iteratively like this? I have very little idea of what is going wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Here is a condensed version of the code I am using. I am using Dislin to plot Solution. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double PI = 3.141592;&#xA;std::complex&amp;lt;double&amp;gt; imaginaryI(0.0,1.0);&#xA;&#xA;int N = 300;&#xA;int M = 300;&#xA;&#xA;std::complex&amp;lt;double&amp;gt; *ComplexSolution= new std::complex&amp;lt;double&amp;gt; [M*N];&#xA;&#xA;for (int i = 0; i &amp;lt; N; i++){&#xA;    std::cout &amp;lt;&amp;lt; i &amp;lt;&amp;lt; std::endl;&#xA;    if( i == N /2)&#xA;        ComplexSolution[i] = 1.0;&#xA;    else ComplexSolution[i] = 0.0;}&#xA;&#xA;double *Solution= new double [M*N];&#xA;double waveK = 4.0;&#xA;double x1 = -3.0, x2 = -1.0, x3 = 1.0, x4 = 3.0;&#xA;double indexN1 = 1.5, indexN2 = 3.5;&#xA;double Length = 40;&#xA;double dx = (x4-x1)/(N-1), dz = Length/(M-1);&#xA;double BETA = 14.0;&#xA;&#xA;double alpha1 = 1.0/(dx*dx), alpha2 = -2.0/(dx*dx);&#xA;&#xA;std::complex&amp;lt;double&amp;gt; *BPM_sub_diagonal = new std::complex&amp;lt;double&amp;gt; [N-1];&#xA;std::complex&amp;lt;double&amp;gt; *BPM_diagonal = new std::complex&amp;lt;double&amp;gt; [N];&#xA;std::complex&amp;lt;double&amp;gt; *workBPM_sub_diagonal = new std::complex&amp;lt;double&amp;gt; [N-1];&#xA;std::complex&amp;lt;double&amp;gt; *workBPM_diagonal = new std::complex&amp;lt;double&amp;gt; [N];&#xA;std::complex&amp;lt;double&amp;gt; *B = new std::complex&amp;lt;double&amp;gt; [N];&#xA;&#xA;int NRHS = 1;&#xA;int INFO;&#xA;&#xA;for (int i = 0; i &amp;lt; N-1; i++)&#xA;    BPM_sub_diagonal[i] = -alpha1;&#xA;&#xA;for (int i = 0; i &amp;lt; N; i++){&#xA;    if(i*dx+x1&amp;gt;x2 &amp;amp;&amp;amp; i*dx+x1&amp;lt;x3)&#xA;        BPM_diagonal[i] = -alpha2 + (4.0*imaginaryI*BETA/dz) - waveK*waveK*indexN2*indexN2 + BETA*BETA;&#xA;    else&#xA;        BPM_diagonal[i] = -alpha2 + (4.0*imaginaryI*BETA/dz) - waveK*waveK*indexN1*indexN1 + BETA*BETA;}&#xA;&#xA;for(int j = 1; j &amp;lt; M; j++){&#xA;    for(int i = 0; i &amp;lt; N; i++)&#xA;        workBPM_diagonal[i] = BPM_diagonal[i];&#xA;&#xA;    for(int i = 0; i &amp;lt; N-1; i++)&#xA;        workBPM_sub_diagonal[i] = BPM_sub_diagonal[i];&#xA;&#xA;    for(int i = 0; i &amp;lt; N; i++){&#xA;        if(i == 0)&#xA;            B[i] = (alpha2 + (4.0*imaginaryI*BETA/dz) + waveK*waveK*indexN1*indexN1 - BETA*BETA)*ComplexSolution[(j-1)*N+i]&#xA;                   + alpha1*ComplexSolution[(j-1)*N+i+1];&#xA;&#xA;        else if(i == N-1)&#xA;            B[i] = (alpha2 + (4.0*imaginaryI*BETA/dz) + waveK*waveK*indexN1*indexN1 - BETA*BETA)*ComplexSolution[(j-1)*N+i]&#xA;                   + alpha1*ComplexSolution[(j-1)*N+i-1];&#xA;&#xA;        else if(i*dx+x1&amp;gt;x2 &amp;amp;&amp;amp; i*dx+x1&amp;lt;x3)&#xA;            B[i] = (alpha2 + (4.0*imaginaryI*BETA/dz) + waveK*waveK*indexN2*indexN2 - BETA*BETA)*ComplexSolution[(j-1)*N+i]&#xA;                   + alpha1*ComplexSolution[(j-1)*N+i+1]&#xA;                   + alpha1*ComplexSolution[(j-1)*N+i-1];&#xA;        else &#xA;            B[i] = (alpha2 + (4.0*imaginaryI*BETA/dz) + waveK*waveK*indexN1*indexN1 - BETA*BETA)*ComplexSolution[(j-1)*N+i]&#xA;                   + alpha1*ComplexSolution[(j-1)*N+i+1]&#xA;                   + alpha1*ComplexSolution[(j-1)*N+i-1];&#xA;    }&#xA;&#xA;    zgtsv_(&amp;amp;N, &amp;amp;NRHS, workBPM_sub_diagonal, workBPM_diagonal, workBPM_sub_diagonal, B, &amp;amp;N, &amp;amp;INFO);&#xA;&#xA;    for(int i = 0; i &amp;lt; N; i++)&#xA;        ComplexSolution[j*N+i] = B[i];&#xA;&#xA;}&#xA;&#xA;for(int i = 0; i &amp;lt; M*N; i++)&#xA;    Solution[i] = std::abs(ComplexSolution[i]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17087" LastEditorUserId="17087" LastEditDate="2015-08-05T10:34:32.497" LastActivityDate="2015-08-06T05:49:12.073" Title="Finite Difference Beam Propagation Method problem" Tags="|linear-algebra|finite-element|computational-physics|lapack|electromagnetism|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="20325" PostTypeId="2" ParentId="2283" CreationDate="2015-07-30T14:41:43.247" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/jameskermode/f90wrap&quot; rel=&quot;nofollow&quot;&gt;f90wrap&lt;/a&gt; is build on top of f2py and automatically generates the Python wrappers for the underlying Fortran code base, including access to derived types. A set of simple &lt;a href=&quot;https://github.com/jameskermode/f90wrap/tree/master/examples&quot; rel=&quot;nofollow&quot;&gt;examples&lt;/a&gt; show how it works.&lt;/p&gt;&#xA;" OwnerUserId="17101" LastActivityDate="2015-07-30T14:41:43.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20326" PostTypeId="2" ParentId="20312" CreationDate="2015-07-30T15:44:38.070" Score="0" Body="&lt;p&gt;These differential equations appear to be linear, so you should be able to solve them analytically. In order to do this you first have to write it as one &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_differential_equation&quot; rel=&quot;nofollow&quot;&gt;matrix differential equation&lt;/a&gt; of the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\dot{\vec{x}} = A\vec{x} + B \vec{u}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done by rewriting the differential equations to an expression for $\ddot{d}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\ddot{d} = M^{-1} F - M^{-1} C \dot{d} - M^{-1} K d.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vector $\vec{x}$ can be defined as $\begin{bmatrix}d &amp;amp; \dot{d}\end{bmatrix}^T$, such that $\dot{\vec{x}}$ is equal to $\begin{bmatrix}\dot{d} &amp;amp; \ddot{d}\end{bmatrix}^T$. By using this definition then the matrix $A$ can be written as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A = &#xA;\begin{bmatrix}&#xA;0 &amp;amp; I \\&#xA;-M^{-1}K &amp;amp; -M^{-1}C&#xA;\end{bmatrix},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $0$ and $I$ are the zero/null and identity matrix receptively, both of the same size as $M$, $C$ and $K$. And similar the matrix $B$ can be written as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;B = &#xA;\begin{bmatrix}&#xA;0 \\&#xA;M^{-1}&#xA;\end{bmatrix},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $0$ is again the zero/null matrix of the same size as $M$, $C$ and $K$. And the vector $\vec{u}$ is equal to $F$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The homogeneous or transient solution ($F=0$) can be found with,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\vec{x}_t(t) = \sum{c_i\vec{v}_ie^{\lambda_it}},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\lambda_i$ are eigenvalues of $A$, with $\vec{v}_i$ the corresponding eigenvectors and $c_i$ are constants which can be found with the help of the initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nonhomogeneous or steady state solution can be found by taking the Laplace transform,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{L}\{\dot{\vec{x}}_{ss}(t)\}(s) = A \mathcal{L}\{\vec{x}_{ss}(t)\}(s) + B \mathcal{L}\{\vec{u}(t)\}(s),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;s X_{ss}(s) = A X_{ss}(s) + B U(s),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;X_{ss}(s) = (s I - A)^{-1} B U(s),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\vec{x}_{ss}(t) = \mathcal{L}^{-1}\{(s I - A)^{-1} B \mathcal{L}\{\vec{u}(t)\}(s)\}(t).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the $\vec{u}$ is constant in time, then this simplifies down to,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\vec{x}_{ss}(t) = -A^{-1} B \vec{u}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The steady state solution can also be found by taking the Laplace transform of the original differential equations. The only difference between this and using $A$ is that the later also gives the derivative of $d_{ss}$ because you solve for $\vec{x}_{ss}(t)$. The total response can be found by adding the two solutions,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\vec{x}(t) = \vec{x}_{ss}(t) + \vec{x}_{t}(t),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the constants $c_i$ should only be solved for by using the initial or boundary conditions when $\vec{x}_{ss}(t)$ has also been added.&lt;/p&gt;&#xA;" OwnerUserId="7031" LastActivityDate="2015-07-30T15:44:38.070" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20327" PostTypeId="1" CreationDate="2015-07-30T16:07:53.107" Score="15" ViewCount="7494" Body="&lt;p&gt;In the wave equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$c^2 \nabla \cdot \nabla u(x,t) - \frac{\partial^2 u(x,t)}{\partial t^2} = f(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we first multiply by a test function $v(x,t)$ before integrating?&lt;/p&gt;&#xA;" OwnerUserId="17029" LastActivityDate="2015-07-30T20:12:18.883" Title="What is the purpose of the test function in Finite Element Analysis?" Tags="|finite-element|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20328" PostTypeId="2" ParentId="20327" CreationDate="2015-07-30T16:37:34.227" Score="15" Body="&lt;p&gt;You're coming at it backwards. The justification is better seen by starting from the variational setting and working towards the strong form. Once you've done this, the concept of multiplying by a test function and integrating can then be applied to problems where you don't start with a minimization problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So consider the problem where we want to minimize (and working formally and not rigorously at all here):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ I(u) = \frac {1}{2} \int_\Omega (\nabla u(x))^2 \; dx $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to some boundary conditions on $\partial\Omega$. If we want this $I$ to reach a minimum, we need to differentiate it with respect to $u$, which is a function. There are several now well trod ways to consider this kind of derivative, but one way it's introduced is to compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I'(u(x),v(x))=\lim_{h\rightarrow 0} \frac{d}{dh}I(u(x)+hv(x))&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h$ is just a scalar. You can see that this is similar to the traditional definition of a derivative for scalar functions of a scalar variable but extended up to functionals like $I$ that give scalars back but have their domain over functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we compute this for our $I$ (mostly using the chain rule), we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ I'(u,v) = \int_\Omega \nabla u \cdot \nabla v \; dx $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Setting this to zero to find the minimum, we get an equation which looks like the weak statement for Laplace's equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \nabla u \cdot \nabla v \; dx = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if we use the Divergence Theorm (aka multi-dimesional integration by parts), we can take a derivative off of $v$ and put it on $u$ to get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ -\int_\Omega \nabla \cdot (\nabla u) v \; dx  + \text {boundary terms} = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this really looks where you start when you want to build a weak statement from a partial differential equation. Given this idea now, you can use it for any PDE, just multiply by a test function, integrate, apply the Divergence Theorem, and then discretize.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-07-30T16:37:34.227" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20330" PostTypeId="2" ParentId="20327" CreationDate="2015-07-30T20:12:18.883" Score="13" Body="&lt;p&gt;As I mention before, I prefer to think about the weak form as a weighted residual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to find an approximate solution $\hat{u}$. Let us define the residual as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R = c^2 \nabla \cdot \nabla \hat{u} - \frac{\partial^2 \hat{u}}{\partial t^2} - f(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the case of the exact solution the residual is the zero function over the domain. We want to find an approximate solution that is &quot;good&quot;, i.e., one that makes $R$ &quot;small&quot;. So, we can try to minimize the norm of the residual (Least square methods, for example), or some average of it. One way of doing it is to compute the weighted residual, i.e., minimize the weighted residual&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int\limits_\Omega wR d\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one important thing about this is that it defines a functional, so you can minimize it. This can work for functions that do not have a variational form. &#xA;I describe  a little bit more in this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19006/why-are-functional-representations-of-systems-important-in-numerical-application/19068#19068&quot;&gt;post&lt;/a&gt;. You can choose the function $w$ in different ways, like being of the same space of the function $\hat{u}$ (Galerkin methods), Dirac delta functions (collocation methods), or a &lt;a href=&quot;https://en.wikipedia.org/wiki/Fundamental_solution&quot; rel=&quot;noreferrer&quot;&gt;fundamental solution&lt;/a&gt; (Boundary Elements Method).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you select the first case, then you will end up with an equation like the one described by @BillBarth.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-07-30T20:12:18.883" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20331" PostTypeId="2" ParentId="20231" CreationDate="2015-07-30T21:53:19.350" Score="3" Body="&lt;p&gt;The problem is not convex, as one can verify using the simplest possible choice: $N=2, d=1$ and with data $x_1=x_2=1, f_1=1, f_2=0$. In that case, your objective function is&#xA;$$&#xA;  F(C) = 2\left(1-\exp(-C/2)\right)^2.&#xA;$$&#xA;Plotting this function shows that it is not convex.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-07-30T21:53:19.350" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20332" PostTypeId="1" CreationDate="2015-07-30T22:41:03.280" Score="2" ViewCount="750" Body="&lt;p&gt;I am curious, if there is a function to convert MPIAIJ (distributed matrices in AIJ format) to a SEQAIJ matrix that lie on a single processor. It is possible to do such an operation for PETSc vectors with &lt;code&gt;VecScatterCreateToAll&lt;/code&gt; or &lt;code&gt;VecScatterCreateToZero&lt;/code&gt;, but I couldn't find a similar function for matrices. It is not a scalable operation obviously, but can be helpful for debugging easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively, I thought the following will work, but the problem with this code is that every process generate a different SEQAIJ matrix. Interestingly, petsc4py doesn't generate an error, but simply leaves the matrix with all zeros.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Input A is an MPIAIJ matrix&#xA;def getSEQAIJ(A): &#xA;    N=A.getSize()[0]&#xA;    B=PETSc.Mat().create(comm=PETSc.COMM_SELF)&#xA;    B.setType(PETSc.Mat.Type.SEQAIJ)&#xA;    B.setSizes(N)&#xA;    B.setUp()&#xA;    rstart, rend = A.getOwnershipRange()&#xA;    for i in xrange(rstart,rend):&#xA;        cols,vals = A.getRow(i) #maybe restore later&#xA;        B.setValues(i,cols,vals,addv=PETSc.InsertMode.INSERT)&#xA;    B.assemble()&#xA;    return B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9608" LastEditorUserId="9608" LastEditDate="2015-07-31T16:10:23.817" LastActivityDate="2019-02-06T04:49:25.380" Title="How to convert MPIAIJ to SEQAIJ matrix in petsc/petsc4py?" Tags="|python|sparse-matrix|petsc|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20343" PostTypeId="1" AcceptedAnswerId="20355" CreationDate="2015-07-31T11:03:23.683" Score="7" ViewCount="170" Body="&lt;p&gt;I'm afraid the title is terrible, but I don't know how to summarize this problem in a few words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using an instrument, we are monitoring the trajectory of several objects during a period of time. We take a lot of measurements of the position of the objects in the field of view of the instrument, so we have a list of triplets composed of (time, X, Y). Usually there are several objects in view, so we have several positions for the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We already have an algorithm that can reconstruct the trajectory of a single object if you give its measurements, however, this algorithm won't work if several objects are provided at the same time. So we are looking for an algorithm to first classify the different observations in different objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of an easy problem, plotted in 2D, containing two objects. It seem they cross paths but actually they approach at different times so it should be possible to classify them without confusion. Sometimes the objects will really cross paths at the same time (get really close). In that case, it is better just to discard the points near the encounter than get some points wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cxl3D.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cxl3D.gif&quot; alt=&quot;two objects&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other scenarios are much more complicated, but it would be OK to discard points from different trajectories that are very close in position and in time. Trajectories can be curved lines, and in fact, almost always they are smoothly curved lines:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AAlaG.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AAlaG.gif&quot; alt=&quot;multiple objects&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hint on how should I approach this problem? &lt;/p&gt;&#xA;" OwnerUserId="7405" LastEditorUserId="-1" LastEditDate="2015-07-31T15:34:14.237" LastActivityDate="2015-08-15T23:04:20.627" Title="Classification of observations depending on its trajectory" Tags="|computer-vision|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20344" PostTypeId="2" ParentId="5368" CreationDate="2015-07-31T18:54:17.967" Score="1" Body="&lt;p&gt;The Pole EXpansion and Selected Inversion (&lt;a href=&quot;https://math.berkeley.edu/~linlin/pexsi/&quot; rel=&quot;nofollow&quot;&gt;PEXSI&lt;/a&gt;) method might be the answer. I have not used this method, but it offers an inversion routine for complex symmetric matrices. It is not specific to tridiagonal matrices, but makes use of sparsity.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2015-07-31T18:54:17.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20345" PostTypeId="1" CreationDate="2015-07-31T21:17:46.983" Score="6" ViewCount="4062" Body="&lt;p&gt;So I'm trying to implement a 9-point stencil discretization to the 2D difussion equation. The stencil is &lt;a href=&quot;http://brunoc69.xtreemhost.com/courses/WS05-06/numerikII/stencil.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, most of the literature deals with a Laplacian that has a constant diffusion coefficient. i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f_t=d\Delta f(x,y). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the problem I'm dealing with has a variable diffusion coefficient, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f_t=\nabla\cdot (d(x,y)\nabla f(x,y)). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you implement that in a 9-point stencil? I've seen in the literature the 5-point stencil version for variable coefficient but not the 9-point one.&lt;/p&gt;&#xA;" OwnerUserId="17075" LastEditorUserId="713" LastEditDate="2015-07-31T21:34:29.453" LastActivityDate="2015-08-12T22:05:57.747" Title="9-point stencil finite difference Laplacian with variable diffusion coefficients" Tags="|finite-difference|fluid-dynamics|advection-diffusion|diffusion|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20346" PostTypeId="2" ParentId="20345" CreationDate="2015-08-01T00:33:59.410" Score="2" Body="&lt;p&gt;Interesting question. I too would like to know if this is mentioned anywhere explicitly. My guess is maybe this is a little too hard and not too useful to be used/taught. Perhaps someone can give a proper reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My calculations (see code below) suggest, if I am not wrong, that for the equation&#xA;$$ u_t = \mathcal{L}u, \qquad \mathcal{L}u = \nabla\cdot(D\nabla u), $$&#xA;there is such a nine-point stencil $L_9$. I am going to assume the central point of the stencil is $(0,0)$ and examine that point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a quick and dirty calculation, which is why it (unusually) uses derivatives of $D$ in the stencil coefficients.&#xA;Here are its coefficients (sorry about the formatting; &lt;em&gt;also the matrices were rotated $90^\circ$ clockwise by accident (sorry!), but they are a pain to retype&lt;/em&gt;):&#xA;$$&#xA;\begin{aligned}&#xA;&amp;amp;&#xA;\frac{D}{6h^2}&#xA;\begin{pmatrix}&#xA; 1 &amp;amp; 4 &amp;amp; 1 \\&#xA; 4 &amp;amp; -20 &amp;amp; 4 \\&#xA; 1 &amp;amp; 4 &amp;amp; 1&#xA;\end{pmatrix}&#xA;\\&amp;amp;+&#xA;\frac{1}{12h}&#xA;\begin{pmatrix}&#xA;-D_y-D_x &amp;amp; -4D_x &amp;amp; D_y-D_x \\&#xA;-4D_y &amp;amp; 0 &amp;amp; 4D_y \\&#xA;-D_y + D_x &amp;amp; 4D_x &amp;amp; D_y+D_x&#xA;\end{pmatrix}&#xA;\\&amp;amp;+&#xA;\frac{1}{24D}&#xA;\begin{pmatrix}&#xA; -D_x D_y &amp;amp; -4 D_x^2-2 D_y^2 &amp;amp; D_x D_y \\&#xA; -2 D_x^2-4 D_y^2 &amp;amp; 12 D_x^2+12 D_y^2 &amp;amp; -2 D_x^2-4 D_y^2 \\&#xA; D_x D_y &amp;amp; -4 D_x^2-2 D_y^2 &amp;amp; -D_x D_y&#xA;\end{pmatrix}&#xA;\\&amp;amp;+&#xA;\frac{1}{12}&#xA;\begin{pmatrix}&#xA; D_{{xy}} &amp;amp; 3 D_{{xx}}+D_{{yy}} &amp;amp; -D_{{xy}} \\&#xA; D_{{xx}}+3 D_{{yy}} &amp;amp; -8 D_{{xx}}-8 D_{{yy}} &amp;amp; D_{{xx}}+3 D_{{yy}} \\&#xA; -D_{{xy}} &amp;amp; 3 D_{{xx}}+D_{{yy}} &amp;amp; D_{{xy}}&#xA;\end{pmatrix}&#xA;\\&amp;amp;+&#xA;\frac{h}{24D}&#xA;\begin{pmatrix}&#xA; 0 &amp;amp; D_x D_{{xx}}+D_{{xy}} D_y &amp;amp; 0 \\&#xA; D_x D_{{xy}}+D_y D_{{yy}} &amp;amp; 0 &amp;amp; -D_x D_{{xy}}-D_y D_{{yy}} \\&#xA; 0 &amp;amp; -D_x D_{{xx}}-D_{{xy}} D_y &amp;amp; 0&#xA;\end{pmatrix}&#xA;\\&amp;amp;+&#xA;\frac{h}{24}&#xA;\begin{pmatrix}&#xA;0 &amp;amp; -D_{xyy} - D_{xxx} &amp;amp; 0 \\&#xA;-D_{yyy} - D_{xxy} &amp;amp; 0 &amp;amp; D_{yyy} + D_{xxy} \\&#xA;0 &amp;amp; D_{xyy} + D_{xxx} &amp;amp; 0&#xA;\end{pmatrix}&#xA;\end{aligned}&#xA;$$&#xA;It's much like the original stencil, but with some corrections. Unfortunately, they look a bit awkward to me, so I can't say much about them beyond saying that they satisfy the equations I need them to satisfy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I calculated the coefficient by brute force: by writing down the linear equations satisfied by the stencil coefficients, such that the approximation at the point $(0,0)$ would satisfy&#xA;$$ L_9u = \mathcal{L}u + \beta h^2\nabla^2\mathcal{L}u + \gamma h^2 \mathcal{L}\mathcal{L}u + O(h^4). $$&#xA;The point is that this kind of &lt;em&gt;method of deferred corrections&lt;/em&gt; only works if in the equation $\mathcal{L}u = \cdots$ the first error term of order $h^2$ looks like $h^2\mathcal{M}\mathcal{L}u$ for some suitable operator $\mathcal{M} = \beta \nabla^2 + \gamma \mathcal{L}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then calculated the error term coefficients&#xA;$$ \beta=\frac{1}{6}, \qquad \gamma = \frac{-1}{12D(0,0)}. $$&#xA;The corresponding modified equation after space discretization is&#xA;$$ \left(1 + \frac{h^2}6\nabla^2 - \frac{h^2}{12D(0,0)}\mathcal{L} \right) u_t = L_9 u. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code that I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(* VarCoefNinePointStencil.nb *)&#xA;&#xA;Clear[ncdiff, ncdiff2]&#xA;ncdiff[d_, u_] := &#xA; D[d[x, y] D[u[x, y], x], x] + D[d[x, y] D[u[x, y], y], y]&#xA;ncdiff2[d_, u_] := D[d[x, y] D[u, x], x] + D[d[x, y] D[u, y], y]&#xA;&#xA;Module[{L9, Lapprox, trunc, conds, sol, target, errTerm1, errTerm2, &#xA;  uvars, remainder, remCond, remSol},&#xA; L9 = Table[Subscript[\[Alpha], i, j], {i, -1, 1}, {j, -1, 1}];&#xA; Lapprox = Normal@Series[&#xA;    Sum[L9[[i + 2, j + 2]] u[i h, j h], {i, -1, 1}, {j, -1, 1}]&#xA;    , {h, 0, 4}];&#xA; uvars = {Derivative[a_, b_][u][0, 0] :&amp;gt; Subscript[u, a, b], &#xA;   u[0, 0] -&amp;gt; Subscript[u, 0, 0]};&#xA; Lapprox = Lapprox /. uvars;&#xA; errTerm1 = ncdiff[d, u];&#xA; errTerm2 = ncdiff[d, u];&#xA; errTerm1 = D[errTerm1, x, x] + D[errTerm1, y, y];&#xA; errTerm2 = ncdiff2[d, errTerm2];&#xA; target = &#xA;  ncdiff[d, u] + h^2 \[Beta] errTerm1 + &#xA;    h^2 \[Gamma] errTerm2 /. {x -&amp;gt; 0, y -&amp;gt; 0};&#xA; target = target /. uvars;&#xA; trunc = Lapprox - target /. {x -&amp;gt; 0, y -&amp;gt; 0};&#xA; trunc = trunc /. uvars;&#xA; conds = Flatten@&#xA;   Table[Coefficient[trunc, Subscript[u, a, b]], {a, 0, 2}, {b, 0, 2}];&#xA; Print[conds // Column];&#xA; sol = Solve[Thread[conds == 0], Flatten@L9];&#xA; If[Head[sol] === Solve || sol === {}, Print[&quot;No solution.&quot;]; Abort[]];&#xA; sol = sol[[1]];&#xA; Print[sol];&#xA; Print[Lapprox - target /. sol // Expand // Collect[#, h] &amp;amp;];&#xA; remainder = Coefficient[Lapprox - target /. sol, h^2];&#xA; remCond = &#xA;  Coefficient[remainder, #] &amp;amp; /@ {Subscript[u, 4, 0], Subscript[u, 3, &#xA;    0]};&#xA; remSol = Solve[Thread[0 == remCond], {\[Beta], \[Gamma]}];&#xA; Print[remSol];&#xA; Print[(1/(24 h^2)) MatrixForm[&#xA;    24 h^2 L9 //. Join[sol, remSol[[1]]] // Expand]];&#xA; Print[Lapprox - target //. Join[sol, remSol[[1]]] // &#xA;   Collect[#, h, Factor] &amp;amp;];&#xA; Collect[remainder, &#xA;  Union@Cases[remainder, Subscript[u, _, _], \[Infinity]], Factor]&#xA; ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-08-12T22:05:57.747" LastActivityDate="2015-08-12T22:05:57.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20347" PostTypeId="2" ParentId="20345" CreationDate="2015-08-01T01:43:29.270" Score="0" Body="&lt;p&gt;I derived my variable 9-point stencil but I'm not sure if its right. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hundsurfer in his book has a variable coefficient case for the 5-point stencil:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_t = \nabla(\cdot(d(x,y)\nabla) f) = \frac{d_{i-1/2,j}(f_{i-1,j}-f_{i,j})-d_{i-1/2,j}(f_{i,j}-f_{i+1,j})}{(\Delta h^2)}+ \frac{d_{i,j-1/2}(f_{i,j-1}-f_{i,j})-d_{i,j+1/2}(f_{i,j}-f_{i,j+1})}{(\Delta h^2)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems he just assumes four &quot;diffusion&quot; fluxes coming from the four edges of the cell, where the diffusion coefficient is calculated at the edge of the cell we're calculating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the limit of $d_{i,j} \rightarrow d$, i.e. where d is constant, we get the standard 5-point stencil that is in books, the internet etc:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d}{h^2}&#xA;\begin{pmatrix}&#xA;  &amp;amp; 1 &amp;amp;  \\&#xA; 1 &amp;amp; -4 &amp;amp; 1 \\&#xA;  &amp;amp; 1 &amp;amp; &#xA;\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I naively tried a similar interpolation for the 9-point stencil, where diffusion fluxes are entering the cell from 8 directions i.e. two horizontal, two vertical, and four diagonal. I used the same weights as the constant coefficient 9-point stencil:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_t = \nabla(\cdot(d(x,y)\nabla) f) = 4\times \frac{d_{i-1/2,j}(f_{i-1,j}-f_{i,j})-d_{i+1/2,j}(f_{i,j}-f_{i+1,j})}{(6\Delta h^2)}+ 4\times\frac{d_{i,j-1/2}(f_{i,j-1}-f_{i,j})-d_{i,j+1/2}(f_{i,j}-f_{i,j+1})}{(6\Delta h^2)}+\frac{d_{i-1/2,j-1/2}(f_{i-1,j-1}-f_{i,j})}{(6\Delta h^2)}+\frac{d_{i+1/2,j-1/2}(f_{i+1,j-1}-f_{i,j})}{(6\Delta h^2)}+\frac{d_{i-1/2,j+1/2}(f_{i-1,j+1}-f_{i,j})}{(6\Delta h^2)}+\frac{d_{i+1/2,j+1/2}(f_{i+1,j+1}-f_{i,j})}{(6\Delta h^2)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I use a midpoint rule to calculate the diffusion at edges e.g.&#xA;$d_{i+1/2,j+1/2}=0.5(d_{i+1,j+1}+d_{i,j})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you take the limit where $d(x,y) \rightarrow d$ you recover your classical 9-point stencil for constant coefficient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{D}{6h^2}&#xA;\begin{pmatrix}&#xA; 1 &amp;amp; 4 &amp;amp; 1 \\&#xA; 4 &amp;amp; -20 &amp;amp; 4 \\&#xA; 1 &amp;amp; 4 &amp;amp; 1&#xA;\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if my naive reasoning is right, but it gave me sensible simulation results, e.g. a circular profile &quot;diffuses&quot; symmetrically as a sphere without deformations. Can anyone comfirm if my reasoning is right or wrong? I'm still a noob. &lt;/p&gt;&#xA;" OwnerUserId="17075" LastEditorUserId="17075" LastEditDate="2015-08-01T02:01:39.447" LastActivityDate="2015-08-01T02:01:39.447" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20348" PostTypeId="2" ParentId="17494" CreationDate="2015-08-01T06:44:48.597" Score="1" Body="&lt;p&gt;PLY is a generic file format that can store any mesh entities with information attached to it. It is mostly used to represent surfacic meshes, and this is what most softwares that use this file format will expect (including Tetgen), but in principle it can be extended to store tetrahedral meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to store tetrahedral meshes in PLY files and use Tetgen to manipulate them, you will need to modify Tetgen or use it as a library (Tetgen is implemented in C++).&#xA;An example of using tetgen as a library is the GEOGRAM library that I'm developping:&#xA;&lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reading and storing PLY files, I am using the following library:&#xA;&lt;a href=&quot;http://w3.impa.br/~diego/software/rply/&quot; rel=&quot;nofollow&quot;&gt;http://w3.impa.br/~diego/software/rply/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do not plan to modify Tetgen / use it as a library, another possibility is to write a file converter that converts between 'tetrahedral mesh stored in PLY file' and native tetrahedral meshes formats supported by tetgen.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2015-08-01T11:57:39.490" LastActivityDate="2015-08-01T11:57:39.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20349" PostTypeId="1" CreationDate="2015-08-01T07:33:47.557" Score="4" ViewCount="216" Body="&lt;p&gt;Given the following system of &lt;strong&gt;partial integro-differential equations&lt;/strong&gt;&#xA;$$ \frac{dX(t)}{dt}=\Lambda-\mu X(t)-\beta X(t)Z(t),\\&#xA;\frac{\partial Y(t,\omega)}{\partial t}+\frac{\partial Y(t,\omega)}{\partial \omega}=-\delta(\omega) Y(t,\omega),\\&#xA;\frac{dZ(t)}{dt}=-\gamma Z(t)+(N-Z(t))\int^{\infty}_{0}\kappa (\omega)Y(t,\omega)d\omega $$&#xA;with the &lt;strong&gt;boundary condition&lt;/strong&gt; &#xA;$$Y(t,0)=\beta X(t)Z(t)$$ and&#xA;&lt;strong&gt;initial conditions&lt;/strong&gt; &#xA;$$X(0)=X_{0},Y(0,\omega)= \sigma(\omega),Z(0)=Z_{0} $$ &#xA;where all parameters $\Lambda, \mu, \beta, \delta(\omega), \gamma, N, \kappa(\omega), X_{0},Z_{0}$ are positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How does one numerically compute a solution for this on Matlab?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; : I was asked to repost this question in scicomp.se -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/1365355/numerically-solving-a-system-of-partial-integro-differential-equations-in-matlab&quot;&gt;Numerically solving a system of partial integro-differential equations in Matlab &lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17122" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-08-01T16:59:09.703" Title="Numerically solving a system of partial integro-differential equations in Matlab" Tags="|matlab|pde|numerical-analysis|integral-equations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20350" PostTypeId="2" ParentId="20305" CreationDate="2015-08-01T09:22:26.817" Score="1" Body="&lt;p&gt;You could use a stencil to do a multidimensional least-squares fit which will account for transverse components.  I've seen this done in finite volume before, [Weller et al. 2009, Weller &amp;amp; Shahrokhi 2014] give details of using upwind-biased stencil on unstructured two-dimensional meshes, but the same techniques can be straightforwardly applied to uniform Cartesian meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen this technique used with flux-form advection that gives stable results, but my own tests with advective form suggest that such a scheme is not stable, likely due to &lt;a href=&quot;https://en.wikipedia.org/wiki/Godunov%27s_theorem&quot; rel=&quot;nofollow&quot;&gt;Godunov's theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[Weller et al. 2009] Voronoi, Delaunay, and block-structured mesh refinement for solution of the shallow-water equations on the sphere. Monthly Weather Review, 137, 4208-4224, &lt;a href=&quot;http://dx.doi.org/10.1175/2009MWR2917.1&quot; rel=&quot;nofollow&quot;&gt;dx.doi.org/10.1175/2009MWR2917.1&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;[Weller &amp;amp; Shahrokhi 2014] Curl-free pressure gradients over orography in a solution of the fully compressible Euler equations with implicit treatment of acoustic and gravity waves. Monthly Weather Review, 142, 4439-4457, &lt;a href=&quot;http://dx.doi.org/10.1175/MWR-D-14-00054.1&quot; rel=&quot;nofollow&quot;&gt;dx.doi.org/10.1175/MWR-D-14-00054.1&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7192" LastActivityDate="2015-08-01T09:22:26.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20351" PostTypeId="1" CreationDate="2015-08-01T14:17:28.977" Score="0" ViewCount="1188" Body="&lt;p&gt;FEA when applied to solid mechanics / structural engineering result in the solution to the equation &#xA;$$ F = K\cdot x $$&#xA;$F$ being the forces, $K$ the stiffness of the system and $x$ represents the resulting displacement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if I were to analyse a live experiment where, lets say, I am applying a point load to a beam, gradually increasing in magnitude. As the load increases the deflection in the beam increases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using some sensors and data-acquisition system, if I were to read this load, into a FE program, then I would have, in the equation above, a scenario of load  which updates, every few seconds, and this should result in new solution to the FE equation. Also, the actual displacement values can be read using a displacement sensor and fed to the FE program for validation / model updation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This creates kind of a loop, and completes the definition of real-time FE solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would such a system have any practical use?&lt;/p&gt;&#xA;" OwnerUserId="6981" LastEditorUserId="713" LastEditDate="2015-08-01T17:00:25.333" LastActivityDate="2015-08-01T17:00:25.333" Title="Are there any possible applications of real-time Finite Element Analysis?" Tags="|finite-element|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20352" PostTypeId="2" ParentId="20351" CreationDate="2015-08-01T14:42:26.807" Score="4" Body="&lt;p&gt;There are hundreds of reasons you might want to do this. Any real-time control system can use an accurate model of the system being controlled to improve the predictor part of the controller. Weather forecasts need to run faster than real time to be useful. I don't know of anyone doing weather with FEM off the top of my head, but you easily could. Etc. Etc.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-08-01T14:42:26.807" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20353" PostTypeId="2" ParentId="20351" CreationDate="2015-08-01T16:45:22.927" Score="0" Body="&lt;p&gt;I think that what you describe is more-or-less how some tanks are able to fire while moving.  They have a differential model of the turret, sensors and actuators.&lt;/p&gt;&#xA;" OwnerUserId="16799" LastActivityDate="2015-08-01T16:45:22.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20354" PostTypeId="2" ParentId="20279" CreationDate="2015-08-01T21:26:15.090" Score="6" Body="&lt;p&gt;I think that you have two problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;your timestep is too big (the CFL condition is not satisfied); and&lt;/li&gt;&#xA;&lt;li&gt;you are not updating the values for $\phi=2\pi$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am not sure about the CFL condition for polar coordinates, but I believed that it should be&#xA;$$2c\frac{\Delta t}{\Delta r \Delta \phi} \leq C_\max \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what implies that $\Delta t &amp;lt; \frac{\Delta r \Delta \phi}{2c}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the values for $\phi=2\pi$, you need to update for $\phi=0$ and $\phi=2\pi$ although you are not looping until the last angular value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These things are corrected in the code below (and it works). One thing that is not working terribly good is the solution for $r=0$... but I have no ideas now on how solve it. But I think that it needs to be solved as a particular case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attach some images for the mode $(1,1)$ of vibration (since I am not good with the animation thing in Python).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.special import jv, jn_zeros&#xA;from mayavi import mlab&#xA;&#xA;&#xA;#%% Parameters&#xA;Nr = 50&#xA;N_phi = 50&#xA;N_steps = 1000&#xA;radius = 5.&#xA;c = 0.5&#xA;dphi = 2*np.pi/N_phi&#xA;dr = 5./Nr&#xA;dt = 0.005  &#xA;if dt&amp;lt; dr*dphi/2/c:&#xA;    # The maximum value for dt is dr*dphi/(2*c)&#xA;    dt = dr*dphi/(4*c)&#xA;&#xA;#%% Initial conditions&#xA;r = np.linspace(0, radius, Nr)&#xA;phi = np.linspace(0, 2*np.pi, N_phi)&#xA;R, phi = np.meshgrid(r, phi)&#xA;X = R*np.cos(phi)&#xA;Y = R*np.sin(phi)&#xA;kth_zero = jn_zeros(1, 1)&#xA;Z = np.cos(phi) * jv(1, kth_zero*R/radius)&#xA;T = np.zeros((N_steps, Nr, N_phi))&#xA;T[0, :, :] = Z.T&#xA;T[1, :, :] = Z.T&#xA;&#xA;#%% Stepping&#xA;k1 = c*dt**2/dr**2&#xA;for t in range(2, N_steps):&#xA;    for i in range(0, Nr-1):&#xA;        for j in range(0, N_phi-1):&#xA;            ri = max(r[i], 0.5*dr)  # To avoid the singularity at r=0&#xA;            k2 = c*dt**2/(2*ri*dr)&#xA;            k3 = c*dt**2/(dphi*ri)**2&#xA;            T[t, i, j] = 2*T[t-1, i, j] - T[t-2, i, j] \&#xA;            + k1*(T[t-1, i+1, j] - 2*T[t-1, i, j] + T[t-1, i-1, j])\&#xA;            + k2*(T[t-1, i+1, j] - T[t-1, i-1, j])\&#xA;            + k3*(T[t-1, i, j+1] - 2*T[t-1, i, j] + T[t-1, i, j-1])&#xA;&#xA;        T[t, i, -1] = T[t, i, 0]  # Update the values for phi=2*pi&#xA;&#xA;&#xA;#%%&#xA;surf = mlab.mesh(X.T, Y.T, 10*T[999], colormap='RdYlBu')&#xA;mlab.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Timestep: 0&#xA;&lt;a href=&quot;https://i.stack.imgur.com/6GYkA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6GYkA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Timestep: 199&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3HH39.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3HH39.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Timestep: 399&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3MpW3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3MpW3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Timestep: 599&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HCLgU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HCLgU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Timestep: 799&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4HQpl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4HQpl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Timestep: 999&#xA;&lt;a href=&quot;https://i.stack.imgur.com/uvpln.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uvpln.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: Please, generate videos and share it in the comments (I want to see). And, use a different colormap, the one you used is not so good.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-03-04T17:27:42.050" LastActivityDate="2017-03-04T17:27:42.050" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20355" PostTypeId="2" ParentId="20343" CreationDate="2015-08-01T22:52:56.643" Score="5" Body="&lt;p&gt;&lt;em&gt;The gist of this answer is that this problem fits into a general framework that is part of machine learning.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this can be done using the standard machinery of clustering / filtering problems in statistics, which are pretty standard. There's a lot of literature on this, so I'm going to present just the basic outline of how this can be done: it's easy to write down a basic model and then improve on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the set of observations be $O=\{o_n\}_n$, with each observation $o = (t, x)$ consisting of time and location. Let $\Lambda$ be the set of all objects' labels (fixed for now). We can write down the likelihood function for all the observations given a specific assignment of labels, and then try to find the assignment of object labels that maximizes log-likelihood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This translates to finding that assignment of labels to observations that is the most likely under some statistical model of objects' behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The likelihood function is, given an assignment of labels $\{\lambda_n\}_n$, the p.d.f. of those observations given this assignment:&#xA;$$ L(\{\lambda_n\}) = \mathbb{P}(o_1,\ldots,o_n\mid \{\lambda_n\}). $$&#xA;We can try to factor this likelihood function as&#xA;$$ L(\{\lambda_n\}) = \prod_\lambda \mathbb{P}(o^\lambda_1) \mathbb{P}(o^\lambda_2\mid o^\lambda_1)\cdots\mathbb{P}(o^\lambda_k \mid o^\lambda_{k-1},o^\lambda_{k-2},\ldots,o^\lambda_1), $$&#xA;where the product goes over all assigned object labels $\lambda$, and each term is the likelihood of observing the same object $\lambda$ at different observations $o^\lambda_1,\ldots,o^\lambda_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To write down the probabilities we can try to model their motion as constant-velocity motion, so that&#xA;$$ \mathbb{P}(o^\lambda_j \mid o^\lambda_{j-1},o^\lambda_{j-2},\ldots) $$&#xA;can be written in terms of $(t_j,x_j), (t_{j-1},x_{j-1}), (t_{j-2},x_{j-2})$ using (say) the normal distribution&#xA;$$ N\left(x_j \mid x_{j-1} + (t_j - t_{j-1})\frac{x_{j-1}-x_{j-2}}{t_{j-1}-t_{j-2}}, \xi \right). $$&#xA;This says that after observing object $\lambda$ at $o^\lambda_{j-2}$, then $o^\lambda_{j-1}$ the expected next location is at $x_{j-1}+\delta t\cdot v$, with some variance $\xi$. For the initial observations one might write down&#xA;$$ \mathbb{P}(o^\lambda_1) = \mathrm{const}, \qquad \mathbb{P}(o^\lambda_2|o^\lambda_1) = N(x_2 \mid x_1, (t_2-t_1)\eta), $$&#xA;where $\mathrm{const}$ is a p.d.f. (up to a multiplicative constant that will drop out later) that is like a penalty for introducing a new object label, and where $\eta$ represents uncertainty in how much an object might move in one time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once this is all done, $-\log L$ becomes a sum of squared errors (although a bit laborious to write out in full) in the observations given an assignment of labels. This sum of squared errors (negative log-likelihood) can be minimized by a simple iterative process: walk through observations in chronological order, and for each next observation pick a label (existing or new) that results in the smallest sum of squared errors over observations so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might already be good enough. If not, you might consider equipping each observation with a hidden state, such as $(t, x, v, a)$ where only $t$ and $x$ are directly observed, and each object also has velocity $v$ and acceleration $a$ that need to be &lt;em&gt;inferred&lt;/em&gt;. You might also have to consider more carefully what the optimal assignment of labels is (the above strategy is a greedy assignment; a common method is the EM algorithm), and how to best determine the total number of objects (e.g., introduce a penalty into $-\log L$ for the total number of objects).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I described here is a kind of Bayesian filtering approach (this is too long already). But this is a pretty standard problem: you have a hidden state (assignment of labels, velocities, etc.), and a statistical model of how that hidden state produces the observations that you have (the likelihood function), and you are trying to &lt;em&gt;infer&lt;/em&gt; the hidden state from the observations (by maximization of log-likelihood). Textbooks that discuss machine learning, Bayesian inference, graphical models would be helpful here.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-08-03T19:11:56.743" LastActivityDate="2015-08-03T19:11:56.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20356" PostTypeId="1" CreationDate="2015-08-02T11:05:31.207" Score="10" ViewCount="235" Body="&lt;p&gt;I am working on solving complex system of 2D PDEs governing the behaviour of plasma in a gas lamp during discharge. Recent tests have shown that because of steep gradients in temperature field and overall explosive nature of a process I won't go anywhere further without adding gas dynamics to the model. As far as I know from my mentor, there will be supersonic shocks traveling across the 5mm lamp. As far as I understand, to even think about somewhat accurate solution I first have to implement some kind of mesh refinement algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read Berger and Colella's LAMR for Shock Hydrodynamics, Colella's short course of block-structured AMR  at Berkley, Structured AMR by Vakili and Martin, Colella, Anghel and Alexander's AMR for Multiscale Nonequilibrium physics. So I understand that there have to be performed somewhat untrivial operations during the interpolation from coarse grid onto finer ones, synchronisations between halfsteps of finer grids, averaging of finer grids onto coarse ones and some kind of flux correction to preserve conservation laws.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That brings to the question: What exactly happens during the time step in Adaptive Mesh Refinement algorithm? I would appreciate full explanation, but here are some specific questions that are likely to arise either way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider parabolic problem on two-level grid, $L_0$ &amp;amp; $L_1$, $L_0$ being the coarse one.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it correct that to get least amount of errors, I have to first advance L0 problem by time step $\tau$ as if there was no mesh refinement at all, then interpolate resulting $L_0$ data to get boundary conditions for $L_1$ problem at $t+\frac{\tau}{2}$ and advance $L_1$ for $\frac{\tau}{2}$, then interpolate L0 data for $L_1$ problem at $t + \tau$ and advance $L_1$ fol half-step again, then average $L_1$ data onto L0 grid and correct $L_0$ points adjacent to L1 boundary for flux conservation using $L_1$ data? Do I understand correctly that those flux fixups do not propagate further into coarse grid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.a. During correction of $L_0$ data using $L_1$, do I recalculate all $L_1$-bordering L0 points at the same time, or am I allowed to use corrected data on $L_0$ $X$-point alongside with data averaged from $L_1$ grid during my correcting of $Y$-point adjacent to $X$? (To illustrate: I am talking about something similar to difference between synchronous and asynchronous Hopfield networks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.b. If it is even possible, how do I synchronize grids for elliptic task? Radiative transfer is a large part of my system, too.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I generate boundary conditions on fine-fine grid boundary?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.a. In some of the papers I've read I noticed the phrase similar to &quot;If possible, boundary conditions are taken from the same-level grid or finer&quot;. How can this be, if at the moment of me updating the first of $L_k$ grids I do not yet have any data from the other $L_k$ or $L_{k+1}$ grids?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.b. If the very concept of fine-fine boundary is not possible or leads to significant errors, then do I have to merge those contacting fine grids and solve the problem on the non-rectangular grid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.c. If I am to solve for the non-rectangular grid, what SLAE solvers besides iterative are good for sparse matrices? I was using Thomas algorithm modified for block-tridiagonal matrices, but as far as I understand, matrix for generic non-rectangular grid won't be tridiagonal.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17130" LastEditorUserId="9667" LastEditDate="2015-08-02T18:39:09.003" LastActivityDate="2015-08-02T18:39:09.003" Title="Time advance in Adaptive Mesh Refinement method" Tags="|pde|adaptive-mesh-refinement|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20360" PostTypeId="1" CreationDate="2015-08-03T12:23:51.327" Score="1" ViewCount="226" Body="&lt;p&gt;Is there a Matlab interface for minpack2?&#xA;minpack2 &lt;a href=&quot;http://ftp.mcs.anl.gov/pub/MINPACK-2&quot; rel=&quot;nofollow&quot;&gt;http://ftp.mcs.anl.gov/pub/MINPACK-2&lt;/a&gt; consists of Fortran programs , but i would like to call the test problems from matlab.&#xA;What can i do ?&lt;/p&gt;&#xA;" OwnerUserId="17137" LastActivityDate="2018-03-12T02:08:05.070" Title="Matlab interface for minpack2" Tags="|matlab|least-squares|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20362" PostTypeId="2" ParentId="135" CreationDate="2015-08-03T23:03:38.557" Score="4" Body="&lt;p&gt;I'm happy to answer that there's a high-quality open source code for this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mcodev31/libmsym&quot; rel=&quot;nofollow&quot;&gt;https://github.com/mcodev31/libmsym&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;libmsym is a C library dealing with point group symmetry in molecules. It can determine, symmetrize and generate molecules of any point group. It can also generate symmetry adapted linear combinations of atomic orbitals for a subset of all point groups and orbital angular momentum (l), and project orbitals into the irreducible representation with the larges component.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have adapted libmysm into &lt;a href=&quot;http://avogadro.cc/&quot; rel=&quot;nofollow&quot;&gt;Avogadro&lt;/a&gt; and a release should come out later in August 2015.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the author is currently working on finishing a manuscript about the details. I'll revise this answer when it's published.&lt;/p&gt;&#xA;" OwnerUserId="14517" LastActivityDate="2015-08-03T23:03:38.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20363" PostTypeId="2" ParentId="19626" CreationDate="2015-08-03T23:16:45.007" Score="3" Body="&lt;p&gt;In statistics the widely used test for checking if the distribution is gaussian is the Jarque-Bera test. Koizumi [1] presents an equivalent test for the multivariate case. I don't know if there something ready to use in C++, Fortran, or Python. However, at first sight, the test looks easy to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] - Koizumi, Kazuyuki, Naoya Okamoto, and Takashi Seo. &quot;On Jarque-Bera tests for assessing multivariate normality.&quot; Journal of Statistics: Advances in Theory and Applications 1.2 (2009): 207-220. &lt;a href=&quot;http://scientificadvances.co.in/admin/img_data/166/images/[5]%20JSATA%2090702%20Kazuyuki%20Koizumi%20et%20al.%20207-220.pdf&quot; rel=&quot;nofollow&quot;&gt;(pdf link)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17141" LastEditorUserId="7733" LastEditDate="2015-08-03T23:45:31.297" LastActivityDate="2015-08-03T23:45:31.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20366" PostTypeId="1" CreationDate="2015-08-04T08:24:51.023" Score="5" ViewCount="189" Body="&lt;p&gt;I'm part of a team trying to generalize a 1D Advection-Diffussion-Reaction code we inherited by extending it to 2D by using dimensional splitting, i.e. solving advection and diffusion for x and y separately with 1D schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Diffusion was (relatively) painless but there are issues with advection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Originally the code had a flux limited, third order upwind-biased scheme for 1D advection which worked like a charm in the 1D case. However, when the same scheme is extended to 2D, crazy instabilities develop. What supposed to be a radial advection-diffusion profile wrinkles in the transverse direction and shoots a &quot;jet-like&quot; instability:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[&lt;img src=&quot;https://i.stack.imgur.com/lgWnH.png&quot; alt=&quot;Instability[1]&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when I use a first order upwind scheme I do get a nice, symmetric radial profile.  The problem with the first order upwind scheme solution is the excessive false diffusion which the higher order scheme does suppress. But the higher order scheme is even more problematic because of the aforementioned instability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if higher order schemes in general are a bad idea in a multidimensional case? Are multidimensional problems more succeptible to &quot;oscillations&quot; caused by high order schemes? What about flux limiters, should 1D flux limiters prevent this oscillations from happening (in my situation, it seems the flux limiter for the third order case doesn't help)? Should I stick to lower order schemes even if they bring false diffusion? This third order scheme I was using is also flux limited, but the flux limit doesn't stop the instabilities...&lt;/p&gt;&#xA;" OwnerUserId="17075" LastActivityDate="2015-08-04T08:24:51.023" Title="Is it generally unstable to use in multidimensional simulations finite difference schemes with higher orders than 2?" Tags="|finite-difference|fluid-dynamics|finite-volume|advection-diffusion|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20367" PostTypeId="1" CreationDate="2015-08-04T09:49:15.390" Score="2" ViewCount="72" Body="&lt;p&gt;I am trying to find how much an engine would turn until it stops accelerating with an acceleration of $a = A \sin^2(x)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Acceleration&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T9aZz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T9aZz.png&quot; alt=&quot;Acceleration&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integrating the acceleration I get velocity $v = A(x/2 - \sin(2x)/4) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integrating the velocity, I get the distance $d = A(x^2/4 - \cos^2(x)/4) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Velocity and distance&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hOdX5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hOdX5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I need to do this calculation on a microcontroller, without using floating point and with a sine look-up table which is basically a $2048\sin(x\pi/512)$ function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I find the distance correctly using the sine look up table? Or what other way could I use?&lt;/p&gt;&#xA;" OwnerUserId="17143" LastEditorUserId="9667" LastEditDate="2015-08-04T17:35:15.773" LastActivityDate="2015-08-04T17:35:15.773" Title="Method for calculating the stopping distance using only integers" Tags="|computational-physics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20368" PostTypeId="1" CreationDate="2015-08-04T10:37:18.123" Score="1" ViewCount="129" Body="&lt;p&gt;Hi I have made a 3D alternating direction explicit scheme for solving the diffusion equation, which will eventually replace a FTCS scheme in model of bubble dynamics in tissue. I have been testing it and have realized that despite the unconditional stability I get oscillations in the system if the Courant number is  greater than ~4. Is there a way round this other than reducing the Courant number?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some more details: The scheme I am using alternating direction explicit ADE is an explicit method which does two sweeps (one upwind and one downwind). In 1D this looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upwind:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C1_new[i]-C_old[i] = (dt/h^2)(C_old[i+1]-C_old[i]-C1_new[i]+C1_new[i-1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Downwind:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C2_new[i]-C_old[i] = (dt/h^2)(C_old[i-1]-C_old[i]-C2_new[i]+C2_new[i+1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Final Average:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C_new[i] = C1_new[i]+C2_new[i]/2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Full details can be found at &lt;a href=&quot;http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1552926&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1552926&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I implement a 3D version and play around with time and space steps I see oscillations occurring from (dt/h^2) values of about 4.5 upwards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a plot of the pt (5,5,5) from my $10\times10\times10$ grid for various Courant numbers. As you can see, there are oscillations for the higher numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/feMJX.jpg&quot; alt=&quot;&quot;&gt;]&lt;/p&gt;&#xA;" OwnerUserId="17146" LastEditorUserId="17146" LastEditDate="2015-08-10T13:05:29.567" LastActivityDate="2015-08-10T13:05:29.567" Title="Reducing oscillations a 3D Alternating direction explicit scheme for the diffusion equation?" Tags="|stability|diffusion|3d|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20369" PostTypeId="1" CreationDate="2015-08-04T21:05:39.683" Score="16" ViewCount="1275" Body="&lt;p&gt;Let &lt;code&gt;x&lt;/code&gt;, &lt;code&gt;y&lt;/code&gt; be two floating-point numbers. What's the right way to compute their mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The naive way &lt;code&gt;(x+y)/2&lt;/code&gt; can result in overflows when &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; are too large. I think &lt;code&gt;0.5 * x + 0.5 * y&lt;/code&gt; maybe better, but it involves two multiplications (which maybe is inefficient), and I am not sure if it is good enough. Is there a better way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another idea I've been playing with is &lt;code&gt;(y/2)(1 + x/y)&lt;/code&gt; if &lt;code&gt;x&amp;lt;=y&lt;/code&gt;. But again, I am not sure how to analyze this and prove that it satisfies my requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, I need a guarantee that the computed mean will be &lt;code&gt;&amp;gt;= min(x,y)&lt;/code&gt; and &lt;code&gt;&amp;lt;= max(x,y)&lt;/code&gt;. As pointed out in &lt;a href=&quot;https://scicomp.stackexchange.com/a/20379/988&quot;&gt;Don Hatch's answer&lt;/a&gt;, maybe a better way of posing this question is: What's an implementation of the mean of two numbers that always gives the most possible accurate result? That is, if &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt; are floating-point numbers, how to compute the floating-point number closest to &lt;code&gt;(x+y)/2&lt;/code&gt;? In this case, the computed mean is automatically &lt;code&gt;&amp;gt;= min(x,y)&lt;/code&gt; and &lt;code&gt;&amp;lt;= max(x,y)&lt;/code&gt;. See &lt;a href=&quot;https://scicomp.stackexchange.com/a/20379/988&quot;&gt;Don Hatch's answer&lt;/a&gt; for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; My priority is robust accuracy. Efficiency is expendable. However, if there are many robust and accurate algorithms, I would pick the most efficient.&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2019-08-18T22:05:42.597" Title="Robust computation of the mean of two numbers in floating-point?" Tags="|floating-point|" AnswerCount="7" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20371" PostTypeId="2" ParentId="20369" CreationDate="2015-08-04T22:15:18.157" Score="18" Body="&lt;p&gt;I think Higham's &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; addresses how one can analyze these types of problems. See Chapter 2, especially exercise 2.8.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this answer I'd like to point out something that isn't really addressed in Higham's book (it doesn't seem to be very widely known, for that matter). If you are interested in &lt;em&gt;proving&lt;/em&gt; properties of simple numerical algorithms such as these, you can use the power of modern SMT solvers (&lt;a href=&quot;https://en.wikipedia.org/wiki/Satisfiability_modulo_theories&quot; rel=&quot;nofollow noreferrer&quot;&gt;Satisfiability Modulo Theories&lt;/a&gt;), such as &lt;a href=&quot;https://github.com/Z3Prover/z3&quot; rel=&quot;nofollow noreferrer&quot;&gt;z3&lt;/a&gt;, using a package such as &lt;a href=&quot;https://hackage.haskell.org/package/sbv&quot; rel=&quot;nofollow noreferrer&quot;&gt;sbv&lt;/a&gt; in Haskell. This is somewhat easier than using pencil and paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I'm given that $0\leq x\leq y$, and I'd like to know if $z=(x+y)/2$ satisfies $x\leq z\leq y$. The following Haskell code &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import Data.SBV&#xA;&#xA;test1 :: (SFloat -&amp;gt; SFloat -&amp;gt; SFloat) -&amp;gt; Symbolic SBool&#xA;test1 fun =&#xA;  do [x, y] &amp;lt;- sFloats [&quot;x&quot;, &quot;y&quot;]&#xA;     constrain $ bnot (isInfiniteFP x) &amp;amp;&amp;amp;&amp;amp; bnot (isInfiniteFP y)&#xA;     constrain $ 0 .&amp;lt;= x &amp;amp;&amp;amp;&amp;amp; x .&amp;lt;= y&#xA;     let z = fun x y&#xA;     return $ x .&amp;lt;= z &amp;amp;&amp;amp;&amp;amp; z .&amp;lt;= y&#xA;&#xA;test2 :: (SFloat -&amp;gt; SFloat -&amp;gt; SFloat) -&amp;gt; Symbolic SBool&#xA;test2 fun =&#xA;  do [x, y] &amp;lt;- sFloats [&quot;x&quot;, &quot;y&quot;]&#xA;     constrain $ bnot (isInfiniteFP x) &amp;amp;&amp;amp;&amp;amp; bnot (isInfiniteFP y)&#xA;     constrain $ x .&amp;lt;= y&#xA;     let z = fun x y&#xA;     return $ x .&amp;lt;= z &amp;amp;&amp;amp;&amp;amp; z .&amp;lt;= y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will let me do this &lt;em&gt;automatically&lt;/em&gt;. Here &lt;code&gt;test1 fun&lt;/code&gt; is the &lt;em&gt;proposition&lt;/em&gt; that $x \leq \mathit{fun}(x,y) \leq y$ for all finite floats $x,y$ with $0\leq x\leq y$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;λ&amp;gt; prove $ test1 (\x y -&amp;gt; (x + y) / 2)&#xA;Falsifiable. Counter-example:&#xA;  x = 2.3089316e36 :: Float&#xA;  y = 3.379786e38 :: Float&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It overflows. Suppose I now take your other formula: $z=x/2+y/2$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;λ&amp;gt; prove $ test1 (\x y -&amp;gt; x/2 + y/2)&#xA;Falsifiable. Counter-example:&#xA;  x = 2.3509886e-38 :: Float&#xA;  y = 2.3509886e-38 :: Float&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Doesn't work (due to gradual underflow: $(x/2)\times2 \neq x$, which might be unintuitive due to all arithmetic being base-2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now try $z=x + (y-x)/2$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;λ&amp;gt; prove $ test1 (\x y -&amp;gt; x + (y-x)/2)&#xA;Q.E.D.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Works! The &lt;code&gt;Q.E.D.&lt;/code&gt; is a &lt;em&gt;proof&lt;/em&gt; that the &lt;code&gt;test1&lt;/code&gt; property holds for all floats as defined above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about the same, but restricted to $x\leq y$ (instead of $0\leq x\leq y$)?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;λ&amp;gt; prove $ test2 (\x y -&amp;gt; x + (y-x)/2)&#xA;Falsifiable. Counter-example:&#xA;  x = -3.1300826e34 :: Float&#xA;  y = 3.402721e38 :: Float&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Okay, so if $y-x$ overflows, how about $z = x + (y/2-x/2)$?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;λ&amp;gt; prove $ test2 (\x y -&amp;gt; x + (y/2 - x/2))&#xA;Q.E.D.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So it seems that among the formulas I've tried here, $x + (y/2 - x/2)$ seems to work (with a proof, too). The SMT solver approach seems to me a much quicker way of answering suspicions about simple floating-point formulas than going through floating-point error analysis with pencil and paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the goal of accuracy and stability is often at odds with the goal of performance. For performance, I don't really see how you can do better than $(x+y)/2$, especially since the compiler will still do the heavy lifting of translating this into machine instructions for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;P.S.&lt;/em&gt; This is all with single-precision IEEE754 floating-point arithmetic. I checked $x \leq x + (y/2-x/2) \leq y$ with double-precision arithmetic (replace &lt;code&gt;SFloat&lt;/code&gt; with &lt;code&gt;SDouble&lt;/code&gt;), and it works too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;P.P.S.&lt;/em&gt; One thing to bear in mind when implementing this in code is that compiler flags like &lt;code&gt;-ffast-math&lt;/code&gt; (some forms of such flags are sometimes turned on &lt;em&gt;by default&lt;/em&gt; in some common compilers) will not result in IEEE754 arithmetic, which will invalidate the above proofs. If you do use flags that enable, e.g., associative addition optimizations, then there's no point in doing anything other than $(x+y)/2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;P.P.P.S.&lt;/em&gt; I got carried away a little looking only at simple algebraic expressions without conditionals. &lt;strong&gt;Don Hatch&lt;/strong&gt;'s &lt;a href=&quot;https://scicomp.stackexchange.com/a/20379/713&quot;&gt;formula&lt;/a&gt; is strictly better.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-08-05T17:45:01.973" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20372" PostTypeId="1" CreationDate="2015-08-04T22:53:12.777" Score="1" ViewCount="82" Body="&lt;p&gt;I am dealing with following 2-dimensional problem in the unit square domain $S_2$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$- \Delta u (x,y) = f \  \text{in} \ S_2,  \hspace{1.5cm} u(x,y) = 0 \ \text{on} \  \partial S_2$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$ is such the analytical solution to problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x,y) = \cos [  2 \pi x  (y - x) ]^2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem should be discretized by 5-point finite difference scheme, which will result in Ax = b system. &#xA;I need to decompose problem into 4 non-overlaping domains. Should I divide  my domain square into 4 parts and then assemble? or assemble and then divide? Main question here is, how the laplacians will look like. &lt;/p&gt;&#xA;" OwnerUserId="17150" LastActivityDate="2015-08-04T22:53:12.777" Title="Non-overlaping Domain decomposition - assemble of Laplacian" Tags="|pde|finite-difference|preconditioning|domain-decomposition|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20373" PostTypeId="2" ParentId="20369" CreationDate="2015-08-04T23:03:11.940" Score="1" Body="&lt;p&gt;Convert to higher precission, add the values there and convert back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There should be no overflow in the higher precission and if both are in the valid floating point range, the calculated number should be inside too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it should be in between them, worst case just half of the larger number if the precission is not sufficient.&lt;/p&gt;&#xA;" OwnerUserId="17151" LastActivityDate="2015-08-04T23:03:11.940" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20374" PostTypeId="2" ParentId="20369" CreationDate="2015-08-04T23:45:26.980" Score="1" Body="&lt;p&gt;Theoretically, &lt;code&gt;x/2&lt;/code&gt; can be computed by subtracting 1 from the mantissa. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, actually implementing bitwise operations like this is not necessarily straightforward, particularly if you don't know the format of your floating point numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can do this, the entire operation is reduced to 3 addition/subtractions, which should be a significant improvement.&lt;/p&gt;&#xA;" OwnerUserId="17112" LastActivityDate="2015-08-04T23:45:26.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20376" PostTypeId="1" AcceptedAnswerId="20378" CreationDate="2015-07-22T15:24:39.447" Score="3" ViewCount="211" Body="&lt;p&gt;I want to numerically (with Matlab) solve Poisson's equation : $ \frac{\partial^2u}{\partial x^2} + \frac{\partial^2u}{\partial y^2} = f(x,y)$ On a rectangular domain using experimental data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From experimental data let's say that I have a $ [1:m \times 1:n] $ matrix that represents the right hand side of the equation. From experimental data I have access to $ \nabla(u)$ which is also a $ [1:m \times 1:n] $ matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is :&lt;/strong&gt; Is it mathematically/physically correct to reduce my mesh to a size of $ [2:m-1 \times 2:n-1] $ and to take $ \nabla(u)(1,:) $, $ \nabla(u)(n,:)$, $ \nabla(u)(:,1)$, $ \nabla(u)(:,n)$ as my Neumann boundary conditions ? Or am I missing something ?&lt;/p&gt;&#xA;" OwnerDisplayName="Luc Rebillout" LastActivityDate="2015-08-05T07:26:47.580" Title="Boundary conditions for solving Poisson's Equation with Experimental Data" Tags="|computational-physics|boundary-conditions|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="20377" PostTypeId="2" ParentId="20376" CreationDate="2015-08-04T18:15:55.160" Score="1" Body="&lt;p&gt;Yes, that is something similar to what I did while implementing a finite difference (FD) code for an electrostatic problem. I'll just present what it did roughly, the computational efficiency can be increased of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach was a bit different, as to be able to simulate more complex geometries. The important matrices for the method were:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A matrix representing the geometry, just to define the edge nodes&lt;/li&gt;&#xA;&lt;li&gt;A matrix that contained the actual points which needed to be calculated (everything within the edge nodes, excluding the edge nodes)&lt;/li&gt;&#xA;&lt;li&gt;A matrix which contained the boundary conditions, whereby you can create it as a matrix of data structures to represent all possible boundaries. The elements are just the edge nodes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When the element that is calculated happens to be on to the boundary, it looks for the type of boundary condition on the neighboring edge. It can contain either a Dirichlet, Neumann, periodic or whatever you invent kind of boundary condition. The details of the condition can be saved in a data structure, to make it easy to impose all kinds of boundaries. Getting the data out of the structure and into your calculation should be easy at this point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It worked, I got the right field solutions!&lt;/p&gt;&#xA;" OwnerUserId="16957" OwnerDisplayName="Kurtovic" LastActivityDate="2015-08-04T18:15:55.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20378" PostTypeId="2" ParentId="20376" CreationDate="2015-08-05T07:24:30.613" Score="0" Body="&lt;p&gt;I eventually found out how to procede.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the work of Dr. De Kat, whose method was exactly the one I wanted to implement. It is explained in the pdf file here  &lt;a href=&quot;http://repository.tudelft.nl/view/ir/uuid:be00fb31-71cb-4859-ab79-67879caadc3c/&quot; rel=&quot;nofollow&quot;&gt;http://repository.tudelft.nl/view/ir/uuid:be00fb31-71cb-4859-ab79-67879caadc3c/&lt;/a&gt; (starting page 49)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To him the best way is to have a region of the flow where Bernoulli's equation can be applied but he does use the Navier-Stokes equation which yields the pressure gradient as his neumann boundary conditions. At least one point must be exactly known in order to procede to a shift of the data after computation.&lt;/p&gt;&#xA;" OwnerDisplayName="Luc Rebillout" LastActivityDate="2015-08-05T07:24:30.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20379" PostTypeId="2" ParentId="20369" CreationDate="2015-08-05T08:03:42.707" Score="8" Body="&lt;p&gt;First, observe that if you have a method that gives a most accurate answer in all cases,&#xA;then it will satisfy your required condition.&#xA;(Note that I say &lt;em&gt;a&lt;/em&gt; most accurate answer rather than &lt;em&gt;the&lt;/em&gt; most accurate answer, since there may be two winners.)&#xA;Proof: If, to the contrary, you have an accurate-as-possible answer that does &lt;em&gt;not&lt;/em&gt; satisfy the&#xA;required condition, that means either &lt;code&gt;answer&amp;lt;min(x,y)&amp;lt;=max(x,y)&lt;/code&gt; (in which case &lt;code&gt;min(x,y)&lt;/code&gt; is a better answer,&#xA;a contradiction), or &lt;code&gt;min(x,y)&amp;lt;=max(x,y)&amp;lt;answer&lt;/code&gt; (in which case &lt;code&gt;max(x,y)&lt;/code&gt; is a better answer, a contradiction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think that means your question boils down to finding a most accurate possible answer.&#xA;Assuming IEEE754 arithmetic throughout, I propose the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if max(abs(x),abs(y)) &amp;gt;= 1.:&#xA;    return x/2. + y/2.&#xA;else:&#xA;    return (x+y)/2.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My argument that this gives a most accurate answer is a somewhat tedious case analysis.&#xA;Here goes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Case &lt;code&gt;max(abs(x),abs(y)) &amp;gt;= 1.&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Subcase neither x nor y is denormalized:&#xA;In this case the computed answer &lt;code&gt;x/2.+y/2.&lt;/code&gt; manipulates the same mantissas&#xA;and therefore gives the exact same answer as computation of &lt;code&gt;(x+y)/2&lt;/code&gt; would yield if we assumed extended exponents to prevent overflow.  This answer may depend on rounding mode but in any case it's guaranteed by IEEE754 to be a best possible answer (from the fact that the computed &lt;code&gt;x+y&lt;/code&gt; is guaranteed to be a best approximation to mathematical x+y, and the division by 2 is exact in this case).&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Subcase x is denormalized (and so &lt;code&gt;abs(y)&amp;gt;=1&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;answer = x/2. + y/2.&#xA;       = y/2. since abs(x/2.) is so tiny compared to abs(y/2.)&#xA;       = the exact mathematical value of y/2&#xA;       = a best possible answer.&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Subcase y is denormalized (and so &lt;code&gt;abs(x)&amp;gt;=1&lt;/code&gt;):&#xA;analogous.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Case &lt;code&gt;max(abs(x),abs(y)) &amp;lt; 1.&lt;/code&gt;:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Subcase the computed &lt;code&gt;x+y&lt;/code&gt; is either non-denormalized or denormalized-and-&quot;even&quot;:&#xA;Although the computed &lt;code&gt;x+y&lt;/code&gt; may not be exact, it is guaranteed by IEEE754 to be&#xA;a best possible approximation to the mathematical x+y.&#xA;In this case the subsequent division by 2 in the expression &lt;code&gt;(x+y)/2.&lt;/code&gt; is exact,&#xA;so the computed answer &lt;code&gt;(x+y)/2.&lt;/code&gt; is a best possible approximation to the mathematical (x+y)/2.&lt;/li&gt;&#xA;&lt;li&gt;Subcase the computed &lt;code&gt;x+y&lt;/code&gt; is denormalized and &quot;odd&quot;:&#xA;In this case exactly one of x,y must also be denormalized-and-&quot;odd&quot;,&#xA;which means the other of x,y is denormalized with the opposite sign,&#xA;and so the computed &lt;code&gt;x+y&lt;/code&gt; is exactly the mathematical x+y,&#xA;and so the computed &lt;code&gt;(x+y)/2.&lt;/code&gt; is guaranteed by IEEE754 to be a best possible approximation&#xA;to the mathematical (x+y)/2.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17156" LastActivityDate="2015-08-05T08:03:42.707" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20380" PostTypeId="2" ParentId="20369" CreationDate="2015-08-05T09:22:04.807" Score="2" Body="&lt;p&gt;Although it may not be super-efficient performance-wise, there is a very simple way to &lt;strong&gt;(1)&lt;/strong&gt; make sure none of the numbers are greater than either &lt;code&gt;x&lt;/code&gt; or &lt;code&gt;y&lt;/code&gt; (no overflows) and &lt;strong&gt;(2)&lt;/strong&gt; keep the floating point as &quot;accurate&quot; as possible (and &lt;strong&gt;(3)&lt;/strong&gt;, as an added bonus, even though subtraction is being used, no values will ever be stored as negative numbers.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float difference = max(x, y) - min(x, y);&#xA;return min(x, y) + (difference / 2.0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In fact, if you &lt;em&gt;really&lt;/em&gt; want to go for accuracy, you don't even need to perform the division on the spot; just return the values of &lt;code&gt;min(x, y)&lt;/code&gt; and &lt;code&gt;difference&lt;/code&gt; which you can use to simplify logically or manipulate later.&lt;/p&gt;&#xA;" OwnerUserId="17162" LastEditorUserId="17162" LastEditDate="2015-08-05T09:40:00.290" LastActivityDate="2015-08-05T09:40:00.290" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20381" PostTypeId="2" ParentId="20369" CreationDate="2015-08-05T14:15:18.983" Score="0" Body="&lt;p&gt;I was thinking along the same line as @Roland Heath but can't comment yet, here's my take:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;x/2&lt;/code&gt; can be computed by subtracting 1 from the &lt;strong&gt;exponent&lt;/strong&gt; (not the mantissa, subtracting 1 from the mantissa is subtracting &lt;code&gt;2^(value_of_exponent-length_of_mantissa)&lt;/code&gt; from the overall value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without restriction of the general case, let's assume &lt;code&gt;x &amp;lt; y&lt;/code&gt;. (If &lt;code&gt;x &amp;gt; y&lt;/code&gt;, relabel the variables. If &lt;code&gt;x = y&lt;/code&gt;, &lt;code&gt;(x+y) / 2&lt;/code&gt; is trivial.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Transform &lt;code&gt;(x+y) / 2&lt;/code&gt; into &lt;code&gt;x/2 + y/2&lt;/code&gt;, which can be performed by two integer subtractions (by one from the exponent) &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;However there's a lower bound on the exponent depending on your representation. If your exponent already is minimal before subtracting 1, this method will require special case handling. A minimal exponent on &lt;code&gt;x&lt;/code&gt; will make &lt;code&gt;x/2&lt;/code&gt; smaller than representable (assuming mantissa is represented with an implicit leading 1).&lt;/li&gt;&#xA;&lt;li&gt;Instead of subtracting 1 from the exponent of &lt;code&gt;x&lt;/code&gt;, shift &lt;code&gt;x&lt;/code&gt;'s mantissa to the right by one (and add the implicit leading 1, if any).&lt;/li&gt;&#xA;&lt;li&gt;Subtract 1 from the exponent of y, if it's not minimal. If it's minimal (y is bigger than x, because of the mantissa), shift the mantissa to the right by one (add implicit leading 1, if any).&lt;/li&gt;&#xA;&lt;li&gt;Shift the new mantissa of &lt;code&gt;x&lt;/code&gt; to the right according to the exponent of &lt;code&gt;y&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Perform integer addition on the mantissae, unless the mantissa of &lt;code&gt;x&lt;/code&gt; has been shifted out completely. If both exponents were minimal, the leading ones will overflow, which is ok, because that overflow is supposed to become an implicit leading one again.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;and one floating point addition.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can't think of any special case here; except for rounding, which also applies to shifting described above.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17167" LastActivityDate="2015-08-05T14:15:18.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20382" PostTypeId="1" AcceptedAnswerId="20384" CreationDate="2015-08-05T15:30:47.453" Score="6" ViewCount="2653" Body="&lt;p&gt;Given an $n_1 \times \cdots \times n_k \times g \times g$ tensor $A$ (i.e. a collection of $g \times g$ matrices) and an $n_1 \times \cdots \times n_k \times g$ tensor $b$ (i.e. a collection of vectors of length $g$) I want to use Numpy to compute the $n_1 \times \cdots \times n_k \times g$ tensor $c$ where each element is the matrix-vector product&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$c_{i_1,\ldots,i_k} = A_{i_1,\ldots,i_k}b_{i_1,\ldots,i_k}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a way to do this using &lt;code&gt;numpy.einsum&lt;/code&gt; but was curious if there was another way to do this or if there is a particular name for this operation? If there is an alternative &lt;em&gt;how does the performance of this alternative compare to using &lt;code&gt;einsum&lt;/code&gt;&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import dot, einsum, zeros_like&#xA;from numpy.linalg import norm&#xA;from numpy.random import randn&#xA;&#xA;n = 10&#xA;g = 4&#xA;&#xA;matrices = randn(n,n,g,g)&#xA;vectors = randn(n,n,g)&#xA;&#xA;# &quot;manual&quot; mat-vec multiplication (slow)&#xA;out1 = zeros_like(vectors)&#xA;for i in range(n):&#xA;    for j in range(n):&#xA;        out1[i,j] = dot(matrices[i,j], vectors[i,j])&#xA;&#xA;# using numpy.ensum (faster)&#xA;out2 = zeros_like(vectors)&#xA;out2 = einsum('...ij,...j-&amp;gt;...i', matrices, vectors)&#xA;&#xA;numpy.isclose(out1, out2).all()  # outputs `True`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6783" LastActivityDate="2015-08-05T18:35:22.467" Title="Alternatives to numpy.einsum" Tags="|linear-algebra|python|numpy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20384" PostTypeId="2" ParentId="20382" CreationDate="2015-08-05T18:35:22.467" Score="9" Body="&lt;p&gt;Pyhton 3.5 will introduce a new operator &lt;code&gt;@&lt;/code&gt;, which was proposed by NumPy devs to be the matrix multiplication operator. You may want to read &lt;a href=&quot;http://legacy.python.org/dev/peps/pep-0465/&quot;&gt;PEP 465&lt;/a&gt;, but the syntax that NumPy will adopt for this operator is &quot;stacked matrix multiplication,&quot; i.e. exactly what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be bits and pieces missing, but some work is already in, see &lt;a href=&quot;https://github.com/numpy/numpy/pull/5878/files&quot;&gt;the relevant PR&lt;/a&gt;, and I think that if you build the latest NumPy development version against a beta release of Python 3.5, you should be able to simply do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;out = matrices @ vectors&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or perhaps:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;out = matrices @ vectors[..., None]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to add a size-1 dimension to the end of your &lt;code&gt;vectors&lt;/code&gt; tensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With earlier versions of Python, if you build the latest development version of NumPy, you should be able to do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;out = np.linalg.matmul(matrices, vectors)  # or vectors[..., None]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These should be very fast, as they combine using a BLAS library to perform the matrix multiplications if available, with C iteration over all the stacked matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't want to get into non stable release territory (although NumPy 1.10 should be out the door in a few weeks), you have to choose:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;einsum&lt;/code&gt; has C iteration speed, but the actual matrix multiplication cannot match BLAS, even though it uses SIMD to be faster than a naive C implementation. Probably best for big &lt;code&gt;n&lt;/code&gt; and small &lt;code&gt;g&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;iteration plus &lt;code&gt;dot&lt;/code&gt; is slow iterating, but fast multiplying (if you have a good BLAS library), so best for small &lt;code&gt;n&lt;/code&gt; and large &lt;code&gt;g&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3399" LastActivityDate="2015-08-05T18:35:22.467" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20385" PostTypeId="2" ParentId="20324" CreationDate="2015-08-06T04:08:19.953" Score="4" Body="&lt;p&gt;When you call Lapack's zgtsv, it doesn't &lt;em&gt;just&lt;/em&gt; solve a tridiagonal system $Ax=b$. What it does first is perform an LU factorization (zgttrf) $A = LU$, where $L,U$ are lower- and upper-tridiagonal matrices, and only then proceeds to solve $LUx=b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you give it the lower, main, and upper diagonals of the matrix $A$, those diagonals are &lt;em&gt;overwritten&lt;/em&gt; by the routines that zgtsv invokes to contain, on output, the lower diagonal of $L$, main diagonal of $U$ and upper diagonal of $U$, respectively. This is a standard idea in linear algebra libraries like Blas and Lapack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This only needs to be done once per matrix: the LU factorization can be reused for future solves of $Ax=b$ (zgttrs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your code does the following thing wrong: it uses the same storage for the lower diagonal of $A$ and for its upper diagonal, meaning that Lapack will necessarily clobber its own output when writing the matrices $L$ and $U$ into the storage of $A$. The solution is to introduce another working vector&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;workBPM_super_diagonal[i] = BPM_sub_diagonal[i];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and pass that to zgtsv instead of passing &lt;code&gt;workBPM_sub_diagonal&lt;/code&gt; to it twice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the iteration $j=10$ when I did this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yFSjq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yFSjq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not too sure what results you are expecting there, but it doesn't blow up (for $N,M=300$). Since Crank-Nicolson wouldn't do a good job of damping oscillations due to discontinuous coefficients or delta-function initial functions, I'm not sure whether all of those oscillations are expected from the equation or if some of them are spurious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reuse the LU factorization between time steps, as your matrix does not change.&lt;/li&gt;&#xA;&lt;li&gt;Do not use raw pointers: it is not necessary given that there are standard C++ types such as &lt;code&gt;vector&lt;/code&gt; and &lt;code&gt;valarray&lt;/code&gt;. Learn about standard C++ features that C++ expects you to use.&lt;/li&gt;&#xA;&lt;li&gt;See &lt;em&gt;The Definitive C++ Book Guide and List&lt;/em&gt;: &lt;a href=&quot;https://stackoverflow.com/q/388242/491171&quot;&gt;https://stackoverflow.com/q/388242/491171&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;As a principle, avoid duplication: the r.h.s. of your equation is specified in &lt;em&gt;six&lt;/em&gt; different places. If there is an error, or if you want to solve an equation with a different r.h.s., that is five places too many that need to be correct at the same time.&lt;/li&gt;&#xA;&lt;li&gt;Decouple your code into modular functions that each solve a small, carefully-defined task. There is no reason why the definition of your equation has to be included right in the FD solver. Unnecessary coupling means that different parts of your program cannot be tested in isolation. C++ includes many tools to design modular testable programs.&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;code&gt;M_PI&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Consider asking on &lt;a href=&quot;https://codereview.stackexchange.com/&quot;&gt;https://codereview.stackexchange.com/&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="713" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-08-06T05:49:12.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20386" PostTypeId="1" CreationDate="2015-08-06T05:30:54.947" Score="8" ViewCount="794" Body="&lt;p&gt;The Sherman-Morrison formula &#xA;$$ (A+uv^T)^{-1} = A^{-1} - \frac{A^{-1}uv^TA^{-1}}{1+v^TA^{-1}u} $$&#xA;results in small errors in relation to the standard matrix inverse operation after each application, as shown &lt;a href=&quot;http://www3.nd.edu/~jeff/Teaching/CBE60478/ShermanMorrison.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. From what I understand, the Sherman-Morrison identity is exact, so the error must come from numerical problems. But where exactly? Also, are there any theoretical bounds for this error? Some sources argue that the error is related to the matrix condition number, but wouldn't that affect the standard inversion too?&lt;/p&gt;&#xA;" OwnerUserId="17171" LastEditorUserId="1804" LastEditDate="2016-05-27T16:32:57.540" LastActivityDate="2016-05-27T16:32:57.540" Title="What is the source of the error in the Sherman-Morrison formula application?" Tags="|linear-algebra|matrix|stability|inverse|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20387" PostTypeId="1" AcceptedAnswerId="26220" CreationDate="2015-08-06T09:24:44.497" Score="11" ViewCount="807" Body="&lt;p&gt;In Nocedal &amp;amp; Wright's book on Numerical Optimization, there is a statement in section 2.2 (page 27), &quot;Generally speaking, it is easier to preserve scale invariance for line search algorithms than for trust-region algorithms&quot;. In that same section, they talk about having new variables that are scaled versions of the original variables, which can help with both line search and trust region. Another approach is preconditioning. For trust region methods, preconditioning is equivalent to having elliptical trust regions and thus, provides scale invariance. However, a similar intuition is not clear for preconditioning for line search. In what ways is line search better suited for scale invariance? Are there some practical considerations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I have a question regarding preconditioning for trust region methods. For a highly ill-conditioned problem, will a good preconditioner reduce both the number of outer Newton iterations and inner CG iterations or only the latter? Since, the trust region is ellipsoidal in the original space, a good preconditioner should lead to an ellipsoid that will match the landscape better. I feel this might reduce the number of outer Newton iterations by forcing the algorithm to take better directions. Is this right?&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2015-08-29T19:55:17.273" LastActivityDate="2017-02-19T21:42:14.350" Title="scale invariance for line-search and trust region algorithms" Tags="|linear-algebra|optimization|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20388" PostTypeId="1" CreationDate="2015-08-06T13:03:55.223" Score="1" ViewCount="143" Body="&lt;p&gt;I need to know what library is the fastest one that can factorize a polynomial of large degree whose coefficients are big integers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried NTL library to factorize a monic polynomial but it is not efficient. &lt;/p&gt;&#xA;" OwnerUserId="14612" LastEditorUserId="20688" LastEditDate="2019-06-22T19:48:32.897" LastActivityDate="2019-06-22T19:48:32.897" Title="C++ Library: What is the Most Efficient Library that Factorizes a Polynomial?" Tags="|c++|libraries|polynomials|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20389" PostTypeId="1" CreationDate="2015-08-06T14:27:15.617" Score="5" ViewCount="206" Body="&lt;p&gt;I have a density function $f(x,y)$ on a rectangular domain $[0,1] \times [0,1]$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My domain is divided into cells of length $[0,h] \times [0,h]$, and I want to compute mass of each cell. I am going through each cell by using two for-loops, but then the computation becomes really slow even when $h$ is still pretty large, for example, $h = 1/400 = 0.0025$, as it amounts to  $400^2 = 160000$ cells to go through.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more efficient way of doing this other than using  two for-loops?&lt;/p&gt;&#xA;" OwnerUserId="11905" LastEditorUserId="11905" LastEditDate="2015-08-06T15:01:54.237" LastActivityDate="2015-10-02T03:56:13.490" Title="Fast computation on 2D rectangular cartesian grid using matlab" Tags="|matlab|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20390" PostTypeId="1" CreationDate="2015-08-06T14:29:06.663" Score="1" ViewCount="400" Body="&lt;p&gt;I  have implemented a basis transformation in C of the following form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kron[A,A]*B*Transpose[kron[A,A]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; are matrices and &lt;code&gt;kron&lt;/code&gt; is the Kronecker product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my naive code is not scalable, and I am looking for a library to achieve this. My starting point was to look in BLAS and LAPACK for Kronecker products and basis contractions (ie. &lt;code&gt;B*A*Transpose[B]&lt;/code&gt;) but from my reading of the documentation these are not in the standard build, is there an efficient way to achieve this using these packages (in particular the contraction)?&lt;/p&gt;&#xA;" OwnerUserId="17175" LastEditorUserId="9667" LastEditDate="2015-08-06T16:27:44.003" LastActivityDate="2015-08-07T11:35:13.270" Title="Kronecker products and basis contractions (ie. B.A.Transpose[B]) in C?" Tags="|linear-algebra|c|lapack|blas|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20391" PostTypeId="2" ParentId="20389" CreationDate="2015-08-06T14:34:54.983" Score="5" Body="&lt;p&gt;If by mass you mean $\int f(x,y)\; dxdy$ over each cell, then your best means of speeding this up is to parallelize it. Each cell is independent, so you should  break the outer loop into $p$ parts and do them all at the same time on $p$ processors. You can't do much better than that. What you want &lt;em&gt;requires&lt;/em&gt; $O(h^{-2})$ integrations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't need a uniform grid of cells where the mass is computed, you could look into adaptive mesh refinement. If there are regions of your $f$ where things are very nice, you might gain substantial speedup by not doing unnecessary work. You would be doing this at the cost of significant overhead in computing and using the result. $400\times400$ may be too small to see the benefit of an adaptive method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, what integration method are you using inside each cell? If that method is quadrature, and the quadrature has too many points, you might benefit from reducing its order inside the cell to something more appropriate. This only drops the constant in the complexity, but it could be an order of magnitude difference or more.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-08-06T14:34:54.983" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20392" PostTypeId="2" ParentId="19187" CreationDate="2015-08-07T04:09:28.080" Score="2" Body="&lt;p&gt;Molecular dynamics, in the computation of interactions inversely proportional to distance. Other N-body codes may share this property, but it's often an explicit goal to minimize or eliminate explicit division for exactly the reason you've given. &lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2015-08-07T04:09:28.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20393" PostTypeId="2" ParentId="1617" CreationDate="2015-08-07T04:20:18.307" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://www.dexy.it/docs/what-is-dexy.html&quot; rel=&quot;nofollow&quot;&gt;Dexy&lt;/a&gt; sounds like it is exactly what you area after. From the site:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dexy is a multi-purpose project automation tool with lots of features designed for working with documents. Dexy is written in Python and has a command-line interface. It's open source software with an MIT license.&lt;/p&gt;&#xA;  &#xA;  &lt;h3&gt;What does Dexy do?&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;Dexy makes it easier to create technical documents by doing the repetitive parts for you. Dexy provides a consistent interface to tools and scripts so you don't have to run them manually. Your project's dexy configuration keeps track of what to run, in which order, and with what parameters. This way, your whole process is captured so anyone can run it using one simple command and the results will be consistent.&lt;/p&gt;&#xA;  &#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;You want to write a blog post with examples showing how to use an API. Dexy will automatically:&lt;/p&gt;&#xA;    &#xA;    &lt;ul&gt;&#xA;    &lt;li&gt;run your example code, saving the results&lt;/li&gt;&#xA;    &lt;li&gt;apply syntax highlighting to your example code (using pygments)&lt;/li&gt;&#xA;    &lt;li&gt;insert the results of API calls and your prettified example code into your post (using jinja)&lt;/li&gt;&#xA;    &lt;li&gt;convert your markdown-formatted blog post to HTML (using python markdown or pandoc).&lt;/li&gt;&#xA;    &lt;li&gt;upload the HTML to the WordPress API in draft mode (using the WordPress API)&lt;/li&gt;&#xA;    &lt;li&gt;publish your blog post when you are finished tweaking it&lt;/li&gt;&#xA;    &lt;/ul&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1232" LastActivityDate="2015-08-07T04:20:18.307" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20395" PostTypeId="1" AcceptedAnswerId="20396" CreationDate="2015-08-07T09:33:37.017" Score="0" ViewCount="369" Body="&lt;p&gt;I would like to have a divergence-free formulation of the Navier-Stokes equations for creeping flow (a.k.a. Stokes equation, N.-S. without the inertial terms) in 3D, for purposes of tracking particle deformation in finite elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also like to be able to specify the usual kinds of boundary conditions (no-slip conditions, moving walls, slip conditions, pressure).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For calculating divergence-free flows, I have found a few options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One is $$\vec{u} = \operatorname{grad} \psi_1 \times \operatorname{grad} \psi_2$$&#xA;The problem with this is that $\psi_1$ and $\psi_2$ are not uniquely determined, and that imposing boundary conditions for non-zero velocities is not trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility I have found is to take the curl of a vector,&#xA;$$\vec{u} = \operatorname{curl} \left(&#xA;\begin{matrix}&#xA;\psi_1\\&#xA;\psi_2\\&#xA;\psi_3&#xA;\end{matrix}&#xA;\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where additional conditions (what kind?) have to be imposed to ensure uniqueness of the solution. Again, specifying boundary conditions is non-trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice or pointers to literature on how to proceed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A clarification: I want to solve the Stokes equation. For my application, it is important that the divergence of the flow field vanishes identically (subject to small errors), also inside an element. If you look at the results of the standard solutions that Comsol delivers with its CFD code, you will find quite large errors for $\operatorname{div} \vec{u}$ for flow regions close to a stagnation point.&lt;/p&gt;&#xA;" OwnerUserId="17183" LastEditorUserId="17183" LastEditDate="2015-08-07T16:25:17.717" LastActivityDate="2015-08-07T16:25:17.717" Title="3D Divergence-free Stokes equations" Tags="|finite-element|fluid-dynamics|comsol|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20396" PostTypeId="2" ParentId="20395" CreationDate="2015-08-07T10:40:02.573" Score="1" Body="&lt;p&gt;As far as I remember it is common to use splitting methods like Chorin's Method to ensure divergence free fields in fluid dynamics. Have a look into Demo 6 of &lt;a href=&quot;http://fenicsproject.org/documentation/dolfin/1.0.1/python/demo/pde/navier-stokes/python/documentation.html&quot; rel=&quot;nofollow&quot;&gt;Fenics on Incompressible Navier Stokes&lt;/a&gt;. Alternatively it is possible to implement certain Element Types like the H(curl)-Nédélec-Element that is often able to compute divergence-free fields. Although it is kind of tricky to know in which situations those elements don't work. &lt;/p&gt;&#xA;" OwnerUserId="16872" LastActivityDate="2015-08-07T10:40:02.573" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20397" PostTypeId="2" ParentId="20390" CreationDate="2015-08-07T11:35:13.270" Score="1" Body="&lt;p&gt;To the best of my knowledge, you are correct that there is no two-sided matrix multiplication routine $BAB^T$ in BLAS/LAPACK (but I could easily have missed it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the symmetric case, I have found code for this operation in &lt;a href=&quot;http://slicot.org/objects/software/shared/doc/MB01RD.html&quot; rel=&quot;nofollow&quot;&gt;Slicot&lt;/a&gt; and &lt;a href=&quot;http://libelemental.org/documentation/dev/blas_like/level3/TwoSidedTrmm.html&quot; rel=&quot;nofollow&quot;&gt;Elemental&lt;/a&gt;. The latter is a library for parallel linear algebra, so it is a different setting. If you can get your hands on a copy of MB01RD from Slicot, it should be best suited to your needs. The library is available free of charge for academic users.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the matrix $C$ in $BCB^T$ is symmetric, then another possibility is changing your algorithm to work with symmetric factors of $C$, for instance $C=LDL^T$ with $D$ diagonal. Now to perform your two-sided conjugation you only have to replace $L$ with $BL$.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2015-08-07T11:35:13.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20400" PostTypeId="2" ParentId="20395" CreationDate="2015-08-07T15:04:11.670" Score="0" Body="&lt;p&gt;You pretty much have to be able to impose a velocity boundary condition different from zero in order to impose a flow that's also divergence-free. If you can't do that in COMSOL, you maybe be out of luck. I would be surprised to learn that you can't specify such a flow. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-08-07T15:04:11.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20401" PostTypeId="2" ParentId="10375" CreationDate="2015-08-07T15:15:26.817" Score="6" Body="&lt;p&gt;In my experience, the polar-Newton method of Higham works much faster (see Chapter 6 of &lt;em&gt;Functions of Matrices&lt;/em&gt; by N. Higham). In &lt;a href=&quot;http://arxiv.org/abs/1507.08366&quot; rel=&quot;noreferrer&quot;&gt;this short note of mine&lt;/a&gt; there are plots that compare this method to first-order methods. Also, citations to several other matrix-square-root approaches are presented, though mostly the polar Newton iteration seems to work the best (and avoids doing eigenvector computations).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% compute the matrix square root; modify to compute inverse root.&#xA;function X = PolarIter(M,maxit,scal)&#xA;  fprintf('Running Polar Newton Iteration\n');&#xA;  skip = floor(maxit/10);&#xA;  I = eye(size(M));&#xA;  n=size(M,1);&#xA;  if scal&#xA;    tm = trace(M);&#xA;    M  = M / tm;&#xA;  else&#xA;    tm = 1;&#xA;  end&#xA;  nm = norm(M,'fro');&#xA;&#xA;  % to compute inv(sqrt(M)) make change here&#xA;  R=chol(M+5*eps*I);&#xA;&#xA;  % computes the polar decomposition of R&#xA;  U=R; k=0;&#xA;  while (k &amp;lt; maxit)&#xA;    k=k+1;&#xA;    % err(k) = norm((R'*U)^2-M,'fro')/nm;&#xA;    %if (mod(k,skip)==0)&#xA;    %  fprintf('%d: %E\n', k, out.err(k));&#xA;    %end&#xA;&#xA;    iU=U\I;&#xA;    mu=sqrt(sqrt(norm(iU,1)/norm(U,1)*norm(iU,inf)/norm(U,inf)));&#xA;    U=0.5*(mu*U+iU'/mu);&#xA;&#xA;   if (err(k) &amp;lt; 1e-12), break; end&#xA;  end&#xA;  X=sqrt(tm)*R'*U;&#xA;  X = 0.5*(X+X');&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="309" LastEditorUserId="309" LastEditDate="2015-08-07T17:36:23.007" LastActivityDate="2015-08-07T17:36:23.007" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20402" PostTypeId="1" CreationDate="2015-08-07T16:48:08.970" Score="4" ViewCount="320" Body="&lt;p&gt;I'm working with pressure-velocity coupled systems. It means that instead of solving 4 different linear systems in segregated approach (1 for pressure and 3 for Ux, Uy, Uz), we can solve only one coupled system. Assuming simple 3D rectangular mesh resulting matrix appears to be 7-diagonal with each element is a block 4x4 of arrow form&#xA;\begin{pmatrix} a_{11} &amp;amp; 0 &amp;amp; 0 &amp;amp; a_{14} \\ 0 &amp;amp; a_{22} &amp;amp; 0 &amp;amp; a_{24} \\ 0 &amp;amp; 0 &amp;amp; a_{33} &amp;amp; a_{34} \\ a_{41} &amp;amp; a_{42} &amp;amp; a_{43} &amp;amp; a_{44} \end{pmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to make efficient GPU-based BiCGStab solver for it, but it seems, that I have problem with preconditioning. On CPU BiCGStab + ILU or AMG works just fine. But on GPU, I can't use ILU family, because they are not suitable. I've tried AINV preconditioner, but it's either too slow and big or not robust enough. Diagonal preconditioner isn't effective at all. Now I'm thinking of polynomial, though I expect it to have similar problems as AINV. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other options that could possibly work with my approach, or the whole idea of GPU BiCGStab is a dead end in the first place? &lt;/p&gt;&#xA;" OwnerUserId="17188" LastActivityDate="2015-08-11T18:15:32.433" Title="Choosing preconditioner for unsymmetric pressure-velocity coupled system" Tags="|linear-solver|sparse-matrix|preconditioning|krylov-method|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20403" PostTypeId="1" CreationDate="2015-08-07T23:26:28.053" Score="0" ViewCount="1523" Body="&lt;p&gt;What is the the simplest way to get started using GPU computing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My interests are primarily in neural networks and I would love to start using GPUs, but my time for learning GPU computing is very limited.  Are there simple libraries for R or something that make it easy to multiply matrices on GPUs?&lt;/p&gt;&#xA;" OwnerUserId="17192" LastEditorUserId="9667" LastEditDate="2015-08-08T18:29:05.990" LastActivityDate="2016-10-12T23:12:59.893" Title="Easiest Way to Begin GPU Computing" Tags="|gpu|r|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20404" PostTypeId="2" ParentId="17577" CreationDate="2015-08-08T01:21:48.777" Score="1" Body="&lt;p&gt;I'm contributor to a machine-learning library called &lt;a href=&quot;http://deeplearning4j.org&quot; rel=&quot;nofollow&quot;&gt;Deeplearning4j&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We see a fair amount of demand for JVM-based tools from sectors like finance, telecoms, government agencies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We developed &lt;a href=&quot;http://nd4j.org&quot; rel=&quot;nofollow&quot;&gt;ND4J&lt;/a&gt;, which supports n-dimensional arrays for Java and includes a Scala wrapper ND4S. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the optimizations are in C/C++, but the framework integrates with Hadoop and Spark, and enables them to run on multiple GPUs. &lt;/p&gt;&#xA;" OwnerUserId="11989" LastActivityDate="2015-08-08T01:21:48.777" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20405" PostTypeId="2" ParentId="20403" CreationDate="2015-08-08T17:06:53.140" Score="2" Body="&lt;p&gt;There is a tutorial on GPU computing in R at &lt;a href=&quot;http://www.r-tutor.com/gpu-computing&quot; rel=&quot;nofollow&quot;&gt;r-tutor.com&lt;/a&gt;. It has various examples you can look at and primarily uses the RPUD package which is open source and also makes use of the non-free RPUDPLUS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally &lt;a href=&quot;http://qiangkou.me/2013/10/06/r-and-gpu/&quot; rel=&quot;nofollow&quot;&gt;this website&lt;/a&gt; has a discussion of a few different packages that aid in GPU computing in R. The packages mentioned are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;gputools&lt;/li&gt;&#xA;&lt;li&gt;HiPLARM&lt;/li&gt;&#xA;&lt;li&gt;rpud&lt;/li&gt;&#xA;&lt;li&gt;magma&lt;/li&gt;&#xA;&lt;li&gt;gcbd&lt;/li&gt;&#xA;&lt;li&gt;OpenCL&lt;/li&gt;&#xA;&lt;li&gt;WideLM&lt;/li&gt;&#xA;&lt;li&gt;cudaBayesreg&lt;/li&gt;&#xA;&lt;li&gt;permGPU&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;but only the first three are discussed in any detail.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-08-08T17:06:53.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20407" PostTypeId="1" CreationDate="2015-08-08T19:49:13.633" Score="1" ViewCount="106" Body="&lt;p&gt;Let   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T=1, K=100, S_0=100, \sigma=0.05, r=0.15. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define $\nu:=\frac{2r}{\sigma^2}-1$&#xA;and $$H(y,z)=\frac{z e^{\pi^2 /4y}}{\pi \sqrt{\pi y}}\int_0^{\infty} e^{-z \cosh(u) -u^2/(4y)} \sinh(u) \sin(\frac{\pi u}{2y})du$$&#xA;then next define &#xA;$$ f(y,z)=\left( \frac{z}{S_0}\right)^{\nu/2}\frac{1}{4KT}\exp\left(-\frac{2(S0+z)}{KT\sigma^2}-\frac{\nu^2\sigma^2y}{8} \right) \times H(\frac{\sigma^2y}{8},\frac{4\sqrt{S_0 z}}{KT\sigma^2}) $$&#xA;and $$ g(y,z)=z\frac{e^{-ry}-e^{-rT}}{rT} $$&#xA;I would like to compute the following double integral &#xA;$$ I=\int_0^{\infty}\left[\int_0^T g(y,z)f(y,z)dy\right]dz. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I tried this with Maple, please see the picture below,&#xA;&lt;a href=&quot;https://i.stack.imgur.com/j4lZH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j4lZH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;it ran out of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to ask whether there is a way to compute the above integral? What should I do to compute the integral?&lt;/p&gt;&#xA;" OwnerUserId="17200" LastEditorUserId="713" LastEditDate="2015-08-08T23:20:10.693" LastActivityDate="2015-08-08T23:20:10.693" Title="How to compute this double integral?" Tags="|linear-algebra|finite-element|numerical-analysis|discretization|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20408" PostTypeId="1" CreationDate="2015-08-08T21:54:30.413" Score="1" ViewCount="220" Body="&lt;p&gt;I tried to solve Hamiltonian system ($Q$ is a vector of all generalized coordinates, $P$ - of generalized momentum)&#xA;$$&#xA;\frac{\mathrm{d} Q}{\mathrm{d} t}=\frac{\partial H}{\partial P} \\&#xA;\frac{\mathrm{d} P}{\mathrm{d} t}=-\frac{\partial H}{\partial Q}&#xA;$$&#xA;where&#xA;$$&#xA;H=\frac{1}{2} \sum_{i=1}^{n}\overrightarrow{p}_{i}^{2} + \frac{k}{2} \sum_{i=2}^{n}(\left \| \overrightarrow{q}_{i}-\overrightarrow{q}_{i-1} \right \|-rest)^2 + \frac{\kappa}{2} \sum_{i=2}^{n-1}\arccos^2\frac{(\overrightarrow{q}_{i}-\overrightarrow{q}_{i-1})\cdot (\overrightarrow{q}_{i+1}-\overrightarrow{q}_{i})}{\left \| \overrightarrow{q}_{i}-\overrightarrow{q}_{i-1} \right \|\left \| \overrightarrow{q}_{i+1}-\overrightarrow{q}_{i} \right \|}&#xA;$$&#xA;with &quot;SymplecticPartitionedRungeKutta&quot; method numerically using Mathematica (description could be found here &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0377042701004927&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0377042701004927&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;\[CurlyKappa] = 20;&#xA;k = 20;&#xA;rest = Sqrt[5];&#xA;n = 3;&#xA;dim = 3;&#xA;Q = Table[Table[Subscript[q, j][i][t], {j, 1, dim}], {i, 1, n}];&#xA;P = Table[Table[Subscript[p, j][i][t], {j, 1, dim}], {i, 1, n}];&#xA;icsQ = {{-1, -1, 0}, {0, 1, 0}, {1, -1, 0}};&#xA;icsP = Table[Table[0, {j, 1, dim}], {i, 1, n}];&#xA;&#xA;H = 1/2 (Sum[P[[i]].P[[i]], {i, 1, n}] + &#xA;     Sum[(Norm[Q[[i]] - Q[[i - 1]]] - rest)^2, {i, 2, n}]*k + &#xA;     Sum[(VectorAngle[Q[[i]] - Q[[i - 1]], &#xA;         Q[[i + 1]] - Q[[i]]])^2, {i, 2, n - 1}]*\[CurlyKappa]);&#xA;Q' = Flatten[D[#, t] &amp;amp; /@ Q];&#xA;P' = Flatten[D[#, t] &amp;amp; /@ P];&#xA;&#xA;Conjugate'[y_[x_][t_]] := 1;&#xA;Abs'[x_] := Sign[x];&#xA;&#xA;var = Flatten[Q~Join~P];&#xA;ics = Flatten[icsQ~Join~icsP];&#xA;eqn = Table[Q'[[i]] == D[H, var[[n*dim + i]]], {i, 1, n*dim}]~Join~&#xA;   Table[P'[[i]] == -D[H, var[[i]]], {i, 1, n*dim}]~Join~&#xA;   Table[(var[[i]] /. t -&amp;gt; 0) == ics[[i]], {i, Length[var]}];&#xA;&#xA;method = {&quot;SymplecticPartitionedRungeKutta&quot;, &#xA;   &quot;PositionVariables&quot; -&amp;gt; var[[1 ;; dim*n]]};&#xA;sol = NDSolve[eqn, var, {t, 0, 10}, Method -&amp;gt; method, &#xA;  WorkingPrecision -&amp;gt; 10, MaxStepSize -&amp;gt; 0.001, &#xA;  MaxSteps -&amp;gt; \[Infinity]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, error pops out:     &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;NDSolve::sprksep: &quot;The Hamiltonian of the differential system in the&#xA;  method does not appear to be in separable form. Try using the method&#xA;  ImplicitRungeKutta with coefficients&#xA;  ImplicitRungeKuttaGaussCoefficients.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But it's easy to see that the Hamiltonian of the system definately has separate form H(p,q)=T(p)+V(q).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. In order to prevent questions. You could notice that I defined two &quot;strange&quot; functions:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Conjugate'[y_[x_][t_]] := 1;&#xA; Abs'[x_] := Sign[x];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Idea: Mathematica doesn't take the derivative from non-analytical functions. And when you take derivative in form &lt;code&gt;D[H,q[t]]&lt;/code&gt;, terms of type &lt;code&gt;D[Conjugate[q[t]],q[t]]&lt;/code&gt; appear. If I don't fix it, I will get an error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;NDSolve::ndnum: Encountered non-numerical value for a derivative at t == 0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17201" LastEditorUserId="17201" LastEditDate="2015-08-10T12:22:38.740" LastActivityDate="2015-08-10T12:22:38.740" Title="Symplectic Partitioned Runge Kutta method in Mathematica" Tags="|runge-kutta|mathematica|" AnswerCount="0" CommentCount="3" ClosedDate="2015-08-11T01:49:36.070" ContentLicense="CC BY-SA 3.0" />
  <row Id="20410" PostTypeId="1" CreationDate="2015-08-09T01:46:49.023" Score="3" ViewCount="104" Body="&lt;p&gt;What's the best method to maximize the value of slow multivariate function? Here is what I know about the function:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Number of parameters is ~ 10.&lt;/li&gt;&#xA;&lt;li&gt;It takes considerable amount of time to compute the value of the function for single variable set. Let's say 30 minutes.&lt;/li&gt;&#xA;&lt;li&gt;Function seems to be concave against every individual parameter.&lt;/li&gt;&#xA;&lt;li&gt;I can run computation in parallel.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All my code is in Java and I'm interested in coding crude version of the optimization algorithm myself rather than using some industry solver.&lt;/p&gt;&#xA;" OwnerUserId="17203" LastEditorUserId="17203" LastEditDate="2015-08-09T18:23:25.207" LastActivityDate="2015-08-09T18:23:25.207" Title="Maximizing slow multi-parameter function" Tags="|optimization|" AnswerCount="0" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="20411" PostTypeId="1" AcceptedAnswerId="20429" CreationDate="2015-08-09T23:31:09.863" Score="0" ViewCount="87" Body="&lt;p&gt;I need to find an equation for the upper bound of $\max \mathbf{w}^T\mathbf{x}_i, \; i=1, \dots N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{w}$ and $\mathbf{x}_i$ are two vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to find a function $f$ which holds the following inequality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \mathbf{w}^T\mathbf{x}_i \leq \mathbf{w}^T \mathbf{z}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{z} = f(\mathbf{x}_i),\; i=1, \dots, N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\mathbf{x}_1 = \begin{pmatrix}x_{11}\\x_{12}\\x_{13}\end{pmatrix}, \; \dots, \mathbf{x}_N = \begin{pmatrix}x_{N1}\\x_{N2}\\x_{N3}\end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{z} = \begin{pmatrix}f(x_{11}, \dots, x_{N1})\\f(x_{12}, \dots, x_{N2})\\f(x_{13}, \dots, x_{N3})\end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example f can be a $\max$ or $\min$ function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the values of $\mathbf{x}_i, \;, i=1, \dots, N$ are known. But $\mathbf{w}$ is unknown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to have $f$ as a function only on $\mathbf{x}_i$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&#xA;$\mathbf{x}_1 = \begin{pmatrix}-10\\1\\3\end{pmatrix}, \; \mathbf{x}_2 = \begin{pmatrix}5\\-3\\-5\end{pmatrix} \implies \mathbf{z} = \max\mathbf{x}_i = \begin{pmatrix}5\\1\\3\end{pmatrix}$&lt;/p&gt;&#xA;" OwnerUserId="11954" LastEditorUserId="11954" LastEditDate="2015-08-10T15:28:04.737" LastActivityDate="2015-08-11T11:53:41.093" Title="what is the upper bound of $\max \mathbf{w}^T\mathbf{x}_i$" Tags="|optimization|numerical-analysis|constrained-optimization|nonlinear-programming|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="20412" PostTypeId="1" AcceptedAnswerId="20413" CreationDate="2015-08-10T03:23:54.903" Score="2" ViewCount="397" Body="&lt;p&gt;I am currently trying to use iterative least squares to solve a system,&#xA;$y = Hx + v$&#xA;where $y$ is a vector of observations, $H$ is the design matrix, and $v$ is the observation error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding, iterative least squares tries to minimize a cost $J$ where $J = \min (Hx-y)^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My pseudocode for this is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while ||x^k-x^{k-1}||&#xA;   H = h(x^{k-1})&#xA;   r = f(x^{k-1}) - y&#xA;   dx = pinv(H)*r&#xA;   x^{k} = x^{k-1} + dx&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $H$ is the Jacobian and the output of $f(x)$ is the expected observations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can $r$ be called the cost of the system because it is the residual/difference between $f(x)$ at estimate $x^{k-1}$ and $y$? Should it be squared in an implementation of iterative least squares?&lt;/p&gt;&#xA;" OwnerUserId="17213" LastActivityDate="2015-08-10T12:26:47.403" Title="Minimizing Cost Functions using Iterative Least Squares" Tags="|iterative-method|least-squares|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20413" PostTypeId="2" ParentId="20412" CreationDate="2015-08-10T12:26:47.403" Score="1" Body="&lt;p&gt;The linear least squares problem $\min \|Hx-y\|^2$ is equivalent to solving&#xA;$H^THx = H^Ty$, known as the normal equation. $H^T$ denotes the transpose, or adjoint matrix. &#xA;There are many ways to solve this, depending on your problem, but the &lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_gradient_method#Conjugate_gradient_on_the_normal_equations&quot; rel=&quot;nofollow&quot;&gt;conjugate gradient method&lt;/a&gt; is a good general method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the pseudo-inverse (pinv) function you're using will calculate $(H^TH)^{-1}H^T$, so &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;x=pinv(H)*y&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This means your iterative method would have converged in the first iteration.&#xA;If performance is not an issue and $H$ is reasonably small, that is the easiest way to solve your problem, but for larger $H$ iterative methods will often be significantly faster. This is particularly the case if $H$ is a sparse matrix, or the matrix-vector product $Hr$ can be calculated via a fast method such as the Fourier transform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As hardmath mentioned, $r$ is the residual vector and $J=\|r\|^2$.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2015-08-10T12:26:47.403" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20415" PostTypeId="1" AcceptedAnswerId="20416" CreationDate="2015-08-10T18:26:04.340" Score="3" ViewCount="215" Body="&lt;p&gt;If one chooses to discetize the advection-diffusion (AD) equation using the standard Galerkin finite element method, stability issues may arise in cases of high Peclet number (i.e., high advection to diffusion ratio).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than using methods like SUPG, can one achieve stable AD solutions (with Galerkin FEM) through the use of multi-grid preconditioning? I have seen and heard talks about how multi-grid solvers can help achieve smoothing and stabilization for a wide range of problems, and was wondering if such a method could be applied to a general AD equation. For instance, I am using PETSc for linear algebra and can use any of its multi-grid preconditioning capabilities.&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2015-08-10T21:30:00.793" Title="stabilizing advection-diffusion with multi-grid?" Tags="|stability|multigrid|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20416" PostTypeId="2" ParentId="20415" CreationDate="2015-08-10T19:42:30.140" Score="5" Body="&lt;p&gt;There are two different flavors of smoothing and stability. The spurious oscillations in convection-diffusion problems are not an artifact of the linear solver but an inevitable artifact of the discretization method. Any linear solver for non-symmetric matrices you use will give you the same wiggly answer, or it's wrong. Multi-grid is one such linear solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When discussing multi-grid, we also talk about smoothing, but that's smoothing between its internal steps. Each step of multi-grid moves the residual from one grid to another and then uses a few iterations of a &quot;smoother&quot;, often Gauss-Seidel, to smooth out the high-frequency components of the error in the solution on that grid. As you go from fine to coarse, you remove lower and lower frequencies of the error. On the coarsest grid, some other solver is often used to deal with the lowest frequency components on this last mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two concepts are separate. If you want to get rid of the wiggles, you either need to use a stabilized method like SUPG, or you need more mesh in the layer that is causing the wiggles. Changing solvers or preconditioners won't help.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-08-10T21:30:00.793" LastActivityDate="2015-08-10T21:30:00.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20418" PostTypeId="1" CreationDate="2015-08-10T21:28:59.817" Score="1" ViewCount="155" Body="&lt;p&gt;I was wondering if someone has implemented the &quot;Tunneling algorithm&quot; for the global minimization of a single variable function in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am hoping to implement it on $[B, A - \lambda I]$. Where $A$ is and $N\times N$ matrix, B is an $N\times M$ matrix, $I$ is the identity and $\lambda$ is the unknown variable in the complex domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is related to &quot;the distance to uncontrollability of a dynamic system&quot;.&lt;/p&gt;&#xA;" OwnerUserId="17220" LastEditorUserId="9667" LastEditDate="2015-08-10T23:44:12.767" LastActivityDate="2015-08-10T23:44:12.767" Title="&quot;Tunneling&quot; optimization algorithm in Matlab" Tags="|optimization|matlab|algorithms|nonlinear-programming|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20419" PostTypeId="2" ParentId="20403" CreationDate="2015-08-10T22:50:19.133" Score="3" Body="&lt;p&gt;If time is of grave concern, I would highly suggest looking at Intels Xeon phi coprocessor. Not only are they nearly or as fast, only require openmp to code for, but Intels customer service on the Intel developer forums is fantastic. I don't know if you can use R, but standards languages such as c, c++ , and fortran can be used. You could also use Intels mkl library directly on the coprocessor for matrix multiplication&lt;/p&gt;&#xA;" OwnerUserId="16889" LastActivityDate="2015-08-10T22:50:19.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20421" PostTypeId="2" ParentId="20288" CreationDate="2015-08-10T23:29:44.260" Score="0" Body="&lt;p&gt;If $\beta_2$ is piece-wise constant the equation is solvable by Fesnel integrals so that you can develop approximte solution based on the boundary conditions.&#xA;You have to introduce a co-moving frame and solve from there.&lt;/p&gt;&#xA;" OwnerUserId="10345" LastActivityDate="2015-08-10T23:29:44.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20422" PostTypeId="2" ParentId="20403" CreationDate="2015-08-11T01:17:44.517" Score="3" Body="&lt;p&gt;I know time may be limited for you, but if you have ~1-3 weeks to really learn CUDA, I highly recommend &lt;em&gt;CUDA by Example: An Introduction to General-Purpose GPU Programming&lt;/em&gt; (&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0131387685&quot; rel=&quot;nofollow&quot;&gt;Amazon.com&lt;/a&gt;). It does a fantastic job of explaining the general concepts of GPU programming, and does a great job of getting one up to speed with NVIDIA's GPU language, CUDA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CUDA is arguably one of the most widely-used GPU languages—certainly within the high-performance computing community (e.g. &lt;em&gt;NASA&lt;/em&gt;). It gives great control over lots of details, but for many tasks, it also requires low-level knowledge of the graphics card. It's not something you'll learn in a day, probably not even in a week, but it can be worth it. &lt;/p&gt;&#xA;" OwnerUserId="9451" LastActivityDate="2015-08-11T01:17:44.517" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20423" PostTypeId="2" ParentId="14818" CreationDate="2015-08-11T02:57:44.820" Score="2" Body="&lt;p&gt;There is also a package for R called rgp.  Visit this link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cran.r-project.org/web/packages/rgp/index.html&quot; rel=&quot;nofollow&quot;&gt;https://cran.r-project.org/web/packages/rgp/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not used rgp as I have only begun to use R seriously but it seemed like a good lead.  I have another one for you that looks really promising but I have a mac and cannot use it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dev.heuristiclab.com/wiki/AdditionalMaterial/ECML-PKDD&quot; rel=&quot;nofollow&quot;&gt;http://dev.heuristiclab.com/wiki/AdditionalMaterial/ECML-PKDD&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17223" LastEditorUserId="-1" LastEditDate="2015-08-18T13:34:29.523" LastActivityDate="2015-08-18T13:34:29.523" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20424" PostTypeId="1" CreationDate="2015-08-11T03:28:07.643" Score="4" ViewCount="143" Body="&lt;p&gt;Suppose I have two sparse matrices, $A$ and $B$, of size $N \times N$. They each have the same sparcity pattern (&quot;footprint&quot;). They each also have values which in theory should be identical, but aren't due to machine precision error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple example of this in MATLAB syntax:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;% Dimensions of square, sparse matrices&#xA;N = 10;&#xA;A = sparse(N,N);&#xA;B = sparse(N,N);&#xA;&#xA;% Add a dozen nonzero entries at random locations&#xA;numEntries = 12;&#xA;rows = randi( N, numEntries,1 );&#xA;cols = randi( N, numEntries,1 );&#xA;&#xA;% Populate the entries, one at a time&#xA;for entry = 1:numEntries&#xA;    i = rows(entry);&#xA;    j = cols(entry);&#xA;&#xA;    % Generate random numbers&#xA;    x = rand(1,1);&#xA;    y = rand(1,1);&#xA;&#xA;    % Populate matrices&#xA;    % Note: (x^2-y^2) / (x-y) == (x+y)*(x-y)/(x-y) == x+y&#xA;    A(i,j) = (x^2-y^2) / (x-y);&#xA;    B(i,j) = x + y;&#xA;&#xA;end&#xA;&#xA;diffMat = A - B;&#xA;disp(diffMat);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obviously the values of $A$ and $B$ &lt;em&gt;should&lt;/em&gt; be identical, but they aren't. The output of this script is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   1.0e-15 *&#xA;&#xA;   (1,2)     -0.222044604925031&#xA;   (2,2)      0.222044604925031&#xA;  (10,3)      0.111022302462516&#xA;   (8,4)     -0.222044604925031&#xA;   (3,5)      0.222044604925031&#xA;  (10,5)      0.555111512312578&#xA;   (3,6)      0.222044604925031&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Suppose a pair of matrices like this that give &lt;em&gt;wildly different&lt;/em&gt; results $x_A$ and $x_B$ for the matrix equations $A x_A = b$ and $B x_B = b$ (same $b$ in each case). Suppose I have confirmed that the values of $A$ and $B$ are identical to less than one part in 1e15 (that is, &lt;code&gt;abs( (A(A~=0)-B(B~=0)) ./ A(A~=0) )  &amp;lt;  1e-15&lt;/code&gt;). Further suppose that the estimated condition number (&lt;code&gt;condest()&lt;/code&gt;) of each matrix is identical. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;My question:  Under what circumstances could this occur?&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;Disclaimer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I originally asked this question because I had this problem in my code. It turned out that my $b$ vectors were not identical due to a small typo in the code that generated them (thanks @JesseChan!). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am still interested if and how this can occur. My guess is that very high condition numbers can produce these kinds of results, but I'm curious if anyone can produce and explain a minimum working example of this behavior. &lt;/p&gt;&#xA;" OwnerUserId="9451" LastEditorUserId="9451" LastEditDate="2015-08-11T19:09:25.320" LastActivityDate="2015-08-11T22:43:06.857" Title="Under what circumstances can two (nearly) identical sparse matrices give different solutions to Mx = b?" Tags="|linear-algebra|matlab|finite-difference|linear-solver|precision|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20425" PostTypeId="1" AcceptedAnswerId="20430" CreationDate="2015-08-11T03:50:54.670" Score="4" ViewCount="1981" Body="&lt;p&gt;I am trying to solve the very simple one dimensional burgers equation which is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\delta U}{\delta t} + \frac{\delta F}{\delta x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the flux F of some variable U is defined as$$ F= \frac{U^2}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I consider a small control volume around the node &lt;strong&gt;&lt;em&gt;i&lt;/em&gt;&lt;/strong&gt;, then I should write the original equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{U_i^{n+1} - U_i^{n}}{\Delta t} + \frac{ F_{i+0.5}^{n} - F_{i-0.5}^{n}}{\Delta x} = 0 $$&#xA;I am using linear interpolation to calculate the &lt;strong&gt;F&lt;/strong&gt; values at the edges of control volume.&#xA;I am defining &lt;strong&gt;&lt;em&gt;F&lt;/em&gt;&lt;/strong&gt; at the node &lt;em&gt;i&lt;/em&gt; as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F[i] = 0.5*0.5*(u[i] + u[i+1])*0.5*(u[i] + u[i+1]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then modifying the function's value as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;unew[i] = u[i] - (dt/dx)*(F[i] - F[i-1]);       //Final Equation//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;However this approach is leading to divergent results which I don't really understand why.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I want to have the discretized equation in the form of &lt;em&gt;Final Equation&lt;/em&gt; because I further want to use adaptive mesh refinement and if the discretized equation is in this form, it is easy to apply.&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2015-08-12T06:00:24.797" Title="How to discretize Burger's equation?" Tags="|pde|finite-difference|fluid-dynamics|mesh|adaptive-mesh-refinement|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20427" PostTypeId="1" CreationDate="2015-08-11T10:25:47.323" Score="1" ViewCount="1350" Body="&lt;p&gt;Suppose I have this matrix in &lt;em&gt;Matlab R2013a&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M = kron(A,B);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; are $N \times N$ matrices, with $N$ a &quot;big&quot; number. Now, I just want to extract a submatrix of &lt;code&gt;M&lt;/code&gt;, composed by taking just few rows from it&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Msub = M(rows,:);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;rows&lt;/code&gt; is a vector of length $m \ll N$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Q:&lt;/em&gt; Is there an efficient way, in &lt;em&gt;Matlab R2013a&lt;/em&gt;, to build &lt;code&gt;Msub&lt;/code&gt; without building &lt;code&gt;M&lt;/code&gt; before? &lt;/p&gt;&#xA;" OwnerUserId="11532" LastEditorUserId="11532" LastEditDate="2015-08-11T10:37:12.153" LastActivityDate="2015-08-15T04:30:17.860" Title="Efficiently extracting a submatrix in Matlab" Tags="|matlab|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20428" PostTypeId="1" CreationDate="2015-08-11T11:38:36.670" Score="1" ViewCount="875" Body="&lt;p&gt;I'm new to factor analysis and I need to compute factor scores using &lt;code&gt;Python&lt;/code&gt;. I have &lt;code&gt;R&lt;/code&gt; code to compute scores and I want to adjust my Python code (&lt;code&gt;decomposition.FactorAnalysis()&lt;/code&gt; settings) to get similar output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;R&lt;/code&gt; code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;-read.csv(&quot;test.csv&quot;, header=F)&#xA;col_names &amp;lt;- names(data)&#xA;drops &amp;lt;- c()&#xA;&#xA;for (name in col_names){&#xA;  st_dev &amp;lt;- sd(data[,name], na.rm = T)&#xA;  if (st_dev == 0){&#xA;    drops &amp;lt;- c(drops, name)&#xA;  }&#xA;}&#xA;&#xA;da_nal &amp;lt;- data[,!(names(data) %in% drops)]&#xA;factanal(na.omit(da_nal), factors = 1, scores = 'regression')$scores&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Its output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Factor1&#xA;1   4.89102190&#xA;2   3.65004187&#xA;3   0.14628700&#xA;4  -0.20255897&#xA;5  -0.01565570&#xA;6  -0.16438863&#xA;7   0.40835986&#xA;8  -0.25823984&#xA;9  -0.20813064&#xA;10  0.09390067&#xA;11 -0.28891296&#xA;12 -0.28882753&#xA;13 -0.26624358&#xA;14 -0.25202275&#xA;15 -0.25181326&#xA;16 -0.15653679&#xA;17 -0.28702281&#xA;18 -0.28865654&#xA;19 -0.23251509&#xA;20 -0.28066125&#xA;21 -0.18714387&#xA;22 -0.24969113&#xA;23 -0.28302552&#xA;24 -0.28712610&#xA;25 -0.29196529&#xA;26 -0.28659988&#xA;27 -0.29502523&#xA;28 -0.15802910&#xA;29 -0.27440118&#xA;30 -0.29083667&#xA;31 -0.29548220&#xA;32 -0.29461059&#xA;33 -0.23594859&#xA;34 -0.29654336&#xA;35 -0.29759659&#xA;36 -0.29085001&#xA;37 -0.29539071&#xA;38 -0.29234303&#xA;39 -0.29702103&#xA;40 -0.27595130&#xA;41 -0.27184361&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Python&lt;/code&gt; code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn import decomposition, preprocessing&#xA;import numpy as np&#xA;&#xA;data = np.genfromtxt('rangir_test.csv', delimiter=',')&#xA;data = data[~np.isnan(data).any(axis=1)]&#xA;data_normal = preprocessing.scale(data)&#xA;fa = decomposition.FactorAnalysis(n_components = 1)&#xA;fa.fit(data_normal)&#xA;for score in fa.score_samples(data_normal):&#xA;    print score &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Its output is quite different from &lt;code&gt;R&lt;/code&gt;'s, but want I them to be similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Python&lt;/code&gt; output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-69.8587183816&#xA;-116.353511148&#xA;-24.1529840248&#xA;-36.5366398005&#xA;-7.87165586175&#xA;-24.9012815104&#xA;-23.9148486368&#xA;-10.047780535&#xA;-4.03376369723&#xA;-7.07428842783&#xA;-7.44222705099&#xA;-6.25705487929&#xA;-13.2313513762&#xA;-13.3253819521&#xA;-9.23993173528&#xA;-7.141616656&#xA;-5.57915693405&#xA;-6.82400483045&#xA;-15.0906961724&#xA;-3.37447211233&#xA;-5.41032267015&#xA;-5.75224753811&#xA;-19.7230390792&#xA;-6.75268922909&#xA;-4.04911793705&#xA;-10.6062761691&#xA;-3.17417070498&#xA;-9.95916350005&#xA;-3.25893428094&#xA;-3.88566777358&#xA;-3.30908856716&#xA;-3.58141292341&#xA;-3.90778368669&#xA;-4.01462493538&#xA;-11.6683969455&#xA;-5.30068548445&#xA;-24.3400870389&#xA;-7.66035331181&#xA;-13.8321672858&#xA;-8.93461397086&#xA;-17.4068326999&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the data itself:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;49.6,34917,24325.4,305,101350,98678,254.8,276.9,47.5,1,3,5.6,3.59,11.9,0,97.5,97.6,8,10,100,0,0,96.93,610.1,100,1718.22,6.7,28,5&#xA;275.8,14667,11114.4,775,75002,74677,30,109,9.1,1,0,6.5,3.01,8.2,1,97.5,97.6,8,8,100,0,0,100,1558,100,2063.17,5.5,64,5&#xA;2.3,9372.5,8035.4,4.6,8111,8200,8.01,130,1.2,0,5,0,3.33,6.09,1,97.9,97.9,8,8,67.3,342.3,0,99.96,18.3,53,1457.27,4.8,8,4&#xA;7.10,13198.0,13266.4,1.1,708,695,6.1,80,0.4,0,4,0,3.1,8.2,1,97.8,97.9,8,8,45,82.7,0,99.68,4.5,80,1718.22,13.8,0,3&#xA;1.97,2466.7,2900.6,19.7,5358,5335,10.1,23,0.5,0,2,0,3.14,8.2,0,97.3,97.2,9,9,74.5,98.2,0,99.64,79.8,54,1367.89,6.4,12,4&#xA;2.40,2999.4,2218.2,0.80,2045,2100,8.9,10,1.5,1,3,0,2.82,8.6,0,97.4,97.2,8,8,47.2,323.8,0,99.996,13.6,24,1249.67,2.7,12,3&#xA;0.59,4120.8,5314.5,0.54,14680,13688,14.9,117,1.1,0,3,0,2.94,3.4,0,97.6,97.7,8,8,11.8,872.6,0,100,9.3,52,1251.67,14,14,2&#xA;0.72,2067.7,2364,3,367,298,7.2,60,2.5,0,12,0,2.97,10.5,0,97.5,97.6,8,8,74.7,186.8,0,99.13,12,57,1800.45,2.7,4,2&#xA;1.14,2751.9,3066.8,3.5,1429,1498,7.7,9,1.6,0,3,0,2.86,7.7,0,97.6,97.8,8,9,76.7,240.1,0,99.93,13.6,60,1259.97,15,8,3&#xA;1.29,4802.6,5026.1,2.7,7859,7789,6.5,45,1.9,0,3,0,2.5,8.2,0,98,98,8,8,34,297.5,0,99.95,10,30,1306.44,8.5,0,4&#xA;0.40,639.0,660.3,1.3,23,25,1.5,9,0.1,0,1,0,2.5,8.2,0,97.7,97.8,8,8,94.2,0,0,100,4.3,50,1565.44,19.2,0,4&#xA;0.26,430.7,608.1,2,33,28,2.5,7,0.4,0,6,0,2.5,8.2,0,97.4,97.4,8,8,76.5,0,0,98.31,8,54,1490.08,0,0,4&#xA;4.99,2141.2,2357.6,3.60,339,320,8.1,7,0.2,0,8,0,2.5,5.9,0,97.3,97.4,8,8,58.1,206.3,0,99.58,13.2,95,1122.92,14.2,8,2&#xA;0.36,1453.7,1362.2,3.50,796,785,3.7,9,0.1,0,9,0,2.5,13.6,0,98,98.1,8,8,91.4,214.6,0,99.74,7.5,53,1751.98,11.5,0,2&#xA;0.36,1657.5,2421.1,2.8,722,690,8.1,8,0.4,0,1,0,2.5,8.2,0,97.2,97.3,11,12,37.4,404.2,0,99.98,10.9,35,1772.33,10.2,8,3&#xA;1.14,5635.2,5649.6,3,2681,2530,5.4,20,0.3,0,1,0,3.1,8.2,0,97.7,97.8,8,11,50.1,384.7,0,99.02,11.6,27,1306.08,16,0,2&#xA;0.6,1055.9,1487.9,1.3,69,65,2.5,6,0.4,0,8,0,2.5,8.2,0,97.9,97.7,8,11,63,137.9,0,99.98,5.1,48,1595.06,0,0,4&#xA;0.08,795.3,1174.7,1.40,85,76,2.2,7,0.2,0,0,0,2.5,8.2,0,97.4,97.5,8,8,39.3,149.3,0,98.27,5.1,52,1903.9,8.1,0,2&#xA;0.90,2514.0,2644.4,2.6,1173,1104,5.5,43,0.8,0,10,0,2.5,13.6,0,97.5,97.5,8,10,58.7,170.5,0,80.29,10,34,1292.72,4,0,2&#xA;0.27,870.4,949.7,1.8,252,240,2.2,31,0.2,0,1,0,2.5,8.2,0,97.5,97.6,8,8,64.5,0,0,100,6.6,29,1483.18,9.1,0,3&#xA;0.41,1295.1,2052.3,2.60,2248,2135,6.0,12,0.8,0,4,0,2.7,8.2,0,97.7,97.7,8,8,71.1,261.3,0,91.86,4.6,21,1221.71,9.4,0,4&#xA;1.10,3544.2,4268.9,2.1,735,730,6.6,10,1.7,0,14,0,2.5,8.2,0,97.7,97.8,8,8,52,317.2,0,99.62,9.8,46,1271.63,14.2,0,3&#xA;0.22,899.3,888.2,1.80,220,218,3.6,7,0.5,0,1,0,2.5,8.2,0,97.2,97.5,8,8,22.5,0,0,70.79,10.6,32,1508.02,0,0,4&#xA;0.24,1712.8,1735.5,1.30,41,35,5.4,7,0.5,0,1,0,3.28,8.2,0,97.8,97.8,9,10,16.6,720.2,0,99.98,4.3,53,1324.46,0,4,2&#xA;0.2,558.4,631.9,1.7,65,64,2.5,7,0.2,0,5,0,2.5,8.2,0,97.7,97.5,8,8,60.7,0,0,99.38,6.1,52,1535.08,0,0,2&#xA;0.21,599.9,1029,1.1,69,70,3.7,85.7,0.1,0,12,0,2.5,8.2,0,97.4,97.5,8,8,48.6,221.2,0,100,5.4,40,1381.44,25.6,0,2&#xA;0.10,131.3,190.6,1.6,28,25,2.9,7,0.3,0,3,0,2.5,8.2,0,97.7,97.8,8,8,58.9,189.4,0,99.93,6.9,42,1525.58,17.4,0,3&#xA;0.44,3881.4,5067.3,0.9,2732,2500,11.2,10,1.5,0,5,0,2.67,8.2,0,97.4,97.3,8,11,14.5,1326.2,0,99.06,3.7,31,1120.54,10.3,10,2&#xA;0.18,1024.8,1651.3,1.01,358,345,4.6,35,0.3,0,2,0,2.5,8.2,0,97.8,97.9,8,10,15.9,790.2,0,100,4.3,48,1531.04,10.5,0,3&#xA;0.46,682.9,784.2,1.8,103,109,2.2,8,0.4,0,4,0,2.5,8.2,0,97.8,97.9,8,8,82.7,166.3,0,99.96,6.4,44,1373.6,13.5,0,2&#xA;0.12,370.4,420.0,1.10,28,25,3.4,10,0.1,0,6,0,2.57,8.2,0,97.6,97.8,8,11,51.6,120,0,99.85,8.1,40,1297.94,0,0,3&#xA;0.03,552.4,555.1,0.8,54,49,3.5,10,0.4,0,0,0,2.5,8.2,0,97.4,97.6,8,10,33.6,594.5,0,100,3.2,41,1184.34,6.6,0,3&#xA;0.21,1256.5,2434.8,0.9,1265,1138,6.3,20,1.3,0,2,0,2.6,8.2,0,98,97.9,8,9,20.1,881,0,99.1,3.9,31,1265.93,7.8,0,3&#xA;0.09,320.6,745.7,1.10,37,25,2.7,8,0.3,0,9,0,2.5,8.2,0,98,97.8,8,8,49.2,376.4,0,99.95,4.3,39,1285.11,0,0,3&#xA;0.08,452.7,570.9,1,18,20,4.7,9,0.6,0,2,0,2.45,8.2,0,97.1,97.1,8,8,19.9,1103.8,0,99.996,2.9,22,1562.61,21.9,0,3&#xA;0.13,967.9,947.2,1,74,65,4.0,25,1.4,0,6,0,2.5,8.2,0,98,98,9,11,30.1,503.1,0,99.999,3.4,55,1269.33,0,0,2&#xA;0.07,495.0,570.3,1.2,27,30,4.3,7,0.5,0,12,0,3.62,8.2,0,98.2,98.2,15,13,29.8,430.5,0,99.7,4.9,40,1461.79,14.6,0,2&#xA;0.17,681.9,537.4,1.1,113,120,2.9,12,0.4,0,8,0,2.5,8.2,0,98.2,98.3,8,8,24,74.3,0,100,5,43,1290.16,0,0,3&#xA;0.05,639.7,898.2,0.40,9,12,3.0,7,0.1,0,1,0,2.5,8.2,0,97.6,97.8,15,11,11.9,1221.1,0,99.996,1.7,40,1372,7,0,4&#xA;0.65,2067.8,2084.2,2.50,414,398,7.3,6,0.7,0,4,0,2.16,8.2,0,97.8,97.9,12,12,60.1,146.3,0,99.96,10.4,44,1059.68,7.4,0,2&#xA;0.12,804.4,1416.4,3.30,579,602,4.2,7,1.8,0,1,0,2.5,8.2,0,98.1,98.3,8,10,8.9,2492.3,0,95.4,2.2,34,1345.76,7,0,2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;P.S. Don't have enough rep to create &lt;code&gt;factor-analysis&lt;/code&gt; tag.&lt;/p&gt;&#xA;" OwnerUserId="17228" LastEditorUserId="17228" LastEditDate="2015-08-12T12:22:33.700" LastActivityDate="2015-08-12T12:22:33.700" Title="How to adjust scikit-learn FactorAnalysis() settings to get scores output similar to one from factanal() in R?" Tags="|python|r|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20429" PostTypeId="2" ParentId="20411" CreationDate="2015-08-11T11:53:41.093" Score="2" Body="&lt;p&gt;As the OP is aware, when $\mathbf{w}$ is nonnegative, an upper bound of the required type can be obtained by taking $\mathbf{z}$ to be the componentwise maximum of the various $\mathbf{x}_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However a simple example shows that no choice of $\mathbf{z}$ is possible when $\mathbf{w}$ is allowed to have a negative entry.  Consider the vectors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{x}_1 = (1,0,0)^T \; \text{ and } \; \mathbf{x}_2 = (-1,0,0)^T $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then whatever choice of $\mathbf{z} = (z_1,z_2,z_3)^T$ is made, there exists $\mathbf{w} = (w_1,w_2,w_3)^T$ for which the inequality $\max \mathbf{w}^T\mathbf{x}_i \leq \mathbf{w}^T \mathbf{z}$ fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, if $z_1 \le 0$, the choice $\mathbf{w} = (1,0,0)^T$ yields $\max \mathbf{w}^T \mathbf{x}_i = 1$, and $\mathbf{w}^T \mathbf{z} = z_1 \not \ge 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if $z_1 \ge 0$, the choice $\mathbf{w} = (-1,0,0)^T$ yields $\max \mathbf{w}^T \mathbf{x}_i = 1$, and $\mathbf{w}^T \mathbf{z} = -z_1 \not \ge 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore no choice of $\mathbf{z}$ is satisfactory for all $\mathbf{w}$.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2015-08-11T11:53:41.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20430" PostTypeId="2" ParentId="20425" CreationDate="2015-08-11T12:49:29.913" Score="4" Body="&lt;p&gt;If you want the simplest possible numerical scheme working for Burger's equation that has your suggested form then you should prefer the so called Lax-Friedrichs method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have the book of LeVeque on Finite Volume Methods for Hyperbolic Problems, look for a very simple formula 4.20 (or a little bit more complex 4.21, but in your suggested form). With that scheme, the oscillations shall disappear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be aware that with the Lax-Wendroff scheme mentioned in other response the oscillations in numerical solution can remain, because it is so called second order accurate method that can not avoid the oscillations in general. The Lax-Friedrichs method is only first order accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the simplicity is your criterion number one, then be aware that you can have still a strange behaviour of numerical solution with Lax-Friedrichs method, the so called clustering of values, see Figure 12.3 in the book of LeVeque. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to avoid such clustering, take the Godunov scheme, see the formula 12.2 that is also first order accurate with no oscillations in numerical solution and no clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know in advance that your initial function (you know you need to know the values of $U(x,0)$) is positive everywhere, then the scheme 12.2 turns to the so called upwind scheme that has very simple form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F_{i-0.5}^n=0.5 (U_{i-1}^n)^2, \quad F^n_{i+0.5}=0.5 (U_i^n)^2  .$$&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2015-08-11T12:49:29.913" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20432" PostTypeId="1" CreationDate="2015-08-11T17:35:43.790" Score="4" ViewCount="705" Body="&lt;p&gt;Time-explicit high order finite element/DG methods for the advection equation tend to have a timestep restriction which looks like&#xA;$$dt &amp;lt; C h/p^2$$&#xA;where $h$ is mesh spacing and $p$ is polynomial degree. I'm having trouble finding a similar order-dependent restriction for high order finite difference stencils.  Is there one known, or does it depend more on the specific stencil used instead of the order of the stencil?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Given Kirill's answer, I'll look instead for the growth of $\rho(A)$ as a proxy for the CFL constant (for example, DG gives $\rho(A) &amp;lt; Ch/p^2$). Is it known how $\rho(A)$ for FD discretizations depends on stencil order or type?&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-08-11T19:27:26.590" LastActivityDate="2015-08-13T14:50:21.503" Title="CFL for high order finite difference" Tags="|finite-difference|cfl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20433" PostTypeId="2" ParentId="20402" CreationDate="2015-08-11T18:15:32.433" Score="3" Body="&lt;p&gt;Both ILU and diagonal scaling are not efficient preconditioners for &quot;real&quot; problems, i.e., if you let the size of your discrete problem become large. In other words, if you insist on timing GPUs vs CPUs with these preconditioners, you are only considering academic problems that are not really of interest -- if the problem is small enough that the use of these preconditioners make sense, then the problem is so small that nobody actually cares about the run time in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do care about run time (because run time is an obstacle to doing actual simulations, not just because you want to do an academic timing study without practical implications), then you need to use preconditioners that make sense for large problems -- and these are typically based on the physical structure of the underlying problem, not just on the matrix values. An excellent example is the one by Silvester and Wathen. We implement and discuss its use in step-22 of the deal.II tutorial (see &lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/step_22.html&quot; rel=&quot;nofollow&quot;&gt;https://www.dealii.org/developer/doxygen/deal.II/step_22.html&lt;/a&gt;); pay particular attention to the discussion in the &quot;Possibilities for extensions&quot; section at the end of the &quot;Results&quot; section.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-08-11T18:15:32.433" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20434" PostTypeId="2" ParentId="20432" CreationDate="2015-08-11T18:45:01.043" Score="5" Body="&lt;p&gt;I'm going to guess that there isn't one. In the usual method of lines for $u_t +au_x=0$, you end up with a system of ODEs of the form&#xA;$$ u_t = Au. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the restriction on the time step comes from &lt;em&gt;(1)&lt;/em&gt; requiring that eigenvalues of $A$ lie in the domain of stability of whatever time-stepping method you use (e.g., if $A$ is skew-symmetric, with imaginary eigenvalues, then forward Euler is always unstable), and &lt;em&gt;(2)&lt;/em&gt; from minimizing error (if both space and time discretizations are second-order, then $\delta t=\delta x$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So based on this reasoning, I'd say no, because there is too much freedom in how you can pick the r.h.s. matrix and the time-stepping method, and the result has less to do specifically with the order of the method, and more to do with other properties, mainly $A$'s spectrum. Of course this doesn't &lt;em&gt;quite&lt;/em&gt; rule out such a formula, but my understanding is that the stability analysis has to be done case-by-case.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-08-11T18:58:08.593" LastActivityDate="2015-08-11T18:58:08.593" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20435" PostTypeId="2" ParentId="20424" CreationDate="2015-08-11T19:22:30.670" Score="3" Body="&lt;p&gt;Glad the answer was in the $b$ vector. The answer to the original equation depends on exactly what method you use to solve the system (backslash has a ton of different options built into it). You usually have that the relative difference in your solutions is $O(\kappa \epsilon)$, where $\epsilon$ is the difference between the matrices and $\kappa$ is the condition number of $A,B$. An easier way of thinking about it is that the order of magnitude of your conditioning illustrates roughly how many digits difference you can expect (for a stable method of solving the system).&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-08-11T22:43:06.857" LastActivityDate="2015-08-11T22:43:06.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20436" PostTypeId="2" ParentId="20424" CreationDate="2015-08-11T21:28:51.467" Score="3" Body="&lt;p&gt;This cannot happen, unless the matrices are very ill-behaved. If you have two nonsingular matrices $A, B$ with &#xA;$$ A-B = O(\epsilon), $$&#xA;then the relative error in solving $Ax=b$ is&#xA;$$&#xA;\begin{aligned}&#xA;\frac{\|A^{-1}b-B^{-1}b\|}{\|A^{-1}b\|}&#xA;&amp;amp;\leq \sup_x \frac{\|A^{-1}x-B^{-1}x\|}{\|A^{-1}x\|}&#xA;\\&amp;amp;\leq \sup_y \frac{\|y - B^{-1}Ay\|}{\|y\|}&#xA;\\&amp;amp;\leq \|I - B^{-1}A\|&#xA;\\&amp;amp;\leq \|B^{-1}(B-A)\|&#xA;\\&amp;amp;\leq \|B^{-1}\| \|A-B\|&#xA;\\&amp;amp;\leq \frac{\kappa(B)}{\|B\|}\|A-B\|&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if $A$ and $B$ are identical to within $\epsilon$, and also have, as you say, identical condition numbers $\kappa(A)=\kappa(B)$, then the relative error can be at most $ \kappa(A)\epsilon/\|A\|. $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, for the error to be on the order of $O(1)$, the condition number of $A$ must be on the order of $\|A\|\epsilon^{-1}$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-08-11T22:27:29.417" LastActivityDate="2015-08-11T22:27:29.417" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20437" PostTypeId="2" ParentId="20425" CreationDate="2015-08-12T06:00:24.797" Score="3" Body="&lt;p&gt;@Peter Frolkovic's answer is a good one, but @Daniel Ruprecht's comment also deserves to be highlighted: &lt;strong&gt;the scheme you are using (centered in space, forward in time) is unstable for any time step size&lt;/strong&gt;.  It's straightforward to see this if you consider instead the advection equation and do a standard von Neumann or method of lines stability analysis.  This is covered in most introductory books on numerics for hyperbolic PDEs or finite difference methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another resource for learning about good methods to solve this type of equations is my &lt;a href=&quot;https://github.com/ketch/HyperPython&quot; rel=&quot;nofollow&quot;&gt;HyperPython&lt;/a&gt; course; in particular, the &lt;a href=&quot;http://nbviewer.ipython.org/github/ketch/HyperPython/blob/master/Lesson_03_High-resolution_methods.ipynb&quot; rel=&quot;nofollow&quot;&gt;3rd lesson&lt;/a&gt; will take you beyond the first order methods mentioned in @Peter Frolkovic's answer.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-08-12T06:00:24.797" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20438" PostTypeId="1" CreationDate="2015-08-12T11:49:06.753" Score="3" ViewCount="101" Body="&lt;p&gt;I obtain numerical discrete data of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;S_{raw}(\omega) = \sum_{j}w_{j} \delta(\omega-\omega_{j})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to compare the result with experimental data the delta peaks need to be broadened according to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;S_{broad}(\omega) = \int d\omega^{\prime} K(\omega,\omega^{\prime}) S_{raw}(\omega^{\prime})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $$ K(\omega, \omega^{\prime})$$ is the broadening kernel i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;K(\omega,\omega^{\prime}) = \frac{\theta(\omega\omega^{\prime})}{\sqrt{\pi}b|\omega|}\exp\left[-\left(\frac{1}{b}\ln\left|\frac{\omega}{\omega^{\prime}}\right| - \frac{b}{4}\right)^2\right].&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $$K(\omega,\omega^{\prime}) = K(\omega-\omega^{\prime})$$ the integral equation above can be interpreted as a convolution and can be efficiently calculated by means of the convolution theorem and FFTs. But the example kernel $K$ does not have this property.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask me now if there is still a way to compute the broadened function by exploiting the convolution theorem and fast FFT algorithms ?&lt;/p&gt;&#xA;" OwnerUserId="17235" LastEditorUserId="9667" LastEditDate="2015-08-12T23:06:02.787" LastActivityDate="2015-08-12T23:06:02.787" Title="Broadening spectral data by using FFT's" Tags="|fourier-analysis|integral-equations|fftw|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20439" PostTypeId="1" CreationDate="2015-08-12T19:24:50.067" Score="1" ViewCount="58" Body="&lt;p&gt;I am looking for a way to simulate behavioral economic models to illustrate the effects of changing parameters. I am familiar with statistical analysis (especially R), but I only have few programming skills beyond R. I think that if there is a way to achieve my goal (I'm not really sure if it is possible at all!) that Octave should be a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually such a model defines some kind of game with actions, strategies and payoffs for every player. Then these actions get evaluated and normally an equilibrium between the players actions/strategies is determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of my desired models is Rabin's idea of Fairness Equilibria. Pages 1286ff can give a short idea of how Rabin defines the game and determines the equilibrium. &lt;a href=&quot;http://pascal.iseg.utl.pt/~depeco/summerschool2007/Rabin.pdf&quot; rel=&quot;nofollow&quot;&gt;LINK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to have a function that depicts that model and enables me to illustrate the effects of different assumptions and situations within the frame of the specific model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: Is it possible at all? Is Octave suitable or should I choose another language/environment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any sort of advice!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to sketch the model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting point is a two player normal form game (Prisoner's Dilemma) with the strategy sets $S_1, S_2$ for every player from which they choose their actions. The payoff function for each player is defined as $\pi_i = S_1 \times S_2$. So player $i$ chooses action $a_i \in S_i$. Now player $j$ has a believe about $i$'s strategy $b_i \in S_i$ and player $i$ has a belief about player $j$'s belief about $i$'s strategy $c_i \in S_i$. &#xA;So the feasible payoffs for $i$ depending on his beliefs are in the set $\Pi(b_j) = \{(\pi_i(a_i,b_j),\pi_j(b_j,a_i)) | a_i \in S_i\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's the framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The routine now has to determine&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\pi_j^h(b_j)$ as the highest payoff for $j$ depending on $i$'s belief that is Pareto efficient in $\Pi(b_j)$&lt;/li&gt;&#xA;&lt;li&gt;$\pi_j^l(b_j)$ as the lowest payoff for $j$ depending on $i$'s belief that is Pareto efficient in $\Pi(b_j)$&lt;/li&gt;&#xA;&lt;li&gt;$\pi_j^{min}(b_j)$ as the worst possible payoff for $j$ depending on $i$'s belief&lt;/li&gt;&#xA;&lt;li&gt;the reference point for $j$: $\pi_j^e(b_j) = \frac{\pi_j^h(b_j) + \pi_j^l(b_j)}{2}$&lt;/li&gt;&#xA;&lt;li&gt;the kindness of $i$ towards $j$: $\kappa_i(a_i,b_j)=\frac{\pi_j(b_j,a_i) - \pi_j^e(b_j)}{\pi_j^h(b_j) - \pi_j^{min}(b_j)}$&lt;/li&gt;&#xA;&lt;li&gt;believed kindness of $j$ towards $i$: $\bar\kappa_j(b_j,c_i)=\frac{\pi_i(c_i,b_j) - \pi_i^e(c_i)}{\pi_i^h(c_i) - \pi_i^{min}(c_i)}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then the Expected Utility is defined as: $U_i(a_i,b_j,c_i) = \pi(a_i,b_j) + \bar\kappa_j(b_j,c_i) \cdot [1+\kappa_i(a_i,b_j)]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to be a Fairness Equilibrium, a pair of strategies $(a_1^*,a_2^*)$ has to fulfill following conditions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$a_i^* \in \text{argmax}_{a_i \in S_i} U_i(a_i,b_j,c_i)$&lt;/li&gt;&#xA;&lt;li&gt;$c_i = b_j = a_i^*$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So my goal is to get a routine that determines these Fairness Equilibria when a certain game is given.&lt;/p&gt;&#xA;" OwnerUserId="17239" LastEditorUserId="17239" LastEditDate="2015-08-17T12:49:02.993" LastActivityDate="2015-08-17T12:49:02.993" Title="Software to simulate behavioral economic models?" Tags="|simulation|octave|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20440" PostTypeId="1" CreationDate="2015-08-12T21:15:08.500" Score="3" ViewCount="73" Body="&lt;p&gt;I'm doing a 2D combustion hydrodynamic simulation and there's a hydrodynamic instability that should be triggered because of the particular physical properties of my system. The key to the instability are temperature gradients perpendicular to the flow direction of my problem. What is a good and &quot;consistent&quot; way to artificially add fluctuations in this perpendicular direction to trigger this instability?&lt;/p&gt;&#xA;" OwnerUserId="17075" LastActivityDate="2015-08-13T12:31:03.250" Title="Adding artificial fluctuations to trigger hydrodynamic instabilities." Tags="|fluid-dynamics|numerical-modelling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20441" PostTypeId="2" ParentId="20440" CreationDate="2015-08-12T23:36:01.473" Score="2" Body="&lt;p&gt;If the problem is truly unstable, round off error is probably enough to kick off the phenomena, though it might take awhile. If there's a small stability region such that you're stuck in a steady state, you might try restarting the flow from the steady solution with a small random or sinusoidal perturbation in the temperature. I prefer this to messing with the velocity and pressure since there's generally no consistency with the continuity equation to worry about. There may be similar arguments to not perturbing the species concentrations since you don't want to have to worry about consistency in the mass. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edited to add: If perturbing your temperature field makes your worry about consistency with your equation of state, you can always add a right-hand-side function to your energy equation and put the perturbation function there for a few time steps instead of relying on a perturbed initial condition.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-08-13T01:34:14.157" LastActivityDate="2015-08-13T01:34:14.157" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20442" PostTypeId="1" AcceptedAnswerId="20443" CreationDate="2015-08-13T02:42:03.270" Score="2" ViewCount="777" Body="&lt;p&gt;I need to calculate the Jacobian matrix of a subroutine F(U). Both F and U are of size N(=O($10^5$)). Using Tapenade, I differentiated the routine in tangent mode. I cannot calculate the full Jacobian directly because of the large memory requirement.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a package that I can use to parse the differentiated FORTRAN routines to get the sparsity pattern? preferably in python.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I plan to naively construct the Jacobian matrix using N tangent calculations, using sparsity pattern to reduce the memory usage. Is there a better way of doing this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would like to stick with Tapenade because of other issues.&lt;/p&gt;&#xA;" OwnerUserId="3640" LastActivityDate="2019-08-08T13:47:15.793" Title="Constructing sparsity pattern of the Jacobian of a FORTRAN subroutine" Tags="|sparse-matrix|fortran|automatic-differentiation|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20443" PostTypeId="2" ParentId="20442" CreationDate="2015-08-13T04:22:20.313" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I need to calculate the Jacobian matrix of a subroutine F(U). Both F and U are of size $N(=O(10^5))$. Using Tapenade, I differentiated the routine in tangent mode. I cannot calculate the full Jacobian directly because of the large memory requirement.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are AD (automatic differentiation) software packages that can do this calculation; I know &lt;a href=&quot;https://yoric.mit.edu/daepack&quot; rel=&quot;nofollow&quot;&gt;DAEPACK&lt;/a&gt; can, because it came out of my PhD advisor's group. It appears that Tapenade does not have sparsity analysis capabilities.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Is there a package that I can use to parse the differentiated FORTRAN routines to get the sparsity pattern? preferably in python.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My intuition is that it probably doesn't exist, because it would require a lot of static analysis. A cursory Google search backs that up.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;I plan to naively construct the Jacobian matrix using N tangent calculations, using sparsity pattern to reduce the memory usage. Is there a better way of doing this?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Probably. It really depends on what you want to do with the Jacobian matrix. If you mostly want to solve linear systems that are relatively well-conditioned or are amenable to preconditioning, I'd suggest instead considering matrix-free approaches, which would allow you to make use of the tangents you calculate. A Jacobian-free Newton-Krylov approach might even be viable, which would make AD unnecessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you really need the entire Jacobian matrix, I'd suggest sparse AD instead.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-08-27T18:01:52.690" LastActivityDate="2015-08-27T18:01:52.690" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20444" PostTypeId="1" AcceptedAnswerId="20461" CreationDate="2015-08-13T08:29:35.067" Score="1" ViewCount="340" Body="&lt;p&gt;Is there any universal method to fill this matrix for any $n$ value:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{A} = \left[ \matrix{n &amp;amp; \sum x_i &amp;amp; \sum x_i^2  &amp;amp; \cdots &amp;amp; \sum x_i^n \cr&#xA;             \sum x_i &amp;amp; \sum x_i^2 &amp;amp; \sum x_i^3 &amp;amp; \cdots &amp;amp; \sum x_i^{n+1} \cr&#xA;              \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \cr&#xA;              \sum x_i^n &amp;amp; \sum x_i^{n+1} &amp;amp; \sum x_i^{n+2} &amp;amp;\cdots &amp;amp; \sum x_i^{n+n}} \right]$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or maybe, it is easier to carry out with Vandermonde matrix to calculate the polynomial coefficients?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be grateful for any suggestions and Fortran code.&lt;/p&gt;&#xA;" OwnerUserId="16846" LastEditorUserId="16846" LastEditDate="2015-08-13T15:27:45.487" LastActivityDate="2018-01-11T20:30:10.000" Title="Polynomial approximation" Tags="|numerical-analysis|statistics|regression|polynomials|" AnswerCount="2" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20447" PostTypeId="2" ParentId="20444" CreationDate="2015-08-13T12:21:50.890" Score="8" Body="&lt;p&gt;$\mathbf{A}$ is an $(n+1) \times (n+1)$ matrix. It can be obtained as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{A} = \left[ \matrix{1 &amp;amp; 1 &amp;amp; 1 &amp;amp; \cdots &amp;amp; 1 \cr&#xA;              x_0 &amp;amp; x_1 &amp;amp; x_2 &amp;amp; \cdots &amp;amp; x_{n} \cr&#xA;              \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \cdots &amp;amp; \vdots \cr&#xA;              x_0^n &amp;amp; x_1^n &amp;amp; x_2^n &amp;amp; \cdots &amp;amp; x_n^{n}}\right] \left[ \matrix{1 &amp;amp; x_0 &amp;amp; x_0^2 &amp;amp; \cdots &amp;amp; x_0^n \cr&#xA;              1 &amp;amp; x_1 &amp;amp; x_1^2 &amp;amp; \cdots &amp;amp; x_1^{n} \cr&#xA;              \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \cdots &amp;amp; \vdots \cr&#xA;              1 &amp;amp; x_n &amp;amp; x_n^2 &amp;amp; \cdots &amp;amp; x_i^{n}}\right] $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the decomposition doesn't involve any summation and can be easily computed using simple indexing and power operations. The matrix multiplication is an effective operator to compute the final matrix $\mathbf{A}$. This is both efficient and easy to implement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix on the right is simply a &lt;a href=&quot;https://en.wikipedia.org/wiki/Vandermonde_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vandermonde matrix&lt;/a&gt; with an added column (right-most). Here is a MATLAB 2-liner to implement this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% x is the input column vector, n is the number of elements e.g. n=length(x)&#xA;W = [ fliplr(vander(x)) (x.^n)];&#xA;A = W'*W;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should work faster than for-looping in MATLAB since &lt;code&gt;vander&lt;/code&gt; function doesn't contain any loops. It is also definitely shorter and cleaner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that making a quick remark is important here. Unlike common belief, when properly implemented matrix multiplication is not $O(N^3)$, but $&amp;lt;O(N^{2.4})$. Many available packages are already very optimized. Using vector instruction sets, along with parallelization brings further speed-up. However, the theoretical complexity never comes to $O(N^2)$ which is achieved via for-looping. This approach is only neater and faster in MATLAB for small sized matrices (up to ~ N&amp;lt;1500).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that, even if the matrix notation is used here, the multiplication could still benefit from the special structure of these matrices. If one is to utilize this property, similar complexity can be obtained.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2018-01-11T20:30:10.000" LastActivityDate="2018-01-11T20:30:10.000" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="20448" PostTypeId="2" ParentId="20440" CreationDate="2015-08-13T12:31:03.250" Score="1" Body="&lt;p&gt;The Gaussian is annoying to add since it isn't compactly supported. The bump function solves that but has nasty higher derivatives that require resolution. I spent a little time once figuring how to get a nicer bump function for just the purpose you describe. Write up athttp://www.scribd.com/mobile/doc/61560918/Behavior-of-the-bump-function-s-derivatives&lt;/p&gt;&#xA;" OwnerUserId="327" LastActivityDate="2015-08-13T12:31:03.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20449" PostTypeId="1" CreationDate="2015-08-13T12:56:17.163" Score="2" ViewCount="553" Body="&lt;p&gt;Consider a set of polytopes $P_i : i=1,2,...,k$ each of which has a&#xA;structure as  $P_i:= \{(x_{i1},x_{i2},..., x_{in})\; |\; x_{ij} \in [a_{ij}, b_{ij}] \subseteq [0,1]\}\;\; \text{for all}\;\; j=1,...,n$   and   $\sum_j x_{ij}=1$.&#xA;We define $P= CH (\cup_i  P_i)$ where $CH$ stands for the convex hull.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other side, we have another set of polytopes $Q_t: t=1,2,..., r$&#xA;each of which defined similarly as $Q_t:= \{(y_{t1},y_{t2},..., y_{tn})\; |\; y_{tj} \in&#xA;[c_{tj}, d_{tj}] \subseteq [0,1]\; \text{for all}\; j=1,...,n \;   \text{and}   \sum_j y_{tj}=1\}$.&#xA;Then we define $Q= CH (\cup_t  Q_t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that the number of polytopes $P_i$, i.e., &lt;em&gt;k&lt;/em&gt; and the number of&#xA;polytopes $Q_t$, i.e., &lt;em&gt;r&lt;/em&gt; are not necessarily equal. However, both polytopes&#xA;$P_i$ and $Q_t$ are defined in $\mathbb{R}^n$ as above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Query&lt;/strong&gt;: Is checking if $P\neq Q$ computationally hard?&lt;/p&gt;&#xA;" OwnerUserId="917" LastEditorUserId="917" LastEditDate="2015-08-18T13:20:01.020" LastActivityDate="2015-08-18T13:20:01.020" Title="Comparison of convex hulls" Tags="|algorithms|computational-geometry|convex-optimization|complexity|convex-hull|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ClosedDate="2015-08-19T17:16:38.303" ContentLicense="CC BY-SA 3.0" />
  <row Id="20450" PostTypeId="2" ParentId="20432" CreationDate="2015-08-13T13:01:53.840" Score="1" Body="&lt;p&gt;At least in one dimensional case for linear advection I have not experienced for finite difference methods the CFL restriction depending on the precision order as you write. Conditionally stable explicit in time well designed numerical schemes have typically the restriction $dt \le C h$,  I have such experience (and proofs) with 2nd and 3rd order accurate schemes, the stencil being at most 5 nodes, i. e. from i-2 to i+2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Opposite to DG methods where degrees of freedom for higher methods are localized at the same geometric location (element), the standard finite difference methods are increasing also the geometric stencil when increasing the precision order. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Standard text books on finite difference (or related finite volume) methods relate the CFL condition to a &quot;domain of influence&quot; that depends on the equation you solve. Maybe some &quot;geometric&quot; clustering of degrees of freedom in DG method introduce the unfavourable influence of CFL condition on the order of method, but my subjective opinion would not expect it with finite difference methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addendum&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was asked for references, here is one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Principles of Computational Fluid Dynamics - Pieter Wesseling, 2001, see Theorem 9.3.3 on so called kappa class of schemes for 1D advection equation $u_t + v  u_x=0$ with constant speed $v$. Several 2nd order schemes belong to this class and one 3rd order accurate (QUICKEST), the stability is reached for Courant number $dt v/h$ less or equal 1. It is so called von Neumann stability (Fourier series analysis).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not work with higher order methods than 3, but I know from papers and conferences that e.g. so called ADER schemes (M. Dumbser use it also for DG) can produce arbitrary order of accuracy and I do not know about any bad behaviour of schemes with respect to the CFL condition, but I might be wrong.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2015-08-13T14:50:21.503" LastActivityDate="2015-08-13T14:50:21.503" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20451" PostTypeId="1" CreationDate="2015-08-13T13:51:36.107" Score="0" ViewCount="46" Body="&lt;p&gt;I am starting to get into scientific computing with a library called Deal.II and I was wondering what the community recommends as good source material that I can learn about scientific material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the Finite Element Method and how to setup problems within the code. How to think about the problems, etc. &lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="9667" LastEditDate="2015-08-13T17:36:09.557" LastActivityDate="2015-08-13T17:36:09.557" Title="Reference Material" Tags="|reference-request|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20452" PostTypeId="2" ParentId="20451" CreationDate="2015-08-13T14:07:24.280" Score="4" Body="&lt;p&gt;This is probably going to get closed as off-topic, but the father of deal.II is active here, and he would point you to &lt;a href=&quot;https://www.dealii.org/8.2.0/doxygen/deal.II/Tutorial.html&quot; rel=&quot;nofollow&quot;&gt;the tutorials&lt;/a&gt; and &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;his scientific computing lectures&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-08-13T14:07:24.280" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20453" PostTypeId="1" CreationDate="2015-08-13T16:44:33.533" Score="1" ViewCount="61" Body="&lt;p&gt;I have the following LP:&#xA;$$&#xA;\begin{array}{ll}&#xA;\text{Minimize} &amp;amp; \sum_{j=1}^n x_j \\&#xA;\text{Subject to} &amp;amp; \sum_{j=1}^n a_{ij} x_j \geq b_i,~~~i\in\{1,\ldots,M\} \\&#xA;                  &amp;amp; 0 \leq x_j \leq 1,~~~j\in\{1,\ldots,n\}&#xA;\end{array}&#xA;$$&#xA;where all $a_{ij} \geq 0, \forall i,j$ and $b_i \geq 0,\forall i$, and these are known. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a quick &lt;em&gt;direct&lt;/em&gt; solution for this LP if it exists. We know that $n$ is much smaller than $M$. This seems similar to the fractional multi-Knapsack problem. I am wondering if there exists a greedy solution that I can use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="1932" LastActivityDate="2015-08-13T16:44:33.533" Title="Solving an LP greedily" Tags="|optimization|linear-programming|combinatorics|" AnswerCount="0" CommentCount="3" ClosedDate="2019-06-09T22:11:49.907" ContentLicense="CC BY-SA 3.0" />
  <row Id="20454" PostTypeId="1" CreationDate="2015-08-13T17:46:01.480" Score="2" ViewCount="61" Body="&lt;p&gt;I've tried looking for efficient parallel algorithms to solve banded systems. SPIKE is a good example but it however requires that the band size is large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any efficient library implementing a parallel algorithm for n-diagonal systems (where n is less than 10)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't found any implementations of Thomas' algorithm which seems promising.&lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2015-08-13T17:46:01.480" Title="Fast library for parallel n-diagonal matrix inversion" Tags="|libraries|banded-matrix|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20455" PostTypeId="1" AcceptedAnswerId="20458" CreationDate="2015-08-13T18:32:48.760" Score="5" ViewCount="247" Body="&lt;p&gt;I'm currently reading &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0021999175901151&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;, and I'm a bit confused about how they compute the accuracy of their algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the aforementioned paper, they investigate the regularized long-wave equation:&#xA;$$&#xA;u_t + u_x + u u_x - u_{xxt} = 0 \tag{1}&#xA;$$&#xA;They discretize the grid by spacing $h$ and time step $\tau$, giving the grid point $(ih,m \tau)$, where $i=0,1,2,\ldots$ and $m=0,1,2,\ldots$. For brevity, they use the notation: $u_i{}^m \equiv u(ih,m \tau)$. Furthermore, they introduce the well-known finite difference operators:&#xA;$$&#xA;\delta_x{}^2 w_i{}^m = \frac{w_{i+1}^m - 2w_i{}^m + w_{i-1}^m}{h^2}&#xA;$$&#xA;$$&#xA;H_x w_i{}^m = \frac{w_{i+1}^m - w_{i-1}^m}{2h}&#xA;$$&#xA;$$&#xA;\Delta_t w_i{}^m = \frac{w_i^{m+1} - w_i{}^m}{\tau}&#xA;$$&#xA;Then they discuss the following scheme to solve equation (1):&#xA;$$&#xA;\Delta_t (1- \delta_x{}^2) w_i{}^m + \frac{1}{2}H_x(w_i{}^{m+1} + w_i{}^{m}) + \frac{1}{2} w_i{}^m H_x(w_i{}^{m+1} + w_i{}^{m}) = 0 \tag{2}&#xA;$$&#xA;Up until here I understand everything, but then they go on and discuss the accuracy of this scheme:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To find the order of accuracy of the scheme, $w$ is replaced by $u$ in equation (2), where $u$ is a solution of equation (1) [...] Thus expanding all the terms about $(i,m+\frac{1}{2})$ shows that the first term approximates $u_t |_i^{m+(1/2)}$ and $u_{xxt}|_i^{m+(1/2)}$ with errors involving  $\tau^2$ and $h^2$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't really understand what the authors are doing here. I don't know what they mean with expanding the terms about $(i,m+\frac{1}{2})$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I learned this is that, for instance, the term $\Delta_t w_i{}^m$ is using a forward difference approximation, and so the truncation error will be $O(\Delta t)$. Can anyone explain what they are doing, or give me a link to some note or books that explain what they are doing?&lt;/p&gt;&#xA;" OwnerUserId="16650" LastEditorUserId="16650" LastEditDate="2015-08-15T22:49:09.110" LastActivityDate="2015-08-15T22:49:09.110" Title="Accuracy of finite differences" Tags="|finite-difference|accuracy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20456" PostTypeId="1" CreationDate="2015-08-13T20:03:43.400" Score="4" ViewCount="122" Body="&lt;p&gt;I am currently writing my dissertation on different methods for pricing barrier options. As part of this, I have implemented a finite differences method for solving one partial differential equation, and another finite differences method for solving another partial differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to create some plots or tables, to show which of these methods are preferable. My problem is, that I have no idea which plots to create. I do not really have much experience with this, so I don't know what will be valuable. Does anyone have any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose it is worth mentioning that I have available an analytical formula for the solution.&lt;/p&gt;&#xA;" OwnerUserId="3618" LastActivityDate="2015-08-13T20:12:01.330" Title="Comparing finite differences methods" Tags="|pde|finite-difference|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20457" PostTypeId="2" ParentId="20456" CreationDate="2015-08-13T20:12:01.330" Score="5" Body="&lt;p&gt;A log-log plot of solution accuracy versus amount of work performed is pretty common, since most methods' errors behave like a power of mesh size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, have a look at how books like &lt;em&gt;Solving Ordinary Differential Equations&lt;/em&gt; by Hairer, Norsett, Wanner, present differences between finite difference methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There can also sometimes be qualitative differences between methods, e.g., a monotone method might be guaranteed to produce nonnegative solutions to a diffusion equation. These can be pointed out directly.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-08-13T20:12:01.330" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20458" PostTypeId="2" ParentId="20455" CreationDate="2015-08-13T21:00:02.010" Score="6" Body="&lt;p&gt;I think you probably already know all this, and maybe it's just the wording that is confusing. I'm going to rephrase what they're doing to make it more explicit. It's exactly the same calculation of truncation errors that you are already familiar with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When approximating an operator $\mathcal{L}$ with an FD operator $L\approx \mathcal{L}$, the truncation error is the operator $E$ in $Lv = \mathcal{L}v + Ev$. If you're solving $\mathcal{L}u=0$, then the approximation $Lu=0$ corresponds to an exact solution of $\mathcal{L}u = -Eu$, with global error given by $\mathcal{L}^{-1}Eu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When they replace the $w$ in the FD formula with an exact solution $u$ of $\mathcal{L}u=0$, they are applying the FD operator $L$ to an exact solution $u$ to get $Lu = \mathcal{L}u + Eu = Eu$ ($\mathcal{L}u=0$). Thus they get the truncation error $Eu$. Technically this is slightly different from applying $L$ to an arbitrary function because the assumption $\mathcal{L}u=0$ may be used in calculating $Eu$, but this is irrelevant when solving a homogeneous equation $\mathcal{L}u=0$ with a zero r.h.s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When they say they expand about $(i,m+\frac12)$, they are trying to evaluate $Eu$ in closed form by expanding $Lu$ in Taylor series. Consider the simpler case:&#xA;$$ \Delta_t u(ih,m\tau) = u_t(ih,m\tau) + \tfrac12 h u_{tt}(ih,m\tau), $$&#xA;but &#xA;$$ \Delta_t u(ih,m\tau) = u_t(ih,(m+\tfrac12)\tau) + \tfrac1{12} h^2 u_{ttt}(ih, (m+\tfrac12)\tau). $$&#xA;Both of these truncation errors are correct, despite having different orders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When they expand about $(ih,(m+\tfrac12)\tau)$, they are computing the expansion&#xA;$$ Lu(ih,m\tau) = \mathcal{L}u(ih,(m+\tfrac12)\tau) + Eu(ih,(m+\tfrac12)\tau). $$&#xA;Since $u$ is an exact solution of $\mathcal{L}u=0$, $\mathcal{L}u$ vanishes at all points, including in between grid points, at $(i, m+\tfrac12)$. This only means that the truncation error $Eu$ in the approximate equation $Lu=0$, or $\mathcal{L}u=-Eu$, is evaluated slightly to the side of the grid points, which would not significantly affect the global error $\mathcal{L}^{-1}Eu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would happen had they expanded around $(i,m)$ instead, as you mentioned about $\Delta_t$? Then they would have gotten the expansion (compare with $\Delta_t$ above)&#xA;$$ Lu(ih,m\tau) = \mathcal{L}u(ih,m\tau) + E_1 u(ih,m\tau), $$&#xA;where $E_1 = O(h)$ is of a lower order than $E = O(h^2)$. Both truncation errors are correct. This is because $E_1$ is too pessimistic for this specific case of $u$ being a solution of $\mathcal{L}u=0$, and it is still technically correct as any function that is $O(h^2)$ is also $O(h)$. The difference between $E_1$ and $E$ is &#xA;$$\mathcal{L}u^{m+1/2} - \mathcal{L}u^{m},$$&#xA; which would indeed be $O(h)$ for an arbitrary function $u$, but which vanishes for a solution $\mathcal{L}u=0$. If you write out $E_1$ in full, and apply $\mathcal{L}u=0$, you should eventually get $O(h^2)$ too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what book to recommend, but LeVeque's Finite Difference Methods I think is quite good. What they are doing is quite conventional, so dicussed in basically any book on FD methods.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-08-14T00:35:38.493" LastActivityDate="2015-08-14T00:35:38.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20460" PostTypeId="1" CreationDate="2015-08-14T07:44:20.350" Score="2" ViewCount="368" Body="&lt;p&gt;I have the optimization problem as below: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;\text{maximize } \sum_{k=1}^{M} \alpha_k {R}_k\\&#xA;&amp;amp;\text{subject to: } \exp \left[ - (2^{{R}_k } -1)  \left(\frac{\tilde{Z} g_{k} p_{\max} + \sigma^2}{g_{pu} p_{pu}}  \right)  \right] \leq q&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\alpha_k$ is the weight factor associated to the $R_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a nonlinear optimization. But I am completely new to optimization. So any help is highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17246" LastEditorUserId="9667" LastEditDate="2015-08-14T19:00:29.870" LastActivityDate="2015-08-17T14:42:08.750" Title="How can I solve a nonlinear optimization problem where constraint contains exponential term?" Tags="|optimization|convex-optimization|nonlinear-programming|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20461" PostTypeId="2" ParentId="20444" CreationDate="2015-08-14T12:29:24.027" Score="4" Body="&lt;p&gt;You can easily do this with for loops. Just use &lt;code&gt;n&lt;/code&gt; to define your loop limits. Here's a fully functional solution for MATLAB (just define n and x first):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%pre-allocate A&#xA;A = zeros(n+1);&#xA;&#xA;%first row:&#xA;for j=0:n&#xA;    A(1,j+1)=sum(x.^j);&#xA;end&#xA;&#xA;%rows 2 through n&#xA;for i=1:n&#xA;    A(i+1,1:n)=A(i,2:n+1); %copy from previous row&#xA;    A(i+1,n+1)=sum(x.^(n+i)); %compute last entry in row&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This method requires &lt;code&gt;(2*n+1)*(2*numel(x))&lt;/code&gt; operations so if &lt;code&gt;x&lt;/code&gt; has about &lt;code&gt;n&lt;/code&gt; values the method is $\mathcal{O}(n^2)$.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastActivityDate="2015-08-14T12:29:24.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20462" PostTypeId="1" CreationDate="2015-08-14T19:53:31.053" Score="2" ViewCount="57" Body="&lt;p&gt;We have a group of observations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y = f(x_1, x_2, x_3) \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have also a forward model $y = f(x_1, x_2)$. The forward model does not include $x_3$ because $x_3$ might include dozens of parameters which we do not understand clearly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we inverted parameter $x_1$ with the forward model $y = f(x_1, 0)$ embedded in our inversion strategy. The best model can fit 80% of the observed data sets. Then we inverted both parameters $x_1$ and $x_2$ with the forward model $y = f(x_1, x_2)$. However, the misfit did not reduce and the inverted $x_2$ is always quite close to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on previous laboratory and numerical experiments, $x_2$ should have significant effects on the observation and its value is obvious larger than zero. So we fix $x_2$ as the experimental observed value and inverted for the optimized model. The misfit from the best inverted model is larger than that with $x_2$ fixed at zero. How can we understand these inversion results? Do they mean that $x_2$ is balanced by $x_3$ or that there was overfitting in inversions with $x_1$ only? &lt;/p&gt;&#xA;" OwnerUserId="17259" LastEditorUserId="9667" LastEditDate="2015-08-15T02:43:13.730" LastActivityDate="2015-08-15T02:43:13.730" Title="Inverted value is not consistent with expectation" Tags="|optimization|constrained-optimization|inverse-problem|curve-fitting|inverse|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20463" PostTypeId="1" AcceptedAnswerId="20465" CreationDate="2015-08-15T04:23:20.997" Score="3" ViewCount="132" Body="&lt;p&gt;I need to get just a random vertex of a polytope. Any will do. The only way I can do this now is to pick a random function (say 0s) to maximize with scipy.optimize.linprog. However, this is wasteful, because I really don't care which vertex I get. I saw something in the docs about getting the internal state of the solver with a callback function, but I don't know enough about the simplex algorithm to understand which paramater I need. How can I just extract the vertex scipy found during Phase 1 and then abort the calculation?&lt;/p&gt;&#xA;" OwnerUserId="17262" LastActivityDate="2015-08-15T17:45:53.003" Title="Use scipy to get any vertex of polytope" Tags="|linear-programming|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20464" PostTypeId="2" ParentId="20427" CreationDate="2015-08-15T04:30:17.860" Score="4" Body="&lt;p&gt;Here is an idea that might help.  It seems that, roughly speaking, the row of a Kronecker product is a Kronecker product of rows.  (I hoped to find a nice reference for this online somewhere but I could not; so here is a little sketch.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, for ${A_{R \times S}} \otimes {B_{M \times L}} = {C_{RM \times SL}}$, the formula for an element of $C$ is&#xA;$$ {c_{m + rM,l + sL}} = {a_{r,s}}{b_{m,l}} $$&#xA;(Reference for this definition is Tolimieri et al., &lt;em&gt;Algorithms for Discrete Fourier Transform and Convolution&lt;/em&gt;, 2ed, chapter 2.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider&#xA;${P_{1 \times S}} \otimes {Q_{1 \times L}} = {Z_{1 \times SL}}$.  Since $Z$ only has one row, it follows that $m=0$ and $r=0$.&#xA;$$ {z_{0,l + sL}} = {a_{0,s}}{b_{0,l}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $i=m+rM$.  If $P$ is the $r$th row of $A$, i.e. ${p_{0,s}} = {a_{r,s}}$, and similarly $Q$ is the $m$th row of $B$, i.e. ${q_{0,l}} = {b_{m,l}}$, then&#xA;$$ {z_{0,l + sL}} = {a_{r,s}}{b_{m,l}} $$&#xA;which is the $i$th row of $C$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding algorithm would be something like this.  To compute the $i$th row of $C$,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r = floor(i/M);&#xA;m = i - r*M;&#xA;kron(A(r,:), B(m,:))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually this will not quite work in Matlab, because the math above uses arrays that index from 0, whereas &lt;a href=&quot;https://stackoverflow.com/questions/4239907/is-zero-based-indexing-available-in-matlab&quot;&gt;Matlab uses 1-based arrays&lt;/a&gt;; unfortunately I don't have Matlab handy to test those implementation details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even so, stacking up a few rows computed along these lines might be more efficient for your application than selecting a subset from the full Kronecker product.&lt;/p&gt;&#xA;" OwnerUserId="17263" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-08-15T04:30:17.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20465" PostTypeId="2" ParentId="20463" CreationDate="2015-08-15T17:45:53.003" Score="4" Body="&lt;p&gt;If you use a 0 objective function, then the solver should stop as soon as it encounters a feasible solution (because that solution will be optimal.)  If the solver is using a 2-phase primal simplex method it will stop immediately after the first phase and the solution will be a vertex of the polyhedron.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, LP feasibility problems like this really aren't much easier than optimization problems with a nonzero objective.  You might be able to adjust the parameters of the solver so that it finds a primal feasible solution more quickly, but that would depend on the details of the solver that you're using.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-08-15T17:45:53.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20466" PostTypeId="1" CreationDate="2015-08-15T19:58:17.640" Score="1" ViewCount="343" Body="&lt;p&gt;I have attempted to code up the simple math required to plot the total field set up by an incident plane wave on a semi-infinite flat plate which can be found &lt;a href=&quot;http://pcwww.liv.ac.uk/~itho17/personal/pdfs/wh_talk.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarise:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\phi_s(r,\theta ) = e^{ikr}/2 \left( w\left[ e^{i\pi/4}\sqrt{2kr}\sin(\frac{\theta + \Theta}{2}) \right]+ w\left[ e^{i\pi/4}\sqrt{2kr}\sin(\frac{\theta - \Theta}{2}) \right]\right) \enspace,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s the scattered potential in polar coordinates according to the article.  $\Theta$ denotes the angle of the incoming plane wave and $k$ is the wave number. Finally&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w(z) = e^{-z^2}(1-\mathrm{erf}(z)) \enspace$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you run the code you will see that I do not get the expected wave field as in the pictures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is as follows:&lt;/strong&gt; Have I made a mistake in my implementation or is the formula given incorrect?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My calculation:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ohwKI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ohwKI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their Calculation:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mMLWe.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mMLWe.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.special import erf&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;&quot;&quot;&quot;Plane wave scattering by a plane wave incident on a seminfinite plane&#xA;&quot;&quot;&quot;&#xA;&#xA;global T&#xA;global k&#xA;T = np.pi/4.0 # Incident plane wave angle&#xA;k = 10        # Wave number&#xA;&#xA;&#xA;def phi_scattered(r,t):&#xA;&#xA;    W = lambda z: np.exp(-z**2)*(1-erf(z))&#xA;    term1 = W(np.exp(1j*np.pi/4.0)*np.sqrt(2*k*r)*np.sin((t+T)/2.0))&#xA;    term2 = W(np.exp(1j*np.pi/4.0)*np.sqrt(2*k*r)*np.sin((t-T)/2.0))&#xA;    res =  np.exp(1j*k*r)/2.0*(term1 + term2)&#xA;&#xA;    return res&#xA;&#xA;def phi_incident(r,t):&#xA;    return np.exp(1j*k*r*np.cos(t-T))&#xA;&#xA;&#xA;&#xA;x1 = np.linspace(-50,50,100)&#xA;x2 = np.linspace(-50,50,100)&#xA;X1, X2 = np.meshgrid(x1,x2)&#xA;&#xA;&#xA;R = np.sqrt(X1**2+X2**2)&#xA;THETA = np.arctan2(X2,X1)&#xA;&#xA;phi = phi_scattered(R,THETA) + phi_incident(R,THETA)&#xA;plt.figure()&#xA;plt.imshow(np.real(phi), vmin = np.min(np.real(phi)), vmax = np.max(np.real(phi)))&#xA;plt.colorbar()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="9667" LastEditDate="2015-08-18T13:20:36.430" LastActivityDate="2015-09-04T18:45:16.003" Title="Computation of plane wave scattering on semi infinite plane" Tags="|pde|python|numerical-analysis|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20467" PostTypeId="2" ParentId="20343" CreationDate="2015-08-15T23:04:20.627" Score="1" Body="&lt;p&gt;If I understand correctly, the form of your problem seems similar to the one faced by a radar or sonar system.  In that case, the sensor periodically outputs a number of &quot;hits&quot; or detections, each consisting of a time and a location in space.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine learning brings powerful general methods to bear on problems of this form (as observed in the answer by Kirill) but in addition there is a substantial literature tailored to this specific problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The task in &lt;em&gt;tracking&lt;/em&gt; is to decide which hits are associated with a single object.  E.g., when many aircraft are up, then deciding which of a radar sweep's detections are associated with the previous sweep's detections is not so obvious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tracking algorithms typically work by having a model that predicts roughly where a next detection is expected, given previous detections, and using this prediction as a guide to associating a new detection with previous ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia of course is often good for a quick survey of a technique to see if it is worth exploring more.  In this case there are several similarly-named articles of very different content.  I suggest that &lt;a href=&quot;https://en.wikipedia.org/wiki/Tracking_system&quot; rel=&quot;nofollow&quot;&gt;Tracking system&lt;/a&gt; is quite off the mark and &lt;a href=&quot;https://en.wikipedia.org/wiki/Track_algorithm&quot; rel=&quot;nofollow&quot;&gt;Track algorithm&lt;/a&gt; is broad but thin; &lt;a href=&quot;https://en.wikipedia.org/wiki/Radar_tracker&quot; rel=&quot;nofollow&quot;&gt;Radar tracker&lt;/a&gt; on the other hand has a solid list of well-known algorithms, starting from the simplest (alpha-beta) through the Kalman filter and its variants, to the particle filter.&lt;/p&gt;&#xA;" OwnerUserId="17263" LastActivityDate="2015-08-15T23:04:20.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20468" PostTypeId="1" CreationDate="2015-08-16T00:20:55.767" Score="18" ViewCount="2459" Body="&lt;p&gt;I am parallelizing code to numerically solve a 5 Dimensional population balance model. Currently I have a very good MPICH2 parallelized code in FORTRAN but as we increase parameter values the arrays become too large to run in distributed memory mode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have access to a cluster with 15 nodes, where each node has two 8 core CPUs and 128GB of RAM. I want to write run a program with MPI-3.0 in shared memory mode so that each process does not generate its own copy of each array. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before I can run anything on the cluster I have to test it on a desktop running Ubuntu. It is essentially one blade of the cluster in that it has two 8 core CPUs and 128 GBs of RAM. I will be writing and testing my code on it, so please gear your responses towards running programs on the Ubuntu computer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read that there is a way to run MPI-3.0 in shared memory mode like OpenMP instead of its default distributed memory mode. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How will I have to alter my code? Do I need to add in calls to other MPI functions like &lt;code&gt;MPI_WIN_ALLOCATE&lt;/code&gt;? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I compile my code to run MPI-3.0 in shared memory mode? Will this be different if it is over several nodes? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Please give sample compilation scripts if you can. I also have only GNU compilers. The cluster I use does not support Intel compilers. &lt;/p&gt;&#xA;" OwnerUserId="17265" LastEditorUserId="17674" LastEditDate="2019-04-05T19:40:13.757" LastActivityDate="2021-08-23T01:05:10.563" Title="How to Run MPI-3.0 in shared memory mode like OpenMP" Tags="|parallel-computing|fortran|numerics|mpi|compiling|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="20469" PostTypeId="1" AcceptedAnswerId="20472" CreationDate="2015-08-16T01:12:31.450" Score="2" ViewCount="544" Body="&lt;p&gt;I am solving a boundary value problem in 2D via an implicit finite difference scheme. Unfortunately, although the problem is well-posed and should have a unique solution, the condition number of the matrix to solve is through the roof (~10^15 for a highly simplified version that &lt;em&gt;is&lt;/em&gt; just the 2D Poisson eqn.; otherwise my full model has a condition number of 10^22). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, the domain is $6.1e7 \le r \le 6.3e7$ and $\epsilon \le \theta \le \pi-\epsilon$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found that if I reduce the radial domain to $8 \le r \le 20$, the condition number drops to ~10,000. This makes me think I need to scale my problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how to do this, however, and I need to do it &lt;em&gt;right&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Would I have to apply the scale factors in both dimensions, or to physical parameters as well?&lt;/strong&gt; That is, if I scaled my radii by a factor $L$ (&lt;em&gt;e.g.&lt;/em&gt; $R = r/L$), would I also have to scale other physical quantities (&lt;em&gt;e.g.&lt;/em&gt; current density $J = j L^3$, or acceleration $A = \frac{\Delta v / L}{\Delta t}$)? &lt;/p&gt;&#xA;" OwnerUserId="9451" LastActivityDate="2015-08-16T20:32:39.743" Title="I need to scale variables to solve a 2D PDE. What are the physical considerations of scaling?" Tags="|pde|finite-difference|computational-physics|electromagnetism|scaling|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20471" PostTypeId="2" ParentId="20460" CreationDate="2015-08-16T17:09:58.947" Score="4" Body="&lt;p&gt;If the $R_k$ are your only optimization variables, then the constraint&#xA;$$&#xA;\exp \left[ - (2^{{R}_k } -1)  \left(\frac{\tilde{Z} g_{k} p_{\max} + \sigma^2}{g_{pu} p_{pu}}  \right)  \right] \leq q&#xA;$$&#xA;is equivalent to&#xA;$$&#xA;- (2^{{R}_k } -1)  \left(\frac{\tilde{Z} g_{k} p_{\max} + \sigma^2}{g_{pu} p_{pu}}  \right)  \leq \ln q&#xA;$$&#xA;which is equivalent to&#xA;$$&#xA;(2^{{R}_k } -1)  &#xA; \leq &#xA; -(\ln q)&#xA;\left(\frac{g_{pu} p_{pu}}{\tilde{Z} g_{k} p_{\max} + \sigma^2}  \right)&#xA;$$&#xA;which is equivalent to&#xA;$$&#xA; 2^{{R}_k }&#xA; \leq &#xA; -(\ln q)&#xA;\left(\frac{g_{pu} p_{pu}}{\tilde{Z} g_{k} p_{\max} + \sigma^2}  \right)+1&#xA;$$&#xA;which is equivalent to&#xA;$$&#xA; {R}_k&#xA; \leq &#xA; \log_2&#xA; \left\{&#xA; -(\ln q)&#xA;\left(\frac{g_{pu} p_{pu}}{\tilde{Z} g_{k} p_{\max} + \sigma^2}  \right)+1\right\}.&#xA;$$&#xA;This is a linear constraint, and your objective function is already linear. So you ended with a standard linear program, for which there is plenty of software (and if you don't want to use any of that, plenty of simple to implement methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transformation above shows an important principle: one can often transform complicated-looking problems into much simpler problems if one just looks at them closely enough. &lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2015-08-17T14:42:08.750" LastActivityDate="2015-08-17T14:42:08.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20472" PostTypeId="2" ParentId="20469" CreationDate="2015-08-16T20:32:39.743" Score="5" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I've found that if I reduce the radial domain to &lt;span class=&quot;math-container&quot;&gt;$8 \leq r \leq 20$&lt;/span&gt;, the condition number drops to ~10,000. This makes me think I need to scale my problem.&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure how to do this, however, and I need to do it right.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Nondimensionalization is partly repeated application of the chain rule, and partly art. The goal is to make as many quantities in your equations as close to 1 as possible. In most cases, it involves scaling both independent variables and dependent variables by &amp;quot;physically relevant&amp;quot; scale factors. Sometimes, these scale factors are obvious (e.g., there is only one length scale that matters, and that length scale is the length/half-length/etc. of the domain), sometimes, they are not (e.g., I have several reference voltages that matter, and I need to pick one).&lt;/p&gt;&#xA;&lt;p&gt;For someone inexperienced, I'd say, focus on the mechanics of nondimensionalization. Pick reference parameters that you think have some meaning (trust your intuition here, or check the literature if you think it's useful), and then nondimensionalize your equations, solve them, and see what happens. See what physical insights you get out of the equations, and think about limiting cases.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Nondimensionalization&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia&lt;/a&gt; is a good reference here. I also like the discussion in Deen's &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0199740283&quot; rel=&quot;noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Analysis of Transport Phenomena&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Would I have to apply the scale factors in both dimensions, or to physical parameters as well?&lt;/strong&gt; That is, if I scaled my radii by a factor &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; (&lt;em&gt;e.g.&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$R = r/L$&lt;/span&gt;), would I also have to scale other physical quantities (&lt;em&gt;e.g.&lt;/em&gt; current density &lt;span class=&quot;math-container&quot;&gt;$J = j L^3$&lt;/span&gt;, or acceleration &lt;span class=&quot;math-container&quot;&gt;$A = \frac{\Delta v / L}{\Delta t}$&lt;/span&gt;)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You don't &lt;em&gt;have&lt;/em&gt; to. It's valid to nondimensionalize only some quantities, particularly if certain variables are difficult to nondimensionalize, or are already nondimensional. Frequently, this situation occurs in combustion applications, where species mass fractions are already nondimensional (though usually vary over several orders of magnitude), and are difficult to scale in such a way that they vary at similar rates (to within a couple orders of magnitude). The best practice is to nondimensionalize your equations as much as you possibly can, since these sorts of scalings act as a natural preconditioner.&lt;/p&gt;&#xA;&lt;p&gt;The only counterargument that I can think of to nondimensionalization is that it does require extra work and thought, and some care to make sure that you implement the scaling factors correctly in your code. Any ill-conditioning due to poor choices of units can sometimes be overcome with judicious use of preconditioners, but usually, nondimensionalization is preferable because of the insight it provides (via the Buckingham pi theorem, nondimensionalization yields the smallest group of parameters that influence your equations).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-08-16T20:32:39.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20474" PostTypeId="1" CreationDate="2015-08-17T07:37:41.993" Score="2" ViewCount="2402" Body="&lt;p&gt;I'm looking for a computationally efficient algorithm for solving the following type of assignment problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two sets of points. Set A has N points and set B has M points. I'd like to establish one-to-many assignments from A to B, where each element in A can have at most two and at least zero matches in B. Obviously, in this bipartite graph the edges have non-zero costs. Also, the matchings are unique - two elements from A cannot be assigned to the same element in B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first thought of using the Hungarian algorithm, but it always finds one-to-one matches, which renders it not directly applicable in my case. The sought algorithm should be able to account 1-to-0 and 1-to-2 assignments as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know any such algorithm from the literature?&lt;/p&gt;&#xA;" OwnerUserId="6580" LastEditorUserId="6580" LastEditDate="2015-08-17T08:20:59.340" LastActivityDate="2016-08-24T22:52:35.883" Title="Algorithms for one-to-many assignment problem" Tags="|optimization|algorithms|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20475" PostTypeId="2" ParentId="20474" CreationDate="2015-08-17T08:06:26.760" Score="1" Body="&lt;p&gt;I would initiate my problem with a Hungarian algorithm. This gives you the minimum cost assignment. Then, I would follow a greedy procedure: Pick a point in A and assign it to the point in B, having the closest distance to the first assignment (obtained via Hungarian). If this point already contains 2 matches, skip to the next one in B (If your assignments are unique, this will never happen). Then, take second point in A and iterate. You could do this efficiently by maintaining a dynamic LUT over the set B.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is of course greedy and by no means optimal. However, my experience is that such algorithms give reasonable results in practice.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2015-08-17T08:06:26.760" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20476" PostTypeId="1" AcceptedAnswerId="20506" CreationDate="2015-08-17T11:18:59.177" Score="4" ViewCount="336" Body="&lt;p&gt;I'm trying to solve a generalized eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax = \lambda Bx, \quad A = A^\top &amp;gt; 0,\; B = B^\top &amp;gt; 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\lambda \approx \sigma$ using &lt;a href=&quot;https://en.wikipedia.org/wiki/Rayleigh_quotient_iteration&quot; rel=&quot;nofollow&quot;&gt;Rayleigh Quotient Iteration (RQI)&lt;/a&gt; (RQI is applied to $B^{-1/2}AB^{-1/2}$, but the $B^{-1/2}$ cancels from the final formulas):&#xA;$$&#xA;(A - \mu_k B) \tilde x_{k+1} = B x_k\\&#xA;x_{k+1} = \frac{\tilde x_{k+1}}{||\tilde x_{k+1}||}\\&#xA;\mu_k = \frac{x_m^\top A x_m}{x_m^\top B x_m}\\&#xA;\mu_0 = \sigma, \quad x_0 = \text{random vector}&#xA;$$&#xA;I've decided to use some iterative method to solve&#xA;$$&#xA;(A - \mu_k B) \tilde x_{k+1} = B x_k&#xA;$$&#xA;for $\tilde x_{k+1}$, since I do not want to store $A$ and $B$ explicitly. I only wish to code computing $Ax$ and $Bx$ products.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix is symmetric, but not positive definite, otherwise I would have used conjugate gradients method. Which method would you suggest? Are there any pitfalls when using Krylov subspace methods with RQI?&lt;/p&gt;&#xA;" OwnerUserId="17125" LastEditorUserId="9667" LastEditDate="2015-08-17T21:48:46.217" LastActivityDate="2015-08-20T13:49:30.743" Title="Appropriate iterative linear solver for an eigenvalue problem" Tags="|linear-algebra|linear-solver|eigensystem|krylov-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20478" PostTypeId="1" AcceptedAnswerId="20803" CreationDate="2015-08-17T21:23:14.510" Score="2" ViewCount="477" Body="&lt;p&gt;Suppose I'm optimizing for an unknown $x\in\mathbb{R}^k.$  I have a linear operator $A(\cdot)$ that maps $x$ to an $n\times n$ symmetric matrix, i.e.,  $A:\mathbb{R}^k\rightarrow\mathbb{R}^{n\times n}.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to solve a problem of the form&#xA;$$&#xA;\begin{array}{rl}&#xA;\min_{x\in\mathbb{R}^k} &amp;amp; f(x)\\&#xA;\textrm{s.t.} &amp;amp; x\in\mathcal C\\&#xA;&amp;amp; \mathrm{exp}(A(x))\cdot v=w,&#xA;\end{array}&#xA;$$&#xA;where $f:\mathbb{R}^k\rightarrow\mathbb{R}$ is convex, $\mathcal C\subseteq\mathbb{R}^k$ is some convex set and $v,w\in\mathbb{R}^n$ are constant vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any chance this problem can be transformed into something convex?  If not (or if so), what would be a good optimization technique/algorithm for problems of this form?&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2015-09-23T06:20:57.200" Title="Optimization with matrix exponential constraint" Tags="|linear-algebra|optimization|convex-optimization|constrained-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20479" PostTypeId="1" CreationDate="2015-08-18T00:59:09.443" Score="1" ViewCount="37" Body="&lt;p&gt;I'm running a potential flow solver, and I have values of the velocity potential $\phi$ on the boundaries. I'd like to compute the spatial evolution kinetic energy in the system. In particular, I'd like to find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T(x) = \int \int_{-\infty}^{\eta(x,t)} \frac{1}{2}(\phi_x^2+\phi_z^2) \ dz \ dt.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I calculate this integral using just boundary values of $\phi$ and its derivatives? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, the more commonly used measure is the temporal evolution of the kinetic energy, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ke(t) = \int  \int_{-\infty}^{\eta(x,t)}\frac{1}{2}(\phi_x^2+\phi_z^2) \ dz \ dx = \int_{\partial A}\phi(\phi_z \ dx - \phi_x \ dz), $$&#xA;with $\partial A$ the boundary of the domain, and we have employed a Green's identity to for $Ke(t)$ based just on values on the boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A solution is to solve for $\phi$ in the interior and then simply compute the 2d integral for $T$, but the boundary is evolving in space and time, so that I'd prefer not to solve Laplace's equation at each time step. &lt;/p&gt;&#xA;" OwnerUserId="8021" LastEditorUserId="8021" LastEditDate="2015-08-18T07:01:48.317" LastActivityDate="2015-08-18T07:10:52.313" Title="Spatial evolution of kinetic energy in free surface flow in terms of values on boundary" Tags="|pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20480" PostTypeId="2" ParentId="20449" CreationDate="2015-08-18T03:49:36.427" Score="0" Body="&lt;p&gt;I am writing some ideas that came to my mind, so take this answer with a grain of salt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we need to find a way if $P=Q$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing that comes to my mind is to compute the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hausdorff_distance&quot; rel=&quot;nofollow&quot;&gt;Hausdorff distance&lt;/a&gt; between $P$ and $Q$. Since each one is the convex hull of the union of a finite sets of polytopes, they are polytopes as well. And it would be enough to compute the Hausdorff distance between the two sets of vertices ($v_P$, $v_Q$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;H(v_P, v_Q) = \max\lbrace h(v_P, v_Q), h(v_Q, v_P)\rbrace\\&#xA;&amp;amp;h(v_P, v_Q)=\max_{a\in v_P} \min_{b\in v_Q}\; d(v_P, v_Q)&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the number of vertices is not the same($n(v_P)\neq n(v_Q)$), then the polytopes are different. We are assuming that they are the same then.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the complexity of computing the Hausdorff distance would be $O(n_P n_Q)$, and they would be equal if $H(v_P, v_Q) =0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To estimate $n_P$ and $n_Q$ we can consider the worst case scenario, i.e., the convex hull preserve all the vertices from the set of polytopes. Then, if the maximum number of vertices that a polytope has is $M_P$ for $P$ and $M_Q$ for $Q$, we have as estimates&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$n_P = kM_P\quad \text{and}\quad n_Q = r M_Q \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What translates in $O(n_P n_Q) = O(kr M_P M_Q)$. If we take into account the computation of the distance function (&quot;norm&quot;). Then we end up with something like $O(krn M_P M_Q)$.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-08-18T03:49:36.427" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20481" PostTypeId="2" ParentId="20479" CreationDate="2015-08-18T07:10:52.313" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can I calculate this integral using just boundary values of $\phi$ and its derivatives?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I hate to be the bearer of bad news, but probably not, no. I detail reasons why below:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It looks like you are invoking the Divergence Theorem when you convert the integral for kinetic energy from a volume integral to a surface integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't use similar logic here to convert the &quot;volume&quot; integral for the spatial evolution kinetic energy to a surface integral, because the form of your integral in two-dimensions doesn't satisfy the hypotheses of the Divergence Theorem. Also, attempting to apply the Divergence Theorem in a single-dimension (that is, the 2-D version only along the $z$ direction) doesn't really achieve what you want because it reduces to the Fundamental Theorem of Calculus, which won't be helpful. (If you already had the antiderivative of your argument, you wouldn't be asking this question.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can also appeal to basic intuition here: the Divergence Theorem is saying that the total change of a quantity in a volume is equal to changes induced by fluxes through the boundary surface. Here, you're not integrating over the whole surface, so some flux information is going to be missing.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-08-18T07:10:52.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20483" PostTypeId="1" CreationDate="2015-08-18T12:12:25.533" Score="3" ViewCount="162" Body="&lt;p&gt;Matrix reorderings are important for many direct solvers. Sometimes the objective is to reduce the bandwith or the generated fill in by LU Decomposition. I am interested in a reordering which reduces the fill in of $A + A^{−1}$. Is there some work, or is it hopeless?&lt;/p&gt;&#xA;" OwnerUserId="17283" LastEditorUserId="9667" LastEditDate="2015-08-18T13:22:01.457" LastActivityDate="2015-08-20T13:08:30.930" Title="Sparse Matrix Reordering" Tags="|matrix|sparse-matrix|graph-theory|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20484" PostTypeId="1" CreationDate="2015-08-18T16:00:27.517" Score="4" ViewCount="164" Body="&lt;p&gt;Consider the following equation on the circle:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dfrac{\partial p(x,t)}{\partial t} = a(x)\dfrac{\partial p(x,t)}{\partial x} \equiv L(p) \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L$ is the operator acting on $p(x,t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I would like to create a matrix of this operator $L$ using FFT/IFFT in Matlab. Note that since our domain is the circle, $x\in[0,2\pi]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking Fourier transform, we get &#xA;$\hat L(p)=  \left[a(x)\dfrac{\partial p(x,t)}{\partial x}\right]^{\wedge}= \left[ a(x)  (D_x \hat{p})^{V}\right]^{\wedge}$, where $\wedge$ and $V$ are FFT and IFFT operations, and $D_x$ is the $x$-derivative matrix in Fourier (formed by multiplying the $k$th fourier coefficient by $ik$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is:&lt;/strong&gt; From the expression I was able to solve the PDE using an ODE solver such as ODE45. But, I am interested in extracting the operator for $L$ to be used for other purposes. Hence, I am trying to extract matrix $\hat{L}$ in the following equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dfrac{\partial \hat{p}(x,t)}{\partial t}=\hat{L}\hat{p}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas ?&lt;/p&gt;&#xA;" OwnerUserId="17286" LastEditorUserId="9667" LastEditDate="2015-08-19T19:56:39.697" LastActivityDate="2015-08-19T22:25:51.247" Title="Computing Fourier representation of space dependent advection operator via FFT" Tags="|pde|fourier-analysis|elliptic-pde|fftw|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20486" PostTypeId="1" CreationDate="2015-08-18T18:57:56.000" Score="7" ViewCount="729" Body="&lt;p&gt;In all descriptions of the Lanczos vector, it's said that the starting vector is random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But let's say I'm only interested in the eigenvector associated with the lowest eigenvalue (as is the case when using Lanczos to find the ground-state of a quantum mechanical system).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wouldn't then &lt;em&gt;any&lt;/em&gt; start vector that has some non-zero overlap with the ground-state do?&lt;/p&gt;&#xA;" OwnerUserId="105" LastActivityDate="2015-08-19T19:32:20.387" Title="Does the Lanczos starting vector have to be random?" Tags="|linear-algebra|eigenvalues|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20487" PostTypeId="2" ParentId="20486" CreationDate="2015-08-18T19:18:32.837" Score="3" Body="&lt;p&gt;If an algorithm starts with a random vector and is known to converge to a correct result, that &lt;em&gt;means&lt;/em&gt; that it can work with an arbitrary vector. How would it succeed starting from an arbitrary random vector otherwise?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only exception might be that there is a set of vectors for which the algorithm fails, but which are picked with essentially zero probability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't specific to Lanczos algorithm either: this would apply to any algorithm that doesn't require a stream of random vectors and doesn't rely on good statistical properties of the RNG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mostly this is done for convenience, because for a fixed vector $u$ the probability that $u^\top v = 0$ is zero for most sensible probability distributions over $v$, thus the picked starting random vector $v$ can be assumed to not be special in any way.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-08-18T19:18:32.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20488" PostTypeId="2" ParentId="20486" CreationDate="2015-08-18T19:20:40.803" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;But let's say I'm only interested in the eigenvector associated with the lowest eigenvalue (as is the case when using Lanczos to find the ground-state of a quantum mechanical system).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You'd want to use a &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/UG/node33.html&quot; rel=&quot;nofollow&quot;&gt;shift-and-invert spectral transformation&lt;/a&gt; with an estimate of the eigenvalue you care about. Lanczos normally finds the largest magnitude eigenvalue and its eigenvector (after some post-processing); inversion transforms the problem to solving the smallest magnitude eigenvalue, and shifting transforms the problem to solving for the eigenvalue closest to an eigenvalue of interest (the shift).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Wouldn't then any start vector that has some non-zero overlap with the ground-state do?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If I remember correctly, if you give Lanczos the exact eigenvector corresponding to a given eigenvalue, it will only generate the Krylov subspace corresponding to that eigenvector-eigenvalue pair, assuming all calculations are performed in exact arithmetic. In double precision arithmetic, with some partial reorthogonalization, it will eventually generate a basis, given enough iterations (with probability 1, since on a set of measure zero, there are pathological inputs, as Kirill points out). So, essentially any starting vector will work.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-08-18T19:20:40.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20489" PostTypeId="1" AcceptedAnswerId="20495" CreationDate="2015-08-18T21:19:42.867" Score="8" ViewCount="1965" Body="&lt;p&gt;&lt;strong&gt;TLDR&lt;/strong&gt;: Are these polynomials really harmonic polynomials, and how can I generate them?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Long version:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to describe an electrostatic potential $\Phi(x,y,z)$ over a source-free volume, by using a polynomial of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\Phi(x,y,z) &amp;amp;= a_{000} + a_{100} x + a_{010} y + \dots + a_{123}xy^2z^3 + \dots \\&#xA;&amp;amp;= \sum_{l,m,n \geq 0}^\infty a_{lmn}x^ly^mz^n \quad .&#xA;\end{align} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The potential is known to me on a grid: I simulated it using a finite element simulation, and I want to find the coefficients $a_{lmn}$ so that I can give a smooth estimate of the potential around the origin. I verified that my potential $\Phi(x,y,z)$ is &quot;benign&quot; in the sense that I can neglect all terms higher than some order $N$. I can somewhat successfully find the coefficients $a_{lmn}$ with a crude fit, but of course the fit-functions $x^ly^mz^n$are not orthogonal in any way, which makes &quot;finding the right order $N$&quot; more important than in should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But since the volume that I am interested in is charge free, I know that the total polynomial must satisfy the Laplace equation $\Delta\Phi(x,y,z) = 0$. I want to use this fact to rewrite the polynomial as a sum of smaller polynomials that, on their own, satisfy the Laplace equation and are orthogonal to each other. I believe such polynomials are called &lt;a href=&quot;https://en.wikipedia.org/wiki/Harmonic_polynomial&quot; rel=&quot;noreferrer&quot;&gt;harmonic polynomials&lt;/a&gt;, and the whole process reminds me of doing multipole expansions, only that instead of sitting far away from the charges that create the potential, I sit between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using horror-inducing Mathematica expressions that transmogrify spherical harmonics to real-valued, cartesian harmonics, I found a set of polynomials $p_{n,m}(x,y,z)$ of degree $n$, with $-n \leq m \leq n$, which as far as I have tested, &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;satisfy the Laplace equation,&lt;/li&gt;&#xA;&lt;li&gt;are orthogonal over the cube $x,y,z \in [-1;1]$&lt;/li&gt;&#xA;&lt;li&gt;are normalizied over the cube $x,y,z \in [-1;1]$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first of these polynomials are (dropping all normalization factors to make them shorter)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;p_{0,0}(x,y,z) &amp;amp;= 1\\ \\&#xA;p_{1,-1}(x,y,z) &amp;amp;= x\\&#xA;p_{1,0}(x,y,z) &amp;amp;= y\\&#xA;p_{1,1}(x,y,z) &amp;amp;= z\\ \\&#xA;p_{2,-2}(x,y,z) &amp;amp;= xy\\ &#xA;p_{2,-1}(x,y,z) &amp;amp;= yz\\&#xA;p_{2,0}(x,y,z) &amp;amp;= x^2+y^2-2 z^2\\&#xA;p_{2,1}(x,y,z) &amp;amp;= xz\\&#xA;p_{2,2}(x,y,z) &amp;amp;= x^2-y^2\\ \\&#xA;p_{3,-3}(x,y,z) &amp;amp;= -3x^2y+y^3\\&#xA;p_{3,-2}(x,y,z) &amp;amp;= xyz\\&#xA;p_{3,-1}(x,y,z) &amp;amp;= y(x^2+y^2-4z^2)\\&#xA;p_{3,0}(x,y,z) &amp;amp;= z(-3(x^2+y^2)+2z^2)\\&#xA;p_{3,1}(x,y,z) &amp;amp;= x(x^2+y^2-4z^2)\\&#xA;p_{3,2}(x,y,z) &amp;amp;= (x^2-y^2)z\\&#xA;p_{3,3}(x,y,z) &amp;amp;= x^3-3xy^2&#xA;\end{align} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I vaguely remember encountering similar polynomials when doing multipole expansions of ion optic elements, such as &lt;a href=&quot;http://cas.web.cern.ch/cas/Belgium-2009/Lectures/PDFs/Wolski-1.pdf&quot; rel=&quot;noreferrer&quot;&gt;magnetic fields used in particle accelerators&lt;/a&gt;. But the generating terms used in ion optics are typically two-dimensional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried finding how to generate such polynomials in the literature, but found nothing that did not use pretty heavy math. I am not entirely sure that &quot;harmonic polynomials&quot; is really the correct term form them. In any case, how can these polynomials be efficiently generated in a way that ensures that they are orthogonal?&lt;/p&gt;&#xA;" OwnerUserId="4633" LastEditorUserId="4633" LastEditDate="2015-08-18T22:43:19.050" LastActivityDate="2015-08-20T11:51:51.497" Title="Generating harmonic polynomials in cartesian coordinates" Tags="|interpolation|spherical-harmonics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20490" PostTypeId="1" CreationDate="2015-08-18T22:35:39.807" Score="1" ViewCount="53" Body="&lt;p&gt;Let $D_{\nu}(.)$ is the parabolic cylinder function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;http://mathworld.wolfram.com/ParabolicCylinderFunction.html&quot; rel=&quot;nofollow&quot;&gt;http://mathworld.wolfram.com/ParabolicCylinderFunction.html&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And $\Gamma(.)$ is the Gamma function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s_y(\mu,\nu,t,z)=2^{\nu}\displaystyle\sum_{k=0}^{\infty}\frac{\Gamma(\nu+k)e^{-(\nu t+z+2kt)^2/(4y)}}{\sqrt{2\pi}y^{1+\mu/2}\Gamma(\nu) k!} D_{\nu+1}(\frac{\nu t+z+2kt}{\sqrt{y}})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$es_y(\mu,\nu,t,x,z)=\displaystyle\sum_{k=0}^{\infty}\frac{(-z)^k}{k!}&#xA;s_y(\mu+k,\nu+k,t,x+z+kt) $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$is_y(\nu,t,r,z,x)=\displaystyle\sum_{l=0}^{\infty}\frac{x^{\nu+2l}}{\Gamma(\nu+l+1)l!}es_y(1+\nu+2l,1+\nu+2l,t,r,z) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Assume that $u,w$ are variables, the other variables are know.&lt;/strong&gt; I would like to compute an integral like this&#xA;$$I=\int_{0}^{\infty}\int_0^{\infty}f(u,w) is_a(2\nu,cu,0,w+d,e\sqrt{w}) dudw$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the integral $I$ can be approximated by truncating the series at large $n$.&#xA;When i did so, the result converges very slowly, (not very accurate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions&lt;/strong&gt;: is there a better way to approximate the above double integral ?&#xA;Do we have a program language that can use to find $s_y(),es_y(.),is_y(.)$ exactly ( I mean without truncating the series)? &lt;/p&gt;&#xA;" OwnerUserId="17200" LastActivityDate="2015-08-18T22:35:39.807" Title="A better way to compute a double integral involving a infinite series?" Tags="|optimization|numerical-analysis|approximation|numerics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20493" PostTypeId="1" AcceptedAnswerId="20584" CreationDate="2015-08-19T09:18:32.257" Score="1" ViewCount="2790" Body="&lt;p&gt;I am trying to follow an algorithm that is described in &lt;a href=&quot;http://www.physics.drexel.edu/~bob/NewBook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elementary Quantum Mechanics in 1D&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to compute eigen-energies and functions in bound states in the basic case in rectangular potential well shown on page 99 as a simple example. I am doing my code in MATLAB. And I can't get the same solution as shown in page 100. I have tried inspecting the algorithm and it seems ok to me, so I am suspecting whether I'm doing something wrong with the Units or scaling or something like that...but I can't seem to see the problem in there either!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm is inspected throughout pages 13-23. I have written the algorithm to be computed in general cases, and I have checked it with the Gaussian well too, it produces same failure results...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, what I want to achieve is to get correct results for just one potential well. The algorithm in that case can be summed to this. In the general case in which you have a potential function approximated with many step functions, the transfer matrix would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_{0, N+1} = E^{-1}(V_0; a_0)K^{-1}(V_0) M_1 M_2 \cdots M_N K(V_{N+1})E(V_{N+1}; a_{N+1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the matrices $E^{-1}, K^{-1}, E, K$ and $M$'s are given in tables for cases where the input energy $E$ that is being calculated is lesser or higher than the potential energy of the step in the specific region.&#xA;&lt;strong&gt;(table 2 page 11 and the M matrices are given in table 3 page 16)&lt;/strong&gt;&#xA;I would love to use latex to write them here as well, but when I write something other than one-line latex code it just doesn't show (I'd appreciate a tip on that too...?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author of the book decomposes the equations make them simpler for computing. Generally, what needs to be done is to calculate the $M$ matrices and then pre-multiply and post-multiply them with the boundary conditions of the potential function.&#xA;$M$ matrices are dependent on the value of the potential step in the according well and its width.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the most easiest case of one rectangular potential well the regions are left and right of the well. The potential energy outside the well is 20eV while inside is 0V. The with of the well is 8 Angstroms (or 0.8nm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;I need more reputation to post more than two links, and I think this book I posted is really good for reference, so I can't post what the solution looks like but tell you that it's on page 100.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, getting back to the problem in hand.&#xA;I would like to calculate the energy eigenvalues in bound states.&#xA;The criteria for bound states is that the energies that are being calculated need to be less then the energy of the potential well on it's left and right boundary region.$E&amp;lt;V_L,V_R$ where $V_L=V_R=20eV$ in this case.&#xA;The MATLAB code makes vector E with values slightly above 0 and slightly less than 20eV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, since the problem for rectangular well is greatly simplified it turns out the only thing you need to compute is this (page 99):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;t_{11}(E) = \cos(ka) + 0.5 * \left(\frac{\kappa}{k} - \frac{k}{\kappa}\right)\sin(ka) &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k = \sqrt{2mE/\hbar}$, in general case it's $E-V_i$ for it's the case on the region where $E&amp;lt;V_i$ but in this case the value of the well is 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And $\kappa = \sqrt{2m(V_{L/R}-E)/\hbar}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code:    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%main.m&#xA;close all&#xA;clear all&#xA;clc&#xA;&#xA;% parameters for constructing the rectangular potential well&#xA;N=101; %number of points         &#xA;xMin = -0.8e-9; % nm    &#xA;xMax = -xMin;       &#xA;x1 = abs(xMin)/2;&#xA;U0 = 0; % well depth&#xA;&#xA;% &#xA;U = ones(N,1)*20;&#xA;x = linspace(xMin,xMax, N);&#xA;&#xA;% creating the potential function U&#xA;for cn = 1 : N&#xA;   if abs(x(cn)) &amp;lt;= x1, U(cn) = U0; end; %if;&#xA;end;&#xA;&#xA;% Test U&#xA;% plot(x,U,'LineWidth',3);&#xA;&#xA;&#xA;% stepWidth_and_their_Pots(U, x) function returns the width dx of each steps &#xA;% through out the potential function U as well as their corresponding potential value Ustep&#xA;[dx, Ustep] = stepWidth_and_their_Pots(U, x);&#xA;&#xA;numE = 50; % scope of input energy levels E to compute &#xA;% computes the transfer matrix element t11(E) and plots it&#xA;V = transferMatrix(U, dx, Ustep, numE);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [dx, Ustep] = stepWidth_and_their_Pots(U, x)&#xA;&#xA;    flag = [];&#xA;    j = 1;&#xA;    for i = 1:length(U)-1&#xA;        if U(i) ~= U(i+1)&#xA;            flag(j) = i+1;&#xA;&#xA;            if i == length(U)-1 &#xA;                break;&#xA;            else j = j+1;&#xA;            end&#xA;        end&#xA;    end&#xA;&#xA;    j = length(flag);&#xA;&#xA;    dx = [];&#xA;    i = 1;&#xA;    for k = j:-1:1&#xA;        if k-1 == 0&#xA;            break;&#xA;        else&#xA;            dx(i) = x(flag(k)) - x(flag(k-1));&#xA;            i = i+1;&#xA;        end&#xA;    end&#xA;&#xA;    Ustep = [];&#xA;    for i = 1:length(flag)-1 &#xA;        Ustep(i) = U(flag(i));&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function V = transferMatrix(U, dx, Ustep, numE)&#xA;&#xA;    %constants&#xA;    hbar = 1.055e-34;      % J.s&#xA;    q = 1.602e-19;         % C&#xA;    m0 = 9.1093897e-31;    % kg&#xA;    m = 0.067*m0;    &#xA;&#xA;    E = linspace(min(U) + 1e-3, U(1)-1e-3, numE);&#xA;    V = [];&#xA;    for j = 1:length(E)&#xA;        kappa = sqrt(2*m/hbar^2 * q * (U(1) - E(j))); % U(1)=U(N) and both &amp;lt; E&#xA;&#xA;        for i = 1:length(dx)&#xA;            if E(j) &amp;gt; Ustep(i)&#xA;                k = sqrt(2*m/hbar^2 * q * (E(j) - Ustep(i)));&#xA;                Ms = [cos(k*dx(i)), -(1/k)*sin(k*dx(i)); k*sin(k*dx(i)), cos(k*dx(i))];&#xA;            elseif E(j) &amp;lt; Ustep(i)&#xA;                k = sqrt(2*m/hbar^2 * q * (Ustep(i) - E(j)));&#xA;                Ms = [cosh(k*dx(i)), -(1/k)*sinh(k*dx(i)); -k*sinh(k*dx(i)), cosh(k*dx(i))];&#xA;            else &#xA;                k = 0;&#xA;                Ms = [1, -dx(i); 0, 1];&#xA;            end  &#xA;        end&#xA;&#xA;        t11 = 0.5 * (Ms(1, 1) + Ms(2, 2)) - 0.5*(kappa*Ms(1,2) + Ms(2,1)/kappa);&#xA;&#xA;        %T = [exp(kappa*xMin) 0; 0, exp(-kappa*xMin)] * 0.5 * [1, -1/kappa; 1, 1/kappa] ...&#xA;             %* Ms * [1, 1; -kappa, kappa] * [exp(-kappa*xMax) 0; 0, exp(kappa*xMax)];&#xA;&#xA;        %test = cos(k*dx) + 0.5*(kappa/k - k/kappa)*sin(k*dx);&#xA;&#xA;        V(j) = t11;       &#xA;    end&#xA;    plot(E, V)&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE EDIT:&#xA;I checked my code and found an error in the mass unit...it was 0.0067 instead of 0.067, which explains one dimension flaw.&#xA;But I still don't get the correct values.&#xA;I have checked my units again and again, and I just can't seem to understand where's the error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;so I have:&#xA;$$\hbar = 1,055 * 10^{-34} Js$$&#xA;$$q = 1,602 * 10^{-19} C$$&#xA;$$m_0 = 9,1093897 * 10^{-31} kg$$&#xA;$$m = 0,067*m_0 = 6,1033 * 10^{-32} kg$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Length scale is in $nm$ and my energies are in $eV$&#xA;Following the formula for $k$ or $\kappa$ I do get that their units are in $m^{-1}$ and that their power is $10^{8}$ which is good since that is multiplied with $nm$ so the values in the matrices $Ms$ should be ok.&#xA;But I'm still not getting the same results as in the book (page 100)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an update picture and I have updated my code for that stupid mistake in mass $m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IQlD7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IQlD7.png&quot; alt=&quot;t11(E) update results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm still not getting the correct results...&lt;/p&gt;&#xA;" OwnerUserId="17291" LastEditorUserId="17291" LastEditDate="2015-08-26T15:17:33.847" LastActivityDate="2015-08-30T06:48:05.107" Title="Transfer Matrix Method in a rectangular potential well" Tags="|matlab|quantum-mechanics|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="20494" PostTypeId="1" CreationDate="2015-08-19T11:57:30.510" Score="2" ViewCount="148" Body="&lt;p&gt;I am using variational data assimilation (4D-VAR) to estimate emissions of anthropogenic greenhouse gases using a rather complex atmospheric transport model. Hence, the optimal solution to my problem is obtained by minimizing a (non-linear) cost functional $J$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model is weakly nonlinear so I can expect the inverse of Hessian of the 4D-VAR cost functional - let's call it $\nabla^2 J^{-1}$ - at the optimal (analysis) point $x_a$ to be a decent approximation to the posterior error covariance matrix $P$. As per standard 4D-VAR theory, the leading eigenpairs of $\nabla^2 J$ (approximately) span the maximum uncertainty space after the assimilation. This all makes sense when we are at a local optimum - i.e., $\nabla^2 J$ is (semi-)positive definite, but what to do if we have not yet reached a local minimum and $\nabla^2 J$ and some of its leading eigenvalues are negative? Is it &quot;safe&quot; to exclude the corresponding eigenpairs from my approximation to the uncertainty space?&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2015-08-20T09:21:19.640" LastActivityDate="2015-08-20T09:21:19.640" Title="Hessian eigenvalues in 4D-VAR data assimilation" Tags="|linear-algebra|matrix|constrained-optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20495" PostTypeId="2" ParentId="20489" CreationDate="2015-08-19T15:22:46.123" Score="7" Body="&lt;h2&gt;Overview&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;These types of polynomials are used in quantum chemistry, potential theory, &lt;a href=&quot;http://people.mbi.ucla.edu/_library/TrainingCourse_Shimming.pdf&quot; rel=&quot;noreferrer&quot;&gt;magnet shimming&lt;/a&gt;, and probably many other branches of science. One problem is that the nomenclature seems to be subtly different between the fields, so the following description is not definitive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us call the solutions of the Laplace equation $\Delta\Phi = 0$, when they are stated in spherical coordinates, by the name of &lt;a href=&quot;https://en.wikipedia.org/wiki/Solid_harmonics&quot; rel=&quot;noreferrer&quot;&gt;solid harmonics&lt;/a&gt;. There are two classes of solid harmonics: Regular solid harmonics, which vanish at the origin, and irregular solid harmonics, which have a singularity at the origin. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solid harmonics can be parametrized using &lt;a href=&quot;https://en.wikipedia.org/wiki/Spherical_harmonics&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;spherical&lt;/em&gt; harmonics&lt;/a&gt; $Y_l^m$. The parametrization of the regular solid harmonics is $r^l Y_l^m(\theta,\phi)$, and irregular solid harmonics can be stated as $r^{-(l+1)} Y_l^m(\theta,\phi)$, which immediately shows the singularity at the origin. &#xA;These parametrizations of the regular and irregular solid harmonics have the benefit that they are orthonormal. (The corresponding scalar product is an integral over the unit sphere).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both regular and irregular solid harmonics are used for &lt;a href=&quot;https://en.wikipedia.org/wiki/Spherical_multipole_moments&quot; rel=&quot;noreferrer&quot;&gt;spherical multipole expansions&lt;/a&gt;: Think of it as a Taylor expansion, or Fourier expansion, only that the base function are not $x^n$ or $\sin(n \omega t)$, but instead solid harmonics. This kind of expansion is very useful when trying to approximate some function that (at least locally) is known to satisfy the Laplace equation. When expanding it with solid harmonics, then even though you might stop at some multipole moment, you can be certain that the expansion still satisfies the Laplace equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the charges are near the origin, and you want to describe the field (caused by these charges) far away from the origin, you would typically use the irregular solid harmonics as base functions: Their poles are on the origin, but that's OK, because all you care about is that the approximation far &lt;em&gt;away&lt;/em&gt; from the origin is a good one. This is the &quot;standard&quot; multipole expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For situations where you want to approximate the field near the origin, and the field is caused by charges far away, you would use the regular solid harmonics as base functions. This is called the &lt;em&gt;interior&lt;/em&gt; multipole expansion, and it is used in beam optics and ion traps. Now the important part: When the regular solid harmonics are expressed in &lt;em&gt;cartesian&lt;/em&gt; coordinates, the results turn out to be surprisingly simple polynomials in $x,y,z$. (Maybe it's not so surprising, after all, you could always do multidimensional Taylor expansion around the origin, and restrict some of the coefficients, so that the total polynomial always satisfies the Laplace equation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While solid harmonics are typically complex functions, a simple linear combination removes all instances of $i$, and you are left with polynomials in $x,y,z$ where all coefficients are real.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Generating Formula&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There is a &lt;a href=&quot;http://en.citizendium.org/wiki/Solid_harmonics#In_total&quot; rel=&quot;noreferrer&quot;&gt;closed formula&lt;/a&gt; for generating real solid harmonics in cartesian coordinates. The resulting polynomials should, as far as I understand it, be orthogonal and normal over the &lt;em&gt;unit sphere&lt;/em&gt;. Those that I checked turned out to be orthogonal over the &lt;em&gt;cube&lt;/em&gt; $|r| \leq 1$, but not normalized! The following Mathematica code implements the generating formula.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gamma[l_, m_, k_] :=&#xA;  (-1)^k 2^(-l) Binomial[l, k] Binomial[2 l - 2 k, l] (l - 2 k)!/(l - 2 k - m)!;&#xA;pi[l_, m_] :=&#xA;  Sum[gamma[l, m, k] (x^2 + y^2 + z^2)^k z^(l - 2 k - m), {k, 0, Floor[(l - m)/2]}];&#xA;a[m_] :=&#xA;  Sum[Binomial[m, p] x^p y^(m - p) Cos[(m - p) Pi/2], {p, 0, m}];&#xA;b[m_] :=&#xA;  Sum[Binomial[m, p] x^p y^(m - p) Sin[(m - p) Pi/2], {p, 0, m}];&#xA;c[l_, m_] :=&#xA;  Sqrt[(2 - KroneckerDelta[m, 0]) (l - m)!/(l + m)!] pi[l, m] a[m];&#xA;s[l_, m_] :=&#xA;  Sqrt[2 (l - m)!/(l + m)!] pi[l, m] b[m];&#xA;genfun[l_, m_] := If[m &amp;gt;= 0, If[m &amp;lt;= l, c[l, m], 0], If[m &amp;gt;= -l, s[l, -m], 0]] // Simplify;&#xA;result =&#xA; Table[Table[genfun[l, m], {m, {0, 1, -1, 2, -2, 3, -3, 4, -4, 5, -5}}], {l, 0, 5}] // TableForm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Map[Simplify[Laplacian[#, {x, y, z}]] &amp;amp;, result, {2}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to ensure that all these polynomials satisfy the Laplace equation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Alternative I&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A different approach is to start from the &lt;a href=&quot;http://cas.web.cern.ch/cas/Belgium-2009/Lectures/PDFs/Wolski-1.pdf&quot; rel=&quot;noreferrer&quot;&gt;two-dimensional multipole expansion of the &lt;em&gt;field&lt;/em&gt;&lt;/a&gt;, and then generalize to the three-dimensional harmonic polynomials. This can be achieved by multiplying the two-dimensional polynomial with the third variable to any power, and then applying a special operator that I can only describe as funky. This procedure is outlined in the book &lt;a href=&quot;https://books.google.de/books?id=8QiR8lCX_qcC&amp;amp;pg=PA273&amp;amp;lpg=PA273&amp;amp;dq=generate%20%22harmonic%20polynomial%22&amp;amp;source=bl&amp;amp;ots=NK6J61AR7I&amp;amp;sig=dA9LYtHweKsPxMLFEDi6m7fRNk0&amp;amp;hl=de&amp;amp;sa=X&amp;amp;ved=0CCgQ6AEwAWoVChMIosLR3e2zxwIVy7YUCh13WAeD#v=onepage&amp;amp;q=generate%20%22harmonic%20polynomial%22&amp;amp;f=false&quot; rel=&quot;noreferrer&quot;&gt;Fundamental World of Quantum Physics&lt;/a&gt;. The corresponding Mathematica code is lengthy, but it can be evaluated rather quickly. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Alternative II&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A third approach is to use Mathematica to build the solid harmonics from the spherical harmonics, then &lt;a href=&quot;https://mathematica.stackexchange.com/questions/45881/are-solid-spherical-harmonics-implemented-in-mathematica&quot;&gt;transforming them&lt;/a&gt; to &lt;em&gt;cartesian&lt;/em&gt; coordinates, and finally &lt;a href=&quot;https://math.stackexchange.com/questions/145080/how-are-the-real-spherical-harmonics-derived&quot;&gt;combining them&lt;/a&gt; in a way that leaves only &lt;em&gt;real-valued&lt;/em&gt; coefficients. This code is short (it is my original approach), but takes a while to execute. Also, Mathematica fails to simplify the polynomials of degree &gt; 10 in a sensible way.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solidHarmonicS[l_?IntegerQ, m_?IntegerQ] := &#xA; Module[{r, \[Theta], \[Phi], xx, yy, zz}, &#xA;  FullSimplify@&#xA;    Evaluate[&#xA;     TransformedField[&quot;Spherical&quot; -&amp;gt; &quot;Cartesian&quot;, &#xA;      r^l SphericalHarmonicY[l, &#xA;        m, \[Theta], \[Phi]], {r, \[Theta], \[Phi]} -&amp;gt; {xx, yy, &#xA;        zz}]] /. {xx -&amp;gt; x, yy -&amp;gt; y, zz -&amp;gt; z}]&#xA;realSolidHarmonics[l_?IntegerQ, m_?IntegerQ] := &#xA; If[m == 0, solidHarmonicS[l, m], &#xA;   If[m &amp;gt; 0, &#xA;    1/Sqrt[2] (solidHarmonicS[l, m] + (-1)^m solidHarmonicS[l, -m]), &#xA;    1/(I Sqrt[2]) (solidHarmonicS[l, &#xA;        m] - (-1)^m solidHarmonicS[l, -m])]] // Simplify&#xA;result = Table[&#xA;   Table[realSolidHarmonics[i, &#xA;     j], {j, {0, 1, -1, 2, -2, 3, -3, 4, -4, 5, -5}}], {i, 0, 5}] // &#xA;  TableForm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4633" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2015-08-20T11:51:51.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20496" PostTypeId="1" CreationDate="2015-08-19T16:00:17.027" Score="0" ViewCount="1208" Body="&lt;p&gt;I'm trying to solve the advection equation $$m_t+(\alpha m)_x=0$$ with $m(0,\cdot)=m_0$ numerically using the first order Godunov scheme. Hence I write $$m_j^{i+1}=m_j^i-\frac{dt}{dx}(m_{j+1/2}^i\alpha_{j+1/2}^i-m_{j-1/2}^i\alpha_{j-1/2}^i)$$ where $m_{j+1/2}^i=m_{j+1}^i$ if $\alpha_{j+1/2}^i&amp;lt;0$ and $m_j^i$ otherwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a Matlab code with $\alpha=1/2$ everywhere:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% parameters&#xA;N=99; M=99; &#xA;&#xA;%Allocate memory&#xA;x=linspace(0,1,M+1); &#xA;mk=zeros(N+1,M+1); a_ph=ones(N,M)*0.5; &#xA;&#xA;%% Set initial conditions&#xA;sigma=0.08;&#xA;mk(1,:)=1/sqrt(2*pi*sigma^2)*exp(-(x-0.5).^2/(2*sigma^2));&#xA;&#xA;q=dt/h;&#xA;&#xA;for i=1:N&#xA;    ai=ak_ph(i,2:M); aim=a_ph(i,1:M-1); %local var&#xA;    B=diag([0,ai.*(ai&amp;gt;=0),0]) + diag([0,ai.*(ai&amp;lt;0)],1) ...&#xA;       - diag([0,aim.*(aim&amp;lt;0),0]) - diag([aim.*(aim&amp;gt;=0),0],-1);&#xA;    mk(i+1,:)=(eye(M+1)-q*B)*mk(i,:)';&#xA;end&#xA;&#xA;[X1,X2]=meshgrid(x1,x2);&#xA;surf(X1,X2,mk','FaceColor','interp','EdgeColor','none')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I compute the solution this is what I get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BjcCf.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BjcCf.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect a wave travelling at a uniform velocity but the amplitude gets smaller in the meantime which is strange. There are also ripples at the peak of the wave. Does anyone have a suggestion?&lt;/p&gt;&#xA;" OwnerUserId="17295" LastEditorUserId="123" LastEditDate="2015-08-21T17:45:28.057" LastActivityDate="2015-08-22T12:41:28.027" Title="Why is Godunov's scheme (for the advection equation) diffusive?" Tags="|matlab|pde|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20497" PostTypeId="1" CreationDate="2015-08-19T18:43:03.413" Score="2" ViewCount="329" Body="&lt;p&gt;I face to the following problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(1+u_x^2)u_{yy} - 2u_xu_yu_{xy} + (1+u_y^2)u_{xx}=0.$$ Problem needs to be discretized and assembled.&#xA;Does anybody know how to proceed in Matlab? &lt;/p&gt;&#xA;" OwnerUserId="17045" LastEditorUserId="17045" LastEditDate="2015-08-20T19:23:47.280" LastActivityDate="2021-04-19T19:31:02.433" Title="Minimal surface finite differences problem - Matlab assemble" Tags="|matlab|finite-difference|nonlinear-equations|discretization|nonconvex|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="20498" PostTypeId="2" ParentId="20486" CreationDate="2015-08-19T19:32:20.387" Score="5" Body="&lt;p&gt;The other answers make some excellent suggestions, but for the sake of completeness, let me give an explicit answer to the two questions you posed: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;No&lt;/strong&gt;, the starting vector does not have to be random, and &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Yes&lt;/strong&gt;, any vector with a non-zero projection onto the desired eigenvector will do. (A random vector is just the simplest reliable way of achieving that in the absence of further information.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is independent of which (or how many) eigenvectors you want to find; as long as your starting vector has non-zero projection onto the corresponding eigenspace, the Lanczos approximations of a given eigenvalue will converge (with a speed depending on, among others, the magnitude of the projection&lt;sup&gt;1&lt;/sup&gt;).&#xA;The details are a bit technical, but you can find them for example in Youcef Saad's &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot; rel=&quot;noreferrer&quot;&gt;book on iterative methods&lt;/a&gt; (in particular, Theorem 6.4 on page 149).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;1. In fact, if the starting vector is an eigenvector (or more generally, lies in the eigenspace), the Lanczos method will give you the corresponding eigenvalue in a single iteration.&#xA;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-08-19T19:32:20.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20499" PostTypeId="2" ParentId="20484" CreationDate="2015-08-19T21:57:36.757" Score="1" Body="&lt;p&gt;The FFT algorithm computes the &lt;a href=&quot;http://mathworld.wolfram.com/DiscreteFourierTransform.html&quot; rel=&quot;nofollow&quot;&gt;discrete Fourier transform&lt;/a&gt; (DFT), which transforms $x_{1:n}$ to $\hat x_{1:n}$ by&#xA;$$ \hat x_k = \sum_{j=1}^{n} x_j \, e^{-2\pi i (j-1) (k-1)/n}. $$&#xA;The DFT is a &lt;em&gt;linear transform&lt;/em&gt;, as is clear from its formula, meaning that it computes multiplication by a matrix $F$:&#xA;$$ \hat x = Fx, \qquad F = \big( e^{-2\pi i (j-1) (k-1)/n} \big)_{k,j}. $$&#xA;IFFT then implements multiplication by $F^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when you are computing $(a(x)(D_x \hat p)^\vee)^\wedge$, you are computing &#xA;$$ \hat L\hat p = F A F^{-1} D_x \hat p, $$&#xA;where $D_x$ multiplies by $i k$ (a diagonal matrix), and $A$ multiplies by $a(x)$ (a diagonal matrix). The only thing to be done here is to construct $F$ explicitly, and to form the matrix product explicitly. There may sometimes be a difference in normalization conventions and such, so $F$ has to be consistent with what the FFT actually computes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice also that $FAF^{-1}$ here is like an application of the change-of-basis formula for the matrix $A$ from space basis to frequency basis.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-08-19T22:25:51.247" LastActivityDate="2015-08-19T22:25:51.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20500" PostTypeId="1" AcceptedAnswerId="20598" CreationDate="2015-08-20T02:37:03.073" Score="8" ViewCount="4707" Body="&lt;p&gt;I'm investigating algorithms to make a mesh watertight for 3d printing. I'd be very excited to implement such algorithms. The initial input is a mesh which is not watertight and I want to understand and implement algorithms to transform it to make it suitable for 3d printing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a good explanation about watertight meshes: &lt;a href=&quot;http://support.ponoko.com/entries/20217167-creating-watertight-meshes-for-3d-printing&quot;&gt;http://support.ponoko.com/entries/20217167-creating-watertight-meshes-for-3d-printing&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard that the key to watertight meshes is shrink wrapping. I'm completely new to meshes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the most common shrink wrapping algorithms? &lt;/li&gt;&#xA;&lt;li&gt;Where can I find open source implementations?&lt;/li&gt;&#xA;&lt;li&gt;Where can I find real world inputs?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17281" LastActivityDate="2015-10-16T08:33:50.273" Title="Shrink wrapping algorithms to make a mesh watertight for 3d printing" Tags="|mesh|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20501" PostTypeId="1" CreationDate="2015-08-20T09:22:39.530" Score="6" ViewCount="374" Body="&lt;p&gt;&lt;em&gt;This is not a typical question with a deterministic answer. If this is not the right place, feel free to close it.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the past one year I have been working on various kinds of inverse problem. Most of them are related to parameter estimation, ranging from small linear differential equation to equations of fluid dynamics. And consistently, I observed that it was easier to find a solution to the non linear problem(using same optimization method, let's say steepest descent). By easy I mean it requires lower number of iterations for the same decrease(very subjective). Although, it is nearly impossible to compare different cases exactly, this trend is surprising to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question here is, did anyone else observe similar behaviour? Is there a qualitatively explanation for this behaviour?    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also raises a follow up question of whether we can quantify(from the initial jacobian may be?) the level of difficulty involve in finding a solution?(or estimate the order of magnitude of the number of optimizer steps)&lt;/p&gt;&#xA;" OwnerUserId="3640" LastEditorUserId="17294" LastEditDate="2015-08-20T14:35:21.757" LastActivityDate="2015-10-19T15:18:36.297" Title="Linear vs Non Linear inverse problems: Does non-linearity help?" Tags="|optimization|inverse-problem|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20504" PostTypeId="2" ParentId="20483" CreationDate="2015-08-20T13:08:30.930" Score="2" Body="&lt;p&gt;You can avoid the explicit computation of $A^{-1}$ in the Newton iteration for the sign of $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Refer to chapter 5 from this &lt;a href=&quot;http://www.maths.manchester.ac.uk/~higham/fm/OT104HighamChapter5.pdf&quot; rel=&quot;nofollow&quot;&gt;book&lt;/a&gt; by Higham, more specifically equation 5.22.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2015-08-20T13:08:30.930" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20505" PostTypeId="1" CreationDate="2015-08-20T13:24:57.730" Score="2" ViewCount="215" Body="&lt;p&gt;I am trying to improve the resolution of a Stokes problem (P2/P1 on unstructured mesh) defined by the matrix $M$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M=&#xA;\begin{pmatrix}&#xA;A_u &amp;amp; 0 &amp;amp; B_u \\&#xA;0 &amp;amp; A_v &amp;amp; B_v\\&#xA;B_u^T &amp;amp; B_v^T &amp;amp; 0 &#xA;\end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Current Implementation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I am currently solving it using a Uzawa method, with a Krylov method accelerated by a preconditioner. This preconditioner is build by additive composition of the matrices $A_p$ and $A_m$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use Petsc to solve each sub-problems, $A_u$, $A_v$, $A_p$ and $A_m$ (in format mpiaij), but I handle the Uzawa method separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_u$ and $A_v$ are solved using cg, ASM preconditioning and level 2 icc:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-ksp_type cg -pc_type asm -sub_pc_type icc -sub_pc_factor_levels 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$A_p$ is solved using cg with AMG preconditioning:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-ksp_type cg -pc_type hypre -pc_hypre_type boomeramg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$A_m$ is solved using cg without preconditioning:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-ksp_type cg -pc_type none&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Goal&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;To improve the speed and allow for easier tuning , I would like to have Petcs do the all the Uzawa process. I am able to pass $M$ in mpiaij format to Petsc. But the tool I use can only pass one, and only one matrix to Petsc, but I could concatenate $M$+$A_p$+$A_m$ and give that to Petsc:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{pmatrix}&#xA;M &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; A_p &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; A_m&#xA;\end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I tell Petsc to separate $M$ from $A_p$/$A_m$ and to use $A_p$ and $A_m$ as preconditioner matrices for the Shur complement in a Uzawa method?&lt;/p&gt;&#xA;" OwnerUserId="11869" LastEditorUserId="11869" LastEditDate="2015-08-20T14:27:06.510" LastActivityDate="2015-08-20T14:27:06.510" Title="preconditioned Uzawa method with Petsc" Tags="|linear-algebra|parallel-computing|petsc|preconditioning|krylov-method|" AnswerCount="0" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20506" PostTypeId="2" ParentId="20476" CreationDate="2015-08-20T13:31:34.163" Score="4" Body="&lt;p&gt;If your matrices are large, why not use a library like &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow&quot;&gt;ARPACK&lt;/a&gt;? The shift-and-invert mode of ARPACK will help you calculate the eigenvalues close to $\sigma$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are interfaces to ARPACK for most high-level programming languages used for numerical computations (Fortran, C, Python, MATLAB, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a quick tutorial for Python can be found &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/tutorial/arpack.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (reasonably small, dense matrices), or &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.sparse.linalg.eigsh.html#scipy.sparse.linalg.eigsh&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (large, sparse matrices, available only in operator form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later edit: I read your post more carefully and noticed you are looking for a way to solve the shift-invert linear system... so why not try &lt;a href=&quot;http://web.stanford.edu/group/SOL/software/minres/&quot; rel=&quot;nofollow&quot;&gt;MINRES&lt;/a&gt; or &lt;a href=&quot;http://web.stanford.edu/group/SOL/software/symmlq/&quot; rel=&quot;nofollow&quot;&gt;SYMMLQ&lt;/a&gt;? Both algorithms are based on a Lanczos (3-term Arnoldi) iteration specifically designed for symmetric (but not necessarily positive definite) matrices. SYMMLQ should perform better if $A$ is non-singular.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2015-08-20T13:49:30.743" LastActivityDate="2015-08-20T13:49:30.743" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20508" PostTypeId="2" ParentId="20021" CreationDate="2015-08-20T14:50:38.737" Score="4" Body="&lt;p&gt;By its anatomy the DGEMM ($C = \alpha AB + \beta C$) is one of the most optimizeable routines in computer science. For historic reason this routine is implemented in FORTRAN or the implementation provides at least an interface which is compatible the old FORTRAN one. If you want to call it from C you have to take care of the function arguments because all of them need to be a pointer (even the scalar values $\alpha$ and $\beta$ or the sizes of the matrices). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to avoid this converting-to-FORTRAN problem, one can also use the so called CBLAS interface which is provided by all main BLAS implementations (ATLAS, OpenBLAS, Intel MKL) as well. This interface allows you to pass all function arguments in a C style way. But it has one big trap door. If you use the classical row-oriented C-style way for storing two dimensional data, the CBLAS interface will copy all data in the FORTRAN colum-major format which costs time and memory. So if you use CBLAS try to use the colum-major storage as in Fortran. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question for the most optimized way is easy to answer: If you have an license of the Intel Compiler Suite or at least of the Intel MKL use this as BLAS library. If not than use OpenBLAS. Both reach more than 90% of the theoretical peak performance of the computer but OpenBLAS has some problems with its performance on multi-socket systems. &lt;/p&gt;&#xA;" OwnerUserId="569" LastActivityDate="2015-08-20T14:50:38.737" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20509" PostTypeId="5" CreationDate="2015-08-20T20:07:41.707" Score="0" Body="&lt;p&gt;The finite-volume method (FVM) is a method for representing and evaluating partial differential equations in the form of algebraic equations. In the finite volume method, volume integrals in a partial differential equation that contain a divergence term are converted to surface integrals, using the divergence theorem. The method is widely used in Computational Fluid Dynamics codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See Wikipedia page on &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_volume_method&quot; rel=&quot;nofollow&quot;&gt;FVM&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-08-24T16:38:18.663" LastActivityDate="2015-08-24T16:38:18.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20510" PostTypeId="4" CreationDate="2015-08-20T20:07:41.707" Score="0" Body="Referring to the discretization of partial differential equations using Finite Volume Method." OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-08-24T16:38:10.287" LastActivityDate="2015-08-24T16:38:10.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20512" PostTypeId="1" AcceptedAnswerId="20513" CreationDate="2015-08-20T21:51:36.183" Score="0" ViewCount="174" Body="&lt;p&gt;Let us say I want to compute the following expression by a numerical integration scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I = \int^{-x}_{-\infty} f(x + y) \, \mathrm dy - \int_{\mathbb R}\bigg(f(x+y) -f(x)\bigg)\, \mathrm  dy&#xA;$$&#xA;for a range of $x$-values $x \in \mathcal O$ for some $\mathcal O \subset \mathbb R$, and where $f(x)$ is some function such that the integrals are defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one thing about this keeps confusing me: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we treated that expression in a `naive', algebraic way, we could decouple the latter integral and get the simpler expression&#xA;$$&#xA;I = \int_{\mathbb R}f(x) \, \mathrm dy - \int^{\infty}_{-x}f(x+y) \, \mathrm dy&#xA;$$&#xA;but clearly,&#xA;$$&#xA;\int_{\mathbb R}f(x) \, \mathrm dy = f(x) \int_{\mathbb R}\, \mathrm dy = \infty &#xA;$$ &#xA;so the decoupling does not make sense mathematically. But here is my confusion: when you solve this numerically, everything is discretized into sums, i.e. $I$ becomes one large sum, and in that sense it does not seem to matter whether you decouple the integral or not. How does this work? Will a decoupling affect the result itself or perhaps just the convergence rate?&lt;/p&gt;&#xA;" OwnerUserId="17170" LastActivityDate="2015-08-20T22:27:57.197" Title="Decoupling integral $\int f(x+ y )-f(x) dy = \int f(x+y) dy - \int f(x) dy$ in a numerical integration scheme?" Tags="|numerical-analysis|integration|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20513" PostTypeId="2" ParentId="20512" CreationDate="2015-08-20T22:27:57.197" Score="4" Body="&lt;p&gt;Your second integral in the first expression for $I$ is ill-defined unless $\lim_{y\to\pm\infty}f(x+y)=f(x)$ (necessary so that it converges), and the first integral is ill-defined unless $\lim_{y\to-\infty}f(x+y)=0$, which proves the expression is well-defined only if $f(x)=0$. So there is absolutely no contradiction there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, however, $f(x)\neq 0$, then both expressions for $I$ are undefined. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, numerical methods can only be expected to handle well-defined mathematical problems. This here is merely an issue of whether your expression makes sense mathematically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may as well just rewrite $$I = -\int_0^\infty f(y)\,dy, $$&#xA;since that is what both of your expressions are equal to &lt;em&gt;when they exist&lt;/em&gt;, and note that $I$ is independent of $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; In &lt;em&gt;principle&lt;/em&gt; there is a way to &lt;em&gt;regularize&lt;/em&gt; a divergent integral in a meaningful way using some kind of a regularization procedure, but that is not the way to go here, because the purpose of regularization is usually not to deal with integrals like $\int_{\mathbb{R}}1\,dy$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-08-20T22:27:57.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20514" PostTypeId="1" CreationDate="2015-08-21T02:50:40.450" Score="5" ViewCount="236" Body="&lt;p&gt;I am trying to solve one dimensional inviscid Burger's equation using adaptive mesh refinement. This is the PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\delta U}{\delta t} + \frac{\delta F}{\delta x} = 0$$&#xA;where the flux F of the variable U is defined as$$ F= \frac{U^2}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this equation, I am defining a control volume around point i, then I should write the original equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{U_i^{n+1} - U_i^{n}}{\Delta t} + \frac{ F_{i+0.5}^{n} - F_{i-0.5}^{n}}{\Delta x} = 0 $$&#xA;I am defining these fluxes as (Ref: &lt;a href=&quot;http://www.astro.uu.se/~bf/course/numhd_course_20100124.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.astro.uu.se/~bf/course/numhd_course_20100124.pdf&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jxM4C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jxM4C.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This scheme works perfectly for any particular mesh. Now, when I am using the adaptive mesh refinement scheme proposed by Berger and Collela (&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0021999189900351&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/0021999189900351&lt;/a&gt;). I am confused about how to calculate fluxes at coarse-fine mesh interface as shown in this figure: (Refinement factor is 4 and &lt;em&gt;stars&lt;/em&gt; are fine mesh points. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/LQuWy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LQuWy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During integrating fine mesh, I need to get these fluxes because these fluxes are used to correct the coarse grid points values to ensure conservation in the grid hierarchy. In this case, I need to correct the values of &lt;em&gt;i-1&lt;/em&gt; and &lt;em&gt;i+1&lt;/em&gt; using these fluxes &lt;strong&gt;which are associated with the fine mesh&lt;/strong&gt;. Since they are &lt;em&gt;boundary&lt;/em&gt; flux for the fine mesh and also, the boundary values changes with each fine grid time step (unlike the coarse mesh because the coarse mesh boundary values are the boundary conditions assigned to PDE), I can not figure out about how to evaluate them. After every fine grid timestep, I know the &lt;strong&gt;u&lt;/strong&gt; values at all four points. &lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2016-03-29T19:47:38.610" Title="Flux at coarse-fine mesh grid interface?" Tags="|pde|finite-difference|finite-volume|mesh|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20515" PostTypeId="1" AcceptedAnswerId="20523" CreationDate="2015-08-21T05:07:50.207" Score="11" ViewCount="6196" Body="&lt;p&gt;I am wondering how Dirichlet boundary conditions in global sparse finite element matrices are actually implemented efficiently. For example lets say that our global finite element matrix was:&#xA;$$K = &#xA;\begin{bmatrix}&#xA;    5  &amp;amp; 2 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 \\&#xA;    2  &amp;amp; 4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0  &amp;amp; 1 &amp;amp; 6 &amp;amp; 3 &amp;amp; 2 \\&#xA;    -1  &amp;amp; 0 &amp;amp; 3 &amp;amp; 7 &amp;amp; 0 \\&#xA;    0  &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 3&#xA;\end{bmatrix}&#xA;\hspace{5mm}\text{and right-hand side vector}\hspace{5mm} &#xA;b = &#xA;\begin{bmatrix}&#xA;  b1 \\&#xA;  b2 \\&#xA;  b3 \\&#xA;  b4 \\&#xA;  b5 \\&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then to apply a Dirichlet condition on the first node ($x_{1}=c$) we would zero out the first row, put a 1 at $K_{11}$, and subtract the first column from the right-hand side. For example our system would become:&#xA;$$K = &#xA;\begin{bmatrix}&#xA;    1  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0  &amp;amp; 4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0  &amp;amp; 1 &amp;amp; 6 &amp;amp; 3 &amp;amp; 2 \\&#xA;    0  &amp;amp; 0 &amp;amp; 3 &amp;amp; 7 &amp;amp; 0 \\&#xA;    0  &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 3&#xA;\end{bmatrix}&#xA;\hspace{5mm}\text{and right-hand side vector}\hspace{5mm} &#xA;b = &#xA;\begin{bmatrix}&#xA;  c \\&#xA;  b2-2\times{c} \\&#xA;  b3-0\times{c} \\&#xA;  b4+1\times{c} \\&#xA;  b5-0\times{c} \\&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all well and good in theory, but if our K matrix is stored in compressed row format (CRS) then moving the columns to the right-hand side becomes expensive for large systems (with many nodes being dirichlet). An alternative would be to not move the columns corresponding to a Dirichlet condition to the right-hand side, i.e. our system would become:&#xA;$$K = &#xA;\begin{bmatrix}&#xA;    1  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    2  &amp;amp; 4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0  &amp;amp; 1 &amp;amp; 6 &amp;amp; 3 &amp;amp; 2 \\&#xA;    -1  &amp;amp; 0 &amp;amp; 3 &amp;amp; 7 &amp;amp; 0 \\&#xA;    0  &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 3&#xA;\end{bmatrix}&#xA;\hspace{5mm}\text{and right-hand side vector}\hspace{5mm} &#xA;b = &#xA;\begin{bmatrix}&#xA;  c \\&#xA;  b2 \\&#xA;  b3 \\&#xA;  b4 \\&#xA;  b5 \\&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This however has a major draw back in that the system is no longer symmetric and so we could no longer use preconditioned conjugate gradient (or other symmetric solvers). One interesting solution that I came across is the &quot;Method of Large Numbers&quot; which I found in the book &quot;Programming Finite Elements in Java&quot; by Gennadiy Nikishkov. This method uses the fact that double precision only contains around 16 digits of accuracy. Instead of putting a 1 in the $K_{11}$ position we place a large number. For example our system becomes:&#xA;$$K = &#xA;\begin{bmatrix}&#xA;    1.0e64  &amp;amp; 2 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 \\&#xA;    2  &amp;amp; 4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0  &amp;amp; 1 &amp;amp; 6 &amp;amp; 3 &amp;amp; 2 \\&#xA;    -1  &amp;amp; 0 &amp;amp; 3 &amp;amp; 7 &amp;amp; 0 \\&#xA;    0  &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 3&#xA;\end{bmatrix}&#xA;\hspace{5mm}\text{and right-hand side vector}\hspace{5mm} &#xA;b = &#xA;\begin{bmatrix}&#xA;  c\times{1.0e64} \\&#xA;  b2 \\&#xA;  b3 \\&#xA;  b4 \\&#xA;  b5 \\&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantages of this method are that it maintains the symmetry of the matrix while also being very efficient for sparse storage formats. My questions then are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How are Dirichlet boundary conditions typically implemented in finite element codes for heat/fluids? Do people use the method of large numbers usually or do they do something else? Is there any disadvantage to the method of large numbers that someone can see? I am assuming that there is probably some standard efficient method used in most commercial and non-commercial codes that solves this problem (obviously I not expecting people to know all the inner workings of every commercial finite element solver, but this problem seems basic/fundamental enough that someone likely has worked on such projects and could provide guidance). &lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="713" LastEditDate="2015-08-21T14:17:09.070" LastActivityDate="2019-04-15T09:35:28.507" Title="How to efficiently implement Dirichlet boundary conditions in global sparse finite element stiffnes matrices" Tags="|finite-element|sparse-matrix|boundary-conditions|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20516" PostTypeId="2" ParentId="14196" CreationDate="2015-08-21T09:44:12.997" Score="1" Body="&lt;p&gt;I realize this is an old question, but when looking for this myself today I found &lt;a href=&quot;http://sourceforge.net/projects/adigator/&quot; rel=&quot;nofollow&quot;&gt;ADiGator&lt;/a&gt;, which is open source, and seems to handle vectors. I haven't tested it yet myself, but it seems to be actively developed.&lt;/p&gt;&#xA;" OwnerUserId="17311" LastActivityDate="2015-08-21T09:44:12.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20517" PostTypeId="1" AcceptedAnswerId="20521" CreationDate="2015-08-21T10:47:24.047" Score="1" ViewCount="285" Body="&lt;p&gt;I am trying to implement my own multigrid method in fenics. Is there any &quot;smart/ fenics&quot; way how to assemble subdomains and obtain restriction/ prolongation operators ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks! &lt;/p&gt;&#xA;" OwnerUserId="17150" LastActivityDate="2015-08-21T16:01:30.223" Title="FENICS subdomains - restriction/ prolongation operators" Tags="|finite-element|fenics|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20519" PostTypeId="1" CreationDate="2015-08-21T14:33:56.680" Score="5" ViewCount="89" Body="&lt;p&gt;Consider a planar graph, where each node is associated with a weight. I would like to partition the graph such that the sum of the node weights in each group satisfy a minimum requirement. However, I would also like as much 'resolution' as possible - that is, I want to maximize the number of groups (minimize the number of nodes per group). Internal edges should be rewarded, to avoid long 'daisy chains' of nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any suggestions as to how I can compute an (approximately) optimal solution? My instinct is to approach this using Monte Carlo, but I'm not sure how I would implement it here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any insights or comments you might have!&lt;/p&gt;&#xA;" OwnerUserId="17312" LastActivityDate="2015-08-21T16:14:40.100" Title="Optimal partitioning of a graph" Tags="|optimization|monte-carlo|graph-theory|combinatorics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20520" PostTypeId="2" ParentId="20468" CreationDate="2015-08-21T15:22:51.700" Score="1" Body="&lt;p&gt;I also found this link on Stack Overflow. I swear I looked for questions like  mine for forever but it seems the best way to search a question on stack overflow is to start to make a question and it will suggest similar posts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway here is the link which does in fact say that you can run MPI3.0+ in shared memory mode like openmp. There are other sources I found which also suggest and some that state you can but do not explain how or give resources to figure it out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/24797298/mpi-fortran-code-how-to-share-data-on-node-via-openmp&quot;&gt;https://stackoverflow.com/questions/24797298/mpi-fortran-code-how-to-share-data-on-node-via-openmp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After following the advice given here in this link I ran into issues with putting USE 'mpi_f08' at the top of my code. I am using gfortran-4.8 which apparently does not currently have enough FORTRAN2008 support to USE mpi_f08. There person in this link probably was using the intel fortran 2008 compiler. You can use USE mpi which covers up to Fortran2003 pretty well. However I do not think that Fortran 2003 has the Type() functions which are needed to use the Remote Memory Access functions, such as MPI_WIN_ALLOCATE_SHARED, so you cannot use them without FORTRAN2008. But I really do not have enough resources to definitely say that for a fact but what I have found does suggest that. &lt;/p&gt;&#xA;" OwnerUserId="17265" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-08-21T15:22:51.700" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20521" PostTypeId="2" ParentId="20517" CreationDate="2015-08-21T15:52:45.723" Score="2" Body="&lt;p&gt;There is currently no documented 'FEniCS way' to do this. However, since FEniCS is a pretty standard finite element code behind all the UFL and code generation magic, you can implement things like transfer operators by yourself. The only difficulty is that they have no built-in mechanism to deal with the inter-level mappings in hierarchically refined meshes and this surely complicates things when you want to efficiently implement geometric multigrid. This is not a limitation of the concept but rather of the current implementation. There is &lt;a href=&quot;https://launchpad.net/fmg&quot; rel=&quot;nofollow&quot;&gt;FMG&lt;/a&gt;, which works around some of these difficulties. However, I have no idea if this package is still actively maintained or if the implementation is good for production use. Given that the FEniCS core is rapidly evolving I would not bet that codes which need to assume certain things about the interior workings have a long lifetime. You can however always use older versions of FEniCS which are compatible if you decide to go this way. If you need this for more than a quick hack it might however be better to implement everything in a feature branch and convince the developers to merge your changes in the development branch once you reach a certain level of stability.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="4260" LastEditDate="2015-08-21T16:01:30.223" LastActivityDate="2015-08-21T16:01:30.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20522" PostTypeId="2" ParentId="20519" CreationDate="2015-08-21T16:14:40.100" Score="2" Body="&lt;p&gt;If all you're looking for is an approximate solution, I would suggest starting with one of the well-known graph partitioning packages, for example METIS. It allows you to attach weights to nodes. Partition it into $P$ groups and check whether your minimal weight sum condition is satisfied. If yes, try partitioning into $P+1$ groups and check again. Then repeat until one of your groups violates the minimal weight sum condition.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-08-21T16:14:40.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20523" PostTypeId="2" ParentId="20515" CreationDate="2015-08-21T16:22:30.987" Score="13" Body="&lt;p&gt;In deal.II (&lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dealii.org&lt;/a&gt; -- disclaimer: I'm one of the principal authors of that library), we do eliminate whole rows and columns, and it is not too expensive overall. The trick is to use the fact that the sparsity pattern is typically symmetric, so you know which rows you need to look into when eliminating a whole column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The better approach, in my view, is to eliminate these rows and columns in the cell matrices, before they are added to the global matrix. There you work with full matrices, so everything is efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never heard of the large-numbers approach and would not use it because surely it will lead to terribly ill-conditioned problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, the algorithms we use in deal.II are described conceptually in lectures 21.6 and 21.65 at &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt; . They closely match your description.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="24963" LastEditDate="2019-04-15T09:35:28.507" LastActivityDate="2019-04-15T09:35:28.507" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20524" PostTypeId="1" AcceptedAnswerId="20551" CreationDate="2015-08-21T22:10:56.283" Score="1" ViewCount="132" Body="&lt;p&gt;I am solving elliptic PDE problem, for which, Euler scheme looks as following: &#xA;$$ \nabla [\gamma ( |\nabla u|^2) \nabla u] = 0,$$&#xA;where&#xA;$$\gamma(|\nabla u|^2) = (1 + |\nabla u|^2)^{-1/2}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aiming to define minimization problem for problem above. Does anybody have any suggestion how it should look like? Or some recommended literature?&lt;/p&gt;&#xA;" OwnerUserId="17150" LastEditorUserId="123" LastEditDate="2015-08-22T13:42:07.997" LastActivityDate="2015-08-24T23:31:27.477" Title="Building minimization optimization problem for 2nd-order elliptic PDE" Tags="|optimization|pde|numerical-analysis|discretization|elliptic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20525" PostTypeId="2" ParentId="20515" CreationDate="2015-08-22T03:43:14.107" Score="2" Body="&lt;p&gt;Zero BCs are easy. For Non zero BC's you can also use Lagrange multipliers. E.g., see &lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch09.d/IFEM.Ch09.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. One advantage of LMs is that you can use any constraint equation, although the system becomes indefinite so you need an appropriate solver.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2015-08-22T04:04:46.690" LastActivityDate="2015-08-22T04:04:46.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20526" PostTypeId="2" ParentId="20496" CreationDate="2015-08-22T05:31:36.350" Score="2" Body="&lt;p&gt;Godunov's scheme is a tricky scheme that &lt;em&gt;chooses the stencil based on the direction of the propagation of the wave&lt;/em&gt;. Because of the hyperbolic character of the equation, we know in advection problems that the information should go from left to right if the wave speed is positive and vice versa for negative wave speed. Godunov's scheme described this trick with the mathematical expression that you have written that in your expression like &quot;$m_{j+1/2}^i=m_{j+1}^i$ if $\alpha_{j+1/2}^i&amp;lt;0$ and $m_j^i$ otherwise&quot;. &#xA;If the sign is reversed it will lead to unphysical oscillations!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why this scheme is diffusive and how to get rid of that&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity I'm assuming wave speed $\alpha$ =1;&#xA;Then the exact solution of this problem, can be written as $$m^{n+1}_i=m^n_{i-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For positive $\alpha$ the finite difference equation can be written as&#xA;$$m^{n+1}_i=m^n_i-CFL*(m^n_i-m^n_{i-1})$$ or&#xA;$$m^{n+1}_I=CFL*m^n_{i-1}+m^n_i(1-CFL)$$ &#xA;here CFL=&lt;a href=&quot;https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&quot; rel=&quot;nofollow&quot;&gt;Courant–Friedrichs–Lewy&lt;/a&gt; number =$\alpha \frac{\delta t}{\delta x}$&#xA;If we  do &lt;a href=&quot;http://www.math.umn.edu/~olver/num_/lnp.pdf&quot; rel=&quot;nofollow&quot;&gt;Von-Newman stability analysis&lt;/a&gt; on that problem, it will restrict our $CFL$ number less than one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still our discretized result is not matching with the exact result.&#xA;Let put $CFL$ =1, then the result will match with exact result. the second term $m^n_i(1-CFL)$ is called numerical viscosity, that is the term that diffuses the result. You can notice that that term increases with reducing the $CFL$ number. That means artificial viscosity is increasing, will decreases the signal amplitude (you have observed this in your code) . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please try to change the $CFL$ number and analyze your result you will understand it clearly. If you want to see the effect of  grid spacing then you should try some higher order &lt;em&gt;TVD&lt;/em&gt; schemes or &lt;em&gt;ENO&lt;/em&gt; with different initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that analysis I discussed here is only valid for forward Euler time discretization and may not be valid for other time discretizations. For that we should do phase velocity plot and numerical diffusion coefficient plot for that scheme. As a rule of thumb we can say odd order schemes are diffusive and even order schemes are dispersive. In most of the good numerical schemes this diffusion and dispersion can be controlled by changing &lt;em&gt;CFL&lt;/em&gt; or &lt;em&gt;cell Reynolds number&lt;/em&gt; and grid size.&lt;/p&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="7733" LastEditDate="2015-08-22T12:41:28.027" LastActivityDate="2015-08-22T12:41:28.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20527" PostTypeId="1" CreationDate="2015-08-22T09:42:59.320" Score="2" ViewCount="95" Body="&lt;p&gt;I'm just starting at scientific computing and I'm seeing that results can be generated merely by plotting the variables. Of course more variables lead to more axes/dimensions so e.g. a three variable system could be or would need to be modeled as a 3D plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I was thinking though was e.g. a 2D case. Does visualization e.g. through some dummy variables (to add dimensions, i.e. to create a 3D visualization out of 2D variables) add anything to simple X-Y plots?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is visualization always linked to the dimensions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does visualization (e.g. an animation) add something more to the &quot;generic plots&quot;? Or is visualization often the same as the plot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about in the case of DEs, which may have multiple variables?&lt;/p&gt;&#xA;" OwnerUserId="17316" LastEditorUserId="17316" LastEditDate="2015-08-22T12:08:51.263" LastActivityDate="2015-08-22T18:31:19.350" Title="What's the role of visualization?" Tags="|visualization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20529" PostTypeId="1" CreationDate="2015-08-22T11:34:52.763" Score="2" ViewCount="332" Body="&lt;p&gt;The governing equations are listed &lt;a href=&quot;https://drive.google.com/file/d/0ByQmnrnCesdIbVJ4RGkzUXNVWlE/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; of my notes on page 4. It's a reproduction of other's paper which solves the equations with COMSOL. The problems arise when I want to solve for the consistent initial condition of the algebraic components, $\phi_1$ and $\phi_2$. They consist of a kinetic reaction term for five reactants&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;a \sum_j i_j&#xA;\end{equation}&#xA;\begin{equation}&#xA;i_j = i_{o,j}\biggl[ \prod_i \biggl(\frac{C_i}{C_{i,o}}\biggr)^{p_{i,j}}\exp\biggl(\frac{\alpha_{aj} F}{RT}(\eta-U_j)\biggr) - \prod_i \biggl(\frac{C_i}{C_{i,o}}\biggr)^{q_{i,j}}\exp\biggl(-\frac{\alpha_{cj} F}{RT}(\eta-U_j)\biggr) \biggr]&#xA;\end{equation}&#xA;where $\eta=\phi_1-\phi_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First try for these two algebraic equations on MINPACK gives $l^2$ norm of the residual around $1.0E-2$ for one dimension case $=500$. Then I twisted the equations so they become one equation of variable $\eta$ (not two coupled equations), the equations listed &lt;a href=&quot;https://drive.google.com/file/d/0ByQmnrnCesdIWmxWUXRFVFN1Nnc/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; on page 4. Remarkably the $l^2$ norm drops to $1.0E-5$. Nevertheless, when I integrate over time they failed on the very first step. It seems that when the equations are all coupled together with this kinetic term, the convergence of the Newton's method failed (I've only tried on the solver DASKR (DASSL's variant) and RADAU5). I'm using method of line and the space discretization is second order. And the Neumann boundary conditions are all coupled back to the governing equations with the ghost points. The Jacobian matrices are generated by Tapenade. The problem is very close to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10905/poisson-nernst-planck-equations-with-ill-conditioned-sparse-matrix&quot;&gt;this&lt;/a&gt; one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore I think (with my little numerical knowledge) the equation of this kind&#xA;\begin{equation}&#xA;\sigma \nabla^2 \phi_1 = a\sum_j i_j &#xA;\end{equation} &#xA;resembles a singular perturbed equation for the coefficient $\sigma/a$ can be of order $1.0E-10$ on the sulfur cathode. $a$ is the surface area of the pore walls per unit volume of the total electrode. From the output data at the boundary there is a one point drop from a relatively smooth inner domain. I'm thinking pseudo-spectral method might solve the problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question is that is it perfectly fine to impose boundary conditions (specifically Neumann type) on PDE as a discretized algebraic equations without putting back into the governing equation with ghost point? I feel like somehow the boundary point won't satisfy the governing equation this way. And when implementing in the IDE or DAE solver I need to explicitly specify they are algebraic term though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One last question: do the initial conditions need to satisfy the boundary conditions? I think most system don't. Or else I have to let the applied current to be zero first and then at $t&amp;gt;0$ crank up to some value, making a discontinuity at the first step. I've found one paper describe this discontinuous initial condition, &lt;a href=&quot;http://www.cs.ucsb.edu/~cse/Files/DAEs_not_ODEs.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAEs not ODEs&lt;/a&gt;  page 371, but I still don't know the technique in solving them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been working this whole summer vacation from scratch and this is my very first time solving a real (non-toy) numerical problems. Any suggestion would be greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quad precision without precondition just solves my first problem regarding ill-conditioned matrix. The third problem (inconsistent initial and boundary condition) is also solved by making initial guessing on the algebraic variables sufficiently close to the values when the source term jumps. But the problem is quad precision implementation is more than 10x slower than 8 bytes double precision. Will it help if I reorder the Jacobian matrix so that it is banded (currently the linear solver is LAPACK with dense matrix)? Or I just need to learn PETSc and make use of the parallel feature and sparse solver? Also, how fined should it be so that the solution can considered fine-grid solution (and hence make error estimation from that)? The second problem above in the main context still puzzles me though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT.II&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5Gw8Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reaction current $\nabla \cdot i_2$&lt;/a&gt; figure from a review &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/aic.690210103/abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; on porous electrode. The author define a dimensionless current density $\delta = \frac{\alpha_a FIL}{RT}(\frac{1}{\kappa}+\frac{1}{\sigma})$ which monitor how non-uniform the current distribution can be. When $\delta=100$, the current at the boundary is very steep already. The system I studied has even higher of this parameter though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT.III&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solve the discontinuity problem regarding the Poisson equation. From the dimensionless parameter above, there is a term $L$. I put $1$ in my code which is non-physical for it means 1m electrode length for consistent unit. After lowering it down to around $1E-5$, the source term becomes smooth. Thanks for Geoff's a priori thought that made me look closer to the physical quantity.&lt;/p&gt;&#xA;" OwnerUserId="16998" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-08-24T17:01:52.940" Title="Ill-conditioned Jacobian matrix from Nernst-Planck equation with Butler-Volmer reactions" Tags="|pde|finite-difference|newton-method|jacobian|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20530" PostTypeId="2" ParentId="20527" CreationDate="2015-08-22T14:09:57.570" Score="3" Body="&lt;p&gt;Visualization, generally, is the science of translating data to insight via graphical representations. There are as many was to do it as you can imagine. For PDEs, there are lots of ways to slice and dice scalar fields and there are also lots of was to show vectors. It just depends on what you're looking for. There are techniques for everything. The answer to all of your questions is &quot;yes&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What problem are you studying?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-08-22T14:09:57.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20531" PostTypeId="2" ParentId="20524" CreationDate="2015-08-22T15:33:13.183" Score="0" Body="&lt;p&gt;Let's assume, that you have discretized problem and on every point you obtained &#xA;evaluation $ g_{ij} $. Therefore $g(u) = 0$ is your system of nonlinear equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let $\gamma$ to be norm function defined as $\gamma(x) = || g(x)||^2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your nonlinear eq. problem is equivalent to the problem &#xA;$$ \text{minimize} \ \  \gamma(x) .$$&lt;/p&gt;&#xA;" OwnerUserId="17045" LastActivityDate="2015-08-22T15:33:13.183" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20532" PostTypeId="2" ParentId="20527" CreationDate="2015-08-22T18:31:19.350" Score="1" Body="&lt;p&gt;Visualization is not a straightforward task and you can do countless things wrong when you try to visualize some complex result. Sometimes you may think you have a super smart and compact way to depict something but your readers will simply not understand what you mean. This refers mainly to artificially adding dimensions to plots, which I think should be generally avoided (though many exceptions exist). For instance, if you have a one-dimensional time-dependent PDE to solve and you plot it as an elevation plot in two dimensions this may seem appealing, since you can display the whole solution in one plot. However, if this is not a standard way to plot results in your community, it might be really confusing and in such cases it may be much more appropriate to show a series of plots at different times and let the user fill in the temporal gaps in his head. As a rule of thumb, I always try to adhere to what the visualization frameworks offer and try to be close to what others did before me, not because I believe it is THE best way, but because I know that readers are used to a certain way of presentation. I think of this in the same way as for graphical user interfaces. You can argue the whole day that combo boxes are a terrible way to make users choose among different options, or that the OK button should be on top rather than left of of the Cancel button in a message dialog, but it is difficult to convince everybody else to adhere to your way of thinking. Of course, when there is no standard way to represent data you can try to be be creative. But you will not often run into situations where you have to visualize something which nobody ever visualized before and it pays to make it easy for the reader to interpret your results. If you try to visualize in unexplored territories and have any doubt, ask the experts. There are visualization chairs at most universities and they are there for a reason.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastActivityDate="2015-08-22T18:31:19.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20533" PostTypeId="1" CreationDate="2015-08-22T19:27:45.533" Score="12" ViewCount="2573" Body="&lt;p&gt;In &lt;a href=&quot;https://math.stackexchange.com/a/1404738/75794&quot;&gt;my answer&lt;/a&gt; to a &lt;a href=&quot;https://math.stackexchange.com/q/1392051/75794&quot;&gt;question on MSE&lt;/a&gt; regarding a 2D Hamiltonian physics simulation, I have suggested using a higher-order &lt;a href=&quot;https://en.wikipedia.org/wiki/Symplectic_integrator&quot; rel=&quot;nofollow noreferrer&quot;&gt;symplectic integrator&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I thought it might be a good idea to demonstrate&#xA;the effects of different time steps on the global accuracy of methods&#xA;with different orders,&#xA;and I wrote and ran a Python/Pylab script to that effect.&#xA;For comparison I chose:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(&lt;em&gt;leap2&lt;/em&gt;) &lt;a href=&quot;https://en.wikipedia.org/wiki/Symplectic_integrator#A_second-order_example&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia's 2nd-order example&lt;/a&gt;&#xA;with which I am familiar, although I know it under the name &lt;em&gt;leapfrog&lt;/em&gt;,&lt;/li&gt;&#xA;&lt;li&gt;(&lt;em&gt;ruth3&lt;/em&gt;) &lt;a href=&quot;https://en.wikipedia.org/wiki/Symplectic_integrator#A_third-order_example&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ruth's 3rd-order symplectic integrator&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;(&lt;em&gt;ruth4&lt;/em&gt;) &lt;a href=&quot;https://en.wikipedia.org/wiki/Symplectic_integrator#A_fourth-order_example&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ruth's 4th-order symplectic integrator&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The strange thing is, whatever timestep I choose, Ruth's 3rd-order method&#xA;seems to be be more accurate in my test than Ruth's 4th-order method,&#xA;even by an order of magnitude.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is therefore: &lt;em&gt;What am I doing wrong here?&lt;/em&gt; Details below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The methods unfold their strength in systems with &lt;em&gt;separable&lt;/em&gt; Hamiltonians,&#xA;i.e. those that can be written as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$H(q,p) = T(p) + V(q)$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; comprises all position coordinates,&#xA;&lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; comprises the conjugate momenta,&#xA;&lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; represents kinetic energy and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; potential energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In our setup, we can normalize forces and momenta&#xA;by the masses they are applied to. Thus forces turn into accelerations,&#xA;and momenta turn into velocities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The symplectic integrators come with special (given, constant) coefficients&#xA;which I will label &lt;span class=&quot;math-container&quot;&gt;$a_1,\ldots,a_n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b_1,\ldots,b_n$&lt;/span&gt;.&#xA;With those coefficients, one step for evolving the system from time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&#xA;to time &lt;span class=&quot;math-container&quot;&gt;$t+\delta t$&lt;/span&gt; takes the form&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$i=1,\ldots,n$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute vector &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; of all accelerations, given vector &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; of all positions&lt;/li&gt;&#xA;&lt;li&gt;Change vector &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; of all velocities by &lt;span class=&quot;math-container&quot;&gt;$b_i\,g\,\delta t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Change vector &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; of all positions by &lt;span class=&quot;math-container&quot;&gt;$a_i\,v\,\delta t$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The wisdom now lies in the coefficients. These are&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\begin{bmatrix}&#xA;a_1 &amp;amp; a_2 \\&#xA;b_1 &amp;amp; b_2&#xA;\end{bmatrix}&#xA;&amp;amp;= \begin{bmatrix}&#xA;\frac{1}{2} &amp;amp; \frac{1}{2} \\ 0 &amp;amp; 1&#xA;\end{bmatrix}&#xA;&amp;amp;&amp;amp;\textsf{(leap2)} \\&#xA;\begin{bmatrix}&#xA;a_1 &amp;amp; a_2 &amp;amp; a_3 \\&#xA;b_1 &amp;amp; b_2 &amp;amp; b_3&#xA;\end{bmatrix}&#xA;&amp;amp;= \begin{bmatrix}&#xA;\frac{2}{3} &amp;amp; -\frac{2}{3} &amp;amp; 1 \\&#xA;\frac{7}{24} &amp;amp; \frac{3}{4} &amp;amp; -\frac{1}{24}&#xA;\end{bmatrix}&#xA;&amp;amp;&amp;amp;\textsf{(ruth3)} \\&#xA;\begin{bmatrix}&#xA;a_1 &amp;amp; a_2 &amp;amp; a_3 &amp;amp; a_4 \\&#xA;b_1 &amp;amp; b_2 &amp;amp; b_3 &amp;amp; b_4&#xA;\end{bmatrix}&#xA;&amp;amp;= \frac{1}{2-\sqrt[3]{2}}&#xA;\begin{bmatrix}&#xA;\frac{1}{2} &amp;amp; \frac{1-\sqrt[3]{2}}{2} &amp;amp; \frac{1-\sqrt[3]{2}}{2} &amp;amp; \frac{1}{2} \\&#xA;0 &amp;amp; 1 &amp;amp; -\sqrt[3]{2} &amp;amp; 1&#xA;\end{bmatrix}&#xA;&amp;amp;&amp;amp;\textsf{(ruth4)}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For testing, I have chosen the 1D initial value problem&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;y''+y &amp;amp;= 0 &amp;amp; y(0) &amp;amp;= 1 &amp;amp; y'(0) &amp;amp;= 0&#xA;\end{align}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\therefore\quad \left(y(t),y'(t)\right) = (\cos t, -\sin t)$$&lt;/span&gt;&#xA;which has a separable Hamiltonian.&#xA;Here, &lt;span class=&quot;math-container&quot;&gt;$(q,v)$&lt;/span&gt; are identified with &lt;span class=&quot;math-container&quot;&gt;$(y,y')$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have integrated the IVP with the above methods over &lt;span class=&quot;math-container&quot;&gt;$t\in[0,2\pi]$&lt;/span&gt;&#xA;with a stepsize of &lt;span class=&quot;math-container&quot;&gt;$\delta t=\frac{2\pi}{N}$&lt;/span&gt; with an integer &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; chosen&#xA;somewhere between &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;.&#xA;Taking &lt;em&gt;leap2&lt;/em&gt;´s speed into account, I tripled &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; for that method.&#xA;I then plotted the resulting curves in phase space and zoomed in at &lt;span class=&quot;math-container&quot;&gt;$(1,0)$&lt;/span&gt;&#xA;where the curves should ideally arrive again at &lt;span class=&quot;math-container&quot;&gt;$t=2\pi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are plots and zooms for &lt;span class=&quot;math-container&quot;&gt;$N=12$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$N=36$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jP4k4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jP4k4.png&quot; alt=&quot;N=12&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/b9EXm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b9EXm.png&quot; alt=&quot;N=12, zoomed&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3bjiP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3bjiP.png&quot; alt=&quot;N=36&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/YauLE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YauLE.png&quot; alt=&quot;N=36, zoomed&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$N=12$&lt;/span&gt;, &lt;em&gt;leap2&lt;/em&gt; with step size &lt;span class=&quot;math-container&quot;&gt;$\frac{2\pi}{3N}$&lt;/span&gt;&#xA;happens to arrive closer to home than &lt;em&gt;ruth4&lt;/em&gt;  with step size &lt;span class=&quot;math-container&quot;&gt;$\frac{2\pi}{N}$&lt;/span&gt;.&#xA;For &lt;span class=&quot;math-container&quot;&gt;$N=36$&lt;/span&gt;, &lt;em&gt;ruth4&lt;/em&gt; wins over &lt;em&gt;leap2&lt;/em&gt;.&#xA;However, &lt;em&gt;ruth3&lt;/em&gt;, with the same step size as &lt;em&gt;ruth4&lt;/em&gt;,&#xA;arrives much closer to home than both the others,&#xA;in all settings I have tested so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the Python/Pylab script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;def symplectic_integrate_step(qvt0, accel, dt, coeffs):&#xA;    q,v,t = qvt0&#xA;    for ai,bi in coeffs.T:&#xA;        v += bi * accel(q,v,t) * dt&#xA;        q += ai * v * dt&#xA;        t += ai * dt&#xA;    return q,v,t&#xA;&#xA;def symplectic_integrate(qvt0, accel, t, coeffs):&#xA;    q = np.empty_like(t)&#xA;    v = np.empty_like(t)&#xA;    qvt = qvt0&#xA;    q[0] = qvt[0]&#xA;    v[0] = qvt[1]&#xA;    for i in xrange(1, len(t)):&#xA;        qvt = symplectic_integrate_step(qvt, accel, t[i]-t[i-1], coeffs)&#xA;        q[i] = qvt[0]&#xA;        v[i] = qvt[1]&#xA;    return q,v&#xA;&#xA;c = np.math.pow(2.0, 1.0/3.0)&#xA;ruth4 = np.array([[0.5, 0.5*(1.0-c), 0.5*(1.0-c), 0.5],&#xA;                  [0.0,         1.0,          -c, 1.0]]) / (2.0 - c)&#xA;ruth3 = np.array([[2.0/3.0, -2.0/3.0, 1.0], [7.0/24.0, 0.75, -1.0/24.0]])&#xA;leap2 = np.array([[0.5, 0.5], [0.0, 1.0]])&#xA;&#xA;accel = lambda q,v,t: -q&#xA;qvt0 = (1.0, 0.0, 0.0)&#xA;tmax = 2.0 * np.math.pi&#xA;N = 36&#xA;&#xA;fig, ax = plt.subplots(1, figsize=(6, 6))&#xA;ax.axis([-1.3, 1.3, -1.3, 1.3])&#xA;ax.set_aspect('equal')&#xA;ax.set_title(r&quot;Phase plot &lt;span class=&quot;math-container&quot;&gt;$(y(t),y'(t))$&lt;/span&gt;&quot;)&#xA;ax.grid(True)&#xA;t = np.linspace(0.0, tmax, 3*N+1)&#xA;q,v = symplectic_integrate(qvt0, accel, t, leap2)&#xA;ax.plot(q, v, label='leap2 (%d steps)' % (3*N), color='black')&#xA;t = np.linspace(0.0, tmax, N+1)&#xA;q,v = symplectic_integrate(qvt0, accel, t, ruth3)&#xA;ax.plot(q, v, label='ruth3 (%d steps)' % N, color='red')&#xA;q,v = symplectic_integrate(qvt0, accel, t, ruth4)&#xA;ax.plot(q, v, label='ruth4 (%d steps)' % N, color='blue')&#xA;ax.legend(loc='center')&#xA;fig.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have checked for simple errors already:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No Wikipedia typo. I have checked the references, in particular&#xA;(&lt;a href=&quot;https://dx.doi.org/10.1016%2F0167-2789%2890%2990019-L&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;,&#xA;&lt;a href=&quot;https://dx.doi.org/10.1016%2F0375-9601%2890%2990092-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;,&#xA;&lt;a href=&quot;https://dx.doi.org/10.1016%2F0021-9991%2891%2990299-Z&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;I have got the coefficient sequence right.&#xA;If you compare with Wikipedia's ordering, note that&#xA;sequencing of operator application works from right to left.&#xA;My numbering agrees with &lt;a href=&quot;https://dx.doi.org/10.1016%2F0021-9991%2891%2990299-Z&quot; rel=&quot;nofollow noreferrer&quot;&gt;Candy/Rozmus&lt;/a&gt;.&#xA;And if I try another ordering nevertheless, results get worse.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My suspicions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Wrong stepsize order: Maybe Ruth's 3rd-order scheme has somehow much smaller&#xA;implied constants, and if the step size were made really small,&#xA;then the 4th-order method would win?&#xA;But I have even tried &lt;span class=&quot;math-container&quot;&gt;$N=360$&lt;/span&gt;, and the 3rd-order method still is superior.&lt;/li&gt;&#xA;&lt;li&gt;Wrong test: Something special about my test lets Ruth's 3rd-order method&#xA;behave like a higher-order method?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7001" LastEditorUserId="6839" LastEditDate="2019-04-07T14:11:32.723" LastActivityDate="2019-04-10T11:15:51.373" Title="Test of 3rd-order vs 4th-order symplectic integrator with strange result" Tags="|python|time-integration|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20535" PostTypeId="2" ParentId="20533" CreationDate="2015-08-22T21:01:27.453" Score="9" Body="&lt;p&gt;Following &lt;em&gt;Kirill&lt;/em&gt;'s suggestion, I ran the test&#xA;with $N$ from a list of roughly geometrically increasing values,&#xA;and for each $N$ computed the error as&#xA;$$\epsilon(N) = \|\tilde z(2\pi) - \tilde z(0)\|_2&#xA;\quad\text{where}\quad \tilde z(t) = \left(\tilde{y}(t),\tilde y'(t)\right)$$&#xA;where $\tilde z$ represents the approximation obtained by numerical integration.&#xA;Here is the result in a log-log plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2bJVc.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2bJVc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;em&gt;ruth3&lt;/em&gt; indeed has the same order $4$ as &lt;em&gt;ruth4&lt;/em&gt; for that test case and&#xA;implied constants of only $\frac{1}{100}$ the magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interesting. I will have to investigate further, perhaps trying other tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, here are the additions to the Python script for the error plot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def int_error(qvt0, accel, qvt1, Ns, coeffs):&#xA;    e = np.empty((len(Ns),))&#xA;    for i,N in enumerate(Ns):&#xA;        t = np.linspace(qvt0[2], qvt1[2], N+1)&#xA;        q,v = symplectic_integrate(qvt0, accel, t, coeffs)&#xA;        e[i] = np.math.sqrt((q[-1]-qvt1[0])**2+(v[-1]-qvt1[1])**2)&#xA;    return e&#xA;&#xA;qvt1 = (1.0, 0.0, tmax)&#xA;Ns = [12,16,20,24,32,40,48,64,80,96,128,160,192,&#xA;      256,320,384,512,640,768,1024,1280,1536,2048,2560,3072]&#xA;&#xA;fig, ax = plt.subplots(1)&#xA;ax.set_xscale('log')&#xA;ax.set_xlabel(r&quot;$N$&quot;)&#xA;ax.set_yscale('log')&#xA;ax.set_ylabel(r&quot;$\|z(2\pi)-z(0)\|$&quot;)&#xA;ax.set_title(r&quot;Error after 1 period vs #steps&quot;)&#xA;ax.grid(True)&#xA;e = int_error(qvt0, accel, qvt1, Ns, leap2)&#xA;ax.plot(Ns, e, label='leap2', color='black')&#xA;e = int_error(qvt0, accel, qvt1, Ns, ruth3)&#xA;ax.plot(Ns, e, label='ruth3', color='red')&#xA;e = int_error(qvt0, accel, qvt1, Ns, ruth4)&#xA;ax.plot(Ns, e, label='ruth4', color='blue')&#xA;ax.legend(loc='upper right')&#xA;fig.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7001" LastActivityDate="2015-08-22T21:01:27.453" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="20536" PostTypeId="2" ParentId="20529" CreationDate="2015-08-23T11:26:42.733" Score="-3" Body="&lt;p&gt;This is a part of a much bigger problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Look at &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/abs/1502.07251&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/abs/1502.07251&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/ftp/arxiv/papers/1502/1502.07251.pdf&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/ftp/arxiv/papers/1502/1502.07251.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ever yours&#xA;Bob Eisenberg&#xA;Chairman emeritus&#xA;Molecular Biophysics&#xA;Rush University Medical Center&#xA;Chicago&lt;/p&gt;&#xA;" OwnerUserId="17322" LastActivityDate="2015-08-23T11:26:42.733" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20537" PostTypeId="2" ParentId="20277" CreationDate="2015-08-23T12:47:40.693" Score="6" Body="&lt;p&gt;I suggest taking a look at Scientific and Engineering C++: An Introduction with Advanced Techniques and Examples by &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0201533936&quot; rel=&quot;noreferrer&quot;&gt;Barton and Nackmann&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that this book was published in 1994 makes it appear to violate your criterion of &quot;modern techniques.&quot; However, Barton and Nackmann were on the cutting edge of what was possible with C++ templates at the time and the innovative techniques they devised for achieving good performance are still used in the latest C++ class libraries.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-08-23T12:47:40.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20538" PostTypeId="1" CreationDate="2015-08-23T18:33:48.070" Score="9" ViewCount="352" Body="&lt;p&gt;Multigrid introductions normally use a rectangular grid. Interpolation of values is then straight forward: Just interpolate linearly on the edge between two adjacent nodes of the coarse grid to find the value of the fine grid node on that edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an FEM application I have a grid which is &quot;topologically&quot; rectangular so that node connections are as on a rectangular grid. However, the nodes are not perfectly aligned on a grid but may travel small distances in order to better fit the geometry, while still maintaing connections as in a perfect rectangular grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mesh looks something like that: &lt;a href=&quot;http://groups.csail.mit.edu/graphics/classes/6.837/F98/TALecture/mesh.gif&quot; rel=&quot;nofollow&quot;&gt;Mesh example&lt;/a&gt;. You see: Connections are &quot;regular rectangular&quot;, but node positions not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can image several &quot;reasonable&quot; geometric interpolation schemes für such a setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general question is: Does multigrid require a perfectly aligned rectangular grid, or will it also work with the situation described above, as long as the interpolation is &quot;good&quot;? Or is it better to use algebraic multigrid in that case? (Which I don't prefer since it's not that intuitive as geometric multigrid.)&lt;/p&gt;&#xA;" OwnerUserId="8138" LastEditorUserId="8138" LastEditDate="2015-08-25T07:56:41.297" LastActivityDate="2015-08-26T02:38:11.873" Title="Multigrid on &quot;not perfectly rectangular&quot; grid" Tags="|multigrid|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20540" PostTypeId="1" CreationDate="2015-08-23T19:41:12.090" Score="3" ViewCount="189" Body="&lt;p&gt;I have been trying to solve the following nonlinear ordinary differential equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$-\Phi''-\frac{3}{r}\Phi'+\Phi-\frac{3}{2}\Phi^{2}+\frac{\alpha}{2}\Phi^{3}=0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Phi'(0)=0,\Phi(\infty)=0.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My solution is supposed to reproduce the following plots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ch3Iq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ch3Iq.jpg&quot; width=&quot;375&quot; height=&quot;300&quot;&gt;&lt;/a&gt;&#xA;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to produce the plots given above, I wrote the following Mathematica code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;α = 0.99;&#xA;Φlower = 0;&#xA;Φupper = 5;&#xA;For[counter = 0, counter &amp;lt;= 198, counter++,&#xA;    Φ0 = (Φlower + Φupper)/2;&#xA;    r0 = 0.00001;&#xA;    Φr0 = Φ0 + (1/16) (r0^2) (2 (Φ0) - 3 (Φ0^2) + α (Φ0^3));&#xA;    Φpr0 = (1/8) (r0) (2 (Φ0) - 3 (Φ0^2) + α (Φ0^3));&#xA;    diffeq = {-Φ''[r] - (3/r) Φ'[r] + Φ[r] - (3/2) (Φ[r]^2) + (α/2) (Φ[r]^3) == 0, Φ[r0] == Φr0, Φ'[r0] == Φpr0};&#xA;    sol = NDSolve[diffeq, Φ, {r, r0, 200}, Method -&amp;gt; &quot;ExplicitRungeKutta&quot;];&#xA;    Φtest = Φ[200] /. sol[[1]];&#xA;    Φupper = If[(Φtest &amp;lt; 0) ||   (Φtest &amp;gt; 1.2), Φ0, Φupper];&#xA;    Φlower = If[(Φtest &amp;lt; 1.2) &amp;amp;&amp;amp; (Φtest &amp;gt; 0),   Φ0, Φlower];&#xA;]&#xA;Plot[Evaluate[{Φ[r]} /. sol[[1]]], {r, 0, 200}, &#xA;     PlotRange -&amp;gt; All, PlotStyle -&amp;gt; Automatic] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the code, I used Taylor expansion at &lt;span class=&quot;math-container&quot;&gt;$r=0$&lt;/span&gt; due to the &lt;span class=&quot;math-container&quot;&gt;$-\frac{3}{r}\Phi'$&lt;/span&gt; term. Moreover, I used shooting method and continually bisected an initial interval from &lt;span class=&quot;math-container&quot;&gt;$\Phi_{\text{upper}}=5$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\Phi_{\text{lower}}=0$&lt;/span&gt; to obtain more and more precise values of &lt;span class=&quot;math-container&quot;&gt;$\Phi(0)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the code above, I was able to produce the plots for &lt;span class=&quot;math-container&quot;&gt;$\alpha = 0.50, 0.90, 0.95,0.96,0.97$&lt;/span&gt;. For example, my plot for &lt;span class=&quot;math-container&quot;&gt;$\alpha = 0.50$&lt;/span&gt; is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1jb45.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1jb45.jpg&quot; width=&quot;375&quot; height=&quot;300&quot;&gt;&lt;/a&gt;&#xA;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my plot for &lt;span class=&quot;math-container&quot;&gt;$\alpha = 0.99$&lt;/span&gt; does not converge to the required plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H0usT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H0usT.jpg&quot; width=&quot;375&quot; height=&quot;300&quot;&gt;&lt;/a&gt;&#xA;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you suggest how I might tackle this problem for &lt;span class=&quot;math-container&quot;&gt;$\alpha = 0.99$&lt;/span&gt;? Also, is there an explanation for the plots shooting upwards and oscillating after a prolonged asymptotic trend towards the positive &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;-axis?&lt;/p&gt;&#xA;" OwnerUserId="17326" LastEditorUserId="22284" LastEditDate="2019-02-17T08:43:44.437" LastActivityDate="2019-02-17T08:43:44.437" Title="Solving an ODE using shooting method" Tags="|ode|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20541" PostTypeId="1" CreationDate="2015-08-23T21:30:14.517" Score="1" ViewCount="45" Body="&lt;p&gt;As a general programming enthusiast and aspiring Bioinformatician student I have an intermediate understanding of computing (languages) as well as Java, and to a lesser extent C++. Having knowledge in a declarative as SQL and the intent to learn a functional as R; I found myself in need of interpreted/scripting language for quick data management tasks. Such would be used in conjunction with a compiled language for more complex algorithms. I am unbiased as I have yet to commit to mastering a single language; albeit I hope to make that decision soon and a major asset like scripting will play a role in that choice. Leading me to the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;When paring a compiled and scripting language, which languages&#xA;&lt;em&gt;fundamentally&lt;/em&gt; compliment the opposing features of the other to the greatest   degree while retaining the ability to communicate by call/implementing their use?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My intent in this case would be able to “plug” one into another to compensate an individual’s shortcomings (if not, at least conceptually) to holistically provide the greatest breath of benefit. Preference would of course be the design and features of the language as in contrast to more creative “hacking” solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(IE &lt;em&gt;Language II&lt;/em&gt; has the benefit… but by nature is poor/slow/lacks at… therefore &lt;em&gt;Language A&lt;/em&gt; would suit it well.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Considerations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Memory Management and the Compile-Time versus Execution-Time Tradeoff&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Data Types, Generics/Reflection, and Inheritance&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Platform/OS Compatibility (Including Source and Code Version)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Ease of use is not so much a consideration as in the case of learning and added utility, but is undeniably a factor in the case of collaboration. I would greatly appreciate any insight as well as the inclusion of anything else that I may have overlooked if applicable. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="17324" LastActivityDate="2015-08-23T21:30:14.517" Title="Manipulating/Extracting Data and Developing Methods - Language Choice" Tags="|optimization|algorithms|numerical-analysis|software|data-analysis|" AnswerCount="0" CommentCount="2" ClosedDate="2015-08-24T16:40:52.563" ContentLicense="CC BY-SA 3.0" />
  <row Id="20543" PostTypeId="2" ParentId="20538" CreationDate="2015-08-24T00:49:22.177" Score="4" Body="&lt;p&gt;Lets say that you have the following grid composed of rectangular elements:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YjjUU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YjjUU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you perform your interpolation assuming a normal structured rectangular grid then you will be introducing errors associated with this inaccurate interpolation. In other words when you restrict your residual vector and when you prolong your error vector there will be errors from the interpolation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if your grid is &quot;close&quot; to being a normal structured cartesian grid then this may work, at least at first, but I suspect one of two things will happen depending on how far off you grid is from being rectangular:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) You might find that the multigrid begins to converge at first. After all initially your error is large anyways and your &quot;approximate&quot; interpolation really just means that some nodes are slightly over represented while some are slightly under respresented. However you may find that the convergence stagnates as the solution becomes more accurate and the interpolation errors become more important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2)Another possibility is that the multigrid does end up converging, but not as fast as it should if you had used the correct interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically by being off with your interpolation you are weighting the importance of certain nodes inaccurately. For example in 2D if you are weighting a given node as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{bmatrix}&#xA;0.25 &amp;amp; 0.5 &amp;amp; 0.25 \\&#xA;0.5 &amp;amp; 1.0 &amp;amp; 0.5  \\&#xA;0.25 &amp;amp; 0.5 &amp;amp; 0.25&#xA;\end{bmatrix}  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;when in truth because your grid is not exactly cartesian it should be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{bmatrix}&#xA;0.25 &amp;amp; 0.55 &amp;amp; 0.25 \\&#xA;0.55 &amp;amp; 1.0 &amp;amp; 0.49 \\&#xA;0.28 &amp;amp; 0.52 &amp;amp; 0.30&#xA;\end{bmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then this will result in some error. Whether this error prevents convegence will likely depend on how far off your grid is from being cartesian. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While AMG is more difficult to understand/implement it sounds like it is the correct method for your grid. Applying geometric multigrid to an &quot;approximate&quot; rectangular grid may work, but I would guess that it is a band-aid solution at best. Hope this helps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;:&#xA;I think there may have been some confusion in my answer. I am not saying that geometric multigrid will only work with cartesian meshes, but rather that defining interpolation (and hence restriction) on cartesian meshes is easy whereas on non-structured meshes this may be difficult. For example consider the case of even a simple 2D domain with a triangular mesh. Refining this mesh is easy - at least conceptually - but how would you define an interpolation operator between the coarse and fine mesh? I prefer AMG simply because it performs more like a &quot;black box&quot; solver, i.e. doesnt need information on the underying mesh, however this is just my person bias/quirk. Geometric multigrid can work as long as you can provide accurate interpolation operators.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-08-26T02:38:11.873" LastActivityDate="2015-08-26T02:38:11.873" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20544" PostTypeId="1" AcceptedAnswerId="20546" CreationDate="2015-08-24T08:06:05.230" Score="0" ViewCount="235" Body="&lt;p&gt;I have a 3000 x 2301 matrix 'xfinfin' in MATLAB that I want to plot in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:2301&#xA;    ome = 0.4 + 0.001*(i-1);&#xA;    plot(ome,xfinfin(:,i),'k.');&#xA;    hold on&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this very quickly leads to MATLAB using 100% RAM and an error shows up in the MATLAB commnad line 'Out of memory'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, this algorithm plots a bifurcation diagram of omega vs x. How do I plot this without MATLAB runnning out of memory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9756" LastActivityDate="2015-08-24T08:54:26.473" Title="bifurcation diagram MATLAB runs out of memory" Tags="|matlab|memory-management|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20545" PostTypeId="2" ParentId="20529" CreationDate="2015-08-24T08:50:23.417" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm thinking pseudo-spectral method might solve the problem?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;A priori&lt;/em&gt;, I don't see why it would. As a continuous expression, the Butler-Volmer equation varies tremendously with small changes in potential. Any discretization inherits this behavior, so I think you're stuck with ill-conditioning no matter what. Maybe you could nondimensionalize the expressions, which would help somewhat. Nondimensionalization doesn't seem to be common practice in the battery simulation literature.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Another question is that is it perfectly fine to impose boundary conditions (specifically Neumann type) on PDE as a discretized algebraic equations without putting back into the governing equation with ghost point?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't see why you couldn't express boundary conditions as explicit algebraic equations, but it would behoove you to eliminate any algebraic equations if possible, since ODE solvers are generally faster than DAE solvers unless the DAE system has significantly sparser Jacobian (or, more generally, the linear systems being solved during the DAE solution algorithm are solved faster than the corresponding linear systems being solved during the ODE solution algorithm).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One last question: do the initial conditions need to satisfy the boundary conditions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, they don't, although when this happens, there is normally a rapid transient associated with the solution satisfying the boundary conditions immediately after time $t = 0$.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But the problem is quad precision implementation is more than 10x slower than 8 bytes double precision. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, this issue is the drawback in using quad precision.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Will it help if I reorder the Jacobian matrix so that it is banded (currently the linear solver is LAPACK with dense matrix)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Well, sure, it would be faster, but you're essentially solving a Poisson equation with a source term, so a preconditioned iterative solver would be faster.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Or I just need to learn PETSc and make use of the parallel feature and sparse solver?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Forget the parallel feature for a second. Mostly, you need to learn how to use PETSc for the sparse solvers. An algebraic multigrid preconditioned GMRES solver will probably work for the linear systems; a similar solver combination worked well for me when solving a Poisson equation with source terms from Butler-Volmer.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, how fined should it be so that the solution can considered fine-grid solution (and hence make error estimation from that)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Run a convergence study and refine your mesh repeatedly until the difference between solutions on successively refined meshes is small enough for you.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-08-24T08:50:23.417" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20546" PostTypeId="2" ParentId="20544" CreationDate="2015-08-24T08:54:26.473" Score="0" Body="&lt;p&gt;May be you should try stem, its a built-in plot function, I think it is what you need.&lt;/p&gt;&#xA;" OwnerUserId="16369" LastActivityDate="2015-08-24T08:54:26.473" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20548" PostTypeId="1" AcceptedAnswerId="20549" CreationDate="2015-08-24T22:26:52.610" Score="0" ViewCount="255" Body="&lt;p&gt;I have this type of question (ODE-constrained optimization) to solve:&#xA;$g(x,p)=0$ is the simulation, where $x$ is state variable and $p$ is parameters aimed to optimize;&#xA;$f(x)$ is the objective function (a measure of merit).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read that some people deal with this by discretize the ODE, but there seems another method called &quot;adjoint method&quot; works better.&#xA;Does anybody know any package on this or have code to share?&lt;/p&gt;&#xA;" OwnerUserId="17333" LastEditorUserId="713" LastEditDate="2015-08-24T22:39:07.290" LastActivityDate="2015-08-24T23:11:05.753" Title="adjoint method package for ODE(PDE)-constrained optimization" Tags="|optimization|pde|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20549" PostTypeId="2" ParentId="20548" CreationDate="2015-08-24T23:11:05.753" Score="3" Body="&lt;p&gt;The adjoint method is a particular way of finding derivatives of the solution to the PDE with respect to the parameters $p$.  In the adjoint method you end up solving a second PDE boundary value problem (the adjoint of the original problem) to find these derivatives. You can then get a gradient of $f$ with respect to $p$ by applying the chain rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be contrasted with the brute force alternative of solving the PDE boundary value problem for perturbed values of the parameters and then computing the derivatives by finite difference approximation.  If the vector $p$ has $m$ parameters, then finite differencing requires you to solve the original PDE $m+1$ times, while the adjoint method requires only one solution of the original PDE and one solution of its adjoint.  Thus the adjoint method can be vastly more efficient.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in practice both the original PDE and the adjoint equation have to be discretized by some kind of finite difference or finite element method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can in theory implement the adjoint method by finding the adjoint equation and then separately discretizing the original PDE and its adjoint problem.  In practice it is often better for numerical accuracy to find the adjoint of the discretized version of the original PDE boundary value problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some Automatic Differentiation (AD) tools support simultaneous development of solvers for the PDE boundary value problem and its adjoint.  The right tools for this will depend very much on your computing environment (programming language, parallelism, etc.)   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-08-24T23:11:05.753" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20550" PostTypeId="2" ParentId="20538" CreationDate="2015-08-24T23:25:21.147" Score="6" Body="&lt;p&gt;Multigrid doesn't need a Cartesian (rectangular), uniform grid. What it needs is that you can define a fine and a coarse level (possibly recursively, if you want to go from a two-level to a multi-level scheme), and that you can define interpolation operators between these levels. The easiest way to explain this is if you indeed have a Cartesian grid, but you can in fact start with any coarse mesh, refine it once, and just like this: you have a finer mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, it is easiest to think of multigrid-suitable meshes not as a fine mesh and how to find the coarser meshes, but to start with a coarse mesh from which you get the finer levels by uniform refinement (i.e., every quadrilateral is subdivided into four smaller ones). Since uniform refinement is always possible, this readily gives you a hierarchy. This is opposed to coarsening, which is not always possible if you are just given an particular mesh, and so makes defining a mesh hierarchy much more complicated. (Which is why people have come up with &lt;em&gt;algebraic&lt;/em&gt; multigrid methods to define the coarse levels based just on the matrix, without thinking of the underlying mesh from which it was created.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-08-24T23:25:21.147" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20551" PostTypeId="2" ParentId="20524" CreationDate="2015-08-24T23:31:27.477" Score="5" Body="&lt;p&gt;For the particular equation you are solving (called the &lt;em&gt;minimal surface equation&lt;/em&gt;), the functional you are trying to minimize is&#xA;$$&#xA;  J(u) = \int_\Omega \sqrt{1 + |\nabla u|^2} \; dx.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a derivation of the equations, as well as a discussion of solution approaches in lectures 31.5 and following here: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; . Of course there are many other sources for the same material as well. For example, nearly every book about the calculus of variations will have your example.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-08-24T23:31:27.477" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20552" PostTypeId="1" CreationDate="2015-08-25T05:54:54.617" Score="1" ViewCount="60" Body="&lt;p&gt;I got this problem thrown at me, unfortunately I lack context at the moment but I thought Maple or Mathematica would solve it anyways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a function $f$ over $x$ and $y$ such as this (Maple)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f := (y-1.298*exp(-1.311*x)-.3435*exp(.3602*x))*(x-3.059865611)*(y+1.875*exp(-.8384*x)+0.7033e-1*exp(.838*x))*(y+.5774*x+3.533)*(.1291*x^2+0.475e-1*y^2+.6927*x+.7080)*(y-.5773*x-3.533)*c;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have boundary condition so that c needs to be determined to satisfy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d^2f}{dx^2}+\frac{d^2f}{dy^2} = -2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solution := solve(diff(f, x, x)+diff(f, y, y) = -2, c)&#xA;fsubs := subs(c = solution, f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So far so good, now I want to integrate &#xA;$$&#xA;\int \left(\frac{df}{dy}-y\right) dx&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integrate(diff(fsubs, y)-y, x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this is where things no longer work. I may be impatient but I thought ten minutes would be enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="17335" LastEditorUserId="9667" LastEditDate="2015-08-25T17:16:54.860" LastActivityDate="2015-08-25T17:16:54.860" Title="CAS Problem with integrals" Tags="|mathematica|integration|maple|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20553" PostTypeId="2" ParentId="20013" CreationDate="2015-08-25T12:54:49.823" Score="1" Body="&lt;p&gt;If you could only use Java, our open source IF97 library provides partial derivatives, see: &lt;a href=&quot;http://www.hummeling.com/IF97&quot; rel=&quot;nofollow&quot;&gt;www.hummeling.com/IF97&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it can be used for proprietary software as well. &lt;/p&gt;&#xA;" OwnerUserId="17337" LastActivityDate="2015-08-25T12:54:49.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20554" PostTypeId="1" CreationDate="2015-08-25T14:27:26.027" Score="1" ViewCount="69" Body="&lt;p&gt;A while ago I came up with an algorithm which can be used to numerically solve optimal control problems, which basically came down to discretizing the control input $u(t)$ and interpolating this to get a continuous function. I tried to optimize the cost function for each discrete value by using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Golden_section_search&quot; rel=&quot;nofollow&quot;&gt;golden search method&lt;/a&gt; and because of the interpolation this would mean I only have to integrate this over a small section in time. The fact that each discrete value of $u(t)$ only affects a small number of elements also allows to use parallel computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the initial problem, I tried to optimize, I used linear interpolation, since the cost function was only a function of $u(t)$ and $u'(t)$. However recently I tried to expand on this, such that I would also be able to use higher order derivatives of $u(t)$. For linear interpolation you only need the discrete points at the start and end of an element, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_{i}(\xi) = u_{i} (1 - \xi) + u_{i+1} \xi,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $u(t)$ from $t_i$ to $t_{i+1}$ is defined as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u(t) = f_i\left(\frac{t - t_i}{t_{i+1} - t_i}\right).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to ensure that $u'(t)$ is continuous (such that second derivative exists) I had to set the slope at the end of an element equal to the slope at the beginning of the next element. I decided on defining this slope using the central difference method, which meant using two extra points, one before and one after the element,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f'_i(0) = \frac{u_{i+1} - u_{i-1}}{2},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f'_i(1) = \frac{u_{i+2} - u_{i}}{2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using these additional constraints yields the following function for an element,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_i(\xi) = u_{i-1} \frac{-\xi^3 + 2\xi^2 - \xi}{2} + u_i \frac{3\xi^3 - 5\xi^2 + 2}{2} + u_{i+1} \frac{-3\xi^3 + 4\xi^2 + \xi}{2} + u_{i+2} \frac{\xi^3 - \xi^2}{2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of the central difference method I also could have used backward or forward difference method, which would require one less point to be used, but I would think that interpolation ought to be symmetrical. These points can also used in the central difference method for the second derivative. Using this as a constraint for the initial and final second derivative yields the following function for an element,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_i(\xi) = u_{i-1} \frac{2\xi^5 - 5\xi^4 + 3\xi^3 + \xi^2 - \xi}{2} + u_i \frac{-3\xi^5 + 15\xi^4 - 9\xi^3 - 2 \xi^2 + 2}{2} + u_{i+1} \frac{3\xi^5 - 15\xi^4 + 9\xi^3 + \xi^2 + \xi}{2} + u_{i+2} \frac{-\xi^5 + 5\xi^4 - 3\xi^3}{2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However will this function have any disadvantages compared to the previous function? For example would the higher order polynomial make it harder to approximate sections of the cost function integral?&lt;/p&gt;&#xA;" OwnerUserId="7031" LastActivityDate="2015-08-25T14:27:26.027" Title="Comparison between of higher order interpolations" Tags="|interpolation|polynomials|optimal-control|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20555" PostTypeId="1" CreationDate="2015-08-25T21:59:01.217" Score="3" ViewCount="283" Body="&lt;p&gt;I want to stop immediately an MPI program, that has N processors and each processor runs a different algorithm (with different computational complexities).&#xA;Processors share information to each other using an unidirectional token ring topology. As soon as one processor ends it should notify its neighbor and all processors should stop immediately as soon as they receive the interruption.&#xA;Here is an sketch of the execution program at each processor:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1. Processor_i(array of n data) {&#xA;2.   while (stopping condition is not satisfied) {&#xA;3.     Run algorithm_i(data) for the ith processor&#xA;4.     Receive information from its source node (at most read 2 messages)&#xA;5.     if (the incomming message indicates interruption)&#xA;6.       break;&#xA;7.     Send its own information to its destiny node (maximum 2 messages)&#xA;8.   }&#xA;9.   Save local solution&#xA;10.  Send interruption message to its destiny node&#xA;11. }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that in the step 3, the computational complexity may be, for example, O(n^2), O(n log n) or O(n^3) for processors 1, 2 and 3, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that the messages are buffering by MPI and the interruption message is not reading immediately.&#xA;I was thinking in writing a file with the stopping instruction, but the filesystem is not shared.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I deal with this problem?&lt;/p&gt;&#xA;" OwnerUserId="17342" LastEditorUserId="17342" LastEditDate="2015-08-28T03:37:25.500" LastActivityDate="2015-08-28T03:37:25.500" Title="What is the best way to stop an asynchronous-heterogeneous MPI program?" Tags="|mpi|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20556" PostTypeId="1" AcceptedAnswerId="20557" CreationDate="2015-08-25T22:29:58.437" Score="1" ViewCount="809" Body="&lt;p&gt;I am trying to apply SVD to large sparse matrices. I already compared the performances of &lt;em&gt;Propack&lt;/em&gt; and &lt;em&gt;irlba&lt;/em&gt; to those of the matlab &lt;code&gt;svd&lt;/code&gt; and &lt;code&gt;svds&lt;/code&gt;. These two packages enhance significantly the computing time but I still have to handle the memory issue. Therefore, I am looking for a free matrix SVD algorithm which allows me to compute M largest singulars values on the fly without having to compute the entire matrix at the entry of the SVD.&#xA;I read about Incremental SVD &#xA;&lt;a href=&quot;http://www.math.fsu.edu/~cbaker/IncPACK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.fsu.edu/~cbaker/IncPACK/&lt;/a&gt; &#xA;and also saw this topic mentioning using Lanczos with Fast Block matrix-vector multiplication via FFTs instead of the SVD &#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/20128/svd-of-large-block-hankel-matrix&quot;&gt;SVD of large block-hankel matrix&lt;/a&gt; &#xA;but since I am not specialist I still can not see an easy way to compute the SVD without bearing the memory cost of computing the entire matrix. Can someone help me to find out?&lt;/p&gt;&#xA;" OwnerUserId="17343" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-10-11T12:40:04.263" Title="On the fly/matrix free SVD of large sparse matrix" Tags="|linear-algebra|matlab|matrix|sparse-matrix|svd|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20557" PostTypeId="2" ParentId="20556" CreationDate="2015-08-26T01:23:03.553" Score="1" Body="&lt;p&gt;Incremental SVD methods will only help you if you want to calculate a truncated (preferably low-rank) SVD. The memory requirements for these algorithms are more or less the ability to store a single column of your large sparse matrix plus the current truncated SVD. If you attempt to calculate the whole SVD with an incremental method, you won't save that much memory; however, if you only calculate the left singular vectors and singular values, you could conceivably save memory if your matrix has many more columns than rows.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-08-26T01:23:03.553" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20560" PostTypeId="1" CreationDate="2015-08-26T16:15:18.890" Score="2" ViewCount="340" Body="&lt;p&gt;My faculty advisor recommended that I take a look at the lattice Boltzmann method as an introduction to scientific computing and potentially an undergraduate honors thesis topic. I cooked up a some Java code (kind of unconventional, but I only know Java and Python right now), and I had a simple 2D simulation. Upon testing a 2D channel flow, I discovered many things wrong with the behavior.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The velocities at the nodes are not stable. In general they all approach zero. By 10,000th time steps, they are on the order of $10^{-15}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Despite the parabolic velocity profile one would expect, the magnitudes of the x-component of the velocity switches between positive and negative along the x-axis.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/cEttk.png&quot; alt=&quot;velocity x-component&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.cims.nyu.edu/~billbao/report930.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference&lt;/a&gt; I'm using explains the general algorithm clearly, but I have yet to find a clear description of the implementation of the no-slip boundary conditions. This is where I think my problem lies. I'm attempting to implement the on-grid boundary conditions (pgs. 3-4) and simulate the Poiseuille flow described in article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How exactly are the no-slip boundary conditions initialized?&lt;/em&gt; Right, now I'm operating under the assumption that you calculate the equilibrium functions based on initial density and velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be an even larger problem with my code if this isn't the problem. The source code is located &lt;a href=&quot;https://github.com/chemx003/Lattice-Boltzmann-2D/tree/master&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; if anyone is curious. Keep in mind this was the most intuitive approach to implementing the LBM for me and not necessarily to most efficient.&lt;/p&gt;&#xA;" OwnerUserId="17344" LastActivityDate="2015-08-26T16:15:18.890" Title="Implementation of no-slip boundary conditions in lattice Boltzmann method fluid simulation" Tags="|fluid-dynamics|boundary-conditions|computational-physics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20561" PostTypeId="2" ParentId="20252" CreationDate="2015-08-27T06:47:07.977" Score="2" Body="&lt;p&gt;You've measured the relative deviation from the initial energy, whereas &quot;energy drift&quot; in MD is only usefully measured as a rate of absolute deviation, and only after the system has equilibrated. (Neither of your simulations looks long enough to measure that equilibration has occurred, e.g. by measuring statistical properties of windows of the energy distribution.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your questions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) You can systematically vary parameters in the program, observe the effect on drift and infer things about the numerical properties of the implementation. (This is what you have already done with the size of the time step.) Other parameters of interest are the atom ordering (e.g. in a water box, if you accumulate all the O-O forces on the O atoms before any of the O-H forces on the O atoms then you will observe different numerical properties than if you accumulate molecule by molecule), simulation cell size (the numerical precision at which you can compute the distance between two atoms around (10,10,10) is rather lower than that you can compute for two atoms around (1,1,1)), and atom masses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) As above, such metrics are size dependent, and drifts are often reported per atom or degree of freedom. Whether drift matters depends on what you are observing. Usually you are observing the statistical properties of some observable, and if those are indistinguishable from correct, then your job is done. A useful approach can be seen here &lt;a href=&quot;https://scholar.google.com/scholar?oi=bibs&amp;amp;cluster=1069342445614725659&amp;amp;btnI=1&amp;amp;hl=en&quot; rel=&quot;nofollow&quot;&gt;https://scholar.google.com/scholar?oi=bibs&amp;amp;cluster=1069342445614725659&amp;amp;btnI=1&amp;amp;hl=en&lt;/a&gt;. Most people talk about drift because it's easy to measure and has a well-defined expected property, but if it walks and talks like a duck, it doesn't matter whether it has fleas unless you're going to use the down.&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2015-08-27T06:47:07.977" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20562" PostTypeId="1" AcceptedAnswerId="20564" CreationDate="2015-08-27T12:09:42.997" Score="1" ViewCount="369" Body="&lt;p&gt;When setting up a finite element system you have to use quadrature to calculate the integrals. I'm having trouble understanding what order rule to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know of some rules of thumb, for example with the 1D Hemholtz you end up with the matrix $M + S$, where $M_{ij} = (\phi_i, \phi_j)$ and $S_{ij} = (\phi_i', \phi_j')$. In this case if $\phi$ is a polynomial of degree $n$, then the integrals in $M$ will of degree $2n$ and the integrals in $S$ will be of degree $2(n-1)$. Using a Gauss rule of order $n$ is recommended here, which integrates $S$ exactly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using DG on the advection equation the &quot;stiffness&quot; matrix becomes $S_{ij} = (\phi_i', \phi_j)$. In this case I found that using an order $n$ Gauss  quadrature rule (which should still integrate $S$ exactly) gives me a singular matrix, and that I have to use a rule of order $n+1$ to get an invertible matrix. Is there some way to know beforehand what order rule to use, or it just determined experimentally?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: As per comments below, I'm using DG with Langrangian elements on a uniform grid to solve the time dependent advection equation $u_t + au_x = 0$. I have periodic boundary conditions on the domain $[0, 2\pi]$ and the initial condition $u(x,0) = \sin(x)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my first attempt coding DG myself, so I'm using central flux and backwards Euler for the time step. Using basis functions of degree $n$ and $\Delta t = h^{n+1}$, the solutions converge to the exact solution in the $L^2$ norm as $O(h^{n+1})$.&lt;/p&gt;&#xA;" OwnerUserId="9728" LastEditorUserId="1804" LastEditDate="2015-08-27T15:59:19.553" LastActivityDate="2015-08-27T16:54:15.507" Title="Quadrature order for finite elements and time dependent discontinuous Galerkin" Tags="|finite-element|numerical-analysis|quadrature|time-integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20563" PostTypeId="1" CreationDate="2015-08-27T14:08:42.827" Score="3" ViewCount="266" Body="&lt;p&gt;Is there an efficient way to measure similarity/distance between two sequences of ranked numbers/letters. The two sequences are of different length, and only have some elements in common?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I have three rank ordered numeric sequences like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sequence A: 1,2,3,4,5,6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sequence B: 2,3,4,5,6,7,8,9,10&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sequence C: 6,3,4,2,5,8,7,10,9&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I guess sequence A and B are more similar, since they have more numbers in common and the common numbers have same order in both sequences. Sequence A and C are less similar since they have less number in common and the common numbers have difference orders in each sequence. Damerau-Levenshtein Distance seems to be an ok option, it measures the similarity between two strings of letters, which considers insertion, deletion, substitution and adjacent transposition of letters. But I also want to consider non-adjacent transposition, i.e., more that two letters between two swapped letters. For example, in sequence C above, '2' and '6' are swapped, but they are not adjacent letters since there are '3' and '4' in between them. Damerau-Levenshtein Distance doesn't seem to take this into account.&lt;/p&gt;&#xA;" OwnerUserId="8191" LastActivityDate="2022-01-18T17:07:00.653" Title="similarity/distance measurement between two ranked sequence" Tags="|algorithms|data-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20564" PostTypeId="2" ParentId="20562" CreationDate="2015-08-27T15:55:37.870" Score="1" Body="&lt;p&gt;Since you have periodic boundary conditions, the system matrix (arising from the volume and flux terms) cannot have full rank (the solution of the stationary equation is unique only up to constants). Hence the well-posedness of the backward Euler step relies on the invertibility of the mass matrix (or at least that the two kernels have only a trivial intersection). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can fairly easily show that the mass matrix will be positive definite &lt;strong&gt;if the quadrature is exact&lt;/strong&gt;. As the product of two polynomials of order $n$ is $2n$, Gauß quadrature of order $n$ is not sufficient and you need at least order $n+1$ (as you have already noted). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The converse is not as obvious, but there's a bit of &quot;finite element folklore&quot; (at least in the engineering community) that each Gauß quadrature point adds $d$ (the space dimension) to the rank of the elemental mass matrix, and full rank is equal to the number of degrees of freedom in each element. In 1D, this is $n+1$, so you need $n+1$ points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, it is sufficient that the highest-order term (i.e., the one in the bilinear form involving the hightest-order derivatives) is integrated exactly (ignoring variable coefficients) and surface terms are integrated using one order higher, but if the highest-order term is itself not invertible (e.g., because of pure Neumann or periodic boundary conditions), you need to make sure the mass matrix is integrated exactly as well.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-08-27T16:54:15.507" LastActivityDate="2015-08-27T16:54:15.507" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20565" PostTypeId="2" ParentId="20442" CreationDate="2015-08-27T18:32:06.690" Score="0" Body="&lt;p&gt;Regarding item 2 of your question, assuming you have the sparsity pattern, it is often possible to compute the Jacobian with far fewer than N evaluations of the function. The idea is that because the Jacobian is sparse, you can evaluate more than a single column with a single function evaluation. Take a look at this paper for the details:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ecommons.cornell.edu/handle/1813/6342&quot; rel=&quot;nofollow&quot;&gt;Software for Estimating Sparse Jacobian Matrices&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-08-27T18:32:06.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20566" PostTypeId="1" CreationDate="2015-08-27T18:44:19.547" Score="2" ViewCount="846" Body="&lt;p&gt;I am trying to do a Molecular Dynamics Simulation of a complex fluid, confined between solid surfaces. I would like to find the flow rate as a function of fluid film thickness, $h$, for a plane poiseuille flow. (The flow rate is not expected to show an $h^3$ dependence, in general, if $h$ is small, in the order of a few nanometers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question I have is, how I can get an estimate for the simulation time required for the system to attain steady state, so the flow rate measurements can be made after such time. If we assumed continuum, we would have this equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \begin{align*}&#xA;\frac{\partial M}{\partial t} &#xA;= \nu \frac{\partial^2 M}{\partial y^2} &#xA;- \frac{1}{\rho} \frac{\partial P}{ \partial x}&#xA;\end{align*}$   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;to describe this unsteady Poiseuille flow($M$ being the Momentum, $\nu$ the kinematic viscosity and $P$ the pressure), with the applied pressure gradient being very high (since it is applied via adding high elementary body forces to the particles) to start with, when compared to the viscous diffusion term.&#xA;The pressure gradient term can be assumed to be constant along the direction of flow $x$ and in time, $t$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if we didn't have the pressure term, we would have a momentum diffusion equation from where we could estimate the time required for the unsteadiness to decay. But, from the above equation (or from any other starting point), it is not obvious to me how one can get an estimate for the simulation time required for steady state to be reached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for your patience and help!&lt;/p&gt;&#xA;" OwnerUserId="3558" LastActivityDate="2015-08-29T02:08:50.510" Title="Estimation of time taken to reach steady state in an MD simulation of Poiseuille flow" Tags="|fluid-dynamics|molecular-dynamics|microfluidics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20567" PostTypeId="2" ParentId="20566" CreationDate="2015-08-28T07:26:32.243" Score="1" Body="&lt;p&gt;I am not sure i understand your setup exactly, but why can't you just use the analytic expression of the laminar solution between parallel plates to get the flow rate? Of course assuming your Reynolds number is small enough so that the laminar solution is globally stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A different approach, which you can apply also to fluid flows in more complex geometries, could be to estimate the decay rate of the least stable eigenmode of the linear stability problem, (again assuming the system is linearly and globally stable) to get an estimate of the time it takes to reach a certain decay of the amplitude of the initial disturbance.&lt;/p&gt;&#xA;" OwnerUserId="6836" LastActivityDate="2015-08-28T07:26:32.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20568" PostTypeId="2" ParentId="20442" CreationDate="2015-08-28T07:28:17.987" Score="0" Body="&lt;p&gt;A cursory Google search revealed &lt;a href=&quot;http://camo.ici.ro/neculai/JACOBIAN.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; resource. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a set of FORTRAN subroutines for determining the Jacobian sparsity pattern of a general multivariate function. What you will want to look at is subroutine &lt;code&gt;RP01A&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C This subroutine automatically generates the sparsity pattern&#xA;C of a sparse Jacobian matrix with an arbitrary or band structure.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is a list of numerical examples (albeit for very small N) that you can use as a starting point for your own problem. I am not sure that their algorithm can handle something like $N = 10^5$, but I think it's worth giving it a shot.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2015-08-28T07:33:36.733" LastActivityDate="2015-08-28T07:33:36.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20569" PostTypeId="1" AcceptedAnswerId="20572" CreationDate="2015-08-28T11:19:12.307" Score="1" ViewCount="745" Body="&lt;p&gt;I am basically looking for a good open source C++ IDE with scientific and visualization libraries built in. Can any one suggest few projects?&lt;/p&gt;&#xA;" OwnerUserId="5003" LastActivityDate="2015-08-28T14:58:24.383" Title="Is there a single C++ IDE packed with necessary scientific libraries for computing purpose?" Tags="|parallel-computing|c++|hpc|libraries|visualization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20570" PostTypeId="1" CreationDate="2015-08-28T13:18:06.900" Score="4" ViewCount="1310" Body="&lt;p&gt;I understand that the constant in the Courant-Friedrichs-Lewy condition is defined as $\mathrm{CFL} = \frac{u \Delta t}{\Delta x}$, where $u$ is the principal coefficient. I came across this post:&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/2927/estimating-the-courant-number-for-the-navier-stokes-equations-under-differing-re&quot;&gt;Estimating the Courant number ...&lt;/a&gt;,&#xA;which basically suggests that the CFL number can be used to decide on the time step size to ensure stability and convergence. I have seen mentioned in many papers that the results were obtained with for instance $\mathrm{CFL} = 0.5$, which would suggest that the CFL is a constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But how is the CFL number defined if the coefficient $u$ changes with time?&lt;/strong&gt;&#xA;I am particularly interested in the solution of hyperbolic inviscid Euler equations, where $u = u$, $u+a$ or $u-a$, where the velocity $u$ varies with time and $a$ is the speed of sound.&lt;/p&gt;&#xA;" OwnerUserId="17362" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-08-31T10:12:28.100" Title="CFL condition for variable coefficients" Tags="|finite-difference|convergence|hyperbolic-pde|cfl|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20571" PostTypeId="1" CreationDate="2015-08-28T13:26:34.833" Score="2" ViewCount="103" Body="&lt;p&gt;I'm trying to develop an inferential procedure for a multivariate dependent  Markov process. Basically, the procedure could be considered as a non linear regression, with a known dependence structure among observation belonging to the same time-point and independent from the others. The non linear trend correspond to a the solution of a matrix differential equation. A more detailed description follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let define a matrix $\Theta$ of parameters $\theta_i$&#xA;\begin{array}{ccc}&#xA;\theta_1 &amp;amp; \theta_4 &amp;amp; \theta_7 \\&#xA;\theta_2 &amp;amp; \theta_5 &amp;amp; \theta_8 \\&#xA;\theta_3 &amp;amp; \theta_6 &amp;amp; \theta_9  \end{array}&#xA;each entry $\theta_i$ could be unconstrained or linearly constrained (both equality and inequality).&#xA;This matrix, multiply by a  given, fixed matrix $V$ and a vector of previous time point $\vec{X}$ observation, governs the derivative of a stochastic process.&#xA;As a consequence, in my objective function $\sum(\vec{Y}-f(\Theta,\vec{X}))^2$, I need to calculate the solutions of the matrix differential equation, $f(\Theta,\vec{X})$. To calculate the solutions, I need to calculate a matrix $A$, combining the entries of $\Theta$ according to $V$, calculate eigen decompositions of $A$ and finally the solution $\hat Y=f(\Theta,\vec{X})$ (exponential). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function is non-convex and has many local minima. Runnig some simulations studies, I verified that the global minimum is located at the true parameters values, and the objective is convex in the closed neighbourhood. &#xA;Until now, I used a constrained Gauss-Newton method to estimate parameters, and, given good initial values, it works. Relaxing some conditions, the method to calculate initial values is not good enough to guarantee the Gauss-Newton method to converge to global minimum (it get stucks in a closed local minimum). So, I'm now looking for a global optimizer. All my code is in R and RCplex and I have some experience with C++. Is Cplex able to solve non-convex,linearly constrained problem and flexible enough to allow me for solve the matrix differential equation? Are there any alternative program? Any suggestion?&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="17360" LastActivityDate="2015-08-28T13:26:34.833" Title="non convex, non linear optimization involving matrix differential equation solution" Tags="|eigenvalues|constrained-optimization|nonlinear-programming|stochastic|nonconvex|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20572" PostTypeId="2" ParentId="20569" CreationDate="2015-08-28T14:58:24.383" Score="2" Body="&lt;p&gt;Try Ceemple &lt;a href=&quot;http://www.ceemple.com&quot; rel=&quot;nofollow&quot;&gt;http://www.ceemple.com&lt;/a&gt;. It a IDE with some scientific computing package.&lt;/p&gt;&#xA;" OwnerUserId="16369" LastActivityDate="2015-08-28T14:58:24.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20573" PostTypeId="1" AcceptedAnswerId="20596" CreationDate="2015-08-28T15:06:16.003" Score="10" ViewCount="4201" Body="&lt;p&gt;I need to calculate $\log(\det (\mathbf M_i))$ where the $\mathbf M_i$'s  are large sparse matrices, which are real, symmetric and positive semi-definite. I hope to have between $10$ and $100$ of those. Physically, each $\mathbf M_i$ is the stiffness matrix of a large system, and if it helps, I know how to write it as $\mathbf M_i=\mathbf B_i^T\mathbf B_i$ (where $\mathbf B_i$ is &lt;strong&gt;not&lt;/strong&gt; a square matrix). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not a computational scientist, so I would prefer using written packages rather than implementing my own algorithms. I currently work in MATLAB and/or Mathematica, but I am also fluent in C++, and am willing to work in any other enviroment if this could help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I've tried so far is calculating the eigenvalues (using MATLAB's &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/eig.html&quot;&gt;&lt;code&gt;eig&lt;/code&gt;&lt;/a&gt;) and summing up their logs, but I suspect that there are other, more efficient ways to do so. For example, I stumbled upon &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S002437959710009X&quot;&gt;this paper&lt;/a&gt;, but I am not sure whether its algorithm is suitable in my case, and I could not find an implemented code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another complication is that it might happen that $\mathbf M_i$ is not invertible, i.e. $\det\mathbf M_i=0$, and in this case I'm interested in the product of all the non-zero eigenvalues (in these cases I know exactly how many zero eigenvalues are there).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="753" LastActivityDate="2021-04-27T14:48:53.307" Title="Calculating the log-determinant of a large sparse matrix" Tags="|matrix|sparse-matrix|computational-physics|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20574" PostTypeId="2" ParentId="20573" CreationDate="2015-08-28T15:49:40.657" Score="2" Body="&lt;p&gt;Algorithmically, a fast, stable route to the determinant is to first convert to symmetric tri-diagonal form using Householder reflections (which is the first step that eig would use on a symmetric matrix; this sub-operation may be visible in matlab). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The determinant of a symmetric tri-diagonal matrix can be found by working along the diagonal in a fairly straightforward way. It requires multiplies and adds at each step, though, so if the final value (or intermediate values) are too large or small to be represented without being in log form, you would need to guard the process against over- or underflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case with zero eigenvalues - I'm not sure offhand what will happen here, if you wind up with 'gaps' in the diagonal (zero diagonal element with 4 adjacent off-diagonal elements also zero') then you can process the submatrices on each side separately. But it might not work out so cleanly. For that case you probably want to just use the 'eig without eigenvectors' op.&lt;/p&gt;&#xA;" OwnerUserId="16933" LastActivityDate="2015-08-28T15:49:40.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20576" PostTypeId="1" CreationDate="2015-08-28T17:39:49.870" Score="1" ViewCount="180" Body="&lt;p&gt;I have this very simple ODE-contrained optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$h(x,x',p,t) = x'-A(p)x-b(p) = 0$, the constraint&lt;/li&gt;&#xA;&lt;li&gt;$g(x(0)) = x_0$, the initial condition with no parameters involved&lt;/li&gt;&#xA;&lt;li&gt;$F = \int (X-X_{obs})^2 dt$, the objective equation &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;According to adjoint method, I need to &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Integrate constraint equation: &#xA;$$x'=A(p)x+b(p)$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Integrate adjoint equation and reverse $\lambda$ in $t$: &#xA;$$\lambda'= A(p)^T \lambda-(X-X_{obs})$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate $\frac{dF}{dp}$: &#xA;$$\frac{dF}{dp} = \int \lambda^T \frac{\partial h}{\partial p} dt,$$&#xA;since &#xA;$$\frac{\partial f}{\partial p} = \frac{\partial g}{\partial p}=0$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But for parameters only show up in $\mathbf{b(p)}$ term, derivatives from adjoint method is inconsistent with derivatives estimated using $\frac{\partial F}{\partial p}$ directly while derivatives for other parameters seem OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m thinking that this inconsistency maybe due to the fact that parameters in $b(p)$ doesn’t affect the calculation of $\lambda$ directly, namely it doesn’t show up in the adjoint equation? But there is also the possibility that I did something wrong in coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any body have any similar experience? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="17333" LastEditorUserId="17333" LastEditDate="2015-08-28T20:06:55.703" LastActivityDate="2015-09-28T00:06:20.330" Title="Inaccurate Derivatives from Adjoint method for ODE-constrained problem" Tags="|ode|constrained-optimization|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="20579" PostTypeId="2" ParentId="20576" CreationDate="2015-08-28T20:11:58.617" Score="2" Body="&lt;p&gt;As discussed in the comments, the equations look fine to me, with the exception of a missing $\lambda$ in the adjoint equation. It should read,&#xA;$$\lambda'= A(p)^T\lambda-(X-X_{obs})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the missing $\lambda$ in the original post is a typo, otherwise you'd be adding a matrix to a vector so your code wouldn't even run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you haven't posted your formula for $\frac{\partial h}{\partial p}$, there could be a mistake there. It should be,&#xA;$$\frac{\partial h}{\partial p} = -A'(p)x - b'(p).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you forgot the $b'(p)$ term here, it might cause the issues you describe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, if the error is large there is probably a bug in the code somewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the error is small, then it could be a optimize-then-discretize vs. discretize-then-optimize issue. These operations only commute if you are discretizing with a Galerkin method. You can check if this is the problem by using very find discretizations in space and time, and seeing if the gradient error goes away.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2015-08-28T20:11:58.617" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20580" PostTypeId="2" ParentId="20576" CreationDate="2015-08-28T23:52:52.083" Score="0" Body="&lt;p&gt;I run the model forward with artificial parameters and then used my 'adjoint method' code to retrieve parameters. I found that as long as I assigned a good enough initial parameter values, the 'adjoint method' code indeed gives derivatives as expected.&#xA;So, the issue I encountered at first is due to, surprisingly, the inaccuracy of direct estimation of dF/dp with finite differences. This is of course because of the complexity/instability of my model equations, which is a very dangerous sign...&#xA;Anyway, issue cleared. Thanks everybody!&lt;/p&gt;&#xA;" OwnerUserId="17333" LastActivityDate="2015-08-28T23:52:52.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20581" PostTypeId="2" ParentId="20566" CreationDate="2015-08-29T02:00:10.197" Score="0" Body="&lt;p&gt;A good estimate can be obtained by looking at:&#xA;$ \begin{align*}&#xA;\frac{\partial M}{\partial t} &#xA;= \nu \frac{\partial^2 M}{\partial y^2} &#xA;\end{align*}$   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is a diffusion equation, that has a solution of the form: $M(t) = \frac{M_0}{\sqrt{4\pi \nu t}} \exp{(-\frac{y^2}{4 \nu t})}$. In theory, therefore, $h^2/\nu $, where $h$ is the fluid film thickness is a fair estimate for the time required to reach steady state. But, in practice, for complicated fluids, the MD simulation seems to be required to be run for much longer,for the fluctuations in the binned averages of the measured quantities to die down.&lt;/p&gt;&#xA;" OwnerUserId="3558" LastEditorUserId="3558" LastEditDate="2015-08-29T02:08:50.510" LastActivityDate="2015-08-29T02:08:50.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20582" PostTypeId="1" CreationDate="2015-08-29T09:23:49.020" Score="5" ViewCount="4746" Body="&lt;p&gt;As far as I know, for getting a unique solution to a PDE we should impose some boundary conditions to the PDE. &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines&quot; rel=&quot;noreferrer&quot;&gt;&quot;The number of required auxiliary conditions is determined by the highest order derivative in each independent variable. &quot;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are: In any numerical method, especially in finite element method, is it valid:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1-) to use more boundary conditions than the highest order derivative in each independent variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2-) to use less boundary conditions than the highest order derivative in each independent variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1-) &lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs00607-010-0138-4&quot; rel=&quot;noreferrer&quot;&gt;In this article&lt;/a&gt; and &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S1007570410003308&quot; rel=&quot;noreferrer&quot;&gt;in this article&lt;/a&gt;, while the highest order derivative of a spatial variable is in third order authors used two boundary conditions for that variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2-) &lt;a href=&quot;http://www.hindawi.com/journals/aaa/2014/819367/&quot; rel=&quot;noreferrer&quot;&gt;In this article&lt;/a&gt; while the highest order derivative of a spatial variable is in third order authors used four boundary conditions for that variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can increase the number of articles like this. &lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="4282" LastEditDate="2015-08-29T09:46:33.747" LastActivityDate="2015-08-29T09:46:33.747" Title="What should be the number of boundary conditions of a PDE" Tags="|pde|finite-element|boundary-conditions|" AnswerCount="0" CommentCount="7" ClosedDate="2015-09-08T02:20:05.250" ContentLicense="CC BY-SA 3.0" />
  <row Id="20583" PostTypeId="1" CreationDate="2015-08-29T11:37:15.500" Score="5" ViewCount="127" Body="&lt;p&gt;For a grid-based numerical simulation, I am looking for a load balancing/partitioning algorithm that not only distributes my grid elements, but also determines (approximates) their respective weights. Does anyone know of existing approaches for this problem or can give me some pointers into which kind of mathematical field I should be looking for a possible solution?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Details&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's say I have &lt;span class=&quot;math-container&quot;&gt;$N_e$&lt;/span&gt; grid elements &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt;, each with a certain computational weight &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;. The &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; are all within an order of magnitude, but I don't know their exact value a priori. The elements are now to be distributed among &lt;span class=&quot;math-container&quot;&gt;$N_p$&lt;/span&gt; processes, preserving their order. I can only measure the cumulative weight of all elements on one process but not their individual contributions. However, I can repartition the grid multiple times to get multiple measurements with different decompositions.&lt;/p&gt;&#xA;&lt;p&gt;What I am searching for is a method that lets me determine an estimate for the &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt; with a relatively low number of decompositions &lt;span class=&quot;math-container&quot;&gt;$N_d$&lt;/span&gt;. The numbers involved  here are in the range of &lt;span class=&quot;math-container&quot;&gt;$N_e = \mathcal{O}(10^9)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$N_p = \mathcal{O}(10^5)$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$N_d = \mathcal{O}(10^2)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1724" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-04-25T23:05:25.880" Title="Load balancing/partitioning with unknown weights" Tags="|domain-decomposition|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20584" PostTypeId="2" ParentId="20493" CreationDate="2015-08-29T18:50:56.147" Score="1" Body="&lt;p&gt;I found the mistake in my code...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's the mass. The author of the book used only the free particle mass while I (taken with some other things I need to calculate) forgot that I was using the mass in GaAs and since I didn't know what to expect for GaAs, I thought I was getting something wrong...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The good news is the algorithm is ok.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so, just use m0 for m in the code and you'll get a nice result, like in the book (page 100)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the code is not fault proof for other potentials though, it still needs testing and tweaking! (!!!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but it works for a one step square potential at least.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Gmkbv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gmkbv.png&quot; alt=&quot;the good result&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I tweaked my code a bit to work on every step potential, if you wish to find bound states:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm pinning in an extra potential function I used for testing.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%main.m&#xA;&#xA;close all&#xA;clear all&#xA;clc&#xA;format long e&#xA;&#xA;N=111;        &#xA;xMin = -5e-9; %nm  &#xA;xMax = -xMin;       &#xA;x = linspace(xMin,xMax, N);&#xA;&#xA;%U = square_potential_well(xMin, x, N);&#xA;&#xA;a = 2e-10;&#xA;b = 5e-10;&#xA;c = 8e-10;&#xA;d = 10e-10;&#xA;U = random_step_pot(x,a,b,c,d,N);&#xA;&#xA;[dx, Ustep] = step_width_and_potential(U, x);&#xA;&#xA;V = tmm(U, dx, Ustep, 200);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function U = square_potential_well(xMin, x, N)&#xA;&#xA;x1 = abs(xMin)/2;         % polovina sirine&#xA;U0 = 0; %dubina jame&#xA;&#xA;% inicijalizacija matrice koja predstavlja potencijalnu energiju jame&#xA;U = ones(N,1)*20;&#xA;&#xA;% kreiranje pravougaone jame sirine 2*x1&#xA;for cn = 1 : N&#xA;   if abs(x(cn)) &amp;lt;= x1, U(cn) = U0; end; %if;&#xA;end;&#xA;&#xA;% Test U&#xA;figure(1)&#xA;plot(x,U,'LineWidth',3);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function U = random_step_pot(x,a,b,c,d, N)&#xA;&#xA;U = ones(N,1);&#xA;Umax = 20;&#xA;U = U*Umax;&#xA;&#xA;&#xA;for cn = 1:N&#xA;    if x(cn) &amp;gt;= a &amp;amp;&amp;amp; x(cn)&amp;lt;=b&#xA;      U(cn) = 0;&#xA;    elseif x(cn)&amp;gt;b &amp;amp;&amp;amp; x(cn)&amp;lt;=c&#xA;      U(cn) = -8;&#xA;    elseif x(cn)&amp;gt;c &amp;amp;&amp;amp; x(cn)&amp;lt;=d&#xA;      U(cn) = 16;&#xA;    end&#xA;end &#xA;&#xA;figure(1)&#xA;plot(x,U, 'LineWidth', 2, 'g')&#xA;xlabel('x [nm]');&#xA;ylabel('U(x) [eV]'); &#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [dx, Ustep] = step_width_and_potential(U, x)&#xA;&#xA;    flag = [];&#xA;%    flag(1)=1;&#xA;    j = 1;&#xA;    for i = 1:length(U)-1&#xA;        if U(i) ~= U(i+1)&#xA;            flag(j) = i+1;&#xA;&#xA;            if i == length(U)-1 &#xA;                break;&#xA;            else j = j+1;&#xA;            end&#xA;        end&#xA;    end&#xA;&#xA;    % break nekad ne radi zato dodajem j vrednost flag koju i treba da ima na kraju&#xA;    j = length(flag);&#xA;&#xA;    dx = [];&#xA;    i = 1; %reinicijalizacija&#xA;    for k = 2:j&#xA;            dx(i) = x(flag(k)) - x(flag(k-1));&#xA;            i = i+1;&#xA;    end&#xA;&#xA;    Ustep = [];&#xA;    for i = 1:length(flag)-1 &#xA;        Ustep(i) = U(flag(i));&#xA;    end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function V = tmm(U, dx, Ustep, countE)&#xA;&#xA;    %konstante&#xA;    hbar = 1.054475e-34;     % Js&#xA;    q = 1.602e-19;         % C naelektrisanje cestice...&#xA;    m0 = 9.1093897e-31; %kg masa slobodnog elektrona&#xA;    m = 0.067*m0;    &#xA;&#xA;&#xA;    %k = [];&#xA;    %kappa = [];&#xA;    E = linspace(min(U) + 1e-3, U(1)-1e-3, countE); % countE je broj ispitivanih energija&#xA;    V = [];&#xA;    %Ms = zeros(2,2,length(E));&#xA;&#xA;% za slucaj pravougaone potencijalne jame za svaku energiju ce biti ispunjen samo prvi if uslov i jednom ce se prolaziti kroz for jer postoji samo jedan jama&#xA;    for j = 1:length(E)&#xA;        kappa = sqrt(2*m/hbar^2 * q * (U(1) - E(j)));&#xA;        M = eye(2);&#xA;        for i = 1:length(dx)&#xA;            if E(j) &amp;gt; Ustep(i)&#xA;                k = sqrt(2*m/hbar^2 * q * (E(j) - Ustep(i)));&#xA;                Ms = [cos(k*dx(i)), -(1/k)*sin(k*dx(i)); k*sin(k*dx(i)), cos(k*dx(i))];&#xA;            elseif E(j) &amp;lt; Ustep(i)&#xA;                k = sqrt(2*m/hbar^2 * q * (Ustep(i) - E(j)));&#xA;                Ms = [cosh(k*dx(i)), -(1/k)*sinh(k*dx(i)); -k*sinh(k*dx(i)), cosh(k*dx(i))];&#xA;            else &#xA;                k = 0;&#xA;                Ms = [1, -dx(i); 0, 1];&#xA;            end %if&#xA;&#xA;            M = M*Ms;&#xA;&#xA;        end&#xA;&#xA;&#xA;        t11 = 0.5 * (M(1, 1) + M(2, 2)) - 0.5*(kappa*M(1,2) + M(2,1)/kappa);&#xA;&#xA;&#xA;        V(j) = t11;       &#xA;    end&#xA;&#xA;    figure(2)&#xA;    plot(E, V, 'r', 'LineWidth', 2);&#xA;    xlabel('E[eV]');&#xA;    ylabel('t_{11}(E)');&#xA;    %axis([0,20,-2.5,1.5]);&#xA;    %print('t11', '-dpng');&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;just remember that if you wish to test the results from the book, work with the free particle mass $m_0$ by changing the value for mass $m$ in tmm.m&lt;/p&gt;&#xA;" OwnerUserId="17291" LastEditorUserId="17291" LastEditDate="2015-08-30T06:48:05.107" LastActivityDate="2015-08-30T06:48:05.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20585" PostTypeId="1" CreationDate="2015-08-29T19:23:02.680" Score="1" ViewCount="1361" Body="&lt;p&gt;I derived a &lt;a href=&quot;https://www.youtube.com/watch?v=Pmu4-fcN4xA&quot; rel=&quot;nofollow noreferrer&quot;&gt;bicopter&lt;/a&gt; dynamical model with two servos and two BLDC motors. And now am trying to simulate it using Matlab.&#xA;As base for simulation I used this &lt;a href=&quot;http://andrew.gibiansky.com/downloads/pdf/Quadcopter%20Dynamics,%20Simulation,%20and%20Control.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and &lt;a href=&quot;https://github.com/gibiansky/experiments/tree/master/quadcopter&quot; rel=&quot;nofollow noreferrer&quot;&gt;this code&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the obtained model always blowing up in case of asymmetrical control input signals. For example, with identical speed on both propellers and 5 degrees tilting angle for one of the thrust vectors, the system is blowing up in less the half second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is simulation plots:&#xA;dt = 0.0001, first 1300 points.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/aELUu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aELUu.png&quot; alt=&quot;sim 1300, dt = 0.0001&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This result obtained for constant 5 degree tilting angle for one of the motors and equal motor speeds. Angular acceleration just grows to the infinity in less than 1 second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Perform a simulation of a copter, from t = 0 through t = 1.&#xA;% As an argument, take a controller function. This function must accept&#xA;% a struct containing the physical parameters of the system and the current&#xA;% gyro readings. The controller may use the strust to store persistent state, and&#xA;% return this state as a second output value. If no controller is given,&#xA;% a simulation is run with some pre-determined inputs.&#xA;% The output of this function is a data struct.&#xA;&#xA;function result = simulate(controller, tstart, tend, dt)&#xA;    % Physical constants.&#xA;    g = 9.81;&#xA;    m = 0.95;&#xA;&#xA;    d = 0.35;&#xA;    h = 0.08;&#xA;&#xA;    I_body = diag([5e-3, 5e-3, 10e-3]);&#xA;    I_react=1e-3;&#xA;    I_gyro=1e-3;&#xA;&#xA;    I = struct('I_body', I_body, 'I_react', I_react, 'I_gyro', I_gyro);&#xA;&#xA;    kd = 0.25;&#xA;&#xA;    kt = 0.00001*9.8;&#xA;    kf = 0.0015*9.8;&#xA;&#xA;    % Simulation times, in seconds.&#xA;    if nargin &amp;lt; 4&#xA;        tstart = 0;&#xA;        tend = 1;&#xA;        dt = 0.0001;&#xA;    end&#xA;    ts = tstart:dt:tend;&#xA;&#xA;    % Number of points in the simulation.&#xA;    N = numel(ts);&#xA;&#xA;    % Output values, recorded as the simulation runs.&#xA;    xout = zeros(3, N);&#xA;    xdotout = zeros(3, N);&#xA;    thetaout = zeros(3, N);&#xA;    thetadotout = zeros(3, N);&#xA;&#xA;    inputout_beta = zeros(2, N);&#xA;    inputout_motor_w = zeros(2, N);&#xA;&#xA;    inputout_beta_speed = zeros(2, N);&#xA;    inputout_beta_acc = zeros(2, N);&#xA;&#xA;    inputout = struct('inputout_beta', inputout_beta, 'inputout_beta_speed', inputout_beta_speed ,...&#xA;        'inputout_beta_acc', inputout_beta_acc,'inputout_motor_w', inputout_motor_w);&#xA;&#xA;    % Struct given to the controller. Controller may store its persistent state in it.&#xA;    controller_params = struct('dt', dt, 'I', I, 'kf', kf, 'd', d, 'h', h, 'm', m, 'g', g);&#xA;&#xA;    % Initial system state.&#xA;    x = [0; 0; 10];&#xA;    xdot = zeros(3, 1);&#xA;    theta = zeros(3, 1);&#xA;&#xA;    % If we are running without a controller, do not disturb the system.&#xA;    if nargin == 0&#xA;        thetadot = zeros(3, 1);&#xA;    else&#xA;        % With a control, give a random deviation in the angular velocity.&#xA;        % Deviation is in degrees/sec.&#xA;        deviation = 300;&#xA;        thetadot = deg2rad(2 * deviation * rand(3, 1) - deviation);&#xA;    end&#xA;&#xA;    ind = 0;&#xA;    for t = ts&#xA;        ind = ind + 1;&#xA;&#xA;        % Get input from built-in input or controller.&#xA;        if nargin == 0&#xA;            i = input(t); %built-in generator&#xA;        else&#xA;            [i, controller_params] = controller(controller_params, thetadot);&#xA;        end&#xA;&#xA;                        % Store simulation state for output.&#xA;        % don't go through the ground&#xA;            if (x(3) &amp;lt; 0)&#xA;                x(3) = 0;&#xA;                theta(1) = 0; % avoid integration windup&#xA;                theta(2) = 0;&#xA;            end&#xA;&#xA;        %%%%% Compute forces, torques, and accelerations.&#xA;        omega = thetadot2omega(thetadot, theta); &#xA;        a = acceleration(i, theta, m, g, kf, kt, kd, xdot);&#xA;        omegadot = angular_acceleration(i, omega, I, d, h, kf, kt);&#xA;&#xA;        %%%% Advance system state. Euler method.&#xA;        omega = omega + dt * omegadot;&#xA;        thetadot = omega2thetadot(omega, theta);&#xA;&#xA;        theta = theta + dt * thetadot;&#xA;        xdot = xdot + dt * a;&#xA;        x = x + dt * xdot;&#xA;&#xA;        linacc(:, ind) = a;&#xA;        angacc(:, ind) = omegadot;a&#xA;        xout(:, ind) = x;&#xA;        xdotout(:, ind) = xdot;&#xA;        thetaout(:, ind) = theta;&#xA;        omegaout(:, ind) = omega;&#xA;        inputout(:, ind) = i;&#xA;&#xA;&#xA;    end&#xA;    % Put all simulation variables into an output struct.&#xA;    result = struct('linear_coord', xout, 'angular_coord', thetaout, 'linear_velocity', xdotout, ...&#xA;                    'angular_velocity',omegaout ,'angular_acc', angacc, 'linear_acc', linacc,'t', ts, 'dt', dt, 'input', inputout);&#xA;&#xA;end&#xA;&#xA;% Arbitrary test input.&#xA;function in = input(t)&#xA;inputout_beta= zeros(2, 1);&#xA;inputout_motor_w= zeros(2, 1);&#xA;&#xA;    inputout_beta(1) = 0.15;%-0.1*cos(t);&#xA;    inputout_beta(2) = 0;%0.1*cos(t);&#xA;    inputout_motor_w(1)= 3000*pi/30;%100*pi/30*cos(t) %rpm to radian per second &#xA;    inputout_motor_w(2)= 3000*pi/30;%101*pi/30*cos(t) %rpm to radian per second &#xA;&#xA;    inputout_beta_speed(1)=0;&#xA;    inputout_beta_speed(2)=inputout_beta_speed(1);&#xA;    inputout_beta_acc = zeros(2, 1);&#xA;&#xA;    in = struct('inputout_beta', inputout_beta, 'inputout_beta_speed', inputout_beta_speed ,...&#xA;        'inputout_beta_acc', inputout_beta_acc,'inputout_motor_w', inputout_motor_w);&#xA;end&#xA;&#xA;% Compute thrust given current inputs and thrust coefficient.&#xA;function F = force(w_motor, beta_motor, kf)&#xA;     w_l=w_motor(1);&#xA;     w_r=w_motor(2);&#xA;     beta_l=beta_motor(1);&#xA;     beta_r=beta_motor(2);&#xA;&#xA;    %Right power plant force&#xA;    f_r=kf*w_r;&#xA;    %Left power plant force&#xA;    f_l=kf*w_l;&#xA;    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;    %Left motor force rot mat&#xA;    R_Pl_Bm=[cos(beta_l), 0, sin(beta_l);&#xA;          0,          1,       0; &#xA;         -sin(beta_l),0, cos(beta_l)];&#xA;    %Right motor force rot mat    &#xA;    R_Pr_Bm=[cos(beta_r), 0, sin(beta_r);&#xA;          0,          1,       0; &#xA;         -sin(beta_r),0, cos(beta_r)] ;&#xA;    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;    F_l=[0; 0; f_l];&#xA;    F_r=[0; 0; f_r];&#xA;    F=R_Pl_Bm*F_l+R_Pr_Bm*F_r;&#xA;end&#xA;&#xA;% Compute torques, given current inputs, length, drag coefficient, and thrust coefficient.&#xA;function T_Bm = torque(input, d, h, k_f, k_d, I)&#xA;     w_l=input.inputout_motor_w(1);&#xA;     w_r=input.inputout_motor_w(2);&#xA;&#xA;     beta_l=input.inputout_beta(1);&#xA;     beta_r=input.inputout_beta(2);&#xA;&#xA;     beta_l_speed=input.inputout_beta_speed(1);&#xA;     beta_r_speed=input.inputout_beta_speed(2);&#xA;&#xA;     beta_l_acc=input.inputout_beta_acc(1);&#xA;     beta_r_acc=input.inputout_beta_acc(2);&#xA;&#xA;    %Right power plant torque&#xA;    tau_drag_r=k_d*w_r;&#xA;    %Left power plant torque&#xA;    tau_drag_l=k_d*w_l;&#xA;&#xA;    %Right power plant force&#xA;    f_r=k_f*w_r;&#xA;    %Left power plant force&#xA;    f_l=k_f*w_l;&#xA;&#xA;    F_l=[0; 0; f_l];&#xA;    F_r=[0; 0; f_r];&#xA;&#xA;    OlOb=[0;d;h];&#xA;    OrOb=[0;-d;h];&#xA;&#xA;    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;    %Left motor force rot mat&#xA;    R_Pl_Bm=[cos(beta_l), 0, sin(beta_l);&#xA;              0,          1,       0; &#xA;             -sin(beta_l),0, cos(beta_l)];&#xA;    %Right motor force rot mat    &#xA;    R_Pr_Bm=[cos(beta_r), 0, sin(beta_r);&#xA;             0,          1,       0; &#xA;             -sin(beta_r),0, cos(beta_r)] ;&#xA;    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;&#xA;     I_gyro = I.I_gyro;&#xA;     I_react = I.I_react;&#xA;&#xA;     % TORQUES&#xA;        %Gyroscopic moments, longitudinal tilting&#xA;        Mgyro_Pl_beta_l=[0;-I_gyro*w_l*beta_l_speed;0];&#xA;        Mgyro_Pr_beta_r=[0;I_gyro*w_r*beta_r_speed;0];&#xA;        %Gyroscopic momentin body frame&#xA;        Mgyro_Bm=R_Pl_Bm*Mgyro_Pl_beta_l+R_Pr_Bm*Mgyro_Pr_beta_r;&#xA;&#xA;        %Fan torques due aerodynamic drag of the propeller&#xA;        Mdrugr_Pr=[0;0;-tau_drag_r];&#xA;        Mdrugl_Pl=[0;0;tau_drag_l];&#xA;        %Drag torques in body frame&#xA;        Mdrug_Bm=R_Pl_Bm*Mdrugl_Pl+R_Pr_Bm*Mdrugr_Pr;&#xA;&#xA;        %Moment from trust vectoring in body frame&#xA;        M_thrv_Bm=cross((R_Pl_Bm*F_l),OlOb)+cross((R_Pr_Bm*F_r),OrOb);&#xA;&#xA;        %Adverse Inertial Reaction moment&#xA;        M_react_Pr=[0;-I_react*beta_r_acc;0];&#xA;        M_react_Pl=[0;-I_react*beta_l_acc;0];&#xA;        %Adverse Inertial Reaction moment, body frame&#xA;        M_react_Bm=R_Pr_Bm*M_react_Pr+R_Pl_Bm*M_react_Pl;&#xA;        %FULL TORQUE&#xA;        %Body frame&#xA;        T_Bm=Mgyro_Bm+Mdrug_Bm+M_thrv_Bm+M_react_Bm;&#xA;end&#xA;&#xA;% Compute acceleration in inertial reference frame&#xA;% Parameters:&#xA;%   g: gravity acceleration&#xA;%   m: mass of quadcopter&#xA;%   kf: thrust coefficient&#xA;%   kt: global drag coefficient&#xA;function A = acceleration(inputs, angles, m, g, kf, kt, kd, linear_velocity)&#xA;    F_g = [0; 0; -g];&#xA;    R_Bm_Wi = rotation(angles);&#xA;    F_Bm = force(inputs.inputout_motor_w, inputs.inputout_beta, kf);&#xA;    %Friction as a force proportional to the linear velocity in each direction.&#xA;    F_a = -kd * linear_velocity;&#xA;    %mV=R*F&#xA;    A = R_Bm_Wi * F_Bm * 1/m + F_g + F_a;&#xA;end&#xA;&#xA;% Compute angular acceleration in body frame&#xA;% Parameters:&#xA;%   I: inertia matrix&#xA;function omegad = angular_acceleration(inputs, omega, I, d, h, kf, kt)&#xA;    T = torque(inputs, d, h, kf, kt, I);&#xA;    omegad = I.I_body\(T - cross(omega, I.I_body * omega));&#xA;end&#xA;&#xA;% Convert derivatives of roll, pitch, yaw to omega.&#xA;function omega = thetadot2omega(thetadot, angles)&#xA;    phi = angles(1);&#xA;    theta = angles(2);&#xA;    psi = angles(3);&#xA;    W = [1,   0,                   -sin(phi);&#xA;         0    cos(phi),    cos(phi)*sin(phi);&#xA;         0   -sin(phi), cos(theta)/cos(phi)];                    &#xA;    omega = W * thetadot;&#xA;end&#xA;&#xA;% Convert omega to roll, pitch, yaw derivatives&#xA;function thetadot = omega2thetadot(omega, angles)&#xA;    phi = angles(1);&#xA;    theta = angles(2);&#xA;    psi = angles(3);&#xA;    W = [1,     sin(phi)*tan(theta),cos(phi)*tan(theta);&#xA;         0      cos(phi),                     -sin(phi);&#xA;         0      sin(phi)/cos(theta), cos(phi)/cos(phi)];&#xA;    thetadot = W * omega;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Newton-Euler equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\ddot{x}  m=[K_f  w_l  c(\beta_l) + K_f  w_r  c(\beta_r)]  [s(\phi)  s(\psi) + c(\phi)  c(\psi)  s(\theta)] + c(\psi)  c(\theta)  [K_f  w_l  s(\beta_l) + K_f  w_r  s(\beta_r)]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\ddot{y}  m =c(\theta)  s(\psi)  [K_f  w_l  s(\beta_l) + K_f  w_r  s(\beta_r)] - [K_f  w_l  c(\beta_l) + K_f  w_r  c(\beta_r)]  [c(\psi)  s(\phi) - c(\phi)  s(\psi)  s(\theta)]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\ddot{z}  m = c(\phi)  c(\theta)  (K_f  w_l  c(\beta_l) + K_f  w_r  c(\beta_r)) - s(\theta)  (K_f  w_l  s(\beta_l) + K_f  w_r  s(\beta_r)) - g  m&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_{xx}  \ddot{\phi}+\dot{\theta}  \dot{\psi}  (I_{zz}-I_{yy} )=K_t  w_l  s(\beta_l) - K_t  w_r  s(\beta_r) + K_f  d  w_l  c(\beta_l) - K_f  d  w_r  c(\beta_r)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_{yy}  \ddot{\theta}+\dot{\psi}  \dot{\phi}  (I_{xx}-I_{zz} )=I_g  \dot{\beta_r}  w_r - I_r  \ddot{\beta_r} - I_g  \dot{\beta_l}  w_l - I_r  \ddot{\beta_l} - K_f  h  w_l  s(\beta_l) - K_f  h  w_r  s(\beta_r)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I_{zz}  \ddot{\psi}+\dot{\theta}  \dot{\phi}  (I_{zz}-I_{yy} )=K_t  w_l  c(\beta_l) - K_t  w_r  c(\beta_r) - K_f  d  w_l  s(\beta_l) + K_f  d  w_r  s(\beta_r)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where:&#xA;$d,h$ - distances to CG&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$I$- Moments of inertia&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k_f,k_t$ - experimentally obtained coefficients&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9GDGr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9GDGr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Problem is independent from simulation step used. I also tried RK4 realization with the same result. &#xA;How can I correct this behavior?&#xA;What is the cause of the singularity?&lt;/p&gt;&#xA;" OwnerUserId="17372" LastEditorUserId="9667" LastEditDate="2015-08-29T23:40:57.160" LastActivityDate="2015-08-29T23:40:57.160" Title="BiCopter simulation in Matlab" Tags="|matlab|simulation|computational-physics|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20588" PostTypeId="1" AcceptedAnswerId="20589" CreationDate="2015-08-29T22:03:31.080" Score="0" ViewCount="211" Body="&lt;p&gt;Can one employ convex optimization for symmetric but not positive definite problems? I tried using MATLAB's quadprog() function to solve this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathrm{min}\frac{1}{2}\mathbf{x}^T\mathbf{H}\mathbf{x}+\mathbf{x}^{T}\mathbf{g}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathrm{s.t.} \mathbf{Ax}=\mathbf{b}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathbf{H}$ is a symmetric but not positive definite. I am attempting to solve Darcy's equation based on the &lt;a href=&quot;http://web.engr.illinois.edu/~amasud/Papers/Masud-Hughes-Darcy-CMAME.pdf&quot; rel=&quot;nofollow&quot;&gt;Variational Multi-Scale (VMS) formulation&lt;/a&gt;. I could rewrite the weak form such that the linear operator is symmetric, but even so quadprog will not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I &quot;normalized&quot; the minimization functional:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathrm{min}\frac{1}{2}\mathbf{x}^T\mathbf{H}^{T}\mathbf{H}\mathbf{x}+\mathbf{x}^{T}\mathbf{H}^T\mathbf{g}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathrm{s.t.} \mathbf{Ax}=\mathbf{b}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ensure that my Hessian is symmetric and positive definite, and quadprog does what i want it to. Why is this the case?&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2015-08-29T22:26:55.107" Title="Convex optimization for symmetric (but not positive definite) problems?" Tags="|matlab|finite-element|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20589" PostTypeId="2" ParentId="20588" CreationDate="2015-08-29T22:26:55.107" Score="2" Body="&lt;p&gt;A quadratic objective function in which the Hessian isn't positive semidefinite isn't a convex objective function and minimizing it isn't convex optimization.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-08-29T22:26:55.107" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20590" PostTypeId="1" CreationDate="2015-08-29T23:06:11.007" Score="4" ViewCount="636" Body="&lt;p&gt;I am analysing the stability of a series of 1D linear equations of the form &#xA;\begin{equation}&#xA;\frac{d}{dt} x = A x&#xA;\end{equation} &#xA;discretised using upwind and central finite volume methods, etc, with the explicit Euler time stepping scheme (to begin with). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have done so far is that I have calculated the eigenvalues of the system, and then plotted them. I then plotted a circle in the imaginary plane that contains all the eigenvalues, using the maximum and minimum eigenvalues. To comment on stability I then look at the radius of this circle and if it was less than one I concluded that stability is obtained. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be grateful if some could tell me if I am doing the right thing here, I have been through lots of books, etc and I got to this stage, but have not found something online or in books that verifies what I'm doing. &#xA;Thank you in advance! &lt;/p&gt;&#xA;" OwnerUserId="17362" LastEditorUserId="1804" LastEditDate="2015-08-30T10:12:07.730" LastActivityDate="2015-08-30T10:12:07.730" Title="linear stability analysis using spectral radius" Tags="|ode|eigenvalues|finite-volume|stability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20591" PostTypeId="2" ParentId="20277" CreationDate="2015-08-29T23:38:47.593" Score="4" Body="&lt;p&gt;The book &quot;Guide to Scientific Computing in C++&quot; by Pitt-Francis &amp;amp; Whiteley was written to answer exactly this sort of thing (use of STL, iterators etc.) it is available via &lt;a href=&quot;http://www.amazon.co.uk/Scientific-Computing-Undergraduate-Computer-Science/dp/1447127358&quot; rel=&quot;nofollow&quot; title=&quot;Amazon link&quot;&gt;Amazon&lt;/a&gt;, or as an e-Book &lt;a href=&quot;http://www.springer.com/us/book/9781447127352&quot; rel=&quot;nofollow&quot; title=&quot;direct from springer&quot;&gt;from publisher&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclosure - I work in the same &lt;a href=&quot;https://www.cs.ox.ac.uk/research/compbio/&quot; rel=&quot;nofollow&quot; title=&quot;Computational Biology Oxford&quot;&gt;research group&lt;/a&gt; as the authors, but still think it's a very good resource for this!&lt;/p&gt;&#xA;" OwnerUserId="4575" LastActivityDate="2015-08-29T23:38:47.593" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20594" PostTypeId="1" CreationDate="2015-08-30T07:49:06.600" Score="2" ViewCount="1577" Body="&lt;p&gt;I have a simple C code with many function calls, which I profiled using gprof. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%   cumulative   self              self     total           &#xA; time   seconds  seconds    calls   s/call   s/call  name    &#xA;71.93    907.85   907.85   201280     0.00     0.00  ForceCalcs&#xA; 2.61   1237.18    33.00     1258     0.03     1.00  Heap&#xA; 1.53   1256.45    19.28     1258     0.02     0.02  AdjustSP&#xA; 0.01   1261.61     0.07     1258     0.00     0.00  Boundary&#xA; 0.00   1261.61     0.00      100     0.00     0.00  Profile&#xA; 0.00   1261.61     0.00        2     0.00     0.00  MakeMap&#xA; 0.00   1261.61     0.00        1     0.00     0.18  Initialize&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have omitted some other function calls from the list just for clarity. This is why the percentages don't add up to 100.&#xA;To compare the results, I compiled it with -O3 command and profiled it again with gprof. The flat profile obtained is like :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Each sample counts as 0.01 seconds.&#xA;% (--- means nothing mentioned)&#xA;%   cumulative   self              self     total &#xA; time   seconds  seconds    calls  ms/call  ms/call  name&#xA;73.66    460.88   460.88   201280     2.29     2.29  ForceCalc&#xA; 1.65    620.85    10.30    ---       ---      ---   Heap&#xA; 0.47    623.80     2.95    ---       ---      ---   AdjustSP&#xA; 0.00    626.10     0.01     1258     0.01     0.01  Boundary&#xA; 0.00    626.10     0.00        1     0.00     0.00  MakeMap&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But now I cannot make much sense as to what optimizations have taken place in my code. For example&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The calls to Initialize() and profile() have no corresponding entry in later profile&lt;/li&gt;&#xA;&lt;li&gt;There are no mentioned calls or msec/call for Heap() and AdjustSP() in the later profile. (This is what I wanted to show by the --- )&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Can someone please explain the meaning of above 2 observations. I think it has got something to do with inline but am not sure.&lt;/p&gt;&#xA;" OwnerDisplayName="user16651" LastEditorUserId="7733" LastEditDate="2015-08-30T14:18:56.753" LastActivityDate="2022-06-03T15:51:16.793" Title="Profiling optimized C code using gprof" Tags="|optimization|c|profiling|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20595" PostTypeId="2" ParentId="20590" CreationDate="2015-08-30T07:53:39.027" Score="3" Body="&lt;p&gt;The eigenvalues of $A$ will only tell you something about the stability of the &lt;em&gt;continuous system&lt;/em&gt; (and here you will need that the real parts of all eigenvalues are less than zero, see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Stability_theory#Linear_autonomous_systems&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;).&#xA;To analyse the stability of the system discretised with forward Euler, you will have to look not only at the eigenvalues of $A$, but of $I + \Delta t A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using forward Euler, your time stepping becomes&#xA;\begin{equation}&#xA;u_{n+1} = u_{n} + \Delta t A u_{n} = \left( I + \Delta t A \right) u_{n}.&#xA;\end{equation}&#xA;Since you are looking at the spectral radius, let $A = U D U^{-1}$ be the eigenvalue decomposition of $A$. Then&#xA;\begin{equation}&#xA;u_{n+1} = u_n + \Delta t U D U^{-1} u_n \Rightarrow U^{-1} u_{n+1} = U^{-1} u_n + D U^{-1} u_n.&#xA;\end{equation}&#xA;Denote as $v := U^{-1} u$ your solution in transformed coordinates which satisfies&#xA;\begin{equation}&#xA;  v_{n+1} = v_{n} + \Delta t D v_n \Rightarrow v_{n+1} = \left( I + \Delta t D \right)^{n+1} v_0&#xA;\end{equation}&#xA;with $v_0 = U^{-1} u_0$ and $u_0$ your given initial solution. Since $D$ is a diagonal matrix, this corresponds to a set of scalar equations&#xA;\begin{equation}&#xA;v_{n+1}^{i} = ( 1 + \Delta t \lambda_{i} ) v_{0}^{i},&#xA;\end{equation}&#xA;where $v_{n+1}^{i}$ is the $i$-th component of $v$. Growth in terms of e.g. the maximum norm of $v$ can now be computed component wise since&#xA;\begin{equation}&#xA;\left| v_{n+1}^{i} \right| = \left| 1 + \Delta t \lambda_{i} \right| \left| v_{0}^{i} \right|.&#xA;\end{equation}&#xA;So if you want to ensure stability in the sense that $\left\| u_{n+1} \right\| \leq \left\| u_{0} \right\|$, you will have to make sure that&#xA;\begin{equation}&#xA;\left| 1 + \Delta t \lambda_{i} \right| \leq 1&#xA;\end{equation}&#xA;for every eigenvalue of $A$.&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="16921" LastEditDate="2015-08-30T08:05:29.473" LastActivityDate="2015-08-30T08:05:29.473" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20596" PostTypeId="2" ParentId="20573" CreationDate="2015-08-30T09:29:12.383" Score="5" Body="&lt;p&gt;Like Brian Borchers says in his comment, the determinant can be determined from the diagonal entries of the Cholesky factor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theory:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the Cholesky factorization &#xA;$$R^TR = S^TMS,$$ &#xA;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$R$ is an upper triangular matrix, and&lt;/li&gt;&#xA;&lt;li&gt;$S$ is a permutation matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then &#xA;\begin{align}&#xA;\text{det}(M) &amp;amp;= \text{det}(SR^TRS^T) \\&#xA;&amp;amp;= \text{det}(S) \text{det}(R^T) \text{det}(R) \text{det}(S^T) \\&#xA;&amp;amp;= \text{det}(R)^2.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $R$ is triangular, its eigenvalues are the diagonal entries, hence&#xA;\begin{align}&#xA;\text{log}~\text{det}(M) &amp;amp;= 2 \text{log}~\text{det}(R) \\&#xA;&amp;amp;= 2 \sum_i \text{log}(R_{ii}).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Code:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab code to do this is, for example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[R,~,S] = chol(M); &#xA;log_det_chol = sum(2*log(diag(R)));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In python, the log determinant can be computed from the sparse cholesky factorization using the package &lt;a href=&quot;https://pythonhosted.org/scikits.sparse/cholmod.html&quot; rel=&quot;nofollow&quot;&gt;scikits.sparse.cholmod&lt;/a&gt;. Paraphrasing from the linked page, the code to do this is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scikits.sparse.cholmod import cholesky&#xA;R = cholesky(M)&#xA;log_det_chol = R.logdet()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you only want to consider the nonzero eigenvalues, you can just sum over the nonzero entries of the diagonal of $R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Test case:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tested this against Matlab's build in det() function on the 2D finite difference Laplace operator with Dirichlet B.C.'s, and got exact agreement for matrices up to around 400-by-400, after which point using matlab's det() yielded Inf, whereas the cholesky method had no problem going up to million-by-million scale on my laptop. Here is the code, for reference:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 20; % M is n^2-by-n^2. Matlab's det() returns Inf for n=25+&#xA;M = gallery('poisson',n); &#xA;[R,~,S] = chol(M); &#xA;log_det_chol = sum(2*log(diag(R))), &#xA;log_det = log(det(M))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Memory considerations:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this method is based on the Cholesky factorization, it inherits the speed and stability from that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large problems, the bottleneck using this method is probably going to be memory. In particular, using a Cholesky factorization with good pivoting (Matlab's is good), you can expect to need $O(s^2)$ memory, where $s$ is the number of degrees of freedom in the smallest &quot;separator front&quot; that partitions the degrees of freedom into distinct pieces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in a 2D problem with an n-by-n grid, the 1D separator front partitioning the domain into two pieces has $s = O(n)$ degrees of freedom. Thus amount of memory required is $O(n^2)$, the same order of magnitude as storing the right hand side. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 3D n-by-n-by-n problem has a 2D separator front with $s = O(n^2)$ degrees of freedom, necessitating $O(n^4)$ memory.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2015-09-07T19:45:10.843" LastActivityDate="2015-09-07T19:45:10.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20597" PostTypeId="2" ParentId="5464" CreationDate="2015-08-30T11:48:21.460" Score="1" Body="&lt;p&gt;I implemented formula given by @Max Hutchinson with CUBlas and Cuda Thrust and compared with online co variance calculation tools. It seems mine producing good results. The code below planned to QDA Bayes. So matrix given may contain more than one class. So multiple co variance matrices is calculated.  I hope it will be useful for someone.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//! Calculates one or more than one coVarianceMatrix given data.&#xA;//  There can be many classes since many covariance matrixes.&#xA;/*!&#xA;    \param inMatrix This vector contains matrix data in major storage. &#xA;    Forexample if inMatrix=[1 2 3 4 5 6] and trialSizes=[2] this means matrix we will work on a matrix like :&#xA;        |1 4 |&#xA;        |2 5 |&#xA;        |3 6 | -&amp;gt; 2 Trials, 3 Features. Columns contains feature rows contains trials (samples)&#xA;    \param trialSizes There can be many classes since many covariance matrixes. Samples from all classes will be given with inMatrix.&#xA;    But we need to know how many trials(samples) we have for each class. &#xA;    For example if inMatrix=[1 2 3 4 5 6 7 8 9 10 11 12] and trialSizes=[2,2] &#xA;    this means matrix we will work on a matrix like :&#xA;        |1 4 |  |7 10 |&#xA;        |2 5 |  |8 11 |&#xA;        |3 6 |  |9 12 |  --&amp;gt; Total number of trials(samples which is total rowCount) 2 + 2 = 4 , &#xA;                             So colSize = inMatrix.size()/4 = 3(feature vector size)&#xA;                         --&amp;gt; There is two element in trialSize vec so each vector has to samples&#xA;*/&#xA;void multiQDACovianceCalculator(std::vector&amp;lt;float&amp;gt;&amp;amp; inMatrix, std::vector&amp;lt;int&amp;gt;&amp;amp; trialSizes)&#xA;{&#xA;    cublasHandle_t handle; // CUBLAS context&#xA;    int classCount = trialSizes.size();&#xA;    int rowSize = std::accumulate(trialSizes.begin(), trialSizes.end(), 0);&#xA;    int dimensionSize = inMatrix.size() / rowSize;&#xA;    float alpha = 1.0f;&#xA;    float beta = 0.0f; // bet =1&#xA;&#xA;    thrust::device_vector&amp;lt;float&amp;gt; d_cov1(dimensionSize * dimensionSize);&#xA;    thrust::device_vector&amp;lt;float&amp;gt; d_cov2(dimensionSize * dimensionSize);&#xA;    thrust::device_vector&amp;lt;float&amp;gt; d_covResult(dimensionSize * dimensionSize);&#xA;&#xA;    thrust::device_vector&amp;lt;float&amp;gt; d_wholeMatrix(inMatrix);&#xA;    thrust::device_vector&amp;lt;float&amp;gt; d_meansVec(dimensionSize); // rowVec of means of trials&#xA;    float *meanVecPtr = thrust::raw_pointer_cast(d_meansVec.data());&#xA;    float *device2DMatrixPtr = thrust::raw_pointer_cast(d_wholeMatrix.data());&#xA;    auto maxTrialNumber = *std::max_element(trialSizes.begin(), trialSizes.end());&#xA;    thrust::device_vector&amp;lt;float&amp;gt; deviceVector(maxTrialNumber, 1.0f);&#xA;&#xA;    cublasCreate(&amp;amp;handle);&#xA;    // Inside of for loop  one covariance matrix calculated each time&#xA;    for (int i = 0; i &amp;lt; trialSizes.size(); i++)&#xA;    {&#xA;        // X*transpose(X) / N&#xA;        alpha = 1.0f / trialSizes[i];&#xA;        cublasSgemm(handle, CUBLAS_OP_N, CUBLAS_OP_T, dimensionSize, dimensionSize, trialSizes[i], &amp;amp;alpha,&#xA;            device2DMatrixPtr, dimensionSize, device2DMatrixPtr, dimensionSize, &amp;amp;beta,&#xA;            thrust::raw_pointer_cast(d_cov1.data()), dimensionSize);&#xA;&#xA;        // Mean vector of each column&#xA;        alpha = 1.0f;&#xA;        cublasSgemv(handle, CUBLAS_OP_N, dimensionSize, trialSizes[i], &amp;amp;alpha, device2DMatrixPtr,&#xA;            dimensionSize, thrust::raw_pointer_cast(deviceVector.data()), 1, &amp;amp;beta, meanVecPtr, 1);&#xA;&#xA;        // MeanVec * transpose(MeanVec) / N*N&#xA;        alpha = 1.0f / (trialSizes[i] * trialSizes[i]);&#xA;        cublasSgemm(handle, CUBLAS_OP_T, CUBLAS_OP_N, dimensionSize, dimensionSize, 1, &amp;amp;alpha,&#xA;            meanVecPtr, 1, meanVecPtr, 1, &amp;amp;beta,&#xA;            thrust::raw_pointer_cast(d_cov2.data()), dimensionSize);&#xA;&#xA;        alpha = 1.0f;&#xA;        beta = -1.0f;&#xA;        //  (X*transpose(X) / N) -  (MeanVec * transpose(MeanVec) / N*N)&#xA;        cublasSgeam(handle, CUBLAS_OP_N, CUBLAS_OP_N, dimensionSize, dimensionSize, &amp;amp;alpha,&#xA;            thrust::raw_pointer_cast(d_cov1.data()), dimensionSize, &amp;amp;beta, thrust::raw_pointer_cast(d_cov2.data()), &#xA;            dimensionSize, thrust::raw_pointer_cast(d_covResult.data()), dimensionSize);&#xA;&#xA;        // Go to other class and calculate its covarianceMatrix&#xA;        device2DMatrixPtr += trialSizes[i] * dimensionSize;&#xA;    }&#xA;    printVector(d_covResult);&#xA;    cublasDestroy(handle);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17332" LastActivityDate="2015-08-30T11:48:21.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20598" PostTypeId="2" ParentId="20500" CreationDate="2015-08-30T11:49:59.707" Score="12" Body="&lt;p&gt;&lt;strong&gt;Making a mesh watertight&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several efficient algorithms to make a mesh watertight, historically, in Stanford, the pionneers of 3D Scanning developed the Zipper algorithm:&#xA;&lt;a href=&quot;https://graphics.stanford.edu/papers/zipper/&quot; rel=&quot;noreferrer&quot;&gt;https://graphics.stanford.edu/papers/zipper/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then many algorithms were developed, you may refer for instance to the following website maintained by friends of mine:&#xA;&lt;a href=&quot;http://www.meshrepair.org/&quot; rel=&quot;noreferrer&quot;&gt;http://www.meshrepair.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has references to several tutorial materials that we wrote with colleagues. See in particular the 'Model repair' chapter of the book that I co-authored with them &lt;a href=&quot;http://www.pmp-book.org/&quot; rel=&quot;noreferrer&quot;&gt;Polygon Mesh Processing&lt;/a&gt; [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On meshrepair.org, you will also find references to implementations (including the Graphite software that I am developing). The latest version of my Graphite software (3.x) has implementation of several mesh repair algorithm. It can be downloaded from:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gforge.inria.fr/frs/?group_id=1465&quot; rel=&quot;noreferrer&quot;&gt;https://gforge.inria.fr/frs/?group_id=1465&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sources of real world examples&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many examples of scanned meshes are available from the Aim at Shape repository:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://visionair.ge.imati.cnr.it/ontologies/shapes/viewmodels.jsp&quot; rel=&quot;noreferrer&quot;&gt;http://visionair.ge.imati.cnr.it/ontologies/shapes/viewmodels.jsp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of the models in this website are available in both raw format and post-processed watertight format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;[1]&lt;/strong&gt; Botsch, Mario, Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Lévy. Polygon mesh processing. CRC press, 2010.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2015-10-16T08:33:50.273" LastActivityDate="2015-10-16T08:33:50.273" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20599" PostTypeId="1" AcceptedAnswerId="20652" CreationDate="2015-08-30T20:44:48.217" Score="5" ViewCount="826" Body="&lt;p&gt;I have the following optimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;&amp;amp;\min_{ X_{1}, \dots,X_{k}  }  \max_{ \theta, \phi } \left|P_{d}(\theta,\phi) - \sum_{k=1}^K \operatorname{Tr}(a_{k}(\theta,\phi)a_{k}^{H}(\theta,\phi)X_{k})\right|\\ &#xA;&amp;amp;\text{subject to}   \sum_{k=1}^K \operatorname{diag}(X_{k}) = \frac{E}{M_{t}N_{t}-(K-1)} 1_{M_{t}N_{t}\times 1}\\&#xA;&amp;amp;X_{k} \succeq 0, k = 1,2, \dots,K&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am pretty sure that this optimization problem is convex (a SDP) so I can use &lt;code&gt;cvx&lt;/code&gt; to solve it. However, I am not sure how I can express it in &lt;code&gt;cvx&lt;/code&gt; language. I am not sure how can sweep through both $\theta$ and $\phi$ to solve the optimization problem. They are defined as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;theta = linspace(0, 180, 181);&#xA;phi = linspace(0, 360, 361);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$K = 5$, $M_{t}=5$, $N_{t} = 5$ and $E$ is some constant.&lt;/li&gt;&#xA;&lt;li&gt;$P_{d}(\theta,\phi)$ is a $181\times361$ matrix ($\theta$ for rows, $\phi$ for columns)&lt;/li&gt;&#xA;&lt;li&gt;$a_{k}(\theta,\phi)$ is a $25\times1$ column vector and $(\cdot)^H$ denotes the Hermitian transpose.&lt;/li&gt;&#xA;&lt;li&gt;$X_{k}$ is the complex optimization variable which is a $25\times25$ matrix. &lt;/li&gt;&#xA;&lt;li&gt;$1_{M_{t}N_{t}\times 1}$ is a vector of ones with given dimensions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I tried to define $a_{k}(\theta,\phi)$  as a 4D vector with $(181, 361, K, 25)$ and store the values for every instance of $\theta$ and $\phi$ but then I couldn't figure out how I can implement $$\sum_{k=1}^K \operatorname{Tr}(a_{k}(\theta,\phi)a_{k}^{H}(\theta,\phi)X_{k})$$&#xA;so that it will generate a $181\times 361$ matrix. &lt;/p&gt;&#xA;" OwnerUserId="17378" LastEditorUserId="17378" LastEditDate="2015-08-31T06:06:55.190" LastActivityDate="2015-09-05T04:33:41.600" Title="Semidefinite Programming Using CVX in Matlab" Tags="|optimization|matlab|cvx|semidefinite-programming|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20600" PostTypeId="1" CreationDate="2015-08-31T01:13:24.957" Score="0" ViewCount="97" Body="&lt;p&gt;Initially I modeled my objective function as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\arg \min \operatorname{Var}(f(x),g(x)) + \operatorname{Var}(c(x),d(x)) + \cdots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$, $g$, $c$, $x$ are linear functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be able to solve the problem with linear solvers I have change the problem as follows ($\operatorname{abs}$ stands for absolute value):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\arg \min\; \operatorname{abs}(f(x)-g(x)) + \operatorname{abs}(c(x)-d(x)) + \cdots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this form correct and has the same meaning as the first model?&lt;/p&gt;&#xA;" OwnerUserId="16535" LastEditorUserId="16535" LastEditDate="2015-08-31T18:07:49.500" LastActivityDate="2015-08-31T18:07:49.500" Title="Can Variance be replaced by absolute value in this optimization problem" Tags="|optimization|linear-programming|quadratic-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20602" PostTypeId="2" ParentId="20570" CreationDate="2015-08-31T07:07:16.767" Score="5" Body="&lt;p&gt;The CFL number (or simply Courant number) is defined locally:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nu = u(x,t) \frac{\Delta t}{\Delta x}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The velocity may vary in time and space and the grid may be non-uniform (in $x$ and/or $t$).  So the CFL number can be different at each point in time and space.&#xA;A necessary condition for convergence of a consistent method is that the CFL condition be satisfied at each point in time and space.  Therefore, to choose the timestep one must enforce a bound on the maximum CFL number (maximum in space).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-08-31T10:12:28.100" LastActivityDate="2015-08-31T10:12:28.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20603" PostTypeId="2" ParentId="20594" CreationDate="2015-08-31T07:07:38.607" Score="3" Body="&lt;p&gt;Two things that may help understanding what's going on:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1) the output of gprof is sometimes hard to parse&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is a tool that helps obtaining a graphical representation called gprof2dot (used as follows in combination with dot):&lt;/p&gt;&#xA;&lt;p&gt;gprof executable | gprof2dot.py | dot -T pdf &amp;gt; profile.pdf&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;gprof2dot can be downloaded from&#xA;&lt;a href=&quot;https://github.com/jrfonseca/gprof2dot&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jrfonseca/gprof2dot&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;dot can be downloaded from: &lt;a href=&quot;http://www.graphviz.org/Download.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.graphviz.org/Download.php&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;2) use a system-wide profiler in conjunction with gprof&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For instance, under Linux, you can use zoom: &lt;a href=&quot;http://freshmeat.sourceforge.net/projects/zoomprofiler&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://freshmeat.sourceforge.net/projects/zoomprofiler&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is freely available.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="1262" LastEditDate="2022-06-03T15:51:16.793" LastActivityDate="2022-06-03T15:51:16.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20604" PostTypeId="2" ParentId="20600" CreationDate="2015-08-31T15:09:26.297" Score="4" Body="&lt;p&gt;Without knowing much about your question it is hard to answer more specifically. So first and foremost - no. These are not equivalent. Minimizing variangce will generally make you converge to some kind of sample &lt;strong&gt;mean&lt;/strong&gt;, whereas minimizing absolute deviation would (generally) make you find some kind of sample &lt;strong&gt;median&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A generally favourable property of sample means is fast convergence (at least, when compared to median). A good property of sample medians is robustness - insensitivity to outliers.&lt;/p&gt;&#xA;" OwnerUserId="11717" LastActivityDate="2015-08-31T15:09:26.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20607" PostTypeId="1" CreationDate="2015-08-31T18:40:48.757" Score="1" ViewCount="609" Body="&lt;p&gt;I have a mixed integer quadratic problem and my objective function is as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\arg \min \operatorname{Var}(f(x),g(x)) + \operatorname{Var}(c(x),d(x)) + \cdots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$, $g$, $c$ $d$ are linear functions and $Var$ indicates variance and $x$ is a vector of 3000 binary variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to solve the problem with mixed integer quadratic solver in gurobi I have to define it in the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ x^T Q x + q^T x $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as the problem is defined in a high level format and there are large set of variables and coefficients , I have no idea how to extract $Q$ and $q$&lt;/p&gt;&#xA;" OwnerUserId="16535" LastEditorUserId="1804" LastEditDate="2015-09-03T06:57:01.073" LastActivityDate="2015-09-03T06:57:01.073" Title="How to formulate variance minimization as a mixed integer quadratic program" Tags="|optimization|quadratic-programming|mixed-integer-programming|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20608" PostTypeId="1" AcceptedAnswerId="20614" CreationDate="2015-08-31T23:35:48.307" Score="5" ViewCount="203" Body="&lt;p&gt;I am trying to enforce the discrete maximum principle (i.e., ensuring non-negative concentrations) for diffusion-type problems that have an anisotropic diffusivity tensor (e.g., tensor dispersion from velocity). For the standard diffusion equation, I could employ convex optimization since I will have a symmetric and positive definite matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, say I am working with the advection-diffusion equation. My problem is now non-symmetric and non-self-adjoint (and I have observed negative concentrations in its formulations), thus I cannot use convex optimization. Recently, I heard that PETSc has the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/SNES/SNESVINEWTONSSLS.html&quot; rel=&quot;nofollow&quot;&gt;Variational Inequality&lt;/a&gt; feature for its SNES solver, and I was told that this kind of solver is amenable for nonlinear problems. Now my question is, can this solver be applied to a problem like advection-diffusion?&lt;/p&gt;&#xA;" OwnerUserId="14576" LastEditorUserId="1804" LastEditDate="2015-09-01T10:17:06.827" LastActivityDate="2015-09-01T10:50:44.477" Title="Can Variational Inequalities handle non-symmetric matrices?" Tags="|optimization|pde|petsc|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20610" PostTypeId="1" AcceptedAnswerId="20612" CreationDate="2015-09-01T02:18:40.677" Score="5" ViewCount="3655" Body="&lt;p&gt;Considering any two cylinders, defined as: the center of their bottoms $A_i$, the radius of their bottom $R_i$, the unit vector $W_i$ of their axis direction, and the length $L_i$ of the cylinders, where $i=1,2$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3vhle.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3vhle.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what is the necessary and sufficient condition of the intersection of the two cylinders?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to numerically determine (by program, e.g., Matlab or C++) whether two given cylinders intersect with each other or not. So whether the problem is numerically solvable and the uniqueness and existence of the solution become important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically, I tried to convert the problem into a linearly constrained least squares problem, but found it is hard to prove the equivalence between the intersection and the solution of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constrained least squares problem is: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;linear constraints: the convex point sets of points between the top and bottoms of the two cylinders, this defines four linear constraints;&lt;/li&gt;&#xA;&lt;li&gt;find the point of which the summed squares of distance to the two cylinder axes is at minimum;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then compare the minimum value with $\displaystyle\sum\limits_{i=1}^2 R_i^2$ to determine whether the cylinders intersect or not. -- It seems counter example can be found when there is no intersection point while the minimum is still less than the criterion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to find a numerically solvable (e.g., numerically convex optimization) necessary and sufficient condition so as to determine whether two cylinders intersect or not?&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2015-09-01T10:24:16.780" Title="How to determine whether two cylinders intersect or not?" Tags="|computational-geometry|convex-optimization|constrained-optimization|least-squares|quadratic-programming|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20612" PostTypeId="2" ParentId="20610" CreationDate="2015-09-01T04:13:50.200" Score="6" Body="&lt;p&gt;David Eberly has a good description of the solution (with pseudocode) here: &lt;a href=&quot;http://www.geometrictools.com/Documentation/IntersectionOfCylinders.pdf&quot;&gt;http://www.geometrictools.com/Documentation/IntersectionOfCylinders.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short summary: like most convex-convex collision detection algorithms, you search systematically for a separating axis between the cylinders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For future reference on similar problems, the authors of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1568814240&quot;&gt;Real-Time Rendering&lt;/a&gt; maintain a &lt;a href=&quot;http://www.realtimerendering.com/intersections.html&quot;&gt;table of links for intersection tests&lt;/a&gt; between different types of primitives.&lt;/p&gt;&#xA;" OwnerUserId="12" LastEditorUserId="12" LastEditDate="2015-09-01T04:27:56.393" LastActivityDate="2015-09-01T04:27:56.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20613" PostTypeId="2" ParentId="20610" CreationDate="2015-09-01T10:24:16.780" Score="3" Body="&lt;p&gt;&lt;strong&gt;Hint&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the two cylinders are parallel, the problem is easy. Otherwise, if the perpendicular distance between the axis exceeds the sum of the radii, there is no intersection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, there is an intersection curve between the two lateral surfaces. It can be expressed analytically: without loss of generality, the second cylinder is vertical with a base at the origin (if not, rotate and translate). Its lateral surface has the implicit equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^2+y^2=R_2^2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any point of the lateral surface of the first cylinder can be reached by the parametric equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_1+U_1R_1\cos(s)+V_1R_1\sin(s)+W_1t,\\&#xA;0\le s\le 2\pi$$ where $U_1$ and $V_1$ form an orthogonal basis with $W_1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plugging the parametric equation in the implicit one, you get an equation which is quadratic in $t$. So you can express $t$ as a function of $s$, analytically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving for $t=0$ and $t=L_1$ (numerically) gives you the useful sections of the curve that belong to the first cylinder, and delimit an $(s,t)$ domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By plugging the equations of the boundaries of this domain in the parametric equation ($z$ component), you should be able to tell if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$0\le A_{1z}+U_{1z}R_1\cos(s)+V_{1z}R_1\sin(s)+W_{1z}t\le L_2$$ occurs in the domain.&lt;/p&gt;&#xA;" OwnerUserId="17391" LastActivityDate="2015-09-01T10:24:16.780" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20614" PostTypeId="2" ParentId="20608" CreationDate="2015-09-01T10:33:42.067" Score="4" Body="&lt;p&gt;It is indeed possible to write an obstacle problem for an advection-diffusion equation as a variational inequality: If $a(u,v)$ is the bilinear form corresponding to your advection-diffusion equation, the corresponding obstacle problem (in your specific case) is finding $u\geq 0$ such that&#xA;$$ a(u,v-u) \geq (f,v-u) \qquad \text{for all } v\geq 0.$$&#xA;In principle, no symmetry or convexity is required for this to be well-posed; however:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There's no guarantee in general that the solution to the obstacle problem coincides with the (non-negative) solution of the advection-diffusion problem. In this case, it might work since there are variational principles for advection-diffusion equations; see, e.g., &lt;a href=&quot;http://www.aero.caltech.edu/~ortiz/Pubs/1985/Ortiz1985.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.aero.caltech.edu/~ortiz/Pubs/1985/Ortiz1985.pdf&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The SNES routine in PETSc is based on a semismooth Newton method, which (like any Newton method) requires uniformly bounded invertibility of the Newton matrix. If the problem is not uniformly convex, this doesn't always hold. (But it might work anyway.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How to feed your variational inequality to PETSc is something I'll leave to the experts.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-09-01T10:50:44.477" LastActivityDate="2015-09-01T10:50:44.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20615" PostTypeId="1" CreationDate="2015-09-01T11:27:36.317" Score="1" ViewCount="97" Body="&lt;p&gt;I am working to solve Poisson's equation in 2D axisymmetric cylindrical coordinates using the Jacobi method. The $L^2$ norm decreases from $\sim 10^3$ on the first iteration (I have a really bad guess) to $\sim 0.2$ very slowly. Then, the $L^2$ norm begins to increase over many iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My final matrix is weakly diagonally dominate, except for the 2nd order Neumann condition at $r = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I make a small tweek to make this work, is it numeric or do I need a new algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My geometry is parallel plates with sharp points at $r = 0$ on both plates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My boundary conditions are&#xA;$$\left. \frac{\partial V}{\partial r} \right|_{r=0} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I would like my second radial BC to be&#xA;$$\left. \frac{\partial V}{\partial r} \right|_{r=\infty} = 0$$&#xA;I settled for&#xA;$$\left. \frac{\partial V}{\partial r} \right|_{r=a} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then Dirichlet conditions at the upper and lower boundaries&#xA;$$V(r, L(r) ) = V_0$$&#xA;$$V(r, U(r) ) = V_L$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&#xA;$$L(r) = \begin{cases}&#xA; &amp;amp; 0 \text{ if } r \geq R_L \\ &#xA; &amp;amp; H_L (1 - \frac{r}{R_L} ) \text{ if } r \leq R_L &#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U(r) = \begin{cases}&#xA; &amp;amp; H \text{ if } r \geq R_U \\ &#xA; &amp;amp; H + H_U (\frac{r}{R_U} - 1 ) \text{ if } r \leq R_U &#xA;\end{cases}$$&lt;/p&gt;&#xA;" OwnerUserId="17394" LastEditorUserId="17394" LastEditDate="2015-09-01T16:44:52.127" LastActivityDate="2015-09-01T16:44:52.127" Title="Jacobi method converging then diverging" Tags="|pde|algorithms|finite-difference|boundary-conditions|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20616" PostTypeId="1" CreationDate="2015-09-01T12:56:32.717" Score="0" ViewCount="44" Body="&lt;p&gt;I want to ask questions like 'what family is this species in, and which other species are in this family?'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of Wikispecies/Wikidata but are there any other good open and linked databases/APIs that would be worth looking at?&lt;/p&gt;&#xA;" OwnerUserId="17395" LastActivityDate="2015-09-01T12:56:32.717" Title="What is the most complete open and linked dataset to use for taxonomy?" Tags="|computational-biology|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20617" PostTypeId="1" AcceptedAnswerId="20632" CreationDate="2015-09-01T14:54:57.887" Score="3" ViewCount="1079" Body="&lt;p&gt;I have the system which is described by several ODE. The solution looks good for me. Now I need to implement the sensitivity analyses of parameters which I used in the model. Therefore, I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I have an output in the form of the vector(solution in time). But the sensitivity routines (such &lt;a href=&quot;http://salib.github.io/SALib/&quot; rel=&quot;nofollow&quot;&gt;SALib&lt;/a&gt;) requires the output to be scalar variable $F(X)$ but not a vector. What should I do? I found in several resources that they model some &quot;true&quot; solution without any variance in parameters and later (when they perform the sensitivity) they compare it with &quot;true&quot; solution via RMSD. Is it correct way of doing that? (could you provide some references)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Parameter boundaries: how to choose the range of parameters? In the some papers the modelers do just ±5% of their values of parameters. What is the most typical way of doing this? based on the literature values and taking [min-5% ; max+5%] will be good? What if I dont have any literature values?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11497" LastEditorUserId="11497" LastEditDate="2015-09-02T12:49:02.880" LastActivityDate="2015-09-02T16:48:10.703" Title="How to perform the sensitivity analyses of ODE with several parameters?" Tags="|numerical-analysis|ode|numerical-modelling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20618" PostTypeId="1" CreationDate="2015-09-01T15:38:54.087" Score="4" ViewCount="437" Body="&lt;p&gt;This question is probably related to chemical engineers that are around this forum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking into writing a solver for a precipitation reaction (struvite to be precise): A+B+C+D &amp;lt;=&gt; E+F&#xA;A, B, C, D, F - liquids; E - solid (for now I ignore the 'solid' part and focus only on mass transfer)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have tried to avoid using combustion-thermo-chemistry libs/classes (which as far as I understand are highly interlinked and dependent of each other) and wrote a simple multispecies transfer solver that performs a forwards reaction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For mass transport source term currently my option is on using 'Laminar finite rate model' as in Fluent, though - I was not able to find out in the literature whether this is appropriate for precipitation reactions (If anyone could answer that then I'd be grateful).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this is where I end up at and in order to move the development forwards (get backwards transfer working) I either need to rewrite some classes from chemistry/thermo/combustion libs, or go with modifying reacting foam, which again is using mentioned libs/classes and I'm not fully familiar with the models used there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd be glad if anyone could give me some pointers on:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) How to use existing combustion/thermo/chemistry libs/classes for my reactions, that as far as my understanding of chemistry goes are not combustive?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Any literature pointers on what models are used on such reactions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;c) If anyone worked on similar stuff maybe you could share your ideas on what would be the best way to approach these type of problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ben&lt;/p&gt;&#xA;" OwnerUserId="17396" LastActivityDate="2016-01-30T05:14:56.740" Title="OpenFOAM precipitation/crystallisation reaction solver" Tags="|fluid-dynamics|computational-chemistry|openfoam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20619" PostTypeId="1" CreationDate="2015-09-01T17:39:08.177" Score="1" ViewCount="208" Body="&lt;p&gt;The equation at the left of the interface is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\displaystyle\frac{\partial C_i}{\partial t} = D_i \nabla^2 C_i - z_i \frac{D_i}{RT}F \nabla \cdot (C_i \nabla \phi_2)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation at the right of the interface is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\displaystyle\frac{\partial \epsilon C_i}{\partial t} = \nabla \cdot \biggl(\epsilon D_i \nabla C_i + z_i \frac{ D_i}{RT}F\epsilon C_i \nabla \phi_2 \biggr) + S(x) &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with an extra source term accounted for pore wall reaction and precipitation (also an extra dependent variable $\epsilon$ for porosity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At $t=1e-5$ the concentration for $C_1$ looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XjnWPl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XjnWPl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At $t=1e-2$ the concentration for $C_1$ looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ewc8I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ewc8I.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But since the value of the source term is negative, I suppose it should be a upward parabola at the interface.&#xA;From time $t=1e-3$ to $t=1e-2$, the distribution for $\phi_2$ changes dramatically at the left of the interface though.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/X9PjM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X9PjM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VHV0t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VHV0t.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some concentrations on the other hand are smooth across the domain though, but $C_1$ and $C_8$, which have the two largest initial condition values, are discontinuous like the picture shown at the interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After $t=1e-2$ the RADAU5 solver converges very slowly (I guess it won't converge to a meaningful values anyway).&#xA;I'm using cell-centered finite volume for it is easier to implement the Neumann boundary conditions. The discontinuous point is at the first cell right at the left of the interface. I'm using $N=100$; for $N=400$ the discontinuous point even show earlier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the problems? And how should I resolve it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's assumed that concentration and potential are continuous at the interface, which in fact they are, but in a &lt;strong&gt;cell-centered grid&lt;/strong&gt;, I'm thinking I can't just naively do nothing at the interface.&lt;/p&gt;&#xA;" OwnerUserId="16998" LastEditorUserId="16998" LastEditDate="2015-09-02T04:58:27.667" LastActivityDate="2015-09-02T04:58:27.667" Title="Discontinuity at Interface" Tags="|pde|finite-difference|finite-volume|diffusion|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20620" PostTypeId="1" CreationDate="2015-09-01T17:57:17.920" Score="0" ViewCount="100" Body="&lt;p&gt;I have several files of data created by LabView of hydraulic pressure testing over the course of a few days. This data is mostly time with various pressure and temperature measurements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you recommend software that will plot several hours worth of data and then let me zoom/pan and select certain intervals? I need to comb through the data a find examples of acceptable pressure holds. Excel is inadequate for this task. &lt;/p&gt;&#xA;" OwnerUserId="17399" LastEditorUserId="9667" LastEditDate="2015-09-01T18:24:26.647" LastActivityDate="2015-09-12T05:11:34.850" Title="Software recommendation for plotting data from lab tests" Tags="|visualization|plotting|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20621" PostTypeId="2" ParentId="20620" CreationDate="2015-09-01T18:15:01.537" Score="2" Body="&lt;p&gt;GnuPlot &lt;a href=&quot;http://www.gnuplot.info/&quot; rel=&quot;nofollow&quot;&gt;http://www.gnuplot.info/&lt;/a&gt; is quite complete and good for generating various types of graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a gallery of the different types of graphs  that it can generate &lt;a href=&quot;http://gnuplot.sourceforge.net/demo/&quot; rel=&quot;nofollow&quot;&gt;http://gnuplot.sourceforge.net/demo/&lt;/a&gt; (as well as example scripts to generate them)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using Linux, then its probably already in your distribution (else it can be installed using the package manager)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using Windows, there are some windows builds available there:&#xA;&lt;a href=&quot;http://ctan.ijs.si/mirror/w32tex/w32/&quot; rel=&quot;nofollow&quot;&gt;http://ctan.ijs.si/mirror/w32tex/w32/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2015-09-01T18:15:01.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20622" PostTypeId="2" ParentId="20620" CreationDate="2015-09-01T18:57:03.267" Score="2" Body="&lt;p&gt;As @nicoguaro mentioned in his comment. If you are familiar with python programming language, I would recommend   matplotlib. It is a Python-based plotting library. You can plot  various types of graphs such as line, scattered,histogram, etc. You can also make 3d plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://matplotlib.org/gallery.html&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; is a gallery of plots produced by matplotlib. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matplotlib also has  a large community so you can find answers to your questions easily.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="4282" LastEditDate="2015-09-01T20:00:31.513" LastActivityDate="2015-09-01T20:00:31.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20623" PostTypeId="2" ParentId="20618" CreationDate="2015-09-02T03:03:42.963" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I'd be glad if anyone could give me some pointers on:&lt;/p&gt;&#xA;&lt;p&gt;a) How to use existing combustion/thermo/chemistry libs/classes for my reactions, that as far as my understanding of chemistry goes are not combustive?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Your best bet is to use &lt;a href=&quot;http://www.cantera.org/docs/sphinx/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cantera&lt;/a&gt;, which include models for liquid-phase thermodynamics. You will probably need to add some functionality yourself, but Cantera is open-source, and Ray Speth, who was at one point the lead developer of Cantera, is a great guy, and you can contact him about what you want to do. (Disclaimer: Ray is an old colleague.)&lt;/p&gt;&#xA;&lt;p&gt;The main issue you're going to run into with other libraries is that is that combustion thermo &amp;amp; kinetics libraries mostly focus on gas-phase chemistry, and for the temperatures and pressures under consideration (excluding engines, most of the time, less than 10 bar, and aside from possibly an initial transient, 700-4000 K) the ideal gas law is a good approximation. Consequently, there just isn't much in the way of non-ideality or fugacity coefficient effects, and you're really going to have to hack these solvers to include that functionality, plus activity coefficient models, if you need them. A colleague of mine several years ago &amp;quot;faked&amp;quot; liquid-phase chemistry for hydrocarbons by treating them as very high-pressure gases, but I don't think that's a good solution for your problem, or for the general case. This largely rules out closed-source libraries (e.g., CHEMKIN-PRO and its competitors). It also rules out many open-source libraries (e.g., TChem, PoKiTT) that focus on gas-phase chemistry exclusively.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;b) Any literature pointers on what models are used on such reactions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It mostly depends on the specific chemical system, and what you're trying to actually model. I think the main difficulty with precipitation can be if you want to understand particle sizes in the precipitate, and where the solid is actually going to go (is it entrained in the liquid and swept away by the main flow? does it just fall to the bottom of your control volume?).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-09-02T03:03:42.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20624" PostTypeId="1" CreationDate="2015-09-02T03:26:58.597" Score="7" ViewCount="877" Body="&lt;p&gt;So first off:&#xA;*** This code is not being used in production software.&lt;br&gt;&#xA;It is a personal project of mine, trying to understand &#xA;approximation theory and advanced curve fitting.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, I'm trying to understand how it works, &#xA;not trying to get a currently existing solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have been trying to implement the &lt;a href=&quot;https://en.wikipedia.org/wiki/Remez_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Remez algorithm&lt;/a&gt; for polynomial approximation.&#xA;I sort of/maybe/kind of have it working (not really).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current solution generate ok polynomials, but &#xA;a)  the coefficients are not converging&#xA;&amp;amp;&#xA;b)  while monitoring the coefficients at each stage, I've noticed that the x-values seem to slip past each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll give some examples to show what I mean.&lt;br&gt;&#xA;My base function I'm trying to model is the square root function with a 4 degree polynomial on the domain [0.25, 1]&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Round 1&#xA;X[0] = 0.25&#xA;X[1] = 0.4&#xA;X[2] = 0.55&#xA;X[3] = 0.7&#xA;X[4] = 0.85&#xA;X[5] = 1&#xA;&#xA;Round 2&#xA;X[0] = 0.25&#xA;X[1] = 0.595076928220583&#xA;X[2] = 0.493988453622788&#xA;X[3] = 0.714333640596557&#xA;X[4] = 1.14135676154991&#xA;X[5] = 1&#xA;&#xA;Round 3&#xA;X[0] = 0.25&#xA;X[1] = 0.638393337463021&#xA;X[2] = 0.63752199068821&#xA;X[3] = 0.538600997945798&#xA;X[4] = 1.07101841739164&#xA;X[5] = 1&#xA;&#xA;Round 4&#xA;X[0] = 0.25&#xA;X[1] = 0.559423143598625&#xA;X[2] = 0.560673538304964&#xA;X[3] = 0.580378820375143&#xA;X[4] = 1.04454592077508&#xA;X[5] = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So here's a look at my actual code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;typename func_t&amp;gt;&#xA;type_t MiniMax(size_t Degree, const type_t&amp;amp; LowerLimit, const type_t&amp;amp; UpperLimit, unsigned char Iterations, func_t F0, func_t F1, func_t F2)&#xA;{&#xA;    // (C) Jacob Wells 2015&#xA;    // This code is licensed under the BSD 3-Clause License&#xA;    // http://opensource.org/licenses/BSD-3-Clause&#xA;&#xA;    if((Degree &amp;lt; 1) || (Iterations &amp;lt; 1))&#xA;    {&#xA;        return (type_t)NAN;&#xA;    }&#xA;&#xA;    const type_t ONE(1), NEG1(-1), ZERO(0);&#xA;    matrix_t&amp;lt;type_t&amp;gt; M(Degree + 2, Degree + 3);&#xA;    vector&amp;lt;type_t&amp;gt; XVal;&#xA;    type_t Delta, Sign, Pow, Err;&#xA;    type_t D1, D2;&#xA;    size_t I, J;&#xA;&#xA;    Delta = (UpperLimit - LowerLimit) / (Degree + 1);&#xA;    XVal.resize(Degree + 2);&#xA;    Coef.resize(Degree + 1);&#xA;    Sign = ONE;&#xA;&#xA;    for(I = 0; I &amp;lt; (Degree + 2); I++)               // Generate our initial x-values&#xA;    {&#xA;        XVal[I] = (I * Delta) + LowerLimit;&#xA;    }&#xA;&#xA;    do&#xA;    {&#xA;        Sign = NEG1;&#xA;        for(I = 0; I &amp;lt; (Degree + 2); I++)&#xA;        {&#xA;            Pow = ONE;&#xA;            M[I][Degree + 1] = Sign;            // Enters the alternating error sign&#xA;            M[I][Degree + 2] = F0(XVal[I]);     // Enters the f(x) value&#xA;            Sign *= NEG1;                       &#xA;            for(J = 0; J &amp;lt;= Degree; J++)        // Evaluates the polynomial for each power&#xA;            {&#xA;                M[I][J] = Pow;&#xA;                Pow *= XVal[I]; &#xA;            }&#xA;        }&#xA;&#xA;        rref(Degree + 2, M);                    // Use Row Reduction Echelon Form to find the polynomial coefficients&#xA;&#xA;        Err = M[Degree + 1][Degree + 2];&#xA;&#xA;        for(I = 0; I &amp;lt;= Degree; I++)            // Copy the coefficients into the polynomial class' array&#xA;        {&#xA;            Coef[I] = M[I][Degree + 2];&#xA;        }&#xA;&#xA;        if(Iterations &amp;gt; 1)&#xA;        {&#xA;            for(I = 1; I &amp;lt;= Degree; I++)        // Use Newton's method to find our new x-values&#xA;            {&#xA;                D1 = nth_deriv(XVal[I], 1) - F1(XVal[I]);&#xA;                D2 = nth_deriv(XVal[I], 2) - F2(XVal[I]);&#xA;&#xA;                if((D1 != ZERO) &amp;amp;&amp;amp; (D2 != ZERO))&#xA;                {&#xA;                    XVal[I] -= (D1 / D2);           &#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;    }while(--Iterations != 0);&#xA;&#xA;    return Err;&#xA;} &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A quick little guide to some of my code:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;F0, F1, &amp;amp; F2 are, respectively, the square root functions, it's first derivative, and it's second derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;nth_deriv calculate the Nth Derivative of the current polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;rref reduces the Matrix to Row Reduction Echelon Form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;matrix_t is a bare bones matrix class I came up with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have done a lot of testing on these functions, and I haven't found a single error with them, so I feel very confident that the problem is in the MiniMax Function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My barebones matrix class&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;typename type_t&amp;gt;&#xA;class matrix_t&#xA;{&#xA;    public:&#xA;    ~matrix_t() {}&#xA;&#xA;&#xA;    matrix_t(size_t ColL, size_t RowL)&#xA;    {&#xA;        Arr.resize(ColL * RowL);&#xA;        RowLen = RowL;&#xA;    }&#xA;&#xA;    type_t* operator [] (size_t I)&#xA;    {&#xA;        return &amp;amp;Arr[I * RowLen];&#xA;    }&#xA;&#xA;    const type_t* operator [] (size_t I) const&#xA;    {&#xA;        return &amp;amp;Arr[I * RowLen];&#xA;    }&#xA;&#xA;    type_t At(size_t I)&#xA;    {&#xA;        return Arr[I];&#xA;    }&#xA;&#xA;&#xA;    private:&#xA;    size_t RowLen;&#xA;    vector&amp;lt;type_t&amp;gt; Arr;&#xA;&#xA;&#xA;&#xA;    matrix_t();&#xA;    matrix_t(const matrix_t&amp;amp; m);&#xA;    void operator = (const matrix_t&amp;amp; m);&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17403" LastEditorUserId="25160" LastEditDate="2019-11-18T19:40:42.790" LastActivityDate="2019-11-18T19:40:42.790" Title="Problems Implementing the Remez Algorithm" Tags="|regression|approximation|polynomials|curve-fitting|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="20625" PostTypeId="2" ParentId="20607" CreationDate="2015-09-02T06:33:50.727" Score="1" Body="&lt;p&gt;As answered elsewhere, for MATLAB you have things like CVX and YALMIP. In YALMIP, you would solve using something like (assuming you have defined function f and g)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = binvar(n,1);&#xA;objective = var(f(x),g(x)) + ...&#xA;optimize([],objective)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A suitable solver will be called (if f and g are linear operators it is a MIQP and if Gurobi is installed and visible on path it will be used)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you simply want to export the numerical data ($Q$ etc), you could do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;export([],objective,sdpsettings('solver','gurobi'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note though, a MIQP with 3000 variables can easily be completely intractable for any solver.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2015-09-02T06:33:50.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20628" PostTypeId="1" CreationDate="2015-09-02T14:28:31.177" Score="0" ViewCount="50" Body="&lt;p&gt;I have implemented in matlab a neural network that uses rprop's algorithm to update its weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strangely the error on the training set does not converge to a local minimum, but oscillates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the graphics plot of the error function on trainingset:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lBWH2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lBWH2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the rprop's algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;function [net] = resilientBackPropagation(net, DW, DB, ETA_PLUS, ETA_MINUS)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for i=1:length(net.W) &#xA;    productDW = net.DW{i}.*DW{i}; &#xA;    productDB = net.DB{i}.*DB{i};&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;indDW_gt_0 = find(productDW &amp;gt; 0);   &#xA;indDB_gt_0 = find(productDB &amp;gt; 0);      &#xA;indDW_lt_0 = find(productDW &amp;lt; 0); &#xA;indDB_lt_0 = find(productDB &amp;lt; 0); &#xA;indDW_eq_0 = find(productDW == 0);&#xA;indDB_eq_0 = find(productDB == 0); &#xA;&#xA;net.deltaMarginW{i}(indDW_gt_0) = min(ETA_PLUS.*net.deltaMarginW{i}(indDW_gt_0), 50);&#xA;net.deltaMarginB{i}(indDB_gt_0) = min(ETA_PLUS.*net.deltaMarginB{i}(indDB_gt_0), 50); &#xA;&#xA;   net.deltaMarginW{i}(indDW_lt_0) = max(ETA_MINUS.*net.deltaMarginW{i}(indDW_lt_0), exp(-6));&#xA;net.deltaMarginB{i}(indDB_lt_0) = max(ETA_MINUS.*net.deltaMarginB{i}(indDB_lt_0), exp(-6));     &#xA;&#xA;    net.W{i}(indDW_gt_0) = net.W{i}(indDW_gt_0)-(sign(DW{i}(indDW_gt_0).*net.deltaMarginW{i}(indDW_gt_0)));&#xA;net.B{i}(indDB_gt_0) = net.B{i}(indDB_gt_0)-(sign(DB{i}(indDB_gt_0).*net.deltaMarginB{i}(indDB_gt_0)));   &#xA;&#xA;  DW{i}(indDW_lt_0) = 0;&#xA;DB{i}(indDB_lt_0) = 0;  &#xA;&#xA;     net.W{i}(indDW_eq_0) = net.W{i}(indDW_eq_0)-(sign(DW{i}(indDW_eq_0).*net.deltaMarginW{i}(indDW_eq_0)));&#xA;net.B{i}(indDB_eq_0) = net.B{i}(indDB_eq_0)-(sign(DB{i}(indDB_eq_0).*net.deltaMarginB{i}(indDB_eq_0)));   &#xA;&#xA;Totale nella Rete Neurale.&#xA;net.DW{i} = DW{i};&#xA;net.DB{i} = DB{i};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;end&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the problem?&#xA;thanks&lt;/p&gt;&#xA;" OwnerUserId="17407" LastEditorUserId="1128" LastEditDate="2015-09-02T16:25:30.583" LastActivityDate="2015-09-02T16:25:30.583" Title="Oscillating convergence plot in my Resilient BackPropagation (RPROP) implementation" Tags="|matlab|" AnswerCount="0" CommentCount="2" ClosedDate="2015-09-02T17:43:11.497" ContentLicense="CC BY-SA 3.0" />
  <row Id="20629" PostTypeId="1" AcceptedAnswerId="20636" CreationDate="2015-09-02T14:44:54.480" Score="37" ViewCount="13968" Body="&lt;p&gt;I have a simple question that is really hard to Google (besides the canonical &lt;em&gt;What Every Computer Scientist Should Know About Floating-Point Arithmetic&lt;/em&gt; paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When should functions such as &lt;code&gt;log1p&lt;/code&gt; or &lt;code&gt;expm1&lt;/code&gt; be used instead of &lt;code&gt;log&lt;/code&gt; and &lt;code&gt;exp&lt;/code&gt;? When should they not be used? How do different implementations of those functions differ in terms of their usage?&lt;/p&gt;&#xA;" OwnerUserId="14933" LastEditorUserId="713" LastEditDate="2015-09-02T16:25:02.797" LastActivityDate="2023-12-30T16:27:43.010" Title="When should log1p and expm1 be used?" Tags="|floating-point|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="20631" PostTypeId="1" CreationDate="2015-09-02T16:15:46.693" Score="1" ViewCount="221" Body="&lt;p&gt;I have implemented in matlab a neural network that uses rprop's algorithm to update its weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strangely the error on the training set does not converge to a local minimum, but oscillates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the graphics plot of the error function on trainingset:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kBz1h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kBz1h.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the rprop's algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [net] = resilientBackPropagation(net, DW, DB, ETA_PLUS, ETA_MINUS)&#xA;&#xA;for i=1:length(net.W) % Ciclo sul Numero di Strati della Rete&#xA;&#xA;    % Calcolo il Prodotto tra le Derivate dei pesi e dei bias di E e E^-1 dell'i-esimo Strato&#xA;    productDW = net.DW{i}.*DW{i}; &#xA;    productDB = net.DB{i}.*DB{i};&#xA;&#xA;    % Prelevo gli Indici dei pesi e dei bias dell'i-esimo Strato con Prodotto Positivo, Negativo e Nullo  &#xA;    indDW_gt_0 = find(productDW &amp;gt; 0);   &#xA;    indDB_gt_0 = find(productDB &amp;gt; 0);      &#xA;    indDW_lt_0 = find(productDW &amp;lt; 0); &#xA;    indDB_lt_0 = find(productDB &amp;lt; 0); &#xA;    indDW_eq_0 = find(productDW == 0);&#xA;    indDB_eq_0 = find(productDB == 0); &#xA;&#xA;    % Calcolo i Margini dei pesi e dei bias con Prodotto Positivo&#xA;    net.deltaMarginW{i}(indDW_gt_0) = min(ETA_PLUS.*net.deltaMarginW{i}(indDW_gt_0), 50);&#xA;    net.deltaMarginB{i}(indDB_gt_0) = min(ETA_PLUS.*net.deltaMarginB{i}(indDB_gt_0), 50); &#xA;&#xA;    % Calcolo i Margini dei pesi e dei bias con Prodotto Negativo&#xA;    net.deltaMarginW{i}(indDW_lt_0) = max(ETA_MINUS.*net.deltaMarginW{i}(indDW_lt_0), exp(-6));&#xA;    net.deltaMarginB{i}(indDB_lt_0) = max(ETA_MINUS.*net.deltaMarginB{i}(indDB_lt_0), exp(-6));     &#xA;&#xA;    % Aggiornamento dei pesi e dei bias della Rete con Prodotto Positivo&#xA;    net.W{i}(indDW_gt_0) = net.W{i}(indDW_gt_0)-(sign(DW{i}(indDW_gt_0).*net.deltaMarginW{i}(indDW_gt_0)));&#xA;    net.B{i}(indDB_gt_0) = net.B{i}(indDB_gt_0)-(sign(DB{i}(indDB_gt_0).*net.deltaMarginB{i}(indDB_gt_0)));   &#xA;&#xA;    % Aggiornamento dei pesi e dei bias della Rete con Prodotto Negativo&#xA;    DW{i}(indDW_lt_0) = 0;&#xA;    DB{i}(indDB_lt_0) = 0;  &#xA;&#xA;     % Aggiornamento dei pesi e dei bias della Rete con Prodotto Nullo&#xA;    net.W{i}(indDW_eq_0) = net.W{i}(indDW_eq_0)-(sign(DW{i}(indDW_eq_0).*net.deltaMarginW{i}(indDW_eq_0)));&#xA;    net.B{i}(indDB_eq_0) = net.B{i}(indDB_eq_0)-(sign(DB{i}(indDB_eq_0).*net.deltaMarginB{i}(indDB_eq_0)));   &#xA;&#xA;    % Memorizzazione delle Derivate dei pesi e dei bias Aggiornati dell'Errore Totale nella Rete Neurale.&#xA;    net.DW{i} = DW{i};&#xA;    net.DB{i} = DB{i};&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ps:the dataset used is banana_dataset and in the plot the x-axis represents the epoch of training and the y-axis the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ETA_MINUS = 0.5;&#xA;ETA_PLUS = 1.2&lt;/p&gt;&#xA;" OwnerUserId="17409" LastEditorUserId="17409" LastEditDate="2015-09-02T17:19:55.770" LastActivityDate="2015-09-02T17:19:55.770" Title="Oscillating convergence in my Resilient BackPropagation (RPROP) implementation" Tags="|optimization|matlab|convergence|nonlinear-programming|regression|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20632" PostTypeId="2" ParentId="20617" CreationDate="2015-09-02T16:48:10.703" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There are 2 possibilities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A) Somehow aggregate the time series into a single number. If you have a &quot;true&quot; solution, yes you can calculate RMS error or some other error measure. If you don't have a true solution to compare against, you can aggregate using some other meaningful statistic (like the sum, or the peak, etc. depending on the application). But in general in this case you're calculating the sensitivity of just one number representing the whole time series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(B) Or, you can calculate the sensitivity of the output value at every timestep. You would create a loop and analyze for every timestep. This is more computationally intensive, but can result in some more dynamic information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Herman, J. D., Kollat, J. B., Reed, P. M., &amp;amp; Wagener, T. (2013). From maps to  movies: high-resolution time-varying sensitivity analysis for spatially distributed watershed models. Hydrology and Earth System Sciences, 17(12), 5109–5125.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;van Werkhoven, K., Wagener, T., Reed, P., &amp;amp; Tang, Y. (2008). Characterization of watershed model behavior across a hydroclimatic gradient. Water Resources Research, 44(1).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As for the parameter boundaries, that is very important and very domain-specific. First look and see if you can find some ranges from another paper where they used your model, and cite those. If not, choosing a percent value is okay, but 5% seems a bit narrow -- you might try 10% instead. The most important thing is that your outputs still make sense. If your ranges produce parameter samples that are &quot;non-physical&quot;, in other words the output has no bearing on reality anymore, then you should narrow them. There is a lot of subjectivity here and it can change your results quite a bit.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(c) Jon Herman, the creator of SALib&lt;/p&gt;&#xA;" OwnerUserId="11497" LastActivityDate="2015-09-02T16:48:10.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20634" PostTypeId="1" AcceptedAnswerId="20637" CreationDate="2015-09-03T03:00:10.680" Score="5" ViewCount="1098" Body="&lt;p&gt;Is the projection method of integrating Navier-Stokes equations exact? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the incompressible flow equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial\mathbf{u}}{\partial t} = -\mathbf{u}\cdot \nabla \mathbf{u} -\nabla p + \mu \Delta \mathbf{u}&#xA;\\&#xA;\nabla\cdot\mathbf{u} = 0&#xA;$$&#xA;along with some meaningful boundary conditions. Using a projection operator $\mathbf P$ defined as &#xA;$$ \mathbf{P} = I -\nabla (\Delta)^{-1}\nabla&#xA;$$ the earlier equations becomes an ODE like so:&#xA;$$\frac{\partial\mathbf{u}}{\partial t} = \mathbf{P}(-\mathbf{u}\cdot \nabla \mathbf{u} + \mu \Delta \mathbf{u}) $$&#xA; because $\mathbf{u}$ is solenoidal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any approximation is involved here or are the two sets of equations exactly same? And, can one discretize and integrate the ODE-looking equation with arbitrary accuracy in space and time using their choice of discretization and integrator because it is an ODE? &#xA;Is finding (analytical) $\mathbf{P}$ the tricky part here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know what might be done in practice in writing a finite difference code e.g. Adams-Bashforth + Crank-Nicholson. A discrete projection operator is obtained by solving a Poisson equation&lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2017-01-23T22:51:08.370" Title="Incompressible Navier-Stokes equations: Is projection method exact?" Tags="|navier-stokes|projection|incompressible|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20636" PostTypeId="2" ParentId="20629" CreationDate="2015-09-03T09:38:56.473" Score="31" Body="&lt;p&gt;We all know that&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\exp(x) = \sum_{n=0}^\infty \frac{x^n}{n!} = 1 + x + \frac12 x^2 + \dots&#xA;\end{equation}&lt;/span&gt;&#xA;implies that for &lt;span class=&quot;math-container&quot;&gt;$|x| \ll 1$&lt;/span&gt;, we have &lt;span class=&quot;math-container&quot;&gt;$\exp(x) \approx 1 + x$&lt;/span&gt;. This means that if we have to evaluate in floating point &lt;span class=&quot;math-container&quot;&gt;$\exp(x) -1$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$|x| \ll 1$&lt;/span&gt; catastrophic cancellation can occur.&lt;/p&gt;&#xA;&lt;p&gt;This can be easily demonstrated in python:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from math import (exp, expm1)&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; x = 1e-8&#xA;&amp;gt;&amp;gt;&amp;gt; exp(x) - 1&#xA;9.99999993922529e-09&#xA;&amp;gt;&amp;gt;&amp;gt; expm1(x)&#xA;1.0000000050000001e-08&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; x = 1e-22&#xA;&amp;gt;&amp;gt;&amp;gt; exp(x) - 1&#xA;0.0&#xA;&amp;gt;&amp;gt;&amp;gt; expm1(x)&#xA;1e-22&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Exact values are&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\exp(10^{-8}) -1 &amp;amp;= 0.000000010000000050000000166666667083333334166666668 \dots \\&#xA;\exp(10^{-22})-1 &amp;amp;= 0.000000000000000000000100000000000000000000005000000 \dots&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In general an &amp;quot;accurate&amp;quot; implementation of &lt;code&gt;exp&lt;/code&gt; and &lt;code&gt;expm1&lt;/code&gt; should be correct to no more than 1ULP (i.e. one unit of the last place). However, since attaining this accuracy results in &amp;quot;slow&amp;quot; code, sometimes a fast, less accurate implementation is available. &lt;s&gt;For example in CUDA we have &lt;code&gt;expf&lt;/code&gt; and &lt;code&gt;expm1f&lt;/code&gt;, where &lt;code&gt;f&lt;/code&gt; stands for fast. According to the &lt;a href=&quot;http://docs.nvidia.com/cuda/cuda-c-programming-guide/#standard-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUDA C programming guide, app. D&lt;/a&gt; the &lt;code&gt;expf&lt;/code&gt; has an error of 2ULP.&lt;/s&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you do not care about errors in the order of few ULPS, usually different implementations of the exponential function are equivalent, but beware that bugs may be hidden somewhere... (Remember the &lt;a href=&quot;https://en.wikipedia.org/wiki/Pentium_FDIV_bug&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pentium FDIV bug&lt;/a&gt;?)&lt;/p&gt;&#xA;&lt;p&gt;So it is pretty clear that &lt;code&gt;expm1&lt;/code&gt; should be used to compute &lt;span class=&quot;math-container&quot;&gt;$\exp(x)-1$&lt;/span&gt; for small &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Using it for general &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is not harmful, since &lt;code&gt;expm1&lt;/code&gt; is expected to be accurate over its full range:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; exp(200)-1 == exp(200) == expm1(200)&#xA;True&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(In the above example &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; is well below 1ULP of &lt;span class=&quot;math-container&quot;&gt;$\exp(200)$&lt;/span&gt;, so all three expression return exactly the same floating point number.)&lt;/p&gt;&#xA;&lt;p&gt;A similar discussion holds for the inverse functions &lt;code&gt;log&lt;/code&gt; and &lt;code&gt;log1p&lt;/code&gt; since &lt;span class=&quot;math-container&quot;&gt;$\log(1+x) \approx x$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$|x| \ll 1$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2023-12-30T16:27:43.010" LastActivityDate="2023-12-30T16:27:43.010" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20637" PostTypeId="2" ParentId="20634" CreationDate="2015-09-03T09:42:33.200" Score="4" Body="&lt;p&gt;The projection scheme is indeed analytic, provided that the initial conditions are incompressible.  You can derive it by taking the divergence of the evolution equation for $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the projection &lt;strong&gt;P&lt;/strong&gt; can be tricky; it's easier with some methods like spectral / pseudospectral methods.  But one has to be careful that numerical error doesn't move the solution off of the incompressible manifold; these errors can add up.  Boundary conditions can also be tricky, because &lt;strong&gt;P&lt;/strong&gt; has to accommodate for those as  well.  Also, the resulting equation is a PDE; the RHS has derivatives in space, so to advance it in time one must perform some sort of spatial discretization to transform it into a system of ODEs.&lt;/p&gt;&#xA;" OwnerUserId="3182" LastActivityDate="2015-09-03T09:42:33.200" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20638" PostTypeId="1" CreationDate="2015-09-03T14:00:20.540" Score="6" ViewCount="692" Body="&lt;p&gt;I have to do some numerical calculus using gamma functions. I am using the &lt;code&gt;tgamma&lt;/code&gt; incluided in the C++ &lt;code&gt;cmath&lt;/code&gt; library. The problem is that the arguments inside the gamma functions are positive and big, so I always get a &lt;code&gt;nan&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that I can use a log-gamma function, &lt;code&gt;lgamma&lt;/code&gt; in C++, to avoid numerical overflow during the calculation. The problem is that I don't know how to apply this in my case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, with something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X=\dfrac{\Gamma(a+b)\Gamma(a-b)}{\Gamma(c)}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a$, $b$ and $c$ are &lt;em&gt;integers&lt;/em&gt; largue enough to produce an overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that I could use a log to obtain $\log(X)=\log(\Gamma(a+b)) + \log(\Gamma(a-b)) -\log(\Gamma(c))$, obtain the result from that operation and make the exponential to get the correct result without overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, what about if I have something like this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X=\sum_i^N p_i\cdot\dfrac{\Gamma(i+b)\Gamma(i-b)}{\Gamma(c)} + \sum_i^N\sum_{j&amp;lt;i} q_i\cdot \dfrac{\Gamma(i+b)\Gamma(j-b)}{\Gamma(c)}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case $p_i$ and $q_i$ are real numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have a sum, so the $\log$ properties are useless. Of course, I could simply substitute all the gammas with log-gammas, but, in that case, how could I retrieve my original result? Can I do something to avoid the overflow?&lt;/p&gt;&#xA;" OwnerUserId="17425" LastEditorUserId="9667" LastEditDate="2015-09-03T19:02:07.397" LastActivityDate="2015-09-03T19:02:07.397" Title="Using Log Gamma function to avoid overflow" Tags="|c++|floating-point|special-functions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20639" PostTypeId="2" ParentId="20638" CreationDate="2015-09-03T15:25:46.847" Score="6" Body="&lt;p&gt;You can avoid overflow by rewriting the sum appropriately. Say you have a sum&#xA;$$ X = \sum_{i=1}^{n} x_i, $$&#xA;where some $x_i$'s are positive and large enough to cause overflow, but $\log x_i$ are of reasonable magnitude. If you find the greatest $\log x_M \geq \log x_i$, then you can rewrite the sum as&#xA;$$ \log X = \log \sum_i e^{\log x_i} = \log x_M + \log \sum_i e^{\log x_i - \log x_M}. $$&#xA;By doing this, only logs are used in the computation. In the remaining sum inside the log we have $\log x_i \leq \log x_M$, so $e^{\log x_i - \log x_M} \leq 1$, which means the value of the remaining sum is at most $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously this doesn't work if $x_i$'s are allowed to sometimes be negative - in that case you might get catastrophic cancellation if they can be negative and relatively large compared to the final value, although there is no problem if they are negative and sufficiently small.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-09-03T15:32:21.410" LastActivityDate="2015-09-03T15:32:21.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20641" PostTypeId="1" CreationDate="2015-09-03T21:29:51.753" Score="1" ViewCount="390" Body="&lt;p&gt;I am a newbie in DFT calculations. I did potential energy scans with GAUSSIAN 09 for different dimers, i.e.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    # b3lyp/6-31G(d) scan test&#xA;&#xA;    title&#xA;&#xA;    0 1&#xA;    Ni 0.0 0.0 0.0&#xA;    Ni 0.0 0.0 r&#xA;&#xA;    r 20.0 19 -1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unexpectedly, the energy at large distances does not converges to zero, but some finite negative value. Why does this happen?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phenomenon seems to be independent of chemical element, functionals, basis set. I am obviously missing something here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="17429" LastActivityDate="2018-02-22T00:43:23.423" Title="Why is potential energy of dimer in DFT scan at large distance not zero?" Tags="|density-functional-theory|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20642" PostTypeId="2" ParentId="20466" CreationDate="2015-09-04T09:27:05.787" Score="1" Body="&lt;p&gt;I cannot directly answer your question, but:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Use &lt;code&gt;contour&lt;/code&gt; instead of &lt;code&gt;imshow&lt;/code&gt;, otherwise your plot is x-flipped. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Use the same limits for axis (that is from -20 to 20) as in the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) As others mentioned, use the same parameters as in the paper (that is &lt;code&gt;k=1&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Styling the plot in the same way as in the paper also makes plot comparing easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've fixed these issues, here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.special import erf&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&quot;&quot;&quot;Plane wave scattering by a plane wave incident on a semi-infinite plane&quot;&quot;&quot;&#xA;&#xA;T = np.pi / 4.0  # Incident plane wave angle&#xA;k = 1.0  # 10  # Wave number&#xA;&#xA;def phi_scattered(r,t):&#xA;    W = lambda z: np.exp(-z**2)*(1-erf(z))&#xA;    term1 = W(np.exp(1j*np.pi/4.0)*np.sqrt(2*k*r)*np.sin((t+T)/2.0))&#xA;    term2 = W(np.exp(1j*np.pi/4.0)*np.sqrt(2*k*r)*np.sin((t-T)/2.0))&#xA;    res =  np.exp(1j*k*r)/2.0*(term1 + term2)&#xA;&#xA;    return res&#xA;&#xA;def phi_incident(r,t):&#xA;    return np.exp(1j*k*r*np.cos(t - T))&#xA;&#xA;x1 = np.linspace(-20, 20, 100)&#xA;x2 = np.linspace(-20, 20, 100)&#xA;X1, X2 = np.meshgrid(x1,x2)&#xA;&#xA;R = np.sqrt(X1**2 + X2**2)&#xA;THETA = np.arctan2(X2, X1)&#xA;&#xA;phi = phi_scattered(R, THETA) + phi_incident(R, THETA)&#xA;&#xA;plt.figure()&#xA;plt.contourf(x1, x2, np.real(phi), cmap=plt.cm.gist_rainbow)&#xA;plt.colorbar()&#xA;plt.contour(x1, x2, np.real(phi), colors='k', linestyles='solid')&#xA;plt.axis('image')&#xA;plt.savefig('scattering.png', dpi=300)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result is:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/L57Bu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L57Bu.jpg&quot; alt=&quot;Styled plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that it's flipped along x-axis.&lt;/p&gt;&#xA;" OwnerUserId="17421" LastEditorUserId="9667" LastEditDate="2015-09-04T18:45:16.003" LastActivityDate="2015-09-04T18:45:16.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20643" PostTypeId="1" AcceptedAnswerId="30610" CreationDate="2015-09-04T10:12:30.543" Score="5" ViewCount="311" Body="&lt;p&gt;What type of numerical methods are there to solve PDE of the sorts of: &#xA;$$\begin{align}&#xA;&amp;amp;f(x,t,u(x,t))u_{xx} - g(x,t,u(x,t))u_{tt} = F(x,t,u(x,t))\\&#xA;&amp;amp;u(x,0)=G_1(x)\\&#xA;&amp;amp;\frac{\partial u(x,0)}{\partial t}=H_1(x)\\&#xA;&amp;amp;u(0,t)=G_2(t) , \frac{\partial u(0,t)}{\partial x} = H_2(t)&#xA;\end{align}$$ &#xA;Where $f,\ g,\ F,\ u \in C^\infty(x,t),\ G_i,\ H_i \in C^\infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically I had in mind the PDE:&#xA;$$u_{xx}u^3-\sin(xt)u_{tt} = u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the general PDE is as above; I looked at Polyanin's second edition Handbook of Nonlinear PDE table of contents, and didn't find something similar, obviously I look at numerical solutions since an analytical solution doesn't seem plausible, but if there is I wouldn't mind. :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also posted this question in &lt;a href=&quot;https://mathoverflow.net/questions/217417/numerical-methods-for-solving-a-hyperbolic-nonlinear-pde&quot;&gt;MathOverflow&lt;/a&gt;, where one answer pointed out that my problem is likely not well-posed.&lt;/p&gt;&#xA;" OwnerUserId="17431" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2018-11-29T15:11:00.610" Title="Numerical methods for solving a mixed type nonlinear PDE" Tags="|pde|numerical-analysis|hyperbolic-pde|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20644" PostTypeId="1" AcceptedAnswerId="20646" CreationDate="2015-09-04T11:22:27.367" Score="11" ViewCount="8150" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;br /&gt;&#xA;I have only built one working numeric solution to 2d Navier-Stokes, for a course.  It was a solution for lid-driven cavity flow.  The course, however, discussed a number of schemas for spatial discretizations and time discretizations.  I have also taken more symbol-manipulation coursework applied to NS.&lt;/p&gt;&#xA;&lt;p&gt;Some of the numeric approaches to handle conversion of the analytic/symbolic equation from PDE to finite difference include:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Euler FTFS, FTCS, BTCS&lt;/li&gt;&#xA;&lt;li&gt;Lax&lt;/li&gt;&#xA;&lt;li&gt;Midpoint Leapfrog&lt;/li&gt;&#xA;&lt;li&gt;Lax-Wendroff&lt;/li&gt;&#xA;&lt;li&gt;MacCormack&lt;/li&gt;&#xA;&lt;li&gt;offset grid (spatial diffusion allows information to spread)&lt;/li&gt;&#xA;&lt;li&gt;TVD&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To me, at the time, these seemed like &amp;quot;insert-name finds a scheme and it happens to work&amp;quot;.  Many of these were from before the time of &amp;quot;plentiful silicon&amp;quot;.  They are all approximations.  In the limit they. in theory, lead to the PDE's.&lt;/p&gt;&#xA;&lt;p&gt;While Direct Numerical Simulation (&lt;a href=&quot;https://en.wikipedia.org/wiki/Direct_numerical_simulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNS&lt;/a&gt;) is fun, and Reynolds Averaged Navier-Stokes (&lt;a href=&quot;https://en.wikipedia.org/wiki/Reynolds-averaged_Navier%E2%80%93Stokes_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;RANS&lt;/a&gt;) is also fun, they are the two &amp;quot;endpoints&amp;quot; of the continuum between computationally tractable, and fully representing the phenomena.  There are multiple families of approaches that live interior to these.&lt;/p&gt;&#xA;&lt;p&gt;I have had CFD professors say, in lecture, that most CFD solvers make pretty pictures, but for the most part, those pictures do not represent reality and that it can be very tough, and take lots of work, to get a solver solution that does represent reality.&lt;/p&gt;&#xA;&lt;p&gt;The sequence of development (as I understand it, not exhaustive) is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;start with the governing equations -&amp;gt; PDE's&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;determine your spatial and temporal discretization -&amp;gt; grid and FD rules&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;apply to the domain including initial conditions and boundary conditions&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;solve (lots of variations on matrix inversion)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;perform gross reality checks, fit to known solutions, etc..&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;build some simpler physical models derived from analytic results&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;test them, analyze, and evaluate&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;iterate (jumping back to either step 6, 3, or 2)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thoughts:&lt;br /&gt;&#xA;I have recently been working with CART models, oblique trees, random forests, and gradient boosted trees.  They follow more mathematically derived rules, and the math drives the shape of the tree.  They work to make discretized forms well.&lt;/p&gt;&#xA;&lt;p&gt;Although these human-created numeric approaches work somewhat, there is extensive &amp;quot;voodoo&amp;quot; needed to connect their results to the physical phenomena they are meant to model. Often the simulation does not substantially replace real-world testing and verification.  It is easy to use the wrong parameter, or not account for variation in geometry or application parameters experienced in the real world.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Has there been any approach to let the nature of the problem define&lt;br /&gt;&#xA;the appropriate discretization, spatial and temporal differencing scheme,&#xA;initial conditions, or solution?&lt;/li&gt;&#xA;&lt;li&gt;Can a high definition solution coupled with the techniques of&#xA;machine learning be used to make a differencing scheme that has much&#xA;larger step sizes but retains convergence, accuracy, and&#xA;such?&lt;/li&gt;&#xA;&lt;li&gt;All of these schemes are accessibly &amp;quot;humanly tractable to derive&amp;quot; -&#xA;they have a handful of elements.  Is there a differencing scheme&#xA;with thousands of elements that does a better job?  How is it derived?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note: I will follow up with the empirically intialized and empirically derived (as opposed to analytically) in a separate question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use of deep learning to accelerate lattice Boltzmann flows.  Gave ~9x speedup for their particular case&lt;/p&gt;&#xA;&lt;p&gt;Hennigh, O. (in press). Lat-Net: Compressed Lattice Boltzmann Flow Simulations using Deep Neural Networks. Retrieved from:&#xA;&lt;a href=&quot;https://arxiv.org/pdf/1705.09036.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1705.09036.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Repo with code (I think):&lt;br /&gt;&#xA;&lt;a href=&quot;https://github.com/loliverhennigh/Phy-Net&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/loliverhennigh/Phy-Net&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;About 2 orders of magnitude faster than GPU, 4 orders of magnitude, or ~O(10,000x) faster than CPU, and same hardware.&lt;/p&gt;&#xA;&lt;p&gt;Guo, X., Li, W. &amp;amp; Ioiro, F. Convolutional Neural Networks for Steady Flow Approximation. Retrieved from:&#xA;&lt;a href=&quot;https://autodeskresearch.com/publications/convolutional-neural-networks-steady-flow-approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://autodeskresearch.com/publications/convolutional-neural-networks-steady-flow-approximation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Others who have looked into the topic about 20 years ago:&lt;/p&gt;&#xA;&lt;p&gt;Muller, S., Milano, M. &amp;amp; Koumoutsakos P. Application of machine learning algorithms to flow modeling and optimization.  Center for Turbulence Research Annual Research Briefs 1999 Retrieved from:&#xA;&lt;a href=&quot;https://web.stanford.edu/group/ctr/ResBriefs99/petros.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/group/ctr/ResBriefs99/petros.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update (2017):&lt;/strong&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://arxiv.org/abs/1712.06567&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; characterises the use of non-gradient methods in deep learning, an arena which has been exclusively gradient based.  While the direct implication of activity is in deep learning, it also suggests that GA can be used as an equivalent in solving a very hard, very deep, very complex problem at the level consistent with or superior to gradient descent based methods.&lt;/p&gt;&#xA;&lt;p&gt;Within the scope of this question, it might suggest that a larger-scale, machine-learning based attack might allow &amp;quot;templates&amp;quot; in time and space that substantially accelerate convergence of gradient-domain methods.  The article goes as far as to say that sometimes going in the direction of gradient descent moves away from the solution.  While in any problem with local optima or pathological trajectories (most high-value real-world problems have some of these) it is expected that the gradient isn't globally informative, it is still nice to have it quantified and validated empirically as it was in this paper and the ability to &amp;quot;jump the bound&amp;quot; without requiring &amp;quot;reduction of learning&amp;quot; as you get in momentum or under-relaxation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update (2019):&lt;/strong&gt;&lt;br /&gt;&#xA;It seems that google now has a contribution &amp;quot;how to find a better solver&amp;quot; piece of the AI puzzle.  &lt;a href=&quot;https://ai.googleblog.com/2018/08/introducing-new-framework-for-flexible.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; This is a part of making the AI make the solver.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update (2020b):&lt;/strong&gt;&lt;br /&gt;&#xA;And now they are doing it, and doing it well...&lt;br /&gt;&#xA;&lt;a href=&quot;https://arxiv.org/pdf/1911.08655.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1911.08655.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is that a 100x speedup for Navier-Stokes solution??&lt;/p&gt;&#xA;&lt;p&gt;It could be argued that they could then deconstruct their NN to determine the actual discretization.  I particularly like figure 4.&lt;/p&gt;&#xA;&lt;p&gt;Update 2020 &lt;a href=&quot;https://arxiv.org/pdf/2006.08762.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update 2022&lt;/strong&gt; - Physics Informed neural networks allow state compression by a factor of 85x with only slight loss.  (&lt;a href=&quot;https://arxiv.org/pdf/2201.03617v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) One of the values of being able to stop, store the state, move it to another computer, and be able to pick it up and restart at the stop-point and not the initial condition is that it can allow reproducibility.  A decent near-estimate of a state means the compute engine can get to the actual with only a little work.  A ML-based &amp;quot;picture&amp;quot; that is good enough can act just like all the compute overhead from the initial state to the good enough estimate.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update 2022:&lt;/strong&gt;&lt;br /&gt;&#xA;ML systems are now becoming much better at pushing back the pareto frontier on chaotic systems.  The demo is a flame front. This suggests there is a form implicit to the learner that captures and predicts the chaotic part in a way that analytical approaches do not.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quantamagazine.org/machine-learnings-amazing-ability-to-predict-chaos-20180418/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quantamagazine.org/machine-learnings-amazing-ability-to-predict-chaos-20180418/&lt;/a&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.120.024102&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.120.024102&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3979" LastEditorUserId="3979" LastEditDate="2022-08-29T03:47:27.953" LastActivityDate="2022-08-29T03:47:27.953" Title="Use of machine learning in computational fluid dynamics" Tags="|finite-difference|fluid-dynamics|machine-learning|numerics|" AnswerCount="2" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="20645" PostTypeId="2" ParentId="20644" CreationDate="2015-09-04T13:59:20.620" Score="7" Body="&lt;p&gt;I think you are mixing a couple different ideas that are causing confusion.  Yes, there are a wide variety of ways to discretize a given problem.  Choosing an appropriate way may look like &quot;voodoo&quot; when you are learning these things in class, but when researchers choose them, they do so drawing on the combined experience of the field, as published in literature.  Therefore they make much more informed choices than a student could.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 1:&lt;/strong&gt; &#xA;If you are solving a problem, and you switch from one scheme to another, your run time will change, the convergence criteria may change, or your asymptotic behavior, but a very important point is that your final converged solution should NOT change.  If it does, you either need to refine your meshes, or there is something wrong with your numerical scheme.  Perhaps you could use some optimization algorithm to create your numerical schemes and improve performance for a specific class of problems, but many times the hand derived schemes are created with mathematically provable optimal convergence/asymptotic behavior for the number of terms involved or mesh type used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the above paragraph doesn't account for things like different turbulence models, which are different mathematical formulations/approximations of the physics, so are expected to have different solutions.  These are again highly studied in the literature, and I don't think programs are at the point that they can look at physical phenomena and produce a mathematical model that properly predicts the response of similar physical systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 2:&lt;/strong&gt;&#xA;Yes, you could derive a scheme that uses the entire mesh at once, using some computer code to do it.  I even feel safe saying that for some meshes, such codes exist, and could give you your scheme in a matter of a couple hours (once you find the code that is).  The problem is that you will never beat Nyquist.  There is a limit to how large of time steps you take, depending on the max frequency of the response of your system, and a limit to how large of mesh cells/elements you can have depending on the spatial frequencies of the solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That doesn't even account for the fact that often the computational work involved in using a more complex scheme is often non-linear with complexity.  The reason most students learn RK4 methods for time integration is that when you start going to methods with a higher order than that, you gaining more evaluations of your derivative faster than you gain orders of your method.  In the spatial realm, higher order methods greatly increase matrix fill-in, so you need less mesh points, but the work you do inverting the sparse matrix increases greatly, at least partially offsetting your gains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure what you are referring to in question three.  Are you talking about turning close solutions to a problem into better solutions?  If so I recommend some light reading on multigrid.  If you are asking about turning decent numerical schemes into amazing ones, I think the rest of my answer at least touches on that.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2015-09-04T13:59:20.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20646" PostTypeId="2" ParentId="20644" CreationDate="2015-09-04T14:00:10.073" Score="19" Body="&lt;p&gt;It's a long-running joke that CFD stands for &amp;quot;colorful fluid dynamics&amp;quot;. Nevertheless, it is used -- and useful -- in a wide range of applications.&#xA;I believe your discontent stems from not sufficiently distinguishing between two interconnected but different steps: creating a mathematical model of a physical process and solving it numerically. Let me comment on these a bit:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;No mathematical (or really, any) model of physical reality is ever &lt;em&gt;correct&lt;/em&gt;; at best it's &lt;em&gt;useful&lt;/em&gt; for predicting the results of measurements in a very precisely demarcated (but hopefully large) set of circumstances. This includes the fact that it must be possible to obtain such a prediction given a specific configuration; this is why we have a whole hierarchy of models from quantum field theory to Newtonian mechanics. In particular, the Navier-Stokes equations do not describe fluid flow, they give a prediction of specific aspects of the behavior of certain fluids under certain conditions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For the more complicated mathematical models (such as the Navier-Stokes equations), you can never obtain an exact solution (and hence prediction), but only a numerical approximation. This is not such a bad thing as it sounds, since the measurements you want to compare them with are themselves never exact. Just as in the choice of models, there's a trade-off between accuracy and tractability -- it makes no sense to spend time or money on getting a more accurate solution than needed. At this point, it becomes purely a question on how to approximate numerically the solution of (in this case) a partial differential equation, which is the subject of a whole mathematical field: numerical analysis. This field is concerned with &lt;em&gt;proving&lt;/em&gt; error estimates for certain numerical methods (again, under certain, explicitly specified, conditions). Your statement &amp;quot;insert-name finds a scheme and it happens to work&amp;quot;, is grossly unfair -- it should be &amp;quot;insert-name finds a scheme and &lt;em&gt;proves&lt;/em&gt; that it works&amp;quot;. Also, these schemes are not pulled out of thin air -- they are derived from well-understood mathematical principles.&lt;/p&gt;&#xA;&lt;p&gt;(For example, finite difference schemes can be derived using Taylor-approximations of a given order. It is certainly possible to -- and some people do -- obtain very high-order difference schemes and implement them, but there's a law of diminishing returns: this can only be automated partially, and hence takes a lot of effort, and certain increasingly restrictive conditions must be satisfied to actually get the corresponding higher accuracy out of them. Also, at some point it's better to use a different scheme altogether such as spectral methods.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The common theme here is that both models and numerical schemes come with a range of applicability, and it is important to pick the right combination for a given purpose. This is precisely why a computational scientist needs to know both the domain science (to know which model is valid in which situation) and mathematics (to know which method is applicable to which model, and to which accuracy)! Ignoring these &amp;quot;use only as directed&amp;quot; labels leads to producing the kind of &amp;quot;computational bullshit&amp;quot; (in the technical sense of Harry Frankfurt) your CFD professors referred to.&lt;/p&gt;&#xA;&lt;p&gt;As to why use a computational model when you have a physical model (such as a wind tunnel): One reason is that running software can be orders of magnitude cheaper than creating a model and putting it in a wind tunnel. Also, it's usually not an either-or: For example, when designing a car or an airplane, you would run hundreds or thousands of simulations to narrow things down, and then only for the final candidate(s) put a model into a wind tunnel.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using machine learning instead of numerical simulation is like saying &amp;quot;having no model is better than having an approximate model&amp;quot;, which I doubt anyone in fluid dynamics (or any other field) would agree with. That being said, it's certainly possible (and actually done) to use machine learning to select unknown &amp;quot;geometry or application parameters&amp;quot; based on agreement with measured data; however, here as well model-based methods such as uncertainty quantification or (Bayesian) inverse problems usually perform much better (and are based on rigorous mathematical principles). Selecting &lt;em&gt;numerical&lt;/em&gt; parameters such as step size or order of the method using machine learning is also possible in principle, but I fail to see the benefit since there's a mathematical theory that tells you precisely how to pick these parameters based on your (mathematical) model.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The paper you link to is about &lt;em&gt;computer graphics&lt;/em&gt;, not &lt;em&gt;computational science&lt;/em&gt;: their goal is not to have an accurate simulation (i.e., a numerical solution of a mathematical model) of a physical process, but something that just looks like one to the naked eye (an extreme case of &amp;quot;colorful fluid dynamics&amp;quot;...) -- that is a very different matter. In particular, there is &lt;em&gt;no&lt;/em&gt; error bound for the output of the trained network compared to the corresponding solution to the Navier-Stokes equations, which is an indispensable part of any numerical method.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;(And your first question starts from a false premise: in &lt;em&gt;every&lt;/em&gt; approach, the problem determines the model, the model determines the discretization, the discretization determines the solver.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-11-23T09:24:49.287" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20647" PostTypeId="1" CreationDate="2015-09-04T15:04:46.497" Score="2" ViewCount="291" Body="&lt;p&gt;I have come across the paper titled:&#xA;&lt;a href=&quot;http://www.researchgate.net/publication/223435655_A_monolithic_fluidstructure_interaction_algorithm_applied_to_the_piston_problem&quot; rel=&quot;nofollow&quot;&gt;A monolithic fluid structure interaction algorithm ... &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 1D grid, at the boundaries the paper uses virtual nodes $x_{0}$ and $x_{N+1}$ (page 372)&#xA;and for instance for velocity v at the boundaries ends up via extrapolation with (page 374)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{0} = 2 x_{1}-x_{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{N+1} = 2 x_{N}-x_{N-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if someone could explain to me where the points N and 0 lie? On the faces or at nodes? Also what is the reason for using this extrapolation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you! &lt;/p&gt;&#xA;" OwnerUserId="17362" LastEditorUserId="17362" LastEditDate="2015-09-07T22:31:16.177" LastActivityDate="2015-09-07T22:31:16.177" Title="FVM - virtual node discretisation" Tags="|finite-volume|discretization|hyperbolic-pde|extrapolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20648" PostTypeId="1" AcceptedAnswerId="20649" CreationDate="2015-09-04T18:38:06.427" Score="3" ViewCount="285" Body="&lt;p&gt;I try to write tests to the implementations of numerical methods.&#xA;The best way to do this is to study the observed order of accuracy and check that it matches with the theoretical order of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I noticed that the observed order of accuracy lies in a band&#xA;$$ [p_{\mathrm{theor}} - \Delta, p_{\mathrm{theor}} + \Delta], $$&#xA;where $p_{\mathrm{theor}}$ is the theoretical order of accuracy and $\Delta$ is some absolute error that must be chosen for any particular method.&#xA;For some of the methods I implement I can take $\Delta=0.05$, for other $\Delta=0.1$ and so forth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is: how should one choose $\Delta$ to be sure that there are no mistakes in the implementation?&lt;/p&gt;&#xA;" OwnerUserId="3254" LastActivityDate="2015-09-04T19:21:41.213" Title="How close observed order of accuracy should be to theoretical order of accuracy?" Tags="|testing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20649" PostTypeId="2" ParentId="20648" CreationDate="2015-09-04T19:21:41.213" Score="2" Body="&lt;p&gt;From practical experience, I find plotting the results of discretization refinement studies best to figure out whether or not the observed order of accuracy is &quot;close enough&quot;. However, writing an automated test of a method's order of accuracy requires some sort of regression to calculate the observed order of accuracy from data, and then defining an appropriate $\Delta$ -- a comparison tolerance -- such that if the observed order of accuracy is within $\Delta$ of the theoretical order of accuracy, the test &quot;passes&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any theory that suggests an appropriate $\Delta$ or an appropriate way to bracket the asymptotic order of convergence region &lt;em&gt;a priori&lt;/em&gt;. I have always written these kinds of tests by reverse-engineering them from plots, so I typically choose $\Delta = .2$ or $\Delta = .3$, because I see the order of accuracy deviate by at most that amount.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In certain cases (e.g., discontinuous initial conditions, or stiff problems), theory can predict how the order of accuracy of a method will degrade; if this additional theory isn't taken into account, then you might need to choose a larger $\Delta$ to compensate. For instance, in Section 8.7 of &quot;Finite-Volume Methods for Hyperbolic Problems&quot;, Randy LeVeque discusses how convergence theory for a first-order upwind method assumes smooth solutions. If instead, a weak solution is discontinuous, the order of accuracy in space (in the $L^{1}$ norm) is one-half instead of one. So in a case like that, if you assume that the true order should be one, you might need to make $\Delta$ as large as 0.6, but in some cases, the degradation of the error is known, and you can adjust the theoretical order of accuracy accordingly.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-09-04T19:21:41.213" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20652" PostTypeId="2" ParentId="20599" CreationDate="2015-09-05T04:33:41.600" Score="4" Body="&lt;p&gt;It's not too difficult to formulate a problem like this in CVX, but unfortunately your instances are too large to be solved in practice by the primal-dual interior point solvers used by CVX.  The problem here is in the minimax objective function, which CVX will convert into &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min t$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$| P_{d}(\theta,\phi)-\mbox{stuff} | \leq t$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for each $\theta$, $\phi$ pair.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each absolute value inequality further gets converted to two linear inequalities.  There are over 65,000 $\theta$, $\phi$ pairs, so you'll end up with an SDP with over 130,000 constraints, which is outside the practical range of solvability (it requires the storage of a dense 130K by 130K matrix which would require over 135 gigabytes of RAM.) Furthermore, you might well run out of memory simply assembling this problem in CVX.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you be interested in solving this for a much smaller range of $\theta$, $\phi$ pairs?  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-09-05T04:33:41.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20653" PostTypeId="1" CreationDate="2015-09-05T10:35:01.447" Score="2" ViewCount="423" Body="&lt;p&gt;I'm currently attempting to turn my code for solving the laplace equation using finite element approximations into an adaptive one using the dual weighted residual as my error estimator: i.e. my indicator for element i: $η_i=(f[i]−∑_0^N A_{ij}U_h[j])∗z[i]$. i.e. I set up f, A and z over a very fine mesh and solve U_h over the normal mesh then interpolate it so i can perform the above operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving for the dual solution z relatively simply; i.e. B(v,z) = Q(v) where B is the stiffness matrix and Q is a vector of 0s with 1 at the quantity of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the errors do not behave how they should; as an example i got the mesh to just increase from 10 in uniform multiples of 10 until 150. However these are the error estimates (as solved above):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.0335462065038&#xA;0.0335462065038&#xA;0.0335462065038&#xA;0.0335462065038&#xA;0.0335462065038&#xA;0.0333608591916&#xA;0.0335462065038&#xA;0.0334614504804&#xA;0.0333158830798&#xA;0.0335462065038&#xA;0.0334317537862&#xA;0.0333344104201&#xA;0.0335441544821&#xA;0.0334327061457&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;They seem to drop then grow, drop then grow repeatedly and i have no idea why, does anybody have any clues what could be causing this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is my code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def rhs(x0):&#xA;    rhsfunction = np.empty(len(x0))&#xA;    for i in range(len(x0)):&#xA;        rhsfunction[i] = math.sin(math.pi*x0[i])&#xA;    return rhsfunction&#xA;&#xA;def Poisson_Stiffness(x0):&#xA;    &quot;&quot;&quot;Finds the Poisson equation stiffness matrix with any non uniform mesh x0&quot;&quot;&quot;&#xA;&#xA;    x0 = np.array(x0)&#xA;    N = len(x0) - 1 # The amount of elements; x0, x1, ..., xN&#xA;&#xA;    h = x0[1:] - x0[:-1]&#xA;&#xA;    a = np.zeros(N+1)&#xA;    a[0] = 1 #BOUNDARY CONDITIONS&#xA;    a[1:-1] = 1/h[1:] + 1/h[:-1]&#xA;    a[-1] = 1/h[-1]&#xA;    a[N] = 1 #BOUNDARY CONDITIONS&#xA;&#xA;    b = -1/h&#xA;    b[0] = 0 #BOUNDARY CONDITIONS&#xA;&#xA;    c = -1/h&#xA;    c[N-1] = 0 #BOUNDARY CONDITIONS: DIRICHLET&#xA;&#xA;    data = [a.tolist(), b.tolist(), c.tolist()]&#xA;    Positions = [0, 1, -1]&#xA;    Stiffness_Matrix = diags(data, Positions, (N+1,N+1))&#xA;&#xA;return Stiffness_Matrix&#xA;&#xA;def NodalQuadrature(x0):&#xA;    &quot;&quot;&quot;Finds the Nodal Quadrature Approximation of sin(pi x)&quot;&quot;&quot;&#xA;&#xA;    x0 = np.array(x0)&#xA;    h = x0[1:] - x0[:-1]&#xA;    N = len(x0) - 1&#xA;&#xA;    approx = np.zeros(len(x0))&#xA;    approx[0] = 0 #BOUNDARY CONDITIONS&#xA;&#xA;    for i in range(1,N):&#xA;        approx[i] = math.sin(math.pi*x0[i])&#xA;        approx[i] = (approx[i]*h[i-1] + approx[i]*h[i])/2&#xA;&#xA;    approx[N] = 0 #BOUNDARY CONDITIONS&#xA;&#xA;return approx&#xA;&#xA;def Solver(x0):&#xA;&#xA;    Stiff_Matrix = Poisson_Stiffness(x0)&#xA;&#xA;    NodalApproximation = NodalQuadrature(x0)&#xA;    NodalApproximation[0] = 0&#xA;&#xA;    U = scipy.sparse.linalg.spsolve(Stiff_Matrix, NodalApproximation)&#xA;&#xA;return U&#xA;&#xA;def Dualsolution(rich_mesh,qoi): #BOUNDARY CONDITIONS?&#xA;    &quot;&quot;&quot;Find Z from stiffness matrix Z = K^-1 Q over richer mesh&quot;&quot;&quot;&#xA;&#xA;    K = Poisson_Stiffness(rich_mesh)&#xA;    Q = np.zeros(len(rich_mesh))&#xA;    if qoi in rich_mesh:&#xA;&#xA;        qoi_rich_node = rich_mesh.tolist().index(qoi)&#xA;        Q[qoi_rich_node] = 1.0&#xA;    else:&#xA;&#xA;        nearest = find_nearest(rich_mesh,qoi)&#xA;        if nearest &amp;lt; qoi:&#xA;&#xA;            qoi_lower_node = rich_mesh.tolist().index(nearest)&#xA;            qoi_upper_node = qoi_lower_node+1&#xA;&#xA;        else:&#xA;&#xA;            qoi_upper_node = rich_mesh.tolist().index(nearest)&#xA;            qoi_lower_node = qoi_upper_node-1&#xA;&#xA;        Q[qoi_upper_node] = (qoi - rich_mesh[qoi_lower_node])/(rich_mesh[qoi_upper_node]-rich_mesh[qoi_lower_node])&#xA;        Q[qoi_lower_node] = (rich_mesh[qoi_upper_node] - qoi)/(rich_mesh[qoi_upper_node]-rich_mesh[qoi_lower_node])&#xA;    Z = scipy.sparse.linalg.spsolve(K,Q)&#xA;    return Z&#xA;&#xA;def Indicators(richx0,basex0,qoi):&#xA;&quot;&quot;&quot;interpolate U, eta_i = r_i z_i = (f_i - sum(AijUj)Zi&quot;&quot;&quot;&#xA;    U = Solver(basex0)&#xA;    U_inter = interp1d(basex0,U)&#xA;    U_rich = U_inter(richx0)&#xA;    f = rhs(richx0)&#xA;    A = Poisson_Stiffness(richx0).tocsr()&#xA;    Z = Dualsolution(richx0,qoi)&#xA;&#xA;    eta = np.empty(len(richx0))&#xA;    for i in range(len(eta)):&#xA;        eta[i] = f[i]&#xA;        for j in range(len(richx0)):&#xA;            eta[i] = eta[i] - A[i,j]*U_rich[j]&#xA;        eta[i] = eta[i] * Z[i]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I understand theres alot of code here but i have no idea where else to turn. I'm almost certain my functions for solving the finite element method are correct (i.e. the Stiffness matrix, nodal quadrature and the solver). Any help would be appreciated alot.&#xA;Thanks, James&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT, this is my plot of the error indicator which must be incorrect but i cannot figure out what is wrong with it.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NUmm2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NUmm2.png&quot; alt=&quot;Error estimate over mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17444" LastEditorUserId="17444" LastEditDate="2015-09-05T20:22:42.053" LastActivityDate="2015-09-05T20:22:42.053" Title="Adaptive Finite Element Method - Laplace" Tags="|finite-element|python|mesh-generation|adaptive-mesh-refinement|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20654" PostTypeId="1" CreationDate="2015-09-06T10:34:44.673" Score="2" ViewCount="609" Body="&lt;p&gt;As I am new to all these stuff I would like to verify whether my procedure is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to calculate the Lennard-Jones potential, potential energy of the system and the forces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The procedure is as follows: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Read the input data: side lenght of the square -a, number of particles -N, the parameters $\sigma$, $\epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) For $i=1,n$ insert randomly n points in x and y axis.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;2a) For $j=i+1,n+1$ calculate the difference to square between&#xA;particles in x and y axis.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;2b) If $r_{ij} \leq 2^{1/6}\sigma$ go back to 2a) step&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;2c) End of the both loops&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;3) For $i=1,n$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;3a) For $j=i+1,+1$ calculate the lennard jones potential $$u_{ij}=4\epsilon[(\sigma/r_{ij})^{12}-(\sigma/r_{ij})^6]$$ where $r_{ij}$ is being calculated in 2a) step&lt;/li&gt;&#xA;&lt;li&gt;3b) calculate the potential energy of the system&#xA;$$U_N=\sum_{i=1}^N\sum_{j=i+1}^N u_{ij}(r_{ij})$$&lt;/li&gt;&#xA;&lt;li&gt;3c) Calculate the force $$F=\sum_{j=1,j\neq i}^NF_{ij}$$ where $$F_{ij}=(-\nabla_ju_{ij}(r_{ij}))$$&lt;/li&gt;&#xA;&lt;li&gt;3d) End of the both loops&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would really grateful for checking, whether it is even correct.&#xA;I would also appreciate any books connected to this problem - I am currently using Allend Tildesley, but it is rather difficult for me at start.&lt;/p&gt;&#xA;" OwnerUserId="16846" LastEditorUserId="16846" LastEditDate="2015-09-06T11:34:31.670" LastActivityDate="2015-09-06T11:34:31.670" Title="Molecular dynamics: Lennard-Jones forces in 2D system" Tags="|fluid-dynamics|fortran|molecular-dynamics|" AnswerCount="0" CommentCount="9" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20655" PostTypeId="1" CreationDate="2015-09-07T03:03:05.553" Score="5" ViewCount="1239" Body="&lt;p&gt;Lots of algorithms for semidefinite programming make use of the Frobenius projection onto the cone of semidefinite matrices:&#xA;$$\mathcal{P}(A) = \min_{X\succeq0} \|A-X\|_{\mathrm{Fro}}^2.$$&#xA;Let's assume $A$ is symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard trick for this projection is to compute the eigendecomposition of $A$ as $A=X\Lambda X^\top$, where $\Lambda$ contains eigenvalues of $A$ and $X$ is an orthogonal set of eigenvectors.  Then, $\mathcal P(A)=X\max(\Lambda,0)X^\top.$  But, requires a full eigen-decomposition, which can be slow!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/525367/matrix-projection-on-a-cone&quot;&gt;This post&lt;/a&gt; recommends computing the projection as $\mathcal{P}(A)=\frac{1}{2}(A+\sqrt{A^\top A})$, where $\sqrt{\cdot}$ denotes the matrix square root.  Disappointly, Matlab's &lt;code&gt;sqrtm()&lt;/code&gt; is slower than &lt;code&gt;eig()&lt;/code&gt; in my tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What other algorithms could be used for this task?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stochastic and deterministic techniques are both OK, so long as they have some convergence guarantee (with high probability?).&lt;/p&gt;&#xA;" OwnerUserId="5129" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-09-07T04:09:24.860" Title="Fast projection onto semidefinite cone" Tags="|linear-algebra|optimization|eigensystem|convex-optimization|semidefinite-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20656" PostTypeId="2" ParentId="20655" CreationDate="2015-09-07T04:09:24.860" Score="2" Body="&lt;p&gt;You can potentially save some effort in the eigenvalue decomposition by finding only those eigenvalues (and the corresponding eigenvectors) that are greater than 0.   If most eigenvalues are non-negative, then you can find the negative eigenvalues and subtract off that part of the eigenvalue decomposition to compute the projection.   The LAPACK routine DSYEVR can be helpful.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't found a faster way to do this computation yet, but I'd be interested in a faster approach to the projection that worked by way of the matrix square root or some other approach.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-09-07T04:09:24.860" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20658" PostTypeId="2" ParentId="20647" CreationDate="2015-09-07T09:29:47.773" Score="1" Body="&lt;p&gt;The finite volume method (FVM) uses cell &lt;em&gt;averages&lt;/em&gt; to represent the numerical solution. For example, $x_i$ indicates the center of the $i$-th grid cell, and $\textbf{U}_i^n$ denotes the numerical solution inside grid cell $i$ at time $t^n$. The fluxes at cell boundaries are indicated by $i \pm 1/2$ subscripts, e.g., using your paper's notation, $\Phi_{i+1/2}$ is the numerical flux at the right boundary of the $i$-th cell (at grid coordinate $x_{i+1/2}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your grid cell indices range from $1$ to $N$. In addition to these, the authors define two &quot;ghost&quot; cells with indices $0$ and $N+1$. These are not part of your &quot;real&quot; mesh configuration. Instead, their definition allows you to compute the fluxes at the &quot;real&quot; boundaries of your grid (i.e., $\Phi_{1/2}$ and $\Phi_{N+1/2}$). In 1D, you can think of cell no $0$ as being immediately to the &quot;left&quot; of grid cell $1$. Similarly, cell $N+1$ lies immediately to the &quot;right&quot; of cell $N$, like in the illustration given &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8538/applying-dirichlet-boundary-conditions-to-the-poisson-equation-with-finite-volum&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution approximation inside the ghost cells is computed using standard extrapolation formulas. &lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-09-07T09:55:10.893" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20660" PostTypeId="1" CreationDate="2015-09-07T14:14:58.947" Score="3" ViewCount="713" Body="&lt;p&gt;would someone be so kind as to explain me the difference between the ordinary givens-rotation and the fast givens-rotation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the fast givens Rotation reduces the Count of operations to approximately the half. How is this possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any hints :-)&lt;/p&gt;&#xA;" OwnerUserId="16641" LastActivityDate="2015-09-07T14:14:58.947" Title="Difference between fast and normal Givens Rotations?" Tags="|numerical-analysis|stability|matrix-factorization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20661" PostTypeId="1" AcceptedAnswerId="20701" CreationDate="2015-09-07T14:31:30.933" Score="1" ViewCount="410" Body="&lt;p&gt;I want to implement an implicit ODE solver, but don't know what to do when the differential equations (DEs) have discontinuities of the form:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;More common type:&#xA;$$\lim_{x\rightarrow 0}\frac{\sin(x)}{x} \enspace .$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Less common, as it occurs in only one DE&#xA;$$\min(\alpha x, \beta) \enspace .$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am trying to implement this myself because it is a set of massively coupled, identical DEs and we want to run them, because of our pipeline, on GPUs in Matlab. I considered adapting/modifying Matlab &lt;code&gt;ode15&lt;/code&gt; but I would still want exact evaluations of the Jacobian, since the system has about 10k state variables, so doing numerical approximations to the Jacobian would be expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have generally considered restarting the ODE solver after the discontinuity but I am not sure how to do that in practice: do I define an $\epsilon$ region around the discontinuity and then simply check if the solver reached it, and then restart?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second type of discontinuities I thought that it's the proper limit of a sequence of function of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\lim_{\gamma \rightarrow \infty}\log\left(\frac{\alpha}{1 + \exp{(-\gamma x)}}\right) \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the Jacobian would be defined piecewise as $\alpha$ for $x&amp;lt;0$, $0$ for $x&amp;gt;0$ and $\frac{\alpha}{2}$ for $x = 0$. &lt;/p&gt;&#xA;" OwnerUserId="17460" LastEditorUserId="9667" LastEditDate="2015-09-10T21:27:29.927" LastActivityDate="2015-09-11T08:09:22.337" Title="Implicit ODE solver with discontinuous derivatives" Tags="|ode|implicit-methods|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="20662" PostTypeId="1" CreationDate="2015-09-07T18:00:48.787" Score="8" ViewCount="154" Body="&lt;p&gt;I have the function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x) = \begin{cases}&#xA;0 \:\: (x &amp;lt; a) \\ 1/2 \:\: (x = a)\\ 1 \:\: (x &amp;gt; a)&#xA;\end{cases}$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a$ is unknown. I can compute the function for any value of $x$, and seek to determine $a$ (to some degree of accuracy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given an initial bracket $x_{0} &amp;lt; a &amp;lt; x_{1}$, I subdivide the bracket by defining &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{2} := \frac{x_{0}+x_{1}}{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and computing $f(x_{2})$. Then I have either $x_{0} &amp;lt; a &amp;lt; x_{2}$ or $x_{2} &amp;lt; a &amp;lt; x_{1}$. I could proceed with this approach until the bracketing values agree to some accuracy, and thus solve for $a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more efficient algorithm?&lt;/p&gt;&#xA;" OwnerUserId="17461" LastActivityDate="2015-09-10T08:03:08.710" Title="Bracketing a discontinuity in a step function" Tags="|roots|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20664" PostTypeId="2" ParentId="20662" CreationDate="2015-09-07T23:57:09.180" Score="7" Body="&lt;p&gt;No. Although the question is phrased in terms of floating-point arithmetic, this is at heart a question about binary search. Binary search is an optimal algorithm among algorithms that rely on a decision tree (e.g., &lt;a href=&quot;https://stackoverflow.com/q/4571478/491171&quot;&gt;https://stackoverflow.com/q/4571478/491171&lt;/a&gt;). Any data structures and algorithms textbook should discuss this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively speaking, if at each step you can make a binary decision (one comparison), you can learn at most one bit of information by partitioning the search region. The optimal behaviour is always to partition the search region in halves. If $a$ can take any of $n=|\mathit{high}-\mathit{low}|/\epsilon_{\mathrm{mach}}$ values, the depth of any (binary) decision tree that contains at least $n$ leaves must be at least $\sim \log_2 n$. Binary search achieves this bound of $\log_2 n$ steps. This analysis applies both the expected behaviour when all $a$'s are equally likely, and also to the worst-case behaviour. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have some other information, then it'd be necessary to be more precise about what it means for $a$ to be &quot;unknown&quot;, what operations are permitted in the algorithm, and what it means to be &quot;more efficient&quot;. (As it is, I interpreted them in the conventional way.)&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-09-08T01:47:11.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20665" PostTypeId="1" AcceptedAnswerId="20673" CreationDate="2015-09-08T03:20:00.913" Score="0" ViewCount="143" Body="&lt;p&gt;I want to solve the following differential equation using control volume approach on a Cartesian mesh:&#xA;$$\frac{\partial T}{\partial t} + \frac{\partial T}{\partial x} + \frac{\partial T}{\partial y}= \lambda[\frac{\partial^2 T}{\partial x^2} + \frac{\partial^2 T}{\partial y^2}]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably, there would be no harm in writing it as ($\lambda$ being contant):&#xA;$$\frac{\partial T}{\partial t} + \frac{\partial [T - \lambda\frac{\partial T}{\partial x}]}{\partial x} + \frac{\partial [T - \lambda\frac{\partial T}{\partial x}]}{\partial y}= 0$$&#xA;For a moment if I define&#xA;$$F = T - \lambda\frac{\partial T}{\partial x}$$ $$G = T - \lambda\frac{\partial T}{\partial y}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if I have a control volume around the point &lt;em&gt;(i, j)&lt;/em&gt; Can I write the flux F on the right edge i.e. &lt;em&gt;i+$\frac{1}{2}$&lt;/em&gt; as:&#xA;$$F_{i+\frac{1}{2}, j}^n = \frac{T_{i,j}^n + T_{i+1,j}}{2}^n  - \lambda\frac{T_{i+1,j}^n - T_{i,j}^n}{dx}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this for all other fluxes, I am writing my discretized equation as (In fact, I want a final form of discretized equation to be like this):&#xA;$$T_{i,j}^{n+1} = T_{i,j}^{n} - \frac{dt}{dx}[{F^n_{i+\frac{1}{2},j} - F^n_{i-\frac{1}{2},j}}]  - \frac{dt}{dy}[{G^n_{i,j+\frac{1}{2}} - G^n_{i,j-\frac{1}{2}}}]$$&#xA;Is this definition correct or there could be some problems with this? I am not very familiar with the finite volume discretization of such hyperbolic PDEs. Is this method stable and accurate enough? I further want to use mesh refinement schemes.&lt;/p&gt;&#xA;" OwnerUserId="16397" LastEditorUserId="14772" LastEditDate="2015-09-10T23:07:35.363" LastActivityDate="2015-09-10T23:07:35.363" Title="How to define fluxes for two dimensional convection-diffusion equation?" Tags="|pde|finite-difference|finite-volume|hyperbolic-pde|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20666" PostTypeId="1" CreationDate="2015-09-08T07:10:10.327" Score="3" ViewCount="97" Body="&lt;p&gt;Given a 2D stochastic differential equation:&#xA;\begin{align}&#xA;\dot{x}_{i}=f_{i}(\textbf{x})+g_{ij}\xi_{j}(t),&#xA;\end{align}&#xA;where $i=2$, $g_{ij}g_{jk}=2\epsilon\delta_{ik}$ and $f(\textbf{x})=-\nabla\phi(\textbf{x})+l(\textbf{x})$ with $\nabla\phi(\textbf{x})\cdot l(\textbf{x})=0$, we have its action functional for trajectories:&#xA;\begin{align}&#xA;S_{T}(x(t))&amp;amp;=\int_{0}^{T}dt\frac{1}{4\epsilon}(\dot{x}_{i}-f_{i})^{2}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us now consider a 2D potential function with double well in $x$-direction: $\phi=\frac{1}{2}(y-1)^{2}-\frac{1}{2}(x-1)^{2}+\frac{1}{4}(x-1)^{4}$, and a given $l(\textbf{x})$. &#xA;I want to minimize the corresponding discretized action for the set of trajectories connecting the two lowest energy points:&#xA;\begin{align}&#xA;S_{T}(x(t))=\Delta t\sum_{k=1}^{M-1}\sum_{i=1}^{2}\Big[\frac{x_{i}^{k+1}-x_{i}^{k}}{\Delta t}-\frac{1}{2}(f_{i}^{k+1}+f_{i}^{k})\Big]^{2},&#xA;\end{align}&#xA;where the trajectory $x(t)$ is divided into $M-1$ time segments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to implement the conjugate gradient method to minimize this nonlinear action?&lt;/p&gt;&#xA;" OwnerUserId="17468" LastActivityDate="2015-09-08T07:10:10.327" Title="How to implement conjugate gradient method to minimize this nonlinear action?" Tags="|optimization|conjugate-gradient|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20667" PostTypeId="2" ParentId="20662" CreationDate="2015-09-08T11:48:16.210" Score="5" Body="&lt;p&gt;See @Kirill's answer, but depending on what sort of hardware you have available, you might be able to leverage parallelism to solve the problem more quickly. For example, say you had a way to evaluate $f$ for $P$ operands simultaneously - you could then repeatedly divide your bracket into $P+1$ intervals instead of $2$. Whether this would actually be &quot;more efficient&quot; or faster is difficult to say, as there will be overheads and various definitions of efficiency.&lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2015-09-10T08:03:08.710" LastActivityDate="2015-09-10T08:03:08.710" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20668" PostTypeId="1" CreationDate="2015-09-08T12:27:21.943" Score="1" ViewCount="58" Body="&lt;p&gt;I have a question on the use of moving mesh to solve the inviscid euler equations. I have solved the following equations: &#xA;$$\frac{\partial}{\partial t}\left[\begin{array}{c}&#xA;\rho\\&#xA;\rho u&#xA;\end{array}\right]=\frac{\partial}{\partial x}\left[\begin{array}{c}&#xA;\rho(u-w)\\&#xA;\rho u(u-w)+p&#xA;\end{array}\right],$$&#xA;where p is a linear function of $p=f(\rho)$ and $w$ is the mesh velocity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used the integral form of FVM, treated the right hand side in its conservative form and have evaluated the results at the cell centres. At the left BC I have wall conditions, so $u_L = w_L = 0$ and at the right hand BC, $u_R = w_R = f(t)$. I have used linear interpolation for the other boundary terms. I am moving the mesh using $dx/dt = w$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that moving the mesh should not effect the solution, however, my results seem to be effected since for instance for the $\rho u (u-w)$ term of the momentum equation, on the right hand side, when $w=0$, I have $\rho_R u_R u_R$ but when $w\neq 0$ I will have $\rho_R u_R (0-0) = 0$. &#xA;Can someone please let me know if I am doing the right thing here, and if it is possible to avoid this issue somehow? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="17362" LastEditorUserId="17362" LastEditDate="2015-09-08T12:46:40.997" LastActivityDate="2015-09-08T12:46:40.997" Title="should boundary conditions be effecting moving mesh results?" Tags="|numerical-analysis|ode|boundary-conditions|finite-volume|discretization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20669" PostTypeId="2" ParentId="20624" CreationDate="2015-09-08T13:06:31.347" Score="4" Body="&lt;p&gt;Let's use the example (approximate the square root function on $[0.25,1.0]$ with a quartic polynomial) to step through your calculations.  I suspect that the code is going to work with only modest changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having chosen an initial set of six interpolation points:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_k = 0.25, 0.4, 0.55, 0.7, 0.85, 1.0\;\; (k=0,\ldots,5) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we proceed to interpolate the function values at these points with a linear combination of a quartic polynomial and values $\pm 1$ that alternate at these interpolation points:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ p(x_k) + (-1) e = c_4 x_k^4 + c_3 x_k^3 + c_2 x_k^2 + c_1 x + c_0 + (-1)^k e = \sqrt{x_k} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This interpolation problem amounts to solving a linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{pmatrix} 1 &amp;amp; x_0 &amp;amp; x_0^2 &amp;amp; x_0^3 &amp;amp; x_0^4 &amp;amp; +1&#xA;                \\ 1 &amp;amp; x_1 &amp;amp; x_1^2 &amp;amp; x_1^3 &amp;amp; x_1^4 &amp;amp; -1&#xA;                \\ 1 &amp;amp; x_2 &amp;amp; x_2^2 &amp;amp; x_2^3 &amp;amp; x_2^4 &amp;amp; +1&#xA;                \\ 1 &amp;amp; x_3 &amp;amp; x_3^2 &amp;amp; x_3^3 &amp;amp; x_3^4 &amp;amp; -1&#xA;                \\ 1 &amp;amp; x_4 &amp;amp; x_4^2 &amp;amp; x_4^3 &amp;amp; x_4^4 &amp;amp; +1&#xA;                \\ 1 &amp;amp; x_5 &amp;amp; x_5^2 &amp;amp; x_5^3 &amp;amp; x_5^4 &amp;amp; -1 \end{pmatrix}&#xA;   \begin{pmatrix} c_0 \\ c_1 \\ c_2 \\ c_3 \\ c_4 \\ e \end{pmatrix}&#xA;=  \begin{pmatrix} \sqrt{x_0} \\ \sqrt{x_1} \\ \sqrt{x_2}&#xA;                \\ \sqrt{x_3} \\ \sqrt{x_4} \\ \sqrt{x_5} \end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course we will resolve this problem each time we change the interpolation points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's look at the solution for your initial set of points:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C0 =  0.195191580257&#xA;C1 =  1.506696296725&#xA;C2 = -1.365826192268&#xA;C3 =  0.938340584388&#xA;C4 = -0.274493119408&#xA; E = -9.0850E-5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You would solve for these using your &quot;bare bones matrix class&quot;.  I computed them using a spreadsheet, putting the $x_k$'s in one column &lt;code&gt;A5:A10&lt;/code&gt;, the function values $\sqrt{x_k}$ in a parallel column &lt;code&gt;C5:C10&lt;/code&gt;, then the matrix of coefficients above in a block &lt;code&gt;E5:J10&lt;/code&gt;, and finally a matrix formula for a column beyond that something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   { = MMULT(MINVERSE(E5:J10),C5:C10) }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am then able to change the interpolation points and (since everything else is given by a spreadsheet formula) quickly observe how the approximation error $e$ changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively you might think that as we converge our interpolation points to the relative extrema of approximation error on the interval, that the error $e$ will decrease in absolute value.  But the opposite will occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The oscillating error $e$ that we are able to interpolate at a non-optimal set of points is &lt;em&gt;less&lt;/em&gt; than the best error of approximation on the interval as a whole.  This is in effect why we go looking in the interval for relative extrema of the error function, the exact function minus the approximating quartic polynomial:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \text{error at } x = \sqrt{x} - p(x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to identify points &lt;em&gt;inside the interval&lt;/em&gt; where the approximation error is bigger (in absolute value) than what we achieved at the interpolation points, namely $\pm e$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is typical (but not inevitable) that the endpoints of the interval will remain among the interpolation points throughout the computation.  Such is the case here, and you kept these two endpoints in your computation.  One of the points you brought into the solution in the second iteration was $x=1.14135676154991$, but that lies outside the interval and should be ignored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take the three extrema you found inside the interval and decide what these should be swapped out with among the original points.  The basic rule is to look at the original points that bracket the new extremum, and swap with the point whose error of approximation is the same &lt;em&gt;sign&lt;/em&gt; as that of the new point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking the first extremum you found, $x_*=0.595076928220583$, we notice that it lies between our original points $0.55$ and $0.7$.  Therefore we will swap it for whichever of those has the same sign error value.  Because $e\approx -9.085\times 10^{-5}$, the error at $x_0 = 0.25$ is negative and alternates in sign with successive interpolation points.  Thus the error was again negative at $x_2 = 0.55$ and positive at $x_3 = 0.7$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We compute the error of approximation at the new point $x_* = 0.595076928220583$, something that is easily added to our spreadsheet:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align*} \text{error at } x_* &amp;amp;= \sqrt{x_*} - p(x_*) \\&#xA;                                   &amp;amp;\approx -3.0107\times 10^{-5}&#xA;\end{align*} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based solely on the criterion of matching signs of errors, we would replace old point $0.55$ with $0.595076928220583$, since both have a negative error value.  But we see that something is wrong!  This new point $x_*$ is clearly not an extremum of the error function; the error there is smaller in absolute value than at all the existing interpolation points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, let's back up and figure out what the actual extrema of the error values are.  For this purpose a graphing calculator application is helpful.  I had never used it before, but GeoGebra gets a fair amount of Community ad spots at Math.SE, and it was pretty easy to plot this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nWfZU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nWfZU.png&quot; alt=&quot;error curve on x = 0.25 to 1.0&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Visually (because high accuracy is not vital at this stage) the six points labelled &lt;code&gt;A,B,C,D,E,F&lt;/code&gt; above are the local extrema, and they become our new interpolation points:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_k = 0.25, 0.31, 0.51, 0.71, 0.92, 1.0\;\; (k=0,\ldots,5) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once these replaced the original points in the spreadsheet, the new value of &quot;equi-oscillating&quot; error value is found, $e \approx -1.6737\times 10^{-4}$.  Note that this is, as theory predicts, larger in absolute value than the &quot;level error&quot; of interpolation for the first set of points:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C0 =  0.192730711777&#xA;C1 =  1.527365848218&#xA;C2 = -1.421661988977&#xA;C3 =  0.999188048845&#xA;C4 = -0.297789990497&#xA; E = -1.6737E-04&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Lather,_rinse,_repeat&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lather, rinse, repeat.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2015-09-08T13:06:31.347" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20670" PostTypeId="2" ParentId="8193" CreationDate="2015-09-08T14:43:28.380" Score="1" Body="&lt;p&gt;Use the MKL link line advisor to determine what library that corresponds to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor&quot; rel=&quot;nofollow&quot;&gt;https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you choose &quot;Single Dynamic Library&quot;, it uses &quot;mkl_rt.lib&quot; and it looks like threading is active.&lt;/p&gt;&#xA;" OwnerUserId="7681" LastActivityDate="2015-09-08T14:43:28.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20671" PostTypeId="1" AcceptedAnswerId="20688" CreationDate="2015-09-08T19:17:01.457" Score="3" ViewCount="530" Body="&lt;p&gt;I'm new to the modelling of inviscid euler equations. I have come across few different upwind schemes that are used instead of central differencing schemes to model such flows, such as flux vector splitting and Godunov. My question is, do I definitely need to know the eigenvectors/eigenvalues of the system if I need to use an upwind scheme for such flow? Is there a way just to apply the normal first order upwind to solve such equations? If so, how is the direction of the flow decided on? Thank you!  &lt;/p&gt;&#xA;" OwnerUserId="17476" LastEditorUserId="17476" LastEditDate="2015-09-09T11:03:43.453" LastActivityDate="2015-09-10T11:19:37.783" Title="upwind schemes for solving inviscid euler equations" Tags="|fluid-dynamics|discretization|hyperbolic-pde|numerical-modelling|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20672" PostTypeId="2" ParentId="20141" CreationDate="2015-09-08T21:57:05.613" Score="1" Body="&lt;p&gt;Looks like you may want to use a Halton Sequence or a Sobol Sequence.  These are sets of points distributed in a square, looking random, but avoiding any two being too close together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The articles on wikipedia are a good start.  &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2015-09-08T21:57:05.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20673" PostTypeId="2" ParentId="20665" CreationDate="2015-09-08T22:05:02.040" Score="1" Body="&lt;p&gt;Edited after some corrections in the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One has to decide at which time level you evaluate the fluxes, i. e.  to add also the index $n$ or $n+1$ to the values of $T, F, G$. The scheme is then explicit (in time) or implicit (in time).  You took the explicit one. After your corrections the scheme looks like standard Finite Volume Method, see e. g.  the book of LeVeque on &quot;FVM for hyperbolic problems&quot;. By the way for positive lambda your equation is parabolic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scheme is stable in general depending on the time discretization, on how large is $\lambda$ or how fine is your mesh. You may use an upwind scheme for the convective part of your fluxes to improve the stability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explicit scheme to be stable you have the restriction on the time step $\delta t $ that must be proportional to $dx^2/\lambda$ or $dy^2/\lambda$ (which one is smaller). If you would choose the implicit scheme with the first order accurate upwind you have no stability restriction on $\delta t$, but you have to solve a system of linear equations with tridiagonal matrix.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2015-09-09T13:45:07.107" LastActivityDate="2015-09-09T13:45:07.107" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20674" PostTypeId="1" AcceptedAnswerId="20994" CreationDate="2015-09-08T22:22:38.167" Score="1" ViewCount="406" Body="&lt;p&gt;I urgently need help with a problem I am having for an assignment for university. I do not expect a full solution, but I am really in need of guidance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am required to write a FEA program in VBA, I have  managed to do that. However, I am having problems when trying to find the inverse transformation of isoparametric quadrilaterals. We require this transformation (from x,y global to a,b local element) to calculate the hydraulic gradients at a point in the element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a system of 2 nonlinear equations with two unknowns:&#xA;$$f_1=\alpha_x+a\beta_x+b\gamma_x+ab\delta_x-C_x$$&#xA;$$f_2=\alpha_y+a\beta_y+b\gamma_y+ab\delta_y-C_y$$&#xA;The only unknowns are a and b. All other symbols are constant per element.&#xA;At first, I tried to solve these via substitution, but soon realised that was a suicide mission. I then opted to solve for the system using matrices and the Newton Method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, however isn't working. Whenever I iterate on the method, the values become unrealistic and clearly don't converge. I am not sure if I made an arithmetic error, but I have been over it by hand and by analysis multiple times without success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would truly appreciate any expertise on the matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is quite an urgent request as well, so haste would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19419/inverse-isoparametric-mappings-for-quadrilateral-finite-elements&quot;&gt;this question&lt;/a&gt; on this site, but it did not help me much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional information:&lt;/strong&gt; This Newton method works when dealing with non-skew elements, i.e. perfect squares and rectangles. When this is the case, only one iteration is required to achieve the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the elements are skewed, the initial result is incorrect, and further results are even worse.&lt;/p&gt;&#xA;" OwnerUserId="17482" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-10-31T00:33:16.783" Title="Inverse Transformation of Isoparametric Quadrilaterals" Tags="|finite-element|newton-method|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20676" PostTypeId="1" CreationDate="2015-09-08T23:27:20.403" Score="2" ViewCount="385" Body="&lt;p&gt;I'm implementing a finite-difference WENO scheme for a simple advection equation&#xA;$$&#xA;\frac{\partial u}{\partial t} + &#xA;\frac{\partial u}{\partial x} = 0&#xA;$$&#xA;with periodic boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not quite sure my scheme is conventional since it is mixing ideas from finite-difference and finite-volume methods in the following way: a time step consists of the reconstruction step and the evolution step. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;During the reconstruction step on the $i$-th interval the solution is approximated with&#xA;$$&#xA;U_i(x) = \frac{\sum_{k=1}^3 \alpha_k \gamma_k(x) P_i^{(k)}(x)}{\sum_{k=1}^3 \alpha_k \gamma_k(x)}&#xA;$$&#xA;where $P_i^{(k)}(x)$ are the quadratic interpolants for the $k$-th small stencil, $\gamma_k(x)$ are the blending functions, so $\sum_k \gamma_k(x)P_i^{(k)}(x)$ is the (unique) fourth order intepolant passing through the points $(x_{i-2}, u_{i-2}),\dots,(x_{i+2}, u_{i+2})$. Finally, $\alpha_k$ are related to the smoothness indicators $\beta_k$ as&#xA;$$&#xA;\alpha_k = \frac{1}{(\beta_k + \epsilon)^2}.&#xA;$$&#xA;Actually, the conventional $u_{i+1/2}$ is simply the value of $U_i(x)$ at $x = x_{i+1/2}$.&lt;/li&gt;&#xA;&lt;li&gt;The evolution step is done in upwind manner using the first order Euler method:&#xA;$$&#xA;\frac{u_i^{n+1} - u_i^n}{\Delta t} + \frac{U_i(x_{i+1/2}) - U_{i-1}(x_{i-1/2})}{\Delta x} = 0.&#xA;$$&#xA;The CFL number $\frac{\Delta t}{\Delta x}$ was taken to be $0.15$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In my numerical experiments I observe instability (in the right half of the plot). It occurs after different number of timesteps, depending on the number of spatial intervals and the CFL number. The number of intervals $N$ was taken to be $N = 100$ and $20N$ time steps was done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a feeling that explicit first order Euler integrator is unstable for the problem. I recall similar behavior of DG methods where the Euler time integrator is unstable if the spatial polynomial degree is greater than 1. What time integrator is suitable for the problem? Is it sufficient to use third order TVD Runge-Kutta or I need a fifth order one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/uranix/9fc7dbec079535e53e2c&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is the code if someone is interested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Kt3Bt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kt3Bt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kb0IZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kb0IZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17125" LastActivityDate="2015-09-09T23:37:20.397" Title="Time integrator for a WENO scheme for advection equation" Tags="|time-integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20677" PostTypeId="1" CreationDate="2015-09-09T10:56:14.580" Score="1" ViewCount="800" Body="&lt;p&gt;I am trying to solve two non-linear equations self-consistently in a Gummel loop. Sometimes (every once in a while), I get to a situation when the loop repeats itself with wrong solutions and a certain error persists. As a simple example, consider the following two equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;y = -x + 1 \\&#xA;&amp;amp;y = \sqrt{x}&#xA;\end{align}&#xA;and suppose the loop reaches to $x=0$ for the input of first equation, which leads to $y=1$ for the input of the second equation. This results in $x=1$ for the input of the first equation, leading to $y=0$, and the situation repeats itself (and of course, does not converge to a correct solution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is a good and comprehensive reference on this particular problem and on the properties of equations which lead to such behaviour. Also, what is the best way to avoid such difficulties in general?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2015-10-09T13:13:15.477" Title="Convergence problem in iterative method" Tags="|iterative-method|nonlinear-equations|convergence|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20678" PostTypeId="2" ParentId="20677" CreationDate="2015-09-09T11:37:42.123" Score="1" Body="&lt;p&gt;In general, the iterative method is not required to converge. This is commonly discussed in numerical analysis textbooks in relation to nonlinear root-finding (e.g., &lt;a href=&quot;https://books.google.ca/books?id=y77n2ySMJHUC&amp;amp;pg=PA86&quot; rel=&quot;nofollow&quot;&gt;here it is in Analysis of Numerical Methods by Isaacson and Keller, p.85--&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are iterating the function&#xA;$$ f(x) = (x-1)^2 $$&#xA;and solving the equation $x = f(x)$ with the roots $x_{1,2} = \frac12(3\pm \sqrt{5}) = 0.38, 2.62$. (This form of the equation is easier to analyse than your version with both variables $x$ and $y$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the iterative method to succeed, according to a standard theorem, it is sufficient for the initial guess to lie in (or to eventually converge to a value in) an interval $(x_-,x_+)$ surrounding a root such that $|f'(x)|\leq \lambda &amp;lt; 1$ for each $x\in(x_-,x_+)$.&#xA;Since $f'(x) = 2(x-1)$, we can see that near the roots $|f'(x)|&amp;gt;1$ ($f'(x_{1,2}) = -1.24, 3.24$). This means that each root is unstable: for each root the interval surrounding it from which initial guesses are guaranteed to converge is empty.&#xA;This unstable behaviour is much easier to see if you start from an initial guess that is already near a root, $x_{1,2}+\epsilon$, instead of $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This issue can be solved, for example, by rewriting the equations in an equivalent form that happens to lead to stable roots.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-09-09T12:57:23.963" LastActivityDate="2015-09-09T12:57:23.963" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20679" PostTypeId="2" ParentId="20671" CreationDate="2015-09-09T13:07:10.830" Score="1" Body="&lt;p&gt;I have experiences with finite volume methods and upwind schemes for nonlinear scalar hyperbolic equations (e.g. $du/dt+df(u)/dx=0$), but not particularly for Euler system of equations. I have checked my sources of literature for a curiosity and after some time my impression is that the most of methods are based on the solution of related Riemann problem, where you can not avoid the computations of eigenvalues (but they are given analytically for standard form of Euler equations). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not found a source where the implementation details would be given in such a form that one may recommend it to someone, say, to realize it easily. If I have to choose one source that is close to such goal (and brief enough), I would choose&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://bender.astro.sunysb.edu/hydro_by_example/compressible/Euler.pdf&quot; rel=&quot;nofollow&quot;&gt;http://bender.astro.sunysb.edu/hydro_by_example/compressible/Euler.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There you find the properties of Euler equation in section 1 on two pages including the eigenvalues, the method based on Riemann problem in section 2 on two pages (at the beginning the first order method 2.1 using a piecewise constant form of numerical solution should be enough) and, finally, the section 3 on Riemann problem and 4 on conservative update.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a direct answer to your question is that I do not expect that you can use an upwind method without using the eigenvalues. Maybe someone else can give better recommendation or different view that I would also appreciate.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2015-09-09T13:07:10.830" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20680" PostTypeId="1" CreationDate="2015-09-09T13:57:35.977" Score="2" ViewCount="543" Body="&lt;p&gt;I Need to to solve an generalized Eigenvalue Problem and to compare two Methods (QR and QZ) concerning their convergence rate and execution time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started with the Basic QR-Algorithm, implemented in Matlab, tested the results and in the last step i included double-shifts and Deflation strategies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now it is time to improve my QZ-Algorithm: I think ist way more complicated, as i did not find any QZ-Algorithm implementations in the Internet. So i grabed the paper of Moler and Stewart and wrote my own implementation. I've been very glad that i did manage to write an algorithm producing the correct results. To have a fair comparison i would like to enhance this algorithm as to include single/double shifts so i can compare the Performance between the QR- and QZ-Algorithm. I think the implicit shift should be more stable, but i want the fastest Solution (less iterations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find the paper here: &lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/746896.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.dtic.mil/dtic/tr/fulltext/u2/746896.pdf&lt;/a&gt;&#xA;And an extensional paper here: &lt;a href=&quot;http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19730018792.pdf&quot; rel=&quot;nofollow&quot;&gt;http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19730018792.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my algorithm so far (works correctly, but without shifting, so it is quite slow):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [lambda, iteration] = QZAlgo(A, B, precision, maxIterations, useHouseholder)   &#xA;    len = length(A);&#xA;&#xA;    % 1. Step: Preparation of Matrices&#xA;    % Transform A to Hessenberg form and B to upper triangular form&#xA;    if (useHouseholder ~= 0)&#xA;        [Q, B] = TriangularFormByHouseholder(B);&#xA;    else&#xA;        [Q, B] = TriangularFormByGivens(B);&#xA;        Q = Q';&#xA;    end&#xA;    A = Q*A;&#xA;    [A, B] = HessenbergFormByGivensWhileMaintainingB(A, B);&#xA;&#xA;    % 2. Step: QZ-Iteration&#xA;    for iteration=1:maxIterations      &#xA;        p = 1;&#xA;        S = A - p*B;&#xA;&#xA;        if (useHouseholder ~= 0)&#xA;            [Q, R] = TriangularFormByHouseholder(S);&#xA;        else&#xA;            [Q, R] = TriangularFormByGivens(S);&#xA;            Q = Q';&#xA;        end&#xA;&#xA;        B1 = Q*B;&#xA;        [B2, Z] = RepairTriangularFormByGivensFromRight(B1);&#xA;&#xA;        B = Q*B*Z;&#xA;        A = Q*A*Z;&#xA;&#xA;        lambda = diag(A) ./ diag(B);&#xA;&#xA;        if (CheckForRequiredPrecision(A, B, precision))&#xA;            break;&#xA;        end&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Triangular-Functions are basically QR-Decompositions. The HessenbergFormByGivensWhileMaintainingB-Function creates an QR-Decomposition of A while repairing changes of B with Givens-Transformations of the right side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Algorithm works so far, but is, as already mentioned, quite slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do anybody know how i can implement some shifting-strategies to the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ANY help, no matter of what Kind and no matter about which shifting strategy, is GREATLY appreciated. I will spend you a crate  of beer of your choice, if we can manage to get this working, if you like ;-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in Advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: If you Need to have a look in some of the subfuctions, i can provide them here, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2: I tried to have a look at CLAPACK (&lt;a href=&quot;http://www.netlib.org/clapack/&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/clapack/&lt;/a&gt;) to figure it out, but the Code is so confusing to me that i had to give it up...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT3: Some of my thoughts about the explicit shift&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Choose shift p: I've Chosen the element in the right bottom Corner of S=A-B as i thought it must be similiar to the procedure of the QR-Algorithm&lt;/li&gt;&#xA;&lt;li&gt;Calculate QR-Decomposition of A-p*B&lt;/li&gt;&#xA;&lt;li&gt;Calculate B'=Q&lt;em&gt;B&lt;/em&gt;Z just the same way i did above&lt;/li&gt;&#xA;&lt;li&gt;Calculate A=SZ + p&lt;em&gt;B = Q(A-p&lt;/em&gt;B) + p*B'&#xA;But this Procedure did not work. I am clueless at this Point :-(&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16641" LastEditorUserId="16641" LastEditDate="2015-09-09T14:16:39.280" LastActivityDate="2015-11-09T00:02:51.720" Title="Improving my QZ-Algorithm (Include Shifts)" Tags="|numerical-analysis|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20681" PostTypeId="2" ParentId="20141" CreationDate="2015-09-09T21:02:13.807" Score="1" Body="&lt;p&gt;You can check &lt;a href=&quot;https://www.jasondavies.com/poisson-disc/&quot; rel=&quot;nofollow&quot;&gt;Poisson-disc&lt;/a&gt;. It produces points that are tightly-packed, but no closer to each other than a specified minimum distance.&lt;/p&gt;&#xA;" OwnerUserId="17493" LastEditorUserId="9667" LastEditDate="2015-09-10T21:35:14.430" LastActivityDate="2015-09-10T21:35:14.430" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20682" PostTypeId="2" ParentId="20680" CreationDate="2015-09-09T23:08:58.617" Score="-1" Body="&lt;p&gt;Why don't your consider the &quot;eigs&quot;, a built-in function in MATLAB ? Are your matrices nonsymmetric? If symmetric, maybe you can also choose FEAST solver or JDQR.m &lt;a href=&quot;http://www.staff.science.uu.nl/~vorst102/JDQR.html&quot; rel=&quot;nofollow&quot;&gt;http://www.staff.science.uu.nl/~vorst102/JDQR.html&lt;/a&gt; for solving your generalized eigenvalue problems.&lt;/p&gt;&#xA;" OwnerUserId="10290" LastActivityDate="2015-09-09T23:08:58.617" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20684" PostTypeId="1" CreationDate="2015-09-09T23:31:24.773" Score="1" ViewCount="290" Body="&lt;p&gt;I am having problems with solving a hyperbolic wave problem with Dirichlet BCs. I have tried reducing the time step sizes, which does not affect the results, and notices increasing the number of nodes makes the results worse. I have concluded the problem is with the boundary or certainly spatial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is 1D, and the grid is structured. I have used FVM with central differencing. At the boundaries I have simply set the term at the boundary equal to the Dirichlet condition (for instance: velocity at east face $v_e = A$, where $A$  is the Dirichlet BC).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there are any ways using which I could get rid of the oscillations in my results by means of changing the way I am implementing the BC or any other quick fixes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also tried upwinding, and did not even obtain convergence. &lt;/p&gt;&#xA;" OwnerUserId="17476" LastEditorUserId="17671" LastEditDate="2016-01-16T17:14:15.033" LastActivityDate="2016-01-16T17:14:15.033" Title="Dirichlet BCs - alternative implementation methods" Tags="|finite-difference|boundary-conditions|finite-volume|discretization|wave-propagation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20685" PostTypeId="2" ParentId="20676" CreationDate="2015-09-09T23:37:20.397" Score="1" Body="&lt;p&gt;I've found this article &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/050637868&quot; rel=&quot;nofollow&quot;&gt;Linear Instability of the Fifth-Order WENO Method&lt;/a&gt; by Rong Wang and Raymond J. Spiteri. It is shown that explicit RK methods are not stable unless their stability region covers a segment $[-\mu i, \mu i], \mu &amp;gt; 0$ on the imaginary axis. I have successfully implemented 5-stage 4-th order strong stability preserving explicit RK method by Spiteri and Ruuth and the instability went away.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I've fount that mixing finite-volume and finite-difference approach results in at best second-order accurate method. So I've completely rewritten the method as finite-volume one, but it seems that simple (polynomial) $\gamma(x)$ exist only for FDM and for FVM they are rational functions. Anyway, $\gamma(x)$ were used only to plot piecewise-smooth WENO approximant and for the numerical method only values at endpoints $\gamma(x_{i\pm 1/2})$ are needed.&lt;/p&gt;&#xA;" OwnerUserId="17125" LastActivityDate="2015-09-09T23:37:20.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20687" PostTypeId="2" ParentId="20620" CreationDate="2015-09-10T04:41:43.750" Score="0" Body="&lt;p&gt;Commercial Software:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Matlab&lt;/li&gt;&#xA;&lt;li&gt;Mathematica&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Free Open Source Tools:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Matplotlib(Python)&lt;/li&gt;&#xA;&lt;li&gt;GNUPlot&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16423" LastActivityDate="2015-09-10T04:41:43.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20688" PostTypeId="2" ParentId="20671" CreationDate="2015-09-10T05:46:06.113" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Do I definitely need to know the eigenvectors/eigenvalues of the system if I need to use an upwind scheme for such flow?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, it is certainly not necessary to use the full eigenstructure of the system, especially for lower order (order 1-2) schemes.  Some of the most commonly used Riemann solvers are Lax-Friedrichs, HLL, HLLE, and HLLC (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Riemann_solver&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;).  Lax-Friedrichs and HLL only require a rough upper bound on the eigenvalues, and the other two only use a little more information; none of them use the eigenvectors.  These solvers are also used implicitly in the so-called central schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the eigenstructure is written down for you in many references, so &quot;knowing&quot; it is easy.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a way just to apply the normal first order upwind to solve such equations?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The eigenvalues (wavespeeds) of the 1D Euler equations are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u-c, u, u+c$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is the local fluid velocity and $c$ is the sound speed.  For subsonic flow, these three speeds will not all have the same sign, so the upwind direction is different for different characteristic fields.  Thus application of an upwind method requires knowing the eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're asking this question, then you would definitely benefit from reading &lt;a href=&quot;https://books.google.com.sa/books?id=zkLtCAAAQBAJ&amp;amp;pg=PA584&amp;amp;lpg=PA584&amp;amp;dq=toro+euler&amp;amp;source=bl&amp;amp;ots=SE9wAw-YY-&amp;amp;sig=ZE4GnqVL020dSne5IgLtyWCKJwo&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0CCsQ6AEwAmoVChMIlO2jiazsxwIVh5caCh3cTg2b&quot; rel=&quot;nofollow&quot;&gt;Toro's book&lt;/a&gt; or &lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot; rel=&quot;nofollow&quot;&gt;LeVeque's book&lt;/a&gt;.  It's difficult to respond to your comments below in a meaningful way unless you are more familiar with the basic discretizations that are used for these equations.  You are trying to extrapolate your knowledge from discretizations of the advection equation, and there is a significant gap.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-09-10T11:19:37.783" LastActivityDate="2015-09-10T11:19:37.783" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20690" PostTypeId="1" CreationDate="2015-09-10T10:03:52.947" Score="1" ViewCount="196" Body="&lt;p&gt;I'm writing up a code to calculate the local energy for electrons in hydrogen like ions for a given wavefunction. My code is giving me weird results, which leads me to believe something is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, am I going about accepting/rejecting moves the right way? And am I doing anything weird in the accumulation phase which gives me the wrong answer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I know from mathematica that my local energy for the trial wavefunction I'm using should be -0.48, however I keep getting a whole splash of results, with little consistency (eg 0.6, -0.38, 0.7 etc.). Am I doing anything weird?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: sorry about the hard to read code, I'm new to this coding thing and am not very sure I'm doing the right thing. Essentially, what I am trying to do is sample my local energy with the probability distribution of my trial wavefunction. I'm not sure if I'm accumulating the sampled energy correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for reference here is my code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;import random &#xA;&#xA;def psi(r): #my trial wavefunction&#xA;    return (math.exp(-1.2*r))&#xA;&#xA;#hamiltonian &#xA;def Ham(r): #the hamiltonian operator&#xA;    return (1.44*psi(r)+(2/r)*(-1.2*psi(r)))&#xA;def Top(r): #numerator of the El expression&#xA;return (-0.5*psi(r)*Ham(r)*r*r)&#xA;def But(r): #denomenator of the El expression&#xA;    return (psi(r)*psi(r)*r*r)&#xA;def Elocal(r): #defines the local energy&#xA;    return (Top(r)/But(r))&#xA;&#xA;def r(x,y,z): &#xA;    return math.sqrt(math.pow(x,2)+math.pow(y,2)+math.pow(z,2))&#xA;&#xA;x0 = random.uniform(0, 50)&#xA;y0 = random.uniform(0, 50)&#xA;z0 = random.uniform(0, 50)&#xA;&#xA;#local energy&#xA;El = 0&#xA;&#xA;#my movement weight&#xA;dr = 0.5&#xA;&#xA;#begin equilibriate&#xA;n = 0&#xA;for n in range (1000):&#xA;&#xA;    dx = random.uniform(-1,1)*dr&#xA;    dy = random.uniform(-1,1)*dr&#xA;    dz = random.uniform(-1,1)*dr&#xA;&#xA;    x = x0+dx&#xA;    y = y0+dy&#xA;    z = z0+dz&#xA;&#xA;    dpsi = psi(r(x,y,z))&#xA;    opsi = psi(r(x0,y0,z0))&#xA;&#xA;    W = (dpsi*dpsi)/(opsi*opsi)&#xA;&#xA;    if W &amp;gt;= random.random():&#xA;        x0 = x&#xA;        y0 = y&#xA;        z0 = z&#xA;    else:&#xA;        x0 = x0&#xA;        y0 = y0&#xA;        z0 = z0&#xA;    n = n+1&#xA;# begin accumulate&#xA;w = 0&#xA;N = 0&#xA;for N in range (1000):&#xA;&#xA;    dx = random.uniform(-1,1)*dr&#xA;    dy = random.uniform(-1,1)*dr&#xA;    dz = random.uniform(-1,1)*dr&#xA;&#xA;    x = x0+dx&#xA;    y = y0+dy&#xA;    z = z0+dz&#xA;&#xA;    dpsi = psi(r(x,y,z))&#xA;    opsi = psi(r(x0,y0,z0))&#xA;&#xA;    W = (dpsi*dpsi)/(opsi*opsi)&#xA;&#xA;    if W &amp;gt;= random.random():&#xA;        x0 = x&#xA;        y0 = y&#xA;        z0 = z&#xA;        El = El+Elocal(r(x,y,z))&#xA;        w = w+1&#xA;    else:&#xA;        x0 = x0&#xA;        y0 = y0&#xA;        z0 = z0&#xA;        El = El&#xA;    N = n+1&#xA;print El/w&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17498" LastEditorUserId="17498" LastEditDate="2015-09-12T02:23:53.733" LastActivityDate="2015-09-12T02:23:53.733" Title="Variational Monte Carlo to calculate local energy of hydrogen like ions in python" Tags="|monte-carlo|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20691" PostTypeId="1" AcceptedAnswerId="20693" CreationDate="2015-09-10T14:09:58.583" Score="3" ViewCount="1013" Body="&lt;p&gt;I need to compute some integrals numerically. The integrand is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x,y) = \left ( \sum_{mn=-j}^{j}A(m,n)\dfrac{\tan^{2j+m+n}(x/2)}{(1+\tan^2(x/2))^{2j}}e^{iy(n-m)} \right )^{N}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: sums are finite. And I have to compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_0^\pi \int_0^{2\pi}f(x,y)dydx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to do this for different values of $N$. I was working in C++ with a simple integrator, but the problem is that the limits in $x$ are $[0,\pi]$. When $j$ is big and $x\rightarrow\pi$, I have overflows/underflows everywhere. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried to use Python/Scipy, with the &lt;code&gt;dblquad&lt;/code&gt; function. However, it takes forever to compute the integral. The results aren't good. I have to compute it more or less a hundred of times, so I need something reasonably fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm really stuck with this. Any idea to get a numerical result from the integral is well received.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; &#xA;Common numerical values as requested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$j\in[20,150]$$&#xA;$$n,m\in[-j,j]$$&#xA;$$N=1,2,...,5$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That means that we have up to ~$\tan^{4jN}(x/2)$ both in numerator/denominator. When $x$~$\pi$ that becomes huge! I've obtained a lot of NaNs in C++.&#xA;$A(m,n)$ is simply a coefficient obtained from some non-zero constants. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Important note: althought the term $e^{iy(n-m)}$ is there, $f(x,y)\in\mathbb{R}$. Imaginary parts always dissapears. &lt;/p&gt;&#xA;" OwnerUserId="17425" LastEditorUserId="17425" LastEditDate="2015-09-10T14:46:51.127" LastActivityDate="2015-09-10T16:46:29.007" Title="Numerical integration with singularities" Tags="|quadrature|scipy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20692" PostTypeId="1" AcceptedAnswerId="20695" CreationDate="2015-09-10T15:56:36.973" Score="2" ViewCount="518" Body="&lt;p&gt;I have an absolute value optimisation problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_x \sum |r-Cx|$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x$ is small around 200 dimension. But $C$ has lots of rows, $C_{30000\times200}$ and $r$ is $30000\times1$. So this will introduce large number of help variables for the absolute value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone recommend a package in Python that can solve it efficiently? I tried CVXOPT but it took 3 hours to solve a slimmed version of 5000 x 200.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it generally quite slow to solve this kind of problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Turns out that CVXOPT solving is not bad at all! Solving time in GLPK is just 12 seconds! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What has taken long is CVX modelling. I used their integrated modelling module&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from cvxopt.modeling import variable, op, dot, matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y = abs(r-C*x) # quick&#xA;objfun = sum(y) # this line takes ages&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suppose the reason the last line is slow is that it is checking convexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By converting the problem myself&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min \sum v\quad s.t.\,-v\leq r-Cx\leq v$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it's now good&lt;/p&gt;&#xA;" OwnerUserId="14305" LastEditorUserId="14305" LastEditDate="2015-09-11T08:12:38.213" LastActivityDate="2015-09-11T08:12:38.213" Title="Python package for large absolute value optimisation" Tags="|python|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20693" PostTypeId="2" ParentId="20691" CreationDate="2015-09-10T16:04:39.133" Score="3" Body="&lt;p&gt;I think there are two things that could be done here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, the way you evaluate &#xA;$$ s(x) = \frac{\tan^{2j+m+n}\frac x2}{(1+\tan^2\frac{x}{2})^{2j}} $$&#xA;seems to be unstable, even though we always have $0\leq s(x)\leq 1$. Thus the overflows can be addressed by rewriting this expression as&#xA;$$ \frac{z^{2j+m+n}}{(1+z^2)^{2j}} = \frac{(1/z)^{2j-m-n}}{(1+z^{-2})^{2j}}, $$&#xA;and using this alternative expression when $z = \tan\frac{x}{2} &amp;gt; 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the oscillatory nature of the integral will lead to trouble when trying to integrate it using an adaptive quadrature routine like scipy's quad (which uses QUADPACK). The problem is that the integration error for the integrand $f(y)$ will behave as some high derivative $f^{(m)}$, which for an oscillatory function $e^{i y p}$ will be large, proportional to $p^m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the way to address this is to use trapezoid rule, which is very  accurate when applied to periodic functions. I think an appropriate method might be to write the integral as&#xA;$$ \approx \int_0^\pi \left( 2\pi N_y^{-1}\sum_{k=0}^{N_y-1} f(x, 2\pi k/N_y) \right)\,dx, $$&#xA;and evaluate the remaining one-dimensional integral using &lt;code&gt;quad&lt;/code&gt;; a suitable value of $N_y$ should be found experimentally/adaptively, it should be quite small - I'm only writing it this way because it seems scipy doesn't already include an adaptive trapezoid quadrature function. The double-exponential rule (which isn't used by quad) is also a good candidate to try here for the outer integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the third thing to try is the change of variable $t = \tan\frac{x}{2}$, with&#xA;$$ \frac{2}{1+t^2}\,dt = dx, \qquad \int_0^\pi (\cdot)dx = \int_0^\infty (\cdot)\frac{2 dt}{1+t^2}.  $$&#xA;This might make it easier for &lt;code&gt;quad&lt;/code&gt; to deal with the function, though I can't be sure without checking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code I wrote in Mathematica to make sure any of this works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Clear[a, f, x2, g1, g2]&#xA;a[m_, n_] := 1/(m^2 + n^2 + 1)&#xA;x2[j_, s_, t_?NumericQ] := &#xA; If[t &amp;gt; 1, (1/t)^(2 j - s)/(1 + (1/t)^2)^(2 j), t^(2 j + s)/(1 + t^2)^(2 j)]&#xA;f[j_, q_, x_, y_] := &#xA; Sum[a[m, n] x2[j, m + n, Tan[x/2]] Exp[I y (m - n)], {m, -j, j}, {n, -j, j}]^q // Re&#xA;g1[j_, q_] := NIntegrate[f[j, q, x, y], {x, 0, \[Pi]}, {y, 0, 2 \[Pi]}]&#xA;g2[j_, q_] := Module[{g},&#xA;  g[x_?NumericQ] := &#xA;   NIntegrate[f[j, q, x, y], {y, 0, 2 \[Pi]}, Method -&amp;gt; &quot;TrapezoidalRule&quot;];&#xA;  NIntegrate[g[x], {x, 0, \[Pi]}, Method -&amp;gt; &quot;DoubleExponential&quot;]&#xA;  ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It evaluated $j=20, N=2$ as &lt;code&gt;3.988585896935342e-6&lt;/code&gt; in about 26s, with relative error $7.9\times 10^{-8}$ (checked with extra precision).&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-09-10T16:46:29.007" LastActivityDate="2015-09-10T16:46:29.007" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20694" PostTypeId="1" AcceptedAnswerId="20717" CreationDate="2015-09-10T16:20:34.727" Score="3" ViewCount="1366" Body="&lt;p&gt;Given variables $a_1$, $b_1$, $c_1$ and $a_2$, $b_2$, $c_2$ representing the vertices of two plane triangles, how might one specify the requirement for the two triangles to intersect as an objective in a mathematical programming solver such as &lt;code&gt;CPLEX&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="17504" LastEditorUserId="17504" LastEditDate="2015-09-12T07:29:04.223" LastActivityDate="2017-03-27T18:18:09.740" Title="Mathematical programming formulation of triangle intersection" Tags="|computational-geometry|convex-optimization|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20695" PostTypeId="2" ParentId="20692" CreationDate="2015-09-10T16:21:26.530" Score="3" Body="&lt;p&gt;This problem is easy to convert into a linear programming problem which will have 60,000 constraints and 60,000 (slack variables) + 200 (x variables.)  I assume that the $C$ matrix is fully dense, but the problem is otherwise sparse.  This should be solvable by a reasonably good LP solver.  I'm somewhat surprised that CVXOPT had such trouble with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You weren't clear in your question about how exactly you solved this in CVXOPT.  Were you using your own LP formulation with the MOSEK solver?  GLPK?  Were you using the l1() or l1blas() function in CVXOPT (these internally solve the LP formulation)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, since you're just solving  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\min \| r-Cx \|_{1}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and not a more general LP, it really isn't necessary to formulate this problem as an LP.  There are many other algorithms for solving this 1-norm minimization problem including iteratively reweighted least squares (IRLS) and subgradient descent methods.  Solving the LP formulation is generally not the most efficient approach.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-09-10T17:23:23.327" LastActivityDate="2015-09-10T17:23:23.327" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20697" PostTypeId="1" AcceptedAnswerId="20718" CreationDate="2015-09-10T16:55:45.427" Score="2" ViewCount="216" Body="&lt;p&gt;Are upwind schemes such as Godunov type methods superior to central differencing schemes? Do the reasons include superiority in modelling hyperbolic problems with Dirichlet BC's? &lt;/p&gt;&#xA;" OwnerUserId="17476" LastEditorUserId="713" LastEditDate="2015-09-10T17:21:01.837" LastActivityDate="2015-09-12T13:03:27.930" Title="Why have specialised upwind schemes been developed to solve hyperbolic equations?" Tags="|fluid-dynamics|numerical-analysis|stability|discretization|hyperbolic-pde|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20698" PostTypeId="2" ParentId="20694" CreationDate="2015-09-10T18:13:26.233" Score="1" Body="&lt;p&gt;For each triangle edge, add a linear constraint corresponding to the equation of the line containing the edge such that the other point is on the correct side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if the line containing the edge $a_1b_1$ is given by $L(x,y)=0$, then add the constraint $L(c_1)L(x,y) \ge 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L(x,y)$ is given by $((x,y)-a_1) \times (b_1-a_1)$, where $\times$ is the 2-dimensional vector product.&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2015-09-10T18:13:26.233" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20701" PostTypeId="2" ParentId="20661" CreationDate="2015-09-11T08:09:22.337" Score="1" Body="&lt;p&gt;With the discussion clearing up some of the confusion in the original post, here is a summary of the discussion so far:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to implement an implicit ODE solver, but don't know what to do when the differential equations (DEs) have discontinuities of the form:&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;More common type: \begin{align}\lim_{x\rightarrow 0}\frac{\sin{x}}{x}\end{align}&lt;/li&gt;&#xA;  &lt;li&gt;Less common, as it occurs in only one DE \begin{align}\min(\alpha x,\beta)\end{align}&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As Kirill points out, the first type of &quot;discontinuity&quot; is removable, which you can prove in this case either using the Squeeze Theorem or L'Hôpital's Rule. Without more specific information, L'Hôpital's Rule is one of the more general strategies for resolving these types of limits, so it's a good heuristic strategy to consider for other parts of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second type of &quot;discontinuity&quot; is really more a point of nondifferentiability.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am trying to implement this myself because it is a set of massively coupled, identical DEs and we want to run them, because of our pipeline, on GPUs in Matlab. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't know of any libraries that implement ODE solvers on GPUs in MATLAB. &lt;a href=&quot;https://code.google.com/p/simengine/&quot; rel=&quot;nofollow&quot;&gt;SimEngine&lt;/a&gt; is an MIT-licensed MATLAB toolbox that seems to implement an ODE solver for the GPU. It also seems to be unsupported (the company that wrote it does not appear to exist anymore, and its website no longer exists).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't recommend implementing this type of software yourself unless you really know what you're doing. As I've said multiple times in the comments, there are BSD-licensed libraries (such as SUNDIALS) that you might be able to modify for your purposes.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I considered adapting/modifying Matlab ode15 but I would still want exact evaluations of the Jacobian, since the system has about 10k state variables, so doing numerical approximations to the Jacobian would be expensive.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You don't necessarily need an &lt;em&gt;exact&lt;/em&gt; Jacobian matrix, but you do need a function that supplies a good enough approximation to the Jacobian matrix using some analytical expression. (W-methods do not require an exact Jacobian matrix, and in practice, many libraries implement methods that only require sufficiently accurate Jacobians.) You are correct that evaluating the Jacobian via finite differencing would be expensive.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have generally considered restarting the ODE solver after the discontinuity but I am not sure how to do that in practice: do I define an $\epsilon$ region around the discontinuity and then simply check if the solver reached it, and then restart?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You have two general options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Ignore discontinuities, in which case your ODE solver will be first-order accurate (for a finite set of ODEs; in the infinite-dimensional case, the error will be of $O(h^{1/2})$, where $h$ is the step size).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want to use a $k$th order method, locate any discontinuities in your right-hand side or any of its derivatives up to order $k - 1$. At each of these discontinuities, restart the integrator. Typically, discontinuities are expressed in terms the roots of an &quot;event function&quot; that is continuously differentiable; discontinuities are then located via rootfinding. This capability is implemented in SUNDIALS, and Hairer, Nørsett, and Wanner probably discuss how to implement it in their textbook.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-09-11T08:09:22.337" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20702" PostTypeId="1" AcceptedAnswerId="20709" CreationDate="2015-09-11T10:37:01.297" Score="1" ViewCount="2190" Body="&lt;p&gt;I want to create a stairs plot with solid horizontal lines and dashed vertical lines. You can see an example of what I am trying to do in this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8833/how-to-compute-the-wavelet-approximation-of-a-function&quot;&gt;question&lt;/a&gt; (scroll down a little bit). In the example, however, they use Python, but I want to do this in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My (simple) code is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k=-4:1:4;&#xA;a= (2*k +1)/2;&#xA;t=-4:0.1:4;&#xA;&#xA;figure()&#xA;stairs(k,a,'r')&#xA;hold on&#xA;plot(t,t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My plot becomes&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bct7Z.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bct7Z.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no idea what to do. Any help is appreciated! &lt;/p&gt;&#xA;" OwnerUserId="17511" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-09-11T15:16:09.413" Title="Stairs plot with solid horizontal lines and dashed vertical lines in MATLAB" Tags="|matlab|" AnswerCount="1" CommentCount="2" ClosedDate="2015-09-25T15:34:10.527" ContentLicense="CC BY-SA 3.0" />
  <row Id="20703" PostTypeId="1" AcceptedAnswerId="20713" CreationDate="2015-09-11T12:00:16.803" Score="4" ViewCount="243" Body="&lt;p&gt;I would like to optimize a function of the following form:&#xA;\begin{equation}&#xA;\sum_{i,j=1}^N c_{i,j} \mathbf{x}_i \cdot \mathbf{x}_j,&#xA;\end{equation}&#xA;where $\mathbf{x}_i \in \mathbf{R}^d$. Is it possible to state this as SDP?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess that an SDP variable would be somehow related to the Gramian $G$ of the vectors $\{\mathbf{x}_i\}$, with the constraint $rank(G) \leq d$. Is it possible to impose such a constraint?&lt;/p&gt;&#xA;" OwnerUserId="15128" LastActivityDate="2015-09-11T22:42:59.353" Title="Rank constrained SDP" Tags="|computational-geometry|constrained-optimization|semidefinite-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20705" PostTypeId="1" CreationDate="2015-09-11T14:05:21.280" Score="0" ViewCount="496" Body="&lt;p&gt;I am using Fourier series to find the analytical solution to the 2D heat equation. The problem is that the integrals which are used to calculate the coefficients of the series cannot be solved analytically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently using GSL's Monte Carlo integration to compute them an this introduces error to the series. My question is will the solution converge to the analytical solution or not? If not, How can I get an estimate of the error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my initial condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;C_0(x,y) = \left\lbrace \begin{array}{lll}&#xA;    0					&amp;amp; &amp;amp;r(x,y) \geq 15\\&#xA;    4- \dfrac{4}{15} r(x,y) 		&amp;amp; &amp;amp;r(x,y) &amp;lt; 15&#xA;    \end{array}\right. \label{ch3-diffusion-2d-circle-init-cond}\\&#xA;  &amp;amp;r(x,y) = \sqrt{(x-75)^2 + (y-50)^2}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this is the integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{mn}={4\over l_x l_y}\int_0^{l_y} \int_0^{l_x} C_0(x,y)\sin\left({m\pi\, x\over l_x}\right)\sin\left({n\pi\, y\over l_y}\right)\,dx\,dy, \quad \forall m,n \in \aleph$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $l_x=l_y = 100$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I get after 500 iterations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sSI54.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sSI54.png&quot; alt=&quot;After 500 iterations&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which hasn't really improved from iteration 100.&lt;/p&gt;&#xA;" OwnerUserId="14772" LastEditorUserId="14772" LastEditDate="2015-09-12T13:17:54.573" LastActivityDate="2015-09-12T13:17:54.573" Title="Using numerical integration to calculate Fourier series' coefficients" Tags="|error-estimation|integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20706" PostTypeId="2" ParentId="20705" CreationDate="2015-09-11T14:12:20.157" Score="1" Body="&lt;p&gt;The quadrature formula you should be using depends on the structure of the integrand, i.e., in your case $\int f(x) \sin(kx) \; dx$. For example, if $f(x)$ is smooth, then using a high order Gaussian integration scheme is likely appropriate. If $f(x)$ is discontinuous, then Gauss formulas are not appropriate, but they may be on the individual segments were $f(x)$ &lt;em&gt;is&lt;/em&gt; smooth. Without more information about the integrand you have, it is not possible to provide a better answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is generically true, however: Monte Carlo integration is almost never the appropriate approach unless you are in high dimensional spaces.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-09-11T14:12:20.157" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20707" PostTypeId="2" ParentId="20697" CreationDate="2015-09-11T14:14:51.937" Score="4" Body="&lt;p&gt;Central differencing schemes are not stable if you have advection dominated problems. There really was no other trivial [1] alternative to developing upwind schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] There are a few other stabilization methods, of course, but in the finite difference of finite volume context, almost everything that was developed over the first 30 years of numerical computations ended up using upwinding in one variation or another.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-09-11T14:14:51.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20708" PostTypeId="1" CreationDate="2015-09-11T14:21:34.773" Score="2" ViewCount="170" Body="&lt;p&gt;I am looking for an efficient algorithm to calculate the surface area of an irregular N-dimensional polytope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a description of this polytope both as coordinates of the vertices as as linear inequalities.&lt;/p&gt;&#xA;" OwnerUserId="17514" LastEditorUserId="9667" LastEditDate="2015-09-11T19:49:40.327" LastActivityDate="2015-09-11T19:49:40.327" Title="Area of convex n-dimensional polytope" Tags="|algorithms|computational-geometry|convex-hull|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20709" PostTypeId="2" ParentId="20702" CreationDate="2015-09-11T15:16:09.413" Score="1" Body="&lt;p&gt;As mentioned in the comments to your original post, with MATLAB you will have to draw the individual line segments. Try this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k=-4:1:4;&#xA;a=(2*k +1)/2;&#xA;t=-4:0.1:4;&#xA;n = size(k,2);&#xA;&#xA;figure(1)&#xA;[xb,yb] = stairs(k,a); % gives us the x and y coordinates &#xA;stairs(k,a,'r--'); % the plot&#xA;hold on&#xA;for i = 1:2:(2*n-2) % size(xb) == size(yb) = 2*n-1&#xA;    % overplot black line segments&#xA;    plot(xb(i:i+1),yb(i:i+1),'k')&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The end result looks as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9jT2i.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9jT2i.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2015-09-11T15:16:09.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20710" PostTypeId="1" AcceptedAnswerId="20711" CreationDate="2015-09-11T17:18:02.310" Score="8" ViewCount="1445" Body="&lt;p&gt;I'm having difficulty understanding how to apply Newton iteration to nonlinear PDEs and then use a fully implicit scheme to time step. For example, I want to solve Burgers equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{t} + u u_{x} - u_{xx} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So discretising time using a Backward Euler&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{t} = \frac{u^{n+1} - u^{n}}{h}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we find that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\frac{u^{n+1} - u^{n}}{h} + u^{n+1} (u^{n+1})_{x} - (u^{n+1})_{xx} = 0 \\&#xA;&amp;amp;u^{n+1} - h (u^{n+1})_{xx} + h u^{n+1} (u^{n+1})_{x} = u^{n} \\&#xA;&amp;amp;(I - h D^{2}) u^{n+1} + h u^{n+1} D u^{n+1} = u^{n} \\&#xA;&amp;amp;(I - h D^{2}) u^{n+1} + N(u^{n+1}) = u^{n} \ \ \ (1)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N$ represents our nonlinear term (note that the nonlinear term is written implicitly). Now, we want to apply Newton iteration to this nonlinear ODE, but here is where I get stuck:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do we just apply Newton Iteration to the LHS of $(1)$, ignoring the $u^{n}$ term, i.e., solve $(I - h D^{2}) u^{n+1} + N(u^{n+1}) = 0$? Or are we supposed to include the $u^{n}$ term? (Just a reminder, I want to time step using a fully implicit scheme after using Newton iteration, so I believe we just want to solve the LHS = 0).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are we then supposed to do with the information from the initial guess and result of our Newton iteration? How do we use this information in our time stepping?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As I'm sure is painfully obvious, I'm quite confused as to how to approach this problem. If someone could give a detailed description of how to apply Newton iteration and time stepping to nonlinear PDEs (not elliptic PDEs though), or could help me with the problem at hand, I would be very grateful. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16852" LastEditorUserId="9667" LastEditDate="2015-09-11T19:54:03.987" LastActivityDate="2015-09-13T17:13:55.430" Title="Newton iteration applied to nonlinear PDE" Tags="|pde|nonlinear-equations|time-integration|newton-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20711" PostTypeId="2" ParentId="20710" CreationDate="2015-09-11T19:15:33.373" Score="7" Body="&lt;p&gt;It's a bit easier to see if you write your equation in the a semi-discretised system of the form $u^{\prime}(t) = F(u(t))$ and with the application of the $\theta$-method and approximating $u^{\prime}(t) \approx (w^{n+1} - w^{n})/\tau$ this gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w^{n+1} - w^{n} - (1-\theta) \tau F(w^n) - \theta\tau F(w^{n+1}) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with unknown vector $w^{n+1}$ and time step $\tau$. Here $u^{\prime}(t)$ is the non discretised time partial derivative and $F(u(t))$ represents your discretised spatial derivatives evaluated at $u(t)$ at time $t$. Using the $\theta$-method gives you a bit of flexibility as you can change the method between fully implicit to fully explicit (and anywhere in between).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation can be solved using Newton iteration,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nu^{k+1} = \nu^{k} - (I - \theta\tau A^{n})^{-1} \left( \nu^{k} - w^{n} - (1-\theta) \tau F(w^n) - \theta\tau F(w^{n+1}) \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k$ is the iteration index ($k\geq0$) and $A^{n}$ is the Jacobian matrix of $F(w^n)$. We use the symbol $\nu^{k}$ for iteration variables such that they are distinguished from solution of the equation at a real time point $u^n$. As you said the iteration needs an initial value, it is perfectly valid to choose $\nu^0 = w^0$ or for a better estimate we can precompute one iteration $\nu^0 = w^0 + \tau F(w^{0})$. Strictly speaking this is the so-called &lt;em&gt;modified&lt;/em&gt; Newton iteration because the Jacobian is not updated during the iteration which is known to work well for stiff PDEs.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2015-09-13T17:13:55.430" LastActivityDate="2015-09-13T17:13:55.430" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20713" PostTypeId="2" ParentId="20703" CreationDate="2015-09-11T22:42:59.353" Score="5" Body="&lt;p&gt;The constraint $\mbox{rank}(X) &amp;lt;= d$ is in general a non-convex constraint.  Sorry.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A commonly used approach is to minimize the Schatten 1-norm of X (the sum of singular values of X) as a surrogate for minimizing the rank.  This works similarly to minimizing the 1-norm of a vector x as a way of minimizing the number of nonzero entries in x.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-09-11T22:42:59.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20715" PostTypeId="1" AcceptedAnswerId="20716" CreationDate="2015-09-12T05:34:31.120" Score="1" ViewCount="282" Body="&lt;p&gt;I'm interested in how much 'signal' is retained from including &lt;code&gt;k&lt;/code&gt; singular values in a Singular Value Decomposition, but I'm having trouble conceptualizing (or creating) a metric for 'signal retention'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can create $A_i$, a matrix which is created from the first $i$ singular values of $A$, but how do I compare $A_i$ with $A$ and get a reliably meaningful metric? I am currently evaluating &lt;code&gt;norm(A_i-A)/norm(A)&lt;/code&gt;, but I'm getting incredibly good results and it doesn't &quot;feel&quot; like the result should be good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by that is that I plot the image of a reconstruction with only the first singular value and it looks nothing like the image I started with, but my analysis shows I have only 10% signal loss. After maybe the first 4 or 5 singular values are included can I start to recognize features of my face in the image, but if I can't even recognize my own face, I have a hard time accepting that it is 90% accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a canonical way for doing this? Surely someone else has performed this kind of analysis with SVD, but I'm having trouble finding how (if?) it is canonically done.&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2015-09-12T08:06:22.010" Title="Compare reconstruction of matrices using SVD" Tags="|matrix|image-processing|svd|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20716" PostTypeId="2" ParentId="20715" CreationDate="2015-09-12T08:06:22.010" Score="3" Body="&lt;p&gt;PSNR, the ratio between the peak power of the true signal and the power of&#xA;the Gaussian noise, measures the amount of mathematical error introduced in an&#xA;image by compression or noise introduction. This would be ideal for your evaluations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PSNR is related to MSE (mean squared error) but uses a logarithmic scale. PSNR between a grayscale image $A$ and its compressed version / reconstruction $\hat{A}$ is defined as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$PSNR(A,\hat{A}) = 10\log_{10}\big(\frac{255}{\lVert A-\hat{A} \rVert _{F}}\big)$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You could also automatically select how many singular values to retain using &lt;a href=&quot;https://en.wikipedia.org/wiki/Akaike_information_criterion&quot; rel=&quot;nofollow&quot;&gt;Akaike information criterion&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2015-09-12T08:06:22.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20717" PostTypeId="2" ParentId="20694" CreationDate="2015-09-12T09:22:44.057" Score="3" Body="&lt;p&gt;The first component you need is a function $orient(p_1,p_2,p_3)$ that computes the orientation between three point, i.e. that gives a positive number if there is a left turn between vectors $\overrightarrow{p_1 p_2}$ and $\overrightarrow{p_1 p_3}$ and a negative number otherwise (and zero if the three points are aligned):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  Orient(p_1,p_2,p_3) = det(\overrightarrow{p_1 p_2}, \overrightarrow{p_1 p_3}) = (x_2-x_1)(y_3-y_1) - (y_2-y_1)(x_3-x_1)&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can determine whether two segments $[p_1 p_2]$ and $[q_1 q_2]$ have an intersection:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  IsectSeg(p_1,p_2,q_1,q_2) = Orient(p_1,q_1,q_2) \times Orient(p_2,q_1,q_2) \le 0 \mbox{ and } Orient(q_1,p_1,p_2) \times Orient(q_2,p_1,p_2) \le 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In English: there is an intersection between $[p_1 p_2]$ and $[q_1 q_2]$ if $[p_1 p_2]$ straddles the supporting line $(q_1 q_2)$ and&#xA;$[q_1 q_2]$ straddles the supporting line $(p_1 p_2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also determine whether a point $p$ belongs to a triangle $q_1,q_2,q_3$ using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  InsideTri(p,q_1,q_2,q_3) = (Orient(p,q_1,q_2) \times Orient(p,q_2,q_3) \ge 0) \mbox{ and } (Orient(p,q_2,q_3) \times Orient(p,q_3,q_1) \ge 0) &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In English: $p$ is inside triangle $(q_1,q_2,q_3)$ if the orientation relative to the three edges $q_1 q_2$, $q_2 q_3$ and $q_3 q_1$ is the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting everything together, there is an intersection between both triangles if:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;   IsectTri(p_1, p_2, p_3, q_1, q_2, q_3) = &#xA;\exists i | InsideTri(p_i, q_1, q_2, q_3) \mbox{ or }&#xA;\exists i | InsideTri(q_i, p_1, p_2, p_3) \mbox{ or }&#xA;\exists i,j,k,l | IsectSeg(p_i, p_j, q_k, q_l)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In English: the two triangles have an intersection if one vertex of one triangle is inside the other triangle, or if there exists an intersection between the edges of the triangles. It is necessary to test for edges intersection (think for instance about two intersecting triangles that form a six-branches star that has no triangle vertex inside the other triangle)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Translated into a constraint, this gives a formula with inequality constraints combined with OR and AND operators (each of the 6 $Inside$ constraint yields two terms, and each of the 9 $IsectSeg$ constraint yields two terms as well). I am unsure of how to translate this into a way that can be understood by a constrained optimization software but there is probably a standard way of doing that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There might be also a shorter / more elegant formulation (but I did not find it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note1: if you know in advance that the triangles are all oriented consistently (all clockwise, or all anticlockwise), then InsideTri() can be made simpler (just test the sign of the $Orient()$ relative to the three edge).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note2: the asked question required a &quot;mathematical programming&quot; answer that could be expressed as a constraint in an optimization program. Now if what you want is simply determining whether two 2D triangles intersect (in a standard programming language that has execution flow and tests), there is a significantly faster approach that avoids some tests, see the 2D section in [1]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://hal.inria.fr/inria-00072100&quot; rel=&quot;nofollow&quot;&gt;https://hal.inria.fr/inria-00072100&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2016-09-27T16:53:56.633" LastActivityDate="2016-09-27T16:53:56.633" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20718" PostTypeId="2" ParentId="20697" CreationDate="2015-09-12T13:03:27.930" Score="4" Body="&lt;p&gt;To elaborate on Wolfgang's answer: since hyperbolic PDE semi-discretizations with centered differences have purely imaginary eigenvalues, they are only neutrally stable.  For linear problems (e.g., the acoustic wave equation or linear Maxwell's equations) this is fine and such methods are commonly used.  For instance, in electromagnetics centered differences are referred to simply as &quot;finite difference time domain&quot; (FDTD) methods and are prevalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For nonlinear hyperbolic PDEs, this neutral stability generally leads to excitation of nonlinear instabilities.  Even if it does not, it still leads to oscillations near points of discontinuity (and discontinuities arise generically in solutions of hyperbolic PDEs).  Upwind spatial discretizations shift the eigenvalues -- especially those corresponding to high-frequency modes -- into the left half-plane, leading to more stable (and less oscillatory) solutions.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-09-12T13:03:27.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20719" PostTypeId="1" CreationDate="2015-09-12T16:18:50.670" Score="13" ViewCount="402" Body="&lt;p&gt;I just read Chapter 3 in &quot;A Multigrid Tutorial&quot; by Briggs/Henson/McCormick,  &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898719505&quot;&gt;link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The text is about Multigrid cycles such as V-cycle, mu-cycle, FMG. What caught my eye: In most iterative procedures one checks whether it has converged to the desired tolerance/accuracy and if so, the procedure stops. But Briggs/Henson/McCormick do not use any convergence checking in the presented schemes. The number of iterations and recursions are just hardcoded and one has to trust that the scheme will converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how is this done in Multigrid normally? Is it usual that the number of iterations/recursions is just hardcoded? I really fear that I will either waste a lot of computation time because I'm over-precise or on the other hand, accuracy will be poor in many cases when I choose a lower number of iterations/recursions.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastEditorUserId="8138" LastEditDate="2015-09-12T16:30:14.860" LastActivityDate="2015-09-19T04:44:25.030" Title="Is it usual to have no convergence checking in Multigrid?" Tags="|multigrid|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20722" PostTypeId="1" CreationDate="2015-09-13T08:52:33.927" Score="4" ViewCount="1738" Body="&lt;p&gt;I'm trying to solve coupled PDE by Crank-Nicolson (CN) and Newton-Raphson method with MATLAB. I have used CN method but not for coupled problem. Please if someone could help let me know to add more details about the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Detail of coupled equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m going to solve Poisson-Nernst-Planck system of equations. The Nernst-Planck equation is&#xA;$$\frac{\partial n_i}{\partial t}=A_i\frac{\partial^2 n_i}{\partial x^2}+B_i\frac{\partial n_i}{\partial x}\frac{\partial \phi}{\partial x}+B_i(n_i\frac{\partial^2 \phi}{\partial x^2}),$$&#xA;where $n_i$ is the concentration of specific ion, $A_i$ and $B_i$ is the coefficient of equation for specific ion, $\phi$ is the electrical potential. The Poisson equation that couples by the previous equation is&#xA;$$\epsilon \frac{\partial^2 \phi}{\partial x^2}=F\sum(n_i*z_i),$$&#xA;where $\epsilon$ is the permittivity of medium, $F$ is the Faraday constant, $n_i$ is the concentration of specific ion and $z_i$ is the ionic charge. So how could I implement Poisson equation into Nernst-Planck and after that implement Newton-Raphson method for linearize it. Or do we need Newton-Raphson method to solve this equation? or we could solve it just by CN method. Please let me know if anybody needs more details or if something is unclear.&lt;/p&gt;&#xA;" OwnerUserId="17525" LastEditorUserId="17525" LastEditDate="2015-09-15T09:41:11.797" LastActivityDate="2016-07-14T20:49:55.720" Title="How could we solve coupled PDE with finite difference method and Newton-Raphson method?" Tags="|finite-difference|nonlinear-equations|coupling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20723" PostTypeId="1" AcceptedAnswerId="20724" CreationDate="2015-09-13T14:44:50.877" Score="1" ViewCount="82" Body="&lt;p&gt;I am using my own Routine to calculate the Cholesky-Factorization of a complex, positive definite symmetric Matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Code Looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void CholeskyDecomposition(Complex **L, Complex **A, const int dim)&#xA;{&#xA;    Complex sum, t;&#xA;    Complex sum1;&#xA;&#xA;    for (int k = 0; k &amp;lt; dim; k++)&#xA;    {&#xA;        // First, calculate the diagonal element of a column&#xA;        sum1 = A[k][k];&#xA;        for (int j = 0; j &amp;lt; k; j++)&#xA;        {&#xA;            Complex product;&#xA;            Cconjmul(product, L[j][k], L[j][k]);&#xA;            Csub(sum1, sum1, product);&#xA;        }&#xA;        Csqrt(&amp;amp;sum1, sum1);&#xA;        L[k][k] = sum1;&#xA;&#xA;        // Now, calculate the elements right the diagonal element&#xA;        for (int i = k + 1; i &amp;lt; dim; i++)&#xA;        {&#xA;            sum = A[k][i];&#xA;            for (int j = 0; j &amp;lt; k; j++)&#xA;            {&#xA;                Cconjmul(t, L[j][k], L[j][i]);&#xA;                Csub(sum, sum, t);&#xA;            }&#xA;            Cdiv(L[k][i], sum, sum1);&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tested this Code against the MATLAB Cholesky-Factorization which results in&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A =&#xA;&#xA;  13.6393 + 0.0000i   1.8844 + 3.4319i  -4.8788 - 4.0195i&#xA;   1.8844 - 3.4319i   5.7265 + 0.0000i   1.1568 - 0.4937i&#xA;  -4.8788 + 4.0195i   1.1568 + 0.4937i   6.6512 + 0.0000i&#xA;&#xA;K&amp;gt;&amp;gt; L = chol(A)&#xA;&#xA;L =&#xA;&#xA;   3.6931 + 0.0000i   0.5102 + 0.9293i  -1.3210 - 1.0884i&#xA;   0.0000 + 0.0000i   2.1454 + 0.0000i   1.3248 - 0.5435i&#xA;   0.0000 + 0.0000i   0.0000 + 0.0000i   1.2927 + 0.0000i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result of my C-Implementation seems not to be totally wrong. The first row is identical to the MATLAB-solution.&#xA;The second and third row contains Errors in the last column:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MATLAB         &amp;lt;--&amp;gt;       C&#xA;1.3248-0.5435i &amp;lt;--&amp;gt; 1.3248+0.08324i&#xA;1.2927+0.0000i &amp;lt;--&amp;gt; 1.3998+0.00000i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me to identify the error source?&lt;/p&gt;&#xA;" OwnerUserId="16641" LastActivityDate="2015-11-12T16:19:46.663" Title="Big errors while calculating Complex Cholesky Factorization" Tags="|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20724" PostTypeId="2" ParentId="20723" CreationDate="2015-09-13T15:01:31.467" Score="1" Body="&lt;p&gt;Sorry for annoying you guys. The error was quite simple: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Now, calculate the elements right the diagonal element&#xA;for (int i = k + 1; i &amp;lt; dim; i++)&#xA;{&#xA;    sum = A[k][i];&#xA;    for (int j = 0; j &amp;lt; k; j++)&#xA;    {&#xA;        Cconjmul(t, L[j][k], L[j][i]);&#xA;        Csub(sum, sum, t);&#xA;    }&#xA;    Cdiv(L[k][i], sum, sum1);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just swap the L's of the complex conjugate multiplication. Has to be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Cconjmul(t, L[j][i], L[j][k]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16641" LastActivityDate="2015-09-13T15:01:31.467" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20725" PostTypeId="1" CreationDate="2015-09-13T18:28:57.620" Score="2" ViewCount="158" Body="&lt;p&gt;I am solving a 1D advection problem of the the form $$d{Q}/dt = [A]{Q}$$ where {Q} is the vector of unknowns and [A] is the matrix of coefficients of spatial discretisation. I have worked out the eigenvalues of [A] to get an idea about the stability of the semi-discretised system and I am getting eigenvalues with positive real parts plus imaginary parts. I know positive real parts are a sign of instability, but my results show the smooth travelling of a wave. I wanted to know what conclusions I can draw from this? &#xA;Thanks! &lt;/p&gt;&#xA;" OwnerUserId="17476" LastEditorUserId="17476" LastEditDate="2015-09-16T08:04:07.380" LastActivityDate="2015-09-16T08:04:07.380" Title="what do positive real parts of eigenvalues mean?" Tags="|fluid-dynamics|eigenvalues|stability|hyperbolic-pde|advection|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20726" PostTypeId="1" CreationDate="2015-09-13T21:03:23.027" Score="4" ViewCount="634" Body="&lt;p&gt;I have a problem with assesing the accuracy of my numerical calculation. I have a 2nd order ODE. It is an eigenvalue problem of the form:&#xA;$$ y'' + ay' + \lambda^2y = 0, $$&#xA;and the boundary condiations are:&#xA;$$ y(0) = y(1) = 0. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation describes a vibrating string, clamped at x=0 and x=1, with a certain mass distribution. I want to be able to calculate the eigenvalues of this equation numerically. I do this by using the Runge-Kutta method to find solutions to the equation with initial conditions:&#xA;$$ y(0) = 0, y'(0) = 1 $$&#xA;with different values of $\lambda$ and then looking for the ones that are zero at x=1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I terminate my search for eigenvalues when I find a function that has&#xA;$$y(1) &amp;lt; \delta,$$&#xA;where $\delta$ is a predefined constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the problem is that I'd like to know how accurate my calculations are, i.e. how do I choose the values of $\delta$ and the stepsize used in computing the values of the potential eigenfunction, so that the error in the eigenvalue is less than, say, $ 10^{-3}$? &lt;/p&gt;&#xA;" OwnerUserId="17529" LastEditorUserId="713" LastEditDate="2015-09-14T03:44:02.890" LastActivityDate="2016-02-22T16:54:15.290" Title="Accuracy of numerical methods in finding eigenvalues" Tags="|ode|eigenvalues|runge-kutta|numerics|accuracy|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="20727" PostTypeId="2" ParentId="223" CreationDate="2015-09-13T21:22:14.400" Score="7" Body="&lt;p&gt;Prior answers to this question have covered most of the salient points, but I want to add one comment with respect to this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;does MKL have the upper hand for some tasks?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The MKL team is in a unique position to know about future Intel instruction sets and their implementations in specific processors.  Furthermore, they have access to proprietary processor simulators and pre-production hardware that no one outside of Intel can use.  Thus, MKL has the upper hand with respect to degree of knowledge about future products and when they obtain this knowledge.  Thus, it should not be too surprising if they produce better implementations of the BLAS than anyone else, at least early in the lifetime of a product with new features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, Intel has been quite open about the AVX-512 instruction set and has provided the &lt;a href=&quot;https://software.intel.com/en-us/articles/intel-software-development-emulator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel® Software Development Emulator&lt;/a&gt; (SDE) that allows developers to emulate AVX-512 instructions on processors that do not support these natively.  Because of this, it will not be too surprising if high-quality open-source implementations of the BLAS are available for Intel processors that support AVX-512 early in the lifetime of these products.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, how much difference it makes to have detailed information about a particular processor versus the fundamentals of dense linear algebra algorithms is not fully resolved.  The following quote addresses this issue better than I can:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In theory, there is no difference between theory and practice. But, in practice, there is.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Full Disclosure: I work for Intel.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2018-01-30T06:13:36.980" LastActivityDate="2018-01-30T06:13:36.980" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20728" PostTypeId="1" CreationDate="2015-09-13T21:27:38.370" Score="2" ViewCount="63" Body="&lt;p&gt;I am a bit of SAGE newb, so I apologize if this is a simplistic question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use SAGE to compute some invariants of elliptic curves defined over rings which aren't the integers $\mathbb{Z}$ or the rationals $\mathbb{Q}$. More specifically, I am interested looking at elliptic curves defined over the ring &#xA;$$&#xA;\mathbb{Z}[1/6][c_4,c_6]&#xA;$$&#xA;or &#xA;$$&#xA;\mathbb{Z}_{(p)}[c_4,c_6]&#xA;$$&#xA;where $p$ is some prime. Can anyone tell me how produce a localized ring such as $\mathbb{Z}[1/6]$ or $\mathbb{Z}_{(p)}$ in SAGE? Or know where in the manual I could find such information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="17207" LastActivityDate="2015-09-13T21:27:38.370" Title="localizing rings in SAGE" Tags="|sage|" AnswerCount="0" CommentCount="1" ClosedDate="2018-11-19T15:15:32.577" ContentLicense="CC BY-SA 3.0" />
  <row Id="20729" PostTypeId="1" CreationDate="2015-09-13T21:48:52.297" Score="0" ViewCount="255" Body="&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/a/8739526/2623899&quot;&gt;This answer&lt;/a&gt; to &lt;a href=&quot;https://stackoverflow.com/q/8739227/2623899&quot;&gt;this question&lt;/a&gt; works only for situations in which the desired solution to the coupled functions is not restricted to a certain range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if, for example, we wanted a solution such that &lt;code&gt;0 &amp;lt; x &amp;lt; 10&lt;/code&gt; and &lt;code&gt;0 &amp;lt; y &amp;lt; 10&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are functions within &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/optimize.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scipy.optimize&lt;/code&gt;&lt;/a&gt; that find roots to a function within a given interval (e.g., &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brentq.html#scipy.optimize.brentq&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;brentq&lt;/code&gt;&lt;/a&gt;), but these work only for functions of &lt;strong&gt;one variable&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does &lt;code&gt;scipy&lt;/code&gt; fall short of providing a root solver that works for multi-variable functions within specific ranges?  How might such a solver be implemented?&lt;/p&gt;&#xA;" OwnerUserId="17530" LastEditorUserId="20688" LastEditDate="2021-12-16T20:22:22.390" LastActivityDate="2021-12-16T20:22:22.390" Title="Solve a pair of coupled nonlinear equations within certain limits" Tags="|optimization|python|nonlinear-equations|constrained-optimization|scipy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20731" PostTypeId="2" ParentId="8057" CreationDate="2015-09-14T09:23:27.957" Score="-3" Body="&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;a=imread('3.jpg');&#xA;    a1=zeros(size(a));&#xA;    for i=1:size(a,1)&#xA;    for j=1:size(a,2)&#xA;    for k=1:3&#xA;    if((a((i-1),j,k)&gt;=a(i,j,k) &amp;amp;&amp;amp; a(i,j,k)&gt;=a((i+1),j,k)) ||(a((i-1),j,k)&gt;=a(i,j,k)&amp;amp;&amp;amp; a(i,j,k)&gt;=a((i+1),j,k))||(a(i,(j-1),k)&gt;=a(i,j,k)&amp;amp;&amp;amp; a(i,j,k)&gt;=a(i,(j+1),k))|| (a(i,(j-1),k)&amp;lt;=a(i,j,k)&amp;amp;&amp;amp; a(i,j,k)&amp;lt;=a(i,(j+1),k))|| (a((i+1),(j+1),k)&gt;=a(i,j,k)&amp;amp;&amp;amp;a(i,j,k)&gt;=a((i-1),(j+1),k))|| (a((i+1),(j+1),k)&amp;lt;=a(i,j,k)&amp;amp;&amp;amp;a(i,j,k)&amp;lt;=a((i-1),(j+1),k))|| (a((i-1),(j-1),k)&gt;=a(i,j,k)&amp;amp;&amp;amp; a(i,j,k)&gt;=a((i+1),(j+1),k))|| (a((i-1),(j-1),k)&amp;lt;=a(i,j,k)&amp;amp;&amp;amp; a(i,j,k)&amp;lt;=a((i+1),(j+1),k)))&#xA;    a1(i,j,k)=round(a(i,j,k));&#xA;    else&#xA;    a1(i,j,k)=round((a((i-1),(j-1),k)+a((i-1),j,k)+a((i-1),(j+1),k)+a(i,(j-1),k)+a(i,(j+1),k)+a((i+1),(j-1),k)+a((i+1),j,k)+a((i+1),(j+1),k))/8);&#xA;    end&#xA;    end&#xA;    end&#xA;    end&#xA;    ??? Subscript indices must either be real positive integers or logicals.&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;please help me &lt;/p&gt;&#xA;" OwnerUserId="17535" LastActivityDate="2015-09-14T09:23:27.957" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20733" PostTypeId="2" ParentId="2267" CreationDate="2015-09-14T10:02:54.997" Score="-3" Body="&lt;p&gt;we are working to open a CFD academy with a CFD starting Courses Free&#xA;&lt;a href=&quot;http://cadcae.net&quot; rel=&quot;nofollow&quot;&gt;http://cadcae.net&lt;/a&gt;&#xA;for example you can see a specific training like &lt;a href=&quot;http://cadcae.net/training/curso-cfd-openfoam&quot; rel=&quot;nofollow&quot;&gt;curso-cfd-openfoam&lt;/a&gt;&#xA;Thanks in advance&#xA;Best regards&lt;/p&gt;&#xA;" OwnerUserId="17536" LastActivityDate="2015-09-14T10:02:54.997" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20734" PostTypeId="2" ParentId="20719" CreationDate="2015-09-14T23:52:13.753" Score="4" Body="&lt;p&gt;Definitely not.  To pick one example, the book &lt;a href=&quot;https://books.google.com/books?id=-og1wD-Nx_wC&amp;amp;lpg=PP1&amp;amp;dq=isbn%3A012701070X&amp;amp;pg=PA53#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Multigrid&lt;/a&gt; has a plot on page 53 (Figure 2.10) that shows the decrease in the residual as a function of the number of V or W cycles.  You would stop cycling when you are satisfied with the size of the residual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source of your confusion may be because some descriptions only describe a single V-cycle.  In some limited cases, because multigrid is such a powerful technique, this may generate a suitable solution.  Also, multigrid can be used as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multigrid_method#Multigrid_preconditioning&quot; rel=&quot;nofollow&quot;&gt;preconditioner&lt;/a&gt;.  In that case, the multigrid is just an accelerator, and the check for convergence happens at a higher level.  But the check should always happen somewhere.&lt;/p&gt;&#xA;" OwnerUserId="17532" LastEditorUserId="17532" LastEditDate="2015-09-19T04:44:25.030" LastActivityDate="2015-09-19T04:44:25.030" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20735" PostTypeId="2" ParentId="20726" CreationDate="2015-09-15T00:01:17.117" Score="0" Body="&lt;p&gt;In principle, you could try to come up with an analytic expression for how the error in step size or $\delta$ affects the answer.  I would not bother.  You will probably get it wrong.  The far, far better method is to vary those numbers and see how much your answer changes.  For example, you could change $\delta$ from 0.01 to 0.001 and see how much the answer changes.  Similarly, you can vary the step size by factors of 2 or 10.&lt;/p&gt;&#xA;" OwnerUserId="17532" LastActivityDate="2015-09-15T00:01:17.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20736" PostTypeId="1" CreationDate="2015-09-15T11:54:43.860" Score="0" ViewCount="880" Body="&lt;p&gt;I want to solve a pair of ODEs using the FEM solver of Abaqus. Is it possible for the user to supply the equation and ask Abaqus solver to solve the equations?&lt;/p&gt;&#xA;" OwnerUserId="17546" LastEditorUserId="20688" LastEditDate="2021-12-28T03:28:17.443" LastActivityDate="2021-12-28T03:28:17.443" Title="abaqus solver question" Tags="|finite-element|ode|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20737" PostTypeId="1" CreationDate="2015-09-15T14:20:15.883" Score="1" ViewCount="1838" Body="&lt;p&gt;Whilst working with MATLAB recently I encountered something odd that I cannot explain. I was using the ode45 solver to solve a system of two coupled second order ODEs. I wasn't convinced about the results, so I tried something easier, just to see where it goes wrong. I uncoupled the equations and I swapped one of them to be a simple harmonic oscillator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quite surprisingly, I did not get what I expected. However when I removed one of the terms from the second equation (which is now uncoupled!) and ran it again, it produced a correct answer. This blows my mind, since there is no link at all between the two equations now. Here is my code after alteration:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Main function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function Ord2ODE&#xA;&#xA;t=0:0.0001:20;&#xA;%x,xdot,y,ydot&#xA;ainit = [0; 1; 0; 0];&#xA;&#xA;[t,a] = ode45(@rhs,t,ainit);&#xA;&#xA;figure;&#xA;plot(t, a(:,1));&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the function rhs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dadt = rhs(t,a)&#xA;&#xA;mu = 0;&#xA;&#xA;x = a(1);&#xA;xdot = a(2);&#xA;y = a(3);&#xA;ydot = a(4);&#xA;&#xA;**Fy = stuff in terms of x, y (a(1), a(2))**&#xA;Fx = stuff in terms of x, y (a(1), a(2))&#xA;&#xA;dadt1 = a(2);&#xA;dadt2 = -2*a(1);  &#xA;dadt3 = a(4);&#xA;dadt4 = -2*a(3) - **Fy**;&#xA;&#xA;dadt = [dadt1;dadt2;dadt3;dadt4];&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with the prblematic bit marked with **  ** (definition and occurance in computation of dadt4). You can see that neither a(3) nor a(4) are present in the calculation of x. The results with and without that term are posted below. Does anyone have any idea why would a term in an uncoupled equation cause this kind of divergence in a solution of the other equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ed3LL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ed3LL.png&quot; alt=&quot;With&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wDRWd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wDRWd.png&quot; alt=&quot;without&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Fy = -mu*y - (1-mu)*y + mu*y/(sqrt(((x+1-mu)^2 + y^2)^3)) + (1-mu)*y/(sqrt(((x-mu)^2 + y^2)^3));&#xA;&#xA;Fx = -mu*(x+1-mu) - (1-mu)*(x-mu) + mu*(x+1-mu)/(sqrt(((x+1-mu)^2 + y^2)^3)) + (1-mu)*(x-mu)/(sqrt(((x-mu)^2 + y^2)^3));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$Fy = -\mu*y - (1-\mu)*y + \frac{\mu*y}{(\sqrt{((x+1-\mu)^2 + y^2)^3)}} + \frac{(1-\mu)*y}{(\sqrt{((x-\mu)^2 + y^2)^3)}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Fx = -\mu*(x+1-\mu) - (1-\mu)*(x-\mu) + \frac{\mu*(x+1-\mu)}{(\sqrt{((x+1-\mu)^2 + y^2)^3)}} + \frac{(1-\mu)*(x-\mu)}{(\sqrt{((x-\mu)^2 + y^2)^3)}}$&lt;/p&gt;&#xA;" OwnerUserId="17529" LastEditorUserId="17529" LastEditDate="2015-09-16T19:34:09.773" LastActivityDate="2015-09-17T12:07:23.387" Title="Unexpected results of MATLAB's ode45" Tags="|matlab|ode|numerics|coupling|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20738" PostTypeId="2" ParentId="20736" CreationDate="2015-09-15T14:29:42.337" Score="3" Body="&lt;p&gt;No, ABAQUS is not a general-purpose PDE solver. It solves the PDE for a number of pre-defined applications such as structural analysis or heat transfer in solids. Of course many physical phenomena are governed by the same equation as heat transfer so you can solve those problems by treating them as heat transfer problems in ABAQUS.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-09-15T14:29:42.337" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20739" PostTypeId="1" CreationDate="2015-09-15T21:16:28.343" Score="3" ViewCount="427" Body="&lt;p&gt;I designed an algorithm to find the global minimum of a function and implemented it in MATLAB. And I also implemented the &quot;Tunneling algorithm&quot; for the global minimum of a function in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now, I want an intelligent way to compare these two algorithms and see which one gets to the global minimum faster. Using the MATLAB tic toc isn't enough. How can I do it?&lt;/p&gt;&#xA;" OwnerUserId="17553" LastEditorUserId="17755" LastEditDate="2015-10-05T04:08:58.403" LastActivityDate="2015-10-05T04:08:58.403" Title="Efficiency of an algorithm" Tags="|matlab|algorithms|approximation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20740" PostTypeId="2" ParentId="20739" CreationDate="2015-09-15T21:20:34.687" Score="8" Body="&lt;p&gt;The usual approach to testing optimization algorithms is to compare how many &lt;em&gt;function evaluations&lt;/em&gt; they need to find the minimum (or get within a fixed tolerance $\varepsilon$ of the minimum). This is easily implemented in any code, and it does not matter in that case how fast your computer is, or what the resolution of the tic/toc mechanism is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of standard problems that are often considered in the optimization literature. The simplest one would be the Rosenbrock function, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Rosenbrock_function&quot;&gt;https://en.wikipedia.org/wiki/Rosenbrock_function&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-09-15T21:20:34.687" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20741" PostTypeId="2" ParentId="2226" CreationDate="2015-09-16T11:13:40.913" Score="4" Body="&lt;p&gt;A complete model of SpMV performance is given in &lt;a href=&quot;http://www.cs.odu.edu/~keyes/papers/pcfd99_gkks.pdf&quot; rel=&quot;nofollow&quot; title=&quot;Towards Realistic Performance Bounds for Implicit CFD codes&quot;&gt;this paper&lt;/a&gt;. It shows clearly that the main limiter is bandwidth, although you can lessen the burden by using multiple vectors. After that you run into instruction issue limitations and a limit on outstanding write instructions I believe.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2015-09-16T11:13:40.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20742" PostTypeId="2" ParentId="20719" CreationDate="2015-09-16T11:21:06.833" Score="5" Body="&lt;p&gt;Yes, it is normal to have no convergence checks in MG for a few reasons. First, if you use a different number of iterates on each pass, then the MG operator is no longer linear, and you would have to use something like FGMRES as an accelerator which can accommodate a nonlinear preconditioner. Second, FMG is an exact solver (reduces error below discretization error) when it works, so checking convergence introduces costly synchronization into the algorithm. You would generally check at the end just to verify convergence.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2015-09-16T11:21:06.833" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20743" PostTypeId="2" ParentId="20729" CreationDate="2015-09-16T11:23:55.143" Score="1" Body="&lt;p&gt;This is multivariate optimization with bound constraints (if you want to look it up in the literature). The &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/tao/&quot; rel=&quot;nofollow&quot;&gt;TAO&lt;/a&gt; software can solve these problems if you want true optimization. If all you want is the solution of nonlinear equations with bounds, you can use the SNESVI solver in &lt;a href=&quot;http://www.mcs.anl.gov/research/projects/petsc/&quot; rel=&quot;nofollow&quot;&gt;PETSc&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2015-09-16T11:23:55.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20747" PostTypeId="2" ParentId="20737" CreationDate="2015-09-16T12:36:42.123" Score="1" Body="&lt;p&gt;From what I understand from your question, the first plot has become unstable, so perhaps if you did a plot with the two methods for the first time period (so up to around 4 seconds) your two solutions will match (looking at your plots they seem to match). &#xA;If that was the case, then try solving the problem with a different solver, or making your time steps smaller to ensure that the problem is not related to the time stepping scheme. &lt;/p&gt;&#xA;" OwnerUserId="17362" LastActivityDate="2015-09-16T12:36:42.123" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20749" PostTypeId="2" ParentId="20737" CreationDate="2015-09-16T16:11:08.177" Score="1" Body="&lt;p&gt;Here is a guess as to what might be going on. ode45 (like all the MATLAB ode solvers) adjusts the step size based on it's estimate of the solution error. All equations are considered in this calculation whether they are coupled or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution errors are compared to two parameters, AbsTol and RelTol. Both of these have default values but sometimes these defaults don't produce an accurate solution. I suggest experimenting with these two parameters (e.g. make them&#xA;an order of magnitude smaller) and see how it affects your solution.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-09-16T16:11:08.177" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20750" PostTypeId="2" ParentId="20684" CreationDate="2015-09-16T20:49:55.270" Score="0" Body="&lt;p&gt;The first thing to do is to construct a simple test case, whose values you know analytically. So you might try a case with 7 nodes, where the solution is a simple function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have a system that can solve the simple case, then it is worth considering what would cause an oscillation in the results; from the wording it seems like you have a dynamics problem, but if this is a static solution then there are convergence acceleration approaches which can help, for example the Anderson mixing scheme for self-consistent problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything in the nature of the problem you are solving which would make it tend to overshoot, like a Coulomb repulsion? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes the stability of the solution can depend on scaling of the variables; if you have a large offset or datum then the differences that an FD scheme depends on can be truncated by their representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally also consider the precision of the variables in use. If you have a fine mesh and the floating point type cannot hold enough precision, then the results are just skipping about in a sparse solution space. So ensure you are using at least double precision and that there is no large datum offset in your BCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without more detail, or results to show the problem, it is hard to offer more specific suggestions. &lt;/p&gt;&#xA;" OwnerUserId="1372" LastActivityDate="2015-09-16T20:49:55.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20752" PostTypeId="1" CreationDate="2015-09-16T22:22:40.823" Score="4" ViewCount="241" Body="&lt;p&gt;I have seen a lot of literature, lecture videos, etc. on solvers/preconditioners for non-symmetric and/or indefinite systems. However, now I want to solve the mixed poisson/Darcy equation using the least-squares finite element method:&#xA;$$\int_\Omega\left(\boldsymbol{u}(\mathbf{x}) + \nabla p(\mathbf{x}) - \boldsymbol{g}(\mathbf{x})\right)\cdot&#xA;\mathbf{A}\left(\boldsymbol{v}(\mathbf{x}) + \nabla q(\mathbf{x})\right)+\left(\nabla\cdot\boldsymbol{u}-f(\mathbf{x})\right)\left(\nabla\cdot\boldsymbol{v}\right)\mathrm{d}\Omega,$$&#xA;where $\boldsymbol{u}/p$ and $\boldsymbol{v}/q$ are the velocity/pressure trial and test functions respectively, $\mathbf{A}$ is a symmetric and positive definite weighting tensor, and $\boldsymbol{g}/f$ are the specific body/volumetric source terms respectively. The above weak form will result in a dicrete first order symmetric and positive definite set of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, what is the &quot;best&quot; solver/preconditioner for this type of problem? I have tried things like the Conjugate gradient method with Jacobi preconditioning, but it seems the number of iterations increases proportionally with problem size (thus, losing some scalability in the strong sense)&lt;/p&gt;&#xA;" OwnerUserId="14576" LastEditorUserId="713" LastEditDate="2015-09-21T07:07:07.833" LastActivityDate="2015-09-21T07:07:07.833" Title="Best solver/preconditioner for least-squares finite element method" Tags="|linear-solver|preconditioning|least-squares|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20753" PostTypeId="2" ParentId="20752" CreationDate="2015-09-17T04:08:53.627" Score="2" Body="&lt;p&gt;Typically multigrid, since least squares FEM usually results in a second order system for which multigrid does well. The convergence of least squares methods is also based on the equivalence of a problem-dependent norm with more standard Sobolev norms, and the same equivalence may sometimes be used to construct appropriate smoothers for multigrid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: for examples, google searches for FOSLS and multigrid yield &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=3&amp;amp;ved=0CCkQFjACahUKEwjjotau_IDIAhUIig0KHUxFBes&amp;amp;url=http%3A%2F%2Famath.colorado.edu%2Fpub%2Ffosls%2Fnonlinelas.pdf&amp;amp;usg=AFQjCNGS7C7deiZzf2DqI9UFWv2eNNzXxQ&amp;amp;sig2=EyxwZuDexILnXYRLfSKb_g&amp;amp;bvm=bv.102829193,d.eXY&amp;amp;cad=rja&quot; rel=&quot;nofollow&quot;&gt;some&lt;/a&gt; &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=6&amp;amp;ved=0CDwQFjAFahUKEwjjotau_IDIAhUIig0KHUxFBes&amp;amp;url=http%3A%2F%2Flukeo.cs.illinois.edu%2Ffiles%2F2005_HeMaMcOl_hoamg.pdf&amp;amp;usg=AFQjCNG_q7m-cinZp8R6DTuGTluHoPFA-A&amp;amp;sig2=Z6kCBQ-ra25PM9S4-Bk7ng&amp;amp;bvm=bv.102829193,d.eXY&amp;amp;cad=rja&quot; rel=&quot;nofollow&quot;&gt;good&lt;/a&gt; &lt;a href=&quot;http://amath.colorado.edu/pub/fosls/hyp1.pdf&quot; rel=&quot;nofollow&quot;&gt;results&lt;/a&gt;. You can also find a good discussion in Gunzberger/Bochev's &quot;Least-Squares Finite Element Methods&quot; book.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-09-18T16:14:20.150" LastActivityDate="2015-09-18T16:14:20.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20756" PostTypeId="2" ParentId="20736" CreationDate="2015-09-17T06:53:53.493" Score="0" Body="&lt;p&gt;I write this as an answer, since I cannot comment. As already explained, Abaqus is not well suited for your problem. You should have a look at Fenics (&lt;a href=&quot;http://fenicsproject.org/&quot; rel=&quot;nofollow&quot;&gt;http://fenicsproject.org/&lt;/a&gt;), which is a general purpose PDE solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I learned that Comsol also provides an interface for user-defined PDE and weak forms. &lt;/p&gt;&#xA;" OwnerUserId="16954" LastEditorUserId="16954" LastEditDate="2015-12-22T11:53:45.723" LastActivityDate="2015-12-22T11:53:45.723" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20758" PostTypeId="2" ParentId="20737" CreationDate="2015-09-17T11:53:06.657" Score="0" Body="&lt;p&gt;Although both equations are uncoupled in the absence of Fy, they are both being solved together and if there is a problem solving one equation this could be a problem solving the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I copied your code and ran it. The equation for y is unstable and quickly returns NaN. However, for me, the result for x is the same whether I include Fy or not.&lt;/p&gt;&#xA;" OwnerUserId="17577" LastEditorUserId="17577" LastEditDate="2015-09-17T12:07:23.387" LastActivityDate="2015-09-17T12:07:23.387" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20759" PostTypeId="1" CreationDate="2015-09-17T12:17:36.933" Score="1" ViewCount="2750" Body="&lt;p&gt;I want to build a new desktop PC for MD simulations of relatively simple protein/carbohydrate systems using mainly GROMACS and Amber force field. No ab initio or QM-MM simulations are intended for now. For instance, I have been working with a protein of approximately 1500 atoms in the presence of small carbohydrates (&amp;lt;100 atoms). For the protein alone, my home PC (i5-4570 CPU 3.20GHz, not GPU-accelerated) does approximately 5 ns/day, so I want something considerably faster than that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which hardware should I choose? Could you suggest low (800 US dollars) to high (2000 US dollars) price configurations with good power considering the current technology available? I am not sure if I need a GPU-accelerated machine, so I am open for different configurations.&lt;/p&gt;&#xA;" OwnerUserId="17578" LastEditorUserId="17578" LastEditDate="2015-09-17T12:39:29.707" LastActivityDate="2021-05-08T11:54:55.030" Title="Good desktop PC for molecular dynamics simulations" Tags="|simulation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20760" PostTypeId="2" ParentId="20759" CreationDate="2015-09-17T13:11:05.657" Score="2" Body="&lt;p&gt;I don't have experience with GROMACS, or even MD simulations, so take my analysis with a grain of salt, but GROMACS provides what appears to be a &lt;a href=&quot;http://www.gromacs.org/GPU_acceleration&quot; rel=&quot;nofollow&quot;&gt;good analysis&lt;/a&gt; of their performance on different hardware configurations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the simulation time per day would depend highly on your molecular system, but they show speedups of 5x using a GTX Titan (the high end or above your price range) over a high end CPU and x3-x4 out of a GTX680 (I would consider this a consumer card, you can get GTX 700-series for a couple hundred dollars on newegg).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on your exact budget you should be able to accelerate your computations by a factor of x3-x5 without too much trouble as long as you do your research on which GPU/CPU combination to buy.  Just looking at their information, I would highly recommend going the GPU accelerated route.&lt;/p&gt;&#xA;" OwnerUserId="1552" LastActivityDate="2015-09-17T13:11:05.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20761" PostTypeId="1" CreationDate="2015-09-17T13:57:32.197" Score="3" ViewCount="334" Body="&lt;p&gt;I am looking to find an algorithm/method by which I can detect regions of flow in a video. The video is that of a grayscale heatmap where the majority of the image is not in motion. The areas that are in motion generally appear as flickering/moving white dots, whereas in the rest of the image the dots do not move. The easiest way to visualize this is to think of a TV screen with noise, but the noise is only changing in certain areas; I would like to detect these areas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current method being implemented to do so is simply to calculate the standard deviation of each pixel over time and use this as a rough estimate of movement. While this does produce the desired image, and it is rather fast, I'm not a huge fan of the method because it doesn't care about how fast things are changing (std(sin(x) == std(sin(2x))).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based off of &lt;a href=&quot;http://live.ece.utexas.edu/publications/2009/jl_icip_2009.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper by the University of Texas&lt;/a&gt;, I tried to estimate regions of flow by the Black and Anandan algorith. Specifically, I used &lt;a href=&quot;http://cs.brown.edu/~black/code.html&quot; rel=&quot;nofollow&quot;&gt;this piece of Matlab code&lt;/a&gt; produced by Brown University. Unfortunately, not only does it take &lt;i&gt;forever&lt;/i&gt; to run, but it really does not seem to be working (shows areas of high flow where there most certainly is not any). I don't have much of a background in image/signal processing so it is entirely possible that I'm messing up the implementation, but I don't think this is the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this: can anyone think of any alternatives I could use that help to estimate regions of flow? Also, am I wrong in thinking that the current method (taking the standard deviation wrt. time) is flawed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any help. &lt;/p&gt;&#xA;" OwnerUserId="17579" LastActivityDate="2015-09-25T14:57:09.043" Title="Estimating Regions of Flow in Grayscale Video" Tags="|matlab|image-processing|signal-processing|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20762" PostTypeId="2" ParentId="20759" CreationDate="2015-09-17T14:09:48.933" Score="6" Body="&lt;p&gt;There is a recently published paper addressing the question of what is the optimal combination of hardware on which to run GROMACS:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kutzner, C., Páll, S., Fechner, M., Esztermann, A., de Groot, B. L., &amp;amp; Grubmüller, H. (2015). Best bang for your buck: GPU nodes for GROMACS biomolecular simulations. Journal of Computational Chemistry, (Spp 1648). &lt;a href=&quot;http://doi.wiley.com/10.1002/jcc.24030&quot;&gt;doi:10.1002/jcc.24030&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-09-17T14:09:48.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20764" PostTypeId="1" AcceptedAnswerId="20770" CreationDate="2015-09-18T08:14:04.500" Score="1" ViewCount="90" Body="&lt;p&gt;I am looking for an algorithm with the following characteristics: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It is used to generate the set of integer vectors $\mathbf k=(k_1,\ldots,k_m)$, where $k_i\leq K_i$, $k_i\geq 0$, and $K_i$ are given positive integers. That is, I am looking for the cartesian product of the sets $\{1,\ldots,K_i\}$.&lt;/li&gt;&#xA;&lt;li&gt;Call $k=k_1+\ldots+k_m$. The output vectors are generated for increasing $k$. So first goes the vector with $k=0$, then the ones with $k=1$, then those for $k=2$, etc.&lt;/li&gt;&#xA;&lt;li&gt;The above is done without full enumeration or use of persistent data structures. That is, given the $i$-th generated vector, $\mathbf k_i$, the function should return a vector $\mathbf k_{i+1}$ that has not been generated before, if one exists. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is fairly easy to generate the $\mathbf k$ vectors without the second requirement, but I am not sure how to handle that one.  &lt;/p&gt;&#xA;" OwnerUserId="17589" LastEditorUserId="651" LastEditDate="2015-09-19T02:45:31.403" LastActivityDate="2015-09-19T02:45:31.403" Title="Algorithm for generating the next m-tuple of integers, subject to constraints" Tags="|algorithms|combinatorics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20766" PostTypeId="1" CreationDate="2015-09-18T15:36:04.347" Score="0" ViewCount="940" Body="&lt;p&gt;I want to solve a system of 7 coupled differential equations and 1 algebraic equation in MATLAB with the method of lines. I could do it for each independent  equation  with some assumptions, but I can't solve these 8 equation together. Can you help me?&lt;/p&gt;&#xA;" OwnerUserId="17594" LastEditorUserId="9667" LastEditDate="2015-09-18T17:18:27.447" LastActivityDate="2015-09-21T07:50:11.740" Title="Solving coupled differential equations and Algebraic equation in MATLAB" Tags="|matlab|pde|nonlinear-equations|method-of-lines|modeling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20767" PostTypeId="2" ParentId="7158" CreationDate="2015-09-18T16:22:41.920" Score="0" Body="&lt;p&gt;If you work backwards from the end formulations, you should find that SUPG is equivalent to RANS with a particular turbulence model or vice-versa. So, you should be able to run an SUPG model at a high Reynolds number and get an answer that's equivalent to having run a RANS model with a particular, but probably nonsensical and very non-physical turbulence model. I haven't done the analysis myself, but it should be straightforward to work out the equivalence.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-09-18T16:22:41.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20769" PostTypeId="2" ParentId="20719" CreationDate="2015-09-18T20:55:05.420" Score="1" Body="&lt;p&gt;In Multigrid used as a &lt;em&gt;solver&lt;/em&gt;, usually the &lt;em&gt;relative norm&lt;/em&gt; of the residual is used as the stopping criterion. As you decrease this ratio - the accuracy of the solution should increase. Further, at the &lt;em&gt;coarsest level&lt;/em&gt; researchers do different things : &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;either solve using a &lt;strong&gt;direct solver&lt;/strong&gt; (no convergence)&lt;/li&gt;&#xA;&lt;li&gt;use &lt;strong&gt;fixed iterations&lt;/strong&gt; (no convergence)&lt;/li&gt;&#xA;&lt;li&gt;use the &lt;strong&gt;successive difference&lt;/strong&gt; between iterations as convergence criterion (not a good method because you could be far away from the solution)&lt;/li&gt;&#xA;&lt;li&gt;Again use &lt;strong&gt;relative norm&lt;/strong&gt; of residual as a stopping criterion.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Method 2 listed above at the coarsest level is good when Multigrid is used as a &lt;em&gt;Pre-conditioner&lt;/em&gt; (Multigrid experts here can comment - I am a beginner).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, &lt;strong&gt;in general convergence is used&lt;/strong&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="14303" LastActivityDate="2015-09-18T20:55:05.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20770" PostTypeId="2" ParentId="20764" CreationDate="2015-09-19T01:08:52.247" Score="1" Body="&lt;p&gt;Modulo an inessential detail, we are asked to generate mixed radix numbers with certain &quot;digit&quot; sums $k=k_1+\ldots+k_m$.  In particular, given $\mathbf{k_i}$ satisfying the sum of components $k$, we are asked to find either the next larger representation having the same component sum, or if none exists, the smallest representation having component sum $k+1$, which we call $\mathbf{k_{i+1}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asking for the &quot;next&quot; $\mathbf{k_i}$ only makes sense if an ordering is imposed among solutions with a fixed component sum $k$.  We will assume a lexicographic ascending order below, but a descending order would be as easy to arrange.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is clear enough: find the rightmost (maximum) index $j$ such that we can increment $k_j$ by one and (in compensation) decrease by one the sum of components with index greater than $j$ in the way that represents the smallest possible ($m-j$ component) mixed radix number (with the specified component sum).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;boundary&quot; case where no such index $j$ is possible, so that we increment $k$ and start over with the smallest possible mixed radix number with component sum $k+1$, is essentially the same as the latter subproblem of the typical step.  It therefore bears discussing how to find the smallest number with a given sum $k$ of entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately this subproblem has an easy solution: we choose digits from right to left in a greedy fashion, as large as possible subject to the constrained sum $k$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is good point to pin down the inessential detail mentioned at the outset.  The Question poses a sum of components $k_i \in \{1,\ldots,K_i\}$ (although confusingly, the condition $k_i \ge 0$ is also stated).  If the components are all required to meet $k_i \ge 1$ (or for that matter any other nonzero lower bounds, possibly determined by the index $i$), we cannot achieve any sum of components less than $m$ (seeing as there are $m$ components to be added).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore it is convenient to subtract one from each component, choosing instead that $k_i \in \{0,\ldots,K_i-1\}$, which makes the analogy with mixed radix representations more explicit.  Correspondingly, the digit sums can then progress from $\sum_{i=1}^m k_i = 0$ to $\sum_{i=1}^m (K_i - 1)$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Least representation of rightmost $m-j$ digits having component sum $c$&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Set $\ell = m$.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;While $\ell &amp;gt; j$:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Set $k_\ell = \max \{c, K_\ell - 1 \}$.&lt;/li&gt;&#xA;  &lt;li&gt;Set $c = c - k_\ell$.&lt;/li&gt;&#xA;  &lt;li&gt;Set $\ell = \ell - 1$.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;Return success if $c = 0$, otherwise fail.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Find rightmost index $j$ such that $k_j$ can be incremented&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Scan from right to left to find an index $j \gt 1$ such that $k_j \gt 0$.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If no such index exists, fail.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Do:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Set $j = j-1$.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;Until $k_j \lt K_j - 1$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For efficiency we can keep a running sum $c$ of entries above index $j$, since we will offset the increment of $k_j$ by decrementing the sum $c$ of components beyond that point.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2015-09-19T01:08:52.247" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20772" PostTypeId="1" CreationDate="2015-09-19T09:24:27.540" Score="0" ViewCount="4864" Body="&lt;p&gt;I want to maximize an objective function using quasi-newton optimization method. But i couldn't find any function for maximizing a function. All inbuilt functions are for minimization. For eg. fminunc in matlab. How can i change my objective function to give it as an input to a minimization function, say fminunc and get the result for maximization.&lt;/p&gt;&#xA;" OwnerUserId="17601" LastActivityDate="2015-09-19T09:29:39.210" Title="Changing min function to max function for optimization" Tags="|optimization|matlab|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20773" PostTypeId="2" ParentId="20772" CreationDate="2015-09-19T09:29:39.210" Score="4" Body="&lt;p&gt;If you want to maximize f(X) subject to specified constraints (if any), then minimize -f(X) subject to those same constraints.  Your optimal objective function will be the negative of -f(X_optimal), where X_optimal is the optimal argument value, which is the same for max f(X) and min -f(X).&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2015-09-19T09:29:39.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20774" PostTypeId="1" CreationDate="2015-09-19T16:21:48.457" Score="1" ViewCount="60" Body="&lt;p&gt;You have questions through out the course that are worth 2 (if you get it perfect), 1 or 0 marks. The professor only takes into account the top 80% of questions, for example 2/2, 2/2, 1/2, 1/2, 0/2 and only 2/2, 2/2, 1/2 would count towards your grade. You are given the number of 2's, 1's, 0's and the total amount of questions ( total # will be divisible by 5). Is there a way to calculate your grade in that course without using if statements, inequalities and recursions (only math functions like subtract, add, multiple, divide, max, min). I've been stuck on this question for a while, if you can give me a hint on how to solve this, it would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17602" LastActivityDate="2015-09-20T15:13:14.880" Title="Is there a way to calculate the top 80% of marks without using if statements or inequalities?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="5" ClosedDate="2015-09-22T03:29:13.420" ContentLicense="CC BY-SA 3.0" />
  <row Id="20775" PostTypeId="1" CreationDate="2015-09-19T16:47:59.673" Score="1" ViewCount="91" Body="&lt;p&gt;I have an object with surface charge $\sigma$ for which I want to caclulate tangential electric field which would correspond to mathematical formulation:&#xA;$$&#xA;\vec{E}(\vec{x}) = \nabla_x \int \frac{\sigma_y}{|\vec{x} - \vec{y}|} dS_y&#xA;$$&#xA;where afterwards normal component $\vec{n} (\vec{E} \cdot \vec{n})$ is substracted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now I have tried to put this surface charge on surface mesh nodes and then transforming the integral to summations (skipping singularity) assuming that integral value on triangle is average of it on nodes. In this way my calculation totally fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As looking why it does not work I started to suspect that higly singular kernel in the integral. To overcome it I look on some different ways:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Interpolating the charge density piecwise linearly and then doing the integration&lt;/li&gt;&#xA;&lt;li&gt;Push the charges inside the suface about a size of mesh triangle&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What other options are available and which do you recommend? Also would it help if I calculate potential either of theese methods and afterwards taking derivative of it? &lt;/p&gt;&#xA;" OwnerUserId="8344" LastActivityDate="2015-09-19T16:47:59.673" Title="Calculating tangential electric field intensity on the boundary" Tags="|electromagnetism|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20776" PostTypeId="2" ParentId="20774" CreationDate="2015-09-19T21:57:47.677" Score="0" Body="&lt;p&gt;Is there a way? Yes, there are even Turing complete programming languages that don't have conditionals (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Unlambda&quot; rel=&quot;nofollow&quot;&gt;unlambda&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most obvious way to do this would be to create a sorted list of the marks by using loops and min/max functions and then just use the first or last 80% of the entries in your list.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-09-20T15:13:14.880" LastActivityDate="2015-09-20T15:13:14.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20777" PostTypeId="2" ParentId="6706" CreationDate="2015-09-20T06:28:52.520" Score="3" Body="&lt;p&gt;I have been using pari/gp (confuration displayed in output below) to calculate determinants of large, dense matrices. For example, I use a quad core i7, 16GB RAM, 32GB swap space on internal ssd, linux laptop, to calculate a matrix 1700x1700 to a precision of a little less than 3000 digits. That takes about 17 hours, using all eight threads, 100% of the time, but only a little RAM. Then, it does the determinant in another six hours, one core uses 100% for the determinant, and kswapd uses another core, 100% of the time (and appears to be able to keep up, and use all the swap space). The determinant is not parallelized in pari/gp (as far as I know, except in that kswapd and the gp instance use separate cores). Here is a specific example (that doesn't use parallel code), calculates a random (floating point), 1000-digit precision, 300x300 matrix determinant:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know it's not matlab, but in my mind, this is a better way.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;=========================== cut here&#xA;default(nbthreads,2); \\this example sets up two but uses only one thread.&#xA;default(threadsize,  500 000 000);&#xA;default(parisize,  1 800 000 000); \\ make sure there is enough RAM allocated&#xA;&#xA;initrand()=&#xA;{&#xA;  cmd = &quot;date +\&quot;%s\&quot;&quot;;           \\ a linux command to get the time&#xA;  seed = eval(externstr(cmd)[1]); \\ evaluate the external command&#xA;  setrand(seed);                  \\ set the random seed in pari/gp&#xA;  print (&quot;Random Number Seed = &quot;, seed); \\ this is what it is&#xA;  return(seed);                   \\ don't really use this value any more&#xA;}&#xA;{&#xA;  default(realprecision, 1000);  \\ accurate to 1000 digits of precision&#xA;  initrand();                    \\ pari/gp seems to need a different seed every time&#xA;  M = matrix(300,300,i,j,random(1.0)); \\ some small random matrix&#xA;  printf (&quot;%10.7f\n&quot;, M[1,1]);   \\ print the first matrix element&#xA;  detM = matdet(M);              \\ calculate the determinant&#xA;  print (detM);                  \\ print the result of the determinant&#xA;  quit(); &#xA;}&#xA;========================================= cut here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the output, which took a minute or so on this eight-year old laptop&#xA;(dual core, 2GB ram, not a fast ThinkPad).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                        GP/PARI CALCULATOR Version 2.7.3 (released)&#xA;                amd64 running linux (x86-64/GMP-6.0.0 kernel) 64-bit version&#xA;             compiled: Mar 20 2015, gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) &#xA;                                 threading engine: pthread&#xA;                       (readline v6.3 enabled, extended help enabled)&#xA;&#xA;                           Copyright (C) 2000-2015 The PARI Group&#xA;&#xA;PARI/GP is free software, covered by the GNU General Public License, and comes WITHOUT ANY WARRANTY WHATSOEVER.&#xA;&#xA;Type ? for help, \q to quit.&#xA;Type ?12 for how to get moral (and possibly technical) support.&#xA;&#xA; parisize = 8000000, primelimit = 500000&#xA;   ***   Warning: new stack size = 1800000000 (1716.614 Mbytes).&#xA; Random Number Seed = 1442728837&#xA; 0.1458753&#xA; -34107869659292788498041296259204073212856192996487539539650250444333175626443030051167504275069295850290060165906721403518710248631573270877318755.16606345550957294257524953917496320048631959234301670865079024142295995123023952749154306602088800645390346232704746782547685913565513791084278569243333238872632785910522536912572116356801097060790810716175151647494785903522152367435398630683566431045727806979724702836066399155098909578586486867501579225868835988817237083450602429253202805656099228929576538193846647490928888795367874033893827126923874835063405546089363260880884557106553366031910609740691344849673406340416580053579023527252144631532815837507760281113528779522639792441184911588021824240226126036961995649658869925314847019638680109421905217088024624421267647700950443089481208575217193882687232501653341331383260930401032570115570957029275695235564734394098177016567882258012726222978777317463869837697774536180341558237759707887932943904115121644125119662351721076540409972736105729&#xA;Goodbye!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course, in this case, the &quot;-341......05729&quot; is the floating point result. I haven't played with integer matrix elements, but I'm sure it is just as good, if not better.&lt;/p&gt;&#xA;" OwnerUserId="17605" LastEditorUserId="17605" LastEditDate="2015-09-20T06:35:37.430" LastActivityDate="2015-09-20T06:35:37.430" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20778" PostTypeId="2" ParentId="20766" CreationDate="2015-09-20T13:20:52.073" Score="2" Body="&lt;p&gt;What you describe is called Differential-Algebraic Equation (DAE) system. Depending on the &lt;em&gt;index&lt;/em&gt; of the system, these can be easy to solve or very hard. Take a look at: &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/7481-manuscript-of--solving-index-1-daes-in-matlab-and-simulink-&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/7481-manuscript-of--solving-index-1-daes-in-matlab-and-simulink-&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solvers &lt;em&gt;ode15s&lt;/em&gt; and &lt;em&gt;ode23t&lt;/em&gt; of Matlab can handle index-1 DAEs. If the system is &lt;em&gt;stiff&lt;/em&gt;, ode15s is more suitable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in the theory behind them, or what is &lt;em&gt;index&lt;/em&gt; and &lt;em&gt;stiff&lt;/em&gt;, take a look at: &lt;a href=&quot;http://www.scholarpedia.org/article/Differential-algebraic_equations&quot; rel=&quot;nofollow&quot;&gt;http://www.scholarpedia.org/article/Differential-algebraic_equations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;I didn't realize the OP was asking specifically for the method of lines. There is a section about DAEs here: &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines&quot; rel=&quot;nofollow&quot;&gt;http://www.scholarpedia.org/article/Method_of_lines&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1602" LastEditorUserId="1602" LastEditDate="2015-09-21T07:50:11.740" LastActivityDate="2015-09-21T07:50:11.740" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20779" PostTypeId="1" CreationDate="2015-09-20T14:37:46.320" Score="0" ViewCount="49" Body="&lt;p&gt;I have solved a numerical problem using two different sets of non-linear governing equations. I want to get an understanding of the stability of the methods relative to each other. To do so, I solving the problem with each method whilst keeping the time stepping schemes, time step and mesh sizes the same. I then run the code, and observe the largest time step using which stable results are produced or convergence is obtained. I wanted to know whether this makes sense. I know this process does not get definite answer about stability of the methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also what I have found that for this particular problem, stable results are still obtained even with CFL numbers greater than 1 (very large time steps). Do the large time steps I try still need to satisfy CFL? &lt;/p&gt;&#xA;" OwnerUserId="17476" LastEditorUserId="17476" LastEditDate="2015-09-20T14:53:45.477" LastActivityDate="2015-09-20T14:53:45.477" Title="comparison of stability of two non-linear methods" Tags="|fluid-dynamics|nonlinear-equations|stability|time-integration|numerical-modelling|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20780" PostTypeId="1" AcceptedAnswerId="20806" CreationDate="2015-09-20T16:16:14.837" Score="1" ViewCount="212" Body="&lt;p&gt;I am developing a finite element simulation and want to evaluate the errors in $H^1$ and $L_2$ norms. The problem is the classical Poisson equation, with Dirichlet B.C.:&#xA;$$-\Delta u=f\mbox{  in }\Omega,$$&#xA;$$u=g_{D}\mbox{ on }\Gamma_{D}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Lagrange bilinear 2/3D elements in the Galerkin approach. The error estimates are the standard&#xA;$$\left\Vert u-u_{h}\right\Vert _{H^{1}\left(\Omega\right)}\leq C\,h\,\left\Vert u\right\Vert _{H^{1}\left(\Omega\right)}$$&#xA;and &#xA;$$\left\Vert u-u_{h}\right\Vert _{L_{2}\left(\Omega\right)}\leq C\,h^{2}\,\left\Vert u\right\Vert _{L_{2}\left(\Omega\right)}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After computing the numerical errors and calculating the experimental order of convergence, how can I evaluate how good is the experimental order? I mean, if I have an order of $2.30$ for the $L_2$ norm, is that good enough? What if it is $1.70$? What criteria can I use?&lt;/p&gt;&#xA;" OwnerUserId="16667" LastEditorUserId="713" LastEditDate="2015-09-20T20:14:05.800" LastActivityDate="2015-09-23T14:29:47.523" Title="Evaluate numerical error estimates" Tags="|finite-element|error-estimation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20782" PostTypeId="1" CreationDate="2015-09-21T04:02:12.200" Score="1" ViewCount="424" Body="&lt;p&gt;I have just started using dl_poly classic to work in Molecular Dynamics simulations. It produces a HISTORY file which records the trajectory after applying boundary conditions. Now, here I don't understand, &lt;strong&gt;how I am suppossed to calculate diffusivity using folded trajectories&lt;/strong&gt;. Secondly is there a way in dl_poly to unfold the trajectories may be using some counters that they might already written (i.e. to do it without altering the source code). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from one of the answers based on a question here, I wrote a code to unfold the folded trajectories &lt;em&gt;(am trying to verify this from a program I wrote myself for the simulation of argon liquid, not dl_poly)&lt;/em&gt; -&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;open(unit=11,file='test.out',status='unknown')&#xA;&#xA;  rx(:,1) = x(:,1)&#xA;  do j=2,tsteps&#xA;  do i=1,nm&#xA;&#xA;    tmp1 = x(i,j-1) - x(i,j) + boxl/2.d0&#xA;    tmp1 = tmp1 - boxL*floor(tmp1/boxL)&#xA;    tmp2 = boxL/2.d0 - tmp1&#xA;    rx(i,j) = rx(i,j-1) + tmp2&#xA;    write(11,*) i,j,x(i,j),ox(i,j),rx(i,j),ox(i,j)-rx(i,j)&#xA;  end do&#xA;  end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the above program, &lt;code&gt;x(i,j)&lt;/code&gt; are folded coordinates, &lt;code&gt;ox(i,j)&lt;/code&gt; are the unfolded ones that are there already (this is the output a simulation program I wrote) and &lt;code&gt;rx(i,j)&lt;/code&gt; are the coordinates I am unfolding from &lt;code&gt;x(i,j)&lt;/code&gt;, where &lt;code&gt;i&lt;/code&gt; corressponds to $i^{th}$ particle and &lt;code&gt;j&lt;/code&gt; corresponds to $j^{th}$ timestep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I am facing with the above program is that, in some cases, I get a difference of &lt;code&gt;+boxl&lt;/code&gt; or &lt;code&gt;-boxl&lt;/code&gt; from the actual unfolded coordinates (i.e. between, &lt;code&gt;ox(i,j)&lt;/code&gt; and &lt;code&gt;rx(i,j)&lt;/code&gt;  )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/12956/molecular-dynamics-diffusion-with-pbc/14273#14273&quot;&gt;answer&lt;/a&gt; I used to write down this logic, but I might written it down wrongly as well.&lt;/p&gt;&#xA;" OwnerUserId="17599" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-05-18T07:19:56.017" Title="Unfolding folded trajectories for Diffusivity calculation - MD" Tags="|molecular-dynamics|advection-diffusion|diffusion|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20783" PostTypeId="1" CreationDate="2015-09-21T08:37:25.690" Score="2" ViewCount="528" Body="&lt;p&gt;My question concerns the following paper: A Conservative Adaptive Projection Method for the Variable Density Incompressible Navier–Stokes Equations (&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999198958909&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0021999198958909&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It describes a second order accurate projection method to solve incompressible Navier Stokes equations. Usually projection methods are based on implicit schemes, but as far as I can figure out, this is based on explicit methods. My motive is to solve incompressible NS equations on a uniform grid using this method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve for the non-linear convection term of momentum equation, the authors first extrapolate the cell-centered velocities to cell faces using predictor-corrector upwind schemes. This is an extrapolation in space (from cell center to cell face) and also an extrapolation in time (from time level $n$ to level $n+\frac{1}{2}$). After they calculate normal velocities on cell faces, they enforce the divergence constraint at level $n+\frac{1}{2}$. For this, they are using the MAC projection in which they write:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vK0qH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vK0qH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where D implies divergence, G implies gradient.&#xA;This equation is used to solve for $\phi^{MAC}$&#xA;But I do not understand how to get $\phi^{MAC}$ from this equation. The term on the right contains two terms in two dimensional case (corresponding to u and v velocity each). &#xA;How should I write the term on LHS in order to get  $\phi^{MAC}$ from this equation?&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2016-02-05T17:48:20.657" Title="MAC Projection in Projection method?" Tags="|finite-difference|fluid-dynamics|numerical-analysis|finite-volume|navier-stokes|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20784" PostTypeId="1" AcceptedAnswerId="20811" CreationDate="2015-09-21T11:52:41.387" Score="1" ViewCount="694" Body="&lt;p&gt;I need to solve a diffusion equation in periodic boundary conditions using fipy but I would like to have a drift term that depends on the position so like this:&#xA;$$ \partial_t u(x,t) = \partial_x^2 u(x,t) + \partial_x(\omega(x)u(x,t)). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new in fipy and I am able to solve diffusion with diffusion coefficient spatially dependent but I am having troubles with this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does someone know how to manage? Or maybe if you know an easier way to solve numerically this equation.&lt;/p&gt;&#xA;" OwnerUserId="17617" LastEditorUserId="1804" LastEditDate="2015-09-24T10:39:05.820" LastActivityDate="2015-09-24T10:54:18.980" Title="Diffusion with space dependent drift in Fipy" Tags="|pde|finite-difference|python|finite-volume|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20786" PostTypeId="1" CreationDate="2015-09-21T13:04:09.923" Score="10" ViewCount="8770" Body="&lt;p&gt;I have my own little subroutine for numerical integration (quadrature), which is a C++ adaptation of an ALGOL program published by Bulirsch &amp;amp; Stoer in 1967 (Numerische Mathematik, 9, 271-278).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to upgrade to a more modern (adaptive) algorithm and wonder whether there are any (free) C++ libraries that provide such. I had a look as GSL (which is C), but that comes with a horrible API (though the numerics may be good). Is there anything else?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A useful API would look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double quadrature(double lower_integration_limit,&#xA;                  double upper_integration_limit,&#xA;                  std::function&amp;lt;double(double)&amp;gt; const&amp;amp;func,&#xA;                  double desired_error_bound_relative=1.e-12,&#xA;                  double desired_error_bound_absolute=0,&#xA;                  double*error_estimate=nullptr);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5472" LastEditorUserId="393" LastEditDate="2015-09-23T14:38:50.807" LastActivityDate="2017-06-27T02:55:26.297" Title="C++ library for numerical intergration (quadrature)" Tags="|c++|quadrature|" AnswerCount="5" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="20787" PostTypeId="1" CreationDate="2015-09-21T13:13:48.963" Score="3" ViewCount="202" Body="&lt;p&gt;I am solving a 1D advection problem of the the form&#xA;$$dQ/dt=[A]Q$$&#xA;where {Q} is the vector of unknowns and [A] is the matrix of coefficients of spatial discretisation. I have worked out the eigenvalues of [A] to get an idea about the stability of the semi-discretised system and I am getting eigenvalues with zero real parts plus imaginary parts. Are there any good references for this?&lt;/p&gt;&#xA;" OwnerUserId="17476" LastActivityDate="2015-09-23T14:54:40.267" Title="what do zero real parts of eigenvalues mean? Any good references?" Tags="|fluid-dynamics|eigenvalues|stability|hyperbolic-pde|advection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20788" PostTypeId="2" ParentId="20783" CreationDate="2015-09-21T14:12:30.270" Score="1" Body="&lt;p&gt;Without entering much into the details of your specific problem, let's write your problem in a simplified manner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \cdot  \left(\frac{\nabla \phi}{\rho}\right) = F $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $F(U,\delta t)$ is already given by the predictor step. The important thing to remember is that $U$ is given on the faces of the cell, thus some interpolation has to be made in order to have all coefficients &quot;in place&quot;. Assuming that the coefficients are sufficiently smooth, the Gauss-Seidel iteration scheme in one of the directions (x, say) for the problem&#xA;in $\Omega=[0,1]^2$ and $u=\phi/\rho $:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$d^2u/dx^2 = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(0)=u(1)$ -Periodic B.C. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;is, using a second order centered discretization, written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_i^{(k+1)} = 2u_i^{(k)} - \frac{1}{2}(h^2F - u_{i-1}^{(k+1)}-u_{i+1}^{(k)})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $h$ is the grid spacing and $u^{(k)}$ is the value of the grid function in the kth iteration. From here, is up to you and the problem at hand which solver to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good, general reference is the book by Saab (2003). The Author has put this version for free in his webpage. On multigrid, the classic paper of Achi Brandtl (1977) is pretty good on the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;APPENDIX&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a linear system $A x = b$, where $A = (a_{ij})$; a fixed point iteration scheme, known as Gauss-Seidel, can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ x^{(k+1)}_i = x^{(k)}_i + \frac{1}{a_{ii}}\left(b-\sum_{i=1}^{j-1}a_{ij}x^{(k+1)}_i -\sum_{i=1}^{j}a_{ij}x^{(k)}_i \right) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, in fact, no iteration over $i$ is needed since the resulting system is explicit in nature (i.e.: the $k+1$ value on the RHS is known from the previous iteration).&lt;/p&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="-1" LastEditDate="2015-10-08T15:22:21.013" LastActivityDate="2015-10-08T15:22:21.013" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20790" PostTypeId="2" ParentId="20787" CreationDate="2015-09-22T02:30:50.467" Score="3" Body="&lt;p&gt;In the 2-D case, this corresponds to an elliptic fixed point (an orbital, I believe). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might look into Lyapunov stability, hopefully someone will be able to recommend a good resource on that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll try to have another look later to see if I can find resources on this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For related material, you might look into limit cycles, or the Poincaré-Bendixson theorem. I've managed to find a reference for the latter. You might also try searching on math.stackexchange for dynamical systems references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Differential Equations, Dynamical Systems, and an Introduction to Chaos&lt;/em&gt; -  Hirsch, Smale, Devaney&lt;/p&gt;&#xA;" OwnerUserId="17112" LastEditorUserId="17112" LastEditDate="2015-09-23T01:41:35.123" LastActivityDate="2015-09-23T01:41:35.123" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20791" PostTypeId="1" AcceptedAnswerId="20804" CreationDate="2015-09-22T03:35:24.273" Score="0" ViewCount="40" Body="&lt;p&gt;I'm a little bit about the SVM.&#xA;I'm actually understanding for which purpose Support Vector Machines are used.&#xA;The aim is to find the biggest (best) hyperplane between two different data clusters.&#xA;The question is here do we apply SVM after we have classified data points or is Classification also a task of SVM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The examples on videos begins always with predefined positive and negative / or blue and red data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="17629" LastActivityDate="2015-09-23T12:56:48.303" Title="Support Vector Machines to use after Classification?" Tags="|support-vector-machines|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20792" PostTypeId="1" AcceptedAnswerId="20800" CreationDate="2015-09-22T04:26:06.133" Score="4" ViewCount="417" Body="&lt;p&gt;I'm looking for a program to obtain meshes to finite element codes 2D and 3D as complete as possible, preferably in fortran 90 or C/C++.&#xA;For example, softwares &quot;Triangle&quot; or  &quot;TetGen&quot; generate meshes but I would like a software that creates meshes with more information (for example, normal vectors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;May you suggest to me some software?&lt;/p&gt;&#xA;" OwnerUserId="17630" LastActivityDate="2015-09-22T17:24:36.533" Title="Finite element mesh software" Tags="|finite-element|fortran|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20795" PostTypeId="1" AcceptedAnswerId="20798" CreationDate="2015-09-22T11:13:02.000" Score="4" ViewCount="2037" Body="&lt;p&gt;I acknowledge the following post where a similar question is posed and a very nice answer has been provided: &#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/11590/is-there-a-mesh-generator-that-will-generate-zero-thickness-elements-for-interfa&quot;&gt;Is there a mesh generator that will generate zero thickness elements for interfaces?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the interface created there consists of 2 noded line elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is as follows: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a convenient method of inserting quad elements with (originally) overlapping nodes, i.e. with a zero thickness. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RSiVC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RSiVC.png&quot; alt=&quot;Quad interface element&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possibly within Gmsh or alternatively an example using a similar open source mesh generator. This would require duplicate nodes along the interface, the resulting mesh following the example has only one node on every coordinate. I require quad elements along the interface to include mixed-mode debonding using a cohesive zone approach. &lt;/p&gt;&#xA;" OwnerUserId="17418" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-09-22T23:36:48.643" Title="Using Gmsh to create a mesh with zero thickness (quad) interface elements" Tags="|finite-element|mesh-generation|gmsh|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20797" PostTypeId="2" ParentId="4822" CreationDate="2015-09-22T14:40:14.977" Score="0" Body="&lt;p&gt;IEEE MICRO 2011 paper &quot;&lt;a href=&quot;http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6045685&amp;amp;tag=1&quot; rel=&quot;nofollow&quot;&gt;GPUs and the future of parallel computing&lt;/a&gt;&quot; shows that state-of-the-art CPUs and GPUs incur ~1700pJ and ~225pJ, respectively, for each floating-point operation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For data storage/access cost in different memories (e.g. SRAM, PCM, eDRAM), one has to use tools like &lt;a href=&quot;http://hplabs.hp.com/people/naveen_muralimanohar/micro07.pdf&quot; rel=&quot;nofollow&quot;&gt;CACTI&lt;/a&gt; and &lt;a href=&quot;https://www.academia.edu/9741921/Exploring_Design_Space_of_3D_NVM_and_eDRAM_Caches_Using_DESTINY_Tool&quot; rel=&quot;nofollow&quot;&gt;DESTINY&lt;/a&gt; (I am co-developer of DESTINY). Also, &lt;a href=&quot;http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5375438&quot; rel=&quot;nofollow&quot;&gt;McPAT&lt;/a&gt; tool can be used for modeling energy consumption of entire processor, while &lt;a href=&quot;http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5732229&quot; rel=&quot;nofollow&quot;&gt;DRAMSim2&lt;/a&gt; is for memory alone. What you may be looking, may be achieved through combination or modification of these (they are open-source). There are many others, but from these, you can find connected references through Google scholar. &lt;/p&gt;&#xA;" OwnerUserId="976" LastActivityDate="2015-09-22T14:40:14.977" CommentCount="0" CommunityOwnedDate="2015-09-22T14:40:14.977" ContentLicense="CC BY-SA 3.0" />
  <row Id="20798" PostTypeId="2" ParentId="20795" CreationDate="2015-09-22T16:09:23.823" Score="4" Body="&lt;p&gt;One way to do it might be to create an internal Line Loop in the input file where the crack originates. To do this, you would create 2 points at each end of the crack in your .geo file (say, points 1,2,3,4), and connect them up with lines. Then, create a Plane Surface from the border of that zero-thickness line loop. When you mesh the geometry, elements will hopefully be placed in this zero-thickness surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If gmsh complains about that, you may have to write a script to post-process the .msh file that comes out of the procedure in my answer in your linked question. You can identify the nodes on the interface using a physical ID. Then, lay down a line of new nodes on top of the interface nodes. Next, loop through all of the elements: Any elements below the interface (physical surface 1, in the example) can keep using the old interface nodes. Any elements above the interface (physical surface 2 in the example) should swap any old interface nodes with the new nodes that you just placed. Lastly, loop through the 2-node interface elements. Replace these with an appropriate quad element using the old and new nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you will have likely read the file into more convenient data structures and manipulated the elements/nodes within a script, you will now have to write all of this out to a valid .msh file, but that shouldn't be too hard to do if you've been able to do everything else up to this point. I'm not saying that this would be a particularly pleasant route, but if you really need the zero-thickness element, then this might be the way to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to be a popular request, so I'm going to write a matlab/python script to do it. I'll come back and edit this when I have a small working example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;===============UPDATE===================&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written a script that implements the procedure that I outlined in the second paragraph. It is too long to paste into a code block here, so I have uploaded it to my github page: &lt;a href=&quot;https://github.com/tjolsen/Mesh_Utilities/tree/master/gmsh_crack&quot; rel=&quot;nofollow&quot;&gt;https://github.com/tjolsen/Mesh_Utilities/tree/master/gmsh_crack&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have included a small working example of a .geo file and some documentation on how to use the code. The script parses a .msh file, creates new nodes and elements, then writes the &quot;cracked&quot; mesh into a new file. Anyone is, of course, welcome to use the code (provided as is...). I would welcome any feedback/improvements to the code. It has some pretty low-hanging performance improvements, which might be necessary if you start using larger meshes than the toy one that I ran on my laptop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if this does/doesn't suit your needs. If it doesn't, it is probably only a small tweak away from being what you need.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastEditorUserId="7378" LastEditDate="2015-09-22T23:36:48.643" LastActivityDate="2015-09-22T23:36:48.643" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20800" PostTypeId="2" ParentId="20792" CreationDate="2015-09-22T17:24:36.533" Score="5" Body="&lt;p&gt;I'd say &lt;a href=&quot;http://geuz.org/gmsh/&quot; rel=&quot;noreferrer&quot;&gt;Gmsh&lt;/a&gt;. I used it for a few finite element projects, and it was mostly easy to work with. The mesh output formats are very parseable, and there's at least one third-party parser (&lt;a href=&quot;https://github.com/inducer/meshpy&quot; rel=&quot;noreferrer&quot;&gt;MeshPy&lt;/a&gt;) that can parse the output. It also has a C++ API, and the mailing list gets enough traffic (probably 10-20 messages a week) that your questions might be answered (in my experience, it's something like a 1-in-2 chance).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-09-22T17:24:36.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20802" PostTypeId="1" AcceptedAnswerId="20807" CreationDate="2015-09-23T05:11:13.747" Score="5" ViewCount="1714" Body="&lt;p&gt;I want to solve this simple advection equation using the finite element method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dc}{dt}+v\cdot\nabla c = f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the best FEM discretization for this? I have tried using the standard/primitive least-squares finite element method for this but it seems I still get oscillations.&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2015-09-27T07:43:45.877" Title="Advection equation using the finite element method" Tags="|finite-element|advection|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20803" PostTypeId="2" ParentId="20478" CreationDate="2015-09-23T06:20:57.200" Score="1" Body="&lt;p&gt;No, this is not possible to cast as a convex problem, as the feasible set generically is nonconvex. Consider, e.g., the case $A = \begin{bmatrix} x &amp;amp; 0\\0 &amp;amp; 2x\end{bmatrix}$, $v = \begin{bmatrix}2\\-1\end{bmatrix}$ and $w=1/2$. This data generates the constraint $2e^x-e^{2x}=1/2$ which has two distinct feasible points ($-.123$ and $.534$). This is of course to be expected, as anything but affine equalities are nonconvex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could be that you can escape this by some logarithmic transformation, although the likelihood is very low. Depends on the specifics of all components of this model.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2015-09-23T06:20:57.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20804" PostTypeId="2" ParentId="20791" CreationDate="2015-09-23T12:56:48.303" Score="2" Body="&lt;p&gt;In machine learning, there are two types of problems (there are more, but start with 2) -- supervised and unsupervised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SVM (also logistic regression, etc) is part of &lt;a href=&quot;https://en.wikipedia.org/wiki/Supervised_learning&quot; rel=&quot;nofollow&quot;&gt;supervised learning&lt;/a&gt; where you are given two datasets. One with predictors and known desired output (blue/red) called training set, another one with only predictors. Your task is to learn the structure from the training set and try to predict the output of the test set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you saw in the video is the training part.&lt;/p&gt;&#xA;" OwnerUserId="14305" LastActivityDate="2015-09-23T12:56:48.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20805" PostTypeId="1" CreationDate="2015-09-23T13:34:17.227" Score="5" ViewCount="391" Body="&lt;p&gt;I have a large matrix $A \in \mathcal{R}^{N\times N}$ which is supposedly positive-definite, but numerically low rank. Instead of $A$, I have its incomplete Cholesky factor $G$, such that $A \simeq GG^\top$. $A$ is very large, so I can't keep it in memory, but $G$ is small enough. I need to &lt;em&gt;numerically solve&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (A^{-1} + D)^{-1} v $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ is a diagonal matrix with positive entries, and $v$ is a vector. I want to do so without forming an $N \times N$ matrix. Is this possible without further approximation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;em&gt;Context: this is the Newton-step in an optimization for variational Bayes under GP. $A$ is a Gram matrix, which I can evaluate any value if needed.&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the matrix inversion lemma, we could rewrite it as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (A - A (D^{-1} + A)^{-1} A)m \simeq G(G^\top m) - GG^\top (D^{-1} + A)^{-1} G(G^\top m)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, the $(D^{-1} + A)^{-1}$ term is still troublesome. (Perhaps a low-rank approximation of it could help?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; If only I had the incomplete Cholesky of $A^{-1} = L L^\top$, things would have been much easier. In this case, matrix inversion lemma gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (D^{-1} - D^{-1} L (I - L^\top D L)^{-1} L^\top D^{-1}) m $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be all computed in the low-rank dimensions. Now, note that the pseudo-inverse of $G$ gives $L$. Hence, an econ-SVD on $G$ would solve it. Is there a better way?&lt;/p&gt;&#xA;" OwnerUserId="1128" LastEditorUserId="1128" LastEditDate="2015-09-24T13:48:06.577" LastActivityDate="2015-10-25T20:41:49.013" Title="Solving $ (A^{-1} + D)^{-1} v $ with low rank Cholesky factors of $A$" Tags="|linear-algebra|numerical-analysis|convex-optimization|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20806" PostTypeId="2" ParentId="20780" CreationDate="2015-09-23T14:29:47.523" Score="3" Body="&lt;p&gt;The theory says that the error follows the estimates you state &lt;em&gt;asymptotically&lt;/em&gt;. It doesn't actually say very much about what the error would be &lt;em&gt;for any given mesh&lt;/em&gt; and in relation to the next mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what you need to do is to compute on finer and finer meshes. You will (or at least should) observe that the convergence rate will approach 2 if you did everything right, and that's all you typically want to demonstrate.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-09-23T14:29:47.523" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20807" PostTypeId="2" ParentId="20802" CreationDate="2015-09-23T14:36:47.147" Score="8" Body="&lt;p&gt;One way or the other, you need to stabilize every discretization for this equation. Traditionally, this was done using methods such as artificial viscosity or its slightly smarter sibling SUPG. But there are many other alternatives to make things work if you'd like to stick with the FEM framework -- e.g., discontinuous Galerkin methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My summary of this issue can be found in lecture 31 here: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-09-23T14:36:47.147" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20808" PostTypeId="2" ParentId="20787" CreationDate="2015-09-23T14:54:40.267" Score="5" Body="&lt;p&gt;Eigenvalues with zero eigenvalue correspond to purely oscillatory modes. You can see it by diagonalising the system. Your matrix $A$ can be written as&#xA;\begin{equation}&#xA;A = P \Sigma P^{-1}&#xA;\end{equation}&#xA;where $\Sigma$ is a diagonal matrix with entries $\lambda_n$ corresponding to the eigenvalues of $Q$. You can now transform your ODE to&#xA;\begin{equation}&#xA;dQ/dt = P \Sigma P^{-1} Q \Rightarrow d (P^{-1}Q)/dt = \Sigma \left( P^{-1} Q \right)&#xA;\end{equation}&#xA;since $A$ and thus $P$ do not depend on $t$. In the transformed coordinate $R:=P^{-1}Q$. the system decouples into &#xA;\begin{equation}&#xA;  dR/dt = \Sigma R&#xA;\end{equation}&#xA;and thus, as $\Sigma$ is diagonal, into a number of independent ODEs&#xA;\begin{equation}&#xA;  d r_n / dt = \lambda_n r_n&#xA;\end{equation}&#xA;with $r_n$ the n-th entry of $R$. This scalar ODE has the solution&#xA;\begin{equation}&#xA;r_n(t) = r_n(0) \exp(\lambda_n t)&#xA;\end{equation}&#xA;By Euler's formula, for $\lambda_n = i a_n$, i.e. zero real part, this becomes&#xA;\begin{equation}&#xA;  r_n(t) = r_n(0) \left( \cos(a_n t) + i\sin(a_n t) \right).&#xA;\end{equation}&lt;/p&gt;&#xA;" OwnerUserId="16921" LastActivityDate="2015-09-23T14:54:40.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20809" PostTypeId="2" ParentId="20761" CreationDate="2015-09-23T15:12:31.877" Score="1" Body="&lt;p&gt;This is not my field any more, but when I was a student, I remember people advocating the use of Markov Chain for this kind of things. Maybe something to search for, although I expect that things have moved on since. Indeed the paper you mention seems to be an evolution of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't really understand why the &quot;Texas method&quot; is not working, not having read the whole paper nor seen your data. Maybe the segmentation part of their algorithm is counter productive in your case, if the morphology of your flow is not compatible with their algorithm? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you talk about &quot;flickering/moving dots&quot;. These are not the same things. Flickering will show as a motion only if the phase of different points are correlated (is it why you use &quot;sin(x)&quot;?).&#xA;In the case of moving dots, they'll show a motion if there's coherence in their amplitude; then particle tracking software could indeed be a solution, as pointed by EngrStudent, depending how big your dots are, which again goes back to a problem of segmentation (check if one of these are working: &lt;a href=&quot;http://www.mashanov.uk/&quot; rel=&quot;nofollow&quot;&gt;http://www.mashanov.uk/&lt;/a&gt;, you may need ImageJ &lt;a href=&quot;http://imagej.nih.gov/ij/download.html&quot; rel=&quot;nofollow&quot;&gt;http://imagej.nih.gov/ij/download.html&lt;/a&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing should still be true in any case: you're saying that the &quot;dots&quot; are only changing in the area of interest. It seems to me that you could increase the speed/ease of detection by doing a Background Subtraction before tracking your flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps a bit, and thank you for having made me think again about my student years...&lt;/p&gt;&#xA;" OwnerUserId="17649" LastEditorUserId="17649" LastEditDate="2015-09-25T14:57:09.043" LastActivityDate="2015-09-25T14:57:09.043" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20810" PostTypeId="1" CreationDate="2015-09-24T08:30:41.823" Score="1" ViewCount="143" Body="&lt;p&gt;I was having some lectures and I didn't quite understand the following: let's say you have a grid like &#xA;$$G=\{ x \in \mathbb{R} : x = x_j = hj, \  j = 0,1,...,n,\ h=1/n\}.$$&#xA;And you write in difference form the following 1D diffusion  equation (comma before index indicates derivative along the index):&#xA;$$-(au_{,1})_{,1}=s, \qquad x \in \Omega := (0,1).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Giving:&#xA;$$ \{ -(a_{j-1} + a_j)u_{j-1} + (a_{j-1}+2a_j+a_{j+1})u_j-(a_j+a_{j+1}  )u_{j+1}\}/2h^2 = s_j$$&#xA;Now let's say you have a discontinuity of $a$ along some interface $\Gamma$. In our 1D problem, let:&#xA;$$a(x) = \epsilon, \ \ 0&amp;lt;x\le x^*, \ \ a(x) = 1, \ \ x^*&amp;lt;x&amp;lt;1.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the boundary conditions $u(0) = 0$ and $u(1) = 1$. This jump can be written:&#xA;$$\epsilon \,\underset{x \uparrow x^*}{\lim u_{,1}} = \underset{x \downarrow x^*}{\lim u_{,1}}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I don't understand: the lecturer claims that by proposing a &lt;em&gt;linear piecewise solution&lt;/em&gt; for the diffusion equation it is found that:&#xA;$$u= cx, \ \  0\le x&amp;lt; x^*, \ \ u=\epsilon c x + 1 -\epsilon c, \ \ x^*\le x\le1.$$&#xA;Where:&#xA;$$c= 1/(x^*-\epsilon x^* + \epsilon).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is done with the &lt;em&gt;difference equation&lt;/em&gt;, assuming $x_j &amp;lt; x^* \le x_{j+1}$, giving:&#xA;$$u_j= \alpha j, \ \  0\le j&amp;lt; k, \ \ u=\beta j + 1 -\beta n, \ \ k+1\le j\le n.$$&#xA;Where,&#xA;$$ \beta =\epsilon \alpha, \ \ \alpha=\left( \epsilon\frac{1-\epsilon}{1+\epsilon} + \epsilon(n-k)+k\right)^{-1}.$$&#xA;Hence:&#xA;$$u_k = x_k/(\epsilon h(1-\epsilon)/(1+\epsilon)+(1-\epsilon)x_k+\epsilon).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How are those Piecewise Linear Solution built? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any input will be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="14772" LastEditDate="2015-10-12T01:48:47.333" LastActivityDate="2015-10-12T01:48:47.333" Title="1-D finite differences with piecewise linear solution" Tags="|finite-difference|numerical-analysis|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20811" PostTypeId="2" ParentId="20784" CreationDate="2015-09-24T10:38:55.087" Score="2" Body="&lt;p&gt;A variable convection term works in the same way as a variable diffusion term:&lt;sup&gt;1&lt;/sup&gt; for (say) $\omega(x) = 1+x^2$ (and Dirichlet conditions for simplicity), you would write&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from fipy import *&#xA;# generate mesh&#xA;nx = 50&#xA;dx = 1.&#xA;mesh = Grid1D(nx=nx, dx=dx)&#xA;x = mesh.cellCenters[0]&#xA;# solution phi on mesh with boundary conditions phi(0)=0, phi(1)=1&#xA;phi = CellVariable(name=&quot;solution variable&quot;, mesh=mesh, value=0.)&#xA;phi.constrain(0., mesh.facesRight)&#xA;phi.constrain(1., mesh.facesLeft)&#xA;# define pde phi_t = phi_xx + ((1+x^2)*phi)_x&#xA;D = 1.&#xA;omega = 1.+x**2&#xA;eqX = TransientTerm() == ExplicitDiffusionTerm(coeff=D) + \&#xA;                         ConvectionTerm(CellVariable(mesh=mesh,value=[omega]))&#xA;# time steps&#xA;timeStepDuration = 0.9 * dx**2 / (2 * D)&#xA;steps = 100&#xA;t = timeStepDuration * steps&#xA;# solve&#xA;for step in range(steps):&#xA;    eqX.solve(var=phi, dt=timeStepDuration)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;&#xA;1. see &lt;a href=&quot;http://ctcms.nist.gov/fipy/documentation/FAQ.html&quot; rel=&quot;nofollow&quot;&gt;http://ctcms.nist.gov/fipy/documentation/FAQ.html&lt;/a&gt; (under &quot;What if my term involves the dependent variable, but not where FiPy puts it?&quot;)&#xA;&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-09-24T10:54:18.980" LastActivityDate="2015-09-24T10:54:18.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20812" PostTypeId="1" CreationDate="2015-09-24T11:17:29.063" Score="4" ViewCount="2792" Body="&lt;p&gt;I have a piece of code in Fortran90 in which I have to solve both a non-linear (with the Newton-Raphson method, for which I have to invert the Jacobian matrix) and a linear system of equations. When I say very small I mean n unknowns for both operations, with $n\leq4$. Unfortunately, $n$ is not known a priori. What do you think is the fastest option? I thought of writing explicit formulas for cases with $n=1,2$ and using other methods for $n=3,4$ (e.g. some functions of the Intel MKL libraries), for the sake of performance. Is this sensible or should I write explicit formulas for the inverse matrix also for $n=3,4$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is going to be called very many times in a Finite Element Method analysis, for this reason I was looking for the fastest solution.&lt;/p&gt;&#xA;" OwnerUserId="17659" LastEditorUserId="713" LastEditDate="2015-09-24T18:55:22.103" LastActivityDate="2018-04-08T08:51:31.030" Title="Efficently invert tiny matrix in Fortran" Tags="|finite-element|matrix|fortran|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="20813" PostTypeId="1" AcceptedAnswerId="20814" CreationDate="2015-09-24T12:02:08.637" Score="1" ViewCount="208" Body="&lt;p&gt;I have an ODE of the form&#xA;$$C_1 d_y u + C_2 (d_y u)^n = C_3 y + C_4$$&#xA;with boundary conditions $u(0) = 0, d_y u(L) = 0$, where $C_1 \to C_4$ are known constants, and where $0 &amp;lt; n \leq 1$ is a real number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Special cases:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If $C_2 = 0$, this is a simple linear ODE and is straightforward to solve.&lt;/li&gt;&#xA;&lt;li&gt;If $C_1 = 0$, I can take the $\sqrt[n]{}$ of both sides and proceed to solve the nonlinear first order ODE.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&#xA;Is there then an appropriate method for solving the general case ($C_1 \neq 0, C_2 \neq 0$)?&lt;/p&gt;&#xA;" OwnerUserId="7941" LastEditorUserId="713" LastEditDate="2015-09-24T18:54:37.007" LastActivityDate="2015-09-27T07:30:52.910" Title="Solving nonlinear boundary value problem" Tags="|ode|nonlinear-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20814" PostTypeId="2" ParentId="20813" CreationDate="2015-09-24T12:56:43.360" Score="2" Body="&lt;p&gt;You simply have a nonlinear boundary value problem here. There is, in general, no technique where you can solve this in one shot, but there are many techniques where you can iteratively solve it via a sequence of solutions $u^{(k)}(y)$ that converge to $u(y)$ as $k\rightarrow \infty$. Examples of this are the fixed point method, pseudo-timestepping, or Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find an overview of the various techniques in many books. There are also lectures 31.5 and following from my video lecture page: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-09-24T12:56:43.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20815" PostTypeId="1" AcceptedAnswerId="20818" CreationDate="2015-09-24T16:53:49.793" Score="3" ViewCount="1265" Body="&lt;p&gt;I have a program that has a nested loop, together with its parent running at $O(n^2)$ complexity performing floating point arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that the performance of the code when compiled with &lt;code&gt;icc&lt;/code&gt; is comparable to that of &lt;code&gt;g++ -ffast-math&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean that icc is switching on an equivalent of the &lt;code&gt;-ffast-math&lt;/code&gt; implicitly?&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="713" LastEditDate="2015-09-24T18:40:01.917" LastActivityDate="2015-09-24T20:21:36.090" Title="performance of icc main.cpp == g++ -ffast-math main.cpp" Tags="|floating-point|computer-arithmetic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20816" PostTypeId="2" ParentId="20815" CreationDate="2015-09-24T17:06:57.887" Score="1" Body="&lt;p&gt;You can find out by asking icc and g++ to print out all the optimization switches that are actually enabled. Look at the verbosity options for each.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-09-24T17:06:57.887" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20817" PostTypeId="2" ParentId="20812" CreationDate="2015-09-24T18:19:13.190" Score="3" Body="&lt;p&gt;For such small matrices I simply call LAPACK ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;! Computes inverse of a matrix&#xA;subroutine MatInv(A,Ainv)&#xA;   implicit none&#xA;   real(8) :: A(:,:),Ainv(:,:)&#xA;   real(8),allocatable :: work(:)&#xA;   integer :: n,info&#xA;   integer,allocatable :: ipiv(:)&#xA;   Ainv=A&#xA;   n=size(Ainv,1)&#xA;   allocate(ipiv(n),work(n))&#xA;   call dgetrf(n,n,Ainv,n,ipiv,info)&#xA;   call dgetri(n,Ainv,n,ipiv,work,n,info)&#xA;   deallocate(ipiv,work)&#xA;end subroutine MatInv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Not worth it to roll your own stuff because of potential bugs and code bloat.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2015-09-24T18:19:13.190" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20818" PostTypeId="2" ParentId="20815" CreationDate="2015-09-24T18:39:03.550" Score="6" Body="&lt;p&gt;To first order approximation, icc does use &lt;code&gt;-ffast-math&lt;/code&gt; by default.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you run &lt;code&gt;icc --help&lt;/code&gt;, it will show you its options. One of the sections is &lt;code&gt;Floating Point&lt;/code&gt;, which begins like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Floating Point&#xA;--------------&#xA;&#xA;-fp-model &amp;lt;name&amp;gt;&#xA;          enable &amp;lt;name&amp;gt; floating point model variation&#xA;            [no-]except - enable/disable floating point semantics&#xA;            fast[=1|2]  - enables more aggressive floating point optimizations&#xA;            precise     - allows value-safe optimizations&#xA;            source      - enables intermediates in source precision&#xA;            strict      - enables -fp-model precise -fp-model except, disables&#xA;                          contractions and enables pragma stdc fenv_access&#xA;            double      - rounds intermediates in 53-bit (double) precision&#xA;            extended    - rounds intermediates in 64-bit (extended) precision&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It doesn't say it here, but in &lt;a href=&quot;https://software.intel.com/en-us/node/522979&quot; rel=&quot;noreferrer&quot;&gt;icc documentation here&lt;/a&gt;, it says that &lt;code&gt;fast=1&lt;/code&gt; is the default.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other options similar to &lt;code&gt;-ffast-math&lt;/code&gt; are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-[no-]prec-sqrt&#xA;          determine if certain square root optimizations are enabled&#xA;-[no-]prec-div&#xA;          improve precision of FP divides (some speed impact)&#xA;-[no-]fast-transcendentals&#xA;          generate a faster version of the transcendental functions&#xA;...&#xA;-[no-]ftz&#xA;          enable/disable flush denormal results to zero&#xA;-[no-]fma&#xA;          enable/disable the combining of floating point multiplies and&#xA;          add/subtract operations&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;E.g., &lt;a href=&quot;https://software.intel.com/en-us/node/522978&quot; rel=&quot;noreferrer&quot;&gt;fused multiply-adds are on by default&lt;/a&gt;, and &lt;a href=&quot;https://software.intel.com/en-us/node/522990&quot; rel=&quot;noreferrer&quot;&gt;precise square root is off by default&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the correspondence between &lt;code&gt;-ffast-math&lt;/code&gt; and &lt;code&gt;-fp-model fast=1&lt;/code&gt; is probably not very precise - it could well be that some other specific options are included in &lt;code&gt;fast-math&lt;/code&gt; (&lt;a href=&quot;https://gcc.gnu.org/wiki/FloatingPointMath&quot; rel=&quot;noreferrer&quot;&gt;which can be seen here&lt;/a&gt;) but excluded in &lt;code&gt;fp-model fast=1&lt;/code&gt; or vice versa. It looks to me like among other things &lt;code&gt;-ffast-math&lt;/code&gt; would disable non-finite floating-point arithmetic (&lt;code&gt;inf&lt;/code&gt;, &lt;code&gt;nan&lt;/code&gt;), which icc doesn't do by default.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-09-24T20:21:36.090" LastActivityDate="2015-09-24T20:21:36.090" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20819" PostTypeId="1" AcceptedAnswerId="20820" CreationDate="2015-09-25T01:44:17.150" Score="8" ViewCount="850" Body="&lt;p&gt;First of all, I hope I am posting this in the correct place. If not, I'm sorry and could you please direct me to where I should post this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt; You are given a set of vectors, $\{\mathbf{a}^i\}_{i=1}^{24}$, and matrices $D_1$ and $D_2$ (both of dimension $[10\times6]$. Let the following be true:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\sum_{i=1}^{24} \mathbf{a}^i = \mathbf{1}\;,$&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{a}^i \geq \mathbf{0}\;,\forall i\;,$ (i.e. all elements positive),&lt;/li&gt;&#xA;&lt;li&gt;$(D_1 + D_2)\mathbf{1} = \mathbf{1}\;.$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then, find vectors $\{\mathbf{b}^i\}_{i=1}^{24}$ and $\{\mathbf{c}^i\}_{i=1}^{24}$ that satisfy the equation, &#xA;$$&#xA;\mathbf{c}^i - D_1 \mathbf{b}^i = D_2 \mathbf{a}^i\;,\quad \forall i\in \{1,2,\dots,24\}\;,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the following constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\sum_{i=1}^{24} \mathbf{c}^i = \mathbf{1}\;,$&lt;/li&gt;&#xA;&lt;li&gt;$\sum_{i=1}^{24} \mathbf{b}^i = \mathbf{1}\;,$&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{c}^i \geq \mathbf{0}\;,\forall i\;,$ (i.e. all elements positive),&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{b}^i \geq \mathbf{0}\;,\forall i\;,$ (i.e. all elements positive).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Could anyone please give me any pointers on how I could go about tackling this problem? I would be interested in finding &lt;em&gt;a&lt;/em&gt; solution (if there are multiple), or even finding out that there are none. I think that this is a kind of optimization problem, but I have no idea about any methods I could use to solve this problem computationally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, this is NOT homework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="17666" LastActivityDate="2015-09-25T02:28:45.153" Title="Linear system solution with inequality constraints - methods?" Tags="|linear-algebra|optimization|constrained-optimization|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20820" PostTypeId="2" ParentId="20819" CreationDate="2015-09-25T02:28:45.153" Score="6" Body="&lt;p&gt;This is a linear programming feasibility problem (since you don't have an objective function to minimize or maximize.)  You can simply use an objective function of $0$ and hand this off to any reasonable LP solver.  You'll either get back a solution or the bad news that the problem is infeasible.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-09-25T02:28:45.153" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20822" PostTypeId="1" CreationDate="2015-09-25T10:05:51.953" Score="2" ViewCount="68" Body="&lt;p&gt;I am wondering what chaotic systems are from the perspective of numerical analysis. I am talking about 'deterministic chaos' such as for instance the 'logistic map' exhibits it. That is, the solution after many steps in the iterative solution is highly dependent on the initial conditions. It is clear that it will also be highly sensitive to numerical errors. But are chaotic systems &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;well posed (Hadamard),&lt;/li&gt;&#xA;&lt;li&gt;well conditioned,&lt;/li&gt;&#xA;&lt;li&gt;always unstable in implementations,&lt;/li&gt;&#xA;&lt;li&gt;susceptible to regularization?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1943" LastEditorUserId="1804" LastEditDate="2015-09-25T10:51:02.020" LastActivityDate="2015-09-25T10:51:02.020" Title="How to classify chaotic systems from a stability perspective" Tags="|stability|condition-number|well-posedness|chaotic-systems|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20823" PostTypeId="1" CreationDate="2015-09-25T12:33:58.960" Score="2" ViewCount="130" Body="&lt;p&gt;I'm trying to make a neural network using MATLAB. But I'm finding that many problems do not get solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the formulas on which I based:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Layers\ in\ Neural\ Network = [0, ..., i, ..., out]&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W_{i,j} = weights\ of\ the\ layer_i\ to\ layer_j\ \in M_{m\ x \ n}\\&#xA;B_{i,j} = weights\ of\ the\ layer_i\ to\ layer_j \in \mathbb{R}^n \\&#xA;d_{i} = delta\ correction\ of\ the\ layer_i\ \in \mathbb{R}^n \\&#xA;out_{j} = output\ signal\ of\ the\ layer_j\ \in \mathbb{R}^n  \\&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update Weights Output layer&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d_{output} = out_{output}·(1-out_{output})·(out_{output} - Y)\\&#xA;W_{j, output}^{+} = W_{j,output} - n·out_{j}^{T}·d_{output} \\&#xA;B_{j, output}^{+} = B_{j,output} + n·d_{output} \\&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update Weights Hidden layers&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d_{i} = out_{i}·(1-out_{i})·(d_{i+1}·W_{i+1, i+2}^T)\\&#xA;W_{i,j}^{+} = W_{i,j} - n·out_{i-1}^{T}·d_{i} \\&#xA;B_{i,j}^{+} = B_{i,j} + n·d_{i} \\&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update Weights First Hidden layer&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d_{1} = out_{1}·(1-out_{1})·(d_{2}·W_{2,3}^T)\\&#xA;W_{1,2}^{+} = W_{1,2} - n·X^T·d_{1} \\&#xA;B_{1,2}^{+} = B_{1,2} + n·d_{1} \\&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code I generated:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;classdef NeuralNetwork &amp;lt; handle&#xA;    properties&#xA;        % Wjk j inputs to k outputs (j inputs/neuron)&#xA;        LW;&#xA;        % Bk (k bias)&#xA;        LB;&#xA;        num_inputs;&#xA;        num_outputs;&#xA;        n_descensParameter;&#xA;    end&#xA;&#xA;    methods&#xA;        function obj = NeuralNetwork(num_inputs, num_outputs, hidden_layers, n_descensParameter)&#xA;            obj.num_inputs = num_inputs;&#xA;            obj.num_outputs = num_outputs;&#xA;            obj.n_descensParameter = n_descensParameter;&#xA;&#xA;            size_ant = obj.num_inputs;&#xA;            for i=1:length(hidden_layers)&#xA;                obj.LW{i} = zeros(size_ant, hidden_layers(i));&#xA;                obj.LB{i} = zeros(1, hidden_layers(i));&#xA;                size_ant = hidden_layers(i);&#xA;            end&#xA;            obj.LW{length(hidden_layers)+1} = zeros(hidden_layers(end), num_outputs);&#xA;            obj.LB{length(hidden_layers)+1} = zeros(1, num_outputs);&#xA;        end&#xA;&#xA;        function InitWeights(obj)&#xA;            b = sqrt(6)/sqrt(size(obj.LW{1}, 2)+obj.num_inputs);&#xA;            obj.LW{1} = -b + (2*b).*rand(size(obj.LW{1}));&#xA;            for i=2:length(obj.LW),&#xA;                obj.LW{i} = -b + (2*b).*rand(size(obj.LW{i}));&#xA;                b = sqrt(6)/sqrt(size(obj.LW{i}, 2)+size(obj.LW{i-1}, 2));&#xA;            end&#xA;        end&#xA;&#xA;        function out = predict(obj, X)&#xA;            inputs = X;&#xA;            for i=1:length(obj.LW)&#xA;                inputs = obj.activationNeuronFunc(inputs*obj.LW{i} + obj.LB{i});&#xA;            end&#xA;            out = obj.cOutput(inputs);&#xA;        end&#xA;        function d = activationNeuronFunc(obj, x)&#xA;            d = (1./(1+exp(-x)));&#xA;        end&#xA;        function d = cOutput(obj, o)&#xA;            d = round(o); % softmax&#xA;        end&#xA;        function d = doutput(obj, o)&#xA;            d = o.*(1-o);&#xA;        end&#xA;        function rval = cicle_backpropagation(obj, X, Y)&#xA;            o = {};&#xA;            inputs = X;&#xA;            for i=1:length(obj.LW)&#xA;                o{i} = obj.activationNeuronFunc(inputs*obj.LW{i} + obj.LB{i});&#xA;                inputs = o{i};&#xA;            end&#xA;&#xA;            LWplus = cell(size(obj.LW));&#xA;            d = cell(size(obj.LW));&#xA;&#xA;            d{end} = obj.doutput(o{end}).*(o{end}-Y);&#xA;            LWplus{end} = obj.LW{end} - obj.n_descensParameter.*(transpose(o{end-1})*d{end});&#xA;            obj.LB{end} = obj.LB{end} + obj.n_descensParameter.*d{end};&#xA;&#xA;            for i=length(obj.LW)-1:-1:2&#xA;                d{i} = obj.doutput(o{i}).*(d{i+1}*transpose(obj.LW{i+1}));&#xA;                LWplus{i} = obj.LW{i} - obj.n_descensParameter.*(transpose(o{i-1})*d{i});&#xA;                obj.LB{i} = obj.LB{i} + obj.n_descensParameter.*d{i};&#xA;            end&#xA;&#xA;            d{1} = obj.doutput(o{1}).*(d{2}*transpose(obj.LW{2}));&#xA;            LWplus{1} = obj.LW{1} - obj.n_descensParameter.*(transpose(X)*d{1});&#xA;            obj.LB{1} = obj.LB{1} + obj.n_descensParameter.*d{1};&#xA;&#xA;            obj.LW = LWplus;&#xA;&#xA;            rval = sum(abs(Y-o{end}));&#xA;        end&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is a small test which should create the XOR expression:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Learn&#xA;a = NeuralNetwork(2,2,[2], 1);&#xA;a.InitWeights()&#xA;X = [1, 1; 1, 0; 0, 1; 0, 0];&#xA;Y = [0, 1; 1, 0; 1, 0; 0, 1];&#xA;for i=1:1500,&#xA;    for j=1:4,&#xA;        a.cicle_backpropagation(c(j,:), b(j,:))&#xA;    end&#xA;end&#xA;% Validate&#xA;a.predict([1,1]) % Must be 0, 1&#xA;a.predict([1,0]) % Must be 1, 0&#xA;a.predict([0,1]) % Must be 1, 0&#xA;a.predict([0,0]) % Must be 0, 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is that it does not learn correctly, I feel that there is an error in the code or formulas that have deduced.&#xA;Could correct me? I'm pretty rookie and I want to learn to program it to understand fully.&lt;/p&gt;&#xA;" OwnerUserId="17669" LastEditorUserId="20688" LastEditDate="2019-05-25T09:54:54.523" LastActivityDate="2019-05-25T09:54:54.523" Title="Correct Weights Neural Network" Tags="|matlab|machine-learning|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20825" PostTypeId="1" AcceptedAnswerId="20845" CreationDate="2015-09-25T15:52:32.273" Score="2" ViewCount="95" Body="&lt;p&gt;I do have a stirred tank reactor with two inlets and one outlet. Several components enter the reactor at inlet 0 and particles at inlet 1. All component from inlet 0 adsorb on the particles from inlet 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can write a mass balance like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; C \cdot V_{liquid} + Q \cdot V_{particles} = 0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C is the concentration in the liquid, Q is the concentration on the particles, V are volumes (or flow rates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ODE may look like (k,l,m are parameters):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{dQ}{dt} = k \, (1 - Q[0])^l * m&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accordingly I can write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{dC}{dt} = - V_{particles}/V_{liquid} *\frac{dQ}{dt}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have two unknowns for every components: C[2] and Q[2] which are the concentrations in the reactor (or at the outlet of the reactor). The ODE model adds two equations for every component, therefore the system is not under determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I write all equations together, it is an implicit DAE model. However, if I say, that the components are mixed instantaneously at time = 0 and adsorption happens afterwards, I can calculate all initial conditions necessary for an ODE system. It seems to work but I am not sure whether this trick is a pitfall or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this assumption correct ? Or do I miss something ? &lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2015-09-27T03:13:51.720" Title="stirred tank model; DAE versus ODE model" Tags="|ode|numerical-modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20826" PostTypeId="1" AcceptedAnswerId="20829" CreationDate="2015-09-25T16:57:55.447" Score="1" ViewCount="314" Body="&lt;p&gt;This question has also been asked at &lt;a href=&quot;https://stackoverflow.com/questions/32785294/max-size-of-set-linear-equations-to-solve-x-axb&quot;&gt;Stack Overflow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a very general question regarding the maximum size of a set of linear equations to be solved by today's fastest hardware, in the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X = AX + B$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where we solve for $X$, and&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$A$: $N\times N$ a sparse matrix of floats; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$B$: $N$-vector of floats.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This becomes $X(I-A) = B$ which is best solved using factorization (and not matrix inversion) as I read &lt;a href=&quot;http://www.johndcook.com/blog/2010/01/19/dont-invert-that-matrix/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know or have a reference to a benchmark or paper which gives some maximum value for $N$ with today's fastest hardware? Assume that I can have that hardware, this is a sort of mental what-if exercise. Most benchmarks I have seen use $N &amp;lt; 10000$. I am thinking about $N&amp;gt;10^7$ or more to be processed within a month.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please consider not only the computational dimension but also storage for $A$. It can be a problem, e.g., assuming $N = 10^6$, storage would be $4\times 10^{12}$ bytes $\approx$ 4 Terabytes for  a totally dense matrix, which is about manageable I guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, can the method to solve the system be parallelized so that I can make the assumption that with parallelization $N$ can become pretty large?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;later addition to question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible application of this is:&#xA;consider N products from factories (e.g. screws, an electric motor, an engine, a car).&#xA;Most products require parts from other products and are used as parts for other more complex products in addition to some quantities ending in the shops - e.g. an electric motor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the $i^{th}$ product we have:&#xA;$X_i = a_{i,1} X_1 + a_{i,2} X_2 + \cdots + a_{i,N} X_N + B_i$&#xA;meaning that the quantity of $X_i$ we aim to produce requires $a_{i,1} X_1$ items from product $X_1$, to $N$, plus $B_i$ which is what ends up in shops for general consumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our aim is to find the solution to this problem given final demands ($B$) and &quot;inter-dependancy&quot; quantities $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess $A$ is sparse but this probably depends on applications, e.g. Toyota states that a basic car consists of 30,000 other products (all broken down to screws). This is why I mention sparse but I also go for the worst case of &quot;mild-denseness&quot;.&lt;/p&gt;&#xA;" OwnerUserId="17672" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-09-26T11:52:14.980" Title="Max size of set linear equations to solve? (X=AX+B)" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20827" PostTypeId="2" ParentId="3450" CreationDate="2015-09-25T17:38:22.310" Score="7" Body="&lt;p&gt;There are several &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_optimization&quot; rel=&quot;nofollow&quot;&gt;Bayesian optimization&lt;/a&gt; techniques you could try. Easiest are based on Gaussian process:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Harold J. Kushner. A new method of locating the maximum of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, pages 86:97–106, March 1964.&lt;/li&gt;&#xA;&lt;li&gt;J. Mockus. The Bayesian approach to global optimization. Lecture Notes in Control and Information Sciences, 38:473–481, 1982.&lt;/li&gt;&#xA;&lt;li&gt;Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proc. International Conference on Machine Learning (ICML), 2010.&lt;/li&gt;&#xA;&lt;li&gt;Andreas Krause, Ajit Singh, and Carlos Guestrin. Near-Optimal sensor placements in Gaussian processes: Theory, efficient algorithms and empirical studies. J. Mach. Learn. Res., 9:235–284, June 2008.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;They operate by forming a posterior over plausible functions give observations so far, and suggesting the next point to quickly learn the function as well as find the global maxima (see &lt;a href=&quot;https://memming.wordpress.com/2012/02/19/active-bayesian-optimization/&quot; rel=&quot;nofollow&quot;&gt;my blog post&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another advantage is that you can estimate the Hessian at the maxima. However, you need to specify a noise model.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastEditorUserId="1128" LastEditDate="2015-09-25T20:05:24.153" LastActivityDate="2015-09-25T20:05:24.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20828" PostTypeId="2" ParentId="20805" CreationDate="2015-09-25T20:03:04.923" Score="2" Body="&lt;p&gt;Y.Z. found a simpler solution without an explicit pseudo-inverse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the matrix inversion lemma twice, he got:&#xA;$$(A^{-1} + D)^{-1}v = GG^\top v - GG^\top D GG^\top v + GG^\top D G (I + G^\top D G)^{-1} G^\top D GG^\top v$$&#xA;which can be evaluated without forming the large $N \times N$ matrix.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2015-09-25T20:03:04.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20829" PostTypeId="2" ParentId="20826" CreationDate="2015-09-25T21:38:07.193" Score="3" Body="&lt;p&gt;For your applications, dense matrix techniques such as the different factorization techniques will not work. Plus your matrix is sparse anyway so you should be using an iterative solver (e.g. Krylov subspace methods). These easily handle problems of the scale you are interested in and there are several sparse matrix solver libraries implemented for parallel architectures (e.g. Petsc, Trilinos, &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot; rel=&quot;nofollow&quot;&gt;others&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several advantages to using sparse matrix techniques:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Storage. Who wants to store a dense matrix? Do you have terabytes of RAM waiting to be used? Most people don't, and you should not want to have to use disk space because that will cripple any sort of algorithm performance. You've already stated that your matrix A is sparse, which generally means that total storage space should only be $O(N)$. So it should only take megabytes to store your matrix and required vectors, not terabytes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Time. Dense matrix factorizations take $O(N^3)$ operations, even for sparse matrices. On a serial processor, a system of equations with $N&amp;gt;10^7$ can easily take on the order of years to compute a factorization. Even when parallelized you still have to deal with storage and communication costs. Sparse methods rely on matrix-vector multiplication, which is an $O(N^2)$ operation for dense matrices but a $O(N)$ operation for sparse matrices. And usually it requires less than $O(N)$ iterations to converge. This will take much less time in general (and can take even less time when a good preconditioner is used).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Parallelization. Because iterative methods rely on matrix-vector multiplication and vector-vector addition, they are highly parallelizable (especially on shared memory architectures, but also distributed memory too). Many libraries already have parallel support (again see &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;) for various architectures.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3834" LastActivityDate="2015-09-25T21:38:07.193" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20830" PostTypeId="2" ParentId="3206" CreationDate="2015-09-25T22:25:14.203" Score="3" Body="&lt;p&gt;I'm not sure if these were added recently but it seems like there are now easy ways to generate random numbers quickly without too much overhead. From this &lt;a href=&quot;https://hplgit.github.io/teamods/MC_cython/main_MC_cython.html&quot; rel=&quot;nofollow&quot;&gt;article&lt;/a&gt; about Monte Carlo simulations in cython we can do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from libc.stdlib cimport rand, RAND_MAX&#xA;r = 1 + int(rand()/(RAND_MAX*6.0)) # random integer 1,...,6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as I understand you don't need to do anything special when compiling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reproducibility during testing you can set a seed&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# srand48(time(0)) # Do it this way in production&#xA;srand48(100) # For reproducibility in testing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17675" LastEditorUserId="17675" LastEditDate="2016-04-13T22:17:53.900" LastActivityDate="2016-04-13T22:17:53.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20832" PostTypeId="1" CreationDate="2015-09-26T01:24:00.463" Score="2" ViewCount="98" Body="&lt;p&gt;This might be better somewhere else, but I'll give it a try here first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm implementing a finite volume scheme for an axisymmetric problem in C, and am looking for a more efficient way to handle all of the necessary parameters each function needs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is a pointer to a struct better, more efficient, or easier in implementation than a list of the function parameters? I personally think the struct is easier to handle when writing the code, but my professor, who loves Fortran, probably doesn't agree. Is the use of structs in scientific computing accepted? Tolerated? Something that should be avoided?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your thoughts.&lt;/p&gt;&#xA;" OwnerUserId="17678" LastActivityDate="2015-10-26T02:44:23.567" Title="Use of structs in Axisymmetric Finite Volume method" Tags="|finite-volume|c|programming-paradigms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20833" PostTypeId="2" ParentId="20832" CreationDate="2015-09-26T02:23:22.210" Score="1" Body="&lt;p&gt;All sorts of programming style are accepted, tolerated, and definitely not avoided. You should look at some open source scientific computing packages out there in the world to see the diverse styles that they are written in. They run the gamut from million-line, common-block heavy Fortran to much more object-oriented C++ or very idiomatic Python. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-09-26T02:23:22.210" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20834" PostTypeId="1" CreationDate="2015-09-26T13:32:38.353" Score="7" ViewCount="3545" Body="&lt;p&gt;I am trying to write a code to calculate the potential and forces, for the same using ewald summation.For this purpose, the formula for potential and force I have used is : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;U = U^{(r)} + U^{(k)} + U^{(bc)} + U^{self}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the k-space contribution of potential is given by &#xA;$$&#xA;U^{(k)} = \frac{1}{2\pi L^{3}}\sum_{\textbf{k}\ne0} \frac{4\pi^2}{k^2}\text {e}^{-\frac{k^2}{4\kappa^2}}|S(\textbf k)|^2 \qquad S(\textbf k) = \sum_{i=1}^N z_i\text{e}^{i\textbf{k}.\text{r}_i}&#xA;$$&#xA;$$&#xA;U^{(r)} = \sum_{j&amp;lt;i} \sum_{\textbf n=0}^{\infty}z_iz_j\frac{\text{erfc}(\kappa|\textbf{r}_{ij}+\textbf n|)}{|\textbf{r}_{ij}+\textbf n|}&#xA;$$&#xA;$$&#xA;U^{(bc)} = \frac{2\pi}{3L^3}\bigg|\sum_{i=1}^Nz_i\textbf r_i \bigg|^2 \qquad U^{(self)} = \frac{\kappa}{\sqrt{\pi}}\sum_{i=1}^Nz_i^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the force equations - &#xA;$$&#xA;\textbf F_i = \textbf F_i^{(k)} +\textbf F_i^{(r)}+\textbf F_i^{(bc)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\textbf F_i^{(k)} = \frac{4\pi z_i}{2\pi L^3}\sum_{\textbf k\ne 0}\frac{\textbf k \text e^{\frac{-k^2}{4\kappa^2}}}{k^2}\bigg(\sin(\textbf k_i. \textbf r_i) \text {Re}(S(\textbf k)) + \cos(\textbf k_i. \textbf r_i) \text {Im}(S(\textbf k))\bigg)&#xA;$$&#xA;and two more equations, which am tired of writing but kind of sure that they are correct ! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently using this method to solve for water molecules molecular dynamics simulation. The problem I am facing is the &lt;strong&gt;calculation of potential is highly dependent on $\kappa$&lt;/strong&gt;, which it can be, but there is vast change in potential if I change $\kappa$ slightly. Secondly &lt;strong&gt;shouldn't the change with respect diminish beyond certain value&lt;/strong&gt;, I don't see that also happening.&#xA;Here is a segment of code used in k-space summation, I have my doubts on this part only :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!calculates the structure factor for each k-vector&#xA;    do i=0,kreq&#xA;        call struct_fact(kx(i),ky(i),kz(i),rx,ry,rz,q,nm,ns,sk_r(i),sk_i(i),cs,sn)&#xA;    end do&#xA;&#xA;    !summing over all k-vectors&#xA;    k_sum = 0.d0&#xA;    const = 1/(4.d0*kappa*kappa)&#xA;    do i=0,kreq&#xA;        k_sum = k_sum + exp(-ksq(i)*const)*( sk_r(i)**2 + sk_i(i)**2 )/ksq(i)&#xA;    end do&#xA;&#xA;    !calculation of force-vector for all sites in k-space&#xA;    ew_kfsx = 0.d0&#xA;    ew_kfsy = 0.d0&#xA;    ew_kfsz = 0.d0&#xA;    do k=0,kreq&#xA;        do i=1,nm&#xA;        do j=1,ns&#xA;            temp = sn(i,j)*sk_r(k) + cs(i,j)*sk_i(k)&#xA;            !print *, ( q(i,j)*temp*kx(i)*exp(-ksq(i)*const) )/ksq(i)&#xA;            ew_kfsx(i,j) = ( q(i,j)*temp*kx(k)*exp(-ksq(k)*const) )/ksq(k)&#xA;            ew_kfsy(i,j) = ( q(i,j)*temp*ky(k)*exp(-ksq(k)*const) )/ksq(k)&#xA;            ew_kfsz(i,j) = ( q(i,j)*temp*kz(k)*exp(-ksq(k)*const) )/ksq(k)&#xA;        end do&#xA;        end do&#xA;    end do&#xA;&#xA;    k_sum = (twopi/boxL**3)*k_sum&#xA;    ew_kfsx = (twopi/boxL**3) * ew_kfsx&#xA;    ew_kfsy = (twopi/boxL**3) * ew_kfsy&#xA;    ew_kfsz = (twopi/boxL**3) * ew_kfsz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I here wish to know, if there is a precise way to find out if my code based on the above formulas are working correctly ?&lt;/p&gt;&#xA;" OwnerUserId="17599" LastActivityDate="2022-02-03T15:56:09.190" Title="Fortran code for Ewald summation" Tags="|fortran|molecular-dynamics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20835" PostTypeId="2" ParentId="20812" CreationDate="2015-09-26T16:30:21.513" Score="5" Body="&lt;p&gt;Here are the routines we use in QMC=Chem for small matrices. QMC=Chem is under GPL license, so if your code is also under GPL, you can use them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: To get the inverse, you need to multiply the result by &lt;code&gt;1.d0/det_l&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine invert2(a,LDA,na,det_l)&#xA; implicit none&#xA; double precision :: a (LDA,na)&#xA; integer          :: LDA&#xA; integer          :: na&#xA; double precision :: det_l&#xA; double precision :: b(2,2)&#xA;&#xA; double precision :: f&#xA; b(1,1) = a(1,1)&#xA; b(2,1) = a(2,1)&#xA; b(1,2) = a(1,2)&#xA; b(2,2) = a(2,2)&#xA; det_l = a(1,1)*a(2,2) - a(1,2)*a(2,1)&#xA; a(1,1) = b(2,2)&#xA; a(2,1) = -b(2,1)&#xA; a(1,2) = -b(1,2)&#xA; a(2,2) = b(1,1)&#xA;end&#xA;&#xA;&#xA;subroutine invert3(a,LDA,na,det_l)&#xA; implicit none&#xA; double precision, intent(inout) :: a (LDA,na)&#xA; integer, intent(in)             :: LDA&#xA; integer, intent(in)             :: na&#xA; double precision, intent(inout) :: det_l&#xA; double precision :: b(4,3)&#xA; integer :: i&#xA; double precision :: f&#xA; det_l = a(1,1)*(a(2,2)*a(3,3)-a(2,3)*a(3,2)) &amp;amp;&#xA;        -a(1,2)*(a(2,1)*a(3,3)-a(2,3)*a(3,1)) &amp;amp;&#xA;        +a(1,3)*(a(2,1)*a(3,2)-a(2,2)*a(3,1))&#xA; do i=1,4&#xA;  b(i,1) = a(i,1)&#xA;  b(i,2) = a(i,2)&#xA;  b(i,3) = a(i,3)&#xA; enddo&#xA; a(1,1) =  b(2,2)*b(3,3) - b(2,3)*b(3,2)&#xA; a(2,1) =  b(2,3)*b(3,1) - b(2,1)*b(3,3)&#xA; a(3,1) =  b(2,1)*b(3,2) - b(2,2)*b(3,1)&#xA;&#xA; a(1,2) =  b(1,3)*b(3,2) - b(1,2)*b(3,3)&#xA; a(2,2) =  b(1,1)*b(3,3) - b(1,3)*b(3,1)&#xA; a(3,2) =  b(1,2)*b(3,1) - b(1,1)*b(3,2)&#xA;&#xA; a(1,3) =  b(1,2)*b(2,3) - b(1,3)*b(2,2)&#xA; a(2,3) =  b(1,3)*b(2,1) - b(1,1)*b(2,3)&#xA; a(3,3) =  b(1,1)*b(2,2) - b(1,2)*b(2,1)&#xA;&#xA;end&#xA;&#xA;&#xA;subroutine invert4(a,LDA,na,det_l)&#xA; implicit none&#xA; double precision, intent(inout) :: a (LDA,na)&#xA; integer, intent(in)             :: LDA&#xA; integer, intent(in)             :: na&#xA; double precision, intent(inout) :: det_l&#xA; double precision :: b(4,4)&#xA; integer :: i,j&#xA; double precision :: f&#xA; det_l =  a(1,1)*(a(2,2)*(a(3,3)*a(4,4)-a(3,4)*a(4,3))  &amp;amp;&#xA;                 -a(2,3)*(a(3,2)*a(4,4)-a(3,4)*a(4,2))  &amp;amp;&#xA;                 +a(2,4)*(a(3,2)*a(4,3)-a(3,3)*a(4,2))) &amp;amp;&#xA;         -a(1,2)*(a(2,1)*(a(3,3)*a(4,4)-a(3,4)*a(4,3))  &amp;amp;&#xA;                 -a(2,3)*(a(3,1)*a(4,4)-a(3,4)*a(4,1))  &amp;amp;&#xA;                 +a(2,4)*(a(3,1)*a(4,3)-a(3,3)*a(4,1))) &amp;amp;&#xA;         +a(1,3)*(a(2,1)*(a(3,2)*a(4,4)-a(3,4)*a(4,2))  &amp;amp;&#xA;                 -a(2,2)*(a(3,1)*a(4,4)-a(3,4)*a(4,1))  &amp;amp;&#xA;                 +a(2,4)*(a(3,1)*a(4,2)-a(3,2)*a(4,1))) &amp;amp;&#xA;         -a(1,4)*(a(2,1)*(a(3,2)*a(4,3)-a(3,3)*a(4,2))  &amp;amp;&#xA;                 -a(2,2)*(a(3,1)*a(4,3)-a(3,3)*a(4,1))  &amp;amp;&#xA;                 +a(2,3)*(a(3,1)*a(4,2)-a(3,2)*a(4,1)))&#xA; do i=1,4&#xA;  b(1,i) = a(1,i)&#xA;  b(2,i) = a(2,i)&#xA;  b(3,i) = a(3,i)&#xA;  b(4,i) = a(4,i)&#xA; enddo&#xA;&#xA;&#xA; a(1,1) =  b(2,2)*(b(3,3)*b(4,4)-b(3,4)*b(4,3))-b(2,3)*(b(3,2)*b(4,4)-b(3,4)*b(4,2))+b(2,4)*(b(3,2)*b(4,3)-b(3,3)*b(4,2))&#xA; a(2,1) = -b(2,1)*(b(3,3)*b(4,4)-b(3,4)*b(4,3))+b(2,3)*(b(3,1)*b(4,4)-b(3,4)*b(4,1))-b(2,4)*(b(3,1)*b(4,3)-b(3,3)*b(4,1))&#xA; a(3,1) =  b(2,1)*(b(3,2)*b(4,4)-b(3,4)*b(4,2))-b(2,2)*(b(3,1)*b(4,4)-b(3,4)*b(4,1))+b(2,4)*(b(3,1)*b(4,2)-b(3,2)*b(4,1))&#xA; a(4,1) = -b(2,1)*(b(3,2)*b(4,3)-b(3,3)*b(4,2))+b(2,2)*(b(3,1)*b(4,3)-b(3,3)*b(4,1))-b(2,3)*(b(3,1)*b(4,2)-b(3,2)*b(4,1))&#xA;&#xA; a(1,2) = -b(1,2)*(b(3,3)*b(4,4)-b(3,4)*b(4,3))+b(1,3)*(b(3,2)*b(4,4)-b(3,4)*b(4,2))-b(1,4)*(b(3,2)*b(4,3)-b(3,3)*b(4,2))&#xA; a(2,2) =  b(1,1)*(b(3,3)*b(4,4)-b(3,4)*b(4,3))-b(1,3)*(b(3,1)*b(4,4)-b(3,4)*b(4,1))+b(1,4)*(b(3,1)*b(4,3)-b(3,3)*b(4,1))&#xA; a(3,2) = -b(1,1)*(b(3,2)*b(4,4)-b(3,4)*b(4,2))+b(1,2)*(b(3,1)*b(4,4)-b(3,4)*b(4,1))-b(1,4)*(b(3,1)*b(4,2)-b(3,2)*b(4,1))&#xA; a(4,2) =  b(1,1)*(b(3,2)*b(4,3)-b(3,3)*b(4,2))-b(1,2)*(b(3,1)*b(4,3)-b(3,3)*b(4,1))+b(1,3)*(b(3,1)*b(4,2)-b(3,2)*b(4,1))&#xA;&#xA; a(1,3) =  b(1,2)*(b(2,3)*b(4,4)-b(2,4)*b(4,3))-b(1,3)*(b(2,2)*b(4,4)-b(2,4)*b(4,2))+b(1,4)*(b(2,2)*b(4,3)-b(2,3)*b(4,2))&#xA; a(2,3) = -b(1,1)*(b(2,3)*b(4,4)-b(2,4)*b(4,3))+b(1,3)*(b(2,1)*b(4,4)-b(2,4)*b(4,1))-b(1,4)*(b(2,1)*b(4,3)-b(2,3)*b(4,1))&#xA; a(3,3) =  b(1,1)*(b(2,2)*b(4,4)-b(2,4)*b(4,2))-b(1,2)*(b(2,1)*b(4,4)-b(2,4)*b(4,1))+b(1,4)*(b(2,1)*b(4,2)-b(2,2)*b(4,1))&#xA; a(4,3) = -b(1,1)*(b(2,2)*b(4,3)-b(2,3)*b(4,2))+b(1,2)*(b(2,1)*b(4,3)-b(2,3)*b(4,1))-b(1,3)*(b(2,1)*b(4,2)-b(2,2)*b(4,1))&#xA;&#xA; a(1,4) = -b(1,2)*(b(2,3)*b(3,4)-b(2,4)*b(3,3))+b(1,3)*(b(2,2)*b(3,4)-b(2,4)*b(3,2))-b(1,4)*(b(2,2)*b(3,3)-b(2,3)*b(3,2))&#xA; a(2,4) =  b(1,1)*(b(2,3)*b(3,4)-b(2,4)*b(3,3))-b(1,3)*(b(2,1)*b(3,4)-b(2,4)*b(3,1))+b(1,4)*(b(2,1)*b(3,3)-b(2,3)*b(3,1))&#xA; a(3,4) = -b(1,1)*(b(2,2)*b(3,4)-b(2,4)*b(3,2))+b(1,2)*(b(2,1)*b(3,4)-b(2,4)*b(3,1))-b(1,4)*(b(2,1)*b(3,2)-b(2,2)*b(3,1))&#xA; a(4,4) =  b(1,1)*(b(2,2)*b(3,3)-b(2,3)*b(3,2))-b(1,2)*(b(2,1)*b(3,3)-b(2,3)*b(3,1))+b(1,3)*(b(2,1)*b(3,2)-b(2,2)*b(3,1))&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17681" LastEditorUserId="17681" LastEditDate="2018-04-08T08:51:31.030" LastActivityDate="2018-04-08T08:51:31.030" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20836" PostTypeId="1" AcceptedAnswerId="20837" CreationDate="2015-09-26T17:31:34.673" Score="6" ViewCount="292" Body="&lt;p&gt;I am trying to think of what this kind of problem is called.  I illustrate it with a telegrapher's equation with (hopefully) standard notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find $u:\Omega\times \mathbb{R} \to \mathbb{R}$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*} &#xA;u_{tt}+u_t-\nabla^2 u &amp;amp;= 0&amp;amp; \mathrm{for}\, (x,t)\in \Omega\times \mathbb{R}\\&#xA;u&amp;amp;= u_{bc} \sin(\omega t)&amp;amp; \mathrm{for}\ (x,t)\in \partial\Omega \times \mathbb{R}&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main point is that I am interested in solving the PDE for all time, or perhaps more precisely I am interested in the limit cycle behavior of the PDE, (after any transients associated to the initial condition have died down).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, I guess a valid problem would be the following.  Fine $u: \Omega \times \mathbb{R}\to \mathbb{R}$ such that &#xA;$$&#xA;\begin{align*}&#xA;u_{tt} +u_t - \nabla^2 u &amp;amp;= 0 &amp;amp;\mathrm{for} (x,t)\in \Omega \times \mathbb{R}\\&#xA;u &amp;amp;= u_{bc} \sin(\omega t)&amp;amp; \mathrm{for} (x,t)\in \partial\Omega \times \mathbb{R}\\&#xA;u(x,t+2\pi/\omega) &amp;amp;= u(x,t)&amp;amp; \mathrm{for} (x,t)\in \Omega \times \mathbb{R}&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this problem have a name?   &lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2015-09-26T20:13:09.523" LastActivityDate="2015-09-26T21:50:55.357" Title="Do the class of PDEs that lack initial conditions have a name?" Tags="|pde|boundary-conditions|hyperbolic-pde|parabolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20837" PostTypeId="2" ParentId="20836" CreationDate="2015-09-26T17:53:06.670" Score="8" Body="&lt;p&gt;Such problems (sometimes called &lt;em&gt;lateral Cauchy problems&lt;/em&gt;) are in general not well-posed (meaning they either lack a solution, or there are infinitely many of them, or the solution is unstable under perturbations of the boundary conditions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For parabolic (or dissipative) equations, it makes sense to study the stationary limit (simply omit the term $u_t$ in the PDE together with the initial condition), but I don't think that's the case for wave-like hyperbolic problems, where information is transported without loss (meaning the influence of the initial conditions &lt;em&gt;never&lt;/em&gt; disappears, unless you have absorbing boundary conditions, in which case the limit solution is the trivial solution $u\equiv 0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your second problem doesn't really make sense mathematically, because the solution $u$ is either completely determined by the periodicity and boundary conditions (if they are compatible), or no solution exists (if they are not).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-09-26T21:50:55.357" LastActivityDate="2015-09-26T21:50:55.357" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20838" PostTypeId="1" CreationDate="2015-09-26T23:58:02.390" Score="2" ViewCount="1026" Body="&lt;p&gt;does anyone know any matlab algorithm that can help me generate a random matrix with REAL EIGEN values?&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="17553" LastEditorUserId="17755" LastEditDate="2015-10-04T19:13:27.927" LastActivityDate="2015-10-04T19:13:27.927" Title="Generation of random Matrix with Real eigen values" Tags="|linear-algebra|matlab|algorithms|matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20839" PostTypeId="2" ParentId="20838" CreationDate="2015-09-27T01:21:02.363" Score="6" Body="&lt;p&gt;You don't mention what kind of distribution you want for the entries of the random matrix, so the simplest recipe that I can think of is to first create a random $n \times n$ matrix $\mathtt{A}$ and to then obtain $\mathtt{B} = \frac{1}{2} ( \mathtt{A} + \mathtt{A}^{\mathtt{T}} )$.  That gives you a symmetric matrix $\mathtt{B}$ with random entries.  Since symmetric matrices have real eigenvalues, the matrix $\mathtt{B}$ will satisfy your requirement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = rand(n)&#xA;B = 0.5 * (A + transpose(A))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's another approach, building on &lt;a href=&quot;https://scicomp.stackexchange.com/a/20840/15&quot;&gt;Bill Barth's answer&lt;/a&gt;. First, create a random diagonal matrix $\mathtt{D}$, then obtain a random orthogonal matrix $\mathtt{Q}$ and, finally, compute the similarity transformation $\mathtt{B} = \mathtt{Q} \, \mathtt{D} \, \mathtt{Q}^{\mathtt{T}}$.  The matrix $\mathtt{B}$ will have the same eigenvalues as $\mathtt{D}$ but will not be diagonal in general.  A simple way to come up with a random orthogonal matrix $\mathtt{Q}$ is to compute the QR decomposition of a random matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;D = diag(rand(n, 1))&#xA;[Q ~] = qr(rand(n))&#xA;B = Q * D * transpose(Q)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-09-27T01:45:01.793" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20840" PostTypeId="2" ParentId="20838" CreationDate="2015-09-27T01:31:50.330" Score="1" Body="&lt;p&gt;If you want to specify the eigenvalues, then you can just put them on the diagonal of an otherwise empty matrix.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-09-27T01:31:50.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20841" PostTypeId="1" CreationDate="2015-09-27T01:38:05.633" Score="4" ViewCount="3377" Body="&lt;p&gt;Suppose that there are $N$ pieces of data, each of which contains $M$ parameter values such that $N &amp;gt;&amp;gt; M$. If we put this information into matrix form ($N$ rows, $M$ columns) and then compute the covariance matrix of this data, $C$, then we can consider that $C_{i,j}$ is the covariance between parameter $i$, $p_i$, and parameter $j$, $p_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have always thought of this as &quot;The amount that $p_i$ would change if $p_j$ changed by 1&quot;. While this may not be &lt;em&gt;exactly&lt;/em&gt; correct (is it correct?), there is at least a fundamental relationship between $p_i$ and $p_j$ which is reflected in $C_{i,j}$ and can consider that if $C_{i,j} = 0$, then there is no fundamental relationship and the change in one will not necessarily reflect a change in the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, consider $\frac{\partial{p_i}}{\partial{p_j}}$, this is the amount of change in $p_i$ due to a small change in $p_j$ and this partial derivative will once again be zero if there is no relationship between $p_i$ and $p_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to consider, in any way, $C_{i,j} \approx \frac{\partial{p_i}}{\partial{p_j}}$? Are there conditions under which this may be the case or any other relationship between these two measures?&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2015-10-27T20:18:45.700" Title="Is there a relationship between the covariance matrix and the partial derivative?" Tags="|statistics|data-analysis|calculus|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20842" PostTypeId="1" AcceptedAnswerId="20876" CreationDate="2015-09-27T02:02:16.413" Score="3" ViewCount="1920" Body="&lt;p&gt;I've been working on calculating large factorials ($N&amp;gt;10^9$) and I was wondering if it wasn't faster to use the GPU to run the calculations on something like openCL. What I realized however was that, not only I didn't know if it was worth it in that case, I couldn't tell when it is or isn't appropriate to use a GPU for number crunching. So, basically, what are the pros and cons of using GPU and CPU for such large numbers ($10^9!$) or when is it worth it at all to do GPU implementation?&lt;/p&gt;&#xA;" OwnerUserId="17686" LastEditorUserId="713" LastEditDate="2015-09-29T22:55:50.393" LastActivityDate="2015-09-29T22:55:50.393" Title="GPU vs CPU calculation" Tags="|parallel-computing|gpu|complexity|" AnswerCount="2" CommentCount="18" ContentLicense="CC BY-SA 3.0" />
  <row Id="20843" PostTypeId="2" ParentId="20834" CreationDate="2015-09-27T02:16:31.540" Score="6" Body="&lt;p&gt;Notice that the summation in $U^{(r)}$ is incorrect.  You want to sum over all the copies of the atoms in the lattice of periodic boxes, not just those whose indices satisfy $i &amp;gt; j$.  In the original box, of course you want to avoid self interactions $i = j$ but just in that one box.  In other words, &lt;em&gt;do not&lt;/em&gt; discard the electrostatic interactions between atom $i$ in the original box with atom $i$ in the neighboring periodic boxes.    If you are using the minimum image convention and the value of $\kappa$ is right (I have not checked it, though, you can do so yourself simply by plotting ${\rm erfc}(\kappa \, r)$ for $r \ge 0$ and seeing if it decays fast enough), then you are already summing correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The summation in the $U^{(k)}$ term goes over all the triplets of integers  (positive and negative) except the one that is zero in all components (i.e., $\mathbf{k} \in \mathbb{Z}^3 \setminus \{ (0, 0, 0) \}$). It is not clear from your Fortran snippet that this is being taken into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a test that is computationally expensive but will make sure your Ewald summation code is working properly.  Just compute the electrostatic energy,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E=\frac{1}{2}\sum_{i=1}^{N}\sum_{\stackrel{j=1}{i\neq j}}^{N} \frac{z_i z_j}{|\mathbf{r}_{ij} |}+\frac{1}{2}\sum_{\stackrel{\mathbf{n}\in\mathbb{Z}^{3%&#xA;}}{\mathbf{n}\neq{0}}}\sum_{i=1}^{N}\sum_{j=1}^{N} \frac{z_i z_j}{|\mathbf{r}_{ij}+\mathbf{n}|},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;over many cubic shells (as many as you need for $E$ to converge up to a desired accuracy), as &lt;a href=&quot;https://scicomp.stackexchange.com/a/19784/15&quot;&gt;specified here&lt;/a&gt;.  The result of your Ewald summation must converge to $E$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing $\kappa$ determines how much the work is distributed between the direct, $U^{(r)}$, and the Fourier space, $U^{(k)}$, parts of the computation.  In principle, you should get the same value for the potential energy even if the values of $\kappa$ change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a tutorial on this topic with an accompanying implementation of the Ewald summation and Particle-Mesh Ewald methods for dispersive (not Coulomb) potentials that you may find helpful.  You can &lt;a href=&quot;http://superadditive.com/notes/particle-mesh-ewald/&quot; rel=&quot;nofollow noreferrer&quot;&gt;find it here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-09-27T16:23:36.553" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20844" PostTypeId="2" ParentId="20842" CreationDate="2015-09-27T02:22:53.960" Score="3" Body="&lt;p&gt;It is a bit old, but the &lt;a href=&quot;http://www.hpcs.cs.tsukuba.ac.jp/~nakayama/cump/&quot; rel=&quot;nofollow&quot; title=&quot;CUDA Multiprecision Arithmetic Library&quot;&gt;CUDA Multiprecision Arithmetic Library&lt;/a&gt; probably supports the operations you need, and reports &lt;a href=&quot;http://www.hpcs.cs.tsukuba.ac.jp/~nakayama/cump/index.php?CUMP%20Performance%20Evaluation&quot; rel=&quot;nofollow&quot;&gt;2-4x speed-ups&lt;/a&gt; vs a CPU socket.  It claims to have a GMP-like interface, so porting you code could be straight-forward, relative to writing custom kernel code.&lt;/p&gt;&#xA;" OwnerUserId="521" LastActivityDate="2015-09-27T02:22:53.960" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20845" PostTypeId="2" ParentId="20825" CreationDate="2015-09-27T03:13:51.720" Score="2" Body="&lt;p&gt;As you say, if the mass balance equation is satisfied at time $t = 0$, then it will be satisfied for all $t &amp;gt; 0$ as well, which implies that you do not need to regard your system as a differential-algebraic system because the algebraic equation is redundant.  To see this, let $$M(t) = C(t) \, V_{\rm liquid} + Q(t) \, V_{\rm particles}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking derivatives with respect to time, we get $$\frac{d M}{d t} = \frac{d C}{d t} \, V_{\rm liquid} + \frac{d Q}{d t} \, V_{\rm particles}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, using&#xA; $$\frac{dC}{dt} = - \frac{V_{\rm particles}}{V_{\rm liquid}} \frac{d Q}{d t},$$ we see that $\frac{d M}{d t} = 0$, which in turn implies that $M(t)$ is constant for all $t$.  If $M(0) = 0$, you can conclude that the mass balance equation is satisfied for all times.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-09-27T03:13:51.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20846" PostTypeId="2" ParentId="20823" CreationDate="2015-09-27T07:10:47.550" Score="2" Body="&lt;p&gt;One thing you should keep in mind, for future reference, is when you go to find the weights for an Artificial Neural Network using the Backpropogation algorithm with Gradient Descent (as you are), you're using a numerical method that will get trapped in a local minima if you're initial guess for the weights aren't good enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you are finding trouble with converging to the right solution, that should be one more thing you look into if you are certain your code follows the math correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now there's two things you might want to experiment with to help with this problem. First, you might want to add in a &lt;strong&gt;Momentum&lt;/strong&gt; term to the Gradient Descent update formula. This momentum term can be as simple as just adding in a weighted version of the last update to the new one. You can get more info &lt;a href=&quot;http://www.willamette.edu/~gorr/classes/cs449/momrate.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also want to consider searching for the weights using a method that might help avoid being stuck at local optimums, like Particle Swarm Optimization, Genetic Algorithms, etc. You could even potentially use a hybrid approach, where you initially use PSO or GA and then use the Backpropogation algorithm to refine the result. Here are some links to help you with these topics; &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle_swarm_optimization&quot; rel=&quot;nofollow&quot;&gt;PSO Link&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow&quot;&gt;GA Link&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-09-27T07:10:47.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20847" PostTypeId="2" ParentId="20813" CreationDate="2015-09-27T07:30:52.910" Score="1" Body="&lt;p&gt;One specific method that you might find interesting is called the Shooting Method. For a nonlinear BVP, it will be an iterative approach, but it is kind of interesting because you treat the BVP like an initial value problem, so you can use normal integrators like Runge-Kutta, etc. Here's a link to give you an idea: &lt;a href=&quot;https://en.wikipedia.org/wiki/Shooting_method&quot; rel=&quot;nofollow&quot;&gt;Link&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-09-27T07:30:52.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20848" PostTypeId="2" ParentId="20802" CreationDate="2015-09-27T07:43:45.877" Score="0" Body="&lt;p&gt;I definitely advocate for discontinuous galerkin methods, as others have noted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to consider Space-Time Discontinuous Galerkin methods (I developed some codes for this at UIUC). Using this approach, you could potentially mesh your elements in a way that would allow for parallelizing your code and avoiding having to solve large matrices.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-09-27T07:43:45.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20849" PostTypeId="1" CreationDate="2015-09-27T08:12:57.717" Score="1" ViewCount="98" Body="&lt;p&gt;I want to solve a set of hyperbolic equations (not the Euler equations) using an upwind type method. I am interested in using a first order upwind scheme and one that is not based on the method of characteristic and does not require the eigenvalues and eigenvector. So far, I have understood that I first need to solve the Reimann problem, to find out the wave speeds (If I'm not mistaken) and then use that information to work out the fluxes at the boundaries using Gudnov type methods. I was wondering if someone could give me a better step to step guide of what I need to do, and recommendation on the type of methods to use? &lt;/p&gt;&#xA;" OwnerUserId="17476" LastActivityDate="2015-10-27T18:18:43.260" Title="solving a hyperbolic set of equations - upwind type method" Tags="|fluid-dynamics|eigenvalues|discretization|hyperbolic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20850" PostTypeId="1" CreationDate="2015-09-27T13:51:01.030" Score="0" ViewCount="122" Body="&lt;p&gt;I need to solve to solve a &quot;large&quot; symmetric sparse linear systems, with matrix size 8000?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I heard about HSL, ITPACK, but I don't know how to use them, and I am working in C language.&lt;/p&gt;&#xA;" OwnerUserId="17690" LastEditorUserId="9667" LastEditDate="2015-09-28T20:57:25.067" LastActivityDate="2015-09-28T20:57:25.067" Title="solver linear system equation" Tags="|linear-algebra|sparse-matrix|c|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20851" PostTypeId="2" ParentId="20850" CreationDate="2015-09-27T14:46:29.490" Score="4" Body="&lt;p&gt;You haven't told us whether your matrix is positive definite or not- this is a very important factor in selecting a solver.  You also haven't told us whether the matrix is well conditioned or poorly conditioned.  A third important question is whether you need a fairly accurate solution (e.g. accurate to 10 digits) or whether a less accurate solution (e.g. accurate to 4 or 5 digits) would be acceptable.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for matrices of this small size (8,000 by 8,000 is quite small by contemporary standards) you should probably be using a direct factorization method rather than an iterative method.  Furthermore, unless you have a lot of these systems to solve it probably isn't worth the added programming effort to use a sparse direct factorization method.  Thus I'd suggest storing the matrix in dense form and using LAPACK (and in particular because of its C language bindings the CLAPACK package) to solve your system of equations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're willing to invest somewhat more time and effort in coding this, and if you're concerned about the performance of the dense matrix implementation then you might consider using the sparse direct factorization approach as implemented (for example) in the SuiteSparse package written by Timothy Davis.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-09-27T15:00:04.247" LastActivityDate="2015-09-27T15:00:04.247" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20853" PostTypeId="2" ParentId="20849" CreationDate="2015-09-27T17:44:31.300" Score="1" Body="&lt;p&gt;If you like using the finite element method you can use the streamline upwind Petrov/Galerkin (SUPG) which is essentially a stabilized Galerkin method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar modifications can also be made to FDM and FVM.&lt;/p&gt;&#xA;" OwnerUserId="17691" LastActivityDate="2015-09-27T17:44:31.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20854" PostTypeId="2" ParentId="20841" CreationDate="2015-09-27T19:29:22.793" Score="4" Body="&lt;p&gt;Let $\mathbf{\theta}$ be a Gaussian random vector with mean $\mathbf{\mu}$ and covariance matrix $\mathbf{\Sigma}_\mathbf{\theta}$. Let $\mathbf{p}_\theta$ denote the joint PDF. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $J_\mathbf{\theta}$ be the objective function, as its negative logarithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J_\mathbf{\theta}=-ln(\mathbf{p}_\theta)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By taking the partial derivatives w.r.t. $\theta_d$ and $\theta_{d'}$, the $(d,d')$ component of the Hessian can be obtained: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H^{(d,d')}(\mathbf{\mu})=\frac{\partial{J(\theta)}}{\partial{\theta_{d}}\partial{\theta_{d'}}}\Bigr|_{\theta=\mu}=(\mathbf\Sigma^{-1}_{\theta})^{(d,d')}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In that case, the Hessian matrix is equal to the inverse of the covariance matrix&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H(\mu)=\mathbf\Sigma^{-1}_{\theta}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is the case when evaluated at the mean value. Yet, for Gaussian random variables, the second derivatives of $J_\mathbf{\theta}$ are constant&#xA;for all $\theta$ because of the quadratic nature. Therefore, Hessian matrix can be computed without obtaining the mean vector $\mathbf{\mu}$. It is also the case that the entries in Hessian matrix carry valuable geometric information about the random vector, such as curvature and conditional variances and etc. &lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2015-09-27T19:29:22.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20855" PostTypeId="1" CreationDate="2015-09-27T20:56:32.697" Score="3" ViewCount="95" Body="&lt;p&gt;Do all BLAS routines compute the respective operation with minimum error ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. Is the reduction in &lt;code&gt;sdot&lt;/code&gt; computed with least error ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to call these routines through &lt;code&gt;gsl/gsl_cblas.h&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="713" LastEditDate="2015-09-29T07:40:58.553" LastActivityDate="2016-09-22T13:10:45.343" Title="Do BLAS routines compute their respective operations with minimum error?" Tags="|c|floating-point|blas|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20856" PostTypeId="1" CreationDate="2015-09-27T21:59:40.387" Score="2" ViewCount="512" Body="&lt;p&gt;I have a matrix which is &quot;almost&quot; like an upper triangular just that the last row has non zero elements. And I want to perform the QR decomposition on that matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know the &quot;name&quot; of such a matrix? And also since I don't want to use the built in matlab QR function for such a matrix, can anyone suggest a better algorithm for me to use?&lt;/p&gt;&#xA;" OwnerUserId="17553" LastEditorUserId="1128" LastEditDate="2015-09-28T19:58:41.497" LastActivityDate="2015-11-19T16:05:59.420" Title="QR decomposition" Tags="|linear-algebra|matlab|algorithms|matrix|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20857" PostTypeId="1" AcceptedAnswerId="20858" CreationDate="2015-09-27T22:46:47.360" Score="0" ViewCount="679" Body="&lt;p&gt;I'm putting together a very simple integration program in FORTRAN using MPI.  I have done this with C and all was well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in my &quot;MAP_REDUCE&quot; call, the comm argument seems problematic.  I have tried several different values and have gotten several error messages when the program runs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note:  the program does compile, but does not run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the program itself (FORTRAN 90):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  c Fortran example                                                      &#xA;  program intega&#xA;  implicit none&#xA;  include '/opt/apps/intel15/mvapich2/2.1/include/mpif.h'&#xA;&#xA;  integer rank,size,ierror,tag,status(MPI_STATUS_SIZE),i&#xA;  real :: integral = 0.00, x = 0.00, total_t = 0.00&#xA;  real :: analy,numres,edif&#xA;&#xA;&#xA;  real, PARAMETER :: x_lower_bound = 0,x_upper_bound = 1&#xA;  real, PARAMETER :: nsteps = 10000000&#xA;  real, PARAMETER :: dx = (x_upper_bound - x_lower_bound)/nSteps&#xA;&#xA;&#xA;  integer :: iMin,iMax,ierr=0,comm=0&#xA;&#xA;&#xA;  call MPI_INIT(ierror)&#xA;  call MPI_COMM_SIZE(MPI_COMM_WORLD,size,ierror)&#xA;  call MPI_COMM_RANK(MPI_COMM_WORLD,rank,ierror)&#xA;  do i = 0,(iMax-1)&#xA;      x = x_lower_bound + dx*(i+0.5)&#xA;      integral = integral + x*x*dx&#xA;   end do&#xA;&#xA;   call MPI_REDUCE(integral,total_t,1,MPI_REAL,MPI_SUM,0,rank,ierr)&#xA;   analy = ((x_upper_bound**3)-(x_lower_bound**3))/3&#xA;   numres = total_t&#xA;&#xA;   if (rank == 0) then&#xA;      edif = analy-numres&#xA;      print *, numres,analy,edif&#xA;   end if&#xA;&#xA;&#xA;  call MPI_FINALIZE(ierror)&#xA;  end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the error file looks like this:  (not the whole thing, just a sample)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[cli_0]: [cli_1]: aborting job:&#xA;Fatal error in PMPI_Reduce:&#xA;Invalid communicator, error stack:&#xA;PMPI_Reduce(1288): MPI_Reduce(sbuf=0x6ac4a8, rbuf=0x6ac4b4, count=1, MPI_REAL, \&#xA;MPI_SUM, root=0, comm=0x1) failed&#xA;PMPI_Reduce(1163): Invalid communicator&#xA;&#xA;aborting job:&#xA;Fatal error in PMPI_Reduce:&#xA;Invalid communicator, error stack:&#xA;PMPI_Reduce(1288): MPI_Reduce(sbuf=0x6ac4a8, rbuf=0x6ac4b4, count=1, MPI_REAL, \&#xA;MPI_SUM, root=0, comm=0x0) failed&#xA;PMPI_Reduce(1163): Invalid communicator&#xA;&#xA;[cli_2]: aborting job:&#xA;Fatal error in PMPI_Reduce:&#xA;Invalid communicator, error stack:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the communicator is the troublemaker.  But not sure what I should be putting in there instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for any suggestions.&lt;/p&gt;&#xA;" OwnerUserId="17692" LastEditorUserId="9667" LastEditDate="2015-09-28T20:14:25.130" LastActivityDate="2015-09-28T20:14:25.130" Title="Comm argument on MPI_Reduce in FORTRAN giving unusual results" Tags="|fortran|mpi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20858" PostTypeId="2" ParentId="20857" CreationDate="2015-09-28T00:23:37.590" Score="2" Body="&lt;p&gt;Your &lt;code&gt;MPI_Reduce&lt;/code&gt; call doesn't have a valid communicator specified. I think you left it out entirely. The place where you have &lt;code&gt;rank&lt;/code&gt; should be the same value everywhere you call it, and it should be a communicator. From the look of your code, it should be &lt;code&gt;MPI_COMM_WORLD&lt;/code&gt; since every rank is doing the operation.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-09-28T01:15:34.400" LastActivityDate="2015-09-28T01:15:34.400" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20861" PostTypeId="1" CreationDate="2015-09-28T08:21:42.207" Score="5" ViewCount="163" Body="&lt;p&gt;I have to solve the following least squares problem:&#xA;\begin{equation}&#xA;\| \left[ \begin{smallmatrix} \mathbf{L} \\ \mathbf{I} \end{smallmatrix} \right]\mathbf{x} - \mathbf{b} \|_2^2&#xA;\end{equation}&#xA;where $\mathbf{L} \in \mathbb{R}^{n\times n}$ is $O(n)$ sparse lower triangular matrix, $\mathbf{I} \in \mathbb{R}^{n \times n}$ is the identity and $\mathbf{b} = \left[ \begin{smallmatrix} \mathbf{b}_1 \\ \mathbf{b}_2 \end{smallmatrix} \right] \in \mathbb{R}^{2n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, solving individual system $\mathbf{Lx} = \mathbf{b}_1$ is of $O(n)$ complexity, by the forward substitution algorithm, but the least squares fit is expensive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am open to &lt;em&gt;any&lt;/em&gt; suggestion, including fast approximate stochastic solvers etc. Of course, it would be perfect if one is aware of a direct method that exploits this kind of structure.&lt;/p&gt;&#xA;" OwnerUserId="15277" LastEditorUserId="27" LastEditDate="2015-09-28T14:03:17.150" LastActivityDate="2015-10-01T08:25:53.807" Title="Large scale triangular least squares" Tags="|linear-solver|sparse-matrix|least-squares|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20862" PostTypeId="2" ParentId="19565" CreationDate="2015-09-28T09:46:01.333" Score="2" Body="&lt;p&gt;To partially answer my question - there is &lt;a href=&quot;http://arxiv.org/abs/1410.1390&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; paper, but coming with a strong assumption on the multiplier value.&#xA;Mingyi Hong, Zhi-Quan Luo, Meisam Razaviyayn &quot;Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems&quot;&lt;/p&gt;&#xA;" OwnerUserId="15277" LastEditorUserId="15277" LastEditDate="2015-11-01T21:43:11.197" LastActivityDate="2015-11-01T21:43:11.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20863" PostTypeId="2" ParentId="20856" CreationDate="2015-09-28T12:59:43.323" Score="4" Body="&lt;p&gt;Look at section 5 of &lt;a href=&quot;http://web.stanford.edu/group/SOL/papers/ggms74.pdf&quot; rel=&quot;nofollow&quot;&gt;http://web.stanford.edu/group/SOL/papers/ggms74.pdf&lt;/a&gt; to see how to update QR decomposition when a row is added to the matrix you wish to factorize.  You may find it not so easy to fully digest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think algorithm 6.16 of &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=3&amp;amp;ved=0CCsQFjACahUKEwjvnff54JnIAhUCFR4KHfAlDgY&amp;amp;url=http%3A%2F%2Fwww.springer.com%2Fcda%2Fcontent%2Fdocument%2Fcda_downloaddocument%2F9783319043241-c1.pdf%3FSGWID%3D0-0-45-1453619-p176445658&amp;amp;usg=AFQjCNHwL1jRh_kEY-7ySZXHAQAQjrn27A&amp;amp;sig2=cBASIpwNUnM5L4pmvXpJ3g&amp;amp;cad=rja&quot; rel=&quot;nofollow&quot;&gt;Chapter 6 of Scientific Computing - An Introduction using Maple and MATLAB&lt;/a&gt; will do the trick for you, allowing you to update in MATLAB an existing QR factorization when you add a row to the matrix you wish to factorize.  Here are the MATLAB m files &lt;a href=&quot;http://www.cs.cornell.edu/cv/GVL4/M-Files/Chapter%206/Chap6.htm&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.cornell.edu/cv/GVL4/M-Files/Chapter%206/Chap6.htm&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2015-09-28T12:59:43.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20864" PostTypeId="1" AcceptedAnswerId="20877" CreationDate="2015-09-28T13:05:58.717" Score="2" ViewCount="126" Body="&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am performing least squares fitting using an objective function of the form $f(\mathbf{x})$ where $\mathbf{x}$ is a vector of parameters containing around 20 elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model function is somewhat expensive to evaluate so I am exploring options for reducing the number of parameters in the model. Are there standard approaches algorithms to tackle this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Current approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After least squares has completed I calculate the covarience matrix $c$ using the Jacobian $\mathbf{J}$ at the solution point $\mathbf{x}_s$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;c = \left(\mathbf{J}^T\mathbf{J}\right)^{-1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By comparing the relative strength of diagonal contributions to this matrix I then calculate a 'dependency' vector for each parameter,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{d} = 1 - \frac{1}{\textrm{diag}(c)\textrm{diag}(c^{-1})}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closer the element in $\mathbf{d}$ is to zero the more dependence the corresponding parameter has on the least squares solution. For example, I arrive at values shown in the radar plot below,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AZUKD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AZUKD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shows some key parameters (the ones tipping towards zero), but many parameters that are close to edge. I think these are candidates for reduction. Either don't include them in the model or fix there values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is don't know &lt;em&gt;a priori&lt;/em&gt; which parameters will be the ones to include and the ones to reduce. I therefore need to come up with an algorithm that dynamically turns parameters on and off. I was going to use the dependency vector to help with this. But I'm sure I'm not the first to have this problem and would like some advice before re-inventing the wheel.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2015-09-28T15:48:47.240" LastActivityDate="2015-09-29T21:49:16.680" Title="Parameter reduction algorithm for least square model" Tags="|linear-algebra|optimization|algorithms|least-squares|numerical-modelling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20867" PostTypeId="1" CreationDate="2015-09-28T21:29:56.643" Score="3" ViewCount="3873" Body="&lt;p&gt;What kind of optimisations does the option &lt;code&gt;-ffast-math&lt;/code&gt; do ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw that the time taken for a simple $O(n^2)$ algorithm being reduced to that of an $O(n)$ algorithm using the option.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2015-09-29T02:57:40.723" LastActivityDate="2015-09-29T07:04:34.110" Title="what does -ffast-math do?" Tags="|optimization|floating-point|computer-arithmetic|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20868" PostTypeId="2" ParentId="20867" CreationDate="2015-09-29T01:40:40.377" Score="7" Body="&lt;p&gt;&lt;a href=&quot;https://gcc.gnu.org/wiki/FloatingPointMath&quot; rel=&quot;noreferrer&quot;&gt;There is a canonical answer to this question in GCC's wiki&lt;/a&gt;, which is presumably maintained, it is by far the most authoritative source of information for this type of question. This question, on the other hand, may eventually go out of date.&#xA;This is all explained in more detail in the wiki, with examples. The following is essentially a quote from it to illustrate how it answers this exact question, with minor comments:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-fno-signaling-nans&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;-fno-trapping-math&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;IEEE standard recommends that implementations allow trap handlers to handle exceptions like divide by zero and overflow. This flags assumes that no use-visible trap will happen.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;-funsafe-math-optimizations&lt;/code&gt; - These optimizations break the laws of floating-point arithmetic and may replace them with the laws of ordinary infinite-precision arithmetic:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Due to roundoff errors the associative law of algebra do not necessary hold for floating point numbers and thus expressions like (x + y) + z are not necessary equal to x + (y + z).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;-ffinite-math-only&lt;/code&gt; - Special quantities such as &lt;code&gt;inf&lt;/code&gt; or &lt;code&gt;nan&lt;/code&gt; are assumed never to appear, this saves the time spent looking for them and handling them appropriately. For example, should $x - x$ always be equal to $0.0$?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;-fno-errno-math&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;disables setting of the errno variable as required by C89/C99 on calling math library routines. For Fortran this is the default.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;-fcx-limited-range&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;causes the range reduction step to be omitted when performing complex division. This uses $a / b = ((ar*br + ai*bi)/t) + i((ai*br - ar*bi)/t)$ with $t = br*br + bi*bi$ and might not work well on arbitrary ranges of the inputs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;-fno-rounding-math&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-fno-signed-zeros&lt;/code&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Due to roundoff errors the associative law of algebra do not necessary hold for floating point numbers and thus expressions like (x + y) + z are not necessary equal to x + (y + z).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Strictly speaking the implications of the last two are not always as intuitive as one might think. For example (see wiki), what about $ -(a - a) = a - a$, is it $+0.0$ or $-0.0$? I believe there is a fair amount of literature on the exact implications, especially by &lt;a href=&quot;http://www.cs.berkeley.edu/~wkahan/&quot; rel=&quot;noreferrer&quot;&gt;Bill Kahan&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Not directly mentioned (I don't see it?), but with &lt;code&gt;-ffast-math&lt;/code&gt;, certain common special functions such as the reciprocal $1/x$ and the square root $\sqrt{x}$ are replaced with less precise versions that are faster, but which still have some &quot;tolerable&quot; error levels (versus 0ulp error required by the standard) - &lt;a href=&quot;http://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html#Errors-in-Math-Functions&quot; rel=&quot;noreferrer&quot;&gt;here, for example, is what precision is usually provided by glibc's libm&lt;/a&gt;. Indeed, this is the most common cause of speedup from &lt;code&gt;-ffast-math&lt;/code&gt;, in code that does a lot of arithmetic with divisions and square roots, almost to the point that I (&lt;em&gt;personally&lt;/em&gt;) think the other suboptions (&lt;code&gt;-ffinite-math-only&lt;/code&gt; and the like especially - signalling &lt;code&gt;NaN&lt;/code&gt;s are quite useful for debugging) cause a bit too much hassle in terms of their cost/benefit.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I saw that the time taken for a simple $O(n^2)$ algorithm being reduced to that of an $O(n)$ algorithm using the option.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I believe this is improbable and it is possible you made a mistake in your analysis. Unsafe floating-point optimizations might make individual expressions somewhat cheaper to evaluate by virtue of having a larger choice of optimizations. But the speedup should always be at most a constant factor. Is it possible you compared an $O(n^2)$ algorithm with an $O(n)$ for insufficiently large $n$?&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-09-29T07:04:34.110" LastActivityDate="2015-09-29T07:04:34.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20869" PostTypeId="2" ParentId="20867" CreationDate="2015-09-29T02:01:53.443" Score="1" Body="&lt;p&gt;An $n^2$ algorithm can be reduced to something that behaves $O(n)$ if, for example, if $n$ is known to the compiler, and is a multiple of the vector size for the vector instructions (if any) supported by the processor. If the compiler can see all of this, it can unroll an inner loop and use vector instructions to do the work. This may reduce the overall operations done to a mere handful and improve the performance substantially. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My read is that &lt;code&gt;fast-math&lt;/code&gt; doesn't enable such optimization, but it could if they are implicitly enabled by the &lt;code&gt;unsafe-math-optimizations&lt;/code&gt; due to associativity restrictions that are disabled therein. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-09-29T02:01:53.443" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20871" PostTypeId="2" ParentId="20855" CreationDate="2015-09-29T07:36:22.080" Score="6" Body="&lt;p&gt;BLAS routines do not typically use stable summation algorithms.&#xA;In the case of gsl, you can look up its source code online - &lt;a href=&quot;http://git.savannah.gnu.org/cgit/gsl.git/tree/cblas/source_dot_r.h&quot;&gt;the source of gsl's &lt;code&gt;sdot&lt;/code&gt; is contained in &lt;code&gt;gsl/cblas/source_dot_r.h&lt;/code&gt;&lt;/a&gt;, and contains this loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  for (i = 0; i &amp;lt; N; i++) {&#xA;    r += X[ix] * Y[iy];&#xA;    ix += incX;&#xA;    iy += incY;&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's just a straightforward sum. The corresponding version in the BLAS library itself is &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d0/d16/sdot_8f_source.html&quot;&gt;here&lt;/a&gt;, and is implemented in essentially the same way.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-09-29T07:43:06.030" LastActivityDate="2015-09-29T07:43:06.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20873" PostTypeId="2" ParentId="19693" CreationDate="2015-09-29T16:51:32.233" Score="0" Body="&lt;p&gt;FYI, the formula from Hastie et al is correct. I just implemented it myself in R and compared the result to what I get from splines::ns() from R. The basis is not identical, but it spans the same space because the fitted values are exactly the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an error somewhere in your implementation. I followed the dumb approach, having implemented seven small functions for four knots: d1-d3, N1-N4, and it worked fine.&lt;/p&gt;&#xA;" OwnerUserId="10302" LastActivityDate="2015-09-29T16:51:32.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20875" PostTypeId="1" AcceptedAnswerId="27974" CreationDate="2015-09-29T20:31:53.180" Score="3" ViewCount="275" Body="&lt;p&gt;Suppose $A\in\mathbb{R}_+^{n\times n}$ is symmetric.  I would like to factorize $A\approx UU^\top$ by solving&#xA;$$&#xA;\begin{array}{rl}&#xA;\min_U &amp;amp; \sum_{ij} \left(A_{ij}\ln\frac{A_{ij}}{[UU^\top]_{ij}}+[UU^\top]_{ij} \right)\\&#xA;\textrm{s.t.} &amp;amp; U\geq0,&#xA;\end{array}&#xA;$$&#xA;for a local minimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective of this problem is KL divergence $\mathrm{KL}(A|UU^\top)$, similar to the one used in &lt;a href=&quot;http://perso.uclouvain.be/paul.vandooren/publications/HoV08.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; and others on nonnegative matrix factorization.  The difference is that I want to optimize as $UU^\top$ rather than $UV^\top$ for matrices $U$ and $V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find papers on this variation of nonnegative factorization, except &lt;a href=&quot;http://perso.uclouvain.be/paul.vandooren/publications/HoV08.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; mentions it briefly under eq.(11) without discussion.  As I understand it, convergence of nonnegative matrix factorization algorithms is well-understood, and I'm hoping to find something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas for how to find a local optimizer using a simple iteration?  Ideally, I'm hoping some method with nice convergence (e.g. multiplicative updates) from nonnegative matrix factorization will be applicable without much adaptation.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2017-10-03T02:43:14.563" Title="Symmetric nonnegative matrix factorization" Tags="|linear-algebra|optimization|nonlinear-programming|nonconvex|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20876" PostTypeId="2" ParentId="20842" CreationDate="2015-09-29T21:18:21.337" Score="8" Body="&lt;p&gt;This may have gone unnoticed in the comments under the original question, but computing $10^9!$ yields a number with 8.5 billion digits, that is it is on the order of $10^{9\cdot 10^9}$. Given that $10^{9\cdot 10^9}=1000^{3\cdot 10^9} \approx 1024^{3\cdot 10^9}=(2^{10})^{3\cdot 10^9}=2^{3\cdot 10^{10}}$, you need approximately $3\cdot 10^{10}$ bits, or around $4\cdot 10^9$ bytes to simply &lt;em&gt;represent&lt;/em&gt; such a number. That's a sizable amount of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine that you want to compute this number in parallel by having each processor compute a part of the factorial. For example, processor $p$ computes $Q_p=\prod_{k=N_{p-1}}^{N_p} k$, then the total can be computed by $N_P! = \prod_{p=1}^P Q_p$ if you choose $N_0=1$ and $P$ is the number of processors. But generally, each one of the $Q_p$ are going to be of the same order of magnitude as the final result (well, not quite, but you get the general idea), so if you have $P$ processors, you need to provision $P$ times as much memory as it takes to store the final result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a GPU, you have &lt;em&gt;many&lt;/em&gt; processors, $P={\cal O}(1000)$. If each of the numbers you have to store already takes 4 GB of memory, then if you have to store it 1000 times over, you very quickly end up with an amount of memory that is not only not available on a typicaly GPU, but in fact not even in regular desktop computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think what this means is that you are biting off more than you can likely chew if you try to compute $10^9!$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-09-29T21:18:21.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20877" PostTypeId="2" ParentId="20864" CreationDate="2015-09-29T21:49:16.680" Score="3" Body="&lt;p&gt;I'll add a real response here, following the comment I just left. As @GeoffOxberry suggests, you might be able to use active subspaces to, in essence, preprocess your objective function and eliminate (linear combinations of) variables. Try the following first. Randomly sample your variables according to some density. One reasonable choice would be to sample according to a Gaussian that is relatively large in a region that may contain the optimum. (Make sure $f$ is well-defined for each sample!) Call these samples $\mathbf{x}_j$ with $j=1,\dots,N$. With 20 variables, I'd say try 40-100 samples, if you can. For each $\mathbf{x}_j$, compute the gradient,&#xA;$$&#xA;\nabla f_j \;=\; \nabla f(\mathbf{x}_j) \;\in\; \mathbb{R}^{20}.&#xA;$$&#xA;Put all these gradient vectors into a matrix&#xA;$$&#xA;\mathbf{G} \;=\; \frac{1}{\sqrt{N}}\begin{bmatrix}&#xA;\nabla f_1 &amp;amp; \cdots &amp;amp; \nabla f_N&#xA;\end{bmatrix} \;\in\; \mathbb{R}^{20\times N},&#xA;$$&#xA;and compute its SVD, $\mathbf{G}=\mathbf{U}\Sigma\mathbf{V}^T$. Look for gaps in the singular values. If you have a big gap in the singular values, then you might have an exploitable active subspace. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the first singular value is much larger than the others, make a plot of $\mathbf{u}_1^T\mathbf{x}_j$ versus $f(\mathbf{x}_j)$, where $\mathbf{u}_1$ is the first left singular vector. Statisticians call this a summary plot. The plot might show some interesting and exploitable characteristics of $f$ that may help the optimization. Or it might not. &lt;/p&gt;&#xA;" OwnerUserId="3652" LastActivityDate="2015-09-29T21:49:16.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20878" PostTypeId="1" CreationDate="2015-09-30T01:19:12.643" Score="7" ViewCount="1740" Body="&lt;p&gt;I want to create a finite element code in fortran. There are many libraries out there on internet. Can someone suggest me a good finite element library in fortran?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ease of use (using fortran language specifically)&lt;/li&gt;&#xA;&lt;li&gt;Well documented&lt;/li&gt;&#xA;&lt;li&gt;free&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17715" LastEditorUserId="1804" LastEditDate="2015-09-30T09:51:04.557" LastActivityDate="2015-09-30T17:19:23.313" Title="Finite element library in fortran" Tags="|finite-element|fortran|libraries|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20879" PostTypeId="1" AcceptedAnswerId="20889" CreationDate="2015-09-30T02:55:05.547" Score="10" ViewCount="639" Body="&lt;p&gt;I was reading some work by Butcher and I came across Pade approximations and the correlation between them and stability functions for some Implicit Runge-Kutta methods. For example, in &lt;a href=&quot;https://en.wikipedia.org/wiki/Pad%C3%A9_table#An_example_.E2.80.93_the_exponential_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; Pade table for the exponential function, we see that the &lt;span class=&quot;math-container&quot;&gt;$(2,1)$&lt;/span&gt; Pade approximation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1+\frac{1}{3}z}{1-\frac{2}{3}z+\frac{1}{6}z^{2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;corresponds to the stability function for the &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods#Radau_IIA_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;RADAU IIA&lt;/strong&gt;&lt;/a&gt; method. Similarly, the &lt;span class=&quot;math-container&quot;&gt;$(2,2)$&lt;/span&gt; Pade approximation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1+\frac{1}{2}z+\frac{1}{12}z^{2}}{1-\frac{1}{2}z+\frac{1}{12}z^{2}} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;corresponds to the stability function of the &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods#Gauss.E2.80.93Legendre_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;2-stage Gauss Runge-Kutta&lt;/strong&gt;&lt;/a&gt; method. However, I'm not sure if other Pade approximations correspond to stability functions; For example, I can't find the numerical method whose stability function corresponds to the &lt;span class=&quot;math-container&quot;&gt;$(2,0)$&lt;/span&gt; Pade approximation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{1-z+\frac{1}{2}z^{2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So my question is two fold (I apologise for asking two questions in one post, it seems like a waste to create two new posts though):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it always possible to generate an Implicit Runge-Kutta scheme whos stability function corresponds to a particular Pade approximation? And if so&lt;/li&gt;&#xA;&lt;li&gt;How do we construct such an implicit scheme?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16852" LastEditorUserId="16852" LastEditDate="2024-03-09T04:21:37.427" LastActivityDate="2024-03-10T12:33:01.340" Title="How to derive an Implicit Runge-Kutta method from Pade approximation" Tags="|stability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20880" PostTypeId="2" ParentId="20878" CreationDate="2015-09-30T05:55:31.737" Score="5" Body="&lt;p&gt;I would recommend writing your own code using PETSc. The most complex part of a parallel FE code is parallel assembly and solve and PETSc takes care of both. The latest version of PETSc also has routines for managing unstructured meshes meaning you can write a performant, scalable FE code in a just a few hundred lines.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2015-09-30T17:19:23.313" LastActivityDate="2015-09-30T17:19:23.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20881" PostTypeId="1" AcceptedAnswerId="20882" CreationDate="2015-09-30T09:57:01.810" Score="1" ViewCount="85" Body="&lt;p&gt;I have experimental data containing horizontal and vertical components of speed and I need to evaluate this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial^2}{\partial x_i \partial x_j}\left(v_iv_j\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denoting $U$ as horizontal component and $V$ as vertical, we should obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial^2}{\partial x_i \partial x_j}\left(v_iv_j\right) \rightarrow \frac{\partial^2}{\partial x^2}(U^2)+\frac{\partial^2}{\partial y^2}(V^2) + 2\frac{\partial^2}{\partial x \partial y}(UV)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used central finite difference of the second order for $\partial^2/\partial x^2$ and finite difference of the first order for mixed derivative. In MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%H and WW are height and width respectively&#xA;for ii=3:H-2&#xA;    for jj=3:WW-2&#xA;&#xA;        %d^2/dx^2&#xA;        A(ii,jj)=(-1/12*U(ii,jj-2)^2+4/3*U(ii,jj-1)^2-5/2*U(ii,jj)^2+4/3*U(ii,jj+1)^2-1/12*U(ii,jj+2)^2)/h^2;&#xA;&#xA;        %d^2/dy^2&#xA;        B(ii,jj)=(-1/12*V(ii-2,jj)^2+4/3*V(ii-1,jj)^2-5/2*V(ii,jj)^2+4/3*V(ii+1,jj)^2-1/12*V(ii+2,jj)^2)/h^2;&#xA;&#xA;        %Mixed derivative&#xA;        C(ii,jj)=(V(ii+1,jj+1)*U(ii+1,jj+1)-V(ii+1,jj-1)*U(ii+1,jj-1)-V(ii-1,jj+1)*U(ii-1,jj+1)+V(ii-1,jj-1)*U(ii-1,jj-1))/(4*h^2);&#xA;&#xA;        %sum of derivatives&#xA;        LGHT(ii,jj)=(A(ii,jj)+B(ii,jj)+2*C(ii,jj));&#xA;&#xA;   end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results are not very satisfactory so far: &quot;wild unconnectivity&quot; all around the screen.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Do I have a mistake somewhere?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Would you use different kind of scheme?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;What kind of data treatment (spline?) would you recommend for obtaining smoother results?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Example of input data:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zFKkM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zFKkM.png&quot; alt=&quot;Example of input data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Output:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XIUAC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XIUAC.png&quot; alt=&quot;Output data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17721" LastEditorUserId="1804" LastEditDate="2015-09-30T15:07:43.387" LastActivityDate="2015-09-30T15:07:43.387" Title="Finite difference for Lighthill source term" Tags="|matlab|finite-difference|data-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20882" PostTypeId="2" ParentId="20881" CreationDate="2015-09-30T10:59:15.157" Score="2" Body="&lt;p&gt;Your reformulation is wrong: The product rule for derivatives is $(uv)' = u'v + uv'$. Applying this in your situation (with $\partial_x$ instead of $\frac{\partial}{\partial x}$ and $\partial_{xy} = \frac{\partial^2}{\partial x\partial y}$ for brevity would yield&#xA;$$\partial_{xy}(uv) = \partial_y(\partial_x u \,v + u\,\partial_x v)&#xA;= \partial_{xy}u \,v + \partial_x u \, \partial_y v + \partial_y u\,\partial_x v + u\,\partial_{xy}v&#xA;$$&#xA;(note that you have four different terms, not three).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to have a second-order accurate scheme (mixing schemes of different accuracy is in general a waste of effort), the standard stencils are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_x u_{ij} \approx \frac{u_{i+1,j}-u_{i-1,j}}{2h},\qquad\partial_y u_{ij} \approx \frac{u_{i,j+1}-u_{i,j-1}}{2h}$$&#xA;and &#xA;$$\partial_{xy} u_{ij} \approx \frac{u_{i+1,j+1}-u_{i+1,j-1}-u_{i-1,j+1}+u_{i-1,j-1}}{4h^2}.$$&#xA;You can either apply the second directly to $(u\,v)_{ij}$, or plug in both stencils on the right-hand side. (Or better yet, both, and compare the results.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-09-30T11:26:56.810" LastActivityDate="2015-09-30T11:26:56.810" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20883" PostTypeId="1" CreationDate="2015-09-30T11:18:57.137" Score="3" ViewCount="100" Body="&lt;p&gt;I would like -if possible at all- to represent and handle missing data (in the statistical sense) within the standard IEEE-754 format. Seeing that for both SNaNs and QNaNs various bit representations are possible, I wonder if they all can arise from standard computations or some can be made inaccessible to paths other than the corresponding explicit bit storage, and thus safe and reserved to represent &quot;empty slots&quot; for missing data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So first of all, does the standard dictate a specific behaviour in representing NaNs or can each processor/system treat them differently? And in the former case, can I do what's hinted at above? (assuming it's safe to detect such values e.g. in C++ via static_cast or a union)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The IEEE standard generically dictates NaN for certain undefined operations, however not the specific payloads. I wonder if there is a standard de facto for this. I would not care if external libraries each have their own payloads, but it's a more serious matter if standard C++ libs are not consistent.&lt;/p&gt;&#xA;" OwnerUserId="5005" LastEditorUserId="5005" LastEditDate="2015-10-01T13:05:20.407" LastActivityDate="2015-10-01T13:05:20.407" Title="IEEE-754 NaNs and missing data" Tags="|floating-point|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20884" PostTypeId="1" CreationDate="2015-09-30T12:30:54.710" Score="1" ViewCount="98" Body="&lt;p&gt;I am facing the following problem. I need to solve numerically a set of coupled equations &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$i\frac{d}{dt}f_{n}^{(i)}(t) = \left[U\cdot n(n-1) + \mu\cdot n\right]f_{n}^{(i)}(t) - \sqrt{n+1}\Phi_i^{*}\ f_{n+1}^{(i)}-\sqrt{n}\Phi_{i}\ f_{n-1}^{(i)}$$&#xA;where $U,\mu$ are just constants and &#xA;$$\Phi_{i} = \sum\limits_{n=1}^{N}\left( f_{n-1}^{(i+1)}(t)\right)^{*}\ f_{n}^{(i+1)}(t)\sqrt{n}$$&#xA;has to be determined self-consistently. These are coupled differential equations where $i = 1,2,\ldots, M$ and $n = 0,1,,\ldots, N$. What kind of numerical methods can be used in order to solve this problem efficiently? To be clear, I am looking for time evolution of each $f_{n}^{(i)}$ (complex numbers).&lt;/p&gt;&#xA;" OwnerUserId="14780" LastEditorUserId="14780" LastEditDate="2015-09-30T14:25:57.983" LastActivityDate="2015-09-30T14:25:57.983" Title="Time dependent self-consistent equations" Tags="|algorithms|ode|nonlinear-equations|computational-physics|computational-chemistry|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20885" PostTypeId="1" CreationDate="2015-09-30T13:14:28.343" Score="4" ViewCount="83" Body="&lt;p&gt;Let's say I'm optimizing something. To pick an arbitrary example, let's say I'm choosing the shape of some part to maximize strength-to-weight ratio. So I get some FEM software, parametrize the shape, and run gradient ascent or whatever to find the optimum shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I do this procedure, I would generally expect that the calculated optimal strength-to-weight ratio is an &lt;em&gt;overestimate&lt;/em&gt; of the actual strength-to-weight ratio for that shape---especially if I'm running the FEM software at low accuracy. Basically, the optimization algorithm will seek out and exploit any inaccuracies in the software to create impossibly high performance metrics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of economics, this basic insight is famously known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Goodhart%27s_law&quot; rel=&quot;nofollow&quot;&gt;Goodhart's Law&lt;/a&gt;. In the context of optimization, how do people refer to this? Is there a term for it? Is there a paper or textbook that I can cite that discusses it?&lt;/p&gt;&#xA;" OwnerUserId="1761" LastActivityDate="2015-09-30T13:14:28.343" Title="Is there a term for Goodhart's Law in the context of optimization?" Tags="|optimization|terminology|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20888" PostTypeId="2" ParentId="20878" CreationDate="2015-09-30T17:00:03.183" Score="2" Body="&lt;p&gt;There is of course a good question why exactly you'd like to create a new finite element library in 2015 when there are already excellent ones out there. There's also the question why you would want to use Fortran in 2015 when that's simply not any more what the community believes is the language to use for new projects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you believe that you have good reasons for both of these questions, then it may be worthwhile looking into PLTMG -- the grandfather of all modern finite element libraries, and written in Fortran. As others have pointed out, PETSc also provides interfaces for Fortran and has finite element support built in.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-09-30T17:00:03.183" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20889" PostTypeId="2" ParentId="20879" CreationDate="2015-09-30T17:27:59.823" Score="9" Body="&lt;p&gt;The only way I could figure out how to do this is to just work backwards, starting with the stability function for implicit Runge-Kutta schemes. I'm not sure if there is a slicker way to do this, this is just the most obvious.&lt;/p&gt;&#xA;&lt;p&gt;Note that the following only applies to implicit Runge-Kutta schemes. Also, I'll be using the Pade &lt;span class=&quot;math-container&quot;&gt;$(2,0)$&lt;/span&gt; approximation (as stated in the question) as an example.&lt;/p&gt;&#xA;&lt;p&gt;Using the fact that the Pade approximation is given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;R(z) &amp;amp;= \frac{P(z)}{Q(z)} \\&#xA;&amp;amp;= \frac{1}{1-z+\frac{z^{2}}{2}}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the stability function for an implicit Runge-Kutta is given by (&lt;span class=&quot;math-container&quot;&gt;$\vec e$&lt;/span&gt; is a vector of ones)&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;R(z) &amp;amp;= \frac{P(z)}{Q(z)} \\&#xA;&amp;amp;= \frac{\det(I-zA+z \vec e b^{T})}{\det(I-zA)}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;we just need to construct matrices such that&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\det(I - zA + z \vec e b^{T}) &amp;amp;= 1 \\&#xA;\det(I - zA) &amp;amp;= 1 - z+\frac{z^{2}}{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Setting&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A = \begin{pmatrix}&#xA;a_{1} &amp;amp; a_{2} \\&#xA;a_{3} &amp;amp; a_{4}&#xA;\end{pmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;gives us&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\det(I - zA) &amp;amp;= 1 - z(a_{1} + a_{4}) + z^{2}(a_{1} a_{4} - a_{2} a_{3}) \\&#xA;&amp;amp;= 1 - z + \frac{z^{2}}{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which gives us conditions on our coefficients&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;a_{1} + a_{4} &amp;amp;= 1 \\&#xA;a_{1} a_{4} - a_{2} a_{3} &amp;amp;= \frac{1}{2}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, as far as I'm aware, we can &lt;em&gt;arbitrarily&lt;/em&gt; choose values for the &lt;span class=&quot;math-container&quot;&gt;$a_{i}$&lt;/span&gt; such that they satisfy the above underdetermined system (for example, &lt;span class=&quot;math-container&quot;&gt;$a_{1} = a_{4} = \frac{1}{2}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_{2} = \frac{1}{3}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a_{3} = -\frac{3}{4}$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;We then solve &lt;span class=&quot;math-container&quot;&gt;$\det(I - zA + z \vec e b^{T})$&lt;/span&gt; in a similar manner, by taking&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\vec e b^{T} = \begin{pmatrix}&#xA;b_{1} &amp;amp; b_{2} \\&#xA;b_{1} &amp;amp; b_{2}&#xA;\end{pmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;constructing &lt;span class=&quot;math-container&quot;&gt;$I - zA + z \vec e b^{T}$&lt;/span&gt; (using the previous matrix we constructed, &lt;span class=&quot;math-container&quot;&gt;$I - zA$&lt;/span&gt;, with the components of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; now known from the &lt;span class=&quot;math-container&quot;&gt;$a_{i}$&lt;/span&gt;), taking the determinant and solving the corresponding system of equations in &lt;span class=&quot;math-container&quot;&gt;$b_{1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b_{2}$&lt;/span&gt;. The matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and vector &lt;span class=&quot;math-container&quot;&gt;$b^{T}$&lt;/span&gt; can now be written in a Butcher tableau.&lt;/p&gt;&#xA;" OwnerUserId="16852" LastEditorUserId="16852" LastEditDate="2024-03-10T12:33:01.340" LastActivityDate="2024-03-10T12:33:01.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20890" PostTypeId="1" CreationDate="2015-09-30T17:51:09.437" Score="6" ViewCount="303" Body="&lt;p&gt;When using many iterative methods, whether for solving linear systems, looking for steady-state convergence in CFD, etc., the semilog plot of the residual often shows &quot;humps&quot; as the residual decays. These humps generally appear in a regular pattern (same number of iterations between them for example). I've seen cases where the valley between the hump can be several orders of magnitude lower than the peak of the hump. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider this image of a residual vs. iteration count (just the black line is important):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lqjRX.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lqjRX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example that I can't embed here is Figure 4 in &lt;a href=&quot;http://enu.kz/repository/2011/AIAA-2011-776.pdf&quot; rel=&quot;noreferrer&quot;&gt;this AIAA paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is it that causes these humps in the residual? Why do they appear with a somewhat regular frequency?&lt;/p&gt;&#xA;" OwnerUserId="391" LastActivityDate="2021-06-19T00:40:31.233" Title="What causes periodic humps in residual plots?" Tags="|iterative-method|convergence|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20891" PostTypeId="2" ParentId="20883" CreationDate="2015-09-30T19:58:47.003" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I wonder if they all can arise from standard computations…&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is implementation-dependent, if a library assigns its own payload to its NaNs, its payload may well clash with your own payload. I'd say this is unlikely to be a huge problem, but check the documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also believe this approach is what R uses for missing data (&lt;a href=&quot;https://cran.r-project.org/doc/manuals/R-lang.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/doc/manuals/R-lang.pdf&lt;/a&gt; Section 3.3.4).&#xA;There are relevant existing answers to related questions on StackOverflow &lt;a href=&quot;https://stackoverflow.com/questions/19800415/why-does-ieee-754-reserve-so-many-nan-values&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/2247447/usefulness-of-signaling-nan&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So first of all, does the standard dictate a specific behaviour in representing NaNs or can each processor treat them differently?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An operation that receives one or more NaNs as inputs should usually return one of those NaNs as output, so the payload is preserved.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;And in the former case, can I do what's hinted at above? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, I believe this was the original idea for why NaNs should have a payload at all. It helps with debugging and is used for representing missing data.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;assuming it's safe to detect such values e.g. in C++ via static_cast or a union&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Indeed, since the binary representation of IEEE754 floats is known, it is safe to just extract the payload &lt;a href=&quot;http://en.cppreference.com/w/cpp/language/reinterpret_cast&quot; rel=&quot;nofollow noreferrer&quot;&gt;with &lt;code&gt;reinterpret_cast&lt;/code&gt;&lt;/a&gt; and a little bit manipulation (e.g., &lt;a href=&quot;https://stackoverflow.com/questions/13982340/is-it-safe-to-reinterpret-cast-an-integer-to-float&quot;&gt;this existing question on SO&lt;/a&gt;). &lt;code&gt;static_cast&lt;/code&gt; doesn't do the right thing here, and doing this with union is (&lt;em&gt;technically&lt;/em&gt;) undefined behaviour but would work correctly anyway.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-09-30T19:58:47.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20892" PostTypeId="1" CreationDate="2015-09-30T20:37:01.050" Score="1" ViewCount="200" Body="&lt;p&gt;I am writing a code to perform hybrid monte carlo molecular dynamics. To do this, I need to have a code to initialize the velocities of all particles according to a maxwell distribution. The code is as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   subroutine thermalize&#xA;        implicit none&#xA;        double precision :: MassInv&#xA;        double precision :: RATIO&#xA;        integer :: i&#xA;&#xA;        !--- temp is temperature set point value&#xA;        STDtemp = sqrt(temp)&#xA;        do i = 1 ,np&#xA;           MassInv = 1.0d0/(mass(position(i)%type))&#xA;           MassInv = sqrt(MassInv*kboltz*kcal2amu)&#xA;&#xA;  !---Gaussian() is a functions that returns gaussian random variable n(0,1)            &#xA;           v(i)%x = Gaussian()*STDtemp*MassInv*ratio&#xA;           v(i)%y = Gaussian()*STDtemp*MassInv*ratio&#xA;           v(i)%z = Gaussian()*STDtemp*MassInv*ratio&#xA;&#xA;&#xA;        enddo&#xA;      end subroutine thermalize&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To measure the temperature&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T = 2.0*KE/(kboltz*(3*np - NCONS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where KE is the kinetic energy, np is the number of atoms, kbotlz is boltzmanns constant. I was trying to run some tests with temp=380, and found that my thermalize function was giving me initial kinetic energies that translated to temperatures around T=550 K. When I turned shake off, and hence ncons = 0, the temperature produced by thermalize was 379. So I think I need to modify the thermalize command to account for the constrained DOF's&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To try to account for the shake, the ratio of the KE with shake and without shake present for a fixed temperature is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1/2kT*(3np-ncons)   = 1/2kT*(3np)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;dividing and rearranging     &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;KE_shake/KE_noshake = (3np-ncons)/np&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thus, I am thinking about multiplying by the sqrt of that ratio. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v(i)%x = Gaussian()*STDtemp*MassInv*ratio*sqrt((3np-ncons)/np)&#xA;v(i)%y = Gaussian()*STDtemp*MassInv*ratio*sqrt((3np-ncons)/np)&#xA;v(i)%z = Gaussian()*STDtemp*MassInv*ratio*sqrt((3np-ncons)/np)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives me the correct temperature (~380). However, when I thermalize my particles, and run an NVE trajectory of a previously equilibrated fluid (temperature should fluctuate around 380), the temperature drops to about 300. I have no idea if this is statistically mechanically correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts?&lt;/p&gt;&#xA;" OwnerUserId="16889" LastEditorUserId="16889" LastEditDate="2015-10-02T01:04:37.687" LastActivityDate="2015-10-02T01:04:37.687" Title="Maxwellian distribution of velocities with Shake algorithm present" Tags="|monte-carlo|molecular-dynamics|constraints|" AnswerCount="0" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20893" PostTypeId="1" CreationDate="2015-09-30T20:44:59.717" Score="3" ViewCount="7204" Body="&lt;p&gt;When assembling all the stiffness matrices for each element together, is the final matrix size equal to the number of joints or elements?&lt;/p&gt;&#xA;" OwnerUserId="17725" LastEditorUserId="9667" LastEditDate="2015-10-01T15:37:25.427" LastActivityDate="2020-02-19T15:18:30.590" Title="Does the global stiffness matrix size depend on the number of joints or the number of elements?" Tags="|finite-element|stiffness|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20895" PostTypeId="1" CreationDate="2015-09-30T22:45:28.387" Score="1" ViewCount="198" Body="&lt;p&gt;I am trying to use ARPACK to diagonalize a sparse Hermitian matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My program is executing without error but returning incorrect eigenvalues.  I have tried to trace the problem back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The work is done by two functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;znaupd&lt;/code&gt; and a function I have defined &lt;code&gt;av&lt;/code&gt; to perform matrix vector multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;znaupd&lt;/code&gt; takes a vector as input and passes a resultant vector to &lt;code&gt;av&lt;/code&gt;.  There is some starting default vector that ARPACK chooses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the output of my program, it seems like the default vector is essentially being chosen to be the zero vector, and &lt;code&gt;znaupd&lt;/code&gt; just passes this same zero vector to &lt;code&gt;av&lt;/code&gt; every time.&lt;/p&gt;&#xA;" OwnerUserId="17727" LastActivityDate="2015-09-30T22:45:28.387" Title="ARPACK- Reverse Communication Interface" Tags="|fortran|arpack|" AnswerCount="0" CommentCount="2" ClosedDate="2018-11-13T14:27:16.733" ContentLicense="CC BY-SA 3.0" />
  <row Id="20896" PostTypeId="2" ParentId="20861" CreationDate="2015-10-01T08:25:53.807" Score="4" Body="&lt;p&gt;This looks isomorphic to &lt;a href=&quot;https://en.wikipedia.org/wiki/Tikhonov_regularization&quot; rel=&quot;nofollow&quot;&gt;Tikhonov regularization&lt;/a&gt; (a.k.a. ridge regression).  Some googling brought up &lt;a href=&quot;http://web.stanford.edu/group/SOL/software/lsqr/&quot; rel=&quot;nofollow&quot;&gt;LSQR&lt;/a&gt; and the newer &lt;a href=&quot;http://web.stanford.edu/group/SOL/software/lsmr/&quot; rel=&quot;nofollow&quot;&gt;LSMR&lt;/a&gt;.  Those links both have implementations in a number of languages.  For large scale problems, Petsc has &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPLSQR.html&quot; rel=&quot;nofollow&quot;&gt;KSPLSQR&lt;/a&gt; built in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on what you mean by 'large scale', mlpack may also work.  Mlpack has a &lt;a href=&quot;http://www.mlpack.org/docs/mlpack-1.0.12/doxygen.php?doc=lrtutorial.html&quot; rel=&quot;nofollow&quot;&gt;tutorial&lt;/a&gt; covering both its command line program as well as its C++ api.&lt;/p&gt;&#xA;" OwnerUserId="17532" LastActivityDate="2015-10-01T08:25:53.807" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20897" PostTypeId="1" AcceptedAnswerId="20902" CreationDate="2015-10-01T08:44:14.483" Score="8" ViewCount="1389" Body="&lt;p&gt;I work on geological problems and I use the Finite Element Method. But this question can be applied on other classical mechanical problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work on implicit 3D surfaces (which represent the limits between two geological layers aka two media).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to impose displacement on these surfaces (Dirichlet Condition). The condition that I impose on surfaces are transfered on the surrounding nodes. I make the assumption that I can use the &quot; classical&quot; way to apply Dirichlet Condition (penalty method in my case) on these nodes, that are NOT on the the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: do you think this assumption is valid? Have you got some references about the subject? As I understand the problem I try to apply Boundary Conditions not on the boundaries...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We try to solve this problem:&#xA;$$&#xA;  \left\{&#xA;    \begin{aligned}&#xA;      \sigma_{ij,j} + F_{i} &amp;amp;= 0 &amp;amp; &amp;amp;\text{in the domain $\Omega$}&amp;amp;\\&#xA;      u_i&amp;amp;= q_{i}&amp;amp; &amp;amp;\text{on the boundary $\Gamma_{q}$}&amp;amp;\\&#xA;      \sigma_{ij}n_{j} &amp;amp;= h_{i}&amp;amp; &amp;amp;\text{on the boundary $\Gamma_{h}$}&amp;amp;\\&#xA;      u_i &amp;amp;= b_{i}&amp;amp; &amp;amp;\text{punctually in the domain $\Omega$}&amp;amp;\\&#xA;    \end{aligned}&#xA;  \right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We run the FEA on a mesh. We basically try to impose Dirichlet Condition on the nodes which are inside the domain $\Omega$ (fourth line). We constrain the displacement value of $u_i$ to $b_i$ for these nodes.&lt;/p&gt;&#xA;" OwnerUserId="17731" LastEditorUserId="17731" LastEditDate="2015-10-01T13:41:06.883" LastActivityDate="2018-12-05T15:38:04.423" Title="FEM: Possible to have boundary conditions &quot;inside&quot; the domain?" Tags="|finite-element|boundary-conditions|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20900" PostTypeId="2" ParentId="20897" CreationDate="2015-10-01T14:06:32.013" Score="0" Body="&lt;p&gt;Well you can model slip between geologic faults (a discontinuity) using constraint equations which can be thought of as loading at the interface instead of the boundary. You simply specify the relative slip and solve the problem. E.g., see the top-left figure in &lt;a href=&quot;https://bitbucket.org/stali/defmod/wiki/Gallery&quot; rel=&quot;nofollow&quot;&gt;https://bitbucket.org/stali/defmod/wiki/Gallery&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is indeed what you want then I would suggest that you use Lagrange multipliers instead of the penalty method. Off course you can use any constraint equation you want but it should make physical sense.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2015-10-01T14:06:32.013" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20901" PostTypeId="1" AcceptedAnswerId="31374" CreationDate="2015-10-01T14:54:20.450" Score="11" ViewCount="8359" Body="&lt;p&gt;I'm working with functions which, in general, are much smoother and better behaved in log-log space --- so that's where I perform interpolation/extrapolation, etc, and that works very well.  &lt;strong&gt;Is there a way to integrate these numerical functions in log-log space?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. I'm hoping to use some sort of simple trapezoidal rule to perform a cumulative integral (e.g. in python, use &lt;code&gt;scipy.integrate.cumtrapz&lt;/code&gt;), to find some $F(r)$ s.t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(r) = \int_0^r y(x) \, dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm hoping to use the values $log(y)$ and $log(x)$, instead of $y$ and $x$ (when possible).&lt;/p&gt;&#xA;" OwnerUserId="5292" LastActivityDate="2022-08-04T15:34:19.957" Title="Integral in log-log space" Tags="|numerics|integration|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20902" PostTypeId="2" ParentId="20897" CreationDate="2015-10-01T15:36:17.347" Score="9" Body="&lt;p&gt;The solution of the equation you are looking for is in the space $H^1$ of functions that have one weak derivative, but in 2d and 3d, this does not imply that the solution is in fact continuous. As a consequence, it is not possible to define the value of a solution at individual points, and equally consequentially, it is mathematically not possible to &lt;em&gt;prescribe&lt;/em&gt; (boundary) values at individual points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I suspect that you're not actually interested in prescribing values for the displacement at individual points, but for all points along a line (in 2d) or surface (in 3d), and this is perfectly valid. (Mathematically speaking, this is so because the trace operator is well defined on $H^1$). At the discrete level, this is then equivalent to prescribing the values on all nodes that lie on this line/surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conceptually, you can think of this as a crack in your domain that is occupied by an infinitely thin device whose displacement you can manipulate and that attached to the two sides of the crack.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-01T15:36:17.347" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="20903" PostTypeId="2" ParentId="20893" CreationDate="2015-10-01T17:41:37.570" Score="3" Body="&lt;p&gt;The number of rows and columns in the final global sparse stiffness matrix is equal to the number of nodes in your mesh (for linear elements). For example if your mesh looked like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PeMNd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PeMNd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then each local stiffness matrix would be 3-by-3. Once all 4 local stiffness matrices are assembled into the global matrix we would have a 6-by-6 global matrix. For example the local stiffness matrix for element 2 (e2) would added entries corresponding to the second, fourth, and sixth rows and columns in the global matrix. For this mesh the global matrix would have the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{bmatrix}&#xA;    * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * \\&#xA;    * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; * \\&#xA;    0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 \\&#xA;    0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * \\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * \\&#xA;    * &amp;amp; * &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * \\&#xA;\end{bmatrix}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where each * is some non-zero value. I assume that when you say joints you are referring to the nodes that connect elements. If this is the case then using your terminology the answer is: the global stiffness matrix has size equal to the number of joints.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="28872" LastEditDate="2020-02-19T15:18:30.590" LastActivityDate="2020-02-19T15:18:30.590" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20904" PostTypeId="1" CreationDate="2015-10-01T19:50:21.260" Score="3" ViewCount="214" Body="&lt;p&gt;This is related to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/20819/linear-system-solution-with-inequality-constraints-methods&quot;&gt;a question I had asked earlier&lt;/a&gt;, with the distinction that earlier I did not have a non-linear objective functional to minimize. The problem is reproduced below with added information. I used MATLAB to solve the problem posed in that question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why I need to minimize the objective functional given below:&lt;/strong&gt; As it turns out, the constraints I had put on the vectors were not enough to guarantee that the vectors be &quot;as different&quot; from each other as possible. Some of the vectors were completely identical. Therefore, I though it might help to maximize the angles between the vectors, kind of driving them towards orthogonality (right?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt; You are given a set of 16 vectors, $\{\mathbf{a}^i\}_{i=1}^{16}$, and matrices $D_1$ and $D_2$ (both of dimension $[10\times6]$. Let the following be true:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\sum_{i=1}^{16} \mathbf{a}^i = \mathbf{1} \in \mathbb{R}^6\;,$&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{a}^i \geq \mathbf{0}\;,\forall i\;,$ (i.e. all elements positive),&lt;/li&gt;&#xA;&lt;li&gt;$(D_1 + D_2)\mathbf{1} = \mathbf{1} \in \mathbb{R}^{10}\;.$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then, find vectors $\{\mathbf{b}^i\}_{i=1}^{16}$ and $\{\mathbf{c}^i\}_{i=1}^{16}$ that minimize the functional:&#xA;$$&#xA;\min \sum_{i,j=1}^{16} \biggl\vert \dfrac{\pi}{2} - \cos^{-1}\biggl(\dfrac{\mathbf{d}^i\cdot \mathbf{d}^j}{||\mathbf{d}^i||||\mathbf{d}^j||}\biggr)\biggr\vert\;,&#xA;$$&#xA;where $\mathbf{d}^i := \left(\begin{array}{c}&#xA;\mathbf{b}^i\\&#xA;\mathbf{c}^i&#xA;\end{array}\right)$, and such that they satisfy the following constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\mathbf{c}^i - D_1 \mathbf{b}^i = D_2 \mathbf{a}^i\;,\quad \forall i\in \{1,2,\dots,16\}\;,$&lt;/li&gt;&#xA;&lt;li&gt;$\sum_{i=1}^{24} \mathbf{c}^i = \mathbf{1} \in \mathbb{R}^{10}\;,$&lt;/li&gt;&#xA;&lt;li&gt;$\sum_{i=1}^{24} \mathbf{b}^i = \mathbf{1} \in \mathbb{R}^6\;,$&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{c}^i \geq \mathbf{0}\;,\forall i\;,$ (i.e. all elements of each vector positive),&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{b}^i \geq \mathbf{0}\;,\forall i\;,$ (i.e. all elements of each vector positive).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Could anyone please give me any pointers on how I could go about tackling this problem? I have no idea about any methods I could use to solve this problem computationally.&lt;/li&gt;&#xA;&lt;li&gt;Any comments on the objective functional I have defined? I was also thinking of $&#xA;\min \sum_{i,j=1}^{16} \biggl(\dfrac{\pi}{2} - \cos^{-1}\biggl(\dfrac{\mathbf{d}^i\cdot \mathbf{d}^j}{||\mathbf{d}^i||||\mathbf{d}^j||}\biggr)\biggr)^2\;,&#xA;$&#xA;but thought that they would be equivalent. Any thoughts? Could I do better than these two?&lt;/li&gt;&#xA;&lt;li&gt;Is there a way to devise a functional to drive them towards &lt;strong&gt;linear independence&lt;/strong&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, this is NOT homework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="17666" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-10-01T21:26:34.420" Title="Find constrained vectors maximizing angles between them - methods?" Tags="|linear-algebra|optimization|constrained-optimization|nonlinear-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20905" PostTypeId="1" CreationDate="2015-10-01T21:02:28.380" Score="5" ViewCount="240" Body="&lt;p&gt;Let $n,m\in \mathbb{N}$ be such that $m\ge n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $M_1\in \mathbb{R}^{n\times n}$, $\{M_{2},M_{3} \} \subset \mathbb{R}^{m\times m}$ be symmetric positive definite and computationally cheap to invert.  You can think of them as a diagonal matrix.  Let $S\in \mathbb{R}^{n\times m}$ be full rank.  Let $B\in \mathbb{R}^m$.  I am interested in solving the following linear system over a range of $\omega \in \mathbb{R}$.&#xA;$$&#xA;\begin{align*}&#xA; \left\lbrack &#xA;   \begin{array}{cc}&#xA;   -i \omega  M_1&amp;amp; S\\&#xA;   -S^\mathrm{T} &amp;amp;  -i\omega M_2+ M_3&#xA;   \end{array}&#xA; \right\rbrack&#xA;\left\lbrack&#xA;  \begin{array}{c}&#xA;U\\Q&#xA;\end{array}&#xA;\right\rbrack = \left\lbrack \begin{array}{c} 0\\B \end{array} \right\rbrack.&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $M_1$ and $M_2$ were both $0$ this would be a saddle point problem and if $M_3$ was zero this would be a Helmholtz problem.  I am interested in solving this equation over a range of values of $\omega$, so reusing decompositions might be useful. This system comes from a Telegrapher's equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am writing to inquire if there is a body of research on these types of problem.  I know the research for the saddle point problems and Helmholtz problem is pretty extensive.  Any ideas or suggestions are appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I am planning on doing, without getting a better idea from here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Writing $U= U_R + i U_I$ and $Q= Q_R + i Q_I$ and rewriting the system by equating real and imaginary parts we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\lbrack&#xA;\begin{array}{cccc}&#xA;  -\omega M_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; S\\&#xA;  0 &amp;amp; \omega M_1&amp;amp; S&amp;amp;0\\&#xA; 0&amp;amp; -S^\mathrm{T}&amp;amp; -\omega M_2 &amp;amp; M_3\\&#xA;  -S^{\mathrm{T}}&amp;amp;0&amp;amp;M_3&amp;amp; \omega M_2&#xA;\end{array}&#xA;\right\rbrack&#xA;\left\lbrack&#xA;\begin{array}{c}&#xA;  U_R\\&#xA;  U_I\\&#xA;  Q_R\\&#xA;  Q_I&#xA;\end{array}&#xA;\right\rbrack&#xA;= &#xA;\left\lbrack &#xA;\begin{array}{c}&#xA; 0\\&#xA; 0\\&#xA; 0\\&#xA; B&#xA;\end{array}&#xA;\right\rbrack.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eliminating $U_I$ and $U_R$ from these we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \left\lbrack&#xA;    \begin{array}{cc}&#xA;    \omega M_3 &amp;amp;&#xA;       S^\mathrm{T} M_1^{-1} S - \omega^2 M_2\\&#xA;       S^\mathrm{T} M_1^{-1} S - \omega^2 M_2&amp;amp; \omega M_3&#xA;    \end{array}&#xA;  \right\rbrack&#xA;  \left\lbrack&#xA;    \begin{array}{c}&#xA;      Q_R\\&#xA;      Q_I&#xA;    \end{array}&#xA;  \right\rbrack&#xA;  = &#xA;  \left\lbrack&#xA;  \begin{array}{c}&#xA;    \omega B\\0&#xA;  \end{array}&#xA;  \right\rbrack.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we have a standard saddle point problem.  I plan on solving it using a Schur complement (inner/outer CG solves) method.  Note that I will use the $\omega M_3$ matrices as &quot;pivots&quot; because there exists $\omega$ for which &#xA;$S^\mathrm{T} M_1^{-1} S - \omega^2 M_2$  is singular.  Having obtained $Q_I$ and $Q_R$ we can use the eliminated equations to recover $U_I$ and $U_R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't like about this approach is that there is no reuse.  I mean the method doesn't achieve any economies of scale even though it will solve many closely related problems.&lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2015-10-04T16:06:15.760" LastActivityDate="2015-10-05T14:32:17.270" Title="What is a good way to solve the following linear system? (repeatedly)" Tags="|linear-algebra|linear-solver|helmholtz-equation|" AnswerCount="1" CommentCount="10" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20906" PostTypeId="1" AcceptedAnswerId="20912" CreationDate="2015-10-01T23:43:49.987" Score="4" ViewCount="545" Body="&lt;p&gt;I know that spectral-methods which are weighted-residual methods can be applied to solve for the incompressible N-S equations. In particular, they are applied to Direct Numerical Simulations (DNS) for studying turbulence. However, I'm still unclear how the continuity condition is enforced in such cases, and what trial functions are chosen.&lt;/p&gt;&#xA;" OwnerUserId="15275" LastActivityDate="2015-10-03T16:25:44.887" Title="How are Spectral Methods applied to CFD? In particular, how is the pressure-velocity coupling implemented?" Tags="|fluid-dynamics|spectral-method|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20907" PostTypeId="2" ParentId="20901" CreationDate="2015-10-02T00:29:40.613" Score="6" Body="&lt;p&gt;You can just change variables.  Setting $a=log(x)$, $b(a)=log(y(x))$. The integral becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F(r)=\int^{log(r)}_{-\infty} exp(a+b) da$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to be a little careful because you are integrating from $-\infty$.  What you have to do exactly will depend on what $y(x)$ looks like.&lt;/p&gt;&#xA;" OwnerUserId="17532" LastActivityDate="2015-10-02T00:29:40.613" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20908" PostTypeId="2" ParentId="20901" CreationDate="2015-10-02T01:58:57.703" Score="4" Body="&lt;p&gt;I do not use python, but if I understand correctly then by &#xA;$$&#xA;F(r)=\int_0^ry(x)dx&#xA;$$&#xA;you are thinking something like&#xA;$$&#xA;\mathbf{F}=\mathrm{integrate}(\mathbf{y},\mathbf{x})&#xA;$$&#xA;where $\mathbf{F}=[F_1,...,F_n]$ is a vector sampling the integral over a grid $\mathbf{x}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However you do not have samples of $x$ and $y$, but rather you have samples of $\hat{x}=\log(x)$ and $\hat{y}=\log(y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the simplest approach would be&#xA;$$&#xA;\mathbf{F}=\mathrm{integrate}(\exp(\mathbf{\hat{y}}),\exp(\mathbf{\hat{x}})) ,&#xA;$$&#xA;but this would be error-prone, because $y(x)$ is not smooth, even though $\hat{y}(\hat{x})$ is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the &lt;a href=&quot;https://en.wikipedia.org/wiki/Trapezoidal_rule&quot; rel=&quot;nofollow&quot;&gt;trapezoidal rule&lt;/a&gt; essentially assumes your input $y(x)$ is piecewise linear. So the simple generalization would be for you to assume that $\hat{y}(\hat{x})$ is piecewise linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, defining $\Delta F_k=F_{k+1}-F_k$, you have&#xA;$$&#xA;\Delta F_k=\int_{x_k}^{x_{k+1}}y(x)dx&#xA;=\int_{\hat{x}_k}^{\hat{x}_{k+1}}e^{\hat{y}(\hat{x})}e^\hat{x}d\hat{x}&#xA;=\int_{\hat{x}_k}^{\hat{x}_{k+1}}\tilde{y}(\hat{x})d\hat{x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, defining $t=(\hat{x}-\hat{x}_k)/\Delta \hat{x}_k$, you have&#xA;$$&#xA;\hat{y}_{k+t} \approx \hat{y}_k + t \Delta \hat{y}_k&#xA;$$&#xA;and $\tilde{y}(t) \approx a e^{bt}$, with $a=e^{\hat{y}_k+\hat{x}_k}$ and $b=\Delta \hat{y}_k+\Delta \hat{x}_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the integral becomes&#xA;$$&#xA;\Delta F_k \approx a \Delta \hat{x} \int_0^1e^{bt}dt &#xA;= a \Delta \hat{x} \frac{e^b-1}{b}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Matlab this would look something like  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dlogx=diff(logx); dlogy=diff(logy); k=1:length(logx)-1;  &#xA;b=dlogx+dlogy; a=exp(logx+logy);  &#xA;dF=a(k).*dlogx.*(exp(b)-1)./b;  &#xA;F=cumsum([0,dF]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Edit: My answer is essentially the same as the much more concise answer that Damascus Steel gave as I was typing. The only difference is I attempted to give a particular solution for the case where the &quot;particular $y(x)$&quot; is a piecewise-linear $\hat{y}(\hat{x})$ discretized over a discrete $\mathbf{\hat{x}}$ mesh, with $F(\hat{x}_1)=0$.)&lt;/p&gt;&#xA;" OwnerUserId="17742" LastEditorUserId="17742" LastEditDate="2015-10-02T02:09:07.877" LastActivityDate="2015-10-02T02:09:07.877" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20909" PostTypeId="2" ParentId="20389" CreationDate="2015-10-02T03:56:13.490" Score="3" Body="&lt;p&gt;In Matlab nested loops can be slow, and commonly a &quot;vectorized&quot; solution that eliminates loops will be faster. Slow looping used to be the norm in Matlab, but this is less true since MathWorks introduced their &quot;Just-In-Time compiler&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, vectorized solutions can &lt;em&gt;still&lt;/em&gt; be faster for some calculations, because many built-in Matlab functions are multi-core (i.e. &lt;em&gt;automatically run in parallel&lt;/em&gt;, when it is efficient to do so).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing the exact details of your problem I cannot say for sure, but here is an approach which might help. This is essentially the &quot;Integral Image&quot; method to efficiently compute moving-averages, with run-time independent of the filter size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, in 1D, say you have a grid $\mathbf{x}=[x_1,...x_{n+1}]$, and a &lt;em&gt;vectorized&lt;/em&gt; function $f=@(x)...$ , and you want to compute integrals of $f$ over the cells, i.e. $I_k=\int_{x_k}^{x_{k+1}}f(x)dx$. Then a simple way is sample $f$ at the cell edges, $\mathbf{f}=f(\mathbf{x})$, and integrate assuming $\mathbf{f}$ is piecewise linear:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f=fnc(x); dx=diff(x); fmid=conv(f,[1,1],'valid')/2; I=fmid.*dx;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is just the trapezoid rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To increase accuracy, you may need to sample $f$ at more than just the end-points of each cell. The trick is then to sample $f$ over a finer grid, but sub-sample the cumulative-mass (CDF). This is as above, but changed slightly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F=cumsum([0,fmid.*dx]); I=diff(F(1:lag:end));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where for example &quot;lag&quot; could be 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2D, this approach would be something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f=bsxfun(fnc,x,y'); dA=diff(x)*diff(y)'; fmid=conv2(f,ones(2),'valid')/4;&#xA;F=cumsum(cumsum(padarray(fmid.*dA,[1,1],'pre')),2);&#xA;I=diff(diff(F(1:lag:end,1:lag:end)),[],2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2015-10-02T03:56:13.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20911" PostTypeId="2" ParentId="20906" CreationDate="2015-10-02T19:46:48.070" Score="2" Body="&lt;p&gt;Frequently &lt;a href=&quot;https://en.wikipedia.org/wiki/Projection_method_%28fluid_dynamics%29&quot; rel=&quot;nofollow&quot;&gt;some sort of velocity-pressure splitting&lt;/a&gt; is employed which gives a step in the method where the velocity and pressure are coupled directly.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-10-02T19:46:48.070" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20912" PostTypeId="2" ParentId="20906" CreationDate="2015-10-03T14:39:58.467" Score="3" Body="&lt;p&gt;For incompressible spectral DNS, often you'll see reference to &lt;a href=&quot;http://dx.doi.org/10.1017/S0022112087000892&quot; rel=&quot;nofollow&quot;&gt;John Kim, Parviz Moin and Robert Moser (1987): Turbulence statistics in fully developed channel flow at low Reynolds number, Journal of Fluid Mechanics, 177, pp 133-166&lt;/a&gt;, which includes details on how continuity is enforced.  You might also find the book &lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-3-642-84108-8&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Spectral Methods in Fluid Dynamics&lt;/em&gt; by Canuto et al., Springer 1988&lt;/a&gt; of interest.&lt;/p&gt;&#xA;" OwnerUserId="327" LastEditorUserId="1804" LastEditDate="2015-10-03T16:25:44.887" LastActivityDate="2015-10-03T16:25:44.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20913" PostTypeId="2" ParentId="20114" CreationDate="2015-10-03T14:54:23.237" Score="6" Body="&lt;p&gt;Initially the Question focused on a product computation, &lt;code&gt;f(x)*g(x)&lt;/code&gt;, where one factor is very small and the other is very large (perhaps large enough to cause overflow in floating point representation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I commented that it might avoid overflow (resp. underflow) if instead we computed (numerical approximations to) $\ln f(x)$ and $\ln g(x)$ and added them.  In exact computation, assuming both $f(x), g(x) \gt 0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) g(x) = \exp ( \ln f(x) + \ln g(x) ) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In floating point computation the relative precision of &lt;code&gt;f(x)*g(x)&lt;/code&gt; would then correspond to the &lt;em&gt;absolute&lt;/em&gt; precision of &lt;code&gt;log(f(x)) + log(g(x))&lt;/code&gt;.  In this sense we have only moved the problem around.  As the OP noted in Comments, the numerical evaluation of $f(x)$ might be exactly zero, despite the true value being only a very small non-zero number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the OP notes, we might have $f(x) = \operatorname{erf}(a) - \operatorname{erf}(x)$ when $x,a$ are both large.  In this case both error function values rapidly approach $1$ from below.  A reasonable library implementation will return exactly $1$ for all &lt;a href=&quot;https://en.wikipedia.org/wiki/Error_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;sufficiently large calling arguments&lt;/a&gt;. Thus the difference $f(x)$ will be naively computed as exactly zero, a clear illustration of &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_of_significance&quot; rel=&quot;nofollow noreferrer&quot;&gt;catastrophic cancellation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oT58F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oT58F.png&quot; alt=&quot;Error function from Wikipedia&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Fig. 1: Graph of error function, from &lt;a href=&quot;https://en.wikipedia.org/wiki/File:Error_Function.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikimedia&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case the root cause of loss of significance is well understood, and in coding we can arrange a less naive evaluation of $f(x)$, one that allows us to accurately evaluate $\ln f(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First note the definition of the &lt;em&gt;complementary error function&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \operatorname{erfc}(x) = 1 - \operatorname{erf}(x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large $x$ we have $\operatorname{erfc}(x)$ close to zero (instead of $\operatorname{erf}(x)$ being close to one).  Consequently:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) = \operatorname{erfc}(x) - \operatorname{erfc}(a) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;avoids catastrophic cancellation (unless $x,a$ are especially close).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As coders we do have responsibility for keeping track of the sign of $f(x)$, since we can only take a logarithm of a positive real number.  This task is easy here, since $\operatorname{erf}(x)$ is monotone increasing, so $f(x) \gt 0$ if and only if $x \lt a$.  Should $x = a$, then $f(x) = 0$ and the product $f(x) g(x) = 0$ as well.  If $x \gt a$, then $f(x) \lt 0$, but we can account for this and work with $\ln |f(x)|$ to compute $\ln |f(x) g(x)|$, fixing up the sign of $f(x) g(x)$ accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a further precaution we will likely want to guard against &lt;em&gt;underflow&lt;/em&gt; in evaluating $\operatorname{erfc}(x)$.  The exponent range for double precision normal values goes down to binary (power of two) $-1022$.  Asymptotically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \operatorname{erfc}(x) \approx O(1/x) e^{-x^2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently we would be at the brink of exponent underflow when $x \approx 25$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid this some mathematical libraries (&lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/erfcx.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MatLab&lt;/a&gt;,&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.special.erfcx.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciPy&lt;/a&gt;) implement the &lt;em&gt;scaled complementary error function&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \operatorname{erfcx}(x) = e^{x^2} \operatorname{erfc}(x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this special function is not available in your library of choice, some self-implementation ideas are &lt;a href=&quot;https://stackoverflow.com/questions/8962542/is-there-a-scaled-complementary-error-function-in-python-available&quot;&gt;kicked around here&lt;/a&gt;, especially Steven G. Johnson's &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/Faddeeva_Package&quot; rel=&quot;nofollow noreferrer&quot;&gt;Faddeeva Package implementation notes&lt;/a&gt; and its algorithms references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Successful handling of this special case (difference of two error function values) does not lead to immediate resolution of all problems with numerical accuracy!  It merely demonstrates that when a cause of catastrophic cancellation is foreseen, we have an opportunity to &quot;code around&quot; it.  Sadly it is often quite time consuming to work out the alternative evaluation and (especially) to test the various paths of computation to ensure consistency.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-10-03T15:04:33.537" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20915" PostTypeId="1" CreationDate="2015-10-03T22:06:45.790" Score="11" ViewCount="448" Body="&lt;p&gt;&lt;strong&gt;Problem statement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented geometric multigrid for $-\nabla^{2}=f$ where $f=\frac{3\pi^{2}}{4}sin \frac{\pi x}{2} sin \frac{\pi y}{2} sin \frac{\pi z}{2}$ on $\Omega \in [0,1]$ on a &lt;strong&gt;unit&lt;/strong&gt; cube. Dirichlet boundaries on the left face, bottom face and front face is &lt;strong&gt;0&lt;/strong&gt;. Neumann boundaries on the top, right and back face are $\frac{\partial u }{\partial n} = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A multigrid method is used to solve the equation. I approximate the ghost points at the Neumann boundary using the &lt;strong&gt;central difference&lt;/strong&gt; formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Method overview&lt;/em&gt; (from comments, confirmed by the author):&#xA;Start from the fine mesh (the final mesh the equation to be solved with), proceed to the coarser mesh to compute the correction, propagate it back and smooth at the end of the multigrid procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Observations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is when I fix my coarsest grid (say to &lt;code&gt;16x16x16&lt;/code&gt;) and measure V-cycles for &lt;em&gt;increasing fine grid sizes&lt;/em&gt;, my V-cycles are &lt;em&gt;not constant&lt;/em&gt;. I read in the book &lt;strong&gt;MULTIGRID&lt;/strong&gt; by &lt;em&gt;Trottenberg et. al&lt;/em&gt;. that we need to use a &lt;strong&gt;modified Full Weighted restriction&lt;/strong&gt; operator to prevent incorrect scaling at Neumann boundaries. Further, I am unable to understand &lt;em&gt;this&lt;/em&gt; modified full restriction operator mentioned in the book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In another example where I implemented a mixed Dirichlet-Neumann problem $-\nabla^{2}=0$ where $u = 1 + x + y + z$ at Dirichlet boundaries I did &lt;strong&gt;&lt;em&gt;not&lt;/em&gt;&lt;/strong&gt; need to use this modified operator for convergence (fixed coarsest grid and increasing finest grid, the V-cycles remained constant).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could the &quot;modified Full Weighted restriction&quot; be causing a deterioration in the convergence rate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please suggest/explain.&lt;/p&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="20688" LastEditDate="2018-03-20T17:36:46.060" LastActivityDate="2021-11-30T01:03:20.403" Title="Increasing V-cycles for constant Coarsest Grid Size and increasing Fine Grid size" Tags="|poisson|multigrid|elliptic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20916" PostTypeId="1" CreationDate="2015-10-04T04:18:32.623" Score="-1" ViewCount="68" Body="&lt;p&gt;The program should first read from the keyboard the values of a and b then estimate the area under f(x) in&#xA;the interval [a, b] using Simpson’s 1/3rd rule and display the estimated area and the error in the estimation. Which that f(x)=sqrt(x+1)+0.25,&#xA;I have an syntax error in data array and this function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float f(float data[])&#xA;{&#xA;    return(sqrt(data[]+1)+0.25);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The rest of my code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include&amp;lt;iostream&amp;gt;&#xA;#include&amp;lt;cmath&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;//prototype function&#xA;float GenerateData(float a ,float b , int n, float data[]);&#xA;float f(float data[]);&#xA;float Simpson(float a, float b, int n,float data[],float I, float J,float simpson);&#xA;float g(float data[]);&#xA;&#xA;//area function&#xA;float f(float data[])&#xA;{&#xA;    return(sqrt(data[]+1)+0.25);&#xA;}&#xA;&#xA;// function after ingreation&#xA;//float g(float data[])&#xA;//{&#xA;//    return((2/3)*sqrt(pow(data[]+1),3)+(0.25*data[])+n);&#xA;//}&#xA;&#xA;int main()&#xA;{&#xA;    //decleration&#xA;    float a,b,simpson;&#xA;    float data[100], I =0, J=0 ;&#xA;    int n;&#xA;&#xA;    //input a b and interval&#xA;    cout&amp;lt;&amp;lt;&quot;given f(x)=sqrt(x+1)+0.25 &quot;&amp;lt;&amp;lt;endl;&#xA;&#xA;    if (a&amp;gt;b)&#xA;    {&#xA;        cout&amp;lt;&amp;lt;&quot;error&quot;;&#xA;    }&#xA;    cout&amp;lt;&amp;lt;&quot;please enter lower limit &quot;;&#xA;    cin&amp;gt;&amp;gt;a;&#xA;    cout&amp;lt;&amp;lt;&quot;please enter upper limit &quot;;&#xA;    cin&amp;gt;&amp;gt;b;&#xA;&#xA;    cout&amp;lt;&amp;lt;&quot;please enter the number of intervals ,even number&quot;;&#xA;    cin&amp;gt;&amp;gt;n;&#xA;    if ( n%2!=0)&#xA;    {&#xA;        cout&amp;lt;&amp;lt;&quot;error&quot;;&#xA;    }&#xA;&#xA;    //output the estimating area&#xA;    cout&amp;lt;&amp;lt; &quot;the estimating area using simpson's rule &quot;&amp;lt;&amp;lt;Simpson( a,  b,  n, data[], I, J,simpson);&#xA;&#xA;    //output the exact area&#xA;    //cout&amp;lt;&amp;lt;&quot;the exact area &quot;&amp;lt;&amp;lt;&#xA;&#xA;    //output the error&#xA;    //cout&amp;lt;&amp;lt;&quot;The Total Error is : &quot;&amp;lt;&amp;lt;endl;&#xA;&#xA;    return 0;&#xA;}&#xA;&#xA;&#xA;&#xA;&#xA;//function generate data&#xA;float GenerateData(float a ,float b , int n,float data[])&#xA;{&#xA;    float xi;&#xA;    //using loop&#xA;    for(int i=0;i&amp;lt;n+1;i++)&#xA;    {&#xA;        data[i]=xi=a+i*(b-a/n);&#xA;    }&#xA;&#xA;    return data[];&#xA;}&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;// function simpson's rule&#xA;float Simpson(float a, float b, int n,float data[],float I, float J,float simpson)&#xA;{&#xA;    float A;&#xA;    //loop&#xA;    for(int i=1;i&amp;lt;n;i++)&#xA;    {   &#xA;        if (i%2!=0)&#xA;        {       &#xA;            I=I+f(data[i]);&#xA;        }&#xA;    }&#xA;&#xA;    //loop&#xA;    for(int i=2;i&amp;lt;n-1;i++)&#xA;    {&#xA;        if (i%2==0)&#xA;    {       &#xA;        J=J+f(data[]);&#xA;    }&#xA;}&#xA;&#xA;  simpson=(b-a/n*3)*(f(a)+(4*I)+(2*J)+f(b));&#xA;  cout&amp;lt;&amp;lt;&quot;The Value of integral under the enterd limits is by using simpson's rule: &quot;&amp;lt;&amp;lt;endl;&#xA;  cout&amp;lt;&amp;lt;simpson&amp;lt;&amp;lt;endl;&#xA;&#xA;  return A;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17759" LastEditorUserId="7567" LastEditDate="2015-10-04T14:24:33.837" LastActivityDate="2015-10-04T17:24:59.380" Title="I have a problem in finding the exact area and estimating the error" Tags="|calculus|" AnswerCount="1" CommentCount="3" ClosedDate="2015-10-05T04:29:12.740" ContentLicense="CC BY-SA 3.0" />
  <row Id="20917" PostTypeId="2" ParentId="20916" CreationDate="2015-10-04T04:58:47.353" Score="1" Body="&lt;p&gt;The following is not valid C:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;return(sqrt(data[]+1)+0.25);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think what you want is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float f(float data[], int n)&#xA;{&#xA;  float fx = 0.0;&#xA;  for(int i=0;i&amp;lt;n;i++){&#xA;    fx += sqrt(data[i]+1)+0.25;&#xA;  }&#xA;&#xA;  return fx;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the integer input n is the size of your data array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On re-reading the question I think what you want is the following function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float f(float x)&#xA;{&#xA;  return sqrt(x+1)+0.25;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where f is called in your simpson function body (and elsewhere) like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(int i=1;i&amp;lt;n;i++){&#xA;   if(i%2!=0){&#xA;     I = I + f(data[i]);&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note 1: There may be problems with your understanding of how to implement simpsons rule - which would make this question on-topic - however I think you need to first get a better understanding of how to write C code before posting a question on this stack exchange. Currently with the improper c code it makes it hard to distinguish if you are having difficulty with understanding how to implement simpsons rule or if its just more coding experience that you need. Good luck and I hope these hints help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 2: You can find the exact area because the integral of the function f(x)=sqrt(x+1)+0.25 can be computed (pen &amp;amp; paper) exactly. Once you have your exact area and your approximate numerical area they can be compared to find the error in your approximation. The exact area is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ Area_{exact} = \int_{a}^{b}\sqrt{x+1}+0.25 dx = (\frac{2}{3}(x+1)^{3/2}+0.25x)|_{a}^{b}$ &lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-10-04T17:24:59.380" LastActivityDate="2015-10-04T17:24:59.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20919" PostTypeId="1" CreationDate="2015-10-04T22:11:39.480" Score="1" ViewCount="298" Body="&lt;p&gt;Suppose I have this set of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a = x + z\qquad (1)$$&#xA;$$b = y + \frac{z}{2}\qquad (2)$$&#xA;$$ z = k_0x\sqrt{y}\qquad (3)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $a$, $b$ $\in \mathbb{R}$ and $k_0 &amp;gt; 0$. The values of $a$ and $b$ assume that neither x nor y can be less than zero, how do I robustly solve this system? I have attempted the Newton-Raphson method and got the following residual and Jacobian:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R(x,y) = \begin{Bmatrix}x+z-a \\y+\frac{z}{2}-b \\ k_0x\sqrt{y} - z\end{Bmatrix} $$&#xA;$$J(x,y) = \begin{bmatrix}1 &amp;amp;&amp;amp; 0 &amp;amp;&amp;amp; 1 \\ 0 &amp;amp;&amp;amp; 1 &amp;amp;&amp;amp; \frac{1}{2} \\ \frac{k_0\sqrt{y}}{2} &amp;amp;&amp;amp; \frac{k_0x}{2\sqrt{y}} &amp;amp;&amp;amp; -1 \end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and here's the MATLAB code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% User-defined inputs&#xA;a = 1;&#xA;b = 0;&#xA;k0 = 10;&#xA;&#xA;% Initial guess&#xA;%u = [1;1;1];&#xA;u = rand(3,1)*2;&#xA;&#xA;% Newton-Raphson solver&#xA;i = 0; max_iters = 30; TOL = 1e-12;&#xA;fprintf('a = %f\nb = %f\nk0 = %f\nInitial guess: (%f,%f,%f)\n\n\tResidual_x\tResidual_y\tResidual_z\tResidual_rel\n',a,b,k0,u(1),u(2),u(3));&#xA;while (true)&#xA;&#xA;    % Residual function&#xA;    F = [u(1)+u(3)-a;u(2)+0.5*u(3)-b;k0*u(1)*sqrt(u(2))-u(3)];&#xA;&#xA;    % Initial residual&#xA;    if i == 0&#xA;        R_init = norm(F);&#xA;    % Maximum iterations&#xA;    elseif i == max_iters&#xA;        fprintf('\tSolution did not converge.\n');&#xA;        break;&#xA;    end&#xA;&#xA;    % Absolute residual&#xA;    R = norm(F);&#xA;&#xA;    % Relative residual&#xA;    R_rel = R/R_init;&#xA;    fprintf('\t%e\t%e\t%e\t%e)\n',norm(F(1)),norm(F(2)),norm(F(3)),R_rel);&#xA;&#xA;    % Check for convergence&#xA;    if (R_rel &amp;lt; TOL)&#xA;        fprintf('\tConverged.\n');&#xA;        break;&#xA;    end&#xA;&#xA;    % Jacobian matrix&#xA;    J = [1,0,1;0,1,0.5;u(1)*k0*sqrt(u(2)),0.5*k0*u(1)/sqrt(u(2)),-1];&#xA;&#xA;    % Solve and update&#xA;    u = u-J\F;&#xA;    i = i + 1;&#xA;end&#xA;fprintf('\nF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it seems this does not always work. If for instance at iterate $i$ I have $y^i = 0$, the solver blows up (because $\frac{1}{\sqrt{0}}$ results in an error). It seems the initial guesses also seems to play a significant role because if you run the above code with randomized guesses you will get a different solution each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, how else (or how better) can I solve what I have above? Or if I stick with this Newton-Raphson scheme, how should I choose my initial guesses?&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2015-10-05T01:48:47.267" Title="Solving this nonlinear system of equations" Tags="|newton-method|nonlinear-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20920" PostTypeId="2" ParentId="20919" CreationDate="2015-10-05T01:37:26.707" Score="2" Body="&lt;p&gt;This can be easily reduced to a sequence of one-variable equations with the help of sage, or any other CAS. (&lt;em&gt;I replaced $\sqrt{y}\mapsto y$ to simplify things&lt;/em&gt;, the only thing that changes is that below only solutions with $y\geq0$ would be valid.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R = PolynomialRing(QQ, 'a,b,k,x,y,z', order='invlex')&#xA;a,b,k,x,y,z = R.gens()&#xA;I = R.ideal(x+z-a, y*y+z/2-b, k*x*y-z)&#xA;# should be 3, meaning finitely many solutions for each set of parameters&#xA;print(I.dimension()) &#xA;I.groebner_basis()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;\big[&amp;amp;z + x -  a,&#xA;\\&amp;amp;y^{2} - \tfrac{1}{2} x -  b + \tfrac{1}{2} a,&#xA;\\&amp;amp;x y -  a y + \tfrac{1}{2} k x^{2} + b k x - \frac{1}{2} a k x,&#xA;\\&amp;amp;a k y - \frac{1}{2}k^{2} x^{2} -  b k^{2} x + \tfrac{1}{2} a k^{2} x + x -  a,&#xA;\\&amp;amp;k^{2} x^{3} + 2 b k^{2} x^{2} -  a k^{2} x^{2} - 2 x^{2} + 4 a x - 2 a^{2}\big]&#xA;\end{aligned}&#xA;$$&#xA;Thus $x$ has to satisfy the cubic equation&#xA;$$ -2 a^2+4 a x-(2+a k^2-2 b k^2) x^2+k^2 x^3 = 0 $$&#xA;(The other command to try is &lt;code&gt;I.elimination_ideal([y,z])&lt;/code&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on $a,k$, there are at most three possible values for $x$. For each possible value of $x$, you can then solve $a=x+z$, followed by the linear equation $y=z/(kx)$, giving at most three distinct solutions. Each solution can then be checked for validity and discarded if necessary.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-10-05T01:48:47.267" LastActivityDate="2015-10-05T01:48:47.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20921" PostTypeId="1" CreationDate="2015-10-05T03:25:12.057" Score="0" ViewCount="561" Body="&lt;p&gt;I have been learining the NURBS theory by the classical textbook &amp;quot;&lt;em&gt;The NURBS Book&lt;/em&gt;&amp;quot; this year. In the chapter 9, the author introduced the method of non-rational B-spline curve interpolation with a &lt;strong&gt;open curve&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have implemented this global B-spline curve interpolation algorithm with the Wolfram &lt;em&gt;Mathematica&lt;/em&gt;. About the details, please see &lt;a href=&quot;http://demonstrations.wolfram.com/preview.html?draft/69841/000034/GlobalBSplineCurveInterpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://mathematica.stackexchange.com/questions/40163/arbitrary-precision-spline-interpolation/84717#84717&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In addition, I know how to generate a closes B-spline curve by the control points. Please see my answer for &lt;a href=&quot;https://mathematica.stackexchange.com/questions/92731/how-to-generate-a-closed-b-spline-curve/92824#92824&quot;&gt;closes B-spline curve&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now let me give a simple example with &lt;em&gt;Mathematica&lt;/em&gt;, I add the option &lt;code&gt;SplineClosed -&amp;gt; True&lt;/code&gt; in the&#xA;&lt;code&gt;BSplineCurve&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;searchSpan[knots_, u0_] :=&#xA; With[{max = Max[knots]},&#xA;  If[u0 == max, Position[knots, max][[1, 1]] - 2,&#xA;   Ordering[UnitStep[u0 - knots], 1][[1]] - 2]&#xA; ]&#xA;&#xA;Options[BSplineInterpolation] = {SplineDegree -&amp;gt; Automatic};&#xA;&#xA;BSplineInterpolation[pts : {{_, _} ..}, opts : OptionsPattern[]] /;&#xA; MatrixQ[pts, NumericQ] :=&#xA; Module[{n, md, sd, paras, knots, coeffMat, ctrlpts},&#xA;  n = Length@pts - 1;&#xA;  sd = OptionValue[SplineDegree] /. Automatic -&amp;gt; 3 /. &#xA;    deg_ :&amp;gt; n /; deg &amp;gt; n;&#xA;  paras =&#xA;   FoldList[Plus, 0, Normalize[(Norm /@ Differences[pts]), Total]] // N;&#xA;  (*calculate the knots*)&#xA;  knots =&#xA;   Join[ConstantArray[0, sd + 1],&#xA;     1/sd (Plus @@ (paras[[# + 1 ;; # + sd]]) &amp;amp; /@ Range[1, n - sd]),&#xA;     ConstantArray[1, sd + 1]] // N;&#xA;  (*calculate the coefficients of matrix*)&#xA;  coeffMat = Function[{u0},&#xA;     With[{i = searchSpan[knots, u0]},&#xA;      Join[ConstantArray[0, i - sd],&#xA;       BSplineBasis[{sd, knots}, #, u0] &amp;amp; /@ Range[i - sd, i],&#xA;       ConstantArray[0, n - i]]]] /@ paras;&#xA;  (*solve the control points of B-Spline curve*)&#xA;  ctrlpts = LinearSolve[coeffMat, pts];&#xA;  (*visualize the result*)&#xA;  Graphics[&#xA;   {BSplineCurve[ctrlpts,&#xA;     SplineClosed -&amp;gt; True, SplineDegree -&amp;gt; sd, SplineKnots -&amp;gt; knots],&#xA;    Red, PointSize[Medium], Point[pts]}]&#xA;  ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Test&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pts = &#xA; {{-1.5, -2}, {-3, -1}, {-2.7, 0.5}, {-1.75, 1.3}, {0.8, -1.5}, &#xA;  {1.5, 0.4}, {0, 2}, {3, 2}};&#xA;BSplineInterpolation[pts]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pDbJu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Obviously, this curve is &lt;strong&gt;not&lt;/strong&gt; continuous and it &lt;strong&gt;doesn't&lt;/strong&gt; pass all the &lt;code&gt;interpolation points&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;another trial(Namely, deleteing the option &lt;code&gt;SplineKnots -&amp;gt; knots&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/mHLLF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&lt;p&gt;I have searched the paper about closed curve interpoaltion by the &lt;strong&gt;Google Scholar&lt;/strong&gt;, however, I didn't discovered any helpful reference.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How to achieve a continuous closed B-spline curve?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14845" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2021-04-25T05:41:07.483" Title="How to interpolate a set of points with a continuous closed B-spline curve?" Tags="|interpolation|mathematica|b-spline|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20923" PostTypeId="1" AcceptedAnswerId="20954" CreationDate="2015-10-05T04:26:18.387" Score="4" ViewCount="145" Body="&lt;p&gt;In a trust region based Newton method, a number of constants are given as inputs to the algorithm that determine the updating rules for the trust region bound. Are these constants chosen arbitrarily or are there some rules that govern these constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance refer to this paper &lt;a href=&quot;http://www.csie.ntu.edu.tw/~cjlin/papers/logistic.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.csie.ntu.edu.tw/~cjlin/papers/logistic.pdf&lt;/a&gt;&#xA;On page 5 (631 in the article) there are a set of equations marked as 10, that determine the trust region bound. In these set of three equations there are 5 constants being used. Though the values of these constants used by the writers are specified on page 9 (635 in the article) however the choice seems to be rather arbitrary and I couldn't find any general rules to set these parameters.&lt;/p&gt;&#xA;" OwnerUserId="17761" LastActivityDate="2015-10-07T12:47:01.570" Title="Trust region - Newton: how to choose constants that determine trust region bound" Tags="|linear-algebra|optimization|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20925" PostTypeId="1" AcceptedAnswerId="20926" CreationDate="2015-10-05T09:52:10.713" Score="4" ViewCount="269" Body="&lt;p&gt;For a large matrix $A$, I need to evaluate the $\log(\det(A))$. I already have it's LDLT decomposition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to evaluate the  $\log\det$ with the elements of the diagonal $D$ of the LDLT decomposition?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16909" LastActivityDate="2015-10-05T12:31:30.773" Title="Does $\log(\det(A))$ equals sum of log of diagonal elements of D in LDLT decomposition?" Tags="|linear-algebra|matrix|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20926" PostTypeId="2" ParentId="20925" CreationDate="2015-10-05T12:31:30.773" Score="5" Body="&lt;p&gt;Yes, it is possible.  By construction, the $L$ matrix is lower triangular with all its diagonal entries equal to one. Therefore, $\det(L) = \det(L^T) = 1$ and, consequently, $\det(L D L^T) = \det(L) \, \det(D) \, \det(L^T) = \det(D)$.  From this, you can conclude that $\log(\det(A)) = \log(\det(D)) = \log(\prod_{i = 1}^n D_{ii}) = \sum_{i = 1}^n \log(D_{ii})$.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-10-05T12:31:30.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20927" PostTypeId="1" CreationDate="2015-10-05T13:15:20.220" Score="2" ViewCount="1278" Body="&lt;p&gt;I need to evaluate Kummer's confluent hypergeometric function for imaginary arguments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$_1F_1(a,b;ix)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $i$ is the imaginary unit, $a,b,x$ are real, and $a,b&amp;gt;0$. Is there a routine available in C/C++? I already saw GSL, but it only computes $_1F_1$ for real arguments.&lt;/p&gt;&#xA;" OwnerUserId="988" LastActivityDate="2016-10-27T07:47:45.707" Title="Kummer's confluent hypergeometric for complex arguments in C/C++?" Tags="|c++|c|special-functions|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20928" PostTypeId="2" ParentId="20905" CreationDate="2015-10-05T13:34:54.633" Score="1" Body="&lt;p&gt;Sorry this is late, but I've been thinking about an iterative solution of this which might be an alternative. So you're solving $(\omega A+B)x = b$ for $x$ with $A$ easily invertible. &#xA;Can you make an iterative version as &#xA;$$\omega A x^{(i+1)} + Bx^{(i)} = b,$$ or equivalently &#xA;$$x^{(i+1)} = \omega^{-1}A^{-1}b - \omega^{-1}A^{-1}Bx^{(i)}.$$ You can precalculate $A^{-1}b$ and $A^{-1}B$, and as @BillBarth said, you could try the $x$ for the previous $\omega$ as your starting guess. Maybe I'm missing something.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="16799" LastEditDate="2015-10-05T14:32:17.270" LastActivityDate="2015-10-05T14:32:17.270" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20929" PostTypeId="1" CreationDate="2015-10-05T13:58:43.057" Score="0" ViewCount="4150" Body="&lt;p&gt;I have a $80\times200$ matrix, and I want to plot its contents with the contourf command in MATLAB, as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\text{[C,h] = contourf(mat,100);}\\&#xA;&amp;amp;\text{set(h,'LineColor','none');}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting image, as you can see, has some step-like discontinuities where the white part meets the coloured part. Is there any way to smooth this region with curve-fitting tool or other methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qXDSQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qXDSQ.jpg&quot; alt=&quot;...&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2015-10-05T23:42:35.243" Title="Having smoother contour plots in MATLAB" Tags="|matlab|plotting|smoothing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20930" PostTypeId="2" ParentId="20929" CreationDate="2015-10-05T14:27:23.963" Score="2" Body="&lt;p&gt;You can regard your original matrix as an image (i.e., a real-valued function $I = I(x, y)$), then you can &lt;a href=&quot;https://en.wikipedia.org/wiki/Gaussian_blur&quot; rel=&quot;nofollow&quot;&gt;apply a Gaussian filter&lt;/a&gt; (convolve $I$ with a Gaussian kernel), and finally you can get the contours of the filtered image/matrix.  MATLAB's Image Processing toolbox already &lt;a href=&quot;http://www.mathworks.com/help/images/ref/imgaussfilt.html&quot; rel=&quot;nofollow&quot;&gt;has the functionality available&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-10-05T14:27:23.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20931" PostTypeId="1" AcceptedAnswerId="20932" CreationDate="2015-10-05T15:37:22.450" Score="0" ViewCount="222" Body="&lt;p&gt;Quadratic Pseudo-Boolean Optimization (QPBO) problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem 1.&lt;/strong&gt; &lt;em&gt;Minimize $\sum_i a_ix_i + \sum_{i&amp;lt;j} a_{ij}x_i x_j$ subject to $x_i\in\{0,1\}\forall i$.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following problem, where the integral constraints are relaxed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem 2.&lt;/strong&gt; &lt;em&gt;Minimize $\sum_i a_ix_i + \sum_{i&amp;lt;j} a_{ij}x_i x_j$ subject to $0\le x_i\le 1\forall i$.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is whether there exists an efficient method that can solve Problem 2 &lt;strong&gt;exactly&lt;/strong&gt; (i.e. output an &lt;strong&gt;optimal&lt;/strong&gt; solution)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for any suggestions !&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2019-05-12T22:20:33.893" Title="Solving a quadratic pseudo-boolean optimization problem where the integral constraints are relaxed" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20932" PostTypeId="2" ParentId="20931" CreationDate="2015-10-05T15:50:47.227" Score="2" Body="&lt;p&gt;Non-convex quadratic programming problems with bounds constraints are in general NP-Hard. Thus you shouldn't expect to find a polynomial time algorithm for this problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These problems can be addressed by various heuristic approaches or by branch and bound methods (although this requires considerable computational effort.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How large are the instances that you're interested in? &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-10-05T15:50:47.227" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20933" PostTypeId="1" CreationDate="2015-10-05T17:14:13.897" Score="0" ViewCount="329" Body="&lt;p&gt;How to find $C$ such that $CC^T =  AA^T + BB^T$, $A$ and $B$ are known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A = \left(\begin{matrix}X\\Y\end{matrix}\right)$, $B = \left(\begin{matrix}0\\cY\end{matrix}\right)$, $c$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify. Cholesky decomposition can give the solution. I was wondering is there any closed form solution?&lt;/p&gt;&#xA;" OwnerUserId="17052" LastEditorUserId="17052" LastEditDate="2015-10-06T18:32:19.763" LastActivityDate="2017-09-25T01:04:01.337" Title="How to find closed form $C$ such that $CC^T = AA^T + BB^T$" Tags="|matrix|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20934" PostTypeId="2" ParentId="20933" CreationDate="2015-10-05T17:53:42.970" Score="5" Body="&lt;p&gt;If A and B are known, then the right hand side is known, and must of necessity be symmetric positive semi-definite.  Therefore, there is a solution C to the stated equation, and this is the (lower triangular) Cholesky factor of $ AA^T + BB^T$ .  Cholesky factorization is a standard routine in any linear algeabra library.  MATLAB's chol computes the upper triangular Cholesky factor, so using that, C would be chol(A * A' + B * B')' .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll leave it to others to determine whether there is a &quot;closed form&quot; solution in terms of X, Y, and c, but nothing jumps out at me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Sorry, did not mean for this to be deleted.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2015-10-06T12:15:46.417" LastActivityDate="2015-10-06T12:15:46.417" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20935" PostTypeId="1" CreationDate="2015-10-05T21:38:25.780" Score="3" ViewCount="666" Body="&lt;p&gt;I am modeling the temperature of the groundwater using heat equation. &#xA;I have Dirichlet BC at the top but at the bottom I have constant temperature equal 12 degrees C (see attached pic). It is look like at the bottom we have zero flux. When I specify the Dirichlet BC I have a &quot;knee&quot; right before &quot;neutral zone&quot;  which is not the case. So far I have 2 ideas how to implement it: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I was thinking maybe the Robyn BC will help here? &lt;/li&gt;&#xA;&lt;li&gt;Maybe I need to implement somehow the [-hh * (T - T_inf)/k] as gradient there. When we have not 12C we will have the flux and when we have 12 then we will have 0 gradient.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;what do you think? is it correct way of tackling this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zQDVO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zQDVO.jpg&quot; alt=&quot;attached image&quot;&gt;&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="11497" LastActivityDate="2015-10-06T14:25:30.753" Title="How to implement boundary conditions in heat equation with no flux and fixed value at the same time? Is it Robyn BC?" Tags="|pde|boundary-conditions|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20936" PostTypeId="2" ParentId="20929" CreationDate="2015-10-05T23:42:35.243" Score="3" Body="&lt;p&gt;Is the white part NaNs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, then you will need to use some sort of extrapolation to smooth that region.&#xA;The function &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/4551-inpaint-nans&quot; rel=&quot;nofollow&quot;&gt;inpaint_nans&lt;/a&gt; may be appropriate (it smoothly fills in NaN regions, essentially by solving a Laplace equation). If more smoothing is needed, you could then follow Juan's approach (i.e. gaussian blur).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing you might consider: I do not know what the purpose of your display is, but you might consider just using imagesc() instead of contourf(). For example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contourf(I,100); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;should be similar to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;imagesc(I); colormap(jet(100));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which just quantizes the colormap used to render the image data.&#xA;(Quantizing the colormap will typically be much faster though, compared to computing 100 contours.)&lt;/p&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2015-10-05T23:42:35.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20937" PostTypeId="2" ParentId="20935" CreationDate="2015-10-06T00:03:07.763" Score="0" Body="&lt;p&gt;Do you have variable thermal diffusivity with depth?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, then if I understand you problem correctly, there is an analytical solution (see &lt;a href=&quot;https://books.google.com/books?id=heJSAwAAQBAJ&amp;amp;pg=PA180&amp;amp;lpg=PA180&amp;amp;dq=%22seasonal+changes+in+subsurface+temperature%22&amp;amp;source=bl&amp;amp;ots=9w1Ts0TEBP&amp;amp;sig=kI3mz_7lx1sxyCMfl4h4ia3PvpM&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0CB0Q6AEwAGoVChMIm6-3yMSsyAIVF0iICh1p7gBT#v=onepage&amp;amp;q=%22seasonal%20changes%20in%20subsurface%20temperature%22&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either case, assuming I understand your problem correctly, then typically the lower boundary condition is zero heat-flux &quot;at infinity&quot; (i.e. Neumann). Here &quot;infinity&quot; means at $\frac{z_0-z}{\lambda}\gg1$, where $\lambda$ is the &quot;skin depth&quot; (see the above link). With this condition, the temperature will end up as the time-average of your surface-forcing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Your comment suggests you have a piecewise-constant diffusivity, dry soil vs. saturated soil (ignoring the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vadose_zone&quot; rel=&quot;nofollow&quot;&gt;vadose zone&lt;/a&gt;). Across a step change in diffusivity, the temperature $T$ and heat flux $q=-KT_z$ will be continuous, but the temperature gradient $Tz$ will be discontinuous. Is the kink in your results at the interface (i.e. water table)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically, piecewise constant conductivities would be handled in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_volume_method&quot; rel=&quot;nofollow&quot;&gt;finite volume method&lt;/a&gt; by assigning the interface-conductivity as the harmonic average of the two adjacent cell-conductivities when computing the flux. (These &lt;a href=&quot;http://web.cecs.pdx.edu/~gerry/class/ME448/notes/pdf/CVFDdiffusion2D.pdf&quot; rel=&quot;nofollow&quot;&gt;notes&lt;/a&gt; give an overview.)&lt;/p&gt;&#xA;" OwnerUserId="17742" LastEditorUserId="17742" LastEditDate="2015-10-06T14:25:30.753" LastActivityDate="2015-10-06T14:25:30.753" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20938" PostTypeId="2" ParentId="20935" CreationDate="2015-10-06T00:22:00.780" Score="1" Body="&lt;p&gt;You cannot (in general) both enforce zero flux &lt;em&gt;and&lt;/em&gt; constant value at the same boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, you don't need a Robin boundary condition. The boundary condition depends on what you wish to enforce. At the top, a Dirichlet condition seems natural. You have already suggested two options at the bottom: Dirichlet (constant, 12 degrees as you said) or Neumann (zero flux, as you said). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this problem, if the bottom boundary condition is a great enough distance from the top there will be only a very small difference between a zero flux Neumann condition and a Dirichlet condition at the mean value of the annual surface temperature. How far is &quot;a great enough distance&quot; depends on the skin depth as mentioned by @GeoMatt22. If you are seeing a bend just above the &quot;neutral zone&quot; with the Dirichlet condition it is probably best to switch to Neumann.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-10-06T00:45:39.733" LastActivityDate="2015-10-06T00:45:39.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20939" PostTypeId="1" CreationDate="2015-10-06T09:38:05.047" Score="6" ViewCount="287" Body="&lt;p&gt;I would like to know the career options for a computational scientist other than getting into software industry. I wish to use the my skills of Numerical Linear Algebra,Parallel Programming, Scientific Visualization,Topological Data Analysis in some Research Lab of any manufacturing company or Scientific company.&lt;/p&gt;&#xA;" OwnerUserId="17783" LastEditorUserId="20688" LastEditDate="2019-06-22T20:22:16.177" LastActivityDate="2019-06-22T20:22:16.177" Title="What are the career options for computational scientists other than being in software Industry?" Tags="|parallel-computing|career-development|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20940" PostTypeId="1" CreationDate="2015-10-06T12:05:17.753" Score="1" ViewCount="53" Body="&lt;p&gt;We have the recurrence relation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$5x_{n+1}-x_n=\frac{1}{3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_0=\frac{1}{12}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;solution:    $y_h=(\frac{1}{5})^nC$   $y_p:  5A-A=\frac{1}{3}  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A=\frac{1}{12}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y=(\frac{1}{5})^nC+\frac{1}{12}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y(0)=C=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will the answer go to &#xA;$ \frac{1}{12}$ or somewhere very close to $\frac{1}{12}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer sheet says very close to. I thought it would be to exactly $\frac{1}{12}$ because for large n the computer would read the fraction as 0 even though it is not? And how is this not right? Please also explain why the fact that for large power n on fractions like here the fact that the number would be read as 0 would not alter this as that was my reasoning.&lt;/p&gt;&#xA;" OwnerUserId="17784" LastActivityDate="2015-10-07T03:42:59.003" Title="problem about simulating recurrence relation" Tags="|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20941" PostTypeId="2" ParentId="20933" CreationDate="2015-10-06T12:56:05.317" Score="1" Body="&lt;p&gt;Substituting A and B into the equation gives you the following symmetric matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$CC^T=K=\left[\begin{matrix}XX^T &amp;amp; XY^T\\YX^T &amp;amp; (1+c^2)YY^T\end{matrix}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find $C$, you could use the following MATLAB commands:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;C = chol(K,'lower')&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Positive definiteness of $K$ should of course be ensured.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2017-09-25T01:04:01.337" LastActivityDate="2017-09-25T01:04:01.337" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20942" PostTypeId="1" AcceptedAnswerId="20947" CreationDate="2015-10-06T14:40:17.460" Score="10" ViewCount="4153" Body="&lt;p&gt;Somebody on my team wants to paralelize IPOPT. (at least some of functions of it). I have not been able to find a GPU implementation of it or a similar package. Nor I've found anything on their docs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is, is there an alternative already implemented on GPU? or at least somebody working on porting it to GPU so we can work toguether?&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="157" LastEditDate="2015-10-06T14:58:13.433" LastActivityDate="2016-03-04T05:33:59.553" Title="Is there a constrained nonlinear optimization library like IPOPT that runs on GPUs?" Tags="|nonlinear-programming|gpu|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20943" PostTypeId="1" CreationDate="2015-10-06T19:01:58.937" Score="0" ViewCount="198" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Warning: Failure at t=6.137539e-04.  Unable to meet integration&#xA;  tolerances without reducing the step size below the smallest value&#xA;  allowed (1.734723e-18) at time t.&#xA;  In ode15s (line 730)&#xA;    In khademi (line 59) &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17594" LastEditorDisplayName="user17774" LastEditDate="2015-10-07T18:41:22.180" LastActivityDate="2015-10-07T18:41:22.180" Title="What is the meaning of this error in MATLAB?" Tags="|matlab|ode|method-of-lines|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20944" PostTypeId="2" ParentId="20943" CreationDate="2015-10-06T19:17:08.250" Score="3" Body="&lt;p&gt;All of the MATLAB ode solvers adjust the integration step size to try to keep the error in the solution less than certain prescribed tolerances. If the error exceeds these tolerances, the step size is reduced until it reaches essentially zero (1.734723e-18).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually this error message arises because the problem is ill-posed. If you terminate the solution at some value of t&amp;lt;6.137539e-04 and look at the solution&#xA;at that time, there is a good chance it will be very large (i.e. the solution is&#xA;going to infinity or negative infinity).&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-10-06T19:17:08.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20945" PostTypeId="1" CreationDate="2015-09-30T09:26:41.940" Score="2" ViewCount="240" Body="&lt;p&gt;When we simulate a bulk molecular system using molecular dynamics in (say) NVE ensemble (microcanonical), we apply some periodic boundary conditions (PBC) to avoid surface effects. What effect does this have on quantities like Pair Distribution function, since it so happens that &lt;strong&gt;part of a molecule is near one edge of the box and the other is in a different edge of the box&lt;/strong&gt;. What effect does this have on PDF, MSD and other such thermodynamic quantities ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS : I am currently simulating SPC/E water molecules using cubic box, in some frames I find water molecules which are split apart, but connected by PBC.&lt;/p&gt;&#xA;" OwnerUserId="17599" OwnerDisplayName="user35952" LastActivityDate="2015-10-06T20:32:29.510" Title="Effect of Periodic Boundary condition in Pair Distribution function" Tags="|molecular-dynamics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20946" PostTypeId="1" AcceptedAnswerId="20989" CreationDate="2015-10-06T21:58:11.607" Score="3" ViewCount="485" Body="&lt;p&gt;I am simulating the sound wave propagation in non-rectangular and asymetric spaces using finite-difference method. I presume linear acoustics equations to be enough (i.e. $\Box p = 0$, $\Box \vec{v} = 0$) and I need to have open and closed boundaries of the system (e.g. node or antinode of acoustic pressure on the wall and the same for velocity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem occurs when I have to deal with the Neumann boundary conditions. The method is unstable even for dense and computationally inconvenient grids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The physics behind the maths allows 3+1 more or less straightforward possible ways how to do the simulation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Model the acoustic pressure (then the Neumann condition is needed at the open/loose boundaries: $\partial_x p = 0$ on the boundary, closed boundaries are OK)&lt;/li&gt;&#xA;&lt;li&gt;Model the velocity potential $\phi$ (not very convenient cause $\partial_x \phi = 0$ on the open boundary and $\partial_t \phi = 0$ on the closed boundary)&lt;/li&gt;&#xA;&lt;li&gt;Model the acoustic velocity (i.e. model d'Alembert wave equation for the vector quantity - not very elegant, that's why we have potential)&lt;/li&gt;&#xA;&lt;li&gt;Reformulate the physics to be e.g. hamiltonian (i.e. have only the first order derivatives), lattice Boltzmann etc. etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any suggestions or experiences please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CFL condition is satisfied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem actually might be the numerical dispersion. The picture shows a system with the source on the left oscillating from -1 to 1 sinusoidally. The values on the picture outside the source area vary between -0.1 and 0.1. On the top and bottom there are Dirichlet boundaries, there is Neumann boundary at the right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ymqS8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ymqS8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FDM with centered time and space was used. In matlab (dx=dy):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A(ii,jj,n+1)=2*A(ii,jj,n)-A(ii,jj,n-1)+c1^2*dT^2/dX^2*(A(ii-1,jj,n)+A(ii+1,jj,n)+A(ii,jj-1,n)+A(ii,jj+1,n)-4*A(ii,jj,n));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17721" LastEditorUserId="17721" LastEditDate="2015-10-12T14:01:18.907" LastActivityDate="2015-10-12T17:15:53.497" Title="Finite difference scheme for 2D sound propagation" Tags="|finite-difference|fluid-dynamics|simulation|wave-propagation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20947" PostTypeId="2" ParentId="20942" CreationDate="2015-10-06T23:33:13.613" Score="15" Body="&lt;p&gt;&lt;strong&gt;tl;dr:&lt;/strong&gt; My general impression from the literature is that speedups are modest (if they exist). The main kernel you'll see in these methods is a sparse-direct method (e.g., sparse LU, sparse LDLT), and memory accesses are irregular; these characteristics don't favor use of GPUs. Also, parallel IPMs are in their infancy. I still suspect people will work on GPU implementations, but I am skeptical much will come of them. (However, distributed-memory IPMs seem slightly more promising and useful.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A few people have worked on interior-point methods (IPMs) for GPUs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.maths.ed.ac.uk/hall/SmGoHa11/ERGO-11-008.pdf&quot; rel=&quot;noreferrer&quot;&gt;Smith, Gondzio, and Hall&lt;/a&gt; developed an IPM for linear programs (LPs) that achieved 4-10x speedups&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.128.428&quot; rel=&quot;noreferrer&quot;&gt;Jung and O'Leary&lt;/a&gt; look at some of the linear algebra kernels in IPMs for LPs and see modest speedups for GPUs on larger problems in their test set&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In general, the papers don't compare their work to highly tuned LP solvers. Jung and O'Leary compare to linprog, which wouldn't be the choice of most practitioners, and the impression I get from scanning the Smith, et al paper is that a serial interior-point method was written from scratch. Given that Hall is very involved in open-source LP solvers, I take that work slightly more seriously. One of the better open-source LP solvers out there is Clp, which Hall maintains, and if they had used that code, it would be mentioned by name. So I'd pay more attention if these codes were accelerating already highly tuned solvers, instead of one-off serial comparisons that aren't state-of-the-art.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, the existing work is for LPs, and you're probably asking for a nonlinear programming (NLP) solver, because that's what IPOPT is. Here is what I know about the NLP solver case:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;parallelizing IPMs is generally challenging (for the LP &amp;amp; SOCP case, see &lt;a href=&quot;http://libelemental.org/&quot; rel=&quot;noreferrer&quot;&gt;Elemental&lt;/a&gt;; you'd use something like the SOCP version (or a QP version) for a general IPM&lt;/li&gt;&#xA;&lt;li&gt;if your NLP doesn't have obvious structure (e.g., isn't a stochastic program, doesn't have obvious bordered-block-angular structure, isn't a PDE-constrained optimization problem where preconditioners for the PDE are well-known), the best linear algebra methods involve sparse-direct methods (e.g., LU, LDLT) with irregular memory accesses that aren't typically suited to GPU architectures (as mentioned before)&lt;/li&gt;&#xA;&lt;li&gt;if your NLP does have obvious structure, you'd want to use an inertia-free line search algorithm that doesn't require an inertia-revealing factorization (building upon &lt;a href=&quot;http://www.mcs.anl.gov/~vzavala/paperinertia.pdf&quot; rel=&quot;noreferrer&quot;&gt;recent work by Chiang and Zavala&lt;/a&gt;), so you could use known preconditioners to help solve the Karush-Kuhn-Tucker system&lt;/li&gt;&#xA;&lt;li&gt;the best parallel matrix interior-point methods available that I know of are those by Gondzio(&lt;a href=&quot;http://www.maths.ed.ac.uk/~gondzio/reports/mtxFree.pdf&quot; rel=&quot;noreferrer&quot;&gt;Implementation paper,&lt;/a&gt; &lt;a href=&quot;http://www.maths.ed.ac.uk/~gondzio/reports/inexactIPM.pdf&quot; rel=&quot;noreferrer&quot;&gt;theory paper&lt;/a&gt;), and by Waechter, et al. (&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/090747634&quot; rel=&quot;noreferrer&quot;&gt;Large-scale interior point overview paper&lt;/a&gt;, &lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs10107-012-0557-4&quot; rel=&quot;noreferrer&quot;&gt;paper going into more detail about implementation of large-scale interior point methods&lt;/a&gt;); there are also algorithms by Petra, Zavala, et al that exploit structure (&lt;a href=&quot;http://www.mcs.anl.gov/~vzavala/paper_pscc.pdf&quot; rel=&quot;noreferrer&quot;&gt;structured nonconvex optimization paper&lt;/a&gt;, &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/130908737&quot; rel=&quot;noreferrer&quot;&gt;Schur complement for interior-point methods paper&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Most research seems to be working on parallelism for the distributed-memory case (which is hard enough). There is some work on the shared-memory case for LP &amp;amp; QP solvers, because that work makes it into commercial codes (e.g., Gurobi, CPLEX, Xpress). Existing work is interesting, but there doesn't seem to be anything compelling for GPUs yet, excepting special applications (e.g., machine learning, for which you can use different algorithms better suited to GPUs).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-10-07T18:14:10.413" LastActivityDate="2015-10-07T18:14:10.413" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20948" PostTypeId="2" ParentId="20940" CreationDate="2015-10-07T03:42:59.003" Score="1" Body="&lt;p&gt;$5(x_{n+1}+c)=x_n+c \Rightarrow  5x_{n+1}-x_n=-4c$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Namely,$c=-1/12$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $b_{n+1}=x_{n+1}-1/12$, then $b_{n+1}=(1/5) b_n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can achieve the following formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b_n=b_0 (\frac 1 5)^{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, $b_0=x_0-\frac 1 {12}=0 \Rightarrow b_n=0$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0=b_n=x_{n}-1/12$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;so we achieve $x_n=1/12$&lt;/p&gt;&#xA;" OwnerUserId="14845" LastActivityDate="2015-10-07T03:42:59.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20949" PostTypeId="2" ParentId="20921" CreationDate="2015-10-07T04:09:25.153" Score="0" Body="&lt;p&gt;It looks as though your first example is working, except for the edge-case of connecting the first and last points. An easy solution, if it works, would be to repeat the first point in &lt;code&gt;pts&lt;/code&gt; at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might check back later, if that doesn't work it might take a while to solve.&lt;/p&gt;&#xA;" OwnerUserId="17112" LastEditorUserId="17112" LastEditDate="2015-10-07T22:56:44.827" LastActivityDate="2015-10-07T22:56:44.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20950" PostTypeId="2" ParentId="422" CreationDate="2015-10-07T05:42:06.523" Score="0" Body="&lt;p&gt;You can use the open source &lt;a href=&quot;http://asl.org.il&quot; rel=&quot;nofollow&quot;&gt;Advanced Simulation Library&lt;/a&gt; which is hardware accelerated (in the case &lt;a href=&quot;http://asl.org.il/benchmarks/multicomponent_flow/#numerical-setup&quot; rel=&quot;nofollow&quot;&gt;high performance computation&lt;/a&gt; is important). It has both &lt;a href=&quot;http://asl.org.il/benchmarks/multicomponent_flow/&quot; rel=&quot;nofollow&quot;&gt;laminar and turbulent flows&lt;/a&gt; and chemical reactions. It is also easy to use, see the source code of the &lt;a href=&quot;http://asl.org.il/doc/Developer-Guide/locomotive_8cc-example.html&quot; rel=&quot;nofollow&quot;&gt;Aerodynamics of a locomotive in a tunnel&lt;/a&gt; benchmark.&lt;/p&gt;&#xA;" OwnerUserId="16442" LastEditorUserId="16442" LastEditDate="2015-10-08T16:31:59.447" LastActivityDate="2015-10-08T16:31:59.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20951" PostTypeId="1" AcceptedAnswerId="20957" CreationDate="2015-10-07T09:34:38.343" Score="4" ViewCount="591" Body="&lt;p&gt;I have several probability distribution histograms of images that I am comparing to test for dissimilarity.  Each histogram has 256 bins, the bins are common for all histograms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The images are taken in very low light conditions, using narrow bandpass filters and illuminated by the same light source (with any other light source excluded).  Each image is to be compared to the 'dark noise' histograms to determine (if any) dissimilarities exist between the low light image and dark noise distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is an effective numerical method to determine for testing for dissimiliarity between image based probability histograms?&lt;/p&gt;&#xA;" OwnerDisplayName="user17774" LastEditorDisplayName="user17774" LastEditDate="2015-10-07T10:03:54.817" LastActivityDate="2015-11-21T12:17:34.790" Title="What are the numerical methods for testing for dissimiliarity between image based probability histograms?" Tags="|image-processing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20952" PostTypeId="2" ParentId="1380" CreationDate="2015-10-07T11:37:29.390" Score="4" Body="&lt;p&gt;In order to formulate the constraints $f \le \max \{ f_1, f_2,...,f_n \}$,&#xA;we create $n$ &lt;strong&gt;binary&lt;/strong&gt; variables, $b_i \in \{0,1\}$, $1 \le i \le n$. &#xA;Let $M$ be the bound of variable $f$, then we only need to add the following constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) $f \le f_i + (1-b_i) M, \forall i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) $\sum_i b_i = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, set $M := max_i f_i - min_i f_i$ if we can estimate the value of $f_i$.&lt;/p&gt;&#xA;" OwnerUserId="17800" LastActivityDate="2015-10-07T11:37:29.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20953" PostTypeId="2" ParentId="2304" CreationDate="2015-10-07T11:37:44.213" Score="1" Body="&lt;p&gt;I can recommend to use the open source &lt;a href=&quot;http://asl.org.il&quot; rel=&quot;nofollow&quot;&gt;Advanced Simulation Library&lt;/a&gt; which is hardware accelerated. It is easy to use, see the source code of this &lt;a href=&quot;http://asl.org.il/doc/Developer-Guide/locomotive_8cc-example.html&quot; rel=&quot;nofollow&quot;&gt;aerodynamics simulation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16442" LastActivityDate="2015-10-07T11:37:44.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20954" PostTypeId="2" ParentId="20923" CreationDate="2015-10-07T12:47:01.570" Score="1" Body="&lt;p&gt;I've converted this from a comment to an answer since no one produced any more elaborate answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, there is a certain arbitrariness. The values you commonly see in books and papers are ones which seem to work well over some reasonable range of problems. They are not handed down from god, and you may be able to be to do better for some problems.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2015-10-07T12:47:01.570" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20955" PostTypeId="1" CreationDate="2015-10-07T14:18:20.927" Score="5" ViewCount="236" Body="&lt;p&gt;Suppose I have a linear system &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\lbrack&#xA;  \begin{array}{cc}&#xA;  M_1&amp;amp; S\\&#xA;  S^{\mathrm{T}}&amp;amp; M_2&#xA;\end{array}&#xA;\right\rbrack \left\lbrack \begin{array}{c} X\\ Y\end{array} \right\rbrack=&#xA;\left\lbrack \begin{array}{c} F\\G\end{array}\right\rbrack,&#xA;$$&#xA; where $M_1\in \mathrm{R}^{n\times n}$ is symmetric positive definite, $M_2\in \mathrm{R}^{m\times m}$ is symmetric positive semi-definite.  $S$ is a full-rank matrix of compatible size.  $X,Y,F,G$ are all vectors of appropriate size.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What factors are relevant in deciding between using GMRES on the big system and using a  Schurr complement technique?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Factors that I would guess are relevant are how expensive $M_1$ is to invert, the condition number of $S$, and whether $M_2$ is the zero matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any links to the literature are appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am asking this question because I need to solve the following system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\lbrack &#xA;  \begin{array}{cccc}&#xA;  M_1&amp;amp; S &amp;amp;0 &amp;amp;\Gamma^{\mathrm{T}} \\&#xA;  -S^{\mathrm{T}}&amp;amp; 0 &amp;amp;\Xi^{\mathrm{T}}&amp;amp; 0\\&#xA;  0 &amp;amp; \Xi &amp;amp; 0 &amp;amp;0 \\&#xA;  \Gamma&amp;amp;0&amp;amp;0&amp;amp;0&#xA;   \end{array}&#xA;\right\rbrack&#xA;\left\lbrack&#xA;\begin{array}{c} x_1\\x_2\\x_3\\x_4&#xA;\end{array}&#xA;\right\rbrack= &#xA;\left\lbrack&#xA;\begin{array}{c}&#xA;f_1\\f_2\\f_3\\f_4&#xA;\end{array}&#xA;\right\rbrack&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, I solve this system using the Schurr complement method but it requires three iterative CG solves.  I am wondering if it would be better to just &quot;brute force it&quot; and use GMRES.&lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2015-10-08T03:01:57.600" LastActivityDate="2015-10-08T13:17:47.377" Title="What factors are relevant when deciding between GMRES and schur complement solves?" Tags="|linear-algebra|linear-solver|gmres|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20956" PostTypeId="2" ParentId="20955" CreationDate="2015-10-07T15:06:28.257" Score="3" Body="&lt;p&gt;You can often manipulate systems like this using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Block_LU_decomposition&quot; rel=&quot;nofollow&quot;&gt;block LU factorization&lt;/a&gt;; the factors you mentioned then come into play.  Preconditioners can then be designed by taking advantage of the block diagonal structure revealed by the factorization, with the blocks being $M_1$ and the Schur complement $M_2-S^TM_1^{-1}S$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An advantage of this approach is that you can also utilize approximate matrix-free inverses for $M_1$ and the Schur complement, which is useful if your matrices are just too big to form and manipulate. For example, for finite element discretizations of incompressible flow, $M_1^{-1}$ can be replaced by a few multigrid iterations. Preconditioners for the Schur complement can also be approximated using matrix-free methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some care should be taken in constructing these preconditioners (for example, &lt;a href=&quot;https://eudml.org/doc/225663&quot; rel=&quot;nofollow&quot;&gt;Wathen&lt;/a&gt; shows cases where substituting a matrix inverse with an iterative method fails because it induces a nonlinear operator instead of a linear one). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: as Wolfgang Bangerth pointed out, this issue can also be solved by employing a flexible CG or GMRES outer iteration.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-10-08T13:17:47.377" LastActivityDate="2015-10-08T13:17:47.377" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20957" PostTypeId="2" ParentId="20951" CreationDate="2015-10-07T17:05:18.690" Score="2" Body="&lt;p&gt;Sounds like a job for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Earth_mover%27s_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Earth Mover's Distance&lt;/a&gt; (or Wasserstein metric).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In computer science, the &lt;strong&gt;earth mover's distance&lt;/strong&gt; (EMD) is a measure of the distance between two probability distributions.&lt;/p&gt;&#xA;&lt;p&gt;Informally, if the distributions are interpreted as two different ways of piling up a certain amount of dirt over the region &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, the EMD is the minimum cost of turning one pile into the other; where the cost is assumed to be amount of dirt moved times the distance by which it is moved.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here's a link to a &lt;a href=&quot;https://github.com/pdinges/python-emd&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python+C implementation&lt;/a&gt; (strictly speaking a Python wrapper).&lt;/p&gt;&#xA;&lt;p&gt;And here's the original implentation by Yossi Rubner: &lt;a href=&quot;http://ai.stanford.edu/%7Erubner/emd/default.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ai.stanford.edu/~rubner/emd/default.htm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8094" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-10-07T20:59:59.380" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20958" PostTypeId="1" AcceptedAnswerId="20973" CreationDate="2015-10-08T08:48:03.360" Score="1" ViewCount="224" Body="&lt;p&gt;I want to solve 7 pde's that are functions of time, radius(j) and length(i). I used the method of lines and converted them to a system of odes in time and it becomes something like this:&#xA;$$dy/dt=((y(i,j)-y(i,j-1))\Delta{r} )+(y(i,j)-y(i-1,j))\Delta{x}$$&#xA;I define all of the equations in one function in MATLAB. This is my function:&#xA;$$dy/dt=f(t,y)$$&#xA;and then I solve those ode's by using ode15s solver. &#xA;Now one of the boundary conditions is an algebraic equation that should be in this function too. It is an algebraic equation of the form: $$y=f(r,x)$$&#xA;so I think if equations have the same appearance, I can write all of them in one function. So I write it in this way: &#xA;$$(10^{-10})\times dy=y-f(r,x)$$&#xA;I put this small coefficient to eliminate $dy$ effect and solve algebraic equation, but MATLAB cant solve it. How can I define this function?&lt;/p&gt;&#xA;" OwnerUserId="17594" LastEditorUserId="20688" LastEditDate="2019-06-22T21:23:41.173" LastActivityDate="2019-06-22T21:24:55.207" Title="How can i define algebraic equation in differential function in MATLAB?" Tags="|matlab|ode|method-of-lines|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20959" PostTypeId="2" ParentId="2393" CreationDate="2015-10-08T09:57:43.937" Score="2" Body="&lt;p&gt;You would need:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;an optimizer and&lt;/li&gt;&#xA;&lt;li&gt;a CFD tool that is capable (flexible) enough to deal with different designs within a concrete concept or even belonging to different concepts.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Following features are crucial for the CFD tool:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;High computation efficiency (speed) in order to treat high amount of&#xA;separate designs in a reasonable time frame.&lt;/li&gt;&#xA;&lt;li&gt;High flexibility and amiability of the numerical techniques in order&#xA;to exclude manual intervention into the search/analysis process.&lt;/li&gt;&#xA;&lt;li&gt;Programmatical geometry manipulation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So I can recommend following software:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.itk.org/&quot; rel=&quot;nofollow&quot;&gt;Insight Toolkit&lt;/a&gt; which provides a good number of optimizers that can be coupled with&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://asl.org.il/&quot; rel=&quot;nofollow&quot;&gt;Advanced Simulation Library&lt;/a&gt; that offers&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;High performance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The library is hardware accelerated, i.e. capable to utilize GPU or FPGA hardware (if available) which means 10-100 speed up vs. CPU based program. Also on the regular CPU it uses advanced features such as SIMD.&lt;/li&gt;&#xA;&lt;li&gt;It is based on dynamic compilation approach which means that there is no need to sacrifice performance for flexibility. This results in up to 10 times speedup vs the standard compilation technique.&lt;/li&gt;&#xA;&lt;li&gt;ASL can be used in cluster infrastructure and in multi-GPU computers.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;High flexibility and amiability:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;strong&gt;mesh-free&lt;/strong&gt; numerical techniques are based on rectangular grid and immersed boundary approaches. These features enable &lt;strong&gt;automated&lt;/strong&gt; design optimization, since no mesh-generation is required. &lt;/li&gt;&#xA;&lt;li&gt;The library is based on dynamic compilation approach which means that there is no need to sacrifice performance for flexibility. This allows to formulate single general and efficient algorithm for different designs/concepts.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://asl.org.il/doc/Developer-Guide/group___geom.html&quot; rel=&quot;nofollow&quot;&gt;Generation and manipulation of geometric primitives&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16442" LastEditorUserId="16442" LastEditDate="2015-12-24T10:50:28.237" LastActivityDate="2015-12-24T10:50:28.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20960" PostTypeId="1" AcceptedAnswerId="20961" CreationDate="2015-10-08T10:14:51.163" Score="7" ViewCount="14663" Body="&lt;p&gt;Is there a Fortran subroutine which performs linear interpolation in one-dimenional data?&#xA;I need something similar to MATLAB function interp1.&lt;/p&gt;&#xA;" OwnerUserId="15018" LastActivityDate="2019-10-10T02:51:17.437" Title="Linear interpolation in Fortran" Tags="|fortran|interpolation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20961" PostTypeId="2" ParentId="20960" CreationDate="2015-10-08T11:32:20.523" Score="10" Body="&lt;p&gt;There is no built-in Fortran functionality to do linear interpolation. You could either use a library or write your own routine. I haven't tried compiling or testing and my fortran may be a bit rusty, but something like the following should work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine interp1( xData, yData, xVal, yVal )&#xA;! Inputs: xData = a vector of the x-values of the data to be interpolated&#xA;!         yData = a vector of the y-values of the data to be interpolated&#xA;!         xVal  = a vector of the x-values where interpolation should be performed&#xA;! Output: yVal  = a vector of the resulting interpolated values&#xA;&#xA;  implicit none&#xA;&#xA;  real, intent(in) :: xData(:), yData(:), xVal(:)&#xA;  real, intent(out) :: yVal(:)&#xA;  integer :: inputIndex, dataIndex&#xA;  real :: minXdata, minYdata, xRange, weight&#xA;&#xA;  ! Possible checks on inputs could go here&#xA;  ! Things you may want to check:&#xA;  !   monotonically increasing xData&#xA;  !   size(xData) == size(yData)&#xA;  !   size(xVal) == size(yVal)&#xA;&#xA;  minXData = xData(1)&#xA;  maxXData = xData(size(xData))&#xA;  xRange = maxXData - minXData&#xA;&#xA;  do inputIndex = 1, size(xVal)&#xA;      ! possible checks for out of range xVal could go here&#xA;&#xA;      ! this will work if x is uniformly spaced, otherwise increment&#xA;      ! dataIndex until xData(dataIndex+1)&amp;gt;xVal(inputIndex)&#xA;      dataIndex = floor((xVal(inputIndex)-minXData)/xRange);&#xA;&#xA;      weight = (xVal - xData(dataIndex))/(xData(dataIndex+1)-xData(dataIndex));&#xA;      yVal(inputIndex) = (1.0-weight)*yData(dataIndex) + ...&#xA;                         weight*yData(dataIndex+1);&#xA;  end do&#xA;end subroutine&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="-1" LastEditDate="2019-10-10T02:51:17.437" LastActivityDate="2019-10-10T02:51:17.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20962" PostTypeId="1" CreationDate="2015-10-08T12:21:19.580" Score="1" ViewCount="107" Body="&lt;p&gt;I am creating a random configuration of particles for a molecular dynamics simulation, where I would like to guarantee a certain density.&#xA;The strategy is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;choose randomly the positions of the particles in $x$- and $y$-axis in square of the side length equal to $a$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;compute the distance between every pair of particles, using the periodic boundaries and minimum image conditions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if the distance is less than $2^{1/6}\sigma$ then we should generate them once again to avoid overlaps.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My problem is that the program for the side length $a=10$, number of particles $n=100$ and the $\sigma=1$ diverges. The maximum $\sigma$ value for which the program gives an output is $0.74$. What is the reason?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I change the length side to a higher value? Or there is a other way to make the system more dense?&lt;/p&gt;&#xA;" OwnerUserId="17809" LastEditorUserId="20688" LastEditDate="2019-06-22T22:49:30.297" LastActivityDate="2019-06-22T22:49:30.297" Title="Creating dense random configuration in for molecular dynamics" Tags="|molecular-dynamics|randomized-algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20963" PostTypeId="2" ParentId="20962" CreationDate="2015-10-08T13:41:37.910" Score="4" Body="&lt;p&gt;As I am assuming by the word random you mean &quot;liquid or gas like&quot; is there any reason you can't use your MD program itself?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pick the number of particles and the volume of the unit cell so&#xA;as to get the density you want &lt;/li&gt;&#xA;&lt;li&gt;Set the particles up on a regular&#xA;lattice such as face centred or body centred cubic with some&#xA;vacancies if required by the number of particles &lt;/li&gt;&#xA;&lt;li&gt;Run a constant&#xA;volume simulation at high enough temperature to melt the solid &lt;/li&gt;&#xA;&lt;li&gt;Re-equilibrate at the temperature of interest&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now I realise this won't exactly satisfy your greater than $2^{1/6}\sigma$ criterion, but to satisfy that exactly will be unphysical and probably not what you want - and any sensible inter-atomic potential will mean there are few such separations. And if you are not using constant volume there are other problems, but I don't see why you can't use a recipe similar to the above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method you outline above is going to get very slow for high density systems, random numbers will just lead to too many atomic overlaps. Unfortunately I can't tell you at what reduced density this will start to cause problems in your set up, but in a related one I did many years ago anything for which ${N \over V}\sigma^3$ was greater than about 0.1 ran into problems if (and this is a big if, it's over 20 years ago) I remember correctly.&lt;/p&gt;&#xA;" OwnerUserId="17475" LastActivityDate="2015-10-08T13:41:37.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20965" PostTypeId="2" ParentId="20960" CreationDate="2015-10-08T16:18:28.203" Score="3" Body="&lt;p&gt;Take a look at the Numerical Recipes book, a classic, found &lt;a href=&quot;http://www.haoli.org/nr/bookfpdf.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. Look in the interpolation chapter. You can find an explanation, plus the code. It's in pdf. A simple google search also produced a &lt;a href=&quot;https://github.com/astrofrog/fortranlib&quot; rel=&quot;nofollow&quot;&gt;github library&lt;/a&gt; by Thomas Robitaille, where a routine called interp1d exists. &lt;/p&gt;&#xA;" OwnerUserId="157" LastActivityDate="2015-10-08T16:18:28.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20966" PostTypeId="1" AcceptedAnswerId="20967" CreationDate="2015-10-08T19:30:31.837" Score="3" ViewCount="580" Body="&lt;p&gt;I am working with symmetric matrices of order $n \times n$ where $n \leq 50$. The diagonal elements of my matrices are a fixed number $d$ and the off diagonal elements are limited to two small numbers $\{a,b\}$.I know $a,b,d$ in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any such matrix I need to be able to determine whether its determinant is negative or not. I must make sure that I do not mistakenly mark a matrix with non-negative determinant as having a negative determinant. I am not concerned with the converse  mistake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What would be a sensible way to determine whether the computed determinant is indeed negative?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The way to go would be to determine the precision guarantee of the algorithms implemented by GNU GSL but unfortunately the documentation does not mention that for these specific functions (&lt;strong&gt;&lt;em&gt;gsl_linalg_LU_decomp&lt;/em&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;em&gt;gsl_linalg_LU_det&lt;/em&gt;&lt;/strong&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already asked this on the relevant GSL mailist but just to get a different perspective, I was wondering, what would be a good way to figure a test that would be 100% precise?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I am just testing whether the determinant is smaller than a rough negative number but I would like to have the smallest possible value $\beta$ such that the determinant is negative whenever it is smaller than $\beta.$&lt;/p&gt;&#xA;" OwnerUserId="4582" LastActivityDate="2015-10-09T23:02:56.217" Title="GSL linear algebra LU/determinant precision" Tags="|linear-algebra|error-estimation|precision|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20967" PostTypeId="2" ParentId="20966" CreationDate="2015-10-08T22:17:32.577" Score="3" Body="&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; I looked up forward LU error bounds in Higham's &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;, Theorem 9.15 (citing &lt;a href=&quot;http://dx.doi.org/10.1007/BF01931295&quot; rel=&quot;nofollow&quot;&gt;Barrlund&lt;/a&gt; and &lt;a href=&quot;http://dx.doi.org/10.1007/BF01994852&quot; rel=&quot;nofollow&quot;&gt;Sun&lt;/a&gt;), for the LU decomposition &#xA;$$A=LU,\quad A+\Delta A=(L+\Delta L)(U+\Delta U)$$ &#xA;it gives the norm-wise error bound&#xA;$$\begin{gather}&#xA;\frac{\|\Delta U\|_F}{\|U\|_2} \leq \frac{\|L^{-1}\|_2\|U^{-1}\|_2\|A\|_2}{1-\|L^{-1}\|_2\|U^{-1}\|_2\|\Delta A\|_2}\frac{\|\Delta A\|_F}{\|A\|_F} =: M, \\&#xA; \|\Delta A\|_\infty \leq n^2\gamma_{3n}\rho_n \|A\|_\infty,&#xA;\end{gather}$$&#xA;where $\gamma_k = k\epsilon/(1-k\epsilon)$ ($\epsilon$ is the unit roundoff), and $\rho_n$ is the growth factor, defined as&#xA;$$\max_{i,j,k} |a^{(k)}_{i,j}|/\max_{i,j} |a_{i,j}|$$&#xA;with $a^{(k)}_{i,j}$ being the matrix elements at $k$-th stage of LU factorization by Gaussian elimination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in principle, so long as you can estimate all of the above numbers, the procedure should produce the correct determinant sign so long as the smallest value on the diagonal of $U$ is not so close to zero that its sign could have been influenced by numerical errors:&#xA;$$ \min_k |u_{kk}| \geq M\|U\|_2 \geq \|\Delta U\|_F. $$&#xA;Of course, this is not very elegant. Higham also gives a component-wise error bound, which should be stricter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; Not likely, but if you know something about how the numbers $a,b$ appear in off-diagonal elements, it is possible there might be a lower bound $m = \min_A |\det A|$. If it so happens that it is easy to calculate, then the determinant's sign can be accurately determined so long as the numerical errors are at most $m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; Eigendecomposition, although more expensive, might also be helpful. For example, GSL (&lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Real-Symmetric-Matrices.html#Real-Symmetric-Matrices&quot; rel=&quot;nofollow&quot;&gt;https://www.gnu.org/software/gsl/manual/html_node/Real-Symmetric-Matrices.html#Real-Symmetric-Matrices&lt;/a&gt;) promises that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The computed eigenvalues are accurate to an absolute accuracy of \epsilon ||A||_2, where \epsilon is the machine precision.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This would directly address the issue of whether the determinant's sign is correct - that would require all eigenvalues to satisfy $|\lambda| &amp;gt; \epsilon \|A\|_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4.&lt;/strong&gt; In general, computation of a sign of some quantity is ill-conditioned when that quantity is small - the determinant's sign would be very sensitive to small changes in the input matrix. So the common solution to this problem is just to look for a way to avoid computing the determinant's sign at all, but I don't know how feasible that is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;5.&lt;/strong&gt; Arbitrary-precision floating-point arithmetic might also help. Although not with GSL, there are libraries (e.g., Eigen) that implement linear algebra in a way that can work with, for example, mpfr.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;6.&lt;/strong&gt; (&lt;em&gt;Edit.&lt;/em&gt;) The $LDL^\top$ decomposition is even easier I think: from Barrlund (Eq. 2.1b), ($D$ is the diagonal from $U$, so I don't think it matters if you actually compute $LU$ instead because this is a perturbation analysis)&#xA;$$ \|\Delta D\|_F \leq \frac{\kappa_2(A)}{1-\kappa_2(A)\frac{\|\Delta A\|_2}{\|A\|_2}} \|\Delta A\|_F = \beta, $$&#xA;which now only needs estimates of $A$'s 2-norm condition number $\kappa_2(A)$ and the normwise error backward error estimates $\|\Delta A\|_2$, $\|\Delta A\|_F$ from above. The correct sign is now guaranteed by something like $|d_{kk}|\geq \beta$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-10-09T23:02:56.217" LastActivityDate="2015-10-09T23:02:56.217" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20968" PostTypeId="1" CreationDate="2015-10-08T23:05:52.667" Score="6" ViewCount="402" Body="&lt;p&gt;I'm not sure how I can represent and solve the following problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a list of sales (timestamp and quantity) and a list of corresponding inventory draws (timestamp and quantity).  What I ultimately want to achieve is to identify inventory draws not accounted for by a sales transaction (ie. missing inventory taken).  To add to the complexity, a draw can correspond to more than one sale and a sale can correspond to more than one draw (many to many).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how I should start this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a sample of real data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/17vRQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/17vRQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17816" LastEditorUserId="17816" LastEditDate="2015-10-10T05:34:51.427" LastActivityDate="2018-11-10T16:02:15.400" Title="Matching/Assignment Problem" Tags="|optimization|algorithms|approximation|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20969" PostTypeId="1" CreationDate="2015-10-09T03:11:56.630" Score="2" ViewCount="1258" Body="&lt;p&gt;I would like to have both the normal (xx) component and shear (xy) component of a 2D (stress) tensor defined on a boundary (y=const, for instance) for an equation which is of the type&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla \cdot \sigma = 0 &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The normal value of $\sigma_{xx}=P$ at $x=0,1$ can be accounted for via a Neumann boundary condition, in the term obtained by integration by parts in weak form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_\Omega  \sigma : \nabla \tau &#xA;=  \int_{\partial\Omega}  P \cdot \tau \, d s .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tau$ is a vector test function for the trial function $\vec{u}$, which physically is the displacement.&#xA;However, I am having difficulty fixing the shear stress at the boundaries $x=0,1$ of the unit square, as a simple example of my goal. How can one best accomplish this in FEniCS?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is similar to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7123/fenics-separate-boundary-conditions-in-normal-and-tangential-direction-of-mesh&quot;&gt;this one&lt;/a&gt;, but as far as I can tell, they are distinct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A minimal running code in FEniCS (in python) which gives incorrect results for $\sigma$ is below. Thank you!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;&#xA;# Mesh Params ################&#xA;E, nu = 1.0, 0.4 #material params&#xA;mesh = UnitSquareMesh(100, 100)&#xA;&#xA;print 'Creating Lagrangian function spaces...'&#xA;Vv = VectorFunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;Vf = FunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;Vt = TensorFunctionSpace(mesh, &quot;Lagrange&quot;, 1)&#xA;&#xA;# P is stress vector of stress values normal and tangent to boundary &#xA;P = Expression(('(x[0]-xc)','(x[0]-xc)'),xc=0.5)&#xA;&#xA;class LeftBoundary(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        tol = 1E-7&#xA;        return on_boundary and abs(x[0]) &amp;lt; tol&#xA;&#xA;class RightBoundary(SubDomain):&#xA;    def inside(self, x, on_boundary):&#xA;        tol = 1E-7&#xA;        return on_boundary and abs(x[0] - 1) &amp;lt; tol&#xA;&#xA;left_boundary = LeftBoundary()        &#xA;right_boundary = RightBoundary()        &#xA;&#xA;boundary_parts = MeshFunction(&quot;size_t&quot;, mesh, mesh.topology().dim() - 1)&#xA;boundary_parts.set_all(0)&#xA;right_boundary.mark(boundary_parts, 1)&#xA;left_boundary.mark(boundary_parts, 2)&#xA;ds = Measure(&quot;ds&quot;)[boundary_parts]&#xA;&#xA;##############################&#xA;# Define variational problem #&#xA;##############################&#xA;print 'Defining test and trial functions...'&#xA;v = TestFunction(Vf)&#xA;tau = TestFunction(Vv)&#xA;phi = TrialFunction(Vf)&#xA;u = TrialFunction(Vv)&#xA;&#xA;# functions for variational problem&#xA;d = u.geometric_dimension()&#xA;epsilon = sym(nabla_grad(u))&#xA;# PLANE STRESS&#xA;sigma = E/(1+nu)* (epsilon) + E*nu/((1-nu**2)) * tr(epsilon)*Identity(d)&#xA;&#xA;# Define normal and tangent directions on mesh&#xA;norm = FacetNormal(mesh)&#xA;tang = as_vector([norm[1], -norm[0]])&#xA;&#xA;tau_n = dot(tau,norm)&#xA;tau_t = dot(tau,tang)&#xA;P_n = dot(P,norm)&#xA;P_t = dot(P,tang)&#xA;&#xA;#################&#xA;# SOLVE&#xA;#################&#xA;au = inner(sigma, sym(nabla_grad(tau)))*dx &#xA;Lu = dot(P_n,tau_n)*ds(1) + dot(P_n,tau_n)*ds(2) + dot(P_t,tau_t)*ds(1) + dot(P_t,tau_t)*ds(2)&#xA;u = Function(Vv)&#xA;solve(au == Lu, u)&#xA;&#xA;## View the results&#xA;sigma = E/(1+nu)* (sym(nabla_grad(u))) + E*nu/((1-nu**2)) * tr(sym(nabla_grad(u)))*Identity(d)&#xA;s_V = project( sigma, Vt)&#xA;sxx = project(s_V[0,0], Vf); &#xA;sxy = project(s_V[0,1], Vf); &#xA;syx = project(s_V[1,0], Vf);  &#xA;syy = project(s_V[1,1], Vf);&#xA;plot(sxx, interactive=True, rescale=False,scalarbar=True)&#xA;plot(syy, interactive=True, rescale=False,scalarbar=True)&#xA;plot(sxy, interactive=True, rescale=False,scalarbar=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17817" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-10-09T15:12:01.927" Title="FEniCS: both normal and shear stress boundary conditions for elasticity?" Tags="|finite-element|boundary-conditions|fenics|" AnswerCount="0" CommentCount="4" ClosedDate="2015-10-11T14:39:12.980" ContentLicense="CC BY-SA 3.0" />
  <row Id="20970" PostTypeId="1" AcceptedAnswerId="20971" CreationDate="2015-10-09T04:39:41.887" Score="5" ViewCount="1970" Body="&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/Explicit_and_implicit_methods&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Explicit methods calculate the state of a system at a later time from&#xA;  the state of the system at the current time, while implicit methods&#xA;  find a solution by solving an equation involving both the current&#xA;  state of the system and the later one.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then why the Crank-Nicolson is considered implicit in time, if a later time is calculated by solving a system of linear equations that depend only on the previous time?&lt;/p&gt;&#xA;" OwnerUserId="11514" LastEditorUserId="27" LastEditDate="2015-10-11T19:30:35.667" LastActivityDate="2015-10-12T02:36:32.563" Title="Why is Crank-Nicolson considered implicit in time?" Tags="|implicit-methods|crank-nicolson|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20971" PostTypeId="2" ParentId="20970" CreationDate="2015-10-09T05:22:54.723" Score="8" Body="&lt;p&gt;A simplification - the Crank-Nicolson method uses the average of the forward and backward Euler methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The backward Euler method is implicit, so Crank-Nicolson, having this as one of its components, is also implicit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More accurately, this method is implicit because $u^{n+1}_i$ depends on $F^{n+1}_i$, not just $F^{n}_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is means that the state at time $t=n+1$ is defined in terms of itself, and not just the previous time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or as stated in another answer, if the $F^{n+1}_i$ term is non-zero, the method is implicit.&lt;/p&gt;&#xA;" OwnerUserId="17112" LastEditorUserId="17112" LastEditDate="2015-10-12T02:36:32.563" LastActivityDate="2015-10-12T02:36:32.563" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="20972" PostTypeId="2" ParentId="20970" CreationDate="2015-10-09T06:08:36.327" Score="3" Body="&lt;p&gt;The Crank-Nicolson method is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{u^{n+1}_{i}-u^{n}_{i}}{dt} = \frac{1}{2}(F^{n+1}_{i}+F^{n}_{i})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method calculates the next state of the system, i.e. $u^{n+1}_{i}$, by solving an equation involving the previous states and the next state. In the case of the heat equation for example we would get a linear system and if we are using finite elements this system would look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M(\frac{u^{n+1}-u^{n}}{dt}) = -\frac{1}{2}K(u^{n+1}+u^{n})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(M+\frac{1}{2}dtK)u^{n+1} = (M-\frac{1}{2}dtK)u^{n}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where M is the mass matrix and K is the stiffness matrix. Because the heat equation is linear we could separate the $u^{n+1}_{i}$'s from the $u^{n}_{i}$'s however the cost of this separation is that we must solve a linear system. In the case of a nonlinear $F^{n+1}_{i}$ and $F^{n}_{i}$ this separation would not be possible and instead we would have to use something like Newtons method to iterate our way to a solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, if we have a timestepping scheme of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{u^{n+1}_{i}-u^{n}_{i}}{dt} = \alpha_{-1}F^{n+1}_{i}+\alpha_{0}F^{n}_{i}+\alpha_{1}F^{n-1}_{i}+\alpha_{2}F^{n-2}_{i}+...$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it is implicit if $\alpha_{-1}$ is non-zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are more familiar with finite differences rather than finite elements then the Crank-Nicolson solution to the heat equation would look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{u^{n+1}-u^{n}}{dt} = \frac{1}{2}A(u^{n+1}+u^{n})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(I-\frac{1}{2}dtA)u^{n+1} = (I+\frac{1}{2}dtA)u^{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $I$ is the identity matrix and $A$ is the finite difference tridiagonal discretization matrix.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-10-09T06:44:44.883" LastActivityDate="2015-10-09T06:44:44.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20973" PostTypeId="2" ParentId="20958" CreationDate="2015-10-09T08:11:53.150" Score="2" Body="&lt;p&gt;Your question is not quite clear, but if your problem is roughly analogous to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{d\mathbf{x}}{dt}=\mathbf{f}[\mathbf{x},y]$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$g[\mathbf{x},y]=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then you just need to use a &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ode15s.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;mass matrix&lt;/a&gt; in your call to ode15s. This is how you &quot;tell&quot; Matlab that you have an algebraic equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, in Matlab pseudo-code you can express the above as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eye(n)*dxdt = f(x,y) % n=length(x)&#xA;  0   *dydt = g(x,y) % y is a scalar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which you would pass to ode15s as a single system&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M*dzdt = h(z) % here z=[x;y] , h=[f;g], and M = blkdiag(eye(n),0)    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The particular syntax will depend on the specific system. The relevant part of the help in the above link begins where it says:&#xA;&quot;If the mass matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is singular, then &lt;span class=&quot;math-container&quot;&gt;$M(t,y)y′ = f(t,y)$&lt;/span&gt; is a system of differential algebraic equations.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Edit: The explanation &lt;a href=&quot;https://en.wikipedia.org/wiki/Differential_algebraic_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; may be more clear.)&lt;/p&gt;&#xA;" OwnerUserId="17742" LastEditorUserId="20688" LastEditDate="2019-06-22T21:24:55.207" LastActivityDate="2019-06-22T21:24:55.207" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20974" PostTypeId="1" CreationDate="2015-10-09T09:14:34.120" Score="2" ViewCount="227" Body="&lt;p&gt;Suppose I have a mesh of the sphere (points on the sphere and a triangulation). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is a good and efficient way to find the triangle which contains a point on the sphere (the point does not need to be a triangulation node)? It would be nice if the operation could be vectorized, to improve Matlab speed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The purpose of this is to construct a simple spherical (linear) interpolation operator on the sphere.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2016-02-08T21:50:03.360" Title="Find triangle which contains point on the sphere" Tags="|interpolation|mesh|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="20975" PostTypeId="1" AcceptedAnswerId="20976" CreationDate="2015-10-09T16:21:54.807" Score="2" ViewCount="150" Body="&lt;p&gt;I am trying to program the pieceewise quadratic lagrange, but I have to deal with these intervals at which the point I am trying to evaluate belongs to. So, essentially you need to have an if statement before we apply the formula, here is a picture of it. &lt;a href=&quot;https://i.stack.imgur.com/Yg09i.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yg09i.jpg&quot; alt=&quot;Lagrange&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to write the if statement, if anyone can help me I would greatly appreciate it, here is the code I have thus far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/*Your experiments should include the following functions on [−1, 1]:&#xA; * f(x) = |ax|&#xA; * f(x) = |ax| + x/2 - x^2 &#xA; * f(x) = 1/(1+ ax^2) &#xA; */&#xA;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;// Piecewise linear Lagrange --- Lagrange Form&#xA;float LagrangeQuad(float x[], float x_eval, float f[], int n);&#xA;&#xA;&#xA;&#xA;int main() {&#xA;    int n = 100;&#xA;&#xA;    // 100 x points&#xA;    float x[n+1];&#xA;    // Step size for interval [-1,1]&#xA;    float h = (1.0 - (-1.0))/100.0;&#xA;    for(int i = 0; i &amp;lt;= n; i++){&#xA;        x[i] = -1 + i*h;&#xA;    }&#xA;&#xA;    //Function values single&#xA;    float alpha = 1.0;&#xA;    float f_0[n+1], f_1[n+1], f_2[n+1];&#xA;    for(int i = 0; i &amp;lt;= n; i++){&#xA;        f_0[i] = fabs(alpha*x[i]);&#xA;        f_1[i] = fabs(alpha*x[i]) + x[i]/2 - pow(x[i],2);&#xA;        f_2[i] = 1/(1 + alpha*pow(x[i],2));&#xA;    }&#xA;&#xA;    // Create new set of x values that we will use to compare the interpolation&#xA;    float x_test[n+1];&#xA;    float h_test = (1.0 - (-1.0))/1000.0;&#xA;    for(int i = 0; i &amp;lt;= n; i++){&#xA;        x_test[i] = -1 + i*h_test;&#xA;    }&#xA;&#xA;    // Compute the Lagrange polynomial single&#xA;    float f0_eval[n+1];&#xA;    float f1_eval[n+1]; &#xA;    float f2_eval[n+1];&#xA;    for(int i = 0; i &amp;lt;= n; i++){&#xA;        f0_eval[i] = LagrangeQuad(x,x_test[i],f_0,n);&#xA;        cout &amp;lt;&amp;lt; f0_eval[i] &amp;lt;&amp;lt; endl;&#xA;    }&#xA;&#xA;    return 0;&#xA;}&#xA;&#xA;float LagrangeQuad(float x[], float x_eval, float f[], int n){&#xA;    float fun;&#xA;    float fun1;&#xA;    float fun2;&#xA;    float xdiff;&#xA;    float xdiff1;&#xA;    float xdiff2;&#xA;    float xdiff3;&#xA;    float xdiff4;&#xA;    float xdiff5;&#xA;    float result;&#xA;&#xA;    for(int i = 0; i &amp;lt; n; i++){&#xA;        if(x_eval &amp;gt;= x[i] &amp;amp;&amp;amp; x_eval &amp;lt;= x[i+1] ){&#xA;        fun = f[i];&#xA;        fun1 = f[i+1];&#xA;        fun2 = f[i+2];&#xA;        xdiff = (x_eval - x[i+1])*(x_eval - x[i+2]);&#xA;        xdiff1 = (x[i] - x[i+1])*(x[i] - x[i+2]);&#xA;        xdiff2 = (x_eval - x[i])*(x_eval - x[i+2]);&#xA;        xdiff3 = (x[i+1] - x[i])*(x[i+1] - x[i+2]);&#xA;        xdiff4 = (x[i+2] - x[i])*(x[i+2] - x[i+1]);&#xA;        }&#xA;    }&#xA;    result = fun*(xdiff/xdiff1) + fun1*(xdiff2/xdiff3) + fun2*(xdiff4/xdiff5);&#xA;&#xA;    return result;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="17826" LastEditDate="2015-10-09T18:21:43.437" LastActivityDate="2015-10-09T18:52:33.320" Title="Piecewise Quadratic Lagrange" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20976" PostTypeId="2" ParentId="20975" CreationDate="2015-10-09T18:17:34.440" Score="1" Body="&lt;p&gt;Ok in your Langrange function, should it be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(int i = 0; i &amp;lt; (n-1)/2; i++){&#xA;    if(x_eval &amp;gt;= x[2*i] &amp;amp;&amp;amp; x_eval &amp;lt;= x[2*i+2] ){&#xA;        fun = f[2*i];&#xA;        fun1 = f[2*i+1];&#xA;        fun2 = f[2*i+2];&#xA;        xdiff = (x_eval - x[2*i+1])*(x_eval - x[2*i+2]);&#xA;        xdiff1 = (x[2*i] - x[2*i+1])*(x[2*i] - x[2*i+2]);&#xA;        xdiff2 = (x_eval - x[2*i])*(x_eval - x[2*i+2]);&#xA;        xdiff3 = (x[2*i+1] - x[2*i])*(x[2*i+1] - x[2*i+2]);&#xA;        xdiff4 = (x_eval - x[2*i])*(x_eval - x[2*i+1]);&#xA;        xdiff5 = (x[2*i+2] - x[2*i])*(x[2*i+2] - x[2*i+1]);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also I think you want an odd number of points, like [x0, x1, x2, x3, x4] that way there is an even number of intervals (in this case 2). In the question it says $n$ must be even but I think this might be referring to the number of intervals and not the number of points? I could be wrong though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&#xA;I think I see now what you did. You make n=100 (even) but have the arrays be length n+1 (odd) which is fine. Just make sure you call your langrange function like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LagrangeQuad(x,x_test[i],f_0,n+1);   //make sure its n+1, not n&#xA;                                     //your arrays are of length n+1 remember!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-10-09T18:52:33.320" LastActivityDate="2015-10-09T18:52:33.320" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20978" PostTypeId="1" CreationDate="2015-10-10T08:50:10.753" Score="3" ViewCount="715" Body="&lt;p&gt;Given a sparse rectangular matrix $A$ (let's say, with dimension $n,m$ and number of non-zero elements $O(n)\sim O(m)$) with entries in $\mathbb Z/2\mathbb Z$  I'm looking for a basis of the kernel as a $\mathbb Z/2\mathbb Z$ vectorial space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that applying Gauss I can easily find them, but it doesn't use the hypothesis that $A$ is sparse, and all the libraries I found use Gauss, and are optimized for dense matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asking you, is there an algorithm (or a C++ library) that does this task using the sparse structure of the matrix, and performs better than Gauss?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, which library will let me work with matrices on an arbitrary field?&lt;/p&gt;&#xA;" OwnerUserId="17807" LastEditorUserId="17807" LastEditDate="2015-10-10T17:16:29.383" LastActivityDate="2015-10-10T17:16:29.383" Title="Kernel of a Sparse Matrix" Tags="|linear-algebra|matrix|c++|libraries|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20979" PostTypeId="2" ParentId="20978" CreationDate="2015-10-10T11:00:07.213" Score="2" Body="&lt;p&gt;To address your last question, the library &lt;a href=&quot;http://www.linalg.org/&quot; rel=&quot;nofollow&quot;&gt;LinBox&lt;/a&gt; contains code for linear algebra (dense and sparse) over finite fields.  There is an &lt;a href=&quot;http://www.linalg.org/linbox-html/examples_2nullspacebasis_8_c-example.html&quot; rel=&quot;nofollow&quot;&gt;example of computing the basis of the nullspace of a sparse matrix over $\mathbb{Z}/p^k\mathbb{Z}$&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you were doing your computation over $\mathbb{R}$, I would have suggested to use a sparse singular value decomposition to get the right singular vectors corresponding to the zero (or smallest) singular values.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2015-10-10T11:05:38.973" LastActivityDate="2015-10-10T11:05:38.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20980" PostTypeId="2" ParentId="10869" CreationDate="2015-10-11T13:17:56.173" Score="0" Body="&lt;p&gt;What  Hristo Iliev said above about 64-bit compilers preferring the SSE and AVX instructions over the FPU (AKA NDP) is absolutely true, at least for Microsoft Visual Studio 2013.  However, for the double precision floating-point operations I was using I found it actually faster, as well as in theory more accurate, to use the FPU.  If it is important to you, I would suggest testing various solutions first, before choosing a final approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When working in Java, I very frequently use the arbitrary-precision BigDecimal data type.  It is just too easy, and one usually does not notice the speed decrease.  Computing the transcendental functions with infinite series and sqrt using Newton's method can take a millisecond or more, but it is doable and quite accurate.&lt;/p&gt;&#xA;" OwnerUserId="17838" LastActivityDate="2015-10-11T13:17:56.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20982" PostTypeId="1" AcceptedAnswerId="21210" CreationDate="2015-10-11T20:13:31.960" Score="5" ViewCount="1525" Body="&lt;p&gt;I am trying to implement adaptive mesh refinement for a finite element code. The code uses (at least for now) linear triangles and so when I do the mesh refinement I want the triangular mesh to remain conformal, i.e. no hanging nodes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2dbgB.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2dbgB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am planning to use the longest-edge bisection method by Rivara [33,34]. Unfortunately I have not been able to access these papers since I have not found a freely downloadable pdf. I have looked at a couple other papers/resources that describe this method, i.e. &lt;a href=&quot;http://www.cs.rpi.edu/~flaherje/pdf/fea8.pdf&quot; rel=&quot;noreferrer&quot;&gt;fea8.pdf&lt;/a&gt; as well as Wolfgang Bangerth's video series &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;noreferrer&quot;&gt;lecture 14 - 18&lt;/a&gt;, however I am having trouble with the details on how this algorithm would actually be implemented. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have done so far is assume an initial coarse mesh, i.e. something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/smZIr.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/smZIr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my program my mesh is represented by an array of binary trees where each original coarse triangle is the root node of one of the binary trees in the array. So for example in the above mesh I would have an array of binary trees of size two. Graphically my trees would look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/znO6g.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/znO6g.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now say for example triangle $e1$ is flagged for refinement. We bisect this triangle by the longest edge to create two new sub triangles:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1oRjt.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1oRjt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and tree structure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xr02Y.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xr02Y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that triangle $f1$ has a hanging node, and worst still the root node in the tree for triangle $f1$ doesn't &quot;know&quot; it has a hanging node along that edge nor does it &quot;know&quot; that this hanging node has number 5. One solution that I came up with would be to have each node representing a triangle to also have additional pointers that point to the neighbours of that triangle. This however quickly makes the array of trees really complicated with pointers between nodes all over the place trying to keep track of which triangles are neighbours of other triangles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In C++ for example my binary tree data structure looks something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;class T&amp;gt;&#xA;class Tree&#xA;{&#xA;  private:&#xA;    struct Node&#xA;    {&#xA;      T element;&#xA;      Node *parent;&#xA;      Node *left;&#xA;      Node *right;&#xA;&#xA;      Node *neigh01;&#xA;      Node *neigh12;&#xA;      Node *neigh20;&#xA;&#xA;      int hnode01;   //store hanging node numbers for this node&#xA;      int hnode12;  &#xA;      int hnode20;   &#xA;&#xA;      bool refine;   //refinement flag saying this node is in need of refinement&#xA;&#xA;      Node(T e) : element(e), parent(NULL), left(NULL), right(NULL), neigh01(NULL), neigh12(NULL), neigh20(NULL), hnode01(-1), hnode12(-1), hnode20(-1)&#xA;      {&#xA;      }&#xA;    };&#xA;&#xA;    //rest of class ...constuctors, methods, etc...&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where you can see I have included additional node pointers that attempt to keep track of where neighbor nodes are stored. Actually implementing this bisection method for adaptive mesh refinement appears to be quite complicated and I am not sure how to go about doing it. Is my idea of having additional pointers to keep track of neighbours the right way to go about solving this? Does anyone have freely available sources that explain the bisection algorithm in more detail, i.e. explaining the data structures used etc? Can anyone point me to an algorithm for adaptive mesh refinement using bisection? The one written in &lt;a href=&quot;http://www.cs.rpi.edu/~flaherje/pdf/fea8.pdf&quot; rel=&quot;noreferrer&quot;&gt;fea8.pdf&lt;/a&gt; seems to indicate that we first run through all nodes that need refinement and split them. We then go through (iteratively?) to correct for hanging nodes. Is this correct/what is usually done? Is my idea of represent my mesh as an array of binary trees a good/standard way of representing a mesh that will be adaptively refined? Basically I need any help I can get in implementing this adaptive mesh refinement algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[33] M.C. Rivara. Design and data structures of a fully adaptive multigrid finite element software. ACM Transactions on Mathematical Software, 10:242 264, 1984.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[34] M.C. Rivara. Mesh refinement processes based on the generalized bisection of simplices. SIAM Journal on Numerical Analysis, 21:604 613, 1984. &lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-10-11T22:47:36.100" LastActivityDate="2016-10-12T17:59:52.490" Title="How to implement adaptive mesh refinement using conformal triangles" Tags="|finite-element|algorithms|c++|adaptive-mesh-refinement|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20983" PostTypeId="1" AcceptedAnswerId="33901" CreationDate="2015-10-12T06:33:15.817" Score="4" ViewCount="371" Body="&lt;p&gt;I am looking at approximating my function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; using a Chebyshev and Legendre series and I ran into this question.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is interpolation using &lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt; Chebyshev nodes the same as representing the function using the first &lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt; Chebyshev coefficients in its Chebyshev series, i.e., is &lt;span class=&quot;math-container&quot;&gt;$$f(x) = \sum_{k=0}^n f(x_k) l_k(x)$$&lt;/span&gt;where &lt;span class=&quot;math-container&quot;&gt;$x_k$&lt;/span&gt; are the Chebyshev nodes and &lt;span class=&quot;math-container&quot;&gt;$l_k(x)$&lt;/span&gt; is the appropriate Lagrange polynomial associated with the &lt;span class=&quot;math-container&quot;&gt;$k^{th}$&lt;/span&gt; node, the same as &lt;span class=&quot;math-container&quot;&gt;$$f(x) = \sum_{k=0}^n a_k T_k(x)$$&lt;/span&gt;where &lt;span class=&quot;math-container&quot;&gt;$T_k(x)$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$k^{th}$&lt;/span&gt; Chebyshev polynomial and &lt;span class=&quot;math-container&quot;&gt;$a_k = \dfrac{\langle f, T_k\rangle}{\langle T_k, T_k\rangle}$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Here the inner product is the inner product using the appropriate weight function for the polynomials, i.e., &lt;span class=&quot;math-container&quot;&gt;$$\langle f, T_k \rangle = \int_{-1}^1 \dfrac{f(y)T_k(y)}{\sqrt{1-y^2}}dy$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Is the same true for Legendre interpolation and Legendre expansion as well, where the inner product is &lt;span class=&quot;math-container&quot;&gt;$$\langle f, P_k \rangle = \int_{-1}^1 f(y)P_k(y)dy$$&lt;/span&gt;? If so, could someone direct me to the proof of this?&lt;/p&gt;&#xA;" OwnerUserId="15195" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2019-11-28T13:58:30.357" Title="Chebyshev and Legendre expansions" Tags="|numerical-analysis|polynomials|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20985" PostTypeId="2" ParentId="20982" CreationDate="2015-10-12T13:25:05.810" Score="3" Body="&lt;p&gt;The way this is typically handled is by running a pass before you actually split cells that determines which &lt;em&gt;other&lt;/em&gt; cells also have to be refined. In pseudo-code, this could look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (cell in cells)&#xA;  if (cell-&amp;gt;is_marked_for_refinement &amp;amp;&amp;amp; cell-&amp;gt;edge_to_split == unassigned)&#xA;  {&#xA;     e = longest edge of cell (0...2);&#xA;     cell-&amp;gt;set_edge_to_split(e);&#xA;     n = cell-&amp;gt;neighbor(e);&#xA;     n-&amp;gt;mark_for_refinement();  // if we split this cell along edge e,&#xA;                                // then we also need to split cell n&#xA;     n_of_n = n-&amp;gt;get_neighbor_index(cell); // find out along which edge&#xA;                                // of n the current cell is&#xA;     n-&amp;gt;set_edge_to_split (n_of_n);&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This process ensures that you first mark all cells that actually need to be split, before you do the splitting.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-12T13:25:05.810" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20986" PostTypeId="2" ParentId="20983" CreationDate="2015-10-12T13:54:24.120" Score="5" Body="&lt;p&gt;What you are doing here is projecting a function onto a basis of a finite dimensional space of polynomials. All you use in the formulas you show is that the basis $T_k$ is orthogonal, i.e., that $\left&amp;lt;T_i,T_j\right&amp;gt;=0$ if $i\neq j$ with regard to some scalar product $\left&amp;lt;\cdot,\cdot\right&amp;gt;$. All of this naturally generalizes to other orthogonal bases of polynomial spaces.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-12T13:54:24.120" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20987" PostTypeId="1" CreationDate="2015-10-12T15:37:31.640" Score="3" ViewCount="218" Body="&lt;p&gt;I have an existing FEM code that solves the linear elasticity problem. I would like to use the same code for large strain rates, still using a simple material law (Saint Venant–Kirchhoff model). [The problem I want to solve is a contact problem in which strain is larger than 1 very locally, close to the contact point]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe I have two steps to take: take into account the nonlinear part of strain $\nabla u \cdot \nabla u^T$, and solve the mechanical balance in the deformed configuration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this be done by successive iterations of the linear elasticity problem, e.g. using iterative mesh deformation and a fixed-point algorithm for the nonlinear strain term? Does this converge? References which would highlight the steps to be taken from small-strain elasticity are welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would for instance like to know whether such an algorithm would work (there may be a couple mistakes in the details, for now it's rather the spirit of it I'm enquiring about):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve $K u + \nabla u^T\cdot \nabla u= Mf$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $(f^i)_{i\leq n}$ be a sequence of functions with $f_n = f$, with each increment small enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $i=0$, $U^0=0$, $K^0=K$.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Solve linear elasticity problem $K^i \delta u^i = M f^i$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Construct configuration $\Omega^{i+1}$, and a change of variable $X^{i+1}$ from $\Omega^{i}$ to $\Omega^{i+1}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;let $U^{i+1} = U^i \circ X^{i+1} + \delta u^i$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate $K^{i+1} = K_{\Omega^{i+1}} + \nabla {U^{i+1}}^T\cdot \nabla$  (there's probably something to do here as we develop $u=U+\delta u$ in the double product, it doesn't write out nicely in tensor notation but should in the linear problem)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Again, the above is not to be taken as something thought over but rather the sort of thing I'd like to find in some report/textbook/publication. But maybe there's a crucial problem I don't see which would be the reason why finite strain codes are not presented as an extension of small strain ones?&lt;/p&gt;&#xA;" OwnerUserId="17850" LastEditorUserId="17850" LastEditDate="2015-10-12T21:13:12.153" LastActivityDate="2015-10-12T21:13:12.153" Title="Finite strain FEM using an existing code that solves small strain elasticity" Tags="|pde|finite-element|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20988" PostTypeId="2" ParentId="19355" CreationDate="2015-10-12T16:01:58.427" Score="2" Body="&lt;p&gt;There are more recent papers on Space-Time Methods. There's one from &lt;a href=&quot;http://www.degruyter.com/view/j/cmam.2015.15.issue-4/cmam-2015-0026/cmam-2015-0026.xml?format=INT&quot; rel=&quot;nofollow&quot;&gt;Steinbach, Space-Time Finite Element&lt;/a&gt; and another from &lt;a href=&quot;http://arxiv.org/pdf/1509.02008v1.pdf&quot; rel=&quot;nofollow&quot;&gt;Langer et. al, Space-Time Isogeometric Analysis&lt;/a&gt; all addressing Parabolic Evolution Problems. In both articles, they describe vividly the variational formulations but in different settings. As the titles suggest, the former uses FEM and latter IgA. I think this gives good information particularly on what you seek.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the last chapter of the second edition of the monograph &lt;a href=&quot;http://www.springer.com/us/book/9783540346586&quot; rel=&quot;nofollow&quot;&gt;Numerical Mathematics,Quatteroni et. al&lt;/a&gt;, there's a section on Space-Time which might also be helpful especially with connections to the $\theta-$schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tensor product Space-Time implementation is very different from non-tensor based ones. The latter is a bit tricky especially for the FEM.&lt;/p&gt;&#xA;" OwnerUserId="7984" LastActivityDate="2015-10-12T16:01:58.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20989" PostTypeId="2" ParentId="20946" CreationDate="2015-10-12T17:15:53.497" Score="1" Body="&lt;p&gt;For numerical simulations of hyperbolic equations finite volume methods have been used in the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://depts.washington.edu/clawpack/&quot; rel=&quot;nofollow&quot;&gt;clawpack&lt;/a&gt; and the book Finite-Volume Methods for Hyperbolic Problems from its author Randall J. LeVeque very helpful for implementing my own code.&#xA;Since finite volume and finite differences are not so different, changing to the finite volume code might be a way to solve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With finite differences one has either problems with numerical dispersion or with numerical diffusion.&#xA;To counter these different flux limiter functions are used to handle this short coming in finite volume codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also just implement your system with clawpacks python api, for reference, check out &lt;a href=&quot;http://www.clawpack.org/pyclaw/gallery/gallery_all.html#id14&quot; rel=&quot;nofollow&quot;&gt;Gallery acoustics&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2015-10-12T17:15:53.497" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="20990" PostTypeId="2" ParentId="20987" CreationDate="2015-10-12T19:27:10.340" Score="2" Body="&lt;p&gt;We only know how to solve general linear problems efficiently, so every algorithm for general nonlinear problems needs to reduce them to a sequence of linear problems. This is what we do in fixed point methods, Newton's method, etc. In your context, all methods to solve such problems reduce the problem to a sequence of (linear) elasticity problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonlinear mechanics is not a field I know really well, so I don't know of many examples. I could suggest looking at &lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/step_44.html&quot; rel=&quot;nofollow&quot;&gt;the step-44 tutorial program of deal.II&lt;/a&gt;, though, which solves what I think comes closest to what you want to do. (Disclaimer: I'm one of the principal authors of deal.II.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-12T19:27:10.340" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20991" PostTypeId="1" CreationDate="2015-10-12T21:12:52.040" Score="9" ViewCount="371" Body="&lt;p&gt;I can't imagine I'm the first to think about the following problem, so I'll be satisfied with a reference (but a complete, detailed answer is always appreciated):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you have a symmetric positive definite $\Sigma \in \mathbb{R}^{n \times n}$. $n$ is thought of as very large, so holding $\Sigma$ in memory is impossible. You can, however, evaluate $\Sigma x$, for any $x \in \mathbb{R}^{n}$. Given some $x \in \mathbb{R}^{n}$, you'd like to find $x^t\Sigma^{-1}x$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first solution that comes to mind is to find $\Sigma^{-1}x$ using (say) conjugate gradients. However, this seems somewhat wasteful - you seek a scalar and in the process you find a gigantic vector in $\mathbb{R}^{n}$. It seems to make more sense to come up with a method to calculate the scalar directly (i.e. without passing through $\Sigma^{-1}x$). I am looking for this kind of method.&lt;/p&gt;&#xA;" OwnerUserId="11717" LastEditorUserId="11717" LastEditDate="2015-10-12T22:05:32.287" LastActivityDate="2015-10-13T19:43:53.583" Title="Iterative &quot;solver&quot; for $x^t \Sigma^{-1} x$" Tags="|linear-algebra|numerical-analysis|iterative-method|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="20992" PostTypeId="2" ParentId="20921" CreationDate="2015-10-13T00:59:29.493" Score="1" Body="&lt;p&gt;The classic book in B-Splines is de Boor, C. (1978). A Practical Guide to Splines. New York: Springer-Verlag, but I spent many weeks on it and was not able to make his algorithms work.  The algorithm in Phillips, G. M., &amp;amp; Taylor, P. J. (1996). Splines and Other Approximations Chapter 6 of Theory and Applications of Numerical Analysis (2nd ed., pp. 131-159). London, San Diago: Academic Press, works well and is easy to understand.&lt;/p&gt;&#xA;" OwnerUserId="17838" LastActivityDate="2015-10-13T00:59:29.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20994" PostTypeId="2" ParentId="20674" CreationDate="2015-10-13T12:45:00.680" Score="1" Body="&lt;p&gt;In the end, I did a check to determine if the element was skew or not. If skew, I solved the transformation via a closed-form solution. If not, I solved it using Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closed form solution I used is similar to that suggested by @nicoquaro. &lt;/p&gt;&#xA;" OwnerUserId="17482" LastEditorUserId="17482" LastEditDate="2015-10-31T00:33:16.783" LastActivityDate="2015-10-31T00:33:16.783" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="20995" PostTypeId="2" ParentId="20165" CreationDate="2015-10-13T17:15:13.497" Score="3" Body="&lt;p&gt;Well there's always the general approach. Let's say we have a particle at a position $\bf r$ and the lattice vectors, i.e. the vectors that forming the sides of the simulation cell, are $\bf a_i$. We can then express $\bf r$ in the basis of $\bf a_i$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${\bf r}=\sum_i f_i{\bf a_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can express this as a matrix vector multiply $${\bf r}={\bf Af}$$ where A is a matrix whose columns are the lattice vectors, and it can be see that f are the fractional coordinates. So defining ${\bf B}={\bf A}^{-1}$ you can find the factional coordinates simply by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${\bf f}={\bf Br}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we want all our images to be in the same cell ... One way of doing this is to ensure $0\le f_i &amp;lt; 1$, and to achieve this we can simply modify $f_i$ to be $f_i - floor(f_i)$ as this is simply translates the particles to equivalent position in the reference cell due to the periodic boundary conditions. So for arbitrary cell (in arbitrary dimensionality) the recipe is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Form ${\bf A}$&lt;/li&gt;&#xA;&lt;li&gt;Invert ${\bf A}$ to form ${\bf B}$, this should always be well determined numerically as the determinant of the matrix is the volume of the cell, and so should be far from zero. You also probably want these vectors anyway for the Ewald sum ...&lt;/li&gt;&#xA;&lt;li&gt;From the position of a particle at $\bf r$ form $\bf f$ by ${\bf f}={\bf Br}$&lt;/li&gt;&#xA;&lt;li&gt;Translate into the reference cell by $g_i=f_i-floor(f_i)$&lt;/li&gt;&#xA;&lt;li&gt;Form the new position in real space ${\bf t}$ by ${\bf t}={\bf Ag}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17475" LastActivityDate="2015-10-13T17:15:13.497" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20996" PostTypeId="2" ParentId="20991" CreationDate="2015-10-13T19:43:53.583" Score="2" Body="&lt;p&gt;I don't think I've heard of any method that does what you want without actually solving $y=\Sigma^{-1}x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only alternative I can offer is if you knew something about the eigenvectors and -values of $\Sigma$. Say you knew that they are $\lambda_i,v_i$, then you can represent $\Sigma=V^T L V$ where the columns of $V$ are the $v_i$, and $L$ is a diagonal matrix with the eigenvalues on the diagonal. Consequently, you have that $\Sigma^{-1}=V^T L^{-1} V$ and you get that&#xA;$$&#xA;  x^T \Sigma^{-1} x = x^T V^T L^{-1} V x = \sum_i \lambda_i^{-1} (v_i^T x)^2.&#xA;$$&#xA;This would of course require you to store &lt;em&gt;all&lt;/em&gt; eigenvalues, i.e., a full matrix $V$. But, if you happened to know that only a few of the eigenvalues of $\Sigma$ are small, say the first $m$, and the rest is so large that you can neglect all terms with $\lambda^{-1}_i$ for $i&amp;gt;m$, then you can approximate&#xA;$$&#xA;  x^T \Sigma^{-1} x = \sum_{i=1}^n \lambda_i^{-1} (v_i^T x)^2 \approx \sum_{i=1}^m \lambda_i^{-1} (v_i^T x)^2.&#xA;$$&#xA;This then only requires you to store $m$ vectors, instead of all $n$ eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in practice it is often equally or more difficult to compute the eigenvalues and eigenvectors, compared to simply solving $y=\Sigma^{-1}x$ multiple times.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-13T19:43:53.583" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="20997" PostTypeId="2" ParentId="20933" CreationDate="2015-10-14T02:34:28.210" Score="3" Body="&lt;p&gt;Probably the Cholesky factor is what you are looking for, as noted by the other answers. However the rectangular matrix $C=[A,B]$ does satisfy $$CC^T=AA^T+BB^T$$&#xA;so is technically a closed form solution.&lt;/p&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2015-10-14T02:34:28.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20998" PostTypeId="1" CreationDate="2015-10-14T08:39:37.860" Score="4" ViewCount="900" Body="&lt;p&gt;Suppose I want to simulate the well-known diffusion partial differential equation in 2D, for example with finite elements or finite differences. Can I directly take physical &lt;strong&gt;diffusion coefficients from the 3D real world or do I have to adjust them for 2D&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2015-10-18T10:50:10.357" Title="Diffusion coefficient when simulating in 2D" Tags="|diffusion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="20999" PostTypeId="1" CreationDate="2015-10-14T09:17:59.967" Score="4" ViewCount="478" Body="&lt;p&gt;I need to compute the pseudo-inverse of a very large rectangular dense matrix without any special structure or properties. I run out of memory/computing power  and have no access to a large parallel computing resource.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the good news is that  I need only one column of the result at a time (for the subsequent calculations). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any iterative algorithm that can compute the 'kth' column  (or at-least progressively build up the first 'k' columns of the pseudo-inverse ?). I'd appreciate any inputs/thoughts on this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I am using MATLAB for now, but the programming environment does not really matter.&lt;/p&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2015-10-14T12:52:23.883" Title="Obtaining column vectors of pseudo-inverse of a matrix" Tags="|linear-algebra|matlab|matrix|svd|inverse|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21000" PostTypeId="2" ParentId="20998" CreationDate="2015-10-14T09:27:23.000" Score="4" Body="&lt;p&gt;That's a modelling problem, not a CS one. This will depend on what is diffusing and on the reasons why 2D is relevant. You may want to give physical details and ask this on physics.SE. Alternatively if you just want to get your hand in with solving diffusion, then the numbers don't have to reflect a physical fact (especially if you have diffusion only, the coefficient will just set the timescale).&lt;/p&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2015-10-14T09:27:23.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21002" PostTypeId="2" ParentId="20999" CreationDate="2015-10-14T12:52:23.883" Score="3" Body="&lt;p&gt;The pseudo inverse $A^+$ fulfills: $A^+b$ is the minimimum norm solution of the least squares problem $\min_x \|Ax-b\|_2^2$. Hence, to calculate $A^+e_k$ by solving the respective optimization problem (which has a different structure, depending on the format of $A$, as Brian Borchers commented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also use an iterative method to solve $A^TA x = A^Te_k$ in the overdetermined case or solve $AA^Ty = e_k$ and $x = A^T y$ in the underdetermined case, respectively. The method of choice depends on further properties of the matrix…&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2015-10-14T12:52:23.883" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21003" PostTypeId="2" ParentId="7248" CreationDate="2015-10-14T13:01:26.210" Score="0" Body="&lt;p&gt;I'm coming a long time after the OP asked the question, but I think a general answer is still missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your code allows you to interact with the linear problem you obtain, the simplest is that you remove the lines and columns corresponding to the nodes in the cylinder. You then solve on the unknown part of the velocity only, and you need to rearrange the output into the numbering of the cartesian grid. This is what is done e.g. in the FEM software rheolef to impose Dirichlet boundary conditions, see &lt;a href=&quot;http://www-ljk.imag.fr/membres/Pierre.Saramito/rheolef/&quot; rel=&quot;nofollow&quot;&gt;http://www-ljk.imag.fr/membres/Pierre.Saramito/rheolef/&lt;/a&gt; (first example of the manual), it works also for FDM. But your linear solver has to be not too specialized of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way is to have a Lagrange multiplier for each node in the cylinder: your constraint is $v_{i,j,k} = 0$ for $x_{i,j,k} \in C$, so you just add lines and columns of 0's with a $1$ at the relevant position for ${i,j,k}$ (e.g. $k*NY*(j*NX+i)$) to the original problem. If your solver cannot solve this new full problem, you can use Uzawa algorithm to solve by iterations of the initial problem and updates of the Lagrange multipliers. See Fortin and Glowinski 1983, Augmented Lagrangian methods&lt;/p&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2015-10-14T13:01:26.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21004" PostTypeId="1" AcceptedAnswerId="21038" CreationDate="2015-10-14T18:42:34.333" Score="3" ViewCount="186" Body="&lt;p&gt;I'm looking for some methods that could be directly applied to the PDE&#xA;$$&#xA;\frac{\partial u}{\partial t} + &#xA;\frac{(u_x)^2}{2} = 0\tag{*}&#xA;$$&#xA;&lt;strong&gt;without&lt;/strong&gt; converting it by $v = u_x$ to the Hopf equation&#xA;$$&#xA;\frac{\partial v}{\partial t} + &#xA;\frac{\partial (v^2/2)}{\partial x} = 0.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, a WENO scheme is perfect, but I'm wondering how to apply a limiter or a numerical flux to $(*)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simple centered scheme&#xA;$$&#xA;\dot u_i(t) = -\frac{1}{2}\left(\frac{u_{i+1}(t) - u_{i-1}(t)}{2h}\right)^2&#xA;$$&#xA;expectedly oscillates when shock (on $v = u_x$) is formed.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/f2Ims.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f2Ims.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17125" LastEditorUserId="17125" LastEditDate="2015-10-14T18:55:08.390" LastActivityDate="2015-10-18T09:49:22.337" Title="Numerical methods for the $u_t + \frac{(u_x)^2}{2} = 0$ equation" Tags="|hyperbolic-pde|oscillations|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21005" PostTypeId="2" ParentId="21004" CreationDate="2015-10-14T18:46:31.397" Score="1" Body="&lt;p&gt;The simplest scheme you could employ is an upwind scheme. It's first-order and introduces artificial viscosity/diffusion, but doesn't require limiters. Probably the next simplest class would be Lax-Wendroff schemes, for which you can find a comprehensive explanation in LeVeque's book on Finite Volume schemes.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-10-14T18:46:31.397" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21006" PostTypeId="1" CreationDate="2015-10-14T21:45:51.827" Score="6" ViewCount="135" Body="&lt;p&gt;Let $A\in \mathbb{R}^{n\times n}$ be symmetric positive semidefinite, and $B\in \mathbb{R}^{n\times n}$ be symmetric positive definite.  Suppose $B$ is block diagonal so it is easy to invert.  (We have access to the entries of $B^{-1}$ in the computer).  Let $\omega \in \mathbb{R}$.  Suppose I want to solve the following symmetric and positive definite system&#xA;$$&#xA;((A-\omega^2 B)B^{-1}(A-\omega^2 B)+\omega^2 B)x = b,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for a range of values of $\omega$.  Note that the RHS, $b$, may depend on $\omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in strategies to compute $x(\omega)$ repeatedly.  As an idealization, suppose I only care about correctness and the time it takes to produce a solution from a given $\omega$.  I mean, any time spent performing decompositions and preprocessing will be considered free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What decompositions / strategies would be useful?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Edit&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I have been playing around with this today.  This is my proposed solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use a combination of a Cholesky decomposition and an eigen decomposition.  Let $C\in \mathbb{R}^{n\times n}$ be a lower triangular matrix such that $CC^\mathrm{T} = B$.  Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*}&#xA;((A&amp;amp;-\omega^2 B)B^{-1}(A-\omega^2 B)+\omega^2 B)\\&#xA;&amp;amp;= \cdots \text{just algebra}\\&#xA;&amp;amp;= C\left((C^{-1} A C^{-\mathrm{T}} - \omega^2 I)(C^{-1} A C^{-\mathrm{T}} - \omega^2 I)+ \omega^2 I) \right)C^{\mathrm{T}}&#xA;\end{align*}&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we take an eigen decomposition of $C^{-1} A C^{-\mathrm{T}}$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C^{-1}A C^{-\mathrm{T}} = Q\Lambda Q^\mathrm{T}$ for $Q\in \mathrm{R}^{n\times n}$, orthonormal, $\Lambda\in \mathbb{R}^{n\times n}$, diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then &#xA;\begin{align*}&#xA; C\big((C^{-1} &amp;amp;A C^{-\mathrm{T}} - \omega^2 I)(C^{-1} A C^{-\mathrm{T}} - \omega^2 I)+ \omega^2 I) \big)C^{\mathrm{T}} \\&#xA;&amp;amp;= \cdots \text{(just algebra)}\\&#xA;&amp;amp;= CQ \big((\Lambda - \omega^2 I)(\Lambda- \omega^2I)+ \omega^2 I\big) Q^\mathrm{T} C^\mathrm{T}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, this system can be trivially inverted, and none of the decompositions depends on $\omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real question is if any of this can be used to design a preconditioner for a system like this that is too large to be factored in this way.  Any ideas on this would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2015-10-15T03:23:08.530" LastActivityDate="2015-10-15T03:23:08.530" Title="What strategies / decompositions would be useful to solve the following linear system repeatedly if I only care about time to solution?" Tags="|linear-algebra|linear-solver|eigensystem|block-decomposition|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21007" PostTypeId="2" ParentId="10678" CreationDate="2015-10-15T06:13:32.220" Score="0" Body="&lt;p&gt;Every element has associated with it a displacement model which expresses the variation of field variable(dependent variable ) in terms of generalised coefficients and independent variables(x,y,z)&#xA;eg: 1D u(x) = a0 +a1x   for 2 noded linear element&#xA;       u(x) = a0 +a1x + a3x^2 for 3 noded quadratic element&#xA; and so on. Here ai s are the generalised coefficients&#xA;Then we eliminate ai s and express the variation of field variable in terms of shape functions and nodal values of the field variable.&#xA;eg: u(x) = N1 u1 + N2 u2&#xA; The function that relates the variation of field variable with the nodal value of the field variable is called the “SHAPE FUNCTION”.  The number of shape functions will depend upon the number of nodes and the number of variables per node.&#xA;The shape functions can therefore be viewed as functions, which denote the contribution of each nodal value at internal points of the element.&#xA;For a two noded element&#xA;At node 1 the contribution of N1 is unity &amp;amp; that of N2 is zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At node 2 the contribution of N2 is unity &amp;amp; that of N1 is zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the midpoint of the element both nodes have equal weightage or influence.&#xA;So shape functions indicate not only how the field variable varies over the element but also how much influence  each nodal value of field variable has at internal points of the element.&#xA;Happy learning:)&lt;/p&gt;&#xA;" OwnerUserId="17884" LastActivityDate="2015-10-15T06:13:32.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21008" PostTypeId="1" CreationDate="2015-10-15T13:00:06.237" Score="2" ViewCount="52" Body="&lt;p&gt;Are there any limitations of particle in cell (PIC) method for high density plasma? To be more specific, is modeling of a narrow channel of high density plasma possible or are there any limitations connected with PIC approximation?&lt;/p&gt;&#xA;" OwnerUserId="11686" LastEditorUserId="1804" LastEditDate="2015-10-15T13:09:00.923" LastActivityDate="2015-10-15T13:09:00.923" Title="Limitations of particle in cell method for high density plasma" Tags="|simulation|modeling|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21009" PostTypeId="1" CreationDate="2015-10-15T13:42:06.417" Score="8" ViewCount="169" Body="&lt;p&gt;In the Non-Negative Matrix factorization (NMF), you basically compute an approximation of a given matrix $V \in \mathbb{R}_{+}^{n \times m}$ into matrices $W$ and $H$ such that $W \in \mathbb{R}_{+}^{n \times r}$ and $H \in \mathbb{R}_{+}^{r \times m}$. Or in other words, $V \approx WH$. For the Sparse NMF with sparsity parameter $\lambda\ge 0 $ and sparsity function $g$, you minimize a cost function of this form:&#xA;$$F(W, H) = \frac{1}{2}\|V - \bar W H\|^2 + \lambda\sum_{ij}g\left(H_{ij}\right) \qquad\text{where } \bar W =\frac W {\|W\|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And $\|\ldots\|$ can be any differentiable norm. For $g(H_{ij}) =H_{ij}$, the steps for updating $H$ and $W$ are suggested &lt;a href=&quot;http://www.maths.tcd.ie/~mnl/store/EggertKorner2004a.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; as follows  for : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}H_{ij} \longleftarrow H_{ij}&amp;amp;\odot\frac{V_i^T \bar W_j}{R_i^T\bar W_j + \lambda}\\&#xA;W_j \longleftarrow W_j&amp;amp;\odot\frac{\sum_i H_{ij}\left[V_i + \left(R_i^T\bar W_j\right)\bar W_j\right]}{\sum_i H_{ij}\left[R_i + \left(R_i^T\bar W_j\right)\bar W_j\right]}\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $R = WH$ and the matrix division and the $\odot$ multiplication  are done&#xA;element-wise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://etd.dtu.dk/thesis/211453/ep08_18_net.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;, it is suggested to accelerate the convergence as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To speed up the convergence, the multiplicative term can be exponentiated with an acceleration parameter $\delta$ larger than one. For each iteration the costfunction is evaluated and if the cost is smaller than in the previous iteration, $\delta$ is increased and if it is larger, then $\delta$ is decreased. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is done in the $H$ update step. Following the notations of the updates above, this will be as follows: &#xA;$$H_{ij} \longleftarrow H_{ij}\odot\left(\frac{V_i^T \bar W_j}{R_i^T\bar W_j + \lambda}\right)^\delta$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can one guarantee that with this acceleration we'll have fewer computational steps until convergence than without. Or how to ensure a minimum number of increasing and decreasing  of $\delta$ until convergence ?   &lt;/p&gt;&#xA;" OwnerUserId="10134" LastEditorUserId="10134" LastEditDate="2015-10-16T17:48:06.157" LastActivityDate="2015-10-16T17:48:06.157" Title="Accelerated convergence for Sparse NMF" Tags="|optimization|convergence|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21010" PostTypeId="1" CreationDate="2015-10-15T14:01:37.513" Score="4" ViewCount="311" Body="&lt;p&gt;I have to maximize&lt;br&gt;&#xA;$$f(x,y)=-\log(xy)$$&#xA;However I need it over the polytope $T=\mathrm{conv}\{(1/2,1/2),(1,2),(2,1)\}$ and this gives me problems: &#xA;Then I get&#xA;$$f_x(x,y)=1/x=0$$&#xA;$$f_y(x,y)=1/y=0 $$&#xA;and I get $x=1$ and $y=1$. However, I know the answer is $(1/2,1/2)$? But I simply don't see why. The point $(1,1)$ is in the convex hull so why isn't this the solution? Is it because it isn't on the &quot;border&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then need to state it's unique by using the below mentioned definitions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &#xA;$$V=\{x_1,...,x_m \}\subset\mathbb{R}$$ and &#xA;$$P=\mathrm{conv}\{x_1,\dots,x_m \}=\{\lambda_1 x_1+\dots+\lambda_m x_m \mid\lambda_i\geq 0,\lambda_1+...+\lambda_m=1\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimization problem &#xA;$$\max_{x\in V} f(x)$$&#xA;has a global maximum at $x \in V$, where&lt;br&gt;&#xA;                $$f(x_0 )=\max\{f(x_1 ),\dots,f(x_m)\}.$$&lt;br&gt;&#xA;This maximum is unique iff $f(x_0 )=f(x_j)$ for a unique $j=1,\dots,m$.&lt;/p&gt;&#xA;" OwnerUserId="17891" LastEditorUserId="1804" LastEditDate="2015-10-15T15:09:46.700" LastActivityDate="2015-11-14T15:57:30.937" Title="Maximizing a function over a polytope" Tags="|optimization|convex-optimization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21011" PostTypeId="2" ParentId="21010" CreationDate="2015-10-15T15:49:37.470" Score="2" Body="&lt;p&gt;The immediate answer to your question is that for constrained optimization problems, the solution does not have to satisfy $\partial f(x,y)/\partial x=0, \partial f(x,y)/\partial y=0$. Consequently, your initial approach to find the minimum won' work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the concrete problem you have here, the objective function is convex, and by definition so is the domain on which you want to minimize it. Consequently, the maximum will lie on the boundary, and you really only have to find the maximum of the objective function on each of the three lines that bound your polygon. Indeed, this will show that the maximum lies where you expect it to be.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-15T15:49:37.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21012" PostTypeId="1" CreationDate="2015-10-15T17:14:11.927" Score="8" ViewCount="4644" Body="&lt;p&gt;Can anybody help me? I heard that Gauss-Newton method compute an aproximation of the Hessian instead of the true Hessian, but, quasi-Newton method too, don't it? what is the differences between them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;any help would be appreciated :)&lt;/p&gt;&#xA;" OwnerUserId="17893" LastActivityDate="2017-10-27T17:11:58.047" Title="Difference between Gauss-Newton method and quasi-Newton method for optimization" Tags="|optimization|algorithms|newton-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21013" PostTypeId="2" ParentId="21012" CreationDate="2015-10-15T19:38:51.040" Score="17" Body="&lt;p&gt;Quasi-Newton methods construct an approximate Hessian for an arbitrary smooth objective function $f(x)$ using values of $\nabla f$ evaluated at the current and previous points.  At each iteration of the method the quasi-Newton approximate Hessian is updated using the gradient evaluated at the latest iterate, $x^{(k)}$.  These approximate Hessians aren't necessarily very good approximations to the actual Hessian of $f$, but they are sufficiently good that you can use them to obtain good search directions in the optimization algorithm.  There are many quasi-Newton methods, of which the most popular is probably BFGS (Broyden-Fletcher-Goldfarb-Shanno.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Gauss-Newton method is an approximate Newtons method that works only with objective functions that can be expressed as a sum of squares &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;f(x)=\sum_{i=1}^{m} f_{i}(x)^{2}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For objective functions of this particular form, if you let $J(x)$ (the Jacobian) be the matrix of first partial derivatives of $f_{i}(x)$ with respect to $x_{j}$, and let $F(x)$ be the vector of $f_{1}(x)$, $f_{2}(x)$, $\ldots$, $f_{m}(x)$, then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\nabla f(x)=J(x)^{T}F(x)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\nabla^{2} f(x) \approx 2J(x)^{T}J(x).&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Gauss-Newton method (and the Levenberg-Marquardt method) use this approximate Hessian and exact gradient in Newton's method.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approximate Hessian in the Gauss-Newton method is not of the same type as the quasi-Newton approximate Hessians (BFGS, DFP, etc.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that one can apply quasi-Newton methods like BFGS to nonlinear least squares problems but you can't apply the Gauss-Newton method to minimizing a general function $f(x)$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2017-10-27T17:11:58.047" LastActivityDate="2017-10-27T17:11:58.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21014" PostTypeId="1" AcceptedAnswerId="21015" CreationDate="2015-10-15T19:52:57.167" Score="5" ViewCount="65" Body="&lt;p&gt;User &lt;a href=&quot;https://scicomp.stackexchange.com/users/651/hardmath&quot;&gt;hardmath&lt;/a&gt;, provided an excellent overview of how to handle overflow when calculating the product of two functions, where one is likely to overflow: &lt;a href=&quot;https://scicomp.stackexchange.com/a/20913/9466&quot;&gt;https://scicomp.stackexchange.com/a/20913/9466&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer also discusses special cases involving $\textrm{erf}$ functions, which was discussed as an example in the question, so there is a discussion on handling $\textrm{erf}$ function underflow:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As a further precaution we will likely want to guard against &lt;em&gt;underflow&lt;/em&gt; in evaluating $\operatorname{erfc}(x)$.  The exponent range for double precision normal values goes down to binary (power of two) $-1022$.  Asymptotically:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;$$ \operatorname{erfc}(x) \approx O(1/x) e^{-x^2} $$&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consequently we would be at the brink of exponent underflow when $x \approx 25$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I would like to understand how to come up with the $x \approx 25$ value.&lt;/strong&gt; I am likely missing some math background here, so it suffices to point me to the topics that would help me understand how to come up with such estimates, &lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-10-15T20:24:56.187" Title="How would I figure out when a function would be at the &quot;brink of underflow&quot;?" Tags="|computer-arithmetic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21015" PostTypeId="2" ParentId="21014" CreationDate="2015-10-15T20:24:56.187" Score="5" Body="&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/Double-precision_floating-point_format&quot;&gt;double-precision floating-point arithmetic&lt;/a&gt;, the number $2^{-1022}$ is the smallest (normal) positive number that can be represented. Computing any smaller positive number results in a loss of precision, as it cannot be represented as a floating-point value. This either means the computed number will be denormal (partial loss of precision) or possibly it could be rounded to zero (complete loss of precision).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of erfc, you can solve for the largest $x$ such that $\mathrm{erfc}(x)$ does not underflow - such that $\mathrm{erfc}(x)$ can be represented with full accuracy. This is done by (numerically) solving the equation&#xA;$$ \mathrm{erfc}(x) = 2^{-1022}, \qquad\Longrightarrow\qquad x = 26.54,$$&#xA;or more easily by solving the approximate equation&#xA;$$ e^{-x^2}=2^{-1022}, \qquad\Longrightarrow\qquad x = \sqrt{1022\log 2} = 26.62.$$&#xA;The number $25$ then comes as a convenient estimate for when problems are likely to occur - most routines might start to give erroneous results before the actual limit.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-10-15T20:24:56.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21016" PostTypeId="1" AcceptedAnswerId="21017" CreationDate="2015-10-15T21:18:22.720" Score="3" ViewCount="917" Body="&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/20913/9466&quot;&gt;In this excellent answer&lt;/a&gt;, it is recommended that one make use of the $\textrm{erfcx}$ function to avoid roundoff error in calculating  dealing with $x &amp;lt; 25$ (approximately). So, one scales their answer by $e^{x^2}$, and then scales back, I suppose (?) to recover the number they are really looking for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if we have the case where we need to calculate $\textrm{erfc}(a) - \textrm{erfc}(b)$? I could write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textrm{erfcx}(a) - \textrm{erfcx}(b) = e^{a^2}\textrm{erfc}(a) - e^{b^2}\textrm{erfc}(b) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but that's not particularly helpful if I want to numerically integrate the function ($x, a, b \in \mathbb{R}$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{a}^{b} e^{x^2}(\textrm{erf}(x) - \textrm{erf(a)})\;\textrm{d}x = \int_{a}^{b} e^{x^2}(\textrm{erfc}(a) - \textrm{erfc(x)})\;\textrm{d}x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $a$ and $b$ are close enough/large enough in magnitude (not large in terms of sign) then we get catastrophic cancellation. For example, try $a = -16.85$ and $b = -16.08$. Rescaling inside the integrand would seem to be unhelpful, rather than helpful!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have Python, &lt;a href=&quot;https://gist.github.com/bmer/eed97c79e8ba2d793e0d&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's the toy problem script on Github Gist&lt;/a&gt;. It implements many of the recommendations provided in this answer to avoid overflow errors, but still runs into underflow problems. &lt;/p&gt;&#xA;" OwnerUserId="9466" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-10-16T03:45:18.327" Title="How can I avoid roundoff error when calculating the difference $\textrm{erfc}(a) - \textrm{erfc}(b)$?" Tags="|numerical-analysis|computer-arithmetic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21017" PostTypeId="2" ParentId="21016" CreationDate="2015-10-15T22:45:46.230" Score="1" Body="&lt;p&gt;I am not sure exactly what you are trying to do.  I am assuming you want to evaluate the integral using some sort of quadrature.  This will work for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*}&#xA;\int_{x=a}^b \exp(x^2) (\mathrm{erf}(x) - \mathrm{erf}(a))\,\mathrm{d} x &#xA;&amp;amp;=&#xA;\frac{2}{\sqrt{\pi}} \int_{x=a}^b \exp(x^2) \int_{\tau = a}^x \exp(-\tau^2) \, \mathrm{d} \tau\, \mathrm{d} x\\&#xA;&amp;amp;= \frac{2}{\sqrt{\pi}} \int_{x=a}^b \int_{\tau= a}^x \exp(x^2-\tau^2) \mathrm{d} \tau\, \mathrm{d}x&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in the triangle of integration, the integrand is strictly positive so you shouldn't have any cancellation.&lt;/p&gt;&#xA;" OwnerUserId="16799" LastActivityDate="2015-10-15T22:45:46.230" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21018" PostTypeId="1" CreationDate="2015-10-16T12:55:28.840" Score="6" ViewCount="595" Body="&lt;p&gt;I am looking to solve a class of SDPs with complex entries, with the semi-definite cone &lt;span class=&quot;math-container&quot;&gt;$S^n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; around 5000 to 15000. Also, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, the number of equality/inequality constraints is close to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I tried &lt;code&gt;SeDuMi&lt;/code&gt; and &lt;code&gt;SDPT3&lt;/code&gt; with &lt;code&gt;CVX&lt;/code&gt;, but I quickly run out of memory for &lt;span class=&quot;math-container&quot;&gt;$n=1000, m=1000$&lt;/span&gt; (I have about 50 GB available). Could you suggest other (free) SDP solvers that I could use?&lt;/p&gt;&#xA;&lt;p&gt;I looked at the documentation for &lt;code&gt;SDPA&lt;/code&gt;, and it claims to have much lower memory usage than the solvers mentioned above. In one of the cases, the memory use is claimed to be around 5% as &lt;code&gt;SDPT3&lt;/code&gt;. The matrices in my problem aren't very sparse, but there are about 20% non-zeros.&lt;/p&gt;&#xA;&lt;p&gt;Is &lt;code&gt;YALMIP&lt;/code&gt; + &lt;code&gt;SDPA&lt;/code&gt; is a good combination for my problem?&lt;/p&gt;&#xA;&lt;p&gt;Please note that I have complex numbers involved, and I would like to avoid changing my formulation to convert everything into real, if possible.&lt;/p&gt;&#xA;" OwnerUserId="17899" LastEditorUserId="20688" LastEditDate="2022-01-04T04:35:27.490" LastActivityDate="2022-01-04T04:36:04.827" Title="Best platform for complex SDPs with n and m around 5-15K?" Tags="|constrained-optimization|convex-optimization|semidefinite-programming|software-recommendation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21019" PostTypeId="1" CreationDate="2015-10-16T14:20:06.253" Score="2" ViewCount="47" Body="&lt;p&gt;So I'm trying to construct a 2 fold oversampled cosine basis in MATLAB. I know how to construct the basis as a square matrix using the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dct(eye(n,n))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;dct&lt;/code&gt; is the discrete cosine transform function and $n\times n$ is the size of my basis but how would I do this if I wanted my basis to be oversampled? Specifically for it to be $n\times 2n$, i.e. a fat matrix such that the columns lose the property of linear independence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="17901" LastEditorUserId="1804" LastEditDate="2015-10-16T15:37:00.317" LastActivityDate="2015-10-16T15:37:00.317" Title="Constructing 2 fold oversampled cosine basis in MATLAB" Tags="|matlab|fourier-transform|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21020" PostTypeId="1" CreationDate="2015-10-16T14:41:23.243" Score="6" ViewCount="721" Body="&lt;p&gt;Is there a fast algorithm for computing the matrix square root of a real symmetric matrix using random matrices or randomized algorithms?&lt;/p&gt;&#xA;" OwnerUserId="1934" LastEditorUserId="20688" LastEditDate="2019-06-22T22:48:03.293" LastActivityDate="2019-06-22T22:48:03.293" Title="Fast algorithm for computing matrix square root using randomized linear algebra?" Tags="|linear-algebra|matrix|algorithms|approximation|randomized-algorithms|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21021" PostTypeId="1" CreationDate="2015-10-16T14:44:11.950" Score="7" ViewCount="161" Body="&lt;p&gt;Let $A\in \mathbb{R}^{n\times n}$ symmetric and positive semidefinite, and $\omega\in \mathbb{R}\setminus\{0\}$.  I am interested in solving the following linear system for a range of values of $\omega$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$((A-\omega^2 I)(A-\omega^2 I)+\omega^2 I)x = b.$$ It may be useful to note that the matrix factors as &#xA;$$&#xA;(A-(\omega^2-i\omega)I)(A-(\omega^2+i\omega)I), &#xA;$$&#xA;where $i^2 = -1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Details:&#xA;$A$ is sparse and I won't have direct access to its entries. The dimension of the null space of $A$ is a non-negligible fraction of $n$.  The dimension of the problem, $n$, will be as big as the computer's RAM will allow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a good way to solve preprocess / precondition this system?  Note that the RHS, $b$, will change when $\omega$ changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notes:  This is a follow up question to this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21006/what-strategies-decompositions-would-be-useful-to-solve-the-following-linear-s&quot;&gt;one&lt;/a&gt;.  The idea of the proposed solution to that question shows that if we could perform an complete eigendecomposition on $A$, we would have a pretty much ideal preprocess.  I have implemented an Lansczos iteration to approximate this eigendecomposition but it doesn't perform as well as I had hoped.  I can explain this idea in more detail as an addendum if there is interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course full answers are appreciated, but they are not expected.  I am mainly looking for ideas to investigate.  Any comments and pointers to the literature are much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note to mods: Is this kind of question acceptable?  I can change it to something more definite if asking for ideas is unacceptable.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Edit&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;This is what I plan on doing.  First note that as $\omega\to \infty$ the matrix starts looking like $I(\omega^4+\omega^2)$, so we are mainly interested in when $\omega$ is comparable to the norm of $A$ and smaller.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To that end, we compute $r$ eigen-pairs of $A$, $(\lambda_i,q_i)\in \mathbb{R}\times \mathbb{R}^{n\times n}$, with the largest eigenvalues.  Then, since these eigenvectors can be made to be orthonormal we have&#xA;$$&#xA;x= \sum_{i=1}^r \alpha_i q_i + \sum_{i={r+1}}^n \alpha_i q_i.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, taking the dot product of both side of the equation with $q_i$ for $1\le i\le r$ we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\alpha_i = \left\langle q_i,b \right\rangle \frac{1}{(\lambda_i - \omega^2)(\lambda_i - \omega^2) + \omega^2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I plan on using this information to construct an initial guess for $x$.  I am still unsure on what preconditioner to use.  &lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-07-25T10:02:06.373" Title="What are some ideas to preprocess / precondition the following linear system?" Tags="|linear-algebra|linear-solver|eigensystem|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21022" PostTypeId="2" ParentId="21018" CreationDate="2015-10-16T14:56:59.940" Score="4" Body="&lt;p&gt;You can try using SCS, either the direct or indirect solver.  SCS uses first-order methods, and hence may be able to solve larger problems than second-order solvers such as SDPT3, SeDuMi, MOSEK, etc.  On the downside, given that it is a first-order solver, it can be very slow - but very slow may be better than not at all.&lt;/p&gt;&#xA;&lt;p&gt;Paper: &lt;a href=&quot;http://web.stanford.edu/%7Eboyd/papers/pdf/scs.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://web.stanford.edu/~boyd/papers/pdf/scs.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;MATLAB code (need to make): &lt;a href=&quot;https://github.com/cvxgrp/scs/tree/master/matlab&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/cvxgrp/scs/tree/master/matlab&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can use SCS under YALMIP or under CVX 3.0 beta. Under CVX 3.0 beta, you can specify&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cvx_solver scs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which will invoke the SCS direct solver.  To invoke the SCS indirect solver, also do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cvx_solver_settings('use_indirect',true);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="20688" LastEditDate="2022-01-04T04:36:04.827" LastActivityDate="2022-01-04T04:36:04.827" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21023" PostTypeId="2" ParentId="21020" CreationDate="2015-10-16T16:20:41.507" Score="4" Body="&lt;p&gt;Randomized algorithms can accurately approximate your matrix if it has rapidly decaying singular values (i.e. a Gaussian sampling of the range is likely to pick up the action of the matrix). Since your matrix is symmetric, you could approximate a truncated eigenvalue decomposition using random linear algebra and simply take the square root of the eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure of any explicit algorithms based on randomized linear algebra (RLA) which approximate the matrix square root explicitly. You could also try iterative algorithms for the matrix square root (Wikipedia shows a few examples) and try to exploit RLA there in matrix-vector multiplications or applying matrix inverses, though this may not work well since inverses typically need the smallest singular values.  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-10-16T16:20:41.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21025" PostTypeId="1" AcceptedAnswerId="21026" CreationDate="2015-10-17T08:35:15.620" Score="3" ViewCount="733" Body="&lt;p&gt;I just started leaning Julia, and my command of it is still very preliminary. I am trying to reorder a Schur decomposition. In Matlab, I could use the ordqz command and just had to specify the ordering by passing a string argument. In Julia, my understanding is that this is achieved by the matrix &lt;em&gt;select&lt;/em&gt; in the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ordschur(S, select)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where S is the Schur object as obtained from  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S=schur(A,B)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I have no clue how to specify the &lt;em&gt;select&lt;/em&gt; matrix to control for the ordering, and looking at the Julia documentation did not prove helpful. Has anybody already used it? Would it be possible to have an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="17913" LastActivityDate="2015-10-17T10:51:12.800" Title="Julia: ordschur command" Tags="|linear-algebra|eigensystem|languages|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21026" PostTypeId="2" ParentId="21025" CreationDate="2015-10-17T10:51:12.800" Score="3" Body="&lt;p&gt;The documentation is a bit terse, I admit. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;..  ordschur(Q, T, select) -&gt; Schur&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Reorders the Schur factorization of a real matrix &lt;code&gt;A=Q*T*Q'&lt;/code&gt; according to the logical array &lt;code&gt;select&lt;/code&gt; returning a Schur object &lt;code&gt;F&lt;/code&gt;. The selected eigenvalues appear in the leading diagonal of &lt;code&gt;F[:Schur]&lt;/code&gt; and the the corresponding leading columns of &lt;code&gt;F[:vectors]&lt;/code&gt; form an orthonormal basis of the corresponding right invariant subspace.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The crucial bit of information, easy to overlook, is &quot;logical array&quot;: this means that &lt;code&gt;select&lt;/code&gt; is a true/false array. The eigenvalues  which &lt;code&gt;select[i]&lt;/code&gt; is true are brought on top. For instance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; A = randn(3,3);&#xA;&#xA;julia&amp;gt; (T,Q) = schur(A);&#xA;&#xA;julia&amp;gt; S = ordschur(Q,T,[false,true,true]);&#xA;&#xA;julia&amp;gt; T&#xA;3x3 Array{Float64,2}:&#xA; 2.12232   0.778057  -0.195052&#xA; 0.0      -0.162527   0.793454&#xA; 0.0       0.0       -1.49427 &#xA;&#xA;julia&amp;gt; S.T&#xA;3x3 Array{Float64,2}:&#xA; -0.162527   0.829117  0.7619   &#xA;  0.0       -1.49427   0.0711313&#xA;  0.0        0.0       2.12232  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The second and third eigenvalue -0.16 and -1.49 are now the leading ones. Note also that the order of the arguments &lt;code&gt;T&lt;/code&gt; and &lt;code&gt;Q&lt;/code&gt; between the two functions is inconsistent (I should probably file a bug about this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to bring on top the eigenvalues with absolute value smaller than 1 (a common use for &lt;code&gt;ordschur&lt;/code&gt;, which in Matlab is achieved through a hardcoded special string parameter), you can use the following very idiomatic line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;select = abs(eig(T)) .&amp;lt; 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2015-10-17T10:51:12.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21027" PostTypeId="5" CreationDate="2015-10-17T12:19:27.507" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2015-10-17T12:19:27.507" LastActivityDate="2015-10-17T12:19:27.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21028" PostTypeId="4" CreationDate="2015-10-17T12:19:27.507" Score="0" Body="For questions related to the collection, processing and interpretation of pixel data from imaging systems." OwnerDisplayName="user17774" LastEditorDisplayName="user17774" LastEditDate="2015-10-18T15:54:47.910" LastActivityDate="2015-10-18T15:54:47.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21029" PostTypeId="1" CreationDate="2015-10-17T16:50:13.757" Score="0" ViewCount="77" Body="&lt;p&gt;Convert the following model into an LP model.  Note that you're not being asked to convert this to standard form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min z = \max (x_1, x_2, x_3, 2000)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t.&#xA;$$-2x_1 + x_2 + x_3 \geq -4$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$3x_1 - 4x_2 + x_3 \leq 12$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_i \geq 0$ for all $i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having difficulty thinking through how to set up an objective function that isn't a formula, but selects the maximum of multiple variables.  I've tried setting up the $\max(x_1, x_2, x_3, 2000)$ as a matrix and as another variable x4, but it doesn't make sense to me on paper.  When I write the problem in Excel, I use &lt;code&gt;=MAX(x1, x2, x3, 2000)&lt;/code&gt; as the objective function but I don't understand the algorithm I would use to solve this problem. I've researched several textbooks and am stuck on setting up an objective function that selects a single maximum value of several variables.&lt;/p&gt;&#xA;" OwnerUserId="17918" LastEditorDisplayName="user17774" LastEditDate="2015-10-18T05:00:21.943" LastActivityDate="2015-10-18T05:00:21.943" Title="Convert the following model into an LP model (not asking for standard form), includes a max (a,b,c,d)" Tags="|linear-programming|model|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21030" PostTypeId="1" AcceptedAnswerId="21055" CreationDate="2015-10-17T17:37:31.840" Score="3" ViewCount="909" Body="&lt;p&gt;I am learning SPH method. At the moment I am trying to implement simulation described in &lt;a href=&quot;http://www.researchgate.net/publication/222543591_Numerical_Simulation_of_Interfacial_Flows_by_Smoothed_Particle_Hydrodynamics&quot; rel=&quot;nofollow noreferrer&quot;&gt;this very good article&lt;/a&gt;. However I don't get how the ghost particles properties are computed:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Position of the ghost particle:&#xA;$$\textbf{x}_{i,G}=2\textbf{x}_w-\textbf{x}_i,$$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;where $\textbf{x}_i$ denotes the $i^{th}$ particle position,  $\textbf{x}_w$ is the rigid boundary instantaneous position, $\textbf{x}_{i,G}$ is the ghost particle position.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Normal velocity component with respect to the boundary:&#xA;$$u_{niG}=2U_{nw}-u_{ni}$$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;where $U_{nw}$ is the local displacement velocity of the rigid boundary with instantaneous position $\textbf{x}_w$, $u_{ni}$ is the normal velocity component of the particle to the boundary, $u_{niG}$ is the normal velocity component of the ghost particle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the local displacement velocity of the rigid boundary with instantaneous position $\textbf{x}_w$, and what is $\textbf{x}_w$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought that if the boundary is rigid (let's say a solid wall) it will have $U_{nw}=0$, and invariant position. However with those assumptions I don't get the ghost particles similar as described in the paper (it is a part of the Fig 15 from the mentioned paper):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/defRE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/defRE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="4334" LastEditorUserId="4334" LastEditDate="2015-10-17T17:45:03.177" LastActivityDate="2015-10-19T21:34:30.493" Title="Smoothed particle hydrodynamics bounduary (ghost particles) properties" Tags="|boundary-conditions|sph|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21031" PostTypeId="2" ParentId="21029" CreationDate="2015-10-17T18:24:59.490" Score="1" Body="&lt;p&gt;&lt;strong&gt;Hint:&lt;/strong&gt; The value $\max(x_1,x_2)$ is the smallest value $t$ for which $t\geq x_1$ and $t\geq x_2$.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2015-10-17T18:24:59.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21032" PostTypeId="1" CreationDate="2015-10-17T19:17:31.110" Score="2" ViewCount="302" Body="&lt;p&gt;What are some of the methods used to test for the exactitude of a numerical solution, given that the analytical solution isn't available, and the numerical solution converges ?&lt;/p&gt;&#xA;" OwnerUserId="14376" LastActivityDate="2015-10-17T23:46:18.463" Title="Mathematical test method for the numerical solution of PDEs?" Tags="|pde|numerical-analysis|numerical-modelling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21033" PostTypeId="2" ParentId="21032" CreationDate="2015-10-17T20:11:33.187" Score="3" Body="&lt;p&gt;Verification of a numerical solution is not fully possible when there is no analytical solution for comparison, but there are still several ways to gain confidence in the &quot;correctness&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your question the meaning of &quot;the numerical solution converges&quot; is not fully clear. &quot;Convergence&quot; in the sense that the numerical code produces a finite answer in finite time is of course necessary for correctness. More strongly, typically a &lt;a href=&quot;https://en.wikipedia.org/wiki/Rate_of_convergence#Convergence_speed_for_discretization_methods&quot; rel=&quot;nofollow&quot;&gt;mesh-refinement study&lt;/a&gt; will be performed to show that the solution converges as the discretization becomes finer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your PDE system is $F(x)=0$, then the error is $\Delta x=x_h-x_*$, where $x_h$ is the discrete solution for mesh-size $h$ and $x_*$ is the exact solution. You would want the convergence to be monotone (i.e. refining mesh does not &lt;em&gt;increase&lt;/em&gt; error), and possibly also show a consistent order of convergence (e.g. $|\Delta x|=O[h^{-p}]$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you could choose some proxy for the &quot;exact&quot; solution. For example, this could be a solution on a very fine mesh, or a solution from another code, e.g. from the literature. (For &lt;em&gt;validation&lt;/em&gt;, rather than &lt;em&gt;verification&lt;/em&gt;, you would use observations from the natural/experimental/engineered system your PDE is modeling.) Then you can use this proxy in the mesh refinement convergence analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather than the solution &lt;em&gt;error&lt;/em&gt;, you can also look at the solution &lt;em&gt;residual&lt;/em&gt;, i.e. $|F_h(x_h)|$. In this case you should normalize the residual so it does not inherently grow with mesh refinement. For example the residual could be averaged, or integrated over the fixed domain size (conceptually, $\int_\Omega F d\Omega = \bar{F}|\Omega|$, with $h=|\Omega|/N$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also reccomend doing &lt;em&gt;code&lt;/em&gt; verification as well. That is, solve some problems that &lt;em&gt;do&lt;/em&gt; have an analytical solution using the same code, and verify that these are correct. (Here, you can &lt;em&gt;manufacture&lt;/em&gt; test cases that by design have an analytical solution; see Bill Barth's answer.)&lt;/p&gt;&#xA;" OwnerUserId="17742" LastEditorUserId="17742" LastEditDate="2015-10-17T23:46:18.463" LastActivityDate="2015-10-17T23:46:18.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21035" PostTypeId="2" ParentId="21032" CreationDate="2015-10-17T23:40:25.017" Score="7" Body="&lt;p&gt;You should also read about the &lt;a href=&quot;http://infoserve.sandia.gov/sand_doc/2000/001444.pdf&quot;&gt;Method of Manufactured Solutions (PDF)&lt;/a&gt; which will show you how to generate analytical solutions to your problem.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-10-17T23:40:25.017" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21037" PostTypeId="1" AcceptedAnswerId="21053" CreationDate="2015-10-18T07:02:21.430" Score="1" ViewCount="1017" Body="&lt;p&gt;I want to solve 3 coupled PDEs equations. They depend on time, radius and length. I used the method of lines (MOL) and converted them to a system of ODEs in time. Now I want to solve them using MATLAB. Here are the equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  &amp;amp;\frac{\partial C_{i,p}}{\partial t} + \frac{u_{cp}C_{i,p}}{r} +     C_{i,p}\frac{\partial u_{cp}}{\partial r} + u_{cp} \frac{\partial C_{i,p}}{\partial r} - D_{eff}\left(\frac{\partial^2 C_{i,p}}{\partial r^2} + \frac{1}{r} \frac{\partial C_{i,p}}{\partial r}\right) = - \frac{(1-\epsilon_f)}{\epsilon_f} \rho_f \frac{\partial q_i}{\partial t}\\&#xA;&amp;amp;\frac{\partial q_i}{\partial t} = k_s (q_i^{eq} - q_i)\\&#xA;&amp;amp;q_i^{eq} = \frac{q_s b P_i}{[1 + (bP_i)^n]^{1/n}}\\&#xA;&amp;amp;q_s = q_{s0}  \exp\left[\eta \left(1 - \frac{T}{T_0}\right)\right]\\&#xA;&amp;amp;b = b_0 \exp\left[\frac{-\Delta H}{R T_0}\left(1 - \frac{T_0}{T}\right)\right]\\&#xA;&amp;amp;\eta = A + B\left(1 -\frac{T_0}{T}\right)\\&#xA;&amp;amp;P_i = C_{i,p}/RT\\&#xA;&amp;amp;T = T_f\\&#xA;&amp;amp;\rho C_{pf}\frac{\partial T_f}{\partial t}-\frac{\lambda_f}{(1-\epsilon_f)} \left[\frac{\partial^{2}T_f}{\partial r^{2}}+\frac{1}{r}\frac{\partial T_f}{\partial r}+\frac{\partial^{2}T_f}{\partial z^{2}}\right] = \rho_f \Delta H_{ads} \frac{\partial q_{i}}{\partial t}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use &lt;code&gt;ode15s&lt;/code&gt; to solve them. I create one matrix where the rows indicate radius nodes and the columns indicate length nodes. I have a problem because they are coupled together and I cant use ode15s for solving them. Because &lt;code&gt;ode15s&lt;/code&gt; solve one equation from time=[0:end] and then solve another. I want to solve all of the equations step by step, first for time=1 then time=2, etc. Can anyone help me solve this set of ODEs using &lt;code&gt;ode15&lt;/code&gt; in MATLAB?&lt;/p&gt;&#xA;" OwnerUserId="17594" LastEditorUserId="17594" LastEditDate="2015-10-19T16:24:13.480" LastActivityDate="2017-07-17T10:43:16.810" Title="How can I solve coupled equations by the method of line(MOL)?" Tags="|matlab|pde|ode|numerical-modelling|method-of-lines|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21038" PostTypeId="2" ParentId="21004" CreationDate="2015-10-18T09:49:22.337" Score="4" Body="&lt;p&gt;This is a Hamilton-Jacobi equation.  You can read about how to apply WENO to such equations in Section 4 of &lt;a href=&quot;http://dx.doi.org/10.1137/070679065&quot; rel=&quot;nofollow&quot;&gt;Chi-Wang Shu's 2009 WENO review paper&lt;/a&gt;, and references therein.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-10-18T09:49:22.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21039" PostTypeId="2" ParentId="20998" CreationDate="2015-10-18T10:50:10.357" Score="2" Body="&lt;p&gt;If you want to obtain the diffusion coefficient for 2D from 3D you have to take a look how the 2D diffusion equation is derived from 3D equation. It will be integrated with respect to one of the spatial dimensions. Diffusion coefficient is a $3 \times 3$ tensor in 3D, in 2D it will be $2 \times 2$ tensor. So in my opinion you cannot simply take the value from 3D equation and substitute it into 2D equation because after integrating it will describe the averaged diffusion in space (in the integrated dimension), so it will have different interpretation, it could be considered rather dispersion coefficient for 2D equations. &lt;/p&gt;&#xA;" OwnerUserId="4334" LastActivityDate="2015-10-18T10:50:10.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21040" PostTypeId="1" AcceptedAnswerId="21043" CreationDate="2015-10-18T13:52:36.897" Score="3" ViewCount="659" Body="&lt;p&gt;In the context of a research project of mine, I am faced with the difficult task of modeling the mixing of two fluids in a container. I would like to achieve the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a &lt;strong&gt;container&lt;/strong&gt; (cylindrical or spherical) containing a &lt;strong&gt;static mixture of gases&lt;/strong&gt; (carbon dioxide + nitrogen), an &lt;strong&gt;inflow channel of oxygen&lt;/strong&gt; into the container, and a &lt;strong&gt;vacuum outflow&lt;/strong&gt; exiting the container, I would like to &lt;strong&gt;simulate the evolution of the mixing of gases inside the container (composition and vortices) and in the outflow channel&lt;/strong&gt;. In fact, the goal is to end up with a crude model of a lung with the particularity of a vacuum outflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone point me to a tool (perferably free) or a set of techniques that could help me build such a simulation? I can program in pretty much any language, and GUI software packages would be ok too&lt;/p&gt;&#xA;" OwnerUserId="17925" LastActivityDate="2015-10-18T17:55:45.543" Title="How to model the mixing of two fluids in a container?" Tags="|fluid-dynamics|simulation|computational-physics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21041" PostTypeId="1" CreationDate="2015-10-18T14:19:45.097" Score="3" ViewCount="275" Body="&lt;p&gt;I am trying to understand how boundary conditions are implemented when one uses the nonlinear LU-SGS algorithm for Euler equations. Most papers describe the Gauss-Seidel sweep over mesh cells, but do not explain how are boundary conditions applied. I could either consider the same BC implementation as in explicit case (i.e. sweep over cells, then add flux corrections on boundaries), or add Jacobian contributions due to boundary conditions to the element Jacobians. The latter leads to a more complicated implementation, however, where one is obliged to check for each element whether it is supposed to receive boundary Jacobian contributions.&#xA;Are you aware of any reference discussing LU-SGS &lt;strong&gt;and&lt;/strong&gt; boundary conditions for compressible Euler/Navier-Stokes equations in more detail?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think it is possible to replace the 'boundary condition Jacobian' approach with something simpler? Would that deteriorate the convergence rate of the solver?&lt;/p&gt;&#xA;" OwnerUserId="4688" LastActivityDate="2018-09-21T13:57:11.530" Title="LU-SGS and boundary conditions" Tags="|fluid-dynamics|boundary-conditions|implicit-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21042" PostTypeId="1" CreationDate="2015-10-18T15:11:36.443" Score="6" ViewCount="849" Body="&lt;p&gt;For my thesis, I look in trajectories of vehicles through an atmosphere at very high velocities. I have a set of equations of motion, which I propagate using a Runge-Kutta-Fehlberg (RKF) 7(8) numerical integrator, one that has already been developed by the department of my university. The entire code is programmed in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my vehicle, I need to find a specific bank angle profile that achieves a specific objective (it's not necessary to understand what a bank angle profile is, just that I need one as an input to my equations of motion). The optimization algorithm provides a (random) bank angle profile, which I use as an input to my simulation. Based on the result of that simulation, the optimization algorithm changes the bank angle profile and evaluates the new trajectory. This keeps on repeating until a bank angle profile is found that achieves the set objective. Let's call the corresponding trajectory that results from the bank angle profile the reference trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I also am developing a guidance algorithm. This algorithm requires the different state variables and derived values (such as altitude or atmospheric density) at a specified interval, let's say ever 1.0 seconds. I want to apply this guidance algorithm to the found reference trajectory. Since the RKF7(8) algorithm has adaptive step size control, which is beneficial during the optimization since it reduces the number of function evaluations, it is most certainly not guaranteed that an output at every 1.0 seconds is provided. So, to achieve this output at ever 1.0 seconds, the integration is performed again with the same initial conditions as the reference trajectory, expect that the step size obtained from the RKF7(8) is modified to ensure the output at every 1.0 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem, however, is that the obtained trajectory in case the step size is modified is different from the reference trajectory. My guess was that since I reduce the step size to achieve the output at every 1.0 seconds, I force the accuracy to increase due to the smaller step size. I also tried modifying the tolerances used in the RKF7(8) algorithm, as well as different order methods (RKF4(5), RKF5(6) and Adams-Bashfort-Moulton). I decreased this tolerance (both absolute and relative) to 1e-15 (which is I think more or less the limit since it's close to machine precision), but I still obtain different trajectories. Same thi To give you an impression of the difference, see the figure below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BIp9M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BIp9M.png&quot; alt=&quot;Altitude as a function of time Altitude as a function of time&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that might have an influence, but I'm not sure about this, is the high velocity. If I reduce the velocity, there is still a small difference but this does not lead to a noticeable difference in the trajectory. However, for my research, it is essential to have these high velocities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone has an explanation for the behavior I showed? If so, what could I do to solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; Based on the comment of Doug Lipinski, I performed several runs with a fixed step-size integrator, Runge-Kutta 4, for a number of step sizes. See the figure below. From this figure, it appears that the solution does not converge to one solution even though the step sizes is very small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wxi6g.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wxi6g.png&quot; alt=&quot;Trajectory propaged using RK4 for different step sizes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17926" LastEditorUserId="17926" LastEditDate="2015-10-19T12:04:55.630" LastActivityDate="2016-07-26T01:16:39.460" Title="Numerical integration using RKF7(8) - different results" Tags="|numerics|integration|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21043" PostTypeId="2" ParentId="21040" CreationDate="2015-10-18T17:55:45.543" Score="3" Body="&lt;p&gt;OpenFOAM is a free, fully functioned CFD package which is capable of such simulations and also deformable meshes if that becomes relevant.  OpenFOAM primarily uses the finite volume method and already has a solver developed for your application - interMixingFoam.  Others are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cfd.direct/openfoam/user-guide/standard-solvers/&quot; rel=&quot;nofollow&quot;&gt;http://cfd.direct/openfoam/user-guide/standard-solvers/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even so, the development of new solvers is pretty straightforward since it is developed to handle the weak form of your equations. There are also some GUIs that exist to help developing the simulation,my preferred one when I was learning OPF is HelyxOS:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://engys.com/products/helyx-os&quot; rel=&quot;nofollow&quot;&gt;http://engys.com/products/helyx-os&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And there is a very active forum community that supplements the somewhat poor documentation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cfd-online.com/Forums/openfoam/&quot; rel=&quot;nofollow&quot;&gt;http://www.cfd-online.com/Forums/openfoam/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, there is an extended project which contains a number of additional features for OPF that are developed by the community:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.extend-project.de&quot; rel=&quot;nofollow&quot;&gt;http://www.extend-project.de&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16463" LastActivityDate="2015-10-18T17:55:45.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21044" PostTypeId="2" ParentId="21042" CreationDate="2015-10-18T21:03:08.873" Score="5" Body="&lt;p&gt;I think the first step is to confirm which of the solutions is more accurate. If you are using a reference implementation of RKF7(8) that has presumably been validated by others on other problems it seems extremely likely that the error is on your end when you're modifying the time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is possible (i.e. not too computationally expensive). I would try performing a convergence study with a simple fixed time step integration scheme (say RK4). If refining the time step leads to the fixed time step method converging to one of the two solutions you have then that gives additional verification of the correct solution. If you find that the &quot;reference solution&quot; is incorrect then you need to look at how you're setting your tolerance. Ideally you should be using a relative tolerance, not absolute (see Wolfgang's answer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the error occurs somewhere in your modifications to the time step I would suggest a slightly different approach: rather than modifying the time step, interpolate the results to the desired time values. If the plot you have shown is typical of the results a simple polynomial interpolation using nearby points should be very accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If successive runs at smaller and smaller fixed step size do not appear to be converging, it may be that your problem is &lt;a href=&quot;https://en.wikipedia.org/wiki/Stiff_equation&quot; rel=&quot;nofollow&quot;&gt;stiff&lt;/a&gt; or that the problem exhibits signs of chaos (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Butterfly_effect&quot; rel=&quot;nofollow&quot;&gt;sensitive dependence on initial conditions&lt;/a&gt;, or in this case small differences in each time step). In the first case you could look at using a different time integrator that is appropriate for stiff ODEs (implicit methods are often used). In the second case I'm not really sure what the best option would be.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-10-19T11:20:46.393" LastActivityDate="2015-10-19T11:20:46.393" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21045" PostTypeId="2" ParentId="21042" CreationDate="2015-10-18T21:04:17.347" Score="5" Body="&lt;p&gt;If you get qualitatively different results from two ODE integrators, then the time step choice of at least one of them is too large. Which one that is is not immediately obvious to say, but if you play with the time step of an integrator (e.g., making it smaller by a factor of ten) and you get a different result, then the time step was too large; if the results are the same then the time step was likely appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would continue to play with the tolerance. 1e-15 is not &lt;em&gt;a priori&lt;/em&gt; close to the round-off level. The round-off is around 1e-16, but only when seen &lt;em&gt;relative&lt;/em&gt; to the size of the variables you are comparing against. If the variables you are comparing against are on the order 1e-12, then 1e-15 is an entirely inappropriately large tolerance. It's also not clear what the tolerance actually compares to -- the error? the residual? Not knowing what it is in the implementation you use, I would continue to play with it to see what happens.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-07-26T01:16:39.460" LastActivityDate="2016-07-26T01:16:39.460" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21046" PostTypeId="1" AcceptedAnswerId="21049" CreationDate="2015-10-19T03:27:19.957" Score="1" ViewCount="389" Body="&lt;p&gt;First off some context. The Traveling Salesman Problem(TSP) is to find the most efficient route passing through a series of points only once. However, there is no perfect function to solve for this in a reasonable amount of time. Because with each additional point, the number of possible routes is greatly increased. This is in terms of a factorial function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where 'n' is the number of points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing a project that is attempting to examine the mathematical properties of different algorithms that provide an answer for the TSP. I am trying to examine the relationship of the time it takes to produce an answer, and how close that answer is to the correct answer. This means that I am checking every possible route using a brute force algorithm, and then will be comparing the other algorithms, and their shortest length to that of the correct answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am comparing the Greedy salesman method, and the brute force method, but I would like some other fairly simple algorithms that can be used for the problem. If there are any common methods that I should do more research into, or any methods that work well, please tell me along with the basic concept behind that process. For example the Greedy salesman method is always going to the closest point, and the brute force method is checking all of the possible routes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate any of the algorithms or methods that are suggested.&lt;/p&gt;&#xA;" OwnerUserId="17804" LastActivityDate="2019-06-04T10:07:34.330" Title="Traveling Salesman Problem" Tags="|optimization|algorithms|c++|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21048" PostTypeId="1" CreationDate="2015-10-19T10:59:47.460" Score="2" ViewCount="182" Body="&lt;p&gt;I have a mesh which represents a 0-genus model. My goal is to construct a homeomorphism from that mesh to its bounding sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to understand a paragraph in &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.42.2762&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.42.2762&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The basic idea&#xA;  of [the algorithms to project a 0-genus mesh on the sphere, referred below] is to let the object grow towards its convex&#xA;  hull and then project it directly onto the sphere. [...]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In our implementation we use a variation of this principle.&#xA;  We first compute a voxelization of the given model. The&#xA;  gradient of the corresponding volumetric distance function&#xA;  is then used to guide the vertices of the given mesh towards&#xA;  a convex configuration. This process can be considered as&#xA;  some kind of ”inverse” deformable surface technique.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here is the link to the article it cites:&#xA;&lt;a href=&quot;http://design.osu.edu/carlson/history/PDFs/shape-kent.pdf&quot; rel=&quot;nofollow&quot;&gt;http://design.osu.edu/carlson/history/PDFs/shape-kent.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please explain me:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How to compute a voxelization of a given model ?&lt;/li&gt;&#xA;&lt;li&gt;What is the &quot;corresponding volumetric distance function&quot; ?&lt;/li&gt;&#xA;&lt;li&gt;How to use its gradient to &quot;guide the vertices of the given mesh to a convex configuration&quot; ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Once we've got a &quot;convex configuration&quot;, projection on the sphere is easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any open source implementation is welcome.&lt;/p&gt;&#xA;" OwnerUserId="17281" LastActivityDate="2015-10-24T16:30:32.610" Title="How to project a 0 genus mesh model on a sphere?" Tags="|mesh|projection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21049" PostTypeId="2" ParentId="21046" CreationDate="2015-10-19T13:13:52.767" Score="1" Body="&lt;p&gt;Randomized algorithms often work surprisingly well. An example is Simulated Annealing or its many variants.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-19T13:13:52.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21051" PostTypeId="2" ParentId="21046" CreationDate="2015-10-19T13:31:18.440" Score="0" Body="&lt;p&gt;For my thesis, I make use of ESA's open source optimization toolbox PaGMO, of which also a Python implementation called PyGMO is available. &lt;a href=&quot;http://esa.github.io/pygmo/tutorials/tsp.html&quot; rel=&quot;nofollow&quot;&gt;This&lt;/a&gt; page provides a written tutorial on how to solve the Traveling Salesman Problem using PyGMO. Since both PaGMO and PyGMO contain a large number of different optimization algorithms, it might be useful to play around these algorithms and see how well they perform.&lt;/p&gt;&#xA;" OwnerUserId="17926" LastActivityDate="2015-10-19T13:31:18.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21052" PostTypeId="2" ParentId="21042" CreationDate="2015-10-19T14:19:17.710" Score="2" Body="&lt;p&gt;One remark (seeing that the two answers given thus far come with quite a few comments, I've decided to post this as a separate answer instead): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you've solved the problem with your ODE integrator, you may wish to consider a Runge-Kutta &lt;em&gt;dense output&lt;/em&gt; mechanism for the solution interpolation at $t_k = k$ [sec]. A dense output scheme may yield significant compute-time savings compared to your approach of re-running your integration with smaller time steps, particularly if you need to repeat the integration many times (with different initial conditions), which is likely the case for your &quot;guidance&quot; optimization algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because the dense output approximation comes (more-or-less) &quot;for free&quot; - the polynomial solution interpolant can be built directly from the output of the Runge-Kutta stages. And you will not need to artificially decrease your time steps below those computed by the R-K adaptive-step logic (i.e., you don't sacrifice efficiency).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will find further information on this forum:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/7362/intermediate-values-interpolation-after-runge-kutta-calculation&quot;&gt;Intermediate values (interpolation) after Runge-Kutta calculation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another excellent reference is the &lt;a href=&quot;https://books.google.it/books?hl=en&amp;amp;id=F93u7VcSRyYC&amp;amp;pg=PA188&amp;amp;redir_esc=y#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;classic book of Hairer and Wanner&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-10-19T15:07:19.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21053" PostTypeId="2" ParentId="21037" CreationDate="2015-10-19T19:54:19.117" Score="1" Body="&lt;p&gt;In short, yes, you can solve the entire semi-discretized PDE at once using &lt;code&gt;ode15s&lt;/code&gt;, or any other ODE solver (ignoring, for the moment, issues of stability and accuracy). I would not arrange the unknowns from the semi-discretized system as a matrix; instead, I would arrange them as one long vector. Although indexing this vector will be cumbersome, expressing the unknowns as a vector better conforms to the interface of MATLAB (and other) ODE solvers, and is consistent with the way the semi-discrete equations are presented in the literature.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-10-19T19:54:19.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21054" PostTypeId="1" CreationDate="2015-10-19T20:16:19.067" Score="0" ViewCount="552" Body="&lt;p&gt;This is a follow up question to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21018/best-platform-for-complex-sdps-with-n-and-m-around-5-15k&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following SDP in standard form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\min_{X\in S^n, X&amp;gt;0} \operatorname{tr}(AX)\\&#xA;&amp;amp;\mbox{subject to}\; \operatorname{tr}(B_iX)\leq b_i, i=1,\dots,m\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $A\in S^n,B_i \in S^n$, $b_i\in \mathbb{R}$ and $m=O(n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone please explain the expected storage complexity of this problem in the general case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that first order solver like SCS will form a solve a $n^2$ sized linear system using gradient information only, and hence the storage complexity will be $O(n^4)$ which reflects the size of gradient matrix which will be $n^2\times n^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this understanding correct? My confusion is regarding the possibility of dual problem being smaller size. Is that taken care of by CVX automatically?&#xA;For example, if $n=1000$ and $m=1000$, it seems like primal form as mentioned above would form the smaller problem. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit After Johan's Answer:&#xA;I am currently using CVX. I tried an example similar to what you gave. Using notation in my post above, I am using $n=100,m=10$. $X\in S^n$. &#xA;If I submit my problem in format above, I get following output:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Calling SCS 1.1.7: &lt;strong&gt;5050 variables, 10 equality constraints&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lin-sys: sparse-indirect, nnz in A = 50500, CG tol ~ 1/iter^(2.00)&#xA;eps = 1.35e-06, alpha = 1.50, max_iters = 10000, normalize = 1, scale = 1.00&#xA;&lt;strong&gt;Variables n = 10, constraints m = 5050&lt;/strong&gt;&#xA;Cones:  sd vars: 5050, sd blks: 1&#xA;Setup time: 3.02e-03s&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now if I submit my problem by forming a dual of this, I get the message that CVX will solve the dual of my dual (i.e. the original problem), and I get exactly same number of constraints and variables as above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is whether CVX is really solving the way you have described ?&#xA;&lt;strong&gt;What is the expected space complexity given the above variables ?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="17908" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-10-21T06:25:16.990" Title="Storage complexity of SDP solver SCS" Tags="|convex-optimization|constrained-optimization|complexity|semidefinite-programming|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21055" PostTypeId="2" ParentId="21030" CreationDate="2015-10-19T21:34:30.493" Score="1" Body="&lt;p&gt;In simple terms, the idea is to get ghost particles such that the interpolated velocity between a (real) particle near the boundary and its corresponding ghost particle satisfies the required boundary condition. This is exactly like in a cell centered finite-difference scheme if you need to apply a Dirichlet boundary condition on the edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagine the figure on the right was generated for a non-zero value of $U_B(t)$. A boundary that is moving left with a certain velocity, at that instant, would give the specific vectors shown in the figure. In general, the boundary does not have to be rigid or fixed. It could be flexible and moving.&lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2015-10-19T21:34:30.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21056" PostTypeId="2" ParentId="21054" CreationDate="2015-10-20T12:29:49.173" Score="3" Body="&lt;p&gt;No, your understanding is not correct. By introducing a slack $s\in R^{m}_+$ you can write the constraints in the form $\operatorname{tr}(B_iX) + s_i = b_i$. You now have a standard semidefinite program, in primal form, over the product of a semidefinite cone of size $n$, and an LP cone of size $m$. The linear system will be a linear system of size $m \times m$ as there are $m$ equalities in the primal ($m$ variables in the dual). A very bad model would be to do what you implicitly hint, i.e., to introduce the elements of $X$ as variables, leading to a linear system of size $O(n^2)$. When solving a primal-dual pair in semidefinite programming, you never explicitly solve for the elements of the primal cone, the matrix as a whole is reconstructed from the solution of the line-search for the duals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stuff &lt;a href=&quot;http://users.isy.liu.se/johanl/yalmip/pmwiki.php?n=Tutorials.AutomaticDualization&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; might be a relevant read (and the linked paper there)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Trivial example in YALMIP (disclaimer, developed by me)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't want YALMIP to interpret this as a problem defined by the individual variables in $X$ and fit data into a dual description, which is default (leads to 125250 variables in dual /125250 equalities in primal)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = sdpvar(500);&#xA;optimize([X&amp;gt;=0,trace(X)==1],trace(randn(500)*X),sdpsettings('solver','scs'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Instead, you want YALMIP to interpret it from a primal point of view (1 equality in primal / 1 variable in dual)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;optimize([X&amp;gt;=0,trace(X)==1],trace(randn(500)*X),sdpsettings('solver','scs','dualize',1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2015-10-21T06:25:16.990" LastActivityDate="2015-10-21T06:25:16.990" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21058" PostTypeId="1" CreationDate="2015-10-20T13:45:33.653" Score="2" ViewCount="170" Body="&lt;p&gt;I'm developing an FEM solver for a coupled system. I have diffusion and potential equations which result in positive definite matrices for each equation, but the coupling makes the overall system matrix non-symmetric, non-positive-definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The solver is planned to run entirely on a GPU. Mesh and data representation is highly optimized for that and standard libraries are therefore not very useful, also because one would need to transfer between GPU and CPU, or had to use CPU-optimized data formats that would not perform well on the GPU.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I plan to use multigrid later on, so I currently use jacobi iterations to solve the equation. That works, let's say, semi-well. It converges not very fast and when errors get small, there are some signs of other problems (which I do not fully understand currently).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm now thinking about my next step. Should I try some CG-based algorithms? Or what would you try or investigate next?&lt;/p&gt;&#xA;" OwnerUserId="8138" LastEditorUserId="8138" LastEditDate="2015-10-20T14:52:54.357" LastActivityDate="2015-10-21T02:24:46.010" Title="Solve FEM matrix from coupled system" Tags="|finite-element|linear-solver|multigrid|conjugate-gradient|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21060" PostTypeId="1" AcceptedAnswerId="21063" CreationDate="2015-10-20T19:56:43.003" Score="3" ViewCount="8632" Body="&lt;p&gt;I am attempting to simulate projectile flight with drag. However, with a timestep of 0.1 seconds, I am consistently getting an error of ~0.1-1%.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while p[1, iterr] &amp;gt;= 0 and iterr &amp;lt; 9999:&#xA;            p0 = p[:, iterr]&#xA;            v0 = v[:, iterr]&#xA;            pp1 = h * v0&#xA;            vp1 = h * f(v0)&#xA;            pp2 = h * (v0 + vp1)&#xA;            vp2 = h * f(v0 + vp1/2)&#xA;            pp3 = h * (v0 + vp2)&#xA;            vp3 = h * f(v0 + vp2/2)&#xA;            pp4 = h * (v0 + vp3)&#xA;            vp4 = h * f(v0 + vp3)&#xA;&#xA;            v[:, iterr + 1] = v0 + (vp1 + 2 * (vp2 + vp3) + vp4)/6&#xA;            #p[:, iterr + 1] = p0 + h * (pp1 + 2 * (pp2 + pp3) + pp4)/6&#xA;            p[:, iterr + 1] = p0 + h * v[:, iterr + 1]&#xA;            iterr += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(v):&#xA;    magv = np.linalg.norm(v)&#xA;    return np.array([k * magv * v[0], k * magv * v[1] - 9.81])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: One line updating position is commented out because I was/am trying to figure out if the two expressions make any significant difference, and they do not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something about this simulation that I am doing wrong, or is 0.1-1% characteristic of the RK4 method?&lt;/p&gt;&#xA;" OwnerUserId="17947" LastEditorDisplayName="user17774" LastEditDate="2015-10-20T19:59:35.000" LastActivityDate="2020-06-13T19:36:10.997" Title="Runge-Kutta Simulation For Projectile Motion With Drag" Tags="|python|numerical-analysis|numpy|runge-kutta|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21061" PostTypeId="2" ParentId="21058" CreationDate="2015-10-21T02:24:46.010" Score="1" Body="&lt;p&gt;The most successful solvers for coupled systems typically apply something like GMRES to the whole system, but then use the block decomposition to derive preconditioners that make use of the properties of individual blocks. If you want to use a multigrid solver, then use it as a solver in the preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you don't state how the system you consider looks like, it's hard to give concrete advice. That said, the general idea of building preconditioners for block systems is discussed in lecture 38 of my video lectures here: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-21T02:24:46.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21062" PostTypeId="1" CreationDate="2015-10-21T03:31:19.883" Score="3" ViewCount="186" Body="&lt;p&gt;I have a sparse $2\times10^5$ by $2\times10^5$ matrix with $3.2\times10^9$ non-zero elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want a sparse solver with out-of-core functionality. I have attempted to use Intel's &lt;code&gt;MKL pardiso&lt;/code&gt;, with a varying degree of success. For smaller problems, it is stellar, but for the problem size above, it is taking a huge amount of &lt;code&gt;RAM&lt;/code&gt; (even with the out-of-core enabled), presumably during fill-in reducing ordering stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;intel&lt;/code&gt; staff has been incredibly helpful with debugging several problems, but I feel that it might be prudent to try a different solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any good reference comparing the performance of solvers for big problems?&lt;/p&gt;&#xA;" OwnerUserId="6716" LastEditorUserId="20688" LastEditDate="2018-04-09T03:02:50.363" LastActivityDate="2018-04-09T03:26:28.443" Title="Suggestions for an out-of-core sparse solver" Tags="|linear-solver|sparse-matrix|hpc|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21063" PostTypeId="2" ParentId="21060" CreationDate="2015-10-21T03:45:17.613" Score="3" Body="&lt;p&gt;This is a popular question but I can't find a readily available answer.  So here are some of the details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us assume that you are solving the equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;m \dot{\mathbf{v}} = m \mathbf{g} - k \lVert \mathbf{v}\rVert \mathbf{v}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the mass of the projectile, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}$&lt;/span&gt; is its velocity, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{g}$&lt;/span&gt; is the acceleration due to gravity, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is a drag coefficient , &lt;span class=&quot;math-container&quot;&gt;$\dot{\mathbf{v}}$&lt;/span&gt; is the time-derivative of the velocity, and &lt;span class=&quot;math-container&quot;&gt;$\lVert \mathbf{v} \rVert$&lt;/span&gt; is the magnitude of the velocity. Also assume that the initial velocity is&#xA;&lt;span class=&quot;math-container&quot;&gt;$&#xA; \mathbf{v}(0) = \mathbf{v}_0 \,.&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we define&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \mathbf{f}(t, \mathbf{v}) := \mathbf{g} - \frac{k}{m}\,\lVert\mathbf{v}\rVert\,\mathbf{v} \,.&#xA;$$&lt;/span&gt;&#xA;we can write our equation as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \dot{\mathbf{v}} = \mathbf{f}(t, \mathbf{v})&#xA;$$&lt;/span&gt;&#xA;The RK4 discretization with a stepsize &lt;span class=&quot;math-container&quot;&gt;$h = t_{n+1} - t_n$&lt;/span&gt; can then be expressed as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \mathbf{v}_{n+1} = \mathbf{v} + \frac{h}{6}(\mathbf{k}_1 + 2\mathbf{k}_2 + 2\mathbf{k}_3 + \mathbf{k}_4)&#xA;$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;  \mathbf{k}_1 &amp;amp;= f(t_n, \mathbf{v}_n) ,\\&#xA;  \mathbf{k}_2 &amp;amp;= f(t_n + \frac{h}{2}, \mathbf{v}_n + \frac{h}{2} \mathbf{k}_1) ,\\&#xA;  \mathbf{k}_3 &amp;amp;= f(t_n + \frac{h}{2}, \mathbf{v}_n + \frac{h}{2} \mathbf{k}_2) &#xA;\end{align}&lt;/span&gt;&#xA;and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \mathbf{k}_4 = f(t_n + h, \mathbf{v}_n + h \mathbf{k}_3) \,.&#xA;$$&lt;/span&gt;&#xA;A similar approach is used to find the position &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; by integrating&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \dot{\mathbf{x}} = \mathbf{v}&#xA;$$&lt;/span&gt;&#xA;with initial position &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}(0) = \mathbf{0}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For our 2D Python implementation we can define the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(t, v, g, m, k):&#xA;    kmv = np.linalg.norm(v)*(k/m)&#xA;    return (g - kmv*v)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The RK4 method can be implemented as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def RK4(tn, xn, vn, h, g, m, k):&#xA;&#xA;    k1v = f(tn, vn, g, m, k)&#xA;    k2v = f(tn + h/2, vn + k1*h/2, g, m, k)&#xA;    k3v = f(tn + h/2, vn + k2*h/2, g, m, k)&#xA;    k4v = f(tn + h, vn + k3*h, g, m, k)&#xA;    vn1 = vn + (k1v + 2*k2v + 2*k3v + k4v)*(h/6)&#xA;&#xA;    k1x = vn&#xA;    k2x = vn + k1v*h/2&#xA;    k3x = vn + k2v*h/2&#xA;    k4x = vn + k3v*h&#xA;    xn1 = xn + (k1x + 2*k2x + 2*k3x + k4x)*(h/6)&#xA;    # xn1 = xn + vn*h + (k1v+k2v+k3v)*h**2/6&#xA;&#xA;    return vn1, xn1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, the iterative process can be implemented using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; theta = math.radians(30)&#xA; v0 = np.array([math.cos(theta), math.sin(theta)])&#xA; x0 = np.array([0, 0])&#xA; t0 = 0&#xA; x = []&#xA; v = []&#xA; t = []&#xA; x.append(x0)&#xA; v.append(v0)&#xA; t.append(t0)&#xA;&#xA; m = 1&#xA; k = 0.01&#xA; g = np.array([0, -9.81])&#xA; h = 0.01&#xA;&#xA; tn = t0&#xA; vn = v0&#xA; xn = x0&#xA; maxt = 1&#xA;&#xA; while (t &amp;lt; maxt):&#xA;   vn, xn = RK4(tn, xn, vn, h, g, m, k)&#xA;   tn = tn + h&#xA;   t.append(tn)&#xA;   x.append(xn)&#xA;   v.append(vn)&#xA;&#xA; print t[-1]&#xA; print x[-1]&#xA; print v[-1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$h = 0.01$&lt;/span&gt; we get the final position &lt;span class=&quot;math-container&quot;&gt;$0.85299327, -4.34011443$&lt;/span&gt;.  The stopping condition that I've used can lead to different end times depending on the value of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a feeling that in the original question stopping conditions are being achieved at different times.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastEditorUserId="6839" LastEditDate="2020-06-13T19:36:10.997" LastActivityDate="2020-06-13T19:36:10.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21064" PostTypeId="2" ParentId="21060" CreationDate="2015-10-21T04:55:51.397" Score="2" Body="&lt;p&gt;Thanks in large part to Biswajit and &lt;a href=&quot;http://ocw.mit.edu/courses/mathematics/18-330-introduction-to-numerical-analysis-spring-2012/lecture-notes/MIT18_330S12_Chapter5.pdf&quot; rel=&quot;nofollow&quot;&gt;this MIT OCW slide, page 4&lt;/a&gt;, and some error on my part. My initial position was set to [0 5.5] instead of [0 0] from one of my previous attempts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my final code, using both Biswajit's answer, and the MIT slides:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;while p[1, iterr] &gt;= 0 and iterr &amp;lt; 9999:&#xA;        p0 = p[:, iterr]&#xA;        v0 = v[:, iterr]&#xA;        t0 = t[0, iterr]&#xA;        vp1 = f(t0, v0)&#xA;        vp2 = f(t0 + h/2, v0 + h * vp1/2)&#xA;        vp3 = f(t0 + h/2, v0 + h * vp2/2)&#xA;        vp4 = f(t0 + h, v0 + h * vp3)&#xA;        v[:, iterr + 1] = v0 + h * (vp1 + 2 * (vp2 + vp3) + vp4)/6&#xA;        p[:, iterr + 1] = p0 + h * v[:, iterr + 1]&#xA;        t[0, iterr + 1] = t0 + h&#xA;        iterr += 1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;def f(t, v):&#xA;    magv = np.linalg.norm(v)&#xA;    return np.array([k * magv * v[0], k * magv * v&lt;a href=&quot;http://ocw.mit.edu/courses/mathematics/18-330-introduction-to-numerical-analysis-spring-2012/lecture-notes/MIT18_330S12_Chapter5.pdf&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt; - 9.81])&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, k is the entire drag coefficient-mass-reference-area term, and $v^2$ is the velocity dependence expressed as $$||\mathbf v||*\mathbf v$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the code needs to be any clearer, I will comment out the parts. Anyone seeing this, notice that acceleration here does not depend on the time of the simulation, however, that is not always true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you to everyone who helped me look at this problem.&lt;/p&gt;&#xA;" OwnerUserId="17947" LastActivityDate="2015-10-21T04:55:51.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21065" PostTypeId="2" ParentId="20782" CreationDate="2015-10-21T05:25:13.317" Score="1" Body="&lt;p&gt;1) You can't, so you need trajectories that do not do such operations, or an analysis implementation that undoes such wrapping on the fly, e.g. by noticing a jump of around the size of the simulation cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Don't know, try the dl_poly documentation. &lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2015-10-21T05:25:13.317" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21066" PostTypeId="1" CreationDate="2015-10-21T07:43:56.210" Score="0" ViewCount="529" Body="&lt;p&gt;Let a Matrix $\mathbf{Y=[A\;b;b^T\;}d]$, where $d$ is a scalar, and $\mathbf{b}$ is a vector.&#xA;How to find the cholesky factorization of $\mathbf{Y}$ if i know the cholesky factorization of $\mathbf{A}$.?&lt;/p&gt;&#xA;" OwnerUserId="17949" LastActivityDate="2015-10-21T07:43:56.210" Title="Cholesky factorization of a block matrix" Tags="|linear-algebra|block-decomposition|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21069" PostTypeId="1" AcceptedAnswerId="21076" CreationDate="2015-10-21T10:32:06.707" Score="2" ViewCount="161" Body="&lt;p&gt;Solving acoustical wave equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;c_0^2\partial_{xx}p-\partial_{tt}=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using forward-time centered-space FDM is not very convenient cause of numerical dispersion etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about using a little physics and solve this as a system of first order equations (linearized Euler equations), i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla p = -\rho_0\partial_t\vec{v}&#xA;$$&#xA;$$&#xA;\partial_t\rho+\rho_0\nabla \cdot\vec{v}=0&#xA;$$&#xA;$$&#xA;c_0^2\partial_t \rho=\partial_tp&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2D case let's denote $u$, $v$ the components of velocity vector, let $\Delta x \equiv \Delta y$ and discretize:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_{i,j}^{n+1}=p_{i,j}^{n}+\frac{\rho_0 c_0^2 \Delta t}{\Delta x}\left(u_{i,j}^{n}-u_{i,j+1}^{n} + v_{i,j}^{n}-v_{i+1,j}^{n} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_{i,j}^{n+1}=u_{i,j}^{n}+\frac{\Delta t}{\rho_0 \Delta x}\left(p^{n}_{i,j}-p^{n}_{i,j+1} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;v_{i,j}^{n+1}=v_{i,j}^{n}+\frac{\Delta t}{\rho_0 \Delta x}\left(p^{n}_{i,j}-p^{n}_{i+1,j} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is that correct?&lt;/li&gt;&#xA;&lt;li&gt;Would that help to get solution with lower numerical noise, dispersion etc.?&lt;/li&gt;&#xA;&lt;li&gt;Would you use forward discretization as above or centered one?&lt;/li&gt;&#xA;&lt;li&gt;How would the CFL condition look like for this case?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17721" LastActivityDate="2015-10-22T07:19:31.223" Title="FDM - Solving acoustical wave equation via first order PDE's" Tags="|pde|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21070" PostTypeId="1" AcceptedAnswerId="21077" CreationDate="2015-10-21T12:42:46.060" Score="4" ViewCount="503" Body="&lt;p&gt;I want to calculate the following equation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\theta  \Gamma \left(\kappa+1,\frac{o}{\theta }\right)-o  \Gamma \left(\kappa,\frac{o}{\theta }\right)}{\Gamma (\kappa)}+o+s$$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$s&amp;gt;0, o&amp;gt;0, \kappa&amp;gt;0, \theta&amp;gt;0$&lt;/span&gt; and in which &lt;span class=&quot;math-container&quot;&gt;$\Gamma(\kappa,x)$&lt;/span&gt; represents the upper incomplete gamma function and &lt;span class=&quot;math-container&quot;&gt;$\Gamma(\kappa)$&lt;/span&gt; the (true) gamma function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current implementation of this equation in c++ using boost &lt;a href=&quot;http://www.boost.org/doc/libs/1_59_0/libs/math/doc/html/math_toolkit/sf_gamma/igamma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tgamma&lt;/a&gt;&#xA;returns an overflow for large &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; values.&#xA;Is there a way to transform this equation?&lt;/p&gt;&#xA;" OwnerUserId="17952" LastEditorUserId="17952" LastEditDate="2019-09-25T12:44:20.780" LastActivityDate="2019-09-25T12:50:17.090" Title="overflow upper incomplete gamma function" Tags="|c++|floating-point|special-functions|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21071" PostTypeId="1" CreationDate="2015-10-21T13:32:33.427" Score="6" ViewCount="277" Body="&lt;p&gt;I want to solve a Sturm-Liouville problem in 1D, i.e.,&#xA;\begin{align}&#xA;[p(x)\ u'(x)]'+q(x)u(x) = f(x)&#xA;\end{align}&#xA;with boundary conditions&#xA;\begin{align}&#xA;u(0)=a \hspace{1cm} u'(0)=b&#xA;\end{align}&#xA;How do I implement these conditions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I did:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I multiply both sides by a test function $v\in\lbrace c\in H^1\big\vert v(0)=0, v'(0)=0 \rbrace$ and integrate over the computational domain, i.e. from 0 to 1 in my case.&#xA;\begin{align}&#xA;-\int\limits_0^1\left(p(x)u'(x)\right)'v(x)dx &#xA;+\int\limits_0^1q(x)u(x)v(x)dx&#xA; =&#xA;\int\limits_0^1f (x)v(x)dx&#xA;\end{align}&#xA; The first term can be transformed by partial integration. &#xA;\begin{align}&#xA;-p(1)u'(1)v(1) + \underset{=0}{\underbrace{p(0)u'(0)v(0)}} +\int\limits_0^1p(x)u'(x)v'(x)dx&#xA;\end{align}&#xA;I want to use hermite cubic splines denoted by $\phi_k^l$ with the property&#xA;\begin{align}&#xA;\left(\phi_k^l\right)^{(m)}(x_j)=\delta_{kj}\delta_{lm}&#xA;\end{align}&#xA;where $(m)$ denotes the 0th or 1st derivative. Inserting this in the weak formulation above we finally arrive at the matrix equation&#xA;\begin{align}&#xA;\nonumber&#xA;-p(1)u^1_{N +1}&amp;amp;\delta_{0l}\delta_{N+1j}&#xA;+a\int\limits_0^1&#xA;dx p(x)\left[\phi_0^0\,'(x)\phi_j^l\,'(x) + q(x)\phi_0^0(x)\phi_j^l(x)\right]\\&#xA;&amp;amp; +b\int\limits_0^1&#xA;dx\, p(x)\left[\phi_0^1\,'(x)\phi_j^l\,'(x) + q(x)\phi_0^1(x)\phi_j^l(x)\right]\\&#xA;&amp;amp;+\sum\limits_{i=1}^{N+1}\sum\limits_{k=0}^1u_i^k&#xA;\int\limits_0^1&#xA;dx\, p(x)\left[\phi_i^k\,'(x)\phi_j^l\,'(x) + q(x)\phi_i^k(x)\phi_j^l(x)\right]\\&#xA;=&amp;amp;\int\limits_0^1f (x)\phi_j^l (x)dx \hspace{2cm}j=1,\dots,N+1\ \ l=0,1&#xA;\end{align}&#xA;Is this derivation correct up to here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix equation is set up by first mapping every element to the standard intervall $[-1,1]$ and then using Gauss Integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Afterwards the global matrix is assembled. Finally boundary conditions have to be implemented according to the above equation. This means we delete the lines and columns corresponding to the coeffiicients of $\phi_0^0$ and $\phi_0^1$. Subtracting the second and third term (the ones with a and b in front) from the left hand side of the above equation on the right hand side and subtraction $p(1)$ at the corresponding position of the matrix we arrive at the final matrix equation. This is visualized in the figure &lt;a href=&quot;https://i.stack.imgur.com/SbHoO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Implementation of boundary conditions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But using the extremely simple case of $p(x)=q(x)=1$, $f(x)=0$ and using only one domain we can find an analytic solution. The comparison between the analytic and the numerical solution is shown in figure &lt;a href=&quot;https://i.stack.imgur.com/H9B94.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;comparision of solutions&lt;/a&gt;.&#xA;Thus there must be a mistake in the above implementation process (other boundary conditions provided at both ends of the domain give correct solutions.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an increasing number of domains the solution does also not converge. Therefore something is obviously wrong but I do not have any idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please help me with this?&lt;/p&gt;&#xA;" OwnerUserId="17953" LastEditorUserId="9667" LastEditDate="2015-10-22T02:06:44.470" LastActivityDate="2015-10-22T02:06:44.470" Title="Boundary conditions in the Finite Element Method at only one side of computational domain" Tags="|finite-element|numerical-analysis|boundary-conditions|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21072" PostTypeId="1" CreationDate="2015-10-21T14:59:34.597" Score="1" ViewCount="422" Body="&lt;p&gt;I am trying to solve the elasticity equation using Crouzeix-Raviart element. Since this can not be solved using continuous FEM I am trying to solve this by discontinuous Galerkin method. In the discontinuous Galerkin formulation the left hand part of equation $a(U,v)$ term has two parts, the first part is the stiffness and the second part provides the stabilization to the equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stabilization term has the jump function $\int[U][v] $, where $[U]=U^+-U^- $ and $[v]=v^+-v^- $. I am finding it difficult to implement this. Could someone please suggest some literature which explains how the jump term is treated in matrix form and is solved.&lt;/p&gt;&#xA;" OwnerUserId="17955" LastEditorDisplayName="user17774" LastEditDate="2015-10-22T02:08:00.717" LastActivityDate="2015-10-22T02:08:00.717" Title="Solving the Jump function in discontinuous Galerkin method" Tags="|finite-element|numerical-analysis|discontinuous-galerkin|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21073" PostTypeId="1" CreationDate="2015-10-21T21:00:25.467" Score="1" ViewCount="655" Body="&lt;p&gt;This is a visualization/programming design problem. I am working on building a geospatial agent based model, meaning that there are multiple agents interacting within some particular spatial container. The movement of the agents and their interactions are based upon some biased random walks and integro-differential equations. I was trying to figure out which package to use for visualizing this, given some technical constraints. I wrote the model prototype in python, but here is the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a way to visualize the interactions between these agents over time.&#xA;I looked at &lt;code&gt;matplotlib&lt;/code&gt;, but the animation features will run into trouble when I add more than a few agents. I believe on the &lt;code&gt;matplotlib&lt;/code&gt; site itself the authors indicate that matplotlib animations are not ready for even medium sized scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to stay open source--and free. The other choices seem to be Mayavi--which uses VTK. I have used this a bit in the past, but was not sure how well Mayavi can handle visualizing a fair number of agents--say between 100-1000. I also was not sure how actively maintained Mayavi was.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vispy is another choice. This is also a very new library, so I am not sure how stable it is. It relies on on an interface to WebGL. I was not sure if this was a good choice to use yet, since not so many people have experience with it--not so many blog posts on Vispy yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally there are also some other hard core math visualization libraries in python like the finite elements solvers: Fenics or others. But I don't have experience with any of these, so not sure if they might work better for this particular problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are any choices I am missing, please suggest them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional Note: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked at agent based modeling software like Netlogo and Repast, etc. I don't really want to use these because they seem to impose a lot of constraints on the visualization and modeling choices--and I have to learn yet another language :). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was a note from the Vispy google group about VTK and mayavi.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Mayavi is based on VTK, an old and big library which uses legacy OpenGL &#xA;(fixed function pipeline). Mayavi itself is mostly written in C++, and &#xA;has trouble attracting contributors. . . . the lead developer has said &#xA;its dying a slow death. &#xA;&#xA;Vispy uses modern opengl and is implemented all the way in Python. It &#xA;only relies on one of several backends to provide a window to render in. &#xA;This makes Vispy more powerful (we can make use of more advanced &#xA;shaders) and flexible. Mayavi and VTK have been around for quite a while &#xA;though, so it will take some time for Vispy to match the functionality. &quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7530" LastEditorUserId="7530" LastEditDate="2015-10-29T18:25:38.493" LastActivityDate="2015-10-29T18:25:38.493" Title="best package choice for visualizing agent based model simulations in python" Tags="|python|visualization|vtk|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21075" PostTypeId="1" CreationDate="2015-10-22T00:46:52.287" Score="1" ViewCount="129" Body="&lt;p&gt;I am solving a problem involving the line with the set of points $(x_3,y_3)$ that are equidistant to two given points $(x_1,y_1)$ and $(x_2,y_2)$. The equation for this line is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(x_3 - x_1)^2 + (y_3 - y_1)^2 = (x_3 - x_2)^2 + (y_3 - y_2)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which boils down to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$2 (x_1 - x_2) x_3 + 2 (y_1 - y_2) y_3 = x_1^2 + y_1^2 - x_2^2 - y_2^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question I have is quite simple: how should I write the right hand side of this equation? I know that there is more than one way to do it and that some alternatives might be more numerically accurate than others, due to floating point errors.&#xA;\begin{align}&#xA;    &amp;amp;x_1^2 + y_1^2 - x_2^2 - y_2^2 = \\&#xA;    &amp;amp;(x_1^2 + y_1^2) - (x_2^2 + y_2^2) = \\&#xA;    &amp;amp;(x_1^2 - x2^2) + (y_1^2 - y_2^2) = \\&#xA;    &amp;amp;(x_1 + x_2) (x_1-x_2) + (y_1 + y_2) (y_1 - y_2) = \\&#xA;    &amp;amp; \vdots&#xA;\end{align}&#xA;How do I compute the right hand sidein the most accurate way?&lt;/p&gt;&#xA;" OwnerUserId="1822" LastEditorUserId="9667" LastEditDate="2015-10-22T02:03:44.633" LastActivityDate="2015-10-22T13:21:32.690" Title="How do I add some floating point numbers, keeping numerical accuracy in mind?" Tags="|numerical-analysis|stability|floating-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21076" PostTypeId="2" ParentId="21069" CreationDate="2015-10-22T07:19:31.223" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is that correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Your system of PDEs looks correct though I didn't double-check the minus signs and such.  The acoustic wave equation is often solved numerically in first-order form.  The choice of second- or first-order form is often based on historical biases or culture in a given field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The one-sided differences you've used lead to an unstable scheme (it doesn't even satisfy the CFL condition!)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Would that help to get a solution with lower numerical noise, dispersion, etc.?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not by itself.  Issues related to numerical errors are essentially the same for both forms.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Would you use forward discretization as above or centered one?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A centered difference would be better than what you have, since it satisfies the CFL condition and is stable for small enough time step sizes.  It is dispersive, so if you have steep gradients in the solution (or if you do not resolve high frequencies sufficiently well) you will get oscillations that are of a numerical origin.  Another approach is to apply an upwind discretization along characteristics.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How would the CFL condition look like for this case?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Geometrically, the CFL condition is the same (the numerical domain of dependence must include the physical domain of dependence).  Algebraically, it is also the same -- for a centered-difference scheme, you need $c\Delta t/\Delta x \le 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One nice reference is &lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot; rel=&quot;nofollow&quot;&gt;LeVeque's book on finite volume methods&lt;/a&gt;; see Section 2.6-2.8 for a full derivation; Chapter 3 for an in-depth discussion of the system; and Chapters 4-6 for numerical discretization and examples.  Chapter 9 is all about acoustics with spatially-varying coefficients.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-10-22T07:19:31.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21077" PostTypeId="2" ParentId="21070" CreationDate="2015-10-22T08:20:13.030" Score="4" Body="&lt;p&gt;This should be obvious, but the first thing you should try is to use the&#xA;&lt;a href=&quot;http://dlmf.nist.gov/8.2.E4&quot; rel=&quot;nofollow&quot;&gt;normalized incomplete gamma function&lt;/a&gt; $Q(a,x)=\frac{\Gamma(a,x)}{\Gamma(a)}$&#xA;for the two summands in the numerator of your fraction (for the first you have to add one recurrence step for $k+1$). In Boost $Q(a,x)$ is called &lt;code&gt;gamma_q&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4853" LastEditorUserId="713" LastEditDate="2015-10-22T20:47:50.793" LastActivityDate="2015-10-22T20:47:50.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21078" PostTypeId="1" AcceptedAnswerId="21079" CreationDate="2015-10-22T09:54:15.830" Score="8" ViewCount="702" Body="&lt;p&gt;I am Computer Scientist and now I am interested in matrix multiplication on GPUs. My research are focused on matrix free finite elements method where I multiply sparse matrix. Sparse matrix could multiply regular or matrix free. In general based on special coordinate function. I have a few general question: How popular is this method? Does any another name exist for this method? I am also looking for books and article concentrate on finite element method especially matrix free multiplication and I consider about general books and article. Because many article based on rather complicated examples like conjugate gradient method or something different. As I said in topic name I try to use this method in process tomography visualization.&lt;/p&gt;&#xA;" OwnerUserId="17965" LastActivityDate="2015-10-22T21:13:56.590" Title="Matrix free finite elements method for visualization in process tomography" Tags="|computational-geometry|computer-vision|matrix-free|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21079" PostTypeId="2" ParentId="21078" CreationDate="2015-10-22T10:27:43.970" Score="7" Body="&lt;p&gt;Matrix-free method is a general name for a class of algorithms, rather than a particular method.&#xA;For example, consider solving the linear equation $Ax=b.$ If you were to solve this this problem using the Gauss elimination method, for example, then you need to pre-compute all elements of $A$ and keep them during the algorithm. If you were to use Krylov-type iterative solvers, you need to multiply the matrix $A$ by a vector at every step. You can do it differently.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pre-compute $A$ and keep it in memory (as a sparse array, perhaps).&lt;/li&gt;&#xA;&lt;li&gt;Do not store $A,$ but re-compute (some of) its elements every time you need to multiply $A$ by a vector.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The first approach is &lt;i&gt;not&lt;/i&gt; a matrix-free method, while the second approach is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question now:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Matrix-free elements have been originally motivated by the lack of operative memory for computations. Although the problem is not solved in general yet, in many applications the remedy is found using the distributed-memory parallel computations or more efficient matrix formats, which allow to store matrix and avoid re-calculation of its elements.&lt;/li&gt;&#xA;&lt;li&gt;Matrix-free method are indeed promising for GPUs, since it may be cheaper to re-compute elements of the matrix on the fast GPU core, than to deliver them to this core from the main memory.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17966" LastActivityDate="2015-10-22T10:27:43.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21080" PostTypeId="2" ParentId="21075" CreationDate="2015-10-22T13:21:32.690" Score="1" Body="&lt;p&gt;Compute each of the additive terms, sort them, then add up the list. If each of your expressions is going to be this short, the cost of the sorting might not kill your performance. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-10-22T13:21:32.690" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21081" PostTypeId="1" CreationDate="2015-10-22T09:08:42.923" Score="0" ViewCount="413" Body="&lt;p&gt;I am Computer Scientist and now I am interested in matrix multiplication on GPUs. My research are focused on matrix free finite elements method where I multiply sparse matrix. Sparse matrix could multiply regular or matrix free. In general based on special coordinate function.&#xA;I have a few general question:&#xA;How popular is this method? Does any another name exist for this method?&#xA;I am also looking for books and article concentrate on finite element method especially matrix free multiplication and I consider about general books and article. Because many article based on rather complicated examples like conjugate gradient method or something different.&lt;/p&gt;&#xA;" OwnerUserId="17965" OwnerDisplayName="Konrad" LastEditorUserId="276" LastEditDate="2017-10-10T00:42:01.550" LastActivityDate="2017-10-10T00:42:01.550" Title="Matrix free finite elements method" Tags="|finite-element|gpu|" AnswerCount="1" CommentCount="6" ClosedDate="2015-10-22T22:43:01.930" ContentLicense="CC BY-SA 3.0" />
  <row Id="21082" PostTypeId="2" ParentId="21081" CreationDate="2015-10-22T14:46:22.967" Score="0" Body="&lt;p&gt;Matrix-free finite elements are relatively well-known. For explicit methods for transient problems, this involves applying the finite element matrix using small reference matrices and geometry-specific transformations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For implicit problems, this is usually done in conjunction with an iterative solver such as CG, GMRES, etc. Note that this typically also requires a preconditioner, many of which may not map well directly to GPUs. For nonlinear implicit problems, this may also be paired with a matrix-free approximation of the Jacobian in a linearization (see for example &lt;a href=&quot;http://users.ices.utexas.edu/~benkirk/dissertation.pdf&quot; rel=&quot;nofollow&quot;&gt;Ben Kirk's thesis&lt;/a&gt; for an application of matrix-free implicit methods in CFD). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For GPU-specific implementations of explicit methods, there is ample literature on matrix-free implementations of finite element (specifically Discontinuous Galerkin) methods; see, for example, &lt;a href=&quot;http://www.springer.com/us/book/9780387720654&quot; rel=&quot;nofollow&quot;&gt;Hesthaven/Warburton&lt;/a&gt; (not GPU specific, but implementation is similar) and &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999109003647&quot; rel=&quot;nofollow&quot;&gt;Klockner et al.&lt;/a&gt;.  For some literature on the GPU implementation of matrix-free implicit solvers, see &lt;a href=&quot;http://arxiv.org/abs/1506.05996&quot; rel=&quot;nofollow&quot;&gt;Remacle, Gandham, Warburton&lt;/a&gt;, where they solve the heat equation using a two-grid overlapping additive Schwarz preconditioner.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-10-22T14:46:22.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21084" PostTypeId="2" ParentId="21078" CreationDate="2015-10-22T19:54:07.190" Score="8" Body="&lt;p&gt;To add to Dmitry's answer (copied over from the deleted version of this question):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrix-free finite elements are relatively well-known. For explicit methods for transient problems, this involves applying the finite element matrix using small reference matrices and geometry-specific transformations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For implicit problems, this is usually done in conjunction with an iterative solver such as CG, GMRES, etc. Note that this typically also requires a preconditioner, many of which may not map well directly to GPUs. For nonlinear implicit problems, this may also be paired with a matrix-free approximation of the Jacobian in a linearization (see for example &lt;a href=&quot;http://users.ices.utexas.edu/~benkirk/dissertation.pdf&quot; rel=&quot;noreferrer&quot;&gt;Ben Kirk's thesis&lt;/a&gt; for an application of matrix-free implicit methods in CFD). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For GPU-specific implementations of explicit methods, there is ample literature on matrix-free implementations of finite element (specifically Discontinuous Galerkin) methods; see, for example, &lt;a href=&quot;http://www.springer.com/us/book/9780387720654&quot; rel=&quot;noreferrer&quot;&gt;Hesthaven/Warburton&lt;/a&gt; (not GPU specific, but implementation is similar) and &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999109003647&quot; rel=&quot;noreferrer&quot;&gt;Klockner et al.&lt;/a&gt;.  For some literature on the GPU implementation of matrix-free implicit solvers, see &lt;a href=&quot;http://arxiv.org/abs/1506.05996&quot; rel=&quot;noreferrer&quot;&gt;Remacle, Gandham, Warburton&lt;/a&gt;, where they solve the heat equation using a two-grid overlapping additive Schwarz preconditioner.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-10-22T21:13:56.590" LastActivityDate="2015-10-22T21:13:56.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21085" PostTypeId="1" AcceptedAnswerId="21086" CreationDate="2015-10-22T21:30:27.217" Score="7" ViewCount="1168" Body="&lt;p&gt;Why do people use the classical least squares approach so often ? If I use the absolute value instead of the power, I immediately know how far away the solution is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ res = \frac{abs(x - x_{model})}{x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the benefit of squaring ?&lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2015-11-06T16:54:05.503" Title="cost function: why use power of two ? abs(x - xhat) / x better?" Tags="|optimization|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21086" PostTypeId="2" ParentId="21085" CreationDate="2015-10-22T22:41:47.203" Score="16" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why do people use the classical least squares approach so often?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Primarily, squaring makes the problem twice-differentiable, thus many different solution methods apply (quasi-Newton methods, Levenberg-Marquardt, Gauss-Newton), and there is still some flexibility with respect to what is being squared (e.g., I can replace $x$ with $f(x)$ for a wide class of functions $f$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From these methods, there are some nice linear algebra benefits, too: we can use Cholesky/sparse Cholesky.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If I use the absolute value instead of the power, I immediately know how far away the solution is...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The absolute value approach (more generally, L1-minimization approaches) can sometimes be tractable, if an LP reformulation is possible, but it can also really slow things down if an LP formulation is not possible (e.g., taking the L1 norm of a difference of nonlinear functions).&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-10-22T22:41:47.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21087" PostTypeId="2" ParentId="21085" CreationDate="2015-10-23T01:46:02.623" Score="12" Body="&lt;p&gt;Athough Geoff Oxberry's answer addresses computational reasons why it's easier to minimize the sum of squared residuals than the sum of absolute values, it doesn't discuss statistical reasons for preferring the least squares solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For problems in which measurement errors are independent and normally distributed, the (appropriately weighted) least squares solution provide a maximum likelihood estimator of the parameters.  Even if the measurement errors aren't independent, it's possible to extend ordinary least squares to correct for correlation of the residuals.  In the linear case there are a number of other important statistical properties that the least squares solution has.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, there are times when the parameter estimate the minimizes the sum of absolute values of the residuals is better from a statistical point of view.  In particular, if there are rare but extremely bad &quot;outlier&quot; measurements that aren't normally distributed, minimizing the sum of the absolute values of the residuals will typically provide a better estimate than least squares.  In &quot;robust regression&quot; the sum of absolute values of the residuals and other related objective functions are often used.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="9667" LastEditDate="2015-10-23T02:47:36.147" LastActivityDate="2015-10-23T02:47:36.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21088" PostTypeId="1" AcceptedAnswerId="21090" CreationDate="2015-10-23T04:32:22.550" Score="1" ViewCount="220" Body="&lt;p&gt;We're currently solving the heat equation as a part of the PDE sequence in class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We've been given the formula:$$T(i, n+1) = T(i,n)+\alpha \left [\frac{T(i+1,n)-2 T(i, n)+T(i-1,n)}{\Delta x^2} \right ] \quad \Delta t$$&#xA;I've got boundary conditions handled. They aren't the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem here is, if one notices, $\alpha$, $\Delta x^2$, and $\Delta t$ are all permanent constants throughout the number of iterations. Thus, they can be pre-evaluated. There's just that one problem where, if the value of $\frac {\alpha \Delta t}{\Delta x^2}$ becomes larger than 0.5, I start running into problems with the equation itself. One can see that when that term = 0.5:&#xA;$$\begin{matrix}&#xA;100 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;100 &amp;amp; 50 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;100 &amp;amp; 50 &amp;amp; 25 &amp;amp; 0 &amp;amp; 0\\&#xA;100 &amp;amp; 67.5 &amp;amp; 25 &amp;amp; 12.5 &amp;amp; 0&#xA;\end{matrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, every row represents a new time instance, and every column is a new discrete x-position element along the 'thin rod' that I am considering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The moment the $\frac {\alpha \Delta t}{\Delta x^2}$ goes below 0.5, however, we are 'good', meaning that this problem ceases to occur, but the propagation of temperature through the 'thin rod' is still really slow, even with a really high temperature at one end, which is held constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the equation provided wrong? Or am I misunderstanding something? Is there some error in sign somewhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: This is indeed a class assignment, but given the nature of the situation, I am inclined to think that it is either my understanding that is fundamentally flawed (beyond the code, just the concept) or I've simply been given the wrong equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: Error in evaluating the matrix. Corrected that.&lt;/p&gt;&#xA;" OwnerUserId="17947" LastEditorUserId="17947" LastEditDate="2015-10-23T05:45:03.683" LastActivityDate="2015-10-23T05:53:45.533" Title="Heat Equation Solution in One Dimension (x, t)" Tags="|pde|numerical-analysis|python|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21089" PostTypeId="2" ParentId="21088" CreationDate="2015-10-23T05:39:23.720" Score="0" Body="&lt;p&gt;You've forgotten a step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is:&#xA;$$T(i, n+1) = T(i,n)+\alpha \left [\frac{T(i+1,n)-2 T(i, n)+T(i-1,n)}{\Delta x^2} \right ] \quad \Delta t$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And you are evaluating:&#xA;$$T(i, n+1) = \alpha \left [\frac{T(i+1,n)-2 T(i, n)+T(i-1,n)}{\Delta x^2} \right ] \quad \Delta t$$&lt;/p&gt;&#xA;" OwnerUserId="17112" LastActivityDate="2015-10-23T05:39:23.720" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="21090" PostTypeId="2" ParentId="21088" CreationDate="2015-10-23T05:53:45.533" Score="1" Body="&lt;p&gt;You've stumbled upon a common problem with explicit schemes. This is a common issue with numerical analysis and it's called the Courant–Friedrichs–Lewy (CFL) condition (&lt;a href=&quot;https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, you have to adjust your time step size and the spatial discretization in order to meet this condition, which very much depends on the physical situation.  Usually, the time step is adjusted, since adaptive meshing (or remeshing and mapping your solution) is somewhat costly.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conceptually, this condition occurs because the &quot;communication&quot; between nodes becomes much slower than the &quot;motion&quot; of the solution.  This means that the temperature, let's think the diffusion of phonons for a more physical conceptualization, moves over multiple cells per time step.  What you want is a condition such that the phonons travel through one cell over many time steps.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CFL condition is a required condition only for explicit numerical schemes and is a necessary condition in both convection and diffusion problems.&lt;/p&gt;&#xA;" OwnerUserId="16463" LastActivityDate="2015-10-23T05:53:45.533" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21091" PostTypeId="1" CreationDate="2015-10-23T06:00:17.253" Score="1" ViewCount="130" Body="&lt;p&gt;I was estimating a falling object's position versus time by using a simple first order step function, where&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=2:length(t) % We're using Euler, so we need an initial previous point to start t=t(i);&#xA;&#xA;V=V+dv;&#xA;&#xA;rho_i=    % Calculate our height and then pull rho from data&#xA;&#xA;K=(m/(mg-.5*C_d*A*V^2*rho_i));&#xA;&#xA; dv=dt/K;&#xA;&#xA;&#xA;&#xA;  pos=pos-(V-V_previous)/2*dt;&#xA;&#xA; V_previous=V;&#xA;&#xA;&#xA; velocity_Vector(i)=V;&#xA; pos_Vector(i)=pos;&#xA; end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I'm wondering, how much will the error be after this thing falls, like many many kilometers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that it's a first order approximation, so there's error...But say I needed to know the bounds on that error.... what would I do to calculate that?&lt;/p&gt;&#xA;" OwnerUserId="17976" LastActivityDate="2015-10-23T06:00:17.253" Title="Euler's first order Error" Tags="|numerical-analysis|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21092" PostTypeId="1" CreationDate="2015-10-23T07:43:31.457" Score="3" ViewCount="54" Body="&lt;p&gt;I am a student doing physics hons and have had very little experience in programming. This semester we are supposed to do a computational project in thermodynamics. I have to solve these two coupled diff eqns:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;ω (p,T ) &amp;amp;= \frac{p ^2}{2m} + \frac{2}{N}\sum_q f(p − q)\,n (q) − \frac{1}{N^2}\sum_{s,t} f(s − t)\,n(s)\,n(t) \quad\text{and} \\&#xA;n(p) &amp;amp;= \frac{1}{\exp\left[\cfrac{ω (p,T ) − \mu}{kT}\right] − 1}&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega$ is the energy per boson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p$ is the momentum of a boson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n(p)$ is the number of bosons in the state with momentum $p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ is a function of the form &#xA;$$&#xA;  f(p)=\frac{1}{2} \left[\epsilon_0- \frac{p^2}{2m}\right]&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\epsilon_0$ is elementary excitation energy at 0 K. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N$ is total no of bosons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T$ is temp and $k$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone guide me to any simple methods to generate some crude solution to this problem? Based on a paper: &quot;Evaluation of speciﬁc heat for superﬂuid helium between 0 - 2.1 K based on nonlinear theory&quot; By Shosuke Sasaki (arXiv:0807.1361v1 [cond-mat.other] 9 Jul 2008&lt;/p&gt;&#xA;" OwnerUserId="17977" LastEditorUserId="7626" LastEditDate="2015-10-23T20:51:35.137" LastActivityDate="2015-10-23T20:51:35.137" Title="Coupled Diff Equation from Bose Einstein distribution" Tags="|numerical-analysis|c++|computational-physics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21093" PostTypeId="1" CreationDate="2015-10-23T05:06:17.190" Score="1" ViewCount="121" Body="&lt;p&gt;I was estimating a falling object's position versus time by using a simple first order step function, where &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=2:length(t) % We're using Euler, so we need an initial previous point to start&#xA;&#xA;    t = t(i);        &#xA;    V = V + dv;        &#xA;    rho_i =    % Calculate our height and then pull rho from data        &#xA;    K = m/(mg - 0.5*C_d*A*V^2*rho_i);        &#xA;    dv = dt/K;    &#xA;    pos = pos - (V - V_previous)/2*dt;        &#xA;    V_previous=V;&#xA;    velocity_Vector(i) = V;&#xA;    pos_Vector(i) = pos;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I'm wondering, how much will the error be after this thing falls, like many many kilometers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that it's a first order approximation, so there's error. But say I needed to know the bounds on that error; what would I do to calculate that?&lt;/p&gt;&#xA;" OwnerDisplayName="John doore" LastEditorUserId="9667" LastEditDate="2015-10-23T19:42:23.133" LastActivityDate="2015-10-23T21:13:26.703" Title="Propagation of error using Euler's first order method" Tags="|finite-difference|computational-physics|" AnswerCount="2" CommentCount="3" ClosedDate="2015-10-24T03:09:23.503" ContentLicense="CC BY-SA 3.0" />
  <row Id="21094" PostTypeId="2" ParentId="21085" CreationDate="2015-10-23T10:57:08.843" Score="1" Body="&lt;p&gt;For one-dimensional data, the point that minimizes the sum of squared distances is the arithmetic mean and the point(s) that minimize the sum of absolute distances is/are the median. A median-like solution may sometimes be prefered over a mean-like solution, but the non-uniqueness and &quot;weird&quot; dependencies make it at elast less tractable theoretically. &lt;/p&gt;&#xA;" OwnerUserId="17979" LastActivityDate="2015-10-23T10:57:08.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21095" PostTypeId="1" CreationDate="2015-10-23T13:27:13.050" Score="5" ViewCount="136" Body="&lt;p&gt;This question is a follow-up to this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21021/what-are-some-ideas-to-preprocess-precondition-the-following-linear-system&quot;&gt;one&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $A\in \mathbb{R}^{n\times n}$ be large, sparse, symmetric and positive definite.  Suppose for I already know $m&amp;lt;n$ eigenpairs of $A$, corresponding to the largest eigenvalues, i.e. $\{( \lambda_i, \vec{q}_i) \}_{i=1}^m \subset \mathbb{R}^+\times  \mathbb{R}^n$.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to solve $A \vec{x}=\vec{b}$ for $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that we expand $\vec{x}$ as $\vec{x}=\sum_{i=1}^m \chi_i \vec{q}_i + \sum_{i=m+1}^n \chi_i \vec{q}_i$.  Now since the eigenvectors are orthogonal, we can take the dot product of both sides of the equation with $\vec{q}_j$, say, and immediately recover that $\chi_j = \vec{q}_j\cdot \vec{b}/ \lambda_j$.  So we define &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{x}_\mathrm{guess} = \sum_{i=1}^m \frac{\vec{q}_i\cdot \vec{b}}{\lambda_i} \vec{q}_i,$$ and use it as the first guess to any iterative solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be shocked if this isn't a standard idea, but I don't know what it is called.  What is it's name?  I would especially like to know what papers to cite. &lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-10-24T23:39:46.957" Title="Does this partial eigen-expansion have a name?" Tags="|linear-algebra|linear-solver|eigensystem|preconditioning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21096" PostTypeId="2" ParentId="21095" CreationDate="2015-10-23T13:46:32.417" Score="6" Body="&lt;p&gt;This is something like a truncated SVD or eigenvector expansion of your solution.  If you take &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_m = \sum_{j=1}^m \frac{q_j\cdot b}{\lambda_j}q_j$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $m=n$, this is the exact solution to $Ax=b$.  If you take instead the $m&amp;lt;n$ eigenvalues in the sum instead, you get an approximation. The problem is that it's a pretty poor approximation; since your eigenvalues are sorted from largest to smallest, $1/\lambda_n$ is the largest term in your series and your error is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\|x-x_m\| = O(1/\lambda_{n})$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, small eigenvalues correspond to low-frequency components in systems, which tend to have the largest components. Approximating your solution using higher frequency (larger magnitude eigenvalues) gives a good approximation of a matrix-vector product, but not the solution of a linear system.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-10-23T13:46:32.417" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21097" PostTypeId="1" CreationDate="2015-10-23T15:18:58.227" Score="5" ViewCount="462" Body="&lt;p&gt;I am using MATLAB's PDE toolbox to solve the differential equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-\nabla\cdot\left(c(x)\nabla u(x)\right) + a(x)u(x) = f(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The particular problem in question is an electrostatic problem, but has the solution (voltage) specified at several points inside the domain. Since the PDE tool is setup to apply boundary conditions along &lt;i&gt;edges&lt;/i&gt;, I thought I might apply these point conditions by the following process:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a boundary condition at point $x_0$, where we want to specify $u(x_0) = b_0$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c(x_0) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a(x_0) = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x_0) = b_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such that at the point $x = x_0$, the above differential equation simply becomes $u(x) = b_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this does not appear to result in the correct solution. Is this a mathematically valid process? If not, is there a preferred method of setting the solution values at certain arbitrary &lt;i&gt;points&lt;/i&gt; in the PDE tool? I am not using the GUI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my understanding, this is a fairly straightforward process using the FEM method. One merely specifies that the corresponding row of the stiffness matrix [K] is entirely 0, with the exception of a 1 at the index of the corresponding node. Then the value of that point is placed in the corresponding row of the solution [b] matrix. Unfortunately, these matrices are not accessible in the PDE tool, hence the question of how to proceed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Resolution&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you all for your help. In the end, it appears that prescribed value points are not currently compatible with MATLAB's &lt;code&gt;adaptmesh&lt;/code&gt; algorithm. I ended up writing my own mesh adaption algorithm and using the &lt;code&gt;assempde&lt;/code&gt; commands to specify the matrix data and solve.&lt;/p&gt;&#xA;" OwnerUserId="17983" LastEditorUserId="17983" LastEditDate="2015-11-06T03:41:41.293" LastActivityDate="2015-11-06T03:41:41.293" Title="Prescribe solution of a PDE at specific points" Tags="|matlab|finite-element|constraints|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21098" PostTypeId="2" ParentId="21093" CreationDate="2015-10-23T20:19:39.340" Score="1" Body="&lt;p&gt;In Euler's method, per step, the error is proportional to $(\Delta t)^2$, and over the entire interval, the cumulative error is of the order $\Delta t$, since the number of steps is proportional to $\frac {1}{\Delta t}$ and the errors add up. So one can put in the error as $$\Delta E \propto \Delta t$$ $$\Rightarrow \Delta E = K \Delta t$$ where $K$ is some constant that is specific to the parameters of the problem, and should be possible to determine using a few different step sizes.&lt;/p&gt;&#xA;" OwnerUserId="17947" LastActivityDate="2015-10-23T20:19:39.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21099" PostTypeId="2" ParentId="21095" CreationDate="2015-10-23T21:08:07.400" Score="6" Body="&lt;p&gt;I don't know for sure whether there is an existing name for this method, but @jessechan's suggestion of &quot;truncated eigenvector expansion&quot; sounds perfectly fine to me (and most people would understand it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question had a second part embedded, namely why is this not what everyone does to get a good initial guess for iterative solvers? The answer to this is that for all interesting cases, computing eigenvectors and eigenvalues is an extremely expensive proposition, far more expensive than doing a few more iterations (which are typically rather cheap). As a consequence, within our community of computational scientists, we most frequently don't try very hard to get good initial guesses for iterative solvers for linear system. If you happen to have a good initial guess (e.g., the previous solution's timestep, or an extrapolation of the previous two time steps), then use it; if you have to work to get an initial guess, forget it -- it's going to be too expensive.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2015-10-24T23:39:46.957" LastActivityDate="2015-10-24T23:39:46.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21100" PostTypeId="2" ParentId="21093" CreationDate="2015-10-23T21:13:26.703" Score="1" Body="&lt;p&gt;You would repeat the computation with various different values of the time step $\Delta t$ and get a table of the form&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\Delta t=0.1, x(T)=12.345$ &lt;/li&gt;&#xA;&lt;li&gt;$\Delta t=0.01, x(T)=12.033$ &lt;/li&gt;&#xA;&lt;li&gt;$\Delta t=0.001, x(T)=12.003$ &lt;/li&gt;&#xA;&lt;li&gt;$\Delta t=0.0001, x(T)=12.0003$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With data such as this, you can extrapolate what the most likely &lt;em&gt;exact&lt;/em&gt; value will be (here simply 12.0), and with this you can then evaluate the error you would have for any of the previous choices for $\Delta t$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-23T21:13:26.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21101" PostTypeId="1" CreationDate="2015-10-24T00:21:29.053" Score="1" ViewCount="110" Body="&lt;p&gt;I'm working on a fluid solver using dual-time stepping and everything works really well, except the convergence in pseudo-time is slow. I'd like to accelerate the convergence. I know multigrid methods are typically used, but I am thinking about exploring another approach that is, hopefully, easier to implement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started reading the literature on sequence convergence acceleration. My sequence in this case is the updated solution to the Navier-Stokes equations in pseudo-time and the sequence, if all goes well, converges to a constant field (alternatively, the pseudo-time derivative converges to zero) as an infinite number of steps are taken. My reading in this area has led me to vector extrapolation methods and the related algorithms, but I am unsure of a few points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sake of argument, consider the Reduced Rank Extrapolation (RRE) method. The algorithm can be found &lt;a href=&quot;https://math.temple.edu/~szyld/reports/RRE.Schwarz.report.pdf&quot; rel=&quot;nofollow&quot;&gt;in Table 3.1 of this report&lt;/a&gt;. In summary, you collect a set of solution vectors (solve your equations over a time step and store the updated values). Then, compute the derivative of the solution (or really just store the increment from the fluid solver since that is what we compute anyway) and the resulting vectors become columns in a big matrix. Find the QR factorization of this big matrix and solve a least-squares problem, and use the result to extrapolate your solution vector. Hopefully your convergence accelerates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, can these actually help in this case? I think they can, but I'm a bit fuzzy on it all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, can these methods be applied locally or do they need to be applied for all cells in the simulation simultaneously? Everything I have read just shows solution vectors. But I don't know if it's $nvar*ncells$ long, or if I can do the QR factorization in each cell to get an extrapolated vector. There's a big difference in cost between point-wise factorization and total-domain factorization in massively parallel simulations. The linked report says it can be done on subdomains, which seems to imply it is at least a &quot;regional&quot; (as in, not pointwise but at least within a processor) QR factorization.&lt;/p&gt;&#xA;" OwnerUserId="391" LastActivityDate="2016-09-14T14:32:35.003" Title="Application of vector extrapolation methods to convergence to a steady state solution" Tags="|fluid-dynamics|nonlinear-equations|convergence|extrapolation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21102" PostTypeId="1" CreationDate="2015-10-24T04:01:18.040" Score="1" ViewCount="58" Body="&lt;p&gt;What equation would you all use to estimate the wind speed over the Mojave Desert in December? I'd like to model the speed of the wind up to say 50 thousand feet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read up about wind speeds on Wikipedia and they give an analytical formula for wind gradient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\\ v_w(h) = v_{10} \cdot \left(  \frac {h} {h_{10}} \right)^ a$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found this link:&#xA;&lt;a href=&quot;http://digital-desert.com/mojave-preserve/geology/02.html&quot; rel=&quot;nofollow&quot;&gt;http://digital-desert.com/mojave-preserve/geology/02.html&lt;/a&gt;&#xA;But it doesn't tell me how to vary altitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would rather be using data that was taken, for example by NOAA, but couldn't find any. Does anybody have a link, algorithm, or recommendation of what I should do?&lt;/p&gt;&#xA;" OwnerUserId="17996" LastActivityDate="2015-10-24T04:01:18.040" Title="Estimating the wind speed above the mojave desert" Tags="|numerical-analysis|" AnswerCount="0" CommentCount="4" ClosedDate="2015-10-26T18:42:19.530" ContentLicense="CC BY-SA 3.0" />
  <row Id="21103" PostTypeId="1" AcceptedAnswerId="21115" CreationDate="2015-10-24T11:56:06.080" Score="8" ViewCount="2285" Body="&lt;p&gt;I have made a solver based on the &lt;a href=&quot;http://docs.sympy.org/dev/modules/diffgeom.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy.diffgeom&lt;/a&gt; library, where I use &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/integrate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scipy.Integrate&lt;/a&gt; to solve the following system of second order differential equations : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;u'' &amp;amp;+ \Gamma^0_{00}(u')^2 + 2\Gamma^0_{01}u'v' + \Gamma^0_{11}(v')^2 = 0,\\ &#xA;v'' &amp;amp;+ \Gamma^1_{00}(u')^2 + 2\Gamma^1_{01}u'v' + \Gamma^1_{11}(v')^2 = 0,&#xA;\end{align}&#xA;where $\Gamma^m_{ij}(u(s),v(s))$ is the Christoffel symbol of second kind. The geodesic solution $u = u(s),v = v(s)$ is a curve defined for the interval $s\in[s_0,s_1]$.&#xA;These equations can be described as a system of first order differential equations by setting $p = u'$, and $q = v'$ :&#xA;\begin{align}&#xA;p' &amp;amp;+ \Gamma^0_{00}p^2 + 2\Gamma^0_{01}pq + \Gamma^0_{11}q^2 = 0\\ &#xA;q' &amp;amp;+ \Gamma^1_{00}p^2 + 2\Gamma^1_{01}pq + \Gamma^1_{11}q^2 = 0&#xA;\end{align}&#xA;with the initial condition given as $u(s_0) = u_0, u'(s_0) = du_0, v(s_0) = &#xA;v_0, v'(s_0) = dv_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a code snippet of my implementation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(y,s,C,u,v):&#xA;    y0 = y[0] # u&#xA;    y1 = y[1] # u'&#xA;    y2 = y[2] # v&#xA;    y3 = y[3] # v'&#xA;    dy = np.zeros_like(y)&#xA;    dy[0] = y1&#xA;    dy[2] = y3&#xA;&#xA;    C = C.subs({u:y0,v:y2}) # Evaluate C for u,v = (u0,v0)&#xA;&#xA;    dy[1] = -C[0,0][0]*dy[0]**2 -\&#xA;           2*C[0,0][1]*dy[0]*dy[2] -\&#xA;             C[0,1][1]*dy[2]**2&#xA;    dy[3] = -C[1,0][0]*dy[0]**2 -\&#xA;           2*C[1,0][1]*dy[0]*dy[2] -\&#xA;             C[1,1][1]*dy[2]**2&#xA;    return dy&#xA;&#xA;def solve(C,u0,s0,s1,ds):&#xA;    s = np.arange(s0,s1+ds,ds)&#xA;    # The Christoffel symbol of 2nd kind, C, is a function of (u,v)&#xA;    from sympy.abc import u,v &#xA;    return sc.odeint(f,u0,s,args=(C,u,v))  # integration method : LSODA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have implemented several generic test cases : torus, sphere, egg carton, and catenoid. However, there seem to be some issue with the solver. On a sphere, for example, the geodesic curve is the great circle (see &lt;a href=&quot;http://msemac.redwoods.edu/~darnold/math55/deproj/Sp99/PaulC/mypaper2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference&lt;/a&gt;). When I try to find the geodesic curve and plot on a sphere (with same parmaters as &lt;a href=&quot;http://msemac.redwoods.edu/~darnold/math55/deproj/Sp99/PaulC/mypaper2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference&lt;/a&gt; provided), the curve starts to veer off. There seem to be some sort of numerical instability that is altering the course of the geodesic curve over the interval $s\in[s_0,s_1]$. Is there any way I can make my solver more stable ? I have tried to reduce the step-size of the solver, but that has not made things any better (visually at least...I could probably try to estimate the convergence rate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0n88d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0n88d.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked the code for the geodesic curve along the equator, and from pole to pole, and I get no deviation :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/qVg82.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qVg82.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &#xA;&lt;a href=&quot;https://i.stack.imgur.com/4mrkL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4mrkL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have pasted the test case on the following link : &lt;a href=&quot;http://pastebin.com/0wTAqW0U&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geodesic on a sphere&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I forgot to mention this, but I am using the following versions &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SymPy : 0.7.7.dev &lt;/li&gt;&#xA;&lt;li&gt;SciPy : 0.16.0&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It should now be possible for anyone to reproduce the same results.&lt;/p&gt;&#xA;" OwnerUserId="4249" LastEditorUserId="4249" LastEditDate="2015-10-25T12:20:59.353" LastActivityDate="2015-10-26T11:04:54.887" Title="Numerical solution of Geodesic differential equations with Python" Tags="|numerical-analysis|python|ode|scipy|numerical-modelling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21106" PostTypeId="1" CreationDate="2015-10-24T13:30:20.637" Score="1" ViewCount="603" Body="&lt;p&gt;I want to solve a integer programming problem with binary variables $x_1,\ldots,x_n.$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a permutation group $G \leq S_n$ such that for every $f \in G$ the vector $\overline{x}_1,\ldots,\overline{x}_n$ is a feasible solution if and only if $\overline{x}_{f(1)}, \ldots ,\overline{x}_{f(n)}$ is a feasible solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering whether there is any clever way to take into account the symmetries given by $G?$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we fix a specific $f \in G$ we can always add the single constraint $x_1 \leq x_{f(1)}$ but I don't see how to fully generalize this for the whole domain of $f$ and perhaps every element of $G.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am well aware of the &lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-3-540-68279-0_17&quot; rel=&quot;nofollow&quot;&gt;following&lt;/a&gt; exposition but I'm yet to see if the presented approaches work well for this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the meantime I wanted to pose this question here in case there is an obvious solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit.&lt;/strong&gt; To clarify the question. I was wondering if there is a way to encode the symmetry of $G$ as constraints of the integer program. I've tried using both CPLEX and Gurobi, setting both to exploit symmetries to the maximum but I am pretty sure they do not get the full range of symmetries and are performing quite a large chunk of redundant computation.&lt;/p&gt;&#xA;" OwnerUserId="4582" LastEditorUserId="4582" LastEditDate="2015-10-24T16:03:41.753" LastActivityDate="2019-02-25T22:03:27.610" Title="Breaking symmetries in a (binary) integer program" Tags="|linear-programming|mixed-integer-programming|symmetry|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21107" PostTypeId="2" ParentId="21106" CreationDate="2015-10-24T14:25:59.567" Score="1" Body="&lt;p&gt;At this point quite a few papers have been written about exploiting symmetry in integer programming and symmetry breaking techniques have been implemented by lots of people and are available in the widely used CPLEX and Gurobi solvers.  So yes, these techniques can be useful in practice.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question is really quite vague.  Could you be more specific about what you would like to know about this topic or what you would like to do with it?&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-10-24T14:25:59.567" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21108" PostTypeId="2" ParentId="21048" CreationDate="2015-10-24T16:30:32.610" Score="3" Body="&lt;p&gt;Here are some elements of answers to the three questions and references to alternative methods for spherical parameterization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. How to compute a voxelization of a given model ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What it means:&lt;/em&gt;&#xA;   It means embedding your surface into a 3D voxel grids, and determining which voxels have an intersection with some triangles of the surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How to do that:&lt;/em&gt;&#xA;   There are several methods to do that, the simplest one is for each triangle, iterating on the voxels inside the bounding box of the triangle, and testing for each voxel whether it has an intersection with the triangle. More sophisticated implementations can use the GPU, that is well suited to this kind of things [1]. It is also possible to use some variants of the Bresenham algorithm [2] to &quot;rasterize&quot; the triangles in 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Computing the distance function&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What it means:&lt;/em&gt; Starting from a voxel grid with voxels that are either empty or that have an intersection with the surface (obtained at step 1.), populate each empty voxel with the distance between its center and the surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How to do that:&lt;/em&gt; There is an efficient algorithm (see survey in [3]) that operates by several &quot;sweepings&quot; over the volume, it is quite easy to implement and very efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Using the gradients to guide the vertices of the mesh to a convex configuration&lt;/strong&gt;  This means solving an evolution equation that will morph the initial surface to a convex surface. The evolution equation uses at each step the gradient of the distance to the initial surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternatives for spherical parameterization&lt;/strong&gt;&#xA;There are other methods that you may use. On the theory side, the approach in [5] is elegant (but probably difficult to implement). If you want a method that is as simple as possible, cut your mesh into two halves and parameterize each half using planar Tutte/Floater parameterization [6] then glue the halves. You can then further optimize the mapping to make the seam between the halves disappear.&#xA;There is an implementation of this approach in my graphite software [7]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://developer.nvidia.com/content/basics-gpu-voxelization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://developer.nvidia.com/content/basics-gpu-voxelization&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://stackoverflow.com/questions/21663613/triangle-rasterization-bresenhams-in-3d&quot;&gt;https://stackoverflow.com/questions/21663613/triangle-rasterization-bresenhams-in-3d&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] 3D distance fields: a survey of techniques and applications, Jones et.al, IEEE Trans. on Vis. and Computer Graphics, 2006&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] math.nyu.edu/~bkleiner/mean_convex_flow.pdf &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] Fundamentals of spherical parameterization, Gotsmann et.al, ACM Siggraph 2003&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] Parameterization and smooth approximation of surface triangulations, Floater, 1996&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[7] Implementation of spherical parameterization in Graphite ver. 2. &#xA;&lt;a href=&quot;https://gforge.inria.fr/frs/download.php/file/33290/Graphite-2-a6.tar.bz2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gforge.inria.fr/frs/download.php/file/33290/Graphite-2-a6.tar.bz2&lt;/a&gt;, src/packages/OGF/parameterizer/algos/spherical_stretch_map_parameterizer.h,.cpp&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-10-24T16:30:32.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21109" PostTypeId="2" ParentId="21106" CreationDate="2015-10-24T23:50:44.290" Score="1" Body="&lt;p&gt;I think you should explore the geometry of the object described by your permutation group. Consider for example if you had three variables and if your permutations allow to permute $x_2$ and $x_3$. Then your optimization problem is posed on the hyperplane $(x_1,x_2,x_2)$.  Similarly, if you allowed the &lt;em&gt;full&lt;/em&gt; permutation group, you'd be optimizing on the line $(x_1,x_1,x_1)$. I would not be surprised if you could show that if your permutation group has $k$ irreducible generators, that the object you are optimizing on has dimension $n-k$ and that you can find representations of this object that are posed with only $n-k$ independent variables. (For example, in the two examples above, $\{x_1,x_2\}$ and $\{x_1\}$ are the independent variables.) This would allow you to simply express the action of the permutation group by reducing the number of variables you have, instead of trying to explicitly represent it in the optimization problem.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-24T23:50:44.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21112" PostTypeId="2" ParentId="20726" CreationDate="2015-10-25T14:49:00.823" Score="1" Body="&lt;p&gt;The exact solution as well as the numerical solution for fixed stepsize $h$ depends analytically on $λ$, at simple eigenvalues there is locally a near linear relationship between $λ$ and the right boundary value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus for a method of order $p$, the numerical eigenvalue for a given stepsize $h$ will lead to a right boundary value of size $O(h^p)$ in the exact solution and thus a distance of the same magnitude to the eigenvalue of the exact solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get an error of magnitude $10^{-3}$ with classical RK4 as integrator one can try $h=0.1$ to $h=0.2$ as step size to get an error of $10^{-4}$ to $10^{-3}$ in the numerical integration and hopefully with a factor close to $1$ the same error magnitude in the eigenvalue. Exact a-priory estimates are difficult, it is easier to estimate the error a-posteriori, for instance using Richardson extrapolation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that the higher oszillation for larger $λ$ leads to consider the requirements of the sampling theorem, i.e., the sampling density has to be high enough. Or in other words, for large $λ$ the problem becomes stiff, the error term could be modified to $O(e^{|λ|}h^p)$ to reflect that. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This should be no problem for the lowest eigenvalue, thus find the solution with error $10^{-4}$ in the root finder for $h=0.1$ and $h=0.2$ and use the extrapolation formula to estimate the error&#xA;$$&#xA;error = \frac{|λ_{0.2}-λ_{0.1}|}{15}&#xA;$$&#xA;for $λ_{0.1}$.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2015-10-25T14:49:00.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21113" PostTypeId="2" ParentId="21060" CreationDate="2015-10-25T15:06:46.420" Score="2" Body="&lt;p&gt;You can probably just use your original code by correcting the step computation for the positions to fall within the RK4 scheme:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while p[1, iterr] &amp;gt;= 0 and iterr &amp;lt; 9999:&#xA;        p0 = p[:, iterr]&#xA;        v0 = v[:, iterr]&#xA;        pp1 = h * v0&#xA;        vp1 = h * f(v0)&#xA;        pp2 = h * (v0 + vp1/2)&#xA;        vp2 = h * f(v0 + vp1/2)&#xA;        pp3 = h * (v0 + vp2/2)&#xA;        vp3 = h * f(v0 + vp2/2)&#xA;        pp4 = h * (v0 + vp3)&#xA;        vp4 = h * f(v0 + vp3)&#xA;&#xA;        v[:, iterr + 1] = v0 + (vp1 + 2 * (vp2 + vp3) + vp4)/6&#xA;        p[:, iterr + 1] = p0 + (pp1 + 2 * (pp2 + pp3) + pp4)/6&#xA;        iterr += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is then a true RK4 implementation in contrast to the other 2 answers where the position update is by the Euler method or something close to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should (or are already doing without documenting it here) use interpolation to get a better value for the time of the zero crossing.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2015-10-25T15:06:46.420" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21115" PostTypeId="2" ParentId="21103" CreationDate="2015-10-26T11:04:54.887" Score="7" Body="&lt;p&gt;The reason the resulting geodesic curve was deviating was because the calculated Christoffel symbol of second kind was incorrect. Using the correct Christoffel symbol :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C =  Matrix([[(0, -tan(v)), (0,0)],[(sin(v)*cos(v),0),(0, 0)]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;results in the proper output (as displayed in the &lt;a href=&quot;http://msemac.redwoods.edu/~darnold/math55/deproj/Sp99/PaulC/mypaper2.pdf&quot; rel=&quot;noreferrer&quot;&gt;reference&lt;/a&gt;) :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3CdiZ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3CdiZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I suppose I have to figure out why the calculated Christoffel symbol was incorrect. But that is another question.&lt;/p&gt;&#xA;" OwnerUserId="4249" LastActivityDate="2015-10-26T11:04:54.887" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21116" PostTypeId="1" CreationDate="2015-10-26T13:08:59.913" Score="0" ViewCount="130" Body="&lt;p&gt;I have to solve a simulation of fluid flow on Ansys, but my prof wants me to run a parallel simulation using the Volume of Fluid method in something with more visible guts than Ansys? Any recommendations, or should I try doing it in Matlab?&lt;/p&gt;&#xA;" OwnerUserId="18015" LastActivityDate="2015-12-09T19:35:19.817" Title="A tweakable software for simulating VOF problem" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21117" PostTypeId="1" AcceptedAnswerId="21122" CreationDate="2015-10-27T01:07:36.007" Score="4" ViewCount="655" Body="&lt;p&gt;I have the following optimization problem:&#xA;$$&#xA;\begin{array}{ll}&#xA;\text{Minimize} &amp;amp; \frac{1}{x_1} + \frac{1}{x_2} + \ldots + \frac{1}{d_n} \\&#xA;\text{Subject to} &amp;amp; A x \leq b &#xA;\end{array}&#xA;$$&#xA;where $x = [x_1~~x_2~~\ldots~~x_n]^T$, $x_1,x_2,\ldots,x_n \in \left\{1,\frac{1}{2},\frac{1}{3},\ldots, \frac{1}{k} \right\}$for some $k$, and $A$ and $b$ have positive entries ($A$ is big but actually very sparse).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dimension $n$ is not too large (I'd say at most 100), but the number of constraints can be huge (hundreds of thousands). They're constraints relating pairs of $x$'s. How can one attempt solving such a problem? Is there a way of transforming this into some standard form that can be solved using some optimization package like Mosek?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, I can get away with solving a continuous relaxation of this problem if the discrete version proved to be very hard. If the variables are continuous, the feasible space becomes a standard convex polytope, but the objective function remains problematic. Any ideas are appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="1932" LastActivityDate="2015-10-27T12:29:58.470" Title="Sum of Inverse of Variables in an Optimization Problem" Tags="|optimization|nonlinear-programming|mixed-integer-programming|combinatorics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21118" PostTypeId="1" CreationDate="2015-10-27T09:32:47.993" Score="1" ViewCount="45" Body="&lt;p&gt;I have the following problem. Let's say we have $x_{jk}$ it is an expression value of gene $j$ in a sample $k$. It is the average of expression levels across the cell types $s_{ij}$, weighted by respective proportions $a_{ki}$ ($i = 1 \cdots N$, $N$ is the disease type):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x_{jk} = \sum_{i=1}^{N} a_{ki}s_{ij}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally this can be expressed as matrix form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA; X = AS&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to do is to solve this equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\min_{A}\ || AS- X||^{2}\\&#xA;&amp;amp;\text{subject to } \left\{&#xA;\begin{array}{c l}      &#xA;    \sum_{i} a_{ki} = 1\\&#xA;    a_{ki} \ge0, \forall i&#xA;\end{array}\right.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically this is solved by using quadratic programming.  The  number of genes is  large (e.g. ~30K). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wandering if there is any good probabilistic model to deal such kind of problem?&lt;/p&gt;&#xA;" OwnerUserId="18026" LastEditorUserId="9667" LastEditDate="2015-10-27T22:32:46.063" LastActivityDate="2015-10-27T22:32:46.063" Title="Probabilistic model to approach problem that is usually dealt with linear programming" Tags="|linear-programming|probability|numerical-modelling|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21119" PostTypeId="1" AcceptedAnswerId="21120" CreationDate="2015-10-27T10:45:59.957" Score="2" ViewCount="537" Body="&lt;p&gt;Where can I find manufactured solutions for the 2d Euler equations, with the complete analytical terms, including the Jacobian of the source term ?&lt;/p&gt;&#xA;" OwnerUserId="18030" LastActivityDate="2015-10-27T22:41:51.103" Title="2d Euler manufactured solutions" Tags="|pde|fluid-dynamics|reference-request|nonlinear-equations|computational-physics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21120" PostTypeId="2" ParentId="21119" CreationDate="2015-10-27T11:20:18.413" Score="5" Body="&lt;p&gt;This 2004 paper by Roy et. al in &lt;em&gt;Int. J. Numer. Meth. Fluids 2004; 44:599–620&lt;/em&gt; (DOI: 10.1002/d.660) should contain exactly what you're looking for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.demec.ufpr.br/CFD/bibliografia/erros_numericos/Roy_et_al_2004.pdf&quot; rel=&quot;noreferrer&quot;&gt;ftp://ftp.demec.ufpr.br/CFD/bibliografia/erros_numericos/Roy_et_al_2004.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can contact the authors to obtain a copy of the FORTRAN code for the source terms (and their derivatives), or generate them yourself using symbolic manipulation software such as Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another paper by the same authors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dept.aoe.vt.edu/~cjroy/Conference-Papers/aiaa2002-3110.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.dept.aoe.vt.edu/~cjroy/Conference-Papers/aiaa2002-3110.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2015-10-27T11:20:18.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21121" PostTypeId="1" CreationDate="2015-10-27T11:34:24.567" Score="2" ViewCount="560" Body="&lt;p&gt;The boundary value problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{cases}&#xA;\dot{x}_i = &#xA;\begin{cases}&#xA;(0.5D^{-1}\psi)_i, \text{ if }(0.5D^{-1}\psi)_i \le 0 \\ &#xA;0 \text{, otherwise}&#xA;\end{cases}&#xA;\\&#xA;\dot{\psi} = 2\Sigma x \\&#xA;x(0) = x^0 \\&#xA;x(T) = 0&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $x^0&amp;gt;0$, $T$ is given, $D&amp;gt;0$ is a diagonal matrix, $\Sigma&amp;gt;0$ is a symmetric matrix.  Actually the constraint is $x(t)\ge 0\forall t$, equivalent to $\dot{x}(t) \le 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 2-dimension case, I try to solve it in MATLAB with this code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [tint, traj] = liq2mon_v3(x0, D, sigma, T)&#xA;&#xA;  tlow = 0;&#xA;  thigh = T;&#xA;  L = length(x0); % L=2 in this example&#xA;  nsegm = 100; % I know it is for final calc only, not for the mesh&#xA;  solinit = bvpinit(linspace(tlow, thigh, nsegm), @guess);&#xA;  sol = bvp5c(@bvp4ode, @bvp4bc, solinit);&#xA;  tint = linspace(tlow, thigh);&#xA;  z = deval(sol, tint);&#xA;  traj = z(1:L, :);&#xA;&#xA;  function xpsi = guess(t)&#xA;    xpsi = zeros(1, L*2);&#xA;    xpsi(1:L) = (x0*(1.0 - 1.0/T*t))';&#xA;  end&#xA;&#xA;  function dydt = bvp4ode(t, y)&#xA;    xcomp = y(1:L); % L=2 in this example&#xA;    psicomp = y(L+1:end);&#xA;    u    = D \ psicomp / 2.0;&#xA;    dx   = min(u, 0); &#xA;    dpsi = 2.0 * sigma * xcomp;&#xA;    dydt = [dx' dpsi']; &#xA;  end&#xA;&#xA;  function res = bvp4bc(ya, yb)&#xA;    xcompa = ya(1:2);&#xA;    xcompb = yb(1:2);&#xA;    res = [(xcompa - x0)' (xcompb)'];&#xA;  end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I get an error &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;*Unable to solve the collocation equations -- a singular Jacobian  encountered&#xA;Error in ==&amp;gt; liq2mon_v3 at 13&#xA;sol = bvp5c(@bvp4ode, @bvp4bc, solinit)*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However if I change the line &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dx = min(u, 0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u(u&amp;gt;0) = 1e-13;&#xA;dx = u;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do get a solution, with $\min(\min(x)) \approx -8\times 10^{-14}$ for the same $x^0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a way to solve this system in MATLAB (or another app) so that the constraint is not violated?&lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;Maybe this could be solved in Python, even with the current constraint violation?&lt;/li&gt;&#xA;&lt;li&gt;Why is the Jacobian singular somewhere within MATLAB solver in this case?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18027" LastEditorUserId="18027" LastEditDate="2015-10-28T16:04:28.617" LastActivityDate="2015-10-28T16:04:28.617" Title="Solving constrained BVP, singular Jacobian" Tags="|matlab|python|ode|boundary-conditions|constrained-optimization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21122" PostTypeId="2" ParentId="21117" CreationDate="2015-10-27T12:29:58.470" Score="3" Body="&lt;p&gt;For the discrete version, it can be cast as a mixed-integer linear program. You just have to note that every element $x_i$ can be written as $x_i = \sum_{j=1}^k \frac{\delta_{ij}}{j}$ where $\sum_{j=1}^k \delta_{ij} = 1$ where $\delta_{ij}$ is binary. Using the same binary variables the inverses are simply $x_{i}^{-1} = \sum_{j=1}^k j\delta_{ij}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the YALMIP code to setup and solve such a problem as a proof-of-concept (disclaimer, the MATLAB Toolbox YALMIP, which interfaces Mosek, is developed by me)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = 5;&#xA;n = 10;&#xA;delta = binvar(n,k,'full')&#xA;x =    delta*(1./(1:k))';&#xA;xinv = delta*(1:k)';&#xA;A = randn(2*n,n);&#xA;b = 10*rand(2*n,1);&#xA;Constraints = [A*x &amp;lt;= b, sum(delta,2) == 1];&#xA;Objective = sum(xinv);&#xA;optimize(Constraints,Objective)&#xA;value(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The continuous version you speak of is a convex program, as the inverse is convex on the positive orthant. An inverse scalar can be reformulated to a model involving second order cones rather easily, and thus you can solve this as a second-order cone program in Mosek. YALMIP does that automatically if you use the convexity-aware cpower operator.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = sdpvar(n,1);    &#xA;Constraints = [A*x &amp;lt;= b, 0 &amp;lt;= x &amp;lt;= 1];&#xA;optimize(Constraints,sum(cpower(x,-1)))&#xA;value(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Manually, you introduce new variables $t_i$ to replace the inverses and add the constraints $x_i^{-1} \leq t_i$, i.e., $1 \leq x_it_i$,  which can be written as the cone constraint $\Big|\Big| \begin{matrix}1\\t_i-x_i\end{matrix}\Big|\Big|\leq x_i+t_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, whether this scales for large instances, that's a completely different question.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2015-10-27T12:29:58.470" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21123" PostTypeId="1" CreationDate="2015-10-27T14:28:49.067" Score="4" ViewCount="2751" Body="&lt;p&gt;I have difficulties with this equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2 u}{d x^2} + u^2 - x^2 = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions: $u(0)=u(1)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know how to solve nonlinear differential equations with Newton's method. If somebody knows could you please explain? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed the comments and ı finally reach these two equations (eqn1 and eqn2). But my problem is that from now on ,  ı do not have any idea to combine these two equations. İf somebody has any idea, ı will be thankful so much for sharing with me. I just applied the newton raphson method&lt;a href=&quot;https://i.stack.imgur.com/hDB7e.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hDB7e.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18032" LastEditorUserId="18032" LastEditDate="2015-10-31T20:32:35.433" LastActivityDate="2022-11-19T02:59:00.273" Title="Solving nonlinear differential equations with Newton's method" Tags="|numerical-analysis|nonlinear-equations|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21124" PostTypeId="2" ParentId="21119" CreationDate="2015-10-27T22:41:51.103" Score="3" Body="&lt;p&gt;You can find information about Manufactured solutions in &lt;a href=&quot;https://github.com/manufactured-solutions&quot; rel=&quot;nofollow noreferrer&quot;&gt;this repository&lt;/a&gt;. Particularly,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs00366-012-0267-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Malaya, Nicholas, et al. &amp;quot;MASA: a library for verification using manufactured and analytical solutions.&amp;quot; Engineering with Computers 29.4 (2013): 487-496.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That have the codes in &lt;a href=&quot;https://github.com/manufactured-solutions/MASA&quot; rel=&quot;nofollow noreferrer&quot;&gt;this GitHub repository&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;MASA (Manufactured Analytical Solution Abstraction) is a library written in C++ (with C, python and Fortran90 interfaces) which provides a suite of manufactured solutions for the software verification of partial differential equation solvers in multiple dimensions. Example formulations include:&lt;/p&gt;&#xA;&lt;p&gt;Heat Equation&lt;br /&gt;&#xA;Laplace's Equation&lt;br /&gt;&#xA;Euler Equations (with and without thermal equilibrium chemistry)&lt;br /&gt;&#xA;Navier-Stokes Equations&lt;br /&gt;&#xA;Reynolds Averaged Navier Stokes with Various Turbulence Models&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-10-27T22:41:51.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21125" PostTypeId="1" CreationDate="2015-10-28T06:59:27.827" Score="2" ViewCount="316" Body="&lt;p&gt;I have coded up a toy implementation of Conjugate Residual and have been testing it.  Both wikipedia and the Saad claim that Conjugate Residual and Conjugate Gradient have similar convergence behavior.  For some examples I have been doing I haven't found this to be true.  For a $128 \times 128$ symmetric indefinite matrix, I don't get (machine) convergence at 400 iterations.  This is at odds with my understanding of Conjugate residual being a Krylov space method that should converge in at most 128 iterations.  (The condition number of this matrix is about 120 and every singular value  is repeated since the eigenvalues come in positive and negative pairs.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am guessing the statement about similar convergence properties refers to the performance on symmetric positive definite matrices, and numerical experiments are consistent with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, what determines the convergence rate of conjugate residual for a symmetric but indefinite matrix?  Also, is the conjugate residual method supposed to converge in at most $n$ iterations, where $n$ is the size of the system?&lt;/p&gt;&#xA;" OwnerUserId="16799" LastActivityDate="2015-10-28T06:59:27.827" Title="Does Conjugate Residual really have convergence properties similar to that of Conjugate Gradient?" Tags="|linear-algebra|linear-solver|iterative-method|krylov-method|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21128" PostTypeId="1" AcceptedAnswerId="21206" CreationDate="2015-10-28T13:53:21.803" Score="5" ViewCount="171" Body="&lt;p&gt;I have a tracker that outputs the trajectory $(x,y,z)$ of an object  (e.g., a &lt;strong&gt;can&lt;/strong&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use these trajectories to train a classifier (i.e., SVM) in order to infer the activity that the person manipulating the object is performing (e.g., drinking from a &lt;strong&gt;can&lt;/strong&gt; or pouring from a &lt;strong&gt;can&lt;/strong&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which kind of features should I use to quantize these trajectories?&lt;/p&gt;&#xA;" OwnerUserId="18040" LastEditorUserId="9667" LastEditDate="2015-10-28T17:01:48.397" LastActivityDate="2015-11-04T01:19:44.447" Title="How to train a model to classify object trajectories?" Tags="|machine-learning|computer-vision|support-vector-machines|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21129" PostTypeId="2" ParentId="21123" CreationDate="2015-10-28T18:57:14.450" Score="1" Body="&lt;p&gt;@GoHokies already had a good link to how to think about nonlinear problems. My own contribution is in lectures 31.5 and following here: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-10-28T18:57:14.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21130" PostTypeId="2" ParentId="21123" CreationDate="2015-10-28T20:23:53.350" Score="1" Body="&lt;p&gt;I understood that you are trying to solve a nonlinear ODE with two Dirichlet conditions. The Newton method is applied to find the root numerically in an iterative manner. In this case, I would try a numerical method to solve this ODE. &#xA;You could do this using Finite Element Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As this problem is nonlinear, you would need to apply the Newton's method. To apply the Newton Method's, you would need to do a Gateaux's differentiation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the Gateaux differentiation, you can then apply the Newton's method. The FEM method to discretize is not mandatory; there are other numerical methods. You can also try  a simple Ritz method; it is even more direct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use the Fenics software to do this (&lt;a href=&quot;http://fenicsproject.org/documentation/tutorial/nonlinear.html&quot; rel=&quot;nofollow&quot;&gt;http://fenicsproject.org/documentation/tutorial/nonlinear.html&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this can help you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best,&lt;/p&gt;&#xA;" OwnerUserId="18044" LastActivityDate="2015-10-28T20:23:53.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21131" PostTypeId="1" CreationDate="2015-10-29T09:20:04.053" Score="3" ViewCount="424" Body="&lt;p&gt;I use software for pollutant propagation on rivers that takes as input a set of parameters ($p_1,p_2,\ldots,p_n$) and creates an output file which is basically a matrix where on each row the concentration of the pollutant in various places along the river at a given timestamp is given. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array} {|r|r|r|r|r...|r|}&#xA;\hline&#xA;\text{TIME} &amp;amp;500\,\text{m}&amp;amp;1000\,\text{m}&amp;amp;1500\,\text{m}&amp;amp;2000\,\text{m}&amp;amp;2500\,\text{m}&amp;amp;...&amp;amp;25000\,\text{m} \\&#xA;\hline&#xA; 2015/12/07 - \text{4:50:00}&amp;amp;0.75&amp;amp;0.71&amp;amp;0.6&amp;amp;0.58&amp;amp;0.55&amp;amp;...&amp;amp;0.12 \\&#xA;\hline&#xA;\hline&#xA; 2015/12/07- \text{4:55:00}&amp;amp;0.71&amp;amp;0.70&amp;amp;0.58&amp;amp;0.56&amp;amp;0.51&amp;amp;...&amp;amp;0.10 \\&#xA;\hline&#xA;\end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what models/theories are suitable to be used to predict, based on a set of given INPUT parameters, the OUTPUT that would be closest to what the software will give as a result?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also how many pairs INPUT/OUTPUT will I need to have in order to minimize the error? &lt;/p&gt;&#xA;" OwnerUserId="18054" LastEditorUserId="20688" LastEditDate="2018-07-18T01:21:13.797" LastActivityDate="2020-12-04T10:06:20.693" Title="what is the best theory/model to use for prediction in multivariate data?" Tags="|data-analysis|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21132" PostTypeId="2" ParentId="2109" CreationDate="2015-10-29T11:13:42.437" Score="6" Body="&lt;p&gt;&lt;strong&gt;Answer to the question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since LBMs model the Navier-Stokes equation, you can achieve incompressible Stokes flow by simply making sure $\mathrm{Re}=UL/\nu\ll1$ and keeping $U\ll1$. Since a stability limit of LBMs already limits $\mathrm{Ma}=U/c_s\sim0.1$ this should not be a problem. LBMs are considered pseudo-compressible, but since $\Delta\rho \sim O\left(\mathrm{Ma}^2\right)$, the fluctuations will be minor and you will have quasi-incompressibility. Now the stability limit may seem restrictive but understand this is in lattice units, in dimensional units the velocities may be as any desired value as long as the Reynolds number is the same in lattice units as in dimensional units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to model different phases i suggest looking into the pseudo-potential method or free-energy based methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now as to if you should use LBMs or FEM for your problem, I think only you can really say after weighing the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2357/disadvantages-of-common-discretization-schemes-for-cfd-simulations?rq=1&quot;&gt;(dis)advantages&lt;/a&gt; of each method. Many times people pick the method they are most comfortable with and will tend to justify it with pseudo-scientific reasons about performance, ease of implementation, popularity, etc. Often times it become a slightly 'religious' debate between LBM users and other conventional CFD users. My advice is: I would suggest finding an opensource code which has examples resembling your problem which you can easily modify and go from there, learning the method it uses as you use it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;As a reply to the answer by @WolfgangBangerth:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;That said, it is also true that LBMs are generally very very slow&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So, yes: it's possible. And, no: it's not a good idea. :-)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my opinion, this answer sounds like the poster has heard of LBMs before but never used the methods. As such i don't think he is in a position to give this advice (let alone have it as the accepted answer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is well known that LBMs are some of the most efficient explicit second-order accurate methods around. The algorithm is highly localized allowing for very efficient parallelization (see &lt;a href=&quot;http://sailfish.us.edu.pl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sailfish CFD&lt;/a&gt;); LBMs are generally very very fast!. Another post discusses it &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11585/comparison-of-lattice-boltzmann-method-vs-traditional-navier-stokes-based-method?lq=1&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Granted it has disadvantages too (high memory requirement, convoluted boundary conditions, difficult to understand connection with 'real-world' equations, etc to name a few), but which method doesn't. I challenge you to give me an example in which equal situations LBM underperforms significantly compared to other methods.&lt;/p&gt;&#xA;" OwnerUserId="18017" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2015-10-29T11:19:24.587" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21133" PostTypeId="1" AcceptedAnswerId="21139" CreationDate="2015-10-29T11:51:47.357" Score="6" ViewCount="1761" Body="&lt;p&gt;I'm a computational chemist working with little computational power and dealing with increasingly demanding chemical systems. My work machines are part of a big network (mixed with windows and linux) and I was asking myself: is it possible to &quot;borrow&quot; computational power from machines around me (with permission from my department, of course)?&lt;/p&gt;&#xA;" OwnerUserId="18058" LastActivityDate="2015-11-02T22:07:27.103" Title="Borrow computational power from machines around me" Tags="|computational-chemistry|hpc|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21134" PostTypeId="1" CreationDate="2015-10-29T12:07:02.337" Score="1" ViewCount="246" Body="&lt;p&gt;How do you approximate a function for interpolated points? I use the natural cubic spline to interpolate points as follows for n = 500 points:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t=[0; 3; 6; 9]&#xA;z=[0; 6; 6; 0]&#xA;plot(t,z,'ro')&#xA;ti=linspace(0,9,500)&#xA;&#xA;zn = natcubicspline(t,z,ti)&#xA;yn = line(ti,zn)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any way approximate a function for these n-many points (for large n)? Or are there ways to treat interpolated points as if they are a function, i.e. find the gradient of the zn vector? Because zn is a vector of constants, this wouldn't necessarily be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: In particular, my data appears to form a 2nd degree polynomial, so I went ahead and used the following Matlab function to fit my data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p = polyfit(transpose(ti),zn,2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which yields coefficient estimates for a 2nd degree polynomial. It does fit the data, but with high error values, and I have to multiply this coefficient vector by a vector [1 z z^2] to get the correct polynomial. Is there any way to streamline this?&lt;/p&gt;&#xA;" OwnerUserId="18060" LastActivityDate="2015-10-30T12:26:00.320" Title="Approximate function following interpolation (Matlab)" Tags="|matlab|interpolation|polynomials|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21136" PostTypeId="1" AcceptedAnswerId="21209" CreationDate="2015-10-29T12:12:44.233" Score="2" ViewCount="515" Body="&lt;p&gt;The solution space of a nonlinear ordinary differential equation (ODE) often includes a separatrix that is unstable in the sense that nearby solutions depart exponentially from it.  The nonlinear Bessel's equation, for instance,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f''[x] + f'[x]/x +f[x](1 – f[x]^2) == 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;has this behavior, with a separatrix satisfying &lt;code&gt;f[0] = 0&lt;/code&gt;, &lt;code&gt;f[∞] = 1&lt;/code&gt;.  What algorithms work well for computing an unstable separatrix from zero to a large value of the independent variable in a nonlinear ODE?&lt;/p&gt;&#xA;" OwnerUserId="15139" LastActivityDate="2015-11-04T01:46:43.080" Title="Algorithm to Compute Separatrix of Nonlinear ODE" Tags="|algorithms|numerical-analysis|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21137" PostTypeId="2" ParentId="21133" CreationDate="2015-10-29T12:21:37.480" Score="6" Body="&lt;p&gt;You could install &lt;a href=&quot;https://boinc.berkeley.edu/&quot;&gt;BOINC&lt;/a&gt; on those machines.  When the computers become idle, the BOINC screensaver/client requests tasks from a server and computes them.  See &lt;a href=&quot;https://boinc.berkeley.edu/wiki/How_BOINC_works&quot;&gt;more information about it here&lt;/a&gt;.  This is the &lt;a href=&quot;https://boinc.berkeley.edu/projects.php&quot;&gt;software used by a number of projects&lt;/a&gt; such as SETI@Home. &lt;a href=&quot;https://boinc.berkeley.edu/trac/wiki/CreateProjectCookbook&quot;&gt;You can create your own project with BOINC&lt;/a&gt; and then put your desktop machines to work.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2015-11-02T22:07:27.103" LastActivityDate="2015-11-02T22:07:27.103" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21138" PostTypeId="1" CreationDate="2015-10-29T13:07:27.560" Score="1" ViewCount="82" Body="&lt;p&gt;I'm trying to learn how to do optimal rocket trajectory planning. I have a process that works but it converges very slowly; I'm looking for help to understand how to speed that up.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The optimal control problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is a simplified version of the &lt;em&gt;linear tangent steering&lt;/em&gt; problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine a point mass (the &quot;rocket&quot;) moving in a 2D plane. No gravity or other external force acts on it. The rocket accelerates itself (up to a maximum), but for now at least it does not change mass. It can instantly rotate to face in any direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to get from a known initial state to a partially-specified final state in minimum time without accelerating above the allowed limit. For my example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Initial position is at the origin&lt;/li&gt;&#xA;&lt;li&gt;Initial velocity is zero&lt;/li&gt;&#xA;&lt;li&gt;Final position is specified in the Y direction (height)&lt;/li&gt;&#xA;&lt;li&gt;Final position is &lt;em&gt;unspecified&lt;/em&gt; in the X direction (down-range)&lt;/li&gt;&#xA;&lt;li&gt;Final velocity is zero in the Y direction (vertical)&lt;/li&gt;&#xA;&lt;li&gt;Final velocity is specified in the X direction (horizontal)&lt;/li&gt;&#xA;&lt;li&gt;Maneuver duration is &lt;em&gt;unspecified&lt;/em&gt; (it's the variable being optimized)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Discretizing the problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm breaking the trajectory up into &lt;em&gt;N&lt;/em&gt; segments of equal duration. In each segment the rocket has a constant acceleration vector, so the optimal trajectory is approximated by a series of parabolic arcs put end-to-end, with matching positions and velocities at the boundaries between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general approach I'm taking is to start with &lt;em&gt;N&lt;/em&gt; very small; say two or three segments. Once I've converged on a solution I can double the number of segments by halving the segment duration and duplicating each segment. Then I iteratively improve that solution until it converges, and repeat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is simple enough that there are a variety of ways of representing the state and constraints. It's possible to just do the integration in closed form instead of having segment integration constraints, for instance, which gives you a very small number of constraints for the final conditions, plus all the acceleration constraints for each of the segments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could opt to connect the segments together with integration constraints instead (specifying that the velocity at the end of a parabolic arc should be consistent with the acceleration over the arc, and that the position should be consistent with the initial and final velocities on that segment). This is more typical and would enable you to go on and add gravity and so forth more readily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accelerations could be an explicit part of the state vector, or they could be derived by comparing the initial and final velocity of each segment.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Solving the problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm solving this by taking a candidate solution and alternating two steps:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Move the solution in the direction of the constrained gradient by some amount&lt;/li&gt;&#xA;&lt;li&gt;Move the solution toward the feasibility boundary if it's violating constraints&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To compute the constrained gradient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{eqnarray*}&#xA;\vec{x} &amp;amp;=&amp;amp; \text{state vector} \\&#xA;\vec{g} &amp;amp;=&amp;amp; \text{objective direction (the unconstrained objective gradient)} \\&#xA;\vec{n}_0, \vec{n}_1, \dotsc &amp;amp;=&amp;amp; \text{constraint directions} \\&#xA;c_0, c_1, \dotsc &amp;amp;=&amp;amp; \text{constraint errors} \\&#xA;\lambda_0, \lambda_1, \dotsc &amp;amp;=&amp;amp; \text{constraint multipliers} \\&#xA;\vec{p} &amp;amp;=&amp;amp; \text{constrained movement direction}&#xA;\end{eqnarray*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $\vec{g}$ vector is just $-1$ in the duration variable and $0$ everywhere else, since I'm just trying to minimize time. The constraint normals $\vec{n}_i$ and errors $c_i$ are evaluated at the current state $\vec{x}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want the movement direction $\vec{p}$ to be perpendicular to all active constraint normals, which we accomplish by subtracting out the parts of $\vec{g}$ that are not perpendicular to the constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{eqnarray*}&#xA;\vec{p} &amp;amp;=&amp;amp; \vec{g} - \lambda_0 \vec{n}_0 - \lambda_1 \vec{n}_1 - \dotsb - \lambda_m \vec{n}_m \\&#xA;\vec{p}^\intercal \vec{n}_0 &amp;amp;=&amp;amp; 0 \\&#xA;\vec{p}^\intercal \vec{n}_1 &amp;amp;=&amp;amp; 0 \\&#xA;\vdots \\&#xA;\vec{p}^\intercal \vec{n}_m &amp;amp;=&amp;amp; 0&#xA;\end{eqnarray*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting the equation for $\vec{p}$ into the $m$ equations for perpendicularity lets us solve for the constraint normal multipliers $\lambda_i$, which can then in turn be put into the first equation to yield $\vec{p}$.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Where I'm stuck&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The next question is how far to move. $\vec{p}$ can be used as-is, but the constraints for this sort of problem are all essentially spherical, so as we reach the optimum they flatten out and movement slows to a crawl. I've tried scaling $\vec{p}$ up by an arbitrary amount but that doesn't help much, and seems inelegant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of a couple of possible things to do. I know what the expected rate of objective change will be, by doing a dot product between $\vec{g}$ and $\vec{p}$. If I could get a second derivative for the expected objective change I could guess how long of a step to take to reach the minimum. I'm getting all tangled up trying to figure out how to do that, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An even closer fit might be to get an entire acceleration vector for movement in the constrained gradient direction, so that we'd move along a parabola instead of a straight line. This would keep the solution closer to the feasibility surface, so the feasibility correction step would have less to do. I'm not too worried about that, though; the feasibility restoration step works pretty well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I headed in the right direction for solving this? Is there a better way? Are there any good references for this sort of problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Here are the primary things I've been working from (along with whatever I've been able to find on the Internet):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Practical Methods for Optimal Control and Estimation Using Nonlinear Programming, 2nd ed.&lt;/em&gt;, by John Betts&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Optimal Control Theory: An Introduction&lt;/em&gt;, by Donald Kirk&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5014" LastActivityDate="2015-10-29T13:07:27.560" Title="How to speed convergence when optimizing a linear objective with nonlinear constraints?" Tags="|constrained-optimization|optimal-control|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21139" PostTypeId="2" ParentId="21133" CreationDate="2015-10-29T14:21:31.707" Score="5" Body="&lt;p&gt;You can use &lt;a href=&quot;https://research.cs.wisc.edu/htcondor/&quot;&gt;HTCondor&lt;/a&gt; that is designed exactly to &quot;steal&quot; cpu cycles from remote machines. It may be a little difficult to setup but I think this may be the best approach.&lt;/p&gt;&#xA;" OwnerUserId="16509" LastActivityDate="2015-10-29T14:21:31.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21140" PostTypeId="1" CreationDate="2015-10-29T14:30:14.130" Score="2" ViewCount="289" Body="&lt;p&gt;I am modelling a 1 dof spring-mass-damper system with friction. As first attempt I modelled the friction according to the simple Coulomb model (figure A here &lt;a href=&quot;http://article.sapub.org/image/10.5923.j.mechanics.20130301.04_006.gif&quot; rel=&quot;nofollow&quot;&gt;http://article.sapub.org/image/10.5923.j.mechanics.20130301.04_006.gif&lt;/a&gt;) and it worked fine.&#xA;Now I want to add the effect of the static friction when v = 0 m/s (figure C &lt;a href=&quot;http://article.sapub.org/image/10.5923.j.mechanics.20130301.04_006.gif&quot; rel=&quot;nofollow&quot;&gt;http://article.sapub.org/image/10.5923.j.mechanics.20130301.04_006.gif&lt;/a&gt;), but I cannot make it work. In fact I would expect to obtain something similar to the first model, but it is not so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations I modelled are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the slipping case ($\dot{x}$ different from 0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ m\ddot{x} + c\dot{x} + kx = -m\ddot{y} - \mu_{d} N $ if $\dot{x} &amp;gt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ m\ddot{x} + c\dot{x} + kx = -m\ddot{y} + \mu_{d} N $ if $\dot{x} &amp;lt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the sticking case ($\dot{x} = 0$) I have 3 equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ m\ddot{x} + c\dot{x} + kx = - m\ddot{y} + kx $ if $ |kx+m\ddot{y}| &amp;lt; \mu_{s} N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $ |kx+m\ddot{y}| &amp;gt; \mu_{s} N$ then the equations are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ m\ddot{x} + c\dot{x} + kx = -m\ddot{y} -\mu_{s} N $ if $ kx+m\ddot{y} &amp;gt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ m\ddot{x} + c\dot{x} + kx = -m\ddot{y} +\mu_{s} N $ if $ kx+m\ddot{y} &amp;lt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N$ is the normal force and $-m\ddot{y}$ represents the logarithmic sweep base excitation given as $\ddot{y} = A*\sin{(\phi(t))}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the starting equation is chosen as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Initial conditions&#xA;q0 = [0;0];&#xA;&#xA;% Choose starting equation&#xA;if q0(2) ~= 0 &#xA;    flag1 = 0;&#xA;else&#xA;    flag1 = 1;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, to switch from one equation to the other set I need events:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [value,isterminal,direction] = events(t,q)&#xA;    value = [q(2),q(2),...]; % Find the times when velocity changes sign &#xA;    isterminal = [1,1,1]; % Stop the integration when the event functions &#xA;                        % are satisfied&#xA;    direction = [1,-1,0]; &#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first 2 terms in the &lt;code&gt;value&lt;/code&gt; look for changes in the velocity direction, but now I need to tell Matlab to choose between one of the 3 equations of the case $\dot{x} = 0$. &#xA;The third condition I would add is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;abs(k*q(1)+m*A*sin(2*pi*(60*f0/(R*log(2.))*(2.^(R/60*t)-1))))-mus*N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But then if I try to add the condition for the 2 remaining equations I get an error on input matrix dimension here&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[t,q,te,qe,ie] = ode45(odefun,tspan,q0(:,1),options);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="11676" LastEditDate="2015-11-02T15:44:31.760" LastActivityDate="2015-11-02T15:44:31.760" Title="ODE events to switch between 5 equations (friction model)" Tags="|matlab|ode|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21141" PostTypeId="1" CreationDate="2015-10-29T15:38:08.303" Score="3" ViewCount="138" Body="&lt;p&gt;I'm looking for some physical phenomena for which an adaptive time stepping algorithm would be ideal. A PDE or ODE that showed very large gradients in time at a small period of time and smoother gradients for the rest of the still necessary simulation. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7023" LastEditorUserId="3691" LastEditDate="2015-11-03T08:04:29.283" LastActivityDate="2015-11-03T08:04:29.283" Title="PDEs appropriate for adaptive time stepping algorithms" Tags="|pde|ode|time-integration|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21145" PostTypeId="2" ParentId="21141" CreationDate="2015-10-29T17:41:29.130" Score="2" Body="&lt;p&gt;Predator-Prey could have the properties you want. Wikipedia has a good description, &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Lotka%E2%80%93Volterra_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.m.wikipedia.org/wiki/Lotka–Volterra_equations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8N46U.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8N46U.jpg&quot; alt=&quot;Wikipedia predator-prey simulation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you could solve very simple decay ODE with an impulse response. You would have three regimes: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Steady state&lt;/li&gt;&#xA;&lt;li&gt;Impulse, in which the system is perturbed over a small time scale&lt;/li&gt;&#xA;&lt;li&gt;Decay back to steady state&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dy}{dt} = s(t) - ky&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make $s(t)$ sharp and the solution will have a similar shape provided $k$ is large. Here $k=\tau^{-1}$ is the decay rate, the inverse of which can be interpreted as the decay lifetime, how quickly the system returns to equilibrium after the perturbation.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2015-10-30T07:26:13.983" LastActivityDate="2015-10-30T07:26:13.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21148" PostTypeId="1" CreationDate="2015-10-29T19:33:47.877" Score="4" ViewCount="176" Body="&lt;p&gt;Suppose $A\in\mathbb{R}^{n\times n}$ is symmetric and positive definite and that we have several symmetric matrices $B_i\in\mathbb{R}^{n\times n}$ that are low-rank and indefinite.  &lt;strong&gt;I need an algorithm for solving the following optimization problem:&lt;/strong&gt;&#xA;$$&#xA;\begin{align*}&#xA;\min_{x\in\mathbb{R}^n}\ &amp;amp; x^\top Ax\\&#xA;\textrm{s.t.}\ &amp;amp; \|x\|_2^2=1\\&#xA;&amp;amp; x^\top B_i x=0\ \forall i\in\{1,\ldots,k\}.&#xA;\end{align*}&#xA;$$&#xA;&lt;strong&gt;Any suggestions?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am hoping it is somehow equivalent to an eigenvalue problem.  Assorted observations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the second constraint is removed, the output is the minimal eigenvector of $A$.&lt;/li&gt;&#xA;&lt;li&gt;The Lagrange multiplier expression is $Ax=\lambda x + \sum_i \mu_iB_ix$.  So, if we somehow know the dual variables $\{\mu_i\}_{i=1}^k$, then $x$ is an eigenvector of $A-\sum_i\mu_i B_i$ with minimal eigenvalue $\lambda$.  So, we could think of $x$ as a very nonlinear function $x(\mu_1,\ldots,\mu_k)$.&lt;/li&gt;&#xA;&lt;li&gt;If one of the $B_i$'s is positive definite, then this problem is not satisfiable (implies $x=0$).  But for my specific problem I can prove the feasible region is nonempty.&lt;/li&gt;&#xA;&lt;li&gt;This can be approached using semidefinite relaxation, but this would be very slow for large $n$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2015-11-03T00:43:37.070" Title="Eigenvalue-style optimization with quadratic constraints" Tags="|optimization|eigensystem|eigenvalues|constrained-optimization|quadratic-programming|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21149" PostTypeId="2" ParentId="21131" CreationDate="2015-10-29T20:16:58.687" Score="1" Body="&lt;p&gt;The question on the minimal # of data points from the second code, the &quot;forward&quot; solver, is a very broad one. &#xA;It depends on the complexity of the underlying phenomenon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the simplest case, the flow is laminar and the pollutant particles are effectively &quot;infinitesimal&quot;; I guess you can approximate this with Fick diffusion (+ mass transport, so a convection-diffusion law) or something similar. The output sample then would then follow a Gaussian law, and there are results from math.statistics and optimal learning regarding the convergence rate of the estimators of such distributions as a function of sample size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(well, without going too deep, the Markov and Chebyshev inequalities could be used:  &lt;a href=&quot;https://en.wikipedia.org/wiki/Chebyshev%27s_inequality#Finite_samples&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Chebyshev%27s_inequality#Finite_samples&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a realistic case, the flow might be turbulent, so the statistics of pollutant transport become much more complex. I can't easily say, but a combination of statistical estimation and numerical PDEs (this sounds like an &quot;uncertainty quantification&quot; type of problem) will help you.&lt;/p&gt;&#xA;" OwnerUserId="9461" LastEditorUserId="9461" LastEditDate="2015-10-30T06:55:46.110" LastActivityDate="2015-10-30T06:55:46.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21151" PostTypeId="1" AcceptedAnswerId="21152" CreationDate="2015-10-30T02:02:13.980" Score="3" ViewCount="579" Body="&lt;p&gt;I am trying out a few forms of polynomial expression optimization, and I'd like to improve of what I've got, if anyone has anything they know is better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementation 1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x-\frac{x^3}{3!}+\frac{x^5}{5!}-\frac{x^7}{7!}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To the best of my knowledge, this has 3 add/subtracts, 21 muls, and 3 divs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementation 2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x(1+x^2(\frac{-1}{3!} + x^2(\frac{1}{5!} + x^2(\frac{-1}{7!}))))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This appears to have 3 adds/subracts, 13 muls, and 3 divs. This is assuming that $x^2$ is precalculated. (I may have counted wrong here.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementation 3:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x(1+\frac{x^2}{3!}(-1 + \frac{x^2}{5*4}(1 - \frac{x^2}{7*6}))))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This appears to have 3 adds/subtracts, 6 muls, and 3 divs. Edit: Again, assuming a precalculated $x^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: In all my factorial calculations, I have not done the $*1$ final multiply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I doing anything wrong here, or is there any way this implementation can be made more computationally efficient?&lt;/p&gt;&#xA;" OwnerUserId="17947" LastEditorUserId="17947" LastEditDate="2015-10-30T02:12:12.470" LastActivityDate="2015-10-30T05:28:35.530" Title="Efficient Implementation of Taylor Series for Sine" Tags="|numerical-analysis|computational-physics|approximation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21152" PostTypeId="2" ParentId="21151" CreationDate="2015-10-30T05:28:35.530" Score="4" Body="&lt;p&gt;Taylor series is not a good way to do this; it takes a lot of terms to get reasonable accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I answered a &lt;a href=&quot;https://stackoverflow.com/a/5778442/224264&quot;&gt;similar question on stackoverflow&lt;/a&gt; a while ago. Here's the body of the answer:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here are some good slides on how to do power series approximations&#xA;  (NOT Taylor series though) of trig functions:&#xA;  &lt;a href=&quot;http://www.research.scea.com/gdc2003/fast-math-functions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.research.scea.com/gdc2003/fast-math-functions.html&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;It's geared towards game programmers, which means accuracy gets&#xA;  sacrificed for performance, but you should be able to add another term&#xA;  or two to the approximations to get some of the accuracy back.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The nice thing about this is that you should also be able to extend it&#xA;  to SIMD easily, so that you could compute the sin or cos of 4 values&#xA;  at one (2 if you're using double precision).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you really really want to use Taylor series, check out &quot;Fast Polynomial Evaluation&quot; in part two of the linked slides; it provides some examples of &lt;a href=&quot;https://en.wikipedia.org/wiki/Estrin%27s_scheme&quot; rel=&quot;nofollow noreferrer&quot;&gt;Estrin's Method&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-10-30T05:28:35.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21154" PostTypeId="1" AcceptedAnswerId="21392" CreationDate="2015-10-30T06:44:09.513" Score="3" ViewCount="359" Body="&lt;p&gt;Central schemes like JST uses artificial dissipation for the stabilization. This modification is an artificial one. Does this additional term makes system inconsistent? Can we expect this term to be zero at the end of the simulation? This is in reference with incompressible flow schemes. In the following references they have used the pressure differences as a dissipation to continuity equations. Is it must to reach these terms to zero at the end of the simulation (steady)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle for Artificial compressibility method lower values of norms of time derivative terms indicates the satisfaction of divergence of velocity. But in the following cases where artificial dissipation is added to continuity equation which will not go zero at the end of pseudo-transience. Will there also norms of time derivative terms indicates the satisfaction of divergence of velocity? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reference 1&lt;/strong&gt;(Link: &lt;a href=&quot;http://heja.szif.hu/ANM/ANM-030110-A/anm030110a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://heja.szif.hu/ANM/ANM-030110-A/anm030110a.pdf&lt;/a&gt;)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/onlKJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/onlKJ.png&quot; alt=&quot;http://heja.szif.hu/ANM/ANM-030110-A/anm030110a.pdf&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reference 2&lt;/strong&gt;(Link: &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999199963155&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0021999199963155&lt;/a&gt;)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nFdhY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nFdhY.jpg&quot; alt=&quot;http://www.sciencedirect.com/science/article/pii/S0021999199963155&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3549" LastEditorUserId="3549" LastEditDate="2015-11-20T05:10:00.730" LastActivityDate="2015-11-29T04:01:06.250" Title="Does artifical dissipation term makes scheme inconsistent?" Tags="|stability|numerics|numerical-modelling|incompressible|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21156" PostTypeId="2" ParentId="21141" CreationDate="2015-10-30T09:24:06.180" Score="1" Body="&lt;p&gt;ODEs modelling chemical reactions (chemical &lt;em&gt;kinetics&lt;/em&gt;) are another good test case. When you have multiple reactions with different &lt;em&gt;rates&lt;/em&gt; (some reactions are blazing fast, while others take a much longer time) you end up with a &lt;em&gt;stiff&lt;/em&gt; system that is a good candidate for adaptive time stepping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example from the classic book by Hairer and Wanner: the &lt;em&gt;Brusselator&lt;/em&gt; (aka &lt;em&gt;the Brussels oscillator&lt;/em&gt;, Chapter II.4, page 170)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\frac{dy_1}{dx} &amp;amp;=&amp;amp; 1 + y_1^2 y_2 - 4 y_1 \\&#xA;\frac{dy_2}{dx} &amp;amp;=&amp;amp; 3y_1 - y_1^2 y_2&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial values $y_1(0) = 1.5$, $y_2(0)=3$, $0 \le x \le 20$.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2015-10-30T09:24:06.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21157" PostTypeId="1" AcceptedAnswerId="21164" CreationDate="2015-10-30T10:17:50.107" Score="1" ViewCount="77" Body="&lt;p&gt;I am trying to implement in Python this ratio:&#xA;$\frac{w_t(i)}{\sum w_t(j)}$ where $w_t(i) = w_{t-1}(i)\cdot\exp{(-x_{t}(i))}$, i.e. the weights are exponentially decreasing without running into underflow problems. I am using this ratio in a formula later. Has anyone had the same problem? Any ideas? I read a bit about the logsumexp trick but I am not sure I can use it here.&lt;/p&gt;&#xA;" OwnerUserId="18075" LastActivityDate="2015-10-30T17:48:06.800" Title="Calculate proportions of exponentially weighted factors avoiding underflow problem" Tags="|numerical-analysis|python|efficiency|numerical-limitations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21158" PostTypeId="1" AcceptedAnswerId="21161" CreationDate="2015-10-30T10:31:20.480" Score="1" ViewCount="2469" Body="&lt;p&gt;I am solving a nonlinear ODE with a regular singularity using MATLAB ODE45 or ODE113.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering what precision and accuracy they have and what one can say about the numerical error. The idea would be to write a statement like &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The solution was found by means of ODE45 which has an accuracy&#xA;  of...&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="18076" LastEditorUserId="123" LastEditDate="2015-11-01T04:33:30.633" LastActivityDate="2015-11-03T10:33:37.540" Title="How can one describe the accuracy of a Runge-Kutta method?" Tags="|matlab|numerical-analysis|ode|error-estimation|runge-kutta|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21159" PostTypeId="1" CreationDate="2015-10-30T11:57:48.047" Score="3" ViewCount="104" Body="&lt;p&gt;We have been developing algorithms for detecting &quot;robust&quot; zeros of multidimensional functions $f: X\to\Bbb R^n$ where $X$ is an $m$-dimensional domain in $\Bbb R^m$. More precisely, for a given $f$, we can compute an $r&amp;gt;0$ such that every continuous $g$, $\|g-f\|&amp;lt;r$ has a nonempty zero set in $X$ (under some quite weak conditions). This can be used for proving that the initial map $f$ has a zero even in cases when it is given only approximately, such as a list of function values in a multidimensional grid and a Lipschitz constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More involved robust properties of the zero set of $f$ could be analysed. For instance we can  find &quot;robust optima&quot; over the zero set with the uncertainty when an additional objective function is given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can these algorithms have some natural applications in engineering and/or scientific computations? In some sense, it generalizes solving systems of equations to the case where the input is uncertain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematics behind the algorithm is nice and nontrivial (and has been the main motivating force for us so far), but the further development makes the best sense if we had specific problems to solve. &lt;/p&gt;&#xA;" OwnerUserId="7212" LastActivityDate="2015-11-02T09:38:24.480" Title="Applications of algorithm for solving systems of equations with uncertainty" Tags="|nonlinear-equations|constrained-optimization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21160" PostTypeId="2" ParentId="21134" CreationDate="2015-10-30T12:26:00.320" Score="2" Body="&lt;p&gt;I think you want to use &lt;code&gt;polyfit&lt;/code&gt; on the original data, not the already interpolated data.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p = polyfit(t,z,2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now &lt;code&gt;p&lt;/code&gt; is your coefficients of the quadratic which interpolates &lt;code&gt;z&lt;/code&gt; at &lt;code&gt;t&lt;/code&gt;. You can evaluate the polynomial at your points with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zi = polyval(p,ti);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since it's a polynomial, you can differentiate it (you could automate this and make it cleaner)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p_prime = [2*p(1), p(2)];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and evaluate this;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zi_prime = polyval(p_prime, ti);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For anything more complicated you could look at &lt;a href=&quot;http://uk.mathworks.com/help/matlab/ref/interp1.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;interp1&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;http://uk.mathworks.com/help/matlab/ref/griddedinterpolant-class.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;griddedInterpolant&lt;/code&gt;&lt;/a&gt;. (&lt;code&gt;griddedInterpolant&lt;/code&gt; returns a function).&lt;/p&gt;&#xA;" OwnerUserId="17671" LastActivityDate="2015-10-30T12:26:00.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21161" PostTypeId="2" ParentId="21158" CreationDate="2015-10-30T12:52:16.223" Score="2" Body="&lt;p&gt;The RK4* method is a fourth-order method, meaning that the local truncation error is on the order of $O(h^5)$, while the total accumulated error is order $O(h^4)$.&lt;/p&gt;&#xA;" OwnerUserId="17966" LastActivityDate="2015-10-30T12:52:16.223" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21162" PostTypeId="1" CreationDate="2015-10-30T13:27:34.140" Score="3" ViewCount="188" Body="&lt;p&gt;Is there any function to create or change a matrix, to have $A_{ij} = \text{abs}(A_{ij})$ in PETSc? If possible it should work with MPIAIJ matrices, not only local.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastActivityDate="2018-03-14T04:03:43.677" Title="Get a matrix with absolute values in PETSc" Tags="|petsc|matrix|" AnswerCount="1" CommentCount="0" ClosedDate="2018-03-15T16:50:57.143" ContentLicense="CC BY-SA 3.0" />
  <row Id="21163" PostTypeId="1" AcceptedAnswerId="21197" CreationDate="2015-10-30T14:07:07.417" Score="6" ViewCount="1414" Body="&lt;p&gt;I wrote a finite element code in fortran 90.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code is really fast, except the meshing process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used &lt;a href=&quot;https://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;noreferrer&quot;&gt;triangle&lt;/a&gt; and &lt;a href=&quot;http://wias-berlin.de/software/tetgen/&quot; rel=&quot;noreferrer&quot;&gt;tetgen&lt;/a&gt; for meshing in 2D and 3D, respectively, so this process is fast, of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for the unit square [0,1]x[0,1] in 2D I have a file with the coordinates of its nodes (for example, a mesh with 5 nodes):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   0.0 0.0  # coordinates of node 1&#xA;2   1.0 0.0  # coordinates of node 2&#xA;3   1.0 1.0  # coordinates of node 3&#xA;4   0.0 1.0  # coordinates of node 4&#xA;5   0.5 0.5  # coordinates of node 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;called &lt;code&gt;coordinate.dat&lt;/code&gt;, which have 4 elements (triangles) with nodes called &lt;code&gt;element.dat&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   1 5 4  # vertices of triangle 1&#xA;2   1 2 5  # vertices of triangle 2&#xA;3   2 3 5  # vertices of triangle 3&#xA;4   5 2 4  # vertices of triangle 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also have a file where each row &lt;code&gt;i&lt;/code&gt; is the number of its initial an final node, called &lt;code&gt;edge.dat&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   1 2  # initial and final node of edge 1&#xA;2   2 3  # initial and final node of edge 2&#xA;3   3 4  # initial and final node of edge 3&#xA;4   4 1  # initial and final node of edge 4&#xA;5   1 5  # initial and final node of edge 5&#xA;6   5 2  # initial and final node of edge 6&#xA;7   2 5  # initial and final node of edge 7&#xA;8   5 4  # initial and final node of edge 8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this files, I need to generate the following information:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Given an element (triangle or tetrahedron), I need to know the number of its sides (edges and faces, respectively). For example, I need to generate the following structure or file, called &lt;code&gt;struct1.dat&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   5 8 4  # triangle 1 has the edges number 5, 8 and 4&#xA;2   1 6 5  # triangle 2 has the edges number 1, 6 and 5&#xA;3   6 2 7  # triangle 2 has the edges number 6, 2 and 7&#xA;4   7 3 8  # triangle 4 has the edges number 7, 3 and 8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(2) Furthermore, given a side (edge or face) I need to know the element numbers of the 2 elements (or only one if the side is on the boundary) shared by that side. For example, I need to generate the following structure (or file) called &lt;code&gt;struct2.dat&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   2 0  # edge number 1 is only on element 2&#xA;2   3 0  # edge number 2 is only on element 3&#xA;3   4 0  # edge number 3 is only on element 4&#xA;4   1 0  # edge number 4 is only on element 1&#xA;5   1 2  # edge number 5 is sharing by elements 1 and 2&#xA;6   3 2  # edge number 6 is sharing by elements 3 and 2&#xA;7   4 3  # edge number 7 is sharing by elements 4 and 3&#xA;8   1 4  # edge number 8 is sharing by elements 1 and 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For both of these structures, &lt;code&gt;struct1.dat&lt;/code&gt; and &lt;code&gt;struct2.dat&lt;/code&gt;, my code is very slow because I used a brute force approach with a lot of loops..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for an algorithm (a paper, or better: a subroutine in fortran available for download) optimized for this? I want to continue using triangle and tetgen, but I am willing to listen to other options.&lt;/p&gt;&#xA;" OwnerUserId="17630" LastEditorUserId="1244" LastEditDate="2015-10-30T14:52:22.247" LastActivityDate="2015-11-11T10:14:06.283" Title="Meshing options to generate number of the sides of and element (tetgen-triangle)" Tags="|finite-element|fortran|mesh-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21164" PostTypeId="2" ParentId="21157" CreationDate="2015-10-30T17:48:06.800" Score="0" Body="&lt;p&gt;The solution may somewhat depend on your application. In general you can keep track of log-weights to minimize underflow issues (i.e. &lt;a href=&quot;https://en.wikipedia.org/wiki/Likelihood_function#Log-likelihood&quot; rel=&quot;nofollow&quot;&gt;log-likelihoods&lt;/a&gt;; then the &quot;logsumexp trick&quot; could apply to the log of your normalization factor). However  some weights will still effectively go to 0 eventually. In &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle_filter#Sequential_importance_resampling_.28SIR.29&quot; rel=&quot;nofollow&quot;&gt;particle filtering&lt;/a&gt; this is known as the &quot;sample degeneracy problem&quot;, and is typically solved with &lt;em&gt;resampling&lt;/em&gt; (e.g. as in the bootstrap filter described in the above link).&lt;/p&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2015-10-30T17:48:06.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21166" PostTypeId="1" CreationDate="2015-10-30T19:47:03.567" Score="1" ViewCount="568" Body="&lt;p&gt;I am new to solving numerically ODES and thus it is difficult for me to judge the reliability/trustworthiness of the results that I have produced for the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am dealing with a 2nd order ODE (I can add a non-linear term to it, too) on the interval $[0,1]$ with regular singularity at $0$ and $1$. &#xA;I converted the ODE into a 1st order one and interpreted it as a IVP by setting an initial condition at $x=1$ motivated by symmetry arguments and let ODE45 integrate from $x=1$ to $x=0$. &#xA;In this way I can produce solutions which is nice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But here comes the problem: I have to use these solutions later on, e.g., I can also compute the $L^2$-norm of the solutions on the interval $[0,1]$ with respect to some measure and what I get as a result seems to depend sensitively on &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1.) the cut-off at $x=0$ (I let ODE45 run until &quot;eps&quot;, the machine precision but I could also take smaller and bigger values)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2.) the smallness of AbsTol and RelTol (both around eps)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3.) the tspan size, i.e., the number of slices into which I cut the interval, like $10^6$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on how I vary the values in (1.)-(3.) I get results which differ from one another and not to a small extent.&#xA;I believe that the code is correctly implemented, rather I guess the problem comes from the regular singularity which leads me to the question how trustworthy ODE45 is for such a problem.&#xA;Does anybody have experiences with such a problem and can recommend what I could do to get more reliable results?&#xA;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation reads $2x~(1-x)~f''(x)+(3-4x)~f'(x)+a~f(x)+b~f^n(x)=0;~~ a,b\in\mathbb{R}, n\in\mathbb{N}.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;not to a small extent&quot; means a change in the result for the normalization from e.g $10^{-2}$ to $10^{20}$ if I solve until $x=eps*100$ vs. $x=eps$.&lt;/p&gt;&#xA;" OwnerUserId="18076" LastEditorUserId="18076" LastEditDate="2015-11-01T12:50:00.340" LastActivityDate="2015-11-01T12:50:00.340" Title="Computing solutions with singularities using MATLAB ODE45" Tags="|matlab|numerical-analysis|ode|stability|runge-kutta|" AnswerCount="0" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21170" PostTypeId="1" AcceptedAnswerId="23846" CreationDate="2015-10-31T09:28:42.233" Score="0" ViewCount="626" Body="&lt;p&gt;I'm writing a VMC simulation for hydrogen and helium atoms, but in both my codes my variational energy for certain wavefunctions is not only statistically different from my expectation value, but it is also lower than my ground state energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the hydrogen atom, I know that the ground state energy is $-0.5Ha$. If I use the trial wavefunction $e^{-1.2r}$ (where $r$ is the distance between my nucleus and electron) I'm getting an energy LOWER than my ground state energy (I've been told that my ground state energy is a lower bound for my variational energy). If I use the exact wavefunction ($e^{-r}$) I get $-0.5Ha$ exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm quite confused by this - I'm guessing since my variational energy is lower than my ground state energy for some trial wavefunctions I must be doing something very wrong - but I do get the exact energy if I use the exact wavefuntion. Why might that be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference here is my code.&#xA;    import math&#xA;    import random&#xA;    import numpy &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;b = numpy.float_(1.2)&#xA;&#xA;def psiT(r): #my trial wavefunction&#xA;    return (math.exp(-b*r))&#xA;&#xA;def Elocal(r): &#xA;    return -0.5*((b*b)-((2*b)/r))-(1/r)&#xA;&#xA;def r(x,y,z): &#xA;    return math.sqrt(math.pow(x,2)+math.pow(y,2)+math.pow(z,2))&#xA;&#xA;ne = 3&#xA;a = 0&#xA;e = numpy.array([random.random() for i in range (ne)])&#xA;&#xA;dr = 1.05&#xA;#equilibriate&#xA;for n in range (10000):&#xA;#move my electrons&#xA;    dt = numpy.array([dr*random.uniform(-1,1) for i in range (ne)])&#xA;&#xA;    ep = e+dt&#xA;#was this move good?  &#xA;    R1 = r(e[0],e[1],e[2])&#xA;    Rp1 = r(ep[0],ep[1],ep[2])&#xA;&#xA;    psip = psiT(Rp1)&#xA;    psi = psiT(R1)&#xA;    W = math.pow(psip/psi,2)  &#xA;&#xA;    if W &amp;gt;= random.random():&#xA;        e = ep&#xA;        a = a+1&#xA;#accumulate&#xA;El = 0&#xA;q = 0&#xA;srt = 0&#xA;for n in range (10000):&#xA;#move my electrons&#xA;    dt = numpy.array([dr*random.uniform(-1,1) for i in range (ne)])&#xA;&#xA;    ep = e+dt&#xA;    #was this move good?  &#xA;    R1 = r(e[0],e[1],e[2])&#xA;    Rp1 = r(ep[0],ep[1],ep[2])&#xA;&#xA;    psip = psiT(Rp1)&#xA;    psi = psiT(R1)&#xA;    W = math.pow(psip/psi,2)    &#xA;&#xA;    if W &amp;gt;= random.random():&#xA;        El = El+Elocal(Rp1)&#xA;        q = q+1&#xA;#ignore srt and s, I use them for error analysis which I have not     included here&#xA;        srt = srt+math.pow(Elocal(Rp1),2)&#xA;        s = math.pow((Elocal(Rp1))-(El/q),2)&#xA;        e = ep&#xA;&#xA;print El/q&#xA;print q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17498" LastEditorDisplayName="user17774" LastEditDate="2015-10-31T18:12:44.037" LastActivityDate="2016-04-26T17:56:24.093" Title="Variational Monte Carlo: Variational energy is lower than ground state energy" Tags="|python|iterative-method|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21172" PostTypeId="1" CreationDate="2015-10-31T12:53:18.680" Score="2" ViewCount="124" Body="&lt;p&gt;I am solving following problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Filtration of water can be described in bi-dimensional case by &#xA;$$- \partial_x(K(x,y) \partial_x u ) - \partial_y (K(x,y) \partial_y u ) = 0, $$&#xA;where $u$ - water level &#xA;$K$ - hydraulic conductivity&#xA;On uniform mesh of $N$, $K$ is defined as&#xA;\begin{align}&#xA;K(x,y) &amp;amp; =  \begin{cases}&#xA;    (x -0.5)^2 + (y - 0.5)^2 - 0.04 &amp;amp;  \geq 0\\&#xA;     &amp;amp;  &amp;lt;0&#xA;  \end{cases}\\ &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;- \partial_x K \partial_x u  \ \ \text{is defined as} \ \  \frac{K(i+ 1/2) \partial_x u_{i+1/2} - K(i - 1/2) \partial_x u_{i - 1/2} }{h}, &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $(i + 1/2)$ refers to mid-points of $(i + 1)th$ interval. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have extended discretization into 2D case and obtained following system (not sure, if its correct)&#xA;\begin{align}&#xA;\begin{bmatrix}&#xA;   c_{i}       &amp;amp; c_{i+1} &amp;amp; 0 &amp;amp; d_{i+1} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    c_{i-1}       &amp;amp; c_{i} &amp;amp; c_{i+1} &amp;amp; 0 &amp;amp; d_{i+1} &amp;amp; 0 &amp;amp; 0&amp;amp; 0 \\&#xA;0       &amp;amp; c_{i-1} &amp;amp; c_{i}  &amp;amp; c_{i+1} &amp;amp; 0 &amp;amp; d_{i+1}&amp;amp;  0 &amp;amp; 0\\&#xA;    d_{i-1} &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots \\&#xA;    0  &amp;amp; 0  &amp;amp; 0 &amp;amp; \dots &amp;amp;\dots &amp;amp; &amp;amp;  c_{i-1} &amp;amp; c_{i} &#xA;\end{bmatrix}&#xA;*&#xA;\begin{bmatrix}&#xA;    u_{1,1}  \\&#xA;    u_{2,1}  \\&#xA;    u_{N-1, 1}  \\&#xA;    \vdots  \\&#xA;    u_{N-1,N-1} \\&#xA;\end{bmatrix} = 0&#xA;\end{align}&#xA;where coefficients are calculated as&#xA;\begin{align}&#xA;c_i  &amp;amp;= + K(i -1/2,j) + K(i +1/2,j) + K(i ,j+1/2) + K(i,j-1/2)\\&#xA;c_{i + 1} &amp;amp;= - K(i+1/2,j) \\&#xA;c_{i- 1} &amp;amp;= - K(i - 1/2,j) \\&#xA;d_{i - 1} &amp;amp;= -  K(i,j - 1/2) \\&#xA;d_{i + 1} &amp;amp;= - K(i,j +1/2) \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;System has to satisfy following BC:&#xA;\begin{align}&#xA;\partial_x u &amp;amp;= 0 \ \  &amp;amp;\text{on} \{0,1 \} \times (0,1)\\&#xA;u &amp;amp;= 10 \ \  &amp;amp;\text{on} \{0 \} \times (0,1) \\&#xA;u &amp;amp;= 0 \ \  &amp;amp;\text{on} \{1 \} \times (0,1)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody give me some advices how to handle them ? &lt;/p&gt;&#xA;" OwnerUserId="17045" LastEditorUserId="17045" LastEditDate="2015-10-31T14:38:52.203" LastActivityDate="2015-10-31T14:38:52.203" Title="Finite difference aproximation - Darcy law" Tags="|finite-difference|fluid-dynamics|boundary-conditions|discretization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21174" PostTypeId="2" ParentId="6968" CreationDate="2015-11-01T02:08:09.537" Score="6" Body="&lt;p&gt;I know its been 2 years.. but here is an example from physics to hammer it home.&#xA;Consider you are solving the Poisson equation: $$\nabla^2\phi =f$$for electrostatic potentials(ϕ) with the right hand side: $$ f=(-ρ/εo) $$ And, ρ(the charge density) is specified at all the grid points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Statement: For Periodic solution, the ∫ρdv=0 condition (described by Ben above) sets the system to be net neutral. This is a nice (physical) way to think about average value of f over the periodic box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's test this statement. For periodic boundary conditions, the integral of the Electric field has to be zero over the surface of the box (you can always find pairs of points on the box surface canceling each other out in the integral). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then by divergence theorem (gauss law), the box should be neutral and this is what we set out to prove.&#xA;$$\oint_S {E_n dA = \frac{1}{{\varepsilon _0 }}} Q_{inside} =0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A little bit of physics here, but I think it provides a nice reinforcement to the discussion here. &lt;/p&gt;&#xA;" OwnerUserId="18090" LastEditorUserId="18090" LastEditDate="2015-11-17T21:37:09.990" LastActivityDate="2015-11-17T21:37:09.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21175" PostTypeId="2" ParentId="21123" CreationDate="2015-11-01T07:20:24.557" Score="1" Body="&lt;p&gt;This is a boundary value problem, not a root finding problem.&#xA;One way to solve it is to replace the derivative with a finite difference, as you did, and iteratively relax to the correct solution.&lt;/p&gt;&#xA;" OwnerUserId="18092" LastActivityDate="2015-11-01T07:20:24.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21176" PostTypeId="1" CreationDate="2015-11-01T08:03:11.140" Score="3" ViewCount="595" Body="&lt;p&gt;In &lt;a href=&quot;http://www.flow3d.com/home/resources/cfd-101/physical-phenomena/boundary-conditions-pressure&quot; rel=&quot;nofollow&quot;&gt;Flow Science&lt;/a&gt;, they provide an example of boundary conditions and how to specify them for CFD simulations. The following is stated:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Pressure Boundary Example&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For example, consider the problem of flow in a section of pipe. On the&#xA;  one hand, if the upstream end of the computational region coincides&#xA;  with the physical entrance to the pipe then a stagnation condition&#xA;  should be used to represent the external ambient conditions as a large&#xA;  reservoir of stationary fluid. On the other hand, if the upstream&#xA;  boundary of the computing region is inside the pipe, and many&#xA;  diameters away from the entrance, then the static pressure condition&#xA;  would be a more reasonable approximation to flow conditions at that&#xA;  location.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which I don't really understand, because inside the pipe, the fluid is already in motion, how come we shall use the static pressure?&lt;/p&gt;&#xA;" OwnerUserId="6673" LastActivityDate="2016-12-28T21:22:39.360" Title="Stagnation or Total conditions in a pipe with moving fluid?" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21177" PostTypeId="1" AcceptedAnswerId="21232" CreationDate="2015-11-01T10:53:21.217" Score="3" ViewCount="681" Body="&lt;p&gt;Let there is triangular  mesh for a simple domain (domain is not important).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this mesh is not uniform (the mesh is produced by adaptive technique), what is the mesh size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a mesh size for each triangle? or this is a number relate to the whole mesh on the domain? How do I compute it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you,&lt;/p&gt;&#xA;" OwnerUserId="10403" LastActivityDate="2015-11-05T17:39:44.403" Title="how can I find the mesh size" Tags="|finite-element|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21178" PostTypeId="2" ParentId="1088" CreationDate="2015-11-01T15:32:00.423" Score="1" Body="&lt;p&gt;You can do it with &lt;a href=&quot;http://www.axiom-developer.org/&quot; rel=&quot;nofollow&quot;&gt;Axiom&lt;/a&gt; (or &lt;a href=&quot;http://open-axiom.org/&quot; rel=&quot;nofollow&quot;&gt;OpenAxiom&lt;/a&gt; or &lt;a href=&quot;http://fricas.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;Fricas&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(1) -&amp;gt; eval( x^2 + 2*x + 3, x=sqrt(2)*t-1)&#xA;(1) -&amp;gt; &#xA;          2&#xA;   (1)  2t  + 2&#xA;                                            Type: Polynomial(AlgebraicNumber)&#xA;(2) -&amp;gt;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18093" LastActivityDate="2015-11-01T15:32:00.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21181" PostTypeId="1" AcceptedAnswerId="21189" CreationDate="2015-11-01T18:21:32.450" Score="3" ViewCount="519" Body="&lt;p&gt;I am implementing a MatLab program to solve the equation given in &lt;a href=&quot;http://ftp.maia.ub.es/dsg/2004/jorba0402.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; paper, which involves solving integrals coming from the variational formulation of the problem. One of them is&#xA;$$\int_{\partial \Omega} f\left(\sum_{i=1}^Nu_i\phi_i\right) \cdot \phi_j d\Omega$$&#xA;where $\phi_j$ is a function such that, in the node $(x_j, y_j)$ of the triangular partition of $\Omega$, its value is 1 and 0 elsewhere. That is, $\{\phi_i\}$ is the standard nodal basis defined for a triangular partition on FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My confusion is how to evaluate this integral. Here is what I think I need to do:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each iteration, we want to evaluate $u^{(k)} = (u_1^{(k)}, ..., u_N^{(k)})^T$ an approximation of $u$ at each node from our triangular partition with $N$ nodes. This integral is only evaluated at the nodes which lie on the boundary $\partial \Omega$. Suppose, for example, that $(x_1, y_1)$ is one of those nodes and we want the value of $u$ at this node (ie, we want u_1 at some iteration). Then, we should have to evaluate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\partial \Omega} f\left(\sum_{i=1}^Nu_i\phi_i(x_1, y_1)\right) \cdot \phi_1(x_1, y_1) d\Omega = \int_{\partial \Omega} f(u_1) d\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am really confused if this is correct, and if it is, how do I calculate this integral?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sorry if the question is confusing, but the paper can give a clear idea of what should be done, if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="18097" LastEditorUserId="7567" LastEditDate="2015-11-02T07:50:57.837" LastActivityDate="2015-11-02T22:07:57.430" Title="Evaluate integral on Boundary - FEM" Tags="|finite-element|numerical-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21182" PostTypeId="2" ParentId="21181" CreationDate="2015-11-01T21:24:11.537" Score="2" Body="&lt;p&gt;You seem to be confused with the definition of the basis $\{\phi_i\}.$ You are right that $\phi_i(x_i,y_i)=1$ and $\phi_i(x_j,y_j)=0$ for all $i\neq j.$ What you probably miss is that $\phi_i$ is linear, i.e. it is non-zero on all triangular elements which touch $(x_i,y_i).$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper describes a standard Galerkin discretisation method. The integral in the left-hand side of your expression can be evaluated analytically, since the function $f(u)=-u(u-a)(u-b)$ is a polynomial.&lt;/p&gt;&#xA;" OwnerUserId="17966" LastActivityDate="2015-11-01T21:24:11.537" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21184" PostTypeId="1" CreationDate="2015-11-02T14:32:51.010" Score="0" ViewCount="281" Body="&lt;p&gt;Is it possible to solve a quadratic cost function which is subject to both nonlinear (quadratic) &amp;amp; linear constriants by SQP methods? If not, what is the best iterative solution for this kind of problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="18101" LastEditorUserId="7567" LastEditDate="2015-11-02T18:03:38.963" LastActivityDate="2015-11-03T21:58:03.797" Title="SQP for Quadratic Cost function optimization subject to linear &amp; nonlinear Constraints" Tags="|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21186" PostTypeId="1" CreationDate="2015-11-02T18:48:48.963" Score="0" ViewCount="1377" Body="&lt;p&gt;For a multi-objective optimization task I want to use the DTLZ5, DTLZ6 and DTLZ7-problems definied by Deb et al. in their Paper &quot;&lt;a href=&quot;http://repository.ias.ac.in/81671/1/104-a.pdf&quot; rel=&quot;nofollow&quot;&gt;Scalable Multi-Objective Optimization Test Problems&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are multiple libraries which implemented these problems like Shark (C++), jMetal (Java), deap (Python) and pagmo (C++ and Python).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They all share their code on GitHub, but unfortunately I may only post two links here, for which I chose jMetal to illustrate the problems that occur in all of these implementations.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;DTLZ5&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In the paper, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;g(\text{x}_M)=\sum_{x_i \in \text{x}_M} x_i^{0.1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;while in &lt;a href=&quot;https://github.com/jMetal/jMetal/blob/master/jmetal-problem/src/main/java/org/uma/jmetal/problem/multiobjective/dtlz/DTLZ5.java#L81&quot; rel=&quot;nofollow&quot;&gt;jMetal&lt;/a&gt; it's&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;g(\text{x}_M)=\sum_{x_i \in \text{x}_M} (x_i - 0.5)^{2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no equivalent to this in the paper, although for DTLZ2 the $g$-function looks this way.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;DTLZ6&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In short: DTLZ6 in jMetal is DTLZ5 in the paper.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;DTLZ7&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I could not find any implementation of the DTLZ7 problem like it is stated in the paper. In contrast, I found at least one paper and other scientific work that referred to the $h$-value of DTLZ7 like &quot;Dynamic Multiobjective Optimization Problems: Test Cases, Approximations, and Applications&quot; by Farina et al. where Deb is even a co-author. Additionally, there is a definition of the DTLZ7-problem on the website of the ETH Zürich where the three co-authors worked. These two sources as well as the libraries listed above describe/implement DTLZ6 instead of DTLZ7.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;TL;DR&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;problem implemented as DTLZ5 is not part of the paper&lt;/li&gt;&#xA;&lt;li&gt;implemented DTLZ6 equals DTLZ5 from the paper&lt;/li&gt;&#xA;&lt;li&gt;same for DTLZ7 and DTLZ6&lt;/li&gt;&#xA;&lt;li&gt;there is no implementation of DTLZ7 out there&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;The Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Is there any paper I could not find that makes my observations obsolete or why do all these sources differ from the paper?&lt;/p&gt;&#xA;" OwnerUserId="18104" LastActivityDate="2015-11-03T09:04:36.853" Title="Definition of the DTLZ 5 - 7 Problems" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21187" PostTypeId="1" CreationDate="2015-11-02T19:03:43.823" Score="4" ViewCount="428" Body="&lt;p&gt;$X$ is a dense matrix of real doubles, typically of size 20 million rows and 500 columns, and $W$ is a diagonal matrix of real,  non-negative doubles stored as a vector. I'm working in C and have looked at the documentation for BLAS but couldn't find a routine that seemed like a perfect fit for this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The machine has over 20 CPU cores and I would like to make use of them if possible. There is plenty of capacity for $X$ and $W$ to be fully in memory but I would like to avoid making copies of data unless really necessary as it is a shared computing resource.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I compute $X'WX$ hundreds of times. At each iteration $X$ is constant but $W$ changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be happy to write my own code for this in C with OpenMP but ideally would use an existing optimized routine if that would be fastest.&lt;/p&gt;&#xA;" OwnerUserId="18106" LastEditorUserId="18106" LastEditDate="2015-11-03T12:37:55.493" LastActivityDate="2015-11-03T17:01:57.210" Title="Optimized parallel routine for $X' W X$ with $W$ diagonal" Tags="|linear-algebra|matrix|performance|lapack|blas|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21189" PostTypeId="2" ParentId="21181" CreationDate="2015-11-02T22:07:57.430" Score="3" Body="&lt;p&gt;So taking your first expression, and giving it a name to make an equation out of it, and substituting a quadrature rule says:&#xA;$$&#xA;\big[F(u)\big]_{j} \approx \sum_q w_q f\Big(\sum_i u_i \phi_i(x_q)\Big)\phi_j(x_q)&#xA;$$&#xA;where $w_q$ and $x_q$ are the weights and points of your quadrature rule. If this is the RHS of your overall system, then $\phi_j$ should probably not appear since this would make a matrix for all $i$ and $j$ in your domain. If this is in the typical left-hand side, then you have additional contribution to the $i,j$ entry of your stiffness matrix. If $x_q$ is not in any of the elements surrounding $x_i$, then you can skip that term in the inner sum since $\phi_i(x_q)=0$ in that case and just add $f(0)$ to your partial sum for that $i$. It might be good if you edited to include the whole equation you're trying to solve, e.g.  $Lu=F(u)$, for a linear operator $L$, or $F(u)={\rm something else}$ to help clarify. Either way, you're probably looking to form a matrix equation to solve for each $u_i$. If $u_i$ is unknown and $f$ is non-linear, then forming a matrix out of this system to solve for $u_i$ is probably impossible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to this inner sum inside $f$ which is presumably non-linear, this is going to be an expensive term to compute. You might be able to quickly build the full $N_qN_i$ table if both $N$s are of reasonable size and $f$ is cheap. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-11-02T22:07:57.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21190" PostTypeId="1" CreationDate="2015-11-03T00:10:47.667" Score="0" ViewCount="94" Body="&lt;p&gt;I am making a Java function to calculate $\int_{0}^{\pi/2}x/sin(x) dx$ using Simpsons rule. When the function equals $0$ or $0/\sin(0)$ should I just add $1$ (limit) or $10^{-10}$ (limit). My professor said to use $10^{-10}$ why would this be?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;public static double simpsonsRuleFunction1(double valueN, double valueA, double valueB, double valueDx) {&#xA;&#xA;    double e = 0.0;&#xA;    double simpsonsRule = 0.0;&#xA;    double valueHolder = 0.0;&#xA;&#xA;    valueN = 2;&#xA;    valueA = 0;&#xA;    valueB = (Math.PI)/2; &#xA;&#xA;    for(int i = 1; i&amp;lt;=valueN+1 ; i++){&#xA;        valueDx = (valueB-valueA)/valueN;&#xA;        e = valueA + ((i-1)*valueDx);&#xA;&#xA;        if (i==1) {&#xA;        // Limit as x -&amp;gt; 0&#xA;        simpsonsRule += Math.pow(10,-10);&#xA;&#xA;        }&#xA;        else if ((i % 2 == 0) &amp;amp;&amp;amp; ( i &amp;gt; 1) &amp;amp;&amp;amp; (i &amp;lt; valueN+1 ))  {&#xA;        simpsonsRule += 4*(e/((Math.sin(e))));&#xA;        }&#xA;        else if ((i % 2 != 0) &amp;amp;&amp;amp; ( i &amp;gt; 1) &amp;amp;&amp;amp; (i &amp;lt; valueN+1 )) {&#xA;        simpsonsRule += 2*(e/((Math.sin(e))));   &#xA;        }&#xA;        else if (i == valueN+1 ) {&#xA;            simpsonsRule += (e/((Math.sin(e))));&#xA;&#xA;        }&#xA;&#xA;    }&#xA;    simpsonsRule = simpsonsRule *((valueDx)/3);&#xA;&#xA;    System.out.println(&quot;\nsimpsonsRule&quot; + simpsonsRule);&#xA;&#xA;     while(Math.abs(valueHolder - simpsonsRule) &amp;gt; Math.pow(10,-6)) {&#xA;           System.out.println(&quot;\nValueHolder&quot; + valueHolder);&#xA;           valueHolder = simpsonsRule;&#xA;           valueN +=2;&#xA;           valueDx = (valueB-valueA)/valueN;&#xA;           simpsonsRule = 0;&#xA;    for(int i = 1; i&amp;lt;=valueN + 1; i++){&#xA;        e = valueA + ((i-1)*valueDx);&#xA;&#xA;        if (i==1) {&#xA;        // Limit as x -&amp;gt; 0&#xA;        simpsonsRule += Math.pow(10,-10);&#xA;&#xA;        }&#xA;        else if (i % 2 == 0) {&#xA;        simpsonsRule += 4*(e/((Math.sin(e))));&#xA;        }&#xA;        else if ((i % 2 != 0) &amp;amp;&amp;amp; ( i &amp;gt; 1) &amp;amp;&amp;amp; (i &amp;lt; valueN + 1)) {&#xA;        simpsonsRule += 2*(e/((Math.sin(e))));   &#xA;        }&#xA;        else if (i == valueN + 1) {&#xA;            simpsonsRule += (e/((Math.sin(e))));&#xA;&#xA;        }&#xA;&#xA;    }&#xA;    simpsonsRule = simpsonsRule *((valueDx)/3);&#xA;&#xA;&#xA;    }&#xA;    return valueN;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18109" LastEditorUserId="7567" LastEditDate="2015-11-03T03:21:28.107" LastActivityDate="2016-01-02T04:47:04.823" Title="Simpsons Rule 10&lt;-6 Error" Tags="|calculus|java|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21191" PostTypeId="1" AcceptedAnswerId="21200" CreationDate="2015-11-03T00:15:01.743" Score="2" ViewCount="523" Body="&lt;p&gt;Consider a Laplace's equation with Dirichlet boundary condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${\nabla ^2}\Phi  = 0$ in a domain $D$ with given Dirichlet Boundary condition: $\Phi=\Phi_o$ at $\partial D$ (smooth, but not analytic boundary)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, that $\Phi_o$ is a periodic function and it changes depending on the position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, $\Phi_o(x,y) = \tan^{-1}(y/x)$ for elliptic domain, ($\frac{x^2}{a^2}+\frac{y^2}{b^2}=1$) and thus $\Phi\in [0,2\pi)$; in this case, the non-continuous boundary condition arises on $(x=a,y=0)$ that divides two distinct values ($\Phi_o(y&amp;gt;0) = 2\pi $ and $\Phi_o(y&amp;lt;0)=0 $).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence, the numerical discontinuity is observed at some of the boundary and deteriorate the solution greatly. Is there any possible treatment to aid this problem? Any comment would be greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="17204" LastEditorUserId="1804" LastEditDate="2015-11-03T18:40:54.487" LastActivityDate="2015-11-03T18:40:54.487" Title="Laplace's equation with periodic Dirichlet boundary conditions" Tags="|pde|boundary-conditions|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21192" PostTypeId="2" ParentId="21190" CreationDate="2015-11-03T02:11:56.000" Score="1" Body="&lt;p&gt;You can simplify your code by removing redundant checks in &lt;code&gt;if&lt;/code&gt; statements. You can also pull out the &lt;code&gt;valueDx&lt;/code&gt; which is constant in the loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double e = 0.0;&#xA;double simpsonsRule = 0.0;&#xA;double valueHolder = 0.0;&#xA;&#xA;valueN = 2;&#xA;valueA = 0;&#xA;valueB = (Math.PI)/2; &#xA;&#xA;valueDx = (valueB-valueA)/valueN;&#xA;simpsonsRule = 1.0 + Math.PI/2;&#xA;for(int i = 2; i&amp;lt;=valueN ; i++){&#xA;    e = valueA + ((i-1)*valueDx);&#xA;&#xA;    if (i % 2 == 0)  {  //even&#xA;        simpsonsRule += 4*(e/((Math.sin(e))));&#xA;    }&#xA;    else { //odd&#xA;        simpsonsRule += 2*(e/((Math.sin(e))));   &#xA;    }&#xA;}&#xA;&#xA;simpsonsRule = simpsonsRule *((valueDx)/3);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also it looks like you repeat this same loop twice in your code for some reason. I havn't run this so there may still be a few bugs especially in the indices etc but this is homework so I wont do it all for you =). You can look at a python or c variant which does what you want on &lt;a href=&quot;https://en.wikipedia.org/wiki/Simpson%27s_rule&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The python implementation is particularly close to yours. To answer your question though, you are evaluating between 0 and $\pi/2$. Thus the line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;simpsonsRule = 1.0 + (Math.PI)/2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;corresponds to adding $f(0) + f(\pi/2)$ where $f(x)=\frac{x}{\sin(x)}$. In the limit $f(0)-&amp;gt;1$ while $f(\pi/2)=\pi/2$.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-11-03T03:04:54.010" LastActivityDate="2015-11-03T03:04:54.010" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21193" PostTypeId="2" ParentId="21187" CreationDate="2015-11-03T05:50:10.527" Score="3" Body="&lt;p&gt;You haven't said anything about how much storage you have available to use for this computation.  A 20 million by 500 array of double precision floats would require 80 gigabytes of RAM to store. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have that much RAM then you'll want to organize the computation to work with smaller strips of the array that you bring in from disk one at a time.  For example, you could work with strips of 25 columns at a time and you'd need only 4 gigabytes of RAM to store each strip.  You'd compute the results for each set of 25 columns and add the resulting 500 by 500 matrices together to get the final result.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your weights $W$ are nonnegative, then you can simply take their square roots and write $W$ as $W=D^{2}$.  Then your product is $(DX)^{T}(DX)$.  You can scale by $D$ in one pass and then call the BLAS routine DGEMM to do the actual matrix multiplications.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your weights $W$ are mixed positive and negative, then I would make a copy of $X$ and scale it by $W$ before performing the matrix multiplication by DGEMM.  Again, if the matrices are too big to fit in memory you can block the computations by hand.      &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Libraries like Intel's MKL and AMD's ACML provide highly optimized implementations of the DGEMM routine.  You should definitely take advantage of the efforts that have been put into optimizing DGEMM in such libraries.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-11-03T05:50:10.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21194" PostTypeId="1" AcceptedAnswerId="21212" CreationDate="2015-11-03T08:38:51.130" Score="2" ViewCount="300" Body="&lt;p&gt;I have a numerical discretization of a partial differential equation that seems to be unstable or stable at a boundary point, depending on what finite difference scheme I am using. Are there standard methods for investigating stability at boundaries?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only know of the von Neumann stability analysis, but that seems to take into account the whole region, and I am particularly interested in instabilities coming from the boundary.&lt;/p&gt;&#xA;" OwnerUserId="15232" LastEditorUserId="123" LastEditDate="2015-11-04T07:25:53.280" LastActivityDate="2015-11-04T07:33:11.170" Title="How can I analyze the stability of a PDE discretization at a boundary?" Tags="|pde|finite-difference|stability|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21195" PostTypeId="2" ParentId="21186" CreationDate="2015-11-03T09:04:36.853" Score="1" Body="&lt;p&gt;The answer is, as expected, another &lt;a href=&quot;http://www.tik.ee.ethz.ch/file/c7c5e610a0c7e26d566a3601e5cce2f4/DTLZ2001a.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;. It was published one year before and has the same title. In this paper, there are in total eight DTLZ problems, and DTLZ5 from this paper is not part of the one from 2002. Renumbering the problems led to my confusion.&lt;/p&gt;&#xA;" OwnerUserId="18104" LastActivityDate="2015-11-03T09:04:36.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21196" PostTypeId="2" ParentId="21158" CreationDate="2015-11-03T10:33:37.540" Score="1" Body="&lt;p&gt;Since &lt;a href=&quot;http://uk.mathworks.com/help/matlab/ref/ode45.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;ODE45&lt;/code&gt;&lt;/a&gt; uses adaptive stepping, you might want to mention the error tolerance (&lt;code&gt;RelTol&lt;/code&gt; and &lt;code&gt;AbsTol&lt;/code&gt;) used. There is some information in the &lt;a href=&quot;http://uk.mathworks.com/help/matlab/ref/odeset.html#zmw57dd0e525830&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;odeset&lt;/code&gt; documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastActivityDate="2015-11-03T10:33:37.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21197" PostTypeId="2" ParentId="21163" CreationDate="2015-11-03T15:05:47.363" Score="4" Body="&lt;p&gt;I have implemented a method that works pretty well for reconstructing edge information in surfacic meshes (or facet information in volumetric meshes), but I am working in C++. I'll try to explain that in an &quot;abstract algorithmic language&quot; that could (hopefully) be translated into Fortran:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that I have a triangle mesh of nt triangles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1:&lt;/strong&gt; I first construct an array of 3*nt &quot;edge records&quot;, each &quot;edge record&quot; has the following fields: (i,j,t) and encodes an edge (i,j) seen from a given triangle (t). I also need to ensure that (i &amp;lt; j).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(note: in Fortran, this could be replaced with three arrays of size 3*nt, one array I[], one array J[] and one array T[], but maybe Fortran90 has &quot;structures&quot;/&quot;records&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To construct the structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;E : array of 3*nt &quot;edge records&quot;&#xA;e = 1&#xA;For t = 1 to nt       &#xA;  get i,j,k the vertices indices of triangle t        &#xA;  E[e].i = min(i,j); E[e].j = max(i,j); E[e].t = t; e = e + 1&#xA;  E[e].i = min(j,k); E[e].j = max(j,k); E[e].t = t; e = e + 1&#xA;  E[e].i = min(k,l); E[e].j = max(k,l); E[e].t = t; e = e + 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2:&lt;/strong&gt; Sort the records by lexicographic sort, i.e. given two records E[e1] and E[e2], E[e1] is supposed to be before E[e2] if E[e1].i &amp;lt; E[e2].i &lt;strong&gt;OR&lt;/strong&gt; (E[e1].i = E[e1].i and E[e1].j &amp;lt; E[e2].j) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(in Fortran, I am unsure of how this can be achieved, you will need to find a generic sorting routine, or maybe will need to implement it on your own if you have three separate I[], J[], T[] arrays. If Fortran90 &#xA;has &quot;structure/records&quot;, maybe the sort() function of the C runtime can be called from Fortran). In the worst case, it is possible to save the array E in a file, call the system sort program on it and read it back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 3:&lt;/strong&gt; Once the &quot;edges record&quot; array is sorted, the 'same edge' seen from two different triangles will be represented by two records that are contiguous. Then you can generate unique edge indices by iterating on it, and incrementing the current edge index each time a different edge is encountered. Edges that have two triangles incident to them will be represented by a sequence of two &quot;edge records&quot; (i,j,t1), (i,j,t2). Then it is easy to generate the arrays that give for each triangle the indexes of its three edges, and for each edge the index of the (up to two) triangles adjacent to it. It also allows to detect invalid meshes, with more than two triangles incident to the same edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also an alternative method, that gives for each vertex the list of triangles incident to it. It is done by &quot;chaining&quot; the &quot;corners&quot; of the triangles (using an array of size 3*nt).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++ implementation in my &quot;geogram&quot; programming library: &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/namespaceGEO.html#af10149ff831242b94bf3c92ac233111a&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/namespaceGEO.html#af10149ff831242b94bf3c92ac233111a&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Download: &lt;a href=&quot;https://gforge.inria.fr/frs/?group_id=5833&quot; rel=&quot;nofollow&quot;&gt;https://gforge.inria.fr/frs/?group_id=5833&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2015-11-11T10:14:06.283" LastActivityDate="2015-11-11T10:14:06.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21198" PostTypeId="2" ParentId="21187" CreationDate="2015-11-03T17:01:57.210" Score="0" Body="&lt;p&gt;Just an improvement on Brian's answer. Instead of using DGEMM, you can use DSYRK to multiply a matrix by its transpose.&lt;/p&gt;&#xA;" OwnerUserId="18116" LastActivityDate="2015-11-03T17:01:57.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21199" PostTypeId="1" CreationDate="2015-11-03T17:50:20.787" Score="3" ViewCount="243" Body="&lt;p&gt;I have a system of ODEs which is (at least moderately) stiff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the class of spectral collocation methods &lt;a href=&quot;https://en.wikipedia.org/wiki/Spectral_method&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Spectral_method&lt;/a&gt; or the related class of weighted residual methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, take a system of ODEs like&#xA;$$&#xA;y'(t) = f(t, y(t))&#xA;$$&#xA;where $f(\cdot,\cdot)$ is nonlinear in my case and leads to a stiff ODE, and I want to solve it on $t \in [0,M]$.  Then approximate the solution $$y(t) \approx \sum_{i=0}^N a_i T_i(t)$$&#xA;where $T_i(\cdot)$ are a basis (e.g. Chebyshev polynomials), $N$ is the maximum order, and the basis is over $[0,M]$ if appropriate.  Note that given $\{a_i\}$, $y'(t)$ can be calculated by differentiating the polynomial basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, evaluate the ODE given a $\{a_i\}$ at the roots of the polynomial basis (which can be shown to be optimal) and find the residual.  Choose $\{a_i\}$ to minimize this residual, or just solve a system of equations with the residual = 0.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: how well does spectral collocation with a Chebyshev basis handle systems that seem stiff when solved with finite differences?  The worry is that the same issues that cause finite differences to have issues (e.g. convergence to a very flat function with no slope) could cause major issues with a finite dimensional approximation of the function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Chebyshev has problems, then is there a better basis which might have less issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A little more context:&lt;/strong&gt; I have implemented spectral collocation with Chebyshev polynomials to solve a stationary Kolmogorov Forward equation in CDFs.  These, of course, need to be strictly positive and weakly monotonically increasing and asymptotically go towards $1$ if it is a proper probability distribution.  I am seeing weird behavior where the pdf drops below $0$ (i.e., it becomes decreasing) or strange jumps at the corners.  It is very hard for me to tell if these are artifacts in the solution scheme or if I have just chosen parameters for the model where no solution exists.  Increasing the number of Chebyshev basis functions doesn't eliminate them, but that could be because they can't, or only slowly, converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Even more context:&lt;/strong&gt;  For what it is worth, in the simplest form of my problem there are two sets of ODEs (which are ultimately coupled):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Parameters&lt;/em&gt;: $\lambda_{\ell}, \lambda_h, r,$ and $\chi$.  In some ways $F_{\ell}'(0), F_h'(0),$ and $g$ are also parameters for the purposes here.  There are two discrete states which leads to the system: $i \in \{\ell,h\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variable $z\in[0,\infty)$ in reality, but these converge pretty quickly so choosing some $z \in [0,\bar{z}]$ for collocation methods is reasonable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;System (1)&lt;/em&gt; The following comes from a stationary Kolmogorov Forward Equation (given a $\gamma(z)$ function such that $\gamma(0) = 0$ and $\lim\limits_{z\to\infty} \gamma(z) = g$.  The latter condition leads to a vanishing derivative term in the ODE (i.e., a singular mass matrix if put in that canonical form.)&#xA;$$&#xA;\begin{align}&#xA;0 &amp;amp;= g F_{\ell}'(z) + \lambda_h F_h(z) - \lambda_{\ell}F_{\ell}(z) + g (F_{\ell}'(0) + F_h'(0))(F_{\ell}(z) + F_h(z)) - g F_{\ell}'(0)\\&#xA;0 &amp;amp;= (g - \gamma(z))F_h'(z) + \lambda_{\ell}F_{\ell}(z) - \lambda_h F_h(z) - g F_h'(0)&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Forget that this is a linear ODE, as the actual one is nonlinear and much trickier.  However, it can still be written in a $M(z, F(z))\cdot F'(z) = \Phi(z, F(z))$ term for some operators and a mass matrix.&lt;/li&gt;&#xA;&lt;li&gt;The trouble seems to come out of the term on the derivative going to $0$ (i.e., a singular mass matrix).&lt;/li&gt;&#xA;&lt;li&gt;It might seem odd that there is a $F_{\ell}'(0)$ and a $F_h'(0)$ in the ODE as parameters but this is not a mistake.  You can easily show that any solution to the ODE will have the $F_i'(0)$ matching these constants by construction.&lt;/li&gt;&#xA;&lt;li&gt;The initial condition is $F_i(0) = 0$.  Furthermore, from any $F_i'(0)$, one can analytically find $\lim\limits_{z\to\infty}F_h(z)$.  With $F_{\ell}(\infty) + F_h(\infty) = 1$, this means we can write it as a BVP if we wish.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;System (2)&lt;/em&gt; The following comes from a Hamilton-Jacobi-Bellman equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;0 &amp;amp;= 1  - (r + \lambda_{\ell})w_{\ell}(z) - g w'_{\ell}(z) + \lambda_h w_h(z)\\&#xA;0 &amp;amp;= 1 - (r + \lambda_h)w_h(z) - (g - \frac{\chi}{2} w_h(z))w_h'(z) + \lambda_h w_{\ell}(z) + \frac{\chi}{4} w_h(z)^2&#xA;\end{align}&#xA;subject to, $w_{\ell}(0) = w_h(0) = 0$ and defined on $z \in [0,\infty)$.  You can analytically find $w_{\ell}(\infty)$ and $w_h(\infty)$ which can help with various methods to set it as a BVP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can show that $\lim\limits_{z\to\infty}(g - \frac{\chi}{2} w_h(z)) = 0$ which leads to a vanishing derivative term in the ODE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Coupled&lt;/em&gt;: In reality, the $\gamma(z)$ from the first solution is &#xA;$$&#xA;\gamma(z) \equiv \frac{\chi}{2}w_h(z)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Leaving Out&lt;/em&gt;: A whole bunch of equilibrium conditions in terms of integrals of $w_i(z)$ and $F_i(z)$ which pin down $g, F'_{\ell}(0)$ and $F'_h(0)$.  Evaluating these concurrently with the solution to the ODEs is the reason that spectral methods are preferable since I can add in more equations and solve everything at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... is what I have written useful?  Maybe not, but you can see the asymptotic singularity in the mass matrix.  Is this stiff?  I think so.  When you solve the second system with finite differences, for example, it has all the hallmarks of a stiff system.&lt;/p&gt;&#xA;" OwnerUserId="9692" LastEditorUserId="9692" LastEditDate="2015-11-06T22:40:14.697" LastActivityDate="2015-11-06T22:40:14.697" Title="Spectral Collocation (or Weighted Residual) Methods to solve Stiff ODEs?" Tags="|ode|spectral-method|stiffness|" AnswerCount="0" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="21200" PostTypeId="2" ParentId="21191" CreationDate="2015-11-03T17:56:46.623" Score="2" Body="&lt;p&gt;This is not just a &quot;numerical discontinuity&quot;. Are you sure that this is the problem you want to solve? $\Phi$ looks like a phase, which is defined up to $2\pi$. That may be what you refer to as &quot;periodic&quot;, although the context makes it unclear as one thinks of space-periodicity of BCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your solution $\Phi$ is up to $2\pi$, I would rather solve e.g. for $u=\cos\Phi$.&lt;/p&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2015-11-03T17:56:46.623" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21202" PostTypeId="1" AcceptedAnswerId="21211" CreationDate="2015-11-03T20:09:36.427" Score="2" ViewCount="1359" Body="&lt;p&gt;I want to solve 7 coupled differential equations in MATLAB. I used the method of lines (MOL) to convert them to a system of DAEs. I then attempted to solve this system using ode15s. Unfortunately an error occurred that said: the index of equation is greater than 1. I searched and know I should reduce the index of equation. But I dont know how can I do this in MATLAB?&#xA;I have simplified equations and assume T=CONSTANT so these  are 3 equations: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$1:'(∂C_(g))/∂t+ u_g  (∂C_(g))/∂z= -k_g A_o (C_(g)-C_(p) |_(r=r_OD ) )+A_o ε_f u_cp |_(r_OD ) C_(p) |_(r_OD )'$&#xA;$2:(∂C_(p))/∂t+(u_cp C_(p))/r+u_cp  (∂C_(p))/∂r-D_eff ((∂^2 C_(p))/(∂r^2 )+1/r  (∂C_(p))/∂r)=-((1-ε_f))/ε_f  ρ_f  (∂q_)/∂t$&lt;br&gt;&#xA;$3:(∂q)/∂t=ks (q^eq-q)$&lt;br&gt;&#xA;$4:q^eq=(qs.b.P_)/〖(1+(b.P_ )^n)〗^(1/n)$&lt;br&gt;&#xA;$5:P=C(p)RT$   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and these are initial and boundry conditions:&lt;br&gt;&#xA;at $t=0$  $q=0$, $cp=0$ and $cg=0$&lt;br&gt;&#xA;at $z=0$ $cg=constant$&lt;br&gt;&#xA;ar $r=riD$  $(∂C_(i,p))/∂r=0$&lt;br&gt;&#xA;at $r=roD$   $Deff.(∂C_(i,p))/∂r=kg.(cg(z)-cp(r=roD)$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I convert them to system of odes in time.&#xA;these are equations:&lt;br&gt;&#xA;$i$ is used for lenght(z).and $j$ is used for radius(r)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$1':dcg(i,j)=((-1).*(Ug).*(((cg(i,j))-(cg(i-1,j)))./((deltaz))))-((kg)*(Ao).*((cg(i,nr2))-(cp(i,nr2))))+((Ao)*(epsilonf).*(Ucp)*(cp(i,nr2)))$&lt;br&gt;&#xA;$2':dcp(i,j)=(((-1).*(Ucp).*(cp(i,j)))./((Ri)+((j-1)*(deltar))))-((Ucp).*(((cp(i,j))-(cp(i,j-1)))./((deltar))))+((Deff).*((((cp(i,j+1)))-(2*cp(i,j))+(cp(i,j-1)))./(deltar^2))+(((1)/((Ri)+((j-1)*(deltar)))).*(((cp(i,j))-(cp(i,j-1)))./((deltar)))))-((((1)-(epsilonf))/(epsilonf))*(ROf).*(dq(i,j)))$&lt;br&gt;&#xA;$3': dq(i,j)=ks*(qeq(i,j)-q(i,j))$  &lt;/p&gt;&#xA;" OwnerUserId="17594" LastEditorUserId="17594" LastEditDate="2015-11-06T16:07:24.883" LastActivityDate="2021-03-13T09:37:22.847" Title="Solving DAE with higher index" Tags="|matlab|pde|method-of-lines|dae|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21203" PostTypeId="2" ParentId="21184" CreationDate="2015-11-03T21:58:03.797" Score="1" Body="&lt;p&gt;Yes, you are looking for quadratically-constraint quadratic programming (QCQP).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would solve your SQP as a sequential QCQP, (You can find results in literature for SQCQP). Most solvers come with a &quot;SOCP&quot;(Second-order cone program) solver - SOCP's subsume QP's and QCQP's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, as I know nothing about your actual problem domain - it's also often totally fair to take a first-order Taylor approximation of your constraints, just as you (presumably) took a second-order Taylor approximation of your cost function for your SQP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a paper that does that (in robotics) is &lt;a href=&quot;http://www.fossen.biz/home/papers/tcst04.pdf&quot; rel=&quot;nofollow&quot;&gt;Fossen's Constrainsed Nonlinear Control Allocation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18065" LastActivityDate="2015-11-03T21:58:03.797" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21204" PostTypeId="1" CreationDate="2015-11-04T00:24:31.553" Score="1" ViewCount="337" Body="&lt;p&gt;Can the following 1D transient advection diffusion and corresponding initial and boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial c}{\partial t} + v\frac{\partial c}{\partial x} - D\frac{\partial^2 c}{\partial x^2} = f \quad\forall x\in(0,1)&#xA;\\ &#xA;c(x=0,t) = 0 \\&#xA;c(x=1,t) = 0 \\&#xA;c(x,t=0) = 0&#xA;$$&#xA;be solved such that we first move the advection and diffusion operators to the RHS, integrate with respect to time, and obtain the following second order ODE:&#xA;$$&#xA;c + vt\frac{\partial c}{\partial x} - Dt\frac{\partial^2 c}{\partial x^2} = tf&#xA;$$&#xA;while assuming that $v,\;D,\;and\;f$ are constants (i.e., they do not vary spatially or temporally)? Because when I tried this and compared the solution to my SUPG and Operator Splitting formulation (assuming the same $h$-size with backward euler), it seems these numerical discretization approach steady state much quicker than the analytical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if $h$-size = 1/100, $\Delta t$ = 0.01, $v$ = 1,  $D$ = 1/150, and $f$ = 1, SUPG/OS comes close to the steady-state solution at $t$ = 1.0, but for the analytical solution to be close, the value of $t$ needs to be somewhere around 10 to have roughly the same closeness to the steady-state solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know if this is an acceptable approach if I want to perform a numerical convergence study comparing SUPG/OS with the analytical solution. I would rather compare the solutions at the exact time levels as opposed to seeing how close they get to the steady-state solution after $T$ amount of time.&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2015-11-04T00:24:31.553" Title="Solving the transient advection-diffusion equation analytically" Tags="|finite-element|advection-diffusion|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21205" PostTypeId="1" AcceptedAnswerId="21207" CreationDate="2015-11-04T00:51:52.630" Score="4" ViewCount="483" Body="&lt;p&gt;I am a beginner in Computational science and FEM. I came across some PDEs which implement time dependent boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not able to visualize exactly a physical scenario of how that would look or what it would mean with reference to a real problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say for example, a wave equation - where the waves are reflected back if BC = 0; what would a changing BC imply here and how would we know the expression for such a changing BC? Same goes for a heat equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain in brief through an example or point me towards a document?&lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="9667" LastEditDate="2015-11-04T06:50:25.870" LastActivityDate="2015-11-05T18:34:46.580" Title="How to physically understand time dependent boundary conditions?" Tags="|boundary-conditions|time-integration|wave-propagation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21206" PostTypeId="2" ParentId="21128" CreationDate="2015-11-04T01:19:44.447" Score="2" Body="&lt;p&gt;Just some quick ideas from someone who works in the realm of feature extraction in physical systems modeling such as this: you want to find the simplest and strongest differentiating characteristics of the different behaviors you're interested in classifying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, though: do you know in advance the full extent of behaviors you wish to classify? If not, the problem will be much harder. If you do, then brainstorm the key non-overlapping differences. Alas, you mention drinking and pouring, but drinking &lt;em&gt;is&lt;/em&gt; a subset of pouring where the target is the drinker's mouth. So you might need some distinguishing characteristics from the data such as the height of the can in absolute space (unless you are allowing for people drinking while sitting, etc. etc.). As you see, assumptions and context are everything in tackling these problems, and expecting to classify human behavior based on a single trajectory of three numbers does not sound like a robust way to solve a problem, &lt;em&gt;as stated&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More importantly then, you should consider what is the ultimate problem you're actually trying to solve? What else can you measure or know about the situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know more about the context, possible features might include measures of the linearity of the trajectory or, conversely, radius of curvature in the trajectory. Also, measure the amount of rotation of the can (but do you just have a single point? In which case you've already lost a valuable feature...), or look for clusters of different patterns of motion and rotation through time, and split up your trajectory into piecewise segments. But none of this is helpful if you don't understand a &lt;em&gt;lot&lt;/em&gt; more about the context of what you're trying to solve.&lt;/p&gt;&#xA;" OwnerUserId="1693" LastActivityDate="2015-11-04T01:19:44.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21207" PostTypeId="2" ParentId="21205" CreationDate="2015-11-04T01:22:11.433" Score="4" Body="&lt;p&gt;For the heat equation imagine a rod that is heated on the left and cooled on the right. Now imagine that instead of a constant prescribed temperature on the left what we want is the heat to steadily rise and fall. This would be an example of a time-dependent boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  \frac{\partial{u}}{\partial{t}} = c\frac{\partial^{2}{u}}{\partial{x}^{2}}\\&#xA;  u(x,0) = f(x)\\&#xA;  u(0,t) = \phi(t) \hspace{4mm} u(1,t) = 0\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where maybe we could model the $\phi(t)=\sin(t)$ so that the temperature rises and falls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly for the wave equation, imagine a string that is fixed at the right end but moves up and down on the left end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional example:&lt;/strong&gt;&#xA;Another example using the heat equation would be a rod with a point source in the middle, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  \frac{\partial{u}}{\partial{t}} = c\frac{\partial^{2}{u}}{\partial{x}^{2}}\\&#xA;  u(x,0) = \left\{&#xA;	     \begin{array}{ll}&#xA;		1/2  &amp;amp; \mbox{if } x = 1/2 \\&#xA;		0 &amp;amp; \mbox{else}&#xA;	     \end{array}&#xA;           \right.\\&#xA;  u(0,t) = 0 \hspace{4mm} u(1,t) = 0\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now this just has two zero Dirichlet conditions on the left and right ends of the rod. We could make this more interesting by adding a time dependent condition to the left end. Perhaps something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  \frac{du}{dx}(0,t) = \min(u,1)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might represent some kind of chemical reaction of the left hand side where we initially have no flux when the temperature is zero there, but later as heat makes it to the wall it results in a flux.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-11-05T18:34:46.580" LastActivityDate="2015-11-05T18:34:46.580" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21208" PostTypeId="2" ParentId="11269" CreationDate="2015-11-04T01:37:24.970" Score="1" Body="&lt;p&gt;SUNDIALS is OK, IMHO, but as an external library of legacy components it does not have the front-end interface capabilities that a python-centric environment does. I built the &lt;a href=&quot;http://pydstool.sf.net&quot; rel=&quot;nofollow&quot;&gt;PyDSTool&lt;/a&gt; system to do this kind of work. It has a high-level set of constructions for dynamical systems and modeling beyond just the solvers. This includes a sophisticated and robust way to specify stopping points that are state or time dependent (called &lt;a href=&quot;http://www.ni.gsu.edu/~rclewley/PyDSTool/Events.html&quot; rel=&quot;nofollow&quot;&gt;&quot;events&quot;&lt;/a&gt;). The events are solved for internally, automatically, and to arbitrary precision. You can then sample the solution very naturally according to any mesh you desire for plotting purposes. See &lt;a href=&quot;http://www.ni.gsu.edu/~rclewley/PyDSTool/Tutorial/Tutorial_VdP.html&quot; rel=&quot;nofollow&quot;&gt;this example&lt;/a&gt; for instance, and the other tutorials and supplied demos to see how to sample and plot with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. PyDSTool can auto-generate C code to generate much faster-running solvers than any integrator that uses callbacks to python-defined functions, which is pretty much any other solution out there.&lt;/p&gt;&#xA;" OwnerUserId="1693" LastActivityDate="2015-11-04T01:37:24.970" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21209" PostTypeId="2" ParentId="21136" CreationDate="2015-11-04T01:46:43.080" Score="2" Body="&lt;p&gt;There are a few and they most typically rely on setting up a boundary value problem. You can find solutions to these in the AUTO continuation package, and the XPP package that provides a (partial) front end to it. However, you'll still have to understand how to set it up because it's not an automated process. There are tutorials for those packages that include that application, I believe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, my own package, &lt;a href=&quot;http://pydstool.sf.net&quot; rel=&quot;nofollow&quot;&gt;PyDSTool&lt;/a&gt;, is meant for dynamical systems analysis and comes with a prototype (and somewhat naive) tool to compute separatrices -- in fact, any geometrically simple sub-manifold around a fixed point. It is described in graphic detail &lt;a href=&quot;http://www.ni.gsu.edu/~rclewley/PyDSTool/Tutorial/Tutorial_saddle_manifolds.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. It uses the shooting method. A similar computation was achieved using my package's PyCont sub-package (uses AUTO internally), which is described &lt;a href=&quot;http://www.math.cornell.edu/~gucken/PDF/gl.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The shooting method will, however, struggle more if you need to cover a very large domain of the independent variable unless its geometry remains relatively &quot;simple&quot;. E.g. it doesn't get tangled or squeezed close to other sub-manifolds. It should be a good starting point for further analysis, though.&lt;/p&gt;&#xA;" OwnerUserId="1693" LastActivityDate="2015-11-04T01:46:43.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21210" PostTypeId="2" ParentId="20982" CreationDate="2015-11-04T05:49:34.040" Score="2" Body="&lt;p&gt;With the help of Wolfgangs answer I think I have figured it out. It is necessary to include additional pointers in my Node class so that each Node in the binary tree can know who are its neighbors. For example my Node and Tree classes look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;class T&amp;gt;&#xA;class Node&#xA;{&#xA;  public:&#xA;    T element;&#xA;    Node *parent;&#xA;    Node *left;&#xA;    Node *right;&#xA;&#xA;  private:&#xA;    Node *neigh01;&#xA;    Node *neigh12;&#xA;    Node *neigh20;&#xA;&#xA;    int hnode01;   //store hanging node numbers for this node&#xA;    int hnode12;  &#xA;    int hnode20;   &#xA;&#xA;    bool refine;   //refinement flag saying this node is in need of refinement&#xA;&#xA;  public:&#xA;    Node(T e) : element(e), parent(NULL), left(NULL), right(NULL), neigh01(NULL), neigh12(NULL), neigh20(NULL), hnode01(-1), hnode12(-1), hnode20(-1)&#xA;    {&#xA;    }&#xA;&#xA;  //rest of methods&#xA;};&#xA;&#xA;&#xA;template &amp;lt;class T&amp;gt;&#xA;class Tree&#xA;{&#xA;  private:&#xA;    int numNodes;&#xA;    int numLeafNodes&#xA;    Node&amp;lt;T&amp;gt; *root;&#xA;    Node&amp;lt;T&amp;gt; *iterator;&#xA;&#xA;  public:&#xA;    Tree();&#xA;    Tree(T e);&#xA;    ~Tree();&#xA;&#xA;  void insert(T e);&#xA;  void deleteTree(Node&amp;lt;T&amp;gt; *leaf);&#xA;&#xA;  //rest of class methods, etc...&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The three additional Node pointers - neigh01, neigh12, &amp;amp; neigh20 - allow us to keep track of any neighbor Nodes. Graphically our tree structure might look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0kwJ3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0kwJ3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VCAJt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VCAJt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LcHkA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LcHkA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QB9Ex.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QB9Ex.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the red lines indicate neighbor Node connections. The important thing to remember when a triangle is refined (i.e. split by longest edge bisection) is that these new child Nodes can only be neighbors with:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;NULL (in the case of a boundary, i.e. no neighbor)&lt;/li&gt;&#xA;&lt;li&gt;each other (each left child Node will always have a neighboring right child Node)&lt;/li&gt;&#xA;&lt;li&gt;neighbors or children of the child Nodes' parent Node (any Nodes that are neighbors or children of a parent Node &lt;strong&gt;may&lt;/strong&gt; be neighbors of the newly created child Nodes)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It is a little tricky, but with some shallow recursive functions I was able to ensure that the correct neighbor pointers were assigned every time a Node is refined. Here is the result for a test mesh before and after refinement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hIC65.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hIC65.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zMYk3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zMYk3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed the mesh remains conformal!&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2015-11-04T05:49:34.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21211" PostTypeId="2" ParentId="21202" CreationDate="2015-11-04T07:30:25.153" Score="4" Body="&lt;p&gt;There are many definitions of DAE index. MATLAB is probably referring to the differentiation index. A few integrators can solve Hessenberg form index-2 DAEs (I believe IDA can do this, along with a few other packages), but most require the index to be 1 or less. Reducing the index of your DAE requires manipulating the underlying equations in the DAE, e.g., using Pantelides' algorithm or the dummy derivative method. Usually, this sort of thing is done in software that has some sort of algebraic manipulation capabilities, like Mathematica, Maple, Sage, or SymPy, so in your case, you would use one of the above approaches to derive an equivalent reduced-index DAE, and then transcribe it into MATLAB and solve it with &lt;code&gt;ode15s&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2015-11-06T07:05:44.273" LastActivityDate="2015-11-06T07:05:44.273" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21212" PostTypeId="2" ParentId="21194" CreationDate="2015-11-04T07:33:11.170" Score="3" Body="&lt;p&gt;If your PDE and your discretization are linear, then your numerical method can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U_{n+1} = A U_n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is a square matrix.  One way to investigate stability is to look at some induced matrix norm of $A$: if it is larger than 1, the discretization is unstable.  This is completely general and is probably what you are looking for.  Below I explain that von Neumann analysis is just a special case of this general approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is a normal matrix (i.e., if $AA^T = A^T A$), then to show stability it is sufficient to check that the eigenvalues of $A$ are not larger than 1 in modulus.  That is because for normal matrices, the induced 2-norm is equal to the modulus of the largest eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the problem has periodic boundary conditions and a standard finite difference discretization is used, then $A$ will be a circulant matrix.  Circulant matrices are normal, and the eigenvectors of circulant matrices are just discrete Fourier modes.  This makes it easy to compute their eigenvalues, and von Neumann analysis is simply a way of determining those eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-11-04T07:33:11.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21213" PostTypeId="2" ParentId="21176" CreationDate="2015-11-04T08:07:49.097" Score="1" Body="&lt;p&gt;I think the answer is in the definition of &quot;Static Pressure&quot;, which is given, according to the British Standards Institutions (&lt;a href=&quot;https://en.wikipedia.org/wiki/BSI_Group&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/BSI_Group&lt;/a&gt;), as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The pressure at a point on a body moving with the fluid&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So the term &quot;static&quot; it is not to be intended as absence of motion.&#xA;Also, every point in a steadily flowing fluid, regardless of the fluid speed in that point, has its own static pressure .&lt;/p&gt;&#xA;" OwnerUserId="11676" LastActivityDate="2015-11-04T08:07:49.097" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21214" PostTypeId="1" CreationDate="2015-11-04T08:45:25.220" Score="8" ViewCount="1179" Body="&lt;p&gt;I very recently started to read up about magnetohydrodynamics (MHD). While I have experience in the fluid part (both theory and numerics), my knowledge about the magneto part is very limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, I am working through the book by &lt;a href=&quot;http://www.cambridge.org/catalogue/catalogue.asp?isbn=9780521794879&quot; rel=&quot;nofollow noreferrer&quot;&gt;Davidson&lt;/a&gt; which is great for learning about physics. I decided that a good first step will be to write my own simple code solving the induction equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} B_t = \nabla \times \left( \mathbf{u} \times B \right). \end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The problem is I do not know how a specific choice of numerical method will perform for this problem nor how good test cases would look like.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I am looking for a good introductory book or script on numerical methods for MHD. Ideally, I hope to find something that is similar to the book by &lt;a href=&quot;http://www.springer.com/us/book/9781441964113&quot; rel=&quot;nofollow noreferrer&quot;&gt;Durran&lt;/a&gt; for geophysical fluid dynamics (GFD) - a thorough introduction to different numerical methods used in the field and analysis of their performance from simple to complex benchmarks problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum&lt;/strong&gt;: To clarify my question a little bit, I am not looking for general introductions into methods that are used in MHD (finite differences, specific integration methods, finite elements, etc). Rather, I am looking for a book that discusses how these methods perform for specific equations related to MHD. For example, what happens if I solve the induction equation with an implicit Euler and centred differences? What changes if I use an upwind stencil instead? The book by Durran does a really great job answering such questions for GFD - I was hoping something similar might be around for MHD, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PS&lt;/strong&gt;: I found the following &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7890/simple-mhd-simulation-code-for-self-education-and-play-with&quot;&gt;question&lt;/a&gt; which is interesting (I will try out the codes linked there), but does not provide an answer for a good &lt;em&gt;book&lt;/em&gt; to understand what is happening in the codes linked there.&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="20688" LastEditDate="2019-07-03T07:09:05.880" LastActivityDate="2019-07-03T07:09:05.880" Title="Good introduction to numerical methods for magnetohydrodynamics (MHD)" Tags="|fluid-dynamics|reference-request|electromagnetism|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21215" PostTypeId="2" ParentId="5321" CreationDate="2015-11-04T10:05:42.677" Score="0" Body="&lt;p&gt;yep this is just a simple Rejection Sampling application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just draw random numbers uniformly distributed in the space defined by the square, say (0,1) .. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;keep track of which numbers you accept (fall inside the circle; radius,r=0.5), and which numbers you reject (fall outside the circle)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;remember to offset the criteria for the r=sqrt(x^2+y^2) so that you are evaluating the random number as its distance from the center of the circle (say [0.5,0.5] = [x,y] if your square is (0,0),(0,1),(1,1),(1,0))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;area circle: pi * r * r&lt;/p&gt;&#xA;&#xA;&lt;p&gt;area square: l*w&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ratio --&gt;  accepted/(accepted+rejected) = area circle/area square&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                    = pi * r * r / l * w&#xA;                                    = pi*(0.5*0.5)/1*1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can just look up Rejection Sampling  ... it's a Monte Carlo Method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quora.com/How-would-you-explain-Monte-Carlo-to-a-12-year-old&quot; rel=&quot;nofollow&quot;&gt;Good Old Buffon's Needle&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18122" LastActivityDate="2015-11-04T10:05:42.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21216" PostTypeId="2" ParentId="7711" CreationDate="2015-11-04T10:11:17.790" Score="1" Body="&lt;p&gt;You may use a sparse factorization algorithm, it means computing matrices $P$, $L$, $U$, such that $M = PLU$ where $P$ is a permutation matrix, $L$ a sparse lower triangular matrix and $U$ a sparse upper triangular matrix. The permutation matrix is there and computed in such a way that $L$ and $U$ remain reasonably sparse (without it it is not the case in general, as indicated in the other answer). Then when $M$ is factorized in this form, it is trivial to solve a linear system with an arbitrary rhs. If $M$ is symmetric definite, then there is a sparse cholesky factorization ($M = PLL^t$), if it is symmetric only, then it can be decomosed as $M = PLDL^t$, with $D$ a diagonal matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several available implementations of sparse factorization: SuperLU, Choldmod, Mumps, TAUCS (depending on whether you need $LU$, $LL^t$ or $LDL^t$). There probably exists MATLAB bindings for most of them.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2015-11-04T10:11:17.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21217" PostTypeId="2" ParentId="21176" CreationDate="2015-11-04T10:24:25.223" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;Because Static Pressure is just another way of saying pressure, but specific to fluid flow/dynamics. It is the pressure associated with something moving &lt;em&gt;with&lt;/em&gt; the fluid.&lt;/p&gt;&#xA;    &#xA;    &lt;p&gt;Dynamic pressure is the kinetic energy per unit volume of a fluid particle. (dynamic pressure usually only for incompressible flows - For compressible flows, try &quot;impact pressure&quot;).&lt;/p&gt;&#xA;    &#xA;    &lt;p&gt;Stagnation pressure (or pitot pressure) is the static pressure at a stagnation point in a fluid flow... aka it is the same idea as the static pressure BUT instead of there being cohesive/constant/&quot;not changing&quot;/&quot;static&quot; fluid motion ... At a stagnation point the fluid velocity is zero and all kinetic energy has been converted into pressure energy &lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;**static pressure + dynamic pressure = stagnation pressure**&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18122" LastActivityDate="2015-11-04T10:24:25.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21218" PostTypeId="2" ParentId="21214" CreationDate="2015-11-04T11:30:30.970" Score="3" Body="&lt;p&gt;From what I understand, you'd like to see which numerical method best simulate the real physics relevant for a particular problem. MHD spans a wide scale of phenomena -- plasma physics (on length scales of ions and electrons) to ideal MHD (on the length scale relevant to accretion disks around blackhole or other compact objects).&lt;br&gt;&#xA;In this case, it's advisable to keep track of what part/scale of physics you'd like simulate and neglect others with proper justification from physics scaling arguments, especially where fluid approximation is or is not suitable and to what error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have referred mostly the book by &lt;strong&gt;Bowers and Wilson, Numerical Modeling in Applied Physics and Astrophysics&lt;/strong&gt;. It introduces the numerical scheme very soon after setting up the basic equations and has an introduction to both Lagrangian and Eulerian schemes. Gabor Toth, an experienced researcher in this field, has lecture notes -- &lt;a href=&quot;http://www-personal.umich.edu/~gtoth/Teach/porto_course.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www-personal.umich.edu/~gtoth/Teach/porto_course.pdf&lt;/a&gt;. It is a good introduction to common problems in MHD simulation (keeping divergence free etc) and principles of code design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding test cases, any research grade code base will have an excellent comparison or at least a list of test problems. FLASH (&lt;a href=&quot;http://flash.uchicago.edu/site/&quot; rel=&quot;nofollow&quot;&gt;http://flash.uchicago.edu/site/&lt;/a&gt;) is one such base extensively used for research. Its user guide (&lt;a href=&quot;http://flash.uchicago.edu/site/flashcode/user_support/flash4_ug_4p3/node39.html#SECTION010120000000000000000&quot; rel=&quot;nofollow&quot;&gt;http://flash.uchicago.edu/site/flashcode/user_support/flash4_ug_4p3/node39.html#SECTION010120000000000000000&lt;/a&gt;) has a good collection and considered as a good set of test problems and MHD code has to satisfy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.: For induction equation, the two-dimensional MHD rotor problem (Balsara and Spicer, 1999) would be a good one. &lt;/p&gt;&#xA;" OwnerUserId="4992" LastActivityDate="2015-11-04T11:30:30.970" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21219" PostTypeId="1" CreationDate="2015-11-04T15:41:34.063" Score="2" ViewCount="120" Body="&lt;p&gt;I need to have a better understanding about how to define ODE events.&#xA;What I know is that if I have my ODE defined as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[t,q,te,qe,ie] = ode45(odefun,tspan,q0(:,1),options);&#xA;options = odeset('Events',@events);&#xA;function [value,isterminal,direction] = events(t,q)&#xA;    value = ...; &#xA;    isterminal = ...; &#xA;    direction = ...; &#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to define the &lt;code&gt;value&lt;/code&gt; as a function of &lt;code&gt;q&lt;/code&gt; and &lt;code&gt;t&lt;/code&gt;in such a way that it goes to zero. So, for example, if I want to find when &lt;code&gt;v = 0&lt;/code&gt; I just need to write&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;value = q(2);&#xA;isterminal = 1;&#xA;direction = 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But is writing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;value = [q(2),q(2)];&#xA;isterminal = [1,1];&#xA;direction = [1,-1];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;equivalent to the previous formulation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And what if I want to stop the integration when &lt;code&gt;q(2) &amp;lt; toll&lt;/code&gt;? I mean, is it possible to stop the integration if &lt;code&gt;q(2)&lt;/code&gt; is very close to xero but not exactly zero? (I am thinking about something like the Karnopp model used to model the sticking condition in this picture &lt;a href=&quot;http://vibrationacoustics.asmedigitalcollection.asme.org/data/journals/jvacek/28878/s_011506vaj1.jpeg&quot; rel=&quot;nofollow&quot;&gt;http://vibrationacoustics.asmedigitalcollection.asme.org/data/journals/jvacek/28878/s_011506vaj1.jpeg&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in the vector &lt;code&gt;te&lt;/code&gt; and &lt;code&gt;qe&lt;/code&gt; I have the time instants and the solution values when the events are encountered. If I have 2 different events to be satisfied (lets call them type 1 and type 2) but I want to plot only the events of type 1, how can I tell Matlab?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, what checks can I perform to be sure the ODE correctly identifies all the events?&lt;/p&gt;&#xA;" OwnerUserId="11676" LastEditorUserId="123" LastEditDate="2015-11-05T06:08:13.740" LastActivityDate="2015-11-05T06:08:13.740" Title="How can I use ODE events in MATLAB?" Tags="|matlab|ode|" AnswerCount="0" CommentCount="1" ClosedDate="2015-11-05T09:05:34.807" ContentLicense="CC BY-SA 3.0" />
  <row Id="21220" PostTypeId="2" ParentId="21214" CreationDate="2015-11-04T17:49:40.530" Score="1" Body="&lt;p&gt;There was a good series of lectures given by Jim Stone at a summer school held at IAS.  The links are here (Stone_Lecturex.pdf)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sns.ias.edu/pitp2/2009files/Lecture_Notes/&quot; rel=&quot;nofollow&quot;&gt;http://www.sns.ias.edu/pitp2/2009files/Lecture_Notes/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16978" LastActivityDate="2015-11-04T17:49:40.530" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21221" PostTypeId="2" ParentId="3315" CreationDate="2015-11-04T17:59:44.917" Score="0" Body="&lt;p&gt;We've been building &lt;a href=&quot;http://www.github.com/graphistry/pygraphistry&quot; rel=&quot;nofollow&quot;&gt;http://www.github.com/graphistry/pygraphistry&lt;/a&gt; to enable doing this from most browsers and notebooks. The idea is to use WebGL to render the big graphs (pan/zoom/etc.), and offload most of the real-time compute (layout, filter, etc.) to a GPU cloud. It's similar to Gephi or Cytoscape, but with more of a focus on big graphs and data analysis, and integrating into the web and notebooks.&lt;/p&gt;&#xA;" OwnerUserId="18129" LastEditorUserId="18129" LastEditDate="2015-11-04T18:06:29.267" LastActivityDate="2015-11-04T18:06:29.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21222" PostTypeId="2" ParentId="3315" CreationDate="2015-11-04T18:48:35.910" Score="0" Body="&lt;p&gt;You may try &quot;Tulip&quot; [1] , I think it can handle quite large graphs (at least I tryed it with 10K to 100K nodes and it worked well).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://tulip.labri.fr/TulipDrupal/&quot; rel=&quot;nofollow&quot;&gt;http://tulip.labri.fr/TulipDrupal/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2015-11-04T18:48:35.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21224" PostTypeId="1" AcceptedAnswerId="21225" CreationDate="2015-11-04T20:48:28.823" Score="2" ViewCount="769" Body="&lt;p&gt;Are there BDF formulas like the ones given &lt;a href=&quot;https://en.wikipedia.org/wiki/Backward_differentiation_formula&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; but one that can be used with implicit-explicit discretization? The right hand side in those formulas is supposed to be implicitly discretized at the current time step. Can I, for instance, replace that slope with multistage Runge-Kutta approximation of the slope and expect it to work? For example,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;BDF2:&#xA;$$\frac{3}{2}[y_n - \frac{4}{3} y_{n-1} + \frac{1}{3} y_{n-2}] = h[f(t_{n},y_{n})], $$&lt;/li&gt;&#xA;&lt;li&gt;Combined BDF2-RK: &#xA;$$\frac{3}{2}[y_n - \frac{4}{3} y_{n-1} + \frac{1}{3} y_{n-2}] = h\big[G\big(f(t_{n-1},y_{n-1}),...,f(t_{n-1}+\Delta t,y_{n-1}+\Delta y))\big],$$&#xA;where G does the Runge-Kutta approximation of the slope at the current time step.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, are there higher-order BDF formulas for second derivates? I could only find a first-order formula.&lt;/p&gt;&#xA;" OwnerUserId="18132" LastEditorUserId="713" LastEditDate="2015-11-04T21:25:12.080" LastActivityDate="2015-11-11T07:37:46.123" Title="BDF methods for implicit-explicit method" Tags="|ode|time-integration|runge-kutta|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21225" PostTypeId="2" ParentId="21224" CreationDate="2015-11-05T06:05:55.980" Score="2" Body="&lt;p&gt;IMEX methods based on BDF methods have been around for a long time; for a relatively recent discussion take a look at &lt;a href=&quot;http://dx.doi.org/10.1016/S0168-9274(97)00056-1&quot; rel=&quot;nofollow&quot;&gt;this paper by Ascher, Ruuth, &amp;amp; Spiteri&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are higher-order accurate BDF methods, but I don't think that is what you mean.  The term BDF is generally understood to refer to a class of methods for first-order differential equations.  If you are interested in multistep methods for second-order differential equations, take a look at Section III.10 of the text by Hairer, Norsett, &amp;amp; Wanner (vol. 1).  Another related but different class of methods are the &quot;second derivative BDF methods&quot;, which use both the first and second derivative; these are discussed in Section V.3 of the same book.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-11-11T07:37:46.123" LastActivityDate="2015-11-11T07:37:46.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21227" PostTypeId="1" CreationDate="2015-11-05T14:13:25.890" Score="0" ViewCount="348" Body="&lt;p&gt;I am trying to build my own simulator of Langevin Equation for the Brownian motion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to this &lt;a href=&quot;http://web.phys.ntnu.no/~ingves/Teaching/TFY4275/Downloads/kap6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;material&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way we calculate the particle position in certain time step is :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8YC96.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8YC96.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;W(u) is a Wiener process. x0, v0, tauB are all constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: How to write a c++ code for this Wiener process integral equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code that I currently used in the simulator.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N=1000;&#xA;tau=0.1;      //0.1s&#xA;t=N*tau;&#xA;tauB=m/gamma; //~ 1e-8s&#xA;&#xA;for(int j=0;j&amp;lt;=N;j++){  // t step&#xA;        sum=0;&#xA;    for(int k=0;k&amp;lt;=j;j++){  // u step&#xA;        sum=sum+(1-exp(-(j-k)*tau/tauB))*ND[j]*tau&#xA;    }&#xA;    x[j]=x0+v0*tauB(1-exp(-j*tau/tauB))+tauB/m*sum&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;x[j] is the particle position at time step j.&#xA;ND[j] is a Normal distribution random value at time step j.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;dW(t)=dU(t)=ND(t)dt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is incorrect.&#xA;There must be something important that I misunderstood in the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me.&lt;/p&gt;&#xA;" OwnerUserId="11614" LastEditorUserId="11614" LastEditDate="2015-11-05T15:11:29.693" LastActivityDate="2015-11-05T18:51:50.040" Title="How to discretize this integral equation? (Langevin Eq)" Tags="|discretization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21228" PostTypeId="2" ParentId="21227" CreationDate="2015-11-05T14:35:22.963" Score="1" Body="&lt;p&gt;The increments, $W(t_{k}) - W(t_{k-1})$, of the Brownian motion $W(t)$ are normally distributed with zero mean and variance equal to $t_{k} - t_{k-1}$.  If ND[j] is a normal random variable with zero mean and variance equal to one, then you want to multiply it by sqrt(tau).  The resulting ND[j] * sqrt(tau) is a normally distributed random variable with zero mean and variance equal to $t_{k} - t_{k-1}$, which is what you want.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2015-11-05T18:51:50.040" LastActivityDate="2015-11-05T18:51:50.040" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21230" PostTypeId="1" AcceptedAnswerId="21235" CreationDate="2015-11-05T16:53:31.293" Score="4" ViewCount="10554" Body="&lt;p&gt;I can write a function to find LCM (lowest common multiple) of an array of integers, but I thought it must have been implemented in numpy or scipy and was expecting something like &lt;code&gt;numpy.lcm()&lt;/code&gt; to do that. I'm surprised to find there is no such thing. Perhaps I'm searching in a wrong place. So, if you know any other library where this simple function is defined, please oblige.&lt;/p&gt;&#xA;" OwnerUserId="18143" LastActivityDate="2019-02-08T17:10:51.370" Title="LCM builtin in Python / Numpy" Tags="|python|scipy|numpy|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21231" PostTypeId="2" ParentId="21097" CreationDate="2015-11-05T17:27:44.297" Score="3" Body="&lt;p&gt;As said in the comments, it is not mathematically appropriate to constrain a single point value in the given BVP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest way to achieve something similar to this is to create small circular punctures into your domain and set the prescribed value at the boundary of those.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/34fUx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/34fUx.png&quot; alt=&quot;PDE Toolbox, prescribing values inside the domain&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With small enough circles the solution should be very close to the solution given by the Lagrange multiplier method proposed in the comments. Furthermore, it is probably easier for you to apply &lt;em&gt;adaptmesh&lt;/em&gt; with this approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wish to use the Lagrange multiplier method, you could implement the refinement iteration by yourself, compute error indicators and apply the function &lt;em&gt;refinemesh&lt;/em&gt; to the mesh iteratively.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2015-11-05T17:27:44.297" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21232" PostTypeId="2" ParentId="21177" CreationDate="2015-11-05T17:39:44.403" Score="3" Body="&lt;p&gt;For each element the so-called local mesh size can be defined e.g. as the maximum edge length. This has applications at least in residual a posteriori error analysis where the elementwise error indicator is often multiplied by the local mesh size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of computing the maximum edge length one can show that the Jacobian determinant of the affine mapping from the reference element to a global element is proportional to the square of the local mesh size. This is a property that I often use when programming adaptive mesh refinement using a posteriori estimators since the code for computing the Jacobian determinant is already there (assuming that the finite element code is implemented using reference elements).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the mesh is not uniform in the sense that for each pair of triangles their local mesh sizes can be bounded by each other (multiplied by some constant), then the so-called 'global mesh size'&amp;mdash;i.e. the maximum of local mesh sizes&amp;mdash;may not be a meaningful quantity. For example, in adaptive refinement one usually is concerned about the error versus number of degrees-of-freedom&amp;mdash;as indicated in the comments.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2015-11-05T17:39:44.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21234" PostTypeId="1" CreationDate="2015-11-05T18:56:43.197" Score="0" ViewCount="1081" Body="&lt;p&gt;I would like to code for a quadtree type meshing but don't know how to do. If anyone can help or can share any starting code?&lt;/p&gt;&#xA;" OwnerUserId="18144" LastActivityDate="2020-05-02T19:42:40.807" Title="Quadtree type Grid" Tags="|mesh-generation|multigrid|grid|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21235" PostTypeId="2" ParentId="21230" CreationDate="2015-11-05T19:17:39.080" Score="3" Body="&lt;p&gt;Why not define it yourself, as suggest in the Rosetta Code examples here:&#xA;&lt;a href=&quot;http://rosettacode.org/wiki/Least_common_multiple#Python&quot; rel=&quot;nofollow&quot;&gt;http://rosettacode.org/wiki/Least_common_multiple#Python&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you could build on this (via &lt;a href=&quot;https://docs.python.org/2/library/fractions.html#fractions.gcd&quot; rel=&quot;nofollow&quot;&gt;this library&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import fractions&#xA;def lcm(a,b): return abs(a * b) / fractions.gcd(a,b) if a and b else 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8094" LastActivityDate="2015-11-05T19:17:39.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21236" PostTypeId="1" AcceptedAnswerId="21254" CreationDate="2015-11-05T19:33:21.157" Score="5" ViewCount="75" Body="&lt;p&gt;I have a system of $N$ ODEs of the form,&#xA;$$&#xA;M(z,F(z)) \cdot F'(z) = \Phi(z,F(z))&#xA;$$&#xA;where the mass matrix is $M(z,F): R\times R^N \to R^{N\times N}$ and $\Phi(z,F):R\times R^N \to R^N$ is (potentially) nonlinear.  The initial condition is $F(0) = 0_N$.  I need to solve this for an arbitrarily large $z$, but it is stationary so I can choose some $z_{\max}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$M(z,F)$ is diagonal &lt;strong&gt;but&lt;/strong&gt; it is asymptotically singular (i.e., as $z\to\infty$, a few of the diagonals approach $0$).  This is why I left it in the mass matrix form, but I could just as easily write it as a $F'(z) = \Phi(z,F)$ form by dividing by the diagonal.&lt;/li&gt;&#xA;&lt;li&gt;This comes from a stationary Kolmogorov Forward equation in CDFs $F(z)$ (where the $N$ equations are an additional discrete state in the distribution, but I don't think that really matters here).  This leads to a few crucial requirements on the solution:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$F(0) = 0_N$ since the minimum of support of the distribution is at $0$&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Weak Positivity&lt;/strong&gt;: $F(z) \geq 0$ for all $z$ to be a valid CDF, where the only $0$ in practice will be at the initial condition.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Weak Monotonicity&lt;/strong&gt;: $F'(z) \geq 0$ for all $z$, since PDFs can't be negative.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Convergence&lt;/strong&gt;: $\lim_{z\to\infty}F(z) &amp;lt; \infty$ to ensure that this is a valid CDF. (In reality, $\lim_{z\to\infty}\sum_{n=1}^{N}F_n(z) = 1$, but I can deal with this through other means)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is this an IVP or a BVP?  Without explaining the details of the setup, the short answer is that I can choose whichever is more convenient and robust.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;QUESTION:&lt;/strong&gt;&#xA;What is the best numerical algorithm (or transformation of the ODE) to solve this problem?  In particular, I would love to maintain positivity and monotonicity, and I suspect that the convergence requirement is making the problem stiff as $z$ gets large (which could also be related to the singular mass matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am willing to sacrifice speed (and even accuracy) for robustness here, but would prefer to use an algorithm with off-the-shelf software.&lt;/p&gt;&#xA;" OwnerUserId="9692" LastEditorUserId="9692" LastEditDate="2015-11-05T19:59:37.330" LastActivityDate="2015-11-08T10:11:42.137" Title="Solving an ODE while maintaining weak positivity and weak monotonicity" Tags="|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21237" PostTypeId="2" ParentId="21097" CreationDate="2015-11-05T20:32:15.147" Score="2" Body="&lt;p&gt;Here is the Lagrange multiplier approach alluded to by Christian Clason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Structurally, I hope you agree that your problem can be put into the form,&#xA;\begin{align}&#xA;\text{argmin}_{u}&amp;amp;\quad \frac{1}{2}||Au - f||^2 \\&#xA;\text{such that}&amp;amp;\quad Bu=g,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$A$ is the PDE operator, &lt;/li&gt;&#xA;&lt;li&gt;$f$ is the PDE right hand side, &lt;/li&gt;&#xA;&lt;li&gt;$B$ is the &quot;observation operator&quot; that takes $u$ and observes it, returning the vector of values of the function at the points of interest, and&lt;/li&gt;&#xA;&lt;li&gt;$g$ is the list of values that the function $u$ is supposed to take at those points.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For elliptic problems like this, $A$ is symmetric and positive, so it has a square root. Thus one could instead replace the objective function with,&#xA;\begin{align}&#xA;\frac{1}{2}||Au-f||^2 &amp;amp;\rightarrow \frac{1}{2}||A^{1/2}u-A^{-1/2}f||^2 \\&#xA;&amp;amp;= \frac{1}{2} u^*Au - u^*f + \frac{1}{2}f^*A^{-1}f \\&#xA;\end{align}&#xA;But since $f^*A^{-1}f$ is a constant value independent of $u$, it doesn't effect the optimal point. Thus we can instead consider the equivalent optimization problem,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\text{argmin}_{u}&amp;amp;\quad \frac{1}{2}u^* A u - u^*f \\&#xA;\text{such that}&amp;amp;\quad Bu=g,&#xA;\end{align}&#xA;The Lagrangian for this problem is,&#xA;$$L = \frac{1}{2}u^* A u - u^*f + \lambda^*(Bu-g).$$&#xA;At the optimal point, the gradient of the Lagrangian must be zero, from which we get the following linear system to be solved,&#xA;$$0 = \begin{bmatrix}\nabla_u L \\ \nabla_\lambda L\end{bmatrix} = \begin{bmatrix}Au-f + B^*\lambda \\ Bu - g\end{bmatrix},$$&#xA;which is the same as the following &quot;KKT system&quot;,&#xA;$$\begin{bmatrix}A &amp;amp; B^* \\ B\end{bmatrix}\begin{bmatrix}u \\ \lambda\end{bmatrix} = \begin{bmatrix}f \\ g\end{bmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This KKT system can be solved directly, or it can be solved iteratively with Krylov methods that work for indefinite systems (e.g. MINRES). If you choose to go the iterative route, there is some &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.5.982&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;well-established theory&lt;/a&gt; by Murphy, Golub, and Wathen that shows the following preconditioner,&#xA;$$P = \begin{bmatrix}A \\ &amp;amp; B A^{-1}B^*\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;clusters the eigenvalues of the KKT matrix onto precisely 3 points (I.e., $\lambda(P^{-1} \text{KKT})$ is a set with 3 elements), and so using this preconditioner, Krylov methods like MINRES will converge in at most 3 iterations. You can then replace $A$ with a preconditioner $\tilde{A}$ (e.g., multigrid). Furthermore, the rank of $B A^{-1}B^*$ is simply the number of points of interest, so you can replace it with the identity and use the preconditioner&#xA;$$\tilde{P} = \begin{bmatrix}&#xA;\tilde{A} \\ &amp;amp; I&#xA;\end{bmatrix}$$&#xA;at the cost of taking a small number of extra number of Krylov iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in terms of discretizations of the problem with finite element methods, you will have to replace operators with their discrete versions, and put mass matrices in appropriate places, yielding the discrete KKT system,&#xA;$$\begin{bmatrix}K &amp;amp; B^T \\ B\end{bmatrix}\begin{bmatrix}u_h \\ \lambda\end{bmatrix} = \begin{bmatrix}M f_h \\ g\end{bmatrix},$$&#xA;and&#xA;$$\tilde{P} = \begin{bmatrix}\tilde{K} \\ &amp;amp; I\end{bmatrix}.$$&#xA;where $K,M$ are the stiffness and mass matrices, $u_h,f_h$ are the discrete versions of $u,f$ (e.g., vector of nodal values), $\tilde{K}$ is the discrete preconditioner, and $B$ is the observation operator that works on the discrete space. E.g., the action of each row of $B$ is to take some weighted linear combination of the nodal values for points in the mesh surrounding the observation node.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2015-11-05T20:38:58.407" LastActivityDate="2015-11-05T20:38:58.407" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21238" PostTypeId="2" ParentId="21230" CreationDate="2015-11-06T08:32:56.090" Score="1" Body="&lt;p&gt;If speed is an issue, you should look at this thread: &lt;a href=&quot;https://stackoverflow.com/questions/15569429/numpy-gcd-function&quot;&gt;https://stackoverflow.com/questions/15569429/numpy-gcd-function&lt;/a&gt; It explains how you can actually easely code a quicker version of &lt;code&gt;gcd&lt;/code&gt; (than the gcd from the &lt;code&gt;fractions&lt;/code&gt; module) in python and then of course a &lt;code&gt;lcm&lt;/code&gt; function from it as in the answer by blochwave.&lt;/p&gt;&#xA;" OwnerUserId="18093" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-11-06T08:32:56.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21239" PostTypeId="1" CreationDate="2015-11-06T10:17:48.563" Score="3" ViewCount="258" Body="&lt;p&gt;I am looking for a LAPACK routine that allows to find a non-trivial solution to the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;A x&lt;/em&gt;&lt;/strong&gt; = &lt;strong&gt;0&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;provided that &lt;strong&gt;&lt;em&gt;A&lt;/em&gt;&lt;/strong&gt; is a &lt;em&gt;n&lt;/em&gt;×&lt;em&gt;n&lt;/em&gt; square singular non-symmetric band matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality &lt;strong&gt;&lt;em&gt;A&lt;/em&gt;&lt;/strong&gt; matrix may not be exactly singular as it is based on some parameter and I use a root finding algorithm to find this parameter (requiring det(&lt;strong&gt;&lt;em&gt;A&lt;/em&gt;&lt;/strong&gt;) = 0, where the determinant is found with DGBTRF and multiplication of the diagonal elements).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only solution I have came up so far is to consider &lt;strong&gt;&lt;em&gt;A&lt;/em&gt;&lt;/strong&gt; a dense matrix matrix, use DGEEV to find its eigenvalues and eigenvectors and take the eigenvector for the eigenvalue closest to zero. However, I believe this is strongly sub-optimal approach. Can anyone suggest a better one?&lt;/p&gt;&#xA;" OwnerUserId="11694" LastActivityDate="2015-11-13T16:24:15.223" Title="Lapack routines for solving A x = 0" Tags="|lapack|" AnswerCount="2" CommentCount="1" ClosedDate="2015-11-14T18:14:51.990" ContentLicense="CC BY-SA 3.0" />
  <row Id="21240" PostTypeId="1" CreationDate="2015-11-06T12:56:09.950" Score="0" ViewCount="1829" Body="&lt;p&gt;I have to implement 'stress-free' boundary conditions for a stokes flow of a 2D-domain (for the left, bottom and right boundaries). The boundary condition are also described mathematically as $2 \eta \dot{\epsilon} = 0$. $\eta$ and $\dot{\epsilon}$ are the viscosity and strain rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I implement such boundary conditions? Would stress-free mean that for the left/right boundary the velocities are: $\frac{\delta v_x}{\delta x} = \frac{\delta v_y}{\delta x} = 0$? And for the bottom: $\frac{\delta v_x}{\delta y} = \frac{\delta v_y}{\delta y} = 0$?&lt;/p&gt;&#xA;" OwnerUserId="18151" LastEditorUserId="18151" LastEditDate="2015-11-06T13:01:41.283" LastActivityDate="2016-01-10T11:53:40.390" Title="How to implement 'stress-free' boundary conditions (stokes flow)?" Tags="|boundary-conditions|navier-stokes|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21241" PostTypeId="1" CreationDate="2015-11-06T15:23:53.453" Score="1" ViewCount="13162" Body="&lt;p&gt;I am looking for a C/C++ implementation of the Hungarian method for solving the linear assignment problem with real-valued cost matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some implementation I found, such as &lt;a href=&quot;http://www2.informatik.uni-freiburg.de/~stachnis/misc.html&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt;, only work for integer costs :(&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions are very appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2017-03-29T20:45:12.290" Title="Looking for a C/C++ implementation of the Hungarian method for real-valued cost matrix" Tags="|optimization|c++|combinatorics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21243" PostTypeId="2" ParentId="21085" CreationDate="2015-11-06T16:54:05.503" Score="2" Body="&lt;p&gt;If your question is &quot;do people sometimes use the $2$-norm to measure distances without full justification?,&quot; then the answer is probably yes. Choosing the correct norm to measure differences between high-dimensional objects is often not simple to decide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One motivation for using the $2$-norm to measure distances between vectors in $\mathbb{R}^n$ is that it is invariant under Euclidean/rigid transformations. If you have some reason to want your algorithm to also be invariant under those transformations, then it is a natural choice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also reasons to use the $1$-norm (often as a convexification of the $0$-&quot;norm&quot; which counts the nonzero entries, if you care about sparsity). See Geoff Oxbury's answer for more on this.&lt;/p&gt;&#xA;" OwnerUserId="3824" LastActivityDate="2015-11-06T16:54:05.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21245" PostTypeId="2" ParentId="7403" CreationDate="2015-11-06T20:34:53.877" Score="9" Body="&lt;p&gt;I'm going against the crowd - the adjugate matrix is in fact very useful for some specialty applications with small dimensionality (like four or less), in particular when you need the inverse of a matrix but don't care about scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two examples include computation of an inverse &lt;a href=&quot;https://en.wikipedia.org/wiki/Homography&quot;&gt;homography&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Rayleigh_quotient_iteration&quot;&gt;Rayleigh quotient iteration&lt;/a&gt; for very small problems (which in addition to being simplified by use of adjugate is numerically better).&lt;/p&gt;&#xA;" OwnerUserId="18155" LastActivityDate="2015-11-06T20:34:53.877" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21246" PostTypeId="2" ParentId="15973" CreationDate="2015-11-06T23:26:07.153" Score="8" Body="&lt;p&gt;Kirill's &lt;a href=&quot;https://scicomp.stackexchange.com/a/15978&quot;&gt;answer&lt;/a&gt; is a good method but it is hampered that the fact that you need to tune it to a specific range in energy, so it is not appropriate if you have a broad-band wavepacket you need to absorb.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;An alternative approach is &lt;strong&gt;exterior complex scaling&lt;/strong&gt;, which is reviewed well in&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Infinite-range exterior complex scaling as a perfect absorber in time-dependent problems. A. Scrinzi. &lt;a href=&quot;http://dx.doi.org/10.1103/PhysRevA.81.053845&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Phys. Rev. A&lt;/em&gt; &lt;strong&gt;81&lt;/strong&gt; no. 5, 053845 (2010)&lt;/a&gt;, &lt;a href=&quot;http://arxiv.org/abs/1002.2520&quot; rel=&quot;nofollow noreferrer&quot;&gt;arXiv:1002.2520&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The idea is to solve the Schrödinger equation not for $\psi(x)$ but for its analytical continuation, branching off from the real line into the complex plane, at an angle, after all the interesting stuff is done. Thus you solve for $\xi\equiv x\in\mathbb R$ if $x&amp;lt;R$, but instead of a real $x&amp;gt;R$ you change to $x=e^{i\theta}(\xi-R)+R$. The advantage of this is that plane waves of the form $\psi(x)=e^{ikx}$ will change into&#xA;$$&#xA;\psi(\xi)&#xA;=\exp(ik(e^{i\theta}(\xi-R)+R))&#xA;=e^{ikR}e^{ik\cos(\theta)(\xi-R)}&#xA;\exp(-k\sin(\theta)(\xi-R)),&#xA;$$&#xA;with an exponentially decreasing amplitude regardless of the value of $k$. For more details, see Scrinzi's paper (which deals with a specific discretization scheme for this method) and references therein.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Another method that can work is a &lt;strong&gt;complex absorbing potential&lt;/strong&gt;, of the form $-i\eta x^n$. This is described in e.g.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Investigation on the reflection and transmission properties of complex absorbing potentials. U.V. Riss and H.‐D. Meyer. &lt;a href=&quot;http://dx.doi.org/10.1063/1.472003&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;J. Chem. Phys.&lt;/em&gt; &lt;strong&gt;105&lt;/strong&gt; no. 4, 1409 (1996)&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and as I see it the main ingredient is having a slightly non-hermitian hamiltonian such that $e^{i Ht}$ will tend to decrease the norm of the wavefunction - and choosing the anti-hermitian part $H-H^\dagger$ cleverly enough that only the parts of the wavefunction on the outside are killed off, without introducing discontinuities which would themselves cause reflections. However, this is not usually perfect and can require a fair bit of fiddling and tuning to get it to work well.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A third method, known as the use of an &lt;strong&gt;absorbing mask&lt;/strong&gt;, is relatively crude: take a 'buffer' region just before the boundary and, at every timestep, multiply your wavefunction by a factor which goes to zero at the boundary. There's a bunch of such masks out there and - at a very rough glance - there don't seem to be any specific choices that stick out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You do want your mask to go down smoothly to zero, or otherwise you're essentially imposing a Dirichlet boundary condition and that will generate reflections. Beyond that, you need to work your specific problem into something that will work for you - which is to say, the method is case-by-case and fairly fiddly to get the reflection coefficient to sufficiently low values.&lt;/p&gt;&#xA;" OwnerUserId="3551" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2015-11-06T23:26:07.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21247" PostTypeId="1" CreationDate="2015-11-07T04:13:30.897" Score="4" ViewCount="2838" Body="&lt;p&gt;This is more of a mathematical question but since we deal with this all the time in computational science, maybe it is relevant in this forum too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am an engineer and I am learning how to model the wave equation in Fenics. To obtain the fully discretised variational form in space and time, I have to implement a numerical method for evolution in time. I have been looking at Sympletic Euler, Crank Nicolson and Newmark Beta method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In literature, it is reported that these methods are conditionally or unconditionally stable for certain values. Focussing on Newmark Beta method, it is reported that this method is unconditionally stable, if we choose the parameters 2β ≥ γ ≥ 1/2. Here beta and gamma are parameters in the Newmark scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: How do mathematicians derive this information about the stability of a method. Whether it is conditionally stable or unconditionally stable, and for which parameter values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone kindly point me in the right direction/ document/ papers? I am not researching this topic so I don't want to dive too deep into the associated mathematical research. I am just curious if there are plain derivations for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2016-05-14T04:06:00.680" Title="How to derive the stability of time stepping schemes?" Tags="|numerical-analysis|hyperbolic-pde|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21248" PostTypeId="1" CreationDate="2015-11-07T04:51:58.030" Score="1" ViewCount="108" Body="&lt;p&gt;I have the following system of equations obtained by implementing Sympletic Euler time scheme to wave equation. I want to model this in Fenics. Here 'u' is the displacement and 'p' is corresponding velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \dot{u} = p $$&#xA;$$ u(x, 0) = u_i$$&#xA;$$ p(x, 0) = 0$$&#xA;$$ u(x, t) = 0 \quad \text{on} \quad \partial \Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t9PYo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t9PYo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read in a documentation that the velocity 'p' should be obtained first and thereafter 'u' should be computed. Also, that it must be ensured that we never have to explicitly compute and store the inverse of the mass matrix M. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $M = uvdx$ and $K = \nabla u \nabla v dx$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the correct way to implement the above system of equations? Can anyone kindly guide on how this can be corrected / refined.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u = interpolate(ui,V)           &#xA;V = FunctionSpace(mesh,&quot;CG&quot;,2)&#xA;&#xA;u_p = function(V)&#xA;u_new = function(V)&#xA;p = interpolate(Constant(0.0),V)&#xA;&#xA;while t =&amp;lt; T&#xA;    bc.apply(M,u)        # apply boundary conditions    &#xA;    solve(M, u_p.vector(), u.vector)    &#xA;    p_new = p.vector() - dt*K*u_p.vector() &#xA;    t += dt              # solve for next timestep&#xA;    u_new.vector()[:] = u.vector() + dt*p_new.vector()&#xA;    u.assign(u_new)      # update old vector&#xA;    p.vector()[:] = p_new&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2015-11-07T04:51:58.030" Title="Need suggestions on how to implement this time stepping for wave equation" Tags="|finite-difference|numerical-analysis|python|fenics|wave-propagation|" AnswerCount="0" CommentCount="1" ClosedDate="2019-06-06T20:36:31.933" ContentLicense="CC BY-SA 3.0" />
  <row Id="21249" PostTypeId="2" ParentId="21247" CreationDate="2015-11-07T04:53:25.157" Score="3" Body="&lt;p&gt;One way is via &lt;a href=&quot;https://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&quot; rel=&quot;nofollow&quot;&gt;von Neumann stability analysis&lt;/a&gt;.  This technique hinges on the fact that &lt;a href=&quot;https://en.wikipedia.org/wiki/Plancherel_theorem&quot; rel=&quot;nofollow&quot;&gt;the Fourier transform can sometimes be regarded as an $L^2$ isometry&lt;/a&gt;.  This and other techniques such as direct norm estimates are discussed in, for example, the first few chapters of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strikwerda, J. C. (2004). Finite difference schemes and partial differential equations (Second edition). Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM).&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2015-11-07T04:53:25.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21252" PostTypeId="1" AcceptedAnswerId="21257" CreationDate="2015-11-08T00:52:06.930" Score="2" ViewCount="69" Body="&lt;p&gt;So I have two versions of a hydrodynamic code that has the same underlying physics. Lets call them code A and B. However code B is more optimized and more object oriented. I was trying to compare the numerical results of both codes and see if they seem compatible. The way I'm doing this is running both codes with the same initial parameters and calculating a particular hydrodynamic value X. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;X is in the order of magnitude of 1E6. To compare both codes I'm essentially doing &#xA;$$X(A)-X(B)= \epsilon,$$&#xA;where $\epsilon$ is some error. The test is giving $\epsilon=10^{-2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would mean that both codes spit out the same X for the first 8 digits. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How significant is this error? I'm using double precision for non-integers, would such a magnitude simply imply a cumulative round off error for the thousands of calculations involved in a hydro code? Or a fundamental difference I'm missing?&lt;/p&gt;&#xA;" OwnerUserId="17075" LastEditorUserId="713" LastEditDate="2015-11-08T01:10:17.613" LastActivityDate="2015-11-08T18:38:30.587" Title="Comparing two versions of the same hydrodynamic code and their error" Tags="|floating-point|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21254" PostTypeId="2" ParentId="21236" CreationDate="2015-11-08T10:11:42.137" Score="7" Body="&lt;p&gt;I assume that the exact solution satisfies the positivity and monotonicity conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most robust method that will guarantee these properties is the Backward Euler method.  It will maintain positivity and monotonicity under arbitrarily large step sizes.  Indeed, if the exact solution satisfies $F(z)\le 1$, Backward Euler will preserve that too.  It is only first-order accurate, so you may need to take small step sizes anyway for the sake of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For arbitrary nonlinear $\Phi$, it is known that &lt;a href=&quot;http://pub.math.leidenuniv.nl/~spijkermn/PUBLICATIONS-GENERAL/Spijker(1983)-Numer-Math-42-271-290.pdf&quot;&gt;there exists no 2nd-order method that preserves positivity under arbitrary step sizes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can show that positivity is preserved under a forward Euler step (with some upper limit on the step size, obviously) then the class of Strong stability preserving (SSP) methods can be used to get higher-order accuracy, but you will have to deal with a restriction on the time step size, and you are likely to lose the high order accuracy anyway due to the asymptotic singularity.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-11-08T10:11:42.137" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21255" PostTypeId="1" AcceptedAnswerId="21256" CreationDate="2015-11-08T17:16:14.560" Score="2" ViewCount="78" Body="&lt;p&gt;Let's say I am solving a simple Poisson problem using a Mixed (DG) finite element method.  If we use orthogonal polynomials as basis functions we can write the finite-dimensional linear system as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\lbrack\begin{array}{cc}I_1&amp;amp; S_{11} \\&#xA;-S_{11}^\mathrm{T}  \end{array} \right\rbrack \left\lbrack\begin{array}{c} q_1\\u_1 \end{array} \right\rbrack = \left\lbrack \begin{array}{c} f_1\\g_1 \end{array} \right\rbrack,&#xA;$$&#xA; for appropriately sized matrices $S_{11},I_1$ and right hand sides $f_1$ and $g_1$, Suppose I solve this linear system and obtain solutions $\hat{q}_1$ and $\hat{u}_1$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider the same underlying PDE with an enriched basis, polynomials of one degree higher, say.   This will result in a new linear system with the following structure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\lbrack&#xA;  \begin{array}{cc}&#xA;    I_1&amp;amp; 0&amp;amp; S_{11}&amp;amp; S_{12}\\&#xA;    0&amp;amp;I_2&amp;amp; S_{21}&amp;amp;S_{22}\\&#xA;    -S_{11}^\mathrm{T} &amp;amp; -S_{21}^{\mathrm{T}} &amp;amp; 0 &amp;amp;0\\&#xA;    -S_{12}^\mathrm{T} &amp;amp; -S_{22}^\mathrm{T}&amp;amp; 0 &amp;amp; 0\\ &#xA;  \end{array} &#xA;\right\rbrack&#xA;\left\lbrack\begin{array}{cc}q_1\\q_2\\u_1\\u_2 \end{array} \right\rbrack=&#xA;\left\lbrack\begin{array}{cc}f_1\\f_2\\g_1\\g_2 \end{array} \right\rbrack,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the vectors $q_2$ and $u_2$ contain all the new degrees of freedom.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question are as follows,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can we use the solution to the smaller problem to construct the solution of the larger problem?&lt;/li&gt;&#xA;&lt;li&gt;Recursion: Suppose we have a fast way to compute the solution of the smaller problem (for all right hand sides) how can we use this method to find solution of the larger problem. I have a vague idea of some sort of preconditioner along the lines of multigrid.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Notes: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can rewrite the large linear system as&#xA;$$&#xA;\left\lbrack&#xA;  \begin{array}{cc}&#xA;    I_1&amp;amp; 0&amp;amp; S_{11}&amp;amp; S_{12}\\&#xA;    0&amp;amp;I_2&amp;amp; S_{21}&amp;amp;S_{22}\\&#xA;    -S_{11}^\mathrm{T} &amp;amp; -S_{21}^{\mathrm{T}} &amp;amp; 0 &amp;amp;0\\&#xA;    -S_{12}^\mathrm{T} &amp;amp; -S_{22}^\mathrm{T}&amp;amp; 0 &amp;amp; 0\\ &#xA;  \end{array} &#xA;\right\rbrack&#xA;\left\lbrack\begin{array}{cc}q_1-\hat{q}_1\\ q_2\\ u_1-\hat{u}_1\\ u_2\\ \end{array} \right\rbrack =&#xA;\left\lbrack \begin{array}{c}0\\f_2\\0\\g_2 \end{array} \right\rbrack,&#xA;$$ and after shuffling the order of the equations and unknowns get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\lbrack \begin{array}{cccc}&#xA;I_1&amp;amp; S_{11}&amp;amp;0&amp;amp; S_{12}\\&#xA;-S_{11}^\mathrm{T}&amp;amp; 0 &amp;amp;-S_{21}^\mathrm{T}&amp;amp; 0\\&#xA;0&amp;amp; S_{21} &amp;amp;I_2&amp;amp;S_{22}\\&#xA;-S_{12}^\mathrm{T}&amp;amp;0 &amp;amp; -S_{22}^\mathrm{T} &amp;amp;0&#xA;\end{array} \right\rbrack&#xA;\left\lbrack \begin{array}{c} q_1-\hat{q}_1\\ u_1-\hat{u}_1\\ q_2\\ u_2\end{array} \right\rbrack =&#xA;\left\lbrack\begin{array}{c}0\\ 0\\ f_2\\g_2 \end{array} \right\rbrack.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing to notice here is that the original linear system appears in the upper left hand corner and the (presumably smaller) related linear system appears in the bottom right hand corner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $I$ matrices are really mass matrices, but since we chose orthogonal polynomials as the basis functions they are diagonal and therefore trivial to invert&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am especially interested in links to the literature or a name of a technique like this that I can use to search the literature.   For the first question I know that we could use previous solution as a guess for an iterative solver but I am looking for something more clever than this.&lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2015-11-09T16:18:51.773" LastActivityDate="2015-11-09T16:18:51.773" Title="Solving new linear system that comes from an $p$ enrichment" Tags="|linear-algebra|linear-solver|poisson|multigrid|spectral-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21256" PostTypeId="2" ParentId="21255" CreationDate="2015-11-08T18:31:57.140" Score="2" Body="&lt;p&gt;What you are thinking of is something that uses the structure of the augmented matrix to make solution of the system simpler. For example, one could be tempted to think of forming the Schur complement with regard to the bottom right $2\times 2$ block of the rewritten system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't think anything like this exists. If it would, then you would have discovered a purely algebraic way to efficiently solve the linear systems that arise from high order discretizations. We don't have such algorithms, though: the best algorithms we have use multigrid, but these algorithms make very specific use of the properties and entries of the matrices we are trying to invert, as opposed to just their algebraic structure. In other words, I don't think that by just reordering terms you will be able to come up with an efficient solver.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-11-08T18:31:57.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21257" PostTypeId="2" ParentId="21252" CreationDate="2015-11-08T18:38:30.587" Score="4" Body="&lt;p&gt;If two different codes agree to 8 digits on some non-trivial quantity, then that is far better than you'd expect in almost all cases. However, there are some cases where the quantity $X$ is poorly chosen, and consequently not indicative. For example, if you considered the total mass in a simulation, then two codes that are both conservative might give the same answer, but differ in almost every other aspect of the solutions they produce. In other words, the choice of $X$ matters. But if your answers are the same to 8 digits, then you're more than likely on the right track with both codes (or they both contain the exact same bug).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-11-08T18:38:30.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21258" PostTypeId="2" ParentId="21240" CreationDate="2015-11-08T18:49:13.980" Score="1" Body="&lt;p&gt;My take on boundary conditions for the Stokes equations are given &lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/step_22.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. It also links to some video lectures that describe the procedure by which you arrive at it. The thing to note is that &quot;stress-free&quot; also involves the pressure.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-11-08T18:49:13.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21261" PostTypeId="1" CreationDate="2015-11-09T12:23:48.487" Score="4" ViewCount="3230" Body="&lt;p&gt;I'm working on an algorithm which should check if two polygons, described by their vertex coordinates, are: one inside the other, are intersecting or are separated&lt;/p&gt;&#xA;&#xA;&lt;p&gt;image below describe this three cases: &lt;a href=&quot;https://i.stack.imgur.com/hl6zL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hl6zL.png&quot; alt=&quot;example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i'm thinking about how to do it but i'm not having any idea. Any suggestion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;solved:&lt;/strong&gt;&#xA;i solved using Shamos-Hoey Algorithm and Winding Number method&lt;/p&gt;&#xA;" OwnerUserId="18171" LastEditorUserId="20417" LastEditDate="2016-10-01T15:39:09.307" LastActivityDate="2016-10-01T15:39:09.307" Title="Algorithm to determine if two polygons intersect" Tags="|computational-geometry|geometry|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21262" PostTypeId="1" AcceptedAnswerId="21268" CreationDate="2015-11-09T17:33:51.667" Score="0" ViewCount="144" Body="&lt;p&gt;MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to vectorize a loop in which each column vector of a 2D matrix (n-by-n) is found by multiplying each single element in a diagonal matrix  with a column vector in another n-by-n 2D matrix. It seems like it would be simple to vectorize, but I must be missing something.  I tried converting the diagonal matrix to a column vector first, but it still needs to do the operation one element at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% example matrix&#xA;B = [ 3     2      .9    2&#xA;      2      4       1    2&#xA;      3      4      -1    0&#xA;     .5     .5      .1    1];&#xA;&#xA;%% find eigenvectors and eigenvalues&#xA; [ve, va] = eig(B,'nobalance');&#xA;&#xA;%% get size of B ... 4&#xA;    [~,q]=size(B);&#xA;&#xA;%% time constants&#xA;    t0=0;&#xA;    t = 2000e-6;&#xA;&#xA;%% pre-allocate memory for Matrix Mn (Mnew) and Mp (Mprime)&#xA;    Mn=zeros(q,1);&#xA;    Mp=Mn;&#xA;&#xA;%% Original code, runs correctly, not vectorized - steps through each column&#xA;for m = 1:q,&#xA;    Mn(:,m) = exp( va(m,m) * t0) * ve(:,m);             &#xA;    Mp(:,m) = exp( va(m,m) * t ) * ve(:,m);            &#xA;end&#xA;&#xA;% quick explanation ^ for each column of Mn and Mp, a single value&#xA;% exp(va(i,i)*t) is multiplied by a column of ve(:,i)&#xA;% vectorizable?&#xA;% Mn(column) = A * ve(column), but A is an individual element in a different vector already&#xA;&#xA;%% New code... vectorized?&#xA;% random starting idea:&#xA;va1 = va * [1; 1; 1; 1];  % changes diagonal matrix into column vector of values, BUT still has to calculate one at a time...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18173" LastActivityDate="2015-11-10T01:51:34.410" Title="Matlab Vectorization of columns in a 2D matrix and single element multiplication" Tags="|matlab|matrix|vectorization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21263" PostTypeId="2" ParentId="21262" CreationDate="2015-11-09T20:30:15.050" Score="0" Body="&lt;p&gt;The results of the following codes are equivalent to the results of &lt;code&gt;for loop&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mp=(exp( diag(va) * t ))' .* ve;&#xA;Mn=(exp( diag(va) * t0 ))' .* ve;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2015-11-09T20:30:15.050" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21264" PostTypeId="2" ParentId="14384" CreationDate="2015-11-09T22:12:12.600" Score="1" Body="&lt;p&gt;This is a path finding problem, but much more complicated than the usual case where only the shortest path is sought.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't seen a problem similar to yours and can't answer whether or not there's a canned solution, or if it's similar enough to one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without the constraint this looks like something dynamic programming could help with, however the constraint is just so unusual (yet its importance so basic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To actually give you &lt;em&gt;some&lt;/em&gt; answer, after ruling out exact solutions I'd run a heuristic; I've worked on similar problems and have had good success with &lt;a href=&quot;https://en.wikipedia.org/wiki/2-opt&quot; rel=&quot;nofollow&quot;&gt;k-opt&lt;/a&gt;. Wikipedia doesn't explain it terribly well; basically, what you do is initialize your solution some way or another (random works well enough, but you could use weighted &lt;a href=&quot;https://en.wikipedia.org/wiki/A*_search_algorithm&quot; rel=&quot;nofollow&quot;&gt;A*&lt;/a&gt; or something without constraint to do better), fix all elements except for a few (2 - 4), and exhaustively search for an improvement in those few. The complexity is worse than linear but much better than brute force.&lt;/p&gt;&#xA;" OwnerUserId="18155" LastActivityDate="2015-11-09T22:12:12.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21266" PostTypeId="1" CreationDate="2015-11-10T00:36:43.280" Score="2" ViewCount="137" Body="&lt;p&gt;Am solving a system of pdes in Fenics and need to write the variational form. Everything looks good expect a term like the following&#xA;$$ \nabla \xi\nabla \cdot U $$&#xA;$\mathbf{U} = (U,V) $ is a vector function and $\xi$ is a scalar function in 2d.&#xA;So I tried the following. Multiply by a vector function $p$ and integrate by part I arrived at the following, assuming that $p$ vanishes on the boundary.&#xA;$$-\int (\xi \nabla \cdot \mathbf{U})\cdot\nabla p\space dx .$$&#xA;But this doesn't look right. Can anybody please help with this. Any help is much appreciated. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="18169" LastEditorUserId="549" LastEditDate="2015-11-10T20:12:51.927" LastActivityDate="2016-04-16T22:37:10.627" Title="Help with Variational Formulation" Tags="|finite-element|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21268" PostTypeId="2" ParentId="21262" CreationDate="2015-11-10T01:51:34.410" Score="0" Body="&lt;p&gt;First you want the eigenvalues as a vector, i.e.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;va = diag(va);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or just call &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/eig.html&quot; rel=&quot;nofollow&quot;&gt;eig&lt;/a&gt; with an output option:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ve, va] = eig(B,'nobalance','vector');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then to multiply the columns you can use either diagonal matrix multiplication&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mp = ve*diag(exp(va*t));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or bsxfun:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mp = bsxfun(@times,ve,exp(va*t)');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2015-11-10T01:51:34.410" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21269" PostTypeId="2" ParentId="21234" CreationDate="2015-11-10T03:24:53.827" Score="2" Body="&lt;p&gt;Writing C++ code from the ground up for adaptive mesh refinement (as part of a PDE solver) is a relatively complicated endeavor and can easily involve thousands of lines of code for even simple problems. Based on the comments it sounds like you may be fairly new to programming (for example unsure how to implement a binary tree). Of course there is nothing wrong with being new at something - we all were at one point - but I think it is important to understand that this is a pretty large project to take on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of resources that would be useful for you, I would &lt;strong&gt;highly&lt;/strong&gt; recommend taking a look at Wolfgang Bangerth's video tutorials &lt;a href=&quot;https://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. These videos actually run through how finite element codes are developed from the ground up. Even though they are centred around finite elements (and not finite differences like you are doing), they contain a lot of valuable information related to adaptive mesh refinement and PDE solver codes in general. Beyond this any good C++ book can teach you about tree data structures...and there is also stackoverflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mentioned the mesh generating tool called Gmsh in one of my comments and I highly recommend downloading for a number of reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Firstly it is easy to use and even has a GUI and can be used on Windows, Linux, and probably Mac (haven't checked Mac personally).&lt;/li&gt;&#xA;&lt;li&gt;It will give you an idea how meshes are stored and what information is important in a mesh.&lt;/li&gt;&#xA;&lt;li&gt;It can create 1D, 2D, and 3D meshes using triangles, quadrilaterals, tetrahedral, ....etc&lt;/li&gt;&#xA;&lt;li&gt;One strategy that you could use in your code is to start with a coarse mesh that you create using Gmsh, and then have this as an input to your program. Your program then performs adaptive mesh refinement on this coarse mesh while solving your PDE. Doing this has the benefit of simplifying your mesh generating process since you wont have to worry about how to generate an initial mesh on a possibly complicated geometry.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now as for your specific question about quadtree data structures, what I would do is create a template QuadTree class in C++. For example the &lt;strong&gt;incomplete&lt;/strong&gt; code for this might look something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#ifndef QUADTREE_H&#xA;#define QUADTREE_H&#xA;&#xA;template&amp;lt;class T&amp;gt;&#xA;class Node&#xA;{&#xA;  public:&#xA;    T object;&#xA;    Node *parent;&#xA;    Node *topLeft;&#xA;    Node *topRight;&#xA;    Node *bottomLeft;&#xA;    Node *bottomRight;&#xA;&#xA;    Node(T obj) : object(obj), parent(NULL), topLeft(NULL), topRight(NULL), bottomLeft(NULL), bottomRight(NULL)&#xA;    {&#xA;    }&#xA;&#xA;    T* get_T();&#xA;    bool isLeaf();&#xA;    //   .&#xA;    //   .&#xA;    // more methods etc&#xA;};&#xA;&#xA;&#xA;template&amp;lt;class T&amp;gt;&#xA;class QuadTree&#xA;{&#xA;  private:&#xA;    int numNodes;      //number of node in tree&#xA;    int numLeafNodes   //number of leaf nodes in tree&#xA;    Node&amp;lt;T&amp;gt; *root;&#xA;    Node&amp;lt;T&amp;gt; *iterator;&#xA;&#xA;  public:&#xA;    QuadTree();&#xA;    QuadTree(T obj);&#xA;&#xA;    void deleteQuadTree(Node&amp;lt;T&amp;gt; *leaf);&#xA;    void setTopLeft(T obj);&#xA;    void setTopRight(T obj);&#xA;    void setBottomLeft(T obj);&#xA;    void setBottomRight(T obj);&#xA;    Node&amp;lt;T&amp;gt;* get_Node();&#xA;    T* get_T();&#xA;    bool isLeaf();&#xA;    //   .&#xA;    //   .&#xA;    //   more methods etc&#xA;};&#xA;&#xA;&#xA;//---------------------------------------------------------&#xA;// writing out all your implementations for each constructor, method, etc&#xA;//---------------------------------------------------------&#xA;//  .&#xA;//  .&#xA;//  .&#xA;&#xA;#endif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By templating this QuadTree we can use it on simple integers (great for testing!) or complicated objects (...that might store information on your quadrilateral elements for example...). I hope this answer is of some help. I may be able to give more suggestions if you have specific questions about what I have written if needed. Your question was a little broad so I wasn't entirely sure where to start but good luck with your project.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="35788" LastEditDate="2020-05-02T19:42:40.807" LastActivityDate="2020-05-02T19:42:40.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21270" PostTypeId="2" ParentId="21266" CreationDate="2015-11-10T09:22:58.033" Score="4" Body="&lt;p&gt;Multiplying term by a vector test function $\mathbf{p}$ and integrating over the domain $\Omega$ gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega \nabla\cdot \mathbf{U}\nabla\xi\cdot\mathbf{p}d\Omega\qquad (1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start with you can write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla\cdot(\mathbf{p}\xi) = \xi\nabla\cdot\mathbf{p} + \mathbf{p}\cdot\nabla\xi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{p}\cdot\nabla\xi = \nabla\cdot(\mathbf{p}\xi) - \xi\nabla\cdot\mathbf{p}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plugging this into (1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega \nabla\cdot \mathbf{U}\left(\nabla\cdot(\mathbf{p}\xi) - \xi\nabla\cdot\mathbf{p}\right) d\Omega\qquad (2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall the Divergence theorem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \nabla\cdot \mathbf{F} d\Omega = \int_\Gamma \mathbf{F}\cdot\mathbf{n}d\Gamma$$&#xA;where $\Gamma$ is the boundary of $\Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying this to the first term in (2) gives the final form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Gamma \nabla\cdot \mathbf{U} \xi(\mathbf{p}\cdot\mathbf{n})d\Gamma - \int_\Omega \nabla\cdot \mathbf{U} \xi\nabla\cdot\mathbf{p}d\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first term is 0 if $\mathbf{p}$ vanishes on the boundary.&lt;/p&gt;&#xA;" OwnerUserId="9728" LastActivityDate="2015-11-10T09:22:58.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21271" PostTypeId="1" CreationDate="2015-11-10T09:42:59.893" Score="1" ViewCount="725" Body="&lt;p&gt;I am interested to know the significance of the energy conservation laws when modelling fluids (or other materials). Am I correct in saying that if energy is conserved then stability is achieved. &#xA;In a way, I need to know why it is important to show that energy is conserved when modelling different phenomena (which can be described by using mass and momentum equation, without the need for the energy equation)&#xA;If you know any reading materials please let me know. &lt;/p&gt;&#xA;" OwnerUserId="17362" LastEditorUserId="276" LastEditDate="2015-11-10T10:28:38.467" LastActivityDate="2015-11-10T15:56:18.830" Title="significance of energy equation and its conservation" Tags="|fluid-dynamics|stability|conservation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21272" PostTypeId="2" ParentId="21271" CreationDate="2015-11-10T10:53:05.573" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I am interested to know the significance of the energy conservation laws when modelling fluids (or other materials).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Energy conservation is usually explicitly modeled with a differential equation when modeling temperature or internal energy matters; the main examples I can think of are compressible flow applications and flows with thermic chemical reactions (like combustion).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Am I correct in saying that if energy is conserved then stability is achieved.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I assume you mean numerical stability? This property typically refers to numerical methods; if a numerical method is convergent, then it is stable. So, assuming you are using convergent numerical methods for your problem, and you assess convergence via calculating the energy, then you could reasonably deduce that your method is stable for your problem if energy is conserved.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In a way, I need to know why it is important to show that energy is conserved when modelling different phenomena (which can be described by using mass and momentum equation, without the need for the energy equation).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If changes in fluid temperature or internal energy are negligible, then the only changes in energy occur due to changes in pressure, density, velocity, or height; so you're computing the energy via Bernoulli's principle. Computing energy in this fashion would help validate numerical results, although there are better methods available (the method of manufactured solutions, for instance). &lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2015-11-10T10:53:05.573" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21274" PostTypeId="2" ParentId="21271" CreationDate="2015-11-10T15:56:18.830" Score="2" Body="&lt;p&gt;Sometimes the energy equation drives the flow. Problems like natural convection are driven by temperature differences. The energy equation is frequently converted into a temperature advection-diffusion equation which can be coupled to the momentum equation through the &lt;a href=&quot;https://en.wikipedia.org/wiki/Boussinesq_approximation_%28buoyancy%29&quot; rel=&quot;nofollow&quot;&gt;Boussinesq approximation&lt;/a&gt;. If there is a heat source or variation in the temperature in the boundary conditions, then solving the energy equation too is the only way to incorporate that information into the overall problem. In compressible flow problems where shocks can create jumps in temperature, it is essential to include the energy equation to have a hope of getting the physics right. In these problems, all of the flow variables end up coupled through the conservation equations and the equation of state for the fluid being modeled.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-11-10T15:56:18.830" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21275" PostTypeId="1" CreationDate="2015-11-10T16:25:59.883" Score="0" ViewCount="968" Body="&lt;p&gt;I have data that I would like to fit with an ellipsoid and I am currently fitting it via the following Matlab commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xs = pts(:,1);&#xA;ys = pts(:,2);&#xA;zs = pts(:,3);&#xA;&#xA;A0 = [xs.^2 ys.^2 zs.^2 xs.*ys ys.*zs xs.*zs xs ys zs ones(size(xs))];&#xA;B0 = [ones(size(xs))];&#xA;&#xA;A = A0'*A0;&#xA;B = A0'*B0;&#xA;&#xA;X = A\B;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I am going to rewrite this into &lt;code&gt;C++&lt;/code&gt; later. For now, I have &lt;code&gt;X&lt;/code&gt; which are the parameters for the general form of an ellipsoid in 3 space. How do I now turn these parameters into a &lt;code&gt;3x3&lt;/code&gt; matrix to calculate $$ (x-C)^T \cdot A \cdot (x-C) $$ so that I can efficiently calculate if a point $x$ is inside the ellipsoid centered at $C$? Furthermore, $C$ is not any part of the original calculation of these parameters, so do I need to move &lt;code&gt;xs&lt;/code&gt;, &lt;code&gt;ys&lt;/code&gt;, and &lt;code&gt;zs&lt;/code&gt; to the original before performing the LS fit?&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2015-11-11T04:24:59.960" Title="Finding matrix form of Ellipsoid given general form" Tags="|linear-algebra|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21277" PostTypeId="2" ParentId="21275" CreationDate="2015-11-10T23:08:05.710" Score="0" Body="&lt;p&gt;Having all the points contained within ellipsoid reminds me of the minimum volume ellipsoid problem. This is easily solved with &lt;a href=&quot;https://en.wikipedia.org/wiki/Ellipsoid_method&quot; rel=&quot;nofollow&quot;&gt;Khachiyan's algorithm&lt;/a&gt;. The MATLAB code is available &lt;a href=&quot;http://de.mathworks.com/matlabcentral/fileexchange/9542-minimum-volume-enclosing-ellipsoid&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. Having said that, I think it is worth reminding you that matrix $A$ is actually the inverse covariance matrix.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2015-11-10T23:08:05.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21278" PostTypeId="2" ParentId="21275" CreationDate="2015-11-11T04:24:59.960" Score="2" Body="&lt;p&gt;You haven't actually defined an ellipsoid with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(x-c)^{T}A(x-c)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An ellipsoid is defined by an inequality &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(x-c)^{T}A(x-c) \leq 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that there isn't an ellipsoid of this form corresponding to every vector $X$ that you might get out of the least squares solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is a solution, then the quadratic terms are associated with $X(1)$, $\ldots$, $X(6)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=[X(1) \; X(4) \; X(6); X(4) \; X(2) \; X(5); X(6) \; X(5) \; X(3)]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-2AC=[X(7); \; X(8); \; X(9)]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C^{T}C=X(10)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is no such solution, or $A$ isn't positive definite, then $X$ doesn't correspond to an ellipsoid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice this is likely to fail to produce a solution.  There are other approaches to fitting ellipsoids to data that do work, but they aren't based on on first fitting a least squares quadratic form as you've done.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-11-11T04:24:59.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21279" PostTypeId="2" ParentId="21214" CreationDate="2015-11-11T05:31:59.643" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://users.monash.edu.au/~dprice/SPH/&quot; rel=&quot;nofollow&quot;&gt;http://users.monash.edu.au/~dprice/SPH/&lt;/a&gt; has references to some introductory notes on MHD in the case of SPH. There's also a number of astrobites on the topic (some of the references may be of some help):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://astrobites.org/2010/12/11/how-to-simulate-a-fluid/&quot; rel=&quot;nofollow&quot;&gt;http://astrobites.org/2010/12/11/how-to-simulate-a-fluid/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://astrobites.org/2011/11/11/what-you-need-to-know-about-sph/&quot; rel=&quot;nofollow&quot;&gt;http://astrobites.org/2011/11/11/what-you-need-to-know-about-sph/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://astrobites.org/2011/06/11/running-your-first-sph-simulation/&quot; rel=&quot;nofollow&quot;&gt;http://astrobites.org/2011/06/11/running-your-first-sph-simulation/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1437" LastActivityDate="2015-11-11T05:31:59.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21280" PostTypeId="2" ParentId="21041" CreationDate="2015-11-11T12:45:27.160" Score="1" Body="&lt;p&gt;Check the article in [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\delta Q_f$&lt;/span&gt; (for a boundary face) must be transformed by a matrix &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; such that,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\delta Q_f = T * \delta Q_L$&lt;/span&gt; ( note &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the left cell and for a boundary face there is no right cell or &lt;span class=&quot;math-container&quot;&gt;$Q_R$&lt;/span&gt; state)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This matrix &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is obtained from the boundary condition itself, for example, a supersonic outlet boundary has &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; (identity matrix) and so on ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, it gets a bit complicated for the wall, and other subsonic BCs. But it is not very hard to derive them yourself. In reference &lt;a href=&quot;https://doi.org/10.2514/3.11685&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, these matrices are already derived for wall and subsonic BC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.2514/3.11685&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1] John T. Batina, &quot;Implicit upwind solution algorithms for three-dimensional unstructured meshes&quot;, &lt;em&gt;AIAA Journal&lt;/em&gt;, Vol. 31, No. 5 (1993), pp. 801-805.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18197" LastEditorUserId="20688" LastEditDate="2018-09-21T13:57:11.530" LastActivityDate="2018-09-21T13:57:11.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21281" PostTypeId="1" AcceptedAnswerId="21282" CreationDate="2015-11-11T16:46:24.717" Score="2" ViewCount="182" Body="&lt;p&gt;I would like to calculate efficiency of parallel alghoritm, using the number of computations instead of time computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In materials from my studies I have a formula like below:&#xA;$$ \eta(n,p) = \frac{\omega(n)}{\omega(n)+h(n,p)} = \frac{1}{1+h(n,p)/\omega(n)}, $$&#xA;where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\eta(n, p)$ - efficiency of parallel program which realized n-sized task on machine with p-number of CPUs&lt;/li&gt;&#xA;&lt;li&gt;$\omega(n)$ - number of computations for a n-sized task&lt;/li&gt;&#xA;&lt;li&gt;$h(n,p)$ - number of I/O operations for a n-sized parallel task &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have an application in two modes which realized 2mln of the same operations on:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;4 threads with 4 CPUs. (with division of operations between all threads)&lt;/li&gt;&#xA;&lt;li&gt;serial computations with 4 CPUs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The formula looks simple but I don't know how to apply it.&#xA;I would like to ask about an explanation of mentioned formula in a real world example.&lt;/p&gt;&#xA;" OwnerUserId="18200" LastEditorUserId="713" LastEditDate="2015-11-11T17:29:22.293" LastActivityDate="2015-11-12T14:23:51.373" Title="Parallel efficiency" Tags="|parallel-computing|performance|efficiency|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21282" PostTypeId="2" ParentId="21281" CreationDate="2015-11-11T18:45:58.260" Score="3" Body="&lt;p&gt;This formula seems to assume that one operation counted by $\omega(n)$ takes as long as one I/O operation counted by $h(n,p)$. So to apply it, you have to be able to count operations in your code and scale them to the same time scale. The further elaboration dividing the numerator and denominator by $\omega$ probably scales this out effectively, but you still have to be able to count operations. Depending on your CPU/co-processor, counting operations can be easier or harder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also need to be able to define for yourself what kind of operations are meant by I/O operations. I.e., does this mean writing to disk or does it include operations to RAM, the network, etc. I presume this formula was derived by substituting $t(n,p) = t_{\rm compute}(n,p) + t_{\rm I/O}(n,p) + \ldots$ into Amdahl's law and turning the crank with $t_{\rm compute}(n,p)=\omega(n,p)/\dot{\omega}(n,p) + \ldots$, etc. I presume also that this was part of an explanation that tries to explain where parts of the times that appear in Amdahl's Law can come from. In order to employ such a formula, you've got to know a lot about your computer and your code. You have to be able to count lots of different types of operations within your code, and you have to know the rate that your computer managed to actually do them at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is much harder than just measuring execution time at each $n$ and $p$ and is not worth approaching.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2015-11-12T14:23:51.373" LastActivityDate="2015-11-12T14:23:51.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21283" PostTypeId="2" ParentId="330" CreationDate="2015-11-11T23:58:29.110" Score="0" Body="&lt;p&gt;In Python there are the tools pep8 and pep257 which will report missing or malformed documentation. elpy for Emacs will also complain about missing documentation. The &lt;a href=&quot;https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt&quot; rel=&quot;nofollow&quot; title=&quot;Numpy&quot;&gt;Numpy docstring conventions&lt;/a&gt; with reStructuredText is good to follow. Test with pep8, pep257 and doctest can be setup with py.test and tox running automatically.&lt;/p&gt;&#xA;" OwnerUserId="17340" LastActivityDate="2015-11-11T23:58:29.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21286" PostTypeId="1" AcceptedAnswerId="21293" CreationDate="2015-11-12T14:20:26.667" Score="3" ViewCount="2619" Body="&lt;p&gt;I'm trying to solve the following first order hyperbolic PDE problem using method of lines:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Hyperbolic Equation: &lt;span class=&quot;math-container&quot;&gt;$u_t = -u_x$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with initial condition: &lt;span class=&quot;math-container&quot;&gt;$u(0,x) = 0, 0 &amp;lt; x &amp;lt; 1$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;boundary condition: &lt;span class=&quot;math-container&quot;&gt;$u(t,0) = 1, t \ge 1$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The solution should be a step function to the right with velocity &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. I'm using centered finite difference to get an approximation of &lt;span class=&quot;math-container&quot;&gt;$u_x$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Following the code &lt;a href=&quot;http://kitchingroup.cheme.cmu.edu/blog/2013/03/07/Transient-heat-conduction-partial-differential-equations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this tutorial&lt;/a&gt;, my code becomes&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.integrate import odeint&#xA;import matplotlib.pyplot as plt&#xA;from scipy.fftpack import diff as psdiff&#xA;&#xA;N = 100 #no. of mesh points&#xA;L = 1.0&#xA;x = np.linspace(0, L, N) #mesh points xi, 0 &amp;lt; xi &amp;lt; 1&#xA;h = x[1] - x[0]&#xA;&#xA;k = -1.0&#xA;def odefunc(u, t):&#xA;    ux = np.zeros(x.shape)&#xA;    u[0] = 1 # boundary condition&#xA;    for i in range(1,N-1):&#xA;        ux[i] = float(u[i+1] - u[i-1])/(2*h) &#xA;    #   ux[i] = float(u[i] - u[i-1])/h &#xA;    dudt = -ux&#xA;    return dudt&#xA;&#xA;init = np.zeros(x.shape, np.float) #initial condition&#xA;tspan = np.linspace(0.0, 2.0, N)&#xA;sol = odeint(odefunc, init, tspan, mxstep=5000)&#xA;&#xA;for i in range(0, len(tspan), 2):&#xA;    plt.plot(x, sol[N-1], label='t={0:1.2f}'.format(tspan[i]))&#xA;&#xA;plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))&#xA;plt.xlabel('t')&#xA;plt.ylabel('u(x,t)')&#xA;&#xA;plt.subplots_adjust(top=0.89, right=0.77)&#xA;plt.savefig('pde.png')&#xA;&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;fig = plt.figure()&#xA;ax = fig.add_subplot(111, projection='3d')&#xA;&#xA;SX, ST = np.meshgrid(x, tspan)&#xA;ax.plot_surface(SX, ST, sol, cmap='jet')&#xA;ax.set_xlabel('x')&#xA;ax.set_ylabel('t')&#xA;ax.set_zlabel('u(x,t)')&#xA;ax.view_init(elev=15, azim=-100) # adjust view so it is easy to see&#xA;plt.savefig('pde-3d.png')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However the resulting graph is not a step function as it should be. What might be the problem here?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yksLE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yksLE.png&quot; alt=&quot;The resulting graph is here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18208" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-11-19T13:44:29.033" Title="Hyperbolic Equation PDE (Python)" Tags="|pde|finite-difference|python|hyperbolic-pde|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21287" PostTypeId="2" ParentId="12954" CreationDate="2015-11-12T14:51:39.397" Score="1" Body="&lt;p&gt;If this is an academic project and you are an university student then you could download student versions of commercial FEA packages like Hyperworks or ANSYS or ABAQUS for free. They have good tutorials to help you model your problem.&lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2015-11-13T00:15:49.600" LastActivityDate="2015-11-13T00:15:49.600" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21289" PostTypeId="1" CreationDate="2015-11-12T16:58:34.703" Score="0" ViewCount="926" Body="&lt;p&gt;I'm trying to solve the following PDE wave equation using method of lines:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Wave Equation: u_tt = u_xx&lt;/p&gt;&#xA;&lt;p&gt;with initial condition: u(0,x) = sin*pi,u_t(0,x)=0,  0 &amp;lt; x &amp;lt; 1&lt;/p&gt;&#xA;&lt;p&gt;boundary condition: u(t,0) = 0, u(t,1) = 0, t &amp;gt;= 1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Edited: deleted the old code and will be posting a new one soon. :)&lt;/p&gt;&#xA;" OwnerUserId="18208" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-11-13T03:13:37.173" Title="Wave Equation PDE" Tags="|pde|finite-difference|python|hyperbolic-pde|wave-propagation|" AnswerCount="2" CommentCount="1" ClosedDate="2015-11-16T18:53:13.007" ContentLicense="CC BY-SA 3.0" />
  <row Id="21290" PostTypeId="2" ParentId="21289" CreationDate="2015-11-12T17:23:33.187" Score="2" Body="&lt;p&gt;Since the wave equation involves a second derivative in time, two initial conditions are necessary: initial displacement $U(x,0)$ and initial velocity $U_t(x,0)$. I don't see the initial velocity condition specified in your question.  This leads me to suspect that there is something wrong with the tutorial you linked to...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I suspected, the code in the tutorial is for the &lt;em&gt;heat equation&lt;/em&gt;, not the &lt;em&gt;wave equation&lt;/em&gt;.  In order to solve the wave equation, you will also need to use a different time stepping scheme altogether.  The one implemented in the tutorial will not work for the wave equation.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2015-11-12T17:23:33.187" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21291" PostTypeId="2" ParentId="21289" CreationDate="2015-11-12T18:24:21.393" Score="3" Body="&lt;p&gt;The problem is that &lt;code&gt;odeint&lt;/code&gt; solves &lt;em&gt;first-order&lt;/em&gt; ODEs of the form $$\dot u(t) = Au(t),$$ but the wave equation is (after discretization in space) a &lt;em&gt;second-order&lt;/em&gt; ODE of the form $$\ddot{u}(t) = Au(t).$$ &#xA;(And in fact, your plot looks quite reasonable for the solution to a parabolic equation.)&#xA;To apply a black-box ODE solver (which is really not such a great idea, by the way), you first need to rewrite the wave equation as a system of first-order equations&#xA;$$ \left\{\begin{aligned} \dot{u}(t) &amp;amp;= v(t),\\ \dot{v}(t) &amp;amp;= Au(t).\end{aligned}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the risk of sounding brusque: Before making more than minor modifications to such code, you should first read up a bit on PDEs to understand &lt;em&gt;what&lt;/em&gt; you're trying to solve. A good starting point could be the Appendix E of &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow&quot;&gt;Randy LeVeque, &lt;em&gt;Finite Difference Methods for Ordinary and Partial Differential Equations&lt;/em&gt;, SIAM 2007&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-11-12T18:24:21.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21292" PostTypeId="2" ParentId="21286" CreationDate="2015-11-12T18:27:52.693" Score="1" Body="&lt;p&gt;Don't have the rep to comment, but you're missing some brackets in &lt;code&gt;odefunc&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(1,N):&#xA;    ux[i] = float(u[i+1] - u[i-1])/(2*h)&#xA;#               parentheses added  ^   ^ &#xA;#   ux[i] = float(u[i] - u[i-1])/h &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="17671" LastEditDate="2015-11-12T19:10:57.630" LastActivityDate="2015-11-12T19:10:57.630" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21293" PostTypeId="2" ParentId="21286" CreationDate="2015-11-12T20:22:55.330" Score="5" Body="&lt;p&gt;First-order hyperbolic equations model conservation laws; as the alternative name &quot;transport equations&quot; suggests, they transport information along so-called &quot;characteristic curves&quot; with a finite speed of propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the simple model equation $u_t + u_x = 0$, you can show (e.g., by separation of variables) that the solution should be of the form&#xA;$$u(t,x) = u(0,x-t),$$&#xA;where the missing information (where $x-t&amp;lt;0$) is taken from the boundary conditions. As you correctly write, with the data in your question you'd expect a step profile moving to the right with constant speed $1$:&#xA;$$u(t,x) = \begin{cases} 1 &amp;amp; x\leq t,\\0&amp;amp; x&amp;gt;t. \end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you can already see that something should be going wrong if you naively use a finite difference approximation: This solution is &lt;em&gt;not&lt;/em&gt; differentiable (it's not even continuous), and yet you try to approximate its derivative using a finite difference quotient. So you need to be a bit more clever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way is to slightly modify the equation so that it &lt;em&gt;does&lt;/em&gt; have a differentiable solution, for example by considering $u_t +u_x = \varepsilon u_{xx}$ for $\varepsilon&amp;gt;0$ very small, and apply a finite difference method to that. It turns out that taking $\varepsilon = \frac{\Delta t}{2}$ (half the discrete time step size) is a good idea; this is called the &lt;em&gt;Lax-Wendroff method&lt;/em&gt;:&#xA;$$U^{n+1}_{j} = U^n_j - \frac{\Delta t}{2\Delta x}(U^n_{j+1}-U^n_{j-1}) + \frac{\Delta t^2}{2\Delta x^2}(U^n_{j+1}-2U^n_j + U^n_{j-1}).$$&#xA;You can show that this is second-order accurate and stable as long as $\Delta t\leq \Delta x$. (This is discussed in Chapter 10 of &lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow&quot;&gt;Randy LeVeque, &lt;em&gt;Finite Difference Methods for Ordinary and Partial Differential Equations&lt;/em&gt;, SIAM 2007&lt;/a&gt;, together with other schemes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you replace your &lt;code&gt;odefunc&lt;/code&gt; routine by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t = np.linspace(0.0, 2.0, N)&#xA;k = t[1] - t[0]&#xA;def odefunc(u, t):&#xA;    ux = np.zeros(x.shape)&#xA;    u[0] = 1 # boundary condition&#xA;    for i in range(1,N-1):&#xA;        ux[i] = (u[i+1] - u[i-1])/(2*h) &#xA;        ux[i] -= k*(u[i+1] - 2*u[i] + u[i-1])/(2*h**2)&#xA;    dudt = -ux&#xA;    return dudt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(note that I've renamed &lt;code&gt;tspan&lt;/code&gt; to &lt;code&gt;t&lt;/code&gt; for simplicity), you should get much better results.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-11-13T14:54:16.443" LastActivityDate="2015-11-13T14:54:16.443" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21294" PostTypeId="2" ParentId="21239" CreationDate="2015-11-13T07:59:16.607" Score="0" Body="&lt;p&gt;I'm not sure this would be much more efficient, but you can try to solve $Ax=v$ where $\|v\|&amp;lt;\varepsilon$. If the result $\|x\|$ is also $\|x\|&amp;lt;\epsilon$ the matrix may not be singular.&lt;/p&gt;&#xA;" OwnerUserId="18215" LastActivityDate="2015-11-13T07:59:16.607" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21295" PostTypeId="1" AcceptedAnswerId="21297" CreationDate="2015-11-13T11:46:09.797" Score="1" ViewCount="2791" Body="&lt;p&gt;I have some data: number of nodes $N$ and error in energy norm corresponing to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen in some references that the rate of convergence is reported by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\| u-u_h\| _E=CN^{\alpha} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I find $C$ and $\alpha$ by MATLAB, I tried by :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;polyfit $(N,\| u-u_h\| _E,1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the answer was not true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I find them.&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="1804" LastEditDate="2015-11-13T12:06:06.817" LastActivityDate="2015-11-13T12:40:26.237" Title="Finding rate of convergence by curve fitting in Matlab" Tags="|matlab|finite-element|convergence|regression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21296" PostTypeId="2" ParentId="21295" CreationDate="2015-11-13T12:04:43.193" Score="1" Body="&lt;p&gt;I think that you should be looking for a rate of convergence in the form&#xA;$$\|u-u_h\|_E \leq Ch^{\alpha}$$&#xA;where $h$ is characteristic mesh size. In your case, a rough estimate would be &#xA;$$h = \frac{1}{\sqrt{N}}$$&#xA;in two dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to evaluate the discretization error on several grids with decreasing $h$ and then use the polynomial fit as&#xA;&lt;code&gt;[alpha, C] = polyfit(lc, error, 1)&lt;/code&gt; where &lt;code&gt;lc&lt;/code&gt; is a vector of logarithms of characteristic lengths &lt;code&gt;lc = [log(h1), log(h2), log(h3), ..., log(hn)]&lt;/code&gt; and &lt;code&gt;error&lt;/code&gt; is a vector of logarithms of corresponding error norms.&lt;/p&gt;&#xA;" OwnerUserId="4688" LastEditorUserId="4688" LastEditDate="2015-11-13T12:40:26.237" LastActivityDate="2015-11-13T12:40:26.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21297" PostTypeId="2" ParentId="21295" CreationDate="2015-11-13T12:05:09.003" Score="6" Body="&lt;p&gt;First of all, rates of convergence are usually given in the form&#xA;$$ \|u-u_h\| \leq C N^\alpha,$$&#xA;rather than equality. Furthermore, rates are &lt;em&gt;asymptotic&lt;/em&gt;, i.e., only have to hold for $N\to \infty$. This means that you're unlikely to find a single $C$ and $\alpha$ such that your equation holds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another reason why your approach doesn't work is that what you're trying to fit is &lt;em&gt;not&lt;/em&gt; a polynomial, since $\alpha$ is a) unknown and b) not an integer (for one thing, it must be negative since the error goes down as $N$ goes up).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What people usually do is look at a &lt;em&gt;double-logarithmic&lt;/em&gt; plot: If you take the logarithm of your equation (or my inequality), you get&#xA;$$ log(\|u-u_h\|) \leq \log(CN^\alpha) = \log(C) + \alpha\log(N).$$&#xA;This is a linear polynomial $ax+b$ in $\log (N)$ with coefficients $a=\alpha$ and $b=\log(C)$ (from which you can find $C=\exp(b)$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you apply &lt;code&gt;polyfit&lt;/code&gt; to $\log(N)$, $\log(\|u-u_h\|)$, you should get an array $[a,b]$ with $\alpha=a$ and $C=\exp(b)$.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2015-11-13T12:05:09.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21298" PostTypeId="2" ParentId="21239" CreationDate="2015-11-13T16:24:15.223" Score="1" Body="&lt;p&gt;LAPACK has xSTERF as routines to find eigenvalues for band matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are only interested in the smallest one, you might find the following answer &lt;a href=&quot;https://mathoverflow.net/questions/24287/what-is-the-best-algorithm-to-find-the-smallest-nonzero-eigenvalue-of-a-symmetri&quot;&gt;interesting&lt;/a&gt;.&#xA;However you need to implement the algorithm yourself.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2015-11-13T16:24:15.223" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21299" PostTypeId="1" CreationDate="2015-11-13T17:35:34.703" Score="4" ViewCount="226" Body="&lt;p&gt;Consider a $GF(2^n)$ field, a $GF(2^k)$ galois fields, where $n=k \times m$ and $GF(2^k)$ is a ground field of $GF(2^n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’d appreciate pointers to papers or suggestions on:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How to find $\log(a)$ and $\exp(a)$ where $a$ is in $GF(2^n)$ given $\log/\exp$ look-up tables of $GF(2^k)$?&lt;/li&gt;&#xA;&lt;li&gt;How to convert the values between $GF(2^n)$ into $GF\left( (2^k)^m \right)$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Specifically I need solution for $n = 16$ (with any combination of integer $m$ and $k$, e.g. $k = 8$, $m = 2$) such that amount of calculations used for conversion is minimal. Generator polynomials for all three fields can be assumed to be known, for example for case of $n = 16$, $k = 8$, $m = 2$:&#xA;$$&#xA;\begin{eqnarray}&#xA;&amp;amp;&amp;amp; GF(2^{16}) :     x^{16} + x^5 + x^3 + x^2 + 1 \\&#xA;&amp;amp;&amp;amp; GF(2^8) :     x^8  + x^4 + x^3 + x^2 + 1 \\&#xA;&amp;amp;&amp;amp; GF\left( (2^8)^2 \right) :  x^2  + 3x  + 1&#xA;\end{eqnarray}&#xA;$$&#xA;Additional background info: Generally I have $\log$ and $\exp$ look-up tables for $GF(2^n)$ and can avoid the conversion problem all together, but $2^n$ tables don't fit into memory-constrained CPU I'm using. Thus I'm interested to calculate $\log$ and $\exp$ of $GF(2^{16})$ using $\log/\exp$ tables of $GF(2^8)$ or $GF(2^4)$. I came across this &lt;a href=&quot;http://web.eecs.utk.edu/~plank/plank/papers/UT-CS-13-717.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, but it explicitly says that $GF(2^{km})$ is not identical to $GF \left( (2^k)^m \right)$, but doesn't offer a way to convert between the two: the result of multiplication using ground and extension fields doesn't match the multiplication result using any other method (presumably because the composite field is not identical to the original field).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. same question is also posted on &lt;code&gt;math.stackexchange&lt;/code&gt;, &lt;a href=&quot;https://mathoverflow.net/questions/223515/how-to-calculate-log-or-exp-of-a-value-in-gf2n-using-log-exp-table-of-gf2k&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18221" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2015-11-19T14:55:13.180" Title="How to calculate log or exp of a value in GF(2^n) using log/exp table of GF((2^k)^m) where n=k*m?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21301" PostTypeId="1" AcceptedAnswerId="21304" CreationDate="2015-11-13T22:39:34.740" Score="5" ViewCount="1599" Body="&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; &lt;em&gt;Should I store a matrix of Nx3 elements row-wise or column-wise, if I'm going to access 3 elements at a time? Does it matter?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm setting up a numerical simulation with matrices of x,y,z positions in 3 dimensions, and I wonder whether it's better to store them row-wise or column-wise in R, Matlab, and Armadillo (C++). Maybe it doesn't matter at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simulation considers N particles, say N=5000, and each is described by 3 coordinates, 3 angles, and 3 sizes. So I've 3 matrices of Nx3 elements. In the code (setting up a linear system of 3N equations), I will need to access positions, sizes, and angles one particle at a time. Therefore, I'm tempted to store those as columns, i.e. those matrices would have 3 rows, and N columns. This is because R, Matlab and Armadillo all have a column-major ordering, so my thinking is that accessing a column will be (ever so slightly) faster. &#xA;On the other hand, maybe there is a penalty of having many columns (memory addresses)? I do not know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By-the-way, I'm mentioning three programming languages because I'll be writing the code in all three (it's not much effort, since the syntax is pretty much the same).&lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2016-05-04T00:22:46.193" Title="matrix storage, many rows or many columns?" Tags="|matlab|r|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21302" PostTypeId="2" ParentId="21301" CreationDate="2015-11-14T00:49:58.807" Score="2" Body="&lt;h3&gt;R, Matlab, and Armadillo (&lt;i&gt;Mat&lt;/i&gt; class) matrices use column-major order so you should create 3xN matrices so that each set of 3 values is stored consecutively in memory.&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A bit more info:&lt;/strong&gt;&lt;br&gt;&#xA;You should store your matrices such that elements which are accessed together are stored sequentially in memory. The memory order of matrix storage is language, data structure, and possibly even implementation dependent. The two storage orders used for matrices are referred to as &lt;a href=&quot;https://en.wikipedia.org/wiki/Row-major_order&quot; rel=&quot;nofollow&quot;&gt;&quot;row-major&quot; or &quot;column-major&quot; order&lt;/a&gt;. In row-major order elements in the same row are stored consecutively in memory and vice versa in column-major.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of choosing the correct arrangement is decreased &lt;a href=&quot;https://en.wikipedia.org/wiki/CPU_cache#Cache_miss&quot; rel=&quot;nofollow&quot;&gt;cache misses&lt;/a&gt; and decreased memory latency. Efficient cache usage can sometimes lead to large speedups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A final note: C and C++ 2D arrays are row-major, but Armadillo's Mat class is column-major since the storage layout was designed to be compatible with BLAS and LAPACK which use Fortran's column-major storage order. Matlab derives much of it underpinnings and syntax from Fortran so it naturally uses column-major storage order. I'm unsure of the history behind R's storage order, but perhaps someone will comment with that information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sources:&lt;/strong&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Row-major_order#Programming_languages&quot; rel=&quot;nofollow&quot;&gt;Wikipedia&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;http://heather.cs.ucdavis.edu/~matloff/R/RProg.pdf&quot; rel=&quot;nofollow&quot;&gt;R for Programmers&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;http://www.mathworks.com/help/matlab/matlab_external/matlab-data.html&quot; rel=&quot;nofollow&quot;&gt;MATLAB documentation&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;http://arma.sourceforge.net/armadillo_nicta_2010.pdf&quot; rel=&quot;nofollow&quot;&gt;Armadillo documentation (pdf)&lt;/a&gt;  &lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-11-14T01:05:55.553" LastActivityDate="2015-11-14T01:05:55.553" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21303" PostTypeId="1" AcceptedAnswerId="21310" CreationDate="2015-11-14T01:07:27.447" Score="4" ViewCount="1069" Body="&lt;p&gt;I have a large number of systems of the form:&#xA;$$A_ix=b,$$&#xA;where each $A_i,i&amp;gt;0$ is a rank one update of $A_{i-1}$ and the $A_i$ are dense matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering whether it is possible to use the classical LU &#xA; algorithm to solve these (rank one updated) versions of the &#xA; initial problem more efficiently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Right now I'm using eigen c++'s &lt;code&gt;PartialPivLU&lt;/code&gt; implementation. &lt;/p&gt;&#xA;" OwnerUserId="479" LastEditorUserId="479" LastEditDate="2015-12-27T15:17:50.763" LastActivityDate="2015-12-27T15:17:50.763" Title="Solve rank one update to LU using plain vanilla LU routine" Tags="|linear-system|matrix-factorization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21304" PostTypeId="2" ParentId="21301" CreationDate="2015-11-14T01:10:00.107" Score="5" Body="&lt;p&gt;Which storage scheme is optimal depends &lt;em&gt;very&lt;/em&gt; strongly on the access&#xA;pattern. You already said that you're accessing them one particle at a&#xA;time, but are you going to access the particles sequentially in order,&#xA;or randomly? Another conventional name for this issue is&#xA;struct-of-arrays vs array-of-structs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key thing here is to know how the memory hierarchy works on a CPU&#xA;(see, e.g.,&#xA;&lt;a href=&quot;https://www.scss.tcd.ie/Jeremy.Jones/CS3021/5%20caches.pdf&quot;&gt;https://www.scss.tcd.ie/Jeremy.Jones/CS3021/5%20caches.pdf&lt;/a&gt;, as well as the many questions on this topic on Stack Overflow). Reads&#xA;from main memory have a very large latency, which will lead to long&#xA;stalls in the execution, unless this is mitigated by prefetching. Each&#xA;read from main memory will read one cache line worth of data, placing&#xA;it in cache, and the reads from cache are comparatively free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for example if you have a matrix $(a_{ij})_{1\leq i,j\leq 3}$,&#xA;and it is stored in column-major order in memory as &#xA;$$ a_{11}, a_{21}, a_{31}, a_{12}, a_{22}, a_{32}, a_{13}, a_{23},&#xA;a_{33}, $$&#xA;then reading $a_{11}$ (&lt;em&gt;or any of&lt;/em&gt; $a_{11},\ldots,a_{23}$) will place all the eight elements from $a_{11}$ to&#xA;$a_{23}$ in cache (assuming correct alignment and a typical 64-byte&#xA;cache line and an element size of 8 bytes), making them free to access.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have several different arrays, a struct-of-arrays, like&#xA;$$ x_{1}, \ldots, \qquad y_{1}, \ldots, \qquad z_{1}, \ldots $$&#xA;and you read $x_1, y_1, z_1$, you will effectively read the first&#xA;eight elements of each array. If you end up using the next seven&#xA;elements immediately, that's fine. If not, then that wasted 7/8 of&#xA;your memory bandwidth by reading 24 elements instead of 3, which is inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If instead you store them as array-of-structs,&#xA;$$ x_1, y_1, z_1, x_2, y_2, z_2, \ldots $$&#xA;then reading $x_1,y_1,z_1$ will read the first eight elements. If you use those next&#xA;elements, there is no great difference, otherwise you waste 5/8 (reading 8 instead of 3) of your memory bandwidth - note this is still better than reading 24 elements as in the previous example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With nine 8-byte elements, if you know you'll access them&#xA;consecutively there should be no big difference either way, as above. If you will&#xA;access them in random order, then reading one set of nine elements&#xA;will lead to either $9\times 8 = 72$ elements being read from memory (if&#xA;stored as struct-of-arrays) or $\lceil \frac{9}{8}\rceil\times 8 = 16$&#xA;elements (if stored as array-of-structs) - this could be a&#xA;very large performance gain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between column-major and row-major, I feel, is a bit of&#xA;red herring here - the key is knowing the access pattern, and making&#xA;sure the data layout matches the access pattern. The next most&#xA;important thing would be &lt;em&gt;prefetching&lt;/em&gt;, which is a technique for&#xA;hiding the relatively large memory access latency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A separate related issue to consider is how you access data that you already read previously, and whether it is likely to still remain in the cache or whether it was purged to make way for other data, causing cache misses - an array of $5000\times 9\times 8$ bytes is $360\,\mathrm{KB}$ and will likely fit entirely in either the L2 or L3 cache. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It gets even more complicated if your program involves multiple threads running on different cores, with potentially separate lower-level caches per core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, incidentally, that for non-sequential access patterns, having nine elements per particle is somewhat inefficient in itself if a cache line is 8 elements long as reading nine elements would be essentially equivalent to reading 16 elements - one technique is to split your data into often-used versus rarely-used parts so that the often-used parts fit better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no penalty for having either many rows or columns. Not sure what you mean by a penalty for &quot;many memory addresses&quot;, but there is no such penalty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the things I said depend on exact properties of the architecture your program will run on. Using an interpreted language like R or Matlab can also affect this severely - for C++ it should hold.&#xA;In general, it is best to use the array-of-structs pattern, unless you have some specific reasons not to.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-11-14T01:34:40.577" LastActivityDate="2015-11-14T01:34:40.577" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21305" PostTypeId="1" AcceptedAnswerId="21309" CreationDate="2015-11-14T07:57:26.123" Score="1" ViewCount="362" Body="&lt;p&gt;For integration of polynomials, the number of qradrature points can be determined easily according to the $(2N-1)$ rule. While in most cases, it's not necessary to use Gaussian quadrature for simple polynomials since they can be calculated analytically, what we encounter in reality are generally complex integrands, such as rational functions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, how can we determin an appropriate number of quadrature points so that we can get a somewhat acceptible solution, and at the same time will not lead to a waste of computational efforts?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastActivityDate="2015-11-14T16:48:26.880" Title="How should I determine the number of quadrature points when using Gaussian quadrature integration?" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21306" PostTypeId="1" AcceptedAnswerId="21308" CreationDate="2015-11-14T11:29:38.163" Score="1" ViewCount="137" Body="&lt;p&gt;I have a set of data to which I want to apply a weighting function, so basically a simple multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My sampled data looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fl9XQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fl9XQ.png&quot; alt=&quot;sampled data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(for those interested, it's the relative luminous intensity of an LED)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weighting function is only available in a tabular form as well, and looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/unetN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/unetN.png&quot; alt=&quot;weighting function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(it's the standard luminosity function, which would probably be available in other formats, but I have another weighting which isn't)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it happens, the points don't coincide on the X-Axis, so I can't just multiply the values, but have to do some interpolation of the weighting function to get the &quot;correct&quot; weighting factor for the points of my sampled data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is already a function / package available for Octave which handles the multiplication of the Y-values with interpolation of the X-values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I'm not a native speaker and am working mostly on electrical engineering problems, my vocabulary of the right keywords ran out quickly. It's probably not all that hard to implement this, but I try not to reinvent the wheel over and over again.&lt;/p&gt;&#xA;" OwnerUserId="18225" LastActivityDate="2017-03-08T23:36:15.173" Title="Applying a weighting function which is sampled at different points" Tags="|octave|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21307" PostTypeId="1" CreationDate="2015-11-14T14:08:29.987" Score="1" ViewCount="33" Body="&lt;p&gt;I have a problem of allocating x between p entities, each entity has a constant $\rho_i$ such that the allocated part should be proportionnal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without loss of generality we can divide the ressources by x and normalise the $\rho_i$ by $\Sigma_i\rho_i$ to see the unconstrained problem has a simple solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_i = n \frac{\rho_i}{\Sigma_i\rho_i} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is to solve a constrained problem, such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ a_i \leq x_i \leq b_i $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an iterative algorithm, starting at $x_i = n \frac{\rho_i}{\Sigma_i\rho_i} $ at each iteration we reallocate the ressources that excess the constraint. Theoretically this works because $\Sigma b_i &amp;gt; 1$ and $\Sigma a_i &amp;lt; 1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the feeling that it could be reformulated as an optimization problem but can't figure out what the objective function would be.&lt;/p&gt;&#xA;" OwnerUserId="18226" LastActivityDate="2015-11-14T14:08:29.987" Title="Allocation under constraint (optimization ?)" Tags="|constrained-optimization|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21308" PostTypeId="2" ParentId="21306" CreationDate="2015-11-14T14:53:39.193" Score="0" Body="&lt;p&gt;I implemented this in a crude way (I'm not using Octave or Matlab very often so I'm not used to vector and matrix data types and how to make the most use of them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function accepts a X-axis value, needs a weighting matrix where the first column contains the x-values and the second column the weighting factors. It also needs two standard values which are chosen if the supplied X-axis value is outside of the range of weighting matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It returns a linear interpolation of the weighting factor, or if the weighting matrix is malformed (multiple entries for one x-point) the larger weighting factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is horrible, does no checks and probably fails for everyone but me. Maybe it's a starting point for someone else.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function retval = Weighting ( x, weightingMatrix, weightingBelowMinimum, weightingAboveMaximum)&#xA;&#xA;if (x &amp;lt; weightingMatrix(1,1))&#xA;  retval = weightingBelowMinimum;&#xA;elseif ( x &amp;gt; weightingMatrix(size(weightingMatrix,1),1))&#xA;  retval = weightingAboveMaximum;&#xA;else&#xA;  smallerWeightIndex = 0;&#xA;  biggerWeightIndex = 0;&#xA;  for i=1:size(weightingMatrix,1)&#xA;    if (x&amp;lt;weightingMatrix(i,1))&#xA;      biggerWeightIndex = i;&#xA;      smallerWeightIndex = i-1;&#xA;      break;&#xA;    endif&#xA;  endfor&#xA;&#xA;&#xA;  if (weightingMatrix(biggerWeightIndex,2)==weightingMatrix(smallerWeightIndex,2))&#xA;    retval = weightingMatrix(biggerWeightIndex,2);&#xA;  elseif (weightingMatrix(biggerWeightIndex,1)==weightingMatrix(smallerWeightIndex,1))&#xA;    retval = max(weightingMatrix(biggerWeightIndex,2),weightingMatrix(smallerWeightIndex,2));&#xA;  else&#xA;    m = (weightingMatrix(biggerWeightIndex,2)-weightingMatrix(smallerWeightIndex,2))/(weightingMatrix(biggerWeightIndex,1)-weightingMatrix(smallerWeightIndex,1));&#xA;    c = weightingMatrix(biggerWeightIndex,2)-(m*weightingMatrix(biggerWeightIndex,1));&#xA;    retval = m*x+c;&#xA;  endif&#xA;endif&#xA;&#xA;&#xA;endfunction&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18225" LastActivityDate="2015-11-14T14:53:39.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21309" PostTypeId="2" ParentId="21305" CreationDate="2015-11-14T15:35:58.220" Score="4" Body="&lt;p&gt;There is a formula for the error term in Gaussian quadrature, so if you bound the derivatives of your integrand, you can bound the error.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can't bound the derivatives of your integrand you can use a Gauss-&#xA;Kronrod rule in which you first use n points to estimate the integral and then reuse those points with n+1 additional points to get a better estimate.  The difference of the two estimates is an estimate of the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This technique of reusing points in a higher order approximation is also used in Clenshaw-Curtis integration.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is standard material in introductory courses in numerical analysis that you can find in many textbooks.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-11-14T16:48:26.880" LastActivityDate="2015-11-14T16:48:26.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21310" PostTypeId="2" ParentId="21303" CreationDate="2015-11-14T18:21:34.180" Score="4" Body="&lt;p&gt;The eta factorization of the basis is a technique widely used in the simplex method for LP to handle rank one column updates.  The same idea can be modified to handle rank one row updates,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll assume that you start with an LU factorization of $A_{0}$ and can solve systems of equations of the form $A_{0}x=b$.  We can also solve systems of the form $yA_{0}=c$ where $c$ is a given row vector and $y$ is the row vector that you are solving for. If $PA_{0}=LU$, then we can solve $yA_{0}=c$ by&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve $wU=c$.&lt;/li&gt;&#xA;&lt;li&gt;Solve $vL=w$.&lt;/li&gt;&#xA;&lt;li&gt;Let $y=vP$. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Suppose that in the first iteration, row $k$ of $A_{0}$ is to be replaced by a row vector $c_{1}$.  Solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$yA_{0}=c_{1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $E_{1}$ be the identity matrix with $y$ in place of row row $k$.  Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E_{1}A_{0}=A_{1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can solve a system of equations $A_{1}x=b$ by &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve $E_{1}u=b$.&lt;/li&gt;&#xA;&lt;li&gt;Solve $A_{0}x=u$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that because $E_{1}$ is nearly the identity matrix except for row $k$, it's trivial to solve $E_{1}u=b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can repeat this process to get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E_{2}E_{1}A_{0}=A_{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\ldots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E_{k} \cdots E_{1}A_{0}=A_{k}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the simplex method it is typical to do this for something like 30 iterations before refactoring $A_{30}$ from scratch.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-11-14T18:21:34.180" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21313" PostTypeId="1" CreationDate="2015-11-15T03:42:44.447" Score="1" ViewCount="205" Body="&lt;p&gt;I have been doing much research on the internet to see if there was a FEM simulator that natively implemented the Jiles-Artherton model whether it be commercial or open source&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, my conclusion is that there is none. Before I make a final verdict though, I would like to check with the community that this statement is correct. Is there any software package that natively implements the Jiles-Artherton model? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking into this because I am working on finding a focal point to my PhD thesis research and I was thinking about creating a simulator for non-linear magnetic simulations. It would include a number of different features such as SPICE simulation that couples with the FEA simulation and parallel computing (both SPICE and FEA simulation). This would also include implementing a Jiles-Artherton model natively into the software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am though concerned that my project is not too much on the researchy side of things and worried this may not be funded since alot of the commercial software packages implement what I would like to do. So, I would also like to ask the community for some feedback, if possible, on the current research topic. Everything is still in an incubation since I start in the coming spring semester. I am asking this because both my professor and I are relatively new to the field.  &lt;/p&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2020-12-14T14:05:29.407" Title="Software Implementing Jiles-Atherton" Tags="|nonlinear-programming|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21315" PostTypeId="2" ParentId="14818" CreationDate="2015-11-15T21:29:17.507" Score="7" Body="&lt;p&gt;I wrote a Python package called &lt;a href=&quot;http://github.com/verdverm/pypge&quot; rel=&quot;nofollow&quot;&gt;PyPGE&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PyPGE is a Symbolic Regression implementation based on Prioritized Grammar Enumeration &lt;strong&gt;(1)&lt;/strong&gt;, not Evolutionary or Genetic Programming. It produces a deterministic Symbolic Regression algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(1)&lt;/strong&gt; Worm, Tony, and Kenneth Chiu. &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=2463486&quot; rel=&quot;nofollow&quot;&gt;&quot;Prioritized grammar enumeration: symbolic regression by dynamic programming.&quot;&lt;/a&gt; Proceedings of the 15th annual conference on Genetic and evolutionary computation. ACM, 2013. GitHub: &lt;a href=&quot;http://github.com/verdverm/pypge&quot; rel=&quot;nofollow&quot;&gt;http://github.com/verdverm/pypge&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18237" LastEditorUserId="21577" LastEditDate="2016-09-04T15:01:02.800" LastActivityDate="2016-09-04T15:01:02.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21316" PostTypeId="1" AcceptedAnswerId="21317" CreationDate="2015-11-15T22:20:02.780" Score="4" ViewCount="284" Body="&lt;p&gt;We consider the problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left\{\begin{matrix}&#xA;k(x)\Delta u(x)=f(x) &amp;amp; \text{ in } \Omega\\ &#xA;u=0 &amp;amp;  \text{ in } \Gamma&#xA;\end{matrix}\right.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Omega \subset \mathbb{R}^2$ open and bounded with smooth boundary $\Gamma=\partial{\Omega}$ and $k(x) \geq a, \forall x \in \Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We consider that $\Omega= \Omega_1 \cup \Omega_2$ and that $k$ is always a constant $k(x)=\left\{\begin{matrix}&#xA;k_1 &amp;amp; \text{ in } \Omega_1 \\ &#xA;k_2 &amp;amp; \text{ in } \Omega_2 &#xA;\end{matrix}\right.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rJjUh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rJjUh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma$ is the boundary between $\Omega_1$ and $\Omega_2$, $\Sigma= \overline{\Omega_1} \cap \overline{\Omega_2}, \Gamma_j=\Gamma \cap \partial{\Omega_j}, j=1,2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We suppose that $\Omega=[-1,1]^2$ and $\Sigma=[-1,1] \times \{ y=0 \}, \Omega_1=\Omega \cap \{ y&amp;gt;0 \}$ and $\Omega_2=\Omega \cap \{ y&amp;lt;0 \}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We construct a grid $\Omega$ so that the interface $\Sigma$ coincides with the sides of the squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IeXuf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IeXuf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We write the problem as $AU=f$ in order to find the approximation of the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the approximations that correspond to the points that are over $\Sigma$ we pick $k1$ and for the approximations that correspond to the points that are under $\Sigma$ we pick $k2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do we do for the approimations that correspond to the points that are on $\Sigma$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:Finally, we don't want the approximation of the solution to be continuous.&#xA;We may take $k=\frac{k_1+k_2}{2}$ for the points on the interface. So do we have to take cases for $y$ in order to give a value to $k$? &lt;/p&gt;&#xA;" OwnerUserId="9422" LastEditorUserId="9422" LastEditDate="2015-11-16T15:14:25.373" LastActivityDate="2015-11-22T10:24:38.150" Title="Points on the interface" Tags="|finite-difference|numerical-analysis|elliptic-pde|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="21317" PostTypeId="2" ParentId="21316" CreationDate="2015-11-15T22:50:00.463" Score="2" Body="&lt;p&gt;Rewrite your problem as&#xA;$$\begin{cases}&#xA;\Delta u(x)=\frac{f(x)}{k_1}&amp;amp; \text{ in } \Omega_1\\ &#xA;\Delta u(x)=\frac{f(x)}{k_2}&amp;amp; \text{ in } \Omega_2\\ &#xA;u=0 &amp;amp;  \text{ in } \Gamma&#xA;\end{cases}$$&#xA;you have to add a condition on an interface, for example a jump condition, e.g.&#xA;$$u^1(x)-u^2(x)=\alpha(x)\quad\text{and}\quad &#xA;u^1_n(x)-u^2_n(y)=\beta(x)&#xA; ,\quad x\in\Sigma$$&#xA; or smoothness (e.g. $\alpha=0=\beta$) of the solution and its normal derivative across the interface. In the above $u^i$ is the solution in the domain $\Omega_i$, $i\in\{1,2\}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the best ways to solve such problem would be &lt;a href=&quot;http://www.math.utah.edu/~mmedvin/Publications/MedvinskyPHDDissertation.pdf&quot; rel=&quot;nofollow&quot;&gt;Difference Potentials Method&lt;/a&gt; (based on good theory, need time to learn, worth the time you spend on it), another is Immersed Interface method(easy to understand, implementation is involved and changes with the problem, less accurate). &#xA;You find a little more under the link provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Regularization approach&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the low order of accuracy is enough for you, you may want to try something very simple. &#xA;You take a &lt;a href=&quot;https://en.wikipedia.org/wiki/Heaviside_step_function&quot; rel=&quot;nofollow&quot;&gt;continuous approximation of step function&lt;/a&gt; and solve a little bit different, but continuous problem, e.g.:&#xA;$$\begin{cases}&#xA;\Delta u(x)=\frac{f(x)}{g_p(x)}&amp;amp; \text{ in } \Omega_1\cup \Omega_2\\ &#xA;u=0 &amp;amp;  \text{ in } \Gamma&#xA;\end{cases}$$&#xA;where (you can choose another approximation of step function)&#xA;$$g_p=\frac{k_2-k_1}{1+e^{-M(x-p)}} + k_1.&#xA;$$&#xA;$M$ supposed to be a big number, the larger the $M$ the sharper the &quot;jump&quot;, i.e. the closest to what you need. You can play with different values to see the effect.&#xA;Note, when $x&amp;lt;p$ the denominator is a big number and so the fraction become close to zero. when $x&amp;gt;p$ the exponential in the denominator is negligible  and therefore the fraction is almost the nominator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, this way you solve a problem without singularities or jumps. However, you solve a little bit different problem, which is asymptotically (as $M\to\infty$) tends to the original one. &lt;/p&gt;&#xA;" OwnerUserId="18215" LastEditorUserId="18215" LastEditDate="2015-11-22T10:24:38.150" LastActivityDate="2015-11-22T10:24:38.150" CommentCount="62" ContentLicense="CC BY-SA 3.0" />
  <row Id="21319" PostTypeId="1" CreationDate="2015-11-16T11:51:11.327" Score="4" ViewCount="90" Body="&lt;p&gt;Suppose I have the discrete values $f(x_i)$ for every discrete value $x_i$ greater than some $\varepsilon$, and I want to numerically calculate the following integral:&#xA;\begin{equation}&#xA; n = \int_\varepsilon^\infty f(x)Z(x)dx&#xA;\end{equation}&#xA;with the known function $Z(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can approximate $f(x)=f(x_i)$ for $x\in[x_{i-},x_{i+}]$ with $x_{i-}=x_i-\dfrac{\Delta x}{2}$ and $x_{i+}=x_i+\dfrac{\Delta x}{2}$,&#xA;and calculate the above integral like:&#xA;\begin{equation}&#xA; n \approx f(x_k)\int_{\varepsilon}^{x_{k+}} Z(x)dx + \sum_{j=k+1}^{\infty} f(x_j)\int_{x_{j-}}^{x_{j+}} Z(x)dx&#xA;\end{equation}&#xA;with $\varepsilon\in[x_{k-},x_{k+}]$. First consider the case of $Z(x)=\sqrt{x-\varepsilon}$, for which the above expression gives a continuously differentiable&#xA;approximation with respect to $\varepsilon$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, consider another function $Z(x)=c$. We can check and see that the above approximation is no longer differentiable when $\varepsilon$ moves continuously from one &#xA;$[x_{i-},x_{i+}]$ to the next $[x_{(i+1)-},x_{(i+1)+}]$. In order to have a continously differentiable approximation to $n$, we can use a piecewise linear interpolation for&#xA;$f(x)$. That is, &#xA;\begin{align}&#xA; &amp;amp; f(x) \approx \dfrac{f(x_{j+1})-f(x_j)}{\Delta x}(x-x_j) + f(x_j)  \qquad\quad   x\in [x_{j-},x_{j+}]~,   \\&#xA; &amp;amp; n \approx \int_{\varepsilon}^{x_{k+}} f(x)Z(x)dx + \sum_{j=k+1}^{\infty} \int_{x_{j-}}^{x_{j+}} f(x)Z(x)dx&#xA;\end{align}&#xA;results in a continuous derivative $\dfrac{dn}{d\varepsilon}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some piecewise interpolation for $f(x)$ in order to make the above mentioned numerical approximation continuously differentiable &#xA;for the function $Z(x) = \dfrac{1}{\sqrt{x-\varepsilon}}$?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2015-11-16T11:51:11.327" Title="Can I make this numerical integration continuously differentiable?" Tags="|interpolation|numerics|integration|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21320" PostTypeId="1" CreationDate="2015-11-16T14:16:04.400" Score="8" ViewCount="299" Body="&lt;p&gt;I have the following question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have two matrices $X, Y$ both of size $m\times p$ and a random i.i.d Gaussian matrix $G$ of size $m \times k$, $m\gg p&amp;gt;k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a fast way to compute $\exp(-XY^T)G$? Perhaps by using the fact that both $X$ and $Y$ are much smaller than $XY^T$? Here, $\exp(\cdot)$ means entry-wise exponent (i.e. of $\textbf{each}$ entry of the matrix). Clearly, without the exponent, it's easy and can be done simply by $X(Y^TG)$, but the problem is that the element-wise exponent is no longer low rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The motivation for the question is multiplying a kernel of the form  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D_{ij}=\exp(-d_{ij})$ or $D_{ij}=\exp(-d_{ij}^2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $d_{ij}=\Vert x_i-y_j \Vert$ and $x_i, y_i \in \mathbb{R}^p$ by a random matrix efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approximated solution is also fine.&lt;/p&gt;&#xA;" OwnerUserId="3761" LastEditorUserId="21193" LastEditDate="2017-12-23T19:14:49.310" LastActivityDate="2018-01-24T08:02:13.153" Title="Fast computation of component-wise $\exp(-XY^T)G$ for random $G$" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21326" PostTypeId="2" ParentId="21299" CreationDate="2015-11-17T12:14:44.760" Score="1" Body="&lt;p&gt;I'm not an expert in Galois Fields, so this answer will not respond to your question directly... but I couldn't help thinking about this problem so this is what I reached (in Matlab):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% reduced table of logarithms (one each 256)&#xA;rd_table = [&#xA;           2&#xA;        5726&#xA;        ....&#xA;];    &#xA;&#xA;two = gf(2,16);&#xA;v = gf(4, 16); &#xA;cnt = 0; &#xA;while length(find(rd_table == v.x)) == 0; &#xA;    v = v * two; &#xA;    cnt = cnt + 1; &#xA;end; &#xA;result = (find(rd_table == v.x)-1)*256+1-cnt&#xA;iterations = cnt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The idea is to find the value in the list of the logarithms; if you can't find it, left-shift the value (in GF(2^16)) and try again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make it work on a microcontroller you will need an &lt;code&gt;uint16&lt;/code&gt; array for the 256 values in &lt;code&gt;rd_table&lt;/code&gt; and an &lt;code&gt;uint8&lt;/code&gt; array of 256 indices in it to speed up the lookup. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, to evaluate the log, you'll need up to 255 cycles (lookfor + left shift). Total complexity &lt;code&gt;O(65536/n/2logn)&lt;/code&gt;, where &lt;code&gt;n&lt;/code&gt; is the length of the table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I put this code snippet on &lt;a href=&quot;http://goo.gl/VjSsfs&quot; rel=&quot;nofollow&quot;&gt;tutorialspoint&lt;/a&gt; for you to play with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="18255" LastActivityDate="2015-11-17T12:14:44.760" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21328" PostTypeId="1" AcceptedAnswerId="21331" CreationDate="2015-11-17T14:14:15.383" Score="0" ViewCount="6635" Body="&lt;p&gt;I'm trying to write a class that uses Newton's law of Gravitation to work out the field of a planet. I've tested my code by inputting values for Earth, for purely vertical motion, so I should get g = -9.8 and I don't. &#xA;I have a class I can use that does every imaginable operation on vectors, called PhysicsVector. I've stared at my code for ages but can't work out what's wrong! I can add the main class if it helps, but I assume I've made some maths mistake in the GravField class here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import java.lang.Math;&#xA;public class GravField{ &#xA;public static final double G = 6.674*Math.pow(10,-11);          //Gravitational constant, same for everything&#xA;private double planetMass=0;                        &#xA;private double planetRadius=0;                      &#xA;private double projectileX=0;&#xA;private double projectileY=0;&#xA;PhysicsVector projectilePosition = new PhysicsVector(a, projectileY);   //Haven't declared a yet, don't know if that's allowed&#xA;PhysicsVector gravityAcceleration = new PhysicsVector();&#xA;&#xA;public GravField(double planetMass, double planetRadius, double projectileX, double projectileY){&#xA;&#xA;    double distance = Math.sqrt(projectileX*projectileX+projectileY*projectileY);   //Distance of projectile from Earth&#xA;    double a = planetRadius + projectileX;              //Newton's law assumes origin at planet centre,&#xA;                                    //my co-ord system starts at planet surface&#xA;&#xA;    double x = (-G*planetMass*projectileX)/(distance*distance*distance);    //Probably some maths mistake here&#xA;    double y = (-G*planetMass*projectileY)/(distance*distance*distance);&#xA;&#xA;    gravityAcceleration.setVector(x, y);                //setVector is in PhysicsVector class and makes vector &#xA;    gravityAcceleration.print2D();                  //xi + yj&#xA;&#xA;&#xA;}&#xA;&#xA;public static double magnitude(PhysicsVector gravityAcceleration){&#xA;&#xA;    double magnitudeOfGravField = gravityAcceleration.magnitude();  //again magnitude is in PhysicsVector class&#xA;    return magnitudeOfGravField;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;" OwnerUserId="18259" LastActivityDate="2015-11-17T18:34:32.117" Title="Simple java gravity simulation" Tags="|simulation|java|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21329" PostTypeId="2" ParentId="21328" CreationDate="2015-11-17T14:30:45.233" Score="0" Body="&lt;p&gt;Newton's law of gravity (divided by the mass of the projectile) is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g = \frac{G m_e \vec{r}}{r^3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hand calculating this for Earth's mass and radius gives a value of $9.93\ m/s^2$ which is close enough for my tastes (I expect error since we are so close to the surface and Earth isn't a point mass).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your error is that you are using $\vec{r}$ in coordinates that have an origin at the surface rather than at the center of the earth.  You need:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double x = (-G*planetMass*a)/(distance*distance*distance);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1552" LastEditorUserId="1552" LastEditDate="2015-11-17T15:11:49.207" LastActivityDate="2015-11-17T15:11:49.207" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21331" PostTypeId="2" ParentId="21328" CreationDate="2015-11-17T18:34:32.117" Score="2" Body="&lt;p&gt;This should be a work for codereview... but here we are, so let's continue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are some modifications to your class to make it better formed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import java.lang.Math;&#xA;&#xA;public class GravField {    &#xA;  private static final double G = 6.674*Math.pow(10,-11);           //Gravitational constant, same for everything&#xA;  private double planetMass = 0;                      &#xA;  private double planetRadius = 0;                        &#xA;  private PhysicsVector planetPosition;&#xA;&#xA;  public GravField(double planetMass, double planetRadius, PhysicsVector planetPosition){&#xA;    this.planetPosition = planetPosition;&#xA;    this.planetMass = planetMass;&#xA;    this.planetRadius = planetRadius;&#xA;  }&#xA;&#xA;  public PhysicsVector gravityAcceleration(PhysicsVector projectile){&#xA;    PhysicsVector distanceVector = projectile.difference(this.planetPosition);  // hope your class can make vector operations&#xA;    double distance = distanceVector.magnitude();   //Distance of projectile from Earth&#xA;    double x;&#xA;    double y;&#xA;    if (distance &amp;gt; this.planetRadius) {&#xA;      x = (-G * this.planetMass * distanceVector.getX()) / (distance * distance * distance);   &#xA;      y = (-G * this.planetMass * distanceVector.getY()) / (distance * distance * distance);&#xA;    } else {    // inside a planet gravity decreases linearly with the distance from the center&#xA;      double magnitude = G * this.planetMass * distance/Math.pow(this.planetRadius, 3.0);&#xA;      x = -magnitude * distanceVector.getX() / distance;&#xA;      y = -magnitude * distanceVector.getY() / distance;         &#xA;    }&#xA;&#xA;    PhysicsVector gravity = new PhysicsVector();&#xA;&#xA;    gravity.setVector(x, y);                //setVector is in PhysicsVector class and makes vector &#xA;&#xA;    return gravity;&#xA;  }&#xA;&#xA;  public double magnitude(PhysicsVector projectile){&#xA;    double magnitudeOfGravField = this.gravityAcceleration(projectile).magnitude();  //again magnitude is in PhysicsVector class&#xA;    return magnitudeOfGravField;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you can create your world at any position and calculate the gravity vector in every point in space, or only its magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you had two worlds, you just need to instantiate two GravFields and you can calculate the total field on your projectile adding the fields resulting from the two worlds you instantiated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enjoy!&lt;/p&gt;&#xA;" OwnerUserId="18255" LastActivityDate="2015-11-17T18:34:32.117" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21332" PostTypeId="1" AcceptedAnswerId="21333" CreationDate="2015-11-17T23:02:14.257" Score="2" ViewCount="684" Body="&lt;p&gt;I am looking at Trefethen and Bau Exercise 37.1:&#xA;I have two normalizations of the Legendre polynomials with corresponding recurrence relations:&#xA;$$P_n(1)=1$$&#xA;which follows&#xA;$$P_n(x) = \frac{2n-1}{n} x P_{n-1}(x) -\frac{n-1}{n}P_{n-2}(x)$$&#xA;and&#xA;$$\|q_n\|=1$$&#xA;which follows&#xA;$$xq_n(x)=\beta_{n-1}q_{n-1}(x)+\alpha_n q_n(x) + \beta_n q_{n+1}(x)$$&#xA;where&#xA;$$\beta_n = \tfrac12 (1-(2n)^{-2})^{-1/2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently, $q_{n+1}$ is proportional to $P_n$. I know that the entries of the Jacobi matrix $t_{ij} = (q_i,xq_j)$ in the $L^2([-1,1])$ inner-product space, and I managed to get the right answer for the $q_j$ with $0$ along the main diagonal, $\beta_{j-1}$ on the upper diagonal and $\beta_j$ along the lower diagonal. However, it doesn't seem that I am getting the right answer for the 1st normalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also,  I am asked to find the relationship between the two Jacobi matrices--my intuition is that they are equal but I can't quite justify it--I think it has to do with the fact that the characteristic polynomials are the same monic polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, I am asked to find a formula for $q_{n+1}(1)=\|P_n\|$ using the Jacobi matrices. I've tried to solve this by equating the entries for the Jacobi matrices (where one includes a term of $||P_n||^2$) and solving. But I can't get the answer $\|P_n\|=\sqrt{\frac{2}{2n+1}}$.&lt;/p&gt;&#xA;" OwnerUserId="18265" LastEditorUserId="713" LastEditDate="2015-11-18T01:00:32.570" LastActivityDate="2016-11-12T18:08:59.587" Title="How to compute the Jacobi matrix (tridiagonal matrix) of a polynomial with a recurrence relationship?" Tags="|linear-algebra|numerical-analysis|matrix|polynomials|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21333" PostTypeId="2" ParentId="21332" CreationDate="2015-11-18T01:39:04.247" Score="1" Body="&lt;p&gt;I'm just going to sketch it out. I think the key mistake is assuming&#xA;that the two matrices are going to be equal---the exercise itself&#xA;notes that $P_n$ and $q_{n+1}$ are proportional but normalized&#xA;differently. Also, I'm not sure how exactly you defined the second matrix — you mention &quot;two Jacobi matrices&quot; but the exercise just says &quot;the two tridiagonal matrices corresponding to the formulas&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $T_{i,j} = (q_i, xq_j)$ has elements $T_{n,n+1} = T_{n+1,n}&#xA;= \beta_n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tridiagonal matrix $P_{i,j}$ defined by the recurrence for $P_n$&#xA;is obtained from rewriting the recurrence as&#xA;$$ x P_n(x) = P_{n,n-1} P_{n-1}(x) + P_{n,n+1} P_{n+1}(x). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not the same as &quot;$P_{ij} = (P_i, xP_j)$&quot; — the tridiagonal matrices $P$&#xA;and $T$ are the matrix representations of &quot;multiplication by $x$&quot; in&#xA;their respective polynomial basis ($q$ or $P$). So the relationship between them comes from the fact that it's the same linear operator applied to different bases. To check, the eigenvalues of both $P$ and $T$ should be the same and be identical to the roots of the appropriate polynomial—the two matrices are similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it is sufficient to use the fact that $P_n(x) = \|P_n\| q_{n+1}(x)$:&#xA;$$ \beta_n = T_{n,n+1} = (q_n, xq_{n+1}) = \left(&#xA;  \frac{P_{n-1}}{\|P_{n-1}\|}, \frac{x P_n}{\|P_n\|} \right)&#xA; = \frac{\|P_{n-1}\|}{\|P_n\|}\, P_{n,n-1}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here getting the value of $\|P_n\|$ is just straightforward&#xA;algebra and for me it came out to $\sqrt{\frac{2}{2n+1}}$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-11-18T01:46:08.913" LastActivityDate="2015-11-18T01:46:08.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21335" PostTypeId="1" AcceptedAnswerId="21339" CreationDate="2015-11-18T13:56:19.233" Score="2" ViewCount="1114" Body="&lt;p&gt;How can I get Gauss-Lobatto points on a quadrilateral or a triangle in $x$-$y$ plane?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am only getting abscissa coordinates and weights by solving Lobatto polynomials using Lobatto quadrature. Please suggest a method to get the $x$-$y$ coordinates&lt;/p&gt;&#xA;" OwnerUserId="18275" LastEditorUserId="9667" LastEditDate="2015-11-18T23:44:39.247" LastActivityDate="2015-11-18T23:44:39.247" Title="How can i get gauss-lobatto points on a quadrilateral?" Tags="|finite-element|spectral-method|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21336" PostTypeId="1" CreationDate="2015-11-18T14:12:31.267" Score="2" ViewCount="54" Body="&lt;p&gt;I have an optimization problem with linear objective, and constraints that are all linear except for one constraint of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S^2+C^2=1$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which corresponds to elements in a rotation matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are effective ways to solve this? I have never tried sequential linear programming, but is that appropriate here?&lt;/p&gt;&#xA;" OwnerUserId="1217" LastEditorUserId="1217" LastEditDate="2015-11-18T15:15:33.463" LastActivityDate="2015-12-14T02:33:29.357" Title="linear objectives and constraint except for S^2+C^2=1" Tags="|constrained-optimization|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21337" PostTypeId="1" CreationDate="2015-11-18T14:58:54.053" Score="2" ViewCount="1356" Body="&lt;p&gt;I am struggling with convergence criteria when performing a Monte carlo simulation on a uniform distribution. Any help would be much appreciated !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I want to sample uniformly a 1D interval (for the sake of simplicity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a random number generator (in Fortran) to draw X values between 0 and 1. Then, how do i choose the number of points N such that I have a good sampling?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the expected mean ( = 0.5) and I can easily compute the average of the positions of my MC points, i.e. μ = (X_1 +... + X_N) / N. I was thinking that I could define a simple criterion such that: μ / &amp;lt; 1% for instance, in order to decide if N is large enough or not...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please can anyone tell me if there is a better way to figure this out?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot !&lt;/p&gt;&#xA;" OwnerUserId="18277" LastActivityDate="2015-11-24T18:06:33.617" Title="How to choose the number of random points in Monte Carlo simulations?" Tags="|algorithms|convergence|monte-carlo|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21338" PostTypeId="1" AcceptedAnswerId="21349" CreationDate="2015-11-18T15:32:40.250" Score="2" ViewCount="188" Body="&lt;p&gt;Let's say we want to solve for the eigen-values of a symmetric matrix of size $n$ x $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Phase 1 of the computation, the matrix is reduced to a tridigonal form using Householder/Arnoldi's reduction.The theory says that this is $O(n^3)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Phase 2 of the computation, this tridigonal form is transformed to a diagonal form by using the &lt;strong&gt;Normalized Simultaneous Iteration (NSI)&lt;/strong&gt;. The two main steps in this iteraion are matrix multiplication of tridigonal matrices and the &lt;strong&gt;qr&lt;/strong&gt; factorization of a tridigonal matrix. Each of these (if implemented specially for tridigonal systems) will be $O(n)$. Since these are to be repeated $n$ times, the total would be $O(n^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to see experimentally that the phase 1 actually reduces the time for computation. The easist way to do this is in &lt;strong&gt;MATLAB&lt;/strong&gt; which uses the &lt;strong&gt;DGEMM&lt;/strong&gt; routine for matrix multiplication and Householder reflector for &lt;strong&gt;qr&lt;/strong&gt; factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : Does the &lt;strong&gt;qr&lt;/strong&gt; algorithm and the &lt;strong&gt;DGEMM&lt;/strong&gt; used in &lt;strong&gt;MATLAB&lt;/strong&gt; take into account if the input matrix is tridigonal and optimize accordingly ? The documentation says that it differentiates between sparse and dense matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have to write a code separately for matrix multiplication of tridigonal systems and a &lt;strong&gt;qr&lt;/strong&gt; using householder reflector, will it be better than the ones provided in &lt;strong&gt;MATLAB&lt;/strong&gt; ?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this using something other than MATLAB , like a C or C++ library ?&lt;/p&gt;&#xA;" OwnerUserId="16319" LastActivityDate="2015-11-19T10:58:28.770" Title="MATLAB : Does the qr algorithm and the DGEMM used in MATLAB take into account if the input matrix is tridigonal and optimize accordingly?" Tags="|matlab|matrix|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21339" PostTypeId="2" ParentId="21335" CreationDate="2015-11-18T16:15:50.677" Score="6" Body="&lt;p&gt;For domains that are logically square or cubic (like your quadrilateral), you can use the &lt;em&gt;tensor product&lt;/em&gt; (dimension-by-dimension) approach. That is, generate the 2D Gauss-Lobatto point &lt;em&gt;matrix&lt;/em&gt; as the tensor product of your 1D Gauss-Lobatto point vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example: if your 1D Gauss-Lobatto points are $(x_1,x_2)$, then in 2D you get the following four points: $(x_i,x_j)_{i,j=1,2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generating the Gauss-Lobatto points on triangular domains is a bit more complicated. Fortunately, there are good references. Here's a couple to get the ball rolling:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The &lt;em&gt;Nodal DG methods&lt;/em&gt; book by Hesthaven and Warburton, appendix A (&lt;a href=&quot;https://books.google.it/books?id=APQkDOmwyksC&amp;amp;printsec=frontcover&amp;amp;source=gbs_ge_summary_r&amp;amp;cad=0#v=onepage&amp;amp;q&amp;amp;f=false&quot;&gt;Google Books&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;The Matlab code from said book (&lt;a href=&quot;http://www.nudg.org/&quot;&gt;link&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://math2.uncc.edu/~shaodeng/TEACHING/math5172/Lectures/Lect_15.PDF&quot;&gt;These&lt;/a&gt; lecture notes on multidimensional Gaussian quadrature.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2015-11-18T16:15:50.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21341" PostTypeId="2" ParentId="21337" CreationDate="2015-11-18T18:08:51.510" Score="3" Body="&lt;p&gt;You may not be able to determine the exact number of points required to obtain 1% error, but you can estimate the order of magnitude of points needed to obtain this accuracy.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Monte Carlo converges at a rate $O(\frac{1}{\sqrt{N}})$, where $N$ is the sample size.  This means the absolute error is bounded as $|\mu-\mu_{approx}|&amp;lt;\frac{C}{\sqrt{N}}$, where C is some constant.   Roughly speaking, this means for every additional digit of accuracy, you will need 100 times more points.  Since you need two digits of accuracy to obtain 1% accuracy (i.e. because 1% is equivalent to 0.01), you will somewhere on the order of $100*100=10^5$ points.  This estimate is not a guarantee that you will obtain 1% accuracy with exactly $10^5$ points because we don't know the constant C.  This constant depends on the properties of the function you are sampling and its order of magnitude may not be estimatable.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2015-11-18T18:08:51.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21343" PostTypeId="1" CreationDate="2015-11-19T00:14:19.120" Score="2" ViewCount="4119" Body="&lt;p&gt;I am trying to solve linear equations in c++ using the Eigen library. For the simple equation Ax=b,  I have a sparse matrix A that is n x n, and known values for b which is n x 1, but I need to find the values for x.  I tried using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = A.colPivHouseholderQr().solve(b);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this solves it for a dense matrix and since my &quot;n&quot; value is quite large, I need to find a way to do this using an implementation that is specific to sparse matrices.  I looked into using SparseLU but I've been having trouble trying figure out the syntax behind it. Eigen online says to write something like this :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SparseLU&amp;lt;SparseMatrix&amp;lt;scalar&amp;gt;, COLAMDOrdering&amp;lt;Index&amp;gt;&amp;gt; solver;&#xA;solver.analyzePattern(A);&#xA;solver.factorize(A);&#xA;x = solver.solve(b);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but when I run this, I get errors saying that they don't know what scalar and Index are.  Has anyone used SparseLU and know the syntax behind implementing it?&lt;/p&gt;&#xA;" OwnerUserId="18280" LastActivityDate="2017-03-10T11:35:24.173" Title="Solving Linear Equations using Eigen" Tags="|c++|eigen|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21344" PostTypeId="1" CreationDate="2015-11-19T03:42:17.407" Score="1" ViewCount="1002" Body="&lt;p&gt;I need to know whether my idea for my final year project could be achieved or not. If its achievable please guide me with the relevant language and other frameworks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The idea&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a piece of paper (A4) on which I draw a circle. Once it is drawn I need to calculate the area and perimeter. In order to calculate the area and perimeter I move the mouse pointer on the image borders. Then the area and perimeter is calculated. Input of the image could be fed into the application via scanning or capturing as an image.&lt;/p&gt;&#xA;" OwnerUserId="18282" LastEditorUserId="9667" LastEditDate="2015-11-19T21:07:49.087" LastActivityDate="2021-10-06T03:50:23.337" Title="Calculate the area and perimeter of a hand-drawn shape" Tags="|algorithms|computational-geometry|image-processing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21345" PostTypeId="1" AcceptedAnswerId="21388" CreationDate="2015-11-19T05:00:35.677" Score="3" ViewCount="374" Body="&lt;p&gt;I understand what Neumann BC means physically and how to imply them. However, I am not able to perfectly understand it the way it is represented mathematically as $\partial u / \partial n$ where $n$ is the outward normal vector to the boundary and $u$ is the solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the normal vector is essential for calculation over surface integral. But what are we doing here? Traction is outcome of partial derivative of the solution vector w.r.t to the normal vector? What does this mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can any one try to simplify this briefly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2015-11-24T17:24:15.150" Title="Understanding Neumann BC" Tags="|finite-element|boundary-conditions|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21347" PostTypeId="2" ParentId="21344" CreationDate="2015-11-19T07:10:24.093" Score="3" Body="&lt;p&gt;Writing my comment idea in more detail: Lets say you have your image input to your application as a black and white pixel array, for example something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OV5Yj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OV5Yj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then each black pixel can be represented with a 1 and each empty white pixel represented with a 0. The corresponding array would look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array}{c c}&#xA;  0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;  0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0\\&#xA;  1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;  1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;  0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0\\&#xA;  0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;\end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have this you can just loop through the two dimensional array and find the approximate area by counting the number of white pixels inside the drawn object. Obviously in with an image with much finer detail the array becomes larger but I think the same idea will work. Even an image that is $1000\times{}1000$ pixels will still only contain 1 million entries and only a fraction of them would be ones. The perimeter is even easer as you simple have to count the number of black pixels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of whether this is a &quot;good&quot; project I can only say that I am glad that I don't have to do it. I think the vast majority of the work you will have to put in to it is going to be in user interface design. How we go from a piece of paper to a filtered image where my algorithm suggestion will work is going to be the hard part. Then again maybe this is for a UI class? Any ways good luck.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-11-19T18:37:59.930" LastActivityDate="2015-11-19T18:37:59.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21348" PostTypeId="2" ParentId="21345" CreationDate="2015-11-19T07:32:07.117" Score="1" Body="&lt;p&gt;When we write $\partial{u}/\partial{n}$ this is just a short-hand for $\nabla{u}\dot{}n$. Thus we are taking the dot product of the gradient and the unit normal. In the case of a rectangular region, for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kr4FQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kr4FQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the unit normal on the left wall is (1,0)  (or (-1,0) depending on your definition). Thus the Neumann condition would be $\partial{u}/\partial{x}=0$ on the left hand wall. Similarily for the bottom edge the unit normal is (0,1) and so the Neumann condition would be $\partial{u}/\partial{y}=0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of the surface integral I think the Wikipedia explaination is pretty good. It says in part &lt;a href=&quot;https://en.wikipedia.org/wiki/Surface_integral&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Alternatively, if we integrate the normal component of the vector field, the result is a scalar. Imagine that we have a fluid flowing through S, such that v(x) determines the velocity of the fluid at x. The flux is defined as the quantity of fluid flowing through S per unit time. This illustration implies that if the vector field is tangent to S at each point, then the flux is zero, because the fluid just flows in parallel to S, and neither in nor out.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-11-19T07:45:43.983" LastActivityDate="2015-11-19T07:45:43.983" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21349" PostTypeId="2" ParentId="21338" CreationDate="2015-11-19T09:06:28.403" Score="4" Body="&lt;p&gt;Here's a short Matlab benchmark. I've used a sparse tridiagonal matrix. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function benchmark_qr&#xA;&#xA;maxit = 25;&#xA;time = zeros(maxit,2);&#xA;&#xA;for iter = 1:maxit&#xA;&#xA;    mdim = 500 * iter;&#xA;&#xA;    A = gallery('tridiag',mdim,-2,3,-2);&#xA;&#xA;    t_start_qr = tic;&#xA;    qr(A);&#xA;    t_elapsed_qr = toc(t_start_qr);&#xA;&#xA;    t_start_eig = tic;&#xA;    eig(A);&#xA;    t_elapsed_eig = toc(t_start_eig);&#xA;&#xA;    time(iter,1) = mdim;&#xA;    time(iter,2) = t_elapsed_qr;&#xA;    time(iter,3) = t_elapsed_eig;&#xA;&#xA;end&#xA;&#xA;figure&#xA;loglog(time(:,1),time(:,2),'rs--')&#xA;hold on&#xA;loglog(time(:,1),time(:,3),'kd--')&#xA;loglog(time(:,1),(time(:,1)/100).^2,'--')&#xA;loglog(time(:,1),(time(:,1)/100).^3,'-.')&#xA;legend('qr','eig','N^2','N^3','Location','northwest');&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results suggest that both &lt;code&gt;eig&lt;/code&gt; and &lt;code&gt;qr&lt;/code&gt; are ${\cal O}(n^2)$ for sufficiently large $n$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qhoNb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qhoNb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2015-11-19T09:06:28.403" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21351" PostTypeId="2" ParentId="20856" CreationDate="2015-11-19T16:05:59.420" Score="2" Body="&lt;p&gt;If the matrix is almost triangular, you can just update the QR to get rid of the unwanted non-zeros. You can do that by using Givens rotations (see Golub and Van-Loan, &quot;Matrix Computations&quot; 4th edition, Chapter 6.5). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use MATLAB's &quot;qrinsert&quot; and add the additional row to the QR factorization.&lt;/p&gt;&#xA;" OwnerUserId="3761" LastActivityDate="2015-11-19T16:05:59.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21352" PostTypeId="1" AcceptedAnswerId="21354" CreationDate="2015-11-19T19:55:59.540" Score="3" ViewCount="2111" Body="&lt;p&gt;I am working on a task for which the object oriented (OO) approach fits well. I am doing it in MATLAB, since I am in the prototyping phase. However, I know that later I will surely have to perform larger scale computations (my university has a cluster). On method is to use mex functions for the computational heavy parts. But it does not work well for MATLAB classes. The other option is to write the code in C++, whose OOP is different from that of MATLAB, but there are many similarities, and the idea is the same. How easy is to use then OpenMP, MPI, PETSC, etc with C++ classes to parallelize my code? The third option would be to neglect OOP, but then I sacrifice the elegance and extensibility and of my program. My questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Do you recommend me to remain with OOP, or switch to the procedural way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Which parallelization technique do you recommend me (OpenMP, MPI, PETSC, etc.)? I do not want to invest enormous amount of time in it.&#xA;I am quite skilled in MATLAB, but have only a basic knowledge in C and nothing in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From one of the comments it turned out that there is no difference if I use standard variables or objects. So to reformulate question 1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is it sure that OOP does not make my life harder when I am going to do the parallelization? I will create a specific application not a general tool; in this case how much OO C++ is difficult to learn? I won't need special data structures, just the loops, if statements and the call of parallelization libraries. Is that a viable solution just to make the class methods parallel (so that the implementation remains hidden from outside) or a complete rewriting is required?&lt;/p&gt;&#xA;" OwnerUserId="18289" LastEditorUserId="18289" LastEditDate="2015-11-19T21:06:51.767" LastActivityDate="2015-11-22T16:58:57.477" Title="Does OOP work easily and efficiently in parallel computations?" Tags="|matlab|parallel-computing|" AnswerCount="3" CommentCount="9" ClosedDate="2015-11-25T20:11:42.627" ContentLicense="CC BY-SA 3.0" />
  <row Id="21353" PostTypeId="2" ParentId="21154" CreationDate="2015-11-19T20:11:12.883" Score="3" Body="&lt;p&gt;All of the schemes I have seen are consistent.  To be concrete, the first reference essentially rewrites their original equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial U}{{\partial t}} + \nabla H(U) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;into&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial U}{{\partial t}} + \nabla H(U)  + h^3 \left( \frac{\partial}{\partial x} (\epsilon \frac{\partial ^3}{\partial x^3} U) + \frac{\partial}{\partial y} (\epsilon \frac{\partial ^3}{\partial y^3} U) \right)= 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unless $U$ is badly behaved (e.g. there is a shock which creates a discontinuity in $U$), as $h$, the mesh spacing, goes to zero, we recover the original equation.  In that way, it behaves like other resolution dependent artifacts (e.g. approximating derivatives with finite differences).  To ensure that it does not cause problems, you have to run the model at different resolutions.&lt;/p&gt;&#xA;" OwnerUserId="17532" LastActivityDate="2015-11-19T20:11:12.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21354" PostTypeId="2" ParentId="21352" CreationDate="2015-11-19T20:38:36.950" Score="3" Body="&lt;p&gt;Without knowing more about your problem, let me just say some general thoughts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can definitely parallelize OOP code using either openMP or MPI. In fact basically all of the professional/commercial software solving computational science problems take advantage of the OOP paradigm because OOP provides significant advantages in encapsulation, extensibility, maintainability, modularization, inheritance etc. I should stress that openMP and MPI are fundamentally different parallelization paradigms: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;openMP&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Shared memory paradigm&lt;/li&gt;&#xA;&lt;li&gt;Useful for speeding up code on a given processor. For example instead of using just 1 of your 4 cores on a quad care processor you can use all 4 cores.&lt;/li&gt;&#xA;&lt;li&gt;All your threads see everything - i.e. all cores can read an write to the same arrays, vectors, etc&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;MPI&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Distributed memory paradigm&lt;/li&gt;&#xA;&lt;li&gt;Useful for breaking up your problem and distributing it across many machines in a cluster.&lt;/li&gt;&#xA;&lt;li&gt;Each separate node acts like a separate computer. Your problem must be broken up and given to each node/computer in the cluster. That node then works on that data only.&lt;/li&gt;&#xA;&lt;li&gt;If you need to access data on a different node then you have to do that through communication calls between the nodes.&lt;/li&gt;&#xA;&lt;li&gt;Communication is relatively expensive and should be minimized. This coupled with the fact that the problem must be broken up means that quite often sequential algorithms must be completely re-thought to work in a distributed environment. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Hybrid&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Here we use both MPI and openMP. For example you might break up your problem and distribute it across many nodes on a cluster. Each node might have 4, 6, 8 ,16 or more cores. On each node you then use openMP to gain further speed-up.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would say that in general parallelization is not a trivial endeavor - especially with MPI. Many times you will need to completely re-think your algorithms so that they work efficiently on a distributed environment. Because of this I think it would be very challenging to learn both C++ and MPI in a short amount of time. C++ alone can take years to truly master. MPI on the other hand isn't difficult from a syntactic perspective - you may only need to know 10-15 subroutine calls - but can be difficult in terms of adjusting your algorithms to handle a distributed memory model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a beginner openMP is probably the easiest and is less likely to require fundamental algorithm changes. You have a slow for loop? You might be able to speed this section of the code up with a simple #pragma statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would definitely recommend using OOP if your code is large and you want it to be able to grow and be maintained over time. Heck even if your code is small OOP is great for creating well designed, modularized code. Learning a language like C++ - especially since you know C already - wont be too difficult (but will take time to become an expert) and it will give you dividends over the long haul. If you are serious about parallelizing your code yourself then you will want to learn both openMP and MPI eventually. In the meantime openMP is probably the easiest and if you are using gcc doesn't require installing as it operates by calling pre-processor flags.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-11-20T07:12:51.220" LastActivityDate="2015-11-20T07:12:51.220" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21355" PostTypeId="2" ParentId="21352" CreationDate="2015-11-19T21:08:19.397" Score="7" Body="&lt;p&gt;Empirically, your comment that &quot;most numerical libraries 'do not like' OOP, that is why many software are written in C or Fortran&quot; is not correct. Instead, I would say that almost all software libraries that have been written over the last 20 years are in one way or another object oriented. For example, PETSc is object oriented (even though it is written in C, but it basically uses classes, inheritance, and virtual functions under the hood), and so is the other large linear algebra library, Trilinos (written in C++). In the finite element context, libMesh, FEniCS/Dolphin, and deal.II (my own project) are all written in C++ in an object oriented way. In fact, even MPI is written in an object oriented way, where communicators are like classes and most functions operate on them (in C++ one would write them as member functions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in reality, regardless of language, the object oriented programming paradigm has long won the battle in scientific computing. This is true whether you write your code in C++ or in C/Fortran -- basically all codes of significant size use the OOP paradigm.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-11-19T21:08:19.397" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21356" PostTypeId="1" CreationDate="2015-11-19T22:17:37.363" Score="2" ViewCount="750" Body="&lt;p&gt;I am trying to model the seismic wave equation and have therefore been reading about discretization schemes and their stability. I recently came across an insightful paper on 'Galerkin FEM methods for wave equation' &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/publications/2010-waves.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.tamu.edu/~bangerth/publications/2010-waves.pdf&lt;/a&gt;. In this document and many other documents, for the newmark beta scheme, the final discretised form of wave equation is in terms of acceleration, velocity and displacement. That is how I have seen it in other documents as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pYXJ5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pYXJ5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However, I have also seen another form of Newmark scheme of wave equation, i.e., the following: (Pg 130)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cM8yl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cM8yl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.lmn.pub.ro/~daniel/ElectromagneticModelingDoctoral/Books/Computational%20EM/Bondeson-%20Computational%20%20Electromagnetics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.lmn.pub.ro/~daniel/ElectromagneticModelingDoctoral/Books/Computational%20EM/Bondeson-%20Computational%20%20Electromagnetics.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I am really confused as to how a wave equation can have 2 different forms of Newmark discretisation. I' ll be very grateful if you could shed some light on this or point me in the right direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The end purpose is to model it in fenics and DEAL 2&lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="7567" LastEditDate="2015-11-21T04:25:08.770" LastActivityDate="2015-11-21T04:25:08.770" Title="Time discretization of wave equation" Tags="|pde|finite-element|time-integration|discretization|wave-propagation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21359" PostTypeId="2" ParentId="21352" CreationDate="2015-11-20T03:03:29.747" Score="2" Body="&lt;p&gt;I think Python is also worth considering along with the languages you mention, as it has several advantages over Matlab for rapid development, but insulates you from a lot of the low-level details that come with using C/C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ecosystem for scientific computing in Python is quite rich. You mentioned PETSc, for which there are &lt;a href=&quot;https://bitbucket.org/petsc/petsc4py&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python bindings&lt;/a&gt;; there are also &lt;a href=&quot;https://bitbucket.org/mpi4py/mpi4py&quot; rel=&quot;nofollow noreferrer&quot;&gt;MPI bindings for Python&lt;/a&gt;; and that's not even scratching the surface of what you can do with Cython, Numba, etc. As for finite element methods, &lt;a href=&quot;http://fenicsproject.org/documentation/tutorial/fundamentals.html#the-poisson-equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;fenics&lt;/a&gt; is fairly popular, though by no means the only choice. I picked up Python in a matter of months, but I only felt competent in C++ after ~4 years of using it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python cannot beat optimized C/C++ code for speed, but the performance gap can be &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2137/efficiency-of-using-petsc4py-vs-c-c-fortran/2140#2140&quot;&gt;as little as 10-40%&lt;/a&gt; with tools like Numba and Cython, together with outsourcing computationally intensive code to PETSc and Trilinos. In my experience, you &lt;em&gt;will&lt;/em&gt; have to worry about memory management and other low-level details when programming in C/C++, which might not be the best use of your time if you're only interested in algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, learning Python does not impede you from learning C++; quite the contrary, the more languages you know, the easier it is to pick up new ones. A practicing computational scientist has to know C, C++, Python, and probably also Fortran and x86-64 assembly. I happen to like Lisp and OCaml too, but those are just for fun.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for object-oriented programming, using this paradigm should not, in principle, hamper the scalability or speed of a large scientific application. The principle performance concern when using OOP is the added cost of virtual function dispatch. It would take too long to talk about that in any amount of detail, but suffice it to say that this cost is often completely negligible compared to the other operations being performed in a typical scientific code, e.g. linear algebra operations. The benefits to the developer, on the other hand, are immense. If anything, good use of OO design principles can help you write parallel code more easily.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-11-22T16:58:57.477" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21360" PostTypeId="1" AcceptedAnswerId="21361" CreationDate="2015-11-20T16:11:32.167" Score="7" ViewCount="4567" Body="&lt;p&gt;I have my 3 dimensional velocity flow-field u, v and w at a given instant of time from DNS using pseudo-spectral method. I need to calculate the energy spectrum ( in Fourier space ) as a function of magnitude of wave-number, i.e. $E(k)$ as a function of $k$. The equation for energy spectrum I used is as follows:&#xA;$$&#xA;E(k) = \int \int \hat{u}_i \hat{u}_i^{*} dA(k)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $dA(k) = 4 \pi k^{2}dk$, $\hat{u}_i$ is the FFT of $\hat{u}_{i}$ and $\hat{u}_i^{*}$ is the transpose conjugate of $\hat{u}_{i}$. $ k = k_{x}^2 + k_{y}^2 + k_{z}^2 $, where $k_{i}$ is the wavenumber along i. It runs from $-N/2$ to $N/2-1$. FFTs were taken using FFTW. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically, I created 2 3-dimensional arrays &lt;code&gt;E&lt;/code&gt; the energy spectrum and &lt;code&gt;mk&lt;/code&gt; the magnitude of wavenumber that span the entire 3D domain. They are initialized as follows ($nx$, $ny$, $nz$ are number of points along $x$, $y$ and $z$ axes ) :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do k = 1, nz&#xA;  do j = 1, ny&#xA;    do i = 1, nx&#xA;      mk(i,j,k) = dsqrt(kx(i)**2 + ky(j)**2 + kz(k)**2)&#xA;      E(i,j,k) = 4*pi*(mk(i,j,k)**2)*(dconjg(uhat(i,j,k))*uhat(i,j,k)+dconjg(vhat(i,j,k))*vhat(i,j,k)+dconjg(what(i,j,k))*what(i,j,k))&#xA;    end do&#xA;  end do&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now comes the confusing part. I need to perform a technique known as 'binning'. This involves dividing the wavenumber range into suitably equal parts and taking the average of the energy that falls into each of the parts. To do this, I collapsed the 2 3D arrays &lt;code&gt;E&lt;/code&gt; and &lt;code&gt;mk&lt;/code&gt; into 1D arrays of length &lt;code&gt;1:nx*ny*nz&lt;/code&gt;. Then, I sorted E in ascending order of mk ( sorting the energy spectrum in increasing order of wavenumber magnitude ). Finally, I added successive values of &lt;code&gt;E&lt;/code&gt; between the range of a particular bin, divided by number of values added and wrote to file for plotting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/djfWe.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/djfWe.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Red curve denotes correct result. The green one is the result produced by my code. As can be seen, the green one shows the general trend but is not adequate. There are a couple of spikes also that occur. Can anyone point the discrepancy in my procedure ? I am willing to share code on request.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT : Upon implementing James' suggestion, my results have significantly improved. (Additionally, I had also made a mistake when digitizing the results from the reference literature). The improved results are shown below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But still, the reference curve gets to go a few wavenumbers more than my curve. For the next wavenumber my code produces a spike.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R4YAd.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R4YAd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15126" LastEditorUserId="15126" LastEditDate="2015-11-21T06:47:29.717" LastActivityDate="2019-06-06T23:27:27.913" Title="computing turbulent energy spectrum from isotropic turbulence flow field in a box" Tags="|fluid-dynamics|numerical-analysis|fortran|fftw|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="21361" PostTypeId="2" ParentId="21360" CreationDate="2015-11-20T19:02:52.573" Score="6" Body="&lt;p&gt;One issue causing the jagged spectra at high wavenumbers is under sampling there. For example consider the 2D analogue of your binning procedure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zlCnN.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zlCnN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't want to sample from the red zones as they will become increasingly under-sampled as you move past a radius of size $|k_{x}|=|k_{y}|$.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-11-20T19:10:21.073" LastActivityDate="2015-11-20T19:10:21.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21362" PostTypeId="1" CreationDate="2015-11-20T21:19:30.900" Score="3" ViewCount="362" Body="&lt;p&gt;I tried to discretize an image into black and white and came into some difficult.  The difference between the letters and paper is pretty clear to our eyes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5VrFQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5VrFQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However a simple thresholding trick doesn't work.  Here we move everything below 0.4 intensity to 0 and everything above 0.4 to 1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kp8jX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kp8jX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's try moving the threshhold to 0.5 and some nasty artifacts occur:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jiVsF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jiVsF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish I could take the &quot;best of both worlds&quot; of trial 1 or trial 2.  Here is the Python code I used... basically the introductory tutorial with a few changes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from skimage import io&#xA;from skimage import color&#xA;import numpy as np&#xA;&#xA;import matplotlib.pyplot as plt&#xA;%matplotlib inline&#xA;plt.rcParams['figure.figsize'] = 7,7&#xA;&#xA;poem = io.imread('IMG_20151120_113157.jpg')&#xA;io.imshow(poem)&#xA;io.show()&#xA;&#xA;poem_gray = color.rgb2gray(poem)&#xA;&#xA;t = 0.5&#xA;poem_gray[poem_gray &amp;lt; t] = 0&#xA;poem_gray[poem_gray &amp;gt; t] = 1&#xA;io.imshow(poem_gray)&#xA;io.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is the histogram for black and white in my image, to justify my threshhold of about &lt;strong&gt;0.45&lt;/strong&gt;  My eyes are playing tricks on me!! Some of the &quot;white section&quot; is as dark as the text.  Is there a more standard method for separating grayscale images?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iK6om.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iK6om.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="4006" LastEditDate="2015-11-20T21:35:36.600" LastActivityDate="2015-12-01T20:18:51.577" Title="How to separate text from the paper on a black and white page?" Tags="|python|discretization|image-processing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21364" PostTypeId="1" AcceptedAnswerId="24071" CreationDate="2015-11-20T22:55:43.583" Score="1" ViewCount="183" Body="&lt;p&gt;Consider a thermo-mechanical coupled problem, where coupling exists from both the sides, mechanical loading producing thermal effects and vice versa. In such a case, is it necessary to always prescribe a thermal boundary condition along with a mechanical boundary condition on the same boundary for the problem to give valid results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example,  when prescribing mechanical load with temperature or prescribing heat flux with displacement. &lt;/p&gt;&#xA;" OwnerUserId="16565" LastEditorUserId="16565" LastEditDate="2016-01-26T21:07:50.560" LastActivityDate="2016-05-27T14:58:39.107" Title="BCs in a coupled problem" Tags="|finite-element|boundary-conditions|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21365" PostTypeId="2" ParentId="21362" CreationDate="2015-11-21T01:42:14.327" Score="6" Body="&lt;p&gt;I believe the best approach here is to use a threshold based on the local average brightness of the image. Setting the threshold to be 90% of the mean value of the 11x11 grid surrounding each pixel gives results that are about as good as you can expect with such a low resolution image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/w5lSh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/w5lSh.png&quot; alt=&quot;Thresholded black and white image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each pixel you just need to compute the mean brightness of the pixels near it. Then, if the pixel's brightness is less than 0.9 (or some threshold of your choosing) times the mean set it to black, otherwise set it to white.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-11-21T01:47:30.360" LastActivityDate="2015-11-21T01:47:30.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21366" PostTypeId="1" AcceptedAnswerId="21369" CreationDate="2015-11-21T07:55:12.373" Score="1" ViewCount="112" Body="&lt;p&gt;I've been working on this code for some hours and it seems I'm doing something wrong which I just can't figure out. I have a function which I integrate and then plot it's values. But the graph I get looks strange. I expected some form of an exponential function but the one I get is far away from my anticipations, and when I try changing some of my constants, it does still behave in a strange manner. This is how the code looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y = 0:0.5:D;&#xA;p = 65.5;&#xA;k = 1;&#xA;D =10:0.5:100; %Create a vector for D&#xA;&#xA;M =zeros(1,181); %an array&#xA;for j = k:181 %For loop for all water values&#xA;&#xA;  M(k) = integral(@(y) Func5(p,D,k,y),0,D(k));&#xA;  k+1;  &#xA;end&#xA;&#xA;plot(D,M)&#xA;title('Hydrostatic pressure in bars');&#xA;xlabel('water level in fot');&#xA;ylabel('pressure i bars');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is the code for my function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function FD = Func5(p,D,k,y)&#xA; w = 40-20*exp(-(0.01*y))*2; %Width of the dam&#xA; FD = p.*(D(k)-y).*w; %Claculate the pressure&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any help.&lt;/p&gt;&#xA;" OwnerUserId="18302" LastEditorUserId="7567" LastEditDate="2015-11-21T09:06:54.080" LastActivityDate="2015-11-21T14:48:21.137" Title="Integrating a function and then plotting it's graph in Matlab" Tags="|matlab|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21367" PostTypeId="2" ParentId="21364" CreationDate="2015-11-21T11:27:53.547" Score="0" Body="&lt;p&gt;It depends on what you understand on valid results. I would say that if for instance you want to determine how much of your strain energy dissipates to heat it would be sufficient to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Periodic_boundary_conditions&quot; rel=&quot;nofollow&quot;&gt;periodic boundary conditions&lt;/a&gt; for the strain on a defined hull of your representative volume element and embed your element into a matrix so that the produced heat is transferred to the matrix.&lt;/p&gt;&#xA;" OwnerUserId="18305" LastEditorUserId="9667" LastEditDate="2016-01-21T22:47:46.233" LastActivityDate="2016-01-21T22:47:46.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21368" PostTypeId="2" ParentId="20951" CreationDate="2015-11-21T12:17:34.790" Score="1" Body="&lt;p&gt;You can also use the Bhattachariyya distance, which is very easy to compute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.m.wikipedia.org/wiki/Bhattacharyya_distance&quot; rel=&quot;nofollow&quot;&gt;https://en.m.wikipedia.org/wiki/Bhattacharyya_distance&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3761" LastActivityDate="2015-11-21T12:17:34.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21369" PostTypeId="2" ParentId="21366" CreationDate="2015-11-21T14:48:21.137" Score="2" Body="&lt;p&gt;I think the main problem you encounter is that your for-loop probably doesn't do what you expect.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = 1;&#xA;M =zeros(1,181);&#xA;for j = k:181 &#xA;  M(k) = integral(@(y) Func5(p,D,k,y),0,D(k));&#xA;  k+1;  # does not increase k&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem here is that the statement &lt;code&gt;k+1&lt;/code&gt; is useless and does not increase &lt;code&gt;k&lt;/code&gt; which i think is what you are after. The problem with this is that &lt;code&gt;j&lt;/code&gt; is still increased each iteration but the values of the integral are assigned to &lt;code&gt;M(k)&lt;/code&gt; which remain &lt;code&gt;M(1)&lt;/code&gt; rather than increase with the loop. Also the argument &lt;code&gt;D(k)&lt;/code&gt; remains unchanged which i guess should instead change with the iteration aswell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;try this instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M =zeros(1,181);&#xA;for j = 1:181&#xA;  M(j) = integral(@(y) Func5(p,D,j,y),0,D(j));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this give me atleast a graph which resembles some sort of exponential function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some general advice about your code: try using comments which add useful information to improve readability. &lt;code&gt;M =zeros(1,181); %an array&lt;/code&gt; is not a useful comment because i can already see this is 'an array', what i cant see is what &lt;code&gt;M&lt;/code&gt; means; a better comment would be &lt;code&gt;% preallocate the results vector&lt;/code&gt; , the same applies for &lt;code&gt;D =10:0.5:100; %Create a vector for D&lt;/code&gt;; what is &lt;code&gt;D&lt;/code&gt;? and for that matter all other variables? Often it is also better to use meaningful variable names like &lt;code&gt;results&lt;/code&gt; or &lt;code&gt;pressure&lt;/code&gt; rather than &lt;code&gt;D&lt;/code&gt; and &lt;code&gt;M&lt;/code&gt;. Furthermore, in a function like &lt;code&gt;Func5&lt;/code&gt; it is useful to include a larger comment block at the top to say what the function calculates (which equation) and what its in/outputs are.&lt;/p&gt;&#xA;" OwnerUserId="18017" LastActivityDate="2015-11-21T14:48:21.137" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21370" PostTypeId="1" CreationDate="2015-11-21T16:51:09.227" Score="1" ViewCount="131" Body="&lt;p&gt;If cardinal basis functions are used (i.e. $\psi_{ij}=1$ iff $i=j$, and 0 otherwise) in interior penalty methods for elliptic equations such as SIPG &amp;amp; NIPG, shoudn't the jump in basis functions when computing surface integrals be [$\psi$] = 0 ? If that is the case the second term (consistency term) is always zero, right? How about the penalty term that has $[\psi]$ ? I am sure I am misunderstanding something here. For the poisson equation&#xA;$-\nabla. (a\nabla u)=0\  \text{in} \  \Omega$ with $u=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the symbol v for the basis functions $\psi$ ,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_{\epsilon}(u,v)=l(v)$$ where  $$a_{\epsilon}(u,v)=\sum_{K\in T_h}\int_K a\nabla u\cdot \nabla v-\sum_{e\in \Gamma_h\cup \Gamma_D}\int_e\{a\nabla u\cdot n_e\}[v]+ \sum_{e\in \Gamma_h\cup \Gamma_D}\int_e\epsilon\{a\nabla v\cdot n_e\}[u] + \sum_{e\in \Gamma_h\cup \Gamma_D}\frac{\sigma_0}{|e|}\int_e[u][v]$$&lt;/p&gt;&#xA;" OwnerUserId="18132" LastEditorUserId="18132" LastEditDate="2015-11-21T20:34:12.093" LastActivityDate="2015-11-21T20:34:12.093" Title="understanding interior penalty jump of basis function" Tags="|finite-element|numerical-analysis|discontinuous-galerkin|elliptic-pde|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="21372" PostTypeId="1" AcceptedAnswerId="21373" CreationDate="2015-11-22T06:27:52.900" Score="12" ViewCount="7269" Body="&lt;p&gt;When reading literatures about finite element method, the term &quot;hanging nodes&quot; can often be encountered. Could anyone tell me what indeed is a hanging node?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="23447" LastEditDate="2017-03-05T21:36:43.023" LastActivityDate="2017-03-05T21:36:43.023" Title="What is a &quot;hanging node&quot; in the finite element meshing?" Tags="|finite-element|mesh-generation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21373" PostTypeId="2" ParentId="21372" CreationDate="2015-11-22T06:42:34.167" Score="17" Body="&lt;p&gt;The following picture illustrates a mesh with a hanging node and a mesh containing no hanging node:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nJhy5.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nJhy5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually with a finite element mesh the vertices are shared with their other neighbouring elements, but the circled node does not belong to the bottom triangle. We call this node a hanging node. This commonly occurs during the process of adaptive mesh refinement. Hanging nodes are either removed - as is done in the above picture by connecting it to another vertex and thus creating two new elements - or by imposing constraints on the system.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-11-22T07:05:16.483" LastActivityDate="2015-11-22T07:05:16.483" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21375" PostTypeId="1" AcceptedAnswerId="21378" CreationDate="2015-11-22T10:04:06.670" Score="2" ViewCount="278" Body="&lt;p&gt;When learning the deal.II FE library, I am a bit confused about the mechanism of its &quot;SparsityPattern&quot; class. Through reading the documentation, I only got to know that it uses the Compressed Row Storage format to store indices of nonzero entries of a sparse matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put my confusion be explicit, suppose I have a quare 10x10 sparse matrix which stores the values corresponding to 10 degrees of freedom, namely &lt;code&gt;dof_handler.n_dofs = 10&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;\begin{pmatrix}&#xA;1 &amp;amp; 2 &amp;amp; 0 &amp;amp;  0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0\\&#xA;3 &amp;amp; 4 &amp;amp; 5 &amp;amp;  0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0\\&#xA;0 &amp;amp; 6 &amp;amp; 7 &amp;amp;  8 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 9 &amp;amp; 10 &amp;amp; 11 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 12 &amp;amp; 13 &amp;amp; 14 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0  &amp;amp; 15 &amp;amp; 16 &amp;amp; 17 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 18 &amp;amp; 19 &amp;amp; 20 &amp;amp; 0  &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 21 &amp;amp; 22 &amp;amp; 23 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp;  0  &amp;amp; 24 &amp;amp; 25 &amp;amp; 26\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp;  0  &amp;amp; 0  &amp;amp; 27 &amp;amp; 28\\&#xA;\end{pmatrix}&#xA;&#xA;&lt;p&gt;But at first, suppose we don't know exactly where the nonzero entries are and can only get a maximal estimate of their number at each row, say &lt;code&gt;max_per_row&lt;/code&gt;, and let &lt;code&gt;max_per_row = 5&lt;/code&gt;. Then the sparsity pattern would be:&lt;/p&gt;&#xA;&#xA;\begin{pmatrix}&#xA;X &amp;amp; X &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; X &amp;amp; X &amp;amp; X\\&#xA;X &amp;amp; X &amp;amp; X &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; X &amp;amp; X\\&#xA;X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X\\&#xA;X &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; X &amp;amp; X &amp;amp; X &amp;amp; X\\&#xA;X &amp;amp; X &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; X &amp;amp; X &amp;amp; X\\&#xA;\end{pmatrix}&#xA;&#xA;&lt;p&gt;One way to get the sparsitypattern is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SparsityPattern  sparsity_pattern;&#xA;sparsity_pattern.reinit(10, 10, 5);&#xA;DoFTools::make_sparsity_pattern(dof_handler, sparsity_pattern);&#xA;sparsity_pattern.compress()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;dof_handler&lt;/code&gt; stores the information of degrees of freedom. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it can also be implemented as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DynamicSparsityPattern dynamic_pattern (dof_handler.n_dofs());&#xA;DoFTools::make_sparsity_pattern (dof_handler, dynamic_pattern);&#xA;constraints.condense (dynamic_pattern);&#xA;SparsityPattern sp;&#xA;sp.copy_from (dynamic_pattern);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*1) Here, when we call &lt;code&gt;sparsity_pattern.reinit(10, 10, 5)&lt;/code&gt;, does it create an empty 10x10 matrix (2D vector) or just a 1x100 vector storing the values row by row? And if it is 10x10 or 1x100, then what is the functionality of third parameter &lt;code&gt;max_per_row=5&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;*2) I think &lt;code&gt;sparsity_pattern.reinit(10, 10, 5)&lt;/code&gt; creates a 10x10 matrix and then using &lt;code&gt;DoFTools::make_sparsity_pattern(dof_handler, sparsity_pattern)&lt;/code&gt; to create a CRS format, i.e, two vectors storing the row columm index of nonzero entries at each row and the index starting a new row respectively, according to the dof information stored in &lt;code&gt;dof_handler&lt;/code&gt;. Does this understanding make sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*3) I know in C++ STL, we can compress a vector to its actuall capacity using its member function &lt;code&gt;vector.shrink_to_fit&lt;/code&gt;. But here if &lt;code&gt;DoFTools::make_sparsity_pattern&lt;/code&gt; has already make a compressed format of 5 entries at each row (according to *2), how can it be compressed again using &lt;code&gt;sparsity_pattern.compress()&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*4) Does &lt;code&gt;DynamicSparsityPattern dynamic_pattern (dof_handler.n_dofs())&lt;/code&gt; creates a &lt;code&gt;(n_dofs) x (n_dofs)&lt;/code&gt; matrix, then what's the difference between &lt;code&gt;DoFTools::make_sparsity_pattern(dof_handler, sparsity_pattern)&lt;/code&gt; and &lt;code&gt;DoFTools::make_sparsity_pattern (dof_handler, dynamic_pattern)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*5) What about &lt;code&gt;constraints.condense (dynamic_pattern)&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;*6) In C++ STL, we know that we can copy a vector &lt;code&gt;v1&lt;/code&gt; of size 4, capacity 8 to a vector &lt;code&gt;v2&lt;/code&gt;, then &lt;code&gt;v2&lt;/code&gt; will be of capacity 4. In this way, can can also compress a vector to its actuall capacity. Is it the same mechanism to use `sp.copy_from (dynamic_pattern)'?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know these questions are so dummy to experienced users of deal.II. But to green learners as me, they are really great hurdles to jump. I sincerely hope someone could be so kind to give some help, and any comments would be greatly appreciated. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="18224" LastEditDate="2015-11-23T07:23:22.563" LastActivityDate="2015-11-23T09:13:09.063" Title="Topics about the deal.II finite element library class &quot;SparsityPattern&quot;" Tags="|c++|sparse-matrix|data-storage|deal.ii|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21376" PostTypeId="1" AcceptedAnswerId="21377" CreationDate="2015-11-22T14:22:51.173" Score="0" ViewCount="144" Body="&lt;p&gt;I have written a code in which I find the approximation of the solution of &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21316/points-on-the-interface&quot;&gt;this elliptic problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I calculated the error using the following part of code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://pastebin.com/7b5mmuRW&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pastebin.com/7b5mmuRW&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I get the following errors for $a=-1, b=1, k_1=k_2=1$ :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;For N=10:  er = 9.9920e-016&#xA;For N=20: er = 2.4425e-015&#xA;For N=30: er = 7.1054e-015&#xA;For N=40: er = 7.7716e-015&#xA;For N=50: er = 1.5765e-014&#xA;For N=60: er = 1.6764e-014&#xA;For N=70: er = 1.0436e-014 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the error should decrease while $N$ increases. &#xA;What have I done wrong at the code above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:Using this code: &lt;a href=&quot;http://pastebin.com/crS4vb1t&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pastebin.com/crS4vb1t&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the following results: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;[rate,Error]=order&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;rate =&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.2221   -2.0880   -0.2637   -2.7620&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Error =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e-014 *&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.0281    0.0241    0.1024    0.1230    0.8342&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $N=4$, $k_1=k_2=1$ the matrix A, with the code that I have written, is the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Columns 1 through 14&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-4     1     0     0     1     0     0     0     0     0     0     0     0     0&#xA; 1    -4     1     0     0     1     0     0     0     0     0     0     0     0&#xA; 0     1    -4     1     0     0     1     0     0     0     0     0     0     0&#xA; 0     0     1    -4     0     0     0     1     0     0     0     0     0     0&#xA; 1     0     0     0    -4     1     0     0     1     0     0     0     0     0&#xA; 0     1     0     0     1    -4     1     0     0     1     0     0     0     0&#xA; 0     0     1     0     0     1    -4     1     0     0     1     0     0     0&#xA; 0     0     0     1     0     0     1    -4     0     0     0     1     0     0&#xA; 0     0     0     0     1     0     0     0    -4     1     0     0     1     0&#xA; 0     0     0     0     0     1     0     0     1    -4     1     0     0     1&#xA; 0     0     0     0     0     0     1     0     0     1    -4     1     0     0&#xA; 0     0     0     0     0     0     0     1     0     0     1    -4     0     0&#xA; 0     0     0     0     0     0     0     0     1     0     0     0    -4     1&#xA; 0     0     0     0     0     0     0     0     0     1     0     0     1    -4&#xA; 0     0     0     0     0     0     0     0     0     0     1     0     0     1&#xA; 0     0     0     0     0     0     0     0     0     0     0     1     0     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Columns 15 through 16&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 0     0&#xA; 0     0&#xA; 0     0&#xA; 0     0&#xA; 0     0&#xA; 0     0&#xA; 0     0&#xA; 0     0&#xA; 0     0&#xA; 0     0&#xA; 1     0&#xA; 0     1&#xA; 0     0&#xA; 1     0&#xA;-4     1&#xA; 1    -4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: Here is the log-log graph of the errors in relation to $N$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FvxKa.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FvxKa.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the errors can't be right. Or am I wrong?&lt;/p&gt;&#xA;" OwnerUserId="9422" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-11-30T20:54:15.417" Title="Calculation of error" Tags="|matlab|finite-difference|numerical-analysis|error-estimation|elliptic-pde|" AnswerCount="1" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="21377" PostTypeId="2" ParentId="21376" CreationDate="2015-11-22T17:42:56.693" Score="4" Body="&lt;p&gt;After a many days of discussions, your problems are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) you choose too simple problem for which you reach the machine zero and therefore cannot observe normal properties&#xA;2) perhaps you do not use the boundary values correctly. With the scheme you work the values should be added to the RHS, but since they are zero there is nothing to add. However, one thing to remember, the boundaries points shouldn't be a part of computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the following code, for the two problems (two different sources), one the simple one you worked with and the second one I suggested to solve in comments.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for Kind=1:2 % define what problem to solve&#xA;    errpre=0;%just to initialize this with something&#xA;&#xA;    for N = 1:7&#xA;&#xA;        %create grid&#xA;        n=2^(N+1);&#xA;        x=linspace(-1,1,n+2);&#xA;        y=linspace(-1,1,n+2);&#xA;        h= abs(x(1)-x(2));&#xA;        [X,Y]=meshgrid(x(2:end-1),y(2:end-1));&#xA;&#xA;        %create descrete operator&#xA;        I = speye(n,n);&#xA;        E = sparse(2:n,1:n-1,1,n,n);&#xA;        D = E+E'-2*I;&#xA;        A = (kron(D,I)+kron(I,D))./(h^2);&#xA;&#xA;        %create RHS and the exact solution&#xA;        if Kind==1&#xA;            f=2* ((X.^2 - 1) + (Y.^2 - 1));&#xA;            ex=(X.^2 - 1) .*(Y.^2 - 1);&#xA;        elseif Kind==2&#xA;            ex = sin(pi*X).*sin(pi*Y);&#xA;            f=-ex*2*pi^2;&#xA;        end&#xA;        %solve the problem&#xA;        ap = (A\f(:));&#xA;&#xA;        %compute the error&#xA;        err  = norm(ex(:)-ap(:),inf)/norm(ap(:),inf);&#xA;&#xA;        %print the error and the rate&#xA;        fprintf('Kind=%d\t n=%d\t err=%-10.8d\t rate=%-4.2f\n',Kind,n,err,log2(errpre/err));&#xA;&#xA;        %keep the old value for the rate calculation at next step&#xA;        errpre=err;&#xA;&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run this code to get &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Kind=1   n=4     err=2.40933816e-16  rate=-Inf&#xA;Kind=1   n=8     err=5.69076036e-16  rate=-1.24&#xA;Kind=1   n=16    err=8.94357034e-16  rate=-0.65&#xA;Kind=1   n=32    err=3.11434148e-15  rate=-1.80&#xA;Kind=1   n=64    err=1.11074876e-15  rate=1.49&#xA;Kind=1   n=128   err=3.77521199e-15  rate=-1.77&#xA;Kind=1   n=256   err=1.42112850e-14  rate=-1.91&#xA;&#xA;Kind=2   n=4     err=1.24859800e-01  rate=-Inf&#xA;Kind=2   n=8     err=3.99615153e-02  rate=1.64&#xA;Kind=2   n=16    err=1.13319182e-02  rate=1.82&#xA;Kind=2   n=32    err=3.01735099e-03  rate=1.91&#xA;Kind=2   n=64    err=7.78424525e-04  rate=1.95&#xA;Kind=2   n=128   err=1.97680908e-04  rate=1.98&#xA;Kind=2   n=256   err=4.98085147e-05  rate=1.99&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18215" LastActivityDate="2015-11-22T17:42:56.693" CommentCount="20" ContentLicense="CC BY-SA 3.0" />
  <row Id="21378" PostTypeId="2" ParentId="21375" CreationDate="2015-11-22T18:34:07.557" Score="2" Body="&lt;p&gt;I will try answer based on my experience with deal.ii. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The &lt;code&gt;max_per_row=5&lt;/code&gt; means that at most there will be 5 non-zeros per row in the matrix. Since we now know this then we do not need to have a $1\times{}100$ matrix but rather a $1\times{}50$. In other words this parameter sets an upper-bound on the memory needed. In reality it is not stored as one vector but rather as two: a row pointer vector and a column indicies vector in accordance with CRS format. See 2).&lt;/li&gt;&#xA;&lt;li&gt;Yes the sparsity pattern holds the row pointers and the column indices as two vectors.&lt;/li&gt;&#xA;&lt;li&gt;We must potentially compress again because while we set &lt;strong&gt;maximum&lt;/strong&gt; of 5 entries per row, there may in fact be less for any given row. One row might have 5 while a different row might only have 3. This results in extra entries in the column index vector that should be removed by compression. Note that these extra entries in the column index vector are often written as $-1$'s since a column index can never be negative and thus if it is $-1$ we know that no non-zero has been assigned for that column index.&lt;/li&gt;&#xA;&lt;li&gt;As the &lt;a href=&quot;https://www.dealii.org/8.3.0/doxygen/deal.II/classDynamicSparsityPattern.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; says the &lt;code&gt;DynamicSparsityPattern&lt;/code&gt; is used to find the sparsity pattern while being compressed at all times. This reduces memory overhead at the expense of cpu time.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This class acts as an intermediate form of the SparsityPattern class. From the interface it mostly represents a SparsityPattern object that is kept compressed at all times. However, since the final sparsity pattern is not known while constructing it, keeping the pattern compressed at all times can only be achieved at the expense of either increased memory or run time consumption upon use. The main purpose of this class is to avoid some memory bottlenecks, so we chose to implement it memory conservative. The chosen data format is too unsuited to be used for actual matrices, though. It is therefore necessary to first copy the data of this object over to an object of type SparsityPattern before using it in actual matrices.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Another viewpoint is that this class does not need up front allocation of a certain amount of memory, but grows as necessary. An extensive description of sparsity patterns can be found in the documentation of the Sparsity patterns module.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://www.dealii.org/8.3.0/doxygen/deal.II/classConstraintMatrix.html#a4c543aa9c09a2eeb1f6c89fcd6e67f0c&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; has this to say:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Condense a sparsity pattern. The name of the function mimics the name of the function we use to condense linear systems, but it is a bit of a misnomer for the current context. This is because in the context of linear systems, we eliminate certain rows and columns of the linear system, i.e., we &quot;reduce&quot; or &quot;condense&quot; the linear system. On the other hand, in the current context, the functions does not remove nonzero entries from the sparsity pattern. Rather, it adds those nonzero entry locations to the sparsity pattern that will later be needed for the process of condensation of constrained degrees of freedom from a linear system.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Since this function adds new nonzero entries to the sparsity pattern, the given sparsity pattern must not be compressed. The constraint matrix (i.e., the current object) must be closed. The sparsity pattern is compressed at the end of the function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;6&quot;&gt;&#xA;&lt;li&gt;We copy the &lt;code&gt;DynamicSparsityPattern&lt;/code&gt; to a regular &lt;code&gt;SparsityPattern&lt;/code&gt; using the copy call because apparently the dynamic one isn't in a nice enough format for later operations. How this is done internally may involve vector copies as you suggest but I am not sure.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Concrete Example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets run through your first &lt;code&gt;SparsityPattern&lt;/code&gt; code snippet (keeping in mind that the exact details of how this is done in deal.ii might differ, but the basic idea I think is correct.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SparsityPattern  sparsity_pattern;&#xA;sparsity_pattern.reinit(10, 10, 5);&#xA;DoFTools::make_sparsity_pattern(dof_handler, sparsity_pattern);&#xA;sparsity_pattern.compress()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Call &lt;code&gt;SparsityPattern  sparsity_pattern;&lt;/code&gt;. This just creates our sparsity pattern object with default constructor. The &lt;code&gt;sparsity_pattern&lt;/code&gt; object contains member variables like &lt;code&gt;row&lt;/code&gt; and &lt;code&gt;col&lt;/code&gt; both of which are vectors (currently of size 0). Note: &lt;code&gt;row&lt;/code&gt; and &lt;code&gt;col&lt;/code&gt; are names I made up. They are called something else in deal.ii.&lt;/li&gt;&#xA;&lt;li&gt;Call &lt;code&gt;sparsity_pattern.reinit(10, 10, 5);&lt;/code&gt;. Our sparsity pattern initializes the row pointer and column index vectors. This might look something like: &lt;code&gt;row.resize(10+1,0);&lt;/code&gt; and &lt;code&gt;col.resize(10*5,-1);&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Call &lt;code&gt;make_sparsity_pattern&lt;/code&gt;. This determines where the non-zeros in our sparse matrix will be, however &lt;strong&gt;some&lt;/strong&gt; of the &lt;code&gt;col&lt;/code&gt; entries will still be &lt;code&gt;-1&lt;/code&gt;'s - i.e. cases where there was less then 5 non-zeros in the row. Otherwise &lt;code&gt;row&lt;/code&gt; is correctly filled and &lt;code&gt;col&lt;/code&gt; contains both &lt;code&gt;-1&lt;/code&gt;'s and other positive column indices indicating the columns where non-zeros exist.&lt;/li&gt;&#xA;&lt;li&gt;Call &lt;code&gt;compress&lt;/code&gt;. This compresses the &lt;code&gt;col&lt;/code&gt; vector by removing any &lt;code&gt;-1&lt;/code&gt;'s.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-11-23T09:13:09.063" LastActivityDate="2015-11-23T09:13:09.063" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21382" PostTypeId="1" CreationDate="2015-11-23T16:33:24.237" Score="5" ViewCount="110" Body="&lt;p&gt;We're currently using a Chirp-Z based implementation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1109/83.784442&quot; rel=&quot;nofollow noreferrer&quot;&gt;R. W. Cox and R. Tong, &quot;Two- and three-dimensional image rotation using the FFT,&quot; &lt;em&gt;IEEE Trans. Image Processing&lt;/em&gt;, vol. 8, no. 9, pp. 1297–1299, Sep. 1999&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;for rotation of astronomical images. I was wondering if there were any algorithms/implementations out there that could also be used to rotate images using FFTs, and were more accurate than the one we're using now. Perhaps any progress made in the past few years in the area?&lt;/p&gt;&#xA;" OwnerUserId="17947" LastEditorUserId="20688" LastEditDate="2018-11-19T01:53:47.223" LastActivityDate="2018-11-19T01:53:47.223" Title="FFT-based Image Rotation Algorithms More Accurate Than Chirp-Z?" Tags="|algorithms|fourier-analysis|image-processing|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21384" PostTypeId="1" CreationDate="2015-11-23T19:27:14.897" Score="1" ViewCount="307" Body="&lt;p&gt;I'm tryin' to implement this method with python3 but i don't solve the problem. This is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python&#xA;# -*- coding: utf-8 -*-&#xA;&#xA;&#xA;import sympy&#xA;import numpy as np&#xA;&#xA;&#xA;x = sympy.symbols('x')&#xA;&#xA;expr = x ** 2 - 2&#xA;fx = sympy.lambdify(x, expr, 'numpy')&#xA;&#xA;def quad_inv(f, x0, x1, x2, tol, maxiter):&#xA;    fx0 = f(x0)&#xA;    fx1 = f(x1)&#xA;    fx2 = f(x2)&#xA;    x3 = None&#xA;    iters = 0&#xA;    error = tol + 1&#xA;    if np.sign(tol) &amp;lt; 0:&#xA;        raise ValueError('La tolerancia debe ser estrictamente positiva.')&#xA;&#xA;    while (error &amp;gt;= tol) &amp;amp; (iters &amp;lt;= maxiter):&#xA;        a = fx1 * fx2 / ((fx0 - fx1) * (fx0 - fx2)) * x0&#xA;        b = fx0 * fx2 / ((fx1 - fx0) * (fx1 - fx2)) * x1&#xA;        c = fx0 * fx1 / ((fx2 - fx0) * (fx2 - fx1)) * x2&#xA;        x3 = a + b + c&#xA;        x0 = x1&#xA;        x1 = x2&#xA;        x2 = x3&#xA;        error = abs((x3 - x0) * (x3 - x1) * (x3 - x2))&#xA;        print(error)&#xA;        iters += 1&#xA;    return[x3, iters]&#xA;&#xA; sol = quad_inv(fx, 1., 1.2, 1.6, 0.0001, 300)&#xA;    print('La raíz aproximada con el Método Inversa Cuadrática es:\n{0}'.format(sol[0]))&#xA;    print('Se han realizado un total de {0} iteraciones.'.format(sol[1]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I only get 1 iteration. I don't find anything about the error cote with this method.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;La raíz aproximada con el Método Inversa Cuadrática es:&#xA;1.4195804195804194&#xA;Se han realizado un total de 1 iteraciones.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5522" LastActivityDate="2015-11-23T19:27:14.897" Title="Inverse quadratic method only makes one iteration - python" Tags="|python|numpy|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21385" PostTypeId="1" CreationDate="2015-11-24T06:57:58.817" Score="1" ViewCount="282" Body="&lt;p&gt;I am looking at the LAPACK DPOSV routine that computes the solution to the real system of linear equations A * X = B. The routine description can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.utah.edu/software/lapack/lapack-d/dposv.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.utah.edu/software/lapack/lapack-d/dposv.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many floating point operations (flops) would a call to this routine take? What is the algorithmic complexity of this subroutine call?&lt;/p&gt;&#xA;" OwnerUserId="18328" LastEditorUserId="7567" LastEditDate="2015-11-24T10:03:36.587" LastActivityDate="2015-11-24T10:03:36.587" Title="Total Flop count for LAPACK DPOSV" Tags="|linear-algebra|lapack|intel-mkl|scalapack|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21386" PostTypeId="2" ParentId="21385" CreationDate="2015-11-24T09:04:49.230" Score="1" Body="&lt;p&gt;There is source code of this function available here: &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/dc/de9/group__double_p_osolve.html&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lapack/explore-html/dc/de9/group__double_p_osolve.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see by following the function calls in the code, it computes the Cholesky factorization of $A$ with &lt;code&gt;dpotrf&lt;/code&gt;, followed by &lt;code&gt;dpotrs&lt;/code&gt;, which solves $AX=B$ by solving two triangular systems (&lt;code&gt;dtrsm&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementations of those functions are pretty standard and should have the usual algorithmic complexities.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-11-24T09:04:49.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21388" PostTypeId="2" ParentId="21345" CreationDate="2015-11-24T17:24:15.150" Score="2" Body="&lt;h2&gt;Basic Concept&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The normal vector corresponds to a physical parameter such as cartesian coordinate $x$. For example, a flux along $x$ direction can be defined by Darcy's law as following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_x(x,t)=\frac{k}{\mu}\frac{\partial{P}}{\partial{x}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Neumann's BC, the flux should be constant or zero at the boundary i.e. $u(0,t)=u(L,t)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, $\frac{k}{\mu}\frac{\partial{P(0,t)}}{\partial{x}}=\frac{k}{\mu}\frac{\partial{P(L,t)}}{\partial{x}}=0$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Consider a system of 2 grids as shown below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5KtWe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5KtWe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we apply the above concept for this system, we can get the following equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Rightarrow \frac{P_i-P_{i-1}}{\Delta x}=\frac{P_1-P_{0}}{\Delta x}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Rightarrow P_1=P_0$&lt;/p&gt;&#xA;" OwnerDisplayName="user5510" LastActivityDate="2015-11-24T17:24:15.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21389" PostTypeId="2" ParentId="21337" CreationDate="2015-11-24T18:06:33.617" Score="0" Body="&lt;p&gt;I have not come across an &quot;explicit&quot; formulation which can define the optimum number of samples for MC simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, a good way would be to estimate the variance of simulation with each sample size, $n$, as $\sigma^2_n$. Then the standard error for $n$ samples would be $\frac{\sigma_n}{\sqrt{n}}$. You could repeat the error estimates by increasing your sample size and stop at that sample size for which the error hits a certain threshold.&lt;/p&gt;&#xA;" OwnerDisplayName="user5510" LastActivityDate="2015-11-24T18:06:33.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21390" PostTypeId="1" CreationDate="2015-11-24T19:58:06.193" Score="2" ViewCount="811" Body="&lt;p&gt;The acoustic wave equation in 2D is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial^2}{\partial t^2}p(x,z,t) = c(x,z)^2\left[\frac{\partial^2}{\partial x^2}p(x,z,t) + \frac{\partial^2}{\partial z^2}p(x,z,t)\right] + s(x,z,t) \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone please give an analytical solution to the above, not worrying about the boundary conditions (assuming infinite boundary)?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$p$ is a scalar field ( pressure )&lt;/li&gt;&#xA;&lt;li&gt;$c$ is the velocity ( a scalar value ) , same everywhere in the domain.&lt;/li&gt;&#xA;&lt;li&gt;$s(x,z,t)$ or source function is at only one point in the domain ( at the center of the 2D / 3D domain).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An IPython snippet of this code for finite differences is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# it is the time step number.&#xA;# dt is the time step size.&#xA;# src is the source&#xA;&#xA;f0 = 100.0    # dominant frequency of source (Hz)&#xA;T = 1.0 / f0  # dominant period&#xA;ist = 100     # shifting of source time function&#xA;src[it] = exp(-1.0 / T ** 2 * ((it - ist) * dt) ** 2)   # This is a Gaussian source&#xA;# Take the first derivative to finally define the source function.&#xA;src = np.diff(src) / dt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The green's function is a bit complicated for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can it be re-defined / adapted for our case of finite differences , given this :&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Acoustic_wave_equation#Cartesian_coordinates&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Acoustic_wave_equation#Cartesian_coordinates&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14662" LastEditorUserId="14662" LastEditDate="2015-11-24T23:35:10.797" LastActivityDate="2015-12-01T16:54:28.890" Title="Analytical Solution to the acoustic / scalar (Inhomogeneous) wave equation with source term" Tags="|finite-difference|wave-propagation|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21391" PostTypeId="2" ParentId="21390" CreationDate="2015-11-24T20:00:39.910" Score="2" Body="&lt;p&gt;You can make up as many solutions to this equation as you want using the &lt;a href=&quot;http://infoserve.sandia.gov/sand_doc/2000/001444.pdf&quot; rel=&quot;nofollow&quot;&gt;Method of Manufactured Solutions (PDF)&lt;/a&gt;. Is there a reason you're focused on this particular case only, or could you use any solution/forcing-function pair?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-11-24T20:00:39.910" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21392" PostTypeId="2" ParentId="21154" CreationDate="2015-11-24T21:27:18.770" Score="2" Body="&lt;p&gt;Perhaps your query is regarding &quot;artificial compressibility&quot; vs &quot;artificial dissipation/viscosity&quot;. Artificial compressibility method is a way to solve the incompressible Navier-Stokes equations where the divergence free condition is satisfied by doing &quot;inner iterations&quot;. This is an alternative to projection methods. It is unrelated to the artificial viscosity that appears (or is added) in solving the hyperbolic equation. Artificial dissipation helps in stabilizing the advection scheme and is not related to the pseudo transient continuance of the artificial compressibility method. The extra dissipation term will not be zero at the end of the inner iterations which is normal. &lt;a href=&quot;https://books.google.co.in/books?id=-GTOBQAAQBAJ&amp;amp;pg=PA644&amp;amp;source=gbs_toc_r&amp;amp;redir_esc=y#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;Pletcher, Tannenhill and Anderson&lt;/a&gt; have a section which discusses this issue.&lt;/p&gt;&#xA;" OwnerUserId="16934" LastEditorUserId="3549" LastEditDate="2015-11-29T04:01:06.250" LastActivityDate="2015-11-29T04:01:06.250" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21393" PostTypeId="1" CreationDate="2015-11-24T22:50:25.070" Score="4" ViewCount="167" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This question was previously posted to Math.SE &lt;a href=&quot;https://math.stackexchange.com/questions/1543214/how-do-i-develop-numerical-routines-for-the-evaluation-of-my-own-special-functio&quot;&gt;here&lt;/a&gt; and had received no answers at the time of this posting.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When performing computational work, I often come across a univariate function, defined in terms of an integral or differential equation, which I would like to rapidly evaluate (say, millions of times per second) over a specified interval to a given precision (say, one part in $10^{10}$). For example, the function&#xA;$$ f(\alpha) = \int_{k=0}^\infty \frac{e^{-\alpha^2 k^2}}{k+1}\ \mathrm{d}k $$&#xA;over the interval $\alpha \in (0,10)$ came up in a recent project. Now it happens that this integral can be evaluated in terms of standard special functions (in particular, $\operatorname{Ei}(z)$ and $\operatorname{erfi}(z)$), but suppose we had a much more complicated function for which no such evaluation was known. Is there a &lt;strong&gt;systematic technique&lt;/strong&gt; I can apply to develop my own numerical routines for the evaluation of such functions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sure plenty of techniques must be out there, as fast algorithms seem to exist for basically all of the common special functions. However, I emphasize that the sort of technique I am looking for should not rely on the function having some particular structure (e.g. recurrence relations like $\Gamma(n+1) = n\Gamma(n)$ or reflection formulas like $\Gamma(z) \Gamma(1-z) = \pi \csc(\pi z)$). Ideally, such a technique would work for just about any (sufficiently well-behaved) function I come across.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can take for granted that I do have some &lt;em&gt;slow&lt;/em&gt; method of evaluating the desired function (e.g. direct numerical integration) to any precision, and that I'm willing to do a lot of pre-processing work with the slow method in order to develop a fast method.&lt;/p&gt;&#xA;" OwnerUserId="11775" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2015-11-25T06:30:42.800" Title="How do I develop numerical routines for the evaluation of my own special functions?" Tags="|numerical-analysis|special-functions|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21394" PostTypeId="2" ParentId="21393" CreationDate="2015-11-24T23:57:28.280" Score="1" Body="&lt;p&gt;In general you can expect that a customized algorithm would be faster for each particular special function that you need to evaluate over various ranges- that's why there are so many different approaches used for particular special functions.  A general purpose approach probably won't be the fastest for all possible functions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For limited ranges of arguments where the function is quite smooth, you might consider computing tables of values to high accuracy and then interpolating these values using a higher order interpolation method.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-11-24T23:57:28.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21395" PostTypeId="1" CreationDate="2015-11-25T00:02:51.463" Score="3" ViewCount="157" Body="&lt;p&gt;I want to solve approximately the following variational problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a function $c:[-1,1]^2\rightarrow [0,1]$, constants $p_1...p_n\in \mathbb{R}^+$, $\alpha_1...\alpha_n\in \mathbb{R}$, and $\beta_1...\beta_n\in \mathbb{R}$ let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$V =\sup_{f_i,g_i}\left\{\sum_{i=1}^n \;\;p_i\cdot \int_0^1\int_0^1  f_i(x,y)\cdot g_i(x,y)\; dx\; dy \;\;: \sum_{i}\alpha_i\cdot f_i = c = \sum_i \beta_i\cdot g_i \right\}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the supremum is taken over all square integrable functions &#xA;$f_i,g_i:[-1,1]^2\rightarrow [-1,1]$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Observe that since all functions have domain $[-1,1]^2$ and range &#xA;$[0,1]$, the value of each integral $\int_{0}^1\int_{0}^1 f(x,y)\cdot g(x,y) \;dx\; dy$ lies between $0$ and $1$. Therefore $V$ lies between $0$ and $\sum_{i} p_i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Assumptions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All functions $f_i$ and $g_i$ and $c$ can be assumed to be square integrable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it simplifies the problem, $c:[-1,1]\rightarrow [0,1]$ can be assumed &#xA;to be $c(x,y)=1$ if $x=y$ and $c(x,y)=0$ otherwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What numerial methods are available to approximate the value of $V$ up to an $\varepsilon$ additive factor? In other words, I want to find a $V'$ such that $|V-V'|\leq \varepsilon$, where $\varepsilon$ is a given precision parameter.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the ratio of convergence of such methods?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18329" LastEditorUserId="18329" LastEditDate="2016-01-18T08:17:12.287" LastActivityDate="2016-01-18T08:17:12.287" Title="Reference Request: Variational Problem" Tags="|optimization|numerical-analysis|approximation|variational-calculus|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="21396" PostTypeId="1" CreationDate="2015-11-25T16:37:44.383" Score="1" ViewCount="48" Body="&lt;p&gt;I have a eulerian hydrodynamic code, which I have recently implemented the Coriolis force in the integrator. Is there a standard test problem for the Coriolis force?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this isn't a Lagrangian code there is no tracking of fluid particles. Right now I am thinking about periodic boundary conditions and setting the entire box to constant density with an initial velocity and watching the entire box evolve. Thankfully since the force is only dependent on velocity and the box's orbital frequency, the medium should evolve homogeneously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be nice to track an individual particle instead of just the velocity evolution of the entire box. If I created an over density, on a static background density, with a non-zero velocity it would be almost analogous. There are some obvious problems with that which make me unsure if it would be a clean test.&lt;/p&gt;&#xA;" OwnerUserId="18344" LastActivityDate="2015-11-25T16:37:44.383" Title="Coriolis test problem" Tags="|fluid-dynamics|testing|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21398" PostTypeId="1" CreationDate="2015-11-26T04:57:59.940" Score="6" ViewCount="5319" Body="&lt;p&gt;I have a system of linear equations that is derived partially from experimental data. Theoretically, the system should have a single, exact solution; however, experimental error causes it to not have an exact solution. Is there a good method to find an approximate solution to this system? My idea for a naive implementation was to solve all pairs of two equations and average the value for the solutions to each variable, but this doesn't extend well as the number of equations increases. Is there any known algorithm to do this?&lt;/p&gt;&#xA;" OwnerUserId="18349" LastEditorUserId="276" LastEditDate="2016-06-01T21:31:36.753" LastActivityDate="2016-06-01T21:31:36.753" Title="Solving a system of linear equations with only an approximate solution" Tags="|algorithms|linear-solver|approximation|linear-system|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21399" PostTypeId="2" ParentId="21398" CreationDate="2015-11-26T09:01:19.010" Score="0" Body="&lt;p&gt;You can use any of the numerical methods that is for linear equation systems. &#xA;Some of them are;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Gauss-Jordan Elimination&lt;/li&gt;&#xA;  &lt;li&gt;Jacobi Method&lt;/li&gt;&#xA;  &lt;li&gt;Gauss-Seidel&lt;/li&gt;&#xA;  &lt;li&gt;Successive Over Relaxation(SOR)&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For more info, you can look at &lt;a href=&quot;http://www.cs.tau.ac.il/~dcor/Graphics/adv-slides/Solving.pdf&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17376" LastActivityDate="2015-11-26T09:01:19.010" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21402" PostTypeId="1" CreationDate="2015-11-26T10:34:28.090" Score="9" ViewCount="4218" Body="&lt;p&gt;Double-precision calculations are significantly slower or more expensive than single-precision calculations. For example, the NVidia Tesla which performs well on doubles is much more expensive then regular GPU. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the same time I do not know about machine learning cases where double-precision is really needed. Currently I think that 64-bit floats are needed by the model only in case then it is over-fitted. In other words doubles are needed only by &quot;bad&quot; models with bad generalization.&lt;/p&gt;&#xA;" OwnerUserId="18351" LastEditorUserId="8094" LastEditDate="2015-11-26T12:52:08.027" LastActivityDate="2024-02-09T22:12:15.817" Title="Are there tasks in machine learning which require double precision floating points?" Tags="|machine-learning|precision|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21403" PostTypeId="1" CreationDate="2015-11-26T10:53:51.677" Score="1" ViewCount="94" Body="&lt;p&gt;I am having problems understanding the boundary conditions of the problem described in this &lt;a href=&quot;https://www.math.nyu.edu/~jbu200/FSI_Papers/Blom_FSI_Piston.pdf&quot; rel=&quot;nofollow&quot;&gt;paper on researchgate&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially the problem consists of a one dimensional fluid chamber in contact with a piston on the right hand side. The problem makes use of the ALE mesh moving technique. The ALE is moved in such a way that the elements remain equidistant, so essentially the mesh velocity varies linearly at all times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that on the left hand side, the velocity of the fluid and the mesh are set to zero whilst the pressure and density are linearly interpolated. On the right hand side, the fluid and mesh velocities are set equal to the piston velocity whilst pressure and density are interpolated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate some guidance on those. &lt;/p&gt;&#xA;" OwnerUserId="17476" LastActivityDate="2015-11-27T17:33:39.980" Title="problem with understanding the fluid boundary conditions of a 1D probelm" Tags="|fluid-dynamics|boundary-conditions|finite-volume|discretization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21404" PostTypeId="1" AcceptedAnswerId="21407" CreationDate="2015-11-26T11:49:42.287" Score="2" ViewCount="68" Body="&lt;p&gt;Given a matrix $M\in\mathbb{R}^{P\times P}$ , is it possible to sample $P$ vectors $u_i\in\mathbb{R}^N$, $i=1..P$ so that $\|u_i-u_j\|=M_{ij}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously for not any $M$ this is possible, i.e. it has to be symmetric, have zero diagonal, etc., but assume $M$ was created from a different set of vectors which satisfied this property.&lt;/p&gt;&#xA;" OwnerUserId="10450" LastActivityDate="2015-12-02T16:01:32.453" Title="Sampling vector so they will have a given euclidean distances matrix" Tags="|linear-algebra|optimization|constrained-optimization|random-sampling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21407" PostTypeId="2" ParentId="21404" CreationDate="2015-11-26T15:59:35.887" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Multidimensional_scaling&quot; rel=&quot;nofollow&quot;&gt;Multidimensional scaling (MDS)&lt;/a&gt; is one algorithm that tries to find such a set of points. It's usually used for visualization, so it's usually used in 2 or 3D spaces, but the optimization procedure itself can be used in arbitrary dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get random samples, you can initialize with random vectors before optimizing, though it is not clear what distribution you'd be sampling from in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDITED: Sample MATLAB code to generate $P$ patterns of size $N$ with the same distances as a reference data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data=mdscale(pdist(reference_data), N, 'Start', 'Random');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or using an Euclidean distances matrix &lt;code&gt;D&lt;/code&gt; of size $P\times P$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data=mdscale(D, N, 'Start', 'Random');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1128" LastEditorUserId="10450" LastEditDate="2015-12-02T16:01:32.453" LastActivityDate="2015-12-02T16:01:32.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21408" PostTypeId="1" CreationDate="2015-11-26T16:25:35.590" Score="1" ViewCount="188" Body="&lt;p&gt;This question is not homework it is recommended exercises to prepare for the final exam. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the family of linear one-step methods defined by $$y_n = y_{n-1} + h(\theta f_n + (1 - \theta)f_{n-1})$$ where $0\leq \theta \leq 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) For what values of $\theta$ in the interval $0\leq \theta \leq 1$ is the associated method convergent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attempted proof:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's first prove convergence of forward Euler i.e. when $\theta = 0$, the forward Euler method says that $$u_{n+1} = u_n + hf(t_n,u_n)$$ and Taylor expansion with remainders of each component gives $$ y_j(t_{n+1}) = y_j(t_n) + hf_j(t_n,y(t_n)) + \frac{1}{2}h^2y_j''(t_n + \theta_j h)$$ $$= y_j(t_n) + hf_j(t_n,y(t_n)) + h\tau_{nj}$$ for some unknown collection of numbers $\theta_j$ between $0$ and $1$. This defines the local truncation error $\tau_n$, the amount by which the exact solution $y$ fails to satisfy the method after one step. Subtraction gives a difference equation for the error $$e_{n+1} = u_{n+1} - y(t_{n+1}) = e_n + h[f(t_n,u_n) - f(t_n,y(t_n))] + h\tau_n$$ Assume $f$ is Lipschitz constant $L$ and the local truncation error satisfies a bound $$||\tau_n|| \leq \tau$$ for all $n$ (which depends on the exact solution) then &#xA;$$||e_{n+1}|| \leq ||e_n|| + hL||e_n|| + h\tau = (1 + hL)||e_n||+ h \tau$$&lt;br&gt;&#xA;$$\leq (1 + hL)^{2}||e_{n-1}|| + (1 + (1 + hl))h\tau $$ &#xA;$$\leq \ldots$$&#xA;$$\leq (1 + hL)^{n+1}||e_0|| + \frac{(1 + hL)^{n+1} - 1}{(1 + hL) - 1}h\tau$$&#xA;since $(1 + hL) \leq e^{hL} = 1 + hL + \frac{1}{2}(hL)^2 + \ldots$, this gives $$||e_n|| \leq e^{nhL}||e_0|| + \frac{1^{nhL} - 1 }{hL}\tau \leq e^{LT}||e_0|| + \frac{e^{LT} - 1}{L}\tau$$ for $0\leq t_n = nh\leq T$. This shows stability. Now the local truncation error satisfies $$h||\tau_n|| = ||\frac{1}{2}h^2y'(t_n + \theta_n h)|| \leq \frac{1}{2}Mh^2$$ assuming $y$ is $C^2$ and $||y''|| \leq u$. Consistency gives a local bound on $\tau$ and stability allows us to conclude convergence $$|u_n - y(t_n)|| \leq e^{LT}||u_0 - y_0|| + \frac{e^{LT} - 1}{LT}\frac{T}{2}Mh \leq O(|u_0 - y_0|) + O(h)$$ Convergence implies that $u_n\rightarrow y_n$ on the whole interval $[0,T]$ as $h\rightarrow 0$ and $u_0\rightarrow y_0$. A similar proof can be done for any one-step method. A little extra work is required for implicit methods such as implicit Euler $$u_{n+1} = u_n + hf(t_{n+1},u_{n+1})$$ because the right hand side depends on the new unknown $u_{n+1}$. Assume you can solve the implicit equation for $u_{n+1}$ at each step so we can estimate the errors as for explicit Euler $$||e_{n+1}|| \leq e_n + hL||e_{n+1}|| + h||\tau_n||$$ collect each terms on the left, and for $hL \leq \frac{1}{2}$ we can then divide by $1 - hL$ to get $$||e_{n+1}|| \leq \frac{1}{1 - hL}(||e_n|| + h||\tau_n||)$$ using the additional inequality $$\frac{1}{1 - hL}\leq 1 + 2hL \ \ \text{for} \ \ hL \leq 1/2$$ gives stability as for Euler's method, and the consistency computation follows. &lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="17826" LastEditDate="2015-11-27T15:07:15.617" LastActivityDate="2015-11-27T15:07:15.617" Title="Family of linear one-step method convergence question" Tags="|numerical-analysis|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="21409" PostTypeId="1" AcceptedAnswerId="21442" CreationDate="2015-11-26T18:25:44.647" Score="2" ViewCount="101" Body="&lt;p&gt;I have to invert large, but densely populated matrices with higher precision arithmetic. Therefore I am looking for something like the PLASMA library, which can do Cholesky or LU factorization in parallel on shared memory machines. Unfortunately it does support only classical float or double arithmetic. Therefore I am looking for a templated library with the same features, but the ability to use arbitrary precision data types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="18353" LastActivityDate="2015-11-30T19:21:12.723" Title="Templated Numerical Linear Algebra in Parallel" Tags="|linear-algebra|parallel-computing|libraries|precision|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21410" PostTypeId="2" ParentId="21402" CreationDate="2015-11-26T18:58:45.547" Score="12" Body="&lt;p&gt;I am not an expert in machine learning, but I can outline the considerations that are relevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numerical calculations in machine learning are generally linear algebra -- either solving linear systems or linear least squares.  For both types of problems, there are well-known backward-stable methods, so I will assume you are using a backward-stable algorithm.  Then you should expect an error of roughly&#xA;$\kappa \epsilon$, where $\kappa$ is the condition number of the problem and $\epsilon$ is unit roundoff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the linear system $Ax=b$, you have $\kappa = \|A\| \|A^{-1}\| = \kappa(A)$, the condition number of the matrix $A$.  For the least squares problem, the condition number can fall anywhere in the range $[\kappa(A),\kappa^2(A)]$; see e.g. the text of Trefethen &amp;amp; Bau for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus for linear systems, single precision will be sufficient as long as $\kappa(A)$ is much less than $10^7$.  For least squares, you may already be in trouble when $\kappa(A)\approx 10^3 - 10^4$.  For large datasets, those are not very large condition numbers.  So it certainly seems plausible that you may need double precision.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2015-11-26T19:13:49.170" LastActivityDate="2015-11-26T19:13:49.170" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21411" PostTypeId="1" CreationDate="2015-11-27T02:24:15.727" Score="2" ViewCount="103" Body="&lt;p&gt;The basis of Molecular Dynamics simulation is the integration of newton's equations of motion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{F}{m} = \frac{d^2r}{dt^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that there are various methods based on Taylor's expansion, to integrate forward in time and calculate the new position. Doing this always leaves an error term in the expansion, however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve for position exactly, I can integrate both sides:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r = \frac{Ft^2}{m} + c_1 t + c_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $r(t=0) = 0$ then $c_2 = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's possible to determine $c_1$ too (?) we have an exact solution, and so I can calculate the next position at $t+dt$ with perfect accuracy, unlike with Taylor expansions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone show me where my understanding is wrong? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="18355" LastActivityDate="2015-11-27T02:24:15.727" Title="Why not solve molecular dynamics trajectories exactly?" Tags="|time-integration|molecular-dynamics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21415" PostTypeId="2" ParentId="14200" CreationDate="2015-11-27T17:24:56.700" Score="4" Body="&lt;p&gt;I am not much into the thing but when we use l-m for training neural nets there is a trick we use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original equation is &#xA;$$\Delta w = - (J^T J)^{-1} J^T e$$&#xA;by subtracting the term $uI$ in order to make the matrix invertable where u is a constant and I is the identity matrix so the equation will become &#xA;$$\Delta w = - (J^T J+uI)^{-1} J^T e$$&#xA;so when $u$ is big enough it will insure that the matrix is invertible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I said I am talking from the NNs point of view there the $u$ parameter is calculated in an iterative fashion and often modified as the algorithm progress.&#xA;So you may need to modify it according to your minimization objective &#xA;and I sorry if I did not make this clear enough.&lt;/p&gt;&#xA;" OwnerUserId="18361" LastEditorUserId="1804" LastEditDate="2015-11-28T10:08:20.893" LastActivityDate="2015-11-28T10:08:20.893" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21417" PostTypeId="1" AcceptedAnswerId="21438" CreationDate="2015-11-27T18:33:51.887" Score="6" ViewCount="3273" Body="&lt;p&gt;I need to store large sparse matrices in Eigen. I cannot find anything in the library except the function below, in &lt;a href=&quot;http://eigen.tuxfamily.org/dox/unsupported/MarketIO_8h_source.html&quot; rel=&quot;noreferrer&quot;&gt;Eigen/Unsupported&lt;/a&gt;. The problem with saveMarket is, that it saves in text format. Due to the size of my matrices I need to store my sparse matrices as binaries. Is there an easy way to adjust the function below to store as a binary. And an easy way to reload the matrix?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;template&amp;lt;typename SparseMatrixType&amp;gt;&#xA;bool saveMarket(const SparseMatrixType&amp;amp; mat, const std::string&amp;amp; filename, int sym = 0)&#xA;{&#xA;  typedef typename SparseMatrixType::Scalar Scalar;&#xA;  std::ofstream out(filename.c_str(),std::ios::out);&#xA;  if(!out)&#xA;    return false;&#xA;&#xA;  out.flags(std::ios_base::scientific);&#xA;  out.precision(64);&#xA;  std::string header; &#xA;  internal::putMarketHeader&amp;lt;Scalar&amp;gt;(header, sym); &#xA;  out &amp;lt;&amp;lt; header &amp;lt;&amp;lt; std::endl; &#xA;  out &amp;lt;&amp;lt; mat.rows() &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; mat.cols() &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; mat.nonZeros() &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;  int count = 0;&#xA;  for(int j=0; j&amp;lt;mat.outerSize(); ++j)&#xA;    for(typename SparseMatrixType::InnerIterator it(mat,j); it; ++it)&#xA;    {&#xA;    ++ count;&#xA;    internal::PutMatrixElt(it.value(), it.row()+1, it.col()+1, out);&#xA;    // out &amp;lt;&amp;lt; it.row()+1 &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; it.col()+1 &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; it.value() &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;}&#xA;  out.close();&#xA;  return true;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8329" LastActivityDate="2020-01-15T09:09:42.813" Title="Eigen - store sparse matrix as binary" Tags="|sparse-matrix|eigen|data-storage|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21418" PostTypeId="1" CreationDate="2015-11-27T19:25:14.203" Score="1" ViewCount="246" Body="&lt;p&gt;I know that FVM is based on the integral form of conservation laws, and FDS is based on PDEs. What I'm confused by, is whether FEM and DGFEM formulations are based on integral or pde form of conservation laws. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have developed FEM formulations starting from integral form of transport equations but I've also found literature on that FEM approximates PDEs, so slightly confused there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also not sure about DGFEM methods, with this method again, I start from the integral form of the equations, not sure if they are also be based on PDEs. &lt;/p&gt;&#xA;" OwnerUserId="17476" LastActivityDate="2015-11-28T19:31:46.137" Title="Are FEM or DGFEM methods based on integrals or PDEs?" Tags="|finite-element|finite-difference|finite-volume|discretization|discontinuous-galerkin|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21419" PostTypeId="1" AcceptedAnswerId="21517" CreationDate="2015-11-27T19:42:19.640" Score="6" ViewCount="1405" Body="&lt;p&gt;Let $x \in R^N$.  From a Spectral Chebyshev collocation method, I have a system of quadratic and linear equations.  Denote them,&#xA;$$&#xA;x^T Q_i x + L_i^T x = 0&#xA;$$&#xA;and &#xA;$$&#xA;A x = 0&#xA;$$&#xA;Furthermore, I know the solution will fulfill&#xA;$$&#xA;B x \geq 0&#xA;$$&#xA;for $Q_i \in R^{N \times N}, L_i \in R^{N}, A\in R^{M \times N}, B\in R^{S\times N}$, and $i = 1,\ldots P$.  To get a sense of sizes, I have $N \approx M\approx S  \approx P \approx 400$.  The matrices $A$ and $B$ would have about $N/2$ nonzeros and each $Q_i$ is going to have around $N/4$ nonzeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; I can solve this with a nonlinear solver, but are there any other methods (and accompanying software implementations) which can solve this and exploit the fact that everything is linear or quadratic?  I have no reason to believe that  $Q_i$ lead to any particular structure (e.g., not necessarily positive semi-definite, etc.).  That said, I am willing to throw caution to the wind and accept local solutions (especially if there is appropriate software I can throw the problem at).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apologies for my general ignorance of these types of problems.  I have looked at QCQP methods, but they seem to focus on inequality constraints.  I am also not sure I can naively put the quadratic constraints as objectives to minimize due to their unknown structure/convexity.&lt;/p&gt;&#xA;" OwnerUserId="9692" LastActivityDate="2015-12-08T17:31:40.973" Title="Algorithm for solving system of quadratic equations and linear equations" Tags="|nonlinear-equations|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21420" PostTypeId="1" CreationDate="2015-11-27T20:16:31.163" Score="2" ViewCount="354" Body="&lt;p&gt;This is a question about the computer algebra system &lt;a href=&quot;http://magma.maths.usyd.edu.au/magma/handbook/&quot; rel=&quot;nofollow&quot;&gt;Magma&lt;/a&gt;. I have been looking for a place to ask this type of question on the SE network and scicomp.SE &lt;a href=&quot;http://meta.math.stackexchange.com/questions/22052/is-either-math-se-or-stackoverflow-an-appropriate-forum-for-questions-about-magm#comment86041_22052&quot;&gt;was suggested to me&lt;/a&gt;; hopefully it finds a home here.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Suppose one has constructed a polynomial algebra $A$ over a ring $R$ in Magma. How does one construct the sub-$R$-algebra of $A$ generated by a given list of elements of $A$?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This seems to me to be a very basic operation so I can't believe there isn't a way to do it, but I haven't so far found it in &lt;a href=&quot;http://magma.maths.usyd.edu.au/magma/handbook/&quot; rel=&quot;nofollow&quot;&gt;the handbook&lt;/a&gt;. (I see functionality to construct subalgebras of matrix algebras and of endomorphism rings of abelian varieties, but not polynomial rings.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 11/3/15:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) As the functionality of Magma may vary with the ring $R$, take $R=\mathbb{Z}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) hardmath suggests in comments to construct the desired subring as the quotient of another polynomial ring. I take him to mean as the image of a map from another polynomial ring. I tried to implement this in a test case as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A := PolynomialRing(IntegerRing(), 3);&#xA;s1 := ElementarySymmetricPolynomial(A,1);&#xA;s2 := ElementarySymmetricPolynomial(A,2);&#xA;s3 := ElementarySymmetricPolynomial(A,3);&#xA;B := PolynomialRing(IntegerRing(),3);&#xA;f := hom&amp;lt; B -&amp;gt;A | s1,s2,s3&amp;gt;;&#xA;S := Image(f);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get a &lt;code&gt;Runtime error: No constructor provided for this type of object&lt;/code&gt; message when Magma tries to implement &lt;code&gt;Image(f)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14405" LastEditorUserId="14405" LastEditDate="2015-12-03T16:21:41.360" LastActivityDate="2015-12-03T16:21:41.360" Title="How to construct a subring of a polynomial ring in Magma" Tags="|symbolic-computation|computer-arithmetic|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21421" PostTypeId="1" CreationDate="2015-11-27T23:14:51.940" Score="9" ViewCount="1366" Body="&lt;p&gt;Is there an overview available over tensor visualization software?&lt;/p&gt;&#xA;&lt;p&gt;My personal preference is:&lt;/p&gt;&#xA;&lt;p&gt;A software which is free, well documented, and offers visualization techniques for different physical second order (or higher-order) tensor fields.&lt;/p&gt;&#xA;&lt;p&gt;Some modules that I would be interested in&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hyperstreamlines (&lt;a href=&quot;https://dl.acm.org/doi/10.5555/951087.951115&quot; rel=&quot;nofollow noreferrer&quot;&gt;eigenvector integration method&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Geodesics as direction, instead of eigenvectors&lt;/li&gt;&#xA;&lt;li&gt;Topological methods (e.g handling of equal eigenvalues, i.e &lt;a href=&quot;https://cgl.ethz.ch/teaching/scivis_common/Literature/DelmarcelleHesselink94.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;degenerate points&lt;/a&gt;, extraction of &lt;a href=&quot;https://georgehaller.com/reprints/smallestFTLE.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite Lyapunov exponent&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Tensor ellipsoids&lt;/li&gt;&#xA;&lt;li&gt;Texture based visualization (&lt;a href=&quot;https://ieeexplore.ieee.org/document/1250379&quot; rel=&quot;nofollow noreferrer&quot;&gt;HyperLIC&lt;/a&gt;,&lt;a href=&quot;https://web.archive.org/web/20170809103936/http://www.zib.de/hotz/publications/paper/hotz04_physicalTensorTexture.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Physical based method&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Handling of asymmetric tensor fields (decomposition of tensor field into symmetric and anti-symmetric parts, and re-glueing of the resulting visualization, or a &lt;a href=&quot;http://www.sci.utah.edu/%7Echengu/Publications/hybridtensorvis_vis11.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;direct approach&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Depending on the physical property of the tensor field, different methods are required to visualize the data.&lt;/p&gt;&#xA;&lt;p&gt;I know about about &lt;a href=&quot;https://docs.enthought.com/mayavi/mayavi/auto/mlab_pipeline_other_functions.html#hyper-streamline&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mayavi&lt;/a&gt;, but there is almost no documentation available for the tensor module, and the module is limited to visualization of a hyperstreamline (singular). The GUI is buggy, and the scripting options seem limited.&lt;/p&gt;&#xA;&lt;p&gt;There appears to be a very dismal selection for visualization of tensor fields.&lt;/p&gt;&#xA;" OwnerUserId="4249" LastEditorUserId="22284" LastEditDate="2022-05-05T00:54:57.870" LastActivityDate="2022-05-05T00:54:57.870" Title="Second order tensor field visualization software" Tags="|visualization|tensor|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21423" PostTypeId="1" AcceptedAnswerId="21425" CreationDate="2015-11-28T08:47:50.803" Score="16" ViewCount="13790" Body="&lt;p&gt;In FEM classes, it's usually taken for granted that the stiffness matrix is positive definite, but I just can't understand why. Could anyone give some explanation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, we can consider the Poisson problem:&#xA;$$ -\nabla^2 u = f,$$&#xA;whose stiffness matrix is:&#xA;$$K_{ij} = \int_\Omega\nabla\varphi_i\cdot\nabla\varphi_j\, d\Omega,$$&#xA;which is symmetric and positive definite. Symmetry is an obvious property, but the positive definiteness is not so explicite to me.&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="713" LastEditDate="2015-11-28T10:01:03.070" LastActivityDate="2023-10-10T13:53:03.313" Title="In FEM, why is the stiffness matrix positive definite?" Tags="|finite-element|matrix|stiffness|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21424" PostTypeId="2" ParentId="21423" CreationDate="2015-11-28T09:45:58.743" Score="3" Body="&lt;p&gt;If stiffness of element is not positive, then the system is not stable. So the model is most likely not correct.  Look at the most basic equation of harmonic oscillator&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$m x''(t) + k x(t) = f(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is unstable if $k$ is negative (look at the roots of the characteristic equation). It means the solution will blow up. The stiffness has to be a restoring force. At least for a physical spring. The stiffness matrix extends this to large number of elements (global stiffness matrix). That is all. But it is the same basic idea. FEM basis is in the stiffness matrix method for structural analysis where each element has a stiffness associated with it. &lt;/p&gt;&#xA;" OwnerUserId="727" LastActivityDate="2015-11-28T09:45:58.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21425" PostTypeId="2" ParentId="21423" CreationDate="2015-11-28T09:59:59.413" Score="21" Body="&lt;p&gt;The property follows from the property of the corresponding (weak form of the) partial differential equation; this is one of the advantages of finite element methods compared to, e.g., finite difference methods.&lt;/p&gt;&#xA;&lt;p&gt;To see that, first recall that the finite element method starts from the weak form of the Poisson equation (I'm assuming Dirichlet boundary conditions here): Find &lt;span class=&quot;math-container&quot;&gt;$u\in H^1_0(\Omega)$&lt;/span&gt; such that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ a(u,v):= \int_\Omega \nabla u\cdot \nabla v \,dx = \int_\Omega fv\,dx \qquad\text{for all }v\in H^1_0(\Omega).$$&lt;/span&gt;&#xA;The important property here is that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ a(v,v) = \|\nabla v\|_{L^2}^2 \geq c \|v\|_{H^1}^2 \qquad\text{for all }v\in H^1_0(\Omega). \tag{1}$$&lt;/span&gt;&#xA;(This follows from Poincaré's inequality.)&lt;/p&gt;&#xA;&lt;p&gt;Now the classical finite element approach is to replace the infinite-dimensional space &lt;span class=&quot;math-container&quot;&gt;$H^1_0(\Omega)$&lt;/span&gt; by a &lt;em&gt;finite-dimensional subspace&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$V_h\subset H^1_0(\Omega)$&lt;/span&gt; and find &lt;span class=&quot;math-container&quot;&gt;$u_h\in V_h$&lt;/span&gt; such that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ a(u_h,v_h):= \int_\Omega \nabla u_h\cdot \nabla v_h \,dx = \int_\Omega fv_h\,dx \qquad\text{for all }v_h\in V_h.\tag{2}$$&lt;/span&gt;&#xA;The important property here is that you are using the same &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and a subspace &lt;span class=&quot;math-container&quot;&gt;$V_h\subset H^1_0(\Omega)$&lt;/span&gt; (a &lt;em&gt;conforming&lt;/em&gt; discretization); that means that you still have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ a(v_h,v_h) \geq c \|v_h\|_{H^1}^2 &amp;gt;0 \qquad\text{for all }v_h\in V_h\setminus\{0\}. \tag{3}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now for the last step: To transform the variational form to a system of linear equations, you pick a basis &lt;span class=&quot;math-container&quot;&gt;$\{\varphi_1,\dots,\varphi_N\}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt;,  write &lt;span class=&quot;math-container&quot;&gt;$u_h =\sum_{i=1}^N u_i\varphi_i$&lt;/span&gt; and insert &lt;span class=&quot;math-container&quot;&gt;$v_h=\varphi_j$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1\leq j\leq N$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt;. The stiffness matrix &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; then has the entries &lt;span class=&quot;math-container&quot;&gt;$K_{ij}=a(\varphi_i,\varphi_j)$&lt;/span&gt; (which coincides with what you wrote).&lt;/p&gt;&#xA;&lt;p&gt;Now take an arbitrary vector &lt;span class=&quot;math-container&quot;&gt;$\vec v=(v_1,\dots,v_N)^T\in \mathbb{R}^N\setminus\{0\}$&lt;/span&gt; and set &lt;span class=&quot;math-container&quot;&gt;$v_h:=\sum_{i=1}^Nv_i \varphi_i\in V_h$&lt;/span&gt;. Then we have by &lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt; and the bilinearity of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; (i.e., you can move scalars and sums into both arguments)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \vec v^T K \vec v = \sum_{i=1}^N\sum_{j=1}^N v_iK_{ij} v_j =&#xA;\sum_{i=1}^N\sum_{j=1}^N a(v_i\varphi_i,v_j\varphi_j) = a(v_h,v_h) &amp;gt;0.$$&lt;/span&gt;&#xA;Since &lt;span class=&quot;math-container&quot;&gt;$\vec v\neq 0$&lt;/span&gt; was arbitrary, this implies that &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is positive definite.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; The stiffness matrix is positive definite because it comes from a &lt;strong&gt;conforming discretization&lt;/strong&gt; of a (self-adjoint) &lt;strong&gt;elliptic partial differential equation&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2022-02-12T11:29:53.080" LastActivityDate="2022-02-12T11:29:53.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21427" PostTypeId="1" CreationDate="2015-11-28T15:00:10.523" Score="5" ViewCount="624" Body="&lt;p&gt;Say I have a set of oriented points at locations $\vec{v_i}$ with each some direction $\vec{n_i}$, in practice they represent the normals of some surface that has been non uniformly sampled. How would I calculate the divergence of this vector field? Standard finite-difference methods would obviously not work because there is no grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I  perhaps assume a grid and use nearest neighbours to infer the vector field at each point (I think it's called &quot;splatting&quot;)? The discretized vector field would then have at each point a weighted average of nearby points. This would then allow me to use standard FD-methods to calculate (or rather estimate) the divergence at each gridpoint. Is this a common method, or are there better methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;Asked it on MO, got an &lt;a href=&quot;https://mathoverflow.net/questions/224741/numerically-calculating-the-divergence-of-a-set-of-oriented-points/224750#224750&quot;&gt;answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18371" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2015-12-17T12:48:40.180" Title="Numerically calculating the divergence of a set of oriented points" Tags="|finite-difference|vector|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21428" PostTypeId="2" ParentId="21418" CreationDate="2015-11-28T19:31:46.137" Score="5" Body="&lt;p&gt;Basically, you have the following set of derivations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Strong form of the PDE -&gt; Finite differences&lt;/li&gt;&#xA;&lt;li&gt;Integral form of the PDE -&gt; Finite volumes&lt;/li&gt;&#xA;&lt;li&gt;Weak (variational) form of the PDE -&gt; Galerkin methods (including FEM and DGFEM)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-11-28T19:31:46.137" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21429" PostTypeId="1" CreationDate="2015-11-28T22:05:44.977" Score="1" ViewCount="261" Body="&lt;p&gt;I understand the checkerboard effect due to the use of collocated grid arrangements in FVM. However, I wanted to know whether this problem is definitely bound to effect the results? For instance, I have modelled some problems on 1D structured grids with a collocated arrangement and have produced smooth results. Perhaps I am mistaken in how exactly the results should be effected? &lt;/p&gt;&#xA;" OwnerUserId="17476" LastActivityDate="2015-11-28T22:05:44.977" Title="Do collocated grid arrangements definitely result in the checkerboard effect?" Tags="|fluid-dynamics|finite-volume|discretization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21430" PostTypeId="1" CreationDate="2015-11-29T02:37:13.360" Score="2" ViewCount="63" Body="&lt;p&gt;I am trying to numerically resolve the equation for an Euler-Bernoulli beam that is inextensible, unshearable, and subject to planar deformations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho I(s) \frac{\partial^2 \theta}{\partial t^2}(s,t) &#xA;= E \frac{\partial}{\partial s} \left[I(s) \left( \frac{\partial \theta}{\partial s}(s,t) - \kappa_p(s,\theta,t) \right) \right]\,,\,\,\,\,\, t&amp;gt;0\,,\,\,\,\,\, 0&amp;lt;s&amp;lt;L\,,$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \theta}{\partial s}(0,t) = \frac{\partial \theta}{\partial s}(L,t) = 0\,,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\theta(s,0) = \frac{\partial \theta}{\partial t}(s,0) = 0\,,$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\kappa_p(s,\theta,t)$ is a preferred curvature. The ends of the beam are force-free and moment-free. Here, $E \gg 1$ so that $\frac{\partial \theta}{\partial s}(s,t) \approx \kappa_p(s,\theta,t)$. I cannot directory approximate $\theta(s,t)$ with $\kappa_p(s,\theta,t)$ since $\kappa_p(s,\theta,t)$ depends on $\theta(s,t)$. I have been reading into what may be a good numerical method to use to resolve this problem, however, I am not finding very good solutions since this equation has a stiff part in both the linear term and the nonlinear forcing term. An implicit method would free me from the minisulce time-step coming from $E$, however, the phase will not be resolved properly. I can use an implicit method for the linear terms, however, using something like multi-level approximation in time for the source term also introduces errors. Also, an implicit method may not resolve the phase well if I take big time-steps. What would be the best way to resolve this problem? If the area moment of inertia, $I(s) = 0$, at the end points, would it be a good idea to simply replace the geometry of the beam where the endpoints have a finite, but small $I(s)$ so that the PDE isn't singular or is there a nice way that I can numerically treat this singularity? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aside: To show that the system is hyperbolic, I can rewrite the PDE as a system of equations and then compute the eigenvalues of the linear part. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \rho I(s) \frac{\partial \theta}{\partial t}&#xA;= \frac{\partial \omega}{\partial s} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial \omega}{\partial t} = E \left[I(s) \left( \frac{\partial \theta}{\partial s}(s,t) - \kappa_p(s,\theta,t) \right) \right] $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if $I(s)$ is strictly positive, then the eigenvalues are $\lambda_{\pm} = \pm \sqrt{E/\rho}$ which is a real quantity in this problem. Thus, the problem is hyperbolic. For non-negative $I(s)$, one recovers the same eigenvalues when taking the limit. Note that the stiffness for the problem is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ S = \frac{|\lambda_+|}{|\lambda_-|} = 1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;however, depending on the numerical method, the variation in $I(s)$, and if I am not careful about the treatment of the singular points (from $I(0) = I(L) = 0$), then I will get numerical stiffness. &lt;/p&gt;&#xA;" OwnerUserId="17368" LastEditorUserId="17368" LastEditDate="2015-11-30T07:15:56.733" LastActivityDate="2015-11-30T07:15:56.733" Title="Resolving a stiff hyperbolic problem with Neumann boundary conditions" Tags="|numerical-analysis|hyperbolic-pde|implicit-methods|wave-propagation|stiffness|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21431" PostTypeId="2" ParentId="21247" CreationDate="2015-11-29T05:29:47.253" Score="0" Body="&lt;h2&gt;Basis of Stability&lt;/h2&gt;&#xA;&lt;p&gt;The stability of a method is derived on the basis of truncation error, an error that accumulates over each time step. In other words, the stability of a solution in space-time domain can be looked from a point of view of its dependence on a future time. Fully explicit schemes are known to be stable for a certain time step size because of the accumulation of truncation error over each time step. If the time step size is beyond the prescribed limit then a high truncation error would normally result in over estimating the final solution. However in implicit schemes the solution estimation depend on the current time step only, so they are independent of the time step size. This independence of time-step in an implicit scheme is gained by the introduction of numerical diffusion effects into the approximating equations, which causes&lt;/p&gt;&#xA;&lt;h2&gt;Explicit Scheme&lt;/h2&gt;&#xA;&lt;p&gt;We calculate the solution for a future time using the solution at the current time. Mathematically, if &lt;span class=&quot;math-container&quot;&gt;$c(t)$&lt;/span&gt; is the solution at the current time and &lt;span class=&quot;math-container&quot;&gt;$c(t+1)$&lt;/span&gt; is the solution at a future time, then for an explicit method we would have the following form of equation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$c(t+1) = f\left[c(t)\right]$$&lt;/span&gt;&#xA;where, &lt;span class=&quot;math-container&quot;&gt;$f\left[c(t)\right]$&lt;/span&gt; is some known function of &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; at current time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and is known. Only unknown in this equation is &lt;span class=&quot;math-container&quot;&gt;$c(t+1)$&lt;/span&gt; on left hand side.&lt;/p&gt;&#xA;&lt;h2&gt;Implicit Scheme&lt;/h2&gt;&#xA;&lt;p&gt;We calculate the solution by solving an equation which contains the solution at both the current time and future time. Using the same convention defined earlier, for an implicit scheme we would have the following equation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$c(t+1) = c(t) + f[c(n+1)]$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this equation right hand side depends upon the quantity you are trying to calculate. Assuming &lt;span class=&quot;math-container&quot;&gt;$f(c)=Ac$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a matrix independent of &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, then the implicit scheme requires solving the following system of linear equations:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$(I-A).c(t+1) = c(t)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Rightarrow c(t+1) = (I-A)^{-1}.c(t)$$&lt;/span&gt;&#xA;i.e. to solve for a solution at a future time you would need to perform matrix inversion.&lt;/p&gt;&#xA;&lt;h2&gt;Pros and Cons of Two Schemes&lt;/h2&gt;&#xA;&lt;p&gt;Explicit schemes are easy to program with only simple calculations performed at each timestep while implicit schemes are more difficult and can involve many iterations per timestep. Explicit schemes are more prone to be unstable and require very small timesteps compared to implicit schemes which are much stable and does not have any constraint on the size of a time step you can take.&lt;/p&gt;&#xA;" OwnerDisplayName="user5510" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-11-29T05:29:47.253" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21433" PostTypeId="1" CreationDate="2015-11-29T19:14:31.037" Score="2" ViewCount="193" Body="&lt;p&gt;I'm trying to solve the following first order parabolic partial differential equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;X \frac{\partial V}{ \partial Q} = -\frac{1}{2} \sigma^2 \frac{\partial^2 V}{\partial X^2} + rV + p \min (X, 0) &#xA;\end{equation*} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to conditions&#xA;\begin{equation*}&#xA;\begin{split}&#xA;V &amp;amp;= 0 \hspace{27pt} \text{as} \hspace{20pt} X \rightarrow \infty \\&#xA;V &amp;amp;= pQ \hspace{20pt} \text{as} \hspace{20pt} X \rightarrow -\infty \\&#xA;\frac{1}{2} \sigma^2 \frac{\partial^2 V}{\partial X^2} - rV &amp;amp;= 0 \hspace{20pt} \text{if} \hspace{20pt} X &amp;lt; 0 \hspace{10pt} \text{and} \hspace{10pt} Q = 0 \\&#xA;\frac{1}{2} \sigma^2 \frac{\partial^2 V}{\partial X^2} - rV &amp;amp;= 0 \hspace{20pt} \text{if} \hspace{20pt} X &amp;gt; 0 \hspace{10pt} \text{and} \hspace{10pt} Q = Q_{max}&#xA;\end{split}&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I understand is that Matlab pdepe solver solves &#xA;\begin{equation*}&#xA;c \Big(x, t, u, \frac{\partial u}{\partial x} \Big) \frac{\partial u}{\partial t} = x^{-m} \frac{\partial }{\partial x} \Big( x^{m} f \Big( x, t, u,  \frac{\partial u}{\partial x} \Big) \Big) + s \Big( x, t, u, \frac{\partial u}{\partial x} \Big)&#xA;\end{equation*} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if redefine my model,&#xA;\begin{equation*}&#xA;X \frac{\partial V}{ \partial Q} = -\frac{1}{2} \sigma^2 \frac{\partial^2 V}{\partial X^2} + rV + p \min (X, 0) &#xA;\end{equation*} &#xA;I can let&#xA;\begin{equation*}&#xA;\begin{split}&#xA;c &amp;amp;= X \\&#xA;f &amp;amp;= -\frac{1}{2} \sigma^2 \frac{\partial V}{\partial X} \\&#xA;s &amp;amp;= rV + p \min (X, 0)  \\&#xA;m &amp;amp;= 0&#xA;\end{split}&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how do I get the boundary condition supposedly in the form &#xA;\begin{equation*}&#xA;p(x,t,u)  + q(x,t) f \Big( x,t,u, \frac{\partial u}{\partial x} \Big) = 0&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and how do I capture conditions as stated above? The domain should be divided into two regions, one (X &amp;lt; 0) for solving upwards, and the other (X &gt; 0) in which solve downwards. Along the boundary X = 0 the solutions must be matched. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a bit lost at this point. Any insight would really help. Thanks! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: &#xA;If I let the following be p,f, and q for the boundary condition&#xA;\begin{equation*}&#xA;\begin{split}&#xA;p &amp;amp;= -rV \\&#xA;f &amp;amp;= -\frac{1}{2} \sigma^2 \frac{\partial V}{\partial X} \\&#xA;q &amp;amp;= 1&#xA;\end{split}&#xA;\end{equation*}&#xA;would this be correct in capturing the above mentioned conditions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code in Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function pde&#xA;global sigma r p&#xA;sigma = 0.5; &#xA;r = 0.01;&#xA;p = 1;&#xA;&#xA;x_inf = 25;&#xA;N = 10;&#xA;M = 20;&#xA;delta_x = x_inf/N;&#xA;q_max = 1;&#xA;delta_q = q_max/M;&#xA;&#xA;m = 0;&#xA;x = linspace(0, delta_x, 2*N);&#xA;t = linspace(0, delta_q, M);&#xA;&#xA;sol = pdepe(m,@pdex1pde,@pdex1ic,@pdex1bc,x,t);&#xA;% Extract the first solution component as u.&#xA;u = sol(:,:,1);&#xA;&#xA;% A surface plot is often a good way to study a solution.&#xA;surf(x,t,u) &#xA;title('Numerical solution computed with 20 mesh points.')&#xA;xlabel('Distance x')&#xA;ylabel('Time t')&#xA;&#xA;% --------------------------------------------------------------&#xA;function [c,f,s] = pdex1pde(x,q,v,DvDx)&#xA;global sigma r p&#xA;c = x;&#xA;f = -(1/2)*sigma*DvDx;&#xA;s = r*v + p*min(x,0);&#xA;% --------------------------------------------------------------&#xA;function u0 = pdex1ic(~)&#xA;u0 = 0;&#xA;% --------------------------------------------------------------&#xA;function [pl,ql,pr,qr] = pdex1bc(xl,vl,xr,vr,q)&#xA;global r&#xA;pl = -r*vl; &#xA;ql = 1;&#xA;pr = -r*vr;&#xA;qr = 1; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any insight at all on the steps to solving this (whether in Matlab with pdepe or not) will be very much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="18208" LastEditorUserId="18208" LastEditDate="2015-11-30T07:55:24.010" LastActivityDate="2015-11-30T07:55:24.010" Title="Modeling First Order Parabolic PDE (Battery Storage Model)" Tags="|matlab|pde|parabolic-pde|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21434" PostTypeId="1" CreationDate="2015-11-29T22:44:19.130" Score="0" ViewCount="44" Body="&lt;p&gt;suppose you have this bayesian network: p(a,b,c) = p(c|b) p(b|a) p(a)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a -&gt; b -&gt; c&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are a and c conditionally independent given b?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yes, why are they independent? How can I show that?&lt;/p&gt;&#xA;" OwnerUserId="17629" LastEditorUserId="17629" LastEditDate="2015-11-30T14:53:49.780" LastActivityDate="2015-11-30T19:41:48.297" Title="Conditional Independence (Bayesian Network)" Tags="|statistics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21435" PostTypeId="2" ParentId="20927" CreationDate="2015-11-30T01:06:34.130" Score="5" Body="&lt;p&gt;One possibility is to wrap the arbitrary precision interval implementation in &lt;a href=&quot;https://github.com/fredrik-johansson/arb/&quot; rel=&quot;nofollow&quot;&gt;Arb&lt;/a&gt;. This will not be as fast as a dedicated double precision implementation, but it might still be fast enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 1: this code requires Arb version 2.8.0 or later.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;acb_hypgeom.h&quot;&#xA;&#xA;void&#xA;hyp1f1ix(double * re, double * im, double a, double b, double x)&#xA;{&#xA;    long prec;&#xA;    acb_t aa, bb, xx, rr;&#xA;    acb_init(aa); acb_init(bb); acb_init(xx); acb_init(rr);&#xA;&#xA;    acb_set_d(aa, a);&#xA;    acb_set_d(bb, b);&#xA;    acb_set_d(xx, x);&#xA;    acb_mul_onei(xx, xx);&#xA;&#xA;    for (prec = 64; ; prec *= 2)&#xA;    {&#xA;        acb_hypgeom_m(rr, aa, bb, xx, 0, prec);&#xA;        if (acb_rel_accuracy_bits(rr) &amp;gt;= 53)&#xA;            break;&#xA;    }&#xA;&#xA;    *re = arf_get_d(arb_midref(acb_realref(rr)), ARF_RND_DOWN);&#xA;    *im = arf_get_d(arb_midref(acb_imagref(rr)), ARF_RND_DOWN);&#xA;&#xA;    acb_clear(aa); acb_clear(bb); acb_clear(xx); acb_clear(rr);&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    double re, im;&#xA;    hyp1f1ix(&amp;amp;re, &amp;amp;im, 3.14, 2.78, 2015.1130);&#xA;    printf(&quot;%.15g %.15g\n&quot;, re, im);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note 2: I have prepared a file arbcmath.h (&lt;a href=&quot;https://github.com/fredrik-johansson/arbcmath&quot; rel=&quot;nofollow&quot;&gt;https://github.com/fredrik-johansson/arbcmath&lt;/a&gt;) that wraps 1F1 and many other functions (2F1, Bessel, incomplete gamma, etc.) for use with C99 complex doubles, so you don't need to implement the wrapper code yourself.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;arbcmath.h&quot;&#xA;&#xA;int main()&#xA;{&#xA;    double complex w;&#xA;    w = ac_hyp1f1(3.14, 2.78, 2015.1130*I);&#xA;    printf(&quot;%.15g + %.15g*I\n&quot;, creal(w), cimag(w));&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3420" LastEditorUserId="3420" LastEditDate="2015-12-29T18:41:24.187" LastActivityDate="2015-12-29T18:41:24.187" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21436" PostTypeId="2" ParentId="21421" CreationDate="2015-11-30T01:12:42.630" Score="5" Body="&lt;p&gt;The program &lt;a href=&quot;http://www.visitusers.org/index.php?title=Main_Page&quot;&gt;VisIt&lt;/a&gt; can do plots of tensor ellipsoids, but I don't think it has anything for hyperstreamlines. While it does make nice plots, I've found VisIt hard to install, if not impossible on some platforms; I know people who have been desperate enough to set up a virtual machine for it, but I haven't done that myself. When it does work, I have found it to be better than Mayavi in terms of documentation and lack of bugs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried to find software for visualizing 2nd-rank tensors, and I agree with you that the options are pretty slim.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-11-30T01:12:42.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21438" PostTypeId="2" ParentId="21417" CreationDate="2015-11-30T06:11:28.350" Score="6" Body="&lt;p&gt;I've rolled my own. Here is a MCVE:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;Eigen/Core&amp;gt;&#xA;#include &amp;lt;Eigen/Sparse&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;&#xA;using namespace Eigen;&#xA;&#xA;typedef Triplet&amp;lt;int&amp;gt; Trip;&#xA;&#xA;template &amp;lt;typename T, int whatever, typename IND&amp;gt;&#xA;void Serialize(SparseMatrix&amp;lt;T, whatever, IND&amp;gt;&amp;amp; m) {&#xA;    std::vector&amp;lt;Trip&amp;gt; res;&#xA;    int sz = m.nonZeros();&#xA;    m.makeCompressed();&#xA;&#xA;    fstream writeFile;&#xA;    writeFile.open(&quot;matrix&quot;, ios::binary | ios::out);&#xA;&#xA;    if(writeFile.is_open())&#xA;    {&#xA;        IND rows, cols, nnzs, outS, innS;&#xA;        rows = m.rows()     ;&#xA;        cols = m.cols()     ;&#xA;        nnzs = m.nonZeros() ;&#xA;        outS = m.outerSize();&#xA;        innS = m.innerSize();&#xA;&#xA;        writeFile.write((const char *)&amp;amp;(rows), sizeof(IND));&#xA;        writeFile.write((const char *)&amp;amp;(cols), sizeof(IND));&#xA;        writeFile.write((const char *)&amp;amp;(nnzs), sizeof(IND));&#xA;        writeFile.write((const char *)&amp;amp;(innS), sizeof(IND));&#xA;        writeFile.write((const char *)&amp;amp;(outS), sizeof(IND));&#xA;&#xA;        writeFile.write((const char *)(m.valuePtr()),       sizeof(T  ) * m.nonZeros());&#xA;        writeFile.write((const char *)(m.outerIndexPtr()),  sizeof(IND) * m.outerSize());&#xA;        writeFile.write((const char *)(m.innerIndexPtr()),  sizeof(IND) * m.nonZeros());&#xA;&#xA;        writeFile.close();&#xA;    }&#xA;}&#xA;&#xA;template &amp;lt;typename T, int whatever, typename IND&amp;gt;&#xA;void Deserialize(SparseMatrix&amp;lt;T, whatever, IND&amp;gt;&amp;amp; m) {&#xA;    fstream readFile;&#xA;    readFile.open(&quot;matrix&quot;, ios::binary | ios::in);&#xA;    if(readFile.is_open())&#xA;    {&#xA;        IND rows, cols, nnz, inSz, outSz;&#xA;        readFile.read((char*)&amp;amp;rows , sizeof(IND));&#xA;        readFile.read((char*)&amp;amp;cols , sizeof(IND));&#xA;        readFile.read((char*)&amp;amp;nnz  , sizeof(IND));&#xA;        readFile.read((char*)&amp;amp;inSz , sizeof(IND));&#xA;        readFile.read((char*)&amp;amp;outSz, sizeof(IND));&#xA;&#xA;        m.resize(rows, cols);&#xA;        m.makeCompressed();&#xA;        m.resizeNonZeros(nnz);&#xA;&#xA;        readFile.read((char*)(m.valuePtr())     , sizeof(T  ) * nnz  );&#xA;        readFile.read((char*)(m.outerIndexPtr()), sizeof(IND) * outSz);&#xA;        readFile.read((char*)(m.innerIndexPtr()), sizeof(IND) * nnz );&#xA;&#xA;        m.finalize();&#xA;        readFile.close();&#xA;&#xA;    } // file is open&#xA;}&#xA;&#xA;&#xA;int main(int argc, char *argv[]){&#xA;    int rows, cols;&#xA;    rows = cols = 6;&#xA;    SparseMatrix&amp;lt;double&amp;gt; A(rows,cols), B;&#xA;&#xA;    std::vector&amp;lt;Trip&amp;gt; trp, tmp;&#xA;&#xA;    trp.push_back(Trip(0, 0, rand()));&#xA;    trp.push_back(Trip(1, 1, rand()));&#xA;    trp.push_back(Trip(2, 2, rand()));&#xA;    trp.push_back(Trip(3, 3, rand()));&#xA;    trp.push_back(Trip(4, 4, rand()));&#xA;    trp.push_back(Trip(5, 5, rand()));&#xA;    trp.push_back(Trip(2, 4, rand()));&#xA;    trp.push_back(Trip(3, 1, rand()));&#xA;&#xA;    A.setFromTriplets(trp.begin(), trp.end());&#xA;    cout &amp;lt;&amp;lt; A.nonZeros() &amp;lt;&amp;lt; endl;   // Prints 8&#xA;    cout &amp;lt;&amp;lt; A.size() &amp;lt;&amp;lt; endl;       // Prints 36&#xA;    cout &amp;lt;&amp;lt; A &amp;lt;&amp;lt; endl;              // Prints the matrix along with the sparse matrix stuff&#xA;&#xA;    Serialize(A);&#xA;&#xA;    Deserialize(B);&#xA;&#xA;    cout &amp;lt;&amp;lt; B.nonZeros() &amp;lt;&amp;lt; endl;   // Prints 8&#xA;    cout &amp;lt;&amp;lt; B.size() &amp;lt;&amp;lt; endl;       // Prints 36&#xA;    cout &amp;lt;&amp;lt; B &amp;lt;&amp;lt; endl;              // Prints the reconstructed matrix along with the sparse matrix stuff&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7433" LastEditorUserId="7433" LastEditDate="2020-01-15T09:09:42.813" LastActivityDate="2020-01-15T09:09:42.813" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21439" PostTypeId="1" CreationDate="2015-11-30T10:26:03.683" Score="5" ViewCount="1983" Body="&lt;p&gt;I am solving the convection-diffusion equation in 2D using Finite Differences with the $\theta$ scheme. The velocity of the fluid and the diffusion coefficient is low in my case (in the range of $10^{-15}$). The boundary conditions are: open boundary on two sides (Robin boundary), and Dirichlet boundary $u=0$ on the other two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I have some questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is the CFL-Number of any importance when solving the Convection Diffusion Equation in 2D using the $\theta$ scheme and Finite Differences? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How does the diffusion coefficient factor into the CFL-Condition?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;s&gt; I know the implicit case is supposed to be stable for all time steps and step sizes, but I get ugly oscillations. Is it only important for the explicit case?&lt;/s&gt; &#xA;(edit: I have no more oscillations)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17890" LastEditorUserId="17890" LastEditDate="2015-12-01T15:27:11.423" LastActivityDate="2015-12-01T15:27:11.423" Title="CFL Condition and Convection Diffusion Equation in 2D" Tags="|finite-difference|stability|advection-diffusion|cfl|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21440" PostTypeId="1" AcceptedAnswerId="21445" CreationDate="2015-11-30T18:04:56.120" Score="1" ViewCount="2147" Body="&lt;p&gt;I'm trying to model the Black-Scholes Equation (transformed into a heat equation) using method of lines in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transformed formula is basically &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;\frac{\partial u}{\partial t} = \frac{\partial^2 u}{\partial x^2} + (k-1)\frac{\partial u}{\partial x} - ku&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $k$ is a constant and with initial condition &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;u(x,0) = \max(e^x - 1, 0)&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;u(a,t) = \alpha \hspace{35pt} u(b,t) = \beta&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is python implementation of the method of lines for the above equation should match the results &lt;a href=&quot;http://www.math.uwaterloo.ca/~hwolkowi//henry/reports/talks.d/t09talks.d/09waterloomatlab.d/mfileshigham.d/bs.m&quot; rel=&quot;nofollow noreferrer&quot;&gt; in the matlab code here&lt;/a&gt;. I can't seem to find where I went wrong. Any insight on the Python code would be really helpful. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.integrate import odeint&#xA;import matplotlib.pyplot as plt&#xA;from scipy.fftpack import diff as psdiff&#xA;&#xA;N = 40&#xA;r = 0.065;&#xA;sigma = 0.8;&#xA;k = float(0.5*sigma**2);&#xA;a = np.log(float(2)/5)&#xA;b = np.log(float(7)/5)&#xA;t0 = 0;&#xA;tf = 5;&#xA;&#xA;x = np.linspace(a, b, N);&#xA;xmesh = np.max(np.exp(a)-1,0)*np.ones(x.shape)&#xA;&#xA;def odefunc(u, t):&#xA;    dudt = np.zeros(xmesh.shape)&#xA;&#xA;    #boundary conditions&#xA;    dudt[0] = u[0]&#xA;    dudt[-1] = (7 - 5*np.exp(-k*t))/5 &#xA;&#xA;    #step size&#xA;    h = b/(N-1)&#xA;&#xA;    for i in range(1, N-1):&#xA;        dudt[i] = ((u[i + 1] - 2*u[i] + u[i - 1]) / h**2) + (k-1)*((u[i] - u[i-1]) / h) - (k*u[i])&#xA;    return dudt&#xA;&#xA;tspan = np.linspace(t0, tf, 20);&#xA;sol = odeint(odefunc, xmesh, tspan)&#xA;&#xA;for i in range(0, len(tspan), 5):&#xA;    plt.plot(xmesh, sol[i], label='t={0:1.2f}'.format(tspan[i]))&#xA;&#xA;# put legend outside the figure&#xA;plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))&#xA;plt.xlabel('X position')&#xA;plt.ylabel('Temperature')&#xA;&#xA;# adjust figure edges so the legend is in the figure&#xA;plt.subplots_adjust(top=0.89, right=0.77)&#xA;plt.savefig('pde.png')&#xA;&#xA;# Make a 3d figure&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;fig = plt.figure()&#xA;ax = fig.add_subplot(111, projection='3d')&#xA;&#xA;SX, ST = np.meshgrid(xmesh, tspan)&#xA;ax.plot_surface(SX, ST, sol, cmap='jet')&#xA;ax.set_xlabel('x')&#xA;ax.set_ylabel('t')&#xA;ax.set_zlabel('u')&#xA;ax.view_init(elev=30, azim=100) # adjust view so it is easy to see&#xA;plt.savefig('pde-transient-heat-3d.png')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iNlJM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iNlJM.png&quot; alt=&quot;3d Plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plot, I believe should look like the following&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JbUl2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JbUl2.jpg&quot; alt=&quot;plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18208" LastEditorUserId="18208" LastEditDate="2015-11-30T22:36:53.020" LastActivityDate="2015-11-30T23:20:30.683" Title="Heat Equation - PDE" Tags="|pde|finite-difference|python|parabolic-pde|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21441" PostTypeId="2" ParentId="15896" CreationDate="2015-11-30T18:42:05.757" Score="1" Body="&lt;p&gt;I don't have enough reputation to comment, but I hope this answer helps you.&#xA;Personally, I find that I learn best through example. &lt;a href=&quot;http://kitchingroup.cheme.cmu.edu/blog/2013/03/07/Transient-heat-conduction-partial-differential-equations/&quot; rel=&quot;nofollow&quot;&gt;Try looking at the code here&lt;/a&gt; to see how MOL was implemented in Python with centered finite difference approximation (an ODE solver was used). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd like to learn more about solving ODEs, parabolic PDEs, and MOL, I suggest &lt;a href=&quot;http://lya.fciencias.unam.mx/pablo/an20072/material/Scientificomputation.pdf&quot; rel=&quot;nofollow&quot;&gt;Heath's Scientific Computation&lt;/a&gt; (Chapter 11) which also provides some examples on solving MOL. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="18208" LastActivityDate="2015-11-30T18:42:05.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21442" PostTypeId="2" ParentId="21409" CreationDate="2015-11-30T19:21:12.723" Score="4" Body="&lt;p&gt;&lt;strong&gt;Elemental&lt;/strong&gt; available at &lt;a href=&quot;http://libelemental.org&quot; rel=&quot;nofollow&quot;&gt;libelemental.org&lt;/a&gt; may have what you ask. From the documentation for release 0.84:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Though Elemental does not yet fully support computation over arbitrary fields, the vast majority of its pieces do. Moving templated implementations into header files is a necessary step in the process and also allowed for certain templating techniques to exploited in order to simplify the class hierarchy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2015-11-30T19:21:12.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21443" PostTypeId="2" ParentId="21434" CreationDate="2015-11-30T19:41:48.297" Score="1" Body="&lt;p&gt;Per edited question: this is super simple. One way to show it is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p(a,c|b) = \frac{p(a,b,c)}{p(b)} = \frac{p(a,b)}{p(b)}p(c|b) = p(a|b) p(c|b)$&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2015-11-30T19:41:48.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21445" PostTypeId="2" ParentId="21440" CreationDate="2015-11-30T22:04:28.233" Score="2" Body="&lt;p&gt;I think that you just plot the wrong thing. The variable &lt;code&gt;xmesh&lt;/code&gt; is the initial condition, not the value of &lt;code&gt;x&lt;/code&gt; (in your particular case it is constant -0.6). But you don't have the same parameters in your Python and Matlab codes, you are also applying badly your initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See here the corrected code (please read it carefully):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;import numpy as np&#xA;from scipy.integrate import odeint&#xA;import matplotlib.pyplot as plt&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;&#xA;def odefunc(u, t):&#xA;    dudt = np.zeros(x0.shape)&#xA;&#xA;    #boundary conditions&#xA;    dudt[0] = u[0]&#xA;    dudt[-1] = (7 - 5*np.exp(-k*t))/5 &#xA;&#xA;    #step size&#xA;    h = b/(N-1)&#xA;&#xA;    for i in range(1, N-1):&#xA;        dudt[i] = (u[i + 1] - 2*u[i] + u[i - 1]) / h**2 +\&#xA;                   (k-1)*(u[i] - u[i-1]) / h - k*u[i]&#xA;    return dudt&#xA;&#xA;&#xA;N = 40&#xA;r = 0.065;&#xA;sigma = 0.8;&#xA;k = r/sigma**2&#xA;a = np.log(2/5)&#xA;b = np.log(7/5)&#xA;t0 = 0&#xA;tf = 5&#xA;&#xA;x = np.linspace(a, b, N);&#xA;x0 = np.exp(x) - 1&#xA;x0[x0 &amp;lt; 0] = 0&#xA;&#xA;tspan = np.linspace(t0, tf, 20);&#xA;sol = odeint(odefunc, x0, tspan)&#xA;&#xA;&#xA;#%% Make a 3d figure&#xA;fig = plt.figure()&#xA;ax = fig.add_subplot(111, projection='3d')&#xA;&#xA;SX, ST = np.meshgrid(x, tspan)&#xA;ax.plot_surface(SX, ST, sol, cmap='YlGnBu_r',&#xA;                rstride=1, cstride=1)&#xA;ax.set_xlabel('x')&#xA;ax.set_ylabel('t')&#xA;ax.set_zlabel('u')&#xA;ax.view_init(elev=30, azim=-115)&#xA;plt.savefig('pde-transient-heat-3d.png')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you plot against &lt;code&gt;x&lt;/code&gt;, you obtain this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D9Vli.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D9Vli.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2015-11-30T23:20:30.683" LastActivityDate="2015-11-30T23:20:30.683" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21446" PostTypeId="1" AcceptedAnswerId="23187" CreationDate="2015-11-30T22:51:37.073" Score="4" ViewCount="617" Body="&lt;p&gt;I am writing a FMM (Fast Multipole Method) algorithm in 3D. I generated the mesh and, currently, I am developing the expansion and the three (M2M, M2L, L2L) translation operators using spherical harmonics. See reference [1], more specifically, these equations are taken from equation numbers 3.36, 3.37, 3.55, 3.56, 3.57.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused about how to apply the operators and I can't seem to grasp how to apply these effectively. So far my aim is to (after the mesh generation of course):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1 - Perform multipole expansions at the finest level&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This as I understand is conducted using these equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\phi(P) = \sum_{n=0}^\infty \sum_{m=-n}^n \frac{M_n^m}{r^{n+1}} Y_n^m(\theta,\phi) \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M_n^m=\sum_i q_i \rho_i^n Y_n^{-m}(\alpha_i,\beta_i) \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above equations $P(r,\theta,\phi)$ are the coordinates of the center of the cube and $Q_i(\rho_i,\alpha_i,\beta_i)$ are the source points present in each cube being evaluated with $q_i$ as their corresponding weight for each source point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2 - Perform the upward pass (M2M)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The translator operators for instance the M2M (i.e. multipole-to-multipole) in the upward pass have the following equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\phi(G) = \sum_{j=0}^\infty \sum_{k=-j}^j \frac{M_j^k}{r^{j+1}} Y_j^k(\theta,\phi)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&#xA;$$M_j^k=\sum_{n=0}^j \sum_{m=-n}^n \frac{O_{j-n}^{k-m} i^{|k|-|m|-|k-m|} A_n^m A_{j-n}^{k-m} \rho^n Y_n^{-m}(\alpha,\beta) }{A_j^k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_n^m=\frac{(-1)^n}{\sqrt{(n-m)!(n+m)!}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above equations, $G(r,\theta,\phi)$ is the center of the parent cube and $P(\rho,\alpha,\beta)$ is the coordinates of the child cube's center.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here's where I am basically stuck at:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is $O_{j-n}^{k-m}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it the multipole expansions I performed before initiating the M2M translations ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;is it the mulipole moment that was used to compute the multipole expansion before starting the M2M (i.e. $M_n^m$)? if so, then each parent will have 8 children in the level directly above it, hence there exists multiple $M_n^m$, how then should I choose which one to implement in the translation M2M operation in place of the $O_{j-n}^{k-m}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;P.S.:&lt;/strong&gt; as you might have noticed already, this is the first time I am implementing the FMM and specifically in 3D. I have to accomplish this as it is a minor step in a larger project with an approaching deadline, hence I appreciate all help anyone can provide.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Reference&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;[1]&lt;/strong&gt; Greengard, Leslie. &lt;a href=&quot;http://cpsc.yale.edu/sites/default/files/files/tr533.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;The rapid evaluation of potential fields in particle systems. ACM Distinguished Dissertations.&quot;&lt;/a&gt; (1988).&lt;/p&gt;&#xA;" OwnerUserId="18387" LastEditorUserId="18387" LastEditDate="2016-02-22T11:28:32.313" LastActivityDate="2016-03-04T20:50:20.390" Title="Fast Multipole Method in 3D" Tags="|numerical-analysis|fluid-dynamics|computational-physics|fast-multipoles|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21447" PostTypeId="2" ParentId="21421" CreationDate="2015-11-30T23:22:55.863" Score="2" Body="&lt;p&gt;I would suggest &lt;a href=&quot;http://www.paraview.org/&quot; rel=&quot;nofollow&quot;&gt;Paraview&lt;/a&gt;, that is similar to VisIt, since both are based on VTK. You can use Python Calculator and scripts (both based on Python), and are described in the &lt;a href=&quot;http://www.paraview.org/paraview-downloads/download.php?submit=Download&amp;amp;version=v4.3&amp;amp;type=data&amp;amp;os=all&amp;amp;downloadFile=TheParaViewGuide-v4.3-CC-Edition.pdf&quot; rel=&quot;nofollow&quot;&gt;manual&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also directly use &lt;a href=&quot;http://www.vtk.org/&quot; rel=&quot;nofollow&quot;&gt;VTK&lt;/a&gt; that has a &lt;a href=&quot;http://www.vtk.org/Wiki/VTK/Examples/Python&quot; rel=&quot;nofollow&quot;&gt;Python interface&lt;/a&gt;. It already has implemented &lt;a href=&quot;http://www.vtk.org/doc/nightly/html/classvtkHyperStreamline.html&quot; rel=&quot;nofollow&quot;&gt;hyperstreamlines&lt;/a&gt; and the other algorithms can be implemented as &lt;em&gt;simple&lt;/em&gt; scripts.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-11-30T23:22:55.863" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21448" PostTypeId="1" CreationDate="2015-12-01T00:08:12.757" Score="0" ViewCount="221" Body="&lt;p&gt;I know that in FVM, it is possible to show that a discretisation scheme is conservative by adding the discrete terms over a few control volumes and showing that all terms cancel apart from those relating to the flux in and out of the entire domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what happens in FEM? My thoughts are: for instance for a 1D case where there are two neighbouring elements sharing a node, and hence 4 equations exists for a 3-element grid, one adds the relevant terms from the 4 equations to see which terms cancel out and which remain in the end. Am I right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="17476" LastActivityDate="2018-08-04T15:06:24.017" Title="Is it possible to show global conservative properties FEM as it is done in FVM?" Tags="|finite-element|finite-volume|discretization|numerical-modelling|conservation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21449" PostTypeId="2" ParentId="21448" CreationDate="2015-12-01T06:34:23.670" Score="2" Body="&lt;p&gt;The way this is usually proven in the finite element context is different, but many finite element schemes satisfy conservation properties. For example, if you think of the Stokes equations, as long as the pressure space contains the piecewise constant functions, then mass is conserved. Similar properties can often be shown for the mixed Laplace equation typically used for porous media flow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is typically more complicated to show such properties for first order conservation equations, but even there it is sometimes possible if finite element spaces are appropriately chosen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the ways you would show this results from the weak form of the equations, with particularly chosen test functions. It doesn't quite work as you suggest in your question.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-01T06:34:23.670" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21450" PostTypeId="1" CreationDate="2015-12-01T11:03:01.170" Score="3" ViewCount="1736" Body="&lt;p&gt;I have a question about the calculation of center of mass (COM) in systems with&#xA;periodic boundary conditions. There is a&#xA;method introduced here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Center_of_mass#Systems_with_periodic_boundary_conditions&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Center_of_mass#Systems_with_periodic_boundary_conditions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But by this method the COM can't be calculated properly, see the image bellow, because the system is extended over the whole box and not only the lower right corner but also the upper right one is considered as periodic image.&#xA;Is there a straightforward way for calculating COM in systems like this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2tSew.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2tSew.png&quot; alt=&quot;a snapshot of the simulated system&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18391" LastEditorUserId="9667" LastEditDate="2015-12-01T16:19:59.887" LastActivityDate="2015-12-01T16:19:59.887" Title="Center of mass in systems with periodic boundary conditions" Tags="|boundary-conditions|computational-geometry|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21451" PostTypeId="1" CreationDate="2015-12-01T12:01:28.287" Score="1" ViewCount="170" Body="&lt;p&gt;I'm working on the Black-Scholes equation, but I'm pretty new to financial modeling. Right now, I am trying to understand the Black-Scholes PDE. I understand that the Black-Scholes equation is given by&#xA;\begin{equation*}&#xA;\frac{\partial C}{\partial t} + \frac{1}{2}\sigma^2 S^2\frac{\partial^2 C}{\partial S^2} + rS \frac{\partial C}{\partial S} - rC = 0&#xA;\end{equation*}&#xA;with initial condition&#xA;\begin{equation*}&#xA;C(S,T) = \max (S-K, 0)&#xA;\end{equation*}&#xA;and boundary conditions&#xA;\begin{equation*}&#xA;C(0,t) = 0 \hspace{35pt} C(S,t) \rightarrow S \text{ as } S \rightarrow \infty&#xA;\end{equation*}&#xA;and $C(S,t)$ is defined over $0 &amp;lt; S &amp;lt; \infty$, $0 \leq t \leq T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be further transformed and simplified into a heat diffusion equation &lt;a href=&quot;http://www.ms.uky.edu/~rwalker/research/black-scholes.pdf&quot; rel=&quot;nofollow&quot;&gt;as described here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we make the following change of variable&#xA;\begin{equation*}&#xA;u = e^{-r\tau}C \hspace{20pt} \text{ or } \hspace{20pt} C = ue^{r\tau}&#xA;\end{equation*}&#xA;and&#xA;\begin{equation*}&#xA;S = e^x \hspace{25pt} \text{ and }  \hspace{25pt} t = T- \tau \hspace{20pt} &#xA;\end{equation*}&#xA;we get the transformed heat equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;\frac{\partial u}{\partial \tau} = \frac{\partial^2 u}{\partial x^2} + (k-1)\frac{\partial u}{\partial x} - ku&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $k = \frac{2r}{\sigma^2}$. The &lt;a href=&quot;http://www.math.uwaterloo.ca/~hwolkowi//henry/reports/talks.d/t09talks.d/09waterloomatlab.d/mfileshigham.d/bs.m&quot; rel=&quot;nofollow&quot;&gt;following matlab code&lt;/a&gt; implements this. My question is, what exactly is the form of the boundary conditions for the the transformed equation? I can't seem to understand the parameters (related to the boundary conditions) given in the Matlab code. Any related literature would be highly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="18208" LastEditorUserId="18208" LastEditDate="2015-12-02T20:36:29.367" LastActivityDate="2015-12-02T20:36:29.367" Title="Boundary Conditions for the given PDE" Tags="|pde|boundary-conditions|parabolic-pde|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21452" PostTypeId="1" CreationDate="2015-12-01T12:27:19.197" Score="1" ViewCount="81" Body="&lt;p&gt;I have a plant that can be modeled with nonlinear equations as&#xA;$$f(x,u,p)=0$$&#xA;where $x$ is the state vector, $u$ the control vector, and $p$ the model parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to control this system, I compute the discrete state-space model&#xA;$$x_{k+1}=A_p x_k+B_p u_k$$&#xA;where $A_p$ and $B_p$ depend on the parameters $p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I consider $N$ different $p$ vectors referring to possible malfunctions of the plant. Thus, I get $N$ possible state-space models. &lt;strong&gt;I am searching for a methodology to cluster these state-space models according to their &quot;similarity&quot; and end up with a subset of $M(&amp;lt;N)$ cluster state-space models.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A trivial example is when the plant has two identical components sharing some task, when one malfunctions ($p_1$), I get the state-space model $(A_{p_1},B_{p_1})$. When the second malfunctions ($p_2$), I get $(A_{p_2},B_{p_2})$. However, the two state-space models are the same and I need only to consider one of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references to papers or techniques are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1602" LastEditorUserId="1602" LastEditDate="2015-12-01T12:56:12.320" LastActivityDate="2015-12-01T12:56:12.320" Title="Clustering state-space models according to similarity" Tags="|clustering|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21454" PostTypeId="1" CreationDate="2015-12-01T14:48:32.980" Score="3" ViewCount="4586" Body="&lt;p&gt;So far what I understand is that two neighbouring elements are conformal if their edges and faces match exactly, whereas with non-conformal elements this is not the case. For instance, h-refinement gives rise to hanging nodes, and h-nonconforming elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then there is the concept of p-nonconforming. Note sure, but I think this applies for instance in DGFEM, where the weight functions are the roots of a polynomial and which do not match at the boundaries? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate it if someone offered a simple explanation for this, particularly in the context of typical structured grids used in DGFEM, FEM and FVM.&lt;/p&gt;&#xA;" OwnerUserId="17476" LastActivityDate="2015-12-01T18:31:51.023" Title="what is the difference between non-conformal and conformal?" Tags="|finite-element|finite-volume|discretization|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21455" PostTypeId="2" ParentId="21454" CreationDate="2015-12-01T15:09:22.917" Score="5" Body="&lt;p&gt;The concept of $p$-nonconforming meshes can also apply to continuous FEM, not just DG.  For continuous FEM, continuity is enforced by enforcing a single set of degrees of freedom on a face shared between two elements. If those elements have varying polynomial degrees, the trace space on the face must be made the same. This may be done by restricting the higher degree element to a lower degree trace (referred to as the minimum rule) or by increasing the degree of the trace space for the lower degree element (referred to as the maximum rule). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For DG, $p$-nonconforming meshes are much simpler to implement, as elements are coupled together weakly through a unique interface flux. The method behaves is pretty much identically so long as the computation of integrals of this flux are computed with sufficiently accurate quadrature (though implementation details may change between uniform $p$ and variable $p$ codes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm less familiar with FVM, but I think $p$-nonconformity may depend on what type of high $p$ FVM you use (wider stencil, compact/multi-resolution, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: As noted by @Christian Waluga, nonconformity here refers to meshes, not FEM.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2015-12-01T18:31:51.023" LastActivityDate="2015-12-01T18:31:51.023" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21456" PostTypeId="2" ParentId="21390" CreationDate="2015-12-01T16:54:28.890" Score="2" Body="&lt;p&gt;I understand that you want to solve the differential equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2 u - \frac{1}{c^2}\frac{\partial^2 u}{\partial t^2} = f(t)\delta(\mathbf{r}) \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for an unbounded domain, $\mathbf{r}$ is the position vector, and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(t) = -\frac{2(t-t_0)}{T^2} e^{-\frac{\left( t-t_0\right)^2}{T^2}} \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;although, we don't need to write this function explicitly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We compute then the Fourier transform of the equation, i.e.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2 u + k^2 u = \hat{f}(\omega)\delta(\mathbf{r}) \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $k^2 = \omega^2/c^2$, and $\hat{f}(\omega)$ the Fourier transform of $f(t)$ that reads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat{f}(\omega) = \frac{i\,\omega\,T\,{e}^{-\frac{{\omega}^{2}\,{T}^{2}}{4}-i\,\omega\,t0}}{\sqrt{2}} \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, the solution to your problem looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(t) = \frac{i}{4 \sqrt{2\pi}}\int_{-\infty}^{\infty} H_0^{(1)}(k|\mathbf{r}|) \hat{f} (\omega) e^{i\omega t} d\omega \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should read about the solution for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Helmholtz_equation&quot; rel=&quot;nofollow&quot;&gt;Helmholtz equation&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_transform&quot; rel=&quot;nofollow&quot;&gt;Fourier transform&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolution&quot; rel=&quot;nofollow&quot;&gt;Convolutions&lt;/a&gt; to understand the procedure.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2015-12-01T16:54:28.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21457" PostTypeId="2" ParentId="21362" CreationDate="2015-12-01T20:18:51.577" Score="0" Body="&lt;p&gt;@DougLipinski's answer is absolutely correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You identified a key clue when you said the problem is &quot;pretty clear to our eyes&quot;:  Understanding human vision is a great path to learning computer vision.  In this case, knowing that human vision is great at handling local contrast would imply that the computer may do better if &quot;everything but&quot; the local contrast is removed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But let's take a moment to consider a general way to approach such problems by starting with a simple question:  How can a good image processing algorithm be found for a specific situation when known methods are unsatisfactory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One great approach is to find a tool that makes it easy to interactively try a bunch of algorithms.  My favorite is &lt;a href=&quot;http://imagej.nih.gov/ij/&quot; rel=&quot;nofollow&quot;&gt;ImageJ&lt;/a&gt;, which has tools that range from the elegantly simple to the bafflingly complex.  Many are built-in, and many more are available as plugins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this specific case, a &lt;a href=&quot;http://imagej.nih.gov/ij/plugins/mean-shift.html&quot; rel=&quot;nofollow&quot;&gt;Mean Shift Filter&lt;/a&gt; with radius 5 should do nicely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once an algorithm is found that meets your specific needs, you can either look at its ImageJ source code, or do a search on the algorithm name to find an implementation in the language of your choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me, that language is generally Python, either via OpenCV, or PIL (the Python Imaging Library), or SciKit-Image, or Pillow, or sometimes the Python bindings for ImageMagick, depending on which package best handles the specific workflow (what supplies the images, and what needs to happen next).&lt;/p&gt;&#xA;" OwnerUserId="18396" LastActivityDate="2015-12-01T20:18:51.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21459" PostTypeId="1" AcceptedAnswerId="21460" CreationDate="2015-12-01T20:46:41.193" Score="0" ViewCount="105" Body="&lt;p&gt;Suppose that we are given a  numerical scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to find the CFL condition , we set $U_j^n= \lambda ^ne^{ik x_j}$ and put it into the numerical scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have shown that the given method is unstable since $|\lambda|&amp;gt;1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, how do we calculate the CFL condition?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get $| \lambda |= \sqrt{1+\gamma^2 \nu^2 \sin^2{(kh)}}$. This can't be $&amp;lt;1$. What do we do in this case? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The given equation is $u_t+ \gamma u_x=0, \gamma&amp;gt;0$. We know that $k \in \mathbb{R}, \nu=\frac{\tau}{h}$ where $\tau$ is the step of the discretization of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wZHvy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wZHvy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: Also suppose that we have a numerical scheme , put into it $U_j^n= \lambda^n e^{ik x_j}$ and get $a_i^n \nu \leq 1$. Can we then say that the method is stable if and only if $a_i^n \nu \leq 1$ ?&lt;/p&gt;&#xA;" OwnerUserId="9422" LastEditorUserId="9422" LastEditDate="2015-12-01T21:49:50.260" LastActivityDate="2015-12-01T22:20:33.203" Title="How do we find the condition?" Tags="|numerical-analysis|stability|cfl|" AnswerCount="1" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="21460" PostTypeId="2" ParentId="21459" CreationDate="2015-12-01T22:20:33.203" Score="3" Body="&lt;p&gt;You assert that for your scheme to be stable you need $|λ|≤1$ which is correct, however you find that all values for $c=γν$ will give $|λ|&amp;gt;1$. This means that the discretization you have applied is unstable and there is no CFL condition for stability. Consider using upwind differencing or an implicit method.&lt;/p&gt;&#xA;" OwnerUserId="18017" LastActivityDate="2015-12-01T22:20:33.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21461" PostTypeId="2" ParentId="2283" CreationDate="2015-12-01T23:07:56.353" Score="2" Body="&lt;p&gt;&lt;code&gt;f2py&lt;/code&gt; supports modern fortran (assumed-shaped arrays and so), see an &lt;a href=&quot;http://arogozhnikov.github.io/2015/11/29/using-fortran-from-python.html#f2py&quot; rel=&quot;nofollow&quot;&gt;example script&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for experiments with fortran code I'd recommend &lt;a href=&quot;https://github.com/mgaitan/fortran_magic&quot; rel=&quot;nofollow&quot;&gt;IPython magic&lt;/a&gt;, which is easy to install and very convenient (it relies on f2py as well).&lt;/p&gt;&#xA;" OwnerUserId="18363" LastEditorUserId="18363" LastEditDate="2016-04-18T20:55:15.230" LastActivityDate="2016-04-18T20:55:15.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21464" PostTypeId="1" AcceptedAnswerId="21480" CreationDate="2015-12-02T10:50:21.920" Score="8" ViewCount="1183" Body="&lt;p&gt;I am trying to compute (in C) a sum like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S = \sum_i \exp( - a_i )$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $10^{4} &amp;lt; a_i &amp;lt; 10^{5}$ are approximately normal distributed. So even if I do the Log-Sum-Exp trick &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S = \exp(\log[\sum_i \exp( - a_i + K )] - K)$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $K = \min_i(a_i)$, I have exponents of order $-10^5$. That means that almost every term vanishes during summation, since only the far right side of the normal-distribution contributes (only about $3$ points of $10^6$!). Is there any clever way to handle a sum like this? Thanks in advance for any suggestions. &lt;/p&gt;&#xA;" OwnerUserId="18403" LastActivityDate="2015-12-05T14:17:38.553" Title="Sum over very small exponentials: Underflow" Tags="|c|statistics|floating-point|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21465" PostTypeId="2" ParentId="21464" CreationDate="2015-12-02T12:00:32.763" Score="2" Body="&lt;p&gt;You are adding positive terms, so you don't need to worry about the imprecision of the final result as if you had negative terms: (10^100+5)-(10^100+3)=2 but (10^100+5)+(10^100+3)=2*10^100 (as long you don't need 100 digits of precision).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case I'd sort the exponents in decreasing order (in the sense of magnitude). Then I'd keep adding just in case there was a carry from the smallest terms.&lt;/p&gt;&#xA;" OwnerUserId="18255" LastActivityDate="2015-12-02T12:00:32.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21469" PostTypeId="1" CreationDate="2015-12-02T16:50:13.857" Score="12" ViewCount="559" Body="&lt;p&gt;I'm currently working with &quot;A Multigrid Tutorial&quot; by Briggs et al, Chapter 8.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The construction of the interpolation operator is given as:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9INAz.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9INAz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then construction of restriction operator and fine grid operator are given as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wr1Wv.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wr1Wv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume we have three grid points x0, x1, x2 with the middle one x1 is fine and the others are coarse. The middle one is interpolated by &lt;code&gt;x1 = x0*w0 + x2*w2&lt;/code&gt;. Therefore, the interpolation operator is (in Matlab):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I = [1, 0, 0; w0, 0, w2; 0, 0, 1]&#xA;&#xA;I =&#xA;&#xA;[  1, 0,  0]&#xA;[ w0, 0, w2]&#xA;[  0, 0,  1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The restriction operator is then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;transpose(I)&#xA;&#xA;ans =&#xA;&#xA;[ 1, w0, 0]&#xA;[ 0,  0, 0]&#xA;[ 0, w2, 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now let's see what would happen if one would restrict and then interpolate directly, what results in a multiplication of &lt;code&gt;I&lt;/code&gt; and &lt;code&gt;transpose(I)&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I*transpose(I)&#xA;&#xA;ans =&#xA;&#xA;[  1,          w0,  0]&#xA;[ w0, w0^2 + w2^2, w2]&#xA;[  0,          w2,  1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would expect that this matrix is something like an identity matrix or would at least have norm 1 or something. But if we would apply x = [1, 1, 1] for lets say w0 = w2 = 0.5, we would get [1.5 1.5 1.5]. I would assume that repeatedly applied restriction-interpolation operations would at least converge to something. But no, in that case all vector components are multiplied by 1.5 on every restriction-interpolation. That seems very strange to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain what's going on?&lt;/p&gt;&#xA;" OwnerUserId="8138" LastEditorUserId="8138" LastEditDate="2015-12-02T21:12:22.280" LastActivityDate="2015-12-07T13:40:19.703" Title="Algebraic Multigrid: Why does the product of interpolation and restriction not result in something with norm 1?" Tags="|linear-algebra|multigrid|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21470" PostTypeId="1" CreationDate="2015-12-02T18:32:18.187" Score="0" ViewCount="5670" Body="&lt;p&gt;I have the function of an electric dipole expressed in cartesian coordinates and I want to create the vector field using Matlab . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function is &#xA;   $$E_z= \frac{p}{4\pi\epsilon_0} \cdot \left(\frac{3z^2}{r^5}- \frac{1}{r^3}\right) $$ and $$E_x=\frac{p}{4\pi\epsilon_0} \frac{3xz}{r^5}\;. $$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code I've come up with is : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear;&#xA;clc;&#xA;p = 1;&#xA;e = 8.85*10^(-12);&#xA;x  =linspace(-5 , 5, 50);&#xA;z = linspace(-5 , 5 ,50);&#xA;[X, Z ] = meshgrid(x,z );&#xA;R=sqrt(X.^2+Z.^2) ; &#xA;EX =( p .* 3 .* X .* Z )./ (4.*pi.*e ./ R.^5);&#xA;EZ = p./( 4 .* pi .* e ) .* ( 3.* Z.^2 ./R.^5 -1./ R.^3);&#xA;quiver ( X , Z , EX , EZ ) ; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it doesn't give me the output I want which looks like this &#xA;&lt;a href=&quot;https://i.stack.imgur.com/5slLQ.gif.&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5slLQ.gif.&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any ideas? I would be grateful!&lt;/p&gt;&#xA;" OwnerUserId="18410" LastActivityDate="2015-12-02T20:07:38.003" Title="Plot vector field in matlab" Tags="|matlab|vector|electromagnetism|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21471" PostTypeId="2" ParentId="21469" CreationDate="2015-12-02T19:11:38.450" Score="8" Body="&lt;p&gt;One simpler explanation - the range of the restriction operator is the coarse grid space, while the range of the interpolation operator is the fine grid space.  Unless the two are equal, interpolation + restriction will not result in an identity matrix, because there will always be components of $x$ which are truncated by the restriction operator and lost.  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-12-02T19:11:38.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21472" PostTypeId="2" ParentId="21470" CreationDate="2015-12-02T19:29:37.583" Score="2" Body="&lt;p&gt;For plotting, it is easier in my opinion to not use meshgrid if you want to scale the arrows. You have a vector field $(E_X, E_Z)$ and you can simply normalize it like in the code below: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear;&#xA;clc;&#xA;p = 1;&#xA;e = 8.85*10^(-12);&#xA;x  =linspace(-5 , 5, 20);&#xA;z = linspace(-5 , 5 ,20);&#xA;&#xA;for i = 1:length(x)&#xA;    for k = 1:length(z)&#xA;        R=sqrt(x(i)^2 + z(k)^2) ; &#xA;        EX =( p * 3 * x(i) * z(k) ) / (4 * pi * e / R^5);&#xA;        EZ = p / ( 4 * pi * e ) * ( 3 * z(k)^2 / R^5 - 1 / R^3);&#xA;        ex(i, k) = EX / sqrt(EX^2 + EZ^2);&#xA;        ez(i, k) = EZ / sqrt(EX^2 + EZ^2);&#xA;    end&#xA;end&#xA;scaleFactor = 0.5; &#xA;quiver ( x , z , ex , ez, scaleFactor, 'LineWidth', 2); &#xA;axis([-5 5 -5 5])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make the arrows look nicer, you can play with Matlab's plotting manipulators. &lt;/p&gt;&#xA;" OwnerUserId="17368" LastActivityDate="2015-12-02T19:29:37.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21473" PostTypeId="2" ParentId="21470" CreationDate="2015-12-02T20:07:38.003" Score="0" Body="&lt;p&gt;Here's how I did it.  Normalize everything so the arrows are same size, and remove NaN's at the origin.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p = 1;&#xA;eps0 = 8.854e-12;&#xA;&#xA;x = linspace(-5,5,20);&#xA;z = x;&#xA;&#xA;[xx,zz] = meshgrid(x,z);&#xA;&#xA;rr = sqrt(xx.^2 + zz.^2);&#xA;&#xA;ex = p/(4*pi*eps0) .* (3.*xx.*zz)./(rr.^5);&#xA;ez = p/(4*pi*eps0) .* (3.*zz.^2./rr.^5 - 1./rr.^3);&#xA;&#xA;E = sqrt(ex.^2 + ez.^2);&#xA;E(isnan(E)) = max(E(:));&#xA;&#xA;figure;&#xA;    quiver(x,z,ex./E,ez./E)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2015-12-02T20:07:38.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21475" PostTypeId="1" CreationDate="2015-12-02T22:49:47.033" Score="1" ViewCount="4897" Body="&lt;p&gt;I have 4 ordinary differential equations that are coupled. The variables in the 4 equations are functions of time and space and one of them is second order in space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{ \partial t_g}{ \partial t} = \frac {hp(t_m-tg)}{A(\rho_gCp_g)}- u\frac { \partial t_g}{ \partial x}   &#xA;\end{equation}&#xA;\begin{equation}&#xA;\frac{ \partial t_m}{ \partial t} = \frac{k_{sub}A_{sub}}{(\rho_mA_mCp_m)+(\rho_{sub}A_{sub}Cp_{sub})}\frac{ \partial^2 t_m}{ \partial x^2} + \frac{ h_mp(\rho_{vg}-\rho_{vm})h_{ads}}{ (\rho_mA_mCp_m)+(\rho_{sub}A_{sub}Cp_{sub})} - \frac {hp(t_m-t_g)}{(\rho_mA_mCp_m)+(\rho_{sub}A_{sub}Cp_{sub}}&#xA;\end{equation}&#xA;\begin{equation}&#xA;\frac { \partial \rho_{vg}}{ \partial t} = - \frac {h_mp( \rho_{vg} -\rho_{vm})}{A} - \frac {u \partial \rho_{vg}}{ \partial x} &#xA;\end{equation}&#xA;\begin{equation}&#xA;\frac { \partial \gamma_m}{ \partial t} = \frac {h_mp(\rho_{vg}-\rho_{vm})}{ \rho_mA_m}&#xA;\end{equation}&#xA;\begin{equation}&#xA;\frac { \gamma_m}{ \gamma_{max}} = \frac {1}{1-c+ \frac{c}{ \phi}}&#xA;\end{equation}&#xA;\begin{equation}&#xA;\phi = 4.09(10^{-9})t_m\rho_{vm}(e^\frac{-5196}{t_m})&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_g$, $t_m$, $\rho_{vg}$, $\rho_{vm}$, and $\gamma_m$ are the variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to solve this in Python. What is the best way to proceed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the Boundary and Initial Conditions corresponding to these equations. But, having never used Python to solve such a problem, I am unsure whether to use a function like &lt;code&gt;odeint&lt;/code&gt; or to type out the forward difference scheme.&lt;/p&gt;&#xA;" OwnerUserId="18412" LastEditorUserId="20625" LastEditDate="2016-06-08T15:27:53.443" LastActivityDate="2016-09-06T16:44:17.983" Title="Multiple Coupled Differential Equation solution in Python" Tags="|python|ode|" AnswerCount="2" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="21476" PostTypeId="1" AcceptedAnswerId="21479" CreationDate="2015-12-03T10:37:13.303" Score="7" ViewCount="774" Body="&lt;p&gt;The problem I want to solve is the displacement formulation of the linear elasticity :&#xA;$$&#xA;\nabla  \cdot \sigma = 0  \quad \text{in} \quad \Omega \\&#xA;\sigma = \lambda ( \nabla \cdot u ) I + \mu (\nabla \cdot u + \nabla \cdot u^t) \quad \text{in} \quad \Omega \\&#xA;u = 0 \quad \text{over} \quad \Gamma_D \\&#xA;\sigma  \cdot n = g \quad \text{ over } \quad \Gamma_N&#xA;$$&#xA;with $u$ the displacement vector, $\sigma$ the stress tensor and $\epsilon$ the deformation tensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weak formulation is the standard one :&#xA;$$ &#xA;a(u,v) = \int_{\Omega} \lambda \; \nabla \cdot u \nabla \cdot v &#xA;+ \int_{\Omega} 2 \mu \; \epsilon (u) : \epsilon (v) \\&#xA;f(v) = \int_{\Gamma_N} g \cdot v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there is the FEM discretization over a tetrahedral or hexahedral mesh using the&#xA;classic lagrange functions basis of degree 1 or 2. &#xA;(P1 or P2 for tet mesh and  Q1 or Q2 for hex mesh)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can assemble the linear system $ A x = B $ without difficulties but I am unable to solve it &#xA;for non trivial problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have reduce my test case to the following :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Omega \; \text{ is the unit cube } \\&#xA;u = 0 \quad \text{ for } z = 0 \\&#xA;\sigma \cdot n = -0.5 \sin(\pi x) \sin(\pi y) \; e_z \quad \text{ for } z=1&#xA;$$&#xA;The mesh is a regular subdivision in tetrahedra (or hexahedra) of the cube.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the Conjugate Gradient to solve the linear system. &#xA;I tested with Jacobi or Gauss-Seidel preconditioners but the CG solver is faster without them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Q1 function basis :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The CG solver does not converge (almost constant residual) when there are more than 300k unknowns (100k vertices)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For Q2 basis :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It starts to be slow at 200k unknowns and after it won't converge at all&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using Jacobi or Gauss-Seidel preconditioner does not improve the convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a comparison view, I tested with the heat equation (scalar field) and a&#xA;similar problem (unit cube, mixed boundary conditions), the CG solver converge quickly with 600k unknowns. &#xA;As the difference is mainly scalar vs vector field, I guess I am missing something in the preconditioning of&#xA;vector problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A linear elasticity problem with 200k unknowns (8000 hexes, 9000 vertices, Q2 basis) and a trivial geometry&#xA;seems pretty small but I am failing at it. And the&#xA;simpler preconditioners do not improve things. So my questions are :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How should I choose my preconditioners for linear elasticity ? &lt;/li&gt;&#xA;&lt;li&gt;How people solving large elasticity problems (millions of unknowns) deal with this kind of issues ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would be very grateful if you have any advices or references on the subject, thank you.&lt;/p&gt;&#xA;" OwnerUserId="17305" LastEditorUserId="17305" LastEditDate="2015-12-04T13:05:11.487" LastActivityDate="2015-12-04T13:05:11.487" Title="Which preconditioning for large linear elasticity problem?" Tags="|finite-element|linear-solver|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21478" PostTypeId="1" CreationDate="2015-12-03T13:05:41.993" Score="1" ViewCount="687" Body="&lt;p&gt;Any differential operator can be written as a finite difference matrix acting on a vector of values. I recently used it to solve $\Delta A = j$ by writing the Laplacian as a finite difference tridiagonal (second order approximation) matrix. I then used SciPy's &lt;code&gt;spsolve&lt;/code&gt;, which solves matrix equations of the form $Ax=b$ by not explicitly inverting the sparse matrix (which blows up really fast).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me, that because of the direct accuracy estimate you get from the finite difference scheme, and the extreme parallellization that can be achieved with methods like &lt;code&gt;spsolve&lt;/code&gt;, this would be an optimal way to solve this equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the case (or is it at least a competitive method)?&#xA;Why are all classes and lectures on numerical methods focusing on all other methods and I can only find sparse mention of the above described method, although it is both intuitive, simple to implement, and at least relatively fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only saw finite difference matrices in light of eigenvalue calculations (in which I believe it is also quite optimal due to extreme optimization of the sparse eigenvalue calculation procedures), and thought of this &quot;integration&quot; technique myself. I might have just overlooked all books and other literature on the subject, hence this question directed to the experts.&lt;/p&gt;&#xA;" OwnerUserId="9669" LastActivityDate="2016-01-06T01:33:07.250" Title="How efficient (compared to &quot;normal&quot; methods) is using a sparse finite difference matrix to solve differential equations?" Tags="|finite-difference|linear-solver|sparse-matrix|integration|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21479" PostTypeId="2" ParentId="21476" CreationDate="2015-12-03T14:56:06.563" Score="4" Body="&lt;p&gt;You can always try AMG as one preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several other methods that are used. Look into papers by Johanes Kraus, Neytcheva, Owe Axelsson. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use separate displacement ordering you can try Schur complement type preconditioners which are very effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may notice that for Incompressible material $\nu = 0.5$ you get $\lambda \rightarrow \infty$ which is a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What people do is they introduce another equation like this&#xA;$$&#xA; \mu \nabla \cdot u - \frac{\mu^2}{\lambda} p = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You use stable pair Q2-Q1 for $u$ and $p$. From this you get a two by two matrix $A$ which you can solve with Schur complement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been working on this for some times now, look for papers by Ali Dorostkar (which is I) and Maya Neytcheva.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one thing I forgot to mention here.&#xA;From Korn's inequality, we know that the block diagonal part of matrix $A$ is spectrally equivalent to $A$ so you can use that as your preconditioner. Meaning that you create a preconditioner&#xA;$$&#xA;D = \begin{bmatrix} A_{11} \\ &amp;amp; A_{22} \end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $Ax$ is a vector (lets name it $v$) in each iteration you apply your preconditioner you have&#xA;$$&#xA;D^{-1}Ax = D^{-1}v&#xA;$$&#xA;but you don't want to invert $D$ so what you do is that you solve another system like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Dy = v&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now since D is block diagonal you just need to solve two smaller systems. Each of those smaller systems you precondition with AMG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this is complicated but I thought maybe this interest you.&lt;/p&gt;&#xA;" OwnerUserId="17327" LastEditorUserId="17327" LastEditDate="2015-12-03T20:17:31.233" LastActivityDate="2015-12-03T20:17:31.233" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21480" PostTypeId="2" ParentId="21464" CreationDate="2015-12-04T06:12:07.057" Score="3" Body="&lt;p&gt;You may use Kahan summation algorithm [1]&#xA;The idea is to reschedule the sum operations in such a way precision loss is limited. The code is very simple (reproduced from [1] below).&#xA;If this does not suffice, you may use multiprecision representations, such as quad doubles [2]. They are supported by several languages / compilers (including GNU c). Finally, if this does not suffice, you may use multi-precision arithmetics based on expansions [3]. I have written an implementation with a C++ class that can be used like a number [4]. Now if the underflow occurs within a single term (i.e., if the exp() underflows), then you may need to use floating point representations with larger number of bits. For instance, you can use MPFR [5]. It has implementations of all classical operators and functions with arbitrary precision (very good library, implemented by friends of mine :-).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function KahanSum(input)&#xA;    var sum = 0.0&#xA;    var y,t                      // Temporary values.&#xA;    var c = 0.0                  // A running compensation for lost low-order bits.&#xA;    for i = 1 to input.length do&#xA;        y = input[i] - c         // So far, so good: c is zero.&#xA;        t = sum + y              // Alas, sum is big, y small, so low-order digits of y are lost.&#xA;        c = (t - sum) - y        // (t - sum) recovers the high-order part of y; subtracting y recovers -(low part of y)&#xA;        sum = t                  // Algebraically, c should always be zero. Beware overly-aggressive optimizing compilers!&#xA;    next i                       // Next time around, the lost low part will be added to y in a fresh attempt.&#xA;    return sum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Kahan_summation_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] J. Shewchuk, Adaptive Precision Floating-Point Arithmetic and Fast Robust Geometric Predicates,  Discrete and Computational Geometry (Impact Factor: 0.69). 07/1996; 18(3). DOI: 10.1007/PL00009321 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;https://gforge.inria.fr/frs/?group_id=5833&quot; rel=&quot;nofollow&quot;&gt;https://gforge.inria.fr/frs/?group_id=5833&lt;/a&gt;, MultiPrecision_psm&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] MPFR: &lt;a href=&quot;http://www.mpfr.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.mpfr.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2015-12-05T14:17:38.553" LastActivityDate="2015-12-05T14:17:38.553" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21481" PostTypeId="2" ParentId="21421" CreationDate="2015-12-04T15:30:14.920" Score="1" Body="&lt;p&gt;It is mostly an academic project, but you could take a look at TEEM &lt;a href=&quot;http://teem.sourceforge.net/download/index.html&quot; rel=&quot;nofollow&quot;&gt;http://teem.sourceforge.net/download/index.html&lt;/a&gt;. A tutorial for using it for second order tensor visualizations maybe found at &lt;a href=&quot;http://cg.cs.uni-bonn.de/en/people/junprof-dr-thomas-schultz/visweek-tutorial-tensors-in-visualization/&quot; rel=&quot;nofollow&quot;&gt;http://cg.cs.uni-bonn.de/en/people/junprof-dr-thomas-schultz/visweek-tutorial-tensors-in-visualization/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18426" LastActivityDate="2015-12-04T15:30:14.920" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21482" PostTypeId="1" CreationDate="2015-12-04T18:34:36.413" Score="5" ViewCount="647" Body="&lt;p&gt;Are there  libraries for conducting parallel AMR on an unstructured grid ? For a finite volume code,  polyhedral cells with arbitrarily shaped faces are as easy to handle as hexahedra, and infact maybe better for AMR because one doesn't have to do anything special for the refined grid. Due to the lack of libraries, I am trying to do this from scratch by splitting  polyhedral cells into pyramids with apex at the centre of the polyhedron. For this purpose, each face is first split into quads to form bases of the pyramids. If the polyhedron happens to be a hexahedron, I merge some of the pyramids (three to be precise) back to get the well known 1 to 8 split for hexahedron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not yet into coding the coarsening stage but I guess that I will need a tree data structure like the oct-tree used for hexs. Do I even need this because, I can arbitrarily merge neighboring cells sharing a face as long as the resulting cell remains convex? What I mean is if a hex has been split into 8 subcells, I don't have to recover the original cell if I decide to do coarsening later on. I haven't consulted any literature, nor did I check thoroughly for existing libraries, which could save me a lot of time.&lt;/p&gt;&#xA;" OwnerUserId="18132" LastEditorUserId="18132" LastEditDate="2015-12-04T19:19:21.297" LastActivityDate="2016-07-24T09:56:10.330" Title="unstructured grid AMR" Tags="|adaptive-mesh-refinement|unstructured-mesh|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21483" PostTypeId="1" CreationDate="2015-12-05T06:41:48.237" Score="5" ViewCount="6846" Body="&lt;p&gt;Now I need to sum up more than one thousands of terms and then make the four-dimmensional integral in my Fortran program. I found that there are some numerical errors. Can you give me some suggestions that how to check and avoid the possible round-off errors for this kind of general problem ?&lt;/p&gt;&#xA;" OwnerUserId="3911" LastActivityDate="2017-01-23T15:53:36.270" Title="How to avoid the round-off errors in the larger calculations?" Tags="|optimization|error-estimation|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21487" PostTypeId="1" CreationDate="2015-12-05T18:40:21.177" Score="4" ViewCount="477" Body="&lt;p&gt;I'm working on a small fun project on the side for a numerical method I've been working a bit with. Roughly, the computational problem I have to solve is the following: Assume you have a collection of $N$ points, call this collection $X$. For each $x_i \in X$, the following steps occur:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: Find $k$ nearest neighbors to $x_i$, call this set $X_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2:  Build a $k \times k$ matrix out of these points with some simple arithmetic. Call this matrix $A^i$. $A^i(j,k)$ depends only on points $y_j,y_k \in X_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3: Build a vector $b^i$ who is 1 only in the row corresponding to the originally chosen point $x^i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4: Solve the linear system $A^if^i =b^i$ with whatever method is best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5: Return the $k \times 1$ vector $f^i$ from Step 4 as well as the indices which map points in $X^i$ back to $X$ (since $X^i$ is a subcollection of $X$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we have those 5 steps to run through for &lt;em&gt;each&lt;/em&gt; point $x_i \in X$. So, if $N = 30,000$, for example, then we have to run through Steps 1-5 30,000 times. Each iteration is independent: there needs to be no communication between running iteration $i$ and iteration $j$. They all rely on the same common dataset, which is $X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done this in Matlab before and it was pretty easy to code up, but even with vectorizing as much as I could, it got a bit slow when N = 30,000 and k = 400. I decided as a good lesson for myself, to try to write it in C++. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thoughts on the C++ code were the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: Step 1 requires computing nearest neighbors. Perhaps it would be helpful to use a tree structure to keep in mind what points are nearby. I used Boost's spatial library to set up an R Tree and I query that for step 1 to build the set $X_i$. Please see &lt;a href=&quot;http://www.boost.org/doc/libs/1_58_0/libs/geometry/doc/html/geometry/spatial_indexes/introduction.html&quot; rel=&quot;nofollow&quot;&gt;http://www.boost.org/doc/libs/1_58_0/libs/geometry/doc/html/geometry/spatial_indexes/introduction.html&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2: Since i needed linear algebra, but my matrices were &quot;small&quot;, I figured something like Trilinos or Petsc would be overkill, so I went with Armadillo. I use armadillo to build the matrices $A^i$, the vectors, and handle the solve step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3: I'm not very experience with OOP, so I may have made a major faux pas here. I made a class Assembler that has the dataset $X$ and the Rtree as member variables (it builds them), and then it has a method I can call that will instantiate an object L, which L then runs steps 1-5 and stores the solution $f^i$ and the indices as member variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4: I have what looks to me to be an embarassingly parallel problem, so I call my Assembler class's method to build up the individual objects L and stick that in a for loop for i=1 to i= N. I then just stuck an OpenMP pragma around the for loop. This gave me pretty good speedups (better than Matlab by a lot, depending on what machine I ran it on).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are the following: Am I on the right track? Anyone have any suggestions or advice on how to improve things? Any suggestions on speed-ups or improved parallelism? If it would be helpful, I can get a link to the code so people can take a look at it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Another question I forgot to ask: each iteration of building object L_i runts through Steps 1-5, but at the end of the day, all I want is to store (for later use) the coefficient vector $f^i$ and the index vector mapping $X_i$ back to $X$ (both of which are member variables object $L_i$ finds during Steps 1-5). Naively, I'm inclined to write these to a csv file or binary file or something, and have one file per object $L_i$, but I'm not exactly sure if that's smart. In MATLAB, I had a sparse matrix $S$ where column $i$ of $S$ was the vector $f^i$ indexed properly with the map that sends $X_i \to X$. Since $k &amp;lt;&amp;lt; N$, $S$ was a sparse matrix. This got super slow in MATLAB, as I guess I wasn't allocating $S$ properly and it was extending itself or something.&lt;/p&gt;&#xA;" OwnerUserId="6567" LastEditorUserId="6567" LastEditDate="2015-12-05T18:48:45.043" LastActivityDate="2015-12-06T15:59:27.120" Title="Advice on handling many &quot;small&quot; matrices in parallel" Tags="|linear-algebra|parallel-computing|c++|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21488" PostTypeId="1" AcceptedAnswerId="21499" CreationDate="2015-12-05T20:39:20.583" Score="5" ViewCount="232" Body="&lt;p&gt;I am in the process of implementing adaptive mesh refinement for a finite element code that solves the Poisson equation. I have had some trouble finding good references on deciding which elements to refine. I have come across the Kelly error estimator from the &lt;a href=&quot;https://www.dealii.org/8.3.0/doxygen/deal.II/classKellyErrorEstimator.html#ab090ebe61ce691160142af39036f92d2&quot; rel=&quot;nofollow noreferrer&quot;&gt;deal.ii&lt;/a&gt; library:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \eta^{2} = \Sigma_{F\epsilon{}\partial{K}}C_{F}\int_{\partial{K}_{F}}(\nabla{u_{h}}\dot{}n)^{2}do&#xA;\end{equation} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions about using this error estimator:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) My first question is about how to solve this integral. Let me run through what I am currently doing. Consider the triangular element: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/siEDk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/siEDk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{1}$, $u_{2}$, and $u_{3}$ is the approximate solution at each node, i.e. $u_{h} = [u_{1},u_{2},u_{3}]$. Now to solve this integral over the first face $F_{1}$, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qcJ5p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qcJ5p.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I calculate $(\nabla{u_{h}}\dot{}n)^{2}$ as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  (\nabla{u_{h}}\dot{}n)^{2} = [(u_{1}\frac{\partial{N_{1}}}{\partial{x}} + u_{2}\frac{\partial{N_{2}}}{\partial{x}} + u_{3}\frac{\partial{N_{3}}}{\partial{x}})n_{x} + (u_{1}\frac{\partial{N_{1}}}{\partial{y}} + u_{2}\frac{\partial{N_{2}}}{\partial{y}} + u_{3}\frac{\partial{N_{3}}}{\partial{y}})n_{y}]^{2}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where for the linear triangle $\frac{\partial{N_{1}}}{\partial{x}}$, $\frac{\partial{N_{2}}}{\partial{x}}$, $\frac{\partial{N_{3}}}{\partial{x}}$, $\frac{\partial{N_{1}}}{\partial{y}}$... are all constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus to calculate $\eta$ I first parameterize face 1 using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  x = x_{1}(1-s) + x_{2}s\\&#xA;  y = y_{1}(1-s) + y_{2}s&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can then calculate the line integral as (for face $F_{1}$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \int_{0}^{1}(\nabla{u_{h}}\dot{}n)^{2}\sqrt{(dx/ds)^{2}+(dy/ds)^{2}}ds =(\nabla{u_{h}}\dot{}n)^{2}\sqrt{(x_{2}-x_{1})^{2}+(y_{2}-y_{1})^{2}}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where again for linear triangles $(\nabla{u_{h}}\dot{}n)^{2}$ is constant. My first question is whether this how we correctly calculate this line integral on each face?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Assuming that I am indeed calculating $\eta^{2}$ correctly, how do we use this to determine which elements should be refined? Do you find the maximum $\eta^{2}$ and then just refine only elements that are a certain percentage of the maximum?&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-12-15T05:43:11.737" LastActivityDate="2015-12-15T05:43:11.737" Title="Using finite element error estimators for adaptive mesh refinement" Tags="|finite-element|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21490" PostTypeId="2" ParentId="21487" CreationDate="2015-12-05T23:51:59.457" Score="2" Body="&lt;p&gt;The linear algebra part of the problem looks like something that would be ideal for use on GPUs; every matrix has the same size and there are a whole bunch of them. You can use the routines available in the &lt;a href=&quot;http://viennacl.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;ViennaCL&lt;/a&gt; for this. ViennaCL has backends for OpenMP, OpenCL and CUDA, so you can, in principle, use it on a multicore CPU or on any GPU; you're not restricted to just NVIDIA hardware as you would be with a CUDA library like cuBLAS. There's also &lt;a href=&quot;https://github.com/clMathLibraries&quot; rel=&quot;nofollow&quot;&gt;clBLAS&lt;/a&gt; and any number of alternatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you decide to use GPUs, you might also be able to build the matrices on the GPU directly, which will save the overhead of moving the memory from the CPU to the GPU. Since you have 30000 matrices of size 400 x 400, this would be a huge savings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you decide not to go with GPUs, you could also try using a library like &lt;a href=&quot;http://math-atlas.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;ATLAS&lt;/a&gt; together with Armadillo, which may provide faster dense linear algebra kernels than the default ones that Armadillo uses. There's more information about using optimized linear algebra libraries on the &lt;a href=&quot;http://arma.sourceforge.net/faq.html&quot; rel=&quot;nofollow&quot;&gt;Armadillo FAQ&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also mentioned using a class to assemble the matrices for the problem and were concerned with whether or not this was a good idea. I doubt this is going to be much of a concern. The performance penalties you run into with object-oriented programming are usually a result of dynamic dispatch, which requires a level of pointer indirection. However, this will be minuscule compared to the floating point operations in building a 400 x 400 matrix. You may want to investigate other spatial data structures than R-trees, like using a &lt;a href=&quot;http://blog.notdot.net/2009/11/Damn-Cool-Algorithms-Spatial-indexing-with-Quadtrees-and-Hilbert-Curves&quot; rel=&quot;nofollow&quot;&gt;Z-ordering&lt;/a&gt; and throwing out the tree data structure altogether. My suspicion is that this will, again, pale in comparison to the linear algebra operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This brings me to my final point: a good profiling tool (&lt;a href=&quot;http://www.scalasca.org/&quot; rel=&quot;nofollow&quot;&gt;scalasca&lt;/a&gt;, &lt;a href=&quot;http://www.cs.uoregon.edu/research/tau/home.php&quot; rel=&quot;nofollow&quot;&gt;TAU&lt;/a&gt;, &lt;a href=&quot;http://www.ompp-tool.com/&quot; rel=&quot;nofollow&quot;&gt;ompP&lt;/a&gt;) will tell you far more about where your code spends the most time, and therefore how to optimize it, than some guy on the internet will.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-12-05T23:51:59.457" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21491" PostTypeId="2" ParentId="21483" CreationDate="2015-12-05T23:59:41.230" Score="1" Body="&lt;p&gt;You can try starting from the smaller elements and adding them and then going to bigger values. This reduces the roundoff error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also try operator precedence for parentheses. To elaborate, assume that you want to compute&#xA;$$&#xA; y = ax + bx^2 + cx^3&#xA;$$&#xA;note that you have 10 arithmetic (2 add + 8 mult) with some roundoff error.&#xA;In your implementation instead if you write like this&#xA;$$&#xA;y = x(a + x(b + cx))&#xA;$$&#xA;you will have 7 operations (2 add + 3 mult) and you may get less roundoff.&lt;/p&gt;&#xA;" OwnerUserId="17327" LastActivityDate="2015-12-05T23:59:41.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21492" PostTypeId="2" ParentId="21487" CreationDate="2015-12-06T15:36:09.017" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For finding the nearest neighbors quickly, a KD-Tree would probably be faster than a R-Tree: a R-Tree supports dynamic updates, that you do not need (and you pay a price for it). Some implementations of KD-Tree are avaible in David Mount's ANN library [1] and in my Geogram library [2]. Note that ANN is not thread-safe (but it is quite easy to make it tread-safe by changing a little bit its source-code / replacing some global variables by function call parameters). Geogram is thread-safe.&#xA;There is also a FLANN library [3] that is thread-safe (but with this one I did not manage to have exactly the nearest neighbors, there was always an approximation, but maybe I did not use it correctly).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another word on nearest-neighbors: if you do not need exactly the k nearest neighbors but can do some approximations, then you may gain a little bit, see the documentation of ANN and FLANN.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With a size of 400x400, maybe using a standard dense linear solver will be the fastest, such as those in LAPACK, especially if you find a version of LAPACK optimized for your processor, using vector instructions (AVX2). GPU may be faster, but it can be a pain to setup and to program. If you want something more modern than LAPACK, you may try also eigen [4], it has many different solvers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To summarize, to answer your question, I would do a big #pragma omp parallel for loop, that retreives the neighbors using a Kd-Tree and solves the linear system using dense linear algebra with AVX instructions for vectorization. In the loop, avoid doing any dynamic memory allocation, since it breaks multithreading (on most OS/c++ runtimes, there is a global lock on dynamic memory allocation/deallocation). Use instead local variables (allocated on the stack, each thread has an independent one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Last but not least&lt;/strong&gt; : I agree with D. Shapero's remark, before starting to optimize anything, it is very important to &lt;strong&gt;run a profiler&lt;/strong&gt;: each time I did that, I was surprised, the bottleneck was absolutely not where I expected it !!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.cs.umd.edu/~mount/ANN/&quot; rel=&quot;nofollow&quot;&gt;https://www.cs.umd.edu/~mount/ANN/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Geogram Kd-Tree class: &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/kd__tree_8h.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/kd__tree_8h.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://www.cs.ubc.ca/research/flann/&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.ubc.ca/research/flann/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;http://eigen.tuxfamily.org&quot; rel=&quot;nofollow&quot;&gt;http://eigen.tuxfamily.org&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2015-12-06T15:59:27.120" LastActivityDate="2015-12-06T15:59:27.120" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21493" PostTypeId="2" ParentId="21230" CreationDate="2015-12-06T15:40:00.413" Score="2" Body="&lt;p&gt;I am not surprised at all that it's not in Numpy. Numpy is focused on floating-point and array/matrix computation, not on number theoretic functions and operations on integers. I can understand it needing internally a &lt;code&gt;gcd&lt;/code&gt; function for some exotic array/stride computation, but it's really not the main point of the library. &lt;a href=&quot;http://doc.sagemath.org/html/en/tutorial/tour_numtheory.html&quot; rel=&quot;nofollow&quot;&gt;Sage&lt;/a&gt; would be the first place I'd look, not numpy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed it has this function, and it looks very fast:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sage: %timeit lcm(range(1,1000))&#xA;100 loops, best of 3: 820 µs per loop&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you are doing number theoretical computations, I'd recommend you to move to Sage instead of pure Python. You'll find that generally it has more of the stuff you need already implemented.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2015-12-06T15:40:00.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21494" PostTypeId="1" CreationDate="2015-12-06T16:33:09.753" Score="3" ViewCount="89" Body="&lt;h1&gt;Short version&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I'm computing the zero-order &lt;a href=&quot;https://en.wikipedia.org/wiki/Hankel_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hankel transform&lt;/a&gt; of a function $f(r)$,&#xA;$$F(k) = \int_0^\infty f(r)J_0(kr)r\,\mathrm{d}r$$&#xA;I know $f(r_n)$ at selected $r_n$ but it is impractical to compute $f$ at additional values of $r$.&#xA;I need the transformed function $F(k)$ to have the best possible accuracy at large values of $k$, especially up to $k\sim 10^4$ but the higher the better, and I'm looking for algorithms or techniques that might aid this process.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Details&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I know a few things about this function on theoretical grounds:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$f(0) = 1$ and $f'(0) = 0$&lt;/li&gt;&#xA;&lt;li&gt;$0 &amp;lt; f(r) \leq 1$ for all $r$&lt;/li&gt;&#xA;&lt;li&gt;$f(r)\to 0$ as $r\to\infty$, roughly like $r^{-4}$&lt;/li&gt;&#xA;&lt;li&gt;the function is smooth and has no singularities&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Beyond that, what I have to work with are a set of numerical abcissas $r_n$ and corresponding function values $f_n$.&#xA;In theory the abcissas are supposed to be logarithmically spaced, $r_n = r_0 \delta^n$, but I think some values got mangled along the way and they're a little off from exact logarithmic spacing.&#xA;Anyway, the point is, it's &lt;em&gt;nearly&lt;/em&gt; impossible to evaluate the function at arbitrary values of $r$.&#xA;For purposes of this question, I have a list of points $(r_n, f_n)$ and that's it.&#xA;Effectively, the abcissas $r_n$ are $0$ (because I know $f(0) = 1$), $10^{-8}$, and then a set of 5000 values spanning the range from roughly $4\times 10^{-5}$ to $900$, roughly logarithmically spaced. A log-log plot of a &lt;a href=&quot;http://static.ellipsix.net/ext-tmp/test_data.dat&quot; rel=&quot;nofollow noreferrer&quot;&gt;representative data set&lt;/a&gt; looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nn8Ch.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nn8Ch.png&quot; alt=&quot;plot of very smoothly varying function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the function is very well-behaved: smooth, slowly varying, high sample rate in the region where it deviates from linearity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to now I've been working with a set of transformed values which I &lt;em&gt;think&lt;/em&gt; were generated using something like linear interpolation and Gaussian quadrature with a constant weight function.&#xA;The original code to do the transformation is missing, which is why I'm not 100% sure if that was the method used.&#xA;Anyway, some of the computed values for $F(k)$ at $k\gtrsim 1000$ are negative; for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k^2      F(k)&#xA;1032.56  7.69276e-08&#xA;1179.5   -6.37344e-08&#xA;1347.35  -2.88371e-07&#xA;1539.08  -2.47056e-07&#xA;1758.09  -4.05768e-07&#xA;2008.28  -2.65087e-07&#xA;2294.06  -2.60826e-07&#xA;2620.51  -2.93459e-07&#xA;2993.42  -2.86159e-08&#xA;3419.4   -3.24329e-07&#xA;3905.99  5.79954e-08&#xA;4461.83  -2.02765e-07&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know, on theoretical grounds, that this cannot be true. Theoretically, $F(k) &amp;gt; 0$ for all $k$. Hence the need for high accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any specialized algorithms or techniques that can be used to improve the accuracy of $F(k)$, especially at large $k$?&#xA;Speed is not an issue; the amount of computing power I can throw at this problem is far more than I can imagine it needing.&lt;/p&gt;&#xA;" OwnerUserId="103" LastEditorUserId="103" LastEditDate="2015-12-07T17:34:51.407" LastActivityDate="2015-12-07T17:34:51.407" Title="Hankel transform with high accuracy" Tags="|fourier-transform|integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21496" PostTypeId="2" ParentId="21494" CreationDate="2015-12-06T17:49:46.120" Score="4" Body="&lt;p&gt;For large values of $k$, the Hankel transform depends primarily on $f(r)$ at small values of $r$.  Your smallest values of $r$ aren't small enough to get good coverage.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example at $k=1 \times 10^{5}$, and your smallest value of $r=4 \times 10^{-5}$, $kr=4$, so you've basically skipped over the first cycle of the Bessel function.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you know that $f(r)$ is smooth, you might try smoothly interpolating values of $f(r)$ and then applying a library routine for computing the Hankel transform from the interpolated values.  The accuracy of this approach will depend on how well your interpolated values of $f(r)$ match the actual values of $f(r)$ for small $r$. If your values of $f(r)$ aren't very precise (as you indicate), then this probably won't work well.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might get a more helpful answer if you gave us more background on your problem.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-12-06T17:49:46.120" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21497" PostTypeId="2" ParentId="21475" CreationDate="2015-12-06T21:34:25.450" Score="1" Body="&lt;p&gt;You have a relation for $\gamma_m$:&#xA;$$&#xA;\gamma_m =  \frac {\alpha_7}{1-c+ \frac{c}{ \phi}} =: f(t_m, \rho_{vm}; t,x)&#xA;$$&#xA;which can be used to get an equation of the form&#xA;$$&#xA;  \frac{\partial \gamma_m}{\partial t} = \beta_\rho(t_m, \rho_{vm})\,\frac{\partial \rho_{vm}}{\partial t} + \beta_t(t_m, \rho_{vm})\,\frac{\partial t_m}{\partial t} + \beta_c(t_m, \rho_{vm})&#xA;$$&#xA;You can then use the expression for the derivative of $\gamma_m$ &#xA;$$&#xA;\frac { \partial \gamma_m}{ \partial t} = \alpha_6(\rho_{vg}-\rho_{vm})&#xA;$$ &#xA;to write&#xA;$$&#xA; \frac{\partial \rho_{vm}}{\partial t} = \alpha(t_m,\rho_{vg},\rho_{vm})  + \beta(t_m, \rho_{vm})\,\frac{\partial t_m}{\partial t}&#xA;$$&#xA;Then the four equations that you have are&#xA;$$&#xA; \begin{aligned}&#xA;\frac{ \partial t_g}{ \partial t} &amp;amp;= \alpha_1\,(t_m-t_g)- u\frac { \partial t_g}{ \partial x}  \\&#xA;\frac{ \partial t_m}{ \partial t} &amp;amp;= \alpha_2\frac{ \partial^2 t_m}{ \partial x^2} + \alpha_3(\rho_{vg}-\rho_{vm}) - \alpha_4(t_m-t_g)\\&#xA;\frac { \partial \rho_{vg}}{ \partial t} &amp;amp;= - \alpha_5( \rho_{vg} -\rho_{vm}) - u\frac {\partial \rho_{vg}}{ \partial x} \\&#xA;\frac{\partial \rho_{vm}}{\partial t} &amp;amp;= \alpha(t_m,\rho_{vg},\rho_{vm})  + \beta(t_m, \rho_{vm})\,\frac{\partial t_m}{\partial t}&#xA;\end{aligned}&#xA;$$&#xA;where $t_g$, $t_m$, $\rho_{vg}$, $\rho_{vm}$ are the variables you want to solve for.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine the spatial derivatives fixed so that these are ordinary differential equations with $t$ as the independent variable. You will still not be able to solve the system because only three of these equations are independent.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2015-12-06T21:34:25.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21498" PostTypeId="1" AcceptedAnswerId="21503" CreationDate="2015-12-07T01:04:12.773" Score="2" ViewCount="1223" Body="&lt;p&gt;I've been given a second-order non-linear ODE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^{2}\theta(s)}{ds^{2}} = sf_{g}\cos{\theta} + sf_{x}\cos{\phi}\sin{\theta}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f_{g}, f_{x}$ and $\phi$ are constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions for $[0,L]$ are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$\frac{d\theta}{ds}\Big|_{s=0} = 0 $&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$\theta (L) = \theta_L$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I will be repeating this numerous times for different values of $L$, implementing the solution method as a function. The $\theta_L$ is just a constant assumed known each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unsure how to proceed with these boundary conditions (are they Neumann, Dirichlet etc.? I don't think so). I struggle when I try to convert the BVP into an IVP for the shooting method and I've read the documentation for bvp4c and ode45 over and over with no progress on this problem. I just can't seem to get started here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone give me some pointers or help? Thank you very much in advance.&lt;/p&gt;&#xA;" OwnerUserId="18442" LastEditorUserId="18442" LastEditDate="2016-11-27T21:02:00.700" LastActivityDate="2016-11-27T21:02:00.700" Title="Numerical method for a BVP with mixed boundary conditions (MATLAB)" Tags="|matlab|boundary-conditions|numerical-modelling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21499" PostTypeId="2" ParentId="21488" CreationDate="2015-12-07T01:05:00.690" Score="2" Body="&lt;p&gt;As mentioned in a comment below the question, the formula you state is incorrect: it should read &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \eta^{2} = \Sigma_{F\epsilon{}\partial{K}}C_{F}\int_{\partial{K}_{F}}([\nabla{u_{h}]}\dot{}n)^{2}do&#xA;\end{equation} &#xA;where $[\cdot]$ is the &lt;em&gt;jump&lt;/em&gt; of the quantity across the face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for your question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Because for linear elements, the gradient is constant. Consequently, if you integrate over a single face $f \subset \partial K_F$, you have that&#xA;\begin{equation}&#xA;  \int_{f}([\nabla{u_{h}]}\dot{}n)^{2}do&#xA;= ([\nabla{u_{h}]}\dot{}n)^{2} \int_{f}do&#xA;= ([\nabla{u_{h}]}\dot{}n)^{2} |f|.&#xA;\end{equation} &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You use the error indicators computed for all cells to identify, for example, the 30% of cells with the largest indicators and then refine these. Alternatively, you can identify those cells with the largest indicators that together amount for 90% of the total error. Take a look, for example, at the documentation of the functions here: &lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/namespaceGridRefinement.html#ad3b68e645838ebeb4f9c55352b56a0b3&quot; rel=&quot;nofollow&quot;&gt;https://www.dealii.org/developer/doxygen/deal.II/namespaceGridRefinement.html#ad3b68e645838ebeb4f9c55352b56a0b3&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-07T01:05:00.690" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21500" PostTypeId="2" ParentId="21478" CreationDate="2015-12-07T01:21:41.967" Score="2" Body="&lt;p&gt;If your question is why the word &quot;sparse matrix&quot; never appears in presentations on finite difference (or finite element, or finite volume) solvers for partial differential equations, then that's because that's always implied: it simply makes no sense to store the matrix in any way that doesn't exploit sparsity. That's true for sequential and parallel applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because it's the only reasonable way to do it, it's not usually mentioned.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-07T01:21:41.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21501" PostTypeId="1" CreationDate="2015-12-07T01:42:50.330" Score="2" ViewCount="194" Body="&lt;p&gt;I am considering using &lt;code&gt;petsc4py&lt;/code&gt; instead of &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; (which is a wrapper for Fortran solvers) for a problem involving the solution of a system of ODEs. The problem has the potential to be stiff. Writing down its Jacobian is very hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have been able to produce reasonable speed gains by writing the RHS functions in &quot;something like C&quot; (using either &lt;code&gt;numba&lt;/code&gt; or &lt;code&gt;Cython&lt;/code&gt;). I'd like to get even more performance out, hence my consideration of PETSc. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Introducing a toy version of the problem: The Amoeba&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Consider the following figure, which defines the basics of an amoeba:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gW542.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gW542.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, an amoeba consists of $N$ nodes and edges, constrained to a 2D space. The position of the nodes describing the amoeba evolve with time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the position of the $n$th ($n \in \{0, 1, ..., N-1\}$) node be described a position vector, $\textbf{x}_n$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textbf{x}_n = \left[\begin{array}{cc} x_{0,n} \\ x_{1,n} \end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the purposes of simplification, assume that the amoeba lives in a world with first order equations of motion, so given an applied force $\textbf{F}$ on a node, the nodal velocity $\dot{\textbf{x}}$ is produced:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \textbf{F} = c \dot{\textbf{x}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c$ is a constant I pick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are very interested in tracking the position of each node,so immediately we produce $2n$ ODEs (one ODE for each spatial component) of interest (assume that edges are elastic springs):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\textrm{d}\textbf{p}_n}{dt} = \text{elastic edge forces} + \textbf{F}_{n, active}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{F}_{n, active}$ is the force generated by the amoeba at node $n$. One can imagine a wide range of rules that could be devised to determine how $\textbf{F}_{n, active}$ is determined, and I suggest that we consider the following rules:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) There are red balls and blue balls inside the amoeba. &#xA;^&#xA;2) Both red and blue balls have &quot;switched-on&quot; (symbolized $r^*$/$b^*$) and &quot;switched-off&quot; (symbolized $r$/$b$) state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Both red and blue balls can be found either in the body of the amoeba where they are always switched off, or at the nodes of the amoeba, where they can be switched on or switched off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) The total number of red balls regardless of location of status is a constant $R$ and similarly for blue balls, a constant $B$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) When red or blue balls are in the body, they are free to jump onto any node they want, but when they are on nodes, some of them might be induced to move to neighbouring nodes due to some sort of &quot;diffusion&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6) Switched on red balls at a node cause a force that attempts to reduce the area of the amoeba&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7) Switched on blue balls at a node cause a force that attempts to increase the area of the amoeba&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8) Switched on red balls tend to switch off blue balls that are switched on, and vice versa, switched on blue balls tend to switch off red balls that are switched on &lt;/p&gt;&#xA;&#xA;&lt;p&gt;9) ...&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Okay, you get my point. I can come up with a model that is fairly convoluted in terms of how the various variables of interest interact.&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Due to the large number of equations involved, it is already tedious to think about writing downa Jacobian. However, since we have access to a computer, it may be that we can write functions governing a particular interaction that do not have neat analytical forms (let alone whether or not their derivatives have neat analytical forms), so we might have a mess of piecewise functions needed to approximate them if we were to go about still trying to produce a Jacobian...&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Things get even worse if my problem involves not just one amoeba, but multiple amoebae (which is when things ARE interesting), with new interaction rules enforcing that amoebae cannot violate each others' areas, and so on.&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;All the toy examples I see of PETSc time stepping problems have Jacobians defined, so I wonder if I would even get a speed gain going from switching to it, if perhaps one of the reasons why I have a high computational cost is due to not being able to provide a Jacobian function?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2015-12-07T01:42:50.330" Title="Is it worth switching to timesteppers provided by PETSc if I can't write down a Jacobian for my problem? Case study with &quot;the amoeba&quot; toy problem" Tags="|python|petsc|time-integration|scipy|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21502" PostTypeId="1" CreationDate="2015-12-07T10:45:04.023" Score="3" ViewCount="189" Body="&lt;p&gt;In a parallel MPI code, I have load balancing issues. My 2D computational domain is distributed on a 2D MPI Cartesian topology, which leads to equal sized 2D sub-domains per MPI process. However, the amount of work to do per domain is determined by the number of &quot;compute points&quot; belonging to each domain. And the density of these compute points is very uneven, leading to some processes having much much more of them than others. Therefore, the overall parallel efficiency of my code might become quite poor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this issue, I want to resize my 2D domain decomposition, to re-balance the number of compute points across my MPI processes. I'm therefore looking for an algorithm permitting me to achieve this optimisation process. The constrains are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The X and Y dimensions &lt;code&gt;(Gx, Gy)&lt;/code&gt; of the global grid to distribute are fixes (integer representing the number of grid points to distribute across MPI processes in each dimensions)&lt;/li&gt;&#xA;&lt;li&gt;The total number of MPI processes is fixed. Let &lt;code&gt;Ntot&lt;/code&gt; be this number, then we have to keep &lt;code&gt;Nx * Ny = Ntot&lt;/code&gt; for any valid &lt;code&gt;(Nx, Ny)&lt;/code&gt; solution.&lt;/li&gt;&#xA;&lt;li&gt;The number of X and Y grid points for each MPI process can be different. The only limit is that the sum of the individual X grid points for all MPI processes should be equal to &lt;code&gt;Gx&lt;/code&gt;. Likewise for &lt;code&gt;Gy&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The density of compute point on the grid is known a priori. Each grid point might have anywhere between zero and a ten such compute points.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any idea of any method / algorithm to solve this problem?&#xA;I'm fully aware that there might be no perfect solution, but any mapping of the grid permitting to improve the load balancing would already be good to take.&lt;/p&gt;&#xA;" OwnerUserId="18449" LastEditorUserId="18449" LastEditDate="2015-12-07T11:35:11.797" LastActivityDate="2015-12-07T18:18:34.670" Title="Balancing number of points in a 2D grid" Tags="|optimization|algorithms|mpi|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21503" PostTypeId="2" ParentId="21498" CreationDate="2015-12-07T12:23:42.830" Score="3" Body="&lt;p&gt;To start you could change it into a system of first-order ODEs&#xA;$$Y(s) := \left(&#xA;  \begin{split}&#xA;    \theta'(s)\\ &#xA;    \theta(s)&#xA;  \end{split}&#xA;\right) &#xA;= &#xA;  \left(\begin{split}&#xA;    Y_1(s)\\ &#xA;    Y_2(s)&#xA;  \end{split}&#xA;\right)\,,$$&#xA;$$ Y'(s) = \left(&#xA;  \begin{split}&#xA;    s f_g \cos(Y_2(s)) + s f_x \cos \phi\sin(Y_2(s))\\&#xA;    Y_1(s)&#xA;  \end{split}&#xA;\right)\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Y(0) = \left(\begin{align} 0\\ t \end{align}\right)\,.$$&#xA;Then the shooting method, parametrised by $t$ as initial conditions, that means use different values of $t$ as an initial condition. Approximate value of $t$ can be find in the style taught in this &lt;a href=&quot;https://www.youtube.com/watch?v=ZMgikZ-lcS8&quot; rel=&quot;nofollow&quot;&gt;lecture&lt;/a&gt;. The easiest root finding method is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bisection_method&quot; rel=&quot;nofollow&quot;&gt;bisection method&lt;/a&gt;. For that you should know a range in which the value of $t$ lies $[t_a t_b]$. Substitute $t_a$, $t_b$ and $t_c$ (mid point of $t_a$ and $t_b$), as three different I.C and solve that using  &lt;a href=&quot;http://personal.egr.uri.edu/sadd/mce372/Higher%20Order%20ODEs.pdf&quot; rel=&quot;nofollow&quot;&gt;ode45&lt;/a&gt; then &lt;a href=&quot;https://www.youtube.com/watch?v=EcrIxpjTho8&quot; rel=&quot;nofollow&quot;&gt;check&lt;/a&gt; weather you are getting closer to $Y_2(L)=\theta_L$, then reduce the search range in bisection algorithm algorithm until you get desired accuracy.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="16930" LastEditDate="2015-12-09T16:46:44.537" LastActivityDate="2015-12-09T16:46:44.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21504" PostTypeId="2" ParentId="21469" CreationDate="2015-12-07T13:40:19.703" Score="4" Body="&lt;p&gt;There are two parts of the answer. First, you don't get the identity because you throw away information during the restriction operation (if you think of a larger mesh than just the three points you consider, then you &quot;forget&quot; the values of ever other node during restriction), and you cannot recover this information during the transpose operation. Consequently, you will not be able to get the &lt;em&gt;identity&lt;/em&gt; operation when applying $I^TI$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, your intuition that you should get something that has a norm around one is correct, and you would if you would choose the prolongation matrix differently. In your statement, you choose it as the transpose of the restriction matrix. This is a convenient choice because it makes the multigrid cycle a &lt;em&gt;symmetric&lt;/em&gt; operator. But the transpose of the restriction is not equivalent to the &lt;em&gt;embedding&lt;/em&gt; matrix $E$ that would simply interpolate the function values from the coarse grid to the fine grid. If you considered the operator $EI$, then this one has the property you are looking for. In particular, $EI$ is &lt;em&gt;idempotent&lt;/em&gt;: if you apply it twice, the result is the same as if you applied it once: $(EI)(EI)=EI$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-07T13:40:19.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21505" PostTypeId="2" ParentId="21502" CreationDate="2015-12-07T18:18:34.670" Score="2" Body="&lt;p&gt;This is typically done with graph partitioning algorithms, and packages such as &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/views/metis&quot; rel=&quot;nofollow&quot;&gt;METIS&lt;/a&gt; and ParMETIS (same link) are often used to do so.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-12-07T18:18:34.670" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21508" PostTypeId="1" CreationDate="2015-12-07T21:42:56.907" Score="0" ViewCount="1715" Body="&lt;p&gt;I'm working on a computational physics assignment and I was looking for some help as I've got stuck!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Write a function to create the finite-difference approximation of the 2nd derivative operator matrix for a staggered grid. Given the inputs $N$ (the size of the matrix) and $\delta x$ (the grid spacing), the function should return the tridiagonal matrix in the form of three arrays $(a,b,c)$. Include the Neumann boundary conditions $u0(−1) = u0(1) = 0$ by modifiying specific elements of $b$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The aim of the assignment is to write a few functions in order to solve this non-linear elliptic equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u'' + 500e^{−100 x^2} − u^3 = 0\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a practice class I wrote a more simple script for solving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u''(x) = f(x) = - e^{x}(-2 + 2x + 5x^2 + x^3)\,.$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from pylab import *&#xA;from scipy import interpolate&#xA;&#xA;def EllipticSolver(Nx):&#xA;    dx = 1.0/Nx&#xA;    pts = linspace(-dx/2.0,1.+dx/2.0,Nx+2)&#xA;    soln = zeros(Nx+2)&#xA;    rhs = zeros(Nx+2)&#xA;    rhs = -exp(pts)*(-2.0 + 2.*pts + 5.*pts**2 + pts**3)&#xA;    soln = tridiagonal2(soln,rhs,dx)&#xA;    return pts[1:-1], soln[1:-1]&#xA;&#xA;def tridiagonal2(dat,d,dx):&#xA;    N = len(dat)&#xA;    print(N)&#xA;    dx2 = 1.0/(dx*dx)&#xA;    a = zeros(N)&#xA;    b = zeros(N)&#xA;    c = zeros(N)&#xA;    a[:] = dx2&#xA;    c[:] = dx2&#xA;    b[:] = -2.0*dx2&#xA;&#xA;    b[1] = -1.0*dx2&#xA;    b[N-2] = -3.0*dx2&#xA;&#xA;    c[1] = c[1]/b[1]&#xA;    d[1] = d[1]/b[1]&#xA;    for j in range(2,N-1):&#xA;        c[j] = c[j]/(b[j] - c[j-1]*a[j])&#xA;        d[j] = (d[j] - d[j-1]*a[j])/(b[j] - c[j-1]*a[j])&#xA;&#xA;    print(a)&#xA;    print(b)&#xA;    print(c)&#xA;    print(d)&#xA;&#xA;    dat[N-2] = d[N-2]&#xA;    for j in range(N-3,0,-1):&#xA;        dat[j] = d[j] - c[j]*dat[j+1]&#xA;&#xA;    return dat&#xA;&#xA;x1, soln1 = EllipticSolver(100)&#xA;x2, soln2 = EllipticSolver(200)&#xA;x3, soln3 = EllipticSolver(400)&#xA;&#xA;figure(1)&#xA;clf()&#xA;plot(x1,soln1,'r-')&#xA;plot(x2,soln2,'g-')&#xA;&#xA;s2 = interpolate.interp1d(x2,soln2,'cubic')&#xA;soln2a = s2(x1)&#xA;&#xA;s3 = interpolate.interp1d(x3,soln3,'cubic')&#xA;soln3a = s3(x2)&#xA;&#xA;diff1 = soln1 - soln2a&#xA;diff2 = soln2 - soln3a&#xA;&#xA;figure(2)&#xA;clf()&#xA;plot(x1,diff1,'r-')&#xA;plot(x2,4.*diff2,'g-')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, Newton-Raphson method, discretise using finite-difference approximations of the derivatives &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u'' = (u[i+1] + u[i-1] - 2u[i]) / dx^2 \,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the first thing I'm gonna have to do is set up some kind of array for $u$ in order to whack it in a for loop and populate it using the above equation I don't know what 'u' should be here, I could use an array of zeros, but I need some values in order to get something for $u''$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example above, I create the &lt;code&gt;rhs&lt;/code&gt; array using the staggered grid I produced called &lt;code&gt;pts&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to make another staggered grid for $u$, and then use that to make a RHS array: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rhs = zeros(N) &#xA;rhs = (ui+1 + ui-1 - 2ui) / dx2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That wouldn't quite work but something along those lines...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also feel like I should be using the boundary conditions to find some initial conditions for $u$ also&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u'(1) = u'(-1) = 0$$&#xA;so then $u''(0) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers would be appreciated, I'm really unsure how to input this extra variable $u$ properly.&lt;/p&gt;&#xA;" OwnerUserId="18461" LastEditorUserId="17671" LastEditDate="2015-12-08T13:31:12.017" LastActivityDate="2015-12-11T11:04:13.223" Title="Finite-difference approximation of the 2nd derivative operator matrix for a staggered grid" Tags="|finite-difference|python|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21509" PostTypeId="2" ParentId="21475" CreationDate="2015-12-08T05:55:07.767" Score="1" Body="&lt;p&gt;Just discretize the all the spatial term of every equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here i'm concentrating only first equation other equation should be done in similar style. &#xA;Lets use &lt;a href=&quot;http://mathfaculty.fullerton.edu/mathews/n2003/differentiation/numericaldiffproof.pdf&quot; rel=&quot;nofollow&quot;&gt;CD4&lt;/a&gt; in space and RK4 in time because CD4 has good Dispersion preserving property and RK4 has relatively good stability. Please try to avoid Euler time integration at any cost if you are using central difference in space and the equation has dominant advection character because of stability issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that we should use some lower order methods at boundary where we can't apply CD4 stencil else if the boundary is periodic we can apply CD4 over the whole domain.&lt;br&gt;&#xA;\begin{equation}&#xA;\frac{ \partial t_g}{ \partial t} = \frac {hp(({t_m}_i-{t_g}_i)}{A(\rho_gCp_g)}- u\frac {  {t_g}_{i-2}-8*{t_g}_{i-1}+8{t_g}_{i+1}-{t_g}_{i+2}}{ 12*\delta x}   &#xA;\end{equation} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Form the initial condition you know $t_g$ and $t_m$ values in the domain at t=0 now we should march in time. &#xA;Lets assume &#xA;\begin{equation}&#xA;F= \frac {hp(({t_m}_i-{t_g}_i)}{A(\rho_gCp_g)}- u\frac {  {t_g}_{i-2}-8*{t_g}_{i-1}+8{t_g}_{i+1}-{t_g}_{i+2}}{ 12*\delta x}  &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation become &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{ d t_g}{ \partial t} =F(t_g)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using low &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0021999180900339&quot; rel=&quot;nofollow&quot;&gt;storage RK4 method&lt;/a&gt;&lt;br&gt;&#xA;this will become &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;{t_g}^0 = {t_g}^n&#xA;\end{equation}&#xA;here you can substitute Initial condition value of $t_g$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${t_g}^1 = {t_g}^0 + \alpha_1 \delta t F({t_g}^0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${t_g}^2 = {t_g}^0+\alpha_2 \delta t F({t_g}^1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${t_g}^3 = {t_g}^0+\alpha_3 \delta t F({t_g}^2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${t_g}^4 = {t_g}^0 + \alpha_4 \delta t F({t_g}^3)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${t_g}^{n+1}={t_g}^4 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\alpha_4=1/4$, $\alpha _3=1/3$, $\alpha _2=1/2$ and $\alpha _1=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like this do this for other equation. Since we are not having any time derivative of $\rho_{vm}$ we should use those algebraic equations to calculate $\rho_{vm}$ using newly calculated other variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that this kind of problem may cause dispersion and numerical diffusion so always use low CFL number -may be less than 0.5 and fine grid (this should be determined by trial and error if you know wave number of initial condition then it may be easy but most of the time we may not have single wave numbered initial condition).&lt;/p&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="16930" LastEditDate="2015-12-11T01:36:18.407" LastActivityDate="2015-12-11T01:36:18.407" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21510" PostTypeId="1" CreationDate="2015-12-08T06:51:39.723" Score="1" ViewCount="30" Body="&lt;p&gt;Given the functional $J_A(y,u) = \frac{1}{2}||y-y_d||^2_{L^2(w,\omega)} + \frac{\lambda}{2}||u||^2_{L^2(w^{-1}, \omega)}$ where w is a function belongs to Muckenhoupt class. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the optimization problem: min $J_A$ subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-$div(A\triangledown{y}) = u$ in $\omega$. Assume matrix $A$ satisfies these inequalities for almost every $x\in \omega$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w(x)|\zeta|^2 \leq C(\zeta^{T}\cdot A)$, &#xA;$\ \zeta^{T}\cdot A \leq D(w(x)|\zeta|^2)$ for some constants $C$ and $D$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y = 0$ on $\partial \omega$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the control constraints $u\in U_A$ where $U_A$ is a nonempty, closed and convex set of $L^{2}(w^{-1}, \omega)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in finding the dual problem + Lagrangian condition for this optimization problem. I also tried quite hard to prove the following inequality but still without any success: $(DJ_A(\bar{u}), (u-\bar{u}))\geq 0$ where $\bar{u} = $ optimal control solution to the optimization problem above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope someone can help me with either questions above. It's quite interesting, but also challenging, so I would really appreciate any help you can give.&lt;/p&gt;&#xA;" OwnerUserId="18464" LastActivityDate="2015-12-08T06:51:39.723" Title="Duality/Lagrangian condition and Variational Inequality of a cost functional" Tags="|finite-element|pde|" AnswerCount="0" CommentCount="1" ClosedDate="2019-06-06T20:32:29.953" ContentLicense="CC BY-SA 3.0" />
  <row Id="21512" PostTypeId="1" CreationDate="2015-12-08T09:38:04.007" Score="7" ViewCount="141" Body="&lt;p&gt;As an example for an ODE course I used the ODE&#xA;$$&#xA;y' = \frac{y}{x} + \frac{1}{\cos(\tfrac{y}{x})}&#xA;$$&#xA;to illustrate domains of existence. Standard substitution $z=y/x$ turns the equation to&#xA;$$&#xA;z' = \frac{1}{x\cos(z)}&#xA;$$&#xA;which can be solved by separation of variables to obtain the solution&#xA;$$&#xA;\phi(x) = x\arcsin(\ln(x/x_0) - \sin(y_0/x_0))&#xA;$$&#xA;with initial value $\phi(x_0)=y_0$&#xA;(and the solution ceases to exists at $x = x_0\exp(1-\sin(y_0/x_0))$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check that my solution is correct, I compared the solution with the output of some numerical solver (&lt;code&gt;ode45&lt;/code&gt; from MATLAB). Since I was to lazy to figure out the precise upper bound at which the solution ceases to exist, I just plugged in some upper bound and looked at the result. As expected, something weired happened beyond the point of existence. However, when comparing the numerical solution with the computed one (which is correct, by the way), I noticed something strange: Although the solution computed by the exact formula has a problem (when the argument of the $\arcsin$ becomes larger than one and from that point on MATLAB plot the real part and neglects the imaginary part of the solution), the &quot;wrong exact solution&quot; somehow follows that &quot;wrong numerical solution&quot; in a very crude, strange but not deniable way. Here is the plot for $x_0=1$, $y_0=0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JyIgI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JyIgI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(the red is the plot of $\phi$ and the blue is the output of &lt;code&gt;ode45&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is that? What is going on with an ODE solver beyond existence? Why is it following the real part of the non-existing complex solution?&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="713" LastEditDate="2015-12-08T17:14:25.420" LastActivityDate="2015-12-08T22:59:24.053" Title="Solving an ODE beyond existence. What's happening?" Tags="|ode|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21513" PostTypeId="2" ParentId="21508" CreationDate="2015-12-08T09:55:51.113" Score="2" Body="&lt;p&gt;On an irregular grid a possible finite difference operator is the tridiagonal matrix $A$  which gives&#xA;$$[AU]_i = \frac{\dfrac{U_{i+1}-U_i}{\delta_{i}} - \dfrac{U_{i}-U_{i-1}}{\delta_{i-1}}}{1/2(\delta_i+\delta_{i-1})}\,,$$&#xA;where $\delta_i = x_{i+1} - x_i$ and $U$ is a vector with $U_i \approx u(x_i)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U$ is the numerical solution you're looking for, so you wont be able to plug it in. You're going to want some sort of function to solve for $U$, so $f(U) = 0$.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="17671" LastEditDate="2015-12-11T11:04:13.223" LastActivityDate="2015-12-11T11:04:13.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21515" PostTypeId="1" CreationDate="2015-12-08T10:40:13.953" Score="1" ViewCount="57" Body="&lt;p&gt;I like to set my scientific simulation programs to generate a picture after a certain number of iterations, such that I can follow what is happening and maybe cancel the simulation before the initially planned iteration. The periodicity of the pictures must be such that they don't affect performance too much. Sometimes, I even create windows to enter the simulations' parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already combined&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fortran with C with OpenGL (the Fortran program generated a C/OpenGL&#xA;code and executed it)&lt;/li&gt;&#xA;&lt;li&gt;Java with Java 2D or Java 3D with GUI created with Swing (NetBeans)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Java has the advantage of portability, but I don't know how slower than Fortran and C it is, because I wrote different programs with each of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you use another solution for real-time image printing or visualization between N [preferably high-performance] iterations?&lt;/p&gt;&#xA;" OwnerUserId="18469" LastActivityDate="2015-12-08T20:37:04.853" Title="What are the computational solutions for periodic visualization of simulation?" Tags="|performance|visualization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21516" PostTypeId="1" CreationDate="2015-12-08T12:28:10.773" Score="1" ViewCount="112" Body="&lt;p&gt;I have some values imported from excel about the annual sell of a product, with 3 var: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;month, &#xA;price, &#xA;sold items&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen how to plot a chart using &lt;em&gt;curve fitting tool&lt;/em&gt; but I need to define a math expression that can describe the relation between these variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do this in matlab? Is there any tool that can achieve this automatically. If you look at the chart should exist a way to use an approximation that can mime the behavior(maybe with spline functions or n-point-hermit), is enough one approximated expression of the model that for the 3 var replicates a similar graph if plotted. &lt;/p&gt;&#xA;" OwnerUserId="18473" LastActivityDate="2015-12-08T12:28:10.773" Title="How to get mathematical model from a data set with MATLAB" Tags="|matlab|statistics|modeling|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="21517" PostTypeId="2" ParentId="21419" CreationDate="2015-12-08T13:01:25.533" Score="4" Body="&lt;p&gt;Quadratically constrained quadratic programming (QCQP) focuses on convex inequalities because those preserve the convexity of the problem. Quadratic equalities do not, so this problem is much harder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any polynomial equation can be re-written as a system of quadratic equations. (For example: $x^3 = 1$ is equivalent to $\{xy = 1,\, x^2 - y = 0\}$.) So solving a set of quadratic equations is not fundamentally easier than solving a set of polynomial equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Wikipedia page on &lt;a href=&quot;https://en.wikipedia.org/wiki/System_of_polynomial_equations&quot; rel=&quot;nofollow&quot;&gt;systems of polynomial equations&lt;/a&gt; explains the basics of how these systems are solved. Commercial packages like Maple and Wolfram can do this, and &lt;a href=&quot;http://ask.sagemath.org/question/11070/find-algebraic-solutions-to-system-of-polynomial-equations/&quot; rel=&quot;nofollow&quot;&gt;apparently&lt;/a&gt; so can the free, open source package Sage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of these packages, compared to a non-linear solver is that you would be guaranteed a solution if one exists. The disadvantage is that the search tree grows exponentially in the number of unknowns, so $N \approx 400$ might be impossible unless the inequalities permit very efficient pruning. &lt;/p&gt;&#xA;" OwnerUserId="18471" LastEditorUserId="18471" LastEditDate="2015-12-08T17:31:40.973" LastActivityDate="2015-12-08T17:31:40.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21518" PostTypeId="1" AcceptedAnswerId="21696" CreationDate="2015-12-08T13:33:08.957" Score="-1" ViewCount="1232" Body="&lt;p&gt;I have to use the LANL2DZpd basis set. It can be obtained from &lt;a href=&quot;https://bse.pnl.gov/bse/portal&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMSL Basis Set Exchange&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But trying to calculate simple Me2Se molecule :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%chk=test.chk&#xA;%NProcShared=12 &#xA;%mem=64300MB&#xA;# B3lyp/gen pseudo=read opt freq scale=0.9806 Freq NMR=GIAO  pop=(Full)&#xA;&#xA; Me2Se&#xA;&#xA;0  1&#xA;6        0.000000000      1.497766000     -0.847272000&#xA;34       0.000000000      0.000000000      0.488258000&#xA;6        0.000000000     -1.497766000     -0.847272000&#xA;1        0.000000000      2.433261000     -0.280015000&#xA;1        0.898342000      1.445398000     -1.468371000&#xA;1       -0.898342000      1.445398000     -1.468371000&#xA;1        0.000000000     -2.433261000     -0.280015000&#xA;1       -0.898342000     -1.445398000     -1.468371000&#xA;1        0.898342000     -1.445398000     -1.468371000&#xA;&#xA;H     0 &#xA;S   3   1.00&#xA;     19.2384000              0.0328280        &#xA;      2.8987000              0.2312040        &#xA;      0.6535000              0.8172260        &#xA;S   1   1.00&#xA;      0.1776000              1.0000000        &#xA;S   1   1.00&#xA;      0.0498000              1.0000000        &#xA;P   1   1.00&#xA;      0.3560000              1.0000000        &#xA;****&#xA;C     0 &#xA;S   7   1.00&#xA;   4233.0000000              0.0012200        &#xA;    634.9000000              0.0093420        &#xA;    146.1000000              0.0454520        &#xA;     42.5000000              0.1546570        &#xA;     14.1900000              0.3588660        &#xA;      5.1480000              0.4386320        &#xA;      1.9670000              0.1459180        &#xA;S   2   1.00&#xA;      5.1480000             -0.1683670        &#xA;      0.4962000              1.0600910        &#xA;S   1   1.00&#xA;      0.1533000              1.0000000        &#xA;P   4   1.00&#xA;     18.1600000              0.0185390        &#xA;      3.9860000              0.1154360        &#xA;      1.1430000              0.3861880        &#xA;      0.3594000              0.6401140        &#xA;P   1   1.00&#xA;      0.1146000              1.0000000        &#xA;P   1   1.00&#xA;      0.0311000              1.0000000        &#xA;D   1   1.00&#xA;      0.5870000              1.0000000        &#xA;****&#xA;Se     0 &#xA;S   2   1.00&#xA;      1.0330000             -3.3224095        &#xA;      0.6521000              3.6003462        &#xA;S   1   1.00&#xA;      0.1660000              1.0000000        &#xA;P   2   1.00&#xA;      2.3660000             -0.1185522        &#xA;      0.3833000              1.0414320        &#xA;P   1   1.00&#xA;      0.1186000              1.0000000        &#xA;P   1   1.00&#xA;      0.0328000              1.0000000        &#xA;D   1   1.00&#xA;      0.3630000              1.0000000        &#xA;****&#xA;! Elements                             References&#xA;! --------                             ----------&#xA;! Na - Hg: P. J. Hay and W. R. Wadt, J. Chem. Phys. 82, 270 (1985).&#xA;! P. J. Hay and W. R. Wadt, J. Chem. Phys. 82, 284 (1985).&#xA;! P. J. Hay and W. R. Wadt, J. Chem. Phys. 82, 299 (1985).&#xA;! &#xA;&#xA;&#xA;&#xA;SE     0&#xA;SE-ECP     3     28&#xA;f-ul potential&#xA;  5&#xA;1    433.1931336            -28.0000000        &#xA;2     83.8952157           -214.3841762        &#xA;2     18.5839139            -65.6918782        &#xA;2      5.3955286            -24.6153932        &#xA;2      1.7474326             -2.4481497        &#xA;s-ul potential&#xA;  5&#xA;0    202.8986193              3.0000000        &#xA;1     78.3820487             62.0295390        &#xA;2     35.0753037            258.8555984        &#xA;2     10.8769543            118.7800153        &#xA;2      2.8005941             38.2355279        &#xA;p-ul potential&#xA;  5&#xA;0     44.3011875              5.0000000        &#xA;1     20.3874206             24.7973458        &#xA;2     23.1889948             63.7575640        &#xA;2      7.9777664             79.0512831        &#xA;2      2.2988146             22.9520183        &#xA;d-ul potential&#xA;  5&#xA;0     73.3628263              3.0000000        &#xA;1     48.3835618             22.4705907        &#xA;2     25.6297211            140.5492887        &#xA;2      7.1705822             63.5781835        &#xA;2      1.3639538              7.0753614   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get geometry like &lt;a href=&quot;https://i.stack.imgur.com/fLxFv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C        0.000000000      0.717597000     -0.898755000&#xA;Se       0.000000000      0.000000000     -0.628062000&#xA;C        0.000000000     -0.717597000     -0.898755000&#xA;H        0.000000000     -0.012348000     -0.279085000&#xA;H        0.743639000      1.161241000     -1.164015000&#xA;H       -0.743639000      1.161241000     -1.164015000&#xA;H        0.000000000      0.012348000     -0.279085000&#xA;H       -0.743639000     -1.161241000     -1.164015000&#xA;H        0.743639000     -1.161241000     -1.164015000]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I am doing wrong and how it can be fixed?&lt;/p&gt;&#xA;" OwnerUserId="18476" LastEditorUserId="18476" LastEditDate="2015-12-08T14:05:16.830" LastActivityDate="2015-12-29T13:55:37.810" Title="Gaussian and external basis set. How to write the input correctly" Tags="|computational-chemistry|quantum-mechanics|" AnswerCount="1" CommentCount="4" ClosedDate="2015-12-30T17:37:37.013" ContentLicense="CC BY-SA 3.0" />
  <row Id="21520" PostTypeId="2" ParentId="21512" CreationDate="2015-12-08T16:36:29.950" Score="6" Body="&lt;p&gt;I'd say the key is when you say &quot;the solution $\phi(x)$&quot;, because the solution to ($x_0=1, y_0=0$)&#xA;$$ \sin(y/x) = \log x $$&#xA;is one of &#xA;$$ y = x\big( \arcsin\log x + 2\pi k \big), \qquad y = x\big( \pi - \arcsin\log x + 2\pi k\big) $$&#xA;for an appropriate integer $k$. The ODE itself is well-defined beyond the location of singularity, unlike the solution that starts at $(1,0)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, there is a slight bug in the stepping function — it &lt;em&gt;should&lt;/em&gt; be able to correctly detect the singularity, but it doesn't. Instead, it steps just past the singularity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the singular points of the ODE in the $(x,y)$-plane are defined by $\cos(y/x)=0$, so in this case it is the line $y=\frac\pi2 x$ that matters. When the ODE solver steps &quot;through&quot; a singularity, it ends up at some value that is at $\tilde x$ just after the location of the singularity $x_{\mathrm{sing}}$, and at $\tilde y$ just above (on the other side of) the singular value $\frac\pi2 \tilde x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It isn't really correct that it follows &quot;the real part of the complex solution&quot;. There is a well-defined real solution that starts at initial conditions $(\tilde x, \tilde y)$, and goes up (as $\arcsin$ above) when $\tilde y&amp;lt;\frac\pi2 \tilde x$, and goes down (as $\pi-\arcsin$ above) when $\tilde y&amp;gt;\frac\pi2\tilde x$. That's why the plot keeps crossing the singular line over and over again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way the plot tracks the line $y=\frac\pi2 x$ consists of individual arcs that are ODE solutions themselves. The locations of those arcs are random because they depend on the numerical errors in the ODE solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There might also be a separate issue with how it is plotted, because the plot sometimes doesn't cross the red line as expected. It could be that the plotting function isn't sampling the solution often enough, but I can't tell from just the plot. On the other hand it could be that the solution instead crosses the line in some intermediate solver steps only.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-12-08T22:59:24.053" LastActivityDate="2015-12-08T22:59:24.053" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21522" PostTypeId="2" ParentId="21515" CreationDate="2015-12-08T20:37:04.853" Score="3" Body="&lt;p&gt;My typical approach to this is to produce output data files that contain data at the required intervals and to visualize them in a batch way with a separate tool (Paraview, VisIt, Tecplot, MATLAB, etc.) at my leisure. With a fast filesystem and good parallel I/O package (HDF5, MPI I/O, ADIOS, etc.), this can often be done without disturbing the performance of your solution process too much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have evidence that producing the picture within your program is currently a bottleneck? I.e., do you have timings or profiling data that shows this? It's not usual that producing output is a bottleneck, but it can be if you are producing it too frequently.  &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2015-12-08T20:37:04.853" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21525" PostTypeId="2" ParentId="20288" CreationDate="2015-12-09T05:39:58.650" Score="3" Body="&lt;p&gt;When you discretize a PDE and your discretization blows up, the correct response is not &quot;let's ignore that result and try a different discretization&quot;.  That's like trying to fix your car by randomly replacing parts.  Instead, you should stop and investigate until you understand what is wrong, then correct that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see three possibilities here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Your code has a bug.&lt;/li&gt;&#xA;&lt;li&gt;Your problem is not well-posed, so the numerical solution is &lt;strong&gt;correctly&lt;/strong&gt; blowing up.&lt;/li&gt;&#xA;&lt;li&gt;The problem is well-posed and your numerical method is correctly implemented, but the method is unstable.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There is not enough information in your question to rule out any of these three possibilities.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-12-09T05:39:58.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21529" PostTypeId="1" AcceptedAnswerId="21530" CreationDate="2015-12-09T14:26:03.663" Score="3" ViewCount="70" Body="&lt;p&gt;I have:&lt;br&gt;&#xA;1) The dark image with few groups of high-brightness pixels and some amount of noise around it.&lt;br&gt;&#xA;2) Number of clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;br&gt;&#xA;1)&lt;br&gt; &lt;a href=&quot;https://i.stack.imgur.com/LIjaQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LIjaQ.png&quot; alt=&quot;Image for clustering&quot;&gt;&lt;/a&gt;&#xA;&lt;br&gt;&#xA;2) 2 clusters&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And i need find centers of bright pixel groups.&lt;br&gt;&#xA;Centers, in that case, should be placed like this:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1R0Qm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1R0Qm.png&quot; alt=&quot;Cluster centers on image&quot;&gt;&lt;/a&gt;&#xA;&lt;br&gt;&#xA;&lt;br&gt;&#xA;&lt;strong&gt;Question:&lt;/strong&gt; Which clustering alghoritm is suitable for this task?&lt;/p&gt;&#xA;" OwnerUserId="18496" LastActivityDate="2015-12-09T15:19:50.567" Title="Clustering pixel clots" Tags="|clustering|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21530" PostTypeId="2" ParentId="21529" CreationDate="2015-12-09T15:19:50.567" Score="2" Body="&lt;p&gt;&lt;strong&gt;If the number of clusters is known&lt;/strong&gt; (like here)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may use Lloyd's clustering [1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is as follows:&#xA;it optimizes a set of cluster centers $p_i$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Initialize the p_i's with an initial guess, or randomly&#xA;&#xA;For each iteration:&#xA;   Compute the cluster associated with each p_i,&#xA;      (the cluster is the set of points nearer to p_i than to the other p_j's)&#xA;   Move each p_i to the weighted centroid of its cluster&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For an image, the iteration can be implented as follows, computing the mass m_i and the centroid g_i of each cluster:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;For each i&#xA;   m_i = 0&#xA;   g_i = (0,0)&#xA;For each pixel (x,y) of the image&#xA;   let i denote the index of the center p_i nearest to (x,y)&#xA;   m_i = m_i + pixel_intensity(x,y)&#xA;   g_i = g_i + pixel_intensity(x,y) * (x,y)&#xA;For each i&#xA;   p_i = (1/m_i)*g_i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since the number of clusters is small, you can find the nearest p_i using a simple loop. If you have a higher number of sites, you may either use a kd-tree, or compute the Voronoi diagram of the sites and iterate on the pixels of each Voronoi cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used this algorithm to cluster the colors of a rubics cube acquired by a lego color sensor, and it works reasonably well while being very easy to implement [3]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If the number of clusters is unknown&lt;/strong&gt;&#xA;then the problem is much more difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may use &quot;mean shift clustering&quot; [2], that will apply a filter-like operation to the image, and make the &quot;modes&quot; appear. It acts like the inverse of a smoothing filter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://en.wikipedia.org/wiki/Lloyd%27s_algorithm&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Lloyd%27s_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_shift&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Mean_shift&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://alice.loria.fr/WIKI/index.php/Graphite/Lego&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/WIKI/index.php/Graphite/Lego&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2015-12-09T15:19:50.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21531" PostTypeId="1" AcceptedAnswerId="21533" CreationDate="2015-12-09T17:11:57.113" Score="0" ViewCount="1722" Body="&lt;p&gt;I have a matrix P of m rows and n columns. P(i,j) contains the price of occupying position i at time j. I want to plot this in Matlab but in a way to get some kind of density plot with different price levels corresponding to different colors.&#xA;An example of the output would be something like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Qo0VP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Qo0VP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10189" LastEditorUserId="10189" LastEditDate="2015-12-09T17:19:35.273" LastActivityDate="2015-12-09T18:38:31.417" Title="How to plot a price level matrix in Matlab?" Tags="|matlab|visualization|plotting|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21532" PostTypeId="2" ParentId="21531" CreationDate="2015-12-09T17:12:55.793" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/contourf.html&quot; rel=&quot;nofollow&quot;&gt;A filled 2-d contour plot&lt;/a&gt; seems to be what you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you have vectors (or matrices) &quot;position&quot; and &quot;time&quot;, the following snippet should work (modulo an apostrophe or two)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contourf(position,time,P);&#xA;colorbar;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17568" LastActivityDate="2015-12-09T17:12:55.793" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21533" PostTypeId="2" ParentId="21531" CreationDate="2015-12-09T17:36:20.817" Score="2" Body="&lt;p&gt;I just found the in-built function I was looking for: &quot;imagesc&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;&amp;gt; imagesc(eye(4)); colorbar&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gives this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pT9ln.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pT9ln.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10189" LastActivityDate="2015-12-09T17:36:20.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21534" PostTypeId="2" ParentId="21531" CreationDate="2015-12-09T18:38:31.417" Score="0" Body="&lt;p&gt;What about just using surface plot? Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;surf(P); view(0,90);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or for more options something like: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;surf(P,'EdgeColor','none','facecolor','interp'); view(0,90); colormap('hot')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2015-12-09T18:38:31.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21535" PostTypeId="2" ParentId="21116" CreationDate="2015-12-09T19:35:19.817" Score="1" Body="&lt;p&gt;I would advise against using MATLAB - you will be spending too much time reinventing the wheel. OpenFOAM has a VoF implementation readily available (interFoam), and you can find hints on adding solidification phase change code &lt;a href=&quot;http://www.cfd-online.com/Forums/openfoam-solving/93620-melting-problem-looking-appropriate-solvers-11.html#post545556&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. There are other, more sophisticated, open source codes such as &lt;a href=&quot;http://sourceforge.net/projects/gfs/&quot; rel=&quot;nofollow&quot;&gt;Gerris&lt;/a&gt; and &lt;a href=&quot;http://www.elmerfem.org/&quot; rel=&quot;nofollow&quot;&gt;Paris&lt;/a&gt;, but you may find it more difficult to add phase change. If you're into finite elements, also have a look at &lt;a href=&quot;http://www.elmerfem.org/&quot; rel=&quot;nofollow&quot;&gt;Elmer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2015-12-09T19:35:19.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21537" PostTypeId="1" CreationDate="2015-12-10T11:49:04.507" Score="0" ViewCount="208" Body="&lt;p&gt;is there a way to add a user defined convergence criteria to an ode solver so that the solution is stopped?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that Matlab uses absolute and relative tolerances but would that suffice in solving CFD equations or is a user-defined criteria needed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="17476" LastActivityDate="2018-05-18T23:51:07.527" Title="maltab ode solver- user defined criteria to stop calculations" Tags="|matlab|fluid-dynamics|ode|convergence|numerical-modelling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21538" PostTypeId="1" AcceptedAnswerId="21539" CreationDate="2015-12-10T13:57:15.243" Score="8" ViewCount="1743" Body="&lt;p&gt;I'm looking for an implementation of the ellipsoid algorithm for linear programming since the application I have in mind has the constraints represented as a separation oracle. Is such an implementation available anywhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there isn't, is there maybe an implementation of the interior point method that works with a separation oracle? (I'm not sure IPM can work with a separation oracle, so it's possible that this part of the question is nonsense.)&lt;/p&gt;&#xA;" OwnerUserId="18503" LastActivityDate="2015-12-10T20:16:40.810" Title="Implementation of LP with separation oracle?" Tags="|linear-programming|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21539" PostTypeId="2" ParentId="21538" CreationDate="2015-12-10T16:42:25.940" Score="11" Body="&lt;p&gt;Here's a longer answer that summarizes my earlier comments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of any implementations of the ellipsoid algorithm that are practically usable for solving LP's. It's easy enough to cook up an implementation of the ellipsoid method in MATLAB but if you try to solve relatively small test problems (e.g. the afiro test problem from NETLIB which has about 50 constraints and 100 variables) you'll quickly discover the double precision arithmetic isn't up to the task.  The problem is that the matrices involved become very ill-conditioned.  If you want to play with this, I'd recommend starting with the discussion in Chavatal's Linear Programming textbook (there's an appendix on the ellipsoid method.)    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interior point methods for LP assume that the constraints are given and don't work with a separation oracle, so that's out of the question. However, you might be interested in interior point cutting plane methods that use similar ideas to effectively construct an LP outer relaxation of the feasible set as they go along. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a nice set of lecture notes on cutting plane approaches by Boyd and Vandenberghe at &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://web.stanford.edu/class/ee392o/localization-methods.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://web.stanford.edu/class/ee392o/localization-methods.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analytic center cutting plane method (ACCPM) of Goffin and Vial is perhaps the best known method of this type.  See the code and references at &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.maths.ed.ac.uk/~gondzio/software/accpm.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.maths.ed.ac.uk/~gondzio/software/accpm.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I'd recommend ACCPM, your problem could also be addressed by a variety of other approaches for general convex optimization problems such as subgradient descent, bundle methods, etc.   My expertise is more in interior point methods for LP and conic optimization problems so you might get a better and more up to date answer from someone who works in this area.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-12-10T20:16:40.810" LastActivityDate="2015-12-10T20:16:40.810" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21540" PostTypeId="2" ParentId="14384" CreationDate="2015-12-10T17:38:30.683" Score="1" Body="&lt;p&gt;I have an hybrid approach using dinamic programming and steepest descent:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start with the straight line from home to school, with constant speed, and calculate energy.&lt;/li&gt;&#xA;&lt;li&gt;Place a point halfway and minimize the energy varying its position, the starting speed and the speed at the point.&lt;/li&gt;&#xA;&lt;li&gt;Recursively subdivide your subpaths, minimizing as per point 2.&lt;/li&gt;&#xA;&lt;li&gt;Exiting from the recursion you can minimize again moving the mid point and varying speed at the end point.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I don't think this will give you the global optimum, but could give a good local minimum (just like an A* algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously the function describing the terrain is assumed to be smooth.&lt;/p&gt;&#xA;" OwnerUserId="18255" LastActivityDate="2015-12-10T17:38:30.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21541" PostTypeId="1" CreationDate="2015-12-10T20:19:24.390" Score="1" ViewCount="45" Body="&lt;p&gt;Assume I have an expression for an matrix initialization, for example the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A[i,i-2*j+k] = B[i-k] * C[i] * D[i+j+k] &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to execute such a process, I could loop over all i, j and k. The problem is that loops are slow in python, compared to e.g. loops in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some way to use compiled packages for operations such as the one above? This should improve performance a lot I would guess.&lt;/p&gt;&#xA;" OwnerUserId="8329" LastActivityDate="2015-12-11T11:02:24.837" Title="How to use compiled python packages for matrix initialization" Tags="|python|matrix|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21542" PostTypeId="2" ParentId="21482" CreationDate="2015-12-10T22:40:06.817" Score="2" Body="&lt;p&gt;I ended up with a simple AMR implementation that does the job for polyhedral cells&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a)  No coarsening. The original coarse grid is always loaded first on which refinement is applied. This gets rid of octtree and such that are needed to keep history of AMR. Also, since the internal data structures for the solver are static (vectors/arrays), it is not possible to refine one cell only efficiently, so re-doing the AMR from scratch is inevitable anyway&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Polyhedral cells are refined into tetrahedrons/pyramids while hexahedrons are split into hexes as I mentioned before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives me a nice AMR simulation but it is only &quot;local&quot; (each processor does its own refinement). It would be nice to have a parallel AMR library for arbitrary polyhedrons.&lt;/p&gt;&#xA;" OwnerUserId="18132" LastActivityDate="2015-12-10T22:40:06.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21544" PostTypeId="1" CreationDate="2015-12-11T05:59:22.600" Score="2" ViewCount="1818" Body="&lt;p&gt;I have been experimenting in building a C++ project for a FDTDS system for Electro-Magnetics.  I have implemented a class [see below] which I called mesh using the Armadillo library. The  3D matrices {Cubes} for the fields Ex, Ey, Ez, Hx, Hy, Hz are declared as private members of the class. The size of these 3D Matrices are determined by the constructor.  During the nuts and bolts part of the program these fields are updated by public functions [see below]    void updateE(void) and void updateH(void). The program works well but is extremely slow.  I have experimented with a timer and have found that the access to individual elements of the 3D matrices  is of the order 2e-5.&#xA;In a similar project I built in the C language but using macros and dynamic memory allocation and pointers I can get accessing times of the order of one thousand sec quicker.  I must be doing something wrong. I am new to both Armadillo and C++.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;armadillo&amp;gt;&#xA;&#xA;using namespace arma;&#xA;&#xA;class mesh&#xA;{&#xA;  private:&#xA;&#xA;    //size of mesh&#xA;    int SizeX;&#xA;    int SizeY;&#xA;    int SizeZ;&#xA;&#xA;   /* declaration of Matrices/Cubes */&#xA;   cube  Hx;&#xA;   ....&#xA;    ....&#xA;&#xA;&#xA;public:&#xA;   mesh(int sizex, int sizey, int sizez); //construct mesh and init values&#xA;   void updateE(void);         // Update Electric field&#xA;   void updateH(void);         // Update Magnetic field&#xA;   .......&#xA;&#xA;&#xA;};&#xA;&#xA;&#xA;mesh::mesh( int sizex, int sizey, int sizez)&#xA;{&#xA;    // allocate size&#xA;    SizeX= sizex;&#xA;    SizeY= sizey;&#xA;    SizeZ= sizez;&#xA;&#xA;    /* memory allocation and zeroed*/&#xA;&#xA;    Hx      = zeros&amp;lt;cube&amp;gt;( SizeX, SizeY - 1, SizeZ - 1);&#xA;&#xA;    ........&#xA;&#xA;&#xA;&#xA;&#xA;}&#xA;&#xA;&#xA;void mesh::updateH(void) {&#xA;  int mm, nn, pp;&#xA;&#xA;   for (mm = 0; mm &amp;lt; SizeX; mm++)&#xA;      for (nn = 0; nn &amp;lt; SizeY - 1; nn++)&#xA;        for (pp = 0; pp &amp;lt; SizeZ - 1; pp++)&#xA;            Hx(mm, nn, pp) = Chxh(mm, nn, pp) * Hx(mm, nn, pp) +&#xA;            Chxey(mm, nn, pp) * (Ey(mm, nn, pp + 1) - Ey(mm, nn, pp) ) -&#xA;            Chxez(mm, nn, pp) *(Ez(mm, nn + 1, pp) - Ez(mm, nn, pp));&#xA;&#xA;   for (mm = 0; mm &amp;lt; SizeX - 1; mm++)&#xA;      for (nn = 0; nn &amp;lt; SizeY; nn++)&#xA;        for (pp = 0; pp &amp;lt; SizeZ - 1; pp++)&#xA;            Hy(mm, nn, pp) = Chyh(mm, nn, pp) * Hy(mm, nn, pp) +&#xA;            Chyez(mm, nn, pp) * (Ez(mm + 1, nn, pp) - Ez(mm, nn, pp)) -&#xA;            Chyex(mm, nn, pp) * (Ex(mm, nn, pp + 1) - Ex(mm, nn, pp));&#xA;&#xA;   for (mm = 0; mm &amp;lt; SizeX - 1; mm++)&#xA;      for (nn = 0; nn &amp;lt; SizeY - 1; nn++)&#xA;        for (pp = 0; pp &amp;lt; SizeZ; pp++)&#xA;            Hz(mm, nn, pp) = Chzh(mm, nn, pp) * Hz(mm, nn, pp) +&#xA;            Chzex(mm, nn, pp) * (Ex(mm, nn + 1, pp) - Ex(mm, nn, pp)) -&#xA;            Chzey(mm, nn, pp) * (Ey(mm + 1, nn, pp) - Ey(mm, nn, pp));&#xA;&#xA;&#xA; return;&#xA; } /* &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18515" LastEditorUserId="18515" LastEditDate="2015-12-11T06:14:17.617" LastActivityDate="2015-12-15T08:31:27.013" Title="Armadillo library appears slow" Tags="|matrix|c++|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21545" PostTypeId="2" ParentId="21541" CreationDate="2015-12-11T11:02:24.837" Score="1" Body="&lt;p&gt;Considering your last comment take a look at &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;numpy.einsum&lt;/code&gt;&lt;/a&gt;; I used it some time ago to do a complicated multidimensional matrix multiplication. It takes a bit to get used to but if you follow the examples you should be able to work it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This module is implemented in C much like the rest of Numpy so you should get decent performance.&lt;/p&gt;&#xA;" OwnerUserId="18017" LastActivityDate="2015-12-11T11:02:24.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21546" PostTypeId="1" CreationDate="2015-12-11T15:17:20.270" Score="3" ViewCount="431" Body="&lt;p&gt;I have the following system of PDEs for which I have given parameters $\gamma, \tau$ and $\mu$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;T_t = &amp;amp;\ \gamma\,(L +\tau F-T)\\&#xA;F_t = &amp;amp; -F_x-(F-LT)\\&#xA;L_t = &amp;amp;\ \mu L_{xx}+(F-LT)\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with no-flux boundary conditions at $x=0$ and $x=\infty$ for $F$ and $L$. The initial conditions are $L=T=F=e^{-x^2}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica failed to produce results because the order of the derivative in the boundary condition for $F$, $(F_x(0)=F_x(\infty)=0)$ is the same as the order of the derivative of $F$ w.r.t $x$ in the PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any alternatives to Mathematica?&lt;/p&gt;&#xA;" OwnerUserId="18521" LastEditorUserId="123" LastEditDate="2016-01-13T13:52:02.190" LastActivityDate="2016-01-13T13:52:02.190" Title="Solving coupled PDEs numerically on a semi-infinite domain with no-flux boundary conditions" Tags="|pde|mathematica|coupling|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21547" PostTypeId="1" CreationDate="2015-12-11T16:41:20.963" Score="3" ViewCount="1302" Body="&lt;p&gt;I have a python code that calculates the solution of the inhomogeneous acoustic wave equation for a 2D medium with any velocity and source configuration. It was implemented using Finite Differences from the following set of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\rho( \mathbf{r}) \mathbf{\nabla} \cdot \left( \frac{1}{\rho (\mathbf{r})}  \mathbf{\nabla} p(\mathbf{r},t) \right) - \frac{1}{c^2(\mathbf{r})} \frac{\partial^2 p(\mathbf{r},t)}{\partial^2 t} = -s(\mathbf{r},t) &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;s(\mathbf{r},t) = \rho( \mathbf{r}) \frac{\partial^2 i_v(\mathbf{r},t)}{\partial^2 t}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and with the propagation velocity &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;c(\mathbf{r})  = \sqrt{\frac{\kappa(\mathbf{r})}{\rho(\mathbf{r})}} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathbf{\kappa}$ is the adiabatic compression modulus of the medium, $i_v$ is the source and $\rho$ is the density of the medium.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FD schema uses 2nd order discretization in time and 4th in space and is implemented for 2D space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is how do I calculate from my numerical simulation the dispersion relation of my code? In fact i want to calculate the phase velocity $v(k) = \frac{\omega(k)}{k}$ dispersion.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to do that to compare with the expected relation from literature. I expect it varying with different grid angles, source frequencies and stability parameters. I know how to input those in my simulation but I dont know how to use the results (2D time panels) to establish the dispersion relation.&lt;/p&gt;&#xA;" OwnerUserId="4092" LastActivityDate="2015-12-16T18:49:36.543" Title="How to calculate dispersion relation from a Finite Difference (FD) wave simulation" Tags="|finite-difference|numerical-analysis|wave-propagation|numerical-limitations|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21549" PostTypeId="2" ParentId="21546" CreationDate="2015-12-11T17:05:27.320" Score="2" Body="&lt;p&gt;Here is something I threw together, it is definitely incorrect, as the program is throwing errors. In particular, a &lt;code&gt;SolutionVariableNumberError&lt;/code&gt; is raised. Likely this is because the equations are written incorrectly. The likely suspect, I'm guessing is in the &lt;code&gt;SourceTerm&lt;/code&gt;'s. This is a start though for how one might go about solving these equations in FiPy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps someone in the forum has more experience with this tool and can elaborate on what is going wrong here. Other than that if you want to continue with this tool, I suggest reading and studying more of FiPy's &lt;a href=&quot;http://www.ctcms.nist.gov/fipy/examples/README.html&quot; rel=&quot;nofollow&quot;&gt;examples&lt;/a&gt;. You could also post to the their mailing list, and ask for some help writing the equations. I've seen others do this on the list, as I have been following it for a while. The project maintainers are quite helpful. (I have no affiliation to the project, I'm just a fan).   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from fipy import CellVariable&#xA;from fipy import DiffusionTerm&#xA;from fipy import Grid1D&#xA;from fipy import ImplicitSourceTerm&#xA;from fipy import numerix&#xA;from fipy import PowerLawConvectionTerm&#xA;from fipy import TransientTerm&#xA;&#xA;mu = 1.&#xA;gam = 1.&#xA;tau = 1.&#xA;&#xA;m = Grid1D(nx=100,Lx=1.)&#xA;&#xA;L = CellVariable(mesh=m, hasOld=True)&#xA;F = CellVariable(mesh=m, hasOld=True)&#xA;T = CellVariable(mesh=m, hasOld=True)&#xA;&#xA;x = m.cellCenters&#xA;&#xA;L.value = numerix.exp(-x**2)&#xA;F.value = numerix.exp(-x**2)&#xA;T.value = numerix.exp(-x**2)&#xA;&#xA;eqn0 = TransientTerm(var=L) == DiffusionTerm(mu, var=L) + (ImplicitSourceTerm(var=F) - ImplicitSourceTerm(var=L*T))&#xA;eqn1 = TransientTerm(var=F) == PowerLawConvectionTerm(-1., F) - (ImplicitSourceTerm(var=F) - ImplicitSourceTerm(var=L*T))&#xA;eqn2 = TransientTerm(var=T) == gam * (ImplicitSourceTerm(1., var=L) + ImplicitSourceTerm(tau, var=F) + ImplicitSourceTerm(-1.,var=T))&#xA;&#xA;eqn = eqn0 &amp;amp; eqn1 &amp;amp; eqn2&#xA;&#xA;for t in range(100):&#xA;    L.updateOld()&#xA;    F.updateOld()&#xA;    T.updateOld()&#xA;    eqn.solve(dt=1.e-3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10225" LastActivityDate="2015-12-11T17:05:27.320" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21550" PostTypeId="1" CreationDate="2015-12-11T21:44:26.183" Score="0" ViewCount="57" Body="&lt;p&gt;Does the order of accuracy of a combination of schemes applied to solve a system of non-linear equations, match those of the same schemes applied to the linearised version of the system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, does linearisation alter the order of accuracy of a scheme?&lt;/p&gt;&#xA;" OwnerUserId="17476" LastEditorUserId="9667" LastEditDate="2015-12-11T23:33:36.990" LastActivityDate="2015-12-11T23:33:36.990" Title="Order of accuracy of linearised vs non-linear system" Tags="|numerical-analysis|nonlinear-equations|discretization|accuracy|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21552" PostTypeId="1" CreationDate="2015-12-12T14:04:23.837" Score="2" ViewCount="47" Body="&lt;p&gt;I have written a sparse quadrature routine to integrate multidimensional Gaussian integrals. I'm trying to show convergence plots in my thesis and demonstrate that the code works correctly. I am in the process of benchmarking this using multidimensional integrals with closed forms. With normal numerical schemes one would assume the relative error to decreases continuously a with higher number of quadrature points. However the smolyak algorithm selects only those tensor products obeying the following criteria:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ell-M+1\leq||i||_1\leq\ell$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $M$ is the number of dimensions in the integral, $\ell$ is the order  and $\|i\|_1$ the summation of the number of quadrature points used for each of the $M$ one-dimensional sets. This results in a sparse tensor product. As we can see when large $\ell$ is used the lower order tensor products are dismissed, particularly when $\ell&amp;gt;&amp;gt;M$. Hence would one still expect the error to decrease continuously with a higher order $\ell$? Within my code I am seeing that for higher $\ell$, the error plateaus, as $M$ is increased this plateaus moves such that the accuracy is decreased. For example when $M$ is 1, the error plateaus at 1E-16 i.e. double precision limit as expected. But when $M=5$ the plateau is at around 1E-12, this seem quite large and I've tested this for several different multidimensional integrals. Is this correct? For $\ell=35$ we have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$31\leq||i||_1\leq35$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we are losing a lot of tensor products. Am I being to pedantic? I will only every require the first four significant figures due to input data. &lt;/p&gt;&#xA;" OwnerUserId="18531" LastEditorUserId="18531" LastEditDate="2015-12-13T09:36:32.363" LastActivityDate="2015-12-13T09:36:32.363" Title="Not sure if sparse quadrature routine is working correctly?" Tags="|quadrature|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21554" PostTypeId="1" AcceptedAnswerId="21556" CreationDate="2015-12-13T04:51:05.833" Score="1" ViewCount="198" Body="&lt;p&gt;I want to frame a higher order &lt;a href=&quot;http://www.holoborodko.com/pavel/numerical-methods/numerical-derivative/central-differences/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Central difference&lt;/a&gt; scheme of about $20^{th}$ order for first derivative. I'm using $20^{th}$ order because I need one scheme with good modified wave number. To find the co-efficient matrix (eg. for CD-2 it is [-1/2,1/2]), I have to solve $Ax=b$ equation. $x=A^{-1}*b$. I tried to solve that equation using &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Build-in inverse (inv(A)) command of matlab, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;solve&quot; command in R, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gauss Seidel&lt;/a&gt; algorithm, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Jacobi_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gauss Jacobi&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conjugate gradient method&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gauss Seidel and Jacobi may fail because of the matrix is not diagonally dominant. Matlab gave answer with a warning and that answer is wrong. R- gave an error message because of poor conditional number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrix $A$ is $20*20$ matrix :&lt;a href=&quot;https://i.stack.imgur.com/k4JSY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k4JSY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrix $b$ is $20*1$ matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OchfX.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OchfX.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any sophisticated algorithm or build-in command of any programming language to solve this? Please help me out.&lt;/p&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="16930" LastEditDate="2015-12-13T04:59:50.590" LastActivityDate="2015-12-13T09:28:50.907" Title="Inverse of &quot;diagonally not dominant matrix&quot;" Tags="|numerical-analysis|matrix|inverse|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21555" PostTypeId="2" ParentId="21554" CreationDate="2015-12-13T05:58:25.323" Score="1" Body="&lt;p&gt;I would not generally expect a &quot;20th order&quot; derivative estimate to typically be very stable/reliable/useful (e.g. due to &lt;a href=&quot;https://en.wikipedia.org/wiki/Runge&amp;#39;s_phenomenon&quot; rel=&quot;nofollow&quot;&gt;well known artifacts&lt;/a&gt; of high-order polynomial interpolation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, a general procedure for deriving finite-difference stencils is to solve an appropriate polynomial interpolation problem. I will use Matlab-style notation, as you mention that program. In 1D, if you have a &quot;stencil-grid&quot; $x=-m:m$, then you can fit an interpolating polynomial $p(x)$ of degree $2m$ to any set of function values $f(x)$. The derivatives $\partial_x^kp$ evaluated at $x=0$ are then central-difference estimates of the corresponding derivatives of $f(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a particular set of function values this would be done via $polyfit$ and $polyder$ in Matlab. To solve for the &lt;em&gt;stencil coefficients&lt;/em&gt; themselves, this is indeed one of the few times I have found a need to actually compute a matrix inverse. The procedure is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) assemble the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vandermonde_matrix&quot; rel=&quot;nofollow&quot;&gt;Vandermonde matrix&lt;/a&gt; $A$ for the stencil $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Invert this matrix via $C=A\backslash I$, where $I$ is the identity matrix (i.e. solve a set of $2m+1$ interpolation problems, where the columns of $I$ are your &quot;$b$&quot; vectors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) The rows of $C$ now give &quot;filters&quot; for the $2m+1$ coefficients of interpolating the polynomial (i.e. the coefficient vector would be $c=Cf$ for given data $f$). So, the second row of $C$ gives the coefficients of your 1st derivative stencil (divide this by $\Delta x$ for a grid with non-unit spacing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that to get the stencils for the first $2m$ derivatives, you would just compute $C(k+1,:)/(k!\Delta x^{k-1})$ for the $k$th derivative. Similarly, $x=-m:m$ assumes central differences on a uniform grid, but an arbitrary $x$ can be used, e.g. to compute off-center derivatives on a non-uniform grid. And you can get &lt;a href=&quot;https://en.wikipedia.org/wiki/Savitzky%E2%80%93Golay_filter&quot; rel=&quot;nofollow&quot;&gt;moving-least-squares stencils&lt;/a&gt; by having a rectangular $A$ matrix (note that the Matlab &quot;$\backslash$&quot; command automatically solves the normal equations in this case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="17742" LastActivityDate="2015-12-13T05:58:25.323" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21556" PostTypeId="2" ParentId="21554" CreationDate="2015-12-13T06:21:21.150" Score="2" Body="&lt;p&gt;One easy way to derive finite-difference approximations to derivatives is as follows. To find the coefficients $c_k$ corresponding to order-$s$ derivative on points $uh,(u+1)h,\ldots,vh$ ($u=-10,v=10,s=1$ in your case), write the condition defining, using Taylor theorem, as&#xA;$$ \sum_{j=u}^{v}c_k e^{j h \partial} = \partial^s, $$&#xA;which should hold asymptotically as $h\to0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introducing a change of variable $z=e^{h\partial}$, this is equivalent to&#xA;$$z^u p(z) = h^{-s}(\log z)^s, $$&#xA;where $p$ is the polynomial $p(z) = c_u + c_{u+1}z + \cdots + c_v z^{v-u}$. Thus&#xA;$$ p(z) = h^{-s}\frac{(\log z)^s}{z^u}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So just compute the approximating Taylor polynomial around $z=1$ for $z^{-u}(\log z)^s$ to order $v-u$, this will be the polynomial with the finite-difference coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $u=-10$, $v=10$, $s=1$, this polynomial is easily computed.&#xA;The coefficients are:&#xA;$$ h^{-1}\left\{\frac{1}{1847560},-\frac{5}{415701},\frac{5}{38896},-\frac{15}{17017},\frac{5}{1144},-\frac{12}{715},\frac{15}{286},-\frac{20}{143},\frac{15}{44},-\frac{10}{11},0,\frac{10}{11},-\frac{15}{44},\frac{20}{143},-\frac{15}{286},\frac{12}{715},-\frac{5}{1144},\frac{15}{17017},-\frac{5}{38896},\frac{5}{415701},-\frac{1}{1847560}\right\}. $$&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-12-13T09:28:50.907" LastActivityDate="2015-12-13T09:28:50.907" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21557" PostTypeId="1" AcceptedAnswerId="21559" CreationDate="2015-12-13T09:51:58.690" Score="4" ViewCount="113" Body="&lt;p&gt;I have two data sets that are quasi-periodic. They have the same period and can be seen clearly by eye. For example when &lt;span class=&quot;math-container&quot;&gt;$x\in(100,200)$&lt;/span&gt;, both of them have about 32 periods.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0PeY8.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0PeY8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Below is a zoom-in of the data:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NAFq9.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NAFq9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can see that the two data sets have quite different nature.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question is, how can I count the exact number of periods(as what we count by eye) by computer? It is best that the algorithm/program can deal with these two different behaved data sets simultaneously.&lt;/strong&gt; I'm using python or c most. If available, code in these two language is preferred.&lt;/p&gt;&#xA;&lt;p&gt;The data is &lt;a href=&quot;https://drive.google.com/file/d/0BzZ3xLk8sH4CN1Nxa0FyWjRsSGM/view?usp=sharing&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. It is about 15mb.&lt;/p&gt;&#xA;" OwnerUserId="16652" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-12-14T08:57:29.870" Title="How to read the number of periods of this complicated graph?" Tags="|algorithms|python|numpy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21559" PostTypeId="2" ParentId="21557" CreationDate="2015-12-13T14:46:53.867" Score="1" Body="&lt;p&gt;You should use a Fourier transform to compute the power spectrum of your signal. Peaks correspond to major frequency components. For an exmaple, see the single sided amplitude spectrum plots here: &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/fft.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.com/help/matlab/ref/fft.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;br&gt;&#xA;It seems I misread your question. If you just want the number of periods that occur in the data set, the relevant frequency is clearly visible in the power spectrum. You are probably interested in the peak corresponding to the lowest frequency present (there are also many integer multiples of this frequency present which may even have larger amplitudes). You can pull out that frequency using some threshold to detect it and multiply by the total duration of the data to get the number of periods in the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the overall frequency for the data to exactly repeat itself is equal to the greatest common factor of all frequencies present. Please see the frequencies obtained below, the lowest frequency peak is what you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZxJ5Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZxJ5Q.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nCu7f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nCu7f.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="16652" LastEditDate="2015-12-14T08:57:29.870" LastActivityDate="2015-12-14T08:57:29.870" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21561" PostTypeId="1" AcceptedAnswerId="21816" CreationDate="2015-12-13T15:28:21.837" Score="9" ViewCount="1685" Body="&lt;p&gt;If I have a physical system which contains a time reversal symmetry (for example a Hamiltonian $H(x,p)=p^2/2m + V(x)$ with $V(x)$ real) and I want to solve the differential equations which describe this system, which solver for ODEs should I use in order to keep the time reversal symmetry (for example in mathematica)? Which solvers break this symmetry?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I want to extend this question. Let us consider a system of coupled first order differential equations $$\dot{a}_1 (t) = f_1(a_1,a_2,a_3,\ldots,a_n;t) \\ &#xA;\dot{a}_2(t) = f_2(a_1,a_2,a_3,\ldots,a_n;t) \\ &#xA;\dot{a}_3(t) = f_3(a_1,a_2,a_3,\ldots,a_n;t) \\ \vdots$$&#xA;What integration method is best used if the underlying system contains a time reversal symmetry?&lt;/p&gt;&#xA;" OwnerUserId="18543" OwnerDisplayName="Merlin1896" LastEditorUserId="123" LastEditDate="2016-01-14T11:33:32.447" LastActivityDate="2016-01-15T07:16:50.497" Title="Which numerical methods preserve time reversal symmetry?" Tags="|ode|time-integration|symmetry|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21562" PostTypeId="1" AcceptedAnswerId="21585" CreationDate="2015-12-13T17:44:58.747" Score="5" ViewCount="3055" Body="&lt;p&gt;I cannot find a reference explaining clearly and rigorously the links between the notions of conservative form for a PDE, a conservative numerical method and discrete conservation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be very grateful for a complete explanation. &lt;/p&gt;&#xA;" OwnerUserId="18545" LastEditorUserId="18545" LastEditDate="2015-12-13T17:54:00.583" LastActivityDate="2015-12-16T12:44:25.267" Title="PDE - Conservative form, conservative methods and discrete conservation" Tags="|pde|discretization|conservation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21563" PostTypeId="1" AcceptedAnswerId="21567" CreationDate="2015-12-13T20:17:31.467" Score="0" ViewCount="830" Body="&lt;p&gt;We have the advection equation $u_t+a u_x=0, a&amp;gt;0, 0&amp;lt;t&amp;lt;T_f, x \in \mathbb{R}$ with initial condition $u(0,x)=u_0(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that we have the following sheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EqJYH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EqJYH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find the CFL condition using the domain of dependence of the finite difference method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate $U_j^{n+1}$ we need the values $U_{j-1}^n$ and $U_{j+1}^n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we get the following domain of dependence: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XZbuV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XZbuV.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The slope of the left line is $\frac{\tau}{h}$ and the slope of the right line is $-\frac{\tau}{h}$, right? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the CFL condition do we require that the left slope is smaller than the right slope? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or have I understood it wrong?&lt;/p&gt;&#xA;" OwnerUserId="9422" LastEditorUserId="9422" LastEditDate="2015-12-13T22:11:15.867" LastActivityDate="2015-12-14T08:56:41.153" Title="Find cfl condition" Tags="|advection|cfl|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21564" PostTypeId="2" ParentId="21557" CreationDate="2015-12-13T20:39:26.027" Score="0" Body="&lt;p&gt;You seem to have different &quot;types&quot; of periods, and different types of maxima and minima. One idea to try is to just compute the set of local extrema. Call the sequence of extremal values $v_j$ at locations $x_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be able to distinguish the different types of maxima and minima by looking at differences in heights between, say, a minimum and the two neighbouring maximums, $|v_j-v_{j-1}|+|v_j-v_{j+1}|$, as well as looking at the size of the &quot;basin&quot; associated to a minimum, $x_{j+1}-x_{j-1}$. The difference in heights would be really high when the data jumps from top to bottom, and you would also get a much smaller difference for the &quot;minor&quot; maxima on the bottom and the top. This can also help get rid of noise if the differences are negligible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing the number of periods would be as simple as counting the number of maxima of the right type, or maybe computing the distance between them as $x_{j+2}-x_j$. This would also allow you to distinguish separately the high-amplitude and low-amplitude oscillations, if necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am reminded of the literature in scientific visualization on using Morse-Smale complexes to extract features from 3d scalar fields (e.g. &lt;a href=&quot;http://dx.doi.org/10.1109/TVCG.2007.70552&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1109/TVCG.2007.70552&lt;/a&gt;), but that's overkill here as the 1d case is really simple on its own.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-12-13T20:39:26.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21566" PostTypeId="2" ParentId="21336" CreationDate="2015-12-13T22:40:57.947" Score="1" Body="&lt;p&gt;If you have a single two-dimensional nonconvex constraint like this (assuming it's not sufficient to relax it to a convex constraint like&#xA;$&#xA;S^2 + C^2 \le 1&#xA;$)&#xA;then I'd recommend simply discretizing the 2d unit ball and solving an LP for each possible value of $(S,C)$.&lt;/p&gt;&#xA;" OwnerUserId="8142" LastEditorUserId="8142" LastEditDate="2015-12-14T02:33:29.357" LastActivityDate="2015-12-14T02:33:29.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21567" PostTypeId="2" ParentId="21563" CreationDate="2015-12-14T00:04:13.257" Score="4" Body="&lt;p&gt;Your image of the numerical domain of dependence is correct. But try to also draw the analytical domain of dependence, maybe this could help you to better understand what is going on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the analytic solution of $u_t+au_x=0$ is $u(t,x)=u_0(x-at)$. So the slope of the actual dependence is $a$. The CFL condition just says that the numerical dependence must include the actual dependence, which means $-\frac{h}{\tau}\leq a \leq \frac{h}{\tau}$ in your case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you compare the analytical solution with the numerical schemes, you will notice that most schemes just interpolate or extrapolate the solution at the previous time step at the required position. The CFL condition then says that you must actually do an interpolation, because an extrapolation would not be stable.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="534" LastEditDate="2015-12-14T08:56:41.153" LastActivityDate="2015-12-14T08:56:41.153" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21568" PostTypeId="1" CreationDate="2015-12-14T03:01:53.937" Score="6" ViewCount="604" Body="&lt;p&gt;I'm working on a voxel-based modelling application and one of the features that I've implemented is a method to do a 3D mesh reconstruction from a series of 2D image slices (similar to an MRI).  I've got a basic brain scan image set that I've been working on, but ultimately I'd like to release a small demo example showing how to do the mesh reconstruction.  However, in order to do this I need the images to be legally open source so that I'm not distributing images that have copy right restrictions.  I've scoured the net trying to find a suitable image set but haven't been able to find anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any resources they could point me to that I could use for my application?&lt;/p&gt;&#xA;" OwnerUserId="18550" LastEditorUserId="9667" LastEditDate="2015-12-14T04:30:32.217" LastActivityDate="2016-01-28T22:10:44.083" Title="Open Source MRI Image Stacks" Tags="|image-processing|data-sets|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21569" PostTypeId="1" CreationDate="2015-12-12T16:29:35.503" Score="8" ViewCount="580" Body="&lt;p&gt;I am implementing from scratch an Hartree-Fock calculation in the STO-3G basis set to perform Born-Oppenheimer molecular dynamics. I have a Restricted Hartree-Fock procedure that can reproduce very well the total energies of Ref. [1] and of Gaussian09 with pre-optimized molecular geometries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, during a Born-Oppenhaimer MD simulation, I will have geometries that can be quite far from equilibrium. I saw that my SCF cycle does not converge for stretched geometries: the SCF cycle for the $CO$ molecule with a bond length of 2.132 bohr converges, while for a bond length of 3.132 bohr I reach the maximal number of allowed iterations (500). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Table 10.4 of Ref. [2] it is clear that an implementation of the DIIS (Direct Inversion of the iterative Subspace) method speedup the SCF convergence. In addition to this speedup, it seems that the convergence for a stretched molecule can be easily reached with this method. For this reason I implemented the DIIS algorithm following Ref. [3].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe I implemented the DIIS method quite correctly, since I was able to obtain a speedup of SCF convergence for pre-optimized molecular geometries. In the following table you can find the number of simple SCF and SCF-DIIS steps I have, for different molecules:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Molecule    SCF     DIIS&#xA;H2O         16      9&#xA;CO          53      19&#xA;HeH+        8       7&#xA;CH4         11      8&#xA;FH          10      7&#xA;O2          41      18&#xA;N2          110     22&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In every molecule I tested I get a speedup, which is quite impressive where a lot of normal SCF cycles were needed. This is a good result, but unfortunately the convergence of stretched molecules is not improved. By doubling the distances (in bohr) of the pre-optimized structure, the DIIS algorithm (as well as the normal SCF) fails to converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;There is some step to add to the original DIIS algorithm of Ref. [3] that I am not considering? How I can make the SCF cycle convergent for stretched molecules, provided that these calculations converge in Gaussian09?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;EDIT&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gaussian09 brakes down as my program does with the option &lt;code&gt;SCF=NoDIIS&lt;/code&gt;. With the option &lt;code&gt;SCF=DIIS&lt;/code&gt; the SCF converges even for very distorted molecules.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[1] A. Szabo and N. Ostlund, &lt;em&gt;Modern Quantum Chemistry&lt;/em&gt;, Dover, 1996.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] T. Helgaker, P. Jørgensen and J. Olsen, &lt;em&gt;Molecular Electronic-Structure Theory&lt;/em&gt;, Wiley, 2000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] P. Pulay, &lt;em&gt;Improved SCF Convergence Acceleration&lt;/em&gt;, Journal of Computational Chemistry, 1982.&lt;/p&gt;&#xA;" OwnerDisplayName="user18279" LastEditorDisplayName="user18279" LastEditDate="2015-12-14T13:36:04.237" LastActivityDate="2015-12-14T13:36:04.237" Title="DIIS method to accelerate SCF convergence for stretched geometries" Tags="|iterative-method|computational-physics|convergence|computational-chemistry|hartree-fock|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="21570" PostTypeId="1" CreationDate="2015-12-14T20:02:42.687" Score="1" ViewCount="220" Body="&lt;p&gt;I am looking for some helps concerning an optimization problem.&#xA;I have an optimization problem defined on two sets  $\mathcal{X}=\{x_i\}_{i=1}^n $ and $\mathcal{Y}=\{y_j\}_{j=1}^m $ and described as follows &#xA;$$&#xA;\begin{equation}&#xA; \begin{aligned}&#xA; &amp;amp; \underset{\alpha, \beta}{\text{max}}&#xA; &amp;amp; &amp;amp; \sum_{i=1}^n \alpha_i x_i^T x_i-\frac{\sum_{i=1}^n \sum_{j=1}^n \alpha_i x_i^Tx_j \alpha_j}{\sum_{i=1}^n \alpha_i}+\sum_{k=1}^m \beta_k y_k^T y_k-\frac{\sum_{k=1}^m \sum_{l=1}^m \beta_k y_k^Ty_l \beta_l}{\sum_{j=1}^m \beta_j }\\&#xA; &amp;amp; \text{subject to} &#xA; &amp;amp; &amp;amp; 0 \leq \alpha \leq C1 \\&#xA; &amp;amp; &#xA; &amp;amp; &amp;amp;   0 \leq \beta \leq C2 \\&#xA; \end{aligned}&#xA;\end{equation}&#xA;$$&#xA;where $C1$ and $C2$ are two given parameters. I'm asking if the above problem can be rewritten in the follwing form&#xA;$$&#xA;\begin{equation}&#xA; \begin{aligned}&#xA; &amp;amp; \underset{X}{\text{max}}&#xA; &amp;amp; &amp;amp;  \frac{ X^T H X}{X}+A^TX\\ %\mathbf {1}_N&#xA; &amp;amp; \text{subject to} &#xA; &amp;amp; &amp;amp; 0 \leq X \leq C \\&#xA; \end{aligned}&#xA;\end{equation}&#xA;$$&#xA;If yes what would be the value of $H$,$A$ and $X$ and if not how can I optimize the first formulation by matlab knowing that the second formulation can be optimized in matlab by CVX package for convex optimization using its &quot;quad_over_lin&quot; function.&lt;br&gt;&#xA;Thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="17055" LastActivityDate="2016-09-10T03:29:08.797" Title="Reformulation of optimization problem" Tags="|quadratic-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21571" PostTypeId="1" AcceptedAnswerId="21573" CreationDate="2015-12-14T21:41:10.530" Score="2" ViewCount="106" Body="&lt;p&gt;I have a certain convex optimization problem which depends on the order of a set of $N$ objects. Meaning, for every possible ordering, I get a different convex optimization problem. My ultimate goal is to find the best optimal solution given all possible orderings. One way of doing that is by solving all $n!$ possible problems. A better way (I think) is to write the whole thing as one MILP and let MOSEK take care of the hard work. I did that as follows: I created $N$ binary variables for every object for a total of $N^2$ binary variables. Say the objects are $F_1,\ldots,F_N$ and the binary variables are $b_{kj}$ where $k,j\in\{1,\ldots,N\}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I let &#xA;$$b_{kj} = \left\{&#xA;\begin{array}{ll}&#xA;1 &amp;amp; \text{if object $F_k$ goes into position $j$;} \\&#xA;0 &amp;amp; \text{otherwise}.&#xA;\end{array}&#xA;\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to guarantee that every object goes into exactly one position, I force the following constraint for every $k$:&#xA;$$\sum_{j=1}^N b_{kj} = 1$$&#xA;Also, in order to guarantee that every position receives exactly one object, I force the following constraint for every $j$:&#xA;$$\sum_{k=1}^N b_{kj} = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This worked for me and I was able to solve the problem using MOSEK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My &lt;strong&gt;first question&lt;/strong&gt; is: is there a better way of doing this? Possibly with fewer than $N^2$ binary variables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My &lt;strong&gt;second question&lt;/strong&gt; is: Let's say I want object $p$ to come before object $q$. How do I force that in the constraints? The only idea I was able to come up with is the following. We must have the following implication for every $k \in \{1,\ldots,N-1\}$.&#xA;$$b_{pk} = 1 \Rightarrow b_{qk} = \ldots = b_{qN} = 0$$&#xA;Which we translate into:&#xA;$$b_{pk} = 0 \vee b_{qk} = \ldots = b_{qN} = 0$$&#xA;which would need additional binary variables to deal with due to the &quot;OR&quot;. Is there a better idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="1932" LastActivityDate="2015-12-14T23:02:27.400" Title="Capturing the order of certain objects in an MILP" Tags="|mixed-integer-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21572" PostTypeId="2" ParentId="21570" CreationDate="2015-12-14T22:56:55.313" Score="1" Body="&lt;p&gt;This question sounds a lot like a homework exercise, so I'll simply give a couple of hints.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Break your objective into the sum of two of two terms, and then use the &quot;epigraph trick&quot; on each separately.  So, you'd have&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$\max s + t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s \leq -\mbox{quad_over_lin(...)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t \leq -\mbox{quad_over_lin(...)}$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Recognize that the two parts of your objective function are already almost in quad_over_lin form.  How can you write &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i=1}^{n} \sum_{j=1}^{n} \alpha_{i} x_{i}^{T}x_{j} \alpha_{j}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\| X\alpha \|_{2}^{2}$?  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-12-14T22:56:55.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21573" PostTypeId="2" ParentId="21571" CreationDate="2015-12-14T23:02:27.400" Score="2" Body="&lt;p&gt;Encoding this constraint in a mixed integer linear programming formulation requires $n^{2}$ variables and doesn't work very well in practice for large $n$.  An alternative is to go beyond integer linear programming to use a constraint programming system with support for the &quot;alldiff&quot; constraint. &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-12-14T23:02:27.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21575" PostTypeId="1" AcceptedAnswerId="21705" CreationDate="2015-12-15T00:31:01.520" Score="6" ViewCount="549" Body="&lt;p&gt;Here is the function I want to minimize:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_i\frac{\rho_{\tau}(1-\alpha-\pmb x_i^{\top}\pmb\beta)}{\sqrt{1+\pmb\beta^{\top}\pmb\beta}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha\in\mathbb{R}$&lt;/span&gt;,  &lt;span class=&quot;math-container&quot;&gt;$\pmb\beta\in\mathbb{R}^p$&lt;/span&gt; are the parameters we want to optimize over and &lt;span class=&quot;math-container&quot;&gt;$\tau\in(0,1)$&lt;/span&gt; and the &lt;span class=&quot;math-container&quot;&gt;$\pmb x_i$&lt;/span&gt; are given and:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rho_{\tau}(x)=\tau\max(0,x)+(1-\tau)\max(-x,0)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is a combination of square root of L2 terms (the denominator) and L1 terms (the numerator). It is not clear to me if this problem can be framed as a convex optimization one (I guess SOCP) and if yes how to reformulate so that it is in standard (SOCP?) form. I guess an answer to either one of these two questions is what I am looking for.&lt;/p&gt;&#xA;&lt;h1&gt;Edit:&lt;/h1&gt;&#xA;&lt;p&gt;This is the objective function described in: He, Xuming; Liang, Hua (1997).&#xA;&lt;em&gt;Quantile regression estimates for a class of linear&#xA;and partially linear errors-in-variables models&lt;/em&gt;. &lt;a href=&quot;http://www3.stat.sinica.edu.tw/statistica/oldpdf/A10n16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ungated copy.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="479" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2015-12-30T18:00:43.440" Title="How to minimize ratio of L1 and square root of L2 norms" Tags="|convex-optimization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21577" PostTypeId="2" ParentId="21544" CreationDate="2015-12-15T08:31:27.013" Score="5" Body="&lt;p&gt;I finally obtained comparable speeds with C++/Armadillo and C/arrays.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Firstly, I switched as suggested by Daniel the order of the loops to column-major order.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Secondly I changed the counters of the loops from int to uword.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally I used the function .at() to disable the bound checking.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18523" LastActivityDate="2015-12-15T08:31:27.013" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21580" PostTypeId="2" ParentId="3262" CreationDate="2015-12-15T15:28:17.183" Score="5" Body="&lt;p&gt;The Eigen Library Documentation also has a nice overview page with a lot of information about different matrix decompositions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://eigen.tuxfamily.org/dox/group__TopicLinearAlgebraDecompositions.html&quot; rel=&quot;noreferrer&quot;&gt;http://eigen.tuxfamily.org/dox/group__TopicLinearAlgebraDecompositions.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="740" LastActivityDate="2015-12-15T15:28:17.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21581" PostTypeId="1" CreationDate="2015-12-15T21:34:11.187" Score="1" ViewCount="344" Body="&lt;p&gt;I need to synchronize intermediate solutions of an optimization problem solved distributively over a number of worker processors. The solution vector is known to be sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have noticed that if I use MPI_AllReduce, the performance is good compared to my own AllReduce implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I believe, the performance can be further improved if AllReduce could communicate only the nonzero entries in the solution vector. I could not find any such implementation of AllReduce.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that MPI_type_indexed can not be used as the indices of the nonzero entries are not known in advance. &lt;/p&gt;&#xA;" OwnerUserId="18571" LastActivityDate="2018-08-11T13:20:59.580" Title="Is there an implementation of MPI_AllReduce which handles sparse data better?" Tags="|mpi|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21583" PostTypeId="1" CreationDate="2015-12-16T07:47:32.747" Score="1" ViewCount="58" Body="&lt;p&gt;Does anybody know if any graph partitioner library such as Metis, Scotch, or Zoltan can (besides splitting a domain), differentiate between internal (i) and boundary (b) nodes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SWWKT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SWWKT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18328" LastActivityDate="2015-12-16T07:47:32.747" Title="Interior nodes of a closed graph?" Tags="|software|computational-geometry|graph-theory|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21584" PostTypeId="1" CreationDate="2015-12-16T11:22:58.720" Score="1" ViewCount="517" Body="&lt;p&gt;How can I estimate how much memory will be needed to find eigenvalues and eigenvectors of a given large sparse matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a real symmetric matrix with roughly $5 \times 10^4$ rows and columns, and an average of $10$ nonzero elements per row. I would like to find the smallest eigenvalue and the corresponding eigenvector, using the built-in &lt;code&gt;Eigensystem&lt;/code&gt; function in &lt;em&gt;Mathematica&lt;/em&gt; (which treats the matrix as sparse and &lt;a href=&quot;http://reference.wolfram.com/language/tutorial/SomeNotesOnInternalImplementation.html#28959&quot; rel=&quot;nofollow&quot;&gt;uses an ARPACK Arnoldi algorithm&lt;/a&gt;). Is there a simple way of estimating how much memory this will take?&lt;/p&gt;&#xA;" OwnerUserId="16906" LastEditorUserId="16906" LastEditDate="2015-12-16T12:22:59.443" LastActivityDate="2015-12-16T12:22:59.443" Title="Memory requirement to find eigenvalues and -vectors of large sparse matrix" Tags="|eigensystem|memory-management|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21585" PostTypeId="2" ParentId="21562" CreationDate="2015-12-16T12:44:25.267" Score="3" Body="&lt;p&gt;I know this topic well for some class of PDEs, so I try to give you a general answer with one example from applications I am familiar with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) &lt;strong&gt;Conservative form of PDE&lt;/strong&gt; - this notion is used when your PDE is derived from a (typically integral) form of some (physical) conservation laws and you write this PDE in the origin form. Very often you can rewrite the PDE in a different form using some rules for derivatives and, even more importantly, using some additional properties for some input data in your PDE, then you call this other form of PDE to be non-conservative to distinguish it from the origin conservative form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance the conservation of tracer transported in a fluid can be modeled by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation} \nonumber  \partial_t \rho + \nabla \cdot (\rho \vec{v} ) = 0 \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this is the conservative form of the PDE for any velocity $\vec{v}$. You can rewrite it using standard rule for derivatives to another (equivalent) form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \partial_t + \nabla \rho \cdot \vec{v} + \rho \nabla \cdot \vec{v} = 0 . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you moreover deal with so called incompressible flows then $\nabla \cdot \vec{v} = 0$, so you obtain finally&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \partial_t \rho + \vec{v} \cdot \nabla \rho = 0 . $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last PDE you might call to be written in a nonconservative form, because you &lt;strong&gt;must&lt;/strong&gt; require $\nabla \cdot \vec{v} = 0$ to say it describes the conservation of transported tracer. Without this condition the last PDE is not conservative in general, it is so called advection equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) &lt;strong&gt;Conservative numerical method&lt;/strong&gt; - if you directly approximate the conservative PDE, then usually your method is conservative if you do everything &quot;right&quot; (some comments on this in the third answer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you implement a numerical method for the non-conservative form of PDE, although your problem is described by a conservative PDE, then you must typically insure (or do some additional work to show) that your method is conservative. If I simplify it, any numerical method derived as an approximation of non-conservative form of some conservative PDE can be in general non-conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.) &lt;strong&gt;Discrete conservation&lt;/strong&gt; - your conservative PDE describes a conservation of some quantity that is defined in the continuous form typically by some integral. This quantity (or its density) is changing in time and space only due to &quot;flow&quot; (&quot;fluxes&quot;) of it. Typically if you compute the conservative PDE on a bounded domain with no flow at its boundary, then this conserved quantity in the whole domain does not change in time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To speak about the same property in a discrete form, you must define how you approximate these quantities in a discrete form and how they are fulfilled in a discrete form. Then you may speak about discrete conservation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance for my first conservative PDE in 1D case the conservative method is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \rho_i^{n+1} = \rho_i^n - \frac{dt}{dx} (\rho_{i+1/2}^n v(x_{i+1/2}) - \rho_{i-1/2}^n v(x_{i-1/2}))  $$&#xA;that shall be valid for $i=1,2,\ldots,I$ (I try to use typical notation for numerical schemes without going into details).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is conservative, because you use consistently the approximation of fluxes, i.e. the left flux for the $i$-th discrete equation is equal to the right flux of the discrete equation for $i-1$. If $v(x_{1/2})=0$ and $v(x_{I+1/2})=0$ then $$dx \sum_{i=1}^I \rho_i^n = dx \sum_{i=1}^I \rho_i^0$$&#xA;for each $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you approximate my third non-conservative form in 1D you might use &#xA;$$ \rho_i^{n+1}= \rho_i^n - dt \, v(x_i) \frac{\rho_{i+1/2}^n - \rho_{i-1/2}^n}{dx} $$&#xA;and that is definitely not conservative numerical method (except for a constant $v$). &lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2015-12-16T12:44:25.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21586" PostTypeId="1" AcceptedAnswerId="21596" CreationDate="2015-12-16T15:10:24.490" Score="4" ViewCount="690" Body="&lt;p&gt;I am looking for some production-ready code to update dense QR and/or Cholesky factorizations (by adding / removing rows and columns or making small-rank updates -- yes, I need all these cases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done my &lt;strike&gt;Google&lt;/strike&gt;literature research, but I have found very little:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Matlab has this &lt;a href=&quot;https://it.mathworks.com/help/matlab/ref/qrupdate.html?requestedDomain=www.mathworks.com&quot; rel=&quot;nofollow&quot;&gt;covered&lt;/a&gt;, but unfortunately this is for a C++ project, so it is not easy to interface with it. I'd much rather have C/C++ or Fortan, especially the &quot;traditional&quot; LAPACK-style code.&lt;/li&gt;&#xA;&lt;li&gt;LAPACK has nothing, as far as I can tell. There was something in LINPACK (LAPACK's precursor), but as far as I can tell it has not been updated in the last 30 years at least.&lt;/li&gt;&#xA;&lt;li&gt;There is some &lt;a href=&quot;http://www.maths.manchester.ac.uk/~clucas/updating/&quot; rel=&quot;nofollow&quot;&gt;code&lt;/a&gt; by Craig Lucas for one of these subtasks (adding/deleting columns only for a QR factorization), and some &lt;a href=&quot;http://anchp.epfl.ch/QRupdate&quot; rel=&quot;nofollow&quot;&gt;code&lt;/a&gt; by Daniel Kressner for adding rows to a QR factorization only. Both of them are only limited to one of the subtasks that I need, and, to quote Kressner's page, &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is not tested thoroughly and should be understood as research code.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Surprisingly, there is mature code for the same task for sparse matrices, in Tim Davis' &lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot; rel=&quot;nofollow&quot;&gt;Suitesparse&lt;/a&gt;. I don't suppose that the code will be efficient on a dense matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Michael Saunders' &lt;a href=&quot;https://web.stanford.edu/group/SOL/software/lusol/&quot; rel=&quot;nofollow&quot;&gt;LUSOL&lt;/a&gt; has the corresponding code for a (sparse) LU factorization (unsymmetric case).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So it seems like there is plenty of code for doing &lt;em&gt;similar&lt;/em&gt; and &lt;em&gt;related&lt;/em&gt; tasks, but not exactly for this problem, which seems surprising.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I overlooking some obvious choice? Is there a library to do it? &lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2015-12-16T20:02:03.300" LastActivityDate="2015-12-17T09:52:29.637" Title="Code to update dense QR and Cholesky factorizations" Tags="|linear-algebra|libraries|lapack|matrix-factorization|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21587" PostTypeId="1" AcceptedAnswerId="21588" CreationDate="2015-12-16T15:20:07.190" Score="9" ViewCount="1108" Body="&lt;p&gt;First of all, hope I've found the right forum for this question, if I haven't please pass me on to one which would fit better.&lt;/p&gt;&#xA;&lt;p&gt;Out of curiosity from an argument with someone who may or may not be more into CPUs than I am. We were arguing about the performance of floating point operations on modern CPUs. Is there any source showing hard-ware performances regarding typical functions such as exp, sin, cos on modern processors?&lt;/p&gt;&#xA;&lt;p&gt;And in particular : can brute force computation compete with tabulation for these functions?&lt;/p&gt;&#xA;" OwnerUserId="18579" LastEditorUserId="18579" LastEditDate="2023-08-14T16:17:05.527" LastActivityDate="2023-08-14T16:17:05.527" Title="Hardware performance, floating point functions" Tags="|performance|floating-point|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21588" PostTypeId="2" ParentId="21587" CreationDate="2015-12-16T16:22:03.503" Score="14" Body="&lt;p&gt;In the 1980's era Intel 80x86 architecture, there was a scalar floating point unit that had instructions like FSIN, FCOS, etc. for computing functions like sin and cos.  These functions were implemented in microcode and might take 100's of CPU cycles to execute.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later, Intel added Streaming SIMD Extensions (SSE) which gave the processor parallel floating point units that could be used to perform floating point operations on multiple operands in parallel.  The SSE instruction set includes basic arithmetic operations (+,-,*,/,sqrt), but the SSE instruction set does not include instructions for functions like sin and cos.  SSE has gone through several versions over the decades with additional instructions and more floating point registers.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the older instructions are still in the architecture, the SSE floating point units are extremely fast.  In practice the SSE instructions are now used almost exclusively for floating point arithmetic with heavily optimized library routines used to compute functions like sin and cos.  These routines will typically compute the function values by evaluating polynomial or rational function approximations of the functions.  Sometimes a small table is used to select which polynomial is used for different ranges of input values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since floating point computation are extremely fast compared to memory accesses, and because a very large table would be required to hold all of the possible values for these functions, straight forward table lookup could not reasonably compete with this approach.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this stackexchange question for more information and links to some specific implementations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/2284860/how-does-c-compute-sin-and-other-math-functions&quot;&gt;https://stackoverflow.com/questions/2284860/how-does-c-compute-sin-and-other-math-functions&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2015-12-16T16:22:03.503" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21589" PostTypeId="2" ParentId="20684" CreationDate="2015-12-16T18:24:01.123" Score="2" Body="&lt;p&gt;My hunch is that you may be prescribing boundary data at both ends of the domain (which leads to oscillations) or incorrectly specifying the boundary data. For a hyperbolic problem, you need to respect the flow of information in your numerical method. For example, consider the linear hyperbolic equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf u_t = \mathbf A \mathbf u_x $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with Dirichlet boundary conditions at the endpoints $x = x_L$ and $x = x_R$. For simplicity, we assume that $\mathbf A$ is a constant coefficient matrix with non-degenerate eigenvalues and linearly independent eigenvectors. By the spectral theorem we can write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf A = \mathbf P \mathbf D \mathbf P^{-1} $$ &#xA;where $\mathbf D$ is a diagonal matrix of eigenvalues and $\mathbf P$ is the respective eigenvector matrix. Using the decomposition, we can rewrite our PDE as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf v = \mathbf P^{-1} \mathbf u $$&#xA;$$ \mathbf v_t = \mathbf D \mathbf v_x $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system is now decoupled and the sign of the eigenvalue will tell you the direction that the characteristics $\xi = x + \lambda t$ are propagating. Characteristics are lines where the solution is constant. For example, for $\lambda_k &amp;gt; 0$, characteristics propagate from left to right. Consider the $(x, t)$-plane of the characteristics. At the initial time, $t = 0$, the initial condition supplies the starting point for the emanating characteristics. For $t &amp;gt; 0$, the characteristics travel from left to right. If we trace any characteristic from the initial time, we can see that the characteristic will eventually exit the domain at $x_R$. If we try to trace any characteristic emanating from $x_L$, we run into the issue that there is no information provided at this boundary point, thus we must prescribe the boundary condition for $v_k$ at $x_L$. If we were to prescribe a boundary condition at $x_R$ as well, this will lead to numerical instability for we are over constraining our system by telling it how the characteristics should behave at the outflow boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are implementing the uncoupled system, then the boundary data that you should impose should still be in terms of the characteristic variables. For example, when solving &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_t = v_x $$&#xA;$$ v_t = u_x $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $-1 \leq x \leq 1$, and Dirichlet boundary conditions $u(-1,t) = f_1(t)$, $v(-1,t) = f_2(t)$, $u(1,t) = f_3(t)$, $u(1,t) = f_4(t)$, it is easy to see that the characteristic variables are $(u + v)$ and $(u - v)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (u+v)_t = (u+v)_x $$&#xA;$$ (u-v)_t = -(u-v)_x $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and that we need to impose an incoming characteristic for $(u+v)$ at $x=-1$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(-1,t) + v(-1,t) = f_1(t) + f_2(t) $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and an incoming characteristic for $(u-v)$ at $x=1$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(1,t) - v(1,t) = f_3(t) + f_4(t) $$ &lt;/p&gt;&#xA;" OwnerUserId="17368" LastActivityDate="2015-12-16T18:24:01.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21590" PostTypeId="2" ParentId="21547" CreationDate="2015-12-16T18:49:36.543" Score="1" Body="&lt;p&gt;I'm trying to answer it with physical intuition and arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume you'll get $p(\mathbf{r},t)$ as solution. Take only one dimension for example i.e. writing as $p(x,t)=\Sigma_n \Sigma_m a_{mn} Sin(k_mx-\omega_nt)$. If you Fourier decompose it in space first, you'll $k_m$s. Then for each $k_m$, do Fourier transform in time to get $\omega_n$. Now plot of $\omega_n$ vs $a_{mn}$ for one n at a time. The peak will correspond to the leading $k_m$ for that $\omega_n$ (Here I'll argue that only leading order matters; this can be refined to find second and later order dependence on $k_m$). Think about this way-- if some $a_{mn}$ is numerically zero for some m (call $m_0$) for one particular n (call $n_o$), this means that the component of wave, $Sin(k_{m_0}x-\omega_{n_{0}}t)$ doen't exist in the full wave.&#xA;Similarly, for other n. This way you'll end up with the pairs $(n,m)$. In general, you'll get different m for each n. Finally, find a fit to this curve or use the plot to define your dispersion relation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please look into care and considerations of discrete fourier transform in space and time which has some limits on your choice of wavelengths ($k_m$) and time period ($\omega_n$) by cell size and time step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sweep long $x, y$ and $z$ separately to get $(k_x,k_y,k_z)$ in each cell to expand this to three dimensions.&lt;/p&gt;&#xA;" OwnerUserId="4992" LastActivityDate="2015-12-16T18:49:36.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21591" PostTypeId="1" AcceptedAnswerId="21597" CreationDate="2015-12-16T20:02:36.013" Score="8" ViewCount="1091" Body="&lt;p&gt;There are lot of articles being written about how the newly launched Intel Xeon Phi will steal the HPC\Super Computer market share from the competitors. Intel Knights is equipped with 72 cores and 4 sockets making it 288 core system. Whereas a single Gtx980 has 2048 CUDA cores. How can these two compete in terms of Computing power (It looks like GTX is way ahead)? Or are they targeting completely different work loads, in which case what are some examples? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.  &lt;/p&gt;&#xA;" OwnerUserId="18586" LastActivityDate="2015-12-17T10:52:28.130" Title="Intel Knights Landing work loads vs NVIDIA GeForce" Tags="|parallel-computing|hpc|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21592" PostTypeId="1" CreationDate="2015-12-16T20:30:15.230" Score="2" ViewCount="721" Body="&lt;p&gt;I recently timed the linsolve direct solver and I was kind of shocked to see that the solver seemed to be scaling quadratically even upto a 1000 dimensions. Specifically I ran the following code and it showed that the cubic coefficient of time taken $a_3 \approx \frac{1}{1000}a_2$. From my recollection the cubic coefficient is not supposed to be so small but more like $\frac{1}{10}a_2$. Why is the routine so fast? I am on a 5 year old macbook pro with only 2 core. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear; close all; clc&#xA;pv = [ 10 20 30 40 50 60 70 80 90 100 200 300 400 500 600 700 800 900 1000];&#xA;tv = nan(length(pv), 1);&#xA;runs=3;&#xA;tv2 = nan(length(pv), 1);&#xA;for pi = 1:length(pv)&#xA;    p = pv(pi);&#xA;    v = randn(p, 1);&#xA;    m = randn(p, p);&#xA;    % Calculate time for mldivide.&#xA;    tic; for i = 1:runs mldivide(m, v); end; tv(pi) = toc/runs * 1e9;&#xA;    % Calculate time for linsolve.&#xA;    tic; for i = 1:runs linsolve(m, v); end; tv2(pi) = toc/runs * 1e9;&#xA;end&#xA;plot(pv, tv, 'b');&#xA;hold on;&#xA;plot(pv, tv2, 'r');&#xA;ylabel('nano-seconds');&#xA;xlabel('dimension');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/91OG5.png&quot; alt=&quot;a busy cat&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="8029" LastActivityDate="2015-12-17T00:07:03.587" Title="Is the &quot;practical&quot; complexity of linsolve direct solver O(n^2) ?" Tags="|matlab|linear-solver|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21594" PostTypeId="2" ParentId="21592" CreationDate="2015-12-16T23:19:08.323" Score="5" Body="&lt;p&gt;Your matrices are far too small to see the asymptoptic $O(n^3)$ running time behavior of the LU factorization used by linsolve.  For very small matrices the overhead of computations surrounding the LU factorization will make it difficult to see the $O(n^3)$ growth.  Furthermore, MATLAB will typically be making use of parallel routines for computing the LU factorization that only start to be efficient at the smaller matrix sizes you've experimented with.  Try your demo again using a range of sizes from $p=500$ up to say $p=20000$ and see what happens.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2015-12-17T00:07:03.587" LastActivityDate="2015-12-17T00:07:03.587" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21595" PostTypeId="2" ParentId="21591" CreationDate="2015-12-16T23:23:43.503" Score="7" Body="&lt;p&gt;CUDA cores aren't at all comparable to the separate processor cores in the Xeon Phi coprocessors.  The Phi coprocessor cores are full fledged processors that can have their own loops, branching, etc. while the CUDA cores are all executing the same operations on various slices of your data.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-12-16T23:23:43.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21596" PostTypeId="2" ParentId="21586" CreationDate="2015-12-17T09:52:29.637" Score="3" Body="&lt;p&gt;You can use the package &lt;a href=&quot;http://sourceforge.net/projects/qrupdate/files/qrupdate/1.1/qrupdate-1.1.1.tar.gz/download&quot; rel=&quot;nofollow&quot;&gt;qrupdate&lt;/a&gt; that is the one linked in Octave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's in Fortran... but it should be self contained. If you need to link it to a C++ project you can look at the Octave &lt;a href=&quot;ftp://ftp.gnu.org/gnu/octave/octave-4.0.0.tar.gz&quot; rel=&quot;nofollow&quot;&gt;source code&lt;/a&gt;, file &lt;code&gt;floatQR.cc&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18255" LastActivityDate="2015-12-17T09:52:29.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21597" PostTypeId="2" ParentId="21591" CreationDate="2015-12-17T10:52:28.130" Score="8" Body="&lt;p&gt;Like Brian said, the Xeon Phi cores are not at all comparable to the CUDA ones. The problem with the Phi is that it's somewhere between two horses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are doing highly parallel floating point calculations, NVIDIA will provide you with something like 3 times the performance at 1/4th of the price. For double precision the gap is smaller, but NVIDIA still ends up being 20% cheaper for the same performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem is very hard to parallelize, the Phi will not help you at all and instead something like an Intel Xeon will give you the best performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sweet spot for the Phi is then something highly parallel, but divergent, i.e. each thread has to do something different. An example of this would be Monte Carlo simulations. They are for instance used in simulations of radiotherapy treatments, where GPUs only give a small (~2x) speed-up over a standard CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intel is also trying to sell the the Xeon Phi on the fact that you only need to rewrite your code minimally. However, for anything that is not trivial to parallelize, the work becomes the same as for a GPU. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2015-12-17T10:52:28.130" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21599" PostTypeId="1" CreationDate="2015-12-17T14:05:30.537" Score="10" ViewCount="372" Body="&lt;p&gt;I have been trying to make a biologically accurate 2D spatial model of tissue layers, where different physiological processes happen. This includes mainly chemical reactions, diffusion and fluxes over boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am making this model in COMSOL Multiphysics, a finite element software package that solves different physics like reaction-diffusion systems, although for my question this might not be really relevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my geometry, I have really small regions between the cells of the tissue layers. These regions serve as openings where diffusion can take place between the cells (junctions). The quality of the mesh is not great here and if I want to improve the quality (mainly by introducing more elements and such), my simulation time increases drastically. The lesser quality mesh also causes convergence to take longer. I added a picture of the geometry to give an idea. I tried different meshes, all with different qualities of the elements and the number of elements ranging from 16000 to 50000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My background in FEM is really limited and I wanted to know if I can tackle this problem in such a way that it: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;doesn't negatively affect the biology (keep the tissue domain sizes/problem etc as biologically accurate as possible),&lt;/li&gt;&#xA;&lt;li&gt;doesn't increase the simulation time drastically,&lt;/li&gt;&#xA;&lt;li&gt;give a better mesh quality. So I really want to know what the best way to go is, since I have already thought of some things.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So can I go with the lesser quality mesh (which is not really bad, but not good either), so that I can keep the small regions for optimum biological accuracy and have a relatively small computation time (and hope I won't run into convergence errors). But maybe there are possibilities that I am missing, for instance: is it possible to make the small domain bigger and then add some kind of factor to the diffusion rates. In other words, if I want to make the domain twice as large, do I factor the diffusion rate with half? Is that even accurate in chemical/physical laws :S.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully I made the problem a bit clear and thank you greatly in advance for the help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8Ptib.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Ptib.png&quot; alt=&quot;Mesh of the tissue model&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18598" LastActivityDate="2016-02-09T20:00:36.363" Title="How to make a good mesh in a biologically accurate model with very small domains" Tags="|finite-element|comsol|computational-biology|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21602" PostTypeId="1" CreationDate="2015-12-17T22:51:17.860" Score="7" ViewCount="175" Body="&lt;p&gt;Lets say I've decomposed a continuous function $y(x)$ over some domain $L_x$ using known finite element method with local basis $Q_i(x)$. Suppose $L$ is divided into $M$ &quot;elements&quot;. If I want to know the function $y(x)$ at a point $x=p$ (where $p$ is not at a nodal value) then I will need to first know which element $p$ is in. I can then interpolate between the element nodes and find the value $y(p)$. I can do this for all $p$ in $L$ and get back my continuous function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, one could search through every element to find the where $p$ lies. This is fine in simple cases, but what about more complex cases when there are 1000's of elements and we are working in 3D? Is there an efficient algorithm that I'm missing that could compute this?&lt;/p&gt;&#xA;" OwnerUserId="18531" LastEditorUserId="17671" LastEditDate="2015-12-18T19:26:15.303" LastActivityDate="2015-12-18T21:23:01.300" Title="Reconstructing a continuous function from finite element method. Is there a faster algorithm for doing so?" Tags="|finite-element|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21603" PostTypeId="2" ParentId="21602" CreationDate="2015-12-18T00:18:10.233" Score="2" Body="&lt;p&gt;You could use a domain decomposition approach; divide the domain into coarse subdomains recursively and search through each one (like a binary tree search in 1D).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know more about the point $p$, you can probably make it more efficient.  For example, if the point $p$ does not change, lookup information can be precomputed and stored during the setup stage. If it changes in time but depends on properties of the solution, it can often be determined by transforming to the reference element and doing the search in parallel. &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2015-12-18T00:18:10.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21604" PostTypeId="2" ParentId="21599" CreationDate="2015-12-18T03:47:22.570" Score="5" Body="&lt;p&gt;With conforming triangular meshes, it will be difficult to make an isotropic mesh which adapts to multiple dramatically different length scales in such a short space without introducing extraneous triangles, some of which may have very large/small angles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not very familiar with them so take this with a grain of salt, but you may have better luck using &lt;a href=&quot;https://en.wikipedia.org/wiki/Mortar_methods&quot;&gt;mortar element methods&lt;/a&gt;. Rather than try to discretize the entire geometry on one mesh, you instead discretize the bulk medium and the junctions on completely separate, non-conforming meshes. The chemical species are modeled separately within each domain, and then coupled globally through the appropriate boundary fluxes; an iterative procedure is used to ensure that all the fluxes match properly across the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method doesn't solve everything for you; it just exchanges the difficulty of getting a nice discretized geometry for the difficulty of coupling the PDEs across the junction boundaries in the correct way, which may be simpler in the end. It also has the distinct advantage of lending itself to parallelism quite naturally.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2015-12-18T03:47:22.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21605" PostTypeId="2" ParentId="21602" CreationDate="2015-12-18T06:33:36.523" Score="4" Body="&lt;p&gt;Don't do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On general meshes, finding an arbitrary point $p$ requires $O(N)$ operations where $N$ is the number of cells. You can get away with significantly less, say $O(\log N)$, if you have a mesh that is constructed hierachically from a coarse mesh by mesh refinement, but unless you use a completely structured grid, finding an arbitrary point inside a mesh is simply a &lt;em&gt;very expensive&lt;/em&gt; operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, once you have the cell $K$ in which $p$ lies, you need to find the location $\hat p$ on the &lt;em&gt;reference&lt;/em&gt; cell $\hat K$ that corresponds to $p$. For this, you have to invert the mapping $\phi_K: \hat K \mapsto K$, and this (in general) requires a nonlinear iteration -- which is also expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence, most algorithms try to avoid this at any cost. For example, when you do integration of functions, you loop over all cells, and on each cell over the quadrature points (which are defined on the reference cell, and so only need to be mapped forward with $\phi_K$). This is a far faster approach that looping over a bunch of quadrature points defined in real space (as opposed to reference space relative to each cell).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, the point is that you need to re-think the overall approach to doing things so that you always start with a loop over all cells, if at all possible.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-18T06:33:36.523" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21606" PostTypeId="2" ParentId="21599" CreationDate="2015-12-18T06:52:18.960" Score="6" Body="&lt;p&gt;You're trying to have your cake and it it too. This does not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a general rule, for problems with features on different length scales, you need meshes that are fine in at least some parts of the mesh. This results in many cells, and this results in long computations, small time steps, and many linear iterations. All of these implications are rather self-explanatory, but one can back them up with mathematical statements that prove that this is so. There is simply not very much you can about it: resolving small features will always be expensive.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2015-12-22T02:40:01.790" LastActivityDate="2015-12-22T02:40:01.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21607" PostTypeId="1" AcceptedAnswerId="21622" CreationDate="2015-12-18T09:53:09.240" Score="4" ViewCount="218" Body="&lt;p&gt;I encounter an optimization problem. The simplified version is like following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denote function $F(x):\mathbf{R}^n\rightarrow\mathbf{R}$, where $F(x)$ is a smooth lower bounded convex function (i.e. $F(x)&amp;gt;0$). The hessian matrix of $F(x)$ is positive definite. However $F(x)$ is not a strongly convex function, namely we can not assume the hessian matrix $H(x)\succ m I_n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose for the sequence $\{\mathbf{x}_k\}_{k=1}^\infty$, the gradients of the $F(x)$ converge to zero:&#xA;\begin{equation}&#xA;\lim_{k\rightarrow\infty} \|\mathbf{g}_k\|_2 = 0,\text{  and  }F(x_{k+1})\le F(x_{k}), k=0,1,...&#xA;\end{equation}&#xA;where $\mathbf{g}_k=\frac{\partial F}{\partial x} |_{x=x_k}$. Do we have the following result&#xA;\begin{equation}&#xA;\lim_{k\rightarrow\infty} F(x_{k}) = \inf _{x\in\mathbf{R}^n} F(x)?&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect $\lim_{k\rightarrow\infty} F(x_k) = \inf_{x\in\mathbf{R}^n} F(x)$. However, I can not find proofs for this result. There are some results about this problem. But unfortunately, either assume the optimum point of $F(x)$ is attainable, namely, there exists $x^*\in\mathbf{R}^n$, such that $F(x^*) = \inf F(x)$ or there exists $x^*\in\mathbf{R}^n$, such that $\lim_{k\rightarrow\infty} x_k=x^*$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could any one help me to solve this problem or give me some comments? Thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="16425" LastEditorUserId="16425" LastEditDate="2015-12-20T08:44:01.833" LastActivityDate="2015-12-22T06:33:08.630" Title="convergence of unconstrained convex optimization" Tags="|optimization|convergence|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21609" PostTypeId="1" CreationDate="2015-12-18T14:12:20.510" Score="1" ViewCount="866" Body="&lt;p&gt;I am currently working on an iterative closest point algorithm (in C++, see &lt;a href=&quot;https://github.com/DeveloperPaul123/Optimum&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the basic premise of an ICP algorithm. You have two point clouds (a target and a reference) and you want to register the reference into the target. You do this by:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Associate Pairs of Points (k-d tree or something similar)&lt;/li&gt;&#xA;&lt;li&gt;Find optimal rotation and translation such that the RMSE of the distance between the transformed points and the target is minimized (in my case I use the Kabsch algorithm with the singular value decomposition).&lt;/li&gt;&#xA;&lt;li&gt;Next, update the reference points with the previously calculated rotation and translation and iterate again using the new reference points and target points.&lt;/li&gt;&#xA;&lt;li&gt;Do this until stopping criteria is met. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My question is how to properly keep track of the rotation and translation. Currently I update the rotation by calculating the total rotation (in degrees) so far and adding to it the new rotation (in degrees). To keep track of the total tranlation I take the last translation (i.e. the total translation so far), multiply it by the newly calculated rotation and then add the newly calculated rotation. In other words:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Assume current look is loop i, newRotation and newTranslation have just been calculated using the Kabsch method, &lt;code&gt;toMatrix()&lt;/code&gt; converts an angle in degrees to a rotation matrix, &lt;code&gt;toDegrees()&lt;/code&gt; converts a rotation matrix to an angle in degrees). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rotation(i) = toMatrix(toDegrees(rotation(i-1)) + toDegrees(newRotation))&#xA;&#xA;translation(i) = (newRotation * translation(i-1)) + newTranslation. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This method seems fundamentally wrong though because my error (Root mean square error) increases with every iteration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data I am using to test this is very well matched, in fact I know the mapping beforehand, so the algorithm should converge rather quickly. I have verified my closest point matching works so I think the calculation of the 'total' rotation and translation is the problem. Is this the proper way to implement this or am I doing something wrong?&lt;/p&gt;&#xA;" OwnerUserId="18613" LastActivityDate="2016-02-08T21:57:32.453" Title="Iterative Closest Point Algorithm" Tags="|optimization|iterative-method|svd|3d|closest-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21610" PostTypeId="1" CreationDate="2015-12-18T17:45:47.753" Score="2" ViewCount="172" Body="&lt;p&gt;I am interested in fixed-step simulation of an ODE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The methods I know of are either variable-step with prescribed error tolerance or fixed-step without error control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there methods known which work on a fixed time grid (for inputs and outputs) AND allow the specification of an error tolerance? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution for the next time step should be calculated as efficient as possible, thus adaptive, e.g. RK12 - RK45. It could try RK12 and increase the order if necessary. Does such a scheme make sense in terms of efficiency? How could it be improved?&lt;/p&gt;&#xA;" OwnerUserId="18615" LastEditorUserId="18615" LastEditDate="2015-12-18T20:46:48.737" LastActivityDate="2015-12-19T10:47:56.190" Title="Fixed-step ODE solver with variable order?" Tags="|ode|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21611" PostTypeId="1" AcceptedAnswerId="21613" CreationDate="2015-12-18T18:39:19.410" Score="5" ViewCount="371" Body="&lt;p&gt;I am given a $n \times n$ matrix $A$ with real entries and define the inner product $$\langle x,y\rangle = x^T A y.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also given an integer $k$ and need to find all binary vectors $x$ such that $\langle x,x \rangle = k.$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By binary vector I mean a vector whose entries are in $\{0,1\}.$ For $n$ small it is not hard to solve this problem - simply test all possible $2^n$ vectors. In practice $n$ can be quite large and there is only a small fraction of vectors that satisfy this condition. Hence I am wondering&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a more efficient way to compute all binary vectors satisfying $\langle x,x \rangle = k$?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am pretty sure this problem is hard in general but I am wondering if there is any way to get away with having to test all $2^n$ vectors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit.&lt;/strong&gt; The matrix $A$ is of the form $(\lambda I-B)^{-1}$ where $\lambda$ is an integer and $B$ is a binary symmetric matrix. $\lambda$ is not an eigenvalue of $B.$&lt;/p&gt;&#xA;" OwnerUserId="4582" LastEditorUserId="4582" LastEditDate="2015-12-18T20:11:08.483" LastActivityDate="2015-12-18T20:11:08.483" Title="Finding all binary vectors with given A-length" Tags="|linear-algebra|matrix|performance|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21612" PostTypeId="1" CreationDate="2015-12-18T19:22:50.240" Score="9" ViewCount="10439" Body="&lt;p&gt;I'm trying to solve a 1D Poisson equation with pure Neumann boundary conditions. I've found many discussions of this problem, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;a href=&quot;https://scicomp.stackexchange.com/questions/12913/poisson-equation-with-neumann-boundary-conditions&quot;&gt;Poisson equation with Neumann boundary conditions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5355/writing-the-poisson-equation-finite-difference-matrix-with-neumann-boundary-cond&quot;&gt;Writing the Poisson equation finite-difference matrix with Neumann boundary conditions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14246/discrete-poisson-equation-with-pure-neumann-boundary-conditions&quot;&gt;Discrete Poisson Equation with Pure Neumann Boundary Conditions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14306/finite-differences-and-neumann-boundary-conditions&quot;&gt;Finite differences and Neumann boundary conditions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And many more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the answers seem unsatisfactory though. For example, the answer in 2) contains a document that explains how the matrix $A$ changes when applying Neumann boundary conditions, but does not explain how to solve the singular system. The comment left in 2) by @Evgeni Sergeev is a reference to a problem with mixed boundary conditions, and not pure Neumann boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, there are some useful things I've found. I do like the second comment given in the 1) by @Sumedh Joshi. It suggests to subtract the mean from the RHS. Other references I've read suggest using Dirichlet BCs at one location in the computational domain which should result in a unique solution, however, I prefer removing the mean since this seams more elegant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem setup is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial x^2} = f, \qquad f = -(2\pi)^2 cos(2\pi x), \qquad 0 \le x\le 1, \qquad \left(\frac{\partial u}{\partial x}\right)_{0} = 0, \qquad \left(\frac{\partial u}{\partial x}\right)_{1} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have ghost points in my computational domain, so my matrix $A$ has some extra zeros (I remove them later). Using central differences everywhere in the computational domain, matrix $A$ takes the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A = \frac{1}{\Delta x^2}&#xA;\left[&#xA;\begin{array}{ccccccccc}&#xA;0  &amp;amp; 0     &amp;amp; 0         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;1  &amp;amp; -2    &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;0  &amp;amp; 1     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; 1       &amp;amp;  0 \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  1        &amp;amp; -2      &amp;amp;  1 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  0        &amp;amp; 0       &amp;amp;  0 \\&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once applying the ghost points and adjusting the matrix $A$, I end up with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A = \frac{1}{\Delta x^2}&#xA;\left[&#xA;\begin{array}{ccccccccc}&#xA;0  &amp;amp; 0     &amp;amp; 0         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;0  &amp;amp; -2    &amp;amp; 2         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;0  &amp;amp; 1     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; 1       &amp;amp;  0 \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  2        &amp;amp; -2      &amp;amp;  0 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  0        &amp;amp; 0       &amp;amp;  0 \\&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Correspondingly, the RHS has changed to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{- 2u_b + 2u_i}{\Delta x^2} = f_b - \frac{-2\theta \Delta x}{\Delta x^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u_b,u_i,f_b,\theta$ are the $u$ at the boundary, $u$ at the first interior point, $f$ at the boundary and the slope of the solution at the boundary (zero in this case) respectively. The interior of this matrix is the same as the one discussed in the document given in the accepted answer in 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I show the setup and results of a small MATLAB script where I print out $A, f, mean(f)$ and the (attempted) solution $u$ using MATLAB's backslash operator. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  A =&#xA;&#xA;   -100.0000  100.0000         0         0         0         0         0         0         0         0         0&#xA;    100.0000 -200.0000  100.0000         0         0         0         0         0         0         0         0&#xA;           0  100.0000 -200.0000  100.0000         0         0         0         0         0         0         0&#xA;           0         0  100.0000 -200.0000  100.0000         0         0         0         0         0         0&#xA;           0         0         0  100.0000 -200.0000  100.0000         0         0         0         0         0&#xA;           0         0         0         0  100.0000 -200.0000  100.0000         0         0         0         0&#xA;           0         0         0         0         0  100.0000 -200.0000  100.0000         0         0         0&#xA;           0         0         0         0         0         0  100.0000 -200.0000  100.0000         0         0&#xA;           0         0         0         0         0         0         0  100.0000 -200.0000  100.0000         0&#xA;           0         0         0         0         0         0         0         0  100.0000 -200.0000  100.0000&#xA;           0         0         0         0         0         0         0         0         0  100.0000 -100.0000&#xA;&#xA;&#xA;  F =&#xA;&#xA;    -19.7392&#xA;    -31.9387&#xA;    -12.1995&#xA;     12.1995&#xA;     31.9387&#xA;     39.4784&#xA;     31.9387&#xA;     12.1995&#xA;    -12.1995&#xA;    -31.9387&#xA;    -19.7392&#xA;&#xA;&#xA;  meanF =&#xA;&#xA;   -9.6892e-016&#xA;&#xA;  Warning: Matrix is singular to working precision.&#xA;  &amp;gt; In main at 121&#xA;&#xA;  u =&#xA;&#xA;     NaN&#xA;     NaN&#xA;     NaN&#xA;     NaN&#xA;     NaN&#xA;     NaN&#xA;     NaN&#xA;     NaN&#xA;     NaN&#xA;    -Inf&#xA;    -Inf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is why is this not working? I thought that removing the mean of the RHS would work as suggested in @Sumedh Joshi's comment. Any help is greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found out that this same exact problem setup works perfectly fine for 12 unknowns (and 15 and much larger, e.g. 100), but does not work for 10 unknowns (as posted). So I suppose that the problem is setup correctly. This still begs the question though, what is going on here? It seems that this may be more of a question regarding numerical analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the matlab results for 12 unknowns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; A =&#xA; 0         0         0         0         0         0         0         0         0         0         0         0         0         0         0&#xA; 0   -1.0    1.0         0         0         0         0         0         0         0         0         0         0         0         0&#xA; 0    1.0   -2.0    1.0         0         0         0         0         0         0         0         0         0         0         0&#xA; 0         0    1.0   -2.0    1.0         0         0         0         0         0         0         0         0         0         0&#xA; 0         0         0    1.0   -2.0    1.0         0         0         0         0         0         0         0         0         0&#xA; 0         0         0         0    1.0   -2.0    1.0         0         0         0         0         0         0         0         0&#xA; 0         0         0         0         0    1.0   -2.0    1.0         0         0         0         0         0         0         0&#xA; 0         0         0         0         0         0    1.0   -2.0    1.0         0         0         0         0         0         0&#xA; 0         0         0         0         0         0         0    1.0   -2.0    1.0         0         0         0         0         0&#xA; 0         0         0         0         0         0         0         0    1.0   -2.0    1.0         0         0         0         0&#xA; 0         0         0         0         0         0         0         0         0    1.0   -2.0    1.0         0         0         0&#xA; 0         0         0         0         0         0         0         0         0         0    1.0   -2.0    1.0         0         0&#xA; 0         0         0         0         0         0         0         0         0         0         0    1.0   -2.0    1.0         0&#xA; 0         0         0         0         0         0         0         0         0         0         0         0    1.0   -1.0         0&#xA; 0         0         0         0         0         0         0         0         0         0         0         0         0         0         0&#xA;&#xA; f =&#xA;&#xA;        0&#xA; -19.7392&#xA; -34.1893&#xA; -19.7392&#xA;  -0.0000&#xA;  19.7392&#xA;  34.1893&#xA;  39.4784&#xA;  34.1893&#xA;  19.7392&#xA;   0.0000&#xA; -19.7392&#xA; -34.1893&#xA; -19.7392&#xA;        0&#xA;&#xA; mean(f)&#xA; ans =&#xA; -9.4739e-016&#xA;&#xA; A(2:end-1,2:end-1)\f(2:end-1)&#xA; ans =&#xA;  0.9167&#xA;  0.7796&#xA;  0.4051&#xA; -0.1065&#xA; -0.6181&#xA; -0.9926&#xA; -1.1297&#xA; -0.9926&#xA; -0.6181&#xA; -0.1065&#xA;  0.4051&#xA;  0.7796&#xA;  0.9167&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2019-12-19T19:35:35.173" Title="Poisson equation finite-difference with pure Neumann boundary conditions" Tags="|finite-difference|boundary-conditions|poisson|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21613" PostTypeId="2" ParentId="21611" CreationDate="2015-12-18T19:57:16.337" Score="4" Body="&lt;p&gt;Are the entries in your $A$ matrix actually real or just integer?  If they are real numbers, how precisely do you need to satisfy the constraint?    Are the entries of $A$ all nonnegative, or could some of the entries be negative?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem is in general NP-Hard, so you shouldn't expect to find a polynomial time algorithm.  You may find that you can do better than checking all $2^{n}$ combinations by using back tracking search.  I'd suggest looking at constraint programming solvers such as Eclipse to do this backtracking search.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try to approach this using integer linear programming software, but most codes for ILP are not setup to find all solutions but rather stop after finding a single optimal solution even if there may be many more solutions.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-12-18T19:57:16.337" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21614" PostTypeId="1" AcceptedAnswerId="21616" CreationDate="2015-12-18T20:16:04.630" Score="9" ViewCount="1010" Body="&lt;p&gt;I was taught in my freshman college computer programming course that using global variables is almost always a bad idea. However, I have found that when designing Fortran programs for very complex simulations, such a practice seems to make things easier; for instance, I never have to worry about passing variables to various subroutines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is am I missing something? Will my bad programming practice eventually come back to harm me?&lt;/p&gt;&#xA;" OwnerUserId="18619" LastActivityDate="2015-12-22T23:03:01.810" Title="Using global variables when doing scientific computing" Tags="|programming-paradigms|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21615" PostTypeId="2" ParentId="21602" CreationDate="2015-12-18T21:23:01.300" Score="2" Body="&lt;p&gt;Similar to comments from Jesse Chan, you could consider breaking up the domain into rectangular (prisms in 3-D) shapes. Then, one option could be to pre-compute a spatial hash table with the finite elements by using a 2-D (or 3-D) coordinate to generate the key. You could then create the table by hashing the vertices of each element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To efficiently find what element to do computation in, you could hash the coordinates of your arbitrary point $p$ and then loop through the elements in the resulting hash table bucket until you find the element you need to do local computation in. If you build the hash function well, you could end up with very quick look ups.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-12-18T21:23:01.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21616" PostTypeId="2" ParentId="21614" CreationDate="2015-12-18T23:37:15.190" Score="11" Body="&lt;p&gt;Benefits of organizing your code to avoid global variables include better control over how variables change.  Nothing worse than hunting down how a global variable is erroneously getting changed or stepped on.  Especially important for large complex systems which pull in libraries or are published as a library for use in other people's code.  Imagine the nightmare of you pulling in a library which has global variables with same names as your global variables ... yikes !!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possible solutions include creating an enclosing construct (language specific :  objects, classes or structs come to mind) to store these variables and to only allow changes to these variables by making calls to the construct's exposed functions.  &lt;/p&gt;&#xA;" OwnerUserId="1399" LastEditorUserId="1399" LastEditDate="2015-12-19T18:56:44.067" LastActivityDate="2015-12-19T18:56:44.067" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21617" PostTypeId="2" ParentId="21614" CreationDate="2015-12-19T03:49:48.613" Score="4" Body="&lt;p&gt;You'll almost certainly get bitten unless they're read-only or unless your code is trivial, in which case it probably wouldn't be scientific computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't try to save typing when you're programming; try to save development and debugging time.&lt;/p&gt;&#xA;" OwnerUserId="4466" LastActivityDate="2015-12-19T03:49:48.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21618" PostTypeId="1" CreationDate="2015-12-19T04:35:55.200" Score="1" ViewCount="153" Body="&lt;p&gt;I know that for a dependent-mesh norm Nitsche' method for boundary conditions is coercive. But how to prove this for the usual norms for $H^1$?&lt;/p&gt;&#xA;" OwnerUserId="18398" LastEditorUserId="6775" LastEditDate="2017-06-10T21:48:31.050" LastActivityDate="2017-06-10T21:48:31.050" Title="Nitsche' method coercivity again" Tags="|numerical-analysis|nitsche-method|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21620" PostTypeId="2" ParentId="21610" CreationDate="2015-12-19T10:47:56.190" Score="5" Body="&lt;p&gt;This is an interesting question.  There is a good reason that no implementation exists using a fixed step size and error control.  &lt;strong&gt;There is no guarantee that you will be able to satisfy a prescribed local error tolerance&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By reducing the step size, you reduce the size of &lt;em&gt;all&lt;/em&gt; terms in the error expansion.  A particular reduction in step size still might not decrease the error (i.e., the sum of all those terms) because of some &quot;lucky/unlucky&quot; cancellations.  However, it's easy to see that &lt;strong&gt;by reducing the step size &lt;em&gt;enough&lt;/em&gt;, you can satisfy any error tolerance&lt;/strong&gt; (at least, in the absence of roundoff).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now what if the step size is fixed, but the order can be increased?  By increasing the order, you guarantee that some of the (previously non-zero) error terms vanish.  However, you also change the constants multiplying all the remaining terms.  In general, some of those constants will be larger and others will be smaller.  Thus &lt;strong&gt;there is no guarantee that the higher-order method will have a smaller error&lt;/strong&gt;.  It most often will, but sometimes the error will be larger.  In fact, if you look closely at state-of-the-art extrapolation solvers, you will find that they account for this possibility by sometimes accepting a solution of lower order than the maximum that has already been computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The danger is that you might increase the order indefinitely without reaching the error tolerance.  The only way to ensure that you will avoid this would be to have a sequence of methods in which &lt;strong&gt;all&lt;/strong&gt; the error constants of the remaining higher-order terms decrease as you increase the order.  Whether such a sequence exists is an open question.  Certainly Gragg-Bulirsch-Stoer extrapolation does not have this property.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using an arbitrary sequence of methods of increasing order (including GBS extrapolation), you would probably find that your approach works most of the time (especially for simpler problems), but occasionally fails completely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One last note: in a real implementation, what matters (for purposes of being able to complete an integration) is not the actual local error, but your estimate of it.  So you might instead want to find a sequence of estimators with the property mentioned above.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2015-12-19T10:47:56.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21622" PostTypeId="2" ParentId="21607" CreationDate="2015-12-20T11:09:37.743" Score="4" Body="&lt;p&gt;What about &#xA;$$ F(x,y) = \frac{x^2}{y} + \frac{\epsilon}{x^2}, \qquad \nabla F(x,y) = \Big( \frac{2x}{y}-\frac{2\epsilon}{x^3}, -\frac{x^2}{y^2} \Big), \qquad x,y&amp;gt;0$$&#xA;with the sequence $(x_k,y_k) = (k, k^2)$? Here $\epsilon$ is a small positive real number that makes the Hessian positive definite instead of semidefinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then $F(x_k,y_k) = 1+\epsilon k^{-2}$ does not converge to the global infimum $\inf F(x,y) = 0$, but the sequence of gradients $$\mathbf{g}_k = \Big(\frac2k-\frac{2\epsilon}{k^3}, -\frac{1}{k^2}\Big)$$ converges to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Informally speaking, the usual proof of convergence based on the inequality $f(y)\geq f(x) + \nabla f(x)^T(y-x)$ would not apply here because while the gradients converge as $k^{-1}$ (not fast enough), the distance between $x_k$ and the closest minimum at $x=0$ diverges as $k$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2015-12-22T06:33:08.630" LastActivityDate="2015-12-22T06:33:08.630" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21623" PostTypeId="1" CreationDate="2015-12-20T11:33:40.247" Score="9" ViewCount="1270" Body="&lt;p&gt;I am looking for a reliable library where I can find &lt;a href=&quot;https://en.wikipedia.org/wiki/Polylogarithm&quot;&gt;polylogarithm&lt;/a&gt; function and newton/secant method for solving non-linear equations. Basically I have something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(x) = x - A*PolyLog(3/2, B*Exp(-t*x))&#xA;f(x) = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(x,y,z) = 0&#xA;g(x,y,z) = 0&#xA;h(x,y,z) = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;involving polylogarithm function. Implementation of Polylogarithm function need to be similar to that of Mathematica or Python (can return complex values) and defined for non-integer value (here we have 3/2). So, Newton method should also work for complex numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The library should be written in &lt;strong&gt;Fortran&lt;/strong&gt; or &lt;strong&gt;C/C++&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14797" LastActivityDate="2016-11-01T17:15:26.867" Title="Library with polylogarithm function" Tags="|c++|fortran|c|libraries|" AnswerCount="2" CommentCount="0" ClosedDate="2017-06-27T15:34:59.930" ContentLicense="CC BY-SA 3.0" />
  <row Id="21625" PostTypeId="1" AcceptedAnswerId="21627" CreationDate="2015-12-21T03:06:20.407" Score="2" ViewCount="185" Body="&lt;p&gt;I work with scientific simulation, but what I like the most in programming is creating graphical user interfaces and 2D or 3D outputs to visualize the evolution of the simulations. I know most input data in scientific simulation are in text files, not in GUIs, and that more usually statistical graphs are scientifically more relevant than 3D movies. But I still can't accept I have to give up on my esteem for GUIs and 3D in science...  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think I should adopt the model-view-controller paradigm in my programs? Can I separate high-performance calculation from visualization using multithread, such that crashes in visualization don't affect the calculations? And sucht that I can switch from text input files to GUIs and back again to my whim. Which software do you recommend?&lt;/p&gt;&#xA;" OwnerUserId="18469" LastActivityDate="2016-01-01T16:53:19.630" Title="Is model-view-controller useful pattern useful to build scientific simulation programs?" Tags="|simulation|3d|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21627" PostTypeId="2" ParentId="21625" CreationDate="2015-12-21T11:54:27.063" Score="4" Body="&lt;p&gt;This answer is my personal opinion. I believe that high performance numerical solvers are not a good place to use MVC pattern. Additional layers and specific data flow will not help to get a better performance. MVC and MVVM (Model-View-ViewModel) are commonly used for GUI developing. If you want to develop pre/post processor where performance is not such important thing, architectural patterns can help you organize a project and make it easier to develop (especially if you work in a team). I believe that GUI for numerical software should be always separated from computational core. Not only by using multi-threading, but it should be located in a separate program. That kind of architecture is commonly found in many commercial numerical softwares like Abaqus or Ansys. If GUI crashes it will not to affect calculations. Solver can run separately on HPC machine where graphical interface is not available. Moreover, GUI can be written using almost any language which provide suitable sets of widgets - Java, C#, or C++ (QT). It depends on target platform and your skills in programming. You can even use some general purpose post/pre processors like GMSH, Paraview or GiD and combine it with your solver. High performance numerical solvers are almost always written in some low level languages like C, C++, Fortran which do not support MVC/MVVM patterns in the same way how Java or C# does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of my PHD thesis I decided to write my own preprocessor using C# and MVVM pattern and separated numerical solver using C++ which consumes input file generated by preprocessor. Such architecture was easier to develop and maintain.&lt;/p&gt;&#xA;" OwnerUserId="1010" LastEditorUserId="1010" LastEditDate="2015-12-21T13:25:34.430" LastActivityDate="2015-12-21T13:25:34.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21628" PostTypeId="1" CreationDate="2015-12-21T16:39:50.390" Score="1" ViewCount="227" Body="&lt;p&gt;I'm currently working on cubic splines interpolation. My question has to do with the end conditions. Instead of using natural or clamped cubic splines, I am asked to use the following condition :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S_{1} = S_{2} $ and $S_{n} = S_{n-1} $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I am not pretty sure how this is applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone help me clear this out?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am editting due to the first comment. On which points do I appply this condition? $S_{1}(x_{2}) = S_{2}(x_{2}) $ is a condition that I've already used and $S_{1}(x_{1}) = S_{2}(x_{2}) $ leads to a contradiction...So is it something like $S_{1}(x_{1}) = S_{2}(x_{1}) $ ?&lt;/p&gt;&#xA;" OwnerUserId="18643" LastEditorUserId="18652" LastEditDate="2015-12-23T18:18:55.413" LastActivityDate="2015-12-23T18:18:55.413" Title="End conditions on cubic splines interpolation" Tags="|numerical-analysis|interpolation|b-spline|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21629" PostTypeId="1" AcceptedAnswerId="21637" CreationDate="2015-12-21T20:38:23.333" Score="3" ViewCount="277" Body="&lt;p&gt;Say I have a random number generator that generates a number within &lt;code&gt;[0, RAND_MAX]&lt;/code&gt;, and &lt;code&gt;RAND_MAX &amp;lt; UINT_MAX&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I generate a random number within &lt;code&gt;[0, i]&lt;/code&gt; such that &lt;code&gt;i&amp;gt;RAND_MAX&lt;/code&gt; and &lt;code&gt;i&amp;lt;UINT_MAX&lt;/code&gt;, while maintaining an even distribution, and without exceeding &lt;code&gt;UINT_MAX&lt;/code&gt; in any calculations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first attempt is to split the range into &lt;code&gt;n&lt;/code&gt; equal parts such that &lt;code&gt;i mod n == 0&lt;/code&gt; by finding the least denominator of &lt;code&gt;i&lt;/code&gt; (using one of the integer factorization sieve algorithms). Each sublist has equal distribution, any can be picked at random. This fails if &lt;code&gt;i&lt;/code&gt; is prime:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 1 2 3 4 5 6&#xA;|___| |_____|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The second attempt is to split the range into two parts &lt;code&gt;a = [0, floor(i/2)]&lt;/code&gt; and &lt;code&gt;b = [floor(i/2)+1, i]&lt;/code&gt;, and then to calculate the distribution as the ratio of the length of the two sub-ranges:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g = gcd(length(a), length(b))&#xA;if random(0, length(a)/g + length(b)/g - 1) &amp;gt;= length(a)/g&#xA;    random(0, floor(i/2))&#xA;else&#xA;    random(floor(i/2)+1, i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Given the following range, the right sub-range is selected twice as often as the left sub-range:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 1 2 3 4 5&#xA;|_| |_____|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately the first call to &lt;code&gt;random()&lt;/code&gt; may recurse forever if &lt;code&gt;gcd()&lt;/code&gt; is ever one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final attempt is to correct the distribution skew introduced by splitting a range of odd length:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if random(1)&#xA;    if random(1)&#xA;        random(0, floor(i/2))&#xA;    else&#xA;        random(floor(i/2) + 1, i)&#xA;else&#xA;    if random(1)&#xA;        random(0, floor(i/2) - 1)&#xA;    else&#xA;        random(floor(i/2), i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which probably doesn't work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For anyone with similar question, the sum of two uniform variables follows a triangular distribution (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Irwin%E2%80%93Hall_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Irwin-Hall Distribution&lt;/a&gt;). This is invalid:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# triangular distribution:&#xA;random(a, floor(b/2)) + random(a, b - floor(b/2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit (on accept):&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two answers I considered are both of high quality. &lt;a href=&quot;https://scicomp.stackexchange.com/a/21630/18645&quot;&gt;This answer&lt;/a&gt; considers RNGs on a per-digit or per-bit basis. &lt;a href=&quot;https://scicomp.stackexchange.com/a/21637/18645&quot;&gt;The accepted answers&lt;/a&gt; consider RNGs numerically, without considering digits or bits. In both cases, the comments are important to read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also posted a follow-up question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21657/maintain-uniform-distribution-across-subranges&quot;&gt;Maintain Uniform Distribution across Subranges&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18645" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-12-24T21:11:53.230" Title="Generate Random Number outside Bounds:" Tags="|algorithms|random-number-generation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21630" PostTypeId="2" ParentId="21629" CreationDate="2015-12-21T21:38:38.543" Score="2" Body="&lt;p&gt;Instead of fixing the provided generator use a reasonable modern choice. It will be faster and have better statistical quality.  Possible examples include the various variants of xorshift+, xorshift* and PCG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To directly respond to the question asked. You can generate a sample, mask out the maximal power of two bits available, generate another and shift that and bit or in the previous.  For a non power of two 'n' perform a rejection method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT 2:&#xA;So using a 64-bit xorshift+ to generate a 64-bit uniform sequence might look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// multiple 'state' blocks to allow for friendly multi-threading&#xA;typedef struct {&#xA;  uint64_t s0;&#xA;  uint64_t s1;&#xA;} rng_state;&#xA;&#xA;// get 64-bits&#xA;inline uint64_t rng_next(rng_state_t* s)&#xA;{&#xA;  uint64_t s1 = s-&amp;gt;s0;&#xA;  uint64_t s0 = s-&amp;gt;s1;&#xA;&#xA;  s-&amp;gt;s0 = s0;&#xA;  s1   ^= s1 &amp;lt;&amp;lt; 23;&#xA;  s-&amp;gt;s1 = s1 ^ s0 ^ (s1 &amp;gt;&amp;gt; 18) ^ (s0 &amp;gt;&amp;gt; 5);&#xA;&#xA;  return s-&amp;gt;s1 + s0; &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Trying to &quot;patch-up&quot; random, given simplifying assumptions that it returns a uniform an N-bit number and extending it to 2N-bits:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inline uint64_t rng_next() {&#xA;   uint64_t r0 = random();&#xA;   return (r0 &amp;lt;&amp;lt; RANDOM_BITS) | random();&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Taking a quick peek at source it looks like the current version of (say) glibc will return 31-bits and by default it uses a power-of-two LCG (very low quality).  The end result of patching up is significantly lower quality and significantly higher runtime cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: All code is typed in post and probably doesn't even compile.&lt;/p&gt;&#xA;" OwnerUserId="18646" LastEditorUserId="18646" LastEditDate="2015-12-22T09:34:12.370" LastActivityDate="2015-12-22T09:34:12.370" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21631" PostTypeId="2" ParentId="19586" CreationDate="2015-12-21T21:58:16.007" Score="24" Body="&lt;p&gt;&lt;a href=&quot;https://joblib.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;Joblib&lt;/a&gt; does what you want.&#xA;The basic usage pattern is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from joblib import Parallel, delayed&#xA;&#xA;def myfun(arg):&#xA;     do_stuff&#xA;     return result&#xA;&#xA;results = Parallel(n_jobs=-1, verbose=verbosity_level, backend=&quot;threading&quot;)(&#xA;             map(delayed(myfun), arg_instances))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;arg_instances&lt;/code&gt; is list of values for which &lt;code&gt;myfun&lt;/code&gt; is computed in parallel.&#xA;The main restriction is that &lt;code&gt;myfun&lt;/code&gt; must be a toplevel function.&#xA;The &lt;code&gt;backend&lt;/code&gt; parameter can be either &lt;code&gt;&quot;threading&quot;&lt;/code&gt; or &lt;code&gt;&quot;multiprocessing&quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can pass additional common parameters to the parallelized function. The body of &lt;code&gt;myfun&lt;/code&gt; can also refer to initialized global variables, the values which will be available to the children.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Args and results can be pretty much anything with the threading backend but results need to be serializable with the multiprocessing backend.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dask.org/&quot; rel=&quot;noreferrer&quot;&gt;Dask&lt;/a&gt; also offers similar functionality. It might be preferable if you are working with out of core data or you are trying to parallelize more complex computations.&lt;/p&gt;&#xA;" OwnerUserId="4000" LastEditorUserId="414" LastEditDate="2019-09-03T10:34:06.227" LastActivityDate="2019-09-03T10:34:06.227" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21632" PostTypeId="1" AcceptedAnswerId="21639" CreationDate="2015-12-22T02:26:57.673" Score="2" ViewCount="232" Body="&lt;p&gt;I'm working in a finite element code where I need to calculate&#xA;$$\int_F b_F$$&#xA;where $F$ is a face of some tetrahedron $K$ in the mesh, and $b_F$ is the usual bubble function defined by&#xA;$$b_F=9\lambda_1\lambda_2\lambda_3$$&#xA;where $\lambda_i$ are the usual barycentric coordinates. Note that $\textrm{supp}(b_F)=K_+\cup K_-$, where $K_+$ and $K_-$ are the thetrahedron sharing the face $F$.&#xA;Where (book o paper) can I find this value?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For example, in 2D (where $K$ is a triangle and $F$ some edge) $b_F=4\lambda_1\lambda_2$ and&#xA;$$\displaystyle\int_F b_F=\frac{2}{3}\,|h_F|$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I need the result when $K$ is a tetrahedron.&lt;/p&gt;&#xA;" OwnerUserId="17630" LastEditorUserId="17630" LastEditDate="2015-12-22T16:17:06.147" LastActivityDate="2015-12-22T22:44:02.040" Title="Integrate over a face of a bubble function" Tags="|finite-element|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21633" PostTypeId="1" CreationDate="2015-12-22T04:18:48.907" Score="2" ViewCount="1213" Body="&lt;p&gt;I'm using power iteration to find the dominant right eigenvector of some large-ish matrices ($1000\times 1000$ to $10000\times 10000$ or so, maybe I'll need to go bigger later) with non-negative elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to know both the left and right eigenvectors corresponding to the largest eigenvalue. Obviously I can find the left eigenvector by doing power iteration on the transpose of the matrix. However, this is quite expensive, and since I already did power iteration to find the right eigenvector and the leading eigenvalue, it seems like there might be a way to use that information to compute the left eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrices are real and have only non-negative elements, but beyond that they have no special properties, e.g. they are not symmetric and they are not stochastic matrices. The leading left and right eigenvectors have all positive elements, and I want to guarantee that numerical errors will not introduce negative elements. (This is why I'm using power iteration rather than any other method to find the leading eigenvector.) Thus I would like to avoid any technique that involves numerically inverting the matrix or anything similar to that, unless this guarantee can be kept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Python/numpy but I'm not attached to it - I'd rather focus on what algorithm I should be using.&lt;/p&gt;&#xA;" OwnerUserId="3746" LastActivityDate="2015-12-22T05:58:27.417" Title="Calculating left eigenvector when I know the right eigenvector" Tags="|linear-algebra|eigenvalues|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21634" PostTypeId="2" ParentId="21633" CreationDate="2015-12-22T05:58:27.417" Score="4" Body="&lt;p&gt;Keep in mind that the Perron-Frobenius theorem applies to non-negative matrices only in the special case that the matrix is irreducible (i.e. some power of $A$ has strictly positive elements.) You need to be sure that your nonnegative matrices are irreducible.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furtheromre, in general you may have multiple complex eigenvalues with the same absolute value as the largest real eigenvalue- the power method isn't guaranteed to converge in this case.  There are specialized methods for finding the right eigenvector corresponding to the dominant real eigenvalue that are guaranteed to converge- you should be using one of those methods.  Some methods compute the left and right dominant eigenvectors at the same time, which would eliminate your problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that you do know the right eigenvector corresponding to the dominant real eigenvalue, then because you know the dominant eigenvalue, finding the left eigenvector corresponding to that eigenvalue is just a matter of solving a system of equations (the equations for a left eigenvector plus an extra equation to normalize the eigenvector to say $\| x \|_{1}=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Perron-Frobenious theorem (for irreducible nonnegative matrices) ensures that there will be a unique (up to normalization) real and positive left eigenvector corresponding to the dominant real eigenvalue.  Thus you should be able to just solve that linear system of equations and be done with it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a practical matter, you might get slightly negative values in your left eigenvector after solving this system of equations.  One option to enforce nonnegativity would be to apply an algorithm for nonnegative least squares (e.g. an active set method such as Parker and Stark's BVLS) to get a nonnegative least squares solution to that linear system.  You'd want to check that this solution was still reasonably close to being a left eigenvector of the original matrix.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't said anything about the sparsity of your $A$ matrix.  This could have a huge impact on whether finding the left eigenvector by some iterative scheme or solving the linear system of equations is faster.      &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2015-12-22T05:58:27.417" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21635" PostTypeId="1" CreationDate="2015-12-22T10:35:08.170" Score="9" ViewCount="695" Body="&lt;p&gt;I am running molecular dynamics simulations of water for testing purposes. The box is quite small, if you ask a guy running classical MD, and relatively large, if you ask a DFT guy: I have 58 water molecules in periodic boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To save CPU time, I am optimizing my cell with a classical force field before running the ab initio MD. I equilibrate the system classically at 300K for 1 ns, then take the last snapshot and use it as input for ab initio MD. My ab initio MD is regular DFT-based Born-Oppenheimer MD with a plane wave basis set and PAW (pseudo)potentials (VASP is the code). In both classical and ab initio simulations I am keeping the temperature constant at 300K using a velocity-rescaling thermostat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am surveying two different ways to make the transition between classical and ab initio:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take the initial velocities and positions from the classical trajectory and import them as initial configuration for the ab initio simulation&lt;/li&gt;&#xA;&lt;li&gt;Freeze the system to zero temperature keeping the classical positions, import that to the DFT code, then quickly (I'm doing it in 0.5 ps at the moment) heat up to 300K&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I was hoping that both strategies would lead to the same average energy after a short (say 10 ps) equilibration period, especially considering that the starting configuration is exactly the same (same initial positions) except for the mentioned temperature trick (initial velocities differ). This is not the case. The figure below shows that the simulation where the system is frozen and then quickly heated up finds an energy region about 1 eV lower in energy than the other one, where the velocities where imported from classical MD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KhhRZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KhhRZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;whether this is to be expected;&lt;/li&gt;&#xA;&lt;li&gt;are there known successful strategies to optimize the transition from classical to ab initio MD;&lt;/li&gt;&#xA;&lt;li&gt;and could you point me towards pertinent literature on the matter?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been running some more tests and -with the limited data I have at the moment- it seems that this might be a system-specific problem. A test with methanol instead of water in a box of the same size showed that the two different initial velocity schemes quickly converge to the same average energy. However, the classical configuration was very close to the quantum one in the case of methanol, that is, the energy at t = 0 was very close to the average energy after convergence. Water is a notoriously difficult system, so perhaps this problem is more or less water-specific. If no answers are added I will try and post one based on my results once I'm done with all the testing.&lt;/p&gt;&#xA;" OwnerUserId="18559" LastEditorUserId="18559" LastEditDate="2016-01-16T08:12:04.747" LastActivityDate="2016-04-04T18:40:40.073" Title="Ways to start ab initio MD from classical MD" Tags="|molecular-dynamics|density-functional-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21636" PostTypeId="2" ParentId="19586" CreationDate="2015-12-22T13:52:46.050" Score="16" Body="&lt;p&gt;What you're looking for is &lt;a href=&quot;http://numba.pydata.org/&quot; rel=&quot;noreferrer&quot; title=&quot;Numba&quot;&gt;Numba&lt;/a&gt;, which can auto parallelize a for loop.&#xA;From their &lt;a href=&quot;http://numba.pydata.org/numba-doc/latest/user/parallel.html#explicit-parallel-loops&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numba import jit, prange&#xA;&#xA;@jit&#xA;def parallel_sum(A):&#xA;    sum = 0.0&#xA;    for i in prange(A.shape[0]):&#xA;        sum += A[i]&#xA;&#xA;    return sum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="7522" LastEditDate="2019-09-03T09:35:18.060" LastActivityDate="2019-09-03T09:35:18.060" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21637" PostTypeId="2" ParentId="21629" CreationDate="2015-12-22T17:13:48.860" Score="2" Body="&lt;p&gt;For the sake of your example, let's say RAND_MAX=12 and i=17. Then do the following procedure: Choose two random numbers $r_1,r_2$ and combine them to a single random number uniformly distributed in $[0,144)$ by computing $r=r_1*12+r_2$. This is of course the wrong interval. You get a uniformly distributed random number in $[0,17)$ by repeating this process until you end up with a number $r$ in this interval; in other words, every time you get a random number $r\ge i$, you discard it and just try again. This guarantees that you end up with a uniform random number.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2015-12-24T02:39:02.300" LastActivityDate="2015-12-24T02:39:02.300" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="21638" PostTypeId="2" ParentId="21614" CreationDate="2015-12-22T17:27:01.673" Score="4" Body="&lt;p&gt;Your view is colored by the fact that you are using Fortran (I assume Fortran 77) where you have only two ways to access variables: (1) they are global variables in some COMMON block, or (2) they have been passed as arguments (typically scalar) to the current function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since 1977, this has been addressed in many ways in all more modern programming languages. An example is the use of structures in C/C++ (composite types or records in other languages) that allow you to group many variables into one and only pass a reference to one structure to a function as an argument. Or, more elegantly in most object oriented programming languages, to group things together in classes and then have a reference or pointer to such a class object passed &lt;em&gt;implicitly&lt;/em&gt; to every member function. These techniques have made dealing with the problem you see unimportant and eliminate the incentive to use global variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reasons not to use global variables have already been given in other answers. My personal favorite among the reasons is that one of the best strategies for reducing the potential for bugs is to &lt;em&gt;limit the scope&lt;/em&gt; of variables. A variable that lives only in a very small part of the program can not create trouble elsewhere. So my goal is often to declare any variable as late as possible and destroy it as early as possible, simply because that makes it so much easier to read and understand code. On the other hand, global variables violate this goal in the most egregious possible way.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-22T17:27:01.673" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21639" PostTypeId="2" ParentId="21632" CreationDate="2015-12-22T17:27:56.440" Score="3" Body="&lt;p&gt;I'll assume the triangular face of your tetrahedron has straight edges since &#xA;that was the case with the edges of the triangle in your 2D example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a straight-edged triangle, there is a simple closed-form expression for&#xA;integrating a polynomial over the triangle. That expression is given, for&#xA;example, in equation 15.26 of this reference:&#xA;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch15.d/IFEM.Ch15.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch15.d/IFEM.Ch15.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That equation, using the notation of the listed reference is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;{\frac{1}{2A}}\int_{\Omega^e}\zeta^i_1\zeta^j_2\zeta^k_3 d\Omega = {\frac{i!j!k!}{(i+j+k+2)!}}, i\ge0, j\ge0, k\ge0.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of defining the triangular, barycentric coordinates as $\lambda_i$, this&#xA;reference uses $\zeta_i$. For your function, $9\lambda_1\lambda_2\lambda_3$, &#xA;i, j, and k in equation 15.26 are all one. So, by simple substitution, your&#xA;integral will be equal $3A/20$ where $A$ is the area of the triangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, there is also a closed-form expression for the integrals of&#xA;a polynomial over a tetrahedron should you need to perform those. It can be&#xA;found as equation 9.21 of reference:&#xA;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch09.d/AFEM.Ch09.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch09.d/AFEM.Ch09.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2015-12-22T22:44:02.040" LastActivityDate="2015-12-22T22:44:02.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21640" PostTypeId="1" AcceptedAnswerId="21648" CreationDate="2015-12-22T19:54:53.127" Score="2" ViewCount="78" Body="&lt;p&gt;When simulation atoms, molecules, colloids, autc., are there programs that define each interacting unity as an object in the OOP sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my own case, I've been simulating magnetic nanoparticles in colloidal dispersion in Fortran, but I have many arrays that are unrelated in the code for each attribute of the nanoparticles. For instance, for 100 nanoparticles, I would have the arrays r(3,100) for 3D position, m(3,100) for 3D magnetic moments, xi(100) for adsorbed molecule surface density, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started my code using Fortran 77 compilers, so I didn't use the TYPE feature of Fortran 2003 that would allow me to put all those arrays inside the same type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I've been using Java a lot these times, so I wondered if I would better define each magnetic nanoparticle as an object. So I would also define methods &quot;move&quot;, &quot;rotate&quot;, etc. in the nanoparticle class. I am not sure about how I would compute the pair interaction itself. I don't see how to define it as object class methods... I guess I would have to do it in the &quot;controller&quot; layer, not in the &quot;model&quot; layer (using MVC paradigm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only hesitate in making this new program because I suspect that defining my nanoparticles as objects would make simulations a lot slower, however beautiful the code would look to me. But if there are other programs that define pair interaction unities that way, there will probably be good reasons to do so and I won't be alone in that... :-)&lt;/p&gt;&#xA;" OwnerUserId="18469" LastEditorUserId="18469" LastEditDate="2015-12-22T20:28:15.227" LastActivityDate="2015-12-23T09:40:41.693" Title="Is it common or to use objects for calculation of pair interactions in physics and chemistry simulations?" Tags="|simulation|fortran|performance|java|molecular-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21641" PostTypeId="1" AcceptedAnswerId="21646" CreationDate="2015-12-22T20:47:36.073" Score="5" ViewCount="1198" Body="&lt;p&gt;As you may know, Fortran is widely and longly (?) used in science. I've heard a lot of people saying that we natural scientists should all migrate to C++ or Java. I've seen people increasingly suggesting also Python, that I thought it was basically a script language like Bash, but which supports floating-point number operations (I have only used Python with that purpose).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, other people I know have been saying that, since Fortran 2003, all the old advantages of languages such as C++ and Java are irrelevant, and that Fortran still has the advantage of being fast, robust and having a lot of code already written for all types of scientific calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another motivation I have to abandon Fortran is that, when I said I used Fortran to some non natural scientists that work with computer, they laughed and asked &quot;Fortran yet?&quot;, but, beside that fact, are there still good reasons to abandon Fortran in science after Fortran 2003? What do C++ and Java have that Fortran don't?&lt;/p&gt;&#xA;" OwnerUserId="18469" LastActivityDate="2015-12-23T09:39:25.723" Title="What do C, C++ and Java have that Fortran 2003 don't?" Tags="|c++|simulation|fortran|c|java|" AnswerCount="3" CommentCount="12" FavoriteCount="0" ClosedDate="2015-12-24T20:43:52.230" ContentLicense="CC BY-SA 3.0" />
  <row Id="21642" PostTypeId="2" ParentId="21641" CreationDate="2015-12-22T22:45:41.017" Score="2" Body="&lt;p&gt;I think this really comes down to what work you are doing with the language. If you are doing numerically computation heavy things, Fortran is a fine language for this and I don't know if C++ or Java would really be advantageous relative to Fortran 2003.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you start diving into applications outside of purely number crunching, like building GUIs, apps, games, back-ends for websites, etc., I think C++ and Java would probably be much better for the job. Part of this is because of all the frameworks and libraries built in those languages to tackle those problems. Now obviously, you could do some number crunching code in Fortran and then build say GUI code in Python/C++/Java and just write a wrapper for the Fortran code to use it and then you could get the best of both worlds. But it just comes down to what's best for the task and constraints at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, that's my 2 cents.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2015-12-22T22:45:41.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21643" PostTypeId="1" CreationDate="2015-12-22T22:59:27.093" Score="0" ViewCount="889" Body="&lt;p&gt;I know that it is possible to determine the theoretical order of accuracy (B) of numerical solutions in FVM (for instance for a steady problem in which only the central differencing scheme (CDS) is used to interpolate, B = 2  since CDS is 2nd order accurate). I know that with FVM the order of accuracy is obtained from the Taylor series. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I interested to know whether there is a method to obtain the convergence rate for FEM solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also particularly interested to find out if there is a way of determining the order of accuracy for DGFEM. I have read in many papers that the convergence rate is normally p+1 where p is degree of the approximating polynomial. However I am uncertain as why this is not effected by the interpolation scheme used for the flux terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="17476" LastEditorUserId="17476" LastEditDate="2015-12-23T13:03:49.327" LastActivityDate="2015-12-23T13:03:49.327" Title="Order of accuracy of DGFEM or FEM" Tags="|finite-element|finite-volume|discontinuous-galerkin|numerical-modelling|accuracy|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21644" PostTypeId="2" ParentId="21614" CreationDate="2015-12-22T23:03:01.810" Score="1" Body="&lt;p&gt;Anything that can be done with a global variable can be done without a global variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as code grows, global variables take on a life of their own and in some very bad cases will totally demodularize the code so that understanding any particular subroutine requires complete understanding of the entire codebase; changes to globals could conceivably happen anywhere at any time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore I think learning to code without globals such that it is instinct and not seen as the harder alternative is the best approach.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2015-12-22T23:03:01.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21645" PostTypeId="2" ParentId="21643" CreationDate="2015-12-22T23:09:59.300" Score="2" Body="&lt;p&gt;This depends on the problem you are solving.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually the elliptic and hyperbolic cases are handled separately because the spectral properties of their operators are very different, and so you have papers/books which answer this question for those specific cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking the expectation is that if you use p-th order polynomials, then DGFEM/FEM will give you between p and p+1 order convergence. The basic strategy for proving this is fairly well understood, but it often needs to be modified when applying it in new situations.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2015-12-22T23:09:59.300" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21646" PostTypeId="2" ParentId="21641" CreationDate="2015-12-22T23:38:06.377" Score="4" Body="&lt;p&gt;One thing that &lt;code&gt;C++&lt;/code&gt; includes that &lt;code&gt;Fortran&lt;/code&gt; doesn't have is extensive support for &lt;a href=&quot;https://en.wikipedia.org/wiki/Template_(C%2B%2B)&quot; rel=&quot;noreferrer&quot;&gt;generic programming&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Template_metaprogramming&quot; rel=&quot;noreferrer&quot;&gt;compile-time code evaluation&lt;/a&gt;. Modern &lt;code&gt;Fortran&lt;/code&gt; allows one to use object-oriented features like &lt;code&gt;C++&lt;/code&gt; (to a degree).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would argue that these generic programming features, along with &lt;code&gt;C++&lt;/code&gt;'s object-oriented model and standard library, allow for the construction of very neat abstract data-structures, algorithms and programming patterns, without any run-time overhead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, if you find yourself using a lot of &lt;em&gt;computer-science&lt;/em&gt; within your scientific programming (i.e binary search trees, hash tables, priority queues, etc, etc) then &lt;code&gt;C++&lt;/code&gt; might be a good choice. The more &lt;em&gt;unstructured&lt;/em&gt; types of scientific computing (those based on unstructured meshes, graphs, networks, sparse matrices, etc, etc) often fall into this type of category. These CS-type language features are a bit of a double-edged sword though -- while some people love them, others hate them with a passion, and much prefer a language like &lt;code&gt;Fortran&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I personally prefer &lt;code&gt;C++&lt;/code&gt; for most projects, I think that &lt;code&gt;Fortran&lt;/code&gt; is a fine language, and the fact that it's old is no reason to think about abandoning it. If you work predominantly with arrays and structured problems it's unlikely that there's a better language choice. At least there's very little incentive to change to a language like &lt;code&gt;C++&lt;/code&gt;!&lt;/p&gt;&#xA;" OwnerUserId="1139" LastActivityDate="2015-12-22T23:38:06.377" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21647" PostTypeId="2" ParentId="21641" CreationDate="2015-12-23T09:39:25.723" Score="1" Body="&lt;p&gt;To expand on Darrens answer a little bit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, Java holds no advantage over Fortran 2003 for Scientific Programming. The syntax is more verbose  (x.add(y) instead of x+y) and the performance is significantly worse. Some of the other JVM languages like Clojure or Scala will at least give you a reasonable syntax to work with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When compared with C++, performance will be about same for numerical workloads. Indeed most Fortran compilers are part of a toolchain also supporting C++, and many (most?) of the optimizations will be the same. &#xA;Like Darren said, C++ supports generic programming, which is implemented in a zero-overhead fashion, meaning you can do advanced abstractions without paying any extra cost. For instance, matrix libraries like Armadillo or Eigen give you a Matlab like syntax, without any performance penalty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main advantage of Fortran is for working with arrays. For pretty much anything else, C++ will be easier, more maintainable and faster.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2015-12-23T09:39:25.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21648" PostTypeId="2" ParentId="21640" CreationDate="2015-12-23T09:40:41.693" Score="2" Body="&lt;p&gt;The problem is the memory layout.&#xA;An object can be viewed in memory like a C struct which contains data and pointers to functions. If you use the OOP, you would have in each object the 3D coordinates, the 3D magnetic moments, and pointers to functions to manipulate this data.&#xA;For instance, if you want to compute the distance between all pairs of particles, you will have to iterate on all the coordinates of the particles. To do this, you will need to iterate over an array of objects. For each object you will have to fetch the 3D coordinates and skip over the rest of the struct (ie other data and eventually pointers to functions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way of using memory is really inefficient for HPC for which memory access is crucial : using consecutive memory is by far more efficient.&#xA;In a few words, you will load a lot of unused data which will badly use your caches, increase your memory bandwidth usage, disable the hardware prefetch possibilities, disable automatic vectorization, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, using functions to access the data in your objects can add some overhead and be an obstacle to automatic vectorization, which is an additional performance loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you are an expert you will take into consideration all these points and write an efficient OO HPC code, but in general the non-OO version will be much more efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to do OO programming, then you should prefer objects containing arrays instead of arrays of objects.&lt;/p&gt;&#xA;" OwnerUserId="17681" LastActivityDate="2015-12-23T09:40:41.693" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21650" PostTypeId="1" AcceptedAnswerId="21658" CreationDate="2015-12-24T00:34:56.220" Score="0" ViewCount="591" Body="&lt;p&gt;I need to devise a algorithm (in Python) that calculates adjacency matrices for the platonic solids. Inputted into the algorythm needs to be the number of polygons meeting at each vertex and the regular polygon on which they are based. Any ideas are welcome as I've tried a few avenues and haven't come up with anything remotely successful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the number of sides &lt;code&gt;n&lt;/code&gt; and let the number of vertexes connected to any particular vertex be &lt;code&gt;m&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a tetrahedron, it has &lt;code&gt;n = 3&lt;/code&gt; and &lt;code&gt;m = 3&lt;/code&gt;. My problem is then going from this to establishing the following adjacency matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 1 1 1&#xA;1 0 1 1&#xA;1 1 0 1&#xA;1 1 1 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is therefore the adjacency matrix for a tetrahedron. With a cube, it becomes more complicated&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 1 1 0 1 0 0 0&#xA;1 0 0 1 0 1 0 0&#xA;1 0 0 1 0 0 1 0&#xA;0 1 1 0 0 0 0 1 &#xA;1 0 0 0 0 1 1 0&#xA;0 1 0 0 1 0 0 1&#xA;0 0 1 0 1 0 0 1 &#xA;0 0 0 1 0 1 1 0     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i need to take the values of n, and m and calculate these matrices. Clearly there are many possible solutions depending on how the algorithm labels the vertexes. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="18667" LastEditorUserId="18667" LastEditDate="2015-12-24T01:09:21.590" LastActivityDate="2015-12-24T21:09:18.050" Title="Calculating adjacency matrix of platonic solids" Tags="|algorithms|matrix|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21652" PostTypeId="2" ParentId="19740" CreationDate="2015-12-24T10:27:38.657" Score="1" Body="&lt;p&gt;After gaining more experience with &lt;a href=&quot;http://asl.org.il&quot; rel=&quot;nofollow&quot;&gt;ASL&lt;/a&gt; I'm convinced that it is the best accelerated open source LBM-solver as of end of 2015. It has following important advantages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;simple C++ API (and only C++ - no need to know OpenCL or Python &lt;strong&gt;even for adding new numerical methods&lt;/strong&gt; (which will automatically become hardware accelerated)!)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://asl.org.il/benchmarks/multicomponent_flow/&quot; rel=&quot;nofollow&quot;&gt;remarkable performance&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;multiphysics: many other physical/chemical phenomena besides CFD (Lattice Boltzmann method is used in ASL)&lt;/li&gt;&#xA;&lt;li&gt;hardware accelerated, can be deployed on CPU/GPU/FPGA/DSP/APU-clusters&lt;/li&gt;&#xA;&lt;li&gt;mesh-free, immersed boundary technology allows to move from CAD directly to simulation (important for automated design optimization).&lt;/li&gt;&#xA;&lt;li&gt;dynamic compilation approach enables an additional layer of &lt;strong&gt;optimization at run-time&lt;/strong&gt; (i.e. for a specific parameters set the application was provided with)&lt;/li&gt;&#xA;&lt;li&gt;open source: AGPL + optional commercial license for closed-source projects.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16442" LastEditorUserId="16442" LastEditDate="2015-12-29T19:38:18.693" LastActivityDate="2015-12-29T19:38:18.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21653" PostTypeId="1" AcceptedAnswerId="21655" CreationDate="2015-12-24T14:48:58.133" Score="7" ViewCount="545" Body="&lt;p&gt;Are there any frameworks that could be classified as &quot;general computational fluid dynamics framework&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm thinking of is a framework that's modular and extensible in order to allow the development of new algorithms pretty much unlimitedly. And scaleability over different computer configurations (laptop - HPC). Do such things exist?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know &lt;a href=&quot;http://fenicsproject.org&quot;&gt;FEniCS&lt;/a&gt;, but my understanding of it is that it's more research oriented (at least now). That is, it doesn't e.g. have a large library or the toolchain for doing e.g. simulation for an engineering design per se.&lt;/p&gt;&#xA;" OwnerUserId="17316" LastActivityDate="2015-12-29T12:36:48.090" Title="Are there any general fluid dynamics frameworks?" Tags="|fluid-dynamics|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21654" PostTypeId="2" ParentId="21653" CreationDate="2015-12-24T17:01:25.767" Score="2" Body="&lt;p&gt;high-order flow solver &lt;a href=&quot;http://www.pyfr.org/&quot; rel=&quot;nofollow&quot;&gt;PyFR&lt;/a&gt; is a GPU based framework written in python created by Imperial College London UK which handles various backends : CUDA, OpenCL OpenMP  ... features :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Governing Equations - Euler, Navier Stokes&lt;/li&gt;&#xA;&lt;li&gt;Dimensionality - 2D, 3D&lt;/li&gt;&#xA;&lt;li&gt;Element Types - Triangles, Quadrilaterals, Hexahedra, Prisms, Tetrahedra, Pyramids&lt;/li&gt;&#xA;&lt;li&gt;Platforms - CPU Clusters, Nvidia GPU Clusters, AMD GPU Clusters&lt;/li&gt;&#xA;&lt;li&gt;Spatial Discretisation - High-Order Flux Reconstruction&lt;/li&gt;&#xA;&lt;li&gt;Temporal Discretisation - Explicit Runge-Kutta&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1399" LastActivityDate="2015-12-24T17:01:25.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21655" PostTypeId="2" ParentId="21653" CreationDate="2015-12-24T20:28:33.217" Score="3" Body="&lt;p&gt;After gaining more experience with &lt;a href=&quot;http://asl.org.il&quot; rel=&quot;nofollow noreferrer&quot;&gt;Advanced Simulation Library&lt;/a&gt; I can recommend it (as I mentioned &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2393/how-can-i-automate-the-process-of-optimizing-the-design-of-a-physical-object/20959#20959&quot;&gt;in another thread&lt;/a&gt;) as a general extensible multiphysics (CFD included) framework&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has following important advantages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;simple C++ API (and only C++ - no need to know OpenCL or Python &lt;strong&gt;even for adding new numerical methods&lt;/strong&gt; (which will automatically become hardware accelerated)!)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://asl.org.il/benchmarks/multicomponent_flow/&quot; rel=&quot;nofollow noreferrer&quot;&gt;remarkable performance&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;multiphysics: many other physical/chemical phenomena besides CFD (Lattice Boltzmann method is used in ASL)&lt;/li&gt;&#xA;&lt;li&gt;hardware accelerated, can be deployed on CPU/GPU/FPGA/DSP/APU-clusters&lt;/li&gt;&#xA;&lt;li&gt;mesh-free, immersed boundary technology allows to move from CAD directly to simulation (important for automated design optimization).&lt;/li&gt;&#xA;&lt;li&gt;dynamic compilation approach enables an additional layer of &lt;strong&gt;optimization at run-time&lt;/strong&gt; (i.e. for a specific parameters set the application was provided with)&lt;/li&gt;&#xA;&lt;li&gt;open source: AGPL + optional commercial license for closed-source projects.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16442" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-12-25T07:59:34.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21657" PostTypeId="1" CreationDate="2015-12-24T21:03:44.983" Score="1" ViewCount="103" Body="&lt;p&gt;&lt;em&gt;Note: this is a continuation of &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21629/generate-random-number-outside-bounds#&quot;&gt;Generate Random Number outside Bounds&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a function (&lt;em&gt;thanks to the previous question&lt;/em&gt;) with the following prototype which returns an integer in the range $[0,b]$, $b \le UINT\_MAX$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;unsigned int randrange(unsigned int b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I want to create a function which returns an integer in the range $[a,b]$, $a \ge INT\_MIN$, $b \le INT\_MAX$, $a \le b$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int randint(int a, int b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can't apply the techniques &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21629/generate-random-number-outside-bounds#&quot;&gt;from the previous question&lt;/a&gt; and also create cross-platform compatible C code. Specifications from &lt;a href=&quot;http://www.iso-9899.info/wiki/The_Standard&quot; rel=&quot;nofollow noreferrer&quot;&gt;the standard&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;an unsigned integer may not necessarily hold a value larger than signed integer (due to padding bits)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$INT\_MAX \le UINT\_MAX$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$\lvert INT \_MIN \rvert$ may exceed UINT_MAX (two's complement)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lvert INT\_MAX \rvert - 1 \le UINT\_MAX$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the range of a signed integer may exceed the range of an unsigned integer (follows from the previous two)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$INT\_MAX - INT\_MIN \le 2 * UINT\_MAX + 1$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;int is the widest type guaranteed by the standard&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$char \le short \le int \le long \le long\ long$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the ranges $[INT\_MIN,-1]$, $[0,INT\_MAX]$ are not guaranteed to be of same length, though they will always be centered around zero&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the ranges for one's complement and sign/magitude are: $[-x,x]$&lt;/li&gt;&#xA;&lt;li&gt;the ranges for two's complement are: $[-(x+1),x]$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;signed integers may implement the following encodings:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;two's complement&lt;/li&gt;&#xA;&lt;li&gt;one's complement&lt;/li&gt;&#xA;&lt;li&gt;sign/magnitude&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;converting an unsigned integer to a signed integer is implementation-defined behaviour if the value of the unsigned integer cannot be represented by the signed integer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, I have two possible approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the &lt;a href=&quot;https://scicomp.stackexchange.com/a/21630/18645&quot;&gt;alternative solution from the linked question&lt;/a&gt;: Generate a particular set of digits/bits rather than to a maximum value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First generate n bits of random information, where n is the number of value bits in a signed integer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(signed int)randrange(INT_MAX);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Generate the sign bit: 0 if positive, 1 if negative. The sign bit must be applied differently depending on the signed integer encoding.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;randrange(1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sign/magnitude has uniform distribution, and therefore is simple to work with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (randrange(1))&#xA;  randombits = -randombits&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One's complement has an extra zero, which should be excluded as it doesn't affect the final range:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (randrange(1)) {&#xA;  // avoid generating negative zero&#xA;  do {&#xA;    randombits = (signed int)randrange(INT_MAX);&#xA;  while (r == INT_MAX);&#xA;  randombits = -randombits&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Two's complement has an extra negative number, which is independent of the even odds of a negative number:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// each number is unique. the odds of a single number appearing in any&#xA;// range, be it [INT_MIN+1,INT_MAX] or [INT_MIN,INT_MAX], are identical&#xA;// (within the range)&#xA;// so compare against any number&#xA;if randombits == 0 {&#xA;  if (randrange(1) {&#xA;    randombits = INT_MIN;&#xA;  }&#xA;} else {&#xA;  // otherwise, even odds for negative&#xA;  randombits = -randombits&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then apply the usual rejection sampling if $randombits &amp;gt; b$ or $randombits &amp;lt; a$, with range-scaling optimizations as described in the original question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach almost seems to good to be true...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a numerical method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Detect when randint's range exceeds the maximum range of randrange:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// outside the bounds of randrange&#xA;if (b &amp;gt;= 0 &amp;amp;&amp;amp; a &amp;lt; 0 &amp;amp;&amp;amp; a + INT_MAX &amp;gt; b) {&#xA;  // following code goes inside this conditional&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Split the range while maintaining uniform distribution. The probability of generating a random number within the bounds of the smaller range is assumed to reflect the size (and probability) ratio of both ranges as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are my assumptions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given two subranges of Z, X and Y, such that $length(X) + length(Y) = length(Z)$, the chance of a random number z from Z appearing in either X or Z is:&#xA;$$P(z\ in\ X) = length(X)/length(Z)$$&#xA;$$P(z\ in\ Y) = length(Y)/length(Z)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the distribution of z in Z is uniform, then to maintain a uniform distribution in both X and Y, only the relative probabilities need to be known:&#xA;$$P(X\ vs\ Y) = length(X)/length(Y)$$&#xA;$$P(Y\ vs\ X) = length(Y)/length(Z)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is good, because length(Z) is incalculable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, instead of calculating a ratio, it can be simulated by a random number r:&#xA;$$P(X\ vs\ Y) = \begin{cases}P(r \ge length(Y)), r = randrange(length(X)) &amp;amp; length(X) \ge length(Y) \\ P(r \le length(Y)), r = randrange(length(Y)) &amp;amp; \text{otherwise} \end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming all that is true, then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Consider (INT_MIN,-1) and (0,INT_MAX)&#xA;// Swap to negative before comparing to avoid overflow&#xA;if (-b &amp;gt; a + 1) {&#xA;  // the positive range is greater&#xA;  if (-randrange(0,b) &amp;lt; a + 1)&#xA;    return randrange(0,b);&#xA;  else&#xA;    return -randrange(1,-a);&#xA;} else if (-b &amp;lt; a + 1) {&#xA;  // the negative range is greater&#xA;  if (randrange(0,-(a+1)) &amp;gt; b)&#xA;    return -randrange(0,-(a+1)) + a;&#xA;  else&#xA;    return randrange(0,b);&#xA;} else {&#xA;  // the ranges are of equal length&#xA;  if (randrange(1))&#xA;    return randrange(0,b);&#xA;  else&#xA;    return randrange(0,b) + a;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This approach also seems to good to be true.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My question: are these approaches valid?&lt;/p&gt;&#xA;" OwnerUserId="18645" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2015-12-28T20:15:01.540" Title="Maintain Uniform Distribution across Subranges" Tags="|algorithms|c|random-number-generation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21658" PostTypeId="2" ParentId="21650" CreationDate="2015-12-24T21:09:18.050" Score="1" Body="&lt;p&gt;As suggested by @hardmath, a simple &lt;a href=&quot;https://en.wikipedia.org/wiki/SageMath&quot; rel=&quot;nofollow&quot;&gt;Sage&lt;/a&gt; solution:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def platonic_solid_adjacency_matrix(n,m):&#xA;    &quot;&quot;&quot;&#xA;    Adjacency matrix of the Platonic solid with `n` sides and in which each vertex has `m` neighbors.&#xA;    &quot;&quot;&quot;&#xA;&#xA;    d = {(6,3):graphs.TetrahedralGraph, (12,3):graphs.HexahedralGraph, &#xA;        (12,4):graphs.OctahedralGraph, (30,3):graphs.DodecahedralGraph, (30,5):graphs.IcosahedralGraph}&#xA;    try:&#xA;        return d[(n,m)]().adjacency_matrix()&#xA;    except KeyError:&#xA;        raise ValueError, &quot;There is no Platonic solid with the supplied parameters&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2015-12-24T21:09:18.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21659" PostTypeId="1" CreationDate="2015-12-24T21:35:17.537" Score="5" ViewCount="3318" Body="&lt;p&gt;The Weierstrass approximation theorem says any continuous function $f(x): [0,1] \to \mathbb{R}$ can be approximated uniformly by polynomials.  Given any $\epsilon$, we can find $p(x) = x^n + \dots $ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ |f(x) - p(x)| &amp;lt; \epsilon $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is always true.  In practice how to do we find $p$?  Let's say $f(x)$ was the step function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) = \left\{ \begin{array}{cc} 0 &amp;amp; x \leq 0 \\ 1 &amp;amp; x &amp;gt; 0\end{array} \right. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we find the polynomial with $\deg p = 100$ which minimizes the tolerance $\epsilon$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \epsilon = \min_{\deg p = 10^2}\left[\max_{x \in [0,1]} | f(x) - p(x) |  \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully I have defined a tractable problem... For example, does &lt;a href=&quot;https://en.wikipedia.org/wiki/Lagrange_polynomial&quot;&gt;Lagrange Interpolation&lt;/a&gt; necessarily minimize $\epsilon$ ?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/d/d9/Dirac_distribution_CDF.svg/512px-Dirac_distribution_CDF.svg.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2015-12-24T22:17:50.640" Title="Approximating a step function with polynomials" Tags="|numerical-analysis|interpolation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21660" PostTypeId="2" ParentId="21659" CreationDate="2015-12-24T22:12:20.407" Score="8" Body="&lt;p&gt;This problem is generally called the &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot;&gt;Minimax Problem&lt;/a&gt;. Unfortunately the step function is not continuous and therefore the Weierstrass approximation theorem does not apply. Any continuous approximation will have $\epsilon \ge 0.5$ since there is a jump of size 1 so the best you can do at that point is split the difference. In fact, $y = 1/2$ is as good as you can do for this problem due to the discontinuity although there are many higher order polynomials that fit better in other norms and just as well in the infinity norm that defines the minimax problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For continuous functions &lt;a href=&quot;https://en.wikipedia.org/wiki/Chebyshev_polynomials&quot;&gt;Chebyshev polynomials&lt;/a&gt; are a good approximation to the minimax solution. There is also a commonly used iterative algorithm call the &lt;a href=&quot;https://en.wikipedia.org/wiki/Remez_algorithm&quot;&gt;Remez algorithm&lt;/a&gt; which approximately solves the minimax problem. The Remez algorithm leverages the &lt;a href=&quot;https://en.wikipedia.org/wiki/Equioscillation_theorem&quot;&gt;equioscillation theorem&lt;/a&gt; which (to paraphrase) says that the polynomial of degree less than or equal to $n$ which has $n+2$ oscillations which are equally above or below the function (i.e. $f(x)\pm C$) on the desired interval is the one which solves the minimax problem for all polynomials of degree less than or equal to $n$. The Remez algorithm iteratively adjusts the coefficients of a polynomial to (approximately) achieve an equioscillating approximation function which is the solution to the minimax problem.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2015-12-24T22:17:50.640" LastActivityDate="2015-12-24T22:17:50.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21661" PostTypeId="2" ParentId="18815" CreationDate="2015-12-24T23:21:29.160" Score="2" Body="&lt;p&gt;Perhaps it would help to decompose $v$ in terms of the (normalized) eigenvectors $w_i$ of the matrix $A$. Let the corresponding eigenvalues be $\lambda_i$, so that $A \cdot w_i = \lambda_i w_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $c_i = &amp;lt;w_i, v&amp;gt;$ then $(I + A + A^2 + ... + A^n) \cdot v = \sum_i c_i (1+\lambda_i + \lambda_i^2 + ... + \lambda_i^n) w_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the eigenvalues that have $|\lambda_i - 1| &amp;gt; \epsilon$ you can use $(1-\lambda^{n+1}_i)/(1-\lambda_i)$ to quickly compute your sum, and for the others you can for example do an explicit summation.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2015-12-24T23:21:29.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21662" PostTypeId="2" ParentId="21657" CreationDate="2015-12-24T23:42:44.087" Score="1" Body="&lt;p&gt;If you have access to floating point numbers, why not generate a $[0, 1)$ random number U and then compute $R = \lfloor a + U (b-a+1)\rfloor$?&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2015-12-24T23:42:44.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21663" PostTypeId="1" CreationDate="2015-12-25T02:08:30.897" Score="3" ViewCount="387" Body="&lt;p&gt;Are there any known issues (such as precision issues) with $\mathsf{MATLAB}$ eig and charpoly functions for large enough $\{-1,0,+1\}$ matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if I change $1$ or $2$ entries between matrices would it be able to find the accurate eigenvalues and characteristic polynomials? I am using size less $40\times40$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My matrices are full rank and symmetric. The condition number of the matrices seems in mid $100$s to $10^{15}$ while absolute value of sum of rows/columns is same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I can get char poly that is good enough for me.&lt;/p&gt;&#xA;" OwnerUserId="7676" LastEditorUserId="7676" LastEditDate="2015-12-25T04:25:15.603" LastActivityDate="2016-08-25T17:27:21.547" Title="Known issues with eigenvalue numerics?" Tags="|matlab|eigenvalues|numerics|precision|numerical-limitations|" AnswerCount="2" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21668" PostTypeId="1" AcceptedAnswerId="21682" CreationDate="2015-12-25T22:39:32.913" Score="1" ViewCount="78" Body="&lt;p&gt;I'm reading a technical report on Simulated Annealing: On the Convergence Time of Simulated Annealing, by Sanguthevar Rajasekaran. You may find it following &lt;a href=&quot;http://repository.upenn.edu/cis_reports/356/?utm_source=repository.upenn.edu%2Fcis_reports%2F356&amp;amp;utm_medium=PDF&amp;amp;utm_campaign=PDFCoverPages&quot; rel=&quot;nofollow&quot;&gt;this link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $G=(V, E)$ is the graph whose vertices are possible solutions to the given problem, and $E$ is the set of edges between the neighbours, the goal is to prove convergence of SA. For that purpose, the author assumed that $G$ is strongly connected and denoted the diameter of $G$, its degree and maximal energy difference between vertices by $D$, $d$ and $\Delta$, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a lemma on the 6th page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $X$ is any state in $V$, then the expected number of steps before a global optimal state is visited starting from $X$ is $\le (\frac 1 d \times e^{-\Delta / T})^{-D}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The probability to visit a neighbour $S_j$ after $S_i$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac 1 d \times e^{\frac{-\Delta E}{T}}$$&#xA;where $\frac 1 d$ is the probability of selecting $S_j$ of all $d$ neighbours of $S_i$, and $e^{\frac{-\Delta E}{T}}$ is the probability to actually move to it. $T$ is held constant in the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the proof of the lemma, the author determines the probability to visit a global optimum $g$ from any state $X$ as $\ge [\frac 1 d \times e^{-\frac \Delta T}]^D$ and finishes the proof:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This implies that the expected number of steps before $g$ is visited&#xA;  is $\le [d e^{\frac \Delta T}]^D$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Would anyone care to explain how do we get the estimate for the number of steps? I'm stuck with it, but am really, really intrigued to figure this out. :)&lt;/p&gt;&#xA;" OwnerUserId="18685" LastActivityDate="2015-12-28T12:02:31.307" Title="Expected number of steps before a global optimum is found with Simulated Annealing" Tags="|optimization|convergence|approximation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21669" PostTypeId="2" ParentId="21663" CreationDate="2015-12-25T23:27:22.450" Score="2" Body="&lt;p&gt;If, as you say, you only really need the characteristic polynomial of your matrices, and not the eigenvalues (solving for the roots of the characteristic polynomial doesn't in general give accurate eigenvalues), and since your matrices are really quite small ($40\times40$) and have only integers, it might be sufficient to just use matlab's symbolic manipulation to calculate the characteristic polynomial exactly, like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; charpoly(sym(gallery('grcar', 10)))&#xA;ans =&#xA;[ 1, -10, 54, -200, 553, -1176, 1935, -2424, 2211, -1328, 401]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which sidesteps any issues with numerical accuracy of &lt;code&gt;eig&lt;/code&gt;. Obviously the (integer) coefficients of the characteristic polynomial can be quite large.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2015-12-25T23:27:22.450" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21670" PostTypeId="1" CreationDate="2015-12-26T00:28:51.327" Score="1" ViewCount="66" Body="&lt;p&gt;I'm trying to solve this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{cases}&#xA;\partial_t E=-k\left([f(\rho)-i.\left[\delta+\frac{1}{2}a\left[\dfrac{\nabla^2_{\bot}}{4}+1-\rho^2\right]\right]]E - 2CP\right)\\&#xA;\partial_t P=-\gamma_{\bot}[(1+i\delta)P + ED]\\&#xA;\partial_t D=-\gamma_{\parallel}(D-\chi(\rho)-\frac{1}{2}(E^*P+EP^*))&#xA;\end{cases}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $f(\rho)$ and $\chi(\rho)$ are radial functions. and the Laplacian is only en x and y (transversal Laplacian).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho$ is the radial coordinate in a cylindrical coordinate system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E=E_x,E_y$ ;and   $E_x , E_y \in \mathbb{C}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$P=P_x,P_y$ ; and  $P_x , P_y \in \mathbb{C}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so $E_x,\ E_y,\ P_x,\ P_y$ are complex functions. which amounts to a total of 9 equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know much about CDF, I tried doing some explicit forward method, since I saw a Navier-Stokes problem being solved that way, but its unstable:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from the equation for the evolution of $P$:&#xA;$$P^{j+1}_{n,m}=-\gamma_{\bot}[(1+i\delta)P^j_{n,m}+E^j_{n,m}D^j_{n,m}]\Delta t + P^j_{n,m}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from the equation for the evolution of $D$: &#xA;$$D^{j+1}_{n,m}=-\gamma_{\parallel}\Delta t (D^j_{n,m}-\chi(\rho)_{n,m}-\frac{1}{2}((E^j_{n,m})^*P^j_{n,m}+E^j_{n,m}(P^j_{n,m})^*))+D^j_{n,m}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from the equation fo the evolution of $E$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t E=i\dfrac{ka}{8}\nabla^2_{\bot}E-k([f(\rho)E-i[\delta+\frac{1}{2}a[1-\rho^2]]E-P)\\$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then: $$\Re(E)^{j+1}_{n,m}= -\frac{ka}{8}\Delta t&#xA;\left(\frac{\Im(E)_{n+1,m}^{j} - 2\Im(E)_{n,m}^{j} + \Im(E)_{n-1,m}^{j}}{\Delta x^2}+\frac{\Im(E)_{n,m+1}^{j} - 2\Im(E)_{n,l}^{m} + \Im(E)_{n,m-1}^{j}}{\Delta y^2}\right)&#xA;-\left(f(\rho)_{n,m}\Re(E)^j_{n,m} + (\delta+\frac{a}{2}[1-\rho^2])\Im(E)^j_{n,m} - \Re(P)^j_{n,m}\right)k\Delta t + \Re(E)^{j}_{n,m}\\$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;witch holds for $\Re(E_x)$ and $\Re(E_y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Im(E)^{j+1}_{n,m}=&#xA;\frac{ka}{8}\Delta t&#xA;\left(\frac{\Re(E)_{n+1,m}^{j} - 2\Re(E)_{n,m}^{j}+\Re(E)_{n-1,m}^{j}}{\Delta x^2} + \frac{\Re(E)_{n,m+1}^{j} - 2\Re(E)_{n,m}^{j} + \Re(E)_{n,m-1}^{j}}{\Delta y^2}\right)&#xA;- \left(f(\rho)_{n,m}\Im(E)^j_{n,m} - (\delta+\frac{a}{2}[1-\rho^2])\Re(E)^j_{n,m} - \Im(P)^j_{n,m}\right)k\Delta t + \Im(E)^{j}_{n,m}\\$$&#xA;witch holds for $\Im(E_x)$ and $\Im(E_y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been reading up on CDF for several weeks, but still feel kind of lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions about winch method should I use, and why, or maybe some book.&lt;/p&gt;&#xA;" OwnerUserId="18687" LastEditorUserId="9667" LastEditDate="2015-12-27T18:54:29.543" LastActivityDate="2015-12-27T21:25:14.857" Title="Solving a nonlinear problem with CDF" Tags="|pde|finite-difference|fluid-dynamics|python|simulation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21671" PostTypeId="2" ParentId="21663" CreationDate="2015-12-26T05:52:47.910" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/21663/known-issues-with-eigenvalue-numerics?rq=1#comment35089_21663&quot;&gt;Kirill's suggestion&lt;/a&gt; to use symbolic linear algebra is on-point.  Another approach is to work out the characteristic polynomial over a number of finite fields, then use the Chinese remainder theorem to reconstruct the characteristic polynomial over $\mathbb{Z}$.  Finite field linear algebra does not suffer from the same kinds of numerical problems as you'll run into with &lt;code&gt;eig&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3525" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-08-25T17:27:21.547" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21672" PostTypeId="1" AcceptedAnswerId="21684" CreationDate="2015-12-26T11:07:17.393" Score="1" ViewCount="299" Body="&lt;p&gt;I want to solve 3 coupled equations. I converted them to a system of odes in time and discrete it in Length and radius. Now I have a problem in one of the equations in first point. Because in this point I have A second order derivative and when I discretize it I dont have $y$ in second point. This is my equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dy}{dy} = \frac{d^{2}y}{dr^{2}}+k(y-q)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the discretized form in matlab:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dy}{dt} = \frac{y(i+2)-2y(i+1)+y(i)}{\Delta{r}^{2}}+k(y(i)-q(i))$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in $i=1$ we dont have points $2$ and $3$ and they will calculate in next line in MATLAB. What should I do?&lt;/p&gt;&#xA;" OwnerUserId="17594" LastEditorUserId="7567" LastEditDate="2015-12-26T17:44:42.867" LastActivityDate="2015-12-28T18:56:40.650" Title="solving PDEs in MATLAB" Tags="|matlab|discretization|method-of-lines|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21673" PostTypeId="1" CreationDate="2015-12-26T16:24:38.110" Score="0" ViewCount="56" Body="&lt;p&gt;As stated in the title I have a raster image (black or white) with no complex or fuzzied components (only horizontal or vertical text, italic or bold are allowed).&#xA;I need to express that raster image in as cheap a way as possibile using vertor representation (I only have one primitive that can draw a rectangle).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first naive approach is to scan the image and use a line for each segment that I find in my scanning proccess.&#xA;If I do that for both the horizontal and vertical direction I can choose the cheapest representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sure that is not the best way, a bold 'T' letter results in a lot of lines with my approach, but it can be decomposed in two rectangles (best solution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what I am asking is, which is the best approach to deal with my particular vectorization problem?&lt;/p&gt;&#xA;" OwnerUserId="15009" LastEditorUserId="7567" LastEditDate="2015-12-26T17:26:26.017" LastActivityDate="2015-12-28T17:45:47.823" Title="Vectorization Algorithm using only rectangle" Tags="|vectorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21674" PostTypeId="2" ParentId="21672" CreationDate="2015-12-27T01:07:15.620" Score="0" Body="&lt;p&gt;I assume you are solving an initial-value problem, so that $y(r, t=0)$ is known. Your temporal discretization will determine what to do -- for example, using forward Euler, you would get:&#xA;$$y(r, t+\delta t) = y(r,t) + \delta \, f(y(r,t)),$$&#xA;where $f$ represents you right-hand side. This is the simplest explicit scheme, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Explicit_and_implicit_methods&quot; rel=&quot;nofollow&quot;&gt;this explanation&lt;/a&gt;, which you probably should not use in practice. There are many alternatives, e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods&quot; rel=&quot;nofollow&quot;&gt;Runge-Kutta methods&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2015-12-27T01:07:15.620" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21675" PostTypeId="1" CreationDate="2015-12-27T04:01:47.723" Score="1" ViewCount="41" Body="&lt;p&gt;Given: Several circular series of map GPS coordinates for several bus routes. The GPS coordinates are not all equal when they run along the same road. The number of GPS coordinates for a single bus route runs from 140-600.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem: When the points are downloaded drawn, the raw routes don't follow each other perfectly and especially when zoomed-out you can't see some of the routes. I want to somehow show the multiple routes that run along a road, mostly likely dashed with multiple colors. But in order to do that, I need to detect the segments of the route that run really close to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be perfect is finding the segments of the routes and mutating their GPS coords to be shared among them. Once that's done I can easily render it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: are there any good algorithms for this purpose, or some combination/tweaks of other algorithms I can use? It all has to be done autonomously in an app, not hand-done/hand-checked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need any other information please ask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/byhoU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/byhoU.png&quot; alt=&quot;Example of 5 routes, 3 on one side of the road and 2 on the other.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18694" LastEditorUserId="18694" LastEditDate="2015-12-27T04:30:26.003" LastActivityDate="2015-12-27T04:30:26.003" Title="Detecting and joining series of line segments that run along each other" Tags="|computational-geometry|geometry|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21678" PostTypeId="2" ParentId="21672" CreationDate="2015-12-27T17:20:30.390" Score="1" Body="&lt;p&gt;Please try to improve your explanation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the index i do you refer to the space variable or time one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, as far as I understood from your post, I think that you need to enforce some boundary condition if you're dealing at the end/beginning of your spatial domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, according to the physics of your problem, you have to enforce Neumann or Dirichlet BCs (&lt;a href=&quot;https://en.wikipedia.org/wiki/Neumann_boundary_condition&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Neumann_boundary_condition&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Dirichlet_boundary_condition&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Dirichlet_boundary_condition&lt;/a&gt;) or maybe other types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you give me a better explanation I can support you in this process.&lt;/p&gt;&#xA;" OwnerUserId="14817" LastActivityDate="2015-12-27T17:20:30.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21680" PostTypeId="2" ParentId="19780" CreationDate="2015-12-28T02:11:17.937" Score="2" Body="&lt;p&gt;I am working through the same book to teach myself Python. My working code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from math import *&#xA;from numpy import *&#xA;&#xA;&#xA;M = 0.0 #set a float variable for Madelung Constant&#xA;L = int(input(&quot;Enter the size of the crystal lattice L/2: &quot;))&#xA;&#xA;for x in range(-L, L+1):&#xA;  for y in range(-L, L+1):&#xA;      for z in range(-L, L+1):&#xA;          if x == y == z == 0:&#xA;              continue&#xA;          r = (x**2 + y**2 + z**2)**-0.5&#xA;          if x + y + z%2 == 0:&#xA;            M = M+r&#xA;          else:&#xA;            M = M-r&#xA;          print(x, y, z)&#xA;print(M)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18702" LastEditorUserId="9667" LastEditDate="2015-12-28T18:22:24.930" LastActivityDate="2015-12-28T18:22:24.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21682" PostTypeId="2" ParentId="21668" CreationDate="2015-12-28T12:02:31.307" Score="1" Body="&lt;p&gt;Suppose the probability to reach the optimum from any state is $p$, in other words at each step the probability of reaching the optimum has a &lt;a href=&quot;https://en.wikipedia.org/wiki/Bernoulli_distribution&quot; rel=&quot;nofollow&quot;&gt;Bernoulli distribution&lt;/a&gt; with parameter $p$. The number of steps required then has a &lt;a href=&quot;https://en.wikipedia.org/wiki/Geometric_distribution&quot; rel=&quot;nofollow&quot;&gt;Geometric distribution&lt;/a&gt;, with mean $1/p$. If you add back the inequality signs you get the result you stated.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2015-12-28T12:02:31.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21683" PostTypeId="1" CreationDate="2015-12-28T13:39:59.347" Score="4" ViewCount="583" Body="&lt;p&gt;I'm developing a solver for solving linear hyperbolic equations of first order with respect to time and spatial derivatives. The formal order of accuracy of the solver must be 5 because I use 5th-order accurate Runge--Kutta method and 5th-order finite-difference WENO for Hamilton--Jacobi equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To verify my solver, I do a convergence study. I have to cases: in one case I have an exponentially growing solution, and in another case I have an exponentially decaying solution. I do not know an exact solution, so I employ the solution on the finest grid as an exact one to measure the discretization error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After measuring the error in $L_2$-norm, I obtain the following results for the case of an exponentially growing solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Resolution  L2-error  L2-order&#xA;  199       3.36e-05  nan&#xA;  399       7.65e-07  5.45&#xA;  799       2.36e-08  5.01&#xA; 1599       5.05e-09  2.22&#xA; 3199       3.01e-09  0.74&#xA; 6399       2.15e-09  0.48&#xA;51199       nan       nan&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have 'weird' numbers in the 'Resolution' column, because spatial step is computed as $$\Delta x = \frac{1}{N+1},$$ where $N$ is the resolution from the 'Resolution' column. This way I can be sure that, for example, each grid point in the solution on the coarsest grid matches each 256th grid point in the solution on the finest grid (52000 / 200 = 256). Then the discretization error is measured for grid functions as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    e = \left(&#xA;              \frac{1}{N+1} &#xA;              \sum u^{(199)}_{i} - u^{(51999)}_{256i}&#xA;        \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data from the table above demonstrate that initially I have the 5th order of accuracy, although I do not get asymptotic convergence to value 5.00. Then, the order of the solution drops significantly, which I can explain with floating-point errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the real issue is that when I do convergence study for the case of an exponentially decaying solution, I do not get the 5th order of accuracy at all:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Resolution  L2-error  L2-order&#xA;199         1.73E-08  nan&#xA;399         1.14E-11  10.56&#xA;799         4.92E-12   1.22&#xA;1599        2.18E-12   1.17&#xA;3199        1.00E-12   1.12&#xA;6399        5.46E-13   0.87&#xA;51199       nan       nan&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Initially, I get ridiculous 10th order of accuracy, which immediately degrade to the 1st order. I should note that in this case the solution varies in range 1e-5 to 1e-12 (it decays). Can the small amplitude of the solution be a reason for this strange behavior of the order of accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is self-convergence test reliable? Do I measure discretization error correctly by subtracting two solutions pointwise?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the reason for drastically different behavior of convergence for the same solver with two different problems?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3254" LastActivityDate="2015-12-28T21:50:04.243" Title="Are self-convergence tests reliable?" Tags="|numerical-analysis|convergence|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21684" PostTypeId="2" ParentId="21672" CreationDate="2015-12-28T15:18:13.927" Score="0" Body="&lt;p&gt;Your problems is with $y(i+2)$  ? &#xA;You have to first put some initial conditiones, witch would be y(0)....y(n).&#xA;Besides the boundarys.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine if you were solving waves in a string. The initial conditions would be the shape of the string... Hence $y(r,t=0)=f(r)$  . That when discretized give the initial values for $y(i+1)$ and $y(i+2)$  .&lt;/p&gt;&#xA;" OwnerUserId="18687" LastEditorUserId="18687" LastEditDate="2015-12-28T18:56:40.650" LastActivityDate="2015-12-28T18:56:40.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21686" PostTypeId="2" ParentId="21673" CreationDate="2015-12-28T17:45:47.823" Score="1" Body="&lt;p&gt;The cheapest representation is clearly if you convert the image into text, with attributes for the font style. You only need one byte per character (plus a couple of bits for the font style), which is always going to be cheaper than having to store coordinates for line segments etc. That's the power of using ancillary information: if you know that an image consists of text, storing the text is far cheaper than storing a &lt;em&gt;representation&lt;/em&gt; of the text.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-28T17:45:47.823" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21687" PostTypeId="1" AcceptedAnswerId="21689" CreationDate="2015-12-28T19:53:37.403" Score="-3" ViewCount="222" Body="&lt;p&gt;I want to solve 7 coupled equations.I use method of line(MOL) and discrete the equation in Length and radius and convert them to a system of ODEs in time.and use ode15s to solve them in MATLAB. But an error occur:         &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Warning: Failure at t=5.422028e-006. Unable to meet integration tolerances without reducing the step size below the smallest value allowed (1.355253e-020) at time t.            &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It means that equations are too stiff. How can i solve them? Is there another method?&#xA;The equations are true.I am sure.&lt;/p&gt;&#xA;" OwnerUserId="17594" LastEditorUserId="17594" LastEditDate="2016-01-10T06:40:46.440" LastActivityDate="2016-01-10T06:40:46.440" Title="How can I solve stiff equations by method of line (MOL)?" Tags="|matlab|pde|ode|stiffness|method-of-lines|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21689" PostTypeId="2" ParentId="21687" CreationDate="2015-12-28T20:10:26.077" Score="5" Body="&lt;p&gt;ode15s is designed to handle stiff systems of ODEs so I doubt if the problem&#xA;you are encountering is that your &quot;equations are too stiff&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is more likely that your spatial discretization has an error for some&#xA;reason or your have some other MATLAB programming error. I suggest the following&#xA;approach to debug this: Set the final integration time for ode15s to be&#xA;something smaller than that when the failure occurs: say 5.2e-6 (I am assuming&#xA;the starting time is zero). Make sure you have enough output times in that&#xA;interval so you can produce reasonable plots of your dependent variables&#xA;at key points in your spatial mesh as a function of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is if you look at these plots you will see that one or more&#xA;of your dependent variables is going to infinity or -infinity as you&#xA;approach the final time. That should give you a clue as to which equation(s)&#xA;are causing problems for ode15s.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2015-12-28T20:10:26.077" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21690" PostTypeId="2" ParentId="21657" CreationDate="2015-12-28T20:15:01.540" Score="0" Body="&lt;p&gt;The general problem with all complicated approaches one can come up (e.g., randomly choosing bits, or rounding floating point number, etc) is to prove that the resulting set of random numbers are really equally distributed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in your case, things would appear to be relatively simple. To draw a number $x \in [A,B)$, draw a random number $y\in [0,B-A)$ and compute $x=y+A$. If $y$ is equally distributed, then so is $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you know that $[A,B)$ is an interval in the signed integers (of the C programming language). Consequently, $[0,B-A)$ is an interval in the unsigned integers for which the question you linked to already has an algorithm.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-28T20:15:01.540" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21691" PostTypeId="2" ParentId="21653" CreationDate="2015-12-28T20:16:32.307" Score="2" Body="&lt;p&gt;OpenFOAM is pretty widely used in the computational fluids community and has a good reputation. I can't say anything based on first hand experience, but it's reasonably well liked and indeed widely used.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-28T20:16:32.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21692" PostTypeId="1" AcceptedAnswerId="21698" CreationDate="2015-12-28T21:45:17.943" Score="2" ViewCount="2383" Body="&lt;p&gt;I am solving the monodimensional diffusion/reaction equation by discretization using the well-known method of lines  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;${\partial c\over\partial t}=D{\partial^2 c\over\partial x^2}+ r\text{    for t&amp;gt;0 and -L&amp;lt;x&amp;lt;L}  \quad[1]$&lt;br&gt;&#xA;$c(x,0)=1 \quad[2], \quad \text{  initial condition}$&lt;br&gt;&#xA;${dc\over dx} \mid_{(0,t)} =0\ \text{  for t&amp;gt;0,} \:[3], \text{  symmetry at the center}$&lt;br&gt;&#xA;$D{dc\over dx} \mid_{(L,t)} =k_L(C_L-C\mid_{(L,t)}) \quad[4]$   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last equation represents that the diffusion at the boundary equals the external mass transfer but I am having trouble to transform it to the method of lines. &#xA;The discretization of the system becomes:&lt;br&gt;&#xA;${dc_i\over dt}  =D{2 c_{i+1}-c_i\over \Delta x^2}+r \text{ for i=1}  \quad[5]$&lt;br&gt;&#xA;${dc_i\over dt}  =D{ c_{i+1}-2c_i+c_{i-1}\over \Delta x^2} +r\text{ for i = 2...N-1} \quad[6]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then, for the last condition I am not sure how to implement it. Discretizing equation [4], I obtain&#xA;$C_i ={k_L\:c_L + {D\over \Delta x} c_{i-1} \over k_L + {D\over \Delta x} } \text{ for i = N} \quad[7]$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think that this is correct given that $C_N$ does not depend on time, is calculated indirectly at each time step from $C_{N-1}$. And that I cannot obtain the derivative for ${dc_{N-1}\over dt}$. But I cannot see how that condition should be discretized. Thank you for your help.  &lt;/p&gt;&#xA;" OwnerUserId="18706" LastActivityDate="2017-08-03T08:24:27.800" Title="Implementation of Neumann boundary condition with method of lines - 1D diffusion/reaction equation" Tags="|pde|finite-difference|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21693" PostTypeId="2" ParentId="21683" CreationDate="2015-12-28T21:50:04.243" Score="3" Body="&lt;p&gt;For all practical purposes, an L2 error of 1e-9 is zero, due to accumulated roundoff. So, for your first example, everything with more than 1600 resolution is exact. For your second example, everything with better than 400 resolution is exact. At these points, you can no longer expect a significant improvement in accuracy by using more points, and the convergence &quot;rates&quot; you compute make no sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To test your code better, you need to find a problem where the error is larger on coarse meshes. (You should make sure that it's the &lt;em&gt;relative&lt;/em&gt;, not &lt;em&gt;absolute&lt;/em&gt; error that's large, so just multiplying the right hand side, initial and boundary values by 1e6 does not count.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2015-12-28T21:50:04.243" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21695" PostTypeId="2" ParentId="21653" CreationDate="2015-12-29T12:36:48.090" Score="1" Body="&lt;p&gt;You might find that a general CFD &quot;framework&quot; might not be the best one for many engineering design problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Advanced Simulation Library (ASL) and OpenFoam (both recommended by others here) will do very well on a large number of applications, but they might be worse than slightly more specific codes that, for example, are better at estimating the motions of floating bodies, ships manoeuvring etc. etc. OpenFoam has managed to be applied reasonably successfully to ship problems; ASL, as far as I know, has not. That same type of situation will occur with many other &quot;general&quot; CFD codes (like a person who is a &quot;Jack of all trades, master of none&quot;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to do a lot more research into CFD solvers, with a clear idea of what engineering applications you will be using them on for the next few years. A fairly large investment of time is needed to learn the intricacies of the &quot;frameworks&quot;, and it would be a pain if they were greatly inferior to others for entire classes of problems you will want to work on in the future.&#xA;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="16571" LastActivityDate="2015-12-29T12:36:48.090" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21696" PostTypeId="2" ParentId="21518" CreationDate="2015-12-29T13:55:37.810" Score="0" Body="&lt;pre&gt;&lt;code&gt;# B3lyp/LANL2DZ extrabasis opt freq Freq NMR=GIAO pop=(Full)&#xA;&#xA;1 0&#xA;&#xA;Z/matrix-XYZ&#xA;&#xA;Se 0&#xA;D 1 1.00&#xA;0.3630000 1.0000000&#xA;****&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the use of extrabasis &lt;/p&gt;&#xA;" OwnerUserId="18476" LastActivityDate="2015-12-29T13:55:37.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21697" PostTypeId="1" CreationDate="2015-12-29T14:52:11.697" Score="4" ViewCount="1392" Body="&lt;p&gt;I have a system of seven nonlinear equations that I want to find their symbolic solutions. The solution will depend on the parameter K, and I should have different solutions by varying the parameter. I want the program to give the ranges of K and the solution associated with each value.&#xA;I tried to solve the system using matlab symbolic toolkit. The code I used is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms K x  y  z  u  v  w  p; &#xA;&#xA;eq1 =  -2*x - 4*u*v + 4*y*p == 0;&#xA;&#xA;eq2 =  -9*y +3*x*p == 0;&#xA;&#xA;eq3 =  -4*z - 4*sqrt(2)*u*w + 4*sqrt(2)*w*p  + 4*K == 0;&#xA;&#xA;eq4 =  -5*u + x*v + 3*sqrt(2)*z*w == 0;&#xA;&#xA;eq5 =  -v + 3*x*u - sqrt(5)*x*w == 0;&#xA;&#xA;eq6 =  -w + sqrt(5)*x*v + sqrt(2)*z*u - sqrt(2)*z*p == 0;&#xA;&#xA;eq7 =  -5*p - 7*x*y - 3*sqrt(2)*z*w == 0;&#xA;&#xA;[x y z u v w p] = solve(eq1, eq2, eq3, eq4,eq5, eq6, eq7)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I run the code using matlab R2015b, and after running for abut 6 hours it either give an empty solution or stops working.&#xA;Any advice?&lt;/p&gt;&#xA;" OwnerUserId="18714" LastEditorUserId="4405" LastEditDate="2015-12-30T10:28:07.617" LastActivityDate="2016-01-01T11:29:22.530" Title="Solving system of 7 nonlinear algebraic equations symbolically" Tags="|matlab|nonlinear-equations|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="21698" PostTypeId="2" ParentId="21692" CreationDate="2015-12-30T10:10:49.110" Score="2" Body="&lt;p&gt;First a general comment. In the case of Neumann boundary condition, opposite to Dirichlet boundary condition where you specify the value of unknown function, you should approximate the PDE also in boundary nodes, it means you should not try to eliminate the unknown functions there as you suggest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, you should use the central difference to approximate the derivative in Neumann boundary condition, otherwise you lose accuracy that is not necessary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although in principle your method can work, you might need a quite fine mesh to obtain a good approximation, because it is like you solve it by MOL on a domain $(h,L-h)$ and with only a first order accurate scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following scheme is second order accurate. The scheme [6] is correct, I suggest to change [5] and [7] as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the index $i$ denote in which space coordinate $x_i$ you approximate your equation. I prefer $x_i = i h$ for $h=L/N$ and $i=0,1,\ldots,N$. Your aim should be to construct a system of ODEs for unknown functions $c_i(t) \approx c(x_i,t)$ for $i=0,1,\ldots,N$, it means including the concentrations at boundary nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using central difference to approximate $dc/dx$ in Neumann boundary conditions and using it to eliminate auxiliary functions $c_{-1}(t)$ and $c_{N+1}(t)$ you should obtain &#xA;$$&#xA;\frac{dc_0}{dt}=D \frac{-2c_0+2c_1}{dx^2}+r \quad [5]&#xA;$$&#xA;and&#xA;$$&#xA;\frac{dc_N}{dt}=D \frac{-2c_N+2c_{N-1}}{dx^2}+\frac{2k_L}{dx}(C_L-c_N)+r \quad [7]&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="27" LastEditDate="2015-12-30T17:42:12.810" LastActivityDate="2015-12-30T17:42:12.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21699" PostTypeId="1" AcceptedAnswerId="21825" CreationDate="2015-12-30T10:36:11.313" Score="10" ViewCount="1756" Body="&lt;p&gt;In the recent question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21697/solving-system-of-7-nonlinear-algebraic-equations-symbolically&quot;&gt;Solving system of 7 nonlinear algebraic equations symbolically&lt;/a&gt;, Brian Borchers experimentally confirmed that Maple can solve a polynomial system that Matlab/Mupad cannot handle. I have heard in the past from people working in the field that Maple has a high-quality implementation of Gröbner bases and related algorithms (which I assume is what is being used here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am tempted to suggest &quot;Matlab is slow on this kind of problems, switch to Maple&quot;, but I would like to have data to back up this statement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a set of benchmark results comparing the speed and effectiveness of Gröbner basis implementations and polynomial system solutions in different computer algebra systems?&lt;/strong&gt; (Maple, Mathematica, Matlab's symbolic toolbox, et cetera).&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="549" LastEditDate="2015-12-30T16:01:17.807" LastActivityDate="2021-06-29T01:44:46.697" Title="Benchmarks for Gröbner bases and polynomial system solution" Tags="|reference-request|polynomials|symbolic-computation|maple|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21700" PostTypeId="5" CreationDate="2015-12-30T10:39:08.703" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2015-12-30T10:39:08.703" LastActivityDate="2015-12-30T10:39:08.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21701" PostTypeId="4" CreationDate="2015-12-30T10:39:08.703" Score="0" Body="Gröbner bases of polynomial ideals, algorithms for their computation, and their use to determine properties of ideals and solve polynomial systems." OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2015-12-30T17:38:58.150" LastActivityDate="2015-12-30T17:38:58.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21702" PostTypeId="2" ParentId="21699" CreationDate="2015-12-30T11:19:42.917" Score="9" Body="&lt;p&gt;Googling &lt;code&gt;benchmark polynomial systems&lt;/code&gt; leads to a few hits, including the University of Mannheim's &lt;a href=&quot;http://krum.rz.uni-mannheim.de/cabench/diractiv.html&quot;&gt;Computer Algebra Benchmark Initiative&lt;/a&gt;. Sadly, most of these are out of date or defunct. The most active seems to be the &lt;a href=&quot;http://wiki.symbolicdata.org&quot;&gt;SymbolicData Wiki&lt;/a&gt;, but as far as I can tell, it only collects benchmark &lt;em&gt;problems&lt;/em&gt;, not benchmark &lt;em&gt;results&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some comparisons (dating back to 1996) of Axiom, Macsyma, Maple, Mathematica, MuPAD, and Reduce solving polynomial systems can be found in &lt;a href=&quot;https://www.informatik.uni-leipzig.de/~graebe/ComputerAlgebra/Publications/WesterBook.pdf&quot;&gt;Hans-Gert Gräbe, &lt;em&gt;About the Polynomial System Solve Facility of Axiom, Macsyma, Maple, Mathematica, MuPAD, and Reduce&lt;/em&gt;, Preprint 11/96 des Instituts für Informatik, Universität Leipzig, Germany, December 1996&lt;/a&gt;. The conclusion is that Axiom, Maple and Reduce win due to their using Gröbner bases (the others did not at this point in time), with Maple coming out slightly ahead of the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also an old comparison on the &lt;a href=&quot;https://www.singular.uni-kl.de/DEMOS/GrP-Karlsruhe-00/Singular/timings.html&quot;&gt;SINGULAR website&lt;/a&gt; showing SINGULAR 2.0 (current as of December 2015 is 4.0.2) beating Maple, among others. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, a more recent publication (&lt;a href=&quot;http://www-polsys.lip6.fr/~jcf/Publications/@ACA/wang_aca2014.pdf&quot;&gt;Yao Sun, Dongdai Lin, and Dingkang Wang. 2015. On implementing signature-based Gröbner basis algorithms using linear algebraic routines from M4RI. ACM Commun. Comput. Algebra 49, 2 (August 2015), 63-64&lt;/a&gt; compare the authors' implementation of a Gröbner basis algorithm with that of Maple, Singular and Magma, with Magma being faster than the other two packages by an order of magnitude (and tying with the authors' implementation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it seems to depend very much on the problem (size as well as structure) and the software version which package is the fastest. Nevertheless, the recommendation to use an actively developed, special-purpose computer algebra system such as Singular, Magma or Maple rather than a general-purpose symbolic computation software is a sound one. This goes double for a toolbox in a &lt;em&gt;numerical&lt;/em&gt; software, which adds another level of overhead and is usually several versions behind the stand-alone software they're based on (MuPAD, previously Maple, in the case of Matlab's toolbox).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2015-12-30T11:47:50.883" LastActivityDate="2015-12-30T11:47:50.883" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21703" PostTypeId="2" ParentId="21699" CreationDate="2015-12-30T15:22:11.357" Score="6" Body="&lt;p&gt;Always keep in mind that the results of any benchmark will depend, in addition to the problem's size, on the base field over which the polynomial ring is defined (rational numbers or integers modulo some power of a prime number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www-polsys.lip6.fr/~jcf/FGb/&quot; rel=&quot;noreferrer&quot;&gt;FGb library&lt;/a&gt; is an actively developed and high-performance implementation of the F5 algorithm.  A benchmark comparing FGb to Magma can be found in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Faugère, J.-C. (2010). FGb: A Library for Computing Gröbner Bases (pp. 84–87). doi:10.1007/978-3-642-15582-6_17&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2015-12-30T15:29:32.183" LastActivityDate="2015-12-30T15:29:32.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21704" PostTypeId="1" AcceptedAnswerId="21708" CreationDate="2015-12-30T17:43:53.493" Score="2" ViewCount="108" Body="&lt;p&gt;I am working on adaptive techniques for solving some elliptic equations.&#xA;The technique is based on residual on elements. My problem is that when I use a predefined tolerance for refining elements, the number of elements that refined are too much, even for a tolerance around $10^{-2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also, tried refining those elements where the error is bigger than half of the worst error, but now the number of elements that refined are too few.&#xA;In both  method, the locations are true locations but I need a strategy for refine elements in the middle of these, not too much and not too few. If I try by refine a fixed percentage of elements, how can I choose this percentage perfectly? Is it dependent on the problem? Is there any better way for that? &#xA;Also, I have another question, each element is refined to children (each triangle is divided to four triangles), is there any better way for adding new elements? It should be noted that the result obtained by this approach are better than bisection.&#xA;If there is good reference  about this, please let me know.&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="1804" LastEditDate="2015-12-30T19:27:53.943" LastActivityDate="2015-12-30T19:27:53.943" Title="Strategies for controlling number of new elements in adaptive mesh refinement" Tags="|reference-request|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21705" PostTypeId="2" ParentId="21575" CreationDate="2015-12-30T18:00:43.440" Score="2" Body="&lt;p&gt;It is non-convex, simply study the special case you get with $x=0, \alpha=0,\tau=1$. Hence, an SOCP formulation is not possible.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2015-12-30T18:00:43.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21706" PostTypeId="2" ParentId="20514" CreationDate="2015-12-30T18:33:38.017" Score="1" Body="&lt;p&gt;Even though these fluxes occur at the boundary of the fine mesh, they are no &lt;em&gt;boundary&lt;/em&gt; fluxes. What you need to do is discretize your equation at the refinement boundary, taking the assymetry of the cells into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How you precisely compute the fluxes will also depend on the time stepping method. The easiest case is using a single time step on all refinement levels. If you use different time steps on different levels, things are more complicated, because you need to 'predict' the coarse values.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2015-12-30T18:33:38.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21707" PostTypeId="2" ParentId="20915" CreationDate="2015-12-30T18:47:53.123" Score="1" Body="&lt;p&gt;Your initial guess can generate a large residual near the Neumann boundary. Depending on the restriction method, this residual might not decrease as desired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'd try is to instead of a V-cycle use an FMG cycle. Because the FMG-cycle starts at the coarsest grid, you will have a reasonable guess near your Neumann boundary condition at finer levels. In my experience, FMG works fine with Neumann boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastEditorUserId="18662" LastEditDate="2015-12-30T21:25:19.320" LastActivityDate="2015-12-30T21:25:19.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21708" PostTypeId="2" ParentId="21704" CreationDate="2015-12-30T19:13:38.860" Score="3" Body="&lt;p&gt;This is not a complete answer but based on my own experience with mesh refinement I felt compelled to write of few ideas/thoughts which would be too long for a comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One idea that I don't think you mentioned would be too refine the top percentage of the elements with greatest error &lt;strong&gt;and then also coarsen the bottom percentage of elements with lowest error&lt;/strong&gt;. This might help with too many elements being coarsened.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the advantages of using the 1:4 refinement:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The angles of the child elements are maximized. In fact the quality of the mesh will be no worse then the original starting mesh because the child elements are congruent to the parent element (you may have to bisect some neighboring triangles to satisfy conformity of the mesh which may produce poor angles).&lt;/li&gt;&#xA;&lt;li&gt;Because each triangle is split into 4 new ones (instead of 2 for bisection) you will get a refined mesh faster (i.e. fewer iterative steps in your solver) then you would for bisection. Thus if you were say solving the Poisson equation you would need fewer successive grids to get to your most refined grid level.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some good resources that I have used include &lt;a href=&quot;http://www.cs.rpi.edu/~flaherje/pdf/fea8.pdf&quot; rel=&quot;nofollow&quot;&gt;fea8&lt;/a&gt;, &lt;a href=&quot;https://www.researchgate.net/publication/26452473_Issues_in_adaptive_mesh_refinement_implementation&quot; rel=&quot;nofollow&quot;&gt;Hannoun&lt;/a&gt;, and the video series by Wolfgang Bangerth &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;video tutorials&lt;/a&gt; (see lectures 15-17). In particular, pay attention to the introduction text for lecture 17. It explains why he refines 30% percent of his elements and how that is optimal in terms of computational time over successive refinement levels.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2015-12-30T19:26:13.393" LastActivityDate="2015-12-30T19:26:13.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21709" PostTypeId="1" CreationDate="2015-12-30T19:42:11.357" Score="0" ViewCount="14190" Body="&lt;p&gt;How can I understand the flow is incompressible or not with this information at 4 edges?&lt;a href=&quot;https://i.stack.imgur.com/OyxVJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OyxVJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17572" LastActivityDate="2017-01-13T13:15:13.423" Title="how can i show flow is incompressible?" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21712" PostTypeId="1" AcceptedAnswerId="21732" CreationDate="2015-12-30T23:29:15.533" Score="3" ViewCount="1021" Body="&lt;p&gt;I have code that attempts to implement a solution to the Schrödinger equation where there is a central potential (more or less im thinking of hydrogen),&#xA;in 1-D using the numerov method to construct the wave function. &#xA;It does not take care of the states (N=1,..) as my goal is to understand the link between the theory and the software.&#xA;Here are some results for some energies, whether&#xA;or not they are eigenvalues, im not sure: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[6.054999999999915, 6.214999999999912, 6.534999999999905]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and a graph&lt;a href=&quot;https://i.stack.imgur.com/rF28D.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rF28D.png&quot; alt=&quot;a look at the square of the wave function&quot;&gt;&lt;/a&gt;.&#xA;The algorithm im trying to implement is noted here in section 6&#xA;&lt;a href=&quot;http://physics.unipune.ernet.in/~cmg/1dPotNumerov.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt;, but my potential is different. My problem is, the central potential has a 1/r at the origin,&#xA;how would I deal with that? (I did a sidestep in the code by adding an arbitrary 0.00001 to eliminate 0). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a more general code in another question &lt;a href=&quot;https://scicomp.stackexchange.com/a/19679/18728&quot;&gt;seen here&lt;/a&gt;, but I do not follow the shift from equation to 3d matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy&#xA;import matplotlib.pyplot as plt &#xA;&#xA;h=np.float64(1)&#xA;m=60&#xA;c=m/2&#xA;estep = 1.0/(100)&#xA;delta = 0.0002&#xA;x=np.linspace(-1000,1000,num=m)&#xA;x=(x - x[m/2]+0.00001 )&#xA;v =  1.0/(x**2)&#xA;sig=None&#xA;E = []&#xA;&#xA;def get_left(wf,k):&#xA;    for i in range(2,int(c)):&#xA;        wf[i] = 2*(( 1- 1/12* h**2* k[i-2]**2)*wf[i-2] - (1 + 1/12 *h**2* k[i-1]**2)*wf[i-1])/(1 + 1/12* h**2* k[i]**2)&#xA;    wf[0:c] = wf[0:c]/(np.sqrt(np.sum(wf[0:c]**2)))&#xA;    return wf&#xA;&#xA;def get_right(wf,k):&#xA;    mi = m-1&#xA;    for i in range(2,int(c)):&#xA;        wf[mi-i] = 2*(( 1- 1/12* h**2* k[mi-i+2]**2)*wf[mi-i+2] - (1 + 1/12 *h**2* k[mi-i+1]**2)*wf[mi-i+1])/(1 + 1/12* h**2*k[mi-i]**2)&#xA;    wf[c:mi] = wf[c:mi]/(np.sqrt(np.sum(wf[c:mi]**2)))&#xA;    return wf&#xA;&#xA;def root_search(epsilonu, epsilonl):&#xA;    en = None&#xA;    it =0&#xA;    while it &amp;lt; 10000:&#xA;        phil =  np.zeros(m)&#xA;        phir =  np.zeros(m)&#xA;        phil[0] =0&#xA;        phil[1] = 0.001&#xA;        phir[m-1]=0&#xA;        phir[m-2]=0.001&#xA;        k = (-v + (epsilonu+epsilonl)/2)&#xA;        phil = get_left(phil,k)&#xA;        phir = get_right(phir,k)&#xA;        diffl = np.diff(phil)&#xA;        diffr = np.diff(phir)&#xA;        erri = diffl[m/2]- diffr[m/2]&#xA;        if erri &amp;lt; delta:&#xA;            plt.plot(x, (phil+phir)**2 )&#xA;            plt.savefig( str('sho'+'.png' ))&#xA;            return (epsilonu+epsilonl)/2&#xA;        if erri &amp;lt; 0 : &#xA;            epsilonl = (epsilonu+epsilonl)/2&#xA;        if erri &amp;gt; 0:&#xA;            epsilonu =epsilonl &#xA;            epsilonl = (epsilonu+epsilonl)/2&#xA;        it+=1&#xA;    return None&#xA;&#xA;def start():&#xA;    epsilonl=0.01&#xA;    eps = epsilonl&#xA;    epsilonu=None&#xA;    iteration=0&#xA;    sig = None&#xA;    while iteration &amp;lt; 100000:&#xA;        phil =  np.zeros(m)&#xA;        phir =  np.zeros(m)&#xA;        phil[0] =0&#xA;        phil[1] = 0.001&#xA;        phir[m-1]=0&#xA;        phir[m-2]=0.001&#xA;        k = (-v + eps)&#xA;        phil = get_left(phil,k)&#xA;        phir = get_right(phir,k)&#xA;        diffl = np.diff(phil)&#xA;        diffr = np.diff(phir)&#xA;        err = diffl[m/2]- diffr[m/2]&#xA;        ssig = 1 if err&amp;gt;0 else -1&#xA;        if sig:&#xA;            if sig != ssig:&#xA;                epsilonu= eps &#xA;                E.append(root_search(epsilonu, epsilonl))&#xA;                print(E)&#xA;        sig = ssig&#xA;        epsilonl = eps&#xA;        eps = eps+estep&#xA;        iteration+=1&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    start()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The theory:&#xA;Im attempting to solve the 1-D Schroedinger equation, &#xA;$$ \left( -\frac{h^2}{2m}\frac{d^2}{dx^2} + V\left(x\right)\right) \phi\left(x\right) = \epsilon \phi\left( x\right) $$&#xA;with V  as a central potential (coulombic force, inverse square relation),&#xA;the final equation looks like (with proper choice of units to give q^2=1, h-1..., energy in hartree) &#xA;$$ \frac{d^2}{dx^2}\phi\left(x\right)  + 2 \left(\epsilon -\frac{1}{x^2} \right) \phi\left(x\right) =0 $$ &lt;/p&gt;&#xA;" OwnerUserId="18728" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2019-06-14T21:29:18.083" Title="ground state from the Schroedinger equation with a central potential what happens to the origin" Tags="|numerical-analysis|computational-physics|quantum-mechanics|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21713" PostTypeId="1" CreationDate="2015-12-31T21:54:36.590" Score="5" ViewCount="588" Body="&lt;p&gt;So I am attempting to solve a 3D poisson problem with Discontinuous Galerkin (interior penalty method). The weak form (written in FEniCS) is as following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = dot(grad(v), grad(u))*dx \&#xA;  - dot(avg(grad(v)), jump(u, n))*dS \&#xA;  - dot(jump(v, n), avg(grad(u)))*dS \&#xA;   + alpha/h_avg*dot(jump(v, n), jump(u, n))*dS \&#xA;   - dot(grad(v), u*n)*ds(1) \&#xA;   - dot(v*n, grad(u))*ds(1) \&#xA;   + (gamma/h)*v*u*ds(1)&#xA;L = v*f*dx - u0*dot(grad(v), n)*ds(1) + (gamma/h)*u0*v*ds(1) + g*v*ds(2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where u/v are the trial/test functions, u0 is Dirichlet BC, g is Neumann BC, h is cell size, n is facet normal, and alpha and gamma are the penalties. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For many 2D unit square problems such as one of the undocumented FEniCS/DOLFIN examples, I have seen alpha and gamma set to 4 and 8 respectively, but when I do a 3D problem, I need much higher values like 40 and 80. How do I systematically determine what the values for alpha and gamma need to be? &lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2016-01-01T15:52:36.957" Title="Choosing the penalty for Discontinuous Galerkin" Tags="|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21715" PostTypeId="2" ParentId="21713" CreationDate="2016-01-01T01:13:49.920" Score="6" Body="&lt;p&gt;The penalty usually scales as $O(p^2/h)$ for polynomial discontinuous Galerkin methods regardless of dimension, but for a more precise constant, it's usually enough to take the penalty large enough to guarantee coercivity with respect to the DG norm.  &lt;a href=&quot;http://webpages.sdsmt.edu/~kshahbaz/shahbazi_05.pdf&quot; rel=&quot;nofollow&quot;&gt;Shahbazi&lt;/a&gt; gives an expression for the penalty over a given face which just depends on $p$ and geometric mapping factors for volume/faces of simplicial elements, which is then refined to include mesh angles by &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0377042706005279&quot; rel=&quot;nofollow&quot;&gt;Epshteyn and Riviere&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For succinctness, Shahbazi's penalty constant $\gamma_f$ over a given face $f$ is roughly&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\gamma_f = \max(c_{K^+},c_{K^-})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K^+, K^-$ are the elements which share that common face, and $c_K$ is an order-dependent constant for each element&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$c_K = \frac{(p+1)(p+d)}{d}\frac{\text{Area of face}}{\text{Volume of element}}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ratio of face area to element volume is $\approx h$, so this has the $O(p^2/h)$ scaling expected (you can also sharpen this slightly by scaling the contributions of non-boundary faces by 1/2). You can also take the global penalty as the max over the individual penalties, but this may worsen conditioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm less sure of what the optimal $\alpha$ constant is, since it's not strictly necessary for well-posedness of SIPDG formulations.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2016-01-01T15:52:36.957" LastActivityDate="2016-01-01T15:52:36.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21717" PostTypeId="1" AcceptedAnswerId="21718" CreationDate="2016-01-01T04:59:11.383" Score="3" ViewCount="970" Body="&lt;p&gt;Ridge regression can be posed as minimizing the following objective function (over &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{2} \lVert Ax - b \lVert_2^2 ~+ \frac{\lambda}{2} \lVert x \lVert_2^2 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which has a closed form solution:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x = (A^TA + \lambda I)^{-1} A^T b $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's say we want to solve this problem for several different values of &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. Then its a good idea to cache the LU decomposition of &lt;span class=&quot;math-container&quot;&gt;$A^TA + \lambda I$&lt;/span&gt;. In MATLAB I this translates to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[L,U] = factor(A'*A + lambda*I)&#xA;x1 = L \ U \ A'*b1&#xA;x2 = L \ U \ A'*b2&#xA;% etc...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here's my question:&lt;/strong&gt; if &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is updated after several solves, is there a simple/efficient way of updating &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;? Or do I have to completely redo the LU factorization?&lt;/p&gt;&#xA;" OwnerUserId="6874" LastEditorUserId="6874" LastEditDate="2020-10-23T20:28:16.300" LastActivityDate="2020-10-23T20:28:16.300" Title="Clever ways to update LU factorization for ridge regression" Tags="|regression|matrix-factorization|update|" AnswerCount="1" CommentCount="1" ClosedDate="2020-10-24T19:22:44.757" ContentLicense="CC BY-SA 4.0" />
  <row Id="21718" PostTypeId="2" ParentId="21717" CreationDate="2016-01-01T05:56:24.087" Score="7" Body="&lt;p&gt;What is the size of your $A$ matrix?  Is $A$ sparse?  Does $A$ have some other special structure?  How many values of $\lambda$ do you want to try?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, you'd use the Cholesky factorization of $A^{T}A+\lambda I$ rather than the LU factorization since $A^{T}A+\lambda I$ is symmetric and positive definite.  However, updating the Cholesky or LU factorization of  a matrix after a full rank diagonal update is not easy in practice (low rank updates are a different matter.)    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One alternative is to do the computation of the regularized solutions using the SVD of $A$.  There's a simple formula for the Tikhonov regularized solution in terms of the SVD so that you only have to compute the SVD of $A$ once.  This is a reasonable way to go for small sized problems (e.g. $A$ has no more than a few thousand rows/columns.) You could also use the eigenvalue decomposition of $A^{T}A$ to do this instead of the SVD of $A$.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large problems, especially where $A$ is sparse, iterative methods are typically used.  You can solve the problem with explicit regularization as a linear least squares problem (hint: work from largest to smallest values of the parameter $\lambda$ and use the previous solution as the starting point for each new value of $\lambda$.)  A more common approach is to use an iteration such as the Landweber iteration or CGLS that will give you an implicitly regularized solution when you stop the iterations before full convergence.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-01-01T05:56:24.087" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21720" PostTypeId="2" ParentId="21697" CreationDate="2016-01-01T11:29:22.530" Score="3" Body="&lt;p&gt;&lt;strong&gt;Don't use Matlab.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The symbolic toolbox in Matlab is useful for computing the derivative, integral or root of simple functions to be used in further numerical computation, but cannot compete with modern computer algebra systems (CAS) such as &lt;a href=&quot;http://www.maplesoft.com/products/maple/&quot; rel=&quot;nofollow&quot;&gt;Maple&lt;/a&gt;, &lt;a href=&quot;https://www.singular.uni-kl.de&quot; rel=&quot;nofollow&quot;&gt;Magma&lt;/a&gt; or &lt;a href=&quot;https://www.singular.uni-kl.de&quot; rel=&quot;nofollow&quot;&gt;Singular&lt;/a&gt;. So if Matlab fails to compute a solution, you should try one of these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you just want to plot the solution(s) as a function of the parameter, that can usually be done within the package (not sure about Magma, though). If you want to use the results in further (numerical) computation, you can use Maple's &lt;code&gt;CodeGeneration&lt;/code&gt; module to generate a Matlab function that evaluates the solution for a given parameter. You could also implement everything within the general purpose &lt;a href=&quot;http://www.sagemath.org&quot; rel=&quot;nofollow&quot;&gt;Sage&lt;/a&gt; framework, which bundles (among many others) Singular together with NumPy, SciPy and Matplotlib (which together provide similar features to Matlab). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing to keep in mind is that solutions provided by a CAS are very likely to be in terms of mathematical objects (such as tenth roots, as Brian Borchers points out is the case in your problem) that cannot be represented numerically -- and hence plotted, or used in further computations -- exactly. So you are once more limited to a numerical approximation, which might(!) be no better than the one you computed. (It usually is, but it's something to be aware of.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-01-01T11:29:22.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21723" PostTypeId="2" ParentId="21625" CreationDate="2016-01-01T16:53:19.630" Score="2" Body="&lt;p&gt;Whilst the MVC/MMVM patterns can be useful where appropriate, I would avoid trying to shoehorn the numerics into a pattern led by your chosen GUI (or other UI) framework. Unless you multi-thread in some fashion, the GUI itself will be unresponsive as the simulation progresses until its completion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention that you wish to view the simulation's evolution as time progresses: for this you could use synchronous callbacks from the numerical code, or if implementing in a separate process you could monitor the standard output (stdout) from the solver if that provides any useful &quot;tick&quot; data. This may involve a non-trivial amount of development work compared to a simple batch solver process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Visual excellence (GUIs, 3D, etc) also do not have to be mutually exclusive with batch solution processes. A number of simulations I performed (coupled FEM-Neural) reported relative complex in-progress data to text terminals and post-processed more detailed visual output.&lt;/p&gt;&#xA;" OwnerUserId="16877" LastActivityDate="2016-01-01T16:53:19.630" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21725" PostTypeId="2" ParentId="21599" CreationDate="2016-01-01T19:33:20.947" Score="1" Body="&lt;p&gt;Resolving small features in FEM will always be costly, there is no getting away from that fact. Your problem seems to be framed in terms of computational burden. In my own case, I was looking at electric field problems in anatomical structures, so had a similar set of problems to your own. The question is usually how detailed a mesh is &quot;good enough&quot; for the particular problem: have you decided on a tolerance for mesh convergence? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility to consider is reducing the element order. By default COMSOL seems to prefer quadratic (2nd order) elements, but if you do not need to resolve derivatives in your solution then linear (1st order) elements will reduce the computational burden significantly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a beginner, I would probably stick with a single FEM for the solution before trying out more advanced techniques like mortar methods. But, as a beginner, remember that finite element analysis is a collection of skills rather than a monolithic ability, and you'll get better with each over time. &lt;/p&gt;&#xA;" OwnerUserId="16877" LastActivityDate="2016-01-01T19:33:20.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21727" PostTypeId="1" CreationDate="2016-01-02T04:44:55.847" Score="7" ViewCount="1613" Body="&lt;p&gt;I would like to efficiently and (to the extent possible) reliably find the Perron-Frobenius eigenvector of non-negative matrices. These are not stochastic matrices, they are typically dense, and their entries differ by many orders of magnitude. They are probably usually irreducible, though if you think of the very small entries as zero they will not be. They are not symmetric. They will be as large as I can make them with reasonable computation time on a laptop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not much of a numerics person and up to now I've tended to treat eigenvector routines as a &quot;back box&quot;. However, in this case none of the off-the-shelf solutions I've tried work very well. Numpy's &lt;code&gt;linalg.eig&lt;/code&gt; is very slow for matrices bigger than $2000\times 2000$ or so, presumably because it always computes every eigenvector, not just the top one. Moreover, the eigenvalue with the largest real part often does not correspond to an eigenvector with all non-negative (or even real) entries, which messes everything up. On the other hand, &lt;code&gt;scipy.sparse.linalg.eigs&lt;/code&gt; is great when it works (despite my matrices being dense), but sometimes I just can't get it to converge no matter what I do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried implementing power iteration by myself, but it's very slow and also suffers from convergence issues - it leaves me feeling that there ought really to be a better way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally I would like an algorithm with the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;faster than power iteration and with better convergence properties&lt;/li&gt;&#xA;&lt;li&gt;able to give me some kind of approximate solution in the case where it can't converge&lt;/li&gt;&#xA;&lt;li&gt;guaranteed not to give me any eigenvector other than the Perron-Frobenius one, i.e. not to give me something with complex elements or a mixture of positive and negative ones&lt;/li&gt;&#xA;&lt;li&gt;able to compute the left and right Perron-Frobenius eigenvectors simultaneously, if that would be more efficient than simply running it twice&lt;/li&gt;&#xA;&lt;li&gt;either available off-the-shelf (in Python or some other language), or explained in a form such that I, as a non-numerics person, can understand how to implement it without worrying too much about the theory behind it&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Does such a thing exist? References to literature giving an overview of how to solve this kind of problem would also be welcome, as I'm sure I'm not the first person to need to do this.&lt;/p&gt;&#xA;" OwnerUserId="3746" LastActivityDate="2016-01-02T16:41:15.827" Title="Numerical computation of Perron-Frobenius eigenvector" Tags="|linear-algebra|eigensystem|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21728" PostTypeId="2" ParentId="21727" CreationDate="2016-01-02T06:30:06.843" Score="6" Body="&lt;p&gt;The algorithm used by &lt;code&gt;eigs&lt;/code&gt; is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Arnoldi_iteration&quot; rel=&quot;nofollow&quot;&gt;Arnoldi iteration&lt;/a&gt; (in shift-and-invert mode).  You could think of it as a sophisticated way of doing an &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverse_iteration&quot; rel=&quot;nofollow&quot;&gt;inverse power iteration&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inverse power iteration method works as follows: if $A$ is your matrix and you know that the Perron root (the spectral radius of $A$) is $\lambda$, then a power iteration on the matrix $B = (A - \lambda I)^{-1}$, where $I$ is the identity matrix, will converge to the Perron-Frobenius eigenvector of $A$, let's call it $v$, much faster than a simple power iteration on $A$.  This is so because $v$ is an eigenvector of the largest eigenvalue of $B$, which will be close to infinity by construction and hence much larger than the next largest eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You still have to obtain an estimate of $\lambda$ but you could use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gershgorin_circle_theorem&quot; rel=&quot;nofollow&quot;&gt;Gershgorin circle theorem&lt;/a&gt; for this purposeor use other techniques (such as the bounds given in [Duan2013]).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, my suggestion is to use something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v = np.ones(A.shape[0])&#xA;eigenvalues, eigenvectors = linalg.eigs(A, k=2, sigma=shift, which='LM', v0=v)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the variable &lt;code&gt;shift&lt;/code&gt; equals your estimate for $\lambda$.  Then check that the resulting eigenvalues are sufficiently far apart and pick the eigenvector corresponding to the Perron root.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if the matrix is not irreducible, you will get more than one eigenvector corresponding to the largest eigenvalue.  Also note that even though $B$ involves a matrix inverse, it is rarely ever necessary (or convenient) to actually compute the inverse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Duan2013] Duan, X., &amp;amp; Zhou, B. (2013). Sharp bounds on the spectral radius of a nonnegative matrix. Linear Algebra and Its Applications, 439(10), 2961–2970. &lt;a href=&quot;http://doi.org/10.1016/j.laa.2013.08.026&quot; rel=&quot;nofollow&quot;&gt;http://doi.org/10.1016/j.laa.2013.08.026&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2016-01-02T16:41:15.827" LastActivityDate="2016-01-02T16:41:15.827" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21729" PostTypeId="1" CreationDate="2016-01-02T10:09:41.173" Score="1" ViewCount="825" Body="&lt;p&gt;I'm trying to compute the &lt;a href=&quot;https://en.wikipedia.org/wiki/Madelung_constant&quot; rel=&quot;nofollow&quot;&gt;Madelung constant&lt;/a&gt; of the ZnS lattice.&#xA;The method is as follows:&#xA;The lattice is a face-centered-cubic with basis $S^-: (0,0,0)$ and $Zn^+: (1/4,1/4,1/4)$.&#xA;The Madelung constant is&#xA;$$\alpha=\sum_j \frac{q_j}{r_j}$$&#xA;where $q_j$ is the charge (in this case divided by the number of neighboring blocks) of $j$&#xA;and $r_j$ the distance to a fixed ion.&#xA;Evjen's method tells us that we need to form blocks of neutral charge to obtain a good&#xA;convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've chosen to start with two cubes and the origin is placed in the center of the face that &#xA;is common to both cubes (so I have a block of size 1x1x2).&#xA;Then, two get more precision I add two cubes in each direction (so in the second iteration&#xA;I have a block of size 3x3x4).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that this method converges to $\alpha\approx 5.35$, when it should be $\alpha\approx 1.64$.&#xA;&lt;strong&gt;Is this the correct block expansion?&lt;/strong&gt;  I've tried with other block and I get the same value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't included the code, because I think that the problem is in the block chosen.&lt;/p&gt;&#xA;" OwnerUserId="13094" LastEditorUserId="13094" LastEditDate="2016-01-10T16:16:09.880" LastActivityDate="2016-01-10T16:16:09.880" Title="Is this the correct way to calculate the Madelung constant?" Tags="|computational-physics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21730" PostTypeId="1" CreationDate="2016-01-02T19:31:11.763" Score="1" ViewCount="56" Body="&lt;p&gt;I  want to show convergence of a finite element  method for a higher order equation.I have a coupled equations that solved together and gives two variables as answer $[u, v]$.&#xA;$$w+\Delta u=0$$&#xA;$$\Delta w=f$$&#xA;for finding the answer&#xA;$$[u,w]=K^{-1}F,$$&#xA;Unfortunately, I am really confused about that and have a lot of questions such as:&#xA;What space should I used?  Should I show convergence for both of variables together or separately?&#xA;If I have a bilinear form what do I need to show (boundedness, coercively and so existence, uniqueness )? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know a good reference that I can find my answers? &lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2016-01-02T19:40:17.037" LastActivityDate="2016-01-02T19:40:17.037" Title="convergence of a method" Tags="|finite-element|reference-request|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21732" PostTypeId="2" ParentId="21712" CreationDate="2016-01-02T21:06:51.267" Score="3" Body="&lt;p&gt;As mentioned by @Kirill, the differential equation for the hydrogen is slightly different. After the separation of variables, you end up with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left[ - \frac{\hbar^2}{2\mu} \left({1 \over r^2}{\partial \over \partial r}\left(r^2 {\partial R(r)\over \partial r}\right) - {l(l+1)R(r)\over r^2} \right) + V(r)R(r) \right]= E R(r),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle_in_a_spherically_symmetric_potential#Hydrogen-like_atoms&quot; rel=&quot;nofollow noreferrer&quot;&gt;rewrite the equation as&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left[ -\frac{1}{2} \frac{d^2}{dy^2} + \frac{1}{2} \frac{l(l+1)}{y^2} - \frac{1}{y}\right] u_l = W u_l$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next code (based on my previous post) computes the eigenvalues/eigenvectors for this equation. See that I added a small value to the denominators, so they don't vanish at zero.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.sparse import diags&#xA;from scipy.sparse.linalg import eigsh&#xA;import matplotlib.pyplot as plt&#xA;&#xA;L = 50*np.pi&#xA;N = 1001&#xA;xa = 0&#xA;xb = L&#xA;x = np.linspace(xa, xb, N)&#xA;dx = x[1] - x[0]&#xA;&#xA;l = 1&#xA;k = 0&#xA;T = -0.5*diags([-2., 1., 1.], [0, -1, 1], shape=(N, N))/dx**2&#xA;U_vec = 0.5*l*(l + 1)/(x**2 + 1e-6) - 1/(np.abs(x) + 1e-6)&#xA;U = diags([U_vec], [0])&#xA;&#xA;H = T + U&#xA;&#xA;vals, vecs = eigsh(H, which='SA')&#xA;&#xA;print(np.round(vals, 6))&#xA;print(np.round([-1/(2*n**2) for n in range(k + l + 1, k + l + 7)], 6))&#xA;&#xA;for k in range(5):&#xA;    vec = vecs[:, k]&#xA;    mag = np.sqrt(np.dot(vecs[:, k],vecs[:, k]))&#xA;    vec = vec/mag&#xA;    plt.plot(x, vec, label=r&quot;&lt;span class=&quot;math-container&quot;&gt;$n=%i$&lt;/span&gt;&quot;% (k+1))&#xA;&#xA;plt.xlabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;&quot;)&#xA;plt.ylabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$\psi(x)$&lt;/span&gt;&quot;)&#xA;plt.xlim(xa, xb)&#xA;plt.legend()&#xA;plt.savefig(&quot;eigenvectors.png&quot;, dpi=600) &#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That gives as solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[-0.125106 -0.0556   -0.031272 -0.020012 -0.013896 -0.010209]&#xA;[-0.125    -0.055556 -0.03125  -0.02     -0.013889 -0.010204]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t1rOY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t1rOY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your repulsive potential (different sign than the Laplacian), I used the same code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.sparse import diags&#xA;from scipy.sparse.linalg import eigsh&#xA;import matplotlib.pyplot as plt&#xA;&#xA;L = 50*np.pi&#xA;N = 1001&#xA;xa = 0&#xA;xb = L&#xA;x = np.linspace(xa, xb, N)&#xA;dx = x[1] - x[0]&#xA;&#xA;T = -0.5*diags([-2., 1., 1.], [0, -1, 1], shape=(N, N))/dx**2&#xA;U_vec = 1/(x**2 + 1e-6)&#xA;U = diags([U_vec], [0])&#xA;&#xA;H = T + U&#xA;&#xA;vals, vecs = eigsh(H, which='SA')&#xA;&#xA;print(np.round(vals, 6))&#xA;&#xA;for k in range(5):&#xA;    vec = vecs[:, k]&#xA;    mag = np.sqrt(np.dot(vecs[:, k],vecs[:, k]))&#xA;    vec = vec/mag&#xA;    plt.plot(x, vec, label=r&quot;&lt;span class=&quot;math-container&quot;&gt;$n=%i$&lt;/span&gt;&quot;% (k+1))&#xA;&#xA;plt.xlabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;&quot;)&#xA;plt.ylabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$\psi(x)$&lt;/span&gt;&quot;)&#xA;plt.xlim(xa, xb)&#xA;plt.legend()&#xA;plt.savefig(&quot;eigenvecs.png&quot;, dpi=600)  &#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with answers&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ 0.000408  0.001207  0.002405  0.004001  0.005997  0.008392]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/povIu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/povIu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although, I am not sure what that means. Because it is not letting the electron to be in the origin pushing it away, and the domain should be really unbounded. It is similar to having a square potential with a round edge at the left. For the more realistic case, you can use nonuniform finite differences and increase them exponentially or absorbing boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2019-06-14T21:29:18.083" LastActivityDate="2019-06-14T21:29:18.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21733" PostTypeId="1" AcceptedAnswerId="21742" CreationDate="2016-01-02T21:19:55.427" Score="12" ViewCount="1266" Body="&lt;p&gt;Suppose $A\in\mathbb{R}^{n\times n}$ is a symmetric, positive definite matrix.  $A$ is big enough that it's expensive to solve $Ax=b$ directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there an iterative algorithm for finding the smallest eigenvalue of $A$ that doesn't involve inverting $A$ in each iteration?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, I'd have to use an iterative algorithm like conjugate gradients to solve $Ax=b$, so repeatedly applying $A^{-1}$ seems like an expensive &quot;inner loop.&quot;  I only need a single eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="5129" LastEditorUserId="4405" LastEditDate="2016-01-04T18:08:13.370" LastActivityDate="2016-01-04T18:08:13.370" Title="Smallest eigenvalue without inverse" Tags="|linear-algebra|eigensystem|eigenvalues|iterative-method|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21734" PostTypeId="1" CreationDate="2016-01-03T00:49:47.297" Score="0" ViewCount="750" Body="&lt;p&gt;I am solving the 1D advection problem given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t}=-c\frac{\partial u}{\partial x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where c is the wave speed, and u is the unknown field variable, and x and t are time and space. I am using central differencing to discretise in space. My initial $u$ profile which is being convected is a step profile given by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u\left(x,t\right)=\left\{ \begin{array}{c}&#xA;1\;  for\;0&amp;lt;x&amp;lt;1\\&#xA;0\;  for\;1&amp;lt;x&amp;lt;2&#xA;\end{array}\right.,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about the boundary conditions. A the moment at the boundaries, if c&gt;0 I have set $u_L=u(1,t)$ (where u(1) is the value of u at the first node), but I end up with a lot of oscillations at the BC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also tried setting $u_L=1$ and $u_R=0$ (R indicates right and L left) and that gives me oscillations too but more sensible ones. I know that with central differencing spurious oscillations are expected but I am not sure about the correct boundary conditions and I would appreciate some help. I searched online for a clear answer but was unsuccessful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also solved the problem with upwinding and by setting $u_L=u(1)$ if c&gt;0 which seems to have worked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="17362" LastEditorUserId="17362" LastEditDate="2016-01-03T12:06:51.790" LastActivityDate="2016-01-03T12:06:51.790" Title="boundary conditions of linear advection problem" Tags="|fluid-dynamics|boundary-conditions|advection|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="21735" PostTypeId="2" ParentId="21734" CreationDate="2016-01-03T10:38:07.060" Score="1" Body="&lt;p&gt;Let me briefly explain one helpful and simple approach how to better understand the boundary conditions for your wave equation with constant speed. The idea is that you can consider your problem on the infinite interval and think about an equivalent definition of boundary conditions to such situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for instance if you extend your initial condition $u(x,0)$ to be valid for all real numbers then you can get the values at your boundary nodes from the exact solution $u(x,t)=u(x-c t,0)$, i.e. $u(0,t)=u(-c t, 0)$ and $u(2,t)=u(2-c t,0)$. You see that for the constant speed $c$ you should prescribe the value of solution only for one boundary node (depending on the sign of $c$), because the other boundary value is in fact defined by the initial condition (for some time)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to have the constant values at boundaries, it means you prescribe $u(0,t)=u_L$ if $c&amp;gt;0$ or $u(2,t)=u_R$ if $c&amp;lt;0$. In a theory you can prescribe the both interdependently of $c$, but then in one node it is not &quot;compatible&quot; with your wave equation in the sense that you prescribe some different &quot;process&quot; than your PDE  at such node (e.g. a boundary layer phenomena).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning the oscillations in nuemrical solution, as you mentioned yourself and you got also correct and good advice in comments, it is due to inappropriate numerical differencing. &lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-01-03T10:38:07.060" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="21736" PostTypeId="1" CreationDate="2016-01-03T14:03:31.690" Score="3" ViewCount="149" Body="&lt;p&gt;Not 100% sure where this question belongs, since I'm not sure if the problem is code related or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've written a program to model the solar system. I am now testing its accuracy. I've checked conservation of angular momentum, and it is perfectly conserved, as is energy - I'm using the velocity verlet algorithm. I've also plotted the path of the planets, their orbits are stable over 1000 years and are definitely circular (I'm sticking to modelling circular orbits).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the planets aren't where they started after one orbital period. They do get back to where they started, just not at the right time. I have no idea why. And it renders a lot of my checks useless. Why might this happen? The effect is significant, about 0.2*10^11m per year. And yet after 1000 years the orbit is still circular, with roughly the same radius. &lt;/p&gt;&#xA;" OwnerUserId="18259" LastEditorUserId="123" LastEditDate="2016-01-04T13:30:10.347" LastActivityDate="2016-01-04T13:30:10.347" Title="Why do planets move at the wrong speed in my solar system model?" Tags="|time-integration|conservation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21737" PostTypeId="1" AcceptedAnswerId="21751" CreationDate="2016-01-03T21:07:47.563" Score="1" ViewCount="611" Body="&lt;p&gt;I'm trying to solve these equations of hypersonic adiabatic flow over a flat plate. I did all the simplifications and got these equations for the stagnation point flow.&#xA;$$\left(Cf''\right)' + f f'' = \left(f'\right)^2-g$$&#xA;$$\left(\frac{C}{p_r} g'\right) + fg'=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a calorically perfect gas, $g=h/he$, and $he = \rm stagnation ~enthalpy$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p_r$ is a constant&#xA;and $C=g^{-1/3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(0)=0$ (over the streamline of the body)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f'(0)=0$ (this is $u$, the velocity over the body which is null)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f''(0)=?$ white suggests to try as a first value $0.664/\sqrt{C_w}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing this for the adiabatic case so &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g(0)=g_{wall}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g'(0)=0$ (no thermic flux)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Control conditions:&lt;br&gt;&#xA;$f'(\delta)=1$ and $g(\delta)=1$ &#xA;I want to integrate from $\delta=0$ to $\delta=6$, in the end, if $f'(\delta)$ is different than $1$ I'll have to change the value assumed for $f''(0)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I solve these equations, someone suggested me using the shooting method but how could I code this in matlab? I would like just a help on how to implement the equations on matlab, not a final solution.&lt;/p&gt;&#xA;" OwnerUserId="18757" LastEditorUserId="18757" LastEditDate="2016-01-07T19:49:32.833" LastActivityDate="2016-01-08T11:01:30.480" Title="Shooting method - Matlab ODE" Tags="|matlab|fluid-dynamics|ode|boundary-conditions|computational-physics|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21738" PostTypeId="1" CreationDate="2016-01-03T21:17:18.010" Score="3" ViewCount="186" Body="&lt;p&gt;The geodesic differential equations are given as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{d^2 x^j}{ds^2} + \Gamma^{\phantom{h}j}_{h\phantom{j}k}\frac{dx^h}{ds}\frac{dx^k}{ds} = 0,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $\Gamma^{\phantom{h}j}_{h\phantom{j}k}$ is the Christoffel symbol of second kind. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to T. A. Moore - A general relativity workbook, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the Riemann curvature tensor quantifies the relative variation of&#xA;  initially parallel geodesics. Since such geodesics only variate relative to each other in a curved space, the Riemann tensor will be zero everywhere in flat space.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If one can conclusively distinguish flat space vs. curved space, can such information have any usefulness when attempting to solve numerically the geodesic differential equations&lt;/p&gt;&#xA;" OwnerUserId="4249" LastActivityDate="2016-01-03T21:17:18.010" Title="Numerically solving geodesic differential equations with a priori knowledge of the Riemann curvature tensor" Tags="|ode|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21739" PostTypeId="2" ParentId="159" CreationDate="2016-01-03T21:18:37.443" Score="1" Body="&lt;p&gt;Several good reasons have been given by others for using libraries, and also for rolling your own routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can sometimes speed up computations for a specific application because you know in advance that you will never need the wide range of values the library routine covers, or the accuracy those routines deliver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, a lot depends on the particular application and how many times the library routine will be called. Why would you call a library routine for Bessel functions billions of times if you only need a few significant figures for a small range of x, and some simpler technique will be sufficient for your needs?&lt;/p&gt;&#xA;" OwnerUserId="16571" LastActivityDate="2016-01-03T21:18:37.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21740" PostTypeId="2" ParentId="5604" CreationDate="2016-01-03T22:38:27.350" Score="4" Body="&lt;p&gt;Although this thread is quite old, I thought it might be useful to have a reference to a peer-reviewed paper for generalizations of some common quadrature rules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nenad Ujevic,&#xA;&quot;A generalization of the modified Simpson’s rule and error bounds&quot;,&#xA;ANZIAM Journal, Vol. 47, 2005.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://journal.austms.org.au/ojs/index.php/ANZIAMJ/article/view/2/1268&quot; rel=&quot;nofollow&quot;&gt;http://journal.austms.org.au/ojs/index.php/ANZIAMJ/article/view/2/1268&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought it would be useful to give a good reference that is freely accessible, and that has references to other papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Alasdair noted above, including derivatives of the endpoints can increase the accuracy remarkably. For example, Ujevic and Roberts showed that adding first derivatives to Simpson's Rule reduces the error to 6th order in the grid spacing, whereas it is 4th order without the derivatives. The Ujevic paper shows that even tighter error bounds can be found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;N. Ujevic and A. J. Roberts, &#xA;A corrected quadrature formula and applications, &#xA;ANZIAM J., 45(E), (2004), E41–E56.&#xA;&lt;a href=&quot;http://anziamj.austms.org.au/V45/E051&quot; rel=&quot;nofollow&quot;&gt;http://anziamj.austms.org.au/V45/E051&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Christian Clason suggested that I move a comment I made into the answer because he thought the references I give are good ones and they might be lost if comments are scrubbed at some stage.)&lt;/p&gt;&#xA;" OwnerUserId="16571" LastEditorUserId="16571" LastEditDate="2016-06-08T15:02:13.433" LastActivityDate="2016-06-08T15:02:13.433" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21742" PostTypeId="2" ParentId="21733" CreationDate="2016-01-04T10:53:49.093" Score="15" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the largest-magnitude eigenvalue $\lambda_{max}$ of $A$ (with, say, &lt;code&gt;eigs('lm')&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then compute the largest magnitude (negative) eigenvalue $\hat{\lambda}_{max}$ of $M = A - \lambda_{max}I$ (again, through a standard call to &lt;code&gt;eigs('lm')&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Observe that $\hat{\lambda}_{max} + \lambda_{\max} = \lambda_{min}(A)$. The reason why this holds is explained &lt;a href=&quot;https://math.stackexchange.com/questions/271864/power-iteration-smallest-eigenvalue&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find your eigenvector $v$ by solving $(A - \lambda_{min} I) v = 0$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-01-04T12:59:26.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21743" PostTypeId="2" ParentId="21736" CreationDate="2016-01-04T13:29:01.580" Score="11" Body="&lt;p&gt;This is expected behavior with the Verlet algorithm.  It is a &lt;em&gt;symplectic integrator&lt;/em&gt;, which means that it will preserve quadratic invariants to within roundoff error -- thus the form planetary orbits is maintained quite accurately (I assume you mean to say that the orbits are &lt;em&gt;elliptical&lt;/em&gt;, not &lt;em&gt;circular&lt;/em&gt;).  However, the method is still a numerical approximation and does introduce error.  The errors introduced affect only the &lt;em&gt;phase&lt;/em&gt; of each orbit.  This means that the planets will move in the correct orbits, but not at the correct speeds.  In order to reduce the phase errors, you can take smaller time steps -- but there is eventually a limit to this, since it will increase the amount of rounding error in the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To learn more about these issues, I recommend the book &lt;a href=&quot;https://books.google.com.pe/books/about/Geometric_Numerical_Integration.html?id=cPTxCAAAQBAJ&amp;amp;hl=en&quot;&gt;Geometric numerical integration&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Studying the long-term stability of the solar system (over millions to billions of years) is a subtle and active research area that makes use of much more accurate numerical methods and even multiple-precision arithmetic.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-01-04T13:29:01.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21747" PostTypeId="1" CreationDate="2016-01-05T19:58:26.797" Score="17" ViewCount="10010" Body="&lt;p&gt;The strong form of a PDE requires that the unknown solution belongs in $H^2$. But the weak form requires only that the unknown solution belongs in $H^1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you reconcile this?  &lt;/p&gt;&#xA;" OwnerUserId="18398" LastEditorUserId="1804" LastEditDate="2016-01-06T11:01:47.287" LastActivityDate="2019-12-09T11:47:35.247" Title="Strong vs. weak solutions of PDEs" Tags="|pde|weak-solution|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21750" PostTypeId="2" ParentId="21747" CreationDate="2016-01-06T11:37:41.987" Score="26" Body="&lt;p&gt;Let's look at the simplest case of Poisson's equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$-\Delta u = f \tag{1}$$&lt;/span&gt;&#xA;on a domain &lt;span class=&quot;math-container&quot;&gt;$\Omega\subset \mathbb{R}^n$&lt;/span&gt; together with homogeneous Dirichlet conditions&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ u|_{\partial\Omega} = 0 \tag{2}$$&lt;/span&gt;&#xA;on the boundary &lt;span class=&quot;math-container&quot;&gt;$\partial\Omega$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;. We assume for now that &lt;span class=&quot;math-container&quot;&gt;$\partial\Omega$&lt;/span&gt; is as smooth as we want (e.g., can be parametrized by a &lt;span class=&quot;math-container&quot;&gt;$C^\infty$&lt;/span&gt; function) -- this will be important later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question now is how to interpret the (purely formal) PDE &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt;. Usually, this is answered in terms of how to interpret the derivative &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt;, but for our purpose it is better to focus on how to interpret the &lt;em&gt;equation&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The PDE &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; is assumed to hold &lt;strong&gt;pointwise for every &lt;span class=&quot;math-container&quot;&gt;$x\in\Omega$&lt;/span&gt;&lt;/strong&gt;. For this to make sense, the right-hand side &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; must be continuous, otherwise we can't speak about pointwise values &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;. This means that the second (classical) derivatives of the solution &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; must be continuous, i.e., we have to look for &lt;span class=&quot;math-container&quot;&gt;$u\in C^2(\Omega)$&lt;/span&gt;. &lt;br/&gt;&lt;br/&gt;&#xA;A function &lt;span class=&quot;math-container&quot;&gt;$u\in C^2(\Omega)$&lt;/span&gt; that satisfies &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; together with the boundary condition &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt; pointwise is called a &lt;strong&gt;classical solution&lt;/strong&gt; (sometimes, unfortunately, also &lt;em&gt;strong solution&lt;/em&gt;). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The requirement that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is continuous is much too restrictive for practical applications. If we only assume &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; to hold &lt;strong&gt;pointwise for almost every &lt;span class=&quot;math-container&quot;&gt;$x\in \Omega$&lt;/span&gt;&lt;/strong&gt; (i.e., everywhere except for sets of Lebesgue measure zero), then we can get away with &lt;span class=&quot;math-container&quot;&gt;$f\in L^2(\Omega)$&lt;/span&gt;. This means that the second derivatives are functions in &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt;, which makes sense if we take &lt;em&gt;weak derivatives&lt;/em&gt; and hence look for &lt;span class=&quot;math-container&quot;&gt;$u\in H^2(\Omega)\cap H^1_0(\Omega)$&lt;/span&gt;. (Remember that for functions &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; that are not continuous, we cannot take the boundary condition &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt; pointwise. Since &lt;span class=&quot;math-container&quot;&gt;$\partial \Omega$&lt;/span&gt; has zero Lebesgue measure as a subset of &lt;span class=&quot;math-container&quot;&gt;$\bar\Omega$&lt;/span&gt;, pointwise almost everywhere doesn't make sense either.)&lt;br/&gt;&lt;br/&gt;&#xA;A function &lt;span class=&quot;math-container&quot;&gt;$u\in  H^2(\Omega)\cap H^1_0(\Omega)$&lt;/span&gt; that satisfies &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt;  pointwise almost everywhere is called a &lt;strong&gt;strong solution&lt;/strong&gt;. Note that it is in general necessary and non-trivial to show that such a solution exists and is unique (which is the case for the example here).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If we are already dealing with weak derivatives, we can also further relax the assumptions on &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. If we take &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; to hold as an &lt;strong&gt;abstract operator equation in &lt;span class=&quot;math-container&quot;&gt;$H^{-1}(\Omega)$&lt;/span&gt;&lt;/strong&gt;, the dual space of &lt;span class=&quot;math-container&quot;&gt;$H^1_0(\Omega)$&lt;/span&gt;, then this makes sense for all &lt;span class=&quot;math-container&quot;&gt;$f\in H^{-1}(\Omega)$&lt;/span&gt; (which is a larger space than &lt;span class=&quot;math-container&quot;&gt;$L^2(\Omega)$&lt;/span&gt;). Pretty much by definition of the dual space and the weak derivative, &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; in this sense is equivalent to the &lt;strong&gt;variational equation&lt;/strong&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\int_\Omega \nabla u(x)\cdot \nabla v(x)\,dx = \int_\Omega f(x)v(x)\,dx \qquad\text{for all }v\in H^1_0(\Omega)\tag{3}.$$&lt;/span&gt;&lt;br/&gt;&#xA;A function &lt;span class=&quot;math-container&quot;&gt;$u\in  H^1_0(\Omega)$&lt;/span&gt; that satisfies &lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt;  is called a &lt;strong&gt;weak solution&lt;/strong&gt;. Again, it is in general necessary and non-trivial to show that such a solution exists and is unique (which is the case for the example here).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, since classical derivatives are also weak derivatives, every classical solution is also a strong solution. Similarly, by the embedding &lt;span class=&quot;math-container&quot;&gt;$H^2(\Omega)\subset H^1(\Omega)$&lt;/span&gt;, every strong solution is also a weak solution. The other directions are more subtle.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;If&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$(3)$&lt;/span&gt; has a unique solution, which moreover satisfies &lt;span class=&quot;math-container&quot;&gt;$u\in H^2(\Omega)$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$f\in L^2(\Omega)$&lt;/span&gt; (rather than just &lt;span class=&quot;math-container&quot;&gt;$H^{-1}(\Omega)$&lt;/span&gt;), then the weak solution is also a strong solution (and for &lt;span class=&quot;math-container&quot;&gt;$n=2$&lt;/span&gt; also a classical solution since in this case &lt;span class=&quot;math-container&quot;&gt;$H^2(\Omega)$&lt;/span&gt; embeds into &lt;span class=&quot;math-container&quot;&gt;$C(\bar\Omega)$&lt;/span&gt;). This property is sometimes called &lt;em&gt;maximal (elliptic) regularity&lt;/em&gt;, and holds for the Poisson equation assuming the boundary &lt;span class=&quot;math-container&quot;&gt;$\partial\Omega$&lt;/span&gt; (and the boundary data) is smooth enough. (This is where the above assumption comes in.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Otherwise, it can happen even for &lt;span class=&quot;math-container&quot;&gt;$f\in L^2(\Omega)$&lt;/span&gt; that the PDE has a weak solution but not a strong solution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If maximal regularity does not hold, it can also happen that the PDE has a unique strong solution (which is hence also a weak solution), but not a unique weak solution. This means that there exist many weak solution in, e.g., &lt;span class=&quot;math-container&quot;&gt;$H^1_0(\Omega)$&lt;/span&gt;, but only one of which is also in &lt;span class=&quot;math-container&quot;&gt;$H^2(\Omega)$&lt;/span&gt; and hence a strong solution. (The actual examples require more complicated spaces; see, e.g.,&#xA;&lt;em&gt;Meyer, Christian; Panizzi, Lucia; Schiela, Anton&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1080/01630563.2011.587074&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Uniqueness criteria for the adjoint equation in state-constrained elliptic optimal control&lt;/strong&gt;&lt;/a&gt;, Numer. Funct. Anal. Optim. 32, No. 9, 983-1007 (2011). &lt;a href=&quot;https://zbmath.org/?q=an:1230.35041&quot; rel=&quot;noreferrer&quot;&gt;ZBL1230.35041&lt;/a&gt;, or more complicated, nonlinear, equations; see, e.g., &lt;a href=&quot;http://www.numdam.org/item/JEDP_2015____A10_0/&quot; rel=&quot;noreferrer&quot;&gt;http://www.numdam.org/item/JEDP_2015____A10_0/&lt;/a&gt;.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2019-12-09T11:47:35.247" LastActivityDate="2019-12-09T11:47:35.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21751" PostTypeId="2" ParentId="21737" CreationDate="2016-01-06T13:38:44.037" Score="0" Body="&lt;p&gt;To change this into a system of first order ODEs, you can define&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Y(x;s) := \left(&#xA;  \begin{split}&#xA;    f''\\ &#xA;    f'\\&#xA;    f\\&#xA;    g'\\&#xA;    g&#xA;  \end{split}&#xA;\right) &#xA;= &#xA;  \left(\begin{split}&#xA;    Y_1\\ &#xA;    Y_2\\&#xA;    Y_3\\&#xA;    Y_4\\&#xA;    Y_5&#xA;  \end{split}&#xA;\right)\,.$$&#xA;Your first equation then gives&#xA;$$ CY_1' + C'Y_1 + Y_3Y_5= &#xA;    (Y_2)^2 - Y_5\,,$$&#xA;or, with&#xA;$C=g^{-1/3}$,&#xA;$C' = -\frac13g^{-4/3}g'$,&#xA;$$ Y_5^{-1/3}Y_1' - \tfrac13Y_5^{-4/3} Y_4Y_1 + Y_3Y_5= &#xA;    (Y_2)^2 - Y_5\,,$$&#xA;and the second differentiated is&#xA;$$\left(\frac{Y_5^{-1/3}}{p_r}+Y_3\right)Y_4' + &#xA;\left(\frac{-\frac13Y_4Y_5^{-4/3}}{p_r}+Y_2\right)Y_4 = 0\,,$$&#xA;with the additional equations&#xA;$$Y_2' = Y_1, \quad Y_3' = Y_2, \quad Y_5' = Y_4\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bringing these together, gives&#xA;$$&#xA;Y'(x;s) = &#xA;\left(\begin{split}&#xA;    Y_5^{1/3}\left(\tfrac13Y_5^{-4/3} Y_4Y_1 - Y_3Y_5 + (Y_2)^2 - Y_5\right)\\ &#xA;    Y_1\\&#xA;    Y_2\\&#xA;    - \left(\frac{Y_5^{-1/3}}{p_r}+Y_3\right)^{-1} &#xA;\left(\frac{-\frac13Y_4Y_5^{-4/3}}{p_r}+Y_2\right)Y_4\\&#xA;    Y_4&#xA;  \end{split}\right)&#xA;$$&#xA;and initial conditions&#xA;$$Y(0;s) := \left(&#xA;  \begin{split}&#xA;    s\\ &#xA;    0\\&#xA;    0\\&#xA;    0\\&#xA;    g_{\text{wall}}&#xA;  \end{split}&#xA;\right) \,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use an ODE solver to integrate this the find $Y(\delta)$ for some $\delta&amp;gt;0$, and change $s$ to match your additional condition &#xA;$$F(s) := Y_2(\delta;s) - 1 = 0\,,$$&#xA;or&#xA;$$F(s) := Y_5(\delta;s) - 1 = 0\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could be by bisection (find 2 values $s_1$ and $s_2$ with $F(s_1)&amp;lt;0$, $F(s_2)&amp;gt;0$, and then &lt;em&gt;home in&lt;/em&gt; by testing $\tilde{s}=0.5(s_1+s_2)$ and replace $s_1$ or $s_2$ as appropriate) or even Newton's method (more work is required to get $F_s(s)$, but it's possible).&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="17671" LastEditDate="2016-01-08T11:01:30.480" LastActivityDate="2016-01-08T11:01:30.480" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21752" PostTypeId="1" AcceptedAnswerId="21957" CreationDate="2016-01-06T15:04:21.340" Score="2" ViewCount="258" Body="&lt;p&gt;I have a discrete 2D temperature field, i. e. a dataset of N points (x, y, T). I would like to compute line integrals at fixed radii, &#xA;$\int_0^{2\pi}T(r,\theta)d\theta$. My first thought was to transform all coordinates into radial coordinates, and then interpolate to a regular grid in $r,\theta$, so the integral would turn into a simple summation. This approach doesn't seem to be ideal, as I lose data at $r=0$. Is there a better way to solve my problem?&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2016-01-29T15:56:32.007" Title="How to compute line integral over circle" Tags="|integration|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21753" PostTypeId="1" CreationDate="2016-01-06T15:43:20.173" Score="4" ViewCount="863" Body="&lt;p&gt;I have two parts, which need to be meshed. One part is located inside another. To simplify the problem, imagine two cubes: one inside another.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/h87gD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h87gD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Geometries are stored in STL files. From CAD point of view, one file contains an internal cube, another contains cube with a hole for an internal cube. All dimensions fits (there are no intersections). My task is to create a conformal tetrahedral mesh suitable for mechanical FEM analysis - without duplicated nodes on shared faces. Each element should contain information to which part it belongs. File format does not matter but Abaqus is preferred.&lt;/p&gt;&#xA;&lt;p&gt;I decided to use gmsh, but I cannot mesh both parts together.&#xA;I tried the following commands:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;File -&amp;gt; Open Internal cube&lt;/li&gt;&#xA;&lt;li&gt;Add volume to internal cube&lt;/li&gt;&#xA;&lt;li&gt;File -&amp;gt; Merge External cube&lt;/li&gt;&#xA;&lt;li&gt;Add volume to external cube&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This produces mesh but elements in internal cube are duplicated. It looks like gmsh does not treat hole (in external cube) as a void. I also tried to use Coherence in gmsh but it failed with error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error   : Self intersecting surface mesh, computing intersections (this could take a while)&lt;/p&gt;&#xA;&lt;p&gt;Error   : Surface mesh is wrong, cannot do the 3D mesh&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I assume, that my geometries are fine:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Internal cube: &lt;a href=&quot;http://pastie.org/10673562&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pastie.org/10673562&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;External cube: &lt;a href=&quot;http://pastie.org/10673564&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pastie.org/10673564&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;How to create such a mesh which satisfy my requirements?&lt;/p&gt;&#xA;" OwnerUserId="1010" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-01-06T15:43:20.173" Title="Meshing two parts together" Tags="|mesh-generation|gmsh|" AnswerCount="0" CommentCount="6" ClosedDate="2016-01-07T15:20:39.777" ContentLicense="CC BY-SA 3.0" />
  <row Id="21754" PostTypeId="1" AcceptedAnswerId="21755" CreationDate="2016-01-07T05:36:30.083" Score="5" ViewCount="5392" Body="&lt;p&gt;I am curious as to what the current status is for symbolic computation, and future outlooks as well. Here are some questions I would like to pose :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is the current outlook for symbolic computation ? &lt;/li&gt;&#xA;&lt;li&gt;Will it replace numerical computation in the near future (is that even a possibility) ?&lt;/li&gt;&#xA;&lt;li&gt;Can symbolic computation outperform numerical computation ?&lt;/li&gt;&#xA;&lt;li&gt;What are advantages of symbolic computation over numerical, and vice versa ?&lt;/li&gt;&#xA;&lt;li&gt;Is symbolic computation being used in any significant computational projects (with regards to size and importance)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4249" LastEditorUserId="4249" LastEditDate="2016-01-17T11:19:38.283" LastActivityDate="2016-01-17T23:31:41.737" Title="Symbolic computation vs. numerical computation" Tags="|symbolic-computation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21755" PostTypeId="2" ParentId="21754" CreationDate="2016-01-07T05:51:09.490" Score="10" Body="&lt;p&gt;I don't think I can answer all of these, but I can give you some thoughts from what I have seen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think symbolic computation is a great tool, but it obviously most useful for problems that have analytical solutions and forms. Not every problem is really represented this way, however. So this is where numeric computation would probably prove a better option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also not sure what you would define as outperforming numeric computation. But symbolic arithmetic should be slower than numeric, so it doesn't outperform with respect to efficiency. It is possible it could be better accuracy-wise, though, when you go and substitute values into the symbolic result versus the numeric result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of using symbolic math, we use it at work (I work on missile algorithms, simulations, machine learning, etc). We usually use it to precompute some intense algebraic quantity, like the eigenvectors and eigenvalues of a 4x4 matrix, and directly code the result to make the computation faster (it helps since our code is running on an embedded system). We use it a lot for that sort of thing. But outside of the embedded code, we don't use it much.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-01-07T05:51:09.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21756" PostTypeId="1" AcceptedAnswerId="21757" CreationDate="2016-01-07T10:13:33.107" Score="1" ViewCount="71" Body="&lt;p&gt;I've got several sets of xyz-data where x,y are coordinates and z is a 'value'. I want to be able to do simple arithmetic operations on these data. The x,y locations are not necessarily the same in the different data sets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There must be a library (preferably python) out there so that I don't have to do all the work by myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does not have to be fancy, linear interpolation between the points is fine and accuracy is not much of an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anybody know if there exists such a library?&lt;/p&gt;&#xA;" OwnerUserId="18792" LastActivityDate="2016-01-07T11:15:13.843" Title="library for arithmetic operations on unstructured xyz" Tags="|libraries|data-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21757" PostTypeId="2" ParentId="21756" CreationDate="2016-01-07T11:15:13.843" Score="3" Body="&lt;p&gt;I would recommend looking at the &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.griddata.html#scipy.interpolate.griddata&quot; rel=&quot;nofollow&quot;&gt;griddata&lt;/a&gt; method in SciPy, which seems to have the functionality you need. Pay attention to the 'fill value' argument if you are looking at points outside your $x,y$ data set.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2016-01-07T11:15:13.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21758" PostTypeId="1" CreationDate="2016-01-07T17:33:42.653" Score="0" ViewCount="1085" Body="&lt;p&gt;I have the ODE below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d}{dt}\pmatrix{x\\ y} = \pmatrix{0 &amp;amp;1\\-a &amp;amp;0}\pmatrix{x\\ y} \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $m=1$ leapfrog method is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_{n+1} = y_{n-1} + 2f_nh \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the $a&amp;lt;0$ state, how do I simply find the condition the solution must satisfy for the method to be successful? Further, how do I enforce this in practice?&lt;/p&gt;&#xA;" OwnerUserId="18797" LastEditorUserId="9667" LastEditDate="2016-01-09T04:57:41.003" LastActivityDate="2016-01-09T04:57:41.003" Title="The condition for stability using the leapfrog method" Tags="|computational-physics|stability|explicit-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21759" PostTypeId="2" ParentId="21758" CreationDate="2016-01-07T22:23:45.977" Score="2" Body="&lt;p&gt;All you need to do is find the stability criteria for whatever scheme (in this case Leap Frog Method) with respect to the simple model equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dz}{dt} = \lambda z$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for complex $\lambda$. Once you have the stability criteria, you need to find the eigenvalues of the matrix you have in your ODE system. Then just make sure each eigenvalue satisfies the stability criteria. Assuming they do, then you know the method should work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is true because if you can diagonalize the ODE system, then you can solve each equation independently. Now that the equations are independent of each other, and since you diagonalized everything, the equations will have the form of the model equation. And so now you can easily check that each independent equation satisfies the stability criteria and ensure it can be integrated. If they can all be integrated individually using your scheme, then the whole system can be integrated with the same scheme (without needing it in a diagonalized form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to leap frog, keep in mind that it has a small set of $\lambda$ values you can have for it to be stable. This may not be the best scheme to use, if you have the ability to do something else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reference I found walking through some of the theory behind the method is located &lt;a href=&quot;http://www.cems.uvm.edu/~tlakoba/math337/notes_4.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-01-07T22:23:45.977" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21761" PostTypeId="1" AcceptedAnswerId="21763" CreationDate="2016-01-08T02:54:28.850" Score="10" ViewCount="12423" Body="&lt;p&gt;What is the difference between the $l^2$ norm and the $L^2$ norm.  I can not find a definitive reference.  &lt;a href=&quot;https://en.wikipedia.org/wiki/Norm_%28mathematics%29#Euclidean_norm&quot;&gt;Wikipedia&lt;/a&gt; uses them interchangeably.&lt;/p&gt;&#xA;" OwnerUserId="17532" LastActivityDate="2016-06-25T17:40:10.430" Title="Difference between l2 norm and L2 norm" Tags="|error-estimation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21762" PostTypeId="2" ParentId="21761" CreationDate="2016-01-08T04:02:24.210" Score="5" Body="&lt;p&gt;The 2-norm for sequences is denoted by $\ell^2$. For functions on the real line $L^2$ is the standard notation of the 2-norm.&lt;/p&gt;&#xA;" OwnerUserId="10143" LastActivityDate="2016-01-08T04:02:24.210" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21763" PostTypeId="2" ParentId="21761" CreationDate="2016-01-08T08:50:11.970" Score="17" Body="&lt;p&gt;Both norms are similar in that they are induced by the scalar product of the respective Hilbert space, but they differ because the different spaces are endowed with different inner products:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For $\mathbb{R}^N$, the Euclidean norm of $v = (v_1,\dots,v_N)^T\in \mathbb{R}^N$ is defined by&#xA;$$\|v\|_{2}^2 = (v,v)_{2} = \sum_{i=1}^N v_i^2.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For $\ell^2$ (the space of real sequences for which the following norm is finite), the norm of $v = \{v_i\}_{i\in\mathbb{N}} \in \ell^2$ is defined by&#xA;$$\|v\|_{\ell^2}^2 = (v,v)_{\ell^2} = \sum_{i=1}^\infty v_i^2.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For $L^2(\Omega)$ (the space of Lebesgue measurable functions on a bounded domain $\Omega\subset\mathbb{R}^d$ for which the following norm is finite), the norm of $u\in L^2(\Omega)$ is defined by&#xA;$$\|u\|_{L^2}^2 = (u,u)_{L^2} = \int_\Omega u(x)^2\,dx.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All this is standard, can be found in any introductory textbook on functional analysis, and is probably already known to you. Since the question is tagged &lt;a href=&quot;/questions/tagged/error-estimation&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;error-estimation&amp;#39;&quot; rel=&quot;tag&quot;&gt;error-estimation&lt;/a&gt;, you are likely interested in the practical difference in using one or the other, for example for finite element discretization. Let's say you have a finite-dimensional subspace $V_h\subset L^2(\Omega)$ which is the span of a finite number of basis functions $\{\varphi_1,\dots,\varphi_N\}$. Then any $u_h\in V_h$ can be written as &#xA;$$u_h = \sum_{i=1}^N u_i \varphi_i. \tag{1}$$&#xA;Since $V_h\subset L^2(\Omega)$, you can of course measure $u_h$ by the $L^2$ norm. &#xA;Alternatively, you can identify $u_h$ with the vector $\vec u:=(u_1,\dots,u_N)^T\in\mathbb{R}^N$ (sometimes called &lt;em&gt;coordinate isomorphism&lt;/em&gt;) and measure $u_h$ by the Euclidean norm of $\vec u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do the two ways of measuring $u_h$ compare? Plugging in the definition $(1)$ yields&#xA;$$ \|u_h\|_{L^2}^2 = (u_h,u_h)_{L^2} = \sum_{i=1}^N\sum_{j=1}^N u_iu_j \int_\Omega \varphi_i(x)\varphi_j(x)\,dx = \vec u^T M_h \vec u,$$&#xA;where $M_h\in \mathbb{R}^{N\times N}$ is the mass matrix with entries $M_{ij} = \int_\Omega \varphi_i(x)\varphi_j(x)\,dx$. By comparison, we have &#xA;$$ \|u_h\|_{\ell^2}^2 := \|\vec u\|_2^2 = \vec u^T \vec u.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both norms are therefore equivalent, i.e., there exist constants $c_1,c_2&amp;gt;0$ such that&#xA;$$ c_1\|u_h\|_{\ell^2}\leq \|u_h\|_{L^2} \leq c_2 \|u\|_{\ell^2}\qquad\text{for all }u_h\in V_h.$$&#xA;So in principle, you could use both norms interchangeably -- if the error goes to zero in one norm, it also goes to zero in the other norm, and with the same rate. &#xA;However, note that while the constants $c_1$ and $c_2$ are independent of $u_h$, they do depend on $V_h$, and in particular on $N$. This is important if you want to compare discretization errors for &lt;em&gt;different&lt;/em&gt; spaces $V_h$ with (say) $N_1&amp;lt;N_2$, in which case you should use a norm that does &lt;em&gt;not&lt;/em&gt; itself depend on $N_1$ or $N_2$, i.e., the $L^2$ norm.&#xA;(You can see this by taking $u_h$ as the constant function $u_h=1$ and compare $\|u_h\|_{\ell^2}$ for different $N$ with $\|u_h\|_{L^2}$ -- the former scales as $\sqrt{N}$, while the latter has the same value for every $N$, since the mass matrix compensates for the scaling.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also a third -- intermediate -- alternative, where the mass matrix is approximated by a diagonal matrix $D_h$ (e.g., by taking as diagonal elements of $D_h$ the sum of the corresponding row of $M_h$), and the norm is taken as $\|u_h\|_{D}^2:=\vec u^T D_h\vec u = \sum_{i=1}^N (D_h)_{ii} u_i^2$; this is usually referred to as &lt;em&gt;mass lumping&lt;/em&gt;. This norm is also equivalent with both the $\ell^2$ and the $L^2$ norm -- and in this case, the constants $c_1$ and $c_2$ when comparing $L^2$ and mass lumping norm do &lt;em&gt;not&lt;/em&gt; depend on $N$. &lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-06-25T17:40:10.430" LastActivityDate="2016-06-25T17:40:10.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21764" PostTypeId="1" CreationDate="2016-01-08T16:12:31.713" Score="4" ViewCount="232" Body="&lt;p&gt;I need some help in numerically solving the nonlinear Poisson's equation for electrons in frequency domain. The steady-state equation is:&#xA;\begin{equation}&#xA;\nabla.(\epsilon\nabla\varphi) = q\left(n_i\exp\left(\dfrac{\varphi-\Phi_n}{V_\text{T}}\right) -N_D\right)&#xA;\,.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In small-signal analysis, we have $\varphi = \varphi_{\text{DC}} + \delta\varphi$, with $\delta\varphi &amp;lt;&amp;lt;\varphi_{\text{DC}}$. &#xA;This yields:&#xA;\begin{equation}&#xA;\nabla.\Big[\epsilon\nabla(\varphi_{\text{DC}} + \delta\varphi)\Big] = &#xA;q\left(n_i\exp\left(\dfrac{\varphi_{\text{DC}} + \delta\varphi-\Phi_n}{V_\text{T}}\right) -N_D\right)&#xA;\end{equation}&#xA;which I can further simplify to:&#xA;\begin{equation}&#xA;\nabla.(\epsilon\nabla\varphi_{\text{DC}}) + \nabla.(\epsilon\nabla\delta\varphi) = &#xA;q\left(n_i\exp\left(\dfrac{\varphi_{\text{DC}} + \delta\varphi-\Phi_n}{V_\text{T}}\right) -N_D\right)&#xA;\,\,.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know how to proceed from here. That is, how should I treat the exponential term on the RHS and what are the basic assumptions in doing so?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-01-08T16:12:31.713" Title="Poisson equation in frequency domain" Tags="|numerical-analysis|nonlinear-equations|poisson|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21765" PostTypeId="5" CreationDate="2016-01-09T07:18:48.563" Score="0" Body="&lt;p&gt;Molecular dynamics simulations aim at solving the equations of motion of the atoms belonging to a molecular system using explicit time propagation and taking into account the effect of temperature on the atomic velocities (as opposite to geometry optimization). The interatomic forces can be described in classical (molecular mechanics, MM), quantum (quantum mechanics, QM) or mixed (QM/MM) frameworks. A number of constraints can be applied to part of the system in order to simplify the problem. For instance, intramolecular bonds are often constrained in MM simulations.&lt;/p&gt;&#xA;" OwnerUserId="18559" LastEditorUserId="18559" LastEditDate="2016-01-09T18:18:52.320" LastActivityDate="2016-01-09T18:18:52.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21766" PostTypeId="4" CreationDate="2016-01-09T07:18:48.563" Score="0" Body="Molecular dynamics simulations aim at solving the equations of motion of the atoms belonging to a molecular system using explicit time propagation and taking into account the effect of temperature on the atomic velocities (as opposite to geometry optimization). The interatomic forces can be described in classical (MM), quantum (QM) or mixed (QM/MM) frameworks." OwnerUserId="18559" LastEditorUserId="18559" LastEditDate="2016-01-09T18:18:41.547" LastActivityDate="2016-01-09T18:18:41.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21768" PostTypeId="1" CreationDate="2016-01-09T13:45:56.357" Score="2" ViewCount="490" Body="&lt;p&gt;I was wondering if anyone could help with understanding the geometric conservation law for moving domains. I came across &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7726/finite-volume-with-cell-averages-vs-cell-totals-for-conservation-equations/7737#7737&quot;&gt;Link1&lt;/a&gt;, and have tried to understood the paper by Farhat et al &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0045782596010286&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link2&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far my understanding is that the two things to consider are 1) the time step to which the cell areas using for integration of flux terms correspond to; 2) the time step at which the mesh velocity ( $\dot{x}$ ) is being evaluated. Finally, the ALE equations must also be satisfied for a uniform flow case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper states a GCL law as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{i}x^{n+1}-A_{i}x^{n}=\int_{t^{n}}^{t^{n+1}}\int_{\partial C_{i}(x)}\dot{x}\overrightarrow{n}d\sigma dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ refers to the area of cells, $C$ refers to the cell areas swept by the  fluxes, t is time, x is the spatial coordinate at the current configuration, and I think $\sigma$ refers to the surface area of integration corresponding $C$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions that I would really appreciate some help with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)  what is $C$ in 1D. Is C referring to the length of the cells or the area, which is set to 1 in 1D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) in the paper (just before equation 16), for a uniform flow, it sets the variables being solved for at different time steps equal to each other. Is that not the definition of steady flow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Finally, is there a way to determine whether a set of numerical results in 1D, obey GCL or not, without actually going through the equations. For instance, my understanding is that the results should stay independent of the moving domains, so if I compared the results obtained using  $\dot{x}$ =0 and $\dot{x} \neq 0$ and showed that they were not the same?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are any simpler papers or examples on GCL, please let me know. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="17362" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-01-11T10:25:56.360" Title="a few questions on understanding geometric conservation law" Tags="|finite-element|fluid-dynamics|finite-volume|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21769" PostTypeId="1" AcceptedAnswerId="21770" CreationDate="2016-01-09T15:19:28.427" Score="4" ViewCount="114" Body="&lt;p&gt;So I'm given a solver that can solve for $x$ in the matrix equation $\underset{=}{A} \underline{x} = \underline{b}$ where $b$ can be anything we specify. (NB: A is an NxN matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now want to find the inverse of matrix A, $\underset{=}{A}^{-1}$, using the solver. How would i go about doing this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first two ideas were this, but I'm not sure if they are relevant:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Find the LU decomposition somehow using this solver, and then finding the inverse of U and L should be simple because of their shape and thus the inverse of A should be easy to calculate through the product of the inverses of U and L.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can i use the fact that $A = R\Lambda R^{-1}$ and subsequently find the eigenvectors and eigenvalues somehow using the equation above?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thank you very much :)&lt;/p&gt;&#xA;" OwnerUserId="18797" LastActivityDate="2016-01-09T15:38:38.353" Title="Finding the matrix inverse given a solver for the matrix equation $Ax=b$" Tags="|matrix|linear-solver|inverse|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21770" PostTypeId="2" ParentId="21769" CreationDate="2016-01-09T15:38:38.353" Score="8" Body="&lt;p&gt;Your two ideas make it much too complicated. If $X$ is the inverse of $A$, &#xA;$$ AX=I, $$&#xA;and $x_i$ is the $i$-th column of $X$ and $e_i$ is the $i$-th column of the identity matrix $I$ ($e_i$ is a vector of all zeros except with $1$ in the $i$-th location), then the columns $x_i$ of the inverse are defined by&#xA;$$ Ax_i = e_i. $$&#xA;All you need to do is solve $n$ systems of linear equations with the vectors $e_i$ in the r.h.s..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After all, think of what it means to represent a (general, black-box) linear operator in a given basis. If the basis is $e_i$, and all you can do is compute the function $x\mapsto Ax$, the matrix elements can be computed as&#xA;$$ A_{ij} = e_i^\top (Ae_j). $$&#xA;In your case, the function is $x\mapsto A^{-1}x$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-01-09T15:38:38.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21771" PostTypeId="1" CreationDate="2016-01-09T19:37:40.347" Score="0" ViewCount="578" Body="&lt;p&gt;On page 18 on this text: &lt;a href=&quot;http://www.dima.uniroma1.it/users/lsa_adn/MATERIALE/FDheat.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.dima.uniroma1.it/users/lsa_adn/MATERIALE/FDheat.pdf&lt;/a&gt; , &#xA;the graph in figure 8 on this page, how would I write a suitable code in matlab or maple that will produce this graph? It's a graph of the different truncation errors of the different finite difference methods (on the last page of this text there are codes for the different finite difference schemes approximations to the exact solution.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my more immediate necessity, I have the PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_t = u_{xx} +\sin(x+t)-\cos(x+t) , \\ u_x(0,t)=-\sin(t) , \\ u_x(1,t)=-\sin(1+t) , \\ u(x,0)=\cos(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I want to find the truncation error of pdsolve's default method in maple and plot it in maple, I was told it should be a linear graph where the $Y$ axis is between $10^{-4}$ and $10^{-8}$. How to implement this in maple?&lt;/p&gt;&#xA;" OwnerUserId="17431" LastEditorUserId="7567" LastEditDate="2016-01-10T02:57:11.497" LastActivityDate="2016-12-24T15:28:40.003" Title="Looking for a matlab/maple code for plotting the truncation error" Tags="|pde|numerical-analysis|parabolic-pde|maple|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21772" PostTypeId="5" CreationDate="2016-01-10T09:36:05.993" Score="0" Body="&lt;p&gt;Gmsh is a free 3D finite element grid generator with a built-in CAD engine and post-processor. Its design goal is to provide a fast, light and user-friendly meshing tool with parametric input and advanced visualization capabilities. Gmsh is built around four modules: geometry, mesh, solver and post-processing. The specification of any input to these modules is done either interactively using the graphical user interface or in ASCII text files using Gmsh's own scripting language.&lt;/p&gt;&#xA;" OwnerUserId="18725" LastEditorUserId="18725" LastEditDate="2016-01-15T01:26:18.103" LastActivityDate="2016-01-15T01:26:18.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21773" PostTypeId="4" CreationDate="2016-01-10T09:36:05.993" Score="0" Body="A three-dimensional finite element mesh generator with built-in pre- and post-processing facilities" OwnerUserId="18725" LastEditorUserId="18725" LastEditDate="2016-01-11T16:21:59.443" LastActivityDate="2016-01-11T16:21:59.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21775" PostTypeId="2" ParentId="21771" CreationDate="2016-01-10T13:40:22.497" Score="1" Body="&lt;p&gt;Just for creating the plot, here's the MATLAB code to generate a graph similar to the one you cited (I've mocked up the data)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Dx = 2.^-(2:8);&#xA;&#xA;% These errors to be generated by your code: mocked up here&#xA;E_FTCS = Dx.^1.7;&#xA;E_BTCS = 0.9*Dx.^1.8;&#xA;E_CN = Dx.^1.75;&#xA;&#xA;E_ideal = Dx.^2;&#xA;&#xA;% Create the plot&#xA;loglog(Dx,E_FTCS,'ko',Dx,E_BTCS,'k*',...&#xA;       Dx,E_CN,'ks',Dx,E_ideal,'k--')&#xA;&#xA;legend({'FTCS','BTCS','CN','ideal'},'Location','northwest')&#xA;xlabel('\Delta x')&#xA;ylabel('Error: ||u-u_e||_2')&#xA;title('\Delta t = 5.0e-6 (constant)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output graph with MATLAB version R2014b:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3CZ9n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3CZ9n.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17671" LastActivityDate="2016-01-10T13:40:22.497" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21777" PostTypeId="1" AcceptedAnswerId="21780" CreationDate="2016-01-11T01:20:06.030" Score="3" ViewCount="8336" Body="&lt;p&gt;I have the following function that has been vectorized so that for every element in input array t, an array is output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@np.vectorize&#xA;def Ham(t):&#xA;    d=np.array([[np.cos(t),np.sqrt(t)],[0,1]],dtype=np.complex128)&#xA;    return d&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting an error: &lt;code&gt;TypeError: only length-1 arrays can be converted to Python scalars&lt;/code&gt;. I believe this error happens when there is conflicts with numpy and python, but I can't see where that would be. Also, if the dtype is not given, the error is &lt;code&gt;ValueError: setting an array element with a sequence.&lt;/code&gt; I'm guessing you can't assign arrays inside arrays in python, but how else can I do this?&lt;/p&gt;&#xA;" OwnerUserId="18832" LastActivityDate="2016-01-11T19:17:32.577" Title="Python Vectorizing a Function Returning an Array" Tags="|python|numpy|special-functions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21779" PostTypeId="2" ParentId="21768" CreationDate="2016-01-11T08:29:28.987" Score="2" Body="&lt;p&gt;1) In 3D, C is a swept volume, in 2D a swept area and in 1D a swept length, i.e. the change in length of a cell between $t^n$ and $t^{n+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Uniform flow is steady, the fact that the grid deforms implies that the simulation is unsteady, but if the GCL is satisfied then the flow itself shouldn't change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Yes, that would be enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, the first (and easiest) paper to introduce the idea of swept volumes is &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/fld.1650080906/full&quot; rel=&quot;nofollow&quot;&gt;Demirdzic &amp;amp; Peric, 1988&lt;/a&gt; for incompressible flows and finite volumes. It seems to have been reinvented later in the context of compressible flows and finite elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999108006165&quot; rel=&quot;nofollow&quot;&gt;Etienne e.a. 2009&lt;/a&gt;, you will find a nice overview. They introduce 3 levels of GCL compliance: exact solution of no-flow, exact solution of uniform flow and identical grid/timestep convergence rates on fixed and deforming grids. Manufactured solutions are used to verify that a code complies.&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2016-01-11T08:29:28.987" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21780" PostTypeId="2" ParentId="21777" CreationDate="2016-01-11T09:46:35.633" Score="6" Body="&lt;p&gt;The problem is that &lt;code&gt;np.cos(t)&lt;/code&gt; and &lt;code&gt;np.sqrt(t)&lt;/code&gt; generate arrays with the length of &lt;code&gt;t&lt;/code&gt;, whereas the second row (&lt;code&gt;[0,1]&lt;/code&gt;) maintains the same size.&#xA;To use &lt;a href=&quot;http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.repeat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;np.vectorize&lt;/code&gt;&lt;/a&gt; with your function, you have to define the output type, and &lt;a href=&quot;http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.repeat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;np.vectorize&lt;/code&gt;&lt;/a&gt; &#xA;&lt;a href=&quot;https://stackoverflow.com/a/14986811/1771479&quot;&gt;isn't really meant as a decorator except for the simplest cases&lt;/a&gt;.&#xA;In &lt;a href=&quot;https://stackoverflow.com/a/24925537/1771479&quot;&gt;this way&lt;/a&gt; however you can generate the function with the right type:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def Ham(t):&#xA;    d=np.array([[np.cos(t),np.sqrt(t)],[0,1]],dtype=np.complex128)&#xA;    return d&#xA;&#xA;HamVec = np.vectorize(Ham, otypes=[np.ndarray])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you can use &lt;code&gt;HamVec&lt;/code&gt; as a function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; x=np.array([1,2,3])&#xA;&amp;gt;&amp;gt;&amp;gt; HamVec(x)&#xA;array([ array([[ 0.54030231+0.j,  1.00000000+0.j],&#xA;       [ 0.00000000+0.j,  1.00000000+0.j]]),&#xA;       array([[-0.41614684+0.j,  1.41421356+0.j],&#xA;       [ 0.00000000+0.j,  1.00000000+0.j]]),&#xA;       array([[-0.98999250+0.j,  1.73205081+0.j],&#xA;       [ 0.00000000+0.j,  1.00000000+0.j]])], dtype=object)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.repeat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;np.vectorize&lt;/code&gt;&lt;/a&gt;  &lt;a href=&quot;https://stackoverflow.com/a/3379505/1771479&quot;&gt;is just a convenience function, it doesn't actually make code run any faster&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;This question might have been more suitable for &lt;a href=&quot;http://stackoverflow.com&quot;&gt;StackOverflow&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; as an answer to the follow up question: the resulting values of the matrix are of type &lt;code&gt;numpy.complex128&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; y = HamVec(x)&#xA;&amp;gt;&amp;gt;&amp;gt; type(y[0][0][0])&#xA;&amp;lt;type 'numpy.complex128'&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you can do for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; y*np.complex('3+2j')&#xA;array([ array([[ 1.62090692+1.08060461j,  3.00000000+2.j        ],&#xA;       [ 0.00000000+0.j        ,  3.00000000+2.j        ]]),&#xA;       array([[-1.24844051-0.83229367j,  4.24264069+2.82842712j],&#xA;       [ 0.00000000+0.j        ,  3.00000000+2.j        ]]),&#xA;       array([[-2.96997749-1.97998499j,  5.19615242+3.46410162j],&#xA;       [ 0.00000000+0.j        ,  3.00000000+2.j        ]])], dtype=object)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18227" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-01-11T19:17:32.577" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21781" PostTypeId="1" AcceptedAnswerId="21851" CreationDate="2016-01-11T10:54:35.143" Score="5" ViewCount="2724" Body="&lt;p&gt;I am using the Newton's method to solve $3\times3$ systems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some particular cases, it turns out that at a given iteration, the Jacobian matrix cannot be inverted and that its determinant is very close to zero (looking at the matrix, there are terms that are around &lt;code&gt;1e+0&lt;/code&gt; and others that are &lt;code&gt;1e-15&lt;/code&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After investigations, it is clear that one variable has no influence on the system when it is close to the solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most clever thing to do with such an issue ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to have an algorithm that can adapt itself to such situations when they happen.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;EDIT&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is about an optical optimisation problem. The point is to add a surface to an optical system so that it fits some optical properties. The Newton's method finds the roots of a function that takes as input the parameters of the surface and outputs the differences between the optical properties computed and the targeted properties. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that if the system is complex enough, then we have convergence. But if the system is too simple, the surface is more spherical and the Jacobian goes to very small values, because some parameters like astigmatism axis become influence-less. &lt;/p&gt;&#xA;" OwnerUserId="18836" LastEditorUserId="18836" LastEditDate="2016-01-18T12:55:22.663" LastActivityDate="2016-01-18T14:08:06.243" Title="Newton's method goes to zero determinant Jacobian" Tags="|nonlinear-equations|convergence|newton-method|jacobian|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21782" PostTypeId="1" AcceptedAnswerId="21785" CreationDate="2016-01-11T11:28:33.550" Score="5" ViewCount="1118" Body="&lt;p&gt;In my research, one of the steps is to choose a numerical method to estimate $\int_a^b f(t)dt$, where $f$ is Lipschitz continuous but not differentiable. For simplicity, I used &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectangle_method&quot;&gt;midpoint rule&lt;/a&gt; but the overall result is not that good. I was thinking whether the problem came from the fact that I didn't select a correct numerical quadrature rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on my understanding, to guarantee convergence, most deterministic quadrature rules require the integrand be certain order differentiable. Since in this case, $f$ is not smooth enough, it's reasonable that the mid-point rule can't guarantee convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My supervisor suggested me to use Monte-Carlo integration. But I am not sure about the convergence condition for this numerical method. I only know regardless of the dimension, this method has a $\frac{1}{\sqrt{N}}$ convergence order. Does this result hold for all $L^1$ function? Do we need any additional assumption for integrand $f$ for the convergence? Could you provide me some resources about the convergence analysis for Monte Carlo integration?&lt;/p&gt;&#xA;" OwnerUserId="5527" LastActivityDate="2016-01-11T15:58:46.943" Title="Convergence of Monte Carlo integration" Tags="|numerical-analysis|monte-carlo|integration|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21783" PostTypeId="1" AcceptedAnswerId="21789" CreationDate="2016-01-07T21:16:17.767" Score="8" ViewCount="209" Body="&lt;p&gt;I am dealing with an optimization problem that has a large number of variables to optimize over - for example let's call these variables $x$, $y$, and $z$ and I wish to minimize the function $f(x,y,z)$. The optimization method that I am using is not able to handle optimizing over all the variables at once. I am instead simplifying the problem by optimizing over a single variable at a time while keeping the other variables fixed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I.e. I fix $y=y_0$, and $z=z_0$ and then optimize the function only over $x$. This 1D optimization yields some optimal value $x^*$. I then fix $x=x^*$, $z=z_0$, then optimize over $y$. I realize that this doesn't necessarily provide me with a globally optimal solution but it should yield a local minimum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering what the name of this method is and where I can find any information about it. Also if there is a more appropriate community to ask, please let me know. Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; the optimization is conducted over $x$, then $y$, then $z$, then $x$, and so on until the solution converges.&lt;/p&gt;&#xA;" OwnerUserId="18841" OwnerDisplayName="user69813" LastEditorUserId="9667" LastEditDate="2016-01-11T20:53:09.297" LastActivityDate="2016-01-12T16:47:51.153" Title="Name of an Optimization Approach to Reduce Size of Variable Space" Tags="|optimization|terminology|" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21784" PostTypeId="2" ParentId="21783" CreationDate="2016-01-10T18:18:24.597" Score="4" Body="&lt;p&gt;The approach you described originally (only one iteration optimizing in each of the three variables x,y,z ) is not guaranteed to converge to the optimal solution unless F(x,y.z) is variable separable into univariate functions.  Therefore, what you describe is not technically an optimization &quot;method&quot;, but an optimization &quot;heuristic&quot;, similar to an operator splitting method or alternating direction implicit (ADI) methods.&lt;/p&gt;&#xA;" OwnerUserId="549" OwnerDisplayName="Paul" LastEditorUserId="549" LastEditDate="2016-01-12T16:47:51.153" LastActivityDate="2016-01-12T16:47:51.153" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21785" PostTypeId="2" ParentId="21782" CreationDate="2016-01-11T15:58:46.943" Score="3" Body="&lt;p&gt;Let the MC integral estimate be&#xA;$$ S_n = \frac{b-a}{n}\sum_{1\leq k\leq n} f(x_k), $$&#xA;where $x_k$ are i.u.d. on the interval $[a,b]$. So long as the function $f$ is $L^1$, the mean exists, and&#xA;$$\mathbb{E}[S_n] = I, \qquad I = \int_a^b f(x)\,dx, $$&#xA;so by the law of large numbers $S_n$ will converge to $I$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the $1/\sqrt{n}$ convergence, the function also needs to be square-integrable:&#xA;$$ \mathbb{V}[S_n] = \frac{|b-a|}{n}\int_a^b \big(f(x)-\bar f\big)^2\,dx, $$&#xA;(where $\bar f$ is the function average on $[a,b]$), so if $f$ is also $L^2$, then the central limit theorem applies, and $S_n$ would be normally distributed with mean $I$ and variance $n^{-1}|b-a|\mathbb{V}[f]$. When the function is &lt;em&gt;not&lt;/em&gt; square integrable, the law of large numbers still applies, but this error distribution is no longer valid, it would have a fat tail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, non-differentiable functions are not &lt;em&gt;that&lt;/em&gt; bad for numerical integration, so long as the singularities are easy to locate, and the algorithm performs some kind of an adaptive interval bisection procedure that isolates the singularities. Alternatively, a common trick is to split the interval by hand so that all singularities are at subinterval endpoints, and invoke quadrature schemes on individual subintervals. I believe most mature quadrature libraries can handle reasonable singularities quite well. In particular, one of the best quadrature schemes, the double-exponential rule, has no problems at all with regular singularities at the interval endpoints. Others, like Gauss-Legendre rules, usually have modified forms to address singularities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't say what your function $f$ is, but my opinion would be that it has to be pretty pathological (e.g., not differentiable anywhere) before MC methods would outperform a mature quadrature scheme in 1d. Not midpoint, though — you'd never expect that to work well.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-01-11T15:58:46.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21786" PostTypeId="1" CreationDate="2016-01-11T16:14:19.597" Score="0" ViewCount="625" Body="&lt;p&gt;I am trying to solve the CDR-Equation in 2D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial c(x,y)}{\partial t} +  \nabla \cdot ( -d\nabla c(x,y) + \vec{v}(x,y) c(x,y))+ a c(x,y)=0\,,$$&#xA;with Boundary Conditions (length of square is $L$):&#xA;$$c(0,y)=0$$&#xA;$$-d\nabla c(L,y) + v(L,y)  (c(L,y))=0\,.$$.&#xA;$$ c(x,y=0)=0$$&#xA;$$ c(x,y=L)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) Why exactly is this equation stiff? Does it depend on the reaction term $ac(x,y)$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) Can I solve the equation with the Crank-Nicholson method? Is the error huge? If yes, what is the best method?&lt;/p&gt;&#xA;" OwnerUserId="17890" LastEditorUserId="17890" LastEditDate="2016-01-12T09:09:21.203" LastActivityDate="2016-01-12T12:23:48.370" Title="Convection diffusion reaction equation (stiffness, solver)" Tags="|pde|stiffness|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21788" PostTypeId="1" AcceptedAnswerId="21790" CreationDate="2016-01-11T17:50:38.827" Score="0" ViewCount="80" Body="&lt;p&gt;I am working on a program that tracks a flying object through space and predicts the future position of said object.  I was given some equations to use, but some of them do not look right, mainly the equation I was given for finding slope:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;slope = (N * sum(t*GCx) - sum(t) * sum(GCx)) / (N * sum(t^2) - sum(t) * sum(t))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would have thought the slope equation to use in the algorithm would have been&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sum for all points((t-average t)*(GCx-average GCx))/sum for all points (t-average t)^2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So which is it and why the difference?  Bare in mind that this will be applied to y and z coordinates later as well.  t is time.&lt;/p&gt;&#xA;" OwnerUserId="16717" LastActivityDate="2016-01-12T00:00:32.693" Title="For finding the track of an object through space(3d) over time, what is the correct slope equation to use in the algorithm?" Tags="|algorithms|regression|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21789" PostTypeId="2" ParentId="21783" CreationDate="2016-01-11T19:17:39.650" Score="9" Body="&lt;p&gt;This is &lt;a href=&quot;https://en.wikipedia.org/wiki/Coordinate_descent&quot;&gt;coordinate descent&lt;/a&gt;. I believe it's used on very large-scale problems when other methods like gradient descent might be too slow (e.g., &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/100802001&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/100802001&lt;/a&gt;). It should converge to a local minimum, but it also would require more steps than something like gradient descent or Newton-type methods.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-01-11T19:17:39.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21790" PostTypeId="2" ParentId="21788" CreationDate="2016-01-12T00:00:32.693" Score="3" Body="&lt;p&gt;Given you are trying to find a path $f(t) = a + b t$ for each 3D component of an object to define its trajectory, you can formulate a Least Square problem to find the values for $a$ and $b$ based on $N$ pieces of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of the least square problem is to minimize the following cost function with respect to $a$ and $b$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J = \frac{1}{2} \sum_{i=1}^{N} ( a + b t_i - y_i )^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;given you are provided data pairs $(t_i,y_i) \forall i \in [1,N]$ and that $y_i$ represents the object's coordinate in which ever dimension you are fitting for (because the cost function above is used for each dimension independently).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you take the derivative of the cost function with respect to $a$ and $b$ and set the two resulting equations equal to 0, you should get the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$aN + b\sum_{i=1}^N t_i = \sum_{i=1}^N y_i$$&#xA;$$a\sum_{i=1}^N t_i + b\sum_{i=1}^N (t_i)^2 = \sum_{i=1}^N y_i t_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving these equations gives you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ a = \frac{(\sum_{i=1}^N (t_i)^2) (\sum_{i=1}^N y_i) - (\sum_{i=1}^N t_i) (\sum_{i=1}^N y_i t_i)}{N\sum_{i=1}^N (t_i)^2 - (\sum_{i=1}^N t_i)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ b = \frac{N\sum_{i=1}^N y_i t_i - (\sum_{i=1}^N t_i)(\sum_{i=1}^N y_i)}{N\sum_{i=1}^N (t_i)^2 - (\sum_{i=1}^N t_i)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $b$ is equivalent to your slope, you can see this equation I derived fits what you were given. This shows you just need to solve this like a typical Least Square problem. If you find a difference between the equations given and the alternate you mentioned, it's because the alternate isn't found using the Least Square approach.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-01-12T00:00:32.693" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21791" PostTypeId="1" CreationDate="2016-01-12T11:49:10.903" Score="1" ViewCount="271" Body="&lt;p&gt;Are there any one dimensional hyperbolic test cases for a 1D uniform flow?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I need the test case for the purpose of showing the satisfaction of geometric conservation law on a moving mesh). &lt;/p&gt;&#xA;" OwnerUserId="17362" LastActivityDate="2016-11-07T22:25:28.777" Title="1D uniform flow test case for compressible flow" Tags="|finite-element|finite-difference|fluid-dynamics|finite-volume|testing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21792" PostTypeId="2" ParentId="21786" CreationDate="2016-01-12T12:23:48.370" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Mathematically speaking, stiffness is meaningless for a single differential equation, and is rather attributed to a set of differential equations that have different time-scales (e.g. when trying to solve two coupled equations with time-scales of 1 second and 1 day, respectively). However, a single equation can also be referred to as stiff if certain numerical integration methods are numerically unstable. Source/sink terms can give rise to stiffness, for instance:&#xA;$$\frac{dy}{dt}=-1000y$$&#xA;is a good example where sink term has given rise to stiffness. That being said, $ac(x,y)$ might be causing the stiffness in your PDE.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Theoretically, you can solve any time-dependent PDE with Crank-Nicolson method. However, you should use adaptive time-steps when solving a stiff equation, otherwise the computational cost is huge. The naive Crank-Nicolson formulation is based on fixed time-step, however, you can derive your customized version with adaptive time-step. But, I do not recommend as it is cumbersome. Your best alternative, in my opinion, is to use method of lines. You can find useful information about this method &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. To give you a rough idea, in the method of lines, you discretize your PDE with respect to spatial variables (and not time). This eventually gives you a set of first order ODEs (ordinary differential equation) with respect to time. Now you can simply use a stiff ODE solver (e.g. Adam's Bashforth method) to integrate the set of ODEs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18413" LastActivityDate="2016-01-12T12:23:48.370" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21793" PostTypeId="2" ParentId="21791" CreationDate="2016-01-12T14:04:19.883" Score="1" Body="&lt;p&gt;P.S. The answer is edited after first remark below it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one means by &quot;1D uniform flow&quot; a passive transport where the velocity is constant, i.e. you want to solve the advection equation $u_t + c u_x = 0$, then the test cases typically differ only by different choices of initial conditions, say the function $u(x,0)$, e.g. a constant impulse (i.e. $u$ equals zero everywhere expect on a finite interval where it is equal to 1), a Gaussian (e.g. $u(x,0)=e^{-x^2}$), or a piecewise linear function (e.g. a triangle shape). You can test the geometric property that the measure of support of solution is preserved in time (e.g. the interval where $u(x,t)$ is positive is simply shifted by speed $c$), see e.g. &lt;a href=&quot;http://www.clawpack.org/pyclaw/gallery/gallery_all.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CLAWPACK&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one does not want to solve the advection equation $u_t +c u_x = 0$, but some &quot;geometric transformation&quot;, e.g. to move a mesh, then the constant speed of flow means that any subset of the initial mesh is simply shifted to a new position by the shift $c t$, where $t$ is time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one means by uniform flow in fact the steady flow, then in the 1D case of advection equation it means to solve $u_t+c(x) u_x = 0$, then I have tested one example with $c(x)=2+\sin(x)$. The exact solution for a given initial function has quite complicated form, it can be found sometimes by e.g. Mathematica, but one property can be checked easily for any initial function - the exact solution is periodic in the sense $u(x,\pi/\sqrt3)=u(x-2 \pi,0)$. I include an animated gif, where $u(x,0)=\sin(x)$ for $x \in (0,4 \pi)$ and the animation runs for $t \in (0,\pi/\sqrt3)$ with periodic boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9S6o8.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9S6o8.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might check in such simulations that the position of intervals where e.g. $u(x,t)$ is positive is shifted from its initial position at $t=0$ by factor $2 \pi$ at $t=\pi/\sqrt3$, similarly for intervals where $u(x,0)$ is negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you compute directly a moving mesh with this speed $c(x)=\sin(x) + 2$, then any subset of initial mesh shall be shifted to the right by factor $2 \pi$ after one time period $t=\pi/\sqrt3$.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-01-12T17:10:04.343" LastActivityDate="2016-01-12T17:10:04.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21794" PostTypeId="2" ParentId="8093" CreationDate="2016-01-12T14:06:28.113" Score="3" Body="&lt;p&gt;Here's a clearer way to transform a 6x6 stiffness matrix - treat it as a 3x3x3x3 tensor:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   For k = 1 To 3&#xA;        For l = 1 To 3&#xA;            For s = 1 To 3&#xA;                For t = 1 To 3&#xA;&#xA;                    rotatedTensor(k,l,s,t) = 0&#xA;&#xA;                    For m = 1 To 3&#xA;                        For n = 1 To 3&#xA;                            For p = 1 To 3&#xA;                                For r = 1 To 3&#xA;&#xA;                                    rotatedTensor(k,l,s,t) += tensor(m,n,p,r) * T(k, m) * T(l, n) * T(s, p) * T(t, r)&#xA;&#xA;                                Next&#xA;                            Next&#xA;                        Next&#xA;                    Next&#xA;&#xA;                Next&#xA;            Next&#xA;        Next&#xA;    Next&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;T&lt;/code&gt; is an ordinary 3x3 rotation matrix. The input is &lt;code&gt;tensor&lt;/code&gt; and the output is &lt;code&gt;rotatedtensor&lt;/code&gt;. Wherever 4 indices appear, convert them to the 2-index form used in the stiffness matrix. That way you can store the input and output as 6x6 matrices and just use the 4 indices to make the code more readable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pseudocode above is independent of the particular mapping from 4-indices to 2-indices. You have to apply that mapping yourself at each point where it refers to a tensor using 4 indices. For example, you might define a function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double tensor(a,b,c,d){&#xA;    if(a==1 and b==1 and c==1 and d==1){&#xA;        return matrix(1,1)&#xA;    }&#xA;    if(a==1 and b==1 and c==2 and d==2){&#xA;        return matrix(1,2)&#xA;    }&#xA;    if(a==1 and b==1 and c==3 and d==3){&#xA;        return matrix(1,3)&#xA;    }&#xA;    if(a==1 and b==1 and c==2 and d==3){&#xA;        return matrix(1,4)&#xA;    }&#xA;    etc ...&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11650" LastActivityDate="2016-01-12T14:06:28.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21795" PostTypeId="1" CreationDate="2016-01-12T21:05:23.200" Score="5" ViewCount="271" Body="&lt;p&gt;My first question, please excuse me if its too basic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a matrix of evenly spaced geographical points; say &lt;code&gt;10 x 10&lt;/code&gt;, which I will call &lt;strong&gt;seed&lt;/strong&gt; points. Each seed point has a lat/long and the spacing can vary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I place $n$ random points on the same matrix. I call these &lt;strong&gt;demand&lt;/strong&gt; points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithm can I use to calculate a variable $n$ seed points which will minimize the distance to all the demand points in the matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The user will decide if he/she needs a single seed or multiple seeds to satisfy all the demand points. If a single seed is selected its similar to an average of the demand points, a centroid. But what if multiple seed points need to be selected, say 2. So the problem becomes, which two seed points if selected would give the minimum distance to all the demand points. Each demand point will connect to a single seed and not both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QQgPm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QQgPm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would ideally like is this problem identified as a MIP / LP problem, its name given (i.e. &quot;Travelling Salesman&quot;). Then I can go and research how to solve it using a solver. But that is only if it is possible with a solver.&lt;/p&gt;&#xA;" OwnerUserId="18853" LastEditorUserId="18853" LastEditDate="2016-01-13T15:51:25.367" LastActivityDate="2016-06-11T22:17:33.710" Title="What category is this problem?" Tags="|linear-algebra|optimization|linear-solver|matrix|linear-programming|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="21798" PostTypeId="1" AcceptedAnswerId="23644" CreationDate="2016-01-13T04:01:54.153" Score="1" ViewCount="36" Body="&lt;p&gt;Im doing a computational biology project in which I simulate evolution under different inheritance rulesets and I am generating phylogenetic trees (beautifully visualised in python with ete3, which I recommend and can be found here: &lt;a href=&quot;http://etetoolkit.org/download/&quot; rel=&quot;nofollow&quot;&gt;http://etetoolkit.org/download/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Can someone point me in the right direction to find and test out some simple metrics that can describe these trees in terms of 'branchy-ness' (you can tell i'm not a bioinformatician or phylogeneticist!). I'm looking for kind of mean-field descriptors of the trees. Kind of like degree distribution for networks...&lt;/p&gt;&#xA;" OwnerUserId="18857" LastActivityDate="2016-04-09T13:23:49.780" Title="Help with some starter metrics on phylogenetic trees?" Tags="|computational-biology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21799" PostTypeId="2" ParentId="21546" CreationDate="2016-01-13T06:25:40.177" Score="2" Body="&lt;p&gt;Seeing what are your boundary conditions, you should use an spectral(Galerkin) method.&#xA;I am trying to solve sometrhing similar, but in 9 dimentions, And the way to treat the field fading in infinity as a boundary condition is with galerkin methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably using a basis like $F_k(x)=e^{-\alpha x^2}\sin(k x)$.&#xA;Though you should look more into the probable basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the boundary conditions are implicit in the choise of the basis, since each function already satisfies them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you should do a series expansion of your initial conditions (like in a fourier series, but with the new basis). witch will give you a system of ODE-s for the coeficients of the series expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this is of any help.&lt;/p&gt;&#xA;" OwnerUserId="18687" LastActivityDate="2016-01-13T06:25:40.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21804" PostTypeId="1" CreationDate="2016-01-13T16:44:00.983" Score="0" ViewCount="96" Body="&lt;p&gt;Between these four different meshes for the same shape, compare them whilst considering the different results for displacement and stress. Moreover, the difference in computational cost and accuracy of the meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/j9faF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j9faF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18866" LastEditorUserId="20688" LastEditDate="2019-06-22T19:12:08.957" LastActivityDate="2019-06-22T19:12:08.957" Title="Comparing the following mesh grids for a star-shaped plate" Tags="|finite-element|mesh-generation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21805" PostTypeId="2" ParentId="21804" CreationDate="2016-01-13T17:06:39.410" Score="1" Body="&lt;p&gt;If you put a Lagrange linear element on the first mesh (upper left with non-convex elements), you're probably going to fail to have an invertable Jacobian, and so you'll fail to be able to compute a stiffness matrix for each element, assemble the system, and compute a result altogether. Given your comment, you've probably noticed this already. Otherwise, these meshes are very different and you don't specify what function space(s) you draw your trial and test functions from, so the effective element size and discretization properties are hard for us to comment on. Assuming the lowest order conforming spaces that lead to solutions, I'd guess that the lower left and upper right meshes are a toss-up for cost and accuracy. They have the same number of degrees of freedom. None of these meshes have enough elements to make accuracy worth thinking about in too much detail. A real problem is probably going to want substantially more elements to give you a good solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inner corners of the star could lead to singularities in the solution or its derivatives (like the classic L-shaped domain for the Laplacian/Poisson problem) depending on the boundary conditions which might indicate that the leading constant in the error estimate is large and that low accuracy is to be expected on such coarse meshes. Have you tried uniformly refining these meshes until you reach the asymptotic regime for the problem?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-01-13T17:06:39.410" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21806" PostTypeId="1" CreationDate="2016-01-13T19:08:51.370" Score="1" ViewCount="104" Body="&lt;p&gt;I am accepted to computational science master program. The master program just started to accept students in our university and today each group of 3 students was assigned to an advisor teacher (we are only 13 students). Our advisor asked us to research for valuable topics in Computational Science, so I thought why don't I ask to experienced scientists in this forum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My group is combined of 1 electronics engineer (me), 1 mathematician and 1 computer engineer. So, our strengths are math and software programming.&lt;/p&gt;&#xA;" OwnerUserId="18868" LastEditorUserId="9667" LastEditDate="2016-01-13T19:53:26.973" LastActivityDate="2016-01-13T19:53:26.973" Title="Which topics are hot in computational science?" Tags="|numerical-analysis|parallel-computing|software|" AnswerCount="0" CommentCount="4" ClosedDate="2016-01-14T02:47:41.240" ContentLicense="CC BY-SA 3.0" />
  <row Id="21807" PostTypeId="2" ParentId="21795" CreationDate="2016-01-13T20:04:07.917" Score="1" Body="&lt;p&gt;This is actually a trivial problem to solve for a moderate number $k$ of &quot;stations&quot; (i.e., the subset of seed points that serve the demand points):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Try $k=1$. For each of your $i=1\ldots N=10^2$ seed points, compute the sum of distances between the demand points and a proposed station at $i$: $D^{(1)}_i=\sum_{\nu=1}^n \|x_{\text{demand},\nu} - x_{\text{seed,i}}\|$. Among all possible station locations, choose the location $i^\ast$ that minimizes $D^{(1)}_i$ and call the distances to the best location $D^{(1)}=D^{(1)}_{i^\ast}=\min_i D^{(1)}_i$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try $k=2$. There are $N(N-1)/2$ possible locations for two stations. Let's say they are at seed points $i,j$. Then compute the sum of distances, $D^{(2)}_{ij}=\sum_{\nu=1}^n \min\left\{\|x_{\text{demand},\nu} - x_{\text{seed,i}}\|,\|x_{\text{demand},\nu} - x_{\text{seed,j}}\|\right\}$. Among these possibilities, select those locations $i^\ast,j^\ast$ that minimizes these distances and call the best distances $D^{(2)}=D^{(2)}_{i^\ast j^\ast}=\min_{i,j} D^{(2)}_{ij}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat for $k=3,4,\ldots$. The effort will quickly grow for larger $k$, but you're probably not interested in the case $k=100$ supply stations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then compare $D^{(1)},D^{(2)},\ldots$ to see which one is smallest, and that is the number of stations you need.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-13T20:04:07.917" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21808" PostTypeId="1" AcceptedAnswerId="21810" CreationDate="2016-01-13T20:43:43.940" Score="4" ViewCount="450" Body="&lt;p&gt;I have a rudimentary question about the differences between finite difference (FDM) and finite volume methods (FVM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In FDM we concentrate on the nodes (points) in space while in FVM we concentrate in the volumes enclosed by faces. Correct me if I am wrong here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the Gauss Divergence theorem is applied on the integral equation of the FVM, we do not calculate the variables on the volume as a whole but on the faces enclosing the volume/cell. And thus we will be calculating on the points in space (the point being on the mid-point of the face generally). I understand the conservative character is lost during this process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how is this different from a Finite Difference Method, where we would be calculating the variables on points as well. Consider a staggered grid, where the point of calculation of the variables could lie exactly on the mid-points of the faces being used for FVM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am starting to believe that the algebraic formulation will be exactly the same, if you do it using an FVM or FDM method. Am I right in assuming this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please point out any inconsistencies in my observations made above.&lt;/p&gt;&#xA;" OwnerUserId="18870" LastEditorUserId="9667" LastEditDate="2016-01-13T22:18:33.110" LastActivityDate="2016-01-14T00:04:41.977" Title="Are the Finite Difference and Finite Volume Methods different after the application of the Gauss Divergence theorem on the FVM?" Tags="|linear-algebra|pde|finite-difference|finite-volume|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21810" PostTypeId="2" ParentId="21808" CreationDate="2016-01-13T21:52:37.837" Score="3" Body="&lt;p&gt;Yes, many methods can be derived in different ways. For example, some finite volume methods for porous media flow can also be derived as staggered grid finite difference methods, as well as mixed finite element methods using particular quadrature rules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is, maybe, not all that surprising given that all try to approximate the underlying physics in some way. At the same time, it may not always be entire obvious how, for example, one might have to choose the quadrature rule to derive a finite difference stencil from a finite element method. But, if you succeed in doing so, this may open up a new and sometimes simpler way of analyzing the convergence of methods because you suddenly have the option of analyzing a finite volume method through the perspective of the finite element method, plus a perturbation that results from using a particular quadrature.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-13T21:52:37.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21811" PostTypeId="1" CreationDate="2016-01-13T23:40:49.403" Score="3" ViewCount="410" Body="&lt;p&gt;I hope this is not a vague question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if any researchers and students in this community have used or considered using 3D graphics software (such as Blender) or a game engine to visualise scientific data as opposed to specialist software.  I understand there are many limitations to using popular 3D graphics for scientific purposes (such as accuracy of the simulation).  Despite this, I have found studies by researchers using these resources to visualise their data (here's an example &lt;a href=&quot;http://www.nature.com/articles/srep04228&quot; rel=&quot;nofollow&quot;&gt;http://www.nature.com/articles/srep04228&lt;/a&gt;), and I am guessing that the existence of free and open-source software can be a benefit to researchers by being a cheaper alternative to hiring visual research groups or developing specialist programs, and may be quite easy to learn  by comparison.  Please correct me if I am wrong on that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if anybody here has ever made a decision to use 3D graphics software or even a game engine in their workflow, and that might be for the reasons above or entirely different ones.  Furthermore, what were the limitations as well as benefits you found from using software not developed for scientists in mind?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Re:  I realise this question is dangerously close to asking for opinions.  But if it is possible to answer factually by being based on experience, that would be great.&lt;/p&gt;&#xA;" OwnerUserId="18869" LastEditorUserId="18869" LastEditDate="2016-01-13T23:51:42.787" LastActivityDate="2016-01-19T19:24:25.027" Title="Can mainstream 3D software be a useful alternative in scientific visualisation?" Tags="|visualization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21813" PostTypeId="1" AcceptedAnswerId="21820" CreationDate="2016-01-14T04:08:00.120" Score="3" ViewCount="154" Body="&lt;p&gt;We know that in classical two-phase Stefan problems, let's say in the temperature distribution of ice-water problem here, the governing PDEs are:&#xA;\begin{equation}&#xA;\left.&#xA;\begin{aligned}&#xA;C_1\frac{\partial T_1}{\partial t}=\nabla\cdot(k_1\nabla T_1),\ in\ \Omega_1\\&#xA;C_2\frac{\partial T_2}{\partial t}=\nabla\cdot(k_2\nabla T_2),\ in\ \Omega_2\\&#xA;\end{aligned}&#xA;\right\}&#xA;\end{equation}&#xA;which is also usually written in the following form:&#xA;\begin{equation}&#xA;C_i\frac{\partial T}{\partial t}=\nabla\cdot(k_i\nabla T),\ in\ \Omega,\\&#xA;\end{equation}&#xA;where&#xA;\begin{equation}&#xA;k_i (C_i)=\left\{&#xA;\begin{aligned}&#xA;k_1 (C_1)\ in\  \Omega_1\\&#xA;k_2 (C_2)\ in\  \Omega_2\\&#xA;\end{aligned}&#xA;\right.&#xA;\end{equation}&#xA;so that it becomes a PDE with space-dependent coefficient, which seems to be reasonable. But after looking into this equation, I found that in this way I am enforcing additional constraints on temperature derivatives at the phase-change interface. I mean, the temperature derivatives with respect to both time and space are discontinuous and thus is not defined at the interface in the original problem. But when combining them together in a single form, I think we are implicitly violating the above conditions. Therefore, I don't think they can be written in the second form (the combined form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am not sure if there is something inappropriate with my point of view. &lt;strong&gt;Perhaps it's just that I am thinking too much?&lt;/strong&gt; Could you please leave your comments about it? &lt;strong&gt;Any comment is welcome! I would greatly appreciate it!&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="11893" LastEditDate="2017-08-17T15:44:42.380" LastActivityDate="2017-08-17T15:44:42.380" Title="Comments needed on the doubts of PDEs in moving boundary problems" Tags="|pde|elliptic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21814" PostTypeId="1" CreationDate="2016-01-14T08:16:04.697" Score="2" ViewCount="244" Body="&lt;p&gt;I am developing a BEM code based on a &lt;a href=&quot;https://www.dealii.org/8.3.0/doxygen/deal.II/step_34.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;deal.ii tutorial&lt;/a&gt;. Consider the Poisson equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Delta u=-f\,,&#xA;$$&lt;/span&gt;&#xA;and its Green's function &lt;span class=&quot;math-container&quot;&gt;$G\left(\mathbf{x},\mathbf{x}'\right)$&lt;/span&gt; with the property&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Delta G=-\delta\left(\mathbf{x}-\mathbf{x}'\right)\,.&#xA;$$&lt;/span&gt;&#xA;Applying Green's 2nd identity leads to the well known boundary integral equation (BIE)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int\limits_{\Omega}u\left(\mathbf{x}\right)\delta\left(\mathbf{x}-\mathbf{x}'\right)\mathrm{d}V&#xA;=\int\limits_{\Omega}f\left(\mathbf{x}\right)G\left(\mathbf{x},\mathbf{x}'\right)\mathrm{d}V&#xA;+\oint\limits_{\partial\Omega}\mathbf{n}\cdot\left[\cdots\right]\mathrm{d}\Gamma&#xA;$$&lt;/span&gt;&#xA;I am considering the integral on left hand side of the equation. It is well known that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int\limits_{\Omega}u\left(\mathbf{x}\right)\delta\left(\mathbf{x}-\mathbf{x}'\right)\mathrm{d}V=\alpha\left(\mathbf{x}'\right)u\left(\mathbf{x}'\right)\,,&#xA;$$&lt;/span&gt;&#xA;where as the factor &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, i.e. the fraction of the solid angle, depends on the location of the point &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}'$&lt;/span&gt;. Typically it is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\alpha\left(\mathbf{x}\right)=\begin{cases}&#xA;0 &amp;amp; \mathbf{x}\notin\Omega\\&#xA;1 &amp;amp; \mathbf{x}\in\Omega\\&#xA;\frac{1}{2} &amp;amp; \mathbf{x}\in\Gamma &amp;amp; \text{smooth boundary}\\&#xA;\frac{\theta}{4\pi} &amp;amp; \mathbf{x}\in\Gamma &amp;amp; \text{3D corner}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&#xA;You get these expressions by simple analytic evaluations of the integral. If the BIE is discretized, these analytic expressions may be not be appropiate any more. Therefore, the author of the tutorial states that the fraction of solid angle may be calculate by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\alpha\left(\mathbf{x}'\right)=1+\oint\mathbf{n}\cdot\nabla G\left(\mathbf{x},\mathbf{x}'\right)\mathrm{d}\Gamma\,,&#xA;$$&lt;/span&gt;&#xA;and in fact uses this term in the discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally my question: &lt;strong&gt;How can be the equation above can be derived?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="14284" LastEditorUserId="20688" LastEditDate="2019-06-22T19:13:53.333" LastActivityDate="2019-06-22T19:13:53.333" Title="Boundary elements method -- calculation of solid angle" Tags="|poisson|elliptic-pde|integral-equations|deal.ii|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21815" PostTypeId="1" AcceptedAnswerId="21827" CreationDate="2016-01-14T10:01:36.707" Score="3" ViewCount="891" Body="&lt;p&gt;Suppose I have the following integral:&#xA;$$\int_{-1}^1 \int_{-1}^1 C_if(x,y)dxdy$$&#xA;where&#xA;\begin{equation}&#xA;C_i=&#xA;\begin{cases}&#xA;C_1 \quad \text{in }\Omega_1\\&#xA;C_2 \quad \text{in } \Omega_2\\&#xA;\end{cases}&#xA;\end{equation}&#xA;So how can I compute it using Gauss quadrature without having to splitting the unit cell into two seperate parts or re-transforming them into another 2 unit cells. I mean, since the integrand is discontinuous over the unit cell, it seems to be not possible to use Guass quadrature directly, so what other information do I need or what treatment should I take to deal with it? &lt;strong&gt;Is it possible to accomplish this goal?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="18224" LastEditDate="2016-01-19T11:37:47.257" LastActivityDate="2016-01-19T11:37:47.257" Title="Using Gauss quadrature for a discontinuous integrand" Tags="|numerical-analysis|quadrature|integration|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21816" PostTypeId="2" ParentId="21561" CreationDate="2016-01-14T11:32:48.413" Score="10" Body="&lt;p&gt;What one usually wants in this situation is to preserve a discrete analog of time symmetry: namely, if the time discretization is applied to solve first forward and then backward in time, the initial condition is recovered.  This is true if the method is invariant under the following substitutions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Delta t \to -\Delta t$$&#xA;$$ a^{n+j} \to a^{n-j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(here $a^n$ is the numerical approximation of the solution $a(t^n)$, so the second substitution is implied by the first).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will give two examples to illustrate.  The explicit Euler method&#xA;$$a^{n+1} = a^n + \Delta t f(a^n)$$&#xA;does not preserve time symmetry; applied backward in time it becomes the implicit Euler method:&#xA;$$a^n = a^{n-1} + \Delta t f(a^n).$$&#xA;On the other hand the midpoint (or &lt;em&gt;leapfrog&lt;/em&gt;) method&#xA;$$a^{n+1} = a^{n-1} + 2\Delta t f(a^n)$$&#xA;does preserve time-reversal symmetry.  Other well-known methods that preserve time-reversal symmetry include the trapezoidal method and (as mentioned in the comments) the Verlet method.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2016-01-15T07:16:50.497" LastActivityDate="2016-01-15T07:16:50.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21820" PostTypeId="2" ParentId="21813" CreationDate="2016-01-14T19:29:35.817" Score="3" Body="&lt;p&gt;Your statement suggests that you assume that because you have a term $\nabla\cdot k \nabla T$ in the PDE, that the derivatives $\nabla T$ need to be continuous. But that's not true. You take derivatives not of $\nabla T$ but of $k\nabla T$ (i.e., the heat flux), and this quantity is continuous [1]. So it is differentiable almost everywhere, and that's all you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, the temperature is continuous both in time and in space. So you can take derivatives in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where it gets a bit more complicated is if you have a moving interface. In that case, if you are sitting at a fixed point $x$ then the time evolution of the temperature at this point, $T(\cdot,x)$ will be continuous, but will probably have a kink at that time where the interface moves across $x$. That's not a problem: it's just at one time instant, and the temperature remains constant, so the time derivative is not defined at that point, but at least it's not a delta function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] All of the above assumes that the strong form of the PDE makes sense. This may not be the case with reentrant corners, etc, but the fundamental points of the statements above remain true.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-14T19:29:35.817" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21821" PostTypeId="1" CreationDate="2016-01-14T20:31:09.460" Score="7" ViewCount="583" Body="&lt;p&gt;While trying to solve a stochastic Gross-Piaevskii equation I have found a problem that can be tracked down to something buggy occuring in the simplest Schrodinger equation possible:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_t \psi = i \partial^2_x \psi$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has the very simple solution of a plane wave:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\psi(t) = A e^{i k x - i\omega t}$, with $\omega = k^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and my problem is that in some cases I cannot recover this very simple solution numerically. This is due to the fact that my code uses a very standar Fast Fourier Transform method to go to k-space, where the evolution is trivial, and then does the inverse FFT to go back to real space. This method naturally enforces periodic boundary conditions in your simulation grid since you are expanding your solution as a sum of periodic functions. What occurs is that, when the initial condition is a plane wave with a periodicity that does not match the domain size, the algorithm is not able to provide the correct solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I show here an example. The problem is clear in $|\psi(t)|$, which should remain constant:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qKGI8.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qKGI8.jpg&quot; alt=&quot;Evolution of the wavefunction amplitude&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The effect is less dramatic in the phase but still present. One can see some ripples that come from the bottom edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jJe7P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jJe7P.png&quot; alt=&quot;Evolution of the phase&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not surprised by the fact that, if I enforce periodic boundary conditions, computing the evolution of a wave which is not periodic in the simulation domain yields problems. However, the method is very standard and use extensively, with packages like XMDS (xmds.org) employing it by default. Therefore, I am surprised by the fact that I did not found any mention of this method failing to solve such an extremely simple example. My question is, is there something that I'm missing? Should I just get over it and assume that this is not a good method if I expect a solution of this kind? Does someone know a reference where this is documented?&lt;/p&gt;&#xA;" OwnerUserId="18881" LastEditorUserId="9667" LastEditDate="2016-01-14T22:50:26.007" LastActivityDate="2016-01-15T18:35:01.297" Title="Solving a simple Schroedinger equation with Fast Fourier Transforms" Tags="|pde|computational-physics|fourier-analysis|wave-propagation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21823" PostTypeId="2" ParentId="21754" CreationDate="2016-01-15T00:05:47.590" Score="6" Body="&lt;p&gt;Before answering much, I want to point out something very important that this question sounds like it should touch on: the fact that numeric computation is kind of meaningless without a symbolic part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't just dump your problem onto a numeric solver and let it chug unless it's a very, very simple one (even then, someone spent a lot of time at the chalkboard designing that part of your CPU that adds two numbers). Normally, you have to pose your problem in as clear terms as possible, often using a pencil, and let it ferment. Can you simplify it? What's important? Anything that will hurt accuracy? Is it even posed right? If this is for work, is it worth your time? Has someone done it? Can you do better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you pick a method to solve it. Maybe your first go you'll use a canned algorithm. Maybe it leaves something to be desired, and you start to specialize. Maybe your problem works much better with something slightly awkward, like a constraint, and then you have to extend your canned solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often, almost all of the hard work is done by a guy with a pencil, paper, and keyboard; the labor of the computer actually making it real is worth peanuts in comparison. If not, then either the computation is very successful, the work was botched, or it was something that there's just really not a lot of space to do much symbolic work on (like machine learning).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might think that as computers progress this becomes less true, but I disagree. As computers get more powerful we throw harder problems at them, these harder problems require more manual effort to even try to approach. That manual effort is a mix of programming and, yes, symbolic manipulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I don't think anyone doubts the outlook of symbolic computation, except maybe folks hunting for exact solutions to nonlinear problems. I wouldn't respect much a person trying to solve a pendulum's motion symbolically nowadays, unless it was for school.&lt;/li&gt;&#xA;&lt;li&gt;No, symbolic computation can't replace numeric. Think about it, something as simple as making a picture brighter is honestly a numeric computation. Even if it consists of just multiplying millions of numbers by a constant, no human can do it.&lt;/li&gt;&#xA;&lt;li&gt;Yes. Be careful though, just because you have an exact solution doesn't mean it's better; sometimes they're too long or incur inaccuracy (just the other day I replaced an exact integral with a numeric one and got better accuracy).&lt;/li&gt;&#xA;&lt;li&gt;Can't really answer this one. Too different, and yet too intertwined.&lt;/li&gt;&#xA;&lt;li&gt;Yes, I'd say anything worthwhile. Read almost any paper on anything computational.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18155" LastEditorUserId="4249" LastEditDate="2016-01-17T23:31:41.737" LastActivityDate="2016-01-17T23:31:41.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21824" PostTypeId="1" CreationDate="2016-01-15T04:00:28.647" Score="2" ViewCount="1436" Body="&lt;p&gt;I am trying to understand the divide and conquer division algorithm that &lt;a href=&quot;https://gmplib.org/manual/Divide-and-Conquer-Division.html&quot; rel=&quot;nofollow&quot;&gt;is used in the GMP bignum arithmetic library&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is very optimised and that makes it somewhat hard to understand. the doc does not really offer an explanation of how it is done, (and the references do not point to an actual algorithm, not one that I could see anyway).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the concept of divide and conquer for sorting, but I am not sure I understand how it can be used in a division.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can $x \over y$ be calculated using divide and conquer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone give a small example of how it is done?&lt;/p&gt;&#xA;" OwnerUserId="18885" LastEditorUserId="18885" LastEditDate="2016-01-15T04:46:04.697" LastActivityDate="2016-01-15T04:46:04.697" Title="Divide and Conquer division algorithm explained (as used in GMP bignum)" Tags="|optimization|algorithms|computer-arithmetic|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21825" PostTypeId="2" ParentId="21699" CreationDate="2016-01-15T05:59:58.430" Score="12" Body="&lt;p&gt;I posted some benchmarks here:&#xA;&lt;a href=&quot;http://www.cecm.sfu.ca/%7Erpearcea/mgb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cecm.sfu.ca/~rpearcea/mgb.html&lt;/a&gt; (&lt;a href=&quot;https://web.archive.org/web/20201202185136/http://www.cecm.sfu.ca/%7Erpearcea/mgb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;archived copy&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;These are for total degree orders.  To solve systems you typically need to do more work.  Timings are for a typical midrange desktop as of 2015 (Haswell Core i5 quad core).&lt;/p&gt;&#xA;&lt;p&gt;The fastest system on one core is Magma, which uses floating point arithmetic and SSE/AVX.  Magma is the strongest system because it has good implementations of FGLM and the Groebner walk (not tested).  These algorithms are used to convert a total degree basis to a lexicographical basis which has a triangular form.  Then you would typically factor polynomials in the lowest variables.&lt;/p&gt;&#xA;&lt;p&gt;mgb is the C library in Maple 2016 which implements the F4 algorithm for total degree and elimination orders.  Its performance is comparable to Magma when it uses multiple cores.&lt;/p&gt;&#xA;&lt;p&gt;FGb is Faugere's implementation of F4.  The version tested here is from his website, and it is faster than the version in Maple.&lt;/p&gt;&#xA;&lt;p&gt;Giac is an open source system with an implementation of F4.  There is a paper describing it &lt;a href=&quot;http://arxiv.org/abs/1309.4044&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://arxiv.org/abs/1309.4044&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Singular is an open source system for many computations in algebraic geometry.  The benchmarks here use &amp;quot;modStd&amp;quot; which is a multi-modular version of the Buchberger algorithm.  You can see the Buchberger algorithm is not competitive with F4.  The basic reason is that F4 amortizes the cost of all the monomial operations.  Aside from that, Singular has reasonably good implementations of FGLM and the Groebner Walk, as well as other algorithms useful for solving.&lt;/p&gt;&#xA;" OwnerUserId="18890" LastEditorUserId="-1" LastEditDate="2021-06-29T01:44:46.697" LastActivityDate="2021-06-29T01:44:46.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21827" PostTypeId="2" ParentId="21815" CreationDate="2016-01-15T11:32:14.233" Score="2" Body="&lt;p&gt;Your question may sound like &quot;one can not use Gauss quadrature for discontinuous integrand&quot; that is not necessary a case. The problem is that a straightforward application of it (without taking into account the discontinuity) can give unsatisfactory accuracy (e.g. adding more quadrature points decrease only slowly the quadrature error). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get an answer that will fit your goal, you should specify in your question  how your sets $\Omega_i$ are described. I can give a reference for one particular case. I am aware of such integration when using level set methods, where an interface can split a domain Omega into two parts, so the sets $\Omega_i$ are given implicitly by a sign of the level set function, because the interface itself is given by the zero level set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you can construct a smooth function $\phi$ (especially that $\nabla \phi$ is defined almost everywhere) such that $\Omega_1 = \{ x : \phi(x) &amp;lt; 0 \}$ and $\Omega_2 = \{x : \phi(x)&amp;gt;0\}$ then search in Google or in scientific databases for terms like &quot;quadrature methods implicitly defined volumes&quot;. I can not recommend a specific publication, because I have not practical experiences with this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly the so called Volume of Fluid Method has to deal with such problem without using a level set function, but a &quot;characteristic functions&quot; for $\Omega_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I wrote before, I did not apply numerical quadrature of this type before and I am only aware of the fact that such problem has been solved many times in level set methods, therefore I do not want to recommend any particular algorithm and I suggest to search for variants suitable to particular needs. Just a quick search gave me this recent &lt;a href=&quot;http://math.lbl.gov/~saye/96629.pdf&quot; rel=&quot;nofollow&quot;&gt;PDF file&lt;/a&gt; being:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;High-Order Quadrature Methods for Implicitly Defined Surfaces and Volumes in Hyperrectangles&lt;/em&gt;&#xA;R. I. Saye&#xA;SIAM Journal on Scientific Computing 2015 37:2, A993-A1019 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly checking it, it claims that the quadrature points (and corresponding positive weights) can be found such that they lie strictly in one subdomain using the information given by level set function. This is of course the direction one should follow in level set methods.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-01-15T13:47:31.237" LastActivityDate="2016-01-15T13:47:31.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21828" PostTypeId="2" ParentId="21821" CreationDate="2016-01-15T18:35:01.297" Score="1" Body="&lt;p&gt;The fundamental solution of the heat equation &#xA;$$ u_t = iu_{xx} $$&#xA;with periodic boundary conditions on $[0,1]$&#xA;can be expressed in closed form in terms of a &lt;a href=&quot;http://mathworld.wolfram.com/JacobiThetaFunctions.html&quot; rel=&quot;nofollow&quot;&gt;Jacobi theta function&lt;/a&gt;,&#xA;$$ \vartheta_3(x, -4\pi t) = 1 + 2\sum_{n\geq1} e^{-4\mathrm{i}\pi^2n^2t}\cos 2\pi n x. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if the initial conditions are discontinuous in the domain (as in the case of a non-periodic IC $e^{ikx}$ with $k$ not an integer multiple of $2\pi$), the solution you should be comparing your numerical results to is not the single wave $e^{i k x-ik^2t}$, which only satisfies the boundary conditions for those specific $k$, but the convolution with the fundamental solution,&#xA;$$ \int_0^1 u_0(y)\vartheta_3(x-y, -4\pi t)\,dy. $$&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-01-15T18:35:01.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21831" PostTypeId="1" CreationDate="2016-01-16T12:56:29.897" Score="1" ViewCount="1575" Body="&lt;p&gt;I am looking for a simple subroutine in Fortran 90 (GNU Compiler) to solve linear equation of the type $AX=B$, where $A$ is an $n\times n$ symmetric matrix stored in the form of symmetric skyline matrix.&#xA;I want a solution of this type:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = skysolve(A,p,X,B,n)&#xA;A = {11,22,13,0,33,24,34,44,55,16,0,0,46,56,66}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;p&lt;/code&gt; is the index of diagonal elements &lt;code&gt;p = {1,2,5,8,9,15}&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be better if I could just have a simple subroutine kind of solution to this situation rather than a library.&lt;/p&gt;&#xA;" OwnerUserId="18908" LastEditorUserId="1804" LastEditDate="2016-01-16T14:23:49.380" LastActivityDate="2016-01-18T17:15:40.723" Title="Skyline solver for AX=B where A is symmetric skyline matrix" Tags="|linear-solver|fortran|data-storage|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21833" PostTypeId="2" ParentId="21831" CreationDate="2016-01-16T14:28:58.227" Score="3" Body="&lt;p&gt;I suggest taking a look at the FEAPpv finite element code written by Robert&#xA;Taylor to accompany this book:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1856176339&quot; rel=&quot;nofollow&quot;&gt;http://www.amazon.com/The-Finite-Element-Method-Fundamentals/dp/1856176339&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can download the fortran source code from this page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ce.berkeley.edu/projects/feap/feappv/&quot; rel=&quot;nofollow&quot;&gt;http://www.ce.berkeley.edu/projects/feap/feappv/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code contains two subroutines, &lt;code&gt;datri&lt;/code&gt; and &lt;code&gt;dasol&lt;/code&gt; that perform the factorization and solution, respectively, of a matrix stored in skyline format; you first call &lt;code&gt;datri&lt;/code&gt; to the factor the matrix and then call &lt;code&gt;dasol&lt;/code&gt; to perform the forward and backward substitutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are sufficiently detailed comments in both subroutines to be able to use them. Here is a simple example with five equations and a symmetric matrix (entries below the diagonal are not included):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      program test&#xA;C&#xA;C  Solve A*U = b      &#xA;C        [2   -2    0    0   -1]&#xA;C        [0    3   -2    0    0]&#xA;C  A =   [0    0    5   -3    0]&#xA;C        [0    0    0   10    4]&#xA;C        [0    0    0    0   10] &#xA;C&#xA;C The solution, U, is: 636.,619.,292.,74.,34.&#xA;C&#xA;        parameter (neqs = 5)&#xA;        real*8 au(7), ad(neqs), b(neqs), energy&#xA;        integer jp(neqs)&#xA;        data au/-2.,-2.,-3.,-1.,0.,0.,4./&#xA;        data ad/2.,3.,5.,10.,10./&#xA;        data b/0.,1.,0.,0.,0./&#xA;        data jp/0,1,2,3,7/&#xA;        call datri(0, au, ad, jp, neqs, neqs)&#xA;        call dasol(0, au, ad, b, jp, neqs, neqs, energy)&#xA;        write(6,*) 'solution', (b(i), i=1,neqs)&#xA;      end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2016-01-17T13:56:39.863" LastActivityDate="2016-01-17T13:56:39.863" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21838" PostTypeId="1" AcceptedAnswerId="21839" CreationDate="2016-01-17T09:46:36.363" Score="3" ViewCount="144" Body="&lt;p&gt;In the FEM we usually use a trick to find the coefficients for shape functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding $M^{-1}$ in the system $MC=I$, where for example in the linear case:&#xA;$$&#xA;\begin{aligned}&#xA;M&amp;amp;=[1 ,x_1,y_1;1 ,x_2,y_2;1 ,x_3,y_3],\\&#xA;C&amp;amp;=[a_1, b_1,c_1;a_2, b_2,c_2;a_3, b_3,c_3],\\&#xA;I&amp;amp;=[1,0,0;0,1,0;0,0,1].&#xA;\end{aligned}&#xA;$$&#xA;and $\phi_1=a1+b_1x_1+c_1y_1$ and also $\phi_2,\phi_3$ are defined in the same way. Since $MC=I$ so $M$ is inverse of $C$. But how do we ensure that $M$ is invertible for a higher degree basis?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="1804" LastEditDate="2016-01-17T10:56:07.003" LastActivityDate="2016-01-17T21:23:42.537" Title="Uniqueness of coefficients for shape functions of higher-order finite elements" Tags="|finite-element|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21839" PostTypeId="2" ParentId="21838" CreationDate="2016-01-17T10:02:47.877" Score="3" Body="&lt;p&gt;This is ensured by choosing the degrees of freedom appropriately. (That's why you need, e.g., additional degrees of freedom on the edges of a triangle for quadratic elements.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, constructing a (local) finite element space consists of choosing a reference element $K$ (e.g., a triangle) and a polynomial space $P$ (e.g., bivariate polynomials on $K$ of total order up to $2$). You then need to find a set of conditions (e.g., point evaluation in some $x_i\in K$) to make the interpolation problem &quot;for given $f\in C(K)$, find $p\in P$ such that $p(x_i) = f(x_i)$ for all $i$&quot; have a &lt;em&gt;unique&lt;/em&gt; solution -- these are then the degrees of freedom you need on each element. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is then a theorem of linear algebra that the matrix $M$ (which is called &lt;em&gt;Vandermonde matrix&lt;/em&gt;) is always invertible. (Note that your equation $MC=I$ is just a compact way of writing the nodal interpolation conditions $\phi_i(x_j,y_j) = \delta_{ij}$.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-01-17T10:51:33.403" LastActivityDate="2016-01-17T10:51:33.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21840" PostTypeId="2" ParentId="21811" CreationDate="2016-01-17T11:00:55.640" Score="3" Body="&lt;p&gt;It seems you have answered your own question.  As another example, computational astrophysicists have used Blender and Sketchfab, for instance, to render simulation output; see e.g. &lt;a href=&quot;http://yt-project.org/doc/visualizing/sketchfab.html&quot; rel=&quot;nofollow&quot;&gt;this page in the yt docs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-01-17T11:00:55.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21844" PostTypeId="1" CreationDate="2016-01-17T22:24:54.737" Score="7" ViewCount="2412" Body="&lt;p&gt;I am interested in computing the ranks of fairly large, the largest being of magnitude $10^6$ x $10^6$, sparse matrices whose entires are all 0, 1, or -1. I have been trying to use Matlab to accomplish this. In particular, my approach has been to use QR-factorization -- that is factoring my matrix $T$ as $TP=QR$ were $P$ is a permutation matrix, $Q$ is orthogonal, and $R$ is upper triangular -- and then counting the number of non-zero entries on the diagonal of $R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Matlab's built in &lt;a href=&quot;http://www.mathworks.com/help/symbolic/qr.html?refresh=true&quot;&gt;QR function&lt;/a&gt; the following Matlab code computes the correct rank for some of my examples. (I have confirmed the rank for some of the smaller matrices against rank(full(T)).)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;load(‘FILE_NAME.dat') &#xA;T = spconvert(FILE_NAME);&#xA;&#xA;tic&#xA;&#xA;[Q,R,P]=qr(T);&#xA;[m,n] = size(T);&#xA;tol = max(size(T))*eps*abs(R(1,1));&#xA;r = 0;&#xA;&#xA;while (r&amp;lt;min(m,n) &amp;amp; abs(R(r+1,r+1)) &amp;gt;= tol)&#xA;r = r+1;&#xA;end&#xA;&#xA;tim=toc&#xA;r&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That said even for some of my smaller matrices this code seems fairly slow and memory intensive. For example, on one of these matrices that is ~70,000 x ~50,000 the above code takes numerous hours and most of the memory on a 64 GB server. (One thing I noticed is that if I save the Q, R, and P computed above to a .mat file the file size seems to grow quickly. For example, the .mat file is ~1 GB when initial matrix is ~15,000 x ~15,000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better (quicker or less memory intensive) way to approach computing the rank of large sparse matrices? For example, a more effective way to implement QR-factorization or even a method entirely different from QR?&lt;/p&gt;&#xA;" OwnerUserId="18925" LastEditorUserId="18925" LastEditDate="2016-01-18T03:47:18.543" LastActivityDate="2016-01-19T21:02:46.870" Title="How to compute the rank of a large sparse matrix in MATLAB" Tags="|matlab|sparse-matrix|matrix|rank|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21845" PostTypeId="2" ParentId="21844" CreationDate="2016-01-17T23:53:00.137" Score="2" Body="&lt;p&gt;A rank revealing LU factorization such as LUSOL might be what you want:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.stanford.edu/group/SOL/software/lusol/&quot; rel=&quot;nofollow&quot;&gt;http://web.stanford.edu/group/SOL/software/lusol/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-01-17T23:53:00.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21846" PostTypeId="1" CreationDate="2016-01-18T08:31:20.750" Score="0" ViewCount="2166" Body="&lt;p&gt;I am currently writing software in c++ which solves the eigenvalue problem of sparse hermitian  matrices. The size of the matrix depends on the user input, but as an estimation it will roughly be 1000x1000. I will have to solve about 300 (again relies on the user input) of them for different initial conditions. What is the fastest way to do it? More specifically, I would like to know if it makes sense to do the calculations on the GPU or would it be faster to do it on the CPU and use openMP for the 300 initial conditions? What c++ library would you suggest to use and why (maybe one has some easy build-in support for GPU)? Currently I am using eigen3 for dense matrices.&lt;/p&gt;&#xA;" OwnerUserId="7040" LastActivityDate="2017-06-27T01:45:36.947" Title="Fastest way to diagonalize square matrices with c++" Tags="|c++|eigensystem|gpu|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21847" PostTypeId="1" AcceptedAnswerId="21849" CreationDate="2016-01-18T11:49:52.190" Score="3" ViewCount="2040" Body="&lt;p&gt;I am interested in generating a 1D non-uniform grid on the interval &lt;code&gt;[0, L]&lt;/code&gt; with &lt;code&gt;N&lt;/code&gt; points, where a region of width $\sigma$ and centred at $\mu$ is at a higher density and where the transition from low and high densities of grid points occurs over a length $\ell$. This is for a finite-difference code, where particular attention is required to a region with large gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my current implementation I specify low and high grid spacings and interpolate between these using a pair of &lt;code&gt;tanh&lt;/code&gt; functions. However in this scheme the total number of points isn't known &lt;em&gt;a priori&lt;/em&gt;. I can then iteratively adjust the low and high densities until the total number of points is &lt;code&gt;N&lt;/code&gt; but this is quite convoluted in practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question then is: can anyone help by describing a scheme for generating such a grid in a simpler way, perhaps something analogous to the way you might generate Gauss-Lobato points &lt;code&gt;x(i) = cos(i PI / N)&lt;/code&gt; where &lt;code&gt;i&lt;/code&gt; is the grid index. &lt;/p&gt;&#xA;" OwnerUserId="7941" LastActivityDate="2016-01-18T13:50:52.350" Title="Generating a non-uniform grid" Tags="|finite-difference|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21848" PostTypeId="1" CreationDate="2016-01-18T12:20:04.143" Score="1" ViewCount="269" Body="&lt;p&gt;I'm planning on building a program that will carry out HPC. I'm aware that C/C++ is significantly better than Python in terms of the speed of computation. However, I'm not yet proficient in C/C++, so do you think it'll be wise to build a fully functioning prototype of sorts in Python and then get it rewritten in C/C++ by others/me in the future.&lt;/p&gt;&#xA;" OwnerUserId="18930" LastActivityDate="2016-01-19T18:09:02.470" Title="Programming language for HPC" Tags="|python|c++|c|hpc|" AnswerCount="2" CommentCount="3" ClosedDate="2016-01-20T03:33:09.550" ContentLicense="CC BY-SA 3.0" />
  <row Id="21849" PostTypeId="2" ParentId="21847" CreationDate="2016-01-18T13:10:40.113" Score="4" Body="&lt;p&gt;You achieve this with equidistribution to a mesh density function $\rho(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you consider $x$ as a continuous map from $\xi \in [0,1]$ into your domain $[0,L]$, then the statement $x$ &lt;em&gt;equidistributes&lt;/em&gt; $\rho$ is equivalent to&#xA;$$\int_0^{x(\xi)} \rho(x') \mathrm{d}x' = \xi \theta\,,$$&#xA;where &#xA;$$\theta = \int_0^L \rho(x) \mathrm{d}x\,.$$&#xA;Not sure if this has a name, but I've been calling it MMPDE0, for &lt;em&gt;moving mesh PDE 0&lt;/em&gt; (rather confusing out of context, but from work in &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0731038&quot; rel=&quot;nofollow&quot;&gt;Huang, Ren, Russell (1994)&lt;/a&gt;)  Another way of stating this, is that you're looking for the $x_i$ for $i=0,\dotsc,N$ that satisfy&#xA;$$\frac1{\theta} \int_{x_{i-1}}^{x_i}\rho(x')\mathrm{d}x' = \frac1N\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, your mesh density function is piecewise linear, so you can find an explicit expression for $\int \rho(x)\mathrm{d}x$ (which will be piecewise quadratic), which you can solve to find the next $x_i$ from $x_{i-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note that this will give $N+1$ points in $[0,L]$ with $x_0 = 0$, $x_N=L$, so adjust for your application as necessary).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a possible algorithm you could use to find your $x_i$: Here, $y_j$ are the ordered knots of your piecewise linear mesh density function &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x_0 = 0$.&lt;/li&gt;&#xA;&lt;li&gt;For $i$ from 1 to $N-1$&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Find the highest $j$ such that&#xA;$$ \int_0^{y_j}\rho(x) \mathrm{d}x &amp;lt; \frac{i \theta}{N},$$&#xA;(trapezium rule)&lt;/li&gt;&#xA;&lt;li&gt;Find $x_i$ such that&#xA;$$\begin{aligned} \int_0^{x_i}\rho(x')\mathrm{d}x' &amp;amp;= &#xA;         \int_0^{y_j}\rho(x')\mathrm{d}x' + &#xA;         \int_{y_j}^{x_i}\rho(x')\mathrm{d}x'\\&#xA;        &amp;amp;= \frac{i\theta}{N}&#xA;      \end{aligned}$$&#xA;(involves solving a quadratic equation in $x_i$)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;$x_N = L$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is generally a good idea to smooth your mesh density function before equidistributing to it. It should still be okay to represent it as a piecewise linear mesh density function, just with a lot more knots.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="17671" LastEditDate="2016-01-18T13:50:52.350" LastActivityDate="2016-01-18T13:50:52.350" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21850" PostTypeId="2" ParentId="21844" CreationDate="2016-01-18T13:20:24.773" Score="6" Body="&lt;p&gt;There are two things you can do that may significantly reduce the computation time and memory used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you aren't using the Q matrix, so don't ask MATLAB to compute it.&#xA;Since it is dense, that will definitely save a lot of memory. You can&#xA;do simply:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R=qr(T);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Second, in sparse factorizations, the ordering of the equations can have&#xA;a dramatic effect on the amount of fill-in and computation time during&#xA;factorization. This is true even for QR and LU factorizations where pivoting&#xA;takes place to insure numerical stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An effective strategy in sparse QR factorizations is to first reorder the &#xA;columns to preserve sparsity. Sometimes that will make a huge difference&#xA;in both memory usage and factorization time-- depending on the matrix.&#xA;That can be done, as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[m,n]=size(T);&#xA;q=colamd(T, [n m]);&#xA;R=qr(T(:,q));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The details of the rank-revealing properties of this algorithm (and implementation) are discussed in this paper by Davis:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.319.4864&amp;amp;rep=rep1&amp;amp;type=pdf&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.319.4864&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2016-01-18T15:39:47.780" LastActivityDate="2016-01-18T15:39:47.780" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21851" PostTypeId="2" ParentId="21781" CreationDate="2016-01-18T14:08:06.243" Score="3" Body="&lt;p&gt;From what you describe, you have an ill-posed problem: The solution is not unique (not even locally). A standard way of dealing with this is the following: Instead of trying to solve $F(x)=y$, where $x$ is your vector of geometrical parameters, $y$ the vector of optical parameters, and $F$ the mapping that computes the latter via the former, you minimize&#xA;$$ J(x) = \frac12\|F(x)-y\|^2 + \frac\alpha2\|x\|^2$$&#xA;for some (small) $\alpha&amp;gt;0$. The last term ensures (local) uniqueness: Among all solutions of $F(x)=y$, it will pick the one with minimal norm (in your case, if any angle will give the same surface, the minimizer will have zero angle).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then compute a minimizer using any optimization method (e.g., BFGS with line search globalization), as described in the book by Nocedal and Wright &lt;em&gt;Numerical Optimization&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For most methods, you need the gradient of $J(x)$, which is given by&#xA;$$\nabla J(x) = \nabla F(x)^T (F(x)-y) + \alpha x,$$&#xA;where $\nabla F(x)^T$ is the transpose of the Jacobian. If it is at all possible, you should try to compute the Jacobian analytically and (approximately) evaluate that. (If you add the mathematical description of your $F$, we might be able to help with that.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-01-18T14:08:06.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21852" PostTypeId="2" ParentId="20078" CreationDate="2016-01-18T15:11:35.880" Score="5" Body="&lt;p&gt;Since you read Riviere's book and know how to assemble element integrals I assume you are familiar with concepts such as &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;transformation to reference elements,&lt;/li&gt;&#xA;&lt;li&gt;numerical quadrature.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These techniques are applied analogously to edge integrals - with two differences: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Integrals are transformed to integrals over edges of the reference element, and then a second transformation brings them to the reference interval [0,1] or [-1,1], such that 1D quadrature rules can be applied.&lt;/li&gt;&#xA;&lt;li&gt;To account for the contributions from neighboring elements, you will obtain multiple terms (contributions in your system matrix) from each integral of the original formulation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My recommendation is to write out the semi-discrete system in full detail before starting to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For another DG-formulation (LDG) you can read up on discretization and implementation details here: &lt;a href=&quot;http://dx.doi.org/10.1016/j.camwa.2015.04.013&quot; rel=&quot;noreferrer&quot;&gt;http://dx.doi.org/10.1016/j.camwa.2015.04.013&lt;/a&gt;&#xA;Many of the techniques explained there apply similarly to IPDG.&#xA;The corresponding code is available on Github: &lt;a href=&quot;https://github.com/FESTUNG/project&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/FESTUNG/project&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18931" LastActivityDate="2016-01-18T15:11:35.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21855" PostTypeId="1" CreationDate="2016-01-18T20:39:01.663" Score="1" ViewCount="291" Body="&lt;p&gt;my name is Lorenzo and I am a postdoc researcher in Italy. My job is related to tomography imaging using synchrotron radiation. This is a new field for me and I start to face with Matlab to write some code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am completely new with tomography imaging and Matlab shows new challenges for me. My actual problem is to create sinograms from a stacking of parallel image projections. For whose are not in the field, a sinogram is a map detecting the projection on the detector of the position of a feature in the sample, as function of angle between the X-Ray beam and the sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I get from an experiment is a series of 2D radiography at different angles, you can see it as a rectangular &quot;volume&quot; where dimensions are the number of rows and columns in the single projection, respectively, and volume is given from the number of angles. A sinogram is just a transversal cut of this volume. This means reading the volume not from lateral side but from the top, so I will create a new array of images, whose dimensions are the number of columns and projection, and array lenght the number of rows in the projections. To give numbers in this experiment I have 4000 projections with size 2048x1370 pixels, so for my computer skills this is a huge computational problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need your help to perform some operations faster. My code in a first part allocates an array to contain all images, this is a 34 Gb array but I have 130 Gb RAM, so no problem. The code to perform this operation uses a loop of imread:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=2:num_proj&#xA;    filename=strcat(path_im,list_proj(i).name);&#xA;    image=imread(filename);&#xA;    imArray(:,:,i)=image;&#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is not the fastest way off course, now it takes 332 seconds to create this array. I have found several solutions to improve this and I will do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second step is to divide for a flat field (image taken without the sample). My code takes the flat and using imdivide divides each image in the array for the flat image:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:size(imArray, 3);&#xA;    imArray(:,:,i)=imdivide(imArray(:,:,i), flat);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this step seems to be fast however it is called 4000 times. Do you have suggestion? Is there a better way to perform it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my greatest problem is how to take in the fastest way the horizontal cut in the projection volume? My basic idea to read the volume from the &quot;top&quot;. is reported below, it does its job, however it takes long time:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:size(sinogram, 3)&#xA;        for k=1:size(sinogram, 1)&#xA;            sinogram(k,:,i)=imArray(i,:,k);&#xA;&#xA;        end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you help me to speed up also this operation? I hope my question is clear, otherwise please ask and I will try to explain it better.&lt;/p&gt;&#xA;" OwnerUserId="18939" LastActivityDate="2016-01-18T20:39:01.663" Title="Extracting sinograms from tomography projections" Tags="|matlab|matrix|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21856" PostTypeId="1" CreationDate="2016-01-19T05:53:04.547" Score="2" ViewCount="700" Body="&lt;p&gt;Solve a matrix equation of the type $AX=B$, where $A$ is an $n \times n$ symmetric matrix stored in the form of symmetric skyline matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the solution given by Bill and some more research on factorization, I used the prototype code provided by &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0045794992903624&quot; rel=&quot;nofollow&quot;&gt;S.H Lee&lt;/a&gt; along with the same test code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    parameter (neqs = 5)&#xA;    real*8 au(7), ad(neqs), b(neqs), energy&#xA;    integer jp(neqs)&#xA;    data au/-2.,-2.,-3.,-1.,0.,0.,4./&#xA;    data ad/2.,3.,5.,10.,10./&#xA;    data b/0.,1.,0.,0.,0./&#xA;    data jp/0,1,2,3,7/&#xA;    CALL REDUCl(neqs,jp,ad,au)&#xA;    CALL SOLVE1(neqs,jp,ad,au,b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results are satisfactory:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;solution   636.        619.        292.        74.        34.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The subroutines REDUC1 and SOLVE1 are as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SUBROUTINE  REDUCl  (N,IP,D,A)&#xA;IMPLICIT  REAL(8) (A-H,O-Z)&#xA;DIMENSION  IP(*),D(*),A(*)&#xA;!FUNCTION  :  CROUT  DECOMPOSITION  A  =  LDU&#xA;!REDUCTION  COLUMN  BY  COLUMN&#xA;DO  11  K=2,N&#xA;K1=K-1&#xA;LK=IP(K)-K1&#xA;KH=IP(K1)-LK+1&#xA;S=D(K)-A(LK+KH)*A(LK+KH)/D(KH)&#xA;DO  22  J=KH+1,K1&#xA;J1=J-1&#xA;LJ=IP(J)-J1&#xA;JH=IP(J1)-LJ+1&#xA;IF  (KH.GT.JH)  JH=KH&#xA;T=A(LK+J)&#xA;DO  33  M=JH,J1&#xA;33  T=T-A(LJ+M)*A(LK+M)&#xA;A(LK+J)=T&#xA;22  S=S-T*T/D(J)&#xA;DO  44 J=KH,K1&#xA;L=LK+J&#xA;44  A(L)=A(L)/D(J)&#xA;11  D(K)=S&#xA;RETURN&#xA;END&#xA;&#xA;SUBROUTINE  SOLVE1  (N,IP,D,A,B)&#xA;IMPLICIT  REAL(8)  (A-H,O-Z)&#xA;DIMENSION  IP(*),D(*),A(*),B(*)&#xA;!FUNCTION  :  SOLVE  FOR  X,  LDUx  =  b  where  U  =  transpose  of  L&#xA;!1.  FORWARD  SUBSTITUTION  :  Lz  =  b&#xA;DO  11  J=2,N&#xA;J1=J-1&#xA;LJ=IP(J)-J1&#xA;JH=IP(J1)-LJ+1&#xA;T=B(J)&#xA;DO  22  M=JH,J1&#xA;22  T=T-A(LJ+M)*B(M)&#xA;11  B(J)=T&#xA;!2.  DIVIDING  BY  DIAGONAL  ELEMENTS  :  Dy  =  2&#xA;DO  33  K=1,N&#xA;33  B(K)=B(K)/D(K)&#xA;!BACKWARD  SUBSTITUTION  :  UX  =  y&#xA;DO  44  K=N,2,-1&#xA;K1=K-1&#xA;LK=IP(K)-K1&#xA;KH=IP(K1)-LK+1&#xA;T=B(K)&#xA;DO  44  J=KH,K1&#xA;44  B(J)=B(J)-T*A(LK+J)&#xA;RETURN&#xA;END&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, now I am stuck in a new problem! this works fine when my matrix is like above but fails (I get wrong answers) for the below two cases:&lt;br/&gt;&#xA;Case 1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    data au/0.6801,0.3846,0.4820,1.4074,1.2731,0.7047,0.1078,0.6171,0.9063,0.5895,0.,0.5260,0.3439,0.8178/&#xA;    data ad/0.3818,0.4524,0.,1.4605,1.7593,1.1887/&#xA;    data b/1.,1.,1.,1.,1.,1./&#xA;    data jp/0,0,2,5,9,14/&#xA;    CALL REDUCl(neqs,jp,ad,au)&#xA;    CALL SOLVE1(neqs,jp,ad,au,b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Case 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!!C  Solve A*U = b&#xA;!!C        [1    0    0     0     0            0         ]&#xA;!!C        [0    1    0     0     0            0         ]&#xA;!!C  A =   [0    0    1     0     0            0         ]&#xA;!!         [0    0    0     1     0            0         ]&#xA;!!C        [0    0    0           1.6667E10   -0.4167E10 ]&#xA;!!C        [0    0    0    0     -0.4167E10    0.1041e10 ]&#xA;data au/-0.4166667E10/&#xA;data ad/1.,1.,1.,1.,1.6667E10,0.1041667E10/&#xA;data b/1.,1.,1.,1.,1.,1./&#xA;data jp/0,0,0,0,0,1/&#xA;Print*,au&#xA;Print*,ad&#xA;CALL REDUCl(neqs,jp,ad,au)&#xA;CALL SOLVE1(neqs,jp,ad,au,b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where am I going wrong? Is it due to some allocation or use of precision?&lt;/p&gt;&#xA;" OwnerUserId="18908" LastEditorUserId="20417" LastEditDate="2016-09-17T13:38:19.953" LastActivityDate="2016-09-17T15:36:01.323" Title="Solve $AX=B$ where $A$ is a skyline matrix" Tags="|sparse-matrix|matrix|fortran|matrix-equations|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21857" PostTypeId="2" ParentId="3107" CreationDate="2016-01-19T09:23:05.670" Score="1" Body="&lt;p&gt;The easiest way to do this is to use the fractional coordinate. The algorithm looks like this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Transform coordinate from cartesian to fractional;&lt;/li&gt;&#xA;&lt;li&gt;Round fractional coordinates to [0, 1);&lt;/li&gt;&#xA;&lt;li&gt;Transform back rounded coordinates to cartesian.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To transform from cartesian coordinates to fractional, you have to multiply the coordinate vector by the inverted cell matrix:&#xA;$$ H^{-1} = \begin{pmatrix}&#xA;a &amp;amp; b \cos \gamma &amp;amp; c \cos \beta\\&#xA;0 &amp;amp; b \sin \gamma &amp;amp; c (\cos\alpha - \cos\beta \cos\gamma)/\sin\gamma\\&#xA;0 &amp;amp; 0             &amp;amp; \sqrt{c^2 - c^2 \cos^2 \beta - c^2 (\cos\alpha - \cos\beta \cos\gamma)^2/\sin^2\gamma}&#xA;\end{pmatrix}^{-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And to transform back to cartesian, you have to multiply the vector by the cell matrix $H$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The good thing is that this also works for orthorombic cells.&lt;/p&gt;&#xA;" OwnerUserId="14379" LastActivityDate="2016-01-19T09:23:05.670" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21858" PostTypeId="2" ParentId="21814" CreationDate="2016-01-19T10:20:00.263" Score="1" Body="&lt;p&gt;If a homogeneous Neumann problem is considered, i.e. $f=0$ in $\Omega$ and $\mathbf{n}\cdot\nabla\phi$ on $\Gamma$, one solution is given by the constant function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \phi\left(\mathbf{x}\right)=\phi_0\,. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the boundary integral equation reads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int\limits_{\Omega}\phi\left(\mathbf{x}'&#xA;\right)\delta\left(\mathbf{x}'-\mathbf{x}\right)\mathrm{d}V'&#xA;=\phi_0\int\limits_{\Omega}\delta\left(\mathbf{x}'-\mathbf{x}\right)\mathrm{d}V'&#xA;=-\phi_0\oint\limits_{\Gamma}\tilde{\mathbf{n}}\cdot&#xA;\left[\nabla' G\left(\mathbf{x}'-\mathbf{x}\right)\right]\mathrm{d}\Gamma'\,,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{\mathbf{n}}$ is the &lt;em&gt;outward normal&lt;/em&gt; of the considered domain. Let the considered domain be the whole space substracted with an interior domain $\Gamma_0$. Then the boundary integral reads&#xA;$$&#xA;-\oint\limits_{\Gamma}\tilde{\mathbf{n}}\cdot&#xA;\left[\nabla' G\left(\mathbf{x}'-\mathbf{x}\right)\right]\mathrm{d}\Gamma'&#xA;=-\oint\limits_{\Gamma_\infty}\tilde{\mathbf{n}}\cdot&#xA;\left[\nabla' G\left(\mathbf{x}'-\mathbf{x}\right)\right]\mathrm{d}\Gamma'+\\&#xA;+\oint\limits_{\Gamma_0}\mathbf{n}\cdot&#xA;\left[\nabla' G\left(\mathbf{x}'-\mathbf{x}\right)\right]\mathrm{d}\Gamma'\,,$$&#xA;where $\mathbf{n}$ is the &lt;em&gt;inward normal&lt;/em&gt;. It can be shown that the integral on $\Gamma_\infty$ is unity (use spherical coordinates). Using this result and the 2nd equation, we have&#xA;$$&#xA;\alpha\left(\mathbf{x}\right)&#xA;=1+\oint\limits_{\Gamma_0}\mathbf{n}\cdot&#xA;\left[\nabla' G\left(\mathbf{x}'-\mathbf{x}\right)\right]\mathrm{d}\Gamma'\,.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2016-01-19T10:20:00.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21859" PostTypeId="2" ParentId="21848" CreationDate="2016-01-19T16:26:44.293" Score="3" Body="&lt;p&gt;It is probably a good idea to stick with Python if you have expertise with that. I am assuming that this means you are using numpy/scipy. Depending on what you exactly want to do, well-written numpy/scipy code can be very fast and even comparable to C performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really need the extra speed of C, you can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Cython&quot; rel=&quot;nofollow&quot;&gt;Cython&lt;/a&gt; so that you keep the majority of the code in Python and write only the bottleneck in C.&lt;/p&gt;&#xA;" OwnerUserId="18948" LastActivityDate="2016-01-19T16:26:44.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21860" PostTypeId="2" ParentId="21856" CreationDate="2016-01-19T17:39:47.127" Score="2" Body="&lt;p&gt;The matrix in your case 2 example has a much higher condition number than the one in the original 5-equation example. Accordingly, errors in the definition of the matrix terms result in larger errors in the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that if you output the matrices from MATLAB with 16 digits of precision, the fortran codes will yield the same solution as MATLAB. That is, the entries in the &lt;code&gt;au&lt;/code&gt; and &lt;code&gt;ad&lt;/code&gt; vectors in your fortran code need to be defined with 16 digits of precision instead of the (roughly) 6 digits of precision in the posted code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, you should be careful to define floating point constants correctly, e.g. 1.d0 instead of 1.e0. This was an oversight in the original 5-equation example that caused no great problem in that case because of the low condition number.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2016-01-20T13:28:38.433" LastActivityDate="2016-01-20T13:28:38.433" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21861" PostTypeId="2" ParentId="21848" CreationDate="2016-01-19T18:09:02.470" Score="0" Body="&lt;p&gt;Python can be used for HPC with no problem. Most computation intensive work will be done with libraries written in C/C++ such as Numpy/Scipy. You can write your key algorithm in C/C++ also. Even if you want to write algorithms in python, there are plenty of methods to gain comparable performance to C/C++. Cython compiles program written with python-like syntax to C; Numba compiles your key algorithms (a python function for example) to C when needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HPC is not only about performance on a single computer, you have the choice to use cluster or GPUs to further enhance the computation performance. Python is good at that also.&lt;/p&gt;&#xA;" OwnerUserId="16299" LastActivityDate="2016-01-19T18:09:02.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21862" PostTypeId="1" CreationDate="2016-01-19T19:08:12.463" Score="3" ViewCount="1779" Body="&lt;p&gt;all, I've tried to find an answer to this question and have come across some related threads but nothing I was able to apply to my problem given my relative lack of experience in this arena.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I basically have a constrained minimization problem using both inequality and equality constraints. I will have more variables (~10) than equality constraints (6), so generally speaking there will be an infinite number of solutions to the system. The wrinkle is that I want my solution vector to have as many zero elements as possible, or more precisely I want to minimize the number of non-zero elements. The real-world application is that each variable represents a trade in a portfolio, so I want to hit my &quot;targets&quot; in the portfolio with the fewest number of necessary trades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial approach was to use the fmincon function in Matlab with the objective function as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fun=@(x)nnz(x);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this approach results in a spurious solution even for the most basic test example I could come up with. Is a problem like this solvable in Matlab? Any help would be much appreciated, I'm really at an impasse. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="18952" LastActivityDate="2016-01-19T20:37:08.953" Title="How can I minimize the number of non-zero elements in the solution vector subject to linear constraints (MATLAB)?" Tags="|matlab|constrained-optimization|cvx|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21863" PostTypeId="2" ParentId="21811" CreationDate="2016-01-19T19:24:25.027" Score="1" Body="&lt;p&gt;There are well designed, open source software packages for scientific visualization. In particular, there is the VTK toolkit, and the two major contenders for scientific visualization -- Visit and Paraview -- are both built on top of it. Given that both are free, there is really no reason for anyone to look for other tools that weren't specifically intended for scientific visualization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, some software can at least interface with other packages. For example &lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;nofollow&quot;&gt;deal.II&lt;/a&gt; has a backend to generate Povray files that can then be rendered via raytracing. (Disclaimer 1: I'm one of the authors of deal.II. Disclaimer 2: I don't think anyone has tested the povray backend in the better part of a decade.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-19T19:24:25.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21864" PostTypeId="2" ParentId="21862" CreationDate="2016-01-19T19:31:28.160" Score="4" Body="&lt;p&gt;A common solution to this problem when you have a cardinality constraint (or an $\ell_0$ regularization term, it's not clear to me from your description), is to relax the non-convex cardinality constraint to either a convex $\ell_1$ constraint, or a convex $\ell_1$ regularization term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for example, a problem like&#xA;$$ \text{minimize}\quad \mathbf{card}(x), \qquad \text{subject to}\quad x\in\mathcal{C} $$&#xA;could be (heuristically) replaced with &#xA;$$ \text{minimize}\quad \|x\|_1, \qquad \text{subject to}\quad x\in\mathcal{C}. $$&#xA;A constraint like $\mathbf{card}(x)\leq k$ could be replaced by a regularization term $\gamma\|x\|_1$, or a constraint $\|x_1\|\leq \beta$, where the appropriate values of $\beta$ or $\gamma$ have to be determined later to make the result match the original constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;em&gt;Convex Optimization&lt;/em&gt; by Boyd, Vandenberghe this is Example 6.4.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-01-19T20:37:08.953" LastActivityDate="2016-01-19T20:37:08.953" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21865" PostTypeId="2" ParentId="21844" CreationDate="2016-01-19T21:02:46.870" Score="2" Body="&lt;p&gt;If you know the rank will be small (say less than 100) and your matrix is square, use &lt;em&gt;eigs(A,100)&lt;/em&gt; to get the 100 largest eigenvalues in magnitude.  The eigenvalues are solved using sparse matrix techniques and would be much faster.  In such a case, there is no need to create a full matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you know the rank will be close to the matrix size, then maybe try &lt;em&gt;eigs(A,100,0)&lt;/em&gt; which should give you the 100 smallest eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="18956" LastActivityDate="2016-01-19T21:02:46.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21866" PostTypeId="1" AcceptedAnswerId="21875" CreationDate="2016-01-20T10:50:14.893" Score="7" ViewCount="672" Body="&lt;p&gt;I am looking for the Butcher tableau of a fourth order accurate Runge-Kutta method with IMEX splitting. I have been reading the ''classical'' paper on the subject by &lt;a href=&quot;http://dx.doi.org/10.1016/S0168-9274(97)00056-1&quot; rel=&quot;noreferrer&quot;&gt;Ascher, Ruuth and Spiteri&lt;/a&gt; as well as a number of works that cite this paper (through Google Scholar).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in all papers I looked at, only methods of order up to three were given. Since the number of order conditions increases very quickly as the order goes up (in his &lt;a href=&quot;http://www.utenti.unife.it/lorenzo.pareschi/talks/imex_slides.pdf&quot; rel=&quot;noreferrer&quot;&gt;slides&lt;/a&gt;, Rascheri states that a general IMEX-RKM has to satisfy 56 conditions), I wonder if such a method has been derived anywhere? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a paper somewhere that states the Butcher tableau for a fourth order IMEX Runge-Kutta method?&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="1804" LastEditDate="2016-01-20T10:53:13.720" LastActivityDate="2016-01-21T05:53:30.730" Title="Fourth order IMEX Runge-Kutta method" Tags="|reference-request|time-integration|runge-kutta|operator-splitting|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21867" PostTypeId="1" CreationDate="2016-01-20T12:40:34.250" Score="2" ViewCount="8121" Body="&lt;p&gt;It is written in a book I'm reading that &#xA;$$\nabla f(x) = \left( \frac{\partial f(x)}{\partial x_1}, \frac{\partial f(x)}{\partial x_2},...,\frac{\partial f(x)}{\partial x_n}\right)$$&#xA;and&#xA;$$\nabla^2 f(x)_{ij} =  \frac{\partial^2 f(x)}{\partial x_i ~\partial x_j}, \qquad \forall i,j=1,...,n.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;strong&gt;2nd-order conditions&lt;/strong&gt;: &#xA;for twice differentiable function $f$, it is convex &lt;strong&gt;if and only if&lt;/strong&gt;&#xA;$$\nabla^2 f(x) \ge 0, \qquad \forall x \in \mathrm{dom} f.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, the function $f(x,y) = \sqrt{x^2+y^2}$ is convex, but does not meet 2nd-order conditions:&#xA;$$&#xA;\begin{aligned}&#xA;\frac{\partial^2 }{\partial x^2} \sqrt{x^2+y^2} &amp;amp;= \frac{y^2}{(x^2+y^2)^{\frac{3}{2}}} \ge 0,\\&#xA;\frac{\partial^2 }{\partial x ~ \partial y} \sqrt{x^2+y^2} &amp;amp;= - \frac{x y}{(x^2+y^2)^{\frac{3}{2}}} \le 0.&#xA;\end{aligned}&#xA;$$&#xA;Can anyone explain this?&lt;/p&gt;&#xA;" OwnerUserId="17800" LastEditorUserId="1804" LastEditDate="2016-01-21T13:02:51.260" LastActivityDate="2016-01-21T13:12:09.430" Title="Second-order derivative condition for convexity" Tags="|convex-optimization|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21868" PostTypeId="2" ParentId="14641" CreationDate="2016-01-20T13:05:47.787" Score="3" Body="&lt;p&gt;We have collected the legal and personal concerns of colleagues and the head of the department and compiled a form that &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;defines the form, content, and scope of software publication&lt;/li&gt;&#xA;&lt;li&gt;names the developer &lt;/li&gt;&#xA;&lt;li&gt;addresses competing interests within the lab&lt;/li&gt;&#xA;&lt;li&gt;can be signed by the head of the department&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;to grant a general but well defined approval for software publication both in final versions or as open source development.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please see this &lt;a href=&quot;https://gitlab.mpi-magdeburg.mpg.de/software/csc-software-publication-form&quot; rel=&quot;nofollow&quot;&gt;gitlab page&lt;/a&gt; for a download of the document and the &lt;code&gt;tex&lt;/code&gt; sources.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2016-08-03T07:43:04.017" LastActivityDate="2016-08-03T07:43:04.017" CommentCount="0" CommunityOwnedDate="2016-01-20T13:05:47.787" ContentLicense="CC BY-SA 3.0" />
  <row Id="21869" PostTypeId="1" CreationDate="2016-01-20T15:06:50.773" Score="7" ViewCount="33369" Body="&lt;p&gt;I want to plot a numerical integral function of some function $f$ using &lt;code&gt;scipy&lt;/code&gt; and &lt;code&gt;matplotlib&lt;/code&gt;. How can I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following but it didn't work (run with &lt;code&gt;ipython %pylab&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy import integrate&#xA;&#xA;def f(x):&#xA;    return x*np.sin(1/x)&#xA;&#xA;X = np.arange(-0.5,0.5,0.001)&#xA;&#xA;#plot(X,f(X))&#xA;&#xA;def F(x):&#xA;    return integrate.quad(f,0,x)&#xA;&#xA;plot(X,F(X))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also tried to vectorize the function &lt;code&gt;F&lt;/code&gt; without success.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastEditorUserId="9667" LastEditDate="2016-01-20T21:14:03.297" LastActivityDate="2021-01-17T13:44:40.130" Title="Plot integral function with scipy and matplotlib" Tags="|python|quadrature|scipy|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21870" PostTypeId="2" ParentId="21869" CreationDate="2016-01-20T18:53:27.437" Score="5" Body="&lt;p&gt;Replace the last line by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot(X, [F(x)[0] for x in X])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That should do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: you can define your function F as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def F(x):&#xA;    try:&#xA;        return [integrate.quad(f, 0, y) for y in x]&#xA;    except TypeError:&#xA;        return integrate.quad(f, 0, x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16655" LastEditorUserId="16655" LastEditDate="2016-01-20T19:44:46.750" LastActivityDate="2016-01-20T19:44:46.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21871" PostTypeId="2" ParentId="21869" CreationDate="2016-01-20T19:35:58.090" Score="8" Body="&lt;p&gt;First of all, your function $x\sin(\frac{1}{x})$ is singular in $x=0$. You might want to add an &lt;code&gt;if&lt;/code&gt; clause like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(x):&#xA;    if abs(x) &amp;lt; 1e-10:&#xA;        res = x&#xA;    else:&#xA;        res  = x*sin(1/x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this does hurt speed (masked arrays would be better).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why your code doesn't work is because&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;scipy.quad&lt;/code&gt; only accepts a single value as left and right boundary&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;scipy.quad&lt;/code&gt; returns a tuple &lt;code&gt;(val,err)&lt;/code&gt; containing the value of the integral and an estimate for the error, so you need to unpack it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This code works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy import integrate&#xA;import matplotlib.pyplot as plt&#xA;%matplotlib inline&#xA;&#xA;def f(x):&#xA;    if (np.abs(x)&amp;lt;1e-10):&#xA;        res = x&#xA;    else:&#xA;        res = x*np.sin(1.0/x)&#xA;    return res&#xA;&#xA;X = np.arange(-0.5,0.5,0.001)&#xA;&#xA;#plot(X,f(X))&#xA;&#xA;def F(x):&#xA;    res = np.zeros_like(x)&#xA;    for i,val in enumerate(x):&#xA;        y,err = integrate.quad(f,0,val)&#xA;        res[i]=y&#xA;    return res&#xA;&#xA;plt.plot(X,F(X))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastActivityDate="2016-01-20T19:35:58.090" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21872" PostTypeId="2" ParentId="21867" CreationDate="2016-01-20T21:34:40.657" Score="3" Body="&lt;p&gt;The comments already mention what you are not considering. For the particular example you mention we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2 f(x,y) = \begin{pmatrix}&#xA;\frac{y^2}{(x^2 + y^2)^\frac{3}{2}} &amp;amp; \frac{-x\,y}{(x^2 + y^2)^\frac{3}{2}}\\&#xA;\frac{-x\,y}{(x^2 + y^2)^\frac{3}{2}} &amp;amp; \frac{x^2}{(x^2 + y^2)^\frac{3}{2}}&#xA;\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the eigenvalues are $0$ and $\frac{1}{\sqrt{x^2 + y^2}}$. That are greater or equal to zero for all $x,y \in \mathbb{R}^+$.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2016-01-20T21:34:40.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21873" PostTypeId="1" AcceptedAnswerId="21876" CreationDate="2016-01-21T02:42:20.363" Score="-2" ViewCount="917" Body="&lt;p&gt;I need to write the fast fourier transform in C++ and I am referring to this formula from wikipedia: &lt;a href=&quot;https://i.stack.imgur.com/6ZUS5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6ZUS5.jpg&quot; alt=&quot;FFT&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for some reason I am not getting the correct output when I simply enter (1,1) (1,1) (1,1) (1,1) for by vector $v$. Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;complex&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;time.h&amp;gt;&#xA;&#xA;using namespace std;&#xA;typedef complex&amp;lt;double&amp;gt; dcomp;&#xA;&#xA;&#xA;int main() {&#xA;    const int N = 4;&#xA;    dcomp sum1 = dcomp(0.0, 0.0);&#xA;    dcomp sum2 = dcomp(0.0, 0.0);&#xA;    dcomp *X = new dcomp[N];&#xA;    dcomp *v = new dcomp[N];&#xA;    for(int i = 0; i &amp;lt; N; i++) {&#xA;        v[i] = dcomp(1.0, 1.0);&#xA;    }&#xA;&#xA;    for(int k = 0; k &amp;lt; N; k++) {&#xA;        for(int m = 0; m &amp;lt; N/2.0; m++) {&#xA;            double inside = (-2*M_PI/N)*(2*m)*k;&#xA;            sum1 = sum1 + v[2*m] * dcomp(cos(inside), sin(inside));&#xA;        }&#xA;        for(int m = 0; m &amp;lt; N/2.0; m++) {&#xA;            double inside1 = (-2*M_PI/N)*(2*m+1)*k;&#xA;            sum2 = sum2 + v[2*m + 1] * dcomp(cos(inside1), sin(inside1));    &#xA;        }&#xA;&#xA;        X[k] = sum1 + sum2;&#xA;        cout &amp;lt;&amp;lt; X[k] &amp;lt;&amp;lt; endl;&#xA;    }&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is quite simple to read if you just look at the formula I am literally doing the same thing but my output is &#xA;(4,4)&#xA;(4,4)&#xA;(4,4)&#xA;(4,4) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and it should be:&#xA;(2,2)&#xA;(0,0)&#xA;(0,0)&#xA;(0,0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me and point out where I went wrong?&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="123" LastEditDate="2016-01-21T06:03:58.867" LastActivityDate="2016-01-21T06:03:58.867" Title="Understanding and implementing the Fast Fourier Transform" Tags="|c++|fourier-transform|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21874" PostTypeId="1" CreationDate="2016-01-21T04:08:58.560" Score="2" ViewCount="606" Body="&lt;p&gt;I need to solve a very large linear system (coming from finite element method). I'm currently using the Intel MKL library, but the system has been delayed more than 20 hours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix of the system is sparse and it is written in Morse (CRS)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm programming in Fortran 90, but I guess the language of the library is not important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More info:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Large of the matrix: 5636087x5636087 or bigger.&lt;/li&gt;&#xA;&lt;li&gt;Type: sparse (CRS), with real numbers, non symmetric.&lt;/li&gt;&#xA;&lt;li&gt;Number of nonzero elements of the matrix: 202516857&lt;/li&gt;&#xA;&lt;li&gt;The matrix is non-symmetric, and I suppose that is well conditioned (coming from finite element method).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17630" LastEditorUserId="16799" LastEditDate="2016-01-21T16:25:25.830" LastActivityDate="2020-02-28T13:52:05.523" Title="Solve a very large linear system (question about a library linear algebra to do this)" Tags="|linear-system|libraries|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21875" PostTypeId="2" ParentId="21866" CreationDate="2016-01-21T05:53:30.730" Score="5" Body="&lt;p&gt;I think the work of Kennedy and Carpenter (mentioned already by @GoHokies) is still the definitive study on this topic.  &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0168927402001381&quot; rel=&quot;noreferrer&quot;&gt;The journal paper can be found here&lt;/a&gt;; for some reason Google Scholar only provides links to the technical report.  It includes methods of up to fifth order that are optimized for a range of properties relevant to convection-diffusion-reaction problems.  Some of the fourth-order methods are compared with fully-implicit methods in &lt;a href=&quot;http://link.springer.com/article/10.1007/BF02728987&quot; rel=&quot;noreferrer&quot;&gt;this follow-up work&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One more fourth-order -- and low-storage -- IMEX method can be found in &lt;a href=&quot;http://arc.aiaa.org/doi/pdf/10.2514/6.2013-3094&quot; rel=&quot;noreferrer&quot;&gt;this 2013 paper of Cavaglieri et. al.&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One comment regarding the order conditions: while it is true that the number of them grows very rapidly for IMEX methods, this number is much smaller if a few simplifying assumptions are invoked -- for instance, if the abscissas are identical.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-01-21T05:53:30.730" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21876" PostTypeId="2" ParentId="21873" CreationDate="2016-01-21T06:03:23.330" Score="5" Body="&lt;p&gt;Rather than having us debug your homework for you, I recommend that you read &lt;a href=&quot;https://jakevdp.github.io/blog/2013/08/28/understanding-the-fft/&quot;&gt;this blog post by Jake Vanderplas&lt;/a&gt; that contains a wonderful explanation of the FFT algorithm along with a highly readable Python implementation.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-01-21T06:03:23.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21877" PostTypeId="1" CreationDate="2016-01-21T12:26:11.397" Score="2" ViewCount="394" Body="&lt;p&gt;I asked this question over at StackOverflow and someone told me that I'd get a better answer here. So here's my problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on a machine learning project which involves doing a Principal Component Analysis on some labeled data and using those labels to extract more valuable information from the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do that, I'm calculating a scatter matrix for each class, and for each pair of classes I need to solve a generalised eigenvalue problem for their scatter matrices, as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$S_i  v = w  (S_j + \beta I) v,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\beta$ is a multiplier and $I$ is the identity matrix. Now, this is the code in python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        jeigenvalues = eigsh(scatter_j, k=10, return_eigenvectors=False, maxiter=100)&#xA;        print('eigenvalues made')&#xA;        beta = betaMult*mean(jeigenvalues)&#xA;        print(beta)&#xA;        print(scatter_j+beta*eye(shape(x_data)[1]))&#xA;        w, v = eigsh(scatter_i,M=scatter_j+beta*eye(shape(x_data)[1]),k=int(numberOfEVs/45), maxiter=100)&#xA;        print(i,j,'done')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;numberOfEVs&lt;/code&gt; is 90 in my current code (so that it's divisible by 45).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem is, at the line where I use the &lt;code&gt;eigsh&lt;/code&gt; for the aforementioned formula, it never gives me an answer. It keeps eating more and more memory without even completing a single iteration (I set its &lt;code&gt;maxiter&lt;/code&gt; input to 1, and it still didn't give an answer). When I don't give the &lt;code&gt;eigsh&lt;/code&gt; function the &lt;code&gt;M&lt;/code&gt; argument (which is the matrix on the right side of the generalised EV problem and it is assumed to be &quot;I&quot; when not specified), it works correctly. But when &lt;code&gt;M&lt;/code&gt; is provided, it becomes unresponsive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The scatter matrices have rather small entries, mostly around 10^-5. I've also tried multiplying the left hand side by the inverse of the RHS matrix, and again it's having the same issue (goes on for a long time without an answer). Is the smallness of these entries the issue? How can I solve it, then?&lt;/p&gt;&#xA;" OwnerUserId="18977" LastEditorUserId="1804" LastEditDate="2016-01-21T12:29:29.027" LastActivityDate="2016-01-21T12:29:29.027" Title="scipy.linalg.sparse.eigsh does not work for generalised eigenvalues" Tags="|python|eigenvalues|machine-learning|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21878" PostTypeId="1" CreationDate="2016-01-21T12:28:45.943" Score="1" ViewCount="790" Body="&lt;p&gt;My question may be elementary, but it is quite essential as I am getting confused.&#xA;Here I am supposed to solve the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=B$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my understanding I have options of using either direct solvers or iterative solvers. I have tried using iterative solver GMRES(It is quite simple in terms of implementation on Fortran 90) but now I would like to use a direct solver for certain reasons and from resources and reviews I have shortlisted three of them: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;UMFPACK &lt;/li&gt;&#xA;&lt;li&gt;SUPERLU  &lt;/li&gt;&#xA;&lt;li&gt;MUMPS  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I think the first two are developed using C and could be called from Fortran 90; The third one is in Fortran 90.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my question is how do I implement this or call function from my Fortran 90 code, ie. is there a reliable procedure to install or call the libraries? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There looks to be tons of literature beyond my scope of understanding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To simplify, I have a sparse symmetric matrix $A$ , stored in skyline format and rhs vector $B$, how do I calculate $x$ using one of the direct sparse solvers in Fortran 90? &lt;/p&gt;&#xA;" OwnerUserId="18908" LastActivityDate="2016-01-21T12:28:45.943" Title="Implementation of a direct solver in Fortran 90?" Tags="|linear-solver|sparse-matrix|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21879" PostTypeId="2" ParentId="21867" CreationDate="2016-01-21T12:58:57.813" Score="5" Body="&lt;p&gt;Consolidating my comments (so that they can be cleaned up): This is a misunderstanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A twice (continuously!) differentiable function $f:\mathbb{R}^n\to \mathbb{R}$ is convex if and only if the Hessian $\nabla^2 f(x)\in\mathbb{R}^{n\times n}$ is positive semi-definite at every $x\in \mathbb{R}^n$. (This definition makes sense since the Hessian is symmetric by Schwarz' theorem if the second derivatives are continuous.) This is sometimes written as &#xA;$$\nabla^2 f(x) \succeq 0 \qquad\text{for all } x\in\mathbb{R}^n$$&#xA;(and more rarely -- since it can lead to misunderstandings -- as $\nabla^2 f(x)\geq 0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @nicoguaro points out in his answer, this is equivalent to the condition that all eigenvalues of $\nabla^2 f(x)$ -- as a function of $x$ -- are nonnegative for every $x\in \mathbb{R}^n$. An equivalent (and often easier to verify, especially for large $n$) condition is that&#xA;$$d^T\nabla^2 f(x)d \geq 0 \qquad\text{for all } d\in\mathbb{R}^n \text{ and }x\in\mathbb{R}^n.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This condition is also easier to work with if you want to &lt;em&gt;rule out&lt;/em&gt; convexity: It's sufficient to find a single $d$ such that $d^T \nabla^2 f(x) d&amp;lt;0$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example (with $x_1 = x$ and $x_2 = y$), this would yield&#xA;$$ &#xA;   \begin{aligned}&#xA;        \begin{pmatrix} d_1 &amp;amp; d_2 \end{pmatrix}&#xA;        \begin{pmatrix}&#xA;            \frac{x_2^2}{(x_1^2 + x_2^2)^\frac{3}{2}} &amp;amp; &#xA;            \frac{-x_1\,x_2}{(x_1^2 + x_2^2)^\frac{3}{2}} \\&#xA;            \frac{-x_1\,x_2}{(x_1^2 + x_2^2)^\frac{3}{2}} &amp;amp; &#xA;            \frac{x_1^2}{(x_1^2 + x_2^2)^\frac{3}{2}}&#xA;        \end{pmatrix}&#xA;        \begin{pmatrix} d_1 \\ d_2 \end{pmatrix}&#xA;        &amp;amp;=&#xA;        \frac{1}{(x_1^2 + x_2^2)^\frac{3}{2}}\left(d_1^2x_2^2 - 2 d_1 x_1x_2d_2 + d_2^2x_1^2\right)\\&#xA;        &amp;amp;= \frac{1}{(x_1^2 + x_2^2)^\frac{3}{2}}\left(d_1x_2-d_2x_1\right)^2\\&#xA;        &amp;amp;\geq 0&#xA;    \end{aligned}&#xA;$$&#xA;for all $x,d\in\mathbb{R}^n$. Hence, $f$ is convex.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-01-21T13:12:09.430" LastActivityDate="2016-01-21T13:12:09.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21880" PostTypeId="1" AcceptedAnswerId="21885" CreationDate="2016-01-21T15:56:24.727" Score="-5" ViewCount="187" Body="&lt;p&gt;I need consultancy for a possible &lt;strong&gt;p=np&lt;/strong&gt; workaround.&#xA;First of all sorry for my English because it is going to be long question:) To better understand the architecture which I design maybe you will need to have experience with matrix operations and a bit object oriented programming. I wanted to tag this question also to computer science but I couldn’t figure out how. Maybe it is not perfect idea but I think it is worth to ask:)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem Description:&lt;/strong&gt;&#xA;I have been thinking about possibilities for solving &lt;strong&gt;&lt;em&gt;p=np&lt;/em&gt;&lt;/strong&gt; problem.  If you would like to read more about it here is the Wikipedia link: &lt;a href=&quot;https://en.wikipedia.org/wiki/P_versus_NP_problem&quot; rel=&quot;nofollow&quot;&gt;wiki&lt;/a&gt;.  A small example stated here (from wiki):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider the subset sum problem, an example of a problem that is easy to verify, but whose answer may be difficult to compute. Given a set of integers, does some nonempty subset of them sum to 0? For instance, does a subset of the set {−2, −3, 15, 14, 7, −10} add up to 0? The answer &quot;yes, because the subset {−2, −3, −10, 15} adds up to zero&quot; can be quickly verified with three additions. There is no known algorithm to find such a subset in polynomial time (there is one, however, in exponential time, which consists of 2n-n-1 tries), but such an algorithm exists if P = NP; hence this problem is in NP (quickly checkable) but not necessarily in P (quickly solvable).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is the idea:&lt;/strong&gt;&#xA;Every day we do the same calculations in multiple PC`s around the world, so the solution I am thinking (in theory) a central database for those calculations to provide result. Long story in short, &lt;strong&gt;i.e.&lt;/strong&gt; when you multiply large matrixes then your application will do a API call from a global database instead of using local resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How does it work:&lt;/strong&gt;&#xA;Lets assume you develop your project in Java. So you will include IIT (Information Is There) Matrix library into your project. Then (i.e.) you will create a new 2D Matrix object instance as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IITMatrix matrix = new IITMatrix(2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you will append new values as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;matrix.add(1,0.599);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First parameter is the dimension and the second one is the actual value. When you append a value to your matrix object, then the matrix automatically will generate/update a hash using &lt;a href=&quot;https://code.google.com/p/lz4/&quot; rel=&quot;nofollow&quot;&gt;LZ4&lt;/a&gt; at background (its called &lt;em&gt;instance variable&lt;/em&gt; for java guys:)). When you need to do an operation with another matrix you will call i.e.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;matrix.multiply(myAnotherMatrix);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now we are using API future instead of your local resources. Multiply method automatically will call below url :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;http://www.mycentraldb.org/iit/multiply/100/300?a=4jh3k5g4hj3g5443kh5g43&amp;amp;b=fjsdfhjsdljsdljsdljsdljsdlkdf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Let`s say &lt;strong&gt;100&lt;/strong&gt; is the rows and &lt;strong&gt;300&lt;/strong&gt; is our columns. &lt;strong&gt;A&lt;/strong&gt; parameter is the generated hash of first matrix and &lt;strong&gt;B&lt;/strong&gt; parameter generated hash of second matrix. Then backend &lt;em&gt;load balancer&lt;/em&gt; will find the matching server (&lt;em&gt;key&amp;amp;value noSQL server&lt;/em&gt;) for 100x300 and it will query &lt;strong&gt;4jh3k5g4hj3g5443kh5g43: fjsdfhjsdljsdljsdljsdljsdlkdf&lt;/strong&gt; for the value. When server will return another Hash as response then your matrix object will parse it and wraps another matrix object. i.e. something like that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IITMatrix result = matrix.multiply(myAnotherMatrix);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Performance advantage:&lt;/strong&gt;&#xA;When you start your project IIT library, application will establish a socket connection until you close application, so each single query will be done in milliseconds without lookup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How the data generated:&lt;/strong&gt;&#xA;   Caching all the numbers is not possible but the exceptional amount of dimensions would be useful. Maybe first target can be combinations of normalized values between -1 and 1 with 6 decimal points (Combination of 2000000). And the server will keep generating hashes by increasing values and when a dimension is done it will create another DB dump for the new one. Using a GPU farm would beneficial. Even a P2P connection or parallel computing MPI network should allow community to share/generate more resources. Also operations will grow i.e. mod, log etc operations will be added into API as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So my question is:&lt;/strong&gt;&#xA; Can this solution fit your daily needs? How it would be better? Should I start? :) I am asking in advance because I don’t want to remove data after days of generation. I have few GPUs to give a shot:)&lt;/p&gt;&#xA;" OwnerUserId="18980" LastActivityDate="2016-01-22T10:57:58.267" Title="Do you think this p=np workaround worth to try?" Tags="|optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21881" PostTypeId="2" ParentId="21874" CreationDate="2016-01-21T18:57:39.877" Score="3" Body="&lt;p&gt;Since the matrix is considerably sparse and well-conditioned (if it is true), I suggest you can try to use Krylov subspace method, which can only use the information of matrix-vector product; such as GMRES(m), CGS, BiCGSTAB, and IDR(s). With aspect of memory and well-conditioned matrix, maybe BiCGSTAB, CGS and IDR(s) are preferable. They are many FORTRAN codes of these above four solvers online. Or even, you can try to use the AGMG method, which have a FORTRAN code in website &lt;a href=&quot;http://homepages.ulb.ac.be/~ynotay/AGMG/&quot; rel=&quot;nofollow&quot;&gt;http://homepages.ulb.ac.be/~ynotay/AGMG/&lt;/a&gt;&#xA;and other method is &lt;a href=&quot;http://www.hsl.rl.ac.uk/catalogue/hsl_mi20.html&quot; rel=&quot;nofollow&quot;&gt;http://www.hsl.rl.ac.uk/catalogue/hsl_mi20.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10290" LastActivityDate="2016-01-21T18:57:39.877" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21882" PostTypeId="2" ParentId="3589" CreationDate="2016-01-21T20:59:23.000" Score="1" Body="&lt;p&gt;Turing instability only happens in specific parameter ranges. If the solution goes from something randomish to uniform, then that means that you're parameter choices give a stable uniform steady state. This means that to getting Turing patterning you need different parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example to look at &lt;a href=&quot;http://www.math.uci.edu/~qnie/Publications/ja62.pdf&quot; rel=&quot;nofollow&quot;&gt;can be found here&lt;/a&gt; where the authors determine parameter ranges for Turing instability and show that inside the range you getting patterning whereas outside you get uniform steady states.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-01-21T20:59:23.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21883" PostTypeId="2" ParentId="3175" CreationDate="2016-01-21T21:13:33.533" Score="4" Body="&lt;p&gt;I don't think is possible given that the nature of solutions to PDEs varies so much. However, a general heuristic you can have in your head is something like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find out what type of equation (Eliptic, Parabolic, Hyperbolic).&lt;/li&gt;&#xA;&lt;li&gt;Look at the coefficients in the problem / the Jacobian. Is the problem stiff? If so you'll want some kind of implicit method (i.e. Crank-Nicolson), otherwise you can get away with some explicit or predictor corrector method.&lt;/li&gt;&#xA;&lt;li&gt;Look at your geometry. Is your geometry simple? Then try a finite difference method. When the geometry is complex, use a finite element method.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This process works for most problems. Parabolic stiff equation on a torus? Give an implicit FEM solver a try. However, this won't work for all problems, you might have to go back and try another method. Or worse: if the problems hard enough, getting a computational method to solve the PDE is a research topic in its own right. That's why computational PDEs is still an active field of research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also remember that any time spent coding counts towards your time spent solving. If you pick a method which takes 5 minutes to write but solves it in 2 days, it is still pragmatically &quot;faster&quot; than a method which takes 3 days to write but solves it in minutes (and you can get a lot more work done). So don't get stuck worrying about &quot;the optimal method&quot; for your problem unless you're going to be solving it for years or are a researcher in numerical methods.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2016-01-21T23:39:20.383" LastActivityDate="2016-01-21T23:39:20.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21884" PostTypeId="1" AcceptedAnswerId="21893" CreationDate="2016-01-21T23:14:08.280" Score="2" ViewCount="1402" Body="&lt;p&gt;For a start I need to implement a 2-D, vertex centered, finite difference scheme, Adaptive mesh refinement serial code. I have the following doubts before starting:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is the input to AMR (say 2-D) &lt;em&gt;always&lt;/em&gt; a box bounding the entire domain ?&lt;/li&gt;&#xA;&lt;li&gt;Is a &lt;em&gt;cell-centerd&lt;/em&gt; scheme easier to implement than &lt;em&gt;vertex centred&lt;/em&gt; scheme ?&lt;/li&gt;&#xA;&lt;li&gt;Do I need to store &lt;em&gt;geometrical&lt;/em&gt; information of cells/vertices (cell means a box and vertex means its 4 corners) ?&lt;/li&gt;&#xA;&lt;li&gt;Is &lt;strong&gt;Quadtree&lt;/strong&gt; the &lt;em&gt;only&lt;/em&gt; method for implementing a 2-D AMR ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14303" LastActivityDate="2016-01-23T03:09:44.267" Title="Implementing Finite Difference Adaptive Mesh Refinement code" Tags="|finite-difference|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21885" PostTypeId="2" ParentId="21880" CreationDate="2016-01-22T00:50:39.143" Score="6" Body="&lt;p&gt;If your question is whether pre-computing results allows you to come up with an algorithm that shows that even NP problems can be solved in P time, then the answer is no. This is so because statements such as &quot;problem A is in P/NP&quot; are stated &lt;em&gt;asymptotically&lt;/em&gt; as the size of the problem gets larger and larger. But you won't be able to provide your database for larger and larger problems: you will run out of resources to store, let alone compute, the answers for problems beyond a certain size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So no, this is no way to prove that P=NP.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="713" LastEditDate="2016-01-22T10:57:58.267" LastActivityDate="2016-01-22T10:57:58.267" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21886" PostTypeId="1" CreationDate="2016-01-22T01:44:32.707" Score="2" ViewCount="88" Body="&lt;p&gt;cannot delete my own question, so I try to overwrite it instead...&lt;/p&gt;&#xA;" OwnerDisplayName="user7098" LastEditorDisplayName="user7098" LastEditDate="2016-05-27T15:41:12.593" LastActivityDate="2016-05-27T15:41:12.593" Title="$(1+x^M)^{1/M}$ need to be able to calculate any order derivatives vs. $x$ and $M$ for $x\ge 0$ and $M\gt 2$" Tags="|simulation|automatic-differentiation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21887" PostTypeId="1" CreationDate="2016-01-22T11:11:40.533" Score="0" ViewCount="351" Body="&lt;p&gt;I'm solving the 3D Diffusion equation&#xA;$$u_t=k(u_{xx}+u_{yy}+u_{zz})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in MATLAB using Fourier techniques.  I assume a 3D Fourier expansion $(e^{-ipx},e^{-imy},e^{-imz})$of the solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Physical space: $u(x,y,z,t)$.&#xA;Fourier Space:  $c(m,n,p,t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substitution and differentiation result in:&#xA; $$c(m,n,p)^{N+1}-c(m,n,p)^{N} = -\frac{k\Delta T}{2} (p^2+m^2+n^2)(c(m,n,p)^{N+1}+c(m,n,p)^{N})$$&#xA;after applying a Crank-Nicolson scheme.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using &lt;code&gt;fftn()&lt;/code&gt; and &lt;code&gt;ifftn()&lt;/code&gt; to forward my coefficients in time and bring them back to physical space.  However I achieve universal decay from the initial condition to zero with no heat flux in any direction, for all time.  Typical time step: &lt;code&gt;0.00001&lt;/code&gt;.  Typical &lt;code&gt;k=0.005&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the problem with my application of &lt;code&gt;fftn()&lt;/code&gt; or the stability of my finite-difference?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;  I've taken an initial condition of $50 \sin(2x)$. I merely took the boundaries of that initial condition and imposed them as the boundary conditions for all time. No good I'm guessing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2:&lt;/strong&gt; I forgot the wavenumber squared on the rhs.  Thanks James!&lt;/p&gt;&#xA;" OwnerUserId="18989" LastEditorUserId="18997" LastEditDate="2016-01-23T09:38:53.023" LastActivityDate="2016-01-23T09:38:53.023" Title="3D Diffusion Equation in Fourier space" Tags="|matlab|finite-difference|numerical-analysis|diffusion|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21888" PostTypeId="2" ParentId="21886" CreationDate="2016-01-22T11:18:52.247" Score="3" Body="&lt;p&gt;One approach is to use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fa%C3%A0_di_Bruno%27s_formula&quot; rel=&quot;nofollow&quot;&gt;Faà di Bruno formula&lt;/a&gt;:&#xA;$$ \big((1+x^M)^{1/M}\big)^{(n)} = \sum \frac{n!}{m_1!1!^{m_1}\cdots m_n! n!^{m_n}} (1/M)^{\underline{m_1+\cdots+m_n}}(1+x^M)^{1/M-m_1-\cdots-m_n} \prod_{j=1}^{n} \big(M^{\underline{\phantom{,}j}} x^{M-j}\big)^{m_j}, \qquad M\notin\mathbb{Z} $$&#xA;Here $a^{\underline{b}}$ is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Pochhammer_symbol&quot; rel=&quot;nofollow&quot;&gt;falling factorial&lt;/a&gt;, and the sum is over all $n$-tuples of integers $m_1,\ldots,m_n\geq0$ such that $\sum_k km_k = n$. This formula is likely to be unstable for very large values of $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A decent alternative is to evaluate the contour integral&#xA;$$ \frac{n!}{2\pi \mathrm{i}}\int_\gamma (1+z^M)^{1/M}\frac{dz}{(z-x)^{n+1}}, $$&#xA;where the contour $\gamma$ is a circle of some radius $R$ surrounding $x$, $z = x+Re^{\mathrm{i}t}$, with $R$ chosen such that all the branch points of $(1+x^M)^{1/M}$ in the complex plane are outside the circle. The choice of $R$ affects the stability of this formula. This is a straightforward Fourier integral, so one can directly use an existing library to compute it, and one can also evaluate many derivatives at once efficiently using FFT.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-01-22T12:27:47.123" LastActivityDate="2016-01-22T12:27:47.123" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21889" PostTypeId="1" AcceptedAnswerId="21899" CreationDate="2016-01-22T11:37:45.717" Score="1" ViewCount="156" Body="&lt;p&gt;I am trying to solve the following problem, where $a_{ij} \ge 0 \ \forall i,j$:&#xA;\begin{align} &#xA;\mbox{minimize}\quad &amp;amp; \sum_{i=1}^m\sum_{j=1}^n (a_{ij}x_{ij}^2 + b_{ij}x_{ij})\\ &#xA;\mbox{subject to}\quad &amp;amp;\sum_{i=1}^m x_{ij} \le 1 \quad \forall j,\\ &#xA;    &amp;amp;\sum_{j=1}^n x_{ij} \le 1\quad \forall i,\\ &#xA;    &amp;amp;x_{ij} \ge 0 \quad \forall i,j.\end{align}&#xA;Could you please suggest me with some methods? (The faster the better.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your answers!&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2016-01-31T18:01:31.343" Title="How to efficently solve: min $\sum_{ij}(a_{ij}x_{ij}^2 + b_{ij}x_{ij})$ s.t" Tags="|optimization|convex-optimization|nonlinear-programming|quadratic-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21892" PostTypeId="1" AcceptedAnswerId="21895" CreationDate="2016-01-22T19:13:32.537" Score="7" ViewCount="243" Body="&lt;p&gt;Assume $X$ and $N$ are two sets of vectors (observations) from two different  normal distributions, where $X$ represents clean data and $N$ represents noise; and $A$ a projection matrix of a filter. the scenario is that our clean data was corrupted by a multiplicative noise via matrix $A$ and an additive noise of $N$. then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Y=A \times X + N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $Y$ are a set of projected vectors from $X$ using $A$, what are solutions to learn this projection matrix and $N$ from training data? Does the Gaussian assumption of $A, N$ and $X$ help to have a better estimation or guide to use a specific solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is matlab code for the training data, noise and a simple projection:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    dataVariance = .10;&#xA;    noiseVariance = .05;  &#xA;    mixtureCenters=randn(13,1);&#xA;    X=randn(13, 1000)*sqrt(dataVariance ) + repmat(mixtureCenters,1,1000);&#xA;&#xA;    %N and A are unknown and we want to estimate them.&#xA;    N=randn(13, 1000)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,1000);&#xA;    A=2*eye(13);&#xA;&#xA;    Y=A*X+N;&#xA;&#xA;    for iter=1:1000&#xA;        A_hat,N_hat = training(X_hat,X,Y);&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;Note: if necessary, for each estimation of $A$, an error can be calculated for an estimation of $N$ using a current $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for iterate=1:1000&#xA;  initiate A&#xA;  estimate N using current A (N=Y-A*X)&#xA;  calculate error of estimation (err=Y-A*X+N)&#xA;  update A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I would prefer not to go for gradient descent approaches.&#xA;&lt;hr&gt;&#xA;I should clarify that the observations of $X$ and $Y$ are time independent and in $X_i$ and $Y_i$, i is not the time. They are just different observations sampled from two normal distributions.&lt;/p&gt;&#xA;" OwnerUserId="18994" LastEditorUserId="18994" LastEditDate="2016-01-28T10:11:05.597" LastActivityDate="2016-01-29T12:00:09.707" Title="Learning parameters of noise and filter coefficients from data where data and noise both have Gaussian distributions" Tags="|machine-learning|probability|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21893" PostTypeId="2" ParentId="21884" CreationDate="2016-01-22T19:51:22.973" Score="4" Body="&lt;p&gt;I'll try to answer your questions one-by-one, though I'm afraid that they are so basic that these answers alone won't be able to help you very much in the long run:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1/ No, one can run AMR on any starting mesh. It doesn't have to be a square. A few examples of adaptive meshes on other domains are here: &lt;a href=&quot;https://github.com/dealii/dealii/wiki/Gallery&quot; rel=&quot;nofollow&quot;&gt;https://github.com/dealii/dealii/wiki/Gallery&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2/ Cell-centered and vertex-centered schemes are, in some sense, dual to each other. You can often think of one in terms of the other defined on a shifted mesh. I imagine both of them to be equally easy/difficult to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3/ As long as you are on the unit square, you can compute the locations of vertices of a cell if you know how you arrived at that cell by mesh refinement. On the other hand, if you start with any other mesh, you will have to store the locations of vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4/ There are other approaches to doing AMR than using quadtrees. For example, you could do longest-edge bisection or red-green refinement for triangles. (Since you already found my video lectures, this would be lecture 15.) Quadtrees are a particularly useful approach for quadrilaterals, though.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-01-23T03:09:44.267" LastActivityDate="2016-01-23T03:09:44.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21895" PostTypeId="2" ParentId="21892" CreationDate="2016-01-23T03:12:39.693" Score="7" Body="&lt;p&gt;So the way I went about formulating the problem was to essentially write the following equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The state that will be estimated, which is defined as a column vector, is the following:&#xA;$$w = [vec(A)^{T},\bar{N}^{T}]^{T}$$&#xA;where $\bar{N}$ is the unknown average $N$ vector and $vec(A)$ is the vectorization operator on the unknown matrix A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the definition above, one can try to estimate $w$ by the following equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w_{i+1} = w_{i} + \eta_1$$&#xA;$$y_{i} = h(w_{i},X_{k}) + \eta_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\eta_j$ are additive white noise vectors based on some variance values, $v_j$, and $X_{k}$ represents some $k^{th}$ data point from your set of values for $X$. You can select the $X_k$ value however you want (I did it randomly). One last equation you need is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ h(w,X) = [\hat{A},\hat{N}][X^{T},1]^{T}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where you extract $\hat{A}$ &amp;amp; $\hat{N}$ from the input $w$. Based on these equations, you can use a Kalman filter to iteratively solve this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented this earlier using an Unscented Kalman filter and validated it was converging to $A$ and $\bar{N}$. I will note that in the event $N$ is constant throughout all the samples, the algorithm I put together converged to $A$ and $N$ exactly (to numerical precision) within a reasonable number of iterations, but it didn't occur that way when $N$ varied randomly for each $X_k$, like you have in your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a sample image showing the convergence of the algorithm (which can be shown by the covariance trace moving towards 0):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ir7iL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ir7iL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I thought I would also post a plot of the $L_{2}$ Error history per iteration. Note that this $L_{2}$ Norm computation is based on the estimated $w$ vector vs the truth one. Now, this first plot is when $N$ is constant through every $X$ vector.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BUqAB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BUqAB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This next picture is when $N$ has a different value for every $X$ based on some Gaussian random sampling. Note, truth $\bar{N}$ in this case is the mixtureCenter variable (trying to estimate the mean $N$).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9e6Kv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9e6Kv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, estimating $N$ when it is constant results in some better estimations than when you have a slightly different $N$ for each $X$ sample, but it still gets a decent estimate in the harder situation. To further improve the convergence rates of the solution, one could do a batch processing via &quot;sensor&quot; fusion techniques. Below I show similar plots as above, but using a batch of 5 random $X$ vectors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mrSiT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mrSiT.png&quot; alt=&quot;Error using Batch of 5 samples with Constant N&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CXqzY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CXqzY.png&quot; alt=&quot;Error using Batch of 5 samples with Random Sampled N&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first of these plots is for when the unknown $N$ is constant and the latter is when you are trying to estimate a mean $N$ based on random $N$ values for each $X$. Within a very small number of iterations, you can see great estimation performance in both examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a pretty helpful link on the subject:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Kalman_filter&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code can be found at the following &lt;a href=&quot;https://github.com/choward1491/KalmanFilterParameterEstimation&quot; rel=&quot;nofollow noreferrer&quot;&gt;github repository&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-01-27T04:02:00.937" LastActivityDate="2016-01-27T04:02:00.937" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21896" PostTypeId="1" AcceptedAnswerId="21990" CreationDate="2016-01-23T16:29:41.480" Score="3" ViewCount="275" Body="&lt;p&gt;Suppose we have an optimization problem &#xA;$$ \mathbf{x} = (x_1, x_2, \ldots, x_m) = \arg\!\min_{\mathbf{x}\in \mathbb{R}^m}f(\mathbf{x})$$&#xA;and a second related problem:&#xA;$$ \mathbf{y} = (y_1, y_2, \ldots, y_n) = \arg\!\min_{\mathbf{y}\in \mathbb{R}^n}g(\mathbf{y})$$&#xA;because we have the relations&#xA;$$ \mathbf{x} = \mathbf{t}(\mathbf{y}) $$&#xA;and &#xA;$$ f(\mathbf{t}(\mathbf{y})) = g(\mathbf{y}) $$&#xA;and $n&amp;gt;m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what is the disadvantage of solving the second problem instead of the first problem if you use a numerical optimizer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example: take for example a function $g(\mathbf{y})$ that only depends on&#xA;$$x_1 = y_1 + y_2$$ $$x_2 = y_1+y_3$$ $$x_3 = y_1 + y_4$$ and $$x_4 =y_1+y_5$$. So minimizing $g(\mathbf{y})$ is equivalent with minimizing&#xA;$$ f(\mathbf{x}) = g(0,x_1,x_2,x_3,x_4) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it harmful to solve the optimization problem the larger problem? Is it possible that a numerical optimizer has more troubles solving the larger problem, even though it doesn't matter which of all the extra solutions it gives?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: I use the Nelder-Mead algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;update: in some cases the optimizer indicates the problem of a &lt;em&gt;degenerate Nelder mead simplex&lt;/em&gt; . Is the root of the problem to be found in the extra variable?&lt;/p&gt;&#xA;" OwnerUserId="19011" LastEditorUserId="19011" LastEditDate="2016-02-01T10:37:08.843" LastActivityDate="2016-02-03T00:14:33.307" Title="Disadvantages of adding an extra variable to an optimization problem" Tags="|optimization|numerical-analysis|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21898" PostTypeId="1" AcceptedAnswerId="21900" CreationDate="2016-01-23T23:49:21.213" Score="1" ViewCount="2511" Body="&lt;p&gt;Could somebody explain to me why periodic boundary conditions are automatically satisfied if you solve your problem assuming a Fourier series?  So, if we assume a Fourier series for our solution, we don't need to impose any type of boundary conditions because the necessary periodic boundary conditions are naturally satisfied? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quick example of the setup I'm referring to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Diffusion equation: $$u_t=k(u_{xx}+u_{yy}+u_{zz})$$&#xA;Assume $$u(x,y,z,t)$$ can be expanded in a Fourier expansion &#xA;$$(e^{-ipz},e^{-imy},e^{-inx}) ,$$&#xA;resulting in automatic satisfaction of periodic BCs.&lt;/p&gt;&#xA;" OwnerUserId="18997" LastEditorUserId="18997" LastEditDate="2016-01-24T02:50:26.903" LastActivityDate="2016-01-24T13:06:13.473" Title="Fourier techniques and periodic boundary conditions" Tags="|pde|fourier-analysis|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21899" PostTypeId="2" ParentId="21889" CreationDate="2016-01-24T01:04:19.170" Score="2" Body="&lt;p&gt;This is a regular positive definite quadratic minimization problem with linear constraints. It should be trivial to solve using one the usual methods for this approach. A simple approach would be, for example, to take the active set method explained in great detail in Nocedal and Wright, &quot;Numerical Optimization&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could of course also take one of the interior point, penalty, or augmented Lagrange methods -- all of these should be able to quickly solve this problem.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-01-31T18:01:31.343" LastActivityDate="2016-01-31T18:01:31.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21900" PostTypeId="2" ParentId="21898" CreationDate="2016-01-24T04:05:38.997" Score="1" Body="&lt;p&gt;Let's assume $k &amp;gt; 0$ is strictly positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically the initial conditions $t=0$ for the diffusion or &lt;a href=&quot;https://en.wikipedia.org/wiki/Heat_equation#Three-dimensional_problem&quot; rel=&quot;nofollow&quot;&gt;&quot;heat&quot; equation&lt;/a&gt; can be rough (discontinuous), and so long as these are square-integrable, the solution operator for the PDE (e.g. with periodic boundary conditions) will instantly smooth out that data to an analytic function at each $t \gt 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we may think of the initial condition as being periodic but discontinuous at the endpoints (which doesn't affect square-integrability), and that at all subsequent times the solution becomes smoothly periodic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if $k$ were replaced by $-k$, so that we have the &quot;backwards&quot; heat equation, then the smoothing property disappears and solutions become unstable.  As the Wikipedia article puts it:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Generally, many different states and starting conditions will tend toward the same stable equilibrium. As a consequence, to reverse the solution and conclude something about earlier times or initial conditions from the present heat distribution is very inaccurate except over the shortest of time periods.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2016-01-24T13:06:13.473" LastActivityDate="2016-01-24T13:06:13.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21901" PostTypeId="2" ParentId="21898" CreationDate="2016-01-24T09:07:05.607" Score="1" Body="&lt;p&gt;Pretty simple. Because each fourier function is periodic, in a way that you will 'choose' $n,m$ and $p$ in your problem, and they will have  be multiples of the boundary frequency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so.. let's say that you have a periodic boundary of length $a$ in z... then $p=\frac{a.k}{2\pi}$ with k being an integer. then $e^{ipz}$ will obey the boundary conditions for all values of k.&lt;/p&gt;&#xA;" OwnerUserId="18687" LastActivityDate="2016-01-24T09:07:05.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21902" PostTypeId="1" AcceptedAnswerId="21907" CreationDate="2016-01-24T10:55:57.580" Score="2" ViewCount="104" Body="&lt;p&gt;It is necessary for me to find the element indicator for poisson equation by linear basis in FEM. Therefore I have to find the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eta_k=h_k||f+\Delta u_h||_{L^2(k)}+\frac{1}{2}h_k^{1/2}||n.\nabla u_h||,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I should approximate $\Delta u_h$ numerically, and I need to second order derivative of the shape functions. But they are linear and this means $\Delta u_h=0$. I can not understand what happen here, and what does $||f+\Delta u_h||_{L^2(k)}$ mean? I am so confuesed by this term. Is there anybody can help  me? Am i wrong?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2016-01-24T13:18:02.037" LastActivityDate="2016-01-25T00:52:27.770" Title="Simple question about find the element indicator" Tags="|finite-element|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21904" PostTypeId="1" CreationDate="2016-01-24T15:12:15.010" Score="4" ViewCount="180" Body="&lt;p&gt;For an optimization routine I needed to compute the derivative of the right-hand side $\: f_u(x_k, u_k)$ of a discrete-time system $x_{k+1} = f(x_k, u_k)$. Since $\: f_u(x_k, u_k)$ includes terms that are divided by $u_k$, it is not possible to evaluate $f(x_k, u_k=0)$. From the physics, it's pretty clear that a limit must exist. The numerics poses a problem here, though. Any idea, how I could circumvent it?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To illustrate the behavior, I added a double logarithmic plot showing $f_u(x_k, u_k)$ as $u \rightarrow 0$. It doesn't make any sense that $f_u$ escapes here. What the plot does not show is that there are multiple sign changes of $f_u$ as $u$ drops below $10^{-4}$, let's say.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/igmoJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/igmoJ.png&quot; alt=&quot;loglog plot of $f_u(x_k, u_k)$ as $u_k \rightarrow 0$&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The function $f_u(x_k, u_k)$ is defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_u(x, u) = \partial_u \int_0^T \int_0^t \sin(x_5 + x_6 \tau + u \frac{\tau^2}{2}) \: d\tau.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had a CAS do the job of analytically integrating and derivating the sine term. As already pointed out this produces some division by $u$ which causes the problems when it comes to evaluating at $u=0$.&lt;/p&gt;&#xA;" OwnerUserId="16701" LastEditorUserId="16701" LastEditDate="2016-01-26T17:41:40.730" LastActivityDate="2016-01-26T17:41:40.730" Title="Does the limit of $\frac{\partial f}{\partial u}$ at $u=0$ exist?" Tags="|optimization|numerical-analysis|jacobian|calculus|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="21905" PostTypeId="2" ParentId="21902" CreationDate="2016-01-24T17:24:42.550" Score="3" Body="&lt;p&gt;The proof goes roughly as follows: Let $u$ be exact solution and $u_h$ discrete solution. Moreover, let $e=u-u_h$ and $\pi_h$ be the interpolation operator from $H^1$ to the discrete finite element space. Facts used in proving the a posteriori error estimate are the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Coercivity of the bilinear form.&lt;/li&gt;&#xA;&lt;li&gt;Galerkin orthogonality.&lt;/li&gt;&#xA;&lt;li&gt;Problem statement and elementwise integration by parts.&lt;/li&gt;&#xA;&lt;li&gt;Cauchy-Schwarz for inner products and for sums.&lt;/li&gt;&#xA;&lt;li&gt;Interpolation estimates $\|e-\pi_h e\|_{0,K} \lesssim h_K |e|_{1,K}$ and $\|e-\pi_h e\|_{0,E} \lesssim h_E^{1/2} |e|_{1,K}$ and the definition of $\|\cdot\|_1$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align*}&#xA;\|u-u_h\|_1^2&amp;amp;\lesssim (\nabla(u-u_h),\nabla(u-u_h))\\&#xA;&amp;amp;=(\nabla(u-u_h),\nabla(e-\pi_h e))\\&#xA;&amp;amp;=(f,e-\pi_h e)+\sum_{K}(\Delta u_h, e-\pi_h e)_K-\sum_{E}([[\nabla u_h\cdot n]],e-\pi_h e)_E\\&#xA;&amp;amp;\lesssim \big(\sum_{K} h_K^2 \|\Delta u_h+f\|_{0,K}^2\big)^{1/2} \big(\sum_{K} h_K^{-2}\|e-\pi_h e\|_{0,K}^2\big)^{1/2}\\&#xA;&amp;amp;\phantom{=}+\big(\sum_{E} h_E \|[[\nabla u_h\cdot n]]\|_{0,E}^2\big)^{1/2} \big(\sum_{E} h_E^{-1}\|e-\pi_h e\|_{0,E}^2\big)^{1/2}\\&#xA;&amp;amp;\lesssim \Big(\big(\sum_{K} h_K^2 \|\Delta u_h+f\|_{0,K}^2\big)^{1/2}+\big(\sum_{E} h_E \|[[\nabla u_h\cdot n]]\|_{0,E}^2\big)^{1/2} \Big)\|u-u_h\|_1&#xA;\end{align*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally you divide by $\|u-u_h\|_1$ to get the bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, note that none of the steps are invalid for $u_h|_K \in P^1(K)$. Thus, $\Delta u_h|_K=0$ should work just fine.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastEditorUserId="10160" LastEditDate="2016-01-24T21:10:24.420" LastActivityDate="2016-01-24T21:10:24.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21907" PostTypeId="2" ParentId="21902" CreationDate="2016-01-25T00:52:27.770" Score="3" Body="&lt;p&gt;It is correct that the error estimator&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eta_k=h_k||f+\Delta u_h||_{L^2(k)}+\frac{1}{2}h_k^{1/2}||n.\nabla u_h||,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;simplifies to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eta_k=h_k||f||_{L^2(k)}+\frac{1}{2}h_k^{1/2}||n.\nabla u_h||,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if you have linear elements. However, the term then is suboptimal. (It does have the right convergence order, but its value is too large.) A finer analysis shows that you can replace it by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eta_k=h_k||f-f_h||_{L^2(k)}+\frac{1}{2}h_k^{1/2}||n.\nabla u_h||,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f_h$ is a piecewise constant interpolant of the function $f$ on each cell. This term is called &quot;data oscillation&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-25T00:52:27.770" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21908" PostTypeId="1" AcceptedAnswerId="21909" CreationDate="2016-01-25T14:28:35.243" Score="2" ViewCount="122" Body="&lt;p&gt;Assume $X$ and $N$ are two sets of observations from two different normal distribution, where $X$ represents clean data and $N$ represents noise; and $A$ a projection matrix of a filter and the relation between $X$ and $Y$ defines as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Y=A \times X + N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;considering that $X_i$ are not sequential samples and are just different observations from a normal distribution (there is no time involved), are we allowed to use a Kalman filter to estimate $A$ and parameters of  $N$? &lt;/p&gt;&#xA;" OwnerUserId="18994" LastEditorUserId="18994" LastEditDate="2016-01-26T07:47:46.043" LastActivityDate="2016-03-26T08:37:37.737" Title="Using kalman filter when samples don't have time index" Tags="|machine-learning|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21909" PostTypeId="2" ParentId="21908" CreationDate="2016-01-25T17:01:47.387" Score="1" Body="&lt;p&gt;My example in your last post, which solves this problem using a Kalman Filter, shows you can use non-time dependent observations, in any order, to estimate $A$ and $N$. It's called using Kalman Filters for parameter estimation.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-01-25T17:01:47.387" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21911" PostTypeId="1" CreationDate="2016-01-26T10:05:58.360" Score="1" ViewCount="103" Body="&lt;p&gt;I'm coding the simplex method and observing that it easily falls into cycling,&#xA;even if &lt;a href=&quot;https://en.wikipedia.org/wiki/Bland%27s_rule&quot; rel=&quot;nofollow&quot;&gt;Bland's rule&lt;/a&gt; is used.&#xA;It seems to me I have found the reason and I would like to check my understanding is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems the problem is in tiny detail of the choice of the pivot row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We choose the pivot row &quot;r&quot; by the codition &lt;/p&gt;&#xA;&#xA;&lt;p&gt;B_r / A_{rc} is minimal over &quot;r&quot; among all &#xA;NONnegative values B_r / A_{rc} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;It seems to me the correct way is to change from NONnegative to strictly positive.&#xA;Is it correct ?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least it helps in my example and seems Okay from theory point of view (well, I am not sure I completely understand the theory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In  textbooks like &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/013255593X&quot; rel=&quot;nofollow&quot;&gt;Hamdy A. Taha&lt;/a&gt; and all internet pages they say about &quot;NONnegativity&quot; )&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is example of cycling with Bland's rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+04 *&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-1.0999   -0.9000   -0.7000         0         0         0         0         0   -0.9000&#xA;    0.0001    0.0001    0.0001    0.0001         0         0         0         0    0.0001&#xA;    0.0010    0.0008    0.0006         0    0.0001         0         0         0    0.0008&#xA;    0.0001         0         0         0         0    0.0001         0         0    0.0001&#xA;         0    0.0001         0         0         0         0    0.0001         0    0.0001&#xA;         0         0    0.0001         0         0         0         0    0.0001    0.0000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+03 *&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     0   -0.2008   -0.4006         0    1.0999         0         0         0   -0.2008&#xA;     0    0.0002    0.0004    0.0010   -0.0001         0         0         0    0.0002&#xA;0.0010    0.0008    0.0006         0    0.0001         0         0         0    0.0008&#xA;     0   -0.0008   -0.0006         0   -0.0001    0.0010         0         0         0&#xA;     0    0.0010         0         0         0         0    0.0010         0    0.0006&#xA;     0         0    0.0010         0         0         0         0    0.0010    0.0004&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+03 *&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     0         0   -0.2500         0    1.1250   -0.2510         0         0   -0.2008&#xA;     0         0    0.0003    0.0010   -0.0001    0.0002         0         0    0.0002&#xA;0.0010         0         0         0         0    0.0010         0         0    0.0008&#xA;     0    0.0010    0.0007         0    0.0001   -0.0013         0         0         0&#xA;     0         0   -0.0007         0   -0.0001    0.0013    0.0010         0    0.0006&#xA;     0         0    0.0010         0         0         0         0    0.0010    0.0004&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+03 *&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     0    0.3333         0         0    1.1667   -0.6677         0         0   -0.2008&#xA;     0   -0.0003         0    0.0010   -0.0002    0.0007         0         0    0.0002&#xA;0.0010         0         0         0         0    0.0010         0         0    0.0008&#xA;     0    0.0013    0.0010         0    0.0002   -0.0017         0         0         0&#xA;     0    0.0010         0         0         0         0    0.0010         0    0.0006&#xA;     0   -0.0013         0         0   -0.0002    0.0017         0    0.0010    0.0004&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+03 *&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     0   -0.2008   -0.4006         0    1.0999         0         0         0   -0.2008&#xA;     0    0.0002    0.0004    0.0010   -0.0001         0         0         0    0.0002&#xA;0.0010    0.0008    0.0006         0    0.0001         0         0         0    0.0008&#xA;     0   -0.0008   -0.0006         0   -0.0001    0.0010         0         0         0&#xA;     0    0.0010         0         0         0         0    0.0010         0    0.0006&#xA;     0         0    0.0010         0         0         0         0    0.0010    0.0004&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is the same without Bland's rule &lt;/p&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+04 *&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-1.0999   -0.9000   -0.7000         0         0         0         0         0   -0.9000&#xA;    0.0001    0.0001    0.0001    0.0001         0         0         0         0    0.0001&#xA;    0.0010    0.0008    0.0006         0    0.0001         0         0         0    0.0008&#xA;    0.0001         0         0         0         0    0.0001         0         0    0.0001&#xA;         0    0.0001         0         0         0         0    0.0001         0    0.0001&#xA;         0         0    0.0001         0         0         0         0    0.0001    0.0000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+03 *&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     0   -0.2008   -0.4006         0    1.0999         0         0         0   -0.2008&#xA;     0    0.0002    0.0004    0.0010   -0.0001         0         0         0    0.0002&#xA;0.0010    0.0008    0.0006         0    0.0001         0         0         0    0.0008&#xA;     0   -0.0008   -0.0006         0   -0.0001    0.0010         0         0         0&#xA;     0    0.0010         0         0         0         0    0.0010         0    0.0006&#xA;     0         0    0.0010         0         0         0         0    0.0010    0.0004&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+03 *&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     0    0.3333         0         0    1.1667   -0.6677         0         0   -0.2008&#xA;     0   -0.0003         0    0.0010   -0.0002    0.0007         0         0    0.0002&#xA;0.0010         0         0         0         0    0.0010         0         0    0.0008&#xA;     0    0.0013    0.0010         0    0.0002   -0.0017         0         0         0&#xA;     0    0.0010         0         0         0         0    0.0010         0    0.0006&#xA;     0   -0.0013         0         0   -0.0002    0.0017         0    0.0010    0.0004&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;simplexMatrix =&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.0e+03 *&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     0   -0.2008   -0.4006         0    1.0999         0         0         0   -0.2008&#xA;     0    0.0002    0.0004    0.0010   -0.0001         0         0         0    0.0002&#xA;0.0010    0.0008    0.0006         0    0.0001         0         0         0    0.0008&#xA;     0   -0.0008   -0.0006         0   -0.0001    0.0010         0         0         0&#xA;     0    0.0010         0         0         0         0    0.0010         0    0.0006&#xA;     0         0    0.0010         0         0         0         0    0.0010    0.0004&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1603" LastActivityDate="2016-01-27T08:05:55.420" Title="Simplex method - cycling and condition &quot;&gt;=&quot; or &quot;&gt;&quot; in choice of pivot row" Tags="|optimization|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21912" PostTypeId="1" CreationDate="2016-01-26T10:43:47.940" Score="5" ViewCount="237" Body="&lt;p&gt;Reposted from: &lt;a href=&quot;https://stackoverflow.com/q/34909093/472610&quot;&gt;https://stackoverflow.com/q/34909093/472610&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work in scientific research. Technical tools aren't always ideal or even really thought through sometimes, but with recent developments one recurrent need (in my field and many other I know of) is that of storing reasonably large datasets (10GB - 1TB) with large amounts of meta-data in a reliable and efficient way. I'm not interested in distributed storage at all here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many &lt;em&gt;ad hoc&lt;/em&gt; file formats with variably well-defined specifications exist. Datasets become naturally more complex over time, or integrate data coming from different fields with their own file-formats, which usually results either a big (inefficient and error-prone) mess or new &lt;em&gt;ad hoc&lt;/em&gt; file-formats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many people have chosen to turn to generic storage formats specifically designed for large datasets by people who know what they are doing, like HDF5. The feedback on this experience is mitigated, and many people criticise especially the complexity of designing an efficient structure for their dataset, and the fact that crashes during processing tend to result in data corruption, which in turn means losing the entire dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know very many things about file-systems, but I have some experience with virtual machines and one thing that blew my mind the first time I created one is that I had to create a virtual hard drive, which would simply sit as a big file in my own file-system. Even better, there was the option of creating a resizeable one, so it could grow as I needed more and more memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are, to people who know these things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would there be non-trivial disadvantages in simply storing large datasets as their own file structure in a virtual file system? It would be platform independent, wouldn't it? &lt;/li&gt;&#xA;&lt;li&gt;Would it be possible to choose a file-system with built-in redundancy or checksum features, such that it could deal with data corruption regardless of the host system (eg ZFS)? Are there portable (or at least unix/linux friendly), open libraries, to create/access/modify such virtual filesystems?&lt;/li&gt;&#xA;&lt;li&gt;Would it be possible to mount such drives easily, say as an external drive (just as an &lt;code&gt;.iso&lt;/code&gt; file for example)? Would you know of any portable &amp;amp; open source tool to do this?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15177" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-01-28T17:26:50.677" Title="Storing large datasets on virtual filesystems" Tags="|data-storage|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21914" PostTypeId="1" AcceptedAnswerId="21942" CreationDate="2016-01-26T15:29:12.463" Score="3" ViewCount="182" Body="&lt;p&gt;Assume I start with an &lt;code&gt;8x8&lt;/code&gt; coarse mesh (see &lt;strong&gt;Fig 1&lt;/strong&gt;) where the vertices (except boundary vertices) represent the unknown variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sKg2R.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sKg2R.jpg&quot; alt=&quot;Figure 1 - coarsest mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After iterative approximation - I find 2 vertices (unknowns) where the error exceeds a certain bound (see &lt;strong&gt;Figure 2&lt;/strong&gt; red dots).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6H1Jz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6H1Jz.jpg&quot; alt=&quot;Red dots show high error vertices&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I further refine the red dots (high error points) (see Figure 3 - only 1 red dot refinement shown). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jsM9C.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jsM9C.jpg&quot; alt=&quot;Green X&amp;#39;s - new unknowns, red X&amp;#39;s - hanging nodes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can I interpolate/prolongate the red X's (hanging nodes) from black X's of the coarser grid ? i.e. I don't need to solve PDE there.&lt;/li&gt;&#xA;&lt;li&gt;Do the black X's and red X's become boundaries for green X's ? (I want to apply Finite Difference at green X's).&lt;/li&gt;&#xA;&lt;li&gt;I believe refinement can be done to any level but generally a 2:1 balance is maintained to have a single hanging node per edge (in 2-D). Do we refine multiple times before updating or just once ?&lt;/li&gt;&#xA;&lt;li&gt;I will be using a quad tree with each tree node containing 4 vertices (with ownership status) - for a vertex centred finite difference implementation but it will require querying adjacent neighbours (H. Samet, 1982 paper) which looks in-efficient. Any suggestions on this ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="14303" LastEditDate="2016-01-28T10:56:17.643" LastActivityDate="2016-01-28T17:21:41.073" Title="Refinement in AMR" Tags="|pde|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21915" PostTypeId="1" CreationDate="2016-01-26T15:55:06.033" Score="8" ViewCount="4818" Body="&lt;p&gt;I have some data defined in an array (an image) and I need to find the curl of a certain function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia has an integral definition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Curl_%28mathematics%29&quot; rel=&quot;noreferrer&quot;&gt;curl&lt;/a&gt; that I like, maybe it can be discrete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla \times F = \frac{1}{|A|} \oint F \cdot d \mathbf{r}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we write this as some kind of finite difference. Let $F = (F_1, F_2, F_3)$. I can think of two different ways to write curl:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{1}{(2\epsilon)^2} \big[ F_2(x+\epsilon, y, z) + F_1(x, y+\epsilon, z) - F_2(x-\epsilon, y, z) + F_1(x, y-\epsilon, z) \Big] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully I wrote that average correctly.  What if I used the corners of the square instead?  Do we get curl still?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{1}{(2\epsilon)^2}\Big[  F(x+\epsilon, y + \epsilon, z) + F(x+\epsilon, y - \epsilon, z)  - F(x-\epsilon, y - \epsilon, z)  - F(x-\epsilon, y + \epsilon, z)   \Big] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what to call these two operators $\nabla_1 $ and $\nabla_2$ who do these operators differ as $\epsilon \ll 1$ ?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My second definition of curl breaks down a little bit.  However all the points $(x,y,z) + \epsilon \,(\pm 1, \pm 1, 0)$ all lie on a circle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{1}{ \pi \epsilon^2} \sum_{\theta = \{ \frac{2\pi k}{n}: 0 \leq k &amp;lt; n \}} F\big[ (x,y,z) + \epsilon( \cos \theta, \sin \theta, 0 )\big] \cdot (-\sin \theta, \cos \theta, 0)  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This operator should be proportional to curl for small $\epsilon \approx 0$, I think, but this is harder to put into a computer.&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="4006" LastEditDate="2016-01-26T16:20:34.983" LastActivityDate="2017-08-08T07:49:49.133" Title="discrete definitions of curl $\nabla \times F$?" Tags="|finite-difference|numerical-analysis|error-estimation|image-processing|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21916" PostTypeId="1" CreationDate="2016-01-26T16:05:46.217" Score="6" ViewCount="181" Body="&lt;p&gt;I need to ask a question about partial derivatives.&#xA;I want to solve this equation (steady state, one dimensional continuity equation):&#xA;$$\frac{\partial (\rho u)}{\partial z}=0$$&#xA;which is equivalent to:&#xA;$$\rho\frac{\partial u}{\partial z}+u\frac{\partial \rho}{\partial z}=0$$&#xA;and discretized by means of finite differences:&#xA;$$\rho_i\frac{(u_i-u_{i-1})}{\delta z}+u_i\frac{(\rho_i-\rho_{i-1})}{\delta z}=0$$&#xA;along with initial conditions like:&#xA;$$z=0 \quad \rho_i=\rho_0 \quad , \quad u_i=u_0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;My questions are:&lt;/em&gt;&lt;/strong&gt; Does this equation has one unique solution? I mean one could increase $u$ and decrease $\rho$ (or the other way around) and have infinite solutions right?&lt;br&gt;&#xA;In this case this equation can describe the conservation of the quantity $(\rho u)$. What about the conservation or $u$ or $\rho$?&lt;br&gt;&#xA;If I could calculate the $\rho$ (density) variable through lets say an equation of state (like $PV=nRT$) which would be variable in $z$ direction can i rewrite the first equation like:&#xA;$$\rho\frac{\partial u}{\partial z}=0$$&#xA;$$\rho_i\frac{(u_i-u_{i-1})}{\delta z}=0$$&#xA;which do have one unique solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to let you know of my intention, I eventually want to solve a set of PDEs:&#xA;$$\frac{\partial \rho}{\partial t}&#xA;+\frac{\partial (\rho u)}{\partial z}=0$$&#xA;$$\frac{\partial C_i}{\partial t}&#xA;+\frac{\partial (u C_i)}{\partial z}=Reactionrate$$&#xA;$$\rho=MW_{mean}\frac{P}{RgT}, MW_{mean}=\sum^{N_{comp}}_{i=1}(Y_i*mw_i), Y_i=\frac{C_i}{\sum^{N_{comp}}_{i=1}{C_i}}$$&lt;/p&gt;&#xA;" OwnerUserId="19040" LastEditorUserId="18870" LastEditDate="2016-02-03T19:11:24.147" LastActivityDate="2016-02-03T19:11:24.147" Title="A simple PDE solution question" Tags="|pde|fluid-dynamics|numerical-modelling|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21917" PostTypeId="2" ParentId="21904" CreationDate="2016-01-26T17:40:45.800" Score="6" Body="&lt;p&gt;Given that $u \frac{\tau^2}{2} \ll 1$, one way of tackling the numerical oscillations, even &lt;em&gt;before&lt;/em&gt; the actual term emerges, is a Taylor approximation in $u$ of the sine term (Thanks to Kirill for the full series):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sin(\underbrace{x_5 + x_6 \tau}_{u_0} + \underbrace{u \frac{\tau^2}{2}}_{\Delta u}) = \sin(x_5 + x_6 \tau) + \sum_{n=1}^N \frac{(-1)^{\lfloor n/2 \rfloor}}{2^n(n-1)!} \tau^{2n}Q_nu^{n} + o(\Delta u^{N}),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Q_n = &#xA;\begin{cases}&#xA;\cos(x_5 + x_6 \tau) &amp;amp; n \text{ odd} \\&#xA;\sin(x_5 + x_6 \tau) &amp;amp; \text{else.}&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introducing this approximation into the original equation and exchanging order of integration and differentiation gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;\begin{array}{ll}&#xA;f_u(x,u) &amp;amp; = \int_0^T \int_0^t \partial_u \left( \sin(x_5 + x_6 \tau) + \sum_{n=1}^N \frac{(-1)^{\lfloor n/2 \rfloor}}{2^n(n-1)!} \tau^{2n}Q_nu^{n} + o(\Delta u^{N}) \right) \: d\tau \\&#xA;&amp;amp; = \int_0^T \int_0^t \sum_{n=1}^N \frac{(-1)^{\lfloor n/2 \rfloor}}{2^n(n-1)!} \tau^{2n}Q_nu^{n-1} + o(\Delta u^{N-1}) \: d\tau.&#xA;\end{array} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A first-order approximation ($N=1$) is then given as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{ll}&#xA;f_u(x,u) &amp;amp; \approx \int_0^T \int_0^t \cos(x_5 + x_6 \tau) \frac{\tau^2}{2} \: d\tau \\&#xA;&amp;amp; = \frac{-3 \cos(x_5)}{x_6^4} + \frac{(6-T^2x_6^2)\cos(x_5 + Tx_6) + 2Tx_6(\sin(x_5) + 2 \sin(x_5 + Tx_6))}{2x_6^4},&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which does not depend on $u$, naturally.&lt;/p&gt;&#xA;" OwnerUserId="16701" LastActivityDate="2016-01-26T17:40:45.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21919" PostTypeId="2" ParentId="21398" CreationDate="2016-01-26T20:15:10.387" Score="3" Body="&lt;p&gt;It seems you are looking for an optimization (here, specifically a fit).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have some experimental data and you want to have a simple model explaining it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest way is using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ordinary_least_squares&quot; rel=&quot;nofollow&quot;&gt;Ordinary Least Squares&lt;/a&gt;. The idea is to minimize the standard deviation or the error you make on the vertical axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another one would be the &lt;a href=&quot;https://en.wikipedia.org/wiki/Principal_component_analysis&quot; rel=&quot;nofollow&quot;&gt;PCA&lt;/a&gt; where you try to find an angle to view your variables to find the maximum information. It's based on the eigenvectors of your system. The advantage is that it only tries to find the best solution to minimize the orthogonal error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many other solutions are possibles but it would be preferable to have more details about what you're exactly looking for and the link between variables.&lt;/p&gt;&#xA;" OwnerUserId="19033" LastActivityDate="2016-01-26T20:15:10.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21921" PostTypeId="2" ParentId="21896" CreationDate="2016-01-26T20:37:57.553" Score="1" Body="&lt;p&gt;Depending of your problem, adding more variables can make the problem extremely easy to solve. All will depend of the structure of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, LP problems are solved using a LU decomposition of the constraint matrix. If the decomposition is easy, the problem will be fast to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example for a sum. You can have a problem where you directly have the sum with only 1s under the diagonal. If you double the size of the problem where you have the value and the sum, then you only have 2 identity matrix one next to the other assuming you compute the sum recursively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem can be solve in a o(N) where the first problem will be in o(N³).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, in that case, having a sparse matrix, although twicce the size, will considerably reduce the overall time to solve.&lt;/p&gt;&#xA;" OwnerUserId="19033" LastActivityDate="2016-01-26T20:37:57.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21922" PostTypeId="1" AcceptedAnswerId="21924" CreationDate="2016-01-26T22:32:23.917" Score="2" ViewCount="3413" Body="&lt;p&gt;A scalar field &lt;strong&gt;f&lt;/strong&gt; approximated on a triangular mesh using the FEM method can be given as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x,y) \approx \bar{f}(x,y) = \sum_{n=1}^3 \left(\phi_n(x,y) \cdot f_n \right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am of the understanding, then, that the gradient of &lt;strong&gt;f&lt;/strong&gt; can be found inside each triangle with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \bar{f}_t = \sum_{n=1}^3 \left(\frac{\partial\phi_n(x,y)}{\partial x} \cdot f_n , \frac{\partial\phi_n(x,y)}{\partial y} \cdot f_n\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $f_t$ is the value at the triangle centroids. Since the value at each point $f_n$ is only defined at single points, it acts as a scalar in the derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, what if we then want to take the gradient of each of the components from the previous solution? One way seems to be to interpolate the gradient results from the triangle centroids back to the given points, and then repeat the process for each component. However, what if this interpolation is unsatisfactory -- is there a way to possibly define the gradient at each &lt;em&gt;point&lt;/em&gt; from data at the &lt;em&gt;triangle centroids&lt;/em&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to be a similiar question posed &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19403/find-the-direction-of-the-gradient-on-a-finite-element-mesh&quot; title=&quot;here&quot;&gt;here&lt;/a&gt;, but I'm not sure if the solution is relevant -- the answer solution seems to involve going through &lt;em&gt;all&lt;/em&gt; the basis functions of adjacent triangles, whereas it seems to me that only one basis function from each adjacent triangle is needed -- the one which has a nonzero value at the observed point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this in mind, I've derived the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \bar{f}_i = \sum_{n=1}^{t_i} \left(\left( \frac{\partial\phi_i^n(x,y)}{\partial x}, \frac{\partial\phi_i^n(x,y)}{\partial y} \right) \frac{f_n}{t_i} \right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;em&gt;i&lt;/em&gt; is the observed point, and $t_i$ is the number of triangles which contain the vertex point &lt;em&gt;i&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, it doesn't seem to work, and I'm not sure if averaging over the number of triangles is appropriate. Can someone point me in the right direction?&lt;/p&gt;&#xA;" OwnerUserId="17983" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-01-26T23:41:35.777" Title="Gradients on 2D FEM Triangular Mesh" Tags="|finite-element|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21923" PostTypeId="1" CreationDate="2016-01-26T22:46:11.180" Score="1" ViewCount="1882" Body="&lt;p&gt;I'm having trouble solving a second order differential equation with Gaussian white noise. The equation I'm solving follows the form: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax'' + Bx' + \sin(x) = i + i_{n}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $i_{n}$ is the Gaussian white noise, and $i$ is a constant source. I'm ultimately interested in $x'$, not $x$. Without the Gaussian source, I've been able to simply use an ODE solver like &lt;code&gt;ode15s&lt;/code&gt; or &lt;code&gt;ode45&lt;/code&gt;. However, with the random source, I don't know how to use an SDE solver to get $x'$. Can anyone with experience in solving SDEs in MATLAB lend some advice?&lt;/p&gt;&#xA;" OwnerUserId="19045" LastEditorUserId="9667" LastEditDate="2016-01-26T23:38:53.010" LastActivityDate="2016-01-27T19:28:14.510" Title="Solving second order SDE with Gaussian white noise for first time derivative in Matlab" Tags="|matlab|ode|stochastic|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21924" PostTypeId="2" ParentId="21922" CreationDate="2016-01-26T23:41:35.777" Score="4" Body="&lt;p&gt;Let me try a quick and dirty answer. Suppose that you have linear shape functions $\phi_n(x,y)$: if you take the gradient of the shape functions (e.g. for computing the energy norm and the stiffness matrix) you end up with a constant function. (Hence the name CST in continuum mechanics: &lt;strong&gt;C&lt;/strong&gt;onstant &lt;strong&gt;S&lt;/strong&gt;train &lt;strong&gt;T&lt;/strong&gt;riangle, since strain is related to the (deformation) gradient tensor of the displacement field.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence if you compute the gradient $\nabla \bar{f}$, this will be piecewise constant on each FE, and is only a crude approximation of $\nabla f$. However Barlow noted super convergence of $\nabla \bar{f}$ in certain interior points of the FE. This observation leads to the following recipe, which is universally adopted in FE post processing programs, when a representation of the gradient $\nabla f$ is required.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;evaluate the gradient field at numerical quadrature points (since most FE's are numerically integrated): for a CST triangle this is a single point at the centroid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;extrapolate the gradient from the quadrature points to the element nodes. For a CST since you have a single quadrature point, no extrapolation is necessary: simply take the centroid value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;at each node take the average of the gradient value extrapolated from all elements that have that node in common. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;use the shape functions to interpolate (inside the FEs) the nodal averaged value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This recipe is indeed very similar to what I understand from the OP question: so I think his basic idea is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The theory for this technique (nodal averaging over a path of elements extrapolating from the super convergent points for the flux) can be found in O.C. Zienkiewicz and J.Z. Zhu, The Superconvergent patch recovery and a posteriori error estimators. Part 1. The recovery technique, Int. J. Numer. Methods Eng., 33, 1331-1364 (1992).&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2016-01-26T23:41:35.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21925" PostTypeId="2" ParentId="21911" CreationDate="2016-01-27T00:58:18.237" Score="4" Body="&lt;p&gt;The short answer to your question is that &quot;nonnegative&quot; is correct.  0 ratios do occur and it is sometimes necessary to perform degenerate pivots in order to ultimately reach a solution which will pass the simplex method's optimality test.  However, you need to carefully deal with near zero values of $B_{r}$ and $A_{rc}$ or else round-off can easily lead to incorrect pivot selections.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notation for the simplex method in tableau form is not completely standardized.  In the following, I'll assume that we're solving a maximization problem, so that variables with positive reduced costs are candidates to enter the basis.  You can easily adjust this for minimization problems.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, because of floating point round off errors, it can be difficult to determine whether $B_{r}$ is 0, positive, or negative (which would be an error condition.)  You'll need to set a tolerance $\epsilon$, and declare $B_{r}$ to be effectively 0 if $| B_{r} | \leq \epsilon$.  If $B_{r} &amp;gt; \epsilon$, then $B_{r}$ is effectively positive.  Similarly, if $B_{r} &amp;lt; -\epsilon$, then $B_{r}$ is effectively negative and you should stop with an error.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, you need to check whether $A_{rc}$ is effectively zero ($|A_{rc} \leq \epsilon$) or whether it is effectively positive.  The basic variable in row $r$ is a candidate to leave the basis only if $A_{rc}$ is effectively positive.   Do not consider row $r$ if $A_{rc}$ is effectively 0 or negative.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In computing the ratio $B_{r}/A_{rc}$ where $A_{rc}$ is effectively positive and $B_{r}$ is effectively 0 you should replace the actual value of $B_{r}$ by 0 and treat the ratio as 0.  For example, suppose $\epsilon=1.0e-8$.  If $A_{rc}=10$ and $B_{r}=-1.0e-16$, then you should treat the ratio as 0.  Similarly, if $A_{rc}=10$ and $B_{r}=+1.0e-16$, then you should treat the ratio as 0.  If you do not do this, then round off errors can easily lead you to incorrectly conclude that the ratio for some row is negative when it's really effectively 0.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, Bland's rule can be stated as follows.  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider only columns where the reduced cost is effectively positive.  Let $c$ be the first column with an effectively positive reduced cost.  (I'm assuming here that the columns are kept in the original order of the variables- you want the entering variable $x_{j}$ with the smallest $j$.)   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider only rows $r$ where $A_{rc}$ is effectively positive.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For those rows under consideration, compute the ratios $B_{r}/A_{rc}$, replacing effectively 0 values of $B_{r}$ with actual values of $0$.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pick $r$ corresponding to the smallest ratio if there is a unique row with this ratio.  If there are multiple rows with the same minimum ratio then pick the one with the smallest index $r$.  (I'm assuming here that you've ordered the rows by the indices of the basic variables- you actually want to pick the leaving basic variable with the smallest index among those that are tied.)   &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-01-27T08:05:55.420" LastActivityDate="2016-01-27T08:05:55.420" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21926" PostTypeId="2" ParentId="21923" CreationDate="2016-01-27T01:13:40.993" Score="3" Body="&lt;p&gt;The most straightforward way to solve your SDE is with an &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler%E2%80%93Maruyama_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler-Maruyama&lt;/a&gt; scheme. This is a simple and effective method for additive noise, i.e., the diffusion/noise term is not a function of the state, as appears to be the case for your example. Here is some Matlab code to solve your system:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 2;        % Order of system&#xA;t0 = 0;       % Initial time&#xA;dt = 1e-3;    % Fixed time step&#xA;tf = 1;       % Final time&#xA;t = t0:dt:tf; % Time vector&#xA;tlen = length(t);&#xA;&#xA;A = 1;&#xA;B = 1;&#xA;c = 1;&#xA;f = @(t,x)[x(2);(-B*x(2)-sin(x(1))+c)/A]; % Drift function&#xA;ep = 1e-1;                                % Size of additive noise&#xA;g = @(t,x)[0;ep];                         % Diffusion function&#xA;&#xA;x = zeros(lt,n); % Allocate output&#xA;x0 = [1;1];&#xA;x(1,:) = x0;     % Set initial condition&#xA;&#xA;seed = 1;  % Seed value&#xA;rng(seed); % Always seed random number generator&#xA;&#xA;% Euler-Maruyama&#xA;for i = 1:tlen-1&#xA;    x(i+1,:) = x(i,:).' + f(t(i),x(i,:))*dt + g(t(i),x(i,:)).*randn(n,1)*sqrt(dt);&#xA;end&#xA;&#xA;figure;&#xA;plot(t,x);&#xA;xlabel('Time');&#xA;ylabel('State');&#xA;legend('x(t)','xdot(t)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is not an adaptive method like &lt;code&gt;ode45&lt;/code&gt;, so you need to ensure that your integration step size, &lt;code&gt;dt&lt;/code&gt;, is sufficiently small. There are many ways to optimize the above code and make it less general (e.g., simplify anonymous functions, pre-calculate normal variates, etc.). You could also try using &lt;code&gt;sde_euler&lt;/code&gt; in my &lt;a href=&quot;https://github.com/horchler/SDETools&quot; rel=&quot;nofollow noreferrer&quot;&gt;SDETools&lt;/a&gt; Matlab toolbox on Github, which has many options akin to those in the ODE suite. See &lt;a href=&quot;https://math.stackexchange.com/a/745513/80812&quot;&gt;this answer&lt;/a&gt; on Math.SE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For methods specialized to second-order SDEs like yours, you could check out &lt;a href=&quot;https://www1.maths.leeds.ac.uk/~grant/BLL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (PDF) by Burrage, et al. For an introduction to numerically solving SDEs, I recommend this paper, which includes many Matlab examples:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Desmond J. Higham, 2001, An Algorithmic Introduction to Numerical Simulation of Stochastic Differential Equations, SIAM Rev. (Educ. Sect.), 43 525–46. &lt;a href=&quot;http://dx.doi.org/10.1137/S0036144500378302&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1137/S0036144500378302&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The URL to the Matlab files in the paper won't work; &lt;a href=&quot;http://personal.strath.ac.uk/d.j.higham/algfiles.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;use this one&lt;/a&gt;. Also, note that random number generator seeding used in the code is now deprecated.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-01-27T19:28:14.510" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21929" PostTypeId="2" ParentId="2779" CreationDate="2016-01-27T10:28:39.950" Score="6" Body="&lt;p&gt;Even if this is an old question, I think that it is worth mentioning &lt;a href=&quot;http://julialang.org/&quot;&gt;Julia&lt;/a&gt;'s particular solution. What this language does is &quot;class-less OOP&quot;: the main constructs are types, i.e., composite data objects akin to &lt;code&gt;struct&lt;/code&gt;s in C, on which an inheritance relation is defined. The types do not have &quot;member functions&quot;, but each function has a type signature and accepts subtypes. For instance, you could have an abstract &lt;code&gt;Matrix&lt;/code&gt; type and subtypes &lt;code&gt;DenseMatrix&lt;/code&gt;, &lt;code&gt;SparseMatrix&lt;/code&gt;, and have a generic method &lt;code&gt;do_something(a::Matrix, b::Matrix)&lt;/code&gt; with specialization&lt;code&gt;do_something(a::SparseMatrix, b::SparseMatrix)&lt;/code&gt;. &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiple_dispatch&quot;&gt;Multiple dispatching&lt;/a&gt; is used to select the most appropriate version to call.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is more powerful than class-based OOP, which is equivalent on dispatching based on inheritance on the first argument only, if you adopt the convention that &quot;a method is a function with &lt;code&gt;this&lt;/code&gt; as its first parameter&quot; (common e.g. in Python). Some form of multiple dispatch can be emulated in, say, C++, but with &lt;a href=&quot;https://en.wikipedia.org/wiki/Double_dispatch&quot;&gt;considerable contortions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main distinction is that methods do not belong to classes, but they exist as separate entities and inheritance can happen on all parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some references:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.julialang.org/en/release-0.4/manual/methods/&quot;&gt;http://docs.julialang.org/en/release-0.4/manual/methods/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://assoc.tumblr.com/post/71454527084/cool-things-you-can-do-in-julia&quot;&gt;http://assoc.tumblr.com/post/71454527084/cool-things-you-can-do-in-julia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://thenewphalls.wordpress.com/2014/03/06/understanding-object-oriented-programming-in-julia-inheritance-part-2/&quot;&gt;https://thenewphalls.wordpress.com/2014/03/06/understanding-object-oriented-programming-in-julia-inheritance-part-2/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2016-01-27T10:28:39.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21930" PostTypeId="1" AcceptedAnswerId="21941" CreationDate="2016-01-27T13:49:41.287" Score="1" ViewCount="150" Body="&lt;p&gt;I have the following problem,&#xA;I have a class of N people and I want them to do stuff by pair,&#xA;but I want them to do this with everybody but as fast as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For N = 4 I got this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A-B C-D&#xA;A-C B-D&#xA;A-D B-C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so in three turns it's finished.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to find an algorithm (simple to explain if possible)&#xA;to solve this problem for any N.&lt;/p&gt;&#xA;" OwnerUserId="19052" LastEditorUserId="651" LastEditDate="2016-01-29T14:01:32.797" LastActivityDate="2016-01-29T14:01:32.797" Title="Round-robin pairings: Everybody need to meet everybody" Tags="|optimization|algorithms|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21931" PostTypeId="1" CreationDate="2016-01-27T15:53:55.843" Score="1" ViewCount="309" Body="&lt;p&gt;I am aware that I could use deconvolution algorithms such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Richardson%E2%80%93Lucy_deconvolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;LUCY&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Wiener_deconvolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weiner&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Blind_deconvolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blind&lt;/a&gt; but I am confused as to how to estimate the PSF (point spread function), which is needed for the LUCY and Weiner algorithms. I am currently using the Widefield Florescence Microscopy setup. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read in a lot of places around the internet that a fluorescent bead that represents a sub resolution object and therefore a impulse function. While I understand that the OTF (optical transfer function) is represented by the image I am getting, how am I superposed to deconvolve the image I have using the very same image? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of the image I am trying to deconvolve: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VYfqX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VYfqX.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a program which automatically singles in on the bead and crops the rest of the image. All I need to do is apply deconvolution to this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My ultimate aim to achieve optical sectioning by deconvolving a bunch of 2D slices I got from a video and representing them with a 3D software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So can someone please explain what I need to do, either theoretically (or in terms of MATLAB, which would be preferred).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;One Possible Solution&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After talking to a few people, I have noticed that EPFL have a &lt;a href=&quot;http://bigwww.epfl.ch/algorithms/psfgenerator/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Java resource&lt;/a&gt; that is callable from MATLAB. It's a PSF generator, based on the specifications of your microscope. This generated PSF can be used with the LUCY and Weiner.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative Solution&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also found &lt;a href=&quot;http://in.mathworks.com/matlabcentral/fileexchange/31945-widefield-fluorescence-microscope-point-spread-function&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; resource, which is a MATLAB library dedicated to what I am working on right now. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dsp.stackexchange.com/questions/28407/what-are-the-figures-of-merit-performance-measurements-i-can-use-for-deconvolut&quot;&gt;This&lt;/a&gt; (I would love it, if someone attempts to answer this question too) is how I will measure the performance of the deconvolution algorithms. I believe that the ultimate result of my deconvolution will depend on the PSF estimation (EPFL resource or Praveen's algorithm) and the deconvolution algorithm (Blind, LUCY, Weiner etc.). I will post a table of all the results here once this is done. &lt;/p&gt;&#xA;" OwnerUserId="19057" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2016-12-20T15:42:48.417" Title="I need to deconvolve the image of a fluorescent bead without the Point Spread Function in MATLAB" Tags="|matlab|algorithms|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21932" PostTypeId="1" CreationDate="2016-01-27T20:30:17.600" Score="1" ViewCount="65" Body="&lt;p&gt;I need some advices to start to solve some problems about fluid motion in a conduit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem domain is shown below: a conduit with some obstacle of different geometric properties. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m7zQ7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m7zQ7.png&quot; alt=&quot;Motion of a fluid in a conduit&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fluid is a &quot;real&quot; and viscous, the viscosity is very high in relation to speed (low speed, very high viscosity). The fluid is in pressure and comes from the blue arrow, I need to know the &quot;speed or amount&quot; of fluid in each output (red arrows). I don't need a real-time simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I approach this kind of problem?&lt;/p&gt;&#xA;" OwnerUserId="19059" LastEditorUserId="9667" LastEditDate="2016-01-27T20:52:10.963" LastActivityDate="2016-01-27T20:52:10.963" Title="Calculate fluid speed in a conduit with obstacles" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="21933" PostTypeId="1" CreationDate="2016-01-27T22:54:53.260" Score="1" ViewCount="352" Body="&lt;p&gt;I have problem to solve stiff equations. Any idea on how to solve this? I have tried &quot;StiffSwitching&quot; but it didnt work. Im solving this using Mathematica 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code. Im sorry if I wrote the code badly but I have fllowed the instructions on how to copy and paste code from Mathematica and this is what I end up with. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;With[{n = 0.6}, ODE3[L_] := {Derivative[1][H][\[Eta]] == -2*F[\[Eta]] - ((1 - n)/(1 + n))*\[Eta]*Derivative[1][F][\[Eta]], &#xA;        F[\[Eta]]^2 - (G[\[Eta]] + 1)^2 + (H[\[Eta]] + ((1 - n)/(1 + n))*\[Eta]*F[\[Eta]])*Derivative[1][F][\[Eta]] == &#xA;         (Derivative[1][F][\[Eta]]^2 + Derivative[1][G][\[Eta]]^2)^((n - 1)/2)*Derivative[2][F][\[Eta]] + &#xA;          Derivative[1][F][\[Eta]]*((n - 1)*(Derivative[1][F][\[Eta]]^2 + Derivative[1][G][\[Eta]]^2)^((n - 3)/2)*(Derivative[1][F][\[Eta]]*Derivative[2][F][\[Eta]] + &#xA;             Derivative[1][G][\[Eta]]*Derivative[2][G][\[Eta]])), 2*F[\[Eta]]*(G[\[Eta]] + 1) + (H[\[Eta]] + ((1 - n)/(1 + n))*\[Eta]*F[\[Eta]])*Derivative[1][G][\[Eta]] == &#xA;         (Derivative[1][F][\[Eta]]^2 + Derivative[1][G][\[Eta]]^2)^((n - 1)/2)*Derivative[2][G][\[Eta]] + &#xA;          Derivative[1][G][\[Eta]]*((n - 1)*(Derivative[1][F][\[Eta]]^2 + Derivative[1][G][\[Eta]]^2)^((n - 3)/2)*(Derivative[1][F][\[Eta]]*Derivative[2][F][\[Eta]] + &#xA;             Derivative[1][G][\[Eta]]*Derivative[2][G][\[Eta]])), F[0] == 0, G[0] == 0, H[0] == 0, F[L] == 0, G[L] == 0}]&#xA;&#xA;&#xA;sol4[L_] := Flatten[NDSolve[ODE3[L], {F, G, H}, \[Eta], &#xA;    Method -&amp;gt; {&quot;Shooting&quot;, &quot;StartingInitialConditions&quot; -&amp;gt; {G[0] == 0, Derivative[1][G][0] == -0.65, F[0] == 0, Derivative[1][F][0] == 0.55, &#xA;        H[0] == 0}}]]&#xA;&#xA;&#xA;Plot[Evaluate[{F[\[Eta]], G[\[Eta]], H[\[Eta]]} /. sol4[14]], {\[Eta],&#xA;   0, 14}, PlotStyle -&amp;gt; Thick, Frame -&amp;gt; True, &#xA; FrameLabel -&amp;gt; {&quot;\[Eta]&quot;, &quot;F, G, H&quot;}, &#xA; Epilog -&amp;gt; {Dashed, Thickness[0.005], &#xA;   Line[{{0, 0.8844}, {8, 0.8844}}]}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got this type of error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;At \[Eta] == 11.976704516552642`, step size is effectively zero; singularity or stiff system suspected. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This only works if I let n=1. But I have to plot for different values of n ranging from 0.6 to 1.4. Below is what plots I got from Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/61kAA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/61kAA.jpg&quot; alt=&quot;This is what Mathematica gives me&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be great if someone could help me and explain to me. Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="19032" LastEditorUserId="19032" LastEditDate="2016-01-27T23:30:21.860" LastActivityDate="2016-01-28T13:42:44.900" Title="Solving stiff equations in Mathematica" Tags="|ode|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21934" PostTypeId="1" CreationDate="2016-01-28T06:42:23.167" Score="5" ViewCount="179" Body="&lt;p&gt;I am seeking of any already written R package which could help in an optimization technique which is called  Difference of convex functions. This technique is sketched &lt;a href=&quot;http://ocw.mit.edu/courses/sloan-school-of-management/15-097-prediction-machine-learning-and-statistics-spring-2012/projects/MIT15_097S12_proj5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and could be very useful for reinforcement learning processes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know something, that could be helpful? &#xA;I have found &lt;a href=&quot;https://cran.r-project.org/web/packages/cccp/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;package called &lt;strong&gt;cccp&lt;/strong&gt;&lt;/a&gt;, but it is all about conic optimization and I don't understand now how could it be used for such a problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to solve the following (a bit simplified) minimization problem, which decomposes in difference of two convex functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min_\mathbf{w} S(\mathbf{w}) =\min_\mathbf{w} \sum_{i=1}^{n}  \min ( \dfrac{|X_i\mathbf{w}|}{\phi}, 1) =  \min_\mathbf{w}~ (S_1(\mathbf{w}) - S_2(\mathbf{w}))$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S_1(\mathbf{w}) =  \sum_{i=1}^{n}  \dfrac{|X_i\mathbf{w}|}{\phi} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S_2(\mathbf{w}) =  \sum_{i=1}^{n}    \left( \dfrac{|X_i\mathbf{w}|}{\phi} - 1   \right)^+$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$(x)^+ = \max(x, 0)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have any suggestions or useful related links with any examples, they would be very appreciated.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. I have found that difference of convex functions optimization problem is closely related with &quot;concave-convex optimization procedure&quot; (CCCP). If anyone has thoughts of how the aforementioned problem could be solved by CCCP, please share!&lt;/p&gt;&#xA;" OwnerUserId="19064" LastEditorUserId="22284" LastEditDate="2019-03-15T21:38:55.593" LastActivityDate="2019-03-15T21:38:55.593" Title="Difference of convex functions optimization problem in R" Tags="|convex-optimization|r|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21935" PostTypeId="1" CreationDate="2016-01-28T09:03:01.520" Score="3" ViewCount="343" Body="&lt;p&gt;I have $n$ boolean variables $x_1,\ldots,x_n$ with associated real-valued costs $c_1,\ldots,c_n$, respectively, and a boolean function $(x_1,\ldots,x_n)\mapsto\Phi(x_1,\ldots,x_n)$ in conjunctive normal form (CNF). I'm looking for a dedicated (ideally free) software package to find an assignment for $x_1,\ldots,x_n$ that satisfies $\Phi$ such that the total cost, that is, the sum of those $c_1,\ldots,c_n$ for which the respective $x_1,\ldots,x_n$ are assigned a true value, is minimised.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current solution is to convert the problem instance to a binary program and solve that with an implementation of the simplex algorithm + branch and cut (I'm using the GLPK package). I'm wondering whether there are software solutions better tailored for my kind of problem.&lt;/p&gt;&#xA;" OwnerUserId="19067" LastActivityDate="2016-10-24T00:26:04.060" Title="optimal SAT solver with weighted variables" Tags="|optimization|mixed-integer-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21936" PostTypeId="2" ParentId="21892" CreationDate="2016-01-28T10:04:49.107" Score="0" Body="&lt;p&gt;Just to summarize the answers given for this question, here I provide a visualization of different methods carried out to learn the projection matrix $A$ given clean samples $X_i$ and noisy samples $Y_i$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HUooa.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HUooa.gif&quot; alt=&quot;learning the projection matrix using different methods: first row: original matrix for 5 different cases, second row: initialization of the matrix, third row: using normal equation, directly derive the projection matrix, fourth row: linear regression using gradient descent and finally, fifth row, using kalman filtering&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first row we can see 5 different projection matrices ($A$) chosen to project $X$ before adding Gaussian noise $N$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second row, you can find the random projections initialized to be learned via different methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third and fourth rows are related to a solution suggested in &lt;a href=&quot;https://cs.stackexchange.com/questions/52154/learning-parameters-of-noise-and-filter-coefficients-from-data-where-data-and-no/52412#52412&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the  third row, using the normal equation, we directly derive the projection matrix:&#xA;$$ \hat A = (X^T \times X)^{-1} \times X^T \times Y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the fourth row, a linear regression using gradient descent is used to learn the projections via $X$ and $Y$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally, the last row, using kalman filtering method suggested by &lt;strong&gt;choward&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can observe, the GD solution and the normal equation are very similar and since in this example, $(X^T \times X)^{-1}$ is feasible to calculate, it sounds a more reasonable solution and can be done in only one step. If the dimensionality of our features were too high, then we can use the GD since both $X$ and $Y$ are Gaussian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The KF solution converges faster than GD, but can not achieve a good performance as GD, I believe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After learning $A$, $N$ can be calculated via: $$\hat N=Y- \hat A \times X$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and further $\hat {\mu_N}$ and $\hat {\Sigma_N}$ can be computed using $\hat N$.&lt;/p&gt;&#xA;" OwnerUserId="18994" LastEditorUserId="-1" LastEditDate="2017-04-13T12:48:31.170" LastActivityDate="2016-01-29T12:00:09.707" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="21937" PostTypeId="2" ParentId="16107" CreationDate="2016-01-28T11:28:17.447" Score="0" Body="&lt;p&gt;No. You have not changed the boundary conditions, the boundary conditions are still Robin type. what you changed is the DEs to both parabolic and used the same boundary conditions. It is in fact surprising to see this. Reasoning is not correct though.&lt;/p&gt;&#xA;" OwnerUserId="19071" LastEditorUserId="1804" LastEditDate="2016-01-28T11:42:25.987" LastActivityDate="2016-01-28T11:42:25.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21938" PostTypeId="2" ParentId="2868" CreationDate="2016-01-28T12:48:49.277" Score="3" Body="&lt;p&gt;Communications in Computational Physics, published with Cambridge University Press&#xA;&lt;a href=&quot;http://www.global-sci.com/&quot; rel=&quot;nofollow&quot;&gt;http://www.global-sci.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Journal of Computational Mathematics: free for both reader and author, published by SMAI &#xA;&lt;a href=&quot;http://www.euro-math-soc.eu/news/14/12/5/smai-founds-new-journal-computational-mathematics&quot; rel=&quot;nofollow&quot;&gt;http://www.euro-math-soc.eu/news/14/12/5/smai-founds-new-journal-computational-mathematics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Journal of Scientific Computing, while not Elsevier, is with Springer, still a commercial publisher.&lt;/p&gt;&#xA;" OwnerUserId="19072" LastActivityDate="2016-01-28T12:48:49.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21939" PostTypeId="2" ParentId="21933" CreationDate="2016-01-28T13:18:28.033" Score="4" Body="&lt;p&gt;When solving a boundary value problem, the shooting method uses&#xA;nonlinear rootfinding to find the initial conditions that would&#xA;satisfy the boundary conditions.&#xA;To see what's going on here, replace your BVP with the IVP corresponding to the first guess that you told the shooting method to start with. That is, replace&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F[L] == 0, G[L] == 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F'[0] == 0.55, G'[0] == -0.65&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and plot the resulting solutions of the IVP. Unlike when solving the BVP, Mathematica will actually give you the solution up to time 11.977 when the singularity occurs. You can clearly see in the plot how the functions blow up — so it is definitely not stiffness causing this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UnZXm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UnZXm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes it clear that your problem is that certain (many?) initial values result in singularities. So the shooting method is not suitable, unless by some coincidence the initial values that it tries just happen to not lead to singularities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I tried setting the starting initial conditions to &lt;code&gt;F'[0] == 0.5, G'[0] == -0.7&lt;/code&gt; (a bit of guesswork and trial and error), it seemed to work correctly, giving a solution that satisfies the BCs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8ZuF9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8ZuF9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-01-28T13:42:44.900" LastActivityDate="2016-01-28T13:42:44.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21940" PostTypeId="2" ParentId="11727" CreationDate="2016-01-28T13:32:36.070" Score="4" Body="&lt;p&gt;There is a paper by Moré, implemented by Nocedal, about that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Jorge J. Moré and David J. Thuente. 1994. Line search algorithms with&#xA;  guaranteed sufficient decrease. ACM Trans. Math. Softw. 20, 3&#xA;  (September 1994), 286-307. DOI &lt;a href=&quot;http://dx.doi.org/10.1145/192115.192132&quot;&gt;http://dx.doi.org/10.1145/192115.192132&lt;/a&gt;&#xA;  (&lt;a href=&quot;http://www.ii.uib.no/~lennart/drgrad/More1994.pdf&quot;&gt;preprint&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="19073" LastEditorUserId="1804" LastEditDate="2016-01-28T13:41:44.860" LastActivityDate="2016-01-28T13:41:44.860" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21941" PostTypeId="2" ParentId="21930" CreationDate="2016-01-28T14:32:47.570" Score="1" Body="&lt;p&gt;As Federico Poloni pointed out, your request is essentially to schedule a &lt;a href=&quot;https://en.wikipedia.org/wiki/Round-robin_tournament&quot; rel=&quot;nofollow&quot;&gt;round-robin tournament&lt;/a&gt;.  If the number of participants is $N$, then there are $N(N-1)/2$ pairs who must meet.  Thus the &quot;as fast as possible&quot; condition requires:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If $N$ is even, $N-1$ rounds are necessary, with $N/2$ pairs meeting in each.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If $N$ is odd, $N$ rounds are necessary, with $(N-1)/2$ pairs meeting in each.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An explicit formula can be given for the round in which each pair of participants (numbered $1$ to $N$) is to meet, but we will naturally treat the cases $N$ even and odd separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;N is even&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider an $N\times N$ table which tells the round in which a pair $(r,c)$ designated by the row $r$ and column $c$ is supposed to meet.  Since we are concerned with scheduling of &lt;em&gt;unordered&lt;/em&gt; pairs, this table should be symmetric in the sense that the entry $(r,c)$ agrees with the entry $(c,r)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we do not need any participant to &quot;meet&quot; themselves, the diagonal entries $(r,r)$ will not designate any actual round, but otherwise the table entries must be distinct in each row and distinct in each column (since a participant can only meet with one other person in any given round).  In this respect we can make the table a symmetric &lt;a href=&quot;https://en.wikipedia.org/wiki/Latin_square&quot; rel=&quot;nofollow&quot;&gt;latin square&lt;/a&gt; by placing the value $N$ along the diagonal and filling the off-diagonal entries with rounds $1$ through $N-1$ as scheduled for $r\neq c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an algorithm to fill the $(r,c)$ entry with a value $f(r,c)$ between $1$ and $N$:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If $r=c$, then $f(r,c) = N$. (&lt;em&gt;diagonal entries&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;If $r\gt c$, then $f(r,c) = f(c,r)$. (&lt;em&gt;entries below diagonal, by symmetry&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;If $c = N$, then $f(r,N) = r$. (&lt;em&gt;entries in last column&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Otherwise $r \lt c \lt N$. Then:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If $r+c$ is even, $f(r,c) = \frac{r+c}{2}$.&lt;/li&gt;&#xA;&lt;li&gt;Otherwise $r+c$ is odd, and we take:&#xA;$$ f(r,c) = 1+ \left[ \left(\frac{r+c+N-1}{2} - 1 \right) \bmod (N-1)\right] $$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The final formula's complexity is occasioned by our need to coerce a value $f(r,c)$ between $1$ and $N-1$ for these off-diagonal entries.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This algorithm implements the method of constructing a schedule reported by &lt;a href=&quot;https://en.wikipedia.org/wiki/%C3%89douard_Lucas&quot; rel=&quot;nofollow&quot;&gt;Édouard Lucas&lt;/a&gt; of primality testing fame &lt;a href=&quot;https://archive.org/stream/p2rcrationsm00lucauoft#page/176/mode/2up&quot; rel=&quot;nofollow&quot;&gt;based on a &quot;clock face&quot;&lt;/a&gt; with positions $1$ to $N-1$ around the circumference and number $N$ at the center, a method &quot;simple and ingenious&quot; that he attributed to Felix Walecki:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Lucas, Edouard (1883). &quot;Les jeux de demoiselles&quot;. Récréations Mathématiques (in French). Paris: Gauthier-Villars. pp. 161–197.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;N is odd&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again we consider an $N\times N$ table whose entries $(r,c)$ tell the round in which participants $r$ and $c$ should meet.  Again the table will be a symmetric latin square, but the significance of the diagonal entries is different.  As only $N-1$ (an even number) of participants can meet in a round, the diagonal entry for $(r,r)$ will here tell us in which round participant $r$ is to be the &quot;bystander&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fill the entry $(r,c)$ with a value between $1$ and $N$ according to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(r,c) = 1 + [ (r+c-1) \bmod N ] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again the formula is a bit fancy so that we get a number between $1$ and $N$, and could here be simplified if the rounds were to be &quot;zero indexed&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed the zero-indexing of participants as well as rounds would allow us to identify this symmetric latin square with the addition table modulo $N$.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2016-01-28T22:58:33.763" LastActivityDate="2016-01-28T22:58:33.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21942" PostTypeId="2" ParentId="21914" CreationDate="2016-01-28T17:21:41.073" Score="2" Body="&lt;p&gt;Discretizations of partial differential equations &quot;enjoy&quot; a property called &lt;em&gt;error pollution&lt;/em&gt; that means that if you don't exactly satisfy the equation at one location (such as the points you identified on the first mesh) as evidenced by the fact that your residual is nonzero there, then that will produce an &lt;em&gt;error&lt;/em&gt; elsewhere as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence, it is not enough to just solve again on the patch of smaller cells in the vicinity of the point you identified where the error is large. This is so because you may make the error smaller on the patch that way, but you're not going to make the error smaller in the rest of the domain unless you solve a &lt;em&gt;global&lt;/em&gt; problem, i.e., including the rest of the mesh. Consequently, the approach is not to interpolate from the black X's to the red X's and then to solve on the green X's, but to solve the whole problem on the entire mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your question 3: You can refine multiple times. In your case, I'd refine around both of the points where you know that the error is large, and only then solve again.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-28T17:21:41.073" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21943" PostTypeId="2" ParentId="21912" CreationDate="2016-01-28T17:26:50.677" Score="1" Body="&lt;p&gt;What you describe sounds like a way where your whole data set is stored in one directory with a set of subdirectories and files within them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think about this differently, then you will come to realize that that's exactly how an XML file is structured: there are hierarchies of sections (subdirectories) and some tags have values (files). In some cases, the values are compressed (e.g., uuencoded, bzip-compressed data) which corresponds to compressing individual files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, what you suggest already exists in hierarchically structured file formats such as XML. The difference simply is that you don't have hundreds or thousands of files sitting around requiring their own metadata, but it's all stored in one file. That's a bit like zipping up the whole directory and compressing it. In essence, that's also what .docx or the OpenOffice file formats do: they're just compressed, tar'ed (or zipped) directory structures.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-28T17:26:50.677" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21944" PostTypeId="2" ParentId="21916" CreationDate="2016-01-28T17:31:42.310" Score="5" Body="&lt;p&gt;The equation $\frac{\partial}{\partial z}(\rho u)=0$ only has a unique solution for the product $\rho u$ (namely a constant), but not for each factor separately. You need a different equation to tease them apart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your context, the equation describes the conservation of momentum. You will need a separate equation for the conservation of mass, which is going to have a term $\frac{\partial}{\partial z}\rho$. Together, these two equations are going to uniquely define both velocity and density.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-28T17:31:42.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21945" PostTypeId="2" ParentId="396" CreationDate="2016-01-28T18:32:04.617" Score="1" Body="&lt;p&gt;From what I know all of the research has focused on weak convergence via extrapolation. &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3540570748&quot; rel=&quot;nofollow&quot;&gt;The book by Kloden, Platen, and Schurz&lt;/a&gt; has a good discussion on them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few problems one has to solve to do &quot;good&quot; Richardson extrapolation with SDEs. For one, the reason why it was interesting was because the error estimates ended up being in the square powers $h^{2n}$ and that won't necessarily be the case with SDEs, especially since it would have to be based on a strong order Runge-Kutta type method and there's no obvious &quot;higher-order choice&quot; like the midpoint method was for ODEs (there is for weak order, which is why this was investigated). This means that its strongest point seems to be diminished. Then you have to come up with a good scheme from interpolating Brownian motion since during the calculation you do a lot of &quot;backfilling&quot; at lower $h$'s, and so you'd have to do that properly to not change the sampling distribution. Lastly, extrapolation methods seemed to work really well for smooth and stable problems, but tend to have problems with more unstable or discontinuous problems. The regularity of SDEs is always pretty weak, and problems seem to stem more from instability (since the noise term also changes the stability conditions A LOT!) than error (which can usually be made lower than the standard deviation of the noise anyways).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those reasons, no one has taken a good look at extrapolation methods for strong approximation. That doesn't mean no one should (I might in the near future), but suggests that getting comfortable with the Runge-Kutta type methods might have a higher payoff. As always in numerical computing, YMMV.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-01-28T18:32:04.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21946" PostTypeId="2" ParentId="19807" CreationDate="2016-01-28T19:02:34.483" Score="7" Body="&lt;p&gt;Pure Neumann problem is unique up to a constant. My two favourite solution strategies:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Modifying the equation $-\Delta u = f$ to $-\Delta u + \varepsilon u = f$ for some small $\varepsilon&amp;gt;0$. If you perform reduced integration this corresponds roughly to adding $\varepsilon$ to the diagonal of your system matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Imposing $\int_\Omega u \,\mathrm{d}x=0$ using Lagrange multiplier. Then your system would be&#xA;$$\begin{bmatrix} A &amp;amp; 1 \\ 1^T &amp;amp; 0 \end{bmatrix} \begin{bmatrix} x \\ \lambda \end{bmatrix} = \begin{bmatrix} f \\ 0 \end{bmatrix}.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10160" LastEditorUserId="10160" LastEditDate="2016-01-28T20:13:17.543" LastActivityDate="2016-01-28T20:13:17.543" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21947" PostTypeId="1" CreationDate="2016-01-28T21:16:27.683" Score="1" ViewCount="2251" Body="&lt;p&gt;So in ${\bf R}^{n\times p}$ we have the Frobenius inner product given by&#xA;$$\langle A, B\rangle=\text{tr}(A^TB)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be interpreted as the Euclidean inner product on ${\bf R}^{np}$.  My understanding is that all inner products on ${\bf R}^{np}$ can be written as&#xA;$$a^TPb$$&#xA;for $P$ positive-definite.&#xA;The best I could do in attempting to extend the Frobenius inner product on ${\bf R}^{n\times p}$ is something of the form&#xA;$$\langle A, B\rangle=\sum_{i=1}^N\text{tr}((X_iAY_i)^T(X_iBY_i))$$&#xA;for $X_i\in{\bf R}^{m_i\times n}$ and $Y_i\in{\bf R}^{p\times q_i}$ all full rank.  However I would like to know if this covers all inner products on ${\bf R}^{np}$, or if maybe it's more complex than necessary due to redundancies.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can find the corresponding $P$ matrix for any specific matrix inner product by taking the standard basis for ${\bf R}^{n\times p}$ and forming the matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{bmatrix}&#xA;    \langle E_1,E_1\rangle &amp;amp; \langle E_1,E_2\rangle &amp;amp; \dots &amp;amp; \langle E_1,E_{np}\rangle \\&#xA;    \langle E_2,E_1\rangle &amp;amp; \langle E_2,E_2\rangle &amp;amp; &amp;amp; \vdots \\&#xA;    \vdots &amp;amp; &amp;amp; \ddots \\&#xA;    \langle E_{np},E_1\rangle       &amp;amp; \dots &amp;amp; \dots &amp;amp; \langle E_{np},E_{np}\rangle&#xA;\end{bmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I don't know if the general form for a matrix inner product I gave above covers all positive-definite matrices $P$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;newer version of this question on MathOverflow: &lt;a href=&quot;https://mathoverflow.net/questions/229675/extending-the-trace-inner-product-to-all-matrix-real-inner-products&quot;&gt;https://mathoverflow.net/questions/229675/extending-the-trace-inner-product-to-all-matrix-real-inner-products&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2016-01-30T01:47:45.880" Title="Extending the Frobenius inner product to all matrix inner products" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21948" PostTypeId="2" ParentId="21568" CreationDate="2016-01-28T22:10:44.083" Score="2" Body="&lt;p&gt;As I mentioned before, you can try with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Visible_Human_Project&quot; rel=&quot;nofollow noreferrer&quot;&gt;Visible Human Project&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also go with &lt;a href=&quot;http://www.slicer.org/publications/gallery&quot; rel=&quot;nofollow noreferrer&quot;&gt;3D Slicer sample data&lt;/a&gt;, e.g., the &lt;a href=&quot;http://www.slicer.org/publications/item/view/1953&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knee Atlas&lt;/a&gt; or the &lt;a href=&quot;http://www.slicer.org/publications/item/view/2037&quot; rel=&quot;nofollow noreferrer&quot;&gt;Brain Atlas&lt;/a&gt;. Their terms are quite permissive:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By downloading these data, you agree to acknowledge our contribution in any of your publications that result form the use of this atlas.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is also &lt;a href=&quot;https://openfmri.org/dataset/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenfMRi&lt;/a&gt;, but you already had that answer in &lt;a href=&quot;https://opendata.stackexchange.com/questions/6661/open-source-mri-image-dataset&quot;&gt;opendata.SE&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:30.123" LastActivityDate="2016-01-28T22:10:44.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21949" PostTypeId="1" AcceptedAnswerId="21955" CreationDate="2016-01-28T22:33:22.383" Score="9" ViewCount="1199" Body="&lt;p&gt;during the development of &lt;a href=&quot;http://magwo.github.io/fullofstars/&quot;&gt;my N-body simulation&lt;/a&gt; with visualisation in WebGL, I devised an optimisation, and I'm wondering if it has a name. I find it unlikely that it has never been done before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works like this: During the first timestep, make an all-pairs iteration. During that iteration, for each particle:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Store all the &lt;strong&gt;close interactions&lt;/strong&gt; in a list - representing all the particles that are close to this one. These interactions will from then on be evaluated each time step. This list would typically contain a handful of entries.&lt;/li&gt;&#xA;&lt;li&gt;Iterate on all the other particles and calculate a &lt;strong&gt;net far force&lt;/strong&gt; that you store with the particle. This net force is thus remembered between timesteps and continuously applied to the particle.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then as the simulation continues past its first timestep, in a round-robin fashion, each timestep update a small number of particles' lists of &lt;strong&gt;close interactions&lt;/strong&gt; and &lt;strong&gt;net far forces&lt;/strong&gt;. So that over a certain number of time steps - say, 1000, all the particles' close interactions and net far forces will have been updated. The ones that you don't update will just check their close interactions and apply the net far force. In this example, the computational complexity of each time step is something like $N^2 / 1000$ instead of $N^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A trick to also making this reasonably accurate is to better identify &quot;close interactions&quot;. Sometimes proximity is not the best indicator - you could also consider mass and relative velocity and so on. &quot;Most significant interactions&quot; might be a better word. Or &quot;most-likely-to change-soon-interactions&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This optimisation allows for &lt;em&gt;a lot&lt;/em&gt; more interacting particles than the all-pairs method, but I'm not sure how to describe it in O() terms, as it does not make a complete solution each timestep, but reuses (slightly incorrect) old information and spreads out the computational effort over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;em&gt;Disclaimer: My webgl simulation also has &quot;vfx&quot; particles that only get &lt;strong&gt;affected&lt;/strong&gt; by gravity and don't reciprocate the effect, so it's not as awesomely fast as it might appear&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So does this optimisation technique have a name?&lt;/p&gt;&#xA;" OwnerUserId="19081" LastEditorUserId="9667" LastEditDate="2016-01-28T22:52:50.027" LastActivityDate="2016-01-29T13:14:32.153" Title="N-body simulation optimisation, looking for name or existing work" Tags="|optimization|algorithms|complexity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21951" PostTypeId="1" CreationDate="2016-01-29T06:34:52.390" Score="2" ViewCount="1009" Body="&lt;p&gt;I have a set of 3d points to which I want to fit two planes. I know the assignment of points to the planes so I don't need any RANSAC or similar. &#xA;Currently, I'm using a PCA-based approach to fit two planes and they are rather orthogonal (+/- 2 deg) but I'd like to integrate this knowledge directly into the plane fitting so that the planes are orthogonal by design. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has someone got an idea for a closed form solution or will I have to do some numerical optimization (should probably work nicely as I already have very good initial values for both planes)?&lt;/p&gt;&#xA;" OwnerUserId="19085" LastEditorUserId="6910" LastEditDate="2016-02-01T23:59:33.333" LastActivityDate="2016-09-22T15:23:16.767" Title="Fitting orthogonal planes to a point set" Tags="|optimization|computational-geometry|geometry|computer-vision|3d|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21952" PostTypeId="1" CreationDate="2016-01-29T08:28:57.660" Score="2" ViewCount="186" Body="&lt;p&gt;I have an expression like the following (anyway to input latex?),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ z^{m}\left( \ln\left(\frac{z}{z-1}\right)-\sum_{k=1}^m \frac{z^{-k}}{k} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for m integer, z complex&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems if I code it the following way, it gives noticeable round-off error in case $|z| \approx 1$. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Z_1=1/Z&#xA;&#xA;    ZV1=LOG(Z/(Z-1))&#xA;    ZK=1.0_q&#xA;    DO K=1,M;          ZK=ZK*Z_1&#xA;      ZV1=ZV1-ZK/K&#xA;    ENDDO&#xA;    ZV1=ZV1/ZK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on a better way to code this expression? Thanks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added note:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have compared with gfortran built-in function Log and Mathematica NIntegrate the following two equivalent expressions,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \ln\left(\frac{z}{z-1}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{0}^{1}\frac{1}{z-x} dx $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $z$ to be complex, and found that these two can differ as big as $10^{-15}$ at small $|z|$. (I pushed NIntegrate to a PrecisionGoal of $10^{-17}$). This difference seems to be responsible for the error of the expression I was seeking help (I can upload a figure to support my claim made here if possible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think this is likely the case? Which result should I trust, from the built-in function or from the numerically converged results? If use Taylor series to help improve numerical convergence, how to treat the case with $|z|\sim 1$, which does not validate any series expansion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notes added:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides the possibility that Mathematica didn't do a good enough job with Log function, it could equally be that NIntegrate does not reach the prescribed precision goal for some input z values. I am trying to clarify this issue with Mathematica and see how they respond.&lt;/p&gt;&#xA;" OwnerUserId="19086" LastEditorUserId="19086" LastEditDate="2016-02-01T17:19:52.700" LastActivityDate="2016-07-20T16:18:45.410" Title="what is the best way to code a formula to reduce roundoff error" Tags="|numerical-analysis|fortran|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21953" PostTypeId="1" CreationDate="2016-01-29T09:44:53.643" Score="2" ViewCount="165" Body="&lt;p&gt;I have a system of ordinary differential equations (ODEs). It is a large system that has dozens of equations and hundreds of parameters. I wish to reduce its size so it becomes computationally more efficient to simulate, and I'm willing to tolerate some error in the reduced model. So the end result should be a system of ODEs that has a smaller number of equations or parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can supposedly use methods of subspace projection to obtain a reduced model. For example Principal component analysis / Proper orthogonal decomposition (POD) with the method of snapshots. With this method a &lt;strong&gt;basis&lt;/strong&gt; is obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads to the following POD algorithm:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Input: the data in the matrix $W$ consisting of the snapshots obtianed by simulating the full systems.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the singular value decomposition so that:&#xA;$$W = UVD' .$$&lt;/li&gt;&#xA;&lt;li&gt;Output: The first $l$ columns of $U$ form the reduced subspace and the basis is then $\Phi = [u_1, ... , u_l]$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If the coefficients of my linear ODE system are in $A$, I can use $\Phi' A \Phi$ to project the system to a reduced subspace and simulate it there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having obtained the simulation results $Y$ in the reduced space, I can do $\Phi Y$ to project the results back to the original space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if $A$ models a biological system with meaningful equations, the same cannot be said for $\Phi' A \Phi$. But is there any way $\Phi$ can be used to remove equations or variables from $A$ to obtain a simpler 'real world' system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I'm not very capable of reading math, so if possible, please explain like I am five.&lt;/p&gt;&#xA;" OwnerUserId="19087" LastEditorUserId="19087" LastEditDate="2016-01-30T15:31:14.797" LastActivityDate="2016-01-30T15:31:14.797" Title="How to obtain the reduced model from a subspace projection method?" Tags="|ode|numerical-modelling|projection|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="21954" PostTypeId="1" AcceptedAnswerId="21959" CreationDate="2016-01-29T12:56:21.787" Score="2" ViewCount="297" Body="&lt;p&gt;There are some technique to generate mesh in a domain. My qustion is that:&#xA;Is there any difference between the results using different techniques for mesh generation?&#xA;If yes which one is better. For example If I solve an equation by regular triangle mesh and by criss-cross mesh wich one must be better (especially in adaptive fem)? &#xA;I think maybe criss-cross mesh is better because of the number of elements. But using more elements leads much computaitional cost!!&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2016-01-29T13:30:11.143" LastActivityDate="2016-02-09T20:13:27.843" Title="Finite element results by different meshes" Tags="|finite-element|adaptive-mesh-refinement|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21955" PostTypeId="2" ParentId="21949" CreationDate="2016-01-29T13:05:27.323" Score="6" Body="&lt;p&gt;The method you describe is somewhat similar to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Barnes%E2%80%93Hut_simulation&quot;&gt;Barnes-Hut algorithm&lt;/a&gt;. The main difference is that you have a single level of  close interactions, whereas the Barnes-Hut has $\log N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Barnes-Hut, all particles are put into an &lt;a href=&quot;https://en.wikipedia.org/wiki/Octree&quot;&gt;octree&lt;/a&gt;, and the gravitational force computed between the elements every node. Computational complexity then drops to $O(N \log N)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turns out what you're proposing does have a name: the &lt;a href=&quot;http://adsabs.harvard.edu/full/1992PASJ...44..141M&quot;&gt;Ahmad-Cohen scheme&lt;/a&gt;. The gain in computational speed is $(N/3.8)^{\frac{1}{4}}$.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="7522" LastEditDate="2016-01-29T13:14:32.153" LastActivityDate="2016-01-29T13:14:32.153" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21957" PostTypeId="2" ParentId="21752" CreationDate="2016-01-29T15:56:32.007" Score="1" Body="&lt;p&gt;An alternative approach that will help with the situation at $r=0$ is to do the interpolation in $(x,y)$ coordinates first, and then evaluate the integral.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-01-29T15:56:32.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21958" PostTypeId="2" ParentId="21952" CreationDate="2016-01-29T22:36:21.883" Score="3" Body="&lt;p&gt;If I properly converted the expression to be calculated into math notation, then what we are dealing with is subtracting off the $m$ leading terms (ignoring a constant of zero) in the power series expansion of $\ln(1 - z^{-1})$ (corresponding to an expansion of $\ln(1+z)$).  This would cause subtractive cancellation and thus amplification of rounding errors relative to the magnitude of the difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \ln(1 - z^{-1}) = \sum_{k=1}^\infty \frac{z^{-k}}{k} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between this power series and its truncation at the $k=m$ index is then scaled by $z^m$.  But multiplying by $z^m$ for $|z| \approx 1$ should not have substantial impact on the relative error of the final result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion is to implement a summation of the tail of the power series, the portion left after the leading terms are subtracted off:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum_{k=m+1}^\infty \frac{z^{m-k}}{k} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will turn my hand to implementing this for complex $z$ in a neighborhood of $1$, but note that the Question asks for something different: a computation valid for $z$ in a neighborhood of the unit circle!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here lies a serious difficulty, in that the natural logarithm is a multi-valued function in the complex plane, and any valid branch cut will intersect the unit circle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principal branch of the natural logarithm is formed to agree with the real natural logarithm on the positive real axis.  The branch cut is then chosen (as a matter of convention) to be the origin and the negative real axis (in the complex plane).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore some clarification is needed of where in the neighborhood of the unit circle (annulus) the computation is expected to be valid (which branch of the natural logarithm are we going to compare to for accuracy).&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2016-01-29T22:44:00.180" LastActivityDate="2016-01-29T22:44:00.180" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="21959" PostTypeId="2" ParentId="21954" CreationDate="2016-01-30T00:22:32.660" Score="7" Body="&lt;p&gt;As a general rule, finite element solutions are more accurate on meshes with cells that (i) deviate less from the optimal shape (which for triangles are equilateral triangles and for rectangles are squares), and (ii) have local symmetries. Symmetries in a quadrilateral mesh would, for example, mean that four cells come together at every vertex where the four incoming edges form two straight lines. For triangles, this would mean that every vertex has exactly 6 adjacent cells, rather than alternating between 4 and 8 for a criss-cross mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason this affects the solution is that (i) the deviation from the optimal shape shows up in the norm of the transformation that appears in the interpolation estimate, and equilateral triangles and squares happen to have the smallest norm; (ii) symmetries allow for Taylor expansions of the error at vertices where certain terms magically cancel, and the solution is of higher order at individual points of the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, as has already been pointed out by @TylerOlsen in the comments, whatever choice of mesh you have, the solution will converge asymptotically as the maximal mesh size $h\rightarrow 0$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-30T00:22:32.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21960" PostTypeId="2" ParentId="21947" CreationDate="2016-01-30T00:37:05.590" Score="2" Body="&lt;p&gt;You can see an inner product as an operation $f(a,b)=\left&amp;lt;a,b\right&amp;gt;$, i.e., it is a bilinear function that (i) returns a non-negative number, (ii) satisfies the relationship $f(a,b)=f(b,a)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For vectors $a,b\in\mathbb R^n$, all bilinear functions that satisfy these properties can be written as&#xA;$$&#xA;  f(a,b) = \sum_{i,j=1}^n a_i P_{ij} b_j&#xA;$$&#xA;where $P$ is symmetric and positive definite. For matrices $a,b\in\mathbb R^{n\times p}$, all such functions can be written as&#xA;$$&#xA;  f(a,b) = \sum_{i,k=1}^n \sum_{j,l=1}^p a_{ij} P_{ijkl} b_{kl}&#xA;$$&#xA;where now $P$ is a tensor of rank 4 that is symmetric in the sense that $P_{ijkl}=P_{klij}$ and positive definite in the sense that $f(a,a)&amp;gt; 0$ for all $a\neq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question boils down to whether every $P$ that satisfies such conditions can be written a form that results from the vectors $X_i,Y_i$. I believe the answer to this is no. This is simply so because the (for simplicity assuming $n=p$) symmetric $P$ has (asymptotically) $n^4/2$ degrees of freedom, whereas the $n$ vectors $X_i,Y_i$ only have $2n^2$ degrees of freedom. In other words, I don't think that for sufficiently large $n$, your approach has sufficiently many degrees of freedom.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-30T00:37:05.590" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21961" PostTypeId="2" ParentId="20070" CreationDate="2016-01-31T08:28:00.210" Score="2" Body="&lt;p&gt;If you are using a Linux based OS, in a machine with Intel CPU and if you have access to Intel MKL (you can obtain a non-commercial copy for free), then the best you can have is PARDISO MKL, which is the Intel version of the  &lt;a href=&quot;http://www.pardiso-project.org&quot; rel=&quot;nofollow&quot;&gt;Pardiso&lt;/a&gt; solver by Olaf Schenk. There is a &lt;a href=&quot;https://github.com/JuliaSparse/Pardiso.jl&quot; rel=&quot;nofollow&quot;&gt;Pardiso wrapper&lt;/a&gt; in Julia, and the performance is in average faster than UMFPACK and MUMPS. Furthermore, you can handle multiple right-hand-sides using BLAS 3 operations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The syntax, however, is a bit more complicated than just a backslash, but you have enough examples to have a good idea how it works and you can take a look at the online documentation to see all the possible options. With the backslash, Julia will check the type of the matrix and it will use the best options, with PARDISO you need to specify the type of matrix yourself.&lt;/p&gt;&#xA;" OwnerUserId="19098" LastActivityDate="2016-01-31T08:28:00.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21962" PostTypeId="2" ParentId="8198" CreationDate="2016-01-31T09:01:53.083" Score="2" Body="&lt;p&gt;I would write this as a comment to the answer by @Mohammad Bazrafshan but since i dont have enough reputation i will write it as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What he suggested works, the official documentation for it can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/integral.html#inputs&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/matlab/ref/integral.html#inputs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that the integral function has the potential to integrate array-valued functions, like vectors and matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it does is component-wise integration of the elements of the array, so it is the same as going through all elements of the array with a loop and performing integration of many scalar-valued functions.&lt;/p&gt;&#xA;" OwnerUserId="19099" LastActivityDate="2016-01-31T09:01:53.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21963" PostTypeId="1" CreationDate="2016-01-31T14:00:02.437" Score="1" ViewCount="746" Body="&lt;p&gt;I would like to construct a surface on a 2D mesh, given a set of Fourier coefficients for the original function $V(x,y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given some $\mathbb{K}\subset \mathbb{Z}^2$ I have a partial double sum of a Fourier series for $V(x,y)$, which could be some potential, for example &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V(x,y)=\frac{1}{\sqrt{x^2+y^2}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x,y)=\sum_{(j,k)\in \mathbb{K}} v_{j,k} \exp(2 \pi i \cdot (jx+ky))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am going to use Matlab for this to begin with so my question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I generate a finite set of Fourier coefficients given $V$ in Matlab?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once I have these coefficients what would the best method to plot them in 3D/contour plot?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7311" LastActivityDate="2016-01-31T14:00:02.437" Title="3D plot from Fourier coefficients in Matlab" Tags="|matlab|fourier-transform|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21966" PostTypeId="2" ParentId="21951" CreationDate="2016-01-31T14:28:40.773" Score="2" Body="&lt;p&gt;Here I devise a novel strategy, based on only 3D points, that I think, would work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will parametrize a 3D plane by a point $\mathbf{p}$ and its normal $\mathbf{n}$. &#xA;Imaging you take a pair of oriented points $\mathbf{p}_1$ and $\mathbf{p}_2$, on the point cloud $\mathbf{P}$ with corresponding normals $\mathbf{n}_1$ and $\mathbf{n}_2$. Let $\mathbf{d}$ denote the vector joining two points, i.e. $\mathbf{d}=\mathbf{p}_1-\mathbf{p}_2$ Ideally, two points would be located on orthogonal planes iff :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;\lvert \angle(\mathbf{n}_1, \mathbf{n}_2) \rvert = \pi \mathbin{/} 2 \\&#xA;\lvert \angle(\mathbf{n}_1, \mathbf{d}) \rvert \leq \pi \mathbin{/} 2 \\&#xA;\lvert \angle(\mathbf{n}_2, \mathbf{d}) \rvert \leq \pi \mathbin{/} 2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More over if one wants to penalize the size constraint (for example when the extent of the planes are known), he/she could as well write:&#xA;$$&#xA;\lVert \mathbf{d} \rVert &amp;lt; \tau_{d}&#xA;$$&#xA;where $\tau_{d}$ is a threshold on the 3D extent of the planes. As I already mentioned, these are ideal cases. In real life, e.g. for the data coming from RGBD sensors, this is never the case. So it is better to introduce a noise threshold, where certain tolerance is maintained for the sake of robustness. Then :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;\pi \mathbin{/} 2-\epsilon_n &amp;lt; \angle(\mathbf{n}_1, \mathbf{n}_2) &amp;lt; \pi \mathbin{/} 2+\epsilon_n \\&#xA;\lvert \angle(\mathbf{n}_1, \mathbf{d}) \rvert &amp;lt; \pi \mathbin{/} 2 + \epsilon_d\\&#xA;\lvert \angle(\mathbf{n}_2, \mathbf{d}) \rvert &amp;lt; \pi \mathbin{/} 2 + \epsilon_d&#xA;$$&#xA;where $\epsilon_d$ and $\epsilon_n$ trades-off noise tolerance to accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, to guarantee the angles to be lying in range $[-\pi, \pi]$, and to be robust to small angles, the formula from &lt;a href=&quot;https://www.researchgate.net/publication/283730711_Point_Pair_Features_Based_Object_Detection_and_Pose_Estimation_Revisited&quot; rel=&quot;nofollow&quot;&gt;Birdal and Ilic 2015 [1]&lt;/a&gt; can be used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\angle(\mathbf{v_1},\mathbf{v_2}) = \tan^{-1}\big( \frac{ \Vert \mathbf{v_1} \times \mathbf{v_2} \rVert}{\mathbf{v_1} \cdot \mathbf{v_2}}\big)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this points, one could always write a RANSAC loop, where random point pairs are tested for the satisfaction of these constraints. However, a better idea (which would make this more efficient and possibly robust) is to employ a somewhat generalized Hough voting procedure, again similar to &lt;a href=&quot;https://www.researchgate.net/publication/283730711_Point_Pair_Features_Based_Object_Detection_and_Pose_Estimation_Revisited&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt; or &lt;a href=&quot;http://far.in.tum.de/pub/drost2010CVPR/drost2010CVPR.pdf&quot; rel=&quot;nofollow&quot;&gt;[2]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We sample the scene for $N$ reference points. We fix each such reference point and pair it with all other $N-1$ points, found on the sampled scene. We compute the pair features, which are in this case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F(\mathbf{p}_1, \mathbf{p}_2) = \big(\angle(\mathbf{n}_1, \mathbf{n}_2), \angle(\mathbf{n}_1, \mathbf{d}), \angle(\mathbf{n}_2, \mathbf{d}), \lVert \mathbf{d} \rVert \big)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For orthogonal planes, given the point pair, the definition is immediate (the normals of two points uniquely define the orthogonal planes). However, finding best such candidate requires care if the scene is cluttered. Thus, we evaluate the aforementioned constraints and create a voting table, similar to &lt;a href=&quot;http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&amp;amp;arnumber=7335508&quot; rel=&quot;nofollow&quot;&gt;[3]&lt;/a&gt;. For a simple plane, such voting table is dimension-less (we only accumulate the number of votes for each - see &lt;a href=&quot;http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&amp;amp;arnumber=7335508&quot; rel=&quot;nofollow&quot;&gt;[3]&lt;/a&gt;). However, for the orthogonal case, some care must be taken. This, I describe as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once an oriented pair of points is found to be on orthogonal planes, the reference point ($\mathbf{p}_1$) defines the first plane. Then the orthogonal counterpart, can freely rotate around the normal of the reference and is also free to slide orthogonally on this infinite reference plane. This creates two degrees of freedom, which can be represented in 2D polar space: ($\theta$, $\rho$). $\theta$ denotes rotation around normal, and $\rho$ is the point to plane distance on the 2D reference plane to the reference point. Note that once the intersection line of the two planes $\mathbf{l}=\{a,b,c,d\}$ is computed, this reduces to point to line distance. The vote ($\theta$, $\rho$) can be casted by transforming the point pair to the origin, and aligning $\mathbf{n}_1$ with $y$ axis. So,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\theta=\angle(\mathbf{l}, \mathbf{x}) \\&#xA;\rho=\frac{\lvert c \rvert}{\mathbf{\sqrt{a^2+b^2}}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{x}$ is the x-axis. (Notice the resemblance to the Hough transform?) The voting is performed locally for each reference point, resulting in $\mathbf{\theta}=\{\theta_1,...,\theta_N\}$ and $\mathbf{\rho}=\{\rho_1,...,\rho_N\}$. Note that this is also a local coordinate system and these entities vary depending on the selected pair point. However, for each reference point, if the pair lies on the orthogonal planes, ideally they should vote for the same local reference frame. The voting also requires quantization of the local reference frame, which can be chosen reasonably depending on the problem size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given all the solutions of the orthogonality for every reference point, the task is then to cluster them. &lt;a href=&quot;https://www.researchgate.net/publication/283730711_Point_Pair_Features_Based_Object_Detection_and_Pose_Estimation_Revisited&quot; rel=&quot;nofollow&quot;&gt;Birdal and Ilic [1]&lt;/a&gt; employ an agglomerative clustering, while other choices are also possible. Note that, for the clustering stage the standard parameterization $[a,b,c,d]$ is a better choice, as it reduces the dimension. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to quantization, noise and artifacts, the plane fitting to this end is a rough one. It is acceptably good but a refinement might be desirable. For that purpose, I would use a non-linear Levenberg Marquardt adjustment procedure, in which the distance from the point cloud to the orthogonal planes are jointly minimized. This can easily be done by minimizing the objective:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E(\mathbf{P}, \{\mathbf{W}_1,\mathbf{W}_2\}) = \sum_{i=1}^M min( Q(\mathbf{p}_i, \mathbf{W}_1), Q(\mathbf{p}_i, \mathbf{W}_2) )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{W}_1=\{\mathbf{W}_1,\mathbf{W}_2\}$ denotes the orthogonal planes, $\mathbf{P} \in \mathbb{R}^3$ is the point cloud, $Q(\mathbf{p}, \mathbf{P})$ is the point to plane distance (like ICP) and $\mathbf{p}_i$ represents each point on the point cloud.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really like to enforce the orthogonality in this stage, it could be done by a trivial regularization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\arg\min_{\mathbf{W1}, \mathbf{W2} } E(\mathbf{P}, \{\mathbf{W}_1,\mathbf{W}_2\})\\&#xA;s.t.&#xA;\mathbf{W}_1^n \cdot \mathbf{W}_2^n = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or minimize:&#xA;$$&#xA;E(\mathbf{P}, \{\mathbf{W}_1,\mathbf{W}_2\}) + \lambda (\mathbf{W}_1^n \cdot \mathbf{W}_2^n)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One drawback is that, points in the scene should cover a significant area of the geometric structure that you're looking for. If there is too much clutter, eveen though the scheme above would still work, it would be better to incorporate some robust functions into the formulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E_{robust}(\mathbf{P}, \{\mathbf{W}_1,\mathbf{W}_2\}) = \Phi(E(\mathbf{P}, \{\mathbf{W}_1,\mathbf{W}_2\})) + \lambda (\mathbf{W}_1^n \cdot \mathbf{W}_2^n)&#xA;$$&#xA;where $\Phi$ is an M-Estimator, such as Tukey's Biweights or Huber function. $\mathbf{W}^n$ is the normal part (first 3 coefficients) of the plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This regularization actually constrains the orthogonality, but uses 8 parameters (4 for each plane). If you would like to reduce the dimension by a re-parameterization, here is how to do it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{x} =\{a,b,c,d,\theta,\rho\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $\{a,b,c,d\}$ denotes the equation of the first plane and $(\theta,\rho)$ are the parameters of the second plane in polar space, around the normal of the first plane as described above. The distance function then involves intersecting the planes, and doing the line-distance computation at each stage (as devised above). This is computationally more cumbersome, but causes the orthogonal planes to move together. Instead of polar parametrization, one could equivalently use a simple vector $\mathbf{v}=(\delta x, \delta y)$, which is the orthogonal, scaled direction vector from the center of the reference point to the intersection line:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{x}_2 =\{a,b,c,d,\mathbf{v}\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both parameterizations $\mathbf{x}$ and $\mathbf{x}_2$ alleviate the need for regularization. One should try and see what would work the best. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This completes the procedure. So, a plane fitting and its refinement is devised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, now I think I must publish this idea. But, anyways, there you go : )&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2016-09-22T15:23:16.767" LastActivityDate="2016-09-22T15:23:16.767" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21967" PostTypeId="2" ParentId="21612" CreationDate="2016-01-31T17:27:26.497" Score="1" Body="&lt;p&gt;First a &lt;strong&gt;short answer&lt;/strong&gt; - it did not work because your matrix is singular. The matrix is singular, because you did not do all steps necessary for the case of zero Neumann boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;longer answer&lt;/strong&gt; - I guess your expectation it should work (I might be wrong) were due to the fact that the mean of right hand side is in your case (numerically almost) zero. There exists a step in this kind of problems where a mean of right hand side is subtracted from it, but this is not enough, and in your case it is not necessary because it is equal 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, the zero mean of $f$ (or zero sum of entries in your $f$) means in your case that there exists a solution to your problem, in fact, there are infinitely many solutions. You still have to do some additional work to find at least one solution from those infinitely many ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me first note how to see easily that your matrix is singular. I mean the matrix in your output from Matlab, i.e. without the first and last row and column with zero entries only. If you sum all rows in your matrix $A$ you get the row with zeros only. This can happen only for singular matrix. In fact it means that you can delete e.g. the last row in your matrix, because it is redundant, it can be expressed from other rows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Luckily, if you do the same with the right hand side $f$ then you get also the zero (it means also the mean of $f$ is zero). If this would not be case, your system has no solution (a brief explanation - it would be like you have the zero row in the matrix with non zero right hand side, such equation can not be fulfilled).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now what to do to find at least one representative solution. In general you must add some additional equation that makes the solution unique. It means you have to add one row in the matrix and corresponding one right hand side. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option you mentioned in your question (and it is the most typical one) is to say e.g. that the last unknown (for which in fact you erased the row in $A$) you prescribe an arbitrary value. It is like you replace the zero Neumann boundary condition with Dirichlet one. So it is like to add the row $(0 \, \ldots \, 0 \, 1)$ to replace the erased one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another one is that you require that the mean of &lt;em&gt;unknowns&lt;/em&gt; has a prescribed value, e.g. zero. It is like you have to add a row with ones only, i.e. $(1 \, \ldots 1)$ to replace the erased row. That is a reason that such option is not very popular, because it changes the structure of your matrix. &lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-01-31T17:27:26.497" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21968" PostTypeId="1" CreationDate="2016-01-31T19:18:36.473" Score="1" ViewCount="109" Body="&lt;p&gt;I have the function $cos(x+t)$; the following PDEs have $cos(x+t)$ as the analytical solution of them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t = u_{xx} -\sin(x+t) + \cos(x+t) \\&#xA;u_x(0,t)=-\sin(t) \\&#xA;u_x(1,t)=-u(1,t)^4-\sin(1+t)+\cos^4(1+t)\\&#xA;u(x,0) = \cos(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ v_t = v_{xx} -\sin(x+t)+\cos(x+t) \\&#xA;v_x(0,t)=-\sin(t)\\&#xA;v_x(1,t)=-\sin(1+t) \\&#xA;v(x,0) = \cos(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I computed the error norm $\| E \| = \sqrt{\sum_{i=0}^{1/h} h\cdot | u_{numerical}(i*h)-\cos(i*h+t_{final})|^2 }$ for a set of values of spacestep $h$, the set is: $\{ 1/16 , 1/32 , 1/64 , 1/128 , 1/256 \}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the plots of $\log_{10} \| E \| $ vs $\log_{10} h$ have been plotted, and I also calcualted the slope of this linear squares fit, for the second PDE above the slope is approximately $2$ for $t_{final} =5$, but for the first PDE above the slope is approximately $0.15$, how did this difference came about?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am adding the code below in maple 2015.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hvals:=[1/16, 1/32, 1/64, 1/128, 1/256]:&#xA;tval:=5:&#xA;dopd:= (x, y, hv, tv)-&amp;gt;  rhs&#xA;                           ( pdsolve&#xA;                             ( x,&#xA;                               y,&#xA;                               numeric,&#xA;                               time = t,&#xA;                               range = 0 .. 1,&#xA;                               spacestep = hv,&#xA;                               timestep = hv,&#xA;                               errorest=true&#xA;                             ):-value&#xA;                                ( t=tv,&#xA;                                  output=listprocedure&#xA;                                )[3]&#xA;                           ):&#xA;  pdes:=[ diff(u(x, t), t) = diff(u(x, t), x, x)-sin(x+t)+cos(x+t),&#xA;          diff(v(x, t), t) = diff(v(x, t), x, x)-sin(x+t)+cos(x+t)&#xA;        ]&#xA;  bcs:= [ [ D[1](u)(0,t)=-sin(t),&#xA;            D[1](u)(1,t)=-sin(1+t),&#xA;            u(x,0)=cos(x)&#xA;          ],&#xA;          [ D[1](v)(0,t)=-sin(t),&#xA;            D[1](v)(1,t)=-v(1,t)^4-sin(1+t)+(cos(1+t))^4,&#xA;            v(x,0)=cos(x)&#xA;          ]&#xA;        ]:&#xA;  for p from 1 by 1 to numelems(hvals) do&#xA;      h:= hvals[p]:&#xA;      for k from 1 by 1 to 2 do&#xA;          sol[k]:= dopd( pdes[k], bcs[k], h, tval);&#xA;    od:&#xA;      E[p]:= sqrt&#xA;             ( h*add&#xA;                 ( (sol[2](j*h)-cos(j*h+tval))^2,&#xA;                   j=0..1/h&#xA;                 )&#xA;             ):&#xA;  od:&#xA;  printf( &quot;\n\n      Step Value     Norm Value\n\n&quot;);&#xA;  seq( printf( &quot; %12a,   %12a\n&quot;,&#xA;               hvals[j], E[j]&#xA;              ),&#xA;       j = 1..numelems(hvals)&#xA;     );&#xA;  pA:=Array(1..2):&#xA;  pA[1]:= plot( [ seq( [ hvals[j], E[j] ],&#xA;                       j=1..numelems(hvals)&#xA;                     )&#xA;                ],&#xA;                labels=[&quot;space\\timestep&quot;, &quot;Norm&quot;],&#xA;                labelfont=[times, bold, 16],&#xA;                labeldirections=[horizontal, vertical],&#xA;                title=&quot;Norm vs step (lin-lin)&quot;,&#xA;                titlefont=[times, bold, 20]&#xA;              ):&#xA;  pA[2]:= plot( [ seq( [ log10(hvals[j]), log10(E[j]) ],&#xA;                       j=1..numelems(hvals)&#xA;                     )&#xA;                ],&#xA;                labels=[&quot;space\\timestep&quot;, &quot;Norm&quot;],&#xA;                labelfont=[times, bold, 16],&#xA;                labeldirections=[horizontal, vertical],&#xA;                title=&quot;Norm vs step (log10-log10)&quot;,&#xA;                titlefont=[times, bold, 20]&#xA;             );&#xA;  plots:-display(pA);&#xA;with(CurveFitting):&#xA;LeastSquares([[log10(hvals[1]),log10(E[1])],[log10(hvals[2]),log10(E[2])],[log10(hvals[3]),log10(E[3])],[log10(hvals[4]),log10(E[4])],[log10(hvals[5]),log10(E[5])]],v);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By the comments that I got I am adding a txt file with the computed errors, a plot of the computed solution of the PDE by maple and an explanation of the default meothd being used by maple:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The first mode of operation uses the default method, which is a centered implicit scheme, and is capable of finding solutions for higher order PDE or PDE systems. The PDE systems must be sufficiently close to a standard form for the method to find the numerical solution.&#xA;The default method uses a second order (in space and time) centered, implicit finite difference scheme to obtain the computed solution. The number of points in the stencil of the finite difference scheme is one greater than the order of each equation.&#xA;  Note: A first order accurate accurate boundary condition (the default) can be used for second order accurate method, and still provide a solution of second order accuracy. Use of boundary conditions with an order of accuracy that is too low degrades the accuracy of the solution, while use of boundary conditions with an order of accuracy that is too high may negatively impact the stability of the problem. For first and second order accurate methods, it is generally recommended to use one or two more boundary points than the differential order of the PDE in the space variable. For third and fourth order accurate methods it is generally recommended to use two or three more boundary points than the differential order of the PDE in the space variable.&#xA;  PDE that are greater than first order in time are solved by converting to an equivalent first order system. Note: The conversion to first order is the expected one, namely that for any dependent variables that are higher than first order, variables representing all lower order time derivatives, and corresponding equations, are appended to the system. This choice of additional variables may not always be desirable, particularly when error control is in use, so often better performance can be obtained by choosing the first order system yourself. For more information on this topic, see pdsolve/numeric/firstorder.&#xA;Requested values that do not lie on a mesh point are computed using a Lagrange interpolation that uses 4 space points and 3 time points. The time points are chosen (whenever possible) so that only one time value is larger than the requested time.&#xA;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m7yu1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m7yu1.jpg&quot; alt=&quot;computed solution for x=1 and between t=0 and t=1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the next file there's the txt file with the errors.&#xA;&lt;a href=&quot;http://www.4shared.com/office/_vKiRORWce/the_computed_errors_for_all_th.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;txt file with errors computed&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope you can help me Kirill or anyone else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S&#xA;I am adding the plot of the absolute value between the computed solution and the actual solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tpKwv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tpKwv.jpg&quot; alt=&quot;difference plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17431" LastEditorUserId="17431" LastEditDate="2016-02-05T17:13:42.847" LastActivityDate="2016-02-05T17:13:42.847" Title="Error norm in maple" Tags="|pde|maple|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="21969" PostTypeId="1" AcceptedAnswerId="21971" CreationDate="2016-01-31T21:50:00.363" Score="1" ViewCount="91" Body="&lt;p&gt;I have a PDE defined over $\mathbb{R}$, for which I don't have the exact solution, and I am to approximate it with finite differences so I need to input some BC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest any good references for dealing with infinite intervals as BCs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possible attempts: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Input $[-M,M]$ as domain for large M and try to guess the exact solution at $x=\pm M$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Because of the symmetry of my IC ($\log\cosh x$), identify left and right boundaries, i.e. $u(-M,t)=u(M,t)$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try some Neumann conditions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try periodic boundaries.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Context&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PDE of interest is&#xA;$$u_{t}=a_{1}u_{xx}+a_{2}\Phi(t)u_{x}^{2}$$&#xA;with $u(0,x)=\log\cosh x$ , defined over $(x,t)\in \mathbb{R}\times [0,1]$, $\Phi(t)$ is the CDF of standard Gaussian and $a_{i}$ are constants. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't have an explicit analytic solution.&lt;/p&gt;&#xA;" OwnerUserId="19107" LastEditorUserId="713" LastEditDate="2016-01-31T22:11:12.200" LastActivityDate="2016-01-31T22:24:51.673" Title="How to deal with PDE over the real line" Tags="|pde|boundary-conditions|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21970" PostTypeId="1" CreationDate="2016-01-31T21:50:21.733" Score="3" ViewCount="360" Body="&lt;p&gt;I've seen a few other posts  on this topic but none have full answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to implement some eigen-decomposition algorithms. I've managed to get the Explicit QR algorithm and the Implicit (Francis) algorithm to compute the eigenvalues. My question however relates to computing the eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My core question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverse_iteration&quot; rel=&quot;nofollow&quot;&gt;Inverse Iteration Algorithm&lt;/a&gt; to get the eigenvectors after completing the QR algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand additionally that I'll need to use the transformation matrices from my QR algorithm (balancing, the hessenberg reduction and the QR decomposition). But I'm more interested right now in whether there is some way to get the eigenvectors from the QR algorithm itself (as &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenvalue_algorithm#Iterative_algorithms&quot; rel=&quot;nofollow&quot;&gt;wikipedia&lt;/a&gt; implies there is).&lt;/p&gt;&#xA;" OwnerUserId="19108" LastActivityDate="2016-01-31T21:50:21.733" Title="Computing eigenvectors from the QR algorithm" Tags="|linear-algebra|eigenvalues|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="21971" PostTypeId="2" ParentId="21969" CreationDate="2016-01-31T22:24:51.673" Score="4" Body="&lt;p&gt;You must have boundary conditions in addition to the PDE itself. For example, a common boundary condition for cases such as yours is $u(t,x)\rightarrow 0$ for $x\rightarrow \pm\infty$ for any $t$. I've seen other cases where the condition was of the form $x^{2n} u(t,x)\rightarrow 0$ for $x\rightarrow \pm\infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of finite difference methods that can solve such equations directly (though there are &quot;infinite element&quot; formulations of the finite element method for it). But you can introduce a coordinate transform, for example $(-1,1) \ni y \mapsto x \in (-\infty,\infty)$ that transforms your equation into one that is posed on a finite subset of the real line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid some complications, let me assume that your original problem had been posed on the half-line $x\in (0,\infty)$ and that we want to transform it to one on the interval $y \in (0,1)$. Then we could (among many other options) choose $y = 1-e^{-x}$, which yields $\frac{\partial}{\partial x}=\frac{\partial y}{\partial x} \frac{\partial}{\partial y} = e^{-x}\frac{\partial}{\partial y} = (1-y)\frac{\partial}{\partial y}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would then replace all derivatives with regard to $x$ by derivatives with regard to $y$ using this expression. This yields a PDE with spatially variable coefficients (depending on $y$) to which you can apply the finite difference method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-01-31T22:24:51.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21972" PostTypeId="1" AcceptedAnswerId="21975" CreationDate="2016-01-31T22:48:33.153" Score="6" ViewCount="295" Body="&lt;p&gt;Update: Someone changed the title to this post to a possible answer (&quot;Fourier decomposition of parametric shapes&quot;) but I changed it to a different title as that makes it clear what I was asking. As I think putting the answer in the title makes it confusing to a new reader.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am learning about programming so this question may be obvious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know that a Fourier series can approximate any sort of waveform&#xA;for example, a square tooth waveform has to be decomposed into curved&#xA;waveforms using Fourier series that have infinite analytic terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is is there an analogue (2D version) algorithm to the known Fourier decomposition algorithm&#xA;for implicit 2 variable equations? An example of what I mean is as follows.&#xA;Consider&#xA;&lt;span class=&quot;math-container&quot;&gt;$$aX^2+bY^2=c$$&lt;/span&gt; is the equation of an ellipse. Is there an algorithm that can express an (infinite?) number the ellipses terms (and maybe terms representing other shapes) in a way to get a rectangle. Say, a rectangle 50 units wide and 100 units high centered at &lt;span class=&quot;math-container&quot;&gt;$(0,0)$&lt;/span&gt;. The output of the algorithm would be an analytic function differentiable everywhere (like a Fourier series). The analogue of the periodicity I would guess is the angle if the equations were converted to parametric form. So this algorithm (or concept) if it exists I would say is Fourier like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would imagine if such an algorithm exists then probably using a &#xA;few terms to approximate the rectangle using  ellipses would result&#xA;in a rectangle with some curvature in its side and the corners of the rectangle wouldn't look sharp, and using enough terms to approximate the rectangle using ellipses would result in a rectangle (if the equation was plotted on the screen by a computer program) and the corners would look sharper and sides more straight. I'm aware there are ways to plot a rectangle but I want to know if this algorithm exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, my question is, is there an algorithm that takes as input equations representing 2D shapes (using some representation) and outputs an analytical expression of a desired shape that may not be analytical everywhere (such as rectangle- and it must be able to do a rectangle shape) and I would suspect this algorithm has similarities to the existing Fourier algorithms.&lt;/p&gt;&#xA;" OwnerUserId="19047" LastEditorUserId="20688" LastEditDate="2019-07-03T06:59:47.507" LastActivityDate="2019-07-03T06:59:47.507" Title="Does this Algorithm (probably Fourier like) Exist for 2D Shapes?" Tags="|fourier-analysis|approximation|" AnswerCount="1" CommentCount="5" ClosedDate="2016-02-03T19:10:43.150" ContentLicense="CC BY-SA 4.0" />
  <row Id="21973" PostTypeId="1" CreationDate="2016-01-31T23:38:45.017" Score="4" ViewCount="809" Body="&lt;p&gt;I've been trying to understand the &lt;a href=&quot;https://en.wikipedia.org/wiki/Fast_multipole_method&quot; rel=&quot;noreferrer&quot;&gt;Fast Multipole Method&lt;/a&gt; but not really getting anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like the fastest mainstream N-body simulation algorithm, and I would like to implement it in an efficient javascript version, mostly for amusement in browsers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My math is pretty weak, but my programming is strong, relatively speaking. For example, I'm not even sure what &quot;expansion&quot; means mathematically in this context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can the implementation of FMM be described simply with words? Or is understanding of the math required?&lt;/p&gt;&#xA;" OwnerUserId="19081" LastActivityDate="2020-07-24T14:32:49.530" Title="Help understanding and implementing fast multipole method for N-body" Tags="|algorithms|simulation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21974" PostTypeId="2" ParentId="21973" CreationDate="2016-01-31T23:42:15.697" Score="1" Body="&lt;p&gt;Actually I might have found a good explanation of the algorithm here: &lt;a href=&quot;https://github.com/davidson16807/fast-multipole-method&quot; rel=&quot;nofollow&quot;&gt;https://github.com/davidson16807/fast-multipole-method&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19081" LastActivityDate="2016-01-31T23:42:15.697" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21975" PostTypeId="2" ParentId="21972" CreationDate="2016-02-01T12:21:20.200" Score="5" Body="&lt;p&gt;You can write a curve as a parametric equation (&lt;span class=&quot;math-container&quot;&gt;$x(t)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y(t)$&lt;/span&gt;) for a range say &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; t &amp;lt; 1$&lt;/span&gt;. Then you can have a Fourier decomposition of both &lt;span class=&quot;math-container&quot;&gt;$x(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y(t)$&lt;/span&gt;.&#xA;A rectangle can be written as such a parametric equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you want to decompose a given function as a sum of functions from a specific class. There is a whole field devoted to this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Representation_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Representation_theory&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I do not think anybody has done it for rectangle-to-ellipses yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you want to cover a rectangle by an infinite number of ellipses, i.e. if you mean Minkowski sum, then yes it can be done.&lt;/p&gt;&#xA;" OwnerUserId="19116" LastEditorUserId="20688" LastEditDate="2019-07-03T06:55:26.773" LastActivityDate="2019-07-03T06:55:26.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21977" PostTypeId="1" CreationDate="2016-02-01T12:43:11.387" Score="8" ViewCount="1183" Body="&lt;p&gt;Briefly, what are the different types of basis used in FEM and why is the nodal basis so popular and advantageous in the finite element context?&lt;/p&gt;&#xA;" OwnerUserId="9506" LastActivityDate="2016-02-01T20:28:15.937" Title="Choice of basis in FEM" Tags="|finite-element|numerical-analysis|basis-set|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21978" PostTypeId="2" ParentId="21977" CreationDate="2016-02-01T14:25:26.053" Score="6" Body="&lt;p&gt;The nodal Lagrange bases are nice because they interpolate the functions at the knots: $$\phi_i(x_j) = \delta_{ij}$$ This means that you can read and plot solutions by looking only at the coefficients $u_j$ in the representation:$$u_h(x) = \sum_j u_j\phi_j(x)$$ That's much nicer than having to evaluate the sum at every point you care about knowing $u_h$.  &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-02-01T14:25:26.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21979" PostTypeId="2" ParentId="21977" CreationDate="2016-02-01T14:31:04.637" Score="4" Body="&lt;p&gt;There are a variety of different bases in FEM, but most involve basis functions which are associated with topological entities, like vertices, edges, faces, and element interiors.  This makes it possible to enforce various types of continuity by ensuring that degrees of freedom for such functions match at shared vertices/edges/faces.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These basis functions can also be defined in a hierarchal fashion (define 1D functions, blend those into 2D functions, blend 2D functions into 3D, etc). Bases defined this way can be used to &lt;a href=&quot;http://link.springer.com/article/10.1007/s00607-007-0236-0#/page-1&quot; rel=&quot;nofollow&quot;&gt;expose sparsity&lt;/a&gt; or guarantee other mathematical properties, though their construction is more involved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nodal bases are simpler way to define such functions, so long as an appropriate number of nodes are placed on the vertices, edges, faces, and interior of an element. Continuity can be enforced by ensuring that two nodal values are identical at shared vertices/edges/faces.  Additionally, if such nodes are co-located at quadrature points, this can be exploited for efficient time-stepping and mass-matrix assembly on quadrilateral and hexahedral elements (this is at the root of the &lt;a href=&quot;https://scholar.google.com/scholar?cluster=6331162253546295141&amp;amp;hl=en&amp;amp;as_sdt=0,47&amp;amp;sciodt=0,47&quot; rel=&quot;nofollow&quot;&gt;Spectral Element Method&lt;/a&gt;).  &lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2016-02-01T14:31:04.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21980" PostTypeId="2" ParentId="21977" CreationDate="2016-02-01T16:30:43.433" Score="10" Body="&lt;p&gt;People use all sorts of bases in practice. For example, people use orthonormal bases in DG methods to ensure that the mass matrix in time stepping schemes is diagonal. People also use hierarchical bases when doing $p$ adaptivity because it makes the construction of constraints at faces where different polynomial degrees come together trivial. For higher order methods, people also use other constructions to minimize the condition number of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, there is a wide variety of bases that are in actual use. We just happen to start teaching the FEM with nodal bases because it's so easy to understand and because they're sufficient for most cases.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-02-01T16:30:43.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21981" PostTypeId="2" ParentId="21977" CreationDate="2016-02-01T17:58:15.680" Score="4" Body="&lt;p&gt;In Engineering nodal bases are a good starting point for solid mechanics problems because the &lt;a href=&quot;https://en.wikipedia.org/wiki/Virtual_work&quot; rel=&quot;nofollow&quot;&gt;principle of virtual work&lt;/a&gt; for the discretised system $\mathbf{K} \mathbf{u} = \mathbf{f}$ reads&#xA;$$&#xA;\delta \mathbf{u}^T \mathbf{K} \mathbf{u} = \delta \mathbf{u}^T \mathbf{f}&#xA;$$&#xA;showing that $\mathbf{K} \mathbf{u}$ are indeed (equivalent) nodal &lt;em&gt;internal&lt;/em&gt; forces, and $\mathbf{f}$ are (equivalent) nodal &lt;em&gt;external&lt;/em&gt; forces. For other bases, although the principle of virtual work is still valid, $\mathbf{u}$ are not nodal displacements, and therefore the meaning of $\mathbf{f}$ is much more abstract.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Early FEM developments were mostly driven by engineering applications, and the intuition of point forces applied at nodes was very important for the method diffusion.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2016-02-01T18:15:19.967" LastActivityDate="2016-02-01T18:15:19.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21982" PostTypeId="2" ParentId="21977" CreationDate="2016-02-01T20:28:15.937" Score="1" Body="&lt;p&gt;Ideally, one would like to have an orthonormal basis with respect to the $L_2$ or energy inner product, or both (e.g., the eigenmodes of the operator at hand), because many matrices would be diagonal and thus very easy to invert.  But that's hard to get (solving eigenproblems is a nonlinear procedure).  Lagrange bases is provides sparse matrices, which in some sense is the next best choice to diagonal, and it very easy to build; so they provide the best of both worlds.  But some more sophisticated choices, such as hierarchical, wavelet bases, Bernstein polynomials, are indeed possible and work better in special applications.&lt;/p&gt;&#xA;" OwnerUserId="19120" LastActivityDate="2016-02-01T20:28:15.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21985" PostTypeId="1" CreationDate="2016-02-02T00:38:16.277" Score="4" ViewCount="2199" Body="&lt;p&gt;I am a physics undergrad and I have MOKE data for a magnetic material (thin permalloy film on a silicon substrate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is one of the hysteresis loops I obtained, plotted using Python:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/etjzn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/etjzn.jpg&quot; alt=&quot;Loop&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The form of the $y$-axis is irrelevant since I am just going to normalise the signal. I am also going to shift the loop horizontally and vertically so that it is centered around zero but that should not be hard to do. My data is just in a standard text file so it boils down to $x$ and $y$ data points.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I obtain the following properties from this?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$M_s$ - saturation magnetisation  &lt;/li&gt;&#xA;&lt;li&gt;$M_r$ - remanent magnetisation  &lt;/li&gt;&#xA;&lt;li&gt;$H_c$ - coercive field  &lt;/li&gt;&#xA;&lt;li&gt;$H_\text{sat}$ - saturation field&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, $M_s$ is just where the graph appears to start being linear at the top but of course it looks very fuzzy with this data, so how would I see where this &quot;linearity&quot; begins? Using the data file of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also any advice on correcting for the sloping saturation at either end of the graph would be appreciated, because in theory of course the saturation should be parallel to the x-axis but various optical effects result in this sloping.&lt;/p&gt;&#xA;" OwnerUserId="19123" LastEditorUserId="9667" LastEditDate="2016-02-02T21:48:49.770" LastActivityDate="2020-07-03T16:42:31.037" Title="Data analysis of a magnetic hysteresis loop" Tags="|electromagnetism|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21986" PostTypeId="1" CreationDate="2016-02-02T00:46:30.973" Score="2" ViewCount="118" Body="&lt;p&gt;I'm working on a Magnetization simulation and writing an algorithm using the metropolis method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a change in energy and a Boltzmann distribution, but, my question is about the trial step. Should the trial position of the magnetization be completely random or should it be a random differential added onto the position of the old magnetization? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My energy basically depends on an angle, so, my range is 0 to Pi, and to get anything near a 50% rejection I'm adding or subtracting approximately pi over 4 from each step - which is far too large. If I reduce it to something reasonably small, then the difference in energy between the old and new energy probability is about 0.999 to 1.001. (here I create a random, small angle and add or subtract it from the current angle of the magnetization vector.)&lt;/p&gt;&#xA;" OwnerUserId="19126" OwnerDisplayName="Joseph" LastEditorUserId="19126" LastEditDate="2016-02-02T11:54:51.597" LastActivityDate="2016-03-29T18:40:30.563" Title="Monte Carlo Metropolis method - trial step algorithm" Tags="|computational-physics|monte-carlo|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="21988" PostTypeId="1" CreationDate="2016-02-02T08:37:17.340" Score="1" ViewCount="1907" Body="&lt;p&gt;I have the system of equations&#xA;\begin{align}&#xA;&amp;amp;A  \frac{\partial u_1}{\partial t} = 1 - u_1 B \frac{\partial u_2}{\partial y}\\&#xA;&amp;amp;\frac{\partial u_2}{\partial t} = \frac{\partial}{\partial y}\left[ e^{u_1} \frac{\partial u_2}{\partial y}\right] \enspace .&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial condition is $u_1(y, 0) = 0$, and $u_2(y&amp;lt;1, 0) = 0$, $u_2(1, 0) = 1$. And the boundary conditions are $u_1(1, t)=1$, $u_2(0, t)=0$, and $u_1(1, t)=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $A$ and $B$ are constants. The value of $A$ is 0.04 and the value of $B$ is 0.9. How can I solve these two PDEs simultaneously in COMSOL? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is not possible in COMSOL please suggest me another software.&lt;/p&gt;&#xA;" OwnerUserId="19129" LastEditorUserId="9667" LastEditDate="2016-02-02T21:40:26.380" LastActivityDate="2016-02-02T21:40:26.380" Title="Solving coupled PDE in COMSOL" Tags="|pde|nonlinear-equations|comsol|" AnswerCount="1" CommentCount="0" ClosedDate="2016-02-03T17:40:41.820" ContentLicense="CC BY-SA 3.0" />
  <row Id="21989" PostTypeId="1" CreationDate="2016-02-02T09:33:25.590" Score="5" ViewCount="372" Body="&lt;p&gt;The cfl condition for linear advection equation &#xA;$$&#xA;u_t + a u_x = 0&#xA;$$&#xA;using a DG method of degree $k$ polynomials, upwind flux and an RK scheme of $k+1$ stage/accuracy is stated to be $\frac{1}{2k+1}$ for $L^2$ stability. However I have not been able to find a paper which proves this though it is mentioned in some papers that the proof exists for k=0,1,2. Please point out any existing literature on this.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="1804" LastEditDate="2016-02-02T10:12:02.833" LastActivityDate="2016-02-02T19:14:04.957" Title="Proof of CFL condition for RKDG scheme" Tags="|reference-request|discontinuous-galerkin|cfl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21990" PostTypeId="2" ParentId="21896" CreationDate="2016-02-02T10:23:26.743" Score="3" Body="&lt;p&gt;Generally speaking, adding a single variable to a continuous optimization problem does not drastically decrease algorithm performance. These sorts of manipulations happen routinely to transform apparently &quot;difficult&quot; problems into &quot;easy ones&quot;; examples are things like smooth reformulations of max functions or absolute values by adding variables and constraints. Other sorts of manipulations increase sparsity, or enable more efficient algorithms to be used, and in these cases, the reformulations decrease execution time compared to solving the original formulation of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exceptions are categories of NP-hard problems, e.g., mixed-integer (linear or nonlinear) programs, when using a global optimization algorithm. The worst-case scaling for those algorithms is exponential, so you could see, for instance, in mixed-integer problems with binary variables, a doubling of execution time when adding a binary variable. However, in working with reformulations of binary mixed-integer linear programs, I haven't seen drastic increases in solve times due to adding variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of black-box algorithms, it would not surprise me if adding a variable or two greatly increased execution times. The problem of a degenerate Nelder-Mead simplex iteration could be a result of your reformulation by adding a variable.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2016-02-03T00:14:33.307" LastActivityDate="2016-02-03T00:14:33.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21991" PostTypeId="1" CreationDate="2016-02-02T14:25:50.220" Score="1" ViewCount="42" Body="&lt;p&gt;I am trying to identify a non-linear plant using the identification toolbox of Matlab.&lt;/p&gt;&#xA;&lt;p&gt;In particular I would like to identify a non-linear arx model, where the nonlinearities are expressed by means of tree partitions.&lt;/p&gt;&#xA;&lt;p&gt;According to Mathworks (&lt;a href=&quot;http://it.mathworks.com/help/ident/ref/treepartition.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://it.mathworks.com/help/ident/ref/treepartition.html&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;TREEPARTITION&lt;/strong&gt; is a nonlinear function y = F(x), where y is scalar and&#xA;x a 1-by-m vector.                      F is a piecewise linear&#xA;(affine) function of x:&lt;br /&gt;&#xA;F(x) = x*L+ [1,x]*C_a + d when x belongs to D_a, where&lt;br /&gt;&#xA;L is a 1-by-m vector, C_k is a 1-by-(m+1) vector,&lt;br /&gt;&#xA;and D_k is partition of the x-space.                                                                   The active partition &lt;strong&gt;D_a is determined as an intersection of&lt;/strong&gt;&#xA;&lt;strong&gt;half-spaces&lt;/strong&gt; by a binary tree as follows:         first a tree with N&#xA;nodes and J levels is initialized. A node at level J is a  terminating&#xA;leaf               and a node at a level j &amp;lt; J has two descendants at&#xA;level j+1. All levels are complete, so N = 2^(J+1)-1.        &lt;strong&gt;The&lt;/strong&gt;&#xA;&lt;strong&gt;partition at node&lt;/strong&gt; r is based on [1,x]*B_r &amp;gt; 0 or &amp;lt;= 0 (move to left or&#xA;right descendant),                 where B_r is chosen to improve the&#xA;stability of least-square computation on the partitions at the&#xA;descendant  nodes. &lt;strong&gt;Then at each node r the coefficients&lt;/strong&gt; C_r of best&#xA;linear approximation of unknown regression function   on D_r are&#xA;computed using penalized least-squares algorithm.&lt;/p&gt;&#xA;&lt;p&gt;[...]&lt;/p&gt;&#xA;&lt;p&gt;When the value of the mapping F, defined by the treepartition object, is computed at x, an &lt;strong&gt;adaptive algorithm&lt;/strong&gt; selects the active node k of the tree on the branch of partitions which contain x.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In this description it is not clear what is the underlying algorithm used to identify the current active node. Indeed, at a first glance, I tought that all the intermediate nodes were decisional nodes and the terminating leafs were containing the unknown coefficients that have to be used for a particular PWA representation.&lt;/p&gt;&#xA;&lt;p&gt;That approach is not correct, indeed as bolded in the citation, every single node has its own set or parameters that best fit the unknown regression function in D_r.&lt;/p&gt;&#xA;&lt;p&gt;Is there a clearer reference and/or explanation about this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14817" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-02-02T14:25:50.220" Title="How to find active region in treepartition Matlab object" Tags="|matlab|machine-learning|modeling|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21992" PostTypeId="2" ParentId="21988" CreationDate="2016-02-02T14:28:03.260" Score="1" Body="&lt;p&gt;Actually that is possible to solve that set of equations in COMSOL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can simply select a 1D domain problem, add physics and in particular use the general ODE/DAE interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can add multiple ODE/DAE equations and, for each of them, define all the conditions stated in your picture.&lt;/p&gt;&#xA;" OwnerUserId="14817" LastActivityDate="2016-02-02T14:28:03.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21993" PostTypeId="1" AcceptedAnswerId="21994" CreationDate="2016-02-02T17:10:52.873" Score="3" ViewCount="201" Body="&lt;p&gt;Lets say I have a system of differential equations which has the form&#xA;$$\dot{C}_{\alpha,\beta,m} = f_{\alpha,\beta,m}(C_{\alpha,\beta,1},\ldots,C_{\alpha,\beta,N};t).$$&#xA;The $f$s are some functions of time and of the $C$s. The exact form is not important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, this can be solved with a simple RK4 algorithm.&#xA;However, there is now the additional (orthogonality) constraint &#xA;$$\sum_{\alpha,\beta}C_{\alpha,\beta,m}^* C_{\alpha,\beta,n} \stackrel{!}{=} \delta_{m,n},$$ which should hold at any time $t$. The star denotes the complex conjugate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is now, how this constraint can be implemented in my integration scheme/RK 4 algorithm.&lt;/p&gt;&#xA;" OwnerUserId="18543" LastActivityDate="2016-02-02T18:45:26.253" Title="Integration of differential equation with orthogonality constraint" Tags="|ode|constrained-optimization|runge-kutta|constraints|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21994" PostTypeId="2" ParentId="21993" CreationDate="2016-02-02T18:45:26.253" Score="6" Body="&lt;p&gt;In general, one cannot expect rk4 to maintain quadratic invariants of the system, it simply doesn't do that. Methods that &lt;em&gt;do&lt;/em&gt; maintain specific invariants have to be specially devised — this usually goes by the name of &lt;em&gt;geometric integration&lt;/em&gt;. Symplectic integrators are the most common kind of these methods, although not for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See, for example, the book &lt;a href=&quot;http://link.springer.com/book/10.1007%2F3-540-30666-8&quot;&gt;&lt;em&gt;Geometric Numerical Integration&lt;/em&gt; by Hairer, Wanner, Lubich&lt;/a&gt;, or &lt;a href=&quot;http://www.unige.ch/~hairer/poly-sde-mani.pdf&quot;&gt;the notes &lt;em&gt;Solving Differential Equations on Manifolds&lt;/em&gt; by Hairer (free online)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the simplest way to incorporate the constraint is to project the approximate solution onto the constraint manifold (&quot;Projection methods&quot;, Example 1.3, p. 21 in the Hairer notes above). Given a system of ODEs &#xA;$$ \dot X = f(t, X), \qquad X^*X=I, $$&#xA;define $\pi(X)$ to be the closest unitary matrix to $X$. If $X = UH$ is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Polar_decomposition&quot;&gt;polar decomposition&lt;/a&gt; of $X$, then $\pi(X) = U$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest is to project onto the constraint manifold after each RK step, $\tilde X_{n+1} = \pi(X_{n+1})$. The second simplest is to rewrite the ODE as &#xA;$$ \dot X = f(t, \pi(X)), $$&#xA;which will incorporate the projection into the RK substeps. (This doesn't actually modify the ODE because $\pi(X)=X$ for any exact solution.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond this, the constraint manifold is called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Stiefel_manifold&quot;&gt;Stiefel manifold&lt;/a&gt;, and there already seem to be some number of articles on google scholar on numerical integration on Stiefel manifolds.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-02-02T18:45:26.253" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="21995" PostTypeId="2" ParentId="21989" CreationDate="2016-02-02T19:14:04.957" Score="4" Body="&lt;p&gt;As far is I know, a proof for the general case has not been presented so far. The most detailed analysis has probably been performed by Krivodonova &amp;amp; Qin 2013 in &lt;a href=&quot;http://dx.doi.org/10.1016/j.apnum.2012.07.008&quot;&gt;this&lt;/a&gt; work. Quoting from the conclusion:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We have derived a closed form expressions for the eigenvalues of the DG spatial discretization applied to the onedimensional linear advection equation with periodic boundary conditions and the upwind flux. [...] We have also shown that $(p + 1)(p + 2)$ is a guaranteed bound on the size of the eigenvalues which can be used to compute the CFL condition for large p. However, we have also proved that the growth rate of the largest eigenvalue is less than $(p + 1)^2$. We conjecture that a more accurate rate is proportional to $(p + 1)^{1.75}$. This is in contrast with the currently&#xA;  assumed quadratic rate for the DGM [13] and various spectral methods [12].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3738" LastActivityDate="2016-02-02T19:14:04.957" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="21996" PostTypeId="1" AcceptedAnswerId="21998" CreationDate="2016-02-03T03:33:14.993" Score="12" ViewCount="269" Body="&lt;p&gt;I work on an inverse problem for my Ph.D. research, which for simplicity's sake we'll say is determining $\beta$ in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L(\beta)u \equiv -\nabla\cdot(k_0e^\beta\nabla u) = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from some observations $u^o$; $k_0$ is a constant and $f$ is known. This is typically formulated as an optimization problem for extremizing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J[u, \lambda; \beta] = \frac{1}{2}\int_\Omega(u(x) - u^o(x))^2dx + \int_\Omega\lambda(L(\beta)u - f)dx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\lambda$ is a Lagrange multiplier. The functional derivative of $J$ with respect to $\beta$ can be computed by solving the adjoint equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L(\beta)\lambda = u - u^o.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some regularizing functional $R[\beta]$ is added to the problem for the usual reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The unspoken assumption here is that the observed data $u^o$ are defined continuously throughout the domain $\Omega$. I think it might be more appropriate for my problem to instead use&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J[u, \lambda; \beta] = \sum_{n = 1}^N\frac{(u(x_n) - u^o(x_n))^2}{2\sigma_n^2} + \int_\Omega\lambda(L(\beta)u - f)dx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x_n$ are the points at which the measurements are taken and $\sigma_n$ is the standard deviation of the $n$-th measurement. The measurements of this field are often spotty and missing chunks; why interpolate to get a continuous field of dubious fidelity if that can be avoided?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives me pause because the adjoint equation becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L(\beta)\lambda = \sum_{n = 1}^N\frac{u(x_n) - u^o(x_n)}{\sigma_n^2}\delta(x - x_n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\delta$ is the Dirac delta function. I'm solving this using finite elements, so in principle integrating a shape function against a delta function amounts to evaluating the shape function at that point. Still, the regularity issues probably shouldn't be dismissed out of hand. My best guess is that the objective functional should be defined in terms of the finite element approximation to all the fields, rather than in terms of the real fields and then discretized after.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find any comparisons of assuming continuous or pointwise measurements in inverse problems in the literature, either in relation to the specific problem I'm working on or generally. Often pointwise measurements are used without any mention of the incipient regularity issues, e.g. &lt;a href=&quot;https://books.google.com/books?id=NMWwCqxS5G0C&amp;amp;lpg=PP1&amp;amp;dq=parker%20geophysical%20inverse%20theory&amp;amp;pg=PR3#v=onepage&amp;amp;q=parker%20geophysical%20inverse%20theory&amp;amp;f=false&quot;&gt;here&lt;/a&gt;. &lt;strong&gt;Is there any published work comparing the assumptions of continuous vs. pointwise measurements? Should I be concerned about the delta functions in the pointwise case?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-02-03T16:09:47.960" Title="pointwise vs. continuous observations in PDE inverse problem" Tags="|inverse-problem|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="21997" PostTypeId="1" CreationDate="2016-02-03T05:26:07.090" Score="4" ViewCount="177" Body="&lt;p&gt;I am looking to numerically solve many different integrals where the integrand is simply the indicator function for a region (i.e. 1 on the region, 0 outside. This is for measuring areas). The dimensions range from 1 to 5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is an efficient quadrature method solving this? It seems like the standard methods which utilize polynomial approximations and derivatives wouldn't help in this case because the function is constant (and then discontinuous). You can think of the problem as having a simple integrand, but a complex region (with a continuous maybe even smoother boundary).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think that adaptive Monte Carlo methods like VEGAS or MISER integration will help out? Or is it just best to take a fine grid, sum up the function values at the grid points, and multiply by a constant (area of grid / number of points)?&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2016-02-03T05:55:36.953" LastActivityDate="2016-02-04T19:40:05.110" Title="Efficient Quadrature Methods for Indicator Functions?" Tags="|numerical-analysis|quadrature|monte-carlo|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21998" PostTypeId="2" ParentId="21996" CreationDate="2016-02-03T09:02:21.070" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The measurements of this field are often spotty and missing chunks; why interpolate to get a continuous field of dubious fidelity if that can be avoided?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You're perfectly right - most of the time, interpolation to a continuous field covering the entire domain is not an option. Think about weather prediction problems, where measurements (point-sources) are available only at selected domain locations. I'd say point-wise data is more the norm than the exception when you consider &quot;real-life&quot; inverse problems.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My best guess is that the objective functional should be defined in terms of the finite element approximation to all the fields (&lt;em&gt;discretize-then-optimize&lt;/em&gt;), rather than in terms of the real fields and then discretized after (&lt;em&gt;optimize-then-discretize&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The two approaches are not equivalent (except for very simple problems). There is a vast body of literature comparing the two approaches (each with its advantages and drawbacks). I'd point you towards Max Gunzburger's &lt;a href=&quot;https://books.google.com/books/about/Perspectives_in_Flow_Control_and_Optimiz.html?id=dWKcdjHmMvYC&amp;amp;redir_esc=y&quot;&gt;monograph&lt;/a&gt; (in particular the end of chapter 2).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any published work comparing the assumptions of continuous vs. pointwise measurements? Should I be concerned about the delta functions in the pointwise case?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can represent your source terms exactly - namely, your source term will be modeled as a (discrete approximation to a) Dirac distribution [&lt;a href=&quot;ftp://ftp.ing-mat.udec.cl/pub/ing-mat/pre-publicaciones/2005/pp05-15.pdf&quot;&gt;Arraya et al., 2006&lt;/a&gt;], &lt;em&gt;or&lt;/em&gt; you can approximate the source term by some regularized function (as is done, for example, in the &lt;a href=&quot;https://www.math.nyu.edu/faculty/peskin/papers/acta_numerica.pdf&quot;&gt;immersed boundary method&lt;/a&gt;). Have a look (for starters) at &lt;a href=&quot;http://arxiv.org/abs/1412.4139&quot;&gt;this recent paper&lt;/a&gt; by Hosseini et al. (and references therein).&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2016-02-03T13:17:09.977" LastActivityDate="2016-02-03T13:17:09.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22000" PostTypeId="1" AcceptedAnswerId="22003" CreationDate="2016-02-03T12:42:29.747" Score="1" ViewCount="538" Body="&lt;p&gt;Assuming that the crosses in the figure below are unknowns in a vertex-centered finite difference scheme in an Adaptive Mesh, how can I calculate the double derivate (Laplacian) at the &lt;strong&gt;Red x&lt;/strong&gt; ? The &lt;strong&gt;Red x&lt;/strong&gt; has only Eastern, Western and Northern neighbours but not a Southern neighbour. Can I take the average of just 3 points which are its immediate neighbours ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o5Etx.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o5Etx.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14303" LastActivityDate="2016-02-03T15:30:21.547" Title="Non-uniform finite difference Adaptive Mesh Refinement" Tags="|pde|finite-difference|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="22001" PostTypeId="1" AcceptedAnswerId="22002" CreationDate="2016-02-03T14:39:19.370" Score="0" ViewCount="403" Body="&lt;p&gt;It is necessary for me to solve a Poisson equation by adaptive finite element method  with transition  elements technique to get conforming mesh.&#xA; For the first local refinement everything is OK, because there is  not any transition  elements, but after that there are some of them. After the first step,&#xA; the error must be estimated on these elements or no? If yes, maybe the error is large and the transition  element must be refine in the next refinement, so they are not transition, if no, this means some parts of domain are without error estimation, or I should find the error on larger elements without considere to hanging nodes? &lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2016-02-06T14:48:05.940" LastActivityDate="2016-02-06T14:48:05.940" Title="How to deal with transition elements in adaptive fem" Tags="|finite-element|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22002" PostTypeId="2" ParentId="22001" CreationDate="2016-02-03T15:28:35.410" Score="3" Body="&lt;p&gt;You will need to estimate the error on all cells, including transition cells. You may then wish to refine these differently, if necessary -- see for example the Red-Green Strategy (which I explain in more detail in lecture 15 at &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-02-03T15:28:35.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22003" PostTypeId="2" ParentId="22000" CreationDate="2016-02-03T15:30:21.547" Score="2" Body="&lt;p&gt;You don't typically form a finite difference stencil involving the equation at these &quot;hanging nodes&quot;, but set the value of this vertex to the mean value of the two adjacent vertices along the edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(See for example lecture 16 here: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-02-03T15:30:21.547" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22004" PostTypeId="1" CreationDate="2016-02-03T16:00:39.020" Score="1" ViewCount="85" Body="&lt;p&gt;So after posting &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21916/a-simple-pde-solution-question&quot;&gt;this&lt;/a&gt; question and reading all your comments I would like to make this new question (update).&lt;br&gt;&#xA;If you consider the three equations presented here:&#xA;$$\frac{\partial \rho}{\partial t}&#xA;+\frac{\partial (\rho u)}{\partial z}=0$$&#xA;$$\frac{\partial C_i}{\partial t}&#xA;+\frac{\partial (u C_i)}{\partial z}=Reactionrate$$&#xA;$$\rho=MW_{mean}\frac{P}{RgT}, MW_{mean}=\sum^{N_{comp}}_{i=1}(Y_i*mw_i), Y_i=\frac{C_i}{\sum^{N_{comp}}_{i=1}{C_i}}$$&#xA;continuity equation, mass balance for each species &amp;amp; an equation to calculate density, what would the discretized form (in $z$ direction, with backwards finite differences) be? I mean, if we consider (for solution purposes) that the variables are $C_i$, $u$ &amp;amp; $\rho$ (Case_1 a, b &amp;amp; c) then it is totally different than considering that the variables are $C_i$, $(uC_i)$, $\rho$ &amp;amp; $(\rho u)$ (Case_2)(as mentioned by &lt;em&gt;Wolfgang Bangerth&lt;/em&gt; at the previous &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21916/a-simple-pde-solution-question&quot;&gt;question&lt;/a&gt;, the term $\frac{\partial (\rho u)}{\partial z}$ implies conservation of quantity $(\rho u)$ in direction $z$, so should we treat this quantity like one?). Next I present the discretized form of both this cases (please correct me if I am wrong, or more precisely, tell me which is the correct one):&lt;br&gt;&#xA;Case_1a&lt;br&gt;&#xA;$$\frac{\partial \rho_j}{\partial t}&#xA;+\frac{(\rho_j u_j - \rho_{j-1} u_{j-1})}{\Delta z}=0$$&#xA;$$\frac{\partial C_{i,j}}{\partial t}&#xA;+\frac{(u_j C_{i,j} -u_{j-1} C_{i,j-1})}{\Delta z}=Reactionrate$$&#xA;$$\rho_j=MW_{mean,j}\frac{P}{RgT}, MW_{mean,j}=\sum^{N_{comp}}_{i=1}(Y_{i,j}*mw_i), Y_{i,j}=\frac{C_{i,j}}{\sum^{N_{comp}}_{i=1}{C_{i,j}}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case_1b&lt;br&gt;&#xA;$$\frac{\partial \rho_j}{\partial t}&#xA;+\rho_j\frac{(u_j - u_{j-1})}{\Delta z}&#xA;+u_j\frac{(\rho_j - \rho_{j-1})}{\Delta z}=0$$&#xA;$$\frac{\partial C_{i,j}}{\partial t}&#xA;+u_j\frac{(C_{i,j} - C_{i,j-1})}{\Delta z}&#xA;+C_{i,j}\frac{(u_j - u_{j-1})}{\Delta z}=Reactionrate$$&#xA;$$\rho_j=MW_{mean,j}\frac{P}{RgT}, MW_{mean,j}=\sum^{N_{comp}}_{i=1}(Y_{i,j}*mw_i), Y_{i,j}=\frac{C_{i,j}}{\sum^{N_{comp}}_{i=1}{C_{i,j}}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Case_2&lt;br&gt;&#xA;$$\frac{\partial \rho_j}{\partial t}&#xA;+\frac{(\rho u)_j - (\rho u)_{j-1}}{\Delta z}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial C_{i,j}}{\partial t}&#xA;+\frac{(u C_i)_j - (u C_i)_{j-1}}{\Delta z}=Reactionrate$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho_j=MW_{mean,j}\frac{P}{RgT}, MW_{mean,j}=\sum^{N_{comp}}_{i=1}(Y_{i,j}*mw_i), Y_{i,j}=\frac{C_{i,j}}{\sum^{N_{comp}}_{i=1}{C_{i,j}}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, should there be any difference detween the results in Case_1a &amp;amp; Case_1b given the same initial $(t=0)$ and boundary $(z=0)$ conditions? What is the correct formulation (mathematically speaking)? Do you think that this set of equations has one unique solution? Because I am almost sure that it does not. What assumption could justify a change like this:&lt;br&gt;&#xA;Case_1c&#xA;$$\frac{\partial \rho_j}{\partial t}&#xA;+\rho_j\frac{(u_j - u_{j-1})}{\Delta z}=0$$&#xA;$$\frac{\partial C_{i,j}}{\partial t}&#xA;+u_j\frac{(C_{i,j} - C_{i,j-1})}{\Delta z}=Reactionrate$$&#xA;$$\rho_j=MW_{mean,j}\frac{P}{RgT}, MW_{mean,j}=\sum^{N_{comp}}_{i=1}(Y_{i,j}*mw_i), Y_{i,j}=\frac{C_{i,j}}{\sum^{N_{comp}}_{i=1}{C_{i,j}}}$$&#xA;which I believe that does have one uniquie solution?&lt;br&gt;&#xA;And referring to the Case_2, this way we have four(*) unknown variables ($C_i$, $(uC_i)$, $\rho$ &amp;amp; $(\rho u)$) which means that we need one more equation? Would an equation like:&#xA;$$\frac{(uC_i)_j}{C_{i,j}}=\frac{(\rho u)_j}{\rho_j}$$&#xA;work? and with given initial and boundary conditions provide one and only one solution to this problem?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;*actually the variables are more than four. If $N_c=Number of Components$ then the variables are $N_c*C_i+N_c*(uC_i)+\rho+(\rho u)$ times the number of discrete points in z direction but the use of the last equation should give the same number of equations and variables along with the boundary conditions.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="19040" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-02-03T16:00:39.020" Title="PDE discretization (via finite difference sheme) question" Tags="|pde|finite-difference|numerical-analysis|fluid-dynamics|numerical-modelling|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22005" PostTypeId="2" ParentId="21996" CreationDate="2016-02-03T16:09:47.960" Score="5" Body="&lt;p&gt;To expand on @GoHokies's answer: If you're interested in regularity questions, you can also ask what &quot;point measurements&quot; really are. In physical practice, you cannot measure anything at a &quot;point&quot;. Rather, you are always going to get some kind of average over some kind of space-time chunk: a thermometer is not a point but an extended object, and it takes time to adjust to the temperature of the medium around it; a concentration measurement device needs a finite sample size; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this means mathematically is that the delta functions in your functional are, really, averages over sufficiently small areas and/or time intervals. Consequently, the right hand sides in the dual equation are also finite, and no regularity issues arise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in practice, you will typically not be able to resolve the small space or time intervals on which you measure with a finite element mesh. That is, on length scales you can resolve, the right hand side &lt;em&gt;does&lt;/em&gt; look singular, and consequently so does the solution. But, since you're &lt;em&gt;already&lt;/em&gt; introducing a discretization error, you can also regularize the characteristic function of the volume over which you measure by a discrete approximation with the same weight; if you do it right, you will introduce an error that is no larger than the discretization error, at the benefit of receiving a perfectly nice right hand side function for the (discrete) dual equation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-02-03T16:09:47.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22006" PostTypeId="1" CreationDate="2016-02-03T16:47:42.703" Score="6" ViewCount="337" Body="&lt;p&gt;I'm used to displacement forumlation of elastic wave equation:&#xA;$$&#xA;\nabla \cdot \sigma (u) + F = \rho \ddot{ u }&#xA;$$&#xA;where $u$ is the primary variable. Recenty I started experimenting with DG and in almost every paper the stress-velocity formulation is used as a &quot;conservative or divergence form&quot;. What's special about this formulation? I can't figure out why they use Inerior Penalty with displacement equation and other fluxes like Lax-Friedrich's with a system of $v, \sigma$.&#xA;I would be thankful for any kind of paper, book or resource.&lt;/p&gt;&#xA;" OwnerUserId="19145" LastActivityDate="2018-11-10T11:06:40.837" Title="Velocity-Stress formulation of Elastodynamics/Wave Equation for beginner" Tags="|discontinuous-galerkin|wave-propagation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22008" PostTypeId="2" ParentId="22006" CreationDate="2016-02-03T19:56:46.003" Score="3" Body="&lt;p&gt;The stress-velocity formulation has been used extensively in DG context on account of the fact that it can lead to a first order system form of the elastic equation. The latter proved to lend itself more easily to a DG formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, recently a DG formulation of the elastic equation in the second-order form has been proposed, which you may find useful, see &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0045782518301920&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6775" LastEditorUserId="6775" LastEditDate="2018-11-10T11:06:40.837" LastActivityDate="2018-11-10T11:06:40.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22011" PostTypeId="1" CreationDate="2016-02-03T22:40:13.450" Score="7" ViewCount="159" Body="&lt;p&gt;We have an elliptic projection &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P: V \rightarrow V_{h}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which satisfies&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Vert u - Pu \Vert_{L^{2}(\Omega_{e})} \leq Ch^{k+1} \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we say anything about $\Vert u - Pu \Vert_{L^{2}( \partial \Omega_{e})}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know if we have use the Trace theorem we can bound it by $h^{k}$, but I would not like to lose the order of convergence.&lt;/p&gt;&#xA;" OwnerUserId="19151" LastEditorUserId="9667" LastEditDate="2016-02-04T00:58:51.687" LastActivityDate="2016-02-04T19:18:24.820" Title="Approximation properties of FEM projections operators on a boundary" Tags="|finite-element|projection|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22013" PostTypeId="1" CreationDate="2016-02-04T08:39:34.820" Score="5" ViewCount="134" Body="&lt;p&gt;I want to use ADMM to solve a LP defined over a graph.&#xA;According to &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Distributed optimization and statistical learning via the alternating direction method of multipliers&#xA;  S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, 2011&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the first primal update ($x^{k+1}$) is given by the solution of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{bmatrix} \rho I &amp;amp; A' \\ A &amp;amp; 0 \end{bmatrix}  \begin{bmatrix} x^{k+1} \\ \nu \end{bmatrix} + \begin{bmatrix} q - \rho(z^k - u^k) \\ -b \end{bmatrix} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, to solve it for $x^{k+1}$ one has to invert the matrix and therefore needs the full vectors $z^k$ and $u^k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because my A is (almost) the incidence matrix, I would like to compute $x^{k+1}_i$ only as a function of $z^k_j$ and $u^k_j$, for $j$ such that $A_{ij} \neq 0$ (in this way, hypothetically, nodes' graph needs fewer communications). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a direct way to do so?&lt;/p&gt;&#xA;" OwnerUserId="19156" LastEditorUserId="19156" LastEditDate="2016-02-09T08:07:41.213" LastActivityDate="2016-02-09T08:07:41.213" Title="ADMM for Linear Program over graph" Tags="|linear-programming|admm|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22016" PostTypeId="2" ParentId="1671" CreationDate="2016-02-04T12:27:04.130" Score="4" Body="&lt;p&gt;HDF5 can use a &quot;shuffling&quot; algorithm where the bytes for N floating point numbers are rearranged so that the first bytes of the N numbers come first, then the 2nd, and so on. This produces better compression ratios after gzip is applied, as it is more likely to produce longer sequences of the same value. See &lt;a href=&quot;http://www.hdfgroup.org/HDF5/doc_resource/H5Shuffle_Perf.pdf&quot; rel=&quot;nofollow&quot;&gt;here for some benchmarks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19161" LastActivityDate="2016-02-04T12:27:04.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22017" PostTypeId="1" AcceptedAnswerId="22028" CreationDate="2016-02-04T16:29:32.423" Score="4" ViewCount="904" Body="&lt;p&gt;When solving fluid flow problems using finite elements you typically end up requiring that $p\in L^2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Darcy flow problems, a popular choice of elements is the Raviart-Thomas element and discontinuous pressure elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Stokes/Navier-Stokes flow a popular choice of elements is the Taylor-Hood element which has piecewise continuous pressure elements. This is actually more continuity than is required, because for a function to be in $L^2$ it does not have to be continuous. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some reason that discontinuous pressure elements are preferred (or necessary) for Darcy flow problems, but not for Stokes/Navier-Stokes flow problems? &lt;/p&gt;&#xA;" OwnerUserId="9728" LastActivityDate="2016-12-04T03:24:41.977" Title="Continuous vs discontinuous pressure elements in fluid flow problems" Tags="|finite-element|fluid-dynamics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22018" PostTypeId="1" CreationDate="2016-02-04T16:57:41.347" Score="2" ViewCount="32" Body="&lt;p&gt;I'm working on a finite volume advection scheme for unstructured meshes which uses a multidimensional polynomial weighted least squares fit for interpolating from cell centres onto faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2D, the advection scheme typically uses a 4x3 stencil of cell values in the domain interior to fit a polynomial that is cubic in x and quadratic in y.  The weights resulting from the SVD are expensive to compute but they can be precomputed during initialisation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two cases where which wish to omit some higher order terms:&#xA;1. Near boundaries (which may be irregular in shape), the stencil is necessarily smaller&#xA;2. If some cell values are very closely spaced on certain meshes, we do not have enough information to sensibly fit high order terms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a technique for dealing with these cases.  Can the SVD help me determine which terms to omit?  Or are there other techniques I could consider? &lt;/p&gt;&#xA;" OwnerUserId="7192" LastActivityDate="2016-02-04T16:57:41.347" Title="Choosing suitable polynomial degree based on information in advection stencil" Tags="|interpolation|least-squares|polynomials|svd|unstructured-mesh|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22020" PostTypeId="2" ParentId="21951" CreationDate="2016-02-04T18:01:03.603" Score="1" Body="&lt;p&gt;I think your problem can be written as an optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\{x_i\}$ is the set of points for plane 1, $\{x_j\}$ for plane 2 respectively. Their orthonormal vectors are $n_1$ and $n_2$ with constraints: $|n_1|=1$, $|n_2|=1$ and $n_1n_2=0$.&#xA;$\{\lambda_i\}$ the set of lagrange multiplier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The functional under constraints reads&#xA;$$&#xA;\sum_i (x_i n_1-c_1)^2 +\sum_j(x_jn_2-c_2)^2+\lambda_1 n_1n_2+\lambda_2(n_1n_1-1)+\lambda_3(n_2n_2-1)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you are looking for the minimum of the functional with parameter set $\{n_{1x},n_{1y},n_{1z},n_{2x},n_{2y},n_{2z},c_1,c_2,\lambda_1,\lambda_2\lambda_3\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you parametrize the conditions into the normal vectors of the planes you can skip the lagrange multiplier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&#xA;Update to answer your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach will guarantee orthogonality (within the numerical accuracy of representing $\lambda_i$). You can easily see this, if you don't use lagrange multipliers but choose spherical coordinates for your normal vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say $\phi_i, \theta_i$ are the spherical coordinates ($\phi_i\in[-\pi,\pi], \theta_i\in[0,\pi]$),  $|n_i|=1$ is then already in the parametrization and the additional condition $n_1n_2=0$ will lead to a condition on $\theta_2$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\cot\theta_2=\frac{\sin\theta_1(\cos \phi_1 \cos\phi_2+\sin \phi_1 \sin\phi_2)}{\cos \theta_1}$ with $\theta_1 \neq \frac{\pi}{2}$, which you can use to express $n_2$ with $\{\phi_1,\theta_1,\phi_2\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you plug this into the posted functional you will see that it obeys this parametrization as well and you are left with the set of free parameters $\{c_1,c_2,\phi_1,\phi_2,\theta_1\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as you can see, if you choose such a map for $\mathbb{R}^3$, you have to deal with all degenerancies of the map, which is unfovarable in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Playing around with a test case I like to add some more information about numerical minimization with lagrange constraints.&#xA;Numerical minimzation routines have problems with saddle points. One way to circumvent such a problem is not to minimize the functional stated above, but to minimze the norm of its gradient $|\nabla f|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my mathematica test case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ is the functional without constraints.&#xA;$g$ are the lagrange multiplier and constraints.&#xA;$ll$ is the functional as written above.&#xA;$lll$ is the norm of the gradient of the functional above.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[256]:= ClearAll[&quot;Global`*&quot;]&#xA;x1 = RandomReal[1, {17, 3}];&#xA;x2 = RandomReal[1, {17, 3}];&#xA;vars = {n1, n2, n3, c1, m1, m2, m3, c2, l1, l2, l3}&#xA;f[n1_, n2_, n3_, c1_, m1_, m2_, m3_, c2_] = &#xA;  Total[(x1.{n1, n2, n3} - c1)^2] + Total[(x2.{m1, m2, m3} - c2)^2];&#xA;g[n1_, n2_, n3_, m1_, m2_, m3_, l1_, l2_, l3_] = &#xA;  l1 ({n1, n2, n3}.{n1, n2, n3} - 1) + &#xA;   l2 ({m1, m2, m3}.{m1, m2, m3} - 1) + l3 ({n1, n2, n3}.{m1, m2, m3});&#xA;ll[n1_, n2_, n3_, c1_, m1_, m2_, m3_, c2_, l1_, l2_, l3_] = &#xA;  f[n1, n2, n3, c1, m1, m2, m3, c2] + &#xA;   g[n1, n2, n3, m1, m2, m3, l1, l2, l3];&#xA;lll[n1_, n2_, n3_, c1_, m1_, m2_, m3_, c2_, l1_, l2_, l3_] := &#xA; Table[D[ll @@ vars, vars[[i]]], {i, 1, Length[vars]}].Table[&#xA;   D[ll @@ vars, vars[[i]]], {i, 1, Length[vars]}]&#xA;s = FindMinimum[&#xA;  lll[n1, n2, n3, c1, m1, m2, m3, c2, l1, l2, &#xA;   l3], {{n1, 1.0}, {n2, 0}, {n3, 1}, {m1, 0.0}, {m2, 1}, {m3, &#xA;    1}, {c1, 1}, {c2, 2}, {l1, 1}, {l2, 2}, {l3, 3}}]&#xA;FullSimplify[lll[n1, n2, n3, c1, m1, m2, m3, c2, l1, l2, l3] /. s[[2]]]&#xA;FullSimplify[g[n1, n2, n3, m1, m2, m3, l1, l2, l3] /. s[[2]]]&#xA;FullSimplify[f[n1, n2, n3, c1, m1, m2, m3, c2] /. s[[2]]]&#xA;{n1, n2, n3}.{n1, n2, n3} - 1 /. s[[2]]&#xA;{m1, m2, m3}.{m1, m2, m3} - 1 /. s[[2]]&#xA;{n1, n2, n3}.{m1, m2, m3} /. s[[2]]&#xA;&#xA;&#xA;Out[259]= {n1, n2, n3, c1, m1, m2, m3, c2, l1, l2, l3}&#xA;&#xA;Out[264]= {6.88327*10^-30, {n1 -&amp;gt; 0.441163, n2 -&amp;gt; -0.600498, &#xA;  n3 -&amp;gt; 0.666916, m1 -&amp;gt; 0.243615, m2 -&amp;gt; 0.795371, m3 -&amp;gt; 0.55501, &#xA;  c1 -&amp;gt; 0.244619, c2 -&amp;gt; 0.855017, l1 -&amp;gt; -1.43598, l2 -&amp;gt; -0.740652, &#xA;  l3 -&amp;gt; -0.329995}}&#xA;&#xA;Out[265]= 1.07099*10^-29&#xA;&#xA;Out[266]= -1.83184*10^-17&#xA;&#xA;Out[267]= 2.17663&#xA;&#xA;Out[268]= 0.&#xA;&#xA;Out[269]= 0.&#xA;&#xA;Out[270]= 5.55112*10^-17&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the solution planes are orthonormal within numerical noise.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2016-02-08T17:40:35.123" LastActivityDate="2016-02-08T17:40:35.123" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22022" PostTypeId="1" AcceptedAnswerId="25903" CreationDate="2016-02-04T19:17:22.703" Score="9" ViewCount="3531" Body="&lt;p&gt;Markov Chains can be used to generate, or auto-complete, text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain#Markov_text_generators&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Markov_chain#Markov_text_generators&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training text is read, and some information about the text is remembered, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The word &quot;I&quot; is usually followed by the word &quot;have&quot;.&#xA;The word &quot;to&quot; is usually followed by the word &quot;go&quot;, and somtimes the word &quot;be&quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be used to generate in text like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A pride of related females and a key species sought for exhibition in the four big cats in breeding programs for ten years, as injuries sustained from continual fighting with rival males exceeding 250 kg (550 lb) in historic times. Until the Lascaux and medieval cultures where they may take to Peru.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Although, this is a lot less effective with many other types of texts.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; Can example images be shown, where an aproach similar to the use of Markov Chains to generate/auto-complete text, has been used to process images?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is this still unrealistically computationally expensive?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I was unable to find examples by searching, I may be lacking the correct vocabulary to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Markov Chains like this could be very computationally expensive, a small 5x5 neighborhood, in monocrome image (black and white only, no shades of grey) has 2^(5^2) ≈ 33 million possible combinations.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Such processing could include: Inpainting, synthisis, improving resolution and and error correction/checking.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Examples of inpainting and synthisis, using other approaches:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inpainting: Gaps in images being fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t5dl6.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t5dl6.png&quot; alt=&quot;inpainting example&quot;&gt;&lt;/a&gt;&#xA;Other inpainting examples and information:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://reference.wolfram.com/language/ref/Inpaint.html&quot; rel=&quot;noreferrer&quot;&gt;https://reference.wolfram.com/language/ref/Inpaint.html&lt;/a&gt; (click &quot;open all&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Synthisis: For example Google Deep Dream, using convolutional neural networks and inceptionism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YG6ky.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YG6ky.jpg&quot; alt=&quot;synthisis example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19166" LastEditorUserId="19166" LastEditDate="2017-01-13T11:29:48.877" LastActivityDate="2021-09-15T05:59:07.993" Title="Markov (Chain) image generators?" Tags="|algorithms|iterative-method|machine-learning|computer-vision|constraints|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="22023" PostTypeId="2" ParentId="22011" CreationDate="2016-02-04T19:18:24.820" Score="4" Body="&lt;p&gt;If you use a sharper trace inequality you get&#xA;$$\begin{aligned}\|u-Pu\|_{0,\partial \Omega}^2 &amp;amp;\lesssim \|u-Pu\|_0 \|u-Pu\|_1 \\&#xA;&amp;amp;\lesssim h^{k+1} h^{k}|u|^2_{k+1} \end{aligned} $$&#xA;implying&#xA;$$\|u-Pu\|_{0,\partial \Omega} \lesssim h^{k+\frac12},$$&#xA;just as Wolfgang suggested in the comments.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2016-02-04T19:18:24.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22024" PostTypeId="2" ParentId="21997" CreationDate="2016-02-04T19:40:05.110" Score="3" Body="&lt;p&gt;I found that the grid summing method beat out both the Cubature package and the adaptive Monte Carlo schemes from the GSL package (standard, VEGAS, and MISER) when it came to both speed an accuracy (in fact, Cubature's hcubature adaptive methods wouldn't converge for abstol&amp;lt;1e-1, showing that it really cannot handle these  kinds of problems). For future reference, I found a paper on a method &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1364923&quot; rel=&quot;nofollow&quot;&gt;called adaptive geometric integration&lt;/a&gt; that someone might wish to try, but for now simply summing over the grid (which is easy to parallelize) seems to be the best option for the effort.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-02-04T19:40:05.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22025" PostTypeId="1" CreationDate="2016-02-04T19:46:49.647" Score="2" ViewCount="3442" Body="&lt;p&gt;Recently I work on mechanical shocks (e.g. impact) in FE (fenics). I've already put together simple timesteppers (Euler, Crank-Nicolson). I use higher order basis, so I think of higher order time integration schemes like RK4. Is it worth the effort in mechanics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find any appropriate example of this method in elastic waves, so maybe it's pointless. I'm not sure, how to apply it in my case, since it's written in &quot;incremental&quot; form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_{i+1} = y_i+\frac{1}{6}(k_1+2k_2+2k_3+k_4)h$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I'm trying to apply it to system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{eqnarray*} \frac{\partial v}{\partial t} - \nabla u &amp;amp;=&amp;amp; f \\  \frac{\partial u}{\partial t} &amp;amp;=&amp;amp; v \end{eqnarray*}$$&lt;/p&gt;&#xA;" OwnerUserId="19145" LastEditorUserId="4912" LastEditDate="2016-02-04T23:12:37.277" LastActivityDate="2021-05-16T10:44:56.130" Title="Runge Kutta for wave equation" Tags="|runge-kutta|wave-propagation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22026" PostTypeId="2" ParentId="1671" CreationDate="2016-02-04T19:50:18.723" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If a function is smooth, there's obviously a lot of correlation between the numbers representing that function, so the data should compress well.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Perhaps the format you require need to store just the offsets from value to neighboring value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately, maybe you could make use of the frequency domain, perhaps even saving these values as a lossless audio file such as &quot;flac lossless&quot;, as you require some of the same properties to a sound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm going to take a different approach to attempting to answer the question which I hope can be of some help. As what you are saying is also that the minimum description length to represent this data is less than providing all the data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Minimum_description_length&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Minimum_description_length&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Effectively a program, computer code, is a good example. And if you don't mind that somthing thats primarily data working by executing, and so also being code, then you could compress your floating point values into somthing like a function or formuli.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this particularly well automatically, and in a realistic amount of compute, is beyond hard. However the Wolfram Language provides some functionality to attempt this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://reference.wolfram.com/language/ref/FindSequenceFunction.html&quot; rel=&quot;nofollow&quot;&gt;https://reference.wolfram.com/language/ref/FindSequenceFunction.html&lt;/a&gt;&#xA;&lt;a href=&quot;https://reference.wolfram.com/language/ref/FindGeneratingFunction.html&quot; rel=&quot;nofollow&quot;&gt;https://reference.wolfram.com/language/ref/FindGeneratingFunction.html&lt;/a&gt;&#xA;&lt;a href=&quot;https://reference.wolfram.com/language/ref/FindFormula.html&quot; rel=&quot;nofollow&quot;&gt;https://reference.wolfram.com/language/ref/FindFormula.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://reference.wolfram.com/language/ref/RSolve.html&quot; rel=&quot;nofollow&quot;&gt;https://reference.wolfram.com/language/ref/RSolve.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19166" LastActivityDate="2016-02-04T19:50:18.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22028" PostTypeId="2" ParentId="22017" CreationDate="2016-02-04T22:59:46.550" Score="4" Body="&lt;p&gt;Using discontinuous pressure spaces has the advantage that the solution is cellwise conservative (because you can test the equation $\nabla \cdot u=0$ with the characteristic function of each element). Thus, there is an advantage for using the $Q_k - P_{-(k-1)}$ element on quadrilateral/hexahedra. People do indeed use this element in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, this implies 3 pressure degrees per cell (in 2d) as opposed to roughly one per cell for the $Q_1$ element. So it's more expensive. But a practical observation is that it is not more accurate, but in fact often &lt;em&gt;less&lt;/em&gt; accurate than just the regular Taylor-Hood element. The choice therefore comes to &quot;better accuracy and cheaper&quot; (Taylor-Hood) vs &quot;locally conservative but more expensive&quot; (discontinuous pressures).&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-12-04T03:24:41.977" LastActivityDate="2016-12-04T03:24:41.977" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="22029" PostTypeId="2" ParentId="22025" CreationDate="2016-02-04T23:25:44.313" Score="4" Body="&lt;p&gt;High order RK methods work fine for a large number of wave propagation (an &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999110005024&quot; rel=&quot;nofollow&quot;&gt;elastic example using a DG discretization&lt;/a&gt;).  The procedure is usually to discretize the spatial derivatives in the first order form of the wave equation.  This is called semi-discretization and it results in a system of ODEs involving a vector of solution coefficients $U$ (for both $u$ and $v$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dU(t)}{dt} = AU(t)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is defined by your choice of discretization and the wave equation you want to solve.  You can then apply Runge-Kutta methods directly to the equation - this involves computing RHS evaluations. Since the wave equation is linear, marching from $t_n$ to $t_n + dt$ with RK4 method requires computing four vectors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{align*}&#xA;k_1 &amp;amp;= AU(t_n)\\&#xA;k_2 &amp;amp;= A(U(t_n) + ({dt/2}) k_1)\\&#xA;k_3 &amp;amp;= A(U(t_n) + ({dt/2}) k_2)\\&#xA;k_4 &amp;amp;= A(U(t_n) + (dt) k_3)&#xA;\end{align*}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then computing the final update $U(t_n + dt) = \frac{dt}{6}(k_1+2k_2+2k_3+k_4)$. Repeat this to march forward in time again.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="123" LastEditDate="2016-02-05T05:17:19.893" LastActivityDate="2016-02-05T05:17:19.893" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22030" PostTypeId="1" CreationDate="2016-02-04T23:47:53.660" Score="3" ViewCount="189" Body="&lt;p&gt;$A$ is a symmetric matrix and is known to be invertible. $X$ is rectangular of size $(N+p) \times N$ with $p &amp;gt; 0$ but full column rank. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we provide an upper bound on the condition number of $X^{T}AX$ based on this information along with the condition number of $A$ and norm of $X$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we use the pseudoinverse of $X$? Do submultiplicative norms directly extend to them?&lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2017-05-21T18:51:11.550" Title="Condition number of $X^{T}AX$" Tags="|linear-algebra|condition-number|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22031" PostTypeId="1" CreationDate="2016-02-05T09:26:30.803" Score="6" ViewCount="283" Body="&lt;p&gt;How does the pseudo inverse of a full column rank matrix change if I rescale a single row?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In more detail the problem is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a fixed matrix $V$ with linear independent columns and &#xA;lots of matrices $D_i$ of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D_i = &#xA;\begin{pmatrix}&#xA;d_i &amp;amp; 0 \\&#xA;0 &amp;amp; E_n &#xA;\end{pmatrix}&#xA;$$&#xA;where $E_n$ is the identity matrix. All matrices are real and have nice condition numbers. So far so sweet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know the pseudoinverse $V^+=(V^*V)^{-1} V^*$ of $V$ and the job is to compute all the pseudo inverses &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(D_i \cdot V)^+$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in a fast way. Any ideas how to exploit the structure of the problem here? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if the problem was the other way round, algebra of pseudo inveses would allow to break it down to inverting a single number and multiplying a single row:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(V \cdot D_i )^+ = D_i^{-1} \cdot V^+$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my applications $V$ is of small size, like $4 \times 3$ for example.&lt;/p&gt;&#xA;" OwnerUserId="19178" LastEditorUserId="19178" LastEditDate="2016-02-05T09:43:18.420" LastActivityDate="2018-05-04T00:30:38.710" Title="Pseudoinverse of perturbed matrix" Tags="|linear-algebra|performance|least-squares|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="22033" PostTypeId="2" ParentId="22030" CreationDate="2016-02-05T11:20:49.877" Score="6" Body="&lt;p&gt;Let's show that we cannot bound the condition number of $X^T A X$ by using only the condition number of $A$ and the norm of $X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $A=I$, so its condition number is exactly $1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $X$ consist of an invertible diagonal block with $p$ rows of zeros padded at the bottom:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ X = \begin{pmatrix} D \\ 0 \end{pmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now $X^T A X = D^2$, and its condition number would be $\|D^{-2}\|\cdot \|D^2\|$ for some choice of matrix norm.  While the norm of $X$ can be used to bound the norm of $D^2$, we have no similar control over how large the norm of $D^{-2}$ can be.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="127" LastEditDate="2017-05-21T18:51:11.550" LastActivityDate="2017-05-21T18:51:11.550" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22034" PostTypeId="1" CreationDate="2016-02-05T11:25:09.690" Score="1" ViewCount="298" Body="&lt;p&gt;I am using finite elements, Newton Raphson technique and MUMPS direct solver, to solve for $P$ in this equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\delta K}{\delta x} \frac{\delta P}{\delta x}= \frac{\delta K B}{\delta x} + A$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A$ and $B$ are constant parameters. $K$ is a parameter varying over space. $P$ is the solution variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K$ is distributed like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EwmNn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EwmNn.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, $K$ becomes zero! So I'll have zeros on the diagonal of my matrix and the solver will eventually print out that the matrix is singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any techniques to still solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="19181" LastEditorUserId="9667" LastEditDate="2016-02-05T19:48:37.613" LastActivityDate="2016-02-05T19:48:37.613" Title="Solving system of equations with zeros on diagonal" Tags="|pde|finite-element|linear-solver|sparse-matrix|" AnswerCount="0" CommentCount="2" ClosedDate="2016-02-10T16:33:15.167" ContentLicense="CC BY-SA 3.0" />
  <row Id="22035" PostTypeId="1" CreationDate="2016-02-05T13:55:20.033" Score="2" ViewCount="2561" Body="&lt;p&gt;I have an ordered list of (2d-)points that are forming a (not axis aligned) rectangle and I'd like to recover that rectangle. Approximations like a minimal enclosing rectangle can't be used so that I'm looking for an algorithm that minimizes the least-squares norm or the hausdorff distance.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rectangle can be a square or rather narrow. The set contains several hundred points and the points cover at least three sides. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current idea looks like that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initial estimation of center, rotation and lenghts from doing a PCA on the point set. (Rotation estimation will be especially bad for squares). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the points will be assigned to the closest side and the mean (rmse?) error recorded. This can be used as an optimization value for a following&#xA;iterative LM-Optimization. &lt;/p&gt;&#xA;" OwnerUserId="19085" LastEditorUserId="19085" LastEditDate="2016-02-05T19:23:26.283" LastActivityDate="2016-02-05T22:05:57.240" Title="Fitting a rectangle to a point set" Tags="|computational-geometry|regression|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="22037" PostTypeId="1" CreationDate="2016-02-05T17:09:35.483" Score="0" ViewCount="512" Body="&lt;p&gt;I am facing a problem with MPI (Fortran). I have a really big matrix at each node and they differ at different nodes. At some point of my calculations, each node needs the matrix from all other nodes, so that the MPI communication is between every node. Each node is a sender at the same time it is also a receiver. The matrix size is already too big to allow me to simply collect data from all other nodes. Currently what I am doing is to combine &lt;code&gt;MPI_ISEND&lt;/code&gt; with &lt;code&gt;MPI_RECV&lt;/code&gt;, but I think this might not be the best solution, thus, I would like to consult you experts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, the matrix on each node is of same size, but they are not replica of each other, their elements are different at different nodes. At some point of time, each node needs the matrix from all other nodes. Thus, each node shall send its own matrix at the same receive data from all other nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be really grateful to you valuable suggestions and maybe solutions. &lt;/p&gt;&#xA;" OwnerUserId="7630" LastEditorUserId="9667" LastEditDate="2016-03-13T23:41:57.577" LastActivityDate="2018-03-06T18:43:57.297" Title="efficient MPI collective non-blocking communication" Tags="|parallel-computing|fortran|mpi|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="22038" PostTypeId="1" AcceptedAnswerId="22050" CreationDate="2016-02-05T20:34:59.793" Score="5" ViewCount="113" Body="&lt;p&gt;A few days ago, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21993/integration-of-differential-equation-with-orthogonality-constraint&quot;&gt;I asked how constraints like&lt;/a&gt; $A^T A = I$ can be implemented if one wishes to integrate differential equations of the form $\dot{A}=f(A,t)$. Kirill was so kind to point out that a projection step on the Stiefel manifold after one integration step (for example after one Runge Kutta step) is sufficient to get the correct solution. This projection $\pi(A)$ can be found via the polar decomposition of the matrix $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is, why a projection after one full RK step is sufficient. If one uses, for example, a 4th order RK method then intermediate sampling points are used. These intermediate points do not necessarily fulfil the constraint $A^T A = I$. Why do I not have to reorthogonalise the matrix A after one intermediate step but only after a full RK4 step? Is this still a valid 4th order method or is there an additional error introduced because the constraint is not fulfilled at the intermediate sampling points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Papers like the one from &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.140.434&quot; rel=&quot;nofollow noreferrer&quot;&gt;Higham (1997)&lt;/a&gt; seem to indicate that it is perfectly fine to use any one-step method between the projections on the manifold and no accuracy is lost. I do not see the reason why this is true.&lt;/p&gt;&#xA;" OwnerUserId="18543" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-02-06T08:41:08.320" Title="Projection on Stiefel manifold after integration step" Tags="|ode|constrained-optimization|runge-kutta|constraints|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22040" PostTypeId="2" ParentId="22035" CreationDate="2016-02-05T22:05:57.240" Score="1" Body="&lt;p&gt;Some thoughts, to be refined with computational trials:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Start with an estimate for the center $X_C,Y_C$, either by PCA as suggested in the OP, or by using the center of a bounding box or circle (fairly cheap computations, and I think we only need an estimate of the center whose error is small relative to the dimensions of the rectangle).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sweep through the data points to calculate angle and distance relative to the estimated center, $(\theta_i,r_i)$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This derived data can be fitted as a periodic function $\theta$ vs. $r$.  Intuitively the &quot;local&quot; maximum values occur at the corners of the rectangle and would be equal if all four corners were well-represented in the data (and thus would be global maxima in the ideal case).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To help with any &quot;missing&quot; side for the rectangle in the data, we reflect all the points through the estimated center, thus achieving more complete coverage of the perimeter.  It seems that we don't need to actually compute those reflections, since the effect on the angle vs. distance derived data is simply to impose a period of $\pi$ rather than $2\pi$ on the model.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Fit (by least-squares) a two segment trigonometric model to the angle vs. distance data, using the half-interval of length $\pi$ that begins and ends at one of the maxima.  (The other intervening maximum separates the two segments.)  That is, an edge of a rectangle with Cartesian equation $y = mx+b$ would have an angle vs. distance relation of the form:  $$ r = \frac{b}{\sin \theta - m \cos \theta} $$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Analyze the errors in fitting for systematic bias in the data points from the two half-intervals.  That is, if the data points in one half-interval for one segment have predominantly positive errors and data points in the other half-interval for the same segment have predominantly negative errors, this is an indication our estimated center needs to move toward the original points that are now giving positive errors (actual radius from estimated center minus the fitted model's predicted radius).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rinse. Lather. Repeat.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="651" LastActivityDate="2016-02-05T22:05:57.240" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22041" PostTypeId="1" CreationDate="2016-02-05T22:29:08.523" Score="3" ViewCount="93" Body="&lt;p&gt;For the past several weeks I was attempting to adapt Lax-Wendroff or some similar scheme for polar coordinates. The process was complicated due to me being unable to find step-by-step derivations of used schemes, so i had to guess the process for cartesian coordinates and to repeat the same for polar ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I managed to do it for MacCormack's scheme, and it exploded even on somewhat smooth, shockwave-less test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to geometry of my system and because of other equations that govern it (I'm trying to add hydrodynamics into 2d plasm discharge model), I was trying to stick to polar coordinates. Hovewer, now I'm quite certain that I won't be able to find ready-for-use scheme for euler equations in polar coordinates. To derive one on my own seems implausible, too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus my questions arise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there ready-to-use robust solvers for inviscid compressible Euler equations with sharp gradients in polar coordinates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are none, how do I attempt to derive one? Which scheme should I try to derive?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it even feasible to try to stick to polar coordinates? Or should I solve Euler equations in cartesian coordinates, and then interpolate values for the rest of my model? If so, should I use uniform grid with cropped cells on the border, or should I use some finite-volume method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I should move away from polar coordinates, which scheme should I use? There are quite a lot of schemes, there are different types of error correction mechanisms, etc - but can someone with experience suggest something?&lt;/p&gt;&#xA;" OwnerUserId="17130" LastActivityDate="2016-02-05T22:29:08.523" Title="Solving compressible inviscid Euler equations with shockwaves in polar coordinates" Tags="|hyperbolic-pde|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22042" PostTypeId="1" CreationDate="2016-02-05T22:29:10.203" Score="2" ViewCount="294" Body="&lt;p&gt;I have a 2D triangle which deforms with each vertex moving by some small ($\sin(x) \approx \tan(x) \approx x$) displacement vector. The displacement of any point in the triangle is linearly interpolated from the displacements at the vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I find the rotation angle of any point in the triangle?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want the rotation angle as a linear function of the vertex displacements and I feel that it should be (again, small displacements only). I tried the following approach but got bogged down in the enormous expressions that appear (hundreds or thousands of terms) and I'm not sure if it's even a correct way. Is there an easier way or does this simplify somehow?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1) Find the deformation gradient $F$ (2x2 matrix) using the displacements and the derivatives of the interpolation functions.&lt;/li&gt;&#xA;&lt;li&gt;2) Find the polar decomposition $F=QS$ where $Q$ is orthogonal and $S$ is symmetric&lt;/li&gt;&#xA;&lt;li&gt;3) Treat $Q$ as a rotation matrix and extract the rotation angle from it&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I also tried more intuitive geometrical ways like averaging the rotation angles of all 3 vertices about the point but they don't seem to give correct looking results.&lt;/p&gt;&#xA;" OwnerUserId="11650" LastEditorUserId="9667" LastEditDate="2016-02-08T21:44:54.437" LastActivityDate="2016-02-10T07:38:10.060" Title="Angle of rotation at a point in a deformed triangle" Tags="|linear-algebra|finite-element|geometry|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22043" PostTypeId="1" CreationDate="2016-02-06T01:04:04.967" Score="0" ViewCount="95" Body="&lt;p&gt;I am having trouble indexing correctly the below statement in C inside a function and then returning it as a pointer. The returning part should not be confusing - hopefully - however the indexing is a nuisance. P can be just about any real number greater or equal to 0 and $O_{j-n}^{k-m}$ is supposed to be a pointer to a 2D array of size: [p+1][2*p+1]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, this condition holds: $O_{j-n}^{k-m}=0$ for $|k-m|&amp;gt;(j-n) $. This is no big deal since a simple if else statement can take care of it, however the nested loops, pointer and indexing situation stated below is the real deal. Any ideas how to take care of this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j=0}^p\sum_{k=-j}^{+j}\sum_{n=0}^{j}\sum_{m=-n}^{n}O_{j-n}^{k-m} $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, so far I am stuck with this algorithm which is a nightmare trying to debug. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (j=0; j&amp;lt;=p; j++){&#xA; for (k=-j; k&amp;lt;=j; k++){&#xA;    for (n=0; n&amp;lt;=j; n++){&#xA;     for (m=-n; m&amp;lt;=n; m++){&#xA;      if ( abs(k-m) &amp;lt;= (j-n) ){&#xA;        printf(&quot;\nIndex[%d][%d]:\t %d&quot;, j-n, k-m, (2*p+1)*(j-n) + k+j-m-n );&#xA;      }&#xA;    }&#xA;  }&#xA;}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="18387" LastEditorUserId="18387" LastEditDate="2016-02-07T15:16:48.157" LastActivityDate="2016-02-07T15:16:48.157" Title="Indexing Nested Loops in C" Tags="|computational-physics|c|fast-multipoles|" AnswerCount="1" CommentCount="3" ClosedDate="2016-02-17T19:50:25.283" ContentLicense="CC BY-SA 3.0" />
  <row Id="22044" PostTypeId="2" ParentId="11203" CreationDate="2016-02-06T05:52:14.083" Score="0" Body="&lt;p&gt;As Geoff says, stiffness is more pragmatic in its definition and use than mathematical. A good practical way to define it is that a stiff problem is one where simple explicit (i.e. Runge-Kutta) methods need a really small time step to be stable, so your simulation never seems to finish. One way to see this is if the eigenvalues of the Jacobian are large, but if you can't tell by eye, just run the simulation with an adaptive Runge-Kutta method (i.e. ode45). If it's taking for ever and seems to get stuck, that means you have to move to a stiff solver (usually more work). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[If you don't have an easy adaptive method to implement, just try a Runge-Kutta method with some reasonable step sizes. If the solution keeps blowing up, it's unstable, which means go to implicit methods]. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-02-06T05:52:14.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22045" PostTypeId="2" ParentId="5088" CreationDate="2016-02-06T06:18:00.140" Score="0" Body="&lt;p&gt;A good way to intuit this is that the &quot;average size of a Wiener process step $dW$&quot; is $\sqrt{\Delta t}$. This is because the variance is $\Delta t$. So if you think about the time scale for anything associated with $b$ as $\sqrt{\Delta t}$ the formula follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, that's just intution. Kloden's &lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-3-662-12616-5&quot; rel=&quot;nofollow&quot;&gt;Numerical Solution of Stochastic Differential Equations&lt;/a&gt; and &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9781611972016&quot; rel=&quot;nofollow&quot;&gt;Taylor Approximations for Stochastic Partial Differentiations&lt;/a&gt; do this more rigorously.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-02-06T06:18:00.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22046" PostTypeId="2" ParentId="18987" CreationDate="2016-02-06T06:21:02.390" Score="1" Body="&lt;p&gt;You can also generate noise spectrums as the solutions to stochastic differential equations. This gives an iterative method for generating the noise timestep by timestep as opposed to solving for the whole spectrum first (i.e. you just solve for one more step of the SDE, and this is your next noise term). &lt;a href=&quot;http://arxiv.org/pdf/1002.4316v1.pdf&quot; rel=&quot;nofollow&quot;&gt;This paper gives a good treatment of some SDEs for your noise type&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-02-06T06:21:02.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22047" PostTypeId="2" ParentId="8925" CreationDate="2016-02-06T06:37:55.003" Score="8" Body="&lt;p&gt;If you want to try a bunch of different solvers, give Julia's &lt;a href=&quot;https://jump.readthedocs.org/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;JuMP modeling framework&lt;/a&gt; a try. It lets you write your model as a JuMP model, and then switch out the solvers with one line of code. For example, for MILP problems you can choose from the Bonmin, Cbc, Couenne, CPLEX, GLPK, Gurobi, and MOSEK solvers. Because of this, if you write it in JuMP, you can just try all the solvers that Geoff mentioned and see what works without having to write a bunch of code. Your own personal tests will be the best source of knowledge for what the fastest algorithms are for your problems.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-02-06T06:37:55.003" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22048" PostTypeId="2" ParentId="20942" CreationDate="2016-02-06T06:50:50.157" Score="3" Body="&lt;p&gt;Generally nonlinear optimization is hard to parallelize because its stepping algorithm is very sequential. GPUs are slower than CPUs so you only get a speedup if you have your problem as something that is highly parallel (i.e. thousands of threads). Thus I wouldn't expect much of a speedup (or any, it might usually be slower) by putting the nonlinear solver on the GPU. That's most likely why no one has done it well, and most people wouldn't try it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if your objective and constraint functions can be calculated in a highly parallel (or vectorized) fashion, you can get a massive speedup by solving your objective/constraint functions on the GPU. This is a common way to use GPUs with nonlinear optimization since it uses GPU acceleration on the hardest (and most parallel) part of the code, and thus is probably the most efficient.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-02-06T06:50:50.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22049" PostTypeId="1" CreationDate="2016-02-06T08:37:32.560" Score="2" ViewCount="131" Body="&lt;p&gt;I consider ordinary wave equation&#xA;$$ u_{tt} - u_{xx} = 0 $$ &#xA;with initial conditions&#xA;$$u(x, 0) = \exp (-2 x^2) \\&#xA;u_t(x, 0) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this problem I approximate $u_{xx}$ with 4-th order difference scheme and find time evolution with symplectic 4-th order method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use following parameters:&#xA;$$x_{min} = -500 \\&#xA;x_{max} = 500 \\&#xA;dx = 0.5 \\&#xA;dt = 0.1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that, for example, in the point $x=0$ this algorithm perfectly works for $t &amp;lt; 1000$ (note that this value equals $x_{max} - x_{min} = 2 x_{max}$ But then evolution looks quite strange. Here is a picture of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4gogv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4gogv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to fix boundary value manually (at every step set $u(x_{min}, t) = u(x_{max}, t) = 0, \ u_t (x_{min}, t) = u_t (x_{max}, t) = 0 $ )&#xA;and also tried to use &quot;exponential suppression&quot; to avoid radiation from boundaries. &#xA;The $u_{xx}$ approximation with finite difference is used only for points which are more than 2 steps distant from boundaries to avoid problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the problem lies in the presence of boundaries, because such symplectic algorithm works fine for harmonic oscillator (where there are no boundaries) for any time value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPD:&lt;/strong&gt; The stencil I use for partial derivative:&#xA;$$u_{xx} (x) = \frac{-u(x-2) + 16 u(x-1) - 30 u(x) + 16 u(x+1) - u(x+2)}{12 (dx)^2} $$&lt;/p&gt;&#xA;" OwnerUserId="19192" LastEditorUserId="19192" LastEditDate="2016-02-08T18:14:53.517" LastActivityDate="2016-02-08T18:14:53.517" Title="Symplectic integration of PDE" Tags="|pde|numerical-analysis|numerics|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="22050" PostTypeId="2" ParentId="22038" CreationDate="2016-02-06T08:41:08.320" Score="4" Body="&lt;p&gt;As a matter of fact, if you project each Runge-Kutta stage you will (in general) &lt;strong&gt;reduce the method to first order accuracy&lt;/strong&gt;; this effect is similar to the order reduction that can occur due to boundary condition implementations.  To explain it briefly, notice that the first stage of the method is just a forward Euler step, which gives a first-order accurate approximation of the solution.  Hence it will be $O(\Delta t^2)$ away from the constraint manifold, and by projecting you introduce an $O(\Delta t^2)$ error that won't go away.  For a more detailed explanation see &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0916072&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So projecting only after a full step is probably best.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-02-06T08:41:08.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22052" PostTypeId="2" ParentId="5473" CreationDate="2016-02-06T12:55:06.897" Score="1" Body="&lt;p&gt;What you want seems to be a convolution; I think the quickest way for achieving it would be the &lt;code&gt;numpy.convolve&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your may have to fix the indices according to your exact needs but I think you would like to try something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;a = [1, 2, 3, 4, 5]&#xA;k = [2, 4, 6, 8, 10]&#xA;&#xA;result = np.convolve(a, k*a[::-1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18093" LastActivityDate="2016-02-06T12:55:06.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22053" PostTypeId="1" CreationDate="2016-02-06T15:21:10.033" Score="2" ViewCount="364" Body="&lt;p&gt;While reading &lt;a href=&quot;http://stanford.edu/~boyd/papers/pdf/admm_distr_stats.pdf&quot; rel=&quot;nofollow&quot;&gt;Boyd's paper on ADMM&lt;/a&gt; I encountered an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem.&lt;/strong&gt; &lt;em&gt;Minimize $f(u) + g(v)$ subject to $Au + Bv = c$, where $f$ and $g$ are closed, proper, convex and differentiable.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denote by $\lambda$ the dual variable. On page 18 of the paper, it is stated that the necessary and sufficient optimality conditions for the above problem are: &#xA;\begin{align}&#xA;Au^* + Bv^* &amp;amp;= c\\&#xA;\nabla f(u^*) + A^T\lambda^* &amp;amp;= 0\\&#xA;\nabla g(v^*) + B^T\lambda^* &amp;amp;= 0.&#xA;\end{align}&#xA;(I changed the names of the variables, sorry for the inconvenience.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider the following special case: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $$u=(x,t)\in \mathcal{X}\times\mathcal{T}, \quad v=(y,z)\in \mathcal{Y}\times\mathcal{Z}, \quad f(u) = h(x) + at, \ a\neq 0$$ &#xA;where &lt;strong&gt;$\mathcal{X},\mathcal{Y},\mathcal{Z},\mathcal{T}$ are closed and convex&lt;/strong&gt;.&#xA;and $$A = \begin{bmatrix}&#xA;-1 &amp;amp; 0\\ &#xA; -1&amp;amp; 0&#xA;\end{bmatrix}, B = \begin{bmatrix}&#xA;1 &amp;amp; 0\\ &#xA;0 &amp;amp; 1&#xA;\end{bmatrix},c=0,\lambda = (\lambda_y,\lambda_z)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(i.e. the problem becomes &lt;em&gt;Minimize&lt;/em&gt; $h(x) + at + g(y,z)$ &lt;em&gt;subject to&lt;/em&gt; $y=x,z=x$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second optimality condition above becomes&#xA;$$\begin{bmatrix}&#xA;\nabla h(x^*)\\ &#xA;a&#xA;\end{bmatrix} + \begin{bmatrix}&#xA;-1 &amp;amp; -1\\ &#xA;0 &amp;amp; 0&#xA;\end{bmatrix}\begin{bmatrix}&#xA;\lambda_y^*\\ &#xA;\lambda_z^*&#xA;\end{bmatrix} = 0$$&#xA;Or equivalently&#xA;$$\begin{bmatrix}&#xA;\nabla h(x^*) - \lambda_y^* - \lambda_z^*\\ &#xA;a&#xA;\end{bmatrix}  = 0,$$&#xA;which can not be achieved because $a\neq 0$!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your discussions!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; The motivation of the above example comes from the following problem I encountered in practice:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimize $(a+c+d)^Tx_1 + b^Tx_2$ subject to $(x_1,x_2)\in \mathcal X$, where $\mathcal X$ is a closed convex set defined by $$\mathcal X = \left\{(x_1,x_2)\middle| \begin{matrix}&#xA;(x_1,x_2) \text{ satisfy some condition } (1), \\ &#xA;x_1 \text{ satisfies some condition } (2),\\ &#xA;x_1 \text{ satisfies some condition } (3)&#xA;\end{matrix}\right\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denote &#xA;\begin{align}&#xA;\mathcal X_1 &amp;amp;= \left\{(x_1,x_2)\mid &#xA;(x_1,x_2) \text{ satisfy condition } (1)\right\}\\&#xA;\mathcal Y &amp;amp;= \left\{x_1 \mid &#xA;x_1 \text{ satisfies condition } (2)\right\}\\&#xA;\mathcal Z &amp;amp;= \left\{x_1 \mid &#xA;x_1 \text{ satisfies condition } (3)\right\}&#xA;\end{align} &#xA;Suppose that the original optimization problem is very hard, but the problems of minimizing any quadratic function over (only) one of the set $\mathcal X_1, \mathcal Y, \mathcal Z$ are easy (minimizing over the intersection of any two sets among them is hard as well). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can thus decompose the problem using ADMM by reformulating it as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimize $a^Tx_1 + b^Tx_2 + c^Ty +d^Tz$ subject to $y=x_1, z=x_1, (x_1,x_2) \in \mathcal X_1, y\in\mathcal Y, z\in\mathcal Z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we apply ADMM now, if $\mathcal X_1, \mathcal Y, \mathcal Z$ are closed and convex? What are the optimality conditions then?&lt;/p&gt;&#xA;" OwnerUserId="7695" LastEditorUserId="7695" LastEditDate="2016-02-07T14:02:29.873" LastActivityDate="2016-02-07T14:02:29.873" Title="Understanding the conditions for which ADMM can be applied" Tags="|optimization|convex-optimization|admm|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="22055" PostTypeId="1" CreationDate="2016-02-06T22:23:04.690" Score="0" ViewCount="247" Body="&lt;p&gt;I have been looking to do a 3D mesh for a shallow tunnel FE analysis.&#xA;The output of mesh node orientation along the element should be counter clock wise direction.&#xA;How and which material helps me best as I am a programming beginner?&lt;/p&gt;&#xA;" OwnerUserId="19199" LastEditorUserId="6910" LastEditDate="2016-02-07T14:15:29.613" LastActivityDate="2016-02-07T17:39:00.677" Title="How can I generate 3D unstructured mesh on MATLAB" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22056" PostTypeId="2" ParentId="22053" CreationDate="2016-02-07T04:41:58.207" Score="2" Body="&lt;p&gt;If I understand the proposed optimization problem correctly, then the only constraint is the equality constraint $Au + Bv=c$ (and $\mathcal{X} = \mathcal{T} = \mathcal{Y} = \mathcal{Z} = \mathbb{R}$ or $\mathbb{C}$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case I think there is no optimal point. To see this, note that by the nature of the matrix $A$, the inequality constraint doesn't care about the value of $t$, so one can choose $t$ to be whatever you want independent of all the other variables. On the other hand, the objective function takes the form,&#xA;$$f(u) + g(v) = at + [\text{stuff not depending on t}].$$&#xA;Thus one can make the objective as large or small as desired by taking $t \rightarrow \pm \infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if I interpreted what you wrote wrong and the sets $\mathcal{X}$, $\mathcal{T}$, etc., are intervals rather then the whole real line, then you could enforce these requirement by either (1) using additional Lagrange multipliers $\nu$ to enforce inequality constraints, or (2) building the constraint into the objective by adding a penalty term that is zero in the feasible set and +infinity outside. Either way, a more detailed analysis is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details on first order conditions for inequality constrained optimization problems, see section 5.5.3 on the KKT optimality conditions on page 243 in Boyd's book,&#xA;&lt;a href=&quot;https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf&quot; rel=&quot;nofollow&quot;&gt;https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2016-02-07T04:41:58.207" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="22058" PostTypeId="2" ParentId="21609" CreationDate="2016-02-07T13:42:25.597" Score="2" Body="&lt;p&gt;Using angles is a very bad idea of storing rotations. They are ambiguous and not always consistently defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Store your pose matrix as an augmented matrix of rotation and translation: $P=[R | t]$. In that convention, a point $\mathbf{x} \in \mathbb{R}^3$ is transformed via the operation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{x}' = P\mathbf{x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$P$ is a $4x3$ matrix, while $\mathbf{x}$ is a column vector of 3D coordinates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let $\mathbf{P}_t$ denote the pose at iteration/time $t$, and $\mathbf{P}_\delta$ is the pose of the current update (Solution from the Kabash, Horn, Point-to-Plane etc). This time they are $4x4$ matrices assembled as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{P}=&#xA;\begin{bmatrix} R &amp;amp; t\\ \mathbf{0} &amp;amp; 1\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your new pose is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{P}_{t+1}=\mathbf{P}_\delta \mathbf{P}_t&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way you could keep track of the matrix, along the minimization process, without having the need of tracking $R$ and $t$ separately. If your case is 2D, you could still do the same thing, only with reduced size matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could see an example of using this convention &lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/47152-icp-registration-using-efficient-variants-and-multi-resolution-scheme&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2016-02-08T21:57:32.453" LastActivityDate="2016-02-08T21:57:32.453" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="22059" PostTypeId="2" ParentId="22043" CreationDate="2016-02-07T13:57:14.343" Score="2" Body="&lt;p&gt;First in your last sum, you have a mistake in the limits (I guess the upper limit would be $n$). Here is the corrected statement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j=0}^p\sum_{k=-j}^{+j}\sum_{n=0}^{j}\sum_{m=-n}^{n}O_{j-n}^{k-m} $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not the case, please update. Next, in the second sum, I believe that upper limit is $k$, not $j$, for the proper nest operation (other-wise it's not a nest, and you could decouple the loops). Correct me if I'm wrong:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j=0}^p\sum_{k=-j}^{+j}\sum_{n=0}^{k}\sum_{m=-n}^{n}O_{j-n}^{k-m} $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, in all cases, your indices have impossible to occur conditions, which might be unnecessary, so, if I re-write, this is the same thing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j=0}^p\sum_{k=0}^{+j}\sum_{n=0}^{k}\sum_{m=-n}^{n}O_{j-n}^{k-m} $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (j=0; j&amp;lt;=p; j++){&#xA;     for (k=0; k&amp;lt;=j; k++){&#xA;        for (n=0; n&amp;lt;=k; n++){&#xA;         for (m=-n; m&amp;lt;=n; m++){&#xA;          if ( abs(k-m) &amp;lt;= (j-n) ){&#xA;            printf(&quot;\nIndex[%d][%d]:\t %d&quot;, j-n, k-m, (2*p+1)*(j-n) + k+j-m-n );&#xA;          }&#xA;        }&#xA;      }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2016-02-07T13:57:14.343" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22060" PostTypeId="2" ParentId="20974" CreationDate="2016-02-07T14:04:39.257" Score="0" Body="&lt;p&gt;In the case of the sphere there is a simple and computationally efficient method in the case the triangles are not far from being equilateral (thanks for the correction in the comments). Suppose we are working on the unit sphere, so the center of the sphere is at the origin. Then in order to find the triangle which contains a certain point $P$ we construct the exterior normals to each triangle and search the one which makes the minimal angle with the vector $\vec{OP}$. The triangle corresponding to the closest normal will contain the point.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="3548" LastEditDate="2016-02-08T21:50:03.360" LastActivityDate="2016-02-08T21:50:03.360" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="22061" PostTypeId="2" ParentId="22055" CreationDate="2016-02-07T17:39:00.677" Score="2" Body="&lt;p&gt;For tetrahedral elements, the easiest way, if not the only way, in MATLAB is with &lt;a href=&quot;http://persson.berkeley.edu/distmesh/&quot; rel=&quot;nofollow&quot;&gt;DistMesh Package&lt;/a&gt;. It is really easy to use.&#xA;I do not know the direction of orientation. However, that should be easy to fix in a few lines of code.&lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2016-02-07T17:39:00.677" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="22062" PostTypeId="1" CreationDate="2016-02-07T20:30:17.387" Score="2" ViewCount="90" Body="&lt;p&gt;Assuming I have a length-$n$ real vector $x$ and have already computed its Fourier transform $\hat x$ (in time $O(n\log n)$), I would like to compute the Fourier transform of $y = x + \delta x$, where $\delta x$ is a small perturbation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does there exist a method for computing the Fourier transform of $y$ in less than $O(n\log n)$ time? Perhaps in linear time (e.g., asymptotically constant number of vector arithmetic operations)? In other words, is it possible to take advantage of knowing an &quot;approximate&quot; Fourier transform ($\hat x$) of the input data ($x+\delta x$)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have in mind the example where $x$ and $y$ are consecutive timesteps in solving a system of ODEs, where $|y-x|=O(\delta t)$ is small, but there might be other examples I don't know of.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-02-07T20:57:50.960" LastActivityDate="2016-02-07T20:57:50.960" Title="Does there exist a Fourier transform algorithm for perturbed data?" Tags="|linear-algebra|fourier-analysis|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="22065" PostTypeId="2" ParentId="20974" CreationDate="2016-02-08T16:36:50.557" Score="3" Body="&lt;p&gt;A generic way of looking up the element in which a point of given coordinates may be is to sort them into a quadtree (octree in 3D). The leaves of the tree will contain only element having an intersection with a given range of coordinates (in your case this could be ranges in $(\theta, \phi)$), the number of elements per leaf is kept smaller than a small enough constant $n$ (around 10 to 20), and the test of whether the point belongs to an element is performed on all of these. This gives a cost of order $n + log (N/n)$ for an $N$-element triangulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your question is about how to test optimally that the point belongs to a triangle, please state it explicitly.&lt;/p&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2016-02-08T16:36:50.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22069" PostTypeId="2" ParentId="22042" CreationDate="2016-02-09T04:14:39.220" Score="1" Body="&lt;p&gt;Your approach seems to be correct, but you shouldn't be getting caught up in &quot;hundreds to thousands&quot; if you're doing it in a reasonable way. In case you're caught up in the mechanics of it, here's the key parts of your process laid out in a little bit more detail. I'm assuming that you can compute the deformation gradient, since you haven't provided any details about how you're interpolating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Compute deformation gradient $\mathbf{F}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Compute the polar decomposition: $\mathbf{F} = \mathbf{RU}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2a) Find eigenvalues and eigenvectors of $\mathbf{C} = \mathbf{F}^T \mathbf{F}$. This tensor has the same eigenvectors $\{\mathbf{v}_1, \mathbf{v}_2\}$ as $\mathbf{U}$, and its eigenvalues $\{\lambda_1^2, \lambda_2^2\}$ are the squares of the eigenvalues of $\mathbf{U}$.&#xA;You seem to be working with 2x2 deformation gradients, so this should again be pretty easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2b) Reconstruct $\mathbf{U}$ using the spectral theorem (&lt;a href=&quot;https://en.wikipedia.org/wiki/Spectral_theorem&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Spectral_theorem&lt;/a&gt;).&#xA;$$&#xA;\mathbf{U} = \sum_i \lambda_i \;\mathbf{v}_i \otimes \mathbf{v}_i &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Solve for $\mathbf{R} = \mathbf{U}^{-1}\mathbf{F}$. Again, 2x2 matrices make this pretty easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Profit (aka: post-process $\mathbf{R}$ however you'd like).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @nicoguaro has noted in a comment above, $\mathbf{F}$ will be constant within the triangle if you have linear interpolation functions, so you would only need to compute this angle of rotation multiple times if you use higher-order interpolation.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2016-02-09T04:14:39.220" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="22070" PostTypeId="2" ParentId="16107" CreationDate="2016-02-09T07:13:19.093" Score="1" Body="&lt;p&gt;The latest version Mathematica solves this problem very well:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Clear[u,v,nx,eqn1,eqn2,ic1,ic2,vars,eqn1,eqn2,ic1,ic2,vars,sol,PDEsol,order,npts];&#xA;npts = 100; order=12; &#xA;nx = Range[0,1,1/(npts-1)];&#xA;{ddx,d2dx2} = Map[NDSolve`FiniteDifferenceDerivative[Derivative[#],nx, &quot;DifferenceOrder&quot; -&amp;gt; order][&quot;DifferentiationMatrix&quot;]&amp;amp;,{1,2}] ;&#xA;u = Array[u$[#][t]&amp;amp;,npts]; v = Array[v$[#][t]&amp;amp;,npts];&#xA;&#xA;eqn1=Thread[D[u,t]==d2dx2.u+2];&#xA;eqn2 = Thread[d2dx2.v+ u==0];&#xA;&#xA;eqn1[[1]]=u[[1]]==ddx[[1]].u+2t;&#xA;eqn1[[-1]]=u[[-1]]==2t-ddx[[-1]].u;&#xA;eqn2[[1]]=v[[1]]==ddx[[1]].v;&#xA;eqn2[[-1]]=v[[-1]]==-ddx[[-1]].v;&#xA;&#xA;ic1=Thread[u==0]/.t-&amp;gt;0;&#xA;ic2=Thread[v==0]/.t-&amp;gt;0;&#xA;&#xA;vars = Flatten[{u, v}] /. x__[t] :&amp;gt; x ; &#xA;PDEsol = First[NDSolve[{eqn1, eqn2, ic1, ic2}, vars, {t, 0, 30}]];&#xA;time = Flatten@First[vars[[1]] /. PDEsol];&#xA;{usol, vsol} = &#xA;  Flatten[Table[&#xA;      Transpose[{nx, ConstantArray[ti, npts], # /. PDEsol}] /. &#xA;       t -&amp;gt; ti, {ti, time[[1]], time[[2]], 0.1}], 1] &amp;amp; /@ {u, v};&#xA;&#xA;&#xA;&#xA;  GraphicsRow[ ListPlot3D[#, PlotRange -&amp;gt; All, PlotTheme -&amp;gt; &quot;Classic&quot;, &#xA;    Mesh -&amp;gt; {50, 50}, Lighting -&amp;gt; &quot;Neutral&quot;] &amp;amp; /@ {usol, vsol}, &#xA; ImageSize -&amp;gt; 800]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S8kAX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S8kAX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6693" LastActivityDate="2016-02-09T07:13:19.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="22071" PostTypeId="1" CreationDate="2016-02-09T08:29:32.257" Score="3" ViewCount="2287" Body="&lt;p&gt;I am simulating unsteady flow around a circular cylinder (using FLUENT). I am facing the following problem. Kindly please help me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OBJECTIVE:&lt;/strong&gt;&#xA;To find out the Reynolds number at which vortex shedding actually starts during a flow around a circular cylinder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; This study is already available in literature. Literature data show that vortex shedding starts at $\text{Re} = 46$ for a circular cylinder. I just want to redo that work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt;&#xA;I ran unsteady simulations from 45 to 50. But, I am not observing any vortex shedding. But, using the same setup, if I run simulations for $\text{Re} &amp;gt; 60$, I am observing vortex shedding and matching values of drag and St. Hence, I feel that I am making mistake in choosing the time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method used:&lt;/strong&gt;&#xA;I came across several posts related to this and many methods to calculate the time step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) From Strouhal number (0.2 for circular cylinder- approx)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{Sr} (0.2) = (\text{frequency} \times \text{diameter}) / \text{velocity}$.&#xA;From the above equation find the frequency. The total time period, $T = 1/\text{frequency}$. Hence, time step $\Delta t = T/25$ (approx).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) From CFL condition&#xA;$\Delta t = C_m X \Delta x / \text{velocity}$&#xA;where, $C_m = 1$ (approx) and $\Delta x$ = minimum cell size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) From domain length&#xA;$\Delta t = \text{domain length} / (20 \times \text{velocity})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried, all the above methods and I am not able to see vortex shedding between $\text{Re} = 45$ and $50$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FLOW SCENARIO:&lt;/strong&gt;&#xA;Unsteady flow around a circular cylinder of $\text{dia} = 0.01\text{m}$ in the Reynolds number range of 45 to 50. The working fluid is water with $\text{viscosity} = 0.001003 \text{kg}/\text{ms}$ and $\text{density} = 998.2 \text{kg}/\text{m}^3$.&lt;/p&gt;&#xA;" OwnerUserId="19225" LastEditorUserId="17671" LastEditDate="2016-02-11T13:51:08.877" LastActivityDate="2018-09-11T03:15:58.033" Title="Time step size for transient simulations of vortex shedding" Tags="|fluid-dynamics|time-integration|vorticity|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="22072" PostTypeId="2" ParentId="22042" CreationDate="2016-02-09T10:19:47.387" Score="0" Body="&lt;p&gt;A much simpler formula is&#xA;$$&#xA;\theta = {1 \over 2} \nabla \times u&#xA;$$&#xA;Where $u$ is displacement. The derivatives of $u$ are obtained multiplying elements of the strain-displacement matrix (&lt;a href=&quot;http://www.uniroma2.it/didattica/TCM/deposito/2-D_elements.pdf&quot; rel=&quot;nofollow&quot;&gt;p8 here&lt;/a&gt;) by components of $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; clarifications:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u$ is a 3D field but since the question is 2D, one component is always zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \times$ means curl&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\theta$ is a vector whose magnitude is the rotation angle. In 2D, it must be parallel to the plane normal so only that component is the angle and the other two are zero.&lt;/p&gt;&#xA;" OwnerUserId="11650" LastEditorUserId="11650" LastEditDate="2016-02-09T23:57:41.063" LastActivityDate="2016-02-09T23:57:41.063" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22074" PostTypeId="1" CreationDate="2016-02-09T12:07:38.363" Score="3" ViewCount="1285" Body="&lt;p&gt;I have a CFD code that I want to run on an HPC cluster. Of course, before it can be run, it first needs to be compiled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best option between compiling on the head node and compiling on one of the compute nodes ? The hardware specifications of head and compute nodes are not necessarily identical, and sometimes all compute nodes aren't the same either : can that change the optimisation behaviour/efficiency of the compiler significantly ? What is the common best practice ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in case of compiling on computation nodes, I would be waiting for the workload manager (e.g. slurm) to grant &quot;RUN&quot; state to the job. The submission script would then compile the software and then run it. Doing otherwise would not be nice to the other users…&lt;/p&gt;&#xA;" OwnerUserId="10018" LastActivityDate="2018-10-18T12:35:27.923" Title="HPC : should I compile on the head node or on a compute node?" Tags="|hpc|compiling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="22075" PostTypeId="1" CreationDate="2016-02-09T12:37:44.767" Score="1" ViewCount="33" Body="&lt;p&gt;I have a question that relates to the classical &quot;longest common subsequence&quot; problem. I'll give the background to the problem, but you could skip to the formulation below if you like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's think of genes on a bacterial chromosome as beads on a string - atomic, unique objects, with an ordering.&#xA;I'm deliberately ignoring some things, and want to look at a simplified model. So let's say the chromosome is the interval&#xA;[1,1000], and there are 1000 genes, one at each integer in the interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose I have a set of 100 chromosomes, each from a different individual in a population, each with their 1000 genes,&#xA;but the ordering is allowed to differ.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, I have 100 different permutations of the integers 1..1000.&#xA;Let's look at an example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 2 3 4 5 6 7 8 9 10&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 8 9 2 3 4 5 6 7 10&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 4 3 2 7 8 9 10 5 6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 4 3 2 10 8 9 5 6 7&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I look at these, I see two things. First, that the first two both contain&#xA;1 2 3 4 5 6 7 10 &#xA;as a subsequence.&#xA;Second, that the final two contain&#xA;1 4 3 2 8 9 5 6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to construct  an algorithm for splitting a list of permutations into subsets like that,&#xA;such that within the subsets, all permutations contain a common subsequence. &#xA;A trivial solution is to make each permutation a subset of size 1 of course, so what I really want&#xA;is to minimize the number of subsets, and maximize the length of the subsequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a lot of previous work on finding the longest common subsequence of a fixed set (&lt;a href=&quot;https://en.wikipedia.org/wiki/Longest_common_subsequence_problem&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Longest_common_subsequence_problem&lt;/a&gt;) but I can't find any works addressing how to partition your strings to maximise the length of common subsequence within subsets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if this is a classical/solved problem also?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS I posted a (slightly less well formulated) version of this on mathOverflow and the organisers sent me over here.&lt;/p&gt;&#xA;" OwnerUserId="19212" LastActivityDate="2016-02-09T12:37:44.767" Title="Clustering by shared subsequence" Tags="|combinatorics|clustering|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22077" PostTypeId="1" AcceptedAnswerId="22079" CreationDate="2016-02-09T16:08:02.313" Score="12" ViewCount="1967" Body="&lt;p&gt;I know that incompressible and compressible flow solvers are specifically designed to solve different types of problems with different fluid properties/flow conditions.  Clearly, among the advantages of using incompressible flow solvers for modeling problems with incompressible fluids is that the energy equation can be neglected, thus reducing the number of variables and equations that need to be solved.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm curious to know about the accuracy of compressible flow solvers in the limit as the fluid properties and flow conditions tend toward being incompressible.  Do compressible flow solvers tend to fail as the fluid/flow being modeled becomes more and more incompressible?  Or do compressible flow solvers perform equally well independently of the compressibility of the fluid/flow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize that this question is a bit broad and may very well depend upon the characteristics of the problem being modeled.  If such is the case, please help me to understand what factors I need to keep in mind when determining the applicability of using a compressible flow solver where otherwise an incompressible flow solver would suffice.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2016-02-09T16:39:43.783" LastActivityDate="2016-02-10T08:25:55.937" Title="Can compressible flow solvers be used to solve incompressible flow?" Tags="|fluid-dynamics|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22078" PostTypeId="2" ParentId="22042" CreationDate="2016-02-09T18:30:44.507" Score="2" Body="&lt;p&gt;Given a small (infinitesimal) 2D displacement field $u = (u_x, u_y)$ the infinitesimal counterclockwise rotation angle $\theta$ is simply&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\theta = \frac 12  \left ( \frac{\partial u_y}{\partial x} - \frac{\partial u_x}{\partial y} \right )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, for $n$ dimensional displacement fields $u\in\mathbb{R}^n$ it can be shown that the infinitesimal rotation is described by the skew symmetric part of the Jacobian:&#xA;$$&#xA;\mathbf{J} = \frac{\partial(u_1,\dots,u_n)}{\partial(x_1,\dots,x_n)}&#xA;$$&#xA;i.e. &#xA;$$&#xA;\mathbf{\Omega} = \frac12 \left ( \mathbf{J} - \mathbf{J}^T \right)&#xA;$$&#xA;This is linked to the well known fact that $\mathrm{so}(n)$ (the skew symmetric square real matrices) form the Lie algebra associated to the special orthogonal group $\mathrm{SO}(n)$ (real orthogonal matrices with determinant 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already noted in another answer, $\frac{\partial u_i}{\partial x_j}$ can be easily computed from the derivatives of the shape functions and the nodal displacement values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For linear shape functions, the Jacobian is constant. This means that $\Delta u = \mathbf{J} \Delta x$, where $\Delta u$, $\Delta x$ are the differences of displacement and coordinates, respectively, evaluated at two distinct points inside the element. If you choose two nodes, then $\Delta u$ is the difference of the nodal displacements, while $\Delta x$ is the vector representing the corresponding edge of the element. For triangles and tetrahedra is it therefore possible (explicit formula left as an exercise) to easily compute the requested quantity directly from nodal coordinates and nodal displacements.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2016-02-10T07:38:10.060" LastActivityDate="2016-02-10T07:38:10.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22079" PostTypeId="2" ParentId="22077" CreationDate="2016-02-09T19:19:24.470" Score="16" Body="&lt;p&gt;The compressible equations are hyperbolic in nature, i.e., they have a finite speed of sound. In practice, this implies that you have to take a time step that is proportional to something like the mesh size divided by the speed of sound. (This is, in its essence, the CFL condition you have to satisfy for stability when using explicit solvers, and for accuracy if you use implicit solvers.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you go to the incompressible limit, then this implies that the speed of sound goes to infinity. With the usual hyperbolic solvers, this means that you need to let the time step go to zero -- i.e., you will not make a lot of progress in your simulations. Consequently, compressible solvers are poorly suited for incompressible problems, and when used for such problems almost always treat them as &lt;em&gt;slightly&lt;/em&gt; compressible problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put another way, there are &lt;em&gt;fundamental&lt;/em&gt; differences between the compressible and incompressible equations, even though one is the limit of the other. This implies that one is well advised to use different codes that tailor to these differences.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-02-09T19:19:24.470" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="22080" PostTypeId="2" ParentId="21954" CreationDate="2016-02-09T19:32:17.240" Score="2" Body="&lt;p&gt;I have written an exhaustive answer on meshing here: &lt;a href=&quot;https://engineering.stackexchange.com/questions/449/meshing-of-complex-geometrical-domains/7326#7326&quot;&gt;https://engineering.stackexchange.com/questions/449/meshing-of-complex-geometrical-domains/7326#7326&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resultant mesh quality is more important than the technique used to create the mesh. Depending on the domain and type of analysis to be carried out, many different meshes can lead to the correct answer with reasonable computational cost. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get a correct answer from analysis, you need to ensure one more thing besides mesh quality which is mesh convergence. Mesh convergence is achieved when further refinement of mesh doesn't increase the accuracy of your result by a significant margin. So practically, your result becomes independent of the mesh and the analysis results are pretty much accurate given all other conditions are taken care of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: A graphic explanation of mesh convergence to be added.&lt;/p&gt;&#xA;" OwnerUserId="17443" LastEditorUserId="-1" LastEditDate="2017-04-13T12:51:48.613" LastActivityDate="2016-02-09T20:13:27.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22081" PostTypeId="2" ParentId="21599" CreationDate="2016-02-09T20:00:36.363" Score="0" Body="&lt;p&gt;You can try:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can use four-noded(quad) elements in place of all tria elements since it is a 2D domain and a lot of tria element will over-stiffen the domain.&lt;/li&gt;&#xA;&lt;li&gt;You can use a meshing program instead of comsol to manually control the size and shape of elements. This way you may be able to control the number of elements and nodes rather than automatically meshing it in comsol.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have a fairly detailed answer on meshing over &lt;a href=&quot;https://engineering.stackexchange.com/a/7326/2918&quot;&gt;here&lt;/a&gt; which you can refer to create a better mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: If you comment with your feedback after trying out manual meshing, I may be able to recommend something specific.&lt;/p&gt;&#xA;" OwnerUserId="17443" LastEditorUserId="-1" LastEditDate="2017-04-13T12:51:48.613" LastActivityDate="2016-02-09T20:00:36.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22083" PostTypeId="2" ParentId="22077" CreationDate="2016-02-10T06:32:14.020" Score="1" Body="&lt;p&gt;The incompressibility assumption is an approximation.  Thus compressible flow solvers -- which don't employ that approximation -- are more accurate but also more expensive.  A compressible solver will give you a perfectly good answer if applied to an &quot;incompressible&quot; problem (i.e. one where compressibility plays no significant role).  It will just take a ridiculously long time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same answer applies to any pair of models where one is a lower-cost approximation of the other.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-02-10T06:32:14.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22084" PostTypeId="2" ParentId="22077" CreationDate="2016-02-10T07:43:09.497" Score="1" Body="&lt;p&gt;&lt;strong&gt;The short answer is:&lt;/strong&gt; Yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now for the long answer.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the other answers point out, it is definitely possible but you would have to adjust your time step accordingly, which will make your simulation be extremely slow compared to if you were using an incompressible solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few years back, I was doing an internship for a private research center which had develop a very robust compressible problem. I was using it to solve an incompressible solver. In order to do it as &lt;em&gt;efficiently&lt;/em&gt; as possible (in terms of CPU time), I had to increase the velocity up to the incompressible limit (Mach $\approx 0.2$) and also increase the viscosity accordingly so the Reynolds number would remain unchanged: $ Re = \dfrac{v D}{\nu} $  &lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="16329" LastEditDate="2016-02-10T08:25:55.937" LastActivityDate="2016-02-10T08:25:55.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22085" PostTypeId="1" AcceptedAnswerId="22086" CreationDate="2016-02-10T08:00:05.767" Score="6" ViewCount="188" Body="&lt;p&gt;I am solving one relatively simple problem with two different algorithm: one which uses brute force while the other is optimized. For a variety of reasons I actually cannot show the codes here but I do not think this is of importance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how I am currently comparing the two algorithms (I use Fortran):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CALL CPU_TIME(t1)&#xA;DO pp = 1,10000&#xA;    **Algorithm_1**&#xA;END DO&#xA;CALL CPU_TIME(t2)&#xA;&#xA;CALL CPU_TIME(t3)&#xA;DO pp = 1,10000&#xA;    **Algorithm_2**&#xA;END DO&#xA;CALL CPU_TIME(t4)&#xA;&#xA;PRINT*, &quot;Algorithm 1 time = &quot;, (t2-t1)/(pp-1)&#xA;PRINT*, &quot;Algorithm 2 time = &quot;, (t4-t3)/(pp-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives me a rough idea of the speed up factor. But my problem is that it only gives a &lt;strong&gt;rough&lt;/strong&gt; idea. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For one case, the average speedup factor is about 460. Sometimes I get 534, other times 380. And that is obtained by running the same executable multiple times in a row. This is the most extreme case; I usually get a range of $ \pm$ 5 %, which still bothers me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a robust way to compare two algorithms? Or are discrepancies in the results expected?&lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="123" LastEditDate="2016-02-23T03:45:31.710" LastActivityDate="2016-02-23T03:45:31.710" Title="How to compare runtimes of two algorithms in a reproducible way" Tags="|fortran|performance|reproducibility|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22086" PostTypeId="2" ParentId="22085" CreationDate="2016-02-10T09:19:08.807" Score="14" Body="&lt;p&gt;To make more robust comparisons (on linux), you can :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) On Intel CPUs the turbo overclocks your CPU. This is controlled by the temperature of the CPU, so it can behave differently from one run to the other. On Linux, you can block the frequency of the CPU as follows. For example, for 2.4GHz:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  echo 1 &amp;gt; /sys/module/processor/parameters/ignore_ppc&#xA;&#xA;  for x in /sys/devices/system/cpu/cpu[0-3]/cpufreq/;do &#xA;    echo 2400000 &amp;gt; $x/scaling_max_freq&#xA;  done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) As long as there is free memory, linux caches your last I/Os. If your program needs to allocate a lot of memory, the first allocation will take time as it will empty the I/O caches to provide some free memory. To make your allocation times more predictable, empty your I/O caches&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;echo 3 &amp;gt; /proc/sys/vm/drop_caches &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;3) Your processes can migrate from one CPU core to another. At each migration, you lose the L1 and L2 caches, so you will experience more cache misses. Also, the migration takes some time. To remove this overhead, bind your process to CPU cores:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;export OMP_PROC_BIND=true&#xA;taskset -c 0-3 ./a.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;4) Before measuring the CPU time, make a little bit of warm up&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) Maybe that algorithm 1 runs faster when it is preceded by algorithm 2, or vice versa. To avoid such a bias, you can interleave measurements of algorithms 1 and 2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  ! DO ALL POSSIBLE INITIALIZATION HERE&#xA;&#xA;  NREP=10000&#xA;  NLOOP=100&#xA;&#xA;  time1 = 0.d0&#xA;  time2 = 0.d0&#xA;&#xA;  DO k=1,NLOOP&#xA;&#xA;    ! WARM UP&#xA;    DO pp = 1,NREP/10&#xA;        **Algorithm_1**&#xA;    END DO&#xA;&#xA;    !MEASURE 1&#xA;    CALL CPU_TIME(t1)&#xA;    DO pp = 1,NREP&#xA;        **Algorithm_1**&#xA;    END DO&#xA;    CALL CPU_TIME(t2)&#xA;&#xA;    time1 = time1 + t2-t1&#xA;&#xA;&#xA;    ! WARM UP&#xA;    DO pp = 1,NREP/10&#xA;        **Algorithm_2**&#xA;    END DO   &#xA;&#xA;    ! MEASURE 2&#xA;    CALL CPU_TIME(t3)&#xA;    DO pp = 1,NREP&#xA;        **Algorithm_2**&#xA;    END DO&#xA;    CALL CPU_TIME(t4)&#xA;&#xA;    time2 = time2 + t4-t3&#xA;&#xA;  ENDDO&#xA;&#xA;  PRINT*, &quot;Algorithm 1 time = &quot;, time1/(dble(NREP)*dble(NLOOP))&#xA;  PRINT*, &quot;Algorithm 2 time = &quot;, time2/(dble(NREP)*dble(NLOOP))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;6) Make sure you are the only user on the machine&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7) Close all other background applications if you are on a desktop computer (firefox, etc)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8) Turn off the network&lt;/p&gt;&#xA;&#xA;&lt;p&gt;9) For Multi-threaded applications, make sure your threads are bound to different physical CPU cores (check in &lt;code&gt;/proc/cpuinfo&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="17681" LastEditorUserId="17681" LastEditDate="2016-02-21T19:32:43.717" LastActivityDate="2016-02-21T19:32:43.717" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22087" PostTypeId="1" CreationDate="2016-02-10T09:20:55.723" Score="0" ViewCount="406" Body="&lt;p&gt;Problem statement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to solve a problem statement using C# as programming language. In the problem system for an input (double/decimal) say $H_i$, the output generated is a form of dataset containing number of parameters ($F_i$, $P_i$ and $T_i$). I somehow have to filter out only those entries in the data set which would satisfy the following conditions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$F_i &amp;gt; F_\min$, where $F_\min$ is some constant;&lt;/li&gt;&#xA;&lt;li&gt;$P_i &amp;gt; P_\min$, where $P_\min$ is some constant; and&lt;/li&gt;&#xA;&lt;li&gt;$T_i &amp;lt; T_\max$, where $T_\max$ is some constant.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there an efficient algorithm I could use in such cases where I could zero in on an optimal set of values for $H_i$ for which the output parameter values are well within the constraints. Also I thought using Genetic Algorithms in this case makes sense but somehow I'm not able to formulate and derive a fitness function for my requirement&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to approach to solve such problem is truly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kindly do not downvote this question as vague.&lt;/p&gt;&#xA;" OwnerUserId="19239" LastEditorUserId="9667" LastEditDate="2016-02-10T16:23:30.910" LastActivityDate="2016-02-10T16:23:30.910" Title="Optimizing multiple output parameters for a given input" Tags="|optimization|algorithms|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="22090" PostTypeId="1" AcceptedAnswerId="22093" CreationDate="2016-02-10T15:36:46.723" Score="5" ViewCount="1242" Body="&lt;p&gt;In the the Finite Element Method (FEM), we attempt to obtain the Weak Form of the described equation. I understand that this is an attempt to reduce the &lt;strike&gt;order&lt;/strike&gt; regularity of the equation, but what are the actual benefits and disadvantages of using the Weak Form? What does regularity mean in this sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've checked out other questions on this topic, the math presented there is higher than my level. If I'd been a better mathematician, I suppose I might have derived my answer. Basically, an answer understood by engineers would be nice.&lt;/p&gt;&#xA;" OwnerUserId="9506" LastEditorUserId="9506" LastEditDate="2016-02-10T16:08:00.417" LastActivityDate="2016-02-12T02:05:35.540" Title="FEM: Obtaining the Weak Form" Tags="|finite-element|weak-solution|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="22091" PostTypeId="2" ParentId="22087" CreationDate="2016-02-10T16:06:00.597" Score="1" Body="&lt;p&gt;Actually what are you trying to do is to solve a nonlinear optimization problem. As far as I understood, you have a dataset and according to the particular input applied, then you get an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to approach this problem would be to consider a grid of points for &lt;em&gt;Hi&lt;/em&gt;&#xA;with respect to evaluate the parameters coming from the dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can formulate your fitness function such that in case the resulting parameters do not respect the constraints, then the cost of your objective will go to 10^53 for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you have evaluated all the grid points, you can choose the best solution and refine the grid to have a better result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider that in this case you don't have any guarantee of the optimality of the obtained solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you can investigate the usage of the Simulated annealing approach to solve your problem (&lt;a href=&quot;https://en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Simulated_annealing&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="14817" LastActivityDate="2016-02-10T16:06:00.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22092" PostTypeId="1" CreationDate="2016-02-10T18:16:29.773" Score="2" ViewCount="1012" Body="&lt;p&gt;I trying to do a simulation where there are two quantities, $\Delta\theta(x,t) = \theta(x,t) - \theta_{o}$ and $\Delta\nu(x,t) = \nu(x,t) - \nu_{o}$. These quantities are chemical concentrations resultant from reactions. There is no significant convective flow only diffusive flow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quantities $\Delta\theta $ $\Delta\nu$ are related to each other by a reaction reaction, this reaction is a source term for both dependent on the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, both variables have source terms that depend on themselves, i.e. if $\Delta\nu$ or $\Delta\theta\neq0$ there are reactions that try to drive them to equilibrium. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Because of this dependence, when I'm setting up my equations, should I put my source terms within the matrix (and not on the b vector)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is FVM still conservative if I have variable dependent sources? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Functional form of the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we don't get confused, let: $\Delta \theta = \theta - \theta_{o} \rightarrow \theta$, because $\theta_o$ is the equilibrium value and we're only interested in quantities away from equilibrium. Then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\delta \theta(t,x)}{\delta t} = \Delta D \Delta \theta(t,x) + Source(\theta,\nu)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice, that I explicitly show the Source depenence on $\theta$ and $\nu$. So when I write the $a_{p}$ term it contains, not just the diffusive terms from $a_E$ and $a_W$, but also from the source. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I solve it in phase space by replacing the time-domain $\theta$ with a frequency domain one. &lt;/p&gt;&#xA;" OwnerDisplayName="user19247" LastEditorDisplayName="user19247" LastEditDate="2016-02-10T22:51:51.473" LastActivityDate="2016-02-12T15:39:18.993" Title="Is the Finite Volume Method conservative when the source term depends on the variable?" Tags="|fluid-dynamics|finite-volume|computational-physics|computational-chemistry|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="22093" PostTypeId="2" ParentId="22090" CreationDate="2016-02-10T20:08:16.440" Score="7" Body="&lt;p&gt;The weak form of the PDE means that you have, in some sense, &quot;fewer constraints on the kinds of functions that you can use&quot; to construct a solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put it into perspective, recall that in FEM, you look for a solution of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j}u_j*\phi_j(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_j$ are the unknown scalar coefficients (which will be solved for) and $\phi_j$ are some known functions.  The main question is:  what kind of functions $\phi_j$ are we allowed to choose?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well... after going through a very long, tedious, and mathematically pedantic proof, we arrive at a very nice conclusion which is very convenient for engineers who want to implement FEM:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;All $\phi_j(x)$ &lt;strong&gt;&lt;em&gt;must&lt;/em&gt;&lt;/strong&gt; have a value of zero on the boundary.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the weak form of the PDE has a weak derivative of maximum order $k$, then it is sufficient that the functions $\phi_j(x)$ have continuity of order $k-1$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Condition #1 is very easy to understand:  $\phi_j(x)=0$ on all points along the boundary of the domain of your problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Condition #2 is not entirely obvious (also not 100% mathematically or pedantically correct either; just &quot;close enough for engineers' sakes&quot;). But condition #2 is the most useful part because it reveals the continuity constraints on $\phi_j$.  For example, take the poisson equation in weak form (assuming homogeneous dirichlet boundary conditions)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega k\nabla u \cdot \nabla v dx=\int_\Omega fv .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation has a weak derivative of maximum order k=1 because the gradient here is, effectively, a first order weak derivative (if the weak form had a laplacian operator $\nabla^2$, then k=2; etc...).  Thus, you can choose a function space such that $\phi_j(x)$ has continuity of order zero.  In other words, you can use functions that are continuous, but not necessarily smooth, across the entire domain.  This gives you a lot of flexibility in being able to construct a set of functions statisfying condtion 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does this matter at all?  Because, in FEM, we also seek a third desireable (albeit not absolutely necessary) condition:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;We want the stiffness matrix to be as sparse as possible.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This third condition matters because it implies that we should use functions $\phi_j$ that are not only zero on the boundary, but zero &quot;almost&quot; everywhere except in small subregions of the domain. Each subregion corresponding to a given $\phi_j$ should only overlap with a small number of other subregions corresponding to other functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can use this condition while exploiting with condition #2 to produce a desirable choice of functions $\phi_j$.  Lagrange polynomials are one such set of functions which satisfies all three conditions.  Of course, Lagrange polynomials are not the only choice, but are probably the simplest to understand from a beginners' perspective.  These functions are zero everywhere except in a small region where they are piecewise linear, quadratic, cubic, etc...  Because $\phi_j(x)$ are only non-zero in small regions for each given $j$, the integrals forming the stiffness matrix are also mostly zero and thus the stiffness matrix will also be very sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just want to caution you that I've watered down a lot of mathematics here and that Condition #2 is only &quot;approximately correct enough for an engineer's perspective&quot; and is not 100% mathematically correct.  I leave it to the abundance of mathematically inclined users on this site to point out how gross an overstatement condition #2 is in reality.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2016-02-10T22:11:30.823" LastActivityDate="2016-02-10T22:11:30.823" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22094" PostTypeId="1" CreationDate="2016-02-11T03:42:09.780" Score="0" ViewCount="5909" Body="&lt;p&gt;In presenting geochemical data, I would like to try a statistical method that presents the data in an ISOCON diagram. This method requires scaling all the data to be the same distance from the origin (i.e., normalizing so that the sums of squares = 1). Result: all data points lie along an arc of a circle centered on the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, data points (elements) are scaled down in such a way that the squared sum of values for altered (y-axis) and unaltered (x-axis) rock become unity, allowing the elements to plot along an arc of a circle centered on the origin at a uniform distance of 1 unit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anyone be helpful in explaining what it would mean and necessary steps to normalize my data so that the sum of squares = 1? This will be used on an x-y scatterplot. Which software would process this type of operation best?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I've calculated for each element for both the x-axis (unaltered) and y-axis (altered) material.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;mean;&lt;/li&gt;&#xA;&lt;li&gt;degrees of freedom;&lt;/li&gt;&#xA;&lt;li&gt;sum of squared errors;&lt;/li&gt;&#xA;&lt;li&gt;variance; and&lt;/li&gt;&#xA;&lt;li&gt;standard deviation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19253" LastEditorUserId="9667" LastEditDate="2016-02-11T15:45:59.087" LastActivityDate="2016-02-11T15:45:59.087" Title="Normalize data so that the sum of squares = 1" Tags="|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22095" PostTypeId="1" CreationDate="2016-02-11T03:48:22.027" Score="0" ViewCount="182" Body="&lt;p&gt;What does it mean: to &lt;strong&gt;force a line through the origin&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I interpret this to be making (forcing) the intercept as (0,0) in the regression procedure of an x-y scatterplot, but am not sure.&lt;/p&gt;&#xA;" OwnerUserId="19253" LastActivityDate="2016-02-11T07:58:06.420" Title="Force a line through the origin" Tags="|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22097" PostTypeId="2" ParentId="22037" CreationDate="2016-02-11T05:42:58.147" Score="1" Body="&lt;p&gt;MPI_ISEND and MPI_IRECV are commonly used for non-blocking communication between processes. If you want to sent data to all processes, collective operations viz. MPI_BCAST or MPI_IBCAST can be used.&lt;/p&gt;&#xA;" OwnerUserId="19254" LastActivityDate="2016-02-11T05:42:58.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22098" PostTypeId="1" AcceptedAnswerId="22113" CreationDate="2016-02-11T06:11:52.727" Score="1" ViewCount="2783" Body="&lt;p&gt;I have a model of a system which consists of diffusing reactants and intermediates. For each variable, $u_i$, the final representative equation looks like the standard reaction-diffusion form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial}{\partial t}u_i(x,t)=f_i(\bar{u}(x,t))-D\frac{\partial^2}{\partial x^2}u_i(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f_i$ is a non-linear function that denotes the reaction term. I am using the finite difference method (FDM) to discretize the spatial derivative with Neumann boundary conditions:&#xA; $$\frac{\partial u_i}{\partial x}{\bigg|}_{x=0, x=L} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the central difference formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_i''(x)\approx\frac{u_i(x-h) - 2u_i(x) + u_i(x+h)}{h^2}.$$&#xA;After reducing the PDE to an ODE, I am using MATLAB's &lt;code&gt;ode15s&lt;/code&gt; (RK4 based solver) to numerically integrate the equations in time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not worked on PDEs much and online resources on FDM (such as even &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_method#Example:_The_heat_equation&quot; rel=&quot;nofollow&quot;&gt;wikipedia&lt;/a&gt;), depict simultaneous discretization of the time derivative, also.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, the discretization shown in these cases, depict the simple Euler integration (either explicit or implicit). Even the Crank-Nicolson scheme is shown like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my approach correct (discretize spatial derivative and solve using an optimized ode solver)? Can there be numerical stability issues? &lt;/p&gt;&#xA;" OwnerUserId="14551" LastEditorUserId="14551" LastEditDate="2016-03-03T06:10:32.677" LastActivityDate="2016-03-03T06:10:32.677" Title="Numerical simulation of a reaction-diffusion system on MATLAB with finite difference discretization of spatial derivative" Tags="|matlab|pde|finite-difference|numerical-analysis|" AnswerCount="3" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="22099" PostTypeId="2" ParentId="22094" CreationDate="2016-02-11T06:17:25.317" Score="2" Body="&lt;p&gt;It's not entirely clear to me what step you struggle with. But for the sake of explanation, assume you have data $x_i, i=1...N$ and you want to compute the normalized data $y_i, i=1...N$ from the $x_i$. Then&#xA;$$&#xA;  y_i = \frac{x_i}{\sqrt{\sum_{j=1}^N x_j^2}}.&#xA;$$&#xA;The denominator is of course the same for every $i$, so you only have to compute it once.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-02-11T06:17:25.317" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22100" PostTypeId="2" ParentId="22098" CreationDate="2016-02-11T07:26:38.007" Score="0" Body="&lt;p&gt;As you said yo can discretize only the space derivative, thus obtainng a set of DAEs (differential-algebraic equations), or discretize both space and time in a way to obtain a set of AEs (algebraic-equations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in a comment, the MOL (method of lines) is a discretization approach which assumes to discretize only the spatial domain and leave the time as a countinuous one. The main advantage is that the time step is chosen automatically by the solver according to the enforcement of some error checking rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would implement a set of AEs, thus discretizing both space and time domains, then is up to you to choose the time step an/or implement an algorithm able to accept/reject a time step size according to some error-checking rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover in case that you would implement this latter formulation, in some cases the numerical scheme can be unstable (in literature there are many examples that show this particular behavior while solving the same set of equations with different numerical methods with fixed stepsize). Finally, when working with nonlinear PDEs (like the one you are using), for a AE implementation you need also to linearize the nonlinear terms around a working point in order to implement some kind of Newton-Raphson method to solve iteratively the nonlinearities. In any case there are many ways to do this : an efficient algorithm was developed by Professor Newman called BAND, which is also available in Fortran routines. This aspect would be automatically managed by the solvers of DAEs such as ode15i, ode23, SUNDIALS suite, DASSL etc.. &lt;/p&gt;&#xA;" OwnerUserId="14817" LastActivityDate="2016-02-11T07:26:38.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22101" PostTypeId="2" ParentId="22095" CreationDate="2016-02-11T07:58:06.420" Score="2" Body="&lt;p&gt;I think that what you are trying to do is to find a line passing through a set of data which is able to best fit that set of data. A Least Square approach, for instance, could be used for this purpose. Force a line to pass through the origin means that if you have the general affine equation for the line as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;y = mx +q&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;than the affine term $q$ is null, $q=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let assume that you have a set of data $Y \in \mathbb{R}^{N}$ evaluated in $X \in \mathbb{R}^{N}$. You want to find a relation between these dataset using a linear relation. You can describe this relation using a linear regression of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Y = \Phi \Theta&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where in general $\Theta \in \mathbb{R}^{N+1}$ is the set of unknowns parameters able to desribe the relation between $X$ and $Y$ while $\Phi \in \mathbb{R}^{N \times N+1}$ is a suitable matrix build starting from $X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let assume that $X$ is defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;X = [ x_1\ x_2\ \cdots\ x_N]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the matrix $\Phi$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Phi =&#xA;\left[&#xA;\begin{matrix}&#xA;x_1 &amp;amp; x_2 &amp;amp; \cdots &amp;amp; x_N &amp;amp; 1 \\&#xA;x_1 &amp;amp; x_2 &amp;amp; \cdots &amp;amp; x_N &amp;amp; 1 \\&#xA;\vdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;x_1 &amp;amp; x_2 &amp;amp; \cdots &amp;amp; x_N &amp;amp; 1 \\&#xA;\end{matrix}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as it is possible to see, the last column of $\Phi$ multiplies the last element of the unknowns $\Theta$, which is considered as an affine term (beacause of the $1$ value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to force the regression line to pass through the origin, just set to zero the last column or neglect it while building the $\Phi$ matrix (be careful because if you remove the last column of $\Phi$ then you have also to reduce the size of $\Theta$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have your $\Phi$ matrix you can solve a linear regression problem (for instance) with the Least Square approach. The objective is to find $\Theta$ able to minimize the following fitness function&#xA;$$&#xA;\|Y-\Phi \Theta\|_2^2&#xA;$$ &#xA;where the operator $\| \cdot \|_2$ represents the two-norm operator of an array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If no constraints are set, the analytical solution of $\Theta$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Theta = (\Phi^{T}\Phi)^{-1}\Phi^{T} Y&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="14817" LastActivityDate="2016-02-11T07:58:06.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22103" PostTypeId="1" AcceptedAnswerId="23203" CreationDate="2016-02-11T12:45:18.803" Score="1" ViewCount="39" Body="&lt;p&gt;I wrote a small simulated annealing library in C++. Right now is just a few classes, a toy project I want to use to test some ideas. But before I move on I want to be sure that it works. Meaning, that if I give it a difficult problem it will work. I won't be fast, or efficient, or whatever. I just want to be sure it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried simple examples, but I would like to benchmark it. Does anybody know of a set of problems to solve? Nonlinear and constrained.&lt;/p&gt;&#xA;" OwnerUserId="157" LastActivityDate="2016-02-24T13:54:37.053" Title="Benchmark an stochastic constrain solver" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22104" PostTypeId="2" ParentId="22092" CreationDate="2016-02-11T14:41:15.200" Score="1" Body="&lt;p&gt;The finite volume method is said to be conservative because you use a relation between surface integrals on the faces of the elements and volume integrals on the elements to construct the matrix to solve. From wikipedia: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;These terms are then evaluated as fluxes at the surfaces of each&#xA;  finite volume. Because the flux entering a given volume is identical&#xA;  to that leaving the adjacent volume, these methods are conservative&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Did you write your own fvm solver? or are you using a package? If you wrote your own you should be able to check that indeed &quot;the flux entering a given volume is identical to that leaving the adjacent volume&quot; because you are familiar with how you build your matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did partially write a fvm solver for turbulent flow like 10 years ago. From what I remember, if everything is done properly, I don't see why in this case your problem should not be conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FVM method is said to be conservative because the way the system is build. It involves using the divergence theorem on a small volume. And when all the math is laid out, you realize that indeed the system is conservative. I don't see why whether the source term depends on the variable or not is important, because you'll integrate as before. But again, to be sure, the best is write down the development of fvm on your equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my bet is that yes, it is still convervative. But to be sure we would have to see how you discretize the integrals and interpolate the average values on the faces of the volumes. If done properly, it should be conservative.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;EDIT: Let's expand this with some math. I'm going to follow (even with some copy/paste) the wikipedia article on the FVM because is available to all. Specifically, the general conservation law using this PDE:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;$ \dfrac{\partial \mathbf{u}}{\partial t} + \nabla\cdot \mathbf{f}(\mathbf{u}) = 0$&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here,  $\bf u$  represents a vector of states and $\bf f $  represents the corresponding flux tensor. Again we can sub-divide the spatial domain into finite volumes or cells. For a particular cell, $i$ , we take the volume integral over the total volume of the cell, $v _{i} $ , which gives,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;$ \int_{v_i} \dfrac{\partial \mathbf{u}}{\partial t} dv + \int_{v_i} \nabla\cdot \mathbf{f}(\mathbf{u})  dv = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now let's write the first integral in terms of the average value $\bar{u}$ of $u$ in that cell. That is, &#xA;$ \int_{v_i} \dfrac{\partial \mathbf{u}}{\partial t} = v_i  \dfrac{\partial \mathbf{\bar{u}}}{\partial t}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to solve the second integral, we use the divergence theorem as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \int_{v_i} \nabla\cdot \mathbf{f}(\mathbf{u})  dv = \oint_{s_i} \mathbf{f}(\mathbf{u}) \cdot \mathbf{\bar{n}} ds $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $s_i$ is the surface that covers the element $i$. That way we get to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ v_i  \dfrac{\partial \mathbf{\bar{u}}}{\partial t} + \oint_{s_i} \mathbf{f}(\mathbf{u}) \cdot \mathbf{\bar{n}} ds = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is when it gets tricky, and how you do it in your equation it may affect your result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now, imagine that you discretize a volume in triangles. Assign to the center the average value. You can use finite differences to get rid of the time derivative, and get the next value of $u$ using an explicit scheme. To calculate the surface integral, you can interpolate the value between the different points (where you have the average of $u$) at the surface. Wikipedia puts it like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Again, values for the edge fluxes can be reconstructed by&#xA;  interpolation or extrapolation of the cell averages. The actual&#xA;  numerical scheme will depend upon problem geometry and mesh&#xA;  construction. ...&#xA;   Finite volume schemes are conservative as cell averages change through&#xA;  the edge fluxes. In other words, one cell's loss is another cell's&#xA;  gain!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;from your equation, you have to figure out how to integrate the source term. From your comment, note that the second integral depends too on $u$. The issue is that you have a time derivative, meaning that in your linear system Ax=b, A will have the coefficients of the next value of $u$, while b will be computed using the current values of $u$. This is of course implementation dependent but I don't see other way to do it. Going back to your equation, (from this point on all is speculation because I don't know what your function is), let's assume that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{v_i} \mathbf{S}(\mathbf{u},v) = v_i \mathbf{S}(\mathbf{\bar{u}},v)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;if this is correct, then you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \dfrac{\partial \mathbf{\bar{u}}}{\partial t} = \dfrac{1}{v_i}\oint_{s_i} \mathbf{f}(\mathbf{u}) \cdot \mathbf{\bar{n}} ds + \mathbf{S}(\mathbf{\bar{u}},v)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be solve numerically. How? it depends on how you solve the time derivative, how you make the interpolation between the nodes to calculate the surface integrals. Note that you practically have the same problem as before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this conservative? well, how is your function source? is the above assumption valid? how do you construct your system? are you writing your code or using a library? But like I said, if things are done properly, and source is &quot;well behaved&quot;, my bet is yes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. Like I said, you have to go through your code. Take a good FVM book and go through the method. I hope this info has helped and pointed you in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="157" LastEditorUserId="157" LastEditDate="2016-02-12T15:39:18.993" LastActivityDate="2016-02-12T15:39:18.993" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22105" PostTypeId="1" AcceptedAnswerId="22106" CreationDate="2016-02-11T15:05:41.523" Score="18" ViewCount="14616" Body="&lt;p&gt;I am solving differential equations that require to invert dense square matrices. This matrix inversion consumes the most of my computation time, so I was wondering if I am using the fastest algorithm available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current choice is &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.inv.html#numpy.linalg.inv&quot;&gt;numpy.linalg.inv&lt;/a&gt;. From my numerics I see that it scales as $O(n^3)$ where n is the number of rows, so the method seems to be Gaussian elimination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations#Matrix_algebra&quot;&gt;Wikipedia&lt;/a&gt;, there are faster algorithms avaliable. Does anyone know if there is a library that implements these? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder, why isn't numpy using these faster algorithms?&lt;/p&gt;&#xA;" OwnerUserId="8329" LastActivityDate="2016-02-11T18:41:21.920" Title="Complexity of matrix inversion in numpy" Tags="|numpy|dense-matrix|inverse|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="22106" PostTypeId="2" ParentId="22105" CreationDate="2016-02-11T15:49:32.937" Score="25" Body="&lt;p&gt;(This is getting too long for comments...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll assume you actually need to compute an inverse in your algorithm.&lt;sup&gt;1&lt;/sup&gt; First, it is important to note that these alternative algorithms are not actually claimed to be &lt;em&gt;faster&lt;/em&gt;, just that they have better asymptotic complexity (meaning the required number of elementary operations grows more slowly). In fact, in practice these are actually (much) &lt;em&gt;slower&lt;/em&gt; than the standard approach (for given $n$), for the following reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The $\mathcal{O}$-notation hides a constant in front of the power of $n$, which can be astronomically large -- so large that $C_1 n^3$ can be much smaller than $C_2 n^{2.x}$ for any $n$ that can be handled by any computer in the foreseeable future. (This is the case for the Coppersmith–Winograd algorithm, for example.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The complexity assumes that every (arithmetical) operation takes the same time -- but this is far from true in actual practice: Multiplying a bunch of numbers with the &lt;em&gt;same&lt;/em&gt; number is much faster than multiplying the same amount of &lt;em&gt;different&lt;/em&gt; numbers. This is due to the fact that the major bottle-neck in current computing is getting the data into cache, not the actual arithmetical operations on that data. So an algorithm which can be rearranged to have the first situation (called &lt;em&gt;cache-aware&lt;/em&gt;) will be much faster than one where this is not possible. (This is the case for the Strassen algorithm, for example.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Also, numerical stability is at least as important as performance; and here, again, the standard approach usually wins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this reason, the standard high-performance libraries (BLAS/LAPACK, which Numpy calls when you ask it to compute an inverse) usually only implement this approach. Of course, there are Numpy implementations of, e.g., Strassen's algorithm out there, but an $\mathcal{O}(n^3)$ algorithm hand-tuned at assembly level will soundly beat an $\mathcal{O}(n^{2.x})$ algorithm written in a high-level language for any reasonable matrix size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;&lt;sup&gt;1&lt;/sup&gt; But I'd be amiss if I didn't point out that this is very rarely really necessary: anytime you need to compute a product $A^{-1}b$, you should instead solve the linear system $Ax=b$ (e.g., using &lt;code&gt;numpy.linalg.solve&lt;/code&gt;) and use $x$ instead -- this is much more stable, and can be done (depending on the structure of the matrix $A$) &lt;em&gt;much&lt;/em&gt; faster. If you need to use $A^{-1}$ multiple times, you can precompute a factorization of $A$ (which is usually the most expensive part of the solve) and reuse that later.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-02-11T15:49:32.937" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="22107" PostTypeId="1" AcceptedAnswerId="22119" CreationDate="2016-02-11T16:21:33.323" Score="2" ViewCount="474" Body="&lt;p&gt;Let $u_h$ be the finite element solution of a fourth order equation (like biharmonic equation), using polynomial degree two.&#xA;If the convergence rate of $u_h$ is $2$, what is the convergence rate of the second order derivatives of $u_h$? Is it possible they do not converge?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="1804" LastEditDate="2016-02-12T18:01:05.617" LastActivityDate="2016-02-12T18:02:55.983" Title="Convergence of the second derivative of the finite element solution" Tags="|finite-element|error-estimation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="22108" PostTypeId="1" CreationDate="2016-02-11T17:08:28.407" Score="3" ViewCount="778" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a graduate student and a researcher. I do not like to rely on commercial products due to cost and also because I feel more comfortable in using and extending existing tools based on languages such as Java, Python, C#, and R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I want to establish a base line of classic neural networks before I extend my work into deep learning. I consider the following essential for my baseline: ANNs for ARMA such as: Simple recurrent networks (SRN), Elman-Jordan (RNN); Time Delay Neural Networks (TDNN); and Gamma Memories. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a single open source tool that can do Simple Recurrent Networks, Elman - Jordan, Time Delay Neural Networks, and Gamma Memories. This will allow me to establish my baseline and then extend my work into deep learning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Attempts&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the only two tools I found so far are both commercial. One is MATLAB Neural Networks Toolbox and the other is a package called Neurosolutions. I found a very similar post &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5110/python-oss-alternatives-for-matlab-neural-network-toolbox-any-intercomparisons&quot;&gt;here&lt;/a&gt;, but the original poster was too general and I didn't see any good answers for my issue. My needs are very specific to the above problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I deeply thank everyone for reading this question and appreciate your kind assistance. &lt;/p&gt;&#xA;" OwnerUserId="19261" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-04-01T15:30:55.693" Title="Open source tool comparable to MATLAB Neural Network Tool Box?" Tags="|python|machine-learning|languages|java|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22109" PostTypeId="2" ParentId="22105" CreationDate="2016-02-11T18:41:21.920" Score="7" Body="&lt;p&gt;You should probably note that, buried deep inside the numpy source code (see &lt;a href=&quot;https://github.com/numpy/numpy/blob/master/numpy/linalg/umath_linalg.c.src&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/numpy/numpy/blob/master/numpy/linalg/umath_linalg.c.src&lt;/a&gt;) the inv routine attempts to call the dgetrf function from your system LAPACK package, which then performs an LU decomposition of your original matrix. This is morally equivalent to Gaussian elimination, but can be tuned to a slightly  lower complexity by using faster matrix multiplication algorithms in a high-performance BLAS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you follow this route, you should be warned that forcing the entire library chain to use the new library rather than the system one which came with your distribution is fairly complex. One alternative on modern computer systems is to look at parallelized methods using packages like scaLAPACK or (in the python world) petsc4py. However these are typically happier being used as iterative solvers for linear algebra systems than applied to direct methods and PETSc in particular targets sparse systems more than dense ones.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-02-11T18:41:21.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22110" PostTypeId="1" CreationDate="2016-02-11T18:59:37.627" Score="3" ViewCount="165" Body="&lt;p&gt;I haven't much experience with conservation laws, shocks, etc. After reformulating my wave equation to 1st order system (velocity-stress):&#xA;$$&#xA;\frac{\partial v}{\partial t} + A \frac{\partial v}{\partial x} + B \frac{\partial v}{\partial y} = 0&#xA;$$&#xA;where: $v=(v_{x},v_{y},\sigma_{x},\sigma_{y},\sigma_{xy}) \\ A = A(\rho, \mu, \lambda)\\ B = B(\rho, \mu, \lambda)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I put my Dirichlet BC to the first component (some constant vx) and obtained a stiff behavior. I've tried the same models with 2nd order equation and saw beautiful waves propagating. Here I get immediate propagation and loads of instabilities with explicit solvers (FD, RK4). The finer the mesh, the more high-frequency oscillations I get. Maybe I'm used to displacements and don't understand this formulation...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&#xA;Is there any way of solving this kind of problem with ordinary tools or should I bother with Riemann solvers, etc. ? My long-term plan is to extend it to DG. There's plenty of papers on Godunov scheme, etc. but I couldn't quite catch what's going on though...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Yes, it's a linear hyperbolic system. I used the term &quot;stiff&quot; in a more 'colloquial' manner. I can't find a stable explicit timestepping scheme for this system with Dirichlet BC. Even for extremely low dt (1e-20), way below CFL I get immediate waves in the whole domain. On the other hand, for implicit, the solution does not propagate correctly.&#xA;For instance, sinusoidal excitation u1 = y = sin(100*t) with 2nd order system looks like this (mesh is obviously too coarse but quite fine):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/diwa4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/diwa4.png&quot; alt=&quot;u&quot;&gt;&lt;/a&gt;&#xA;For 1st order system cosinusoidal excitation v1 = y' = 100*cos(100*t) like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HDadW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HDadW.png&quot; alt=&quot;v&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same mesh size, the same timestepping (Crank-Nicolson). I suppose this is not physically correct...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The wave does not propagate from the middle too:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CaxR1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CaxR1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19145" LastEditorUserId="123" LastEditDate="2016-02-12T16:15:11.847" LastActivityDate="2016-02-12T16:15:11.847" Title="My discretization of a wave equation in first-order form does not give correct solutions. What should I do?" Tags="|hyperbolic-pde|runge-kutta|wave-propagation|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="22111" PostTypeId="1" AcceptedAnswerId="23128" CreationDate="2016-02-11T21:24:14.900" Score="2" ViewCount="796" Body="&lt;p&gt;After running several calculations with numpy, I end with the mean vector and covariance matrix for a state vector. Is there a way with numpy or scipy to sample a random vector around this mean and covariance?&lt;/p&gt;&#xA;" OwnerUserId="19265" LastActivityDate="2016-02-15T13:45:54.490" Title="(numpy/scipy) Build a random vector given mean vector and covariance matrix" Tags="|python|numpy|scipy|random-sampling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22112" PostTypeId="2" ParentId="22090" CreationDate="2016-02-12T02:05:35.540" Score="3" Body="&lt;p&gt;It's precisely the opposite of what you had. You use the weak form to increase the regularity (or alternatively, allow you to solve problems with less regularity). Integration always smooths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intuition is actually quite simple: For &quot;most&quot; functions, if you take an integral, then its derivative exists (its derivative is the thing you just integrated!). Many times you might want to solve a PDE which doesn't necessarily have derivatives, so the idea is that you just solve the PDE for the integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common example deals with step functions. A step function is not differentiable, but in a sense its &quot;weak derivative&quot; is the Dirac delta (as in, the integral of a step function has a derivatives which integrate like a Dirac delta), so to make everything work you just take integrals everywhere. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can have a PDE for things which aren't strictly differentiable, and you just tell people &quot;well if you look at the integral it works&quot;. That's the weak form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Note: There are a lot more details. Example, Sobolev Spaces are a generalization of Lebesgue spaces. Since $L^2$ functions aren't functions but equivalence classes of functions which are equal almost everywhere according to some measure, they don't have point-wise values, and so they only make sense in terms of how they integrate (again by how integrals &quot;smoothen&quot;). Etc. But it's all the say idea: smoothen using the integral, and define the derivative via integration by parts.]  &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-02-12T02:05:35.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22113" PostTypeId="2" ParentId="22098" CreationDate="2016-02-12T02:38:50.030" Score="3" Body="&lt;p&gt;Systems biologist studying reaction-diffusion systems here. The method that you started with is probably the most common. In fact, I would say that it's the first thing to try for this type of problem. If you just let $A$ be the tridiagonal matrix (1,-2,1), then you can write the system as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ u_{i+1} = Au_{i} + f(u_{i})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{i}$ is the vector of all reactants and $f$ is in its vectorized form. This could then be thrown into any ODE solver. This is the Method of Lines (MOL) approach that FancyPants noted in a succinct form. For many reaction-diffusion problems this is sufficient. If your instability comes from stiff reaction equations (i.e. the non-spatial model is stiff), then using a stiff ODE solver in this form is usually sufficient. Not only that, but because of the highly vectorized form of this equation, you can easily make use of GPGPU/Xeon Phi/multi-node computing to &quot;brute force&quot; the solution (in MATLAB, this takes little/to no extra effort).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you have high diffusion constants (or your diffusion is a highly variable equation), you may not be able to get the system stable enough for the MOL method to work. If that's the case, there are two ways that you can go. One common way is the Crank-Nicholson method. A quick intuition is that it is simply a midpoint method in space and a midpoint method in time. Just about any computational PDEs text will outline the method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this will result in a large implicit system which you will have to solve using a nonlinear solver like Newton's method. The real problem is that your implicit equations are all coupled, meaning they cannot be solved in parallel which can hurt for large systems. Thus one of the state-of-the-art methods in this field are the &lt;a href=&quot;http://www.math.uci.edu/~qnie/Publications/pdf/pdf.24.pdf&quot; rel=&quot;nofollow&quot;&gt;Implicit Integration Factor Methods&lt;/a&gt;. For example, if you look at Equation 26 in the linked paper, if you were to compute your function at previous values as some large constant $C$, then you see that the implicit equation decouples to that it's independent at each point in space. This means you can run a nonlinear solver at each point in space (meaning a much smaller Jacobian for faster solutions (and you can easily for solve it pen/paper instead of numerically approximating it), and you can solve at each point in space in parallel/GPU/etc.) and get massive practical speedups over the Crank Nicholson Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary: first try the method of lines with ODE45, if that doesn't work try stiff solvers. Else go to Crank-Nicholson (if you have solve of that code lying around), but if you will be solving lots of these types of problems, try an Implicit Integration Factor Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: If you're looking at periodic problems, you may also want to try &lt;a href=&quot;http://www.mff.cuni.cz/veda/konference/wds/proc/pdf13/WDS13_116_m6_Rybar.pdf&quot; rel=&quot;nofollow&quot;&gt;Spectral methods&lt;/a&gt;. Also, I am assuming you're using a 1D system. If not, look at operator splitting methods (either ADI or for integration factor methods), or finite element methods.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2016-02-12T04:40:13.990" LastActivityDate="2016-02-12T04:40:13.990" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="22114" PostTypeId="1" CreationDate="2016-02-12T02:51:18.280" Score="1" ViewCount="123" Body="&lt;p&gt;I was wondering if anyone knows about the differences between the large HPC resources available to US researchers. I am currently using a startup allocation (through the NSF GRFP) on XSEDE but was thinking about helping my PI apply for a research allocation. However, I was wondering what the other options are. I have heard of Blue Waters, how does it differ from XSEDE in terms of resources, queue wait times, etc? Are there any other similar resources that I should consider?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(BTW, I am surprised that if I google Blue Waters vs XSEDE there's no nice blog post about this!)&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-03-13T03:45:22.387" Title="Comparison of US HPC Resources" Tags="|parallel-computing|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22115" PostTypeId="2" ParentId="22114" CreationDate="2016-02-12T03:14:54.153" Score="1" Body="&lt;p&gt;Blue Waters is a single system, and XSEDE is a consortium that has many systems available.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-02-12T03:14:54.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22117" PostTypeId="2" ParentId="22098" CreationDate="2016-02-12T10:53:48.547" Score="1" Body="&lt;p&gt;What you're doing is called &lt;strong&gt;the method of lines&lt;/strong&gt; and it is the most common way of discretizing an evolution PDE.  An example of how to do it stably for reaction-diffusion problems (written by me) is available &lt;a href=&quot;http://nbviewer.jupyter.org/github/ketch/AMCS252/blob/master/8_A_reaction-diffusion_model_for_pattern_formation.ipynb&quot; rel=&quot;nofollow&quot;&gt;here in Python&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2016-02-12T16:11:18.157" LastActivityDate="2016-02-12T16:11:18.157" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22119" PostTypeId="2" ParentId="22107" CreationDate="2016-02-12T17:57:30.180" Score="2" Body="&lt;p&gt;&lt;strong&gt;Yes&lt;/strong&gt;, for several reasons.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;First, it is instructive to look at how one usually proves convergence rates. In the standard setting, where your discrete approximation $u_h\in V_h$ is in the same space $V$ as the true solution $u$, the error $\|u-u_h\|_V$ in the natural space $V$ (where you have continuity and coercivity of the bilinear form) is bounded by the interpolation error of $u$ by functions in $V_h$ (this is &lt;em&gt;Céa's lemma&lt;/em&gt;). So all you have to do is to get an estimate of the interpolation error $\|u- I_h u\|_V$. If $V=H^l$, the space of $l$ times weakly differentiable functions, and $V_h$ is the space of piecewise polynomials of degree $k$ on a given mesh, &lt;em&gt;and&lt;/em&gt; your exact solution is in $H^{k+1}$ for $k+1\geq l$, then you can show that&#xA;$$ \| u- I_h u \|_{H^l} \leq C h^{k+1-l} |u|_{H^{k+1}},\tag{1}$$&#xA;where $|u|_{H^{s}}$ is the $H^s$ seminorm of $u$, i.e., the $L^2$ norm of all partial derivatives of order $s$, and $C&amp;gt;0$ is a constant independent of $h$ and depending only on the geometrical properties of the mesh. (The proof works by first considering a reference element, where the error can be bounded by a constant, and then transforming the norms to each element and summing; the power of $h$ in the estimate comes purely from how the different $H^s$ norms scale under affine transformations.) This means that the smoothness of the true solution limits the polynomial degree you can use to get better convergence. (You can always use higher degrees, of course, but you won't get better accuracy from them.) &#xA;You can use different methods (e.g., Aubin-Nitsche lemma) to trade a lower order error norm for higher powers of $h$, i.e., an estimate of the form $(1)$ for the error $\|u-u_h\|_{H^l}$ usually holds also for smaller values of $l$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to your specific case. The natural regularity of the biharmonic equation is $H^2$, since the weak form involves second derivatives. You are also using piecewise quadratic polynomials. This means $l=k=2$, such that you would need to have the true solution in $H^3$, which is more than the natural regularity, and hence does not necessarily hold for the test problem you are solving. (But $u\in H^2$ is enough to get from $(1)$ a convergence order of $\mathcal{O}(h^2)$ in $H^0=L^2$, which is what you observe. You should also get $\mathcal{O}(h)$ in $H^1$, i.e., for the first derivatives, which is a useful test.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rule of thumb:&lt;/strong&gt; You lose one order of convergence for each derivative, and you gain one order for each polynomial degree, up to a point determined by the regularity of the exact solution you are trying to approximate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;However, you are not in the standard setting: Functions in $H^2$ are  continuously differentiable across element boundaries (apply the usual argument to the first derivatives), which is not the case for the standard piecewise polynomial spaces (which are only continuous across boundaries). Hence, $(1)$ doesn't apply in this case. Intuitively, since you are not enforcing the correct continuity across element boundaries, you might get the function values right (enough), but not the derivatives -- and especially not the second derivatives. (Here's where the test with the first derivatives comes in handy.) Instead, you also need an &quot;interior&quot; penalty method to enforce the required continuity of $H^2$ functions across element boundaries (such as &lt;em&gt;discontinuous Galerkin methods&lt;/em&gt;; an alternative would be &lt;em&gt;mixed methods&lt;/em&gt;). But how to do this would be a new question.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-02-12T18:02:55.983" LastActivityDate="2016-02-12T18:02:55.983" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22121" PostTypeId="1" AcceptedAnswerId="22122" CreationDate="2016-02-13T09:41:35.930" Score="0" ViewCount="370" Body="&lt;p&gt;For a system $\mathbf{x=Da}$, there exist a lot of algorithms to estimate sparse vector $\mathbf{a}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to know the big-O mathematical complexity of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) orthogonal matching pursuit (OMP) both with fixed sparsity and error tolerance criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) L1-magic (which is based on interior point methods) algorithms. &lt;/p&gt;&#xA;" OwnerUserId="17949" LastActivityDate="2016-02-13T14:59:27.037" Title="Mathematical Complexity of Sparse Solvers" Tags="|linear-solver|complexity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="22122" PostTypeId="2" ParentId="22121" CreationDate="2016-02-13T12:56:45.640" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://dsp.rice.edu/sites/dsp.rice.edu/files/cs/FastL1.pdf&quot; rel=&quot;nofollow&quot;&gt;This paper&lt;/a&gt; and &lt;a href=&quot;http://ecovision.mit.edu/~sai/12S990/Yang10-SIAM.pdf&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt; give a good overview of the complexity of different methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find the feature sign search algorithm to be very fast, robust and useful. You could check it &lt;a href=&quot;http://ai.stanford.edu/~hllee/softwares/nips06-sparsecoding.htm&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, along with the paper :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Efficient sparse coding algorithms&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Honglak Lee, Alexis Battle, Rajat Raina, and Andrew Y. Ng.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;NIPS 2006&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2016-02-13T14:59:27.037" LastActivityDate="2016-02-13T14:59:27.037" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="22123" PostTypeId="1" AcceptedAnswerId="22126" CreationDate="2016-02-13T15:37:23.103" Score="3" ViewCount="267" Body="&lt;p&gt;I need to implement a structural analysis code, and I turn to you for advice. My needs are simple:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Library must be integrable in a C++ code&lt;/li&gt;&#xA;&lt;li&gt;I want to express a weak form, without manual intervention&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The second point leaves libraries like &lt;code&gt;deal.II&lt;/code&gt; out of my scope. I've tried &lt;code&gt;FEniCS&lt;/code&gt;, but it is a nightmare to make it work with Xcode, I am constantly finding discrepancies between console and Xcode, and I need to debug my code!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any alternatives?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="19279" LastActivityDate="2016-04-16T11:49:06.443" Title="FEM libraries with weak forms" Tags="|finite-element|c++|libraries|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="22124" PostTypeId="1" AcceptedAnswerId="22125" CreationDate="2016-02-13T22:37:08.137" Score="3" ViewCount="52" Body="&lt;p&gt;I have a conic section in the real projective plane.  This is represented by its real symmetric 3×3 matrix.  I verify that the conic section is real and non-degenerate by computing the eigenvalues of the matrix, finding that it's of full rank and indefinite, and follow a different code path if it isn't.  The conic could be very close to singular though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can find points of the conic section by intersecting it with a line, which is easy using the matrix, or by finding a tangent incident on a point, which is easy using the inverse of the matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to define an arbitrary but consistent cyclic orientation of the conic, such that for any point on the conic, I can get a nonzero vector tangent to the conic and directed in some consistent direction along the conic.  What is a good way to do this?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like a way that is numerically stable in all cases, and is not too difficult to implement.  I can use a matrix library, so the method could contain matrix operations or other matrix decompositions. &lt;/p&gt;&#xA;" OwnerUserId="5372" LastActivityDate="2016-02-14T00:14:49.803" Title="Find a consistent cyclic orientation on a conic section" Tags="|linear-algebra|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22125" PostTypeId="2" ParentId="22124" CreationDate="2016-02-14T00:14:49.803" Score="2" Body="&lt;p&gt;The tangent vector $v$ at $x\in\mathbb{RP}^2$ to $x^{\top}Ax=0$ would satisfy $x^\top Av=0$, and also belong to the tangent space, so $x^\top v = 0$. So it must be the cross product &#xA;$$ v = x\wedge Ax, \qquad\text{or}\qquad v = -x\wedge Ax, $$&#xA;and you can pick freely one of these orientations.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-02-14T00:14:49.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22126" PostTypeId="2" ParentId="22123" CreationDate="2016-02-14T18:56:02.123" Score="5" Body="&lt;p&gt;You may not be able to express the weak form in deal.II as a mathematical formula, but you come pretty close. For elasticity, the bilinear form reads&#xA;$$&#xA;  a({\mathbf \varphi}_i, {\mathbf \varphi}_j) =&#xA;  \left(&#xA;    \lambda \nabla\cdot {\mathbf \varphi}_i, \nabla\cdot {\mathbf \varphi}_j&#xA;  \right)_\Omega&#xA;  +&#xA;  \left(&#xA;    \mu \nabla\mathbf \varphi_i, \nabla\mathbf \varphi_j&#xA;  \right)_\Omega,&#xA;  +&#xA;  \left(&#xA;    \mu \nabla\mathbf \varphi_i, \nabla\mathbf \varphi_j^T&#xA;  \right)_\Omega,&#xA;$$&#xA;and the corresponding code you would write is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  for (unsigned int q_point=0; q_point&amp;lt;n_q_points; ++q_point)&#xA;    for (unsigned int i=0; i&amp;lt;dofs_per_cell; ++i)&#xA;      {&#xA;        const Tensor&amp;lt;2,dim&amp;gt; phi_i_grad&#xA;          = fe_values[displacements].gradient (i,q_point);&#xA;        const double phi_i_div&#xA;          = fe_values[displacements].divergence (i,q_point);&#xA;&#xA;        for (unsigned int j=0; j&amp;lt;dofs_per_cell; ++j)&#xA;          {&#xA;            const Tensor&amp;lt;2,dim&amp;gt; phi_j_grad&#xA;              = fe_values[displacements].gradient (j,q_point);&#xA;            const double phi_j_div&#xA;              = fe_values[displacements].divergence (j,q_point);&#xA;&#xA;            cell_matrix(i,j)&#xA;              +=  (lambda_values[q_point] *&#xA;                   phi_i_div * phi_j_div&#xA;                   +&#xA;                   mu_values[q_point] *&#xA;                   double_contract(phi_i_grad, phi_j_grad)&#xA;                   +&#xA;                   mu_values[q_point] *&#xA;                   double_contract(phi_i_grad, transpose(phi_j_grad))&#xA;                  ) *&#xA;                  fe_values.JxW(q_point);&#xA;          }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last couple of lines assembling the cell matrix are, in effect, a 1:1 transcription of the formula above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More details here: &lt;a href=&quot;https://dealii.org/developer/doxygen/deal.II/group__vector__valued.html&quot;&gt;https://dealii.org/developer/doxygen/deal.II/group__vector__valued.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-02-14T18:56:02.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23127" PostTypeId="1" AcceptedAnswerId="23132" CreationDate="2016-02-15T12:11:33.980" Score="2" ViewCount="466" Body="&lt;p&gt;I have two line segments given by their endpoints $(a_1,a_2)$, $(b_1,b_2)$ in $R^3$ and want to know if they have the same length (up to some error), so that the naive test looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|\, \Vert a_1-a_2\Vert- \Vert b_1-b_2\Vert\, | &amp;lt; d_\text{threshold}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I need to test a large set of segments so I would like to get rid of the &lt;code&gt;sqrt&lt;/code&gt; within the norm computation. Each segment only shows up once in the full computation so that I can precompute and store lenghts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $a$ is the squared norm of $a_1-a_2$, I can write the equation as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|\sqrt{a} - \sqrt{b}| &amp;lt; d_\text{threshold}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I square both sided, I only get a &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a^2 - 2 \sqrt{a} \sqrt{b} + b^2 &amp;lt; d_\text{threshold}^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which again forces me to compute the roots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something obvious?&lt;/p&gt;&#xA;" OwnerUserId="19085" LastEditorUserId="9667" LastEditDate="2016-02-15T19:51:52.827" LastActivityDate="2016-02-24T18:14:36.010" Title="Fast comparison of line segments lengths" Tags="|computational-geometry|geometry|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23128" PostTypeId="2" ParentId="22111" CreationDate="2016-02-15T13:45:54.490" Score="1" Body="&lt;p&gt;If random vector $X$ has variance $S$, then $LX$ has variance $LSL^\top$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So generate whatever random variables with mean 0 and identity covariance matrix, then transform it $LX+\mu$, where $\mu$ is your mean vector and $LL^\top$ equals to your covariance matrix. You can find $L$ by cholesky decomposition.&lt;/p&gt;&#xA;" OwnerUserId="14305" LastActivityDate="2016-02-15T13:45:54.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23129" PostTypeId="2" ParentId="23127" CreationDate="2016-02-15T14:50:35.257" Score="2" Body="&lt;p&gt;Generally speaking, the $\mathtt{sqrt}$ function is going to be the slowest part of that. Fortunately, if their lengths are the same, then the squares of their lengths are also the same. Therefore, you can just omit that part and compare the squares of the norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any way to make that go too much faster using cross-platform code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you really want to get it running quickly, you can always take a look at what processor you're running on and find out if it supports vector instructions (ie, SSE or AVX) and use those intrinsics. &lt;a href=&quot;https://software.intel.com/sites/landingpage/IntrinsicsGuide/&quot; rel=&quot;nofollow&quot;&gt;https://software.intel.com/sites/landingpage/IntrinsicsGuide/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By doing this, you can take advantage of instruction-level parallelism in the computation and achieve huge speedups, albeit at the cost of your time. It's possible that your compiler is already doing this for you, however, especially if you compiler with (for gcc) -march=native. You should check the assembly that comes out of the compiler before spending time vectorizing the code yourself.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2016-02-15T14:50:35.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23130" PostTypeId="2" ParentId="23127" CreationDate="2016-02-15T15:18:09.433" Score="1" Body="&lt;p&gt;Instead of putting a threshold on the differences, put a threshold on the ratio:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{ \lVert \mathbf{b}_2-\mathbf{b}_1 \rVert}{ \lVert \mathbf{a}_2-\mathbf{a}_1 \rVert}&amp;lt; \tau&#xA;$$&#xA;You could then reduce this to:&#xA;$$&#xA;r =\frac{ \lVert \mathbf{b}_2-\mathbf{b}_1 \rVert^2}{ \lVert \mathbf{a}_2-\mathbf{a}_1 \rVert^2}&amp;lt; \tau^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and get rid of the square root. If you are disturbed by the fact that $r$ depends on the order selected, than you could consider a short if statement.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2016-02-15T15:18:09.433" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23131" PostTypeId="1" AcceptedAnswerId="29222" CreationDate="2016-02-15T16:22:53.290" Score="7" ViewCount="324" Body="&lt;p&gt;I am looking to implement GMSh into a simualtor that I am going to create. I am looking to integrate the geo, mesh, and post processor modules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, looking online, it appears the documentation for the GMSH function calls are non existent. I was wondering if anyone on the forum has ran into some sort of documentation for the program. I do know that this exists but it is not extensive enough: &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.pdf&quot; rel=&quot;nofollow&quot;&gt;http://gmsh.info/doc/texinfo/gmsh.pdf&lt;/a&gt;. From the PDF, it says that the API documentation is not complete. Does a rough draft of the documentation exist somewhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AS a side question, what is the practicality of integrating GMSH in a simulator given its current state? Would it be better for me to look off of other open source programs on how they created a basic CAD and mesh package or are there better open source alternatives that I can use? I did find this site: &lt;a href=&quot;http://www.robertschneiders.de/meshgeneration/software.html&quot; rel=&quot;nofollow&quot;&gt;http://www.robertschneiders.de/meshgeneration/software.html&lt;/a&gt; and it appears that GMSH is the better tool compared to the others. The only downfall is that GMSH does not have adequate documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is that I could create my own from scratch; however, I do not have the experience nor the background to do this. Unless, someone would be able to provide some really good source material, I might be able to pull it off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So after talking to the creators of the program, there is no documentation of the API.&lt;/p&gt;&#xA;" OwnerUserId="18230" LastEditorUserId="18230" LastEditDate="2016-02-19T13:45:47.707" LastActivityDate="2018-04-02T18:03:18.660" Title="Does some form of documentation of GMSH exist?" Tags="|software|computational-geometry|mesh-generation|geometry|documentation|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="23132" PostTypeId="2" ParentId="23127" CreationDate="2016-02-15T18:32:27.543" Score="4" Body="&lt;p&gt;You could also rewrite the expression to avoid square roots entirely. Reorder $a,b$ so that $a&amp;gt;b&amp;gt;0$, and let the inequality be&#xA;$$ \sqrt{a}-\sqrt{b}&amp;lt;\tau. $$&#xA;This is equivalent to&#xA;$$ a &amp;lt; \tau^2 + 2\tau\sqrt{b} + b \qquad\Leftrightarrow\qquad 2\tau\sqrt{b} &amp;gt; a-\tau^2-b, $$&#xA;which is in turn equivalent to&#xA;$$ \Big( a &amp;lt; \tau^2+b \quad\text{or}\quad 4\tau^2b&amp;gt;(a-\tau^2-b)^2 \Big) $$&#xA;which does not involve square roots.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-02-15T18:37:45.313" LastActivityDate="2016-02-15T18:37:45.313" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23133" PostTypeId="2" ParentId="23127" CreationDate="2016-02-15T22:12:51.953" Score="1" Body="&lt;p&gt;If $a$ is the squared norm of $a_1-a_2$, then the equation can written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$|\sqrt{a} - \sqrt{b}| &amp;lt; d_\text{threshold}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But squaring both sides results in:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a - 2 \sqrt{a} \sqrt{b} + b &amp;lt; d_\text{threshold}^2$$&#xA;or&#xA;$$ - 2 \sqrt{a} \sqrt{b} &amp;lt; d_\text{threshold}^2 - (a + b)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so&#xA;$$(a + b - d_\text{threshold}^2)&amp;lt; 2\sqrt{a} \sqrt{b} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $c = a + b - d_\text{threshold}^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus $$c &amp;lt; 2\sqrt{a} \sqrt{b} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If c &gt;= 0 then the inequality becomes&#xA;$$ c^2 &amp;lt; 4ab$$&#xA;Otherwise use&#xA;$$ c^2 &amp;gt; 4ab$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inequality flips due to effectively multiplying by the negative value $c$.&#xA;From a programming standpoint, I like the fact that very few operations would be required to determine inclusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original answer neglected the fact that  $d^2$ could be larger than $a + b$.&lt;/p&gt;&#xA;" OwnerUserId="19295" LastEditorUserId="19295" LastEditDate="2016-02-24T18:14:36.010" LastActivityDate="2016-02-24T18:14:36.010" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23134" PostTypeId="2" ParentId="7189" CreationDate="2016-02-15T23:33:12.187" Score="1" Body="&lt;p&gt;This problem lends itself to a highly vectorized form. As you noted, the ADI method gives a few steps of tridiagonal systems. Since it's in the form of linear equations, you can use &lt;a href=&quot;https://devblogs.nvidia.com/parallelforall/parallel-direct-solvers-with-cusolver-batched-qr/&quot; rel=&quot;nofollow&quot;&gt;CUsolver&lt;/a&gt; and &lt;a href=&quot;https://developer.nvidia.com/cublas&quot; rel=&quot;nofollow&quot;&gt;CUblas&lt;/a&gt; to call parallel GPU versions of standard linear algebra routines. Using these, you should be able to take the explicit code and just change the inner loop to an appropriate CUsolver call and solve it in a way where the code looks almost exactly like a CPU implementation, but with the matrix operations done on the GPUs via library calls.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-02-15T23:33:12.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23135" PostTypeId="1" CreationDate="2016-02-16T05:52:32.807" Score="7" ViewCount="10267" Body="&lt;p&gt;For those familiar with more of these tools than I am, what are the pros and cons of the various tools available? Right now I exclusively use Tecplot for CFD visualization, but it leaves a lot to be desired. The vector graphics are okay, but not great, and it's not clear to me how to have the fonts be correctly generated raw by LaTeX. For 2D lineplots I prefer python/matplotlib for pgf graphics with great LaTeX operability, but python lacks flowfield visualization stuff. Scripting in Tecplot is okay, and reproducing identical figures but with different data is pretty easy by recording macros and editing them. It's also very easy to get up and running. Paraview and visit I haven't used for anything nontrivial, and they seem to have a high barrier to entry. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me, matplotlib takes a little more learning to get started, but after that you can produce excellent publication quality vector plots in the blink of an eye, far faster and better than in Matlab. Is it possible to do the same with paraview or visit compared to Tecplot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if you had a new PhD student what would you push them towards for the best quality figures, and what would your workflow look like?&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2020-09-05T15:44:45.693" Title="CFD visualization workflow: Visit vs Paraview vs Tecplot and others" Tags="|visualization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23137" PostTypeId="1" AcceptedAnswerId="23141" CreationDate="2016-02-16T11:22:33.270" Score="4" ViewCount="324" Body="&lt;p&gt;I'm looking to solve a matrix equation and not sure where to start looking for resources.&#xA;The equation is&#xA;$$AX + XB = C\,,$$&#xA;where $A\in\mathbb{R}^{n\times n}$, $B\in\mathbb{R}^{m\times m}$, $C\in\mathbb{R}^{n\times m}$ and $X\in\mathbb{R}^{n\times m}$.&#xA;(Alternatively $AX + XA = C$ with $n=m$ would be useful.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can see that I could change it into a system&#xA;$$A'\mathbf{x}' = \mathbf{b}'\,,$$&#xA;with $A' \in \mathbb{R}^{nm\times nm}$, ${\mathbf{x}\in \mathbb{R}^{nm}}$, ${\mathbf{b}\in \mathbb{R}^{nm}}$, but was wondering if there were any shortcuts I could take.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastActivityDate="2016-02-16T13:49:15.003" Title="Resources for solving mixed left and right matrix equations" Tags="|linear-algebra|matrix|linear-solver|reference-request|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23138" PostTypeId="1" CreationDate="2016-02-16T11:27:42.257" Score="2" ViewCount="82" Body="&lt;p&gt;Given that $R$ is a rectangular matrix, $D$ is a diagonal, square matrix and $S$ being a square matrix along with the fact that both $D$ and $S$ are invertible. $S$ in this specific case can be assumed to be symmetric, positive-definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R$ is actually the first few columns of the discrete Fourier Transform matrix. Thus, $R^{T}R=Id $. Note $RR^{T}$ need not be $Id$. Also, $R$ has both dimensions of the same order (but with more rows than columns). $R$ also has a full row rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to obtain $D$ in terms of the given matrices, which are $R$ and $S$&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="15120" LastEditDate="2016-02-16T15:34:44.167" LastActivityDate="2016-02-16T15:34:44.167" Title="Solve for $D$ in $R^{T}DSDR = Id$" Tags="|linear-solver|matrix|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23139" PostTypeId="2" ParentId="23135" CreationDate="2016-02-16T12:35:53.923" Score="7" Body="&lt;p&gt;I'd venture the guess that most people in computational science use either Visit or Paraview for flow visualizations. These are simply the two most widely used programs I use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's true that there is a bit of a barrier in the beginning, but my students are quite proficient after using it for a class period or two. If you want to see an interactive demonstration of how they are used, take a look at videos 11 and 32 here:&#xA;&lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="7681" LastEditDate="2018-07-09T23:10:17.663" LastActivityDate="2018-07-09T23:10:17.663" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23140" PostTypeId="2" ParentId="533" CreationDate="2016-02-16T13:14:39.897" Score="0" Body="&lt;p&gt;From &lt;a href=&quot;http://www.icp.uni-stuttgart.de/~icp/mediawiki/images/b/b8/DFT.pdf&quot; rel=&quot;nofollow&quot;&gt;Chapter 9 of Bob Gotwal's lecture notes on &lt;em&gt;Computational Chemistry for Chemistry Educators&lt;/em&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Advantages&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most significant advantage to DFT methods is a significant increase in computational accuracy without the additional increase in computing time. DFT methods such as B3LYP/6-31G(d) are oftentimes considered to be a standard model chemistry for &lt;strong&gt;many applications.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disadvantages&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the main disadvantages of DFT methods is the challenge in determining the most appropriate method for a particular application. The practitioner should, prior to choosing a DFT method, consult the literature to determine the suitability of that choice for that particular problem or application. As such, DFT usage tends to favor the more sophisticated user. In general practice (including educational environments), the B3LYP/6-31g(d) model chemistry is considered by most to be a good general-purpose choice&lt;/p&gt;&#xA;" OwnerUserId="19303" LastEditorUserId="1804" LastEditDate="2016-02-18T14:56:28.303" LastActivityDate="2016-02-18T14:56:28.303" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23141" PostTypeId="2" ParentId="23137" CreationDate="2016-02-16T13:42:55.523" Score="5" Body="&lt;p&gt;This is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Sylvester_equation&quot;&gt;Sylvester equation&lt;/a&gt;, although normally the matrices would all be square. Even for rectangular matrices, the &lt;a href=&quot;http://dx.doi.org/10.1145/361573.361582&quot;&gt;Bartels-Stewart method&lt;/a&gt; applies (also &lt;a href=&quot;https://www.cs.cornell.edu/cv/ResearchPDF/Hessenberg.Schur.Method.pdf&quot;&gt;&lt;em&gt;A Hessenberg-Schur Method for the Problem $AX+XB=C$&lt;/em&gt;, Golub, Nash, Van Loan&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to use the Schur decompositions&#xA;$$ A = URU^\top, \qquad B^\top = VSV^\top, $$&#xA;so that $AX+XB=C$ is transformed into&#xA;$$ RY+YS^\top = F, \qquad Y = U^\top XV, F = U^\top CV. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $R$ is upper-triangular and $S^\top$ lower-triangular, this can be used to solve the system directly. If $Y = [y_1, \ldots, y_m]$ are the columns of $Y$, and $f_k$ are the columns of $F$, then the equation for $y_k$ depends only on $y_{k+1},\ldots,y_m$, and can be solved with back-substitution:&#xA;$$ (R+s_{kk}I)y_k = f_k - \sum_{j&amp;gt;k} s_{kj}y_j. $$&#xA;When using the real Schur decomposition, this would sometimes be a coupled linear equation in $y_k, y_{k+1}$ instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also shows it doesn't matter if $m\neq n$. To convert to the usual Sylvester equation with $m=n$ one could also just pad the matrices with zeros to obtain an equivalent problem.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-02-16T13:49:15.003" LastActivityDate="2016-02-16T13:49:15.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23142" PostTypeId="1" AcceptedAnswerId="23143" CreationDate="2016-02-16T14:16:56.733" Score="2" ViewCount="540" Body="&lt;p&gt;I am confused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;a href=&quot;http://www.math.udel.edu/~fjsayas/anIntro2FEM.pdf&quot; rel=&quot;nofollow&quot;&gt;read&lt;/a&gt; an introduction to finite element method where it was derived for the poisson equation:&#xA;$$-\Delta u + cu = f,\qquad, u = g_0 \text{ on Dirichlet boundaries},\qquad\partial_n u  g_1  \text{ on Neumann boundaries}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It introduced the concept of the weak form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Find u such that&lt;/em&gt; $u=g_0$ on Dirichlet boundaries&#xA;$$\int_\Omega \nabla u \bullet \nabla v +c \int_\Omega uv = \int_\Omega  fv + \int g_1 v $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And derives the method from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In another paper I was reading however, a variational formulation of the Poisson equation was presented:&#xA;$$\Delta u + k = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which was then formulated as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Minimize&lt;/em&gt; $\int_\Omega \nabla u \bullet \nabla u - 2ku$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t  $u=0$ on boundary&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;My Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In the paper they say that is it now trivial to use the variational formulation to use finite element methods. But I don't understand the connection with the weak formulation and hence I do not understand the link with the finite element method.&lt;/p&gt;&#xA;" OwnerUserId="19241" LastEditorUserId="1804" LastEditDate="2016-02-16T14:38:35.197" LastActivityDate="2016-02-16T15:55:43.617" Title="Finite element method applied to variational problem/functional VS weak formulation" Tags="|finite-element|variational-calculus|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23143" PostTypeId="2" ParentId="23142" CreationDate="2016-02-16T14:37:34.107" Score="4" Body="&lt;p&gt;The connection is quite straightforward (but note that your two examples have different boundary conditions and are thus &lt;em&gt;not&lt;/em&gt; equivalent). Assume that you know (e.g., from physical considerations) that your unknown function minimizes the functional&#xA;$$J(u) = \int_\Omega |\nabla u|^2 - 2fu\,dx$$&#xA;over all functions in a certain function space (here, $u\in H^1_0(\Omega)$, the space of all weakly differentiable functions which vanish at the boundary).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as in standard calculus, a necessary condition for $u$ to be a minimizer is that the derivative vanishes at $u$; specifically, that the &#xA;&lt;em&gt;directional derivative&lt;/em&gt; satisfies&#xA;$$J(u;v) := \lim_{t\to 0} \frac{J(u+tv)-J(u)}{t} = 0 \qquad\text{for all }v\in H^1_0(\Omega)$$&#xA;(intuitively, $J$ does not further decrease by going a small step in any direction $v$ from $u$). Now, inserting the definition, canceling what you can, and taking the limit yields&#xA;$$0=J(u;v) = 2\int_\Omega \nabla u\cdot \nabla v\,dx - 2\int_\Omega f v\,dx \qquad \text{for all } v\in H^1_0(\Omega),$$&#xA;which is precisely the weak formulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you now replace $H^1_0(\Omega)$ by a finite-dimensional subspace $V_h$ of (say) piecewise linear polynomials defined on a triangulation, you arrive (after rearranging and dividing by $2$) at the finite element formulation&#xA;$$\int_\Omega \nabla u_h\cdot \nabla v_h\,dx = \int_\Omega f v_h\,dx \qquad \text{for all } v_h\in V_h.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Neumann boundary conditions, you proceed similarly, but the functional now involves a boundary integral similar to the volume integral for the right-hand side. For nonhomogeneous Dirichlet conditions, you'd write $u=u_g + u_0$, where $u_g$ is a suitable function with $u=g$ on the boundary and $u_0$ satisfies $u=0$ on the boundary and is similarly characterized as a minimizer (or solution of a weak form).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; In principle, you could also first replace $u\in H^1_0(\Omega)$ by $V_h$ in $J$, i.e., minimize $J(u_h)$ over all $u_h\in V_h$ and then compute the derivative, and end up at the same equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also discretize the functional and then apply a minimization algorithm to $J(u_h)$, e.g., steepest descent or Newton's method. (Or vice versa, formulate the minimization algorithm in function space and discretize each iteration.) For a linear equation, though, this would be equivalent to applying Richardson iteration to the weak formulation or just solving it, respectively, so there's no real gain. But for a nonlinear problem (where the weak formulation would be a nonlinear PDE), this so-called &lt;em&gt;direct method&lt;/em&gt; can make sense and is actually used in practice.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-02-16T15:55:43.617" LastActivityDate="2016-02-16T15:55:43.617" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23144" PostTypeId="1" CreationDate="2016-02-16T16:35:41.163" Score="5" ViewCount="1436" Body="&lt;p&gt;I am trying to evaluate this integral directly using numerical integration functions in Mathematica and Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_0^\infty {y^{-(n+1)}\prod_{i=1}^n \gamma(a_i+1,b_i y) }dy&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\gamma(a,x)=\int_0^x dt\ t^{a-1}e^{-t}$ is the lower incomplete Gamma function. In my actual problem $n$ can take values from 10 to 100, where $a_i$&#xA;can take values from 100 to 1000.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the Python code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.integrate as integrate&#xA;import scipy.special as special&#xA;&#xA;&#xA;def integrand(a,b,y):&#xA;    l=len(a)&#xA;    x=pow(y,-(l+1))&#xA;    for ai,bi in zip(a,b):&#xA;        # x*=special.gammainc(ai+1,bi*y) &#xA;        x*=1/bi*special.gammainc(ai+1,bi*y)&#xA;    return x&#xA;&#xA;ns=[2,4,5,6]&#xA;&#xA;print integrate.quad(lambda x: integrand([n for n in ns],[0.1,0.8,0.3,0.4],x),0,np.inf)&#xA;print integrate.quad(lambda x: integrand([10*n for n in ns],[0.1,0.8,0.3,0.4],x),0,np.inf)&#xA;print integrate.quad(lambda x: integrand([100*n for n in ns],[0.1,0.8,0.3,0.4],x),0,np.inf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the corresponding output is  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(8.231566916434924e-05, 2.7894916135933814e-10)$&lt;br&gt;&#xA;$(5.884546215247849e-09, 1.1634128468162877e-08)$&lt;br&gt;&#xA;$(0.0, 0.0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is disappointing considering the modest $n$=4. In the second output $a_i$ ~ $60$ and the error is greater for one order of magnitude than the resulting integral. Don't need to comment the third trial case with $a_i$ ~ $600$, with zero error!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course that is not magic, I am using the full integration interval $[0,\infty]$, and ask for numerical tricks like reducing integration interval and any other treatment to get reasonable results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made some easy progress by multiplying by a constant ($\prod_r 1/b_i$) and the numbers becomes more stable within all the range of my application. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, by plotting the integrand I see an unimodal curve which span the ranges $[5,60]$, $[50,600]$ and $[500,6000]$ respectively, vanishing outside. If I truncate the integration intervals respectively the results are quite acceptable and agree with Wolfram Mathematica results. I tried other larger sets, and find the intervals by trial/error. So I will be happy if I can find the proper intervals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I find the interval for any sets of $a_i$ and $b_i$ and a multiplicative constant that made the numbers stable ?&lt;/p&gt;&#xA;" OwnerUserId="19307" LastEditorUserId="19308" LastEditDate="2016-03-08T04:19:22.177" LastActivityDate="2016-03-08T04:19:22.177" Title="Numerical integration of a single integral" Tags="|python|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23145" PostTypeId="2" ParentId="23144" CreationDate="2016-02-16T17:36:25.403" Score="2" Body="&lt;p&gt;Quadrature on improper integrals can always be problematic. If you're finding large errors, I'd suggest one of two things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;As choward suggested, use a change of variable trick to change the integral to have finite bounds, then use quadrature as usual.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a quadrature method that uses basis functions appropriate for your problem. In this case, the Laguerre Polynomials (or Rational Chebyshev on semi-infinite intervals) work (ref Boyd: &lt;em&gt;Chebyshev and Fourier Spectral Methods&lt;/em&gt;.) &lt;a href=&quot;https://en.wikipedia.org/wiki/Gauss%E2%80%93Laguerre_quadrature&quot; rel=&quot;nofollow&quot;&gt;Laguerre Quadrature should work better.&lt;/a&gt; There's even &lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/routines.polynomials.laguerre.html&quot; rel=&quot;nofollow&quot;&gt;a numpy module for it.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2016-02-16T17:36:25.403" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23146" PostTypeId="2" ParentId="22123" CreationDate="2016-02-16T22:26:23.020" Score="3" Body="&lt;p&gt;I myself have not had much experience with computational science. Stil relativity new. However, with the experience that I have had, I will try to answer with the best of my ability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would have to agree with the other users here. Try to get FENics support to help troubleshoot your issues with XCODE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that does not work, Deal.ii is not that bad. It takes care of much of the hard work of programming everything. With its extensive use of tutorials and documentation, I would not dismiss it quite yet. Sure, it takes a little bit of effort to get up and running, but comparatively speaking, I think that it is one of the best tools there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for converting to the weak for and estimating via quadrature, that may take a bit of research. &lt;/p&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2016-02-16T22:26:23.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23147" PostTypeId="1" AcceptedAnswerId="23157" CreationDate="2016-02-17T00:51:54.077" Score="-1" ViewCount="228" Body="&lt;p&gt;Our task is to implement a factorization routine that given A in a suitably efficient data structure returns the factors $L$ and $U$ where $L$ is unit lower triangular and $U$ is upper triangular. In order to guarantee that pivoting is not required you may restrict $A$ to be a diagonally dominant matrix. You must also write routines that solve systems $Lv = f$ and $Uv = f$ for arbitary vectors $v$ and $f$ and $L$ and $U$ matrices that result from your factorization routine. Your code must be as efficient as possible in terms of the number of comptuations and the storage required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $A\in\mathbb{R}^{n\times n}$ be a nonsymmetric and nonsingular matrix with zero/nonzero element structure that has nonzero elements on the main diagonal, i.e., $\alpha_{i,j}\neq 0$, the first superdiagonal, i.e., $\alpha_{i,i+1}\neq 0$, the first subdiagonal, i.e., $\alpha_{i-1,i}\neq 0$, the fourth superdiagonal, i.e., $\alpha_{i,i+4}\neq 0$, and the fourth subdiagonal, i.e., $\alpha_{i-4,i}\neq 0$. For $n = 15$ this has the form &#xA;$$&#xA; A =&#xA;\begin{pmatrix}&#xA;* &amp;amp; *  &amp;amp; 0  &amp;amp; 0  &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;* &amp;amp; *  &amp;amp; *  &amp;amp; 0  &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\  &#xA;0 &amp;amp; *  &amp;amp; *  &amp;amp; *  &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; *  &amp;amp; *  &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;* &amp;amp; 0  &amp;amp; 0  &amp;amp; *  &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; *  &amp;amp; 0  &amp;amp; 0  &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; *  &amp;amp;  0  &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; *  &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * \\ &#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, I have to perform the $LU$ factorization in as efficiently as possible, so far what I have done is store this matrix $A$ in a storage efficient manner. But I am not sure how to get $L$ and $U$ in the same manner, any suggestions is greatly appreciated. Here is my code so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            /* &#xA;     * File:   main.cpp&#xA;     * Author: Morgan Weiss&#xA;     *&#xA;     * Created on February 15, 2016, 8:21 PM&#xA;     */&#xA;&#xA;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt; &#xA;&#xA;double** banded_matrix(int n);&#xA;double get(double** A, int n, int i, int j);&#xA;void set(double** A, int n, int i, int j, double val);&#xA;void create(double** A, int n, int flag);&#xA;&#xA;&#xA;int main()&#xA;{&#xA;    int N = 4;&#xA;&#xA;    //Initialize A&#xA;    double **A = banded_matrix(N);&#xA;    create(A,N,0);&#xA;&#xA;&#xA;    //Initialize L &#xA;    double L[N][N];&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        for (int j = 0; j &amp;lt; N; j++){&#xA;            L[i][j] = 0; &#xA;        }          &#xA;    }&#xA;&#xA;    //Initialize U&#xA;    double U[N][N];&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        for (int j = 0; j &amp;lt; N; j++){&#xA;            U[i][j] = 0; &#xA;        }          &#xA;    }&#xA;&#xA;    //Set L[i][i] = 1&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        L[i][i] = 1;&#xA;    }&#xA;&#xA;    //Solve for L and U&#xA;    for (int j = 0; j &amp;lt; N; j++) //loop over columns&#xA;    {&#xA;        //Compute U&#xA;        for (int i = 0; i &amp;lt;= j; i++) &#xA;        {           &#xA;            double sum = 0;&#xA;            if (i &amp;gt; 0) &#xA;            {&#xA;                for (int k = 0; k &amp;lt; i; k++){&#xA;                    sum += L[i][k]*U[k][j];&#xA;                }&#xA;            }&#xA;            U[i][j] = get(A,N,i,j) - sum;&#xA;        }&#xA;&#xA;        //Compute L&#xA;        for (int i = j+1; i &amp;lt; N; i++) &#xA;        {&#xA;            double sum = 0;&#xA;            for (int k = 0; k &amp;lt; j; k++){&#xA;                sum += L[i][k]*U[k][j];&#xA;            }&#xA;&#xA;            L[i][j] = (get(A,N,i,j) - sum)/U[j][j];    &#xA;&#xA;        }&#xA;    }&#xA;&#xA;&#xA;    //Initialize result matrix R&#xA;    double R[N][N];&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        for (int j = 0; j &amp;lt; N; j++){&#xA;            R[i][j] = 0;&#xA;            for (int k = 0; k &amp;lt; N; k++){&#xA;                R[i][j] += L[i][k]*U[k][j];   &#xA;            }        &#xA;        }          &#xA;    }&#xA;&#xA;    //Print R&#xA;    std::cout &amp;lt;&amp;lt; &quot;R = &quot; &amp;lt;&amp;lt; std::endl;&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        for (int j = 0; j &amp;lt; N; j++) {&#xA;            std::cout &amp;lt;&amp;lt; R[i][j] &amp;lt;&amp;lt; &quot;   &quot;;        &#xA;        }&#xA;        std::cout &amp;lt;&amp;lt; std::endl;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; std::endl;&#xA;&#xA;&#xA;&#xA;    //Print A&#xA;    std::cout &amp;lt;&amp;lt; &quot;A = &quot; &amp;lt;&amp;lt; std::endl;&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        for (int j = 0; j &amp;lt; N; j++) {&#xA;            std::cout &amp;lt;&amp;lt; get(A,N,i,j) &amp;lt;&amp;lt; &quot;   &quot;;        &#xA;        }&#xA;        std::cout &amp;lt;&amp;lt; std::endl;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; std::endl;&#xA;&#xA;    //Print U&#xA;    std::cout &amp;lt;&amp;lt; &quot;U = &quot; &amp;lt;&amp;lt; std::endl;&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        for (int j = 0; j &amp;lt; N; j++) {&#xA;            std::cout &amp;lt;&amp;lt; U[i][j] &amp;lt;&amp;lt; &quot;   &quot;;        &#xA;        }&#xA;        std::cout &amp;lt;&amp;lt; std::endl;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; std::endl;&#xA;&#xA;    //Print L&#xA;    std::cout &amp;lt;&amp;lt; &quot;L = &quot; &amp;lt;&amp;lt; std::endl;&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        for (int j = 0; j &amp;lt; N; j++) {&#xA;            std::cout &amp;lt;&amp;lt; L[i][j] &amp;lt;&amp;lt; &quot;   &quot;;        &#xA;        }&#xA;        std::cout &amp;lt;&amp;lt; std::endl;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; std::endl;&#xA;&#xA;    return 0;&#xA;}&#xA;&#xA;void create(double** A, int n, int flag) {&#xA;    if (flag == 0) {&#xA;        for (int i=0; i&amp;lt;n-1; i++) {&#xA;            for (int j=0; j&amp;lt;n-1; j++) {&#xA;                if (i==j) {&#xA;                    set(A,n,i,j,2.);&#xA;                } else if (abs(i-j)==1 || abs(i-j)==4) {&#xA;                    set(A,n,i,j,1.);&#xA;                }&#xA;            }&#xA;        }&#xA;    } else if (flag==1) {}&#xA;}&#xA;&#xA;double** banded_matrix(int n) {&#xA;    double **data = new double *[n];&#xA;    data[0] = new double[3];&#xA;    for(int i = 1; i &amp;lt;= 3; i++){&#xA;        data[i] = new double[4];&#xA;    }&#xA;    for(int i = 4; i &amp;lt;= n-3; i++){&#xA;        data[i] = new double[5];&#xA;    }&#xA;    for(int i = n-4; i &amp;lt;= n-2; i++){&#xA;        data[i] = new double[4];&#xA;    }&#xA;    data[n-1] = new double[3];&#xA;    return data;&#xA;}&#xA;&#xA;&#xA;double get(double** A, int n, int i, int j) {&#xA;    double result;&#xA;    // main tridiagonal&#xA;    if(abs(i-j) &amp;lt; 2){&#xA;        if(i == 0){&#xA;            result = A[i][j];&#xA;        } else if(i &amp;lt;= 3){&#xA;            result = A[i][j-(i-2)];&#xA;        } else if(i &amp;gt;= 3){&#xA;            result = A[i][j-(i-1)];&#xA;        }&#xA;    // 4th superdiagonal&#xA;    } else if (j - i == 4){&#xA;        if(i == 0 || i == n-1) {&#xA;            result = A[i][2];&#xA;        } else if((i &amp;gt;= 1 &amp;amp;&amp;amp; i &amp;lt;= 3) || (i &amp;gt;= n - 4 &amp;amp;&amp;amp; i &amp;lt;= n - 2)) {&#xA;            result = A[i][3];&#xA;        } else {&#xA;            result = A[i][4];&#xA;        }&#xA;    // 4th subdiagonal&#xA;    } else if (i - j == 4){&#xA;        result = A[i][0];&#xA;    // the rest of the matrix    &#xA;    } else {&#xA;        result = 0.;&#xA;    }&#xA;    return result;&#xA;}&#xA;&#xA;void set(double** A, int n, int i, int j, double val) {&#xA;    // main tridiagonal&#xA;    if(abs(i-j) &amp;lt; 2){&#xA;        if(i == 0){&#xA;            A[i][j] = val;&#xA;        } else if(i &amp;lt;= 3){&#xA;            A[i][j-(i-2)] = val;&#xA;        } else if(i &amp;gt;= 3){&#xA;            A[i][j-(i-1)] = val;&#xA;        }&#xA;    // 4th superdiagonal&#xA;    } else if (j - i == 4){&#xA;        if(i == 0 || i == n-1) {&#xA;            A[i][2] = val;&#xA;        } else if((i &amp;gt;= 1 &amp;amp;&amp;amp; i &amp;lt;= 3) || (i &amp;gt;= n - 4 &amp;amp;&amp;amp; i &amp;lt;= n - 2)) {&#xA;            A[i][3] = val;&#xA;        } else {&#xA;            A[i][4] = val;&#xA;        }&#xA;    // 4th subdiagonalf&#xA;    } else if (i - j == 4){&#xA;        A[i][0] = val;&#xA;    // the rest of the matrix    &#xA;    } else {&#xA;        std::cout &amp;lt;&amp;lt; &quot;cannot set element (&quot; &amp;lt;&amp;lt; i &amp;lt;&amp;lt;&quot;,&quot; &amp;lt;&amp;lt; j &amp;lt;&amp;lt;&quot;) in matrix&quot; &amp;lt;&amp;lt; std::endl;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is my best attempt, if anyone has any suggestions on the way I implemented this, I would appreciate it.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="17826" LastEditDate="2016-02-21T05:46:49.780" LastActivityDate="2016-02-22T10:36:08.080" Title="$LU$ factorization" Tags="|linear-algebra|numerical-analysis|c++|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23148" PostTypeId="2" ParentId="23135" CreationDate="2016-02-17T02:10:17.040" Score="4" Body="&lt;p&gt;I don't know why do you want vector graphics for your visualizations. It works ok for 2D cases, but in 3D I believe that there is need for raster images. In Paraview you can export to PDF, for example. Also, you might find Mayavi interesting. The next example generates a vector image (use with caution this simple example is 1.8 MB).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from mayavi import mlab&#xA;&#xA;# Test data: Matlab `peaks()`&#xA;x, y = np.mgrid[-3:3:50j,-3:3:50j]&#xA;z =  3*(1 - x)**2 * np.exp(-x**2 - (y + 1)**2) \&#xA;   - 10*(x/5 - x**3 - y**5)*np.exp(-x**2 - y**2) \&#xA;   - 1./3*np.exp(-(x + 1)**2 - y**2) &#xA;&#xA;surf = mlab.surf(x, y, z, colormap='RdYlBu', warp_scale=0.3, representation='wireframe', line_width=0.5)&#xA;mlab.outline(color=(0, 0, 0))&#xA;axes = mlab.axes(color=(0, 0, 0), nb_labels=5)&#xA;axes.title_text_property.color = (0.0, 0.0, 0.0)&#xA;axes.title_text_property.font_family = 'times'&#xA;axes.label_text_property.color = (0.0, 0.0, 0.0)&#xA;axes.label_text_property.font_family = 'times'&#xA;mlab.savefig(&amp;quot;vector_plot_in_3d.pdf&amp;quot;)&#xA;mlab.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/87121.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/87121.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As a final comment, I would say that you can generate good visualizations in Mayavi/Paraview, Tecplot or matplotlib, but you will have to invest some time.&lt;/p&gt;&#xA;&lt;p&gt;Here some examples in Paraview:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ClytI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ClytI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nYoAE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nYoAE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vsPxH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vsPxH.png&quot; alt=&quot;http://www.paraview.org/gallery/&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2020-09-05T15:44:45.693" LastActivityDate="2020-09-05T15:44:45.693" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23149" PostTypeId="1" CreationDate="2016-02-17T14:31:48.670" Score="1" ViewCount="168" Body="&lt;p&gt;I am a beginner user of &lt;code&gt;R&lt;/code&gt;. I am trying to maximize log likelihood function with the bounded parameters. The function is a kind of gamma mixture model which try to capture unobserved heterogeneity across individual. I chose the &lt;code&gt;optim's L-BFGS-B&lt;/code&gt; given that the function contains two parameters (i.e., &lt;code&gt;alpha&lt;/code&gt; and &lt;code&gt;r&lt;/code&gt;) which should have positive values. However, the optimization does not take place and returns the initial parameters with the  message below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Sample data&#xA;data&amp;lt;-matrix(c(1,36,547,2,54,464,3,92,415,4,114,1106,5,10,1038),ncol=3,byrow=TRUE)&#xA;colnames(data)&amp;lt;-c(&quot;ID&quot;,&quot;freq&quot;,&quot;time&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\min_{r, \alpha}\; r \log \alpha + \sum_{j=0}^{n_i-1}\log(r+j) - (n_i + r)\log(t_{is} + t_{ic} + \alpha)\\&#xA;&amp;amp;\text{such that}\\&#xA;&amp;amp;\qquad r &amp;gt; 0\quad \text{(shape parameter)}\\&#xA;&amp;amp;\qquad \alpha &amp;gt; 0\quad \text{(scale parameter)}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code below I use following terms for function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;ni&lt;/em&gt; = data[i,2] ## ni is the frequency of individual i&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;tis&lt;/em&gt;+&lt;em&gt;tic&lt;/em&gt; = data[i,3] ## tis+tic is the total time of individual i&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Likelihood function I want to optimize&#xA;ll &amp;lt;- function(theta){&#xA;alpha&amp;lt;-theta[1]  #scale parameter &amp;gt;0&#xA;r&amp;lt;-theta[2] #shape parameter &amp;gt;0&#xA;&#xA;ll_i=c()&#xA;s=c()&#xA;&#xA;for(i in 1:5){&#xA;&#xA;for(j in 0:data[i,2]-1){&#xA;s[j+1]&amp;lt;-log(r+j+1)&#xA;}&#xA;&#xA;ll_i[i]&amp;lt;-r*log(alpha)+sum(s)-(data[i,2]+r)*log(alpha+data[i,3])&#xA;s=c()&#xA;}&#xA;&#xA;print(ll_i)&#xA;ll&amp;lt;-sum(ll_i)&#xA;return(-ll)&#xA;}&#xA;&#xA;# Initial parameters&#xA;init&amp;lt;-c(alpha=1,r=5)&#xA;# L-BFGS-B optimization&#xA;ans&amp;lt;-optim(par=init,fn=ll,method=&quot;L-BFGS-B&quot;,lower=c(1e-10,1e-10),upper=c(Inf,Inf))&#xA;&#xA;$message&#xA;[1] &quot;CONVERGENCE: NORM OF PROJECTED GRADIENT &amp;lt;= PGTOL&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any hint towards a successful optimization?&lt;/p&gt;&#xA;" OwnerUserId="19320" LastEditorUserId="19320" LastEditDate="2016-02-23T10:16:22.590" LastActivityDate="2016-02-23T10:16:22.590" Title="Problem in using N-BFGS-B in optim" Tags="|optimization|r|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23150" PostTypeId="1" CreationDate="2016-02-17T15:50:57.573" Score="2" ViewCount="79" Body="&lt;p&gt;Consider a discrete constrained optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{q}_*^h= \arg \min {\cal J}^h(\mathbf{x}^h[\mathbf{q}^h],\mathbf{q}^h)&#xA;$$&#xA;subject to the (weak-form) constraint&#xA;$$&#xA;F^h[\mathbf{x}^h(\mathbf{q}^h),\mathbf{q}^h](\mathbf{w}^h) = 0 \; \forall \, \mathbf{w}^h&#xA;$$&#xA;with $\mathbf{x}^h \in \Omega^h$ (the discrete domain) and $\mathbf{w}^h$ the discrete test functions. We can assume $F^h$ is a consistent and stable discretization of a corresponding continuous problem on the discrete domain $\Omega^h$. The first order necessary conditions that define our local minimum $\mathbf{x}^*$, with $\lambda_*^h$ the discrete adjoint variable (or Lagrange multiplier) are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F^h[\mathbf{x}^h,\mathbf{q}_*^h](\mathbf{w}^h) = 0 \;\; \forall \, \mathbf{w}^h \\&#xA;F_{\mathbf{x}}^h[\mathbf{x}^h,\mathbf{q}_*^h](\mathbf{w}^h,\lambda_*^h) = {\cal J}_\mathbf{x}[\mathbf{x}^h,\mathbf{q}_*^h] \;\; \forall \, \mathbf{w}^h \\&#xA;F_{\mathbf{q}}^h[\mathbf{x}^h,\mathbf{q}_*^h](\mathbf{w}^h,\lambda_*^h) = {\cal J}_\mathbf{q}[\mathbf{x}^h,\mathbf{q}_*^h] \;\; \forall \, \mathbf{w}^h&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How sensitive is the discrete optimum $\mathbf{q}^h_*$ to &lt;em&gt;consistent&lt;/em&gt; perturbations to the primal problem? By consistent I mean a discrete perturbation that vanishes when evaluated at the &lt;em&gt;exact&lt;/em&gt; (continuous) KKT triplet $(\mathbf{u},\lambda,\mathbf{q})$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F^h[\mathbf{x}^h(\mathbf{q}^h),\mathbf{q}^h](\mathbf{w}^h) + \Delta^h [\mathbf{x}^h,\mathbf{q}^h](\mathbf{w}^h) = 0 \; \forall \, \mathbf{w}^h&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What bounds do we need to prove on $\Delta^h$ (or its derivative) to guarantee the perturbed KKT system still converges to $(\mathbf x_*^h,\mathbf q_*^h)$? I'm aware the answer may well be problem dependent, so I'd welcome some references to existing literature for discrete model problems (linear diffusion, etc.).&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2016-02-17T15:50:57.573" Title="Convergence of KKT equations for discrete parameter estimation problems" Tags="|pde|constrained-optimization|optimal-control|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23152" PostTypeId="2" ParentId="23144" CreationDate="2016-02-17T19:12:14.637" Score="4" Body="&lt;p&gt;Improper integrals like this are often no match for &lt;a href=&quot;https://en.wikipedia.org/wiki/Tanh-sinh_quadrature&quot; rel=&quot;nofollow&quot;&gt;double exponential quadrature&lt;/a&gt;. I can compute your integrals with &lt;a href=&quot;http://mpmath.org&quot; rel=&quot;nofollow&quot;&gt;mpmath&lt;/a&gt;. For larger n, it appears that a higher precision needs to be used.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from mpmath import mp, gammainc, quad, power, inf, nstr&#xA;&#xA;def integrand(a,b,y):&#xA;    l=len(a)&#xA;    x=power(y,-(l+1))&#xA;    for ai,bi in zip(a,b):&#xA;        x*=gammainc(ai+1,0,bi*y,regularized=True) / bi&#xA;    return x&#xA;&#xA;ns=[2,4,5,6]&#xA;&#xA;mp.pretty = True&#xA;mp.dps = 30&#xA;print quad(lambda x: integrand([n for n in ns],[0.1,0.8,0.3,0.4],x),[0,inf],error=True)&#xA;print quad(lambda x: integrand([10*n for n in ns],[0.1,0.8,0.3,0.4],x),[0,inf],error=True)&#xA;mp.dps = 100&#xA;print nstr(quad(lambda x: integrand([100*n for n in ns],[0.1,0.8,0.3,0.4],x),[0,inf],error=True),20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(0.0000823156691641082138376342855035, 1.0e-41)&#xA;(0.0000000156728839596991659411718340463, 1.0e-28)&#xA;(1.6721149546712801741e-12, 1.0e-32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Splitting the interval at one point also does the trick:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; mp.dps = 30&#xA;&amp;gt;&amp;gt;&amp;gt; print quad(lambda x: integrand([100*n for n in ns],[0.1,0.8,0.3,0.4],x),[0,5000,inf],error=True)&#xA;(1.6721149546712801740899041739e-12, 1.0e-32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you don't know which parameters to use, you could make it automatic: check the error estimate from quad() and, if it's too large, either double the precision or subdivide the interval in two and recurse.&lt;/p&gt;&#xA;" OwnerUserId="3420" LastEditorUserId="3420" LastEditDate="2016-02-17T19:20:55.003" LastActivityDate="2016-02-17T19:20:55.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23155" PostTypeId="1" CreationDate="2016-02-17T23:22:36.393" Score="3" ViewCount="735" Body="&lt;p&gt;I'm trying to assemble global matrices for the following system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}&#xA; K&amp;amp; Q\\ &#xA; Q^T&amp;amp;S &#xA;\end{bmatrix} \begin{bmatrix}&#xA;u_h\\ &#xA;p_h&#xA;\end{bmatrix} = \begin{bmatrix}&#xA;f_u\\ &#xA;f_p&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In which, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ K, Q, Q^T, S $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are matrix generated from element integral, which I know how to compute. Likewise, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$f_u, f_p$$&lt;/span&gt; are vectors generated from the integral over the element (right-hand side), which I'm able to compute. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question that I have is how to compute the &lt;em&gt;global matrix&lt;/em&gt;? I know how to compute if it was a simple &lt;span class=&quot;math-container&quot;&gt;$KU = F$&lt;/span&gt; system. I'm not sure how to go about it in a coupled case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have matrices all elements, but I don't know how to assemble a global matrix system. Can I just create a global matrix for &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$p_h$&lt;/span&gt;, and then assume they are block matrices and append them together in a big matrix? My problem is that, in a global matrix, each node won't be coupled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that Taylor-Hood elements are used, so &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; nodes, and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; nodes. So each node has a varying # degrees of freedom, either 2 or 3. &lt;/p&gt;&#xA;" OwnerUserId="19328" LastEditorUserId="20688" LastEditDate="2019-05-25T10:00:57.587" LastActivityDate="2019-05-25T10:00:57.587" Title="How to assemble global matrix (for a coupled) problem?" Tags="|finite-element|matlab|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23156" PostTypeId="2" ParentId="23155" CreationDate="2016-02-18T01:01:45.033" Score="-1" Body="&lt;p&gt;I would highly recommend using PETSc, especially if you want to assemble in parallel, and if you are planning to use preconditioners created by combining preconditioners for individual fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assembly in PETSc is as simple as it gets. Look at some of the examples (many are for coupled problems as well). Parallel assembly and solve are the most complicated part of an FE code and PETSc takes care of both.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2016-02-18T01:01:45.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23157" PostTypeId="2" ParentId="23147" CreationDate="2016-02-18T03:47:48.483" Score="0" Body="&lt;p&gt;I am all for creating your own algorithm in order to do whatever. But I have to ask, why not use a linear algebra package written in C++ that can do LU factorization? I can see having to write your own if there are specific design requirements that the packages do not meet. If there aren't any special requirements, then why create more work for yourself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, these packages have gone through the fire and flames of testing to make them efficient. I will link a few to these in the answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://math.nist.gov/tnt/jama_doxygen/class_JAMA__LU.html&quot; rel=&quot;nofollow&quot;&gt;http://math.nist.gov/tnt/jama_doxygen/class_JAMA__LU.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.gnu.org/software/gsl/&quot; rel=&quot;nofollow&quot;&gt;http://www.gnu.org/software/gsl/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some low level stuff - &lt;a href=&quot;http://www.netlib.org/blas/&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/blas/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a list of some more Linear Algebra packages:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Comparison_of_linear_algebra_libraries&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Comparison_of_linear_algebra_libraries&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2016-02-18T03:47:48.483" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23158" PostTypeId="1" CreationDate="2016-02-18T09:31:46.307" Score="4" ViewCount="273" Body="&lt;p&gt;I am trying to implement a solver for the Euler/Navier Stokes equations. I have a problem implementing boundary conditions for the wall. I am using an unstructured solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of literature says that at the wall you simply reflect the normal velocity, and then pose it as a Riemann problem. So let me take the momentum equation. The U here is $\rho u$. The F (flux) is $\rho u^2+p$.&#xA;$u$ is velocity, $\rho$ is density, $p$ is pressure. Assume that the grid is such that $u$ is normal velocity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the way literature guides toward implementing this boundary condition is to take ghost values such that boundary conditions at the wall are satisfied. So reflect the velocity, and copy density, pressure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The velocity will come to zero at the wall using this. But if you look at the flux, it will not. I have compared Lax-Friedrichs, HLL and I don't see how F becomes $p$ as it should. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference I will put up this link &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arc.aiaa.org/doi/abs/10.2514/6.2014-2923&quot; rel=&quot;nofollow&quot;&gt;http://arc.aiaa.org/doi/abs/10.2514/6.2014-2923&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2016-02-18T09:31:46.307" Title="Reflecting boundary condition posed as a Riemann problem" Tags="|boundary-conditions|finite-volume|navier-stokes|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23159" PostTypeId="1" CreationDate="2016-02-18T12:21:53.483" Score="3" ViewCount="420" Body="&lt;p&gt;My question falls under purview of turbulent flows. I want to add an initial perturbation, for which I have a given energy spectrum (say$ E(k)=ak^4e^{-bk^2} $). The steps involved in getting these perturbation is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choose random values for $u\in (-1,1)$&lt;/li&gt;&#xA;&lt;li&gt;Fourier transform:  $u(x,y,z)\rightarrow\hat{u}(k)$&lt;/li&gt;&#xA;&lt;li&gt;Make $\hat{u}$ solenoidal $\hat{u}'=\hat{u}-\hat{k}.\hat{u}/k^2$ &lt;/li&gt;&#xA;&lt;li&gt;Scale $\hat{u}'$ with the given energy spectrum&lt;/li&gt;&#xA;&lt;li&gt;Take inverse transform:  $\hat{u}'(k)\rightarrow u(x,y,z)$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How to proceed when I have to give perturbations at discrete points, in which case $u(x,y,z)$ is available at discrete points $(x_i,y_j,z_k)$?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastActivityDate="2016-02-18T12:21:53.483" Title="How to obtain values in physical space for a given spectrum?" Tags="|fourier-transform|fftw|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23160" PostTypeId="2" ParentId="23155" CreationDate="2016-02-18T15:28:26.737" Score="3" Body="&lt;p&gt;I think you may be viewing the problem the wrong way round. The rows of your matrix correspond to your discretized equations, which depend on the degrees of freedom in the problem, not the physical locations in space. You are free (theoretically) to number the $u$ nodes and $p$ nodes however you please and you will still get $K$, a 9x9 matrix $Q$, a 9x4 matrix and $S$ a 4x4 matrix, specific to your numbering (and choice of element). The spatial coupling comes through the integrals which define these matrices, you shouldn't thing that there is anything special about the face that some nodes are spatially co-located. Indeed there are discontinuous finite element methods in which individual variables can have multiple nodes located at a single point in space.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-02-18T15:28:26.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23161" PostTypeId="2" ParentId="23155" CreationDate="2016-02-18T17:04:41.997" Score="0" Body="&lt;p&gt;If I understand your question correctly it is&#xA;primarily about programming rather than a conceptual FE&#xA;one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general purpose FE programs, it is almost always assumed&#xA;that different nodes have different sets of degrees of freedom (dof).&#xA;In order to assemble each element into the global matrix, a key&#xA;piece of information you need is a &quot;map&quot; from each degree of&#xA;freedom in the element matrix to the appropriate equation in the&#xA;global matrix. Here is a very simple way to create that map:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create an array (the map) with dimensions max_dofs_per_node x number_of_nodes &#xA;in the model. (In your case 3 x n)&lt;/li&gt;&#xA;&lt;li&gt;For each node in the model, set the entries in the column &#xA;of the map matrix corresponding to the node to 1 if the dof is&#xA; active at that node and 0 otherwise.&lt;/li&gt;&#xA;&lt;li&gt;Now iterate over all entries in the map matrix and assign a global&#xA;equation number if the entry is 1. Start the equation numbers at&#xA; one and increment each time.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;During assembly, for each element node, you can easily determine&#xA;the global equation number by referring to the appropriate column&#xA;in the map matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fastest way to create a sparse matrix in MATLAB is by first&#xA;creating a matrix of &quot;triplets&quot; (row, column, value) where row&#xA;and column are the appropriate equation numbers from the map matrix&#xA;and then passing the triplets to the MATLAB sparse function.&#xA;That approach is discussed in this note by Davis:&#xA;&lt;a href=&quot;http://blogs.mathworks.com/loren/2007/03/01/creating-sparse-finite-element-matrices-in-matlab/&quot; rel=&quot;nofollow&quot;&gt;http://blogs.mathworks.com/loren/2007/03/01/creating-sparse-finite-element-matrices-in-matlab/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-02-18T17:04:41.997" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23162" PostTypeId="2" ParentId="533" CreationDate="2016-02-18T17:49:27.197" Score="-3" Body="&lt;p&gt;DFT creates a basis for multiscale simulations molecular dynamics, Monte Carlo methods. But it rigorous methods and extensive approaches to model a atomic system using quantum mechanics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Main drawbacks would be the timescale FEMTO SECOND to PICO SECOND only and  expensive. But these are helpful in providing insights at atomic scale.&lt;/p&gt;&#xA;" OwnerUserId="19316" LastActivityDate="2016-02-18T17:49:27.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23164" PostTypeId="1" CreationDate="2016-02-18T20:43:08.473" Score="6" ViewCount="3231" Body="&lt;p&gt;I want to simulate the electric fields in simple circuits using Python and only free software. My first goal is to reproduce the images given in &lt;a href=&quot;https://www.tu-braunschweig.de/Medien-DB/ifdn-physik/ajp000782.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(1)&lt;/a&gt; which are made by the commercial &lt;a href=&quot;http://www.ansys.com/Products/Electronics/ANSYS-Maxwell&quot; rel=&quot;nofollow noreferrer&quot;&gt;ANSYS Maxwell&lt;/a&gt; software. Where to start for such a project? Which libraries are suitable to attack this problem by solving Maxwell's equations numerically? And how to model and implement the circuit geometry? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want a full solution but I am asking for a guide of how to attack this problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cNLuc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cNLuc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Müller, Rainer. &quot;A semiquantitative treatment of surface charges in DC circuits.&quot; American Journal of Physics 80.9 (2012): 782-788. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9417" LastEditorUserId="20688" LastEditDate="2019-06-22T18:55:48.827" LastActivityDate="2019-06-22T18:55:48.827" Title="Simulate electric fields due to surface charges in simple circuits using python" Tags="|finite-element|python|electromagnetism|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23166" PostTypeId="1" AcceptedAnswerId="23167" CreationDate="2016-02-19T01:52:44.543" Score="4" ViewCount="342" Body="&lt;p&gt;Specifically, how does one go about writing code that works with, say, both MPICH and OpenMPI?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently in the process of cleaning up the build scripts and code for a distributed-memory stochastic mixed-integer linear programming package we aim to release this Spring. One of idiosyncrasies of the code (or rather, one of the libraries we rely on from our collaborators) is that it will only work with MPICH, and it will not build or run with OpenMPI. What sort of things should I look to be implementing in an MPI implementation-independent fashion? (e.g., a coworker has pointed out that OpenMPI and MPICH represent communicators differently.) My goal is to try and patch the software so it will work with both OpenMPI and MPICH, and any high-level guidance about how to do that (hopefully with some references) would be extremely helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Here's an example error message from the compiler (OS X, Clang from Xcode 7.2):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PIPS/PIPS-IPM/Core/QpStoch/sTree.C:147:9: error: no matching function for call to 'MPI_Comm_create'&#xA;        ierr = MPI_Comm_create(mpiWorldGroup, childGroup, &amp;amp;childComm); assert(ierr==MPI_SUCCESS);&#xA;               ^~~~~~~~~~~~~~~&#xA;/usr/local/Cellar/open-mpi/1.10.1_1/include/mpi.h:1271:20: note: candidate function not viable: cannot convert argument of incomplete type 'MPI_Group' (aka 'ompi_group_t *') to 'MPI_Comm' (aka 'ompi_communicator_t *')&#xA;OMPI_DECLSPEC  int MPI_Comm_create(MPI_Comm comm, MPI_Group group, MPI_Comm *newcomm);&#xA;                   ^&#xA;1 error generated.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I should make more clear: the errors are in a library I am helping some collaborators get ready for a new release. I was not the original author, but I have patched up the code to add functionality I needed, plus I helped improve the build system. So this question is really geared more towards the situation where a developer is somewhat familiar with MPI, can write MPI-standard-compliant code, but is now trying to debug someone else's code and replace it with MPI-standard-compliant code. I'm looking for common pitfalls, basically.&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2016-02-19T17:41:25.147" LastActivityDate="2016-02-19T18:08:11.287" Title="How does one write MPI-implementation-independent code?" Tags="|mpi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23167" PostTypeId="2" ParentId="23166" CreationDate="2016-02-19T01:54:28.923" Score="6" Body="&lt;p&gt;Always use the correct types as specified by the standard. MPI_Comm for your communicators not int unless you're in Fortran. Etc., etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should be relatively easy. What problems are you really having?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edited to add in response to update: It looks like the type of &lt;code&gt;childGroup&lt;/code&gt; is not an &lt;code&gt;MPI_Group&lt;/code&gt; but an &lt;code&gt;MPI_Comm&lt;/code&gt;, so you should either fix that or use a temporary &lt;code&gt;MPI_Group&lt;/code&gt; to extract the group from the communicator locally. It looks like these two types are typdefed to int or something in OpenMPI, so the code &quot;works&quot; there and not under MPICH where they may each be their own opaque structures.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2016-02-19T18:08:11.287" LastActivityDate="2016-02-19T18:08:11.287" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23168" PostTypeId="2" ParentId="23166" CreationDate="2016-02-19T03:33:23.313" Score="6" Body="&lt;p&gt;Bill Barth already gave great advice. My advice is to not read the MPICH or OpenMPI documents. Read the MPI standard instead. As far as standards are concerned, it's actually quite readable. Implementations are, in my experience, quite conforming to the standard, but they add and extend them. These extensions are part of the problem that makes code incompatible.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-02-19T03:33:23.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23169" PostTypeId="2" ParentId="23164" CreationDate="2016-02-19T05:23:15.113" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Ansys#ANSYS_Maxwell&quot; rel=&quot;nofollow&quot;&gt;ANSYS Maxwell&lt;/a&gt; is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_element_method&quot; rel=&quot;nofollow&quot;&gt;Finite Element&lt;/a&gt; solver for Electromagnetism. So, I assume that you are looking for a Finite Element package that has a Python interface (or is written in Python).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some popular options like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sfepy.org/doc-devel/index.html#&quot; rel=&quot;nofollow&quot;&gt;SfePy&lt;/a&gt;;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://fenicsproject.org/&quot; rel=&quot;nofollow&quot;&gt;FEniCS&lt;/a&gt;; and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.agros2d.org/&quot; rel=&quot;nofollow&quot;&gt;Agros2D&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The last one provides a Python interface and a (nice) graphic interface, and is based in Hermes2D (so it supports hp- refinement). I would suggest Agros2D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, there are many more programs that can be used with Python. You can just make a search for &quot;Finite Elements Python&quot;.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2016-02-19T15:48:00.550" LastActivityDate="2016-02-19T15:48:00.550" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23170" PostTypeId="1" AcceptedAnswerId="23171" CreationDate="2016-02-19T12:12:25.703" Score="2" ViewCount="309" Body="&lt;p&gt;Suppose I want to numerically solve for $f(x,k)$ the one-dimensional Boltzmann equation for electrons in steady-state condition, given as:&#xA;\begin{equation}&#xA;\left( \dfrac{\hbar k}{m} \right) \dfrac{\partial f}{\partial x}&#xA;+ E \dfrac{\partial f}{\partial k} = \dfrac{f-f_0}{\tau}&#xA;\end{equation}&#xA;where all other symbols represent constant values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to discretize the equation, I have defined the $(x,k)$ grid and&#xA;I identify the coordinates of real space and momentum space with $x[i]$ and $k[j]$, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left( \dfrac{\hbar k_j}{2m} \right)&#xA;\Big[f(i+1,j) - f(i-1,j)\Big] \Delta k + &#xA;\dfrac{E}{2} \Big[ f(i,j+1) - f(i,j-1) \Big]\Delta x &#xA;= \dfrac{f(i,j) - f_0(i,j)}{\tau} \Delta x \Delta k&#xA;\end{equation}&#xA;Now, the above matrix equation gives a left-hand side matrix which is not stable for arbitrary values of constants and results in very ugly oscillations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen that many people use a so-called &quot;staggered grid&quot; (consisting of direct/adjoint grid nodes), or use various splitting methods (even/odd or positive/negative splitting) in order to solve this problem. The more I read on these stabilization schemes, the more confused I get about the main idea. Can somebody help me with an easy-to-understand explanation or reference on the basic idea and the details of its implementation in such cases?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-02-19T12:43:48.903" Title="Stability in discretization of a PDE" Tags="|finite-difference|stability|discretization|numerics|operator-splitting|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23171" PostTypeId="2" ParentId="23170" CreationDate="2016-02-19T12:43:48.903" Score="3" Body="&lt;p&gt;Analogous equations are considered in different applications e.g. stationary advection equation with right hand side. Looking to your equation through this view, you should try one-sided finite differences instead of central differences that are known to produce oscillations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance if your input data are such that &#xA;$$&#xA;\dfrac{\hbar k}{m} \ge 0 \,, \quad E \ge 0&#xA;$$&#xA;then you should use backward differences obtaining&#xA;\begin{equation}&#xA;\left( \dfrac{\hbar k_j}{m} \right)&#xA;\Big[f(i,j) - f(i-1,j)\Big] \Delta k + &#xA;E \Big[ f(i,j) - f(i,j-1) \Big]\Delta x &#xA;= \dfrac{f(i,j) - f_0(i,j)}{\tau} \Delta x \Delta k&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If those coefficients have different signs, choose forward differencing. I expect that the oscillations will disappear, otherwise the other data may produce the difficulties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only drawback is that the suggested &quot;upwind&quot; method in this answer is only first order accurate, so you may need to use fine grids to obtain satisfactory results. If this is an issue for you, you can use one-sided finite difference formulas of second order accuracy, see e.g. wikipedia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. The staggered grids are used, to my experience, when you have more than one unknown function, e.g. at least two PDEs.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-02-19T12:43:48.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23172" PostTypeId="1" CreationDate="2016-02-19T12:50:27.983" Score="2" ViewCount="1978" Body="&lt;p&gt;I am trying to figure out at exactly what dimension it is better to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Strassen_algorithm&quot; rel=&quot;nofollow&quot;&gt;Strassen algorithm&lt;/a&gt; rather than the standard multiplication. I know that there is 18 addition and subtraction in Strassen algorithm but I don't know how I must calculate it. This is a homework so I don't want direct answer without any explanation. Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19347" LastEditorUserId="9667" LastEditDate="2016-02-19T15:50:19.183" LastActivityDate="2016-02-19T15:50:19.183" Title="strassen algorithm vs. standard multiplication for matrices" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="23173" PostTypeId="2" ParentId="23172" CreationDate="2016-02-19T14:46:43.560" Score="7" Body="&lt;p&gt;The answer to this question depends heavily on the particular details of the computer that you're using.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In modern implementations, conventional matrix multiplication implementation (in the form of highly optimized versions of the BLAS xGEMM function) use blocked algorithms that are carefully tuned to match the cache size of the processor.  In comparison, Strassen's algorithm is extremely cache unfriendly, and this makes it difficult to get good performance on contemporary processors.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One recent Arxiv preprint claims that a Strassen like algorithm can give modest performance improvements (up to about 25% faster) than Intel's MKL for matrices of size $N=3000$ or larger on an Intel processor based system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Austin R. Benson and Grey Ballard.  A Framework for Practical Parallel Fast Matrix Multiplication. &lt;a href=&quot;http://arxiv.org/pdf/1409.2908.pdf&quot;&gt;http://arxiv.org/pdf/1409.2908.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's another important issue to consider here- Strassen like algorithms are less numerically accurate than conventional blocked matrix-matrix multiplication.  In some applications that inaccuracy can cause problems at the application level.  Because of this, and because the performance improvements are not that large for reasonably sized matrices, few people make use of Strassen like algorithms in practice.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-02-19T14:46:43.560" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23174" PostTypeId="1" CreationDate="2016-02-19T18:13:28.513" Score="1" ViewCount="1061" Body="&lt;p&gt;To solve the heat equation from implicit scheme subject to Neumann boundary condition we can write:&#xA;$$&#xA;T_i^{j+1}-T_i^{j}=\alpha (T_{i+1}^{j+1}-2T_{i}^{j+1}+T_{i-1}^{j+1})&#xA;$$&#xA;$$&#xA;\textbf{A} T^{n+1} = T^n &#xA;$$&#xA;$$&#xA;-\alpha T_{i-1}^{n+1}+(1+2 \alpha)T_i^{n+1}-\alpha T_{i+1}^{n+1} =T_i^n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initial and boundary conditions are :&#xA;$$ &#xA;T(L/4,t)=100 \quad ,   \frac{\partial T(0,t)}{\partial x} =\frac{\partial T(L,t)}{\partial x} =0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where alpha is $kdt/dx^2$, $k$ is the coefficient of thermal conductivity and L is the length of the rod. from the boundary condition we have :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T(2,j)= T(0,j)&#xA;T(n+1,j)=T(n-1,j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so we have a tridiagonal matrix with main diagonal of ($1+2\alpha$) and subdiagonal and superdiagonal of $-\alpha$. The first and last row of the matrix has changed by Neumann.B.C to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A(1,1)    A(1,2)  :    1+2\apha    -2\alpha&#xA;A(n,n-1)  A(n,n)  :   -2\alpha     1+2\alph&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is my implementation of the problem in FORTRAN:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program heat_rod&#xA;!&#xA;! To solve the heat eauation for a rod with &#xA;! Tt(x,t) =sigma* Txx(x,t)&#xA;! T(x,t) = 0 and T(L/4,t)= 100, &#xA;! N.B.C : Tx(0,t) = Tx(L,t) = 0&#xA;! x  over [0,1] and t over [0,0.1]&#xA;! Variables :&#xA;! dl, dd, du subdiagonal, maindiagonal and superdiagonal elements respectively&#xA;! To plot the result:&#xA;! echo 'splot &quot;data&quot; nonuniform matrix with lines' | gnuplot --persist&#xA;&#xA;implicit none&#xA;&#xA;integer,parameter :: n = 11, m = 31      ! n=21, m=81&#xA;integer :: i,j,index_x&#xA;real(8) :: alph,l,time&#xA;real(8) :: dl(n),dd(n),du(n),d_x,d_t,sigma&#xA;real(8) :: T_new(n),T_old(n)&#xA;&#xA;&#xA;open(unit=7, file=&quot;T.txt&quot;)&#xA;&#xA;l = 1.d0&#xA;time = 1.d-1&#xA;d_x = l/real(n-1)&#xA;d_t  = time/real(m-1)&#xA;sigma = 1.d0 ! coefficient of termal conductivity&#xA;alph = (sigma * d_t)/(d_x*d_x)&#xA;!print *, alph&#xA;if (alph &amp;gt; 0.5) then&#xA;print*, &quot;Solution may be unstable&quot;&#xA;!stop&#xA;endif&#xA;&#xA;du = -alph&#xA;dl = -alph&#xA;&#xA;!du(1) = 0.d0         !Dirichlet.B.C&#xA;du(1) = -2.d0 * alph  !Neumann B.C&#xA;!dl(n) = 0.d0         !Dirichlet B.C&#xA;dl(n) = -2.d0 * alph  !Neumann B.C&#xA;&#xA;dd = 1.d0 + 2.d0 * alph&#xA;!dd(1) = 1.d0&#xA;!dd(n) = 1.d0&#xA;!dd(1) = 1.d0 + 2.d0 * alph&#xA;!dd(n) = 1.d0 + 2.d0 * alph&#xA;&#xA;T_old = 0.d0&#xA;index_x = l/4 *n&#xA;!print *, index_x&#xA;T_old(index_x) = 100.&#xA;!print *,T_old&#xA;&#xA;! print the x positions at the first row&#xA;write(7,120) n,((i-1)*d_x,i=1,n)&#xA;120 format (I3,999f12.2)&#xA;&#xA;write(7,110) 1,(T_old(i),i=1,n)&#xA;&#xA;do j = 1,m&#xA;   call tridag(dl,dd,du,T_old,T_new,n)&#xA;   write(7,110) (j)*d_t ,(T_new(i),i=1,n)&#xA;   T_old = T_new&#xA;enddo&#xA;&#xA;! Printing the results&#xA;!write(*,120) ((i-1)*h,i=2,nx-1)&#xA;!120 format (7x,' t',7x,'x = ',f5.2,9f11.2)&#xA;!do j = 1,m&#xA;!    write(7,110) (T_plot(i,j),i=1,n)&#xA;!enddo&#xA;110 format (f6.3,999f12.6)&#xA;&#xA;!close(7)&#xA;&#xA; contains&#xA;&#xA;SUBROUTINE tridag(a,b,c,r,u,n)&#xA;INTEGER n,NMAX&#xA;REAL(8) :: a(n),b(n),c(n),r(n),u(n)&#xA;PARAMETER (NMAX=500)&#xA;! Solves for a vector u(1:n) of length n the tridiagonal linear set &#xA;! a(1:n) , b(1:n) , c(1:n) , and r(1:n) are input vectors and are not modified.&#xA;! Parameter: NMAX is the maximum expected value of n .&#xA;INTEGER j&#xA;REAL(8) :: bet,gam(NMAX)      !One vector of workspace, gam is needed.&#xA;if(b(1).eq.0.)pause 'tridag: rewrite equations'&#xA;!If this happens then you should rewrite your equations as a set of order N − 1, with u2&#xA;!trivially eliminated.&#xA;bet=b(1)&#xA;u(1)=r(1)/bet&#xA;do  j=2,n&#xA;!Decomposition and forward substitution.&#xA;gam(j)=c(j-1)/bet&#xA;bet=b(j)-a(j)*gam(j)&#xA;if(bet.eq.0.)pause 'tridag failed'&#xA;!Algorithm fails; see below.&#xA;u(j)=(r(j)-a(j)*u(j-1))/bet&#xA;enddo &#xA;do  j=n-1,1,-1&#xA;!Backsubstitution.&#xA;    u(j)=u(j)-gam(j+1)*u(j+1)&#xA;enddo &#xA;return&#xA;END SUBROUTINE&#xA;end program&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8cDFD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8cDFD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I doubt about the the way I defined the boundary conditions.&#xA;How can I validate the simulation results? Thanks in advance for your guide or comment.&lt;/p&gt;&#xA;" OwnerUserId="9994" LastEditorUserId="9994" LastEditDate="2016-02-20T11:38:20.233" LastActivityDate="2016-02-20T11:38:20.233" Title="Heat equation from implicit scheme with Neumann B.C" Tags="|finite-difference|numerical-analysis|fortran|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23176" PostTypeId="1" CreationDate="2016-02-20T11:21:57.807" Score="1" ViewCount="345" Body="&lt;p&gt;I can not undrestand when we have jump on interior boundaries, if the solution be continuous. In  the interior boundaries, derivatives have only different unit outward normal vector and this means we can discard all interior derivatives terms. But in some text i see that they are different and the notation of $[\frac {\partial u}{\partial n}]=\frac {\partial u^+}{\partial n}-\frac{\partial u^-}{\partial n}$ is used. Is it related to the test space?&#xA;In fact, I am working on adaptive finite element methods for Poisson's equation, based on the book: &lt;em&gt;The finite element method,Theory,implementations and applications  written by M. larson and F. Bengzon,&lt;/em&gt;  and on this book to find a posteriori error estimate for this equation by linear fem, they used jump for $\frac {\partial u}{\partial n}$&#xA;$$\eta_k=h_k\| f+\Delta u_h\|_{L^2(k)}+\frac{1}{2}h_k^{\frac{1}{2}}\|[n.\nabla u_h]\|_{L^2({\partial k}-\partial{\Omega})}$$&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2016-02-22T17:39:07.880" LastActivityDate="2016-02-23T22:54:17.140" Title="Jump conditions on interior boundaries" Tags="|finite-element|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23177" PostTypeId="1" CreationDate="2016-02-20T21:22:42.583" Score="1" ViewCount="173" Body="&lt;p&gt;I am looking for an algorithm to search a substring in a string.  I know there are a number of wellknown Algorithms like Knuth-Morris-Pratt, for instance, and I suppose most preimplemented  functions use one of those. However, I remember a lecture, long ago, where the lecturer said that the usefullness of such an algorithm depends partly on the size of the alphabet. KMP and BMH obviously work fine on an ordinary 26-letter alphabet. But what do I do if I have a much smaller alphabet (say, DNA: 4 letters or simply a binary)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there an particular algorithms that work well on very small alphabets? &#xA;I googled and supposed I should easily find something as searching through DNA is rather common, but to no avail. Any help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please do not downvote, this is my first question in this subcommunity, I do not know yet what is considered too basic here and what is ok.&lt;/p&gt;&#xA;" OwnerUserId="19366" LastEditorUserId="9667" LastEditDate="2016-02-20T23:30:15.233" LastActivityDate="2016-02-20T23:30:15.233" Title="String search algorithm on small alphabet" Tags="|algorithms|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23178" PostTypeId="1" CreationDate="2016-02-20T22:01:32.290" Score="3" ViewCount="713" Body="&lt;p&gt;I have to write a program where I have to perform matrix-vector multiplication and the matrix is sparse matrix. Most sparse matrices available online are in Harwell-Boeing format and they have to be read in compressed sparse row format in order to do operations on it. How should I read such a file and convert into compressed sparse row format in a C programming framework?&lt;/p&gt;&#xA;" OwnerUserId="17783" LastActivityDate="2016-02-20T23:01:38.227" Title="How to read a Harwell-Boeing Matrix file format into a compressed sparse row format in a C program?" Tags="|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23179" PostTypeId="2" ParentId="23127" CreationDate="2016-02-20T22:51:17.930" Score="2" Body="&lt;p&gt;I want to submit a separate answer to this question, because there is a way to &lt;em&gt;automatically&lt;/em&gt; simplify such formulas to remove square roots. I will use sage and &lt;a href=&quot;http://www.usna.edu/CS/qepcadweb/B/QEPCAD.html&quot; rel=&quot;nofollow&quot;&gt;QEPCAD&lt;/a&gt; (both free). &lt;a href=&quot;http://doc.sagemath.org/html/en/reference/interfaces/sage/interfaces/qepcad.html&quot; rel=&quot;nofollow&quot;&gt;The tutorial is here.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inequality &#xA;$$ |\sqrt{a}-\sqrt{b}|&amp;lt;t $$&#xA;is equivalent to the statement that there exist $u, v$ such that&#xA;$$ \exists u,v: \qquad (u-v)^2 &amp;lt; t^2, \quad u^2 = a, \quad\text{and}\quad v^2 = b. $$&#xA;These are quantified polynomial in-/equalities and can be simplified with QEPCAD using the following simple session in sage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var('a,b,u,v,t')&#xA;qepcad(qf.E([u,v], qf.and_((u-v)^2 &amp;lt; t^2, u^2 == a, v^2 == b)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output gives the equivalent conditions for the original inequality to hold:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a &amp;gt;= 0 /\ b &amp;gt;= 0 /\ [ t^4 - 2 b t^2 - 2 a t^2 + b^2 - 2 a b + a^2 &amp;lt; 0 \/ t^2 - b - a &amp;gt; 0 ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In other words, it is equivalent to &#xA;$$ (t^4 - 2 b t^2 - 2 a t^2 + b^2 - 2 a b + a^2 &amp;lt; 0) \lor (t^2 - b - a &amp;gt; 0). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nice thing about this approach is that it is free from human error.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-02-20T22:51:17.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23180" PostTypeId="2" ParentId="23178" CreationDate="2016-02-20T23:01:38.227" Score="4" Body="&lt;p&gt;Tim Davis' SuiteSparse package has a function for reading matrices in that&#xA;format:&#xA;&lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot; rel=&quot;nofollow&quot;&gt;http://faculty.cse.tamu.edu/davis/suitesparse.html&lt;/a&gt;&#xA;It appears that you have to download the entire SuiteSparse package but if all&#xA;you want is to read Harwell-Boeing files, you really need only one routine&#xA;from that package: RBio/Source/RBio.c. Note, you will also need a couple of&#xA;include files that are also part of the package in order to compile that function.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-02-20T23:01:38.227" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23181" PostTypeId="1" CreationDate="2016-02-22T00:01:20.563" Score="10" ViewCount="256" Body="&lt;p&gt;I am looking for credible references stating how much resources supercomputers spend on coordinating versus doing actual task-related work. Resources could be available processing power but even Watts seem like a valid unit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe one of my professors or text books once said that in massively parallel systems, up to half of the available processing power is spent on coordinating the task and message passing. Unfortunately, I can't seem to find this reference or any other material about this proportion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize this will differ a lot depending on the supercomputer architecture and modern implementations probably are more efficient in this regard, so an overview of this metric across multiple architectures or evolutions (before and after dedicated message passing hardware) would be even better.&lt;/p&gt;&#xA;" OwnerUserId="19372" LastActivityDate="2016-02-22T16:26:35.330" Title="Communication overhead in supercomputing" Tags="|hpc|mpi|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23182" PostTypeId="1" CreationDate="2016-02-22T02:03:52.293" Score="0" ViewCount="111" Body="&lt;p&gt;I implemented an $LU$ factorization algorithm on c++ for a $n$ by $n$ matrix $A$, and I wanted to now write a routine that solves the systems $Lv = f$ and $Uv = f$. How would you recommend writing such a routine. I am aware of using a linear algebra package on c++ in which I could take the inverse of $L$ and $U$ and solve for $v$ to make the expressions equal to. Although, I was wondering if there is a simpler way of solving a system like this in c++. Any suggestions is greatly appreciated. I made a post to this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/23147/lu-factorization/23157#23157&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-02-22T04:05:34.433" Title="$LU$ factorization routine to solve system $Lv = f$ and $Uv = f$" Tags="|numerical-analysis|c++|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23183" PostTypeId="2" ParentId="23182" CreationDate="2016-02-22T02:48:04.683" Score="3" Body="&lt;p&gt;The basic strategy here is to take advantage of the known structure of the two matrices (lower and upper triangular respectively). This means that the inversion can be found efficiently through forward/backward substitution. For example, the operation of the &lt;em&gt;first&lt;/em&gt; row of $L$ on $v$ gives $L_{00}x_0=f_0$, which (for the cases you've been told to consider in which pivoting is unnecessary) can be solved through simple scalar division. Given your knowledge of $x_0$, you can then generate $\tilde{f}_i=f_i-L_{i0}x_0$ for  $i\in [1,N-1]$. At the end of this process you have N-1 unknowns, an N-1 dimensional right hand side and an $N-1\times N-1$ lower triangular matrix to work with. This recursion forms the outer loop of the solve. The key is to store $L$ and $U$ such that the substitution process accesses contiguous static memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you haven't already, I strongly recommend reading the relevant subsection of Numerical Recipes by Press et al. The latest edition is even in C++. I would also suggest adding the homework tag, as you have in your previous question. &lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-02-22T02:48:04.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23184" PostTypeId="2" ParentId="23181" CreationDate="2016-02-22T03:14:04.453" Score="10" Body="&lt;p&gt;A long standing favorite benchmark in high performance computing has been the HPLinpack benchmark, which measures the speed of a computer system in floating point operations per second while solving a very large, dense, linear system of equations.  It is assumed that the solution takes $2/3n^{3}+2n^{2}$ floating point operations and the tester is allowed to vary $n$ to achieve maximum performance.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The benchmark measures include RPEAK (the theoretical maximum number of floating point operations per second for the system) and RMAX (the maximum achieved number of operations per second in the HPLinpack benchmark.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's typical for RPEAK to be a substantial fraction of RMAX, indicating that on this benchmark task, current supercomputers can achieve a significant fraction of their theoretical peak performance.  For example, in the November 2015 TOP500 supercomputer rankings, the fastest machine,  Tianhe-2, has RPEAK=54.902 petaflops and RMAX=33.863 petaflops.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the HPLinpack benchmark is widely viewed as not being representative of current workloads.  HPlinpack results typically overstate the performance of supercomputers in actual applications by a large factor.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A new benchmark, called HPCG, is under development.  This benchmark involves operations commonly performed in iterative methods for the solution of large sparse systems of equations arising from discretized PDE's.  This workload is far more challenging for high performance computers.  It's also much more representative of what supercomputers are used for in practice.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some early results from HPCG are coming in at less than 5% of RPEAK.  For example, Tianhe-2 has RPEAK=54.902 petaflops and HPCG at 0.58 petaflops (see reference below to a presentation on HPCG.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The TOP500 HPLinpack benchmarks can be found at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.top500.org/&quot;&gt;http://www.top500.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A presentation on HPCG can be found at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hpcg-benchmark.org/downloads/isc15/HPCG-ISC15-FINAL-SLIDES_update1.pdf&quot;&gt;http://www.hpcg-benchmark.org/downloads/isc15/HPCG-ISC15-FINAL-SLIDES_update1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HPCG's website is at &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.hpcg-benchmark.org/&quot;&gt;http://www.hpcg-benchmark.org/&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-02-22T03:14:04.453" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23185" PostTypeId="1" CreationDate="2016-02-22T04:14:18.923" Score="1" ViewCount="518" Body="&lt;p&gt;I want to solve two dimensional Navier Stokes equations on a staggered grid for the case of Taylor-Green Vortex. My initial conditions are standard sine and cosine functions. As I am aware, I should use periodic boundary conditions on all the four boundaries. I am facing some trouble regarding implementation of this BC as I could not find enough literature about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose, if my index ranges from $0$ to $N$. Then, is this the correct way to use periodic BC?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_\text{vel}(N) = u_\text{vel}(1)\\&#xA;u_\text{vel}(0) = u_\text{vel}(N-1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and similarly for $v_\text{vel}$ and pressure variables?&lt;/p&gt;&#xA;" OwnerUserId="16397" LastEditorUserId="9667" LastEditDate="2016-02-22T04:25:42.607" LastActivityDate="2016-02-22T04:25:42.607" Title="Periodic boundary conditions for solving Navier Stokes Equations on a Staggered Grid" Tags="|finite-difference|fluid-dynamics|boundary-conditions|navier-stokes|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23186" PostTypeId="2" ParentId="23147" CreationDate="2016-02-22T10:36:08.080" Score="0" Body="&lt;p&gt;Have you tried running your code? I think it will segfault. The routine &quot;banded_matrix&quot; twice sets the same values of data, and then in the 3rd loop sets a couple of non-existing elements.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2016-02-22T10:36:08.080" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23187" PostTypeId="2" ParentId="21446" CreationDate="2016-02-22T11:57:14.143" Score="2" Body="&lt;p&gt;So this answer is supposed to help others trying to implement the shift operators themselves while experiencing delimited knowledge about the method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fast multipole method's algorithm is dependent upon the following steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- At the finest level, start calculating multipole expansions in each and every partition. Usually you take the expansion at the center of that partition - but maybe for certain applications one might venture and try different locations? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Start the upward pass. This is just the aggregation of the previously calculated expansions and translating them up the tree until the coarsest level is reached. The aggregation is just the multipole-to-multipole shift from a current expansion situated at this level to it's parent. Usually if the partitioning is based on an octree then each 8 children are summed after translation to their corresponding parent. And this goes on all the way reaching level 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3- Start the downward pass. In this pass there are two shift operations involved: multipole-to-local and local-to-local. The multipole-to-local shift starts from level 2 and goes all the way down to the finest level. In this translation, which by the way is the most computationally demanding part of the FMM algorithm, we take into consideration the effect of all the expansions in an interaction list of every single expansion we are trying to compute and perform the operation. This step by itself is a bit complicated to explain here, but I suppose you have the picture by now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;4- Now the second step of the downward pass: local-to-local shift operation. This is simply the segregation (a more descriptive word here might be: decomposition) of each parent into its children. This starts at the coarsest level and goes all the way down to the finest. Also one must be careful to add the M2L and L2L contribution to one another for each expansion in the same cube/partition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;5- Finally evaluation of the potential field is put into practice. This has two options: far-field potential evaluation and near-field potential evaluation. They depend on the location of the source points with respect to each of the target points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This is merely a very simple overview of the translation theory utilized within the FMM. The sole aim of this explanation is to help beginners become more familiar with the details and to point them to the right track using relatively simple words.&lt;/em&gt; &lt;strong&gt;For more details, please check out the reference included in the original question above.&lt;/strong&gt;  &lt;/p&gt;&#xA;" OwnerUserId="18387" LastEditorUserId="18387" LastEditDate="2016-03-04T20:50:20.390" LastActivityDate="2016-03-04T20:50:20.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23188" PostTypeId="2" ParentId="23181" CreationDate="2016-02-22T16:26:35.330" Score="6" Body="&lt;p&gt;The honest answer is that we don't know. The answer depends heavily on what is actually being run and what code the user has written. As Brian Borchers points out, there's a big difference between two benchmarks where we have all the code and supposedly know what that code is doing, but there's much disagreement about how representative this code is of what users of supercomputers are actually doing. Without detailed source code analysis and some heavy instrumentation of real codes on real machines, finding this ratio is next to impossible. There are some projects that are starting to collect data that might be able to get the community close to answering this question, but it is not at all settled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, the question isn't really even clear. If a cluster node's communication card has a processor on it which can only be used for communication, how do you count the time that this card spends idle not handling communication (nor anything else)? I.e., what counts as &quot;available processing power&quot;? Do we count badly written programs which have unoptimized compute and communications routines the same as optimized? What if someone uses a known anti-pattern in their code which deliberately under-utilizes the CPU? What about embarrassingly parallel programs that do not communicate at all (these do get run on supercomputers, I promise you)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't waste your time trying to quantify an off-the-cuff remark in a book or from your professor. These kinds of statements are there to remind us that parallel programming is hard and generally done poorly. Supercomputers also aren't perfectly designed to eliminate or optimize away all the waste. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-02-22T16:26:35.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23189" PostTypeId="2" ParentId="19689" CreationDate="2016-02-22T18:14:43.260" Score="2" Body="&lt;p&gt;You say you use an explicit time stepping scheme for convection. Then you don't need a non-linear solver at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{u_i^{n+1}-u_i^n}{\Delta t} + u_j^n \frac{\partial u_i^n}{\partial x_j} + \frac{\partial \tilde{p}}{\partial x_i}- \nu \nabla^2 u_i^{n+1} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where subscripts denote spatial coordinate directions (i = 1,2,3 in 3D), and superscripts, the time step. This scheme is consistent, and is called a semi-implicit scheme. The non-linear term can be transferred to the right hand side and just becomes part of the residual at each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a fully implicit scheme without Newton iterations, you could use the &quot;characteristics-Galerkin&quot; method. This is a semi-Lagrangian method in which you approximate the total derivative implicitly:&#xA;$$&#xA;\frac{Du}{Dt}|^{n+1} \approx \frac{u^{n+1} - u^n \circ X(x,t^n)}{\Delta t}&#xA;$$&#xA;where $X$ denotes the characteristic passing through $(x,t^{n+1})$, which can be computed, for instance, by solving an ODE. (So finally this may not be much cheaper than a Newton method.) For details, you can check out&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mofdi El-Amrani and Mohammed Seaid, &quot;An L2-projection for Galerkin-characteristic solution of incompressible flows&quot;, SIAM J. Sci. Comput., Vol 33 No 6, pp 3110-3131, 2011.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can try out the method in FreeFEM++, if you want to. It has a convenient &quot;convect&quot; operator for this.&lt;/p&gt;&#xA;" OwnerUserId="5541" LastActivityDate="2016-02-22T18:14:43.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23190" PostTypeId="1" CreationDate="2016-02-23T00:57:40.090" Score="3" ViewCount="129" Body="&lt;p&gt;I am trying to solve $Ax=b$ using the conjugate gradient method. However, it is important to me to obtain a bound not only on the usual residual $||b-Ax_k||_2$ but also on the quantity $||b-Ax_k||_1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it true that $||b-Ax_k||_1 \leq C ||b-Ax_0||_1$ for all iterations $k$ where $C$ is some constant? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In other words, is it true that the conjugate gradient amplifies $1$-norm of the residual by at most a constant factor?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we can choose $C=\sqrt{n}$ (where $A \in \mathbb{R}^{n \times n}$) then the above fact is trivial since its true with the 1-norm replaced by the 2-norm and using the bound $||\cdot||_1 \leq \sqrt{n} ||\cdot||_2$. However, I'd like to $C$ to be a constant not depending on dimension $n$. &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;If the answer to question one is negative, is there some modification of the conjugate gradient which achieves such a bound?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19379" LastActivityDate="2016-02-23T00:57:40.090" Title="Conjugate gradient: the 1-norm of the residual" Tags="|linear-algebra|conjugate-gradient|krylov-method|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23191" PostTypeId="1" CreationDate="2016-02-23T09:45:09.080" Score="10" ViewCount="861" Body="&lt;p&gt;This might be the wrong stackexchange site for this question.  math.SE, cs.SE, programmers.SE, and of course stackoverflow are all possibilities.  I'm hoping to reach an audience that might actually &lt;em&gt;use&lt;/em&gt; this function, to get some feedback on how they'd like it to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C's &lt;a href=&quot;http://en.cppreference.com/w/c/numeric/math/nextafter&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;double nextafter(double x, double y)&lt;/code&gt;&lt;/a&gt; function returns the next representable value from x, in the direction of y.  (commonly + or - infinity, but can be anything).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm tidying up GNU libc/libm's &lt;a href=&quot;https://www.sourceware.org/git/gitweb.cgi?p=glibc.git;a=blob;f=math/s_nextafter.c;h=dfa5e860c40f7ae31843d13bd48d183378a4fc6b;hb=HEAD&quot; rel=&quot;noreferrer&quot;&gt;implementation&lt;/a&gt; to compile to nicer code.  (It currently extracts each double to two 32bit integers.  It's pretty clunky and uses lots of branches, and doesn't take advantage of &lt;code&gt;int64_t&lt;/code&gt;.).  If I'm going to change it at all, I might as well make as much improvement as possible, but using an FP compare-equal would change the results in DAZ mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How should it behave when the denormals-are-zero and/or flush-to-zero are active?&lt;/strong&gt;  On x86, &lt;a href=&quot;http://www.cita.utoronto.ca/~merz/intel_c10b/main_cls/mergedProjects/fpops_cls/common/fpops_set_ftz_daz.htm&quot; rel=&quot;noreferrer&quot;&gt;there are two separate flags&lt;/a&gt; which sacrifice gradual underflow for performance.  By default, neither is set.  One or both can be set (they're often used together, but don't have to be):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;DAZ is an input filter: When an FP math / compare instruction reads its inputs, denormals are considered +/-0.0.  So a compare between two denormals finds they're equal.  Arithmetic can easily produce denormal results, though.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;FTZ is an output filter: Denormal inputs work normally, with no effect on compare instructions.  However, ordinary math instructions can't generate denormals.  Denormal results are flushed to +/-0.0.  &lt;code&gt;DBL_MIN/2 + DBL_MIN/1.5&lt;/code&gt; should give the same result as without FTZ (e.g. if those input denormals are loaded as constants).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In both cases (esp DAZ), one might argue that the next representable value below &lt;code&gt;DBL_MIN&lt;/code&gt; (the smallest normal number) was &lt;code&gt;+0.0&lt;/code&gt;, and that the next number below &lt;code&gt;+0.0&lt;/code&gt; or &lt;code&gt;-0.0&lt;/code&gt; was &lt;code&gt;-DBL_MIN&lt;/code&gt;.  Implementing this would require checking the FTZ/DAZ flags.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might argue instead that in DAZ mode, two different denormals should be considered equal, because they &lt;em&gt;do&lt;/em&gt; compare equal.  Thus, it's correct to return &lt;code&gt;y&lt;/code&gt; when &lt;code&gt;x&lt;/code&gt; equals &lt;code&gt;y&lt;/code&gt;, as required by &lt;a href=&quot;http://pubs.opengroup.org/onlinepubs/9699919799/functions/nextafter.html&quot; rel=&quot;noreferrer&quot;&gt;POSIX&lt;/a&gt;, and by &lt;a href=&quot;http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf&quot; rel=&quot;noreferrer&quot;&gt;the C11 standard&lt;/a&gt; (7.12.11.3 pg 256 and Annex F.10.8.3 pg 529).  (The motivation for this specification is for &lt;code&gt;nextafter(0.0, -0.0)&lt;/code&gt; to return &lt;code&gt;-0.0&lt;/code&gt;, and vice versa.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 2nd behaviour allows an extremely fast &amp;amp; compact implementation (no taken branches and 3 not-taken branches polluting the branch-predictor, and ~9 clock cycles latency on Intel Haswell (for comparison, an FP multiply takes 5 cycles) for the common case (x!=y, and x!=0, and the result doesn't overflow(infinity) or underflow (denormal)).  Less than half the total code size compared to glibc's current version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The C11 standard says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Even though underflow or overflow can occur, the returned value is independent of the&#xA;  current rounding direction mode.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But DAZ isn't a rounding &lt;em&gt;direction&lt;/em&gt; mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;boost::math::ulp&lt;/code&gt; function's documentation observes that &quot;our experience is that &lt;code&gt;std::nextafter&lt;/code&gt; often breaks depending which optimizations and hardware flags are in effect&quot;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Consistency with other implementations is desirable for glibc's &lt;code&gt;nextafter&lt;/code&gt; function, but consistency with glibc's existing behaviour may take precedence.  The current implementation is not influenced at all by any FP settings, because it uses only integer comparisons on the IEEE float bit patterns.  Consistency with &lt;code&gt;nexttoward&lt;/code&gt;, and with &lt;code&gt;nextafterf(float,float)&lt;/code&gt; / &lt;code&gt;nextafterl(long double, long double)&lt;/code&gt; is also important.  I haven't looked into the long double versions yet.  Integer compares are clunkier there, because &lt;code&gt;int64_t&lt;/code&gt; can't hold 80 bits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using integer-only compares is only slightly slower / bulkier to implement (using a 64bit integer compare, and the sign of x and y, instead of just an FP compare and the sign of x).  IEEE floats are cleverly designed so their bits can be compared as sign-magnitude integers (and thus as 2's complement integers with a check for the both-negative case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does &lt;code&gt;nextafter&lt;/code&gt; behave on other systems (OS X, MSVC++, anything else)?  If people are interested in testing it, I might write a little test program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you &lt;em&gt;wish&lt;/em&gt; &lt;code&gt;nextafter&lt;/code&gt; behaved?  (&lt;code&gt;nextafter&lt;/code&gt; is pretty much fully specified except for DAZ/FTZ, so wishes other than that might be better fulfilled by different functions.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did I miss anything?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I know the current implementation is &quot;good enough&quot; for such a rarely-used function.  It's ugly, but it's used so rarely that it doesn't have much impact.  (Actually, is it rarely used?  Has anyone ever seen code that used it a lot?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of the motivation here is that I enjoy optimizing things in assembly language, and integer ops on FP data is interesting, especially on x86-64 where integer instructions can be used on the vector registers that are also used for &lt;code&gt;float&lt;/code&gt;s and &lt;code&gt;double&lt;/code&gt;s (but not 80bit &lt;code&gt;long double&lt;/code&gt; for &lt;code&gt;nextafterl&lt;/code&gt;, or for the direction arg of &lt;code&gt;nexttowards&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="18583" LastEditorUserId="18583" LastEditDate="2016-05-29T01:24:57.773" LastActivityDate="2016-05-29T01:24:57.773" Title="Implementing std::nextafter: Should denormals-are-zero mode affect it? If so, how?" Tags="|c|floating-point|computer-arithmetic|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23192" PostTypeId="2" ParentId="16052" CreationDate="2016-02-23T10:58:39.450" Score="1" Body="&lt;p&gt;As long as you have accurate values for the roots (or extrema) of the oscillatory part of your integrand, Longman's method (as I described in &lt;a href=&quot;https://scicomp.stackexchange.com/a/64&quot;&gt;this answer&lt;/a&gt;) remains applicable. All you have to do is evaluate a bunch of integrals with intervals in between the roots using your favorite quadrature method, and treat these integrals as the terms of some alternating series. You can then use any number of convergence acceleration methods (Euler, Levin, Weniger, etc.) to &quot;sum&quot; this alternating series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, in &lt;a href=&quot;https://math.stackexchange.com/a/173927&quot;&gt;this math.SE answer&lt;/a&gt;, I evaluated an infinite integral whose oscillatory part is a product of two Bessel functions.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-02-23T10:58:39.450" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23193" PostTypeId="1" CreationDate="2016-02-23T11:45:35.650" Score="2" ViewCount="407" Body="&lt;p&gt;I just started working on my master thesis in the theoretical particle physics and my supervisor asked me to start working with &lt;a href=&quot;https://www.nikhef.nl/~form/&quot; rel=&quot;nofollow&quot;&gt;FORM&lt;/a&gt;, a script interpreter with basic knowledge of lorentz and dirac structures implemented, for the evaluation of some feynman diagrams. As my first glimpse on how FORM looks and works it doesn't seem to be very user friendly and intuitive. So I started to look around a bit and found that sympy, which I know the basics of, also has a &lt;a href=&quot;http://docs.sympy.org/latest/modules/physics/hep/gamma_matrices.html&quot; rel=&quot;nofollow&quot;&gt;module&lt;/a&gt; for lorentz indices and dirac matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So before I work more into one of these I thought I would seek some solid advice from someone who maybe has worked with both or was once confronted with the same decision. Has there ever been a benchmark of these and if FORM is the faster option is it worth the inconveniences? Is there maybe a third option considered superior over the two mentioned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought that would be on-topic here, but please correct me if I'm wrong.&lt;/p&gt;&#xA;" OwnerUserId="15261" LastActivityDate="2016-02-23T11:45:35.650" Title="Evaluation of feynman diagrams: FORM vs. Sympy vs. other" Tags="|software|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23194" PostTypeId="1" CreationDate="2016-02-23T14:42:14.043" Score="5" ViewCount="1583" Body="&lt;p&gt;I googled about some code on the complex polygamma function especially C++ code, but can't find anything. Does anyone know where to find such code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complex digamma function does exist but not the complex polygamma function.&lt;/p&gt;&#xA;" OwnerUserId="19384" LastEditorUserId="20688" LastEditDate="2019-05-17T19:21:59.440" LastActivityDate="2020-12-17T19:35:22.667" Title="I am searching for a C++ code implementing the complex polygamma function" Tags="|c++|special-functions|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="23195" PostTypeId="2" ParentId="23194" CreationDate="2016-02-23T17:04:34.680" Score="5" Body="&lt;p&gt;Looking at what the python scipy library does for its special functions, the polygamma is found by returning the digamma if the zeroth derivative is requested, otherwise return $(-1)^{n+1}\Gamma(n+1)\zeta(n+1,z)$ where $\Gamma$ is the gamma function and $\zeta$ the two argument Riemann zeta function. Assuming that this identity holds for the complex numbers, it may explain why you can't find any explicit code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact to add a reference, this is also the method used in the gnu implementation of libstdc++, see &lt;a href=&quot;https://gcc.gnu.org/onlinedocs/libstdc++/libstdc++-html-USERS-4.4/a01203.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="19263" LastEditDate="2016-02-23T19:11:14.303" LastActivityDate="2016-02-23T19:11:14.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23196" PostTypeId="1" AcceptedAnswerId="23197" CreationDate="2016-02-23T17:21:27.460" Score="0" ViewCount="73" Body="&lt;p&gt;I'm trying to express the following semidefinite program: for given $A \in R^{m \times n}$ and a scalar $\epsilon \in (0,1)$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\gamma_{2}^{\epsilon}(A):= \min\,t\\&#xA;&amp;amp;\text{subject to} \left( \begin{array}{ccc}&#xA;W_1 &amp;amp; B  \\&#xA;B^T &amp;amp; W_2  \\ \end{array} \right)\succcurlyeq 0,\, \operatorname{diag}(W_1)\leq t,\, \operatorname{diag}(W_2)\leq t,\\ &#xA;&amp;amp;\forall (i,j) \in \{1,...,m\} \times \{1,...,n\}: -\epsilon\leq A_{ij}-B_{ij},\, A_{ij}-B_{ij} \geq \epsilon.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $B \in R^{m \times n} ,\, W_1 \in R^{m \times m}$, $W_1 \in R^{n \times n}$ and $t \in R$ are the decision variables, in a form that is acceptable by the semidefinite-quadratic-linear program solver &lt;a href=&quot;http://www.math.nus.edu.sg/~mattohkc/sdpt3.html&quot; rel=&quot;nofollow&quot;&gt;SDPT3&lt;/a&gt;. I know that the first thing I should do is to convert it to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Semidefinite_programming#Equivalent_formulations&quot; rel=&quot;nofollow&quot;&gt;standard SDP form&lt;/a&gt;. I tried to do it, but I got stuck on how to model the objective function. Any help on modelling this program in an SDPT3-acceptable form would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19385" LastEditorUserId="20688" LastEditDate="2022-01-04T04:34:14.407" LastActivityDate="2022-01-04T04:34:14.407" Title="Express the $\gamma_{2}^{\epsilon}$ SemiDefinite program in a form that is acceptable by SDPT3" Tags="|convex-optimization|semidefinite-programming|modeling|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23197" PostTypeId="2" ParentId="23196" CreationDate="2016-02-23T19:44:03.930" Score="2" Body="&lt;p&gt;In order to do this, you'll need to convert inequalities to equality constraints by introducing slack variables and then add these slack variables to your matrix variable as an additional LP block.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;X=\left[ \begin{array}{ccc}&#xA;W_{1} &amp;amp; B &amp;amp; 0 \\&#xA;B^{T} &amp;amp; W_{2} &amp;amp; 0 \\&#xA;0     &amp;amp;  0    &amp;amp; \mbox{diag}(v) \\&#xA;\end{array} \right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;v=\left[ \begin{array}{c}&#xA;s_{1} \\&#xA;s_{2} \\&#xA;\vdots \\&#xA;s_{p} \\&#xA;t \\&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In SDPT3, your $X$ matrix will be a block diagonal matrix with an $m+n$ by $m+n$ symmetric positive semidefinite block for $W_{1}$, $W_{2}$, $B$, and $B^{T}$, and a diagonal (or LP variable) block $v$ of length $2mn+3$ for the slack variables and $t$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easy to express the objective $\min t$ as the minimum of the trace of $CX$, where $C$ is a matrix with a 1 in the lower right corner and all other entries 0.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your constraints will then be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{diag}(W_{1})+v_{1} - v_{p+1} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{diag}(W_{2})+v_{2} - v_{p+1} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{i,j}-B_{i,j} - v_{2+i+(j-1)m} = -\epsilon $ for $i=1, 2, \ldots, m$, $j=1, 2, \ldots, n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{i,j}-B_{i,j} + v_{2+mn+i+(j-1)m} = \epsilon $ for $i=1, 2, \ldots, m$, $j=1, 2, \ldots, n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of these linear equality constraints can be easily written as a linear equality constraint involving entries in $X$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-02-23T19:54:19.513" LastActivityDate="2016-02-23T19:54:19.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23200" PostTypeId="1" CreationDate="2016-02-24T06:13:01.077" Score="8" ViewCount="649" Body="&lt;p&gt;In &lt;a href=&quot;https://scicomp.stackexchange.com/a/10883/1502&quot;&gt;this previous thread&lt;/a&gt; the following multiplicative way to combine symmetric preconditioners $P_1$ and $P_2$ for the symmetric system $Ax=b$ was suggested:&#xA;\begin{align}&#xA;P_\text{combo}^{-1} :=&amp;amp; P_1^{-1} + P_2^{-1}(I - A P_1^{-1})\\&#xA;=&amp;amp;P_1^{-1} + P_2^{-1} - P_2^{-1} A P_1^{-1}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This combined preconditioner is not symmetric. However, I have tried using it in conjugate gradient anyways in several different contexts, and the method always seems to converge just fine. Why is this?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example 1:&lt;/strong&gt; Random matrices.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Testing multiplicative combination of preconditioners&#xA;n = 500;&#xA;[U,S,~] = svd(randn(n,n)); A = U*S*U';&#xA;[W1,S1,~] = svd(randn(n,n)); noise1 = W1*S1*W1';&#xA;[W2,S2,~] = svd(randn(n,n)); noise2 = W2*S2*W2';&#xA;P1 = A + 0.5 * noise1;&#xA;P2 = A + 0.5 * noise2;&#xA;&#xA;solve_P = @(x) P1\x + P2\x - P2\(A*(P1\x));&#xA;&#xA;b = randn(n,1);&#xA;x_true = A\b;&#xA;pcg(A,b,1e-6,n);&#xA;pcg(A,b,1e-6,n,P1);&#xA;x = pcg(A,b,1e-6,n,solve_P);&#xA;norm(x_true - x)/norm(x_true)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the output I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pcg converged at iteration 127 to a solution with relative residual 9.9e-07.&#xA;pcg converged at iteration 62 to a solution with relative residual 6.8e-07.&#xA;pcg converged at iteration 51 to a solution with relative residual 8.1e-07.&#xA;relative error= 4.23e-07&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example 2:&lt;/strong&gt; Combining multigrid with incomplete cholesky for solving the Poisson equation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 50; N = n^2;&#xA;A = gallery('poisson',n); %Laplacian on n-by-n grid, zero dirichlet BC&#xA;&#xA;L = ichol(A);&#xA;solve_P1 = @(x) L'\(L\x);&#xA;% Combinatorial multigrid: http://www.cs.cmu.edu/~jkoutis/cmg.html&#xA;solve_P2 = cmg_sdd(A); &#xA;&#xA;solve_P = @(x) solve_P1(x) + solve_P2(x) - solve_P1(A*solve_P2(x));&#xA;&#xA;b = randn(N,1);&#xA;x_true = A\b;&#xA;pcg(A,b,1e-6,N);&#xA;pcg(A,b,1e-6,N,solve_P1);&#xA;pcg(A,b,1e-6,N,solve_P2);&#xA;x = pcg(A,b,1e-6,N,solve_P);&#xA;disp(['relative error= ', num2str(norm(x_true - x)/norm(x_true),3)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For this I get the results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pcg converged at iteration 131 to a solution with relative residual 8.4e-07.&#xA;pcg converged at iteration 44 to a solution with relative residual 6e-07.&#xA;pcg converged at iteration 19 to a solution with relative residual 7e-07.&#xA;pcg converged at iteration 12 to a solution with relative residual 4.7e-07.&#xA;relative error= 5.2e-07&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I've also seen the same qualitative behavior on matrices arising in more complicated/realistic situations.&lt;/li&gt;&#xA;&lt;li&gt;Given an incorrect solution to $Ax=b$, the error $e$ and the residual $r$ are related by the error equation $Ae=r$. One can view this combo preconditioner as approximately solving the original equation using $P_1$ instead of $A$, then approximately solving the error equation with $P_2$ instead of $A$, then adding the approximate error back to correct the original approximate solution.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-09-09T14:30:51.183" Title="Why does conjugate gradient work with this nonsymmetric preconditioner?" Tags="|preconditioning|conjugate-gradient|krylov-method|symmetry|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23202" PostTypeId="1" AcceptedAnswerId="23209" CreationDate="2016-02-24T13:51:32.063" Score="26" ViewCount="10065" Body="&lt;p&gt;I have learnt about Finite Element Method (also a little on other numerical methods) but I don't know what are exactly definition of these two errors and differences between them?&lt;/p&gt;&#xA;" OwnerUserId="4670" LastEditorUserId="1804" LastEditDate="2016-02-25T10:22:44.570" LastActivityDate="2016-02-25T11:20:46.113" Title="What are differences between 'a priori' and 'posteriori' error estimate in numerical analysis?" Tags="|finite-element|numerical-analysis|error-estimation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23203" PostTypeId="2" ParentId="22103" CreationDate="2016-02-24T13:54:37.053" Score="0" Body="&lt;p&gt;All I could find is  this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mat.univie.ac.at/~neum/glopt/test.html#test_constr&quot; rel=&quot;nofollow&quot;&gt;http://www.mat.univie.ac.at/~neum/glopt/test.html#test_constr&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it has a paper with different functions and results. But no code that I could use.&lt;/p&gt;&#xA;" OwnerUserId="157" LastActivityDate="2016-02-24T13:54:37.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23204" PostTypeId="1" CreationDate="2016-02-24T14:19:55.563" Score="1" ViewCount="47" Body="&lt;p&gt;I have the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a matrix with n rows and m columns. Some elements of the matrix are unavailable. For each column, you have a set containing a number of zeros and ones which must be distributed over the available elements in that column. The total number of zeros and ones in a set is equal to the number of available elements in the corresponding column. The question is: How should the values of the sets be distributed over the elements in the corresponding colum, such that the number of rows for which the total number of ones is less than 20% of the available elements in that row, is minimized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does someone knows a similar problem which has a polynomial time algorithm solving the problem to optimality, or knows a polynomial time algorithm solving the problem to optimality (if it exists)?&lt;/p&gt;&#xA;" OwnerUserId="19400" LastActivityDate="2016-02-24T14:19:55.563" Title="Optimal distribution of zeros and ones over matrix" Tags="|optimization|matrix|combinatorics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23205" PostTypeId="1" AcceptedAnswerId="23206" CreationDate="2016-02-24T14:32:03.820" Score="2" ViewCount="2015" Body="&lt;p&gt;In Gmsh mesh option list ,there is a smoothing concept, what does this mean and what is the effect of its number to the mesh quality?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastActivityDate="2016-02-24T17:57:37.120" Title="what is the meaning of mesh smoothing steps in Gmsh?" Tags="|mesh|smoothing|gmsh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23206" PostTypeId="2" ParentId="23205" CreationDate="2016-02-24T17:57:37.120" Score="5" Body="&lt;p&gt;Meshing algorithms can place the vertices of triangles and tetrahedra in a wide variety of ways, but they are us usually essentially constructive (i.e new vertices are introduced, existing vertices stay where they are). Disappointingly, this can cause the meshes which are generated to be unsuitable for numerical calculation. Mesh smoothing, at least in the case of gmsh, is essentially a post-processing step, which takes an existing mesh and iteratively deforms the positions (but not connectivity) of the vertices such that the coordinates better satisfy the numerical discretization of an elliptic PDE, effectively spreading out jumps in the anisotropy across a finite area, reducing the extrema in mesh quality. For more details, see for example &lt;a href=&quot;http://perso.uclouvain.be/vincent.legat/teaching/documents/meca2170-jfr-cours4.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; for a talk given by one of the GMSH authors about elliptic meshing. This is part of a wider class of problems called moving mesh partial differential equations (MMPDES).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the algorithm is iterative, you would hope that it converges to a fixed state, for a give mesh resolution. How many steps it will take to get there will depend on the original mesh and the other constraints you apply.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-02-24T17:57:37.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23207" PostTypeId="1" AcceptedAnswerId="23212" CreationDate="2016-02-25T03:18:41.297" Score="7" ViewCount="1056" Body="&lt;p&gt;This may be a trivial question, but I've always wondered...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the classical, &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_coefficient&quot; rel=&quot;nofollow&quot;&gt;central finite difference schemes&lt;/a&gt;, if I'm interested in determining the second derivative, does applying the first derivative scheme recursively (2x) give the same answer as performing the second derivative scheme once? What about the behavior of the truncation error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if I should code a first-central FD and just call it recursively, or have a second-central FD separately to handle second derivatives. What are the differences and why choose one approach over the other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The link I provided should help clarify my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I'll be more explicit. If I go to the link above, I can choose a 3-point stencil which evaluates at $x_{i-1},x_0,x_{i+1}$. I have two options to calculate a second derivative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 1: choose a first derivative central scheme, second order accurate, with coefficents $\{-1/2,0,1/2\}$. If I apply this recursively, I obtain the second derivative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 2: choose a second derivative central scheme, second order accurate, with coefficients $\{1,-2,1\}$. I only have to apply this once to obtain the second derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, are they the same? I suspect the truncation errors are different, but I'm not sure.&lt;/p&gt;&#xA;" OwnerUserId="19407" LastEditorUserId="19378" LastEditDate="2016-02-26T19:34:35.063" LastActivityDate="2016-02-26T19:34:35.063" Title="Finite difference recursion and higher order" Tags="|algorithms|finite-difference|numerical-analysis|discretization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23208" PostTypeId="1" AcceptedAnswerId="23543" CreationDate="2016-02-25T09:00:23.540" Score="1" ViewCount="1086" Body="&lt;p&gt;I'm currently working with a system where the long range Coulomb interactions are described with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ewald_summation&quot; rel=&quot;nofollow&quot;&gt;Ewald summation method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using DL_POLY Classic. The Ewald summation method used is described in pages 46-48 in the user manual which I unfortunately can't link because of my low ranking.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working with two papers which use the same computational methods. Again, I can't link anything but you can find the papers in &lt;strong&gt;arxiv&lt;/strong&gt;. The names are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;B. Guillot, N. Sator; Carbon dioxide in silicate melts: A molecular dynamics simulation study&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;B. Guillot, N. Sator; A computer simulation study of natural silicate melts. Part I: Low pressure properties&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These papers describe the parameters as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The long range coulombic interactions are accounted for by a Ewald sum with a constant $\alpha L= 5-7$ (where $\alpha$ is the width of the charge distribution on each ion and $L$ is the size of the simulation box) and cut off distance $(r_{cut})$ of 10 - 11 Å, the summation in the reciprocal space being evaluated for all $k$ vectors with $|k|L/2\pi&amp;lt;6-7$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;DL_POLY requires 4 values for ewald-summation, $\alpha~,k_{x},k_{y}$ and $k_{z}$, where (according to the user manual) $\alpha$ is the convergence parameter and $k_{x,y,z}$ are the maximum k-vector indexes in x,y or z-direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now I would need to find out the correct convergence parameter and the values for $k_{x,y,z}$. Can I get these values straight from the data supplied in the paper, and if yes, how?&lt;/p&gt;&#xA;" OwnerUserId="19186" LastEditorUserId="19186" LastEditDate="2016-03-30T09:08:32.977" LastActivityDate="2016-04-01T08:54:03.457" Title="Choosing the correct Ewald sum parameters" Tags="|molecular-dynamics|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23209" PostTypeId="2" ParentId="23202" CreationDate="2016-02-25T11:20:46.113" Score="33" Body="&lt;p&gt;Error estimates usually have the form&#xA;$$ \|u - u_h\| \leq C(h),$$&#xA;where $u$ is the exact solution you are interested in, $u_h$ is a computed approximate solution, $h$ is an approximation parameter you can control, and $C(h)$ is some function of $h$ (among other things). In finite element methods, $u$ is the solution of a partial differential equation and $u_h$ would be the finite element solution for a mesh with mesh size $h$, but you have the same structure in inverse problems (with the regularization parameter $\alpha$ in place of $h$) or iterative methods for solving equations or optimization problems (with the iteration index $k$ -- or rather $1/k$ -- in place of $h$). &#xA;The point of such an estimate is to help answer the question &lt;em&gt;&quot;If I want to get within, say, $10^{-3}$ of the exact solution, how small do I have to choose $h$?&quot;&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between a priori and a posterior estimates is in the form of the right-hand side $C(h)$:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In &lt;strong&gt;a priori&lt;/strong&gt; estimates, the right-hand side depends on $h$ (usually explicitly) and $u$, but not on $u_h$. For example, a typical a priori estimate for the finite element approximation of Poisson's equation $-\Delta u = f$ would have the form&#xA;$$ \|u-u_h\|_{L^2} \leq c h^2 |u|_{H^2},$$&#xA;with a constant $c$ depending on the geometry of the domain and the mesh. In principle, the right-hand side can be evaluated prior to computing $u_h$ (hence the name), so you'd be able to choose $h$ before solving anything. In practice, neither $c$ nor $|u|_{H^2}$ is known ($u$ is what you're looking for in the first place), but you can sometimes get order-or-magnitude estimates for $c$ by carefully going through the proofs and for $|u|$ using the data $f$ (which is known). The main use is as a qualitative estimate -- it tells you that if you want to make the error smaller by a factor of four, you need to halve $h$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In &lt;strong&gt;a posteriori&lt;/strong&gt; estimates, the right-hand side depends on $h$ and $u_h$, but not on $u$. A simple &lt;em&gt;residual-based&lt;/em&gt; a posterior estimate for Poisson's equation would be&#xA;$$ \|u-u_h\|_{L^2} \leq c h \|f+\Delta u_h\|_{H^{-1}},$$&#xA;which could in theory be evaluated &lt;em&gt;after&lt;/em&gt; computing $u_h$. In practice,  the $H^{-1}$ norm is problematic to compute, so you'd further manipulate the right-hand side to get an &lt;em&gt;element-wise&lt;/em&gt; bound&#xA;$$ \|u-u_h\|_{L^2} \leq c \left(\sum_{K} h_K^2 \|f+\Delta u_h\|_{L^2(K)} + \sum_{F} h_K^{3/2} \|j(\nabla u_h)\|_{L^2(F)}\right),$$&#xA;where the first sum is over the elements $K$ of the triangulation, $h_K$ is the size of $K$, the second sum is over all element boundaries $F$, and $j(\nabla u_h)$ denotes the jump of the normal derivative of $u_h$ across $F$. This is now fully computable after obtaining $u_h$, except for the constant $c$. So again the use is mainly qualitative -- it tells you which elements give a larger error contribution than others, so instead of reducing $h$ uniformly, you just select some elements with large error contributions and make those smaller by subdividing them. This is the basis of &lt;em&gt;adaptive finite element methods&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-02-25T11:20:46.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23210" PostTypeId="1" AcceptedAnswerId="23213" CreationDate="2016-02-25T12:05:55.100" Score="1" ViewCount="231" Body="&lt;p&gt;If we have a discrete saddle point problem with the coefficient matrix&#xA;$$ \mathcal{A} = &#xA; \begin{bmatrix}&#xA;   A &amp;amp; B^T \\&#xA;   B &amp;amp; 0&#xA; \end{bmatrix},&#xA;$$&#xA;then $\mathcal{A}$ is invertible, supposing $B$ has full rank and $A$ is positive semidefinite (which hold in my case), if $\mathrm{ker}(A)\cap\mathrm{ker}(B) = \{0\}$. The basis for the kernel or null space of a matrix can be calculated by &lt;code&gt;null(A)&lt;/code&gt; in MATLAB. But how should I interpret mathematically and calculate the intersection of the two kernels?&lt;/p&gt;&#xA;" OwnerUserId="18289" LastEditorUserId="9667" LastEditDate="2016-02-25T15:46:08.057" LastActivityDate="2016-02-25T16:20:24.980" Title="Calculate the intersection of two matrix kernels in MATLAB" Tags="|matlab|nullspace|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23211" PostTypeId="1" CreationDate="2016-02-25T14:58:04.293" Score="1" ViewCount="43" Body="&lt;p&gt;I want to discretize and numerically solve the following PDE:&#xA;\begin{equation}&#xA;v(k)\dfrac{\partial f}{\partial x} + E(x)\dfrac{\partial f}{\partial k} = S\{f\}&#xA;\end{equation}&#xA;using finite volume (box integration) method.&#xA;Since the direct discretization results in an oscillatory solution, I have used the upwind discretization scheme. This eliminates the oscillations and the solution satisfies the maximum principle. However, if I have my $x$-grid as different $x_i$ points with $i=1,2,...,N$, the upwind discretization schemes loses the connection between $f(x_1)$ (the boundary value) and $f(x_2)$ because I am always calculating the right-sided difference on the second grid node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try to use the central difference on $x_2$, oscillations and instabilities return to the solution, and when I use the upwind scheme, the boundary points are disconnected from their adjacent nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to solve this issue?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-02-25T14:58:04.293" Title="Stability in Discretization of 1D Stationary Boltzmann equation" Tags="|stability|discretization|numerics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23212" PostTypeId="2" ParentId="23207" CreationDate="2016-02-25T15:44:11.977" Score="5" Body="&lt;p&gt;Short answer: yes (in exact arithmetic). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll have to use the centered difference formula evaluated at $x \pm \frac{1}{2}\delta x$, like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_x = \frac{u(x + \frac{1}{2}\delta x) - u(x - \frac{1}{2}\delta x)}{\delta x} + {\cal O}(\delta x^2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, applying the formula recursively, you arrive at the &quot;standard&quot; second order finite-difference stencil:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;u^{\rm FD}_{xx} &amp;amp;=&amp;amp; \frac{u_x(x + \frac{1}{2}\delta x) - u_x(x - \frac{1}{2}\delta x)}{\delta x} \\&#xA;&amp;amp;=&amp;amp; ... \\&#xA;&amp;amp;=&amp;amp; \frac{u(x + \delta x) - 2 u(x) + u(x - \delta x)}{(\delta x)^2} \\ &#xA;&amp;amp;=&amp;amp; u_{xx} + \frac{(\delta x)^2}{12}u_{xxxx} + \text{higher order terms.} &#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This recursive &quot;trick&quot; works with other finite-difference formulas as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I (for one) would not implement such a scheme recursively. The computational cost (defined as the total FLOP count) of evaluating a second derivative through the recursive scheme is higher than that of the &quot;classic&quot; formula, with no extra benefit to accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use $u(x+\delta x)$ and $u(x - \delta x)$, this is equivalent to the standard second derivative approximation but with twice the grid-width, hence the leading error term will be $\frac{(2\delta x)^2}{12}u_{xxxx}$, 4 times larger that above.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17671" LastEditDate="2016-02-26T05:49:03.367" LastActivityDate="2016-02-26T05:49:03.367" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23213" PostTypeId="2" ParentId="23210" CreationDate="2016-02-25T16:20:24.980" Score="1" Body="&lt;p&gt;In MATLAB, null([A; B]) will find an orthogonal basis for the intersection of the null spaces of A and B.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems unlikely that you really want to find this basis, but it's not clear from your question what you're actually trying to do.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-02-25T16:20:24.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23214" PostTypeId="2" ParentId="6791" CreationDate="2016-02-25T20:23:19.313" Score="0" Body="&lt;p&gt;You can fix the eigenvectors to have the same sign in Python like this:&#xA;eigvects = np.sign(eigvects[0,:]).transpose()*eigvects&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That will fix the 1st eigenvector component to positive for all eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't figure out the LAPACK signing routine either vs ARPACK which would be nice to know so that eigh signs could be replicated, keeping your simulation predictable.&lt;/p&gt;&#xA;" OwnerUserId="19418" LastActivityDate="2016-02-25T20:23:19.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23215" PostTypeId="1" CreationDate="2016-02-25T20:30:17.877" Score="1" ViewCount="58" Body="&lt;p&gt;I am looking at a real-world resource allocation problem that is cast as a quadratically-constrained linear program of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\max\langle f,x\rangle&#xA;$$&#xA;subject to&#xA;$$&#xA;\begin{aligned}&#xA;m \leq\,\, &amp;amp; x \leq M, \\&#xA;A\cdot x &amp;amp; \leq b, \\&#xA;A_{eq}\cdot x &amp;amp; = b_{eq}, \qquad\text{and} \\&#xA;\langle x,Q\cdot x\rangle &amp;amp; \leq \alpha.&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a small problem, that is, $x\in\mathbb{R}^{55}$, and it can be solved in much less than a second using Matlab's $\verb|fmincon|$ to obtain an optimal solution $x^*$. The question is the following: is there an &quot;intelligent&quot; (read: systematic/efficient; again, hacks are welcome) way to determine an admissible box constraint of the form&#xA;$$&#xA;\tilde{m}\leq x*\leq \tilde{M}\ ? \qquad(1)&#xA;$$&#xA;That is, if $B$ is the region described by the box constraint (1) and $F$ is the feasible region of the original problem, then it should be the case that&#xA;$$&#xA;B\subseteq F.&#xA;$$&#xA;It is fine if $B$ has empty interior, as long as the relative interior with respect to the affine subspace that contains $B$ is not empty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To provide some context, the components of the optimal solution $x^*$ are communicated to production departments that operate &lt;em&gt;independently&lt;/em&gt; and $B$ represents allowable deviations from their targets. Let me point out that, as is implicit in the description of the problem, the original constraints are hard constraints and it is not meaningful to violate them; i.e., to have a box $B$ around $x^*$ that extends beyond the feasible region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Any pointers to the literature, high-level hints to the right direction, or quick and dirty approaches to the following problem will be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19414" LastEditorUserId="9667" LastEditDate="2016-02-25T22:30:44.400" LastActivityDate="2016-02-25T22:30:44.400" Title="Admissible box constraint for a quadratically constrained linear program" Tags="|linear-programming|quadratic-programming|" AnswerCount="0" CommentCount="2" ClosedDate="2019-06-06T20:30:34.123" ContentLicense="CC BY-SA 3.0" />
  <row Id="23216" PostTypeId="1" AcceptedAnswerId="23217" CreationDate="2016-02-25T23:11:13.863" Score="1" ViewCount="6136" Body="&lt;p&gt;&lt;br&gt;Yesterday I was wondering how floats are handled in a computer and what they look like in binary... I learnt about the &lt;a href=&quot;https://en.wikipedia.org/wiki/Single-precision_floating-point_format&quot; rel=&quot;nofollow&quot;&gt;single-precision floating-point&lt;/a&gt; and I tried to see the limit of that format...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first wrote this little Python (3.4) script&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a,b = 1,1&#xA;&#xA;while (a+b)-a-b == 0:&#xA;    a*=2.0&#xA;&#xA;while (a+b)-a-b != 0:&#xA;    b+=1.0&#xA;&#xA;print(a,b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I wanted to see what specific exponent value of &lt;strong&gt;a&lt;/strong&gt; could be create a lack of precision in the result &lt;strong&gt;(a+b)-a-b&lt;/strong&gt;, and if increasing the value of &lt;strong&gt;b&lt;/strong&gt; could correct it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output was :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;9007199254740992.0 2.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I noticed that &lt;strong&gt;2^53 = 9007199254740992.0&lt;/strong&gt; and I though it was a bit disturbing : the binary representation of &lt;strong&gt;53&lt;/strong&gt; is &lt;strong&gt;110101&lt;/strong&gt;, and this really doesn't look like any specific value...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;&lt;strong&gt;2.0&lt;/strong&gt;&quot; let me a little less sceptical, as you just have to add &lt;strong&gt;1&lt;/strong&gt; to increase the exponent value of &lt;strong&gt;b&lt;/strong&gt;. It made more sense, but I realised I couldn't explain it properly too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I modified my script to see the actual binary value of everything I was doing :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import struct&#xA;&#xA;def binary(f):&#xA;    return str(bin(struct.unpack('!i',struct.pack('!f',f))[0])).replace('0b', '')&#xA;&#xA;def status(a,b):&#xA;    return binary(a),binary(b),binary(a+b),binary(-a),binary(-b),binary(-a-b),binary((a+b)-a-b)&#xA;&#xA;a,b = 1,1&#xA;&#xA;while (a+b)-a-b == 0:&#xA;    a*=2.0&#xA;    print(status(a,b))&#xA;&#xA;&#xA;print(&quot;\n&quot;)&#xA;&#xA;while (a+b)-a-b != 0:&#xA;    b+=1.0&#xA;    print(status(a,b))&#xA;&#xA;print(a,b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But what I saw didn't help me. I can't figure out why the lack of precision happens at that specific value of &lt;strong&gt;a&lt;/strong&gt;, and why increasing &lt;strong&gt;b&lt;/strong&gt; just once would correct it.&lt;br&gt;&lt;br&gt;&#xA;Could anyone help me understanding this ?&lt;br&gt;&lt;br&gt;&#xA;(I apologize in advance for my really poor English)&lt;/p&gt;&#xA;" OwnerUserId="19420" LastEditorUserId="19420" LastEditDate="2016-02-26T14:24:10.270" LastActivityDate="2016-02-26T14:24:10.270" Title="Addition and subtraction of two floats in Python" Tags="|python|floating-point|precision|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23217" PostTypeId="2" ParentId="23216" CreationDate="2016-02-26T00:39:27.783" Score="4" Body="&lt;p&gt;You've run into the limits inherent in &lt;a href=&quot;https://en.wikipedia.org/wiki/Double-precision_floating-point_format&quot; rel=&quot;nofollow&quot;&gt;double precision floating point&lt;/a&gt; numbers, which python uses as its default float type (this is the same as a C double).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These model real numbers as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(-1)^s \left(1+\sum_{i=1}^{52}\frac{b_{52-i}}{2^i}\right)\times 2^{e-1023}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $s$ is 0 or 1 (1 bit), $e$ is an 11 bit number and the $b_i$s fill the remaining bits. This indeed has a maximum value with integer stride of  $2^{53}$ = 9007199254740992.0. After this value, the numbers which can be modelled start to increase by 2 each time, and the addition of 1.0 (i.e. b) underflows. On the other hand, you can still add 2.0 just fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model effectively attempts to balance range (the size of exponents it accepts) with accuracy (the size of the quantised step it accepts). If you'd still like to play with single precision numbers and your python installation includes numpy, they can be accessed using the &lt;a href=&quot;http://docs.scipy.org/doc/numpy-1.10.1/user/basics.types.html&quot; rel=&quot;nofollow&quot;&gt;numpy.float32&lt;/a&gt; function. Note that you will have to set your constants to be float32 as well, otherwise python automatic type conversion may prevent you seeing anything.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="19263" LastEditDate="2016-02-26T00:57:25.907" LastActivityDate="2016-02-26T00:57:25.907" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23219" PostTypeId="1" CreationDate="2016-02-26T08:52:51.633" Score="4" ViewCount="120" Body="&lt;p&gt;I've been having fun playing around with a simple model for reaction in a co-flow fluidized bed (I'm afraid I'm a chemical engineer, not a computer scientist). Unfortunately, simple physical assumptions often lead to &lt;em&gt;terrifying&lt;/em&gt; equations. I've reduced the problem to solving the following set of simultaneous integro-differential equations for the 2 unknown functions $g(s)$ (on $s\in[s_0,s_{max}]$) and $C(z)$ (on $z\in [0,L]$), where $f(C,s)$ is a known function and $\alpha$ and $\tau$ are constants:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\frac{dC}{dz} = \alpha \int_{s_0}^{s_{max}} f(C(z),s) g(s) ds$$&#xA;$$g(s) = \frac{\exp\left(-\frac{L}{\tau}\int_{s_0}^{s} \frac{ds}{\int_{0}^{L}f(C(z),s)dz}\right)}{\frac{\tau}{L} \int_{0}^{L}f(C(z),s)dz}$$&#xA;and the one BC:&#xA;$$C|_{z = 0} = C_0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I've tried the brute-force approach: finite differences. I divided the intervals  $[0,L]$ and $[s_0,s_{max}]$ into $n$ points, and defined the $2n$ unknown variables $C_1, C_2,\ldots,C_n$ and $g_1, g_2,\ldots,g_n$, which are just the values of my unknown functions at these points. I then approximately the derivatives and integrals above using FD expressions, and attempted to solve the system of $2n$ non-linear algebraic equations using $\texttt{fsolve}$ in MATLAB. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps predictably, MATLAB doesn't like this, and fsolve struggles to give a solution for medium-sized $n$ ($n=100$ returns 'no solution found', and $n = 250$ is still running after half an hour.) My questions would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are there any ways of solving &lt;em&gt;large&lt;/em&gt; systems of simultaneous non-linear equations, that are more efficient than $\texttt{fsolve}$. Preferably I'd be looking for an algorithm already implemented in MATLAB.&lt;/li&gt;&#xA;&lt;li&gt;Even more optimistically, are there any other possible approaches to solving these types of highly non-linear integro-differential equations?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6799" LastActivityDate="2016-02-26T08:52:51.633" Title="The numerical solution of a (very ugly) set of integro-diferential equations" Tags="|finite-difference|" AnswerCount="0" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23220" PostTypeId="1" AcceptedAnswerId="23226" CreationDate="2016-02-26T11:17:11.307" Score="0" ViewCount="52" Body="&lt;p&gt;I read &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999107003269&quot; rel=&quot;nofollow&quot;&gt;reference (1)&lt;/a&gt;，but I am confused about how to introduce a distance vector in Lorentz force like the author does it in equation (11):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rm J\times B = -J\cdot \nabla \left(&#xA;B\times r\right)+\left(J\cdot \nabla B\right)\times r$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rm J$ is a current, $\rm B$ is magnetic field, $\rm r$ is the distance vector&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and equation (8)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rm \omega\times u=u\cdot\nabla\left(&#xA;\omega\times r\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega$ is the rotating speed vector of the reference frame relative to the absolute inertial frame, $\rm r$ is the distance vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, how to introduce a distance vector when decompose a force?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author says that $\rm -J\cdot\nabla \left(B\times r\right)$ is a globally conservative term while $\rm \left(J\cdot\nabla B\right)\times r$ is a locally conservative term, and he does not explain it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we judge a term is a globally conservative term or a locally conservative term?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Ni, Ming-Jiu, et al. &quot;A current density conservative scheme for incompressible MHD flows at a low magnetic Reynolds number. Part II: On an arbitrary collocated mesh.&quot; Journal of Computational Physics 227.1 (2007): 205-228.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19408" LastEditorUserId="9667" LastEditDate="2016-02-26T19:38:40.443" LastActivityDate="2016-02-26T23:40:28.527" Title="How to introduce a distance vector when decompose a force?" Tags="|fluid-dynamics|electromagnetism|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23222" PostTypeId="1" CreationDate="2016-02-26T17:20:16.873" Score="7" ViewCount="676" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VDsOd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VDsOd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are $n$ points in a 2-D plane and each is given by its $x$ and $y$ coordinates. They are stored in an array in an ascending order with respect to $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All points are connected together by line segments ($n\choose 2$ lines). Within the complete graph created by these line segments, how can we find a line which has the most number of intersections?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the given example, $AD$ (marked green) is the line with the most intersections (4) in a set 6 points.&lt;/p&gt;&#xA;" OwnerUserId="19429" LastEditorUserId="20688" LastEditDate="2018-03-30T05:12:00.400" LastActivityDate="2019-05-25T04:07:06.383" Title="How can I find a line segment with the most intersections along with the coordinates of the intersection points?" Tags="|computational-geometry|geometry|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23223" PostTypeId="1" CreationDate="2016-02-26T18:49:17.263" Score="4" ViewCount="247" Body="&lt;p&gt;This may be a trivial question, and I apologize if so. Consider the following simple problem: We have a 2D, regular grid of points (say $X = [0,5000] \times [0,5000]$) spaced uniformly by units of 1 (e.g., $(0,0), (0,1), (1,0) \in X$ and there exists a function $f(x,y)$ that we aim to approximate on $X$.  Let's assume it's nice and smooth, at least $C^2$. Assume that we have data samples on a uniform subgrid (say every 100 points in a direction we have a sample value $f(x_i,y_i)$). We all know there exists many different methods to approximate such a function. What I'm curious about is suggestions on which methods provide me the &lt;em&gt;fastest&lt;/em&gt; evaluation of a new given point &lt;em&gt;not&lt;/em&gt; on the subgrid (e.g., at a point $(50,25)$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 1: Multivariate polynomial interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could choose a multivariate polynomial basis of some fixed degree (let's assume 3rd order) so , $p(x,y) = c_0 + c_1 x + c_2y + c_3 x^2 + c_4 xy + c_5 y^2 + c_6 x^3 + c_7 x^2 y + c_8 xy^2 + c_n y^3$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of advantages to this method: storage is small (we just need to store the coefficients, which is a very small number relative to the number of points in the grid) and evaluation on a new point is relatively fast (we have several multiplications, plus a few additions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, we also have to fit this polynomial. This requires us to solve a least squares problem, where we will have a matrix of $N \times d$ where d is the number of coefficients and $N$ is the number of sample points. I &lt;em&gt;believe&lt;/em&gt; my best option is an SVD solve (using maybe Armadillo or Eigen). This will definitely be the slowest part of the whole ordeal, so there is significant up front cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next disadvantage is issues such as Runge's Phenomenon. I am usually quite cautious about fitting with multivariate polynomials due to concerns such as poor accuracy in certain regions. Furthermore, the choice of basis significantly affects the solvability of the system. If this were 1D, I'd do Chebyschev polynomials if possible. Is there a better basis for such a problem in 2D?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Option 2: Piecewise Polynomials on the square grids&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a fan of this option. I can use linear or quadratic elements on the actual subregions (of say 100 by 100 units). The disadvantage is that I'm required to store the $N$ known function samples. So this requires more storage, so copying and moving around this information is more significant than moving around for example the polynomial coefficients (which were a fixed, tiny number). Evaluation should be quite quick. I need to know the values at my vertices of the cell, and given a new point (x,y), I need to compute which square cell it's in. I assume I can do this by dividing $x$ and $y$ by the stride (say 100) and flooring it to find out the bottom left vertex of the cell the new point is in. The huge advantage i see here is that I'm not solving large systems to get coefficients for an approximating function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only disadvantage I really see is the case where my grid spacing varies (say 100 by 100 cells for a bit, then 50 by 50 cells). Given a new point to evaluate a function value at, (x,y), I need to efficiently and &lt;em&gt;quickly&lt;/em&gt; compute which cell (x,y) is in, then do my piecewise polynomial evaluation in that cell quickly. Maybe I'm being dumb, but what is a good efficient method to quickly compute which cell a point lies in when the cells are not all the same size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3: Radial Basis Function methods&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This requires solving large systems &lt;em&gt;and&lt;/em&gt; lots of coefficients, so I'm not seeing an advantage here. If I had irregularly sampled points, I would consider this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary: Speed is of the essence for me; I will have lots of incoming points and evaluations of the function value at these points desired and I need to also as quickly as possible set up the approximating function (e.g. solving for polynomial or RBF coefficients). Suggestions would be great. Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I was unclear in my question. I am aiming for real-time (or as near as possible) fitting of an incoming dataset (as described above) with some method (polynomials, piecewise polys, whatever), which will then subsequently, in real-time receive many requests for evaluations. I will receive more than just one dataset, possibly many per second (tens? hundreds?). Hence, the upfront cost of building the interpolation/fitting scheme and subsequent evaluations both needs to be fast. I will definitely need to parallelize as much as possible.&lt;/p&gt;&#xA;" OwnerUserId="6567" LastEditorUserId="6567" LastEditDate="2016-02-27T20:23:33.833" LastActivityDate="2016-02-27T20:23:33.833" Title="Question on speed and accuracy comparisons of different 2D curve fitting methods" Tags="|algorithms|numerical-analysis|interpolation|curve-fitting|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23224" PostTypeId="2" ParentId="23223" CreationDate="2016-02-26T19:40:29.913" Score="2" Body="&lt;p&gt;First of all, you can turn the fitting problem for 1 and 3 into an iterative optimization problem if you don't want to deal directly with solving a large system of equations using a matrix approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;strong&gt;2&lt;/strong&gt;, you can use a hash table and design a spatial hash function to map a coordinate to the appropriate subregion, which would be very fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;strong&gt;3&lt;/strong&gt;, given you use RBFs that are locally impacting the fit to the data, you could use a KD Tree with an m-Nearest Neighbors algorithm to find the RBFs that impact a given input point and only use those to compute the value. I have found this as very efficient in my implementations. The tough point is figuring out how many RBFs to use and how large to set their Region of Influence to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other Idea&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have storage requirements, you could also consider doing Locally Weighted Regression using a local linear or quadratic fit. I have built codes to do this on large data sets with 9-D data points and it was quite fast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The benefit to this algorithm is you don't have to fit any models beforehand. You end up using a set of data close to your input data point that you want to find the function value at, and do a local fit using a simpler model (linear, quadratic), so it's fast. If you use a KD Tree to find the M Nearest Neighbors, it should be quite quick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final Thoughts&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am suspecting option 2 might be your best balance between speed and reduced data storage, especially if you use a hash table/hash function pair to map the input points to the appropriate subregion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I almost forgot, but for method &lt;strong&gt;1&lt;/strong&gt;, you could also incorporate regularization on the coefficients of your polynomial model. This will help you avoid Runge's Phenomenon and should give you the fastest model to evaluate lots of points relative to the other approaches.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-02-26T19:57:35.733" LastActivityDate="2016-02-26T19:57:35.733" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23226" PostTypeId="2" ParentId="23220" CreationDate="2016-02-26T23:40:28.527" Score="1" Body="&lt;p&gt;I believe your first two questions are both just applications of the general vector calculus identity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{a}\times\mathbf{b}=\mathbf{a}\cdot\nabla_\mathbf{x}\mathbf{b}\times\mathbf{x}-\mathbf{a}\cdot\nabla_\mathbf{x}(\mathbf{b}\times\mathbf{x})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I've taken the liberty of relabelling the coordinate vector from $\mathbf{r}$ to $\mathbf{x}$. The easiest way of seeing this is probably coordinatewise, noting that $[\nabla_\mathbf{x}\mathbf{x}]_{ij}=\frac{\partial x_j}{\partial x_i}=\delta_{ij}$. We then get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\epsilon_{ijk}(a_m\partial_m b_j)x_k-a_m\partial_m(\epsilon_{ijk}b_j x_k) = \epsilon_{ijk}(a_m\partial_m b_j)x_k - \epsilon_{ijk}(a_m\partial_m b_j)x_k - a_m\epsilon_{ijk}b_j \delta_{km}=\epsilon_{imj}b_ja_m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case with rotation vector, $\mathbf{\omega}$ is a spatially constant vector (solid body rotation) so that $\mathbf{u}\cdot\nabla \mathbf{\omega}$ vanishes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your last question is more open to interpretation, but I believe here globally conservative is being used in the sense of &quot;divergence form&quot;, while locally conservative means in the sense of, the force is conservative travelling with the vector $\mathbf{J}$.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-02-26T23:40:28.527" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23228" PostTypeId="1" CreationDate="2016-02-27T18:13:06.053" Score="1" ViewCount="394" Body="&lt;p&gt;I am trying to solve the following optimization problem:&#xA;\begin{align}&#xA;&amp;amp;\min\limits_{s} \rm{tr}\left(S^T S\right) + \mathrm{tr}\left(\left(S^T S\right)^{-2}\right)\\&#xA;&amp;amp;\text{subject to }\rm\left\|S^TS-Z^TZ\right\|_{norm}\leq \epsilon\end{align}&#xA;Where $\rm \|\cdot\|_{norm}$ could be either the Frobenius norm or the $\ell_1$-norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, $\rm S^TS$ belongs to the ball with radius $\epsilon$ and center $\rm Z^TZ$. Actually, I want to solve my problem with $\rm Q = S^TS$ which is a symmetric positive semi-definite matrix, thus, a convex problem to solve but I couldn't find another way to impose that property on my solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have tried first with &lt;code&gt;minFunc&lt;/code&gt; toolbox by using the lagrange expression, and the Frobenius norm for the constraint $\rm \|\cdot\|_{norm}$ i.e., &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min\limits_{s} \rm \mathrm{tr}\left(S^T S\right) + \mathrm{tr}\left(\left(S^T S\right)^{-2}\right) + \lambda\cdot\mathrm{tr}\left(\left(S^T S - Z^TZ\right)\left(S^T S - Z^TZ\right)^T\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it stops after few iterations with a very high error about $10^{2}$ for a small $S\in\mathrm{R}^{10\times10}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options.Method = 'cg';%'lbfgs'; &#xA;options.maxIter = 100000000;&#xA;options.MaxFunEvals = 20000000000000;&#xA;lambda = 0.001;&#xA;x0 = rand(100,1);&#xA;Z = rand(15,10);&#xA;funObj = @(x)myfunc(x, lambda, Z);&#xA;[sol, f_val] = minFunc(funObj,x0,options);&#xA;&#xA;function [f,g] = myfunc(x, lambda, Z)&#xA;s = reshape(x, [sqrt(numel(x)),sqrt(numel(x))]);&#xA;f = trace(s'*s) + trace(inv(s'*s)^2) + lambda* trace((s'*s - Z'*Z)*(s'*s - Z'*Z)');&#xA;g = 2*s -4*s*(inv(s'*s))^3 + lambda*(2*s*s'*s + 2*s'*s*s -2*s*Z'*Z -2*Z'*Z*s);&#xA;g = g(:);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried also with &lt;code&gt;minConf_PQN&lt;/code&gt; using the $\ell_1$-norm constraint but returns this error: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;close to singular or badly scaled. Results may be inaccurate. RCOND = NaN&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is the code that I have used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;funObj = @(x)myfunc(x);&#xA;&#xA;Z = rand(15,10);&#xA;S0 = load('S_true.mat')&#xA;L=S0'*S0-Z'*Z; &#xA;tau = sum(abs(L(:))); %true tau&#xA;r = reshape(Z'*Z, [size(Z,2)^2,1]);&#xA;funProj = @(w)sign(w).*projectRandom2C(abs(w'w-r),tau);&#xA;sol = minConf_PQN(funObj,x0,funProj,options);&#xA;&#xA;function [f,g] = myfunc(x)&#xA;&#xA;s = reshape(x, [sqrt(numel(x)),sqrt(numel(x))]);&#xA;&#xA;f = trace(s'*s) + trace(inv(s'*s)^2);&#xA;&#xA;g = 2*s -4*s*(inv(s'*s))^3;&#xA;g = g(:);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help solving my problem would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3880" LastEditorUserId="10134" LastEditDate="2016-03-07T15:29:12.457" LastActivityDate="2016-03-07T15:29:12.457" Title="How to solve a constrained optimization problem using minFunc or minConf" Tags="|matlab|convex-optimization|linear-programming|nonlinear-programming|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23230" PostTypeId="2" ParentId="6791" CreationDate="2016-02-27T19:34:21.957" Score="1" Body="&lt;p&gt;The other answers suggest normalizing eigenvectors so that their first component is positive, but this seems bad advice to me: if the computed eigenvectors are &lt;code&gt;v = (1e-17, 1)&lt;/code&gt; and &lt;code&gt;w =(-1e-17, 1)&lt;/code&gt;, then normalizing them to &lt;code&gt;v_n = (1e-17, 1)&lt;/code&gt; and &lt;code&gt;w_n =(1e-17, -1)&lt;/code&gt; and checking their distance with &lt;code&gt;norm(v_n, w_n)&lt;/code&gt; identifies them as far apart when they are in fact very close to one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice is that you are looking at the problem at the wrong level: you don't need to normalize your output, you need to change the way in which you make comparisons between vectors. Compare two vectors $a,b$ by computing their angle, or correlation: $\frac{|a^Tb|}{\lVert a \rVert \lVert b\rVert}$. If this quantity is close to 1, then the two vectors differ only by a phase.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2016-02-28T21:52:43.093" LastActivityDate="2016-02-28T21:52:43.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23232" PostTypeId="1" CreationDate="2016-02-27T20:58:58.173" Score="1" ViewCount="944" Body="&lt;p&gt;I am having a really hard time getting any kind of reliable / consistent result from my Metropolis code. I have torn it apart and am now examining just the randomness in my random number generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I decided I would just run many iterations over a spread from -1 to 1 and it should give me an average of 0. I was completely surprised by the lack of precision. My code does give results about zero, but, from -200 to 200 with almost an equal distribution throughout the range. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to find a way to bring this closer together. I have tried more runs (knowing 10,000 isn't really that many) but even up to a billion runs, the averages can range just as much if not more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to fix this randomness?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    program test_rand&#xA;    real*8 x, y, z, EAx, EAy, EAz, Tx, Ty, Tz&#xA;    integer j, i&#xA;&#xA;    CALL init_random_seed&#xA;&#xA;    Do i = 1, 10000&#xA;        EAx = 0.D0&#xA;        EAy = 0.D0&#xA;        EAz = 0.D0&#xA;        Tx = 0.D0&#xA;        Ty = 0.D0&#xA;        Tz = 0.D0&#xA;        Do j=1,10000&#xA;            CALL RANDOM_NUMBER(x)&#xA;            CALL RANDOM_NUMBER(y)&#xA;            CALL RANDOM_NUMBER(z)&#xA;            EAx = 2.0D0 * x - 1.0D0&#xA;            EAy = 2.0D0 * y - 1.0D0&#xA;            EAz = 2.0D0 * z - 1.0D0&#xA;            Tx = EAx + Tx&#xA;            Ty = EAy + Ty&#xA;            Tz = EAz + Tz&#xA;        end do&#xA;        write(50,*) i, Tx, Ty, Tz&#xA;    end do&#xA;    end program test_rand&#xA;&#xA;    !   initialize a random seed from the system clock at every run (fortran 95 code)&#xA;    subroutine init_random_seed()&#xA;    INTEGER :: i, n, clock&#xA;    INTEGER, DIMENSION(:), ALLOCATABLE :: seed&#xA;    CALL RANDOM_SEED(size = n)&#xA;    ALLOCATE(seed(n))&#xA;    CALL SYSTEM_CLOCK(COUNT=clock)&#xA;    seed = clock + 37 * (/ (i - 1, i = 1, n) /)&#xA;    CALL RANDOM_SEED(PUT = seed)&#xA;    DEALLOCATE(seed)&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19126" LastEditorUserId="19126" LastEditDate="2016-02-27T21:28:12.860" LastActivityDate="2016-02-27T21:28:12.860" Title="Generating Random Numbers in Fortran for Metropolis method" Tags="|fortran|statistics|random-number-generation|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23234" PostTypeId="1" CreationDate="2016-02-28T13:13:21.250" Score="1" ViewCount="116" Body="&lt;p&gt;Webster equation is a popular generalization of 1D wave equation used for ducts of variable cross-section $S \equiv S(x)$. Assuming harmonicity in time, the spatial equation for propagation of variable $\Phi(x)$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial^2 \Phi}{\partial x^2} + \frac{\partial (\ln S(x))}{\partial x}\frac{\partial \Phi}{\partial x} + k^2\Phi^2 = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What FD scheme would be appropriate assuming that we have boundary conditions at $\Phi(0)$ and $\Phi(L)$, i.e. no $\partial_x\Phi(0)$ etc.?&lt;/p&gt;&#xA;" OwnerUserId="17721" LastActivityDate="2016-02-28T18:16:09.133" Title="Finite difference scheme for Webster equation" Tags="|finite-difference|boundary-conditions|wave-propagation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23235" PostTypeId="1" AcceptedAnswerId="23236" CreationDate="2016-02-28T15:18:20.403" Score="0" ViewCount="752" Body="&lt;p&gt;I am very new to optimization. so, sorry If I ask a simple question. I have a problem with a dozen of variables. I want to use Augmented Lagrangian for solving the problem. I write a code based on the method(in matlab). Unfortunately, the code doesn't work And I don't know how to understand what's going wrong? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to debug an augmented lagrangian program? Is there a method for checking variable changes to understand the source of the bug? or could you provide some hints or insight on how to debug an augmented lagrangian code?&lt;/p&gt;&#xA;" OwnerUserId="8223" LastActivityDate="2016-02-28T16:51:58.457" Title="How to understand what's going wrong in a code for solving a problem with augmented lagrangian?" Tags="|optimization|convex-optimization|constrained-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23236" PostTypeId="2" ParentId="23235" CreationDate="2016-02-28T16:15:46.217" Score="5" Body="&lt;p&gt;It's very difficult to provide a useful answer to this question because you haven't provided enough details about your problem and what you have already tried.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various conditions under which the augmented Lagrangian method is certain to converge asymptotically to an optimal solution.  You should make sure that the problem you're trying to solve satisfies these conditions so that there's a sound theoretical basis for what you're doing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the augmented Lagrangian method, you solve an unconstrained minimization problem within each major iteration.  If the procedure that you're using to solve that unconstrained problem isn't working properly than the overall method can easily fail to converge to a solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The augmented Lagrangian method makes use of a penalty parameter that can be adjusted after each major iteration of the algorithm.  In theory, the method is guaranteed to converge if a large enough penalty parameter is used.  In practice, using too large a penalty parameter can make it hard to solve the unconstrained minimization problem and slow down convergence.&lt;br&gt;&#xA;Starting with a bad value of the parameter (too big or too small) or adjusting the parameter incorrectly (not increasing the penalty enough) can lead to slow convergence or complete failure.  In my experience, it's almost always necessary to run some experiments to adjust the penalty parameter for good convergence on a new class of problems.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't said whether the method is failing entirely or simply converging slowly on your problem.  For users who are used to the fast quadratic convergence of Newton based methods, it can be frustrating to see the slower convergence of augmented Lagrangian methods.  You're often limited in practice to getting solutions that are only accurate to a few digits.  If you need solutions that are accurate to 8-10 digits, than augmented Lagrangian methods are probably not the best way to go.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendations for debugging this would be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Make sure you've got a theoretical convergence result for the class of problems you're considering.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Make sure that your code is correctly minimizing the augmented Lagrangian in each iteration.  You may be able to get away with an imprecise solution later, but for debugging I'd just try for the best accuracy you can get.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adjust the penalty parameter so that you're getting reasonably fast convergence to feasibility.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You should obviously be monitoring the feasibility of the solution after each major iteration.  Assuming the constraints are reasonably scaled, simply computing the norm of the constraint violations is usually a good measure.  You can also track the evolution of the Lagrange multipliers to see whether they're converging to some fixed values.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you've got access to some other reliable solver, you can solve your problem using that solver first so that you know what an optimal solution looks like.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-02-28T16:51:58.457" LastActivityDate="2016-02-28T16:51:58.457" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23237" PostTypeId="2" ParentId="23234" CreationDate="2016-02-28T17:40:56.083" Score="2" Body="&lt;p&gt;This is not something I've worked on, so I do not know if there is a recommended approach to this, only how I would approach this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm assuming you can evaluate $C(x) = (\ln(S(x)))_x$ wherever you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\phi$ is prescribed at $0$ and $L$, I would discretise on $x_i = L\frac{i}{N+1}$ for $i$ from $1$ to $N$. Then the finite difference operators could be combined into a matrix $A$ and boundary condition correction term vector $\mathbf{b}$ with&#xA;$$\begin{gathered}&#xA;A = \frac{1}{\Delta x^2}\delta^2_x + \frac1{2\Delta x}\mathrm{diag}(C(x_i))\delta_{2x}\,,\\&#xA;\mathbf{b}_i = &#xA;\begin{cases}&#xA;  \left(\frac1{\Delta x^2} - \frac1{2\Delta x}c(x_1)\right)\phi(0)\quad&amp;amp;\text{if}\quad i=1\,,\\&#xA;  \left(\frac1{\Delta x^2} + \frac1{2\Delta x}c(x_N)\right)\phi(L) &amp;amp;\text{if}\quad i=N\,,\\&#xA;0&amp;amp;\text{otherwise}\,,&#xA;\end{cases}&#xA;\end{gathered}&#xA;$$&#xA; so that&#xA;$$\left[\phi_{xx} + C(x)\phi_x\right]_{x_i} \approx (A\underline{\phi} + \mathbf{b})_i\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the solution, I would use Newton's method, looking for the root of&#xA;$$F(\underline{\phi}) = A\underline{\phi} + \mathbf{b} + k^2 \mathrm{diag}(\phi_i^2)$$&#xA;which has tridiagonal Jacobian&#xA;$$\mathrm{J}(\underline{\phi}) = A + 2k^2 \mathrm{diag}(\phi_i)\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(For completion) If there were a Neumann boundary condition, for example&#xA;$$\phi_x(0) = \alpha\,,$$&#xA;I would include $\phi(0)$ in the solution vector $\underline{\phi}$, then I would construct $\mathrm{b}$ by considering a &lt;em&gt;ghost point&lt;/em&gt; at $x = -\Delta x$ satisfying&#xA;$$\frac{\phi(\Delta x) - \phi(-\Delta x)}{2\Delta x} = \alpha\,.$$&#xA;I believe mixed boundary conditions wouldn't be much more complicated than non-zero Neumann.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="17671" LastEditDate="2016-02-28T18:16:09.133" LastActivityDate="2016-02-28T18:16:09.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23239" PostTypeId="1" AcceptedAnswerId="23240" CreationDate="2016-02-29T10:42:32.103" Score="0" ViewCount="3701" Body="&lt;p&gt;I'm trying to find inverse of a matrix using c++ library &lt;a href=&quot;http://arma.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;armadillo&lt;/a&gt; or &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;eigen&lt;/a&gt;. But first row of the matrix is wrong, please note that other elements are correct. I'm unable to understand what mistake I committed, please help me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;armadillo&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;Eigen/Dense&amp;gt;&#xA;using namespace std;&#xA;using namespace arma;&#xA;using namespace Eigen;&#xA;&#xA;int main()&#xA;{&#xA;    vec a;&#xA;int h=5;&#xA;&#xA;    MatrixXd b(5,5);&#xA;    MatrixXd im(5,5);&#xA;&#xA;     a &amp;lt;&amp;lt;-2 &amp;lt;&amp;lt; -1 &amp;lt;&amp;lt; 0 &amp;lt;&amp;lt; 1 &amp;lt;&amp;lt; 2;&#xA;&#xA;   for(int i1=0;i1&amp;lt;h;i1++)&#xA;   {&#xA;&#xA;            b(i1,0)=1;&#xA;            b(i1,1)=a(i1);&#xA;            b(i1,2)=a(i1)*a(i1)+1/12;&#xA;            b(i1,3)=pow(a(i1),3)+a(i1)/4;&#xA;            b(i1,4)=1/80+pow(a(i1),2)/2+pow(a(i1),4);&#xA;&#xA;&#xA;    }&#xA;&#xA;    im=b.inverse();&#xA;&#xA;    cout&amp;lt;&amp;lt;im&amp;lt;&amp;lt;endl;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output of current code is&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GNPOo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GNPOo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Correct result is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   0.00469  -0.06042   1.11146  -0.06042   0.00469&#xA;   0.10417  -0.70833   0.00000   0.70833  -0.10417&#xA;  -0.06250   0.75000  -1.37500   0.75000  -0.06250&#xA;  -0.08333   0.16667   0.00000  -0.16667   0.08333&#xA;   0.04167  -0.16667   0.25000  -0.16667   0.04167&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16930" LastActivityDate="2016-02-29T11:55:21.043" Title="Inversion of a matrix using c++ library armadillo or eigen" Tags="|c++|eigen|armadillo|" AnswerCount="1" CommentCount="0" ClosedDate="2016-03-01T01:59:37.060" ContentLicense="CC BY-SA 3.0" />
  <row Id="23240" PostTypeId="2" ParentId="23239" CreationDate="2016-02-29T11:55:21.043" Score="6" Body="&lt;p&gt;Please make following changes in your code and check the answer&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        b(i1,0)=1;&#xA;        b(i1,1)=a(i1);&#xA;        b(i1,2)=a(i1)*a(i1)+1.0/12.0;&#xA;        b(i1,3)=pow(a(i1),3)+a(i1)/4;&#xA;        b(i1,4)=1.0/80.0+pow(a(i1),2)/2+pow(a(i1),4);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3549" LastActivityDate="2016-02-29T11:55:21.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23241" PostTypeId="1" CreationDate="2016-02-29T11:58:55.907" Score="5" ViewCount="139" Body="&lt;p&gt;Computers can work only with &lt;a href=&quot;https://en.wikipedia.org/wiki/Computable_number&quot; rel=&quot;nofollow&quot;&gt;computable numbers&lt;/a&gt;, while most of the algorithms are based on analysis of real numbers (real analysis). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I heard of the existence of &lt;a href=&quot;https://en.wikipedia.org/wiki/Computable_analysis&quot; rel=&quot;nofollow&quot;&gt;computable analysis&lt;/a&gt; I immediately had these questions: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can it help numerical analysis in some way? &lt;/li&gt;&#xA;&lt;li&gt;Can we for example, help prove convergence of algorithms that are not convergent in the reals? &lt;/li&gt;&#xA;&lt;li&gt;Can we find different algorithms that work for computables faster than they do for reals?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3740" LastEditorUserId="10134" LastEditDate="2016-03-07T15:29:18.083" LastActivityDate="2016-03-07T15:29:18.083" Title="Are there any benefits of computable analysis to numerical algorithms" Tags="|numerical-analysis|convergence|approximation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23242" PostTypeId="1" CreationDate="2016-02-29T12:49:34.210" Score="0" ViewCount="357" Body="&lt;p&gt;I have been doing finite element analysis using Matlab. I look for many examples and tutorials producing only the stiffness matrix letting elements being weightless. However, in my case, I need to do a soil deformation where the self-weight of elements is significant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any guidance will be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="19199" LastEditorUserId="20688" LastEditDate="2019-06-22T20:48:44.767" LastActivityDate="2019-06-22T20:48:44.767" Title="How to create node to node lumping" Tags="|finite-element|matlab|reference-request|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="23245" PostTypeId="2" ParentId="23228" CreationDate="2016-03-01T09:39:05.427" Score="4" Body="&lt;p&gt;Not a direct answer to your title question, but I think you are better off attacking this problem from the semidefinite domain instead. Trivial approach is to linearize the objective at some initial guess, solve the linearized problem, perform a line-search along the computed direction, and repeat until the objective doesn't improve. The code below does this in an implementation using YALMIP (disclaimer: developed by me). It terminates in a second or so, using an SDP solver such as Mosek or SDPT3.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Z = rand(15,10);&#xA;&#xA;% Take a step D from current solution Qi, Q = Qi+D&#xA;D = sdpvar(10);&#xA;% Initial guess Q = Qi = Z'*Z&#xA;Qi = Z'*Z;&#xA;% Values to test in brute-force line-search&#xA;alpha = logspace(-4,3,100);&#xA;% Run some iterations&#xA;for i = 1:10&#xA;    % Linearized objective&#xA;    Objective = trace(Qi + D) + trace(Qi^-2 - Qi^-2*D*Qi^-1 - Qi^-1*D*Qi^-2)&#xA;    % Solve linear SDP&#xA;    optimize([Qi+D&amp;gt;=0, norm(Qi + D - Z'*Z,'fro') &amp;lt;= .01],Objective)&#xA;    % Perform naive line-search&#xA;    for j = 1:length(alpha)&#xA;        Qtest = Qi + alpha(j)*value(D);&#xA;        if min(eig(Qtest))&amp;gt;0  &amp;amp;&amp;amp; norm(Qtest - Z'*Z,'fro')&amp;lt;=0.01&#xA;            Objtest(j) = trace(Qtest) + trace(Qtest^-2);&#xA;        else&#xA;            Objtest(j) = inf;&#xA;        end&#xA;    end&#xA;    if all(Objtest &amp;gt;= trace(Qi) + trace(Qi^-2))&#xA;        break&#xA;    end&#xA;    plot(alpha,Objtest)&#xA;    % Pick best step&#xA;    [~,best] = min(Objtest);&#xA;    % Update solution&#xA;    Qi = Qi + alpha(best)*value(D);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2016-03-01T09:56:53.620" LastActivityDate="2016-03-01T09:56:53.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23246" PostTypeId="1" CreationDate="2016-02-23T16:39:41.170" Score="4" ViewCount="81" Body="&lt;p&gt;I have a problem that I've framed out in a particular way, but I don't know if I'm re-inventing the wheel here. Is there an existing literature base in this problem? Does it have a corresponding term of art? Where would you recommend that I start reading about this type of problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic problem that I have is that I need to make predictions about where ants will be in a large terrain. I have all sorts of information about the terrain -- its elevation, bodies of water, the flora cover, the distances to cities and other human settlements, the weather -- all of these things plausibly influence where and how ants move across this terrain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to the terrain data, I have data about where ants have historically been found with lat/longs to a high degree of precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also know that ants will &lt;em&gt;generally&lt;/em&gt; be moving from one end of the terrain to the other. But there is not a hard requirement that they always move closer to the opposite end at every step. We just know that they're starting at one end and want to go somewhere on the other end. The &quot;box&quot; of terrain is just a very wide rectangle, and I'm comfortable with assuming that ants want to go from the longest side to the opposite longest side. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plausibly some ant could go from one corner to the farthest corner, but it's more likely that they'll go to a nearer point because the path is shorter, so probably the path is less costly. This is where the calculus of variations comes in: I want to make inferences about the plausible paths that ants will take. Likewise, an ant could make a several U-turns to avoid some obstacle (like a suddenly steep elevation, or crossing a body of water), and doing so would incur a cost cost, but &lt;em&gt;probably&lt;/em&gt; won't make several such U-turns, if judicious planning could avoid it ahead of time by taking an alternative route. This is similar to how water will usually flow down hill, but special circumstances can make a geyser.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conceptually, the cost function is a function of the terrain data, but the cost itself is unknown. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The straight-up machine learning approach would just be to use the terrain data to make inferences about where ants will be directly as some sort of regression problem -- this can be done any number of arbitrary ways, but I wonder if doing so without considering paths will lead to implausible inferences, such as making predictions that ants might be at any point along a line, &lt;em&gt;except&lt;/em&gt; some vast swath in the middle where the model predicts probability 0, or that there is a &quot;dotted line&quot; of alternating high- and low-probability regions. I wouldn't really believe that model, and would tend to think that it's overfitting the data. It seems like making inferences about the cost function, however indirectly, would impose a certain amount of regularization on the solution driven purely by the &quot;pure&quot; machine learning regression problem, so incorporating that information would improve our inferences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So where would I start reading about this style of problem?&lt;/p&gt;&#xA;" OwnerUserId="15249" OwnerDisplayName="user777" LastActivityDate="2016-03-01T09:52:50.087" Title="Calculus of Variations with unknown cost function but some data" Tags="|regression|machine-learning|optimization|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="23247" PostTypeId="2" ParentId="23228" CreationDate="2016-03-01T14:04:23.323" Score="4" Body="&lt;p&gt;Adding another answer, as I just realized that the problem is easily solved as a linear SDP. Let $Q=S^TS$ and you have the objective $\mathrm{trace}~Q + \mathrm{trace}~Q^{-2}$. Introduce an upper bound $X\succeq Q^{-1}$ and minimize  $\mathrm{trace}~Q + \mathrm{trace}~X^{2}$. At optimality you will have $X= Q^{-1}$. The constraints $X\succeq Q^{-1}$ and $Q\succeq 0$ is converted to an LMI using a Schur complement, and the objective is convex quadratic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also here implemented in YALMIP&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Q = sdpvar(n);&#xA;X = sdpvar(n);&#xA;Model = [[X eye(n);eye(n) Q]&amp;gt;=0, norm(Q-Z'*Z)&amp;lt;= .01]&#xA;Objective = trace(Q) + trace(X*X);&#xA;optimize(Model,Objective)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2016-03-01T14:16:36.553" LastActivityDate="2016-03-01T14:16:36.553" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23248" PostTypeId="2" ParentId="23242" CreationDate="2016-03-01T14:26:03.937" Score="1" Body="&lt;p&gt;A load due to gravity or self-wieght is commonly referred to as a body force in continuum&#xA;mechanics. Finite element texts often use this term when referring to this type of loading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each finite element, an equivalent nodal force vector due to body forces can be calculated&#xA;as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ {\bf f}^e_i = \int N_i \left\{ \begin{array}{c} &#xA;b_x \\ b_y &#xA;\end{array} \right\} dV $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N_i$ is the element shape function at the ith node, &#xA;$b_x$ the body force in the x direction, and $b_y$ is the body force&#xA;in the y direction. For a gravity load in the negative y direction, $b_y = -\rho g$&#xA;where $g$ is the acceleration due to gravity and $\rho$ is the material&#xA;density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The element ${\bf f}^e_i$ are assembled into the global load vector&#xA;in a manner analogous to how element stiffness matrices are assembled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to learn more, most finite element texts that emphasize structural&#xA;analysis discuss this procedure in sections describing calculation of &#xA;equivalent nodal loads. For example, see section 4.2 in&#xA;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/097900490X&quot; rel=&quot;nofollow&quot;&gt;http://www.amazon.com/Finite-Element-Procedures-K-J-Bathe/dp/097900490X&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-03-01T14:26:03.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23250" PostTypeId="1" AcceptedAnswerId="25568" CreationDate="2016-03-01T16:23:37.413" Score="16" ViewCount="273" Body="&lt;p&gt;After doing some mathematics related to the stability of elements in 3D Stokes problem I was slightly shocked to realize that $P_2-P_1$ is not stable for an arbitrary tetrahedral mesh. More precisely, in case you have an element where all nodes and three out of four facets lie on the boundary of the domain with a Dirichlet condition, you end up getting a singular matrix. This is in fact fairly trivial to conclude from the weak form of the Stokes system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tested the only commercial Stokes code I have an access to (COMSOL) and it allowed me to create such a mesh. Upon clicking solve I get 'Error: Singular matrix' as expected. (I am under the impression that COMSOL uses $P_2-P_1$ for its creeping flow module.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/00at2.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/00at2.png&quot; alt=&quot;The bad mesh.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To further test that the problem was not related to other configurations I tried the following mesh and everything works as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WEH9F.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WEH9F.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt; Is this kind of constraint taken into account in (adaptive or non-adaptive) mesh generators? I see from various research papers that this element seems to be quite popular. Are these kind of boundary instabilities generally disregarded as insignificant when choosing a method to use? More importantly, what does it really mean to have a &lt;em&gt;stable finite element&lt;/em&gt;, i.e., what kind of mesh-dependent instabilities are too much so that we conclude that the method is bad?&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2016-11-16T10:18:14.310" Title="Usefulness of elements with mesh-dependent stability" Tags="|stability|navier-stokes|unstructured-mesh|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23252" PostTypeId="1" CreationDate="2016-03-02T10:18:24.940" Score="1" ViewCount="46" Body="&lt;p&gt;I have a matrix, $X$, it is symmetric. I project $C \odot X$ and $D\odot X$ to semidefinite cone. $C$ is a Gramian matrix with some elements near zero and of course semidefinite, with one row and column equal to zero. $D$ is a matrix of ones except one column and row equal to zero (not the same column and row as $D$). Both are symmetric. After projection, I want to obtain $X':=\mathrm{proj}(C\odot X)./C$, (element wise division) but &#xA;some values became very large, may because of corresponding elements in&#xA;the matrix $C$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to correctly obtain $X'$?&lt;/p&gt;&#xA;" OwnerUserId="8223" LastEditorUserId="1804" LastEditDate="2016-03-02T11:10:15.710" LastActivityDate="2016-03-03T04:50:14.747" Title="How to compute $\mathrm{proj}_{SDP}(C\odot X)./C$ without numerical problems?" Tags="|floating-point|semidefinite-programming|projection|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23253" PostTypeId="1" CreationDate="2016-03-02T11:32:27.510" Score="3" ViewCount="211" Body="&lt;p&gt;The free-streaming term in stationary 1D Boltzmann equation satisfies the equilibrium distribution function, that is:&#xA;\begin{equation}&#xA;\text{Fr}\{f\} := v(k)\dfrac{\partial f(x,k)}{\partial x} &#xA;- \dfrac{d\varepsilon_\text{sub}(x)}{dx}\dfrac{\partial f(x,k)}{\partial k}&#xA;\end{equation} &#xA;equals zero by putting $f_0(x,k) = \exp\left(-\dfrac{k^2}{2m}-\varepsilon_\text{sub}(x)\right)$, with $v(k)=\dfrac{k}{m}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, after discretization:&#xA;\begin{equation}&#xA;\text{Fr}\{f(x_i,k_j)\} :=&#xA;\dfrac{k_j}{m}\Big[f(x_{i+1},k_j) - f(x_i,k_j)\Big] - &#xA;\dfrac{\varepsilon_\text{sub}(x_{i+1})-\varepsilon_\text{sub}(x_i)}{\Delta x}  \Big[f(x_i,k_{j+1}) - f(x_i,k_j)\Big]&#xA;\end{equation}&#xA;the equilibrium function $f_0(x_i,k_j) = \exp\left(-\dfrac{k_j^2}{2m}-\varepsilon_\text{sub}(x_i)\right)$ no longer is a solution to our discretized equation (Of course, the solution gets closer and closer to the equilibrium distribution function by refining the grid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is a very important property for our discretization to satisfy $\text{Fr}\{f_0\} = 0$ by construction, could you suggest a suitable discretization scheme in this regard?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-03-02T19:34:10.650" Title="Boltzmann equation and equilibrium distribution function" Tags="|pde|finite-difference|finite-volume|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23254" PostTypeId="1" AcceptedAnswerId="23256" CreationDate="2016-03-02T13:06:40.720" Score="2" ViewCount="255" Body="&lt;p&gt;I have implemented the Richardson extrapolation of the Euler-Maruyama method to 4th order, to estimate the moments of SDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Euler-Maruyama works, and I would expect the Richardson extrapolation to work too, because when I suppress the noise contribution it gives the correct solution in the resulting ordinary differential equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I add the noise term, I don't get the moments (the variance, in particular) I would expect, as I do with the regular Euler method, so apparently I am doing something wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my MATLAB code for the extrapolation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [t, y] = Richardson_weak_4th_order(a, b, t_interval, y0, h)&#xA;&#xA;[t1, y1] = euler_maruyama(a, b, t_interval, y0, h);&#xA;[~, y2]  = euler_maruyama(a, b, t_interval, y0, h/2);&#xA;[~, y3]  = euler_maruyama(a, b, t_interval, y0, h/4);&#xA;[~, y4]  = euler_maruyama(a, b, t_interval, y0, h/8);&#xA;&#xA;% Extrapolation&#xA;y2 = y2(1:2:end,:);&#xA;y3 = y3(1:4:end,:);&#xA;y4 = y4(1:8:end,:);&#xA;&#xA;t = t1;&#xA;y = 1/21*(64*y4 - 56*y3 + 14*y2 - y1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For simplicity, I am testing with an Ornstein-Uhlenbeck process&#xA;$$&#xA;\mathbb{d} X_t = (m - X_t)\mathbb{d}t + \sigma \mathbb{d}W_t&#xA;$$&#xA;for which $\mathbb{E}(X) \simeq m$ and $\operatorname{Var}(X) \simeq \frac{\sigma^2}{2}$. The results are OK with the Euler method, but the variance is quite bigger with the extrapolation. What am I missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; The problem is Richardson extrapolation was not implemented on the estimated moments (&lt;strong&gt;as it should&lt;/strong&gt;), but to realizations of the process.&lt;/p&gt;&#xA;" OwnerUserId="19467" LastEditorUserId="19467" LastEditDate="2016-03-02T15:00:28.367" LastActivityDate="2016-03-02T16:03:42.223" Title="Problem with Richardson extrapolation method for weak convergence in SDE" Tags="|ode|convergence|stochastic|diffusion|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23255" PostTypeId="1" AcceptedAnswerId="23931" CreationDate="2016-03-02T14:37:12.863" Score="4" ViewCount="702" Body="&lt;p&gt;I'm looking for a benchmark for the lid-driven cavity problem in 3D to compare the results of my code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2D I used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0021999182900584&quot; rel=&quot;noreferrer&quot;&gt;U. K. N. G. Ghia, K. N. Ghia and C. T. Shin (1982)&#xA;&lt;em&gt;High-Re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method&lt;/em&gt;. Journal of computational physics.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know a benchmark work for the cavity driven in 3D? Which is the &lt;em&gt;classic&lt;/em&gt; paper to compare 3D codes?&lt;/p&gt;&#xA;" OwnerUserId="17630" LastEditorUserId="1804" LastEditDate="2016-03-02T14:59:46.163" LastActivityDate="2016-11-29T14:33:23.793" Title="Lid-driven Cavity benchmark in 3D. Classical paper to compare" Tags="|reference-request|navier-stokes|3d|benchmarking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23256" PostTypeId="2" ParentId="23254" CreationDate="2016-03-02T15:19:04.223" Score="2" Body="&lt;p&gt;You are applying Richardson extrapolation to increasingly-accurate &lt;em&gt;independent&lt;/em&gt; sample paths of an OU process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would this work even for independent identically normally distributed random variables $y_1,y_2,y_3,y_4\sim\mathrm{N}(\mu,\sigma^2)$? These have already &quot;converged&quot; to the true distribution $\mathrm{N}(\mu,\sigma^2)$, and you might expect the extrapolation to just reproduce the variables, but&#xA;$$ \tfrac1{21}(64y_4-56y_3+14y_2-y_1) \sim \mathrm{N}(\mu, 16.8\sigma^2). $$&#xA;If they are independent, that destroys any chance that this linear combination would do the right thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to a cursory search on Google Scholar, Richardson extrapolation can indeed be helpful. But I think it's necessary to be careful about how you pick your samples for it, and how independent they are. The above argument with $y_i$ would apply as well when $y_i$ are estimates of a sample moment of the generated process paths: the variance of the sample moments will be amplified by the process of Richardson extrapolation, so this is something you need to handle carefully.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So Richardson extrapolation can only be applied, with a little care, to sample moments, but not at all not sample paths. For sample moments specifically, it would indeed help: independent $y_k$ would be distributed as $\mathrm{N}(\mu,2^{-2(k-1)}\sigma^2)$, and then&#xA;$$ \tfrac1{21}(64y_4-56y_3+14y_2-y_1) \sim \mathrm{N}(\mu, 0.7\sigma^2). $$&#xA;If you were to do the same thing with $y_k$'s coming from the &lt;em&gt;same&lt;/em&gt; sample, then by my calculations the variance would (for sample means) be $ 0.15\sigma^2$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-03-02T16:03:42.223" LastActivityDate="2016-03-02T16:03:42.223" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23257" PostTypeId="1" AcceptedAnswerId="23502" CreationDate="2016-03-02T16:52:29.780" Score="2" ViewCount="80" Body="&lt;p&gt;I understand how MH work, I'm able to use it to simulate e.g. 2D Ising model. What I don't understand is what you actually take average of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run the simulation, it reaches equlibrium after some time:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xzsfR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xzsfR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Now each new state is going to be the same, so I see no point in taking the average or variance of the sequence that would follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I have to run the simulation each time over and over again from the beginning and take averages of the reached equilibrium states? This seems very inefficient to me.&lt;/p&gt;&#xA;" OwnerUserId="18910" LastActivityDate="2016-03-29T18:25:01.550" Title="Use of Metropolis-Hasting algorithm for gathering statistics" Tags="|monte-carlo|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23258" PostTypeId="1" CreationDate="2016-03-02T18:16:31.357" Score="1" ViewCount="304" Body="&lt;p&gt;I am trying to solve a singular linear least square problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$minimize: \phantom{2} ||Ax-b||^2 \\&#xA;subject \phantom{2} to: \phantom{2} x \ge 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $ A \in R^{n \times m} $, and $ n\lt m$. here m is a very large number and the matrix $A$ is dense and almost full.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The least square solution for this is equivalent to solve (maybe not equivalent due to the constraints, just to show the reasoning):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A^TAx=A^Tb,\phantom{2} x \ge 0 \phantom{,,,} (*)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but here $A^TA$ is singular because $ n\lt m$. So my question ends up how to solve the above constrained singular linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only idea I came up with is using SVD like the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A^TA=VSV^T=[V_1 \phantom{,,} V_2] &#xA;       \begin{bmatrix}&#xA;        S_{11} &amp;amp; 0 \\&#xA;        0 &amp;amp; 0 \\&#xA;        \end{bmatrix} &#xA;        \begin{bmatrix}&#xA;        V_1^T \\&#xA;        V_2^T \\&#xA;        \end{bmatrix}&#xA;=V_1S_{11}V_1^T$$&#xA;then setting $ x=V_1z $ leads to $ z=V_1^Tx $, and solving $(*)$ equals to solving &#xA;$$S_{11}z=V_1^TA^Tb, \phantom{,} V_1z \ge0$$&#xA;which can be reformulated as,&#xA;$$minimize: \phantom{2} \frac{1}{2}z^TS_{11}z-b^TAV_1z \\&#xA;subject \phantom{2} to: \phantom{2} V_1z \ge 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, unfortunately, because the matrix $A$ is large and dense, doing SVD needs large memory and is slow. When doing this using MATLAB it prompted me saying &quot;out of memory&quot;. Is there any other more efficient way to solve this singular least square problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Is it possible there is no solution to the reformulated minimization problem? I know the objective function is convex since $S_{11}$ is positive definite, and of course the linear constraints are convex. But is it possible the it is infeasible, that the linear constraints might contradict with each other?&lt;/p&gt;&#xA;" OwnerUserId="8213" LastEditorUserId="1804" LastEditDate="2016-05-27T20:44:11.187" LastActivityDate="2016-12-16T20:23:38.660" Title="Efficient methods to solve large dense singular least square problem (linear system)" Tags="|least-squares|svd|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23259" PostTypeId="2" ParentId="23253" CreationDate="2016-03-02T19:34:10.650" Score="2" Body="&lt;p&gt;This is an interesting problem that occurs in a similar form also for other  mathematical models given by PDEs. I have no experience with your particular formulation, but I can try to inspire you by suggestions for similar problems or give some comments that can be useful in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be quite important to know why you need your numerical scheme to produce exact solution for special function $f_0$. I am familiar e.g. with such requirement for so called density driven flows where for special form of given density that is variable only in the direction of gravity one expects that the solution, here a pressure, is approximated (or used exactly) such that &lt;em&gt;no artificial numerical flow is created where the exact flow is zero&lt;/em&gt;. Another very well-known case are so called well-balanced numerical schemes for shallow water problems that produce zero numerical flow for hydrostatic situations for arbitrary bottom topography. Therefore it might be useful to understand what is the motivation in your case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, here are two brief suggestions. In the case of density driven flow there exist so called consistent velocity approximations to resolve similar problem. The idea is that the unknown solution (there the pressure) is expressed as a difference of two functions where one of them is the exact response for the special variable density. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case it might be that instead of solving for $f$ you solve for $\tilde f$ such that $f=\tilde f - f_0$ where $f_0$ is your equilibrium distribution that should be uniquely defined for your input data. To obtain the PDE for $\tilde f$ you plug $f$ into your PDE. If your problem is $\hbox{Fr}\{f\}=0$ then you obtain $\hbox{Fr}\{\tilde f\}=\hbox{Fr}\{\tilde f_0\}$, and , clearly, the solution is $\tilde f = f_0$, if you use the same discretization for the both sides (that is natural).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another solution to your requirement is a special interpolation in the construction of numerical approximation, but this looks quite complicated in your case if the function $\epsilon_{\hbox{sub}}$ can have a general form. Where such procedure is successful is e.g. a numerical solution for one-dimensional stationary advection-diffusion equation where in the case of constant input coefficients one obtains an exponential form of exact solution. Motivated by this, one propose a numerical scheme using exponential profile assumption for numerical solution between two grid points that is appropriate also for variable input coefficients and as a &quot;side product&quot; it can produce the exact solution in the case of constant input coefficients.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-03-02T19:34:10.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23260" PostTypeId="1" CreationDate="2016-03-02T19:54:00.263" Score="2" ViewCount="3256" Body="&lt;p&gt;I am trying to learn the Lattice-Boltzmann method and was looking for some good beginner resources explaining the method. I have been looking at some codes online, but have been having trouble understanding them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone suggest some good introductory resources about the Lattice-Boltzmann method? If you know of any codes that might be useful then please link to them as well (Matlab preferred).  &lt;/p&gt;&#xA;" OwnerUserId="19470" LastEditorUserId="18017" LastEditDate="2016-03-04T16:08:40.283" LastActivityDate="2023-01-15T20:01:24.810" Title="Introduction to Lattice Boltzmann methods" Tags="|matlab|reference-request|lattice-boltzmann-methods|" AnswerCount="1" CommentCount="2" ClosedDate="2016-03-23T15:32:33.820" ContentLicense="CC BY-SA 3.0" />
  <row Id="23261" PostTypeId="1" AcceptedAnswerId="23277" CreationDate="2016-03-03T03:12:34.890" Score="3" ViewCount="700" Body="&lt;p&gt;I want to solve the PDE equation numerically. For this, I started my study with something simple; heat equation&#xA;$$&#xA;\frac{\partial u}{\partial t}=\frac{\partial^2 u}{\partial^2 x}&#xA;$$&#xA;with the initial condition&#xA;$$&#xA;u(x,0)=1\qquad(-1&amp;lt;x&amp;lt;1)&#xA;$$&#xA;I assumed the constant is set to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My next step was to turn this into discrete equation. I used&#xA;$$&#xA;\frac{\partial u}{\partial t}=\frac{u(t+\Delta t)-u(t)}{\Delta t},\qquad&#xA;\frac{\partial^2 u}{\partial x^2}=\frac{u(x+2\Delta x)+u(x)-2u(x+\Delta x)}{(\Delta x)^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Arranging all these,&#xA;$$&#xA;\frac{u(x,t+\Delta t)-u(x,t)}{\Delta t}&#xA;=&#xA;\frac{u(x+2\Delta x,t)+u(x,t)-2u(x+\Delta x,t)}{(\Delta x)^2}&#xA;$$&#xA;$$&#xA;u(x,t+\Delta t)=u(x,t)+\frac{\Delta t}{(\Delta x)^2}\left(&#xA;u(x+2\Delta x,t)+u(x,t)-2u(x+\Delta x,t)&#xA;\right)&#xA;$$&#xA;so, at arbitrary $x$, after time $\Delta t$, the value will be function of $u$ at $x+2\Delta x$ and $x+\Delta x$, before $\Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my algorithm, and I wrote a code for Matlab&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc; clear all;&#xA;pt=1000;                    %%% number of points&#xA;xsta=-5;                    %%% x start&#xA;xend=5;                     %%% x end&#xA;x=linspace(xsta,xend,pt);   %%% creating x space -5 ~ 5&#xA;t=linspace(0,2,1000);       %%% creating t space 0 ~ 2 sec&#xA;&#xA;dx=abs(x(2)-x(1))/pt;       %%% x interval&#xA;dt=abs(t(2)-t(1))/pt;       %%% t interval&#xA;&#xA;u=zeros(pt,pt);             %%% creating u function&#xA;&#xA;u0=zeros(1,pt);             %%% initial value&#xA;for i=1:pt&#xA;    if x(i)&amp;gt;=-1 &amp;amp;&amp;amp; x(i)&amp;lt;=1&#xA;        u0(i)=1;&#xA;    end&#xA;end                         %%% initial value&#xA;&#xA;u(1,:)=u0;                  %%% inserting initial value to t=0;&#xA;&#xA;for i=2:pt                  %%% t step. it starts from 0+dt&#xA;    for j=1:pt-2;           %%% x step. it ends at xend - dx*2 due to algorithm&#xA;        u(i,j)=u(i-1,j)+((  u(i-1,j+2) + u(i-1,j) -2*u(i-1,j+1)  )/dx)*dt;&#xA;    end&#xA;end&#xA;&#xA;figure();&#xA;plot(x,u(150,:))            %%% plotting at t=0.2983 sec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But when I plot, It gives weird function, and does not look like heat equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, the scale is off by $10^{36}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RWb7M.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RWb7M.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am writing my code for first time, so I don't know what is wrong with my code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any suggestion of better algorithm for this PDE?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, this is just start. I am going to use more complicated, coupled PDE for my project. Is there any good algorithm for this?&lt;/p&gt;&#xA;" OwnerUserId="19465" LastActivityDate="2016-03-13T08:43:37.743" Title="Algorithm suggestion for PDE - example: heat equation" Tags="|pde|computational-physics|parabolic-pde|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23262" PostTypeId="2" ParentId="23252" CreationDate="2016-03-03T04:50:14.747" Score="1" Body="&lt;p&gt;That you get really big numbers is no accident.  Consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C = \left(\begin{array}{ccc}&#xA;2 &amp;amp; 1 &amp;amp; \epsilon\\&#xA;1 &amp;amp; 2 &amp;amp; 1\\&#xA;\epsilon &amp;amp; 1 &amp;amp; 2&#xA;\end{array}\right)&#xA;$$&#xA;and&#xA;$$&#xA;X = \left(\begin{array}{ccc}&#xA;-1 &amp;amp; -1 &amp;amp; 1\\&#xA;-1 &amp;amp; 1 &amp;amp; 1\\&#xA;1 &amp;amp; 1 &amp;amp; 1\\&#xA;\end{array}\right).&#xA;$$&#xA;Note that $X$ has eigenvalues $\{1, \pm 2\}$ and $C$ has eigenvalues within $2\epsilon$ of $\{2 - \sqrt{2}, 2, 2 + \sqrt{2}\}$; these are not close to singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can compute that&#xA;$\mathrm{proj}_{SDP}(C .*X)$ is very close to (entrywise within $10^{-3} + \epsilon$ or so of) the matrix&#xA;$$&#xA;\left(\begin{array}{ccc}&#xA;0.111 &amp;amp; 0.474 &amp;amp; 0.124\\&#xA;0.474 &amp;amp; 2.131 &amp;amp; 0.969\\&#xA;0.124 &amp;amp; 0.969 &amp;amp; 2.007\\&#xA;\end{array}\right),&#xA;$$&#xA;which has some very nonzero entries where $C$ has $\epsilon$'s.&lt;/p&gt;&#xA;" OwnerUserId="3525" LastActivityDate="2016-03-03T04:50:14.747" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23265" PostTypeId="2" ParentId="23260" CreationDate="2016-03-03T17:39:26.643" Score="5" Body="&lt;p&gt;Back in August 2011, I followed a beginner's course (LBM workshop) on LBM in Canada.&#xA;The resources for that course are still up and contain a nice tutorial covering a wide range of topics: theory, coding setup, boundary conditions, multiphase, etc.&lt;/p&gt;&#xA;&lt;p&gt;The simpler tutorials are in Matlab, for the more advanced tutorials some knowledge of C/C++ is required. To be frank, if you plan on doing anything worthwhile in CFD using LBM you will probably need to step away from Matlab at some point anyway.&lt;/p&gt;&#xA;&lt;p&gt;As for literature which contains some nice coding samples:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/kr/book/9783540279815&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lattice Boltzmann Modeling: An Introduction for Geoscientists and Engineers&lt;/a&gt; by Sukop &amp;amp; Thorne&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-0-85729-455-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lattice Boltzmann Method&#xA;Fundamentals and Engineering Applications with Computer Codes&lt;/a&gt; by Mohamad&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As for other literature which is a required reading to understand LBMs:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://global.oup.com/academic/product/the-lattice-boltzmann-equation-9780198503989?cc=nl&amp;amp;lang=en&amp;amp;&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Lattice Boltzmann Equation:&#xA;For Fluid Dynamics and Beyond&lt;/a&gt; by Succi&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.springer.com/la/book/9783540669739&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lattice-Gas Cellular Automata and Lattice Boltzmann Models: An Introduction&lt;/a&gt; by Wolf-Gladrow&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Some opensource codes to get you started:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.palabos.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Palabos&lt;/a&gt; - written in C++ for multi-CPU applications. Contains a cavity example in 2D and 3D.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sailfish.us.edu.pl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sailfish CFD&lt;/a&gt; - written in Python with a CUDA/OpenCL backend using the pyCuda/pyOpenCl API for multi-GPU applications. Also contains a cavity example in 2D. (my personal favorite)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18017" LastEditorUserId="20688" LastEditDate="2023-01-15T20:01:24.810" LastActivityDate="2023-01-15T20:01:24.810" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23266" PostTypeId="2" ParentId="21635" CreationDate="2016-03-03T20:11:53.590" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;after a short (say 10 ps) equilibration period&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You say yourself that the re-equilibration period is 'short'. Have you tried waiting longer to see if the two re-initialized systems &lt;a href=&quot;https://en.wikipedia.org/wiki/Transient_state&quot; rel=&quot;nofollow&quot;&gt;converge&lt;/a&gt;, and if so at what rate?&lt;/li&gt;&#xA;&lt;li&gt;Velocity-rescaling is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Flying_ice_cube&quot; rel=&quot;nofollow&quot;&gt;notoriously naive&lt;/a&gt; thermostat. Maybe you could replace it with something more realistic? (Berendsen, Nose-Hoover, etc.)&lt;/li&gt;&#xA;&lt;li&gt;If you're at all concerned about sampling a 'large' system, you can make the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ergodic_hypothesis&quot; rel=&quot;nofollow&quot;&gt;ergodic principle&lt;/a&gt; and parallel computing work for you: Sample &lt;em&gt;multiple&lt;/em&gt; re-initialized &lt;em&gt;realizations&lt;/em&gt; from the tail of your classical trajectory (taking snapshots of the classically equilibrated system). Then you can run multiple instances of your quantum code, each using a different re-initialization phase-space configuration, and average the outputs. Since the separate realizations evolve independently of each-other, the simulations are &lt;em&gt;embarrassingly parallel&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Sorry I can't offer any authoritative sources...&lt;/p&gt;&#xA;" OwnerUserId="7497" LastEditorUserId="7497" LastEditDate="2016-04-04T18:40:40.073" LastActivityDate="2016-04-04T18:40:40.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23267" PostTypeId="1" AcceptedAnswerId="23268" CreationDate="2016-03-03T20:40:33.640" Score="0" ViewCount="360" Body="&lt;p&gt;I searched the web and found some C code for calculating the determinant of a $n\times n$ matrix. This code however seems timing complexity, and run pretty slow especially when handling a larger matrix, for instance, $1000\times1000$ matrix. I need calculate the determinant of a covariance matrix of a multivariate normal distribution, I want to know is there any rapid way, or code for this purpose, thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="19482" LastEditorUserId="9667" LastEditDate="2016-03-03T20:56:38.350" LastActivityDate="2016-03-03T20:58:49.383" Title="Is there any rapid way to calculate the determinant of NXN covariance matrix?" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23268" PostTypeId="2" ParentId="23267" CreationDate="2016-03-03T20:58:49.383" Score="4" Body="&lt;p&gt;You cannot accurately compute the determinant of a large matrix in floating point arithmetic. Every algorithm I know of would be terrifyingly slow anyway, but the reason why people don't compute determinants for large matrices is not because of run time: it is because of accuracy issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reasons this is so because, in essence, when you want to compute the determinant, you need to compute the eigenvalues. There are many ways, the simplest being to just compute an LU decomposition of the matrix. But some of them may be small and some of them may be large. If you multiply together the eigenvalues, this does not create a problem if there are only a few of them. But if you have a large number of rather disparate numbers, all of which are only inaccurately known, you get into trouble because of cancellation effects unless the eigenvalues you have happen to be distributed rather specially distributed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why this is so, consider a $1000\times 1000$ matrix whose eigenvalues happen to all be around 10. The first problem is that the determinant would be $10^{1000}$, too large to be represented by double precision, even though each individual eigenvalue is moderately sized. Secondly, if you multiply many numbers, each with their own finite accuracy, you'll amplify these inaccuracies to the point where you don't have much accuracy left in the end result.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-03-03T20:58:49.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23269" PostTypeId="2" ParentId="23261" CreationDate="2016-03-03T21:01:19.833" Score="4" Body="&lt;p&gt;You really don't want to solve the heat equation with an explicit time stepping scheme. You need to choose the time step so incredibly small that you won't make any progress towards the end time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Explanation in lecture 27: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; .)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-03-03T21:01:19.833" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23270" PostTypeId="2" ParentId="20942" CreationDate="2016-03-04T05:33:59.553" Score="5" Body="&lt;p&gt;I'm a little late to the party, but the short answer is that yes it's possible to parallelize an interior point method for GPUs, but whether or not that is successful depends on the structure of the problem.  In terms of existing software, &lt;a href=&quot;http://www.optimojoe.com/products/optizelle/&quot; rel=&quot;noreferrer&quot;&gt;Optizelle&lt;/a&gt; can do it.  Grab the development branch until a new release occurs in the near future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situations differ slightly depending on whether or not the original problem contains equalities or inequalities.  There's a variety of ways to do this, but, in my opinion, the best way to do this for problems with only inequalities constraints is using an inexact trust-region method Newton method combined with an primal dual interior point method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For inequalities only, the basic inexact trust-region Newton method can be found in Nocedal and Wright's Numerical Optimization on page 171 or on Conn, Gould, and Toint's Trust-Region Methods on page 205.  This algorithm can be combined successfully with a primal-dual interior point method by essentially using the modified truncated-CG method from page 890 of the paper An Interior Point Method for Large-Scale Nonlinear Programming by Byrd, Hribar, and Nocedal.  Personally, I don't like how they setup their interior point system, so I wouldn't use their interior point formulation, but that's preference.  NITRO is a good algorithm.  As far as the interior point details, &lt;a href=&quot;http://www.optimojoe.com/products/optizelle/&quot; rel=&quot;noreferrer&quot;&gt;Optizelle&lt;/a&gt;'s manual explains how to do this in its manual.  I probably ought to post an updated manual, but the development branch is current.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case with both inequality and equality constraints, I believe the best algorithm is combining the inexact trust-region composite-step SQP method from Heinkenschoss and Ridzal in a paper titled A Matrix-Free Trust-Region SQP Method for Equality Constrained Optimization.  Basically, the process of tacking on an interior point method works pretty much the same as the unconstrained case except that the quasinormal step needs to be safeguarded as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the parallelization opportunities, the algorithms I reference above work well because these algorithms can be implemented matrix-free.  Specifically, &lt;a href=&quot;http://www.optimojoe.com/products/optizelle/&quot; rel=&quot;noreferrer&quot;&gt;Optizelle&lt;/a&gt;'s implementation for the problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min\limits_{x\in X}\{ f(x)  : g(x)=0, h(x)\geq 0\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Requires that the user provide an implementation for&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$f(x), \nabla f(x), \nabla^2 f(x)\partial x$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$g(x), g^\prime(x)\partial x, g^\prime(x)^*\partial y, (g^{\prime\prime}(x)\partial x)^* \partial y$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$h(x), h^\prime(x)\partial x, h^\prime(x)^*\partial y, (h^{\prime\prime}(x)\partial x)^* \partial y$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It doesn't care where these implementations come from or how their parallelized.  They can be done in shared memory, distributed memory, or GPUs.  It does not matter.  What works best for a particular problem, depends on the structure.  In addition, it requires the user to provide linear algebra for&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;init: Memory allocation and size setting&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;copy: y &amp;lt;- x (Shallow.  No memory allocation.)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;scal: x &amp;lt;- alpha * x&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;axpy: y &amp;lt;- alpha * x + y&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;innr: innr &amp;lt;- &amp;lt;x,y&amp;gt;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;zero: x &amp;lt;- 0&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;rand: x &amp;lt;- random&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;prod: Jordan product, z &amp;lt;- x o y&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;id: Identity element, x &amp;lt;- e such that x o e = x&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;linv: Jordan product inverse, z &amp;lt;- inv(L(x)) y where L(x) y = x o y&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;barr: Barrier function, barr &amp;lt;- barr(x) where x o grad barr(x) = e&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;srch: Line search, srch &amp;lt;- argmax {alpha \in Real &amp;gt;= 0 : alpha x + y &amp;gt;= 0} where y &amp;gt; 0&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;symm: Symmetrization, x &amp;lt;- symm(x) such that L(symm(x)) is a symmetric operator&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These operations can be done in serial, parallel, distributed memory, shared memory, or on GPUs.  It does not matter.  What's best depends on the problem structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, there's the linear systems and there are three that can be provided:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Preconditioner for $\nabla^2 f(x)$&lt;/li&gt;&#xA;&lt;li&gt;Left preconditioner for $g^\prime(x)g^\prime(x)^*$&lt;/li&gt;&#xA;&lt;li&gt;Right preconditioner for $g^\prime(x)g^\prime(x)^*$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Each of these preconditioners can be implemented in either serial or parallel, distributed memory or shared, or on GPUs.  Basically, the first preconditioner is the preconditioner for the truncated-CG system associated with the optimalitity systems.  The last two preconditioners are used for the augmented system solves associated with the composite step SQP algorithm.  In general, this is where you're going to get your biggest performance boost.  Imagine if the constraint $g$ represented some kind of PDE.  Then, the preconditioner for $g^\prime(x)g^\prime(x)^*$ corresponds to a forward PDE solve followed by an adjoint PDE solve.  Note, if they were square, $(g^\prime(x)g^\prime(x)^*)^{-1}=g^\prime(x)^{-*}g^\prime(x)^{-1}$.  For a huge number of PDE formulations, such as finite difference methods with explicit time integrators, these solves can be very well parallelized on a GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the algorithms in &lt;a href=&quot;http://www.optimojoe.com/products/optizelle/&quot; rel=&quot;noreferrer&quot;&gt;Optizelle&lt;/a&gt; do work on symmetric cone problems, which include bound, second-order cone, and semidefinite constraints.  Nevertheless, in general, the linear cone solves will tend to out perform it.  Basically, linear cone solves can reduce the feasibility and optimality solves done to a really compact system that can be Choleski factored.  Since &lt;a href=&quot;http://www.optimojoe.com/products/optizelle/&quot; rel=&quot;noreferrer&quot;&gt;Optizelle&lt;/a&gt; works with nonlinear systems, it can't really do that.  At least, I don't know how.  On top of that, there are restrictions on the size of the SDP blocks that &lt;a href=&quot;http://www.optimojoe.com/products/optizelle/&quot; rel=&quot;noreferrer&quot;&gt;Optizelle&lt;/a&gt; can handle.  The operator &lt;code&gt;linv&lt;/code&gt; above requires the inverse of the SDP matrices and that inverse is really expensive for large blocks.  In addition, there's an extra safe guard that requires a Choleski factorization.  These factorizations don't really parallelize well on a GPU.  At least, I don't know of an implementation that does parallelize well.  Anyway, the bottom line is that if it's a linear cone program, use a linear cone solver like CSDP or SDPT3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR; Use &lt;a href=&quot;http://www.optimojoe.com/products/optizelle/&quot; rel=&quot;noreferrer&quot;&gt;Optizelle&lt;/a&gt;.  It's free, open source, and BSD licensed.  I've scaled it to something like half a billion variables and it worked fine.  I've run it with GPUs and it worked fine.  Whether or not it works &lt;em&gt;well&lt;/em&gt; with a GPU depends on whether or not the operations above parallelize well on a GPU.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2016-03-04T05:33:59.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23271" PostTypeId="1" AcceptedAnswerId="23292" CreationDate="2016-03-04T05:52:05.657" Score="9" ViewCount="1556" Body="&lt;p&gt;I am reading a paper &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0021999175901151&quot; rel=&quot;noreferrer&quot;&gt;[1]&lt;/a&gt; where they solve the following non-linear equation &#xA;\begin{equation}&#xA;u_t + u_x + uu_x - u_{xxt} = 0&#xA;\end{equation}&#xA;using finite difference methods. They also analyse the stability of the schemes using the Von Neumann's stability analysis. However, as the authors realize, this is only applicable to linear PDE's. So the authors work around this by &quot;freezing&quot; the non-linear term, i.e. they replace the $uu_x$ term with $Uu_x$, where $U$ is &quot;considered to represent locally constant values of $u$.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is two-fold: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: how to interpret this method and why does it (not) work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2: could we also replace the $uu_x$ term with the $uU_x$ term, where $U_x$ is &quot;considered to represent locally constant values of $u_x$&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Eilbeck, J. C., and G. R. McGuire. &quot;Numerical study of the regularized long-wave equation I: numerical methods.&quot; Journal of Computational Physics 19.1 (1975): 43-57.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16650" LastEditorUserId="9667" LastEditDate="2016-03-04T19:40:14.930" LastActivityDate="2016-12-26T15:49:55.933" Title="What does the Von Neumann's stability analysis tell us about non-linear finite difference equations?" Tags="|pde|finite-difference|numerical-analysis|nonlinear-equations|stability|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23272" PostTypeId="1" AcceptedAnswerId="23275" CreationDate="2016-03-04T13:20:00.077" Score="1" ViewCount="495" Body="&lt;p&gt;I need to calculate the inverse of a dense matrix, with some elements taking values as high as 1e9 and some around 1e2. What would be the best method to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I am more concerned about the &lt;strong&gt;precision&lt;/strong&gt; of the answer rather than minimizing the computational time. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I explicitly need to find the &lt;code&gt;inverse&lt;/code&gt; and not the solution of any linear system &lt;code&gt;Au=b&lt;/code&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Size:&lt;/strong&gt; The size of the matrix is below $100 \times 100$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Formula&lt;/strong&gt; The formula for Pade Approximation of order 3,3 can be given as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left(120I_{n\times n}+60A+12A^2+A^3\right)\times\left(120I_{n\times n}-60A+12A^2-A^3\right)^{-1}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;A&lt;/code&gt; is the matrix whose exponential is to be found out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the Pade Approximation for order 3,3. I am trying to implement a Pade approximation of order 13,13 in which the coefficient of $I_{n\times n} \text{is}$ &lt;code&gt;17643225600&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Implementation:&lt;/strong&gt; I am implementing the Pade Approximation for finding the exponential of a matrix in which I need to find the inverse of the &quot;denominator&quot; matrix.&lt;/p&gt;&#xA;" OwnerUserId="18879" LastEditorUserId="18879" LastEditDate="2016-03-04T14:25:21.887" LastActivityDate="2016-03-04T15:22:48.397" Title="Calculate inverse of dense matrix with entries of very different magnitude" Tags="|linear-algebra|stability|inverse|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23273" PostTypeId="1" AcceptedAnswerId="23274" CreationDate="2016-03-04T13:46:04.053" Score="0" ViewCount="116" Body="&lt;p&gt;I am using FEM to do an assignment on a heat conduction problem on a complex domain, which needs me to get the variation of the temparature distribution subject to the variation of boundary conditions, and its exact solution is unknown. Also I have to show the correctness and convergence of my solutions numerically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am considering to compute the mean temperature over the whole domain, and then with the time variation of this mean value, I can get a curve denoting the evolvement of mean temparature. Then using different refinement levels, I can get a bounch of these curves. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to show its convergence, could I just put these curves on the paper with some description like this:&quot;Obviously, it can be seen from the figure that with the increasing refinement level, the mean temparature variation is converging.&quot; Or do I have to compute the mean differences of these curve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, I am not sure whether it's appropriate to use this mean temparature variation as a way to verify the correctness of my solution. Even though I can show the convergence, it is still not necessary to be correct because I don't have the exact solution. Also, even if the mean value is correct, I  still can't show the correctness on the whole domain. Maybe the values at some points are larger while smaller elsewhere. Who knows? (After all, no correct solution can be refered to.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone have some good suggestions?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastActivityDate="2016-03-04T14:18:36.640" Title="What should I put on the paper to show the correctness and convergence of my solution?" Tags="|numerical-analysis|convergence|error-estimation|verification|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23274" PostTypeId="2" ParentId="23273" CreationDate="2016-03-04T14:18:36.640" Score="4" Body="&lt;p&gt;You should definitely do a mesh refinement study (some conferences and journals require this). You should compare the results of this study to a known, analytical solution. You can either get this solution from the literature or generate one using the &lt;a href=&quot;http://infoserve.sandia.gov/sand_doc/2000/001444.pdf&quot;&gt;Method of Manufactured Solutions&lt;/a&gt; (PDF warning). You should demonstrate that your solution is converging at the expected rate as $h$ decreases as given by the &lt;em&gt;a priori&lt;/em&gt; error estimate for your problem if one exists, or you should compute the rate and state it if such an estimate does not (say in a hard nonlinear problem). There may be a nearby problem that gives a rate that you can compare to. If an estimate for the convergence rate of the average of the solution exists or you can derive it, then you can give a result based on that as well. Nevertheless, you need a known solution to compare to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes, when no analytic solution exists and an MMS one is hard to compute, people compute a solution on a so-called &quot;overkill&quot; mesh (i.e., as fine as you can manage to compute) and pretend like that one is the analytic one and compare to that. This approach is considered reasonable in some branches of PDE simulation, but better to have an answer at hand in my opinion. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-03-04T14:18:36.640" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23275" PostTypeId="2" ParentId="23272" CreationDate="2016-03-04T15:22:48.397" Score="3" Body="&lt;p&gt;Depending on the condition number of the matrix $A$, computing Padé approximations of the matrix exponential $e^A$ is horribly ill-conditioned, especially for such high orders. If a problem is ill-conditioned, no amount of mathematical trickery will allow a stable computation; however, often you can change the problem to one that is better conditioned (and still leads to the desired result). What to do depends on the precise nature of your assignment:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If the goal is an approximation of the matrix exponential, look at one of the better ways in Clive Moler and Charles van Loan's seminal paper &lt;a href=&quot;http://www.cs.cornell.edu/cv/ResearchPDF/19ways+.pdf&quot; rel=&quot;nofollow&quot;&gt;Nineteen dubious ways to compute the exponential of a matrix&lt;/a&gt;, SIAM Rev., 45(1), 3–49. (In particular, #2 on page 10ff discusses Padé approximations, and you should consider the scaling and squaring method #3.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If it has to be a Padé approximation, there are more robust ways to computing it via SVDs instead of direct inversion; see, e.g. Pedro Gonnet, Stefan Güttel, and Lloyd N Trefethen's paper &lt;a href=&quot;http://www.guettel.com/download/gonnet_guettel_trefethen.pdf&quot; rel=&quot;nofollow&quot;&gt;Robust Padé approximation via SVD&lt;/a&gt;, SIAM Rev., 55(1), 101–117. (This paper deals with Padé approximation of real functions, not matrix functions, so some further work is required.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Otherwise, for the matrix size and type you are looking at, and since you don't care so much about performance, I don't see a better way than using a direct solver (e.g., LU decomposition) from a  high-quality library to solve &#xA;for each unit vector $e_i$, $i=1,\dots N$ the linear system $B m_i = e_i$ and set $M=[m_1,\dots,m_N]$. Depending on the software framework, you should use the highest precision data type available to minimize round-off errors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-03-04T15:22:48.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23277" PostTypeId="2" ParentId="23261" CreationDate="2016-03-04T20:56:59.490" Score="3" Body="&lt;p&gt;While i agree with Wolfgang that its best to choose Crank-Nicolson time stepping, i think it is incorrect to assert explicit time stepping results in 'incredibly small' time steps or even to suggest your script is not functioning because of explicit time stepping. You can make it work just fine, but you need to get your discretization correct and to implement the boundary conditions properly.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Problems:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;First, your discretization of your spatial second derivative is incorrect. I like to do this using the Finite Volume method because you can not go wrong. You define an average temperature over a control volume $[x-\frac{1}{2}\Delta x,x+\frac{1}{2}\Delta x]$ where $x$ is the node at the center of the volume element with size $\Delta x$:&#xA;$$\bar{u} = \frac{1}{\Delta x}\int_{x-\frac{1}{2}\Delta x}^{x+\frac{1}{2}\Delta x} u dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you integrate the heat equation: &#xA;$$\partial_{t}u=a\partial_{x}^{2}u$$&#xA;over the control volume:&#xA;$$\frac{1}{\Delta x}\int_{x-\frac{1}{2}\Delta x}^{x+\frac{1}{2}\Delta x} \partial_{t}u dx=a\frac{1}{\Delta x}\int_{x-\frac{1}{2}\Delta x}^{x+\frac{1}{2}\Delta x} \partial_{x}^{2}u dx$$&#xA;$$\partial_{t}\bar{u} =\frac{a}{\Delta x}\left.\partial_{x}u \right|_{x-\frac{1}{2}\Delta x}^{x+\frac{1}{2}\Delta x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit: The result here is that the temperature in the control volume is spatially uniform, i.e. the average temperature $\bar{u}$ at node $x$. For $\Delta x$ small enough, this become increasingly more accurate. The change in time of $\bar{u}$ then becomes dependent of the incoming and outgoing flux of heat at the boundaries located at $x-\frac{1}{2}\Delta x$ and $x+\frac{1}{2}\Delta x$. The choice of these coordinates make this a staggered grid where the nodes do not align with the boundaries.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the second-order accurate discretization of the derivate, we get:&#xA;$$\partial_{t}\bar{u}&#xA;	=	\frac{a}{\Delta x}\left[\frac{\bar{u}\left(x+\Delta x\right)-\bar{u}\left(x\right)}{\Delta x}-\frac{\bar{u}\left(x\right)-\bar{u}\left(x-\Delta x\right)}{\Delta x}\right]&#xA;	=	\frac{a}{\Delta x^{2}}\left[\bar{u}\left(x+\Delta x\right)-2\bar{u}\left(x\right)+\bar{u}\left(x-\Delta x\right)\right]&#xA; $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is different from your discretization. &lt;em&gt;Edit: since the averaged temperature at node $x$ is used exclusively in these equations i will drop the 'bar' from $\bar{u}$&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit: For abritrary coordinate $\tilde{x}$, second-order accuracy is shown by Taylor expansion:&lt;/em&gt;&#xA;$$u\left(\tilde{x}-\frac{\Delta x}{2}\right)=u\left(\tilde{x}\right)-\frac{\Delta x}{2}\left.\frac{du}{dx}\right|_{\tilde{x}}+\frac{1}{2}\left(\frac{\Delta x}{2}\right)^{2}\left.\frac{d^{2}u}{dx^{2}}\right|_{\tilde{x}}+O\left(\Delta x\right)^{3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u\left(\tilde{x}+\frac{\Delta x}{2}\right)=u\left(\tilde{x}\right)+\frac{\Delta x}{2}\left.\frac{du}{dx}\right|_{\tilde{x}}+\frac{1}{2}\left(\frac{\Delta x}{2}\right)^{2}\left.\frac{d^{2}u}{dx^{2}}\right|_{\tilde{x}}+O\left(\Delta x\right)^{3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Subtracting to cancel the second derivative gives:&lt;/em&gt;&#xA;$$u\left(\tilde{x}+\frac{\Delta x}{2}\right)-u\left(\tilde{x}-\frac{\Delta x}{2}\right)=\Delta x\left.\frac{du}{dx}\right|_{\tilde{x}}+O\left(\Delta x\right)^{3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;rewriting to an expression for the first derivative:&lt;/em&gt;&#xA;$$\left.\frac{du}{dx}\right|_{\tilde{x}}=\frac{u\left(\tilde{x}+\frac{\Delta x}{2}\right)-u\left(\tilde{x}-\frac{\Delta x}{2}\right)}{\Delta x}+O\left(\Delta x\right)^{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;substituting in the averaged heat equation and filling in the $\tilde{x}=x-\frac{1}{2}\Delta x$ and $\tilde{x}=x+\frac{1}{2}\Delta x$ respectively gives the discretized equation found before.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, you need to define boundary conditions. Since you didn't specify i am going to assume you assume zero-gradient boundary conditions. Since $x$ is at the center of the node, you can view it as a staggered grid. This means that the boundaries of the domain are in between nodes at $x_{b0}=x_1-\frac{1}{2}\Delta x$ and $x_{bf}=x_N+\frac{1}{2}\Delta x$, where $x_{1}$ and $x_{N}$ are the first and $N$th node respectively. We can set the gradients at $x_{b0}$ and $x_{bf}$ by introducing 'ghost' nodes outside the domain at $x_0$ and $x_{N+1}$ and then the gradients are defined as:&#xA;$$\partial_{x}u\left(x_{b0}\right)=\frac{u\left(x_{1}\right)-u\left(x_{0}\right)}{\Delta x}=0\quad\partial_{x}u\left(x_{bf}\right)=\frac{u\left(x_{N+1}\right)-u\left(x_{N}\right)}{\Delta x}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So simply setting $u\left(x_{0}\right)=u\left(x_{1}\right)$ and $u\left(x_{N+1}\right)=u\left(x_{N}\right)$ will ensure zero-gradients at the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we are in a position to solve the equations. &lt;em&gt;Edit: For each node we solve the discretized heat equation and use the information at the boundaries for the first and last node. For simplicity each node is identified by a subscript $i$, i.e. $u_i=u\left(x_i\right)$:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_{t}\bar{u}_{i}=\frac{a}{\Delta x^{2}}\left[\bar{u}_{i+1}-2\bar{u}_{i}+\bar{u}_{i-1}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;written out explicitly:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_{t}\bar{u}_{1}	=	\frac{a}{\Delta x^{2}}\left[\bar{u}_{2}-2\bar{u}_{1}+\bar{u}_{0}\right]$$&#xA;$$\partial_{t}\bar{u}_{2}	=	\frac{a}{\Delta x^{2}}\left[\bar{u}_{3}-2\bar{u}_{2}+\bar{u}_{1}\right]$$&#xA;    $$\vdots	$$&#xA;$$\partial_{t}\bar{u}_{N-1}	=	\frac{a}{\Delta x^{2}}\left[\bar{u}_{N}-2\bar{u}_{N-1}+\bar{u}_{N}\right]$$&#xA;$$\partial_{t}\bar{u}_{N}	=	\frac{a}{\Delta x^{2}}\left[\bar{u}_{N+1}-2\bar{u}_{N}+\bar{u}_{N-1}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit: Here $u_0$ and $u_{N+1}$ are the values of the temperature at the ghost nodes previously determined from the boundary conditions. We substitute those in to get:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_{t}\bar{u}_{1}	=	\frac{a}{\Delta x^{2}}\left[\bar{u}_{2}-\bar{u}_{1}\right]$$&#xA;$$\partial_{t}\bar{u}_{N}	=	\frac{a}{\Delta x^{2}}\left[-\bar{u}_{N}+\bar{u}_{N-1}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, we define a system of equations with vector:&#xA;$$\vec{u}=\begin{bmatrix}u\left(x_{1}\right) &amp;amp; u\left(x_{2}\right) &amp;amp; \cdots &amp;amp; u\left(x_{N-1}\right) &amp;amp; u\left(x_{N}\right)\end{bmatrix}^{T}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;which allows us to write these equations as a system of equations:&lt;/em&gt;&#xA;$$\partial_{t}\begin{bmatrix}\bar{u}_{1}\\&#xA;\bar{u}_{2}\\&#xA;\vdots\\&#xA;\bar{u}_{N-1}\\&#xA;\bar{u}_{N}&#xA;\end{bmatrix}=\frac{a}{\Delta x^{2}}\begin{bmatrix}-1 &amp;amp; 1\\&#xA;1 &amp;amp; -2 &amp;amp; 1\\&#xA; &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots\\&#xA; &amp;amp;  &amp;amp; 1 &amp;amp; -2 &amp;amp; 1\\&#xA; &amp;amp;  &amp;amp;  &amp;amp; 1 &amp;amp; -1&#xA;\end{bmatrix}\begin{bmatrix}\bar{u}_{1}\\&#xA;\bar{u}_{2}\\&#xA;\vdots\\&#xA;\bar{u}_{N-1}\\&#xA;\bar{u}_{N}&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so we get the linear system:&#xA;$$\partial_t\vec{u}=\boldsymbol{A}\cdot\vec{u}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&#xA;$$A=\frac{a}{\Delta x^{2}}\left[\begin{array}{ccccc}&#xA;-1 &amp;amp; 1\\&#xA;1 &amp;amp; -2 &amp;amp; 1\\&#xA; &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots\\&#xA; &amp;amp;  &amp;amp; 1 &amp;amp; -2 &amp;amp; 1\\&#xA; &amp;amp;  &amp;amp;  &amp;amp; 1 &amp;amp; -1&#xA;\end{array}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All rows are described by our discretized equation, except in the first and last column where the values for the 'ghost' nodes outside of the domain are used to account for the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's not worry about discretization of the time derivative, we will simply let matlab integrate this system using an appropriate 'ode' function. This has as advantage that matlab choses an proper timestep for discretization which makes sure that the integration is stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below you will find an implementation of the above linear system and the result.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Code and result:&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function main()&#xA;    clc, clear all, close all;&#xA;&#xA;    a = 1;                     %% thermal diffusivity&#xA;    pt=100;                    %%% number of points&#xA;    xsta=-5;                    %%% x start&#xA;    xend=5;                     %%% x end&#xA;    x = linspace(xsta,xend,pt);   %%% creating x space -5 ~ 5&#xA;&#xA;    % create discretization matrix&#xA;    A = zeros(pt,pt);&#xA;    A(1,1) = -1; &#xA;    A(1,2) = 1;&#xA;    for i = 2:pt-1&#xA;        A(i,i-1) = 1;&#xA;        A(i,i) = -2;&#xA;        A(i,i+1) = 1;&#xA;    end&#xA;    A(pt,pt-1) = 1; &#xA;    A(pt,pt) = -1;&#xA;&#xA;    % initial condition&#xA;    u0 = zeros(pt,1);&#xA;    u0(abs(x)&amp;lt;1) = 1;&#xA;&#xA;    function dudt = pde(t, u, Fo_x)&#xA;        dudt = Fo_x*A*u;&#xA;    end&#xA;&#xA;    nt = 5; % increase nt if you want more time points (not needed for accuracy)&#xA;    Fo = 1; % Fo is a dimensionless diffusion coefficient, a*t/l^2&#xA;    Fo_x = Fo*(pt-1); % Fo_x is diffusion coefficient based on grid&#xA;    tspan = linspace(0,1,nt);&#xA;    [dimt,u] = ode45(@pde, tspan, u0, [], Fo_x);&#xA;&#xA;    figure, plot(x,u,'o')&#xA;    legendCell = cellstr(num2str((dimt*(xend-xsta)^2/a), 't=%-d s'));&#xA;    legend(legendCell)&#xA;    xlabel('spatial coordinate, x')&#xA;    ylabel('temperature, u')&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SLz8G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SLz8G.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18017" LastEditorUserId="18017" LastEditDate="2016-03-13T08:43:37.743" LastActivityDate="2016-03-13T08:43:37.743" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23278" PostTypeId="2" ParentId="23261" CreationDate="2016-03-04T21:59:52.257" Score="3" Body="&lt;p&gt;So you can solve this problem with an Explicit time stepping scheme, as &lt;strong&gt;nluigi&lt;/strong&gt; mentioned, but it comes at a cost just as &lt;strong&gt;Wolfgang&lt;/strong&gt; stated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the problem statement, the problem is formulated using an Explicit Euler time stepping scheme with (arguably) a second order central difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stability region of Explicit Euler can be found to be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta t \leq \frac{2}{|\lambda|_{max}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $|\lambda|_{max}$ is the largest modulus of the eigenvalues taken from the differential operator $A$ that allows the following discretization to be true:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial \textbf{u} }{\partial t} = A \textbf{u}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Second Order Central Difference scheme for space, and assuming Dirichlet boundary conditions, one can find the following expression for the largest eigenvalue modulus:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ |\lambda|_{max} = \frac{2 C}{\Delta x^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C \gt 1$ and $\Delta x$ represents the distance between points in the spacial discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you substitute this result into the stability result for Explicit Euler, you find:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Delta t \leq \frac{\Delta x^2}{C}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This result implies that as you shrink the distance between points, you must shrink the time step an order of magnitude more to ensure the solution remains stable. This is often unrealistic for higher precision simulations. This is the problem that &lt;strong&gt;Wolfgang&lt;/strong&gt; was mentioning and why you should consider another scheme. But as long as you stay within the stability region, you can integrate with an Explicit scheme just fine, as &lt;strong&gt;nluigi&lt;/strong&gt; mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To work around this, it's common to either use an implicit time stepping scheme or use an adaptive explicit scheme (which is what ode45 in Matlab uses).&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-03-04T22:06:04.537" LastActivityDate="2016-03-04T22:06:04.537" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23279" PostTypeId="1" CreationDate="2016-03-04T22:32:40.323" Score="12" ViewCount="3632" Body="&lt;p&gt;I was wondering what is a more optimized open source BLAS/LAPACK package with respect to modern multi-core processors (Haswell and beyond).  Is there any distribution that can attain performance close to that of Intel MKL for instance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to Intel, which libraries may perform well on AMD architectures for Zen gen1 and Zen gen2 ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping that the answer can be based on actual performance analysis work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&#xA;Michael&lt;/p&gt;&#xA;" OwnerUserId="19493" LastEditorUserId="24110" LastEditDate="2019-09-13T20:40:53.347" LastActivityDate="2019-09-13T20:40:53.347" Title="Optimized open source BLAS / LAPACK package" Tags="|optimization|lapack|blas|multicore|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="23284" PostTypeId="1" CreationDate="2016-03-05T14:54:56.940" Score="0" ViewCount="248" Body="&lt;p&gt;I have an equation of the form $J^Te=f$, where $e$ and $f$ are known vectors and $J$ is an unknown matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I efficiently compute $J^T(JJ^T)^{-1/2}e$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My motivation to address this problem is that, as a part of a bigger problem I need to compute matrix $J$ then compute $J^T(JJ^T)^{-1/2}e$. However, computation of matrix $J$ is expensive and $J$ results to be a large matrix, thereby the determination of product-transpose-squareroot-inverse of it is taking a lot of time. Hence, I'm planning to bypass the expensive computation of $J$ with the help some known vectors $e$ and $f$ and the relation $J^Te=f$.&lt;/p&gt;&#xA;" OwnerUserId="19499" LastEditorUserId="19499" LastEditDate="2016-03-05T21:28:31.340" LastActivityDate="2016-03-09T01:49:51.770" Title="Fast computation of square root inverse of matrix, matrix being determined from Ax=b form" Tags="|linear-algebra|matrix|inverse|matrix-free|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23285" PostTypeId="1" CreationDate="2016-03-05T15:18:48.330" Score="4" ViewCount="151" Body="&lt;p&gt;I have an interesting problem from my research that I have been struggling to solve. One part of the problem involves generating all possible matrices, where each set contains three integer vectors, of a given volume $V$. That is the determinant of the matrix should be $V$. The constraint is that the maximum of the absolute value of the integer components is bounded by $N$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is , suppose a $3 \times 3$ matrix $A$ with components $a_{ij}$, then &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$a_{ij} \in \mathbb{Z}$&lt;/li&gt;&#xA;&lt;li&gt;$\max (|a_{ij}|) \leq N$.&lt;/li&gt;&#xA;&lt;li&gt;$\det(A) = V$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have a MATLAB code that, I believe, scales as $O(N^8)$, and so, highly inefficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I am thinking about this is as a lattice of integer points. I need three vectors with components $(a_{11}, a_{12}, a_{13})$, $(a_{21}, a_{22}, a_{23})$, and $(a_{31}, a_{32}, a_{33})$. These components are all integers and will be the rows of the matrix $A$. The volume $V$ of the parallelepiped is the determinant of the matrix $V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I create $N_1 = (2N+1)^3$ number of lattice points. I choose two vectors, which will create $N_1 \choose 2$ combinations. And then I enumerate through the $N_1$ choices for the third vector and check if the determinant of the matrix is $V$. Any ideas to speed up this process? Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested, the reason I want to generate these matrices is listed as another question here - &lt;a href=&quot;https://math.stackexchange.com/questions/1683555/unique-sub-lattices-of-a-given-volume&quot;&gt;https://math.stackexchange.com/questions/1683555/unique-sub-lattices-of-a-given-volume&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1&lt;/strong&gt;: Explaining the final goal of the problem might help set this up better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the solution of all the possible matrices be denoted by the set $S_V =\{ A \}$. I am only interested in a finite subset of matrices $Q_{V} = \{Q \} \subset S_V$, such that &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For any two matrices $Q_1$ and $Q_2$ that belong to $Q_V$, $Q_1&#xA;    \cdot Q_2^{-1}$ is a non-integer matrix. That is any matrix in $Q_V$ cannot be expressed as an integer linear combination of any other matrix in $Q_V$ (They do not belong to the same lattices).&lt;/li&gt;&#xA;&lt;li&gt;For all matrices $A \in S_V$, there exists a matrix $Q \in Q_V$, such that $A \cdot Q^{-1}$ is an integer matrix. That is any matrices in $S_V$ can be expressed as an integer linear combination of a matrix in $Q_V$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This subset $Q_V$ ends up being pretty small. For example, for V = 3, there are only 13 matrices in the set $Q_V$ for any $N&amp;gt;3$.&lt;/p&gt;&#xA;" OwnerUserId="3215" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-04-06T04:21:56.340" Title="Efficient way to generate a list of possible matrices (all integer components) with a determinant $V$" Tags="|linear-algebra|optimization|matrix|efficiency|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23286" PostTypeId="1" CreationDate="2016-03-05T15:43:04.733" Score="10" ViewCount="1015" Body="&lt;p&gt;I need to compute a lot of $3\times3$ matrix inverses (for Newton iteration polar decomposition), with very small number of degenerate cases ($&amp;lt;0.1\%$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explicit inverse (via matrix minors divided by determinant) seems to work, and is about ~32~40 fused flops (depending on how I compute reciprocal of the determinant). Not considering the det scale factor, it's only 18 fused flops (each of the 9 elements is of the form ab-cd, 2 fused flops).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a way to compute inverse of $3\times 3$ using fewer than 18 (with arbitrary scale) or 32 (with proper scale, considering reciprocal 1 op) fused flops?&lt;/li&gt;&#xA;&lt;li&gt;Is there an economical way (using ~50 f-flops) to compute a backwards-stable left inverse of a $3\times 3$ matrix?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm using single-precision floats (iOS game). The backwards stability is interesting new concept for me and I want to experiment. Here's &lt;a href=&quot;http://arxiv.org/pdf/1201.6035v1.pdf&quot; rel=&quot;noreferrer&quot;&gt;the article&lt;/a&gt; that provoked the thought. &lt;/p&gt;&#xA;" OwnerUserId="7128" LastEditorUserId="10134" LastEditDate="2016-03-07T15:29:22.457" LastActivityDate="2018-04-03T10:40:26.467" Title="The fast, and The Backward-Stable (left) $3\times 3$ matrix inverse" Tags="|matrix|matrix-equations|inverse|matrix-factorization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23287" PostTypeId="1" CreationDate="2016-03-05T18:41:26.120" Score="2" ViewCount="2206" Body="&lt;p&gt;&lt;a href=&quot;https://mathoverflow.net/questions/232811/solving-linear-systems-with-ill-conditioned-matrices&quot;&gt;As per suggestions of the people from MathOverflow, I'm reposting my question here:&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently trying to solve a linear system $Ax = B$, where the matrix $A$ is ill conditioned (i.e. nearly singular), with a condition number of $~10^7$. The aforementioned linear system arises from a finite difference discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematical model for my problem is a PDE with derivatives of $x$ and $t$. Therefore, I'm solving a linear system of a discretized mesh of points with interval $\Delta x$ for each time step $\Delta t$. I'm already using centered differences for time, but because the problem has mixed derivatives (i.e. dependent of both $x$ and $t$), the problem still falls into a linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how can I find a solution to this linear system? The most common solution I saw is doing preconditioning on matrix $A$, so it gets better conditioned. But, due to my engineering background, I hardly understand what must be done to precondition matrix $A$. I saw many different methods (like Jacobi and ILU factorization), but I don't know how to apply them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a sidenote, I'm doing this on MATLAB, so if anyone know any in-built function that can help me, it would be very appreciated. I tried gmres but it didn't work (the solution still &quot;exploded&quot; after a few steps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Here is the equation as asked&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;EIr\left[\frac{\partial^4\theta}{\partial x^4}-6\left(\frac{\partial\theta}{\partial x}\right)^2\frac{\partial^2\theta}{\partial x^2}\right]-EAr\left[\frac{\partial u_x}{\partial x}\frac{\partial^2\theta}{\partial x^2}+\frac{\partial^2 u_x}{\partial x^2}\frac{\partial\theta}{\partial x}+1.5r^2\frac{\partial^2\theta}{\partial x^2}\left(\frac{\partial\theta}{\partial x}\right)^2\right]-I_pr\omega\left[2\frac{\partial^2\theta}{\partial x \partial t}\frac{\partial\theta}{\partial x}+\frac{\partial\theta}{\partial t}\frac{\partial^2\theta}{\partial x^2}\right]-m_pr\frac{\partial u_x}{\partial x}\frac{\partial^2\theta}{\partial t^2}+m_pr\frac{\partial^2 u_x}{\partial t^2}\frac{\partial\theta}{\partial x}-(x+u_x)m_pr\frac{\partial^3\theta}{\partial x \partial^2 t}+(x+u_x)m_pr\left(\frac{\partial\theta}{\partial t}\right)^2\frac{\partial\theta}{\partial x}+m_pg\sin\theta = 0&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="19505" LastEditorUserId="19505" LastEditDate="2016-03-06T19:35:23.173" LastActivityDate="2016-03-06T19:35:23.173" Title="Solving linear systems with ill-conditioned matrices" Tags="|matlab|linear-solver|preconditioning|condition-number|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23290" PostTypeId="2" ParentId="23258" CreationDate="2016-03-06T05:51:57.550" Score="4" Body="&lt;p&gt;Pretty sure that what you want is Non-negative least squares.  Plenty of implementations already exist, so you shouldn't have to write any code to get going.  &lt;a href=&quot;https://en.wikipedia.org/wiki/Non-negative_least_squares&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Non-negative_least_squares&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19509" LastActivityDate="2016-03-06T05:51:57.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23291" PostTypeId="2" ParentId="23287" CreationDate="2016-03-06T05:55:48.433" Score="1" Body="&lt;p&gt;The two simplest solutions to solving ill posed problems are 1- truncated SVD, and 2 - Tikhonov regularization, which are actually related to one another but implemented in different ways.  Plenty of details are available online about both of these methods; Wikipedia should be sufficient to get you going.  I would start with Tikhonov regularization, and then move to #1 if you want a fully automatic method. &lt;/p&gt;&#xA;" OwnerUserId="19509" LastActivityDate="2016-03-06T05:55:48.433" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23292" PostTypeId="2" ParentId="23271" CreationDate="2016-03-06T09:41:15.580" Score="1" Body="&lt;p&gt;What you are saying is referred to as linearization. It is a common technique used in the analysis of non-linear PDE's. What is done is to cast equations in the format,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_t+Au=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here A is a matrix resulting from the linearization of the equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to your questions,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;As you are thinking, it works to some extent, but does not to some other extent. The utility is that stability can be proven for linear systems but not readily for non-linear systems. So the linear results are extended to the non-linear systems. Often, different methods are adopted for particular cases. For example,&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$uu_x = \frac{1}{2}(u^2)_x$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is the conservation form. So,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_t+\frac{1}{2}(u^2)_x = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when represented in a finite volume sense gives limits on the evolution of u. &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;What is the utility of doing the replacement. You will remove the equation from a wave equation form. Which would mean that the solutions would not behave as a wave equation. So in the stability analysis, the test solutions would have to be completely different and un-physical as well. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2016-03-06T09:41:15.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23293" PostTypeId="1" AcceptedAnswerId="23923" CreationDate="2016-03-06T11:54:57.153" Score="0" ViewCount="137" Body="&lt;p&gt;Can you provide a hash functions $F(x) | F(x) = F(\overline{x}), x \in {\{0;1\}}^n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\overline{x} $ is $x$ where all bits are swapped: $0 \rightarrow 1, 1 \rightarrow 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically it will help me improve my MAX-CUT algorithm&lt;/p&gt;&#xA;" OwnerUserId="19513" LastActivityDate="2016-05-06T02:08:30.430" Title="Symmetric hash function" Tags="|optimization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23294" PostTypeId="2" ParentId="6863" CreationDate="2016-03-06T12:25:28.950" Score="5" Body="&lt;p&gt;One author is being practical, while the other is being rigorous.  The short answer is that &lt;strong&gt;stability of the frozen-coefficient problems is proven to guarantee stability of the variable-coefficient problem if the coefficients and solution are smooth enough.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know how much smoothness is needed, prepare to get your hands dirty.  The fundamental reference on the parabolic side is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Fritz, John.  &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/cpa.3160050203/abstract&quot; rel=&quot;noreferrer&quot;&gt;On integration of parabolic equations by difference methods: I. Linear and quasi-linear equations for the infinite interval.&lt;/a&gt; Communications on Pure and Applied Mathematics, 5(2):155--211 (1952).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For hyperbolic problems, see&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Strang, Gilbert.  &lt;a href=&quot;http://link.springer.com/article/10.1007%2FBF01386051?LI=true#page-1&quot; rel=&quot;noreferrer&quot;&gt;Accurate partial difference methods II&lt;/a&gt;.  Numerische Mathematik 6, 37-46 (1964).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One additional note: Ascher claims that his examples 5.1 and 5.6 demonstrate how schemes that are von Neumann stable in the smooth case can be unstable in the non-smooth case.  But both examples are misleading.  Example 5.1, involving the KdV equation, actually has a smooth solution for all time.  Meanwhile, Example 5.6 is still a linear constant-coefficient problem; only the solution is discontinuous.  The oscillations that appear there are due to numerical dispersion and not instability.  So these two examples do not demonstrate the pathologies that can arise.  Unfortunately, I don't know of one that does.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2016-03-06T12:38:54.110" LastActivityDate="2016-03-06T12:38:54.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23295" PostTypeId="1" AcceptedAnswerId="23299" CreationDate="2016-03-06T14:35:29.893" Score="1" ViewCount="659" Body="&lt;p&gt;I have a problem of the form,&#xA;\begin{align}&#xA;minimize_{y} maximize_{x}&amp;amp;\quad x^T y - y^T (B\odot x x^T) y\\&#xA; s.t. &amp;amp;x\in [l,u]\\&#xA;  &amp;amp;Ay=b   &#xA;\end{align}&#xA;How to efficiently solve this problem? $B$ is a semidefinite matrix.&#xA;What I have done: you can dual the inner problem, and get a semidefinite problem with two sdp variables. But this approach is too slow, and most of the solvers are unable to warmstart, which I need. I want a rapid method with warmstart ability, i.e. when I add a simple constraint, I can use the previous iteration results and not start from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:Unfortunately, I have some additional difficulties, $y$ is integer. Although I relax it to values between $[-1,1]$. If I use the semidefinite relaxation, for $y$, is there any efficient algorithm for this problem? In this way, the nonconvexity of function for a fixed $x$ which @Brian Borchers mentioned, will be resolved. This method, is good for my application, since, I can at least use warmstart for $x$. What do you suggest?&lt;/p&gt;&#xA;" OwnerUserId="8223" LastEditorUserId="8223" LastEditDate="2016-03-10T05:53:35.700" LastActivityDate="2016-03-10T08:50:53.830" Title="What method do you suggest to solve this minimax, quadratic in both variables problem?" Tags="|optimization|convex-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23296" PostTypeId="1" CreationDate="2016-03-06T15:02:58.350" Score="0" ViewCount="1229" Body="&lt;p&gt;I would like to incorporate a CFD topic in a project I currently have. The deadline is a little less than a month from now. Basically, what I would like to do is solve some equations and simulate fluid behavior. The programming language I will use to accomplish this in is C, however I intend to rely on MATLAB for later visualizations and better flexibility - after getting my solutions for the PDEs from my C code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently a &lt;strong&gt;beginner&lt;/strong&gt; in CFD and would like to boost my understanding by getting myself involved in such an interesting project - at least from my perspective!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any interesting yet &lt;strong&gt;simple&lt;/strong&gt; examples out there to help me accomplish this? I so far have not one idea in mind, especially since most of the topics circulating the web are kind of &lt;em&gt;'advanced'&lt;/em&gt; for someone in my situation as of yet. Any references, tutorials, ideas or the like would be greatly appreciated !! &lt;/p&gt;&#xA;" OwnerUserId="18387" LastActivityDate="2016-03-07T23:32:09.000" Title="Simple Simulation Examples in Computational Fluid Dynamics" Tags="|numerical-analysis|fluid-dynamics|simulation|computational-physics|numerical-modelling|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="23297" PostTypeId="2" ParentId="23293" CreationDate="2016-03-06T15:03:22.203" Score="0" Body="&lt;p&gt;I am not aware of an established symmetric hash function but it is easy to add this property to an existing function &lt;code&gt;hash&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our &lt;code&gt;my_hash&lt;/code&gt; arbitrarily decides that all data with a positive first bit is $x$ and with a negative first bit is $\overline{x}$. If the input &lt;code&gt;data&lt;/code&gt; is $x$ we return &lt;code&gt;hash(i)&lt;/code&gt; otherwise &lt;code&gt;bit_inverse(hash(i))&lt;/code&gt;. See pseudo code below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def my_hash(data)&#xA;    # arbitrary way to identify if data is already inverted&#xA;    if first bit of data is set&#xA;        return hash(data)&#xA;    else&#xA;        return bit_inverse(hash(data))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Before using this, make sure the negative effect on collisions and hash value distribution is not relevant to your problem.&lt;/p&gt;&#xA;" OwnerUserId="18622" LastEditorUserId="18622" LastEditDate="2016-03-06T15:22:44.170" LastActivityDate="2016-03-06T15:22:44.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23298" PostTypeId="2" ParentId="23293" CreationDate="2016-03-06T15:15:48.490" Score="1" Body="&lt;p&gt;The following hash function is symmetric. Note that the bit shifting is arbitrary and really you don't have to do it. The point is that you make sure the terms you use to create the hash are symmetric (in this case addition and multiplication) and that you use the key and its bit-flipped version to compute the hash index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In C/C++:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// symmetric hash function&#xA;unsigned char sym_hashfunc( unsigned char key ){&#xA;    unsigned int flip = ~key;&#xA;    unsigned int left = (key + flip);&#xA;    unsigned int right = (key * flip);&#xA;    return (left&amp;gt;&amp;gt;3) ^ (right&amp;gt;&amp;gt;1);&#xA;}&#xA;&#xA;int main(int argc, const char * argv[]) {&#xA;&#xA;    // some random input key&#xA;    unsigned int key = 12;&#xA;    unsigned int flipkey = ~key; // find bit flipped version&#xA;&#xA;    // print results. Should show same hash value&#xA;    printf(&quot;Hash1 = %u, Hash2 = %u\n&quot;,sym_hashfunc(key),&#xA;                                      sym_hashfunc(flipkey));&#xA;&#xA;    // exit&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-03-06T15:15:48.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23299" PostTypeId="2" ParentId="23295" CreationDate="2016-03-06T16:28:13.800" Score="4" Body="&lt;p&gt;You can write your inner problem as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\max_{x} x^{T}y - x^{T}Cx&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;x \in [l,u]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C=\mathrm{diag}(y)B\mathrm{diag}(y)$.  This is a box constrained QP.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $C=\mathrm{diag}(y) B \mathrm{diag}(y)$, $C$ is also positive semidefinite, so the inner problem is a concave maximization problem.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can easily solve the inner problem using a variety of methods (e.g. an active set method.)  You can also compute the sensitivity of the optimal value to changes in $y$.  With those sensitivities, you could then apply a first order method to solve the outer minimization problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a variety of algorithms for solving convex-concave minimax problems &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\min_{y} \max_{x} f(x,y)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$ is a function that is concave in $x$ for fixed $y$ and convex in $y$ for fixed $x$.  See for example chapter 3 in the book by Rustem and Howe, &lt;em&gt;Algorithms for Worst-Case Design and Applications to Risk Management&lt;/em&gt;.  These algorithms can work with just about any method for solving the inner problem as a subroutine.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, $f$ is not convex as a function of $y$ with $x$ fixed.  You also have additional complicating constraints.  You can still try this general approach, but the conventional analysis won't ensure convergence.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-03-08T01:33:28.227" LastActivityDate="2016-03-08T01:33:28.227" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23300" PostTypeId="2" ParentId="23296" CreationDate="2016-03-06T17:14:56.153" Score="4" Body="&lt;p&gt;There are many simple cases that you could look at, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2D-flow over a backward-facing step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lid-driven cavity flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Inviscid) wave resistance of thin ships or travelling pressure distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They should be easy to do in a month. There are also hidden depths (e.g. 3D-turbulence effects, non-linearity, etc) to the problems that could give years of interest, fascination and frustration.&lt;/p&gt;&#xA;" OwnerUserId="16571" LastActivityDate="2016-03-06T17:14:56.153" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23302" PostTypeId="1" CreationDate="2016-03-06T23:56:26.897" Score="-1" ViewCount="759" Body="&lt;p&gt;I'm trying to solve a nonlinear partial differential equation&#xA;\begin{equation}&#xA;L(u_{xxtt},u_{xx}u_{tt},u_{xt}^2,u_{xt},u_{tt})=0&#xA;\end{equation}&#xA;using finite difference methods. In order to remove the second derivatives with respect to time, I substitute&#xA;\begin{equation}&#xA;v := u_t&#xA;\end{equation}&#xA;such that it becomes&#xA;\begin{equation}&#xA;L(v_{xxt},u_{xx}v_{t},v_{x}^2,v_{x},v_{t}) = 0&#xA;\end{equation}&#xA;So in my algorithm, I first implicitly calculate $v$ at the next time step using central difference approximations (in order to ensure that local truncation error is second order both in time and space), i.e.&#xA;\begin{align}&#xA;(v_i{}^m)_{xx} &amp;amp; = (v^m_{i+1} - 2 v_i{}^m + v^m_{i-1})/\Delta x^2  \\&#xA;(v_i{}^m)_x &amp;amp; = (v^m_{i+1}-v^m_{i-1}) / 2\Delta x \\&#xA;(v_i{}^m)_t &amp;amp; = (v_i^{m+1} - v_i{}^{m-1})/2 \Delta t&#xA;\end{align}&#xA;Once I've calculated $v_i^{m+1}$, I calculate $u_i^{m+1}$ as follows&#xA;\begin{equation}&#xA;v_i{}^m = (u_i{}^m)_t = \frac{u^{m+1}_i - u_i^{m-1}}{2 \Delta t} \implies u^{m+1} = 2 \Delta t v_i{}^m + u_i^{m-1} \tag{1}&#xA;\end{equation}&#xA;I would like to determine the stability of this program, and &lt;strong&gt;any&lt;/strong&gt; tips on how to do that are greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I first wanted to determine the stability using von Neumann's method, but I don't think this is possible (but please correct me if I'm wrong). Let me explain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the answer given to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/23271/what-does-the-von-neumanns-stability-analysis-tell-us-about-non-linear-finite-d?noredirect=1#comment36598_23271&quot;&gt;my previous question&lt;/a&gt;, I've decided to &quot;freeze&quot; $u_{xx}$ and $v_x$ in order to linearize the PDE&#xA;\begin{equation}&#xA;L(v_{xxt},v_{x},v_{t}) = 0&#xA;\end{equation}&#xA;However, if I substitute equation $(1)$ into my finite difference operator, then I will get terms involving $u^{m+2}, u^{m+1}, u^m, u^{m-1}, u^{m-2}$. Thus, if I now substitute in the following (as per usual for the von Neumann stability analysis)&#xA;\begin{equation}&#xA;u_i{}^m = A^m \exp(\hat{i} k i \Delta x) \; , \; \hat{i}:= \sqrt{-1}&#xA;\end{equation}&#xA;then I will end up with a fifth order polynomial in $A$&#xA;\begin{equation}&#xA;aA^4 + bA^3 + cA^2 + dA + e = 0&#xA;\end{equation}&#xA;This means it will not be possible to solve for $A$ and thus I cannot determine the amplification factor. Hence, unless I've missed something, it's not possible to use the von Neumann analysis to determine stability of the scheme.&lt;/p&gt;&#xA;" OwnerUserId="16650" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-03-07T10:39:39.067" Title="Stability analysis for coupled nonlinear system of partial differential equations" Tags="|pde|finite-difference|numerical-analysis|nonlinear-equations|stability|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23303" PostTypeId="1" CreationDate="2016-03-06T19:21:52.470" Score="3" ViewCount="83" Body="&lt;p&gt;The problem I have may be really simple, but still getting a hard time solving it. So I have the Euler rotation equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I_{1}\dot{\omega}_{1}+\left(I_{3}-I_{2}\right)\omega_{2}\omega_{3}=\lambda_{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I_{2}\dot{\omega}_{2}+\left(I_{1}-I_{3}\right)\omega_{3}\omega_{1}=\lambda_{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I_{3}\dot{\omega}_{3}+\left(I_{2}-I_{1}\right)\omega_{1}\omega_{2}=\lambda_{3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $I_{i}$ are the moments of inertia about the principal axes of rotation and $\omega_{i}$ the time dependent angular velocities about each axis. In general $I_{i}\neq I_{j}, i\neq j$, or don't satisfy any situations in which they can be reduced to easier relations. $\lambda_{i}$ is a normal distributed random number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that these equations are non-linear and, in general, have no analytic solution. The question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\qquad$ Is there a good numerical integrator for these equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using C++ and have looked into the LAPACK package, but I'm kind of confused on how to use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the reference from Skowron and Gould (arXiv:1203.1034 [astro-ph.EP]); however I really don't know how to implement this algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If somebody could help me in finding an open source integrator or a reference where they talk about the implementation of this code, it would be great.&lt;/p&gt;&#xA;" OwnerDisplayName="user19519" LastActivityDate="2016-03-07T01:34:40.617" Title="Numerical Euler Rotation Equation" Tags="|computational-physics|software|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="23304" PostTypeId="2" ParentId="23285" CreationDate="2016-03-07T03:50:02.537" Score="3" Body="&lt;p&gt;I was able to get an answer on the math stackexchange site. You could find the answer &lt;a href=&quot;https://math.stackexchange.com/questions/1683555/unique-sub-lattices-of-a-given-volume&quot;&gt;here&lt;/a&gt;! It supposedly has to do with Hermite Normal Forms! Thank you, &lt;a href=&quot;https://math.stackexchange.com/users/117522/heterotic&quot;&gt;Heterotic&lt;/a&gt; for the answer.&lt;/p&gt;&#xA;" OwnerUserId="3215" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-03-07T03:50:02.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23306" PostTypeId="2" ParentId="23296" CreationDate="2016-03-07T09:08:01.430" Score="0" Body="&lt;p&gt;You can go through the following MOOC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about&quot; rel=&quot;nofollow&quot;&gt;http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going through the lessons and coding the examples and exercises will help you understand and build the tools you require. With that you can try out more substantial problems. &lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2016-03-07T09:08:01.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23307" PostTypeId="2" ParentId="23302" CreationDate="2016-03-07T10:39:39.067" Score="1" Body="&lt;p&gt;You can always try to find the roots numerically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in terms of alternate ways of analysis. If you are still referring to the same paper, then it shows that there is an analytic solution to the equation. What you can try to do is substitute this solution in your numerical scheme and see whether you can do something with it. &lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2016-03-07T10:39:39.067" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23309" PostTypeId="2" ParentId="23296" CreationDate="2016-03-07T23:32:09.000" Score="1" Body="&lt;p&gt;In my opinion, the simplest way to start is with some 1D compressible flow. e.g. Sod's shock tube problem. Then maybe extend that to 2D for visuals. Incompressible flow is &amp;quot;trickier&amp;quot; than compressible flow.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/253qX.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/253qX.gif&quot; alt=&quot;Density vs Time in Sod's shock tube&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16934" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-03-07T23:32:09.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23311" PostTypeId="1" AcceptedAnswerId="23312" CreationDate="2016-03-08T01:41:13.507" Score="1" ViewCount="1123" Body="&lt;p&gt;I am starting to dabble in MPI. I am fairly new to this area and I am currently reading the MPI standard. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to write my first MPI program, a simple hello world program, on a cluster that I have. I am not sure if anyone is familiar with the IBM bladecenter. Supposedly, there is a midplane that connects all of the blades together (if someone could confirm/deny this in their experience, that would be great). I was going to run a simple program using the send and receive C++ commands. I think that I will run the example program in the point-to-point communication section of the standard. I have typed up the program below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is this, say I have two computers (computer A and computer B) that are connected to each other in some way (take your pick but in this case, it would be the bladecenter so they are connected via a midplane). Through the MPI standard, how does computer A communicate with computer B through the connection? Also, if I run my program on computer A, will computer B be needing to run the same program? How will MPI know that the rank is on computer B and not on some other thread of computer A? Does MPI_Comm_rank get the rank for every single processor/core that it can see?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the example program that I am using, is that meant for 2 processes on the same computer or, would this work for 2 computers? If not, then how would I modify the program to work for 2 computers or n number of computers?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;mpi.h&quot;&#xA;main(int argc, char **argv)&#xA;{&#xA;     char message{20];&#xA;     int myRank;&#xA;     MPI_Status status;&#xA;     MPI_Init(&amp;amp;argc, &amp;amp;argv);&#xA;     MPI_Comm_rank(MPI_COMM_WORLD, &amp;amp;myRank);&#xA;     if(myRank == 0)&#xA;     {&#xA;          strcpy(message, &quot;Hello, there&quot;);&#xA;          MPI_SEND(message, strlen(message) + 1, MPI_CHAR, 1, 99, MPI_COMM_WORLD);&#xA;     }&#xA;     else if(myRank == 1)&#xA;     {&#xA;          MPI_Recv(message, 20, MPI_CHAR, 0, 99, MPI_COMM_WORLD, &amp;amp;status);&#xA;          printf(&quot;recieved: %s:\n&quot;, message);&#xA;     }&#xA;     MPI_Finalize();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I apologize if these questions are obvious. Again, this is my first time dealing with MPI and any help would be greatly appreciated, thank you&lt;/p&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2018-06-06T17:10:37.490" Title="How does MPI differentiate between two computers?" Tags="|parallel-computing|c++|mpi|hpc|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23312" PostTypeId="2" ParentId="23311" CreationDate="2016-03-08T01:56:47.833" Score="3" Body="&lt;p&gt;The standard is purposefully silent on these issues preferring to leave such issue to library implementers to figure out. There's no need to standardize such mechanisms. Most network hardware has a unique (enough) identifier (MAC address or other GUID), and most operating systems associate a hostname with each node. These together are enough for MPI libraries to assign tasks to ranks in a consistent way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't need to know where the ranks end up in most cases. The MPI standard doesn't want you to worry about it. The mechanics inside an MPI program are generally independent of the arrangement, though the library itself has to keep track to send messages to the right place behind the scenes. The standard is designed so that you don't have to worry about it for correctness, though there may be some performance differences.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-03-08T01:56:47.833" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23313" PostTypeId="2" ParentId="23284" CreationDate="2016-03-08T02:16:23.113" Score="1" Body="&lt;p&gt;The problem is wildly underdetermined. Of the infinitely many equations for $J$ of the equation $J^T e=f$ is the (rank-one) matrix&#xA;$$&#xA;  J^T = \frac{f e^T}{\|e\|^2}.&#xA;$$&#xA;For this, you have that&#xA;$$&#xA; JJ^T = \frac{e f^Tf e^T}{\|e\|^4} = \|f\|^2 \frac{e e^T}{\|e\|^4}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, this matrix satisfies&#xA;$$&#xA;  (JJ^T)(JJ^T) = \|f\|^4 \frac{e e^T}{\|e\|^4} \frac{e e^T}{\|e\|^4}&#xA;  = \|f\|^4 \frac{e e^T e e^T}{\|e\|^8}&#xA;  = \|f\|^4 \frac{e e^T}{\|e\|^6} = \frac{\|f\|^2}{\|e\|^2} JJ^T.&#xA;$$&#xA;This means that&#xA;$$&#xA;  (JJ^T)^{1/2}&#xA;  = \frac{\|e\|}{\|f\|} JJ^T.&#xA;$$&#xA;This allows you to compute the quantity you are interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: To see how the last line follows, take the square of the left and right hand sides (i.e., multiply each side with itself) to obtain&#xA;$$&#xA;  JJ^T&#xA;  = \frac{\|e\|^2}{\|f\|^2} (JJ^T) (JJ^T).&#xA;$$&#xA;This is, up to division by the factor on both sides, the same as before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how to proceed then: We know that $A := JJ^T = c ee^T$ where $c=\|f\|^2/\|e\|^4$. In other words, if you multiply $A$ by $e$, you get a multiple of $c\|e\|^2 e$ from it. Conversely, this means that applying $A^{-1}$ to this direction $e$, you get one over this multiple, i.e., $A^{-1}e=\frac{1}{c\|e\|^2}e$. (This inverse does not strictly exist -- I'm just taking the pseudo-inverse $A^\dagger$ when I write $A^{-1}$). This brings us back to the equation&#xA;$$&#xA;  (JJ^T)^{-1/2}&#xA;  = \frac{\|f\|}{\|e\|} A^{-1}&#xA;$$&#xA;and consequently&#xA;$$&#xA;  (JJ^T)^{-1/2}e&#xA;  = \frac{\|f\|}{\|e\|} A^{-1}e&#xA;  = \frac{\|f\|}{c\|e\|^2} e&#xA;  = \frac{\|f\|\|e\|^4}{\|f\|^2\|e\|^2} e&#xA;  = \frac{\|e\|^2}{\|f\|} e.&#xA;$$&#xA;With this, you get&#xA;$$&#xA;  J^T(JJ^T)^{-1/2}e&#xA;  = \frac{1}{\|f\|} fe^Te&#xA;  = \frac{\|e\|^2}{\|f\|} f.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this said, this is just one choice for $J$. Other choices may yield different results.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-03-09T01:49:51.770" LastActivityDate="2016-03-09T01:49:51.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23314" PostTypeId="1" CreationDate="2016-03-08T08:35:26.097" Score="5" ViewCount="266" Body="&lt;p&gt;In N queens problem &lt;a href=&quot;https://en.wikipedia.org/wiki/Eight_queens_puzzle&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Eight_queens_puzzle&lt;/a&gt;, trying to find solution by backtracking encounters difficulties quite fast (even for SWI-Prolog, &lt;a href=&quot;http://swish.swi-prolog.org/example/clpfd_queens.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://swish.swi-prolog.org/example/clpfd_queens.pl&lt;/a&gt;, which solves N=100 in 0.2 sec, entering N=150 in the link above did not reach a solution in a reasonable amount of time (&quot;time limit exceeded&quot;), and as for ECLiPSe (another Prolog with constraint poropagation) it chokes much, much earlier..).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, local search works incredibly well on this problem (&lt;a href=&quot;https://stackoverflow.com/questions/1863531/n-queens-probiem-how-far-can-we-go&quot;&gt;https://stackoverflow.com/questions/1863531/n-queens-probiem-how-far-can-we-go&lt;/a&gt;, answer by the user chesslover), as the solutions seem to be reachable easily enough by hill climbing (or rather, in this case, hill descending with occasional short range climbing), starting from a random position in the space of possible configurations (each column is occupied by exactly one queen).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which other well known problems appear to have similar properties (say, a part from examples given in &lt;a href=&quot;https://en.wikipedia.org/wiki/Local_search_(optimization)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Local_search_(optimization)&lt;/a&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="18877" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2018-05-03T00:04:36.277" Title="Striking examples of success of local search algorithms" Tags="|optimization|algorithms|combinatorics|heuristics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23315" PostTypeId="1" AcceptedAnswerId="23326" CreationDate="2016-03-08T10:57:08.783" Score="3" ViewCount="289" Body="&lt;p&gt;I do a lot of general processing of data on my desktop computer. It can be stuff like extracting timeseries of point measurements from a large number of NetCDF files, or converting csv-files into binary or making a lot of plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, would it be useful to have a Xeon Phi for this kind of work? I try to use trivial parallelisation on my 8-core CPU whenever I can, using things like xargs to run several copies of my python scripts. Would I be able to use a Xeon Phi in the same way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some experience using OpenMP and MPI on Xeon Phi, so I have an idea of how that works, but I never thought of trying it in a more interactive manner, and I only had access to one for a limited time, so I can't easily check this myself.&lt;/p&gt;&#xA;" OwnerUserId="19544" LastActivityDate="2016-03-09T08:49:35.270" Title="How useful is a Xeon Phi for a general workstation" Tags="|parallel-computing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23316" PostTypeId="1" CreationDate="2016-03-08T11:59:10.520" Score="1" ViewCount="101" Body="&lt;p&gt;Ok so I'm a complete beginner in computational modelling (I use analytical methods of physics typically) but I would like to model an anisotropic, aperiodic (but not random) finite array of metallic nanorods, all of which have their long axis orientated in the same direction as eachother, but this direction can be completely arbitrary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am interested in is the charge oscillations in each nanorod (localised surface plasmons) which can electromagnetically couple to neighbouring nanorods and generate a collective charge density wave (plasmon/polariton) which can in principle propagate through the system. For example I could impose a periodic driving of the charge density in one nanorod at a particular frequency and see how it decomposes into the different allowed momentum states of the system and propagates. I want to have a simulation where I can see the propagation of these waves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that because of the aperiodicity I can't use periodic boundary conditions. I believe this means that a full 3D electromagnetic simulation is too computationally heavy (I have a good spec iMac but nothing better). So I assume I have to do a 2D model, but I have no idea if this makes any sense for my system and how to implement. These nanorods in principle can have any orientation in 3-dimensions and the EM field is intrinsically 3D so how can I map this problem to 2D. How reliable is this approach for modelling the reality?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm hoping for some suggestions on what software to use and how to get started and perhaps some insight into this 3D to 2D mapping problem. I have access to COMSOl but happy to try other (free) software.&lt;/p&gt;&#xA;" OwnerUserId="18258" LastActivityDate="2016-03-08T11:59:10.520" Title="Representing a 3D system in 2D (Electromagnetic modelling)" Tags="|finite-element|finite-difference|computational-physics|comsol|electromagnetism|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23320" PostTypeId="2" ParentId="338" CreationDate="2016-03-08T17:51:13.123" Score="6" Body="&lt;p&gt;As part of the lab the developed and maintains StochKit.  I am happy to hear that it is highly recommended in the previous answers.  However, I wanted to update everyone.  There already is a python wrapper for StochKit: &lt;a href=&quot;https://github.com/gillespy/gillespy&quot; rel=&quot;noreferrer&quot;&gt;GillesPy&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also recommend you checkout &lt;a href=&quot;http://www.stochss.org&quot; rel=&quot;noreferrer&quot;&gt;StochSS&lt;/a&gt;, This is an fully functional modeling and simulation IDE that uses StochKit and as one of it's simulations engines.  It also includes spatial simulation with the python package PyURDME.  Model building, and 3D visualization tools (plus cloud computing integration).&lt;/p&gt;&#xA;" OwnerUserId="19553" LastEditorUserId="19553" LastEditDate="2016-03-20T22:38:48.173" LastActivityDate="2016-03-20T22:38:48.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23321" PostTypeId="2" ParentId="6791" CreationDate="2016-03-08T19:34:26.200" Score="0" Body="&lt;p&gt;The above concern with a result of positive or negative 1e-17 as the first eigenvector value (which suggests a rounding error on essentially a 0 value could render your scaled eigenvector invalid) is not impacted by the ARPACK eigsh routine (Python, or eigs in MATLAB) - it consistently reproduces identical eigenvectors only differing by +/-1 scalar.  If you use another method which may introduce rounding errors, I would suggest increasing precision to avoid that issue if it presents itself.&lt;/p&gt;&#xA;" OwnerUserId="19418" LastActivityDate="2016-03-08T19:34:26.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23322" PostTypeId="1" AcceptedAnswerId="23324" CreationDate="2016-03-08T20:30:37.780" Score="0" ViewCount="2679" Body="&lt;p&gt;I have 2 pandas datasets which have 'RF' field in common.  The &lt;code&gt;optionsdata&lt;/code&gt; variable is ~60K, which map to at most the &lt;code&gt;histdata&lt;/code&gt; fields of 5K.  Using &lt;code&gt;np.where&lt;/code&gt; is incredibly slow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for j in range(0,len(optionsdata)):&#xA;    optionsmap[j] = np.where(histdata.ix['RF'].str.match(optionsdata.RF[j]))[0][0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a much faster way to do this?  All I need is the row # for each value in &lt;code&gt;optionsdata&lt;/code&gt; which corresponds to the RF row in &lt;code&gt;histdata&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should note the field compared is a string such as 'NYMEX_01_MAR_2016'&lt;/p&gt;&#xA;" OwnerUserId="19418" LastEditorUserId="19418" LastEditDate="2016-03-09T03:48:22.390" LastActivityDate="2016-03-09T08:12:10.063" Title="Python Pandas / Numpy indexing faster than np.where between 2 arrays" Tags="|python|" AnswerCount="1" CommentCount="3" ClosedDate="2016-03-12T19:20:30.497" ContentLicense="CC BY-SA 3.0" />
  <row Id="23323" PostTypeId="1" AcceptedAnswerId="23330" CreationDate="2016-03-09T06:42:32.997" Score="7" ViewCount="1445" Body="&lt;p&gt;Suppose we use a preconditioned iterative solver for a linear system. If the initial state for the solver can be chosen very close to the exact solution - does this reduce requirements for the preconditioner?&lt;/p&gt;&#xA;" OwnerUserId="4325" LastEditorUserId="1804" LastEditDate="2016-03-09T08:55:56.320" LastActivityDate="2018-03-28T06:55:19.777" Title="role of initial guess for iterative linear solver" Tags="|linear-solver|preconditioning|krylov-method|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23324" PostTypeId="2" ParentId="23322" CreationDate="2016-03-09T08:12:10.063" Score="1" Body="&lt;p&gt;It is not 100% clear to me what your dataframes look like, but I believe you could do this with the &lt;a href=&quot;http://pandas.pydata.org/pandas-docs/stable/merging.html#database-style-dataframe-joining-merging&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;merge&lt;/code&gt;&lt;/a&gt; function of pandas. Your code would be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;optionsmap = pandas.merge(optionsdata,histdata,on=&quot;RF&quot;,how=&quot;left&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2016-03-09T08:12:10.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23326" PostTypeId="2" ParentId="23315" CreationDate="2016-03-09T08:49:35.270" Score="3" Body="&lt;p&gt;In short: No. Sadly the Xeon Phi does not work like a 60 core cpu, but more like a GPU including its own dedicated memory.&lt;br&gt;&#xA;This means that all software has to be written to exploit the card, through things like OpenCL, Intels OpenMP, TBB, etc.&#xA;The advantage is that for applications already written to use OpenMP or OpenCL, recompiling with the intel libraries should be enough to start using the Xeon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your case, that is unlikely to be relevant however. There is nothing like xargs for the Xeon Phi. Additionally, the problems you describe could easily be bound by the speed of the harddrive, rather than CPU. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2016-03-09T08:49:35.270" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23327" PostTypeId="1" CreationDate="2016-03-09T10:11:41.920" Score="2" ViewCount="249" Body="&lt;p&gt;I just got a new Macbook and decided to install everything from scratch instead of migrating. Previously I've always installed SciPy etc. through MacPorts, but lately MacPorts as a whole seems not to be as cleanly maintained as it once was, and I spend a lot of time tracking down problems with it. I also generally used MacPorts (rather than pip or easy_install) to install Python libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was wondering if there is a better way to install SciPy and friends these days. I know that there are a number of all-in-one packages such as Anaconda, but I don't know if that's better or worse than using MacPorts to install the lot. I also know that Homebrew exists as a more recent alternative to MacPorts, but in searching for comparisons between the two I haven't gleaned much useful information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is whether it is generally recommended to install Anaconda or another similar package, or to use MacPorts or Homebrew, or some other solution. I realise this question could be a little opinion-based, so I'd appreciate it if answers would stick to relevant facts about the various options, as they relate to maintaining a scientific Python distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if I do use MacPorts or Homebrew, should I generally reach for the package manager when I want to install a new library, or is it better to just install the basic packages that way and use pip for everything else?&lt;/p&gt;&#xA;" OwnerUserId="3746" LastActivityDate="2016-06-08T18:17:42.123" Title="What is the currently recommended way to install the SciPy ecosystem on OS X?" Tags="|python|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23328" PostTypeId="2" ParentId="23327" CreationDate="2016-03-09T10:58:16.943" Score="0" Body="&lt;p&gt;I rather suspect that the answers to this will be almost entirely opinion based, with the right answer depending on what you already have (or need) on your system but in terms of mostly verifiable facts I claim:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Writing your own Homebrew recipes involves using Ruby, which looks better on a cv and is slightly more useful these days than tcl as used by Macports. This only matters when the available packages don't support what you want to do, but for heavy scientific computation this is depressingly common.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Macports is still a good deal stricter about building its entire tool chain rather than using mac supplied tools. This is most likely to matter to you if you're compiling against python, when getting the flags right for the build process gets a bit more insane, but the final result can be made more gcc friendly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Macports had significantly more packages the last time I checked, although the implementation of the variants system can mean that packages from different maintainers don't play nicely together, particularly when openmpi and compiler choices are important.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;pip plus virtualenv makes for nice sandbox environments when packages don't play nicely together (I'm thinking in particular of various tools under the aegis of the FEniCS project). I've been told that pip uninstall doesn't always do what you expect, but I haven't ever run into that myself.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I mostly use macports and haven't dared to upgrade to El Capitan yet. &lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-03-09T10:58:16.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23330" PostTypeId="2" ParentId="23323" CreationDate="2016-03-09T13:01:58.223" Score="12" Body="&lt;p&gt;Practical experience shows that trying to get good initial iterates has little value. For example, in the context of solving partial differential equations, if you take the solution from one mesh, interpolate it onto a finer mesh, and use that as the starting guess for something like a CG iteration to solve the same problem on the finer mesh, then it turns out that you will only need a negligible number of iterations less than if you had started the iteration with a zero vector. The gain is not worth the code or the time to do the interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation is very different, however, if you have a nonlinear problem, or if you used a fixed point iteration rather than something like CG.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-03-09T13:01:58.223" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23335" PostTypeId="1" CreationDate="2016-03-09T22:02:45.543" Score="3" ViewCount="6286" Body="&lt;p&gt;Hi I have a code below that solves non linear coupled PDE's given Dirichlet boundary conditions.  However I need to implement periodic boundary conditions.    The periodic boundary conditions are troubling me, what should I add into my code to enforce periodic boundary conditions?  Thanks for the help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, $t\geq 0$ and $x\in [0,1]$. Here are the coupled equations, below that I provide my code&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\partial^2_tu(x,t) +\partial_t u(x,t) -a u(x,t)-\partial^2_x u(x,t)+b u(u^2+f^2)=0\\&#xA;\partial^2_tf(x,t) +\partial_t f(x,t) -a f(x,t)-\partial^2_x f(x,t)+bf(u^2+f^2)=0\\&#xA;u(x,0)=f(x,0)=z(x),\ &#xA;\partial_t u(x,0)=\partial_t f(x,0)=0&#xA;\end{eqnarray}&#xA;where $a,b&amp;gt;0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those are the initial conditions, but now I need to impose periodic boundary conditions.  These can be mathematically written as $u(0,t)=u(1,t)$ and $\partial_x u(0,t)=\partial_x u(1,t)$, the same holds for $f(x,t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program coupledPDE&#xA;&#xA;integer, parameter :: n = 10, A = 20 !total number of position and time steps, respectively&#xA;real, parameter :: h = 0.1, k = 0.05 !step size for position and time respectively , x goes from 0 to 1, t goes from 0 to 1 also.&#xA;real, dimension(0:n) :: u,v,w,f,g,d !these variables store my solutions.&#xA;integer:: i,m !loop indices for space and time respectively&#xA;real:: t,R,x,c1,c2,c3,eta,a,b  &#xA;&#xA;R=(k/h)**2.&#xA;a=1.0&#xA;b=1.0&#xA;c1=(2.+a*k**2.-2.0*R)/(1+k/2.)&#xA;c2=R/(1.+k/2.)&#xA;c3=(1.0-k/2.)/(1.0+k/2.)&#xA;c4=b*k**2./(1+k/2.)&#xA;&#xA;&#xA;u=0.0 !Boundary conditions, need to make periodic..&#xA;f=0.0 !boundary conditions, but not correct.&#xA;&#xA;do i = 0,n+1 &#xA;  x = real(i)*h    &#xA;  w(i) = z(x)  !u(x,0)&#xA;  d(i) = z(x)  !f(x,0)&#xA;end do&#xA;&#xA;do i=1,n !discretization for 1st time step only, as usual for hyperbolic PDE's&#xA;  v(i) = (c1/(1.+c3))*w(i) + (c2/(1.+c3))*(w(i+1)+w(i-1)) -(c4/(1.+c3))*w(i)*((w(i))**2.+(d(i))**2.) !\partial_t u(x,0)=0&#xA;  g(i) = (c1/(1.+c3))*d(i) + (c2/(1.+c3))*(d(i+1)+d(i-1)) -(c4/(1.+c3))*d(i)*((w(i))**2.+(d(i))**2.) !\partial_t f(x,0)=0&#xA;end do&#xA;&#xA;&#xA;do m=1,A &#xA;&#xA;   do i=1,n-1 !Discretization equation for all times after the 1st step&#xA;       u(i)=c1*v(i)+c2*(v(i+1)+v(i-1))-c3*w(i)-c4*v(i)*((v(i))**2.+(g(i))**2.) &#xA;       f(i)=c1*g(i)+c2*(g(i+1)+g(i-1))-c3*d(i)-c4*g(i)*((v(i))**2.+(g(i))**2.) &#xA;   end do &#xA;   write(*,*)'check if u-f=0', u-f !this should be zero&#xA;   print*, &quot;the values of f(x,t+k) for all m=&quot;,m&#xA;   print &quot;(//3x,i5,//(3(3x,e22.14)))&quot;,m,f,u&#xA;&#xA;end do&#xA;&#xA;end program coupledPDE&#xA;&#xA;function z(x)&#xA;real, intent(in) :: x&#xA;real :: pi&#xA;pi=4.0*atan(1.0)&#xA;z = sin(pi*x)&#xA;end function z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks for reading, I am new to here, so if I should reformat my question in a more proper way please let me know.  &lt;/p&gt;&#xA;" OwnerUserId="19570" LastEditorUserId="123" LastEditDate="2016-03-10T04:47:08.913" LastActivityDate="2016-03-10T08:47:10.403" Title="How do I program periodic boundary conditions?" Tags="|pde|finite-difference|numerical-analysis|boundary-conditions|fortran|" AnswerCount="1" CommentCount="1" ClosedDate="2016-03-14T22:21:27.860" ContentLicense="CC BY-SA 3.0" />
  <row Id="23336" PostTypeId="2" ParentId="23335" CreationDate="2016-03-10T08:47:10.403" Score="6" Body="&lt;p&gt;Typically, you would add &quot;guard cells&quot;, that is (for u) u(-1) and u(n+1) with your notation. Before each integration step:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u(n+1) = u(0)&#xA;u(-1) = u(n)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and similarly for the other variables. If you use higher order derivatives, you could also define u(-2) and u(n+2), etc.&lt;/p&gt;&#xA;" OwnerUserId="16340" LastActivityDate="2016-03-10T08:47:10.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23337" PostTypeId="2" ParentId="23295" CreationDate="2016-03-10T08:50:53.830" Score="3" Body="&lt;p&gt;With integer variables, the whole problem can be cast as a MILP after some tricks from integer and quadratic programming. Perhaps not what you want to do in your case, but at least this is a guaranteed global approach&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Multiplication between binary and continuous can be written using linear constraints and auxiliary variables.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Polynomials of binaries can be written using linear constraints and auxilliary variables.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Bounded integer variables can be written using binary variables.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The optimal solution to the inner QP can be written using KKT conditions, involving products between $y$ and $x$, and complementarity.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Complementarity constraints can be handled using additional binary variables and constraints.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;First normalize the notation and assume the inner program is to minimize $c^T(y)x + \frac{1}{2}x^TQ(y)x$, subject to constraints $Ex\leq f$. The optimal solution is defined by the KKT conditions &#xA;$$Q(y)x + c(y) + E^T\lambda=0\\ \lambda^T(f - Ex) = 0\\\lambda\geq 0\\f-Ex\geq 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any point satisfying the optimality conditions, the objective is equal to $\frac{1}{2}(c(y)^Tx-f^T\lambda)$.In the outer program where we wish to minimize the objective, we thus want to minimize $-\frac{1}{2}(c(y)^Tx-f^T\lambda)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We note that we have an equality involving a quadratic function of $y$ multiplied with $x$ (thus linearizable), a complementarity condition (linearizable) and a bilinear term in the objective (linearizable). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quite a bit of different things to put together. In the code below, I will use the MATLAB Toolbox YALMIP (disclaimer: developed by me) to test this. It has built-in functionality for the linearizations, and MILP-modelling of the complementarity.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Define a random problem&#xA;n = 4;&#xA;B = randn(n);&#xA;B = B*B';&#xA;A = ones(1,n);&#xA;b = 1;&#xA;U = rand(n,1);&#xA;L = -rand(n,1);&#xA;E = [eye(n);-eye(n)];&#xA;f = [U;-L];&#xA;&#xA;% Define decision variables&#xA;x = sdpvar(n,1);&#xA;% y is modelled as a selection from possible values&#xA;PossibleY = [-1;0;1];&#xA;Selector = binvar(n,length(PossibleY),'full');&#xA;y = Selector*PossibleY;&#xA;ConstrainSelection = sum(Selector,2) == 1&#xA;&#xA;% Inner problem min_x c(y)'*x + .5*x'*Q(y)*x&#xA;lambda = sdpvar(length(f),1);&#xA;c = -y;&#xA;Q = 2*(B.*(y*y'));&#xA;&#xA;Stationary = Q*x+c+E'*lambda == 0;&#xA;Complementarity = complements(lambda&amp;gt;=0, f - E*x&amp;gt;=0);&#xA;Objective = 0.5*(c'*x - f'*lambda);&#xA;&#xA;% Objective we actually maximized in inner and whish o minimize in&#xA;% the outer problem&#xA;OuterObjective = -Objective;&#xA;% Linearize binary*binary and binary*continuous &#xA;LinearizedStationary = binmodel([Stationary,[L&amp;lt;=x&amp;lt;=U]]);&#xA;[LinearizedObjective,Cuts] =  binmodel(OuterObjective,[L&amp;lt;=x&amp;lt;=U]);&#xA;% Solve the problem&#xA;optimize([LinearizedStationary, &#xA;          Complementarity,&#xA;          Cuts,&#xA;          A*y == b,ConstrainSelection],LinearizedObjective)&#xA;% Check that the linearization actually worked&#xA;[value(LinearizedObjective) -value(c'*x + .5*x'*Q*x)]&#xA;&#xA;% Small sanity check. Check by solving everything for fixed x, keeping&#xA;% kkt conditions, to ensure the new y still renders x optimal.&#xA;Objective = value(x)'*y - y'*(B.*(value(x*x')))*y;&#xA;[LinearizedObjective,Cuts] =  binmodel(Objective);&#xA;LinearizedStationary = binmodel([Q*value(x)+c+E'*lambda == 0]);&#xA;optimize([Cuts,A*y == b,lambda&amp;gt;=0, lambda'*(f - E*value(x))==0,LinearizedStationary],LinearizedObjective)&#xA;value(Objective)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2016-03-10T08:50:53.830" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23338" PostTypeId="1" CreationDate="2016-03-10T09:28:52.123" Score="1" ViewCount="82" Body="&lt;p&gt;I have the following two PDEs which I want to check for stability:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t= u_{xx} , \ u(x,0)=1 , \ u_x(1,t)=-hu^4(1,t) , \ u_x(0,t) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t = u_{xx}-\sin(x+t)+\cos(x+t) , \ u(x,0)=\cos(x) ,\ u_x(1,t)=-hu^4(1,t)-\sin(1+t)+h\cos^4(1+t) , \ u_x(0,t)=-\sin(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first I got that $d/dt (u,u) = d/dt \int_0^1 u^2 dx = 2[-hu^5(1,t)-\int_0^1 u_x^2dx$, but how to estimate $u^5(1,t)$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just before I forget, h is some parameter greater than zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="17431" LastActivityDate="2016-03-10T09:28:52.123" Title="Stability of two PDEs" Tags="|pde|parabolic-pde|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23339" PostTypeId="1" CreationDate="2016-03-10T09:59:13.037" Score="0" ViewCount="387" Body="&lt;p&gt;Consider the following eigenvalue problem&#xA;\begin{equation}&#xA;\mathcal {L} x(s) = \lambda x(s),&#xA;\end{equation}&#xA;where &#xA;\begin{equation}&#xA;\mathcal {L} = \alpha \partial^4_s + (s^2-1)\partial^2_s + s \partial_s + 1,&#xA;\end{equation}&#xA;with $\alpha = \rm{const}$ and $s\in[0,1]$. In order to find the eigenvalues and the eigenfunctions, the operator $\mathcal L$ has to be translated into a matrix form. Suppose to use finite differences, what is the matrix that satisfies the boundary conditions $u(0) = u_s(0)=0$ and $u_{ss}(1)=u_{sss}(1)=0$? &lt;/p&gt;&#xA;" OwnerUserId="18560" LastEditorUserId="14772" LastEditDate="2016-03-12T13:11:21.230" LastActivityDate="2016-03-12T13:11:21.230" Title="Boundary conditions generalized eigenvalue problem" Tags="|finite-difference|boundary-conditions|eigenvalues|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23340" PostTypeId="1" AcceptedAnswerId="23366" CreationDate="2016-03-10T12:30:13.987" Score="3" ViewCount="2602" Body="&lt;p&gt;What are the state of the art methods for fast 2D convolution approximation?&lt;br&gt;&#xA;I'm familiar with SVD based multiplication and &lt;a href=&quot;http://arxiv.org/abs/1402.5649&quot; rel=&quot;nofollow&quot;&gt;cross approximation&lt;/a&gt; approaches, but would be thankful to get additional references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kernel size is typically ~3x3 - 11x11, not separable, with input matrices from sizes ~200x200 - 1000x1000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="10226" LastEditorUserId="10226" LastEditDate="2016-03-14T07:48:35.460" LastActivityDate="2016-04-09T16:43:33.187" Title="Methods for fast approximation of convolution" Tags="|matrix|approximation|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="23341" PostTypeId="2" ParentId="23339" CreationDate="2016-03-10T14:26:43.857" Score="1" Body="&lt;p&gt;With finite difference methods the typical philosophy for imposing boundary conditions on derivatives is to introduce enough extra &quot;ghost&quot; points, outside the computational domain, such that total number of unknowns matches the total number of equations, including the boundary conditions. For example to implement a zero  Neumann condition at $s=0$ we add a point $u(-\Delta x)$ and an equation $u(-\Delta s) = u(\Delta s)$. This could either be done explicitly with an expanded solution vector and another row in the matrix, or by substituting into the other discrete equations so that&#xA;$\partial_{ss} u (0) \approx \frac{u(\Delta s) -2u(0)+u(-\Delta s)}{2\Delta s^2} = \frac{u(\Delta s)-u(0)}{\Delta s^2}$.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-03-10T14:26:43.857" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23343" PostTypeId="1" AcceptedAnswerId="23349" CreationDate="2016-03-10T23:25:09.120" Score="11" ViewCount="2806" Body="&lt;p&gt;There are many methods for diagonalizing matrices, probably the most widely used is the combination of Householder transformations and the QR algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any superior method for diagonalizing (large, non-sparse)real symmetric matrices? Superiority can be a bit muddy, so I define it as fast, numerically stable, does not require large amounts of extra memory and lends itself to parallelization and vectorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Meta: I have been a bit torn about the correct place to ask this, another candidate would have been the Mathemathics exchange, they also have a numerical linear algebra tag.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="10264" LastEditorUserId="10264" LastEditDate="2016-03-11T01:01:45.260" LastActivityDate="2023-10-21T07:07:20.293" Title="What is the state of the art algorithm for diagonalizing real symmetric matrices?" Tags="|linear-algebra|algorithms|parallel-computing|eigenvalues|performance|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23344" PostTypeId="1" AcceptedAnswerId="23350" CreationDate="2016-03-11T19:06:32.740" Score="3" ViewCount="1491" Body="&lt;p&gt;It is a commonly known fact that the Rayleigh Quotient converges cubically (&lt;a href=&quot;http://www.ams.org/journals/mcom/1974-28-127/S0025-5718-1974-0405823-3/S0025-5718-1974-0405823-3.pdf&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt;), while the Power Iteration may converge slowly if the difference between the dominant and second-dominant eigenvalue is small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing this, when would it ever be beneficial to use the Power Iteration over the Rayleigh Quotient Iteration? The only possible upside that I can see with the Power Iteration is that it consists of a matrix-vector multiplication $O(n^2)$, while the Rayleigh Quotient iteration requires the solving a system of equations $O(n^{3})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a specific size threshold that needs to be reached before the Power Iteration becomes superior? What about for matrices with a specific structure?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Parlett, Beresford N. &quot;The Rayleigh quotient iteration and some generalizations for nonnormal matrices.&quot; Mathematics of Computation 28.127 (1974): 679-693.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19588" LastEditorUserId="19588" LastEditDate="2016-03-14T12:20:44.347" LastActivityDate="2016-03-14T12:20:44.347" Title="Power Iteration over Rayleigh Quotient Iteration?" Tags="|linear-algebra|eigenvalues|iterative-method|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23345" PostTypeId="1" CreationDate="2016-03-12T02:17:06.853" Score="8" ViewCount="252" Body="&lt;p&gt;If there is an $(n+1)$-dimensional integral of the form&#xA;$$ \int_{[0,1]^{n+1}} f(x, y)\,\mathrm{d}^n x \,\mathrm{d}y,$$&#xA;normally one would evaluate this using a multi-dimensional integration library over the whole domain, $[0,1]^{n+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But are there some conditions where it might make sense to perform the integral over $y$ separately, using a one-dimensional quadrature, and then use the multi-dimensional integration library to evaluate the integrand over the other $n$ coordinates?&#xA;$$ \int_{[0,1]^n}g(x)\,\mathrm{d}^nx, \qquad g(x) = \int_0^1 f(x,y)\,\mathrm{d}y. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might make sense, for example, if $f$ is especially smooth as a function of $y$, but not $x$. But how smooth exactly does it have to be in this case?&#xA;My guess was that it almost never makes sense because far too many of the 1-d quadrature evaluation points would be &quot;wasted&quot;, but I'm not so sure this always applies. Is this guaranteed by the design of the high-dimensional integration methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my own case, $f$ is black-box, but piecewise-smooth in $y$, and has an unknown amount of kinks and jumps in $x$ at unknown locations, and $n$ is quite high ($n\geq 4$), so the integral over $x$ has to be done with something specifically for many dimensions. The integral over $y$ can be done with something regular like &lt;code&gt;quadgk&lt;/code&gt;. In this example, the function is smooth enough in $y$ that it almost seems to work, but repeated integration ends up being 30 times slower, so I am wondering if the approach is misguided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know where this is already discussed in the literature, that would be helpful too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example.&lt;/strong&gt; &lt;em&gt;(of why this is not trivial)&lt;/em&gt; Consider an &quot;easy&quot; integral, which is very smooth, unlike what I'm really interested in:&#xA;$$ \int_{[0,1]^n} e^{-x_1 x_2\cdots x_n}\,\mathrm{d}^n x = \mathrm{F}\left(\begin{array}{}\{1,\ldots,1\}_n\\\{2,\ldots,2\}_n\end{array}\middle| -1\right). $$&#xA;We could do naive $n$-dimensional Monte Carlo on the integrand, or naive $(n-1)$-dimensional Monte Carlo on the integrand integrated once w.r.t. $x_1$, which is $g(x_{2:n}) = -(e^{-a}-1)/a$ (where $a=x_2\cdots x_n$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With some algebra, I calculated that the variance of $(n=5)$-dimensional $N$-point MC estimate is $0.00244N^{-1}$, and it is $0.00167N^{-1}$ for the $4$-dimensional integral of $g$, for a variance reduction by a factor of $1.5$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a &lt;em&gt;paltry&lt;/em&gt; reduction in variance: it would be negated by using $1.5$ times as many samples points, and this is offset by the fact that the internal integrand might be more than $1.5$ times slower to evaluate. If the function $g = (1-e^{-a})/a$ above happens to be more than $1.5$ times slower, this represents a &lt;em&gt;net loss&lt;/em&gt; of accuracy, keeping computation time fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presumably the same kind of tradeoff applies when considering a deterministic rule for integrating over $x$. The Monte Carlo method makes this analysis much easier than the general case, because integrating over $y$ acts like a very straightforward variance reduction technique. &lt;strong&gt;But I'm really much more interested in deterministic methods, which I haven't been able to analyze as easily.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-03-15T21:06:50.090" LastActivityDate="2016-03-15T21:06:50.090" Title="When is it advantageous to iterate integrals numerically?" Tags="|numerical-analysis|reference-request|quadrature|integration|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23346" PostTypeId="1" CreationDate="2016-03-12T10:44:20.330" Score="1" ViewCount="1604" Body="&lt;p&gt;In CVX, I encounter a problem. I want to multiply a Matrix of 2x4 with its transpose. I know the result must be positive definite. However, it couldn't let me do the multiplication directly. Says: Disciplined convex programming error:Only scalar quadratic forms can be specified in CVX. What can I do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cvx_begin&#xA;    variable power_allocation(length(anchor_coordinate.x),1)&#xA;    minimize sum(power_allocation)&#xA;    subject to&#xA;    Matrix = process_matrix_inv'*(observation_matrix(:,:,t*n+1)'*diag(power_allocation.*path_loss(:,t*n+1))*&#xA;observation_matrix(:,:,t*n+1)+prior_infor(:,:,t*n))*process_matrix_inv;&#xA;    A = Matrix(1:2,3:6)*D*Matrix(1:2,3:6)';&#xA;    trace_inv(Matrix(1:2,1:2)-A) &amp;lt;= MSE_limit;&#xA;    power_allocation &amp;lt;= power_max;&#xA;    power_allocation &amp;gt;= power_min;&#xA;    cvx_end&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="19590" LastEditorUserId="302" LastEditDate="2016-03-12T17:11:54.560" LastActivityDate="2016-03-12T17:11:54.560" Title="Matrix transpose multiplication" Tags="|matrix|cvx|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23348" PostTypeId="2" ParentId="23346" CreationDate="2016-03-12T15:35:16.303" Score="4" Body="&lt;p&gt;In CVX, you can use &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;quad_form(A*x-b,Q)&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;to implement &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;(Ax-b)^{T}Q(Ax-b)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the section of the manual on scalar quadratic forms.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-03-12T15:35:16.303" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23349" PostTypeId="2" ParentId="23343" CreationDate="2016-03-12T23:40:26.373" Score="11" Body="&lt;p&gt;The QR/Francis algorithm is the go-to choice for dense eigenproblems, but there are a few competitors around:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Jacobi_eigenvalue_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Jacobi algorithm&lt;/a&gt; (like QR, another algorithm with an unfortunate name, which can be confused with a method to solve linear systems...). &lt;strong&gt;Main idea:&lt;/strong&gt; choose a nondiagonal element of $A_k$; apply a suitable 2x2 Givens rotations $A_{k+1}=Q_kA_kQ_k^*$ to zero it out; then repeat. In each new iteration the entries that you have zeroed out at the previous steps become nonzero again, so the matrix is always full; nevertheless, one can prove that the off-diagonal entries converge to zero under some assumptions on how selected them in the different step. &lt;strong&gt;Literature pointer&lt;/strong&gt; studying stability: Demmel, Veselic 1992, &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.139.8311&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Jacobi's method is more accurate than QR&lt;/em&gt;&lt;/a&gt;. Essentially, they show that when the eigenvalues span several orders of magnitude, this method gets better accuracy for the smaller ones. Note that between 1992 and now there has been another major improvement to the QR algorithm (the MRRR method by Parlett and Dhillon); I am not expert enough in the area to tell if this has bridged the gap.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Spectral divide-and-conquer algorithms based on doubling iterations (which are &lt;em&gt;not&lt;/em&gt; the classical &lt;a href=&quot;https://en.wikipedia.org/wiki/Divide-and-conquer_eigenvalue_algorithm&quot; rel=&quot;noreferrer&quot;&gt;divide and conquer algorithms&lt;/a&gt; mentioned in a comment). These algorithms rely on BLAS level-3 operations only, so (given an implementation of products, inversions, projections etc., which pays proper attention to communication and caching issues) they should scale better to matrices of dimension larger than a few thousands.&#xA;&lt;strong&gt;Main idea&lt;/strong&gt;: apply an iteration such as Newton for the matrix sign $M_0=A$, $M_{k+1}=\frac12 (M_k + M_k^{-1})$ under which the eigenvectors are preserved and each eigenvalue converges to either $-1$ or $1$; then compute kernels of $(\lim M_k)\pm I$ to determine two complementary invariant subspaces of $A$; project $A$ onto these two subspaces, and repeat for the two smaller problems. &lt;strong&gt;Literature pointer&lt;/strong&gt; studying stability: Nakatsukasa and Higham, 2012, &lt;a href=&quot;http://eprints.ma.man.ac.uk/1824/&quot; rel=&quot;noreferrer&quot;&gt;Stable and Efficient Spectral Divide and Conquer Algorithms for the Symmetric Eigenvalue Decomposition and the SVD&lt;/a&gt;. They construct a variant of the iteration which requires no matrix inverses and converges extremely fast, and prove the stability of the resulting method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Both methods can be faster / more accurate than QR under certain conditions; there is research interest in the numerical linear algebra community, but they are less mature than QR, which has been the main horse in the race for decades, used by virtually everyone in numerical computing. I would not recommend their use instead of QR to a practitioner in a different field who just want to get the job done; but if you are doing research on eigenvalue methods or you have a problem for which QR encounters trouble you may want to experiment with them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for implementations, LAPACK includes a Jacobi routine to compute the SVD, &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d1/d5e/dgesvj_8f_source.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;*GESVJ&lt;/code&gt;&lt;/a&gt;, and a Matlab implementation of QDWH (the algorithm described in the Nakatsukasa and Higham's paper) is on &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/36830-symmetric-eigenvalue-decomposition-and-the-svd&quot; rel=&quot;noreferrer&quot;&gt;Matlab file exchange&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2016-03-13T08:25:25.257" LastActivityDate="2016-03-13T08:25:25.257" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23350" PostTypeId="2" ParentId="23344" CreationDate="2016-03-13T05:34:16.150" Score="5" Body="&lt;p&gt;Power and Rayleigh iterations are fundamentally different beasts, because the former uses only matrix-vector products with $A$, while the latter uses matrix-vector products with $(A+zI)^{-1}$. This is a very important point -- neither algorithms require explicit access to the matrix itself, but only certain actions of the matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The choice to chose one over the other is heavily weighted by the application-specific complexity of each operation, which does not always scale according to the figures you have quoted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the case where $A$ is actually dense, but matrix-vector products can be efficiently computed. For example, gravity is a &lt;em&gt;dense&lt;/em&gt; interaction, meaning that every body interacts with every other body. If the strengths of the interactions between $n$ bodies are tabulated as the $n\times n$ matrix $A$, then it will contain $n^2$ nonzero elements. For very large $n$, these $n^2$ elements might not even fit in memory. However, the matrix-vector product $Ax$ can be computed in just $O(n)$ complexity using the fast multipole method, &lt;em&gt;without ever having to form the matrix&lt;/em&gt;. Accordingly, it is possible to compute $\lambda_\max(A)$ using power iterations, even as $n$ scales into the billions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, when $A$ is a sparse matrix, there often exist a reordering of the columns of the matrix such that $(A+zI)^{-1}x$ can be computed in near-linear or linear complexity. After an initial symbolic factorization step, only $\approx O(n)$ work is required to compute each Rayleigh iteration. In this case, there is very little reason to even consider power iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I should note that power iterations is a suboptimal Krylov subspace method. Wherever power iterations can be used, the Lanczos iterations can do better. In fact, using exact arithmetic, Lanczos is guaranteed to converge after $n$ iterations.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2016-03-13T22:39:16.083" LastActivityDate="2016-03-13T22:39:16.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23352" PostTypeId="1" CreationDate="2016-03-13T12:33:02.947" Score="5" ViewCount="938" Body="&lt;p&gt;I am working on implementing the Fourier beam propagation method in C++.  I am really more of a programmer than a physicist but I think I have a good understanding of what I am trying to do.  Here is what I am trying to implement.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NwsJf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NwsJf.png&quot; alt=&quot;Beam Propagation algorithm flow chart&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Two questions.  In the flow chart he says you should perform the diffraction and after that do the refraction.  From the equation however it looks like you apply the refraction first, do the FFT and then apply the diffraction.  Also in the formula for diffraction I notice that the has kz as a function of omega, kx, and another kx.  Is that a typo where he should have ky instead of another kx?&lt;/p&gt;&#xA;&lt;p&gt;I am including my code below. I am trying to create the diffraction pattern through a circular aperture. I am not really sure what is an appropriate size for the cell grids. Should it be on the order of a wavelength? I am just getting blank output and most in the inverse transform are NaN.  I think the issue lies with how I calculate &lt;span class=&quot;math-container&quot;&gt;$k_z$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$k_x$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$k_y$&lt;/span&gt;, see comments in code.  Any help would be appreciated in getting this working.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;complex&amp;gt;        &#xA;#include &amp;lt;cmath&amp;gt;        &#xA;#include &amp;lt;iostream&amp;gt;        &#xA;#include &amp;lt;stddef.h&amp;gt;        &#xA;#include &amp;quot;fourier.hpp&amp;quot;        &#xA;#include &amp;lt;omp.h&amp;gt;        &#xA;#include &amp;lt;fstream&amp;gt;        &#xA;#include &amp;lt;fftw3.h&amp;gt;        &#xA;#include &amp;lt;cstring&amp;gt;            &#xA;        &#xA;const int N = 1000;        &#xA;        &#xA;void save(const char* fn, std::complex&amp;lt;double&amp;gt;* g)        &#xA;{        &#xA;        std::ofstream out(fn);    &#xA;        out &amp;lt;&amp;lt; &amp;quot;P5 &amp;quot; &amp;lt;&amp;lt; N &amp;lt;&amp;lt; &amp;quot; &amp;quot; &amp;lt;&amp;lt; N &amp;lt;&amp;lt; &amp;quot; 255 &amp;quot;;    &#xA;            &#xA;        double max = 0.0;    &#xA;        for(int i = 0; i &amp;lt; N * N; i++)    &#xA;        {    &#xA;                max = fmax(std::abs(g[i]), max);    &#xA;        }    &#xA;            &#xA;        for(int i = 0; i &amp;lt; N * N; i++)    &#xA;        {    &#xA;                unsigned char c = 255.0 * std::abs(g[i]) / max;    &#xA;                out.write((const char*)&amp;amp;c, 1);    &#xA;        }    &#xA;            &#xA;        out.close();    &#xA;}        &#xA;        &#xA;int main()        &#xA;{           &#xA;        typedef std::complex&amp;lt;double&amp;gt; complex;    &#xA;        const complex I(0.0, 1.0);    &#xA;        complex* e_prime = new std::complex&amp;lt;double&amp;gt;[N * N];&#xA;        complex* e = new std::complex&amp;lt;double&amp;gt;[N * N];&#xA;        &#xA;        memset(e, 0, sizeof(std::complex&amp;lt;double&amp;gt;) * N * N);&#xA;        &#xA;        const double lambda = 500.0e-9;&#xA;        const double dz = 1.0e-8; //forward step size&#xA;        const double n = 1.0; //index of retraction (just a constant 1)&#xA;        const double k0 = 2.0 * M_PI / lambda;&#xA;        const double cell_size = lambda / 4.0; //I want the grid to have a 1/4 wavelength resolution&#xA;        &#xA;        &#xA;        //create a circle in the electric field grid&#xA;        for(int y = 0; y &amp;lt; N; y++)&#xA;        {&#xA;                for(int x = 0; x &amp;lt; N; x++)&#xA;                {&#xA;                        int dx = x - N / 2;&#xA;                        int dy = y - N / 2;&#xA;                        &#xA;                        if(dx * dx + dy * dy &amp;lt; 1000)&#xA;                                e[y * N + x] = 1.0;&#xA;                        else&#xA;                                e[y * N + x] = 0.0;&#xA;                                &#xA;                }&#xA;        }&#xA;        &#xA;        save(&amp;quot;slit.ppm&amp;quot;, e);&#xA;        &#xA;        //apply the refraction step&#xA;        for(int y = 0; y &amp;lt; N; y++)&#xA;        {&#xA;                for(int x = 0; x &amp;lt; N; x++)&#xA;                {&#xA;                        e[y * N + x] *= std::exp(-I * n * dz * k0);&#xA;                }&#xA;        }&#xA;        &#xA;        //Perform the fourier transform of the refracted electric field&#xA;        fftw_plan p = fftw_plan_dft_2d(N, N, (fftw_complex*)e, (fftw_complex*)e_prime, FFTW_FORWARD, FFTW_ESTIMATE | FFTW_NO_SIMD);&#xA;        fftw_execute(p);&#xA;        fftw_destroy_plan(p);&#xA;        &#xA;        //save the fourier transform for reference&#xA;        save(&amp;quot;ft.ppm&amp;quot;, e_prime);&#xA;        &#xA;        //apply the diffraction step&#xA;        for(int y = 0; y &amp;lt; N; y++)&#xA;        {&#xA;                for(int x = 0; x &amp;lt; N; x++)&#xA;                {&#xA;                        /*&#xA;                         * This is where I am a bit confused.  So far I have not&#xA;                         * found where in the math to account for the size of the&#xA;                         * grid cells.  I figure since in the spatial domain you would&#xA;                         * multiply by the cell_size in the frequency domain I suppose you&#xA;                         * would divide.&#xA;                         * &#xA;                         * I think this may be the issue because I am getting kx and ky &amp;gt; k0&#xA;                         */&#xA;                        double kx = x / cell_size;&#xA;                        double ky = y / cell_size;&#xA;                        std::cout &amp;lt;&amp;lt; std::sqrt(complex(k0 * k0 - kx * kx - ky * ky)) &amp;lt;&amp;lt; std::endl;&#xA;                        //I think you would calculate kz = sqrt(k0^2 - kx^2 - ky^2)&#xA;                        e_prime[y * N + x] *= std::exp(-I * dz * std::sqrt(complex(k0 * k0 - kx * kx - ky * ky)));&#xA;                }&#xA;        }&#xA;        &#xA;        //do the inverse fft&#xA;        p = fftw_plan_dft_2d(N, N, (fftw_complex*)e_prime, (fftw_complex*)e, FFTW_BACKWARD, FFTW_ESTIMATE | FFTW_NO_SIMD);&#xA;        fftw_execute(p);&#xA;        fftw_destroy_plan(p);&#xA;       &#xA;        //this should be what the diffraction pattern looks like.&#xA;        save(&amp;quot;diffracted.ppm&amp;quot;, e);&#xA;        &#xA;        delete[] e;&#xA;        delete[] e_prime;&#xA;        return 0;&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is the slit I am diffracting through.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7WZdj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7WZdj.jpg&quot; alt=&quot;Slit (Electric field input)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the FFT. (Look at the corners)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7mSo6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7mSo6.jpg&quot; alt=&quot;Incorrect FFT&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I didn't bother including the last image of the diffraction pattern because it is just black.&lt;/p&gt;&#xA;" OwnerUserId="19602" OwnerDisplayName="chasep255" LastEditorUserId="23791" LastEditDate="2021-08-26T23:21:56.383" LastActivityDate="2023-05-19T04:07:25.850" Title="Help with Fourier beam propagation method" Tags="|computational-physics|c++|fourier-transform|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="23353" PostTypeId="2" ParentId="23352" CreationDate="2016-03-13T13:33:04.570" Score="1" Body="&lt;p&gt;Your cell_size should be the length of the box, not the resolution of each cell, so use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; const double cell_size = lambda / 4.0 * N;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;instead. And I also think you should be doing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    double kx = (x-N/2)*2*M_PI/cell_size;&#xA;    double ky = (y-N/2)*2*M_PI/cell_size;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A couple of side notes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using FFTs you should pick your grid size ($N$) to be a power of 2 for optimal efficiency. Also, you should pick units of nanometres so that you don't have those significant exponents knocking around and potentially messing up precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; In case you copy and pasted this, I originally made a typo of (x-N/2) instead of (y-N/2). Now corrected.&lt;/p&gt;&#xA;" OwnerUserId="11739" OwnerDisplayName="lemon" LastActivityDate="2016-03-13T13:33:04.570" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="23354" PostTypeId="1" CreationDate="2016-03-13T14:17:13.107" Score="2" ViewCount="176" Body="&lt;p&gt;I am in the process of building a robotics physics engine. I have been using the Linear ODE $x' = Ax + b$ for the core of my physics integration, but have never found a really good solution method for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem gets harder than the standard ODE solution because my A matrix is singular. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of an $A$ matrix and a $b$ matrix from my simulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{vmatrix}&#xA;0 &amp;amp;  0  &amp;amp; 0.64 &amp;amp; 0 &amp;amp; 0.64 &amp;amp; 1.27 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp;  0  &amp;amp; 0    &amp;amp; 0 &amp;amp; -0.64 &amp;amp; 1.27 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0.64 \\&#xA;0 &amp;amp;  0  &amp;amp; -29.53 &amp;amp; 0 &amp;amp; -28.2 &amp;amp; -56.4 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp;  0  &amp;amp; 1.33 &amp;amp; -2.67 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 171.2 &amp;amp; -1.33 \\&#xA;0 &amp;amp;  0  &amp;amp; 1.33 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1.33 \\&#xA;0 &amp;amp;  0  &amp;amp; 2.67 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 2.67 \\&#xA;0 &amp;amp;  0  &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp;  0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp;  0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp;  0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 28.2 &amp;amp; -56.4 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -29.53 \\&#xA;\end{vmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{vmatrix}&#xA;0\\&#xA;0\\&#xA;9.76\\&#xA;0\\&#xA;171.2\\&#xA;0\\&#xA;0\\&#xA;0\\&#xA;0\\&#xA;-9.76\\ &#xA;\end{vmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to several previous questions, the solution to this ODE is the expression. &#xA;$$&#xA;x(t)=e^{At}y+\int_0^t e^{As}b ds&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, the integral of this is the hard part. According to &lt;a href=&quot;http://www.wolframalpha.com/input/?i=integral%20of%20e%5E%28A%20s%29*b*ds%20from%200%20to%20t&quot; rel=&quot;nofollow&quot;&gt;Wolfram Alpha&lt;/a&gt;, the solution is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_0^t e^{As}b ds = \frac{b(e^{At}-1)}A&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is equivelant to the taylor series:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;bt + \frac{1}2Abt^2 + \frac{1}6A^2bt^3 + \frac{1}{120}A^3bt^4 + ...&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current, inefficient solution relies on evaluating this taylor series. I cannot evaluate the solution directly, since I cannot invert A. I have tried using $A^+$, however this does not yield the correct solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better solution to this problem? I am currently taking Calc II, and have attempted to teach myself the Linear Algebra necessary to build this software. Therefore I don't know of all the tricks that might be used to tame this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the theano python library to perform these calculations, though if I can get a solution using scipy/numpy I can figure out how to do it in theano.&lt;/p&gt;&#xA;" OwnerUserId="19265" LastEditorUserId="10134" LastEditDate="2016-03-15T14:17:59.463" LastActivityDate="2016-03-15T14:17:59.463" Title="Methods for solving $x'=Ax+b$ for small, sparse, singular $A$" Tags="|ode|simulation|scipy|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23355" PostTypeId="2" ParentId="23354" CreationDate="2016-03-13T15:57:05.103" Score="9" Body="&lt;p&gt;Any general-purpose ODE solver should be able to handle this linear coupled system of ODE very easily, for example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html&quot;&gt;scipy.integrate.ode&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;CVODE from the Sundials solver suite; it appears to have Python bindings &lt;a href=&quot;https://pypi.python.org/pypi/python-sundials/0.5&quot;&gt;here&lt;/a&gt;, and perhaps there are others.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This kind of thing is typically discussed in any textbook on numerical methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, computing the matrix exponential is a bit tricky — see &quot;Nineteen Dubious Ways&quot; by Moler, Van Loan, your truncated Taylor series approach is their Method 1. There is already a &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.expm.html&quot;&gt;matrix exponential function in scipy&lt;/a&gt;, so you should prefer using that to a naive implementation. Truncated Taylor series wouldn't even work for a general $1\times 1$ case because of numerical instability when $A&amp;lt;0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also recommend against doing it this way due to the likely catastrophic cancellation in $A^{-1}(e^{At}-1)$ — this formula is inaccurate even for small scalar $A$ and is difficult to use even if you used an accurate matrix exponential, like &lt;code&gt;scipy.linalg.expm&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your formula for the general solution $x(t)$ seems suspicious: Wolfram Alpha gave you an answer assuming that $A$ is a scalar number, which is the main reason that formula assumes $A$ is invertible and has the $b$ on the wrong side of the matrix.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-03-13T16:42:01.263" LastActivityDate="2016-03-13T16:42:01.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23356" PostTypeId="1" CreationDate="2016-03-13T19:50:00.753" Score="1" ViewCount="387" Body="&lt;p&gt;I need to calculate the lagrange polynomial  which approximates $e^x$ at $101$ points, the points $\frac{k}{101^2}$ for $k\in\{0,1,2\dots 100\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import java.math.*;&#xA;import org.apache.commons.math3.analysis.polynomials.PolynomialFunctionLagrangeForm;&#xA;&#xA;public class interpolation {&#xA;&#xA;&#xA;    public static void main(String[] args) {&#xA;        // TODO Auto-generated method stub&#xA;        int N = 100;&#xA;&#xA;        double [] X = new double [N+1];&#xA;        double [] Y = new double [N+1];&#xA;&#xA;        // Here we give the computer the points:&#xA;&#xA;        for(int k=0; k&amp;lt;=N ;k++){&#xA;            double a = k/(N*N*1.0);&#xA;            double b = Math.exp(a);&#xA;            X[k]=a;&#xA;            Y[k]=b;&#xA;        }&#xA;&#xA;        // here we build the polynomial&#xA;&#xA;         PolynomialFunctionLagrangeForm pol = new PolynomialFunctionLagrangeForm( X , Y );&#xA;&#xA;         double [] P = pol.getCoefficients();&#xA;&#xA;         // here we print the polynomial&#xA;&#xA;         for(int i=P.length -1 ;i &amp;gt;= 0;i--){&#xA;             System.out.print( P[i] + &quot;    &quot;);&#xA;         }&#xA;&#xA;&#xA;    }&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It gives good results for $N=5,10$ but bad ones for $N=50,100$ (it doesn't resemble the Taylor expansion at all). I would like help in calculating this polynomial more precisely. Thank you kindly.&lt;/p&gt;&#xA;" OwnerDisplayName="user19606" LastEditorDisplayName="user19606" LastEditDate="2016-03-14T00:37:16.433" LastActivityDate="2016-03-14T22:16:49.717" Title="Calculating lagrange polynomial for 100 points?" Tags="|numerical-analysis|interpolation|polynomials|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="23357" PostTypeId="1" AcceptedAnswerId="23850" CreationDate="2016-03-13T22:29:32.817" Score="2" ViewCount="1357" Body="&lt;p&gt;I want to solve PDE system, which consists of Poisson equation, continuity equations for electron and hole with drift-diffusion equation numerically, by using method called Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method is explained in several literatures, including&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://nanohub.org/resources/1565/download/ddmodel_solution_details_word.pdf&quot; rel=&quot;nofollow&quot;&gt;https://nanohub.org/resources/1565/download/ddmodel_solution_details_word.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(But I'm not sure I understand this method at all)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{gathered}&#xA;  \frac{\partial }{{\partial x}} \left( {\varepsilon \frac{\partial }{{\partial x}}V} \right) =  - q\left( {p - n + N_D^ +  - N_A^ - } \right) \hfill \\&#xA;  \frac{\partial }{{\partial x}}{J_n} = q\left( {\frac{{\partial n}}{{\partial t}} + \frac{{n - {n_0}}}{{{\tau _n}}} - {G_{op}}} \right) \hfill \\&#xA;  \frac{\partial }{{\partial x}}{J_p} =  - q\left( {\frac{{\partial p}}{{\partial t}} + \frac{{p - {p_0}}}{{{\tau _p}}} - {G_{op}}} \right) \hfill \\&#xA;  {J_n} =  - qn{\mu _n}\frac{\partial }{{\partial x}}V + q{D_n}\frac{\partial }{{\partial x}}n \hfill \\&#xA;  {J_p} =  - qp{\mu _p}\frac{\partial }{{\partial x}}V - q{D_p}\frac{\partial }{{\partial x}}p \hfill \\ &#xA;\end{gathered}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Arranging these, I got&#xA;$$&#xA;\begin{gathered}&#xA;  0 = \frac{\partial }{{\partial x}} \cdot \left( {\varepsilon \frac{\partial }{{\partial x}}V} \right) + q\left( {p - n + N_D^ +  - N_A^ - } \right) \hfill \\&#xA;  0 = \frac{{\partial n}}{{\partial t}} + {\mu _n}\frac{{\partial n}}{{\partial x}}\frac{{\partial V}}{{\partial x}} + {\mu _n}n\frac{{{\partial ^2}V}}{{\partial {x^2}}} - {D_n}\frac{{{\partial ^2}n}}{{\partial {x^2}}} + \frac{{n - {n_0}}}{{{\tau _n}}} - {G_{op}} \hfill \\&#xA;  0 = \frac{{\partial p}}{{\partial t}} - {\mu _p}\frac{{\partial p}}{{\partial x}}\frac{{\partial V}}{{\partial x}} - {\mu _p}p\frac{{{\partial ^2}V}}{{\partial {x^2}}} + {D_p}\frac{{{\partial ^2}p}}{{\partial {x^2}}} + \frac{{p - {p_0}}}{{{\tau _p}}} - {G_{op}} \hfill \\ &#xA;\end{gathered}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what I understand from general Newton's method for solving equation system is following; I have system of equation&#xA;$$&#xA;\begin{aligned}&#xA;f_1(x,y,z)=0 \\&#xA;f_2(x,y,z)=0 \\&#xA;f_3(x,y,z)=0&#xA;\end{aligned}&#xA;\qquad\rightarrow\qquad&#xA;\mathbf{F}(x,y,z)=0&#xA;$$&#xA;and initial value (starting value)&#xA;$$&#xA;\mathbf{X}_0=(x_0,y_0,z_0)&#xA;$$&#xA;Then I create Jacobian Matrix&#xA;$$&#xA;\mathbf{J}(x,y,z)=&#xA;\left[&#xA;\begin{array}{ccc}&#xA;\partial_xf_1 &amp;amp; \partial_yf_1 &amp;amp; \partial_zf_1\\&#xA;\partial_xf_2 &amp;amp; \partial_yf_2 &amp;amp; \partial_zf_2\\&#xA;\partial_xf_3 &amp;amp; \partial_yf_3 &amp;amp; \partial_zf_3&#xA;\end{array}&#xA;\right]&#xA;$$&#xA;The $\mathbf{X}$ that give $\mathbf{F}=0$ is &#xA;$$&#xA;\mathbf{X}_{n+1}=\mathbf{X}_n-\mathbf{J}^{-1}(\mathbf{X}_n)\mathbf{F}(\mathbf{X}_n)&#xA;\qquad&#xA;(n=0,1,2...)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got this idea, and Now I want to apply this into the poisson, continuity, drift-diffusion eq. system.&#xA;First, I have my system of equation, with independent variables $V, n, p$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;0 =&amp;amp; f_1(V,n,p)= \frac{\partial }{{\partial x}} \left( {\varepsilon \frac{\partial }{{\partial x}}V} \right) + q\left( {p - n + N_D^ +  - N_A^ - } \right) &#xA;\\&#xA;0 =&amp;amp; f_2(V,n,p)= \frac{{\partial n}}{{\partial t}} + {\mu _n}\frac{{\partial n}}{{\partial x}}\frac{{\partial V}}{{\partial x}} + {\mu _n}n\frac{{{\partial ^2}V}}{{\partial {x^2}}} - {D_n}\frac{{{\partial ^2}n}}{{\partial {x^2}}} + \frac{{n - {n_0}}}{{{\tau _n}}} - {G_{op}}&#xA;\\&#xA;0 =&amp;amp; f_3(V,n,p)= \frac{{\partial p}}{{\partial t}} - {\mu _p}\frac{{\partial p}}{{\partial x}}\frac{{\partial V}}{{\partial x}} - {\mu _p}p\frac{{{\partial ^2}V}}{{\partial {x^2}}} + {D_p}\frac{{{\partial ^2}p}}{{\partial {x^2}}} + \frac{{p - {p_0}}}{{{\tau _p}}} - {G_{op}}&#xA;\end{aligned}&#xA;$$&#xA;I assumed that the initial condition might be&#xA;$$&#xA;V(x)=\left\{&#xA;\begin{aligned}&#xA;0 \qquad &amp;amp; (x_1&amp;lt;x&amp;lt;0)\\&#xA;k_BT\ln\left(\frac{N_AN_D}{n_i^2}\right)\qquad &amp;amp;(0&amp;lt;x&amp;lt;x_2)\\&#xA;\end{aligned}\right.&#xA;\qquad&#xA;n(x)=\left\{&#xA;\begin{aligned}&#xA;N_D \qquad &amp;amp; (x_1&amp;lt;x&amp;lt;0)\\&#xA;\frac{n_i^2}{N_A} \qquad &amp;amp;(0&amp;lt;x&amp;lt;x_2)\\&#xA;\end{aligned}\right.&#xA;\qquad&#xA;p(x)=\left\{&#xA;\begin{aligned}&#xA;\frac{n_i^2}{N_D} \qquad &amp;amp; (x_1&amp;lt;x&amp;lt;0)\\&#xA;N_A \qquad &amp;amp;(0&amp;lt;x&amp;lt;x_2)\\&#xA;\end{aligned}\right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got stuck when I was creating Jacobian $ \mathbf{J}_{11}$&#xA;$$&#xA;\partial_Vf_1=\frac{\partial }{\partial V}\left[&#xA;\frac{\partial }{{\partial x}} \left( {\varepsilon \frac{\partial }{{\partial x}}V} \right) + q\left( {p - n + N_D^ +  - N_A^ - } \right)&#xA;\right]&#xA;\stackrel{?}{=}&#xA;\frac{\partial }{{\partial x}} \left( {\varepsilon \frac{\partial }{{\partial x}}} \right)&#xA;$$&#xA;Apparently, I got an Jacobian element which is just operator!&#xA;Am I proceeding correctly?&#xA;If I may go further, I have my Jacobian that looks like&#xA;$$&#xA;\mathbf{J}(V,n,p)=&#xA;\left[&#xA;\begin{array}{ccc}&#xA;\frac{\partial }{{\partial x}} \left( {\varepsilon \frac{\partial }{{\partial x}}} \right)&#xA;&amp;amp; -q&#xA;&amp;amp; q&#xA;\\&#xA;\mu_n\frac{\partial^2 n}{\partial x^2}+mu_n n\frac{\partial^2 }{\partial x^2}&#xA;&amp;amp; \frac{{\partial }}{{\partial t}} + {\mu _n}\frac{{\partial^2 V}}{{\partial x^2}} + {\mu _n}\frac{{{\partial ^2}V}}{{\partial {x^2}}} - {D_n}\frac{{{\partial ^2}}}{{\partial {x^2}}} + \frac{1}{{{\tau _n}}}&#xA;&amp;amp; 0&#xA;\\&#xA;-\mu_p\frac{\partial^2 p}{\partial x^2}-mu_p p\frac{\partial^2 }{\partial x^2}&#xA;&amp;amp; 0&#xA;&amp;amp; \frac{{\partial }}{{\partial t}} - {\mu _p}\frac{{\partial^2 V}}{{\partial x^2}} - {\mu _p}\frac{{{\partial ^2}V}}{{\partial {x^2}}} + {D_p}\frac{{{\partial ^2}}}{{\partial {x^2}}} + \frac{1}{{{\tau _p}}}&#xA;\end{array}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;=&#xA;\left[&#xA;\begin{array}{ccc}&#xA;\frac{\partial }{{\partial x}} \left( {\varepsilon \frac{\partial }{{\partial x}}} \right)&#xA;&amp;amp; -q&#xA;&amp;amp; q&#xA;\\&#xA;\mu_n\frac{\partial^2 n}{\partial x^2}+\mu_n n\frac{\partial^2 }{\partial x^2}&#xA;&amp;amp; \frac{{\partial }}{{\partial t}} + 2{\mu _n}\frac{{\partial^2 V}}{{\partial x^2}} - {D_n}\frac{{{\partial ^2}}}{{\partial {x^2}}} + \frac{1}{{{\tau _n}}}&#xA;&amp;amp; 0&#xA;\\&#xA;-\mu_p\frac{\partial^2 p}{\partial x^2}-\mu_p p\frac{\partial^2 }{\partial x^2}&#xA;&amp;amp; 0&#xA;&amp;amp; \frac{{\partial }}{{\partial t}} - 2{\mu _p}\frac{{{\partial ^2}V}}{{\partial {x^2}}} + {D_p}\frac{{{\partial ^2}}}{{\partial {x^2}}} + \frac{1}{{{\tau _p}}}&#xA;\end{array}&#xA;\right]&#xA;$$&#xA;I don't have any idea how to apply inverse matrix of this &quot;operator&quot; matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct result? &lt;/p&gt;&#xA;" OwnerUserId="19465" LastEditorUserId="19465" LastEditDate="2016-03-14T02:07:41.193" LastActivityDate="2016-04-26T21:30:12.477" Title="How to implement Newton method in solving 1D PDE system? (ie. Poisson eq, continuity eq, drift-diffusion eq.)" Tags="|computational-physics|newton-method|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="23361" PostTypeId="1" CreationDate="2016-03-14T18:07:23.640" Score="0" ViewCount="606" Body="&lt;p&gt;I need to solve a set of 5 PDEs for functions $u(x,t)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked up &lt;a href=&quot;http://www.mathworks.com/help/matlab/math/partial-differential-equations.html?refresh=true#f1-697343&quot; rel=&quot;nofollow&quot;&gt;the Matlab function&lt;/a&gt; &lt;code&gt;pdepe&lt;/code&gt;. It looked perfect for my case, until I read the line:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$f(x,t,u,\partial u/\partial x)$ is a flux term and $s(x,t,u,\partial u/\partial x)$ is a source term. The flux term must depend on $\partial u/\partial x$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my problem for 4/5 of my equations $f(x,t,u,\partial u/\partial x)$ doesn't depend on $\partial u/\partial x$, and in 1/5 of my equations $f(x,t,u,\partial u/\partial x)=0$. In my equations, there's no second derivative of $u$ with respect to $x$.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does this mean i can't use pdepe in order to obtain a solution for my problem?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Oddly enough, in &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/pdepe.html&quot; rel=&quot;nofollow&quot;&gt;the link&lt;/a&gt; for the &lt;code&gt;pdepe&lt;/code&gt; function, that line I mentioned before: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...The flux term must depend on $\partial u/\partial x$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;does not appear there. also, I would expect that since having $f(x,t,u,\partial u/\partial x)$ that does not depend on $\partial u/\partial x$ is just a special case, it wouldn't in any way prevent me from obtaining a solution.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;So all in all, I want to know if I can use &lt;code&gt;pdepe&lt;/code&gt; even if $f(x,t,u,\partial u/\partial x)$ does not depend on $\partial u/\partial x$?&lt;/li&gt;&#xA;&lt;li&gt;If not, what would happen if I try to solve it anyways? and what other method I can use to solve my set of PDEs?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My equations look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us use the form of matlab: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;c(x,t,u,∂u/∂x)∂u/∂t=(x^−m)*∂/∂x((x^m)*f(x,t,u,∂u/∂x))+s(x,t,u,∂u/∂x)&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;code&gt;u&lt;/code&gt; is a vector with 5 components, as I have 5 equations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have &lt;code&gt;m=0&lt;/code&gt; in all of  my equations.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eq. 1: c=1, f=0, s=A(u)&#xA;&#xA;eq. 2: c=1, f=-u(2).*B(u(4)), s=C(u) &#xA;&#xA;eq. 3: c=1, f=-u(3).*B(u(5)), s=C(u) &#xA;&#xA;eq. 4: c=1./B(u(4)), f=-u(4), s=D(u) &#xA;&#xA;eq. 5: c=1./B(u(5)), f=-u(5), s=E(u)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;A&lt;/code&gt;,&lt;code&gt;B&lt;/code&gt;,&lt;code&gt;C&lt;/code&gt;,&lt;code&gt;D&lt;/code&gt;,&lt;code&gt;E&lt;/code&gt; are some functions of &lt;code&gt;u=[u(1);u(2);u(3);u(4);u(5)]&lt;/code&gt; but not of &lt;code&gt;x&lt;/code&gt;,&lt;code&gt;t&lt;/code&gt;,&lt;code&gt;du/dx&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it's important to note that for a 1st order ODE it can always be chosen that f=0, so that all of the terms can go into s (and in that case, obviously f does not depend on du/dx). here i just chose a convinient way to represent my equations.&lt;/p&gt;&#xA;" OwnerUserId="19617" LastEditorUserId="19617" LastEditDate="2016-03-19T19:59:56.970" LastActivityDate="2016-03-19T19:59:56.970" Title="How to solve my PDE's in Matlab?" Tags="|matlab|pde|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23362" PostTypeId="2" ParentId="23361" CreationDate="2016-03-14T19:04:38.473" Score="5" Body="&lt;p&gt;The short answer is that the &lt;code&gt;pdepe&lt;/code&gt; function does not strictly require that the flux&#xA;term be a function of $\partial u/\partial x$ or that the pde even contain a flux term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However,  &lt;code&gt;pdepe&lt;/code&gt; is designed to work best for this case. The&quot;pe&quot; in the function&#xA;name stands for parabolic-elliptic; this is the class of PDE it was designed for. Based on&#xA;your description, your system of PDE is likely hyperbolic. How well  &lt;code&gt;pdepe&lt;/code&gt; will work&#xA;in your case depends on the specific equations and boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numerical solution of hyperbolic PDEs can be quite challenging. Many books and papers&#xA;have been written on this topic. However, many of the basic numerical techniques for&#xA;solving hyperbolic PDEs rely on adding some &quot;artificial diffusion&quot; to the equations. &#xA;In 1D this amounts to adding a small flux term that &lt;strong&gt;does&lt;/strong&gt; depend on &#xA;$\partial u/\partial x$. You can achieve the same effect by doing this manually in &#xA; &lt;code&gt;pdepe&lt;/code&gt;. The downside of this approach is that some experimentation is required&#xA;to determine the right amount of artificial diffusion; too much and the solution may&#xA;be overly smeared out, too little and the solution will show some undesirable oscillations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know more, you might take a look at this classic text by Strikwerda:&#xA;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898715679&quot;&gt;http://www.amazon.com/Difference-Schemes-Partial-Differential-Equations/dp/0898715679&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Core MATLAB doesn't have a function designed specifically for hyperbolic PDEs. But Shampine, one&#xA;of the authors of &lt;code&gt;pdepe&lt;/code&gt;, has written such a function for MATLAB and it can be downloaded here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://faculty.smu.edu/shampine/current.html&quot;&gt;http://faculty.smu.edu/shampine/current.html&lt;/a&gt;&#xA;(See the section titled Hyperbolic PDEs)&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-03-14T19:04:38.473" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="23363" PostTypeId="2" ParentId="18788" CreationDate="2016-03-14T22:08:59.470" Score="2" Body="&lt;p&gt;There exists a clever algorithm for doing this, based on ideas due to Cayley, using the &lt;a href=&quot;https://en.wikipedia.org/wiki/B%C3%A9zout_matrix&quot; rel=&quot;nofollow&quot;&gt;Bézout matrix&lt;/a&gt;. I learned this from &lt;a href=&quot;http://dx.doi.org/10.1007/s00211-014-0635-z&quot; rel=&quot;nofollow&quot;&gt;[1]&lt;/a&gt;, which describes an algorithm for a generic version of this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we want to find all the critical points of the function $\|f(s)-f(t)\|^2$ (viewing $f$ as a function $\mathbb{R}\to\mathbb{R}^2$), a sufficient condition is that&#xA;$$ P_s(t) := (f(s)-f(t))^\top f'(s) = 0, \qquad Q_s(t) := (f(s)-f(t))^\top f'(t) = 0, $$&#xA;which is a pair of simultaneous trigonometric equations in $s$ and $t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Form the Bézout matrix as the matrix with entries $B_{ij}(s)$ defined by the expansion&#xA;$$ \frac{P_s(x)Q_s(y) - P_s(y)Q_s(x)}{x-y} = \sum_{i,j} B_{ij}(s) \phi_i(x) \phi_j(y). $$&#xA;Here $\phi_i$ are a suitable basis for the function $f,P,Q$. Typically they are either monomials, or Chebyshev polynomials, but presumably can also be elementary trigonometric functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those values of $s$ for which the exists a solution $t$ to the pair of equations $P_s(t) = Q_s(t) = 0$ cause the matrix $B_{ij}(s)$ to be singular. In other words, if we expand the matrix $B_{ij}(s)$ in the basis $\phi_i(s)$ as&#xA;$$ B_{ij}(s) = \sum_k B_{ij}^{(k)}\phi_k(s), $$&#xA;for a solution $s$ to exist it is necessary that&#xA;$$ \det B_{ij}(s) = 0, $$&#xA;which is a &lt;em&gt;polynomial eigenvalue problem&lt;/em&gt; (the polynomial's coefficients in the basis $\phi$ are the matrices $B_{ij}^{(k)}$). This is a well-known problem, and has standard, although possibly difficult algorithms, such as the &lt;code&gt;polyeig&lt;/code&gt; function in Matlab. In particular it can turned into a &lt;em&gt;linear&lt;/em&gt; eigenvalue problem for $s$ by constructing the block-companion matrix (if $\phi$ are monomials) or the block-colleague matrix (if $\phi$ are Chebyshev), and solving that &lt;em&gt;linearized&lt;/em&gt; version of the PEP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may also be helpful to partition the $(s,t)$-square into some number of subsquares and use the Bézout matrix approach on individual subsquares, as this can substantially lower the degree of the polynomial eigenvalue problem and its linearized version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1007/s00211-014-0635-z&quot; rel=&quot;nofollow&quot;&gt;Computing the common zeros of two bivariate functions via Bézout resultants&lt;/a&gt; by Yuji Nakatsukasa, Vanni Noferini, Alex Townsend&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-03-15T21:57:26.077" LastActivityDate="2016-03-15T21:57:26.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23366" PostTypeId="2" ParentId="23340" CreationDate="2016-03-15T12:52:36.977" Score="4" Body="&lt;p&gt;Fast filter approximations have been studied for a long time, especially to implement IIR filters, like Gaussians and their derivatives. You may want to reuse such concepts, with keywords like &lt;em&gt;integral image&lt;/em&gt;, &lt;em&gt;summed-area tables&lt;/em&gt;,  &lt;em&gt;box filter&lt;/em&gt;, &lt;em&gt;recursive filtering&lt;/em&gt;. You can start from a recent review:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ipol.im/pub/art/2013/87/article.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Survey of Gaussian Convolution Algorithms&lt;/a&gt;, 2013&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;and dig other cited papers.&lt;/p&gt;&#xA;&lt;p&gt;I am thinking about approximating an impulse response as a sum of boxes, on top of the others. The next one could be of interest &amp;quot;We have presented a method for automatically approximating an arbitrary&#xA;2-D filter by a box filter&amp;quot;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.contrib.andrew.cmu.edu/%7Ebpires/paperMedia/box_filters.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Approximating image filters with box filters&lt;/a&gt;, 2011&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mia.uni-saarland.de/Publications/gwosdek-ssvm11.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Theoretical Foundations of Gaussian Convolution by Extended Box Filtering&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1016/S0165-1684(02)00243-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sum-box technique for fast linear filtering&lt;/a&gt;, 2002&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://nyu.pure.elsevier.com/en/publications/boxlets-a-fast-convolution-algorithm-for-signal-processing-and-ne&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boxlets: A fast convolution algorithm for signal processing and neural networks&lt;/a&gt;, 1998&lt;/li&gt;&#xA;&lt;li&gt;with some &lt;a href=&quot;http://nghiaho.com/?p=1159&quot; rel=&quot;nofollow noreferrer&quot;&gt;Octave code&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I am not sure though you can gain a lot in number of operations, at least for separable filters.&lt;/p&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-03-15T12:52:36.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23367" PostTypeId="2" ParentId="23345" CreationDate="2016-03-15T18:41:32.467" Score="4" Body="&lt;p&gt;&lt;strong&gt;Clarification:&lt;/strong&gt; My answer is specifically written for adaptive integration routines with deterministic error control &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/Cubature&quot; rel=&quot;nofollow&quot;&gt;like this one&lt;/a&gt;. It becomes moot for sparse grid and Monte Carlo based integration routines, whose error control is not performed in the manner described below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A significant cost of automatic black-box, tensor-product-based integration routines is error control, from two aspects&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Wasted function evaluations. All adaptive integration work by estimating the integrand and the error using a low-order rule or coarser partitioning, and repeating this with progressively higher-order rules or finer partitions until the error requirements are satisfied. Nested integration rules allow &lt;em&gt;some&lt;/em&gt; of work done in previous steps to be recycled, but often not all.&lt;/li&gt;&#xA;&lt;li&gt;In an effort to conserve function evaluations, highly nested rules like Gauss-Kronrod or Newton-Cotes are often used in adaptive integration codes. Nested quadrature rules are suboptimal quadrature rules, in that they are considerably less accurate than optimal rules (e.g. Gauss-Legendre and Clenshaw Curtis) for a particular class of functions over a fixed quadrature order. In other words, nested rules make less efficient use of function evaluations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The inefficiencies associated with error control in the $y$ dimension can be entirely removed using a semi-analytic treatment over $y$. Suppose that $f(x,y)$ were smooth (i.e. not just piecewise smooth) in $y$. We are free to pick an optimal quadrature rule, and to derive an a prior bound on the quadrature order $r$ in order to guarantee&#xA;$$\left|\sum_{k=1}^r w_k f(x,y_k) - \int_{[0,1]}f(x,y)\,\mathrm{d}y\right| \le \epsilon\qquad \text{for all }x\in[0,1]^n,$$ &#xA;using, say, the Lipschitz constant of $f(x,y)$ over $y$ for all $x$. In a semi-analytical treatment of variable error control, we may even let the quadrature order $r$ vary as a function with respect to $x$ and $\epsilon$ to further enhance efficiency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replacing numerical steps by optimal, semi-analytical ones results in enhanced efficiency for the outer integration routine. Considering that $g(x)$ lies at the heart of an $n$-dimensional integral, any small gains made in the evaluation of $g(x)$ will be greatly magnified over the full integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case where $f(x,y)$ is only piecewise smooth, optimal quadrature rules can be developed so long as the boundaries of the piecewise segments are known, so similar arguments hold. Even if the boundaries are not known, heuristically, Clenshaw-Curtis is often able to handle piecewise smooth functions very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give an application example, this exact issue arose for me in the evaluation of volume-to-volume singular integrals &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/Cubature&quot; rel=&quot;nofollow&quot;&gt;in this paper&lt;/a&gt;, and my treatment is similar to the one proposed above. As a rule of thumb, it is always advisable to remove as many dimensions as possible using analytical arguments before feeding the problem through a black-box integration routine.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2016-03-15T20:55:00.567" LastActivityDate="2016-03-15T20:55:00.567" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="23369" PostTypeId="1" CreationDate="2016-03-15T22:30:04.807" Score="4" ViewCount="122" Body="&lt;p&gt;When we create numerical models of a real-world system, we usually go through a few phases (from Abramowitz 2010):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a &lt;strong&gt;perceptual&lt;/strong&gt; model, where we consider the relevant components of the system&lt;/li&gt;&#xA;&lt;li&gt;a &lt;strong&gt;conceptual&lt;/strong&gt; model, where we consider the relationships between those components&lt;/li&gt;&#xA;&lt;li&gt;a &lt;strong&gt;mathematical&lt;/strong&gt; model, where we come up with equations and formulae to express the conceptual model&lt;/li&gt;&#xA;&lt;li&gt;a &lt;strong&gt;numerical&lt;/strong&gt; model, that allows the computer to estimate the results of the mathematical model on a given spatiotemporal scale&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Divide by zero errors are a pretty common cause of premature model death. Sometimes that's just due to poor programming. But if we exclude that possibility (assume a perfect programmer), &lt;strong&gt;is a divide by zero error an indication that the conceptual model is wrong?&lt;/strong&gt; That is, is it reasonable to assume that the equivalent of a divide-by-zero error never happens in the real world? I guess that this question presupposes that mathematics is in some way &quot;real&quot;, comments on the relevance of that would also be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;references&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Abramowitz, Gab 2010, &lt;em&gt;Model Independence in Multi-Model Ensemble Prediction&lt;/em&gt;. Australian Meteorological and Oceanographic Journal 59: 3–6. &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.222.5811&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.222.5811&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1232" LastActivityDate="2016-03-16T10:21:12.563" Title="Is a divide by zero error an indication of a bad conceptual model?" Tags="|numerical-modelling|modeling|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23370" PostTypeId="1" AcceptedAnswerId="23384" CreationDate="2016-03-16T08:36:35.003" Score="1" ViewCount="123" Body="&lt;p&gt;In my view, it is needed,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;To save computational time.&lt;/li&gt;&#xA;&lt;li&gt;To define accurate boundary conditions at interface where properties changes drastically.&lt;/li&gt;&#xA;&lt;li&gt;To solver 2 phase problems as single phase only.&lt;/li&gt;&#xA;&lt;li&gt;To capture breaking and joining of droplets/splashing.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are there any other reasons which I am missing? Please guide.&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="19632" LastEditorUserId="19632" LastEditDate="2016-03-18T09:15:18.460" LastActivityDate="2016-03-18T09:15:18.460" Title="In which cases an interface tracking/capturing method needed along with Navier-Stokes solver for flow?" Tags="|fluid-dynamics|navier-stokes|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23371" PostTypeId="2" ParentId="23369" CreationDate="2016-03-16T10:21:12.563" Score="10" Body="&lt;p&gt;We should keep in mind that models are just representations of a portion of reality (a narrow portion), therefore a divide  by zero error or other mathematical error (negative concentration, for instance) is not necessarily a flaw in the conceptual model, only that the model is used beyond its proper range of application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's consider a model for the growth of &lt;em&gt;Saccharomyces cerevisiae&lt;/em&gt;, based on Monod equation at aerobic conditions (consuming glucose and oxygen). I make the approximation that there is a lot of glucose so that it is not limiting and the equations become easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My conceptual model would be: &quot;biomass grows by using oxygen from the medium, besides, I know it will take other compounds such as glucose, ammonium, phosphates, and metals&quot;.&#xA;The mass balance of the process in a batch reactor can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  \frac{dX}{dt} =Y_{X/S} \; \mu_{MAX}\frac{C_O}{K+C_O}X $$&#xA;$$  \frac{dC_O}{dt} =-\mu_{MAX}\frac{C_O}{K+C_O}X $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I know that &lt;em&gt;Saccharomyces cerevisiae&lt;/em&gt; also consumes ammonium as a source of nitrogen. If I want to follow the ammonium concentration in the process, I must write:&#xA;$$  \frac{dC_{NH}}{dt} =-Y_{NH/S} \; \mu_{MAX}\frac{C_O}{K+C_O}X $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with this equation is that if ammonium is limiting, the concentration can go below zero and become negative as there is nothing in the Monod equation taking into account this limitation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My point is that the conceptual model is not wrong, but its mathematical realisation changes depending on the range of application. This is a trivial case where the range of application is very easy to see as it depends on deciding upon which are the limiting compounds, but in other models, knowing whether the mathematical realisation follows the conceptual model can be much more complex. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum up, a divide by zero is not necessarily a conceptual model issue. It can be that you are applying your model where it is not supposed to hold.  &lt;/p&gt;&#xA;" OwnerUserId="18706" LastActivityDate="2016-03-16T10:21:12.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23372" PostTypeId="2" ParentId="23357" CreationDate="2016-03-16T11:17:38.150" Score="2" Body="&lt;p&gt;Here's an extended discussion about how to do it for a single, nonlinear PDE: &lt;a href=&quot;https://dealii.org/developer/doxygen/deal.II/step_15.html&quot; rel=&quot;nofollow&quot;&gt;https://dealii.org/developer/doxygen/deal.II/step_15.html&lt;/a&gt; . You may also want to watch lectures 31.5 and following on nonlinear PDEs here: &lt;a href=&quot;http://www.math.tamu.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.tamu.edu/~bangerth/videos.html&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I co-wrote that program and these are my lectures.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-03-16T11:17:38.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23373" PostTypeId="1" CreationDate="2016-03-16T11:57:36.170" Score="1" ViewCount="104" Body="&lt;p&gt;I am trying to simulate an optic-fiber sensor (in-fiber interferometer) to study its respond to temperature. The method I am using is finite-difference time-domain (FDTD), and I come out with a large memory usage problem. The wavelength of the light source centered at 1.55&amp;mu;m while the optical fiber has diameter of 125&amp;mu;m and the length of it is set to order of 10&lt;sup&gt;-3&lt;/sup&gt;m. This make the number of cells become very large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that I had tried to reduce the length by 1/62.5 which the memory usage drop to a more reasonable value (7.8Gb) but the result become weird, in which the amplitude of the signal at point of interest is higher than the source (I don't know why).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that many people are using beam propagation method (BPM) instead of FDTD. Is FDTD suitable for this case (with some modification on the model) which the wavelength is much much smaller than the geometry, or is there another method which is more suitable to compute? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The structure of the sensor:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/N2nuY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N2nuY.png&quot; alt=&quot;SMF - single mode fiber; MMF - multimode fiber&quot;&gt;&lt;/a&gt;&#xA;SMF - single mode fiber (core diameter: 9&amp;mu;m); MMF - multimode fiber (core diameter: 105&amp;mu;m)&lt;/p&gt;&#xA;" OwnerUserId="19633" LastEditorUserId="19633" LastEditDate="2016-03-21T03:49:10.087" LastActivityDate="2016-03-21T03:49:10.087" Title="Suitable method for simulation of in-fiber interferometer" Tags="|finite-difference|electromagnetism|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23375" PostTypeId="1" CreationDate="2016-03-16T19:05:58.137" Score="2" ViewCount="76" Body="&lt;p&gt;How to numerically find the solution for $a$ and $b$ of this equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_r \frac {\mathrm B_{z_r}(a+m_r+1,b)}{\mathrm B_{z_r}(a+m_r,b)}=K$$&#xA;where $m_r$ are non-negative integers, $0&amp;lt;K&amp;lt;1$, $0&amp;lt;a$, $0&amp;lt;b$, and $\mathrm B_{z}(a,b)=\int_0^zx^{a-1}(1-x)^{b-1}dx$ is the incomplete beta function&lt;/p&gt;&#xA;" OwnerUserId="19308" LastActivityDate="2016-03-16T21:44:15.620" Title="Solving $\sum_r \frac {\mathrm B_{z_r}(a+m_r+1,b)}{\mathrm B_{z_r}(a+m_r,b)}=K$ for $a$ and $b$" Tags="|numerical-analysis|nonlinear-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23378" PostTypeId="2" ParentId="23373" CreationDate="2016-03-16T21:38:00.603" Score="1" Body="&lt;p&gt;Long, axially-invariant (continuous or periodic) structures are best solved with some &quot;propagation ansatz&quot;, e.g in BPM if I remember correctly one assumes that the modal profile of the incoming wave varies slowly in the lateral direction (which is in fact what happens in weak confinement setting, i.e. when the index contrast is low, such as in an optical fiber). This makes the paraxial approximation to the wave equation possible, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A fiber Bragg grating is periodic (for a while), so perhaps you could simulate a single period of it and therefore impose the Bloch condition on the solution in the propagation direction. The &lt;code&gt;mpb&lt;/code&gt; code written by the &quot;ab initio&quot; MIT group is made exactly to extract the propagation diagram (frequency vs wavenumber) of periodic structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are no strong nonlinearities (low laser powers) and no fast transients, the propagation can be solved in the linear stationary regime and the above approximations hold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum up, both the paraxial and the periodic-coefficient settings are approximations. I'd suggest you get acquainted with &lt;code&gt;mpb&lt;/code&gt; (it is a linux code though) since it lets you simulate a single period of the FBG with high resolution, and repeat the analysis for various values of the refractive index contrast.&lt;/p&gt;&#xA;" OwnerUserId="9461" LastEditorUserId="9461" LastEditDate="2016-03-17T12:24:12.053" LastActivityDate="2016-03-17T12:24:12.053" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23379" PostTypeId="1" CreationDate="2016-03-16T21:41:10.557" Score="5" ViewCount="311" Body="&lt;p&gt;I am using the method by Brandt in the FAS Multigrid algorithm to estimate the truncation error in a 1-D flow problem and then use that as the basis for generating a mesh(r-adaptation) in 1-D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The process is standard, in that, transform the equations to a computational domain $\xi$, perform/run computations and map the solution $u$ and $x$ back to the physical domain $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;FLOW PROBLEM&lt;/em&gt; in computational domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_{\xi\xi} - \left(5 x_\xi + \frac{x_{\xi\xi}}{x_\xi}\right) u_\xi = 0 \quad\tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions $u(\xi=0)=0$ and $u(\xi=1) =1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;MESH EQUATION&lt;/em&gt; in computational domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_{\xi\xi} + \frac{2 e_\xi}{e} x_\xi = 0\qquad\tag{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions $x(\xi=0)=0$ and $x(\xi=1)=1$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e$ is the local truncation error from equation-1 and $e_\xi$ its derivative. I am &lt;em&gt;using a simple second order central finite difference&lt;/em&gt; scheme for the linear coupled problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things to know:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Using the FAS methodology, once an accurate truncation error estimate is made(which requires a reasonably fine grid for the flow problem, say N=65), it is then treated as an known input to equation-2.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The analytical solution to equation-1 is : $u(x) = \frac{exp(5 x(\xi)-1)}{exp(5)-1} $ and equation-2 treating the coefficient as known, is  smooth that is: $x(\xi)= \frac{1- exp\left[-2\frac{ e_\xi}{e} \xi\right]}{1- exp\left[-2\frac{ e_\xi}{e}\right]}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;3.Truncation error from flow problem:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7y3Sv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7y3Sv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.a Coefficient $\frac{e_\xi}{e}$: &lt;a href=&quot;https://i.stack.imgur.com/YaNQa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YaNQa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;I have written code in FORTRAN 77 &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I have tried using the thomas algorithm for equation-2 and it never seems to give a reasonable solution other than $x(\xi)=0$ for all interior points. Why is this? Can it be that the system is weakly diagonally dominant?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I then switched the method to solve equation-2 to a simple Seidel iterative scheme, and that seems to work, but the solution to $x(\xi)$ is not smooth like the analytical counterpart. Why ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8de2g.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8de2g.png&quot; alt=&quot;The mesh points mapped to the original physical domain&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Typically what happens is the mesh points would re-distribute accordingly and the flow problem is well resolved. In my case, the solution I obtain for $x(\xi)$ seems to completely change the behavior of the flow problem implying it is obviously not a valid solution. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Any suggestions relevant to any of the above or any insight will be much appreciated.&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="19629" LastEditorUserId="19629" LastEditDate="2016-04-07T20:36:17.200" LastActivityDate="2016-07-18T16:03:15.367" Title="Mesh Generation in 1-D" Tags="|mesh-generation|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23380" PostTypeId="2" ParentId="23375" CreationDate="2016-03-16T21:44:15.620" Score="3" Body="&lt;p&gt;If you have any sort of guess, you can use the following representation:&#xA;$$ g(a,b) = \sum_r \frac{B_{z_r}(a+m_r + 1,b)}{B_{z_r}(a+m_r,b)} - K$$&#xA;Which should be $g(a^*,b^*)=0$ when you find the best solution,$(a^*,b^*)$. You can then use a method like &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow&quot;&gt;Newton&lt;/a&gt;'s for solving nonlinear equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have no clue what initial guess to make for the $(a,b)$ pair, you could formulate the problem as an optimization problem with the cost function:&#xA;$$ J = \frac{1}{2} g(a,b)^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this cost function, you could technically try any optimization approach, though I would recommend some approach that can try to find a global minimum. Some examples might be using &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow&quot;&gt;Genetic Algorithms&lt;/a&gt; or Global/Local &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle_swarm_optimization&quot; rel=&quot;nofollow&quot;&gt;Particle Swarms&lt;/a&gt; to find the global minimum.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-03-16T21:44:15.620" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="23381" PostTypeId="1" AcceptedAnswerId="23388" CreationDate="2016-03-16T22:31:35.860" Score="3" ViewCount="473" Body="&lt;p&gt;Given some sparse unitary square matrix $A$ ($dim=2^n$ if it matters), is there an algorithm to decompose $A$ into a Kronecker/tensor product of smaller unitary matrices? In other words: decompose unitary $A$ into $$A=U_1 \otimes  U_2 \otimes ... \otimes U_k$$ where each $U_i$ is &lt;strong&gt;individually&lt;/strong&gt; unitary and $\otimes$ denotes the Kronecker product. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that in general, the inverse of a Kronecker product is not unique and may not even exist, but if it does, all I would care about is a single tensor decomposition, not an exhaustive list. It wouldn't matter to me what the individual phases are for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I assumed the matrix was a Kronecker product, but in the case that it isn't, is there a way to check?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: I found a resource &lt;a href=&quot;https://math.stackexchange.com/questions/60399/method-to-reverse-a-kronecker-product&quot;&gt;here&lt;/a&gt; describing a general outline on how to do the inverse Kronecker product and how to check if it is a Kronecker product in the first place. It looks like the problem can be solved in polynomial time. If however, I demand unitary matrices, is there a similar scheme?&lt;/p&gt;&#xA;" OwnerUserId="19641" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-04-16T23:53:59.970" Title="Algorithm to decompose a sparse unitary matrix into a Kronecker product of smaller unitary matricies" Tags="|linear-algebra|matrix|tensor-decomposition|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23383" PostTypeId="2" ParentId="1617" CreationDate="2016-03-16T23:51:33.837" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.knime.org/&quot; rel=&quot;nofollow&quot;&gt;Knime&lt;/a&gt; or &lt;a href=&quot;https://rapidminer.com/&quot; rel=&quot;nofollow&quot;&gt;Rapidminer&lt;/a&gt; could likely handle all the requirements as well. Whether they should is up for grabs :)  &lt;/p&gt;&#xA;" OwnerUserId="19643" LastActivityDate="2016-03-16T23:51:33.837" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23384" PostTypeId="2" ParentId="23370" CreationDate="2016-03-17T13:30:38.900" Score="1" Body="&lt;p&gt;Free surfaces occur when there are several phases present with sharp(ish) interfaces free to move. Some methods allow to treat all the phases as a single mathematical problem (e.g. level set or phase field), while some others track explicitly this free surface (e.g. ALE or immersed boundary methods).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Can it save computational time? If you use a method as ALE method and solve only for one phase while simplifying the model of the other (e.g. assuming a perfect fluid with uniform pressure), the computational domain will be smaller but will usually have a more complex time-evolving geometry. This may not save time. On the other hand, if quantities like density are very different between the two phases, the condition number of the linear system associated will be much lower with ALE-like methods.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Two parts here:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;a) Is it more precise? This will all depend on the meshing, a refined mesh in the location of strong gradients of physical properties can also be efficient, although the thickness of &quot;interfaces&quot; in implicit interface methods generally needs to be a few times the mesh size there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) What about the boundary conditions at the interface? Complex boundary conditions are usually easier to write at an explicitly tracked boundary, because you can usually discretise them directly rather than rewrite them in terms of quantities such as the level set function.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Your formulation is misleading. If you have two homogeneous phases and solve only for one using ALE-like methods, yes you will go from space-dependent parameters in an implicit interface method to homogemeous-parameter equqtion, but in general you'll think of it the other way: tracking the interface leads to two problems in two sub-domains.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Explicitly tracking the interface will require you to &lt;strong&gt;model&lt;/strong&gt; the (de)coalescence events rather than just &quot;capture&quot; them. In general, if notheing is done, (de)coalescence events will give rise to mesh self-intersection. You need to prevent this by an explicit numerical modelling in explictly tracked interface methods, while for implicit interface methods this will happen seamlessly -- which doesn't mean that this will happen with the right physics.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2016-03-17T13:30:38.900" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23385" PostTypeId="1" CreationDate="2016-03-17T15:49:57.023" Score="9" ViewCount="2536" Body="&lt;p&gt;Let $A\in \mathbb{R}^{n\times n}$, symmetric and positive definite.  Suppose it takes $m$ units of work to multiply a vector by $A$.  It is well known that performing the CG algorithm on $A$ with condition number $\kappa$ requires $\mathcal{O} (m\sqrt{\kappa})$, units of work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, of course, being a $\mathcal{O}$ statement this is an upper-bound.  And the CG algorithm can always terminate in zero steps with a lucky initial guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we know if there exists a RHS and an initial (unlucky) guess that will require $\mathcal{\Theta}(\sqrt{\kappa})$ steps?  Put another way, is worst-case work-complexity of CG really $\Theta( m \sqrt{\kappa})$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question arises when I tried to determine if the benefit of a preconditioner (lower $\kappa$) outweighed its cost (higher $m$).  Right now, I am working with toy problems and would like to have a better idea before I implement anything in a compiled language.&lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2016-03-21T13:29:15.173" LastActivityDate="2016-03-21T17:19:38.597" Title="What is the worst case complexity of Conjugate Gradient?" Tags="|conjugate-gradient|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23387" PostTypeId="1" CreationDate="2016-03-17T22:37:35.383" Score="1" ViewCount="225" Body="&lt;p&gt;Let $A$ be positive-definite and $C$ diagonal positive-definite, consider the problem of solving the following equation for $\bf x$&#xA;$$A{\bf x}+C\begin{bmatrix}&#xA;e^{x_1}   \\ &#xA; \vdots \\ &#xA;e^{x_n}&#xA;\end{bmatrix}=b.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe nonlinear conjugate gradient can solve this, however the nonlinear part is pretty benign as far as nonlinearities go, is there a trick to put this into a form where conjugate gradient could solve it?&lt;/p&gt;&#xA;" OwnerUserId="6849" LastActivityDate="2016-04-18T00:04:18.517" Title="Can this equation be solved with the conjugate gradient method?" Tags="|optimization|conjugate-gradient|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23388" PostTypeId="2" ParentId="23381" CreationDate="2016-03-17T23:51:41.670" Score="1" Body="&lt;p&gt;After some thought, I realized the condition of being unitary is essentially irrelevant to the problem. This is because of the mixed-product and inverse identities of the Kronecker product. So given a unitary $A$, we have: $$A^{-1}=A^{\dagger} \implies U_1^{-1}\otimes...\otimes U_k^{-1}=U_1^{\dagger}\otimes...\otimes U_k^{\dagger}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$AA^{\dagger}=AA^{-1}=I=U_1 U_1^{\dagger} \otimes ... \otimes U_k U_k^{\dagger}$$&#xA;Where $I$ is the identity matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This last line implies that each individual $U_j U_j^{\dagger}=aI$ for some complex $a$. To make $U_j$ unitary is then just a matter of scaling by a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the inverse Kronecker product for a unitary matrix is automatically unitary up to a constant scaling factor. So to perform this inverse product, one just needs to follow the solution outlined by karahane in the math stackexchange &lt;a href=&quot;https://math.stackexchange.com/questions/60399/method-to-reverse-a-kronecker-product&quot;&gt;here&lt;/a&gt;. A possible followup question would be whether one can get a piece-by-piece unitary nearest Kronecker product approximation. To be self-contained, the academic references for the solution are given below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Golub G, Van Loan C. Matrix Computations, The John Hopkins University Pres. 1996 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Van Loan C., Pitsianis N., Approximation with Kronecker Products, Cornell University, Ithaca, NY, 1992 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genton MG. Separable approximations of space–time covariance matrices. Environmetrics 2007; 18:681–695.&lt;/p&gt;&#xA;" OwnerUserId="19641" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-03-17T23:51:41.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23389" PostTypeId="1" AcceptedAnswerId="23392" CreationDate="2016-03-18T06:09:28.853" Score="4" ViewCount="577" Body="&lt;p&gt;I'm a bit confused about the relationships between these two approximation methods mentioned in the title. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does this kind of interpolation also belongs to the field of spectral methods? &lt;/li&gt;&#xA;&lt;li&gt;Are the Lagrange interpolants we get from using the roots of orthogonal polynomials also orthogonal? &lt;/li&gt;&#xA;&lt;li&gt;It's likely to get mixed with these two methods, could someone please clarify their differences?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;br&gt;&#xA;Let me use the Chebyshev polynomials as an example:&lt;br&gt;&#xA;(1) Using the Chebyshev polynomials as basis functions, then $f(x)$ is approximated as&#xA;\begin{equation}&#xA;f(x)\simeq\sum\limits_{n=0}^Na_nT_n (x)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Interpolating at the $(N+1)$ roots, $x_0, x_1, ..., x_k,...,x_N$, of the Chebyshev polynomial $T_{N+1}(x)$, then the interpolation of $f(x)$ is&lt;br&gt;&#xA;\begin{equation}&#xA;f(x)\simeq P_N(x)=\sum\limits_{k=0}^N f(x_k)L_k(x)&#xA;\end{equation}&#xA;where $L_k(x)$ is the interpolant function at $x=x_k$.&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="18224" LastEditDate="2016-03-19T14:52:37.250" LastActivityDate="2016-04-29T04:56:11.603" Title="Interpolation with the roots of orthogonal polynomials &amp; Spectral expansion" Tags="|interpolation|approximation|spectral-method|special-functions|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23390" PostTypeId="2" ParentId="23385" CreationDate="2016-03-18T14:58:32.070" Score="9" Body="&lt;p&gt;The answer is a resounding yes. The convergence rate bound of $(\sqrt{\kappa}-1) / (\sqrt{\kappa}+1)$ is sharp over the set of symmetric positive definite matrices with condition number $\kappa$. In other words, knowing nothing more about $A$ than its condition number, CG really can take $\sim\sqrt{\kappa}$ iterations to converge. Loosely speaking, the upper-bound is attained if the eigenvalues of $A$ are uniformly distributed (i.e. &quot;peppered&quot;) within an interval of condition number $\kappa$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a more rigorous statement. Deterministic versions are more involved but work using the same principles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theorem&lt;/strong&gt; (Worst-case choice of $A$). Pick any random orthogonal matrix $U$, let $\lambda_1,\ldots,\lambda_n$ be $n$ real numbers uniformly sampled from the real interval $[1,\kappa]$, and let $b=[b_1;\ldots;b_n]$ be $n$ real numbers sampled i.i.d. from the standard Gaussian. Define $$A=U\mathrm{diag}(\lambda_1,\ldots,\lambda_n)U^T.$$ Then in the limit $n\to\infty$, conjugate gradients will convergence with probability one to an $\epsilon$ accurate solution of $Ax=b$ in no less than $\Omega(\sqrt{\kappa}\log\epsilon^{-1})$ iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Proof.&lt;/em&gt; The standard proof is based on optimal Chebyshev polynomial approximations, using techniques found in a number of places, such as &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9781611970937&quot; rel=&quot;nofollow&quot;&gt;Greenbaum's book&lt;/a&gt; or &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898715342&quot; rel=&quot;nofollow&quot;&gt;Saad's book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2016-03-21T15:54:39.703" LastActivityDate="2016-03-21T15:54:39.703" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23392" PostTypeId="2" ParentId="23389" CreationDate="2016-03-18T16:44:16.260" Score="6" Body="&lt;p&gt;I hope I understood the question correctly. They try to compute exactly the same thing, so they really are equivalent. I'll use Chebyshev polynomials because they are easy to analyze.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a function $f(x)$ on $[-1,1]$, the spectral interpolant is the truncation of&#xA;$$ \begin{aligned}&#xA;f(x) &amp;amp;= \sum_{n\geq0} \bar a_n T_n(x), \\&#xA;\bar a_n &amp;amp;= \frac{1+[n&amp;gt;0]}{\pi}\int_{-1}^1 f(x)T_n(x)\frac{dx}{\sqrt{1-x^2}} \\&amp;amp;= \frac{1+[n&amp;gt;0]}{\pi} \int_{0}^\pi f(\cos\theta)\cos(n\theta)\,\mathrm{d}\theta.&#xA;\end{aligned} $$&#xA;The $N$-th degree Lagrange interpolant, using the roots of $T_{N+1}(x)$ is given by &#xA;$$ a_n = \frac{1+[n&amp;gt;0]}{N+1}\sum_{k=0}^N f(x_k) T_n(x_k), \qquad x_k = \cos\frac{\pi (k+\frac12)}{N+1}. $$&#xA;This uses the fact that $\sum_{k=0}^N T_m(x_k)T_n(x_k)$ is zero when $m\neq n$, $m,n\leq N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula for $a_n$ is nothing but the discrete cosine transform (type-II) applied to the function values at $x_k$, due to $T_n(x_k) = \cos \pi n(k+\frac12)/(N+1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are not, strictly speaking, the same, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula for $a_n$ is a trapezoidal rule approximation to the Fourier cosine integral in the formula for the exact coefficients $\bar a_n$. The trapezoidal rule is known to be exponentially accurate for smooth periodic functions (&lt;a href=&quot;http://dx.doi.org/10.1137/130932132&quot; rel=&quot;noreferrer&quot;&gt;Trefethen-Weideman 2014&lt;/a&gt;), which $f(\cos\theta)$ is. Since for a spectral interpolant you would still have to evaluate the integral somehow, the Lagrange interpolant with the roots as nodes is just a way of evaluating that integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to compute the exact difference between $a_n$ and $\bar a_n$, by expanding the sum for $a_n$ using the full series for $f$, and using the identity &#xA;$$ \sum_{k=0}^{N} \cos(j\theta_k)\cos(n\theta_k) = \frac{N+1}{2}\big( [N+1\setminus j-n](-1)^{(j-n)N/(N+1)} + [N+1\setminus j+n](-1)^{(j+n)N/(N+1)}\big), $$&#xA;and for a complex-differentiable function $g(\theta)=f(\cos\theta)$ that is holomorphic in the region of the complex plane $|\Im \theta|&amp;lt;\alpha$, heuristically the error appears to be something on the order&#xA;$$ a_n - \bar a_n \sim |\bar a_{N+1-n}| \lesssim e^{-\alpha(N+1-n)}. $$&#xA;So for smooth functions this error decays very quickly and becomes negligible, so the Lagrange and the spectral interpolants can be considered identical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit&lt;/em&gt;. &lt;strong&gt;What is the relationship between $\sum a_nT_n(x)$ and $\sum f(x_k) \ell_k(x)$?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $a_n$ be defined as above, let $f_1(x) = \sum_{n=0}^{N} a_n T_n(x)$, and let $f_2(x) = \sum_{k=0}^N f(x_k) \ell_k(x)$, where $\ell_k(x) = \prod_{j\neq k} (x-x_j)/(x_k-x_j)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both $f_1(x)$ and $f_2(x)$ are polynomials in $x$ of degree $N$, by construction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the above definition of $a_n$, together with&#xA;$$T_n(x_k) = \cos(n\theta_k), \qquad \theta_k = \pi(k+\tfrac12)/(N+1) $$&#xA;we can check that $f_1(x_k) = f(x_k)$, using the identity&#xA;$$ \sum_{n=0}^{N} \frac{1+[n&amp;gt;0]}{N+1} \cos(n\theta_j)\cos(n\theta_k) = [j=k]. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore $f_1(x)$ and $f_2(x)$ are (non-identically-zero) polynomials of degree $N$ that pass through the same $N+1$ points, and therefore are the same polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the form of the interpolating polynomial in terms of $a_n$ makes the relationship with the Chebyshev series of the function $f(x)$ clearer than the Lagrange interpolation form.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-03-19T18:56:32.367" LastActivityDate="2016-03-19T18:56:32.367" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="23393" PostTypeId="1" AcceptedAnswerId="23394" CreationDate="2016-03-18T19:03:35.487" Score="0" ViewCount="146" Body="&lt;p&gt;In PETSc, I need to generate a random number between 1 and 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the include &lt;code&gt;cstdlib&lt;/code&gt;, the classical trick is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(double)rand() / (double)RAND_MAX&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In PETSc, the constant &lt;code&gt;RAND_MAX&lt;/code&gt; does not seem to exist. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a work around? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though it is ugly, I plan to do something similar to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(rand() % 10000) / 10000.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19667" LastActivityDate="2016-03-18T19:35:44.090" Title="PETSc - how computing a random number between 0 and 1?" Tags="|petsc|random-number-generation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23394" PostTypeId="2" ParentId="23393" CreationDate="2016-03-18T19:35:44.090" Score="1" Body="&lt;p&gt;To elaborate somewhat on Paul's point, generating pseudo-random numbers in parallel requires some amount of care. Usually, when you generate PRNs, you seed the generator with something like the system time down to the millisecond. However, if you do this in a parallel program, you may well seed the generator with the same time on every process, giving you the same pseudo-random sequence on each processor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc does have a function &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/PetscRandomCreate.html&quot; rel=&quot;nofollow&quot;&gt;PetscRandomCreate&lt;/a&gt;, but the documentation for it even admonishes you that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...it should NOT be used for sophisticated parallel Monte Carlo methods since it will very likely not have the correct statistics across processors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can decide for yourself whether this caveat will apply to your case. Either way, it's always a good idea to use something like a &lt;a href=&quot;https://en.wikipedia.org/wiki/Q%E2%80%93Q_plot&quot; rel=&quot;nofollow&quot;&gt;QQ plot&lt;/a&gt; to check that the random numbers you're generating have the distribution that you think they do.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-03-18T19:35:44.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23395" PostTypeId="1" CreationDate="2016-03-18T19:58:36.250" Score="0" ViewCount="56" Body="&lt;p&gt;I am working on a model in COMSOL and I am usin different meshes from coarse to fine keeping the other parameters constant. And, I got different maximum velocities for all the meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I do to achieve similar maximum velocities for all the simulations?&lt;/p&gt;&#xA;" OwnerUserId="19668" LastEditorUserId="9667" LastEditDate="2016-03-19T15:32:06.273" LastActivityDate="2016-03-20T17:52:49.770" Title="Different Meshes Different Maximum velocities" Tags="|finite-element|fluid-dynamics|comsol|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23396" PostTypeId="1" CreationDate="2016-03-18T21:06:00.137" Score="2" ViewCount="88" Body="&lt;p&gt;&lt;em&gt;Note: this question is not strictly related to Matlab or any other environment (even though I would prefer that you refer to my Matlab code).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've &lt;code&gt;n = 10000&lt;/code&gt; equally spaced elements between &lt;code&gt;-pi&lt;/code&gt; and &lt;code&gt;pi&lt;/code&gt; in an array produced like this (in Matlab):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 10000;    &#xA;x = linspace(-pi, pi, n);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I create a second array &lt;code&gt;x_r&lt;/code&gt;, which is basically contains the elements of &lt;code&gt;x&lt;/code&gt; rounded to the third decimal place:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_r = chop(x, 3);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I calculate the &lt;code&gt;sin&lt;/code&gt; of all elements of &lt;code&gt;x&lt;/code&gt; and I call the resulting array &lt;code&gt;y&lt;/code&gt;. I do the same for &lt;code&gt;x_r&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y = sin(x);&#xA;y_r = sin(x_r);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I try to sum all elements of &lt;code&gt;y&lt;/code&gt; and then all elements of &lt;code&gt;y_r&lt;/code&gt;, that is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s = sum(y);&#xA;s_r = sum(y_r);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I try to print &lt;code&gt;s&lt;/code&gt; and &lt;code&gt;s_r&lt;/code&gt; with &lt;code&gt;100&lt;/code&gt; decimal digits after the dot, I obtain these huge numbers (and probably the first one is not finished):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s = -0.0000000000003170828901848210175171876477281318270923932134408573801920283585786819458007812500000000&#xA;s_r = -0.0000000000001331495677603378169351344695314764976501464843750000000000000000000000000000000000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I want to find the mean of &lt;code&gt;y&lt;/code&gt; and &lt;code&gt;y_r&lt;/code&gt;, which I save respectively in &lt;code&gt;m&lt;/code&gt; and &lt;code&gt;m_r&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m = mean(y);&#xA;m_r = mean(y_r);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I try to print &lt;code&gt;m&lt;/code&gt; and &lt;code&gt;m_r&lt;/code&gt; with 100 decimal digits, I obtain other two huge numbers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m = -0.0000000000000000317082890184820990794524483366356918770088064486473850012160369260527659207582473755&#xA;m_r = -0.0000000000000000133149567760337822432508902790402516613129923689416538035157344666004064492881298065&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is (as the title of this question indicates) which of the two operations is more robust to digit cancellation, the &lt;code&gt;sum&lt;/code&gt; or the &lt;code&gt;mean&lt;/code&gt; operation? And why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that the mean operation is &lt;strong&gt;less robust&lt;/strong&gt; to digit cancellation because it does more calculations, but I'm not sure, and I would not know if this would even be a right explanation. From the results, I'm not able to deduce anything useful honestly.&lt;/p&gt;&#xA;" OwnerDisplayName="user9925" LastActivityDate="2016-03-21T14:17:14.367" Title="Is more robust to digit cancellation the sum operator or the mean operator?" Tags="|matlab|floating-point|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23398" PostTypeId="1" AcceptedAnswerId="23401" CreationDate="2016-03-18T23:33:12.997" Score="0" ViewCount="519" Body="&lt;p&gt;&lt;strong&gt;Setup (complete, but all very standard):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is how to best calculate the cumulative integral of a function which comes out of Spectral Collocation with a chebyshev basis.  Take some function $f : [0, \bar{z}] \to \mathbb{R}$ approximated by&#xA;$$&#xA;f(z) \approx \sum_{n=0}^{N-1}d_n T_n(z),\, z \in [0, \bar{z}]&#xA;$$&#xA;where $T_n(z)$ is a basis of Chebyshev polynomials adapted to the $[0, \bar{z}]$ domain. Denote the vectors of coefficients as $d\in \mathbb{R}^N$.  Calculate the Chebyshev polynomial roots (adapted to the $[0, \bar{z}]$ domain) and define them as,&#xA;$$&#xA;\vec{z}_{\mathrm{int}} \equiv \{z_1,\ldots z_{N}\}\in \mathbb{R}^{N}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the complete set of nodes including boundary values as (with $z_0 \equiv 0$ and $z_{N+1} \equiv \bar{z}$ as $\vec{z}\equiv \{0, z_1,\ldots z_N, \bar{z}\}\in \mathbb{R}^{N+2}$&#xA;Now, define the basis matrices as&#xA;\begin{align}&#xA;B &amp;amp;\equiv \begin{bmatrix}&#xA;T_0(z_0)&amp;amp; \ldots &amp;amp; T_{N-1}(z_0)\\&#xA;\ldots &amp;amp; &amp;amp; \ldots\\&#xA;T_0(z_{N+1}) &amp;amp; \ldots &amp;amp; T_{N-1}(z_{N+1})&#xA;\end{bmatrix} \in \mathbb{R}^{(N+2)\times N}\\	&#xA;B' &amp;amp;\equiv \begin{bmatrix}&#xA;T_0'(z_0)&amp;amp; \ldots &amp;amp; T'_{N-1}(z_0)\\&#xA;\ldots &amp;amp; &amp;amp; \ldots\\&#xA;T_0'(z_{N+1}) &amp;amp; \ldots &amp;amp; T_{N-1}'(z_{N+1})&#xA;\end{bmatrix} \in \mathbb{R}^{(N+2)\times N}&#xA;\end{align}&#xA;Then, given the coefficient matrix $d$, you can find $f$ or the derivative $f'$ at every point in $\vec{z}$ with&#xA;\begin{align}&#xA;\vec{f} &amp;amp;\equiv \{f(z_n)\}_{n=0}^{N+1} = B \cdot d\in \mathbb{R}^{N+2}\\		&#xA;\vec{f}' &amp;amp;\equiv \{f'(z_n)\}_{n=0}^{N+1} = B' \cdot d\in \mathbb{R}^{N+2}\\		&#xA;\end{align}&#xA;with the function at the interior nodes as $\vec{f}_{\mathrm{int}} \equiv \vec{f}(1:N) \in \mathbb{R}^N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we can find the weighting vector for Chebyshev-Gauss quadrature (&lt;a href=&quot;https://en.wikipedia.org/wiki/Gaussian_quadrature&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Gaussian_quadrature&lt;/a&gt;) on the exact same roots $\vec{z}_{\mathrm{int}}$, and call it $\omega \in \mathbb{R}^N$.  With this, we can approximate integrals for some $g(z)$ with $g(\vec{z}_{\mathrm{int}}) \equiv \{g(z) | z \in \vec{z}_{\mathrm{int}}\}$&#xA;$$&#xA;\int_0^{\bar{z}} g(z) d z \approx \omega \cdot g(\vec{z}_{\mathrm{int}})&#xA;$$&#xA;(note that this quadrature scheme does not use the endpoints).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Problem:&lt;/strong&gt;&#xA;Define the cumulative function,&#xA;$$&#xA;F(z) \equiv \int_0^z f(\tilde{z}) d\tilde{z}&#xA;$$&#xA;If all I cared about was $F(\bar{z})$, then I have a nice approximation,&#xA;$$&#xA;F(\bar{z}) \approx \omega \cdot \vec{f}_{\mathrm{int}}&#xA;$$&#xA;But what an approximation for the cumulative integral, $F(z)$ for all $z \in \vec{z}$ given only the above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Current Solution:&lt;/strong&gt;&#xA;Note that as I am not able to evaluate $f(z)$ at other points, I cannot naively use Gauss-Chebyshev quadrature as the quadrature roots depend on the domain of integration.  My, grossly imprecise, method is to use trapezoidal integration at the unevenly spaced chebyshev roots.  Or,&#xA;$$&#xA;\Delta \vec{z} = \{\vec{z}(n) - \vec{z}(n-1)\}_{n=1}^{N+1}&#xA;$$&#xA;Then, to calculate the integral to one of the nodes,&#xA;$$&#xA;\int_0^{z_{n-1}}f(\hat{z}) d \hat{z} \approx \frac{1}{2}\Delta \vec{z}(0:n) \cdot (\vec{f}(0:n-1) + \vec{f}(1:n))&#xA;$$&#xA;I can even get fancier and calculate this all in one step,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\Omega \equiv \frac{1}{2}\begin{bmatrix}0 &amp;amp;0 &amp;amp; 0 &amp;amp;0 &amp;amp; \ldots &amp;amp; 0 \\&#xA;\Delta \vec{z}(0:1) &amp;amp; &amp;amp; 0 &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0 \\&#xA;\Delta \vec{z}(0:2) &amp;amp; &amp;amp;  &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0 \\&#xA;\ldots &amp;amp; &amp;amp;  &amp;amp;   &amp;amp; &amp;amp; \ldots \\&#xA;\Delta \vec{z}(0:N)&amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp;&#xA;\end{bmatrix}\in \mathbb{R}^{(N+2)\times (N+1)}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which gives the complete set of integrals as,&#xA;$$&#xA;\vec{F} =\Omega \cdot (\vec{f}(0:N) + \vec{f}(1:N+1))\in\mathbb{R}^{N+2}&#xA;$$&#xA;This is especially useful for me because I am solving a spectral collocation method, so having auto-differentiation within the calculation of the residual makes the problem solvable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there better approaches?&lt;/strong&gt;&#xA;I am hoping that there is a more precise way of calculating these partial integrals.  For example, I found &lt;a href=&quot;https://math.stackexchange.com/questions/344073/integrating-non-uniform-grid-data-from-an-accelerometer&quot;&gt;https://math.stackexchange.com/questions/344073/integrating-non-uniform-grid-data-from-an-accelerometer&lt;/a&gt; which gives some ideas for a non-uniform Simpson's Rule, but it seems intractable to get a quadratic-form anything like my $\Omega$ above.  Has anyone done that work, or is there a method I am forgetting about?&lt;/p&gt;&#xA;" OwnerUserId="9692" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2024-01-23T09:55:45.097" Title="Calculating integrals for a function approximated by Chebyshev polynomials" Tags="|quadrature|collocation|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23399" PostTypeId="2" ParentId="23387" CreationDate="2016-03-18T23:38:59.867" Score="2" Body="&lt;p&gt;The trick is to view your equation as the first-order optimality conditions of an unconstrained optimization problem $\min_x f(x)$. Recall that the first-order optimality conditions are just $\nabla f(x) = 0$. So in your case, you only need to set&#xA;$$&#xA; f(x) = -b^T x + \tfrac{1}{2} x^T A x + \sum_{i=1}^n C_{ii} e^{x_i},&#xA;$$&#xA;where I'm assuming $A$ is symmetric (you didn't specify) and $C_{ii}$ is the $i$-th diagonal element of $C$.&lt;/p&gt;&#xA;" OwnerUserId="324" LastActivityDate="2016-03-18T23:38:59.867" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23401" PostTypeId="2" ParentId="23398" CreationDate="2016-03-19T08:30:44.017" Score="5" Body="&lt;p&gt;If you know the Chebyshev expansion for $f(z)$, why don't you formally integrate the polynomials using the recurrence relation for Chebyshev polynomials ? The Clenshaw-Curtis method is based on this approach (combined with an intelligent use of the FFT). &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2016-03-19T08:30:44.017" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23402" PostTypeId="1" CreationDate="2016-03-19T08:56:23.710" Score="2" ViewCount="347" Body="&lt;p&gt;The solvepde function was introduced in MATLAB R2016a. I am able to solve my system of PDEs if there are no time delays involved. Does anyone know how to include time delays in the solvepde function?&lt;/p&gt;&#xA;" OwnerUserId="19673" LastActivityDate="2016-03-22T17:31:30.757" Title="How to use time delays in the solvepde function in MATLAB for a system of PDEs?" Tags="|pde|nonlinear-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23403" PostTypeId="1" AcceptedAnswerId="31433" CreationDate="2016-03-19T11:54:33.807" Score="6" ViewCount="228" Body="&lt;p&gt;I need to compute the &lt;a href=&quot;https://en.wikipedia.org/wiki/Two-sided_Laplace_transform&quot; rel=&quot;nofollow&quot;&gt;two-sided (bilateral) Laplace transform&lt;/a&gt; of a &lt;em&gt;numerically&lt;/em&gt; given function $F$,&#xA;$$&#xA;I(t) = \int_{-\infty}^{+\infty} {dx} \, e^{-x} \, F(x + t) ~,&#xA;$$&#xA;where $F(x)$ has some sharp features, e.g., at $ x = \{ 0, x_p, \cdots \} $, has finite support, and vanishes sufficiently fast for $ x &amp;lt; 0 $ to prevent a divergence of the integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the sharp features, I have to divide the integration region to some subregions, for instance,&#xA;\begin{align}&#xA;I(t) &amp;amp;= \int_{-\infty}^{-\Delta} {dx} \, e^{-x} \, F(x + t)&#xA; + \int_{-\Delta}^{+\Delta} {dx} \, e^{-x} \, F(x + t) \\&#xA;&amp;amp;\quad + \int_{+\Delta}^{x_p - \Delta} {dx} \, e^{-x} \, F(x + t)&#xA; + \int_{x_p - \Delta}^{x_p + \Delta} {dx} \, e^{-x} \, F(x + t) \\&#xA;&amp;amp;\quad + \int_{x_p + \Delta}^{+\infty} {dx} \, e^{-x} \, F(x + t) ~,&#xA;\end{align}&#xA;where $\Delta &amp;gt; 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to know the best and most precise way to perform such an integral numerically (quadrature methods). Is there something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Gauss%E2%80%93Laguerre_quadrature&quot; rel=&quot;nofollow&quot;&gt;Gauss-Laguerre quadrature&lt;/a&gt; for such an integration?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also considered a transformation of the integration variable,&#xA;$$&#xA;u = e^{-x} ~,&#xA;$$&#xA;to absorb the exponential factor,&#xA;$$&#xA;I(t) = \int_{0^+ = \, u(x \rightarrow +\infty)}^{+\infty = \, u(x \rightarrow -\infty)} {du} \, F(-\ln(u) + t) ~,&#xA;$$&#xA;and performing the transformed integration by a &lt;a href=&quot;https://en.wikipedia.org/wiki/Tanh-sinh_quadrature&quot; rel=&quot;nofollow&quot;&gt;Tanh-Sinh quadrature&lt;/a&gt;; yet I am not sure if this is the best method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, one can take&#xA;$$&#xA;f(x,\omega) = e^{-\alpha (x - \omega)} \, \Theta(x - \omega)&#xA;$$&#xA;where $\Theta$ denotes the Heaviside step function and $\alpha &amp;gt; 0$. Hence, the ‘width’ of the peak at $\omega$ can be varied by $\alpha$.&#xA;The two-sided Laplace transform of $f$ will be&#xA;$$&#xA;\mathcal{L}[f](\omega) := \int_{-\infty}^{+\infty} dx \, e^{-x} \, f(x, \omega) = \frac{ e^{-\omega} }{1 + \alpha} ~.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="15235" LastEditorUserId="15235" LastEditDate="2016-03-24T17:59:41.823" LastActivityDate="2019-05-15T16:03:15.667" Title="Numerical computation of two-sided (bilateral) Laplace transform" Tags="|quadrature|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23404" PostTypeId="1" CreationDate="2016-03-19T16:34:33.160" Score="1" ViewCount="269" Body="&lt;p&gt;I have a one-dimensional set of points, i.e. $(n,y_n), 1\leq n \leq N$. I want to fit them with a linear combination of $k$ &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectangular_function&quot; rel=&quot;nofollow&quot;&gt;rectangular functions&lt;/a&gt; in a least-squared-error sense. Each rectangle is parametrised by a left edge, right edge, and height. It is similar to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lebesgue_integration&quot; rel=&quot;nofollow&quot;&gt;Lebesgue sum&lt;/a&gt; but I am looking at horizontal strips of flexible width and height to achieve minimum error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to be a really straightforward and very linear problem, but I don't see this facility in MATLAB curve fitting toolbox, and the internet is not helping at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sure (I think) I can formulate the equations and write the code, it might take me a couple of hours. But should I do it? Is there some reason it's not widely used? Is it actually ill-defined (in the sense there is no optimum)?&lt;/p&gt;&#xA;" OwnerUserId="3740" LastEditorUserId="3740" LastEditDate="2016-03-29T15:04:28.877" LastActivityDate="2016-04-05T09:56:35.723" Title="Fit curve with rectangles" Tags="|optimization|interpolation|curve-fitting|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="23405" PostTypeId="1" CreationDate="2016-03-19T19:35:01.267" Score="1" ViewCount="481" Body="&lt;p&gt;I have a simple question that I wasn't able to find an answer to. I have a function &lt;code&gt;f(x)&lt;/code&gt; where &lt;code&gt;x&lt;/code&gt; is some vector, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function f=f(x)&#xA;    f=x.^4;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I want to integrate with bounds as vectors:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=1:0.1:10;&#xA;x_0=2*ones(size(x));&#xA;g=integral(@f,x_0,x);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I want to get &lt;code&gt;g&lt;/code&gt; as a vector &lt;code&gt;g(i)=integral(@f,x_0(i),x(i))&lt;/code&gt;. However, this will return an error that the bounds have to be scalars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously I can use a loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=1:0.1:10;&#xA;x_0=2*ones(size(x));&#xA;for i=1:length(x)&#xA;    g(i)=integral(@f,x_0(i),x(i));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but that can be very inefficient if I have big vectors that are repeatedly integrated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simple efficient way to do this vectorized without a loop?&lt;/p&gt;&#xA;" OwnerUserId="19617" LastEditorUserId="4474" LastEditDate="2016-03-21T18:18:57.950" LastActivityDate="2016-06-19T22:31:09.320" Title="Matlab integral with vectorized bounds, without using a loop" Tags="|matlab|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23407" PostTypeId="1" AcceptedAnswerId="23409" CreationDate="2016-03-19T21:40:24.233" Score="1" ViewCount="59" Body="&lt;p&gt;I would like to fit a diffusion coefficient as function of e.g. salt, pH etc. If I use a linear combination of all variables, I will have to apply constraints because the model fails, if the diffusion coefficient will get too large or negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there exists a function that behaves similar to the logistic function (asymptotic between 0 and a certain limit) but instead it asymptotically reaches -1 and +1. In this way I would avoid constraints and bounds. At the same time, negative parameters would mean that the corresponding variables do decrease diffusion and vice versa.&lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2016-03-19T22:53:36.660" Title="limiting function as cost function: logistic function between -a and +a" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23409" PostTypeId="2" ParentId="23407" CreationDate="2016-03-19T22:53:36.660" Score="4" Body="&lt;p&gt;Tangens Hyperbolicus behaves as you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/math/a/6/8/a682968c1934c1bbc2799a2d6c963921.png&quot; alt=&quot;Tanh formula&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/8/87/Hyperbolic_Tangent.svg/490px-Hyperbolic_Tangent.svg.png&quot; alt=&quot;Tanh Graph&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="18622" LastActivityDate="2016-03-19T22:53:36.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23410" PostTypeId="1" CreationDate="2016-03-20T01:39:31.780" Score="2" ViewCount="2683" Body="&lt;p&gt;I want to write a MATLAB code for Roe solver for 1D shallow water equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anyone who can help me with that?&lt;/p&gt;&#xA;" OwnerUserId="19679" LastEditorUserId="9667" LastEditDate="2016-03-20T03:53:10.510" LastActivityDate="2016-03-25T04:02:18.330" Title="Code for shallow water equation" Tags="|matlab|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23412" PostTypeId="2" ParentId="23389" CreationDate="2016-03-20T06:23:34.707" Score="2" Body="&lt;p&gt;Thanks for Kirill's detailed answer, which clarifies all the confusion in my head. According to Kirill's answer and the materials he provided, now I want to generalize it a bit to common cases.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us suppose $\{F_n(x)\}$ is a set of orthogonal polynomials on $[-1,1]$, i.e.,&lt;br&gt;&#xA;\begin{equation}&#xA;\int_{-1}^1 F_m(x)F_n(x)w(x)dx=g_m\delta_{mn},&#xA;\end{equation}&#xA;and $f(x)$ is a continuous function we want to approximate in [-1,1].  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(1)&lt;/strong&gt; Using $\{F_n(x)\}$ as basis functions, we get&#xA;\begin{equation}&#xA;f(x)=\sum_{n=0}^\infty a_nF_n(x),&#xA;\end{equation}&#xA;we approximate it with a truncated version,&#xA;\begin{equation}&#xA;f(x)=\sum_{n=0}^N a_nF_n(x),&#xA;\end{equation}&#xA;where $a_n$ can be computed from&#xA;\begin{equation}&#xA;a_n=\frac{1}{g_n}\int_{-1}^1 f(x)F_n(x)w(x)dx.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(2)&lt;/strong&gt; We use the the $(n+1)$ roots of $F_{N+1}(x)$ to interpolate it:&#xA;\begin{equation}&#xA;f(x)\simeq P_N(x)=\sum_{n=0}^N a_nL_n(x).&#xA;\end{equation}&#xA;Here, since $P_N(x)$ is an N-th degree polynomial, it can also be expressed using $F_n(x), n=1,2,...,N$, because $\{F_n(x)\}_{n=0}^N$ is a base for the polynomial sapce $P_l, l\leq N$, so we get:&#xA;\begin{equation}&#xA;f(x)\simeq P_N(x)=\sum_{n=0}^N c_n F_n(x).&#xA;\end{equation}&#xA;also, $P_N(x_k)=f(x_k)$ at the (N+1) interpolation points, which means&#xA;\begin{equation}&#xA;f(x_k)=\sum_{n=0}^N c_n F_n(x_k).&#xA;\end{equation}&#xA;multiply this equation by $w_k F_m(x_k)$ on both sides and compute the sum on $x_k$,&#xA;\begin{equation}&#xA;\sum \limits_{k=0}^N f(x_k)F_m(x_k) w_k=\sum \limits_{k=0}^N w_k F_m(x_k) \sum_{n=0}^N c_n F_n(x_k)=\sum_{n=0}^N c_n \sum \limits_{k=0}^N F_m(x_k) F_n(x_k) w_k.&#xA;\end{equation}&#xA;Since $(m+n)\leq 2N$, $F_m(x)F_n(x)$ is a polynomial of degree less than or equal to 2N. From the principle of Gauss quadrature, the following equation holds exactly:&#xA;\begin{equation}&#xA;\int_{-1}^1 F_m(x)F_n(x)w(x)dx=\sum \limits_{k=0}^N F_m(x_k) F_n(x_k) w_k=g_m\delta_{mn},&#xA;\end{equation}&#xA;Therefore,&#xA;\begin{equation}&#xA;\sum \limits_{k=0}^N f(x_k)F_m(x_k) w_k=\sum_{n=0}^N c_n g_m\delta_{mn}=c_n g_n,&#xA;\end{equation}&#xA;thus,&#xA;\begin{equation}&#xA;c_n=\frac{1}{g_n}\sum \limits_{k=0}^N f(x_k)F_m(x_k) w_k,&#xA;\end{equation}&#xA;which, as Kirill has stated in the answer, is just a rectangular rule approximation of the intergral $a_n$ listed above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In conclusion&lt;/strong&gt;, the two forms to approximate $f(x)$ are almost the same. Again, thanks to Kirill's answer.&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="18224" LastEditDate="2016-04-29T04:56:11.603" LastActivityDate="2016-04-29T04:56:11.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23414" PostTypeId="2" ParentId="23398" CreationDate="2016-03-20T09:31:24.227" Score="0" Body="&lt;p&gt;There seems to be a problem with your setup. Chebyshev polynomials are defined on the interval [-1,1], not on a general domain as you have specified. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you need to scale your domain to [-1,1] to be able to use collocation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other problem of doing a general quadrature could simple be broken down to doing the quadrature over the domain [0,z] scaled to [-1,1]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you want to fix the number of points, then you have to basically use some kind of further approximation. One way would be to break down the domain into smaller sub-domains, scale each one to [-1, 1], do the quadrature to the nearest point to z, and then do an approximate quadrature for the remaining part. &lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2016-03-20T09:31:24.227" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23415" PostTypeId="1" AcceptedAnswerId="23500" CreationDate="2016-03-20T10:14:32.293" Score="2" ViewCount="289" Body="&lt;p&gt;I am going to calculate a trajectory, by using a pre-calculated vector field. The values of the field are known on a grid which is quadratic in the horizontal direction, un-evenly spaced in the vertical, and evenly spaced in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm planning to use a 4th order Runge-Kutta integrator, and my question is if I should simply select my time step such that the three different times in the Runge-Kutta calculation exactly matches three consecutive time steps in the data, or if I will gain anything by implementing some kind of interpolation scheme to allow using a shorter time step in the integrator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The spatial and temporal resolution of the pre-calculated dataset is such that the trajectory will seldom move further than the spatial step during one time step.&lt;/p&gt;&#xA;" OwnerUserId="19544" LastActivityDate="2016-03-29T11:23:23.067" Title="Does it have an effect to interpolate data before using Runge-Kutta?" Tags="|interpolation|runge-kutta|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23416" PostTypeId="2" ParentId="23415" CreationDate="2016-03-20T11:55:01.440" Score="2" Body="&lt;p&gt;Basically &lt;a href=&quot;http://www.scholarpedia.org/article/Runge-Kutta&quot; rel=&quot;nofollow&quot;&gt;what RK does&lt;/a&gt; can be seen as some sort of interpolation.&#xA;As you can see on Figure 1, the advantage of RK compared to Euler is its greater stability region so I think you should run a series of tests to check whether you are indeed inside the stability region.&#xA;If you chose to interpolate first on your own, then you are going to introduce additional correlations that should not be there... I think it would be better to reduce the timestep in the computation of the vector field if you can.&lt;/p&gt;&#xA;" OwnerUserId="19678" LastActivityDate="2016-03-20T11:55:01.440" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23417" PostTypeId="1" CreationDate="2016-03-20T16:39:08.693" Score="1" ViewCount="192" Body="&lt;p&gt;Recently, I took a course on numerical methods where we learned about Galerkin's Method of Weighted Residuals. We were told that it forms the basis of the Finite Element Method .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in most introductory texts I've seen on FEM, they start with a simple example where they manually write out the equations to minimize the potential energy at each node. For instance, see this &lt;a href=&quot;http://www.msm.cam.ac.uk/phase-trans/mphil/MP7-2.pdf&quot; rel=&quot;nofollow&quot;&gt;example of a system of springs&lt;/a&gt;. There are no test functions, so it doesn't seem to be the MWR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a name for this other method?&lt;/li&gt;&#xA;&lt;li&gt;What are the practical and numerical differences between it and the Galerkin Method of Weighted Residuals? (e.g., is it easier to set up for certain problems? Does it have better numerical precision or stability?)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19689" LastActivityDate="2016-03-20T16:39:08.693" Title="Is there a difference between the Galerkin MWR and other techniques called Finite Element Method" Tags="|finite-element|" AnswerCount="0" CommentCount="9" ClosedDate="2016-03-22T20:02:21.970" ContentLicense="CC BY-SA 3.0" />
  <row Id="23418" PostTypeId="2" ParentId="23395" CreationDate="2016-03-20T17:52:49.770" Score="1" Body="&lt;p&gt;The finite element methods, i.e., what COMSOL uses internally, does not &lt;em&gt;solve&lt;/em&gt; the problem exactly. It only (i) provides an &lt;em&gt;approximation&lt;/em&gt; to the solution, and (ii) does so in a way for which we can show that the approximation &lt;em&gt;converges&lt;/em&gt; to the exact solution of the problem as mesh becomes finer and finer. This typically implies that on meshes that are &lt;em&gt;too coarse&lt;/em&gt; the difference between the approximation and the exact solution will be large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convergence, however, also implies that as you make your mesh finer and finer, the difference will become smaller and smaller. In other words, as long as you see significant differences between the solution on one mesh and the next, you can expect that both are inaccurate and unreliable. On the other hand, if you see no significant differences on two or more finer and finer meshes, you may hope that the approximation is reasonably close to the exact solution and can be trusted to a certain degree.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-03-20T17:52:49.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23420" PostTypeId="1" CreationDate="2016-03-20T18:45:43.450" Score="1" ViewCount="594" Body="&lt;p&gt;How can I make a scatter plot like the following, where the density of points represents the magnitude of the scalar field at that point. Consider for example the probability density for an atom:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/81WCb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/81WCb.jpg&quot; alt=&quot;Example plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I managed to plot something like this as a color plot (see my question and the pictures here: &lt;a href=&quot;https://stackoverflow.com/questions/36113331/pcolormesh-in-polar-coordinates&quot;&gt;https://stackoverflow.com/questions/36113331/pcolormesh-in-polar-coordinates&lt;/a&gt;) but I didn't find any option to make such a scatter plot as above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A requirement is that it should be done with free software for linux. Preferences:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pgfplots or pstricks&lt;/li&gt;&#xA;&lt;li&gt;mathplotlib or another python library&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am interested in such types of plots in 2d as well as in 3d.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-03-21T03:14:44.787" Title="How to make scatter plots of scalar fields with free software?" Tags="|python|plotting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23421" PostTypeId="2" ParentId="23420" CreationDate="2016-03-21T03:14:44.787" Score="3" Body="&lt;p&gt;Points for a scatterplot of a probability density function can simply be sampled &#xA;from it. I used the &lt;code&gt;Psi(X1, X2)&lt;/code&gt; you defined in your linked post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of sample points, alpha and the marker type can be tuned. The axis can be renormalized to the dimensions of th meshgrid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/49wc4.png&quot; alt=&quot;Scatter plot of crossreferenced field&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code below I take the absolute value of the input matrix because it had negative values. You might want to set negative values to 0 instead and add a different colored scatterplot to the same axis if you have positive and negative data. &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pylab as plt&#xA;def sample(mat, count=10000):&#xA;    shape = mat.shape&#xA;    magn = np.abs(mat) # input should not be negative&#xA;    cumsum = magn.cumsum()&#xA;&#xA;    samples = [None] * count&#xA;    for i, rand in enumerate(np.random.random(count) * magn.sum()):&#xA;        idx = np.argmax(cumsum &amp;gt; rand)&#xA;        samples[i] = (idx % shape[0], idx / shape[1])&#xA;&#xA;    return samples&#xA;&#xA;sample_x, sample_y = zip(*sample(psi(X1,X2)))&#xA;plt.figure()&#xA;plt.scatter(sample_x, sample_y, alpha=0.25, marker=&quot;.&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18622" LastActivityDate="2016-03-21T03:14:44.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23423" PostTypeId="1" CreationDate="2016-03-21T10:02:35.833" Score="5" ViewCount="1285" Body="&lt;p&gt;In elementary chemistry textbooks you often have pictures like the following one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iB73G.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iB73G.png&quot; alt=&quot;Orbitals&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any conventions how to get them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure, but I guess that it are contour plots with only one iso-surface of constant probability density $|\psi|^2$ such that the (integrated) probability for an electron to be inside of on the blue volumes, is 90 %.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, give those conditions a unique result and how to solve it numerically? Maybe you could add an example using python mayavi or something like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that my question is not of how to visualize the probability density via colormap or scatter plots, that's conceptually clear.&lt;/p&gt;&#xA;" OwnerUserId="11911" LastActivityDate="2019-02-19T23:52:52.957" Title="What determines the usual chemistry textbook plots of atom orbitals?" Tags="|python|computational-physics|computational-chemistry|plotting|vtk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23424" PostTypeId="2" ParentId="23423" CreationDate="2016-03-21T10:45:05.640" Score="6" Body="&lt;p&gt;Your intuition is right, for example in &lt;a href=&quot;https://de.wikipedia.org/wiki/Atomorbital#/media/File:AOs-3D-dots.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;3D Orbitals (German Wikipedia)&lt;/a&gt; the caption explicitly states that 90% iso-surfaces are used. I have however seen different percentages before where the results look similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did you check the &lt;a href=&quot;http://docs.enthought.com/mayavi/mayavi/auto/example_atomic_orbital.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mayavi Example Atomic Orbital&lt;/a&gt;? If you remove the phase-coloring and find the additional parameter to &lt;code&gt;contour&lt;/code&gt; that sets the cutoff, it produces the textbook plots.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gx1sg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gx1sg.jpg&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;&lt;sub&gt;(source: &lt;a href=&quot;http://docs.enthought.com/mayavi/mayavi/_images/example_atomic_orbital.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;enthought.com&lt;/a&gt;)&lt;/sub&gt;  &lt;/p&gt;&#xA;" OwnerUserId="18622" LastEditorUserId="22284" LastEditDate="2019-02-19T23:52:52.957" LastActivityDate="2019-02-19T23:52:52.957" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="23426" PostTypeId="2" ParentId="23396" CreationDate="2016-03-21T14:17:14.367" Score="1" Body="&lt;p&gt;The first thing to realize is that reality doesn't work in decimal. And &lt;code&gt;sin(x)&lt;/code&gt; in particular has no respect for decimals - it works in radians. So when you truncate &lt;code&gt;x_r&lt;/code&gt; by up to &lt;code&gt;0.001&lt;/code&gt;, that really is 6.36619772368E-4'th of a right angle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I then see that &lt;code&gt;y_r&lt;/code&gt; has so many trailing zeroes, I assume there is a massive loss of precision. This probably is because MATLAB has to revert to direct numerical approximation with finite precision, while the non-rounded version allows a more direct formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means digit cancellation is just straight-out irrelevant in the second case.&lt;/p&gt;&#xA;" OwnerUserId="19697" LastActivityDate="2016-03-21T14:17:14.367" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23428" PostTypeId="2" ParentId="23385" CreationDate="2016-03-21T15:06:49.543" Score="0" Body="&lt;p&gt;Taking this as my original question: Do we know if there exists a RHS and an initial (unlucky) guess that will require $\Theta(\sqrt{\kappa})$&#xA;steps?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer to the question is &quot;no&quot;.  The idea of this answer comes from the comment from Guido Kanschat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Claim:  For any given condition number $k$, there exists a matrix $A$, with that condition number for which the CG algorithm will terminate in at most two steps (for any given RHS and initial guess).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider $A\in \mathbb{R}^{n\times n} $ where $A=\mathrm{diag}(1,\kappa,\kappa,\ldots, \kappa)$.  Then the condition number of $A$ is $\kappa$.  Let $b\in \mathbb{R}^n$ be the RHS, and denote the eigenvalues of $A$ as $\lambda_i$ where&#xA;$$\lambda_i = \left\{\begin{array}{ll}1 &amp;amp; i=1\\&#xA;\kappa &amp;amp; i\not= 1&#xA; \end{array} \right. .&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We first consider the case where $x^{(0)} \in \mathbb{R}^n$, the initial guess, is zero.  Denote $x^{(2)}\in \mathbb{R}^n$ as the second estimate of $A^{-1}b$ from the CG algorithm.  We show that $x^{(2)} =A^{-1}b$ by showing $\langle x^{(2)}-A^{-1}b, A(x^{(2)}-A^{-1}b)\rangle =0$.  Indeed, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA; \langle x^{(2)}-A^{-1}b, A(x^{(2)}-A^{-1}b)\rangle &amp;amp;=&#xA; \left\| x^{(2)}-A^{-1}b \right\|_A^2 \\&#xA;&amp;amp;=\min_{p\in \mathrm{poly}_{1} } \left\| (p(A)-A^{-1}) b \right\|_A^2\\&#xA;&amp;amp;=\min_{p\in \mathrm{poly}_{1} } \sum_{i=1}^n (p(\lambda_i) - \lambda_i^{-1})^2 \lambda_i b_i^2 \\&#xA;&amp;amp;\le \sum_{i=1}^n (\widehat{p}(\lambda_i) - \lambda_i^{-1})^2 \lambda_i b_i^2&#xA;= 0&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where we use the first order polynomial $\widehat{p}$ defined as $\widehat{p}(x)= (1+\kappa-x)/\kappa$.  So we proven the case for $x^{(0)}= 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $x^{(0)} \not = 0$, then $x^{(2)}= \overline{x^{(2)}}+ x^{(0)}$ where $\overline{x^{(2)} }$ is the second estimate of the CG algorithm with $b$ replaced with $\overline{b} = b-A x^{(0)}$.  So we have reduced this case to the previous one.  &lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2016-03-21T17:19:38.597" LastActivityDate="2016-03-21T17:19:38.597" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23430" PostTypeId="2" ParentId="23410" CreationDate="2016-03-21T17:47:35.753" Score="3" Body="&lt;p&gt;You can have a look at Randall J LeVeque's &lt;a href=&quot;http://www.cambridge.org/nl/academic/subjects/mathematics/numerical-analysis/finite-volume-methods-hyperbolic-problems&quot; rel=&quot;nofollow&quot;&gt;book&lt;/a&gt; on Finite Volume Methods for Hyperbolic Problems. He provides you with the necessary equations and algorithms. You can use this to build your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need additional help, you can also have a look at &lt;a href=&quot;http://www.clawpack.org/book.html&quot; rel=&quot;nofollow&quot;&gt;CLAWPACK&lt;/a&gt;, his own set of codes that solve a variety of example problems which include Shallow water equations. I believe you should be able to find what you are looking for, in his examples. From there on, you just have to convert the code to MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="16601" LastActivityDate="2016-03-21T17:47:35.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23431" PostTypeId="2" ParentId="23405" CreationDate="2016-03-21T21:15:17.347" Score="1" Body="&lt;p&gt;I think &lt;a href=&quot;http://ch.mathworks.com/help/matlab/ref/arrayfun.html&quot; rel=&quot;nofollow&quot;&gt;arrayfun&lt;/a&gt; is what you are looking for: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arrayfun(@(x_1, x_2) integral(@(y) y.^4, x_1, x_2), x_0, x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18916" LastActivityDate="2016-03-21T21:15:17.347" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23436" PostTypeId="1" CreationDate="2016-03-22T15:32:00.330" Score="1" ViewCount="50" Body="&lt;p&gt;I am trying to solve Travelling Salesman Problems using tabu search. I have been able to successfully find &quot;near enough&quot; optimal solutions (as well as one optimal, yay!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the moment I am using sub-tour reversal as the local search precedure. The basic gist of it is similar to 2-opt, but rather than swapping two cities, it reverses the subtour between and including those two cities. I.e if we have a tour [1, 2, 3, 4, 5, 6, 1] and we swap city 2 and 5 we get [1, 5, 4, 3, 2, 6, 1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I am using two &quot;for&quot; loops to find the best neighborhood from the initial tour. But as you know, going through every single swap is a very time consuming process, which limits me to TSP problems of less than 500 cities. Anything above 1000 cities will take my program hours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, can I apply some other method than two or loops to find the best neighborhood in more efficient (fast)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now my &quot;for&quot; loops look like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int col = 1;&#xA;for (int i = 1; i &amp;lt; initialSolution.length-1; i++) {&#xA;    for (int j = col; j &amp;lt; initialSolution.length-1; j++) {&#xA;        if (i == j) {&#xA;            continue;&#xA;        }&#xA;        if ((i == 1) &amp;amp;&amp;amp; (j == initialSolution.length-2)) {&#xA;            continue;&#xA;        }&#xA;        if ((j == 1) &amp;amp;&amp;amp; (i == initialSolution.length-2)) {&#xA;            continue;&#xA;        }&#xA;        // LOCAL SEARCH PRECEDURE&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;such that I do not repeat previously reversed tours, and such that I do not reverse the whole tour.&lt;/p&gt;&#xA;" OwnerUserId="19710" LastEditorUserId="19710" LastEditDate="2016-03-22T16:45:05.267" LastActivityDate="2016-03-22T16:45:05.267" Title="Alternative to two &quot;for&quot; loops in finding best neighborhoods for TSP?" Tags="|nonlinear-programming|approximation|graph-theory|java|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23438" PostTypeId="2" ParentId="23410" CreationDate="2016-03-22T16:08:16.843" Score="2" Body="&lt;p&gt;There is a project named FullSWOF which solves Shallow Water Equations with many solvers. Unfortunately it is written by C++. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.univ-orleans.fr/mapmo/soft/FullSWOF/&quot; rel=&quot;nofollow&quot;&gt;http://www.univ-orleans.fr/mapmo/soft/FullSWOF/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can download the project here &lt;a href=&quot;http://4share.vn/f/2b1f131b131c1b1a/FullSWOF_1D-1.02.01_win.zip&quot; rel=&quot;nofollow&quot;&gt;http://4share.vn/f/2b1f131b131c1b1a/FullSWOF_1D-1.02.01_win.zip&lt;/a&gt; This is my copied zip file of the project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also read my work here &lt;a href=&quot;https://www.researchgate.net/publication/292617263_Numerical_methods_for_Shallow_water_equations&quot; rel=&quot;nofollow&quot;&gt;https://www.researchgate.net/publication/292617263_Numerical_methods_for_Shallow_water_equations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that report I wrote MATLAB codes for Shallow Water Equations with Rusanov, HLL and MUSCL-Hancock solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="7472" LastEditorUserId="7472" LastEditDate="2016-03-25T04:02:18.330" LastActivityDate="2016-03-25T04:02:18.330" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23439" PostTypeId="2" ParentId="23415" CreationDate="2016-03-22T17:03:55.983" Score="1" Body="&lt;p&gt;It would likely be best to not use the interpolation since I assume you'd default to cubic splines which is $\mathcal{O}(h^3)$ whereas RK4 is $\mathcal{O}(h^4)$. You'd have to do an error analysis to see if this would decrease your accuracy, and I think it's likely that it would. That said, if you do a really high order interpolation, then you're fine. To know how high of an order you need to do, you really have to look at the equation, put in the error term, check the expansion, and see where it ends up.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-03-22T17:03:55.983" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23440" PostTypeId="2" ParentId="23402" CreationDate="2016-03-22T17:13:32.690" Score="1" Body="&lt;p&gt;If you have constant time delays, you can introduce new variables for the delayed variables. For example, you can let $y = x(t-\tau)$. Then by the chain rule you get $\dot{y} = \dot{x}(t-\tau)$... so you end up with a lot of extra variables, but it will work if you aren't changing your timesteps around and make them a multiple / divisor of $\tau$ (if you have multiple $\tau$, then it has to be a common divisor or common multiple). If you set a good initial condition on $y$ you know have a system of non-delay equations which it can solve with chosen fixed timesteps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your delay equations are more complex, then the solver needs to take this into account. solvepde does not do that, so I would not recommend it for this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Almost forgot to note, if you use a divisor of $\tau$, then you need $\tau / h$ new variables in order to always hit one you know... this is easier to implement as an array that shifts everything down 1 rather than with a whole bunch of variables in a standard solver like solvepde, but YMMV)&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2016-03-22T17:31:30.757" LastActivityDate="2016-03-22T17:31:30.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23441" PostTypeId="2" ParentId="10922" CreationDate="2016-03-22T18:38:19.823" Score="8" Body="&lt;p&gt;From my experience Julia isn't ready for (scientific) everyday use yet (I'm talking about the stabilized version 0.4 of march 2016). The language itself is nice, feature rich and consistent; a refreshing step forward from both matlab and python. There are certainly uses cases where Julia is a good choice even at this early stage. But if you need reliable and mature scientific libraries I cannot recommend making the move now. The main problems I encountered are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The packages outside of Julia's core aren't reliable. They break with updates, change, get replaced, sometimes are incomplete or simply broken.&lt;/li&gt;&#xA;&lt;li&gt;Julia's parallelism features (imo the most promising potential matlab killer features) are are immature. You will encounter segmentation faults, memory leaks, crashes and disappointing performance. Sometimes they don't play well with other parts of julia, for example some of the solvers for linear systems or packages outside of the core. While these features sound promising, they often enough failed for me. Almost never was it enough to simply write &lt;code&gt;@parallel&lt;/code&gt; in front of the for loop, where in theory it should.&lt;/li&gt;&#xA;&lt;li&gt;Many little things, small bugs and issues one could live with: not so nice and sometimes erroneous call stack traces, a little buggy interpreter history, slow loading of packages, bad performance of one or another package/function, etc. What annoyed me the most is the PyPlot package, a wrapper for matplotlib, currently there's no alternative to it. It's really great that it's there and it mostly works. But if you need to plot data, be prepared for very slow performance and problems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This will all get better. I am confident that some day julia will be superior to matlab and python in almost every aspect. Chances are great that innovative packages will be developed for it. It seems like there already is a big community. Even now there is a wealth of packages with use cases ranging from opencl to webservers. Using python or c libraries is very easy, so you probably won't hit a wall in terms of library availability. A great strength of Julia will be the effortlessness with which one can glue together high performance functionality from various sources in a modern and consistent high level language (see answers above). But as a whole I found it to be neither mature nor stable enough to be used productively.&lt;/p&gt;&#xA;" OwnerUserId="19711" LastActivityDate="2016-03-22T18:38:19.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23442" PostTypeId="2" ParentId="19193" CreationDate="2016-03-22T18:49:28.687" Score="1" Body="&lt;p&gt;You can take a look at the truncation errors in your discretization. For example, for your time discretization, the truncation error would be &#xA;$$&#xA;\frac{\Delta t}{2} \frac{\partial^2 T}{\partial t^2}&#xA;$$&#xA;An implicit scheme guarantees stability but not accuracy, as the truncation errors can be quite high if large time-steps are taken. Moreover, these errors can add up as you run for longer times. The actual acceptable time-step usually depends on the problem at hand.&lt;/p&gt;&#xA;" OwnerUserId="4837" LastActivityDate="2016-03-22T18:49:28.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23443" PostTypeId="1" CreationDate="2016-03-22T22:41:26.600" Score="1" ViewCount="106" Body="&lt;p&gt;The documentation of PETSc shows an example of usage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  PetscRandomCreate(PETSC_COMM_SELF,&amp;amp;r);&#xA;  PetscRandomSetType(r,PETSCRAND48);&#xA;  PetscRandomGetValue(r,&amp;amp;value1);&#xA;  PetscRandomGetValueReal(r,&amp;amp;value2);&#xA;  PetscRandomDestroy(&amp;amp;r);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the example random 1, we can find also the use of PetscRandomSeed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it seems not to be enough for attaching a PetscRandom variable to an object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following code works well for any number of procs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#undef __FUNCT__&#xA;#define __FUNCT__ &quot;TSInjectCreate&quot;&#xA;PetscErrorCode  TSInjectCreate(MPI_Comm comm,TSInject *ininject){&#xA;  PetscErrorCode ierr;&#xA;  size_t         t;&#xA;  PetscInt       i;&#xA;  PetscReal       r;&#xA;  TSInject       inject;&#xA;&#xA;  PetscFunctionBegin;&#xA;  PetscValidPointer(ininject,1);&#xA;  *ininject = NULL;&#xA;  ierr                       = PetscClassIdRegister(&quot;TSInject&quot;,&amp;amp;TSINJECT_CLASSID);CHKERRQ(ierr);&#xA;  ierr = PetscHeaderCreate(inject,TSINJECT_CLASSID,&quot;TSInject&quot;,&quot;Injection of SDC&quot;,&quot;TS&quot;,comm,TSInjectDestroy, TSInjectView);CHKERRQ(ierr);&#xA;  ierr = PetscRandomCreate( comm,&amp;amp;inject-&amp;gt;ran);CHKERRQ(ierr);&#xA;  ierr = PetscRandomSetType(inject-&amp;gt;ran, PETSCRAND48);CHKERRQ(ierr);&#xA;  ierr = PetscRandomSeed(inject-&amp;gt;ran);CHKERRQ(ierr);&#xA;  ierr = PetscRandomGetValue(inject-&amp;gt;ran, &amp;amp;r);CHKERRQ(ierr);&#xA;  *ininject = inject;&#xA;&#xA;  PetscFunctionReturn(0);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But as soon as I use &lt;code&gt;PetscRandomGetValue(inject-&amp;gt;ran, &amp;amp;r);&lt;/code&gt; in another function, it works only when mpi is launched with one proc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error comes exactly (and weirdly) from the comparison:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PetscErrorCode  TSInjection(TSInject inject, Vec X)&#xA;{&#xA;  PetscReal       r,err;&#xA;  Vec             Y;&#xA;  PetscErrorCode ierr;&#xA;&#xA;  PetscFunctionBegin;&#xA;  ierr = PetscRandomGetValue(inject-&amp;gt;ran, &amp;amp;r);CHKERRQ(ierr);&#xA;ierr = VecView(X, 0);CHKERRQ(ierr);    // works&#xA;  if(r  &amp;gt; inject-&amp;gt;proba){ PetscFunctionReturn(0);}&#xA;        ierr = VecView(X, 0);CHKERRQ(ierr);      // stucks the execution&#xA; ...          &#xA;  PetscFunctionReturn(0);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I replace &lt;code&gt;r&lt;/code&gt; by 0.1 for example, everything works fine. What is all the more surprising is that displaying r doesn't stuck the execution.&lt;/p&gt;&#xA;" OwnerUserId="19667" LastEditorUserId="19667" LastEditDate="2016-03-23T15:32:00.410" LastActivityDate="2016-03-23T18:08:11.310" Title="PETCs - How should one initialize PetscRandom?" Tags="|petsc|c|random-number-generation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23444" PostTypeId="1" CreationDate="2016-03-22T22:47:26.387" Score="1" ViewCount="42" Body="&lt;p&gt;I've got an elliptic problem in a domain with two materials in a checkerboard arrangement. Both materials are highly dissimilar, being one much more compliant than the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use classic linear lagrange finite element, because of the corners of the checkerboard, there will be singularities that will make the solution gradient not converge with the mesh refinement. How can I prove this fact? I am looking for a proof based on the fact that the material properties uses the same mesh discretization, but it's piecewise constant. Which references would be good for this proof? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2016-03-22T22:47:26.387" Title="Gradient convergence on a checkerboard domain in finite element" Tags="|error-estimation|elliptic-pde|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23445" PostTypeId="1" CreationDate="2016-03-22T23:47:17.257" Score="2" ViewCount="3409" Body="&lt;p&gt;I understand the basic idea of imaginary time propagation method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The wavefunction $\psi(x,t)$ as a superposition of energy eigenstates $\phi_m(x)$:&#xA;$$&#xA;\psi(x,t)=\sum_m \phi_m(x)e^{-iE_mt/\hbar}&#xA;$$&#xA;In imaginary time, ${t}\rightarrow{-it}$,&#xA;$$&#xA;\psi(x,-it)=\sum_m \phi_m(x)e^{-E_mt/\hbar}=e^{-E_ot/\hbar}\bigg(\phi_o+\phi_1e^{-(E_1-E_o)t/\hbar}+\phi_1e^{-(E_2-E_o)t/\hbar}+........\bigg)&#xA;$$&#xA;As time $t$ increases the terms with greater exponents will decay more rapidly, leaving behind the ground state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how do i make a program or subroutine to find the ground state wavefunction using imaginary time propagation for a basic one dimensional problem, preferably in Fortran ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ref to &lt;a href=&quot;https://books.google.co.in/books?id=zTO-jzbfD3wC&amp;amp;pg=PA225&amp;amp;lpg=PA225&amp;amp;dq=an%20introduction%20to%20computational%20physics%20Pang%20gross%20pitaevskii&amp;amp;source=bl&amp;amp;ots=FL731i0gCm&amp;amp;sig=3jRigEhwNlItlbzETf-v3atrjBQ&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwiaz_H7tO3LAhXPHo4KHbkLDbgQ6AEINzAD#v=onepage&amp;amp;q=an%20introduction%20to%20computational%20physics%20Pang%20gross%20pitaevskii&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;7.12&lt;/a&gt; in an introduction to computational physics by Pang.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The macroscopic state of Bose–Einstein condensate is described by the&#xA;Gross–Pitaevskii equation:&#xA;$$&#xA;i\hbar\frac{\partial \Psi(x,t)}{\partial t}=\frac{-\hbar^{2}}{2m}\frac{\partial^2  \Psi(x,t)}{\partial x^2}+V_{ext}\Psi(x,t)+g|\Psi(x,t)|^2\Psi(x,t)&#xA;$$&#xA;where $\psi(x,t)$ is the macroscopic wavefunction, $m$ and $g$ are two system-related parameters, and $V_{ext}(x)$ is the external potential. Develop a program that solves this equation numerically. Consider various potentials, such as a parabolic or a square well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I apply Finite-difference method or Crank-Nickelson method, similar to simple diffusion equation, by substituting for $\frac{\partial \Psi(x,t)}{\partial t}, \frac{\partial^2  \Psi(x,t)}{\partial x^2}, |\Psi(x,t)|^2\Psi(x,t), \Psi(x,t)$, after inputting the imaginary time to get rid of '$i$' ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or how do I apply imaginary time evolution to Schrodinger equation of very basic 1D system ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; It'd be very helpful if someone can give me advice on where to start.&lt;/p&gt;&#xA;" OwnerUserId="14801" LastEditorUserId="14801" LastEditDate="2016-04-02T10:12:44.647" LastActivityDate="2021-02-24T08:11:51.537" Title="imaginary time propagation to find ground state wavefunction" Tags="|approximation|quantum-mechanics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23448" PostTypeId="1" CreationDate="2016-03-23T03:20:50.680" Score="1" ViewCount="210" Body="&lt;p&gt;I would like to know if I should  switch to OPENFOAM for my task.&#xA;I work only with Cartesian grids, right now in 2d, rectangular domains only.&#xA;If $\mathbf{w} = (w_1, w_2)$ and suppose I want to solve&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\partial_t(\mathbf{w}) = \nabla(\mathrm{div} \mathbf{w}) = \mathbf{f}$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;subject to Dirichlet boundary condition and an initial condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to do is discretize in the following way and solve:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\dfrac{w_1^{(n+1)} - w_1^n}{\tau} - \partial_{xx}w_1^{(n+1)} = f_1 +  \partial_{xy}w_2^n  \\   \dfrac{w_2^{(n+1)} - w_2^n}{\tau} -&#xA;  \partial_{yy}w_2^{(n+1)} = f_1 + \partial_{yx}w_1^{(n+1)}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Need 2 tridiaogonal solves ( so Ax = b is trivial)&lt;/li&gt;&#xA;&lt;li&gt;I have made my choice of time stepping, and also the discretization. &lt;/li&gt;&#xA;&lt;li&gt;I might modify my time stepping and discretization later.&lt;/li&gt;&#xA;&lt;li&gt;Right now, I wrote a program myself in C&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Basically I want to avoid doing the  routine tasks like organize the MAC grid, store source values at the cell centre, compute $\partial_{xy}w_2^n$  given $w_2^n$ etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q1) Is OPENFOAM overkill for my task and be more of a nuisance to actually implement the scheme I want to try ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q2) Am I better off doing what I am doing now, having a working code in C, which means though that each time I want to try something new, I have to change my program ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q3) Is it possible to use direct methods instead of iterative methods to solve my system in OPENFOAM ?&lt;/strong&gt;  &lt;/p&gt;&#xA;" OwnerUserId="7728" LastEditorUserId="7728" LastEditDate="2016-03-23T05:02:23.623" LastActivityDate="2016-04-22T05:21:12.377" Title="Is OPENFOAM a good idea for direction/operator splitting in parabolic equation" Tags="|software|finite-volume|libraries|openfoam|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23449" PostTypeId="2" ParentId="23448" CreationDate="2016-03-23T04:20:05.903" Score="1" Body="&lt;p&gt;OpenFoam will definitely satisfy your needs. Though you may feel some restrictions of solvers,still it mostly resolves all the domains. Also Open Source platform has its advantage of large user-base and support by community. You can always give a try. &lt;/p&gt;&#xA;" OwnerUserId="19539" LastActivityDate="2016-03-23T04:20:05.903" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23451" PostTypeId="1" CreationDate="2016-03-23T12:06:11.570" Score="4" ViewCount="288" Body="&lt;p&gt;&lt;a href=&quot;https://books.google.com/books?isbn=0387400656&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nocedal and Wright&lt;/a&gt;&#xA;on Conjugate Gradient Methods, p. 123, describe a&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;restart strategy ... whenever two consecutive gradients are far from orthogonal&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;$\qquad {{| \nabla f_k^T \ \nabla f_{k-1} |}&#xA;\over { \|\nabla f_k\|^2 }} \ge \nu $,&#xA;with $\nu$ typically 1/10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone comment on CG with such restarts, or point to test cases on the web ?&lt;br&gt;&#xA;Or is the &quot;popular choice max $( 0, \beta^{PR} )$&quot;&#xA;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Nonlinear_conjugate_gradient_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nonlinear_conjugate_gradient_method&lt;/a&gt;)&#xA;good enough, satisficing ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(A good answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/507/bfgs-vs-conjugate-gradient-method&quot;&gt;bfgs-vs-conjugate-gradient-method&lt;/a&gt; says,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Anecdotal evidence points to restarting being a tricky issue,&lt;br&gt;&#xA;  as it is sometimes unnecessary and sometimes very necessary.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Well, that's generally true of a lot of things (taxes come to mind).&lt;br&gt;&#xA;Test cases with plots of $\beta_k$ or $\theta_k$ might be interesting.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;A possibly silly test case that led to the question is CG on an ill-conditioned quadratic in 2d:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.optimize import fmin_cg&#xA;&#xA;n = 2&#xA;cond = 100&#xA;eigenvalues = np.linspace( 1./cond, 1, n )&#xA;xmin = 1000 * np.ones( n )&#xA;&#xA;def fprime( x ):&#xA;    return eigenvalues * (x - xmin)&#xA;def f( x ):&#xA;    return (x - xmin) .dot( eigenvalues * (x - xmin)) / 2&#xA;&#xA;x0 = np.zeros( n)&#xA;&#xA;ret = fmin_cg( func, x0, fprime )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Added:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;for &lt;code&gt;n&lt;/code&gt; in [1,2,3,4,5] this takes 80 81 6 40 9 iterations,&#xA;721 722 28 94 27 function evaluations.&#xA;(Is CG generally &lt;em&gt;very&lt;/em&gt; sensitive to the linesearch ?)&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://reference.wolfram.com/language/tutorial/UnconstrainedOptimizationConjugateGradientMethods.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematica conjugate gradient minimizer&lt;/a&gt;&#xA;has a RestartThreshold with default 1/10.&#xA;But, sorry, I don't speak Mathematica.&#xA;Any native speaker care to try this ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-04-10T09:19:51.963" Title="Nonlinear conjugate gradient restart threshold 1/10" Tags="|convergence|conjugate-gradient|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23455" PostTypeId="1" CreationDate="2016-03-23T15:47:06.930" Score="3" ViewCount="1236" Body="&lt;p&gt;A CSV file is characterized by a header, describing the n datarows to come. The header is a text string separated by a separator. A CSV header might look like (C1)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Date;Time;ZIP-Code;Address;Temperature&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Assume we have a second CSV file (C2) with this structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Date;ZIP-Code;Address;Temperature&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;C1 and C2 are similar, but they are not the same: C2 lacks &quot;Time&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume another C3:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Time;Date;Address;ZIP-Code;Temperature&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here the same items as in C1 are present, but the order is different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am after is a metric which would give me the similarity between two sets, including the relative closeness of items with these two sets. In other words, if the order of items varies within one set, but the cardinality is the same, the similarity value should be bigger compared to different cardinality or if the cardinality is the same but the items itself vary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came up with this preliminary mental measure. I could plot in a matrix &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;             Date;Time;ZIP-Code;Address;Temperature&#xA;Date           1 &#xA;Time                1&#xA;ZIP-Code                    1&#xA;Address                            1&#xA;Temperature                                 1&#xA;&#xA;&#xA;             Date;Time;ZIP-Code;Address;Temperature&#xA;Time           0    1     0         0       0&#xA;ZIP-Code       0    0     1         0       0&#xA;Address        0    0     0         1       0&#xA;Temperature    0    0     0         0       1&#xA;&#xA;&#xA;             Date;Time;ZIP-Code;Address;Temperature&#xA;Temperature    0    0      0       0        1&#xA;Address        0    0      0       1        0&#xA;ZIP-Code       0    0      1       0        0&#xA;Time           0    1      0       0        0&#xA;Date           1    0      0       0        0&#xA;&#xA;&#xA;&#xA;             Date;Time;ZIP-Code;Address;Temperature&#xA;Head1           0   0       0       0       0&#xA;Head2           0   0       0       0       0&#xA;Head3           0   0       0       0       0&#xA;Head4           0   0       0       0       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My sense of similarity would be &quot;the more structure / pattern in the matrix&quot;, the more similar the tow CSV headers are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is a measure like Dice-Distance, Cosine-Similarity, Jaccard-Index which would be of help?&lt;/p&gt;&#xA;" OwnerUserId="1856" LastActivityDate="2016-03-24T18:59:47.590" Title="Similarity of two CSV files (or more?)" Tags="|data-analysis|data-sets|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23456" PostTypeId="2" ParentId="23443" CreationDate="2016-03-23T18:08:11.310" Score="1" Body="&lt;p&gt;To solve this issue, a work-around is too employ a DIY random function. In my context, I have given up with PetscRandom to write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PetscErrorCode TSInjectRandInt(TSInject inject, PetscInt *r)  // RAND_MAX assumed to be 32767&#xA;{&#xA;    PetscFunctionBegin;&#xA;    inject-&amp;gt;next = inject-&amp;gt;next * 1103515245 + 12345;&#xA;    *r = PetscAbs((PetscInt)(inject-&amp;gt;next/65536) % 32768);&#xA;    PetscFunctionReturn(0);&#xA;}&#xA;&#xA;&#xA;PetscErrorCode TSInjectRandReal(TSInject inject, PetscReal *r)  {&#xA;  PetscInt i;&#xA;&#xA;  PetscFunctionBegin;&#xA;  TSInjectRandInt(inject, &amp;amp;i);&#xA;  *r = ((PetscReal)i)/32767.0;&#xA;  PetscFunctionReturn(0);&#xA;}&#xA;&#xA;PetscErrorCode TSInjectRandSeed(TSInject inject, PetscInt seed)&#xA;{&#xA;    PetscFunctionBegin;&#xA;    inject-&amp;gt;next = seed;&#xA;    PetscFunctionReturn(0);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, one should notice that RAND_MAX is fixed and that the generator is clearly very easy. It has the only advantage of working. &lt;/p&gt;&#xA;" OwnerUserId="19667" LastActivityDate="2016-03-23T18:08:11.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23457" PostTypeId="1" CreationDate="2016-03-23T21:03:00.207" Score="2" ViewCount="620" Body="&lt;p&gt;Suppose I have a large, sparse, $m \times n$ matrix $A$, with $m \gt n$ and $\text{rank}(A) = n$. I wish to solve $Ax=b$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I know that $A$ has the following characteristics:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$A$ is somewhat diagonally-dominant (it may have a few side-bands offset quite a bit from the main diagonal, and a few rows do not fit the band structure). &lt;/li&gt;&#xA;&lt;li&gt;$A$ is not symmetric.&lt;/li&gt;&#xA;&lt;li&gt;$A$ is not positive-definite. &lt;/li&gt;&#xA;&lt;li&gt;$rank(A) = n$  (a unique solution exists!).&lt;/li&gt;&#xA;&lt;li&gt;$\text{cond}(A^T A)$ is quite high: $10^{\gt 20}$. &lt;/li&gt;&#xA;&lt;li&gt;$m$ is only slightly larger than $n$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Suppose I require an accuracy of one part in $10^{\gt 9}$, or roughly 9 digits of precision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This unique problem is neither under- nor over-determined, but it involves a rectangular matrix. It is not in the realm of either least-squares or minimum length. This places constraints on which approaches would be suitable to solve it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I could use Gaussian elimination, SVD (projecting into the orthogonal complement to the null space), or a few other methods. I also know several other methods (e.g. LU decomposition) require a square matrix, so they could be used to solve $\left(A^T A\right) x = A^T b$, though I'm concerned that the high condition number of $A^T A$ would be prohibitive for these approaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What methods would be appropriate to solve this system of equations, and what are their relative advantages/disadvantages?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Which method(s) hold up best against high condition numbers? &lt;/li&gt;&#xA;&lt;li&gt;Which method(s) offer the best accuracy? &lt;/li&gt;&#xA;&lt;li&gt;Which method(s) are the quickest / most efficient? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9451" LastActivityDate="2016-03-23T21:29:12.807" Title="Methods for solving rectangular, full-rank systems of equations -- which is best?" Tags="|linear-algebra|algorithms|numerical-analysis|linear-solver|condition-number|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23458" PostTypeId="2" ParentId="23457" CreationDate="2016-03-23T21:29:12.807" Score="1" Body="&lt;p&gt;I suggest Tim Davis' SPQR which is part of his SuiteSparse package:&#xA;&lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot; rel=&quot;nofollow&quot;&gt;http://faculty.cse.tamu.edu/davis/suitesparse.html&lt;/a&gt;&#xA;I believe it will do a reasonable job of satisfying your three criteria.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-03-23T21:29:12.807" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23459" PostTypeId="1" CreationDate="2016-03-24T11:55:44.543" Score="4" ViewCount="194" Body="&lt;p&gt;In numerical fluid dynamics there are field methods like finite-volume, finite-element, etc. and particle methods like Smoothed-Particle-Hydrodynamics – SPH and others. Both approaches have advantages and disadvantages depending on the application. Both, in principal completely different approaches, can be combined using a Voronoi-Diagram. These Voronoi-Cells which behave like particles fill out the complete regarded space and are moving according to the physical laws (conservation of momentum and energy; see also&#xA;&lt;a href=&quot;http://ivancic.de/cfd2k/WhatIsCFD2k.html&quot; rel=&quot;nofollow&quot;&gt;http://ivancic.de/cfd2k/WhatIsCFD2k.html&lt;/a&gt;). These particles also can interact with each other (exchange of momentum, energy, etc. ==&gt; the Voronoi cells/particles obey the Navier-Stokes Equations). Such a Voronoi approach can combine the advantages of field and particle methods and therefore lead to better results in the numerical fluid dynamics (e.g. for turbulent flows which cannot be predicted accurately up to now).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I do not know any numerical method to apply it for such Voronoi-Particles in order to describe physical flows correctly. Does anybody know such a numerical method or is anyone interested in developing such a method together with me? I am an aerospace engineer very familiar with fluid dynamics, turbulence and their physical and thermodynamic laws but sadly I am not an expert in numerical mathematics which is necessary to derive such a new approach.&lt;/p&gt;&#xA;" OwnerUserId="19741" LastEditorUserId="7567" LastEditDate="2016-03-28T05:59:41.770" LastActivityDate="2022-12-28T06:45:14.110" Title="combination of field and particle methods for fluid dynamics" Tags="|fluid-dynamics|computational-physics|numerical-modelling|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23465" PostTypeId="2" ParentId="23455" CreationDate="2016-03-24T18:59:47.590" Score="1" Body="&lt;p&gt;Useful inclusion measures depend on whether you want to check pairwise similiarity or containment. For similiarity the Jaccard index is intuitive. For containment I would adapt it to &lt;code&gt;what ratio of columns of my 'gold standard file' are in the file to check&lt;/code&gt; which is a common feature used in natural language problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the order of columns in a file, an idea is to find the overlapping sequences of columns. Assume both files have the exactly same columns but they are reordered. You can find overlapping sections, and then count the sum of overlaps (length of the section -1, because a overlap of 1 is no overlap) and normalise to the maximum possible. See example below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;File 1: A | B | C | D | E&#xA;File 2: D | E | C | A | B&#xA;overlapping sections: (D, E), (C), (A, B)&#xA;normalisation: sum of overlaps / max possible overlap&#xA;result: ( 1 + 0 + 1 ) / 4 = 0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A combination (sum, product?) of these two will cover some patterns in your matrices, but fail at other patterns a human would identify, i.e. having all columns in reverse order leads to 0 for the second metric. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem would be inconsistent column names. If the string names don't exactly match, everything above breaks down. In that case, you might want to look into matching functions (matching dependencies) for data cleaning before.&lt;/p&gt;&#xA;" OwnerUserId="18622" LastActivityDate="2016-03-24T18:59:47.590" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23466" PostTypeId="1" CreationDate="2016-03-24T22:18:31.840" Score="2" ViewCount="401" Body="&lt;p&gt;The 2-norm condition number can be easily extended to rectangular matrices. I'm wondering if the inequality for the product of matrices still holds in that case, i.e.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\operatorname{cond}(AB) \leq \operatorname{cond}(A)\operatorname{cond}(B)$&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="127" LastEditDate="2017-04-01T09:14:54.960" LastActivityDate="2017-04-01T09:14:54.960" Title="Condition Number of Rectangular Matrices" Tags="|linear-algebra|matrix|svd|condition-number|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23468" PostTypeId="2" ParentId="23466" CreationDate="2016-03-25T02:24:04.500" Score="3" Body="&lt;p&gt;I guess I figured out the answer to my question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose the SVD of $A = U \Sigma V^\ast$  (where $V^\ast$ is the conjugate transpose of the matrix $V$). Noting the fact that the unitary transformations $U$ and $V$ preserve the 2-norm, $\|Ax\|_{2}$ for any unit vector $x$ can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\frac{\|Ax\|_{2}}{\|x\|_{2}} = \|Ax\|_{2}&#xA;&amp;amp;= \|U \Sigma V^\ast x\|_{2} \\&#xA;&amp;amp;= \| \Sigma V^\ast x\|_{2} \\&#xA;&amp;amp;= \| V^\ast \Sigma x\|_{2} \\ &#xA;&amp;amp;= \|\Sigma x\|_{2}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence $\|Bx\|_{2} \leq \sigma_\max(B)$ and $\|Bx\|_{2} \geq \sigma_\min(B)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $y = Bx$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;\|ABx\|_{2} = \|Ay\|_{2} \leq \sigma_\text{max}(A) \|y\|_{2} \leq \sigma_\max(A) \sigma_\max(B)&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;\|ABx\|_{2} = \|Ay\|_{2} \geq \sigma_\min(A) \|y\|_{2} \geq \sigma_\min(A) \sigma_\min(B)&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the above statements are true for all $x$, they are true both for the minimum and maximum. Thus, &#xA;\begin{equation*}&#xA;\sigma_\max(AB) = \max \|ABx\|_{2} \leq \sigma_\max(A) \sigma_\max(B)&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&#xA;\begin{equation*}&#xA;\sigma_\min(AB) = \min \|ABx\|_{2} \geq \sigma_\min(A) \sigma_\min(B)&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On dividing the above two equations, we obtain &#xA;\begin{equation*}&#xA;\operatorname{cond}_{2}(AB) \leq \operatorname{cond}_{2}(A) \operatorname{cond}_{2}(B)&#xA;\end{equation*}&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="127" LastEditDate="2017-04-01T09:12:37.083" LastActivityDate="2017-04-01T09:12:37.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23469" PostTypeId="2" ParentId="23459" CreationDate="2016-03-25T03:20:54.053" Score="0" Body="&lt;p&gt;I don't know very much about the application you have in mind, but combinations of finite element or finite volume methods with particle methods typically go by the name &quot;Particle-In-Cell&quot; (or, for short, PIC) methods. You may want to do a literature search. These methods are quite popular in geodynamics.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-03-25T03:20:54.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23471" PostTypeId="1" CreationDate="2016-03-25T14:14:25.167" Score="1" ViewCount="143" Body="&lt;p&gt;I wonder what is the difference between both functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In C++, the difference between new and malloc is easier: new allocates memory by calling the constructor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about PETSc ? In the example, it seems to be pretty close:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in dm/examples/tutorials/ex6.c:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;struct _p_FA {&#xA;  MPI_Comm   comm[3];&#xA;  PetscInt   xl[3],yl[3],ml[3],nl[3];    /* corners and sizes of local vector in DMDA */&#xA;  PetscInt   xg[3],yg[3],mg[3],ng[3];    /* corners and sizes of global vector in DMDA */&#xA;  PetscInt   offl[3],offg[3];            /* offset in local and global vector of region 1, 2 and 3 portions */&#xA;  Vec        g,l;&#xA;  VecScatter vscat;&#xA;  PetscInt   p1,p2,r1,r2,r1g,r2g,sw;&#xA;};&#xA;typedef struct _p_FA *FA;&#xA;FA         fa;&#xA;PetscNew(&amp;amp;fa);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In  ksp/ksp/examples/tutorials/ex42.c:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PetscMalloc(sizeof(struct _p_CellProperties),&amp;amp;cells);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The rule is just: using PetscMalloc for a structure, and PetscNew for a pointer of structure?&lt;/p&gt;&#xA;" OwnerUserId="19667" LastActivityDate="2016-03-25T14:14:25.167" Title="PETSc - difference between PetscNew and PetscMalloc" Tags="|petsc|c|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23472" PostTypeId="1" CreationDate="2016-03-25T14:31:33.330" Score="2" ViewCount="1825" Body="&lt;p&gt;I recently heard that in Linux, &lt;code&gt;/dev/random&lt;/code&gt; and &lt;code&gt;/dev/urandom&lt;/code&gt; are pseudo random number generators but not true random number generator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am curious to know the difference between pseudo &amp;amp; true random number? In other words What is the difference between pseudo-randomness &amp;amp; true-randomness?&lt;/p&gt;&#xA;" OwnerUserId="19754" LastActivityDate="2017-06-29T01:39:15.967" Title="Difference between true &amp; pseudo random numbers" Tags="|random-number-generation|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23474" PostTypeId="1" CreationDate="2016-03-25T20:56:41.177" Score="1" ViewCount="498" Body="&lt;p&gt;I am interested in solving a sequence of linear systems of the form:&#xA;$$A x_i = b_i$$&#xA;That is, all the systems use the same matrix $A$ but they have different right hand sides. The matrix $A$ is &lt;strong&gt;sparse&lt;/strong&gt; symmetric positive definite. &#xA;I am currently using CHOLMOD to solve each system. Obviously, I factorize $A$ once (Cholesky factorization) and then I perform the forward/backward solves sequentially. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am now trying to parallelize these operations. I would like to find the $x_i$'s in parallel. On my first attempt using &lt;code&gt;pthreads&lt;/code&gt;, I couldn't get more than 2X speed-up on a quad-core i7 machine. I am guessing that I couldn't go beyond 2X because the operations that I am trying to parallelize are so fast that their runtime is comparable to the time required to read the Cholseky factorization matrices from the memory and to send them to each core. I am not sure of that as I am not very experienced in parallel programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is there a better way of doing this? Would I expect higher speed-ups if I used other programming paradigms? Also, are you aware of any solvers out there that already perform this parallelization. (I am not interested in packages that parallelize one system solve).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem seems embarrassingly parallel and I expected speed-ups very close to 4X on a quad-core machine as there are no dependencies between the tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="1932" LastActivityDate="2016-03-25T22:01:33.273" Title="Linear Systems with Multiple Right Hand sides" Tags="|parallel-computing|linear-solver|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23475" PostTypeId="2" ParentId="23474" CreationDate="2016-03-25T22:01:33.273" Score="7" Body="&lt;p&gt;It's possible that your performance is limited by the memory bandwidth of your system.  It's not at all uncommon to have a situation where two or three of your cores can use all of the available memory bandwidth and make it impossible to achieve higher parallel speedups because there isn't enough bandwidth to allow all of the cores to work at full speed.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A relatively simple test is to run John McCalpin's STREAM benchmark with 1, 2, 3, and 4 cores to see how performance scales.  Don't be surprised if it bogs down after two cores.   If your system is bandwidth limited in this way and if your code is memory intensive (typical for sparse direct solvers), then this could explain your results.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see some STREAM benchmark results at McCalpin's web site.  One result for a generic core-i7-2600 system shows very little improvement in going from one to four cores.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The STREAM benchmark can be found at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cs.virginia.edu/stream/&quot;&gt;https://www.cs.virginia.edu/stream/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-03-25T22:01:33.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23476" PostTypeId="2" ParentId="23459" CreationDate="2016-03-26T13:26:18.147" Score="3" Body="&lt;p&gt;Besides the PIC methods that Wolfgang Bangerth refers to there are some approaches in computer graphics or more specifically in computer animation that explicitly make use of Voronoi cell discretizations to solve the Navier-Stokes or Euler equations (without the viscous terms). In contrast to the PIC approaches, there the computational mesh is not constant but it changes during the simulation as the particles move. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two methods I have in mind are are published by Brochu et al. (&lt;a href=&quot;http://dl.acm.org/citation.cfm?doid=1778765.1778784&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?doid=1778765.1778784&lt;/a&gt;) and Sin et al. (&lt;a href=&quot;http://dl.acm.org/citation.cfm?doid=1599470.1599502&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?doid=1599470.1599502&lt;/a&gt;). There, the particles carry all information and a Voronoi diagram is constructed in each time step to compute pressure corrections. Both methods are able to compute single-phase and pseudo-two-phase flows. It is important to note that these approaches are for computer animation, i.e., for visually plausible simulations and are not designed for highly accurate flow prediction. Maybe these two references might help in designing a method that you have in mind. &lt;/p&gt;&#xA;" OwnerUserId="19766" LastActivityDate="2016-03-26T13:26:18.147" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23477" PostTypeId="2" ParentId="14214" CreationDate="2016-03-26T19:51:00.880" Score="-1" Body="&lt;p&gt;The solution is here for windows machine.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Type first &gt;&gt;pwd&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then you will see the path of the matlab folder , for example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C:\Program Files\MATLAB\R2012a\bin&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right click on bin file and choose properties and then &quot;security&quot;, then &quot;Edit&quot; to change the permission to &quot;Allow&quot; for all Users, click oK,  and you can go back to matlab and test it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks M Merdan &lt;/p&gt;&#xA;" OwnerUserId="19767" LastActivityDate="2016-03-26T19:51:00.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23478" PostTypeId="2" ParentId="23459" CreationDate="2016-03-26T20:25:19.837" Score="4" Body="&lt;p&gt;There are some interesting numerical schemes for incompressible Euler equations that involve power diagrams (a generalization of Voronoi diagrams), there is a survey paper by Jean-Marie Mirebeau there:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://hal.archives-ouvertes.fr/hal-01237356&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.archives-ouvertes.fr/hal-01237356&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The idea is based on a connection that was discovered by Yann Brenier between :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;the projection operator onto the set of volume-preserving vector fields&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the problem of optimal transport&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;More details on Yann Brenier's homepage:&#xA;&lt;a href=&quot;http://www.cmls.polytechnique.fr/perso/brenier/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cmls.polytechnique.fr/perso/brenier/&lt;/a&gt;&#xA;see in particular:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;polar decomposition of vector fields&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The problem of optimal transport has a certain form (semi-discrete) that naturally makes Voronoi diagram (or power diagrams) appear. The nice point is that the theory of optimal transport manipulates a very general class of mathematical objects (probablity measures), that can be both continous functions and sums of Dirac masses. This is when using a continuous function on one side and a sum of Dirac masses on the other side that Voronoi diagrams appear. More importantly, since the theory encompasses both cases, this is not a discretization, this is an application of the theory in a special case. Therefore, all the mathematical properties are preserved by the discrete version.&lt;/p&gt;&#xA;&lt;p&gt;Recent numerical schemes were developed for some forms of Euler fluids, including:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Merigot and Mirebeau's paper: &lt;a href=&quot;http://arxiv.org/abs/1505.03306&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://arxiv.org/abs/1505.03306&lt;/a&gt;&#xA;(where initial and final boundary conditions are fixed, i.e. the goal is to compute a certain geodesic that describes the motion of the fluid)&lt;/li&gt;&#xA;&lt;li&gt;Power particles (a computer graphics paper that uses very similar notions, but with only initial boundary conditions fixed): &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=2766901&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dl.acm.org/citation.cfm?id=2766901&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I think that these methods are interesting, because they can be directly derived from the principle of least action, and they can embed some conservation laws directly in their formulation, without needing to enforce additional constraint. I think this is very elegant.&lt;/p&gt;&#xA;&lt;p&gt;The basic components for these numerical solution mechanisms is solving (some form of) the Monge-Ampere equation. I published a method to do that in 3D (my method is a 3D extension of a 2D multi-level solver by Merigot), the tech report is here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://arxiv.org/abs/1409.1279&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://arxiv.org/abs/1409.1279&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(I published later a version in Mathematical Modeling and Analysis J.)&lt;/p&gt;&#xA;&lt;p&gt;The source-code is part of my GEOGRAM open-source library:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/BrunoLevy/geogram&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/BrunoLevy/geogram&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Other references: there is also an interesting simulator for astrophysics (AREPO) that uses similar notions, reference is there:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://wwwmpa.mpa-garching.mpg.de/%7Evolker/arepo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://wwwmpa.mpa-garching.mpg.de/~volker/arepo/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A version of the algorithm that works for free-surface fluids (partial optimal transport) that I published in Journal of Computational Physics:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0021999121007336?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0021999121007336?via%3Dihub&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Applications in cosmology, that we published in Physical Review Letters:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.128.201302&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.128.201302&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.129.251101&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.129.251101&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2022-12-28T06:45:14.110" LastActivityDate="2022-12-28T06:45:14.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23479" PostTypeId="1" CreationDate="2016-03-26T21:18:02.270" Score="1" ViewCount="78" Body="&lt;p&gt;Suppose $A\in\mathbb{R}^{n\times n}$ is a banded matrix, i.e., a matrix with all of its nonzero elements on the main diagonal, i.e., $\alpha_{i,i}\neq 0$, the first superdiagonal, i.e., $\alpha_{i,i+1}\neq 0$, through the $k$-th superdiagonal, i.e., $\alpha_{i,i+k}\neq 0$, the first subdiagonal, i.e., $\alpha_{i-1,i}\neq 0$, through the $k$-th subdiagonal, i.e., $\alpha_{i,i+k}\neq 0$. All elements not on these diagonal are $0$. For $k = 4$ and $ n = 15$ the pattern is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{pmatrix}&#xA;* &amp;amp; *  &amp;amp; *  &amp;amp; *  &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;* &amp;amp; *  &amp;amp; *  &amp;amp; *  &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\  &#xA;* &amp;amp; *  &amp;amp; *  &amp;amp; *  &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;* &amp;amp; *  &amp;amp; *  &amp;amp; *  &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;* &amp;amp; *  &amp;amp; *  &amp;amp; *  &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; *  &amp;amp; *  &amp;amp; *  &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; *  &amp;amp; *  &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; *  &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * \\&#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * \\ &#xA;0 &amp;amp; 0  &amp;amp; 0  &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; * \\ &#xA;\end{pmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we need to store this matrix as efficiently as possible. I wrote my code in C++ and the output is correct and the way I did it I believe saves the storage since I do not store any of the zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code that I did:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double** banded_matrix(int n) {&#xA;    double **data = new double *[n];&#xA;    data[0] = new double[3];&#xA;    for(int i = 1; i &amp;lt;= 3; i++){&#xA;        data[i] = new double[4];&#xA;    }&#xA;    for(int i = 4; i &amp;lt;= n-3; i++){&#xA;        data[i] = new double[5];&#xA;    }&#xA;    for(int i = n-4; i &amp;lt;= n-2; i++){&#xA;        data[i] = new double[4];&#xA;    }&#xA;    data[n-1] = new double[3];&#xA;    return data;&#xA;}&#xA;&#xA;double get(double** A, int n, int i, int j) {&#xA;    double result;&#xA;    // main tridiagonal&#xA;    if(abs(i-j) &amp;lt; 2){&#xA;        if(i == 0){&#xA;            result = A[i][j];&#xA;        } else if(i &amp;lt;= 3){&#xA;            result = A[i][j-(i-2)];&#xA;        } else if(i &amp;gt;= 3){&#xA;            result = A[i][j-(i-1)];&#xA;        }&#xA;    // 4th superdiagonal&#xA;    } else if (j - i == 4){&#xA;        if(i == 0 || i == n-1) {&#xA;            result = A[i][2];&#xA;        } else if((i &amp;gt;= 1 &amp;amp;&amp;amp; i &amp;lt;= 3) || (i &amp;gt;= n - 4 &amp;amp;&amp;amp; i &amp;lt;= n - 2)) {&#xA;            result = A[i][3];&#xA;        } else {&#xA;            result = A[i][4];&#xA;        }&#xA;    // 4th subdiagonal&#xA;    } else if (i - j == 4){&#xA;        result = A[i][0];&#xA;    // the rest of the matrix    &#xA;    } else {&#xA;        result = 0.;&#xA;    }&#xA;    return result;&#xA;}&#xA;&#xA;void set(double** A, int n, int i, int j, double val) {&#xA;    // main tridiagonal&#xA;    if(abs(i-j) &amp;lt; 2){&#xA;        if(i == 0){&#xA;            A[i][j] = val;&#xA;        } else if(i &amp;lt;= 3){&#xA;            A[i][j-(i-2)] = val;&#xA;        } else if(i &amp;gt;= 3){&#xA;            A[i][j-(i-1)] = val;&#xA;        }&#xA;    // 4th superdiagonal&#xA;    } else if (j - i == 4){&#xA;        if(i == 0 || i == n-1) {&#xA;            A[i][2] = val;&#xA;        } else if((i &amp;gt;= 1 &amp;amp;&amp;amp; i &amp;lt;= 3) || (i &amp;gt;= n - 4 &amp;amp;&amp;amp; i &amp;lt;= n - 2)) {&#xA;            A[i][3] = val;&#xA;        } else {&#xA;            A[i][4] = val;&#xA;        }&#xA;    // 4th subdiagonalf&#xA;    } else if (i - j == 4){&#xA;        A[i][0] = val;&#xA;    // the rest of the matrix    &#xA;    } else {&#xA;        cout &amp;lt;&amp;lt; &quot;cannot set element (&quot; &amp;lt;&amp;lt; i &amp;lt;&amp;lt;&quot;,&quot; &amp;lt;&amp;lt; j &amp;lt;&amp;lt;&quot;) in matrix&quot; &amp;lt;&amp;lt; endl;&#xA;    }&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    int N = 15;&#xA;&#xA;    //Initialize A&#xA;    double **A = banded_matrix(N);&#xA;    create(A,N,0);&#xA;//Print A&#xA;    cout &amp;lt;&amp;lt; &quot;A = &quot; &amp;lt;&amp;lt; endl;&#xA;    for (int i = 0; i &amp;lt; N; i++) {&#xA;        for (int j = 0; j &amp;lt; N; j++) {&#xA;           cout &amp;lt;&amp;lt; get(A,N,i,j) &amp;lt;&amp;lt; &quot;   &quot;;        &#xA;        }&#xA;       cout &amp;lt;&amp;lt; endl;&#xA;    }&#xA;  cout &amp;lt;&amp;lt; endl;&#xA;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I just want to know if my method is efficient or if I should have done it a different way.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastActivityDate="2016-03-28T05:53:51.190" Title="Creating a matrix that saves storage" Tags="|matrix|banded-matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23480" PostTypeId="2" ParentId="23472" CreationDate="2016-03-26T22:19:18.907" Score="2" Body="&lt;p&gt;Most random numbers are in fact pseudo-random: they use a seed that is fed to a perfectly deterministic algorithm which can then generate series of numbers which are periodic but with a &lt;strong&gt;very large and complicated period&lt;/strong&gt; so they usually appear random.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has drawbacks of course since it's not really random, but also advantages, among which is &lt;strong&gt;reproducibility&lt;/strong&gt;: if you store the seed, using this seed again will give you the exact same list of &quot;random&quot; numbers...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are &lt;a href=&quot;https://www.christianpinder.com/articles/pseudo-random-number-generation/&quot; rel=&quot;nofollow&quot;&gt;a&lt;/a&gt; &lt;a href=&quot;http://en.cppreference.com/w/cpp/numeric/random&quot; rel=&quot;nofollow&quot;&gt;few&lt;/a&gt; &lt;a href=&quot;http://engineering.mit.edu/ask/can-computer-generate-truly-random-number&quot; rel=&quot;nofollow&quot;&gt;links&lt;/a&gt; as well as another to &lt;a href=&quot;https://www.random.org/&quot; rel=&quot;nofollow&quot;&gt;a source of real random numbers&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19678" LastActivityDate="2016-03-26T22:19:18.907" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23481" PostTypeId="1" CreationDate="2016-03-27T00:52:58.237" Score="1" ViewCount="776" Body="&lt;p&gt;For my program assignment I need to write a preconditioned steepest descent algorithm. I have psedo-code from my professor which is here:&lt;a href=&quot;https://i.stack.imgur.com/1OSr3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1OSr3.jpg&quot; alt=&quot;PSD&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From reading in my text on this method they say that $P$ is the preconditioned matrix but they do not define it in any way shape or form. I am not sure at all, but by my professors psedo-code I would guess that $M$ is $P$? I honestly have no idea if that is the case. I have tried to search online for this but I seem to be getting information that is not very consistent with the way our book/professor presents the material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: $M$ is just a general precondition matrix; i.e. Jabobi, or any other precondition matrix. The problem I seem to be having now is that I do not really understand the pseudo-code. I have all the ingredients necessary to start but I am not sure how to begin. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: the book I am using is,  A. Quarteroni, R. Sacco, and F. Saleri, Numerical Mathematics,&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="17826" LastEditDate="2016-03-27T15:51:10.900" LastActivityDate="2016-03-27T15:51:10.900" Title="Preconditioned Steepest Descent" Tags="|linear-algebra|algorithms|numerical-analysis|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="23482" PostTypeId="1" CreationDate="2016-03-27T01:38:19.023" Score="1" ViewCount="108" Body="&lt;p&gt;In the (iterative) Conjugate Gradient (CG) algorithm: &lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Conjugate_gradient_method&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial search direction $p_{0}$ is set to the initial residual $r_{0}$. But I can't see why this choice? Can I get this choice of $p_{0}$ from any of the equations presented in the article?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="19692" LastActivityDate="2016-03-27T04:08:34.550" Title="Conjugate Gradient, initial direction set to initial residual" Tags="|numerical-analysis|numerics|conjugate-gradient|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23483" PostTypeId="2" ParentId="23482" CreationDate="2016-03-27T04:08:34.550" Score="1" Body="&lt;p&gt;Consider the equation&#xA;$\mathbf{p}_k=\mathbf{r}_k-\sum_{i&amp;lt;k} \frac{\mathbf{p}_i^T\mathbf{A}\mathbf{r}_k}{\mathbf{p}_i^T\mathbf{A}\mathbf{p}_i}\mathbf{p}_i$ for $k=0$. Equivalently, and in terms of what's actually going on, we are correcting the search direction of steepest descent to be $\mathbf{A}$ orthogonal to the previous search directions. For the first step we &lt;em&gt;have&lt;/em&gt; no other search directions and just look along the residual directly.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-03-27T04:08:34.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23484" PostTypeId="1" AcceptedAnswerId="23511" CreationDate="2016-03-27T06:01:31.180" Score="3" ViewCount="318" Body="&lt;p&gt;I am working on an astrophysical research in which we relate the cumulative number of Damped Lyman Alpha HI clouds/galaxies, namely their number densities, $\frac{dN_{DLA}}{dz}(&amp;gt;M, z=0),$ to the hosting Dark Matter halo mass function, $\frac{dn_{h}}{d(\log M)}$, and the incidence cross section area, $\sigma_{DLA}$, in the Local Universe through the following integral equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dN_{DLA}}{dz}\left(&amp;gt;\log(M_{min}), z=0\right) = \int_{\log(M_{min})}^{\infty}\frac{dn_{h}}{d(\log M)}(\log M) \times\sigma_{DLA}(\log M) \times d(\log M)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I happen to have observational information of the number density on the left-hand side and Dark Matter mass function on the right-hand side (with known functional forms.) However, I have no information on the parametrization of the incidence cross section, $\sigma_{DLA}(\log M)$. I would like to receive some hints as to how to model this function numerically and if there could be any degeneracy, or non-uniqueness of its functional form, given the fact that it is not known a priori? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiar with few programming languages but just don't know how to tackle the problem numerically in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your help is greatly appreciated.   &lt;/p&gt;&#xA;" OwnerUserId="14338" LastEditorUserId="10134" LastEditDate="2016-03-27T17:51:06.183" LastActivityDate="2016-03-29T20:15:49.387" Title="How to Solve an Integral Equation for an Unknown Integrand numericlaly?" Tags="|numerical-analysis|python|computational-physics|numerical-modelling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23487" PostTypeId="1" AcceptedAnswerId="23494" CreationDate="2016-03-28T03:09:39.350" Score="-2" ViewCount="2496" Body="&lt;p&gt;I have a working C++ function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/// Structure for surface nodes&#xA;// Each node has a current x- and y-position and a reference x- and y-position.&#xA;struct nstruct {&#xA;/// Constructor&#xA;node_struct() {&#xA;    x = 0;&#xA;    y = 0;&#xA;    x_ref = 0;&#xA;    y_ref = 0;&#xA;    Ax = 0;&#xA;    Ay = 0;&#xA;}&#xA;/// Elements&#xA;double x; // current x-position&#xA;double y; // current y-position&#xA;double x_ref; // reference x-position&#xA;double y_ref; // reference y-position&#xA;double Ax; // node  (x-component)&#xA;double Ay; // node  (y-component)&#xA;};&#xA;/// Structure for object&#xA;struct pstruct {&#xA;/// Constructor&#xA;pstruct() {&#xA;    num_nodes = pnodes;&#xA;    center.x;&#xA;    center.y;&#xA;    center.x_ref;&#xA;    center.y_ref;;&#xA;    node = new node_struct[num_nodes];&#xA;}&#xA;/// Elements&#xA;int num_nodes; // number of surface nodes&#xA;node_struct center; // center node&#xA;node_struct *node; // list of nodes&#xA;};&#xA;&#xA;//&#xA;void Function(pstruct particle) {&#xA;&#xA;for (int X = 0; X &amp;lt; Nx; ++X) {&#xA;   for (int Y = 0; Y &amp;lt; Ny; ++Y) {&#xA;       Ax[X][Y] = 0;&#xA;       Ay[X][Y] = 0;&#xA;   }&#xA;}&#xA;&#xA;for (int n = 0; n &amp;lt; particle.num_nodes; ++n) {&#xA;   int xStart = static_cast &amp;lt;int&amp;gt; (particle.node[n].x - 3.0);&#xA;   int xEnd = static_cast &amp;lt;int&amp;gt; (particle.node[n].x + 3.0);&#xA;   int yStart = static_cast &amp;lt;int&amp;gt; (particle.node[n].y - 3.0);&#xA;   int yEnd = static_cast &amp;lt;int&amp;gt; (particle.node[n].y + 3.0);&#xA;   for (int X = xStart; X &amp;lt; xEnd; ++X) {&#xA;       for (int Y = yStart; Y &amp;lt;= yEnd; ++Y) {&#xA;          **// here I want to use Lagrangian Interpolation**&#xA;            const double xDistance = X - particle.node[n].x;&#xA;            const double yDistance = Y - particle.node[n].y;&#xA;            const double delta = dirac_4(xDistance, yDistance);&#xA;          // dirac_4 : is Dirac Delta function&#xA;            Ax[X][Y] += (particle.node[n].Ax * delta);&#xA;            Ay[X][Y] += (particle.node[n].Ay * delta);&#xA;        }&#xA;    }&#xA;}&#xA;return;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Its is an immersed boundary problem. I have Lagrangian points over an object (circular, or any shape), corresponding acceleration, etc. What I want to do, is to interpolate flow variables over  Euler mesh using information of Lagrangian points (defined in pstruct and nstruct). &#xA;I would like to use Lagrangian Interpolation instead of dirac_4 function to compute Ax and Ay. How I will implement Lagrangian/Polynomial interpolation in such a situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="18144" LastEditorUserId="18144" LastEditDate="2016-03-29T01:35:14.460" LastActivityDate="2016-03-29T01:35:14.460" Title="How to implement Lagrangian/Polynomial Interpolation for my C++ Code?" Tags="|c++|interpolation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23488" PostTypeId="2" ParentId="23479" CreationDate="2016-03-28T05:53:51.190" Score="1" Body="&lt;p&gt;Reiterating some of points made by Tobias and Wolfgang Bangerth:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I see no &lt;code&gt;delete&lt;/code&gt; to free memory! I highly suggest running valgrind on your code to find any memory leaks.&lt;/li&gt;&#xA;&lt;li&gt;Storing your matrix with two dimensional arrays probably isnt the best idea especially for large systems. Instead I suggest looking into other &lt;a href=&quot;http://netlib.org/linalg/html_templates/node90.html&quot; rel=&quot;nofollow&quot;&gt;sparse matrix storage formats&lt;/a&gt;, e.g. Compressed Row Storage (CRS), Compressed Column Storage, ...&lt;/li&gt;&#xA;&lt;li&gt;using std::vector is also a good idea&lt;/li&gt;&#xA;&lt;li&gt;I think writing your functions to return double pointer variables is needlessly messy.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An example of using the CRS format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;// Sample test matrix&#xA;// [2,1,0,0,1,0,0,0&#xA;//  1,2,1,0,0,1,0,0&#xA;//  0,1,2,1,0,0,1,0&#xA;//  0,0,1,2,1,0,0,1&#xA;//  1,0,0,1,2,1,0,0&#xA;//  0,1,0,0,1,2,1,0&#xA;//  0,0,1,0,0,1,2,1&#xA;//  0,0,0,1,0,0,1,2]&#xA;&#xA;&#xA;// In CRS this matrix can be represented by the three arrays:&#xA;int rows[9] = {0,3,7,11,15,19,23,27,30};&#xA;int columns[30] =  {0,1,4,0,1,2,5,1,2,3,6,2,3,4,7,0,3,4,5,1,4,5,6,2,5,6,7,3,6,7};&#xA;double values[30] =    {2,1,1,1,2,1,1,1,2,1,1,1,2,1,1,1,1,2,1,1,1,2,1,1,1,2,1,1,1,2};&#xA;&#xA;&#xA;int main()&#xA;{&#xA;    // matrix multiply y = A*x&#xA;    double x[8] = {1,2,3,4,5,6,7,8};&#xA;    double y[8];&#xA;    for(unsigned int i=0; i&amp;lt;8; i++){&#xA;        y[i] = 0;&#xA;        for(unsigned int j=rows[i]; j&amp;lt;rows[i+1]; j++){&#xA;            y[i] += values[j]*x[columns[j]];&#xA;        }&#xA;    }&#xA;&#xA;    // print arrays y &#xA;    for(unsigned int i=0; i&amp;lt;8; i++){&#xA;        std::cout&amp;lt;&amp;lt;y[i]&amp;lt;&amp;lt;std::endl;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the CRS format is efficient because it stores no non-zeros. Another nice things about this format is that by storing the sparse matrix with three one dimensional arrays means your memory is contiguous.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2016-03-28T05:53:51.190" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23489" PostTypeId="1" CreationDate="2016-03-28T07:41:15.667" Score="7" ViewCount="157" Body="&lt;p&gt;I am in the process of collecting material for a class in computational methods. It will include introductions into numerical methods for ordinary differential equations (Runge-Kutta methods, multi-step methods) and parabolic and hyperbolic partial differential equations (finite differences, spectral methods, finite volumes) and how to implement and use them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As part of that class, I would very much like to show the students an example where they can compare results from a numerical simulation with a (very simple) real world experiment. Unfortunately, I did not really find a suitable candidate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am asking if one of you can think of a very simple experiment that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;can be conducted in a normal class room (not a lab)&lt;/li&gt;&#xA;&lt;li&gt;needs only ingredients I can buy in a normal supermarket (or drugstore)&lt;/li&gt;&#xA;&lt;li&gt;can be modelled by a relatively simple mathematical equation - think diffusion equation or linear advection equation&lt;/li&gt;&#xA;&lt;li&gt;does not involve anything dangerous like fire (obviously) ;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot in advance for your help!&lt;/p&gt;&#xA;" OwnerUserId="16921" LastActivityDate="2016-03-29T09:18:49.130" Title="Very simple (real) experiment for computational methods class" Tags="|numerical-modelling|experiment|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23490" PostTypeId="1" CreationDate="2016-03-28T10:58:11.737" Score="1" ViewCount="138" Body="&lt;p&gt;I've been learning CFD theory for 4 months (which includes FEA, FDM AND FVM) and now I want to start running simulations. As a novice to CFD software, I would appreciate some advice on where to start, e.g., OpenFoam, Matlab, Fortran? (I am comfortable with programming and learning coding languages). &lt;/p&gt;&#xA;" OwnerUserId="19539" LastEditorUserId="17368" LastEditDate="2016-03-31T01:40:35.390" LastActivityDate="2016-03-31T01:40:35.390" Title="CFD implementation in software" Tags="|finite-element|finite-difference|fluid-dynamics|simulation|" AnswerCount="0" CommentCount="2" ClosedDate="2016-03-31T01:41:32.560" ContentLicense="CC BY-SA 3.0" />
  <row Id="23491" PostTypeId="2" ParentId="23489" CreationDate="2016-03-28T14:31:34.023" Score="6" Body="&lt;p&gt;Trace the arc of a ping pong ball you are shooting through the room using a slingshot. The equations that describe this are trivial (gravity acts downward, friction acts in the opposite direction of velocity and is proportional to the square of the speed). You can record the arc on a cell phone and I'm sure there is cell phone software that allows you to take apart a video frame by frame so you can determine the position (and initial velocity) using the first couple of frames.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All you need is a ping pong ball, a rubber band (for the slingshot), a tape measure (behind the person shooting the ball), and a cell phone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(All of this is inspired by having written this article: &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/100788604&quot;&gt;http://epubs.siam.org/doi/abs/10.1137/100788604&lt;/a&gt; ).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-03-28T14:31:34.023" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23492" PostTypeId="2" ParentId="23489" CreationDate="2016-03-28T15:59:50.867" Score="3" Body="&lt;p&gt;What about some of the classical physics examples shown on &lt;a href=&quot;http://www.myphysicslab.com/index.html&quot; rel=&quot;nofollow&quot;&gt;myphysicslab&lt;/a&gt;? A pendulum or spring is relatively easy to make / obtain, and furthermore you can observe the period, amplitude and chaoticity of the motion without special tools.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2016-03-28T15:59:50.867" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23493" PostTypeId="1" CreationDate="2016-03-28T16:23:26.120" Score="2" ViewCount="1495" Body="&lt;p&gt;I wanted to clarify the idea of the exact line search in steepest descent method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An exact line search involves starting with a relatively large step size ($\alpha$) for movement along the search direction $(d)$ and iteratively shrinking the step size until a decrease of the objective function is observed. Is this correct?&#xA;I.e. the only condition suppose to be satisfied is: $f(x_{k+1})&amp;lt;f(x_k)$, where $x_{k+1} = x_k + \alpha_kd_k$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In exact line search the step size $\alpha$ is not necessary to be fixed. Is this true?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is true then what is the difference of the exact line search from the backtracking line search? I know that backtracking line search is based on Armijo-Gooldstein condition or Wolfe's condition. &lt;/p&gt;&#xA;" OwnerUserId="19782" LastActivityDate="2016-03-28T16:23:26.120" Title="Exact line Search in Steepest descent" Tags="|optimization|algorithms|numerical-analysis|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23494" PostTypeId="2" ParentId="23487" CreationDate="2016-03-28T16:29:43.080" Score="0" Body="&lt;p&gt;So a general Lagrange Interpolation in 2D can be written as the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x,y) = \sum_i^N f_i L_i(x,y) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ L_i(x,y) = \prod_{j,j\neq i}^N \frac{(x-x_j)(y-y_j)}{(x_i-x_j)(y_i-y_j)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the above formulas, you should be able to program this up. The thing to note is that when $N$ is large, you end up with high order polynomial interpolants, which can result in high errors between data points. I would consider another approach unless you are interpolating using a few data points or you are certain this sort of fit would work well in your situation.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-03-28T16:29:43.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23496" PostTypeId="1" CreationDate="2016-03-28T20:26:24.697" Score="1" ViewCount="64" Body="&lt;p&gt;Given a domain $\Omega$ with 2 subdomains $\Omega_1,\Omega_2$ as figure below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EzZ5i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EzZ5i.png&quot; alt=&quot;Domain&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Level set equation $$ \partial_t\varphi + U\cdot\nabla\varphi =0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U$ is velocity only presents inside $\Omega_1$. We assume $U$ is irrotational, i.e. $\nabla \times U=0$, which implies that the velocity ﬁeld can be derived from a potential function $\Phi$ defined in $\Omega$ such that $U=\nabla\Phi$. The potential equation is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{cases}&#xA;-\nabla\cdot(\nabla\Phi)&amp;amp;=-\beta_S \quad \text{in }\Omega_1\\&#xA;-\nabla\cdot(\nabla\Phi)&amp;amp;=0 \quad \text{in }\Omega_2&#xA;\end{cases}&#xA;$$ with $\beta_S$ given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve this level set equation, especially with unfitted mesh but I don't know the problem is well-posed or not.&lt;/p&gt;&#xA;" OwnerUserId="4670" LastEditorUserId="4670" LastEditDate="2016-03-29T07:17:48.023" LastActivityDate="2016-03-29T07:17:48.023" Title="Check the well-posedness of the problem with level set equation" Tags="|finite-element|numerical-analysis|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23498" PostTypeId="2" ParentId="23489" CreationDate="2016-03-29T09:18:49.130" Score="2" Body="&lt;p&gt;Elastica theory / Euler Bernoulli beam theory is a very good candidate. Equations are relatively simple (especially in the linear case) and there is generally a very good agreement between experiments and mathematical model. Experimental setup is easy (if you work with very slender beams and low loads) and physical parameters to be identified are geometrical (very easy to measure with a caliper) and material Young Modulus (which could be estimated from engineering books, or left as a tunable parameter to be estimated by some preliminary experiment.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g., with three point bending, record the load--deflection curve. Use initial tangent stiffness to estimate Young Modulus, and then compare experimental to analytical and analytical to numerical results. Afterwards you change boundary conditions and solve other cases...&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2016-03-29T09:18:49.130" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23499" PostTypeId="1" CreationDate="2016-03-29T10:38:15.513" Score="6" ViewCount="933" Body="&lt;p&gt;So given an objective function $f({\bf x})$, I would like to include a positivity constraint when I perform the fixed point iteration:&#xA;$${\bf x}^{(t+1)}={\bf x}^{(t)} - \text{H}_f^{-1}\nabla f({\bf x}^{(t)}),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for ${\bf x}^{(0)}&amp;gt;{\bf 0}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way I've found to do it is to add a log barrier to the objective function so it becomes &#xA;$$g({\bf x}) = f({\bf x})+\frac{1}{n}\sum_{i=1}^n\log{\bf x}_i,$$&#xA;since this will cause $\nabla g$ to shoot to infinity as one approaches $\bf 0$ from the positive orthant.  And then to modify the iteration step size to prevent the step size from being large enough to cross all the way over the log barrier to the other side:&#xA;$${\bf x}^{(t+1)}={\bf x}^{(t)} - \frac{1}{2}\min\Big[2, \frac{\left\|{\bf x}^{(t)}\right\|}{\left\|\text{H}_g^{-1}\nabla g({\bf x}^{(t)})\right\|}\Big]\text{H}_g^{-1}\nabla g({\bf x}^{(t)}).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the best way to enforce a positivity constraint on Newton's Method, or is there a better way? &lt;/p&gt;&#xA;" OwnerUserId="6849" LastActivityDate="2016-03-29T10:38:15.513" Title="Best way to add a positivity constraint to Newton's Method" Tags="|optimization|constrained-optimization|newton-method|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23500" PostTypeId="2" ParentId="23415" CreationDate="2016-03-29T11:23:23.067" Score="0" Body="&lt;p&gt;I did a bunch of tests, and it seems the answer to my question is that it does have an effect to use interpolation with Runge-Kutta. However, using a shorter timestep in the integrator than in the input data does not seem to have a large effect (I still need to interpolate, though, since the Runge-Kutta method evaluates the derivatives at the halfway point in time). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have only tested this for a case where the ratio between spatial and temporal resolution in the input data are such that I will usually move only a fraction (1/10 - 1/5) of the spatial grid spacing during one timestep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, the choice of interpolation scheme seems to have an effect in some cases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://folk.ntnu.no/nordam/160329/interpolation_2_small.png&quot; alt=&quot;Particle transport 1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://folk.ntnu.no/nordam/160329/interpolation_1_small.png&quot; alt=&quot;Particle transport 1&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the figures, I have transported particles for 2 days (top) and 3 days (bottom), using 4th order Runge-Kutta, with the interpolation scheme given by the legend. The x and y coordinates are in meters. For each scheme, I have used different timesteps: 2 hours, 1 hour, 0.5 hours and 0.25 hours. The timestep in the input data is 1 hour. Essentially, lowering the timestep below that of the input data has very little effect on the results. Going from linear interpolation to cubic splines has a large effect in one of the two cases shown below, though. It took about three times as long to run the program with splines interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess the only way to be sure which approach is &quot;most correct&quot; would be to get higher resolution data from the same source, and then compare trajectories derived from the high and low resolution data.&lt;/p&gt;&#xA;" OwnerUserId="19544" LastEditorUserId="-1" LastEditDate="2017-03-10T09:42:38.143" LastActivityDate="2016-03-29T11:23:23.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23502" PostTypeId="2" ParentId="23257" CreationDate="2016-03-29T11:46:43.963" Score="2" Body="&lt;p&gt;Your post actually contains two questions:&lt;/p&gt;&#xA;&lt;h2&gt;1) What should you calculate&lt;/h2&gt;&#xA;&lt;p&gt;This first question will be answered by defining what you are studying. If it is the magnetic properties of your system (usual interest of Ising models) you can calculate the energy of your state by calculating the sum of the energy of each components:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ U_i = - \sum_i \textbf{m}_i\cdot \textbf{E}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$m_i$&lt;/span&gt; is the magnetic moment of the component &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; the electric field. Yet this represents only an instantaneous energy of the whole system which you can't observe directly in a real physical system due to his natural fluctuations. Contrarly to your saying, in equilibrium at microscopic level, the energy is not always the same for each configurations at equilibrium. The algorithm has a non-zero probability to accept higher energy configurations and I recommend you to test higher temperature systems Hence we observe the mean of a serie of records (forming your data): &lt;span class=&quot;math-container&quot;&gt;$$ U = \frac{1}{n} \sum_i^n U_i$$&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of records you'll keep. Furthermore you can calculate standard deviation and other statistical properties that are useful for any physical calculations (check any good statistical and computational physics textbook like the Shang-Keng Ma or the one of Werner Krauth).&lt;/p&gt;&#xA;&lt;h2&gt;2) What data must I use to calculate it?&lt;/h2&gt;&#xA;&lt;p&gt;This second question is a bit more challenging as you have to define if your system has reached equilibrium. For simple Ising models this is not really a problem since you'll just observe the fluctuations of energy in you're last records. If they are small enough it's a win but if you're working in non-micro-canonical ou non-canonical ensembles for systems with highers degrees of freedom, it can become quite tedious to define whether or not your model finally converged. Usually if your data did not converge to an acceptable mean result, you should continue your simulation starting from the configuration you end up with. Else you can (more or less) arbritrary define a starting point for your statistical analysis, depending on the convergence of your data. You also have to take some time between the different records of your values for them to be statistically independent. For example, only record one out of one thousand configuration.&lt;/p&gt;&#xA;" OwnerUserId="19790" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-03-29T18:25:01.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23503" PostTypeId="1" CreationDate="2016-03-29T12:51:26.913" Score="1" ViewCount="742" Body="&lt;p&gt;Given the advection equation for an incompressible flow field&#xA;$$\frac{\partial c}{\partial t} + \mathrm{Pe} \frac{\partial c}{\partial x} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what would the best method be for discretizing this without introducing any numerical diffusion or oscillations? Specifically when we have step changes in boundary conditions, and time (and space) dependent velocity $v(x,t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book &lt;em&gt;Numerical Methods for Problems with Moving Fronts&lt;/em&gt; by Bruce A. Finlayson goes into great detail on this problem when you have step changes as boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He recommends filtered leapfrog as the best &lt;strong&gt;finite difference&lt;/strong&gt; method:&#xA;$$c^{n+1}_i = \frac{\alpha}{2}\left(c_i^n + c_i^{n+2}\right) + (1-\alpha)c_i^{n-1} - &#xA;\frac{\mathrm{Pe}\Delta t}{\Delta x}\left(c_{i+1}^n - c_{i-1}^n\right)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;em&gt;filtered&lt;/em&gt; means $\alpha = 1$) and Taylor-Galerkin as the best &lt;strong&gt;finite element&lt;/strong&gt; method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{6}\left(c_{i+1}^{n+1} - c_{i+1}^{n}\right) &#xA;+ \frac{2}{3}\left(c_{i}^{n+1} - c_{i}^{n}\right) &#xA;+ \frac{1}{6}\left(c_{i-1}^{n+1} - c_{i-1}^{n}\right) &#xA;= -\frac{\mathrm{Pe}\Delta t}{\Delta x} \left(c^{n}_{i+1} - c^{n}_{i-1}\right) &#xA;+ \frac{\mathrm{Pe}^2\Delta t^2}{\Delta x^2}\left(c^{n}_{i+1} - 2c_i^n + c^{n}_{i-1}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I was wondering what the consensus is today? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested the mentioned methods, and they both suffer from numerical and oscillation. But perhaps this can be managed by adjusting $\Delta t$ and $\Delta x$ in relation to the velocity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/7423/19794&quot;&gt;This&lt;/a&gt; answer to a related question regarding Crank-Nicholson and the advection equation states that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Crank-Nicolson is not necessarily the best method for the advection equation. It is second order accurate and unconditionally stable, which is fantastic. However it will generate (as with all centered difference stencils) spurious oscillation if you have very sharp peaked solutions or initial conditions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;but gives no alternative methods.&lt;/p&gt;&#xA;" OwnerUserId="19794" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-03-30T08:53:31.083" Title="Discretization method for advection equation without numerical diffusion" Tags="|discretization|advection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23504" PostTypeId="1" AcceptedAnswerId="23535" CreationDate="2016-03-29T13:39:35.613" Score="5" ViewCount="410" Body="&lt;p&gt;Assume I know eigen-pairs $(\epsilon_i,\vec\phi_i)$ for matrix operator $\hat H$ that $ \hat H \vec \phi_i = \epsilon_i \vec \phi_i $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have &lt;strong&gt;slightly perturbed&lt;/strong&gt; $\hat H' = \hat H + \hat R$ were perturbation $\hat R$ is small with respect to $\hat H$ (in some sence, e.g. Frobenius norm ?). Now I would like to find eigen-pairs of the perturbed $\hat H'$. &#xA;$\hat H' \vec \phi'_i = \epsilon'_i \vec \phi'_i $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect there should exist &lt;strong&gt;some efficient iterative algorithm&lt;/strong&gt; which use the knowledge of solution $ \hat H \vec \phi_i = \epsilon_i \vec \phi_i $ to obtain solution of $\hat H' \vec \phi'_i = \epsilon'_i \vec \phi'_i $ with modest computational effort ( measured in CPU flops ) especially if I work with &lt;strong&gt;sparse matrices&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/etemplates/node402.html&quot; rel=&quot;nofollow&quot;&gt;Davidson Method&lt;/a&gt; what I search for ? Or, in general, I can use the known solution of un-perturbed Hamiltonian to construct some preconditioner and initial vectors for &lt;a href=&quot;https://en.wikipedia.org/wiki/Arnoldi_iteration&quot; rel=&quot;nofollow&quot;&gt;Arnoldi&lt;/a&gt;/&lt;a href=&quot;https://en.wikipedia.org/wiki/Lanczos_algorithm&quot; rel=&quot;nofollow&quot;&gt;Lancozs&lt;/a&gt; iteration.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;background:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not expert in linear algebra, nor iterative methods. I'm just trying to familiarize myself with the field by googling. I downloaded some papers about Davidson / Arnoldi / Lanczos methods, but at first look I wasn't able to extract clear answer to even the very basic question &lt;em&gt;if it is relevant to my problem.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read wikipage &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenvalue_perturbation&quot; rel=&quot;nofollow&quot;&gt;Eigenvalue perturbation&lt;/a&gt; but it doesn't concern with explicit numerical algorithm and it computational cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mostly I'm interested in case when $\hat H$ and $\hat H'$ are hermitian (ground state Hamiltonian of some quantum system), but for more general understanding I do not want to limit the question on that case.&lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="4696" LastEditDate="2016-03-31T08:21:14.270" LastActivityDate="2016-03-31T14:54:22.387" Title="Eigenvectors of slightly perturbed matrix" Tags="|eigenvalues|iterative-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23505" PostTypeId="1" AcceptedAnswerId="33153" CreationDate="2016-03-29T13:43:57.357" Score="2" ViewCount="61" Body="&lt;p&gt;I asked a similar question on the Mathematics stack exchange &lt;a href=&quot;https://math.stackexchange.com/questions/1710701/ordering-of-eigenvectors-to-maximise-trace-of-diagonalising-matrix&quot;&gt;here&lt;/a&gt; without much success, so I thought I'd ask it with a more practical bent here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have a Hermitian matrix $H$ with (for the moment) distinct eigenvalues. Let $\{u_1,u_2,\ldots,u_n\}$ be an orthonormal eigenbasis. Most linear algebra packages I know will order these vectors so that the corresponding eigenvalues are in ascending order. However, instead I would like them to be ordered so that the trace of the diagonalising matrix $U$ is maximised; i.e. I would like to find $\max\limits_{\sigma\in S(n)}\sum_j|u_{\sigma(j)j}|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to avoid simply exhaustively searching the space? Possibly in special cases or to find an approximate solution? For example if $H$ is close to diagonal one would expect $U$ to be close to the identity and thus each $u_j$ to have a unique maximum at different indices; one could then find the location of the maximum for each eigenvector separately and order appropriately, but this breaks down as soon as an eigenvector does not have a unique maximum or two eigenvectors have their maxima at the same index.&lt;/p&gt;&#xA;" OwnerUserId="19795" LastActivityDate="2019-07-31T18:02:36.943" Title="Ordering of eigenvectors to maximise trace of diagonalising matrix" Tags="|linear-algebra|optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23510" PostTypeId="1" AcceptedAnswerId="23512" CreationDate="2016-03-29T16:09:09.287" Score="1" ViewCount="103" Body="&lt;p&gt;There are two independent tetrahedral FEM grids. Second grid is subset of the first. By subset, I mean: nodes from the second grid are exactly in the same positions as some nodes from the first grid. The numbering of nodes is completely different in both grids. I need to renumber nodes from the second grid according to nodes from first one to properly transfer the solution. To do it I need to create some kind of map between nodes ids. Most naive solution is to create a dictionary where the keys contain nodes positions and value is node 'id'. But using arrays of doubles as a keys is not a great idea due to floating point math (precision issue). &lt;/p&gt;&#xA;" OwnerUserId="1010" LastActivityDate="2016-03-29T17:22:54.820" Title="Common nodes in two FEM grids" Tags="|python|mesh|unstructured-mesh|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23511" PostTypeId="2" ParentId="23484" CreationDate="2016-03-29T16:10:09.337" Score="1" Body="&lt;p&gt;First, let me reframe your problem in simpler notation: by defining&#xA;\begin{align}&#xA;\ell &amp;amp;:= \log M \\&#xA;\ell_0 &amp;amp;:= \log M_{\text{min}} \\&#xA;DN(\ell) &amp;amp;:= \frac{dN_{DLA}}{dz}(\log M, z=0) \\&#xA;Dn(\ell) &amp;amp;:= \frac{dn_h}{d(\log M)}(\log M) \\&#xA;f(\ell) &amp;amp;:= \sigma_{DLA}(\log M)&#xA;\end{align}&#xA;you can write your equation as&#xA;$$DN(\ell_0) = \int_{\ell_0}^{\infty} Dn(\ell)\, f(\ell)\, d\ell$$&#xA;You can apply the fundamental theorem of calculus to this and easily determine that&#xA;$$f(\ell) = -\frac{1}{Dn(\ell)}\frac{d}{d\ell}DN(\ell)$$&#xA;Or in your notation,&#xA;$$\sigma_{DLA}(\log M) = -\biggl(\frac{dn_h}{d(\log M)}(\log M)\biggr)^{-1}\frac{d}{d(\log M)}\frac{dN_{DLA}}{dz}(\log M, z=0)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know $\frac{dN_{DLA}}{dz}(\log M, z=0)$ and $\frac{dn_h}{d(\log M)}(\log M)$ for a range of values of $\log M$, then you can easily calculate the unknown function $\sigma_{DLA}$ over that same range of $\log M$ (maybe except for a bit near the endpoints) using numerical differentiation. Any numerical library you care to use should include functions for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only know $\frac{dN_{DLA}}{dz}(\log M_{\text{min}}, z=0)$ at the &lt;em&gt;one&lt;/em&gt; value of $M = M_{\text{min}}$, then your problem is underconstrained, because $\sigma_{DLA}$ could be basically anything. Specifically, you could guess pretty much any form for $\sigma_{DLA}(\log M)$ (any form that allows the integral to be defined, anyway), and up to an overall normalization constant, you can satisfy your equation with that form. So with only one data point for $\frac{dN_{DLA}}{dz}$, you cannot determine anything useful about $\sigma_{DLA}$. (I'm not sure if that's what you meant.)&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2016-03-29T16:10:09.337" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23512" PostTypeId="2" ParentId="23510" CreationDate="2016-03-29T17:22:54.820" Score="5" Body="&lt;p&gt;Hashing floating-point numbers can indeed lead to weird results, especially if the node positions can be perturbed by some small amount or if there are denormalized values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You included the Python tag, so I assume that's what you're using and that you have scipy. A quick and dirty solution would be to construct a &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.KDTree.html&quot;&gt;kd-tree&lt;/a&gt; of the fine grid points, then for each point in the coarse grid, compute its nearest neighbor in the fine grid. This will work in $O(n \log n)$ rather than the $O(n^2)$ of a naive nested loop, without the issues of floating-point weirdness. There may be a faster way to do the same thing, but in these applications you very often need a spatial tree data structure in the first place.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-03-29T17:22:54.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23513" PostTypeId="2" ParentId="23503" CreationDate="2016-03-29T17:48:21.777" Score="2" Body="&lt;p&gt;If your aim is to solve one-dimensional advection equation with variable velocity having minimal numerical diffusion and no unphysical oscillations and you have no other requirements (mass conservation?!) then in my opinion the so called semi-Lagrangian schemes may be your choice. By the way, using the notation Pe in your equation is a little bit confusing to me as it reminds a &quot;Peclet&quot; number that relates advection and diffusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The semi-Lagrangian methods are based on the fact that the solution is constant along characteristic curves (the &quot;characteristics&quot;), therefore to determine $c_i^{n+1}$ you have to find a characteristic $X(t)$ that ends in time $t^{n+1}$ in the grid point $x_i$. You can obtain it numerically by solving ordinary differential equation &#xA;$$&#xA;\dot X(t) = v(X(t),t) , \,\, X(t^{n+1})=x_i \,, \,\, t \in (t^n,t^{n+1})&#xA;$$&#xA;back in time. Computing then the value of numerical solution at time $t^n$ in the position $X(t^n)$ will give you the value of $c_i^{n+1}$. This last task shall be done by some proper numerical interpolation of nodal values $c_j^n$ that will give you no unphysical values (e.g. a linear interpolation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the characteristics will cross the boundary of your domain (interval) for some $\tilde t \in (t^n,t^{n+1})$ then you must exploit the boundary condition, typically a given value at time $\tilde t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This class of methods has a great advantage of no formal restriction on the time step of method. Of course, you must work with velocity $v(x,t)$ that is smooth enough. If you have sinks or sources in your velocity field, e.g. points where two characteristics meet or start, then some careful treatment in the method shall be realized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you for some reasons prefer purely Eulerian type of methods (e.g. finite difference or finite volume or finite element method) then you should be aware that if you allow no unphysical oscillations then you can have at most second order accurate method, so to obtain accurate numerical solution (&quot;less numerical diffusion&quot;), fine grids shall be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addendum&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method called &quot;Taylor-Galerkin finite element method&quot; (FEM) in the question is closely related to the so called Lax-Wendroff method for finite volume (or difference) method. In fact, if one uses a mass lumping in the left hand side of FEM, so it takes the form $c_i^{n+1}-c_i^n$, then it is identical to Lax-Wendroff method for constant Pe. This method is second order accurate, but it results in numerical oscillations. To avoid them, a so called limiter must be used that reduces the scheme in some grid points to the direction of first order accurate scheme. This topic is very well described in e.g. book of R. LeVeque on Finite Volume Methods for Hyperbolic Problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better choice than Lax-Wendroff concerning the accuracy (a so called phase error) is the so called Fromm method that includes in the right hand side also the numerical values $c_{i-2}^n$ or $c_{i+2}^n$ depending on the sign of Pe. To have no oscillations, again a limiter must be used, but this is the case for any second order accurate scheme. This method would be my choice if its restriction on the time step is not an issue (the CFL condition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More advanced option than the Fromm scheme with a limiter is the approach e.g. of Chi-Wang Shu and collaborators On maximum-principle-satisfying high order schemes for scalar conservation laws or Maximum-principle-satisfying high order finite volume weighted essentially nonoscillatory schemes for convection-diffusion equations. This would be my recommendation if more time to the implementation can be invested.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-03-30T08:53:31.083" LastActivityDate="2016-03-30T08:53:31.083" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23514" PostTypeId="2" ParentId="21986" CreationDate="2016-03-29T18:40:30.563" Score="3" Body="&lt;p&gt;Well I guess that this question could take place in both computational science and physics, depending on the kind of physicist or computational scientist stubbling upon it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet for your answer, as I saw some solid MC algorithm, I advise you to put your rejection goal (or acceptation) as a constant inside your code and vary your trial size proportionnaly to the actual trial size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean is that you calculate the rate of rejection $\frac{n_{rejected}}{n_{total}}$, compare it with the goal and update your trial size to, let's say, something like $\pm10\%$ of it $$\delta\theta_{new}=\delta\theta\pm0.1\delta\theta$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that if the trial size is larger you'll end up with larger $e^{\Delta U}$ meaning more rejection so check the $\pm$ sign wisely depending on the rate of rejection. Also verify that $n_{total}$ is only considered with entries recorded since the last update of $\delta\theta$.&#xA;This stuff should converge to a solution exploring a useful portion of the phase space and avoiding very low probable region at lower temperatures.&lt;/p&gt;&#xA;" OwnerUserId="19790" LastActivityDate="2016-03-29T18:40:30.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23515" PostTypeId="2" ParentId="23484" CreationDate="2016-03-29T20:15:49.387" Score="1" Body="&lt;p&gt;I am going to use the notation that &lt;strong&gt;David Z&lt;/strong&gt; used, to help simplify things. I will also define the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \langle g,f\rangle_k = \int_{l_k}^\infty g(l)f(l)dl $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's assume you can compute $DN(l)$ and $Dn(l)$ at a set of $k$ values for $l_0$, which I will just call $l_k$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's also assume you can approximate $f(l)$ with a finite basis of size $m\leq k$, like so:&#xA;$$ f(l) = \sum_{j=1}^m a_j \phi_j(l) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\phi_j$ is some basis function and $a_j$ are coefficients to solve for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this representation, you should end up with $k$ equations with $m$ unknowns, with the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ DN(l_k) = \sum_{j=1}^m a_j \langle Dn,\phi_j\rangle_k \;\;\;\;\;\;\forall k $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the number of numerically unique values for $l_k$ that you can compute, you should be able to find a suitable basis to approximate $f(l)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only have one value for $l_k$, you might just want to assume a constant value for $f(l)$ and compute that approximation. &lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-03-29T20:15:49.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23516" PostTypeId="1" AcceptedAnswerId="23520" CreationDate="2016-03-30T09:41:01.877" Score="1" ViewCount="65" Body="&lt;p&gt;this is a classical problem, but I need help to pinpoint what I am missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem:&#xA;In MATLAB &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(exp(1) + 10^12) - 10^12  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;gives you a double which equal to e, up to 5 correct digits. But I thought it would be 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One has exp(1) + 10^12 = 1.000000000002718e+12, where 1.000000000002718 contains 16 digits, the precision we are working at. All that is left of e is 2.718, but (exp(1) + 10^12) - 10^12 = 2.7182 (I have left out the digits following the last decimal 2), meaning I have one more digit saved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question:&#xA;Where or how is the extra digit stored? I know that the last the 53 bit for unsigned 64 bit IEEE is used for rounding, is it related to my question?&lt;/p&gt;&#xA;" OwnerUserId="19807" LastActivityDate="2016-03-30T12:05:54.910" Title="MATLABs double arithmetic" Tags="|matlab|error-estimation|computer-arithmetic|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23517" PostTypeId="2" ParentId="23208" CreationDate="2016-03-30T09:46:57.377" Score="1" Body="&lt;p&gt;I have limited experience with Ewald, but I found this thesis to be helpful when working with it. I do not know if it contains the answer to your question, but it might contain useful references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A458575&amp;amp;dswid=9118&quot; rel=&quot;nofollow&quot;&gt;http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A458575&amp;amp;dswid=9118&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19807" LastActivityDate="2016-03-30T09:46:57.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23519" PostTypeId="2" ParentId="23208" CreationDate="2016-03-30T10:43:35.657" Score="0" Body="&lt;p&gt;I have much more experience with DL_POLY 4 (being a contributor to the code) which is a successor to DL_POLY classic, but you can get &quot;sensible&quot; values for the Ewald parameters by use of the procedure outlined in section 3.2.5.1 &quot;Ewald sum and SPME&quot; on page 89 of the manual where it suggests including a &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ewald precision 1d-6&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;directive in your CONTROL file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also consider using the SPME method for the Ewald simulation rather than the classic formulation as you seem to be doing. For large systems this should be very much quicker. It is described on pages 48-50 of the manual, and to use it simply replace &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ewald&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;wherever you use it in the CONTROL file with&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;spme&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you go this route you might want to consider using an optimised FFT, such as FFTW, as the one supplied with DL_POLY classic is not that quick.&lt;/p&gt;&#xA;" OwnerUserId="17475" LastActivityDate="2016-03-30T10:43:35.657" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23520" PostTypeId="2" ParentId="23516" CreationDate="2016-03-30T11:58:58.373" Score="1" Body="&lt;p&gt;You can use &lt;code&gt;dec2bin(typecast((exp(1) + 1e12) - 1e12, 'uint64'))&lt;/code&gt; to obtain the binary representation of the double precision floating point.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;100000000000101101111110000101010001011000101000101100000000000&#xA;  100001001101101000110101001010010100010000000000101100000000000&#xA;  100000000000101101111110000000000000000000000000000000000000000&#xA;  100000000000101101111100111011011001000101101000011100000000000&#xA;  100000000000101101111101101111110100100001111111110010000000000&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;are the binary representations of&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;e&lt;/li&gt;&#xA;&lt;li&gt;e + 1e12&lt;/li&gt;&#xA;&lt;li&gt;(e + 1e12) - 1e12&lt;/li&gt;&#xA;&lt;li&gt;2.718&lt;/li&gt;&#xA;&lt;li&gt;2.7182&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2016-03-30T11:58:58.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23521" PostTypeId="2" ParentId="23516" CreationDate="2016-03-30T12:05:54.910" Score="1" Body="&lt;p&gt;Firstly, I can't guarantee it, but I suspect that matlab will return a signed double rather than an unsigned, so you've probably only got 52 bits to play with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However to actually address the question, consider a clock that has stopped with the hour hand on 1 o'clock. It has a terrible precision, but if you look at it at 12.30, it's accurate to within half an hour. Similarly, the $(1/2)^{52}$ step in the representation of doubles is only useful to tell us the &lt;strong&gt;maximum&lt;/strong&gt; truncation error in the representation of mathematics, the actual accuracy can be far better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a concrete example of something closer to the bound, compare your example with the result of  $(\pi+10^{12})-10^{12}$.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-03-30T12:05:54.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23522" PostTypeId="1" CreationDate="2016-03-30T12:58:55.013" Score="0" ViewCount="92" Body="&lt;p&gt;I have a function evaluated on a regular 5D grid with 21 points per dimension (so $21^{5}$ points total).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to evaluate the integral of the function over all 5 dimensions, so I was planning on using one of the composite Newton-Cotes formulae (i.e. trapezium rule, Simpson's rule, Boole's rule etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Assuming we can't re-evaluate the function on a different grid, is there a way of approaching this that is significantly better than Newton-Cotes?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If we are using Newton-Cotes, is higher-order always better? More specifically is it always better to divide the data into a smaller number of intervals evaluated with a higher-order scheme rather than a larger number of intervals evaluated with a lower-order scheme?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6700" LastEditorUserId="127" LastEditDate="2017-04-21T02:42:20.277" LastActivityDate="2017-04-21T02:42:20.277" Title="Choice of Newton-Cotes formulae for regularly gridded multi-dimensional data" Tags="|numerical-analysis|quadrature|integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23523" PostTypeId="1" AcceptedAnswerId="23533" CreationDate="2016-03-30T14:44:18.453" Score="0" ViewCount="68" Body="&lt;p&gt;Given a set of integers $\{l_1,l_2,\ldots,l_n\}$, where each integer is associated with a set $m_k\in\{-l_k,-l_k+1,\ldots,l_k\}$, I need to find all combinations $\{m_1,m_2,\ldots,m_n\}$ that sum to a given integer $M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, let's say the input is $\{1,3\}$ and $M=2$. This means that $m_1\in\{-1,0,1\}$ and $m_2\in\{-3,-2,\ldots,3\}$. The desired combinations are then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\{m_1,m_2\}=\{-1,3\},\{0,2\},\{1,1\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For only two integers $l_1$ and $l_2$ this is easy, but a bit more involved for a general number. I have a brute force algorithm implemented which finds &lt;em&gt;all&lt;/em&gt; possible combinations and then picks those with the right sum, but this quickly becomes too slow when the sets grow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm wondering if there is a fast algorithm to find these (doesn't have to be the fastest in existence if sacrificing a little speed gives one easier to implement)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure this is a well known problem, but googling I couldn't find anything.&lt;/p&gt;&#xA;" OwnerUserId="7265" LastActivityDate="2016-04-01T16:27:05.440" Title="Picking n integers from n different sets summing to a given value" Tags="|combinatorics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23524" PostTypeId="1" AcceptedAnswerId="23526" CreationDate="2016-03-30T15:06:43.780" Score="3" ViewCount="520" Body="&lt;p&gt;I wrote a number of 1D/2D FE and FD programs as a bachelor student, but the main problem I continually came into contact with was gradient shocks related to convection/diffusion problems in convection-dominated flows. From my limited research this seems to be an inherent problem in continuous Galerkin FE methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm interested in jumping into the DG FE world. I've been doing some reading on the various forms of DG available, but I'm not really sure where to start. I've read about hybrid CG/DG methods, various limiters, h/p adaptivity, etc. All of this is really interesting, but difficult to find a beginning to start with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it worth just focusing on the first paper by Reed and Hill from '73? Is there a more recent paper/textbook/lecture notes that gives a decent intro into DG methods without any overly specific information for a particular application? I'm looking to start but don't know where..&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="549" LastEditDate="2016-03-30T16:14:42.973" LastActivityDate="2016-03-31T07:15:01.847" Title="Intro to DG Finite Element methods" Tags="|finite-element|reference-request|discontinuous-galerkin|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23525" PostTypeId="2" ParentId="23522" CreationDate="2016-03-30T15:46:06.897" Score="4" Body="&lt;p&gt;First, it can be found that Newton-Cotes can suffer from Runge's Phenomena when you use higher order version of it (because of the underlying Lagrange interpolants used in the formulation). So I suspect there is some optimal Newton-Cotes approach which does a balance of using a refined number of intervals with a lower order approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, a quadrature that might be better is Gauss Quadrature, which uses $n$ points to integrate a $2n-1$ order polynomial (approximation) exactly.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-03-30T16:01:47.827" LastActivityDate="2016-03-30T16:01:47.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23526" PostTypeId="2" ParentId="23524" CreationDate="2016-03-30T16:22:06.330" Score="10" Body="&lt;p&gt;For parabolic/elliptic PDE's, I highly recommend Beatrice Riviere's book: &lt;strong&gt;Discontinuous Galerkin methods for solving elliptic and parabolic equations: theory and implementation&lt;/strong&gt;.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For hyperbolic PDE's and general (i.e. nonlinear) conservation laws, I recommend Hesthaven &amp;amp; Warburton's book: &lt;strong&gt;Nodal discontinuous Galerkin methods: algorithms, analysis, and applications&lt;/strong&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both books include matlab code that you can step through in a debugger to understand how they are really implemented.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="3441" LastEditDate="2016-03-30T16:24:53.230" LastActivityDate="2016-03-30T16:24:53.230" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23527" PostTypeId="1" AcceptedAnswerId="23528" CreationDate="2016-03-30T17:36:44.097" Score="1" ViewCount="154" Body="&lt;p&gt;It is necessary for me to find the shape functions on $L^2(\Omega)$(piecewise constant functions),&#xA;I searched a lot but that, could not find anything, all of them are about higher degree polynomial.&#xA;If we define the space as follow:&#xA;$${\cal Dp0}=\{v \in L^2(\Omega):v|_T \in {\cal p0(T)}, \forall T \in {\cal T}\}$$ &#xA;where ${\cal T}$ is triangulation of the domain.&#xA;How can I find them?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2016-03-31T07:56:55.073" LastActivityDate="2016-03-31T07:56:55.073" Title="How to construct shape functions in the $L^2(\Omega)$" Tags="|finite-element|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23528" PostTypeId="2" ParentId="23527" CreationDate="2016-03-30T18:46:21.610" Score="5" Body="&lt;p&gt;$p_0(T)$ is the space of all functions that are &lt;em&gt;constant&lt;/em&gt; on cell $T$. The shape function that corresponds to this is the function that is 1 on cell $T$ and zero everywhere else. Its derivative is then obviously zero in the cell itself, as well as outside the cell, but it is undefined at the interface.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-03-30T18:46:21.610" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23530" PostTypeId="2" ParentId="19255" CreationDate="2016-03-30T20:58:24.203" Score="0" Body="&lt;p&gt;A common trick to accelerate SVD of skinny matrices is to firstly apply a QR decomposition to it and than you do a SVD in the smaller matrix R. This gives you A = QR = QSVD, where QS is orthogonal, then QSVD is an SVD decomposition. Maybe, MATLAB does check the shape of the matrix before applying the SVD and use some similar trick.&lt;/p&gt;&#xA;" OwnerUserId="17141" LastActivityDate="2016-03-30T20:58:24.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23533" PostTypeId="2" ParentId="23523" CreationDate="2016-03-31T05:49:41.167" Score="1" Body="&lt;p&gt;I thought this might be a fun problem to solve, so I cranked out a solution for it based on the comment I made in the problem statement. The class representing the solution, which is in C++, can be found via the following link:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/choward1491/Bones/tree/master/src/Algorithms&quot; rel=&quot;nofollow&quot;&gt;Combinatorial Solution Class&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main.cpp file can be written with the following example code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &quot;CombinatorialSoln.hpp&quot;&#xA;&#xA;&#xA;int main( int argc, char** argv ){&#xA;&#xA;// Initialize combinatorial problem&#xA;CombinatorialSoln soln;&#xA;std::vector&amp;lt;int&amp;gt; iset {1, 3, 2, 5, 6, 3};&#xA;soln.setIntegerSet(iset);&#xA;&#xA;// Compute combinations&#xA;CombinatorialSoln::Combos combos = soln.computeCombinations(-5);&#xA;&#xA;// print results&#xA;for(int i = 0; i &amp;lt; combos.size(); i++ ){&#xA;    combos[i].print();&#xA;}&#xA;&#xA;// Print basic results&#xA;soln.showStatistics();&#xA;&#xA;// Complete&#xA;return 1;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-04-01T16:27:05.440" LastActivityDate="2016-04-01T16:27:05.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23535" PostTypeId="2" ParentId="23504" CreationDate="2016-03-31T14:54:22.387" Score="2" Body="&lt;p&gt;One of the best strategies for finding particular eigenvalues of linear operators is shift-and-invert Lanczos; if you're looking for an eigenvalue of the matrix $A$ close to the value $\sigma$, then you run the Lanczos algorithm on the operator $(A - \sigma I)^{-1}$. The spectrum of this operator will be much better separated than the spectrum of $A$, and since the Lanczos algorithm is great at finding extreme eigenvalues, it will pick out the eigenvalue close to $\sigma$ much faster. Of course, one then has to solve linear systems for $A - \sigma I$, which may be expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you know the eigenvalue decomposition of $H$, so you can easily solve linear systems $(H - \sigma I)\phi = f$. For a given $f$, the solution $\phi$ should be fairly close to the solution of the system $(H' - \sigma I)\phi' = f$ for the perturbed Hamiltonian, provided that $\sigma$ is not too large. So you can leverage your solution of the unperturbed problem by using $H - \sigma I$ as a preconditioner for $H' - \sigma I$ in a Krylov subspace method such as MINRES, rather than a generic and possibly more expensive method such as $LDL^*$-factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a blunt approach, you can take $\sigma$ to be the eigenvalue $\epsilon_i$ of $H$; better yet, you can use the usual perturbation theory estimates from quantum mechanics for $\epsilon_i'$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This strategy is generally most effective if you're looking for the ground state or the first few excited states. I haven't tried the Davidson method myself so I don't know how one might apply the same reasoning as for shift-and-invert Lanczos. This is discussed in the book &lt;a href=&quot;https://books.google.com/books?isbn=1611970733&quot; rel=&quot;nofollow&quot;&gt;Numerical Methods for Large Eigenvalue Problems&lt;/a&gt; by Yousef Saad, which is (in my opinion) the best resource on this subject. Saad has also done a lot of work on finding interior eigenvalues, which is a substantially harder problem.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-03-31T14:54:22.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23536" PostTypeId="1" AcceptedAnswerId="26329" CreationDate="2016-03-31T16:17:36.280" Score="11" ViewCount="6458" Body="&lt;p&gt;I try to familiarize myself with iterative eigenvalue solvers such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Lanczos_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Lanczos&lt;/a&gt;. So I tried rewrite it to python directly according to wiki. But it doesn't seem to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it approximates quite well the largest eigenvalue $\epsilon_{max}$ of matrix $A$&lt;/li&gt;&#xA;&lt;li&gt;but the rest of eigenvalues obtained by solution of projected tridiagonal matrix $T$ is distributed like uniformly over the interval $( \epsilon_{min}$, $\epsilon_{max} )$ of original matrix $A$&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I woud expect that it will find $m$ larges eigenvalues&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;the vectors $v_i$ (matrix $V$) generated by Lancozs iteration seems to be still orthogonal to each other, so I guess the numerical unstability of Lancozs is not the problem&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Python code:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def Lanczos( A, v, m=100 ):&#xA;    n = len(v)&#xA;    if m&amp;gt;n: m = n;&#xA;    # from here https://en.wikipedia.org/wiki/Lanczos_algorithm&#xA;    V = np.zeros( (m,n) )&#xA;    T = np.zeros( (m,m) )&#xA;    vo   = np.zeros(n)&#xA;    beta = 0&#xA;    for j in range( m-1 ):&#xA;        w    = np.dot( A, v )&#xA;        alfa = np.dot( w, v )&#xA;        w    = w - alfa * v - beta * vo&#xA;        beta = np.sqrt( np.dot( w, w ) ) &#xA;        vo   = v&#xA;        v    = w / beta &#xA;        T[j,j  ] = alfa &#xA;        T[j,j+1] = beta&#xA;        T[j+1,j] = beta&#xA;        V[j,:]   = v&#xA;    w    = np.dot( A,  v )&#xA;    alfa = np.dot( w, v )&#xA;    w    = w - alfa * v - beta * vo&#xA;    T[m-1,m-1] = np.dot( w, v )&#xA;    V[m-1]     = w / np.sqrt( np.dot( w, w ) ) &#xA;    return T, V&#xA;&#xA;# ---- generate matrix A&#xA;n = 50; m=10&#xA;sqrtA = np.random.rand( n,n ) - 0.5&#xA;A = np.dot( sqrtA, np.transpose(sqrtA) )&#xA;&#xA;# ---- full solve for eigenvalues for reference&#xA;esA, vsA = np.linalg.eig( A )&#xA;&#xA;# ---- approximate solution by Lanczos&#xA;v0   = np.random.rand( n ); v0 /= np.sqrt( np.dot( v0, v0 ) )&#xA;T, V = Lanczos( A, v0, m=m )&#xA;esT, vsT = np.linalg.eig( T )&#xA;VV = np.dot( V, np.transpose( V ) ) # check orthogonality&#xA;&#xA;#print &quot;A : &quot;; print A&#xA;print &quot;T : &quot;; print T&#xA;print &quot;VV :&quot;; print VV&#xA;print &quot;esA :&quot;; print np.sort(esA)&#xA;print &quot;esT : &quot;; print np.sort(esT)&#xA;&#xA;plt.plot( esA, np.ones(n)*0.2,  '+' )&#xA;plt.plot( esT, np.ones(m)*0.1,  '+' )&#xA;plt.ylim(0,1)&#xA;plt.show( m )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;illustration:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;blue  - eigenvalues of matrix $A$&lt;/li&gt;&#xA;&lt;li&gt;green - eigenvalues of matrix $T$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OffMk.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OffMk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="1804" LastEditDate="2017-03-06T19:35:15.170" LastActivityDate="2020-07-11T06:02:31.000" Title="Quality of eigenvalue approximation in Lanczos method" Tags="|eigenvalues|iterative-method|eigensystem|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23537" PostTypeId="1" CreationDate="2016-03-31T18:35:38.087" Score="1" ViewCount="165" Body="&lt;p&gt;I am trying to minimize a non-linear vector-valued function in MATLAB.  As a test case for my code, I try to minimize a function whose solution I know apriori.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that one of the solutions is widely distributed in scale.  They are [175,164,854,3.7e5,6000];&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the 3.7e5 has a wide difference in scale with the rest of the solution vector. As a result, despite changing tolerances, algorithms etc., the best that MATLAB can find for my x(4) solution is 5.2e5 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I know that using a diagonal (or close to diagonal) scaling can help in finding the minimizing solution. However, from my understanding, the Hessian matrix is needed to obtain the scaling factor, i.e. Ideally I would like to scale the solutions so that the Hessian of the objective function becomes the identity matrix at/near the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem is, the Hessian of a vector-valued function is a Tensor and not a matrix.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a solution, or any other method for handling this problem ?&lt;/p&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2016-03-31T19:19:18.463" Title="Scaling a vector-valued non-linear function for numerical optimization/minimization" Tags="|optimization|matlab|constrained-optimization|nonlinear-programming|scaling|" AnswerCount="0" CommentCount="1" ClosedDate="2019-06-06T20:26:43.560" ContentLicense="CC BY-SA 3.0" />
  <row Id="23539" PostTypeId="2" ParentId="22108" CreationDate="2016-04-01T02:59:17.067" Score="1" Body="&lt;p&gt;Python in conjunction with Theano.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many deep learning and neural network packages developed in Python are built on top of or are just higher level interfaces to Theano.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out &lt;a href=&quot;http://deeplearning.net/tutorial/&quot; rel=&quot;nofollow&quot;&gt;http://deeplearning.net/tutorial/&lt;/a&gt; for more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Please note that I would have added this post as a comment, as opposed to an answer, but unfortunately I don't have enough rep points yet.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="19834" LastEditorUserId="19834" LastEditDate="2016-04-01T15:30:55.693" LastActivityDate="2016-04-01T15:30:55.693" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23540" PostTypeId="1" AcceptedAnswerId="23555" CreationDate="2016-04-01T03:29:30.560" Score="2" ViewCount="1292" Body="&lt;p&gt;I just knew how to do Newton-Raphson iteration in time-independent 1D nonlinear differential equation. Then I applied to time-dependent 1D nonlinear differential equation, and I got confused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is just test time-dependent 1D nonlinear differential equation I want to solve (I just made this up from heat equation, to make it nonlinear)&#xA;$$&#xA;\frac{\partial u}{\partial t}+\frac{\partial^2 u}{\partial x^2}+u^2=0&#xA;$$&#xA;For this equation, I assumed I would need initial condition, and boundary condition&#xA;$$&#xA;u(x,t=0)=\left\{&#xA;\begin{aligned}&#xA;1\qquad&amp;amp;(-1\leq x\leq 1)\\&#xA;0\qquad&amp;amp;\text{otherwise}&#xA;\end{aligned}\right.,\qquad&#xA;u(-5,t)=0,\quad u(5,t)=0&#xA;$$&#xA;I set the $x$ range of $-5 \leq x \leq 5$, and planned to create code in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started with space discretization by using second-order difference&#xA;$$&#xA;\frac{\partial u}{\partial t}+\frac{u_{i+1}-2u_{i}+u_{i-1}}{\Delta x^2}+u_{i}^2=0&#xA;$$&#xA;where $i$ denotes space discretization number&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I applied Backward-Euler discretization for time discretization, which is&#xA;$$&#xA;\frac{d q(x,t)}{dt}=f(t,x,q(x,t))\qquad\longrightarrow\qquad&#xA;\frac{q_{j+1}-q_{j}}{\Delta t}=f_{j+1}&#xA;$$&#xA;where $j$ denotes time discretization number&#xA;(this is just intermediate step for me. I am planning to apply Trapezoidal Rule - Second order Backward Difference Formula (TR-BDF2) later)&#xA;Now applying this, the equation looks like&#xA;$$&#xA;\frac{u_{i,j+1}-u_{i,j}}{\Delta t}+\frac{u_{i+1,j+1}-2u_{i,j+1}+u_{i-1,j+1}}{\Delta x^2}+u_{i,j+1}^2=0&#xA;$$&#xA;Again, $i$ and $ j$ denotes space and time discretization respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I applied Newton-Raphson(NR) scheme, which is for give equation&#xA;$$&#xA;F(u)=0&#xA;$$&#xA;the solution is determined by iteration&#xA;$$&#xA;F'(u^{k})\delta u^{k}=-F(u^{k}), \qquad \qquad u^{k+1}=u^{k}+\delta u^{k}&#xA;$$&#xA;(I'm running out of space for discretization) where k denotes NR iteration number, and if $F(u)$ was system of equations, $F'(u)\delta u$ should be Jacobian.&#xA;Do I have to set initial and boundary condition for $\delta u_{i,j}$ as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my equation looks like&#xA;$$&#xA;\frac{\delta u_{i,j+1}^{k}-\delta u_{i,j}^{k}}{\Delta t}&#xA;+&#xA;\frac{\delta u_{i+1,j+1}^{k}-2\delta u_{i,j+1}^{k}+\delta u_{i-1,j+1}^{k}}{\Delta x^2}&#xA;+2u_{i,j+1}^{k}\delta u_{i,j+1}^{k}&#xA;=&#xA;-\frac{u_{i,j+1}^{k}-u_{i,j}^{k}}{\Delta t}&#xA;-\frac{u_{i+1,j+1}^{k}-2u_{i,j+1}^{k}+u_{i-1,j+1}^{k}}{\Delta x^2}&#xA;-(u_{i,j+1}^{k})^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where I got stuck. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I proceed after this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something? like another boundary/initial conditions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I only know $u_{i,1}^{0}$ and $\delta u_{i,1}^{0}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are like, 11 unknowns due to backward differences ($j+1$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some literature says that I should solve this equation for &quot;each time step&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, is this right way to apply Backward-Euler scheme and NR scheme to the time-dependent nonlinear differential equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any good example solving time dependent nonlinear differential equation with Newton-Raphson iteration?&lt;/p&gt;&#xA;" OwnerUserId="19465" LastEditorUserId="19465" LastEditDate="2016-04-01T13:24:52.070" LastActivityDate="2016-04-02T07:40:42.213" Title="Implementation of Backward-Euler scheme, Newton-Raphson iteration scheme to time dependent nonlinear differential equation" Tags="|nonlinear-equations|newton-method|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23541" PostTypeId="1" CreationDate="2016-04-01T07:32:22.650" Score="1" ViewCount="614" Body="&lt;p&gt;I would like to solve the following optimization problem&#xA;\begin{align}&#xA;\min_{x\in [0,1]^n} x^T p+ \frac{1}{2\lambda} x^T Q x&#xA;\end{align}&#xA;$Q$ is a positive semidefinite matrix.  $\lambda&amp;gt;0$ is a constant (a real number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve this problem repeatedly, with the slight change of $p$ and $Q$ in each iteration. The length of the vector $x$ is at most 1000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a method as fast as possible, which can use the result of previous solve of the problem. Is there any source code available for the method? And if there is please provide a reference to it. &lt;/p&gt;&#xA;" OwnerUserId="8223" LastEditorUserId="4274" LastEditDate="2016-07-21T03:13:16.037" LastActivityDate="2016-07-21T03:13:16.037" Title="What is the fastest method for solving a quadratic programm repeatedly,( warmstarted)?" Tags="|optimization|convex-optimization|quadratic-programming|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23542" PostTypeId="1" AcceptedAnswerId="23583" CreationDate="2016-04-01T07:40:30.800" Score="1" ViewCount="447" Body="&lt;p&gt;I know that in outflow boundary we assume a zero normal gradient condition and use upwind scheme for approximation. However, I saw this sentence in a book which I do not understand; &quot;&lt;strong&gt;Convective fluxes are usually assumed to be independent of the coordinate normal to an outflow boundary.&lt;/strong&gt;&quot; What does it exactly mean and why are we allowed to make that assumption?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;source: &lt;a href=&quot;https://books.google.de/books?id=ye3tCAAAQBAJ&amp;amp;pg=PA77&amp;amp;lpg=PA77&amp;amp;dq=Convective+fluxes+are+usually+assumed+to+be+independent+of+the+coordinate+normal+to+an+outflow+boundary.&amp;amp;source=bl&amp;amp;ots=ww-zQFrFBS&amp;amp;sig=26nEaCGoy5qxxz1zyXPIEe8lhQo&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwibk6qnuu7LAhVH6xQKHSgJCR8Q6AEIHTAA#v=onepage&amp;amp;q=Convective%20fluxes%20are%20usually%20assumed%20to%20be%20independent%20of%20the%20coordinate%20normal%20to%20an%20outflow%20boundary.&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.  &lt;/p&gt;&#xA;" OwnerUserId="19836" LastEditorUserId="19836" LastEditDate="2016-04-02T22:33:23.663" LastActivityDate="2016-04-06T18:29:27.140" Title="Outflow boundary condition" Tags="|fluid-dynamics|boundary-conditions|finite-volume|navier-stokes|numerical-modelling|" AnswerCount="1" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23543" PostTypeId="2" ParentId="23208" CreationDate="2016-04-01T08:54:03.457" Score="1" Body="&lt;p&gt;I found out the problem. Turns out it wasn't in the Ewald summation parameters. The paper that I'm trying to figure out uses the word &quot;equilibrate&quot;, so I assumed (incorrectly) that the writers had used the &quot;&lt;strong&gt;equilibration n&lt;/strong&gt;&quot; -command in DL_POLY. It seems that in this particular case using the equilibration command messes up the simulation and causes it to form structures which are not possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your support! &lt;/p&gt;&#xA;" OwnerUserId="19186" LastActivityDate="2016-04-01T08:54:03.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23545" PostTypeId="1" AcceptedAnswerId="23546" CreationDate="2016-04-01T11:03:01.220" Score="0" ViewCount="82" Body="&lt;p&gt;In  standard finite element method, when we want to find $L^2$ error of a solution, we only find the error on Gaussian points that are inside of each triangle, for $DG$ method can we do this in a same way or we should add some boundary error term, because of the jump?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastActivityDate="2016-04-01T11:23:26.290" Title="How to find $L^2$ error for discontinuous Galerkin method" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23546" PostTypeId="2" ParentId="23545" CreationDate="2016-04-01T11:23:26.290" Score="3" Body="&lt;p&gt;Assuming that you're talking about the $L_2$ error in the functions themselves, and not doing something complicated with gradient reconstruction or anything, then restricting to the sum of the numerical integrals of the errors inside each element should be fine. Remember that the element boundaries are of zero measure in an integral over the domain and that the error at the jumps is finite. &lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-04-01T11:23:26.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23547" PostTypeId="1" AcceptedAnswerId="23549" CreationDate="2016-04-01T14:22:37.563" Score="10" ViewCount="2467" Body="&lt;p&gt;Recently, I came across a comment claiming that almost all researchers doing Monte Carlo methods are doing it wrong. It went on to elaborate that merely choosing different seeds for different instances of a PRNG such as the Mersenne Twister is not sufficient to ensure unbiased results as &lt;em&gt;bad collisions&lt;/em&gt; can occur. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Mersenne_Twister#Disadvantages&quot;&gt;Wikipedia article on the Mersenne Twister&lt;/a&gt; seems to corroborate:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Multiple Mersenne Twister instances that differ only in seed value (but not other parameters) are not generally appropriate for Monte-Carlo simulations that require independent random number generators, though there exists a method for choosing multiple sets of parameter values.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have to admit, I'm guilty as charged. But so are all the other implementations of parallel Monte Carlo libraries I've seen so far, in particular &lt;a href=&quot;http://alps.comp-phys.org/mediawiki/index.php/Main_Page&quot;&gt;ALPS&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Wikipedia article also references two papers offering remedy:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;strong&gt;&lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/DC/dgene.pdf&quot;&gt;Dynamic Creation (DC)&lt;/a&gt;&lt;/strong&gt; scheme (1998) picks parameter sets for the MTs based on the hypothesis that they are independent if the corresponding characteristic polynomials are coprime.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;&lt;a href=&quot;http://www.iro.umontreal.ca/~lecuyer/myftp/papers/jumpf2.pdf&quot;&gt;Jump Ahead for $\mathbb F_2$-linear RNGs&lt;/a&gt;&lt;/strong&gt; (2008). I reckon it is similar to the leap-frog method for LCGs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both methods have been coautored by Matsumoto and Nishimura, the original authors of the Mersenne Twister algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm afraid I'm not very knowledgeable in number theory or algebra and do not fully grasp the above schemes or the maths behind the Mersenne Twister. My questions are primarily of practical nature:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How much do I really need to worry about introducing bias to my simulations when not employing such a scheme if next to nobody cares about it in practice (at least in my community)?&lt;/li&gt;&#xA;&lt;li&gt;If I were to implement one of these counter-measures, am I right to assume that the Jump-Ahead one is better suited as it is based on a firm theory and is the more modern method?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14467" LastEditorUserId="14467" LastEditDate="2016-04-01T15:01:45.290" LastActivityDate="2016-04-03T15:28:12.607" Title="Parallel Mersenne Twister for Monte Carlo" Tags="|parallel-computing|monte-carlo|random-number-generation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23548" PostTypeId="1" CreationDate="2016-04-01T17:11:14.877" Score="3" ViewCount="212" Body="&lt;p&gt;I am kind of new in finite elements and I am solving simple &quot;Poisson nonlinear&quot; problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$- \nabla ((1 + u^2) \nabla u) = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u = 0 \ \text{on} \  \Omega $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Newton solver, where I have defined step as following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J = s * ( -g )$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where  $ J = \int (2 \delta u \nabla u_h) \nabla v + (( 1 + u_h^2) \nabla \delta u ) \nabla v  dx $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;g = $\int (1 + u^2 ) \nabla u \nabla v - f v \ dx$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to check my solution and monitor energy, however I am bit stuck here. I know that in linear case it would be QP function, but how is my strain energy function defined here? &lt;/p&gt;&#xA;" OwnerUserId="19785" LastEditorUserId="19785" LastEditDate="2016-04-02T22:39:47.287" LastActivityDate="2016-04-02T22:39:47.287" Title="Solving a nonlinear poisson equation via variational minimization" Tags="|optimization|finite-element|nonlinear-equations|poisson|variational-calculus|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23549" PostTypeId="2" ParentId="23547" CreationDate="2016-04-01T17:59:49.677" Score="10" Body="&lt;p&gt;Like you say, using the Mersenne Twister for parallel computations is almost always done incorrectly, as the correct method is tricky to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By far the easiest and best answer would be to move away from the Mersenne Twister entirely, and use something like the &lt;a href=&quot;http://www.pcg-random.org/&quot;&gt;PCG family&lt;/a&gt;, which provides multiple streams out of the box. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Mersenne Twister is known to &lt;a href=&quot;http://www.iro.umontreal.ca/~lecuyer/myftp/papers/testu01.pdf&quot;&gt;fail several statistical tests&lt;/a&gt;, while also being slower than newer RNGs such as the PCG and XorShift+ families.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason the Mersenne Twister is so widely used today is mainly a result of the RNGs before it being far worse, both in performance and quality. It also helped that the original authors open sourced a highly performant implementation. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2016-04-01T17:59:49.677" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23551" PostTypeId="1" CreationDate="2016-04-02T02:42:47.220" Score="1" ViewCount="316" Body="&lt;p&gt;I need to solve a tridiagonal system (positive definite, diagonally dominant)  $Ax = b$  in a time stepping loop.  $A \in \mathbb{R}^{N \times N}$ remains constant but $b$ changes during each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking which of the following would be faster :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute and store $\mathbf{z_i}$ that solves $A\mathbf{z_i} = \mathbf{e_i} \; \forall 1\leq i\leq N$. Then any $b \in \mathbb{R}^{N}$ is a linear combination of $\mathbf{z_i}$ and $x = b_i \mathbf{z_1} + \dotsb b_N \mathbf{z_N}$  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Naively solve $A x = b$ for each given $b$ by tridiagonal matrix algorithm.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I thought (1)  will save the time taken for computing the tridiagonal solves since the $z$ essentially gives the inverse of $A$. However, I calculated that one solve for $\mathbf{e_i}$ takes $8N$ flops, so that is $8N \times N$ to compute all the $\mathbf{z_i}$. Then for each  given $b$, it takes $N^2$ flops to compute $x$.  If I take $n$ time-steps, that is a total of $8N^2 + nN^2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast (2) needs $n\times 8N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I correct ? Where is (2)  gaining the advantage ?&lt;/p&gt;&#xA;" OwnerUserId="7728" LastActivityDate="2017-01-13T14:12:05.753" Title="solving tridiagonal system with multiple right hand sides" Tags="|numerical-analysis|linear-solver|banded-matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23552" PostTypeId="1" CreationDate="2016-04-02T03:46:11.007" Score="3" ViewCount="290" Body="&lt;p&gt;I am solving an unsteady flow using the dual-time Navier-Stokes equation in which I write my momentum equation as:&#xA;$$\frac{\partial u}{\partial \tau} + \frac{\partial u}{\partial t} + \frac{\partial u^2}{\partial x} + \frac{\partial uv}{\partial y} =  -\frac{\partial p}{\partial x}  + \frac{\partial^2 u}{\partial x ^2} + \frac{\partial^2 u}{\partial y ^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the time levels in 'physical time, t' goes like n-1, n, n+1, and for that of 'pseudo time, $\tau$ goes like q-1, q, q+1. Then, with the combination of forward and central difference, I can discretize this equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u^{q+1}-u^{q}}{d\tau} + \frac{3u^{q+1}-4u^{n}+u^{n-1}}{2dt} + C = -P + D$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where C, P and D are the discretized version of convection, pressure and diffusion terms respectively. I use a forward difference for pressure, central difference for diffusion terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Iterations are done in pseudo time unless the local steady state is achieved on which I simply assign $u^{n+1} = u^{q+1}$. My concern is of stability at this moment. I seem to infer that both dt and $\tau$ affects the stability but I can not come to a mathematical expression.&#xA;Therefore I am looking forward to gain some knowledge about how to calculate the CFL number (viscous and advection) for this particular scheme.&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2016-04-06T22:36:57.987" Title="Stability Criterion for this Explicit Scheme" Tags="|finite-difference|numerical-analysis|fluid-dynamics|stability|cfl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23553" PostTypeId="1" AcceptedAnswerId="23556" CreationDate="2016-04-02T05:24:40.843" Score="0" ViewCount="43" Body="&lt;p&gt;I am trying to figure out how to deal coupling. I am trying to model a polymer chain. I have to deal with the angle between adjacent angles, dihedral angles, and vibration. I am able to compute all three separately: I create an acceleration matrix by computing the forces. The acceleration matrix has rows=dimension and columns=particles. Each vector in the matrix represents a monomer in real space. Then, I multiply the acceleration matrix with the time step squared and add it to the initial positions. This gives me how the monomers evolve. However, I am not sure how I would go about in coupling all these three. Since most programs execute code sequentially, how would I go around to this? Is there a general approach? I am thinking of two approaches: calculating an effective acceleration that includes all acceleartion along with their coupling and correcting the acceleration after obtaining all accelerations.&#xA;Are there other approaches? Any advice or references would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19740" LastActivityDate="2016-04-02T08:17:21.243" Title="Modeling Coupling: vibrational,rotational, etc" Tags="|molecular-dynamics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23555" PostTypeId="2" ParentId="23540" CreationDate="2016-04-02T07:40:42.213" Score="1" Body="&lt;p&gt;You seem to be confused with what are unknowns in your algebraic equations (as mentioned also in the comment by  origimbo).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let say you have $N$ inner nodes for space discretization, i.e. $\Delta x = 10/(N+1)$. The values $u_{i,1}$ shall be obtained from initial condition (a more usual notation is $u_{i,0}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notion &quot;you have to solve your algebraic equations in each time step&quot; means that in each time step the values $u_{i,j}$ are known (either from initial condition or from previous time step) and your unknowns (the values to be found) are only $u_{i,j+1}$. Now this is the task for NR method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When applying NR method, you have to choose the values $u^0_{i,j+1}$, typically you take $u^0_{i,j+1}=u_{i,j}$. It means $F(u)=0$ represents $N$ nonlinear algebraic equations having $N$ unknowns $u_{1,j+1}$ up to $u_{N,j+1}$ having the form&#xA;$$&#xA;\frac{\delta u_{i,j+1}^{k}}{\Delta t}&#xA;+&#xA;\frac{\delta u_{i+1,j+1}^{k}-2\delta u_{i,j+1}^{k}+\delta u_{i-1,j+1}^{k}}{\Delta x^2}&#xA;+2u_{i,j+1}^{k}\delta u_{i,j+1}^{k}&#xA;=&#xA;-\frac{u_{i,j+1}^{k}-u_{i,j}}{\Delta t}&#xA;-\frac{u_{i+1,j+1}^{k}-2u_{i,j+1}^{k}+u_{i-1,j+1}^{k}}{\Delta x^2}&#xA;-(u_{i,j+1}^{k})^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In above, the values $u_{i,j}$, $u_{*,j+1}^k$ are known, the unkowns in above are only three (but in each equation different ones), namely $\delta u_{i-1,j+1}^k$, $\delta u_{i,j+1}^k$, and $\delta u_{i+1,j+1}^k$. The boundary conditions are $\delta u_{0,j+1}^k=0$ and $\delta u_{N+1,j+1}^k=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, the sign in your PDE before the diffusion term is wrong, it should be minus.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-04-02T07:40:42.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23556" PostTypeId="2" ParentId="23553" CreationDate="2016-04-02T08:17:21.243" Score="1" Body="&lt;p&gt;A famous and precise algorithm for updating positions and speed in molecular dynamics are Verlet algorithm and it's variations (velocity and leapfrog).&#xA;You can see the Verlet algorithm as a Newton-like integrator and velocity or leapfrog as Runge-Kutta versions of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll find the maths, which are really simple in any molecular simulation textbook you'll see it's quite near what you're doing (here is a short version &lt;a href=&quot;http://www.fisica.uniud.it/~ercolessi/md/md/node21.html&quot; rel=&quot;nofollow&quot;&gt;http://www.fisica.uniud.it/~ercolessi/md/md/node21.html&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their is no problem regarding the calculation of your acceleration vector summing all forces in your atoms as long as your time step is around ten times shorter than the shorter period of your system (roughly). Else you will miss parts of the dynamics or it will be totally incoherent due to drifts of positions.&lt;/p&gt;&#xA;" OwnerUserId="19790" LastActivityDate="2016-04-02T08:17:21.243" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23559" PostTypeId="2" ParentId="23547" CreationDate="2016-04-02T21:55:04.237" Score="3" Body="&lt;p&gt;If you want to use MT, you can use &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/&quot; rel=&quot;nofollow&quot;&gt;SFMT&lt;/a&gt; as your PRNG and &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/JUMP/&quot; rel=&quot;nofollow&quot;&gt;SFMT jump&lt;/a&gt; to generate multiple streams.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can simply initialise MT with one seed, and then jump ahead by e.g. $1 \cdot 10^{60}$, $2 \cdot 10^{60}$, $3 \cdot 10^{60}$ … steps to generate multiple streams. Jumping is somewhat expensive, but you only need to do it once when you initialise your PRNGs.&lt;/p&gt;&#xA;" OwnerUserId="11633" LastActivityDate="2016-04-02T21:55:04.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23560" PostTypeId="1" CreationDate="2016-04-02T22:19:52.600" Score="0" ViewCount="250" Body="&lt;h2&gt;Background &amp;amp; Problem formulation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm trying to write a simple program in C++ that performs adaptive numerical integration of vector valued integrands (in one variable), i.e. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_a^b \bar{f}(x) \ dx $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\bar{f} = \{f_1, f_2, ..., f_n\}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there are good libraries that can do this (for example &lt;a href=&quot;http://www.gnu.org/software/gsl/&quot; rel=&quot;nofollow&quot;&gt;GSL&lt;/a&gt; or &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/Cubature&quot; rel=&quot;nofollow&quot;&gt;Cubature&lt;/a&gt;) but I cannot use any of these because they are under GNU GPL. On the other hand I think these libraries are a bit to involved for my simple problems. But if there is library that I can use I will probably use it!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I need is something simple like an adaptive &lt;a href=&quot;https://en.wikipedia.org/wiki/Gaussian_quadrature#Gauss.E2.80.93Kronrod_rules&quot; rel=&quot;nofollow&quot;&gt;Gauss-Kronrod&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Gaussian_quadrature#Gauss.E2.80.93Lobatto_rules&quot; rel=&quot;nofollow&quot;&gt;Gauss-Lobatto&lt;/a&gt; solver, but &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%E2%80%93Cotes_formulas&quot; rel=&quot;nofollow&quot;&gt;Newton-Cotes&lt;/a&gt; probably suffices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been using a Gauss-Lobatto algorithm before that is inspired by &lt;a href=&quot;http://www.bnikolic.co.uk/nqm/1dinteg/gausslobatto.html&quot; rel=&quot;nofollow&quot;&gt;this code&lt;/a&gt; and ideally I would just modify that one to be able to handle vector integrands. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before doing so I tried to &quot;vectorize&quot; a simpler algorithm, namely an adaptive mid-point rule algorithm (i.e. is a rectangular Newton-Cotes). While doing so I realized that it is very difficult to &quot;vectorize&quot; the algorithm and maintain &lt;a href=&quot;https://en.wikipedia.org/wiki/Tail_call&quot; rel=&quot;nofollow&quot;&gt;tail-recursion&lt;/a&gt;. And I noticed significant loss of performance (~30%) when removing the tail-recursion in this simple algorithm. So I imagine the impact can be even bigger in an adaptive Gauss-Lobatto algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to summarize, here are some of my questions:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Q1&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When writing my own adaptive algorithm should I avoid using recursion if I cannot maintain the tail-recursion?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Q2&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Is there a library I can use that solves my problem and is under a less restrictive license than GNU GPL? (LGPL should be fine).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="19849" LastEditorUserId="414" LastEditDate="2016-10-12T12:13:28.423" LastActivityDate="2016-12-13T17:11:46.157" Title="Adaptive numerical integration of a univariate vector integrand" Tags="|algorithms|c++|quadrature|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23562" PostTypeId="1" CreationDate="2016-04-03T00:24:56.577" Score="6" ViewCount="938" Body="&lt;p&gt;Recently, I am trying to solve 2D Euler Equation using FV-WENO reconstruction. For the smooth initial problem, component-wise WENO reconstruction works well. My problem is, if I want to solve double mach reflection problem, do I need to do characteristic decomposition? It seems that doing characteristic decomposition in 2D is not as easy as that in 1D.&lt;/p&gt;&#xA;" OwnerUserId="18916" LastActivityDate="2018-06-26T03:11:03.003" Title="Is it necessary to do characteristic decomposition to solve 2D Euler Equation?" Tags="|finite-volume|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23564" PostTypeId="2" ParentId="23547" CreationDate="2016-04-03T15:28:12.607" Score="1" Body="&lt;p&gt;Really only you can answer the question about simulation bias and if it is acceptable in your application.  The standard procedure I use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Set a pseudo random sequence as a benchmark (standard Monte Carlo) using a high # of simulations (in risk management 10,000 is often used, in other fields 100,000 to 1M may be used).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Run your RNG over the same input data for a subset of data (we use 1 year but that is often overkill).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare the results using statistics which describe features of the data you actually use for making conclusions / decisions.  We use percentiles (1,5,25,50,75,95,99), absolute error, standard deviation of the error.  All of this is relative to your benchmark.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you have the analysis, you can use your own judgement as to if the RNG bias is acceptable.&lt;/p&gt;&#xA;" OwnerUserId="19418" LastActivityDate="2016-04-03T15:28:12.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23567" PostTypeId="1" CreationDate="2016-04-04T00:04:26.273" Score="5" ViewCount="1028" Body="&lt;p&gt;I have a standard set of linear equations $Ax=b$ where the Hessian matrix $A$ has the special block structure as shown:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=&#xA;\begin{pmatrix}&#xA;T &amp;amp; U\\ &#xA;U^T &amp;amp; V&#xA;\end{pmatrix}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x=&#xA;\begin{pmatrix}&#xA;c\\ &#xA;d&#xA;\end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $A$ is square Hessian matrix and each dimension is of size $n = c_n + d_n$. This compound dimension is quite large, typically $10^6-10^{10}$. The gradient vector $b$ has the same block structure as $x$. This set of equations is solved for a full Newton step to an iterative procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can think of the matrices $T$ and $V$ as two different basis expansions of a given problem with the coupling between these two bases represented by $U$. This set of linear equations is usually solved through CG or an associated method as the matrix $A$ is too large to store.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is interesting about this is that the &quot;coupling&quot; block $U$ is quite small. Often, the $c$ and $d$ blocks are solved alternatively without consideration of $U$; however, this can lead to somewhat slow convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would appear that this can be solved in a somewhat decoupled manner where were $Ax$ Hessian-vector guesses can be split as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax_1 = T*c + U*d$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax_2 = U^T*c + V*d$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, $Ax_1$ could be updated on every third iteration of $Ax_2$. As the computation of $Ax_1$ is much more costly than $Ax_2$ this would be quite advantageous. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that this kind of system is well described somewhere, but have been unable to google the correct combination of words. Does anyone know what this kind of system of equations is called?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if anyone has a good recommendation for material on approximate Newton methods for ill-conditioned systems that would be quite helpful as well.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Full disclosure: I asked the same question &lt;a href=&quot;https://math.stackexchange.com/posts/1711816/edit&quot;&gt;here&lt;/a&gt; with a 100 point bounty without luck. I decided to try again here as this community has more experience with practical solutions.&lt;/p&gt;&#xA;" OwnerUserId="4947" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-04-09T14:19:57.877" Title="Solving a set of linear equations with block structure and weak coupling" Tags="|optimization|linear-solver|nonlinear-equations|newton-method|conjugate-gradient|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="23568" PostTypeId="1" CreationDate="2016-04-04T05:58:08.800" Score="2" ViewCount="56" Body="&lt;p&gt;In  Single-Photon Emission Computerized Tomography (SPECT) parallel beam reconstruction using Maximum-Likelihood Expectation–Maximization(MLEM), is it sufficient to scan the object around 180 degree?  or it is required to perform full 360 scan?&lt;/p&gt;&#xA;" OwnerUserId="19859" LastActivityDate="2016-04-04T05:58:08.800" Title="SPECT reconstrction using MLEM" Tags="|linear-algebra|algorithms|iterative-method|inverse-problem|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23569" PostTypeId="1" AcceptedAnswerId="23574" CreationDate="2016-04-04T06:22:41.760" Score="1" ViewCount="76" Body="&lt;p&gt;I couldn't find a simple answer to this thing I'm trying to do in Matlab. I have a function file that does some calculation for a given parameter &lt;em&gt;a&lt;/em&gt;: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[]=my_function(...,a)&#xA;...&#xA;Calculations to obtain some functions y_1(x),y_2(x),... for a given parameter a.&#xA;...&#xA;&#xA;figure(1);&#xA;plot(x,y_1);&#xA;title('y_1(x)');&#xA;xlabel('x');&#xA;ylabel('y_1');&#xA;set(legend(sprintf('y_1(x) for a=%g',a)),'interpreter','latex','Location','best');&#xA;&#xA;figure(2);&#xA;plot(x,y_2);&#xA;title('y_2(x)');&#xA;xlabel('x');&#xA;ylabel('y_2');&#xA;set(legend(sprintf('y_2(x) for a=%g',a)),'interpreter','latex','Location','best');&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm interested in plotting y_1(x) for different values of &lt;em&gt;a&lt;/em&gt; (say 5 different values), all in a single figure (with a proper legend of course). Likewise the same for y_2(x),y_3(x), etc....&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Whats the simplest way to do it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was thinking of making my_function return the plot/figure handle for each of the y_i's and then make a new script/function file that will do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a=[0,1,3,5,7];&#xA;&#xA;h_1=my_function(...,a(1));&#xA;h_2=my_function(...,a(2));&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and somehow combine the figures from h_1 and h_2, but I'm not sure how to do it properly with the legend and everything.&lt;/p&gt;&#xA;" OwnerUserId="19617" LastEditorUserId="19617" LastEditDate="2016-04-04T15:16:01.010" LastActivityDate="2016-04-04T21:53:40.700" Title="Handling figures built for different parameters in Matlab" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23570" PostTypeId="1" AcceptedAnswerId="31417" CreationDate="2016-04-04T11:46:45.153" Score="6" ViewCount="513" Body="&lt;p&gt;I'm working in laboratories where molecular dynamics data are almost always analysed usign block average as stated in the famous Allen and Tildesley book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We divide the datas in blocks of size &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; on which we compute our interesting quantities. We consider that for large enough blocks, the fluctuations converge so: &lt;span class=&quot;math-container&quot;&gt;$$ s=\lim_{M \rightarrow \infty}\frac{M\sigma^2(&amp;lt;X&amp;gt;_M)}{\sigma^2(X)}$$&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is called the statisical inefficiency and, when converged for M large enough, is an estimation of the spacing between uncorrelated records. You can then evaluate your statistical error:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sigma^2(&amp;lt;X&amp;gt;_{run})=\frac{s}{N}\sigma^2(X)$$&lt;/span&gt;&#xA;for a run of N records.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I stumbled across &lt;a href=&quot;http://www.helsinki.fi/~rummukai/lectures/montecarlo_oulu/lectures/mc_notes5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this set of lecture notes&lt;/a&gt; that starts its chapter saying explicitly how block averaging is a naive and mediocre method to evaluate statistical error. Resampling methods seem to be more accurate and easier to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then have two questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) How are resampling methods an &quot;upgrade&quot; of block averaging? The author states that it's due to the potential large variation of value between blocks but on simple systems which converge really fast to equilibrium, I still have an order of magnitude of difference between bootstrapping and block average. Can the starting values be sufficient to impact the statistical error evaluation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Why is block average so widely used in the molecular dynamics community? I suspect that this reference books did propagate this methods but it was published at a time where resampling became of common use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure that any of my colleagues ever asked this question and my knowledge of statistical analysis is to low for me to really defend my view.&lt;/p&gt;&#xA;" OwnerUserId="19790" LastEditorUserId="23791" LastEditDate="2019-04-13T09:42:55.240" LastActivityDate="2019-04-13T09:42:55.240" Title="Is resampling more accurate than block average for statistical analysis of data?" Tags="|statistics|error-estimation|molecular-dynamics|data-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23573" PostTypeId="2" ParentId="14148" CreationDate="2016-04-04T15:18:46.027" Score="1" Body="&lt;p&gt;A &quot;hand-waving&quot; argument may be possible to explain the need for a positive coefficient for the stability of the time integration scheme. I don't know what that is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same condition can be derived rigorously through von Neumann stability analysis. &lt;a href=&quot;http://ocw.mit.edu/courses/mathematics/18-336-numerical-methods-for-partial-differential-equations-spring-2009/lecture-notes/MIT18_336S09_lec14.pdf&quot; rel=&quot;nofollow&quot;&gt;von Neumann Stability analysis&lt;/a&gt;. This link examines a few different schemes.&lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2016-04-04T15:18:46.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23574" PostTypeId="2" ParentId="23569" CreationDate="2016-04-04T16:11:13.330" Score="2" Body="&lt;p&gt;The most idiomatic MATLAB way to do this would be to store the different &lt;code&gt;y&lt;/code&gt; results in an array and then plot the results with a single plot call.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = 1:5;&#xA;x = linspace(0,2*pi);&#xA;y = sin(bsxfun(@times, a.', x));&#xA;figure&#xA;plot(x,y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is equivalent to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = linspace(0,2*pi);&#xA;for a = 1:5&#xA;    y(ii,:) = sin(a*x);&#xA;end&#xA;figure&#xA;plot(x,y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can plot multiple things on the same axes by calling the &lt;code&gt;hold&lt;/code&gt; function. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = linspace(0,2*pi);&#xA;figure&#xA;hold on&#xA;for a = 1:5&#xA;    y = sin(a*x);&#xA;    plot(x,y)&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;br&gt;&#xA;You can make your legend as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = 1:5;&#xA;x = linspace(0,2*pi);&#xA;&#xA;figure&#xA;hold on&#xA;for ii = 1:numel(a)&#xA;    y(i,:) = sin(a*x);&#xA;    legendStrs{ii} = sprintf('y_%d(x) for a=%g', ii, a(ii));&#xA;end&#xA;h = plot(x,y);&#xA;legend(h,legendStrs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="7733" LastEditDate="2016-04-04T21:53:40.700" LastActivityDate="2016-04-04T21:53:40.700" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23577" PostTypeId="2" ParentId="23340" CreationDate="2016-04-04T20:58:52.917" Score="3" Body="&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;:&lt;br&gt;&#xA;I'm new(er) to 'R' and so I don't know the sparse FFT library (&lt;a href=&quot;https://dsp.stackexchange.com/questions/2317/what-is-the-sparse-fourier-transform&quot;&gt;SE&lt;/a&gt;,&lt;a href=&quot;http://groups.csail.mit.edu/netmit/sFFT/code.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIT&lt;/a&gt;,&lt;a href=&quot;https://www.eecs.berkeley.edu/~kannanr/assets/project_ffft/ISITExtendedNoisyDFT-v5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Berk&lt;/a&gt;) off the top of my head.  This stuff has been published for ~2 years.  I would be surprised if it doesn't exist - it should and would be valuable addition to the language if it doesn't. If I were highly motivated (not at the moment), I would make it myself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The question is:&lt;/strong&gt;&lt;br&gt;&#xA;You are wanting to convolute a, presumably zero-padded, 3x3 to 11x11 matrix with a 200x200 to 1000x1000 matrix very quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt;&lt;br&gt;&#xA;I'm not &quot;hip&quot; on the state of the art on this, sorry.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My comment from above:&lt;/strong&gt;&lt;br&gt;&#xA;One might use the ideas of compressed sensing the sense of very sparse cloud of Dirichlet samples. I wonder if there is a sample-at-a-time approach that allows stopping once the resultant change is below a threshold. That might be fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I mean by compressed sensing from sparse Dirichlet samples is this:&lt;/strong&gt;&lt;br&gt;&#xA;Uniformly randomly sample single pixels across the large picture and one-at-a-time add to your convolution result.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that you are looking for &quot;pac-man&quot; in the screen image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is pac-man&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0F7wc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0F7wc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the game-screen image:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/shGq2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/shGq2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you find the &quot;pac-man&quot; in the image using convolution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is rev0 code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(png)             #image load&#xA;library(pracma)          #matlab like matrix utils&#xA;library(microbenchmark)  #for very tight timing&#xA;library(spatstat)        #for gaussian blur&#xA;&#xA;#raw data&#xA;im1 &amp;lt;- readPNG(source = &quot;./data/pacman1.png&quot;,native=F) #pac man&#xA;im2 &amp;lt;- readPNG(source = &quot;./data/pac_game.png&quot;)         #game board&#xA;&#xA;#convert to grayscale (aka 2d matrix)&#xA;im1.g &amp;lt;- rot90(0.2126*im1[,,1] + 0.7152*im1[,,2] + 0.0722*im1[,,3],k=-1)&#xA;im2.g &amp;lt;- rot90(0.2126*im2[,,1] + 0.7152*im2[,,2] + 0.0722*im2[,,3],k=-1)&#xA;&#xA;#normalize&#xA;im1.g &amp;lt;- im1.g/max(im1.g)&#xA;im2.g &amp;lt;- im2.g/max(im2.g)&#xA;&#xA;#housekeeping on original &#xA;rm(im1,im2)&#xA;&#xA;# #check images&#xA;# image(im1.g,col = gray.colors(256))&#xA;# image(im2.g,col = gray.colors(256))&#xA;&#xA;#classic method for image registration using convolution&#xA;&#xA;#make low intensity values negative, &#xA;# (improves curvature for subpixel approaches.)&#xA;im1.g &amp;lt;- im1.g - 0.1&#xA;im2.g &amp;lt;- im2.g - 0.1&#xA;&#xA;#    zero pad the smaller&#xA;im1b &amp;lt;- 0 * im2.g&#xA;temp &amp;lt;- size(im1.g)&#xA;&#xA;im1b[1:temp[1],1:temp[2]] &amp;lt;- im1.g&#xA;&#xA;#convolute&#xA;fim1 &amp;lt;- fft(im1b)&#xA;fim2 &amp;lt;- fft(im2.g)&#xA;im12 &amp;lt;- Re(fft(fim1*fim2,inverse=T))&#xA;&#xA;#ground negatives&#xA;im12[which(im12&amp;lt;0)] &amp;lt;- 0&#xA;&#xA;#scale to height&#xA;im12 &amp;lt;- im12/max(im12)&#xA;&#xA;#plot the result&#xA;image(im12, col = terrain.colors(256), new = TRUE)&#xA;&#xA;drape.plot(1:nrow(im12), 1:ncol(im12), (im12), border = NA, theta = 25, phi = 55, )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The resulting 2d image is:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/spcpf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/spcpf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The drape plot is:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OforH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OforH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I don't exactly like what is going on, I am reasonably confident that the one of the highest 6 peaks happens at the actual &quot;packman&quot;.  Personally I would prefer to use the edges with a gaussian blur so that I am not confusing &quot;Speedy&quot; with Pac-Man, but you are caring about compute speed, not necessarily registration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I can wrap the &quot;convolution&quot; part in benchmark and get compute time&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# benchmark convolute&#xA;mybench &amp;lt;- microbenchmark({&#xA;                           fim1 &amp;lt;- fft(im1b);&#xA;                           fim2 &amp;lt;- fft(im2.g);&#xA;                           im12 &amp;lt;- Re(fft(fim1*fim2,inverse=T));}, &#xA;                          times=100)&#xA;#display value&#xA;print(mybench)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The displayed results are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Unit: milliseconds&#xA;...&#xA;     min       lq     mean   median       uq      max neval&#xA; 9.33394 9.665923 10.00031 9.816518 9.940719 14.81459   100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now there are 64512 pixels in our image.  For larger images we could have millions of pixels.  Performing a convolution there can be really expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's uniformly randomly sub-select 10% of the pixels, and perform the convolution on them, with a sparse fft transform.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#make empty&#xA;im2b &amp;lt;- matrix(0, nrow = nrow(im2.g), ncol = ncol(im2.g))&#xA;&#xA;#uniformly randomly sample 10% of pixels&#xA;idx2b &amp;lt;- sample(x = 1:length(im2.g), size = floor(length(im2.g)/10),replace = FALSE)&#xA;im2b[idx2b] &amp;lt;- im2.g[idx2b]&#xA;&#xA;fim1 &amp;lt;- fft(im1b);&#xA;fim2 &amp;lt;- fft(im2b);&#xA;im12 &amp;lt;- Re(fft(fim1 * fim2, inverse = T));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With only 10% of pixels, the registration property of the convolution actually improves.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/csHrr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/csHrr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The true location retains its high value in the convolution even when much of the data is removed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ILCjZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ILCjZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&gt; INSERT UPDATED BENCHMARK from SPARSE FFT HERE &amp;lt;--&#xA;note: the following is work in progress    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can consider each pixel of the sub-sampled 'im2.g' as its own image, the sum of which comprise the full background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we then look for is a way to make successive convolutions.  The first way may be more expensive, but then we want to find a way to make it less expensive.  I know symbolically that we can treat it as a dirac-delta.  The forward transform is going to be very cheap.  I know that a spatial tight variance function has a wide variance in wavenumber.  &lt;/p&gt;&#xA;" OwnerUserId="3979" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2016-04-09T16:43:33.187" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="23581" PostTypeId="2" ParentId="23562" CreationDate="2016-04-05T09:01:49.747" Score="0" Body="&lt;p&gt;Usually, a Riemann solver or approximate Riemann solver is used to solve test cases which require shock capturing such as this one. It is an extension of the upwinding of the characteristics variables to non-linear equations, so I assume it is what you mean by characteristic decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 2D Riemann problem is indeed much more complex than the 1D problem. See for instance &lt;a href=&quot;http://www3.nd.edu/~dbalsara/Numerical-PDE-Course/Appendix_MHD/abstract.shtml&quot; rel=&quot;nofollow&quot;&gt;this course&lt;/a&gt; on multidimensional Riemann solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for many applications, the 2D effects at the corner of the cells are neglected and only the 1D Riemann problems along the edges are solved. This approximation might be sufficient to solve your test case.&lt;/p&gt;&#xA;" OwnerUserId="19808" LastActivityDate="2016-04-05T09:01:49.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23582" PostTypeId="2" ParentId="23404" CreationDate="2016-04-05T09:56:35.723" Score="1" Body="&lt;p&gt;Assuming that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the rectangles do not overlap at all on the $x$ axis, and that&lt;/li&gt;&#xA;&lt;li&gt;the sum of the rectangles should never exceed the value of the original function at any point,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;the following simple dynamic programming algorithm will calculate the optimal set of at most $k$ rectangles w.r.t. least squared error in $O(n^2k)$ time and $O(nk)$ space in the worst case.  Although this might seem prohibitive for large $n$, there are some refinements that mean that many inputs will take much less time -- and if necessary, some shortcuts that improve the runtime at the cost of sacrificing optimality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define $f(i, j)$ to be the least error that can be achieved on the subsequence $1, \dots, i$ ($i \le n$) using at most $j \le k$ rectangles.  To avoid confusion about which data points are supported by which rectangles, assume all rectangles begin and end at positions having fractional part 0.5.  Also define $g(i, j)$ to be the least error that can be achieved for the subsequence $i, ..., j$ using a single rectangle that ends at $x$ co-ord $j + 0.5$ and begins at any position $\ge i + 0.5$ and $\le j - 0.5$ and having fractional part 0.5.  Finally, let $adjErr(i, j, h)$ be the sum of squared residuals over the range $i, ..., j$ under the assumption that a single rectangle of height $h$ spans this region.  Then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(i, j) = \min_{0 \le m &amp;lt; j}{(f(m, j-1) + g(m+1, i))}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g(i, j) = \min_{i \le m &amp;lt; j}{(adjErr(i, m, 0) + adjErr(m+1, j, min_{m&amp;lt;r\le j}x_r))}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$adjErr(i, j, h) = \sum_{m=i}^j(x_m - h)^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overall lowest possible error is given by $f(n, k)$.  An actual solution (i.e. a set of rectangles) having this best-possible error level can be found by tracing back through the DP matrix, looking for the $m$ value that allowed $f(i, j)$ to take its minimum value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is useful, comment and I'll come back to flesh out the details a bit more.&lt;/p&gt;&#xA;" OwnerUserId="19826" LastActivityDate="2016-04-05T09:56:35.723" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23583" PostTypeId="2" ParentId="23542" CreationDate="2016-04-05T10:04:35.390" Score="1" Body="&lt;p&gt;I understand it as a Neumann boundary condition &#xA;$\frac{\partial F}{\partial n} = 0$ where $F$ is the convective flux and $n$ is the unit normal vector at the boundary. This is the most convenient boundary condition to solve numerically the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Independent of the coordinate normal to [the boundary]&quot; means that, in the local coordinate system of the boundary defined by the vectors $(n, t_1, t_2)$, the field $F$ may locally vary only along the tangential coordinates $(t_1, t_2)$. Hence $\frac{\partial F}{\partial n} = 0$.&lt;/p&gt;&#xA;" OwnerUserId="19808" LastEditorUserId="19808" LastEditDate="2016-04-06T18:29:27.140" LastActivityDate="2016-04-06T18:29:27.140" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23584" PostTypeId="2" ParentId="23445" CreationDate="2016-04-05T13:15:34.540" Score="3" Body="&lt;p&gt;It's a bit heavy, but not all of it must be read to get a useable method :&#xA;&lt;a href=&quot;http://arxiv.org/abs/cond-mat/0303241&quot; rel=&quot;nofollow&quot;&gt;Computing the ground state solution of Bose-Einstein condensates by a normalized gradient flow, Bao and Du&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe -- it's been a while since I worked on it -- that you have to move to imaginary time (as you said, by using $-it$ instead of $t$). &#xA;Then you want to propagate along time the equation obtained (something like :&#xA;$\frac{\partial \Psi}{\partial t} = \frac{1}{2} \Delta \Psi - V(x,y)\Psi - \beta |\Psi|^2 \Psi$ ), which can be done as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each time-step of this propagation can then be separated in two &quot;sub-steps&quot; (basically, it means to take into account two different parts of the righ hand side of the above equation), using &quot;Time splitting spectral method&quot; (TSSP) described p. 10 in the above reference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then solve these &quot;sub-timesteps&quot; on $|\Psi|^2 = \rho$.&#xA;One is easier to solve in Fourier space, the other is exactly solvable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each iteration, the energy can then be computed and you just need to set a convergence criterion to get your ground state. &lt;/p&gt;&#xA;" OwnerUserId="19724" LastActivityDate="2016-04-05T13:15:34.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23585" PostTypeId="1" AcceptedAnswerId="23587" CreationDate="2016-04-05T15:39:14.420" Score="2" ViewCount="1054" Body="&lt;p&gt;I am trying to write a program to compute the advection equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t +u_x = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the spectral method for the spatial derivative $u_x$ and the leapfrog method for the time derivative $u_t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That will give the an equation for the next time step:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{j}^{k+1} = u_{j}^{k-1} - \frac{\Delta t}{n}[f^{-1}(iN \pi f(U_{j}^{k}))]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N = -n,...,n-1$, $f = $ fourier transform,  $f^{-1} = $ inverse fourier transform, $n$ = number of discretized points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I need two initial values (vectors) I use ode45 to compute the solution $u_{j}^{k+1}$ at time $\Delta t$ and having already the solution $u_{j}^{k}$ at $t = 0$ we can use those two vectors to start the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get wrong results, but I can't see what I am doing wrong. This is a plot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qwrim.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qwrim.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe someone can see what I am doing wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CODE:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = 0.2;&#xA;A = 1;&#xA;n = 2^8;&#xA;deltat = 2*A/n;&#xA;h = 2*A/n;   %step size for N&#xA;x = -A:h:A-h; %Discretization of N&#xA;tm = 0:deltat:2;&#xA;u = @(x) exp(-(x/w).^2);&#xA;N = -n/2:n/2-1;   &#xA;&#xA;sol = zeros(length(tm),length(x)); % rows are space / columns are time;&#xA;&#xA;u0 = u(x); % initial condition at t = 0&#xA;[t, ksol] = ode45(@f,[0 deltat], u0);&#xA;u1 = ksol(end,:);&#xA;sol(2,:) = u1;&#xA;sol(1,:) = u0;&#xA;&#xA;for k = 3:length(tm)+1&#xA;    f_hat = fft(sol(k-1,:),n);    % fft transform&#xA;    umiddle = deltat/n * ifft(1i * pi * N * f_hat); %% inverse fft&#xA;    sol(k,:) = sol(k-2,:) - umiddle;&#xA;&#xA;end&#xA;&#xA;mesh(abs(real(sol(2:end,:))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  function sol = f(~,u)&#xA;    n = 2^8;&#xA;    N = -n/2:n/2-1;   &#xA;    D = diag(1i*N*pi); % diagonal matrix (* each u0 element)&#xA;    sol = -ifft(D*fft(u,n));&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19690" LastEditorUserId="19690" LastEditDate="2016-04-05T16:23:42.640" LastActivityDate="2016-04-05T16:44:25.037" Title="Numerically computing the advection equation" Tags="|matlab|fourier-analysis|advection|spectral-method|fourier-transform|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23587" PostTypeId="2" ParentId="23585" CreationDate="2016-04-05T16:44:25.037" Score="8" Body="&lt;p&gt;I see several issues:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The DFT computed with &lt;code&gt;fft&lt;/code&gt; puts the zero mode at the beginning of the array, and if you want to compute the derivative, it is necessary to apply &lt;code&gt;fftshift&lt;/code&gt;/&lt;code&gt;ifftshift&lt;/code&gt; to the array &lt;code&gt;N&lt;/code&gt; to make sure the derivative is correct. It is easy to see for yourself what the correct expression is by working it out with pen and paper, and see also the documentation for &lt;code&gt;fftshift&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;You divide the derivative term by $n$, but I don't see why this is correct: the highest frequency mode in the solution on a mesh of size $N$ will be $u(x) = \cos(\pi x N/(2A))$, which has derivative of order $N$. Yet the way you compute the derivative, the highest frequency mode will have a derivative $\leq 1$, which doesn't seem right.&lt;/li&gt;&#xA;&lt;li&gt;I think you also lost a factor of two in the derivative term, because the approximation to $u_t$ should be $(u^{k+1}-u^{k-1})/(2\delta t)$.&lt;/li&gt;&#xA;&lt;li&gt;Your $\delta t$ is too large. If we analyze the stability of your scheme in the von Neumann sense, substituting the eigenfunctions $u(t,x) = e^{\mathrm{i}(\lambda t-\mu x)}$, the dispersion relation is $\mu\,\delta t = \sin(\lambda\,\delta t)$. The highest frequency mode on a mesh on $[-A,A]$ of $N$ points is $(-1)^j = \cos(\pi N\frac{(x/A)+1}{2})$, so $\delta t$ should be at most $2A/(\pi N)$ for stability.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here are the changes, and this gives me the expected result of a wave propagating at speed 1.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w = 0.2;&#xA;A = 1;&#xA;n = 2^8;&#xA;deltat = 2*A/(pi*1.001*n); %%% Try changing 1.001 to 0.999&#xA;h = 2*A/n;   %step size for N&#xA;x = -A:h:A-h; %Discretization of N&#xA;tm = 0:deltat:2;&#xA;u = @(x) exp(-(x/w).^2);&#xA;N = -n/2:n/2-1;&#xA;&#xA;sol = zeros(length(tm),length(x)); % rows are space / columns are time;&#xA;&#xA;u0 = u(x); % initial condition at t = 0&#xA;%[t, ksol] = ode45(@f,[0 deltat], u0);&#xA;%u1 = ksol(end,:);&#xA;sol(1,:) = u0;&#xA;u1 = u0 + deltat * ifft(-1i*pi*ifftshift(N).*fft(sol(1,:)));&#xA;sol(2,:) = u1;&#xA;&#xA;for k = 3:length(tm)+1&#xA;    f_hat = fft(sol(k-1,:));    % fft transform&#xA;    umiddle = ifft(-1i * pi * ifftshift(N) .* f_hat); %% inverse fft&#xA;    sol(k,:) = sol(k-2,:) + 2*deltat * umiddle;&#xA;    u1  = sol(k-1,:);&#xA;end&#xA;&#xA;[meshX,meshT] = meshgrid(x, tm);&#xA;figure;&#xA;pcolor(meshT, meshX, real(sol(2:end,:)));&#xA;shading flat;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-04-05T16:44:25.037" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23592" PostTypeId="1" AcceptedAnswerId="23619" CreationDate="2016-04-05T21:28:13.590" Score="5" ViewCount="789" Body="&lt;p&gt;What is the most efficient and numerically stable algorithm for computing the inverse CDF $F^{-1}(y)$ of a probability function, assuming that both the PDF $f(x)$ and the CDF $F(x)$ are known analytically but the inverse CDF is not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, this is the same as finding the root of the nonlinear function $G(x) \equiv F(x) - y$, with $x \in \mathbb{R}$ for a given $y \in (0, 1)$. We know/assume that $G(x)$ is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;monotonically non-decreasing (being $F(x)$ a CDF);&lt;/li&gt;&#xA;&lt;li&gt;differentiable at least $k \ge 1$ times (its first derivative is $G^\prime(x) = f(x)$);&lt;/li&gt;&#xA;&lt;li&gt;continuous (we assume that $f(x)$ is not a delta function).&lt;/li&gt;&#xA;&lt;li&gt;I am particularly interested in the case in which the shape of the PDF is a generic mixture of Gaussian distributions multiplied by a polynomial of arbitrary degree (in this case, the CDF can be computed analytically).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are several standard methods for root-finding of generic nonlinear functions (see for example Chapter 9 of &lt;a href=&quot;http://www.fing.edu.uy/if/cursos/fiscomp/extras/numrec/book/f9.pdf&quot; rel=&quot;nofollow&quot;&gt;Numerical Recipes&lt;/a&gt;). The method I would use is  &lt;a href=&quot;https://en.wikipedia.org/wiki/Brent%27s_method&quot; rel=&quot;nofollow&quot;&gt;Brent's method&lt;/a&gt;, with perhaps a couple of steps of &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow&quot;&gt;Newton-Raphson&lt;/a&gt; in the end to refine the root (if at all); but I wonder if there are better ways given the assumptions above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, note that I need to compute the inverse CDF for a large number ($\sim 10^5$) of distinct CDFs within this class; a lookup table or pre-computation is not feasible.&lt;/p&gt;&#xA;" OwnerUserId="19884" LastEditorUserId="19884" LastEditDate="2016-04-05T22:42:20.883" LastActivityDate="2016-04-07T21:26:23.027" Title="Efficient and stable computation of inverse CDF" Tags="|newton-method|probability|roots|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23600" PostTypeId="1" AcceptedAnswerId="23601" CreationDate="2016-04-06T12:35:12.990" Score="3" ViewCount="87" Body="&lt;p&gt;A common approach to image reconstruction is to solve the convex optimization problem&#xA;\begin{equation}&#xA;\text{minimize}  \quad \frac12 \| Ax - b \|^2 + \gamma \| Dx \|_1&#xA;\end{equation}&#xA;where $b$ is a vector of measurements (perhaps a blurry, noisy image), $A$ is a linear transformation that maps the reconstructed image to the vector of measurements (often $A$ is a convolution operator), and $D$ is a wavelet or tight frame analysis operator.  The optimization variable is $x$, an image stored as a rectangular array of numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the most popular choices for $D$ (and what are the most popular and easiest to use software implementations of $D$)?  I've experimented with both curvelet and shearlet tight frames.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good choice of $D$ that is included in Matlab's wavelet toolbox?&lt;/p&gt;&#xA;" OwnerUserId="3274" LastActivityDate="2016-04-24T09:00:14.047" Title="What are the most popular wavelet or tight frame regularizers for image reconstruction problems?" Tags="|image-processing|inverse-problem|wavelet|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23601" PostTypeId="2" ParentId="23600" CreationDate="2016-04-06T14:22:36.503" Score="2" Body="&lt;p&gt;EDIT: some code made available] From a personal experience, and in the work of colleagues, we observed that multi-band dual-tree wavelets, a type of low-redundancy oriented decompositions based on Hilbert pair of wavelets, were an interesting alternative to higher redundancy transformations (curvelets, shearlets). You can have a two-fold or four fold redundancy. Even if you can have fewer directions than with other transforms (a survey on 2D geometric wavelet-like representations is given in &lt;a href=&quot;http://dx.doi.org/10.1016/j.sigpro.2011.04.025&quot; rel=&quot;nofollow noreferrer&quot;&gt;A panorama on multiscale geometric representations&lt;/a&gt;, 2011, L. Jacques &lt;em&gt;et al.&lt;/em&gt;), the Hilbert properties yields useful statistical properties with Gaussian noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UthpX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UthpX.png&quot; alt=&quot;Dual tree of an 8-band dual-tree wavelet&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Caveat: such design sometimes requires to relax the tight frame condition (depending on the optimization used), see &lt;a href=&quot;http://dx.doi.org/10.1109/TSP.2011.2173684&quot; rel=&quot;nofollow noreferrer&quot;&gt;Relaxing Tight Frame Condition in Parallel Proximal Methods for Signal Restoration&lt;/a&gt;, IEEE TSP, 2012. Another type of image restoration with the same wavelets and no issue with the tight frame condition  is discussed in &lt;a href=&quot;http://dx.doi.org/10.1109/ICIP.2015.7351220&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sparse adaptive template matching and filtering for 2D seismic images with dual-tree wavelets and proximal methods&lt;/a&gt;, 2015, ICIP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those dual-tree wavelets originate from &lt;a href=&quot;http://dx.doi.org/10.1109/TIP.2006.875178&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image Analysis Using a Dual-Tree $M$-Band Wavelet Transform&lt;/a&gt;, 2006, and the improvement over curvelets was shown, for instance (for standard denoising), in &lt;a href=&quot;http://dx.doi.org/10.1109/TSP.2008.921757&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Nonlinear Stein Based Estimator for Multichannel Image Denoising&lt;/a&gt;, 2008. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT] The 2D code is now available, embedded in a denoising toolbox: &lt;a href=&quot;http://fr.mathworks.com/matlabcentral/fileexchange/56705-m-band-2d-dual-tree--hilbert--wavelet-multicomponent-image-denoising&quot; rel=&quot;nofollow noreferrer&quot;&gt;$M$-band 2D dual-tree (Hilbert) wavelet multicomponent image denoising&lt;/a&gt;, so you might need a little work. I hope I can work on a cleaner implementation. Recently related discussions in &lt;a href=&quot;https://dsp.stackexchange.com/questions/29634/how-to-implement-a-j-level-m-band-wavelet-transform-of-an-image/29638#29638&quot;&gt;How to implement a $j$-level $M$-band wavelet transform of an image?&lt;/a&gt; and &lt;a href=&quot;https://dsp.stackexchange.com/questions/30310/code-for-a-wavelet-based-hilbert-transform&quot;&gt;Code for a wavelet based hilbert transform?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2016-04-24T09:00:14.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23603" PostTypeId="2" ParentId="23567" CreationDate="2016-04-06T20:43:42.163" Score="2" Body="&lt;p&gt;Consider the alternating directions method&#xA;$$&#xA;x_{k+1}=T^{-1}(c-Uy_{k}),\qquad y_{k+1}=V^{-1}(d-U^{T}x_{k}).&#xA;$$&#xA;If $\|U\|$ is sufficiently small (in a rigorously defined sense described&#xA;below) then the sequence $\{x_{k},y_{k}\}$ is guaranteed to converge&#xA;to a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But conjugate gradients will always converge faster. To see this,&#xA;define the matrix $P$ as the version of $A$ in which the coupling&#xA;is eliminated altogether, &#xA;$$&#xA;P=\begin{bmatrix}T &amp;amp; 0\\&#xA;0 &amp;amp; V&#xA;\end{bmatrix},&#xA;$$&#xA;and note that the alternating directions method is really just the&#xA;preconditioned iterations&#xA;$$&#xA;Pz_{k+1}+(A-P)z_{k}=b\iff z_{k+1}=b-P^{-1}(A-P)z_{k}.&#xA;$$&#xA;It is a well-known theorem that preconditioned conjugate gradient&#xA;(PCG) with $P$ as the preconditioner converges as fast or faster&#xA;than the corresponding fixed-point iterations. But of course this&#xA;may not matter if the alternating directions method already convergences&#xA;very quickly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Theorem.&lt;/strong&gt; Define $\alpha=\sigma_{\max}(U)$ and $\beta=\sqrt{\lambda_{\min}(T)\lambda_{\min}(V)}$.&#xA;If $\alpha&amp;lt;\beta$, then the alternating directions iteration converges&#xA;within&#xA;$$&#xA;\frac{\beta}{\beta-\alpha}\log\epsilon^{-1}\text{ iterations,}&#xA;$$&#xA;and PCG with $P$ as the preconditioner converges within&#xA;$$&#xA;\sqrt{\frac{\beta}{2(\beta-\alpha)}}\log2\epsilon^{-1}\text{ iterations.}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof. After preconditioning, the matrix $P^{-1/2}AP^{-1/2}=I+G$&#xA;where $G=P^{-1/2}(A-P)P^{-1/2}$ is written explicitly as&#xA;$$&#xA;G=\begin{bmatrix}0 &amp;amp; T^{-1/2}UV^{-1/2}\\&#xA;V^{-1/2}U^{T}T^{-1/2} &amp;amp; 0&#xA;\end{bmatrix},&#xA;$$&#xA;and the spectral radius for the iteration matrix is&#xA;$$\begin{align*}&#xA;\rho(G) &amp;amp; =\|T^{-1/2}UV^{-1/2}\|\\&#xA; &amp;amp; \le\|U\|\sqrt{\|T^{-1}\|\|V^{-1}\|} = \frac{\alpha}{\beta}&#xA;\end{align*}$$&#xA;where we note that if $A$ is positive definite, then $\|A^{-1}\|^{-1}=\lambda_{\min}(A)$. If the spectral radius is strictily less than 1, then the iterated sequence is convergent. &#xA;Finally, the condition number for $P^{-1/2}AP^{-1/2}$ is equal to&#xA;$\kappa=(1+\rho(G))/(1-\rho(G))=(\beta+\alpha)/(\beta-\alpha)$. Gradient descent converges in&#xA;$\frac{1}{2}(\kappa+1)\log\epsilon^{-1}$ iterations, and PCG gradients converges&#xA;in $\frac{1}{2}\sqrt{\kappa+1}\log2\epsilon^{-1}$ iterations.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;References&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Shewchuk, Jonathan Richard. &quot;An introduction to the conjugate gradient method without the agonizing pain.&quot; (1994).&lt;/li&gt;&#xA;&lt;li&gt;Saad, Yousef. Iterative methods for sparse linear systems. Siam, 2003.&lt;/li&gt;&#xA;&lt;li&gt;Greenbaum, Anne. Iterative methods for solving linear systems. Vol. 17. Siam, 1997.&lt;/li&gt;&#xA;&lt;li&gt;Kelley, C.T. Iterative methods for linear and nonlinear equations. Siam, 1995.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2016-04-09T14:19:57.877" LastActivityDate="2016-04-09T14:19:57.877" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23604" PostTypeId="2" ParentId="23552" CreationDate="2016-04-06T22:36:57.987" Score="1" Body="&lt;p&gt;It is done in the standard way through von Neumann stability analysis. The CFL condition on the pseudo timestep is straight forward. The restrictions due to the viscosity and other terms are a &lt;em&gt;little&lt;/em&gt; complicated, but they can still be obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following paper the authors workout the time-step restriction on the pseudo time for an explicit Euler scheme for Artificial Compressibility and backward Euler scheme for the real time (in place of the BDF2 that you have). They show how the pseudo timestep is affected by the real timestep, viscosity and the AC parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;McHugh, P. R., and John D. Ramshaw.&lt;/strong&gt; &quot;Damped artificial compressibility iteration scheme for implicit calculations of unsteady incompressible flow.&quot; &lt;em&gt;International journal for numerical methods in fluids&lt;/em&gt; 21.2 (1995): 141-153.&lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2016-04-06T22:36:57.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23606" PostTypeId="1" AcceptedAnswerId="23617" CreationDate="2016-04-07T02:05:10.460" Score="-1" ViewCount="151" Body="&lt;p&gt;I have a computer simulation system of bodies connected by springs, so their movement is governed by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{n+1} = x_n-\Delta tk(x_n-r)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $r$ is the idea distance between every two bodies, and $\Delta t$ is the timestep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Backward Euler method, I end up with the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{n+1} = x_n-\Delta tk(x_{n+1}-r)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which simplifies to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{n+1} = \frac{x_n+\Delta tkr}{1+\Delta tk}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then find that $\Delta tk(x_{n+1}-r)= x_n-x_{n+1} $ so I know the magnitude of my movement, and I can easily calculate a direction vector to apply to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I calculate such interactions between each two sets of bodies that are joined by a spring (I have about 28 bodies), and then add up the movement vectors for each body and apply the movement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where I run into a problem. The numbers I get just keep growing and growing through the iterations until the reach infinity and my simulation explodes, unless I set a small timestep $\Delta t$. I thought Backward Euler should be able to avoid this problem and be unconditionally stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the problem on my implementation, or is there a limit to Backward Euler's stability too that is governed by the time step $\Delta t$&lt;/p&gt;&#xA;" OwnerUserId="19906" LastEditorUserId="123" LastEditDate="2016-04-08T04:06:04.867" LastActivityDate="2016-04-08T04:06:04.867" Title="Why is this method for simulating a system of springs and masses unstable?" Tags="|ode|numerical-modelling|implicit-methods|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23607" PostTypeId="1" AcceptedAnswerId="23618" CreationDate="2016-04-07T06:20:20.337" Score="2" ViewCount="432" Body="&lt;p&gt;Hello I hope I'm asking this in the right community, please feel free to redirect me some better place if you don't think it fits here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I learned when I went to university half an eternity ago, modern CPUs typically have large pipelines for instructions to gain performance by being able to do many simple things in parallell.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; // A typical code block inside a loop that I fear would clog up a pipeline.&#xA; if(l1)     a = ...;&#xA; elseif(l2) a = ...;&#xA; ...&#xA; else        a = ...;&#xA;&#xA; // A different way to write it as a sum that would avoid&#xA; //  branching but at a possible cost of more instructions (?)&#xA; a = (!!l1)*... + (!!l2)*... + ()*...;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Would modern compilers know how to (and &lt;em&gt;allow&lt;/em&gt; themselves to) avoid branching in the first case or should I as an implementer put effort into helping the compiler by planning and rewriting my code as logical arithmetic expressions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is optimization for speed for number-crunching applications.&lt;/p&gt;&#xA;" OwnerUserId="18579" LastActivityDate="2016-04-08T14:41:28.520" Title="Branch avoidance for performance with modern compilers?" Tags="|performance|compiling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23609" PostTypeId="1" AcceptedAnswerId="23610" CreationDate="2016-04-07T10:46:42.220" Score="15" ViewCount="654" Body="&lt;p&gt;I regularly compete in so called &quot;Programming Contests&quot;, where you solve difficult algorithmic problems with your own code and problem solving skills during a limited time-frame. For referential examples of what those might look like, search for competitions like e.g. Google Code Jam, or ACM-ICPC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(If you know what programming contests are, you can skip the following paragraph)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In these contests, you can compete either individually or in teams, at a physical site or online, and the goal is to solve as many problems as possible before the contest is over. Each problem is associated with certain constraints which your solution must fulfill, such as running time, utilized memory, etc. The problem difficulty can range from &quot;obvious how to solve&quot; to &quot;demanding great knowledge or heavy ingenuity to crack&quot;. The main goal is of course to have fun, but success may in some cases lead to cash prizes, honor, and perchance even an interview with recruiters of top-notch companies like Google.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(If you skipped the above paragraph, stop skipping, because here comes my question)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Through programming contests I have been able to develop a genuine interest for algorithmics, computer science in general, and an overall sense of meaning when it comes to coding and developing my problem-solving skills. I would like to do so in scientific computing as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; &lt;em&gt;Do there exist any competitions in the same vein as the purely algorithmic kind, but with a focus on numerical analysis, optimization et cetera?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried Googling, but couldn't find any at first glance…&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum:&lt;/strong&gt; I got the suggestion to check out Project Euler, but that isn't really what I was looking for. What I want is more an arena to practice the more &quot;dirty&quot; kind of numerics, like getting creative with solution schemes for pde's, solving numerical optimization problems, et cetera. Just for comparison, Kaggle is a site where you can practice data mining, machine learning etc. in various contests held regularly. I want something like that, but for implementing and getting creative with numerical schemes and the likes.&lt;/p&gt;&#xA;" OwnerUserId="17610" LastEditorUserId="17610" LastEditDate="2016-04-07T12:52:41.000" LastActivityDate="2016-04-12T08:28:31.913" Title="Scientific Programming Contests" Tags="|optimization|numerical-analysis|complexity|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23610" PostTypeId="2" ParentId="23609" CreationDate="2016-04-07T13:49:45.993" Score="12" Body="&lt;p&gt;I don't know of any current contests, but you can definitely have a look at the SIAM 100-digit challenge. It's a set of 10 problems for which the contest required 10 correct digits per problem. All problems are of the type &quot;if you do it blindly, you'll only get a couple of digits&quot; (unless you resorted to multi-precision arithmetic with in some cases a huge amount of digits needed). See here for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hundred-dollar,_Hundred-digit_Challenge_problems&quot;&gt;Wikipedia page with just the challenges and the 10 digit solutions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book &quot;&lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898717969&quot;&gt;The SIAM 100-Digit Challenge&lt;/a&gt;&quot; discusses all ten problems and provides different approaches to get 10 or even more digits correct. I heartily recommend it. But it's fun to try first without peeking and then read about all neat numerical algorithms that exist to solve those problems. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2016-04-07T13:49:45.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23612" PostTypeId="1" AcceptedAnswerId="23614" CreationDate="2016-04-07T15:10:28.007" Score="0" ViewCount="166" Body="&lt;p&gt;I have a sort of reverse problem, and I'm not sure if there is a simple solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a tridiagonal Hermitian matrix:&#xA;$$&#xA;A = \begin{bmatrix}&#xA;0 &amp;amp; a_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;a_1 &amp;amp; 0 &amp;amp; a_2 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; a_2 &amp;amp; 0 &amp;amp; a_3 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; a_3 &amp;amp; 0 &amp;amp; a_n \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; a_n &amp;amp; 0 \\&#xA;\end{bmatrix}&#xA;$$&#xA;where $a_n$ are unknown; however, I know $A$'s eigenvalues:&#xA;$$&#xA;\lambda = \{\lambda_1, \lambda_2, \lambda_3, \lambda_n\}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find the $a_n$'s that would give me the known eigenvalues that I have.  So I'm after the matrix that gives me these eigenvalues.  I also know the order of the eigenvalues, so I that, say, the first eigenvalue belongs to the largest eigenvector, and the second to the second largest eigenvector.  I don't know the eigenvectors though, and I don't really care what they are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked into closed-form equations to calculate the characteristic polynomials for a matrix of the form of $A$; however, I'm not sure how easy it is to go from these to $a_n$'s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19912" LastActivityDate="2016-04-09T14:04:32.100" Title="Construct tridiagonal matrix from eigenvalues" Tags="|matrix|eigenvalues|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23614" PostTypeId="2" ParentId="23612" CreationDate="2016-04-07T20:04:28.037" Score="2" Body="&lt;p&gt;As a first thought, you could formulate the problem by creating the following set of nonlinear equations: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g_i(\textbf{a}) = det(A(\textbf{a})-\lambda_i I)=0 \;\;\;\;\;\forall i \in [1,n]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you could try solving it through some root-finding or optimization approach. Note that there's a &lt;a href=&quot;https://en.wikipedia.org/wiki/Tridiagonal_matrix&quot; rel=&quot;nofollow&quot;&gt;recursive formula for Tridiagonal determinants&lt;/a&gt; that should reduce the above equations to the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g_i(\textbf{a}) = f_{n}^{(i)}=0 \;\;\;\;\;\forall i \in [1,n]$$&#xA;$$where \;\;\;\;\;f_{j}^{(i)}=-\lambda_i f^{(i)}_{j-1} - a_{j-1}^2 f^{(i)}_{j-1}, f^{(i)}_0 = 1, f^{(i)}_{-1} = 0 \;\;\;\;\;\forall i \in [1,n]$$&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-04-09T14:04:32.100" LastActivityDate="2016-04-09T14:04:32.100" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23617" PostTypeId="2" ParentId="23606" CreationDate="2016-04-07T21:05:27.947" Score="5" Body="&lt;p&gt;The implicit Euler method is unconditionally stable alright, but what you are doing is not the implicit Euler method. Rather, what you do is compute &lt;em&gt;where the particle would be&lt;/em&gt; at the end of the time step using only 2-particle interactions, compute the location update, and then sum these updates up for all particle interactions. But the implicit Euler method would compute all of these updates where you use as $x_{n+1}$ the &lt;em&gt;final position using all interactions&lt;/em&gt;, not just the one interaction you are currently considering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method you describe is called &quot;operator splitting&quot;, and it is not unconditionally stable even if each building block (each interaction separately) is discretized in a way that is unconditionally stable.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-07T21:05:27.947" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23618" PostTypeId="2" ParentId="23607" CreationDate="2016-04-07T21:09:30.793" Score="2" Body="&lt;p&gt;Yes, modern compilers use branch avoidance if possible. For example, they would pull common subexpressions in the assignments to the variable &lt;code&gt;a&lt;/code&gt; out of the if/else branches; they would then see if perhaps what remains in the computation of the assignments is simple enough to have a formulaic expression that allows computation without branching, or through a conditional assignment that depends on a condition flag rather than an executed branch instruction. If expressions are simple enough, the compiler could also employ techniques such as the one you suggest, but that's clearly only a win if what you multiply with is cheap enough to compute for all possible branches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But like all optimizations, what compilers can do is, in reality, rather limited. It often frees you from purposefully obscuring code using micro-optimizations, but it does not free you from thinking about the bigger picture.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-07T21:09:30.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23619" PostTypeId="2" ParentId="23592" CreationDate="2016-04-07T21:26:23.027" Score="3" Body="&lt;p&gt;The assumptions you have are no more specific than what you need to assume to make something like Newton's method work. In fact, you don't even assume enough to make the problem unique: you only assume that $F(x)$ is non-decreasing, when of course you need it to be &lt;em&gt;monotonically increasing&lt;/em&gt; to make finding an answer unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence, there is really nothing very much that could be exploited here, other than the fact that if you assume strict monotonicity, then the root is unique. Thus, what you suggest in terms of methods makes perfect sense.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-07T21:26:23.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23621" PostTypeId="1" AcceptedAnswerId="23623" CreationDate="2016-04-08T02:54:55.280" Score="2" ViewCount="779" Body="&lt;p&gt;I wanted to try to implement part of the method in the following article using Fourier transforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.shodor.org/media/content/jocse/student_submissions/nocito2010/nocito2010_pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.shodor.org/media/content/jocse/student_submissions/nocito2010/nocito2010_pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I am using the FFT to approximate the gradients at each point and using this to calculate the force on each body.  I am currently getting strange results.  When I add just one massive body to the system I get the following as my potential which seems incorrect. I am aware that the nature of the Fourier transform causes the wrap around which would distort the spherical shape but I would still not expect this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZEKW8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZEKW8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code.  One issue I am having is that in the paper you are supposed to divide by |k|^2.  However at some points there is a zero which causes issues.  That is why I clamp it to 0.1.  Not sure if there is a better way to handle that.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;arrayfire.h&amp;gt;&#xA;#include &amp;lt;memory&amp;gt;&#xA;#include &amp;lt;cstdlib&amp;gt;&#xA;#include &amp;lt;algorithm&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;cstring&amp;gt;&#xA;&#xA;const int N = 2;&#xA;const int GRID_DIM = 1024;&#xA;const double CELL_SIZE = 100.0;&#xA;const double G = 1.0;&#xA;const double DT = 0.001;&#xA;&#xA;struct Vec2&#xA;{&#xA;    float x, y;&#xA;    Vec2(float x = 0, float y = 0) :&#xA;        x(x), y(y) { }&#xA;};&#xA;&#xA;int wrapIndex(int i, int i_max) &#xA;{&#xA;    return ((i % i_max) + i_max) % i_max;&#xA;}&#xA;&#xA;&#xA;af::array makeGrid(const Vec2* p, const float* m)&#xA;{&#xA;    float* grid_h = new float[GRID_DIM * GRID_DIM];&#xA;    memset(grid_h, 0, sizeof(float) * GRID_DIM * GRID_DIM);&#xA;&#xA;    for(int i = 0; i &amp;lt; N; i++)&#xA;    {&#xA;        int x = wrapIndex(roundf(p[i].x / CELL_SIZE), GRID_DIM);&#xA;        int y = wrapIndex(roundf(p[i].y / CELL_SIZE), GRID_DIM);&#xA;        grid_h[y * GRID_DIM + x] += m[i];&#xA;    }&#xA;&#xA;    af::array grid(GRID_DIM, GRID_DIM, grid_h);&#xA;    delete[] grid_h;&#xA;    return grid;&#xA;}&#xA;&#xA;&#xA;int main()&#xA;{&#xA;    af::setBackend(AF_BACKEND_CUDA);&#xA;    af::info();&#xA;    Vec2* p = new Vec2[N];&#xA;    Vec2* v = new Vec2[N];&#xA;    float* m = new float[N];&#xA;&#xA;    std::fill(v, v + N, 0.0f);&#xA;    std::fill(m, m + N, 1.0f);&#xA;//  for(int i = 0; i &amp;lt; N; i++)&#xA;//      p[i] = Vec2(rand() % GRID_DIM / 5 + GRID_DIM / 2, rand() % GRID_DIM / 5 + GRID_DIM / 2);&#xA;&#xA;    //add one massive object at the center&#xA;    m[0] = 1000000.0;&#xA;    p[0] = Vec2(GRID_DIM * CELL_SIZE / 2, GRID_DIM * CELL_SIZE / 2);&#xA;&#xA;    m[1] = 0.0;&#xA;    p[1] = Vec2(GRID_DIM * CELL_SIZE / 4, GRID_DIM * CELL_SIZE / 4);&#xA;&#xA;    af::array rng = af::range(GRID_DIM);&#xA;    af::array k2 = 2.0 * af::Pi * af::select(rng &amp;gt; GRID_DIM / 2, (rng - GRID_DIM) , rng) / (GRID_DIM * CELL_SIZE);&#xA;    //af::array k2 = 2.0 * af::Pi * rng / (GRID_DIM * CELL_SIZE);&#xA;    k2(0) = 1.0;&#xA;    k2 = af::tile((k2 * k2).T(), GRID_DIM) + af::tile(k2 * k2, 1, GRID_DIM);&#xA;&#xA;    af::Window wnd(GRID_DIM, GRID_DIM);&#xA;&#xA;    //load the grid with bodies&#xA;    af::array grid = makeGrid(p, m);&#xA;    while(!wnd.close())&#xA;    {&#xA;        af::timer t = af::timer::start();&#xA;&#xA;        //fft the density grid&#xA;        grid = af::fft2(grid);&#xA;&#xA;        //apply the formula from the paper&#xA;        grid *= -G / (af::Pi * k2);&#xA;        //invert the fft to get the potentials&#xA;        grid = af::ifft2(grid);&#xA;        grid = af::abs(grid);&#xA;&#xA;        //compute the gradients in the potential&#xA;        af::array dx, dy;&#xA;        af::grad(dx, dy, grid);&#xA;&#xA;        float* dx_h = dx.host&amp;lt;float&amp;gt;();&#xA;        float* dy_h = dy.host&amp;lt;float&amp;gt;();&#xA;&#xA;        //use the gradients to time step each of the bodies&#xA;        #pragma omp parallel for simd&#xA;        for(int i = 0; i &amp;lt; N; i++)&#xA;        {&#xA;            int xi = wrapIndex(roundf(p[i].x / CELL_SIZE), GRID_DIM);&#xA;            int yi = wrapIndex(roundf(p[i].y / CELL_SIZE), GRID_DIM);&#xA;&#xA;            v[i].x += dx_h[yi * GRID_DIM + xi] * DT / m[i];&#xA;            v[i].y += dy_h[yi * GRID_DIM + xi] * DT / m[i];&#xA;            p[i].x += v[i].x * DT;&#xA;            p[i].y += v[i].y * DT;&#xA;        }&#xA;        af::freeHost(dx_h);&#xA;        af::freeHost(dy_h);&#xA;&#xA;        //display the potential&#xA;&#xA;        wnd.image(af::abs(grid) / af::max&amp;lt;float&amp;gt;(af::abs(grid)));&#xA;        //remake the new grid&#xA;        grid = makeGrid(p, m);&#xA;&#xA;        std::cout &amp;lt;&amp;lt; &quot;dt = &quot; &amp;lt;&amp;lt; af::timer::stop(t) &amp;lt;&amp;lt; std::endl;&#xA;&#xA;    }&#xA;&#xA;    delete[] p;&#xA;    delete[] v;&#xA;    delete[] m;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is another less saturated photo of the potential for a single point.  I can't figure out why it is not approximately round / radially symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6sFp3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6sFp3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19602" LastEditorUserId="123" LastEditDate="2016-04-08T16:37:57.127" LastActivityDate="2016-04-08T16:37:57.127" Title="How do I avoid divide-by-zero when solving the Poisson equation with Fourier transforms?" Tags="|fourier-analysis|poisson|elliptic-pde|fourier-transform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23623" PostTypeId="2" ParentId="23621" CreationDate="2016-04-08T04:20:40.080" Score="6" Body="&lt;p&gt;You are solving Poisson's equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2 \Phi = 4\pi G \rho.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that if $\Phi$ is a solution, then so is $\Phi+C$ for any constant $C$.  Furthermore, $C$ will have no effect on your simulation since the forces depend only on derivatives of $\Phi$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The divide-by-zero issue is just a manifestation of this degeneracy.  In the Fourier-transformed Poisson equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat{\Phi}_k = \frac{-G}{\pi |k|^2} \hat{\rho}_k,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the $k=0$ mode corresponds to the constant part of $\Phi$.  In practice you can simply discard it.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-04-08T04:20:40.080" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23627" PostTypeId="1" CreationDate="2016-04-08T12:28:39.523" Score="0" ViewCount="494" Body="&lt;p&gt;I was told by a class mate that the smallest exponent that we can represent by a single-precision floating-point number (which uses 8 bits for the exponent) is $-126$ and the greatest is $127$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that exponents are represented using excess or bias notation, that is, we represent all exponents, including the negative ones, using positive binary numbers, which we obtain by summing to the real exponent (which can be negative) the &lt;em&gt;bias&lt;/em&gt;. In the single-precision floating-point system, the bias is $2^{\text{# of bits in exponent - 1}} - 1$, which in our case is $2^{8 - 1} - 1 = 2^7 - 1 = 128 - 1 = 127$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain the decimal exponent from the biased one, we can subtract $-127$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, since $0$ has a special representation in this system, the first &lt;em&gt;biased&lt;/em&gt; exponent is $1$, so $1 - 127 = -126$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't understand why the greatest exponent isn't $128$, since, again, the first biased exponent would be $1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing?&lt;/p&gt;&#xA;" OwnerDisplayName="user9925" LastActivityDate="2016-04-08T14:08:51.320" Title="Why the greatest exponent that can be represented in single-precision floating-pointer numbers is 127 (and not 128)?" Tags="|floating-point|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23628" PostTypeId="2" ParentId="23627" CreationDate="2016-04-08T14:08:51.320" Score="4" Body="&lt;p&gt;Exponent $-127$ is reserved for representing zero and denormal numbers, and the exponent $+128$ is reserved for representing infinities and NaNs, as described here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Single-precision_floating-point_format#Exponent_encoding&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Single-precision_floating-point_format#Exponent_encoding&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for normalized numbers, the exponent range is $[-126,127]$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-04-08T14:08:51.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23630" PostTypeId="2" ParentId="23607" CreationDate="2016-04-08T14:41:28.520" Score="2" Body="&lt;p&gt;To add to Wolfgang's answer, there are many expressions which look like they could be optimized by a Sufficiently Smart Compiler, but for which a general use of this optimization would be unsafe. As you have pointed out, rewriting a conditional to use boolean arithmetic involves computing every possible branch, then computing a kind of weighted sum; this does more computing but won't slow you down due to branching. As a silly example, this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (x &amp;gt; 0)&#xA;    return x;&#xA;return -x;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;can be trivially transformed into&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bool b = (x &amp;gt; 0);&#xA;return b * x + (!b) * (-x);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But what about this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (fabs(x) &amp;gt; eps)&#xA;    return sin(x) / x;&#xA;return 1.0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The branch is there to avoid dividing by zero, so you would not want the compiler to perform the same optimization here. Pretty much any operation can, in principle, trigger a &lt;a href=&quot;https://www.gnu.org/software/libc/manual/html_node/FP-Exceptions.html&quot; rel=&quot;nofollow&quot;&gt;floating-point exception&lt;/a&gt;; adding or multiplying two doubles can overflow. So a reasonable optimizing compiler might choose not to eliminate a branch, even if you as the programmer know that the expression inside is innocuous. You would then have to optimize it yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, while using boolean arithmetic to get rid of conditionals can be a useful trick, it makes your code much less readable and the intent should be in comments or documented somewhere.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-04-08T14:41:28.520" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23631" PostTypeId="1" AcceptedAnswerId="26752" CreationDate="2016-04-08T15:28:01.573" Score="0" ViewCount="2091" Body="&lt;p&gt;Suppose I have an algorithms which I want to implement ona GPU. The algorithm consists of a main loop, and all iterations of the loop can be run in parallel. Also, each iteration of the loop has an inner loop whose iterations can be run in parallel. Lets say that I need &lt;code&gt;N&lt;/code&gt; iterations of the main loop, and &lt;code&gt;M&lt;/code&gt; iterations of the inner loop (per main loop iteration), and that my GPU has &lt;code&gt;L&lt;/code&gt; cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;code&gt;N+N*M &amp;lt;= L&lt;/code&gt;, I can run everything in parallel. But if this is not the case, I need to decide what to run sequentially. How should I make this decision? For example, if &lt;code&gt;N=10&lt;/code&gt;, &lt;code&gt;M=5&lt;/code&gt;, &lt;code&gt;L = 20&lt;/code&gt;, when should I choose each of these options (or any other options)?:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Run all main iterations in parallel, and all inner loop sequentially.&lt;/li&gt;&#xA;&lt;li&gt;Run all main iterations sequentially, and all inner loop in parallel.&lt;/li&gt;&#xA;&lt;li&gt;Run all main iterations in parallel, two of the inner loops in parallel and the rest sequentially.&lt;/li&gt;&#xA;&lt;li&gt;Run three of the main iterations in parallel, run each of their inner loops in parallel, run the rest of the main iterations and their inner loops sequentially.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19924" LastActivityDate="2017-04-29T22:23:12.530" Title="How to decide how to parallelize nested loops on GPU" Tags="|parallel-computing|performance|gpu|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23632" PostTypeId="2" ParentId="23327" CreationDate="2016-04-08T15:42:40.383" Score="1" Body="&lt;p&gt;The only missing part I see to the previous answer is the Anaconda package that you mentioned.&#xA;I just wanted to add that it seems appropriate for a 'lazy' installation via graphical installer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside is that it installs a lot of packages, and some of them may be uninteresting for your use.&lt;/p&gt;&#xA;" OwnerUserId="19724" LastActivityDate="2016-04-08T15:42:40.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23634" PostTypeId="1" AcceptedAnswerId="23635" CreationDate="2016-04-08T18:30:08.923" Score="0" ViewCount="1129" Body="&lt;p&gt;I'm trying to use CVX to solve SDP problem.&#xA;I have a constraint with positive definite matrix, but if i read the document of CVX, I can only find variable with positive semidefinite matrix.&#xA;Can anyone know how I should define positive definite matrix in CVX?&lt;/p&gt;&#xA;" OwnerUserId="19931" LastActivityDate="2016-04-08T18:50:07.583" Title="Positive definite matrix in CVX" Tags="|cvx|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23635" PostTypeId="2" ParentId="23634" CreationDate="2016-04-08T18:50:07.583" Score="2" Body="&lt;p&gt;As a practical matter, in floating point arithmetic there's no clear distinction between a positive definite matrix and a positive semidefinite matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to characterize positive semidefinite and positive definite matrices is by the eigenvalues.  However, any computation of the eigenvalues of $X$ will have some round off error, and you can never be sure whether the smallest eigenvalue is exactly 0, slightly negative, or slightly positive.  So, you end up using some kind of tolerance and declaring the matrix to be positive semidefinite if the most negative eigenvalue is smaller in absolute value than this tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some codes consider a matrix to be effectively positive definite if it has a Cholesky factorization, but this approach has similar issues with round off errors.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to select a value $\tau$ and consider $X$ to be positive definite if all eigenvalues of $X$ are greater than or equal to $\tau$ or equivalently consider $X$ to be positive definite if $X-\tau I$ is positive semidefinite.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-04-08T18:50:07.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23636" PostTypeId="2" ParentId="20834" CreationDate="2016-04-08T19:29:58.593" Score="1" Body="&lt;p&gt;I was having a similar problem as the one you had.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The k-space, real part, and self-interaction are fairly dependent on $\kappa$, once you sum them up, the effects of $\kappa$ will cancel out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check my (Python) implementation at this link:&#xA;&lt;a href=&quot;https://github.com/chemlab/chemlab/blob/180287718380ae635aa6401cbd183d93e3bdc81c/chemlab/md/ewald.py&quot; rel=&quot;nofollow&quot;&gt;https://github.com/chemlab/chemlab/blob/180287718380ae635aa6401cbd183d93e3bdc81c/chemlab/md/ewald.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19802" LastActivityDate="2016-04-08T19:29:58.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23637" PostTypeId="1" CreationDate="2016-04-08T21:00:01.053" Score="1" ViewCount="84" Body="&lt;p&gt;I want to solve a system of coupled nonlinear 1-D PDE $(\partial_{tt} + \alpha\partial_t)u_i(x,t)=\partial_{xx}(\sum_{j=1}^{j&amp;lt;i}ju_j(x,t)+i\sum_{j=i}^{n}u_j(x,t))-\sin(u_i(x,t))+f$, using method of lines and for this I defined a spatial grid of about $N\approx1000$ points. Since my PDE is second order both in time and space, the number of ODEs that I have to solve is about $2n*N$ which needs to be run for enough time to get the solutions. For this large system of coupled ODEs I used an ODE solver (tested both implicit and explicit) and although the fastest is ODE23, yet it takes a long time when my $\delta x=0.01$. Since I have access to a multicore system, I was wondering if I could solve this system in a scalable method. I do not think it is applicable using MATLAB or MATHEMATICA. What other options do I have?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% spatial grid&#xA;x=xl:dx:xu;&#xA;%intial condition&#xA;u0(1:n) = somdefunction (x)&#xA;u0(n+1:2n)=0;&#xA;%ode solver&#xA;[t,u]=ode113(@fun,tspan,u0,options)&#xA;% pde &#xA;function ut = fun(t,u)&#xA;uxx = 4*del2(u(1:n),dx);&#xA;%boundary conditions&#xA;% pde&#xA;ut1(1:n) = u(n+1:2n);&#xA;ut2(1:n) = uxx - sin(u);&#xA;ut = (ut1;ut2);&#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14751" LastEditorUserId="14751" LastEditDate="2016-04-11T22:33:09.067" LastActivityDate="2016-04-11T22:33:09.067" Title="What is the best option in terms of library or software to solve this system of hyperbolic PDEs?" Tags="|ode|parallel-computing|hyperbolic-pde|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23638" PostTypeId="1" AcceptedAnswerId="23639" CreationDate="2016-04-08T21:21:16.487" Score="0" ViewCount="283" Body="&lt;p&gt;I've been reading books which refers to vector update operations of the form: y := y + a&lt;em&gt;x, where y and x are vector variables and a is a scalar as SAXPY. I understand a&lt;/em&gt;x plus y part, but why &quot;single precision&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm seeing it in the context of complexity estimates of iterative linear solvers, where I assume that most variables (the vectors and matrices) are double precision, not single. But vector updates are still described as SAXPY.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="19692" LastActivityDate="2022-03-31T16:21:45.010" Title="Single Precision a x plus y (SAXPY) terminology" Tags="|linear-solver|complexity|numerics|precision|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23639" PostTypeId="2" ParentId="23638" CreationDate="2016-04-08T22:42:44.157" Score="0" Body="&lt;p&gt;This is before my time, but the old &lt;a href=&quot;https://web.archive.org/web/20160915040442/https://www.cisl.ucar.edu/zine/96/fall/articles/4.ieee.formats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cray Single Precision&lt;/a&gt; was 64-bit, and so the name stuck for supercomputer users who got used to it. &amp;quot;Sax pee&amp;quot; is just more natural to say than &amp;quot;Dax pee&amp;quot;, besides.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="38994" LastEditDate="2022-03-31T15:37:34.090" LastActivityDate="2022-03-31T15:37:34.090" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="23640" PostTypeId="1" CreationDate="2016-04-08T23:14:39.120" Score="4" ViewCount="216" Body="&lt;h1&gt;Intro:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I am developing a Galerkin FEM code in matlab - starting small with a simple 1D ODE. The equation I'm trying to solve is: $$a u_x = cos(x), x \in [0, 2\pi]$$ Which has a known exact solution $$ u = sin(x)/a $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I solve the equation using my code on an odd number of linear elements (nodes = N+1), the solution is as expected:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Qo0ab.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Qo0ab.png&quot; alt=&quot;Sample FEM Solution - Odd number Elements&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I'm running into a problem when I set the number of elements to an even number, in which case the solution vector is just filled with NaNs and I receive the following warning/error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Warning: Matrix is singular to working precision&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most of the questions I've looked up on mathworks answers have to do with incorrectly constructed matrices - that's not the case here. I'm not sure if I'm running into a numerical issue, where a zero is too small, or if this is an implementation issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I construct both the elemental stiffness matrices and the RHS vector in the same way, and I can't tell a difference between the two systems from a quick comparison of their respective &lt;strong&gt;Ax&lt;/strong&gt;=&lt;strong&gt;b&lt;/strong&gt; structures.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Methodology&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I'm generating the elemental stiffness matrices individually, where &lt;em&gt;N&lt;/em&gt; is a basis function. The weak form of the ODE above for an elemental stiffness matrix is:&#xA;$$ I_e = \int_\Omega \underline{N} a u_x dx = \int_\Omega \underline{N} cos(x) dx $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using integration by parts (i.e. &lt;em&gt;magic&lt;/em&gt;), this equation turns into this:&#xA;$$ (au) \underline{N} |_{\partial \Omega L}^{\partial \Omega R} - \int_\Omega (au) \underline{N}_x dx $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I substitute a continuous value of u for the weighted sum of node values ($\hat{u}$), as such:&#xA;$$ u = N_1 \hat{u}_1 + N_2 \hat{u}_2 = \underline{N}^\intercal \underline{\hat{u}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is finally equal to the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ I_e = a \underline{N}^\intercal \underline{\hat{u}} \underline{N} |_{\partial \Omega L}^{\partial \Omega R} - a \int_\Omega \underline{N}^\intercal \underline{\hat{u}} \underline{N}_x dx  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the first term in $I_e$ cancels out except for the boundaries, in which case $u$ is already 0, I'm assuming that term can be safely ignored. The resulting elemental stiffness takes the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ I_e = \begin{bmatrix} -a \int_\Omega N_1 N_{1x} dx &amp;amp; -a \int_\Omega N_1 N_{2x} dx \\ -a \int_\Omega N_2 N_{1x} dx &amp;amp; -a \int_\Omega N_2 N_{2x} dx \end{bmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Code&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The entire code is in a repository on my github &lt;a href=&quot;https://github.com/cbcoutinho/simple1D_CG-FE&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but the most important part is here below. All of the functions called from this script have been checked and verified to work properly, namely basis_1D(), lgquad(), and lgwt(). Feel free to check the repo if you're curious&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The portion of the code that constructs &lt;strong&gt;A&lt;/strong&gt; and &lt;strong&gt;b&lt;/strong&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for kk = 1:numElem&#xA;    % Initialize elemental stiffness matrix&#xA;    Ie  = zeros(2);&#xA;&#xA;    % Anonymous function that maps natural coordinate to global coordinate&#xA;    X   = @(s)basis_1D(1,1,s,0)*coordPairs(kk,1) + ...&#xA;                basis_1D(1,2,s,0)*coordPairs(kk,2); % X = x(s)&#xA;&#xA;    % Anonymous function that calculates the Jacobian as a function of the natural coordinate&#xA;    J   = @(s)basis_1D(1,1,s,1)*coordPairs(kk,1) + ...&#xA;                basis_1D(1,2,s,1)*coordPairs(kk,2); % J = dx/ds &#xA;&#xA;    for ii = 1:2&#xA;        for jj = 1:2&#xA;            % Anonymous function to what I denote above as N_x&#xA;            fun1    = @(s) basis_1D(1,ii,s,1)./J(s);&#xA;&#xA;            % Anonymous function to what I denote above as N&#xA;            fun2    = @(s) basis_1D(1,jj,s,0);&#xA;&#xA;            % The two functions combined together, as well as the jacobian for numerical quadrature&#xA;            fun     = @(s) fun1(s)*fun2(s).*J(s);&#xA;&#xA;            Ie(ii,jj) = -a*lgquad(fun,[],-1,1); % Calling Guass-Legendre Quad routine&#xA;&#xA;            II = (kk-1)+ii;&#xA;            JJ = (kk-1)+jj;&#xA;            A(II,JJ) = A(II,JJ) + Ie(ii,jj); % Assembling global matrix&#xA;        end&#xA;&#xA;        fun1    = @(s) cos(X(s));&#xA;        fun2    = @(s) basis_1D(1,ii,s,0);&#xA;        fun     = @(s) fun1(s)*fun2(s).*J(s); % Combined function for RHS&#xA;&#xA;        % Calling Gauss-Legendre Quad routine and assembling RHS&#xA;        b((kk-1)+ii) = b((kk-1)+ii) + lgquad(fun,[],-1,1);&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone give me an idea of what my issue might be?&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="19791" LastEditDate="2016-04-09T13:28:08.700" LastActivityDate="2016-04-09T13:28:08.700" Title="Galerkin FEM error when using even number of elements" Tags="|linear-algebra|matlab|finite-element|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23641" PostTypeId="1" AcceptedAnswerId="23645" CreationDate="2016-04-09T02:11:10.120" Score="2" ViewCount="1409" Body="&lt;p&gt;I have seen sparse matrix-matrix multiplication commonly referred to as SpGEMM, which means general/generalised sparse matrix-matrix multiplication. I've seen it once or twice (forgot where) as SpMM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what is &quot;general&quot; referring to? The type of sparse matrix? Does there exist a non-general sparse matrix-matrix multiplication algorithm? All the routines/kernels I've come across are designed for sparse matrices which doesn't assume a structure, so isn't the word &quot;general&quot; a bit redundant? SpMM seems like a more fitting acronym.&lt;/p&gt;&#xA;" OwnerUserId="19692" LastActivityDate="2016-06-23T14:07:19.537" Title="Sparse Matrix Matrix multiplication terminology (SpGEMM or SpMM?)" Tags="|matrix|sparse-matrix|numerical-modelling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23642" PostTypeId="1" AcceptedAnswerId="23661" CreationDate="2016-04-09T05:37:50.997" Score="0" ViewCount="183" Body="&lt;p&gt;Does anyone know of a book that will outline which quantum chemical methods are appropriate for what problems? I am trying to make informed choices before I start using computational resources. It is very difficult to figure out if a particular method is appropriate for the material and problem I am looking at. If there was a book that would outline which method to use for a given problem it would be extremely helpful. Say if I was going to use  Density Functional Theory which methods PBE, B3LYP, MO6 etc are most appropriate for finding the minimum energy structure, the correct band gap, etc for material categories such as: polymers, small molecules, metals, perovskites etc. Or what calculations do most DFT methods typically fail at where wave function approaches would be a safer bet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perchance has anyone read: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Density Functional Theory&#xA;edited by Eberhard K.U. Gross, Reiner M. Dreizler&lt;/p&gt;&#xA;" OwnerUserId="19937" LastEditorUserId="19937" LastEditDate="2016-04-13T23:22:21.110" LastActivityDate="2017-05-28T10:07:40.133" Title="Quantum Chemical Calculations is there a book for which method to use with what problem?" Tags="|computational-chemistry|quantum-mechanics|density-functional-theory|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23643" PostTypeId="2" ParentId="23642" CreationDate="2016-04-09T10:08:48.950" Score="0" Body="&lt;p&gt;I have recently been involved in DFT calculations with wavelets. For this topic, I suggest &lt;a href=&quot;https://comphys.unibas.ch/publications/Goedecker1998d.pdf&quot; rel=&quot;nofollow&quot;&gt;Wavelets and Their Application: For the Solution of Partial Differential Equations in Physics&lt;/a&gt;, 1998. &lt;/p&gt;&#xA;" OwnerUserId="19501" LastActivityDate="2016-04-09T10:08:48.950" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23644" PostTypeId="2" ParentId="21798" CreationDate="2016-04-09T13:23:49.780" Score="0" Body="&lt;p&gt;In Natural Language Processing the terms &quot;bushy&quot; and &quot;straggly&quot; are used to describe tree structure of sentence grammar parses. &quot;Bushy&quot; trees are flatter rather than deeper. &quot;Straggly&quot; trees branch deeply to the right or left.  As far as metrics, you could use tree depth as well as straggliness which has be quantified as by &lt;a href=&quot;https://cs.stanford.edu/~keithw/www/belay-pamuk-sylvestro-winstein.pdf&quot; rel=&quot;nofollow&quot;&gt;Belay et al.&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We calculate the straggliness of a sentence to be the maximum depth of&#xA;  the tree (calculated by counting the maximum stack depth of the parse&#xA;  trees) divided by the number of phrases in the sentence (as determined&#xA;  by the number of lines in the parse tree). In addition, we took the&#xA;  average and standard deviation of the bushiness and straggliness of&#xA;  each sentence and used those as features as well.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4873" LastActivityDate="2016-04-09T13:23:49.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23645" PostTypeId="2" ParentId="23641" CreationDate="2016-04-09T14:05:39.777" Score="3" Body="&lt;p&gt;I think this is a hold-over from the naming convention for the same routines in &lt;a href=&quot;http://www.netlib.org/blas/#_level_3&quot; rel=&quot;nofollow&quot;&gt;BLAS&lt;/a&gt;, where there are routines like &lt;code&gt;DGEMM&lt;/code&gt; for double-precision, general matrix/matrix multiply, &lt;code&gt;DSYMM&lt;/code&gt; for symmetric matrices, &lt;code&gt;ZHEMM&lt;/code&gt; for complex Hermitian matrices, etc. The reason for the different routines is that, if you know the matrix is symmetric or Hermitian, you can use just the upper triangle of it to compute the result of a matrix multiply compared to the general case. Perhaps by saying SpGEMM they are implying that they also considered optimized routines for sparse symmetric matrices.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-04-09T14:05:39.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23646" PostTypeId="1" AcceptedAnswerId="23648" CreationDate="2016-04-09T14:05:50.027" Score="0" ViewCount="204" Body="&lt;p&gt;If we have the weak form of  an equation like Poisson equation with discontinuous Galerkin method, to get the weak form of continuous discontinuous Galerkin method what terms are changed?&#xA;Is it true if just remove the term contained $[u][v]$ in the following:&#xA;$$a(u,v)= (f,v)$$ where$$a(u,v)=\sum_{T \in \cal T}(\nabla u, \nabla v)_T-\sum_{E \in \cal \partial T}(&amp;lt;n.\nabla u&amp;gt;, [v])_E-(n.\nabla u,v)_{\partial \Omega}+\alpha \sum_{E \in \cal \partial T}([u], &amp;lt;n.\nabla v&amp;gt;)_E+\alpha (u-g,n.\nabla v)_{\partial \Omega}+\sum _{E \in \cal \partial T}\beta h^{-1}([u],[v])_E+ \beta h^{-1}([u-g],[v])_{\partial \Omega}$$&lt;/p&gt;&#xA;" OwnerUserId="10403" LastActivityDate="2016-04-09T16:05:22.950" Title="weak form of an equation by continuous discontinuous galerkin method" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23648" PostTypeId="2" ParentId="23646" CreationDate="2016-04-09T16:05:22.950" Score="1" Body="&lt;p&gt;Work backwards (disintegrate by parts) until the point where you introduce the elements.Take the elements out. This will turn all the jump terms back into continuous ones. Integrate by parts forwards without elements, eliminate things that are zero (by BCs or whatever), then introduce the elements. There's no magic steps here. You might get lucky with some operators (linear, symmetric, positive definite ones seem likely) that dropping the terms you want to drop reduces things to what you want, but it's unlikely this process will work in more general or non-linear cases.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-04-09T16:05:22.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23649" PostTypeId="1" AcceptedAnswerId="23659" CreationDate="2016-04-09T16:23:53.967" Score="5" ViewCount="2542" Body="&lt;p&gt;I often hear the phrase &quot;good&quot; or &quot;bad&quot; parallel scaling/efficiency.&#xA;What exactly do people mean when they say that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let $p = 1,\ldots 16$ be the number of processing elements, and A and B be parallel algorithms and consider weak parallel scaling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the efficiency of $A$ decreases linearly from $100\%$ (at $p=1$) to $30\%$ (at $p=16$), and the efficiency of B drops from $100\%$ (at $p=1$) to $20\%$ (at $p=2$), but stays constant at $20\%$ for $p=2,4,8,16$, which has better parallel scaling?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not seen this put in concrete terms before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="19692" LastActivityDate="2016-04-15T20:30:13.453" Title="What is &quot;good&quot; parallel scaling?" Tags="|parallel-computing|mpi|efficiency|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23650" PostTypeId="1" CreationDate="2016-04-09T21:47:41.703" Score="1" ViewCount="69" Body="&lt;p&gt;I need to build some CNN's using very large numbers of photos, to classify which photos contain certain features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've run into a number of road blocks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As much as I love both Linux and virtualization, I've found that Linux-only solutions like Berkeley's Caffe and NYU's pre-trained OverFeat don't work for large scale data aren't viable because virtualization has a lot of problems in an HPC and big data scenario, especially when the tools depend heavily on specific hardware and drivers to be effective (even if you have the hardware and drivers, getting it to work on a VM is something different).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found a number of solutions in R, Python, and ImageJ which work well on the small scale or with specialized types of photos like medical images, satellite or microscopic images, but not for real world photos with lots of &quot;noise&quot; and features which can vary a good bit between photos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a Windows-friendly or platform-agnostic scalable solution that's free and minimally complicated?&lt;/p&gt;&#xA;" OwnerUserId="19091" LastActivityDate="2016-04-10T10:25:39.880" Title="Convolutional Neural Nets for Large Scale Image Classification on Windows?" Tags="|software|image-processing|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23652" PostTypeId="1" CreationDate="2016-04-10T11:03:42.037" Score="2" ViewCount="181" Body="&lt;p&gt;I've read in several articles/answer written by CS (grad) students or other people that the advanced knowledge of pure math that's required for certain areas of computer vision is sometimes too high for the CS student themselves, and requires the involvement actual math people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do wonder if it's true at all? I don't certainly pose this question from a negative view. As a new computer vision researcher who just switched from pure math, I find the math behind CV interesting, but the questions asked to be of different nature. But can the CV researchers with CS background not answer those math questions?&lt;/p&gt;&#xA;" OwnerUserId="19948" LastActivityDate="2016-04-10T11:03:42.037" Title="Pure math questions arising in computer vision, and the need for mathematicians to solve them" Tags="|computational-geometry|computer-vision|" AnswerCount="0" CommentCount="8" ClosedDate="2016-04-12T00:51:14.710" ContentLicense="CC BY-SA 3.0" />
  <row Id="23654" PostTypeId="1" AcceptedAnswerId="24001" CreationDate="2016-04-10T12:16:11.297" Score="1" ViewCount="742" Body="&lt;p&gt;The lifting operator $\mathbf{r}(\mathbf{v_h})$ for the '2nd version' of Bassi-Rebay scheme for elliptic problems in $d$ dimensions is defined as&#xA;$$&#xA;\int_{\Omega_h} \mathbf{w}_h \cdot \mathbf{r}(\mathbf{v}_h)\,\mathrm{d}\mathbf{x} = -\int_{E} \{\mathbf{w}_h\}\cdot \mathbf{v}_h \,\mathrm{d}S, &#xA;$$ &#xA;where&#xA;$$&#xA;\mathbf{v}_h \in \mathbf{V}_h,\:\mathbf{w}_h \in \mathbf{V}_h\\&#xA;V_h = \{v_h \in L^2(\Omega_h) : \bigl. v_h\bigr|_K\in \mathbb{P}_k(K) \:\forall K\in \mathcal{T}_h\}\\&#xA;\mathbf{V}_h = \{v_h \in \bigl(L^2(\Omega_h)\bigr)^d : \bigl. v_h\bigr|_K\in \bigl(\mathbb{P}_k(K)\bigr)^d \:\forall K\in \mathcal{T}_h\},&#xA;$$&#xA;and $E$ is an edge of element belonging to triangulation $\mathcal{T}_h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weak form for Laplace equation discretized with BR2 then contains terms such as&#xA;$$&#xA;\int_{\Omega_h} \bigl(\nabla v_h\bigr) \cdot \mathbf{r}\bigl([\![u_h]\!]\bigr)\,\mathrm{d}\mathbf{x},\quad u_h, v_h \in V_h,&#xA;$$&#xA;for which the definition of the lifting operator can be used to convert the integral over $\Omega_h$ into surface integral over element edge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also face terms such as&#xA;$$&#xA;\int_{E} [\![v_h]\!] \cdot \mathbf{r}\bigl([\![u_h]\!]\bigr)\,\mathrm{d}S.&#xA;$$&#xA;How should these be evaluated? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only approach I can think of is to use the definition of the lifting operator with all test functions $\mathbf{w}_h$ belonging to one element in order to assemble a local linear system where the expansion coefficients of $\mathbf{r}(\mathbf{v}_h)$ are unknowns (i.e. I would assume that $\mathbf{r}(\mathbf{v}_h)$ can be represented as a linear combination of basis functions). If I do that, then I can work with the lifting operator restricted to element traces. This seems to be a bit strange to me though and I'm afraid I completely missed the point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remark - notation:&#xA;Let $E$ be an internal edge shared by two elements $K^{+}$ and $K^{-}$ and let $\mathbf{n}^{+}$ and $\mathbf{n}^{-}$ be outer normals of $K^{+}$ and $K^{-}$ on this edge. The jump $[\![\cdot]\!]$ and average $\{\cdot\}$ operators are defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$[\![v_h]\!] = v_h^{+}\mathbf{n}^{+} + v_h^{-}\mathbf{n}^{-}$ for $v_h \in V_h$&lt;/li&gt;&#xA;&lt;li&gt;$[\![\mathbf{v}_h]\!] = \mathbf{v}_h^{+} \cdot \mathbf{n}^{+} + \mathbf{v}_h^{-} \cdot \mathbf{n}^{-}$ for $\mathbf{v}_h \in \mathbf{V}_h$&lt;/li&gt;&#xA;&lt;li&gt;$\{v_h\} = \frac{1}{2}(v_h^{+} + v_h^{-})$, $v_h \in V_h$&lt;/li&gt;&#xA;&lt;li&gt;$\{\mathbf{v}_h\} = \frac{1}{2}(\mathbf{v}_h^{+} + \mathbf{v}_h^{-})$, $\mathbf{v}_h \in \mathbf{V}_h$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The superscript ${}^{\pm}$ refers to trace quantities restricted to edge (face in 3D) from $K^{+}$ and $K^{-}$, respectively.&lt;/p&gt;&#xA;" OwnerUserId="4688" LastEditorUserId="4688" LastEditDate="2016-04-10T17:00:49.473" LastActivityDate="2016-05-18T08:51:37.257" Title="Discretization of lifting operator in BR2 scheme" Tags="|finite-element|discontinuous-galerkin|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23655" PostTypeId="2" ParentId="10416" CreationDate="2016-04-10T14:06:18.270" Score="4" Body="&lt;p&gt;The White-Christoph formula can be used to estimate the skin-friction coefficient as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C_f\approx\frac{0.455}{\ln^2(0.06Re_x)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this correlation, one can relate $C_f$ to $y^+/y_1$.  Assuming that $x=1$, the following estimate is given for the initial spacing $y_1$ off the wall as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_1=\frac{y^+}{Re}\left(\frac{0.2275}{\ln^2(0.06Re_x)}\right)^{-\frac{1}{2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have studied and validated this on a flat plat for Reynolds numbers up to $10^9$, for a nominal submarine hull at full scale Reynolds numbers, and also for a ducted propulsor using both $q-\omega$ and Spalart-Allmaras turbulence models.  In the case of the propulsor it's important to use the tip velocities when computing the $y^+$ estimate.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ref: W.H. Brewer, &lt;a href=&quot;https://drive.google.com/open?id=0B6Njc_M02jWOTUFaV0UwcXZIaUE&quot; rel=&quot;nofollow&quot;&gt;On simulating tip-leakage vortex flow to study the nature of cavitation inception&lt;/a&gt;, 2002, pp. 40-43, pp. 70-73.&lt;/p&gt;&#xA;" OwnerUserId="4873" LastEditorUserId="4873" LastEditDate="2016-04-12T00:00:28.523" LastActivityDate="2016-04-12T00:00:28.523" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23657" PostTypeId="1" CreationDate="2016-04-10T20:12:00.580" Score="12" ViewCount="1536" Body="&lt;p&gt;So-called &lt;a href=&quot;https://en.wikipedia.org/wiki/Quadtree&quot; rel=&quot;nofollow&quot;&gt;quadtree&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Octree&quot; rel=&quot;nofollow&quot;&gt;octree&lt;/a&gt; grids are quite attractive for applications requiring adaptive mesh refinement. They are for example used in &lt;a href=&quot;http://gfs.sourceforge.net/wiki/index.php/Main_Page&quot; rel=&quot;nofollow&quot;&gt;Gerris&lt;/a&gt; and &lt;a href=&quot;http://www.physics.drexel.edu/~olson/paramesh-doc/Users_manual/amr.html&quot; rel=&quot;nofollow&quot;&gt;Paramesh&lt;/a&gt;. Is anyone aware of a good file format for such grids, and supporting visualization software? See also &lt;a href=&quot;http://gfs.sourceforge.net/wiki/index.php/FAQ#Why_create_a_new_visualisation_tool_like_GfsView.3F_Can.27t_you_use_existing_tools_like_Mayavi.2FVTK.2C_OpenDX_etc....3F&quot; rel=&quot;nofollow&quot;&gt;this answer&lt;/a&gt; in the Gerris FAQ. The only potential candidate that I'm currently aware of is the VTK &lt;a href=&quot;http://www.vtk.org/doc/nightly/html/classvtkHyperOctree.html&quot; rel=&quot;nofollow&quot;&gt;HyperOctree&lt;/a&gt; class, but there seems to be almost no documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a specific example of a mesh, consider a sparsely refined octree grid, consisting of $10^4$ boxes, each containing $8 \times 8 \times 8=512$ cells. My current strategies in Visit/Paraview are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write the grid as a unstructured VTK file. This is expensive for larger datasets, and does not exploit the 'structuredness' of the quad/octree.&lt;/li&gt;&#xA;&lt;li&gt;Collect neighboring 'boxes' in the tree into larger blocks, and write a block-structured grid. This involves quite a bit of extra code, and loses the connectivity information embedded in the quad/octree.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; If there is currently no viable format for directly writing and visualizing an octree grid, I would appreciate suggestions for a file format that can be used to write the octree as a block-structured grid. Ideally, this format:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Has a relatively simple definition/implementation, ideally open source and with C/Fortran compatibility.&lt;/li&gt;&#xA;&lt;li&gt;Can efficiently handle a large number of grid blocks (e.g. 1000 or more).&lt;/li&gt;&#xA;&lt;li&gt;Allows to easily specify the connectivity of grid blocks, in particular at refinement boundaries.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm currently using &lt;a href=&quot;https://wci.llnl.gov/simulation/computer-codes/silo&quot; rel=&quot;nofollow&quot;&gt;Silo&lt;/a&gt; for this purpose, but I'm looking for something that scores better on the above three points (smaller, faster, simpler).&lt;/p&gt;&#xA;" OwnerUserId="18662" LastEditorUserId="18662" LastEditDate="2016-04-13T21:13:59.887" LastActivityDate="2016-04-13T21:13:59.887" Title="Visualization of quadtree &amp; octree grids" Tags="|visualization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23659" PostTypeId="2" ParentId="23649" CreationDate="2016-04-11T07:47:52.120" Score="12" Body="&lt;p&gt;&lt;em&gt;Good&lt;/em&gt; is a relative term, and it will depend on the nature of the problem, the nature of the algorithm, and properties of the hardware involved.  The only absolute reference point is ideal scaling (100% efficiency).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can claim your scaling is good if it is better than what anyone else has achieved for the same problem, or if it's &quot;close&quot; to ideal for large numbers of processors.  For example, in &lt;a href=&quot;http://www.davidketcheson.info/assets/papers/pyclaw-sisc.pdf&quot;&gt;this paper&lt;/a&gt; (disclaimer: citing my own work because it's what I'm most familiar with) we achieved about 95% efficiency weak scaling from 1 to 65K processes and claimed that was good.  It was nothing exceptional given the algorithms and hardware involved, but we did avoid making any major mistakes that would have ruined the efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both examples you give seem very poor for most problem domains.  In the second example, you actually have anti-scaling -- when running with 2 or 4 processes at 20% efficiency, wall clock time will actually be &lt;strong&gt;greater&lt;/strong&gt; than for a serial run.  That's definitely bad scaling!&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2016-04-11T11:50:53.490" LastActivityDate="2016-04-11T11:50:53.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23660" PostTypeId="1" CreationDate="2016-04-11T11:50:58.100" Score="2" ViewCount="407" Body="&lt;p&gt;I recently started to perform natural resonance theory (NRT) analysis with &lt;a href=&quot;http://nbo6.chem.wisc.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NBO6&lt;/a&gt;. This program may automatically create a chemical markup language (&lt;a href=&quot;https://en.wikipedia.org/wiki/Chemical_Markup_Language&quot; rel=&quot;nofollow noreferrer&quot;&gt;cml&lt;/a&gt;) file with the treated structures. Unfortunately I don't speak cml well enough to judge if the file generated is any good.&lt;/p&gt;&#xA;&lt;p&gt;What I would like to do is to convert this generated file into an image. That means, that the included 3D coordinates have to be flattened to 2D. I think the problem lies at this step as I get an erroneous structure reported.&lt;/p&gt;&#xA;&lt;p&gt;As a test molecule I chose &lt;a href=&quot;https://en.wikipedia.org/wiki/Acetic_acid&quot; rel=&quot;nofollow noreferrer&quot;&gt;acetic acid&lt;/a&gt; and I am including a total of three resonance structures (It gets worse when I use more). The first and main contributor is displayed just fine. The second has missing formal charges. These are not explicitly given in the cml file, so they would need to be calculated. Openbabel is certainly able to do that, but for some reason, it is not working here. The third structure is completely ripped apart. The atoms should be at the same position as in the first and second image.&lt;/p&gt;&#xA;&lt;p&gt;I am using a self-compiled local version of openbabel (2.3.2) with the following command (test.cml at the end):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  ~/local/tools/openbabel-install/bin/babel -icml test.cml -opng test.png&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I obtain the following image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0KPgm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0KPgm.png&quot; alt=&quot;openbabel output&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Along with the error messages:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;==============================&#xA;*** Open Babel Error  in DoHCounts&#xA;  In atom a1 the number of explicit hydrogens exceeds the hydrogenCount attribute.&#xA;==============================&#xA;*** Open Babel Error  in DoHCounts&#xA;  In atom a1 the number of explicit hydrogens exceeds the hydrogenCount attribute.&#xA;==============================&#xA;*** Open Babel Error  in DoHCounts&#xA;  In atom a1 the number of explicit hydrogens exceeds the hydrogenCount attribute.&#xA;3 molecules converted&#xA;3 errors 109 audit log messages&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't have any idea how to access those audit log messages (&lt;code&gt;/var/log/messages: Permission denied&lt;/code&gt;) and if they are in any way helpful.&lt;br /&gt;&#xA;However, the error messages already seem to indicate, that the generated cml file is faulty.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;?xml version=&amp;quot;1.0&amp;quot;?&amp;gt;&#xA;&amp;lt;cml title=&amp;quot;Resonance forms from NRT analysis&amp;quot;&amp;gt;&#xA; &amp;lt;molecule id=&amp;quot;1&amp;quot; title=&amp;quot;Wgt=64.49%&amp;quot;&amp;gt;&#xA;  &amp;lt;atomArray&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a1&amp;quot; elementType=&amp;quot;C&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.40&amp;quot; y3=&amp;quot;-0.11&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a2&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.93&amp;quot; y3=&amp;quot;0.86&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a3&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.70&amp;quot; y3=&amp;quot;-0.70&amp;quot; z3=&amp;quot;0.89&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a4&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.70&amp;quot; y3=&amp;quot;-0.70&amp;quot; z3=&amp;quot;-0.89&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a5&amp;quot; elementType=&amp;quot;C&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.09&amp;quot; y3=&amp;quot;0.13&amp;quot; z3=&amp;quot;-0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a6&amp;quot; elementType=&amp;quot;O&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.78&amp;quot; y3=&amp;quot;-1.05&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;2&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a7&amp;quot; elementType=&amp;quot;O&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.65&amp;quot; y3=&amp;quot;1.21&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;2&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a8&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;1.73&amp;quot; y3=&amp;quot;-0.80&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;  &amp;lt;/atomArray&amp;gt;&#xA;  &amp;lt;bondArray&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a2&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a3&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a4&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a5&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a5 a6&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a5 a7&amp;quot; order=&amp;quot;2&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a6 a8&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;  &amp;lt;/bondArray&amp;gt;&#xA; &amp;lt;/molecule&amp;gt;&#xA; &amp;lt;molecule id=&amp;quot;2&amp;quot; title=&amp;quot;Wgt=23.43%&amp;quot;&amp;gt;&#xA;  &amp;lt;atomArray&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a1&amp;quot; elementType=&amp;quot;C&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.40&amp;quot; y3=&amp;quot;-0.11&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a2&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.93&amp;quot; y3=&amp;quot;0.86&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a3&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.70&amp;quot; y3=&amp;quot;-0.70&amp;quot; z3=&amp;quot;0.89&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a4&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.70&amp;quot; y3=&amp;quot;-0.70&amp;quot; z3=&amp;quot;-0.89&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a5&amp;quot; elementType=&amp;quot;C&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.09&amp;quot; y3=&amp;quot;0.13&amp;quot; z3=&amp;quot;-0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a6&amp;quot; elementType=&amp;quot;O&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.78&amp;quot; y3=&amp;quot;-1.05&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a7&amp;quot; elementType=&amp;quot;O&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.65&amp;quot; y3=&amp;quot;1.21&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;3&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a8&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;1.73&amp;quot; y3=&amp;quot;-0.80&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;  &amp;lt;/atomArray&amp;gt;&#xA;  &amp;lt;bondArray&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a2&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a3&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a4&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a5&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a5 a6&amp;quot; order=&amp;quot;2&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a5 a7&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a6 a8&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;  &amp;lt;/bondArray&amp;gt;&#xA; &amp;lt;/molecule&amp;gt;&#xA; &amp;lt;molecule id=&amp;quot;3&amp;quot; title=&amp;quot;Wgt=4.20%&amp;quot;&amp;gt;&#xA;  &amp;lt;atomArray&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a1&amp;quot; elementType=&amp;quot;C&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.40&amp;quot; y3=&amp;quot;-0.11&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a2&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.93&amp;quot; y3=&amp;quot;0.86&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a3&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.70&amp;quot; y3=&amp;quot;-0.70&amp;quot; z3=&amp;quot;0.89&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a4&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;-1.70&amp;quot; y3=&amp;quot;-0.70&amp;quot; z3=&amp;quot;-0.89&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a5&amp;quot; elementType=&amp;quot;C&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.09&amp;quot; y3=&amp;quot;0.13&amp;quot; z3=&amp;quot;-0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a6&amp;quot; elementType=&amp;quot;O&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.78&amp;quot; y3=&amp;quot;-1.05&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;3&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a7&amp;quot; elementType=&amp;quot;O&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;0.65&amp;quot; y3=&amp;quot;1.21&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;atom id=&amp;quot;a8&amp;quot; elementType=&amp;quot;H&amp;quot; hydrogenCount=&amp;quot;0&amp;quot; x3=&amp;quot;1.73&amp;quot; y3=&amp;quot;-0.80&amp;quot; z3=&amp;quot;0.00&amp;quot; lonePair=&amp;quot;0&amp;quot;/&amp;gt;&#xA;  &amp;lt;/atomArray&amp;gt;&#xA;  &amp;lt;bondArray&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a2&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a3&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a4&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a1 a5&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a5 a7&amp;quot; order=&amp;quot;3&amp;quot;/&amp;gt;&#xA;   &amp;lt;bond atomRefs2=&amp;quot;a6 a8&amp;quot; order=&amp;quot;1&amp;quot;/&amp;gt;&#xA;  &amp;lt;/bondArray&amp;gt;&#xA; &amp;lt;/molecule&amp;gt;&#xA;&amp;lt;/cml&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you look at the coordinate information, you will see, that they are identical. The bond array just changes along with the lone pairs. It is also obvious from where the above mentioned errors stem from. Each entry has &lt;code&gt;hydrogenCount=&amp;quot;0&amp;quot;&lt;/code&gt;, which is maybe correct because there are explicit hydrogens. But I am not sure. I found an &lt;a href=&quot;https://sourceforge.net/p/openbabel/bugs/947/&quot; rel=&quot;nofollow noreferrer&quot;&gt;open bug report&lt;/a&gt; referring to this attribute.&lt;/p&gt;&#xA;&lt;p&gt;Now that I seem to not find a reason for this behaviour I need to find an alternative. The currently only viable and reliable method for me is to interpret the output of the NRT by hand.&lt;/p&gt;&#xA;&lt;h3&gt;Resulting from this I have a few questions:&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the produced cml file correct? Does it contain errors and hence the problem is not with openbabel?&lt;/li&gt;&#xA;&lt;li&gt;If the file is correct, are there alternatives to openbabel, which treat it correctly and which are able to convert it to a 2D image (structural representation)? (I would prefer a commandline tool.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The output should be similar to this manually created image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/77cQZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/77cQZ.jpg&quot; alt=&quot;(somewhat) correct/ expected output&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried other programs, so far with not much luck.&lt;br /&gt;&#xA;&lt;a href=&quot;http://www.cambridgesoft.com/software/overview.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chemdraw&lt;/a&gt; can read the cml file and produces the structures with the correct bonds and the atoms are in the correct position. However, charges are missing and all molecules are at the same position, thus making it quite hard to obtain a good image. It also fails to read the title statement.&lt;br /&gt;&#xA;&lt;a href=&quot;http://jchempaint.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JChemPaint&lt;/a&gt; while technically able to read cml, fails to read this file. I could pinpoint it to the third structure, which produces an error and aborts. If only the first two are included it also fails to produce the charges.&lt;br /&gt;&#xA;&lt;a href=&quot;http://www.acdlabs.com/resources/freeware/chemsketch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChemSketch&lt;/a&gt; cannot read cml formatting.&lt;/p&gt;&#xA;" OwnerUserId="19956" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-04-11T11:50:58.100" Title="What are alternatives to openbabel to convert CML file to image?" Tags="|software|computational-chemistry|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23661" PostTypeId="2" ParentId="23642" CreationDate="2016-04-11T12:10:38.770" Score="3" Body="&lt;p&gt;&lt;strong&gt;There is no universal answer to this.&lt;/strong&gt; You will have to find publications that have similar set-ups and already did the benchmarking, or you will have to do it yourself. Once you understand how the methods work, and any quantum/ computational chemistry book is sufficient for that, you have a very general understanding of what might go wrong. It is also very dependent on whatever you can afford.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can head over to chemistry.se, where I have written a summary about DFT. Shameless self-promotion: &lt;a href=&quot;https://chemistry.stackexchange.com/q/27302/4945&quot;&gt;DFT Functional Selection Criteria&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19956" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:07.170" LastActivityDate="2016-04-11T12:10:38.770" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23662" PostTypeId="2" ParentId="23649" CreationDate="2016-04-11T13:14:28.077" Score="0" Body="&lt;p&gt;If you are looking for a general criterion, maybe linear scaling would be the 'ideal' scaling. Hence, I would say that the closer to a straight line your scaling is, the better it is. &lt;/p&gt;&#xA;" OwnerUserId="19724" LastActivityDate="2016-04-11T13:14:28.077" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23663" PostTypeId="2" ParentId="23649" CreationDate="2016-04-11T14:03:05.777" Score="1" Body="&lt;p&gt;As mentioned already, there's no absolute definition for good scaling. All you can really have is some sense of whether or not the scalability your code achieves is better, worse or in line with whichever other codes doing the same sort of computation do achieve in similar environments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, some computing centres sometimes (try to) enforce some rules to make sure their computing resources are used (not too in)effectively. For example, I remember of one of them for which, in order to be granted access, you had to propose a scalability curve for a typical use case of you code. From that, you would only be allowed to run up to a number of processes for which the parallel effectiveness of the code would be over 75%. This magic 75% efficiency threshold wouldn't mean that your code's scalability would be good when above, but simply that it would just be too bad below...&lt;/p&gt;&#xA;" OwnerUserId="18449" LastActivityDate="2016-04-11T14:03:05.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23665" PostTypeId="1" CreationDate="2016-04-12T00:25:03.140" Score="1" ViewCount="755" Body="&lt;p&gt;I'm working on some problems that ultimately boil down into a simple assembly of an overdetermined system of equations, $Ax=b$, where $A$ is $m \times n$ for $m \gg n$. I'm leveraging Armadillo's C++ library, which I'm linking against MKL. I computed my root mean square error on my training data by computing $E = \frac{\|Ax-b\|}{\sqrt{m}}$ and noticed that my errors and solution $x$ varied significantly per run of the test (in the second or third significant digit, sometimes some coefficients were off by a factor of 2). This troubled me, and so I ran the application with MKL_NUM_THREADS=1 forcing the number of threads down to 1, and every time I ran I got the same answer (with minor, far in the noise differences), and the error $E$ fell by several orders of magnitude. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solve the linear system using the very simple call arma::solve(X,A,B) where $X$ contains two columns of $n$ coefficients we aim to solve for, and $B$ contains two columns of $m$ data points. The condition number is likely  large (forgot to check it), but it is still within what I would guess is the &quot;safe range&quot; (NumPy's linalg.lstsq never complained when I tried it using that).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm guessing this is an error somewhere in my matrix assembly, or maybe something I'm fundamentally misunderstanding about multi-threaded MKL being used by armadillo. Is this an expected behavior or a sign that there is definitely something wrong in the code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any suggestions!&lt;/p&gt;&#xA;" OwnerUserId="6567" LastActivityDate="2016-04-12T01:28:44.593" Title="Armadillo Multi-threaded Linear Solve Yielding Different Answers" Tags="|parallel-computing|matrix|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23666" PostTypeId="2" ParentId="23665" CreationDate="2016-04-12T01:28:44.593" Score="2" Body="&lt;p&gt;Parallel code implies a different order of summation for your matrix and any other floating point operations within the solver. It's entirely possible that the matrix and solution you get are especially harmed by this if the condition number is really bad.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-04-12T01:28:44.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23667" PostTypeId="2" ParentId="23609" CreationDate="2016-04-12T07:02:37.657" Score="3" Body="&lt;p&gt;One alternative would be top coder data science competitions. It doe not fit your description, but you can encounter very often contests which are a combination of algorithms - combinatorial - numerical calculus - machine learning. I would keep an eye on those. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A link to the home page for data science competitions is &lt;a href=&quot;https://www.topcoder.com/challenges/data/active/?pageIndex=1&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4838" LastActivityDate="2016-04-12T07:02:37.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23668" PostTypeId="2" ParentId="23609" CreationDate="2016-04-12T08:28:31.913" Score="2" Body="&lt;p&gt;Have you seen the competitions organised by GECCO (Genetic and Evolutionary Computation Conference)?&#xA;&lt;a href=&quot;http://gecco-2016.sigevo.org/index.html/HomePage#&amp;amp;panel1-1&quot; rel=&quot;nofollow&quot;&gt;http://gecco-2016.sigevo.org/index.html/HomePage#&amp;amp;panel1-1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several different competitions for the 2016 conference on the page at:&#xA;&lt;a href=&quot;http://gecco-2016.sigevo.org/index.html/Competitions&quot; rel=&quot;nofollow&quot;&gt;http://gecco-2016.sigevo.org/index.html/Competitions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the competitions can be attempted even though you have no intention of attending the conference.&lt;/p&gt;&#xA;" OwnerUserId="16571" LastActivityDate="2016-04-12T08:28:31.913" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23669" PostTypeId="1" CreationDate="2016-04-12T12:53:39.057" Score="3" ViewCount="171" Body="&lt;p&gt;I am a beginner in CFD having written few codes for laminar flow cases using SIMPLE and some other explicit solvers. Now, I want to use my solvers and some other models to solve for the turbulent flow cases. In this regard, problem of decaying turbulence got my attention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I absolutely do not know about how to make the transition from laminar to turbulent, how to write a code for it. I need some basic guidelines, papers and literature that can help me to write a code for this particular problem using the Navier-Stokes equations. And also, understand the basic concepts of the same. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2016-04-13T11:51:34.900" Title="Decaying turbulence and simulation" Tags="|navier-stokes|fluid-dynamics|laminar-flow|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23670" PostTypeId="2" ParentId="23669" CreationDate="2016-04-12T14:18:32.087" Score="3" Body="&lt;p&gt;The first thing to keep in mind is that there is no one turbulence model that works well in all situations.  You need to choose the right model for the right situation.  See this link for reference: &lt;a href=&quot;https://engineering.stackexchange.com/questions/336/which-turbulence-models-are-suitable-for-cfd-analysis-on-a-streamlined-vehicle-b/344#344&quot;&gt;https://engineering.stackexchange.com/questions/336/which-turbulence-models-are-suitable-for-cfd-analysis-on-a-streamlined-vehicle-b/344#344&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many types of models, all with varying degrees of accuracy and computational cost.  Generally speaking: the more accurate the model is, the more computational cost it incurs.  So you need to consider very carefully how much computational expense you can afford versus how much accuracy you really need.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that said, it is not trivial to convert laminar code to one that handles turbulence.  Depending on the model, you may need up to 7 additional variables and equations, and heuristics to specify these new values at the boundaries.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on whether you're considering compressible or incompressible flow, you may need to modify your momentum and energy equations with additional terms which include these new variables, along with varying expressions for the turbulent viscosity and turbulent Prandtl number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation:  have a look at OpenFoam, an open source cfd solver with lots of turbulence models already implemented.  You can get a good idea of how to modify your code from this (assuming your solver uses finite volume method). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Admittedly, OpenFoam has a steep learning curve and may take a while to master navigating it.   But I think this is your best bet when it comes to learning implementation of generalized cfd code.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="-1" LastEditDate="2017-04-13T12:51:48.613" LastActivityDate="2016-04-13T11:51:34.900" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23671" PostTypeId="2" ParentId="23657" CreationDate="2016-04-13T04:06:51.043" Score="3" Body="&lt;p&gt;In my case when I was doing the multilevel refinement, I used to write grid files at different levels separately. Usually the level 0 is a base coarse mesh which remains fixed. And for level 1 and up, I used different files for different patches. Say if you have 5 fine patches on level 1, my strategy was to write five different files at the end of my program, something like Level_1_Patch_1,Level_1_Patch_2... which could dynamically be done in C (never worked in Fortran).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        char filename1[64];&#xA;        sprintf(filename1, &amp;quot;Patch%d.dat&amp;quot;, patch_number);&#xA;        &#xA;        FILE *file;&#xA;        file= fopen(filename1, &amp;quot;wb&amp;quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I should mention here that I used a finite difference, structured grid system.&lt;/p&gt;&#xA;&lt;p&gt;So once I had all the levels and their .PLT or .DAT files. I import all of them in TecPLOT. For example, when I was writing my two-level code with a couple of fine patches on level 1, your mesh should look like this in Tecplot.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DnMam.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DnMam.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am not very familiar with handling unstrucutred grids but structured ones are handled well in Tecplot&lt;/p&gt;&#xA;" OwnerUserId="16397" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-04-13T04:06:51.043" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23673" PostTypeId="1" CreationDate="2016-04-13T11:47:32.523" Score="2" ViewCount="228" Body="&lt;p&gt;I want to solve the problem below &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;      \begin{aligned}&#xA;        \eta u-\Delta u &amp;amp;=f, &amp;amp;\text{in $\Omega$}\\&#xA;      \end{aligned}&#xA;    \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Omega=(0,1)\times (0,1)$, My boundary are like this $\frac{\partial u}{\partial n} = 0$ if $y=0$ or $1$, and $u=0$ if $x=0$ or $1$. Using ghost nodes can handle the low order discretization of the neumann boundary, but the problem that i'm facing is that I want to include the four corner points in &#xA;the neumann boundary not in the dirichlet one, and this causes me problem since I cannot use the interior equation on the corners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope my question was clear.&lt;/p&gt;&#xA;" OwnerUserId="10258" LastActivityDate="2016-04-13T18:26:02.360" Title="Apply second order finite difference discretization for mixed boundary condition" Tags="|boundary-conditions|discretization|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23674" PostTypeId="1" CreationDate="2016-04-13T17:43:14.527" Score="1" ViewCount="145" Body="&lt;p&gt;I implement Runge--Kutta method for time integration of the system of nonlinear conservation laws&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_t + f(u)_x = 0. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the system is nonlinear, we have to recompute time step &lt;code&gt;dt&lt;/code&gt; on each time step. Then we have to find new current time by using code like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cur_time = cur_time + dt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, if we want to integrate to a very large time, then &lt;code&gt;cur_time&lt;/code&gt; will be large while time step &lt;code&gt;dt&lt;/code&gt; will be small, therefore, the error due to the floating-point arithmetics can accumulate significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to avoid this? If time step were constant, I could handle it, but what to do with the nonconstant time step?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 2016-04-17.&lt;/strong&gt; I add an example to demonstrate that the floating-point arithmetics introduce significant error even for a small number of step and small ratio &lt;code&gt;cur_time/dt&lt;/code&gt;. For simplicity, I use constant time step &lt;code&gt;dt&lt;/code&gt; here, nevertheless, the issue with nonconstant time step should be similar. Code is in Python programming language:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-js prettyprint-override&quot;&gt;&lt;code&gt;dt = 0.2&#xA;n = 100&#xA;cur_time = [0]&#xA;for i in range(n):&#xA;    cur_time.append(cur_time[-1] + dt)&#xA;cur_time[-5:]  # Shows last five time points.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which leads to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;19.199999999999964,&#xA;19.399999999999963,&#xA;19.599999999999962,&#xA;19.79999999999996,&#xA;19.99999999999996&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Therefore, even with small time of integration and small number of time steps, the time points are slightly corrupted by accumulation of error due to the floating-point arithmetics.&lt;/p&gt;&#xA;" OwnerUserId="3254" LastEditorUserId="3254" LastEditDate="2016-04-16T21:02:26.280" LastActivityDate="2016-04-18T12:35:19.230" Title="Accurate computation of the current time in time integrator" Tags="|floating-point|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23675" PostTypeId="1" AcceptedAnswerId="23677" CreationDate="2016-04-13T18:10:26.927" Score="3" ViewCount="122" Body="&lt;p&gt;In the book &lt;strong&gt;Mixed Finite Element Methods and Applications&lt;/strong&gt; by Boffi, Brezzi, and Fortin  there is a pretty long discussion about why the Raviart-Thomas (RT) projection is only defined for functions in $H^1$ (Remark 2.5.1).  In addition, For DG methods there are similar projections, only defined for functions in $H^1.$   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To set the stage for my question, I introduce a linear elliptic PDE.  Let $\Omega\in \mathbb{R}^d$ with $d\in \{2,3\}$ be a bounded polyhedral domain. Suppose a source function $\sigma\colon \Omega\to \mathbb{R}$, is in $L^2(\Omega)$,  the Dirichlet boundary condition $u_d\colon \partial\Omega_D\to \mathbb{R}$, and the Neumann boundary condition $\vec{g}_N\cdot\vec{\eta}\colon \partial\Omega_N \to \mathbb{R}$ are all functions with enough regularity such that the following PDE is well defined.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our PDE is&#xA;\begin{align*}&#xA;  0 &amp;amp;= \vec{q} + \vec{\nabla} u&amp;amp;&amp;amp; x\in \Omega,\\&#xA;  \sigma &amp;amp;= \vec{\nabla} \cdot \vec{q}&amp;amp;&amp;amp; x\in \Omega,\\&#xA;  u_d &amp;amp;= u&amp;amp;&amp;amp; x\in \partial\Omega_D,\\&#xA;  \vec{g}_N\cdot \vec{n}&amp;amp;= \vec{q}\cdot \vec{n} &amp;amp;&amp;amp; x\in \partial\Omega_N.&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are weak restrictions that we need to place on the data (the domain, the source term, and the boundary conditions) so that $\vec{q}\in H^1(\Omega)$?  Am I correct in assuming that we need some sort of elliptic regularity?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;References to the literature are appreciated.  I have many papers dealing with elliptic regularity but they all deal with even simpler PDEs (homogenous BC, smooth boundary, no Neumann data, etc).   &lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2016-04-13T18:59:17.427" LastActivityDate="2016-04-13T18:59:17.427" Title="In mixed elliptic formulation, what are the weakest requirements to ensure the flux is in $H^1$?" Tags="|finite-element|discontinuous-galerkin|elliptic-pde|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23676" PostTypeId="2" ParentId="23673" CreationDate="2016-04-13T18:26:02.360" Score="2" Body="&lt;p&gt;Mathematically speaking, you can only pose boundary conditions on parts of the boundary that has nonzero $d-1$ dimensional measure. In other words, it doesn't make sense to say that you want a particular boundary condition to hold at individual points. Whether you do or do not do anything at a corner point, you will nevertheless converge to the same solution. In other words, just don't bother with figuring out whether or not you have Dirichlet or Neumann conditions at these corners.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-13T18:26:02.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23677" PostTypeId="2" ParentId="23675" CreationDate="2016-04-13T18:28:39.547" Score="2" Body="&lt;p&gt;I don't think you get this from your setup. If $\sigma$ is only in $L^2$, then the equation $\nabla \cdot q = \sigma$ would suggest that you only get $q \in H_\text{div}$, regardless of what boundary conditions you have or how smooth the domain's boundary is.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-13T18:28:39.547" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23678" PostTypeId="2" ParentId="23674" CreationDate="2016-04-13T18:30:42.037" Score="4" Body="&lt;p&gt;This shouldn't happen. You get floating point errors if &lt;code&gt;dt&lt;/code&gt; is, say, $10^{-13}$ times &lt;code&gt;cur_time&lt;/code&gt;. But this would mean that you have already done around $10^{13}$ time steps, which I think you are unlikely to have done, just because that would have taken a humongously long time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the likelihood that you do so many time steps that round-off is an error is pretty small.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-13T18:30:42.037" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23679" PostTypeId="1" AcceptedAnswerId="24494" CreationDate="2016-04-13T19:35:26.183" Score="3" ViewCount="590" Body="&lt;p&gt;I'am trying to implement a path following algorithm based on MPC (Model Predictive Control), found in this paper : &lt;a href=&quot;http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/189/pdf/imm189.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Path Following Mobile Robot in the Presence of Velocity Constraints&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Principle:&lt;/strong&gt; Using the robot model and the path, the algorithm predict the behavior of the robot over N future steps to compute a sequence of commands $(v,\omega)$ to allow the robot to follow the path without overshooting the trajectory, allowing to slow down before a sharp turn, etc.&lt;br&gt;&#xA;$v:$ Linear velocity&lt;br&gt;&#xA;$\omega:$ Angular velocity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The robot:&lt;/strong&gt; I have a non-holonomic robot like this one (Image extracted from the paper above) :&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/K6MRY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K6MRY.png&quot; alt=&quot;Non-holonomic robot with castor wheel. (Extracted from the paper above)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here is my problem:&lt;/strong&gt; Before implementing on the mobile robot, I'am trying to compute the needed matrices (using Matlab) to test the efficiency of this algorithm. At the end of the matrices computation some of them have dimension mismatch&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I did:&lt;/strong&gt;&lt;br&gt;&#xA;The quote below is from §4 (4.1, 4.2, 4.3, 4.4) p6-7 of the paper.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h2&gt;4.1 Model&lt;/h2&gt;&#xA;  &#xA;  &lt;p&gt;$z_{k+1} =  Az_k + B_\phi\phi_k + B_rr_k$ (18) &#xA;  with:&lt;br&gt;&#xA;  $A = \begin{bmatrix} 1 &amp;amp; Tv \\ 0 &amp;amp; 1 \end{bmatrix}$&#xA;  $B_\phi = \begin{bmatrix} {T^2\over2}v^2\\ Tv  \end{bmatrix}$&#xA;  $B_r = \begin{bmatrix} 0 &amp;amp; -Tv \\ 0 &amp;amp; 0 \end{bmatrix}$&lt;br&gt;&#xA;  $T$: sampling period&lt;br&gt;&#xA;  $v$: linear velocity&lt;br&gt;&#xA;  $k$: sampling index (i.e. $t= kT$)&lt;br&gt;&#xA;  $z_k:$ the state vector $z_k = (d_k, \theta_k)^T$ position and angle difference to the reference path&lt;br&gt;&#xA;  $r_k:$ the reference vector $r_k = (0, \psi_k)^T$ with $\psi_k$ is the reference angle of the path at step k &lt;/p&gt;&#xA;  &#xA;  &lt;h2&gt;4.2 Criterion&lt;/h2&gt;&#xA;  &#xA;  &lt;p&gt;The predictive receding horizon controller is based on a minimization of the criterion&lt;br&gt;&#xA;  $J= \Sigma^N_{n=0} (\hat{z}_{k+n} - r_{k+n})^T Q(\hat{z}_{k+n} - r_{k+n}) + \lambda\phi^2_{k+n}$, (20)&lt;br&gt;&#xA;  Subject to the inequality constraint&lt;br&gt;&#xA;  $ P\begin{bmatrix} v_n \\ v_n\phi_n \end{bmatrix} \leq q,$&lt;br&gt;&#xA;  $n=0,..., N,$&lt;br&gt;&#xA;  where $\hat{z}$ is the predicted output, $Q$ is a weight matrix, $\lambda$ is a scalar weight, and $N$ is prediction horizon.&lt;/p&gt;&#xA;  &#xA;  &lt;h2&gt;4.3 Predictor&lt;/h2&gt;&#xA;  &#xA;  &lt;p&gt;An n-step predictor $\hat{z}_{k+n|k}$ is easily found from iterating (18). Stacking the predictions $\hat{z}_{k+n|k},n = n,...,N$ in the vector $\hat{Z}$ yields&lt;br&gt;&#xA;  $\hat{Z} = \begin{bmatrix} \hat{z}_{k|k} \\ \vdots \\ \hat{z}_{k+N|k}\end{bmatrix} = Fz_k + G_\phi\Phi_k + G_rR_k$  (22)&lt;br&gt;&#xA;  with&lt;br&gt;&#xA;  $\Phi_k = \begin{bmatrix} \phi_k, \ldots, \phi_{k+N}\end{bmatrix}^T$,&lt;br&gt;&#xA;  $R_k = \begin{bmatrix} r_k, \ldots, r_{k+N}\end{bmatrix}^T$,&lt;br&gt;&#xA;  and&lt;br&gt;&#xA;  $F = \begin{bmatrix}I &amp;amp; A &amp;amp; \ldots &amp;amp; A^N \end{bmatrix}^T$&lt;br&gt;&#xA;  $G_i = \begin{bmatrix} 0 &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0 &amp;amp; 0 \\ B_i &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0 &amp;amp; 0 \\ AB_i &amp;amp; B_i &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots \\ \vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; 0 &amp;amp; 0 \\ A^{N-1}B_i &amp;amp; \ldots &amp;amp; AB_i &amp;amp; B_i &amp;amp; 0 \end{bmatrix}$&lt;br&gt;&#xA;  where index $i$ should be substituted with either $\phi$ or $r$&lt;/p&gt;&#xA;  &#xA;  &lt;h2&gt;4.4 Controller&lt;/h2&gt;&#xA;  &#xA;  &lt;p&gt;Using the N-step predictor (22) simplifies the criterion (20) to&#xA;  $J_k = (\hat{Z}_k - R_k)^T I_q (\hat{Z}_k - R_k) + \lambda\Phi^T_k\Phi_k$, (23)&#xA;  where $I_q$ is a diagonal matrix of appropriate dimension with instances of Q in the diagonal. The unconstrained controller is found by minimizing (23) with respect to $\Phi$:&lt;br&gt;&#xA;  $\Phi_k = -L_zz_k - L_rR_k$, (24)&lt;br&gt;&#xA;  with&lt;br&gt;&#xA;  $L_z = (lambda + G^T_wI_qG_w)^{-1}G^T_wI_qF$&#xA;  $L_r = (lambda + G^T_wI_qG_w)^{-1}G^T_wI_q(Gr - I)$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'am trying to compute $\Phi_k = -L_zz_k - L_rR_k$ but the dimension of $L_r$ and $R_k$ does not match for matrix multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parameters are :   &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$T=0.1s$&lt;/li&gt;&#xA;&lt;li&gt;$N=10$&lt;/li&gt;&#xA;&lt;li&gt;$\lambda=0.0001$&lt;/li&gt;&#xA;&lt;li&gt;$Q=\begin{bmatrix} 1 &amp;amp; 0 \\ 0 &amp;amp; \delta \end{bmatrix}$ with $\delta=0.02$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I get :&lt;br&gt;&#xA;$R_k$ a (11x2) matrix (N+1 elements of size 2x1, transposed)&lt;br&gt;&#xA;$G_w$ a (22x11) matrix&lt;br&gt;&#xA;$G^T_w$ a (11x22) matrix&lt;br&gt;&#xA;$I_q$ a (22x22) matrix&lt;br&gt;&#xA;$F$ a (22x2) matrix&lt;br&gt;&#xA;$G_r$ a (22x22) matrix    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;so Lz computation gives (according to the matrix sizes):&lt;br&gt;&#xA;$L_z=(scalar + (11x22)(22x22)(22x11))^{-1} (11x22)(22x22)(22x22)$&lt;br&gt;&#xA;a (11x2) matrix.&lt;br&gt;&#xA;as $z_k$ is (2x1) matrix, doing $L_zz_k$ from (24) is fine.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and Lr computation gives (according to the matrix sizes):&lt;br&gt;&#xA;$L_r=(scalar + (11x22)(22x22)(22x11))^{-1} (11x22)(22x22)((22x22) - (22x22))$&lt;br&gt;&#xA;a (11x22) matrix.&lt;br&gt;&#xA;as $R_k$ is (11x2) matrix, doing $L_rR_k$ from (24) is not possible.&lt;br&gt;&#xA;I have a (11x22) matrix multiplicated by a (11x2) matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure I'm missing something big here but unable to see what exactly.&#xA;Any help appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="19991" LastEditorUserId="19991" LastEditDate="2016-04-17T08:32:57.290" LastActivityDate="2016-07-20T16:53:57.317" Title="Mobile robot path following using model predictive control (MPC)" Tags="|optimization|matrix|simulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23680" PostTypeId="1" AcceptedAnswerId="23686" CreationDate="2016-04-14T01:33:32.380" Score="3" ViewCount="2133" Body="&lt;p&gt;I have many 3x3 real symmetric matrices for which I need to determine the eigenvalues. &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenvalue_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; gives a nice non-iterative algorithm for this case, which I have translated into C++:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;boost/math/constants/constants.hpp&amp;gt;&#xA;&#xA;template&amp;lt;typename Real&amp;gt;&#xA;std::vector&amp;lt;Real&amp;gt; eigenvalues(const Real A[3][3])&#xA;{&#xA;    using boost::math::constants::third;&#xA;    using boost::math::constants::pi;&#xA;    using boost::math::constants::half;&#xA;&#xA;    static_assert(std::numeric_limits&amp;lt;Real&amp;gt;::is_iec559,&#xA;                  &quot;Template argument must be a floating point type.\n&quot;);&#xA;&#xA;    std::vector&amp;lt;Real&amp;gt; eigs(3, std::numeric_limits&amp;lt;Real&amp;gt;::quiet_NaN());&#xA;    auto p1 = A[0][1]*A[0][1] + A[0][2]*A[0][2] + A[1][2]*A[1][2];&#xA;    auto diag_sq = A[0][0]*A[0][0] + A[1][1]*A[1][1] + A[2][2];&#xA;    if (p1 == 0 || 2*p1/(2*p1 + diag_sq) &amp;lt; std::numeric_limits&amp;lt;Real&amp;gt;::epsilon())&#xA;    {&#xA;        eigs[0] = A[0][0];&#xA;        eigs[1] = A[1][1];&#xA;        eigs[2] = A[2][2];&#xA;        return eigs;&#xA;    }&#xA;&#xA;    auto q = third&amp;lt;Real&amp;gt;()*(A[0][0] + A[1][1] + A[2][2]);&#xA;    auto p2 = (A[0][0] - q)*(A[0][0] - q) + (A[1][1] - q)*(A[1][1] -q) + (A[2][2] -q)*(A[2][2] -q) + 2*p1;&#xA;    auto p = std::sqrt(p2/6);&#xA;    auto invp = 1/p;&#xA;    Real B[3][3];&#xA;    B[0][0] = A[0][0] - q;&#xA;    B[0][1] = A[0][1];&#xA;    B[0][2] = A[0][2];&#xA;    B[1][1] = A[1][1] - q;&#xA;    B[1][2] = A[1][2];&#xA;    B[2][2] = A[2][2] - q;&#xA;    auto detB = B[0][0]*(B[1][1]*B[2][2] - B[1][2]*B[1][2])&#xA;              - B[0][1]*(B[0][1]*B[2][2] - B[1][2]*B[0][2])&#xA;              + B[0][2]*(B[0][1]*B[1][2] - B[1][1]*B[0][1]);&#xA;    auto r = invp*invp*invp*half&amp;lt;Real&amp;gt;()*detB;&#xA;    if (r &amp;gt;= 1)&#xA;    {&#xA;        eigs[0] = q + 2*p;&#xA;        eigs[1] = q - p;&#xA;        eigs[2] = 3*q - eigs[1] - eigs[0];&#xA;        return eigs;&#xA;    }&#xA;&#xA;    if (r &amp;lt;= -1)&#xA;    {&#xA;        eigs[0] = q + p;&#xA;        eigs[1] = q - 2*p;&#xA;        eigs[2] = 3*q - eigs[1] - eigs[0];&#xA;        return eigs;&#xA;    }&#xA;&#xA;    auto phi = third&amp;lt;Real&amp;gt;()*std::acos(r);&#xA;    eigs[0] = q + 2*p*std::cos(phi);&#xA;    eigs[1] = q + 2*p*std::cos(phi + 2*third&amp;lt;Real&amp;gt;()*pi&amp;lt;Real&amp;gt;());&#xA;    eigs[2] = 3*q - eigs[0] - eigs[1];&#xA;&#xA;    return eigs;&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    float M[3][3];&#xA;    M[0][0] = 1.25e6;&#xA;    M[1][0] = 1.25e6;&#xA;    M[0][1] = 1.25e6;&#xA;    M[1][1] = 1.25e6;&#xA;    M[0][2] = 0.0;&#xA;    M[2][0] = 0.0;&#xA;    M[1][2] = 0.0;&#xA;    M[2][1] = 0.0;&#xA;    M[2][2] = 0.0;&#xA;&#xA;    auto eigs = eigenvalues&amp;lt;float&amp;gt;(M);&#xA;    std::cout &amp;lt;&amp;lt; &quot;Eigenvalues: &quot;;&#xA;    std::cout.precision(std::numeric_limits&amp;lt;float&amp;gt;::digits10 + 5);&#xA;    std::cout &amp;lt;&amp;lt; std::fixed &amp;lt;&amp;lt; std::scientific;&#xA;    for (auto eig : eigs)&#xA;    {&#xA;        std::cout &amp;lt;&amp;lt; eig &amp;lt;&amp;lt; &quot;, &quot;;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; '\n';&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The eigenvalues of the matrix are $2.5\times10^6$, 0, and 0. However, the program returns $2.5\times10^6$, $0.0625$, and $0$. Yes, the ratio of the second to the first is roughly the float epsilon, and $q$ and $p$ are nearly equal. But is there a way to stabilize this algorithm so that the loss of precision is not so dramatic?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="9667" LastEditDate="2018-06-03T23:31:55.203" LastActivityDate="2018-06-06T00:06:06.853" Title="Stabilizing a 3x3 real symmetric matrix eigenvalue calculation" Tags="|c++|matrix|eigenvalues|floating-point|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23681" PostTypeId="1" AcceptedAnswerId="23717" CreationDate="2016-04-09T06:42:55.747" Score="3" ViewCount="254" Body="&lt;p&gt;I am solving incompressible 2D Navier-Stokes equations with zero y-component velocity. The geometry is a simple 2D pipe of a length $L$ and diameter $W$ and there is only two boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Non-slip on the pipe walls: $u = 0$&lt;/li&gt;&#xA;&lt;li&gt;Flow velocity at inlet $u = U_{max}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;and so, the two equations I have are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial u}{\partial x} = 0$$&#xA;$$ \frac{\partial P}{\partial x} = \mu \frac{\partial^2 u}{\partial y^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using finite difference discretization I came up with the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.continuity equation:&#xA; $$ u_{i,j} = u_{i+1,j} $$&#xA;2. Pressure using forward difference:&#xA;$$P_{i,j} = P_{i+1,j} - \frac{\mu \triangle x}{(\triangle y) ^2}(u_{i,j+1} + u_{i,j-1} - 2u_{i,j})$$&#xA;3. Pressure using backward difference:&#xA;$$P_{i,j} = P_{i-1,j} + \frac{\mu \triangle x}{(\triangle y) ^2}(u_{i,j+1} + u_{i,j-1} - 2u_{i,j})$$&#xA;4. Manipulating equation (2) to get an equation for $u$:&#xA;$$u_{i,j} = \frac{u_{i,j+1}}{2} + \frac{u_{i,j-1}}{2} - (\frac{P_{i+1,j} - P_{i,j}}{\frac{2\mu \triangle x}{(\triangle y) ^2}}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I shall then use Gauss–Seidel method to solve the previous equations, but since I made two finite difference equation for u and P using the same partial differential equation I can't tell whether this is correct or not as I am getting very weird results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is my approach correct?&lt;/p&gt;&#xA;" OwnerUserId="13154" OwnerDisplayName="Algo" LastEditorUserId="13154" LastEditDate="2016-04-16T04:55:53.777" LastActivityDate="2016-04-19T18:39:18.890" Title="Is this finite difference approach correct?" Tags="|pde|finite-difference|iterative-method|navier-stokes|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="23684" PostTypeId="1" CreationDate="2016-04-14T11:55:26.950" Score="2" ViewCount="433" Body="&lt;p&gt;Consider this integro-differential heat equation taken from &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.12.0/reference/optimize.nonlin.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciPy documentation page&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\nabla^2 P = \alpha \left(\iint_\Omega \cosh(P)dx dy \right)^2 &#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which was found in &lt;a href=&quot;https://scicomp.stackexchange.com/q/10177/720&quot;&gt;this&lt;/a&gt; question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the considered python code the Newton-Krylov iterations are applied and perform quite nice for $\alpha=10$. However this method fails when $\alpha$ is changed from 10 to 15 or more. What might be the reason for this behaviour and is it possible to improve the situation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any references are appreciated, thank you.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: in the first part of the question I asked about possible physical background behind this problem. Several days after I decided to address to scipy-dev newsgroup and almost immediately received a reply from Pauli Virtanen:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is just some random equation. There's no physical or otherwise&#xA;  background for it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This information makes further discussion of the problem much less interesting, so I's rather close this question.&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-04-19T18:49:39.883" Title="Why do Newton-Krylov iterations stagnate in this problem?" Tags="|reference-request|scipy|newton-method|" AnswerCount="2" CommentCount="3" ClosedDate="2016-04-19T19:40:56.690" ContentLicense="CC BY-SA 3.0" />
  <row Id="23685" PostTypeId="1" AcceptedAnswerId="23688" CreationDate="2016-04-14T13:43:13.433" Score="1" ViewCount="517" Body="&lt;p&gt;I need to efficiently rotate a 2D (and 3D) vector in a CUDA kernel. I was thinking about generating random unitary rotation matrices. I don't need to know the angle, it just has to be randomly distributed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some clever way to avoid the computational complexity of the division and the square root for the norm? Or another way to rotate a vector?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need this for a 2D GPU off-lattice Monte-Carlo simulation, but a 3D solution would be nice too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="19989" LastEditorUserId="19989" LastEditDate="2016-04-14T14:11:18.930" LastActivityDate="2016-04-15T18:19:31.807" Title="Efficiently rotate vector in 2D (and 3D)" Tags="|linear-algebra|c++|monte-carlo|blas|cuda|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23686" PostTypeId="2" ParentId="23680" CreationDate="2016-04-14T15:12:07.767" Score="8" Body="&lt;p&gt;This is trying to compute the eigenvalues by computing the roots of the characteristic polynomial. In this case, the characteristic polynomial is $p(t) = t^3-2t^2x$, $x=1.25\times 10^6$, and zero is a root of multiplicity two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is generally no good way of computing double roots of a polynomial to precision better than $O(\sqrt{\epsilon})$, because if the polynomial is specified by its coefficients (as here) there exists a perturbation in the coefficients of magnitude $O(\epsilon)$ that causes the double root to split into two roots separated by $O(\sqrt{\epsilon})$. The same should apply to perturbations of $O(\epsilon)$ in the input matrix too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual advice is that having a backward-stable algorithm (one which computes the exact result for a perturbed input) is the goal, and if you have a problem that is &lt;em&gt;so&lt;/em&gt; sensitive to tiny perturbations in the input then you are probably asking the wrong question, although there are some exceptions. So I'd say the important question here is why it is important for you to accurately compute double eigenvalues of a matrix.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-04-14T15:12:07.767" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23688" PostTypeId="2" ParentId="23685" CreationDate="2016-04-14T22:50:20.480" Score="1" Body="&lt;p&gt;Some of the standard methods for random rotations in 3D (&lt;em&gt;Sphere Point Picking&lt;/em&gt;) are listed on &lt;a href=&quot;http://mathworld.wolfram.com/SpherePointPicking.html&quot; rel=&quot;nofollow&quot;&gt;Mathworld&lt;/a&gt;. On a CPU, Marsaglia's method is quite efficient, because it avoids expensive $\sin$ and $\cos$ computations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Marsaglia's method isn't really suitable for a GPU, because it sometimes rejects pairs of random numbers. If you have access to fast trigonometric functions, the approach of equation (6) to (8) at the &lt;a href=&quot;http://mathworld.wolfram.com/SpherePointPicking.html&quot; rel=&quot;nofollow&quot;&gt;page linked above&lt;/a&gt; probably works better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mathworld.wolfram.com/CirclePointPicking.html&quot; rel=&quot;nofollow&quot;&gt;Circle point picking&lt;/a&gt; in 2D is simpler. In any case, I don't think you can avoid the computation of a vector norm / square root, which is often implemented in hardware nowadays. Drawing random numbers or computing trigonometric functions is probably significantly more expensive.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastEditorUserId="7733" LastEditDate="2016-04-15T18:19:31.807" LastActivityDate="2016-04-15T18:19:31.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23689" PostTypeId="1" CreationDate="2016-04-14T23:36:27.327" Score="0" ViewCount="84" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Suppose that $y(t)$ is the exact solution of the ivp $$y'(t)=f(t,y(t)), y(0)=y_0$$ and $u(t)$ is any approximation to $y(t)$ with $u(0)=y(0)$. Define the error $e(t)=y(t)-u(t)$. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How can I show that $e(t)$ satisfies the ivp $$e'(t)=f(t,u(t)+e(t))-u'(t), e(0)=0$$ And if we say that $f(t,y)=\lambda y$ for some constant $\lambda$, how can we solve the ivp from my previous question to show that $u(t)+e(t)=y(t)$?&lt;/p&gt;&#xA;" OwnerUserId="20012" LastActivityDate="2016-05-15T02:27:53.793" Title="IVP Using Numerical Methods" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23690" PostTypeId="1" AcceptedAnswerId="23693" CreationDate="2016-04-15T01:16:56.043" Score="1" ViewCount="654" Body="&lt;p&gt;I am trying to use the subroutine &lt;code&gt;twofft&lt;/code&gt; which in turn uses subroutine &lt;code&gt;four1&lt;/code&gt; in the book(Numerical Recipes in Fortran) to compute the FFT of two double precision data arrays. If you carefully look at the &lt;code&gt;twofft&lt;/code&gt; routine, the arrays &lt;code&gt;fft1&lt;/code&gt; and &lt;code&gt;fft2&lt;/code&gt; are declared as complex which are sent as arguments to &lt;code&gt;four1&lt;/code&gt; which declares the same two arrays as double precision( or real if you are using the real version). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this allowed?  I am unable to obtain the correct FFT for my data. Any suggestion how to get around this problem? I just want to perform a quick FFT and a deconvolution and not have to go through the entire process of understanding the code. &lt;/p&gt;&#xA;" OwnerUserId="19629" LastEditorUserId="10134" LastEditDate="2016-04-15T11:14:44.497" LastActivityDate="2016-04-15T12:26:11.927" Title="FFT (Numerical Recipes in FORTRAN 77)" Tags="|data-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23691" PostTypeId="2" ParentId="23689" CreationDate="2016-04-15T02:05:15.140" Score="2" Body="&lt;p&gt;For the first question, notice that&#xA;\begin{align*}&#xA;e' &amp;amp;= (y-u)' &#xA;\\&#xA;&amp;amp;= y' - u' &#xA;\\&#xA;&amp;amp;= f(t,y) - u' \tag{1}.&#xA;\end{align*}&#xA;In equation (1) we can use $y=u+e$.  This gives&#xA;$$&#xA;e' = f(t,u+e)-u'.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second question, if $y'=\lambda y$, then using separation of variables gives $y(t) = y_0 e^{\lambda t}$.  Since $f(t,y) =\lambda y$, the differential equation for $e$ takes the form&#xA;\begin{align*}&#xA;e' &amp;amp;=  f(t,u+e) - u'  &#xA;\\&#xA;&amp;amp;= \lambda (u+e)-u'.&#xA;\end{align*}&#xA;Put $e$ on the left hand side, and $u$ on the right hand side.  This gives&#xA;$$&#xA;e' -\lambda e = - (u'-\lambda u).&#xA;$$&#xA;Using the integrating factor $\mu(t) = \exp{(-\lambda t)}$, we obtain&#xA;$$&#xA;(\mu(t) e )' = - (\mu(t) u)'&#xA;\implies&#xA;\int (\mu(t) e )'\,dt =- \int (\mu(t) u)'\,dt.&#xA;$$&#xA;Hence, $\mu(t) e = -\mu(t) u +C$, where $C$ is a constant of integration.  Using the initial condition $e(0)=0$, one finds that $C=y_0$.  Solving for $e$ then gives&#xA;$$&#xA;e(t) = -u(t) + \frac{y_0}{\mu(t)}.&#xA;$$&#xA;Notice that $1/\mu(t) = \exp{( \lambda t)}$, so &#xA;$$&#xA;e(t) = -u(t) + \frac{y_0}{\mu(t)}&#xA;=&#xA;-u(t) +y_0\exp{( \lambda t)}&#xA;=&#xA;-u(t)+y(t).&#xA;$$&#xA;Thus, $u(t)+e(t) =y(t)$.&lt;/p&gt;&#xA;" OwnerUserId="11604" LastActivityDate="2016-04-15T02:05:15.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23692" PostTypeId="1" CreationDate="2016-04-15T03:37:14.377" Score="2" ViewCount="526" Body="&lt;p&gt;The goal is to couple system of two nonlinear differential equations by applying appropriate space and time discretization and Newton-Raphson scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations system is&#xA;$$\left[&#xA;\begin{array}{c}&#xA;F_{1}(u,v)       \\&#xA;F_{2}(u,v)&#xA;\end{array}\right]&#xA;=&#xA;\left[&#xA;\begin{aligned}&#xA;-\frac{\partial^2 u}{\partial x^2} &amp;amp;= v     \\&#xA;\frac{\partial v}{\partial t}&#xA;&amp;amp;=\frac{\partial J(u,v)}{\partial x}&#xA;\end{aligned}&#xA;\right]&#xA;$$&#xA;where $u(x,t), v(x,t)$ is function of time and space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice $F_{1}$ is differential equation in space exclusively, and $F_{2}$ is partial differential equation in space and time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discretization in space and time are subscript $i$ and superscript $j$ respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $F_1$, discretizing in space gives&#xA;$$&#xA;F_{1}(u,v)\quad\rightarrow\quad&#xA;\frac{u_{i+1}-2u_{i}+u_{i-1}}{\Delta x^2}+v_{i}=0&#xA;$$&#xA;The next is discretizing $F_2$ in space.&#xA;$$&#xA;F_{2}(u,v)\quad\rightarrow\quad&#xA;\frac{\partial v}{\partial t}&#xA;=&#xA;\frac{J_{i+1}-J_{i}}{\Delta x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assumption is made; $J_{i+1}$ is function of $u_{i}$, $u_{i+1}$, $v_{i}$ and $v_{i+1}$ and  $J_{i}$ is function of $u_{i-1}$, $u_{i}$, $v_{i-1}$ and $v_{i}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In discretizing $F_{2}$ in time, the method TR-BDF2(Trapezoidal Rule - Second-order Backward Differentiation Formula) is applied, which is&#xA;$$&#xA;\frac{\partial q(z)}{\partial t}&#xA;=&#xA;f(z,t)&#xA;\quad\rightarrow\quad&#xA;\frac{q^{j+1}-q^{j}}{\Delta t}=&#xA;\frac{f^{j+1}+f^{j}}{2}&#xA;\quad\rightarrow\quad&#xA;2q^{j+1}-\Delta t f^{j+1}&#xA;-2q^{j}-\Delta t f^{j}&#xA;=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once again, superscript $j$ and $j+1$ denotes discretization in time, not power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying TR-BDF2 method to space discretized $F_{2}$ gives&#xA;$$&#xA;F_{2}(u,v)\quad\rightarrow\quad&#xA;2v_{i}^{j+1}- \frac{\Delta t}{\Delta x}(J_{i+1}^{j+1}-J_{i}^{j+1})&#xA;-2v_{i}^{j}- \frac{\Delta t}{\Delta x}(J_{i+1}^{j}-J_{i}^{j})&#xA;=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, there are two equations, one in only discretized in space and the other discretized in both time and space. How do these two equations be coupled in Newton-Raphson scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does applying TR-BDF2 to $F_{1}$ valid? Since $F_{1}$ has to remain invariant through time...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, $q$ is zero in $F_{1}$, so $f^{j+1}+ f^{j}=0$ (??) Then there are two equations both in time and space discretized.&#xA;$$&#xA;F_{1}(u,v)\quad\rightarrow\quad&#xA;\frac{u_{i+1}^{j+1}-2u_{i}^{j+1}+u_{i-1}^{j+1}}{\Delta x^2}+v_{i}^{j+1}&#xA;+\frac{u_{i+1}^{j+1}-2u_{i}^{j}+u_{i-1}^{j}}{\Delta x^2}+v_{i}^{j}&#xA;=0&#xA;$$&#xA;$$&#xA;F_{2}(u,v)\quad\rightarrow\quad&#xA;2v_{i}^{j+1}- \frac{\Delta t}{\Delta x}(J_{i+1}^{j+1}-J_{i}^{j+1})&#xA;-2v_{i}^{j}- \frac{\Delta t}{\Delta x}(J_{i+1}^{j}-J_{i}^{j})&#xA;=0&#xA;$$&#xA;Applying Newton-Raphson Scheme method to this system is,&#xA;$$&#xA;u^{k+1}=u^{k}+\delta u^{k},&#xA;\qquad\qquad&#xA;\left[&#xA;\mathbf{J}(u^{k})&#xA;\right]\delta u^{k}&#xA;=&#xA;-&#xA;\left[&#xA;\mathbf{F}(u^{k})&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where Jacobian and $\mathbf{F}$ is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[&#xA;\begin{array}{*{8}c}&#xA;U_{1D} &amp;amp; U_{1U} &amp;amp;        &amp;amp;       &amp;amp; V_{1D} &amp;amp;      &amp;amp;      &amp;amp;       \\&#xA;U_{1L} &amp;amp; U_{1D} &amp;amp; U_{1U} &amp;amp;       &amp;amp;        &amp;amp;V_{1D}&amp;amp;      &amp;amp;       \\&#xA;       &amp;amp; U_{1L} &amp;amp; \ddots &amp;amp; U_{1U}&amp;amp;        &amp;amp;      &amp;amp;\ddots&amp;amp;      \\ &#xA;       &amp;amp;        &amp;amp; U_{1L} &amp;amp; U_{1D}&amp;amp;        &amp;amp;      &amp;amp;      &amp;amp;V_{1D}\\&#xA;U_{2D} &amp;amp; U_{2U} &amp;amp;        &amp;amp;       &amp;amp; V_{2D} &amp;amp;V_{2U}&amp;amp;      &amp;amp;         \\&#xA;U_{2L} &amp;amp; U_{2D} &amp;amp; U_{2U} &amp;amp;       &amp;amp; V_{2L} &amp;amp;V_{2D}&amp;amp;V_{2U}&amp;amp;          \\&#xA;       &amp;amp; U_{2L} &amp;amp; \ddots &amp;amp; U_{2U}&amp;amp;        &amp;amp;V_{2L}&amp;amp;\ddots&amp;amp;V_{2U}  \\ &#xA;       &amp;amp;        &amp;amp; U_{2L} &amp;amp; U_{2D}&amp;amp;        &amp;amp;      &amp;amp;V_{2L}&amp;amp;V_{2D}\\&#xA;\end{array}&#xA;\right]&#xA;\left[&#xA;\begin{array}{c}&#xA;\delta u_2\\&#xA;\vdots \\&#xA;\delta u_{n-1}\\&#xA;\delta v_2\\&#xA;\vdots \\&#xA;\delta v_{n-1}&#xA;\end{array}&#xA;\right]&#xA;=&#xA;-&#xA;\left[&#xA;\begin{array}{c}&#xA;F_1(i=2)\\&#xA;\vdots\\&#xA;F_1(i=n-1)\\&#xA;F_2(i=2)\\&#xA;\vdots\\&#xA;F_2(i=n-1)&#xA;\end{array}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $F_{1,2}(i)$ vector is inserting current $i$ value for $j$ terms and guess value $i$ for $j+1$ (next time) terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and each components are (D=main diagonal, U=first upper diagonal, L=first lower diagonal)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;U_{1D}&#xA;&amp;amp;=&#xA;\frac{d}{du_i^{j+1}}F_{1}^{j+1}&#xA;=-\frac{2}{\Delta x^2}&#xA;&amp;amp; (2\leq i\leq n-1)&#xA;\\&#xA;U_{1U}&#xA;&amp;amp;=&#xA;\frac{d}{du_{i+1}^{j+1}}F_{1}^{j+1}&#xA;=\frac{1}{\Delta x^2}&#xA;&amp;amp; (2\leq i\leq n-2)&#xA;\\&#xA;U_{1L}&#xA;&amp;amp;=&#xA;\frac{d}{du_{i-1}^{j+1}}F_{1}^{j+1}&#xA;=\frac{1}{\Delta x^2}&#xA;&amp;amp; (3\leq i\leq n-1)&#xA;\\&#xA;V_{1D}&#xA;&amp;amp;=&#xA;\frac{d}{dv_{i}^{j+1}}F_{1}^{j+1}&#xA;=1&#xA;&amp;amp; (2\leq i\leq n-1)&#xA;\\&#xA;U_{2D}&#xA;&amp;amp;=&#xA;\frac{d}{du_i^{j+1}}F_{2}^{j+1}&#xA;=-\frac{\Delta t}{\Delta x^2}\left(&#xA;\frac{dj_{i+1}^{j+1}}{du_{i}^{j+1}}&#xA;-&#xA;\frac{dj_{i}^{j+1}}{du_{i}^{j+1}}&#xA;\right)&#xA;&amp;amp; (2\leq i\leq n-1)&#xA;\\&#xA;U_{2U}&#xA;&amp;amp;=&#xA;\frac{d}{du_{i+1}^{j+1}}F_{2}^{j+1}&#xA;=-\frac{\Delta t}{\Delta x^2}\left(&#xA;\frac{dj_{i+1}^{j+1}}{du_{i+1}^{j+1}}&#xA;\right)&#xA;&amp;amp; (2\leq i\leq n-2)&#xA;\\&#xA;U_{2L}&#xA;&amp;amp;=&#xA;\frac{d}{du_{i-1}^{j+1}}F_{2}^{j+1}&#xA;=\frac{\Delta t}{\Delta x^2}\left(&#xA;\frac{dj_{i-1}^{j+1}}{du_{i-1}^{j+1}}&#xA;\right)&#xA;&amp;amp; (3\leq i\leq n-1)&#xA;\\&#xA;V_{2D}&#xA;&amp;amp;=&#xA;\frac{d}{dv_i^{j+1}}F_{2}^{j+1}&#xA;=2-\frac{\Delta t}{\Delta x^2}\left(&#xA;\frac{dj_{i+1}^{j+1}}{dv_{i}^{j+1}}&#xA;-&#xA;\frac{dj_{i}^{j+1}}{dv_{i}^{j+1}}&#xA;\right)&#xA;&amp;amp; (2\leq i\leq n-1)&#xA;\\&#xA;V_{2U}&#xA;&amp;amp;=&#xA;\frac{d}{dv_{i+1}^{j+1}}F_{2}^{j+1}&#xA;=-\frac{\Delta t}{\Delta x^2}\left(&#xA;\frac{dj_{i+1}^{j+1}}{dv_{i+1}^{j+1}}&#xA;\right)&#xA;&amp;amp; (2\leq i\leq n-2)&#xA;\\&#xA;V_{2L}&#xA;&amp;amp;=&#xA;\frac{d}{dv_{i-1}^{j+1}}F_{2}^{j+1}&#xA;=\frac{\Delta t}{\Delta x^2}\left(&#xA;\frac{dj_{i-1}^{j+1}}{dv_{i-1}^{j+1}}&#xA;\right)&#xA;&amp;amp; (3\leq i\leq n-1)&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again,  $J_{i+1}$ is function of $u_{i}$, $u_{i+1}$, $v_{i}$ and $v_{i+1}$ and  $J_{i}$ is function of $u_{i-1}$, $u_{i}$, $v_{i-1}$ and $v_{i}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct way?&lt;/p&gt;&#xA;" OwnerUserId="19465" LastActivityDate="2016-04-15T03:37:14.377" Title="Applying Newton-Raphson method to system of two differential equations, one time independent, one time dependent" Tags="|newton-method|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23693" PostTypeId="2" ParentId="23690" CreationDate="2016-04-15T08:03:10.207" Score="4" Body="&lt;p&gt;No, it's illegal by the Fortran standard, but that said most compilers will let you get away with it (if the debugging options are not turned on and provided a correct interface is not in scope at the calling point) as long as you are passing Double Complex to double precision and the arrays of the later are twice the size of the former, and similarly complex to real - I say this because in your question you say &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the arrays fft1 and fft2 are declared as complex which are sent as&#xA;  arguments to four1 which declares the same two arrays as double&#xA;  precision&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which won't work - you have to pass DOUBLE complex to double precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said I wouldn't touch the NR FFT routines with a barge pole. Instead I would install FFTW (&lt;a href=&quot;http://www.fftw.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.fftw.org/&lt;/a&gt;) and use that. If you're using linux you can probably apt-get it or similar.&lt;/p&gt;&#xA;" OwnerUserId="17475" LastEditorUserId="17475" LastEditDate="2016-04-15T12:26:11.927" LastActivityDate="2016-04-15T12:26:11.927" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23694" PostTypeId="1" CreationDate="2016-04-15T09:14:49.040" Score="1" ViewCount="82" Body="&lt;p&gt;I need to find $X$ for which the following expression is minimized:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \min ||Y - F^{-1}(X)||_2 + ||X||_1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $Y$ is an array (of length approx 44000, an &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/audioread.html&quot; rel=&quot;nofollow&quot;&gt;audio sample I will be reading&lt;/a&gt; using Matlab's &lt;code&gt;audioread( )&lt;/code&gt; function) and $F^{-1}$ is the &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ifft.html&quot; rel=&quot;nofollow&quot;&gt;inverse fast Fourier transform&lt;/a&gt; &lt;code&gt;ifft( )&lt;/code&gt;. Subscripts indicate 2-norm and 1-norm respectively.&#xA;Could someone help me going about implementing it using Matlab?&lt;/p&gt;&#xA;" OwnerUserId="20013" LastEditorUserId="651" LastEditDate="2016-04-15T11:10:22.933" LastActivityDate="2016-04-15T11:10:22.933" Title="Need help writing the code for the following optimization" Tags="|optimization|matlab|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="23695" PostTypeId="1" CreationDate="2016-04-15T19:38:08.183" Score="1" ViewCount="295" Body="&lt;p&gt;I am using staggered grid FDTD for solving elastic wave equation. A description of which can be found at (geodynamics.usc.edu/~becker/teaching/557/reading/Virieux1987.pdf). I have generated a synthetic seismogram (description below) using it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Synthetic seismogram&lt;/strong&gt; : To generate a synthetic seismogram, a source is placed at top and an array of receiver is laid along the surface. A synthetic seismogram can be visualize as in this youtube animation (www.youtube.com/watch?v=_Gjr5EWNIQk)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Problem is the spurious oscillations generated in FDTD simulation for elastic wave. I have taken care of grid dispersion which requires 10 grid points per wavelength (minimum). still I feel there is a lot of dispersion in it.&#xA;The model I have taken is a homogeneous elastic model with absorbing (damp) type  boundary conditions along all the edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vp = 2000 m/s; Vs = 1500 m/s;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grid spacing: dx = dv = 5m; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;time sampling: dt = 1msec&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: Ricker (20Hz). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is the response of at one receiver (k/a trace) and a snap of wave &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YxVqE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YxVqE.jpg&quot; alt=&quot;wave&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 1&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently I don't have privileged to add more than two link so I edited and joined two pics of traces for soft and hard types of sources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corrected trace after applying pressure source (soft type) is shown now. But still there are wriggles trailing the wave&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7SMLo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7SMLo.jpg&quot; alt=&quot;Trace_for_soft_n_hard_source&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20015" LastEditorUserId="20015" LastEditDate="2016-04-21T05:02:23.837" LastActivityDate="2016-04-21T05:02:23.837" Title="Unwanted Oscillation in FDM simulation of elastic wave equation" Tags="|finite-difference|hyperbolic-pde|wave-propagation|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23696" PostTypeId="2" ParentId="23649" CreationDate="2016-04-15T20:30:13.453" Score="0" Body="&lt;p&gt;For simplicity, let's assume A and B are solving the same problem. The speedup for alg B at 16 processor cores is about 4 whereas for A it is 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For general information, weak scaling means the amount of work per core is held constant so generally some aspect of problem size is increased in proportion to the number of processors. In this example, as the problem size and number of processors are increased, the amount of useful work done by each processor in both A and B decreases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alg A is &quot;better&quot; in the sense that time to solution for number of cores from 1 to 16 is always better than Alg B. Of course if the efficiency trends would continue then Alg B would be the 'better' choice at 18 cores and above. In fact, Alg B if scaling truly had a floor at 0.2, it might be very attractive for very large problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As other answers have pointed out, and efficiency of order 0.2 or 0.3 is generally considered poor though. My personal opinion is that time to solution is what matters and the best algorithm is the one that gets me the answer soonest within the constraint of available computational power. The question would be more interesting if there was a crossing in performance say if Alg A dropped to 10 % efficiency at 16 cores. There are simulation codes that have different parallel algorithms to choose from that are 'better' for different architectures or even different numbers of available cores. Parallelism always has a cost in communication and often enough also in greater numbers of floating point operations.&lt;/p&gt;&#xA;" OwnerUserId="3450" LastActivityDate="2016-04-15T20:30:13.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23700" PostTypeId="1" AcceptedAnswerId="23709" CreationDate="2016-04-16T08:11:06.840" Score="4" ViewCount="209" Body="&lt;p&gt;I am interested in evolving the (cubic) self-focusing nonlinear Schrödinger equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$i\frac{\partial \psi}{\partial t}+\frac{1}{2}\frac{\partial^2 \psi}{\partial x^2}+\left|\psi\right|^2\psi=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from an initial condition $\psi(x,t_0)=f(x)$, where I am going to restrict my attention to a finite region $x\in [a,b]$. Since I'm not too familiar with the &quot;standard&quot; methods for evolving this equation, the proper choice of algorithm/method isn't obvious to me. For example, I don't know if using a &quot;spectral&quot; method would be better than doing straight integration/evolution. The methods I have read about are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.phys.ksu.edu/personal/washburn/pdf/washburn_thesis_chapter4.pdf&quot; rel=&quot;noreferrer&quot;&gt;Split-Step Fourier Transform&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://math.arizona.edu/~mcl/Miller/MillerLecture11.pdf&quot; rel=&quot;noreferrer&quot;&gt;Inverse Scattering Transform&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782598002576&quot; rel=&quot;noreferrer&quot;&gt;Interpolation&lt;/a&gt; Approaches &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would like a bit of guidance on both what is that &quot;norm&quot; for handling this evolution, as well as what is most efficient. I would also very much appreciate references to sources which directly or indirectly answer my question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please forgive me if this is a vague or overly simple question.&lt;/p&gt;&#xA;" OwnerUserId="9562" LastActivityDate="2016-04-17T06:55:28.407" Title="What is the preferred method for evolving the Nonlinear Schrödinger Equation?" Tags="|pde|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23701" PostTypeId="2" ParentId="20230" CreationDate="2016-04-16T11:05:59.420" Score="0" Body="&lt;p&gt;Yes, you can solve such system (up to several million equations) on an ordinary PC. You will need to use a very accurate implementation of the CG method. The problem is with (a) storing such large matrix in the RAM; and (b) efficient implementation of the major mathematical operation, namely: the matrix-by-vector multiplication. There is a solver that satisfies the above-mentioned requirements; please follow the link below (in particular, that webpage provides an example of a very large system, arising from the Finite Element Method):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://members.ozemail.com.au/~comecau/CMA_Sparse.htm&quot; rel=&quot;nofollow&quot;&gt;http://members.ozemail.com.au/~comecau/CMA_Sparse.htm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20010" LastActivityDate="2016-04-16T11:05:59.420" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23702" PostTypeId="2" ParentId="1445" CreationDate="2016-04-16T11:18:58.573" Score="-5" Body="&lt;p&gt;There is little point in trying to learn the Finite Element Method if a particular textbook doesn't contain really working, well-tested and well-commented codes. There is a book that comes with a CD that contains fully working implementation of the method and algorithms described in the book. The following webpage provides a brief description of the book and an example from it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://members.ozemail.com.au/~comecau/quad_shell.htm&quot; rel=&quot;nofollow&quot;&gt;http://members.ozemail.com.au/~comecau/quad_shell.htm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The book is available from the Amazon website:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0646930818&quot; rel=&quot;nofollow&quot;&gt;http://www.amazon.com/Computational-Geometry-Surfaces-Application-Analysis/dp/0646930818&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. &lt;/p&gt;&#xA;" OwnerUserId="20010" LastActivityDate="2016-04-16T11:18:58.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23704" PostTypeId="1" CreationDate="2016-04-16T13:08:34.690" Score="1" ViewCount="1674" Body="&lt;p&gt;I'm working on a custom &lt;strong&gt;discrete and constrained multi-objective optimization&lt;/strong&gt; problem and I'd like to know which libraries or platforms that implement algorithms like &lt;code&gt;SPEA2&lt;/code&gt; and &lt;code&gt;NSGA&lt;/code&gt; in &lt;strong&gt;C++&lt;/strong&gt; and that allow the addition of custom algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like it to be easily customizable to my problem, efficient, cross-platform and possibly easy to use.&#xA;Some of them (like &lt;code&gt;EO&lt;/code&gt;, &lt;code&gt;MOMH&lt;/code&gt; or &lt;code&gt;Open beagle&lt;/code&gt; don't seem to maintained any more). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.tik.ee.ethz.ch/pisa/?page=pisa.php&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;PISA&lt;/code&gt;&lt;/a&gt; (written in C) doesn't seem to fit my needs since I wish to add a method using multiple populations for the optimization process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help (Propositions of other platforms are welcome)&lt;/p&gt;&#xA;" OwnerUserId="20023" LastActivityDate="2020-08-06T10:36:06.043" Title="Which C++ Multi-objective Optimization libraries allows the addition of custom problems and custom algorithms?" Tags="|optimization|c++|constrained-optimization|libraries|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23705" PostTypeId="2" ParentId="23704" CreationDate="2016-04-16T15:12:47.790" Score="0" Body="&lt;p&gt;Would the COCO framework be useful?&#xA;See the competitions at GECCO 2016 (and earlier conferences).&#xA;&lt;a href=&quot;http://gecco-2016.sigevo.org/index.html/HomePage#&amp;amp;panel1-1&quot; rel=&quot;nofollow&quot;&gt;http://gecco-2016.sigevo.org/index.html/HomePage#&amp;amp;panel1-1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16571" LastActivityDate="2016-04-16T15:12:47.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23706" PostTypeId="1" CreationDate="2016-04-16T20:18:48.237" Score="1" ViewCount="76" Body="&lt;p&gt;Suppose I want to solve the below equation numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dy}{dx}=y&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to normalize the space discretization by choosing&#xA;$$&#xA;a\bar{x}=x&#xA;$$&#xA;where I assume $\bar{x}$ is unity.&#xA;Then the equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{a}\frac{dy}{d\bar{x}}=y&#xA;\qquad\rightarrow\qquad&#xA;\frac{dy}{d\bar{x}}=ay&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, my discretization becomes,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{y_{i+1}-y_{i}}{\Delta \bar{x}}=ay&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which I don't think it makes sense, since $\Delta \bar{x}$ must be unity, so is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;y_{i+1}-y_{i}=ay&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;right?&lt;/p&gt;&#xA;" OwnerUserId="19465" LastEditorUserId="17368" LastEditDate="2016-04-17T02:34:35.663" LastActivityDate="2016-04-17T02:34:35.663" Title="Does scaling factor affect discretization?" Tags="|discretization|scaling|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23709" PostTypeId="2" ParentId="23700" CreationDate="2016-04-17T06:55:28.407" Score="5" Body="&lt;p&gt;You won't find universal agreement among experts on a single best way to solve this equation.  On the other hand, since it is a PDE in one spatial dimension there are many approaches that will give good accuracy in a reasonable amount of time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very simple place to start is a &lt;strong&gt;pseudospectral discretization in space with Runge-Kutta integration in time&lt;/strong&gt;; you can learn about such methods in a lot of good books, or you can find a quick introduction with examples and code &lt;a href=&quot;https://github.com/ketch/PseudoSpectralPython&quot;&gt;here&lt;/a&gt; (disclaimer: that links to something that I wrote).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main issue you'll find with that approach is that the second-derivative term leads to stiffness and forces you to use a time step of order $(\Delta x)^2$.  There are many ways of avoiding this restriction; the simplest is the split-step method you mention, but it introduces its own significant errors.  For other, more accurate ways to deal with the stiffness, you could look at using &lt;strong&gt;an integrating factor&lt;/strong&gt;, &lt;strong&gt;exponential methods&lt;/strong&gt;, or &lt;strong&gt;IMEX additive Runge-Kutta methods&lt;/strong&gt;, all of which are discussed and tested for a variety of PDEs in &lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/publication/PDF/2005_111.pdf&quot;&gt;this excellent paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I won't attempt to comment on approaches 2-3 from your list, as I am not an expert on either.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-04-17T06:55:28.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23710" PostTypeId="2" ParentId="23674" CreationDate="2016-04-17T07:08:42.587" Score="2" Body="&lt;p&gt;Your updated question implies that you need to know the current time very precisely -- so precisely that a difference on the order of $10^{-14}$ is unacceptable.  If you need to track something to an accuracy better than a few times double-precision roundoff error, the only alternative is to use something other than double-precision numbers.  You could use higher-precision floating-point numbers (see &lt;a href=&quot;http://mpmath.org/&quot; rel=&quot;nofollow&quot;&gt;mpmath&lt;/a&gt;) or rational arithmetic (see &lt;a href=&quot;http://www.sympy.org/&quot; rel=&quot;nofollow&quot;&gt;sympy&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If possible, you may want to cast $\Delta t$ to a double-precision float when using it elsewhere in order to avoid slowing down the whole code.  But then when your time variable reaches $T$, your solution may as well be considered as a solution at $\hat{T} = T + \epsilon$ where $\epsilon$ is related to double-precision roundoff anyway).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2016-04-17T09:15:22.290" LastActivityDate="2016-04-17T09:15:22.290" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23711" PostTypeId="2" ParentId="23684" CreationDate="2016-04-17T10:24:59.947" Score="0" Body="&lt;p&gt;Generally speaking, poor convergence (or no convergence at all) of an iterative method that is solving a system of linear algebraic equations is attributed to the fact that the corresponding matrix is ill-conditioned. In mechanical problems, that may be observed if the model is &quot;loose&quot; (no sufficient boundary conditions applied, or if the model consists of materials with very different elastic properties, like, say, steel and foam). For heat transfer problems, that would probably correspond to cases when the physical model consists of different materials with vastly different heat transfer coefficients. &lt;/p&gt;&#xA;" OwnerUserId="20010" LastActivityDate="2016-04-17T10:24:59.947" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23712" PostTypeId="2" ParentId="19936" CreationDate="2016-04-17T10:29:07.260" Score="-5" Body="&lt;p&gt;You just need a finite element that does not manifest shear locking. There is software that implements such element, based on the Mindlin model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://members.ozemail.com.au/~comecau/plate_analytical.htm&quot; rel=&quot;nofollow&quot;&gt;http://members.ozemail.com.au/~comecau/plate_analytical.htm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This software was developed by our company; but it is impossible to provide here full mathematical details of the method implemented in the program. As an alternative, you may consider using an approach used in the following book (it is devoted to shell elements, i.e with curved surfaces, and is based on the Kirchhoff-Love geometric hypothesis; still, that element does not manifest shear locking either, and you may consider adaptation of that element for your purposes):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0646930818&quot; rel=&quot;nofollow&quot;&gt;http://www.amazon.com/Computational-Geometry-Surfaces-Application-Analysis/dp/0646930818&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one more thing to be considered in terms of shear locking. In the theory of plates and shells, the displacements (deflections) field is the principal function that defines all other parameters of the mechanical behavior. Rotations of the nodes represent the 1st derivatives of the displacements; bending moments are the second derivatives; shear forces correspond to the 3rd derivatives. Quite obvious that if the deflections are not interpolated by polynomials of sufficiently high order, shear forces would just not be evaluated with any acceptable degree of precision at all. &lt;/p&gt;&#xA;" OwnerUserId="20010" LastEditorUserId="20010" LastEditDate="2016-04-17T10:54:58.983" LastActivityDate="2016-04-17T10:54:58.983" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23714" PostTypeId="1" AcceptedAnswerId="23753" CreationDate="2016-04-17T15:38:18.120" Score="0" ViewCount="59" Body="&lt;p&gt;I'm not sure if this SE site is the best one for this question, so let me know where it should be moved to if you think it doesn't belong here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After learning about the quadratic formula, I'm interested in visualizing how the nature of the roots of a quadratic equation, $ax^2+bx+c$ change with respect its discriminant. I'd like to visualize the discrete nature of the roots of a quadratic equation over all combinations of $a, b, c$ in $b^2-4ac$, i.e. whether the roots are complex, equal, unequal, rational, or irrational, etc. I'm not sure I fully grasp the nature of my question itself and how to get a mathematically sound answer to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is interesting to me because it deals with discrete properties emerging from continuous ones. I'm wondering what is the best way of visualizing/answering this – should I do random sampling? where can I read more about problems of this nature (i.e. discrete properties on continuous variables)?&lt;/p&gt;&#xA;" OwnerUserId="19718" LastEditorUserId="19718" LastEditDate="2016-04-18T12:53:51.737" LastActivityDate="2016-04-21T14:21:32.553" Title="How to choose a good distribution for visualizing phase changes in the nature of the roots of a quadratic equation" Tags="|discretization|numerics|random-sampling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23716" PostTypeId="2" ParentId="23704" CreationDate="2016-04-17T20:38:05.037" Score="0" Body="&lt;p&gt;I'm not sure if it exactly has SPEA2 or NSGA, but check out &lt;a href=&quot;https://dakota.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;DAKOTA&lt;/a&gt;. It has all manner of genetic algorithms that can tackle discrete and continuous optimization problems. You can either link to it directly, or use it as a driver which calls your programs as a black box. It's all open source so I'm sure you could add your own custom algorithms, though I can't speak to how easy that is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall it's a fantastic and very capable program for optimization (and many other related problems).&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2016-04-17T20:38:05.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23717" PostTypeId="2" ParentId="23681" CreationDate="2016-04-17T23:00:21.770" Score="3" Body="&lt;p&gt;There are two aspects to your question I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Do your equations match the physical problem you're trying to model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Do your finite difference equations converge to the continuous ones as dx and dt approach zero?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I'd like to address question 1, since this affects question 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;: Assuming you're looking for a general (developing) flow solution, this image is a helpful illustration:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/8/84/Development_of_fluid_flow_in_the_entrance_region_of_a_pipe.jpg/1280px-Development_of_fluid_flow_in_the_entrance_region_of_a_pipe.jpg&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the entrance region (ER) extends to a critical point where the flow reaches a fully developed (FD) state. In the FD region, there's only 1 non-zero component of velocity, however, in ER there are multiple components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The governing equation (for the entire domain) are momentum and continuity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \mathbf{u}}{\partial t} + \mathbf{u} \bullet \nabla \mathbf{u} = - \nabla p + \frac{1}{Re} \nabla^2 \mathbf{u}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \bullet \mathbf{u} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;http://ingforum.haninge.kth.se/armin/AR_2000/HL2008/NAVIER_STOKES_EQ.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, this simplifies to (assuming $u_{\theta} = 0, \frac{\partial}{\partial \theta} = 0$, which also implies that we're assuming a laminar solution)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;r:&#xA;$\frac{\partial u_r}{\partial t} + u_r \frac{\partial u_r}{\partial r} + u_z \frac{\partial u_r}{\partial z} = -\frac{\partial p}{\partial r} + \frac{1}{Re} \left[ \frac{1}{r} \frac{\partial}{\partial r} \left( r \frac{\partial u_r}{\partial r} \right) - \frac{u_r}{r^2} + \frac{\partial^2 u_r}{\partial z^2} \right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;z:&#xA;$&#xA;\frac{\partial u_z}{\partial t} + u_r \frac{\partial u_z}{\partial r} + u_z \frac{\partial u_z}{\partial z}&#xA;=&#xA;-\frac{\partial p}{\partial z} + \frac{1}{Re}&#xA;\left[ \frac{1}{r} \frac{\partial}{\partial r} \left( r \frac{\partial u_z}{\partial r} \right)&#xA;+ \frac{\partial^2 u_z}{\partial z^2}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuity:&#xA;$\frac{1}{r} \frac{\partial (r u_r)}{\partial r} +\frac{\partial u_z}{\partial z} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this, we may apply general inlet BCs as you suggested:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_r = u_{r,inlet}(r,0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_z = u_{z,inlet}(r,0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likely outlet BCs might be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_r = 0 $ (more fewer unknowns than $\frac{\partial u_r}{\partial z} = 0$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u_z}{\partial z} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;: Since there are many different approaches to solve equations in question 1, I will just briefly comment on some approaches rather than writing finite difference equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may discretize in space and time and then apply a solution method (you suggested Gauss-Seidel). You may use staggered variables, where velocity and pressure are located on the cell face and center respectively, or you may use a collocated scheme, but then you will need to compute fluxes for the advection term in order to avoid pressure checkerboard phenomena. Typically, 2nd-order central difference schemes are applied to these equations for spatial derivatives, unless higher order discretization is needed. Since it seems you've written your equations for steady state, I'll assume that you're not interested in transient behavior and suggest using 1st-order (explicit Euler) time marching.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="8174" LastEditDate="2016-04-19T18:39:18.890" LastActivityDate="2016-04-19T18:39:18.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23718" PostTypeId="1" CreationDate="2016-04-18T03:23:17.913" Score="1" ViewCount="65" Body="&lt;p&gt;I have a simulated system as shown in the following image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vTnKn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vTnKn.png&quot; alt=&quot;bodies in simulation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L_0$ is attached to two other bodies $L_1$ and $L_2$. Furthermore, body $L_3$ is also in the simulation (it is attached to $L_2$ but that is not important). Each body has a position, so $L_0$'s position is $\vec{P}^0 = (x_0, y_0)$, $L_1$'s position is $\vec{P}^1 = (x_1, y_1)$, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L_0$'s movement is governed by two forces, a linear attraction force to bodies it is attached to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec{A}_0 = \frac{\Delta P}{\Delta t} = k(\vec{P}^1 - \vec{P}^0) + k(\vec{P}^2 - \vec{P}^0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which adheres to Hook's Law of springs: $\vec{F} = -kr$ where $r$ is the distance between the two bodies. and a non-linear repulsion force which acts on $L_0$ from all other bodies:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec{R} = \frac{\Delta P}{\Delta t} = \vec{U}^{01} \frac{Gm^0m^1}{|\vec{P}^0 - \vec{P}^1|^2} + \vec{U}^{02} \frac{Gm^0m^2}{|\vec{P}^0 - \vec{P}^2|^2} + \vec{U}^{03} \frac{Gm^0m^3}{|\vec{P}^0 - \vec{P}^3|^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\vec{U}^{01}$ is the direction vector from $L_1$ to $L_0$, so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec{U}^{01} = \frac{\vec{P}^1 - \vec{P}^0}{|\vec{P}^1 - \vec{P}^0|}$ and the same is true for $\vec{U}^{02}$ and $\vec{U}^{03}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This force adheres to Newton's law of gravity between two bodies with masses $m^1$ and $m^2$, $\vec{F} = \frac{Gm^1m^2}{r^2}$ where $r$ is the distance between the two bodies (so the repulsion increases quadratically as the bodies get closer)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In essence then, $L_0$'s movement is governed by the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\Delta P}{\Delta t} = k(\vec{P}^1 - \vec{P}^0) + k(\vec{P}^2 - \vec{P}^0) + \vec{U}^{01} \frac{Gm^0m^1}{|\vec{P}^0 - \vec{P}^1|^2} + \vec{U}^{02} \frac{Gm^0m^2}{|\vec{P}^0 - \vec{P}^2|^2} + \vec{U}^{03} \frac{Gm^0m^3}{|\vec{P}^0 - \vec{P}^3|^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my simulation, I want to use the implicit (Backward Euler) method of the discretized equation, so I discretize to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec{P}_{n+1}^0 = \vec{P}_n^0 + \Delta tk(\vec{P}_n^1 - \vec{P}_n^0) + \Delta tk(\vec{P}_n^2 - \vec{P}_n^0) + \Delta t\vec{U}_n^{01} \frac{Gm^0m^1}{|\vec{P}_n^0 - \vec{P}_n^1|^2} + \Delta t\vec{U}_n^{02} \frac{Gm^0m^2}{|\vec{P}_n^0 - \vec{P}_n^2|^2} + \Delta t\vec{U}_n^{03} \frac{Gm^0m^3}{|\vec{P}_n^0 - \vec{P}_n^3|^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then substitute $P_{n+1}^0$ instead of $P_n^0$ at the appropriate places to get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec{P}_{n+1}^0 = \vec{P}_n^0 + \Delta tk(\vec{P}_n^1 - \vec{P}_{n+1}^0) + \Delta tk(\vec{P}_n^2 - \vec{P}_{n+1}^0) + \Delta t\frac{\vec{P}_{n+1}^0 - \vec{P}_n^1}{|\vec{P}_{n+1}^0 - \vec{P}_n^1|} \frac{Gm^0m^1}{|\vec{P}_{n+1}^0 - \vec{P}_n^1|^2} + \Delta t\frac{\vec{P}_{n+1}^0 - \vec{P}_n^2}{|\vec{P}_{n+1}^0 - \vec{P}_n^2|} \frac{Gm^0m^2}{|\vec{P}_{n+1}^0 - \vec{P}_n^2|^2} + \Delta t\frac{\vec{P}_{n+1}^0 - \vec{P}_n^3}{|\vec{P}_{n+1}^0 - \vec{P}_n^3|} \frac{Gm^0m^3}{|\vec{P}_{n+1}^0 - \vec{P}_n^3|^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where I'm stuck. How do I solve for $P_{n+1}^0$? Can I isolate it? do I need to use Newton's method, and if so how do I do that for a 2-coordinate vector?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example talks about $L_0$, but I actually have to do it for each of the bodies described. So I hope the answer I get could be generalized for $L_1$, $L_2$, and $L_3$&lt;/p&gt;&#xA;" OwnerUserId="19906" LastEditorUserId="19906" LastEditDate="2016-04-18T08:09:52.100" LastActivityDate="2016-04-18T08:09:52.100" Title="How to solve bring my implicit equation to closed form?" Tags="|ode|implicit-methods|" AnswerCount="0" CommentCount="3" ClosedDate="2019-06-06T20:22:18.477" ContentLicense="CC BY-SA 3.0" />
  <row Id="23720" PostTypeId="1" CreationDate="2016-04-18T08:09:46.443" Score="3" ViewCount="245" Body="&lt;p&gt;Consider the following Schrodinger equation for the harmonic oscillator with real $x$:&#xA;$$&#xA;−ψ''(x) + x^ 2 ψ(x) = Eψ(x).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solve the last equation using shooting method and implicit Runge-Kutta integration. For the eigenvalue $E=1$, the corresponding eigenfunction looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Z5iq3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z5iq3.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I set $x=4$ as $+∞$ and $x=-4$ as $-∞$ . From the Fig, we see that the eigenfunction exponentially decay at $x=\pm 4$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when eigenvalue $E=17$, the corresponding eigenfunction is too complicated so that it won't exponentially decays at $x=\pm 4$ as shown on the following Fig.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2wNE5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2wNE5.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, the eigenfunction decays at $x=\pm 6$ as shown on the following Fig.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/j72Be.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j72Be.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numerical infinity should be large (like $x=6$) for large eigenvalue $E$, and small (like $x=4$) for small eigenvalue. If independent from the eigenvalue, I uniformly set numerical infinity as a very large number, it would waste computational resource. So, is there any formula to determine the exact numerical infinity based on different inputted eigenvalue please?&lt;/p&gt;&#xA;" OwnerUserId="12010" LastEditorUserId="12010" LastEditDate="2016-04-18T20:48:34.250" LastActivityDate="2016-04-26T13:32:31.123" Title="Determine numerical infinity for Schrodinger equation $−\psi''(x) + x^ 2 \psi(x) = E\psi(x)$" Tags="|numerical-analysis|ode|boundary-conditions|eigenvalues|runge-kutta|" AnswerCount="2" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="23721" PostTypeId="2" ParentId="23720" CreationDate="2016-04-18T08:27:40.083" Score="2" Body="&lt;p&gt;You should maybe look at the analytical form of these solutions &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_harmonic_oscillator&quot; rel=&quot;nofollow&quot;&gt;(here's the wikipedia page)&lt;/a&gt;, this would probably help you to derive a relation between the energy level $n$ and the typical length of your state (I would look at the $H_n$ functions given in the link above).&lt;/p&gt;&#xA;" OwnerUserId="19724" LastActivityDate="2016-04-18T08:27:40.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23722" PostTypeId="2" ParentId="23674" CreationDate="2016-04-18T11:45:02.390" Score="0" Body="&lt;p&gt;Also don't get misleaded by the display of your computer regarding precision and remember that error does not stack the same way depending on your calculations for the same values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np # Personnal comfort, not relevant here.&#xA;dt = 0.2&#xA;n  = 100&#xA;add_time, mul_time = np.zeros(n), np.zeros(n)&#xA;for i in range(n):&#xA;    if i == 0:&#xA;        add_time[i], mul_time[i] = 0.2&#xA;    else:&#xA;        add_time[i] = add_time[i-1] + dt&#xA;        mul_time[i] = (i-1)*dt&#xA;add_time[-5:], mul_time[-5]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which leads to the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(array([ 19.2,  19.4,  19.6,  19.8,  20. ]), array([ 19.2,  19.4,  19.6,  19.8,  20. ]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But for demonstration purpose I had the following print:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print &quot;add_time: {0:18.18f}\n {1:18.18f}\n {2:18.18f}\n {3:18.18f}\n {4:18.18f}\n mul_time: {5:18.18f}\n {6:18.18f}\n {7:18.18f}\n {8:18.18f}\n {9:18.18f}\n&quot;.format(add_time[-5], add_time[-4], add_time[-3], add_time[-2], add_time[-1], mul_time[-5], mul_time[-4], mul_time[-3], mul_time[-2], mul_time[-1])&#xA;add_time: 19.199999999999963762&#xA;19.399999999999963052&#xA;19.599999999999962341&#xA;19.799999999999961631&#xA;19.999999999999960920&#xA;mul_time: 19.200000000000002842&#xA;19.400000000000002132&#xA;19.600000000000001421&#xA;19.800000000000000711&#xA;20.000000000000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My initial guess is also that this works well because dt is set at the value of 0.2 yet you'll always end up with a round-up error due to floats precision. But still, if you can store your values in advance with a minimum set of operation, as for time-step in this short example, do it.&#xA;The lesser number of operations, the smaller the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: One can note the following on simple precision when substracting the expected value from the arrays for the printing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;add_time: -0.000000000000035527&#xA;0.000000000000035527&#xA;0.000000000000039080&#xA;0.000000000000039080&#xA;0.000000000000039080&#xA;mul_time: 0.000000000000003553&#xA;0.000000000000003553&#xA;0.000000000000000000&#xA;0.000000000000000000&#xA;0.0000000000000000000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We are at a precision setup where the errors are not even relevant to be printed.&#xA;Hence, I actually advise to set a flag at each time step change and to calculate the new time step with something like, say:&#xA;$$t_n = t_o + i*dt_n$$&#xA;where $t_o$ is the time at which time-step was changed, $dt_n$ the new time step and $i$ the number of iteration with this new value.&lt;/p&gt;&#xA;" OwnerUserId="19790" LastEditorUserId="19790" LastEditDate="2016-04-18T12:35:19.230" LastActivityDate="2016-04-18T12:35:19.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23723" PostTypeId="2" ParentId="23714" CreationDate="2016-04-18T12:44:31.173" Score="1" Body="&lt;p&gt;Well, for the visualization of the discrete and real/imaginary character of the roots, you can simply do a 3D plot of $$ \frac{-b\pm \sqrt{b^2-4ac}}{2a} $$ against $a$, $b$, and $c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give you a surface linking parabolas/square roots/hyperbolas on the different ($aOc$), ($aOb$), ... planes where the surface disappears because the roots become imaginary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also do the same kind of experiment for the real and imaginary parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand the&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;two equal, two unequal part&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As for the rational, irrational part, it's a bit old for me, but I think that the solution will be rational only if $\Delta$ can be written as a fraction of squares, but I don't know how you could use that anyway... You might want to ask on &lt;a href=&quot;http://math.stackexchange.com&quot;&gt;Math&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19678" LastEditorUserId="19678" LastEditDate="2016-04-18T12:49:07.320" LastActivityDate="2016-04-18T12:49:07.320" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23725" PostTypeId="1" CreationDate="2016-04-19T05:28:24.870" Score="1" ViewCount="123" Body="&lt;p&gt;I need to calculate the second variation of displacement interpolation function $u = \sum N_a u_a$ in Finite Element Analysis, where $N_a$ are the shape functions and $u_a$ are the nodal values. Someone told me the second variation is 0. The reason is as shown below. But I don't understand why $\delta u_a$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you guys please tell me if the second variation of displacement interpolation function really equals to zero? If it is, could you please explain to me why $\delta u_a$ equals constant? Actually, I think $\delta u$ is arbitrary, so I don't think the second variation of displacement is equal zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u = \sum_{a=1}^{n} N_a u_a$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta u = \delta\sum_{a=1}^{n}N_a u_a = \sum_{a=1}^{n}N_a \delta u_a$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As $\delta u$ is constant (This is what I don't understand). Thus&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta^2 u_a = 0 \enspace .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta^2 u = \delta^2\sum_{a=1}^{n}N_a u_a = \sum_{a=1}^{n}N_a \delta^2 u_a = 0 \enspace .$$&lt;/p&gt;&#xA;" OwnerUserId="20061" LastEditorUserId="9667" LastEditDate="2016-04-19T19:18:03.230" LastActivityDate="2016-04-21T23:34:07.927" Title="The second variation of displacement interpolation function in Finite Element Method" Tags="|finite-element|pde|nonlinear-equations|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23727" PostTypeId="1" CreationDate="2016-04-19T09:38:32.060" Score="2" ViewCount="361" Body="&lt;p&gt;I am having an unconstrained optimization problem, where the Hessian is positive semidefinite and block diagonal. The function is strictly convex, hence, the curvature condition ($ s^{T}_{k}y_{k} &amp;gt; 0$) is supposed to be always satisfied. Here, $s_{k}$ and $y_{k}$ are the usual secant pairs, where $s_{k} = x_{k}-x_{k-1}$ and $y_{k}=g_{k}-g_{k-1}$, where $g_{k}$ is the gradient at iteration $k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am noticing that this curvature condition is indeed getting satisfied if I take the entire variable vector and gradient vector. But I was hoping that it will be satisfied at the block level, also, for all the blocks. i.e., by restricting $s$ and $y$ to block level vectors, for all the blocks. This is not being observed for all the blocks. My intuition says that it should happen because the blocks of a PSD matrix are also PSD. Can somebody give more understanding about this?&lt;/p&gt;&#xA;" OwnerUserId="14478" LastActivityDate="2016-04-19T09:38:32.060" Title="Quasi Newton method for block diagonal Hessian" Tags="|linear-algebra|newton-method|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23729" PostTypeId="1" CreationDate="2016-04-19T12:24:49.810" Score="1" ViewCount="215" Body="&lt;p&gt;I would like to know how to compute the &quot;best&quot; cell center for pressure calculation in case of concave polyhedral cells, if it makes any sense. My face centers are defined as faces barycenters and I'm just wondering how to correctly determine the pressure-point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hint?&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="9667" LastEditDate="2016-04-21T15:49:44.667" LastActivityDate="2016-04-27T13:02:55.453" Title="Concave Cells and Finite Volume Method" Tags="|finite-volume|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23730" PostTypeId="2" ParentId="23725" CreationDate="2016-04-19T13:16:07.467" Score="1" Body="&lt;p&gt;The variation operator applies to the functions, not the constants.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Variations work similarly to derivatives.  If your interpolation functions are piecewise linear, the first variation becomes piecewise constant and the second variation becomes zero.  &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2016-04-19T20:07:24.880" LastActivityDate="2016-04-19T20:07:24.880" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23733" PostTypeId="2" ParentId="23194" CreationDate="2016-04-19T14:28:30.740" Score="10" Body="&lt;p&gt;I implemented a complex polygamma function in the Julia standard library (&lt;a href=&quot;https://github.com/JuliaLang/julia/pull/7125&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JuliaLang/julia/pull/7125&lt;/a&gt;).   It is possible to call this from C++ by &lt;a href=&quot;https://docs.julialang.org/en/v1/manual/embedding/&quot; rel=&quot;nofollow noreferrer&quot;&gt;linking to the libjulia library&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The easiest way to do this is to call Julia's &lt;code&gt;@cfunction&lt;/code&gt; construct to get a C/C++ function pointer to the compiled function.  Then you can call it normally like any other C++ function.&lt;/p&gt;&#xA;&lt;p&gt;In particular (1) download and install Julia, (2) launch Julia and use its &lt;a href=&quot;https://docs.julialang.org/en/v1/stdlib/Pkg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;package manager&lt;/a&gt; to &lt;code&gt;add SpecialFunctions&lt;/code&gt; (to install the Julia &lt;a href=&quot;https://github.com/JuliaMath/SpecialFunctions.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpecialFunctions.jl package&lt;/a&gt; that now contains my &lt;code&gt;polygamma&lt;/code&gt; code), (3) compile your C++ code linked with &lt;code&gt;libjulia&lt;/code&gt;.  For example, the following C++ program loads Julia and gets a C++ function pointer &lt;code&gt;polygamma&lt;/code&gt; to call the Julia polygamma function on &lt;code&gt;std::complex&amp;lt;double&amp;gt;&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-cxx prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;julia.h&amp;gt;&#xA;JULIA_DEFINE_FAST_TLS() // only define this once, in an executable (not in a shared library) if you want fast code.&#xA;&#xA;#include &amp;lt;complex&amp;gt;&#xA;typedef std::complex&amp;lt;double&amp;gt; (*polygamma_ptr)(int m, std::complex&amp;lt;double&amp;gt; z);&#xA;polygamma_ptr polygamma = NULL;&#xA;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;int main(int argc, char *argv[])&#xA;{&#xA;    jl_init();&#xA;&#xA;    // get a C function pointer to the Julia polygamma function compiled for&#xA;    // (int, complex&amp;lt;double&amp;gt;) arguments:&#xA;    jl_eval_string(&amp;quot;import SpecialFunctions&amp;quot;);&#xA;    jl_value_t *ret = jl_eval_string(&amp;quot;@cfunction(SpecialFunctions.polygamma, ComplexF64, (Cint, ComplexF64))&amp;quot;);&#xA;    polygamma = (polygamma_ptr) jl_unbox_voidpointer(ret);&#xA;&#xA;    // example: call polygamma(3, 1.7+4.2i) and output it.&#xA;    std::cout &amp;lt;&amp;lt; &amp;quot;polygamma(3, 1.7+4.2i) = &amp;quot; &amp;lt;&amp;lt; polygamma(3, std::complex&amp;lt;double&amp;gt;(1.7, 4.2)) &amp;lt;&amp;lt; &amp;quot;\n&amp;quot;;&#xA;&#xA;    jl_atexit_hook(0);&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is &lt;code&gt;polygamma(3, 1.7+4.2i) = (-0.0184344,0.0162077)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20069" LastEditorUserId="20069" LastEditDate="2020-12-17T19:35:22.667" LastActivityDate="2020-12-17T19:35:22.667" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23734" PostTypeId="2" ParentId="23379" CreationDate="2016-04-19T14:33:29.667" Score="3" Body="&lt;p&gt;You have&#xA;$$&#xA;\begin{gathered}&#xA;u_{xx} - 5u_x = 0 \qquad \text{for} \quad x\in[0,1]\,,\\&#xA;u(0) = 0\,,\\&#xA;u(1) = 1\,,&#xA;\end{gathered}&#xA;$$&#xA;which has analytic solution&#xA;$$u(x) = \frac{\exp(5x)-1}{\exp(5)-1}\,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a computation variable $\xi$ such that $x(\xi)$, we can express derivatives of $u$ in terms of $\xi$&#xA;$$&#xA;\begin{gathered}&#xA;u_\xi = u_x x_\xi\,,\\&#xA;u_{\xi\xi} = u_xx_{\xi\xi} + u_{xx}x_\xi^2\,,&#xA;\end{gathered}&#xA;$$&#xA;hence can transform the ODE into&#xA;$$&#xA;\begin{gathered}&#xA;u_{\xi\xi} - \left(5 x_\xi + \frac{x_{\xi\xi}}{x_\xi}\right)u_\xi = 0\,,\\&#xA;u(\xi = 0) = 0\,,\\&#xA;u(\xi = 1) = 1\,.&#xA;\end{gathered}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equidistribution principal for a monitor function $M(x)$ (&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/S0036142993256441&quot; rel=&quot;nofollow noreferrer&quot;&gt;Huang Russell 1997&lt;/a&gt;, &lt;a href=&quot;https://scicomp.stackexchange.com/a/21849/17671&quot;&gt;Generating a non-uniform grid&lt;/a&gt;) can be expressed as&#xA;$$\left(M(x) x_\xi\right)_\xi = 0\,,$$&#xA;which I think you've rearranged to give&#xA;$$x_{\xi\xi} - \frac{M_\xi}{M}x_\xi = 0\,,$$&#xA;which will put a high density of points where $M$ is high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A finite difference solution on a uniform mesh is given by&#xA;$$&#xA;\left(\frac{1}{\Delta x^2}\delta^2_x - \frac{5}{2\Delta x}\delta_{2x}\right)U = \mathbf{b}&#xA;$$&#xA;where&#xA;$$\mathbf{b} = \left(0,\dotsc,0,\frac{1}{\Delta x^2} - \frac{5}{2\Delta x}\right)^T\,,$$&#xA;and $\delta_{2x}$ and $\delta^2_x$ are the central difference operators for the first and second derivatives respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If solving this ODE with finite differences, we could set the monitor function to be the truncation error of the method. Taking a uniform mesh on 20 points, we can plot the error $E_i = |u(x_i) - U|$ seen in the figure below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y8K3M.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y8K3M.jpg&quot; alt=&quot;Truncation error for uniform finite difference solution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Using any monitor function which goes to zero should ring alarm bells, since we'll get zero mesh density, but I'll carry on for now adding $10^{-8}$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking $M_i = E_i$, equidistributing to the linear interpolant of $M_i$ and solving with non-uniform finite differences, I got the error shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PMkg5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PMkg5.jpg&quot; alt=&quot;Truncation error for non-uniform FD solution with M=E&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of a smoothing algorithm [Zegeling2007, Chapter 7 of Adaptive Computing:Theory and Algorithms by Tao Tang and Jinchao Xu (2007, Scientific Press)],[&lt;a href=&quot;http://oai.cwi.nl/oai/asset/2016/2016A.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Verwer, Blom, Furzeland and Zegeling (1988)&lt;/a&gt;],&#xA;$$&#xA;  \tilde{M}_i = \sum_{j=0}^N M_j \left(\frac{\sigma}{\sigma+1}\right)^{|i-j|}\,,&#xA;$$&#xA;guarantees that &#xA;$$\frac{\sigma}{\sigma+1}\leq \frac{x_{i+1}-x_i}{x_i-x_{i-1}} \leq \frac{\sigma+1}{\sigma}\,.$$&#xA;Using this with $\sigma=2$, I get the errors shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ucTAH.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ucTAH.jpg&quot; alt=&quot;Truncation error for non-uniform FD solution with M=Etilde&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not a great advert for adaptivity, but this was just a quick experiment. I did try to iterate this process, using the smoothed error on the non-uniform mesh to define a third mesh etc., which gave lower errors, but this process did not converge.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-04-19T14:33:29.667" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="23735" PostTypeId="1" CreationDate="2016-04-19T16:22:29.737" Score="2" ViewCount="511" Body="&lt;p&gt;Consider the PDE&#xA;\begin{equation}&#xA;\frac{d^2}{dx^2} g(x) = \frac{d}{dx} \delta(x-x_0),&#xA;\end{equation}&#xA;with $x, x_0 \in [0,1]$ and $g(0)=g(1)=0$. What is the best method to discretise the derivative of the delta function using finite differences?&lt;/p&gt;&#xA;" OwnerUserId="18560" LastEditorUserId="18560" LastEditDate="2016-04-20T13:09:11.763" LastActivityDate="2016-04-20T17:00:07.143" Title="Discretize Poisson equation with derivative of delta function as source" Tags="|finite-difference|poisson|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23736" PostTypeId="2" ParentId="23684" CreationDate="2016-04-19T18:49:39.883" Score="5" Body="&lt;p&gt;I'm not sure if this is the answer, but consider the case as $\alpha \rightarrow \infty$. In this case you're trying to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\iint_{\Omega} \cosh(P) dxdy = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And since cosh looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/7/76/Sinh_cosh_tanh.svg/504px-Sinh_cosh_tanh.svg.png&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that integrating it over any domain will never give you zero. If the diffusion of $P$ ($\nabla^2 P$) is not large enough to counteract this limit then you'll likely have problems.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-04-19T18:49:39.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23737" PostTypeId="1" AcceptedAnswerId="23740" CreationDate="2016-04-19T21:00:43.153" Score="4" ViewCount="630" Body="&lt;p&gt;My problem is &#xA;$$\text{minimize}: \phantom{2} f(x) \\&#xA;\text{subject to }: \phantom{2} x_4 \ge 0$$&#xA;where $x=(x_1,x_2,x_3,x_4)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the fourth component $x_4$ of the desired local minimizer is greater than zero, so the inequality constraint at the solution is inactive. However, there are other local minimizers that are infeasible. So, if one simply drops the bound constraint during the Newton iteration it might converge to an infeasible point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is:&lt;/strong&gt; What algorithm is the most effective one to solve this bound constraint problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried log-barrier method&#xA;$$\text{minimize}: \phantom{2} g(x)=f(x) - u \log(x_4)$$&#xA;with $u \rightarrow 0$. For example, $u_1=1, u_2=1\times 10^{-1},\cdots,u_7=1\times 10^{-6}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But at some $u_k$, say $u_3=1\times 10^{-3}$ the unconstrained minimization cannot converge, even though it converges at $u_1=1$ and $u_2=1 \times 10^{-1}$. I used line-search. I see that during the Newton iteration $g(x)$ is decreasing but the norm of the gradient $||\nabla g||$ doesn't. I cannot figure out the possible reason. Maybe ill-conditioning?&lt;/p&gt;&#xA;" OwnerUserId="8213" LastEditorUserId="9667" LastEditDate="2016-04-19T22:44:06.353" LastActivityDate="2017-07-05T17:08:01.283" Title="Simple bound constrained optimization problem" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23738" PostTypeId="5" CreationDate="2016-04-19T22:48:37.770" Score="0" Body="&lt;p&gt;&lt;strong&gt;From Wikipedia:&lt;/strong&gt; In numerical analysis, Newton's method (also known as the Newton–Raphson method), named after Isaac Newton and Joseph Raphson, is a method for finding successively better approximations to the roots (or zeroes) of a real-valued function.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2016-04-20T01:25:03.120" LastActivityDate="2016-04-20T01:25:03.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23739" PostTypeId="4" CreationDate="2016-04-19T22:48:37.770" Score="0" Body="Method for finding successively better approximations to the roots (or zeroes) of a real-valued function" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2016-04-20T01:24:56.603" LastActivityDate="2016-04-20T01:24:56.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23740" PostTypeId="2" ParentId="23737" CreationDate="2016-04-19T23:00:41.000" Score="5" Body="&lt;p&gt;There is really no particularly good answer that is generically true for any $f(x)$. My recommendation is to read the book by Nocedal &amp;amp; Wright on Numerical Optimization (one of my all-time favorite books) which covers the log-barrier method you use and many others. For example, you could consider active set methods, or the quadratic penalty method, etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-19T23:00:41.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23743" PostTypeId="1" AcceptedAnswerId="23750" CreationDate="2016-04-20T13:19:08.620" Score="2" ViewCount="187" Body="&lt;p&gt;This is a 16-bit PRNG function, transcribed from assembly to C for easier reading:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#define LOW(exp)  ((exp) &amp;amp; 0x00FF)&#xA;#define HIGH(exp) (((exp) &amp;amp; 0xFF00) &amp;gt;&amp;gt; 8)&#xA;&#xA;uint16_t prng(uint16_t v) {&#xA;&#xA;    uint16_t low  = LOW(v);&#xA;    uint16_t high = HIGH(v);&#xA;&#xA;    uint16_t mul_low  = low  * 5;&#xA;    uint16_t mul_high = high * 5;&#xA;&#xA;    // need to check for overflow, since final addition is adc as well&#xA;    uint16_t v1    = LOW(mul_high) + HIGH(mul_low) + 1;&#xA;    uint8_t  carry = HIGH(v1) ? 1 : 0;&#xA;&#xA;    uint16_t v2 = (LOW(v1) &amp;lt;&amp;lt; 8) + LOW(mul_low);&#xA;&#xA;    return (v2 + 0x11 + carry);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Original  transcription by @sagara, according to @EternisedDragon; minor edits by me. Assembly and some explanations available at &lt;a href=&quot;https://stackoverflow.com/questions/36745601/how-is-the-carry-flag-being-set-in-this-assembly-code&quot;&gt;https://stackoverflow.com/questions/36745601/how-is-the-carry-flag-being-set-in-this-assembly-code&lt;/a&gt;.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been trying to identify this PRNG, wanting to find out if it falls under a common classification. I've been going through &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_random_number_generators&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia's list of random number generators&lt;/a&gt;, like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_feedback_shift_register&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear feedback shift register algorithms&lt;/a&gt;, but all of these seem way more complex than the simple function above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does this function look familiar to anyone?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to research the properties of this PRNG but first wanted to see if there's any existing literature.&lt;/p&gt;&#xA;" OwnerUserId="20085" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-04-21T15:19:36.653" Title="What PRNG function is this?" Tags="|algorithms|c|random-number-generation|assembly|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23744" PostTypeId="1" AcceptedAnswerId="23749" CreationDate="2016-04-20T14:51:11.160" Score="0" ViewCount="126" Body="&lt;p&gt;Can any on help me understand how from the simple delta function on the right hand and performing the integration we can get values as in the left? Note that $h = x_i + 1 - x_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;diffusion&lt;/em&gt; and &lt;em&gt;mass&lt;/em&gt; matrices are given by&#xA;$$D_{ij} = \int_0^L \frac{d\phi_i}{dx}\frac{d\phi_j}{dx} dx =&#xA;\left\{\begin{matrix}&#xA;\frac{1}{h_1} &amp;amp;\text{if } i=j=1,\\&#xA;\frac{1}{h_{i-1}} + \frac{1}{h_{i}} &amp;amp;\text{if } i=j\neq 1 \text{ and } N_E +1,\\&#xA;\frac{1}{h_{N_E}} &amp;amp;\text{if } i=j=N_E+1,\\&#xA;-\frac{1}{h_{i}} &amp;amp;\text{if } j=i+1,\\&#xA;-\frac{1}{h_{i-1}} &amp;amp;\text{if } j=i-1,\\&#xA;0 &amp;amp;\text{otherwise.}&#xA;\end{matrix}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M_{ij} = \int_0^L \phi_i \phi_j dx =&#xA;\left\{\begin{matrix}&#xA;\frac{1}{3}h_1 &amp;amp;\text{if } i=j=1,\\&#xA;\frac{1}{3}(h_{i-1} + h_i) &amp;amp;\text{if } i=j\neq 1 \text{ and } N_E +1,\\&#xA;\frac{1}{3}h_{N_E} &amp;amp;\text{if } i=j=N_E+1,\\&#xA;\frac{1}{6}h_{i} &amp;amp;\text{if } j=i+1,\\&#xA;\frac{1}{6}h_{i-1} &amp;amp;\text{if } j=i-1,\\&#xA;0 &amp;amp;\text{otherwise.}&#xA;\end{matrix}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the shape functions satisfy the condition $\phi_i(x_j) = \delta_{i,j}$ is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kronecker_delta&quot; rel=&quot;nofollow&quot;&gt;Kronecker delta&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pozrikidis, Constantine. Introduction to finite and spectral element methods using MATLAB. CRC Press, 2005.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="9667" LastEditDate="2016-04-20T18:43:14.427" LastActivityDate="2016-04-20T18:43:14.427" Title="Derivation in the FEM method" Tags="|finite-element|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23745" PostTypeId="1" AcceptedAnswerId="23752" CreationDate="2016-04-20T15:06:09.737" Score="2" ViewCount="5458" Body="&lt;p&gt;I am a bit confused with nodal basis functions $h_i$ and the corresponding lagrange polynomials $l_i$. For linear functions, it's quite clear, on $[x_0,x_1]$ the nodal basis is $h_0 = l_0 = 1-x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But with quadratic basis functions, there are $l_0^1 = 1-x^2$ and $l_0^2=\frac{(x-1)(x-2)}{2}$ associated with the node and the interval $[x_0,x_1]$. Further, the function $l_{-1}^3 = \frac{(x-0)(x-1)}{2}$ still has support from the left side as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://hplgit.github.io/INF5620/doc/pub/sphinx-fem/._main_fem003.html#construction-principles&quot; rel=&quot;nofollow noreferrer&quot;&gt;this graphic&lt;/a&gt; this seems to be resolved by taking an interval $[x_0,x_2]$ for quadratic elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently trying to build some matrix $\langle h_i,h_j\rangle = \int h_i h_j$ and try to figure out, which functions need to be integrated together on an interval $[x_i, x_{i+1}]$. My current example is just a periodic 1D-Interval with edges $e_i = [x_i, x_{i+1}]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now i am confused, if i want to use the $h_i$ functions for this, or $l_i$ functions, and if i need to have distinctions if i am using the corresponding function to the left or right side or how i match the overlapping intervals when using functions, which have a support bigger than one interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how my functions are overlapping on $[0,1]$ (so for example $1-x^2$ has support on two elements). &#xA;&lt;a href=&quot;https://i.stack.imgur.com/hnxIC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hnxIC.png&quot; alt=&quot;centered functions&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2cEPs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2cEPs.png&quot; alt=&quot;left sided functions&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/oRcvY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oRcvY.png&quot; alt=&quot;right sided functions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there are six functions on each interval and three on each node, just as the graphic linked above. But there the interval $[0,2]$ is used, so that the $1-x^2$ function is $0$ outside with some inner node not belonging to other elements, while i have only nodes on the limits of the elements, so the function has support on two elements as well.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="14700" LastEditDate="2016-04-21T09:00:37.257" LastActivityDate="2016-04-22T11:55:52.193" Title="Nodal basis functions and lagrange polynomials" Tags="|finite-element|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23746" PostTypeId="1" AcceptedAnswerId="23747" CreationDate="2016-04-20T15:14:40.960" Score="1" ViewCount="88" Body="&lt;p&gt;I am learning how to use pseudo random number generators but the instructor just told us how they work without explaining why they work.&#xA;For example, can one prove that the numbers generated by LCG methods are really uniform deviates in [0,1)? Experiments show that it looks like yes. But can we prove this mathematically? I am always puzzled by the basic philosophy behind pseudo random numbers. If the algorithm is deterministic, how can the outcome be random?&lt;/p&gt;&#xA;" OwnerUserId="14407" LastEditorUserId="20688" LastEditDate="2019-06-22T22:28:30.933" LastActivityDate="2019-06-22T22:28:30.933" Title="Pseudo random numbers" Tags="|reference-request|random-number-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23747" PostTypeId="2" ParentId="23746" CreationDate="2016-04-20T16:46:52.030" Score="1" Body="&lt;p&gt;Psudorandom numbers generated via methods like LCG are generated via a discrete dynamical system (i.e. a system where $x_n = f(x_{n-1})$). For these types of systems, tools such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Ergodic_theory&quot; rel=&quot;nofollow&quot;&gt;Ergodic Theory&lt;/a&gt; can be applied to learn the the long-term distribution of such processes. This is one way to get the long-run probability distribution which generalizes well and can thus be used to develop better methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These psudorandom numbers are taken to be cycles in the dynamical system which have a long period length and which have an asymptotically uniform distribution. For example, you can come up with a system where $x_n = x_1$  for every $n=5$. This would mean that your solution is a 5-cycle. This wouldn't be very good because then it would repeat a lot. So psudorandom number generators are made to have very long periods. &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/randstream.list.html&quot; rel=&quot;nofollow&quot;&gt;Mathworks has a good table which shows the periods of common random number generators&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a long period with the points distributed almost uniformly, this matches what you'd intuitively believe is psudo-random uniform.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-04-20T16:46:52.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23748" PostTypeId="2" ParentId="23735" CreationDate="2016-04-20T17:00:07.143" Score="4" Body="&lt;p&gt;As noted by others, this PDE is not well-posed in the Strong Form. However, you can take some function which asymtopically approximates the Dirac Delta and use that (where the error in the approximation is below truncation error). For example, you can use&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \eta_\epsilon = \epsilon^{-n} \eta(x/\epsilon) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \eta(x) = \exp(-1/(1-|x|^2)) \chi_{|x|&amp;lt;1} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $n$ is the dimension.You can easily take the derivative of this and choose a small enough $\epsilon$ such that this is close enough to the Dirac delta.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-04-20T17:00:07.143" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23749" PostTypeId="2" ParentId="23744" CreationDate="2016-04-20T17:57:46.297" Score="4" Body="&lt;p&gt;The basis functions seem to be piecewise linear with $\phi_i(x_j) = \delta_{ij}$, so&#xA;$$ \phi_i(x) = &#xA;\begin{cases}&#xA;\frac{1}{h_{i-1}}(x-x_{i-1}) \qquad &amp;amp;x\in[x_{i-1},x_i]\,,\\&#xA;\frac{-1}{h_{i}}(x-x_{i+1}) \qquad &amp;amp;x\in[x_{i},x_{i+1}]\,,\\&#xA;0 &amp;amp; \text{otherwise}\,,&#xA;\end{cases}&#xA;$$&#xA;and as such&#xA;$$ \frac{\mathrm{d}\phi_i}{\mathrm{d}x}(x) = &#xA;\begin{cases}&#xA;\frac{1}{h_{i-1}} \qquad &amp;amp;x\in[x_{i-1},x_i]\,,\\&#xA;\frac{-1}{h_{i}} \qquad &amp;amp;x\in[x_{i},x_{i+1}]\,,\\&#xA;0 &amp;amp; \text{otherwise}\,.&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example to get you started, the first integral, we have to show&#xA;$$&#xA;  \int_0^L \frac{\mathrm{d}\phi_i}{\mathrm{d}x}&#xA;           \frac{\mathrm{d}\phi_j}{\mathrm{d}x} \mathrm{d}x =&#xA;  \frac{1}{h_{i-1}} + \frac{1}{h_{i}}&#xA;  \quad \text{when } i=j\neq 1 \text{ and } N_E+1\,.&#xA;$$&#xA;With $i=j$ and with the above definition&#xA;$$&#xA;  \begin{split}&#xA;    \int_0^L \left(\frac{\mathrm{d}\phi_i}{\mathrm{d}x}\right)^2 \mathrm{d}x&#xA;      &amp;amp;= 0 + \int_{x_{i-1}}^{x_i} \frac1{h_{i-1}^2}\mathrm{d}x +&#xA;         \int_{x_{i}}^{x_{i+1}} \frac1{h_{i}^2}\mathrm{d}x + 0 \\&#xA;      &amp;amp;= \left[\frac{x}{{h_{i-1}}^2}\right]_{x_{i-1}}^{x_i}&#xA;         + \left[\frac{x}{{h_{i}}^2}\right]_{x_{i}}^{x_{i+1}} \\&#xA;      &amp;amp;= \frac{1}{h_{i-1}} + \frac{1}{h_{i-1}}\,.&#xA;  \end{split}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="17671" LastActivityDate="2016-04-20T17:57:46.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23750" PostTypeId="2" ParentId="23743" CreationDate="2016-04-20T18:39:18.357" Score="8" Body="&lt;p&gt;A little playing with the sequence of numbers generated by the C code shows that the sequence is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z_{i+1}=5z_{i}+273 \mod 2^{16}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a linear congruential generator (LCG).  It's easy to show that this LCG has full period (See theorem 7.1 in Law's &lt;em&gt;Simulation Modeling and Analysis, 5th ed.&lt;/em&gt; and check the three conditions.)   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find the generator in any of the references that I checked, but that doesn't mean it hasn't been published somewhere.  It will suffer from all of the faults of other LCG's (in particular it will do badly on tests of uniformity in higher dimensions.)   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-04-20T18:39:18.357" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="23751" PostTypeId="2" ParentId="23745" CreationDate="2016-04-20T18:58:46.870" Score="1" Body="&lt;p&gt;I don't completely understand your question. The shape functions for a quadratic element are plotted below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cbXaA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cbXaA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the dimension of the space is 3, one needs 3 functions in the basis. They are well defined since they must satisfy the conditions $\phi_i(x_j) = \delta_{ij}$. This gives 3 conditions, which completely determines a quadratic polynomial.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="549" LastEditDate="2016-04-21T01:34:54.043" LastActivityDate="2016-04-21T01:34:54.043" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23752" PostTypeId="2" ParentId="23745" CreationDate="2016-04-21T11:35:49.567" Score="3" Body="&lt;p&gt;By using basis functions, you are stating how a variable changes &lt;em&gt;between&lt;/em&gt; nodes by knowing only information &lt;em&gt;at&lt;/em&gt; the nodes. These nodes are used to 'fit' a set of polynomial basis functions, and then the field variable you are interested in is then a summation of each basis function with the corresponding value of your field variable at each node. Mathematically, you would write that as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \phi = \sum_{i=1}^N H_i\hat{\phi}_i, \forall i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N$ is the number of basis functions, and $H$ is the basis function itself, and $\hat{\phi}$ is the value of your field variable at the nodes. This is equivalent to the dot product of the vector of basis functions and the vector of nodal values, respectively:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \phi = \underline{H} \cdot \underline{\hat{\phi}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every set of basis functions, the required number of points is dependent on the order of the basis function you want to use. For a set of linear basis functions (order = 1), you need 2 points. For quadratic (order = 2), you need 3 points, and so forth. Basically, you need the lowest number of points to fully define the polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quadratic basis function has the form:&#xA;$$ H_i = \alpha_i + \beta_i x + \gamma_i x^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is quadratic, you need three points. Let's say we have $x = [x_1, x_2, x_3]$, then you could set up the following three equations for your first basis function to determine the polynomial coefficients, where it equals 1 at $x_1$ and 0 at the other nodes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{r c  l}&#xA;H_1 = \alpha_1 + \beta_1 x_1 + \gamma_1 x_1^2 = 1 \\&#xA;H_1 = \alpha_1 + \beta_1 x_2 + \gamma_1 x_2^2 = 0 \\&#xA;H_1 = \alpha_1 + \beta_1 x_3 + \gamma_1 x_3^2 = 0 \\&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: It's important to state that at a basis node $x_i$, only the corresponding basis function $H_i$ has a non-zero value of 1, all the others are 0. In other words $H_i(x_j) = \delta_{ij}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could solve this anyway you like, but I prefer putting it all together and solving for the coefficients using linear algebra:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{matrix}&#xA;1 &amp;amp; x_1 &amp;amp; x_1^2 \\&#xA;1 &amp;amp; x_2 &amp;amp; x_2^2 \\&#xA;1 &amp;amp; x_3 &amp;amp; x_3^2 \\&#xA;\end{matrix} \right] \left[\begin{matrix}&#xA;\alpha_1 &amp;amp; \alpha_2 &amp;amp; \alpha_3 \\&#xA;\beta_1 &amp;amp; \beta_2 &amp;amp; \beta_3 \\&#xA;\gamma_1 &amp;amp; \gamma_2 &amp;amp; \gamma_3 \\&#xA;\end{matrix}\right] = \left[\begin{matrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 1 \\&#xA;\end{matrix}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plugging in your solution for $\alpha_i, \beta_i, $ and $\gamma_i, \forall i$ back into the basis equations gives you your set of fitting polynomials, which combine together using our original equation of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \phi = \sum_{i=1}^N H_i\hat{\phi}_i, \forall i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used a sample set of values (y = [2 3 1.5]) at x = [-1, 0, 1], and this was the end result. You see how a quadratic function fits through the three original points, and that the three basis functions themselves are the 'weights' of the sample points showing how the field variable changes between the basis nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XDmG7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XDmG7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="20098" LastEditDate="2016-04-21T19:31:58.890" LastActivityDate="2016-04-21T19:31:58.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23753" PostTypeId="2" ParentId="23714" CreationDate="2016-04-21T14:21:32.553" Score="2" Body="&lt;p&gt;Because the polynomial $ax^2+bx+c$ and $x^2+\frac{b}{a}x + \frac{c}{a}$ have the exact same roots, let's first simplify the problem by only considering the roots of the equation&#xA;$$&#xA;  x^2+b'x + c' = 0&#xA;$$&#xA;where you can think of $a'=\frac{b}{a}, c'=\frac{c}{a}$. This shows that really only two parameters matter. You could then, for example, plot the (real parts of) the two roots in the $z$-direction of a 3d plot, and let the $x$- and $y$-axes be $a'$ and $b'$. Or, if you're interested in the discriminant, you may choose the $x$-axis as $b'^2-4c'$ and the $y$-axis as $b'$, $c'$, or any other quantity independent of the discriminant. You will then get two surfaces for the two roots wherever they are real.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the following plot shows the two roots for all values where the discriminant is non-negative:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sgftx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sgftx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along the parabola, the discriminant is zero, and the two roots coincide. You could easily create a similar plot for other choices of the axes as functions of $b',c'$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-21T14:21:32.553" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23754" PostTypeId="2" ParentId="23720" CreationDate="2016-04-21T14:33:21.653" Score="3" Body="&lt;p&gt;You look at the question the wrong way. First, let us consider the &lt;em&gt;exact&lt;/em&gt; problem and its solution:&#xA;$$&#xA;  -\psi''(x) + x^2 \psi(x) = E\psi(x),&#xA;$$&#xA;with boundary values $\psi(\pm\infty)=0$. Now you want to consider truncating your domain somewhere, i.e., you want to consider a &lt;em&gt;different&lt;/em&gt; problem that reads&#xA;$$&#xA;  -\tilde\psi''(x) + x^2 \tilde\psi(x) = E\tilde\psi(x),&#xA;$$&#xA;with boundary values $\tilde\psi(\pm x_\infty)=0$ where $x_\infty$ is a &lt;em&gt;finite&lt;/em&gt; termination point of your domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, these two problems will have different answers, $\psi$ and $\tilde\psi$. The question you are asking is, in essence, how large you have to choose $x_\infty$. But this cannot be answered without also saying how small you want the &lt;em&gt;difference&lt;/em&gt; $\psi-\tilde\psi$ to be! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you choose $x_\infty$ small, then you can solve the problem of course, but $\psi-\tilde\psi$ will be large. On the other hand, if you want the difference to be small, you will likely have to choose $x_\infty$ to be large. In general, if you require that $\|\psi-\tilde\psi\|\le\varepsilon$, then there will be some value $\bar x(\varepsilon)$ so that any $x_\infty\ge \bar x(\varepsilon)$ will be a valid choice of the domain size.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-04-26T13:32:31.123" LastActivityDate="2016-04-26T13:32:31.123" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23756" PostTypeId="1" AcceptedAnswerId="23759" CreationDate="2016-04-21T19:58:31.133" Score="1" ViewCount="91" Body="&lt;p&gt;I want to run a simulation which involves rates between different states. Each state is label by a pair of indices $(m,n)$, so that a certain rate $R_{(m,n)\rightarrow(m',n')}$ requires four indices in total.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I try to do is to convert these double indices into a single index $j$. This would allow me to use standard matrix multiplication methods to calculate e.g. the product of two matrices&#xA;$$A_{j_1,j_2} = (B  C)_{j_1,j_2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do see, how to calculate the 1D index, by defining&#xA;$$j(n,m) = \sum_{n'=0..n-1}M(n) + m$$&#xA;Here $M(n)$ is the number of different $m$'s I have for a given $n$. This is the crucial part, the number of different $m$'s is different for each $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, i do not see how to get $m$ and $n$ back, in case $j$ is given and for a generic $M(n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a standard problem in computer science?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what strategy can I use to do matrix multiplications when I have double indices?&lt;/p&gt;&#xA;" OwnerUserId="8329" LastEditorUserId="8329" LastEditDate="2016-04-21T20:19:30.087" LastActivityDate="2016-04-21T20:46:34.310" Title="Use double index in matrix multiplication" Tags="|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23757" PostTypeId="1" CreationDate="2016-04-21T19:59:52.093" Score="0" ViewCount="66" Body="&lt;p&gt;A factory has $A$ and $B$ products. $A$ is made with $4X + 2Y$ raw materials. $B$ is made $2X + 4Y$ raw materials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to maximize total profit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;amount of profit $A$ per item, amount of profit $B$ per item and&lt;/li&gt;&#xA;&lt;li&gt;number of $X$ and number of $Y$ raw materials.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of A which will be produced,&lt;/li&gt;&#xA;&lt;li&gt;number of B which will be produced,&lt;/li&gt;&#xA;&lt;li&gt;total profit.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Input:&#xA;profitA = $8, profitB = $6,&#xA;numberOfX = 600, numberOfY = 480,&#xA;Output:&#xA;numberOfA = 120,&#xA;numberOfB = 60,&#xA;totalprofit = $1320.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My solution is brute force algorithm. I find maximum number of $A$ will can produce and I decrease it one by one then compare result and get maximum profit. But this is not efficient. Is there a algorithm that solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Formula = mA*pA + mB*pB =&amp;gt; maximum&#xA;pA: profit A, pB: profit B,&#xA;mA: number of product A, mB: number of product B,&#xA;Producing mA number A are required 4*mA number X and 2*mB number Y,&#xA;producing mB number B are required 2*mB number X and 4*mB number Y.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20099" LastEditorUserId="9667" LastEditDate="2016-04-21T20:52:27.363" LastActivityDate="2016-04-22T23:30:08.167" Title="Maximize Result For 4 variable" Tags="|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23759" PostTypeId="2" ParentId="23756" CreationDate="2016-04-21T20:46:34.310" Score="2" Body="&lt;p&gt;Won't this work? You create a map from $(m,n)$, the key-pair, to $j$, the value. For this you could use &lt;code&gt;std::map&lt;/code&gt;. The reverse map to go from $j$ to $(m,n)$ is just a vector of pairs. Then you can use $j$-like index to perform the matrix operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I simplifying your problem too much?&lt;/p&gt;&#xA;" OwnerUserId="20098" LastActivityDate="2016-04-21T20:46:34.310" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23760" PostTypeId="2" ParentId="23757" CreationDate="2016-04-21T22:15:58.727" Score="1" Body="&lt;p&gt;Express the objective function for profit: &quot;maximize $P_o = n_A p_A + n_B p_B$&quot; in terms of $n_A$ (or $n_B$). If we produce $n_A$ items then from the constraints we have:&#xA;$$n_B = \min (\frac{n_X - 4 n_A}{2}, \frac{n_Y - 2 n_A}{4}).$$&#xA;Thus we have a max-min problem. I believe this can be solved using auxiliary constraint technique like-so:&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/10792139/using-min-max-within-an-integer-linear-program&quot;&gt;https://stackoverflow.com/questions/10792139/using-min-max-within-an-integer-linear-program&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20098" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-04-22T23:30:08.167" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23761" PostTypeId="2" ParentId="23725" CreationDate="2016-04-21T23:34:07.927" Score="0" Body="&lt;p&gt;I am not sure I understand your question. This is how I understand second variation (skipping much of the details). Imagine you have a simple linear spring whose stored energy is $W = {1\over2} k \thinspace u^2$ for spring stiffness $k$ and displacement $u$. Then the first variation gives $\delta W = \delta u \thinspace k \thinspace u$, and the second variation gives $\text{d}\delta W = \delta u \thinspace k \thinspace \text{d}u.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the second variation can be expressed as $\text{d}u = N_i \text{d}u_i$ depending on the chosen discretization and shape functions. Thus the quadratic nature of energy expression lends itself to computing second variation.&lt;/p&gt;&#xA;" OwnerUserId="20098" LastActivityDate="2016-04-21T23:34:07.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23762" PostTypeId="1" CreationDate="2016-04-22T05:13:09.843" Score="2" ViewCount="543" Body="&lt;p&gt;I wish to solve the &lt;strong&gt;two-dimensional Navier Stokes equations using multigrid&lt;/strong&gt; method on a collocated grid using the Predictor-Corrector method mentioned below. But first, let me elaborate on what I had already did. &lt;strong&gt;Previously, I used a staggered grid and a artificial compressibility based solver.&lt;/strong&gt; So my governing equations were:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jlDPH.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jlDPH.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since my solver is explicit (the time term here is just to do the iterations). To implement multigrid in this kind of methodology, I defined the defect/residual on fine grid as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cMebS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cMebS.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This defect/residual and also the solution on the fine grid is then restricted to the coarse grid (Full Storage Approximation Scheme) using simple injection operator. &lt;strong&gt;&lt;em&gt;Since the equations on the coarse grid remain almost the same in appearance, except that their RHS is not zero and instead is defined as&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eAHG4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eAHG4.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;where the dashed terms are &lt;em&gt;m&lt;/em&gt; and not the corrections itself.&lt;/strong&gt;&#xA;So, after I solve the coarse mesh equations to find the correction, I interpolate them to fine mesh to correct the fine mesh solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method works just fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Now, I want to implement multigrid on a colocated grid with the following predictor-corrector based discretization:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DptHU.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DptHU.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the three steps are predictor, corrector and update steps respectively. &lt;strong&gt;Predictor is essentially a forward difference based on the values at old time level while corrector uses a backward difference using the values at predictor time level.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How will I define the defect/residual on the fine mesh for this scheme? Any help will be appreciated. And if there are any problems in my existing approach, please let me know.&lt;/p&gt;&#xA;" OwnerUserId="16397" LastActivityDate="2017-01-21T23:43:27.777" Title="How to define residual in multigrid approach?" Tags="|finite-difference|fluid-dynamics|discretization|navier-stokes|multigrid|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23763" PostTypeId="1" CreationDate="2016-04-22T07:38:51.460" Score="6" ViewCount="455" Body="&lt;p&gt;I'm using a lot of linear solvers and preconditioners, but mostly, they are magical black boxes to me. Since I'll also have to implement some of them in future, I would like to learn a bit more, exactly how they work an which one is &quot;good&quot; for what.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I'm quite fit in the basics (linear algebra, numerics), I do not need a book that covers the full theory. I'm more looking for an extensive, well-understandable description of the various algorithms. Something like a &quot;cook book&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any recommendations?&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-04-23T17:08:52.100" Title="&quot;Cookbook&quot; about iterative linear solvers and preconditioners" Tags="|linear-solver|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23764" PostTypeId="2" ParentId="23763" CreationDate="2016-04-22T07:52:49.913" Score="15" Body="&lt;p&gt;Have a look at &lt;em&gt;Templates for the Solution of Linear Systems:&#xA;Building Blocks for Iterative Methods&lt;/em&gt; (Barrett et al.). You can find it &lt;a href=&quot;http://www.netlib.org/templates/templates.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's why I'm recommending this over other references:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the &quot;flowchart of iterative methods&quot; in appendix D (last page)&lt;/li&gt;&#xA;&lt;li&gt;covers both linear solvers and preconditioners,&lt;/li&gt;&#xA;&lt;li&gt;it is short (100 pages or so),&lt;/li&gt;&#xA;&lt;li&gt;does not go into too much detail,&lt;/li&gt;&#xA;&lt;li&gt;has a list of references if you're curious to find out more about a particular solver/preconditioner,&lt;/li&gt;&#xA;&lt;li&gt;the writing style is very clear and the authors are well-known experts in the field,&lt;/li&gt;&#xA;&lt;li&gt;it is free to download and print.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Caveat: this book was written in 1994 so it may not describe some of the more recent developments...&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2016-04-23T17:08:52.100" LastActivityDate="2016-04-23T17:08:52.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23765" PostTypeId="1" AcceptedAnswerId="23843" CreationDate="2016-04-22T08:54:45.150" Score="0" ViewCount="104" Body="&lt;p&gt;I have a model given by a system of differential equations &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{dy}{dt}=f(y)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $y=(y_1,y_2,y_3)$  and $f:\mathbb{R}^3 \to \mathbb{R}^3$. The system works as follows : integrate the ode's with initial values $(y_1^0,y_2^0,y_3^0)$ until $y_1$ reaches a certain value (say $y_1=0$), get the time $t_f$ when this happens and start again with the initial values  $(y_1^0 ,y_2^0, y_3(t_f))$. All this until some stopping condition is encountered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : what software can be used to simulate such a system? It would be preferable to be open source, but any suggestion is welcomed.&lt;/p&gt;&#xA;" OwnerUserId="1302" LastEditorUserId="123" LastEditDate="2016-05-03T07:20:29.533" LastActivityDate="2016-05-03T07:20:29.533" Title="Integrating a dynamical system until an algebraic condition is satisfied" Tags="|ode|software|numerics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23767" PostTypeId="1" CreationDate="2016-04-22T10:08:24.480" Score="1" ViewCount="299" Body="&lt;p&gt;I am trying to solve following set of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A(i,i-2)*u(i-2) + A(i,i-1)*u(i-1) + &lt;strong&gt;(A(i,i)+β(i) )*u(i)&lt;/strong&gt; + A(i,i+1)*u(i+1) + A(i,i+2)*u(i+2)= &lt;strong&gt;B(i) + β(i)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where i=1:1000000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If values of &lt;strong&gt;β varies(several orders)&lt;/strong&gt; greatly then what would be the best strategy to solve this equation numerically. Should I expect oscillations in the result?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the possible remedy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;*Is it okay if I add &quot;10000*u(i)&quot; on both sides of equations while solving iteratively?*&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;Structure of Coeffcient Matrix will roughly look like this (if i=1:9, while I am trying to do it for i=1:10^6): &#xA;$$&#xA;\begin{bmatrix}&#xA; A11 + β1&amp;amp;  A12&amp;amp;  A13&amp;amp;  0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp; \\ &#xA; A21&amp;amp;  A22 + β2&amp;amp;  A23&amp;amp;  A24&amp;amp;  0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp; \\ &#xA; A31&amp;amp;  A32&amp;amp;  A33 + β3&amp;amp;  A34&amp;amp;  A35&amp;amp;  0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp; \\ &#xA; 0&amp;amp;    A42&amp;amp;  A43&amp;amp;  A44 + β4&amp;amp;  A45&amp;amp;  A46&amp;amp;  0&amp;amp;    0&amp;amp;    0&amp;amp; \\ &#xA; 0&amp;amp;    0&amp;amp;    A53&amp;amp;  A54&amp;amp;  A55 + β5&amp;amp;  A56&amp;amp;  A57&amp;amp;  0&amp;amp;    0&amp;amp; \\ &#xA; 0&amp;amp;    0&amp;amp;    0&amp;amp;    A64&amp;amp;  A65&amp;amp;  A66 + β6&amp;amp;  A67&amp;amp;  A68&amp;amp;  0&amp;amp;\\ &#xA; 0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;    A75&amp;amp;  A76&amp;amp;  A77 + β7&amp;amp;  A78&amp;amp;  A79&amp;amp; \\ &#xA; 0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;  A86&amp;amp;  A87&amp;amp;  A88 + β8&amp;amp;  A89&amp;amp; \\ &#xA; 0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;    0&amp;amp;  A97&amp;amp;  A98&amp;amp;  A99 + β9&amp;amp; \\ &#xA; \end{bmatrix}&#xA;$$&#xA; Here A(i,i)=A(i,i-2)+A(i,i-1)+A(i,i+1)+A(i,i+2)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  **β(i,i)&amp;gt;0&#xA;  β(i,i) &amp;gt;&amp;gt; A(i,i-2)+A(i,i-1)+A(i,i+1)+A(i,i+2)**&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also there is a huge variation in values of β.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;**Order of every entry in matrix is ~10^-2, except diagonal entries and β which vary from order of 1 to 10^6.&lt;/p&gt;&#xA;" OwnerUserId="18157" LastEditorUserId="4405" LastEditDate="2016-08-24T16:11:41.313" LastActivityDate="2017-07-20T22:04:17.563" Title="Solving a linear system whose matrix has imbalanced diagonal entries" Tags="|linear-algebra|computational-physics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23769" PostTypeId="2" ParentId="23745" CreationDate="2016-04-22T11:55:52.193" Score="1" Body="&lt;p&gt;My first answer focused on how to define the basis functions themselves; however, that didn't answer your question. Here I will describe how that relates to the mass/stiffness matrices.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Background&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I'm going to explain how to build a mass matrix based on a continuous Galerkin approach to the finite element method using the 1D diffusion problem as an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla^2 \phi - f = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick step back into the FEM formulation, we want to convert our problem into a finite element problem. We start by making the assumption that we can represent the true solution by only using a finite number of nodes. We'll call the true solution $ \phi$ and the finite element solution $\hat{\phi}$. Our solution probably won't be perfect because of the approximation, but it'll be good enough. Another way of saying that is by calling it a residual:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{R}\left( \hat{\phi} \right) = \nabla^2\hat{\phi} -f \cong 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're now going to multiply our residual function with our finite element test functions, integrate that over the domain, and set the integral to zero (excellent analogy for why &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16331/motivation-behind-galerkin-method&quot;&gt;here&lt;/a&gt;). We end up with this statement:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \underline{N} \mathcal{R} \left( \hat{\phi} \right) \partial \Omega = \int_\Omega \underline{N} \left( \nabla^2\hat{\phi} -f \right) \partial \Omega =  0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega \underline{N} \left( \nabla^2\hat{\phi} \right) \partial \Omega =  \int_\Omega \underline{N} \left( f \right) \partial \Omega $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\underline{N}$ is a vector of basis functions (length = number of nodes) used to discretize the system. $\hat{\phi}$ is the approximate solution to our problem, which is interpolated by a polynomial within each element. For the linear case, traversing along the element $s$ = [-1 1]:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \hat{\phi}(s) = N_1(s)\tilde{\phi}_1 + N_2(s)\tilde{\phi}_2 = \underline{N}^T \underline{\tilde{\phi}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, for the quadratic case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \hat{\phi}(s) = N_1(s)\tilde{\phi}_1 + N_2(s)\tilde{\phi}_2 + N_3(s)\tilde{\phi}_3 = \underline{N}^T \underline{\tilde{\phi}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're going to discretize the domain using 3 linear elements ($l_{1-3}$), which results in 4 nodes ($\tilde{\phi}_{1-4}$). Let's also assume we have Dirichlet boundaries at the ends $(\tilde{\phi}_{1} = \tilde{\phi}_{L} ~,~ \tilde{\phi}_{4} = \tilde{\phi}_{R})$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/oiQ3B.png&quot; height=&quot;100&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It's common to construct the mass matrix in a block pattern, where we first create an elemental mass matrix corresponding to each element, and then fill it into the global mass matrix. Therefore, we are going to isolate our original equation to the first element:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega_1} \underline{N} \left( \nabla^2\hat{\phi} \right) \partial \Omega_1 =  \int_{\Omega_1} \underline{N} \left( f \right) \partial \Omega_1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And we are going to replace $\hat{\phi}$ with $\underline{N}^T \underline{\tilde{\phi}}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega_1} \underline{N} \left( \nabla^2 (\underline{N}^T \underline{\tilde{\phi}}) \right) \partial \Omega_1 =  \int_{\Omega_1} \underline{N} \left( f \right) \partial \Omega_1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm not going to go into these steps, but needless to say we can use integration by parts to convert the second derivative into a first derivative, after which you end up with this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega_1} \frac{\partial\underline{N}}{\partial x}  \frac{\partial\underline{N}^T}{\partial x} \underline{\tilde{\phi}} \partial \Omega_1 =  \int_{\Omega_1} \underline{N} \left( f \right) \partial \Omega_1 $$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;The answer to your question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Now I think you're asking, how do you turn this integral into an elemental matrix, which you are later going to add to the global matrix. And specifically, what is the influence of nodes using basis functions of higher order than 1? Well, going back to the linear element case, there are two nodes per element. That means the size of $\underline{N}$ is $(2,1)$. The resulting elemental block matrix is then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{matrix}&#xA;\int_{\Omega_1} \frac{\partial N_1}{\partial x}  \frac{\partial N_1}{\partial x} \partial \Omega_1 &amp;amp; \int_{\Omega_1} \frac{\partial N_1}{\partial x}  \frac{\partial N_2}{\partial x} \partial \Omega_1 \\&#xA;\int_{\Omega_1} \frac{\partial N_2}{\partial x}  \frac{\partial N_1}{\partial x} \partial \Omega_1 &amp;amp; \int_{\Omega_1} \frac{\partial N_2}{\partial x}  \frac{\partial N_2}{\partial x} \partial \Omega_1 \\&#xA;\end{matrix} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a quadratic element, there are 3 nodes. The resulting elemental block matrix has to reflect that, and is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{matrix}&#xA;\int_{\Omega_1} \frac{\partial N_1}{\partial x}  \frac{\partial N_1}{\partial x} \partial \Omega_1 &amp;amp; \int_{\Omega_1} \frac{\partial N_1}{\partial x}  \frac{\partial N_2}{\partial x} \partial \Omega_1 &amp;amp; \int_{\Omega_1} \frac{\partial N_1}{\partial x}  \frac{\partial N_3}{\partial x} \partial \Omega_1 \\&#xA;\int_{\Omega_1} \frac{\partial N_2}{\partial x}  \frac{\partial N_1}{\partial x} \partial \Omega_1 &amp;amp; \int_{\Omega_1} \frac{\partial N_2}{\partial x}  \frac{\partial N_2}{\partial x} \partial \Omega_1 &amp;amp; \int_{\Omega_1} \frac{\partial N_2}{\partial x}  \frac{\partial N_3}{\partial x} \partial \Omega_1 \\&#xA;\int_{\Omega_1} \frac{\partial N_3}{\partial x}  \frac{\partial N_1}{\partial x} \partial \Omega_1 &amp;amp; \int_{\Omega_1} \frac{\partial N_3}{\partial x}  \frac{\partial N_2}{\partial x} \partial \Omega_1 &amp;amp; \int_{\Omega_1} \frac{\partial N_3}{\partial x}  \frac{\partial N_3}{\partial x} \partial \Omega_1 \\&#xA;\end{matrix} \right]&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-04-22T11:55:52.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23771" PostTypeId="2" ParentId="8930" CreationDate="2016-04-22T14:25:13.033" Score="1" Body="&lt;p&gt;A more recent publication has come out with a new method for solving the 3x3 polar decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://eprints.ma.man.ac.uk/2352/01/covered/MIMS_ep2015_66.pdf&quot; rel=&quot;nofollow&quot;&gt;An algorithm to compute the polar decomposition of a 3x3 matrix&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm really surprised OP was needing 15 iterations for the iterative method!)&lt;/p&gt;&#xA;" OwnerUserId="7745" LastActivityDate="2016-04-22T14:25:13.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23772" PostTypeId="2" ParentId="23767" CreationDate="2016-04-22T15:48:31.593" Score="1" Body="&lt;p&gt;Answer depends on my assumptions about your question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a typo in your expression for the $i^{\text{th}}$ equation's diagonal term, which you say is $A_{i,i} + \beta_i$ at the top of the page. Then if I try using this with your expression for $A_{i,i}$ below the matrix, then I get $1+2\beta_i$ in the diagonal. &lt;em&gt;Assuming that this is indeed a typo&lt;/em&gt;, then ...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is $\beta_i &amp;gt;&amp;gt; B_i$ also? If so, given that $\beta_i &amp;gt;&amp;gt; \sum_{k=i-2,k\ne i}^{i+2} A_{i,k}$, your solution vector $u_i$ is centered around $\{1,1,1,...\}$. This is good news, because then this could be your initial guess for any iterative scheme.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can also use direct solver with 1M equations, and should be able to get a reasonable performance with recent laptops and desktops.&lt;/p&gt;&#xA;" OwnerUserId="20098" LastEditorUserId="20098" LastEditDate="2016-04-22T15:56:23.057" LastActivityDate="2016-04-22T15:56:23.057" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23774" PostTypeId="2" ParentId="23765" CreationDate="2016-04-22T16:02:11.993" Score="0" Body="&lt;p&gt;You can also try SUNDIAL suite from LLNL. See &lt;a href=&quot;https://computation.llnl.gov/casc/sundials/main.html&quot; rel=&quot;nofollow&quot;&gt;https://computation.llnl.gov/casc/sundials/main.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20098" LastActivityDate="2016-04-22T16:02:11.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23775" PostTypeId="2" ParentId="22071" CreationDate="2016-04-22T16:42:31.223" Score="-1" Body="&lt;p&gt;Whatever the literature available for flow past a cylinder, I think it is right. If you print the results in between your simulation, you will see the vortex shedding which is not exactly the same as with $\text{Re}=60$ but it is quite fluctuating and then along with increasing $\text{Re}$, its frequency increases and that is the reason to have vortex appearance at $\text{Re}&amp;gt;50$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to see the difference in vortex shedding, just output that into the file after every 5000 or 10000 iterations&lt;/p&gt;&#xA;" OwnerUserId="20118" LastEditorUserId="20688" LastEditDate="2018-09-11T03:15:58.033" LastActivityDate="2018-09-11T03:15:58.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="23777" PostTypeId="1" CreationDate="2016-04-22T18:38:15.177" Score="1" ViewCount="97" Body="&lt;p&gt;Which one of the following is suitable for judging convergence in Two-Fluid Flow Solver?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Absolute Residual (L^2-Norm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Relative Residual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Fraction Change in Velocity, Pressure and Volume Fraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am asking this question because due to &lt;strong&gt;large value of momentum exchange coefficients&lt;/strong&gt; one can get very high value of absolute residual even when solutions are close to actual value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other criterion used?&lt;/p&gt;&#xA;" OwnerUserId="18157" LastEditorUserId="18157" LastEditDate="2016-04-22T21:32:48.780" LastActivityDate="2016-04-23T04:37:20.833" Title="Convergence Criteria for Two Fluid Flow Solver" Tags="|numerical-analysis|fluid-dynamics|simulation|computational-physics|numerical-modelling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23778" PostTypeId="1" AcceptedAnswerId="23781" CreationDate="2016-04-22T18:39:06.137" Score="4" ViewCount="3856" Body="&lt;p&gt;In &lt;a href=&quot;http://web.stanford.edu/class/ee364b/lectures/subgradients_notes.pdf&quot; rel=&quot;nofollow&quot;&gt;these notes&lt;/a&gt; (section 2.3), it is stated that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;A point $x^*$ is a minimizer of a function $f$ (&lt;strong&gt;not necessarily convex&lt;/strong&gt;) if and only if $f$ is subdifferentiable at $x^*$ and $0 \in\partial f(x^*).$&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Could anybody provide me with references for a proof of the above statement? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a reference where we can learn more about subgradients of non-convex functions? &#xA;In Section &lt;strong&gt;3. Calculus of subgradients&lt;/strong&gt; of the above notes,many properties of subgradients are presented for convex functions. I would like to know which ones among these properties still hold for non-convex functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2016-04-22T19:48:54.040" Title="Subgradients of non-convex functions" Tags="|optimization|convex-optimization|nonconvex|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23780" PostTypeId="2" ParentId="23778" CreationDate="2016-04-22T19:00:11.767" Score="1" Body="&lt;p&gt;In terms of the subdifferential definition used in the notes, the statement is immediate by definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more general notion of subdifferential, Proposition 2.3.2 on page 38 of Clarke, F. H. (1990), &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9781611971309&quot; rel=&quot;nofollow&quot;&gt;&quot;Optimization and Nonsmooth Analysis&quot;&lt;/a&gt; says:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $f$ attains a local minimum or maximum at $x$, then $0\in \partial f (x).$&lt;/p&gt;&#xA;" OwnerUserId="20104" LastEditorUserId="20104" LastEditDate="2016-04-22T19:15:25.480" LastActivityDate="2016-04-22T19:15:25.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23781" PostTypeId="2" ParentId="23778" CreationDate="2016-04-22T19:05:49.233" Score="8" Body="&lt;p&gt;The fact that $x^*$ is a (global!) minimizer of $f$ if and only if $0\in\partial f(x^*)$ is already fully explained in the notes you linked to -- it's really that simple, but here's the argument again for the sake of completeness. Assume that $x^*$ is a global minimizer of $f$. Then, by definition,&#xA;$$&#xA;f(x) - f(x^*) \geq 0 = 0^T (x-x^*) \qquad\text{for any }x\in\mathbb{R}^n,&#xA;$$&#xA;which is exactly the definition of the convex subdifferential (and subdifferentiability). For the other direction, you just swap the last equality around to see that one definition implies the other. (Note that this argument only works for &lt;em&gt;global&lt;/em&gt; minimizers -- this is where the convexity of $f$ really comes in, because for convex functions, every minimizer is a global minimizer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to the other properties for non-convex functions: The short answer is &lt;strong&gt;none&lt;/strong&gt;. Note that the reverse direction assumes that $f$ is subdifferentiable, i.e., that the set $\partial f(x^*)$ is non-empty. This doesn't sound like much (indeed, it's always true for any (locally finite) convex function), but it pretty much only holds for convex functions. In fact, you can show that if the subdifferential is non-empty everywhere, then $f$ is convex (see &lt;a href=&quot;https://math.stackexchange.com/q/1499059&quot;&gt;https://math.stackexchange.com/q/1499059&lt;/a&gt;). So all the other properties -- which are statements about some elements of the subdifferential -- are in general true only in the vacuous sense (as statements about the empty set).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are further generalizations of subdifferentials for non-convex functions, which are non-empty for larger (but still restricted) classes of functions and admit similar properties (in particular, necessary optimality conditions). Needless to say, the larger the class, the trickier they are to work with. One prominent example is Clarke's generalized gradient of locally Lipschitz continuous functions (see, e.g. Chapter 10  &lt;a href=&quot;http://dx.doi.org/10.1007/978-1-4471-4820-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clarke, &lt;em&gt;Functional Analysis, Calculus of Variations and Optimal Control&lt;/em&gt;, Springer 2013&lt;/a&gt;). You can find even more generalized derivatives in &lt;a href=&quot;http://dx.doi.org/10.1007/978-3-540-71333-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schirotzek, &lt;em&gt;Nonsmooth analysis&lt;/em&gt;, Springer 2007&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-04-22T19:48:54.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23782" PostTypeId="1" CreationDate="2016-04-22T20:30:15.513" Score="7" ViewCount="990" Body="&lt;p&gt;Analytically, the expression&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\exp(\ln(x))-x \enspace,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;should give 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in Matlab, it does not.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = linspace(1, 10, 10);&#xA;exp(log(x)) - x;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for $x \in [1, 10]$, the answer is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0   &#xA;0   &#xA;-4.44089209850063e-16   &#xA;0   &#xA;0   &#xA;0   &#xA;0   &#xA;-1.77635683940025e-15   &#xA;1.77635683940025e-15    &#xA;1.77635683940025e-15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is happening and how can I fix this?&lt;/p&gt;&#xA;" OwnerUserId="19465" LastEditorUserId="123" LastEditDate="2017-01-04T10:16:07.800" LastActivityDate="2017-01-12T06:47:54.623" Title="Why is $\exp(\ln(x))-x\neq0$ in floating point arithmetic?" Tags="|floating-point|precision|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23783" PostTypeId="1" CreationDate="2016-04-22T20:33:10.110" Score="2" ViewCount="200" Body="&lt;p&gt;Consider a voice recording split into it's phonemes as our sample $S=(s_1,...,s_k) \in \Omega = P^k$. The number of phonemes is $|P| = 40$. Then I have a word $w = (w_1,...,w_n) \in P^n$. I want to analyze with what chance I can reconstruct $w$ by just cutting out parts of the sample and chaining them together in the right order. I assume that $p(s_i=p) = x_p$ for $i=1,..,n$ and $p \in P$, i.e. that each sample symbol $s_i$ is chosen independently from the others and the distribution of the phonemes is given by $x_j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I wanted to know what the probability is that one can reconstruct a word &quot;in principle&quot; by taking single phonemes from the sample and chaining them in the right order. One can easily calculate this by naively iterating through all possible samples. This is obviously not feasable for larger k. Instead, I take the set $X = \{w_i : 1 \leq i \leq n\}$ of all required phonemes. Let $R_A^k$ be all the samples of length k where all phonemes in $A \subseteq X$ are contained in the sample (i.e. $A$ is reconstructable). Note that $p(S \in R_X^k)$ is my answer. The states $Q=\{R_A : A \subseteq X\}$ form a Bayesian Network where the transitions represent going from $k$ to $k+1$. For instance, if one is in state $R_A$ one ends up in $R_{A \cup \{p\}}$ with probability $x_p$ for $p \in X \setminus A$. Now I can iteratively calculate the probability of $w$ being reconstructable in $O(k \cdot 2^{|X|})$ and this suffices for my data (in one dataset, the biggest $n$ is $10$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to know the probability for reconstructing the word using $c$ substrings in $S$. For instance, if I want to reconstruct the word $w=$ACEG and I have the sample $S=$CEGBAF then I can reconstruct $w$ as A+CEG using 2 substrings. I try to group all samples into relevant states so that I can iteratively compute over k like before. My idea is to save the longest available substring in each position of w, as well as the start position and length of the longest streak in S (the largest substring at the end of $S$ that is in $w$), so $q=((q_1,...,q_n),a,l)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: $w=$ABCDE and the state $q=((2,1,2,1,0),3,1)$. This means we gathered two substrings &quot;AB&quot; and &quot;CD&quot;, the last streak in S was &quot;C&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively there are $n! \cdot \frac{n \cdot (n+1)}{2}$ such states. One can cut down this number by observing constraints in $q$: $(a=0 \wedge l=0) \vee (0 \lt l \leq q_a \lt n-a)$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented this in python (Note that string indexing starts at zero here):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python&#xA;import itertools&#xA;&#xA;def enum_states(n):&#xA;    states = [(i,) for i in xrange(0,n+1)]&#xA;    for i in xrange(n-1):&#xA;        states = [q + (x,) for q in states for x in xrange(max(0, q[-1]-1), n - i)]&#xA;    states = [(q, a, l) for q in states for a in xrange(0, n) for l in xrange(0, n-a+1) if (a==0 and l==0) or (l &amp;gt; 0 and l &amp;lt;= q[a] and q[a] &amp;lt; n-a)]&#xA;    return states&#xA;def print_states(states):&#xA;    i = 0&#xA;    for q in states:&#xA;        tup = q[0]&#xA;        a = q[1]&#xA;        l = q[2]&#xA;        tup = [str(e) for e in tup]&#xA;        if l == 0:&#xA;            print ''.join(tup) + &quot;  &quot;,&#xA;        else:&#xA;            print ''.join(tup[:a]) + &quot;&amp;lt;&quot; + ''.join(tup[a:a+l]) + &quot;&amp;gt;&quot; + ''.join(tup[a+l:]),&#xA;        i += 1&#xA;        if i%8 == 0:&#xA;            print&#xA;&#xA;states = enum_states(10)&#xA;#print_states(states)&#xA;print len(states)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For $n=10$ there are $|Q|=978926$ states! My original goal was to compute the transition matrix from the bayesian network. There are at most $(n+1)$ transitions from each state (one for each phoneme in $w$ and one for all phonemes not in $w$). The transition matrix $M$ would therefore be very sparse and could be stored efficiently in a CSR Matrix. However iterating over k going one step at a time would be too slow for large k. Instead one can calculate $M^k$ using fast exponentiation. But the resulting matrix would not be sparse any more and just be too big to work with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I could articulate my problem clearly. To sum up:&#xA;How can I compute the probability of a word being reconstructable using $c$ substrings from a sample of length $k$ efficiently? Am I missing something? Maybe there's a smarter way of doing this.&lt;/p&gt;&#xA;" OwnerUserId="20125" LastActivityDate="2018-07-12T10:03:07.627" Title="Probability of reconstructing a word using c substrings from a random sample" Tags="|linear-algebra|python|sparse-matrix|complexity|probability|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23784" PostTypeId="2" ParentId="23782" CreationDate="2016-04-22T20:42:20.600" Score="10" Body="&lt;p&gt;The numeric precision is not perfect. You get rounding errors during your computation. When working with floats, don't check if they are = 0, but check if their absolute distance to 0 is smaller than some epsilon.&lt;/p&gt;&#xA;" OwnerUserId="20125" LastActivityDate="2016-04-22T20:42:20.600" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23785" PostTypeId="2" ParentId="23783" CreationDate="2016-04-22T20:51:58.270" Score="1" Body="&lt;p&gt;I don't know the answer to your question, but there is almost certainly a large amount of literature on it in the computational biology community. This is because your problem is in essence how genomes are reconstructed: genome sequencers break the many copies of the same DNA (which is a string consisting of 4 letters) into small pieces at essentially random locations (i.e., you get many many substrings that will partially overlap), and can then read these substrings. The goal of &quot;sequence assembly&quot; is then to recover the long string from the many substrings that, hopefully, cover the long string many times over.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this is the community (and associated literature) I would start to look  into to get your question answered.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-22T20:51:58.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23787" PostTypeId="2" ParentId="23777" CreationDate="2016-04-23T04:37:20.833" Score="1" Body="&lt;p&gt;Normally in any numerical solution of PDEs, you have the conjugate pairs, say, (displacement, force), OR (pressure, velocity) that contribute to the energy of the system. Let's notate these abstractly as (degree of freedom, its conjugate), or (DOF, CJG). In order to cover a wide range of application, one should check for convergence of&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Relative residual (of the conjugate) values against some reference value, say L2-norm of the residual in system,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Iterate value of the DOF against some reference value, could be L2 or L1-norm of DOF in the system, and&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Verify that $\Delta $Energy due to &quot;incremental DOF times residual&quot; is small compared to, say, the energy input to the system.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;1 verifies that the residual goes to zero relative to some norm, 2 helps when large changes in DOFs are seen even at low values of residuals, and 3 ensures there is energy balance in the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is very rare to check for absolute value of residuals (at least in solid mechanics, my field).&lt;/p&gt;&#xA;" OwnerUserId="20098" LastActivityDate="2016-04-23T04:37:20.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23788" PostTypeId="1" AcceptedAnswerId="23789" CreationDate="2016-04-23T08:33:01.893" Score="2" ViewCount="911" Body="&lt;p&gt;On a grid I am having the values of a physical quantity say for example Temperature, at the E,W,N,S and P node all of them being calculated using a second order discretization scheme. I want a second order accurate estimate of the value of Temperature at point X. How to get that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note- I can get a second order accurate value of temperature at point IP1 (using second order polynomial fitting with point E, W and P) and at point IP2 (using second order polynomial fitting with point N, S and P). Both IP1 and IP2 are at distance Δx/4Δx/4 from point P.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kindly refer to the stencil for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eFBLq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eFBLq.png&quot; alt=&quot;Stencil&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17615" LastEditorUserId="17615" LastEditDate="2016-04-23T08:50:20.940" LastActivityDate="2016-04-23T20:18:19.570" Title="Second order interpolation scheme" Tags="|finite-difference|numerical-analysis|computational-physics|discretization|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23789" PostTypeId="2" ParentId="23788" CreationDate="2016-04-23T13:03:49.413" Score="2" Body="&lt;p&gt;In 2D you would need a bi-quadratic interpolation function such as $(a_0 + a_1 x + a_2 x^2)(b_0 + b_1 y + b_2 y^2)$ giving rise to 9 undetermined coefficients. Thus you need 9 points where the values are known then you can determine these coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the least you need incomplete polynomial up to 2nd order for a 2nd order estimate, i.e., $\{1,x,y,x^2,y^2,xy\}$ with 6 coefficients, thus you need 6 points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With 5 points and 6 coefficients you will have an under determined linear least squares problem. You can convert it to an over determined system by adding more points, especially if you are on, say, a finite difference grid. You can find details in any numerical analysis text book. &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)&quot; rel=&quot;nofollow&quot;&gt;See also this&lt;/a&gt;. I give some basics below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your goal is to find the polynomial function coefficients - the function itself is linear in the coefficients. Now express the sum of the square of error between interpolated values $v^I$ and actual values $v^A$ at $n&amp;gt;6$ points as:&#xA;\begin{align*}&#xA;E = \sum_{i=1}^n(v^I(a_p, \mathbf x_i) - v^A_i)^2 \qquad&amp;amp; v^I \text{ is linear in } a_p \text{ evaluated at } \mathbf x_i \\&#xA;= \sum_{i=1}^n(v^I_p(\mathbf x_i) a_p - v^A_i)^2\qquad&amp;amp; \text{p}^{th} (1...6) \text{ coefficient of } v^I \text{ at } \mathbf x_i \\&#xA;{{\partial E}\over{\partial a_k}} = \sum_{i=1}^n 2(v^I_p(\mathbf x_i) a_p - v^A_i) v^I_p(\mathbf x_i)\delta_{pk} = 0\qquad&amp;amp; \text{minimize w.r.t. } a_k \\&#xA;\implies \sum_{i=1}^n v^I_k(\mathbf x_i) v^I_p(\mathbf x_i) a_p = \sum_{i=1}^n v^I_k(\mathbf x_i) v^A_i\qquad&amp;amp;&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly if you have an over determined system, you can also go with the approach of bi-quadratic functions with 9 coefficients (on a 3x3 grid) at each point.&lt;/p&gt;&#xA;" OwnerUserId="20098" LastEditorUserId="20098" LastEditDate="2016-04-23T20:18:19.570" LastActivityDate="2016-04-23T20:18:19.570" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="23790" PostTypeId="2" ParentId="23782" CreationDate="2016-04-23T17:11:30.490" Score="7" Body="&lt;p&gt;[EDIT] An alternate view: 64-bit floating numbers represent a discrete set $S$. For a function $f$ to be exactly invertible,  it should be a bijection from $S$ to $S$.  Suppose we are interested in a fast growing function like $\exp$. At one point, $\exp x &amp;gt; x$.&#xA;If $M$ is the maximum element from $S$, then $f(M)\ge M$. The strict version $f(M)&amp;gt; M$ is not  allowed, because there is no higher number that $M$ in $S$. Morally, a monotonic function acting like a bijection on a finite set is a permutation, and has very limited degrees of freedom, it is likely to be limited to the identity, or or a counter-identity which reverts the order of elements in the set. Outside these trivial examples, monotonic functions cannot be bijections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, it is due to finite-precision numerical computations (for instance &lt;code&gt;64-bit&lt;/code&gt; floating point), that especially affect functions involving irrationals numbers. Integer or sum-of-powers-of-two calculations may be less sensitive.  If you dare to plot the numerical error, you get something like:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JGAOg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JGAOg.png&quot; alt=&quot;the plane of nummerical errors&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;code&gt;Matlab&lt;/code&gt; code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=linspace(1,10000,10000);&#xA;y = exp(log(x))-x;&#xA;plot(x,y,'.');&#xA;xlabel('x');&#xA;ylabel('y');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you can see that the maximum error spreads with the value. If you want to do something, you can check and threshold a relative error, especially if your variable spread over a large scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It get even weirder, if you just swap the functions, say: $\log(\exp(x))-x$, the error is in this case numerically zero.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=linspace(1,700,10000);&#xA;y = log(exp(x))-x;&#xA;plot(x,y,'.');&#xA;xlabel('x');&#xA;ylabel('y');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RlaCF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RlaCF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="19501" LastEditDate="2017-01-12T06:47:54.623" LastActivityDate="2017-01-12T06:47:54.623" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23791" PostTypeId="1" CreationDate="2016-04-23T17:17:11.693" Score="0" ViewCount="100" Body="&lt;p&gt;I have a symmetric $ 3 \times 3 $ matrix $A$ and I need to compute the eigenvectors and eigenvalues of this. I know that I can use something like Lapack, but I also know that this can be computed analytically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The physics suggest that this will have only real eigenvalues, but the analytic expression allows for imaginary numbers.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a simply analytic expression for this?&lt;/li&gt;&#xA;&lt;li&gt;If I can generate the characteristic equation analytically, would it be most efficient to solve this equation numerically to obtain the eigenvalues?&lt;/li&gt;&#xA;&lt;li&gt;Is it just as simple / efficient to simply use something like Lapack for this?&lt;/li&gt;&#xA;&lt;li&gt;Any other ideas?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="164" LastActivityDate="2016-04-23T17:25:22.357" Title="Most efficient way to compute eigenvectors / values of this matrix?" Tags="|linear-algebra|eigenvalues|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23792" PostTypeId="2" ParentId="23791" CreationDate="2016-04-23T17:25:22.357" Score="2" Body="&lt;p&gt;You may want to consider a  &quot;A Robust Eigensolver for 3 × 3 Symmetric Matrices&quot; by &#xA;David Eberley: &lt;a href=&quot;http://www.geometrictools.com/Documentation/RobustEigenSymmetric3x3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.geometrictools.com/Documentation/RobustEigenSymmetric3x3.pdf&lt;/a&gt;&#xA;There is also this question:&#xA;&lt;a href=&quot;https://dsp.stackexchange.com/questions/1911/analytical-expression-for-the-eigenvectors-of-a-3x3-real-symmetric-matrix&quot;&gt;https://dsp.stackexchange.com/questions/1911/analytical-expression-for-the-eigenvectors-of-a-3x3-real-symmetric-matrix&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20104" LastEditorUserId="-1" LastEditDate="2017-04-13T12:47:29.437" LastActivityDate="2016-04-23T17:25:22.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23793" PostTypeId="1" AcceptedAnswerId="23794" CreationDate="2016-04-23T17:43:02.130" Score="1" ViewCount="105" Body="&lt;p&gt;What is computational complexity for computing perimeter of a polygon of $n$ vertices? The polygon is not necessarily regular and can be convex or non-convex.&lt;/p&gt;&#xA;" OwnerUserId="16876" LastEditorUserId="549" LastEditDate="2016-04-23T19:10:46.407" LastActivityDate="2016-04-24T08:14:20.330" Title="computational complexity for computing perimeter of a polygon" Tags="|algorithms|computational-geometry|complexity|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23794" PostTypeId="2" ParentId="23793" CreationDate="2016-04-23T19:02:50.470" Score="1" Body="&lt;p&gt;Either one way or the other, the polygon is composed of vertices and the links between vertices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically this is expressed as an ordered sequence of vertices. In this case, you just sum up the consecutive vertex to vertex distances and this is $O(N)$. The main reason is that the single distance computation is $O(1)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\lVert \mathbf{v}_i - \mathbf{v}_j \rVert = \sqrt{(\mathbf{v}^x_i-\mathbf{v}^x_j)^2+(\mathbf{v}^y_i-\mathbf{v}^y_j)^2}&#xA;$$&#xA;and you simply perform a single pass over the vertices&#xA;$$&#xA;P = \sum\limits_{i=2}^N \lVert \mathbf{v}_i - \mathbf{v}_{i-1} \rVert &#xA;$$&#xA;where $P$ is the perimeter. For closed polygons, you just add another distance:&#xA;$$&#xA;P_C = \sum\limits_{i=2}^N \big(\lVert \mathbf{v}_i - \mathbf{v}_{i-1} \rVert \big) +\lVert \mathbf{v}_1 - \mathbf{v}_{N} \rVert &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the polygon is presented as a vertex &amp;amp; edge list (similar to a graph), then you iterate over all the edges and once again sum up the edge lengths. You should take care of not adding the contributions multiple times. This can easily be done with the help of a hashtable, making the entire complexity to be $O(N)$, again.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2016-04-24T08:14:20.330" LastActivityDate="2016-04-24T08:14:20.330" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23797" PostTypeId="1" CreationDate="2016-04-24T00:52:51.837" Score="6" ViewCount="412" Body="&lt;p&gt;I'm working with the cylindrical coordinates. I'm using the central difference to convert the radial part of Laplace operator into a matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^2 u = \frac{\partial ^2u}{\partial r^2}+\frac{1}{r}\frac{\partial u}{\partial r}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which in the discretized form gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^2 u_i = \frac{u_{i+1} - 2u_i +u_{i-1}}{h^2}+\frac{1}{r_i}\frac{u_{i+1} -u_{i-1}}{2h}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &quot;h&quot; is the step size and $\ r_i = i*h $. Thus:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^2 u_i = \frac{u_{i+1} - 2u_i +u_{i-1}}{h^2}+\frac{1}{i*h}\frac{u_{i+1} -u_{i-1}}{2h}=\frac{1}{h^2}[u_{i+1}(1+\frac{1}{2i})-2u_i+u_{i-1}(1-\frac{1}{2i})]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the off diagonals: coefficients of $(u_{i+1})$&#xA;and $(u_{i-1})$ will have the $1/r_i$ dependence, so that would affect the matrix symmetry. This is because in the matrix, the element &quot;ij&quot; will depend on $1/r_i$ while its transpose &quot;ji&quot; will depend on $1/r_j$. Hence, the first few elements of the matrix will look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{matrix}&#xA;        -2 &amp;amp; 1.5 &amp;amp; 0 &amp;amp; \cdots \\&#xA;        0.75 &amp;amp; -2 &amp;amp; 1.25 &amp;amp; 0 &amp;amp; \cdots\\&#xA;        0 &amp;amp; \frac{5}{6} &amp;amp; -2 &amp;amp; \frac{7}{6} &amp;amp; \cdots\\&#xA;        \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots \\&#xA;        \end{matrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the Laplace operator is hermitian, why isn't the matrix symmetric? What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="20146" OwnerDisplayName="user334106" LastActivityDate="2016-05-03T16:34:03.030" Title="Matrix representation of the radial Laplace operator isn't symmetric as supposed" Tags="|numerical-analysis|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23798" PostTypeId="2" ParentId="23797" CreationDate="2016-04-24T04:31:26.763" Score="1" Body="&lt;p&gt;Start with the original definition and discretize consistently:&#xA;\begin{align*}&#xA;  \frac{1}{r}\frac{\partial}{\partial r} \left( r \frac{\partial u}{\partial r} \right) &amp;amp;= \frac{1}{r_i \thinspace h} \left( r_{i+\frac{1}{2}} \left[ \frac{u_{i+1} - u_i}{h} \right] - r_{i-\frac{1}{2}} \left[ \frac{u_i - u_{i-1}}{h} \right] \right) \\&#xA;  &amp;amp;= \frac{1}{i \thinspace h^2} \left( (i+\tfrac{1}{2}) \left[ u_{i+1} - u_i \right] - (i-\tfrac{1}{2}) \left[ u_i - u_{i-1} \right] \right) \\&#xA;  &amp;amp;= \frac{1}{i \thinspace h^2} \left( (i+\tfrac{1}{2}) u_{i+1} - 2 \thinspace i \thinspace u_i + (i-\tfrac{1}{2}) u_{i-1} \right)&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That should result in the symmetric matrix you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="20098" LastActivityDate="2016-04-24T04:31:26.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23799" PostTypeId="1" CreationDate="2016-04-24T10:24:24.590" Score="2" ViewCount="1184" Body="&lt;p&gt;I want to program an integration routine in C++ using the GSL library but for complex functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should I split my integrand to apply the gsl_integration_qag function on it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just integrate the real and imaginary parts by themselves is not correct I guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following function:&#xA;$$S(a,N) = S(a, \infty) - \sum^{\infty}_{n = 1} \frac{1}{n + N}$$&#xA;where $S(a, \infty)$ is some real constant.&#xA;The functions I later intend to integrate are using the harmonic sum as a basis but are just sums of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integration I intend to do is the Mellin transform which is given by:&#xA;$$f(x) = \frac{1}{2 \pi i} \int_C dn x^{-n} f^{M}(n).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is much approciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Any idea on a C++ program for complex numerical integration?&lt;/p&gt;&#xA;" OwnerUserId="19384" LastEditorUserId="19384" LastEditDate="2016-04-24T16:05:01.077" LastActivityDate="2016-07-24T11:01:20.007" Title="Complex Numerical Integration using GSL" Tags="|c++|integration|complex-analysis|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="23800" PostTypeId="1" AcceptedAnswerId="23801" CreationDate="2016-04-24T13:51:57.990" Score="12" ViewCount="11946" Body="&lt;p&gt;I am totally confused between ODEs which I am familiar with, and differential algebraic equations (DAE) and algebraic differential equations (ADE). Are they the same but just different names or what is the key difference between them (their nature and solution methods)?&lt;/p&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="20688" LastEditDate="2022-06-15T21:51:28.257" LastActivityDate="2022-06-15T21:51:28.257" Title="ODEs vs DAE vs ADE?" Tags="|ode|numerical-modelling|dae|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="23801" PostTypeId="2" ParentId="23800" CreationDate="2016-04-24T14:59:07.510" Score="12" Body="&lt;p&gt;At least one difference is that in a system of ODEs, all the equations are differential, e.g.: &#xA;$$&#xA;\dot{x}=f(x,y)\\&#xA;\dot{y}=g(x,y)&#xA;$$&#xA;whereas the definition of DAEs that I'm familiar with includes some non-differential (i.e. algebraic) equations in the set, e.g.:&#xA;$$&#xA;\dot{x}=h(x,y)\\&#xA;y=l(x,y)&#xA;$$&#xA;where $l$ is non-trival, and its solution can't be easily substituted into the first equation to simplify. These get more complex when there are more algebraic terms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;DAEs are more challenging numerically; the challenges they entail are similar to but sometimes more severe than those attending stiff problems.  A very thorough explanation of DAEs and how to solve them numerically can be found in &lt;a href=&quot;http://www.springer.com/us/book/9783540604525&quot;&gt;volume II of the text by Hairer and Wanner&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="123" LastEditDate="2016-04-25T05:58:25.317" LastActivityDate="2016-04-25T05:58:25.317" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23802" PostTypeId="2" ParentId="23800" CreationDate="2016-04-24T17:22:49.553" Score="8" Body="&lt;p&gt;Differential-algebraic equations (DAE) are equations of the form $F(t,x,x')=0$, with the unknown function being $x(t)$. So in a way are generalizations of ODEs. A nice place to start is &lt;a href=&quot;http://www.scholarpedia.org/article/Differential-algebraic_equations&quot;&gt;here&lt;/a&gt;. On the other hand an algebraic differential equation is a totally different thing. The wikipedia page gives an overview, but basically is an equation involving differential operators on a differential algebra.&lt;/p&gt;&#xA;" OwnerUserId="1302" LastEditorUserId="1302" LastEditDate="2016-04-24T18:30:13.890" LastActivityDate="2016-04-24T18:30:13.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23803" PostTypeId="1" AcceptedAnswerId="23813" CreationDate="2016-04-24T20:04:46.823" Score="2" ViewCount="557" Body="&lt;p&gt;I'm interested in a function that would allow me to weight my system similar to using the error function; however computing the error function at scale would be a bottleneck. Is there something like erf, but that is much faster to compute and with the following properties?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given any value [-inf, inf], returns an output in the range [-1, 1], goes through (0,0), and allows me change the slope at (0,0) to tune into my system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was considering a Hermitian  polynomial, but then I would be constrained within a certain domain... &lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="276" LastEditDate="2017-10-10T00:48:25.037" LastActivityDate="2017-10-10T00:48:25.037" Title="Function similar to erf that is fast at scale and allows for changing the slope at 0?" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23805" PostTypeId="1" AcceptedAnswerId="23806" CreationDate="2016-04-25T00:26:36.430" Score="2" ViewCount="149" Body="&lt;p&gt;Is there a divide and conquer algorithm for optimizing weakly unimodal continuous functions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding more details: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My function has a flat line on the left and right and then there is a global minima in between but this global minima repeats for a while on what would seem like a straight line. To the left of the minima and right of the flat line on left, the behavior is monotonic decreasing. To the right of the minima and left of the flat line on right, the behavior is monotonic increasing. Also, the function is a positive function that is it's range is always &gt;0. In addition, the function is defined only above zero. i.e, it's domain is always &gt;0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see the main issue being that the initial bracket of three points may not be ideal. If the initial bracket was ideal, then does golden section search converge to optimal solution? Or is the issue bigger-even if the initial bracket was ideal? &lt;/p&gt;&#xA;" OwnerUserId="16876" LastEditorUserId="16876" LastEditDate="2016-04-26T06:40:37.357" LastActivityDate="2016-04-26T06:40:37.357" Title="Divide and conquer for optimizing weakly unimodal continuous function?" Tags="|optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23806" PostTypeId="2" ParentId="23805" CreationDate="2016-04-25T01:42:01.977" Score="4" Body="&lt;p&gt;No, there is not.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that your weakly unimodal function might be constant except for a very small &quot;bump&quot;.  No algorithm that evaluates the function a finite number of times can distinguish such a function from the constant function unless it gets lucky. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you had such an algorithm and ran it on a constant test function $f(x)=c$ for which it stopped after $k$ function evaluations (all of which had the same value.)  Next run it on a function $g$ that has the same constant value at each of those $k$ points but also has a hidden bump that doesn't hit any of those points.  Your algorithm would stop after the same $k$ iterations with an incorrect result.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Divide and conquer algorithms for global minimization of non-convex functions require additional properties of the function  (for example Lipschitz continuity) in order to deal with this issue.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-04-25T01:42:01.977" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23807" PostTypeId="1" AcceptedAnswerId="23808" CreationDate="2016-04-25T01:51:29.560" Score="5" ViewCount="167" Body="&lt;p&gt;Are there papers that deal with the (nondimensionalized) Stokes equation for incompressible fluid flow in a &quot;doubly mixed&quot; form like the following?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;0&amp;amp;=\underline{\epsilon} + \frac{1}{2} (\vec{\nabla} \vec{u}+ (\vec{\nabla} \vec{u})^{T})\\&#xA;\vec{f}&amp;amp;= \vec{\nabla} \cdot \underline{\epsilon} + \nabla p\\&#xA;0&amp;amp;= \vec{\nabla} \cdot \vec{u}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or using Einstein notation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;0 &amp;amp;= \epsilon_{ij}+ \frac{1}{2} (u_{i,j}+u_{j,i})\\&#xA;f_i &amp;amp;= \epsilon_{ij,j}+p_{,i}\\&#xA;0&amp;amp;= u_{i,i}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, much like the mixed form for linear elasticity we write the symmetric tensor $\underline{\epsilon}$ as the symmetric gradient of $\vec{u}$.  I am looking for any references in the literature for finite element methods that solve the Stokes equation this way (as a system of three first order PDE's).  &lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2016-05-05T17:55:07.547" LastActivityDate="2016-05-05T17:55:07.547" Title="Stokes Equation in &quot;two-fold saddle point&quot; form?" Tags="|finite-element|navier-stokes|elliptic-pde|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23808" PostTypeId="2" ParentId="23807" CreationDate="2016-04-25T06:21:11.200" Score="2" Body="&lt;p&gt;You can for instance have a look at the different HDG papers by Cockburn et. al. They use a similar form to formulate their hybridizable approaches. An overview is given in this review paper: &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045793013004568&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0045793013004568&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4260" LastActivityDate="2016-04-25T06:21:11.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23809" PostTypeId="2" ParentId="23797" CreationDate="2016-04-25T06:49:58.310" Score="1" Body="&lt;p&gt;I think the answer might be in the equation you're looking at. Consider solving the equation $Ax=b$, where $A=\nabla^2$, $x$ is the unknown and $b$ is the right hand side. Multiplying the left and right hand side by the volume ($r dr d\theta dz$), will result in a symmetric matrix $A$ on the left hand side, right? Then you'd be solving the modified system $Dx = f$ where $D = dV A$ and $f = dV b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if there is a way to symmetrize the operator without modifying other parts of the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @nameRakes pointed out, this would become&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;  dV&#xA;  \frac{1}{r}\frac{\partial}{\partial r} \left( r \frac{\partial u}{\partial r} \right) &amp;amp;= \frac{k}{h} \left( r_{i+\frac{1}{2}} \left[ \frac{u_{i+1} - u_i}{h} \right] - r_{i-\frac{1}{2}} \left[ \frac{u_i - u_{i-1}}{h} \right] \right) \\&#xA;  &amp;amp;= \frac{k}{\thinspace h} \left( (i+\tfrac{1}{2}) \left[ u_{i+1} - u_i \right] - (i-\tfrac{1}{2}) \left[ u_i - u_{i-1} \right] \right) \\&#xA;  &amp;amp;= \frac{k}{\thinspace h} \left( (i+\tfrac{1}{2}) u_{i+1} - 2 \thinspace i \thinspace u_i + (i-\tfrac{1}{2}) u_{i-1} \right)&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $dV=rk,k=drd\theta dz$. You'll notice that the equation is slightly different now and you're only dividing by $h$ instead of $h^2$, but this is okay so long you treat your RHS the same (by multiplying by $V$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is symmetric, I'm not sure about Hermitian though.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="8174" LastEditDate="2016-04-26T19:12:47.567" LastActivityDate="2016-04-26T19:12:47.567" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23811" PostTypeId="2" ParentId="23800" CreationDate="2016-04-25T08:32:00.137" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://mathoverflow.net/questions/225840/index-reduction-of-differential-algebraic-equations-by-hand/226517#226517&quot;&gt;Here is an identical copy of an answer on MO&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One intuitive way to understand a &lt;a href=&quot;http://www.scholarpedia.org/article/Differential-algebraic_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAE&lt;/a&gt; is to interpret it as a dynamical system which can be controlled by some input signals, whose output signals have to satisfy some (equational) constraints. For a typical multibody system, the input signals are the forces perpendicular to the constraints, the output signals are the positions of the bodies, and the (equational) constraints on the output signals are fixed distances between the bodies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input signals must now control the dynamical system in such a way that the output signals always satisfy the constraints. This is difficult for a multibody system, because the forces only control the rate of change of the velocities, and the velocities only control the rate of change of the positions, while only the positions must satisfy the constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reducing the index is easy in theory, because if we assume that the positions satisfy the constraints at the current time instance, then we can just replace the constraints on the positions by constraints on the velocities ensuring that the positions will continue to satisfy their constraints. In practice however, we don't want to throw away the constraint on the positions after we determined the constraints on the velocities, but we do have to throw away some of the initial (differential) equations, if we don't want to end with an overdetermined system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining the constraints on the velocities from the constraints on the positions might be tedious in practice, but at least it is straightforward (and canonical) once you understood the principle. The constraint $c(y,t)=0$ implies $\frac{d}{dt}c(y(t),t)=0=\frac{\partial c}{\partial y}*\frac{d}{dt}y+\frac{\partial c}{\partial y}$. This is not an (equational) constraint yet, because $\frac{d}{dt}y$ is not a variable but only the derivative of a variable. But the other differential equations allow us to express $\frac{d}{dt}y$ as a function of the variables, in our case $\frac{d}{dt}y=v$ for $v=\dot{y}$, so we get the equational constraint $0=\frac{\partial c}{\partial y}*v+\frac{\partial c}{\partial y}$ (or rather $0=\frac{\partial c}{\partial y}*\dot{y}+\frac{\partial c}{\partial y}$ if you manage to not get confused by using $\dot{y}$ as a variable instead of the derivative of a variable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Throwing away some of the initial (differential) equations is less straight forward (or canonical). If we can use a constraint equation like $y_1^2+y_2^2=1$ to determine $y_1$ as a function of the other variables (i.e. $y_1(t)=\sqrt{1-(y_2(t))^2}$ in this case), then we can throw away the differential equation for $y_1$, i.e. a differential equation of the form $\frac{d}{dt}y_1=\dots$. But we might have also decided to throw away the differential equation for $y_2$ instead, because the constraint also allow to determine $y_2$ as a function of the other variables. But no matter how easy it is to throw something away, this can easily destroy some symmetry of the system we didn't want to destroy, or we might be forced to switch which equation we throw away during the numerical simulation and thereby introduce undesired artifacts. So this part makes index reduction really challenging in practice.&lt;/p&gt;&#xA;" OwnerUserId="534" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2016-04-25T08:32:00.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23812" PostTypeId="2" ParentId="23799" CreationDate="2016-04-25T08:57:35.903" Score="1" Body="&lt;p&gt;I think the way to go is to parametrize the integral with a real parameter and integrate the real and imaginary parts seperately. Implementing this is subtle, the implementation depends from case to case.&lt;/p&gt;&#xA;" OwnerUserId="19384" LastActivityDate="2016-04-25T08:57:35.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23813" PostTypeId="2" ParentId="23803" CreationDate="2016-04-25T10:48:35.967" Score="5" Body="&lt;p&gt;You are probably looking for &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;noreferrer&quot;&gt;Sigmoid functions&lt;/a&gt;, i.e., functions with an 'S'-shape. Some examples besides the error function are &#xA;$\tanh(x)$, $\arctan{x}$, $\frac{x}{\sqrt{1+x^2}}$, $\frac{x}{1+|x|}$ and $\frac{1}{1+e^{-x}}$, which are illustrated in &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function#/media/File:Gjl-t%28x%29.svg&quot; rel=&quot;noreferrer&quot;&gt;this figure&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With approriate scaling and an offset, they can all go through $(0,0)$ and have a range of $(-1,1)$. An easy way to control their slope at $x = 0$ is to introduce a change of variable $x \to \lambda x$.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2016-04-25T10:48:35.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23814" PostTypeId="1" CreationDate="2016-04-25T12:55:27.857" Score="6" ViewCount="118" Body="&lt;p&gt;I have implemented Carlson's algorithm for $E(k)$ from &lt;em&gt;Numerical computation&#xA;of real or complex elliptic integrals&lt;/em&gt; (available from &lt;a href=&quot;http://arxiv.org/pdf/math/9409227v1.pdf&quot;&gt;ArXiv eprint&lt;/a&gt;, see&#xA;also &lt;a href=&quot;http://dlmf.nist.gov/19.8.E6&quot;&gt;DLMF&lt;/a&gt;). It is essentially his formula (46) which simultaneously&#xA;computes $K(k), E(k)$ for &lt;strong&gt;complex&lt;/strong&gt; $k$ using the arithmetic-geometric sequence $a_m, b_m$ to&#xA;get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$K(k) = \frac{\pi}{a_n+b_n}, \quad E(k) = \left( \left(\frac{a_0+b_0}{2} \right)^2&#xA;-\sum\limits_{m=1}^{n} 2^{m-2}(a_m - b_m)^{2}\right) K(k)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately with IEEE 64 or 80 bit floating point arithmetic the expression for $E(k)$ is accurate only up to $|k| \approx 1000,\,$ for larger arguments severe rounding/cancellation problems occur (e.g. for $k=10^3-10i$ the relative error in the real part is $17\epsilon,\;$for $k=10^4-10i$ it is $150\epsilon,\;$ and for $k=10^6-10i$ it is $24000\epsilon$). For very large $|k|$ I could use the logarithmic form, but is seems relative useless if there is no reliable&#xA;algorithm for medium $|k|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a known stable algorithm for this range, ie. which can compute $E(k)$ for $|k| &amp;gt; 100$ without using multi-precision?&lt;/p&gt;&#xA;" OwnerUserId="4853" LastEditorUserId="4853" LastEditDate="2016-04-25T13:23:24.573" LastActivityDate="2016-05-09T08:57:18.993" Title="Numerical computation of the complex elliptic integral $E(k)$ for medium $|k|$" Tags="|floating-point|special-functions|complex-arithmetic|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23815" PostTypeId="1" CreationDate="2016-04-25T15:20:11.213" Score="1" ViewCount="305" Body="&lt;p&gt;I am using the PETSc libraries, in particular using the SNES solver package to solve a nonlinear matrix equation.  Is there a way of parallelising the solution to this problem problem without using the DMDA distributed array methods? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked through the pets manual and the coverage of parallel nonlinear solvers is not well covered. &lt;/p&gt;&#xA;" OwnerUserId="20151" LastActivityDate="2016-04-25T15:49:34.420" Title="The alternative to using PETSc's SNES solvers in parallel without using the DMDA methods" Tags="|parallel-computing|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23816" PostTypeId="1" CreationDate="2016-04-25T15:25:01.047" Score="1" ViewCount="89" Body="&lt;p&gt;Given a block structured grid with a refinement factor of two, what are some of the common ways to discretize a Laplacian near a refinement boundary? See also the picture below, in which $u_H$ and $u_h$ indicate coarse and fine values with the given indices. I'm particularly interested in the computation of $\nabla^2 u$ on the 'fine' side of the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would like to have a discretization that is second order accurate with as small a stencil as possible. A conservative discretization would also be valuable, meaning that the 'fluxes' $\nabla u$ on the fine and coarse side of the boundary match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ebTWq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ebTWq.png&quot; alt=&quot;Cell-centered values near a refinement boundary&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18662" LastEditorUserId="18662" LastEditDate="2016-04-26T11:55:50.160" LastActivityDate="2016-04-26T11:55:50.160" Title="How to discretize Laplacian near refinement boundary" Tags="|laplacian|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23817" PostTypeId="2" ParentId="23815" CreationDate="2016-04-25T15:49:34.420" Score="2" Body="&lt;p&gt;There is nothing about &lt;code&gt;SNES&lt;/code&gt; that requires &lt;code&gt;DMDA&lt;/code&gt;.  Create a &lt;code&gt;Vec&lt;/code&gt; distributed however you would like the state and residual to be distributed, then use &lt;code&gt;SNESSetFunction()&lt;/code&gt; and (optionally) &lt;code&gt;SNESSetJacobian()&lt;/code&gt;. See &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/src/snes/examples/tutorials/ex2.c.html&quot; rel=&quot;nofollow&quot;&gt;SNES ex2.c&lt;/a&gt; for an example that does not use &lt;code&gt;DM&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2016-04-25T15:49:34.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23820" PostTypeId="2" ParentId="23816" CreationDate="2016-04-25T17:04:07.017" Score="1" Body="&lt;p&gt;I think one option is to interpolate $u$ to the left face of (3,1) first. Then you can use a modified Laplacian stencil to take into account the half-distance between $u$ at (3,1) and $u$ on the face.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-04-25T17:04:07.017" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23821" PostTypeId="2" ParentId="392" CreationDate="2016-04-25T18:19:54.980" Score="0" Body="&lt;p&gt;This is an old question, but I think the &lt;a href=&quot;https://en.wikipedia.org/wiki/Mozilla_Public_License&quot; rel=&quot;nofollow&quot;&gt;Mozilla Public License&lt;/a&gt; is worth mentioning as a middle ground between permissive licenses (BSD, MIT) and strong copyleft licenses (GPL). MPL code can be used and redistributed, but the MPL code and any modifications to it must be made available. For example, a company can take some MPL code, make their own improvements to it, and distribute it in a proprietary closed-source software package, so long as they make their modified version of the original MPL code available. They're not obligated to release all of their own source code, as they would with GPL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the BSD license, there's the fear that a corporation could take your code and improve it without giving back these contributions to you or the community at large, under the rationale that the improvements they make to it confer a competitive advantage. (Matt Knepley's answer suggests that not every acts this way though). On the other hand, many people might avoid GPL code altogether. The MPL avoids both of these potential pitfalls, at least in principle.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-04-25T18:19:54.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23822" PostTypeId="1" AcceptedAnswerId="23834" CreationDate="2016-04-25T18:30:24.567" Score="5" ViewCount="363" Body="&lt;p&gt;I have a linear system of equations that can be expressed $$A=B+AB,$$ where $A$ and $B$ are real, symmetric matrices. I would like to solve for $A$ given $B$. At present, I solve for $A$ directly via $$A=B(I-B)^{-1},$$ with the inverse computed via $LU$ factorization. For most relevant inputs $B$, the inversion goes well, but of course, sometimes $(I-B)$ is simply ill-conditioned. Is there a way to solve for $A$ without explicitly computing a matrix inverse?&lt;/p&gt;&#xA;" OwnerUserId="14469" LastEditorUserId="14469" LastEditDate="2016-04-25T19:37:59.950" LastActivityDate="2016-04-26T14:24:55.350" Title="Solving $A=B+AB$ without matrix inverse" Tags="|linear-algebra|matrix-factorization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23823" PostTypeId="2" ParentId="19694" CreationDate="2016-04-25T18:52:11.967" Score="0" Body="&lt;p&gt;Sorry if my comment is way too late but I just joined this community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may consider Bender's decomposition or Dantzig-Wolfe decomposition .. I've done similar thing : &#xA;min {c'x + E [Q(x)]| x&gt;=0}; where Q(x)= min {q'y|Wy =T(x)-d, y&gt;=0}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was able to solve it efficiently using Bender's decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good Luck,&lt;/p&gt;&#xA;" OwnerUserId="20156" LastActivityDate="2016-04-25T18:52:11.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23824" PostTypeId="1" CreationDate="2016-04-25T19:26:53.203" Score="0" ViewCount="48" Body="&lt;p&gt;I'm looking for an algorithm to take the entire vector space of length n Boolean vectors and partition it into vectors that are the same up to a rotation of the entries.&#xA;For example if n=3 the partitions would be:&#xA;-{(0,0,0)}&#xA;-{(0,0,1),(0,1,0),(1,0,0)}&#xA;-{(0,1,1),(1,1,0),(1,0,1)}&#xA;-{(1,1,1)}&#xA;One (seemingly slow) way to do this would be to generate the list of all possible Boolean vectors of length n and at each entry of the list generate all vectors that are similar to the original and delete these from the list.&#xA;Any ideas on a faster algorithm?&lt;/p&gt;&#xA;" OwnerUserId="20157" LastActivityDate="2016-04-25T21:10:44.800" Title="Grouping Boolean vectors by similarity up to a rotation" Tags="|linear-algebra|algorithms|" AnswerCount="2" CommentCount="0" ClosedDate="2016-04-25T21:20:07.917" ContentLicense="CC BY-SA 3.0" />
  <row Id="23826" PostTypeId="2" ParentId="23824" CreationDate="2016-04-25T19:38:01.187" Score="0" Body="&lt;p&gt;You can start by computing the sum of the boolean elements per vector. This would give you a permutation invariant metric per vector. You could then cluster the vectors, having same (or similar) sums into same buckets. You could easily use a hashtable for this. The operation is then completed in $O(N)$ time.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2016-04-25T19:38:01.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23827" PostTypeId="1" AcceptedAnswerId="23830" CreationDate="2016-04-25T20:04:39.177" Score="1" ViewCount="1307" Body="&lt;p&gt;I want to know what are the differences between two data structures in MATLAB: Structure Arrays and Cell Array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In what circumstances we can use each of them? &lt;/p&gt;&#xA;" OwnerUserId="20160" LastEditorUserId="9667" LastEditDate="2016-04-25T21:31:24.887" LastActivityDate="2016-04-26T00:29:39.093" Title="Difference between Data structure objects and cell array objects in matlab" Tags="|matlab|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23828" PostTypeId="1" CreationDate="2016-04-25T20:36:30.887" Score="1" ViewCount="66" Body="&lt;p&gt;I have a collection of different scrap aluminium alloys. I want to mix them together to make new alloys with customer-defined compositions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes this will involve little more than melting down several tonnes of scrap aluminium cans to make ingots, which are then sold to manufacturers of aluminium cans so that they can make more aluminium cans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes the required product will be very different to any of the relatively cheap scrap metal. This will require mixing different scrap metals with relatively expensive additions (known in the trade as &quot;master alloys&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I schedule production in the order A, B, C, D and apply the simplex algorithm to each product using Excel solver. The first time I do this, there will be a certain number of tonnes of each type of scrap. The algorithm will calculate the cheapest mixture of scrap and master alloys to make product A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I do the same for product B, I have to deduct the quantities of scrap and master alloys used whilst making product A (I can't use the same material twice!). I clearly have to do the same when I make product C, then product D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, my material constraints change each time I make a product. And the sequence of material constraints will be different for each production sequence. The sequence A, B,C, D will give a different order of material constraints to the order D, C, B, A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that I can find the cheapest &lt;strong&gt;total&lt;/strong&gt; cost simply by aggregating the whole day's production into one huge block of metal and running Excel solver on that. However, that doesn't tell me how to achieve that minimum cost in a real production schedule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have N products to make in any given day. Is there a way of finding the globally cheapest production schedule order without trying out all N! permuatations?&lt;/p&gt;&#xA;" OwnerUserId="20161" LastActivityDate="2016-04-25T20:36:30.887" Title="I'm using linear programming for production planning. Does the order in which I make products affect the cost?" Tags="|linear-programming|complexity|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23829" PostTypeId="1" CreationDate="2016-04-25T20:59:49.720" Score="1" ViewCount="100" Body="&lt;p&gt;I have to vectors $X_1$ and $X_2$ with 3 dimensional points $p_i$ and $p_j$ contained. As long as $X_1$ is not empty, I want to find the closest pair $p_i$ and $p_j$. The point $p_i$ of this pair I need to remove from $X_1$ and add to $X_2$. To make the following analysis easier, both $X_1$ and $X_2$ contain $n$ points. The lower border for the closest pair problem is $O(n(log(n)))$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the closest pair I utilize at the moment a search structure (Octree, as I am only interested in expected/average run-time). With the Octree containing the complete $X_2$ vector, I can find the closest neighbor for each Point in $X_1$ in $O(log(n))$. So I iterate through $X_1$ $n$ times, leading to $O(n(log(n)))$ for 1 closest pair. I can remove the point from $X_1$ and add to the Octree without effecting the run-time. But I have to do this $n$ times (until $X_1$ is empty), leading to $O(n^2(log(n)))$ in total. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Any solution which is not perfect but better than this, if no answer to (b) can be given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) The theoretical lower complexity border of this problem, preferably with the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jan&lt;/p&gt;&#xA;" OwnerUserId="20162" LastEditorUserId="1804" LastEditDate="2016-04-25T21:37:41.783" LastActivityDate="2016-10-30T15:35:00.060" Title="Expected runtime complexity of repeated closest Point Pair search" Tags="|complexity|closest-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23830" PostTypeId="2" ParentId="23827" CreationDate="2016-04-25T21:10:35.217" Score="2" Body="&lt;p&gt;Both structures and cell arrays are the most flexible data types &lt;em&gt;built&lt;/em&gt; into matlab (as opposed to creating your own class), and that flexibility comes with a price - physical memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not near a matlab-enabled machine, otherwise I would show you some examples; however, to get an idea of what I mean, create some data in matlab and then enter 'whos'. This will show you the memory usage of each data type. Cell arrays and structures are way more memory hungry than arrays, simply because you can combine different days types into a single variable (strings, booleans, numerics, etc.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to your question between cell arrays and structures, this is more of a question of preference in terms of writing, editing, and later understanding your code. I like structures because naming variable fields is easier to understand when multiple people are working on a project. Cell arrays take slightly more afterthought, because everything is based on a numeric index: what was cellData{:,2} again? It's easier with a structure: structData.address(:), oh ya those were addresses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a firm believer that matlab should be used so that it's as easy as possible to read and write. With bigger and more complicated projects, matlab itself becomes a bottleneck, and it would be better to write your code in another faster language. &lt;/p&gt;&#xA;" OwnerUserId="19791" LastActivityDate="2016-04-25T21:10:35.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23831" PostTypeId="2" ParentId="23824" CreationDate="2016-04-25T21:10:44.800" Score="0" Body="&lt;p&gt;Provided the length $N$ of the vectors is large enough to merit it, this is a perfect problem for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Discrete_Fourier_transform&quot; rel=&quot;nofollow&quot;&gt;discrete Fourier transform&lt;/a&gt;. The DFT of a vector $x$ is the vector $\mathcal F x$ with entries&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(\mathcal F x)_k = \sum_{n = 0}^{N - 1}x_ne^{-2\pi ikn/N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $P_nx$ is the operator which cyclically permutes the vector $x$ by $n$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(P_nx)_k = x_{(k + n)\mod N}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then the relation between cyclic permutation and Fourier transform is very simple:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(\mathcal F P_n x)_k = e^{2\pi ikn/N}(\mathcal F x)_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you've computed the DFT of two vectors, with this relation, you can check in $\mathcal O(N)$ time whether they are cyclic permutations of each other by checking whether their DFTs are related according to the last equation. The sum formula for the DFT suggests that it can only be computed in $\mathcal O(N^2)$ time, but by using some cleverness, the DFT can be computed in $\mathcal O(N\log N)$ time; an algorithm to do this is called a fast Fourier transform or FFT. There is probably an implementation of FFTs in whatever language you're using.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-04-25T21:10:44.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23832" PostTypeId="2" ParentId="23827" CreationDate="2016-04-26T00:29:39.093" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/matlab/matlab_prog/cell-vs-struct-arrays.html&quot; rel=&quot;nofollow&quot;&gt;As noted here&lt;/a&gt;, cell arrays and structs are both general enough to do the same things. However, one you access numerically (cell arrays) whereas the other you access fields by name (structs). In general, you choose the one that makes sense for readability. For example, if you want to loop through a bunch of things, probably put it in a cell array since the numeric indexing is convenient (though you can use structfun). If you want to store heterogeneous data in a convenient package to recall by name (such as different size matrices), then store it as a struct.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-04-26T00:29:39.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23833" PostTypeId="1" AcceptedAnswerId="23858" CreationDate="2016-04-26T05:19:48.290" Score="1" ViewCount="1310" Body="&lt;p&gt;I have the following simple nonlinear equations with two unknowns only:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\{&#xA;\begin{array}{c}&#xA; \int_1^2{\dfrac{ e^{{a_1} x+{a_2} x^3}}{1+x^2}} \, dx=1 \\[13pt]&#xA; \int_1^2{ x^2 e^{{a_1} x+{a_2} x^3}} \, dx=\dfrac{1}3 \\&#xA;\end{array}&#xA;\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand it is almost impossible to obtain all the possible numerical solutions in the complex field $\mathbb{C}$. So any numerical complex solution per a specific given initial value with good enough accuracy would be OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By symbolic-numerical computation, it is easy to obtain one solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\{\quad&#xA;\begin{array}{ccrcr}&#xA; {a_1}&amp;amp;= &amp;amp; 2.083619981922075 &amp;amp;-&amp;amp; 2.163052112144277\; i \\&#xA; {a_2}&amp;amp;= &amp;amp; -0.144311264409679 &amp;amp;+&amp;amp; 1.206590594556891\; i \\&#xA;\end{array}&#xA;\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I found it  difficult to handle such a problem using the built-in integral functions in Matlab since such  &lt;code&gt;integral&lt;/code&gt; and &lt;code&gt;quadgk&lt;/code&gt; functions as accept &lt;code&gt;function handle&lt;/code&gt;-form only integrands. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My try is first composing the nonlinear equations into user defined functions, then solve it per iteration algorithm (e.g. Newton's method) with a specific initial value), but when I tried to pass symbolic variables or global variables $a_1$, $a_2$, problems (error messages) would happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though symbolic computation is possible to obtain a solution, it is too slow especially for more general nonlinear equations. Since the solution I need is only numerical one, I wonder: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to solve such a problem by Matlab without using the symbolic computation? If the integral function has to accept symbolics $a_1$, $a_2$, how to handle it in Matlab (or C++)?&lt;/p&gt;&#xA;" OwnerUserId="20166" LastEditorUserId="20166" LastEditDate="2016-04-26T08:46:42.560" LastActivityDate="2016-04-28T00:35:19.737" Title="How to solve the problem without using symbolic computation" Tags="|matlab|nonlinear-equations|iterative-method|integration|symbolic-computation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23834" PostTypeId="2" ParentId="23822" CreationDate="2016-04-26T05:35:12.640" Score="8" Body="&lt;p&gt;If $A$ and $B$ are real symmetric, then $A=B+AB$ if and only if the product $AB$ is also real symmetric. In turn, $AB=BA$ holds if any only if $A$ and $B$ share a common eigendecomposition. This latter statement gives a recipe for computing $A$ given $B$.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Given $B$, compute its eigendecomposition $B=V\Lambda V^T$, in which $V$ is the orthonormal set of of eigenvectors, and $\Lambda = \mathrm{diag}(\lambda_1,\dots,\lambda_n)$ is the diagonal matrix of real eigenvalues.&lt;/li&gt;&#xA;&lt;li&gt;Compute the diagonal matrix $S= \mathrm{diag}(s_1,\dots,s_n)$ in which each $s_k$ is defined with respect to the corresponding $\lambda_k$ as in $$s_k=\frac{\lambda_k}{1-\lambda_k}\text{ for all }k\in\{1,\ldots,n\}.$$&lt;/li&gt;&#xA;&lt;li&gt;Output $A=VSV^T$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In fact, this is the most numerically stable procedure possible for solving the desired equation. The computed $A$ is exactly real symmetric by construction. Also, limiting all matrix-matrix operations to orthogonal transformations minimizes the numerical error that are propagated from the problem data to the computed solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, we will none-the-less encounter ill-conditioning if the eigenvalues of $B$ are close to 1. But this latter phenomenon is physical, and reflects the underlying ill-conditioning of the equation to be solved. In this latter case, we may reformulate the problem and solve it in a least-squares sense, using essentially the same steps as the ones presented above.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2016-04-26T14:24:55.350" LastActivityDate="2016-04-26T14:24:55.350" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23835" PostTypeId="2" ParentId="23822" CreationDate="2016-04-26T10:32:29.857" Score="4" Body="&lt;p&gt;Your equation is a &lt;em&gt;Sylvester&lt;/em&gt; equation&#xA;$$&#xA;AB - A = -B,&#xA;$$&#xA;for which there are numerous solution algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would definitely recommend the &lt;em&gt;Bartels-Stewart&lt;/em&gt; algorithm that bases on real Schur decompositions. These can be computed for general square matrices in a stable fashion and avoiding complex arithmetics; cf. Chapter 16.1 in the book &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; by N. Higham. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, all algorithms assume that the equation is uniquely solvable, which in your case is equivalent to that $B$ does not have $1$ as eigenvalue. For this case, I have never come across any theory that may have helped to build a numerical algorithm on it. (And I have done quite some research on that)&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-04-26T10:32:29.857" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23837" PostTypeId="2" ParentId="83" CreationDate="2016-04-26T13:11:32.040" Score="2" Body="&lt;p&gt;How about CMA-ES? It has Python bindings and is well suited to nonconvex, nonlinear optimization problems and I've used it quite a bit:&#xA;&lt;a href=&quot;https://www.lri.fr/~hansen/cmaesintro.html&quot; rel=&quot;nofollow&quot;&gt;https://www.lri.fr/~hansen/cmaesintro.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Installation through pip:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install cma&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's some sample code from their website:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import cma&#xA;help(cma)  # &quot;this&quot; help message, use cma? in ipython&#xA;help(cma.fmin)&#xA;help(cma.CMAEvolutionStrategy)&#xA;help(cma.CMAOptions)&#xA;cma.CMAOptions('tol')  # display 'tolerance' termination options&#xA;cma.CMAOptions('verb') # display verbosity options&#xA;res = cma.fmin(cma.Fcts.tablet, 15 * [1], 1)&#xA;res[0]  # best evaluated solution&#xA;res[5]  # mean solution, presumably better with noise&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20172" LastActivityDate="2016-04-26T13:11:32.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23840" PostTypeId="1" CreationDate="2016-04-26T15:03:23.353" Score="1" ViewCount="152" Body="&lt;p&gt;I'm trying to minimize the unconstrained scalar sum of a quadratic convex function (to which a convex optimizer is readily applied) and a non-linear and non-convex function which is differentiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;General purpose non-linear optimizers are not providing adequate results, which I assume is due to the high dimensionality of my problem (&gt;=10000), so I was wondering if there was a better way to go about minimizing my function given that it is decomposable into a convex and differentiable non convex part.&lt;/p&gt;&#xA;" OwnerUserId="20177" LastActivityDate="2016-04-26T15:03:23.353" Title="Minimization of the sum of convex function and non-linear non convex function" Tags="|nonlinear-programming|nonconvex|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23842" PostTypeId="1" AcceptedAnswerId="23873" CreationDate="2016-04-26T15:25:58.167" Score="2" ViewCount="562" Body="&lt;p&gt;I am struggling finding pseudo Code for a non-linear fit of the following function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y = a\, x^b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Package NLS in R does perform well, but utilizing external software is not practicable in my program anymore. I would like to avoid this by implementing a short class by myself. Literature links including the solution are also welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I already have implemented the log transform with linear square fit. That is not the solution I am looking for. I need to implement NLS fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update 2, giving reason why the log transformed method is not sufficient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My original data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nGAMj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nGAMj.png&quot; alt=&quot;Allometry function related to forestry&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The log transformed data with a linear model line in red, one can see that the log transformed residuals are a heavy influence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OfnGM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OfnGM.png&quot; alt=&quot;And its log transformed counterpart, with fitted line in red&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original data again, the red line is the power function with parameters derived from the log transformed method, the blue line is the power function predicted with package NLS2 in R: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/p5oqx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p5oqx.png&quot; alt=&quot;The original data with power function derived from the log transform method in red and the better function derived with R-package NLS2 in blue&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly the log-transformed residuals have too much influence, the NLS method forces the prediction on the stable part of the model.&lt;/p&gt;&#xA;" OwnerUserId="20162" LastEditorUserId="20162" LastEditDate="2016-04-29T17:43:15.813" LastActivityDate="2016-08-13T06:46:48.783" Title="Pseudo Code for non linear power function fit needed" Tags="|nonlinear-equations|curve-fitting|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23843" PostTypeId="2" ParentId="23765" CreationDate="2016-04-26T15:53:10.177" Score="2" Body="&lt;p&gt;MATLAB's ODE solvers can use what are known as &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/odeset.html&quot; rel=&quot;nofollow&quot;&gt;Event functions&lt;/a&gt; to achieve this. I also link &lt;a href=&quot;http://www.mathworks.com/matlabcentral/answers/19910-event-function-for-ode-solver&quot; rel=&quot;nofollow&quot;&gt;someone who posted a good quick example to show how to use it&lt;/a&gt; that is close to your use-case.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-04-26T15:53:10.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23845" PostTypeId="1" AcceptedAnswerId="24013" CreationDate="2016-04-26T17:06:28.743" Score="0" ViewCount="309" Body="&lt;p&gt;Please refer to &lt;a href=&quot;http://stanford.edu/~boyd/papers/pdf/admm_distr_stats.pdf&quot; rel=&quot;nofollow&quot;&gt;Boyd et al.'s convergence analysis of ADMM&lt;/a&gt; (Chapter 3 and Appendix A).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Why do we need $f$ and $g$ to be convex?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see the need of this assumption. &lt;strong&gt;If the convexity assumption is removed, the analysis is still valid.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2016-05-19T15:01:44.997" Title="On Boyd et al.'s convergence analysis of ADMM: Why do we need the convexity assumption?" Tags="|optimization|convex-optimization|admm|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23846" PostTypeId="2" ParentId="21170" CreationDate="2016-04-26T17:56:24.093" Score="2" Body="&lt;p&gt;The trial wavefunction, $\exp(-1.2r)$,  does not respect the cusp condition - the derivatives of the wavefunction need to cancel the $1/r$ Coulomb term.   Without a correct cusp condition, the local energy diverges at the origin, which is likely causing convergence problems with the value of the integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the leading term should be $\exp(-Zr)$ (where $Z$ is the nuclear charge).   To create a trial wavefunction for hydrogen with a correct cusp, try adding higher powers of $r$, $\exp(-r + a r^2)$, where $a$ is the variational parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Monte Carlo integration also has an error - both accepted and rejected moves should be included in the sum (this code only includes accepted moves).  Otherwise individual energy values have the wrong weight in the final integral. And the final value of energy (E1) should be divided by the total number in the sum (10000), not just the number of acceptances.&lt;/p&gt;&#xA;" OwnerUserId="20181" LastActivityDate="2016-04-26T17:56:24.093" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23847" PostTypeId="2" ParentId="23845" CreationDate="2016-04-26T18:04:57.300" Score="6" Body="&lt;p&gt;To consolidate my comments: The proof of inequality (A.2) rests on the fact that the solutions $x^{k+1}$ and $z^{k+1}$ of the substeps (3.2) and (3.3), respectively, are &lt;em&gt;global&lt;/em&gt; minimizers, so you can compare them to the saddle points $x^*$ and $z^*$, respectively. (Here, Boyd uses the subdifferential characterization, but I'm sure you can also derive it using the optimality directly.) But this is only guaranteed if the subproblems are convex (where any minimizer is a global minimizer), which is the case if $f$ and $g$ are convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's not saying ADMM &lt;em&gt;can't&lt;/em&gt; work for nonconvex problems -- indeed, this is currently a very hot topic in nonsmooth optimization and image processing -- but the fact that you can only work with local minimizers makes the analysis (and practice) significantly more involved.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-04-26T18:04:57.300" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23848" PostTypeId="2" ParentId="23762" CreationDate="2016-04-26T18:56:52.100" Score="1" Body="&lt;p&gt;The residual should just be the equation you wrote where all terms are moved to one side (plus or minus a negative sign), you can compute this after solving for you unknowns (which seems to be implicit for the second step as you indicated). There may be additional storage required to compute the residual from current and previous time steps since you need information at both time levels, but it likely won't be too much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try to integrate this computation into your code, which may help in terms of performance, but I prefer to apply this kind of computation after computing the solution and only doing so every 100 or so time steps.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-04-26T18:56:52.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23849" PostTypeId="1" CreationDate="2016-04-26T20:01:19.557" Score="4" ViewCount="1408" Body="&lt;p&gt;Given a near singular covariance matrix, the standard method of 'fixing' it seems to be to add a small damping coefficient $c&amp;gt;0$ to the diagonal, which serves to bump all the eigenvalues up by this amount.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to determine the optimal value of this damping coefficient without resorting to some iterative procedure where I jump back and forth between inverting the matrix to look for infinities and NaNs and making $c$ increasingly small?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could look at the minimum eigenvalue fairly cheaply, but it seems like the correct choice for $c$ would still also depend on both the entries of the matrix as well as its size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, it's more important that my algorithm always works than it is that the perturbation is as small as possible, so I'm willing to accept more error for the sake of a procedure with some sort of guarantee that it will always work.&lt;/p&gt;&#xA;" OwnerUserId="6849" LastActivityDate="2016-04-26T20:01:19.557" Title="Fixing a near singular covariance matrix" Tags="|linear-algebra|numerical-analysis|matrix|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="23850" PostTypeId="2" ParentId="23357" CreationDate="2016-04-26T21:30:12.477" Score="2" Body="&lt;p&gt;Here is a really good book on implementing these equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analysis and Simulation of Semiconductor Devices&#xA;by S. Selberherr&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before solving the full set of drift diffusion equations, an initial guess would be solving only the Poisson equation at equilibrium.  This is also discussed in the book.&lt;/p&gt;&#xA;" OwnerUserId="7681" LastActivityDate="2016-04-26T21:30:12.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23851" PostTypeId="1" CreationDate="2016-04-27T03:56:26.207" Score="2" ViewCount="50" Body="&lt;p&gt;I took a FFT of sound in a box generated by a frequency sweep over a range of frequencies, and have an array of frequencies and their corresponding FFT amplitudes.  According to models for the acoustic waves, certain frequencies should be resonant frequencies, which certainly shows in the data as Gaussian-like peaks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there might be some program or algorithm that can quickly determine the peaks in the spectrum as well as their standard deviations (if assumed to be Gaussian).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have a brute force way of doing, that is to check each point and their values on Igor Pro, and finding the value of $f$ such that the spectrum amplitude, $A = A_0 e^{-1/2}$ where $A_0$ is the peak.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is due to the fact that the equation of the Gaussian is&#xA;$$&#xA;A_0 e^{-\frac{f^2}{2\sigma_f}}&#xA;$$&#xA;where $\sigma_f$ is the standard deviation in $f$, so that at $f=\sigma_f$, the exponential is $e^{-1/2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there was a way to automate this.  Finding the peaks themselves would probably involve some sort of discrete derivative to find the extrema (which would have to ignore turning points due to noise fluctuations).  Then perhaps I could compare values over a range about the peaks, but of course the comparisons won't be exact equality, but instead finding the closest values, since I am working with discrete data.&lt;/p&gt;&#xA;" OwnerUserId="20186" LastEditorUserId="9667" LastEditDate="2016-04-27T05:25:48.667" LastActivityDate="2016-04-27T05:25:48.667" Title="Finding errors in frequency from a Fast Fourier Transform from Gaussian fitting" Tags="|algorithms|fourier-analysis|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23852" PostTypeId="1" AcceptedAnswerId="26325" CreationDate="2016-04-27T04:26:13.783" Score="10" ViewCount="971" Body="&lt;p&gt;I'm working on a project where I have two adv-diff coupled domains through their respective source terms (one domain adds mass, the other subtracts mass). For brevity, I'm modeling them in steady state. The equations are your standard advection-diffusion transport equation with a source term look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial c_1}{\partial t} = 0 = \mathcal{F}_1 + \mathcal{Q}_1(c_1,c_2) \\&#xA;\frac{\partial c_2}{\partial t} = 0 = \mathcal{F}_2 + \mathcal{Q}_2(c_1,c_2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathcal{F}_i$ is diffusive and advective flux for species $i$, and $\mathcal{Q}_i$ is the source term for species $i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been able to write a solver for my problem using the Newton-Raphson method, and have completely coupled the two domains using a block mass matrix, ie:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F_{coupled} = &#xA;\left[\begin{array}{c c}&#xA;A_1 &amp;amp; 0 \\&#xA;0 &amp;amp; A_2 \\&#xA;\end{array}\right]\underbrace{ \left[\begin{array}{c}&#xA;c_{1,i} \\&#xA;c_{2,i} \\&#xA;\end{array}\right] }_{x_i}&#xA; - \left[\begin{array}{c}&#xA;b_1(c_{1,i}, c_{2,i}) \\&#xA;b_2(c_{1,i}, c_{2,i}) \\&#xA;\end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term $F_{coupled}$ is used to determine the Jacobian matrix and update both $c_1$ and $c_2$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{J}(x_i) \left[x_{i+1} - x_i \right] = -\mathcal{F}_{coupled}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x_{i+1} = x_i - \left(\mathcal{J}(x_i)\right)^{-1} \mathcal{F}_{coupled}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To speed things up, I don't calculate the Jacobian every iteration - right now I'm playing with every five iterations, which has seemed to work well enough and keep the solution steady.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is: I'm going to be moving to a larger system where both domains are in 2D/2.5D, and calculating the Jacobian matrix is going to quickly deplete my available computer resources. I'm building this model to be used in an optimization setting later on, so I also can't be behind the wheel at every iteration tuning the damping factor, etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I right to be looking elsewhere for a more robust and algorithm for my problem, or is this is as good as it gets? I've looked a bit into Quasi-linearization, but am not sure how applicable it is w.r.t. to my system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other slick algorithms that I may have missed that can solve a system of nonlinear equations without resorting to re-calculating the Jacobian as offen?&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="9667" LastEditDate="2017-03-04T15:57:26.073" LastActivityDate="2018-06-16T21:16:57.473" Title="Methods of solving non-linear advection-diffusion systems beyond Newton-Raphson?" Tags="|nonlinear-equations|newton-method|advection-diffusion|coupling|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="23854" PostTypeId="1" CreationDate="2016-04-27T11:06:41.737" Score="7" ViewCount="290" Body="&lt;p&gt;I'm searching for robust smoothers for geometric multigrids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By robust I mean:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Effective for high order approximations (say spectral element, spectral Discontinuous Galerkin),&lt;/li&gt;&#xA;&lt;li&gt;Parallel (suitable for co-processors),&lt;/li&gt;&#xA;&lt;li&gt;Effective for heterogeneity and anisotropy problems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From what I can gather, Schwarz type smoothers may be promising &lt;a href=&quot;http://www.mcs.anl.gov/~fischer/pubhtml/IP109-frame.pdf&quot; rel=&quot;noreferrer&quot;&gt;(Fischer et al)&lt;/a&gt;; and block/line/ plane, and ILU smoothers are also recommended &lt;a href=&quot;https://books.google.com/books?id=-og1wD-Nx_wC&quot; rel=&quot;noreferrer&quot;&gt;(Trottenberg et al)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other state-of-the-art smoothers I should consider?&lt;/p&gt;&#xA;" OwnerUserId="20004" LastEditorUserId="19501" LastEditDate="2016-04-27T18:41:37.987" LastActivityDate="2021-11-30T13:02:10.227" Title="Robust smoothers for geometric multigrid" Tags="|linear-algebra|linear-solver|iterative-method|multigrid|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23858" PostTypeId="2" ParentId="23833" CreationDate="2016-04-28T00:35:19.737" Score="8" Body="&lt;p&gt;You can solve this numerically in Python without symbolic computation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;from __future__ import print_function, division&#xA;import numpy as np&#xA;from numpy import exp&#xA;from scipy.integrate import quad&#xA;from scipy.optimize import root&#xA;&#xA;def f1(a1, a2, x):&#xA;    return exp(a1 * x + a2 * x * x * x) / (1 + x * x)&#xA;&#xA;def f2(a1, a2, x):&#xA;    return exp(a1 * x + a2 * x * x * x) * x&#xA;&#xA;def g(a1, a2, f):&#xA;    def r(x):&#xA;        return f(a1, a2, x).real&#xA;    def i(x):&#xA;        return f(a1, a2, x).imag&#xA;    y = quad(r, 1, 2)[0] + quad(i, 1, 2)[0] * 1j&#xA;    return y&#xA;&#xA;def func(Z):&#xA;    a1 = Z[0] + Z[1] * 1j&#xA;    a2 = Z[2] + Z[3] * 1j&#xA;    y1 = g(a1, a2, f1) - 1&#xA;    y2 = g(a1, a2, f2) - 1/3&#xA;    return np.array([y1.real, y1.imag, y2.real, y2.imag])&#xA;&#xA;X0 = np.array([2.0, -2.0, -0.1, 1.0])&#xA;print(root(func, X0, tol=1e-12))&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;fjac: array([[-0.30890916,  0.05007557,  0.09254457,  0.94525291],&#xA;       [-0.12460792, -0.41710616, -0.89760745,  0.06925448],&#xA;       [ 0.41309815, -0.83196711,  0.34050112,  0.14573825],&#xA;       [ 0.84758358,  0.36241635, -0.26418811,  0.28365668]])&#xA;     fun: array([ -2.88657986e-15,  -2.49800181e-15,  -1.89293026e-14,&#xA;        -2.43138842e-14])&#xA; message: 'The solution converged.'&#xA;    nfev: 24&#xA;     qtf: array([ -9.32740208e-12,   6.42466839e-12,  -3.53520661e-12,&#xA;        -1.78427395e-12])&#xA;       r: array([ -3.43272029,   0.72043771, -14.00467579, -11.98268077,&#xA;        -2.65005807,  13.29034808, -12.53526771,  -7.04373163,&#xA;         2.58188876,  -7.03891799])&#xA;  status: 1&#xA; success: True&#xA;       x: array([ 2.08361998, -2.16305211, -0.14431126,  1.20659059])&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that I've changed the definition of the second function to use an &lt;code&gt;x&lt;/code&gt; term instead of an &lt;code&gt;x^2&lt;/code&gt; term so that it matches your solution :) I guess this was a typo in your question.&lt;/p&gt;&#xA;" OwnerUserId="20197" LastActivityDate="2016-04-28T00:35:19.737" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23860" PostTypeId="1" AcceptedAnswerId="23875" CreationDate="2016-04-28T04:14:34.523" Score="0" ViewCount="516" Body="&lt;p&gt;I have a function &lt;code&gt;f(x,k1,k2)&lt;/code&gt; and I am trying to minimize it over &lt;code&gt;x&lt;/code&gt; for different values of &lt;code&gt;(k1,k2)&lt;/code&gt; on a 2d grid like so&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i,k1 in enumerate(np.logspace(-3.3,-1,20)):&#xA;    for j,k2 in enumerate(np.logspace(-3.3,-1,20)):&#xA;        if j==0:&#xA;            initial_guess = best_x_0&#xA;        else:&#xA;            initial_guess = best_x&#xA;        res = minimize(f,initial_guess,args=(k1,k2),bounds=((0.001,1),),tol=0.01,method='L-BFGS-B')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For given &lt;code&gt;(k1,k2)&lt;/code&gt; this is a 1d problem and the function is relatively well behaved with only 1 minimum. &#xA;However, evaluating it is very costly, ranging from a few seconds to up to about 10 minutes depending on the parameters. &#xA;Obviously, there must be a more efficient way of solving this than treating it as many independent minimization problems. If the points &lt;code&gt;(k1,k2)&lt;/code&gt; are close enough, and if I already have the minimum for one point, the minimum for the points around it should not be very different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked at what Scipy has to offer but I did not find anything ideal for this purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The functions in &lt;code&gt;scipy.optimize.minimize_scalar&lt;/code&gt; do not require an initial guess and so I dont know how to take advantage of the 2d grid structure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also ran into issues using the &lt;code&gt;'L-BFGS-B'&lt;/code&gt; method of &lt;code&gt;scipy.optimize.minimize&lt;/code&gt;. &#xA;For example, using &lt;code&gt;k1=k2=0.000501187233627&lt;/code&gt; and an initial guess of &lt;code&gt;x=0.02&lt;/code&gt; it converges to &lt;code&gt;x=0.022114610909&lt;/code&gt; in 8 function evaluations. But if I use the same initial guess with &lt;code&gt;k1=k2= np.logspace(-3.3,-1,20)[0] = 0.000501187233627272527534957103&lt;/code&gt;, when clearly there should not be any difference it get stuck on about &lt;code&gt;x=0.01999926424&lt;/code&gt; performing useless evaluations such as &lt;code&gt;x=0.019966155,0.01999995105,0.0199951834&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to accomplish this?&lt;/p&gt;&#xA;" OwnerUserId="4099" LastActivityDate="2017-02-28T11:12:03.493" Title="Repeated 1d minimization with similar parameters (scipy)" Tags="|optimization|python|convex-optimization|scipy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23861" PostTypeId="1" CreationDate="2016-04-28T06:09:58.567" Score="0" ViewCount="329" Body="&lt;p&gt;I have a numerical code which is MPI+OpenMP (hybrid) parallelized and an available computational resource of 32 nodes with 16 cores on each node. The code has been tested for MPI scalability up to 16 cores on different nodes and OpenMP scalability on 16 cores on a single node. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I assume the hybrid parallelization to be scalable on $32\times 16$ cores?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: MPI+OpenMP is done in such a way that MPI processes are launched at the beginning itself and shared memory parallelization is done within each process using a few PARALLEL DO loops.&lt;/p&gt;&#xA;" OwnerUserId="19254" LastEditorUserId="19254" LastEditDate="2016-04-30T06:07:27.037" LastActivityDate="2016-04-30T06:07:27.037" Title="MPI+OpenMP Scalability" Tags="|parallel-computing|mpi|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23863" PostTypeId="1" CreationDate="2016-04-28T11:00:43.533" Score="2" ViewCount="169" Body="&lt;p&gt;I need to solve, as mentioned, Schrodinger's equation in a Bunimovich stadium-shaped infinite potential well with Dirichlet BC Numerically (this isn't possible analytically). In order to do so, I need to define the discrete Hamiltonian matrix of the system. Since the shape isn't defined well in any coordinates system, I have no idea how to do so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally (as I did with a square well), I'd define the size of the well and then solve 1D TISE for each dimension using MATLAB eig function. Can it be solved similarly? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20205" LastEditorUserId="20205" LastEditDate="2016-04-28T11:08:04.213" LastActivityDate="2016-04-28T11:08:04.213" Title="Solving Schrodinger's Equation Numerically in a Bunimovich Stadium" Tags="|computational-physics|quantum-mechanics|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="23864" PostTypeId="1" CreationDate="2016-04-28T13:35:14.087" Score="5" ViewCount="1639" Body="&lt;p&gt;I want to elaborate on a statement I read in Acton's &quot;Numerical methods that work&quot;, paragraph &quot;Exponential fitting&quot;, page 252.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Computationally we are being asked to fit only the parameters $A$ and $B$ in the equation $y = A\exp (-at) + B\exp (-bt)$&#xA;  when we have observed a sample at several times to produce&#xA;  a set of $(t_i, y_i)$ pairs. It is a simple least-squares fit that&#xA;  generally requires only a desk calculator.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;OK I get this part. But then the author goes on :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unfortunately there is a companion problem that looks&#xA;  only slightly more complicated — until you try it! We again&#xA;  have $(t_i, y_i)$ readings from a radioactive sample, but the&#xA;  decaying materials are not known, hence the decay rates $a$&#xA;  and $b$ must also be fitted. The answer to this problem lies&#xA;  in the chemical rather than the computer laboratory, and the&#xA;  sooner the hopeful innocent can be sent there and away&#xA;  from the computer room, the better off everyone will be.&#xA;  For it is well known that &lt;strong&gt;an exponential equation of this&#xA;  type in which all four parameters are to be fitted is extremely&#xA;  ill conditioned&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(Emphasis is mine.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't understand this statement, so I started to think how I would have proceed if faced with the same problem. Considering the simplicity of this function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ y = A\exp (-at) + B\exp (-bt) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;seen as a function of $(A,a,B,b)$, I would go straighforward to try some sort of gradient-based method because I can easily compute the partial derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I call $E(y)$ my error function, the problem resort to minimizing the quantity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ E(y) = \sum_{i=1}^{N} \left( y(t_i) - y_i \right) ^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again the partial derivatives with respect to $A$, $a$, $B$ or $b$ are easy to compute, and I don't see the problem coming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So where is the problem and what is it ?&lt;/p&gt;&#xA;" OwnerUserId="7784" LastEditorUserId="7784" LastEditDate="2016-04-28T15:15:07.423" LastActivityDate="2016-05-01T22:56:28.313" Title="Fitting with a linear combination of exponentials" Tags="|curve-fitting|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="23865" PostTypeId="2" ParentId="23861" CreationDate="2016-04-28T14:57:01.540" Score="6" Body="&lt;p&gt;No. You need to test it to that scale, especially if you have MPI calls within OpenMP regions.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-04-28T14:57:01.540" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23867" PostTypeId="1" CreationDate="2016-04-28T15:46:13.227" Score="7" ViewCount="17006" Body="&lt;p&gt;I am looking for a C++ library for Fast Fourier Transform (FFT) in high precision (e.g., using high precision real data types similar to &lt;code&gt;mpfr_t&lt;/code&gt; in &lt;a href=&quot;http://www.mpfr.org/mpfr-current/mpfr.html&quot; rel=&quot;noreferrer&quot;&gt;MPFR&lt;/a&gt; or &lt;code&gt;cpp_dec_float&lt;/code&gt; in &lt;a href=&quot;http://www.boost.org/doc/libs/master/libs/multiprecision/doc/html/boost_multiprecision/tut/floats.html&quot; rel=&quot;noreferrer&quot;&gt;BOOST&lt;/a&gt;). The preference is for open-source or, if not available, at least &quot;free for academic research&quot; libraries. I have tried standard precision optimized implementations such as FFTW in &lt;code&gt;long double&lt;/code&gt;, but that proved to be insufficient. An ideal solution would link to BOOST since I am using special function implementations from that library. &lt;/p&gt;&#xA;" OwnerUserId="20104" LastActivityDate="2016-04-29T15:30:28.193" Title="C++ libraries for Fast Fourier Transform in high precision" Tags="|c++|precision|fourier-transform|fftw|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23868" PostTypeId="2" ParentId="23861" CreationDate="2016-04-28T15:46:35.327" Score="2" Body="&lt;p&gt;As a general rule, I would test out the combination of cores and nodes, since scalability depends on so many factors: having enough to do on each node and then core, task / data parallelism in your problem, how well the code is architected to perform MPI communications without interrupting the openMP threads, etc. Most of the times one can avoid MPI calls from within openMP loops, and I have.&lt;/p&gt;&#xA;" OwnerUserId="20098" LastActivityDate="2016-04-28T15:46:35.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23869" PostTypeId="1" CreationDate="2016-04-28T18:08:08.767" Score="3" ViewCount="176" Body="&lt;p&gt;What would you recommend as an all-in-one cluster management software for a Beowulf cluster? We have about 7(most of them are 8 core machines and 1 is a 48 core machine) very good compute nodes in our research group and they are currently managed using Rocks and run Centos 6. We have been doing MPI-based computations, particularly, molecular simulations (Molecular Dynamics and Monte Carlo Simulations) reliably but we would now like to update the software. I suppose doing a system upgrade would break packages and reinstalling the cluster is a better option. I am wondering if we should use Rocks again or go for some other all-in-one solutions. We would want something that's easy to maintain (we all have experience with linux but not server management) and has LTS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my questions are: 1. Is there a Debian-based option like PelicanHPC that is good for deploying on a long term basis? 2. Is building a beowulf cluster based on Debian/Ubuntu/Centos 7 from scratch a good idea? &lt;/p&gt;&#xA;" OwnerUserId="3558" LastEditorUserId="3558" LastEditDate="2016-05-03T15:30:41.907" LastActivityDate="2016-05-03T15:30:41.907" Title="Beowulf Cluster For MD simulations" Tags="|molecular-dynamics|hpc|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23870" PostTypeId="2" ParentId="23864" CreationDate="2016-04-29T00:40:50.597" Score="5" Body="&lt;p&gt;The problem is ill-conditioned in the sense that very small changes to the $(y_{i},t_{i})$ data can lead to much larger relative changes in the best fitting parameters.  If you do the arithmetic in limited precision (and even IEEE double precision is can be inadequate in practice) then the precision of your floating point arithmetic may not be adequate to obtain a reasonably accurate solution to the nonlinear least squares problem.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-04-29T00:40:50.597" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23871" PostTypeId="2" ParentId="23867" CreationDate="2016-04-29T05:19:57.967" Score="1" Body="&lt;p&gt;ARPREC (&lt;a href=&quot;http://crd-legacy.lbl.gov/~dhbailey/mpdist/&quot; rel=&quot;nofollow&quot;&gt;http://crd-legacy.lbl.gov/~dhbailey/mpdist/&lt;/a&gt;) is an arbitrary-precision C++/Fortran package and contains an FFT implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apfloat (&lt;a href=&quot;http://www.apfloat.org/apfloat/&quot; rel=&quot;nofollow&quot;&gt;http://www.apfloat.org/apfloat/&lt;/a&gt;) is another C++ arbitrary-precision library which contains FFT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of the above packages have been included in the FFTW benchmark (&lt;a href=&quot;http://www.fftw.org/speed/&quot; rel=&quot;nofollow&quot;&gt;http://www.fftw.org/speed/&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2016-04-29T05:19:57.967" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23872" PostTypeId="1" CreationDate="2016-04-29T08:39:37.780" Score="3" ViewCount="151" Body="&lt;p&gt;In order to numerically solve the following differential equation:&#xA;\begin{equation}&#xA; \text{Fr}\{f\} := v(k)\dfrac{\partial f(z,k)}{\partial z} - F(z) \dfrac{\partial f(z,k)}{\partial k} = -\dfrac{f-f_0}{\tau}&#xA;\end{equation}&#xA;I have used the finite volume method, and have discretized the left hand side to:&#xA;\begin{equation}&#xA;\text{Fr}'\{f\} :=&#xA; v(k_j)\Delta k  \Bigg[   f(z_{i+},k_j) - f(z_{i-},k_j) \Bigg] - &#xA; F(z_i)\Delta z \Bigg[ f(z_{i},k_{j+})   -   f(z_{i},k_{j-}) \Bigg] &#xA;\end{equation} &#xA;for every box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The so-called flux averaging approximation,&#xA;\begin{align}&#xA; &amp;amp; f(z_{i+},k_j) = \dfrac{f(z_i,k_j)+f(z_{i+1},k_j)}{2} \\&#xA; &amp;amp; f(z_{i-},k_j) = \dfrac{f(z_i,k_j)+f(z_{i-1},k_j)}{2}&#xA;\end{align}&#xA;will lead to instability especially if the flux term is weak.&#xA;This could be avoided by applying the upwind scheme. In this case:&#xA;\begin{equation}&#xA; f(z_{i+},k_j) = &#xA; \begin{cases}&#xA;f(z_{i},k_j) &amp;amp; v(k_j)&amp;gt;0 \\&#xA;f(z_{i+1},k_j) &amp;amp; v(k_j)&amp;lt;0&#xA;\end{cases}&#xA;\end{equation}&#xA;\begin{equation}&#xA; f(z_{i-},k_j) = &#xA; \begin{cases}&#xA;f(z_{i-1},k_j) &amp;amp; v(k_j)&amp;gt;0 \\&#xA;f(z_{i},k_j) &amp;amp; v(k_j)&amp;lt;0&#xA;\end{cases}&#xA;\end{equation}&#xA;I have implemented the above upwinding method, and my results seem accurate for very small values of $F(z_i)$ throughout the system.&#xA;However, when $F(z_i)$ gets large, the obtained results lose accuracy and deviate from the correct result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason to this deviation, I guess, is that the analytical solution to $\text{Fr}\{f\}=0$ is an exponential function. Since the equations are linearly discretized, the discretization cannot follow the large exponential changes accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any idea for increasing the accuracy of the above discretization while holding on to unconditional stability?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastEditorUserId="123" LastEditDate="2016-05-03T07:06:17.190" LastActivityDate="2016-05-03T07:06:17.190" Title="High order unconditionally stable discretization for a scalar hyperbolic PDE" Tags="|pde|finite-volume|discretization|hyperbolic-pde|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23873" PostTypeId="2" ParentId="23842" CreationDate="2016-04-29T08:50:49.313" Score="5" Body="&lt;p&gt;Chapter &lt;strong&gt;10&lt;/strong&gt; of Numerical Analysis (Richard L. Burden, J. Douglas Faires) gives good readable pseudo code for Newtons method. The start parameters are taken from the solution of the linear problem of the log transformed data. (see comment of Christian Clason giving a&#xA;&lt;a href=&quot;http://mathworld.wolfram.com/LeastSquaresFittingPowerLaw.html&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt;&#xA;(Linear Solution, Mathworld Wolfram) )&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while not reached convergence criteria&#xA;      calucate vector of residuals F(x) and Jacobian matrix J(x)&#xA;      solve J(x)y = -F(x)&#xA;      set x = x + y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here $x = a,b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-F(x)$ is for $n$ points a vector with length $n$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the $ith$ element is $- a\, x_i^b +  y_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the Jacobian $J(x)$ is a $n,2$ Matrix with the $ith$ row:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J_{i,1} = x_i^b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J_{i,2} = a\,log(x_i)\,x_i^b $&lt;/p&gt;&#xA;" OwnerUserId="20162" LastEditorUserId="20162" LastEditDate="2016-08-13T06:46:48.783" LastActivityDate="2016-08-13T06:46:48.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23874" PostTypeId="2" ParentId="23867" CreationDate="2016-04-29T15:30:28.193" Score="3" Body="&lt;p&gt;You might want to take a look at the Eigen C++ matrix class library. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;http://eigen.tuxfamily.org/index.php?title=Main_Page&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an FFT class in the unsupported section of the library but my impression is that it is relatively mature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a snippet of code using this FFT class with the BOOST multi-precision classes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;complex&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;Eigen/Core&amp;gt;&#xA;#include &amp;lt;unsupported/Eigen/FFT&amp;gt;&#xA;&#xA;#include &amp;lt;boost/multiprecision/cpp_dec_float.hpp&amp;gt;&#xA;&#xA;using boost::multiprecision::cpp_dec_float_50;&#xA;&#xA;using std::cout;&#xA;using std::endl;&#xA;&#xA;void twoSinTest() {&#xA;  const double pi = 2.*std::acos(0.);&#xA;#if 0&#xA;  typedef double FFTFloat;&#xA;#else&#xA;  typedef cpp_dec_float_50 FFTFloat;&#xA;#endif&#xA;  double Fs = 1000;            // Sampling frequency&#xA;  double T = 1 / Fs;           // Sampling period&#xA;  int L = 1000;               // Length of signal&#xA;  std::vector&amp;lt;FFTFloat&amp;gt; timebuf(L);&#xA;  typedef std::complex&amp;lt;FFTFloat&amp;gt; C;&#xA;  std::vector&amp;lt;C&amp;gt; freqbuf;&#xA;  double t = 0;&#xA;  for (int i = 0; i &amp;lt; L; i++) {&#xA;    timebuf[i] = 0.7*std::sin(2 * pi * 50 * t) + std::sin(2 * pi * 120 * t);&#xA;    t += T;&#xA;  }&#xA;  Eigen::FFT&amp;lt;FFTFloat&amp;gt; fft;&#xA;  fft.fwd(freqbuf, timebuf);&#xA;  cout &amp;lt;&amp;lt; freqbuf.size() &amp;lt;&amp;lt; endl;&#xA;  for (int i = 0; i &amp;lt; freqbuf.size(); i++) {&#xA;    const C &amp;amp;f = freqbuf[i];&#xA;    cout &amp;lt;&amp;lt; i &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; f &amp;lt;&amp;lt; endl;&#xA;  }&#xA;} &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-04-29T15:30:28.193" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23875" PostTypeId="2" ParentId="23860" CreationDate="2016-04-29T22:37:58.317" Score="1" Body="&lt;p&gt;One approach is to be strategic about setting the initial values and the order in which the grid values of k1 and k2 are evaluated. In particular, you start with the 2x2 grid of lowest and highest values for k1, k2. Then consider, mid-points between the grid values, use &lt;em&gt;averages&lt;/em&gt; of x solutions for the neighboring grid points as the initial_guess at a new grid point. You now have 3x3 grid with solutions. Go to midpoints, obtain 5x5 grid, etc. You will of course need to save all of these solutions rather than the best of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the optimization method: since you have a 1d problem, I would try ‘Newton-CG’ with explicitly computed first and second derivatives if possible. This normally speeds up the convergence. &lt;/p&gt;&#xA;" OwnerUserId="20104" LastActivityDate="2016-04-29T22:37:58.317" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23876" PostTypeId="2" ParentId="23864" CreationDate="2016-04-30T01:16:16.273" Score="3" Body="&lt;p&gt;In addition to being ill-posed (as discussed by @BrianBorchers), the problem is also difficult to solve because it is not convex. The original least squares problem is convex because it only contains a sum of squares in the unknowns $A,B$ if $a,b$ are given. But if all four parameters are unknowns, then the exchange $A,a \leftrightarrow B,b$ in four-dimensional space leaves the objective function untouched. In other words, the problem will have (at least) two minima and consequently cannot be convex unless the optimum satisfies $A=B,a=b$; however, in that case, it is easy to see that there are infinitely many optima of the form $A'+B'=A+B$ and the problem is at least not &lt;em&gt;strictly&lt;/em&gt; convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the problem not being (strictly) convex, you will run into all sorts of numerical difficulties when trying to solve your least-squares problem.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-04-30T01:16:16.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23877" PostTypeId="1" CreationDate="2016-04-30T21:50:06.390" Score="2" ViewCount="235" Body="&lt;p&gt;I have read the paper of &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0377221713006747&quot; rel=&quot;nofollow&quot;&gt;Haase and Muller (2014)&lt;/a&gt; where they present the linear reformulations of the multinomial logit choice probabilities and compare different approaches. I have tried for an exercise to code their models, particularly the variant of &lt;a href=&quot;http://sydney.edu.au/business/__data/assets/pdf_file/0006/28878/itls_wp_09_07.pdf&quot; rel=&quot;nofollow&quot;&gt;Haase (2009)&lt;/a&gt;, and to solve it with Gurobi 6.5. The issue is that I obtain really long running times. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In their numerical investigation they used CPLEX 12 on a 64-bit Windows Server 2008 with 1 Intel Xeon 2.4 GHz processor and 24 GiB of RAM. On the other hand I am using Python + Gurobi on a 64-bit Windows 8 with 2 Six-Core AMD Opteron 2.00 GHz processors and 32 GiB of RAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that there can be differences between Gurobi and CPLEX in running times on different models/programs. In some cases the Gurobi is better, and in some cases the CPLEX is better. Also, I understand that my machine is not very representative and I hope that 0.4 GHz doesn't make such a big difference. But the difference is really huge. For some small problems, that Haase and Muller needed a little bit more than a minute to solve them, on my machine I need more than two hours. Is it possible that choice of the solver and machine can matter that much? Can someone, please, check my code, because I am newbie in this game.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the mathematical model written in LaTeX:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \max F = \sum_{i\in I} \sum_{j\in J} x_{ij} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;s.t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \sum_{j\in J} y_j = r $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \tilde{x}_{i} + \sum_{j\in J} \leq 1, \forall i \in I $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ x_{ij} - \frac{\phi_{ij}}{1 + \phi_{ij}} y_j \leq 0, \forall i \in I, j \in J $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ x_{ij} - \phi_{ij}\tilde{x}_i \leq 0, \forall i \in I, j \in J $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ y_i \in \{0, 1\}, \forall j \in J $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ x_{ij} \geq 0, \forall i \in I, j \in J $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \tilde{x}_i \geq 0, \forall i \in I $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $ \phi_{ij} = \frac{ e^{v_{ij}} }{\sum_{k\in M\setminus J} e^{v_{ik}}} $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code in Python.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from gurobipy import *&#xA;import math&#xA;import numpy as np&#xA;&#xA;n = 200 # The number of customers&#xA;m = 60 # The number of possible facility locations.&#xA;x_coords = 30 * np.random.random_sample(n + m)&#xA;y_coords = 30 * np.random.random_sample(n + m)&#xA;I = { i: (float(x_coords[i]), float(y_coords[i])) for i in range(n) } # The customers' locations.&#xA;M = { j: (float(x_coords[n + j]), float(y_coords[n + j])) for j in range(m) } # The possible facility locations.&#xA;J = {i: M[i] for i in range(m-10)} # J is subset of M. The competitor has located facilities in M\J.&#xA;MdiffJ = {i: M[m-10+i] for i in range(10)} # Competitor's facility locations. |M\J| = 10 as in the paper.&#xA;d = {}&#xA;v = {}&#xA;for i in I:&#xA;    for j in J:&#xA;        d[i,j] = abs(I[i][0] - J[j][0]) + abs(I[i][1] - J[j][1]) # Rectangular distances.&#xA;        v[i,j] = -0.2 * d[i,j]&#xA;r = round(0.2 * len(J)) # == 0.2 * |J|, as in the paper.&#xA;&#xA;&#xA;model = Model()&#xA;&#xA;# Creating variables&#xA;y = {}&#xA;x_tilda = {}&#xA;x = {}&#xA;for j in J:&#xA;    y[j] = model.addVar(vtype=GRB.BINARY, name=&quot;y_%i&quot; % j)&#xA;for i in I:&#xA;    x_tilda[i] = model.addVar(lb=0, name=&quot;x_tilda_%i&quot; % i)&#xA;    for j in J:&#xA;        x[i,j] = model.addVar(lb=0, name=&quot;x_%i%i&quot; % (i,j))&#xA;&#xA;# Integrating variables&#xA;model.update()&#xA;&#xA;# Setting the objective.&#xA;obj = LinExpr()&#xA;obj += quicksum(quicksum(x[i,j] for j in J) for i in I)&#xA;&#xA;# Setting the objectiv as a maximization problem.&#xA;model.setObjective(obj, GRB.MAXIMIZE)&#xA;&#xA;# Computing phi (constants)&#xA;phi = {}&#xA;for i in I:&#xA;    for j in J:&#xA;        phi[i,j] = math.e**v[i,j] / (sum(math.e**v[i,k] for k in MdiffJ))&#xA;&#xA;# Adding constraints.&#xA;model.addConstr(quicksum(y[j] for j in J) - r == 0, &quot;sum(y_*) == r&quot;)&#xA;for i in I:&#xA;    model.addConstr(x_tilda[i] + quicksum(x[i,j] for j in J) &amp;lt;= 1, &quot;x_tilda_%i+sum(x_%i*) &amp;lt;= 1&quot; % (i,i))&#xA;    for j in J:&#xA;        model.addConstr(x[i,j] - (phi[i,j] / (1 + phi[i,j])) * y[j] &amp;lt;= 0, &quot;x_%i%i - frac * y_%i &amp;lt;= 0&quot; % (i,j,j))&#xA;        model.addConstr(x[i,j] - phi[i,j] * x_tilda[i] &amp;lt;= 0, &quot;x_%i%i - phi_%i%i * x_tilda_%i &amp;lt;= 0&quot; % (i,j,i,j,i))&#xA;&#xA;# Computing the optimal solution.&#xA;model.optimize()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20239" LastEditorUserId="20239" LastEditDate="2016-04-30T22:15:18.233" LastActivityDate="2016-04-30T22:15:18.233" Title="Very long running time for Haase and Muller (2014) coded in Python + Gurobi" Tags="|python|mixed-integer-programming|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23878" PostTypeId="2" ParentId="16423" CreationDate="2016-04-30T22:08:43.030" Score="2" Body="&lt;p&gt;Maybe a dynamic programming approach would fit as it it seems to be close to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Knapsack_problem&quot; rel=&quot;nofollow&quot;&gt;knapsack problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you go for the PSO, I think you could express the quantity each person gets from a particular supply as a variable to your problem. Then you'll have to express your constraints and your objective function. You should have an &lt;a href=&quot;https://en.wikipedia.org/wiki/Integer_programming&quot; rel=&quot;nofollow&quot;&gt;ILP&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that you could use the PSO algorithm to search for good solutions. Here is a paper found from google scholar (is seems to be cited many many times)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;www.computer.org/csdl/proceedings/wcci/2002/7282/02/01004478.pdf&lt;/p&gt;&#xA;" OwnerUserId="20023" LastEditorUserId="20023" LastEditDate="2016-05-01T07:45:56.993" LastActivityDate="2016-05-01T07:45:56.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23879" PostTypeId="1" AcceptedAnswerId="23895" CreationDate="2016-05-01T00:26:56.890" Score="5" ViewCount="204" Body="&lt;p&gt;First I will give the relevant information for my question, and then I'll ask the question.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;$\large{\textrm{Background}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For evolving the nonlinear Schrodinger equation (NLS), one typically uses [a variant of] the &lt;a href=&quot;https://en.wikipedia.org/wiki/Split-step_method&quot;&gt;Split-Operator&lt;/a&gt;, in which one split the nonlinear and linear components of the differential equation and treats them seperately, one after another. The version of the NLS I'm concerned with is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \psi}{\partial t}=\frac{i}{2}\frac{\partial^2 \psi}{\partial x^2}+i|\psi|^2\psi=[\hat{D}+\hat{N}(x,t)]\psi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we have defined &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi=\psi(x,t),\,\,\hat{D}=\frac{i}{2}\frac{\partial^2}{\partial x^2},\,\,\hat{N}(x,t)=i|\psi|^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As outlined quite succinctly in the previously cited wiki page, the general evolution $\psi(x,t)\to\psi(x,t+\Delta t)$ for &lt;strong&gt;sufficiently small&lt;/strong&gt; step size $\Delta t$ is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(x,t+\Delta t)=e^{\Delta t(\hat{D}+\hat{N})}\psi(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To actually evolve this with a computer, we can split/factor the exponential operator in the line above to a obtain an algorithm that is correct to order $(\Delta t)^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(x,t+\Delta t)\approx e^{\Delta t(\hat{D})}e^{\Delta t(\hat{N})}\psi(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Realizing that the second derivative in the $\hat{D}$ operator turns into a multiplication by $-k^2$ in Fourier-space, we can perform this evolution on a computer by calculating, in a discretized fashion of course, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(x,t+\Delta t)\approx\mathcal{F}^{-1}\left(e^{-\frac{i}{2}\Delta t (k^2)}\mathcal{F}\left(e^{i\Delta t |\psi^2|}\psi(x,t)\right)[k,t]\right)[x,t]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be interpreted as follows: we first &quot;evolve&quot; the nonlinear part of the equation, and then we take a time step $\Delta t$ by the &quot;evolution&quot; of the linear part of the equation (in Fourier space). With this scheme, one can start with an initial wave-form/function $\psi(x,t_0)$ and evolve forward in time.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;$\large{\textrm{Question}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How does one use higher order algorithms on the NLS?&lt;/strong&gt; By using the Baker-Campell-Hausdorff formula for factorizing exponentials of operators, one can come up with a simple &lt;strong&gt;third order&lt;/strong&gt; algorithm,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(x,t+\Delta t)\approx e^{\frac{\Delta t}{2}\hat{D}}e^{\Delta t(\hat{N})}e^{\frac{\Delta t}{2}\hat{D}}\psi(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if we try to attack this in the same way we did before, we now have to evaluate $\hat{N}(x,t+\frac{\Delta t}{2})$, &lt;strong&gt;which involves evaluating $\psi(x,t+\frac{\Delta t}{2})$, which is the function we are trying to evolve!&lt;/strong&gt; More precisely, if we only start off with an initial wave-form/function $\psi(x,t)$, then we don't have access to $\psi(x,t+\frac{\Delta t}{2})$. How can I resolve this problem.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; After writing this I realized that by evaluating $e^{\frac{\Delta t}{2}\hat{D}}\psi(x,t)$, it's possible we are obtaining an approximation for $\psi(x,t+\frac{\Delta t}{2})$, which would answer my question affirmatively. However I'm going to keep this question up in case I'm incorrect.&lt;/p&gt;&#xA;" OwnerUserId="9562" LastEditorUserId="9562" LastEditDate="2016-05-01T17:43:04.423" LastActivityDate="2016-05-02T15:25:10.730" Title="Evolving nonlinear Schrodinger equation with higher-order algorithms?" Tags="|nonlinear-equations|operator-splitting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23881" PostTypeId="2" ParentId="23864" CreationDate="2016-05-01T09:44:01.143" Score="3" Body="&lt;p&gt;I can report a personal experience on trying to fit  fluorescence decays by a sum of two exponentials. We had a quantity of repeated measurements in the same condition, and several different conditions. Non-convexity, and non-uniqueness of the solutions have already been dealt with, the problem is ill-posed, so I will focus here on practical &quot;data&quot; aspects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Out of my best efforts (back in 2010), in one of the cleanest cases, the fit was fine enough, except at the beginning.&#xA;Some caveats follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/22WwR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/22WwR.png&quot; alt=&quot;Double exponential fit&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Offset&lt;/strong&gt;: in measurements, you often have an initial dead-time (before some $t_0$ time), flat signal, whose amplitude may inform you about a data offset. That offset is troublesome, because you could fit it with a lot of low-decay exponentials. Very often, maybe because of the limited length of the data frame, the  signal does not bow down to the initial offset level, and I suspect that the final level might not be the same. One more parameter to estimate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Initial time&lt;/strong&gt;: normally, the model is insensitive to time-shifts. But a measured signal has a beginning. Here, we have to choose a $t_0$. Not too late, but if too early, the sharp peak might be distorted by instrumental response (convoluted), and the bi-exponential model is not valid anymore. However, this part is highly energetic, and pumps a lot in optimization. As you can see, the difference between a fit and the data shows a peculiar not-quite-exponential residue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Location&lt;/strong&gt;: past the initial time, you have to choose a fit window, where the model is the most valid or stable. I have observed that when I slide the fit window, fitting parameters tend to vary, before becoming invalid on the right end. In fact, one can observe in $\log$ domains that stuff is not always stationnary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sampling&lt;/strong&gt;: often, when running experiments, you choose a fixed sampling. It affects differently fatty signals (low decay) and skinny ones (sharp decay), both on the choice of an initial time, and on the fitting windows choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Noise&lt;/strong&gt;: of course, signal fluctuate, rarely  uniformly, and you sometimes get spurious random spikes that affect the  fit.  Should one filter the data first? Not very sure, it can provide biases. I have tested some combinations of nonlinear geometric filters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, on $100$ signals in the same condition, I usually had large variations on estimated parameters that should have been the same, e. g. from $20\%$ to $2$-fold variation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since then, novel algorithms have been designed, imposing for instance positivity, as in &lt;a href=&quot;http://arxiv.org/abs/1401.2305&quot; rel=&quot;nofollow noreferrer&quot;&gt;Semi-infinite optimization with sums of exponentials via polynomial approximation&lt;/a&gt;, 2014, and many others. Yet, data acquisition issues are not to neglect.&lt;/p&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="19501" LastEditDate="2016-05-01T22:56:28.313" LastActivityDate="2016-05-01T22:56:28.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23882" PostTypeId="1" CreationDate="2016-05-01T12:48:47.643" Score="14" ViewCount="6934" Body="&lt;p&gt;I'm developing an FEM simulation. For early testing, I will use simple self-written mesher and visualisation of the mesh graph. But I want to prepare my program to use data generated by an existing mesher and output it to existing visualisation tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a recommended (quasi-)standard for file format and internal data format for (FEM) meshes?&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="8138" LastEditorUserId="8138" LastEditDate="2016-05-01T13:11:14.510" LastActivityDate="2017-05-10T14:41:10.017" Title="What is a common file/data format for a mesh (for FEM)?" Tags="|finite-element|mesh|graph-theory|" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23884" PostTypeId="2" ParentId="23882" CreationDate="2016-05-01T14:16:16.873" Score="3" Body="&lt;p&gt;You might try either &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#File-formats&quot; rel=&quot;nofollow&quot;&gt;Gmsh's MSH file format&lt;/a&gt; or &lt;a href=&quot;http://web.stanford.edu/class/me469b/handouts/gambit_write.pdf&quot; rel=&quot;nofollow&quot;&gt;GAMBIT neutral file format&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4873" LastActivityDate="2016-05-01T14:16:16.873" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23885" PostTypeId="2" ParentId="23882" CreationDate="2016-05-01T16:19:03.047" Score="1" Body="&lt;p&gt;Besides MSH file format cited in Wes's answer, I am also using Gamma3's MESH file format. If you are programming in C/C++, there is an easy-to-use software library, LM6, for loading/saving in this file format [1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.rocq.inria.fr/gamma/gamma/Membres/CIPD/Loic.Marechal/Research/LM6.html&quot; rel=&quot;nofollow&quot;&gt;https://www.rocq.inria.fr/gamma/gamma/Membres/CIPD/Loic.Marechal/Research/LM6.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-05-01T16:19:03.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23886" PostTypeId="2" ParentId="23882" CreationDate="2016-05-01T18:26:34.527" Score="9" Body="&lt;p&gt;The short answer is no, there is not a standard format. But there are some common ones, like Gmsh for input/output and VTK for output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before making a decision you need to find out what do you want to do. If you want to have your (small) program for a while, then you can pick the format that best suit to your taste and needs. If you are planning to change to a &quot;bigger&quot; program in the future, it would be a good idea to start managing their file formats right now. If you are planning to use commercial pre/post-processors then I suggest that you stick to their formats, since they &quot;update&quot; those with every new version and it might be a pain in the head.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below, I list some (open source) options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pre-processing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://gmsh.info/&quot;&gt;Gmsh&lt;/a&gt;, with its &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#File-formats&quot;&gt;fileformat&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.freecadweb.org/&quot;&gt;FreeCAD&lt;/a&gt; as a preprocessor. This one creates &lt;a href=&quot;http://www.calculix.de/&quot;&gt;Calculix Input files&lt;/a&gt; that are compatible with &lt;a href=&quot;http://www.3ds.com/products-services/simulia/&quot;&gt;Abaqus&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Post-processing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Visualization with &lt;a href=&quot;http://www.paraview.org/&quot;&gt;Paraview&lt;/a&gt;/&lt;a href=&quot;http://code.enthought.com/projects/mayavi/&quot;&gt;Mayavi&lt;/a&gt;. Then use &lt;a href=&quot;http://www.vtk.org/VTK/img/file-formats.pdf&quot;&gt;VTK&lt;/a&gt; file formats. You can check the Paraview user guide &lt;a href=&quot;http://www.paraview.org/paraview-downloads/download.php?submit=Download&amp;amp;version=v5.0&amp;amp;type=data&amp;amp;os=all&amp;amp;downloadFile=ParaViewGuide-CE-v5.0.0.pdf&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;You can also do post-processing in Gmsh.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.salome-platform.org/&quot;&gt;Salome-Meca&lt;/a&gt; is another option for post-processing, it works fine with VTK.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2016-05-01T19:15:38.550" LastActivityDate="2016-05-01T19:15:38.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23887" PostTypeId="2" ParentId="23882" CreationDate="2016-05-01T19:54:53.470" Score="3" Body="&lt;p&gt;There is actually a standard for this: &lt;a href=&quot;https://en.wikipedia.org/wiki/ISO_10303&quot; rel=&quot;nofollow&quot;&gt;ISO/TS 10303&lt;/a&gt; (start with parts 1380 to 1386). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prior to being hijacked by ISO, this initiative, which began back in the 1980s, was known as PDES/STEP. See &lt;a href=&quot;https://www.pdesinc.org/index.html&quot; rel=&quot;nofollow&quot;&gt;https://www.pdesinc.org/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't believe anybody much uses it unless they are working in an environment where it is a mandatory requirement. A large number of international committees did an excellent job in subverting the original good intention, and produced a standard that attempts to cover everything from the describing the overall management structure of a company to the correct format for writing dates and times (and no, I'm not joking).&lt;/p&gt;&#xA;" OwnerUserId="20246" LastEditorUserId="9667" LastEditDate="2016-05-01T20:06:14.217" LastActivityDate="2016-05-01T20:06:14.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23890" PostTypeId="1" AcceptedAnswerId="23894" CreationDate="2016-05-01T22:31:23.763" Score="0" ViewCount="201" Body="&lt;p&gt;I am comparing several algorithms (moments and matrix products) for real-time computing in terms of numerical complexity in elementary operations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT] Algorithms are very similar in terms of loops and operations. All operations are performed in a scalar way (not matrix/vector product), with elements stored in lookup tables or computed on the fly. Let us say I want to check whether Algoritm 1 uses more operations that Algoritm 2. I will consider only additions (or subtractions), multiplies and non-integer exponentiations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have so far split the operations in three classes: sums/differences, products and non-integer powers: &#xA; $(+,-)$, $(\times)$ and $(\hat{})$. I am not interested in asymptotics, &lt;em&gt;e.g.&lt;/em&gt; I am not using the Landau symbol $O()$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT] For instance, if Algorithm 1 and 2 have [$n^2$ $(+,-)$/$2n^2$ $(\times)$] and [$n^3$ $(+,-)$/$3n^2$ $(\times)$] respectively, may I say that Algorithm 1 is plausibly more efficient.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are these three classes sufficient for a basic comparision, is it still meaningful to assume the same complexity for plus and minus?  &lt;/li&gt;&#xA;&lt;li&gt;Should I include a metric for loops, for instance? Or is it so very dependent on software/hardware that such algorithmic complexity is becoming worthless?&lt;/li&gt;&#xA;&lt;li&gt;What would be a nice compact notation? For $a x^{0.3}+b y^{0.7}$, I am thinking about something like: $$(1_+,2_\times,2_{\hat{}})$$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="19501" LastEditDate="2016-05-04T09:30:53.220" LastActivityDate="2016-05-04T09:30:53.220" Title="Notations for algorithmic complexity in elementary operations" Tags="|algorithms|complexity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23892" PostTypeId="1" CreationDate="2016-05-02T08:51:57.163" Score="2" ViewCount="227" Body="&lt;p&gt;&lt;em&gt;Note: I have edited the original question to try to be clearer.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to simulate a gravitational nbody problem with a Barnes-Hut algorithm, in which the bodies start in a homogeneous sphere with random velocities. Because of this, the initial angular momentum is almost zero. I have tested the code with things like the solar system and compared its results to a brute force algorithm and it seems to work fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite using very low time steps (I'm using a velocity verlet integrator, if that matters) the angular momentum increases a lot, not being conserved at all. This image shows a typical increase of the momentum (later I will show more detailled data)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/d6ZBw.png&quot; alt=&quot;L increase&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that the energy is being conserved perfectly but the angular momentum keeps increasing with the bodies being accelerated. All the angular momentums are computed from the actual center of mass of the cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a body has a close encounter and gets expelled from the cluster, both the angular momentum and the energy increase a lot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2oHT7.png&quot; alt=&quot;A body gets expelled&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried different timesteps (from $\delta t= 10^{-8}t_0$ to $\delta t=10^{-3}t_0$) &lt;del&gt;and gravitational softening parameters $\epsilon$ (from $F_{ij} \propto r_{ij}(\epsilon + r_{ij}^3)^{-1}$) from $1$ light year to $10^{-5}$ light years (the initial separation between bodies is $6$ ly),&lt;/del&gt; and they all have the same general results (there are 8 series in each line but the deviation is minimal):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/P2u8T.png&quot; alt=&quot;Evolution of the system&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sudden rise of the lines is caused by a body being expelled by a close encounter. My principal concern is the initial increase of the angular momentum, because it doesn't come with a variation in the total energy. Because real galaxies have angular momentum, I thought that maybe I should start the simulation with a non zero angular momentum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the standard initial conditions in this kind of problems? They involve a small initial angular momentum or they use a &quot;spinning&quot; cluster to begin the simulation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Which variation in angular momentum and energy can be considered reasonable in nbody simulations?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit: I wasn't aplying gravitational softening, but it does not change the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: This is the relevant code of the angular momentum, I can see a flaw but who knows...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double acc = 0;&#xA;for(int i=0;i&amp;lt;nstars;i++){&#xA;    double r[3] = { x[i]-cmx , y[i]-cmy , z[i]-cmz };&#xA;    double v[3] = { vx[i] , vy[i] , vz[i] };&#xA;    double li[3] = { r[1]*v[2]-r[2]*v[1], -(r[0]*v[2] - r[2]*v[0]),  r[0]*v[1] - r[1]*v[0] };&#xA;    // Li = Rᵢ×Vᵢ&#xA;    acc += m[i]*norm(li);&#xA;    // acc += m⋅R×V&#xA;    }&#xA;return acc;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;cmx, cmy, cmz are the coordinates of the mass (just checked, it doesn't move a bit and it's where it should be). Using a fixed reference frame makes no difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Just tried gravitational softening (it was badly implemented). It helps a lot to avoid bodies fly away, but makes no difference in the constant increase of the angular momentum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/nHIaX.png&quot; alt=&quot;gravitational softening&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="20251" LastEditorUserId="20251" LastEditDate="2016-05-04T06:40:57.923" LastActivityDate="2016-05-04T12:29:45.747" Title="Initial conditions of nbody problem" Tags="|simulation|conservation|" AnswerCount="0" CommentCount="18" ContentLicense="CC BY-SA 3.0" />
  <row Id="23893" PostTypeId="2" ParentId="23872" CreationDate="2016-05-02T13:48:09.917" Score="2" Body="&lt;p&gt;Your problem looks very much like &quot;implicit upwind method for advection equation&quot;, let me comment it from this point of view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us think that $f=f(z,k,t)$, $f(z,k,0)=f_0(z,k)$ and we search for $f(z,k,\tau)$ that fulfills for $t \in (0,\tau)$ the advection equation (I use a shorter notation for partial derivatives):&#xA;\begin{equation}&#xA;\partial_t f + v(k) \partial_z f - F(z) \partial_k f = 0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your upwind scheme is so called implicit full upwind (the first order accurate) method, in which you try to solve for $f(z,k,\tau)$ the advection equation in &lt;strong&gt;one&lt;/strong&gt; time step. For small values of $v$ and $F$ it can work, i.e. the results can have satisfactory accuracy, but for large values the accuracy must be very poor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the implicit upwind scheme has no restriction on the time step (say your $\tau$) due to stability reasons, it has a natural restriction on it due to accuracy. In fact, the standard CFL condition is pretty good guess for such accuracy restriction, it means&#xA;$$&#xA;\tau \le h \max \limits_{i,j} \left(|v(k_j)| + |F(z_i)| \right)^{-1}&#xA;$$&#xA;where $h=\min \{\Delta k, \Delta z\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for any serious reasons your $\tau$ is given and fixed, you might look to your problem like the suggested advection equation and solve it with Lagrangian methods if you significantly violate the above condition for $\tau$. Such type of method can be implemented with arbitrary large $\tau$. &lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-05-02T16:50:34.200" LastActivityDate="2016-05-02T16:50:34.200" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23894" PostTypeId="2" ParentId="23890" CreationDate="2016-05-02T15:20:30.687" Score="3" Body="&lt;p&gt;The categories you use were meaningful a couple of decades ago when floating point operations were expensive compared to all other kinds of things processors did. But they are no longer: for example, floating point additions and multiplications today take only a few clock ticks and consequently not much longer than integer operations such as those you do when performing loop index increments or the jump at the beginning or end of a loop; and floating point operations take far less time than loading data from memory (by an order or magnitude or more), in particular if that data is not in a cache.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, the performance of algorithms can no longer be measured adequately using only the measures you count in your post. To give an example I have recently come across, my collaborator Martin Kronbichler measured the time it takes (or, rather, the memory bandwidth, which is the inverse, normalized by the amount of data) for simple operations such as the dot product between two vectors. If your measures were relevant, you'd get a constant line (because the number of operations &lt;em&gt;per vector element&lt;/em&gt; is constant). But in reality, these curves are far from horizontal lines. Take a look here to see examples: &lt;a href=&quot;https://github.com/dealii/dealii/pull/2517&quot; rel=&quot;nofollow&quot;&gt;https://github.com/dealii/dealii/pull/2517&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-05-02T21:07:03.193" LastActivityDate="2016-05-02T21:07:03.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23895" PostTypeId="2" ParentId="23879" CreationDate="2016-05-02T15:25:10.730" Score="4" Body="&lt;p&gt;Your added edit to the question points exactly the way this needs to be done: You first evolve your field using a half-step with the $\hat D$ operator, then a full step with the $\hat N$ operator, and then another half step with $\hat D$. These are done one after the other, i.e., the result of one operation is the input for the next.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in finding more about this, the scheme you describe is commonly referred to as the &quot;Strang splitting&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-05-02T15:25:10.730" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23896" PostTypeId="1" CreationDate="2016-05-02T17:31:09.573" Score="2" ViewCount="147" Body="&lt;p&gt;I've got a uniform quadrangular mesh and for each node there's a vector quantity $u$ defined. I also have a non-aligned material interface across the mesh. Now I need that vector quantity to have a zero jump across the interface&#xA;$[u\cdot n]=0$ where $n$ is the interface normal. I want to project $u$ into another space that satisfies this condition. Could I use Raviart-Thomas finite element for this case? I have never worked with RT elements and I was wondering if they could be built using an arbitrary direction instead of the element's normal (recall that the interface normal is not aligned with the elements and also that I am only interested in having $[u\cdot n]=0$, whereas the other components can have non-zero jumps) Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2016-05-03T02:01:38.493" Title="Projecting a vector field onto a H(div) space" Tags="|finite-element|projection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23897" PostTypeId="2" ParentId="23896" CreationDate="2016-05-03T02:01:38.493" Score="1" Body="&lt;p&gt;In the interior of cells, the Raviart-Thomas functions are continuous. As a consequence, the normal component is of course also continuous and the jump is zero. That may not be the case at places where the interface intersects cell boundaries, though. The set of these intersection points, however, consists of a finite number of points along a curve in 2d, or similar in 3d, i.e., a lower-dimensional manifold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I'm surprised that you need the condition $[u \cdot n]=0$. Typically, one would require that the &lt;em&gt;flux&lt;/em&gt; across an interface separating two materials is continuous. The flux is usually a material constant &lt;em&gt;times&lt;/em&gt; the vector-valued solution.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-05-03T02:01:38.493" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23898" PostTypeId="1" AcceptedAnswerId="23905" CreationDate="2016-05-03T08:47:24.980" Score="1" ViewCount="878" Body="&lt;p&gt;I ask this question because I do not see (m)any examples and I'd like to avoid to spend hours or days to work into a mesher only to see that it is impossible:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the sketch below. As you see, I have a box with a mesh. Inside this box, there's also a circle. &lt;strong&gt;This is sort of an &quot;internal boundary&quot; and I would like that some nodes exactly stick on the circle and the edges between &quot;inner&quot; and &quot;outer&quot; elements approximate the circle shape as good as possible.&lt;/strong&gt; In real world, the circle could be a way more complex structure, and there could even be inner-inner surfaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few questions arise:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does this task have a &quot;name&quot; (to facilitate internet searches)? &lt;/li&gt;&#xA;&lt;li&gt;Is it a task that every reasonable mesher can handle or do I need something special?&lt;/li&gt;&#xA;&lt;li&gt;I currently try GMESH. Is that possible with GMESH? (An example would be greatly appreciated.)&lt;/li&gt;&#xA;&lt;li&gt;Or do you recommend something different?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Pm1AT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Pm1AT.jpg&quot; alt=&quot;Sketch&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-09-28T14:05:30.483" Title="Meshing: Handle an internal boundary" Tags="|mesh|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23899" PostTypeId="2" ParentId="23829" CreationDate="2016-05-03T12:27:46.273" Score="1" Body="&lt;p&gt;I could reduce the problem to $O(n^2)$ runtime in the following manner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Build a search structure (KDTree or Octree, the KDTree garanties $O(n(log(n))$ here) with $X_2$ points contained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each $p_i$ in $X_1$ find its closest point $p_j$. Build a List $L$ containing the pairs $p_i$,$p_j$ in addition with distance $d_{i,j}$ as a triplet. Noting the distance is only needed to avoid later computations which would not effect the runtime complexity. The whole procedure is done in $O(n(log(n))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then do:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$while \ X_1 \ not empty$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \  \ \ \  \  \ \ \ iterate \ through \ L \ to \  find \ the \ triplet \  with \  the \ minimal \  distance$ linear time&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \  \ \ \  \  \ \ \ add \ p_i \ from \ triplet (in \ following \ named \ p_{i \ min}) \ to \ X_2$ constant time&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \  \ \ \  \  \ \ \ remove \ p_{i \ min} \ from \ X_1 \ and \ its \ containing \ triplet \ from \ L$ - constant time&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \  \ \ \  \  \ \ \ check \ for \ each \ triplet \  in \  L \ if \ the \ distance \  between \ the \ found \ p_i \ and \ the \ triplet \\ memeber \ p_i \ &amp;lt;= d_{i,j}$ linear time&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \  \ \ \   \  \ \ \  \  \ \ \   \  \ \ \  if \ yes \ replace \ triplet(p_i,p_j,d_i,j)  \ with \ a \ new triplet (p_i,p_{i \ min}, d_{i, i \ min})$  constant time&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$end$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The while loop is exectuted n times. Inside the loop only two linear operations are done, leading to a total execution of $n(n!)$ which is of complexity $O(n^2)$. As step one (generatint the Search structure and building the list $L$ is $O(n(log(n))$, the total complexity is limited by $O(n^2)$.&lt;/p&gt;&#xA;" OwnerUserId="20162" LastActivityDate="2016-05-03T12:27:46.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23901" PostTypeId="2" ParentId="23807" CreationDate="2016-05-03T15:49:01.650" Score="2" Body="&lt;p&gt;There is also some work on this in the literature on least-squares finite element methods, in which everything is reformulated as a first-order system; see for example &lt;a href=&quot;http://www.ams.org/mcom/1994-63-208/S0025-5718-1994-1257573-4/&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; by Bochev and Gunzburger. They use the velocity/pressure/vorticity formulation rather than velocity/pressure/strain rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since everything is positive-definite with LSFE, there's no LBB condition to satisfy (!) and you can use P1 elements for everything, instead of e.g. Taylor-Hood or MINI elements or what have you. I think the downside is that the divergence-free condition is not satisfied exactly with LSFE.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-05-03T15:49:01.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23902" PostTypeId="2" ParentId="23898" CreationDate="2016-05-03T16:02:18.130" Score="1" Body="&lt;p&gt;Meshing a geometry with an &quot;internal region&quot; is not an operation that all meshers can perform. Some allow you to specify one or more internal regions but then automatically assume these are holes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not seen a gmsh example like you show but am not certain it cannot create&#xA;such a mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the netgen mesher:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://sourceforge.net/projects/netgen-mesher/&quot; rel=&quot;nofollow&quot;&gt;https://sourceforge.net/projects/netgen-mesher/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;has an example that is very similar to yours. If you install Netgen 6.1, load the small geometry example, netgen6.1/share/netgen/squarecircle.in2d, and then push the &quot;Generate Mesh&quot; button in the GUI, you will get a mesh very similar to the one you show above. This mesh can be written to a file in a variety of formats including gmsh.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-05-03T16:02:18.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23903" PostTypeId="2" ParentId="23898" CreationDate="2016-05-03T16:26:01.407" Score="1" Body="&lt;p&gt;If you have the resources to obtain a license, &lt;a href=&quot;https://cubit.sandia.gov/&quot; rel=&quot;nofollow&quot;&gt;Cubit&lt;/a&gt; from Sandia National Labs has this capability. I have used it often in the past to generate many meshes like the one in your example.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2016-05-03T16:26:01.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23904" PostTypeId="2" ParentId="23797" CreationDate="2016-05-03T16:34:03.030" Score="6" Body="&lt;p&gt;What Charlie is saying is basically that the discrete Laplacian is Hermitian, but not with regard to the Cartesian inner product. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When defined in polar coordinates, the continuous Laplacian $\nabla^2$ satisfies &#xA;\begin{equation}&#xA;\int u \nabla^2 v rdrd\varphi = \int \nabla^2 v u r dr d\varphi &#xA;\end{equation}&#xA;for any $u(r, \varphi)$ and $v(r, \varphi)$. For the discrete Laplacian, the integrals become sums weighted by $hr_i$, or in matrix notation&#xA;\begin{equation}&#xA;\langle u, WAv \rangle = \langle A u,Wv \rangle,&#xA;\end{equation}&#xA;where $W_{ii} = hr_i$ and $\langle x, y \rangle = x^Ty$. Hence, $WA$, or in Charlie's notation $dVA$, is Hermitian but $A$ is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see a problem in solving the eigenvalue problem for $WA$ instead of $A$: if $x$ is an eigenvector of $WA$, then $W^{-1}x$ is an eigenvector of $A$ with the same eigenvalue.&lt;/p&gt;&#xA;" OwnerUserId="5245" LastActivityDate="2016-05-03T16:34:03.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23905" PostTypeId="2" ParentId="23898" CreationDate="2016-05-03T16:37:50.470" Score="3" Body="&lt;p&gt;What you mention is common, at least in my field. What you need to consider is the definition of the regions. The inside region is defined by one contour/loop: the circle. The outer region is defined by two contours/loops: the square and the circle. There are different ways of representing these. One of them is &lt;a href=&quot;https://en.wikipedia.org/wiki/Boundary_representation&quot; rel=&quot;nofollow noreferrer&quot;&gt;BRep&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done this type of meshes in different software: Gmsh, Triangle, Ansys, Abaqus. Particularly in Gmsh, you can see the &lt;code&gt;.geo&lt;/code&gt; file below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Point(1) = {-2, -2, 0, 1.0};&#xA;Point(2) = {2, -2, 0, 1.0};&#xA;Point(3) = {2, 2, 0, 1.0};&#xA;Point(4) = {-2, 2, 0, 1.0};&#xA;Point(5) = {-1, 0, 0, 1.0};&#xA;Point(6) = {0, -1, 0, 1.0};&#xA;Point(7) = {1, 0, 0, 1.0};&#xA;Point(8) = {0, 1, 0, 1.0};&#xA;Point(9) = {0, 0, 0, 1.0};&#xA;Line(1) = {1, 2};&#xA;Line(2) = {2, 3};&#xA;Line(3) = {3, 4};&#xA;Line(4) = {4, 1};&#xA;Circle(5) = {5, 9, 6};&#xA;Circle(6) = {6, 9, 7};&#xA;Circle(7) = {7, 9, 8};&#xA;Circle(8) = {8, 9, 5};&#xA;Line Loop(9) = {8, 5, 6, 7};&#xA;Plane Surface(10) = {9};&#xA;Line Loop(11) = {3, 4, 1, 2};&#xA;Plane Surface(12) = {9, 11};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the mesh is the same as shown before&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E7Q6K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E7Q6K.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2016-09-28T14:05:30.483" LastActivityDate="2016-09-28T14:05:30.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23906" PostTypeId="1" CreationDate="2016-05-03T19:28:44.130" Score="4" ViewCount="63" Body="&lt;p&gt;I am solving the nonlinear Schrodinger equation (NLSE), &#xA;$$A_t+iA_{xx}+i|A|^2A=0$$&#xA;where $A$ is a complex valued function, which can be written as $A=ae^{i\theta}$ for $a,\theta$ real. Now, for particular sets of initial conditions, I find that phase dislocations occur, and hence there are singularities in the frequency, $\omega=-\theta_t$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, do phase singularities effect the accuracy of the model? Furthermore, physically, why can rapid variations in the phase occur in an equation that is derived under the WKB assumption? Does this mean the equation is not uniformly valid? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example will be provided below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a split-step pseudo spectral method. My code is written in Fortran but I have found I can reproduce the results with a Matlab code I found online, which is provided below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial condition takes the form &#xA;$$A=sech(x)e^{iCx^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $C$ a constant representing a chirp to the signal, which can lead to packet focusing due to dispersion. The evolution of the packet is shown in figure 1. &lt;a href=&quot;https://i.stack.imgur.com/y5e1M.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y5e1M.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the phenomena in question occurs around $t=0.20$ when the amplitude $a$ goes to 0, and hence the phase is not well defined. This leads to singularities in $\omega$. A better picture of this is shown in figure 2.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mU2yZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mU2yZ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We see that as the amplitude goes towards 0 at the front and rear of the group, i.e. the packet splits, the phase starts to become singular. Note, I've decimated $\omega$ by a factor 10 to make my point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, the amplitude $a$ heuristically seems to vary slowly, but the phase clearly has rapid variations where the packet amplitude becomes small. Does this mean my results are no longer physically? Finally, and this is off topic, what sets the sign of $\omega$ at these singularities? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, to check the numerical accuracy of the scheme I confirm the energy is conserved to one part in $10^6$. Furthermore, there exist exact solutions to the NLSE, which I confirm stay solutions for long times with the resolution and step size used. Finally, the CFL condition for linear wave speeds ($u=1$ here) is obeyed. I have played around with the resolution and time steps, and find they give the same results.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab script: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  %  This Matlab script file solves the nonlinear Schrodinger equation&#xA;  %  with the first order split-step Fourier method due to Hardin and Tappert. &#xA;  %                              Andre Weideman, 1995&#xA;  %&#xA;  N=512; %number of fourier modes&#xA;  dt = 0.001;  %time step&#xA;  M=300; % number of time steps&#xA;  t=0:dt:M*dt-dt; &#xA;  L=4*pi; % size of the domain &#xA;  I=sqrt(-1); &#xA;  % J=100;&#xA;  h = L/N;                       % Space step&#xA;  n = [-N/2:1:N/2-1]';           % Indices&#xA;  x = n*h;            % Grid points&#xA;  %----------------------------------------&#xA;  % IC: Chirped sech packet&#xA;  u=sqrt(2)*sech(x).*exp(-I*x.^2); &#xA;  %----------------------------------------&#xA;  U = u;           % Compute initial condition; save it in U&#xA;  e = -4*n.*n*pi*pi/L/L;         % Squares of wavenumbers.&#xA;  for m = 1:1:M                  % Start time evolution&#xA;  u = exp(dt*I*(abs(u).*abs(u))).*u;  % Solve nonlinear part of NLS&#xA;  c = fftshift(fft(u));                 % Take Fourier transform&#xA;  c = exp(dt*I*e).*c;                   % Advance in Fourier space&#xA;  u = ifft(fftshift(c));                % Return to physical space                   &#xA;  U = [U u];                           &#xA;  end        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8021" LastActivityDate="2016-05-03T19:28:44.130" Title="Phase dislocations and numerical accuracy" Tags="|pde|computational-physics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23907" PostTypeId="1" AcceptedAnswerId="23910" CreationDate="2016-05-03T20:14:05.223" Score="0" ViewCount="32" Body="&lt;p&gt;Sentiment analysis using Machine Learning is a hot topic. Which is the best classifier to use based on the amount of training data available?&lt;/p&gt;&#xA;" OwnerUserId="20269" LastEditorUserId="4873" LastEditDate="2016-05-04T03:47:08.720" LastActivityDate="2016-05-04T03:47:08.720" Title="What is the Best and easiest way to create a Classifer for Sentiment Analysis" Tags="|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23908" PostTypeId="1" CreationDate="2016-05-03T20:49:17.600" Score="2" ViewCount="1116" Body="&lt;p&gt;I'm trying to solve the steady-state heat equation with a space-dependent thermal diffusivity in FreeFem. I.e. $$\nabla\cdot(\kappa(x,y)\nabla T) = 0$$&#xA;I'm hoping to read in from a file locations $(x_i,y_i)$ around which diffusivity changes. So, for the sake of this example, you might think of something of the form $$\kappa(x,y) = \sum_i \frac 1{\|(x,y)-(x_i,y_i)\|}$$&#xA;So what I would like would be something along the lines of (this obviously won't compile, because I haven't declared all of the variables etc. I'm just outlining)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;func real kappa() {&#xA;   for (i=0;i&amp;lt;n;++i)&#xA;      k += 1/sqrt((x-x(i)) + (y-y(i))^2);&#xA;   return k;&#xA;}&#xA;&#xA;problem heat(uh,vh,solver=GMRES) =&#xA;   int2d(Th)( (dx(uh)*dx(vh) + dy(uh)*dy(vh)) * kappa ) // * kappa doesn't work!&#xA;   + on(1,uh=1) + on(2,uh=0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where I've taken Dirichlet b.c. of 1 on one side, and 0 elsewhere. The problem is that I can't write this function on one line (because it needs a loop), and FreeFem doesn't let me call a user-defined function within the problem definition like this. So either I'm missing something, or I need another way to do this. Pointers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, I think the reason is that the objects &lt;code&gt;dx(uh)&lt;/code&gt; and co. are of type &lt;code&gt;Vh&lt;/code&gt; and so I need an object of the same type. Which means I must build my function values onto that type.&lt;/p&gt;&#xA;" OwnerUserId="8160" LastEditorUserId="8160" LastEditDate="2016-05-03T21:53:19.860" LastActivityDate="2016-05-04T09:55:22.027" Title="FreeFem user-defined function" Tags="|finite-element|pde|numerical-analysis|elliptic-pde|" AnswerCount="1" CommentCount="3" ClosedDate="2016-05-11T19:39:38.110" ContentLicense="CC BY-SA 3.0" />
  <row Id="23909" PostTypeId="1" CreationDate="2016-05-03T21:46:06.757" Score="1" ViewCount="172" Body="&lt;p&gt;Quite often I do fit some ODE or DAE systems to my data (small to medium sized problems). Via the assimulo package, I found Casadi and read a bit about the language modellica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Casadi offers automatic differentiation and an interface to ipopt. As far as I understood. It is designed to solve and optimize ODE or DAE systems but until now I havent found a report of someone using it for datafitting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be reasonable to use casadi (or jmodellica) for such a task ? The combination of Ipopt with Casadi seems to be very powerfull.&lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2016-05-03T21:46:06.757" Title="Is casadi suitable for data fitting?" Tags="|optimization|automatic-differentiation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23910" PostTypeId="2" ParentId="23907" CreationDate="2016-05-03T21:50:40.820" Score="0" Body="&lt;p&gt;If you have no data, you can use hand-written rules targeting certain words.  However, this requires careful tuning, and is time-consuming.   Naïve Bayes using a bag of words representation is fairly simple, and works well if you have very little data.  If you have a modest amount of data, MaxEnt and SVM tend to do better than Naïve Bayes in terms of accuracy.  If you have a huge amount of data SVM or kNN can become too slow, so Naïve Bayes again is preferable.  Reference: &lt;a href=&quot;https://web.stanford.edu/~jurafsky/slp3/7.pdf&quot; rel=&quot;nofollow&quot;&gt;https://web.stanford.edu/~jurafsky/slp3/7.pdf&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I was starting from scratch, I would use TextBlob: &lt;a href=&quot;http://textblob.readthedocs.io/en/dev/classifiers.html&quot; rel=&quot;nofollow&quot;&gt;http://textblob.readthedocs.io/en/dev/classifiers.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4873" LastActivityDate="2016-05-03T21:50:40.820" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23911" PostTypeId="2" ParentId="21301" CreationDate="2016-05-04T00:22:46.193" Score="0" Body="&lt;p&gt;Column-major order seems to be more natural. For example suppose if you want to save movie to file picture by picture then you are using column order, and that is very intuitive and nobody would save it in row-major order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are programmer in C/C++ you should use some higher level libraries for matrices (Eigen, Armadillo,...) with default column-major order. Only maniac would use raw C pointers with row-major order, although C/C++ offers something that reminds matrix indexing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity everything with row-major order should be considered as at least strange formed. Slice by slice is simply natural order and it means column-major order (like Fortran). Our fathers/mothers had a very good reasons why they chose it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately before it became clear several interesting libraries were created in row-major order, probably due to lack of experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify recall the definition of row-major order where right index vary faster in one step through memory eg A(x,y,z) it is z-index, it means that in memory pixels from different slices are adjacent, what we wouldn't want. For movie A(x,y,t) the last index is time t. It is not hard to imagine that it is simply impossible to save movie in row-major mode.&lt;/p&gt;&#xA;" OwnerUserId="20272" LastActivityDate="2016-05-04T00:22:46.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23912" PostTypeId="2" ParentId="4796" CreationDate="2016-05-04T00:26:43.993" Score="2" Body="&lt;p&gt;Column-major order seems to be more natural. For example suppose if you want to save movie to file picture by picture then you are using column order, and that is very intuitive and nobody would save it in row-major order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are programmer in C/C++ you should use some higher level libraries for matrices (Eigen, Armadillo,...) with default column-major order. Only maniac would use raw C pointers with row-major order, although C/C++ offers something that reminds matrix indexing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity everything with row-major order should be considered as at least strange formed. Slice by slice is simply natural order and it means column-major order (like Fortran). Our fathers/mothers had a very good reasons why they chose it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately before it became clear several interesting libraries were created in row-major order, probably due to lack of experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify recall the definition of row-major order where right index vary faster in one step through memory eg A(x,y,z) it is z-index, it means that in memory pixels from different slices are adjacent, what we wouldn't want. For movie A(x,y,t) the last index is time t. It is not hard to imagine that it is simply impossible to save movie in row-major mode.&lt;/p&gt;&#xA;" OwnerUserId="20272" LastActivityDate="2016-05-04T00:26:43.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23913" PostTypeId="1" CreationDate="2016-05-04T01:12:33.943" Score="2" ViewCount="320" Body="&lt;p&gt;I also posted this in the physics forum,  but maybe here it fits better.&#xA;I am trying to solve a generalized eigenvalue problem raised by linear stability analysis&#xA;$$AV=\lambda BV.$$&#xA;$A$ and $B$ are non-symmetric complex valued matrices.  The set of equations I am trying to solve are linearized flow,thermal,solutal fields.(I end up with 3 ODE equations, 4th order for the flow, 2nd order for both thermal and solutal)&#xA;The system is nondimentionalized, and discretized using 2nd order central finite element method in to $N$ nodes.&#xA;Then $A$ is the discretized linear operator for my u,T,C, respectively. &#xA;However, this leads to an order of $N^4$ for some of the terms in $A$, order $N^2$ for $B$. In addition to that, the wave number from the linear stability analysis also introduce some large number to $A$. The order of magnitude of $A$ and $B$ is really different. I am using LAPACK solver (ZGGEV) to solve the problem and the eigenvalues I am getting ranges from +/-$10^6$ to $10^2$. I am wondering if there is a way to scale the problem/eigenvalue such that the information can be interpreted?&#xA;Any recommendation/thoughts would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20274" LastActivityDate="2016-08-11T21:27:08.903" Title="Generalized Eigenvalue Problem from linear stability analysis" Tags="|fluid-dynamics|eigenvalues|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23915" PostTypeId="2" ParentId="23852" CreationDate="2016-05-04T06:06:17.063" Score="0" Body="&lt;p&gt;At first I considered to add only a remark, but the space was not enough, so I add some brief description of my experiences with this topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, looking to your notation of $F_{coupled}$ I do not see the coupled form, I suppose that $b_1$ and $b_2$ shall be both dependent on $c_{1,i}$ and $c_{2,i}$. Moreover if $A_1$ and $A_2$ are matrix representations of approximations of $\mathcal{F}_1$ and $\mathcal{F}_2$ then they should not depend only on $c_i$, but also on neighbor values, but this may be only a wrong understanding of your notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a general comment I would like to add that using analytical Jacobian seems to be the only way to obtain a quadratic convergence of nonlinear iterative solver (i.e. the Newton-Raphson solver in your case). Did you observe it in your case? It is quite important, because otherwise there could be some misunderstanding in your approximations (linearization).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all applications I was involved in (some of them included large scale computations) we had never issue with time consumption of assembling the Jacobian, the most time consuming issue was always applying a linear solver. The analytical Jacobian (if available) has been always in applications I was working on the preferred choice because of quadratic convergence. In few cases such nonlinear solver produces a matrix that causes problems to the convergence of iterative linear solver, so we tried to use a simpler linearization than analytical Jacobian to help the linear solver. Such a trade off between behavior of nonlinear and linear algebraic solver depending on the linearization of nonlinear algebraic system was always tricky and I could not give a general recommendation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you are right that the drawback (or property) of analytical Jacobian for system of PDEs is that it produces coupled algebraic system, so if you decoupled such system in any way, e.g. solving separately the approximation of each PDE by, say, iterative splitting method, then again you loose the quadratic convergence of global solver. But then at least if you solve separately each discretizied (decoupled) PDE, you can again speed up the solution for this particular problem by using Newton-Raphson method.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-05-04T06:06:17.063" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23916" PostTypeId="2" ParentId="4796" CreationDate="2016-05-04T08:54:32.097" Score="3" Body="&lt;p&gt;The choice of row-major / column-major indexing can have a significant impact on performances because of the way memory and cache works, and the way multiple indices are converted into a linear index.&#xA;Internally, memory is a single 1-dimensional array, and the elements of a $m\times n$ matrix will be arranged linearly:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;element $m_{i,j}$ will be stored at index $i \times m + j$ if row-major order is used&lt;/li&gt;&#xA;&lt;li&gt;element $m_{i,j}$ will be stored at index $j \times n + i$ if column-major order is used&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now imagine the following algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i from 1 to m&#xA;   for j from 1 to n&#xA;      do something with m(i,j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If row-major order is used, then this will traverse all the linear indices $i \times m + j$ sequentially, resulting in good memory locality, whereas if column-major order is used, then successive memory accesses will be scattered in memory. Consequences can be dramatic especially when virtual memory / swapping enters the scene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusions:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;yes, it has an importance, but the choice depends on the way data is accesed. For the previous example, if column-order is used, what you can do is simply swapping the two loops. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;rule of thumb: the quickly-varying index should be mapped to successive locations in memory.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;more importantly, &lt;strong&gt;measuring / benchmarking&lt;/strong&gt; the impact of the choice is fundamental, since it depends on many parameters (the size of the data, the size of the cache, the way the used language maps multiple indices to a linear index, the way the operating system manages virtual memory, the way loops are nested in the linear algebra library that you use...)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-05-04T08:54:32.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23917" PostTypeId="2" ParentId="23908" CreationDate="2016-05-04T09:55:22.027" Score="1" Body="&lt;p&gt;Yeah, I got the type wrong. This is possible, but the function needs to be evaluated over an object of the same type. For example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh Th=buildmesh(...);&#xA;fespace Vh(Th,P1);&#xA;Vh uh, vh;&#xA;&#xA;Vh kappa = 0;&#xA;for(int i = 0; i &amp;lt; n; ++i) {&#xA;   kappa += 1/sqrt((x - c(i,0))^2 + (y - c(i,1))^2);&#xA;}&#xA;&#xA;problem heat(uh,vh) =&#xA;int2d(Th)((dx(uh) * dx(vh) + dy(uh) * dy(vh)) * kappa)&#xA;+ on(1,uh=1) + on(2,uh=0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8160" LastActivityDate="2016-05-04T09:55:22.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23920" PostTypeId="1" CreationDate="2016-05-04T14:14:54.837" Score="5" ViewCount="121" Body="&lt;p&gt;Regarding the numerical stabilization of two-dimensional advection equation,&#xA;\begin{equation}&#xA;\dfrac{\partial f}{\partial t} + \Big(\dfrac{d\varepsilon_1(k)}{dk}\Big)\dfrac{\partial f}{\partial z}&#xA;- \Big( \dfrac{d\varepsilon_2(z)}{dz} \Big)\dfrac{\partial f}{\partial k} = 0&#xA;\end{equation}&#xA;I have encountered some texts that propose using entropy functions. For example, it is mentioned that the choice of &#xA;\begin{equation}&#xA;h(f,z,k) = fe^{\varepsilon_1+\varepsilon_2}&#xA;\end{equation}&#xA;will do the job for the above equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the given function (or any other entropy function) increasing the numerical stability? What is the main idea behind using such functions?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-05-04T14:14:54.837" Title="Using entropy functions for increasing numerical stability" Tags="|pde|stability|hyperbolic-pde|numerics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23922" PostTypeId="2" ParentId="23898" CreationDate="2016-05-05T07:19:11.997" Score="3" Body="&lt;p&gt;Besides the softwares pointed-at in other answers, if you are working in 2D, then Shewchuk's triangle software [1] can do the job. If you are working in 3D, then Si's tetgen software [2] has this functionality. To optimize the mesh (2D or 3D), you can use the MMGTools suite [3],[4]. My own software GEOGRAM [5] also has some mesh optimization functionalities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow&quot;&gt;https://www.cs.cmu.edu/~quake/triangle.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://wias-berlin.de/software/tetgen/&quot; rel=&quot;nofollow&quot;&gt;http://wias-berlin.de/software/tetgen/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://www.mmgtools.org/&quot; rel=&quot;nofollow&quot;&gt;https://www.mmgtools.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;https://gforge.inria.fr/projects/mmg3d/&quot; rel=&quot;nofollow&quot;&gt;https://gforge.inria.fr/projects/mmg3d/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-05-05T07:19:11.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23923" PostTypeId="2" ParentId="23293" CreationDate="2016-05-06T02:08:30.430" Score="1" Body="&lt;p&gt;If $F(x)$ is a non-symmetric hash, you can always make it symmetric by defining $F'(x) = F(x) + F(\tilde x)$ where $+$ can be any symmetric operation (bitwise xor, for example).&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2016-05-06T02:08:30.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23924" PostTypeId="1" CreationDate="2016-05-06T04:07:11.480" Score="3" ViewCount="1683" Body="&lt;p&gt;I was previously running my neural networks using the Lasagne library to build and train neural networks in Theano on an NVIDIA GTX 750 Ti. I'm using a genetic algorithm to tune the hyperparameters of my neural network, so I bought a &lt;a href=&quot;https://us.msi.com/Graphics-card/GTX-780Ti-GAMING-3G.html#hero-overview&quot; rel=&quot;nofollow&quot;&gt;GTX 780 Ti&lt;/a&gt; to speed up training each neural network. The 780 Ti should have over 3 times the single precision floating point performance of the &lt;a href=&quot;http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-750-ti&quot; rel=&quot;nofollow&quot;&gt;750 Ti&lt;/a&gt;, but I don't see any speed-up at all.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I checked GPU utilization in GPU-Z and it is the same for both GPUs (mid-high 60's%). Why is the beefier GPU using the same percentage of resources, but not running any faster?&lt;/li&gt;&#xA;&lt;li&gt;I uninstalled all NVIDIA software including the driver (restarting twice in the process), then re-installed everything using the CUDA 7.5 Toolkit Installer and then I installed the CUDNN 7.5 (5?) libraries/headers/binaries&#xA;I restarted my neural network and it was still performing as poorly as before.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some probably important information:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This is on Windows 7&lt;/li&gt;&#xA;&lt;li&gt;For nvcc I'm using &lt;code&gt;cl-version=2013&lt;/code&gt; because that's what I have and it works fine&lt;/li&gt;&#xA;&lt;li&gt;CPU is an Intel Core i5-2500K @ 3.3GHz&lt;/li&gt;&#xA;&lt;li&gt;8 GB system memory&lt;/li&gt;&#xA;&lt;li&gt;Data is loaded off of a Samsung SSD 850 EVO 500GB drive&lt;/li&gt;&#xA;&lt;li&gt;3 GB of GPU memory (other specs can be found at the link above)&lt;/li&gt;&#xA;&lt;li&gt;Card is plugged into a PCI-E Gen2 (1x16) slot&lt;/li&gt;&#xA;&lt;li&gt;The networks vary a lot, but they're all multilayer perceptrons with between 1-5 hidden layers&lt;/li&gt;&#xA;&lt;li&gt;15-150 neurons per layer, all DenseLayers&lt;/li&gt;&#xA;&lt;li&gt;75 inputs, 10 outputs&lt;/li&gt;&#xA;&lt;li&gt;Currently no drop-out or Gaussian noise before any layers&lt;/li&gt;&#xA;&lt;li&gt;Batch size is 1024&lt;/li&gt;&#xA;&lt;li&gt;Data is shuffled before each epoch&lt;/li&gt;&#xA;&lt;li&gt;Hidden layer activation functions are all elu&lt;/li&gt;&#xA;&lt;li&gt;Output activation function can be sigmoid, tanh, or elu&lt;/li&gt;&#xA;&lt;li&gt;Loss function is mean squared error&lt;/li&gt;&#xA;&lt;li&gt;Updates use nesterov momentum&lt;/li&gt;&#xA;&lt;li&gt;Training data size is 274MB with 934918 samples&lt;/li&gt;&#xA;&lt;li&gt;Validation data size is 30.5MB with 103880 samples&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My first thought was that my networks were too small to make any difference on a small vs. big GPU, but the bigger GPU is stressed just as much as the littler one when running, plus these networks were running faster on the GRID K520 GPUs on Amazon EC2 (on Ubuntu) (both the K520 and 780 Ti use the Kepler GPU architecture while the 750 Ti was Nvidia's first Maxwell card).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be causing my new card to be so slow? How can I get it to run faster than the older crappier card?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Running the nn on Ubuntu I see a significant speed-up, not as big as I hoped, but definitely noticeable. I'm probably not making full use of the GPU with my small networks. One possible reason for the slowness in Windows is WDDM mode vs. TCC mode: https:// devtalk.nvidia.com /default/topic/895331/cuda-setup-and-installation/tesla-k20-vs-titan-x-performance-for-the-same-code/post/4724884/#4724884. I'm going to try installing my old card in a PCI-E 1x slot and make that the display driver and use my new card for computation only&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit the second: Here's what one of my neural networks would look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input_var = T.matrix('inputs')&#xA;input = lasagne.layers.InputLayer(shape=(1024, 75), input_var=input_var)&#xA;network = lasagne.layers.DenseLayer(input, 40,&#xA;            nonlinearity=lasagne.nonlinearities.elu,&#xA;            W=lasagne.init.GlorotUniform())&#xA;network = lasagne.layers.DenseLayer(network, 113,&#xA;            nonlinearity=lasagne.nonlinearities.elu,&#xA;            W=lasagne.init.GlorotUniform())&#xA;network = lasagne.layers.DenseLayer(network, 76,&#xA;            nonlinearity=lasagne.nonlinearities.elu,&#xA;            W=lasagne.init.GlorotUniform())&#xA;output = lasagne.layers.DenseLayer(&#xA;            network, num_units=10,&#xA;            nonlinearity=lasagne.nonlinearities.tanh)&#xA;prediction = lasagne.layers.get_output(output)&#xA;loss = lasagne.objectives.squared_error(prediction, target_var)&#xA;loss = loss.mean()&#xA;updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate=0.5, momentum=0.5)&#xA;&#xA;test_prediction = lasagne.layers.get_output(output, deterministic=True)&#xA;test_loss = lasagne.objectives.squared_error(test_prediction, target_var)&#xA;test_loss = test_loss.mean()&#xA;&#xA;train_fn = theano.function([input_var, target_var], loss, updates=updates)&#xA;val_fn = theano.function([input_var, target_var], test_loss)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20297" LastEditorUserId="20297" LastEditDate="2016-05-12T14:19:23.720" LastActivityDate="2016-07-14T11:22:58.560" Title="Why am I not seeing faster neural network training after upgrading to a vastly better GPU?" Tags="|python|performance|machine-learning|gpu|cuda|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23927" PostTypeId="2" ParentId="23782" CreationDate="2016-05-06T22:12:18.040" Score="0" Body="&lt;p&gt;If you have the symbolic toolbox you can fix the problem by &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms x&#xA;y = exp(log(x))-x&#xA;% y = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1030" LastActivityDate="2016-05-06T22:12:18.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23928" PostTypeId="1" CreationDate="2016-05-07T03:07:38.533" Score="3" ViewCount="2074" Body="&lt;p&gt;I'm trying to solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2u + ku = f\text{ in } \Omega,$$&#xA;$$u=0 \text{ on }\partial\Omega,$$&#xA;where $\Omega$ is a unit square and $k,f$ have real and imaginary components as $k=k_r+k_ij$ and $f(x,y)=f_r(x,y) + j\cdot f_i(x,y)$, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach to solve this problem is to assume the solution has both a real and imaginary component as well:  $u=u_r + ju_i$.  Substituting this int the PDE above, we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2u_r + j\nabla^2u_i+ (k_ru_r-k_iu_i)+j(k_iu_r+k_ru_i) = f_r+jf_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equating reals with reals and imaginaries with imaginaries, we obtain a system of two coupled PDE's:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2u_r + (k_ru_r-k_iu_i) = f_r$$&#xA;$$\nabla^2u_i + (k_iu_r+k_ru_i) = f_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multipling each equation by a separate test function $v_r$, $v_i$, respectively, and integrating to get the weak forms, I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-(\nabla u_r,\nabla v_r) + k_r(u_r,v_r)-k_i(u_i,v_r) = \int_\Omega f_rv_r$$&#xA;$$-(\nabla u_i,\nabla v_i) + k_i(u_r,v_i)+k_r(u_i,v_i) = \int_\Omega f_iv_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding the two equations I get a single equation:&#xA;$$-(\nabla u_r,\nabla v_r) + k_r(u_r,v_r)-k_i(u_i,v_r) &#xA;  -(\nabla u_i,\nabla v_i) + k_i(u_r,v_i)+k_r(u_i,v_i) = \int_\Omega f_rv_r+\int_\Omega f_iv_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then choose a finite dimensional space of piecewise polynomial functions V such that $u_r,v_r,u_i,v_i\in V$.  That is, I choose all trial and test functions to come from the same space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried using Method of Manufactured Solutions to test my implementation in two cases.  In case 1, I assume that the solution is real and of the form $u=sin(\pi x)sin(\pi y)$, which yields a source term $f = (-2\pi^2+2j)sin(\pi x)sin(\pi y)$.  In case 2, I assume the solution has both real and imaginary parts and is of the form $u=sin(\pi x)sin(\pi y) + j sin(\pi x)sin(\pi y)$, which yields a source term $f= [(-2\pi^2-2) + j(-2\pi^2+2) ]sin(\pi x)sin(\pi y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For case 1, the error decreases as I refine both the mesh size and the polynomial order, as expected.  However, in case 2, I do not get convergence at all.  Clearly, it could be a programming error, but I want to make sure that my approach is sound.  In particular:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it ok to separate the complex equation into a system of real PDE's?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Am I wrong for multiplying real test functions into the separated equations?  Should I have multiplied the complex equation by a complex test function of the form $v_r+j_vi$?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it ok to add the two weak forms together to come up with a single variational expression?  I've seen this done when using mixed finite element spaces for the stokes equation, but I'm not entirely sure that this is ok for this problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2020-04-23T08:53:09.013" Title="Weak Formulation of Helmholtz equation with a complex coefficient and complex source term" Tags="|finite-element|helmholtz-equation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23929" PostTypeId="1" AcceptedAnswerId="23939" CreationDate="2016-05-07T07:32:49.277" Score="0" ViewCount="2772" Body="&lt;p&gt;I would like to sole the spring mass equation of the form:&#xA;$M\ddot{Y}+B\dot{Y}+KY = F_y(t)$, where $M :$ mass, $B :$ Damping, $K : $ stiffness, $F_y :$ external force and $Y :$ displacement in transverse direction; using &lt;strong&gt;Runge-Kutta 4th-order method&lt;/strong&gt;.&#xA;I modify RK4 C++ code as: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/*&#xA;Definition of the y'(t) = f1(t,y,y') = y' by the definition&#xA;*/&#xA;double f1(double t, double y, double v)&#xA;{&#xA;  double d1y;&#xA;  d1y = v;&#xA;  return d1y;&#xA;}&#xA;&#xA;/*&#xA;Definition of the y&quot;(t) = f2(t,y,y')&#xA;*/&#xA;double f2(double t, double y, double v, double Fy)&#xA;{&#xA; double d2y;&#xA; double M = 0.03715; // 10.0;&#xA; double B = 0.0;&#xA; double K = 0.149; &#xA; d2y = (-Fy-B*v-K*x)/M; &#xA; return d2y;&#xA;}&#xA;&#xA;/*-----------------------------------------------------------------------&#xA;Program to solve 1st order Ordinary Differential Equations&#xA;y'(t) = d1y(t,y)&#xA;y&quot;(t) = d2y(t,y)&#xA;method: 4th-order Runge-Kutta method */&#xA;&#xA;double rk4_2nd(double(*d1x)(double, double, double),&#xA;           double(*d2x)(double, double, double, double),&#xA;           double ti, double xi, double vi, double tf, double Fy,&#xA;           double&amp;amp; xf, double&amp;amp; vf)&#xA;{&#xA;  double h,t,k1x,k2x,k3x,k4x,k1v,k2v,k3v,k4v;&#xA;&#xA;  h = tf-ti;&#xA;  t = ti;&#xA;&#xA;  k1x = h*d1x(t,xi,vi);&#xA;  k1v = h*d2x(t,xi,vi,Fy);&#xA;&#xA;  k2x = h*d1x(t+h/2.0,xi+k1x/2.0,vi+k1v/2.0);&#xA;  k2v = h*d2x(t+h/2.0,xi+k1x/2.0,vi+k1v/2.0,Fy);&#xA;&#xA;  k3x = h*d1x(t+h/2.0,xi+k2x/2.0,vi+k2v/2.0);&#xA;  k3v = h*d2x(t+h/2.0,xi+k2x/2.0,vi+k2v/2.0,Fy);&#xA;&#xA;  k4x = h*d1x(t+h,xi+k3x,vi+k3v);&#xA;  k4v = h*d2x(t+h,xi+k3x,vi+k3v,Fy);&#xA;&#xA;  xf = xi + (k1x + 2.0*(k2x+k3x) + k4x)/6.0;&#xA;  vf = vi + (k1v + 2.0*(k2v+k3v) + k4v)/6.0;&#xA;&#xA;  return 0.0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem I am facing is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;RK4 is called inside a function e.g.Function(void) and this Function() is called inside main where a time loop runs as form of iterations, How can I define ti, tf, to RK4 function for my problem.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; void Function() {&#xA;&#xA; Y0[n] = Y[n] - YC; // n =  number of objects i.e. n = 1 at the moment&#xA; // Y : current displacement; YC: Initial/reference position = constant&#xA; V0[n] = V[n]; // Velocity at (t-1) = velocity at (t)&#xA; FY[n] += -k * (Y0[n]); // external force; k : spring constant&#xA;&#xA; rk4_2nd(f1, f2, t0, Y0[n], V0[n], t1, FY[n], Y[n], V[n]); &#xA;} &#xA;&#xA;int main() {&#xA;  N = 0;&#xA;do {&#xA;  N = N + 1;&#xA;  Function();&#xA; } while (N &amp;lt; NMAX);&#xA; return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18144" LastEditorUserId="18144" LastEditDate="2016-05-09T08:24:07.267" LastActivityDate="2016-05-09T16:46:57.093" Title="Equation of motion by RK4 method" Tags="|c++|runge-kutta|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23930" PostTypeId="2" ParentId="23928" CreationDate="2016-05-07T16:10:59.340" Score="3" Body="&lt;p&gt;Let $A$ and $B$ be real matrices, and $b$ and $c$ be real vectors. Then if we define $z$ as the complex solution to a complex system of equations,&#xA;$$(A+jB)z = (b+jc)$$&#xA;then $z$ is equivalently given as $z=x+jy$, where&#xA;$$\begin{bmatrix}A &amp;amp; -B \\ B &amp;amp; A\end{bmatrix}\begin{bmatrix}x \\ y\end{bmatrix} = \begin{bmatrix}b \\ c\end{bmatrix}$$&#xA;These simple facts extend to &lt;em&gt;linear operator&lt;/em&gt; equations. That is, suppose that $A+jB$ were the complex Helmholtz differential operator that you are trying to invert. Then you can follow these identical steps to obtain two sets of real, coupled operator equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your original equations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Yes. This is simply the matrix representation of complex numbers extended to linear operators.&lt;/li&gt;&#xA;&lt;li&gt;This step is fine, because complex testing / basis functions also have real representations, and can be split in the same way that the equations themselves are split.&lt;/li&gt;&#xA;&lt;li&gt;This is where you've made a mistake. To give a finite-dimensional analogy, you have effectively collapsed the 2x2 system shown above (with $2n$ real equations and $2n$ real unknowns) into $(A+B)x+(A-B)y = b+c$, which is an underdetermined system with $n$ real equations and $2n$ real unknowns. You then made this problem well-defined by setting $y=0$. So you're really solving $(A+B)x = (b+c)$, and this is right only when $B=0$ and $c=0$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you remove that last step where you have added the two equations together, the system should converge for all complex cases.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastActivityDate="2016-05-07T16:10:59.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23931" PostTypeId="2" ParentId="23255" CreationDate="2016-05-07T16:29:55.917" Score="1" Body="&lt;p&gt;I consider this to be the &quot;classic&quot; 3D-Lid-Driven Cavity (LDC) incompressible flow benchmark paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Guj, G. &amp;amp; Stella, F. A vorticity-velocity method for the numerical of 3D incompressible flows. J. Comput. Phys. 298, 286–298 (1993).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, I developed a 200 line 3-D LDC incompressible flow solver in fortran:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/charliekawczynski/short_LDC_fortran&quot; rel=&quot;nofollow&quot;&gt;https://github.com/charliekawczynski/short_LDC_fortran&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results from this code look good, but I have not compared them with literature yet.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-05-07T16:29:55.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23932" PostTypeId="1" AcceptedAnswerId="23945" CreationDate="2016-05-07T21:47:01.277" Score="5" ViewCount="465" Body="&lt;p&gt;I was playing with the idea that a sine function is periodic. But even within one period there are symmetries, namely the second fourth of a period is the mirror image of the first fourth and the second half is minus the first half. So if I could approximate the sine function from $0$ to $\pi/2$, then I would have enough to approximate it everywhere for real values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the approximation I wanted to use a polynomial of order $N$, and I wanted find it such that the polynomial minimizes the error between it and sine between $0$ and $\pi/2$. Therefore I came up with the following problem formulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\min_{\vec{a}} \int_0^{\frac{\pi}{2}} {\left(\sin(x) - \sum_{n=0}^N{a_n x^n}\right)^2 dx}. \tag{1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be solved mostly analytically by setting the derivative of $(1)$ with respect to each $a_n$ equal to zero:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_0^{\frac{\pi}{2}} {x^m \sum_{n=0}^N{a_n x^n}dx} = \int_0^{\frac{\pi}{2}} {x^m \sin(x) dx}, \quad \forall\; m=0,1,\dots, N \tag{2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sum_{n=0}^N{\frac{a_n \left(\frac{\pi}{2}\right)^{n+m+1}}{n+m+1}} = \left\{\begin{array}{l,l} m!\left(\frac{\pi}{2}\right)^{m-1}\sum_{n=0}^{\frac{m-1}{2}}\frac{\left(\frac{-4}{\pi^2}\right)^n}{(m-2n-1)!}, &amp;amp; \text{if } m \text{ odd} \\ m!\left((-1)^{\frac{m-2}{2}}+\left(\frac{\pi}{2}\right)^{m-1}\sum_{n=0}^{\frac{m-2}{2}}\frac{\left(\frac{-4}{\pi^2}\right)^n}{(m-2n-1)!}\right), &amp;amp; \text{else}\end{array}\right.\tag{3}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this problem then turns into solving a matrix equation ($Ax=b$).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;But here is my question; if I increase $N$ I would expect getting smaller and smaller errors up to some point when the computational errors of calculating the polynomial coefficients get to big, mainly probably due to the sum of the right hand side of $(3)$. But how could these computational errors be decreased? Namely initially the error get smaller, but beyond a polynomial of order 8 the error seems to grow exponentially as can be seen in the following figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XICoT.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XICoT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error of the 8th order polynomial is at most $2.152\times 10^{-8}$ (while an 7th/8th order Taylor polynomial at $x=0$ has maximum error of $1.569\times 10^{-4}$). I just wonder if I could do something to increase the order and decrease the error to something closer to $10^{-15}$, but even using the &lt;a href=&quot;http://nl.mathworks.com/help/symbolic/vpa.html&quot; rel=&quot;noreferrer&quot;&gt;variable-precision arithmetic&lt;/a&gt; in MATLAB does not seem to affect the error at all.&lt;/p&gt;&#xA;" OwnerUserId="7031" LastActivityDate="2016-05-09T23:17:19.580" Title="Polynomial order of an approximation of a section of sine and numerical accuracy" Tags="|approximation|polynomials|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23934" PostTypeId="1" AcceptedAnswerId="23935" CreationDate="2016-05-08T11:47:59.417" Score="1" ViewCount="62" Body="&lt;p&gt;I have a &lt;a href=&quot;https://www.jasondavies.com/poisson-disc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;random point generator&lt;/a&gt; which takes a distance $d$ and fills an area with points such that distance between any two points is no less that $d$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xXrEM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xXrEM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to control the number of points in the area, but how to choose $d$? What is needed is just a rough guess since the best $d$ is found using a binary search algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point generation is time consuming so the better the initial guess, the faster the heuristic search will be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how to guess $d$?&lt;/p&gt;&#xA;" OwnerUserId="4591" LastEditorUserId="4591" LastEditDate="2016-05-08T12:03:08.927" LastActivityDate="2016-05-08T17:47:13.597" Title="Finding optimal point distance to get desired number of random points in an area" Tags="|approximation|heuristics|random-sampling|search|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23935" PostTypeId="2" ParentId="23934" CreationDate="2016-05-08T15:11:40.377" Score="0" Body="&lt;p&gt;Assume the points can be reordered in a square grid. Then increasing distance between points will grow the squares of the grid. Therefore the number of points will be inversely proportional to $d^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since area is known for some units (pixels in my case), we can tell the number of points $n_{max}$ for the distance $d_{min}=1$ pixel, which is simply the number of pixels in the area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The estimated number of points then becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$n=n_{max}\cdot(\frac{d_{min}}{d})^2=\frac{n_{max}}{d^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d=(\frac{n_{max}}{n})^{\frac{1}{2}}$$&lt;/p&gt;&#xA;" OwnerUserId="4591" LastEditorUserId="4591" LastEditDate="2016-05-08T17:47:13.597" LastActivityDate="2016-05-08T17:47:13.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23936" PostTypeId="2" ParentId="14627" CreationDate="2016-05-08T16:31:39.177" Score="1" Body="&lt;p&gt;Dirichlet BCs are, by definition, a prescribed value at the boundary. If setting u(boundary) = 0 is unsettling to you, then consider the alternative of shrinking your domain so that you're only solving for the &lt;strong&gt;unknowns&lt;/strong&gt; on the interior. Terms in the Navier-Stokes will reach to the boundary (where the velocity is known) but these velocities do not experience changes in momentum (they're purely kinematic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One reason for including the boundaries themselves (and often ghost points) is to allow an easy change between Dirichlet BCs, where the boundary values are known, and Neumann BCs, where the values on the boundary must be solved for. The added point(s) are just a means to an end.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="8174" LastEditDate="2016-05-08T18:21:58.753" LastActivityDate="2016-05-08T18:21:58.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23937" PostTypeId="1" CreationDate="2016-05-08T19:43:34.667" Score="7" ViewCount="119" Body="&lt;p&gt;I am trying to solve a bunch of equations for the zeros of the derivative of an analytic function, and I would like to know if there exist methods that exploit this structure to provide better performance than the standard algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I am using Mathematica's &lt;a href=&quot;https://reference.wolfram.com/language/ref/FindRoot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;FindRoot&lt;/code&gt;&lt;/a&gt; function, which I understand relies on Newton's method. (I iterate over quasirandom seeds as described &lt;a href=&quot;https://mathematica.stackexchange.com/questions/36898/using-nsolve-in-the-complex-plane/57821#57821&quot;&gt;here&lt;/a&gt;.) My problem, however, has additional, structure, of the form&#xA;$$f'(z)=0$$&#xA;for an analytic $f$, so maybe there is a better way to do this. So: &lt;strong&gt;are there methods that exploit this structure to provide better performance&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To be a bit more explicit and provide some context in case it's useful: I am writing a Mathematica package to solve the saddle-point equations for high-order harmonic generation (as explained e.g. &lt;a href=&quot;http://arxiv.org/abs/1304.2413&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), which are of the form&#xA;$$&#xA;\left\{\begin{aligned}&#xA;(p(t,u)-A(u))^2+\gamma^2 &amp;amp; =0 \\&#xA;(p(t,u)-A(t))^2+\gamma^2 &amp;amp; = \omega,&#xA;\end{aligned}\right.&#xA;\tag{1}&#xA;$$&#xA;where $p(t,u)=\frac{1}{i\varepsilon+t-u}\int_u^tA(\tau)\mathrm d\tau$ for $\varepsilon$ a small, positive constant, $\gamma&amp;gt;0$ is fixed, and $\omega&amp;gt;0$ is a parameter. Here $A(t)$ is the vector potential of a laser field and can be a three-dimensional vector, but a simple example is $A(t)=A_0\sin(t)$. These equations can be seen as looking for the zeros of both partial derivatives of &#xA;$$&#xA;S(t,u)=\gamma^2(t-u)-\omega t+\int_u^t(p(t,u)-A(\tau))^2\mathrm d\tau.&#xA;$$&#xA;$S$ can be assumed to be known explicitly but in some cases it is relatively awkward (with &lt;a href=&quot;http://reference.wolfram.com/language/ref/LeafCount.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;LeafCount&lt;/code&gt;&lt;/a&gt;s in the &gt;6000 range).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some notes on the behaviour of this system:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I am interested in a specific box in the complex plane for $t$ and $u$. In some regimes some roots can wander off the top and bottom of this box (i.e. to higher $|\mathrm{Im}(t)|$ than prescribed), and in those cases the roots would mostly be ignored anyway. It would be nice to have them as it simplifies the accounting, but they're not crucial.&lt;/li&gt;&#xA;&lt;li&gt;Some roots which are mostly not of interest can also wander off the sides of the box, and those would definitely get ignored anyway. However, I have no guarantees on the number of roots in my chosen box, and it takes some fiddling after I've got the entire curves w.r.t. $\omega$ to design a function $g(t_s,u_s)$ that will take a root $(t_s,u_s)$ and tell me whether it's a keeper or not.&lt;/li&gt;&#xA;&lt;li&gt;The regime $A_0^2\gg\gamma^2$ is of particular interest. In this case the roots will mostly have real $t$ roots, and as $\omega$ varies they will approach each other, have avoided crossings, and veer off into nonzero $\mathrm{Im}(t)$. When $A_0^2\gg\gamma^2$ these avoided crossings can be very tight and happen very quickly with respect to variations in $\omega$, but I'm OK with having to deal with those $\omega$ regions separately.&lt;/li&gt;&#xA;&lt;li&gt;At the moment I have an explicit expression for $S(t,u)$ which I differentiate symbolically with Mathematica, and this gives derivatives with &lt;code&gt;LeafCount&lt;/code&gt; roughly 6 to 8 times those of $S$. They're not stuff I'd like to simplify by hand.&lt;/li&gt;&#xA;&lt;li&gt;I would appreciate methods which integrate well with Mathematica (either already implemented, existent as third-party tools, or relatively easy to implement), but if none are available I'm interested in any methods in this area.&lt;/li&gt;&#xA;&lt;li&gt;Sometimes I do have access to reasonable guesses for the roots (via e.g. a related, simpler $A(t)$, or using the results from the previous $\omega$) but I would rather avoid this or at least have methods which work even without such guesses.&lt;/li&gt;&#xA;&lt;li&gt;If a simpler model is desired, setting $\gamma=0$ can yield good estimates for $t$ (but not necessarily for $u$).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3551" LastEditorUserId="-1" LastEditDate="2017-04-13T12:56:10.040" LastActivityDate="2016-06-08T03:10:15.187" Title="Are there special methods for solving $f'(z)=0$ for analytic $f$?" Tags="|newton-method|complex-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23938" PostTypeId="2" ParentId="23937" CreationDate="2016-05-08T20:48:24.713" Score="3" Body="&lt;p&gt;While it's especially well-suited for polynomial systems, &lt;a href=&quot;http://arxiv.org/abs/1011.1091&quot; rel=&quot;nofollow&quot;&gt;Smale's $\alpha$-theory&lt;/a&gt; gives a way to be certain that the Newton iterates for a nonlinear analytic system will converge, and also gives a criterion for when two distinct initial guesses do or do not converge to the same root.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use it, you'll need both derivatives and bounds on the growth of the Taylor coefficients of the function you're interested in. This is especially nice for polynomials because the Taylor coefficients after the degree are 0. With this in mind, you might be better off transforming your problem by multiplying out the $(i\epsilon + t - u)^{-1}$ term, since that could put a singularity in the complex plane near the region you're interested in. That's just a hunch on my part though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Newton's method can be very finicky about the initial guess, so often you have to resort to slow methods (e.g. Picard) for a few iterations before switching to Newton. Knowing how many iterations of the slow method to use can be guesswork. By using Smale's theory, you can do the bare minimum number of iterations of an initial slow method. So it's a performance boost as well as a guarantor of robustness.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-05-08T20:48:24.713" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23939" PostTypeId="2" ParentId="23929" CreationDate="2016-05-08T20:59:43.023" Score="3" Body="&lt;p&gt;This update to your code should be what you want if I understand how you want to implement it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void Function(double currentTime, double nextTime) {&#xA; for( int i = 0; i &amp;lt; n; i++ ){&#xA;    Y0[i] = Y[i] - YC; // n =  number of objects i.e. n = 1 at the moment&#xA;    // Y : current displacement; YC: Initial/reference position = constant&#xA;    V0[i] = V[i]; // Velocity at (t-1) = velocity at (t)&#xA;    FY[i] += -k * (Y0[i]); // external force; k : spring constant&#xA;&#xA;    rk4_2nd_me1(f1, f2, currentTime, Y0[i], V0[i], nextTime, FY[i], Y[i], V[i]); &#xA; }&#xA;} &#xA;&#xA;int main() {&#xA;  double currentTime = 0.0, finalTime = 1.0;&#xA;  double dt = (finalTime-currentTime)/static_cast&amp;lt;double&amp;gt;(NMAX);&#xA;  int N = 0;&#xA;  do {&#xA;    currentTime = N*dt;&#xA;    N = N + 1;&#xA;    Function(currentTime,currentTime+dt);&#xA;  } while (N &amp;lt; NMAX);&#xA; return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now based on this, I assume that &lt;code&gt;Y0, V0, Y, V, FY, YC&lt;/code&gt; are all global variables.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-05-09T16:46:57.093" LastActivityDate="2016-05-09T16:46:57.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23940" PostTypeId="1" CreationDate="2016-05-09T05:57:42.163" Score="5" ViewCount="506" Body="&lt;p&gt;So I have an optimization problem of the form&#xA;$$\text{maximize}\hspace{3mm}f(A):{\bf R}^{K\times K}\rightarrow{\bf R}$$&#xA;$$\text{subject to}\hspace{19mm}A^T{\bf 1}=\bf{1}$$&#xA;$$\hspace{33mm}A\geq 0$$&#xA;where the constraint set is thus the manifold of left stochastic matrices (each column sums to 1).  The objective function $f$ is not concave and thus my goal is to very quickly find a local maxima (speed is important).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $f$ is a nasty piece of work, nevertheless the gradient $\nabla f$ is analytically computable.  However even without the constraint, setting the gradient equal to zero and solving for the extrema is intractable, and thus I can't imagine that back solving for Lagrange multipliers is doable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method I've tried is manifold gradient ascent using some code found &lt;a href=&quot;http://www.manopt.org/reference/manopt/manifolds/multinomial/multinomialfactory.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; from the Manopt package.  They refer to the manifold of left stochastic matrices as the &lt;em&gt;multinomial manifold&lt;/em&gt;, and the code provides methods for basically projecting the gradient onto the tangent space, and then retracting this new gradient onto the manifold itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately as far as retractions go it's fairly primitive I think, and it involves entry-wise dividing the projected gradient by the current point and then exponentiating it, which causes overflow unless you make your step-sizes quite small (however I don't fully understand the intricacies of manifold optimization so I could be wrong about this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could compute the Hessian numerically and try to run a 2nd order optimization on the multinomial manifold, but I'd have to do some reading to figure out exactly how to do this.  Basically I need something that is &lt;em&gt;fast&lt;/em&gt;, speed is more important than accuracy, as I have to do this thousands of times and it's only 1 step in a larger coordinate ascent algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient $\nabla f$ is complicated and thus each evaluation of it is quite costly, however the one saving grace is that the dimension of $A$ is relatively small, almost certainly $&amp;lt;100$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best approach here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As requested, here is the function and its gradient:&#xA;\begin{align}&#xA;f(A )&amp;amp;= \Big[-\psi\big(\sum_{i=1}^KC_{ij}\big)+\sum_{i=1}^KA_{ij}(\psi(C_{ij})-\log C_{ij})\Big]_{j=1:K} \cdot \big(\sum_{n=2}^NA^{n-2}\big)\cdot {\bf x}\\&#xA;&amp;amp;+ \sum_{n=1}^N[\log B_{in}]_{i=1:K}^T\cdot A^{n-1}\cdot{\bf x}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\nabla f(A) &amp;amp;= [\psi(C_{ij}) - \log A_{ij} -1]_{i=1:K,\hspace{1mm}j=1:K}\cdot \text{diagm}\Big(\big(\sum_{n=2}^NA^{n-2}\big)\cdot{\bf x}\Big)\\&#xA;&amp;amp;+ \sum_{n=3}^{N}\sum_{r=0}^{n-3}\Big((A^r)^T\cdot\Big[-\psi\big(\sum_{i=1}^KC_{ij}\big)+\sum_{i=1}^KA_{ij}(\psi(C_{ij})-\log C_{ij})\Big]_{j=1:K}^T \cdot {\bf x}^T\cdot (A^{n-3-r})^T\Big)\\&#xA;&amp;amp;+ \sum_{n=2}^N\sum_{r=0}^{n-2}(A^r)^T\cdot[\log B_{in}]_{i=1:K}\cdot{\bf x}^T\cdot(A^{n-2-r})^T&#xA;\end{align}&#xA;Where $\psi$ is the digamma function, $A$ and $C$ are $K\times K$ matrices, $B$ a $K\times N$ matrix and ${\bf x}$ is a length $K$ vector, all real-valued.  The function $\text{diagm}$ converts a vector to a diagonal matrix.  Also I checked my derivation of the gradient numerically, so you can be sure its correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm writing this in Julia, and I tried using the NLopt package both using a gradient-based method and a derivative free method, but they were even slower than my original manifold approach.&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="6849" LastEditDate="2018-03-11T19:15:08.130" LastActivityDate="2020-11-21T12:51:40.377" Title="Optimization on the manifold of stochastic matrices" Tags="|optimization|constrained-optimization|julia|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="23942" PostTypeId="2" ParentId="23814" CreationDate="2016-05-09T08:34:49.097" Score="6" Body="&lt;p&gt;After some searching I found two formulas to solve the accuracy problem.&#xA;The two modifications are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In range $|k| \ge 40$ the (asymptotic) series from &lt;a href=&quot;http://functions.wolfram.com/08.01.06.0010.02&quot;&gt;http://functions.wolfram.com/08.01.06.0010.02&lt;/a&gt; is used (it is valid for $|k|&amp;gt;1,\,$ but convergence is suboptimal for small $k$):&#xA;\begin{multline*}&#xA;E(z)=\sqrt{-z}&#xA;+\frac{\ln(-z)}{\sqrt{-z}}\sum_{n=0}^\infty \frac{\left(-\frac{1}{2}\right)_{n+1}^2 z^{-n}}{n!(n+1)!}\\&#xA;+\frac{1}{\sqrt{-z}}\sum_{n=0}^\infty \frac{\left(-\frac{1}{2}\right)_{n+1}^2 z^{-n}}{n!(n+1)!}&#xA;\Big(2\psi(n+1)-2\psi(n+\tfrac{1}{2})+\frac{1}{n+1}\Big)&#xA;\end{multline*}&#xA;Note Wolfram's different definition of $E$ so we actually have $z = k^2$.&#xA;To avoid overflows in computing $-k^2$ for very large $k,$  $\sqrt{-z}\,$ is computed in-line using three cases and $\ln(-z)\,$ is based on $\ln(k).$ Up to six terms are used in each sum, but only one for $|k| &amp;gt; \epsilon^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For certain values on the branch cut, i.e. $k \in \mathbb{R}, 4 &amp;lt; |k| \le 40\,$ the Legendre relation &lt;a href=&quot;http://dlmf.nist.gov/19.7.E3&quot;&gt;http://dlmf.nist.gov/19.7.E3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E\left(1/k\right)=(1/k)\left(E\left(k\right)\pm iE\left(k^{\prime}\right)-{k^{\prime}}^{2} K\left(k\right)\mp ik^{2} K\left(k^{\prime}\right)\right)$$ &#xA;is applied, where special care is taken to avoid cancellation for the&#xA;sums in the real and imaginary parts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After these two changes the relative errors are bounded by about $5 \epsilon$ for IEEE double.&lt;/p&gt;&#xA;" OwnerUserId="4853" LastEditorUserId="4853" LastEditDate="2016-05-09T08:57:18.993" LastActivityDate="2016-05-09T08:57:18.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23943" PostTypeId="2" ParentId="23327" CreationDate="2016-05-09T17:30:00.760" Score="0" Body="&lt;p&gt;I use pip as a package manager and PyCharm as my environment.  I had Anaconda installed for awhile, but found it difficult to integrate libraries that weren't part of the Anaconda distribution. &lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2016-05-09T17:30:00.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23945" PostTypeId="2" ParentId="23932" CreationDate="2016-05-09T23:08:55.427" Score="3" Body="&lt;p&gt;By using orthonormal polynomial basis functions, then you do not need to solve a linear set of equations in order to get the polynomial coefficients, similar to the &lt;a href=&quot;http://mathworld.wolfram.com/FourierSeries.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier series&lt;/a&gt; coefficients. Because for large $N$ the matrix, of the linear set of equations, will get &lt;a href=&quot;http://mathworld.wolfram.com/ConditionNumber.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ill-conditioned&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is at least required that the basis functions are orthogonal, but normalized functions help with the ease of approximation of a function. In order to get slightly simpler basis functions (without transcendental coefficients induced by the upperbound of the interval of $\pi/2$) the interval will be changed into $[0,1]$, such that the function we want to approximate becomes $\sin\left(\frac{\pi}{2}x\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two functions are orthogonal when there inner product is equal to zero. This inner product for this interval can be written as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left&amp;lt;f(x), g(x)\right&amp;gt; = \int_0^1 {f(x) g(x) dt}, \tag{1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a functions $f(x)$ is normalized when $\left&amp;lt;f(x), f(x)\right&amp;gt;=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to get similar converges of the approximation each next orthonormal polynomial basis function will contain a higher power of $x$. The first normalized basis function of order zero will then become,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_0(x) = 1. \tag{2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each following orthonormal polynomial basis function can then be found with,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_k^*(x) = x^k - \sum_{i=0}^{k-1}\left&amp;lt;x^k, p_i(x)\right&amp;gt; p_i(x), \tag{3a}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_k(x) = \frac{p_k^*(x)}{\sqrt{\left&amp;lt;p_k^*(x), p_k^*(x)\right&amp;gt;}}. \tag{3b}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next three basis functions for example then become,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{}&#xA;p_1(x) = \sqrt{3} \left(1 - 2 x\right) \\&#xA;p_2(x) = \sqrt{5} \left(1 - 6 x + 6 x^2\right) \\&#xA;p_3(x) = \sqrt{7} \left(1 - 12 x + 30 x^2 - 20 x^3\right)&#xA;\end{array} \tag{4}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $n$th order approximation of a function $f(x)$ on the interval $[0,1]$ can then simply be found with,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_n(x) = \sum_{i=0}^n {\left&amp;lt;f(x), p_i(x)\right&amp;gt; p_i(x)}. \tag{5}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following figure it can be seen that the errors keeps decreasing for higher order of approximation of $\sin\left(\frac{\pi}{2}x\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6Wnbs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6Wnbs.png&quot; alt=&quot;Absolute errors of different order of approximation.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7031" LastEditorUserId="7031" LastEditDate="2016-05-09T23:17:19.580" LastActivityDate="2016-05-09T23:17:19.580" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23946" PostTypeId="1" CreationDate="2016-05-10T03:15:11.573" Score="0" ViewCount="1382" Body="&lt;p&gt;In COMSOL, is there a way to get it to export a txt file with all the eigenfrequencies found during an eigenfrequency analysis? I suppose that it could be done through the MATLAB interface or through the COMSOL GUI.&lt;/p&gt;&#xA;" OwnerUserId="20336" LastActivityDate="2016-05-10T08:50:18.550" Title="COMSOL: get list of eigenfrequencies" Tags="|matlab|comsol|" AnswerCount="1" CommentCount="1" ClosedDate="2016-05-11T15:57:43.497" ContentLicense="CC BY-SA 3.0" />
  <row Id="23947" PostTypeId="2" ParentId="23946" CreationDate="2016-05-10T08:50:18.550" Score="1" Body="&lt;p&gt;Under results node, there are Export and Reports options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right click on Export will give you some options for exporting your data. &lt;/p&gt;&#xA;" OwnerUserId="20191" LastActivityDate="2016-05-10T08:50:18.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23948" PostTypeId="1" AcceptedAnswerId="23953" CreationDate="2016-05-10T11:53:59.873" Score="25" ViewCount="9608" Body="&lt;p&gt;I was very surprised when I started to read something about non-convex optimization in general and I saw statements like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Many practical problems of importance are non-convex, and most&#xA;  non-convex problems are hard (if not impossible) to solve exactly in a&#xA;  reasonable time. (&lt;a href=&quot;https://inst.eecs.berkeley.edu/~ee127a/book/login/l_ncp_main.html&quot; rel=&quot;noreferrer&quot;&gt;source&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In general it is NP-hard to find a local minimum and many algorithms may get stuck at a saddle point. (&lt;a href=&quot;http://www.offconvex.org/2016/03/22/saddlepoints/&quot; rel=&quot;noreferrer&quot;&gt;source&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm doing kind of non-convex optimization every day - namely relaxation of molecular geometry. I never considered it something tricky, slow and liable to get stuck. In this context, we have clearly many-dimensional non-convex surfaces ( &gt;1000 degrees of freedom ). We use mostly first-order techniques derived from steepest descent and dynamical quenching such as &lt;a href=&quot;http://users.jyu.fi/~pekkosk/resources/pdf/FIRE.pdf&quot; rel=&quot;noreferrer&quot;&gt;FIRE&lt;/a&gt;, which converge in few hundred steps to a local minimum (less than number of DOFs). I expect that with the addition of &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_gradient_descent&quot; rel=&quot;noreferrer&quot;&gt;stochastic noise&lt;/a&gt; it must be robust as hell. (Global optimization is a different story)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I somehow cannot imagine how the &lt;a href=&quot;https://en.wikipedia.org/wiki/Potential_energy_surface&quot; rel=&quot;noreferrer&quot;&gt;potential energy surface&lt;/a&gt; should look like, to make these optimization methods stuck or slowly convergent. E.g. very pathological PES (but not due to non-convexity) is &lt;a href=&quot;http://disciplinas.stoa.usp.br/pluginfile.php/388757/mod_resource/content/0/PhysRevLett.97.170201.pdf&quot; rel=&quot;noreferrer&quot;&gt;this spiral&lt;/a&gt;, yet it is not such a big problem. &lt;strong&gt;Can you give illustrative example of pathological non-convex PES?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I don't want to argue with the quotes above. Rather, I have feeling that &lt;strong&gt;I'm missing something here. Perhaps the context.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="20417" LastEditDate="2017-04-10T15:22:28.587" LastActivityDate="2021-08-05T05:38:50.560" Title="Why should non-convexity be a problem in optimization?" Tags="|optimization|nonconvex|" AnswerCount="5" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23949" PostTypeId="2" ParentId="14666" CreationDate="2016-05-10T12:17:09.237" Score="2" Body="&lt;p&gt;Let me try to help, but please correct me if I am wrong because I am just pulling that from my hat :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will just expand the computation to help to see what is happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us write $A = (X^TY)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so you want to compute $\sum_k{a_{ik}x_{ki}\nu_i}$ for every $i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $a_{ik}=\sum_l{x_{li}y_{lk}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left(diag(X^TYX)⋅v\right)_i =\sum_k{\sum_l{x_{li}y_{lk}}x_{ki}\nu_i} $ wich is $O(n^3)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let us bring the hypothesis: Since $Y$ is symmetric, $y_{lk}=y_{kl}$. In the sum, there is only the product $x_{li}x_{ki}$ that is also symmetric with respect to $k$ and $l$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So $\left(diag(X^TYX)⋅v\right)_i = 2\times\sum_k{\sum_{l&amp;gt;k}{x_{li}y_{lk}}x_{ki}\nu_i} + \sum_{k}{x_{ki}y_{kk}x_{ki}\nu_i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is $n$ times $\frac{n(n+1)}{2}$ computations. So you can divide computation by almost $2$ but not $n$. &lt;/p&gt;&#xA;" OwnerUserId="20324" LastActivityDate="2016-05-10T12:17:09.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23950" PostTypeId="1" CreationDate="2016-05-10T12:42:49.180" Score="2" ViewCount="178" Body="&lt;p&gt;In the family of Lattice Boltzmann (LB) methods, like many others, one deals with three types of node; namely, &lt;strong&gt;fluid node&lt;/strong&gt;, &lt;strong&gt;solid node&lt;/strong&gt;, and &lt;strong&gt;boundary node&lt;/strong&gt;. (I know; the boundary node is a subtype of fluid node.)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;What are the exact definitiones of these nodes? (mathematically, or physically)&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;What is the type of red node in the figure?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IROzT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IROzT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15043" LastActivityDate="2016-05-10T13:29:00.350" Title="Definitiones of solid, fluid, and boundary nodes in the context of LBM" Tags="|boundary-conditions|lattice-boltzmann-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23951" PostTypeId="2" ParentId="23950" CreationDate="2016-05-10T13:29:00.350" Score="1" Body="&lt;p&gt;Unless you use specific boundary conditions like an immersed boundary method, the gray part does not exist for the computation after the boundary nodes have been selected. As far as the computation goes, black nodes represent the wall and no other wall exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Blue nodes have all degrees of freedom. If you use D2Q9, all 9 propagation directions are used, so mathematically it is the normal LBM equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On black nodes, elements that were streamed from directions toward the wall have to be propagated back in another direction as if they bounced on the wall (at least with bounce back boundary conditions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red node is exactly as the black ones. Depending on how you decide wich nodes should represent the wall. You may have nodes that actually are inside the wall and still considered as boundary nodes or as it is on your figure, boundary nodes completely outside the wall. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On yellow nodes, unless the boundary is moving, no computation is needed. If you do not want to make an exception for it, yellow nodes can be considered as blue nodes and computation on those nodes will be disregarded. This has no influence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I've been useful.&lt;/p&gt;&#xA;" OwnerUserId="20324" LastActivityDate="2016-05-10T13:29:00.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23952" PostTypeId="2" ParentId="20027" CreationDate="2016-05-10T13:45:58.897" Score="2" Body="&lt;p&gt;The reason it looks like it's just shifted is an illusion. One step prediction plots can be very deceiving for the unseasoned. Basically what's happening is that your time series is not very predictable, and the model is spitting something very close to the last data point. Your model is close to $y(t+1) = y(t) + \epsilon$. Therefore, the prediction looks like it's shifted to the future by 1.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2016-05-10T13:45:58.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23953" PostTypeId="2" ParentId="23948" CreationDate="2016-05-11T00:07:44.037" Score="27" Body="&lt;p&gt;The misunderstanding lies in what constitutes &quot;solving&quot; an optimization problem, e.g. $\arg\min f(x)$. For mathematicians, the problem is only considered &quot;solved&quot; once we have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;A candidate solution:&lt;/strong&gt; A particular choice of the decision variable $x^\star$ and its corresponding objective value $f(x^\star)$, AND&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;A proof of optimality:&lt;/strong&gt; A mathematical proof that the choice of $x^\star$ is globally optimal, i.e. that $f(x) \ge f(x^\star)$ holds for every choice of $x$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;When $f$ is convex, both ingredients are readily obtained. Gradient descent locates a candidate solution $x^\star$ that makes the gradient vanish $\nabla f(x^\star)=0$. The proof of optimality follows from a simple fact taught in MATH101 that, if $f$ is convex, and its gradient $\nabla f$ vanishes at $x^\star$, then $x^\star$ is a global solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When $f$ is nonconvex, a candidate solution may still be easy to find, but the proof of optimality becomes extremely difficult. For example, we may run gradient descent and find a point $\nabla f(x^\star)=0$. But when $f$ is nonconvex, the condition $\nabla f(x)=0$ is necessary but no longer sufficient for global optimality. Indeed, it is not even sufficient for &lt;em&gt;local&lt;/em&gt; optimality, i.e. we cannot even guarantee that $x^\star$ is a local minimum based on its gradient information alone. One approach is to enumerate all the points satisfying $\nabla f(x)=0$, and this can be a formidable task even over just one or two dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When mathematicians say that most problems are impossible to solve, they are really saying that &lt;em&gt;the proof of (even local) optimality is impossible to construct&lt;/em&gt;. But in the real world, we are often only interested in computing a &quot;good-enough&quot; solution, and this can be found in an endless number of ways. For many highly nonconvex problems, our intuition tells us that the &quot;good-enough&quot; solutions are actually globally optimal, even if we are completely unable to prove it!&lt;/p&gt;&#xA;" OwnerUserId="11716" LastActivityDate="2016-05-11T00:07:44.037" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23955" PostTypeId="2" ParentId="23948" CreationDate="2016-05-11T05:13:23.240" Score="8" Body="&lt;p&gt;An example of a tricky low dimensional problem could be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T3A2w.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T3A2w.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given you hit a local minima, how can you be sure it's anything close to as good as the global minima? How do you know if your result is a unique optimal solution, given it is globally optimal? How can you create an algorithm robust to all the hills and valleys so it doesn't get stuck somewhere? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example like this is where things can get difficult. Obviously, not all problems are like this, but some are. What's worse is, in a setting in industry, the cost function may be time consuming to compute AND have a problematic surface like the one above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Real Problem Example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example I could tackle at work is doing an optimization for a missile guidance algorithm that could be robust at many launch conditions. Using our cluster, I could get the performance measurements I need in about 10 minutes for a single condition. Now to adequately judge robustness, we would want at least a sample of conditions to judge against. So let's say we run six conditions, making an evaluation of this cost function take one hour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nonlinear missile dynamics, atmospheric dynamics, discrete time processes, etc result in a pretty nonlinear reaction to changes in the guidance algorithm, making the optimization hard to solve. The fact this cost function will be non-convex makes the fact it is time consuming to evaluate a big issue. An example like this is where we would strive to get the best we can in the time we are given.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-05-11T05:13:23.240" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23956" PostTypeId="2" ParentId="23948" CreationDate="2016-05-11T09:44:36.883" Score="7" Body="&lt;p&gt;The problem is that of saddle points, discussed in the post which you linked. From the abstract of one of the &lt;a href=&quot;http://arxiv.org/abs/1602.05908&quot; rel=&quot;noreferrer&quot;&gt;linked articles&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, in general it is hard to guarantee that such algorithms even converge to a local minimum, due to the existence of complicated saddle point structures in high dimensions. &lt;strong&gt;Many functions have degenerate saddle points such that the first and second order derivatives cannot distinguish them with local optima&lt;/strong&gt;. In this paper we use higher order derivatives to escape these saddle points: we design the first efficient algorithm guaranteed to converge to a third order local optimum (while existing techniques are at most second order). We also show that it is NP-hard to extend this further to finding fourth order local optima.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Essentially you can have functions where you have saddle points that are indistinguishable from local minima when looking at the 1st, 2nd and 3rd derivatives. You could solve this by going to a higher order optimizer, but they show that finidng a 4th order local minimum is NP hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend reading the article, as they also show several examples of such functions. For instance the function $x^2y+y^2$ has such a point at (0,0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use a number of heuristics to escape such points, which may work for many (most?) real world examples, but cannot be &lt;strong&gt;proven&lt;/strong&gt; to always work.&lt;br&gt;&#xA;In the &lt;a href=&quot;http://www.offconvex.org/2016/03/22/saddlepoints/&quot; rel=&quot;noreferrer&quot;&gt;blog post&lt;/a&gt; you linked they also discuss the conditions under which you can escape such saddle points in polynomial time. &lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2016-05-11T09:44:36.883" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23962" PostTypeId="1" CreationDate="2016-05-12T13:11:50.900" Score="1" ViewCount="90" Body="&lt;p&gt;I have the following system of coupled differential equations (the Bogoliubov-de Gennes equations for a certain geometry):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{cases}&#xA;-\frac{1}{2}u''(r) + \big[\frac{1}{2r^2} + f_1(r)\big]u(r) + f_2(r)v(r)=\omega u(r)\\&#xA;-\frac{1}{2}v''(r) + \big[\frac{1}{2r^2} + f_1(r)\big]v(r) + f_2(r)u(r)=-\omega v(r),&#xA;\end{cases}&#xA;$$&#xA;where $f_i(r)$ are known functions which go to zero for large $r$ and are finite for small $r$. This is in fact an eigenvalue problem, because I need to find the value of $\omega$ for which this system has (normalizable) solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I would like to solve this system numerically: hence find the value of $\omega$ such that the system has a solution and then also the functions $u(r),v(r)$. I am familiar with numerical recipes to solve coupled differential equations, but now that this is an eigenvalue problem as well, I am at a loss. Could anyone give me some pointers on how to tackle such a problem? Some references where this is described would be appreciated as well!&lt;/p&gt;&#xA;" OwnerUserId="8051" LastActivityDate="2016-05-12T13:11:50.900" Title="Coupled system of differential equations with eigenvalue" Tags="|eigensystem|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23963" PostTypeId="1" AcceptedAnswerId="23964" CreationDate="2016-05-12T21:18:51.137" Score="7" ViewCount="243" Body="&lt;p&gt;In Julia it appears that one picks up some error terms when doing finite differences using matrix multiplication versus shifts and addition.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; n = 1000&#xA;1000&#xA;&#xA;julia&amp;gt; hessM = circshift(eye(n),-1) + circshift(eye(n),1) - 2* eye(n);&#xA;&#xA;julia&amp;gt; function hess(x::Vector)&#xA;         return circshift(x,-1) + circshift(x,1) - 2*x&#xA;       end&#xA;hess (generic function with 1 method)&#xA;&#xA;julia&amp;gt; x = rand(n);&#xA;&#xA;julia&amp;gt; maximum(hessM * x - hess(x))&#xA;0.0&#xA;&#xA;julia&amp;gt; x = rand(n)/300;&#xA;&#xA;julia&amp;gt; maximum(hessM * x - hess(x))&#xA;8.673617379884035e-19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I understand that this is floating point error, since if I do &lt;code&gt;x = rand(n) / 256&lt;/code&gt; (or any power of 2 for that matter) the error goes away, but if we divide by something as simple as 3 the error crops up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But where exactly is the floating point error coming in when running the above code? And why is it sensitive to the division? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit: in fact, I have localized the difference to &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; x = rand(n)/3;&#xA;&#xA;julia&amp;gt; maximum(hessM * x + 2 * eye(n) * x - circshift(eye(n),-1) * x - circshift(eye(n),1)*x)&#xA;1.1102230246251565e-16&#xA;&#xA;julia&amp;gt; x = rand(n);&#xA;&#xA;julia&amp;gt; maximum(hessM * x + 2 * eye(n) * x - circshift(eye(n),-1) * x - circshift(eye(n),1)*x)&#xA;0.0&#xA;&#xA;julia&amp;gt; x = rand(n) * 3;&#xA;&#xA;julia&amp;gt; maximum(hessM * x + 2 * eye(n) * x - circshift(eye(n),-1) * x - circshift(eye(n),1)*x)&#xA;8.881784197001252e-16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What gives?&lt;/p&gt;&#xA;" OwnerUserId="1481" LastActivityDate="2021-07-02T01:46:29.117" Title="Where does the floating point error come from? (Finite difference using matrix multiplication versus shifts and adding.)" Tags="|floating-point|julia|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23964" PostTypeId="2" ParentId="23963" CreationDate="2016-05-13T02:21:21.787" Score="6" Body="&lt;p&gt;&lt;em&gt;&lt;strong&gt;Edit (July 2021)&lt;/strong&gt;&lt;/em&gt;: it appears that the behavior will be changed as a side effect of the move of the default PRNG from Mersenne Twister to Xoshiro in the 1.7 release of Julia. See comments below.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;It seems that this is tied to how Julia generates random numbers; I've &lt;a href=&quot;https://github.com/JuliaLang/julia/issues/16344&quot; rel=&quot;nofollow noreferrer&quot;&gt;opened a discussion on the Julia Language site&lt;/a&gt;. The current implementation of Julia's random number generator for the default range &lt;code&gt;[0,1)&lt;/code&gt; for floats (in other words, calling simply &lt;code&gt;rand()&lt;/code&gt;) always produces a &lt;code&gt;0&lt;/code&gt; in the least significant bit for some reason or another (unlike MATLAB, for example). A side effect of this is that floating point errors are suppressed (since these are often errors coming from rounding/cancellation to the lowest bit). Multiplying/dividing by a power of 2 do not change the significand in the floating point representation, however multiplying/dividing by a non-power does. So generically after multiplying/dividing by some non-power, the least significant bit can be either 1 or zero, and now floating point error occurs as usually expected.&lt;/p&gt;&#xA;" OwnerUserId="1481" LastEditorUserId="1481" LastEditDate="2021-07-02T01:46:29.117" LastActivityDate="2021-07-02T01:46:29.117" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="23968" PostTypeId="1" AcceptedAnswerId="23986" CreationDate="2016-05-13T15:40:33.483" Score="11" ViewCount="2636" Body="&lt;p&gt;I am trying to simulate decaying homogeneous isotropic turbulence. As initial condition I want a divergence-free vector-field, i.e, $\mathrm{div} U = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I initialize random velocity field in an uniform grid such that above condition is satisfied?&lt;/p&gt;&#xA;" OwnerUserId="20363" LastEditorUserId="20688" LastEditDate="2019-06-22T22:47:06.910" LastActivityDate="2023-02-11T18:45:24.920" Title="Constructing random divergence-free velocity fields" Tags="|fluid-dynamics|randomized-algorithms|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23970" PostTypeId="2" ParentId="21247" CreationDate="2016-05-14T02:08:41.480" Score="1" Body="&lt;p&gt;To show stability of a numerical scheme, as stated above, Von Neumann analysis is typically the way to go. It looks at a single Fourier mode of the scheme to find the amplification factor, and from that, one may deduce the sufficient conditions of the parameters such that the scheme is stable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To wit, we show the Backward Euler scheme for the Cauchy problem for the diffusion equation is strongly $\ell_2$ stable, for any $\mu&amp;gt;0$, where $\mu = \frac{\Delta t}{\Delta x^2}$. Letting $w^{n+1}_l$ denote the approximation at time $\Delta t\cdot n$ and $\Delta x\cdot l$ in the domain, and define $h:=\Delta x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that $w^n_l = e^{ilh\xi}$, that is $w^n_l$ is a single Fourier mode. Hence $w^{n+1}_l = g(\xi)w^n_l = g(\xi)e^{ilh\xi}$, and we seek $g(\xi)$, the amplification factor. So&#xA;\begin{eqnarray*}&#xA;w^n_l &amp;amp;=&amp;amp; -\mu(w^{n+1}_{l-1} + w^{n+1}_{l+1}) + (1+2\mu)w^{n+1}_{l}\\&#xA;  &amp;amp;=&amp;amp; -\mu  g(\xi)(e^{i(l-1)h\xi} + e^{i(l+1)h\xi}) + (1+2\mu)e^{ilh\xi}g(\xi)\\&#xA;  &amp;amp;=&amp;amp; g(\xi)\left(1+2\mu(1-cos(h\xi))\right)e^{ilh\xi}\\&#xA;\end{eqnarray*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we claim that the scheme is unconditionally stable. Hence &#xA;$$&#xA;|g(\xi)| = |\frac{1}{1+2\mu(1-cos(h\xi))}|\leq 1&#xA;$$&#xA;or &#xA;$$&#xA;1 \leq |{1+2\mu(1-cos(h\xi))}| &#xA;$$&#xA;We find $4\mu \geq 1+2\mu(1- cos(h\xi)) \geq 1$, conditional of $\mu$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar analysis is carried out for other schemes to find the sufficient conditions of the parameters so that the scheme is stable. Specifically, the analysis for Crank Nicolson is more complicated since it has half a Forward Euler step and half a Backward Euler step, and is correspondingly stable for $\mu\leq 1$.&lt;/p&gt;&#xA;" OwnerUserId="17678" LastActivityDate="2016-05-14T02:08:41.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23971" PostTypeId="2" ParentId="23968" CreationDate="2016-05-14T03:12:26.863" Score="14" Body="&lt;p&gt;Create two random scalar fields $f$ and $g$ and set the velocity to: &#xA;$$&#xA;u=\nabla f \times \nabla g&#xA;$$&#xA;which is guaranteed to be divergence free. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-05-14T03:12:26.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23972" PostTypeId="2" ParentId="21247" CreationDate="2016-05-14T04:06:00.680" Score="4" Body="&lt;p&gt;Another way you can find stability is to assume first you are solving the following model problem for complex &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\dot{x} = f(t,x) = \lambda x$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with exact solution: &lt;span class=&quot;math-container&quot;&gt;$x(t) = x_0 e^{\lambda t}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then you define your scheme, let's start with Explicit Euler:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ x_{k+1} = x_{k} + h \; f(t,x_{k}) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the step size. Based on this scheme, you substitute the value for your simple equation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ x_{k+1} = x_{k} + h \lambda x_{k} = (1 + h \lambda ) x_{k} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We then can say that an equation is stable if the following is true:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \left| \frac{x_{k+1}}{x_{k}} \right| = \left| \sigma \right| \le 1 \;\;\;\;\;\forall k $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$\left| \cdot \right|$&lt;/span&gt; is the complex modulus in this case. With this, we can find the ratio &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \sigma = (1 + h\lambda_{Re}) + i h \lambda_{Im}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\lambda_{Re} = Re[\lambda]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\lambda_{Im} = Im[\lambda]$&lt;/span&gt;. We can then work out and obtain the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \left| \sigma \right|^2 \le 1 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ (1+h \lambda_{Re})^2 + h^2 \lambda_{Im}^2 \le 1$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ 1 + 2 h \lambda_{Re} + h^2 (\lambda_{Re}^2 +\lambda_{Im}^2) \le 1$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ 2 h Re[\lambda] + h^2 \left| \lambda \right|^2 \le 0 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ 2 Re[\lambda] + h \left| \lambda \right|^2 \le 0 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ h \le \frac{-2 Re[\lambda]}{\left| \lambda \right|^2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now first, given &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is positive, the term on the right hand side must be positive. This implies that &lt;span class=&quot;math-container&quot;&gt;$Re[\lambda] \lt 0 $&lt;/span&gt; must be true for the problem to potentially be stable.&lt;/p&gt;&#xA;&lt;p&gt;For purely real &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;, you can simplify the expression to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ h \le \frac{2 }{\left| \lambda \right|} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now given you have discretized your problem to the following system equations:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \dot{\textbf{u}} = A \textbf{u}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is some matrix, specifically a differential operator when solving Partial Differential Equations, you can then theoretically transform this system using the eigensystem of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A=Q \Lambda Q^{-1}$&lt;/span&gt;, by doing the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \dot{\textbf{u}} = A \textbf{u}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \dot{\textbf{u}} = Q \Lambda Q^{-1} \textbf{u}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ Q^{-1} \dot{\textbf{u}} = \Lambda Q^{-1} \textbf{u}$$&lt;/span&gt;&#xA;Define &lt;span class=&quot;math-container&quot;&gt;$\textbf{z} = Q^{-1} \textbf{u}$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \dot{\textbf{z}} = \Lambda \textbf{z}$$&lt;/span&gt;&#xA;or&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \dot{z}_{j} = \lambda_{j} z_{j} \;\;\; \forall j$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\lambda_{j}$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$j^{th}$&lt;/span&gt; eigenvalue of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Now that the system is diagonalized, you can find that the most limiting step size will be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ h \le 2 \min_{j} \left(\frac{-Re[\lambda_{j}]}{\left| \lambda_{j} \right|^2}\right) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which will be based on the eigenvalue with the largest modulus in this case&lt;/p&gt;&#xA;&lt;h1&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/h1&gt;&#xA;&lt;p&gt;Provided you have some scheme, you can derive/look up it's stability criteria relative to the model problem. After you have this stability criteria, you can apply it to problems by, if the scheme is conditionally stable, finding eigenvalues of your matrix operator &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and find the most limiting step size using the eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-05-14T04:06:00.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23973" PostTypeId="1" CreationDate="2016-05-14T07:31:15.817" Score="7" ViewCount="337" Body="&lt;p&gt;Suppose we have a sparse matrix $A$.  Is there any way to compute just the sparsity pattern of $A^2 = A \cdot A$ (I do not actually need to know what exactly the nonzero value are) faster than to compute the full sparse product $A \cdot A$? &lt;/p&gt;&#xA;" OwnerUserId="20263" LastEditorUserId="20417" LastEditDate="2016-09-30T18:41:30.063" LastActivityDate="2016-09-30T18:41:30.063" Title="Compute sparsity pattern of $A^2$" Tags="|linear-algebra|matrix|sparse-matrix|" AnswerCount="0" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="23974" PostTypeId="1" CreationDate="2016-05-14T07:46:34.653" Score="4" ViewCount="140" Body="&lt;p&gt;Given a symmetric matrix $A$, the Lanczos algorithm outputs $C$ such that $C^{T}AC$ is tridiagonal. Is there a  generalization of this such that $C^{T}AC$ is banded of specific width $w$? Note that $C$ can be rectangular.&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="15120" LastEditDate="2016-05-14T17:37:33.580" LastActivityDate="2016-05-14T18:23:22.447" Title="Solve for $C$ such that $C^{T}AC$ is banded of given width" Tags="|linear-algebra|sparse-matrix|banded-matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23976" PostTypeId="1" AcceptedAnswerId="23977" CreationDate="2016-05-14T12:51:08.407" Score="2" ViewCount="189" Body="&lt;p&gt;I have a reference value $R$ and a modelled value $M$. $M$ is generated using a stochastic algorithm with parameters $a$ and $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is to tune $a$ and $b$ so that $M$ is as close as $R$ possible (heuristically). But each run of $M$ takes around 30 seconds. Any &quot;good&quot; algorithm will be jammed by this computation, seemingly. Even so, what is better (than others) algorithm of choice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we are to allow $\vert M-R\vert$ converges quickly initially (and accept it to slowly converge in later stages), what types of algorithm should I look for? My colleague suggests Particle Swarm Optimisation (PSO). Is that a good choice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: We need &quot;good&quot; fit but not &quot;best&quot; fit. Our tolerance is fairly large.&lt;/p&gt;&#xA;" OwnerUserId="20373" LastActivityDate="2016-05-14T15:16:13.460" Title="What kind of optimisation algorithm is suitable for a computationally expensive function?" Tags="|optimization|complexity|stochastic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23977" PostTypeId="2" ParentId="23976" CreationDate="2016-05-14T13:30:11.940" Score="5" Body="&lt;p&gt;Response surface models (a kind of surrogate model) are often used in situations like this.  The idea is to sample values of the parameters $a$ and $b$, compute $R(a,b)$ at each point, and then build a regression model (typically quadratic or even higher order) of the function.  You can then optimize over the the fitted model.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-05-14T13:30:11.940" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23978" PostTypeId="2" ParentId="17504" CreationDate="2016-05-14T13:55:44.787" Score="4" Body="&lt;p&gt;You  might also want to have a look at:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Elmer &lt;a href=&quot;https://www.csc.fi/web/elmer&quot; rel=&quot;nofollow&quot;&gt;https://www.csc.fi/web/elmer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Kratos &lt;a href=&quot;http://www.cimne.com/kratos/&quot; rel=&quot;nofollow&quot;&gt;http://www.cimne.com/kratos/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;OpenFoam &lt;a href=&quot;http://www.openfoam.com/&quot; rel=&quot;nofollow&quot;&gt;http://www.openfoam.com/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;CaeLinux &lt;a href=&quot;http://caelinux.com/CMS/&quot; rel=&quot;nofollow&quot;&gt;http://caelinux.com/CMS/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20374" LastActivityDate="2016-05-14T13:55:44.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23979" PostTypeId="2" ParentId="23976" CreationDate="2016-05-14T15:16:13.460" Score="2" Body="&lt;p&gt;&lt;strong&gt;Active learning&lt;/strong&gt; (aka &lt;strong&gt;experimental design&lt;/strong&gt;) strategies are suitable for this. One of them being the response surface modeling suggested by &lt;a href=&quot;https://scicomp.stackexchange.com/a/23977/1128&quot;&gt;Brian Borchers&lt;/a&gt;. The idea is to choose the next point to evaluate to learn about the optimal M as fast as possible. Here are some old and new papers to start from:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Harold J. Kushner. A new method of locating the maximum of an arbitrary multipeak curve in the&#xA;presence of noise. Journal of Basic Engineering, pages 86:97–106, March 1964.&lt;/li&gt;&#xA;&lt;li&gt;J. Mockus. The Bayesian approach to global optimization. Lecture Notes in Control and Information Sciences, 38:473–481, 1982.&lt;/li&gt;&#xA;&lt;li&gt;Julien Villemonteix, Emmanuel Vazquez, and Eric Walter. An informational approach to the global optimization of expensive-to-evaluate functions. Journal of Global Optimization, September 2008, cs/0611143.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1128" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-05-14T15:16:13.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23980" PostTypeId="2" ParentId="23974" CreationDate="2016-05-14T18:23:22.447" Score="7" Body="&lt;p&gt;Yes. The block Lanczos algorithm &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/etemplates/node250.html&quot;&gt;http://www.netlib.org/utk/people/JackDongarra/etemplates/node250.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;produces a block triangular matrix where you control the block size, hence the bandwidth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly, one can argue that a block tridiagonal matrix is not a &quot;proper&quot; banded matrix as there regular patches of certain zeros within the band. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a proper banded matrix then you should do a two-sided reduction to upper quasi upper Hessenberg form with $r&amp;gt;1$ subdiagonals using, say, Householder reflections, see&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0167819111000482&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0167819111000482&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this article is not freely available to you, then much of the same information is freely available here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www8.cs.umu.se/research/uminf/reports/2010/014/part1.pdf&quot;&gt;http://www8.cs.umu.se/research/uminf/reports/2010/014/part1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an approach which is distinctly different from the Lanczos algorithms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases the decision to seek out a form which is not mathematically optimal stems from a realization that it is no longer the raw flop count which determines the run time. Reducing memory operations, communication and the need synchronization is frequently much more important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A term which is relevant to your problem is &quot;symmetric band reduction&quot;.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2016-05-14T18:23:22.447" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="23984" PostTypeId="1" AcceptedAnswerId="23987" CreationDate="2016-05-16T01:26:54.607" Score="4" ViewCount="1421" Body="&lt;p&gt;I need to solve a linear system of the form&#xA;$$Ax = b$$&#xA;where $A$ is upper Hessenberg matrix with the lower bandwidth equal to 1, $b$ is the RHS vector and $x$ is the solution vector. I have a C++ routine based on &quot;C.B. Moler, Algorithm 423, Linear Equation Solver, C.A.C.M. 15 (1972), p. 274.&quot; that solves the system. However, I wish to use LAPACK if available. Since I could not find any LAPACK routines for 'Hessenberg matrix linear system solving', I used the &lt;code&gt;dgetrf&lt;/code&gt; and &lt;code&gt;dgetrs&lt;/code&gt; routines, but they are way slower than the C++ implementation for Hessenberg matrices. Sample Output is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;Bench-marking Hessenberg matrix solve routines (size = 400)&#xA;            Function   Time-taken(s)         Residual&#xA;            --------   -------------         --------&#xA;          C++ dech()        0.000367               xx&#xA;          C++ solh()        0.000140       0.00000004&#xA;        LPK dgetrf()        0.004051               xx&#xA;        LPK dgetrs()        0.000110       0.00000005&#xA;&#xA;Final results&#xA;  C++ factorization is 11.027X faster than LAPACK.&#xA;  C++ bf solve is 0.790X slower than LAPACK.&#xA;&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is: is there any routine in LAPACK that specializes in handling linear system solves with Hessenberg matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;b&gt;NOTE&lt;/b&gt;:&#xA;Dense matrix solving using &lt;code&gt;dgetrf&lt;/code&gt; and &lt;code&gt;dgetrs&lt;/code&gt; is way faster in LAPACK, as expected.&lt;/p&gt;&#xA;" OwnerUserId="15045" LastActivityDate="2019-05-17T19:18:48.167" Title="Solving linear system $Ax=b$ with Hessenberg matrix using lapack" Tags="|linear-solver|lapack|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23986" PostTypeId="2" ParentId="23968" CreationDate="2016-05-16T02:04:40.273" Score="7" Body="&lt;p&gt;A purely random initial condition will not resemble realistic turbulence and will take quite some time to reach a realistic decay state. If you want good results it's more complicated than just ensuring zero divergence. Usually the velocity field is generated in spectral space to give the proper spectral energy distribution and is then transformed to physical space to initialize the simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Section 4.1 of the following paper outlines one way to do this:&lt;br&gt;&#xA;Huidan Yu, Sharath S. Girimaji, Li-Shi Luo, DNS and LES of decaying isotropic turbulence with and without frame rotation using lattice Boltzmann method, Journal of Computational Physics, Volume 209, Issue 2, 1 November 2005, Pages 599-616, ISSN 0021-9991, &lt;a href=&quot;http://dx.doi.org/10.1016/j.jcp.2005.03.022&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1016/j.jcp.2005.03.022&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This section begins:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We conduct the simulations in a 3D periodic cube with various resolutions $N^3$. The initial incompressible homogeneous isotropic velocity field $\vec{u}_0$ (${\nabla} \cdot \vec{u}_0 = 0$) is generated in spectral space $\kappa$ with the following energy spectrum in a prescribed range $\kappa_{\min} \le \kappa \le \kappa_{\max}$ and a random phase (cf. details in [47]):&lt;br&gt;&#xA;  \begin{align}&#xA;\hat{E}(\kappa,0) = \left\{&#xA;\begin{array}&#xA;0.038\kappa^m \exp(-0.14\kappa^2) &amp;amp; \kappa\in[\kappa_{\min},\kappa_{\max}],\\&#xA;0 &amp;amp; \kappa\notin[\kappa_{\min},\kappa_{\max}].&#xA;\end{array}&#xA;\right.~~~~~~~~~~~~(16)&#xA;\end{align}&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Then the velocity field is transferred to physical space.  In what follows, we use $m = 4$ or $2$ in Eq. (16) to investigate the effect of $m$ on the energy spectrum and other quantities.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[47]  T. Miyauchi, T. Ishizu, Direct numerical simulation of homogeneous isotropic turbulence – decay of passive scalar fluctuation, in&#xA;  Preprint vol. JSME No. 914-2, 1991, pp. 166–168.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You might also want to look at:&lt;br&gt;&#xA;Consistent initial conditions for the DNS of compressible turbulence&#xA;Ristorcelli, J. R. and Blaisdell, G. A., Physics of Fluids, 9, 4-6 (1997), DOI:&lt;a href=&quot;http://dx.doi.org/10.1063/1.869152&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1063/1.869152&lt;/a&gt;&lt;br&gt;&#xA;which could also be used for incompressible turbulence by using only the first step in their procedure which involves generating an appropriate divergence-free velocity field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That paper chooses a solenoidal (zero divergence) velocity field with spectrum $E(\kappa) = A\kappa^4\exp(-2\kappa^2/\kappa_p^2)$ and random phase components. $A$ determines the velocity amplitude. $\kappa_p$ determines the peak in the spectral energy and is set equal to $12$ in that paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that these two methods are equivalent with appropriate choices of parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;br&gt;&#xA;As for the specifics of generating a zero-divergence velocity with a desired energy spectrum, that question has been asked &lt;a href=&quot;https://scicomp.stackexchange.com/questions/23159/how-to-obtain-values-in-physical-space-for-a-given-spectrum&quot;&gt;here&lt;/a&gt; but has not been answered. Perhaps someone can go back and answer that old question now.&lt;/p&gt;&#xA;" OwnerUserId="7733" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-05-19T13:13:52.567" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23987" PostTypeId="2" ParentId="23984" CreationDate="2016-05-16T06:08:43.230" Score="3" Body="&lt;p&gt;There is no (readily available) solver for systems in Hessenberg form inside LAPACK. However, you can build one yourself using LAPACK routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, use Givens rotations to reduce your system to upper triangular form. Then do backward substitution on the transformed system. You construct Given rotations with xROTG. You apply them with xROTF. You solve a triangular system with xTRTRS&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an inverse iteration for Hessenberg systems built into LAPACK, which could presumably be tricked into solving your system, but I would not bother. Orthogonal transformations will never get you into trouble. &lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2016-05-16T06:08:43.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23990" PostTypeId="1" CreationDate="2016-05-16T17:10:11.807" Score="8" ViewCount="176" Body="&lt;p&gt;This is an entry-level question for multiway matrix decompositions. I have a set/population &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; of entities (here biological cells) for each of which I also have a number (&lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;) of flavours of length &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; vectors (genetic-like measurements for which &lt;span class=&quot;math-container&quot;&gt;$N \gg k$&lt;/span&gt;) which characterise them. Now if I take just one type of vector, and look at its values across all of the entities (cells), I do a PCA since I effectively have a &lt;span class=&quot;math-container&quot;&gt;$k \times N$&lt;/span&gt; (depending on convention) array which I can explore in the conventional. If I repeat this analysis &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; times for each flavour of the vector, I get &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; very similar analyses but I would like to take an overall view. My question is this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;is it possible (or reasonable) to approach this as a multiway analysis on an &lt;span class=&quot;math-container&quot;&gt;$l \times k \times N$&lt;/span&gt; data source says using a Tucker factorisation/tensor decomposition?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;is this the most appropriate approach?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;are there any available resources, papers or books that I should read to understand this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The aim is to understand how the different vector flavours (types of genetic-like information) differ in the characterisation of the cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently working with the &lt;code&gt;rTensor&lt;/code&gt; package in R which I think may be based upon a similar one in MATLAB. I have a maths background and am in principle not afraid of tensors although this would be the first time I have used them for data analysis.&lt;/p&gt;&#xA;" OwnerUserId="5261" LastEditorUserId="20688" LastEditDate="2018-11-06T21:18:54.010" LastActivityDate="2018-11-06T21:18:54.010" Title="Tucker factorisation to compare multiple PCA decompositions?" Tags="|matlab|r|tensor-decomposition|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="23991" PostTypeId="1" CreationDate="2016-05-16T17:12:14.513" Score="1" ViewCount="116" Body="&lt;p&gt;I am trying to solve a problem, which I find quite hard, like, &lt;em&gt;headache-hard&lt;/em&gt;. I have to solve the following set of $M$ nonlinear equations:&#xA;$$F(X)=\begin{bmatrix}f_1 (X)\\f_2 (X)\\...\\f_M (X)\\ \end{bmatrix}=0,X\in{}R^{N\times{}M}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_i\left(X\right)=x_{*,i}-F^{-1}\left[k\left(\omega{}\right)+\sum_{j=1}^Mq\left(\omega{}\right)\dot{}F\left[H^{\left(j\right)}\left(x_{*,j}\right)\right]\right]=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $F$ is the Fourier transform and $H$ a nonlinear function, $k$ and $q$ are constants. $H$ has the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H\left(x\right)=r{\left(\frac{x}{s}\right)}^{\alpha{}},\&#xA;s,\ r,\ x\in{}R^N$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The highest $\alpha$ is, the harder it is to find a solution. I tried both simulated annealing and Jacobian-Free Newton Krylov numerical methods, without much success. I believe an issue here might be the way I state the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should &lt;strong&gt;JFKN&lt;/strong&gt; or &lt;strong&gt;SA&lt;/strong&gt; compute a perturbation/update on each matrix element separately? It does not sound reasonable to me, as I am dealing with timeseries, each column of $X$ being a timeseries of length $N$; I guess perturbation should be flexible yet consistent from one element to the next. Ha! A bit of insight or advice would be very much appreciated here!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="14268" LastEditorUserId="14268" LastEditDate="2016-05-18T11:10:36.930" LastActivityDate="2016-05-19T01:22:49.937" Title="Solving a large system of nonlinear equations, where timeseries are the unknown" Tags="|nonlinear-equations|nonlinear-programming|newton-method|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="23992" PostTypeId="1" CreationDate="2016-05-16T17:18:16.137" Score="1" ViewCount="191" Body="&lt;p&gt;I'm going through this &lt;a href=&quot;http://inmabb.criba.edu.ar/revuma/pdf/v54n1/v54n1a08.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article defines function function $\phi_n^\mu(x)$ that is orthonormal on $L^2$ with measure $dm = dx$:&#xA;\begin{equation}&#xA;	\phi^\mu_n =\left(\frac{\gamma_\mu(n)}{\Gamma(\mu+\frac{1}{2})}\right)^\frac{1}{2}\frac{1}{\sqrt{2^n}n!}e^{-\frac{x^2}{2}}H^\mu_n(x)x^\mu;  \mu,n\in{N},&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it defines differential operator (one-dimensional case, $d=1$, $\mu_1=\mu$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation} \label{eq:hcal}&#xA;	\mathcal{H_\mu} = \frac{1}{2}\left( &#xA;		 \sum^d_{k=1}\left[&#xA;		 	\mathcal{D}^2_{\mu_k}-\frac{2\mu_k}{x} \left(&#xA;		 		\frac{\partial}{\partial x}+\mu_k-1&#xA;		 	\right)&#xA;		 \right] -|x|^2 &#xA;	\right),&#xA;\end{equation}&#xA;where \begin{equation} \label{eq:dunkl}&#xA;	\mathcal{D}_\mu\phi(x)=\phi'(x) + \frac{\mu}{x}\left(&#xA;		\phi(x)-\phi(-x)&#xA;	\right),&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is said (in the paper) that $\phi$ is the eigenfunction of $\mathcal{H}^\mu$ operator, i.e. $-2(\mathcal{H}^\mu \phi_n^\mu) (x) = 2(n+\mu+1/2)\phi_n^\mu(x)$ for one-dimensional case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the functions written in Python: dunkl2 for $\mathcal{D}_\mu$, and calh for $\mathcal{H}_\mu$ operator:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def dunkl2(n,m,func,x): &#xA;return misc.derivative(func,x,0.001,2) + 2*m*(misc.derivative(func,x,0.001))/x - m*(func(x)-func(-x))/x**2&#xA;&#xA;def calh(n,m,func,x):&#xA;return -dunkl2(n,m,lambda x: func(n,m,x),x) + 2*m/x*misc.derivative(lambda x: func(n,m,x),x,0.001) + (2*m/x*(m-1)+x**2)*func(n,m,x)&#xA;&#xA;for i in range(0,10):&#xA;print(&quot;n =&quot;, i, &quot;; m =&quot;, 10-i, &quot;; x =&quot;, 1, &quot;; (Cal.H * f)(x) / f(x) =&quot;, calh(i,10-i,phi,1)/phi(i,10-i,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I test my Python functions by simply dividing both sides of the differential equation by $\phi_n^\mu(x)$ and checking if the result is $2(n+\mu+1/2)$, and this gives me the set of numbers below. But, as you can see, there's something wrong, at least because the resulting coefficient depends not only on $n$ and $\mu$, but on $x$ also.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where is my mistake? I can assure you that this phi I use is indeed orthonormal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 0 ; m = 9 ; x = 1 ; (Cal.H * f)(x) / f(x) = 108.999910168&#xA;n = 1 ; m = 8 ; x = 1 ; (Cal.H * f)(x) / f(x) = 74.9999101676&#xA;n = 2 ; m = 7 ; x = 1 ; (Cal.H * f)(x) / f(x) = 75.0000086295&#xA;n = 3 ; m = 6 ; x = 1 ; (Cal.H * f)(x) / f(x) = 49.0000086295&#xA;n = 4 ; m = 5 ; x = 1 ; (Cal.H * f)(x) / f(x) = 49.0000089668&#xA;n = 5 ; m = 4 ; x = 1 ; (Cal.H * f)(x) / f(x) = 31.0000089668&#xA;n = 6 ; m = 3 ; x = 1 ; (Cal.H * f)(x) / f(x) = 30.9999828753&#xA;n = 7 ; m = 2 ; x = 1 ; (Cal.H * f)(x) / f(x) = 20.9999828754&#xA;n = 8 ; m = 1 ; x = 1 ; (Cal.H * f)(x) / f(x) = 20.9999722442&#xA;n = 9 ; m = 0 ; x = 1 ; (Cal.H * f)(x) / f(x) = 18.9999722445&#xA;&#xA;n = 0 ; m = 9 ; x = 2 ; (Cal.H * f)(x) / f(x) = 77.5000026668&#xA;n = 1 ; m = 8 ; x = 2 ; (Cal.H * f)(x) / f(x) = 61.0000026667&#xA;n = 2 ; m = 7 ; x = 2 ; (Cal.H * f)(x) / f(x) = 53.9999972207&#xA;n = 3 ; m = 6 ; x = 2 ; (Cal.H * f)(x) / f(x) = 41.4999972198&#xA;n = 4 ; m = 5 ; x = 2 ; (Cal.H * f)(x) / f(x) = 36.5000043004&#xA;n = 5 ; m = 4 ; x = 2 ; (Cal.H * f)(x) / f(x) = 28.0000043004&#xA;n = 6 ; m = 3 ; x = 2 ; (Cal.H * f)(x) / f(x) = 24.9999862338&#xA;n = 7 ; m = 2 ; x = 2 ; (Cal.H * f)(x) / f(x) = 20.4999862334&#xA;n = 8 ; m = 1 ; x = 2 ; (Cal.H * f)(x) / f(x) = 19.4999826454&#xA;n = 9 ; m = 0 ; x = 2 ; (Cal.H * f)(x) / f(x) = 18.9999826456&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18584" LastEditorUserId="18584" LastEditDate="2016-05-17T16:17:30.163" LastActivityDate="2016-05-17T16:17:30.163" Title="Generalized Hermite Function as eigenfunction of a differential operator" Tags="|python|scipy|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="23994" PostTypeId="2" ParentId="23968" CreationDate="2016-05-17T09:13:13.683" Score="2" Body="&lt;p&gt;If you are in 2D and if you want more physical setups, I suggest you consider &lt;a href=&quot;https://en.wikipedia.org/wiki/Potential_flow&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;potential flows&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various ways to construct these &lt;em&gt;potential flows&lt;/em&gt; which are always divergence free and which satisfy certain boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the wikipedia article, you find the &lt;a href=&quot;https://en.wikipedia.org/wiki/Potential_flow#Power_laws&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;power laws&lt;/em&gt;&lt;/a&gt;, that describes flows around plates, edges, or in corners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there are other generating functions, that may give you, e.g., the flow around an idealized airfoil.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-05-17T09:13:13.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="23995" PostTypeId="1" CreationDate="2016-05-17T14:01:16.433" Score="5" ViewCount="4813" Body="&lt;p&gt;I am interested in 2D channel flow of an incompressible Stokes fluid (Re &amp;lt;&amp;lt; 1), with periodic boundary conditions in the x-direction and no-slip at the walls in the y-direction. I have existing code that solves this using a stream function formulation which is tested and works correctly. The details of the numerical  implementation shouldn't matter, but for reference I use finite differences in the y-direction, and a Fourier method in the x-direction.&lt;/p&gt;&#xA;&lt;p&gt;In the stream-function formulation the pressure is eliminated via the incompressibility condition, however occasionally it is useful to examine it in order to gain some insight. To be clear, this pressure is an auxiliary field, and the accuracy of the simulation isn't dependent on it.&lt;/p&gt;&#xA;&lt;p&gt;My question is how can this pressure be recovered from the stream function object? I also have the raw velocities in case they are easier to work with. This general problem appears to be called the Pressure Poisson Equation (PPE) but I'm having some difficulties understanding it.&lt;/p&gt;&#xA;&lt;h2&gt;My attempt&lt;/h2&gt;&#xA;&lt;p&gt;Begin with the Stokes equation (subject to some force &lt;span class=&quot;math-container&quot;&gt;$\bf f$&lt;/span&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$0 = \eta \nabla^2{\bf v} - \nabla p + {\bf f} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Taking the divergence of this, and applying incompressibility, &lt;span class=&quot;math-container&quot;&gt;${\bf 0} = {\bf \nabla}\cdot{\bf v}$&lt;/span&gt;, we find:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla^2 {p} = \nabla \cdot {\bf f}, \qquad {\rm (1)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;i.e., a Poisson equation for the pressure.&lt;/p&gt;&#xA;&lt;h3&gt;Doubly Periodic&lt;/h3&gt;&#xA;&lt;p&gt;In a doubly periodic system (i.e., periodic in x and y) we can easily calculate the pressure (defined up to a constant), using the boundary conditions that gradients in the pressure are subject to periodic BCs: &lt;span class=&quot;math-container&quot;&gt;$\nabla p(x, y) = \nabla p(x+L,y)$&lt;/span&gt; etc. This can easily be solved in Fourier space.&lt;/p&gt;&#xA;&lt;h3&gt;Channel flow&lt;/h3&gt;&#xA;&lt;p&gt;This is where I am unsure - in the periodic direction I know the boundary condition, but in the wall normal direction it's not clear what the boundary condition should be. Eq 16 in &lt;a href=&quot;https://arxiv.org/pdf/1011.3589.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ref&lt;/a&gt; (excluding advection, and assuming time-independent BCs) suggest that both forces and velocity must appear in the BC:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;${\bf n}\cdot\nabla p = {\bf n}\cdot\left(f + \eta \nabla^2 {\bf v} \right)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This seems consistent with &lt;a href=&quot;http://people.math.umass.edu/%7Ejohnston/PAPERS/JCP_1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ref 2&lt;/a&gt; Eq 1.9 (which sets &lt;span class=&quot;math-container&quot;&gt;$\bf f = 0$&lt;/span&gt;) where the BC reads:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\partial p/\partial{\bf n} = {\bf n} \cdot \eta \nabla^2 {\bf v}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is this an appropriate way to find the pressure in channel geometries, along with Eq. (1) above? If not are there any alternative approaches, perhaps involving the streamfunction directly? Ease of implementation is preferred over efficiency as this is generally a one-off calculation.&lt;/p&gt;&#xA;" OwnerUserId="7941" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-07-20T14:27:09.270" Title="Recovering pressure from velocity or streamfunction fields" Tags="|fluid-dynamics|boundary-conditions|navier-stokes|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23996" PostTypeId="2" ParentId="23995" CreationDate="2016-05-17T20:49:22.760" Score="0" Body="&lt;p&gt;The answers you've provided are valid, however, if you're looking for a simpler approach that is specific to your problem, then you may use $\partial p / \partial n = 0$ on a no-slip, no-flow through wall. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The justification behind this, I believe, is that the Dirichlet conditions imply velocity is known and, if steady, means no net forces exist at the boundary. Looking at the momentum equation on the boundary normal to the wall, you'll see that all forces are zero (since the boundary is kinematic). Therefore $\partial p / \partial n = 0$ is a valid BC.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="8174" LastEditDate="2016-06-17T03:29:05.397" LastActivityDate="2016-06-17T03:29:05.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23998" PostTypeId="1" CreationDate="2016-05-17T23:44:22.220" Score="0" ViewCount="2737" Body="&lt;p&gt;I am currently using &lt;a href=&quot;https://github.com/WarrenWeckesser/odeintw&quot; rel=&quot;nofollow&quot;&gt;the wrapper odeintw&lt;/a&gt; for scipy.integrate.odeint to solve my equations since they are complex-valued.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, I have 3 coupled first-order differential equations with 2 independent variables&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial C}{\partial t} &amp;amp;= \frac{i}{a}(AB - cd) - \frac{C}{t_1}\\&#xA;\frac{\partial A}{\partial t} &amp;amp;= \frac{2ib^2}{a}cC - \frac{A}{t_2}\\&#xA;\frac{\partial B}{\partial t} &amp;amp;= ied + f&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the capital letters A, B, and C are my functions; the lower case letters a, b, c, t1, t2, and d are simply parameters; z and t are my independent variables. I've been able to solve the first two equations by simply making B equal to a constant (i.e. neglecting the 3rd equation) and inputting certain initial conditions/parameter values. The code is shown below and provides the real and imaginary part of the solutions separately (note: some of the variable names are different, but the equations are equivalent): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from odeintw import odeintw&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;def Wfunc(W, t, hbar, Pm, Em, Ep, T1, T2, d):&#xA;    N, Pp = W&#xA;    return [(1j/hbar)*(Pp*Ep - Em*Pm) - N/T1,&#xA;            (2j*(d**2)/hbar)*Em*N - Pp/T2]&#xA;&#xA;W0 = np.array([1+2j, 3+4j])&#xA;t = np.linspace(0, 5, 1001)&#xA;hbar = 1.&#xA;Pm = 4 - 2j&#xA;Em = 2.5&#xA;Ep = 10.&#xA;T1 = 100.&#xA;T2 = 10&#xA;d = 0.1&#xA;W, infodict = odeintw(Wfunc, W0, t, args=(hbar, Pm, Em, Ep, T1, T2, d),&#xA;                  full_output=True)&#xA;&#xA;plt.figure(1)&#xA;plt.clf()&#xA;color1 = (0.5, 0.4, 0.3)&#xA;color2 = (0.2, 0.2, 1.0)&#xA;plt.plot(t, W[:, 0].real, color=color1, label='N.real', linewidth=1.5)&#xA;plt.plot(t, W[:, 0].imag, '--', color=color1, label='N.imag', linewidth=2)&#xA;plt.plot(t, W[:, 1].real, color=color2, label='Pp.real', linewidth=1.5)&#xA;plt.plot(t, W[:, 1].imag, '--', color=color2, label='Pp.imag', linewidth=2)&#xA;plt.xlabel('t')&#xA;plt.grid(True)&#xA;plt.legend(loc='best')&#xA;&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works as intended for the 2 equations with same independent variable, but now I want to introduce my other independent variable, z, and am running into a bit of difficulty. Are there any efficient ways to solve these 3 equations simultaneously? Extending the code for cases involving large intervals over the independent variables and more complex differential equations would be of interest.&lt;/p&gt;&#xA;" OwnerUserId="20405" LastEditorUserId="9667" LastEditDate="2016-05-18T17:50:19.640" LastActivityDate="2016-05-18T17:50:19.640" Title="Solving coupled differential equations with multiple independent variables" Tags="|pde|python|numerical-modelling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="23999" PostTypeId="2" ParentId="23998" CreationDate="2016-05-18T04:37:33.207" Score="3" Body="&lt;p&gt;You can integrate the third equation in $z$. Then you know $B$ up to a constant, right? Then $z$ is no longer an independent variable. Since it's then a 2-variable linear problem you can then just solve it analytically.&lt;/p&gt;&#xA;" OwnerUserId="19722" LastActivityDate="2016-05-18T04:37:33.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24001" PostTypeId="2" ParentId="23654" CreationDate="2016-05-18T08:51:37.257" Score="1" Body="&lt;p&gt;That seems to be the right way to solve for the lifting operators, see for example Appendix A in &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/fld.2161/abstract&quot; rel=&quot;nofollow&quot;&gt;&quot;Discontinuous Galerkin methods for the Navier-Stokes equations&#xA;using solenoidal approximations&quot; by Montlaur et al&lt;/a&gt; (there is also an authors copy if you search for the title), it is for the Compact DG Method but as far as I'm aware all of the lifting operators are equivalent up to some small change to modify the behavior slightly (e.g. CDG is a compact modification of the Local Discontinuous Galerkin method) &lt;/p&gt;&#xA;" OwnerUserId="20410" LastActivityDate="2016-05-18T08:51:37.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24002" PostTypeId="1" CreationDate="2016-05-18T15:21:14.017" Score="1" ViewCount="835" Body="&lt;p&gt;I would like to link my Fortran 90 application to a library which would give me Delaunay triangulation (for a 2D set of points) and tetrahedralization (for a 3D set of points). As suggested in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2026/fastest-delaunay-triangulation-libraries-for-sets-of-3d-points&quot;&gt;Fastest Delaunay triangulation libraries for sets of 3D points&lt;/a&gt; thread I am going to try TETGEN, which functions as both, a library and a command line utility. By default, TETGEN gives you 3D Delaunay &lt;strong&gt;tetrahedralization&lt;/strong&gt;, and I would like to know if I can also obtain Delaunay &lt;strong&gt;triangulation&lt;/strong&gt; for a set of 2D points using TETGEN. As was pointed out in the comments below, this can be accomplished using TRIANGLE via an API similar to TETGEN, but I would still like to link my code to a single library and not two different ones depending on the dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can I use TETGEN with a 2D set of points in addition to 3D? Any other suggestions on what library I can use? The thread above also mentions CGAL. &lt;/p&gt;&#xA;" OwnerUserId="20414" LastEditorUserId="20688" LastEditDate="2019-03-15T14:59:43.343" LastActivityDate="2019-03-15T15:06:40.730" Title="Can TETGEN generate triangulation of a 2D point set?" Tags="|software|mesh-generation|delaunay-triangulation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24003" PostTypeId="1" CreationDate="2016-05-18T16:42:32.440" Score="2" ViewCount="129" Body="&lt;p&gt;I'm familiar with kmeans to cluster data points, but not text.. So I have one column of words in cvs format, which I want to cluster those which have similar word or more, and then mapping those cluster to numbers as index, those index numbers needed to be added as a second column. I know there is scipy packahes and word2vec in python, but this is the first time for me dealing with clustering text.. Any idea on how to do this??&lt;/p&gt;&#xA;" OwnerUserId="14682" LastActivityDate="2016-05-20T09:11:44.053" Title="Clustering similar words and mapping to numbers in Python" Tags="|python|mapping-strategy|clustering|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24004" PostTypeId="2" ParentId="15891" CreationDate="2016-05-18T21:35:41.563" Score="-2" Body="&lt;p&gt;Here is a &lt;a href=&quot;https://github.com/paulknysh/blackbox&quot; rel=&quot;nofollow&quot;&gt;solution&lt;/a&gt; to your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Description of a mathematical method is provided in this &lt;a href=&quot;http://arxiv.org/pdf/1605.00998.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20419" LastActivityDate="2016-05-18T21:35:41.563" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24005" PostTypeId="1" CreationDate="2016-05-18T21:46:20.630" Score="16" ViewCount="1560" Body="&lt;p&gt;I am learning the theory behind DG-FEM methods using the Hesthaven/Warburton book and I am a bit confused about the role of the 'numerical flux.' I apologize if this is a basic question, but I have looked and not found a satisfactory answer to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the linear scalar wave equation:&#xA;$$\frac{\partial u}{\partial t} + \frac{\partial f(u)}{\partial x} = 0$$&#xA;where the linear flux is given as $f(u) = au$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As introduced in Hesthaven's book, for each element $k$, we end up with $N$ equations, one for each basis function, enforcing that the residual vanishes weakly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R_h(x,t) = \frac{\partial u_h}{\partial t} + \frac{\partial au_h}{\partial x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{D^k} R_h(x,t) \psi_n(x) \, dx = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fine. So we go through integration by parts once to arrive at the 'weak form' (1) and integrate by  parts twice to get the 'strong form' (2). I will adopt Hesthaven's sort-of-overkill but easily generalized surface integral form in 1D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) $$ \int_{D^k} \left( \frac{\partial u_h^k}{\partial t} \psi_n-au_h^k\frac{d \psi_n}{d x} \right)\, dx = - \int_{\partial D^k} \hat{n}\cdot(au_h)^*\psi_n \,dx \qquad 1 \leq n\leq N$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2)&#xA;$$ \int_{D^k} R_h \psi_n\, dx = \int_{\partial D^k} \hat{n}\cdot \left( au_h^k-(au_h)^* \right)\psi_n \,dx \qquad 1 \leq n\leq N $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we choose a numerical flux? Why don't we use the value of $au_h^k$ at the boundary in (1) instead of using a flux? Yes, it's true that the value of this quantity may be multiply defined across elements, but each equation is only over 1 element $D^k$, so why does this matter? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, the boundary term of the second integration by parts clearly yields a different quantity $au_h^k$ the second time in (2), which makes no sense to me. We are doing the same operation! Why wouldn't the two boundary terms just cancel, making (2) useless? How have we introduced new information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly I am missing something crucial to the method, and I would like to fix this. I have done some real and functional analysis, so if there is a more theory-based answer regarding the formulation, I would like to know!&lt;/p&gt;&#xA;" OwnerUserId="20148" LastActivityDate="2019-02-06T04:37:55.910" Title="Role of the numerical flux in DG-FEM" Tags="|finite-element|discontinuous-galerkin|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24006" PostTypeId="2" ParentId="10068" CreationDate="2016-05-18T23:28:38.360" Score="-3" Body="&lt;p&gt;Here is a &lt;a href=&quot;https://github.com/paulknysh/blackbox&quot; rel=&quot;nofollow&quot;&gt;code&lt;/a&gt; that allows to efficiently optimize expensive black-box functions using multicore CPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A description of mathematics behind the code is given &lt;a href=&quot;http://arxiv.org/pdf/1605.00998.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20419" LastActivityDate="2016-05-18T23:28:38.360" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24007" PostTypeId="2" ParentId="6784" CreationDate="2016-05-18T23:32:04.940" Score="-1" Body="&lt;p&gt;I'd just add that one of the keys here is being able to scale optimization method on &lt;strong&gt;multicore CPUs&lt;/strong&gt;. If you can perform several function evaluations simultaneously, it gives you a speedup equal to a number of cores involved. Compare this to just using slightly more accurate response model, which makes you 10% more efficient or so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend to look at &lt;a href=&quot;https://github.com/paulknysh/blackbox&quot; rel=&quot;nofollow&quot;&gt;this code&lt;/a&gt;, it can be useful for people having access to many cores. A mathematics behind it is described in this &lt;a href=&quot;http://arxiv.org/pdf/1605.00998.pdf&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20419" LastEditorUserId="-1" LastEditDate="2016-05-19T19:15:31.620" LastActivityDate="2016-05-19T19:15:31.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24008" PostTypeId="1" CreationDate="2016-05-19T00:43:44.650" Score="0" ViewCount="1080" Body="&lt;p&gt;I am in a begginers programming course and we got a little project. I chose to simulate the three body problem using the Euler method. Even though the system is chaotic there are some special cases which produce symmetric trajectories. I get most of them, but the most important one doesn't come out. It's called figure 8 and supposed to be stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure that the initial conditions are fine. And the program should be as well, as I get the other expected results. &#xA;So the question is whether there is an error in my program or the Euler method doesn't work for this specific case. (I would then change to the Runge-Kutta method)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate any kind of help!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;&#xA;using namespace std; &#xA;&#xA;double const  ms=1.0000000000000000;&#xA;double const  me=1.0000000000000000;&#xA;double const  mm=1.0000000000000000;&#xA;double const  G=1.000000000000000000;&#xA;double const  dt=0.0001000000000000;&#xA;&#xA;/* 1=Sonne,2=Erde, 3=Mond */&#xA;&#xA;double fsx(double x1, double y1, double x2, double y2, double x3, double y3){&#xA;  double dv;&#xA;dv=G*(me/((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))*((x2-x1)/sqrt((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))+G*(mm/((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1)))*((x3-x1)/sqrt((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1))); &#xA;/*Projektion der Gravitationskraft auf Körper 1 auf x-Richtung */&#xA;  return dv;&#xA;}&#xA;&#xA;double fsy(double x1, double y1, double x2, double y2, double x3, double y3){&#xA;  double dv=G*(me/((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))*((y2-y1)/sqrt((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))+G*(mm/((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1)))*((y3-y1)/sqrt((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1))); /*Projektion der Gravitationskraft auf Körper 1 auf y-Richtung */&#xA;  return dv;&#xA;}&#xA;&#xA;double fex(double x1, double y1, double x2, double y2, double x3, double y3){&#xA;  double dv;&#xA;dv=G*(ms/((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))*((x2-x1)/sqrt((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))+G*(mm/((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1)))*((x3-x1)/sqrt((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1))); &#xA;/*Projektion der Gravitationskraft auf Körper 1 auf x-Richtung */&#xA;  return dv;&#xA;}&#xA;&#xA;double fey(double x1, double y1, double x2, double y2, double x3, double y3){&#xA;  double dv=G*(ms/((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))*((y2-y1)/sqrt((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))+G*(mm/((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1)))*((y3-y1)/sqrt((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1))); /*Projektion der Gravitationskraft auf Körper 1 auf y-Richtung */&#xA;  return dv;&#xA;}&#xA;&#xA;double fmx(double x1, double y1, double x2, double y2, double x3, double y3){&#xA;  double dv;&#xA;dv=G*(me/((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))*((x2-x1)/sqrt((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))+G*(ms/((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1)))*((x3-x1)/sqrt((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1))); &#xA;/*Projektion der Gravitationskraft auf Körper 1 auf x-Richtung */&#xA;  return dv;&#xA;}&#xA;&#xA;double fmy(double x1, double y1, double x2, double y2, double x3, double y3){&#xA;  double dv=G*(me/((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))*((y2-y1)/sqrt((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1)))+G*(ms/((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1)))*((y3-y1)/sqrt((y3-y1)*(y3-y1)+(x3-x1)*(x3-x1))); /*Projektion der Gravitationskraft auf Körper 1 auf y-Richtung */&#xA;  return dv;&#xA;}&#xA;&#xA;&#xA;double euler(double (f)(double,double,double,double,double,double), double x1, double y1, double x2, double y2, double x3, double y3, double vi){&#xA;  double vf=vi + f(x1, y1, x2, y2, x3, y3)*(dt);&#xA;  return vf;&#xA;}&#xA;&#xA;&#xA;&#xA;int main() {&#xA;float xf3, yf3, tmax;&#xA;  ofstream out(&quot;project1.dat&quot;);&#xA;&#xA;  ti=0.00000000000000000, xi1=-0.97000436, yi1=0.24308753, vxi1=0.466203685, vyi1=0.43236573, tmax=3.000000000000, xi2=-xi1, vxi2=vxi1, yi2=-yi1, vyi2=vyi1, xi3=0.0000000000000, vxi3=-2*vxi1, yi3=0.0000000000000000, vyi3=-2*vyi1;&#xA;  for(tf=0.0;tf&amp;lt;=tmax;tf=tf+dt){&#xA;    vxf1=euler(fsx,xi1,yi1,xi2,yi2,xi3,yi3,vxi1);&#xA;    vyf1=euler(fsy,xi1,yi1,xi2,yi2,xi3,yi3,vyi1);&#xA;    vxf2=euler(fex,xi2,yi2,xi1,yi1,xi3,yi3,vxi2); /* Gravitationskraft und damit auch fx und fy sind symmetrisch -&amp;gt; es reichen 2 Funktionen */&#xA;    vyf2=euler(fey,xi2,yi2,xi1,yi1,xi3,yi3,vyi2);&#xA;    vxf3=euler(fmx,xi3,yi3,xi2,yi2,xi1,yi1,vxi3);&#xA;    vyf3=euler(fmy,xi3,yi3,xi2,yi2,xi1,yi1,vyi3);&#xA;    xf1=xi1+(vxf1)*(dt);&#xA;    yf1=yi1+(vyf1)*(dt);&#xA;    xf2=xi2+(vxf2)*(dt);&#xA;    yf2=yi2+(vyf2)*(dt);&#xA;    xf3=xi3+(vxf3)*(dt);&#xA;    yf3=yi3+(vyf3)*(dt);&#xA;&#xA;    out &amp;lt;&amp;lt; tf &amp;lt;&amp;lt; &quot; &quot;&amp;lt;&amp;lt; xf1 &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; yf1 &amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt; vxf1 &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; vyf1&amp;lt;&amp;lt;  &quot; &quot;&amp;lt;&amp;lt; xf2 &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; yf2 &amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt; vxf2 &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; vyf2&amp;lt;&amp;lt; &quot; &quot;&amp;lt;&amp;lt; xf3 &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; yf3 &amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt; vxf3 &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; vyf3 &amp;lt;&amp;lt; endl;&#xA;    ti=tf;&#xA;vxi1=vxf1;&#xA;    vyi1=vyf1;&#xA;    vxi2=vxf2;&#xA;    vyi2=vyf2;&#xA;    vxi3=vxf3;&#xA;    vyi3=vyf3;&#xA;&#xA;    xi1=xf1;&#xA;    yi1=yf1;&#xA;    xi2=xf2;&#xA;    yi2=yf2;&#xA;    xi3=xf3;&#xA;    yi3=yf3;&#xA;  }&#xA;out.close();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The grafic (it's supposed to be a perfect 8): &lt;a href=&quot;https://i.stack.imgur.com/ELH9n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ELH9n.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20422" LastEditorUserId="9667" LastEditDate="2016-05-19T03:16:28.350" LastActivityDate="2016-05-20T17:36:27.587" Title="Three body problem in C++" Tags="|finite-difference|c++|simulation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24010" PostTypeId="2" ParentId="24008" CreationDate="2016-05-19T03:08:15.457" Score="2" Body="&lt;p&gt;So something must be up with your code. I know this because when I just wrote a C++ code to solve this problem with my simulation framework, I got the following result using Explicit Euler:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eSpyB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eSpyB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend breaking frequent computations into functions, especially functions that can do vector related math. You might consider representing the positions and velocities of each body with an array, like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double p1[2], v1[2]; // These represent position and velocity of body 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you could write functions to compute a vector magnitude and a unit vector like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double hypot( double x, double y ){&#xA;  double t; // modified based on code from wikipedia on &quot;Hypot&quot; page&#xA;  x = abs(x);&#xA;  y = abs(y);&#xA;  if( x == 0.0 ){ return y; }&#xA;  if( y == 0.0 ){ return x; }&#xA;  t = min(x,y);&#xA;  x = max(x,y);&#xA;  t = t/x;&#xA;  return x*sqrt(1.0+t*t);&#xA;}&#xA;&#xA;double mag( double * vec ){&#xA;  return hypot(vec[0],vec[1]);&#xA;}&#xA;&#xA;void toUnit( double * vec ){&#xA;  double mag_ = mag(vec);&#xA;  vec[0] /= mag_; vec[1] /= mag_;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And use the functions by simply doing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double r[2] = {p1[0]-p2[0],p1[1]-p2[1]};&#xA;double magr = mag(r);&#xA;toUnit(r);&#xA;double accel_mag = G*m1/(magr*magr);&#xA;double dv2dt[2] = {accel_mag*r[0],accel_mag*r[1]};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Getting a habit of building frequently used computations into independent functions will not only make it take less time to build new codes, but debugging your problems will also be easier since if you have a mistake in some of the math, you only have to fix maybe one function even though this function is used in many places.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-05-20T17:36:27.587" LastActivityDate="2016-05-20T17:36:27.587" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24011" PostTypeId="1" CreationDate="2016-05-19T10:07:23.057" Score="2" ViewCount="1059" Body="&lt;p&gt;I have the task to fit a rotated rectangle of known size into an image like &lt;a href=&quot;https://i.stack.imgur.com/5HIuV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5HIuV.png&quot; alt=&quot;this&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a synthetic test case, in the real application, everything is rather blurred. The rectangle has to cover as much as possible of the black region, while covering as little as possible of the grey region  while the white region is ignored. I don't always see all corners, it could e.g. be possible that I just have the upper half of the rectangle and a much larger white area. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've so far implemented a straight forward gradient descent to search in the 3d space (x,y, rotation) and optimize for the difference of black and gray pixels within the current pose of the rectangle. This approach is not very fast so I'd like to ask for alternatives. &lt;/p&gt;&#xA;" OwnerUserId="19085" LastActivityDate="2016-05-20T20:09:02.483" Title="Fitting rectangle to segments in image" Tags="|optimization|image-processing|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24012" PostTypeId="2" ParentId="6784" CreationDate="2016-05-19T13:15:51.060" Score="2" Body="&lt;p&gt;I think you should start with:&#xA;GECCO Workshop on Real-Parameter Black-Box Optimization Benchmarking (BBOB 2016)&#xA;&lt;a href=&quot;http://numbbo.github.io/workshops/index.html&quot; rel=&quot;nofollow&quot;&gt;http://numbbo.github.io/workshops/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will find many different algorithms that have been used in previous competitions, and that&#xA;have been compared on a common basis.&#xA;If you start elsewhere, you will soon drown in the hundreds of papers that claim their methods &#xA;and algorithms perform better than others with little actual evidence for those claims. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Until recently, it was, to be frank, a disgraceful state of affairs and all power to INRIA, &#xA;GECCO and many others for the effort they have made in establishing a framework for rational &#xA;comparisons.&lt;/p&gt;&#xA;" OwnerUserId="16571" LastActivityDate="2016-05-19T13:15:51.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24013" PostTypeId="2" ParentId="23845" CreationDate="2016-05-19T15:01:44.997" Score="2" Body="&lt;p&gt;It took me quite some time to figure out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the convexity assumption is removed, then the following equation (line 9, page 108) no longer holds:&#xA;$$\partial L_\rho(x^{k+1},z^k,y^k) = \partial f(x^{k+1}) + A^Ty^k + \rho A^T (Ax^{k+1} + Bz^k - c).$$&#xA;More accurately, this equation only holds if $\partial f(x^{k+1}) \neq \emptyset$ (which always holds if $f$ is convex). Similarly for $g$.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2016-05-19T15:01:44.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24014" PostTypeId="1" CreationDate="2016-05-19T15:23:57.170" Score="-1" ViewCount="530" Body="&lt;p&gt;Is there a method to check for unconditional stability or positive-definiteness of large matrices in MATLAB?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I know that matrices with property M (positive main diagonal elements and negative off-diagonal elements, with the main diagonal elements in each row or column being larger than the summation of off-diagonal elements) result in an unconditionally stable system of equations. But I cannot decide for other matrices.&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-05-20T04:47:12.513" Title="Stability of matrix equations in MATLAB" Tags="|matlab|stability|numerics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24016" PostTypeId="2" ParentId="24011" CreationDate="2016-05-19T17:25:27.650" Score="2" Body="&lt;p&gt;You might be able to do better than simple gradient descent using an algorithm like &lt;a href=&quot;https://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm&quot; rel=&quot;nofollow&quot;&gt;BFGS&lt;/a&gt;, which uses the history of the gradients as well as the approximate solutions to try and approximate the Hessian of the objective functional. A good explanation of how the algorithm works, modifications for large systems, modifications in the event of non-convexity, etc. can be found in &lt;a href=&quot;https://books.google.com/books?isbn=0387400656&quot; rel=&quot;nofollow&quot;&gt;Nocedal and Wright&lt;/a&gt;. BFGS is one of a more general family of &lt;a href=&quot;https://en.wikipedia.org/wiki/Quasi-Newton_method&quot; rel=&quot;nofollow&quot;&gt;quasi-Newton methods&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of your parameters (the orientation) is defined not in a vector space but rather in the group $S^1$; moreover, you get the same fit for some orientation $\theta$ as for $\theta + \pi$. So there's some degeneracy in the solution due to the fact that the image you're fitting has a discrete symmetry, namely reflection in one axis. So the solution might not be quite as simple as just using BFGS instead of gradient descent, but it's a good place to start.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-05-19T17:25:27.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24017" PostTypeId="1" CreationDate="2016-05-19T19:53:08.940" Score="2" ViewCount="224" Body="&lt;p&gt;Could anyone suggest the most computationally efficient method for finding amplitude at a given frequency having a noisy wide band signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific about a task. I have some physical dynamical system. I'm introducing sinusoidal input with exact frequency and amplitude. And looking for the amplitude at the same frequency on the output signal. While output signal is subject to noise and affected by other physical inputs at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is required to detect some aspects of physical model. Experiments shows that this kind of test is sensitive to physical changes I want to detect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I quite limited in resources. I only have small 32 bit MCU that is capable only of fixed-point arithmetic. Or very slow emulated floating point. That's why I'm asking about most effective solutions to the task.&lt;/p&gt;&#xA;" OwnerUserId="20433" LastEditorUserId="9667" LastEditDate="2016-09-22T19:33:54.050" LastActivityDate="2020-04-06T13:02:20.343" Title="Amplitude at a given frequency in a wide band signal" Tags="|fourier-transform|signal-processing|fixed-point|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24018" PostTypeId="2" ParentId="24014" CreationDate="2016-05-19T20:27:15.127" Score="0" Body="&lt;p&gt;I suspect that you also want a method which is not to computational expensive. So depending on whether or not you think that this is not too computational expensive you could use the following. Look at whether the &lt;a href=&quot;http://mathworld.wolfram.com/PositiveDefiniteMatrix.html&quot; rel=&quot;nofollow&quot;&gt;determinants of each (upper left) sub-matrix&lt;/a&gt; is bigger than zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With upper-left submatrices I mean the following; for $A\in \mathbb{R}^n$ then,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\det\left(A[1,\cdots,k;1,\cdots,k]\right) &amp;gt; 0\quad \forall\; k=1,\cdots,n.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also has the advantage that it can be terminated whenever you encounter a determinant which is smaller or equal to zero. Depending on how many CPU core you have available you could speed this up by do this in parallel with &lt;code&gt;parfor&lt;/code&gt;. Only I do not know if you can still terminate when you encounter a bad determinant.&lt;/p&gt;&#xA;" OwnerUserId="7031" LastActivityDate="2016-05-19T20:27:15.127" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24019" PostTypeId="1" CreationDate="2016-05-19T23:55:10.893" Score="1" ViewCount="43" Body="&lt;p&gt;in systems biology why is the use of delay differential equations better than ordinary differential equations i.e. compartment models in delay modelling? is there a data independence angle in models parameter estimation?&lt;/p&gt;&#xA;" OwnerUserId="20435" LastEditorUserId="20435" LastEditDate="2016-05-20T02:58:29.633" LastActivityDate="2016-05-20T02:58:29.633" Title="How come the use of delay differential equations in model parameter estimation better than ordinary differential equations?" Tags="|pde|ode|computational-biology|" AnswerCount="0" CommentCount="4" ClosedDate="2016-05-20T21:48:15.410" ContentLicense="CC BY-SA 3.0" />
  <row Id="24020" PostTypeId="2" ParentId="24014" CreationDate="2016-05-20T04:47:12.513" Score="3" Body="&lt;p&gt;In general, your best bet is to understand where your matrix came from and be able to show that it is theoretically positive definite.  The computational approaches to showing that a matrix is positive definite (including eigenvalues, Cholesky factorization, and checking determinants of principal minors) are all impractical for large sparse matrices.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your matrices are dense and of merely medium size (you can easily fit them into memory) then checking for symmetry and then computing the Cholesky factorization is probably the most practical way to check for positive definiteness.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-05-20T04:47:12.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24021" PostTypeId="1" CreationDate="2016-05-20T08:49:17.140" Score="2" ViewCount="3546" Body="&lt;p&gt;I'm currently developing a simulation and I intend to use VTK for visualisation. Furthermore, I fell a bit stupid right now, as I do not find a good example on how to generate the data for VTK. There are many examples that start from reading the data and then do some visualisation with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BUT what is with the other side? I will have a FEM mesh with up to billions of nodes on many computing nodes. How do people feed this into &quot;something&quot; (probably a file) suitable as input for VTK? Is it possible with VTK to already do some pre-processing in the distributed simulation application in order to reduce the amount of data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think, a link to a good example or tutorial would be more than helpful. Currently I'm just lost in the amount of information available.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-05-21T17:49:57.693" Title="How do you generate data for VTK?" Tags="|vtk|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24023" PostTypeId="2" ParentId="11665" CreationDate="2016-05-20T11:15:06.030" Score="1" Body="&lt;p&gt;To give some background to the answer given by LutzL, here is how you'd set up your linear system to start with. You want to find an image $L$ whose gradient $\nabla L$ is as close as possible to your target vector field $T=(T_x, T_y)$. In other words:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\arg\min_{L} \left\|\nabla L - T \right\|&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll actually end up solving the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Delta L = \frac{\partial T_x}{\partial x} + \frac{\partial T_y}{\partial y}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\Delta$ is the Laplacian operator. Here's why. Let's start from the beginning using matrix algebra (with a slightly imperfect notation):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\arg\min_L \left\|&#xA;\begin{bmatrix}&#xA;D_x \\&#xA;D_y&#xA;\end{bmatrix}&#xA;L -&#xA;\begin{bmatrix}&#xA;T_x \\&#xA;T_y&#xA;\end{bmatrix}&#xA;\right\|.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $D_x$ and $D_y$ are $x$ and $y$ derivative operator matrices, $L$ is your image represented as a column vector, and $T_x$ and $T_y$ are also in vector form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then proceed minimising the above as you would with any least-squares problem, which gives you the linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;(D_x D_x + D_y D_y)L = D_x T_x + D_y T_y&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where indeed $D_x D_x + D_y D_y$ is the matrix form of the Laplacian operator $\Delta$ used above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Turns out this is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Circulant_matrix?oldformat=true#Applications&quot; rel=&quot;nofollow&quot;&gt;circulant matrix&lt;/a&gt;, which has the property that the linear system can be solved using the FFT as described by LutzL.&lt;/p&gt;&#xA;" OwnerUserId="20441" LastActivityDate="2016-05-20T11:15:06.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24024" PostTypeId="2" ParentId="24021" CreationDate="2016-05-20T12:33:10.367" Score="7" Body="&lt;p&gt;If you want to keep your FE code independent of the VTK API, writing a file that VTK can read is the most straightforward way to do that. The file formats are&#xA;documented here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.vtk.org/wp-content/uploads/2015/04/file-formats.pdf&quot;&gt;http://www.vtk.org/wp-content/uploads/2015/04/file-formats.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The legacy format is simpler but the XML format provides access to some of the newer features of VTK. Creating VTK objects from files in either of these formats is straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Writing a file also allows you to use visualization applications built using&#xA;the VTK API instead of writing your own, for example, ParaView:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.paraview.org/&quot;&gt;http://www.paraview.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-05-20T12:33:10.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24026" PostTypeId="1" AcceptedAnswerId="24032" CreationDate="2016-05-20T17:11:43.147" Score="5" ViewCount="3677" Body="&lt;p&gt;I'm having trouble understanding how GMRES works. I've read the part in Saad's book and a few others but still I am confused. Can someone provide me a numerical example to understand it better? Or if that's not possible maybe a walk through the algorithm step by step?&lt;/p&gt;&#xA;" OwnerUserId="20447" LastEditorUserId="1804" LastEditDate="2016-05-21T15:29:38.487" LastActivityDate="2021-03-17T06:06:25.607" Title="A numerical GMRES example" Tags="|linear-algebra|krylov-method|gmres|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24027" PostTypeId="1" CreationDate="2016-05-20T17:13:44.133" Score="1" ViewCount="64" Body="&lt;p&gt;I am trying to work out a second order or higher order correction for the method of Spectral deferred Correction (SDC). Specifically using as a corrector a second order or third order multi-step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In SDC, the timestep $[t_{n},t_{n+1}] $ is subdivided in m quadrature nodes $[t_{n,1},t_{n,2},...,t_{n,m-1},t_{n,m}] $. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ODE problem is written in its integral form:&#xA;$$ \begin{equation}&#xA;\label{eq:ODEPicard}&#xA;\begin{array}{ccl}&#xA;\phi(t) &amp;amp; = &amp;amp; \phi_{0} + \displaystyle\int_{0}^{t} F(\tau,\phi(\tau))d\tau&#xA;\end{array}&#xA;\end{equation}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And later rewritten in function of the error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \begin{equation}&#xA;\begin{array}{ccl}&#xA;\label{eq:CorrectionEmeasure}&#xA;\delta(t) &amp;amp; = &amp;amp; \phi_{0} + \displaystyle\int_{0}^{t} [F(\tau,\tilde{\phi}+\delta(t)) - F(\tau,\tilde{\phi})]d\tau +   E(t,\tilde{\phi})&#xA;\end{array}&#xA;\end{equation}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\tilde{\phi} $ denotes the approximate solution. This previous eq is discretized and based on the definitions of the error $\delta(t)$ and the residual $E(t,\tilde{\phi})$ can be discretize by a r-multi-step method as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \begin{equation}&#xA;\label{eq:CorrectionMultiStepDiff1}&#xA;\begin{array}{ccl}&#xA;\displaystyle\sum^{r}_{j=0}{\{\alpha_{j}\tilde{\phi}^{[k]}_{i-j} + \displaystyle\sum_{w=0}^{j} \alpha_{w} I^{i-j}_{i-j-1}(F(\tilde{\phi}^{[k-1]})) \}}&#xA;-  \{F(t_{i-1},\tilde{\phi}_{i-1}^{[k]} ) - F(t_{i-1},\tilde{\phi}_{i-1}^{[k-1]})\} = 0&#xA;\end{array}&#xA;\end{equation} $ \&#xA;Being $I^{i+1}_{i}(F)$ an interpolatory quadrature rule&#xA;$\begin{equation}&#xA;\label{eq:QuadratureInterpolatory}&#xA;  \int^{t_{i+1}}_{t_{i}} F(\tau,\phi(\tau))d\tau \approx I^{i+1}_{i}(F) = \int^{t_{i+1}}_{t_{i}} p(\tau)\omega(\tau)d\tau = \displaystyle\sum_{j=1}^{n} \omega_{j}F(t_{j})&#xA;\end{equation}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, this as every multistep method needs an initialization in the first step. What I have done is to resolve that initialization by RK2 or midpoint rule. But furthermore, the first subinterval $[t_{n,1},t_{n,2}]$ require values of the previous r-1 subnodes values. This is a problem, because as it is pointed out in &lt;a href=&quot;http://people.cs.vt.edu/~asandu/Public/For/Michael/Layton_2008_Picard.pdf&quot; rel=&quot;nofollow&quot;&gt;Layton&lt;/a&gt; [1]. $\delta(t)$ is continous in the interval $[t_{n},t_{n+1}]$ but may be discontinous in the previous one. I do not totally understand why this happen, so if someone could explain me this I would be so grateful, I think it is because of the quadrature term but I don't still see it. Well, but she say that this produces a lack of smoothness in the solution and the order or correction is only 1 per step instead of r for an r-step.  And later in that paper she proposes some solutions for this, (page 5), but it is not clear what she exactly mean by &quot;fixed starting values&quot; and &quot;variable starting values&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have tried to keep the order of accuracy of the corrector (specially I have focused in Second order Adams Bashforth) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\displaystyle \beta_{1} = \frac{\Delta t_{i-1}^2}{2\Delta t_{i-2}} \\&#xA;\displaystyle \beta_{2} = \frac{2\Delta t_{i-1}\Delta t_{i-2} + \Delta t_{i-1}^2 }{2\Delta t_{i-2}}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \begin{equation}&#xA; \phi_{i}^{[k+1]} = \phi_{i-1}^{[k+1]} + \beta_{1}[F(t_{i-1},\phi_{i-1}^{[k+1]}) - F(t_{i-1},\phi_{i-1}^{[k]})] + \beta_{2}[F(t_{i-2},\phi_{i2}^{[k+1]}) - F(t_{i-2},\phi_{i-2}^{[k]})] +I^{i}_{i-1}(\phi^{[k]}) &#xA;\end{equation}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the first I tried it was the use of midpoint rule in the first subinterval, but as it is pointed out in &lt;a href=&quot;https://www.math.uh.edu/~jingqiu/papers/idc1.pdf&quot; rel=&quot;nofollow&quot;&gt;IDC&lt;/a&gt; [2] this lack of smoothness (this concept is also redefined in this paper), it has a degree of smoothness one, in the discrete sense of the numerical set. And later I have tried to use other initialization but also lacking of smoothness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is does anybody has an idea how to obtain a second or higher order smoothness in the discrete sense based on a combination of Multistep and other method (maybe multistage). Or how to initialize at each subinterval the multistep method not to suffer order reduction due to the lack of smoothness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. If someone could clarify me also the comment that Layton does in the page 5 of [1] about continuity of the error it would be great too!&lt;/p&gt;&#xA;" OwnerUserId="20448" LastActivityDate="2016-05-20T17:13:44.133" Title="Second and Higher Order Order Corrector in Spectral Deferred Correction" Tags="|pde|discretization|time-integration|spectral-method|collocation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24028" PostTypeId="2" ParentId="24011" CreationDate="2016-05-20T20:09:02.483" Score="1" Body="&lt;p&gt;You can speed up your process by taking multi-scale approach, i.e. perform template matching on coarse scale, then upscale and refine parameters. Of course, the number of levels and step size depends on your problem and it may be tricky to determine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can take a more analytical approach by first extracting edges from the image (I guess Canny would work best in this case) and then use either &lt;strong&gt;Hough Transform&lt;/strong&gt; or &lt;strong&gt;RANSAC&lt;/strong&gt; (RANdom SAmple and Concensus) approach to extract all lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using some assumptions about the rectangle color (black or white foreground to grey background), you can tell which side of each line goes inside the rectangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, having at least 2 lines will constrain your optimization problem enough so that you can solve this without optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach may target for rectangle corners, i.e. using &lt;strong&gt;Harris Corner Detector&lt;/strong&gt; and then determining rectangle orientation from patch around each corner.&lt;/p&gt;&#xA;" OwnerUserId="4591" LastActivityDate="2016-05-20T20:09:02.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24031" PostTypeId="2" ParentId="24026" CreationDate="2016-05-21T14:55:46.213" Score="6" Body="&lt;p&gt;The intuition behind all Krylov's subspace methods is the following. Given a square matrix $A$ and a compatible vector $b$, there exists a unique monic polynomial $p$ such that $p(A)b = 0$. Mathematically, this is a consequence of the axiom of choice and Cayley's theorem which ensures that $q(A) = 0$ where $q$ is the characteristic polynomial of $A$. If $A$ is nonsingular, then $p(A)b=0$ can be transformed in $A w(A)b = b$ where $w$ is a polynomial closely related to $p$. In short, the solution $x = A^{-1}b$ can be written in the form $x = w(A)b$. Typically, $w$ has degree $n-1$, and so it is not suitable for practical computations. But this expression for $x$ is &lt;em&gt;the&lt;/em&gt; reason why we seek approximations of similiar type, i.e. $x_k \in K_k(A,b)$.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2016-05-21T15:02:06.763" LastActivityDate="2016-05-21T15:02:06.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24032" PostTypeId="2" ParentId="24026" CreationDate="2016-05-21T15:24:24.183" Score="11" Body="&lt;p&gt;GMRES is indeed one of the hardest to understand Krylov methods. As you correctly state, the algorithm computes in each step $m$ a new approximation $x^m$ to the solution of $Ax=b$ as a minimizer of $\|b-Ax\|_2$ over the Krylov space $K_m(A,b) = \mathrm{span}\{b,Ab,A^2b,\dots,A^{m-1}b\}$. (I'm assuming for simplicity that $x^0 =0$). Clearly, if  $A\in\mathbb{R}^{n\times n}$, then $x^n$ is the desired solution. The whole point of GMRES is to compute each minimizer $x^{m}$ via the Arnoldi process by reusing as much information from the previous steps as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivation proceeds in two steps. First, recall that the Arnoldi process (which is an algorithm analogous to Gram-Schmidt to compute an orthonormal basis of $K_m(A,b)$) yields after $m$ steps the orthonormal vectors $q_1,\dots,q_{m+1}$ with $q_1 = b/\beta$, $\beta = \|{b}\|$, and scalars $h_{ij}$, $1\leq i\leq j+1\leq m+1$ such that &#xA;\begin{equation}&#xA;    AQ_m = Q_m H_m + h_{m+1,m}q_{m+1}e_m^T =: Q_{m+1}\bar H_m,&#xA;\end{equation}&#xA;where $Q_{m+1}=[q_1,\dots,q_{m+1}] \in \mathbb{R}^{n\times (m+1)}$, $\bar H_m = (h_{ij})_{i,j} \in \mathbb{R}^{(m+1)\times m}$, and $e_m$ is the $m$th unit vector. Since every $x\in K_m(A,b)$ can be written as a linear combination of $q_1,\dots,q_m$, i.e., $x = Q_m\xi$ for some $\xi\in\mathbb{R}^m$, we have that&#xA;\begin{equation}&#xA;    \begin{aligned}[t]&#xA;        b-Ax         &amp;amp;= b - AQ_m \xi\\&#xA;        &amp;amp;= \beta q_1 - Q_{m+1}\bar H_m \xi \\&#xA;        &amp;amp;= Q_{m+1}(\beta e_1 - \bar H_m\xi).&#xA;    \end{aligned}&#xA;\end{equation}&#xA;Since $Q_{m+1}$ is unitary, we hence have that&#xA;\begin{equation}&#xA;   \min_{x\in K_m(A,b)} \|{b-Ax}\|_2^2 = \min_{\xi\in\mathbb{R}^m} \|{\beta e_1 - \bar H_m\xi}\|_2^2.&#xA;\end{equation}&#xA;The right-hand side is a (typically much) smaller least squares problem that can be solved by QR decomposition of $\bar H_m$: If $\bar H_m = GR$, where $G$ is unitary (I'm not using $Q$ here to avoid confusion) and $R$ is upper triangular, then &#xA;$$\|\bar H_m\xi - \beta e_1\|^2_2 = \|\beta G^T e_1 -R\xi\|^2_2.$$&#xA;Since $\bar H_m\in\mathbb{R}^{(m+1)\times m}$, the matrix $R$ has the form&#xA;$R=\begin{pmatrix}R_m\\0\end{pmatrix}$ with $R_m\in\mathbb{R}^{m\times m}$ invertible. Furthermore, we can partition $G^T e_1 =: (d_m,\rho_m)\in\mathbb{R}^{m+1}$ with $d_m\in\mathbb{R}^m$ and $\rho_m \in \mathbb{R}$. Thus,&#xA;$$\|\beta G^T e_1 -R\xi\|^2_2 = \|R_m\xi - \beta d_m\|_2^2 + |\beta \rho_m|^2.$$&#xA;The minimum is therefore attained at $\xi^m := \beta R_m^{-1} d_m$, for which the residual is $\beta |\rho_m|$. This residuum -- which is used as a stopping criterion -- can therefore be evaluated without computing $x^m$.&#xA;Furthermore, $\bar H_m$ already has upper Hessenberg form, hence this can be done quite fast by a single sweep of Givens rotations (whose product -- in matrix form -- is exactly $G^T$). The approximation $x^m$ is then simply obtained from $x^m =Q_m\xi^m$ (but this only needs to be done at the very end).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is exactly what the GMRES algorithm does, but instead of starting from scratch each time, it alternates a step of the Arnoldi process and an update of the QR factorization to add a column: In step $m$,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A new set of $q_{m+1}$ and $h_{1,m},\dots,h_{m+1,m}$ are computed using the Arnoldi process. (This gives a new column of $Q$ as well as of $\bar H$.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The extended matrix $\bar H_{m+1}$ is brought to upper triagonal form by applying $m+1$ Givens rotations to the new column -- of which the first $m$ have already been obtained during the previous steps, and only the last one   has to be computed to zero out the bottom-most element. (This gives a new column of $R$ in $\bar H = GR$; note that the new rotation leaves the previous columns unchanged -- this is the key of GMRES.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;These Givens rotations are also applied to the right-hand side $e_1\in \mathbb{R}^{m+1}$ to obtain the vector $(d^m,\rho_m)$. Again, only the newest rotation needs actually be applied, because the older ones were already applied in the previous iterations. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If we are satisfied with the current relative residuum $|\rho_m|$ (or have reached the maximum number of iterations), the final iterate is computed via $x^m = \beta Q_m R_m^{-1}d^m$ (which is cheap since $R_m$ is small and upper triangular).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since the steps become increasingly more expensive with $m$ (more data has to be stored, and the number of arithmetic operations grows quadratically), the algorithm is usually &lt;em&gt;restarted&lt;/em&gt; after $M$ steps: $x^M$ is computed as above, and the procedure starts over with $m=0$ and a new initial point $x^0=x^M$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going into more detail here would get a bit long, but let me know and I'll try to fill in any gaps.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-05-25T19:42:27.493" LastActivityDate="2016-05-25T19:42:27.493" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24034" PostTypeId="2" ParentId="24021" CreationDate="2016-05-21T16:17:46.503" Score="2" Body="&lt;p&gt;As said above, the VTK legacy format is simple and straightforward and you should be able to write a subroutine to create the output file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in your case, you are dealing with billions of nodes, resulting in a huge (or several) file. For such amount of data, you must use compressed binary format. I assume you run your code in parallel on a cluster. The XML format provides interesting parallel features for reading multiple output from different processes for instance and has been especially designed to read data from massive parallel computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know if VTK will suit your needs. You can take a look at the HDF5 library &lt;a href=&quot;https://www.hdfgroup.org/HDF5/&quot; rel=&quot;nofollow&quot;&gt;https://www.hdfgroup.org/HDF5/&lt;/a&gt; which can be used with MPI to perform efficient I/O operations. It provides C and Fortran interfaces. NETCDF or CGNS libraries are also good alternatives. Depending on your data, you can choose to use an existing format or develop your own driver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, about visualization software, you should take a look at Paraview mentioned above and also Visit &lt;a href=&quot;https://wci.llnl.gov/simulation/computer-codes/visit/&quot; rel=&quot;nofollow&quot;&gt;https://wci.llnl.gov/simulation/computer-codes/visit/&lt;/a&gt; which both support massive data visualization through parallel pipelines and remote cluster access. Paraview is built with VTK so it should offer optimizations to read VTK files. Visit uses filters and ghost nodes to speed up the visualization displaying only the required cells. Both have a good documentation I advise you to read.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="20454" LastEditDate="2016-05-21T17:49:57.693" LastActivityDate="2016-05-21T17:49:57.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24038" PostTypeId="1" CreationDate="2016-05-22T13:34:20.707" Score="1" ViewCount="64" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GAtlT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GAtlT.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;The figure below shows the PCA projections of inputs which are 14 meteorological features, (i.e. wind, temperature, humidity, pressure, and so on.)&#xA;I would like to use any technique to make it more separable than this, The ISOMAP method is also used instead of PCA, but it gave a non-separable distribution as well! Any suggestions would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17524" LastActivityDate="2016-05-22T13:34:20.707" Title="How to make the following data separable for the classification into three classes?" Tags="|machine-learning|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24039" PostTypeId="2" ParentId="3341" CreationDate="2016-05-22T23:21:19.297" Score="9" Body="&lt;p&gt;The integral in question is also known as the Boys function, after the British chemist Samuel Francis Boys who introduced its use in the early 1950s. A few years ago, I needed to compute this function in double precision, as fast as possible but accurately. I managed to achieve a relative error on the order of $10^{-15}$ across the entire input domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is generally advantageous to use different approximations for small and large arguments, where the optimal switch-over between &quot;large&quot; and &quot;small&quot; is best determined experimentally, and is in general a function of $m$. For my code, I defined &quot;small&quot; arguments as those satisfying condition $a \le m + 1{1\over 2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large arguments, I compute &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{F}_m(a) = {1\over 2}\gamma\left(m + {1\over 2}, a\right) \times p \times p, \space \space p = a^{-{1\over 2}\left(m+ {1\over 2}\right)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This order of operations avoids premature underflow. As we need only the lower incomplete gamma function of half-integer orders here rather than a fully general lower incomplete gamma function, it is advantageous from a performance perspective to compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\gamma \left(m + {1\over 2}, a\right) = \Gamma \left(m + {1\over 2}\right) - \Gamma\left(m + {1\over 2}, a\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using tabulated values of $\Gamma \left(m + {1\over 2}\right)$ and computing $\Gamma\left(m + {1\over 2}, a\right)$ according to&#xA;&lt;a href=&quot;https://math.stackexchange.com/questions/724068/expressing-upper-incomplete-gamma-function-of-half-integer-order-in-terms-of-gam&quot;&gt;this answer&lt;/a&gt;, while carefully avoiding the issue of subtractive cancellation through use of a fused multiply-add operation. A potential further optimization is to observe that for sufficiently large $a$, $\gamma \left(m + {1\over 2}, a\right) = \Gamma \left(m + {1\over 2}\right)$ to within a given floating-point precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For small arguments, I started with a series expansion for the lower incomplete gamma function from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;A. Erdelyi, W. Magnus, F. Oberhettinger, and F.G. Tricomi, &quot;Higher &#xA;Transcendental Functions, Vol. 2&quot;. New York, NY: McGraw-Hill 1953&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and modified it to compute the Boys function $\mathrm{F}_{m}(a)$ as follows (truncating the series when the term is sufficiently small for a given precision):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{F}_{m}(a) = {1\over 2}\frac{1}{m + {1\over 2}}\exp(-a)\left(1+\sum_{n=1}^{\infty}\frac{a^{n}}{(1 + m + {1\over 2}) \times\space  ...\space \times (n + m + {1\over 2})}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also interesting and potentially important special cases for low orders of the Boys function, particularly $m = 0, 1, 2, 3$. First, we have $\mathrm{F}_{0}(a) = \sqrt{\frac{\pi}{4a}}\mathrm{erf}\left(\sqrt{a}\right)$, where $\mathrm{erf}$ is the error function provided in Fortran 2008 as the elemental function &lt;code&gt;ERF&lt;/code&gt; and in C/C++ as the standard library functions &lt;code&gt;erf&lt;/code&gt; and &lt;code&gt;erff&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For fast computation when $m = 1, 2, 3$, I use custom minimax polynomial approximations for small arguments, say $a \lt {2{1\over 2}}$, and forward recursion $\mathrm{F}_{m}(a) = \frac{1}{2a}\left(\left(2m-1\right)\mathrm{F}_{m-1}(a) - \exp(-a)\right)$, for large ones, where issues with subtractive cancellation in the latter are mitigated by the use of fused multiply-add operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where function values have to be computed for a given $a$ across multiple orders $m$, one would want to compute the function value for the highest value of $m$ directly, i.e. as discussed above, then use the numerically stable backward recursion $\mathrm{F}_{m-1} = \frac{1}{2m-1} \left(2a \space \mathrm{F}_{m}(a) + \exp\left(-a\right)\right)$ to compute all the other function values.&lt;/p&gt;&#xA;" OwnerUserId="20458" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-06-06T17:51:19.773" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24041" PostTypeId="1" AcceptedAnswerId="24046" CreationDate="2016-05-23T10:42:19.670" Score="0" ViewCount="446" Body="&lt;p&gt;i have a question about a FE problem im working on.&#xA;I made a finite element model of an linear elastic block of material (double striped block) attached with a rigid connection to the environment (colored block). A force is applied in the bottom right node. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nodal displacements are calculated according to the Direct Stiffness Method.&#xA;The system of equations is simply K&lt;em&gt;U = F or U = C&lt;/em&gt;F with C=K^-1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The displacement of the top right node needs to be constrained to a specified trajectory. By adding lagrange multipliers to the global stiffness matrix i have succesfully constrained the possible displacements of the top right node to a line. To be clear the force is the input, the displacement is the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The new system of equations is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bwwxo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bwwxo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to extend the problem and therefore it is needed for me to specify a nonlinear trajectory. &#xA;As far as i know the direct stiffness method with lagrange multipliers does not work in this case for it only allows to add constraints of the form y = a*x+b.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best method to add a nonlinear displacement constraint to this system of linear equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been searching and reading papers in the subject but so far i have not found a sound answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope you guys can help me and i am lloking forward to your answer,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;J.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Dglzb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dglzb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20465" LastActivityDate="2016-05-23T18:08:12.480" Title="FEM, Direct Stiffness Method with a nonlinear displacement constraint in one node" Tags="|finite-element|nonlinear-equations|constraints|stiffness|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24042" PostTypeId="1" CreationDate="2016-05-23T13:54:40.040" Score="0" ViewCount="1327" Body="&lt;p&gt;I want to solve the 2D Laplace's equation:&#xA;$$ \frac{\partial^2 T}{\partial x^2 } + \frac{\partial^2 T}{\partial y^2 } = 0 $$&#xA;with boundary conditions: T(x=0)=T(x=1)=T(y=1)=0 and T(y=0)=1 on a 1024x1024 discrete grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I do this?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it the Discrete Sine Transform I should use? If so which of them?&lt;/p&gt;&#xA;" OwnerUserId="3622" LastActivityDate="2016-05-27T20:41:43.410" Title="How do I solve Laplace's equation in 2D using spectral methods?" Tags="|pde|numerical-analysis|fourier-analysis|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24043" PostTypeId="1" AcceptedAnswerId="24050" CreationDate="2016-05-23T15:45:21.050" Score="1" ViewCount="111" Body="&lt;p&gt;I have a system with below coefficient matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ C = \begin{pmatrix} A &amp;amp; B^T \\ B &amp;amp; D \end{pmatrix},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, $A$ and $D$ are square and positive definite. Furthermore, if $B$ be square, then $B$ will be positive definite (it is optional). It is possible that the condition number of $C$ will be large. How can solve this system to avoid numerical problem as much as possible.&lt;/p&gt;&#xA;" OwnerUserId="20467" LastEditorUserId="1804" LastEditDate="2016-05-27T15:42:07.643" LastActivityDate="2016-06-01T19:06:28.057" Title="Numerical method for solving a system with positive definite blocks" Tags="|linear-algebra|linear-solver|condition-number|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24046" PostTypeId="2" ParentId="24041" CreationDate="2016-05-23T17:39:55.600" Score="0" Body="&lt;p&gt;The key point to recognize is that the Lagrange multiplier method is applicable to &#xA;both nonlinear and linear constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you can write your nonlinear constraint equation as&#xA;$$g(U_s) = 0$$&#xA;you have a set of N+1 nonlinear equations to solve as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ K_{ss}U_s + \lambda\frac{\partial g}{\partial U_s} = F_s$$&#xA;$$ g(U_s) = 0$$&#xA;where N is the number of finite element equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These can be solved for $U_s$ and $\lambda$ using the Newton-Raphson method.&#xA;If you are not already familiar with using Newton-Raphson for solving a system of&#xA;nonlinear equations, you can find many references on the web.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introductory FEM courses almost always emphasize linear formulations and so&#xA;present the Lagrange multiplier technique as you show above. In intermediate rigid body dynamics courses, in either physics or engineering departments, the equations of motion and constraints are almost always nonlinear, so the Lagrange multiplier method is presented from that point of view. So, if you want to learn more, I suggest taking a look at a dynamics text used in such a course.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2016-05-23T18:08:12.480" LastActivityDate="2016-05-23T18:08:12.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24047" PostTypeId="1" CreationDate="2016-05-23T20:07:07.940" Score="3" ViewCount="117" Body="&lt;p&gt;I'm currently developing a large simulation that will run on many nodes, and, as far as possible, on GPUs. I'm now considering to build it using libraries from the Trilinos project. I would for sure need Kokkos, Tpetra and some linear solver libraries. I also need to deal with a mesh and with construction of FEM equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have the following pros and cons:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pro:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I get a large number of libraries.&lt;/li&gt;&#xA;&lt;li&gt;My solution will be somewhat standardized and therefore more interoperable with other libraries, software, developers, ...&lt;/li&gt;&#xA;&lt;li&gt;There's an user community, however, of course, much smaller than for many other programming topics.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Contra:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Installation is tedious. I'm now on day 6...&lt;/li&gt;&#xA;&lt;li&gt;According to the hands on tutorial, Tpetra seems to be not very well established yet.&lt;/li&gt;&#xA;&lt;li&gt;Compiling for CUDA also seems to be tricky and not very mature.&lt;/li&gt;&#xA;&lt;li&gt;The risk that Trilinos does not support something I need. It would probably be very difficult to understand and extend it by myself. Particularly GPU-stuff.&lt;/li&gt;&#xA;&lt;li&gt;Same risk for bugs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm looking for some more arguments pro and contra Trilinos, or some advices considering when you would use it and when not. I know that it might be a bit opinion based, but I would be very glad if I you could share some experience and knowledge with me.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-05-23T20:07:07.940" Title="What are some important pros and contras considering using Trilinos?" Tags="|trilinos|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24048" PostTypeId="1" CreationDate="2016-05-24T00:12:17.787" Score="1" ViewCount="665" Body="&lt;p&gt;I'm currently in the process of writing a self-consistent Schrodinger-Poisson solver for a device heterstructure (High Mobility Electron Transistor). The algorithm is based off of this journal&lt;sup&gt;(1)&lt;/sup&gt;. I am having a few difficulties however. It seems like I can either have Dirichlet BCs or Neumann BCs at the substrate (i.e. the right end of the structure). Below is a snapshot of a Dirichlet and Neumann BC, respectively where $ x_n=0 $ for the first, and $\partial x/\partial n = 0$ for the second plot. Note that the boundary in question $\partial \Omega$ is at the right end of the plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V5P1L.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V5P1L.jpg&quot; alt=&quot;My plots&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm really interested in having the boundary $\partial\Omega $ be like the plot below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jkGOZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jkGOZ.png&quot; alt=&quot;Desired plot&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt; (1) A self‐consistent solution of Schrödinger–Poisson equations using a nonuniform mesh. Tan, I‐H. and Snider, G. L. and Chang, L. D. and Hu, E. L., Journal of Applied Physics, 68, 4071-4076 (1990)&lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Below outlines the discretization scheme that I am using.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;My plan is to simulate an AlGaAs/GaAs HEMT stack that looks almost&#xA;identical to the one in the journal that I referenced above (The&#xA;stack is in Fig. 3). There's only a few Angstroms of difference in&#xA;some of my layers. I begin with a simple trial potential $$&#xA;V_{trial}(x) = -q\phi(x) + \Delta E_c(x) $$ Initially, this is a&#xA;simple line that begins at 0.7 eV with a step function of 0.2 eV&#xA;that spans the AlGaAs region.&lt;/li&gt;&#xA;&lt;li&gt;I then solve the Schrodinger equation using the finite difference scheme. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The function Schrodinger($V_{trial}$) gets called first.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [prob_Density, eig_Vals] = schrodinger(phi)&#xA;p = parameters;&#xA;n = p.grid_points;&#xA;t0 = p.hbar^2 / (2*p.m_AlGaN*p.dz^2);&#xA;&#xA;A = zeros(n,n);&#xA;&#xA;% Define first and last rows&#xA;A(1,1) = 2*t0; A(1,2) = -2*t0; A(n-1,n) = -2*t0; A(n,n) = 2*t0;      &#xA;&#xA;for i=2:(n-1)  % Create Hamiltonian&#xA;    A(i,i) = 2*t0 + phi(i);&#xA;    A(i+1,i) = -t0; A(i,i+1) = -t0; &#xA;    A(i-1,i) = -t0; A(i,i-1) = -t0;&#xA;end&#xA;&#xA;[eig_states, eig_Vals] = eig(A);  % Solve for eigensolutions&#xA;&#xA;prob_Density = eig_states.*conj(eig_states);  % Calculate probability function&#xA;&#xA;end  % End of function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Using the result &quot;prob_Density&quot; from step 2, I then solve for the current distributions using $$n(x) = \sum_{k=1}^{m} \psi_k^* (x) \psi_k (x) n_k$$ where $$n_k = \frac{m^*}{\pi \hbar^2} \int_{E_k}^{\infty} \frac{1}{1+e^{(E-E_F)/kT}} dE $$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Next, I use the $n(x)$ and solve Poisson's equation. I once again use the finite difference scheme with the Poisson's equation. My Dirichlet boundary condition for x=0 is defined at the Schottky barrier height. My second boundary is of the Neumann type, where the derivative is zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code for the &quot;Charge Density&quot; (i.e. $n(x)$) and Poisson solver are written below, respectively.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%#### FIND CHARGE DISTRIBUTION ########&#xA;function charge_density = n_x(prob_Density, eig_Vals, Ef, final_Eigstate)&#xA;&#xA;p = parameters;&#xA;&#xA;n = p.grid_points;&#xA;n_array = zeros(n,1);&#xA;const = (p.m_AlGaAs ./ (pi*p.hbar^2));&#xA;&#xA;fermi_dirac = @(E) 1./(1+exp((E-Ef)./p.Vt)); % Unit eV&#xA;&#xA;for eig_state = 1:final_Eigstate&#xA;&#xA;    Ek = eig_Vals(eig_state,eig_state); % First,second,third .. eigenenergy&#xA;    nk = const .* integral(fermi_dirac, Ek, inf)*p.q;   %  [m]^-2&#xA;    n_x_arr = prob_Density(:,eig_state).*nk;    % [m]^-1 * [m]^-2 -&amp;gt; [m]^-3&#xA;    n_array = n_array + n_x_arr;&#xA;end&#xA;&#xA;charge_density = n_array; % Electron volumetric density [m]^-3&#xA;&#xA;&#xA;end % End program&#xA;&#xA;&#xA;%######### START POISSON SOLVER ###########&#xA;function phi = poisson(n_x)&#xA;&#xA;first_boundary = 0.7; % eV&#xA;second_boundary = 0;    &#xA;&#xA;p=parameters;&#xA;n = p.grid_points;&#xA;&#xA;% Create Poisson matrix elements &#xA;A = zeros(n,n);&#xA;&#xA;for i = 2:n-1&#xA;   A(i,i) = -2; &#xA;   A(i+1,i) = 1; &#xA;   A(i-1,i) = 1; &#xA;   A(i,i+1) = 1; &#xA;   A(i,i-1) = 1;&#xA;end&#xA;&#xA;A(1,1) = -2; &#xA;A(1,2) = 1;&#xA;A(n,n) = -1; % -1 a result of Neumann BC&#xA;A(n,n-1) = 1;&#xA;&#xA;F = zeros(n,1);&#xA;&#xA;%% RHS of matrix equation (the 'B' n x 1 term) AU=B, U=A\B&#xA;h = p.dz;&#xA;&#xA;for i = 2:n-1 &#xA;   f = p.q*(p.ND(i) - n_x(i))./p.eps_hemt(i);&#xA;   F(i) = f*h^2;&#xA;end&#xA;&#xA;f1 = p.q*(p.ND(1) - n_x(1))./p.eps_hemt(1);&#xA;fn = p.q*(p.ND(n) - n_x(n))./p.eps_hemt(n);&#xA;&#xA;F(1) = f1*h^2 - first_boundary;&#xA;&#xA;F(n) = fn*h^2 - second_boundary;&#xA;&#xA;&#xA;phi = (A\F);                      %  Voltage profile&#xA;&#xA;end&#xA;%##### END POISSON SOLVER&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Finally, I use the result from the Poisson solver and return to Step 2. I once again run the function Schrodinger($V_{new}$) and iterate through this process until a self consistent solution is found. This method brings me to the very first plot you see above, and I'm aiming to simulate the third plot. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20468" LastEditorUserId="20468" LastEditDate="2016-05-24T07:42:40.130" LastActivityDate="2017-11-21T08:53:56.117" Title="Issues with self-consistent Poisson-Schrodinger solver" Tags="|boundary-conditions|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24050" PostTypeId="2" ParentId="24043" CreationDate="2016-05-24T07:56:13.917" Score="3" Body="&lt;p&gt;Please notice that you assumptions do not preclude singular matrices. A specific example is&#xA;\begin{equation}&#xA;C = \begin{bmatrix} I &amp;amp; I \\ I &amp;amp; I \end{bmatrix}&#xA;\end{equation}&#xA;where $I$ denotes the identity matrix of dimension $I$. This example also shows that the condition that $B$ is positive definite is not necessarily useful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate the fact that you have simplified your question by removing the context and presenting the bare mathematical facts. However, I fear that you may have gone too far and that important information has been lost, i.e. you matrix appears worse than it really is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that the matrix is not to ill conditioned, then you can solve &#xA;\begin{equation}&#xA;Cx = \begin{bmatrix} A &amp;amp; B^T \\ B &amp;amp; D \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}  = \begin{bmatrix} f_1 \\ f_2 \end{bmatrix}&#xA;\end{equation}&#xA;by passing to the Schur complement system&#xA;\begin{equation}&#xA;S x_2 = (D - B A^{-1} B^T) x_2 = f_2 - B A^{-1} f_1&#xA;\end{equation}&#xA;which you solve for $x_2$. This is attractive, if $D$ is small, and it is easy to solve systems with coefficient matrix $A$. It is less attractive, if $D$ is large as $S$ is likely a dense matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the system is ill-conditioned, then the best course of action is to do a QR factorization with column pivoting $AP = QR$. By inspecting $R$ you will at least get information about, say, the numerical rank of $A$ and you might find that $f$ is in the range of your matrix anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The rank revealing QR factorization with column pivoting $AP = QR$ is a closely related to the Gram-Schmidt algorithm for computing an orthogonal basis for the column space of $A$. Here $P$ is simply a permutation matrix which is often picked to ensure that the diagonal entries of the upper triangular matrix $R$ are monotone decreasing. The strength of having a factorization of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ AP = Q R = Q \begin{bmatrix} R_{11} &amp;amp; R_{12} \\ 0 &amp;amp; R_{22} \end{bmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is that you can start to make estimates of the singular values of $A$ in terms of the matrices $R_{11}$ and $R_{22}$. A good book which is freely available from MIT's website is &quot;Matrix Computations&quot; by Golub and van Loan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.mit.edu/ehliu/Public/sclark/Golub%20G.H.,%20Van%20Loan%20C.F.-%20Matrix%20Computations.pdf&quot; rel=&quot;nofollow&quot;&gt;http://web.mit.edu/ehliu/Public/sclark/Golub%20G.H.,%20Van%20Loan%20C.F.-%20Matrix%20Computations.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among the many topics discussed you will find QR factorizations. I am not familiar with the following report on rank revealing QR factorizations, but the authors are of a caliber where I am confident that this report is also good&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.mcs.anl.gov/pub/tech_reports/reports/P559.pdf&quot; rel=&quot;nofollow&quot;&gt;ftp://ftp.mcs.anl.gov/pub/tech_reports/reports/P559.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;QR factorization with column pivoting is implemented in LAPACK. This is a link to the relevant node&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/lapack/lug/node42.html&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/lapack/lug/node42.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2016-06-01T19:06:28.057" LastActivityDate="2016-06-01T19:06:28.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24051" PostTypeId="2" ParentId="24017" CreationDate="2016-05-24T11:01:58.893" Score="0" Body="&lt;p&gt;Since you're using a microcontroller, how about implementing a bandpass filter in hardware (i.e. using analog electronics, not digital computation). This might be the fastest option if you're sampling an analog signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for digital filters, bandpass filtering and the Goertzel algorithm (as you mentioned) are good ways to go, and there are ways to optimize them for fixed-point arithmetic.&lt;/p&gt;&#xA;" OwnerUserId="20473" LastActivityDate="2016-05-24T11:01:58.893" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24053" PostTypeId="1" CreationDate="2016-05-24T11:50:47.770" Score="1" ViewCount="219" Body="&lt;p&gt;I have implemented a constitutive equation of elastic materials (Hooke's law) in my 3D weakly compressible SPH solver based on [1]. The coding seems to be correct.&#xA;To verify the implementation I started to check the solver through simple test cases. One of them is a rotating planar object in zero gravity (initial conditions contain the velocity field of rigidbody-like rotation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The offdiagonal members of the Cauchy-stress tensor (concerning each particle) should remain zero, but they start to rise as the body deviates from its initial configuration. The larger the deviation the larger the shear-stress in the particles. This results in an oscillatory motion of the body instead of pure rotation.&#xA;In [1] very large deflections occur, however in my solver these are obstructed by the spurious shears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the reason of this behaviour?&#xA;Due to the Jaumann-rate the stress-derivative should be objective. Is it true?&#xA;Are there any non-trivial steps in the calculations of [1] that are not introduced?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] J.P. Gray, J.J. Monaghan, R.P. Swift. SPH elastic dynamics, 2001.&#xA;URL:&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782501002547&quot; rel=&quot;nofollow&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0045782501002547&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20475" LastActivityDate="2022-02-05T06:04:50.553" Title="How to make objective elastic SPH model?" Tags="|sph|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24054" PostTypeId="2" ParentId="21931" CreationDate="2016-05-24T12:07:18.117" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have read in a lot of places around the internet that a fluorescent&#xA;  bead that represents a sub resolution object and therefore a impulse&#xA;  function. While I understand that the OTF (optical transfer function)&#xA;  is represented by the image I am getting, how am I superposed to&#xA;  deconvolve the image I have using the very same image?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is what we do for two-photon microscopes. Is your bead much smaller than the point spread function? If so then, as you mention, you can approximate the bead an impulse. From this perspective, the bead itself is a delta function, and the recorded image of it is your estimate the PSF. Deconvolving the image of the bead with itself would give a delta function at the bead's location, which is exactly the assumption behind this method. It's important to average several frames to reduce the noise, subtract the background, normalize, and crop the image to remove unnecessary background. You could also fit a parametric model of the PSF (e.g. a Gaussian to approximate the central lobe, or a more complicated model). Here, you would again consider the bead to be a delta function, and the model would be fit to the image of the bead. If your bead is larger and you don't want to treat is as a delta function, I could imagine treating it as a known sphere (if you know the size). In that case, you'd have to do a deconvolution to recover the PSF.&lt;/p&gt;&#xA;" OwnerUserId="20473" LastActivityDate="2016-05-24T12:07:18.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24055" PostTypeId="1" AcceptedAnswerId="24056" CreationDate="2016-05-24T14:09:48.467" Score="1" ViewCount="76" Body="&lt;p&gt;Suppose I have to solve the 2-D heat equation in a rectangular domain using the finite difference method, for the boundary conditions say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_1$ is the temperature of the right side of the rectangle,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_2$ is the temperature of the top side,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_3$ is the temperature of the left side,&#xA;and $T_4$ is the temperature at the bottom.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;           T2&#xA;    _______________&#xA;   |               |&#xA;T3 |               |T1&#xA;   |               |&#xA;   |_______________|&#xA;          T4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;at the vertices points of the rectangle, how to set the boundary conditions? do I need to take the average value of the two temperatures?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="14930" LastEditorUserId="14930" LastEditDate="2016-05-25T01:04:39.440" LastActivityDate="2016-05-25T01:04:39.440" Title="How to set the temperature at the vertices points for a rectangular domain?" Tags="|finite-difference|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24056" PostTypeId="2" ParentId="24055" CreationDate="2016-05-24T14:27:53.643" Score="2" Body="&lt;p&gt;I believe you cannot apply two boundary conditions at the same point. You need to choose between T2 and T3 for the top-left point for instance. The average might be acceptable but I'm not sure of the physical meaning though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, with a fine discretization, it should not be a problem since the consequence of this choice will be insignificant on the solution.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-05-24T14:27:53.643" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24057" PostTypeId="1" CreationDate="2016-05-24T18:47:04.310" Score="4" ViewCount="215" Body="&lt;p&gt;I'm fooling around with the decision on how to build a multiphysics simulation for too long now (also several questions in this forum). First, I thought it would be possible/necessary to write most of it from scratch and by combining various libraries. (I did this in Matlab and Python to get familiar with the model, so I basically have an idea how to do it.) Now I learned more about more integrated platforms, for example Trilinos and PETSc. Then I learn about MOOSE that pretty much claims to be the jack of all trades. And probably a dozen other platforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me as an unexperienced PhD student, it seems to be almost impossible to do an educated decision about which platform to use. So I really, really need the help of the community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Requirements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multiphysics and nonlinear equations&lt;/li&gt;&#xA;&lt;li&gt;Various regions with different physics and internal boundaries to each other&lt;/li&gt;&#xA;&lt;li&gt;Very large simulations on many computing nodes&lt;/li&gt;&#xA;&lt;li&gt;In these days it's probably mandatory to leverage GPUs.&lt;/li&gt;&#xA;&lt;li&gt;Geometry will be generated programmatically and it would be very convenient to have an integrated mesher that can generate meshes from parametrically defined geometries (signed distance functions for example).&lt;/li&gt;&#xA;&lt;li&gt;Can be integrated into own software&lt;/li&gt;&#xA;&lt;li&gt;Liberal license, like BSD or (L)GPL&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If you started such a project right now, what FEM platform would you count on?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-05-24T18:47:04.310" Title="Once and for all: Which FEM plattform should I use for a very large multiphysics simulation?" Tags="|finite-element|linear-solver|mesh|multiphysics|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="24059" PostTypeId="1" AcceptedAnswerId="24068" CreationDate="2016-05-24T20:14:19.037" Score="0" ViewCount="178" Body="&lt;p&gt;I'd like to know how to solve in MATLAB the following integer optimization problem :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\underset{B,D}{\arg\min} \Vert{Y-XDB}\Vert_{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $X,Y$ are given matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraint is the following : the matrix $D$ must be a diagonal matrix and its elements must be 0 or 1.&lt;/p&gt;&#xA;" OwnerUserId="20482" LastEditorUserId="4244" LastEditDate="2016-05-25T15:16:58.553" LastActivityDate="2016-05-27T09:57:41.193" Title="Integer programming with Matlab" Tags="|optimization|matlab|constrained-optimization|" AnswerCount="1" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24060" PostTypeId="1" CreationDate="2016-05-25T07:10:21.300" Score="3" ViewCount="857" Body="&lt;p&gt;I have always come across the terms &quot;conservative&quot; as opposed to &quot;non-conservative&quot; forms of equations in fluid mechanics. Is there a good reference that someone can share to clearly distinguish the two or shed some light on what they mean and what are their advantages/disadvantages?   &lt;/p&gt;&#xA;" OwnerUserId="19629" LastActivityDate="2016-05-25T10:01:54.590" Title="conservative v non-conservative" Tags="|pde|" AnswerCount="1" CommentCount="2" ClosedDate="2016-05-29T03:48:08.963" ContentLicense="CC BY-SA 3.0" />
  <row Id="24061" PostTypeId="2" ParentId="24060" CreationDate="2016-05-25T07:31:06.697" Score="6" Body="&lt;p&gt;In short, you can recognize a conservative formulation if a divergence operator is involved in the equation. For instance, the mass conservation equation is naturally written in conservative form :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \rho}{\partial t} + div  (\rho \mathbf{u})=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By integrating over $\Omega$ (a reference volume) and with the flux-divergence theorem :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Omega} \frac{\partial \rho}{\partial t} \mathrm{d}\Omega + \int_{\partial \Omega} \rho \mathbf{u} \cdot \mathbf{n} \mathrm{d}S = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can understand that the quantity of mass in the reference volume is equal to the variation of $\rho$ in time (first term) + the sum of the fluxes ($\rho \mathbf{u}$) across the surface boundary S with respect to the normal $\mathbf{n}$ pointing outward, that is the difference between what is going in and what is leaving $\Omega$. It is conservative in the sense that if we sum all the contributions acting on the volume $\Omega$, the result will be constant at each time, it means that mass is not lost or created.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a pure advection equation (non-conservative) : &#xA;$$\frac{\partial \phi}{\partial t} + \mathbf{u} \cdot \nabla\phi=0$$&#xA;the theorem below does not apply (no divergence operator). It means that the quantity $\phi$ is advected through the computational domain so if we place ourselves in an eulerian formalism looking at the quantity $\phi$ in a reference volume $\Omega$, the result can be 1 at t=dt and 0.5 at t=t+dt =&gt; non-conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is recommended to solve equations in conservative form as they respect the conservation principle but sometimes an additional advection equation is required (multifluid simulations), that one can be very tedious to discretize without tremendous numerical problems.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="20454" LastEditDate="2016-05-25T10:01:54.590" LastActivityDate="2016-05-25T10:01:54.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24062" PostTypeId="1" AcceptedAnswerId="24063" CreationDate="2016-05-25T15:15:13.307" Score="0" ViewCount="328" Body="&lt;p&gt;I am doing a time evolution analysis for a structure and have already got the displacement result, which is a vector including x and y components. And I want to plot the time evolution of x and y displacement respectively for a point in the structure. But every time I use plot selection over time, it shows the evolution of displacement magnitute. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get the single component evolution?&#xA;In addition, it seems that the magnitute plot does not include signs (all of them are positive), am I right?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastActivityDate="2016-05-25T16:06:24.497" Title="How to plot selection over time for only components, but not magnitute. of a vector in paraview?" Tags="|plotting|paraview|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24063" PostTypeId="2" ParentId="24062" CreationDate="2016-05-25T16:06:24.497" Score="0" Body="&lt;p&gt;The magnitude plot may display the norm of the vector so it is obviously non-negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may be able to get what you are looking for by taking the dot product of your vector with the vectors from the canonical base. If your data is a vector $\vec{u}=\begin{pmatrix}x\\y\end{pmatrix}$ then you can recover the x component by taking&#xA;$\vec{u} \cdot \begin{pmatrix}1\\0\end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should be able to do that with the calculator (see Filters) and the buttons iHat, jHat.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-05-25T16:06:24.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24065" PostTypeId="1" CreationDate="2016-05-26T05:30:54.737" Score="2" ViewCount="417" Body="&lt;p&gt;I want to solve Navier Stokes equations on a &lt;strong&gt;collocated grid&lt;/strong&gt;. Earlier, I was using a MacCormick scheme based solver where I discretized predictor step in forward differences and corrector step in backward differences as mentioned in this paper  (&lt;em&gt;An explicit finite-difference scheme for simulation of moving particles&lt;/em&gt;, &lt;a href=&quot;http://dl.acm.org/citation.cfm?id=1127013&quot; rel=&quot;nofollow&quot;&gt;http://dl.acm.org/citation.cfm?id=1127013&lt;/a&gt;). This  works, but the convergence is very slow at high-grid resolutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this regard, can anyone suggest me some &lt;strong&gt;explicit&lt;/strong&gt; NS solvers that are not very complicated and could be converged in relatively shorter times. Also, it should be mentioned here that I am writing NS equations in &lt;em&gt;artificial compressibility&lt;/em&gt; framework.&lt;/p&gt;&#xA;" OwnerUserId="16397" LastEditorUserId="16397" LastEditDate="2016-05-28T12:56:09.043" LastActivityDate="2016-05-28T12:56:09.043" Title="Collocated Grid Navier Stokes Solver" Tags="|finite-difference|numerical-analysis|fluid-dynamics|explicit-methods|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24066" PostTypeId="1" CreationDate="2016-05-26T12:31:28.763" Score="5" ViewCount="5014" Body="&lt;p&gt;The image below illustrates the kinetic scheme I am trying to model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V3rw4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V3rw4.png&quot; alt=&quot;scheme&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first example focuses on the conversion of the wood-oil to non-volatiles and volatiles. The rate equations are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r_w = \frac{dC_w}{dt} = -(K_{nv} + K_v)\,C_w \\&#xA;r_{nv} = \frac{dC_{nv}}{dt} = K_{nv}\,C_w \\&#xA;r_{v} = \frac{dC_{v}}{dt} = K_{v}\,C_w \\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve for the concentrations, I use the &lt;code&gt;odeint&lt;/code&gt; solver in SciPy as demonstrated in the Python code below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dt = 0.0001                           # time step, delta t&#xA;tmax = 25                             # max time, s&#xA;t = np.linspace(0, tmax, num=tmax/dt) # time vector&#xA;nt = len(t)                           # total number of time steps&#xA;&#xA;def rates(c, t):&#xA;    &quot;&quot;&quot;&#xA;    w = wood-oil as conc[0]&#xA;    nv = non-volatiles as conc[1]&#xA;    v = volatiles as conc[2]&#xA;    &quot;&quot;&quot;&#xA;    Knv = 0.3&#xA;    Kv = 0.8&#xA;    rw = -(Knv + Kv) * c[0]&#xA;    rnv = Knv * c[0]&#xA;    rv = Kv * c[0]&#xA;    return [rw, rnv, rv]&#xA;&#xA;cc = sp.odeint(rates, [1, 0, 0], t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above code displays the following plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GsTJh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GsTJh.png&quot; alt=&quot;plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my next approach I attempt to account for the reactions that consume the non-volatiles and volatiles; the rate equations are displayed below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r_w = \frac{dC_w}{dt} = -(K_{nv} + K_v)\,C_w \\&#xA;r_{nv} = \frac{dC_{nv}}{dt} = K_{nv}\,C_w - K_{r1}\,{C_{nv}}^{2.5} - K_{cr}\,{C_{nv}}^{0.9} - K_{c1}\,{C_{nv}}^{1.1} \\&#xA;r_{v} = \frac{dC_{v}}{dt} = K_{v}\,C_w + K_{cr}\,{C_{nv}}^{0.9} - K_{d}\,{C_{v}}^{0.9} - K_{a}\,{C_{v}}^{1.4} - K_{g}\,{C_{v}}^{0.8} - K_{r2}\,{C_{v}}^{0.7}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the code I use to solve for the concentrations is shown below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def dCdt(c, t):&#xA;    &quot;&quot;&quot;&#xA;    w = wood-oil as conc[0]&#xA;    nv = non-volatiles as conc[1]&#xA;    v = volatiles as conc[2]&#xA;    &quot;&quot;&quot;&#xA;    Knv = 0.3&#xA;    Kv = 0.8&#xA;    Kr1 = 9.2e7;    r1 = 2.5&#xA;    Kcr = 4.1e-5;   cr = 0.9&#xA;    Kc1 = 3.7e5;    c1 = 1.1&#xA;    Kd = 8.0e-4;    d = 0.9&#xA;    Ka = 6.1e-6;    a = 1.4&#xA;    Kg = 1.8e-4;    g = 0.8&#xA;    Kr2 = 37.0e5;   r2 = 0.7&#xA;    rw = -(Knv + Kv)*c[0]&#xA;    rnv = Knv*c[0] - Kr1*c[1]**r1 - Kcr*c[1]**cr - Kc1*c[1]**c1&#xA;    rv = Kv*c[0] + Kcr*c[1]**cr - Kd*c[2]**d - Ka*c[2]**a - Kg*c[2]**g - Kr2*c[2]**r2&#xA;    return [rw, rnv, rv]&#xA;&#xA;cc2 = sp.odeint(dCdt, [1, 0, 0], t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this approach does not work and I get an error from Python about &lt;code&gt;invalid value encountered in double_scalars&lt;/code&gt;. It appears that the &lt;code&gt;odeint&lt;/code&gt; solver isn't capable of handling the different reaction orders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my next approach is to solve the system with the SciPy &lt;code&gt;ode&lt;/code&gt; solver. The code for this is shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def dcdt(t, c):&#xA;    &quot;&quot;&quot;&#xA;    w = wood-oil as c[0]&#xA;    nv = non-volatiles as c[1]&#xA;    v = volatiles as c[2]&#xA;    &quot;&quot;&quot;&#xA;    Knv = 0.3&#xA;    Kv = 0.8&#xA;    Kr1 = 9.2e7;    r1 = 2.5&#xA;    Kcr = 4.1e-5;   cr = 0.9&#xA;    Kc1 = 3.7e5;    c1 = 1.1&#xA;    Kd = 8.0e-4;    d = 0.9&#xA;    Ka = 6.1e-6;    a = 1.4&#xA;    Kg = 1.8e-4;    g = 0.8&#xA;    Kr2 = 37.0e5;   r2 = 0.7&#xA;    rw = -(Knv + Kv)*c[0]&#xA;    rnv = Knv*c[0] - Kr1*c[1]**r1 - Kcr*c[1]**cr - Kc1*c[1]**c1&#xA;    rv = Kv*c[0] + Kcr*c[1]**cr - Kd*c[2]**d - Ka*c[2]**a - Kg*c[2]**g - Kr2*c[2]**r2&#xA;    return [rw, rnv, rv]&#xA;&#xA;# store concentrations&#xA;Coil = np.ones(nt)      # bio-oil concentration&#xA;Cnvol = np.zeros(nt)    # non-volatiles concentration&#xA;Cvol = np.zeros(nt)     # volatiles concentration&#xA;&#xA;# Setup the ode integrator where 'dopri5' is Runge-Kutta 4th order&#xA;r = sp.ode(dcdt).set_integrator('dopri5', nsteps=10000)&#xA;r.set_initial_value([1, 0, 0], 0)&#xA;&#xA;# integrate the odes for each time step then store the results&#xA;k = 1&#xA;while r.successful() and r.t &amp;lt; tmax-dt:&#xA;    r.integrate(r.t+dt)&#xA;    Coil[k] = r.y[0]&#xA;    Cnvol[k] = r.y[1]&#xA;    Cvol[k] = r.y[2]&#xA;    k += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the &lt;code&gt;ode&lt;/code&gt; approach does not work and I receive a warning about the system being stiff. Does anyone have suggestions on how to solve this system of rate equations in Python when the reaction order is not one?&lt;/p&gt;&#xA;" OwnerUserId="5146" LastEditorUserId="5146" LastEditDate="2016-10-12T15:11:54.777" LastActivityDate="2016-11-11T16:00:15.160" Title="Solve rate equations with different reaction orders using SciPy ode" Tags="|python|computational-chemistry|scipy|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24068" PostTypeId="2" ParentId="24059" CreationDate="2016-05-27T09:57:41.193" Score="0" Body="&lt;p&gt;This kind of problem can be solved using heuristic methods, like particle swarm optimization or genetic algorithms.&lt;/p&gt;&#xA;" OwnerUserId="20482" LastActivityDate="2016-05-27T09:57:41.193" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24071" PostTypeId="2" ParentId="21364" CreationDate="2016-05-27T14:58:39.107" Score="2" Body="&lt;p&gt;It is very much problem dependent, but the fact that it is coupled doesn't matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if you have BC's properly defined for the thermal model, ignoring the mechanical aspects, and if you have BC's properly defined for the mechanical model, ignoring the thermal aspects, you have BC's properly defined for the coupled system. It's not a bad idea to take a &quot;divide-and-conquer&quot; approach to coupled problems, setting up and solving each problem individually, then doing a one-way coupling, then doing the full coupling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, there's a few heuristics to keep in mind when solving such problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For each submodel, in each dimension, you need the same number of BC's as the highest order derivative in the problem. For the heat equation, this is typically 2nd order in space, and thus you need two BC's specified&lt;/li&gt;&#xA;&lt;li&gt;What type of BC you use (Dirichlet, Neumann, Robin) and where you apply it depends on what you are modeling -- remember, you can apply BC's at $\infty$&lt;/li&gt;&#xA;&lt;li&gt;For the heat equation, and others, you can't have Neumann type BC's on every boundary in a steady state problem without add'l information. You need to specify the temperature somewhere or sometime (Note: this isn't necessary a hard-and-fast rule for all models you will encounter, but if it comes up, take a second look)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20509" LastActivityDate="2016-05-27T14:58:39.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24072" PostTypeId="5" CreationDate="2016-05-27T15:30:36.467" Score="0" Body="" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-05-27T15:30:36.467" LastActivityDate="2016-05-27T15:30:36.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24073" PostTypeId="4" CreationDate="2016-05-27T15:30:36.467" Score="0" Body="This question is about equations where the unknown itself is a matrix such as Sylvester or Riccati equations. For systems of linear equations (where the unknown is a vector), use &quot;linear-system&quot;." OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-05-27T15:30:36.467" LastActivityDate="2016-05-27T15:30:36.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24077" PostTypeId="1" AcceptedAnswerId="24078" CreationDate="2016-05-27T18:52:55.093" Score="1" ViewCount="160" Body="&lt;p&gt;I have implemented a Quasi Newton method for my problem, where I use the Hessian matrix approximation based approach. Hence, there is a linear system solve in every iteration. I solve the linear system using Conjugate Gradients and use the compact representation given in &lt;a href=&quot;http://users.iems.northwestern.edu/~nocedal/PDFfiles/representations.pdf&quot; rel=&quot;nofollow&quot;&gt;Representations of quasi-Newton matrices and their use in limited memory methods&lt;/a&gt; to get matrix vector products. I am noticing that the linear system is solved in very less number of iterations, 20 to 30 usually. This happens for even very large problems (one million variables). Is this observed in general when working with Quasi Newton? I can think of one possible reason for this fast convergence. Since, I ensure the curvature condition on the Quasi Newton vector pair, the Hessian approximation is guaranteed to be strictly positive definite and the Quadratic form in Conjugate Gradients has a &quot;nice&quot; shape. I am not sure whether this sufficiently explains what is happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Based on the accepted answer below, I have this question. The situation in the Quasi Newton approach, where the linear system can be provably solved in very less number of iterations is very interesting. Are there many works where this has been favorably exploited?&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2016-05-28T18:38:00.030" LastActivityDate="2016-05-28T18:38:00.030" Title="The linear system in Quasi Newton method" Tags="|newton-method|conjugate-gradient|quasi-newton|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24078" PostTypeId="2" ParentId="24077" CreationDate="2016-05-27T20:15:13.140" Score="3" Body="&lt;p&gt;It is a well-known theorem that conjugate gradients is guaranteed to converge (in exact arithmetic) within $r+1$ steps for an identity-plus-rank-$r$ matrix, regardless of its size. In finite precision, you would find that in the worst case, CG would nearly stagnate for the first $r+1$ iterations, then rapidly converge towards the solution for every iteration after that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The limited memory Hessian approximation is exactly an identity-plus-rank-$r$ matrix, where $r$ is (possibly a small scalar multiple of) the number of vectors you are currently carrying. Hence CG is guaranteed to solve it in $O(r)$ iterations. Assuming that the matrix-vector products with the Hessian are efficiently performed, this is a total time complexity of $O(nr^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, CG is not the most efficient way to implement quasi-Newton. Instead of forming the limited memory approximation of the Hessian and then inverting it afterwards, you can form the limited memory approximation of the &lt;em&gt;Hessian-inverse&lt;/em&gt; directly, using steps no more expensive than the ones you have been performing. Then the matrix solve at each iteration is replaced by a single matrix-vector multiplication, which can be performed in $O(nr)$ complexity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proofs for all of these statements can be found in any standard textbook on nonlinear optimization, such as Nocedal &amp;amp; Wright.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastActivityDate="2016-05-27T20:15:13.140" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24079" PostTypeId="2" ParentId="24042" CreationDate="2016-05-27T20:41:43.410" Score="3" Body="&lt;p&gt;So many options!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be solved analytically using separation of variables. So that's good. Gives us something to check against, make sure that we are doing this right, and a good way of checking convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The non-continuous aspect isn't a problem. Might need a few more terms to converge, but will still converge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1024x1024 I think will be overkill, to be honest, depending of course on your required accuracy, even a quick and dirty finite difference solution will give good results there. No big deal, though whatever you do, I highly recommend coding in general and setting the grid to be much coarser while testing. Like 2x2 or 3x3. That makes it much easier to see when (probably not if) you mess something up and make it explode, not to mention that it's much faster for the computer to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, enough preamble, basic idea: you choose a series solution to approximate the true solution and determine the coefficients that best satisfy the problem.&#xA;Let's look how I would approach this problem. First, let's think about what we want our solution to look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Theta(x,y)=\sum_{i=0}^n\sum_{j=0}^ma_{i,j}F_i(x)G_j(y)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note, I'm using $\Theta(x,y)$ to denote our approximate solution to the true solution $T(x,y)$) We have to choose the $F_i$'s &amp;amp; $G_j$'s (our basis functions) and determine the $a_{i,j}$'s that best solve the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you can choose anything you want for $F_i$'s &amp;amp; $G_j$'s, as long as they are all linearly independent (orthogonality is nice, but not strictly required). So let's choose basis functions that make our life easier. For a problem like this, I would chose basis functions that automatically satisfy the boundary conditions. So in our $x$-coordinate, let's use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F_i(x) = \sin(i\pi x)$ for $i=1...n-1$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is $0$ at at $x=0$ and $x=1$ for all $i$, and independent of $G_j$, so those BC's are taken care of for $x$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $y$-coordinate is a little trickier (if we use $\sin (j\pi x)$, we can't satisfy our non-homogeneous BC), but not too bad:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$G_j(y) = \cos((j+1/2)\pi y)$ for $j=1...m$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This satisfies our BC of $T=0$ at $y=1$. And $G_j(0) = 1$ for all $j$, so that's something we can work with. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's look back at our desired equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Theta(x,y)=\sum_{i=0}^n\sum_{j=0}^ma_{i,j}F_i(x)G_j(y) = \sum_{i=1}^{n-1}\sum_{j=1}^ma_{i,j}\sin(i\pi x)\cos((j+1/2)\pi y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note: since we've handled the 3 of the 4 homogeneous BC, I've adjusted the limits of the summation to go from $1$ to $n-1$ and $1$ to $m$ rather than $0$ to $n$ and $0$ to $m$ )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, now we need to find the coefficients, $a_{i,j}$, which give the best solution. Well, first, we need to define what we mean by &quot;best&quot;. Ideally, we want our approximate solution $\Theta(x,y)$ to equal the real solution $T(x,y)$, or at least minimize $\Theta(x,y)-T(x,y)$ in some averaged sense. But we don't know what $T(x,y)$ is, in general, all we have to work with is the governing equation (Laplace's Eq, in this case). In other words, we will chose our $a_{i,j}$ in such a way that $\Theta(x,y)$ approximately satisfy the governing equation, so let's plug our solution into that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 \Theta}{\partial x^2} + \frac{\partial^2 \Theta}{\partial y^2} = 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{i=1}^{n-1}\sum_{j=1}^m-i^2\pi^2 a_{i,j}\sin(i\pi x)\cos((j+1/2)\pi y) - \sum_{i=1}^{n-1}\sum_{j=1}^m(j+1/2)^2\pi^2a_{i,j}\sin(i\pi x)\cos((j+1/2)\pi y) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, we cannot choose a set of $a_{i,j}$ that ensures that this equation is satisfied everywhere (i.e. for all $x$ and $y$), and the deviation from the solution is called the &quot;residual&quot;. The &quot;best&quot; solution would be to use the Galerkin approach to minimize the sum of square of the residual across the entire domain. But it involves performing integrations and can get rather messy and involved in the general case (though for this case, it may be easier, since you can take advantage of the orthogonality of trig functions and everything is nice and linear).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can just set the residual to be zero at $(n-1)$ in discrete points $x$ and $(m-1)$ in discrete points $y$. This is called collocation tends to be much simpler than the Galerkin approach. You can use equal spaced nodes using this technique, but you'll get better results if you use Chebyshev nodes (that will increase node density near the boundaries, and prevent Runge's phenomena from giving you trouble). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either approach will give you $(n-1)\times (m-1)$ linear equations to solve simultaneously for the $((n-1)\times m)$ $a_{i,j}$'s. Obviously, that's not enough equations, but remember, we still have that final, non-homogeneous BC to take care of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Theta(x,0)= \sum_{i=1}^{n-1}\sum_{j=1}^ma_{i,j}\sin(i\pi x)=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can use this to provide the remaining necessary constraints to fully solve for our coefficients. Again, we can use collocation to specify $\Theta(x,0)=1$ at $(n-1)$ specific values of $x$ or we can use a Galerkin approach (which would be a similar procedure to a Fourier transform in this case). Either way, you'll get another $(n-1)$ equations to work with, so you have $(n-1)\times (m)$ linear equations in $(n-1)\times m$ unknowns. And it's just a matter of using your linear solver of choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also a technique called the &quot;Boundary Element Method&quot;, which I think is really cool and something you may be interested in. Basically you only discretize and solve along the boundary and can calculate the solution at any point on the interior after the fact. It effectively reduces the dimensionality of the problem by one (i.e. a $2D$ becomes $1D$) and is especially useful where the ratio of the boundary area to the volume is small. It's limited in application though, I think the governing equation has to be homogeneous, for example, but it's very powerful when it can be used. &lt;/p&gt;&#xA;" OwnerUserId="20509" LastActivityDate="2016-05-27T20:41:43.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24080" PostTypeId="1" CreationDate="2016-05-27T20:42:10.337" Score="5" ViewCount="215" Body="&lt;p&gt;Suppose I have a real symmetric matrix. I would like to tell wether it has at least $k$ strictly positive eigenvalues, but using only additions (no multiplications). Is there a method that I could use? I thought that maybe Gershgorin theorem could be useful, since I need only to add row (or column) elements, but it doesn't guarantee that eigenvalues are different from zero: for example,&#xA;$$&#xA;M = \begin{pmatrix}&#xA;1  &amp;amp; -1 &amp;amp; 0 &amp;amp; 0\\&#xA;-1 &amp;amp;  3 &amp;amp; -1 &amp;amp; -1\\&#xA;0  &amp;amp; -1 &amp;amp; 2 &amp;amp; -1\\&#xA;0  &amp;amp; -1 &amp;amp; -1 &amp;amp; 2&#xA;\end{pmatrix}&#xA;$$&#xA;has eigenvalues $0,1,2,4$, but Gershgorin's theorem would give me estimates&#xA;$$&#xA;\begin{align*}&#xA;1 \pm 1 &amp;amp;= [0,2]\\&#xA;3 \pm 3 &amp;amp;= [0,6]\\&#xA;2 \pm 2 &amp;amp;= [0,4]\\&#xA;2 \pm 2 &amp;amp;= [0,4]&#xA;\end{align*}&#xA;$$&#xA;and I wouldn't know, looking at the estimates only, wethere there is any eigenvalue $&amp;gt;0$.&lt;/p&gt;&#xA;" OwnerUserId="20512" LastActivityDate="2016-05-30T11:53:51.633" Title="Is there an eigenvalue estimation method more accurate than Gershgorin's, which uses no multiplication?" Tags="|linear-algebra|eigenvalues|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24081" PostTypeId="1" AcceptedAnswerId="24082" CreationDate="2016-05-27T21:17:06.733" Score="0" ViewCount="130" Body="&lt;p&gt;I am learning about FEM methods and nonlinear optimization. I would like to try my nonlinear trust region solver on some simple nonlinear problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be good example to implement for beginner? I am aiming to minimize nonlinear energy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks!&lt;/p&gt;&#xA;" OwnerUserId="17150" LastActivityDate="2016-05-28T02:47:19.520" Title="NONLINEAR ENERGY MINIMIZATION EXAMPLE" Tags="|optimization|finite-element|linear-solver|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24082" PostTypeId="2" ParentId="24081" CreationDate="2016-05-28T02:47:19.520" Score="2" Body="&lt;p&gt;Use the minimal surface equation. Here is a version without line search or trust region: &lt;a href=&quot;https://dealii.org/8.4.1/doxygen/deal.II/step_15.html&quot; rel=&quot;nofollow&quot;&gt;https://dealii.org/8.4.1/doxygen/deal.II/step_15.html&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I am one of the authors of the library to which this program belongs.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-05-28T02:47:19.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24085" PostTypeId="1" AcceptedAnswerId="24108" CreationDate="2016-05-28T07:19:37.433" Score="4" ViewCount="653" Body="&lt;p&gt;I am being asked for one of my classes to solve 2D Laplace equations with mixed boundary conditions using the Conjugate Gradient method. The equations and &#xA;conditions are given as:&#xA;$$&#xA;\frac{\partial^2p}{\partial x^2} + \frac{\partial^2p}{\partial y^2}=0\\&#xA;p = 0\text{ at x=0}\qquad p=k\text{ at x=1}\\&#xA;\frac{\partial p}{\partial y}=0\text{ at y=0, 1}&#xA;$$&#xA;The problem can be represented as $\textbf{Ap}=\textbf{b}$, and should be solvable by finding matrices $\textbf{A}$ and $\textbf{b}$ and applying the Conjugate &#xA;Gradient algorithm on them. I know that the general form of the $\textbf{A}$ matrix can be written as:&#xA;$$&#xA;\begin{pmatrix}&#xA;\textbf{J} &amp;amp; \textbf{I} &amp;amp; \cdots &amp;amp; \textbf{0}\\&#xA;\textbf{I} &amp;amp; \textbf{J} &amp;amp; \cdots &amp;amp; \textbf{0}\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots\\&#xA;\textbf{0} &amp;amp; \cdots &amp;amp; \textbf{I} &amp;amp; \textbf{J}&#xA;\end{pmatrix}&#xA;\text{with }\textbf{J}=&#xA;\begin{pmatrix}&#xA;4&amp;amp;-1&amp;amp;\cdots&amp;amp;0\\&#xA;-1 &amp;amp; 4 &amp;amp;\cdots &amp;amp; 0\\&#xA;\vdots&amp;amp;\vdots&amp;amp;\ddots&amp;amp;\vdots\\&#xA;0&amp;amp;\cdots&amp;amp;-1&amp;amp;4&#xA;\end{pmatrix}&#xA;$$&#xA;where $\textbf{I}$ is the unit matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Translation of the Dirichlet conditions makes the matrix $\textbf{A}$ and vector $\textbf{b}$ become&#xA;$$\textbf{A}=&#xA;\begin{pmatrix}&#xA;\textbf{I}&amp;amp;\textbf{0}&amp;amp;\textbf{0}&amp;amp;\cdots&amp;amp;\textbf{0}&amp;amp;\textbf{0}\\&#xA;\textbf{I}&amp;amp;\textbf{J}&amp;amp;\textbf{I}&amp;amp;\cdots&amp;amp;\textbf{0}&amp;amp;\textbf{0}\\&#xA;\textbf{0}&amp;amp;\textbf{I}&amp;amp;\textbf{J}&amp;amp;\cdots&amp;amp;\textbf{0}&amp;amp;\textbf{0}\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots\\&#xA;\textbf{0}&amp;amp;\textbf{0}&amp;amp;\textbf{0}&amp;amp;\cdots&amp;amp;\textbf{J}&amp;amp;\textbf{I}\\&#xA;\textbf{0}&amp;amp;\textbf{0}&amp;amp;\textbf{0}&amp;amp;\cdots&amp;amp;\textbf{0}&amp;amp;\textbf{I}\\&#xA;\end{pmatrix}&#xA;\textbf{b}=&#xA;\begin{pmatrix}&#xA;0\\0\\\vdots\\k\\k&#xA;\end{pmatrix}&#xA;$$&#xA;while the Neumann conditions will modify matrix $\textbf{J}$ as&#xA;$$\textbf{J}=&#xA;\begin{pmatrix}&#xA;1&amp;amp;-1&amp;amp;0&amp;amp;\cdots&amp;amp;0&amp;amp;0\\&#xA;-1&amp;amp;4&amp;amp;-1&amp;amp;\cdots&amp;amp;0&amp;amp;0\\&#xA;0&amp;amp;-1&amp;amp;4&amp;amp;\cdots&amp;amp;0&amp;amp;0\\&#xA;\vdots&amp;amp;\vdots&amp;amp;\vdots&amp;amp;\ddots&amp;amp;\vdots&amp;amp;\vdots\\&#xA;0&amp;amp;0&amp;amp;0&amp;amp;\cdots&amp;amp;4&amp;amp;-1\\&#xA;0&amp;amp;0&amp;amp;0&amp;amp;\cdots&amp;amp;-1&amp;amp;1&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, this has made my matrix $\textbf{A}$ become asymmetric and indefinite. What makes it not usable for the Conjugate Gradient algorithm...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know how to still apply the conjugate gradient method on this problem, without having to use a preconditioner?&lt;/p&gt;&#xA;" OwnerUserId="20513" LastActivityDate="2016-05-30T19:05:51.693" Title="2D Laplace problem with mixed boundary conditions using Conjugate Gradients" Tags="|algorithms|boundary-conditions|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24087" PostTypeId="1" AcceptedAnswerId="24192" CreationDate="2016-05-28T10:13:30.700" Score="1" ViewCount="1185" Body="&lt;p&gt;Is there any version of GSA (Gravitational Search Algorithm)[1] implemented with C++ or even C#?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I found was implemented using MATLAB which is not good for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;[1]&lt;/strong&gt; Rashedi, Esmat, Hossein Nezamabadi-Pour, and Saeid Saryazdi. &lt;a href=&quot;https://www.researchgate.net/publication/222853813_GSA_a_Gravitational_Search_Algorithm&quot; rel=&quot;nofollow&quot;&gt;&quot;GSA: a gravitational search algorithm.&quot;&lt;/a&gt; Information sciences 179.13 (2009): 2232-2248.&lt;/p&gt;&#xA;" OwnerUserId="20517" LastEditorUserId="9667" LastEditDate="2016-06-10T02:20:16.503" LastActivityDate="2016-06-10T12:41:28.860" Title="GSA Search Algorithm in C++" Tags="|optimization|c++|heuristics|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24088" PostTypeId="2" ParentId="14733" CreationDate="2016-05-28T10:38:29.207" Score="2" Body="&lt;p&gt;The differential operator is &quot;grad after div&quot; and so (in the continuous version) is symmetric and negative semi-definite. To get a symmetric discretization your discretization of the gradient has to be the (negative) adjoint of your discrete divergence. With finite differences this says, for example, that forward differences for the divergence imply backward differences for the gradient. Boundary conditions need special care. &lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2016-05-28T10:38:29.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24090" PostTypeId="2" ParentId="24080" CreationDate="2016-05-28T13:39:44.033" Score="7" Body="&lt;p&gt;Conclusions are indeed possible, but Gershgorin's circle theorem must be supplemented with other results. Let &#xA;\begin{equation}&#xA;\lambda_1 \leq  \lambda_2 \leq \lambda_3 \leq \lambda_4&#xA;\end{equation}&#xA;denote the eigenvalues of $M$. Let $N$ denote the lower 3 by 3 corner of $M$, i.e, &#xA;\begin{equation}&#xA;N = \begin{bmatrix}  3 &amp;amp;-1 &amp;amp; -1 \\  -1 &amp;amp; 2 &amp;amp; -1 \\ -1 &amp;amp; -1 &amp;amp; 2 \end{bmatrix}&#xA;\end{equation}&#xA; and let &#xA;\begin{equation}&#xA;\mu_1 \leq  \mu_2 \leq \mu_3&#xA;\end{equation}&#xA;denote the eigenvalues of $N$. Then by Cauchy's interlacing theorem&#xA;\begin{equation}&#xA;\lambda_1 \leq \mu_1 \leq \lambda_2 \leq \mu_2 \leq \lambda_3 \leq \mu_3 \leq \lambda_4.&#xA;\end{equation}&#xA;The eigenvalues &quot;interlace&quot; much like the teeth of a zipper. Unfortunately, no conclusion's can be drawn from studying Gershgorin's intervals for $N$. However, conclusions are possible by passing to $K$, the upper two by two corner of $N$, i.e.&#xA;\begin{equation}&#xA;K = \begin{bmatrix} 3 &amp;amp; -1 \\ -1 &amp;amp; 2 \end{bmatrix}.&#xA;\end{equation}&#xA;Let $\nu_1 \leq \nu_2$ denote the eigenvalues of $K$. By Gershorin's theorem, they are both positive. By Cauchy's theorem we have&#xA;\begin{equation}&#xA;\mu_1 \leq \nu_1 \leq \mu_2 \leq \nu_2 \leq \mu_3&#xA;\end{equation}&#xA;Since $\nu_1$ is positive, we can conclude $\mu_2$ and $\mu_3$ are positive. It follows that $\lambda_3$ and $\lambda_4$ are both positive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can push further using only additions. By inspection, we see that $N$ is irreducibly diagonally dominant, because it is weakly diagonally dominant in rows 2 and 3, and strictly diagonally dominant in row 1. It follows that $N$ is non-singular. Since Gershgorin's theorem implies that $0 \leq \mu_1$, we now know that $0&amp;lt;\mu_1$. It follows immediately, that $0 &amp;lt; \lambda_2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the remaining eigenvalue of $M$, i.e. $\lambda_1$, we observe that the row sums of $M$ are all $0$. Therefore, $\lambda_1 = 0$, and $v_1 = (1,1,1,1)^T$ is a corresponding eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This procedure generalizes as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using comparisons we establish that a matrix $M$ is symmetric and has positive diagonal entries. Using additions we can determine that it is also weakly diagonally dominant. By Gershgorin's theorem it follows that it is semi-definite. If there is a single row which is stricly diagonally dominant, then the matrix is irreducibly diagonally dominant, hence non-singular, hence positive definite. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, we obtain a new matrix $N$ by deleting, a weakly diagonally dominant row and the corresponding column from $M$. The new matrix will be strictly diagonally dominant in at least one row, because we &lt;em&gt;did&lt;/em&gt; remove at least one nontrivial off diagonal element when we created $N$. As before we conclude that $N$ is positive definite. By Cauchy's theorem it follows that $M$ has at least $n-1$ positive eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know if it is possible to determine if zero is an eigenvalue of $M$ using additions only.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2016-05-30T11:53:51.633" LastActivityDate="2016-05-30T11:53:51.633" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24091" PostTypeId="1" AcceptedAnswerId="24092" CreationDate="2016-05-28T15:18:03.483" Score="0" ViewCount="177" Body="&lt;p&gt;I want to compare the number $x$ with $x' = x + \Delta x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have selected an absolute tolerance $atol = 0.1$, then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;|\Delta x| &amp;lt; atol \implies x' \approx x&#xA;\end{equation} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have selected a relative tolerance $rtol = 0.1$, then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left|\frac{\Delta x}{x}\right| &amp;lt;rtol \implies x' \approx x&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens when I set $rtol = atol = 0.1$? Does the relative tolerance check override the absolute tolerance check?&lt;/p&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2016-05-28T16:30:39.690" Title="`lsoda` from `odepack`: does relative tolerance override absolute tolerance, or the other way around?" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24092" PostTypeId="2" ParentId="24091" CreationDate="2016-05-28T16:30:39.690" Score="1" Body="&lt;p&gt;That very much depends on how it is implemented in a specific software, so you need to check either the documentation or the source code. For &lt;code&gt;lsoda&lt;/code&gt;, the &lt;a href=&quot;https://people.sc.fsu.edu/~jburkardt/f77_src/odepack/odepack.f&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; states:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The estimated local error in Y(i) will be controlled&#xA;so as to be roughly less (in magnitude) than&#xA;&#xA;EWT(i) = RTOL*ABS(Y(i)) + ATOL     if ITOL = 1, or&#xA;EWT(i) = RTOL*ABS(Y(i)) + ATOL(i)  if ITOL = 2.&#xA;&#xA;Thus the local error test passes if, in each&#xA;component, either the absolute error is less than&#xA;ATOL (or ATOL(i)), or the relative error is less&#xA;than RTOL.&#xA;&#xA;Use RTOL = 0.0 for pure absolute error control, and&#xA;use ATOL = 0.0 (or ATOL(i) = 0.0) for pure relative&#xA;error control.  Caution:  Actual (global) errors may&#xA;exceed these local tolerances, so choose them&#xA;conservatively.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So in this case, whichever tolerance is satisfied overrides the other. &lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-05-28T16:30:39.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24095" PostTypeId="1" CreationDate="2016-05-29T01:59:40.250" Score="1" ViewCount="83" Body="&lt;p&gt;This question is a continuation of &lt;a href=&quot;https://scicomp.stackexchange.com/a/20095/7023&quot;&gt;this answer&lt;/a&gt; (I can't comment) If we were going to manufacture a solution for a problem with discontinuous coefficients, I understand that the solution should have discontinuous gradient in the interface normal between the coefficients discontinuity, but this would not be arbitrary for problems where the interface is strangely defined. How could we go about this? Is there an automated process to manufacture a solution with discontinuous gradient in the interface?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, when going from the strong form to the weak form, shouldn't we have a boundary term in the interface?&#xA;$$&#xA;\int_{\Omega_1} k_1\nabla u_1 \cdot \nabla v_1 \; d\Omega + \int_{\Omega_2} k_2\nabla u_2 \cdot \nabla v_2 \; d\Omega = \\&#xA;\int_\Omega f v \;d\Omega + \int_{\partial \Omega_1}k_1 \nabla u_1 \cdot n_1\cdot v_1\; d\partial \Omega +\int_{\partial\Omega_2}k_2 \nabla u_2 \cdot n_2 \cdot v_2 \;d\partial \Omega \;\;\forall v &#xA;$$&#xA;This term is not going to vanish unless our solution has a gradient such that the flux will be zero. Say that we have two materials $k_1$ and $k_2$ separated by $\Gamma = \partial \Omega_1 \cap \partial \Omega_2$&lt;/p&gt;&#xA;" OwnerUserId="7023" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-06-03T17:58:29.087" Title="Manufacturing a solution for non-smooth coefficients in elliptic problems" Tags="|pde|convergence|elliptic-pde|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24096" PostTypeId="1" CreationDate="2016-05-29T02:03:36.500" Score="2" ViewCount="62" Body="&lt;p&gt;In Jacobi SVD algorithm as given &lt;a href=&quot;http://www.math.pitt.edu/~sussmanm/2071Spring08/lab09/index.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; every pair of columns of the matrix is orthogonalized until convergence. I want to know that how does the order of selection of the pair of columns affect the convergence of the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on implementing this algorithm. In my program even though I am covering every pair of columns but still with a certain order of selection of columns the algorithm doesn't converge even after thousands of iteration. But with ordering like cyclic ordering, the algorithm converges in 10-12 iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobi rotation of one pair interferes with rotation of another pair. I want to know which order is the best for converging the algorithm in the minimum number of steps.&lt;/p&gt;&#xA;" OwnerUserId="20527" LastActivityDate="2016-05-29T02:03:36.500" Title="Optimal ordering in Jacobi SVD algorithm" Tags="|linear-algebra|svd|jacobian|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24097" PostTypeId="1" CreationDate="2016-05-29T07:21:29.550" Score="-3" ViewCount="132" Body="&lt;p&gt;I wrote a code in MATLAB that solves parabolic equation of two space (heat equation:$u_t = u_{xx} + u_{yy}$ )with ADI (Alternating direction implicit) method (finite difference method).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in order to test my code for initial function I want to write a function with two variables that is like &quot;M&quot; ,I mean that in $x-y$ points of &quot;M&quot; it's $z = 1$ and in the other points $z = 0$! I don't have any idea that how I can write this. suppose $x$ is in $[a,b]$ and $y$ is in $[c,d]$.&lt;/p&gt;&#xA;" OwnerUserId="20528" LastEditorUserId="17671" LastEditDate="2016-05-30T16:04:41.787" LastActivityDate="2016-05-30T16:04:41.787" Title="How to write a function that its plot is like a &quot;M&quot; in MATLAB" Tags="|matlab|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24098" PostTypeId="2" ParentId="24097" CreationDate="2016-05-29T18:03:51.370" Score="0" Body="&lt;p&gt;I suppose that the simplest thing for you to do is to draw your $M$ on a piece of quadrille paper, i.e. paper with pre-printed squares. Embed your $M$ inside a square $S$ of a suitable size, say, 20 by 20 squares. The square $S$ will correspond to your box, i.e. $[a,b] \times [c,d]$. Do not bother with too much detail, but fill out each of the 400 small sub squares as needed to get a nice $M$. Create a table with $T$ with 400 entries and let $T(k)=1$ if an only if cell $k$ is part of your $M$ and $T(k)=0$ otherwise.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2016-05-29T18:03:51.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24099" PostTypeId="1" CreationDate="2016-05-30T00:08:15.400" Score="1" ViewCount="1826" Body="&lt;p&gt;I apologize if some of my questions are naive; I am very new to computer simulations and fluid-dynamics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am going to start a PhD in early 2017, and I would like to bone-up on some Computational Fluid Dynamics to prepare. The PhD (and my background) is mostly physics-based analogous gravity, but there is a very hefty fluid-dynamics component to the course. Ideally, there will be theoretical components, simulations, and experiments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked into the most widely used solvers, and I stumbled onto a cfd-online wike, with a list of three different software suites: &lt;strong&gt;OpenFOAM, SU2, and Palabos&lt;/strong&gt;. Since fluid dynamics will only be a portion of my PhD, and simulations a portion of a portion, I'm hoping to choose one pony and stick to it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problems will be primarily dealing with will be stationary and non-stationary liquid flows in ridged-walled channels (including open-channel flows with obstacles such as airfoils, and bathtub vortex problems). I am not sure whether or not the no-slip condition will be important, but measurements of vorticity and surface waves will be. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am hoping that somebody could tell me which method would be the most efficient. I have read in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/11585/comparison-of-lattice-boltzmann-method-vs-traditional-navier-stokes-based-method&quot;&gt;another stackexchange post&lt;/a&gt; that the Lattice Boltzmann Equation (LBE) method can be easily parallel-ized. I have also read on &lt;a href=&quot;https://www.researchgate.net/post/Lattice_Boltzmann_equation_vs_Navier-Stokes_equations2&quot; rel=&quot;nofollow noreferrer&quot;&gt;reasearchgate&lt;/a&gt; that LBE &lt;em&gt;can&lt;/em&gt; be more efficient than Navier-Stokes (NS) methods, but one often requires a finer-mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also interested in which method (meaning, which of the above programs in bold) would be the easiest in which to utilize GP-GPU, and parallel GPU systems. &lt;/p&gt;&#xA;" OwnerUserId="20533" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2018-12-30T20:28:30.497" Title="New to CFD, Lattice Boltzmann or Navier-Stokes?" Tags="|fluid-dynamics|navier-stokes|gpu|lattice-boltzmann-methods|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24100" PostTypeId="1" CreationDate="2016-05-30T03:36:37.430" Score="6" ViewCount="236" Body="&lt;p&gt;I’m using a standard Crank-Nicholson algorithm to solve this system of two coupled diffusion equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\dot{u} - \dot{v} = \frac{\partial}{\partial x} \left( \alpha(x) \frac{\partial u}{\partial x}  \right) &#xA;\label{1}&#xA;\tag{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\dot{v} - \dot{u} = \frac{\partial}{\partial x} \left( \beta(x) \frac{\partial v}{\partial x}  \right) &#xA;\label{2}&#xA;\tag{2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; are fully determined functions of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. BCs are Dirichlet conditions only. I ordered the solution vector as &lt;span class=&quot;math-container&quot;&gt;$u_1, v_1, u_2, v_2, \ldots$&lt;/span&gt; to obtain a band matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear system is obtained by writing discretized &lt;span class=&quot;math-container&quot;&gt;$\eqref{1}$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$\eqref{2}$&lt;/span&gt; for the first node, then these two equations for the second node, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results I get are not completely nonsensical, but oscillations are so huge that the data cannot be exploited. Basically, solutions will be rather OK for half of the time steps, and the other ones will be extremely low negative values with little physical sense. There is no improvement when changing &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; or trying different BCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I failed at understanding where the phenomenon comes from. Could the problem be ill-posed? Or does it have something to do with the way equations are written/ordered?&lt;/p&gt;&#xA;" OwnerDisplayName="user20536" LastEditorUserId="20688" LastEditDate="2020-03-29T08:10:04.000" LastActivityDate="2020-03-29T08:10:04.000" Title="Is this system of diffusion equations well-posed?" Tags="|pde|diffusion|crank-nicolson|well-posedness|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24102" PostTypeId="2" ParentId="24097" CreationDate="2016-05-30T07:32:03.273" Score="1" Body="&lt;p&gt;Not a function, but as Kirill said in the comments, you could use &lt;code&gt;imread&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could convert a 100*100 pixel image of an M into a 100-by-100 matrix with something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M_bmp = imread('M.bmp') ;&#xA;M = 1 - double(M_bmp(:,:,1))/255;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which will pull out the red channel of the bitmap file and scale it to 1.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastActivityDate="2016-05-30T07:32:03.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24107" PostTypeId="2" ParentId="24099" CreationDate="2016-05-30T14:01:00.540" Score="1" Body="&lt;p&gt;Each method has its pros and cons. LBM ans NS solvers are both efficient and reliable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The no-slip condition may be important for physical meanings, otherwise it may be enough for you to solve Euler's equations (inviscid fluid), depending on your needs. Since you mentioned vorticity, the Navier-Stokes equation can be written in a velocity-vorticity formulation rather than the standard pressure-velocity coupling so it may be interesting for you.&#xA;You did not say if you will be working in a compressible framework, that may change a lot of things if you are dealing with high or low Mach number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the parallelism, I can't answer for LBM but for NS, again it depends on your needs and your solver (explicit or implicit), you surely can parallelize on multiple CPU if you are familiar with this concept, but if your goal is not to spend too much time in developing your own code, then yes it may be relevant to use a software.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-05-30T14:01:00.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24108" PostTypeId="2" ParentId="24085" CreationDate="2016-05-30T14:57:18.053" Score="1" Body="&lt;p&gt;For the matrix $A$ and vector $b$ subtract the first (block) row from the second row, and, analogously, subtract the last row from the one before last. In such way you eliminate the unknowns for Dirichlet boundary condition and the resulting (remaining) matrix $A$ becomes again symmetric and positive definite.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-05-30T14:57:18.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24109" PostTypeId="2" ParentId="24100" CreationDate="2016-05-30T15:19:13.960" Score="6" Body="&lt;p&gt;I think it might be ill-posed, since the time-dependent parts are linearly dependent.  If you add your two time-dependent equations together, you get a time-independent equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(\alpha(x)u_x)_x + (\beta(x) v_x)_x = 0.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do your initial conditions satisfy this equation?&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-05-30T15:19:13.960" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24110" PostTypeId="2" ParentId="24085" CreationDate="2016-05-30T19:05:51.693" Score="0" Body="&lt;p&gt;I think Peter's answer is more likely what you're looking for, but to provide an additional view:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can move your BCs to the right hand side:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ap = b$&#xA;$A(p_{interior}+p_{BCs}) = b$&#xA;$A p_{interior} = b - Ap_{BCs}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting system should be symmetric, or nearly symmetric, and might require slight modification similar to what Peter suggests (depending on if your data is cell centered or cell cornered). The advantage with this approach is that the BCs can be perceived as a source term, and removed from the left hand side.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-05-30T19:05:51.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24111" PostTypeId="1" CreationDate="2016-05-30T20:47:18.160" Score="3" ViewCount="201" Body="&lt;h2&gt;Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I have a PDE that I'm trying to solve with spectral methods. The solution $y$ is always positive, and decays as $y \propto e^{-ax}$ for large $x$. The domain is $[0, \infty)$. (There are actually two independent variables but let's not worry about that right now.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, although whatever numerical solution I obtain (using the Galerkin method) seems to have good absolute accuracy, it has poor relative accuracy towards larger $x$ due to the large dynamic range of the solution (coming from the exponential decay). I'm wondering how best to improve the relative accuracy of my solution. Any suggestions would be appreciated!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Example test problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is not the actual PDE I'm trying to solve (which is actually 2d), but it illustrates the difficulty I'm having:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(1 + e^{-x})y' = -y + 2xe^{-x},\quad y(0) = 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analytic solution is $x^2/(1+e^x)$ (plotted as dashed red below), which is always positive. However, solving this with a Laguerre function basis set (recombined so that each basis function is $0$ at $x = 0$) of order 20 yields the blue curve below, which diverges from the analytic solution (and even becomes negative) when it becomes small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qkFuF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qkFuF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What I've tried/considered&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I considered making the transformation $y = e^{-A(x)}$ and then solving for $A$. This would probably work, but unfortunately makes the equations nonlinear, which I was hoping not to deal with.&lt;/li&gt;&#xA;&lt;li&gt;I tried using Laguerre functions as a basis set, which have exponential decay built in. This didn't really work, though, as you can see above -- solutions still have low relative accuracy towards larger $x$.&lt;/li&gt;&#xA;&lt;li&gt;I'm planning to substitute $y = e^{-ax} z$ and then solve for $z$. I don't know $a$ a priori, but can estimate it. I think this will work all right, but I'm wondering if there's a more general method that I'm unaware of.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19722" LastEditorUserId="19722" LastEditDate="2016-06-01T16:52:31.573" LastActivityDate="2016-06-01T16:52:31.573" Title="Achieving high relative accuracy (vs. absolute accuracy) using spectral methods" Tags="|pde|spectral-method|accuracy|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="24112" PostTypeId="2" ParentId="24100" CreationDate="2016-05-30T21:45:03.597" Score="1" Body="&lt;p&gt;I was curious about your problem and it was not difficult for me to test CN method for your two equations. Added - parts beginning with &quot;added&quot; were written afterwards to address the comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point given by David is important. Here are three plots for initial condition (added - that do not fulfill the suggested condition $\alpha u_{xx} + \beta v_{xx}=0$), the first time step and the 5th one, computed by implicit Euler method, the blue curve is u, the other one is v:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3znwr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3znwr.png&quot; alt=&quot;Initial condition - blue is u, the other one is v &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/irYjo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/irYjo.png&quot; alt=&quot;u and v after small first time step&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fi2QA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fi2QA.png&quot; alt=&quot;u and v after 5 time steps&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The diffusion coefficients $\alpha=\beta=1$ are constant. Added - The solution changes rapidly in the first time step for arbitrary small time step that is unusual. Afterwards standard numerical diffusion can be observed (the 5th time step).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same computations with Crank-Nicolson are strange as you observed (the 1st and 5th time step):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q921o.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q921o.png&quot; alt=&quot;The first time step with Crank-Nicolson&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lhCWk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lhCWk.png&quot; alt=&quot;The 5th time step with Crank-Nicolson&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose someone can give you later comment on your PDEs directly, but from this simplified numerical analysis I would guess that you have no chance to obtain reasonable solution with Crank-Nicolson method (added - if your initial functions do not fulfill $(\alpha u_x)_x + (\beta v_x)_x=0$). I doubt if the numerical solution with implicit Euler method is reasonable, but if yes, it suggests that for initial condition that do not fulfill this PDE described by David, you obtain for arbitrary small time a jump in the solution, so you have some &quot;infinite&quot; speed of evolution. Added - The situation is even worse for explicit Euler methods as it is noted in the comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is only heuristic answer, but at least it confirms your doubts about Crank-Nicolson method for your problem. Below is the code in Mathematica that produce the pictures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added - if your initial functions fulfill the required stationary PDE (but in the approximative form using e.g. finite difference method as in the provided code) then you may use implicit ($\theta=1$ in the code) or explicit ($\theta=0$) Euler method or Crank-Nicolson method ($\theta=0.5$) for time discretization, they will work because these methods preserve this property  also for evolved numerical functions if it is fulfilled for initial functions.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Model data&#xA;t0 = 0.; T = 0.1; L = 10.; alpha = 1.; beta = 1.; &#xA;u0[x_] := If[Abs[x - 3] &amp;lt;= 1., 1., 0.]; &#xA;v0[x_] := If[Abs[x - 7] &amp;lt;= 1., -1., 0.]; &#xA;&#xA;Method's input&#xA;Nd = 5; M = 20; theta = 0.5; \[Tau] = T/Nd; h = L/(M + 1); &#xA;Mat = SparseArray[{}, {2*M, 2*M}]; &#xA;ps = Array[{}, {2*M}]; &#xA;&#xA;Method&#xA;For[i = 1, i &amp;lt;= M, i++, If[i != 1, Mat[[i,i - 1]] = (-alpha)*\[Tau]*theta]; Mat[[i,i]] = h^2 + 2*alpha*\[Tau]*theta; &#xA;   If[i != M, Mat[[i,i + 1]] = (-alpha)*\[Tau]*theta]; Mat[[i,i + M]] = -h^2; ]&#xA;For[i = M + 1, i &amp;lt;= 2*M, i++, If[i != M + 1, Mat[[i,i - 1]] = (-beta)*\[Tau]*theta]; Mat[[i,i]] = h^2 + 2*beta*\[Tau]*theta; &#xA;   If[i != 2*M, Mat[[i,i + 1]] = (-beta)*\[Tau]*theta]; Mat[[i,i - M]] = -h^2; ]&#xA;&#xA;solver = LinearSolve[Mat]; &#xA;&#xA;For[i = 0, i &amp;lt;= M + 1, i++, u[i, 0] = u0[i*h]; v[i, 0] = v0[i*h]]&#xA;&#xA;For[n = 1, n &amp;lt;= Nd, n++, &#xA;  For[i = 1, i &amp;lt;= M, i++, ps[[i]] = h^2*(u[i, n - 1] - v[i, n - 1]) + (1 - theta)*\[Tau]*alpha*(u[i - 1, n - 1] - 2*u[i, n - 1] + &#xA;         u[i + 1, n - 1]); ps[[M + i]] = h^2*(v[i, n - 1] - u[i, n - 1]) + (1 - theta)*\[Tau]*beta*(v[i - 1, n - 1] - 2*v[i, n - 1] + &#xA;         v[i + 1, n - 1])]; r = solver[ps]; u[0, n] = u[M + 1, n] = v[0, n] = v[M + 1, n] = 0.; &#xA;   For[i = 1, i &amp;lt;= M, i++, u[i, n] = r[[i]]; v[i, n] = r[[M + i]]]; ]&#xA;&#xA;n = 0; ListLinePlot[{Table[{i*h, u[i, n]}, {i, 0, M + 1}], Table[{i*h, v[i, n]}, {i, 0, M + 1}]}, PlotRange -&amp;gt; All]&#xA;n = 1; ListLinePlot[{Table[{i*h, u[i, n]}, {i, 0, M + 1}], Table[{i*h, v[i, n]}, {i, 0, M + 1}]}, PlotRange -&amp;gt; All]&#xA;n = 5; ListLinePlot[{Table[{i*h, u[i, n]}, {i, 0, M + 1}], Table[{i*h, v[i, n]}, {i, 0, M + 1}]}, PlotRange -&amp;gt; All]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-05-31T14:17:57.647" LastActivityDate="2016-05-31T14:17:57.647" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24113" PostTypeId="1" CreationDate="2016-05-31T08:42:58.693" Score="4" ViewCount="183" Body="&lt;p&gt;I'd like to simulate how several particles mix together. For example, how do they settle when you throw them in a bucket? How do they assemble in zero-gravity? I might also want that they are &quot;sticky&quot; or that they can crack. I might also want to add some not-so-rigid things like drops of glue or snippets of strings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is not to have 100% accuracy in the physics, but more to end up with particle clusters that look similiar and have similiar geometric properties as their real-world counterparts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that this is a task which can get arbitrary difficult. Currently, I'm just looking for a &lt;strong&gt;good starting point&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Terms and words to do a good internet research&lt;/li&gt;&#xA;&lt;li&gt;A good paper that can serve as a starting point&lt;/li&gt;&#xA;&lt;li&gt;A real good book, probably from game physics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-05-31T21:55:19.593" Title="Mixing some particles together - &quot;Game physics for engineers&quot;?" Tags="|computational-physics|particle|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24114" PostTypeId="2" ParentId="24113" CreationDate="2016-05-31T11:30:36.237" Score="0" Body="&lt;p&gt;Sounds like a challenging endeavor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For deformable object, check out &lt;strong&gt;level set methods&lt;/strong&gt;, e.g. this (old fashioned by still informative) website by &lt;a href=&quot;https://math.berkeley.edu/~sethian/2006/Explanations/level_set_explain.html&quot; rel=&quot;nofollow&quot;&gt;Sethian&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2016-05-31T11:30:36.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24115" PostTypeId="1" CreationDate="2016-05-31T14:00:12.677" Score="1" ViewCount="375" Body="&lt;p&gt;Solving system of constrained linear and non-linear equations in MATLAB&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm solving a FEM problem in MATLAB with use of the direct stiffness method. The problem is now formulated as a system of nn linear equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$KU=F$ or $CF=U$ with $C = K^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the problem is formulated in 3-Dd and has over 200 nodes. The model is used as a deformation estimator, the force is always the input and the deformation output. So there are a lot of linear equations. Now this problem needs to be extended, 2 types of constraints need to be implemented. Some of the nodes have to be constrained so that they only can move in a circular motion around a given point. And some of the nodes have to be bound with a maximum amount of deformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in total there will hundreds of equations of the form equations of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a_1 X(1) + a_2 X(2) + \cdots - F(1) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;about 10 equality constraints of a nonlinear form like (could also be a sinusoid):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X(1)^2 + X(2)^2 = L$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and 10 inequality constraints of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X(1) &amp;lt; 30\pi/180$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best method to solve a nonlinear system of equations with constraints of this type simultaneously? Is it possible to use fsolve? Or fmincon? I've been looking into solvers but I can't find a solid conclusion on what solver can handle all these equation types at once and what would be most efficient in terms of computation time.&lt;/p&gt;&#xA;" OwnerUserId="20465" LastEditorUserId="20465" LastEditDate="2016-06-01T10:23:08.537" LastActivityDate="2016-06-01T10:23:08.537" Title="Solving system of constrained linear and non-linear equations in MATLAB" Tags="|matlab|finite-element|nonlinear-equations|constrained-optimization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24116" PostTypeId="1" AcceptedAnswerId="35899" CreationDate="2016-05-31T15:16:39.337" Score="2" ViewCount="253" Body="&lt;p&gt;I need to find the (unique) root of a nonlinear function $f(x)$, $x \in \mathbb{R}$.&#xA;For the record, $f(x)$ is the CDF of a probability density minus a constant $0 &amp;lt; p &amp;lt; 1$ (I am inverting the CDF; see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/23592/efficient-and-stable-computation-of-inverse-cdf&quot;&gt;this question&lt;/a&gt;). The pdf is defined as a 1-D mixture of Gaussians times a polynomial. $f$ is analytical but relatively expensive to compute, so I would like to speed up the root-finding algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have been using &lt;code&gt;fzero&lt;/code&gt; in MATLAB, which implements Brent's method.&#xA;In several places in this forum (see also the question above) I have seen recommended Alefeld-Potra-Shi's method instead, which makes me think it is state-of-the art (I am not particularly familiar with the root-finding literature).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are the performance differences between Brent's and Alefeld-Potra-Shi's methods, and when is one method expected to be better than the other?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If Alefeld-Potra-Shi is potentially better for my problem, I might work on a MATLAB port of Alefeld-Potra-Shi, which I could not find so far (possibly there is an Octave implementation, and a Julia one &lt;a href=&quot;https://github.com/JuliaLang/Roots.jl/blob/master/src/fzero.jl&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="19884" LastEditorUserId="19884" LastEditDate="2016-05-31T15:25:02.697" LastActivityDate="2020-09-11T13:02:31.000" Title="Difference between Brent's and Alefeld-Potra-Shi for root finding" Tags="|matlab|efficiency|roots|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24117" PostTypeId="1" AcceptedAnswerId="24130" CreationDate="2016-05-31T15:59:58.853" Score="1" ViewCount="290" Body="&lt;p&gt;I have searched through the archives without success. Apparently, the question is simple:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What linear algebra library can I use that is parallel (shared memory) but &lt;strong&gt;without&lt;/strong&gt; OpenMP?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I've seen, OpenMP seems a requirement. Since I need to interact with colleagues using a Mac, I cannot opt for OpenMP (no, I cannot ask them to compile gcc). The usage would be a classic FEM, so matrices, vectors, and decompositions (or some kind of parallel linear solver).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intel's TBB is ok, of course, pthreads are good, standard C++ &lt;code&gt;std::thread&lt;/code&gt; would be the perfection. But I'm not looking for perfection :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="19279" LastActivityDate="2016-06-01T16:26:44.553" Title="Parallel linear algebra without OpenMP" Tags="|linear-algebra|matrix|parallel-computing|linear-solver|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24119" PostTypeId="1" CreationDate="2016-05-31T17:59:55.703" Score="1" ViewCount="797" Body="&lt;p&gt;I would like to solve a dense linear system the form in python&#xA;$$&#xA;L\left(\boldsymbol{x}\right):=\left[\gamma^+\left[\boldsymbol{A}+\frac{1}{2}\boldsymbol{B}^{-1}\right]&#xA;+\gamma^-\left[\boldsymbol{A}-\frac{1}{2}\boldsymbol{B}^{-1}\right]\right]\cdot\boldsymbol{x}=\boldsymbol{b}&#xA;$$&#xA;I thought it is a good idea &lt;strong&gt;not&lt;/strong&gt; to explicitly compute the inverse of $\boldsymbol{B}$. Therefore, I implemented the Operator $L$ to solve $L\left(\boldsymbol{x}\right)=\boldsymbol{b}$ using Krylov methods like &lt;strong&gt;cg&lt;/strong&gt; or &lt;strong&gt;gmres&lt;/strong&gt;. I am using the &lt;code&gt;scipy.sparse.linalg.LinearOperator&lt;/code&gt; class for the operator &lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.sparse.linalg.LinearOperator.html#scipy.sparse.linalg.LinearOperator&quot; rel=&quot;nofollow&quot;&gt;see the docs here&lt;/a&gt;. The product $\boldsymbol{B}^{-1}\cdot\boldsymbol{x}$ computed by another Krylov iteration or an L-U decomposition depending on system size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for larger problems I would like to improve the rate of convergence of the outer iteration. I neither have sparse matrices nor an explicit representation of my matrix. Therefore, as far as I know the &lt;em&gt;classical preconditioners&lt;/em&gt; for Krylov methods like &lt;strong&gt;ilu&lt;/strong&gt; or &lt;strong&gt;Jacobi's method&lt;/strong&gt; are not applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other methods which can be used? And are there python libraries for these methods?&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2016-06-01T18:11:44.217" Title="Preconditioning of two step iteration for dense matrices" Tags="|linear-algebra|python|scipy|preconditioning|dense-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24120" PostTypeId="2" ParentId="24119" CreationDate="2016-05-31T19:37:13.380" Score="1" Body="&lt;p&gt;I would try to circumvent the problem entirely by the substitution $x=By$. Then your equation reduces to&#xA;$$ \left[ (\gamma^+ + \gamma^-) AB + \frac{1}{2} (\gamma^+-\gamma^-) I \right] y = b.$$&#xA;If $AB$ is sparse, then I would first attempt a direct solve, and failing that, I would apply GMRES with a sparse preconditioner. If successful, then you can recover $x$ from $y$ using your current strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any extra information about the properties of the matrices $A$ and $B$ will be vital to the selection of the optimal solution strategy. It would be great to know the physical problem which generated the matrices, their dimensions, sparsity pattern, and any special mathematical properties inherited from the underlying problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: On a desktop I would want to avoid large dense matrices. On a larger parallel machine I would give ScaLAPACK a try as the LU factorization algorithm is built on top of the matrix-matrix multiplication operation. This is one of the few kernels which are compatible with the hardware of today. It has high arithmetic intensity, so the processors can run at a high fraction of their peak flop rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, I would also investigate if my dense matrix admits a good sparse approximation. I would treat my matrix as a vector and sort all components by their absolute value. This would immediately reveal if the overwhelming majority of the entries are tiny relative to the others. I would construct a preconditioner for my dense matrix by dropping all entries below a suitable threshold.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2016-06-01T18:11:44.217" LastActivityDate="2016-06-01T18:11:44.217" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24121" PostTypeId="2" ParentId="24113" CreationDate="2016-05-31T20:36:22.037" Score="4" Body="&lt;p&gt;My experience has been in visual effects where simulation time is not as important as in games, but is still more important than in engineering.  The methods I'm familiar with are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Material_point_method&quot; rel=&quot;nofollow&quot;&gt;material point method&lt;/a&gt;:  A continuum description of materials&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Discrete_element_method&quot; rel=&quot;nofollow&quot;&gt;discrete element method&lt;/a&gt;:  A discrete description&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Peridynamics&quot; rel=&quot;nofollow&quot;&gt;Peridynamics&lt;/a&gt;: Nonlocal continuum description if fracture is important&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For details on the Material Point Method in a visual effects context, see &lt;a href=&quot;http://web.cs.ucla.edu/~dt/theses/jiang-thesis.pdf&quot; rel=&quot;nofollow&quot;&gt;Jiang's 2015 UCLA dissertation&lt;/a&gt; titled &quot;The Material Point Method for the Physics-Based Simulation of Solids and Fluids.&quot;  The references cited in that work will give you an idea of what the method has been used for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Discrete Element Method, a good book is &quot;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/111856720X&quot; rel=&quot;nofollow&quot;&gt;Understanding the Discrete Element Method&lt;/a&gt;&quot; (2014) by Matuttis and Chen (ISBN: 978-1-118-56720-3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Peridynamics there are two quite different approaches - bond-based and state-based.  You can read Gerstle's &quot;Introduction to Practical Peridynamics&quot; to get a feel for the method.  A test implementation of the bond-based method can be found on &lt;a href=&quot;https://github.com/bbanerjee/ParSim/tree/master/Matiti/src&quot; rel=&quot;nofollow&quot;&gt;my github&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All these methods can be expensive but remove the need for extensive user input to get the physics to work correctly.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastEditorUserId="7626" LastEditDate="2016-05-31T21:55:19.593" LastActivityDate="2016-05-31T21:55:19.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24122" PostTypeId="2" ParentId="24113" CreationDate="2016-05-31T20:46:24.250" Score="0" Body="&lt;p&gt;Check this out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.fluidix.ca/&quot; rel=&quot;nofollow&quot;&gt;http://www.fluidix.ca/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll find references for almost every example + some nice CUDA implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't help with CFD, but for solids there's Discrete Element Method and Peridynamics. The key idea behind them is essentially the same.&lt;/p&gt;&#xA;" OwnerUserId="19145" LastActivityDate="2016-05-31T20:46:24.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24124" PostTypeId="2" ParentId="24111" CreationDate="2016-06-01T05:06:34.490" Score="2" Body="&lt;p&gt;So I went ahead and implemented a code in Matlab that can solve this problem using a spectral approach, utilizing a simple polynomial basis. Using a simple polynomial basis of order 20 resulted in the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eAfSI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eAfSI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tnR2I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tnR2I.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I did do that you might not have was define the basis to exist on a domain that might have better numerical stability (especially as x increases). I made it so my basis was defined on the domain $\zeta \in [-1,1]$. Then I used the mapping between $\zeta$ and $x$ to modify the integrals and do the necessary computation needed to find the coefficients for the basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now based on my solution, I question if you have some implementation error that leads to the inaccurate solution you have. I would expect your solution to be at least as good as what my simple polynomial basis produces, assuming implementation is correct.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-06-01T05:15:27.130" LastActivityDate="2016-06-01T05:15:27.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24125" PostTypeId="1" AcceptedAnswerId="27413" CreationDate="2016-06-01T10:17:25.790" Score="2" ViewCount="341" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Let $A\in\mathbb{R}^{n\times n}$ and its inverse be partitioned $$A = \begin{pmatrix}&#xA;A_{11} &amp;amp; A_{12}\\&#xA;A_{21} &amp;amp; A_{22}\\&#xA;\end{pmatrix},\:\:&#xA;A^{-1} = \begin{pmatrix}&#xA;\tilde{A_{11}} &amp;amp; \tilde{A_{12}}\\&#xA;\tilde{A_{21}} &amp;amp; \tilde{A_{22}}\\&#xA;\end{pmatrix}$$&#xA;  where $A_{11}\in\mathbb{R}^{k\times k}$ and $\tilde{A_{11}}\in\mathbb{R}^{k\times k}$.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;a.) Show that if, $S = A_{22} - A_{21}A_{11}^{-1}A_{12}$, the Schur complement of $A$ with respect to $A_{11}$ exists then $A$ is nonsingular iff $S$ is nonsingular.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;b.) Show that $S^{-1} = \tilde{A_{11}}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Attempted solution a.) The Schur complement arises as a result of performing a block Gaussian elimination by multiplying the matrix $A$ from the right with the block lower triangular matrix $$L = \begin{pmatrix}&#xA;I_n &amp;amp; 0\\&#xA;-A_{22}^{-1}A_{21} &amp;amp; I_m\\&#xA;\end{pmatrix}$$&#xA;where $I_n$ is a $n\times n$ identity matrix. &#xA;\begin{align*}&#xA;AL &amp;amp;= \begin{pmatrix}&#xA;A_{11} &amp;amp; A_{12}\\&#xA;A_{21} &amp;amp; A_{22}\\&#xA;\end{pmatrix} \begin{pmatrix}&#xA;I_{n} &amp;amp; 0\\&#xA;-A_{22}^{-1}A_{21} &amp;amp; I_m\\&#xA;\end{pmatrix} = \begin{pmatrix}&#xA;A_{11} - A_{12}A_{22}^{-1}A_{21} &amp;amp; A_{12}\\&#xA;0 &amp;amp; A_{22}\\&#xA;\end{pmatrix}\\&#xA;&amp;amp;= \begin{pmatrix}&#xA;I_n &amp;amp; A_{12}A_{22}^{-1}\\&#xA;0 &amp;amp; I_m\\&#xA;\end{pmatrix}\begin{pmatrix}&#xA;A_{11} - A_{12}A_{22}^{-1}A_{21} &amp;amp; 0\\&#xA;0 &amp;amp; A_{22}\\&#xA;\end{pmatrix}&#xA;\end{align*}&#xA;As you can see after multiplication with the matrix $L$ the Schur complement appears in the upper $n\times n$ block. I am not really sure where to go from here any suggestions is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="11604" LastEditDate="2017-07-11T15:48:42.037" LastActivityDate="2017-07-14T20:05:16.773" Title="Schur complement of a matrix $A$" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24129" PostTypeId="1" AcceptedAnswerId="24133" CreationDate="2016-06-01T14:20:44.520" Score="0" ViewCount="115" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider $S\in\mathbb{R}^{n\times n}$ whose nonzero elements have the following pattern for $n = 8$: &#xA;  $$\begin{pmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \mu_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \mu_2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \mu_3 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \alpha &amp;amp; \beta &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \gamma &amp;amp; \delta &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \delta_1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \delta_2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \delta_3 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;\end{pmatrix}$$&#xA;  The pattern generalizes to any $n$ easily. Assume that for any $n$, $S$ is a nonsingular matrix.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;a.) We have considered several basic transformations (Gauss transforms, Gauss Jordan transforms, elementary permutations, Household reflectors) that can be used to compute the factorizations efficiently. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Using what ever combination of these transformations, describe an algorithm to compute stably a factorization of $S$ for any $n$ that can be used to solve $Sx = b$. Your algorithm should be designed to require as few computations as possible. Your solution must include a description of how you exploit the structure of the matrix and its factors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Attempted solution - Right off the back to me this matrix looks ugly so I am thinking of putting it into a better form so essentially applying permutations to the matrix. In part of my professors solution he says: One way to approach this problem performs an priori symmetric permutation i.e., re ordering the rows and columns with the same permutation. Consider $P^T S P$ where $$P = \begin{pmatrix} e_1 &amp;amp; e_2 &amp;amp; e_3 &amp;amp; e_6 &amp;amp; e_7 &amp;amp; e_8 &amp;amp; e_4 &amp;amp; e_5\end{pmatrix}$$&#xA;I am wondering about the logic of how we gets $P$. Note that $e_i$ just denotes the position of the $1$. For example $e_1$ is just an $8$ by $1$ matrix with $1$ in the first entry followed by $0$'s in the remaining entries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So that is question $1$ for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now applying this we get $$\tilde{S} = P^T S P = \begin{pmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \mu_1\\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \mu_2\\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \mu_3\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \delta_1 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \delta_2 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; \delta_3 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \alpha &amp;amp; \beta\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \gamma &amp;amp; \delta\\&#xA;\end{pmatrix}$$&#xA;Alright, so now we have a better form to the matrix. Now I want to apply the Gauss transform to this matrix in order to get $U$ in the $LU$ factorization. Now since all the nonzero entries that we need to stress over is in the 2 last columns I believe we just need to find $M_6$ and $M_7$ to get my $U$ i.e., I am applying $$M_{n-1}\ldots M_1 A^{(n-1)} = U$$ I hope that is clear if not please let me know. Now once I do this correctly I can simply apply a forward and backward solve to compute the system $Sx = b$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions or comments in regards to this question are greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastActivityDate="2016-06-01T20:17:36.917" Title="$LU$ Factorization of a nonsingular matrix with a particular pattern" Tags="|linear-algebra|algorithms|complexity|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24130" PostTypeId="2" ParentId="24117" CreationDate="2016-06-01T16:26:44.553" Score="3" Body="&lt;p&gt;For the relatively large problems you are targeting, I suggest&#xA;you focus on the sparse matrix solver because that's where you'll&#xA;get the biggest benefit from parallelism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the comments it wasn't clear if you had access to a high-quality&#xA;multi-threaded BLAS library on all your target platforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do, and are also able to compile FORTRAN code, the MUMPS solver&#xA;is in my opinion the best free sparse solver for FE applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mumps.enseeiht.fr/&quot; rel=&quot;nofollow&quot;&gt;http://mumps.enseeiht.fr/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is actually designed for distributed memory parallelism (MPI) but it does&#xA;an amazingly-good job with just a multi-threaded BLAS on multicore&#xA;machines. It also has an out-of-core option which is very useful for&#xA;solving large problems on desktop machines with limited memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second option is SPOOLES.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/linalg/spooles/spooles.2.2.html&quot; rel=&quot;nofollow&quot;&gt;http://www.netlib.org/linalg/spooles/spooles.2.2.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an older sparse solver, not quite at the state of the art,&#xA;but it can be built with pthreads and it is written entirely in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, a third option is PaStiX. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://pastix.gforge.inria.fr/files/README-txt.html&quot; rel=&quot;nofollow&quot;&gt;http://pastix.gforge.inria.fr/files/README-txt.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't actually tried it, myself, so can't comment further except&#xA;to say that it, too, supports pthreads.  &lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-06-01T16:26:44.553" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24131" PostTypeId="2" ParentId="4890" CreationDate="2016-06-01T18:36:19.413" Score="-6" Body="&lt;p&gt;Ripplons' Biot-Savart    &lt;a href=&quot;http://www.ripplon.com/software.html&quot; rel=&quot;nofollow&quot;&gt;http://www.ripplon.com/software.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;eight more characters for the stupid post minimum character count constraint.&lt;/p&gt;&#xA;" OwnerUserId="20562" LastActivityDate="2016-06-01T18:36:19.413" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24132" PostTypeId="2" ParentId="4890" CreationDate="2016-06-01T19:12:33.750" Score="3" Body="&lt;p&gt;For this purpose you need to use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_electromagnetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;simulation software&lt;/a&gt;. One of the most common methods in Electromagnetics would be &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_element_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite element method&lt;/a&gt;, but you can also find &lt;a href=&quot;https://en.wikipedia.org/wiki/Boundary_element_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boundary Element Methods&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite Difference Methods&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some common software in EM are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ansys.com/Products/Electronics/ANSYS-Maxwell&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ansys Maxwell&lt;/a&gt;; and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cst.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CST Studio&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A lot of people is also using &lt;a href=&quot;https://www.comsol.com/comsol-multiphysics&quot; rel=&quot;nofollow noreferrer&quot;&gt;COMSOL Multiphysics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would say that this feature is implemented in a lot of different commercial and opensource applications. If your problem is 2D and you want opensource I would suggest &lt;a href=&quot;http://www.agros2d.org/features/magnetic/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agros 2D&lt;/a&gt;, here an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TUZ8z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TUZ8z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2016-06-01T19:12:33.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24133" PostTypeId="2" ParentId="24129" CreationDate="2016-06-01T20:17:36.917" Score="1" Body="&lt;p&gt;Somebody has been studying the Spike algorithm for solving tridiagonal linear systems using two processors :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless, while permuting the matrix is a valuable theoretical tool, the operation is unnecessary in practice and requires additional time in the form of data movement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead focus on the central 2 by 2 block which engages the 2 unknowns $x_4$ and $x_5$ and nothing else. You permutation reveals why this system is non singular, right? Once this system has been solved using a simple LU factorization, it is straight forward to recover $x_1, x_2$, and $x_3$ and $x_6, x_7$, and $x_8$. If $f$ denotes the right hand side of the (Spike) system $Sx = f$, then you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_j + \mu_j x_5 = f_j, \quad j=1,2,3,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta_j x_4 + x_{5 + j} = f_{5+j}, \quad j = 1,2,3.$$&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2016-06-01T20:17:36.917" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24134" PostTypeId="5" CreationDate="2016-06-01T21:42:22.433" Score="0" Body="&lt;p&gt;This tag has been deprecated because it is vague, and is a candidate for blacklisting. Consider instead using tags like &lt;a href=&quot;/questions/tagged/linear-algebra&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;linear-algebra&amp;#39;&quot; rel=&quot;tag&quot;&gt;linear-algebra&lt;/a&gt; (for systems of linear equations), &lt;a href=&quot;/questions/tagged/linear-solver&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;linear-solver&amp;#39;&quot; rel=&quot;tag&quot;&gt;linear-solver&lt;/a&gt; (for algorithms to solve systems of linear equations), or &lt;a href=&quot;/questions/tagged/linear-programming&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;linear-programming&amp;#39;&quot; rel=&quot;tag&quot;&gt;linear-programming&lt;/a&gt; (for optimization problems consisting only of a linear objective and linear constraints).&lt;/p&gt;&#xA;" OwnerUserId="276" LastEditorUserId="276" LastEditDate="2016-06-01T21:42:22.433" LastActivityDate="2016-06-01T21:42:22.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24135" PostTypeId="4" CreationDate="2016-06-01T21:42:22.433" Score="0" Body="This tag has been deprecated because it is vague, and is a candidate for blacklisting. Consider instead using tags like [tag:linear-algebra] (for systems of linear equations), [tag:linear-solver] (for algorithms to solve systems of linear equations), or [tag:linear-programming] (for optimization problems consisting only of a linear objective and linear constraints)." OwnerUserId="276" LastEditorUserId="276" LastEditDate="2016-06-01T21:42:22.433" LastActivityDate="2016-06-01T21:42:22.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24139" PostTypeId="1" CreationDate="2016-06-02T10:39:01.440" Score="2" ViewCount="253" Body="&lt;p&gt;I have a 3D integral that is almost a radial convolution of the form&#xA;$$ \int d^{3}k'h(\mathbf{k'})g(|\mathbf{k-k'}|) $$&#xA;and I am looking for a fast and efficient algorithm (e.g. FFT) to solve it numerically. In the following I give you a brief description of the functions I am using and the problems I encountered.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;The functions and the radial case&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Consider the Fourier transform of a spherical top hat function:&#xA;$$ \tilde{W}_{i}(k,R_{i},R_{i+1})=\frac{1}{V_{i}}\int W_{i}(r)e^{i\mathbf{k\cdot r}}d^{3}r\\=\frac{4\pi}{V_{i}k}\int_{R_{i}}^{R_{i+1}}\sin(kr)rdr\equiv\tilde{W}_{i}(k)$$&#xA;where $R_{i}$ are some radiuses. If $f(k)$ is a general function (it comes from an interpolation, so there is no analytical formula) the convolution integral&#xA;$$ h_{ij}(k)=\int d^{3}k'f(|\mathbf{k'}|)\tilde{W_{i}}\left(|\mathbf{k}-\mathbf{k'}|\right)\tilde{W}_{j}\left(|\mathbf{k}-\mathbf{k'}|\right)$$&#xA;is radial, thus it can be solved with a one-dimensional FFT algorithm.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;The $k$-derivative and the convolution $d_{ij}^{(2)}(\mathbf{k})$&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Considering now the $k$-derivative of one of the two window function, some terms in the integral are no longer radial:&#xA;$$\int d^{3}k'f(|\mathbf{k'}|)\tilde{W_{i}}\left(|\mathbf{k}-\mathbf{k'}|\right)\frac{\partial\tilde{W}_{j}\left(|\mathbf{k}-\mathbf{k'}|\right)}{\partial k}	=	\\ = \int d^{3}k'f(k')\frac{\tilde{W_{i}}\left(|\mathbf{k}-\mathbf{k'}|\right)}{|\mathbf{k}-\mathbf{k'}|}\frac{\partial\tilde{W}_{j}\left(|\mathbf{k}-\mathbf{k'}|\right)}{\partial|\mathbf{k}-\mathbf{k'}|}\left(k-\frac{\mathbf{k}\cdot\mathbf{k}'}{k}\right) \\&#xA;	\equiv	kd_{ij}^{(1)}(k)-\frac{\mathbf{k}}{k}\cdot d_{ij}^{(2)}(\mathbf{k}).$$&#xA; Particularly the integral $d_{ij}^{(2)}$   is still a convolution but it isn't radial:&#xA;$$d_{ij}^{(2)}(\mathbf{k})=\int d^{3}k'f(|\mathbf{k'}|)\mathbf{k'}\frac{\tilde{W_{i}}\left(|\mathbf{k}-\mathbf{k'}|\right)}{|\mathbf{k}-\mathbf{k'}|}\frac{\partial\tilde{W}_{j}\left(|\mathbf{k}-\mathbf{k'}|\right)}{\partial|\mathbf{k}-\mathbf{k'}|}$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Problems&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Now, solving the integral by using common integration procedures is not really doable, because the functions $\tilde{W}_{i}(k)$ oscillate really rapidly so the numerical integral takes a lot of time and the result is never really reliable and it depends a lot on the chosen precision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand a three dimensional FFT algorithm is really memory consuming. In fact $\tilde{W_{i}}(k)$ is a rapidly oscillating function and it needs a lot of points to be well sampled (around 5000/7000 points for each dimension) thus in three dimensions the matrices become really big. But for example I don't know anything about possible methods to parallelize the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also checked other possibilities like Hankel transforms, Bessel functions for the angular part or Legendre polynomials, but without any success. If it was possible to rewrite it in the form of a 2-dimensional convolution, perhaps it would also be enough.&lt;/p&gt;&#xA;" OwnerUserId="20567" LastEditorUserId="20567" LastEditDate="2016-06-02T10:44:18.567" LastActivityDate="2016-06-02T10:44:18.567" Title="Solving a 3D (almost radial) convolution with FFT" Tags="|algorithms|computational-physics|fourier-transform|integration|fftw|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24140" PostTypeId="1" CreationDate="2016-06-02T11:12:28.067" Score="2" ViewCount="490" Body="&lt;p&gt;I'm currently evaluating how I could implement several algorithms on a parallel computer with a large number of nodes, each also equipped with a powerful GPU. It turns out that for many algorithms, it would be useful to &quot;just&quot; have a datastructure that is distributed over many nodes and on whose elements you can run a function in the style of a for-each kernel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, it could &lt;em&gt;for example&lt;/em&gt; boil-down to a distributed implementation of C++'s STL. It would also mean, that you could just impelement the algorithm in a familiar mean and would (mostly) not need to bother about parallel details. It would also be cool to have Python bindings: Write some high-level python code and the low-level distributed stuff happens magically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some research about existing projects in this direction:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There's for example &lt;a href=&quot;https://en.wikipedia.org/wiki/Stapl&quot; rel=&quot;nofollow&quot;&gt;STAPL&lt;/a&gt; but I could not really find significantly more information, and did not get a response on my email.&lt;/li&gt;&#xA;&lt;li&gt;There are linear algebra packages such as Trilinos and PETSc, but they are mostly designed for linear algebra and they are also huge frameworks which need significant effort to be incorporated into own projects.&lt;/li&gt;&#xA;&lt;li&gt;GPU support is generally poor (I'm aware that you can not straight forward use a GPU for general purpose data structures.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Are there any serious efforts on distributed general purpose data structures that could be used as a basis for parallelization of existing code? I would be happy to get some references, papers or search words.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-10-06T17:24:55.633" Title="State of the art in parallel data structures" Tags="|python|parallel-computing|c++|mpi|" AnswerCount="3" CommentCount="2" ClosedDate="2016-10-08T21:03:08.847" ContentLicense="CC BY-SA 3.0" />
  <row Id="24141" PostTypeId="1" CreationDate="2016-06-02T12:25:23.817" Score="0" ViewCount="78" Body="&lt;p&gt;I am working with the equations for incompressible viscous fluid:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\partial_t \vec{\omega} + (\vec{u}\cdot\nabla)\vec{\omega} = \nu\nabla^2\vec{\omega}&#xA;$$&#xA;$$&#xA;\nabla^2 \vec{\psi} = -\vec{\omega}&#xA;$$&#xA;$$&#xA;\vec{u}=\nabla \times \vec{\psi}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with an usual notation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\omega$ ... vorticity&lt;/li&gt;&#xA;&lt;li&gt;$u$ ... velocity&lt;/li&gt;&#xA;&lt;li&gt;$\nu$ ... kinematic viscosity&lt;/li&gt;&#xA;&lt;li&gt;$\psi$ ... stream function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I need to discretize the system (by means of finite differences). Since this is very common tasks, the schemes must be somewhere available, but I was not lucky in googling this time. Could you please share a solution?&lt;/p&gt;&#xA;" OwnerUserId="17721" LastActivityDate="2016-06-02T12:25:23.817" Title="Finite differences for incompressible viscous fluid equations" Tags="|pde|finite-difference|navier-stokes|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24142" PostTypeId="1" AcceptedAnswerId="24150" CreationDate="2016-06-02T12:37:05.087" Score="2" ViewCount="584" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Consider $A\in\mathbb{R}^{n\times n}$ whose nonzero elements are restricted to the main diagonal the strict upper triangular part, and the first subdiagonal. For $n = 8$ the locations that must be zero are indicated and the positions that may be nonzero are indicated by $\alpha_{ij}$:&#xA;  $$\begin{pmatrix}&#xA;\alpha_{11} &amp;amp; \alpha_{12} &amp;amp; \alpha_{13} &amp;amp; \alpha_{14} &amp;amp; \alpha_{15} &amp;amp; \alpha_{16} &amp;amp; \alpha_{17} &amp;amp; \alpha_{18}\\&#xA;\alpha_{21} &amp;amp; \alpha_{22} &amp;amp; \alpha_{23} &amp;amp; \alpha_{24} &amp;amp; \alpha_{25} &amp;amp; \alpha_{26} &amp;amp; \alpha_{27} &amp;amp; \alpha_{28}\\ &#xA;0 &amp;amp; \alpha_{32} &amp;amp; \alpha_{33} &amp;amp; \alpha_{34} &amp;amp; \alpha_{35} &amp;amp; \alpha_{36} &amp;amp; \alpha_{37} &amp;amp; \alpha_{38}\\&#xA;0 &amp;amp; 0 &amp;amp; \alpha_{43} &amp;amp; \alpha_{44} &amp;amp; \alpha_{45} &amp;amp; \alpha_{46} &amp;amp; \alpha_{47} &amp;amp; \alpha_{48}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \alpha_{54} &amp;amp; \alpha_{55} &amp;amp; \alpha_{56} &amp;amp; \alpha_{57} &amp;amp; \alpha_{58}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \alpha_{65} &amp;amp; \alpha_{66} &amp;amp; \alpha_{67} &amp;amp; \alpha_{68}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \alpha_{76} &amp;amp; \alpha_{77} &amp;amp; \alpha_{78}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \alpha_{87} &amp;amp; \alpha_{88}\\&#xA;\end{pmatrix}$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;i.) Suppose the subdiagonal elements $\alpha_{i+1,i} \neq 0$ (this is called an unreduced Hessenberg matrix). Determine a necessary and sufficient condition for $A$ to be nonsingular.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Attempted solution - If $\det(A)\neq 0$ then $A$ is nonsingular.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ii.) Describe an efficient algorithm to solve $Ax = b$ via factorization and determine the order computational complexity, i.e., give $k$ in $O(n^k)$. Your solution should include a description of how you exploit the structure of the matrix and how it influences the structure of your factors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Attempted solution - I am thinking of just using the $LU$ factorization and getting $A$ such that $A = L + D + L^T$ then I can just calculate $Lx$, $Dx$, and $L^T x$ and sum the results (Carl Christian) recommended this in another exercise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also since $A$ is almost upper trapezoidal we could simply apply the Gauss transform matrices $M_1, M_2,\ldots, M_7$ to get $U$ then we can easily find $L$ and then we would just use a forward and backward solve to compute $Ax = b$. This will still result in $O(n^2)$ computations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways these type of questions are challenging for me, if anyone has any suggestions I would greatly appreciate it. Also, I want to know what constitutes as a complete solution for b.) as in what do I need to show in my solution to satisfy the conditions asked.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="20417" LastEditDate="2017-03-23T15:58:02.080" LastActivityDate="2017-03-24T10:26:57.053" Title="Determine a sufficient condition for a Hessenberg matrix to be nonsingular" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24143" PostTypeId="2" ParentId="24140" CreationDate="2016-06-02T12:53:38.460" Score="0" Body="&lt;p&gt;I've had the same question recently. One interesting development are the &lt;a href=&quot;http://www.hsafoundation.com/&quot; rel=&quot;nofollow&quot;&gt;HSA specifications&lt;/a&gt; targeting heterogeneous hardware that are supposedly implemented in &lt;a href=&quot;https://gcc.gnu.org/gcc-6/changes.html&quot; rel=&quot;nofollow&quot;&gt;GCC 6.1&lt;/a&gt; (I haven't tried it yet). Sounds like the idea is to unify memory access and floating point operations across heterogeneous devices with one programming model.&lt;/p&gt;&#xA;" OwnerUserId="6702" LastActivityDate="2016-06-02T12:53:38.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24144" PostTypeId="1" AcceptedAnswerId="24147" CreationDate="2016-06-02T22:57:20.877" Score="0" ViewCount="297" Body="&lt;p&gt;A user has this nice graph of f(x) on his profile and I want to learn how to do it in matlab or Gnu octave. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pIl9T.png?s=128&amp;amp;g=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pIl9T.png?s=128&amp;amp;g=1&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(x) = -(x*x*x +x + 1)&#xA;(0,0) = center&#xA;color(x,y) = arctan2(x, f(x))&#xA;lum(x,y) = sin(hypot(x, f(x))/alpha)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But when I try in matlab I fail:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;&amp;gt; y= -(x.*x.*x+x+1)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;&amp;gt; x = linspace(0,2*pi,100);&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;&amp;gt; plot(x,y)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matlab code does not make a plot that resembles the nice visual graph. I understand that I must also accomplish the effects, but can you help me on the way? Is there a program that is more or less suitable to make a plot like the above?&lt;/p&gt;&#xA;" OwnerUserId="3566" LastActivityDate="2016-06-03T18:22:44.823" Title="Nice plot of f(x)" Tags="|matlab|plotting|octave|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24145" PostTypeId="1" CreationDate="2016-06-03T15:52:17.047" Score="1" ViewCount="281" Body="&lt;p&gt;I am solving a non-linear advection equation of the form $u_t + f(u)_x = 0$ where $f(u)$ is a complicated function of $u$. I am solving this equation using a first order fully implicit scheme (backward Euler in time and upwind for space) with second order linearization using Newton's method for non-linear convergence. In order to analyze the results (whether they make sense physically) I wanted to get an idea of the general characteristics that they may exhibit (like shocks and rarefactions) as a function of the nature of $f(u)$. However, in the literature, every treatment of non-linear advection that I find ultimately ends up dealing with inviscid Burgers equation. While I understand the rationale, I would like to look at some other examples. Specifically, my flux function can be convex or concave i.e., $f^{\prime\prime}(u)$ can be $&amp;lt; 0$ or $&amp;gt; 0$ and I wanted to look at cases where the non-linear wave-speeds $f^\prime(u)$ are something other than $u$ itself. Any relevant reference is much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4837" LastEditorUserId="4837" LastEditDate="2016-08-26T14:39:14.697" LastActivityDate="2016-08-26T14:39:14.697" Title="Numerical solution of non-linear advection equation other than inviscid burgers" Tags="|nonlinear-equations|advection|nonconvex|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24146" PostTypeId="1" CreationDate="2016-06-03T16:33:07.073" Score="1" ViewCount="124" Body="&lt;p&gt;I want to solve a system of ordinary differential equations with Matlab. I need this to solve a mechanical engineering related problem. &#xA;If $n$ is the number of degrees of freedom of my mechanical system, then the number of variables (and also of equations, obviously) is $n^2$.&#xA;I can easily numerically integrate the ODE for example for $n=6$. I can tell that the problem is stiff because ode45 takes a very long time, while ode15s takes seconds and gives accurate results.&#xA;For a 13 degrees of freedom system (i.e. 169 variables) instead the computations are extremely slow and that's because the step size is tiny, at the beginning is $\approx 10^{-18}$, after one minute it becomes $\approx 10^{-15}$ and it gets bigger and bigger but at some point I have to stop the computation and the integration hasn't reached $10^{-10}$ (whereas with $n=6$ I easily integrate from $0$ to $100$).&#xA;I was wondering if this is something I could expect because of the 169 variables. What can I do in this cases? I tried all the available solvers for stiff problems but it's not any better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I understand that the 6 degrees of system and the 13 degrees of freedom system are just different models so I can't make a comparison, but from the mechanical point of view the bigger one is not much more complex.&#xA;I'm very naive for what concerns numerical solution of ODEs and I expected that doubling the size of the problem (thus solving 4x equations) would have led to (roughly) 4x computation time. Of course it cannot be as simple as this, but I'm surprised by the difference between the two models. The computation time was &quot;seconds&quot; for the small system and it has become &quot;neverending&quot; for the big one!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I eventually discovered what my problem was! As I said, the ODE was related to a mechanical engineering problem. The issue was that the variables represented actual physical quantities the were in different units and &lt;strong&gt;very&lt;/strong&gt; different orders of magnitude! I just scaled the problem by means of appropriate scalar factors and the computation time decreased significantly! Obviously, you have to scale the solution back to the right units, but it's not a big deal. For additional information I recommend this paper &lt;a href=&quot;http://dx.doi.org/10.1137/S0895479803434914&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1137/S0895479803434914&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20582" LastEditorUserId="20582" LastEditDate="2017-09-30T16:05:58.017" LastActivityDate="2017-09-30T16:05:58.017" Title="Stiffness emerges as number of ODEs increases" Tags="|matlab|ode|stiffness|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24147" PostTypeId="2" ParentId="24144" CreationDate="2016-06-03T18:22:44.823" Score="4" Body="&lt;p&gt;Using Python and &lt;a href=&quot;http://mpmath.org/&quot;&gt;mpmath&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib&#xA;import mpmath&#xA;f = lambda z: z**3-z+1&#xA;mpmath.cplot(f, points=100000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Easy peasy.... See &lt;a href=&quot;http://mpmath.org/doc/current/plotting.html#complex-function-plots&quot;&gt;here&lt;/a&gt; for the documentation on &lt;code&gt;mpmath.cplot&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2016-06-03T18:22:44.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24149" PostTypeId="1" AcceptedAnswerId="24174" CreationDate="2016-06-03T18:31:08.033" Score="0" ViewCount="140" Body="&lt;p&gt;Recently, I read a paper &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/nla.2004/abstract&quot; rel=&quot;nofollow&quot;&gt;[1]&lt;/a&gt; and then I want to handle the two-dimensional linear integro-differential equation&#xA;\begin{equation*}&#xA;-\triangle u + q\Big(\frac{\partial u}{\partial x} + \frac{\partial u}{\partial y}&#xA;\Big) + \gamma\int^{y}_0\int^{x}_0 \mathcal{K}(x, y, s,t) u(s,t)ds dt = g(x, y),\quad\&#xA;(x, y) \in \Omega.&#xA;\end{equation*}&#xA;with the homogeneous Dirichlet boundary condition, where $\Omega = (0, 1)\times (0, 1)$ &#xA;is the unit square, $\triangle$ is the two-dimensional Laplace operator, $\mathcal{K}(x,&#xA;y, s, t)$ is the integral kernel function, $g(x, y)$ is a given function, and $q$ and $&#xA;\gamma$ are prescribed positive parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After approximating the derivatives by the nine-point centered differences and the integral &#xA;by the trapezoidal quadrature formula, with the stepsizes being both equal to $h = \frac{1}&#xA;{\sqrt{m} + 4}$, and normalizing the discretized coefficient matrix and right-hand side by &#xA;multiplying $h^2$, we obtain a linear system of the form&#xA;\begin{equation*}&#xA;(T + \zeta B +  \nu K)x =  b,&#xA;\end{equation*}&#xA;where $\zeta = qh$ and $\nu = \gamma h^4$ are positive constants.&#xA;\begin{equation*}&#xA;T = T_1 \otimes I + I \otimes T_1 \in \mathbb{R}^{m\times m}\quad\&#xA;\mathrm{and}\quad\ B = B_1 \otimes I + I \otimes B_1 \in \mathbb{R}^{m\times m}&#xA;\end{equation*}&#xA;are the two-dimensional discrete matrices corresponding to the difference and the integral operators,&#xA;\begin{equation*}&#xA;T1 = \frac{5}{2} I - \frac{4}{3}(E_1 + E^{T}_1) + \frac{1}{12}(E_2 + E^{T}_2)\quad\ &#xA;\mathrm{and}\quad\ B_1 = \frac{2}{3}(E_1 - E^{T}_1) - \frac{1}{12}(E_2 - E^{T}_2)&#xA;\end{equation*}&#xA;are the one-dimensional discrete matrices corresponding to the difference and the integral operators, &#xA;with $E_1,~E_2\in \mathbb{R}^{\sqrt{m}\times\sqrt{m}}$ being given by,&#xA;\begin{equation*}&#xA;E_1 = \begin{pmatrix}&#xA;0 &amp;amp;1                     \\&#xA;  &amp;amp;0 &amp;amp; 1                 \\&#xA;  &amp;amp;  &amp;amp; 0 &amp;amp; 1             \\&#xA;  &amp;amp;  &amp;amp;   &amp;amp;\ddots &amp;amp;\ddots \\&#xA;  &amp;amp;  &amp;amp;   &amp;amp;       &amp;amp;0     &amp;amp; 1\\&#xA;  &amp;amp;  &amp;amp;   &amp;amp;       &amp;amp;      &amp;amp; 0&#xA;\end{pmatrix}\quad\ {\rm and}\quad\ &#xA;E_2 = \begin{pmatrix}&#xA;0 &amp;amp;0 &amp;amp; 1                     \\&#xA;  &amp;amp;0 &amp;amp; 0 &amp;amp; 1                 \\&#xA;  &amp;amp;  &amp;amp;\ddots &amp;amp;\ddots &amp;amp;\ddots \\&#xA;  &amp;amp;  &amp;amp;   &amp;amp;0      &amp;amp;0     &amp;amp; 1  \\&#xA;  &amp;amp;  &amp;amp;   &amp;amp;       &amp;amp;0     &amp;amp; 0  \\&#xA;  &amp;amp;  &amp;amp;   &amp;amp;       &amp;amp;      &amp;amp; 0&#xA;\end{pmatrix}&#xA;\end{equation*}&#xA;and $K = (K_{ij})\in\mathcal{R}^{m\times m}$ is defined by&#xA;\begin{equation*}&#xA;k_{ij} = x_{i_1}y_{i_2}\mathcal{K}(x_{i_1}, y_{i_2}, x_{j_1}, y_{j_2}),&#xA;\end{equation*}&#xA;with &#xA;\begin{equation*}&#xA;i = \sqrt{m}(i_2 - 1) + i_1 \quad\ {\rm and}\quad\ j = \sqrt{m}(j_2 - 1) + j_1,\quad&#xA;i_1,~i_2,~j_1,~j_2 = 1,2,\ldots,\sqrt{m}.&#xA;\end{equation*}&#xA;Here we choose&#xA;\begin{equation*}&#xA;\mathcal{K}(x,y,s,t) = \frac{\mu(\sqrt{\psi} + 4)}{xy} \Big[\sqrt{\psi}(y - t) + (x - s)&#xA;\Big],&#xA;\end{equation*}&#xA;with $\psi$ being a positive constant, so that&#xA;\begin{equation*}&#xA;K_{ij} = \frac{\mu(\sqrt{\psi} + 4}{\sqrt{m} + 4}\Big[\sqrt{\frac{\psi}{m}}(i - j)+ &#xA;\Big(1 - \sqrt{\frac{\psi}{m}}\Big) (i_1 - j_1)\Big],\quad\ i,j = 1,2,\ldots,m.&#xA;\end{equation*}&#xA;For this problem, the matrix $T$ is symmetric positive definite, and the matrices $B$ &#xA;and $K$ are skew-symmetric. Besides, both $T$ and $B$ are sparse, but $K$ is dense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problems: (1) How to evaluate and assemble the dense matrix $K$ by MATLAB? (There are $i_1,i_2,j_1,j_2,i,j$ make me confused!)&#xA;(2) How to know that the dense matrix $K$ is skew-symmetric? &#xA;(3) It seems that we have $i_1 = i_2,~j_1 = j_2$?&lt;/p&gt;&#xA;" OwnerUserId="10290" LastEditorUserId="10290" LastEditDate="2016-06-03T19:50:54.997" LastActivityDate="2016-06-07T19:37:41.097" Title="Discretized matrix from the integral kernel function" Tags="|matlab|finite-difference|dense-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24150" PostTypeId="2" ParentId="24142" CreationDate="2016-06-03T19:16:35.280" Score="3" Body="&lt;p&gt;Write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{A} = \begin{bmatrix} \mathrm{r}^{\top} &amp;amp; \alpha_{18}\\ \mathrm{U} &amp;amp; \mathrm{c}\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathrm{U} \in \mathbb{R}^{(n-1) \times (n-1)}$ is an upper triangular matrix. There is a permutation matrix $\mathrm{P}$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{\mathrm{A}} \mathrm{\mathrm{P}} = \begin{bmatrix} \alpha_{18} &amp;amp; \mathrm{r}^{\top}\\ \mathrm{c} &amp;amp; \mathrm{U}\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;whose determinant is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\det (\mathrm{AP}) = \det (\mathrm{A}) \cdot \underbrace{\det(\mathrm{P})}_{=\pm1} = \det (\mathrm{U}) \cdot  (\alpha_{18} - \mathrm{r}^{\top} \mathrm{U}^{-1} \mathrm{c})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As $\mathrm{U}$ is upper triangular, its determinant is the product of its entries on the main diagonal. Thus, if there are no zero entries on the main diagonal of $\mathrm{U}$, then $\mathrm{U}$ is invertible. If $\mathrm{U}$ is invertible and $\alpha_{18} \neq \mathrm{r}^{\top} \mathrm{U}^{-1} \mathrm{c}$, then we have $\pm \det (\mathrm{A}) \neq 0$, i.e., $\mathrm{A}$ is non-singular. To summarize, we have the following sufficient condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(\displaystyle\prod_{i=1}^{n-1} u_{ii} \neq 0\right) \land \left(\alpha_{18} \neq \mathrm{r}^{\top} \mathrm{U}^{-1} \mathrm{c}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if $\mathrm{U}$ is invertible, then $\mathrm{U}^{-1} \mathrm{c}$ is the unique solution to the linear system $\mathrm{U} \mathrm{y} = \mathrm{c}$, whose augmented matrix is $[\mathrm{U}\,|\,\mathrm{c}]$, which is a submatrix of $\mathrm{A}$ (namely, its last $n-1$ rows).&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2017-03-24T10:26:57.053" LastActivityDate="2017-03-24T10:26:57.053" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="24152" PostTypeId="1" AcceptedAnswerId="24162" CreationDate="2016-06-04T08:07:54.913" Score="0" ViewCount="81" Body="&lt;p&gt;I have a little question that might be basic for some experts, but right now, its not clear for me.&#xA;I want to implement temperature depending viscosity in a finite difference scheme (incompressible fluid). Therefore, I get for the viscous term&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial}{\partial x_j} \left( \nu \left(\frac{\partial u_i}{\partial x_j} + \frac{\partial u_j}{\partial x_i}\right) \right) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I equate this using the product rule (noting that $\nu$ is spatial variable):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \nu}{\partial x_j} \left( \frac{\partial u_i}{\partial x_j} + \frac{\partial u_i}{\partial x_j} \right) + \nu \left( \frac{\partial^2 u_i}{\partial x_j \partial x_j} + \frac{\partial^2 u_j}{\partial x_j \partial x_i} \right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Making use of Schwarz's theorem, I can switch the cross derivatives in the last term. Because of the incompressibility condition ($\frac{\partial u_i}{\partial x_i} = 0$), I expect the last term to vanish, i.e. ending up with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \nu}{\partial x_j} \left( \frac{\partial u_i}{\partial x_j} + \frac{\partial u_i}{\partial x_j} \right) + \nu \frac{\partial^2 u_i}{\partial x_j \partial x_j} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question now is rather simple: Is this correct?? I had some doubts since I remember some discussions that in case of temperature dependent viscosity one needs to implement cross-derivatives of the velocity field, and that these cross-derivatives are somehow tricky regarding the order of the numerical scheme. However, I don't need them here and this confuses me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be happy if someone could help me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&#xA;Marius&lt;/p&gt;&#xA;" OwnerUserId="20588" LastActivityDate="2016-06-05T20:41:10.297" Title="Implementing temperature depending viscosity in a finite-difference scheme" Tags="|finite-difference|fluid-dynamics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24156" PostTypeId="1" AcceptedAnswerId="24158" CreationDate="2016-06-04T17:04:21.103" Score="1" ViewCount="74" Body="&lt;p&gt;As the title suggests, I'm looking for a very accurate algorithm to find the area under an oscillatory  function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, I would like to integrate the following graph&lt;a href=&quot;https://i.stack.imgur.com/CfMK4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CfMK4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently using simpson's method to integrate this, but I was wondering if there are more accurate methods.&lt;/p&gt;&#xA;" OwnerUserId="16650" LastEditorUserId="16650" LastEditDate="2016-06-04T20:28:08.360" LastActivityDate="2016-06-04T20:28:08.360" Title="Looking for an accurate algorithm to find the area of a oscillatory function" Tags="|algorithms|integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24158" PostTypeId="2" ParentId="24156" CreationDate="2016-06-04T17:48:11.277" Score="6" Body="&lt;p&gt;You can look up various quadratures. One method that should fair better is Gauss Quadrature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also recommend looking into any adaptive quadrature schemes. There are many of them out there, so searching up algorithms for adaptive integration would help you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Adaptive_quadrature&quot;&gt;This wikipedia link&lt;/a&gt; should give you some insight into adaptive integration.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-06-04T17:48:11.277" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24161" PostTypeId="1" CreationDate="2016-06-04T22:02:08.147" Score="4" ViewCount="89" Body="&lt;p&gt;I have a 3-tensor $A = A_{ijk}$ with each dimension between 9 and 25 (roughly), and an integer $n &amp;gt; 0$.  I would like the factor this tensor as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{ijk} = \sum_{0 \le \alpha \lt n} B_{\alpha i} C_{\alpha j} D_{\alpha k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;such that that $B$, $C$, and $D$ are optimally conditioned possibly rectangular matrices.  I'm not sure whether optimally should mean max, sum, or product of $\operatorname{cond}(B)$, $\operatorname{cond}(C)$, $\operatorname{cond}(D)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there algorithms to do this?  Software?  I expect it's an NP-complete problem, but $A$ is fixed and I'm willing to spend quite a lot of compute time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Clarification&lt;/strong&gt;: As noted in the comments, I'm specifically interested in the overcomplete case $n &amp;gt; \operatorname{rank}(A)$, where more terms than necessary are used in order to improve the conditioning.&lt;/p&gt;&#xA;" OwnerUserId="447" LastEditorUserId="447" LastEditDate="2016-06-06T04:13:46.817" LastActivityDate="2016-06-06T04:13:46.817" Title="Optimally conditioned 3-tensor factorization" Tags="|tensor-decomposition|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24162" PostTypeId="2" ParentId="24152" CreationDate="2016-06-05T20:41:10.297" Score="1" Body="&lt;p&gt;I think there are two parts to your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first part is the correctness of the analytic expression you've written. All of the expressions you've written, as far as I can tell, are correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part is the correctness (or better, favorableness) of the resulting finite difference discretization. It turns out that different resulting discretizations (e.g. the one you'd get from the first and last expressions) have different conservation properties. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @Bill Barth suggested, a finite element, or finite volume, approach has nice conservation properties. You can read about different finite difference discretizations and their conservation properties here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vasilyev, O. V. High Order Finite Difference Schemes on Non-uniform Meshes with Good Conservation Properties. J. Comput. Phys. 157, 746–761 (2000).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you use the first expression, as this will have nice conservation properties.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-06-05T20:41:10.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24164" PostTypeId="1" CreationDate="2016-06-06T10:51:04.693" Score="0" ViewCount="1955" Body="&lt;p&gt;I have a large vector with complex numbers. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I check whether it a conjugate symmetry vector?&lt;/li&gt;&#xA;&lt;li&gt;If not, is there any way to transform it?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20363" LastEditorUserId="20417" LastEditDate="2016-06-06T14:30:03.317" LastActivityDate="2016-06-07T17:01:45.893" Title="Conjugate symmetry in MATLAB?" Tags="|matlab|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24166" PostTypeId="2" ParentId="24164" CreationDate="2016-06-06T13:18:25.130" Score="2" Body="&lt;p&gt;You reverse the order of the components of a vector with the &quot;flip&quot; command. Complex conjugation is done with the &quot;conj&quot; command. So, you will want to check if conj(flip(x)) equals x or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The term conjugate symmetric vector was new to me. It appears to be a term used exclusively by some members of the signal processing community. A vector $x \in \mathbb{C}^n$ is conjugate symmetric if and only if $x = J_n\, \bar{x}$, where $J_n$ denotes the $n$ by $n$ anti-diagonal exchange matrix and $\bar{x}$ denotes complex conjugation. The action of $J_n$ on a vector $x \in \mathbb{C}^n$ consists of reversing the order of the elements of $x$.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2016-06-07T17:01:45.893" LastActivityDate="2016-06-07T17:01:45.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24167" PostTypeId="1" AcceptedAnswerId="24171" CreationDate="2016-06-06T16:23:27.417" Score="1" ViewCount="110" Body="&lt;p&gt;Let $\Omega\subset \mathbb{R}^d$, $d\in \{2,3\}$ be an open bounded polygonal/polyhedral set.  Suppose I want to solve the following pde&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\vec{q}+\vec{\nabla}u &amp;amp;=0\,&amp;amp;x&amp;amp;\in \Omega\\&#xA;\vec{\nabla} \cdot \vec{q} &amp;amp;= f\, \quad&amp;amp; x&amp;amp;\in \Omega\\&#xA;u&amp;amp;=0\,&amp;amp; x&amp;amp;\in \partial\Omega_{\mathrm{dir}}\\&#xA;\vec{q}\cdot \vec{\eta}&amp;amp;= 0\,&amp;amp; x&amp;amp;\in \partial\Omega_{\mathrm{neu}}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid complications assume that the measure of the Dirichlet boundary is nonzero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose further that there is triangulation of $\Omega$, $T$, such that the source function $f$ is piecewise polynomial with respect to $T$.  Suppose that $T$ has no &quot;hanging nodes&quot;.  Suppose I want to approximate the solution to the pde using a the same triangulation $T$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I choose polynomial trial/test spaces sufficiently rich, can I generate the exact solution to the pde?   For example, if $f$ is only piecewise constant, can I choose the first Raviart Thomas space that contains (continuous) piecewise affine functions for the trail/test space for $\vec{q}$? and an appropriate trail/test space for $u$ and generate the exact solution?  This result is true for $d=1$, but I haven't been able to prove it to myself for $d=2$ or higher.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this seems true and I haven't been able to prove it, I am looking for a counter example, a proof, or at least a sketch of a proof.  I appreciate any ideas.  I am mainly interested in proofs/ideas that allow for domains that are not convex.&lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2016-06-07T14:27:49.783" LastActivityDate="2016-06-07T14:27:49.783" Title="Does a mixed method solve this elliptic pde exactly if the source function is piecewise polynomial?" Tags="|finite-element|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24168" PostTypeId="1" CreationDate="2016-06-06T16:33:01.497" Score="1" ViewCount="80" Body="&lt;p&gt;I am considering approximating the operator of the BVP &#xA;\begin{cases}&#xA;-u''+u'=g,&amp;amp;\quad x\in [a,b]\\ u'(a)=-1, u'(b)=1,&#xA;\end{cases}&#xA;by a matrix. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I tried to use the idea of finite difference method. I first consider the operator of the homogenuous BVP &#xA;\begin{cases}&#xA;-u''+u'=\lambda u,&amp;amp;\quad x\in [a,b]\\ u'(a)=u'(b)=0.&#xA;\end{cases}&#xA;Suppose $[a,b]$ is divided into uniform subintervals and each subinterval $(x_i,x_{i+1})$is represented by the midpoint $\bar{x}_i$, mesh size $h=x_{i+1}-x_{i}$. The first order differential operator is approximated by &#xA;$$\dfrac{\partial }{\partial x}|_{\frac{1}{2}}=\dfrac{1}{2}(\dfrac{\partial }{\partial x}|_{0}+\dfrac{\partial }{\partial x}|_{1})=\dfrac{1}{2}(0+\dfrac{1 }{h}(u_{\frac{3}{2}}-u_{\frac{1}{2}})),$$&#xA;where we use the Neumann BC. &#xA;Similarly, we get &#xA;$$\dfrac{\partial }{\partial x}|_{N-\frac{1}{2}}=\dfrac{1}{2h}(-u_{N-\frac{3}{2}}+u_{N-\frac{1}{2}}),$$&#xA;for the last element. In this way, we can approximate the $\frac{\partial}{\partial x}$ of the BVP by the matrix&#xA;$$D=\begin{bmatrix}&#xA;-1  &amp;amp; 1     &amp;amp;       &amp;amp;        &amp;amp;         \\&#xA;-1  &amp;amp; 0 &amp;amp; 1 &amp;amp;  &amp;amp;          \\&#xA; &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp;      \\&#xA;         &amp;amp;  &amp;amp; -1 &amp;amp; 0 &amp;amp; 1 \\&#xA;        &amp;amp;        &amp;amp;      &amp;amp; -1      &amp;amp; 1&#xA;\end{bmatrix}.$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second order operator, &#xA;$$\dfrac{\partial^2 }{\partial x^2}|_{\frac{1}{2}}=\dfrac{1}{h}(\dfrac{\partial }{\partial x}|_{1}-\dfrac{\partial }{\partial x}|_{0})=\dfrac{1}{h^2}(u_{\frac{3}{2}}-u_{\frac{1}{2}}),$$&#xA;for the first element. In this way, we can approximate the $\frac{\partial^2}{\partial x^2}$ of the BVP by the matrix&#xA;$$DD=\begin{bmatrix}&#xA;-1  &amp;amp; 1     &amp;amp;       &amp;amp;        &amp;amp;         \\&#xA;1  &amp;amp; -2 &amp;amp; 1 &amp;amp;  &amp;amp;          \\&#xA; &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp;      \\&#xA;         &amp;amp;  &amp;amp; 1 &amp;amp; -2 &amp;amp; 1 \\&#xA;        &amp;amp;        &amp;amp;      &amp;amp; 1      &amp;amp; -1&#xA;\end{bmatrix}.$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for the inhomogenous case, we have &#xA;$$\dfrac{\partial }{\partial x}|_{\frac{1}{2}}=\dfrac{1}{2}(\dfrac{\partial }{\partial x}|_{0}+\dfrac{\partial }{\partial x}|_{1})=\dfrac{1}{2}(-1+\dfrac{1 }{h}(u_{\frac{3}{2}}-u_{\frac{1}{2}})),$$&#xA;then I don't know how to handle the constant, since it's independent of $u$. Is there anyway to include the BC into the matrix?&lt;/p&gt;&#xA;" OwnerUserId="5527" LastActivityDate="2016-06-07T18:16:30.823" Title="Approximate Neumann BVP operator by a matrix" Tags="|finite-difference|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24169" PostTypeId="2" ParentId="24168" CreationDate="2016-06-06T18:00:50.587" Score="1" Body="&lt;p&gt;If your domain is defined as $a = x_0 &amp;lt; x_1 &amp;lt; \dotsb &amp;lt; x_N = b$&#xA;you can add &lt;em&gt;ghost points&lt;/em&gt;&#xA;$$&#xA;  \begin{aligned}x_{-1} = a-\Delta x\,,\\&#xA;  x_{N+1} = b+\Delta x\,,\end{aligned}&#xA;$$&#xA;then impose the boundary conditions as&#xA;$$&#xA;\begin{aligned}&#xA;  u'(a) = \frac{u_1 - u_{-1}}{2\Delta x}\,, \\&#xA;  u'(b) = \frac{u_{N+1} - u_{N-1}}{2\Delta x}\,.&#xA;\end{aligned}&#xA;$$&#xA;Rearranging for $u_{-1}$ and $u_{N+1}$, you can express the finite difference operator by changing the matrix &lt;em&gt;and&lt;/em&gt; including a boundary correction vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example,&#xA;$$&#xA;\begin{aligned}&#xA;u''(a) &amp;amp;= \frac{u_1 -2u_0 + u_{-1}}{\Delta x^2}\\&#xA;&amp;amp;= \frac{u_1 - 2u_0}{\Delta x^2} + \frac{u_1 - 2\Delta x u'(a)}{\Delta x^2}\\&#xA;&amp;amp;= \frac{2u_1 - 2u_0}{\Delta x^2} - \frac{2u'(a)}{\Delta x}\,.&#xA;\end{aligned}$$&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="17671" LastEditDate="2016-06-06T18:07:11.763" LastActivityDate="2016-06-06T18:07:11.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24171" PostTypeId="2" ParentId="24167" CreationDate="2016-06-07T03:15:45.077" Score="3" Body="&lt;p&gt;The conjecture is false. Consider just the simplest case,&#xA;$$&#xA;  -\Delta u = 1&#xA;$$&#xA;in $\Omega=[0,1]^2$, and assume that $\partial u/\partial n=0$ all around (and fix the mean value of $u$ to make the problem well posed). For this case, the weak solution and the solution of the mixed formulation equals the strong solution of the Laplace equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do the equivalent of a Taylor expansion around the origin (one of the corners of the domain). If the solution were polynomial, this expansion would have to terminate after finitely many terms. But you will find that it doesn't. In fact, the solution has a (rather weak) singularity at each of the corners (think something like $u \propto x^{7/4}y^{7/4}$ or similar -- a function for which a sufficiently high derivative becomes singular), something you can't get if the solution were indeed polynomial.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-06-07T03:15:45.077" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24172" PostTypeId="2" ParentId="24168" CreationDate="2016-06-07T08:34:07.823" Score="1" Body="&lt;p&gt;An answer to your question &quot;how to handle the constant&quot; is that it has to be moved to the right hand side of your linear algebraic system. In another words this part of inhomogeneous Neumann boundary condition shall NOT be included to the matrix, because, as you mentioned, it is independent of $u$. It will cause the only non-zero term in the right hand side of your system of linear algebraic equations if you discretize your homogeneous BVP. &lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-06-07T18:16:30.823" LastActivityDate="2016-06-07T18:16:30.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24174" PostTypeId="2" ParentId="24149" CreationDate="2016-06-07T19:37:41.097" Score="1" Body="&lt;p&gt;(1) It appears that $(i_1,i_2),(j_1,j_2)$ are coordinate pairs, such that $(j_1,j_2)$ are the integer coordinates on the source grid and $(i_1,i_2)$ are the integer coordinates on the observer grid. The connection formulas for $i$ and $j$ involving these quantities simply flatten the grid structure and give you unique indices for each point in the grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) I suggest you interchange $i,j$ in your expression for $K_{ij}$ and see for yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) No, recall from (1) that $i,j$ depend on $i_1,i_2,j_1,j_2$.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2016-06-07T19:37:41.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24175" PostTypeId="1" CreationDate="2016-06-08T07:12:18.937" Score="3" ViewCount="215" Body="&lt;p&gt;I am an undergraduate, and enthusiastic about &lt;em&gt;HPC&lt;/em&gt;. I am currently familiar with the tools &lt;strong&gt;OpenMP, MPI, CUDA, OpenCL, thrust libraries etc&lt;/strong&gt;. But I want to know the core functioning of these tools, I want to know how exactly these environments work to get the most of the hardware. It would be great if someone would suggest me some reading material or even some MOOC videos. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading the material it would be great if I could get a very good idea how the library is written. And maybe someday even I could write one if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="20623" LastEditorUserId="20623" LastEditDate="2016-06-08T07:17:49.543" LastActivityDate="2019-01-13T15:25:03.487" Title="HPC reading material" Tags="|parallel-computing|hpc|gpu|cuda|opencl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24176" PostTypeId="2" ParentId="24175" CreationDate="2016-06-08T12:35:27.283" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;https://www.openmprtl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel OpenMP Run Time library&lt;/a&gt; has been open sourced, so you might have look at a few functions in there. There are many open source MPI libraries (MPICH2, OpenMPI, MVAPICH2, etc.), and the MPI interface is standardized, so reading the standards document can be enlightening. OpenMP is also standardized, so you might look to that as well. CUDA is basically a proprietary NVIDIA language, so don't expect to get too much information about its inner workings. Thrust is also open, so you can poke around at it on GitHub if you like, though you might spend some time looking at the C++ STL and Boost before you do that.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="24110" LastEditDate="2019-01-13T10:14:51.523" LastActivityDate="2019-01-13T10:14:51.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24179" PostTypeId="1" CreationDate="2016-06-08T14:50:14.180" Score="3" ViewCount="920" Body="&lt;p&gt;Suppose I have a complicated structure given in 2 or 3 dimensions as a polygon mesh. For example, this could represent a &quot;cave&quot; or an assembly of irreguar shaped particles or a tree, whatever. Now I'd like to &quot;smoothen&quot; this structure to end up with a &lt;strong&gt;finer &lt;em&gt;and&lt;/em&gt; smoother&lt;/strong&gt; polygon mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's &lt;strong&gt;not&lt;/strong&gt; just about refining the mesh and it's also &lt;strong&gt;not&lt;/strong&gt; just about some sort of rastering and then anti-aliasing filtering, it's really about getting from a coarse edgy polygonization/mesh to a fine, more smoothly curved polygonization/mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I'm looking for some ideas on how to do it: Books, papers, search terms. (I do not even know the name of the problem...)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(It will be living in the realm of nano particles if that plays a role.)&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-02-27T11:40:44.343" Title="How to &quot;smoothen&quot; (not just refine) a 2D/3D polygonal mesh" Tags="|computational-geometry|mesh|smoothing|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24180" PostTypeId="1" CreationDate="2016-06-08T15:25:18.013" Score="1" ViewCount="242" Body="&lt;p&gt;I am interested in implementing pressure inlet boundary conditions for the 2D compressible Euler equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My equation of state is an ideal gas ($p=\rho R T$) which is thermally prefect but &lt;em&gt;not&lt;/em&gt; calorically perfect, i.e. $c_p\equiv c_p(T)$ is a non-constant functions of $T$ alone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary condition specifies total pressure $p_t$, total density $\rho_t$, and angle of attack $\alpha$ on the boundary. I would like to know how to compute a consistent state at the boundary assuming an isentropic and adiabatic process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see from the &lt;a href=&quot;http://www.afs.enea.it/project/neptunius/docs/fluent/html/ug/node239.htm#sec-pinlet-calc&quot; rel=&quot;nofollow&quot;&gt;ANSYS Fluent manual&lt;/a&gt;, that in case of a perfect gas one does the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Static pressure on the boundary $p_b$ is set equal to the static pressure in the adjacent cell $p_i$&lt;/li&gt;&#xA;&lt;li&gt;Mach number on the boundary is computed from the isentropic relation&#xA;$$ p_t = p_b\left(1+ \frac{\gamma-1}{2}M^2\right)^{\gamma/(\gamma-1)}$$&lt;/li&gt;&#xA;&lt;li&gt;Static temperature on the boundary $T_b$ is computed from the isentropic relation&#xA;$$ T_t = T_b\left(1+\frac{\gamma-1}{2}M^2\right)$$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any references on implementing this kind of boundary condition in case of a ideal gas which is solely thermally perfect is highly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first approach was&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;set the mach number on the boundary equal to that of the adjacent cell&lt;/li&gt;&#xA;&lt;li&gt;solve the adiabatic process equation&#xA;$$ h_t = h(T) + \frac{1}{2}M^2 c(T)^2$$&#xA;for $T$ to determine the temperature on the boundary&lt;/li&gt;&#xA;&lt;li&gt;Solve $s(T,p) = s(T_t,p_t)$ for $p$ to determine the pressure on the boundary&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, this approach leads to a flow which differs significantly from what I see in ANSYS. In fact, the computed mach numbers are much lower.&lt;/p&gt;&#xA;" OwnerUserId="20631" LastEditorUserId="20688" LastEditDate="2019-06-22T18:56:44.007" LastActivityDate="2019-06-22T18:56:44.007" Title="Implementing pressure inlet boundary condition" Tags="|fluid-dynamics|ansys-fluent|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24182" PostTypeId="2" ParentId="24179" CreationDate="2016-06-08T21:36:08.367" Score="4" Body="&lt;p&gt;For just mesh smoothing, you can start by looking at &lt;a href=&quot;https://en.wikipedia.org/wiki/Laplacian_smoothing&quot; rel=&quot;noreferrer&quot;&gt;Laplacian smoothing&lt;/a&gt; and some of the references therein. The idea is to update the position of every vertex in the interior of the mesh by replacing it with the average of its neighbors. There are loads of more sophisticated ways of doing this by using different operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're doing both surface mesh refinement and smoothing at the same time, you can do better than just adding more points to every triangle by computing estimates of local mesh curvature; &lt;a href=&quot;https://computergraphics.stackexchange.com/questions/1718/what-is-the-simplest-way-to-compute-principal-curvature-for-a-mesh-triangle&quot;&gt;this answer&lt;/a&gt; might give you some inspiration. More generally, you can construct a local spline approximation of the surface and use this to decide where to add new points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://books.google.com/books?isbn=1568814267&quot; rel=&quot;noreferrer&quot;&gt;Polygon Mesh Processing&lt;/a&gt; by Botsch, while a little thin on some of the details, covers the general ideas pretty well and will at least give you the terms needed to describe your problem.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-06-08T21:36:08.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24185" PostTypeId="2" ParentId="24179" CreationDate="2016-06-09T01:14:11.077" Score="5" Body="&lt;p&gt;As mentioned in the answer by @DanielShapero, you can follow an approach based on local approximations of the curvature for your nodes. In the post he suggest, there is an article by Desbrun. I would suggest to check another article by him: &lt;a href=&quot;https://hal.archives-ouvertes.fr/file/index/docid/71778/filename/RR-4808.pdf&quot; rel=&quot;noreferrer&quot;&gt;Anisotropic Polygonal Remeshing&lt;/a&gt; [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option that comes to my mind is to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Catmull%E2%80%93Clark_subdivision_surface&quot; rel=&quot;noreferrer&quot;&gt;Catmull-Clark&lt;/a&gt; subdivision algorithm [2]. Where you start with a mesh and add new points in the middle of the original face (split step) and then you move the original points using a weighted average of their closest (newly created) points (average step). Khan-Academy has an interactive version of it in this link: &lt;a href=&quot;https://www.khanacademy.org/partner-content/pixar/modeling-character/modeling-subdivision/p/interactive-subdivision-in-3d&quot; rel=&quot;noreferrer&quot;&gt;https://www.khanacademy.org/partner-content/pixar/modeling-character/modeling-subdivision/p/interactive-subdivision-in-3d&lt;/a&gt;, so you can check if that helps you.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;[1]&lt;/strong&gt; Pierre   Alliez,   David   Cohen-Steiner,   Olivier   Devillers,   Bruno   Levy,   Mathieu   Desbrun. Anisotropic  Polygonal  Remeshing. [Research  Report]  RR-4808,   INRIA.  2003. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;[2]&lt;/strong&gt; &lt;a href=&quot;http://people.eecs.berkeley.edu/~sequin/CS284/PAPERS/CatmullClark_SDSurf.pdf&quot; rel=&quot;noreferrer&quot;&gt;Recursively generated B-spline surfaces on arbitrary topological meshes&lt;/a&gt;. Computer-Aided Design 10 (6): 350. doi:10.1016/0010-4485(78)90110-0.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2016-06-09T01:14:11.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24186" PostTypeId="2" ParentId="24179" CreationDate="2016-06-09T12:07:18.987" Score="6" Body="&lt;p&gt;To complement the two answers from Daniel Shapero and Nicoguaro:&#xA;Basically, there are two ways of smoothing a mesh, &lt;strong&gt;subdivision&lt;/strong&gt; (generate new vertices) and &lt;strong&gt;smoothing&lt;/strong&gt; (move the points in such a way that the obtained shape is smoother). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Subdivision&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To grasp the intuition, imagine you want to &quot;smoothen&quot; a 2D square. The 2D square is not smooth because it has corners, so let's cut the corners out, then you obtain an octagon (a &quot;stop sign&quot;), it has more corners, but they are wider (smoother). If you do that again, you obtain a polygon with 16 sides, that is a reasonably good approximation of a circle (do that an additional couple of times if it's not smooth enough). Now there are some theoretic results, telling you how to cut the corners in such a way that if you do that an infinite number of times, you will get a smooth (tangent continous) shape. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;When can I use subdivision ?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use subdivision, you need a mesh that is rather coarse, with well shaped elements. You need also to know that there are subdivision methods for meshes composed of quads (e.g., Catmull-Clark, Doo-Sabin) and subdivision methods for meshes composed of triangles (e.g. Butterfly, Loop, sqrt(3)). You will obtain best results with squares, but you need a very nice input mesh (e.g., a mesh designed by a computer graphics artist). If your mesh does not have a nice quad-like structure, you may try to use our &quot;anisotropic polygonal remeshing&quot; method cited in the other answers (disclosure: I'm a co-author of this article).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further reading on subdivision: see Zorin et.al's SIGGRAPH course notes [1] &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Smoothing&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other idea to smooth a mesh is to move the vertices in such a way that the obtained mesh is smoother. Here is some intuition about the idea: a flat mesh, where each vertex is exactly at the barycenter of its neighbors, is perfectly smooth. So one possible idea is to iteratively move all the vertices towards the center of their neighbors in interleaved loops:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(1) compute the barycenters of the neigbors&#xA;(2) move the points to the barycenter&#xA;(3) goto 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you do that, it works quite well, but you will observe that the mesh shrinks more and more, so you can do that with a correction phase, making the volume of the mesh constant (clearly, you need a closed mesh to do that):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(1) compute the barycenters of the neigbors&#xA;(2) move the points to the barycenter&#xA;(3) compute the volume of the mesh&#xA;(4) scale all point coordinates by the cubic root of the ratio between the new volume and the initial volume&#xA;(3) goto 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, instead of correcting the volume a-posteriori, why not directly compute a volume-preserving smoothing ? This is (roughly) what Desbrun's &quot;curvature flow&quot; method cited in Shapero's answer does, by exploiting a relation between the curvature at a point and the volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine that you have some points of the mesh that need to keep their original locations, like &quot;control nodes&quot; that will be &quot;locked&quot; while all the other points are free to move. A possibility to do that, still saying that making each point as near as possible to the barycenter of its neighbors, is using least-squares: you minimize the squared distance between each point and the barycenter of its neighbors subject to the constrained points. This is called Discrete Smooth Interpolation [2],[3] (method invented by Mallet, my Ph.D. thesis advisor). It also corresponds to Laplacian smoothing (see the PMP book [4] that we wrote with colleagues, cited also in the other answers, in particular chapter 3) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;So which method should I use ?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mainly depends where the mesh comes from. Subdivision is very cheap, but only works for nice designed meshes. Curvature flow is reasonably simple, and will quite easily remove the high frequency noise from a scanned mesh. Least-squares smoothing / Laplacian smoothing needs to solve a least-squares problem (boils down to solving a linear system), which is more involved. This is the price to pay if you need to introduce constraints / control points / handles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of completeness, I also mention spectral methods, that do a Fourier-like decomposition of the shape, see my articles and SIGGRAPH course notes on the topic [5,6], but it is extremely costly (solve eigenproblem), it's overkill in most situations (but the math. is interesting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://mrl.nyu.edu/publications/subdiv-course2000/&quot; rel=&quot;noreferrer&quot;&gt;http://mrl.nyu.edu/publications/subdiv-course2000/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Discrete Smooth Interpolation, Mallet, ACM Transactions on Graphics, 1989&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://alice.loria.fr/index.php/publications.html?redirect=0&amp;amp;Paper=smoothing@1999&quot; rel=&quot;noreferrer&quot;&gt;http://alice.loria.fr/index.php/publications.html?redirect=0&amp;amp;Paper=smoothing@1999&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] Polygon Mesh Processing, CRC press, Botsch, Kobbelt, Alliez, Levy, &lt;a href=&quot;http://www.pmp-book.org/&quot; rel=&quot;noreferrer&quot;&gt;http://www.pmp-book.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] Laplacian Eigenfunctions, towards an algorithm that understands geometry, Levy, SMI 2006 invited talk&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] Manifold Harmonics, Vallet and Levy, Eurographics / Computer Graphics Forum, 2008&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[7] Spectral Geometry Processing, Levy and Zhang, SIGGRAPH and SIGGRAPH ASIA course (first given in 2010)&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-06-09T12:07:18.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24187" PostTypeId="1" AcceptedAnswerId="29035" CreationDate="2016-06-09T18:03:41.063" Score="6" ViewCount="248" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Recall that a unit lower triangular matrix $L\in\mathbb{R}^{n\times n}$ is a lower triangular matrix with diagonal elements $e_i^{T}L e_i = \lambda_{ii} = 1$. An elementary unit lower triangular column form matrix, $L_i$, is an elementary unit lower triangular matrix in which all of the nonzero subdiagonal elements are contained in a single column. For example, for $n = 4$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;$$L_1 = \begin{pmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;\lambda_{21} &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;\lambda_{31} &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\&#xA;\lambda_{41} &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;\end{pmatrix} \ \ \ L_2 = \begin{pmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; \lambda_{32} &amp;amp; 1 &amp;amp; 0\\&#xA;0 &amp;amp; \lambda_{42} &amp;amp; 0 &amp;amp; 1\\&#xA;\end{pmatrix} \ \ \ L_3 = \begin{pmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; \lambda_{43} &amp;amp; 1\\&#xA;\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our first task was to show that any unit lower triangular column form matrix, $L_i\in\mathbb{R}^{n\times n}$, can be written as the identity matrix plus an outer product of two vectors, i.e., $L_i = I + v_i w_i^{T}$ where $v_i\in\mathbb{R}^{n\times n}$ and $w_i\in \mathbb{R}^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;solution - Since only the $i$-th column of $L_i$ differs from the identity matrix the outer product $v_i w_i^{T}$ must have the same structure. This implies that $w_i = e_i$ and it follows that $v_i$ is added to the $i$-th column of $I$ to define $L_i e_i$. Since only elements below the main diagonal element are different from $I$, it follows that $v_i$ has a &quot;lower&quot; structure to its potentially nonzero elements. This is often indicated in the notation by using $l_i$ instead of the generic $v_i$. The conditions on the vector are $$l_i^{T}e_j = \begin{cases}0 \ &amp;amp; 1\leq j \leq i\\&#xA;\lambda_{ji} \ &amp;amp; i+1\leq j \leq n&#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the expression is $L_i = I + l_i e_i^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question I have is the following: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;i.) Suppose $L_i\in\mathbb{R}^{n\times n}$ and $L_j\in\mathbb{R}^{n\times n}$ are elementary unit lower triangular column form matrices with $1\leq i &amp;lt; j \leq n-1$. Consider the matrix product $B = L_i L_j$. Determine an efficient algorithm to compute the product and its computational and storage complexity.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;ii.) Suppose $L_i\in\mathbb{R}^{n\times n}$ and $L_j\in\mathbb{R}^{n\times n}$ are elementary unit lower triangular column form matrices with $1\leq j \leq i \leq n-1$. Consider the matrix product $B = L_i L_j$. Determine an efficient algorithm to compute the product and its computational and storage complexity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The only difference from (i) and (ii) are the inequalities as you can see. I have been told that (i) requires no computation but I don't understand why. I am quite confused about these types of problems. Any suggestions are greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="20417" LastEditDate="2016-10-01T15:39:15.077" LastActivityDate="2018-03-13T22:00:09.623" Title="Efficient algorithm for a matrix product" Tags="|linear-algebra|algorithms|matrix|complexity|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24192" PostTypeId="2" ParentId="24087" CreationDate="2016-06-10T02:18:08.020" Score="2" Body="&lt;p&gt;Doing a fast search in GitHub you can find&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A &lt;a href=&quot;https://github.com/aoyuan/GSA&quot; rel=&quot;nofollow&quot;&gt;MATLAB implementation&lt;/a&gt;;&lt;/li&gt;&#xA;&lt;li&gt;A &lt;a href=&quot;https://github.com/PaintingInAir/Optimal&quot; rel=&quot;nofollow&quot;&gt;Python implementation&lt;/a&gt;; and&lt;/li&gt;&#xA;&lt;li&gt;A &lt;a href=&quot;https://github.com/edgnik/gsa-cpu-gpu&quot; rel=&quot;nofollow&quot;&gt;C implementation&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2016-06-10T02:18:08.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24193" PostTypeId="1" CreationDate="2016-06-10T07:09:08.750" Score="7" ViewCount="3537" Body="&lt;p&gt;Are there benchmarks for how many multiplications of various integer types compared to floating point types can be achieved per second on modern CPUs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to get some hint if it would be worth to make a c implementation of an integer arithmetics algorithm. I.e. would there be speed gains to be had on modern CPUs compared to iterative floating point arithmetic methods. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For those curious about the algorithm: &lt;a href=&quot;https://math.stackexchange.com/a/1818752/213607&quot;&gt;This answer&lt;/a&gt; inspired me to write &lt;a href=&quot;https://math.stackexchange.com/questions/1819741/matrices-over-integer-fields-to-solve-complex-polynomials?lq=1&quot;&gt;a question&lt;/a&gt; which lead to the idea of doing matrix monomials over integer vectors to approximate roots of polynomials over more complicated fields. For example finding integer roots of &lt;a href=&quot;https://math.stackexchange.com/questions/1819488/quaternion-interpolation-in-3d/1819573#1819573&quot;&gt;quaternion&lt;/a&gt;s representing 3 dimensional rotations. &quot;Which operation if I apply it N times becomes this rotation?&quot; The simplest solution is probably a rotation.&lt;/p&gt;&#xA;" OwnerUserId="18579" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-06-11T07:52:30.297" Title="Integer vs float multiplication performance, modern CPUs" Tags="|reference-request|iterative-method|performance|benchmarking|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24194" PostTypeId="2" ParentId="24087" CreationDate="2016-06-10T12:27:34.517" Score="1" Body="&lt;p&gt;The algorithm described by Rashedi et al in the paper you mentioned is not strictly a gravitational search algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In their equation (7), the magnitude of the force of attraction is inversely proportional to the distance between agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rashedi et al claim that they get better results with equation (7) instead of the (classically) correct &lt;b&gt;inverse square law&lt;/b&gt;. Therefore, the descriptor &quot;gravitational&quot; is misleading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an &lt;b&gt;inverse square law&lt;/b&gt;, the denominator should be $R^3$, not $R$, in equation (7).&lt;/p&gt;&#xA;" OwnerUserId="16571" LastEditorUserId="16571" LastEditDate="2016-06-10T12:41:28.860" LastActivityDate="2016-06-10T12:41:28.860" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24195" PostTypeId="1" CreationDate="2016-06-10T15:54:30.763" Score="3" ViewCount="514" Body="&lt;p&gt;I need to implement a 3D version of the Corner Transport Upwind (CTU) finite volume method (in python); and so I've been reading Leveque, &quot;Finite Volume Methods for Hyperbolic Problems&quot; which I think is quite good. Shown below is the critical section regarding how to generalize CTU for linear advection to arbitrary velocity fields (non-constant in space).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand something about the flux updates $G_{i, j-1/2}$ and $G_{i, j+1/2}$. Suppose $v_{i, j-1/2}$ is negative and $v_{i, j+1/2}$ is positive but they have the same magnitude. Then there is transverse flux across both $G_{i, j-1/2}$ and $G_{i, j+1/2}$, and hence those flux updates will be non-zero. Further, $G_{i, j-1/2}$ will be positive (the sign flips due to the sign of the velocity right?) and $G_{i, j+1/2}$ will be negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These cells are then used in the flux differencing formula for cell $Q_{ij}$ as: $G_{i, j+1/2} - G_{i, j-1/2}$. Because the two terms had opposite sign before, the minus sign causes them to add together, rather than cancel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That doesn't seem right to me. With respect to cell $Q_{ij}$, I thought these transverse terms were meant to account for parts of the wave that don't flux into the cell, but in this case the donor-cell wave traveling from $Q_{i-1, j}$ into $Q_{ij}$ is stretching on the top and bottom equally, so shouldn't the flux into $Q_{ij}$ be the same as if the transverse fluxes weren't there at all, as if $v_{i, j-1/2}$ and $v_{i, j+1/2}$ were both zero? That is, the donor cell flux into $Q_{ij}$ sweeps out the same area as if $v_{i, j-1/2}$ and $v_{i, j+1/2}$ were both zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xr5nt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xr5nt.png&quot; alt=&quot;Page 454-455 from Leveque FVM for Hyperbolic Problems.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20634" LastEditorUserId="20634" LastEditDate="2016-06-10T16:12:17.367" LastActivityDate="2016-06-10T16:12:17.367" Title="Corner Transport Upwind for Linear Advection in Arbitrary Velocity Field" Tags="|finite-difference|numerical-analysis|finite-volume|advection|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24196" PostTypeId="1" CreationDate="2016-06-10T16:30:14.177" Score="1" ViewCount="97" Body="&lt;p&gt;I am trying to solve the drift-diffusion model with generation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial N_e}{\partial t} = \alpha(x) \left| \Gamma_e (x,t) \right| - \frac{\partial \Gamma_e}{\partial x} (x,t)$$&#xA;$$\frac{\partial N_i}{\partial t} = \alpha(x) \left| \Gamma_e (x,t) \right| - \frac{\partial \Gamma_i}{\partial x} (x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Gamma_e (x,t) = -\mu_e(x) E(x) N_e(x,t) - D_e(x) \frac{\partial N_e}{\partial x} (x,t)$$&#xA;$$\Gamma_i (x,t) =  \mu_i(x) E(x) N_i(x,t) - D_i(x) \frac{\partial N_i}{\partial x} (x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was looking at &lt;a href=&quot;https://en.wikipedia.org/wiki/Upwind_scheme&quot; rel=&quot;nofollow&quot;&gt;upwinding schemes&lt;/a&gt; but am not sure if I can use this method when there is some diffusion? If so how would I handle the $0^{\mathrm{th}}$ and $2^{\mathrm{nd}}$ order terms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to use finite differencing.&lt;/p&gt;&#xA;" OwnerUserId="17394" LastActivityDate="2016-06-10T16:30:14.177" Title="PDE solver for drift-diffusion with generation" Tags="|pde|numerical-modelling|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24197" PostTypeId="2" ParentId="24193" CreationDate="2016-06-10T20:57:01.173" Score="4" Body="&lt;p&gt;In general, the answers is no.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modern CPU's excel at problems which have high arithmetic intensity and are implemented using floating point arithmetic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few figures will help explain the discrepancy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the Haswell chip the MULPS instructions performs 8 single precision floating point multiplications. The latency is 5 CPU cycles and the reciprocal throughput is 0.5. This means that your have to wait 5 cycles for first instruction to complete, but if you have a stream of independent operands, then you can eventually retire 2 instruction pr. cycle. This yields a peak rate of 16 single precision multiplications pr. cycle. In comparison the IMUL instruction multiplies a single pair of 32 bit integers. The latency is 3 cycles and the reciprocal throughput is 1. If you have a stream of independent operands, then the peak rate is 1 multiplication pr. cycle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Achieving the peak flop rate it only possible for a few very specialized types of computations, such as doing inner products on vectors which fit in the L1 cache on a single core. Matrix-matrix multiply, the kernel for all blocked implementations of Gaussian elimination frequently achieves 70-80% of the theoretical peak on the Top500 computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sparse arithmetic typically has a very low arithmetic intensity and is incompatible with modern computer architecture. Computers on the Top500 list execute the sparse benchmark (Conjugate Gradient) at about 2% of the peak flop rate. The best computer on the list does does less than 20%. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can make some sparse algorithms run at a high fraction of the peak flop rate, but it requires extremely aggressive optimizations which are tailored to the specific task at hand. Attempting to boost the computational speed with integer computations will just lower the arithmetic intensity even further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to increase the accuracy of, say, double precision floating point calculations by clever algorithms which return the result, as if it had been computed using a smaller unit round off error, and then rounded to double precision. If extreme accuracy is required, then this approach is far more likely to be successful.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2016-06-10T20:57:01.173" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24198" PostTypeId="1" CreationDate="2016-06-11T00:13:08.367" Score="1" ViewCount="141" Body="&lt;p&gt;I'm trying to solve the optimization problem below with MatLab, but I'm unsure of how to modify the constraints in the quadprog function (or how to add the constraints with the Portfolio object). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the formulation below, omega is the weight of the asset in the portfolio (which we are trying to find).&#xA;The capital sigma is the covariance matrix of asset returns.&#xA;mu is the expected return.&#xA;delta is the maximum allowed deviation of the expected return of an asset i, from an estimated value mu(i).&#xA;gamma is a bound for the amount of shorting allowed.&#xA;l and h are lower and upper limits for the weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is trying to find the optimal portfolio with weights between l and h y considering only the worst outcome for the amount of short positions that exceed gamma.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xeAAf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xeAAf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20657" LastActivityDate="2016-06-11T00:13:08.367" Title="Conservative Short Sales Portfolio Optimization with MatLab code help" Tags="|optimization|matlab|constrained-optimization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24199" PostTypeId="1" AcceptedAnswerId="25912" CreationDate="2016-06-11T18:27:07.180" Score="1" ViewCount="757" Body="&lt;p&gt;I'm interested in using a SWT to perform Multi Resolution Analysis over 3D data arrays. However I could not find any software package that implements it. The Matlab Toolbox of wavelets (the most complete I think) only has 3D DWT and 2D SWT, so my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There exists a working implementation of 3D SWT out there?&lt;/li&gt;&#xA;&lt;li&gt;If not, how can I implement it? (some references maybe)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7230" LastEditorUserId="9667" LastEditDate="2016-06-12T16:58:37.810" LastActivityDate="2017-01-10T20:13:20.923" Title="3D Stationary Wavelet Transform implementations" Tags="|wavelet|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24200" PostTypeId="1" CreationDate="2016-06-11T22:56:32.773" Score="2" ViewCount="1375" Body="&lt;p&gt;I want to obtain the &lt;a href=&quot;https://en.wikipedia.org/wiki/Convolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolution&lt;/a&gt; of two discretized &lt;em&gt;real&lt;/em&gt; functions $f$ and $g$,&#xA;$$&#xA;c(t) = \int_{-\infty}^{+\infty} \mathrm{d}{x} \, f(x) \, g(t-x)&#xA;\tag{1}&#xA;$$&#xA;via &lt;a href=&quot;https://en.wikipedia.org/wiki/Discrete_Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;discrete Fourier transform&lt;/a&gt; (DFT).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a concrete example, let $x \in [-6, 6]$ be the $x$-axis on which the functions are defined, with&#xA;$$&#xA;f(x) = \Theta(x) \, e^{-x} ~, \\&#xA;g(x) = \Pi(x/2) ~,&#xA;$$&#xA;where $\Theta(x)$ is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Heaviside_step_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heaviside step-function&lt;/a&gt; and $\Pi(x)$ is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rectangular_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;rectangular function&lt;/a&gt;.&#xA;Then the exact analytical expression for the convolution will be (see also figure below)&#xA;$$&#xA;C(t) = &#xA;\begin{cases} &#xA;   0 , &amp;amp; t \leq -1 \\&#xA;   1 - e^{t + 1} , &amp;amp; -1 &amp;lt; t \leq 1 \\&#xA;   e^{-t} (e - e^{-1}) , &amp;amp; t \geq 1 &#xA;\end{cases} &#xA;$$&#xA;&lt;img src=&quot;https://i.stack.imgur.com/XDefd.png&quot; width=&quot;400&quot; alt=&quot;exact convolution of f and g&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $x$-axis is discretized into $N = 2^K$ points. DFT (and inverse DFT) can be used to obtain the &lt;a href=&quot;https://en.wikipedia.org/wiki/Circular_convolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;periodic&lt;/em&gt; (&lt;em&gt;circular&lt;/em&gt;) convolution&lt;/a&gt; $c_P$ of $f$ and $g$,&#xA;$$&#xA;c_P[n] = DFT^{-1} ( DFT(f) \cdot DFT(g) ) ~.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to what I have seen in text-books (e.g. section 2.4 of &lt;a href=&quot;http://www.worldcat.org/oclc/822975038&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;), the &lt;em&gt;aperiodic&lt;/em&gt; (‘infinite’) convolution (corresponding to infinite ‘signals’ for $f$ and $g$) can be obtained by zero-padding $f$ and $g$ arrays at the &lt;em&gt;end&lt;/em&gt; by $N-1$ zeros.&#xA;Then the discretized convolution corresponding to Eq. 1 should be obtained as&#xA;$$&#xA;c[n] = DFT^{-1} ( DFT(f_{padded}) \cdot DFT(g_{padded}) )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to do this in Python using &lt;code&gt;scipy.fftpack&lt;/code&gt; (see the code &lt;a href=&quot;https://ideone.com/RG81pG&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). But the results of this discrete convolution does &lt;em&gt;not&lt;/em&gt; match with the analytic one (see figure); namely, the scale and the position of the discrete convolution is not correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/cg45D.png&quot; width=&quot;400&quot; alt=&quot;exact vs. discrete convolution of f and g&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the correct way to perform the discrete (infinite) convolution of $f$ and $g$?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;*Note: The correct result is actually obtained by padding $f$ by $N-1$ zeros at the &lt;em&gt;end&lt;/em&gt;, re-arranging (center-padding) $g$ as $\hat{g} := (g_{+} ~|~ N-1 ~ \text{zeros} ~|~ g_{-})$, where $g_{\pm}$ denotes the right/left half of $g$, and applying DFT as&#xA;$$&#xA;c[n] = DFT^{-1} ( DFT(f_{padded}) \cdot DFT(\hat{g}) ) ~;&#xA;$$&#xA;finally one should divide elements of $c$ by $\sum_{i} g_i$.&#xA;But I cannot understand the reason behind this.&lt;/p&gt;&#xA;" OwnerUserId="15235" LastEditorUserId="15235" LastEditDate="2016-06-12T11:28:47.560" LastActivityDate="2016-08-02T06:31:36.533" Title="Convolution of two real functions using discrete Fourier transform (FFT): zero-padding and normalization" Tags="|fourier-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24201" PostTypeId="1" CreationDate="2016-06-12T04:46:44.350" Score="1" ViewCount="175" Body="&lt;p&gt;I've been trying to find an applicable PDE solver for cases such as this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8SG0O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8SG0O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although when dealing with stiff equations in the complex domain, applying existing packages has been problematic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked into FiPy but it doesn't seem to be able to handle complex numbers well. A possible method was to split each function and parameters into 2 vectors denoting its real and imaginary part, although its applicability in dealing with the nonlinear PDE shown appears to complicate matters more. I'm also now looking through the documentation for FEniCS and seeing if it can be helpful (although once again complex variables seem to be problematic). I was also looking into making my own solver using basic methods such as Runge-Kutta but the solutions seem to vary wildly when I change parameters/domains. I have been looking into rezoning/adaptive stretching techniques but I will only be implementing that once I've gotten a basic PDE solver that can handle simple cases first. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you happen to have any recommendations for any particular packages or methods? I am currently using Python but am open to any recommendations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example of boundary/initial conditions would be: &#xA;$$N(z,0) = \frac{1}{2} \cos(10^{-5}) \\ P(z,0) = 10^9 \sin(10^{-5}) \\ E(z,0) = 10^{10} \\ N(0,t) = \frac{1}{2} e^{\frac{-t}{10^6}} \\ P(0,t) = 10^9 e^{\frac{-t}{10^6}} \\ E(0,t) = 10^{10} e^{\frac{-t}{10^6}} \\ \\$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(My apologies for the quite high values. This is after a rescaling to make the values closer in range, but they still are quite large. I will be working on modifying this so any suggestions with your own scalings are more than welcome.)&lt;/p&gt;&#xA;" OwnerUserId="20407" LastEditorUserId="20407" LastEditDate="2016-06-13T16:25:33.137" LastActivityDate="2016-06-13T16:25:33.137" Title="Applicable solvers for nonlinear coupled PDEs" Tags="|pde|stiffness|complex-analysis|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24202" PostTypeId="2" ParentId="24201" CreationDate="2016-06-12T16:12:08.353" Score="4" Body="&lt;p&gt;The approach of using two equations for the real and imaginary parts of an equation is often used. It may lead to more cumbersome formulas, but it is definitely possible and common. An example where this is done in deal.II (a library that I maintain) is here: &lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/step_29.html&quot; rel=&quot;noreferrer&quot;&gt;https://www.dealii.org/developer/doxygen/deal.II/step_29.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-06-12T16:12:08.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24203" PostTypeId="1" CreationDate="2016-06-12T17:11:36.810" Score="5" ViewCount="693" Body="&lt;p&gt;I am currently working on a physics problem that turns into a non-linear boundary value problem. I need an efficient numerical solver that I could run on my laptop with i5 dual core CPU. I am discretizing my 15 equations on an N x N x N cubic grid using fourth order finite difference derivatives. I am interested in using Newton-Krylov methods since the Jacobian that is generated is sparse. My ultimate goal is to solve this system on a 128 x 128 x 128 cube, which means I will be solving almost 15 million equations for the same number of variables. I am already in the process of learning SUNDIALS packages but I recently found the PETSc library, which many people seem to be praising for quality of the code and efficiency. PETSc also seems to have a large number of preconditioners needed for Krylov methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I would like to ask how do both packages compare for serial computations since I would like to run the solver on my laptop (at least for testing purposes on a bit coarser grid). I might eventually move to my university's HPC cluster to solve the system on a finer grid. Also, my intention is to use these packages on a long term basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to high performance computing so kindly excuse my ignorance. I would really appreciate any suggestions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S- The equations I am working on come from Non-Abelian gauge theories and are in Tensor form. Individual components (15) equations are long and complicated...so I actually compute these equations using mathematica and then discretize them. These equations are similar to Maxwell's equations but more complicated and non-linear in the fields. I do not know how much this would help here but the equations are as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(D_\mu F^{\mu \nu})^a = g \epsilon_{abc} (D_\nu \phi)^b \phi^c$, and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D_\mu (D^\mu \phi)^a = - \lambda (\phi^b \phi^b -v^2) \phi^a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here Einstein's summation convention is implied. The indices $\mu$ and $\nu$ run from 0 to 3. And index $a$ runs from 1 to 3. $D$ represents gauge covariant derivatives and $\phi$ and $F$ represent Higgs and gauge fields strength. You can check this article for more details....page 9. &lt;a href=&quot;http://www-thphys.physics.ox.ac.uk/people/MaximeGabella/higgs.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www-thphys.physics.ox.ac.uk/people/MaximeGabella/higgs.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20675" LastEditorUserId="20675" LastEditDate="2016-06-12T19:58:41.930" LastActivityDate="2016-06-15T08:19:01.407" Title="PetSc vs Sundials for serial numerical computations?" Tags="|finite-difference|parallel-computing|petsc|krylov-method|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24205" PostTypeId="1" AcceptedAnswerId="24214" CreationDate="2016-06-13T05:51:35.283" Score="1" ViewCount="95" Body="&lt;p&gt;I have quite simple problem of FEM solution of 1D differential non-linear equation. Altough the problem itself is simple, the numerical solution of the arising system of non-linear algebraic equations takes hours. The system is tridiagonal type, typical for 1D FEM problems.  At the moment I am using Picard or Newton methods. And those are the only methods I have experience with. However they do not seem to give good results.&#xA;Are there other methods, modifications to those or solvers anyone could recommend? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: The problem is solution of 1D diffusve wave equation. The details can be found &lt;a href=&quot;http://www.ibwpan.gda.pl/docs/ahem/ahem61str111.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. &#xA;The system is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textbf{M}\cdot{\textbf{x}}={\textbf{r}}$$&#xA;with&#xA;$$M_{i,i-1}=(1-\omega)\cdot \frac{\Delta x}{2}+\theta\cdot \Delta t \cdot \frac{1}{2}\left( \frac{1}{n}\frac{(x_{j-1}-z_{j-1})^{5/3}}{ \sqrt{\frac{x_{j}-x_{j-1}}{\Delta x}}}+ \frac{1}{n}\frac{(x_{j}-z_{j})^{5/3}}{ \sqrt{\frac{x_{j}-x_{j-1}}{\Delta x}}}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M_{i,i}=\omega\cdot \Delta x+\theta\cdot \Delta t \cdot \frac{1}{\Delta x}\left [ \frac{1}{2}\left( \frac{1}{n}\frac{(x_{j-1}-z_{j-1})^{5/3}}{ \sqrt{\frac{x_{j}-x_{j-1}}{\Delta x}}}+ \frac{1}{n}\frac{{}(x_{j}-z_{j})^{5/3}}{ \sqrt{\frac{x_{j}-x_{j-1}}{\Delta x}}}\right)&#xA;+\frac{1}{2}\left( \frac{1}{n}\frac{(x_j-z_j)^{5/3}}{ \sqrt{\frac{x_{j+1}-x_j}{\Delta x}}}+ \frac{1}{n}\frac{{}(x_{j+1}-z_{j+1})^{5/3}}{ \sqrt{\frac{x_{j+1}-x_j}{\Delta x}}}\right) \right] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M_{i,i+1}=(1-\omega)\cdot \frac{\Delta x}{2}+\theta\cdot \Delta t \cdot &#xA;\frac{1}{2}\left( \frac{1}{n}\frac{(x_j-z_j)^{5/3}}{ \sqrt{\frac{x_{j+1}-x_j}{\Delta x}}}+ \frac{1}{n}\frac{{}(x_{j+1}-z_{j+1})^{5/3}}{ \sqrt{\frac{x_{j+1}-x_j}{\Delta x}}}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $\textbf{r}$ constant. So concluding the non-linearity is quite strong as there is 1 by square root and additionally unknown value to the power of $5/3$. &#xA;For solution of system of linear equations I use lu decomposition for sparse matrices. I do not consider those systems large, this word was added by moderator. The systems are of size 1000x1000.&lt;/p&gt;&#xA;" OwnerUserId="4334" LastEditorUserId="4334" LastEditDate="2016-06-14T05:32:58.103" LastActivityDate="2016-06-14T12:12:38.413" Title="Efficient solution of large systems of non linear algebraic equations" Tags="|finite-element|nonlinear-equations|convergence|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24206" PostTypeId="1" AcceptedAnswerId="24218" CreationDate="2016-06-13T07:17:13.947" Score="1" ViewCount="1744" Body="&lt;p&gt;I'm having difficulty with numerically solving the inviscid burgers equation.Godunov's scheme is used in most of what I've found in literature . Now my question is if using a crank nicolson shceme is wrong or not? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} + u\frac{\partial u}{\partial x}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with this BC:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;at $t&amp;gt; 0 ,x=0$  :  $u=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using a finite volume method: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial }{\partial t}\int udv + \int u\frac{\partial u}{\partial x}dv=0&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t}\Delta v +(uuA) _{e}-(uuA) _{w}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} + F_{e}u_{e}-F_{w}u_{w}=0&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F=\frac{u}{\Delta x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and for discretization in time I used Crank-Nicolson &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{p}^{n+1}-u_{p}^{n}=\frac{\Delta t}{2}(-F_{e}u_{e}+F_{w}u_{w})^{n+1}+\frac{\Delta t}{2}(-F_{e}u_{e}+F_{w}u_{w})^{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the final form of equation is as (upwind scheme) :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(1+\frac{\Delta t}{2}F_{e})u_{P}^{n+1}=(-\frac{\Delta t}{2}F_{w}) u_{W}^{n+1} +u_{P}^{n}+\frac{\Delta t}{2}(F_{w}u_{w}-F_{e}u_{e})^{n}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and for the first block :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(1+\frac{\Delta t}{2}F_{e})u_{P}^{n+1}= u_{P}^{n}+\frac{\Delta t}{2}(-F_{e}u_{e})^{n}+\Delta t F_{w}u_{w}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and after that I tried to solve a set of linear algebraic equation. The result in every time step seems to converge but with time passing the magnitude of velocity is also increasing which is an indication of a mistake.&lt;/p&gt;&#xA;" OwnerUserId="16922" LastActivityDate="2016-06-16T18:43:55.483" Title="Numerical solution of burgers equation with finite volume method and crank-nicolson" Tags="|pde|finite-volume|crank-nicolson|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24207" PostTypeId="1" AcceptedAnswerId="24210" CreationDate="2016-06-13T12:32:40.593" Score="3" ViewCount="421" Body="&lt;p&gt;I am running octave and i have been trying ode45, ode54, ode23 etc to integrate the equation `&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q''(t) = B\cos(Q)\sin(\omega t)$$&#xA;$$Q(t=0)=0.$$ When the time interval to be integrated increases, the execution time increases non-linearly. Integrating from t=0 to t=T takes 1.2 seconds. For 10T it takes 12 seconds, fine! But for 100T it takes 500 seconds, instead of the expected 120. Why is that? Can i get around it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to integrate about 1000 times the interval that took 1.2 seconds. 1200 seconds would be fine. But after 4 hours i don't have any results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I had made a couple of mistakes in the actual equation, now its right. The script looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;U=1e-12; # Unit factor to make computation precise and fast. &#xA;&#xA;B = 0.7 * 6.2 / 2 * 1e26*U^2;&#xA;w = 6*pi/800 * 1e17*U;&#xA;&#xA;OPT = odeset (&quot;InitialStep&quot;,1e-16/U,&quot;MaxStep&quot;, 4e-16/U, ...&#xA;&quot;RelTol&quot;, 1e-1, &quot;AbsTol&quot;, 1e-1);&#xA;&#xA;tSpan=[0 1e-12/U]; #150 ps =&amp;gt; 15 deg from 0 deg. &#xA;init=[0 0]; &#xA;&#xA;tic&#xA;[t, g]=ode23( @(t,G)[G(2); B*cos(G(1))*sin(w*t)],tSpan,init,OPT);&#xA;t = t*U;&#xA;g=g(:,1);&#xA;toc&#xA;plot(t,g) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20686" LastEditorUserId="20686" LastEditDate="2016-06-14T17:26:49.000" LastActivityDate="2016-06-15T03:29:54.047" Title="Computational time not proportional to integration interval in ODE-solver?" Tags="|matlab|ode|time-integration|octave|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="24208" PostTypeId="1" CreationDate="2016-06-13T13:10:43.453" Score="1" ViewCount="38" Body="&lt;p&gt;Lately, I was reading some seminal papers on Fractional Step Algorithms and I found this one:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Kim, D., Choi, H. A Second-Order Time-Accurate Finite Volume Method for Unsteady&#xA;Incompressible Flow on Hybrid Unstructured Grids. Journal of Computational Physics, Volume 162, Issue 2, pp. 411-428 (2000)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And I am confused on as to how to use/define the boundary conditions  for the auxiliary variables  in the method proposed. Specifically, in this paper the    use of a second auxiliary velocity is proposed, $\hat{u_i}$, and a certain    boundary condition is proposed for it in equation (19). Anyway, the    proposed integration method described by equations (12)-(15) seem not to require  $\hat{u_i}$   on the boundaries; the rotational part of the momentum    equation, equation (12), is solved for $\delta \hat{u}$ which, I think, it    can be extrapolated from the cell-center to the faces (in specific, the boundary faces); and equation (13) only requires the surface integration of the left-hand side which only involves the pressure. My question is: Is the latter assertion (the extrapolation of $\delta \hat{u}$) correct? If not, How can I couple the    boundary condition $\hat{u_i} = u_i^{n+1}+O(\Delta t^2)$ with equation (12) and equation    (14)? How can be implemented?&lt;/p&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="14659" LastEditDate="2016-06-13T13:19:10.743" LastActivityDate="2016-06-13T13:19:10.743" Title="BC's for intermediate velocities in Implicit Fractional Step Methods" Tags="|finite-volume|navier-stokes|projection|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24210" PostTypeId="2" ParentId="24207" CreationDate="2016-06-13T17:05:29.420" Score="8" Body="&lt;p&gt;Substantially edited, since the original poster changed his equation...  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the MATLAB (and Octave) ODE solvers dynamically adjust the step size as needed to maintain an accurate solution.  If the integrator starts to use smaller step sizes then the process slows down.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the particular form of your ODE, the $\sin(wt)$ factor has a period of about 0.003, so substantially smaller time steps will be required.  It's also smart to put in a maximum time step limit to stop the solver from stupidly deciding to use too large a time step.    A time step of 0.0001 (1.0e-16/U) seems about right to me for this problem, and that's what the code given by the OP uses, along with a maximum time step of 0.0004 (4e-16/U).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're interested in solving this equation for time spans of up to 1000 (1000/U), so we're looking at taking ten million steps or so (more if the solver decides that it needs to cut the time step.)  Given that we have to evaluate the function multiple times per time step and all of the other overhead of the ODE solver, this is a fair bit of work.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried running the script under MATLAB for periods of 1, 10, 100, and 1000, and got completely consistent solutions with elapsed times of 0.3&#xA;seconds, 3.8 seconds, 32, seconds, and 322 seconds.  This scales linearly,&#xA;so it doesn't appear that the solver got bogged down taking tiny time steps.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to check the accuracy of the solution, I tried decreasing the max time step by a factor of 10 and reran.  I got essentially the same solutions, in elapsed times that were about 10 times longer.  Thus I'm confident that the supplied initial time step and maximum time step are OK and the solution produced by MATLAB is reasonably accurate (to say 2 digits.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried allowing a larger maximum time step and found that MATLAB's ode23() was tricked into taking steps that were too long and the resulting solution (although very fast) was completely inaccurate.  You need to be careful with limiting the maximum time step for this equation because of the fast oscillation in the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried tightening the relative and absolute error tolerances to 1.0e-3 (MATLAB's default) from 1.0e-1.  The solutions were very similar but the solution times roughly doubled.            &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran some quick checks using ode45 instead of ode23 and got essentially the same solutions, but the execution time was about twice as long.  This isn't surprising, since we're time step limited and ode45 does about twice as many function evaluations per time step as ode23.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran the script on the same machine under Octave, and the runs took 1 second, 10 seconds, 166 seconds, ... (the last one is still running after more than 45 minutes.) The solutions I obtained looked very similar to the MATLAB solutions which is good.  However, this is not scaling linearly, indicating to me that the Octave implementation of ode23 is probably shortening the time step when it really doesn't have to.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend that you give this example to the developers of the Octave ODE package and ask them to look into the poor performance in comparison with MATLAB ode23().      &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-06-15T03:29:54.047" LastActivityDate="2016-06-15T03:29:54.047" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24212" PostTypeId="2" ParentId="24203" CreationDate="2016-06-14T04:42:10.260" Score="3" Body="&lt;p&gt;In general, you can do more with PETSc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SUNDIALS is a collection of ODE solvers (in CVODE, Adams-Bashforth and BDF methods; in ARKODE, ARKIMEX methods) and DAE solvers (IDAS implements a BDF method) with sensitivity capabilities (the CVODES and IDAS variants), and a nonlinear solver (KINSOL). There are a few Krylov solvers in there (at least GMRES and CG) and a couple other fixed-point type methods, like Anderson iteration, plus interfaces to a few solvers, like KLU, and SuperLU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc has Krylov subspace methods, nonlinear solvers, ODE and DAE solvers, linear and nonlinear preconditioners (including a multigrid method), mesh management features, and interfaces to external software that provide more of these types of features, including a CVODE interface to SUNDIALS that uses GMRES.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only things in SUNDIALS that aren't in PETSc right now (and may even be in newer versions of PETSc) are BDF methods, and BDF-based sensitivity calculations. PETSc has an implementation for sensitivity computations, may use different algorithms for computing sensitivities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc, on the other hand, has loads of stuff that SUNDIALS doesn't have, like more linear solvers, more preconditioners (as far as I remember, SUNDIALS doesn't have any native preconditioners), nonlinear preconditioning, mesh management, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SUNDIALS is easier to learn, but PETSc will probably pay off more for you in the long run.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2016-06-14T04:42:10.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24213" PostTypeId="1" CreationDate="2016-06-14T05:00:10.097" Score="1" ViewCount="187" Body="&lt;p&gt;I have implemented a Quasi-Newton method based on the Hessian approximation. I am noticing that the algorithm takes too many iterations to converge, even though it does converge. What I am not able to understand is the really small step size ($l_{\infty} \text{norm} \le 10^{-5}$) for many of these iterations. The problem is ill-conditioned and I read that Quasi-Newton can degenerate to steepest descent for ill-conditioned problems. Is that what is happening? If so, is there any intuition about the step size being so small? I have tried the damping modification given in section 18.3 from Nocedal and Wright. However, it is not improving the step length, since, the damping criterion itself becomes very small. Is there some other principled way to ensure larger step size? Will larger step size help with faster convergence?&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2016-06-14T14:07:51.560" LastActivityDate="2016-06-14T14:07:51.560" Title="Quasi Newton taking very small steps" Tags="|newton-method|quasi-newton|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="24214" PostTypeId="2" ParentId="24205" CreationDate="2016-06-14T12:12:38.413" Score="1" Body="&lt;p&gt;Looking to the paper you referred to in EDIT, my first guess for possible problems in your nonlinear solver is that in fact the nonlinear system you present is different (simplified) than the one given in the paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your description you do not mention what to do if during iterations one obtains e.g. $x_{j+1}-x_j \approx 0$ when you divide by zero or if $x_j &amp;lt; z_j$ when you get complex numbers. Moreover following the paper you should have instead of $\sqrt{\frac{x_{j+1}-x_j}{\Delta x}}$ the term $\sqrt{\frac{|x_{j+1}-x_j|}{\Delta x}}$. The nonlinearity in your algebraic system comes from the approximation of nonlinear diffusion coefficient in PDE where the cases I mentioned are in the paper defined differently (``regularized'').&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do follow the paper also in these definitions (it means you simply did not describe it in your question), you should really specify more precisely what you do not like in the behavior of your solvers, but such ``piecewise'' defined non-linearity as mentioned in the paper can deteriorate the behavior of nonlinear iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, I would expect that if you solve a problem where those cases can not happen e.g. a propagation of a wave given initially by $h(x,0)=2+sin(x)$ for $z \equiv 0$ and boundary conditions $h(-\pi/2,t)=1$ and $h(\pi/2,t)=3$, you really should get a quadratic convergence of Newton solver for enough small time step $\Delta t$. I use the same notation as in the referred paper, it means the system is in fact $M h = r$ and $x$ is the space variable, so the discrete unknowns are $h_j$ and not $x_j$. &lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-06-14T12:12:38.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24215" PostTypeId="1" CreationDate="2016-06-14T13:47:08.580" Score="2" ViewCount="177" Body="&lt;p&gt;In d'Halluin et al. (2005) (&lt;a href=&quot;http://imajna.oxfordjournals.org/content/25/1/87.short&quot; rel=&quot;nofollow&quot;&gt;http://imajna.oxfordjournals.org/content/25/1/87.short&lt;/a&gt;) the authors claim that the correlation integral&#xA;$$ I(x) = \int_{-\infty}^{\infty} V(x+y) f(y) dy $$&#xA;can be approximated with FFT methods by&#xA;$$ I_{k} = IFFT((FFT(V)_{k})(FFT(f))_{k}^{*}) $$&#xA;where $(.)^{*}$ denotes the complex conjugate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to program this in Matlab with $V = \max(90 e^{y} - K,0)$ and $f$ being the probability density function of the normal distribution with $\mu = 0.1$ and $\sigma = 0.2$. Analytically I get the result $I(0) = 8.783$ but in whatever way I try the FFT approximation the results are nowhere close to that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My math background is really weak and I don't have any intuition or knowledge regarding the FFT method. I calculated a set of values (powers of 2) for $V()$ and $f()$, did both element-wise and matrix multiplications of the FFTs of the two vectors and checked the real part of the IFFT, but I never get the desired result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong? Can someone tell me how I get to the result?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code of one of the ways I did it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S = 90;&#xA;K = 100;           &#xA;mu = 0.1;&#xA;sig = 0.2;&#xA;AA = 2.5;&#xA;N = 255;             &#xA;x = (linspace(-AA,AA,N+1))';&#xA;&#xA;V = max(S.*exp(x) - K,0);&#xA;ffff = normpdf(x,mu,sig);&#xA;&#xA;I_k = real(ifft(fft(V).*conj(fft(ffff))));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20693" LastEditorUserId="9667" LastEditDate="2016-06-14T18:52:35.363" LastActivityDate="2016-06-15T08:31:23.520" Title="Approximating improper integral with FFT" Tags="|matlab|fourier-transform|integration|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24216" PostTypeId="2" ParentId="24203" CreationDate="2016-06-14T18:12:48.407" Score="3" Body="&lt;p&gt;As mentioned in Geoff Oxberry's more complete answer, it should be noted that &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSSUNDIALS.html&quot; rel=&quot;nofollow&quot;&gt;PETSc includes TSSUNDIALS, an interface to SUNDIALS&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you configure PETSc with the &lt;code&gt;--download-sundials&lt;/code&gt; option (see &lt;code&gt;python2 ./configure  --help | grep -A 2 sundials&lt;/code&gt; for other related options, such as using an existing SUNDIALS library), then you can use at least some of the SUNDIALS functionality from PETSc and thus might have an easier time making comparisons between SUNDIALS and PETSc's other TS implementations.&lt;/p&gt;&#xA;" OwnerUserId="3824" LastEditorUserId="3824" LastEditDate="2016-06-15T08:19:01.407" LastActivityDate="2016-06-15T08:19:01.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24217" PostTypeId="1" CreationDate="2016-06-14T21:53:36.700" Score="2" ViewCount="484" Body="&lt;p&gt;It is easy to prove that,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\{p_1(x,y)=1-x-y\;,\;p_2(x,y)=x\;,\;p_3(x,y)=y\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is a Lagrangian basis of $\mathbb{P}_1(\hat{T})$ (polynomials of total degree less that 1 living on $\hat T$), where $\hat T$ is the triangle with vertices $a_1=(0,0)$, $a_2=(1,0)$ and $a_3(0,1)$. It is clear that $p_i(a_j)=1$ if $i=j$ and $p_i(a_j)=0$ if $i\neq j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also can calculate a basis function generating $\mathbb{P}_2(\hat T)$. But calculate basis functions of $\mathbb{P}_3(\hat T)$, $\mathbb{P}_4(\hat T)$ it is too dificult, so I think that there exists a general formula to write in my finite element code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There exists a formula to compute the basis function sets of $\mathbb{P}_k(\hat T)$, with $k$ any integer?&lt;/p&gt;&#xA;" OwnerUserId="17630" LastActivityDate="2016-06-15T09:46:10.100" Title="General Lagrange basis formula (usual problem in finite element context)" Tags="|finite-element|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24218" PostTypeId="2" ParentId="24206" CreationDate="2016-06-14T23:44:56.087" Score="2" Body="&lt;p&gt;If I understand correctly, you are using a centered finite difference in space and the implicit trapezoidal method in time.  That scheme is unconditionally absolutely stable, but will generate spurious oscillations.  So you should expect to see some increase in the maximum value of $u$, but it shouldn't blow up.  If it blows up, you have an implementation error (bug).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll add that this is a lousy method for a purely hyperbolic PDE, since it generates oscillations and isn't very accurate for large time steps anyway.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-06-14T23:44:56.087" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24219" PostTypeId="2" ParentId="24217" CreationDate="2016-06-15T02:09:11.693" Score="2" Body="&lt;p&gt;I hope I'm correct in interpreting your question as asking for a constructive approach for generating the Lagrange bases on the reference triangle, apologies if this is more simplistic than you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's introduce a third (dependent) spatial variable, $\zeta = 1 - x -y$. Then  your basis is just the set ${\zeta,x,y}$. In the $P_2$ case we're free to place the node points at the vertices of the triangle, and the midpoints of the edges. We can identify the corner nodes with &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a quadratic function which is 0 at $\zeta=0,\frac{1}{2}$ (i.e proportional to $\zeta (\zeta-\frac{1}{2}$))&lt;/li&gt;&#xA;&lt;li&gt;a quadratic function which is 0 at $x=0,\frac{1}{2}$ (proportional to $x (x-\frac{1}{2}$))&lt;/li&gt;&#xA;&lt;li&gt;a quadratic function which is 0 at $y=0,\frac{1}{2}$ (proportional to $y (y-\frac{1}{2}$))&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Meanwhile the edge points can be identified with&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a quadratic function which is zero on $x = 0$ and $y=0$ (ie. proportional to  $xy$)&lt;/li&gt;&#xA;&lt;li&gt;a quadratic function which is zero on $\zeta = 0$ and $y=0$ (ie. proportional to  $\zeta y$)&lt;/li&gt;&#xA;&lt;li&gt;a quadratic function which is zero on $x = 0$ and $\zeta=0$ (ie. proportional to  $\zeta y$)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With $P_3$ we just extend the concept again; a corner point takes a cubic function zero on $\zeta =0,\frac{1}{3},\frac{2}{3}$ hence proportional to $\zeta(\zeta-\frac{1}{3})(\zeta-\frac{2}{3})$, with the others cycling through. Now for the others, we carry on with a process of elimination; for example there are 3 points not on the lines $\zeta =0$ or $\zeta=1/3$. Two of these lie on the line $x=0$ (similarly $y=0$) hence the remaining point takes a function proportional to $\zeta (\zeta-1/3)x$ [or $\zeta (\zeta-1/3)y$ for its friend. Symmetric rotation takes us through all points except the internal point. This requires a function which vanishes on $\zeta = 0$, $x=0$ and $y=0$, thus proportional to $xy\zeta$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way finding the functions reduces to an essentially graphical exercise of finding $k$ lines through all but one of $T_{k+1}$ points arranged in a triangle. Calculating the normalisation constants is left as an exercise for the reader.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-06-15T02:09:11.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24220" PostTypeId="1" CreationDate="2016-06-15T05:37:06.207" Score="3" ViewCount="1827" Body="&lt;p&gt;I have an $n$ by $m$ numpy array representing a rectangular lattice $L$, where each site contains a one or a zero, representing two different materials. I'm modelling heat flow across this lattice. The idea is that the conductivity between two lattice sites depends on the material in both sites.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve the heat equation I need to create from this a sparse $nm\times nm$ matrix $C$, where each of the $nm$ rows and columns corresponds to a cell in my original matrix, and where the $(i,j)^\text{th}$ element represents the thermal conductance between sites $i$ and $j$ on the lattice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is whether there is an efficient way to implement this in numpy. Currently I have nested Python loops that iterate over every element of $L$ and add corresponding entries to $C$, but for large lattices this obviously gets rather slow. Is there a way to do this using numpy primitives rather than Python loops?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For clarity, my current code looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = sp.lil_matrix((w*h,w*h))&#xA;&#xA;def coords_to_index(x, y):&#xA;    return x*h+y&#xA;&#xA;def setlink(p1x, p1y, p2x, p2y):&#xA;    i = coords_to_index(p1x,p1y)&#xA;    j = coords_to_index(p2x,p2y)&#xA;    ci = grid[p1x,p1y]&#xA;    cj = grid[p2x,p2y]&#xA;    if ci==cj:&#xA;        d = 1&#xA;    else:&#xA;        d = 1e-3 # conductance is much smaller between different materials&#xA;    A[i,j] = d&#xA;    return d&#xA;&#xA;for x in range(w):&#xA;    for y in range(h):&#xA;        d = 0.&#xA;        if x&amp;gt;0:&#xA;            d += setlink(x,y,x-1,y)&#xA;        else:&#xA;            d += 1&#xA;        if x&amp;lt;w-1:&#xA;            d += setlink(x,y,x+1,y)&#xA;        else:&#xA;            d += 1&#xA;        if y&amp;gt;0:&#xA;            d += setlink(x,y,x,y-1)&#xA;        else:&#xA;            d += 1&#xA;        if y&amp;lt;h-1:&#xA;            d += setlink(x,y,x,y+1)&#xA;        else:&#xA;            d += 1&#xA;        i = coords_to_index(x,y)&#xA;        A[i,i] = -d&#xA;&#xA;A = A.tocsr()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3746" LastEditorUserId="3746" LastEditDate="2016-06-15T07:20:18.740" LastActivityDate="2016-06-15T07:20:18.740" Title="Efficiently creating an adjacency matrix from a lattice in numpy" Tags="|python|numpy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24221" PostTypeId="1" CreationDate="2016-06-15T05:45:32.503" Score="0" ViewCount="92" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Suppose you are attempting to solve &lt;span class=&quot;math-container&quot;&gt;$Ax = b$&lt;/span&gt; using linear stationary iteration method defined by &lt;span class=&quot;math-container&quot;&gt;$$x_k = G x_{k-1} + f$$&lt;/span&gt; that is consistent with &lt;span class=&quot;math-container&quot;&gt;$Ax = b$&lt;/span&gt;, i.e., for which &lt;span class=&quot;math-container&quot;&gt;$f = (I - G)A^{-1}b$&lt;/span&gt;. Suppose the eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; are real and such that &lt;span class=&quot;math-container&quot;&gt;$|\lambda_1| &amp;gt; 1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|\lambda_i| &amp;lt; 1$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$2\leq i \leq n$&lt;/span&gt;. Also, suppose that &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; has &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; linearly independent eigenvectors, &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1\leq i \leq n$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;a.) Show that there exists an initial condition &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$x_k\rightarrow x = A^{-1}b$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;b.) Does your answer give a characterization of selecting &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; that could be used in practice to create an algorithm that would ensure convergence?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Solution - Let &lt;span class=&quot;math-container&quot;&gt;$e^{(k)} = x_k - x$&lt;/span&gt; be the error on step &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. We know that &lt;span class=&quot;math-container&quot;&gt;$$e^{(0)} = \sum_{i=1}^{n}\alpha_i z_i, \ \ \|z_i\| = 1$$&lt;/span&gt; Since &lt;span class=&quot;math-container&quot;&gt;$|\lambda_i| &amp;lt; 1$&lt;/span&gt; then we can conclude that &lt;span class=&quot;math-container&quot;&gt;$$\lim_{k\rightarrow \infty} e^{(k)} = 0$$&lt;/span&gt; which implies that there is a fixed point &lt;span class=&quot;math-container&quot;&gt;$x^* = A^{-1} b$&lt;/span&gt;. Thus there exists an &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$x_k\rightarrow x^* = A^{-1}b$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am not really sure if this is correct. Any suggestions are greatly appreciated. This is not homework. I am preparing for a qualifier exam at the end of August in Computational Mathematics.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-06-17T07:10:00.303" Title="Linear stationary iteration method" Tags="|linear-algebra|iterative-method|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24222" PostTypeId="2" ParentId="24221" CreationDate="2016-06-15T08:03:19.303" Score="3" Body="&lt;p&gt;This is not correct. To conclude that the error converges, you need that &lt;em&gt;all&lt;/em&gt; eigenvalues (and thus the spectral radius) of the iteration matrix are less than $1$; but the problem states that $\lambda_1&amp;gt;1$. So the standard theory is not applicable. (In fact, since you don't use any property of $x_0$, you have &quot;shown&quot; convergence of the iteration. This should have made you suspicious.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not giving the full answer, but here's a hint: Your approach of writing the initial error as a linear combination of basis vectors $z_i$ is right, but you have to choose the &lt;em&gt;right&lt;/em&gt; basis -- if you look at the statement again (and the standard counterexample for why $\rho(G)&amp;lt;1$ is necessary for convergence), you should see that there's an obvious choice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; Since you insist:&lt;/p&gt;&#xA;&#xA;&lt;blockquote class=&quot;spoiler&quot;&gt;&#xA;  &lt;p&gt; As the statement of the problem suggests, expand the error $e_0$ in the basis $z_i$ of eigenvectors of $G$. Then any $x_0$ such that $\alpha_1=0$ will do. (Naturally, to actually find such an $x_0$, you'd have to already know $x$.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-06-17T07:10:00.303" LastActivityDate="2016-06-17T07:10:00.303" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24223" PostTypeId="2" ParentId="24215" CreationDate="2016-06-15T08:31:23.520" Score="1" Body="&lt;p&gt;Even though this is a strict requirement, the authors explicitly mention that $x$ is equally spaced in the space of $log(S)$. They also generate it this way in the pseudocode. In that case $\bar{V}(log S) = V(S)$. I think you do not satisfy that. And in the paper, it is also the case that $\bar{f}$ is a probability density. This is just to warn in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their resolution to interpolation along non-equally spaced intervals comes at a later stage, where they explicitly interpolate the discrete values of the result onto the original $S$ grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, theoretically, the equation is an altered &lt;a href=&quot;https://en.wikipedia.org/wiki/Cross-correlation&quot; rel=&quot;nofollow&quot;&gt;cross-correlation&lt;/a&gt;, where $f$ is not a complex conjugate. Then in that case, the formula is correct (not an approximation) - with periodic border treatment of course.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2016-06-15T08:31:23.520" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24224" PostTypeId="2" ParentId="24217" CreationDate="2016-06-15T09:46:10.100" Score="2" Body="&lt;p&gt;There is no closed formula for the general case, but there are standard procedures. I'm assuming you are looking for triangular Lagrange elements, but the adaption to other cases is straightforward. Say you are looking for the nodal basis $\{\psi_1,\dots,\psi_m\}$ of $P_k(T)$, where $T$ is the reference triangle with vertices $(0,0)$, $(0,1)$, and $(1,0)$, and you have $m:=\frac12(k+1)(k+2)$ (the dimension of $P_k(T)$) different nodes $\xi_1,\dots,\xi_m\in T$ (where $\xi_i=(x_i,y_i)$). A nodal basis satisfies by definition &#xA;$$\label{1}\psi_i(\xi_j) = \delta_{ij} = \begin{cases} 1 &amp;amp; \text{if } i=j, \\ 0 &amp;amp; \text{else.}\end{cases} \tag{1}$$&#xA;If the nodes are chosen correctly, this uniquely determines the $\psi_i$. (In fact, this condition is exactly what determines how many nodes have to lie on the different vertices, edges, and the interior of $T$). This means that you can use this condition to solve for the unknown coefficients of the $\psi_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way (an alternative way is given by origimbo) to do this is to write &#xA;$$\psi_i(\xi_j) = \sum_{n=1}^m a_{in} p_n(\xi_j),$$&#xA;where $p_n$ are the monomials spanning $P_k(T)$, i.e., $1,x,y,x^2,xy,y^2,\dots$ Inserting this into \eqref{1} leads to the following system of linear equations:&#xA;$$&#xA;\begin{pmatrix} p_1(\xi_1) &amp;amp; p_2(\xi_1) &amp;amp;\dots \\ p_1(\xi_2) &amp;amp; p_2(\xi_2) &amp;amp; \dots\\ \vdots &amp;amp; \vdots &amp;amp;\ddots \end{pmatrix}&#xA;\begin{pmatrix} a_{11}&amp;amp; a_{21} &amp;amp;\dots\\ a_{12} &amp;amp; a_{22} &amp;amp; \dots \\\vdots &amp;amp; \vdots &amp;amp;\ddots \end{pmatrix}&#xA;=&#xA;\begin{pmatrix} 1&amp;amp; 0 &amp;amp;\dots\\ 0 &amp;amp; 1 &amp;amp; \dots \\\vdots &amp;amp; \vdots &amp;amp;\ddots \end{pmatrix}&#xA;$$&#xA;(The first matrix is called &lt;em&gt;Vandermonde matrix&lt;/em&gt;.)&#xA;This system you can solve for the unknown $a_{ij}$ either symbolically (e.g., using SymPy or Matlab's Symbolic Computing Toolbox -- no need to do this by hand!) or numerically (e.g., using Matlab's backslash, although Vandermonde matrices can get quite ill-conditioned for large $k$ and arbitrary $\xi_j$, although there are choices that minimize the condition number). &lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-06-15T09:46:10.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24225" PostTypeId="1" CreationDate="2016-06-15T10:22:54.743" Score="2" ViewCount="179" Body="&lt;p&gt;I need to model a &quot;fishing rod&quot; in 2 dimensions by joining several &quot;rigid sticks&quot; by flexible/elastic joints. The joints act as plate/torsion springs with different spring constants. The &quot;fishing rod&quot; will bend at these joints. The segments of the rod become lighter and the springs softer, towards the tip.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only input data are the length and weight of each piece of rod and the respective spring constant by which it is attached to the previous one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to see how this object bends and moves when one end (the handle) is exposed to torque (i.e. how the fishing rod bends when you throw the bait. - There is no need to include torque around the object's own axis.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is not a problem involving stress analysis by FEM. It is about movement in a time frame of seconds. I will later try to extend this model into biomechanics.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can follow (and have simulated) the Lagrange transform for the double pendulum, but for more than two segments, the Lagrange transform becomes &quot;virtually impossible&quot; to compute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.m.wikipedia.org/wiki/Multibody_system&quot; rel=&quot;nofollow&quot;&gt;Multi body dynamics&lt;/a&gt; software can model multi linked objects  (and many more things). See &lt;a href=&quot;https://youtu.be/TtHFRWshfhI&quot; rel=&quot;nofollow&quot;&gt;this video&lt;/a&gt; for a N-link pendulum simulation. How can the movements of such a complex object be computed!?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I could just get a grip on how the &quot;physics engine&quot; for multiple, elastic/flexible joints work, in this kind of software, I could maybe use this to model the fishing rod.&lt;/p&gt;&#xA;" OwnerUserId="9385" LastEditorUserId="9385" LastEditDate="2016-06-18T15:34:16.197" LastActivityDate="2016-06-18T15:34:16.197" Title="How does Multi Body Dynamics software work for flexible joints?" Tags="|simulation|computational-physics|time-integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24227" PostTypeId="1" AcceptedAnswerId="24228" CreationDate="2016-06-15T23:55:35.110" Score="4" ViewCount="307" Body="&lt;p&gt;I have an ODE for a scalar function $u=u(t)$ of the form:&#xA;$$&#xA;\frac{du}{dt}=L(u).&#xA;$$&#xA;Here the function $L=L(u)$ satisfies:&#xA;$$&#xA;L(0)=0, \quad L'(u)\le0.&#xA;$$&#xA;Then it is easy to see that the solution $u=u(t)$ has the following property:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(i) If the initial value $u(0)\ge0$, then $u(t)\ge0$ for any $t&amp;gt;0$;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(ii) If the initial value $-M\le u(0)\le M$ with $M&amp;gt;0$, then $-M\le u(t)\le M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: I need to find a numerical method to solve this ODE and under &lt;strong&gt;arbitrary&lt;/strong&gt; step size maintain the two properties or only one of two. I only know that the Euler backward method has these two properties. Does anyone of you know high order (e.g. third-order) methods to solve this problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any links to literature or for further reading would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20711" LastEditorUserId="9667" LastEditDate="2016-06-16T02:19:26.027" LastActivityDate="2016-06-16T04:11:25.197" Title="Solve an ODE with positivity-preserving property unconditionally" Tags="|ode|time-integration|implicit-methods|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24228" PostTypeId="2" ParentId="24227" CreationDate="2016-06-16T01:18:09.983" Score="6" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The two properties are usually called positivity-preserving and monotonicity-preserving (makes it easier to find this question).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Looking at &lt;a href=&quot;http://www.ams.org/journals/mcom/2006-75-254/S0025-5718-05-01794-1/S0025-5718-05-01794-1.pdf&quot;&gt;http://www.ams.org/journals/mcom/2006-75-254/S0025-5718-05-01794-1/S0025-5718-05-01794-1.pdf&lt;/a&gt; and &lt;a href=&quot;http://homepages.cwi.nl/~willem/DOCART/SIAM_HRS.pdf&quot;&gt;http://homepages.cwi.nl/~willem/DOCART/SIAM_HRS.pdf&lt;/a&gt; it seems that implicit Euler is the exception among implicit linear multistep methods in that only implicit Euler allows for timesteps to be arbitrary while preserving monotonicity (section 5.3). They say the restrictions on implicit linear multistep methods' timesteps are not much better than on explicit multistep methods.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a discussion of monotonicity-preserving Runge-Kutta methods in Hairer-Wanner Vol.II, Section IV.11: implicit Euler is the only one among them having a threshold factor of $\infty$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;See also &lt;a href=&quot;http://www.cscamm.umd.edu/tadmor/pub/linear-stability/Gottlieb-Shu-Tadmor.SIREV-01.pdf&quot;&gt;http://www.cscamm.umd.edu/tadmor/pub/linear-stability/Gottlieb-Shu-Tadmor.SIREV-01.pdf&lt;/a&gt;, which discusses high-order strong stability-preserving methods and gives an overview of available methods (not unconditionally SSP, though).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-06-16T04:11:25.197" LastActivityDate="2016-06-16T04:11:25.197" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24229" PostTypeId="2" ParentId="24206" CreationDate="2016-06-16T02:44:09.453" Score="1" Body="&lt;p&gt;Thank you for your clarification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Thomas algorithm solves $Ax=b$ when, e.g., A is a tridiagonal matrix (there are other special cases I believe, but this is not one of them). Your &quot;final form of equation&quot; does not appear to be in this form, nor does it look like this is possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @David Ketcheson mentioned, implicit time marching is not very attractive for problems like this. Usually the diffusion term is treated implicitly in time, and for good reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explicit time marching has numerical stability restrictions due to the advection and diffusion terms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta t &amp;lt; C_1 \frac{\Delta x}{\max(U)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta t &amp;lt; C_2 \frac{\Delta x^2}{\nu}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;respectively. The latter is often more restrictive (for diffusion dominant problems or when node points are highly clustered). Often, the advection term is treated explicitly and the CFL condition is used to determine the largest allowable time step when the diffusion term is not a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @David Ketcheson mentioned in his comment, explicit Euler is not a good choice for central differencing since it is unconditionally unstable. However, upwinding schemes with explicit time marching is frequently used in high Reynolds number CFD, so I imagine that it is applicable here, however, you &lt;em&gt;cannot&lt;/em&gt; use central differencing since you have no diffusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for spatial discretization, there is a lot of literature on upwinding schemes.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="8174" LastEditDate="2016-06-16T18:43:55.483" LastActivityDate="2016-06-16T18:43:55.483" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24231" PostTypeId="2" ParentId="24225" CreationDate="2016-06-16T16:05:07.177" Score="2" Body="&lt;p&gt;As @BillGreene mentioned, this would be better modeled as a continuous elastic body. It would be something like a cantilever beam subject to its own weight and a force at the tip. You can use, &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler%E2%80%93Bernoulli_beam_theory&quot; rel=&quot;nofollow&quot;&gt;Euler-Bernoulli&lt;/a&gt; beams for this, where the equations are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  \cfrac{\partial^2 }{\partial x^2}\left(EI\cfrac{\partial^2 w}{\partial x^2}\right) = - \mu\cfrac{\partial^2 w}{\partial t^2} + q(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you can discretize your domain and obtain a time-integration scheme, you can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_method&quot; rel=&quot;nofollow&quot;&gt;Finite Differences&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_element_method&quot; rel=&quot;nofollow&quot;&gt;Finite Elements&lt;/a&gt; for this. A common one is &lt;a href=&quot;https://en.wikipedia.org/wiki/Verlet_integration&quot; rel=&quot;nofollow&quot;&gt;Verlet integration&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec x_{n+1}=2\vec x_n-\vec x_{n-1}+\vec a_n\,\Delta t^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where you express your time stepping as a function of the past two time steps and the force in your &lt;em&gt;nodes&lt;/em&gt;. You can see an implementation &lt;a href=&quot;http://subprotocol.com/system/introducing-verlet-js.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I think that your idea might work. It will not be a correct representation of the physics of the problem, but I think that it might make sense. I was checking a couple of physics simulations software like &lt;a href=&quot;http://physion.net/&quot; rel=&quot;nofollow&quot;&gt;Physion&lt;/a&gt;, and they do not have rotational springs. So, you will need to implement that part. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding on how you do this, you can formulate the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lagrangian_mechanics&quot; rel=&quot;nofollow&quot;&gt;Lagrangian&lt;/a&gt; and then obtain the equations of motion. An example for the n-links pendulums is &lt;a href=&quot;http://www.moorepants.info/blog/npendulum.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. Where the author used Python and SymPy as part of the task.After this, you will, probably, find a pattern to &lt;em&gt;assemble&lt;/em&gt; your stiffness and mass matrices and then you can use as many links as you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all cases, you will need a way to compute the &quot;forces&quot; between the nodes in your system and a time-integration technique, that is commonly explicit (and &lt;a href=&quot;https://en.wikipedia.org/wiki/Symplectic_integrator&quot; rel=&quot;nofollow&quot;&gt;symplectic&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2016-06-16T16:05:07.177" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24232" PostTypeId="1" CreationDate="2016-06-16T16:57:58.397" Score="4" ViewCount="8613" Body="&lt;p&gt;Many sources present the Euler, Verlet, velocity Verlet, and leapfrog algorithms for integrating Newton's equations.  Based on the order of accuracy, it is agreed that velocity Verlet, Verlet, and leapfrog are superior to the Euler method.  It can also be shown that the Verlet and velocity methods are algebraically equivalent.  What are the advantages and disadvantages of using velocity Verlet or leapfrog integrators?  Is one more accurate or expensive than the other?&lt;/p&gt;&#xA;" OwnerUserId="20722" OwnerDisplayName="Ian" LastEditorUserId="1804" LastEditDate="2016-06-16T17:48:35.530" LastActivityDate="2016-06-19T22:21:30.930" Title="Comparison of velocity Verlet and leapfrog algorithms" Tags="|ode|time-integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24236" PostTypeId="1" AcceptedAnswerId="24248" CreationDate="2016-06-17T03:20:03.193" Score="0" ViewCount="191" Body="&lt;p&gt;I am trying to run a pbs file which runs a script on a cluster. The script prompts for a user input. I wish to write this user input in my pbs file, what is a good way to do this?&lt;/p&gt;&#xA;" OwnerUserId="20720" LastActivityDate="2016-06-18T13:33:38.703" Title="Running a script that asks for user input from a pbs file" Tags="|hpc|pbs|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24237" PostTypeId="1" AcceptedAnswerId="24238" CreationDate="2016-06-17T03:27:47.510" Score="0" ViewCount="294" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Let $A\in\mathbb{R}^{n\times n}$ is symmetric positive definite and consider solving linear system $Ax = b$. Show that the symmetric Gauss-Seidel iteration converges for any $x_0$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Solution - Since $A$ is symmetric positive definite and $A = D - L - U$ is the usual partitioning into diagonal, strict lower and strict upper triangular elements, we have $$P_{sgs} = (D - L)D^{-1}(D - U) = CC^T$$ where $C$ is nonsingular lower triangular with positive diagonal elements and it follows that $P_{sgs}$ is symmetric positive definite. Since $A$ and $P$ are both symmetric positive definite, then we can use the result that if the symmetric matrix $2P - A$ is positive definite then the iteration converges. Computing $2P - A$ yields \begin{align*}&#xA;2\left[ (D-L)D^{-1}(D - U)\right] - A&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before I continue my professors solution has $U = L^T$ then after pushing around matrices $2P - A = A + 2LD^{-1}L^T$ in which we concludes that since this is the sum of two symmetric positive definite matrices and is therefore also symmetric positive definite. Therefore, Symmetric Gauss-Seidel iteration converges for any $x_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not intuitive to me why he lets $U = L^T$ and the algebra or shifting of matrices that yield the solution does not make any sense to me at all. Any suggestions is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastActivityDate="2016-06-17T04:00:38.727" Title="Show the symmetric Gauss-Seidel converges for any $x_0$" Tags="|linear-algebra|numerical-analysis|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24238" PostTypeId="2" ParentId="24237" CreationDate="2016-06-17T04:00:38.727" Score="2" Body="&lt;p&gt;$U=L^{T}$ because the original matrix $A$ is symmetric.  You should be able to do the algebraic simplification without too much trouble.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-06-17T04:00:38.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24239" PostTypeId="1" AcceptedAnswerId="26266" CreationDate="2016-06-17T08:56:09.830" Score="0" ViewCount="153" Body="&lt;p&gt;I have a large scale system with the following equations of motions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M\ddot z(t)+K z(t)=F(t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M$ is the mass matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$K$ is the stiffness matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I want to solve the equation for displacement $z(t)$, the eigenvalues and the eigenvectors with load dependent Ritz Analysis ( the algorithm is described succinctly in &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.10.7257&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FhtF3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FhtF3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that I can use &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;APRACK to for large scale eigenvalue problems&lt;/a&gt;, but what about load dependent Ritz Analysis? Is there already a solver ( commercial, open source, all doesn't matter) out there that does this kind of thing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to be clear, I am looking for a programming library that I can use, not a commercial software that I can't call programmatically .&lt;/p&gt;&#xA;" OwnerUserId="690" LastEditorUserId="690" LastEditDate="2016-07-21T07:35:05.110" LastActivityDate="2017-02-24T10:36:22.513" Title="Existing solver for large scale load dependent Ritz vectors problem" Tags="|numerical-analysis|eigensystem|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24240" PostTypeId="2" ParentId="24239" CreationDate="2016-06-17T11:44:20.353" Score="0" Body="&lt;p&gt;The commercial program, SAP 2000, &lt;a href=&quot;http://www.comp-engineering.com/SAPManE.htm&quot; rel=&quot;nofollow&quot;&gt;http://www.comp-engineering.com/SAPManE.htm&lt;/a&gt; has an option for dynamic analysis with Ritz vectors. Take a look at page 74 in the Basic Analysis Reference Manual.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-06-17T11:44:20.353" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24241" PostTypeId="2" ParentId="24232" CreationDate="2016-06-17T15:03:02.853" Score="6" Body="&lt;p&gt;The shortest answer is that the Verlet / Leapfrog methods are symplectic and time reversible, and these are desirable numerical properties that reflect the physical reality of certain simulation problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sympletic means that when the methods are guaranteed to conserve total energy (more correctly, the Hamiltonian) in conservative simulation problems. Take for instance, the simulation of the earth spinning around the sun. Without an external dissipation of energy, the earth is guaranteed to remain indefinitely in orbit. But using Euler or RK4 to perform the simulation, for example, the earth would eventually leave orbit or crash into the sun, due to a small numerical drift in the total energy that builds up over a long period of simulation. Symplectic methods like Verlet / Leakfrog explicitly preserve the total energy, so the numerical solution is also guaranteed to keep the earth indefinitely in orbit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Time-reversibility means that the (fixed-step) method can take $k$ steps forward in time, followed by $k$ steps backwards in time, and arrive at the same initial conditions used to start the simulation. We often apply time-reversible numerical methods to time-reversible physical processes; we then expect the numerical solution to have long-time behaviour similar to that of the exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these two properties come at a sacrifice of accuracy and stability. The RK2 method is a comparable second-order method that is significantly more accurate, and the BDF2 is a second-order method that is significantly more stable. Also, these methods are explicit (i.e. efficient, because they can be computed without solving a nonlinear system of equations) only for certain physical processes; their most general form is implicit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hairer, Lubich, Wanner, &quot;Geometric Numerical Integration&quot;, Springer 2006&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT v2:&lt;/strong&gt; the Leapfrog and Verlet methods are (theoretically) identical. The following quote is from Pg. 7 of Hairer Lubich Wanner.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This basic method, or its equivalent formulation&#xA;  given below, is called the Stormer method in astronomy, the Verlet method in molecular dynamics, the leap-frog method in the context of partial differential equations, and it has further names in other areas.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2016-06-19T22:21:30.930" LastActivityDate="2016-06-19T22:21:30.930" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24242" PostTypeId="1" CreationDate="2016-06-17T15:09:17.570" Score="2" ViewCount="1385" Body="&lt;p&gt;I would like to solve 3D transient incompressible Navier-Stokes with FEM, Newton method, Schur-based preconditioner, Lagrangean P2/P1 elements (no stabilization), in a rigid pipe discretized with tetrahedrons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fluid is initially at rest, I impose constant inlet and outlet pressure boundary conditions (and therefore constant pressure gradient), and I would like to reach Poiseulle steady-state solution after a while. Reynolds number is within the laminar regime, and CFL holds! After couple hundred of timesteps, pressure becomes unstable and everything breaks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I impose pressure b.c. by adding -$\int_\Gamma p \eta N_i dx$ at the moment equation, where $\eta$ is the normal outward unitary vector, and $N_i$ is the $P_2$ basis function associated to the boundary nodes. Nothing out of the standard, I think.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I visualize the result of this integral in Paraview, the boundary looks like a checkerboard: there are nonzero values at the d.o.f. associated to the edges, but nearly ZERO values at the d.o.f associated to the vertices! The explanation I used to convince myself is that the shape functions associated to vertices have zero integral when they are restricted to triangles (actually, unless I have made a bizarre mistake, this also holds for $P_2$ shape functions associated to the vertices of the triangles have zero integral). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.: the velocity profile around the last successful time-steps before stoping has also a kind of chessboard profile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Few issues puzzle me:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) everything works fine if I keep the same pressure gradient, mesh, Reynolds number etc. but impose parabolic velocity profile (given by Poiseulle formulae), or constant velocity (and then I have the Poiseulle profile after the entrance length).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) the code which evaluates the pressure integral above is the exactly the same used by another (solid mechanics) solver, validated with inhomogeneous &amp;amp; transient traction boundary conditions, which uses P1 elements instead!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) if I refine the mesh, I still have the checkerboard pattern, but the pressure imposition setup somehow works! I did not expect mesh dependency.  To be clear, this checkerboard pattern is not the pressure, but the result of the integral above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My quite vague question is: what am I missing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if the problem statement is clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw., do you know any other way to impose pressure? I've already heard about introducing 1's at the (nonexistent) boundary pressure entries of the continuity equation ($\nabla \cdot v = 0$), but I am uncomfortable with this idea, if I don't use any stabilization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hint is highly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="10470" LastEditorUserId="10470" LastEditDate="2016-06-18T08:21:29.780" LastActivityDate="2017-05-08T07:19:16.207" Title="Pressure boundary condition in Navier-Stokes equations" Tags="|finite-element|boundary-conditions|navier-stokes|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24243" PostTypeId="1" CreationDate="2016-06-17T16:33:22.780" Score="2" ViewCount="938" Body="&lt;p&gt;I know that when modeling incompressible flow in a pipe, it is common practice that when you have a velocity inlet, there should be a zero pressure gradient at the inlet as well.  Also, when you have a pressure outlet, you should set the velocity gradient to zero at the outlet.  But when you have both a pressure inlet and pressure outlet, should one set the velocity gradients at both inlet and outlet to zero?  Would this lead to a stable solution for pipe flow?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2016-06-20T18:23:01.343" Title="How should I set the velocity conditions when you have a pressure inlet and pressure outlet?" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24244" PostTypeId="2" ParentId="24243" CreationDate="2016-06-17T19:55:04.263" Score="0" Body="&lt;p&gt;Which method are you using? FEM, FVM, FDM, other? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In FEM, at least, when you impose velocity inlet, you leave pressure field (not the pressure gradient) untouched.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the pressure outlet, if you only set pressure, you are automatically assuming that the normal component of the velocity gradient at the outlet is zero [1], which is not wrong if the pipe is straight and if your pressure is &quot;well-behaved&quot; (e.g., no backflow, recirculation - otherwise you need extra terms in your variational formulation to handle it [2]).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you mean by &quot;stable solution&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &quot;Efficient Solvers for Incompressible Flow Problems&quot; Turek (1999) has a very interesting discussion about hidden boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] e.g. &quot;A comparison of outlet boundary treatments for prevention&#xA;of backflow divergence with relevance to blood flow simulations&quot; Moghadam et al (2011).&lt;/p&gt;&#xA;" OwnerUserId="10470" LastActivityDate="2016-06-17T19:55:04.263" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24245" PostTypeId="2" ParentId="6701" CreationDate="2016-06-18T01:30:02.397" Score="0" Body="&lt;p&gt;Yes as @wolfgang_bangerth says in his answer, you simply replace division by multiplication of matrix inverse and times by matrix multiplication. Things get interesting when the blocks become comparable or larger than available memory in which case the block operations can be parallelized by ScaLapack or other distributed linear algebra LU decomposition. For serial solvers, individual LU decompositions of the blocks can be written to disk to during forward phase of the solve to create an out-of-core solver that get's around memory limitations - at least for a some scales. Even more interesting, the tri-diagonal solver can be done parallelized itself with a block-cyclic technique while the block operations can or can not be parallelized and this can result in some truly fast solvers - see this &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=2&amp;amp;ved=0ahUKEwjd2aqes7DNAhXC5iYKHcl9BGMQFggmMAE&amp;amp;url=http%3A%2F%2Fdspace.mit.edu%2Fbitstream%2Fhandle%2F1721.1%2F94353%2F10ja009_full.pdf%3Fsequence%3D1&amp;amp;usg=AFQjCNGh9T1fyv9Pe3041KeTSCaDXr8g1w&amp;amp;sig2=s_37iXXdyQmZdcaqv14iog&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="3450" LastActivityDate="2016-06-18T01:30:02.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24246" PostTypeId="1" CreationDate="2016-06-18T09:13:46.577" Score="1" ViewCount="64" Body="&lt;p&gt;I am simulating in COMSOL a system of 3 coupled PDEs (parabolic &amp;amp; elliptic) along with 10 stiff ODEs. In order to have the system working, I am downsizing the time step size too much to achieve convergence. I am lucky enough that I am doing 1D simulations which don't take too much time, but this drives me to think about more complex 3D cases in which the distretized system will result in hugh DOF to be solved, with the very low step size this takes too much time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So, is there computationally efficient method to solve such system (e.g. downsizing step size for ODEs but having larger ones for PDEs. Just imagining)&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="20025" LastActivityDate="2016-06-18T09:13:46.577" Title="Stiff ODEs coupled with PDEs (computational efficiency)" Tags="|pde|efficiency|stiffness|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24247" PostTypeId="1" CreationDate="2016-06-18T13:23:37.690" Score="0" ViewCount="912" Body="&lt;p&gt;I have simple code, which flags nodes with in region enclosed by cylinder. On implementing the code, the result is mild tilt of the cylinder observed case with $\theta=90^{\circ}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm for checking any point inside arbitrarily oriented cylinder is as follows. Let $\vec{r}$ be the vector joining center $\vec{c}$ and arbitrary point $\vec{o}$, and $$\vec{r}=\vec{c}-\vec{o}.$$ For orientation vector $\vec{o}$, the projection of $\vec{r}$ on it is,&#xA;$$ u = \vec{r}\cdot\vec{o}.$$&#xA;Therefore, the perpendicular vector is,&#xA;$$\vec{p}=\vec{r}-u\cdot\vec{o}.$$&#xA;For a cylinder of length $2l$ and radius $a$, check&#xA;$$\vec{p}.\vec{p}&amp;lt;a^2  \hspace{1cm} \text{for } -l\leq u \leq l. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The actual issue:&lt;/strong&gt;&#xA;The above algorithm is implemented in Fortran. The code checks for points in Cartesian grid if inside the cylinder. Following being the test case:&#xA;The cylinder makes an angle $\theta=90^{\circ}$ in the yz-plane with respect to y-axis. Therefore, the orientation vector $\vec{o}$ is (0, 1, 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 1:&lt;/strong&gt;&#xA;Orientation vector is assigned directly with $\vec{o}=(0.0,1.0,0.0)$. This results in perfect cylinder with $\theta=90^{\circ}.$&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9yQkc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9yQkc.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 2:&lt;/strong&gt;&#xA;Orientation vector is specified with intrinsic Fortran functions with double precision accuracy &lt;code&gt;dsin&lt;/code&gt; and &lt;code&gt;dcos&lt;/code&gt; with $\vec{o}=(0.0, \sin(\pi/2.0), \cos(\pi/2.0))$ with $\pi$ value assigned with more than 20 significant decimal points. The resulting cylinder results in mild tilt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lwlUs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lwlUs.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&#xA;The highlighted region indicates the extra material due to tilt of cylinder with respect to Cartesian axes. I also tried &lt;a href=&quot;https://stackoverflow.com/questions/2157920/why-define-pi-4atan1&quot;&gt;architecture specific maximum precision &quot;pi&quot; value.&lt;/a&gt; This also results in same problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shows like the actual angle made by cylinder is not $90^\circ$. Can anyone suggest valid solution for this problem. I need to use the inbuilt trigonometric functions for arbitrary angles and looking for accurate cell flagging method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: All operations are performed with double precision accuracy.&lt;/p&gt;&#xA;" OwnerUserId="20723" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-06-18T23:38:55.573" Title="Fortran round-off error with floating point operations" Tags="|computational-geometry|fortran|floating-point|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24248" PostTypeId="2" ParentId="24236" CreationDate="2016-06-18T13:33:38.703" Score="0" Body="&lt;p&gt;You might check out the concept of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Here_document&quot; rel=&quot;nofollow&quot;&gt;&quot;here document&quot;&lt;/a&gt; assuming your shell is bash. That way you can write, in your batch (pbs, lsf, sge, slurm, or whatever) script, the text of what you'd like to respond to the prompt with, capture it in your batch script for reproducibility purposes, and format it more or less how you like (based on the constraints of the prompting program). &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-06-18T13:33:38.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24249" PostTypeId="1" CreationDate="2016-06-18T17:22:42.430" Score="1" ViewCount="105" Body="&lt;p&gt;I am engaged with the syntheses and DFT calculations of coordination complexes. I wonder how can I set up the basis set for carried out using B3LYP method and mixed basis sets of LanL2DZ for Pd and 6-31+G(d) for all other atoms, in the use of DFT theory by Gaussian 09W. can any one please explain for me?  &lt;/p&gt;&#xA;" OwnerUserId="20725" LastEditorUserId="9667" LastEditDate="2016-06-19T03:26:39.560" LastActivityDate="2016-06-19T03:26:39.560" Title="Choose of basis set" Tags="|optimization|density-functional-theory|" AnswerCount="0" CommentCount="4" ClosedDate="2016-07-01T14:52:37.387" ContentLicense="CC BY-SA 3.0" />
  <row Id="24250" PostTypeId="2" ParentId="24247" CreationDate="2016-06-18T23:38:55.573" Score="2" Body="&lt;p&gt;It would be really nice to look at the code, compiler you are using and compiler options. However, I can suggest couple of things to test right away:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of specifying 0.0 and 2.0 as in your text, use 0.0_dp and 2.0_dp, whenever you use any constants. That will make sure that double precision will be used for them. (dp=kind(0.0d0)) Moreover, I would say that multiplication by 0.5_dp is preferred to division by 2.0_dp.&#xA;Say, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tmp_sin=dsin(pid*0.5_dp)&#xA;tmp_cos=dcos(pid*0.5_dp)&#xA;o=/0.0_dp,tmp_sin,tmp_cos/&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another thing to try, would be to create a double-precision variable pid_over_two that will have a value of pi/2 with &gt;20 significant digits and try your code (with your compiler settings) on that.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Make sure that you &lt;strong&gt;do not&lt;/strong&gt; compile this code with -no-prec-div option that allows for non-IEEE standard division.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Make sure that when you output the data for visualisation, you are using the correct format specifiers&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These are the basic advices I can suggest without looking inside of the code.&#xA;Also, it would be valuable to see, what output does your code (under your compiler/platform/settings) produce for:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dsin(pid/2.0)&#xA;dsin(pid/2.0_dp)&#xA;dsin(0.5_dp*pid)&#xA;dcos(pid/2.0)&#xA;dcos(pid/2.0_dp)&#xA;dcos(0.5_dp*pid)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2016-06-18T23:38:55.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24251" PostTypeId="2" ParentId="24243" CreationDate="2016-06-19T04:08:19.857" Score="2" Body="&lt;p&gt;There are a few things to consider.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I would suggest focusing on which BCs are most appropriate for the velocity since pressure is just a constraint and its BCs will follow.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the physical situation you are trying to model? Here are &lt;em&gt;some&lt;/em&gt; BC scenarios for duct mean pressure gradient-driven flows along the $x$ direction:&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A) Very long duct - Fully periodic BCs with many axial cells&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ u_{ghost,inlet}  = u_{interior,outlet}, \quad u_{ghost,outlet} = u_{interior,inlet}, \\  &#xA;  p_{ghost,inlet}  = p_{interior,outlet}, \quad p_{ghost,outlet} = p_{interior,inlet} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This models a very long duct. The flow, laminar or turbulent, leaves the exit and re-enters in the entrance. The only condition on this BC is that the duct must be longer (usually a few times longer) than the largest length scale of the flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;B) Fully developed duct - several BC options for velocity with 1 axial cell&lt;/strong&gt;&#xA;A fully developed duct with 1 axial cell can be simulated using several implementations. Consider the following cases&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{B.1)} \qquad u_{ghost,inlet}  = u_{interior,outlet}, \qquad u_{ghost,outlet} = u_{interior,inlet} \\&#xA;\qquad \qquad p_{ghost,in}  = p_{interior,outlet}, \qquad p_{ghost,outlet} = p_{interior,inlet} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{B.2)} \qquad  \left. \frac{\partial u}{\partial x} \right|_{inlet}  = 0, \qquad \left. \frac{\partial u}{\partial x} \right|_{outlet} = 0 \qquad \left. \frac{\partial p}{\partial x} \right|_{inlet} = 0, \qquad \left. \frac{\partial p}{\partial x} \right|_{inlet} = 0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This models a fully developed duct. Both BCs enforce $\partial_x () = 0$ for all variables except pressure, which varies linearly. The mean pressure gradient enters the momentum equation as a source term. Why are the two BCs equivalent? Because there's only one interior axial cell for ghost points to reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;C) Developing duct - Dirichlet inlet/Neumann outlet velocity BCs with many axial cells&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ u(inlet) = u_{inlet}, \qquad \left. \frac{\partial u}{\partial x} \right|_{outlet} = 0, \qquad \left. \frac{\partial p}{\partial x} \right|_{inlet} = 0, \qquad p_{outlet} = 0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This models a developing duct. The outlet is fully developed, and the length of the duct must be longer than the &lt;a href=&quot;https://en.wikipedia.org/wiki/Entrance_length&quot; rel=&quot;nofollow&quot;&gt;development length&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;D) The duct flow you propose - Neumann inlet/Neumann outlet velocity BCs with many axial cells&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \left. \frac{\partial u}{\partial x} \right|_{inlet} = 0, \qquad \left. \frac{\partial u}{\partial x} \right|_{outlet} = 0, \qquad p(inlet) = p_{inlet}, \qquad p(outlet) = p_{outlet} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does this model? The answer is a bit subtle. If the flow is laminar, we expect the same results as in case B) and unchanging along $x$. If the flow is turbulent however, then we have a problem? The inlet BC appears to come from the outlet of a fully developed duct flow, the outlet appears to head towards a fully developed duct flow, and the interior is turbulent. But how could this be? A laminar fully developed duct flow is not likely to become turbulent and return to a laminar state with no intermediate damping mechanisms. Unless there are additional mechanisms that result in the laminarization of the flow after the flow has reached a turbulent state (far from the fully developed inlet), this model is not physically meaningful.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;As far as the convergence of the BCs you suggest. You may see convergence if you're in a laminar or, maybe, a transition regime, however, there is no guarantee that you'll see convergence in a turbulent regime for any Reynolds number.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="8174" LastEditDate="2016-06-20T18:23:01.343" LastActivityDate="2016-06-20T18:23:01.343" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24252" PostTypeId="1" CreationDate="2016-06-19T06:23:45.230" Score="1" ViewCount="1961" Body="&lt;p&gt;This is my first question.  I have the following &lt;a href=&quot;https://gist.github.com/user20730/95ed4f414c55eda87f176ba195d7c9f5&quot; rel=&quot;nofollow noreferrer&quot;&gt;data&lt;/a&gt; that I'd like to approximate as a parametric function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;y = a + (bx_1 + cx_2 + dx_3 + ex_1x_2 + fx_1x_3 + gx_2x_3 + hx_1x_2x_3 + i)*(j*\sin(kx_1x_2x_3) + l\cos(mx_1x_2x_3))&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;testdata &amp;lt;- read.csv('..path_to_file/gistfile1.txt', sep = &quot;&quot;)&#xA;plot(1:35,testdata$X0, col = 'blue', pch = 19, ylim = c(0,8),type = &quot;l&quot;,lwd = 2, xlab = &quot;x&quot;, ylab = &quot;y_i&quot;,cex.lab = 2)&#xA;lines(1:35,testdata$X1, col = 'red', pch = 19, ylim = c(0,1),lwd = 2)&#xA;lines(1:35,testdata$X1.1, col = 'green', pch = 19, ylim = c(0,1),lwd = 2)&#xA;lines(1:35,testdata$X8, col = 'magenta', pch = 19, ylim = c(0,1),lwd = 2)&#xA;leg.txt &amp;lt;- (c(&quot;y&quot;,&quot;x1&quot;,&quot;x2&quot;,&quot;x3&quot;))&#xA;legend(18,8,leg.txt, col  =  c('blue','red','green','magenta'), lty = 1,lwd= 2,border = 'white')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;: &lt;a href=&quot;https://i.stack.imgur.com/ENmqj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ENmqj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D48Dp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D48Dp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions how to modify the function shown below so as to get a closer fit?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NpTpa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NpTpa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20730" LastEditorUserId="19501" LastEditDate="2016-08-14T20:03:13.890" LastActivityDate="2016-08-14T20:03:13.890" Title="Curve fitting for oscillating data" Tags="|regression|curve-fitting|modeling|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="24253" PostTypeId="1" AcceptedAnswerId="24255" CreationDate="2016-06-19T07:46:47.720" Score="0" ViewCount="183" Body="&lt;p&gt;NVIDIA's Tesla GPUs for Servers includes market segments called &lt;code&gt;&quot;Hyperscale HPC&quot;&lt;/code&gt; and &lt;code&gt;&quot;Strong-Scale HPC&quot;&lt;/code&gt;. How different is &lt;code&gt;&quot;Strong-Scale&quot;&lt;/code&gt;&lt;em&gt;ing&lt;/em&gt;  from  &lt;code&gt;&quot;Hyperscale&quot;&lt;/code&gt;&lt;em&gt;ing&lt;/em&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2016-06-23T15:18:08.270" Title="Hyperscale Vs Strong-scale HPC" Tags="|gpu|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24255" PostTypeId="2" ParentId="24253" CreationDate="2016-06-19T14:11:34.487" Score="4" Body="&lt;p&gt;Those are both marketing words. Either ask NVIDIA what they mean by them, or ignore them. NVIDIA appears to be using them to classify different product lines, at either different cost levels or different levels of parallelism. You can almost certainly ignore these marketing terms safely. They have no definite meaning to me.  &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="16799" LastEditDate="2016-06-23T15:18:08.270" LastActivityDate="2016-06-23T15:18:08.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24256" PostTypeId="1" AcceptedAnswerId="24257" CreationDate="2016-06-21T00:28:58.777" Score="4" ViewCount="604" Body="&lt;p&gt;I coded a finite element method with linear basis elements for the problem $$-u'' = f(x), x\in[0,1], u(0) = u(1) = 0$$ The nodes are uniformly spaced and I will denote them as $x_i$. I initially calculated the error of my method as $$\text{error} = \max_i |u(x_i) - \hat{u}(x_i)|$$&#xA;where $\hat{u}$ denotes the FEM solution. I kept getting approximately 0 as the error, and I'm thinking that's because the FEM solution interpolates the exact solution $u$ at the nodes $x_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then went back and got an approximation for $$\max_{x\in[0,1]} |u(x) - \hat{u}(x)|$$ and I got second-order convergence (as expected).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is does the FEM (with linear/quadratic/cubic piecewise basis functions) always give back an interpolation of the exact solution, and so should I expect an error of zero at the nodes?&lt;/p&gt;&#xA;" OwnerUserId="7322" LastEditorUserId="1804" LastEditDate="2016-06-21T16:21:28.970" LastActivityDate="2016-06-21T16:21:28.970" Title="Finite element error for second order ODE at nodes equal to zero" Tags="|finite-element|convergence|elliptic-pde|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24257" PostTypeId="2" ParentId="24256" CreationDate="2016-06-21T00:32:08.363" Score="3" Body="&lt;p&gt;For this simple elliptic PDE, the finite-element approximation is indeed exact at the nodes (i.e., coincides with the true solution); this is usually called &lt;em&gt;superconvergence (at the nodes)&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it's such a nice homework problem, instead of the (simple) proof, here's a hint: Use the weak formulation with a hat function centered on $x_i$ as a test function, Galerkin orthogonality, and partial integration.&lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="1804" LastEditDate="2016-06-21T13:53:42.967" LastActivityDate="2016-06-21T13:53:42.967" CommentCount="9" CommunityOwnedDate="2016-06-21T14:00:38.473" ContentLicense="CC BY-SA 3.0" />
  <row Id="24258" PostTypeId="1" CreationDate="2016-06-21T03:07:30.343" Score="0" ViewCount="43" Body="&lt;p&gt;The simple average is commonly used to combine the predictions out of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive models to get more accurate predictions? &lt;/p&gt;&#xA;" OwnerUserId="17524" LastActivityDate="2016-06-21T03:38:38.540" Title="How are the outcomes that generated from different predictive models combined to get more accurate predictions?" Tags="|machine-learning|statistics|data-analysis|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24259" PostTypeId="2" ParentId="24258" CreationDate="2016-06-21T03:30:59.190" Score="2" Body="&lt;p&gt;Bagging, Boosting, and Bayesian Model Averaging/Combination are all widely used techniques for doing this.  These are discussed in many textbooks on machine learning.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-06-21T03:30:59.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24260" PostTypeId="2" ParentId="24258" CreationDate="2016-06-21T03:38:38.540" Score="1" Body="&lt;p&gt;Provided you have raw data you could use in this process, one could use the various different models and treat them as basis functions of sorts that you wish to merge together in a least square sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could then merge the various models using a least square fit based on whatever data you have at your disposal. This is certainly different than simple averaging.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-06-21T03:38:38.540" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24262" PostTypeId="1" CreationDate="2016-06-21T09:46:35.033" Score="0" ViewCount="1961" Body="&lt;p&gt;I am doing a project on the finite element method. I want to know how to form the mass matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please point me out to the resources on the finite element method, where the procedure of assembling a mass matrix is described in detail.&lt;/p&gt;&#xA;" OwnerUserId="20751" LastEditorUserId="20688" LastEditDate="2019-06-22T20:45:59.677" LastActivityDate="2019-06-22T20:45:59.677" Title="How is the mass matrix formed in finite element methods?" Tags="|finite-element|reference-request|" AnswerCount="1" CommentCount="1" ClosedDate="2016-06-27T15:50:04.637" ContentLicense="CC BY-SA 4.0" />
  <row Id="24264" PostTypeId="1" CreationDate="2016-06-21T14:22:56.927" Score="2" ViewCount="126" Body="&lt;p&gt;I am interested in the numerical solution of the following system of non-linear partial-differential algebraic equations, where the independent variables are $X$ and $T$, representing non-dimensional space and time, respectively. The fields are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f = f(X,T)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ g = g(T)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \xi = \xi(T)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \psi = \psi(T)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system of 5 equations can be defined by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;\frac{\partial f}{\partial X} = \frac{\partial F}{\partial T}-AH\left( I \psi + Jf \right) + B \frac{\partial f^2}{\partial T^2} - K q ~~~~~~~~~~(1)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial g}{\partial T} = q ~~~~~~~~~~(2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial q}{\partial T} = C \left( f-g \right) + L \left( 2 D q^2 - E \left|q \right| q \right) - G H \left( I \xi + Jq \right) ~~~~~~~~~~(3)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;\frac{\partial \xi}{\partial T} = I \xi + Jq ~~~~~~~~~~(4) &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;\frac{\partial \psi}{\partial T} = I \psi + Jf ~~~~~~~~~~(5)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $F$ is the Burgers conservation form, defined by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F = f^2/2 &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $A-E,G-L$ are constants and $q=q(T)$ is introduced to reduce the system to first order equations only, excluding equation (1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested if anyone has suggestions for how to approach numerically solving the above system of equations in a fully coupled manner?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a note, I have already managed to solve the above system, but using a de-coupled approach, i.e: solve equations (2) - (4) using a standard ODE solver with an initial condition for $f = f(0,T)$ and $g = g(0)$. The newly calculated value of $q$ can then be substituted into equation (1) to find $f$ for the next space-step. This is repeated until the final space-step is reached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="20752" LastActivityDate="2016-07-05T10:02:59.370" Title="numerical approach for system of non-linear partial-ordinary differential equations" Tags="|nonlinear-equations|hyperbolic-pde|method-of-lines|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24265" PostTypeId="2" ParentId="24262" CreationDate="2016-06-21T16:45:15.380" Score="1" Body="&lt;p&gt;Kind of a broad question. You can clarify what exactly you need, but basically the stiffness matrix comes from strain energy (stress times strain) and the mass matrix comes from kinetic energy (mass times velocity squared). Assembly from the elemental to global mass matrix is the same as the stiffness matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chapter 16 will be a good start.&#xA;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/MFEMD.d/Home.html&quot; rel=&quot;nofollow&quot;&gt;http://www.colorado.edu/engineering/CAS/courses.d/MFEMD.d/Home.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16541" LastActivityDate="2016-06-21T16:45:15.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24266" PostTypeId="2" ParentId="24264" CreationDate="2016-06-21T17:04:58.777" Score="1" Body="&lt;p&gt;If there are only &lt;strong&gt;continuous&lt;/strong&gt; solutions in your problem, the finite difference method is enough. However, if there exists shock, the finite difference/finite volume WENO/ENO suit for your purpose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, you should first discretize the systems in space and make it to a systems of ODEs. Then, use Runge-Kutta methods to evolve in time.&lt;/p&gt;&#xA;" OwnerUserId="20711" LastActivityDate="2016-06-21T17:04:58.777" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24267" PostTypeId="1" CreationDate="2016-06-21T17:43:56.313" Score="2" ViewCount="90" Body="&lt;p&gt;So I have a collection of systems of equations, basically $n$ systems of equations, each composed of $k$ equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{a_1x_{1j}}{a_1x_{1j} + \cdots + a_kx_{kj}} + \log x_{1j} + 1 - B_{1j} = 0$$&#xA;$$\frac{a_2x_{2j}}{a_1x_{1j} + \cdots + a_kx_{kj}} + \log x_{2j} + 1 - B_{2j} = 0$$&#xA;$$\vdots$$&#xA;$$\frac{a_kx_{kj}}{a_1x_{1j} + \cdots + a_kx_{kj}} + \log x_{kj} + 1 - B_{kj} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $j = 1,...,n$, with the restriction that each $(x_{1j},...,x_{kj})$ lies on the simplex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see the systems are all very similar to each other, the only difference being the $B_{ij}$ constant at the end.  Also if it helps all the $a_i$ and $B_{ij}$ are positive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering first if these systems are actually simple enough that they have analytic solutions, and if not, then upon finding the solution to one of them (probably by using some sort of constraint-modified Newton's method), whether I can basically use this solution to immediately find the solution to the others.&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="6849" LastEditDate="2016-06-21T18:18:16.420" LastActivityDate="2016-06-22T05:02:51.007" Title="Finding quick solution to a collection of systems of fairly simple but nonlinear equations" Tags="|optimization|nonlinear-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24268" PostTypeId="1" AcceptedAnswerId="24273" CreationDate="2016-06-21T23:24:29.600" Score="1" ViewCount="581" Body="&lt;p&gt;I'm attempting to implement velocity Verlet with a harmonic oscillator in C, but I have some errors:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Energy is not being conserved&lt;/li&gt;&#xA;&lt;li&gt;Energy oscillates much more than I would expect&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I believe the error has to do with my position function or definitions, as the PE is blowing up much faster than the KE is going down, but I have not been able to determine the error. It's also possible that the KE is not going down as fast as it should, I suppose.&#xA;I guess it should be mentioned that I'm new to programming in C.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.H&amp;gt;&#xA;#include &amp;lt;math.H&amp;gt;&#xA;&#xA;double position(double x0, double omega, double v0, double t)&#xA;{&#xA;    return x0 * cos( omega * t) + (v0/omega) * sin( omega * t);&#xA;}&#xA;&#xA;double velocity(double x0, double omega, double v0, double t)&#xA;{&#xA;    return -x0 * omega * sin(omega * t) + v0 * cos( omega * t);&#xA;}&#xA;&#xA;double acceleration(double x0, double omega, double v0, double t)&#xA;{&#xA;    return -x0 * omega*omega * cos( omega * t) - v0 * omega * sin( omega * t);&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    double x0, v0, ao, x, xu, v, a, pe, ke, te, ft, t, m , dt, k, omega;&#xA;    t=0;&#xA;    ft = 10.;&#xA;    dt = .01;&#xA;    x0 = 0;&#xA;    v0 = 1;&#xA;    m = 1;&#xA;    k= 1;&#xA;    omega = .1;&#xA;&#xA;    for(t=0;t&amp;lt;=ft;t+=dt)&#xA;    {&#xA;&#xA;        if(t==0)&#xA;        {&#xA;            x = position(x0, omega, v0,t);&#xA;            v = velocity (x0, omega, v0, t);&#xA;            a = acceleration(x0, omega, v0, t);&#xA;        }&#xA;&#xA;        else &#xA;        {&#xA;            a = (-k * x)/m;&#xA;            v = v + (0.5*a*dt);&#xA;            x = x + (v * dt);&#xA;            ao = acceleration(x, omega, v, t);&#xA;            v = v + (0.5*ao*dt);&#xA;        }&#xA;&#xA;        ke= 0.5 * m * v * v;&#xA;        pe = 0.5 * k * x * x;&#xA;        te = pe + ke;&#xA;        //printf(&quot;position at time %f = %f\n&quot;,t,x);&#xA;        //printf(&quot;velocity at time %f = %f\n&quot;,t,v);&#xA;        //printf(&quot;acceleration at time %f = %f\n&quot;,t,a);*/&#xA;        printf(&quot;energy at time %f = K: %f; V: %f; Total: %f\n\n&quot;,t,ke,pe,te);&#xA;&#xA;    }&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20757" LastEditorUserId="9667" LastEditDate="2016-06-22T19:15:48.917" LastActivityDate="2016-06-22T19:15:48.917" Title="Implementing velocity verlet for harmonic oscillator in C gives error wrt conservation of energy" Tags="|computational-physics|c|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24269" PostTypeId="2" ParentId="24267" CreationDate="2016-06-22T00:47:14.197" Score="1" Body="&lt;p&gt;Notice that, in your problem, these systems are decoupled with each other. To be more precise, the $j$-th system only contains variables $x_{1j},\dots,x_{kj}$. Therefore, you only need to solve each system separately using Newton's method (or using some build-in functions in MATLAB or python).&lt;/p&gt;&#xA;" OwnerUserId="20711" LastActivityDate="2016-06-22T00:47:14.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24270" PostTypeId="2" ParentId="24267" CreationDate="2016-06-22T05:02:51.007" Score="0" Body="&lt;p&gt;Not sure if this would totally solve your problem, but it might be a step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So lets take your $j^{th}$ system and just frame it as the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \textbf{F}(\textbf{x}_j^{*}) - \textbf{B}_j = \textbf{0} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given you find the solution to that, let's look at the $i^{th}$ system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \textbf{F}(\textbf{x}_i^{*}) - \textbf{B}_i = \textbf{0} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's try to relate the $i^{th}$ system to the $j^{th}$ system like so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \textbf{F}(\textbf{x}_i^{*}) - \textbf{B}_i = \textbf{0} $$&#xA;$$ \textbf{F}(\textbf{x}_j^{*} + \Delta \textbf{x}) - (\textbf{B}_j + \Delta_{ij})= \textbf{0} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left(\textbf{F}(\textbf{x}_j^{*}) - \textbf{B}_j\right) + \left( \left.\frac{\partial \textbf{F}}{\partial\textbf{x}}\right|_{\textbf{x}_{j}^{*}}\Delta \textbf{x} - \Delta_{ij} \right)= \textbf{0} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The left term goes to zero based on the solution to the $j^{th}$ system. You end up with the equation:&#xA;$$ \Delta \textbf{x} = \left[\left.\frac{\partial \textbf{F}}{\partial\textbf{x}}\right|_{\textbf{x}_{j}^{*}}\right]^{-1} \Delta_{ij}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By solving this equation above, you &lt;em&gt;might&lt;/em&gt; be able to quickly get to the solution for the $i^{th}$ equation using knowledge of the $j^{th}$. What's cool about this is you only need to potentially solve the $j^{th}$ equation the hard way and then you could create a factored version of $\left[\left.\frac{\partial \textbf{F}}{\partial\textbf{x}}\right|_{\textbf{x}_{j}^{*}}\right]$ to make the solution of the above equation as efficient as possible for the remaining systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing to note, however, is you will need to experiment with this since I am unsure how sensitive the solution will be to small changes in the $\textbf{B}$ vector. The above solution should primarily work if the sensitivity to differences in $\textbf{B}$ are small.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-06-22T05:02:51.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24271" PostTypeId="1" CreationDate="2016-06-22T08:16:59.857" Score="2" ViewCount="1343" Body="&lt;p&gt;How to define a variable which is an integral involving cauchy principal value inside in any computer programming language? I want to know how to break down the procedure step by step from a computational science viewpoint. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the integral I encounter:&#xA;$S(\omega') = \int_0^\infty d \omega \frac{1}{\omega} \left( \mathcal{P} \left( \frac{1%&#xA;}{\omega' - \omega} \right) + \mathcal{P} \left( &#xA;\frac{1}{\omega' + \omega} \right) \right) \ $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LHS is a function of &lt;code&gt;w'&lt;/code&gt;. The RHS is a function of &lt;code&gt;w'&lt;/code&gt;. For example, &lt;code&gt;x(t) = 2t+1&lt;/code&gt; defines the variable &lt;code&gt;x&lt;/code&gt; in terms of &lt;code&gt;t&lt;/code&gt;. &lt;code&gt;t = 1&lt;/code&gt; means &lt;code&gt;x = 3&lt;/code&gt;. &#xA;For example in Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t = 1;&#xA;x = 2t + 1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I want to define the variable &lt;code&gt;S&lt;/code&gt; in terms of &lt;code&gt;w'&lt;/code&gt;. For example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w' = 1;&#xA;S = 2w' + 1; %&amp;lt;----(*) But what is S in terms of w' in my question?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then &lt;code&gt;S = 3&lt;/code&gt; here. But it is not such a simple expression here. I need the expression of &lt;code&gt;S&lt;/code&gt; or set up the integral on the R.H.S so I can get the expression of &lt;code&gt;S&lt;/code&gt; in terms of &lt;code&gt;w'&lt;/code&gt; like in line (*). For example in Matlab, is it needed to do it using &lt;code&gt;sym&lt;/code&gt; (symbolic) or do I have to do it by fourier transform &lt;code&gt;fourier&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the R.H.S is a definite integral of &lt;code&gt;w&lt;/code&gt;. But since it will disappear, &lt;code&gt;w'&lt;/code&gt; is the only variable of &lt;code&gt;S&lt;/code&gt; here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or to cut off the question into parts of just defining Cauchy principal value. I browse in matlab documentation &lt;a href=&quot;http://www.mathworks.com/help/symbolic/int.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mathworks.com/help/symbolic/int.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;int(1/(x - 1), x, 0, 2, 'PrincipalValue', true)&lt;/code&gt; is what I encountered. But now I am not calculating improper integral. I want to define the Cauchy principal value of $$\frac{1}{x}$$ as in complex analysis and distribution. i.e. $$P\left(\frac{1}{x}\right) = \lim_{\varepsilon\rightarrow 0^+} \left[\int_a^{b-\varepsilon} 1/x\,\mathrm{d}x+\int_{b+\varepsilon}^c 1/x\,\mathrm{d}x\right]$$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related question:&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/19275657/python-find-principal-value-of-an-integral-numerically&quot;&gt;Sokhotski–Plemelj theorem&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20760" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2017-07-22T20:39:00.790" Title="Defining Cauchy principal value in MATLAB (or Scilab/Maple)?" Tags="|matlab|computational-physics|octave|maple|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24273" PostTypeId="2" ParentId="24268" CreationDate="2016-06-22T11:31:16.933" Score="3" Body="&lt;p&gt;Your second acceleration &lt;code&gt;ao&lt;/code&gt; in the &lt;code&gt;else&lt;/code&gt; statement does not make sense. Since it assumes that the calculated position and velocity are the initial conditions at $t=0$ and then you evaluate it at the current time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you change the expression for &lt;code&gt;ao&lt;/code&gt; to the same as you used for &lt;code&gt;a&lt;/code&gt;, but using the updated position, then you should get the expected results.&lt;/p&gt;&#xA;" OwnerUserId="7031" LastActivityDate="2016-06-22T11:31:16.933" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24274" PostTypeId="1" CreationDate="2016-06-22T14:04:42.253" Score="8" ViewCount="4109" Body="&lt;p&gt;Using the SciPy/NumPy libraries, Python is a pretty cool and performing platform for scientific computing. &lt;strong&gt;I just wonder: When I have to go parallel (multi-thread, multi-core, multi-node, gpu), what does Python offer?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm mostly looking for something that is fully compatible with the current NumPy implementation. The coolest thing would be an implementation of NumPy and some other basic functionality (list comprehension for example) that is fully parallel and also can use GPUs. But I'm probably asking for a bit too much ;-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(It's not that we are unable to switch to solution in C/C++, it's more that we'd really like to leverage the advantages of Python above these languages also in a parallel enivronment.)&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-06-23T19:30:57.577" Title="What does Python offer for distributed/parallel/GPU computing?" Tags="|python|scipy|numpy|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24275" PostTypeId="2" ParentId="24274" CreationDate="2016-06-22T20:11:01.417" Score="2" Body="&lt;p&gt;Among the things you can try is &lt;a href=&quot;http://pympi.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;pyMPI&lt;/a&gt; for MPI support in python. We have a good number of users at our HPC site that use it for parallelism. It's not an automated way to get there through NumPy, but you should look at its features carefully.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-06-22T20:11:01.417" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24276" PostTypeId="2" ParentId="16331" CreationDate="2016-06-22T20:38:00.697" Score="2" Body="&lt;p&gt;While this question is old and has been answered by plenty of smart people, I just want to jot down the intuition I use to explain the Galerkin method to people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal in our situation is to find as close of a solution as we can to some continuous residual equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r(u) = 0$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us define the $i^{th}$ basis function as $\phi_{i}(x)$, define the approximate solution as $u_{h} = \sum_{i}^{n} a_{i} \phi_{i}(x)$, and define the residual as $r(u_{h}(x))$, the Galerkin projection ends up being:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{\Omega} r(u_{h}(x)) \phi_i(x) dx = 0 \;\;\; \forall i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This integral expression can be viewed as an inner product written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left&amp;lt;  r(u_{h}), \phi_i\right&amp;gt; = 0 \;\;\; \forall i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the perspective of this inner product, the Galerkin Projection forces the residual error to be orthogonal to the choice of basis functions. So while there might be true error associated with using a lower dimensional representation of the solution, the Galerkin Projection aims to minimize the error component associated with the chosen basis.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-06-22T20:38:00.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24278" PostTypeId="1" CreationDate="2016-06-22T23:49:54.257" Score="0" ViewCount="3008" Body="&lt;p&gt;The Schrodinger equation for time-dependent Hamiltonian is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$i\hbar\frac{d}{dt}\psi(t) = H(t)\psi(t) \, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to implement solve the Schrodinger equation for time-dependent Hamiltonian in ODE 45. However, because the Hamiltonian $H(t)$ is dependent on time, I do not know how to do interpolation in ode45. Can you give me some hints?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;psi0 = [0 1];&#xA;H = [1 0;0 1]*cos(t); %this is wrong, I do not know how to implement this and pass it to ode45&#xA;hbar = 1;&#xA;t    = [0:1:100];&#xA;[T, psi] = ode45(dpsi, t, psi);&#xA;function dpsi = f(t, psi, H, psi0)&#xA;dpsi = (1/i)*H*psi;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2016-06-26T12:23:23.990" Title="Solving Time dependent Schrodinger equation using MATLAB ode45" Tags="|matlab|computational-physics|quantum-mechanics|octave|maple|" AnswerCount="1" CommentCount="9" ClosedDate="2016-07-21T21:43:52.683" ContentLicense="CC BY-SA 3.0" />
  <row Id="24279" PostTypeId="2" ParentId="24274" CreationDate="2016-06-23T00:51:11.370" Score="6" Body="&lt;p&gt;Here's a few options that are relatively easy to work with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One node - &lt;a href=&quot;https://docs.python.org/3/library/multiprocessing.html&quot;&gt;multiprocessing&lt;/a&gt; is the most straightforward thing to do.  &lt;code&gt;multiprocessing.map&lt;/code&gt; works well for an embarrasingly parallel problem.&lt;/li&gt;&#xA;&lt;li&gt;GPU - pyCUDA allows you to do some GPU level programming with just python.  I haven't toyed with it that much.&lt;/li&gt;&#xA;&lt;li&gt;MPI - mpi4py- Exposes MPI interface at the python level.  If you're familiar with MPI, this is useful.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There's a ton of options out there.  &lt;a href=&quot;https://wiki.python.org/moin/ParallelProcessing&quot;&gt;Here's&lt;/a&gt; a page that attempts to catalog some of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I wanted to give more links, but apparently I don't have the rep for that.  A quick search on these terms should turn up documentation)&lt;/p&gt;&#xA;" OwnerUserId="20772" LastActivityDate="2016-06-23T00:51:11.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24280" PostTypeId="2" ParentId="2283" CreationDate="2016-06-23T02:48:02.837" Score="1" Body="&lt;p&gt;Here is a list of Fortran wrappers for Python:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mgaitan/fortran_magic/issues/4&quot; rel=&quot;nofollow&quot;&gt;https://github.com/mgaitan/fortran_magic/issues/4&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5401" LastActivityDate="2016-06-23T02:48:02.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24281" PostTypeId="1" CreationDate="2016-06-23T05:32:52.413" Score="7" ViewCount="909" Body="&lt;p&gt;I'm having some performance problems with my code dealing with the multiplication of big sparse matrices (stiffness and aerodynamic influence coefficient matrices).&#xA;Mainly I have to multiply such matrices frequently in my code with each other. Basically, one should of course not multiply sparse matrices but rather use matrix-vector multiplications. However, in my case, I can hardly prevent matrix-matrix multiplications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now...&#xA;I get sparse matrices in CSC, CSR and COO formats as an input. (Not sensible as well, but I cannot influence this interface -.-).&#xA;Are there rules of the thumb that help me decide which format the matrices in my multiplication should best have to end up with a high(er) performance code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know Intel's &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-mkl-csrcsc&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;mkl_dcsrcsc&lt;/code&gt;&lt;/a&gt; (multiplies CSR-matrix with CSC-matrix) for example, cannot find a &lt;code&gt;mkl_dcsccsr&lt;/code&gt; (multiplies CSC-matrix with CSR-matrix) on the other hand. Is it, therefore, sensible for &lt;span class=&quot;math-container&quot;&gt;$A \cdot B$&lt;/span&gt; to convert &lt;span class=&quot;math-container&quot;&gt;$A_\text{CSC}$&lt;/span&gt; from CSC-format into CSR-format &lt;span class=&quot;math-container&quot;&gt;$A_\text{CSR}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B_\text{CSR}$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$B_\text{CSC}$&lt;/span&gt;? Might it even be sensible to take a long way and convert &lt;span class=&quot;math-container&quot;&gt;$A_\text{CSC}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$A_\text{CSR}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B_\text{CSR}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$B_\text{COO}$&lt;/span&gt; (in order to enable &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-mkl-csrcoo&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;mkl_dcsrcoo&lt;/code&gt;&lt;/a&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I find out which format I should best use for my matrices and which functions/routines I should best use for my matrix-matrix multiplications?&lt;/p&gt;&#xA;" OwnerUserId="19831" LastEditorUserId="20688" LastEditDate="2019-05-14T20:55:39.730" LastActivityDate="2019-05-14T20:55:39.730" Title="Sparse matrix format and sparse-matrix sparse-matrix multiplication" Tags="|matrix|sparse-matrix|performance|blas|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24282" PostTypeId="1" AcceptedAnswerId="24285" CreationDate="2016-06-23T10:42:04.667" Score="5" ViewCount="344" Body="&lt;p&gt;In the previous question &quot;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/16331/motivation-behind-galerkin-method/24272#24272&quot;&gt;Motivation behind Galerkin method&lt;/a&gt;&quot;, Paul gives a good and easy-to-understand explanation indicating that the Galerkin method is a kind of projection method. Can anyone explain why the Collocation method also belongs to the projection methods? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find a reference &quot;&lt;a href=&quot;https://www.wiwi.uni-augsburg.de/vwl/maussner/pap/hm2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Projection methods and the curse of dimensionality&lt;/a&gt;&quot; and know the collocation method employs the Dirac delta function as the weight function. &lt;/p&gt;&#xA;" OwnerUserId="20078" LastEditorUserId="20688" LastEditDate="2020-01-19T20:08:41.443" LastActivityDate="2020-01-19T20:08:41.443" Title="Motivation behind Collocation Method" Tags="|discretization|spectral-method|collocation|galerkin|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24284" PostTypeId="1" AcceptedAnswerId="24287" CreationDate="2016-06-23T11:21:31.127" Score="3" ViewCount="165" Body="&lt;p&gt;&lt;strong&gt;tldr: Can space-time Galerkin schemes applied to convection-diffusion problems lead to effects on the convection velocity?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For time $t\in (0,1)$ and the spatial variable $\xi \in (0,1)$, I am considering the viscous Burger's equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot z + \frac 12\partial_\xi (z^2) - \nu \partial_{\xi\xi}z = 0 \tag{*} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with a viscosity parameter $\nu$. I have zero Dirichlet conditions in space and as the initial condition $z(0)$, I use a step function, which is one on one half of the domain and zero on the other half.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With finite elements in space and Runge-Kutta in time, I get to the expected numerical results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IFPCU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IFPCU.png&quot; alt=&quot;FEM in space + Runge-Kutta in time&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if I discretize the equation with a space-time Galerkin scheme, I get a picture, where, apart from artefacts due to low-dimensional bases, the convection speed is obviously too fast. (The front reaches the border too early)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fLyFq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fLyFq.png&quot; alt=&quot;low-rank space-time Galerkin with 12*12 basis functions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, if I replace the factor $\frac 12$ by the factor $0.375$ in the equation $(*)$, then the convection looks right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4JOCs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4JOCs.png&quot; alt=&quot;low-rank space-time Galerkin with corrected convection speed with 12*12 basis functions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this factor $0.375$ works for all kind of experiments, namely for various viscosity parameters $\nu$ and sizes of the space-time bases, I am almost sure that this is not a bug in my code. Thus, my question is: &lt;strong&gt;Is there a known issue concerning space-time discretizations of transient convection-diffusion problems that cause this corrections in the convection speed?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I use continuous Galerkin in time as well, so that there is no interpretation of the time-space scheme, as a space discretization plus time stepping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit (of the first edit):&lt;/strong&gt;&#xA;Here is the result of the space-time Galerkin simulation for a finer resolution, a smooth initial condition that is consistent with the Dirichlet conditions, and a shorter time frame to avoid the &quot;clash&quot; of the wave with the zero Dirichlet conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6h0T0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6h0T0.png&quot; alt=&quot;full solution and space-time Galerkin with 20*18 basis functions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As before, in the low-order space-time Galerkin approach (right) the wave speed is too high if compared to the FEM/Runge-Kutta solution (left).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT (and confession)&lt;/strong&gt; I must admit that it &lt;em&gt;was&lt;/em&gt; an error in the implementation. After @BillBarth's answer and after that Joost from &lt;a href=&quot;http://www.nutils.org/about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;nutils&lt;/a&gt; provided me a benchmark simulation of the same problem with high-fidelity space-time finite elements I was convinced to revise my code bottom-up. Anyways, this is how it could look like with a low-dimensional (12, 12) degrees of freedom in (space, time) space time Galerkin scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sAMly.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sAMly.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2016-10-30T08:12:02.827" LastActivityDate="2016-10-30T08:12:02.827" Title="Space-time Galerkin of Burgers changes the convection speed" Tags="|finite-element|discretization|parabolic-pde|space-time-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24285" PostTypeId="2" ParentId="24282" CreationDate="2016-06-23T11:44:20.277" Score="12" Body="&lt;p&gt;I would not insist on demanding a geometrical meaning from Galerkin methods in general. There is a connection, but it becomes less meaningful as you extend it further and further. (In a sense, it would be better to call Galerkin methods &quot;generalized projection methods&quot;.) To really understand the connection between collocation and Galerkin methods requires quite a bit of mathematics (functional analysis and measure theory, to be precise), but I'll try to summarize the salient points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea behind Galerkin methods is the following: Assume you have an equation $F(x) = 0$ for some (possibly nonlinear) operator $F:X\to Y$ between (infinite-dimensional) spaces $X$ and $Y$, i.e., you are trying to find $\bar x\in X$ with $F(\bar x)=0$. It is a theorem in functional analysis that $F(x) = 0$ if and only if &#xA;$$\langle F(x),y^*\rangle_Y = 0\qquad\text{for all } y^*\in Y^*,$$ &#xA;which is the dual space of $Y$. (Bear with me, there will be examples later.) This is still something you can't solve, but if you replace $X$ by a &lt;em&gt;finite-dimensional&lt;/em&gt; subspace $X_h\subset X$ and $Y^*$ by a finite-dimensional subspace $Y^*_h\subset Y^*$, you can try to find $\bar x_h \in X_h$ such that $$\label{1}&#xA;\langle F(\bar x_h),y^*_h\rangle_Y = 0\qquad\text{for all } y^*_h \in Y_h^*, \tag{1}&#xA;$$ &#xA;or equivalently, since $Y^*_h$ is finite-dimensional, for all basis functions $y^*_1,\dots,y^*_n$ of $Y^*_h$. If $X_h$ and $Y_h$ are chosen correctly, this gives you $n$ independent conditions for $n$ unknown coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's specialize a bit: If $Y=X$ is a Hilbert space, you can take $Y^*_h = X_h$; this is called a &lt;em&gt;Ritz-Galerkin method&lt;/em&gt;. Furthermore, if $F(x)=x-z$ for some $z\in X$, what we're actually doing is looking for $x_h\in X_h$ such that $x_h-z$ is orthogonal to $X_h$ -- these are exactly the conditions that characterize the orthogonal projection of $z$ onto the subspace $X_h$. Hence the name &lt;em&gt;(generalized) projection method&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for collocation (and the promised example): If $Y\neq X$ and thus $Y_h^* \neq X_h$, one speaks of &lt;em&gt;Petrov-Galerkin methods&lt;/em&gt;. In the specific case that $Y=C(E)$, the space of continuous functions on some closed set $E$ (e.g., the real interval $[0,1]$), the dual space is given by the space $\mathcal{M}(E)$ of Borel measures on $E$, and the duality pairing between a continuous function $u$ and a measure $\mu$ is given by &#xA;$$\langle u,\mu\rangle_C = \int_E u(x) \,d\mu(x).$$ &#xA;An important example of Borel measures are the Dirac measures $\delta_{x_0}$ for $x_0\in E$, which are &lt;em&gt;defined&lt;/em&gt; by their action as&#xA;$$\langle u,\delta_{x_0} \rangle_C := u(x_0).$$ &#xA;Now we're almost there: If we take $Y^*_h$ as the subspace spanned by Dirac measures for $n$ distinct points $x_1,\dots,x_n\in E$, the Galerkin conditions \eqref{1} become&#xA;$$ F(\bar u_h)(x_i)  = \langle F(\bar u_h),\delta_{x_i}\rangle_Y = 0\qquad\text{for all } 1\leq i \leq n.$$&#xA;Similarly, if $X=C(D)$ for some space $D$ (e.g., $D=[0,1]$ as well), you can choose $X_h$ as the subspace of piecewise linear (on a given set of intervals $[x_1,x_2],\dots,[x_{n-1},x_n]\subset D$ and continuous functions on $D$ and uniquely represent $\bar u_h\in X_h$ by its values $\bar u_h(x_i)$ for $1\leq i\leq n$. Then what you actually have to find is the values $\bar u_h(x_i)$ for $1\leq i\leq n$ such that $F(\bar u_h)(x_i) = 0$ for all $1\leq i\leq n$ -- which is precisely collocation.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-06-23T11:44:20.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24287" PostTypeId="2" ParentId="24284" CreationDate="2016-06-23T13:32:40.353" Score="1" Body="&lt;p&gt;In your third picture, there are still a lot of spurious oscillations. You clearly know your mesh is under-resolved by at least a factor of 2. I'm guessing that the factor of 0.5 in the second term of (*) is externally imposed and that you shouldn't be arbitrarily changing it. Each space-time Galerkin method is equivalent to a finite difference scheme in space and time, depending on your choice of quadrature. I don't know what quadrature you've done here, and I don't have the equivalences memorized, but if this doesn't work out to something that's symplectic, you're very likely to go too fast or too slow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend refining the mesh until the oscillations stop first and see if you pick up the required wave speed. If not, you should work backwards and from one timestep (i.e. one time test function) of your space-time Galerkin method with whatever quadrature method you are using and see what FDM you have. It's probably not sympletic and is likely messing up everything. &lt;/p&gt;&#xA;" OwnerUserId="27" LastEditorUserId="27" LastEditDate="2016-06-23T21:43:45.333" LastActivityDate="2016-06-23T21:43:45.333" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24288" PostTypeId="1" CreationDate="2016-06-23T13:40:38.167" Score="1" ViewCount="40" Body="&lt;p&gt;Let $f:(0,1)\rightarrow [0,1]$ be a $\mathcal{C}^{\infty}$ function.&#xA;We say that a function $D_r^f:\{1,...,r\}\rightarrow [0,1]$ is an $r$-discretization of $f$ if $$D_r^f(j) = \frac{1}{|a(j)-b(j)|}\cdot\int_{(a(j),b(j))} f(x)\;dx$$ where $a(j)=\frac{(j-1)}{r}$ and $b(j) = \frac{j}{r}$. In other words, we divide the interval $(0,1)$ in $r$ consecutive sub-intervals  and $D^f_r(j)$ is the average of $f$ on the $j$-th sub-interval $(a(j),b(j))$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any two $\mathcal{C}^{\infty}$ functions $f,g:(0,1)\rightarrow [0,1]$, we let we consider the following inner product. $$\langle f,g\rangle = \int_{(0,1)} f(x)\cdot g(x)\; dx$$&#xA;On the other hand we define the inner product on $r$-discretizations as follows. &#xA;$$\langle D_r^f,D_r^g\rangle = \sum_{j=1}^r D_r^f(j)\cdot D_r^g(j).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Given an $\varepsilon \in (0,1)$, how large must $r$ be so that $|\langle D_r^f,D_r^g \rangle - \langle f,g\rangle| \leq \varepsilon$? &lt;/li&gt;&#xA;&lt;li&gt;Is it enough to assume that $f,g$ are $\mathcal{C}^{\infty}$ functions in order to be able to prove a bound on $r$ in terms of $\varepsilon$? If not, what are the weakest conditions $f$ and $g$ must satisfy? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Obs: This question was asked at mathoverflow some days ago (with no answers). Now I realize that scicomp may be the most adequate place to ask it. &lt;/p&gt;&#xA;" OwnerUserId="20779" LastActivityDate="2016-06-23T13:40:38.167" Title="Inner Products vs Discretizations of Functions" Tags="|numerical-analysis|discretization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24289" PostTypeId="2" ParentId="23641" CreationDate="2016-06-23T14:07:19.537" Score="2" Body="&lt;p&gt;The specification for the sparse BLAS ( see &lt;a href=&quot;http://www.nist.gov/customcf/get_pdf.cfm?pub_id=50993&quot; rel=&quot;nofollow&quot;&gt;http://www.nist.gov/customcf/get_pdf.cfm?pub_id=50993&lt;/a&gt; for an overview of the specification) documents a routine for the multiplication of a sparse matrix and a dense matrix. This operation is commonly referred to as SpMM. The document does not specify a routine for multiplication of a sparse matrix and a second sparse matrix for reasons described on page 243 of the overview document. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, multiplication of two sparse matrices is an important operation and is often referred to as SpGEMM to distinguish it from the sparse BLAS version. See, for example, &lt;a href=&quot;http://www.sandia.gov/~gmballa/talks/LA15.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.sandia.gov/~gmballa/talks/LA15.pdf&lt;/a&gt;, for more information on SpGEMM.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-06-23T14:07:19.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24290" PostTypeId="1" CreationDate="2016-06-23T14:53:56.767" Score="2" ViewCount="82" Body="&lt;p&gt;I've been running some complicated Finite Element Models. In most cases, the stress repartition seemed to be absolutely correct. However, on several point (complicated geometry), it appears that stress is way beyond the overall stress in that area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same goes for calculus of hydrostatic pressure. While all the model ranges between -150;150 kPa, there are some areas that reach the insane values of 300 or even 1MPa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I be sure the singularities in those points does not alter stress repartition to a point where the model is not realistic ?&lt;/p&gt;&#xA;" OwnerUserId="20191" LastActivityDate="2016-06-23T14:59:57.057" Title="Can singularity screw up your model?" Tags="|finite-element|numerical-modelling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24291" PostTypeId="2" ParentId="24290" CreationDate="2016-06-23T14:59:57.057" Score="4" Body="&lt;p&gt;Yes!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any reentrant corner has a singularity at the corner (thus the classic L-shaped domain problem). There's lots of literature on this, but I don't know that it's a completely solved problem.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-06-23T14:59:57.057" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24292" PostTypeId="2" ParentId="10011" CreationDate="2016-06-23T16:02:18.667" Score="1" Body="&lt;p&gt;There is &lt;a href=&quot;http://cimg.eu/&quot; rel=&quot;nofollow&quot;&gt;CImg&lt;/a&gt;, a header-only library that is fairly performant but not very tweakable. It is mainly about image processing, but it also offers some functions for drawing graphs and axes, albeit limited (you get the graph options you get, and that is it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The biggest quirk with it is just getting used to the way the image is stored internally. The color channels are separated, so you've got contiguous bytes for blue, then green, then red. To output to a screen or file, you've got to interleave them back together. The sample code is good and makes it easy to figure things out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used it (it can draw a few graphs fast enough to keep up with a live video stream at 60fps) but also look longingly at matplotlib and the like.&lt;/p&gt;&#xA;" OwnerUserId="20782" LastActivityDate="2016-06-23T16:02:18.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24293" PostTypeId="1" CreationDate="2016-06-23T16:38:43.350" Score="6" ViewCount="1560" Body="&lt;p&gt;I've recently finished an introductory course on the finite element method from a more mathematical perspective (following Brenner and Scott) and we were introduced to the finite element mass matrix in elliptic problems as the matrix arising from terms without a derivative. For example, a one-dimensional Helmholtz type equation with the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-u(x)'' + au(x) = f(x), \quad 0 &amp;lt; x &amp;lt; 1, \quad a&amp;gt;0\\&#xA; u(0) = u(1) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;has a corresponding weak formulation that requires us to find $u$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_0^1 u' v'dx + a\int_0^1uv dx = \int_0^1fvdx \quad \forall v \in H^1_0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H^1_0 = \{v \in H^1 : v(0) = v(1) = 0\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choosing $S \subset H^1_0$ to be a conforming finite dimensional subset with a basis $\{ \phi_i \}_{i=1}^N$, and saying $u = \sum_{j = 0}^N u_j \phi_j $, we get the linear problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;(\pmb{K} + a \pmb{M})U = F &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K_{ij} = \int_0^1 \phi_i' \phi_j' dx$ is the stiffness matrix and $M_{ij} = \int_0^1 \phi_i \phi_j dx$ is the mass matrix. The finite element method typically proceeds by choosing $S$ to be the space of piecewise polynomials for example. This formulation extends naturally to higher dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this previous post: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19704/how-to-formulate-lumped-mass-matrix-in-fem&quot;&gt;How to formulate lumped mass matrix in FEM&lt;/a&gt;, there are various ways to lump the mass matrix. For example, by summing the off-diagonal terms: $M_{ii} = \sum M_{ij}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what is the justification of this? Is there mathematical reasoning why this should give a consistent method? Is there a way to quantify the error introduced by doing this? I've seen an explanation that justifies mass matrix lumping in the context of mechanics where this assumption implies that the mass of the system is concentrated at discrete points, but how does this generalize to more general elliptic PDE problems?&lt;/p&gt;&#xA;" OwnerUserId="20526" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-09-11T23:08:46.940" Title="Effects of Lumping Mass Matrix" Tags="|finite-element|pde|matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24294" PostTypeId="2" ParentId="24274" CreationDate="2016-06-23T19:30:57.577" Score="2" Body="&lt;p&gt;IPython has a package for parallel computing called &lt;a href=&quot;https://github.com/ipython/ipyparallel&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;ipyparallel&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An overview from their &lt;a href=&quot;http://ipyparallel.readthedocs.io/en/stable/&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This architecture abstracts out parallelism in a very general way,&#xA;  which enables IPython to support many different styles of parallelism&#xA;  including:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Single program, multiple data (SPMD) parallelism.&lt;/li&gt;&#xA;  &lt;li&gt;Multiple program, multiple data (MPMD) parallelism.&lt;/li&gt;&#xA;  &lt;li&gt;Message passing using MPI.&lt;/li&gt;&#xA;  &lt;li&gt;Task farming.&lt;/li&gt;&#xA;  &lt;li&gt;Data parallel.&lt;/li&gt;&#xA;  &lt;li&gt;Combinations of these approaches.&lt;/li&gt;&#xA;  &lt;li&gt;Custom user defined approaches.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;Most importantly, IPython enables all types of&#xA;  parallel applications to be developed, executed, debugged and&#xA;  monitored interactively. Hence, the &lt;code&gt;I&lt;/code&gt; in IPython. The following are&#xA;  some example usage cases for IPython:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Quickly parallelize algorithms that are embarrassingly parallel using a number of simple approaches. Many simple things can be&#xA;  parallelized interactively in one or two lines of code.&lt;/li&gt;&#xA;  &lt;li&gt;Steer traditional MPI applications on a supercomputer from an IPython session on your laptop.&lt;/li&gt;&#xA;  &lt;li&gt;Analyze and visualize large datasets (that could be remote and/or distributed) interactively using IPython and tools like&#xA;  matplotlib/TVTK.&lt;/li&gt;&#xA;  &lt;li&gt;Develop, test and debug new parallel algorithms (that may use MPI) interactively.&lt;/li&gt;&#xA;  &lt;li&gt;Tie together multiple MPI jobs running on different systems into one giant distributed and parallel system.&lt;/li&gt;&#xA;  &lt;li&gt;Start a parallel job on your cluster and then have a remote collaborator connect to it and pull back data into their local IPython&#xA;  session for plotting and analysis.&lt;/li&gt;&#xA;  &lt;li&gt;Run a set of tasks on a set of CPUs using dynamic load balancing.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="20786" LastActivityDate="2016-06-23T19:30:57.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24295" PostTypeId="1" CreationDate="2016-06-23T21:51:11.240" Score="1" ViewCount="35" Body="&lt;p&gt;To be specific, let's consider Support Vector Machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 1 vs all case, one has $n$ binary classifiers where $n$ is the number of classes. In each classifier one has two labels 0, 1 and for a given test data set, one can predict $\rm Prob(0)$ and $\rm Prob(1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how to compute prediction probability $\rm Prob(i)$ of a given test data point in class $i=1,2,\cdots,n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In sklearn, it seems that ${\rm Prob}(i) = {\rm Prob}(1;i)/\sum_{j=1}^n {\rm Prob}(1;j)$. How to interpret it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 1 vs 1 case, one has $n(n-1)$ binary classifiers. Then how to compute prediction probability of classes from binary classifiers?&lt;/p&gt;&#xA;" OwnerUserId="2102" LastActivityDate="2016-06-23T21:51:11.240" Title="How to calculate multi-class prediction probabilities using One vs All and One vs One classifiers" Tags="|algorithms|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24297" PostTypeId="1" CreationDate="2016-06-23T23:45:15.087" Score="1" ViewCount="214" Body="&lt;p&gt;I am trying to simulate elastoplastic seismic wave propagation using &lt;a href=&quot;https://bitbucket.org/fenics-apps/fenics-solid-mechanics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fenics Solid Mechanics Application&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The app. provides some quasi-static demos to show elastoplastic behaviour in a cube/ beam/ square. The weak form they implement is taken from &lt;a href=&quot;http://link.springer.com/chapter/10.1007%2F978-1-4020-9090-5_18&quot; rel=&quot;nofollow noreferrer&quot;&gt;page 199 of this paper&lt;/a&gt; written by the developers themselves, and is given as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IpPXb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IpPXb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the terms are defined in a paragraph below the equation in the paper. The equation is very similar to the weak form obtained for linear elasticity. It must be noted that in computational plasticity, return mapping algorithms (Simo and Hughes 1998) are popular and hence the application uses a 'closest point projection' return mapping algorithm along with Newton method. The stress (sigma) in the linear form 'L' is the residual stress (or stress from previous iteration). In the LHS (or bilinear form 'a') the variable 'C' is the consistent elastoplastic tangent moduli. Other variables are Body force 'f'; traction 'h'; test function 'v'; and trial function 'u'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far so good. Now, for elastic linear wave propagation the weak form implemented is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cYiT0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cYiT0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the test function 'w' is the same as the test function 'v' in equation (6) and (7). The only new term is the 'intertia term', where $/rho$ is the mass density of material and $/ddot{u}$ is the acceleration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering the above 2 equations, is it fair to say that I can implement an elastoplastic wave using the following weak form?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1coW1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1coW1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $C_{ep}$ is the same as $C$ in equation (6).&lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2016-06-23T23:45:15.087" Title="Weak form for elastoplastic wave propagation" Tags="|finite-element|fenics|wave-propagation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24300" PostTypeId="1" AcceptedAnswerId="24319" CreationDate="2016-06-24T14:07:06.730" Score="3" ViewCount="269" Body="&lt;p&gt;I am trying to use a projection method that deals with the viscous effects implicitly to model flow around a cylinder. I'm having trouble figuring out what the boundary conditions should be, particularly on the inflow and outflow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think we can consider the Stokes equations without loss of generality: &#xA;$$ \mathbf{u}_t = \Delta \mathbf{u} - \nabla p $$&#xA;$$ \nabla \cdot\mathbf{u} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we discretize explicitly in time (ignoring the pressure term) we get:&#xA;$$ \frac{\mathbf{u}^{*} - \mathbf{u}^n}{\delta} = \Delta\mathbf{u}^{n}$$&#xA;This leads to:&#xA;$$ \mathbf{u}^{n+1}=\mathbf{u}^* - \delta\nabla p^{n+1}$$&#xA;Taking the divergence of this equation yields the pressure Poisson equation. The boundary conditions can be found by dotting with the normal:&#xA;$$\nabla p^{n+1} \cdot\mathbf{n} = \frac{(\mathbf{u}^* - \mathbf{u}^{n+1})}{\delta}\cdot\mathbf{n}$$&#xA;where $\mathbf{u}^*\cdot\mathbf{n}$ can be computed and $\mathbf{u}^{n+1}\cdot\mathbf{n}$ is given as a boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if we discretize implicitly in time we get:&#xA;$$ \frac{\mathbf{u}^* - \mathbf{u}^n}{\delta} = \Delta \mathbf{u}^{*}$$&#xA;This is the diffusion equation for $\mathbf{u}^*$, and requires boundary conditions on $\mathbf{u}^*$. The most obvious choice is setting $\mathbf{u}^* = \mathbf{u}^{n+1}$ on the boundary. However if we do that, for the pressure Poisson equation we end up with $\nabla p\cdot\mathbf{n} = 0$ everywhere (even at the inflow and outflow). This seems incorrect to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should the correct boundary conditions on the pressure Poisson equation be in this case? &lt;/p&gt;&#xA;" OwnerUserId="9728" LastEditorUserId="8174" LastEditDate="2016-06-27T16:36:47.217" LastActivityDate="2016-06-28T12:31:02.690" Title="Implicit projection method with inflow boundary conditions" Tags="|fluid-dynamics|projection|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24301" PostTypeId="1" AcceptedAnswerId="24305" CreationDate="2016-06-24T16:05:30.940" Score="0" ViewCount="203" Body="&lt;p&gt;I am a little confused about the connection between variables for the plain advection pde:&#xA;$$u_t+au_x=0$$&#xA;So initially I thought $x$ and $t$ are independent and $u$ is a function of those, but then we can write the same PDE and say that there are characteristics and then we have ODE for them:&#xA;$$\frac{dx}{dt}=a$$&#xA;So, now $x$ is a function of $t$. So are they dependent or not?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2016-06-26T08:41:11.627" Title="hyperbolic equation and characteristics" Tags="|hyperbolic-pde|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24302" PostTypeId="1" AcceptedAnswerId="24311" CreationDate="2016-06-24T18:21:19.013" Score="0" ViewCount="160" Body="&lt;p&gt;I want to solve the advection equation: $u_t+au_x=0, a &amp;gt; 0$. Here is our method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U_j^{n+1}=U_{j-1}^n-\left(\frac{ak}{h}-1\right)(U_{j-1}^n - U_{j-2}^n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to answer the following question: &quot;Say we fix $h$, then for which values of $k$ is this method exact? Note that there's more than one value.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the following is exact: $U_j^{n+1}=U_{j-1}^n$, which corresponds to $k=\frac{h}{a}$. But what is the other value of $k$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7322" LastActivityDate="2016-06-27T10:09:34.497" Title="Exact finite difference method for advection equation" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24304" PostTypeId="2" ParentId="24300" CreationDate="2016-06-25T08:22:43.947" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The most obvious choice is setting $\mathbf{u}^* = \mathbf{u}^{n+1}$ on the boundary.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is true if your velocity is described by Dirichlet BCs, however, if your velocity BCs are described by Neumann BCs (fully developed flow, e.g.), you should use $\frac{\partial \mathbf{u}^*}{\partial n} = 0$. This, it sounds to me, is what you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-06-25T08:22:43.947" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24305" PostTypeId="2" ParentId="24301" CreationDate="2016-06-26T08:41:11.627" Score="1" Body="&lt;p&gt;You are correct in your initial assumption. x and t are independent and u is  the dependent variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your confusion is because you are assuming that $\frac{dx}{dt} = a$ is the solution to the equation. Notice that $u$ is not in this ODE. Basically this is an equation for the symmetry of the PDE. After getting this equation, you can write &lt;/p&gt;&#xA;&#xA;&lt;p&gt;u = F(x-at)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is the actual solution since you will know u at t = 0 by specifying the initial conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;x and t are not dependent on each other. $\frac{dx}{dt} = a$ merely states that the solution is fully specified on lines that are solutions of this ODE, and thus are called characteristics. &lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2016-06-26T08:41:11.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24306" PostTypeId="1" CreationDate="2016-06-26T09:55:01.600" Score="0" ViewCount="248" Body="&lt;p&gt;I'm simulating a biological phenomena with reaction diffusion equations. There are multiple diffusing materials and there are some complex relations about consumption and production of such materials. What is the best way to simulate these interactions? Is finite elements method appropriate? and is there any tool, which solves the PDE (with any method) and give me distribution of chemicals at each time step so that I apply my computations on them?&lt;/p&gt;&#xA;" OwnerUserId="20808" LastEditorUserId="9667" LastEditDate="2016-06-26T20:02:25.200" LastActivityDate="2016-06-26T20:02:25.200" Title="Reaction-diffusion equations" Tags="|finite-element|pde|numerical-analysis|diffusion|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24307" PostTypeId="2" ParentId="24278" CreationDate="2016-06-26T12:23:23.990" Score="1" Body="&lt;p&gt;Possibly this is what you want?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function schrodingerEqn&#xA;psi0 = [0 1];&#xA;hbar = 1;&#xA;t    = [0:1:100];&#xA;fh = @(t, psi) f(t, psi, hbar);&#xA;[T, psi] = ode45(fh, t, psi0);&#xA;figure; plot(T, real(psi(:,2)));&#xA;end&#xA;&#xA;function dpsi = f(t, psi, hbar)&#xA;dpsi = 1/(hbar*i)*[1 0;0 1]*cos(t)*psi;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-06-26T12:23:23.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24308" PostTypeId="1" CreationDate="2016-06-26T14:36:33.153" Score="1" ViewCount="257" Body="&lt;p&gt;I am writing a Fortran program that does fairly light weigt computation on fairly large data files. The output must be readable by some Java-script (that i have no insight in) and preferably also by other languages/programs like PHP and Matlab. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using formatted data the I/O requires about 50 times more time than the processing, while if using unformatted data, the I/O and compute times are about equal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to my question: is there some &quot;format&quot; that is portable between Fortran, and other languages but is still I/O efficient?&lt;/p&gt;&#xA;" OwnerUserId="20686" LastActivityDate="2016-06-26T14:48:59.773" Title="A fast I/O file format readable by Fortran, Java, PHP?" Tags="|fortran|io|java|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24309" PostTypeId="2" ParentId="24308" CreationDate="2016-06-26T14:48:59.773" Score="3" Body="&lt;p&gt;HDF5 appears to have some JavaScript support via its &lt;a href=&quot;https://github.com/HDFGroup/h5serv&quot; rel=&quot;nofollow&quot;&gt;HDF Server&lt;/a&gt; product. You might be able to use this, though it's a web API and not a native reader. Do you need to use this on the backend (like via node.js) on the JS side, or over the web? It might make a big difference in performance.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-06-26T14:48:59.773" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24311" PostTypeId="2" ParentId="24302" CreationDate="2016-06-27T10:09:34.497" Score="1" Body="&lt;p&gt;Let $c=a k/h$ denote so called Courant number. Your scheme is reasonable only for $c \in [1,2]$. It is exact for $c=1$, as you mentioned, and for $c=2$ in the sense that if the values at time level $n$ are exact, they will be exact also at time level $n+1$. It is unusual to use a scheme that is inprecise (and very likely unstable) for small time step $k$, i. e.  when $c \in (0,1)$, you may include standard first order   upwind scheme for this case. There exist schemes that solve your advection equation for arbitrary time step $k$ and even for variable $a$, e. g. modified method of characteristics. &lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-06-27T10:09:34.497" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24315" PostTypeId="1" CreationDate="2016-06-27T22:50:32.033" Score="1" ViewCount="412" Body="&lt;p&gt;I have a 2D square mesh and a 3D beam shaped mesh and I want to propagate a seismic wave in them. I am trying to simulate them using Open source FEM codes (fenics). I have left the top surface to be free because that is where we would see a response (ground motion). I have also set the other boundaries except the bottom one to zero Dirichlet condition. These will cause the waves hitting them to reflect back. Usually, these would be absorbing boundary conditions or PML but for now I keep them reflective. I believe that the input seismic excitation will be set as a Dirichlet condition on the bottom boundary (2D/ 3D). In literature, a Ricker Wavelet pulse is considered suitable to represent a seismic input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is the input source given as a Dirichlet condition on bottom boundary or is it given as a point source ricker wavelet somewhere within the mesh close to the bottom boundary?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If it is a point source ricker wavelet; how would I implement it in fem mesh using c++ or python at the point where I want it? Does anyone have a piece of code that would help? Is it added as the source term '$f$' in the '$f∗v∗dx$' part of the weak form of wave equation (where $v$ is the test function) or is it projected as an initial condition on the mesh??&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for your guidance!&lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="16705" LastEditDate="2016-06-27T23:28:57.057" LastActivityDate="2016-06-27T23:28:57.057" Title="How to add a Ricker Wavelet (Mexican Hat) to a 2D/ 3D fem mesh?" Tags="|finite-element|boundary-conditions|fenics|wave-propagation|wavelet|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24317" PostTypeId="1" CreationDate="2016-06-28T03:23:06.513" Score="1" ViewCount="146" Body="&lt;p&gt;I would like to find the intersection points e.g. $M_x$ and $M_y$ as in the attached figure. The boundary (solid line) is defined by the Lagrangian points.&#xA;I am working in C++ (basic - medium proficiency).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this regard, I need a computer algorithm or code to get the intersection points both in x- and y-in a separate arrays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h8MQf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h8MQf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18144" LastActivityDate="2016-06-28T03:23:06.513" Title="Finding boundary intersection points with Cartesian grid?" Tags="|mesh-generation|grid|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24318" PostTypeId="1" CreationDate="2016-06-28T10:22:47.110" Score="3" ViewCount="121" Body="&lt;p&gt;I am working on an program to compute the structure factor of a given configuration of particles, and I need an efficient algorithm to generate all the possible vectors with integer coordinates and magnitude between $n-\delta$ and $n+\delta$, where $\delta$ is small compared to $n$. This is equivalent to finding all the solutions to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(n-\delta)^2&amp;lt;\| \vec v \|^2&amp;lt;(n+\delta)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\vec v \in \mathbf{Z}^3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, I only need to find half of the solutions because the other half will be given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec v'=-\vec v$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best algorithm to solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS This is my first time posting here so any help with the tags would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20342" LastEditorUserId="20342" LastEditDate="2016-06-28T14:13:04.080" LastActivityDate="2016-06-28T22:34:09.443" Title="Algorithm to generate all vectors of integers with magnitude between $n\pm \delta$" Tags="|combinatorics|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24319" PostTypeId="2" ParentId="24300" CreationDate="2016-06-28T12:31:02.690" Score="1" Body="&lt;p&gt;tldr: &lt;strong&gt;Reformulate the projection and avoid the need for boundary conditions on the pressure.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you are misinterpreting the projection scheme. In all formulations that I know, the pressure is never really computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It rather goes like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute a tentative velocity $u^*$ approximating $u^{n+1}$&lt;/li&gt;&#xA;&lt;li&gt;Project $u^*$ onto the space of divergence-free functions to obtain $u^{n+1}$ by solving &#xA;$$ \begin{bmatrix} I &amp;amp; \nabla \\ \nabla \cdot &amp;amp; 0 \end{bmatrix}&#xA;   \begin{bmatrix} u^{n+1} \\ \phi \end{bmatrix} = \begin{bmatrix} u^{*} \\ 0 \end{bmatrix} $$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some remarks here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You may compute the $u^*$ as you suggest (setting $\nabla p^{n+1}=0$). Typically, one rather uses a guess for the pressure gradient.&lt;/li&gt;&#xA;&lt;li&gt;In step 2. one computes the actual velocity, so that the given boundary conditions apply. However,&lt;/li&gt;&#xA;&lt;li&gt;The function $\phi$ is not the pressure, so that it might be difficult to interprete boundary conditions that include the pressure (like Navier-conditions). There are ways to relate $\phi$ to the pressure; cf., e.g. [1].&lt;/li&gt;&#xA;&lt;li&gt;Furthermore, the system in 2. can be brought into the &lt;em&gt;Pressure Poisson Equation&lt;/em&gt; that you have. But at the expense of another spatial differentiation (taking the divergence) and the additional need of boundary conditions. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want to really compute the pressure, you can use the relation to $\phi$ (this is often sufficient for the next pressure gradient guess) or solve the actual &lt;em&gt;Pressure Poisson Equation&lt;/em&gt; derived from your actual equation and when you have the velocity computed. There is some issues about the right boundary conditions, but also some good answers to that in [2].&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[1] P. M. Gresho. On the theory of semi-implicit projection methods for&#xA;  viscous incompressible flow and its implementation via a finite element&#xA;  method that also introduces a nearly consistent mass matrix. I: Theory.&#xA;  Int. J. Numer. Methods Fluids, 11(5):587–620, 1990.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[2] P. M. Gresho and R. L. Sani. Incompressible flow and the finite element&#xA;  method. Vol. 2: Isothermal laminar flow. Wiley, Chichester, UK, 2000.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-06-28T12:31:02.690" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24320" PostTypeId="1" AcceptedAnswerId="32868" CreationDate="2016-06-28T13:16:47.280" Score="3" ViewCount="377" Body="&lt;p&gt;I am looking for a discrete Fourier transform (DFT) library that can be run with MPI on Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, in other languages (C, Fortran) &lt;a href=&quot;http://www.fftw.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FFTW&lt;/a&gt; is used. There's a Python wrapper for FFTW called &lt;a href=&quot;https://hgomersall.github.io/pyFFTW/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyFFTW&lt;/a&gt;, it does support multithreading but seemingly not MPI.&lt;/p&gt;&#xA;" OwnerUserId="19111" LastEditorUserId="20688" LastEditDate="2019-06-11T23:16:28.973" LastActivityDate="2019-06-11T23:16:28.973" Title="MPI support for discrete Fourier transform (DFT) in Python" Tags="|python|mpi|hpc|fourier-transform|fftw|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24321" PostTypeId="1" AcceptedAnswerId="24324" CreationDate="2016-06-28T15:08:20.080" Score="3" ViewCount="245" Body="&lt;p&gt;I am actually confused about time stepping for PDEs. Before, I was distretizing time using backward Euler method for implicit formulation and I get a system of Algebraic equations to solve. Now during my research when I read papers, I find they  disctretize space only leaving time derivative which yields system ODEs?&#xA;What is the difference between two approaches and when I use one not another?&lt;/p&gt;&#xA;" OwnerUserId="20025" LastActivityDate="2016-06-29T07:35:07.563" Title="Discretizing time term for PDEs" Tags="|pde|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24322" PostTypeId="1" CreationDate="2016-06-28T15:16:36.090" Score="2" ViewCount="1534" Body="&lt;p&gt;I have been reading about the heat equation and I am confused about uniqueness in the case when the domain is bounded and when it is not. In the book I am following, it is common to write the heat equation over [0,1], with zero values on the boundaries and shows that a series solves that equation. I can follow that, and the claim that the solution is unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the same book can give an example on unbounded domain and write the solution as an integral of the heat kernel and again it is a solution and it is unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, do boundary conditions affect uniqueness? Looks like not as I can write a unique solution in both cases. What is the effect then?&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="9667" LastEditDate="2016-06-29T20:52:15.130" LastActivityDate="2016-12-27T00:01:52.167" Title="heat equation on bounded and unbounded domain" Tags="|boundary-conditions|parabolic-pde|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24323" PostTypeId="1" AcceptedAnswerId="24325" CreationDate="2016-06-28T15:59:37.927" Score="2" ViewCount="99" Body="&lt;p&gt;I am having a hard time understanding the very first example in &lt;a href=&quot;http://www2.isye.gatech.edu/people/faculty/Alex_Shapiro/TutorialSP.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;A Tutorial on Stochastic Programming&quot;&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically the authors show that one can formulate the stochastic variant of &lt;strong&gt;(1.2)&lt;/strong&gt; with &lt;strong&gt;Eq (1.8)&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;My question:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;How should one interpret &lt;strong&gt;(1.8)&lt;/strong&gt; (i.e. how do the new inequalities allow us to explore two different scenarios in the problem?)&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;The deterministic problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZfmMc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZfmMc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xt0F0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xt0F0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Stochastic re-formulation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ILDem.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ILDem.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZYEkb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZYEkb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20832" LastActivityDate="2016-06-28T16:10:42.643" Title="From deterministic to stochastic LP formulations" Tags="|optimization|convex-optimization|constrained-optimization|linear-programming|stochastic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24324" PostTypeId="2" ParentId="24321" CreationDate="2016-06-28T16:05:50.020" Score="4" Body="&lt;p&gt;If you just discretize space and turn your PDE system into a system of ODEs, you end up being able to take advantage of ODE integration libraries/codebases, which might make your life easier since you only spend the effort doing the spatial discretization and then get to just throw in those dynamics into a sort of black box ODE integrator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you discretize time and space, you really put the pressure on yourself to completely tackle and solve the problem on your own. This gives you more control over what time integration methods you want to implement and will likely make it easier to implement implicit schemes, for example. However, you do have more room for error and this approach can be more effort on paper and in implementation, depending on the scale of the problem.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-06-28T16:05:50.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24325" PostTypeId="2" ParentId="24323" CreationDate="2016-06-28T16:10:42.643" Score="2" Body="&lt;p&gt;(1.8) is a simple reformulation of the deterministic LP (1.2)  It's still a deterministic LP.   This may be somewhat confusing since the authors are going back forth between a deterministic LP formulation (assuming that $d$ is known) and a stochastic LP formulation (in which we're minimizing the expected value over random values of $d$.)  See (1.9) for the expected value formulation corresponding to the deterministic $d$ formulation in (1.8).   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-06-28T16:10:42.643" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24326" PostTypeId="1" AcceptedAnswerId="24328" CreationDate="2016-06-28T16:12:04.337" Score="3" ViewCount="423" Body="&lt;p&gt;Recently, I obtain a linear system, $Ax = b$, where $A$ is a nonsingular, strictly diagonally dominant $M$-matrix. Then I also got a matrix splitting $A = S - T$, where $S$ is also a nonsingular, strictly diagonally dominant $M$-matrix. So I establish a stationary iteration scheme as follows, &#xA;$$Sx^{(k+1)} = Tx^{(k)} + b.$$&#xA;According to numerical results, it seems that this iterative scheme is always convergent. Is it rational from theoretical analysis? So, can we prove the convergence of this iterative scheme? i.e., &#xA;show $\rho(S^{-1}T) &amp;lt; 1$, where $\rho(\cdot)$ is the spectral radius.&lt;/p&gt;&#xA;" OwnerUserId="10290" LastActivityDate="2016-06-28T18:40:05.483" Title="Convergence conditions of a stationary iteration method for linear systems" Tags="|iterative-method|operator-splitting|matrix-equations|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24328" PostTypeId="2" ParentId="24326" CreationDate="2016-06-28T18:40:05.483" Score="2" Body="&lt;p&gt;You can prove convergence by satisfying the spectral radius relationship you note, choosing $S$ and $T$ such that $\rho(S^{-1}T) &amp;lt; 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This comes about by first writing two equations based on your operator splitting:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Sx = Tx + b$$&#xA;$$ Sx^{(k+1)} = Tx^{(k)} + b $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x$ is exact solution and $x^{(k)}$ is the $k^{th}$ iteration's solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now assume error can be defined as: $e^{(k)} = x - x^{(k)}$. Based on this relationship and the two equations above, you end up with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Se^{(k+1)} = Te^{(k)} $$&#xA;$$ e^{(k+1)} = (S^{-1}T) e^{(k)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the resulting equation, the condition that needs to be met for $e^{(k+1)}$ to converge as $k \rightarrow \infty$ is for $\rho(S^{-1}T) &amp;lt; 1$, as you mentioned. How you prove this spectral radius inequality depends on the form of $S^{-1}T$ based on your choices for $S$ and $T$.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-06-28T18:40:05.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24329" PostTypeId="1" CreationDate="2016-06-28T21:31:30.057" Score="1" ViewCount="316" Body="&lt;p&gt;I'm relatively new to fenics and I just looked through all questions related to Dirichlet boundary conditions. I don't seem to find a well-described question or answer about what I'm about to ask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm solving a PDE-constrained optimization problem and I have &quot;measured data&quot; from experiment defined on all nodes of my mesh. I want to impose part of the &quot;measured data&quot; to my Dirichlet boundary condition and impose 0 Neumann boundary condition to the nodes where the &quot;measured data&quot; is not specified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some reading, I think I can read in this &quot;measured data&quot; together with mesh as meshdata or mesh_function, since it is defined on mesh nodes. It seems possible, though I haven't figured out how to do this exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even with that &quot;measured data&quot; loaded in, I still don't know how to impose them on part of boundary nodes as Dirichlet boundary conditions and impose 0 Neumann bc for the rest of the boundary nodes. Applying them pointwisely seems to be very inefficient. Tagging subdomains doesn't seem to apply to this situation very well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in my case, I 'd like to have an array, &lt;strong&gt;measuredData&lt;/strong&gt; that is of the same size as the coordinates &lt;strong&gt;x&lt;/strong&gt;, so that I can do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mesh = UnitSquareMesh(3,2)&#xA;V = VectorFunctionSpace(mesh,&quot;CG&quot;,1)&#xA;measD0 = Expression(&quot;measuredData[0]&quot;)&#xA;measD1 = Expression(&quot;measuredData[1]&quot;)&#xA;bc0 = DirichletBC(V.sub(0), measD0, on_boundary)&#xA;bc1 = DirichletBC(V.sub(1), measD1, on_boundary)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But now I haven't figured out a way to overload Expression to make it see my measuredData.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keen to hear experienced fenics' users' opinions! Thanks in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: As Bill Barth clarified, the Dirichlet boundaries are well connected and so are the Neumann boundaries. So this problem is well-posed.&lt;/p&gt;&#xA;" OwnerUserId="20835" LastEditorUserId="20835" LastEditDate="2016-06-28T22:29:52.530" LastActivityDate="2016-06-28T22:29:52.530" Title="imposing &quot;measured data&quot; to Dirichlet boundary conditions in fenics" Tags="|optimization|finite-element|boundary-conditions|fenics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24330" PostTypeId="2" ParentId="24329" CreationDate="2016-06-28T21:41:40.373" Score="2" Body="&lt;p&gt;I don't know how to do this in FEciCS, but that question should probably be asked of the developers or their mailing list per this SE's policy. That being said, I'm not sure such a problem is well-posed. Maybe someone will correct me, but even if you can figure out how to enforce inter-mixed point-wise Dirichlet and Neumann conditions on the same edge or face (2d vs. 3d) in FEniCS, you may get into mathematical trouble. If the condition transitions from one kind to the other on that boundary, such that the two regions are compact, connected, or something else, then I think you should be OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the conditions randomly imposed at different points, or do you have consistent regions of the boundary where they are applied?&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-06-28T21:41:40.373" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24331" PostTypeId="2" ParentId="24318" CreationDate="2016-06-28T22:34:09.443" Score="3" Body="&lt;p&gt;What about a simple nested loop to give you one octant of the solution, which can then be copied due to symmetry:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$i$ from 0 to $d+n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$j$ from 0 to $\sqrt{(d+n)^2-i^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k$ from $\sqrt{(d-n)^2-i^2-j^2}$ to $\sqrt{(d+n)^2-i^2-j^2}$,&#xA;where the minimum bound is 0 if $i^2+j^2 &amp;gt; (d-n)^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to round off to the 'smallest' integer range.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2016-06-28T22:34:09.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24332" PostTypeId="2" ParentId="24321" CreationDate="2016-06-29T07:35:07.563" Score="3" Body="&lt;p&gt;The technique of discretising the spatial derivatives in a problem and leaving the temporal evolution as continuous is known as the &quot;method of lines&quot;, there's lots of stuff on google about it, e.g. &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines&quot; rel=&quot;nofollow&quot;&gt;http://www.scholarpedia.org/article/Method_of_lines&lt;/a&gt;. It's known as the method of lines because you literally solve the problem by solving individual equations (ODEs) along lines of constant $x$ (say).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One other possible advantage to using method of lines (besides those mentioned in the other answer) is avoiding the problem of &lt;strong&gt;consistency&lt;/strong&gt;. When solving a PDE you can encounter the problem where your approximation of it is inconsistent with the PDE that you're trying to solve, and so you actually end up solving the &lt;em&gt;wrong&lt;/em&gt; PDE. But by reducing the PDE to a system of ODEs you avoid the possibility of this happening (there is no equivalent of inconsistency when solving ODEs).&lt;/p&gt;&#xA;" OwnerUserId="20776" LastActivityDate="2016-06-29T07:35:07.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24334" PostTypeId="2" ParentId="24252" CreationDate="2016-06-29T09:15:17.187" Score="5" Body="&lt;p&gt;&lt;strong&gt;Numerical judgement of model choice:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You model 36 observations with a model consisting of 12 or 13 predictor variables. This is most likely not a good model. Even if you reach a high $R^2_{adj}$, you most likely model a random pattern. Try to compare a computed $AIC$ (Akaike information criterion) or $BIC$ (Bayesian information criterion)  of this model to the one of a  simple sinus function with 2 parameters. You might see that your model gets rejected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Explain-ability of model choice:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general in Statistics you want to be able to explain the interaction of variables in your model. For example you analyze Death Rates $DR_i$ of a population $P_i$. You find out that $DR_i$ increases exponentially with extreme outside temperatures $T$. So you want to fit a model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$DR_i = T_{Deviation}*(T -T_{mean})²$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you would most likely find biological reasons, for example $T_{mean}$ reflects the average temperature of the natural surroundings of $P_i$. The other fitted model parameter $t_{Deviation}$ reflects $P_i$'s resistance (warm or cold blooded). $P_i$ is cold blooded an so its fitted $t_{Deviation}$  is larger than the one of warm blooded species $P_j$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Re-usability of your model:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not very often the best looking fit the best statistical fit. Extrapolate your prediction line and see if it acts like you would expect it to do or not. Even if you do not have to predict extrapolated values you can take this as a measurement how well the model fits the natural circumstances. If no extrapolation is possible, most likely a second data set of measurements would not work on your model. Such a model is not transferable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a limited number of fields you can ignore such stuff of course. 3-D reconstruction of surfaces in CAD for example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Online resource to generate models from 2d data sets:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will find a regression tool for nonlinear fits with up to three variables &#xA;&lt;a href=&quot;http://www.xuru.org/rt/NLR.asp#Manually&quot; rel=&quot;nofollow&quot; title=&quot;Online Nonlinear Regression -  accessed on 02.07.2016&quot;&gt;here&lt;/a&gt;. You type in your $x,y$ matrix and it predicts a huge variety of different models.&#xA;I think earlier versions had AIC included but maybe it is also somewhere else on this page.&lt;/p&gt;&#xA;" OwnerUserId="20162" LastEditorUserId="20162" LastEditDate="2016-08-13T21:11:41.730" LastActivityDate="2016-08-13T21:11:41.730" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24335" PostTypeId="1" CreationDate="2016-06-29T20:52:09.050" Score="1" ViewCount="35" Body="&lt;p&gt;I am working on moving least square approximation. To well-define of this method, the following matrix of size $n \times m$ must be of rank $m$.&#xA;For $m=3$&#xA;$${\bf A }=\begin{pmatrix}&#xA;1&amp;amp;x_1&amp;amp;y_1\\&#xA;1&amp;amp;x_2&amp;amp;y_2\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots\\&#xA;1&amp;amp;x_n&amp;amp;y_n&#xA;\end{pmatrix},$$&#xA;where $x_i,y_i$ are point's coordinates lie in the supprt and $n$ is the  number of them.&#xA;Also, for $m=6$&#xA;$${\bf A }=\begin{pmatrix}&#xA;1&amp;amp;x_1&amp;amp;y_1&amp;amp;x_1^2&amp;amp;x_1y_1&amp;amp;y_1^2\\&#xA;1&amp;amp;x_2&amp;amp;y_2&amp;amp;x_2^2&amp;amp;x_2y_2&amp;amp;y_2^2\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots \\&#xA;1&amp;amp;x_n&amp;amp;y_n&amp;amp;x_n^2&amp;amp;x_n y_n&amp;amp;y_n^2&#xA;\end{pmatrix}.$$&#xA;If for the first one, there are only $3$ points in the support and these points are the vertices of a triangle, can we sure that $rank(A)=3?.$&#xA;Does anyone what the best strategy for finding support size is? For uniform distributed points usually we can take $r=1.1*\delta$  for $m=3$ and $r=2.1*\delta$ for $m=6$, where $\delta$ is the distance between points. But for a mesh obtained by the adaptive technique or non-uniform this is an important thing that should be choose carefully not too large and not too small.&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2016-06-29T20:57:13.880" LastActivityDate="2016-06-29T20:57:13.880" Title="Size of the support" Tags="|mesh-generation|meshless-methods|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24337" PostTypeId="2" ParentId="24322" CreationDate="2016-06-29T22:18:47.323" Score="2" Body="&lt;p&gt;This question is more related with mathematics than Scientific Computing. But I will mention some things that come to my mind. I am not trying to be rigorous here, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are mentioning more than one thing here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You are changing your domain. In one case I suppose that you have $x \in [0,1]$, but for the other one $x \in [0, \infty)$.&lt;/li&gt;&#xA;&lt;li&gt;The effect of having different boundary conditions.&lt;/li&gt;&#xA;&lt;li&gt;Uniqueness dependance on boundary conditions.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Regarding &lt;strong&gt;1&lt;/strong&gt;, you need to consider that the functions that you allow in your solution are different when you change the domain. For one you would have something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u = A e^{-kx} + B e^{kx},\quad x \in [0,1],$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the second you will enforce $B = 0 $, to satisfy $\lim_{x\rightarrow \infty} u &amp;lt; \infty$, thus&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u = A e^{-kx} \quad x \in [0,\infty),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;assuming that $k$ is positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;strong&gt;2&lt;/strong&gt;, I can mention that the boundary conditions change the values that your constants $A$ and $B$ would take, assuming that the solution is unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About &lt;strong&gt;3&lt;/strong&gt;, for some boundary conditions the solutions is not unique. Your problem can have infinite solutions, e.g., $\frac{d^2 u}{dx^2} = 0$, with Neumann conditions at both sides. Your problem can have zero solutions if your boundary conditions cannot be satisfied. And, I suppose, that you can have multiple solutions for non-linear boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2016-06-29T22:18:47.323" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24338" PostTypeId="1" AcceptedAnswerId="24339" CreationDate="2016-06-29T23:38:05.883" Score="6" ViewCount="214" Body="&lt;p&gt;I want to find the roots for $\kappa$ for the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sqrt{\alpha - 1} \cos{\left (\frac{\sqrt{2} \sqrt{\alpha - 1}}{2 \sqrt{\epsilon}} \right )} \cosh{\left (\frac{\sqrt{2} \sqrt{\alpha + 1}}{2 \sqrt{\epsilon}} \right )} - \sqrt{\alpha - 1} \\&#xA; -\frac{1}{\sqrt{\alpha + 1}} \sin{\left (\frac{\sqrt{2} \sqrt{\alpha - 1}}{2 \sqrt{\epsilon}} \right )} \sinh{\left (\frac{\sqrt{2} \sqrt{\alpha + 1}}{2 \sqrt{\epsilon}} \right )} = 0 \enspace ,$$&#xA;where $\alpha=\sqrt{1 + 4\epsilon\kappa^2}$. This equation has infinite roots, but I am interested in the first $N$ of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option to solve this problem is to use Newton method, the problem is to define the initial points, since the values of the function can be quite high, as can be seen below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IIqUy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IIqUy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem comes from finding the eigenvalues of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[\frac{d^2 u}{ds^2} - \epsilon \frac{d^4u}{ds^4} + \kappa^2\right]u = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then I can obtain an approximation using perturbation methods, i.e., the eigenvalues are approximated by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa_n^2 = n^2\pi^2 + \epsilon n^4\pi^4$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for small $\epsilon$. Then, for small values of $\epsilon$, I can use the values $\kappa_n^2$ as initial guesses to the Newton algorithm. But when $\epsilon$ increases these initial guesses fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I know the original differential equation, I can use FEM or FDM to find the eigenvalues, but I am interested in other methods. Below, you can see the comparison using FDM (1001 points), perturbation solution and Newton method (using the perturbation solution as initial guess). All the curves are for $\epsilon=10^{-3}$, but I couldn't make those guesses to work for greater values of $\epsilon$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CQn7o.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CQn7o.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Is there any other method to solve this problem? Maybe some kind of transformation that can be applied to the equation?&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-03-21T03:20:56.977" LastActivityDate="2017-03-21T03:20:56.977" Title="Roots of a function for eigensystem" Tags="|eigensystem|newton-method|roots|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24339" PostTypeId="2" ParentId="24338" CreationDate="2016-06-30T00:38:56.860" Score="5" Body="&lt;p&gt;I suspect the main problem is the magnitude of the values. If you divide through by $\cosh$ to make all the numbers smaller, then &lt;a href=&quot;https://github.com/ApproxFun/ApproxFun.jl&quot;&gt;ApproxFun&lt;/a&gt; doesn't seem to have a problem finding all the roots.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function lhs(ε, α)&#xA;    A, B = sqrt((α-1)/(2ε)), sqrt((α+1)/(2ε))&#xA;    return sqrt(α*α-1)*(cos(A) - 1/cosh(B)) - sin(A)*tanh(B)&#xA;end&#xA;&#xA;u = Fun(x -&amp;gt; lhs(1e-3, x), [1.0, 10.0], 1024)&#xA;r = ApproxFun.roots(u)&#xA;r2 = [fzero(x -&amp;gt; lhs(1e-3, x), r[i]*(1+1e-6*rand())) for i in 1:length(r)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This finds all the roots in $\alpha\in[1,10]$, for $\epsilon=10^{-3}$, $\kappa^2\lesssim 25000$, using an 1024-term Chebyshev series, taking a very small amount of time. This seems to be accurate to about $10^{-8}$ and the roots can be polished off with Newton's method or similar.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-06-30T00:38:56.860" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24340" PostTypeId="2" ParentId="24264" CreationDate="2016-06-30T17:07:00.610" Score="0" Body="&lt;p&gt;So I have managed to find two approaches for solving the coupled system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Solve equations (2) - (4) using a standard ODE solver with an initial condition for $f_0=f(0,T)$ and $g_0=g(0)$. The newly calculated value of $q_0$ can then be substituted into equation (1) to find $f$. Use fixed point iteration with under-relaxation until the residuals for $f$ and $g$ converge and then progress to the next space step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Convert the system to a DAE system by moving everything to the RHS in (2-5) and discretising all terms in time using finite differences. This can be solved using a DAE solver, and specifying a mass matrix with zeros along the diagonal for (2-5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both methods work, however method (1) takes many iteration to converge, even with under-relaxation and method (2) becomes extremely demanding on memory when there are many time points&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following suggestions in the comments, another method is to discretise in $X$, rather than in $T$ and time-march, leading to the following ODE system (after some re-arrangement):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial f}{\partial T} = \gamma$$&#xA;$$\frac{\partial}{\partial T}(\frac{f^2}{2} + B \gamma) = \frac{\partial f}{\partial X} + ...$$&#xA;$$ ... $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\gamma$ is a new variable to remove the 2nd order derivative in (1). However, I am unsure how to treat the term: $\frac{f^2}{2} + B \gamma$ on the LHS. Does anyone have any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="20752" LastEditorUserId="20752" LastEditDate="2016-07-05T10:02:59.370" LastActivityDate="2016-07-05T10:02:59.370" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="24341" PostTypeId="1" CreationDate="2016-06-30T17:52:05.313" Score="2" ViewCount="81" Body="&lt;p&gt;I am solving a partial differential algebraic equation (PDAE) system which has the following dependent variables:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f=f(X,T)$ and $g=g(T)$, along with a few others&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current method for coupling is to use fixed-point iteration with constant under-relaxation factors for the two coupled variables $f$ and $g$. The issue is that convergence (determined from the norm of the residuals) is too slow. To resolve this I have tried to implement dynamic under-relaxation using Aitken's method. However, this doesn't seem to offer any speed-up - it actually often requires more iterations to converge than using constant relaxation factors. This is the case even if I apply upper and lower bounds on the relaxation factors calculated by Aitken's method. Reducing the starting relaxation factors to small values (~0.01) also doesn't seem to help much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial thoughts are that the Aitken's method is not accounting for the coupling between the two variables - the relaxations factors $\omega_f$ and $\omega_g$ are calculated with no consideration for the variation of each other. I have tried having a look at the literature, but I can only find the application of Aitken's method for dynamic relaxation for a single variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions for improving the performance of Aitken's method in this case would be much appreciated&lt;/p&gt;&#xA;" OwnerUserId="20752" LastActivityDate="2016-06-30T17:52:05.313" Title="Dynamic Successive Over/under Relaxation (SOR) with several variables" Tags="|pde|iterative-method|convergence|coupling|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24342" PostTypeId="1" AcceptedAnswerId="27199" CreationDate="2016-06-30T17:56:54.757" Score="4" ViewCount="379" Body="&lt;p&gt;A problem I have had on my mind recently has been a compact way to compute the size of an $N$-Dimensional Polynomial basis of some order $p$, where a linear basis is $p=1$. I have attempted searching for such a formula online but haven't been able to stumble upon one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to believe a formula for this computation exists, though, because this should be a question that statisticians or others who build higher dimensional models ask when figuring out what polynomial basis order to use, based on the size of their data sets, when they want to build a model using polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to know is, is there any compact formula for this question that I have not been able to find via my search online? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another related question is, is there a clever algorithmic way to represent and compute individual $N$-Dimensional polynomial basis terms such that one could target a specific basis function in the set using a single index? Because then one could do a simple loop through all the basis functions to do any necessary computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I am fine with someone mentioning these answers don't really need to be answered explicitly due to some reasoning. It's just that, as far as my experience has shown, these questions seem important to answer to tackle certain modeling problems based on polynomials. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like how do I estimate what polynomial order to use to model a data set to avoid overfitting (without explicit regularization)? Do people just avoid answering this question by using something like K-fold Cross Validation to find the appropriate polynomial order? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then, how can I compactly program software that can represent an $N$-D order $p$ polynomial basis in a way that could simply be looped through without needing to store data for each polynomial term? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the underlying questions of this post I am trying to get answered.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-07-05T20:56:30.133" LastActivityDate="2021-08-11T20:41:16.110" Title="Computing size of N-Dimensional Polynomial Basis and Efficient Representation of Basis" Tags="|polynomials|basis-set|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24343" PostTypeId="1" CreationDate="2016-06-30T20:28:04.593" Score="2" ViewCount="1026" Body="&lt;p&gt;In what sense is the weighted Frobenius norm &quot;adimensional&quot;/&quot;scale-invariant&quot; for any symmetric positive definite weight matrix $W$? If we plug in a positive diagonal matrix into $W$ wee see that $||A||_W$ is not always $||TA||_W$ for a one-to-one transformation $T$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See page 196 of Nocedal's book on numerical optimization and page 60 of Fletcher's &quot;Practical methods of numerical optimization&quot; for reference.&lt;/p&gt;&#xA;" OwnerUserId="20853" LastEditorUserId="9667" LastEditDate="2016-06-30T21:57:28.033" LastActivityDate="2016-06-30T21:57:28.033" Title="Weighted Frobenius norm in BFGS" Tags="|optimization|quasi-newton|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="24344" PostTypeId="1" CreationDate="2016-06-30T21:00:51.523" Score="3" ViewCount="359" Body="&lt;p&gt;I'm working on a MD simulation using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lennard-Jones_potential&quot; rel=&quot;nofollow&quot;&gt;Lennard-Jones potential&lt;/a&gt; and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Verlet_integration#Velocity_Verlet&quot; rel=&quot;nofollow&quot;&gt;Velocity Verlet&lt;/a&gt; algorithm. The code compiles and runs, but the energy for each time step is not conserved; it decreases constantly when run. The magnitude of the potential energy is much larger than that of the kinetic energy, but I haven't been able to detect where the error is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help you can offer!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.H&amp;gt;    &#xA;#include &amp;lt;math.H&amp;gt;&#xA;#include &amp;lt;stdlib.H&amp;gt;&#xA;struct particle&#xA;{&#xA;    double x[3],v[3],a[3];&#xA;};&#xA;&#xA;double sigma, epsilon, m, dt;&#xA;#define N 1000&#xA;struct particle particles[N];&#xA;&#xA;double rfinder(int id_a, int id_b)&#xA;{&#xA;    int i;&#xA;    double rtot,dist,dist2;&#xA;    dist2 = 0;&#xA;    for(i=0;i&amp;lt;3;i++)&#xA;    {&#xA;        dist = particles[id_a].x[i] - particles[id_b].x[i];&#xA;        dist2 += dist * dist;&#xA;    }&#xA;    rtot = sqrt(dist2);&#xA;    return rtot;&#xA;}&#xA;&#xA;double calcacceleration()&#xA;{&#xA;    int b,c,i;&#xA;    double dist,r,r7,r13,sigma6,sigma12, r12, sor6, sor12, pe;&#xA;    pe = 0;&#xA;    for(b=0;b&amp;lt;N;b++)&#xA;    {&#xA;        for(i=0;i&amp;lt;3;i++)&#xA;        {&#xA;            particles[b].a[i] = 0;&#xA;        }&#xA;    }&#xA;&#xA;    for(b=0;b&amp;lt;N;b++)&#xA;    {&#xA;        for(c=0;c&amp;lt;N;c++)&#xA;        {&#xA;            if(b!=c)&#xA;            {&#xA;                double dist2;&#xA;                dist2 = 0;&#xA;                for(i=0;i&amp;lt;3;i++)&#xA;                {&#xA;                    dist = particles[b].x[i] - particles[c].x[i];&#xA;                    dist2 += dist * dist;&#xA;                }&#xA;                if (dist2&amp;lt;8*8)&#xA;                {&#xA;                    r = sqrt(dist2);&#xA;                    //printf(&quot;%d %d %f\n&quot;,b,c,r);&#xA;                    double rinv = 1.0/r;&#xA;                    double r2 = rinv*rinv;&#xA;                    double r4 = r2*r2;&#xA;                    double r6 = r4 * r2;&#xA;                    r7 = r4*r2*rinv;&#xA;                    r13 = r7*r4*r2;&#xA;                    r12 = r6 * r6;&#xA;                    sigma6 = sigma*sigma*sigma*sigma*sigma*sigma;&#xA;                    sigma12 = sigma6*sigma6;&#xA;                    sor6 = sigma6*r6;&#xA;                    sor12 = sigma12*r12;&#xA;                    for(i=0;i&amp;lt;3;i++)&#xA;                    {&#xA;                        dist = particles[b].x[i] - particles[c].x[i];&#xA;                        particles[b].a[i] += -(0.5 * (4 * epsilon *((-12 * sigma12 * r13) +(6 * sigma6 * r7))) * ((dist/r))/m);&#xA;                        particles[c].a[i] +=  (0.5 * (4 * epsilon *((-12 * sigma12 * r13) +(6 * sigma6 * r7))) * ((dist/r))/m);&#xA;                        pe += 0.5*(4*epsilon*(sor12 - sor6));&#xA;                    }&#xA;                }&#xA;            }&#xA;        }&#xA;    }&#xA;    return pe;&#xA;}&#xA;&#xA;void position_mover()&#xA;{&#xA;    double x, x0, v;&#xA;    int p, i;&#xA;    for(p=0;p&amp;lt;N;p++)&#xA;    {&#xA;        for(i=0;i&amp;lt;3;i++)&#xA;        {&#xA;            particles[p].x[i] = particles[p].x[i] + (particles[p].v[i] * dt);&#xA;        }&#xA;    }&#xA;    return;&#xA;}&#xA;&#xA;double velocity_mover()&#xA;{&#xA;    double ke;&#xA;    int p, i;&#xA;    ke = 0;&#xA;    for(p=0;p&amp;lt;N;p++)&#xA;    {&#xA;        for(i=0;i&amp;lt;3;i++)&#xA;        {&#xA;            particles[p].v[i] = particles[p].v[i] + (0.5 * particles[p].a[i] * dt);&#xA;            ke += 0.5 * m * particles[p].v[i] * particles[p].v[i];&#xA;        }&#xA;    }&#xA;    return ke;&#xA;}&#xA;&#xA;void output_to_file()&#xA;{&#xA;int p;&#xA;FILE * fp;&#xA;fp = fopen(&quot;data.xyz&quot;, &quot;a&quot;);&#xA;fprintf(fp,&quot;%d \n\n&quot;,N);&#xA;for(p=0;p&amp;lt;N;p++)&#xA;{&#xA;    fprintf(fp,&quot;H %f %f %f\n&quot;,particles[p].x[0],particles[p].x[1],particles[p].x[2]);&#xA;}&#xA;fclose(fp);&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;int p, i, j, k;&#xA;double ft,t,pe, ke,te;&#xA;sigma = 1;&#xA;epsilon = 1;&#xA;FILE * fp;&#xA;fp = fopen(&quot;data.xyz&quot;, &quot;w&quot;);&#xA;fclose(fp);&#xA;printf(&quot;What timestep would you like?\n&quot;);&#xA;scanf(&quot;%lf&quot;, &amp;amp;dt);&#xA;printf(&quot;In seconds, what is the final time?\n&quot;);&#xA;scanf(&quot;%lf&quot;, &amp;amp;ft);&#xA;printf(&quot;What mass would you like?\n&quot;);&#xA;scanf(&quot;%lf&quot;, &amp;amp;m);&#xA;p = 0;&#xA;//sets initial positions of atoms to be evenly spaced in a 10 * 10 * 10 cube&#xA;for(i=0;i&amp;lt;10;i++)&#xA;{&#xA;    for(j=0;j&amp;lt;10;j++)&#xA;    {&#xA;        for(k=0;k&amp;lt;10;k++)&#xA;        {&#xA;            particles[p].x[0] = i*2;&#xA;            particles[p].x[1] = j*2;&#xA;            particles[p].x[2] = k*2;&#xA;            p++;&#xA;        }&#xA;    }&#xA;}&#xA;&#xA;/*sets initial velocities and accelerations of atoms to 0 to get rid of any&#xA;garbage in memory */&#xA;for(i=0;i&amp;lt;3;i++)&#xA;{&#xA;    for(p=0;p&amp;lt;N;p++)&#xA;    {&#xA;        particles[p].v[i] = 0;&#xA;        particles[p].a[i] = 0;&#xA;    }&#xA;}&#xA;&#xA;&#xA;for(t=0;t&amp;lt;ft;t+=dt)&#xA;{&#xA;output_to_file();&#xA;calcacceleration();&#xA;velocity_mover();&#xA;position_mover();&#xA;pe = calcacceleration();&#xA;ke = velocity_mover();&#xA;te = ke + pe;&#xA;printf(&quot;Time: %f\n PE: %f\n KE: %f\n TE: %f\n&quot;,t,pe, ke, te);&#xA;}&#xA;printf(&quot;Done!\n&quot;);&#xA;return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20757" LastEditorUserId="20757" LastEditDate="2016-06-30T21:13:14.667" LastActivityDate="2016-06-30T21:13:14.667" Title="Error in MD simulation with Lennard-Jones potential in C, resulting in a lack of conservation of energy" Tags="|simulation|c|molecular-dynamics|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="24345" PostTypeId="1" CreationDate="2016-06-30T21:36:50.507" Score="2" ViewCount="1311" Body="&lt;p&gt;I need to find the smallest few eigenvalues of a Hamiltonian (&lt;a href=&quot;https://en.wikiversity.org/wiki/Quantum_Simulation/Exact_diagonalization&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;exact&lt;/em&gt; diagonalization&lt;/a&gt;) I use Python, and SciPy's built-in sparse eigenvalue solver. I notice, however, that for my small system (only a 40k x 40k matrix) the program takes hours, maybe even days. I began to suspect something was up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing I note is that my matrix should have a zero eigenvalue. Will that cause havoc in the eigenvalue solvers? If so, what is the way to remedy this?&lt;/p&gt;&#xA;" OwnerUserId="20855" LastEditorUserId="20855" LastEditDate="2016-06-30T21:58:16.047" LastActivityDate="2016-07-01T07:11:43.823" Title="Zero Eigenvalues in Lanczos Algorithm" Tags="|linear-algebra|python|eigenvalues|quantum-mechanics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24346" PostTypeId="2" ParentId="24345" CreationDate="2016-06-30T22:14:07.730" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/tutorial/arpack.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciPy tutorial&lt;/a&gt; explicitly states&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note that ARPACK is generally better at finding extremal eigenvalues: that is, eigenvalues with large magnitudes. In particular, using &lt;code&gt;which = 'SM'&lt;/code&gt; may lead to slow execution time and/or anomalous results. A better approach is to use shift-invert mode.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and goes on to describe that. Basically, if $\lambda$ is the smallest by magnitude eigenvalue of $A$, then &#xA;$\nu = \frac{1}{\lambda}$&#xA;is the &lt;em&gt;largest&lt;/em&gt; by magnitude eigenvalue of $A^{-1}$. (With the same trick, you can get the eigenvalue closest to a given $\sigma$ by looking at the largest for $(A-\sigma I)^{-1}$.) Hence, to compute, say, the three smallest by magnitude eigenvalues, you can instead use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eigenval, eigenvec = eigsh(A, 3, sigma=0, which='LM')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you already know that $A$ has a zero eigenvalue and is therefore not invertible (and you have some idea where the next closest are), you can instead use a nonzero shift, e.g., &lt;code&gt;sigma=0.01&lt;/code&gt; to shift your matrix away from singular. If &lt;code&gt;sigma&lt;/code&gt; is small enough, that should still get you the ones closest to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Also, as noted in my comment, make sure you have a proper optimized BLAS such as OpenBLAS installed -- for sparse eigenvalues, SciPy calls ARPACK (written in Fortran) which in turn makes heavy use of BLAS.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; If it's the smallest &lt;em&gt;algebraic&lt;/em&gt; eigenvalue you're after, you can use &lt;a href=&quot;https://scicomp.stackexchange.com/a/21742/1804&quot;&gt;this neat trick&lt;/a&gt;: If you have an estimate $\sigma$ of the largest eigenvalue (say, by computing it using &lt;code&gt;which='LM'&lt;/code&gt;), you can use that to shift all eigenvalues to be negative -- so that the smallest algebraic eigenvalues are also the largest magnitude ones. Then simply use &lt;code&gt;which='LM'&lt;/code&gt; on $A-\sigma I$ and shift back.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-07-01T07:11:43.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24347" PostTypeId="2" ParentId="24345" CreationDate="2016-06-30T22:53:58.463" Score="3" Body="&lt;p&gt;While the smallest eigenvalues of a sparse 40k x 40k matrix might be obtained in a reasonable amount of time using ARPACK and some handy tricks through the SciPy wrapper, as mentioned here, the problem will scale at best as $N^2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any larger eigenvalue problems I almost always recommend &lt;a href=&quot;http://slepc.upv.es/&quot; rel=&quot;nofollow&quot;&gt;SLEPc&lt;/a&gt;, which also has a Python wrapper (through perhaps not as natural as the C versions) and in many cases calls parallel versions of the routines SciPy would.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-06-30T22:53:58.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24348" PostTypeId="1" CreationDate="2016-07-01T03:37:49.310" Score="2" ViewCount="268" Body="&lt;p&gt;During the past ~4.5 years of my PhD, I have written an MD code, based largely on LAMMPS and NAMD, however reparallelized to suit my computational scenario better. One important thing to mention possibly for this discussion is that the code runs in SPDP mode to take advantage of the wide vector registers on xeon phi coprocessors. I am currently working with acetaminophen in water, however I cannot seem to rationalize the behavior I am observing with the temperature vs time data. I am using GAFF force field for acetaminophen and tip3p for water. The langevin thermostat is being used for temperature maintenance, and is being ran with a relaxation time of 2 ps. The shake algorithm is being used to constrain the motion of all bonded hydrogens. Currently, the system is a 2.75nm acetaminopehn nanocrystal embedded in a previously equilibrated box of acetaminopen + tip3p water. These are fairly large simulation I am trying to run, and consist of about 2600 acetaminophen molecules in about 30K water molecules for about 150K total atoms.  I have noticed that the system equilibrates at ~321 K, even though the set point is 320 K. I have figured out through testing that the set point offset is somehow related to the shake algorithm being applied to the solid nanocrystal. Currently,  I have the following test cases. I have more plots to show, but the website wont let me post more than 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;solid liquid shake : this is the case where shake is applied to all hydrogens in the system (solid acetaminophen nanocrystal + liquid acetaminophen +. The system was ran in the NPT ensemble with langevin thermostat (2 ps relaxation time) and berendsen barostat (1 ps relaxation time). The timestep was 1.0fs. As you can see from the first image, the system equilibrates at ~321 K, and is clearly above the 320 K set point.&lt;a href=&quot;https://i.stack.imgur.com/ddSx0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ddSx0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fluid-system shake: To test if there is a bug in my shake code that appears only when applied to acetaminophen, I ran an equilibration of a fluid mixture of acetaminophen and water. This system has no embedded solid sphere, and was generated using the insert-molecules and solvate command from gromacs. The following image shows the system equilibrating at 320 K correctly. Not sure what is so specific about the embedded sphere case thats making it bug out.&lt;a href=&quot;https://i.stack.imgur.com/LjJhi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LjJhi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one final thing of note is that I have tried varying the friction constant of the langevin thermostat. Increasing the friction accelerations temperature convergence to the steady state value, but does not affect the steady state value itself (i.e. 321 K)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My specific questions are (1) Do others share any concern regarding the system equilibrating at a slightly higher (~0.8-1.0 K) temperature than the set point. I.E. Is it a general phenomenon for the shake algorithm to possible induce temperature set point offset? Is this kind of offset even considered strange? (2) Does anyone see something I am missing? I have been pounding my head on a desk for about a week now. Do others have thoughts as to why the shake algorithm, when applied to my embedded solid nanocrystal, causes the system to miss the set point, but not when the shake algorithm is applied to an acetaminophen + water fluid mixture? Any help is greatly appreciated. I am the only one in my research group who does MD, and cant ask these kind of very detailed questions to anyone else.&lt;/p&gt;&#xA;" OwnerUserId="20859" LastEditorUserId="9667" LastEditDate="2016-07-01T13:20:41.587" LastActivityDate="2016-07-01T13:20:41.587" Title="temperature set point maintenance with Langevin thermostat + shake algorithm" Tags="|molecular-dynamics|molecular-mechanics|lammps|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24349" PostTypeId="1" AcceptedAnswerId="24354" CreationDate="2016-07-01T10:09:36.277" Score="5" ViewCount="137" Body="&lt;p&gt;Admittedly, I'm new to the subject, so this is probably a really simple question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume I want to solve the (large sparse) linear system Ax = b multiple times with algebraic multigrid and b changes (slightly) every time. As I understand it, AMG selects its coarse grid based on the smoothness of the error, which depends on b, so this would have to be done anew every time b changes, although A stays the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct? And isn't this a huge bargain compared to geometric multigrid (when applicable)?&lt;/p&gt;&#xA;" OwnerUserId="20867" LastActivityDate="2016-07-01T19:59:07.003" Title="Does algebraic multigrid reuse its coarse grids?" Tags="|linear-solver|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24350" PostTypeId="1" CreationDate="2016-07-01T12:11:40.753" Score="0" ViewCount="423" Body="&lt;p&gt;I wrote a Matlab code for solving 2D Poisson equation $u_{xx} + u_{yy} + f(x,y) = 0$ on $[a,b]\times [c,d]$ with neumann boundary condition on $x = b$  and the other boundary conditions are dirichlet,I used finite difference methods.My code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [u,err] = poissonFDM(a,b,c,d,j1,j2,fun,g,typebound,bound,uex,nit)&#xA;dltx = (b - a)/(j1+1); dlty = (d - c)/(j2+1);&#xA;switch typebound &#xA;    case 'Neumann'&#xA;        dim = (j1+1)*j2;&#xA;        D = speye(j1+1);&#xA;        T = ones(j1+1,1); &#xA;        T = [T,-4*T,T]; T(j1,1) = 2*T(j1,1);&#xA;        T = spdiags(T,[-1;0;1],j1+1,j1+1);&#xA;        A1 = D;A = T;&#xA;        for i = 1 :((j2)- 2)&#xA;             A = blkdiag(A,T);&#xA;             A1 = blkdiag(A1,D);&#xA;        end&#xA;        A = blkdiag(A,T);&#xA;        A(1:(j1+1)*(j2-1),j1+2:(j1+1)*j2) = A(1:(j1+1)*(j2-1),j1+2:(j1+1)*j2) + A1;&#xA;        A(j1+2:(j1+1)*j2,1:(j1+1)*(j2-1)) = A(j1+2:(j1+1)*j2,1:(j1+1)*(j2-1)) + A1;&#xA;        h = max(dltx,dlty);&#xA;        xh = (a:dltx:b); yh = (c:dlty:d)';&#xA;        boundL = g(a,yh);&#xA;        boundB = g(xh,c);boundU = g(xh,d);&#xA;        gR = bound(b,yh);&#xA;        k = 1;&#xA;        v = zeros(dim,1);f = zeros(dim,1);Uex = zeros((j1+1)*j2,1);&#xA;        for s = 1:j2&#xA;             for r = 1:(j1+1)&#xA;             f(r,s) = fun(r*dltx + a,s*dlty + c);&#xA;                if(nargin == 12)&#xA;                    Uex(k) = uex(r*dltx+a,s*dlty+c);&#xA;                end&#xA;             v(k) = -(h^2)*f(r,s);   &#xA;             k = k + 1;&#xA;             end&#xA;        end&#xA;        %Bottom Border     &#xA;        v(1:j1) = v(1:j1) - boundB(2:j1+1)';&#xA;        v(j1+1) = v(j1+1) - gR(1);&#xA;        %Left Border&#xA;        v(1:j1:j1*j2 - j1 + 1) = v(1:j1:j1*j2 - j1 + 1) - boundL(2:j2+1);&#xA;        %Right Border&#xA;        v(j1+1:j1+1:(j1+1)*j2) = v(j1+1:j1+1:(j1+1)*j2) - 2*h*gR(2:j2+1);&#xA;        %Up Border&#xA;        v((j1+1)*j2-j1:(j1+1)*j2 -1) = v((j1+1)*j2-j1:(j1+1)*j2-1) - boundU(2:j1+1)';&#xA;        v((j1+1)*j2)= v((j1+1)*j2 ) - gR(j2+2);&#xA;        %solving linear system with conjugate gradiant&#xA;        %U = conjgrad(A,v);&#xA;        U = A\v;&#xA;        %err = U - Uex;&#xA;end &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I doubt about accuracy of my code ,I suppose that $u = \sin(x+y)$.when I test my code with this :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u = poissonFDM(0,pi,0,pi,11,11,@(x,y)2*sin(x+y),@(x,y)sin(x+y),'Neumann',@(x,y)cos(x+y),@(x,y)sin(x+y));&#xA;surf(u)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so the graph will be this :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eamMC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eamMC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this graph is far from the graph of $ u = sin(x+y)$. What is wrong with my code ??and about the method I used ,I should say that I approximate all the derivatives with the central finite difference formula!Can anyone help me? &lt;/p&gt;&#xA;" OwnerUserId="20528" LastActivityDate="2016-07-06T11:34:16.750" Title="What is wrong with my code for solving Poisson equation with one side Neumann boundary condition?" Tags="|matlab|finite-difference|poisson|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24351" PostTypeId="1" CreationDate="2016-07-01T17:39:18.647" Score="1" ViewCount="45" Body="&lt;p&gt;Looking at the plain heat equation $u_t=u_{xx}$ the explicit scheme for it would look like the following iteration:&#xA;$$u_{m,n+1}=\rho u_{m-1,n}+(1-2\rho)u_{m,n}+\rho u_{m+1,n}$$&#xA;I noticed this equation resembles some probability equation: basically If I look at it as:&#xA;$$u_{m,n+1}=p_1 u_{m-1,n}+p_2u_{m,n}+p_3u_{m+1,n}$$&#xA;I notice $p_1+p_2+p_3=1$ and if I restrict those numbers to be between $[0,1]$ if as in case of probabilities it matches the stability criteria. Is there some sort of connection between probability and FDM here? It looks to me as $u_{m,n+1}$ is some sort of expectations but I can't make my argument complete so I would appreciate some help on that.&lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2016-07-01T17:39:18.647" Title="explicit scheme stability restriction" Tags="|parabolic-pde|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24352" PostTypeId="2" ParentId="24350" CreationDate="2016-07-01T18:50:36.920" Score="1" Body="&lt;p&gt;It's unclear what the problem is to begin with. Why do you doubt the accuracy of your solution? If you substitute the numerical result into the governing equation do you return a zero residual? A central difference implementation should be fine for this problem, except at the boundaries of course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try simplifying your problem to find the error: remove the forcing term $f(x,y)$ so you're solving a Laplace equation, simplify the boundary conditions (Dirichlet on all sides perhaps), reduce the order so you're solving the 1D problem, and so fourth.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-07-01T18:50:36.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24353" PostTypeId="2" ParentId="10675" CreationDate="2016-07-01T19:03:56.830" Score="1" Body="&lt;p&gt;There are &lt;em&gt;many&lt;/em&gt; methods to solve your problem, and choosing the appropriate one is sometimes more of an art than anything else.  A few things are definitely true, though:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$10^5 \times 10^5$ is not unthinkably large in the world of scientific computing, there are many tools to deal with problems of this size &lt;/li&gt;&#xA;&lt;li&gt;GPU-accelerated eigenvalue solvers are still relatively new and typically not the preferred tool&lt;/li&gt;&#xA;&lt;li&gt;It is too large to fit onto a single compute node or GPU, which brings about the problem of scalability&lt;/li&gt;&#xA;&lt;li&gt;You should distribute your problem across multiple nodes to deal with the memory storage issue, many libraries deal with this and the gold standard is &lt;a href=&quot;http://slepc.upv.es/&quot; rel=&quot;nofollow&quot;&gt;SLEPc&lt;/a&gt; which has several routines that you can experiment with&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The crux: it's not clear what method is quickest for finding those eigenvalues, often depending on the matrix structure and conditioning, but the appropriate tools are available to find out.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-07-01T19:03:56.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24354" PostTypeId="2" ParentId="24349" CreationDate="2016-07-01T19:59:07.003" Score="3" Body="&lt;p&gt;In algebraic multigrid there are usually two steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Setup: Here we compute the $A$ matrices and interpolation matrices ($W$, $W'$) at each grid level. This is based on computing c-points and f-points which are entirely derived from information from the matrix $A$ at each level. We do not need the right hand side $b$ vector to complete this setup step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Solve: Once we have our A matrices and interpolation matrices at each level we perform the recursive multigrid algorithm:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;do $n_{1}$ smoothing steps on $Ax=b$&lt;/li&gt;&#xA;&lt;li&gt;compute residual $r=b-Ax=Ae$&lt;/li&gt;&#xA;&lt;li&gt;compute $W'r$ &lt;/li&gt;&#xA;&lt;li&gt;set $e = 0$&lt;/li&gt;&#xA;&lt;li&gt;recursively solve $A_{c}e_{c}=W'r$ ($A_{c}$ and $e_{c}$ are the coarse level $A$ matrix and error vector)&lt;/li&gt;&#xA;&lt;li&gt;correct $x = x + We_{c}$&lt;/li&gt;&#xA;&lt;li&gt;do $n_{2}$ smoothing steps on $Ax=b$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In summary the setup step can be done once for many different choices of right hand side vector $b$ because finding the $A$ matrices and interpolation matrices ($W$, $W'$) at each level does not involve the right hand side vector, but rather only information from the $A$ matrix.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2016-07-01T19:59:07.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24356" PostTypeId="1" CreationDate="2016-07-01T20:20:12.747" Score="0" ViewCount="190" Body="&lt;p&gt;Actually I am reading a book about the Lattice Boltzmann methods, and here is a quotation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;After introducing BGKW approximation, the Boltzmann equation (without external forces) can be approximated as,  $$\dfrac{\partial f}{\partial t}+c\cdot \nabla f=\dfrac1\tau\left(f^{eq}-f\right)\tag{1}$$&#xA;  In lattice Boltzmann method, the above equation is descretized and assumed it is valid along specific diresctions, linkages. Hence, the discrete Boltzmann equation can be written along a specified direction as,&#xA;  $$\dfrac{\partial f_i}{\partial t}+c_i \nabla f_i=\dfrac1\tau\left(f_i^{eq}-f_i\right)\tag{2}$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I cannot understand how can I pass from equation $(1)$ to $(2$) (notice also the absence of the dot product between $c_i$ and $\nabla f_i$) can you please explain? is it a typo?&lt;/p&gt;&#xA;" OwnerUserId="14930" LastEditorUserId="9667" LastEditDate="2016-07-01T21:08:05.520" LastActivityDate="2016-07-01T21:36:14.220" Title="Explain step in Lattice Boltzmann method" Tags="|lattice-boltzmann-methods|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24358" PostTypeId="2" ParentId="24356" CreationDate="2016-07-01T21:36:14.220" Score="1" Body="&lt;p&gt;In (1), $f$ is an unknown function of $t$, $x$ and $\xi$. Here $\xi$ denotes the moment of a particle. Note that (1) is only a scalar equation for $f=f(t,x,\xi)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From (1) to (2), we actually discretize the space of $\xi$, from continuous space $-\infty&amp;lt;\xi&amp;lt;\infty$ to discretize space $\xi_1$, $\xi_2$, $\dots$, $\xi_n$. Now (2) is a set of $n$ equations for functions $f_i=f_i(t,x)$, $i=1,\dots,n$.&lt;/p&gt;&#xA;" OwnerUserId="20711" LastActivityDate="2016-07-01T21:36:14.220" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24360" PostTypeId="1" AcceptedAnswerId="24371" CreationDate="2016-07-02T16:55:40.840" Score="0" ViewCount="186" Body="&lt;p&gt;I have a linear elastic wave propagation code and an elastoplastic wave propagation code based on FEniCS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, I keep the 2D mesh (100, 100) fineness unit square and give a source wave of $\sin(40 \pi t)$ in form of Dirichlet BC on one edge of square. For Newmark time discretization method, using $\beta = 0.25$ gives a constant acceleration method. For the linear code, in Paraview, I observe a fast propagating symmetrically distributed wave while for the elastoplastic code, I see much slower propagation and not a symmetrical propagation. It looks like its going more in 1 direction than equally spreading out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Young's modulus for both the simulations is 1 and so is the mass density. The Poisson's ratio is similar as well. The elastoplastic wave propagation has another parameter introduced from plasticity theory called 'the yield stress', i.e., after which the material experiences plastic deformation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&#xA;- What would be a good way to verify the results of both the codes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be a good idea to keep the yield stress abnormally high? If the yield stress is never exceeded, then the result should be similar to elastic wave propagation, right? Would the value of Young's modulus matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: The purpose is to simulate seismic waves.&lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="9667" LastEditDate="2016-07-03T02:24:30.917" LastActivityDate="2016-07-04T20:54:04.880" Title="What would be a simple approach to validate a wave propagation code?" Tags="|finite-element|fenics|wave-propagation|verification|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24361" PostTypeId="1" CreationDate="2016-07-02T19:12:00.547" Score="1" ViewCount="72" Body="&lt;p&gt;How difficult is it to program an automatic software using &lt;code&gt;C&lt;/code&gt; that is able to read astronomy images of up to 4096x4096 pixels (or other sensor geometries), search for bright or faint blobs, calculate its subpixel centroid, compare with the same locations of other the previous or next image, look for other blobs in those and eventually find the same one by its linear motion/equal motion intervals if the images were taken with equal time gaps in between?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, as an additional feature, take subsequent images with equal time gaps in between and overlay them such that very faint blobs become visible by signal-accumulation and noise-averaging, and do the same as outlined above until linear motions are found for the same blob.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are key skills one needs to have in &lt;code&gt;C&lt;/code&gt; in order to program this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What computer hardware would be reasonable if wanting the program to need no more than 1 minute for 3 subsequent images?&lt;/p&gt;&#xA;" OwnerUserId="20876" LastActivityDate="2016-07-03T18:44:52.400" Title="Automatic motion recognition" Tags="|software|image-processing|pattern-recognition|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24362" PostTypeId="2" ParentId="24361" CreationDate="2016-07-02T20:26:46.100" Score="2" Body="&lt;p&gt;You would greatly benefit from knowledge in signal processing, but some specific useful knowledge would be in computer vision and estimation (like Kalman filters or alpha beta filters). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to tracking blobs, you could use blob detector algorithms to find meaningful blobs, use some descriptor like SIFT to describe the blob and then do feature matching between images to figure out matching blobs. Then based on the matches and time differences between images, you could do some sort of Kalman filter estimation of speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the common libraries you can use to solve this problem, as I mentioned in my comment below, is OpenCV. It covers most common computer vision and image processing problems you might end up with. I will note that you may have to experiment with various algorithms because they don't always perform as robustly as they could. When I worked in the Computer Vision group at the Jet Propulsion Lab, I found OpenCV failed to implement various algorithms to be as fast and robust as they could be when compared to the in house algorithms made by NASA engineers. NASAs codes were far superior even for simple things like blob detection and feature descriptors. &lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-07-03T18:44:52.400" LastActivityDate="2016-07-03T18:44:52.400" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24363" PostTypeId="1" AcceptedAnswerId="24364" CreationDate="2016-07-02T22:39:11.440" Score="2" ViewCount="310" Body="&lt;p&gt;$\DeclareMathOperator{\diag}{diag}$&#xA;Consider the generalized eigenproblem $A\mathbf{x}=\lambda B\mathbf{x}$. When solving PDEs numerically (specifically, I am interested on finding the Dirichlet eigenmodes of the 2D laplacian in orthogonal coordinates, i.e. $\nabla^2 u=\lambda u$) this problem takes the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A_1X + XA_2^\text{T} = \lambda B \circ X$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $X\in\mathbb{R}^{m\times n}$ is our eigenvector (the eigenfunction $u$ sampled at the meshpoints), $A_1\in\mathbb{R}^{m\times m}$ and $A_2\in\mathbb{R}^{n\times n}$ are differentiation matrices, $B\in\mathbb{R}^{m\times n}$ is the Jacobian determinant (sampled at the meshpoints) and the symbol $\circ$ denotes  element-wise multiplication. I am only interested on the eigenmodes corresponding to the smallest eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the help of the Kronecker product, the problem can be vectorized as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (I \otimes A_1 + A_2 \otimes I)\mathbf{x} = \lambda \diag(B)\mathbf{x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{x}\in\mathbb{R}^{mn}$ is a vector formed by stacking the columns of $X$ and $\diag{(B)}$ is a diagonal matrix formed with the columns of $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with this is that the resulting matrices are unnecessarily sparse and have size of $mn\times mn$, and when I try to solve it for modestly large sizes of $m$ and $n$, say 200, my computer runs out of memory. Clearly that is not the way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that modern eigenvalue algorithms are highly sophisticated, and that coding one by myself is not the best option. So I am looking for an iterative solver that only depends on the computation of the matrix-vector products with $A$ and $B$ and that allows the user to provide them as a black-box.&lt;/p&gt;&#xA;" OwnerUserId="17612" LastActivityDate="2018-05-24T19:56:53.143" Title="Eigenvectors of Black-box matrix" Tags="|linear-algebra|pde|eigenvalues|matrix-equations|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24364" PostTypeId="2" ParentId="24363" CreationDate="2016-07-03T01:57:36.907" Score="5" Body="&lt;p&gt;I recommend &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;noreferrer&quot;&gt;ARPACK&lt;/a&gt; which provides matrix-free routines for generalized eigenvalue problems with examples in their documentation for this purpose.  This method is one of the most widely used, especially for relatively large problems where you are only searching for a few eigenvalues/vectors (such as the smallest, in your case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, eigenvalue problems of the size $10^4$ are not especially large and are likely solvable on any modern workstation. However, if this problem gets bigger the gold standard is &lt;a href=&quot;http://slepc.upv.es/&quot; rel=&quot;noreferrer&quot;&gt;SLEPc&lt;/a&gt;, which is a library containing many eigenvalue solvers and can distribute your problem across many compute nodes. &lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-07-03T01:57:36.907" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24365" PostTypeId="2" ParentId="24361" CreationDate="2016-07-03T10:09:59.927" Score="1" Body="&lt;p&gt;The answer to quantify the difficulty is not trivial to give. It depends already hugely on your $C$ programming skills. As $C$ is not accounted to be an easy language, a more trivial answer is not so easy. Is there a reason to not use $C++$? You can still write $C$-style code in $C++$ and are not forced to use object oriented coding style if you want to improve a basic but essential function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, look for available libraries, e.g. &lt;a href=&quot;http://opencv.org/&quot; rel=&quot;nofollow&quot;&gt;openCV&lt;/a&gt;. That one will greatly improve your blob detection. It is a frequently used picture analysis library. Most likely the blob detection is your runtime crucial step. Your input data  is of size ~ 16 mio points here. Finding the centroid is ignorable (can be done in linear time). As the number of blobs should be far lower than 16 mio, the last part of your algorithm might be even with an $n^2$ algorithm acceptable fast for you. OpenCV usage will give you most likely maximum run-time efficiency and short and readable code for this first step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computer specs: In general the described problem does not sound computational hard. Most likely a standard computer will do the trick. You can test maybe a plot detection on a test picture with Photoshop or Gimp to get a feeling. If you run into time issues: a better graphic card will only help you if you are doing GPU - computing. Most likely you will do not. Increasing the number of cores will only help you if you utilize multithreading. You need to be sure that you are able to do so. Memory should not be an issue. 4kx4k pixels is nothing. &lt;/p&gt;&#xA;" OwnerUserId="20162" LastActivityDate="2016-07-03T10:09:59.927" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24366" PostTypeId="2" ParentId="24363" CreationDate="2016-07-03T15:00:20.037" Score="1" Body="&lt;p&gt;You are right that writing an efficient and robust eigenvalue solvers for large sparse systems is a difficult task that should only be done yourself as the very last resort. Spencer already gives the major players in his answer (ARPACK for single-node multithreaded computing, which is part of MATLAB and SciPy, and SLEPc for distributed computing, which can be accessed in Python via &lt;a href=&quot;http://pythonhosted.org/slepc4py/&quot; rel=&quot;nofollow&quot;&gt;slepc4py&lt;/a&gt;).&#xA;Both solve generalized eigenvalue problems, and both can take a procedure that computes matrix-vector products instead of a matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are interested in the smallest magnitude eigenvalues, you probably need to make use of the shift-and-invert strategy (i.e., compute the smallest magnitude eigenvalues of $A^{-1}$. Of course, you never actually compute inverse matrices; instead, whenever you need to compute $A^{-1}v$ for some vector $v$, you solve $Aw=v$ and use $w$ for $A^{-1}v$. &#xA;There are several strategies to accelerate this, based on the fact that ARPACK/SLEPc also only require a procedure that provides the application of the inverse (the obvious ones should already be implemented in ARPACK/SLEPc):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Since you need to solve linear systems repeatedly, you compute a LU- or Cholesky factorization of the matrix $A$ ahead of time and use the (much faster) backward/forward substitution part of the linear solve during the iteration. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In your case, you probably don't want to build the big tensor product matrix in the first place; in this case it makes sense to use an iterative solver such as CG for the solution of $Aw=v$ as well (especially if you can provide a good preconditioner). In this way, the whole eigenvalue solver  only requires a procedure to perform the matrix-vector multiplication. (You probably have to play with the tolerances a bit to get good results.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-07-03T15:00:20.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24367" PostTypeId="1" AcceptedAnswerId="24370" CreationDate="2016-07-03T15:30:39.913" Score="1" ViewCount="347" Body="&lt;p&gt;I would like to draw a texture (in Matlab - Psychtoolbox) which is composed of a sinusoidally-modulated isoluminant (magenta &amp;amp; cyan) grating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have the RGB matrices of the two phases (for magenta and cyan) and I need them to get blended into the background gray at zero-crossing points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please help me in formulating the formula of the matrix I need to use in order to create such texture (Figure attached). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/5hXee.jpg&quot; alt=&quot;![enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="20888" LastEditorUserId="9667" LastEditDate="2016-07-03T20:17:54.747" LastActivityDate="2016-07-03T20:26:09.457" Title="Matlab: Creating a matrix that represents a colored, sinusoidally-modulated grating" Tags="|matlab|visualization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24369" PostTypeId="1" AcceptedAnswerId="24959" CreationDate="2016-07-03T17:36:41.663" Score="1" ViewCount="556" Body="&lt;p&gt;I'm currently trying to implement a FEM to solve a type of wave equation with homogeneous Dirichlet boundary conditions by using standard $\mathcal{P}_1$ triangle elements and an explicit scheme for the time discretization. &#xA;In every discrete time step, the solution of a linear system with the mass matrix&#xA;$$M_{i,j} = \int_{\Omega}\varphi_i(x) \varphi_j(x) \, dx$$&#xA;occurs, where $i,j \in \mathcal{N_{\mathrm{inner}}}$ are the inner nodes and $\varphi_i$ are the piecewise linear, global continuous, nodal basis functions. Thus, i would like to replace $M$ by a good diagonal approximation $L$, namely the $\textit{lumped mass matrix}$.&#xA;Now, i've encountered two types to lumping techniques. First, a row-sum technique&#xA;$$L_{i,j} = \delta_{i, j} \sum_{k \, \in \, \mathcal{N_{\mathrm{inner}}}} M_{i,k}.$$&#xA;And second, a nodal quadrature technique&#xA;$$L_{i,j} = \int_{\Omega} I_h(\varphi_i(x) \varphi_j(x)) \, dx,$$&#xA;where $I_h$ is the piecewise linear interpolator. In case of $\mathcal{P_1}$-elements, the above can be rewritten as&#xA;$$L_{i,j} = \delta_{i, j} \sum_{T \, \in \, \omega(i)} \frac{|T|}{3},$$&#xA;with $\omega(i)$ being the patch of elements that share the inner node $i \in \mathcal{N_{\mathrm{inner}}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of homogeneous Neumann boundary conditions, we replace $\mathcal{N_{\mathrm{inner}}}$ by the set $\mathcal{N}$ of all nodes on the mesh. Then, both approaches are the same, because $\sum_{i \, \in \, \mathcal{N}} \varphi_i(x) \equiv 1$ on $\bar{\Omega}$.&#xA;But for homogeneous Dirichlet boundary conditions, this is not the case anymore and both techniques result in different matrices (though, the two lumped diagonal matrices are only different on nodes $i \in \mathcal{N_{\mathrm{inner}}}$ that are nodes of an element, which touches the boundary).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{Now my Question}$: which one of these lumped mass matrices should I use?&#xA;I've seen in some applications, that people want—for conservation reasons—that the row sum of both the original mass matrix and the lumped mass matrix are equal, but i've also seen that people want the node quadrature approach, because, in least in theory, the resulting fully discrete scheme shouldn't lose any order of accuracy.&lt;/p&gt;&#xA;" OwnerUserId="10005" LastActivityDate="2016-09-11T22:53:16.227" Title="Mass Lumping in case of Dirichlet boundary conditions" Tags="|finite-element|discretization|wave-propagation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24370" PostTypeId="2" ParentId="24367" CreationDate="2016-07-03T20:26:09.457" Score="0" Body="&lt;p&gt;I am not sure if this is what you want. But it looks to me that you want to generate a colormap that goes from cyan to magenta passing by gray. Based on &lt;a href=&quot;https://stackoverflow.com/a/24488819/3358223&quot;&gt;this answer&lt;/a&gt; I provide the code below. You can interpolate the values to obtain your colormap.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Data&#xA;x = linspace(0, 4*pi, 201);&#xA;y = x;&#xA;[X, Y] = meshgrid(x, y);&#xA;Z = sin(X);&#xA;&#xA;% Colormap&#xA;c1 = [1.0, 0.0, 1.0];&#xA;c2 = [0.5, 0.5, 0.5];&#xA;c3 = [0.0, 1.0, 1.0];&#xA;map = [c1; c2; c3];&#xA;n1 = 50;&#xA;n2 = 50;&#xA;cmap = [linspace(c1(1),c2(1),n1), linspace(c2(1),c3(1),n2)&#xA;        linspace(c1(2),c2(2),n1), linspace(c2(2),c3(2),n2)&#xA;        linspace(c1(3),c2(3),n1), linspace(c2(3),c3(3),n2)];&#xA;&#xA;% Plot&#xA;fig = figure&#xA;imshow(Z)&#xA;colormap(cmap')&#xA;print(fig,'MyPlot','-dpng')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the output&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5brho.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5brho.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-07-03T20:26:09.457" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24371" PostTypeId="2" ParentId="24360" CreationDate="2016-07-03T20:50:07.777" Score="2" Body="&lt;p&gt;I agree with the suggestion of starting with a simple problem and&#xA;with the elastic solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the simplest wave problem is the 1D, infinite bar/string.&#xA;The analytical solution to this problem is well-known, e.g.&#xA;&lt;a href=&quot;http://mathworld.wolfram.com/dAlembertsSolution.html&quot; rel=&quot;nofollow&quot;&gt;http://mathworld.wolfram.com/dAlembertsSolution.html&lt;/a&gt;&#xA;You can model this with a strip of elements in the x-direction.&#xA;You want zero-stress boundary conditions on all edges/faces.&#xA;If you define an initial condition (displacement) as a &quot;pulse&quot; of some &#xA;shape in the&#xA;center of the bar, one wave will move to the right and one to&#xA;the left as time increases (as you can see from the analytical solution).&#xA;Obviously you can't create an infinitely-long FE model. Rather, you want&#xA;stop the FE solution at a time before the two waves reach the ends of &#xA;your model and reflect back toward the center. The FE solution before&#xA;this time can be compared with the analytical one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the linear, elastic case, the modulus you use doesn't really&#xA;matter. For the plastic case, as you suggest, you should get the&#xA;elastic solution if you define a high yield stress.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-07-03T20:50:07.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24372" PostTypeId="1" CreationDate="2016-07-04T12:18:14.090" Score="1" ViewCount="149" Body="&lt;p&gt;In my program I have a complex symmetric tridiagonal matrix. In order to perform some algorithmic optimizations I am searching for a (ideally linear) mapping from $n\times n$ complex symmetric tridiagonal to $2n \times 2n$ real symmetric tridiagonal. Does such a mapping exist?&lt;/p&gt;&#xA;" OwnerUserId="20767" LastActivityDate="2016-08-03T19:12:40.117" Title="Mapping from n x n complex symmetric tridiagonal to 2n x 2n real symmetric tridiagonal" Tags="|matrix|lapack|symmetry|complex-arithmetic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24373" PostTypeId="1" CreationDate="2016-07-04T15:16:20.397" Score="4" ViewCount="561" Body="&lt;p&gt;Suppose we are given an RRQR factorization for some matrix $A \in \mathbb{R}^{m \times n}$, $A\Pi = QR$ where $m &amp;gt; n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a cheap way to update $A' = A + uv^{\top}$ given this factorization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of how to update the QR factorization given a rank-one update, but I am unsure how to update $\Pi$ as well to guarantee the rank-revealing aspect.&lt;/p&gt;&#xA;" OwnerUserId="20899" LastEditorUserId="20688" LastEditDate="2018-05-04T04:22:45.627" LastActivityDate="2022-11-10T17:09:07.787" Title="Rank-one Update to a Rank Revealing QR (RRQR) Factorization?" Tags="|linear-algebra|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24374" PostTypeId="1" CreationDate="2016-07-04T15:50:24.240" Score="2" ViewCount="512" Body="&lt;p&gt;I am using the FDFD method to calculate eigenmodes for an empty wave guide.&#xA;It's a 1x1meter structure with a PEC boundary. (Here I have 6x6 points to make it simple). Sounds simple, but I can't get it right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each mesh point my source free E-Field is allocated and calculated using the Helmholtz-equation:&#xA;$(∇^2-\lambda)E(x,y) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And since I can use a central differential quotient&#xA;$∇E_{ij}(x,y) = (-4E_{ij} +E_{i+1,j}+E_{i-1,j}+E_{i,j+1}+E_{i,j-1})/\Delta x^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am getting a matrix like this one (on the central diagonal the values are $4$ and the rest is of the diagonals are $-1$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2IGdF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2IGdF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far so good. But now I need to apply 2 changes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;eliminate non existing points (on the boundary each point has only 3x dots of the central differential quotient and not 4, so one of them must be set to 0). And on the edges there are only 2x dots instead of 4.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apply PEC for which $ E_{tan}(boundary)=0$ which means that on the boundary all tangetial fields must be equal 0.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Doing so I get the &lt;strong&gt;&quot;1. A-Matrix&quot;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nX0YA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nX0YA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;But here comes my problem:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Eigenvalues (for me here Eigenmodes) that I get are not the ones, that I calculate analytically. &#xA;My guess is that the 1st or 2nd adjustment that I am applying is somehow wrong. Could you please tell me whether I am apply my matrix adjustments and PEC implementation correctly?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to Peter Frolkovič, I noticed that there were some mistakes in my notations. I corrected the $∇$ to $∇^2$ and the $E_k$ to $E_{ij}$.&#xA;Also regarding the boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it's PEC I'm after, the E-Field is 0 inside PEC. So naturally all E-Field components on all boundary points should be 0. But if I do so, I really have to set all boundary points to zero in the Matrix A and this gives me the &lt;strong&gt;&quot;2. A-Matrix&quot;&lt;/strong&gt; (see the 2nd image above)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But solving the eigenvalue problem with this matrix results in some very weird Eigenvalues. So that's why I thought that not all E-Field components on the boundary should be 0 but only the tangetial ones. So I only set $ E_{tan}(boundary)=0$. But as described this didn't help much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here an Example with my Grid (see first image above):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above image you can see how my grid is defined (blue numbers are the node numbers). As described, at each node I solve the Helmholtz PDE from above. Lets take node number $7$ as an example. The corresponding equation would be: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(∇^2-\lambda)E_7 = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $∇^2E_7 = (-4E_7+E_1+E_8+E_{13})/ \Delta x^2$ (here are only 3 nodes of the central differential quotient, since node number 7 doesn't have a left node)&#xA;$(∇^2-\lambda)E_7 = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(-4E_7+E_1+E_8+E_{13}) = \Delta x^2 \lambda E_7 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for Dirichlet $E_7 =0$ which means that the equation simplifies to &#xA;$(E_1+E_8+E_{13}) = 0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I understand it right? If so, then the only way I know how to implement this in an eigevalue equation is by editing the main matrix - so by setting the corresponding matrix value (node) to 0... so in MATLAB&#xA; $A(7,7)=0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As &lt;em&gt;Peter Frolkovič&lt;/em&gt; pointed out, matrix entries are only the coefficients of the unknowns - I agree, but how else can I influence the node $E_7$ in this case, if not by doing as I described?&lt;/p&gt;&#xA;" OwnerUserId="20900" LastEditorUserId="20900" LastEditDate="2016-07-06T18:29:16.487" LastActivityDate="2016-07-06T18:29:16.487" Title="finite difference frequency domain Eigenvalue matrix to get eigenmodes" Tags="|finite-difference|eigenvalues|eigensystem|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24375" PostTypeId="2" ParentId="24372" CreationDate="2016-07-04T17:30:43.640" Score="1" Body="&lt;p&gt;I will assume you mean complex symmetric s.t. $A = A^T$ and not Hermitian $A = A^H$, which is different. Let $B \in \mathbb{R},[2n \times 2n]$ be your new tri-diagonal matrix. A simple option is each imaginary component is mapped $n$ places down the appropriate diagonal:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Main diag.: $b_{i,i} = Re(a_{i,i}), \;b_{i+n,i+n} = Im(a_{i,i}) \qquad \text{for} \qquad i = 1,\ldots,n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sub-diag.: $b_{i+1,i} = Re(a_{i+1,i}), \; b_{i+1+n,i+n} = Im(a_{i+1,i}) \qquad \text{for} \qquad i = 1,\ldots,n-1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Super-diag.: $b_{i,i+1} = Re(a_{i,i+1}), \; b_{i+n,i+1+n} = Im(a_{i,i+1}) \qquad \text{for} \qquad i = 1,\ldots,n-1$&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastEditorDisplayName="user20857" LastEditDate="2016-07-04T18:47:32.167" LastActivityDate="2016-07-04T18:47:32.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24376" PostTypeId="1" AcceptedAnswerId="24413" CreationDate="2016-07-04T18:41:06.353" Score="2" ViewCount="293" Body="&lt;p&gt;Are there numerical methods of solving the following fully nonlinear time-dependent PDE:&#xA;$$\nabla^2u\left(\textbf{r}(t), \dot{r}(t), t\right)=f\left(\textbf{r}(t), \dot{r}(t), t\right),$$&#xA;for $\textbf{r}(t)$, where $\nabla^2 = \frac{\partial^2}{\partial^2 x}+\frac{\partial^2}{\partial^2 y}+\frac{\partial^2}{\partial^2 z}$ and $\mathbf{r}(t)=x(t)\mathbf{i}+y(t)\mathbf{j}+z(t)\mathbf{k}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could FACR (Fourier Analysis Cyclic Reduction) be used to solve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, could you recommend a reference that would help me numerically solve this or a similar problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;thanks&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="20904" LastEditorUserId="20904" LastEditDate="2016-07-08T15:11:41.017" LastActivityDate="2016-07-08T15:11:41.017" Title="Numerical Methods for Solving a Fully Nonlinear Time-Dependent PDE?" Tags="|pde|nonlinear-equations|reference-request|time-integration|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="24377" PostTypeId="2" ParentId="24360" CreationDate="2016-07-04T20:54:04.880" Score="1" Body="&lt;p&gt;I've found the Aldridge-Blake solution for spherical wave propagation quite useful for verification.  You can find the code in Prof. Brannon's repository at &lt;a href=&quot;https://csmbrannon.net/2012/03/13/aldridge-spherical-source-verification-test-for-dynamic-continuum-codes/&quot; rel=&quot;nofollow&quot;&gt;https://csmbrannon.net/2012/03/13/aldridge-spherical-source-verification-test-for-dynamic-continuum-codes/&lt;/a&gt;.  Unfortunately, the solution is too complex (no pun intended) to post on this forum.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2016-07-04T20:54:04.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24378" PostTypeId="1" AcceptedAnswerId="24379" CreationDate="2016-07-04T21:21:05.357" Score="0" ViewCount="1357" Body="&lt;p&gt;I am trying to solve the following problem via a finite difference approximation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_t = k \, u_{xx}$, on $0 &amp;lt; x &amp;lt; L$ and $t &amp;gt; 0$;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(0,t) = u(L,t) = 0$;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x,0) = f(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I take $u(x,0) = f(x) = x^2$ for my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programming is not my forte at all, so I am having some issues with the implementation. Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; ## This program is to implement a Finite Difference method approximation&#xA;## to solve the Heat Equation, u_t = k * u_xx,&#xA;## in 1D w/out sources &amp;amp; on a finite interval 0 &amp;lt; x &amp;lt; L. The PDE&#xA;## is subject to B.C: u(0,t) = u(L,t) = 0,&#xA;## and the I.C: u(x,0) = f(x).&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;# definition of solution to u_t = k * u_xx&#xA;def u(x,t):&#xA;    return u&#xA;&#xA;# definition of initial condition function  &#xA;def f(x):&#xA;    return x^2&#xA;&#xA;# parameters    &#xA;L = 1&#xA;T = 10&#xA;N = 10&#xA;M = 10&#xA;s = 0.25&#xA;&#xA;# uniform mesh&#xA;x_init = 0&#xA;x_end = L&#xA;dx = float(x_end - x_init) / N&#xA;&#xA;# time discretization&#xA;t_init = 0&#xA;t_end = T&#xA;dt = float(t_end - t_init) / M&#xA;&#xA;t = np.zeros(M+1)&#xA;t = np.arange(t_init, t_end, dt)&#xA;&#xA;# Boundary Conditions&#xA;for m in xrange(0, M):&#xA;    t[m] = m * dt&#xA;    u(0, t[m]) = 0&#xA;    u(N, t[m]) = 0&#xA;&#xA;# Initial Conditions&#xA;for j in xrange(0, N):&#xA;    x[j] = j * dx&#xA;    u(x[j], 0) = f(x[j])&#xA;&#xA;# finite difference scheme&#xA;for j in xrange(1, N-1):&#xA;    u(x[j],t[m+1]) = u(x[j],t[m]) + s * ( u(x[j+1],t[m]) - &#xA;    2 * u(x[j],t[m]) + u(x[j-1],t[m]) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So in particular, I am wondering (1) if my function definitions are okay? (2) is my uniform mesh and time discretization okay? (3) am I selecting reasonable values for the parameters? (I know that the approximation is more accurate for smaller $\Delta x$ and $\Delta t$, with the disadvantage of increased computing time..)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(4) how are my loops (for the initial/boundary conditions)? I get the feeling I need a nested loop for the finite difference scheme.. One for loop for the values of j and another for loop going through each value m in time. Is this correct? ... Also, I keep getting the error: u(0, t[m]) = 0 &quot;can't assign to function call.&quot; What is wrong there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for the help! As I mentioned, my programming skills are not very strong so I feel like a fish out of water doing this.&lt;/p&gt;&#xA;" OwnerUserId="20906" LastActivityDate="2016-11-10T08:56:18.133" Title="Help implementing finite difference scheme for heat equation" Tags="|pde|finite-difference|python|parabolic-pde|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24379" PostTypeId="2" ParentId="24378" CreationDate="2016-07-04T23:10:59.883" Score="3" Body="&lt;p&gt;So your first big issue is one you bring up in point &lt;strong&gt;(4)&lt;/strong&gt;, where you say you get the error '&lt;em&gt;u(0, t[m]) = 0 &quot;can't assign to function call&quot;&lt;/em&gt;'. You're trying to store data by assigning data to a function. That won't work. The way you're implementing this code, you actually should make &lt;code&gt;u(x,t)&lt;/code&gt; be represented by a 2-D array that can be accessed via &lt;code&gt;u[m][j]&lt;/code&gt;, where &lt;code&gt;m&lt;/code&gt; is associated with the $m^{th}$ time value, and &lt;code&gt;j&lt;/code&gt; is associated with the $j^{th}$ spatial value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you also note, you should have a nested for loop. Do an outside loop for time, and then an inner loop for looping through the spatial discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now assuming your value &lt;code&gt;s&lt;/code&gt; makes sense for your problem, the other potential issue I see is your step size, &lt;code&gt;dt&lt;/code&gt;. I am pretty sure this time stepsize won't be stable based on the value of &lt;code&gt;dx&lt;/code&gt; since it seems you're just using an Explicit Euler time stepping scheme. Try picking some &lt;code&gt;dt&lt;/code&gt; that is proportional to &lt;code&gt;dx*dx&lt;/code&gt;, like &lt;code&gt;dt=0.1*dx*dx&lt;/code&gt;, and see how it fairs for you.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-07-04T23:10:59.883" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24384" PostTypeId="2" ParentId="24378" CreationDate="2016-07-05T10:04:45.710" Score="5" Body="&lt;p&gt;Don't worry about programming skills, everyone's a beginner at some point. It's a good start actually. I'm just going to give some hints assuming that this is an exercise. Actually, choward's answer is complete, this is just a rephrasing of his suggestions in, I hope, a more beginner friendly language. So, if you do accept an answer it should be his ;)&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your definition of the solution does not make sense in Python. I would suggest storing it in a &lt;code&gt;NxM&lt;/code&gt; numpy array. That way you can easily define the initial and boundary conditions with the slice notation (i.e. &lt;code&gt;U[:,0] = x**2&lt;/code&gt; where &lt;code&gt;x&lt;/code&gt; is an 1-dim array, see &lt;code&gt;numpy.linspace&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It seems that you are using the central difference scheme for the second order derivative in space. The definition is usually &#xA;$$&#xA;u_{xx}(x_i) = \frac{u_{i+1} - 2u_i + u_{i-1}}{h^2}&#xA;$$&#xA;where $h$ is the step size in space (your &lt;code&gt;dx&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You are using a explicit time stepping scheme called &lt;em&gt;explicit Euler&lt;/em&gt; or &lt;em&gt;forward Euler&lt;/em&gt; which is &lt;em&gt;conditionally stable&lt;/em&gt;. This means that your discretisation has to fulfil a certain condition in order for your solution to remain bounded or, in other words, that your time step has to be small enough. Another around this would be to use an implicit method such as &lt;em&gt;backward Euler&lt;/em&gt; which is &lt;em&gt;unconditionally stable&lt;/em&gt;. Note that, while this is fine for the heat equation which is intrinsically stable, it's maybe not the best choice for neutral or unstable problems since your numerical solution may suffer from over-stabilisation (a.k.a. numerical damping). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You have only implemented the inner loop. You would need an outer loop to advance in time as well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20916" LastEditorUserId="20916" LastEditDate="2016-07-05T10:13:38.693" LastActivityDate="2016-07-05T10:13:38.693" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24386" PostTypeId="1" CreationDate="2016-07-05T13:59:28.870" Score="2" ViewCount="120" Body="&lt;p&gt;To implement hybridizable discontinuous Galerkin methods, one needs finite element spaces defined on the skeleton of the mesh. deal.II has support for HDG through FE_FaceQ class which provides discontinuous Lagrange functions supported on the skeleton of the mesh. I want to know if there are other libraries which provide such FE spaces for &quot;embedded DG&quot; (EDG) and &quot;interior embedded DG&quot; (IEDG) methods. EDG methods use continuous trace spaces while IEDG methods use trace space continuous on interior edges but discontinuous on boundary edges.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2016-07-05T13:59:28.870" Title="FEM libraries with trace spaces" Tags="|finite-element|libraries|discontinuous-galerkin|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24388" PostTypeId="1" AcceptedAnswerId="24392" CreationDate="2016-07-05T15:01:43.977" Score="1" ViewCount="1220" Body="&lt;p&gt;I have recently acquired a server that I will be using for distributed simulations. However, I was wondering how do I determine which MPI version the server is compatible with?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason that I am asking this is that I acquired it from a reseller who buys old computer equipment (for those wondering, it is an IBM blade center. Some of the blades have DDR2 installed and others are dated around 2011) and resells it. There was one professor in my university who was using an older server. He was trying to code for distributed systems but he was saying that it was not compatible with the latest version of MPI (he also said that he was having a hard time getting older versions of MPI working). So, my question is how can I make sure that the version of MPI I use is compatible with the server?&lt;/p&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2016-07-05T20:44:10.617" Title="How to determine MPI compatibility" Tags="|parallel-computing|mpi|hpc|" AnswerCount="2" CommentCount="3" ClosedDate="2016-07-06T15:00:04.143" ContentLicense="CC BY-SA 3.0" />
  <row Id="24389" PostTypeId="2" ParentId="24388" CreationDate="2016-07-05T15:57:07.587" Score="1" Body="&lt;p&gt;Since there is a set of &lt;a href=&quot;https://www.mpi-forum.org/docs/docs.html&quot; rel=&quot;nofollow&quot;&gt;MPI standards&lt;/a&gt;, any of the available stacks should state, which &lt;a href=&quot;https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions&quot; rel=&quot;nofollow&quot;&gt;for example standard level they are compliant with&lt;/a&gt;. Your customer should be able to state which level of standard compliance they need. There's probably very few codes out there that need 3.1 at this point, but some people like to be on the cutting edge. For each function in an MPI standard level, you ought to be able to be able to find by searching which MPI standard document introduced it and then find a library that supports that level of standard. There are lots more MPI libraries than MPICH and OpenMPI, but lots of other libraries are based on MPICH. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-07-05T15:57:07.587" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24390" PostTypeId="1" CreationDate="2016-07-05T17:35:18.683" Score="3" ViewCount="129" Body="&lt;p&gt;Does anyone know of references discussing the solution of 4th order PDEs by way of spectral/hp-finite element methods?  Specifically, I'm interested in the extension of the spectral element method, which uses high-order Lagrange polynomial interpolants (at a suitable distribution of nodes) as shape functions.  In this case, the resulting finite elements are C0 continuous, which is sufficient for 2nd order PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For fourth order PDEs, C1 continuous finite elements are required. Hence, cubic (or quintic) Hermite interpolation is typically used to construct the shape functions, as they produce finite elements with this property.  I'm interested in how one might best construct higher order &quot;spectral&quot; elements that are C1 continuous and have the fast convergence properties associated with the spectral element method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware that an alternative approach would be to split the 4th order PDE into two 2nd order PDEs.  However, I think this would require solving for the 2nd derivative of the unknown, which may become infinite at points on the boundary in my case.  Thus, I'm interested in methods where I need only solve for the unknown and its first derivative to avoid this trouble.&lt;/p&gt;&#xA;" OwnerUserId="9601" LastActivityDate="2016-07-05T17:35:18.683" Title="Spectral/hp-finite elements for 4th order PDEs" Tags="|finite-element|pde|spectral-method|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24391" PostTypeId="1" CreationDate="2016-07-05T19:42:31.933" Score="2" ViewCount="110" Body="&lt;p&gt;I apologize if this is a relative simple question. I am very new to setting up a cluster computer and I am attempting to make heads and tails of the information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was reading on setting up a cluster with MPICH in ubuntu. And it seemed like it was a &quot;simple&quot; process. The process was a simple 1,2,3 step process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I came upon process managers such as openMosix. From the steps outlines in the MPICH (&lt;a href=&quot;https://help.ubuntu.com/community/MpichCluster&quot; rel=&quot;nofollow&quot;&gt;https://help.ubuntu.com/community/MpichCluster&lt;/a&gt;) it seemed like I don't need to setup a process manangmenet but at the same time, these programs help to manage parallel jobs. And MPICH has an extension to talk to the process manager. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, when would I use a process manager? Would this be on large clusters? Or should I always install process managers and they are used in conjunction with MPI? What is the difference between an MPI (whether it be MPICH or OpenMPI) and the process manager? Or does the process manager utilize MPI? &lt;/p&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2016-08-04T22:36:47.650" Title="Need help understanding cluster process managers and MPI" Tags="|parallel-computing|mpi|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24392" PostTypeId="2" ParentId="24388" CreationDate="2016-07-05T20:44:10.617" Score="2" Body="&lt;p&gt;MPI is software that runs on your computer(s). Which implementation and version of MPI you can use therefore depends on the installed operating system. If you are for example running GNU/Linux, you can usually choose between different versions of MPICH and OpenMPI in your package manager.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Special network features such as Infiniband can benefit MPI and require approriate hardware, but basic ethernet is already sufficient.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2016-07-05T20:44:10.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24393" PostTypeId="2" ParentId="24391" CreationDate="2016-07-05T22:09:27.593" Score="1" Body="&lt;p&gt;You should use some sort of process or resource management (aka a scheduler) once more than one person at a time will try to use the cluster. Unless you have some other way to guarantee that person A won't use person B's nodes, you need a scheduler. Sometimes, in some small labs, this is done by agreement of two people who literally sit next to each other, but even that is bound to fail, so installing an open source scheduler is the route to go. This isn't for the faint of heart, but it's also not terribly hard. There are lots of documents out there to help you.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-07-05T22:09:27.593" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24396" PostTypeId="2" ParentId="24374" CreationDate="2016-07-06T06:27:54.683" Score="1" Body="&lt;p&gt;Looking to your &quot;2. A-Matrix&quot; my first guess is that you might misunderstand the implementation of so called Dirichlet boundary conditions when one prescribes given values for the unknown solution at boundary. For instance the zero Dirichlet boundary conditions mean&#xA;$$&#xA;E_{i,j}=0&#xA;$$&#xA;for some indices $(i,j)$ for which the corresponding grid points lie on boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the standard treatment is that you view the above condition as a linear algebraic equation (although a trivial one), so you include it into your system of linear algebraic equations, so a corresponding row and column must be present in the matrix. This row has a trivial form having $1$ on the diagonal and zero elsewhere. The other rows are not influenced by this boundary condition! Especially it does not mean that you replace in the matrix the entries corresponding to boundary nodes by zeroes, the matrix entries are defined by coefficients before the unknowns, not by the unknowns themselves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my suggestion is that you check this issue in your code. In &quot;2. A-Matrix&quot; it seems that you have zero rows that can not happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other standard approach to treat Dirichlet boundary conditions is that you eliminate these values, it means you do not include them into unknowns, so no rows and columns for them exist in the matrix. If you have the zero Dirichlet boundary conditions, it means simply that you delete the rows and columns that would correspond to these values. In general you modify all linear algebraic equations for $i,j$ when at least one neighbor lies on the boundary (but not $i,j$-node itself, it is inner node). You might follow this approach in the &quot;1. A-Matrix&quot;, but I am not sure if completely.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-07-06T06:27:54.683" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24397" PostTypeId="2" ParentId="24350" CreationDate="2016-07-06T11:34:16.750" Score="0" Body="&lt;p&gt;The question you wrote down is very puzzling. To start with, taking the same code and pasting it while using the same command you used to generate the output yields an error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 - You specified 'u' (lower-case) as a function output, yet it is used nowhere in your code (you only use 'U' -- upper case). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 - When I substitute 'u' for 'U' -- as one expects -- I get a vector. Meaning I cannot reproduce the output via 'surf' as you did. Please synchronize your question for us to understand how to assist. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3 - Out of curiosity, 'Uex' is used where exactly after calculating it in the inner-most loop?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4 - Some other variables seem not be used at all, however I will assume this was done on purpose to simplify the question...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5 - Would it not be easier if you used a more simplified function -- taking into consideration how you'd expect the boundaries to react -- for debugging purposes ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;6 - Why are there no comments in your code. This is a bad programming habit, even for a relatively simple function and not to mention an inquiry post.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum it up, consider monitoring the residual (every $k^{th}$ iteration?), simplify your input function and reduce your boundaries to Dirichlet. An even more useful tool is to insert a breakpoint(s) inside your function, to further break down your code and debug effectively. If this still would not work, consider fixing your question by making it reproducible on our side and please: &lt;strong&gt;comments&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck !&lt;/p&gt;&#xA;" OwnerUserId="18387" LastActivityDate="2016-07-06T11:34:16.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24398" PostTypeId="1" AcceptedAnswerId="24400" CreationDate="2016-07-06T12:11:41.413" Score="0" ViewCount="321" Body="&lt;p&gt;I am solving a 3D heat transfer equation with variable boundaries (insulated, convective, radiative or free) using a F.D.M. technique. My geometry of choice is a cube. The purpose of my work is to get more familiar with such problems -- out of interest -- and their parallelizations using both distributed (MPI) and shared memory (OpenMP/Pthreads).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make this question as neat as possible, I will avoid tediously long expressions -- as is usually the case in 3D -- and go straight to the point. My stencil is a 7-point kernel. I am solving on my cube by traversing the discretization of the inner points using an SOR approach (i.e. excluding boundary surfaces, vertices and corners). Hence I find myself confused about how one usually solves such equations via Finite Differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To set-up a (mass) matrix, instead of the naive point-by-point stencil approach is greatly perplexing. I say this because per each row of the mass matrix I end up having only 7 non-zero elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if my cube is discretized equally in x, y and z directions while having 100 points in each respectively, then my overall mass matrix dimensions would become: $(100x100x100)^2 = 1,000,000-by-1,000,000$ with only 7 nnz elements per row. This is too sparse and inefficient from the perspective of memory and &lt;em&gt;certain&lt;/em&gt; algorithm applications.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes me wonder how inexperienced I currently am when trying to formulate a solution for such problems. That being said, how does one usually formulate such a huge sparse matrix to tackle such problems ? Surely my naive approach isn't the best nor the only solution out there (duh). Any explanation and/or examples would be very welcome. &lt;/p&gt;&#xA;" OwnerUserId="18387" LastActivityDate="2016-07-06T16:55:16.010" Title="Heat Equation in 3D mass Matrix set-up" Tags="|algorithms|finite-difference|fluid-dynamics|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24399" PostTypeId="1" AcceptedAnswerId="24402" CreationDate="2016-07-06T12:57:49.217" Score="7" ViewCount="433" Body="&lt;p&gt;In deriving a Newton scheme, I end up with a Jacobian matrix of the form $J=I+ab^T$ where $a,b$ are vectors. For practical reasons, I want to approximate it by a symmetric positive definite matrix. Symmetrizing is easy: replace it by $J \approx I+\frac 12(ab^T+ba^T)$. But the (now real) eigenvalues may still be negative, so I further approximate $J \approx \tilde J := I+\frac 12\alpha (ab^T+ba^T)$ where I want to choose $\alpha$ so that $\tilde J$ is s.p.d. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To choose $\alpha$, I need the eigenvalues of the matrix $ab^T$. This ought to be simple enough, but I must be missing the key step to find a closed-form expression for the eigenvalues. Clearly, there will be at most two non-zero eigenvalues, and the corresponding eigenvectors must lie in the plane spanned by the vectors $a,b$. The eigenvalues are then the minimal and maximal value of the Rayleigh quotient&#xA;$$&#xA;  R(x) := \frac{x^T (ab^T) x}{x^T x}&#xA;  = \frac{(a^Tx) (b^Tx)}{x^T x}&#xA;$$&#xA;over all $x \in \text{span}(a,b)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that there is a closed form expression for the minimal and maximal value of $R(x)$, but its form eludes me. &lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-07-06T20:48:45.720" LastActivityDate="2016-07-12T19:08:51.347" Title="Eigenvalues of $ab^T$" Tags="|eigenvalues|newton-method|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24400" PostTypeId="2" ParentId="24398" CreationDate="2016-07-06T16:55:16.010" Score="1" Body="&lt;p&gt;I am not certain I understand your question but it appears to have to do with storage and manipulation of sparse matrices. If so, then I suggest you take a&#xA;a look at the lecture notes from a course Professor Saad teaches on sparse matrix computations:&#xA;&lt;a href=&quot;http://www-users.cselabs.umn.edu/classes/Spring-2014/csci8314/&quot; rel=&quot;nofollow&quot;&gt;http://www-users.cselabs.umn.edu/classes/Spring-2014/csci8314/&lt;/a&gt;&#xA;As you will see, lecture 2 discusses different ways of representing sparse matrices. Lecture 3 specifically discusses the issue you are concerned about-- finite difference equations. Later lectures discuss solution techniques for linear equations when the matrices are sparse.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-07-06T16:55:16.010" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24401" PostTypeId="1" AcceptedAnswerId="24417" CreationDate="2016-07-06T21:43:06.373" Score="5" ViewCount="1344" Body="&lt;p&gt;C++ eigen library does vectorization for different architecture, like SSE, NEON etc. In their documentation they mentioned that, Eigen vectorization is not compiler dependent. But most modern compilers like gcc does the vectorization automatically if the vectorization flag is enabled using -O3 flag. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, why Eigen or any other libraries does this hard coded  vectorization when compilers does this automatically for us?    &lt;/p&gt;&#xA;" OwnerUserId="20940" LastEditorUserId="20940" LastEditDate="2016-07-07T16:16:49.070" LastActivityDate="2016-07-08T09:30:49.497" Title="Why do libraries need hand-vectorized code instead of compiler auto vectorization" Tags="|c++|matrix|vectorization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24402" PostTypeId="2" ParentId="24399" CreationDate="2016-07-07T00:11:58.277" Score="7" Body="&lt;p&gt;This is sometimes known as Buzano's inequality (&lt;a href=&quot;http://www.jstor.org/stable/2159168&quot; rel=&quot;nofollow&quot;&gt;http://www.jstor.org/stable/2159168&lt;/a&gt;). In general, if $\|x\|=1$, $P=xx^{\top}$ is a projection operator, so a simple application of Cauchy-Schwarz leads to&#xA;$$ 2|b^{\top}xx^{\top}a|-|b^{\top}a| \leq |b^{\top}(2P-I)a| \leq \|a\|\,\|b\|, $$&#xA;which gives Buzano's inequality&#xA;$$ |b^{\top}xx^{\top}a| \leq \frac{\|a\|\,\|b\|+|a^{\top}b|}{2}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without loss of&#xA;generality let $a=(1,0)$,&#xA;$b=(p,q)$ ($p^2+q^2=1$, $q\geq0$) and&#xA;$x = (\alpha, \beta)$ ($\alpha^2+\beta^2=1$, $\alpha\geq0$). Finding the extrema of&#xA;$ b^{\top}xx^{\top}a = \alpha(\alpha p+\beta q)$&#xA;is easily done with Lagrange multipliers, giving the values&#xA;$$ \frac{p\pm1}{2}. $$&#xA;Here $p = \frac{a^{\top}b}{\|a\|\|b\|} = \cos\angle ab$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this &#xA;$$ \frac{a^{\top} b-\|a\|\|b\|}{2} \|x\|^2 \leq a^{\top} xx^{\top}b \leq \frac{ a^{\top}b+\|a\|\|b\|}{2} \|x\|^2 $$&#xA;(with both inequalities being tight) so the admissible values of $\alpha$ are given by&#xA;$$ 1 + \tfrac12(p\pm1)\alpha \|a\|\|b\| &amp;gt; 0, \qquad \frac{-2}{\|a\|\|b\| + a^{\top}b} &amp;lt; \alpha &amp;lt; \frac{2}{\|a\|\|b\|-a^{\top}b}. $$&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-07-12T19:08:51.347" LastActivityDate="2016-07-12T19:08:51.347" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24403" PostTypeId="2" ParentId="14673" CreationDate="2016-07-07T03:17:26.477" Score="0" Body="&lt;p&gt;I am just looking at &lt;a href=&quot;https://www.sam.math.ethz.ch/sam_reports/reports_final/reports2013/2013-05.pdf&quot; rel=&quot;nofollow&quot;&gt;Käppeli and Mishra&lt;/a&gt;, &quot;Well-balanced schemes for the Euler equations with gravitation&quot;. They make one key observation -- use the enthalpy to reconstruct values at cell boundaries -- and derive from this a formulation where the truncation errors of flux and source terms cancel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper is surprisingly easy to read; kudos to the authors.&lt;/p&gt;&#xA;" OwnerUserId="3987" LastActivityDate="2016-07-07T03:17:26.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24404" PostTypeId="1" CreationDate="2016-07-07T03:44:57.030" Score="1" ViewCount="193" Body="&lt;p&gt;If I have an equation given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\displaystyle Y = \frac{a^2}{d^2}\frac{(1-c^2\frac{c}{a})}{(1-b^2)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I expand $a,b,c,d$ in a Taylor series, where $a$ is truncated at the $A^{th}$ order, $b$ is truncated at the $B^{th}$ order, $c$ is truncated at the $C^{th}$ order, and $d$ is truncated at the $D^{th}$ order, how do I go about determining the order of the truncation error for the full equation  $ Y$?&lt;/p&gt;&#xA;" OwnerUserId="19407" LastActivityDate="2023-01-06T09:38:44.707" Title="How to determine the truncation error with products and quotients" Tags="|linear-algebra|finite-difference|numerical-analysis|computational-physics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24405" PostTypeId="2" ParentId="24404" CreationDate="2016-07-07T04:40:24.173" Score="2" Body="&lt;p&gt;Given two truncated Laurent series, $f = p(x)+O(x^p)$, $g = q(x)+O(x^q)$, the orders will be&#xA;$$ f+g = p(x) + q(x) + O(x^{\min(p,q)}), $$&#xA;$$ fg = p(x) q(x) + O(x^{\min(p+q_0, q+p_0)}), $$&#xA;$$ 1/g = 1/q(x) + O(x^{q-2q_0}). $$&#xA;where $p_0$ and $q_0$ are the degrees (possibly negative) of the first nonzero monomials in $p$ and $q$. So for each subexpression it is necessary to determine the leading term (which is possibly non-constant, and might be singular), then it is straightforward algebra.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2016-07-07T04:40:24.173" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24406" PostTypeId="1" AcceptedAnswerId="24674" CreationDate="2016-07-07T11:54:20.900" Score="3" ViewCount="3434" Body="&lt;p&gt;I am teaching a basic course on computational physics and for the last part of the course I will introduce freshman physics undergraduates to finite-element modelling methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am preparing a COMSOL model that solves for the eigenfrequencies and shape of the eigenmodes of the unperturbed 3D geometry of e.g. a tuning fork.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, I don't actually know what the FEM algorithm is doing: What matrices are set up, what goes into each element, what equations are solved for...?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer should give an account of what the FEM algorithm does to go from the meshed 3D geometry to the solved eigenfrequencies and the shape of the eigenmodes of the whole structure.&lt;/p&gt;&#xA;" OwnerUserId="20935" LastEditorUserId="20935" LastEditDate="2016-08-09T11:28:36.050" LastActivityDate="2016-08-12T15:25:33.463" Title="Understanding Finite-Element Modal Analysis" Tags="|linear-algebra|finite-element|numerical-analysis|computational-physics|comsol|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="24408" PostTypeId="2" ParentId="24406" CreationDate="2016-07-07T12:26:35.463" Score="1" Body="&lt;p&gt;If you are familiar with the standard FEM analysis works, the idea of modal analysis is straightforward. In standard FEM analysis, you transfer the time-dependent elastic wave equation (ignoring damping for now)&#xA;$$ \ddot{r}(t) + A r(t) = f(t) \tag{1}\label{1},$$&#xA;which is a mathematical model describing the behavior of the displacement $r(t)$ using the differential operator $A$ (which includes the material law; in the case of linear elasticity, $Ar = \mu \Delta r + (\mu+\lambda)\nabla(\nabla\cdot r)$, where $\lambda$ and $\mu$ are the Lamé parameters) into the system of linear equations&#xA;$$ M \ddot{\vec{r}}(t) + K \vec r(t) = M\vec f(t),$$&#xA;where $K$ is the stiffness matrix and $M$ is the mass matrix, as you write in your question. The way this is done in finite element methods is by projecting the infinite-dimensional system \eqref{1} onto a finite-dimensional subspace (this is called &lt;em&gt;Galerkin approach&lt;/em&gt;), which in finite element methods is chosen as the span of piecewise polynomials on each cell of the mesh. There are already a lot of answered questions about how this works on this site (e.g., &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10678/basic-explanation-of-shape-function&quot;&gt;this one&lt;/a&gt;), so I'm not going to go into more detail here; I'll just point out that in the specific case of piecewise linear functions, the finite-dimensional approximation to the solution of \eqref{1} is uniquely determined by its values at the nodes of the mesh, and this is exactly what $\vec r$ represents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, the eigenmodes -- which are stationary states! -- satisfy the resonance equation&#xA;$$ Ar = \lambda r,$$&#xA;where $\lambda$ is the eigenvalue. Proceeding as for the time-dependent case \eqref{1}, you arrive at the finite-dimensional (generalized) eigenvalue problem&#xA;$$ K\vec r = \lambda M\vec r.$$&#xA;This is now a standard problem in numerical linear algebra, for which several methods exist (which, it should be pointed out, work by primarily computing &lt;em&gt;eigenvectors&lt;/em&gt;, from which the eigenvalues can be obtained easily). In particular, since $K$ and $M$ are large and sparse, a Krylov method such as Arnoldi would be most efficient. This is exactly what COMSOL does (using the FORTRAN library ARPACK, which implements these methods). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(You are right that eigenvectors are not unique; usually -- and in particular in Krylov methods -- you want to have them normalized, to $r^TMr=1$ as Bill Greene notes, see page 55 of the &lt;a href=&quot;http://hpc.mtech.edu/comsol/pdf/Structural_Mechanics_Module/StructuralMechanicsModuleUsersGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;COMSOL manual&lt;/a&gt;; this only leaves the choice of sign, which is arbitrary and often even random (due to the initialization of the method).)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-07-08T10:29:37.950" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24409" PostTypeId="1" CreationDate="2016-07-07T13:33:39.337" Score="1" ViewCount="477" Body="&lt;p&gt;I am a little confused about the truncation error and what it actually means. So, For example I have an explicit approximation to the heat equation&#xA;$$\frac{u_{j}^{i+1}-u_{j}^{i}}{\delta t}=\frac{u_{j-1}^{i}-2u_{j}^{i}+u_{j+1}^{i}}{\delta x^2}$$&#xA;Now, I pick a point $(x_j,t_i)$ and do the Taylor approximation of each grid node function in the above equation. Then I choose $(x_j,t_{i+1})$ and do the same exercise and local truncation error is exactly the same. Intuitively, I would expect the local disretization(or truncation) error be a little larger in the second case because the second $x$ derivative is approximated with a distance $\delta t$ but it is not. So does it mean that since this scheme involves 4 grid nodes, if I pick any of them and do that Taylor expansion I will get exact same local error? Perhaps I am confused what is the meaning of the local truncation error in FDM.&lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2017-02-03T06:56:46.347" Title="discretization error for the explicit scheme for heat equation" Tags="|parabolic-pde|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24410" PostTypeId="2" ParentId="24409" CreationDate="2016-07-07T13:52:34.943" Score="0" Body="&lt;p&gt;The spatial discretization and time discretization is treated differently. The Taylor approximation is used to determine the order of the spatial discretization. This creates an equation of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{du}{dt} = A u$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where A is the matrix representing the finite difference operator. Then we use methods such as the Von Neumann method to determine stability etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are correct in assuming that the error will be different for the two cases, and this will come out in the Von Neumann analysis.&lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2016-07-07T13:52:34.943" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24411" PostTypeId="2" ParentId="24409" CreationDate="2016-07-07T15:32:38.970" Score="1" Body="&lt;p&gt;local truncation error arises when we neglect higher order terms which are assumed to be of negligible effect -- but then again the threshold is set depending on the task at hand. As for your question, you have a heat equation in 1D (a very typical parabolic PDE) that you intend to solve via finite differences. &#xA;$$u_t = u_{xx} $$&#xA;What you decided to use to approximate the above Partial Differential Equation is a typical forward (Euler) in time central in space scheme (FTCS). So in other words, you did the following: &#xA;$$u_t \approx  \frac{u_i^{n+1}-u_i^n}{\delta t}$$ &#xA;$$u_{xx} \approx \frac{u_{i-1}^n-2u_i^{n}+u_{i+1}^n}{\delta x^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next is an accuracy investigation. To accomplish this we Taylor expand each of the terms excluding about which we expand -- in our case to facilitate our work, let's go for $u_i^n \approx u(x,t)$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t} \Bigg |_x \approx u(t+\delta t) = u(t) + u'(t)\delta t + \frac{1}{2}u''(t)\delta t^2 + \frac{1}{6}u'''(t)\delta t^3 + \mathcal{O}(\delta t^4) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial x} \Bigg |_t \approx u(x+\delta x) = u(x) + u'(x)\delta x + \frac{1}{2}u''(x)\delta x^2 + \frac{1}{6}u'''(x)\delta x^3 + \frac{1}{24}u^{iv}(x)\delta x^4 + \mathcal{O}(\delta x^5) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial x} \Bigg |_t \approx u(x-\delta x) = u(x) - u'(x)\delta x + \frac{1}{2}u''(x)\delta x^2 - \frac{1}{6}u'''(x)\delta x^3 + \frac{1}{24}u^{iv}(x)\delta x^4+ \mathcal{O}(\delta x^5) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now substituting in the above expression/scheme, we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t = u'(t) + \frac{1}{2}u''(t)\delta t + (hot)... \rightarrow u_t \approx u'(t) + \mathcal{O}(\delta t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{xx} =  u''(x) + \frac{1}{12}u^{iv}(x)\delta x^2 + (hot)... \rightarrow u_{xx} \approx u''(x) + \mathcal{O}(\delta x^2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $hot$ is an abbreviation for Higher Order Terms and $\mathcal{O}$ is the truncation/neglected terms. Therefore, as can be seen, this particular scheme is considered order one accurate in time and 2 in space due to its truncated terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you still would like to dig deeper, which I encourage you, then consider investigating the difference between the local truncation error and the global truncation error. While you are at it, might as well check out the Von Neumann stability condition analysis!&lt;/p&gt;&#xA;" OwnerUserId="18387" LastActivityDate="2016-07-07T15:32:38.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24413" PostTypeId="2" ParentId="24376" CreationDate="2016-07-07T23:19:06.987" Score="2" Body="&lt;p&gt;To keep things simple, we will approximate $\nabla^2u(\textbf{x},\dot{\textbf{x}},t)$ with some operator $L[u(\textbf{x},\dot{\textbf{x}},t)]$ that uses finite difference approximations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla^2u(\textbf{x},\dot{\textbf{x}},t) \approx L[u(\textbf{x},\dot{\textbf{x}},t)] = \sum_{k=1}^{d} \frac{u(\textbf{x}-h\hat{\textbf{e}}_k,\dot{\textbf{x}},t) - 2u(\textbf{x},\dot{\textbf{x}},t) + u(\textbf{x}+h\hat{\textbf{e}}_k,\dot{\textbf{x}},t)}{h^2}   $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $d$ is the dimension of $\textbf{x}$, $h \ll 1$, and $\hat{\textbf{e}}_k$ is the unit vector in the $k^{th}$ direction. Based on this approximation, we can then view the solution to this problem as finding a path based on the constraining &quot;dynamics&quot; you have listed. This can then be viewed from the perspective of Calculus of Variations, where we wish to find an optimal path. This allows us to specify a cost function to minimize, such as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J[\textbf{x}] = \int_{t_1}^{t_2}\left( L[u(\textbf{x},\dot{\textbf{x}},t)] - f(\textbf{x},\dot{\textbf{x}},t)\right)^2 dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $t_1$ and $t_2$ bound the time frame you care about, and your optimal path $\textbf{x}$ will minimize the cost. To make this path more easily approximated, let's approximate $\textbf{x}(t)$ by some basis, expressed as the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \textbf{x}_{h}(t) = \sum_{j}^{n} \textbf{a}_{j} \phi_{j}(t)$$&#xA;$$ \dot{\textbf{x}}_{h}(t) = \sum_{j}^{n} \textbf{a}_{j} \dot{\phi}_{j}(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By using this approximation, $\textbf{x}_{h}(t)$, in place of $\textbf{x}(t)$, the problem can be tackled by finding the coefficients $\textbf{a}_j$ such that the cost function is minimized. This optimization looks like it will be ugly, but this is at least one way you might go about solving it.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-07-08T02:59:32.743" LastActivityDate="2016-07-08T02:59:32.743" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24415" PostTypeId="1" AcceptedAnswerId="24420" CreationDate="2016-07-08T02:52:38.183" Score="2" ViewCount="977" Body="&lt;p&gt;I have a two-dimensional mesh generated by &lt;a href=&quot;https://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;triangle&lt;/a&gt; (the mesh generator software is not relevant). This software generates a perfect mesh for approximate the solution by piecewice linear functions (P1 mesh).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for the unit square [0,1]x[0,1] in 2D I have a file with the coordinates of its nodes (for example, a mesh with 5 nodes):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   0.0 0.0  # coordinates of node 1&#xA;2   1.0 0.0  # coordinates of node 2&#xA;3   1.0 1.0  # coordinates of node 3&#xA;4   0.0 1.0  # coordinates of node 4&#xA;5   0.5 0.5  # coordinates of node 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;called &lt;code&gt;coordinate.dat&lt;/code&gt;, which have 4 elements (triangles) whose conectivity is given by the file called &lt;code&gt;element.dat&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   1 5 4  # vertices of triangle 1&#xA;2   1 2 5  # vertices of triangle 2&#xA;3   2 3 5  # vertices of triangle 3&#xA;4   5 2 4  # vertices of triangle 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I need to program problems where the finite element spaces belong to  arbitrary polynomial degree, for example, approximate the solution by polynomials of degree 2, 3, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this, I need to build a new mesh with new nodes and a new conectivity file with more nodes. An example of the general mesh that I need:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9o7u9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9o7u9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already know how to calculate the coordinate of all new nodes on each element (it is just a convex combination easy to calculate) but I can't get a easy way to generate the new conectivity file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been thinking for several days how to program a general method for this, but I could not come up with any.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do you know some method to create meshes P2, P3, ... from a mesh P1?&lt;/strong&gt;&#xA;or more precisely, &lt;strong&gt;How can I build the new conectivity file?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="17630" LastActivityDate="2016-07-09T05:54:05.580" Title="Two-dimensional mesh in fem: generating P1, P2, P3,... mesh from a P1 mesh" Tags="|finite-element|mesh-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24417" PostTypeId="2" ParentId="24401" CreationDate="2016-07-08T09:30:49.497" Score="12" Body="&lt;p&gt;It is true that compilers are getting better and better at auto-vectorization, and for basic coefficient-wise operations like &lt;code&gt;2*A-4*B&lt;/code&gt; a library like Eigen cannot do much better than recent compilers. However, for slightly more complicated expressions like matrix products, reductions, transposition, powers, etc. the compiler cannot do much. On the other hand, Eigen can take advantage of the higher level knowledge of the expression semantic to explicitly vectorize them. Moreover, complex scalar types are not vectorized by compilers. You can check by yourself by disabling Eigen's explicit vectorization (&lt;code&gt;-DEIGEN_DONT_VECTORIZE&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="4337" LastActivityDate="2016-07-08T09:30:49.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24419" PostTypeId="1" AcceptedAnswerId="24425" CreationDate="2016-07-08T11:27:58.207" Score="3" ViewCount="821" Body="&lt;p&gt;I need to process some polygons in 3D. They are typically loaded from an OFF or STL file. Then I need to do some transformations (rotation, move, resize), I'd like to check whether points are inside the polygon or not, whether polygons overalp, and I'd like to merge several polygons into one. I'd also like to have raw access on points and surface indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that there are many libraries for 2D, but it's difficult to find something for 3D. Preferably usable with Python/NumPy.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-07-10T14:59:18.290" Title="Library for polygon processing in 3D" Tags="|computational-geometry|mesh-generation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24420" PostTypeId="2" ParentId="24415" CreationDate="2016-07-09T05:37:30.550" Score="1" Body="&lt;p&gt;The mesh for Pk elements is usually exactly the same as for P1. The only difference is that you also need to keep track of your edge indices for higher orders and associate degrees of freedom (DOFs) with edges (from P2 on) and elements (from P3 on). This is only a matter of defining local indices for nodes and edges with respect to your reference element, mapping to the actual mesh element and doing some index bookkeeping. Computing indices is quite straightforward if you figured that out and does not involve working with refined mesh data structures. The advantage of this technique is that you do not rely on a specific &quot;placement&quot; of DOFs on mesh nodes and generalizes easily to more general finite elements. I suggest you start over from scratch and consult the existing literature on the implementation of finite element methods. There is even a book with the title &quot;Understanding and Implementing the Finite Element Method&quot; by Gockenbach. While I have not read it myself, I am pretty sure that he explains these basics better than I ever could in this post. Other popular introductions usually also have a chapter on the implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you intended to do is however not a useless exercise. A popular approach for visualizing higher order polynomials is to use refined meshes, see also &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7468/fenics-visualizing-high-order-elements/7469#7469&quot; title=&quot;this question&quot;&gt;this question&lt;/a&gt;. In this case you need to deal with an interpolation to a refined mesh. Interpolation is easy if you know the vertices and refining is usually done recursively, i.e., you refine once, then again, etc. In this way it is easy to keep track of connectivities if you have a suitable index for your edges. Setting that up is a straightforward task since edges can be defined as a tuple of node indices which you can (again) define once and for all for your reference element and map to each of the actual mesh elements. However, since your question is related to setting up higher order elements, I will not elaborate on mesh refinement here.&lt;/p&gt;&#xA;" OwnerUserId="4260" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-07-09T05:54:05.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24422" PostTypeId="1" CreationDate="2016-07-09T18:52:18.047" Score="2" ViewCount="283" Body="&lt;p&gt;We have a system of equations, wherein the BC of one PDE is coupled with the source term of another PDE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a regular 2D unit grid in x and y.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two PDEs to be solved&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first PDE (elliptic diffusion problem) is defined only at $y = 1$, acting along the x-axis (i.e. it acts in the x-direction and only along the top of the cartesian grid). This x-axis is discretized with a fixed grid-spacing, generating a finite number of nodes. Let this set of nodes/co-ordinates be represented by $‘X’$.&lt;/li&gt;&#xA;&lt;li&gt;The second PDE is a time-varying diffusion problem. This is defined only along the y-axis, but for all x-nodes (i.e. for all $‘X’$), where the 1st PDE is being solved.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;br /&gt;&#xA;PDE1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla.(S \nabla a) = f(\omega,a)$$&#xA;BC1:   $\frac{\partial a}{\partial x} = 1$, at $x=1$ (Neumann)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BC2:   $a = 0$, at $x=0$ (Dirichelet)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br /&gt;&#xA;&lt;br /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PDE2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial B}{\partial t} = \nabla.( \left( \begin{array}{cc}&#xA;0 &amp;amp; 0 \\&#xA;0 &amp;amp; D  \end{array} \right) \nabla B)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BC1:   $B = 0$, at $y=0$ for all $'X'$, i.e. along the bottom face&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BC2:   $\frac{\partial B}{\partial y} = g(\omega, a)$, at $y=1$ (Neumann)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ and $g$ are linear functions in $\omega(x,y,t)$ and $a(x,y,t)$. $B$ is defined in the 2-D grid as $B(x,y,t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Importantly&lt;/strong&gt;, $$\omega=B_{y=1} \text{  for all  }  'X'$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e., the BC2 of the 2nd PDE couples with the Implicit Source Term of the 1st PDE along the top face of the Cartesian mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How would one approach this problem in order to obtain a numerical solution via a FD/FV implementation?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2016-07-17T16:15:34.797" Title="Coupling Boundary Condition of one PDE with source term of another PDE" Tags="|pde|boundary-conditions|finite-volume|mesh|coupling|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24423" PostTypeId="1" AcceptedAnswerId="24426" CreationDate="2016-07-10T08:21:59.193" Score="9" ViewCount="457" Body="&lt;p&gt;I have a sizable model (~5000 lines) written in C. It is a serial program, with no random number generation anywhere. It makes use of the FFTW library for functions using FFT - I do not know the details of the FFTW implementation , but I assume the functions therein are deterministic also (correct me if I'm in err).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue I cannot understand is that I am getting small differences in the results for identical runs on the same machine (same compiler, same libraries).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use double-precision variables, and to output the result in variable &lt;code&gt;value&lt;/code&gt; for example, I issue:&#xA;&lt;code&gt;fprintf(outFID, &quot;%.15e\n&quot;, value);&lt;/code&gt; or&lt;br&gt;&#xA;&lt;code&gt;fwrite(&amp;amp;value, 1, sizeof(double), outFID);&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I would constantly get differences such as:&lt;br&gt;&#xA;2.07843469652206&lt;strong&gt;4&lt;/strong&gt;e-16 vs. 2.07843469652206&lt;strong&gt;3&lt;/strong&gt;e-16&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have spent much time trying to figure out why this is. I initially thought one of my memory chips have gone bad, and I've ordered and replaced them, to no avail. I subsequently also tried running my code on a colleague's Linux machine, and I get differences of the same nature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be causing this? It is a small issue now, but I wonder if it is the &quot;tip of the iceberg&quot; (of a serious problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought I would post here instead of StackOverflow in case someone working with numerical models might have come across this issue. If anyone can shed light on this, I'd be much obliged. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Followup to comments:&lt;/strong&gt;&lt;br&gt;&#xA;Christian Clason and Vikram: first, thank you for your attention to my question. The articles you linked to suggest that: 1. rounding errors limits the accuracy, and 2. different code (such as introducing seemingly harmless print statements) can affect the results up to the machine epsilon. I should clarify that I am not comparing the effects &lt;code&gt;fwrite&lt;/code&gt; and &lt;code&gt;fprintf&lt;/code&gt; functions. I am using one OR the other. In particular, the &lt;em&gt;same&lt;/em&gt; executable is used for both runs. I am simply stating the issue occurs whether I use &lt;code&gt;fprintf&lt;/code&gt; OR &lt;code&gt;fwrite&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the code path (and executable) is the same, and the hardware is the same. With all these external factors held constant, where does the randomness come from, fundamentally? I suspected the bit flip happened due to faulty memory not retaining a bit correctly, which is why I replaced the memory chips, but that doesn't seem to be the issue here, I verified and you indicated. My program outputs thousands of these double precision numbers in a single run, and there are always a random handful that have random bit flips.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Followup to Christian Clason's first comment: Why is $2\cdot 10^{-16}$ the same as 0 within machine precision? The smallest positive number for a double is 2.22e-308, so shouldn't that be equal to 0? My program outputs thousands of values in the 10^-16 range (ranging from 1e-15 to 8e-17) and we've been seeing meaningful variations in our research project, so I'm hope we've not been looking at nonsensical numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Followup #2&lt;/strong&gt;:&lt;br&gt;&#xA;This is a plot of the time series output by the model, to aid in the offshoot discussion in the comments.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Kwsh2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kwsh2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20962" LastEditorUserId="20962" LastEditDate="2016-07-10T23:12:16.700" LastActivityDate="2016-08-24T13:57:01.317" Title="Small, unpredictable results in runs of a deterministic model" Tags="|floating-point|precision|computer-arithmetic|" AnswerCount="4" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="24425" PostTypeId="2" ParentId="24419" CreationDate="2016-07-10T14:59:18.290" Score="3" Body="&lt;p&gt;You may try GEOGRAM [1], a programming library that we are developing with my research team. It has import for several formats, including STL. If you love templates, there is also CGAL that has many functionalities [2] (but with it you will not have by-index access to the elements as you requested, CGAL uses pointers). There is also the GNU Triangulated Surface Library [3] with some interesting functionalities. Finally, you may try libIGL, it has index-based data structures (like geogram) and uses templates (like CGAL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need a graphic user interface, then you can use Graphite [5], it is a shell around Geogram (also developed by my research team).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] GEOGRAM: &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] CGAL: &lt;a href=&quot;http://www.cgal.org&quot; rel=&quot;nofollow&quot;&gt;http://www.cgal.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] GTS: &lt;a href=&quot;http://gts.sourceforge.net/&quot; rel=&quot;nofollow&quot;&gt;http://gts.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] libIGL: &lt;a href=&quot;http://libigl.github.io/libigl/&quot; rel=&quot;nofollow&quot;&gt;http://libigl.github.io/libigl/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] Graphite: &lt;a href=&quot;http://alice.loria.fr/software/graphite/doc/html/&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/software/graphite/doc/html/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-07-10T14:59:18.290" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24426" PostTypeId="2" ParentId="24423" CreationDate="2016-07-10T19:14:24.363" Score="9" Body="&lt;p&gt;There are aspects of modern computing systems that are inherently non-deterministic that can cause these kinds of differences.  As long as the differences are very small in comparison with the required accuracy of your solutions, there probably isn't any reason to worry about this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of what can go wrong based on my own experience.  Consider the problem of computing the dot product of two vectors x and y.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$d=\sum_{i=1}^{n} x_{i}y_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing this dot product requires computing the products $x_{i}y_{i}$ and then adding up the results.  The floating point multiplications should produce precisely the same results every time.  If the floating point additions are computed in the same order each time then the sum should be identical.  However, since floating point addition is not associative, you might get different results if the product of two vectors is computed in such a way that the additions are performed in different order.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you might compute the product of two vectors first as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$d=((x_{1}y_{1})+(x_{2}y_{2}))+(x_{3}y_{3})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$d=(x_{1}y_{1})+((x_{2}y_{2})+(x_{3}y_{3}))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How might this happen?  Here are two possibilities.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Multithreaded computations on parallel cores.  Modern computers typically have 2, 4, 8, or even more processor cores that can work in parallel.  If your code is using parallel threads to compute a dot product on multiple processors, then any random perturbation of the system (e.g. the user moved his mouse and one of the processor cores has to process that mouse movement before returning to the dot product) could result in a change in the order of the additions.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ALignment of data and vector instructions.  Modern Intel processors have a special set of instructions that can operate on (for example) for floating point numbers at a time.  These vector instructions work best if the data is aligned on 16 byte boundaries.  Typically, a dot product loop would break the data up into sections of 16 bytes (4 floats at a time.)  If you rerun the code a second time the data might be aligned differently with the 16 byte blocks of memory so that the additions are performed in a different order, resulting in a different answer.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can address point 1 by making your code run as a single thread and disabling all parallel processing.  You can address point 2 by requiring memory allocation to align blocks of memory (typcially you'd do this by compiling the code with a switch such as -align.)  If your code is still giving results that vary then there are other possibilities to look at. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://software.intel.com/en-us/articles/getting-reproducible-results-with-intel-mkl&quot;&gt;documentation from Intel&lt;/a&gt; discusses issues that can lead to non reproducibility of results with the Intel Math Kernel Library.  &lt;a href=&quot;https://software.intel.com/en-us/articles/consistency-of-floating-point-results-using-the-intel-compiler&quot;&gt;Another document&lt;/a&gt; from Intel that discusses compiler switches to use with Intel's compilers.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-07-10T22:49:32.730" LastActivityDate="2016-07-10T22:49:32.730" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24427" PostTypeId="1" CreationDate="2016-07-11T04:31:28.710" Score="2" ViewCount="170" Body="&lt;p&gt;I am trying to determine the order of my numerical method for resolving a fluid-structure interaction problem using the immersed boundary method. I am using Crank-Nicolson to resolve the fluid equations and the immersed body is updated in time using an explicit operator splitting + projection method. The CFL condition for the fluid equations is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ CFL = |u_\max(t)| \frac{\Delta t}{\Delta x} \,. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran one set of simulations where I keep the timestep size fixed and refine the spatial resolution of the fluid and body meshes. This way, I'm able to determine the spatial order of accuracy of my scheme. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, when I try to keep my spatial resolution fixed and vary the timestep, I sometimes violate the CFL condition. Not wanting to resort to very minuscule timestep sizes, I did a convergence study where I held the CFL number fixed and varied the spatial and temporal resolutions proportionality. This tells me the overall order of my entire numerical method, however, does not distinguish between the spatial and temporal convergence rates. Is it true that in this case, if the temporal accuracy is less than the spatial accuracy, then the results will tell me the temporal convergence rate? &lt;/p&gt;&#xA;" OwnerUserId="17368" LastActivityDate="2016-07-14T14:05:12.320" Title="Interpreting convergence study results, fixed CFL" Tags="|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24428" PostTypeId="1" AcceptedAnswerId="29741" CreationDate="2016-07-11T10:04:19.383" Score="6" ViewCount="318" Body="&lt;p&gt;Basically, I am trying to compute the SVD of a large Hermitian matrix $H$ using Lanczos iteration, while $H$ consists if a Toeplitz kernel $K$, which should be able to help speed up the matrix-vector computations(using FFT) over conventional Lanczos algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;H(i,j) = P_i * K_{ij}(i-j) * P^*_j&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, I need the largest few eigenvalues and eigenvectors of $H$, and I was wondering which Lanczos algorithms exactly meet this demand? I notice the &lt;a href=&quot;http://sun.stanford.edu/~rmunk/PROPACK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PROPACK&lt;/a&gt; package, but it seems to me it's not designed for this specific case here and the fact that it's written in Fortran...&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="20688" LastEditDate="2018-06-19T13:18:33.737" LastActivityDate="2018-06-19T13:40:01.990" Title="Lanczos algorithms for Hermitian system with Toeplitz kernel" Tags="|linear-algebra|matrix|iterative-method|svd|numerics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24429" PostTypeId="1" CreationDate="2016-07-11T12:19:03.310" Score="3" ViewCount="300" Body="&lt;p&gt;more specifically, $C x \log_2(x)- K = 0$, where $C$ and $K$ are constants ($\log_2$ means log base 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was solving a topcoder problem, &lt;a href=&quot;https://community.topcoder.com/stat?c=problem_statement&amp;amp;pm=3561&amp;amp;rd=6519&quot; rel=&quot;nofollow&quot;&gt;SortEstimate&lt;/a&gt;, which requires us to solve the aforementioned equation.&#xA;(Side question: can it be called a polynomial? No? Then what is the name of such an equation?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used Binary search to solve this in C++ and was able to do so successfully to high accuracy. But then I came across a rather different solution which I think,  having studied Numerical Analysis in college, may be using one of methods for root finding. I want to find the method, but haven't found anything useful on google or other numerical analysis resources available online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's &lt;a href=&quot;https://ideone.com/UoPf7Q&quot; rel=&quot;nofollow&quot;&gt;the code&lt;/a&gt; – also $K$ in the equation is the variable &lt;code&gt;time&lt;/code&gt; here &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;using namespace std;&#xA;class SortEstimate&#xA;{&#xA;public:&#xA;&#xA;    double howMany(int c, int time){&#xA;&#xA;    long double x = 7; &#xA;    long double r = double(time)/double(c); &#xA;    for (int i = 0; i&amp;lt;1000; i++) { &#xA;      x = x - (x * log(x)/log(2.0) - r)/(1.0 + log(x)/log(2.0)); &#xA;    } &#xA;    return x; &#xA;&#xA;  } &#xA;};&#xA;&#xA;int main()&#xA;{&#xA;    SortEstimate sort;&#xA;    cout&amp;lt;&amp;lt;sort.howMany(1,8)&amp;lt;&amp;lt;endl;//4&#xA;    cout&amp;lt;&amp;lt;sort.howMany(2,16)&amp;lt;&amp;lt;endl;//4&#xA;    cout&amp;lt;&amp;lt;sort.howMany(37,12392342)&amp;lt;&amp;lt;endl;//23105&#xA;    cout&amp;lt;&amp;lt;sort.howMany(1,2000000000)&amp;lt;&amp;lt;endl;//7.6375e+07&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the &lt;code&gt;for&lt;/code&gt; loop, what does the statement mean? All I can think is it should be a numerical method. Any information would be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not the right community to ask this doubt, then please refer me to the correct one.&lt;/p&gt;&#xA;" OwnerUserId="20972" LastEditorUserId="4474" LastEditDate="2016-07-11T17:20:36.483" LastActivityDate="2016-07-12T17:31:50.640" Title="Fastest method forfinding a solution to x*log(x)" Tags="|numerical-analysis|c++|roots|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24430" PostTypeId="1" CreationDate="2016-07-11T12:22:00.333" Score="1" ViewCount="93" Body="&lt;p&gt;I am pretty new to the CFD and I wanted to start with Chorin's projection. The starting problem is just a free jet flowing in the investigated area. I got terrible oscillations almost immediately and I think it is due to the wrong treated boundary conditions for intermediate velocity $u^*$. There is just a noob mistake somewhere in the process:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Calculation of the $u^*$ using:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u^* = u^{n}+\Delta t(u^n\cdot\nabla)u^n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Solving the Poisson equation with the source term $\frac{\Delta t}{\rho}\nabla \cdot u^*$ and then updating the velocity:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u^{n+1} = u^* + \frac{\Delta t}{\rho}\nabla p^{n+1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Where in the process should the BC for $u^*$ step in?&lt;/li&gt;&#xA;&lt;li&gt;How can I use a boundary condition $u^*=u^{n+1}$ if I don't have the $u^{n+1}$ yet?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I need zero Neumann BC for velocity and zero Dirichlet BC for pressure.&lt;/p&gt;&#xA;" OwnerUserId="17721" LastActivityDate="2016-07-11T12:22:00.333" Title="Oscillations in Chorin's method due to the BC" Tags="|finite-difference|fluid-dynamics|navier-stokes|projection|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24431" PostTypeId="1" AcceptedAnswerId="24432" CreationDate="2016-07-11T12:27:49.577" Score="-2" ViewCount="153" Body="&lt;p&gt;In this bit of code, the X and Y arrays should be identical but for some reason, that I CANNOT figure out for the life of me, X[0] is always 1 rather than 0. I have tried initializing the whole array to 0, putting X[0]=0 in directly, using a constant, and no matter what it is always one and the problem doesn't occur for the Y array. Any insight would be great. Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#define N 3&#xA;#define M 3&#xA;#define A 0.0&#xA;#define B 1.0&#xA;#define C 0.0&#xA;#define D 1.0&#xA;&#xA;&#xA;int main()&#xA;    double h_x;&#xA;    h_x= (double) 1/(N+1);&#xA;    double h_y;&#xA;    h_y= (double) 1/(M+1);&#xA;    double X[N+2];&#xA;    double Y[M+2];&#xA;    double xhalf[N+1];&#xA;    double yhalf[M+1];&#xA;    int i,j;&#xA;    double o,k,l;&#xA;&#xA;    X[0]=C; X[N+2]=D; Y[0]=C; Y[M+2]=D; xhalf[0]=h_x/2; yhalf[0]=h_y/2;&#xA;    for(i=1; i&amp;lt;=N; i++)&#xA;    {&#xA;&#xA;        X[i]= X[i-1]+h_x;&#xA;        xhalf[i]=xhalf[i-1]+h_x;&#xA;        printf(&quot;%e\n&quot;, X[i]);&#xA;    }&#xA;    for(i=1; i&amp;lt;=M; i++)&#xA;    {&#xA;&#xA;        Y[i]=Y[i-1]+h_y;&#xA;        yhalf[i]=yhalf[i-1]+h_y;&#xA;        printf(&quot;%e\n&quot;, Y[i]);&#xA;    }&#xA;return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.25&#xA;1.50&#xA;1.75&#xA;2.5e-1&#xA;5.0e-1&#xA;7.5e-1&lt;/p&gt;&#xA;" OwnerUserId="20973" LastActivityDate="2016-07-16T14:11:44.040" Title="Array initialization in C" Tags="|c|computer-arithmetic|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24432" PostTypeId="2" ParentId="24431" CreationDate="2016-07-11T12:45:50.610" Score="0" Body="&lt;p&gt;I ran your code and it is giving correct output, I think.&#xA;You should check the code in your system.Also there was a syntax error in your code.&lt;br/&gt;&#xA;check the output I'm getting in the stdout section: &lt;a href=&quot;https://ideone.com/9siAoO&quot; rel=&quot;nofollow&quot;&gt;https://ideone.com/9siAoO&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20972" LastActivityDate="2016-07-11T12:45:50.610" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24433" PostTypeId="2" ParentId="24429" CreationDate="2016-07-11T15:42:52.480" Score="8" Body="&lt;p&gt;This equation is not polynomial. Assuming both $K$ and $C$ are positive (as in your linked problem), then the solution of $C x \ln_2(x) - K = 0$ can be found in terms of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lambert_W_function&quot; rel=&quot;nofollow&quot;&gt;Lambert $W$ function&lt;/a&gt; or the &lt;a href=&quot;https://en.wikipedia.org/wiki/Wright_Omega_function&quot; rel=&quot;nofollow&quot;&gt;Wright $\omega$ function&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x = \frac{D}{W_0(D)} = \frac{D} {\omega\left(\ln\left( D \right)\right)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D = \ln(2)K/C$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1007%2FBF02124750&quot; rel=&quot;nofollow&quot;&gt;Corless, et al. 1996&lt;/a&gt; is a good reference for implementing the Lambert $W$. You can find some basic Matlab code to compute it numerically &lt;a href=&quot;http://blogs.mathworks.com/cleve/2013/09/02/the-lambert-w-function/&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;, which should be trivial to convert to C++. The performance of this code could be improved by using series and asymptotic expansions to find a better initial guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for what the C++ code in your question is doing, it appears to be nothing more than a naïve implementation of &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow&quot;&gt;Newton's method&lt;/a&gt;. The &lt;code&gt;for&lt;/code&gt; loop just iterates one the recurrence relation a large number of times. It assumes that the system will converge after 1,000 steps. This is probably a reasonable assumption, but is quite inefficient. The Matlab code I linked to uses a higher order scheme, &lt;a href=&quot;https://en.wikipedia.org/wiki/Halley%27s_method&quot; rel=&quot;nofollow&quot;&gt;Halley's method&lt;/a&gt;, and a convergence condition based on a tolerance.&lt;/p&gt;&#xA;" OwnerUserId="4474" LastEditorUserId="4474" LastEditDate="2016-07-12T17:31:50.640" LastActivityDate="2016-07-12T17:31:50.640" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24434" PostTypeId="1" CreationDate="2016-07-11T16:53:23.547" Score="1" ViewCount="215" Body="&lt;p&gt;I am looking for an algorithm that fits a parabola to a set of data points. However, no data point may be below the parabola. All points must be above the parabola. Is there an standard algorithm for finding that parabola?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a 2D problem. I have about 20 points.&lt;/p&gt;&#xA;" OwnerUserId="20361" LastActivityDate="2016-07-13T20:17:50.250" Title="Finding an enclosing parabola for a set of points" Tags="|curve-fitting|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24435" PostTypeId="2" ParentId="24434" CreationDate="2016-07-11T17:41:49.547" Score="1" Body="&lt;p&gt;I have never heard of the need for solving a problem like this, so I am unsure if there's a standard algorithm for your question. Without the need to have points above the parabola, this problem would be a typical least square problem. But due to your constraint, you need to formulate the problem differently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there's a number of ways one might approach this, but one idea I have is to formulate the problem as the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ J(\vec{\beta}) = \frac{1}{2N}\sum_{i=1}^{N}\lbrace \delta_i^2 + \gamma\max(\delta_i,0)^2\rbrace$$&#xA;where&#xA;$$ \delta_i = f(x_i,y_i, \vec{\beta}) - z_i$$&#xA;$$ \vec{\beta} = \left[\beta_1, \beta_2, \beta_3,\beta_4, \beta_5, \beta_6\right]^{T} $$&#xA;$$ f(x,y,\vec{\beta}) = \beta_1 + \beta_2x + \beta_3 y + \beta_4 x^2 + \beta_5 y^2 + \beta_6 x y $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and where $\gamma$ is a coefficient that amplifies a penalization term for when the current parabola fit has data points that fall below it. Based on this formulation, you can try to minimize $J(\vec{\beta})$ with respect to $\vec{\beta}$ through some iterative optimization procedure.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-07-11T23:32:20.957" LastActivityDate="2016-07-11T23:32:20.957" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="24440" PostTypeId="1" CreationDate="2016-07-12T19:44:54.763" Score="7" ViewCount="141" Body="&lt;p&gt;I don't think I've ever seen any scientific computational software that I would consider &quot;good&quot; in terms of software design. I don't mean in terms of functionality, but in terms of good high-level design, coding standards, readability, etc. Most things I've encountered have some degree of bad code smells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are some examples of really good computational science software engineering?&lt;/strong&gt; Why do you like it? Preferably open source so we can see it, any language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To minimize the subjective nature of the question, and because there's no one &quot;right way&quot; to do things, &lt;strong&gt;please give concrete examples of things like software design patterns, agile development practices, regression testing, and similar things implemented in scientific codes.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://kotaku.com/5975610/the-exceptional-beauty-of-doom-3s-source-code&quot; rel=&quot;nofollow&quot;&gt;I like this post titled &quot;The Exceptional Beauty of Doom3's Source Code.&quot;&lt;/a&gt; I'd love to see a similar example in the scientific world.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastEditorUserId="4736" LastEditDate="2016-07-12T22:11:54.697" LastActivityDate="2016-07-12T22:11:54.697" Title="What is some of the best software design you've seen in a numerical code?" Tags="|software|" AnswerCount="0" CommentCount="4" ClosedDate="2016-07-12T22:27:21.477" ContentLicense="CC BY-SA 3.0" />
  <row Id="24441" PostTypeId="2" ParentId="24423" CreationDate="2016-07-12T20:53:41.567" Score="1" Body="&lt;p&gt;While it's true that expression term evaluation order changes may very well occur due to multi-core/multi-thread processing scenarios, don't forget that there may be (even though it's a long shot) some kind of hardware design flaw at work.&#xA;Remember the Pentium FDIV problem?  (See &lt;a href=&quot;https://en.wikipedia.org/wiki/Pentium_FDIV_bug&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Pentium_FDIV_bug&lt;/a&gt; ).&#xA;Some time ago, I worked on pc-based analog circuit simulation sofware.   Part of our methodology involved developing regression-test suites, which we would run against nightly builds of the software.   With many of the models we developed, iterative methods (e.g. Newton-Raphson (&lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Newton%27s_method&lt;/a&gt;) and Runge-Kutta) were used extensively in the simulation algorithms.  With analog devices, it is often the case that internal artifacts, such as voltages, currents, etc., occur having extremely small numeric values.  These values, as part of the simulation process, are incrementally varied over (simulated) time.  The magnitude of these changes may be extremely small, and what we often observed was that subsequent FPU operations on such delta values bordered on the &quot;noise&quot; threshold of the FPU's precision (64-bit floating has a 53-bit mantissa, IIRC).  That, coupled with the fact that we often had to introduce &quot;PrintF&quot; logging code into models to allow debugging (ah, the good ol' days!), practically guaranteed sporadic results, on a daily basis!  So, what's all this mean?   You have to expect to see differences under such circumstances, and the best thing to do is define and implement a way to decide (magnitude, frequency, trend etc) when/how to ignore them.&lt;/p&gt;&#xA;" OwnerUserId="20988" LastActivityDate="2016-07-12T20:53:41.567" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24442" PostTypeId="2" ParentId="11276" CreationDate="2016-07-13T02:05:41.793" Score="9" Body="&lt;p&gt;&lt;a href=&quot;https://gitlab.com/wlandry/ftensor&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;FTensor&quot;&gt;FTensor&lt;/a&gt; is a lightweight, header only, fully templated library that includes ergonomic summation notation.  It has been tested extensively in 2, 3, and 4 dimensions, but should work fine for any number of dimensions.&lt;/p&gt;&#xA;" OwnerUserId="17532" LastEditorUserId="17532" LastEditDate="2021-05-25T18:35:17.623" LastActivityDate="2021-05-25T18:35:17.623" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24445" PostTypeId="1" CreationDate="2016-07-13T13:45:56.687" Score="0" ViewCount="1033" Body="&lt;p&gt;I have been running a finite-element simulation where, for the sake of convergence, I chose to stay in small deformation hypothesis (in comsol, &lt;em&gt;include non-linear geometry&lt;/em&gt; box not checked for who that may concern). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my simulation is made-up so that geometry actually undergoes very large deformations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under comsol, I've read that rotation matrix is computed in a different manner (some kind of linear version). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : in a finite-element simulation with a presence of a rigid body, do the rotation matrix of that rigid body is computed AFTER the displacement field computation or BEFORE ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other terms, does the fact that the rotation matrix is linearized affects the displacement of the rigid body ?&lt;/p&gt;&#xA;" OwnerUserId="20191" LastActivityDate="2016-07-13T22:52:42.717" Title="How does rigid body rotation affect resuts in a simulation?" Tags="|finite-element|comsol|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24446" PostTypeId="1" CreationDate="2016-07-13T16:05:21.833" Score="1" ViewCount="106" Body="&lt;p&gt;I am looking for a mesh partitioner, where I can specify overlap, for example h = 3. &#xA;I have looked into metis, but I wasn't able to find such a functionality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other package which provides this functionality?&lt;/p&gt;&#xA;" OwnerUserId="17150" LastEditorUserId="1804" LastEditDate="2016-07-13T16:17:52.303" LastActivityDate="2016-07-13T16:17:52.303" Title="Mesh partitioner with user-defined overlap" Tags="|finite-element|computational-geometry|mesh-generation|mesh|partitioning|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24447" PostTypeId="1" CreationDate="2016-07-13T19:23:02.297" Score="1" ViewCount="98" Body="&lt;p&gt;I am trying to calculate the three-term connection coefficients&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;Λ_{l,m}^{d_1,d_2,d_3} = ∫_{-∞}^∞ φ^{(d_1)}(x) φ^{(d_2)}_l(x) φ^{(d_3)}_m(x) dx&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for Daubechies wavelets numerically using Python. &lt;a href=&quot;https://www.nada.kth.se/utbildning/grukth/exjobb/rapportlistor/2004/rapporter04/besora_jordi_04091.pdf&quot; rel=&quot;nofollow&quot;&gt;This report (p. 20)&lt;/a&gt; gives a nice overview of how to calculate them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a &lt;a href=&quot;https://manuels.github.io/db_cc3/docs/cc3_2.html&quot; rel=&quot;nofollow&quot;&gt;(well documented) implementation&lt;/a&gt; in python and for the two-term connection coefficients had no problems but when I compare my 3-term results with the ones of Resnikoff and Wells (1998) I find lots of errors (only the (l=m) coefficients appear to be correct).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am starring at the code for weeks now, but I cannot find any errors. I would be very pleased if somebody would give me a hint where the error might be.&lt;/p&gt;&#xA;" OwnerUserId="20998" LastActivityDate="2016-07-13T19:23:02.297" Title="Computing 3-term Connection Coefficients for Wavelets" Tags="|python|wavelet|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24448" PostTypeId="2" ParentId="24428" CreationDate="2016-07-13T19:34:35.410" Score="3" Body="&lt;p&gt;How large are we talking, here? If the problem can fit onto a single node, you can likely just solve it through Hermitian matrix computations and call it a day, see &lt;a href=&quot;http://slepc.upv.es/&quot; rel=&quot;nofollow&quot;&gt;SLEPc&lt;/a&gt; for fast implementations of this. Note that SLEPc is written in C but has Fortran and Python wrappers as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addressing your problem more directly: I do not know of any SVD/EVD solvers that will exploit your Toeplitz kernal and matrix structure without modification. That said, it shouldn't be hard to construct something like this yourself. A common method for extracting the largest eigenvalues (singular values) of a matrix is the Arnoldi method, usually implemented using &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow&quot;&gt;ARPACK&lt;/a&gt;. Here, you can simplify modify the matrix-vector multiplication subroutine yourself to exploit the Toeplitz structure through an FFT, as you said.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastEditorDisplayName="user20857" LastEditDate="2016-07-13T19:42:46.100" LastActivityDate="2016-07-13T19:42:46.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24449" PostTypeId="2" ParentId="24434" CreationDate="2016-07-13T20:17:50.250" Score="3" Body="&lt;p&gt;Choward already sketched a good approach, I am just going to elaborate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the equation for a parabola in the plane $\mathbb{R}^2$ is given by the zero locus of a quadratic equation $$f_a(x,y) = a_{11}x^2 + a_{12}xy + a_{22}y^2 + a_{1}x + a_{2}y + a_{0} = 0,$$ where $a = (a_{11}, a_{12},  a_{22}, a_{1}, a_{2}, a_{0}) \in \mathbb{R}^6$ is the vector of unknown coefficients. However, in gnereal such equation can describe any conic sections, such as a hyperbola, an ellipse and can even degenerate to a pair of striaght lines (that happens whenever the plynomial $f_a(x,y)$ is reducible). To make sure you get a parabola, you need to impose the restriction $g(a) = a_{12}^2 - 4 a_{11} a_{22} = 0$ (and to make sure the polynomial is irreducible). If you want all of your sample points $(x_j,y_j), \,\,\, j=1,...,N$  to be above the parabole, you simply have to impose also the linear with respect to $a$ restriction $l_j(a) = f_a(x_j,y_j) \geq 0$ for all $j=1,...,N$ as well as let's say $a_{11} \geq 0$. Now take the least square functional $$J(a) = \sum_{j=1}^{N} \left(f_a(x_j,y_j)\right)^2$$ which is a quadratic (so convex) function of $a \in \mathbb{R}^6$ and find its minimum on the set of restrictions $$R = D \cap A$$ where&#xA;$$D = \{a \in \mathbb{R}^6  \, :\,  l_j(a) = f_a(x_j,y_j) \geq 0 ,\, a_{11}\geq 0 \}$$ &#xA;$$A = \{a \in \mathbb{R}^6  \, :\,  g(a) = a_{12}^2 - 4 a_{11} a_{22} = 0\}.$$ In order to do that, you can form the Lagrange multiplier cost function &#xA;$$L(a,\lambda) = J(a) + \lambda g(a)$$ and look for its minimum on the convex polyhedral set (e.g. a set determined by linear inequalities like in our case)&#xA;$$P = \{(a,\lambda) \in \mathbb{R}^6 \times \mathbb{R}  \, :\,  l_j(a) = f_a(x_j,y_j) \geq 0 ,\, a_{11}\geq 0\} = D \times \mathbb{R}.$$ This is a convex probelm with quadratic cost function $L$ and linear restricitions $P$, so I beleive most programing languages would have standard functions that would solve this kind of calculus problem: find the minimum of $L(a,\lambda)$ on $P$. I think one can simplify the whole thing by taking $a_{11} = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a parabola given by $y = f_a(x) = a_{11} x^2 + a_1 x + a_0$ you simply have $f_a(x,y) = f_a(x) - y$ and $g(a) \equiv 0$ so the problem simplifies significantly and there is no need for Lagrange multipliers. &lt;/p&gt;&#xA;" OwnerUserId="20964" LastActivityDate="2016-07-13T20:17:50.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24450" PostTypeId="2" ParentId="24445" CreationDate="2016-07-13T21:15:44.693" Score="2" Body="&lt;p&gt;You question is unclear.  However, rotations in the context of linear mechanics are often confusing. Therefore a response is probably needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I will assume you have a deformable body and a rigid body in your simulation, i.e., there are two objects that may interact.&#xA;2) I will assume the deformable body is subject to a torque while the rigid body has a constant rotational velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a typical FE simulation under these conditions, the rigid body is treated as a node at which the Euler equations are solved.  If contact is expected then some extra computations are done.  That means that rotations are taken care of reasonably accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the deformable body is linear elastic (or small-strain elastic-plastic/viscoelastic), rotations are typically accounted for by using an objective stress rate (such as the Jaumann rate).  More accurate results are obtained by unrotating the stress and rate of deformation using the deformation gradient, updating the stress, and then rotating back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same approach can be used for nonlinear materials, even at large strains.  However, a more convenient approach for &quot;simple&quot; material models is to use a large deformation strain measure and use related constitutive models.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2016-07-13T21:15:44.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24451" PostTypeId="2" ParentId="24445" CreationDate="2016-07-13T22:38:58.077" Score="1" Body="&lt;p&gt;Due to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler%27s_rotation_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler's rotation theorem&lt;/a&gt; a rigid body in 3D space has 3 rotational degrees of freedom, (plus 3 displacement degrees of freedom for a total of 6 degrees of freedom.). This typically means that the rotation matrix (which has 9 components) is &lt;em&gt;not&lt;/em&gt; the primary unknown but, when needed, is computed from the rigid body's rotational DOFs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several different choices for the three rotation parameters are possible, see &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Rotation_formalisms_in_three_dimensions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rotation formalisms in three dimensions&lt;/a&gt;&quot;. In general the dependence between the rotation parameters and the rotation matrix is non linear, but when $\sin\theta \approx \theta$, simplified expression can be obtained. E.g., for a small $\theta$ rotation about the $z$ axis, we have&#xA;\begin{equation}&#xA;\begin{bmatrix} \cos\theta &amp;amp; -\sin\theta &amp;amp; 0 \\&#xA;\sin\theta &amp;amp; \cos\theta &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 1&#xA;\end{bmatrix}&#xA;\approx&#xA;\begin{bmatrix} 1 &amp;amp; -\theta &amp;amp; 0 \\&#xA;\theta &amp;amp; 1 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 1&#xA;\end{bmatrix}&#xA;\end{equation} &#xA;As you can see the &quot;linearised&quot; rotation matrix is correct up to $\theta^2$ terms, so in general it is not possible to use small rotation theory when $\theta$ is not small with respect to unity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by Biswajit Banerjee in his &lt;a href=&quot;https://scicomp.stackexchange.com/a/24450/1744&quot;&gt;answer&lt;/a&gt;, formulations with finite rotations and small/infinitesimal strains are possible, but I would not call them &quot;fully linear&quot; because the dependence of the rotation matrix from the rotation parameters is still non-linear.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-07-13T22:52:42.717" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24452" PostTypeId="5" CreationDate="2016-07-14T10:17:11.620" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-07-14T10:17:11.620" LastActivityDate="2016-07-14T10:17:11.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24453" PostTypeId="4" CreationDate="2016-07-14T10:17:11.620" Score="0" Body="R is a language and environment for statistical computing and graphics that provides a wide variety of statistical (linear and nonlinear modelling, classical statistical tests, time-series analysis, classification, clustering, ...) and graphical techniques and is highly extensible. (Questions about problems and bugs encountered when using R belong on the R-help mailing list (https://stat.ethz.ch/mailman/listinfo/r-help).)" OwnerUserId="6585" LastEditorUserId="1804" LastEditDate="2016-07-14T10:33:40.423" LastActivityDate="2016-07-14T10:33:40.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24454" PostTypeId="5" CreationDate="2016-07-14T10:19:43.273" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-07-14T10:19:43.273" LastActivityDate="2016-07-14T10:19:43.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24455" PostTypeId="4" CreationDate="2016-07-14T10:19:43.273" Score="0" Body="Data Analysis is the process of evaluating data using analytical and logical reasoning to examine each component of the data provided. " OwnerUserId="6585" LastEditorUserId="6585" LastEditDate="2016-07-14T13:48:52.580" LastActivityDate="2016-07-14T13:48:52.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24456" PostTypeId="1" CreationDate="2016-07-14T11:00:46.923" Score="0" ViewCount="137" Body="&lt;p&gt;I would like to know some GNU libraries or lists of downloadable Tersoff potentials. I couldn't find that many of them, and I also would like to know if there is a reason as to why this is the case.&lt;/p&gt;&#xA;" OwnerUserId="19740" LastActivityDate="2020-01-04T06:32:31.340" Title="List of Tersoff potentials?" Tags="|molecular-dynamics|computational-chemistry|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24457" PostTypeId="2" ParentId="23924" CreationDate="2016-07-14T11:07:51.610" Score="5" Body="&lt;p&gt;I suppose, you right and your network is not that big to 100%-utilize the GPU. The bottle-neck here seems to be not the GPU itself, but the transfer rate between RAM and VRAM and here the difference between 750 Ti and 780 Ti is not that significant. You can try to improve the training speed by hiding the latency of memory transfer - you have to assure you transfer new data set to the card, while it's calculating the previous one. You may also need to split you task in to smaller portions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also important to notice, that 750 Ti uses a newer (Maxwell) GPU-architecture, while 780 Ti uses the predecessor (Kepler) GPU-generation. See &lt;a href=&quot;https://scicomp.stackexchange.com/questions/12960/maxwell-vs-kepler-for-gpu-computation&quot;&gt;here&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="6585" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-07-14T11:22:58.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24458" PostTypeId="2" ParentId="24427" CreationDate="2016-07-14T14:05:12.320" Score="2" Body="&lt;p&gt;If you are trying to isolate spatial error, you will want to choose a fixed but very small time step so you can guarantee that temporal error is orders of magnitude smaller than spatial for every mesh considered. And obviously if you want to isolate temporal error, do the opposite, choose a very fine spatial discretization and vary the time step.&lt;/p&gt;&#xA;" OwnerUserId="3309" LastActivityDate="2016-07-14T14:05:12.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24459" PostTypeId="1" CreationDate="2016-07-14T15:24:51.827" Score="1" ViewCount="38" Body="&lt;p&gt;Consider linear equation $Au = f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to solve it with iterative method (assuming $A$ is good). &#xA;First order iterative method is:&#xA;$$&#xA;u^{k+1} = u^k - \alpha_{k+1}(Au^k - f),&#xA;$$&#xA;The second degree method is:&#xA;$$&#xA;u^{k+1} = u^k - \alpha_{k+1}(Au^k - f) - \beta(u^k - u^{k-1}).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For both methods we can define iteration parameters $\alpha_k$, $\beta_k$ via minimax problem which solution is Chebyshev polynomials. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is good, but it seems to me, that convergence speed is the same for both cases and is&#xA;$$&#xA;\|\varepsilon^{k}\| \leq \frac{2\sigma^k}{1+\sigma^{2k}}\|\varepsilon^{0}\|,&#xA;$$&lt;br&gt;&#xA;where $u - u^k = \varepsilon^{k}$ approximation error after the $k$-th iteration and $\sigma$ is constant dependent on operator spectrum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only idea I have, that first order iterations optimal for chosen $k$ for which coefficients are computed, while second-order iteration is optimal on every step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would greatly appreciate any work on this to clear-up those details.&lt;/p&gt;&#xA;" OwnerUserId="16620" LastActivityDate="2016-07-14T15:24:51.827" Title="Difference between Chebyshev first and second degree iterative methods" Tags="|linear-algebra|optimization|numerical-analysis|linear-solver|iterative-method|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24460" PostTypeId="1" AcceptedAnswerId="24465" CreationDate="2016-07-14T18:22:26.127" Score="9" ViewCount="1534" Body="&lt;p&gt;I have an objective function $E$ dependent on a value $\phi(x, t = 1.0)$, where $\phi(x, t)$ is the solution to a PDE. I am optimizing $E$ by gradient descent on the &lt;em&gt;initial condition&lt;/em&gt; of the PDE: $\phi(x, t = 0.0)$. That is, I update $\phi(x, t = 0.0)$ and then have to integrate the PDE to compute my residual. That means, if I were to do a line search for the gradient descent step size (call it $\alpha$), for every potential value of $\alpha$ I would have to integrate the PDE all over again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case that would be prohibitively expensive. Is there another option for adaptive gradient descent step size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not just looking for mathematically principled schemes here (although of course that's better if something exists), but would be happy with anything that is generally better than a static step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="20634" LastActivityDate="2016-10-17T19:10:02.810" Title="Adaptive gradient descent step size when you can't do a line search" Tags="|optimization|pde|conjugate-gradient|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="24461" PostTypeId="2" ParentId="20722" CreationDate="2016-07-14T20:49:55.720" Score="2" Body="&lt;p&gt;As mentioned by Matt Knepley, this is naturally formulated as a system of partial differential algebraic equations. Because you're in Matlab, you could consider doing the spatial discretization yourself (e.g. finite difference, finite volume, finite element) to obtain a system of DAE's, then use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Method_of_lines&quot; rel=&quot;nofollow&quot;&gt;method of lines&lt;/a&gt; to step forward in time. You can discretize both the Nernst-Planck and Poisson equations the same way in space. This should give a list of equations like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial n_{i,x_j}}{\partial t} = f(\{n_{i,x_j}\}, \{\phi_{x_j}\})$$&#xA;$$0 = g(\{n_{i,x_j}\}, \{\phi_{x_j}\})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $x_j$ refers to the value of the variable at a particular discretized $x$ coordinate, and the functions $f$ and $g$ depend on your spatial discretization and will be different for each spatial location. If you have $N$ species, you should have $N$ of the first equation at each grid point and one of the second, leading to the same number of unknowns and equations (I don't know about your boundary conditions, so you can treat them as extra equations, substitute them in algebraically to the discretized equations at the edges, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're going to use CN time stepping, this will naturally lead to a non-linear system of equations you have to solve at each time step, which you could do using the Newton-Raphson method. However, unless you are particularly set on CN time stepping, I would suggest considering Matlab's &lt;code&gt;ode15s&lt;/code&gt; (see &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/ode15s.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt;), which is reasonably good at time stepping through coupled, nonlinear (index-1) DAE's like this one. It uses variable order, variable size stepping, and I've found it to work well for coupled Nernst-Planck style equations. To couple the differential and algebraic equations in &lt;code&gt;ode15s&lt;/code&gt;, you will need to pass a (singular) mass matrix to &lt;code&gt;ode15s&lt;/code&gt;. It will take care of doing the Newton-Raphson iterations for you, and you can either let it estimate the Jacobian numerically or provide a function which provides the analytical Jacobian.&lt;/p&gt;&#xA;" OwnerUserId="21008" LastActivityDate="2016-07-14T20:49:55.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24462" PostTypeId="1" AcceptedAnswerId="24474" CreationDate="2016-07-14T22:13:42.473" Score="1" ViewCount="130" Body="&lt;p&gt;I am new to machine learning and statistical analysis and am having trouble figuring how I should go about a problem I have. I believe that I understand the gradient descent algorithm and how it optimizes the parameters of a function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, so far I am have mostly seen examples of gradient descent being applied to, univariate and multivariate, first order linear equations. For linear regression models, &lt;code&gt;numpy&lt;/code&gt; vector operations are an easy choice for computing the few components used in gradient descent, i.e. the hypothesis, loss, gradient, etc. However what if I want to apply gradient descent to a multivariate nonlinear equation, specifically one that has different functions across its input variables. Here is an example of such a function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_1, x_2) = \theta_1 x_1 + \theta_2 x_2^{\theta_3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case the first term is linear but the second is not and requires its own gradient function. The various gradient descent functions I have seen would not be able to optimize this function without hard changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose my question is, how do I generalize a gradient descent algorithm so that it can optimize multivariate nonlinear functions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach so far has been to allow a list of gradient functions (matching the number of parameters) to be passed to my gradient descent function and to apply these gradients, in parallel, to their respective parameters assuming the order is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this seem like a sound approach to what I am trying to accomplish? Or perhaps is what I am trying to accomplish unnecessary and reveals a fundamental misunderstanding on my part?&lt;/p&gt;&#xA;" OwnerUserId="21010" LastEditorUserId="21010" LastEditDate="2016-07-14T23:33:43.673" LastActivityDate="2016-07-18T14:32:50.133" Title="Vectorizing list of different functions for Gradient Descent" Tags="|optimization|python|nonlinear-equations|numpy|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24463" PostTypeId="1" CreationDate="2016-07-14T23:18:47.850" Score="1" ViewCount="679" Body="&lt;p&gt;I have a system of PDEs in cylindrical coordinates that needs to be solved:&#xA;1. Continuity equation&#xA;2. Incompressible Navier stokes ( in r &amp;amp; z coordinates)&#xA;3. Heat transfer equation with both conduction and convection terms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to solve these equations using PDE toolbox?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the link from the MATLAB website I referred to:&#xA;&lt;a href=&quot;http://www.mathworks.com/help/pde/ug/equations-you-can-solve.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/pde/ug/equations-you-can-solve.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I do not think my equations would fit into this form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is discretization by the finite difference method a better way? &#xA;Is application of the SIMPLE algorithm by Patankar necessary?&lt;/p&gt;&#xA;" OwnerUserId="21011" LastEditorUserId="123" LastEditDate="2016-07-15T19:23:48.963" LastActivityDate="2019-05-14T08:38:00.637" Title="Can I model laminar incompressible fluid flow and heat transfer in MATLAB's PDE toolbox?" Tags="|matlab|pde|finite-difference|fluid-dynamics|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24464" PostTypeId="1" CreationDate="2016-07-15T02:59:00.347" Score="1" ViewCount="1071" Body="&lt;p&gt;I have measurements of a quantity on a 3d grid. My measurements are distributed on four x-y planes similar to what is shown in the image below. The measurements roughly follow a Cartesian grid but have varying spacing and gaps. The measurements are more sparse in the z-direction but I expect the quantity to vary by an order of magnitude more in the x and y direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole domain is not convex but for now I will concentrate only on the subset of data points where the measurement planes overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YCAkx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YCAkx.png&quot; alt=&quot;data positions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in linearly interpolating the quantity in 3D (in the x-y region where the measurement planes overlap), integrating the quantities over all space, and determining gradients in each direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of trilinear interpolation along a cartesian grid (e.g. scipy.interpolate.RegularGridInterpolator) and barycentric interpolation with Delaunay triangulation (e.g. scipy.interpolate.LinearNDInterpolator). Is there an advantage to choosing one method over another?&#xA;I also found a gradient calculation in matplotlib.tri.TriInterpolator.gradient. Is this a good choice? &lt;/p&gt;&#xA;" OwnerUserId="14391" LastEditorUserId="14391" LastEditDate="2016-07-16T21:10:41.407" LastActivityDate="2016-07-21T15:51:22.447" Title="linearly interpolate and determine gradients for data on non-uniform grid" Tags="|interpolation|scipy|delaunay-triangulation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24465" PostTypeId="2" ParentId="24460" CreationDate="2016-07-15T09:04:36.850" Score="15" Body="&lt;p&gt;I'll begin with a general remark: first-order information (i.e., using only gradients, which encode slope) can only give you directional information: It can tell you that the function value decreases in the search direction, but not for how long. To decide how far to go along the search direction, you &lt;em&gt;need&lt;/em&gt; extra information (gradient descent with constant step lengths can fail even for convex quadratic problems). For this, you basically have two choices:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use &lt;strong&gt;second-order information&lt;/strong&gt; (which encodes curvature), for example by using Newton's method instead of gradient descent (for which you can always use step length $1$ sufficiently close to the minimizer).&lt;/li&gt;&#xA;&lt;li&gt;Trial and error (by which of course I mean using a proper &lt;strong&gt;line search&lt;/strong&gt; such as Armijo).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If, as you write, you don't have access to second derivatives, and evaluating the obejctive function is very expensive, your only hope is to compromise: use enough approximate second-order information to get a good candidate step length such that a line search needs only $\mathcal{O}(1)$ evaluations (i.e., at most a (small) constant multiple of the effort you need to evaluate your gradient).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possibility is to use &lt;strong&gt;Barzilai--Borwein step lengths&lt;/strong&gt; (see, e.g.,  &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=A066745B191ECBDE04DE09AA8EACF912?doi=10.1.1.22.4212&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow&quot;&gt;Fletcher: &lt;em&gt;On the Barzilai-Borwein method&lt;/em&gt;. Optimization and control with applications, 235–256, Appl. Optim., 96, Springer, New York, 2005&lt;/a&gt;). The idea is to use a finite difference approximation of the curvature along the search direction to get an estimate of the step size. Specifically, choose $\alpha_0&amp;gt;0$ arbitrary, set $g^0:=\nabla f(x^0)$ and then for $k=0,...$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Set $s^k = -\alpha_k^{-1} g^k$ and $x^{k+1}=x^k+s^k$&lt;/li&gt;&#xA;&lt;li&gt;Evaluate $g^{k+1}=\nabla f(x^{k+1})$ and set $y^k = g^{k+1}-g^{k}$&lt;/li&gt;&#xA;&lt;li&gt;Set $\alpha_{k+1} = \frac{(y^k)^Ty^k}{(y^k)^Ts^k}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This choice can be shown to converge (in practice very quickly) for quadratic functions, but the convergence is &lt;em&gt;not&lt;/em&gt; monotone (i.e., the function value $f(x^{k+1})$ can be larger than $f(x^k)$, but only once in a while; see the plot on page 10 in Fletcher's paper). For non-quadratic functions, you need to combine this with a line search, which needs to be modified to deal with the non-monotonicity. One possibility is choosing $\sigma_k \in (0,\alpha_k^{-1})$ (e.g., by backtracking) such that&#xA;$$ f(x^k - \sigma_k g^k) \leq \max_{\max(k-M,1)\leq j\leq k} f(x^j) - \gamma \sigma_k (g^k)^Tg^k,$$&#xA;where $\gamma\in(0,1)$ is the typical Armijo parameter and $M$ controls the degree of monotonicity (e.g., $M=10$). There's also a variant that uses gradient values instead of function values, but in your case the gradient is even more expensive to evaluate than the function, so that doesn't make sense here. (Note: You can of course try to blindly accept the BB step lengths and trust your luck, but if you need any sort of robustness -- as you wrote in your comments -- that would be a really bad idea.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative (and, in my opinion, much better) approach would be to use this finite difference approximation already in the computation of the search direction; this is called a &lt;strong&gt;quasi-Newton method&lt;/strong&gt;. The idea is to incrementally build an approximation of the Hessian $\nabla^2 f(x^k)$ by using differences of gradients. For example, you could take $H_0=\mathrm{Id}$ (the identity matrix) and for $k=0,\dots$ solve&#xA;$$H_{k}s^{k} = -g^{k},\label{cc1}\tag{1}$$&#xA;and set&#xA;$$H_{k+1} = H_k + \frac{(y^k-H_ks^k)^T(s^k)^T}{(s^k)^Ts^k}$$&#xA;with $y^k$ as above and $x^{k+1} = x^k +s^k$. (This is called &lt;em&gt;Broyden update&lt;/em&gt; and is rarely used in practice; a better but slightly more complicated update is the &lt;em&gt;BFGS update&lt;/em&gt;, for which -- and more information -- I refer to Nocedal and Wright's book &lt;a href=&quot;http://www.springer.com/us/book/9780387303031&quot; rel=&quot;nofollow&quot;&gt;Numerical Optimization&lt;/a&gt;.) The downside is that a) this would require solving a linear system in each step (but only of the size of the unknown which in your case is an initial condition, hence the effort should be dominated by solving PDEs to get the gradient; also, there exist update rules for approximations of the &lt;em&gt;inverse&lt;/em&gt; Hessian, which only require computing a single matrix-vector product) and b) you still need a line search to guarantee convergence...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Luckily, in this context there exists an alternative approach that makes use of every function evaluation. The idea is that for $H_k$ symmetric and positive definite (which is guaranteed for the BFGS update), solving \eqref{cc1} is equivalent to minimizing the quadratic model&#xA;$$q_k(s) = \frac12 s^T H_k s + s^T g^k.$$&#xA;In a &lt;strong&gt;trust region method&lt;/strong&gt;, you would do so with the additional constraint that $\|s\| \leq \Delta_k$, where $\Delta_k$ is an appropriately chosen trust region radius (which plays the role of the step length $\sigma_k$). The key idea is now to choose this radius adaptively, based on the computed step. Specifically, you look at the ratio&#xA;$$ \rho_k := \frac{f(x^k)-f(x^k+s^k)}{f(x^k)-q_k(s^k)}$$&#xA;of the actual and predicted reduction in function value. If $\rho_k$ is very small, your model was bad, and you discard $s^k$ and try again with $\Delta_{k+1}&amp;lt;\Delta_k$. If $\rho_k$ is close to $1$, your model is good, and you set $x^{k+1}=x^k+s^k$ and increase $\Delta_{k+1}&amp;gt;\Delta_k$. Otherwise you just set $x^{k+1}=x^k+s^k$ and leave $\Delta_k$ alone. To compute the actual minimizer $s^k$ of $\min_{\|s\|\leq \Delta_k} q_k(s)$, there exist several strategies to avoid having to solve the full constrained optimization problem; my favorite is &lt;a href=&quot;http://www.ii.uib.no/~trond/publications/papers/trust.pdf&quot; rel=&quot;nofollow&quot;&gt;Steihaug's truncated CG method&lt;/a&gt;. For more details, I again refer to Nocedal and Wright.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-10-17T19:10:02.810" LastActivityDate="2016-10-17T19:10:02.810" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24467" PostTypeId="2" ParentId="24431" CreationDate="2016-07-16T14:11:44.040" Score="1" Body="&lt;p&gt;Alright I did some fixing and debugging. Quite a few mistakes I would say, I think you're a beginner in C/Programming ? Anyway find the code below...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;&#xA;#define N 3&#xA;#define M 3&#xA;#define A 0.0&#xA;#define B 1.0&#xA;#define C 0.0&#xA;#define D 1.0&#xA;&#xA;&#xA;int main(int argc, char *argv[]){&#xA;&#xA;double h_x = (double) 1/(N+1);&#xA;double h_y = (double) 1/(M+1);&#xA;double X[N+2];&#xA;double Y[M+2];&#xA;double xhalf[N+1];&#xA;double yhalf[M+1];&#xA;int i,j;&#xA;double o,k,l;&#xA;&#xA;X[0] = C; X[N+1] = D;&#xA;Y[0] = C; Y[M+1] = D;&#xA;xhalf[0] = h_x/2;&#xA;yhalf[0] = h_y/2;&#xA;&#xA;// before&#xA;printf(&quot;\nhx= %f&quot;, h_x); printf(&quot;\th_y= %f\n&quot;, h_y);&#xA;printf(&quot;\nX[0]= %f&quot;, X[0]); printf(&quot;\tX[N+1]= %f&quot;, X[N+1]);&#xA;printf(&quot;\nY[0]= %f&quot;, Y[0]); printf(&quot;\tY[M+1]= %f\n\n&quot;, Y[M+1]);&#xA;&#xA;for(i=1; i&amp;lt;=N; i++){&#xA;    X[i]= X[i-1]+h_x;&#xA;    xhalf[i]=xhalf[i-1]+h_x;&#xA;    printf(&quot;X[%d] = %e\n&quot;, i, X[i]);&#xA;}&#xA;&#xA;for(i=1; i&amp;lt;=M; i++){&#xA;    Y[i]=Y[i-1]+h_y;&#xA;    yhalf[i]=yhalf[i-1]+h_y;&#xA;    printf(&quot;Y[%d] = %e\n&quot;, i, Y[i]);&#xA;}&#xA;&#xA;// after&#xA;printf(&quot;\nhx= %f&quot;, h_x); printf(&quot;\th_y= %f\n&quot;, h_y);&#xA;printf(&quot;\nX[0]= %f&quot;, X[0]); printf(&quot;\tX[N+1]= %f&quot;, X[N+1]);&#xA;printf(&quot;\nY[0]= %f&quot;, Y[0]); printf(&quot;\tY[M+1]= %f\n\n&quot;, Y[M+1]);&#xA;&#xA;return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First, when you insert a code for us to help you out, please copy-paste it fully working (compilable) -- include libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, please paste a working version that actually works and so we dont have to work our way from scratch. In this case why was the opening braces for the main function missing ??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, a good practice is not to leave the main function empty like you did, check my code...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fourth, you were displaying the indexing of your arrays starting from 1, i.e. you skipped the first value and that is why you did not see a zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fifth, you initialize your arrays as (N+2) and (M+2) for X and Y - respectively. However in C you do not access the (N+2)th and (M+2)th index. This is why after compiling and during execution, we got a &quot;stack smashing detected&quot; error. In other words, for example, you initialize: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;double array[5];&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can only use array[0] up until array[4]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anything besides that, keep up the good work ! We've all been there :)   &lt;/p&gt;&#xA;" OwnerUserId="18387" LastActivityDate="2016-07-16T14:11:44.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24469" PostTypeId="1" CreationDate="2016-07-17T10:09:46.227" Score="1" ViewCount="191" Body="&lt;p&gt;I have a quick question regarding Rigorous Coupled Wave Analysis (RCWA) for numerically solving problems (in optics). I am trying to simulate a device which is non-periodic in x, y directions (but is longitudinally homogenous in z-direction). Is this possible? If so, would you be able to point me toward some readings or materials (haven't been able to find any)? Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="21029" LastActivityDate="2016-07-17T10:09:46.227" Title="RCWA for Non-Periodic Structure?" Tags="|numerical-analysis|simulation|computational-physics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24470" PostTypeId="1" AcceptedAnswerId="24471" CreationDate="2016-07-17T11:04:53.330" Score="3" ViewCount="2469" Body="&lt;p&gt;I have some code that does exactly this, but I do not like to use things I do not understand. Here is the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N1=N+1;&#xA;cheb=cos(pi*(0:N)/N)';&#xA;unif=linspace(-1,1,N1)';&#xA;if N&amp;lt;3&#xA;    x=cheb;&#xA;else&#xA;    x=cheb+sin(pi*unif)./(4*N);&#xA;end&#xA;P=zeros(N1,N1);&#xA;%N1xN1 zero matrix&#xA;xold=2;&#xA;%eps= epsilon!&#xA;while max(abs(x-xold))&amp;gt;eps&#xA;&#xA;xold=x;   &#xA;P(:,1)=1;&#xA;P(:,2)=x;&#xA;%set first collumn entries to 1 (P(:,1) = 1);&#xA;%set second collumn entries to x (P(:,2) = x);&#xA;for k=2:N&#xA;    P(:,k+1)=( (2*k-1)*x.*P(:,k)-(k-1)*P(:,k-1) )/k;&#xA;    %%Bonnets formula ;)&#xA;end&#xA;x=xold-( x.*P(:,N1)-P(:,N) )./( N1*P(:,N1));&#xA;end&#xA;&#xA;%---chebyshev differentiation matrix----------------------------------------    ---------&#xA;x=-x;&#xA;X=repmat(x,1,N1);%set every collumn of X to x&#xA;Xdiff=X-X'+eye(N1);&#xA;L=repmat(P(:,N1),1,N1);&#xA;L(1:(N1+1):N1*N1)=1;&#xA;D=(L./(Xdiff.*L'));&#xA;D(1:(N1+1):N1*N1)=0;&#xA;D(1)=-(N1*N)/4;&#xA;D(N1*N1)=(N1*N)/4;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I really want  understand, theoretically, how this matrix is formed and then more importantly, implemented in MATLAB.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up until the comment &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% --- chebyshev differenti....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can not seem to find any documentation relating to the formation of this matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: The syntax I am ok with, it is the actual formation of the matrix I am having trouble with. I have posted this on MSE and it had been put on hold. I had a look on the help centre and this site seemed to fit the description.  &lt;/p&gt;&#xA;" OwnerDisplayName="user21030" LastEditorUserId="1804" LastEditDate="2016-07-26T11:17:47.490" LastActivityDate="2016-07-26T11:17:47.490" Title="How do I form the Chebyshev differentiation matrix in MATLAB?" Tags="|matlab|numerical-analysis|spectral-method|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24471" PostTypeId="2" ParentId="24470" CreationDate="2016-07-17T11:41:37.330" Score="3" Body="&lt;p&gt;I'm assuming that you know how Chebyshev collocation methods work (but if not, let me know and I'll explain a bit more); a good introduction is Nick Trefethen's &lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/spectral.html&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Spectral Methods in Matlab&lt;/em&gt;&lt;/a&gt; as well as his &lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/ATAP/&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;Approximation Theory and Approximation Practice&lt;/em&gt;&lt;/a&gt; (in particular Chapter 21). &#xA;(But note that this code does &lt;em&gt;Legendre&lt;/em&gt; collocation, not Chebyshev collocation!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Lagrange form of the interpolation polynomial, you can derive an explicit form of the differentiation matrix $D$:&#xA;$$ D_{ij} = \ell'_j(x_i) = &#xA;\begin{cases} &#xA;\frac{\lambda_j}{\lambda_i(x_i-x_j)} &amp;amp; \text{for }i\neq j,\\&#xA;\frac{x_j}{1-x_j^2} &amp;amp; \text{for }i= j,&#xA;\end{cases}$$&#xA;where $\ell_j$ are the Langrange nodal polynomials (i.e. $\ell_j(x_k) = \delta_{jk}$) and $\lambda_j$ are the (reciprocal) denominators in the Langrage polynomial $\ell_j$. This is exactly what the final code block computes (in a particularly compact form):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;Xdiff=X-X'+eye(N1)&lt;/code&gt; computes all possible differences of $x_i$ and $x_j$ and adds one for $i=j$ (i.e., on the diagonal)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;L=repmat(P(:,N1),1,N1)&lt;/code&gt; computes the denominators of $\lambda_j$ from the Chebyshev Vandermonde matrix&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;L(1:(N1+1):N1*N1)=1&lt;/code&gt; replaces the diagonal entries with one (since there's no $\lambda_j$ for $i=j$)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;D=(L./(Xdiff.*L'))&lt;/code&gt; now computes all entries $D_ij$ simultaneously (it should be obvious for $i\neq j$, the special case $i=j$ is taken care of by the above modifications)&lt;/li&gt;&#xA;&lt;li&gt;The remaining lines modify &lt;code&gt;D&lt;/code&gt; to account for boundary conditions (the code computes only the matrix corresponding to the interior points).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(For reference, the original code comes from &lt;a href=&quot;https://mathworks.com/matlabcentral/fileexchange/5015-legendre-collocation-differentiation&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-07-17T12:21:31.687" LastActivityDate="2016-07-17T12:21:31.687" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24473" PostTypeId="2" ParentId="24422" CreationDate="2016-07-17T16:15:34.797" Score="1" Body="&lt;p&gt;As written, it's not clear to me that it's actually fully coupled. That is, the solution to $B$ depends (via its BC2) on the solution to $a$, but assuming $\omega$ is simply some specified function of position and time, it looks like $a$ doesn't depend on the solution to $B$. If that's the case, then simply solve for $a$ on a 1D mesh first, then you have the (constant in time) BC for $B$ and you can solve it as though there were no coupling. This can be done using a number of techniques for applying BC's within the FD/FV method, but it didn't appear that this was what you were asking. If $\omega$ is a function of $B$, then the system would be fully coupled. I'll assume that they're fully coupled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're confident that the diffusion in PDE2 is not going to ever have a component in the $x$ direction, and you think FD/FV is good for your problem (I so no reason why they wouldn't be), I would consider casting this as two separate, coupled 1D grids to avoid needlessly calculating zero-fluxes in the $x$ direction in the diffusing region. The first grid would be 1D in the $x$ direction, over which PDE1 can be defined. Then, you could repeat a 1D $y$ direction grid at each discretization point in $x$, and define PDE2 over each one of those. Then it's more like an equivalent 1+1D system. To solve that in a coupled system, as you suggested, you can use FD/FV to discretize in space, then step forward using the method of lines. Because $a$ doesn't appear with a time-derivative anywhere, the discretized problem will be a system of (index-1) differential algebraic equations (DAE's).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you've done the FD/FV, you would end up with an equation like this&#xA;$$0 = f_1(\omega, \{a_i\})$$&#xA;at each interior mesh point in the $x$ mesh, and&#xA;$$\frac{\partial B}{\partial t} = f_2(\{B_i\})$$&#xA;at each interior mesh point on the $y$ meshes where $f_1$ and $f_2$ are functions which depend on your discretization choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a number of ways to solve index-1 DAE's including (1) simple implicit Euler time stepping, (2) the idas module of the &lt;a href=&quot;http://computation.llnl.gov/projects/sundials-suite-nonlinear-differential-algebraic-equation-solvers&quot; rel=&quot;nofollow&quot;&gt;SUNDIALS&lt;/a&gt; suite, which has been wrapped for some other languages (Python, Julia...), or perhaps (3) Matlab's &lt;code&gt;ode15s&lt;/code&gt;, which also enables solution of index-1 DAE's by passing in a singular mass matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, as you said, the coupling would come through the boundary conditions, which you can implement a number of ways. One common approach is to use &quot;ghost points,&quot; or fictitious points added on each side of the mesh. So for example, for the right side of the $x$ mesh, say it has index $N_x$, you could add an additional equation for the fictitious point $N_{x}+1$,&#xA;$$a_{N_{x}+1} = a_{N_x}$$&#xA;and simply write the grid point equation for $a_{N_x}$ as you would for any interior point (typically as a function of $a_{N_{x}-1}, a_{N_x}, a_{N_{x}+1}$). The above is a case that's trivial enough that you could choose to substitute this algebraic equation into the discretization for point $N_x$ on the $x$ mesh, but when things get uglier, you can simply leave them as extra equations. For example, looking at a top point for the $B$ mesh(es) at $x$-index $x_i$, you could have the $y$-index point $N_{y}+1$ defined with the equation&#xA;$$\left(B_{x_i,N_{y}+1} - B_{x_i,N_y}\right)/\Delta y = g(\omega,a_{x_i})$$&#xA;where $\Delta y$ is the grid spacing in the $y$ direction. Then, again, if you can solve for $B_{x_i,N_y}$, you could substitute it into the $x_i, N_{y}$ equation, but if $\omega$ were some non-linear function of $B$, you could solve this algebraic equation along with the others in your DAE system. You can certainly change the discretization scheme to get higher order accuracy, but hopefully this demonstrates the idea.&lt;/p&gt;&#xA;" OwnerUserId="21008" LastActivityDate="2016-07-17T16:15:34.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24474" PostTypeId="2" ParentId="24462" CreationDate="2016-07-18T10:46:33.877" Score="1" Body="&lt;p&gt;Gradient descent (sometimes also called &lt;em&gt;steepest descent&lt;/em&gt;) for the minimization of a function $f:\mathbb{R}^n\to\mathbb{R}$ consists in picking $x^0\in\mathbb{R}^n$ and then setting for $k=0,\dots$&#xA;$$x^{k+1} = x^k - \sigma_k \nabla f(x^k),$$&#xA;where $\sigma_k&amp;gt;0$ is a suitable step length (e.g., chosen via Armijo line search) and $\nabla f(x)\in\mathbb{R}^n$ is the &lt;em&gt;gradient&lt;/em&gt; of $f$ evaluated in $x$, i.e., the vector&#xA;$$\nabla f(x) = (\partial_{x_1} f(x),\dots,\partial_{x_n} f(x))^T,$$&#xA;where $\partial_{x_i}f$ denotes the partial derivative of $f$ with respect to $x_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most black-box optimization routines such as Matlab's &lt;code&gt;fminunc&lt;/code&gt; or &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html&quot; rel=&quot;nofollow&quot;&gt;&lt;code&gt;scipy.optimize&lt;/code&gt;&lt;/a&gt; accept as input a function that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;takes as input the current point $x$ and&lt;/li&gt;&#xA;&lt;li&gt;outputs the value $f(x)$ (as a scalar) and the gradient $\nabla f(x)$ (as a vector of the same size and shape of $x$).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How&lt;/em&gt; you compute the value and the gradient is irrelevant for the optimizer (but, since this will be done often, you should try to do it as efficiently as possible). In many cases, you can calculate the derivatives by hand (or ask &lt;code&gt;sympy&lt;/code&gt;) to get an explicit formula. If you can't calculate the derivative, you need to use a finite difference approximation, but that can severely limit the performance of the gradient method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you're likely minimizing a loss function, so you have a bunch of pairs $(x_j,y_j)$ and wish to minimize&#xA;$$f(\theta) = \sum_{j} (\theta_1 x_{j,1} + \theta_2 x_{j,2}^{\theta_3}-y_j)^2$$&#xA;over $\theta=(\theta_1,\theta_2,\theta_3)^T\in\mathbb{R}^3$. &#xA;Here, the gradient is given by&#xA;$$\nabla f(\theta) = &#xA;\begin{pmatrix} &#xA;\sum_{j}2x_{j,1} (\theta_1 x_{j,1} + \theta_2 x_{j,2}^{\theta_3}-y_j)\\&#xA;\sum_{j}2x_{j,2}^{\theta_3}(\theta_1 x_{j,1} + \theta_2 x_{j,2}^{\theta_3}-y_j)\\&#xA;\sum_j 2\theta_2 x_{j,2}^{\theta_3}\log x_{j,2}(\theta_1 x_{j,1} + \theta_2 x_{j,2}^{\theta_3}-y_j)&#xA;\end{pmatrix} \in \mathbb{R}^3.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-07-18T14:32:50.133" LastActivityDate="2016-07-18T14:32:50.133" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24476" PostTypeId="1" AcceptedAnswerId="24495" CreationDate="2016-07-18T13:04:44.937" Score="3" ViewCount="1677" Body="&lt;p&gt;I have a large matrix, side-length is about $n\geq 1000$. I need to do element-wise multiplication of this matrix with another matrix many, many times. I make this process by:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://arma.sourceforge.net/docs.html#vectorise&quot; rel=&quot;nofollow&quot;&gt;Vectorizing&lt;/a&gt; (through armadillo) this matrix, i.e., flattening it in a single array.&lt;/li&gt;&#xA;&lt;li&gt;Splitting this array over many cores. Each core will take a part of this matrix and do its multiplication.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now I realized that for Mathematical reasons, I only need the upper-triangular part of this matrix. &lt;strong&gt;How can I vectorize only the upper-triangular part and discard the other components of the matrix?&lt;/strong&gt; Simply looping is not that good. Do I have any other options?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My matrices are Armadillo complex matrices of &lt;code&gt;arma::cx_mat&lt;/code&gt;. And my current vectorization function is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;typedef double Real;&#xA;typedef std::vector&amp;lt;std::complex&amp;lt;Real&amp;gt; &amp;gt; VecType;&#xA;arma::cx_mat myMatrix;&#xA;//fill my matrix...&#xA;VecType  vectorizedMatrix = arma::conv_to&amp;lt;VecType&amp;gt;::from(myMatrix);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I find &lt;code&gt;std::vector&lt;/code&gt; very useful, because I do element-wise multiplication very efficiently using standard algorithms with &lt;code&gt;std::transform()&lt;/code&gt;, which is best optimized (AFAIK).&lt;/p&gt;&#xA;" OwnerUserId="5410" LastEditorUserId="5410" LastEditDate="2016-07-18T14:50:31.100" LastActivityDate="2016-08-19T18:12:42.560" Title="Vectorize a part of a huge matrix in C++" Tags="|c++|matrix|vectorization|armadillo|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24477" PostTypeId="1" CreationDate="2016-07-18T19:02:30.847" Score="7" ViewCount="215" Body="&lt;p&gt;My problem at hand pertains to choosing a distance measure for use in locally weighted regression. In my particular problem, I have a data set that is upwards of 10 dimensions, where the variables have different units (think distances, speeds, angles).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I am formulating the problem is along the following lines:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, given some matrix $M$, we will compute a local Least Square solution, given some local set of data about $\textbf{x}$ defined as $\lbrace \left(\textbf{x}_i, y_i\right)\rbrace_{i=1}^N$, using the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \beta^*(\textbf{x},M) = \text{arg}\min_{\beta} \frac{1}{2N} \sum_{i=1}^N \text{d}(\textbf{x}_i,\textbf{x},M) \left(f(\textbf{x}_i,&#xA;\beta) - y_i\right)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\text{d}(\textbf{u},\textbf{v},M) &amp;amp;= \sqrt{(\textbf{u}-\textbf{v})^TM^TM(\textbf{u}-\textbf{v})}\\&#xA;f(\textbf{x},M) &amp;amp;= \sum_{j=1}^p\beta_j(\textbf{x},M)\phi_j(\textbf{x})&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, we wish to find $M$ such that this solution is minimized at a set of separate training points, $\lbrace \left(\hat{\textbf{x}}_i,\hat{y}_i \right) \rbrace_{i=1}^P$, defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;M^{*} &amp;amp;= \arg \min_{M} \frac{1}{2P} \sum_{i=1}^P \left(f(\hat{\textbf{x}}_i,&#xA;\beta^*(\hat{\textbf{x}}_i,M)) - \hat{y}_i\right)^2&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\text{d}(\cdot,\cdot,\cdot)$ is the distance measure for the data set based on some weighting matrix $M$, $\mathbf{\beta}_j(\textbf{x},M)$ is the $j^{th}$ term in the local least square coefficient solution based on some weighting matrix $M$, and $\phi_j$ is the $j^{th}$ basis function used to represent the unknown function locally. The whole point of this is to construct a distance measure that can be used with a whole dataset $D$ and allow for an efficient and accurate non-parametric representation. Note as well that this distance measure would come into play using a KD-Tree for finding the local dataset used in the local Least Square solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on this formulation, the optimization problem for $M^*$ appears non-convex, so I would like to know if there's any other approaches that might be better? I haven't had much success thus far finding any information, though I might not know the right thing to search for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to note that I actually have tried scaling each term of the vector, based on the variance in its possible values, and this did not really work robustly enough. I should also note for the angular measurements, I actually find the smallest signed angle difference in the $(\textbf{u}-\textbf{v})$ operation (ie $359^o-6^o=-7^o)$.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="25160" LastEditDate="2017-12-31T09:47:14.573" LastActivityDate="2018-07-29T13:01:45.613" Title="Methods to Estimate Optimal Distance Measure for Multidimensional Data Set" Tags="|optimization|regression|distance-measure|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24479" PostTypeId="1" AcceptedAnswerId="24487" CreationDate="2016-07-19T02:10:41.530" Score="9" ViewCount="3308" Body="&lt;p&gt;Is it pointless to use gradient based optimization algorithms if you can only provide a numerical gradient? If not, why provide a numerical gradient in the first place if it is trivial to perform finite differentiation for the optimization library itself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT]&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Just to clarify, my question indeed is in a more general sense than a specific application. Although my field of application happens to be likelihood optimization under various statistical frameworks.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;My issue with automatic differentiation is that there always seems to be a catch. Either the AD library can't propagate to external library calls (like BLAS) or you have to rework your workflow so drastically that it makes it a pain to deal with... especially if you're working with type sensitive languages. My gripes with AD are a separate issue altogether. But I want to believe!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I guess I need to better formulate my question but I'm doing a poor job of it. If have an option to either use a derivative-free optimization algorithm or a derivative based optimization algorithm with the caveat that I can only give it a numerical gradient, which one on average will be superior? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21051" LastEditorUserId="21051" LastEditDate="2016-07-19T22:25:38.623" LastActivityDate="2016-07-20T19:05:10.670" Title="Is providing approximate gradients to a gradient based optimizer useless?" Tags="|optimization|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24480" PostTypeId="2" ParentId="24479" CreationDate="2016-07-19T03:57:28.180" Score="17" Body="&lt;p&gt;If your objective is smooth, then using finite difference approximations to the derivative is often more effective than using a derivative free optimization algorithm.  If you have code that computes the derivatives exactly then it is normally best to use that code rather than to use finite difference approximations.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although some optimization libraries will compute finite difference approximations for you automatically using heuristics to determine the step size parameters, it can be better to use your own routines to compute the finite difference approximations either because you have better knowledge of the appropriate step sizes or because of special structure in the function that your code can exploit.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option that is often worth while is to use automatic differentiation techniques to produce a subroutine that computes the analytical derivatives from the source code for computing the objective function itself.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-07-19T03:57:28.180" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24481" PostTypeId="1" CreationDate="2016-07-19T04:16:18.550" Score="2" ViewCount="2245" Body="&lt;p&gt;I'm supposed to integrate differential equations for $r$ and $\theta$ in order to simulate orbital motion. The differential equation I used for $r$ is second order and $\theta$ is first order. The end goal is to graph $r(t)$, $\theta(t)$, which I'll probably do with matplotlib in python. I was wondering if anyone with prior knowledge of orbits and Runge-Kutta could let me know where I went wrong with my code? It compiles without error but isn't returning values that I expected. This is my code with parameters describing Earth's orbit: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt; &#xA;#define N 3         /* number of first order equations */&#xA;#define dist 0.00001        /* stepsize in t*/&#xA;#define MAX 1       /* max for t */&#xA;#define MAXTHETA 2*M_PI&#xA;&#xA;FILE *output;           /* internal filename */&#xA;&#xA;void runge4(double x, double y[], double step);&#xA;double f(double x, double y[], int i);&#xA;&#xA;&#xA;int main()&#xA;{&#xA;double t, y[N];&#xA;int j;&#xA;&#xA;void runge4(double x, double y[], double step); /* Runge-Kutta function */&#xA;&#xA;double f(double x, double y[], int i);      /* function for derivatives */&#xA;&#xA;output=fopen(&quot;kepler.rtf&quot;, &quot;w&quot;);                   /* external filename */&#xA;&#xA;y[0]= 1.02343552273569;                                       /* initial position */&#xA;y[1]= 6.18068911;                                       /* initial velocity */&#xA;y[2] = 0;                                               /* initial angle */&#xA;&#xA;fprintf(output, &quot;0\t%f\t%f\n&quot;, y[0], y[2]); /* prints time and position */&#xA;&#xA;for (j=1; j*dist&amp;lt;=MAX ;j++)         /* time loop */&#xA;{&#xA;    t=j*dist;&#xA;    runge4(t, y, dist);&#xA;&#xA;    fprintf(output, &quot;%f\t%f\t%f\n&quot;, t, y[0], y[2]); /* prints time, position, theta */&#xA;}&#xA;&#xA;fclose(output);&#xA;}&#xA;&#xA;void runge4(double x, double y[], double step)&#xA;{&#xA;double h=step/2.0,          /* the midpoint */&#xA;    t1[N], t2[N], t3[N],        /* temporary storage arrays */&#xA;    k1[N], k2[N], k3[N],k4[N];  /* for Runge-Kutta */&#xA;int i;&#xA;&#xA;for (i=0;i&amp;lt;N;i++) t1[i]=y[i]+0.5*(k1[i]=step*f(x, y, i));&#xA;for (i=0;i&amp;lt;N;i++) t2[i]=y[i]+0.5*(k2[i]=step*f(x+h, t1, i));&#xA;for (i=0;i&amp;lt;N;i++) t3[i]=y[i]+    (k3[i]=step*f(x+h, t2, i));&#xA;for (i=0;i&amp;lt;N;i++) k4[i]=        step*f(x+step, t3, i);&#xA;&#xA;for (i=0;i&amp;lt;N;i++) y[i]+=(k1[i]+2*k2[i]+2*k3[i]+k4[i])/6.0;&#xA;}&#xA;&#xA;double f(double x, double y[], int i)&#xA;{&#xA;    double k = 39.4784176044, m = 0.000003003, h = 0.00000300299, l = 0.0000188781;&#xA;    if (i==0) return(y[1]);         /* derivative of first equation (dr/dt)*/&#xA;    if (i==1) return((-k/(m*y[0]*y[0])) + (l*l)/(m*y[0]*y[0]*y[0])); /* derivative of second equation (d^2r/dt^2) */&#xA;    else return(h/(y[0]*y[0])); /* derivative of theta (d(theta)/dt) */&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The second order differential equation I used for the position, $r$, is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot{r} = \frac{-k}{mr^2} + \frac{l^2}{mr^3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the parameter $l$ is the angular momentum and $k$ is the gravitational constant multiplied my the mass of the object that the planet is orbiting, in this case the sun. The first order differential equation I used for $\theta$ is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot{\theta} = \frac{h}{r^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$h = l\left(\frac{m_1 m_2}{m_1 + m_2}\right)^{-1}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $m_1$ and $m_2$ are the masses of the two bodies in the system, and the mass term in the equation for $h$ is the reduced mass. All of the units of parameters are in AU, years, and solar masses. If anybody could point me in the right direction that would be awesome.&lt;/p&gt;&#xA;" OwnerUserId="21052" LastEditorUserId="9804" LastEditDate="2016-07-19T17:00:10.097" LastActivityDate="2019-07-08T22:05:44.113" Title="4th Order Runge Kutta: Integration of Differential Equations for Planetary Orbit" Tags="|numerical-analysis|runge-kutta|integration|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24482" PostTypeId="1" CreationDate="2016-07-19T05:35:45.360" Score="2" ViewCount="60" Body="&lt;p&gt;In 2D, the &lt;em&gt;hat functions&lt;/em&gt; are the usual function basis for the finite dimentional space $\{v\in \mathcal{C}:v|_T\in P_1(T)\,\forall T\}$: set of continuously functions that are polynomials less than or equal to one over each element (triangle) $T$ of the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, which are the function bases for $\{v:v|_e\in P_k(e)\,\forall e\}$? (space of functions that are polynomials less than or equal to one on edge $e$ of the mesh).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, &lt;strong&gt;how can I define a base of polynomials less than or equal to one over an single edge $e$ of the mesh?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="17630" LastActivityDate="2016-07-19T05:35:45.360" Title="Basis over edges of a mesh" Tags="|finite-element|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24483" PostTypeId="1" AcceptedAnswerId="24484" CreationDate="2016-07-19T16:28:03.750" Score="2" ViewCount="41" Body="&lt;p&gt;I am trying to find points of a surface whose normal is parallel with the 3 dimensional axes (i.e. local maxima, minima and saddle points in x, y and z dimensions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function of the surface is defined as $f(u,v)$ which is continuous and gives a 3D vector indicating the position of the surface for a given set of parameters u and v.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the points I require I understand that I need to find the derivatives of the surface with respect to u and v. The relevant points will occur when:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\delta f_i(u,v)}{\delta u}=0$ &amp;amp; $\frac{\delta f_i(u,v)}{\delta v}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $i$ is the respective dimension (x,y or z) and $0\leq u\leq1$ &amp;amp; $0\leq v\leq1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of strategy should I take to solving this problem? I was thinking it may be possible to use a numerical method with the partial derivatives to iteratively find solutions. But then how would I remove the solution as you would in univariate root finding?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any insights would be really useful.&lt;/p&gt;&#xA;" OwnerUserId="7837" LastActivityDate="2016-07-19T21:13:47.817" Title="Finding common roots of two multivariate equations" Tags="|numerical-analysis|roots|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24484" PostTypeId="2" ParentId="24483" CreationDate="2016-07-19T21:13:47.817" Score="1" Body="&lt;p&gt;The derivatives $\partial_u f(u,v)$ and $\partial_v f(u,v)$ are two vectors tangent to yor surface that in general span the tangent space to the surface at the point $f(u,v)$. Then the normal vector to the surface at that point is $n(u,v)=\partial_u f(u, v)\times\partial_v f(u,v)$ and you want it parallel to say the coordinate direction $e_z=(0,0,1)^T$. Then yo want to solve the equation $n(u,v)\times e_z=(\partial_u f(u, v)\times\partial_v f(u,v))\times e_z=0.$ After applying a simplification formula for a double cross-product like this one, you get equations like the ones you have written. To solve the system of equations numerically maybe you can simply use Newton's method or something of that kind. &lt;/p&gt;&#xA;" OwnerUserId="20964" LastActivityDate="2016-07-19T21:13:47.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24485" PostTypeId="1" AcceptedAnswerId="24488" CreationDate="2016-07-19T21:53:12.857" Score="5" ViewCount="1038" Body="&lt;p&gt;I know of at least two ways of dealing with Dirichlet boundary conditions in finite-difference schemes (and, to a lesser extent, finite-element schemes). Here I'm thinking of solving Poisson's equation $\Delta u = f$, but the question is more general.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Treat the function values at boundary nodes as further unknown variables $u_i$, but add equations of the form $u_i = g(x_i)$ to the resulting system of equations. In total we'll have $n_\text{inner} + n_\text{boundary}$ equations and unknowns, where $n_\text{inner}$ is the number of nodes in the interior of the domain and $n_\text{boundary}$ is the number of nodes on the boundary.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't include variables for the fixed function values at boundary nodes. In total we'll have $n_\text{inner}$ equations and unknowns.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The two approaches have the following advantages and disadvantages:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Setting up the equations in the second approach is slightly more complicated, because nodes which are neighbours of boundary nodes have to be treated slightly differently. This seems to be the main selling point of the first approach.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With the first approach, the resulting system matrix might not be symmetric. (Symmetry is very nice for iterative solvers.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The second approach is closer to the theoretical treatment of the undiscretized case: With zero Dirichlet conditions, we look for a solution in the Sobolev space $H_0^1(\Omega)$ instead of $H^1(\Omega)$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As soon as more than one differential operator is involved, for instance in the Stokes equation, the first approach gets more messy. One has to introduce zero rows in some of the resulting matrices.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The question:&lt;/strong&gt; Which of the two approaches is the current recommended best practice? Are there further advantages or disadvantages which I've overlooked?&lt;/p&gt;&#xA;" OwnerUserId="21062" LastEditorUserId="21062" LastEditDate="2016-07-21T12:58:02.250" LastActivityDate="2016-07-21T12:58:02.250" Title="Best practice for dealing with Dirichlet boundary conditions in finite-difference schemes: add artificial unknowns?" Tags="|finite-difference|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24486" PostTypeId="1" CreationDate="2016-07-20T03:20:45.340" Score="2" ViewCount="118" Body="&lt;p&gt;Suppose $A\in\mathbb{R}^{n\times c}$,$u\in\mathbb{R}^n$,$n\gg c$. &#xA;The time complexity of eigenvalue decomposing directly for matrix $AA^T+\text{diag}(u)$ is $O(n^3)$. And it is easy to avoid $O(n^3)$ for matrix $AA^T+I$. So can we avoid $O(n^3)$ for matrix $AA^T+\text{diag}(u)$? Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jay.&lt;/p&gt;&#xA;" OwnerUserId="21065" LastActivityDate="2016-07-20T03:20:45.340" Title="Eigenvalue decomposition of the sum: $AA^T$ + diag($u$)" Tags="|linear-algebra|eigenvalues|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24487" PostTypeId="2" ParentId="24479" CreationDate="2016-07-20T08:42:50.707" Score="11" Body="&lt;p&gt;To complement Brian's excellent answer, let me give a bit of (editorial) background. Derivative-free optimization methods are defined as methods that only make use of function evaluations, and are basically all variations of &quot;sample the admissible set more or less systematically and save the best function value&quot; -- that's all you &lt;em&gt;can&lt;/em&gt; do given the information. These methods can be roughly subdivided in&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Stochastic methods&lt;/em&gt;, where the selection of samples is fundamentally random (meaning that randomness is a crucial component; there can be other, deterministic components). These methods are often motivated by physical or biological processes and have corresponding names like &quot;simulated annealing&quot;, &quot;genetic algorithms&quot;, or &quot;particle swarm/firefly/anthill method&quot;. There is rarely any convergence theory beyond &quot;if you try long enough, you will hit all points (including the minimizer) with probability $1$&quot; (whether that will happen -- with any probability -- before the heat death of the universe is another matter...) As a mathematician, I would consider these methods as a last resort: If you don't know &lt;em&gt;anything&lt;/em&gt; about your function, this is all you can do, and you might get lucky.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Deterministic methods&lt;/em&gt;, where the selection of samples is not random, i.e., based purely on previous function evaluations. The most famous example is probably the Nelder--Mead simplex method; others are &lt;a href=&quot;http://epubs.siam.org/doi/10.1137/S003614450242889&quot;&gt;generating set search&lt;/a&gt; methods. &lt;em&gt;It is important to realize that this can only work if there is any (exploitable) relation between the value of the function at different points -- i.e., some smoothness of the function.&lt;/em&gt; In fact, the convergence theory for, e.g., the Nelder--Mead method is based on constructing a &lt;em&gt;non-uniform&lt;/em&gt; finite-difference approximation of the gradient based on the function values at the vertices of the simplex and showing that it convergences to both the exact gradient and zero as the simplex contracts to a point. (The variant based on a standard finite-difference approximation is called &lt;em&gt;compass search&lt;/em&gt;.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Model-based methods&lt;/em&gt;, where the function values are used to build a local model of the function (e.g., by interpolation), which is then minimized using standard (gradient-/Hessian-based) methods. Since a finite difference approximation is equivalent to the exact derivative of a polynomial interpolant, the classical &quot;numerical gradient&quot; approach falls into this class as well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As you can see, the boundaries between these classes are fluid, and often just a matter of interpretation. But the moral should be clear: &lt;strong&gt;Make sure you use all available information about the function you're minimizing.&lt;/strong&gt; To quote Cornelius Lanczos:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A lack of information cannot be remedied by any mathematical trickery.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;After all, if you don't know &lt;em&gt;anything&lt;/em&gt; about your function, it might as well be completely random, and minimizing a random value is a fool's errand...&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-07-20T11:28:48.610" LastActivityDate="2016-07-20T11:28:48.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24488" PostTypeId="2" ParentId="24485" CreationDate="2016-07-20T09:26:31.467" Score="3" Body="&lt;p&gt;I think the best approach depends on the solver that you are using. With a direct method, I would incorporate Dirichlet boundary conditions by modifying the right-hand side (option 2). The main advantage, as you have already listed, is that the resulting matrix is symmetric. This not only saves you storage, but also allows for (much) more efficient algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also other advantages. For example, suppose you want to incorporate an irregular boundary in your domain. Option 1 now becomes quite complicated, in particular when the boundary values depend on each other or on the solution -- for example with Neumann or mixed boundary conditions. Option 2 would be the way to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With an iterative method, it depends. For example, when using geometric multigrid, it often makes sense to use &lt;em&gt;ghost cells&lt;/em&gt; near your domain boundaries so that you can use the same smoother everywhere.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastEditorUserId="18662" LastEditDate="2016-07-20T12:49:29.957" LastActivityDate="2016-07-20T12:49:29.957" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24490" PostTypeId="2" ParentId="23995" CreationDate="2016-07-20T14:27:09.270" Score="2" Body="&lt;p&gt;So in case if you want to get pressure field there are two variants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) Simple one&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you have already solve the problem in terms of stream function ($\psi$), you get velocity field from the definition of stream function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_{x} = \partial \psi / \partial y,\quad v_{y} = -\partial \psi / \partial x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then using momentum equation you can obtain field of pressure gradient, in coordinates form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial p / \partial x = \eta \left(\partial^2 v_{x}/ \partial x^2 + \partial^2 v_{x}/ \partial y^2\right) + f_{x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial p / \partial y = \eta \left(\partial^2 v_{y}/ \partial x^2 + \partial^2 v_{y}/ \partial y^2\right) + f_{y}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you don't set any boundary conditions for pressure (e.g. pressure at infinity point), pressure field will not be unique. Pressure field will be unique with accuracy to some additive constant. Knowing pressure derivatives in each point of your mesh you can integrate this equations and get pressure field. Before integrating you should set pressure to some value in some particular point of mesh. &#xA;Integration could be done simply by formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p_{i+1, j} = p_{i, j} + g_{i, j} \Delta x_{i} \quad p_{i, j+1} = p_{i, j} + g_{i, j} \Delta y_{j}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) Another one&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, you can solve elliptic problem to find out the pressure field. As you mentioned, pressure equation is obtained after taking divergence of the momentum equation.&#xA;The Neumann boundary condition in case of your geometry is following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p_{x} = \eta \partial^2 v_{x} / \partial x^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on the $x = 0, x = L$ lines.&lt;/p&gt;&#xA;" OwnerUserId="21058" LastActivityDate="2016-07-20T14:27:09.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24491" PostTypeId="1" CreationDate="2016-07-20T16:03:23.720" Score="3" ViewCount="220" Body="&lt;p&gt;I am working on a project involving steady solutions for the Navier-Stokes Equations. In the past I've only worked with the unsteady Navier-Stokes, so some of this is new to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, at each iteration for some problems, one may need to resort to a relaxation parameter at each iteration, so letting $\tilde{\boldsymbol{u}}^n$ be the velocity field at an iteration $n$, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\boldsymbol{u}^n = \gamma \tilde{\boldsymbol{u}}^n + (1 - \gamma) \boldsymbol{u}^{n-1}&#xA;$$&#xA;For some value of $\gamma$ between zero and one. This seems to work alright for my purposes, but I have a couple of questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it necessary to relax both pressure and velocity? It seems for my purposes relaxing only velocity seems to be fine, and the pressure still converges to the desired physical solution. I do worry, however, that there are problems with consistency if the pressure is not also relaxed.&lt;/li&gt;&#xA;&lt;li&gt;If the pressure must also be relaxed, how does the pressure relxation parameter relate to the velocity relaxation parameter? Would they simply be the same value of $\gamma$? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Potentially relevant details:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I am using a finite element code with P2 for velocity and P1 for pressure.&lt;/li&gt;&#xA;&lt;li&gt;I am prescribing a Dirichlet inflow with a parabolic profile and a Neumann outflow.&lt;/li&gt;&#xA;&lt;li&gt;I am solving the coupled monolithic system (and not using the pressure Schur Complement approach).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance to anyone who has any answers for me!&lt;/p&gt;&#xA;" OwnerUserId="21074" LastEditorUserId="21074" LastEditDate="2016-07-20T19:01:08.093" LastActivityDate="2016-10-20T15:49:27.063" Title="Relaxation Parameters for Steady Navier-Stokes" Tags="|finite-element|pde|fluid-dynamics|nonlinear-equations|navier-stokes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24492" PostTypeId="2" ParentId="21952" CreationDate="2016-07-20T16:18:45.410" Score="1" Body="&lt;p&gt;You could try asking Herbie, a piece of software that was designed to find more accurate formulas&#xA;&lt;a href=&quot;http://herbie.uwplse.org/&quot; rel=&quot;nofollow&quot;&gt;http://herbie.uwplse.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21075" LastActivityDate="2016-07-20T16:18:45.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24493" PostTypeId="2" ParentId="20634" CreationDate="2016-07-20T16:45:17.940" Score="4" Body="&lt;p&gt;&lt;em&gt;Note: I originally posted an answer that I was not 100% pleased with, so I have revised it heavily. 1/20/2017&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The projection method is not in an exact approximation to the full system in general. There are a few reasons for this, but the most immediate has to do with the boundary conditions. Let's look at the continuous setting first. Consider the unsteady Navier-Stokes problem in a sufficiently regular domain $\Omega$ with appropriate boundary conditions, given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\begin{split}&#xA;\label{UnsNS}&#xA;\frac{\partial \boldsymbol{u}}{\partial t} -\nu\nabla \cdot (\nabla \boldsymbol{u} + \nabla \boldsymbol{u}^T) + ( \boldsymbol{u} \cdot \nabla) \boldsymbol{u} + \triangledown p = \boldsymbol{f} \quad \textit{in } \Omega \times \lbrack t_0, t_n \rbrack  \\&#xA;\nabla \cdot \boldsymbol{u} = 0 \quad \textit{in } \Omega \times \lbrack t_0, t_n \rbrack \\&#xA;\mathcal{B}\left( \boldsymbol{u}, p \right) = \boldsymbol{g} \quad \textit{on } \partial \Omega \times \lbrack t_0, t_n \rbrack  \\&#xA;\boldsymbol{u}(\boldsymbol{x},0) = \boldsymbol{u}_0 \quad \textit{at} \quad t=t_0 &#xA;\end{split}&#xA;\end{align}&#xA;Where $\mathcal{B}$ is an appropriate trace operator describing the boundary conditions. For simplicity, let's assume that $\boldsymbol{u}=0$ on $\partial \Omega$.&#xA; At a step $n$ the standard Chorin-Temam projection method consists of three steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We introduce an intermediate velocity field $\tilde{\boldsymbol{u}}^n$ and separate the velocity and pressure computations in two parts. We compute $\tilde{\boldsymbol{u}}^n$ as follows:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\frac{1}{\Delta t} (\tilde{\boldsymbol{u}}^n - \boldsymbol{u}^{n-1} ) - \nu \Delta \tilde{\boldsymbol{u}}^n + (\tilde{\boldsymbol{u}}^n \cdot \nabla ) \tilde{\boldsymbol{u}}^n = \boldsymbol{f}^n  \\&#xA;\tilde{\boldsymbol{u}}^n =0 \, \, \text{on} \, \, \partial \Omega&#xA;\end{align*}&#xA;Note we enforce boundary conditions here.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Next, we use the intermediate velocity field to compute the pressure: $$\frac{1}{\Delta t} (\boldsymbol{u}^{n} - \tilde{\boldsymbol{u}}^{n} ) = -\nabla p^n$$ &#xA;By exploiting the fact that $\nabla \cdot \boldsymbol{u}^n = 0$, we can apply the divergence operator to both sides and formally eliminate $\boldsymbol{u}^n$. We now have a Poisson problem for the Pressure:&#xA;$$ -\Delta t \Delta p^{n} = - \nabla \cdot \tilde{\boldsymbol{u}}^n $$ &lt;/li&gt;&#xA;&lt;li&gt;Apply the Helmholtz Principle to get the final velocity $\boldsymbol{u}^n$: $$\boldsymbol{u}^n = \tilde{\boldsymbol{u}}^n - \Delta t \nabla p^n$$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are a couple of things to notice here. The first is that we only have a second order operator for the velocity in step (1), and accordingly we can only control the trace operator (and hence the boundary conditions) during this step. The next thing to notice is that during step (2), we have a second-order operator for the pressure. This implicitly introduces a boundary condition on the pressure; in this case we have imposed homogenous Neumann boundary conditions for pressure. Such boundary conditions may not be accurate for the true pressure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because our final velocity computed in step (3) relies on step (2), our boundary conditions are no longer guaranteed to be satisfied (as we enforced them during the first step). Some people have suggested that because of this the intermediate velocity field $\tilde{\boldsymbol{u}}^n$ should be regarded as the 'true' velocity field; however this is equally problematic because the intermediate field is not divergence free in general. So we have two choices: a velocity field that respects incompressibility but not the boundary conditions, and a velocity field that respects the boundary conditions but not incompressibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in the numerical setting there are other reasons why this solution is not &quot;exact&quot;; in particular the &lt;em&gt;splitting error&lt;/em&gt; induced by separating the velocity and pressure by means of the intermediate velocity field is nonzero. You can observe this easily by looking at the problem in the algebraic setting. Our standard N-S matrix is given by: \begin{align}&#xA;\begin{split}&#xA;\label{UnsAlgProb}&#xA; \begin{bmatrix} &#xA;		     A &amp;amp; B^{T} \\&#xA;		    B &amp;amp; 0  \end{bmatrix} \begin{bmatrix} \boldsymbol{u}^{n} \\ \boldsymbol{p}^{n} \end{bmatrix}&#xA;		=&#xA;		 \begin{bmatrix} \boldsymbol{f} + \frac{1}{\Delta t} \boldsymbol{u}^{n-1}  \\ \boldsymbol{0} \end{bmatrix}&#xA;\end{split}&#xA;\end{align}&#xA;However, for Chorin-Temam we are solving the following block LU system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\begin{split}&#xA;\label{projA}&#xA; \begin{bmatrix} &#xA;		     A &amp;amp; 0 \\&#xA;		    B &amp;amp; -BB^T  \end{bmatrix}  \begin{bmatrix} &#xA;		     I &amp;amp; \Delta t B^T \\&#xA;		    0 &amp;amp; I  \end{bmatrix} \begin{bmatrix} \boldsymbol{u}^{n} \\ \boldsymbol{p}^{n} \end{bmatrix}&#xA;		=&#xA;		 \begin{bmatrix} \boldsymbol{f} + \frac{1}{\Delta t}  \boldsymbol{u}^{n-1}  \\ \boldsymbol{0} \end{bmatrix}&#xA;\end{split}&#xA;\end{align}&#xA;Or, similarly, reinterpreting the method as the &lt;em&gt;algebraic projection method&lt;/em&gt; (noting that the mass matrices $M$ correspond to the identity operator) we have a similar block LU system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\begin{split}&#xA;\label{algprojA}&#xA; \begin{bmatrix} &#xA;		     A &amp;amp; 0 \\&#xA;		    B &amp;amp; -BM^{-1}B^T  \end{bmatrix}  \begin{bmatrix} &#xA;		     I &amp;amp; \Delta t M^{-1}B^T \\&#xA;		    0 &amp;amp; I  \end{bmatrix} \begin{bmatrix} \boldsymbol{u}^{n} \\ \boldsymbol{p}^{n} \end{bmatrix}&#xA;		=&#xA;		 \begin{bmatrix} \boldsymbol{f} + \frac{1}{\Delta t}  M\boldsymbol{u}^{n-1}  \\ \boldsymbol{0} \end{bmatrix}&#xA;\end{split}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases, the difference between the 'exact' system and the projected one is nonzero. This can be verified easily by performing the matrix multiplication. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will also note that if one replaces the matrix $\Delta t M^{-1} B^{T}$ with $\Delta t A^{-1} B^{T}$ in the (1,2) block of the upper triangular matrix for the algebraic projection method, one can regain control of the boundary conditions (as $A$ corresponds to a second-order operator). This resolves our issue with boundary conditions, though the splitting error remains nonzero. &lt;/p&gt;&#xA;" OwnerUserId="21074" LastEditorUserId="21074" LastEditDate="2017-01-23T22:51:08.370" LastActivityDate="2017-01-23T22:51:08.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24494" PostTypeId="2" ParentId="23679" CreationDate="2016-07-20T16:53:57.317" Score="2" Body="&lt;p&gt;I found the solution to my problem : I was misunderstanding $R_k$ notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_k:$ the reference vector $r_k = (0, \psi_k)^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$R_k = \begin{bmatrix} r_k, \ldots, r_{k+N}\end{bmatrix}^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute $R_k$, I was replacing each value of $r_k$ and then, doing the transpose. But instead I have to do the transpose first, and then replace the values of $r_k$. Doing this, I get $R_k$ a (22x1) matrix and the computation works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some tests with Matlab, but didn't manage to really make it work and as I didn't have the time to continue to work on it, it's in stand by. Anyway, if someone reading this, managed or trying to make a similar algorithm work, I'm interested to talk about it (but this is out of the scope of my original question).&lt;/p&gt;&#xA;" OwnerUserId="19991" LastActivityDate="2016-07-20T16:53:57.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24495" PostTypeId="2" ParentId="24476" CreationDate="2016-07-20T18:10:43.500" Score="1" Body="&lt;p&gt;So I invented my own solution. Critique, suggestions are highly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I learned that elements are stored in a matrix as &lt;a href=&quot;http://arma.sourceforge.net/docs.html#Mat&quot; rel=&quot;nofollow&quot;&gt;column-major&lt;/a&gt;. So I assumed that using &lt;code&gt;std::copy()&lt;/code&gt; with an iterator over a single column will just give consecutive elements, hence better speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;typename T&amp;gt;&#xA;std::vector&amp;lt;T&amp;gt; vectorizeMatrix_upperTriangular_columnMajor(const arma::Mat&amp;lt;T&amp;gt;&amp;amp; mat)&#xA;{&#xA;    std::vector&amp;lt;T&amp;gt; output((mat.n_cols*(mat.n_cols-1))/2);&#xA;    typename arma::Mat&amp;lt;T&amp;gt;::const_iterator it = mat.begin() + mat.n_rows; //iterator at rows to skip in every step, starts at second column&#xA;    long toSkipInVec = 0;&#xA;    for(int i = 1; i &amp;lt; mat.n_cols; i++) //Starts with 1 to skip the diagonal&#xA;    {&#xA;        std::copy(it, it + i, output.begin() + toSkipInVec);&#xA;        toSkipInVec += i;&#xA;        it += mat.n_rows;&#xA;    }&#xA;    return output;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5410" LastActivityDate="2016-07-20T18:10:43.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24496" PostTypeId="2" ParentId="24423" CreationDate="2016-07-20T18:26:07.900" Score="1" Body="&lt;p&gt;While floating point rounding from async operations may be the issue, I suspect that it is something more banal.  The use of uninitialized variable that is adding randomness to your otherwise deterministic code.  It is a common issue that is often overlooked by developers because when you run in debug mode all variables are initialized to 0 on declaration.  When not running in debug mode, the memory assigned to a variable has whatever value the memory had before the assignment.  The memory is not zeroed on assignment as an optimization.  If this is happening in your code, it will be easy to fix, less so in the libraries code.&lt;/p&gt;&#xA;" OwnerUserId="3870" LastActivityDate="2016-07-20T18:26:07.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24497" PostTypeId="1" CreationDate="2016-07-20T18:42:01.187" Score="1" ViewCount="87" Body="&lt;p&gt;I'd like to solve the elastodynamics equation for a problem with fast scales from an impact (although no shocks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho\ddot{\mathbf{u}} - \nabla \cdot \sigma(\mathbf{u}) = \mathbf{f}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In structural dynamics, this equation is typically solved with the Newmark scheme with Lagrange elements for the spatial domain discretization. The Newmark scheme is implicit, but I believe given the fast scales, it will be cheaper to solve with an explicit scheme. To avoid inverting the mass matrix from the second-order derivative term, I have seen that a common technique is to lump the mass matrix into a diagonal matrix. What's the accuracy of this technique? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other methods involve using DG for the spatial discretization, which gives a block diagonal matrix that can be easily inverted, could this be cheaper and more accurate than the above approach? What would be the right way to approach this problem? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2016-07-20T18:42:01.187" Title="Time integration for elastodynamics" Tags="|time-integration|discontinuous-galerkin|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24498" PostTypeId="2" ParentId="24479" CreationDate="2016-07-20T19:05:10.670" Score="4" Body="&lt;p&gt;Your question asks about gradient-based optimizers, so I think Brian was right on.&#xA;I would only share, since I am currently struggling with that myself, some of the issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problems with finite difference are 1) performance, because you have to re-evaluate the function again for each dimension, and 2) it can be tricky to choose a good step size. If the step is too large, the assumption of linearity of the function may not hold. If the step is too small, it may run into the noise in the function itself, because derivatives amplify noise. The latter can be a real problem if the function involves solving differential equations.&#xA;If it is possible to calculate the gradients analytically, or using sensitivity equations, it will certainly be more accurate and maybe faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's another approach that you can try if you haven't invested too much time already in the software, and that is to run it with complex arithmetic.&#xA;It's called &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_differentiation#Complex_variable_methods&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;complex step differentiation&lt;/em&gt;&lt;/a&gt;.&#xA;The basic idea is when you evaluate the function, if you want its gradient with respect to parameter X, you set the imaginary part of X to a very small number &lt;em&gt;eps&lt;/em&gt;.&#xA;After you do the calculation, the imaginary part of the function's value, divided by &lt;em&gt;eps&lt;/em&gt;, is the gradient with respect to X.&#xA;When you want the gradient with respect to Y, you have to do it all again, of course.&#xA;What's interesting about it is that &lt;em&gt;eps&lt;/em&gt; can be made very small.&#xA;The reason it works is that the normal rules of differential calculus are precisely mirrored in the rules of complex arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I regard it as &lt;em&gt;not&lt;/em&gt; a panacea, because it's not always easy to do a complicated function in complex arithmetic, it's not worth it if the gradient can be calculated analytically, and in the case of differential equations it's exactly equivalent to &lt;a href=&quot;http://computation.llnl.gov/projects/sundials-suite-nonlinear-differential-algebraic-equation-solvers/toms_cvodes_with_covers&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;sensitivity equations&lt;/em&gt;&lt;/a&gt;, which I'm doing as necessary.&lt;/p&gt;&#xA;" OwnerUserId="1262" LastActivityDate="2016-07-20T19:05:10.670" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24499" PostTypeId="1" CreationDate="2016-07-20T22:02:57.867" Score="1" ViewCount="62" Body="&lt;p&gt;In a particular implementation(Finite volume advection using upwind) of adaptive mesh refinement the error square estimate for a cell C is given as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sum_{i = x,y,z} vol * \frac{1}{12} * h^{2} * (\frac{ds}{di})^{2}&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $vol $ is the volume of the cell, $h $ is the size of the cell and ${ds}/{di} $ is the derivative of the solution $s $ in each of the direction $i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I still fail to understand that where is this error expression coming from and is the error expression specific for the scheme being implemented which is first order accurate in space or, can it be applied for any other scheme too. I tried to find the truncation error in space for upwind scheme but the expression doesn't matches the expression given above. What i also tried, is to use the same expression for error estimate for a second order accurate central differencing scheme for diffusion scheme( nomal 5 point stencil in 2D or 7 point stencil in 3D) and it seems to work for that though I face feeble issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other better ideas or pointers for error estimation to dynamically refine the mesh is also welcome.&lt;/p&gt;&#xA;" OwnerUserId="17615" LastActivityDate="2016-07-20T22:02:57.867" Title="Square error estimate for adaptive mesh refinement" Tags="|fluid-dynamics|finite-volume|error-estimation|adaptive-mesh-refinement|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24500" PostTypeId="2" ParentId="23541" CreationDate="2016-07-20T22:46:19.783" Score="2" Body="&lt;p&gt;You could try any iterative algorithm, such as gradient descent or L-BFGS.  To speed up solving each instance after the first, use the solution to the prior instance as the initial value for solving the next instance.  Here the heuristic is that if $p,Q$ has changed only slightly, then we hope the optimal solution will be affected by only a small amount.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2016-07-20T22:46:19.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24501" PostTypeId="1" CreationDate="2016-07-21T04:08:05.657" Score="2" ViewCount="959" Body="&lt;p&gt;I am wondering is there any robust, well-tested, accurate open source FEM solver package for Windows? I would like to use to power the engine of my structural engineering application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FEM package should come with&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Meshing algorithm&lt;/li&gt;&#xA;&lt;li&gt;Static solver&lt;/li&gt;&#xA;&lt;li&gt;Modal Analysis&lt;/li&gt;&#xA;&lt;li&gt;Everything that is standard for structural engineering application&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I don't need it to run on clusters, because my application should be able to run on personal laptops. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Code_Aster&quot; rel=&quot;nofollow&quot;&gt;Code_Aster&lt;/a&gt;, but it is not supported natively on Windows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can find &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_finite_element_software_packages&quot; rel=&quot;nofollow&quot;&gt;a list of such packages at wiki&lt;/a&gt;, but I am not sure which one is alive and under development, and which one is a dead-end project. &lt;/p&gt;&#xA;" OwnerUserId="690" LastEditorUserId="690" LastEditDate="2016-07-24T06:38:54.573" LastActivityDate="2016-09-15T13:48:05.500" Title="Open source FEM implementation for Windows" Tags="|finite-element|computational-physics|open-source|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24502" PostTypeId="1" CreationDate="2016-07-21T09:28:15.667" Score="3" ViewCount="310" Body="&lt;p&gt;I have managed to make my model converge fairly well and achieve large displacements and deformations. It exhibits stress and strain good continuity by solving it under small displacement and deformation assumption. In other terms, engineering strain is computed instead of Green Lagrange strain, stress tensor is Cauchy and not Piola Kirchhoff, and material frame is associated to spatial frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the small displacement/deformation assumption is only for no more than 5% strain, but actually, the strain reaches 50% ; it moves 1cm into an 8cm solid box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know also that every single FEM can only be an approximation of reality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I expect about the validity of the model ? Does it mean absolutely nothing because it is quite nonsense (but the only thing is that the error is highered) ? &lt;/p&gt;&#xA;" OwnerUserId="20191" LastActivityDate="2016-08-13T10:30:45.420" Title="Produce large displacement under small displacement approximation?" Tags="|finite-element|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24506" PostTypeId="1" CreationDate="2016-07-21T11:55:36.447" Score="1" ViewCount="82" Body="&lt;p&gt;I was trying to implement a moving mesh algorithm using Gauss Seidel, so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a pde like this (Hamilton - Jacobi eq)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{cases}\phi_t+H(\phi_x)=0 &amp;amp; [-1,1]\times [0,T]\\\phi(x,0)=\phi_0 &amp;amp; t=0\end{cases}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I do this each step:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a) Use a standard solver to approximate solution (one step more in time)&#xA;b) Do 5 Gauss Seidel iteration to adapt grid &#xA;c) interpolate solution on the new grid&#xA;d) repeat until t=T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I have some problems with (b), maybe my weight function is not properly calculated:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;equidistribution principle says:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega(\phi_{j+1/2})(x_{j+1}-x_{j})-\omega(\phi_{j-1/2})(x_{j}-x_{j-1})=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that my GS iteration is (using $n,n+1$ as time's index)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega(\phi^n_{j+1/2})(x^n_{j+1}-x^{n+1}_{j})-\omega(\phi^n_{j-1/2})(x^{n+1}_{j}-x^{n+1}_{j-1})=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now if I want to use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega=\sqrt{1+\alpha|\phi_x|^2}$ , $\alpha&amp;gt;0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a central differencing for derivatives $\phi_x=\frac{1}{2}\left(\frac{\Delta^+_x\phi_j}{\Delta^+_xx_j}+\frac{\Delta^-_x\phi_j}{\Delta^-_xx_j}\right)$ , what's my linear system to solve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A=\begin{bmatrix}1 &amp;amp; 0 &amp;amp;0 &amp;amp;0 &amp;amp; \dots &amp;amp; 0\\\dots \\0 &amp;amp;\omega_{j-1/2} &amp;amp; -\omega_{j+1/2}-\omega_{j-1/2} &amp;amp; \omega_{j+1/2} &amp;amp; \dots &amp;amp; 0 \\\dots\\0&amp;amp; 0 &amp;amp; \dots &amp;amp; 0 &amp;amp;\dots &amp;amp;1\end{bmatrix}&#xA;x=\begin{bmatrix}x^{n+1}_1 \\x^{n+1}_2 \\\dots \\x^{n+1}_j\\\dots \\x^{n+1}_M  \end{bmatrix}&#xA;b=\begin{bmatrix}-1 \\0 \\\dots \\0\\\dots \\1  \end{bmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ax=b? I tried this with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega_{j+1/2}=\omega\left(\frac{1}{2}\left(\frac{\phi(x_{j+1})-\phi(x_{j})}{x_{j+1}-x_{j}}+\frac{\phi(x_{j})-\phi(x_{j-1})}{x_{j}-x_{j-1}}\right)\right)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\omega_{j-1/2}=\omega\left(\frac{1}{2}\left(\frac{\phi(x_{j})-\phi(x_{j-1})}{x_{j}-x_{j-1}}+\frac{\phi(x_{j-1})-\phi(x_{j-2})}{x_{j-1}-x_{j-2}}\right)\right)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;maybe this are wrong? my reference is &lt;a href=&quot;http://www.math.pku.edu.cn/pzhang/articles/34.pdf&quot; rel=&quot;nofollow&quot;&gt;this article&lt;/a&gt; (2.2.1 par)&lt;/p&gt;&#xA;" OwnerUserId="21085" LastEditorUserId="21085" LastEditDate="2016-07-21T12:44:27.963" LastActivityDate="2016-07-21T12:44:27.963" Title="Gauss Seidel moving mesh AMR hamilton Jacobi" Tags="|matlab|pde|algorithms|numerical-analysis|nonlinear-equations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24508" PostTypeId="1" AcceptedAnswerId="24510" CreationDate="2016-07-21T16:30:17.723" Score="6" ViewCount="2854" Body="&lt;p&gt;I want to approximate the rotation in SO(3) (ie. 3D rotation) that minimizes some objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a SO(3) rotation representation that lends itself to energy minimization. I'm writing a framework that performs minimization on shape parameters, where the goodness of the fit is given solely based on some arbitrary objective function which is a function of a rotation matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example solution would be to convert euler angles to matrix representation, minimize the objective function as a function of the angle parameters, and convert back to a matrix. This effectively to reparameterizing the objective function into some other rotation representation. However, one problem with euler angles is that the representation does not preserve uniformity and is prone gimbal locking problems. Thus I am afraid it may give biased results or create local minima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have heard it is possible to optimize based on quaternion or axis angle representations of 3D rotations, but I have been unable to find a good source or description.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Added)&#xA;More specifically. As suggested by Choward in the comments, a solution using quaternions would formaly be:&#xA;Find some quaternion,$q^*$, that satisfies: $q^*=\arg \min f(R(q))$, where $R(q)$ is a rotation matrix as a function of the quaternion, and $f(\cdot)$ is some cost function being minimized as a function of the rotation matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: typo&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit2: Added clarification based on comment.&lt;/p&gt;&#xA;" OwnerUserId="21089" LastEditorUserId="21089" LastEditDate="2016-07-21T17:17:05.333" LastActivityDate="2016-07-26T10:48:15.467" Title="Approximating rotation matrix of arbitrary objective function" Tags="|optimization|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24510" PostTypeId="2" ParentId="24508" CreationDate="2016-07-21T20:42:56.763" Score="3" Body="&lt;p&gt;So your problem statement is to find some quaternion, $q^*$, such that it satisfies the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ q^* = \text{arg} \min f(R(q))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f(\cdot)$ is some cost function, and $R(q)$ is a matrix created from some quaternion $q$. Now, this problem can be viewed as a more general problem, such as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \textbf{x}^* = \text{arg} \min J(\textbf{x})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\textbf{x} \in \mathbb{R}^4$, and $J(\cdot)$ is a more general cost function. In your case, you can tackle it in a variety of ways. If you can compute $\nabla J$ with respect to $\textbf{x}$, then you can use various gradient based methods. If not, then doing a gradient-free approach can work. Note that you will need to ensure you normalize the input $\textbf{x}$ within $J(\cdot)$ before you feed it into your original $f(R(\cdot))$ cost function.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-07-21T20:42:56.763" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24511" PostTypeId="1" AcceptedAnswerId="24512" CreationDate="2016-07-21T23:32:53.373" Score="9" ViewCount="6815" Body="&lt;p&gt;Background: My PhD was in 'Computational Science'. My dissertation was on the analysis of X-Ray Diffraction Data and analysis of thermally perturbed nuclei in the overall dynamic analysis of the molecular electron density for solid state physics. The takeaway? It was very much based in science.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion Computational Science is the pursuit of science, &quot;... a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe&quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Science&quot;&gt;wiki&lt;/a&gt;), via computational means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most position fo 'Data Science', however, seem more like 'data analysis' types of jobs. That is, heavy SQL queries, using pre-built R and Python models (linear regression, etc.) to draw conclusions from structured and unstructured data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is Computational Science a superset of Data Science? Are they interchangeable? Is Data Science an actual 'science'? Is Computational Science an actual 'science'?&lt;/p&gt;&#xA;" OwnerUserId="164" LastActivityDate="2016-07-22T04:55:58.847" Title="What is the difference between Computational Science and Data Science?" Tags="|data-analysis|" AnswerCount="1" CommentCount="1" ClosedDate="2016-07-25T18:13:47.490" ContentLicense="CC BY-SA 3.0" />
  <row Id="24512" PostTypeId="2" ParentId="24511" CreationDate="2016-07-22T04:55:58.847" Score="10" Body="&lt;p&gt;They are not interchangeable. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Computational science tends to refer more to HPC, simulation techniques (differential equations, molecular dynamics, etc.), and is usually referred to as scientific computing. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Data science tends to refer to computationally-intensive data analysis, like &quot;big data&quot;, bioinformatics, machine learning (optimization), Bayesian analyses using MCMC, etc. I think it's the same as what used to be referred to as computational statistics. It was the infusion of computer science with statistics, but many of the techniques that were developed dropped the rigorous Fisherian &quot;statistical testing&quot; (clustering, cross-validation techniques, data visualization) but kept the data part.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The most clear explanation of it came to me when I was teaching a workshop on Julia for Data Science and Scientific Computing. The data scientists wanted to learn Julia in order to do fast &quot;big data&quot; analysis, i.e. regressions and other GLMs on large data. The computational scientists (scientific computerers?) wanted to know how to easily write code to solve large linear systems on HPCs and GPUs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice those are two ways of saying the exact same computations, but with very different meanings. So in some sense similar, but still distinct (and there is cross-over between the disciplines, like using machine learning to learn parameters for PDEs from data). &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-07-22T04:55:58.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24513" PostTypeId="1" CreationDate="2016-07-22T11:22:35.557" Score="0" ViewCount="68" Body="&lt;p&gt;(Variant of Stats Exercise to Protect the Innocent).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting for $x = 0$ I can do one of two operations $\pm 1$ or $\pm 4$.  Let $T$ be one iteration of this step.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the mean and standard deviation after 10 steps mod 10 ? i.e. the distribution of $T^{10}(0) \mod 10 $ &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the mean and standard deviation after 100 steps mod 100 ? i.e. t he distribution of $T^{100}(0) \mod 100 $ &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the mean and standard deviation after 1000 steps mod 1000 ? i.e. t he distribution of $T^{1000}(0) \mod 1000 $ &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The exam had asked for 10 decimal places accuracy.  Also the problem has been completely altered but original in spirit.  Notice these do not look for limits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this be classified under a general theme?&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="4006" LastEditDate="2016-07-22T20:33:08.130" LastActivityDate="2016-07-22T20:33:08.130" Title="Exact distribution of random walk after 10 or 100 steps" Tags="|numerical-analysis|iterative-method|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24514" PostTypeId="1" AcceptedAnswerId="24538" CreationDate="2016-07-22T11:44:19.223" Score="3" ViewCount="897" Body="&lt;p&gt;I am working on block preconditioning and seemingly it is common to write customised Krylov solvers for them. Within each solver, the individual block linear system with preconditioners are approximated by a few runs of AMG V-cycle or a Gauss Seidel sweep. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods are however used as smoothers in scipy, or even FEniCs supported PETSc. I would like to have them as native solvers, like it is possible to do in MATLAB. Just to run a single AMG cycle standalone, for example. However, I am working in a python environment (FEniCs + Scipy + Numpy) and need guidance for the same. Are there any libraries that I can make use of to carry out these?&lt;/p&gt;&#xA;" OwnerUserId="18849" LastActivityDate="2016-07-26T09:25:23.967" Title="Python environments for AMG and Gauss Seidel as solvers instead of preconditioners" Tags="|linear-algebra|python|preconditioning|multigrid|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24515" PostTypeId="2" ParentId="24502" CreationDate="2016-07-22T14:48:52.150" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I know also that every single FEM can only be an approximation of reality.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By the FEM you just obtain an approximation to an (hopefully) &lt;a href=&quot;https://en.wikipedia.org/wiki/Well-posed_problem&quot; rel=&quot;nofollow&quot;&gt;well posed problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logical steps are as follows.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You first set a problem by choosing appropriate equations (congruence, constitutive equations, balance), a geometric domain, material constitutive parameters, and appropriate boundary and initial conditions. If the problem is well-posed this defines a unique &lt;em&gt;solution&lt;/em&gt; $L$.&lt;/li&gt;&#xA;&lt;li&gt;By the FEM you obtain an approximation $L_\text{FEM} \approx L$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The small strain/displacement assumption affect the equations that define $L$ (in the sense that you end up with a set of linear congruence and balance equations), and presumably you have a well-defined problem, and therefore are able to easily compute a converged solution $L_\text{FEM}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real question is if $L$ is physically meaningful or not, and of course this question has nothing to do with the particular method you used to compute an approximation to it. This is a question about the equations you are going to solve: are they able to model the physics of your problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is impossible to answer this question without analysing your specific problem and your solution. However in general it is very dangerous to apply continuum mechanics equations outside their domain of validity: why a solution $L$ defined under small displacement assumptions should be physically valid for finite displacements? In my own experience this is almost never the case, apart from some trivial problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations of linear mechanics can be presented as the linearisation of more general non-linear equations. Intuitively this means that you search for a solution on the tangent to a curve, instead of for a solution on the curve itself: you can make a strong point about the fact that you will end very far from the correct solution.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2016-07-22T14:48:52.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24516" PostTypeId="1" CreationDate="2016-07-22T15:56:23.177" Score="5" ViewCount="271" Body="&lt;p&gt;I am trying to understand the stability of a finite difference equation on the staggered grid.&lt;/p&gt;&#xA;&lt;p&gt;I could understand the Von Neumann stability analysis for the collocated grid for a simple acoustic equation  as explained in &lt;a href=&quot;http://www.math.umn.edu/%7Eolver/num_/lnp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; and in this  &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7156/von-newman-stability-analysis-for-2d-acoustic-wave-equation-explicit&quot;&gt;Von Newman stability analysis for 2D acoustic wave equation explicit&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However the problem is to analyse the wave equation which is presented in &lt;strong&gt;coupled form&lt;/strong&gt; on &lt;strong&gt;staggered grid&lt;/strong&gt; such as :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://geodynamics.usc.edu/%7Ebecker/teaching/557/reading/Virieux1987.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://geodynamics.usc.edu/~becker/teaching/557/reading/Virieux1987.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Above equations are better shown as in appendix A  of following&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.9512&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.9512&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions for stability analysis of this kind pde problems?&lt;/p&gt;&#xA;" OwnerUserId="20015" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-07-23T12:10:57.350" Title="Stability analysis for a hyperbolic PDE on staggered grid" Tags="|finite-difference|numerical-analysis|stability|hyperbolic-pde|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24518" PostTypeId="1" AcceptedAnswerId="24526" CreationDate="2016-07-22T16:58:45.347" Score="0" ViewCount="230" Body="&lt;p&gt;I am making a code in C that requires the equivalent of Matlab's '\' command for a linear system of the form AX=B where A is an NxN matrix and X, B are Nx1 vectors- i.e a code that performs X=A\B that is optimized to recognize it will be receiving a symmetric positive-definite sparse matrix. I was wondering if there is any open source code in C to do this operation.&lt;/p&gt;&#xA;" OwnerUserId="20973" LastActivityDate="2016-07-25T05:26:14.847" Title="Open Source Linear Algebra Library" Tags="|linear-algebra|linear-solver|matrix|c|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24519" PostTypeId="2" ParentId="24513" CreationDate="2016-07-22T20:18:54.827" Score="2" Body="&lt;p&gt;One way to do this is to note that since step sizes are independent of step directions, conditional on knowing the sequence of step sizes, the distribution of $T^n(0)$ is a sum of two random walks of $k$ and $n-k$ steps with step sizes 1 and 4, respectively (both of which are independent binomial random variables), where $k$ is the number of steps of size 1, which itself has a binomial distribution, independent of the random walks. Since the limits on the problem are pretty small, the probability of observing a given value of $T^n(0)$ can be calculated directly in time $O(n^3)$ from the binomial pmf's, and then expectation and standard deviation modulo $n$ calculated from that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this can be classified as a question about &lt;a href=&quot;https://en.wikipedia.org/wiki/Law_of_total_expectation&quot; rel=&quot;nofollow&quot;&gt;iterated expectation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, for $n=10$ this gives me mean $4.3798828125$ and s.d. $2.871979661$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-07-22T20:24:11.280" LastActivityDate="2016-07-22T20:24:11.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24520" PostTypeId="2" ParentId="20968" CreationDate="2016-07-23T01:59:28.427" Score="2" Body="&lt;p&gt;You don't say what is the condition when an inventory draw is allowed to match a sale, but I'm going to assume it is: both the timestamp and the item must match, if we want a particular sale to be matched to a particular inventory draw.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is right, your problem is separable by (time,item)-pair: for each timestamp that appears in the dataset and each item, look only at the sales &amp;amp; inventory draws that occur for that timestamp and that item and see if any of the inventory draws can't be matched up to a sale; then move on to the next pair.  This breaks down the problem into smaller pieces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given your problem statement, the following algorithm is the best you can do, for each (timestamp,item)-pair:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $d_1,\dots,d_m$ denote the quantities of the inventory draws (for that (time,item)-pair).  Let $s_1,\dots,s_n$ denote the quantities of the sales orders (for that (time,item)-pair).  Then:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If $d_1+\dots + d_m &amp;lt; s_1 + \dots + s_n$, something is wrong with your records (but no inventory draw is suspicious).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If $d_1+\dots + d_m = s_1 + \dots + s_n$, nothing is suspicious.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If $d_1+\dots + d_m &amp;gt; s_1 + \dots + s_n$, then at least one of the inventory draws is suspicious.  Realistically, there will typically be no way to tell &lt;em&gt;which&lt;/em&gt; inventory draw was at fault.  More precisely, let $\delta = (d_1+\dots + d_m) - (s_1 + \dots + s_n)$; any subset of draws whose quantities sum to $\delta$ &lt;em&gt;could&lt;/em&gt; be suspicious.  In other words, for any set $S \subseteq \{1,2,\dots,m\}$ such that $\sum_{i \in S} d_i = \delta$, the set $S$ could be suspicious.  You could enumerate all sets $S$, but typically there will be many sets.   All you know is that one of those sets is a set of suspicious inventory draws, but you won't know which.  Typically, you won't be able to put your finger on any one inventory draw and say that it is definitely problematic or definitely not suspicious (you can only say draw $i$ is definitely problematic if $i$ is present in &lt;em&gt;all&lt;/em&gt; sets $S$ such that $\sum_{i \in S} d_i = \delta$; you can only say that draw $i$ is definitely not suspicious if $i$ is not present in &lt;em&gt;any&lt;/em&gt; of those sets $S$).  Even worse, there will typically be exponentially many possible sets $S$, so finding all of them will probably take intractively long, if you have many orders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, realistically, the total quantity of inventory draws for some item at some time exceeds the total quantity of sales for that item at that time, then about all you can say is that at least one of those inventory draws seems suspicious; flag them all for a human to inspect.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To do better than this, you'll need additional information that goes beyond just timestamp, item, and quantity.  For instance, you might have additional information, such as the value of the item, the identity of the worker who did the inventory draw, and so on -- that might help you do better.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2016-07-23T01:59:28.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24521" PostTypeId="2" ParentId="3242" CreationDate="2016-07-23T02:10:54.060" Score="1" Body="&lt;p&gt;If you're using random forests -- and you should be; they're just better than decision trees, for most purposes -- then yes, you can measure a &quot;confidence interval&quot;.  A random forest is a collection of trees.  Each tree gives you a prediction.  The random forest classifier takes the majority vote (or average) of those predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get a confidence score, you can now look at the distribution of the predictions from all of the trees.  If there are 1000 trees, each one has made one vote, so you can look at the distribution of the 1000 votes for that input.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For instance, if you're doing boolean classification, the random forests classifier will take the majority vote of those 1000 votes; but you can also calculate what fraction of those 1000 votes agree with the output of the classifier, and use that as a confidence score.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For instance, if you're doing regression, the random forests predictor may take the average or median of the 1000 values from the 1000 trees; but you can also sort those 1000 values, find the 5th and 95th quartiles, and use those as a sort of 90% &quot;confidence interval&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Good libraries will do all this for you, and may also implement some more sophisticated methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's lots more written about this in the literature on random forests.  See, e.g., &lt;a href=&quot;https://stats.stackexchange.com/q/56895/2921&quot;&gt;https://stats.stackexchange.com/q/56895/2921&lt;/a&gt;, &lt;a href=&quot;https://stats.stackexchange.com/q/12425/2921&quot;&gt;https://stats.stackexchange.com/q/12425/2921&lt;/a&gt;, &lt;a href=&quot;http://blog.datadive.net/prediction-intervals-for-random-forests/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://blog.datadive.net/prediction-intervals-for-random-forests/&lt;/a&gt;, &lt;a href=&quot;http://blog.revolutionanalytics.com/2016/03/confidence-intervals-for-random-forest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://blog.revolutionanalytics.com/2016/03/confidence-intervals-for-random-forest.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4274" LastEditorUserId="-1" LastEditDate="2017-04-13T12:44:17.923" LastActivityDate="2016-07-23T02:10:54.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24522" PostTypeId="2" ParentId="24508" CreationDate="2016-07-23T04:22:32.237" Score="1" Body="&lt;p&gt;Basically the quaternions are the four dimensional Euclidean space together with a division algebra structure. The three dimensional space is the same as the imaginary part of the quaternions. A rotation of three space is given by a unit quarternion $q$ which acts on an imaginary quaternion $x$ by the formula $x \mapsto qx\bar{q}$. Now, one way you can implement rotations is the following: fix a vector in three space, say $\hat{a}=(a_1,a_2,a_3)^T$, write it as an imaginary quaternion $a=a_1 i + a_2 j + a_3 k$ and then a rotation around the vector $\hat{a}$ by angle $\theta$ is written as a unit quaternion in the form $$q = ( \cos{\theta}) + (\sin{\theta}) a.$$ Its inverse is its conjugat $$\bar{q}=  ( \cos{\theta}) - (\sin{\theta}) a.$$&#xA;Thus, your cost function depends on four parameters $J(\theta,a_1,a_2,a_3) = J(q,\bar{q})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: $ \hat{a}$ needs to be a unit vector so that $a$ is a unit imaginary quaternion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: In general, for any nonzero $q$, a transformation $x \mapsto q x q^{-1}$ of an imaginary quaternion $x$, is also a rotation so normalizing quaternions is not really necessary. This is true for example because $q^{-1} = \frac{1}{|q|^2} \bar{q}$, so $$q x q^{-1} = q x \left( \frac{\bar{q}}{|q|^2} \right) = \left(\frac{q}{|q|}\right) x \left(\frac{\bar{q}}{|q|}\right).$$ So it seem like you can end up with optimizing a function $J(q,\bar{q})$ without constraints.&lt;/p&gt;&#xA;" OwnerUserId="20964" LastEditorUserId="20964" LastEditDate="2016-07-23T20:06:32.530" LastActivityDate="2016-07-23T20:06:32.530" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24523" PostTypeId="2" ParentId="24508" CreationDate="2016-07-23T21:52:01.857" Score="3" Body="&lt;p&gt;I agree that quaternions are the way to go, as stated in other answers, but let me add a brief note on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation&quot; rel=&quot;nofollow&quot;&gt;axis-angle&lt;/a&gt; representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a vector &#xA;\begin{equation}&#xA;\mathbf{v} = \begin{bmatrix}v_1\\v_2\\v_3\end{bmatrix} &#xA;\end{equation}&#xA;let&#xA;\begin{align}&#xA;\theta &amp;amp; = \lvert \mathbf{v} \rvert &amp;amp; &#xA;\mathbf{e} &amp;amp; = \frac{\mathbf{v}}{\lvert \mathbf{v} \rvert}  &amp;amp;&#xA;[\mathbf{v}]_\times &amp;amp; = &#xA;\begin{bmatrix}&#xA;0 &amp;amp; -v_3 &amp;amp; v_2 \\&#xA;v_3 &amp;amp; 0 &amp;amp; -v_1 \\&#xA;-v_2 &amp;amp; v_1 &amp;amp; 0&#xA;\end{bmatrix}&#xA;\end{align}&#xA;and&#xA;\begin{equation}&#xA;\mathbf{R} = \exp([\mathbf{v}]_\times) &#xA;\end{equation}&#xA;where $\exp$ is the matrix exponential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be shown that $\mathbf R$ is a rotation matrix corresponding to a rotation of magnitude $\theta$ about axis $\mathbf e$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative expression of $\mathbf{R}$ is the Rodrigues' rotation formula:&#xA;\begin{equation}&#xA;\mathbf{R} = \mathbf{I} + \sin\theta \, [\mathbf{e}]_\times + (1-\cos\theta) \,[\mathbf{e}]_\times^2&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of this representation is that you can optimise with respect to the three independent parameters $(v_1, v_2, v_3)$. The disadvantage of course is that you have to evaluate a matrix exponential (!) or $\sin(\lvert \mathbf v \rvert)$ and $\cos(\lvert \mathbf v \rvert)$. However a quick and dirty implementation using this representation is almost trivial, especially in a high level language that, like scipy, has a matrix exponential.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2016-07-26T10:48:15.467" LastActivityDate="2016-07-26T10:48:15.467" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24524" PostTypeId="2" ParentId="24501" CreationDate="2016-07-24T06:36:20.900" Score="-2" Body="&lt;p&gt;Maybe FreeFem would be your best choice ;) &lt;a href=&quot;http://www.freefem.org/&quot; rel=&quot;nofollow&quot;&gt;website&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21085" LastActivityDate="2016-07-24T06:36:20.900" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24525" PostTypeId="1" AcceptedAnswerId="24531" CreationDate="2016-07-25T01:02:22.743" Score="0" ViewCount="714" Body="&lt;p&gt;I want to make sure I am clear with all the reasons why oscillations and how wild they get. I will summarize what I understood from the books with respect to the application of the Crank-Nicolson to the parabolic PDE. Consider the PDE&#xA;$$u_t=au_x+bu_{xx}$$&#xA;and some initial data $f(x)$. Assume there are some boundary conditions but they are irrelevant for discussion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem #1: If the initial data is rough, step function for example, then the oscillations might occur, but those are &quot;controlled&quot; and do decay, but slowly. So, I will not have an explosive solution but close to the region of the discontinuity I might have wiggles. However, if the time step is fine enough I can prevent them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem #2: if $a&amp;gt;&amp;gt;b$, or even of $b=0$, regardless of the initial condition, using central differences for the $au_x$ term will bring me trouble(implicit or explicit in time). This problem has nothing to do with kinks in the initial data. And this issue is a more serious one as the solution does explode. Regardless how small my time and space grid is I will have oscillations. Would I ever have problems if the initial data is very smooth?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did I get this two common problems correctly? I just want to make sure they have distinct roots! Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="1611" LastEditDate="2016-07-25T12:37:53.257" LastActivityDate="2016-07-25T18:26:14.280" Title="spurious oscillations Crank-Nicolson" Tags="|parabolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24526" PostTypeId="2" ParentId="24518" CreationDate="2016-07-25T05:26:14.847" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CHOLMOD&lt;/a&gt; is definitely the go-to for if you want a direct solver. It's what MATLAB and Julia use for solving A\b with a positive-definite sparse matrix, and likely many others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you may not want to use a direct solver for large sparse matrices. Since direct solvers are $\mathcal{O}(n^3)$, they tend to not do very well as the problem grows in size. Instead, people use iterative methods to iterative get solutions $Ax_n \approx b$ where the solution $x_n \rightarrow x$ as the number of iterations $n \rightarrow \infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, since your matrix is positive definite, conjugate-gradient methods are well suited for the problem. There are tons of implementations &lt;a href=&quot;https://people.sc.fsu.edu/~jburkardt/c_src/cg/cg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;like this one&lt;/a&gt; if you don't want a whole library. But then there are multigrid methods and all kinds of other things. There is a &lt;a href=&quot;https://scicomp.stackexchange.com/questions/81/what-guidelines-should-i-follow-when-choosing-a-sparse-linear-system-solver&quot;&gt;whole CSSE question devoted to picking the right method&lt;/a&gt;. If you want a library that would make it easy to  swap around to different solvers and see which one is fastest, try &lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/linearsolvertable.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PetsC&lt;/a&gt; (I linked the page which shows all of the solvers). If you're going to put some time into it and performance really matters, I would definitely go with PetsC.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-07-25T05:26:14.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24527" PostTypeId="2" ParentId="24514" CreationDate="2016-07-25T05:39:11.597" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/pyamg/pyamg&quot; rel=&quot;nofollow&quot;&gt;pyAMG&lt;/a&gt; is a good library for algebraic multigrid solvers. I don't use it in Python, but I have used the &lt;a href=&quot;https://github.com/cortner/PyAMG.jl&quot; rel=&quot;nofollow&quot;&gt;Julia wrapped version&lt;/a&gt; and have found it good enough that I will be using it in my &lt;a href=&quot;https://github.com/ChrisRackauckas/DifferentialEquations.jl&quot; rel=&quot;nofollow&quot;&gt;DifferentialEquations.jl&lt;/a&gt; pretty soon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For others, you may want to try &lt;a href=&quot;https://bitbucket.org/petsc/petsc4py&quot; rel=&quot;nofollow&quot;&gt;petsc4py&lt;/a&gt;. These are wrappers for PETSc. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2016-07-25T21:48:11.047" LastActivityDate="2016-07-25T21:48:11.047" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24529" PostTypeId="1" AcceptedAnswerId="24540" CreationDate="2016-07-25T12:37:07.900" Score="0" ViewCount="643" Body="&lt;p&gt;I have used the free, multibody dynamics software MBDyn for a while now. It is a good program and also fits my needs. There are good manuals, basic tutorials and examples to be found. There is also a mailing list. The plotting program I had to write myself but there are even suggestions for how to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as I get further into MBDyn I find slight problems constructing the models. I can't say if it is only my own lack of background in engineering and algebra that is the problem. (MBDyn interfaces with the graphical environment Blender, both for input and output, but for me a text interface for input is sufficient, so far.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is:&#xA;What software for multibody dynamics is available on the market?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in connecting solid objects by elastic and deformable joints, like hinges, ball joints etc. I wish to see how the bodies move when forces are applied and &quot;springs and damping&quot; interact with the connected bodies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never worked with the Finite Elements Method (FEM). I don't think that is what I need. I think that is more for modeling continuous structures like fluids, gases, solid materials etc. (I wish to model beams, but I think that this also done in multibody dynamics (MBDyn does it well). I don't need any detailed internal information from the beam. I just need to know &quot;a little bit how it bends&quot;.) I do not need any information about internal forces in the objects, just how they move.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So please tell, what multibody dynamics software exist? And, if possible, what are their differences?&lt;/p&gt;&#xA;" OwnerUserId="9385" LastEditorUserId="20688" LastEditDate="2018-04-05T14:50:33.040" LastActivityDate="2018-04-05T14:50:33.040" Title="What multibody dynamics softwares exist?" Tags="|computational-physics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24530" PostTypeId="1" CreationDate="2016-07-25T12:54:08.390" Score="1" ViewCount="537" Body="&lt;p&gt;I have some data in the form of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       X  Y  Z    Amplitude&#xA;       0  0  0    0.00&#xA;       1  0  0    0.00&#xA;       2  0  0    0.00&#xA;       3  0  0    0.00&#xA;       ...&#xA;      27 30  8    0.25&#xA;      28 30  8    0.25&#xA;      29 30  8    0.25&#xA;      30 30  8    0.25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I want to show them in a 3D plot, with Amplitude either shown in color or with isosurface shapes.&#xA;The number of elements in each vector exceeds 8000, so I cannot use meshgrid command (results in insufficient memory error).&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-12-23T01:30:44.703" Title="Generating 4D Plots in Matlab" Tags="|matlab|plotting|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24531" PostTypeId="2" ParentId="24525" CreationDate="2016-07-25T13:38:33.823" Score="4" Body="&lt;p&gt;The two problems you mention indeed have different roots. The first one is a consequence of &lt;em&gt;numerical dispersion&lt;/em&gt; and the other one of &lt;em&gt;numerical instability&lt;/em&gt;. Let me elaborate a little:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem #1:&#xA;The PDE in question is non-dispersive, i.e. waves of any frequency and wavenumber will travel equally fast - at speed $a$. However, after discretisation, this is no longer true. In fact, for the Crank-Nicolson scheme, the wavespeed is reduced somewhat to (roughly)&#xA;$$&#xA;a\left( 1 - \frac{2 + \lambda^2}{12}(\xi \Delta x)^2 \right),&#xA;$$&#xA;where $\lambda = \Delta t / \Delta x$ is the time to space mesh ratio, and $\xi$ is the wavenumber of a propagating wave. This means that waves of higher wavenumber will travel at a slower wavespeed than waves of lower wavenumbers. Now, assume that our initial solution can be expressed in terms of a Fourier transform. Then, if it has sharp gradients, it will typically have contributions from a large range of wavenumbers (i.e. both high and low), and hence, the solution &quot;decomposes&quot; into a train of waves travelling at different speeds. This typically looks like the oscillations you describe. These oscillations are no cause of instability and will not grow in time. In fact, if $b \geq 0$ they are dissipated along with the rest of the solution. Clearly, a small $\Delta x$ and $\Delta t$ reduces the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem #2:&#xA;The issue you describe here is due to numerical instability. This is seen e.g. for the FTCS (Forward in Time, Central in Space) discretisation which is a common text book example. Von Neumann analysis shows that this discretisation is unconditionally unstable, even for very smooth initial data. However, don't dismiss central differences as useless. There are ways to modify the discretisation such that central differences can be used &quot;almost everywhere&quot;, except near the domain boundaries, without experiencing any instability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;:&#xA;Below is an example of a stencil that uses central differences almost everywhere, yet is stable for the following advection problem with $a &amp;gt; 0$:&#xA;$$&#xA;u_t + au_x = 0 \\&#xA;u(x,0) = f(x) \\&#xA;u(0,t) = g(t).&#xA;$$&#xA;Let us discretise in space with a difference matrix $D = P^{-1}Q$ such that the semi-discrete problem becomes&#xA;$$&#xA;\mathbf{u}_t + aP^{-1}Q\mathbf{u} = \sigma P^{-1} (u_0 - g(t)) \mathbf{e}_0.&#xA;$$&#xA;Here, $\mathbf{u}=(u_0, u_1, \dots, u_{n-1}, u_n)^T$ is the semi-discrete solution vector defined on a mesh with grid spacing $\Delta x$. The right-hand side is known as a &lt;em&gt;simultaneous approximation term&lt;/em&gt; (SAT). It weakly imposes the boundary condition for $u_0$. Here, $\mathbf{e}_0 = (1,0,0,\dots)^T$ and $\sigma$ is a scalar that we will soon specify. The matrices involved are given by&#xA;$$&#xA;P = \Delta x \, \text{diag}(1/2, 1, 1, \dots, 1, 1, 1/2),&#xA;$$&#xA;$$&#xA;Q = &#xA;\begin{pmatrix}&#xA;-1/2 &amp;amp; 1/2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots \\&#xA;-1/2 &amp;amp; 0 &amp;amp; 1/2 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots \\&#xA;0 &amp;amp; -1/2 &amp;amp; 0 &amp;amp; 1/2 &amp;amp; 0 &amp;amp; \dots \\&#xA;&amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \\&#xA;&amp;amp; \dots &amp;amp; 0 &amp;amp; -1/2 &amp;amp; 0 &amp;amp; 1/2 \\&#xA;&amp;amp; &amp;amp; 0 &amp;amp; 0 &amp;amp; -1/2 &amp;amp; 1/2&#xA;\end{pmatrix}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To show that this discretisation is stable, note first that $P$ is symmetric and positive definite. Note further that $Q$ satisfies the so called &lt;em&gt;summation-by-parts&lt;/em&gt; (SBP) property&#xA;$$&#xA;Q + Q^T = \text{diag}(-1, 0, \dots, 0, 1).&#xA;$$&#xA;Now, let us perform a stability analysis:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define the norm $\| \mathbf{u} \|^2 = \mathbf{u}^T P \mathbf{u}$. Then it follows (I'll leave a few details for you to fill in) that&#xA;\begin{align}&#xA;\| \mathbf{u} \|^2_t &amp;amp;= \mathbf{u}_t^T P \mathbf{u} + \mathbf{u}^T P \mathbf{u}_t \\&#xA;&amp;amp;= -a \mathbf{u}^T (Q + Q^T) \mathbf{u} + 2 \sigma u_0 (u_0 - g(t)) \\&#xA;&amp;amp;= -a u_n^2 + u_0^2 (a + 2 \sigma) - 2 \sigma u_0 g(t).&#xA;\end{align}&#xA;Choosing $\sigma = -a$, then adding and subtracting $a g(t)^2$ gives the final result&#xA;$$&#xA;\| \mathbf{u} \|^2_t = a \left( g(t)^2 - u_n^2 - (u_0 - g(t))^2 \right) \leq a g(t)^2.&#xA;$$&#xA;Thus, the norm of the solution is bounded by the boundary data and hence, by definition, the discretisation is stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;:&#xA;In summary, this discretisation uses the central difference stencil everywhere except at the first and last grid points. It is stable, and will remain stable upon discretisation in time, provided a suitable $\Delta t$ is chosen.&lt;/p&gt;&#xA;" OwnerUserId="19512" LastEditorUserId="19512" LastEditDate="2016-07-25T18:26:14.280" LastActivityDate="2016-07-25T18:26:14.280" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24533" PostTypeId="1" CreationDate="2016-07-25T19:15:01.993" Score="2" ViewCount="130" Body="&lt;p&gt;Consider an ODE of the form:&#xA;$$ u'(t)=-\frac{1}{\varepsilon}u(u-\frac{1}{2})(u-1) $$&#xA;with the initial value&#xA;$$ u(0)=u_0. $$ Here $\varepsilon&amp;gt;0$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is easy to verify that $u\equiv0$ and $u\equiv1$ are two stable equilibriums for the ODE. And the ODE has the property:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if $u_0&amp;lt;\frac{1}{2}$, then $u(t)\rightarrow0$ as $t\rightarrow\infty$;&lt;/li&gt;&#xA;&lt;li&gt;if $u_0=\frac{1}{2}$, then $u(t)\equiv\frac{1}{2}$;&lt;/li&gt;&#xA;&lt;li&gt;if $u_0&amp;gt;\frac{1}{2}$, then $u(t)\rightarrow1$ as $t\rightarrow\infty$;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the problem, $\varepsilon$ is very small. I want to design a numerical scheme that has two property:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the time step size $\Delta t$ is independent of $\varepsilon$. &lt;/li&gt;&#xA;&lt;li&gt;the numerical solutions enjoy has the same asymptotic property as the exact solution.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have tried the Euler backward scheme:&#xA;$$ u_{n+1} = u_n -\frac{\Delta t}{\varepsilon}u_{n+1}(u_{n+1}-\frac{1}{2})(u_{n+1}-1). $$&#xA;However, this scheme, which is a nonlinear algebraic equation for $u_{n+1}$, does not necessarily have a unique solution for any $\varepsilon$ and $\Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find a high-order scheme (at least third-order) for this ODE. It seems that the Runge-Kutta method does not fit my purpose.&lt;/p&gt;&#xA;" OwnerUserId="20711" LastEditorUserId="20711" LastEditDate="2016-07-26T13:15:41.473" LastActivityDate="2016-07-27T05:47:14.847" Title="Solving ODE with multiple equilibriums" Tags="|ode|nonlinear-equations|stiffness|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24534" PostTypeId="2" ParentId="24530" CreationDate="2016-07-25T20:49:53.423" Score="1" Body="&lt;p&gt;Since you have only 8 slices along the z, it's not unreasonable to make it an animation slicing through z (x-y plot for z=0, then for z=1, ...). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if you try to make the 4D plot, likely you won't be able to see much beyond the front. You can use transparencies, but it still won't be possible to view the middle. So you can plot around the outside. So instead of a full meshgrid, you can plot the outside edges of the box with color + a little bit of transparency, and then the first two rows inward. That should save a lot of memory and may be able to fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or try to get some time on an HPC's large memory node. Some of the ones on XSEDE are made with visualization in mind. You may be able to get some time via the &lt;a href=&quot;https://portal.xsede.org/allocations/announcements#trial&quot; rel=&quot;nofollow&quot;&gt;Comet trial allocations&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-07-25T20:49:53.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24537" PostTypeId="1" AcceptedAnswerId="24653" CreationDate="2016-07-26T09:09:17.260" Score="8" ViewCount="1113" Body="&lt;p&gt;I am facing a problem as follows: I have a box full of points with a certain unknown distribution and I would like to calculate its Voronoï Diagram. The problem is that the number of points is so huge that this may be impossible to do for the full distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I have planned to do that for just a region inside the box, where the number of points was not that big. In order to do so I need to know how to calculate the minimum region that may affect to the Voronoi diagram of a certain smaller region inside that box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, I would like to compute the Voronoï diagram of the points inside the small cube of the figure below that fits with the Voronoï diagram that would have the points of the full box storing the smallest Voronoï diagram possible on the memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/J6ZeD.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/J6ZeD.png&quot; alt=&quot;Explanation of the problem.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21134" LastActivityDate="2016-08-08T19:53:42.597" Title="Computing the Voronoi diagram of a region inside a box" Tags="|voronoi-diagrams|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24538" PostTypeId="2" ParentId="24514" CreationDate="2016-07-26T09:25:23.967" Score="2" Body="&lt;p&gt;FEniCS tends to hide the details about the actual matrices it builds, and prevent easy manipulation of them. As far as I can tell this is a design decision, as they are trying to create an all-encompassing closed pipeline for discretizing and solving PDES. The particular details of matrix storage and matrix elements are considered an implementation detail that only developers should be concerned with. The available linear algebra backend formats change from version to version, breaking old code. There is no built in function to export a sparse matrix to the scipy format (.toarray() converts to a dense matrix!); you have to specify a backend type and then write your own exporting function specifically for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For someone who just wants to specify a PDE and get a solution, maybe this is fine, but for someone like you or me who wants to experiment with building different preconditioners, solvers, investigate spectral properties of the matrices, etc, this is frustrating. So, typically what I do is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get the matrix out of FEniCS and into scipy&lt;/li&gt;&#xA;&lt;li&gt;Do the linear algebra in scipy, using my own code or packages like pyamg&lt;/li&gt;&#xA;&lt;li&gt;Bring the results back into FEniCS for post-processing.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Anyways, I do this so often that it only took me a few minutes to whip up the following simple code that does this process, testing both a simple Gauss-Seidel solver and a multigrid solver from pyAMG on a simple elliptic PDE:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from dolfin import *&#xA;import numpy as np&#xA;import scipy.sparse as sps&#xA;import scipy.sparse.linalg as spla&#xA;from pyamg import *&#xA;&#xA;dolfin.parameters.linear_algebra_backend = 'Eigen'&#xA;&#xA;def fenics_eigen_to_scipy_csr(assembled_fenics_form):&#xA;    row,col,val = as_backend_type(assembled_fenics_form).data()&#xA;    M = sps.csr_matrix((val,col,row))&#xA;    M.eliminate_zeros()&#xA;    return M&#xA;&#xA;def do_gauss_seidel(b, A, tol=1e-5, maxiter=1000):&#xA;    dd = A.diagonal()&#xA;    D = sps.dia_matrix(A.shape)&#xA;    D.setdiag(dd)&#xA;    L = sps.tril(A, -1)&#xA;    U = sps.triu(A, 1)&#xA;&#xA;    solve_DL = spla.factorized(D + L)&#xA;    normb = np.linalg.norm(b)&#xA;    x = np.zeros(b.shape)&#xA;    for ii in range(maxiter):&#xA;        normr = np.linalg.norm(b - A*x)&#xA;        if normr/normb &amp;lt; tol:&#xA;            break&#xA;        x = solve_DL(b - U*x)&#xA;    return x&#xA;&#xA;&#xA;mesh = RectangleMesh(Point(0.0, 0.0), Point(1.0, 1.5), 100, 200)&#xA;V = FunctionSpace(mesh, 'CG', 1)&#xA;u = TrialFunction(V)&#xA;v = TestFunction(V)&#xA;a = inner(nabla_grad(u), nabla_grad(v))*dx + 10*u*v*dx&#xA;&#xA;A = fenics_eigen_to_scipy_csr(assemble(a))&#xA;b = np.random.rand(V.dim())&#xA;&#xA;x_true = spla.spsolve(A, b)&#xA;&#xA;x_gs = do_gauss_seidel(b, A, maxiter=1000)&#xA;print 'relative error in gauss-seidel solve:', np.linalg.norm(x_true - x_gs)/np.linalg.norm(x_true)&#xA;&#xA;ml = ruge_stuben_solver(A)&#xA;print ml&#xA;x_amg = ml.solve(b, tol=1e-10, maxiter=2)&#xA;print &quot;relative error in amg solve:&quot;, np.linalg.norm(x_true - x_amg)/np.linalg.norm(x_true)&#xA;&#xA;# Get the vector back into a FEniCS Function and plot it:&#xA;xtrue_fct = Function(V)&#xA;xtrue_fct.vector()[:] = np.copy(x_true)&#xA;plot(xtrue_fct, interactive=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2016-07-26T09:25:23.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24539" PostTypeId="1" CreationDate="2016-07-26T11:17:32.820" Score="0" ViewCount="294" Body="&lt;p&gt;This is a follow-up question to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/24470/how-do-i-form-the-chebyshev-differentiation-matrix-in-matlab&quot;&gt;How do I form the Chebyshev differentiation matrix in MATLAB?&lt;/a&gt; The goal of the following code is to solve the Poisson problem:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [PerInfError]=Poisson2Dn0dc1sev12(N)&#xA;%Solve -nabla^2u=f(x,y) on Omega=[-1,1]^2, with u(Gamma)=x^3,  leading&#xA;%to I(U) =int(0.5(nabla.u)^2-fu)dxdy&#xA;%Exact solution u(x,y) = sin(pix)sin(piy)+x^3 +y.^3&#xA;%Current example has f(x,y) = 2pi^2sin(pix)sin(piy)+6x+6y&#xA;%Using GLL nodes over 1 spectral element&#xA;&#xA;%------------GLL Nodes, weights and differentiation matrix-----------------&#xA;N1=N+1;&#xA;cheb=cos(pi*(0:N)/N)';&#xA;unif=linspace(-1,1,N1)';&#xA;if N&amp;lt;3&#xA;    x=cheb;&#xA;else&#xA;    x=cheb+sin(pi*unif)./(4*N);&#xA;end&#xA;P=zeros(N1,N1); %N1xN1 zero matrix&#xA;xold=2;&#xA;&#xA;while max(abs(x-xold))&amp;gt;eps&#xA;    xold=x;&#xA;    P(:,1)=1;&#xA;    P(:,2)=x;&#xA;    %set first collumn entries to 1 (P(:,1) = 1);&#xA;    %set second collumn entries to x (P(:,2) = x);&#xA;    % and so on, the collumns of this matrix are the legendre polynomials&#xA;    for k=2:N&#xA;        P(:,k+1)=( (2*k-1)*x.*P(:,k)-(k-1)*P(:,k-1) )/k;&#xA;        %%Bonnets formula ;)&#xA;    end&#xA;    % Roots of (1-x^2)L'_N&#xA;    x=xold-( x.*P(:,N1)-P(:,N) )./( N1*P(:,N1));&#xA;end&#xA;&#xA;%---chebyshev differentiation matrix-------------------------------------------------&#xA;x=-x;&#xA;X=repmat(x,1,N1);%set every collumn of X to x&#xA;Xdiff=X-X'+eye(N1); % X MINUS ITS TRANSPOSE YOU IDIOT&#xA;L=repmat(P(:,N1),1,N1);&#xA;L(1:(N1+1):N1*N1)=1;&#xA;D=(L./(Xdiff.*L'));&#xA;D(1:(N1+1):N1*N1)=0;&#xA;D(1)=-(N1*N)/4;&#xA;D(N1*N1)=(N1*N)/4;&#xA;%---GLL Quad Weights-------------------------------------------------------&#xA;&#xA;w=2./(N*N1*P(:,N1).^2);&#xA;&#xA;%---matrix of GLL x and y coords-------------------------------------------&#xA;&#xA;Xcoord = repmat(x(2:N),1,N-1)';&#xA;Ycoord = repmat(x(2:N),1,N-1);&#xA;&#xA;%---defining the known exact solution uEx----------------------------------&#xA;&#xA;UEx=sin(pi.*Xcoord).*sin(pi.*Ycoord);&#xA;&#xA;%----- RHS contribution --- zero dirichlet boundary conditions-------------&#xA;b=zeros((N-1)^2,1);&#xA;A= zeros((N-1)^2,(N-1)^2);&#xA;for k = 1:(N-1)^2;&#xA;    if mod(k,N-1)==0;&#xA;        n = N-1;&#xA;    else&#xA;        n = mod(k,N-1);&#xA;    end&#xA;    m = 1+floor((k-1)./(N-1));&#xA;&#xA;    %----Full RHS for non zero dirichlet---------------------------------------&#xA;&#xA;    b(k,1)= w(m+1).*w(n+1).*(2*(pi.^2).*sin(pi.*Xcoord(m,n))...&#xA;                                      .*sin(pi.*Ycoord(m,n)));&#xA;&#xA;    %--- LHS contribution---split into the three contributions-----------------&#xA;    AY1=zeros(N-1,N-1);&#xA;    AX1=zeros(N-1,N-1);&#xA;&#xA;    for ii = 1:N-1;&#xA;        AX1(ii,n)=sum(w(n+1).*w(:).*D(:,m+1).*D(:,ii+1)); %for A=A1=A2&#xA;        AY1(m,ii)=sum(w(m+1).*w(:).*D(:,n+1).*D(:,ii+1)); %for A=A1=A2&#xA;    end&#xA;&#xA;    AXYSum=AX1+AY1;&#xA;    A(k,:)=reshape(AXYSum,1,(N-1)^2);&#xA;end&#xA;%---Solving the resulting linear system----equating the Error--------------&#xA;&#xA;UEstV= linsolve(A,b);&#xA;UEst=reshape(UEstV,N-1,N-1);&#xA;Error=abs(UEst-UEx);&#xA;PerInfError=norm(UEx-UEst,inf)./norm(UEx,inf);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is what I think the LHS should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Omega}\nabla u \nabla v\approx w_l\sum_{i = 0}^{N}\sum_{m = 0}^{N}w_mD_{mi}D_{mk}u_{il} + w_k\sum_{j = 0}^{N}\sum_{n = 0}^{N}w_nD_{nj}D_{nl}u_{kj}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $D$ is the differentiation matrix mentioned in the original post. I got to this result by letting $u(x,y) = \sum_{i = 0}^{N}\sum_{j = 0}^{N}u_{ij}h_i(x)h_j(y)$ and $v(x,y) = h_k(x)h_l(y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the matrix, from this, should look something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MdKOs.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MdKOs.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for some blocks being diagonal and others being &quot;Full&quot; is that the first term in the above expression only makes sense when $i=k$, this is because it represents the derivative in the x direction and the second term when $j=l$ for the same reason but in the y direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to understand how the author has implemented this. I am being quite vain here, assuming that he has done it this way.&lt;/p&gt;&#xA;" OwnerDisplayName="user21030" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-07-26T11:51:08.513" Title="How to form the stiffness matrix for the Poisson equation using a spectral method" Tags="|matlab|elliptic-pde|spectral-method|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24540" PostTypeId="2" ParentId="24529" CreationDate="2016-07-26T12:29:30.943" Score="1" Body="&lt;p&gt;Your question is really too broad for this site.&#xA;There are dozens of commercial and open source&#xA;multibody dynamics codes, many of which can be identified&#xA;by simple Google search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you didn't restrict your question to open source, I'll&#xA;mention two commercial packages that have been very widely used for more&#xA;than 20 years:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MSC/Adams&#xA;&lt;a href=&quot;http://www.mscsoftware.com/product/adams&quot; rel=&quot;nofollow&quot;&gt;http://www.mscsoftware.com/product/adams&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LMS/DADS&#xA;&lt;a href=&quot;https://www.plm.automation.siemens.com/en_us/products/lms/virtual-lab/legacy-applications/dads.shtml&quot; rel=&quot;nofollow&quot;&gt;https://www.plm.automation.siemens.com/en_us/products/lms/virtual-lab/legacy-applications/dads.shtml&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you take a look at the survey paper,&#xA;Computational strategies for flexible multibody systems, by Wasfy and Noor that focuses on adding&#xA;flexibility effects to rigid body dynamics codes. It also mentions the two codes I list above.&#xA;A pdf version of this paper is currently available here: &#xA;&lt;a href=&quot;http://hosting.umons.ac.be/html/mecara/grasmech/PaperWasfyNoor.pdf&quot; rel=&quot;nofollow&quot;&gt;http://hosting.umons.ac.be/html/mecara/grasmech/PaperWasfyNoor.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This site (&lt;a href=&quot;http://sbel.wisc.edu/Courses/ME451/2014/&quot; rel=&quot;nofollow&quot;&gt;http://sbel.wisc.edu/Courses/ME451/2014/&lt;/a&gt;), contains a course on multibody dynamics taught&#xA;at the University of Wisconsin. Among other useful documents, it contains a copy of a book on this&#xA;topic by Edward Haug, one of the original authors of the DADS code. This book discusses different&#xA;approaches for multi- rigid body dynamics so should help to answer your question about differences&#xA;among the various codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, you say you are not interested in FEM. But you should understand that, from a software point&#xA;of view, in recent years, the distinction between FE codes that perform nonlinear structural dynamics&#xA;and multibody dynamics codes that include flexiblity effects has become very blurred. On the one hand,&#xA;the Abaqus FE code (&lt;a href=&quot;http://www.3ds.com/products-services/simulia/products/abaqus/&quot; rel=&quot;nofollow&quot;&gt;http://www.3ds.com/products-services/simulia/products/abaqus/&lt;/a&gt;)&#xA;has a wide variety of joints and links for constructing mechanisms. On the other hand, the multibody dynamics&#xA;code you mention, MBDyn, has a relatively general library of (essentially) finite elements for modeling&#xA;flexible bodies. You can easily find several papers where MBDyn has been used for solving structural analysis&#xA;problems that, traditionally, would have been solved by a FE structural code.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-07-26T12:29:30.943" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24541" PostTypeId="2" ParentId="24529" CreationDate="2016-07-26T12:54:34.933" Score="1" Body="&lt;p&gt;The open source code &lt;a href=&quot;http://www.hotint.org/&quot; rel=&quot;nofollow&quot;&gt;HOTINT&lt;/a&gt; also has the capability to simulate multi-body systems, with and without deformation. This software can also be coupled with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Discrete_element_method&quot; rel=&quot;nofollow&quot;&gt;DEM&lt;/a&gt; code &lt;a href=&quot;http://www.cfdem.com/liggghts-open-source-discrete-element-method-particle-simulation-code&quot; rel=&quot;nofollow&quot;&gt;LIGGGHTS&lt;/a&gt; to model the interaction of multi-body systems with particles or fluids (via the &lt;a href=&quot;https://en.wikipedia.org/wiki/Smoothed-particle_hydrodynamics&quot; rel=&quot;nofollow&quot;&gt;SPH&lt;/a&gt; approach).&lt;/p&gt;&#xA;" OwnerUserId="3510" LastActivityDate="2016-07-26T12:54:34.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24542" PostTypeId="2" ParentId="24501" CreationDate="2016-07-26T13:04:36.953" Score="1" Body="&lt;p&gt;The are &lt;a href=&quot;https://sourceforge.net/projects/asterwin/&quot; rel=&quot;nofollow&quot;&gt;Windows ports&lt;/a&gt; of the open source FEM code &lt;a href=&quot;https://en.wikipedia.org/wiki/Code_Aster&quot; rel=&quot;nofollow&quot;&gt;Code_Aster&lt;/a&gt;, although these ports are not up to date with the main project, which is available on Linux or FreeBSD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code_Aster is part of the &lt;a href=&quot;http://www.salome-platform.org/&quot; rel=&quot;nofollow&quot;&gt;Salome&lt;/a&gt; software collection, see also &lt;a href=&quot;https://en.wikipedia.org/wiki/Salome_(software)&quot; rel=&quot;nofollow&quot;&gt;Wikipedia an Salome&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3510" LastActivityDate="2016-07-26T13:04:36.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24543" PostTypeId="2" ParentId="24533" CreationDate="2016-07-26T13:32:17.440" Score="0" Body="&lt;p&gt;I don't quite understand what the problem is with the direct Euler method. Let $f(u) =u(u-1/2)(u-1)$ so the equation is $u'=-(1/ \varepsilon)f(u)$ and use the scheme $u_{n+1}=u_n - (\Delta t / \varepsilon)f(u_n) = F(u_n)$. To study the behavior of the numerical solutions, we study the dynamics of the map $u \mapsto F(u)$. Check that $F(0)=0, F(1/2)=1/2, F(1)=1$. Then &#xA;$F(u)=u(1-(\Delta t/\varepsilon)(u-1/2)(u-1))=ug(u)$. For $u \in (0,1/2)$, $g(u)&amp;lt;1, g'(u)&amp;lt;0$, so it's strictly decreasing,  and thus. $F(u)&amp;lt;u$. Hence for any $u_n \in (0,1/2), u_{n+1}=F(u_n)&amp;lt;u_n,$ so $g(u_{n+1})&amp;lt;g(u_n)&amp;lt;g(u_{n-1})&amp;lt;...g(u_0):=q &amp;lt;1$ as long as $u_0 \in (0,1/2)$. Now $u_{n+1}=F(u_n)=u_ng(u_n)&amp;lt;q u_n &amp;lt; q^2 u_{n-1}&amp;lt;...&amp;lt;q^{n+1} u_0$. Now you see that when $n$ goes to infinity $u_n$ converges to $0$. The rest is handled the same way. &lt;/p&gt;&#xA;" OwnerUserId="20964" LastEditorUserId="20964" LastEditDate="2016-07-26T15:52:26.000" LastActivityDate="2016-07-26T15:52:26.000" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24544" PostTypeId="2" ParentId="8998" CreationDate="2016-07-26T15:07:15.587" Score="1" Body="&lt;p&gt;The Pros are the &quot;tools&quot; built into each functional language: It is so easy to filter data, it is so easy to iterate over data and it is so much easier to come up with a clear and concise solution to your problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only Con is, that you have to get your head around this new kind of thinking: It could take some time to learn what you have to know.&#xA;Others in the SciComp domain don't really use those languages, which means you can't get that much support :(&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in functional-scientific-languages, I developed one&#xA;&lt;a href=&quot;https://ac1235.github.io&quot; rel=&quot;nofollow&quot;&gt;https://ac1235.github.io&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21141" LastActivityDate="2016-07-26T15:07:15.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24545" PostTypeId="2" ParentId="8998" CreationDate="2016-07-26T16:08:00.447" Score="12" Body="&lt;p&gt;I have maybe a unique perspective on this because I am a HPC practitioner with a scientific computation background as well as a functional programming language user. I don't want to equate HPC with scientific computation, but there is considerable intersection, and so that is the point of view I take in answering this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Functional languages are unlikely to be adopted in HPC for now primarily because HPC users and customers genuinely care about achieving as close to peak performance as possible. It is true that when code is written in a functional way it naturally exposes parallelism that can be exploited, but in HPC that is not enough. Parallelism is only one piece of the puzzle in achieving high performance, you must also take into account a wide range of micro-architectural details and doing this generally requires very fine-grained control over the execution of code, that control is not available in any functional languages that I know of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I have high hopes this may change. I have noticed a trend that researchers are starting to realize that a lot of these micro-architectural optimizations can be automated (to an extent). This has bred a zoo of source-to-source compiler technology where a user inputs a &quot;specification&quot; of the computation they want to have happen, and the compiler outputs C or Fortran code which realizes that computation with the optimizations and parallelism necessary to efficiently use the target architecture. Incidentally this is what functional languages are well adapted to doing: modeling and analyzing programming languages. It is no accident that the first major adopters of functional languages were compiler developers. With a few notable exceptions I have not seen this actually take hold yet, but the ideas are there, and functional languages are the ideal tool for realizing those ideas.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2016-07-26T18:08:07.940" LastActivityDate="2016-07-26T18:08:07.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24546" PostTypeId="2" ParentId="24533" CreationDate="2016-07-26T16:26:13.543" Score="4" Body="&lt;p&gt;There are a couple of questions implicit in your post:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How does one deal with non-uniqueness of the algebraic equations generated by any implicit numerical method?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Typically you have a very good initial guess -- the previous time step solution.  This will usually ensure that e.g. Newton's method converges to the correct root.  For extremely large values of $\Delta t$ this is not true; I don't believe that there is a completely generic way to get around this other than by not taking $\Delta t$ too large.  In your particular case you know that the correct solution is real and lies in the interval $[0,1]$, which may help.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How does one ensure high-order convergence for arbitrarily stiff problems?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Use a Runge-Kutta method with high stage order, or use a high order linear multistep method.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-07-26T16:26:13.543" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24547" PostTypeId="1" AcceptedAnswerId="24549" CreationDate="2016-07-26T16:26:31.377" Score="1" ViewCount="101" Body="&lt;p&gt;I have a robust RNG that generates random 32-bit (unsigned) ints. As is probably well known, for metropolis MC simulation, a random number between 0 and 1 is needed to determine acceptance/rejection of a move, so I need to convert my 32-bit int into a (double-precision) floating point value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what is best practice for converting my 32-bit int into a float? I think I have the following two options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;(double) getRandomNumber() / 4294967295&lt;/code&gt; (2^32 - 1); range [0,1]&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;(double) getRandomNumber() / 4294967296&lt;/code&gt; (2^32); range [0,1)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first option seems straightforward, with a range of [0,1]. There would be 2147483648 values in the range [0,0.5) and 2147483648 values in the range (0.5, 1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second option, we have a range of [0,1), and 0.5 is represented exactly among the 2^32 possible floats. There is symmetry in that 50% of the values are in the range [0, .49999999977], and the other 50% are in the range [0.5, .99999999977].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there details in the int-&gt;double conversion that I am missing? What's standard practice for a range for probabilistic simulations of this type? Is it [0,1], (0,1], [0,1), or (0,1)? Or am I splitting hairs over a trivial matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note I've read that division is less efficient than multiplication, so I do intend to precompute the reciprocal and multiply by that value instead in my actual implementation.&lt;/p&gt;&#xA;" OwnerUserId="21132" LastActivityDate="2016-07-26T17:37:47.147" Title="RNG float range for metropolis monte carlo" Tags="|floating-point|monte-carlo|random-number-generation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24548" PostTypeId="1" CreationDate="2016-07-26T17:18:49.600" Score="2" ViewCount="94" Body="&lt;p&gt;To randomly permute a list one has to select one of $n!$ possibilities, which quickly goes outside of the capabilities of conventional PRNGs.  Even Mersenne Twister fails with a list of length greater than 2080 entries, and probably fails much sooner in practice because there's no guarantee that its state is used in a way that eliminates duplicate outcomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in the process of designing an algorithm to shuffle a list from inadequate state, and designing this as a library function (rather than as an application-specific solution), what would normally be the design criteria that should be met (or maximised or minimised) first, and what compromises are likely to be the most benign?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is in the general case.  Obviously some applications are oblivious to problems that others could turn into a catastrophic failure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, how do you evaluate the quality of a shuffle when you know in advance it must be imperfect (in the same way as you know a PRNG is imperfect)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I ask, and as an example of the sorts of things I'd like to detect, is that during development of a shuffle algorithm I got a couple of obviously-bad results but wasn't sure what the best way to formalise them as tests would be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An acutely problematic shuffle, for example would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;static uint32_t n;     // number of elements in list to shuffle&#xA;static uint32_t m;     // n rounded up to a power-of-two, minus one&#xA;static uint32_t state; // set to a random value to randomise order&#xA;&#xA;void shuffle_seed(uint32_t seed) {&#xA;  state = s;&#xA;}&#xA;&#xA;uint32_t next_index(void) {&#xA;  do {&#xA;    seed = (state * 1103515245 + 12345) &amp;amp; m;&#xA;  } while (state &amp;gt;= n);&#xA;  return state;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can call this function &lt;code&gt;n&lt;/code&gt; times and be guaranteed &lt;code&gt;n&lt;/code&gt; unique results between &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;n&lt;/code&gt;... so it's a permutation; but when you compare the result for one seed to that for another you find that they're the same sequence starting at different points.  Also, every second index is odd and the others even.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another, slightly more advanced design gave partial substring matches rather than the whole thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can choose only $n$ or $n^k$ (for smallish $k$) different permutations out of $n!$, ones containing similar sub-strings are probably the ones you'd want to eliminate first.  I expect there are a variety of other kinds of poor choice that I haven't thought of.&lt;/p&gt;&#xA;" OwnerUserId="21143" LastEditorUserId="21143" LastEditDate="2016-07-30T21:58:44.373" LastActivityDate="2016-07-30T21:58:44.373" Title="How to evaluate &quot;quality&quot; of set of attainable random permutations" Tags="|random-number-generation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24549" PostTypeId="2" ParentId="24547" CreationDate="2016-07-26T17:37:47.147" Score="0" Body="&lt;p&gt;I think you're splitting hairs on something trivial. Maybe for a cryptographic RNG this detail might matter, but I'd be surprised if your choice here ever affected any result. For one thing, any bias of this size would only even be detectable if you generate something like 2^30 numbers in a go, which at that point you should be looking at using a longer period RNG sequence anyways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would just go with the [0,1] approach since getting exactly 0.5 could lead to some nasty bugs that would be hard to find later (part of the code uses &amp;lt;0.5 for reject/accept, other parts &gt;0.5, for almost every random number this would still work but this bug would be something that would be hard to reproduce and only rear its head in the long (and therefore important) runs).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-07-26T17:37:47.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24550" PostTypeId="2" ParentId="24548" CreationDate="2016-07-26T17:47:23.317" Score="2" Body="&lt;p&gt;You can do some spot checking. Testing for all $n!$ possibilities is impractical, but you can make some unit tests to make sure that after doing sufficiently many shuffles certain properties of the distribution are correct. For example, make the array $[1,2,\ldots,n]$ and check (for $x=2,5,7,12$)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the mean value for the index of $x$ after shuffling approximately $n/2$?&lt;/li&gt;&#xA;&lt;li&gt;Is the variance of the index of $x$ after shuffling close to its theoretical moment?&lt;/li&gt;&#xA;&lt;li&gt;Is the average distance between $x_i$ and $x_j$ close the theoretical mean distance ($i$ and $j$ in the test set)&lt;/li&gt;&#xA;&lt;li&gt;Is $x_i&amp;gt;x_j$ (indices after shuffling) nearly half the time?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And you can keep going until you feel comfortable that it's achieving at least approximately the shuffle's theoretical statistics. (Is this what you were asking?)&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-07-26T17:47:23.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24551" PostTypeId="2" ParentId="24533" CreationDate="2016-07-26T18:08:36.880" Score="2" Body="&lt;p&gt;If you relax the independence between $\epsilon$ and $\Delta t$ to be &quot;approximate&quot;, i.e. use a method which has a really large stable region even if it's not A-stable, then there are plenty of methods which can get good performance. A standard set of methods for this are the &lt;a href=&quot;https://en.wikipedia.org/wiki/Backward_differentiation_formula&quot; rel=&quot;nofollow&quot;&gt;Backward Differentiation Formulas (BDF)&lt;/a&gt; or the Numerical Differentiation Formulas (NDF). Usually these implementations get quite complicated because the lower order BDF/NDF methods have better stability, so they are normally implemented as variable-timestep and variable-order. For lower tolerance but faster solving one usually uses Rosenbrock methods of orders 2/3 (which is also easier to implement). [Note that although the BDF methods are not A-stable, the stability region always includes the full negative real axis, meaning that for a lone real-valued ODE like the one you have here, these methods are stable for any $\epsilon$ and $\Delta t$. But that fact doesn't generalize to systems of equations or complex-valued ODEs].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So these give implicit methods. However, as you noticed, the implicit methods do not necessarily have a unique solution for what is the next step. How do you choose one? Normally, you assume that your previous timestep is close enough that whatever root finder you use (Newton's method, or others) is going to converge to &quot;the right one&quot; (here you have a positive, a zero, and a negative solution so it's easy to tell when it fails). This has a timestep assumption in there that the derivation of the implicit method doesn't account for, which could lead to some unexpected issues. If this ends up giving you issues, one way to deal with this is to make some kind of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Predictor%E2%80%93corrector_method&quot; rel=&quot;nofollow&quot;&gt;Predictor-Corrector method&lt;/a&gt; where you use a Runge-Kutta 4/5 step to produce an estimate for the step (or split the timestep and do a few low-order but stable explicit steps to get an estimate), and use that estimate as the initial guess for Newton's method. This estimate should be closer to the true step, at least enough that Newton's method will now converge to the right place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dealing with stiff problems has a lot of research behind it. Not only that, but it has years of software development. So if you're really looking for solvers which make a good compromise between speed and accuracy on hard problems, you should try to stick to tried-and-tested software. You can directly call &lt;a href=&quot;http://www.unige.ch/~hairer/software.html&quot; rel=&quot;nofollow&quot;&gt;some of the classics&lt;/a&gt;, or you can use wrappers in a higher level language (&lt;a href=&quot;http://www.mathworks.com/help/matlab/math/choose-an-ode-solver.html&quot; rel=&quot;nofollow&quot;&gt;MATLAB&lt;/a&gt;,&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html&quot; rel=&quot;nofollow&quot;&gt;Python&lt;/a&gt;, &lt;a href=&quot;https://github.com/ChrisRackauckas/DifferentialEquations.jl&quot; rel=&quot;nofollow&quot;&gt;Julia (wrappers to Hairer and other algorithms coming really soon!)&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2016-07-27T05:47:14.847" LastActivityDate="2016-07-27T05:47:14.847" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24552" PostTypeId="2" ParentId="24548" CreationDate="2016-07-26T18:11:21.400" Score="2" Body="&lt;p&gt;To permute a list, there is no need to enumerate the n! possibilities. Shuffling the list should be sufficient. Pseudo code (from wiki article) for shuffling would be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i from n−1 downto 1 do&#xA;    j ← random integer such that 0 ≤ j ≤ i&#xA;    exchange a[j] and a[i]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For c++11, std::shuffle along with a good RNG should work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle&quot; rel=&quot;nofollow&quot;&gt;This wiki article&lt;/a&gt; was my source. The article does note the following in the &quot;Pseudorandom generators&quot; subsection&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An additional problem occurs when the Fisher–Yates shuffle is used with a pseudorandom number generator or PRNG: as the sequence of numbers output by such a generator is entirely determined by its internal state at the start of a sequence, a shuffle driven by such a generator cannot possibly produce more distinct permutations than the generator has distinct possible states. Even when the number of possible states exceeds the number of permutations, the irregular nature of the mapping from sequences of numbers to permutations means that some permutations will occur more often than others. Thus, to minimize bias, the number of states of the PRNG should exceed the number of permutations by at least several orders of magnitude.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This problem can be avoided by using a true RNG that depends on processor noise such as std::random_device*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;* although I guess random_device also has caveats: &lt;a href=&quot;http://www.pcg-random.org/posts/cpps-random_device.html&quot; rel=&quot;nofollow&quot;&gt;http://www.pcg-random.org/posts/cpps-random_device.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21132" LastEditorUserId="21132" LastEditDate="2016-07-26T18:34:15.777" LastActivityDate="2016-07-26T18:34:15.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24553" PostTypeId="2" ParentId="8998" CreationDate="2016-07-26T18:17:07.097" Score="8" Body="&lt;p&gt;The quick summary is that &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Numerical computing uses mutability/side-effects to achieve most of its speedups and reduce allocations (many functional programming structures have immutable data)&lt;/li&gt;&#xA;&lt;li&gt;Lazy evaluation can be rough to use with numerical codes.&lt;/li&gt;&#xA;&lt;li&gt;Either you are developing a package where dropping down to the lowest levels for performance really matters (C/Fortran or now Julia) (in these you can also edit the assembler code as necessary), or you are writing a script which uses these fast libraries so you mostly tend to care about development time (and so you choose Julia/MATLAB/Python/R). Functional languages tend to sit in a weird middle ground which is helpful in other disciplines, but not as helpful here.&lt;/li&gt;&#xA;&lt;li&gt;Most of the numerical algorithms for differential equations, optimization, numerical linear algebra, etc. are written/developed/proved in terms of convergence, i.e. you have approximation $x_n$ and you want to get $x_{n+1}$. The natural style to implement these algorithms is a loop. (There are some algorithms written recursively like some multigrid algorithms, but these are much more rare.)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These facts together make functional programming not seem necessary to most users.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-07-26T18:17:07.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24554" PostTypeId="2" ParentId="20030" CreationDate="2016-07-26T23:47:29.547" Score="1" Body="&lt;p&gt;OK, so you want to select M points from a given set of N points in the Euclidean plane, so that the sum of pairwise distances of the selected points is maximal, correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard local search algorithm is pretty fast and offers a pretty good approximation. The runtime is linear in N and quadratic in M. Its approximation ratio is 1 - 4/M. This means that the ratio gets better as M increases. For instance, for M=10 it gets 60% the optimal value, and for M=50 it gets 92% the optimal value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm works also for Euclidean spaces of general dimension. In this case, the problem is NP-hard. But on the plane, it is not known if it is NP-hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source is &lt;a href=&quot;https://arxiv.org/abs/1607.04557&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. Hope this helps!&#xA;Best, Alfonso&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;" OwnerUserId="21147" LastActivityDate="2016-07-26T23:47:29.547" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24555" PostTypeId="1" CreationDate="2016-07-27T03:03:46.843" Score="8" ViewCount="356" Body="&lt;p&gt;It's common parlance to say that Linux is faster, and for good reasons. But as stated in the title, how much does choice of OS matter for performance of scientific computing code? For something things which the operating system is more directly involved (such as playing games or video rendering), sure there can be a difference. But if I run some codes in MATLAB, solve PDEs in Julia, or are doing some linear algebra on a GPU using CUDA: is there actually a measurable performance difference between Windows, MacOS, and Linux?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if anyone has performed benchmarks to see if such a difference exists. Maybe differences exist when using some software like MATLAB because of differences in the compilers they use? Or maybe CUDA drivers are measurably faster for one OS than the other? Or do OS's sometimes interfere a lot like like the &lt;a href=&quot;https://stackoverflow.com/questions/19944429/cuda-performance-penalty-when-running-in-windows&quot;&gt;WDDM CUDA issue&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I know there are other reasons for choosing an operating system which is beyond the scope of this question. I am specifically wondering about performance.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-08-01T14:20:53.223" Title="How much does choice of OS matter for performance of scientific computing code?" Tags="|performance|benchmarking|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24556" PostTypeId="1" CreationDate="2016-07-27T06:27:36.340" Score="3" ViewCount="683" Body="&lt;p&gt;I have measurements of magnetic field on a 3d grid. My measurements are distributed on four x-y planes similar to what is shown in the image below. The measurements roughly follow a Cartesian grid but have varying spacing and gaps. The measurements are more sparse in the z-direction but I expect the magnetic field to vary by an order of magnitude more in the x and y direction. Each vector component of the magnetic field ($B_x$, $B_y$, $B_z$) is measured at different points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the image below I show the measurement points for $B_x$. The points for $B_y$, $B_z$ are distributed similarly but not at the exact same points. The whole domain is not convex but I am only looking at the region inside the black rectangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Rjifz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rjifz.png&quot; alt=&quot;measurement positions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to calculate the curl of the magnetic field to determine the current density. E.g. $j_z = 1/\mu_{0} (\partial B_y / \partial x - \partial B_x / \partial y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the data is non-uniform and $B_x$ and $B_y$ are not measured at same locations I can not use simple finite difference relations. I have tried three approaches &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;interpolate the $B_x$ and $B_y$ with scipy.interpolate.gridata(..., method='linear') to a common grid and then take 1st order differences between the point and divide by the spacing.&lt;/li&gt;&#xA;&lt;li&gt;interpolate the $B_x$ and $B_y$ with scipy.interpolate.gridata(..., method='linear') to a common grid and then take 2nd order differences between the points with gradient and divide by the spacing.&lt;/li&gt;&#xA;&lt;li&gt;Triangulate the $B_x$ and $B_y$ positions with  scipy.spatial.Delaunay and pass the triangulation and data to matplotlib.tri.LinearTriInterpolator which has a gradient method. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I plotted the magnetic field and $j_z$ for each of the three methods. Only the third (left to right) plot seems to be a curl of the field and gives me the magnitude that I expect from the experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CxJzV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CxJzV.png&quot; alt=&quot;current&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;scipy.interpolate.griddata uses scipy.interpolate.LinearNDInterpolator which triangulates the positions and uses linear barycentric interpolation. The same is true for matplotlib.tri. Are these results expected? If all methods are using linear interpolation why is the third method more accurate than taking differences of the interpolated values? &lt;/p&gt;&#xA;" OwnerUserId="14391" LastActivityDate="2016-07-27T06:27:36.340" Title="Gradients of non-uniformly sampled data in 3D space" Tags="|finite-difference|interpolation|scipy|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24557" PostTypeId="1" CreationDate="2016-07-27T10:18:18.677" Score="2" ViewCount="231" Body="&lt;p&gt;I want to discretize and numerically solve the equation:&#xA;\begin{equation}&#xA;v(k)\dfrac{\partial f}{\partial z} + F(z)\dfrac{\partial f}{\partial k} = \alpha f(z,k) \,\,.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discretization with central differences,&#xA;\begin{equation}&#xA;v(k_j)\left(\dfrac{f(z_{i+1},k_j) - f(z_{i-1},k_j)}{2\Delta z}\right) +&#xA;F(z_i)\left(\dfrac{f(z_{i},k_{j+1}) - f(z_i,k_{j-1})}{2\Delta k}\right) = &#xA;\alpha f(z_i,k_j)&#xA;\end{equation}&#xA;results in spurious oscillations. Using upwind scheme,&#xA;\begin{equation}&#xA;v(k_j)\left(\dfrac{f(z_{i},k_j) - f(z_{i-1},k_j)}{\Delta z}\right) +&#xA;F(z_i)\left(\dfrac{f(z_{i},k_{j}) - f(z_i,k_{j-1})}{\Delta k}\right) = &#xA;\alpha f(z_i,k_j)   &#xA;\qquad {\color{blue}{\bigg[ v(k_j)&amp;gt;0 \,,\, F(z_i) &amp;gt; 0 \bigg]}} &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;v(k_j)\left(\dfrac{f(z_{i},k_j) - f(z_{i-1},k_j)}{\Delta z}\right) +&#xA;F(z_i)\left(\dfrac{f(z_{i},k_{j+1}) - f(z_i,k_{j})}{\Delta k}\right) = &#xA;\alpha f(z_i,k_j)   &#xA;\qquad {\color{blue}{\bigg[ v(k_j)&amp;gt;0 \,,\, F(z_i) &amp;lt; 0 \bigg]}} &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;v(k_j)\left(\dfrac{f(z_{i+1},k_j) - f(z_{i},k_j)}{\Delta z}\right) +&#xA;F(z_i)\left(\dfrac{f(z_{i},k_{j}) - f(z_i,k_{j-1})}{\Delta k}\right) = &#xA;\alpha f(z_i,k_j)   &#xA;\qquad {\color{blue}{\bigg[ v(k_j)&amp;lt;0 \,,\, F(z_i) &amp;gt; 0 \bigg]}} &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;v(k_j)\left(\dfrac{f(z_{i+1},k_j) - f(z_{i},k_j)}{\Delta z}\right) +&#xA;F(z_i)\left(\dfrac{f(z_{i},k_{j+1}) - f(z_i,k_{j})}{\Delta k}\right) = &#xA;\alpha f(z_i,k_j)   &#xA;\qquad {\color{blue}{\bigg[ v(k_j) &amp;lt; 0 \,,\, F(z_i) &amp;lt; 0 \bigg]}} &#xA;\end{equation}&#xA;removes these oscillations but produces inaccurate results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other simple scheme to improve the accuracy while keeping on to stability of discretizaion? Can I combine the trapezoidal rule with upwind scheme? What does the resulting discretization look like?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-08-11T11:28:40.073" Title="Combining trapezoidal rule with upwind scheme" Tags="|finite-difference|finite-volume|discretization|numerics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24559" PostTypeId="2" ParentId="24557" CreationDate="2016-07-27T15:29:38.467" Score="3" Body="&lt;p&gt;I would recommend to try a second order accurate upwind scheme. There are many of them. Let me show two of them, I use them for solving nonstationary advection equation when the solution is approaching steady state. I call them like in the book of LeVeque on hyperbolic problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I write them as the finite difference scheme for the term&#xA;$$&#xA;v \frac{\partial f(z)}{\partial z} \,, &#xA;$$&#xA;and the extension to your 2D equation shall be straightforward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first scheme is derived from (time dependent) Fromm scheme and it takes the form&#xA;$$&#xA;v \, \frac{f(z_{i+1}) + 3 f(z_i)-5 f(z_{i-1}) +f(z_{i-2})}{4 \Delta z} \hbox{ ,  if  } v&amp;gt;0 ,&#xA;$$&#xA;and&#xA;$$&#xA;v \, \frac{-f(z_{i+2}) + 5 f(z_{i+1})-3 f(z_{i}) - f(z_{i-1})}{4 \Delta z} \hbox{ ,  if  } v&amp;lt;0 .&#xA;$$&#xA;For time dependent problems the related Fromm scheme gives for many problems the most accurate numerical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second scheme can be derived from Beam-Warming second order upwind scheme and it takes the form&#xA;$$&#xA;v \, \frac{3 f(z_{i}) - 4 f(z_{i-1}) + f(z_{i-2})}{2 \Delta z} \hbox{ ,  if  } v&amp;gt;0 ,&#xA;$$&#xA;and&#xA;$$&#xA;v \, \frac{-f(z_{i+2}) + 4 f(z_{i+1})-3 f(z_{i}) }{2 \Delta z} \hbox{ ,  if  } v&amp;lt;0 .&#xA;$$&#xA;The related scheme for nonstationary advection equation is less accurate than the Fromm scheme for many examples. The advantage is that it has a stencil that is one-sided, and it might produce less oscillations than the Fromm scheme. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You must be aware that any second order scheme can produce unphysical oscillations in numerical solution. Depending on your problem (e.g. the smoothness of initial condition and  of given velocity field) for enough fine grids the oscillations can be negligible. If it is not the case you must use so-called limiting procedures to reduce the order of accuracy in some grid points. The resulting scheme is then nonlinear, because the coefficients are depending on the unknown numerical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I repeat that I recommend to try it, it is not a definite solution, but if you need to care about the accuracy, it is a direction you should follow.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-08-11T11:28:40.073" LastActivityDate="2016-08-11T11:28:40.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24561" PostTypeId="1" AcceptedAnswerId="24563" CreationDate="2016-07-27T19:48:54.697" Score="1" ViewCount="2109" Body="&lt;p&gt;I have recently come across the &lt;a href=&quot;https://en.wikipedia.org/wiki/Strang_splitting&quot; rel=&quot;nofollow&quot;&gt;Strang splitting&lt;/a&gt; and have some questions. For the differential equation of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ dy/dt = (L_1 + L_2)y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strang splitting implement the time splitting as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{eqnarray*}&#xA;\tilde y_1 &amp;amp; = &amp;amp; e^{L_1 \Delta t/2} y_0, &#xA;&amp;amp; &amp;amp; \bar y_1 = e^{L_2 \Delta t} \tilde y_1,&#xA;&amp;amp; &amp;amp; y_1 = e^{L_1 \Delta t/2} \bar y_1 \\&#xA;\tilde y_2 &amp;amp; = &amp;amp; e^{L_1 \Delta t/2} y_1,&#xA;&amp;amp; &amp;amp; \bar y_2 = e^{L_2 \Delta t} \tilde y_2,&#xA;&amp;amp; &amp;amp; y_2 = e^{L_1 \Delta t/2} \bar y_2 \\&#xA;... \\&#xA;\tilde y_n &amp;amp; = &amp;amp; e^{L_1 \Delta t/2} y_{n-1},&#xA;&amp;amp; &amp;amp; \bar y_n = e^{L_2 \Delta t} \tilde y_n,&#xA;&amp;amp; &amp;amp; y_n = e^{L_1 \Delta t/2} \bar y_n \\&#xA;\end{eqnarray*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from the equations above, it is obviously that the half time step with $y_1$ and $\tilde y_2$ can be combined into a single time step, so it is equivalent to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{eqnarray*}&#xA;\tilde y_1 &amp;amp; = &amp;amp; e^{L_1 \Delta t/2} y_0,&#xA;&amp;amp; &amp;amp; \bar y_1 = e^{L_2 \Delta t} \tilde y_1 \\&#xA;\tilde y_2 &amp;amp; = &amp;amp; e^{L_1 \Delta t} \bar y_1,&#xA;&amp;amp; &amp;amp; \bar y_2 = e^{L_2 \Delta t} \tilde y_2 \\&#xA;... \\&#xA;\tilde y_n &amp;amp; = &amp;amp; e^{L_1 \Delta t} \bar y_{n-1},&#xA;&amp;amp; &amp;amp; \bar y_n = e^{L_2 \Delta t} \tilde y_n \\&#xA;y_n &amp;amp; = &amp;amp; e^{L_1 \Delta t/2} \bar y_n &#xA;\end{eqnarray*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This look like that it is exactly the same as the first order time splitting scheme except the first and last half time step, and the computation is faster with the reduction. Am I missing something here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, what is the time splitting method with four order? How does it look like &lt;strong&gt;explicitly&lt;/strong&gt;? I am looking for some numerical algorithms for solving nonlinear Schroedinger equation.&lt;/p&gt;&#xA;" OwnerUserId="3444" LastActivityDate="2016-07-28T16:01:14.237" Title="Strang splitting" Tags="|pde|nonlinear-equations|time-integration|operator-splitting|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24562" PostTypeId="2" ParentId="24561" CreationDate="2016-07-27T20:07:53.887" Score="1" Body="&lt;p&gt;You can combine those two steps since both of them involve the same operator ($L_1$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the fourth-order method I recommend that you check out &lt;a href=&quot;https://stackoverflow.com/a/18298180/128966&quot;&gt;my answer to a similar question on Stack Overflow&lt;/a&gt; and also Section 4 in the following paper (see also the discussion below):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yoshida, H. (1990). Construction of higher order symplectic integrators. Physics Letters A, 150(5–7), 262–268. &lt;a href=&quot;http://doi.org/10.1016/0375-9601(90)90092-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doi.org/10.1016/0375-9601(90)90092-3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $a = L_1$ and $b = L_2$. The analysis of this type of schemes is done using the Baker-Campbell-Hausdorff (BCH) formula, which tells us that the expression for $c$ in&#xA;\begin{equation}&#xA;  \exp\{t a\} \exp\{t b\} = \exp\{t c\},&#xA;\end{equation}&#xA;is given in terms of Lie brackets involving $a$ and $b$ (recall that $[a, b] = ab - ba$):&#xA;\begin{equation}&#xA;  c = a + b&#xA;  + \frac{t}{2} \left[a, b\right]&#xA;  + \frac{t^2}{12} \left( \left[a, \left[a, b\right]\right] + \left[b, \left[b, a\right]\right] \right)&#xA;  + \frac{t^3}{24} \left[a, \left[b, \left[b, a\right]\right]\right]&#xA;  + \dotsb&#xA;\end{equation}&#xA;The BCH formula for the particular case of the Strang splitting is given by&#xA;\begin{equation}&#xA;  \exp\left\{\tfrac{t}{2} a\right\} \exp\left\{t b\right\} \exp\left\{\tfrac{t}{2} a\right\} = \exp\left\{t c_1 + t^3 c_3 + t^5 c_5 + \dotsb \right\},&#xA;\end{equation}&#xA;where&#xA;\begin{align*}&#xA;  c_1 &amp;amp;= a + b, \\&#xA;  c_3 &amp;amp;= -\frac{1}{24} \left[a, \left[a, b\right]\right] + \frac{1}{12} \left[b, \left[b, a\right]\right], \\&#xA;  &amp;amp;\dotsb&#xA;\end{align*}&#xA;In general, $c_3$ does not vanish and this implies that the scheme is second order. In order to construct higher order methods, one can choose a splitting such that the appropriate terms in the BCH formula vanish (the paper by Yoshida cited above discusses this in depth).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Particular choices of $a$ and $b$ will yield symplectic methods but from the discussion above we see that the order of the scheme does not necessarily have to do with it being symplectic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional useful references are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hairer, E., Lubich, C., &amp;amp; Wanner, G. (2010). Geometric numerical integration (Vol. 31). book, Springer, Heidelberg.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Leimkuhler, B., &amp;amp; Reich, S. (2004). Simulating Hamiltonian dynamics (Vol. 14). book, Cambridge University Press, Cambridge.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-07-28T16:01:14.237" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24563" PostTypeId="2" ParentId="24561" CreationDate="2016-07-28T08:29:55.167" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This look like that it is exactly the same as the first order time splitting scheme except the first and last half time step, and the computation is faster with the reduction. Am I missing something here?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You're exactly correct, as others have already said.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, what is the time splitting method with four order?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are many higher-order splitting schemes, some developed very recently.  &lt;a href=&quot;http://www.asc.tuwien.ac.at/~winfried/splitting/&quot; rel=&quot;nofollow&quot;&gt;This is the most comprehensive list I know of&lt;/a&gt; and includes various methods of order 4.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-07-28T08:29:55.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24564" PostTypeId="2" ParentId="65" CreationDate="2016-07-28T08:31:55.367" Score="2" Body="&lt;p&gt;A new resource for high-order splitting schemes that lists quite a few can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.asc.tuwien.ac.at/~winfried/splitting/&quot; rel=&quot;nofollow&quot;&gt;http://www.asc.tuwien.ac.at/~winfried/splitting/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-07-28T08:31:55.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24565" PostTypeId="2" ParentId="24557" CreationDate="2016-07-28T09:06:04.717" Score="1" Body="&lt;p&gt;If at all possible don't use a forward difference derivative formula since it doesn't always provide convergence in all cases since it leads to a form of extrapolation. Instead always use a backward difference derivative. Remember: consistency + stability = convergence and forward difference derivatives do not guarantee consistency. Its a general rule of thumb I always use and a mistake thats not easy to always see in formulae. BTW, central difference second derivative formulas are always consistent.&lt;/p&gt;&#xA;" OwnerUserId="21161" LastActivityDate="2016-07-28T09:06:04.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24566" PostTypeId="2" ParentId="1273" CreationDate="2016-07-28T09:23:23.990" Score="1" Body="&lt;p&gt;For integration use a gaussian quadrature rule when dealing with exponentials as a rule of thumb. By way of example, integrating a cubic function over a small area with simpsons rule or trapezoidal rule doesn't need a large N, but using them for an exponential function leads to huge N required to achieve convergence. So always go for an exponential based interpolant (i.e: gaussian quadrature rule). If this fails, then you need to plot the function to see how quickly it is oscillating and how it dies/rises with the area of integration.&lt;/p&gt;&#xA;" OwnerUserId="21161" LastActivityDate="2016-07-28T09:23:23.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24567" PostTypeId="1" CreationDate="2016-07-28T09:47:55.853" Score="0" ViewCount="55" Body="&lt;p&gt;My program uses a lot of ellipsoid shaped polygons in 2 and 3 D. So far, I create them by the simple, well-known &lt;a href=&quot;https://en.wikipedia.org/wiki/Ellipsoid#Parameterization&quot; rel=&quot;nofollow&quot;&gt;parametrization about two angles&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'd like to have my ellipses a bit dented and distorted. Currently, I just need &lt;em&gt;some&lt;/em&gt; sort of non-perfect ellipsoids, later I'd like to tune it to realisitic production defects. The ellipsoids should roughly look like they were drawn/modeled by a not-so-talented child or like they were made of thin aluminium and someone dented them with its fingers. So, the size of dents is roughly in the order of 1/10 to 1/5 of the ellipsoid size.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there any parametrizations to get distorted and dented ellipsoid? Or does this problem have a name to find literature about it?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2016-07-30T05:08:21.707" Title="Parametrization of distorted and dented ellipsoids" Tags="|computational-geometry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24568" PostTypeId="1" AcceptedAnswerId="24715" CreationDate="2016-07-28T11:55:39.230" Score="1" ViewCount="100" Body="&lt;p&gt;In &lt;a href=&quot;https://www.researchgate.net/publication/222466209_Global_properties_of_pseudospectral_methods&quot; rel=&quot;nofollow&quot;&gt;this&lt;/a&gt; paper, on page 243, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d_{jk} = \frac{a_j}{a_k(x_j - x_k)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $$a_k = \prod_{l = 0;l\neq k}^{N}(x_k-x_l)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, $a_k$ requires evaluating N multiplications. Why does the author say, &quot;$N^2$ operations are required to compute $a_j$&quot;?&lt;/p&gt;&#xA;" OwnerDisplayName="user21030" LastActivityDate="2016-08-15T14:13:44.920" Title="What is meant by &quot;operations&quot;?" Tags="|algorithms|complexity|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24570" PostTypeId="1" CreationDate="2016-07-28T17:10:56.040" Score="3" ViewCount="178" Body="&lt;p&gt;I am doing a nested integral via quadrature. To give a definite example, lets say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_0^2dx \left[x + \int_0^x dy \, 2y\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So effectively I'm integrating $x + x^2$ from 0 to 2 (although obviously my actual problem can't be analytically solved like this). Its implemented in code something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inner(x) = quad(y-&amp;gt;2*y, 0, x, tolerance=t_inner)&#xA;quad(x + inner(x), 0, 2, tolerance=t_outer)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT: Also note, in this example $x$ appears only as a limit to the inner integral, but my actual problem it appears in the integrand as well. Generally I just mean that the inner &lt;code&gt;quad&lt;/code&gt; call depends on $x$ in some way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of this example is that for small $x$, the first term is bigger than the second, whereas for larger $x$, the second term (the inner integration) dominates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would dictate that I could speed up my code by setting the inner quadrature error tolerance (&lt;code&gt;t_inner&lt;/code&gt;) large when $x$ is small, since error on the inner integral at small $x$ contributes less to my overall error. Then I can increase &lt;code&gt;t_inner&lt;/code&gt; when I get to large $x$ when this inner quadrature really matters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is there some general prescription for figuring out the optimal way to do this kind of thing? Is there some way I can just choose my &quot;global&quot; tolerance, &lt;code&gt;t_outer&lt;/code&gt;, and have &lt;code&gt;t_inner&lt;/code&gt; picked automatically as a function of $x$? &lt;/p&gt;&#xA;" OwnerUserId="21166" LastEditorUserId="21166" LastEditDate="2016-07-28T21:14:43.703" LastActivityDate="2016-09-01T00:26:59.653" Title="Adaptive tolerance for nested quadrature" Tags="|numerical-analysis|quadrature|error-estimation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24571" PostTypeId="2" ParentId="24570" CreationDate="2016-07-28T21:53:36.870" Score="0" Body="&lt;p&gt;I haven't fully thought this through, but it seems like t_inner should be changing by function of the derivative of the inner integral with respect to x. For example, lets say we have a function t_inner = A*(B - d(2y)/dx). You can pick values for A and B by defining boundary conditions at the integral limits, or by setting t_outer = t_inner at the value of x where x = inner(x). Intuitively it seems like you don't need both A and B (either A=1 or B=0) but I'm not sure which is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, as you're iterating through x you can then calculate a new t_inner for each x by looking at the numerical change in inner(x) over the previous iteration. As long your x spacing is fairly smooth with respect to inner(x) it seems like this should work.&lt;/p&gt;&#xA;" OwnerUserId="21170" LastActivityDate="2016-07-28T21:53:36.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24572" PostTypeId="1" CreationDate="2016-07-16T10:57:19.957" Score="1" ViewCount="647" Body="&lt;p&gt;When I use Matlab's &lt;code&gt;surf&lt;/code&gt; function, I notice that I can't re-perform its interpolation, which produces unreliable results. Do you know what does &lt;code&gt;surf&lt;/code&gt; do under the hood?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following code compares &lt;code&gt;spline&lt;/code&gt; and &lt;code&gt;surf&lt;/code&gt; interpolation: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z= linspace(-1,1,4001);&#xA;x = linspace(-2*pi,2*pi,150);&#xA;sharp_gausian = @(y,sigma) exp((-y.^2).*sigma);&#xA;&#xA;% initializing low-resolution surf&#xA;for j=1:length(z)&#xA;    psi_grid(j,:) = sharp_gausian(x + sin(5*z(j)),10);&#xA;end&#xA;subplot(2,2,1)&#xA;s =surf(x,z, psi_grid);&#xA;set(s,'linestyle','none')&#xA;xlabel('$x$' , 'FontSize', 14, 'Interpreter', 'latex');&#xA;ylabel('$z$', 'FontSize', 14, 'interpreter', 'latex','rot',0);&#xA;title('$Nx = 150$')&#xA;&#xA;% using higher resolution grid and interpolating the function using spline.&#xA;x_fine = linspace(-2*pi,2*pi,1500);&#xA;for j=1:length(z)&#xA;    psi_grid2(j,:) = spline(x, psi_grid(j,:), x_fine);&#xA;end&#xA;subplot(2,2,2)&#xA;s2 =surf(x_fine,z, psi_grid2);&#xA;set(s2,'linestyle','none')&#xA;xlabel('$x$' , 'FontSize', 14, 'Interpreter', 'latex');&#xA;ylabel('$z$', 'FontSize', 14, 'interpreter', 'latex','rot',0);&#xA;title('$Nx = 1500$')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15188" OwnerDisplayName="Amir Sagiv" LastEditorUserId="20688" LastEditDate="2017-04-16T16:40:12.933" LastActivityDate="2017-04-16T16:40:12.933" Title="How does Matlab `surf` perform interpolation?" Tags="|matlab|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24573" PostTypeId="1" CreationDate="2016-07-29T03:32:18.477" Score="2" ViewCount="100" Body="&lt;p&gt;I'd like to find &lt;em&gt;numerically&lt;/em&gt; a solution to a sparse system of 2000000 polynomial equations of degree 3 with 50000 variables and integer coefficients (or at least to decide whether or not a solution exists). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Question&lt;/em&gt;: Is there an algorithm and a supercomputer for which this computation is tractable today?&lt;/p&gt;&#xA;" OwnerUserId="9824" LastEditorUserId="9824" LastEditDate="2016-07-30T01:25:46.963" LastActivityDate="2016-07-30T01:25:46.963" Title="Is the numerical resolution of this huge sparse polynomial system tractable?" Tags="|numerical-analysis|complexity|numpy|polynomials|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24574" PostTypeId="2" ParentId="20030" CreationDate="2016-07-29T09:46:52.683" Score="0" Body="&lt;p&gt;One solution is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Create the bounding rectangle in $O(n)$ time&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Make &lt;strong&gt;M&lt;/strong&gt; artificial even distributed points inside this bounding rectangle, some &lt;strong&gt;M&lt;/strong&gt; are more difficult than others. In your case four in the corners of the rectangle and one in the center&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Build a KD-Tree of your &lt;strong&gt;N&lt;/strong&gt; input points. $O(n(log(n)))$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Query each of your &lt;strong&gt;M&lt;/strong&gt; points for nearest neighbor in the KD-Tree. $O(m(log(n)))$ time&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The whole algorithm should be correct and is bounded to $O(n(log(n)))$. The only tricky part is as said before the distribution of the artificial &lt;strong&gt;M&lt;/strong&gt; points. I assume larger prime numbers should be rather difficult. But I assume there are efficient algorithms for this. It is quite trivial for $\sqrt{M} \in \mathbb{N}$, as then only a grid needs to be generated and the points are the grid centers.   &lt;/p&gt;&#xA;" OwnerUserId="20162" LastEditorUserId="20162" LastEditDate="2016-07-29T15:41:43.900" LastActivityDate="2016-07-29T15:41:43.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24575" PostTypeId="1" AcceptedAnswerId="24576" CreationDate="2016-07-29T09:55:48.383" Score="4" ViewCount="211" Body="&lt;p&gt;I have a real valued function, let's call it $f(\mathbf{x}, \mathbf{y})$, which I would like to maximise with respect to $\mathbf{x}\in\mathrm{R}^d$ and minimise it with respect to $\mathbf{y}\in\mathrm{R}^q$. After a while I realised that I am looking for a solution that is a saddle point. I have no experience with such kind of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone inform me what algorithms there are for dealing with such problems? I am working in Julia, so in case anyone knows some implementation in Julia that would help me even further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; this was originally posted in the CrossValidated forum, but I was suggested to move it here instead.&lt;/p&gt;&#xA;" OwnerUserId="21177" LastEditorUserId="21177" LastEditDate="2016-08-05T10:16:45.473" LastActivityDate="2016-08-05T10:16:45.473" Title="Looking for saddle point in scalar function with multiple parameters" Tags="|optimization|julia|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24576" PostTypeId="2" ParentId="24575" CreationDate="2016-07-29T10:20:00.770" Score="4" Body="&lt;p&gt;That depends on whether $f$ is differentiable with respect to $x$ and $y$, and whether the function is convex/concave in $x$/$y$. In the simplest case, you can just write down the necessary optimality conditions&#xA;$$ \begin{pmatrix} \nabla_x f(\bar x,\bar y) \\ \nabla_y f(\bar x,\bar y) \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} $$&#xA;for the saddle point $(\bar x,\bar y)$, where $\nabla_x f(x,y)\in \mathbb{R}^d$ is the gradient with respect to $x$ etc., and apply a Newton method to that system of nonlinear equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could use an iterative method (variously called &lt;em&gt;ascent-descent&lt;/em&gt;, &lt;em&gt;Arrow--Hurwicz&lt;/em&gt; or &lt;em&gt;alternating directions&lt;/em&gt; method): Start with $x^0,y^0$ and set&#xA;$$ \begin{aligned} &#xA;x^{k+1} &amp;amp;= x^k + \alpha_k \nabla_x f(x^k,y^k)\\&#xA;y^{k+1} &amp;amp;= y^k - \alpha_k \nabla_y f(x^k,y^k)&#xA;\end{aligned} $$&#xA;for a suitable choice of step sizes $\alpha_k&amp;gt;0$. There are various versions that use $x^{k+1}$ in place of $x^k$ in the update for $y$ or (after reordering the iteration) vice versa, or include an extrapolation step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $f$ is not differentiable but convex/concave, similar approaches are possible by using proximal mappings instead of gradients; the currently most widely used approach for the special case $f(x,y) = g(x)+h(y)$ is known under the name &lt;em&gt;primal-dual hybrid gradient method&lt;/em&gt; (or often, after the authors of a paper that proposed it, &lt;em&gt;Chambolle--Pock method&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these are fairly straightforward to implement in Matlab (and hence easily ported to Python or Julia).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; I should point out that in contrast to nonlinear optimization, there's no general theory of finding saddle points of nonconvex differentiable functions (as far as I and Google know); all works I am familiar with either assume convexity/concavity or a very specific structure for $f$ (e.g., being the difference of convex functions or coming from the Lagrangian of a constrained optimization problem). The above is merely a description of the two rough classes of approaches used in these papers. &lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-07-29T17:20:38.007" LastActivityDate="2016-07-29T17:20:38.007" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="24577" PostTypeId="2" ParentId="16138" CreationDate="2016-07-29T12:18:44.360" Score="2" Body="&lt;p&gt;Six deflation methods for sparse PCA were discussed in &lt;a href=&quot;http://web.stanford.edu/~lmackey/papers/deflation-nips08.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hotelling's deflation and orthogonal Hotelling's deflation;&lt;/li&gt;&#xA;&lt;li&gt;Projection deflation and orthogonal projection deflation;&lt;/li&gt;&#xA;&lt;li&gt;Schur complement deflation;&lt;/li&gt;&#xA;&lt;li&gt;Reforming sparse PCA problem.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Personally I prefer the orthogonal projection deflation and it can be extended to the generalized eigenvalue problem. Consider the problem as,&#xA;$$&#xA;min_v v^TAv,s.t.v^TBv=1.&#xA;$$&#xA;At $t$-th iteration, obtain vector $q_t\propto(I-Q_{t-1}Q_{t-1}^TB)v_t$ where $q_t^TBq_t=1$ and $Q_{t-1}=[q_{1:(t-1)}]$. This can be thought as a result of the weighted &lt;a href=&quot;https://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process&quot; rel=&quot;nofollow&quot;&gt;Gram-Schmidt process&lt;/a&gt; from the set $\{v_t\}$. Then the matrix $A$ is updated as,&#xA;$$&#xA;A_{t+1}=(I-B^Tq_tq_t^T)A_t(I-q_tq_t^TB),&#xA;$$&#xA;where the matrix $B$ can be kept intact. It works fine for non-sparse generalized eigenvalue problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I further tested this for sparse generalized eigenvalue problems, using the algorithm in &lt;a href=&quot;https://arxiv.org/abs/1408.6686&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. The process mentioned here only guarantees that,&#xA;$ (v_t)^TA_sv_t=0, \forall{t&amp;lt;s} $. Thus $V^TBV\neq I$. The orthogonality property, unfortunately, has to be realized in other ways. &lt;/p&gt;&#xA;" OwnerUserId="21179" LastEditorUserId="21179" LastEditDate="2016-08-03T05:05:48.713" LastActivityDate="2016-08-03T05:05:48.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24578" PostTypeId="2" ParentId="24555" CreationDate="2016-07-29T17:35:33.337" Score="2" Body="&lt;p&gt;There should be little difference in principle as the underlying tool kits are usually similar if not the same: libraries, compilers (and hardware). In practice, there can be improvement from using 'thin' nodes. That is no GUI, no email running in the back ground or any of the other many processes an OS may have going to facilitate the user experience.  It can be easier to set up a 'server' os in linux straight from a standard distro that can result in a much leaner machine for running a floating point intensive code. Even then, the advantages would not be very noticeable as long as there is no user activity aside from the running code.&lt;/p&gt;&#xA;" OwnerUserId="3450" LastActivityDate="2016-07-29T17:35:33.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24579" PostTypeId="1" AcceptedAnswerId="24607" CreationDate="2016-07-29T17:47:06.947" Score="1" ViewCount="566" Body="&lt;p&gt;I am using the open-source CHLOMOD (as here &lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot; rel=&quot;nofollow&quot;&gt;http://faculty.cse.tamu.edu/davis/suitesparse.html&lt;/a&gt;) in order to solve a linear system Ax=b (performing A/b=x) in my domain decomposition code but I am unsure how to use it in my C serial code on a regular CPU rather than a GPU. Could someone give me an example of how to import this into some example  of code that solves for a system of equations? I am fairly new to C and am unsure how to do this. Thank you so much!&lt;/p&gt;&#xA;" OwnerUserId="20973" LastActivityDate="2016-08-02T03:28:31.590" Title="Linear solve using CHLOMOD in C" Tags="|matrix|sparse-matrix|c|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24580" PostTypeId="1" AcceptedAnswerId="24588" CreationDate="2016-07-29T19:12:13.497" Score="2" ViewCount="962" Body="&lt;p&gt;I find it kind of counter intuitive, that the result of an advection gets more smeared out at the borders when decreasing the timestep (which should make it more accurate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let there be a equally spaces grid $x_1, \dots, x_n$ with a constant 1D advection $v=1$. Let the initial values be &#xA;$$u^0=[0, 0, 1, 1, 0, 0]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the corresponding divergence field&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla\cdot (uv) = [0,0,1,0,-1,0]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a timestep of $\delta_t = 1$ we get for one timestep with the explicit euler forward time scheme and forward differences for the derivative:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u^1=[0, 0, 0, 1, 1, 0]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But with a smaller timestep $\delta_t=0.5$ we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u^{0.5}=\left[0,0,\frac{1}{2}, 1, \frac{1}{2}, 0\right]\\&#xA;u^1=\left[0,0,\frac{1}{4}, \frac{1}{2}, \frac{1}{2}, \frac{1}{4}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So while the CFL number requires a smaller timestep for stability, in this case a smaller number leads to unsharp boundaries and decreasing the $\delta_x$ helps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some other stablity condition than CFL and Peclet numbers for this problem?&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="14700" LastEditDate="2016-07-29T21:28:27.763" LastActivityDate="2022-01-11T16:33:03.340" Title="More Smearing with decreasing timestep in advection problems" Tags="|stability|advection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24582" PostTypeId="1" AcceptedAnswerId="24599" CreationDate="2016-07-29T21:38:29.787" Score="3" ViewCount="1445" Body="&lt;p&gt;Given a $M\times M\times M$ 3D cube grid $C$, how can I map the 3D grid-coordinates to 1D coordinates such that &lt;strong&gt;every&lt;/strong&gt; cell's nearest neighbors can be represented by a continuous range of corresponding 1D values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the 3D-to-1D mappings could be Z-indexing, but I don't know how to access the neighbors in the way I described above.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why do I need this:&lt;/strong&gt; I'm writing an OpenCL kernel function, that processes particles. In each cell of the grid $C$ can be arbitrary (non-negative :) number of particles. The particles are stored in memory sorted by their cell's id given by the 1D coordinates. And to get the best GPU performance when reading neighbors, I need the neighboring particles to be close in memory (i.e. the 1D coordinates of a grid cells that are close to each other in space are also close).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want more of the context, you can see my questions at different SE sites:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/38646131/nearest-neighbors-in-cuda-particles&quot;&gt;Nearest Neighbors in CUDA particles&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gamedev.stackexchange.com/questions/126269/z-order-curve-how-to-query-all-nearest-neighbors&quot;&gt;Z-order curve — how to query all nearest neighbors?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21183" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-08-01T19:26:22.657" Title="Nearest Neighbors in radius via Z-order order curve?" Tags="|algorithms|opencl|nearest-neighbors|spatial-data|particle|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24583" PostTypeId="1" CreationDate="2016-07-30T03:06:19.653" Score="2" ViewCount="111" Body="&lt;p&gt;There are lots of higher order time splitting method as shown by the &lt;a href=&quot;http://www.asc.tuwien.ac.at/~winfried/splitting/&quot; rel=&quot;nofollow&quot;&gt;list &lt;/a&gt; with real and complex coefficients $a_i, b_i, c_i$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;[e^{c_s \Delta t \hat C}] e^{b_s \Delta t \hat B} e^{a_s \Delta t \hat A} ...   [e^{c_1 \Delta t \hat C}] e^{b_1 \Delta t \hat B} e^{a_1 \Delta t \hat A} u&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not clear which one should I choose, so are there any advantage of complex coefficients over real coefficients? Also, which method is &lt;em&gt;better&lt;/em&gt; in practices. Intuitively, the least step $s$ and real coefficients seems an easy choice.&lt;/p&gt;&#xA;" OwnerUserId="3444" LastEditorUserId="3444" LastEditDate="2016-07-30T03:17:57.437" LastActivityDate="2016-07-30T07:55:03.513" Title="High order time splitting methods" Tags="|pde|nonlinear-equations|time-integration|operator-splitting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24584" PostTypeId="2" ParentId="24567" CreationDate="2016-07-30T05:08:21.707" Score="1" Body="&lt;p&gt;You can distort the elliposid uniformly by applying a linear map ($\mathbf{F}$, a 3x3 matrix in 3D) such that&#xA;$$&#xA;  \mathbf{x}_{\text{distorted}} = \mathbf{F} \cdot \mathbf{x}&#xA;$$&#xA;If the map depends on position ($\mathbf{x}$) you can get more complex deformations.  You could search for such maps in the literature but it will probably be easier to find what you want by trial and error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more physically based, but computationally tractable approaches, see the paper &quot;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2726347/&quot; rel=&quot;nofollow&quot;&gt;Morphology and interaction between lipid domains&lt;/a&gt;&quot; by Ursell et al., PNAS,  2009 Aug 11; 106(32): 13301–13306.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2016-07-30T05:08:21.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24585" PostTypeId="2" ParentId="24583" CreationDate="2016-07-30T07:55:03.513" Score="3" Body="&lt;p&gt;The design of the &quot;best&quot; splitting schemes is discussed and investigated at length in &lt;a href=&quot;http://arxiv.org/abs/1604.01179&quot; rel=&quot;nofollow&quot;&gt;this recent paper&lt;/a&gt; (disclosure: I am one of its authors).  In short, the most commonly used criterion is the size of the leading truncation error term coefficients.  This can generally be made smaller by using a larger number of stages, and the tradeoff can be worthwhile if your step size is limited by accuracy considerations in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any inherent &lt;em&gt;advantage&lt;/em&gt; to methods with complex coefficients; generally that is a disadvantage since you have to do complex arithmetic which requires more operations.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-07-30T07:55:03.513" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24586" PostTypeId="1" AcceptedAnswerId="24587" CreationDate="2016-07-30T12:30:54.413" Score="2" ViewCount="2211" Body="&lt;p&gt;I am not sure if my question is on topic or not and if not please let me know. I have regularly spaced gridded data(output of  a weather forecast simulation software) and I have latitude and longitudes as 2D arrays along with geopotential height(2D array). The data is in netCDF  or GRIB format. Ultimately  I am looking to make a contour plot of the heights data and I am looking to use Python's scipy and matplotlib package. I initially need to do an interpolation of the various data points. From the scipy package what would be the best interpolation function that I should choose for my use case which has structured grid data  ?&#xA;Is something like the linked API ideally suited for me ? Initially I went ahead and tried &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.griddata.html&quot; rel=&quot;nofollow&quot;&gt;griddata &lt;/a&gt; but this seems for unstructured grid data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.interpolate.RegularGridInterpolator.html&quot; rel=&quot;nofollow&quot;&gt;Regular Grid Interpolation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user16555" LastActivityDate="2016-07-30T13:01:35.360" Title="Contour plot interpolation recommendation" Tags="|python|interpolation|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24587" PostTypeId="2" ParentId="24586" CreationDate="2016-07-30T13:01:35.360" Score="1" Body="&lt;p&gt;If your data are regularly spaced, you do not need an interpolation procedure, you can directly plot a contour through a &lt;code&gt;contour&lt;/code&gt; or a &lt;code&gt;pcolor&lt;/code&gt; command. That why I don't understand why you need to initially interpolate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Contouring algorithms often use marching cube method (and its variations) to render a contour plot. This kind of algorithm only work with a structured grid. For an unstructured grid, routines likes &lt;code&gt;griddata&lt;/code&gt; perform a Delaunay triangulation or similar and indeed do an interpolation to recover data on an structured grid where the contouring algorithm can be applied.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-07-30T13:01:35.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24588" PostTypeId="2" ParentId="24580" CreationDate="2016-07-30T14:41:28.483" Score="7" Body="&lt;p&gt;You solve the 1D-advection equation with &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; a constant velocity:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial u}{\partial t} + c\frac{\partial u}{\partial x}=0~~~~~~~~(1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;When you discretize this equation (with an explicit scheme in time and an upwind scheme for a positive velocity &lt;span class=&quot;math-container&quot;&gt;$c \geq 0$&lt;/span&gt; in space for instance), you get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{u_i^{n+1} - u_i^n}{\Delta t}+c\frac{u_i^n-u_{i-1}^{n}}{\Delta x} = 0 ~~~~~~~~(2)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;BUT&lt;/strong&gt; here you &lt;strong&gt;approximate&lt;/strong&gt; your solution and by doing so you commit an error, called &lt;strong&gt;the truncation error&lt;/strong&gt;. Let's find it :&lt;/p&gt;&#xA;&lt;p&gt;The key idea is to use the Taylor expansion for each of your discretized quantities:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u_i^{n+1} = u_i^n + \Delta t \frac{\partial u}{\partial t}+\frac{\Delta t^2}{2}\frac{\partial^2 u }{\partial t^2}+O(\Delta t^3)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u_{i-1}^{n} = u_{i}^n - \Delta x \frac{\partial u}{\partial x}+\frac{\Delta x^2}{2}\frac{\partial^2 u }{\partial x^2}+O(\Delta x^3)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then you plug those expressions into (2). You should find:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial u}{\partial t} + c\frac{\partial u}{\partial x} + \frac{1}{2}\Delta t \frac{\partial^2 u }{\partial t^2} - c \frac{\Delta x}{2} \frac{\partial^2 u }{\partial x^2} = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;You recognize the first two terms, it is your equation (1) and by putting the rest in the right-hand size, you find that this equation is now equal to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial u}{\partial t} + c\frac{\partial u}{\partial x}= Err$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$$Err = - \frac{1}{2}\Delta t \frac{\partial^2 u }{\partial t^2} + c \frac{\Delta x}{2} \frac{\partial^2 u }{\partial x^2} ~~~~~~~(3)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is &lt;strong&gt;NOT&lt;/strong&gt; equation (1). Let's simplify a little this expression. You use the fact that from (1), you derive in time:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial^2 u}{\partial t^2} + c\frac{\partial^2 u}{\partial t\partial x}=0$$&lt;/span&gt;&#xA;Omitting mathematical arguments, you can switch partial derivatives and by using (1) again, you get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial^2 u}{\partial t^2} = c^2\frac{\partial^2 u}{\partial x^2} ~~~~~(4)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Plug (4) into (3) and by introducing the CFL number : &lt;span class=&quot;math-container&quot;&gt;$$CFL=\frac{c\Delta t}{\Delta x}$$&lt;/span&gt; you should end up with:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Err = \frac{c\Delta x}{2}\frac{\partial^2 u}{\partial x^2}(1-CFL)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the error you commit by approximating derivatives. The truncation error depends on the second derivative in space which is called a &lt;strong&gt;diffusion&lt;/strong&gt; term, hence a &lt;strong&gt;smearing&lt;/strong&gt; effect (think about the heat equation). Now you understand that when your CFL tends to 1 (that is to say a larger &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; fixed), the error vanishes and so you solve exactly (1), that is your first result. On the other hand, when your CFL decreases (small &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt;), the error gets larger and your solution is &lt;strong&gt;smeared&lt;/strong&gt;, as you have experienced in your second example. Indeed, decreasing &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; helps because by doing so you increase your CFL even if your time step is fixed by your stability condition. You recover the fact the results get better with a refined mesh, fortunately !&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="36438" LastEditDate="2022-01-11T16:33:03.340" LastActivityDate="2022-01-11T16:33:03.340" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="24589" PostTypeId="2" ParentId="24537" CreationDate="2016-07-30T20:40:36.713" Score="2" Body="&lt;p&gt;Seems like the experts are not answering your question so I will try to provide an idea. But before I do that I strongly suggest that you look up in the literature for some sophisticated methods that have been already developed. However, without guaranteeing that this is a good or fast or efficient suggestion, I propose the following methodology. Keep in mind, I may have made some mistakes, so I do not guarantee that everything is fully correct, but I hope the idea of the method gives you some approach that will help you solve your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $V$ be the set of your points in the whole &quot;big&quot; cube. Fix your &quot;small&quot; cube $C$ somewhere in the big cube and let $ V_C$ be the set of points that are contained in $C$, i.e. $V_C = V \cap C.$ Initially set $V'_C=V_C$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1:&lt;/strong&gt; Generate the Voronoi diagram $Vor(V'_C)$. For each point $v \in V'_C$ denote by $Vor(v)$ its Voronoi cell, which is a convex polyhedron in three-space. Furthermore, denote by $W(v)$ the vertices of the Voronoi cell centered at $v \in V'_C$ and by $W(V'_C) = \cup_{v \in V'_C} W(v)$ the vertices of all Voronoi&#xA;cells from the Voronoi diagram $Vor(V'_C)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2:&lt;/strong&gt; Color all points from $V'_C$ and all Voronoi vertices $W(V'_C)$ white.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 3:&lt;/strong&gt; For each Voronoi vertex $w \in W(V'_C)$ draw the Delaunay sphere centered at $w$, that is the sphere with center $w$ and radius the distance between $w$ and one of the points from $V'_C$ whose Voronoi cell has $w$ as a vertex (it doesn't matter which point, there are several but the result is always the same).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Case 3.1.&lt;/em&gt; If the Delaunay sphere of $w$ is contained in the cube $C$, color $w$ black.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Case 3.2.&lt;/em&gt; If the Delaunay sphere is not contained in the cube $C$ but it doesn't contain any point from $V$ in its (open) interior, color the point $w$ black.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Case 3.3.&lt;/em&gt; If the Delaunay sphere of $w$ contain points from $V$ in its (open) interior, (1) add the points from $V$ contained in the interior of the sphere to the set $V'_C$ and (2) keep the color of the point $w$ white.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 4:&lt;/strong&gt; For each point $v \in V'_C$ check if all Voronoi vertices $W(v)$ of its Vornoi cell are black. If not all of them are black, keep the color of $v$ white. If they are black, color $v$ black.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 5:&lt;/strong&gt; Check if all points of the original set $V_C$ are black.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Case 5.1.&lt;/em&gt; If they are all black, the Voronoi diagram $Vor(V'_C)$ restricted to the cube $C$ is the local portion of the global Voronoi diagram $Vor(V)$ restricted to $C$. End.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Case 5.2.&lt;/em&gt; If there are white vertices in $V_C$, then go back to Step 1. In Step 1, when generating the new Voronoi diagram $Vor(V'_C)$, one keeps the Voronoi cells around black points from $V'_C$ the same, keeps all black Voronoi vertices from $W(V'_C)$ and makes alteration only in relation to the white ones.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="20964" LastActivityDate="2016-07-30T20:40:36.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24590" PostTypeId="1" CreationDate="2016-07-31T10:51:24.200" Score="1" ViewCount="49" Body="&lt;p&gt;I have discrete time series emerging from the numerical simulations.&#xA;It means that the time series can be slightly noisy.&#xA;The time series should &quot;obey&quot; to the following formula:&#xA;$$ \psi(t) = \sum_{i=1}^{N} a_i \exp(\alpha_i t) \sin (\omega_i t + \phi_i) $$&#xA;where $N$ is not known in advance (it can be 1, 2, or 20) depending on a given simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE 2016-08-05: Unknown parameters here are $a_i$, $\alpha_i$, $\omega_i$, $\phi_i$, and $N$, although the only thing I need is to extract $\alpha_i$ and $\omega_i$ from these time series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previously, when $N=1$ was guaranteed, I used nonlinear least-squares algorithm. It turned out that this algorithm requires an extremely good initial guess to find a right fit.&#xA;I'm not sure how to provide a good initial guess when $N$ is not known in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most robust and best way to extract parameters I need from these time series?&#xA;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE 2016-08-05: The following is the code that demonstrates an example time series. This example is purely artificial, though, as I don't have a real time series emerging from numerical simulations right now.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;alpha(1) = 0.03;&#xA;alpha(2) = 0.05;&#xA;alpha(3) = -0.01;&#xA;&#xA;a(1) = 1e-5;&#xA;a(2) = 1e-5;&#xA;a(3) = 1e-5;&#xA;&#xA;omega(1) = 0.9;&#xA;omega(2) = 0.125;&#xA;omega(3) = 0.001;&#xA;&#xA;phi(1) = pi/2;&#xA;phi(2) = pi/3;&#xA;phi(3) = pi;&#xA;&#xA;t = linspace(0, 100, 1000);&#xA;d = 0 * t;&#xA;&#xA;for i = 1:3&#xA;    d = d + a(i) * exp(alpha(i)*t) .* sin(omega(i)*t + phi(i)); &#xA;end&#xA;&#xA;plot(t, d, 'o-');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5lTeh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5lTeh.png&quot; alt=&quot;Example time series&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3254" LastEditorUserId="3254" LastEditDate="2016-08-05T12:30:42.760" LastActivityDate="2016-08-05T12:30:42.760" Title="Best way to to find fitting parameters for time series of decaying-growing oscillator type" Tags="|regression|signal-processing|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24591" PostTypeId="1" CreationDate="2016-07-31T19:06:42.693" Score="3" ViewCount="328" Body="&lt;p&gt;I'm currently trying to approximate the following type of &lt;strong&gt;wave equation&lt;/strong&gt; (in weak formulation):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\Omega \subset \mathbb{R}^d$ ($d=2$) be some polygonal domain. We seek a function $u \in L^2\left(0, T; H^1_0(\Omega)\right)$ with $u' \in L^2\left(0, T; L^2(\Omega)\right)$, $u'' \in L^2\left(0, T; H^{-1}(\Omega)\right)$ and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{equation}&#xA; \langle u''(t), v \rangle_{H^{-1}(\Omega) \times H^1_0(\Omega)} + \langle \nabla u(t), \nabla v\rangle_{L^2(\Omega)} = \langle f(t), v\rangle_{L^2(\Omega)},&#xA;\end{equation}&#xA;$$&#xA;for almost every $t \in (0,T)$ and all $v \in H^1_0(\Omega)$ and with initial conditions&#xA;$$&#xA;\begin{equation}&#xA; u(0) = u_0, \ \ u'(0)=v_0,&#xA;\end{equation}&#xA;$$&#xA;with data $f \in L^2\left(0, T; L^2(\Omega)\right)$, $u_0 \in H^1_0(\Omega)$ and $v_0 \in L^2(\Omega)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discretization&lt;/strong&gt;: First, we just discretize the problem in space by using a finite element space $V_h$ of continuous, piecewise-linear polynomials and then use an (explicit) centred difference quotient of second order for the time derivative &quot;$u''(t)$&quot; with time step size $\Delta t = \frac{T}{M}$, $M \in \mathbb{N}$, and end up with a discrete scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation} \label{leapfrogscheme}&#xA;	\left\langle \frac{\left( u_h^{n-1} - 2 u_h^{n} + u_h^{n+1} \right)}{(\Delta t)^2}, v_h \right\rangle_{L^2} + \langle \nabla u_h^{n}, \nabla v_h \rangle_{L^2} = \langle f^n, v_h \rangle_{L2},&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all $v_h \in V_h$ and with given start values $u_h^{0}$ and $u_h^{1}$, where the approximation of the Galerkin solution $u_h$ at time $t_n = n \Delta t$ is denoted by $u_h^n$ and we define $f^n := f(t_n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As start values, we take for instance the $L^2$-projection (for all $v_h \in V_h$):&#xA;\begin{align} &#xA;	\langle u_h^0, v_h \rangle_{L^2} &amp;amp;:= \langle u_0 , v_h \rangle_{L^2}, \\&#xA;	\langle u_h^1, v_h \rangle_{L^2} &amp;amp;:= \left\langle u_0 + (\Delta t) v_0 + \frac{(\Delta t)^2}{2}\left( f^{0} + \Delta u_h^0 \right), v_h \right\rangle_{L^2},&#xA;\end{align}&#xA;where we used the second order Taylor expansion&#xA;\begin{align}&#xA;	u_h(\Delta t) \approx u_h(0) + (\Delta t) u_h'(0) + \frac{(\Delta t)^2}{2} u_h''(0)&#xA;\end{align}&#xA;in time $t=0$ to define $u_h^{1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Error analysis&lt;/strong&gt;: Here comes the part i'm having some trouble with, because i haven't found any good literature that covers this topic. &lt;br&gt;&#xA;I basically want to figure out, what kind of convergence rate in a norm i can optimally expect (given the solution is smooth enough) and which discrete norm i should take to verify that my computed solutions resolve this rate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0713048&quot; rel=&quot;nofollow&quot;&gt;Garth A. Baker - &quot;Error Estimates for Finite Element Methods for Second Order Hyperbolic Equations&quot;&lt;/a&gt;, i found that, if additionally $u \in L^\infty\left(0, T; H^2(\Omega)\right)$, $u' \in L^2\left(0, T; H^2(\Omega)\right)$ and the third and fourth time derivative of $u$ exists and is in $L^2\left(0, T; H^2(\Omega)\right)$, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; \max_{n} \| u(t_n) - u_h^{n} \|_{L^2(\Omega)} \leq C \, ( h^2 + \Delta t^2 ),&#xA;\end{equation}&#xA;where $h&amp;gt;0$ is the spatial mesh size of an uniform mesh that defines the finite element space $V_h$, i.e. all elements $T$  are of size $\mathrm{diam}(T) \approx h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For stability reasons, one has to take a sufficient small time step length $\Delta t \approx h_{\min}$ (CFL-condition). So, if we replace $\Delta t$ by $h$ and rewrite (in dimension d=2) $h = N^{-\frac{1}{2}}$, where $N$ denotes the degrees of freedom (here: number of inner nodes of the finite element mesh), then one should hopefully see an optimal error of &#xA;\begin{equation}&#xA; \max_{n} \| u(t_n) - u_h^{n} \|_{L^2(\Omega)} \leq C \, N^{-1}.&#xA;\end{equation}&#xA;So, this is basically an error analysis in the $L^\infty\left(0, T; L^2(\Omega)\right)$-norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this - so far - how people would measure the convergence of such a hyperbolic problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now it gets a little bit more complicated by considering domains with reentrant corners, so that we can only expect the solution to be in $u(t) \in H^{1+\delta}(\Omega)$, for some $0 &amp;lt; \delta &amp;lt; 1$. This would damp the $L^2$-error of $u(t) - u_h(t)$ for a fix $t \in [0,T]$ down to $\mathcal{O}\left(h^{1+\delta}\right)$ instead of $\mathcal{O}\left(h^{2}\right)$ (i.e. down to $\mathcal{O}(N^{-\frac{1+\delta}{2}}$) if we would use an uniform mesh of size $h$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get around this drawback, i would like to use local refined meshes to still get the optimal $L^2$-error of $\mathcal{O}\left(N^{-1}\right)$ for a fix $t$. The problem now is that i still have to ensure stability by taking $\Delta t \approx h_{\min}$ - this depends on the smallest element diameter, which decreases drastically for locally refined meshes. So, the computational work we put into the time discretization isn't really comparable anymore with our number of interior nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would someone measure the convergence for this kind of discretization? I would also be grateful for some literature in general that considers the error analysis of the wave equation by using finite elements.&lt;/p&gt;&#xA;" OwnerUserId="10005" LastActivityDate="2016-08-02T00:25:01.927" Title="Measure the convergence rate of a discretization of a wave equation" Tags="|finite-element|convergence|hyperbolic-pde|wave-propagation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24592" PostTypeId="2" ParentId="20786" CreationDate="2016-07-31T19:09:53.113" Score="3" Body="&lt;p&gt;Take a look at &lt;a href=&quot;http://headmyshoulder.github.io/odeint-v2/doc/boost_numeric_odeint/getting_started/overview.html&quot; rel=&quot;nofollow&quot;&gt;Odeint&lt;/a&gt;. It is now part of Boost and it includes the&#xA;Bulirsch-Stoer algorithm among others. To start you can see &lt;a href=&quot;http://headmyshoulder.github.io/odeint-v2/examples.html#simple1dexample&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; a very simple example.&lt;/p&gt;&#xA;" OwnerUserId="21198" LastActivityDate="2016-07-31T19:09:53.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24593" PostTypeId="2" ParentId="20786" CreationDate="2016-08-01T10:18:07.077" Score="1" Body="&lt;p&gt;I have had success with the &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/Cubature&quot; rel=&quot;nofollow&quot;&gt;Cubature library&lt;/a&gt; (it is written in C, though). It is aimed at multidimensional integration with a relatively low number of dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://mint.sbg.ac.at/HIntLib/&quot; rel=&quot;nofollow&quot;&gt;HIntLib library&lt;/a&gt; is written in C++ and it has routines for adaptive quadrature (cubature). &lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2016-08-01T10:18:07.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24595" PostTypeId="1" AcceptedAnswerId="24601" CreationDate="2016-08-01T12:36:22.607" Score="0" ViewCount="201" Body="&lt;p&gt;I am given 2 unstructured (i.e. do not posses any special sparsity pattern like banded/triangular/etc.) sparse matrices $A$ and $B$ of dimension ($n$ x $n$) and density $d$ each (thus each matrix contains $n^2 d$ non zero elements).&lt;br&gt;&#xA;In terms of probability, for any element $A(i,j)$,&lt;br&gt;&#xA;$p (A[i,j] \neq 0) = d$ and independent of all other elements of $A$.&lt;br&gt;&#xA;Similar probability distribution occurs for $B$.&lt;br&gt;&#xA;I wanted to know if it is possible to calculate the expected density of $C = AB$ in terms of $n$ and $d$ i.e. the distribution $p( C[i,j] \neq 0 )$?&lt;/p&gt;&#xA;" OwnerUserId="21204" LastEditorUserId="21204" LastEditDate="2016-08-01T18:10:33.357" LastActivityDate="2016-08-01T21:21:52.427" Title="Multiplication of random sparse matrices" Tags="|sparse-matrix|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24596" PostTypeId="1" CreationDate="2016-08-01T13:40:53.593" Score="1" ViewCount="447" Body="&lt;p&gt;I am trying to find some package of Discontinuous Galerkin (DG) method for solving hyperbolic and parabolic equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my research, I focus on designing new schemes for some very simple equations and boundary condition and doing numerical analysis. Thus, I need the package which is easy for me to do possible modification in algorithm. Moreover, python is better. I am not familiar with C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone tell me what's a suitable package for me?&lt;/p&gt;&#xA;" OwnerUserId="20711" LastEditorUserId="36438" LastEditDate="2022-10-25T21:50:08.010" LastActivityDate="2022-10-26T12:14:59.020" Title="Package for Discontinuous Galerkin method" Tags="|libraries|discontinuous-galerkin|software-recommendation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24597" PostTypeId="2" ParentId="24555" CreationDate="2016-08-01T14:20:53.223" Score="3" Body="&lt;p&gt;I remember reading two or three years ago statistics about OS distribution for the List of the 500 fastest computers world wide. Linux/unix based running computer number was 498, Windows based 2 and no MacOS. Unfortunately I failed to get to the statistics again. &lt;a href=&quot;https://en.wikipedia.org/wiki/Supercomputer&quot; rel=&quot;nofollow&quot; title=&quot;wikipedia&quot;&gt;Wikipedia&lt;/a&gt;'s SuperComputer article just states:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Although most modern supercomputers use the Linux operating system, each manufacturer has its own specific Linux-derivative, and no industry standard exists, partly due to the fact that the differences in hardware architectures require changes to optimize the operating system to each hardware design&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if this article does not give the numbers of the distribútion, it gives at least a clear trend to Linux based computers. In a field where every bit of performance counts (and the money needed to adapt the system to the super computer specification most likely does not, so I do not assume the the open source nature is the major benefit here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a rather normal workstation Linux tends to also have better resource management, e.g. less memory usage, and tends to be more stable. Yet this is a personal experience of mine wihout statistics behind it and should not be generalized without confirmation (Win 7 vs Ubuntu 14.04).&lt;/p&gt;&#xA;" OwnerUserId="20162" LastActivityDate="2016-08-01T14:20:53.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24598" PostTypeId="2" ParentId="24200" CreationDate="2016-08-01T15:06:25.460" Score="4" Body="&lt;p&gt;You want to calculate $$c(t)=\int_{-6}^6dx f(x)g(t-x)$$. Your functions are not periodic, but you pad them with zeros so that assuming them to be repeated periodically does not mess up the calculation of the convolution. You decided to pad them with zeros from $6$ to $18$. So you now want to calculate $$c(t)=\int_{-6}^{18}dx f(x)g(t-x)$$ by discrete Fourier transform on the interval from $-6$ to $18$. This means that you need to sample your functions at $N$ equally-spaced points $x_j=j\Delta x - 6$, $j=0,\dots N-1$, where $\Delta x=24/N$. Note the shift by $-6$ in the definition of the $x_j$ that is necessary because your interval starts at $-6$. We can now approximate the integral  by a sum:&#xA;$$c(t)\approx\Delta x\sum_{j=0}^{N-1}f(x_j)g(t-x_j).$$&#xA;Notice the factor of $\Delta x$ that you forgot in your calculation and that explains the wrong amplitude you found. Next we consider only a discrete set of&#xA;$t$s so that $t-x_j$ is also one of the discrete points at which you have sampled your functions. It is convenient to set $t=x_n-6$ for some $n$, because then $t-x_j=x_n-6-x_j=x_{n-j}$ and the continuous convolution you want to calculate is approximated by a discrete convolution of the form that can be calculated by discrete Fourier transform &#xA;$$c(x_n-6)\approx\Delta x\sum_{j=0}^{N-1}f(x_j)g(x_{n-j}).$$&#xA;It is this shift by $-6$ that explains why the graphs do not line up in your figure.&lt;/p&gt;&#xA;" OwnerUserId="21195" LastEditorUserId="21195" LastEditDate="2016-08-02T06:31:36.533" LastActivityDate="2016-08-02T06:31:36.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24599" PostTypeId="2" ParentId="24582" CreationDate="2016-08-01T19:26:22.657" Score="5" Body="&lt;p&gt;What you are asking is not possible, because you cannot order all cells in such a way that neighbors are always in a continuous range. Suppose we try to construct continuous neighbor lists in 3D. Each interior cell $C_i$ has six neighbors, so it should appear six times in a neighbor list. The only way to construct six &lt;em&gt;continuous&lt;/em&gt; neighbor list is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$C_{i-5}$ to $C_{i}$&lt;/li&gt;&#xA;&lt;li&gt;$C_{i-4}$ to $C_{i+1}$&lt;/li&gt;&#xA;&lt;li&gt;..&lt;/li&gt;&#xA;&lt;li&gt;$C_{i}$ to $C_{i+5}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These lists contain 11 different cells. However, the left and right $x$-neighbor of $C_i$ already have $5+5+1=11$ different cells in their neighbor list (their only shared neighbor is $C_i$). Because there are also $y$ and $z$-neighbors, we cannot have continuous neighbor lists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could instead compute a Morton number $m_i$ for each cell (i.e., use the Z-curve), and then store these numbers in a sorted 1D array. Given any cell, you can compute the Morton number of a neighbor in time $O(1)$, and do a binary search in time $O(\log N)$, where $N$ is the number of cells.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2016-08-01T19:26:22.657" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24600" PostTypeId="2" ParentId="24537" CreationDate="2016-08-01T19:53:05.300" Score="1" Body="&lt;p&gt;I understand your question as: I want to draw Voronoi diagram for a subset of points such that it is same as the one which is obtained when considering the complete set of points. &#xA;Voronoi diagrams are drawn by first joining neighboring points and then drawing a plane perpendicular to the line at the midpoint. You do this for all the nearest neighbors and you have a voronoi diagram in the neighborhood of a point. Do this for all the points and you have voronoi diagram for all the points. You see, voronoi diagrams are defined locally. There is no second nearest neighbor or third nearest neighbor effect. Only first nearest neighbor effect. So all you have to do to get a voronoi diagram with a subset of point is to identify the points in the subregion of interest, connect them to all of their respective nearest neighbors, and draw a plane passing through midpoints of these line segment and perpendicular to the line segment. This diagram will be same for a local region whether you consider a sub region or complete region. &lt;/p&gt;&#xA;" OwnerUserId="3888" LastEditorUserId="3888" LastEditDate="2016-08-02T07:00:45.713" LastActivityDate="2016-08-02T07:00:45.713" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24601" PostTypeId="2" ParentId="24595" CreationDate="2016-08-01T21:21:52.427" Score="3" Body="&lt;p&gt;I took a go at this and hopefully I am able to write this properly to articulate the logic. In this logic, I am assuming summation of terms from $A$ and $B$ used to compute a term in $C$ can't cancel out to equal $0$ and that instead $C$ only has zero terms based on the sparseness of $A$ and $B$. Please let me know if you spot any errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;p\left( \left\{C(i,j) \neq 0\right\} \right) &amp;amp;= p\left( \bigcup_{k=1}^{n} \left\{ A(i,k)B(k,j) \neq 0 \right\} \right)\\&#xA;&amp;amp;= 1 - p\left( \bigcap_{k=1}^{n} \left\{ A(i,k)B(k,j) \neq 0 \right\}^c \right)\\&#xA;&amp;amp;= 1 - \prod_{k=1}^n p\left( \left\{ A(i,k)B(k,j) \neq 0 \right\}^c \right)\\&#xA;&amp;amp;= 1 - \prod_{k=1}^n \left( 1 - p\left( \left\{ A(i,k)B(k,j) \neq 0 \right\} \right) \right)\\&#xA;&amp;amp;= 1 - \prod_{k=1}^n \left( 1 - p\left( \left\{ \left\{A(i,k) \neq 0 \right\} \bigcap \left\{ B(k,j) \neq 0 \right\} \right\} \right) \right)\\&#xA;&amp;amp;= 1 - \prod_{k=1}^n \left( 1 - p\left( \left\{A(i,k) \neq 0 \right\} \right) p\left(\left\{ B(k,j) \neq 0 \right\} \right) \right)\\&#xA;&amp;amp;= 1 - \prod_{k=1}^n \left( 1 - d^2 \right)\\&#xA;&amp;amp;= 1 - \left( 1 - d^2 \right)^n\\&#xA;&amp;amp;\approx nd^2\;\;\;\;\; \forall d^2 \ll 1&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-08-01T21:21:52.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24602" PostTypeId="1" AcceptedAnswerId="24603" CreationDate="2016-08-01T21:58:50.187" Score="6" ViewCount="675" Body="&lt;p&gt;I was reading about TR methods and there are some terms, which are confusing for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says, method is globaly convergent. What does it really mean? Converges to global minima, or converges for sure to local minima, if there is any? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks for clarification!&lt;/p&gt;&#xA;" OwnerUserId="17150" LastEditorUserId="17150" LastEditDate="2016-08-02T11:30:28.683" LastActivityDate="2016-08-02T11:30:28.683" Title="Global convergence in trust region algorithm" Tags="|optimization|convergence|nonconvex|trust-regions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24603" PostTypeId="2" ParentId="24602" CreationDate="2016-08-01T22:19:42.120" Score="7" Body="&lt;p&gt;Standard terminology in nonlinear (i.e., derivative-based) optimization is that &quot;global convergence&quot; means &quot;convergence to a stationary point no matter where you start from&quot; (where the limit may in fact depend on the starting point). This is in contrast to &lt;em&gt;local&lt;/em&gt; convergence, which requires that you start sufficiently close to a stationary point; if your initial point is too far away, the method need not converge at all. This is famously the case for Newton's method, which on the other hand (for nice enough functions) converges quadratically (i.e., &lt;em&gt;much&lt;/em&gt; faster than steepest descent). Much of nonlinear optimization is concerned with trying to get rid of this requirement of a good enough initial guess (also called &quot;globalization&quot;, of which trust region methods are one approach) without destroying the fast convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is that if you are using algorithms (only) based on derivatives,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;it is impossible to distinguish local from global minimizers (since derivatives only carry local information), and&lt;/li&gt;&#xA;&lt;li&gt;as soon as you hit a stationary point, the derivative by definition vanishes and hence the algorithm must terminate.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(Another reason for the terminology is that for a large class of practically relevant problems (convex problems), any stationary point is in fact a (global!) minimizer.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, &lt;em&gt;global optimization&lt;/em&gt; is concerned with actually computing global minimizers (for which, &lt;a href=&quot;https://en.wikipedia.org/wiki/There_ain%27t_no_such_thing_as_a_free_lunch&quot;&gt;TANSTAAFL&lt;/a&gt;, there is much less convergence theory).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-08-02T06:50:41.497" LastActivityDate="2016-08-02T06:50:41.497" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24604" PostTypeId="2" ParentId="24537" CreationDate="2016-08-01T23:01:10.900" Score="1" Body="&lt;p&gt;The simplest way of doing this is to surround your iner box with a bigger box that contains at least all the nearest neighbours of the points within your inner box.  Note that there will be an issue arising when the inner box is close to the edge of the encompassing data box: you have no external points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculating a Voronoi/Delaunay tessellation may be more subtle than you might think.  One of the issues is how to decide precisely whether a point is on one side or other of a tessellation plane/line.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is the very complete &quot;CGAL&quot; C++ library for doing this at &lt;a href=&quot;http://www.cgal.org/&quot; rel=&quot;nofollow&quot;&gt;http://www.cgal.org/&lt;/a&gt;.  My colleagues and I have used this in several published papers in astrophysics: it appears to be rock-solid in addressing all the potential pitfalls in creating these tessellations.&lt;/p&gt;&#xA;" OwnerUserId="21129" LastActivityDate="2016-08-01T23:01:10.900" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24605" PostTypeId="2" ParentId="24570" CreationDate="2016-08-01T23:44:22.000" Score="1" Body="&lt;p&gt;If the two terms in your integral have substantially different behavior, the better approach is likely to split the integral into two separate integrals. This way, you can choose parameters for each integrand separately.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-01T23:44:22.000" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24606" PostTypeId="2" ParentId="24591" CreationDate="2016-08-02T00:25:01.927" Score="2" Body="&lt;p&gt;You are asking very complicated questions for which there are likely no answers that can be rigorously proven.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you go back for a second and ask the same question for the solution of the Laplace equation, then you are asking whether, given that $u\in H^{1+\delta}$, you can find an algorithm that constructs a mesh with $N$ cells so that $\|u-u_h\|_{L_2} = {\cal O}(N^{-\frac{1+\delta}{2}})$ in 2d. I don't know whether that's actually true for the $L_2$ norm, but we know since the work of Binev, Dahmen, DeVore that there are algorithms to construct meshes so that $\|u-u_h\|_{H^1} = {\cal O}(N^{-1})$ if the solution is in $H^2$. That's already a very deep result that was only proven in the 2000s, if I recall correctly. Definitely not trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, it will not be very difficult to show experimentally that a mesh created by a less sophisticated algorithm will not satisfy this guarantee. As a consequence, the concrete choice of the adaptive mesh matters -- a lot!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you go further and ask whether this all can be translated into similar results for the wave equation. I bet that this is not possible with today's mathematical strategies. The solution of the wave equation is less regular, because the wave equation is hyperbolic. So there is less to work with in order to prove the embeddings of the solution spaces of the wave equation into the approximation spaces used in the theorem mentioned above. You then also want to couple the mesh size and the time step size, which further complicates things. I suspect that nobody in the world today can prove results for this. Or, at the very least, the half dozen people who could do so, already know all of the things I mentioned above.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-02T00:25:01.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24607" PostTypeId="2" ParentId="24579" CreationDate="2016-08-02T03:08:11.707" Score="3" Body="&lt;p&gt;Here is how I call CHOLMOD from a C program. It uses as input my own C++ structure for a sparse matrix (in standard Compressed Row Storage format).&#xA;There are several gotchas: understanding the conventions for lower and upper triangular (stype parameter) was what took me some time (CHOLMOD silently fails if this is not correct).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;suitesparse/cholmod.h&amp;gt;&#xA;#include &amp;lt;suitesparse/cholmod_check.h&amp;gt;&#xA;&#xA;namespace {&#xA;    using namespace OGF;&#xA;    /**&#xA;     * \brief Solves a linear system with CHOLMOD.&#xA;     * \param[in] M a reference to a matrix. The matrix is de-initialized &#xA;     *  on exit.&#xA;     * \param[out] x_out a pointer to the solution vector&#xA;     * \param[in] b_in a pointer to the right hand side&#xA;     * \retval true if the linear system could be solved&#xA;     * \retval false otherwise&#xA;     */&#xA;    bool solve_linear_system_with_cholmod(&#xA;        SparseMatrix&amp;amp; M, double* x_out, const double* b_in&#xA;    ) {&#xA;        ogf_assert(M.m() == M.n());&#xA;        index_t n = M.n();        &#xA;        NLSparseMatrix&amp;amp; MM = *M.implementation();&#xA;&#xA;        // Compute required nnz (works if MM in symmetric and&#xA;        // MM is in full storage mode).&#xA;        index_t nnz = 0;&#xA;        for(index_t i=0; i&amp;lt;n; ++i) {&#xA;            NLRowColumn&amp;amp; Ri = MM.row[i];&#xA;            for(index_t jj=0; jj&amp;lt;Ri.size; ++jj) {&#xA;                if(Ri.coeff[jj].index &amp;lt;= i) {&#xA;                    ++nnz;&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        // Step 1: initialize CHOLMOD library&#xA;        //----------------------------------------------------&#xA;        static cholmod_common c ;&#xA;        static bool cholmod_initialized = false;&#xA;        if(!cholmod_initialized) {&#xA;            cholmod_start(&amp;amp;c) ;&#xA;            cholmod_initialized = true;&#xA;        }&#xA;&#xA;        // Step 2: translate sparse matrix into cholmod format&#xA;        //----------------------------------------------------&#xA;&#xA;        cholmod_sparse* A = cholmod_allocate_sparse(&#xA;            n, n, nnz,    // Dimensions and number of non-zeros&#xA;            false,        // Sorted = false&#xA;            true,         // Packed = true&#xA;            1,            // stype (-1 = lower triangular, 1 = upper triangular)&#xA;            CHOLMOD_REAL, // Entries are real numbers&#xA;            &amp;amp;c&#xA;        );&#xA;&#xA;        int* colptr = (int*)A-&amp;gt;p;&#xA;        int* rowind = (int*)A-&amp;gt;i;&#xA;        double* val = (double*)A-&amp;gt;x;&#xA;&#xA;        // Convert Geogram Matrix into CHOLMOD Matrix&#xA;        index_t count = 0 ;&#xA;        for(index_t i=0; i&amp;lt;n; ++i) {&#xA;            colptr[i] = int(count);&#xA;            NLRowColumn&amp;amp; Ri = MM.row[i];&#xA;            for(index_t jj=0; jj&amp;lt;Ri.size; ++jj) {&#xA;                const NLCoeff&amp;amp; C = Ri.coeff[jj];&#xA;                index_t j = C.index;&#xA;                if(j &amp;lt;= i) {&#xA;                    val[count] = C.value;&#xA;                    rowind[count] = int(j);&#xA;                    ++count;&#xA;                }&#xA;            }&#xA;        }&#xA;        geo_assert(count == nnz);&#xA;        colptr[n] = int(nnz);&#xA;&#xA;        /*&#xA;        geo_assert(cholmod_check_sparse(A,&amp;amp;c) != 0);&#xA;        if(n &amp;lt; 10) {&#xA;            cholmod_write_sparse(stdout,A,NULL,NULL,&amp;amp;c);&#xA;        }&#xA;        cholmod_print_sparse(A,&quot;A&quot;,&amp;amp;c);&#xA;        */&#xA;&#xA;        // Step 2: construct right-hand side&#xA;        cholmod_dense* b = cholmod_allocate_dense(n, 1, n, CHOLMOD_REAL, &amp;amp;c) ;&#xA;        Memory::copy(b-&amp;gt;x, b_in, n * sizeof(double)) ;&#xA;&#xA;        // Step 3: factorize and solve&#xA;        cholmod_factor* L = cholmod_analyze(A, &amp;amp;c) ;&#xA;        geo_debug_assert(cholmod_check_factor(L,&amp;amp;c) != 0);&#xA;&#xA;        if(!cholmod_factorize(A, L, &amp;amp;c)) {&#xA;            std::cerr &amp;lt;&amp;lt; &quot;COULD NOT FACTORIZE !!!&quot; &amp;lt;&amp;lt; std::endl;&#xA;        }&#xA;&#xA;        cholmod_dense* x = cholmod_solve(CHOLMOD_A, L, b, &amp;amp;c) ;&#xA;        Memory::copy(x_out, x-&amp;gt;x, n * sizeof(double)) ;&#xA;&#xA;        // Step 4: cleanup&#xA;        cholmod_free_factor(&amp;amp;L, &amp;amp;c) ;&#xA;        cholmod_free_sparse(&amp;amp;A, &amp;amp;c) ;&#xA;        cholmod_free_dense(&amp;amp;x, &amp;amp;c) ;&#xA;        cholmod_free_dense(&amp;amp;b, &amp;amp;c) ;&#xA;&#xA;        // To be called at exit.&#xA;        // cholmod_finish(&amp;amp;c) ;&#xA;&#xA;        // Commented-out sanity check&#xA;        /*&#xA;        vector&amp;lt;double&amp;gt; check(n, 0.0);&#xA;        M.mult(x_out,check.data());&#xA;        double err = 0.0;&#xA;        for(index_t i=0; i&amp;lt;n; ++i) {&#xA;            err += geo_sqr(b_in[i] - check[i]);&#xA;        }&#xA;        err = ::sqrt(err);&#xA;        std::cerr &amp;lt;&amp;lt; &quot; CHOLMOD || Ax - b || = &quot; &amp;lt;&amp;lt; err &amp;lt;&amp;lt; std::endl;&#xA;        */&#xA;&#xA;        return true;&#xA;    }&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2016-08-02T03:28:31.590" LastActivityDate="2016-08-02T03:28:31.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24608" PostTypeId="1" CreationDate="2016-08-01T18:01:41.230" Score="1" ViewCount="65" Body="&lt;p&gt;Can you recommend a good book that discusses several methods for the numerical solution of time-dependent and time-independent Schrödinger equation? I have searched the internet several times but could not find good references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be very helpful if the book also contained matlab/python examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve the TDSE in strong laser field, and trying to implement the split operator method or Crank-Nicholson method.&lt;/p&gt;&#xA;" OwnerUserId="21210" OwnerDisplayName="hsinghal" LastActivityDate="2016-08-02T06:37:12.670" Title="Book Recommondation for the time-dependent Schrödinger equation" Tags="|computational-physics|software|" AnswerCount="0" CommentCount="7" ClosedDate="2016-08-05T17:26:17.637" ContentLicense="CC BY-SA 3.0" />
  <row Id="24610" PostTypeId="1" CreationDate="2016-08-03T04:23:25.213" Score="0" ViewCount="85" Body="&lt;p&gt;We know &lt;a href=&quot;https://en.wikipedia.org/wiki/Sod_shock_tube&quot; rel=&quot;nofollow noreferrer&quot;&gt;shock tube&lt;/a&gt; problem will give discontinuous solution of primitive variables ($\rho$, $v$, $p$). Will it give discontinuous result in flux terms?&#xA;$F =[\rho u, \rho u^2 +p, \rho e_v]^T$. I tried that on one &lt;a href=&quot;http://www.itam.nsc.ru/OLD2/flowlib/SRC/sod.f&quot; rel=&quot;nofollow noreferrer&quot;&gt;FORTRAN&lt;/a&gt; code of analytical solution on shock tube problem. I have plotted  $\rho u$ in the following figure&#xA;&lt;a href=&quot;https://i.stack.imgur.com/e8UcZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e8UcZ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here all the flux quantities are having discontinuity. In gas dynamics for steady 1D flow one of the governing equations is $\rho_1u_1= \rho_2u_2$, but the figure doesn't show such phenomena. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it because shock tube problem is transient and equation $\rho_1u_1= \rho_2u_2$ is steady?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, then how to quantify conservativeness in analytical solution?&lt;/p&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="16930" LastEditDate="2016-08-03T11:05:07.127" LastActivityDate="2016-08-03T17:12:55.673" Title="Conservatives in shock tube" Tags="|fluid-dynamics|computational-physics|conservation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24611" PostTypeId="2" ParentId="24610" CreationDate="2016-08-03T17:12:55.673" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Will it give discontinuous result in flux terms?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, the flux terms in general may be discontinuous.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it because shock tube problem is transient and equation $\rho_1 u_1= \rho_2 u_2$ is steady?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I believe the &quot;governing equation&quot; you are referring to with $\rho_1 u_1 = \rho_2 u_2$ is part of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rankine%E2%80%93Hugoniot_conditions#The_jump_condition&quot; rel=&quot;nofollow&quot;&gt;Rankine-Hugoniot jump conditions&lt;/a&gt;. This form is only valid for a stationary shock. For a general case with a (potentially) moving shock this becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_s (\rho_2 - \rho_1) = \rho_2 u_2 - \rho_1 u_1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If so, then how to quantify conservativeness in analytical solution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The terms which are temporally conserved are the conserved variables set integrated over the &lt;em&gt;entire&lt;/em&gt; domain (mass density, momentum density, total energy density): $[\rho, \vec{p}, e]$ where $\vec{p} = \rho \vec{u}$ and $e = \frac{P}{\gamma - 1} + \frac{1}{2} \rho \vec{u} \cdot \vec{u}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you need to be careful about what happens at the boundaries, as it's quite easy for the boundaries to be non-conserving (for example, if you use Dirichlet boundary conditions and force a certain $\rho$, $\vec{p}$, and $e$ at the wall).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try evaluating all of the Rankine-Hugoniot jump conditions locally in the domain, and how well these relations hold will give you some indication of if your solution is correct or not. This method should be independent of boundary condition issues, though I'm not sure this method necessarily demonstrates conservation of mass/momentum/energy of the solution.&lt;/p&gt;&#xA;" OwnerUserId="1295" LastActivityDate="2016-08-03T17:12:55.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24612" PostTypeId="1" AcceptedAnswerId="24700" CreationDate="2016-08-03T17:14:23.947" Score="6" ViewCount="1977" Body="&lt;p&gt;I am trying to solve for the allowed wavefunctions and energies for a 1D quartic potential well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this I am using the patching method (&lt;a href=&quot;https://engineering.dartmouth.edu/microeng/otherweb/henning/papers/jap-sub95.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://engineering.dartmouth.edu/microeng/otherweb/henning/papers/jap-sub95.pdf&lt;/a&gt;). I integrate from x = 0 to x = patch and from x = patch to x = end with the Numerov method. I then find the zeros of the function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{left}(patch)  *\frac{\mathrm{d} \psi_{right}(patch) }{\mathrm{d} x} - \psi_{right}(patch)  *\frac{\mathrm{d} \psi_{left}(patch) }{\mathrm{d} x} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where left means integration from x = 0 to x = patch, and right means integration from x = end to x = patch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am able to find the correct wavefunctions for a quartic well of small depth. The potential:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Hcuy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Hcuy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the first four wavefunctions that were found:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Gp4z5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gp4z5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue I am having is that when I increase the well depth, I am unable to find the correct wavefunctions. For the following deeper well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OWDkS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OWDkS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first found wavefunction (below) for this deeper well is clearly not the first allowed energy because it has too many zero crossings:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E2pPE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E2pPE.png&quot; alt=&quot;First found wavefunction for deeper well&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I inspect the minimizing function that I find the roots of, I see that it does not cross zero at low energies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect the issue for this is that in the deeper well, the two potential wells can be considered separately because there is so little wavefunction overlap. I would, however, like my solver to be robust enough to find the isolated wavefunctions, or at the very least recognize when the two potential wells can be considered separately. I would also like my solver to solve asymmetric wells, which I am not sure it can do right now. Is there a way I can adapt my code to solve these two issues?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please find my code attached: &lt;a href=&quot;https://drive.google.com/drive/folders/0B1OuUJ2C7IHAcHRkcm1JLVpLMWM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/drive/folders/0B1OuUJ2C7IHAcHRkcm1JLVpLMWM&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, here is the MATLAB code reproduced below. The main function is the first code block and is the script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Outputs&#xA;% E_allowed; % Allowed eigenvalues&#xA;% wavefunctions_allowed; % Allowed eigenfunctions&#xA;&#xA;%% Inputs&#xA;potential_function; % potential function, energy in eV&#xA;z_axis; % the 1D problem&#xA;h; % z_step, z(2) - z(1), distance in meters&#xA;mass_particle;&#xA;E_guess_max; E_guess_min;&#xA;E_guess_step;&#xA;E_disp;&#xA;&#xA;%% Set up&#xA;% Find eigenfunctions and eigenenergies for an arbitrary potential function&#xA;if(~any(strcmp(who,'MAX_SHRO_LOOP')))&#xA;    MAX_SHRO_LOOP = 200; % Maximum runs for the bisection loop&#xA;end&#xA;&#xA;if(~any(strcmp(who,'TWO_PASS_PATCHING')))&#xA;    TWO_PASS_PATCHING = 1; % Maximum runs for the bisection loop&#xA;end&#xA;&#xA;if(~any(strcmp(who,'E_guess_step')))&#xA;    E_guess_step = 1e-4; % MUST BE SET ACCORDINGLY&#xA;    %MUST BE SMALLER THAN THE SPACING BETWEEN ADJACENET ENERGY LEVELS&#xA;    %MUST BE LARGE SUCH THAT WE INCREMENT E AFTER EIGEN FOUND WE SEE CHANGE&#xA;end&#xA;&#xA;if(~any(strcmp(who,'z_patch_vec')))&#xA;    % the x patches that can be tried&#xA;    z_patch_vec = 3 : numel(z_axis) - 2;&#xA;end&#xA;&#xA;if(~any(strcmp(who,'z_patch_index')))&#xA;    z_patch_index = round(numel(z_patch_vec) / 2); % two pass z patching&#xA;end&#xA;&#xA;if(~any(strcmp(who,'init_small')))&#xA;    init_small = 1e-100; % have to adjust to get good results&#xA;    % if determinant is too small then increase this number--if the eigenvalues&#xA;    % are found too quickly&#xA;    % if NaNs in wavefunctions then decrease&#xA;end&#xA;&#xA;&#xA;if(~any(strcmp(who,'num_to_find')))&#xA;    % adaptive eigenenergy finding...&#xA;    num_to_find = [30 60 100 500 Inf];&#xA;end&#xA;&#xA;if(~any(strcmp(who,'accuracy_to_find')))&#xA;    accuracy_to_find = [1e-16 1e-8 1e-8 1e-8 1e-8]; &#xA;    % after each num_to_find has been reached, change the accuracy&#xA;end&#xA;&#xA;% E in eV&#xA;if(~any(strcmp(who,'E_guess_min')))&#xA;    E_guess_min = min(potential_function);   &#xA;end&#xA;&#xA;&#xA;if(~any(strcmp(who,'E_guess_max')))&#xA;    E_guess_max = max([potential_function(1) potential_function(end)]);    &#xA;end&#xA;&#xA;if(~any(strcmp(who,'E_disp')))&#xA;    E_disp = 0;&#xA;end&#xA;&#xA;&#xA;&#xA;&#xA;%%&#xA;mass_particle = mass_particle / q;&#xA;&#xA;Es_guessed = [];&#xA;crossfires_guessed = [];&#xA;&#xA;% the calculated stuff...&#xA;E_allowed = [];&#xA;wavefunctions_allowed = [];&#xA;left_allowed_wavefunctions = [];&#xA;right_allowed_wavefunctions = [];&#xA;&#xA;E_guess = E_guess_min;&#xA;E_bracket_low = E_guess_min;&#xA;E_bracket_high = E_guess_min;&#xA;&#xA;% for displaying stuff&#xA;E_guess_range = abs(E_guess_max - E_guess_min);&#xA;curr_percent = 10;&#xA;fprintf('\nShrodinger: ');&#xA;&#xA;z_patches = []; % the x_patches tried&#xA;if_loop_counter = 1; % the counter for the adaptive x pacthing&#xA;&#xA;while E_guess &amp;lt; E_guess_max&#xA;&#xA;    % adapative eigenenergy error finding&#xA;    EIGEN_ERR = accuracy_to_find(sum(numel(E_allowed) &amp;gt; num_to_find) + 1);  &#xA;    % adaptive x patching&#xA;    z_patch = z_patch_vec(z_patch_index);&#xA;    % print progress bar&#xA;    if( 100 * abs(min(potential_function) - E_guess) / E_guess_range &amp;gt;= curr_percent )&#xA;        fprintf(1,'%2d%%... ',(curr_percent));&#xA;        curr_percent = curr_percent + 10;&#xA;    end&#xA;&#xA;    % Find a good x_patch value&#xA;&#xA;    % Low energy bracket&#xA;    T = h^2 * 2 * mass_particle * (potential_function - E_bracket_low) / (12 * h_bar^2); % T function&#xA;    guess_left_bracket_low  = Numerov_Left(0, init_small, T);&#xA;    guess_right_bracket_low = Numerov_Right(0, init_small, T);&#xA;    [crossfire_mat_bracket_low val_low] = make_crossfire_mat(guess_left_bracket_low, guess_right_bracket_low, z_patch, h);&#xA;&#xA;    % high energy bracket&#xA;    E_bracket_high = E_bracket_low + E_guess_step;&#xA;    T = h^2 * 2 * mass_particle * (potential_function - E_bracket_high) / (12 * h_bar^2); % T function&#xA;    guess_left_bracket_high  = Numerov_Left(0, init_small, T);&#xA;    guess_right_bracket_high = Numerov_Right(0, init_small, T);&#xA;    crossfire_mat_bracket_high = make_crossfire_mat(guess_left_bracket_high, guess_right_bracket_high, z_patch, h);&#xA;    %--this code can be deleted...&#xA;&#xA;    % poor patching crossfire returns NaN, try another patching value&#xA;    if(isnan(crossfire_mat_bracket_low) ||  isnan(crossfire_mat_bracket_high))&#xA;&#xA;        if_loop_counter = 1 + if_loop_counter;&#xA;        z_patch_index = mod(z_patch_index + 1, numel(z_patch_vec)) + 1;&#xA;        z_patch = z_patch_vec(z_patch_index);&#xA;&#xA;        if(if_loop_counter &amp;gt; numel(z_patch_vec))&#xA;            disp('patching'); % all the x patchings were tried, none worked...&#xA;            break;&#xA;        end&#xA;&#xA;    end&#xA;    %--this code can be deleted...&#xA;&#xA;    %--this if statement (but not the code in the block) can be deleted&#xA;    % only continue if the x patching is okay and neither crossfire is not NaN&#xA;    if(~isnan(crossfire_mat_bracket_low) &amp;amp;&amp;amp; ~isnan(crossfire_mat_bracket_high))&#xA;&#xA;        if_loop_counter = 1; % reset the x patch counter so we can do it again later if needed&#xA;        z_patches = [z_patches; z_patch]; % the x_patches we have tried&#xA;&#xA;        crossfires_guessed = [crossfires_guessed; val_low];&#xA;        Es_guessed = [Es_guessed; E_bracket_low];  &#xA;&#xA;        %-------------------------- bracket guesses --------------------------%&#xA;        % Want to bound the eigenenerngy between the low and high brackets&#xA;        % Now having found a good x_patch, let us continue to bracket the eigenvalue guess&#xA;&#xA;        % low energy bracket is either the min energy (if 1st try) or the&#xA;        % previous eigenenergy + guess_step&#xA;&#xA;        % high energy bracket search&#xA;        % determinant has to flip signs&#xA;        while(crossfire_mat_bracket_low * crossfire_mat_bracket_high &amp;gt; 0)&#xA;&#xA;            E_bracket_high = E_bracket_high + E_guess_step;&#xA;&#xA;            T = h^2 * 2 * mass_particle * (potential_function - E_bracket_high) / (12 * h_bar^2); % T function&#xA;&#xA;            guess_left_bracket_high  = Numerov_Left(0, init_small, T);&#xA;            guess_right_bracket_high = Numerov_Right(0, init_small, T);&#xA;&#xA;            [crossfire_mat_bracket_high val] = make_crossfire_mat(guess_left_bracket_high, guess_right_bracket_high, z_patch, h);&#xA;            crossfires_guessed = [crossfires_guessed; val];&#xA;            Es_guessed = [Es_guessed; E_bracket_high];  &#xA;&#xA;        end&#xA;&#xA;        counter = 1; % counter for the binary search&#xA;        found_flag = 0; % 1 = found an eigenenergy&#xA;&#xA;        % set the first guess to be the low bracket&#xA;        E_guess_previous = E_bracket_low;&#xA;        crossfire_mat_previous = crossfire_mat_bracket_low;&#xA;&#xA;        % guess&#xA;        E_guess = (E_bracket_high + E_bracket_low) / 2;&#xA;&#xA;        % binary search&#xA;        % search until MATLAB resolution reached or iteration counter satisfied&#xA;        while(abs(E_guess_previous - E_guess) &amp;gt; EIGEN_ERR &amp;amp;&amp;amp; counter &amp;lt; MAX_SHRO_LOOP &amp;amp;&amp;amp; ~found_flag)&#xA;&#xA;            if(E_guess &amp;gt; E_guess_max)&#xA;                found_flag = 1;&#xA;                break&#xA;            end&#xA;&#xA;            % for the bisection guess&#xA;            T = h^2 * 2 * mass_particle * (potential_function - E_guess) / (12 * h_bar^2); % T function&#xA;            guess_left  = Numerov_Left(0, init_small, T);&#xA;            guess_right = Numerov_Right(0, init_small, T);        &#xA;            [crossfire_mat val] = make_crossfire_mat(guess_left, guess_right, z_patch, h);&#xA;&#xA;            crossfires_guessed = [crossfires_guessed; val];&#xA;            Es_guessed = [Es_guessed; E_guess];  &#xA;&#xA;            % E guess correct&#xA;            if(crossfire_mat == 0)&#xA;&#xA;                E_allowed = [E_allowed; E_guess];&#xA;                right_allowed_wavefunctions = [right_allowed_wavefunctions, guess_right];&#xA;                left_allowed_wavefunctions = [left_allowed_wavefunctions, guess_left];&#xA;                found_flag = 1; % will not add to the allowed stuff at the end of the bisection loop&#xA;&#xA;            % guessed too low, no sign change&#xA;            elseif(crossfire_mat * crossfire_mat_previous &amp;gt; 0)&#xA;&#xA;                E_bracket_low = E_guess;&#xA;&#xA;            else % guessed too high and the sign changed&#xA;&#xA;                E_bracket_high = E_guess;&#xA;&#xA;            end&#xA;&#xA;            % diagnostic display&#xA;            if(E_disp)&#xA;                disp(E_guess);&#xA;            end&#xA;&#xA;            % go onto the next bisection loop iteration&#xA;            E_guess_previous = E_guess;&#xA;            crossfire_mat_previous = crossfire_mat; &#xA;            E_guess = 0.5 * (E_bracket_low + E_bracket_high); % New guess&#xA;&#xA;            counter = counter + 1;            &#xA;&#xA;        end&#xA;&#xA;        % if the binary search stopped&#xA;        % stopped because the difference between two eigenenergies was so&#xA;        % small or because the max iterations was reached...not because&#xA;        % determinant was zero&#xA;        if(~found_flag)&#xA;&#xA;            E_allowed = [E_allowed; E_guess];&#xA;            right_allowed_wavefunctions = [right_allowed_wavefunctions, guess_right];    &#xA;            left_allowed_wavefunctions = [left_allowed_wavefunctions, guess_left];&#xA;&#xA;        end&#xA;&#xA;        % increment the brackets for the next run&#xA;        E_bracket_low = E_allowed(end) + E_guess_step;&#xA;        E_bracket_high = E_bracket_low + E_guess_step;&#xA;&#xA;    end&#xA;&#xA;end&#xA;&#xA;&#xA;%%&#xA;% Clean up wavefunctions&#xA;% wavefunction correction&#xA;wavefunctions_allowed = zeros(numel(z_axis), numel(E_allowed));&#xA;patching = [];&#xA;for i = 1 : numel(E_allowed)&#xA;&#xA;     [wavefunctions_allowed(:,i) patch_out] = wavefunction_patch(left_allowed_wavefunctions(:,i), right_allowed_wavefunctions(:,i), z_axis, z_patches(i), E_allowed(i), potential_function);&#xA;     patching = [patching; patch_out];&#xA;&#xA;end&#xA;&#xA;% flag_zerocross_eigenlevel = 1; % so far ok&#xA;% % num nodes ~= the energy level&#xA;% for i = 1 : numel(E_allowed)&#xA;%    &#xA;%     % zero crossings not commensurate with eigenvalue level&#xA;%     if(numel(find_roots(wavefunctions_allowed(:,i))) ~= i - 1)&#xA;%         flag_zerocross_eigenlevel = 0;&#xA;%         break&#xA;%     end&#xA;%     &#xA;% end&#xA;% &#xA;% if(~flag_zerocross_eigenlevel)&#xA;%     &#xA;%     %disp('ERROR');&#xA;%     % the wells are too seperate, consider them as isolated!!&#xA;%     &#xA;% end&#xA;&#xA;&#xA;%% Two pass patching, now we do it again...having found the optimal patching where the agreement is closest. Necessary?&#xA;% the calculated stuff...&#xA;if(TWO_PASS_PATCHING)&#xA;&#xA;    E_allowed = [];&#xA;    wavefunctions_allowed = [];&#xA;    left_allowed_wavefunctions = [];&#xA;    right_allowed_wavefunctions = [];&#xA;&#xA;    % E in eV&#xA;    E_guess_min = min(potential_function);&#xA;    E_guess_max = max([potential_function(1) potential_function(end)]);&#xA;&#xA;    E_guess = E_guess_min;&#xA;    E_bracket_low = E_guess_min;&#xA;    E_bracket_high = E_guess_min;&#xA;&#xA;    % for displaying stuff&#xA;    E_guess_range = abs(E_guess_max - E_guess_min);&#xA;    curr_percent = 10;&#xA;    fprintf('\nShrodinger: ');&#xA;&#xA;    while E_guess &amp;lt; E_guess_max &amp;amp;&amp;amp; numel(E_allowed) &amp;lt; numel(patching)&#xA;&#xA;        % adapative eigenenergy error finding&#xA;        EIGEN_ERR = accuracy_to_find(sum(numel(E_allowed) &amp;gt; num_to_find) + 1);  &#xA;&#xA;        % second pass patching&#xA;        z_patch = patching(numel(E_allowed) + 1);&#xA;&#xA;        % print progress bar&#xA;        if( 100 * abs(min(potential_function) - E_guess) / E_guess_range &amp;gt;= curr_percent )&#xA;            fprintf(1,'%2d%%... ',(curr_percent));&#xA;            curr_percent = curr_percent + 10;&#xA;        end&#xA;&#xA;        % Low energy bracket&#xA;        T = h^2 * 2 * mass_particle * (potential_function - E_bracket_low) / (12 * h_bar^2); % T function&#xA;        guess_left_bracket_low  = Numerov_Left(0, init_small, T);&#xA;        guess_right_bracket_low = Numerov_Right(0, init_small, T);&#xA;        [crossfire_mat_bracket_low val_low] = make_crossfire_mat(guess_left_bracket_low, guess_right_bracket_low, z_patch, h);&#xA;&#xA;        % high energy bracket&#xA;        E_bracket_high = E_bracket_low + E_guess_step;&#xA;        T = h^2 * 2 * mass_particle * (potential_function - E_bracket_high) / (12 * h_bar^2); % T function&#xA;        guess_left_bracket_high  = Numerov_Left(0, init_small, T);&#xA;        guess_right_bracket_high = Numerov_Right(0, init_small, T);&#xA;        crossfire_mat_bracket_high = make_crossfire_mat(guess_left_bracket_high, guess_right_bracket_high, z_patch, h);&#xA;&#xA;        crossfires_guessed = [crossfires_guessed; val_low];&#xA;        Es_guessed = [Es_guessed; E_bracket_low];  &#xA;&#xA;        %-------------------------- bracket guesses --------------------------%&#xA;        % Want to bound the eigenenerngy between the low and high brackets&#xA;        % Now having found a good x_patch, let us continue to bracket the eigenvalue guess&#xA;&#xA;        % low energy bracket is either the min energy (if 1st try) or the&#xA;        % previous eigenenergy + guess_step&#xA;&#xA;        % high energy bracket search&#xA;        % determinant has to flip signs&#xA;        while(crossfire_mat_bracket_low * crossfire_mat_bracket_high &amp;gt; 0)&#xA;&#xA;            E_bracket_high = E_bracket_high + E_guess_step;&#xA;&#xA;            T = h^2 * 2 * mass_particle * (potential_function - E_bracket_high) / (12 * h_bar^2); % T function&#xA;&#xA;            guess_left_bracket_high  = Numerov_Left(0, init_small, T);&#xA;            guess_right_bracket_high = Numerov_Right(0, init_small, T);&#xA;&#xA;            [crossfire_mat_bracket_high val] = make_crossfire_mat(guess_left_bracket_high, guess_right_bracket_high, z_patch, h);&#xA;            crossfires_guessed = [crossfires_guessed; val];&#xA;            Es_guessed = [Es_guessed; E_bracket_high];  &#xA;&#xA;        end&#xA;&#xA;        counter = 1; % counter for the binary search&#xA;        found_flag = 0; % 1 = found an eigenenergy&#xA;&#xA;        % set the first guess to be the low bracket&#xA;        E_guess_previous = E_bracket_low;&#xA;        crossfire_mat_previous = crossfire_mat_bracket_low;&#xA;&#xA;        % guess&#xA;        E_guess = (E_bracket_high + E_bracket_low) / 2;&#xA;&#xA;        % binary search&#xA;        % search until MATLAB resolution reached or iteration counter satisfied&#xA;        while(abs(E_guess_previous - E_guess) &amp;gt; EIGEN_ERR &amp;amp;&amp;amp; counter &amp;lt; MAX_SHRO_LOOP &amp;amp;&amp;amp; ~found_flag)&#xA;&#xA;            if(E_guess &amp;gt; E_guess_max)&#xA;                found_flag = 1;&#xA;                break&#xA;            end&#xA;&#xA;            % for the bisection guess&#xA;            T = h^2 * 2 * mass_particle * (potential_function - E_guess) / (12 * h_bar^2); % T function&#xA;            guess_left  = Numerov_Left(0, init_small, T);&#xA;            guess_right = Numerov_Right(0, init_small, T);        &#xA;            [crossfire_mat val] = make_crossfire_mat(guess_left, guess_right, z_patch, h);&#xA;&#xA;            crossfires_guessed = [crossfires_guessed; val];&#xA;            Es_guessed = [Es_guessed; E_guess];  &#xA;&#xA;                % E guess correct&#xA;            if(crossfire_mat == 0)&#xA;&#xA;                E_allowed = [E_allowed; E_guess];&#xA;                right_allowed_wavefunctions = [right_allowed_wavefunctions, guess_right];&#xA;                left_allowed_wavefunctions = [left_allowed_wavefunctions, guess_left];&#xA;                found_flag = 1; % will not add to the allowed stuff at the end of the bisection loop&#xA;&#xA;                % guessed too low, no sign change&#xA;            elseif(crossfire_mat * crossfire_mat_previous &amp;gt; 0)&#xA;&#xA;                E_bracket_low = E_guess;&#xA;&#xA;            else % guessed too high and the sign changed&#xA;&#xA;                E_bracket_high = E_guess;&#xA;&#xA;            end&#xA;&#xA;            % diagnostic display&#xA;            if(E_disp)&#xA;                disp(E_guess);&#xA;            end&#xA;&#xA;            % go onto the next bisection loop iteration&#xA;            E_guess_previous = E_guess;&#xA;            crossfire_mat_previous = crossfire_mat; &#xA;            E_guess = 0.5 * (E_bracket_low + E_bracket_high); % New guess&#xA;&#xA;            counter = counter + 1;            &#xA;&#xA;        end&#xA;&#xA;        % if the binary search stopped&#xA;        % stopped because the difference between two eigenenergies was so&#xA;        % small or because the max iterations was reached...not because&#xA;        % determinant was zero&#xA;        if(~found_flag)&#xA;&#xA;            E_allowed = [E_allowed; E_guess];&#xA;            right_allowed_wavefunctions = [right_allowed_wavefunctions, guess_right];    &#xA;            left_allowed_wavefunctions = [left_allowed_wavefunctions, guess_left];&#xA;&#xA;        end&#xA;&#xA;        % increment the brackets for the next run&#xA;        E_bracket_low = E_allowed(end) + E_guess_step;&#xA;        E_bracket_high = E_bracket_low + E_guess_step;&#xA;&#xA;    end&#xA;&#xA;    %%&#xA;    % Clean up wavefunctions&#xA;    % wavefunction correction&#xA;    wavefunctions_allowed = zeros(numel(z_axis), numel(E_allowed));&#xA;    patching_second_pass = [];&#xA;    for i = 1 : numel(E_allowed)&#xA;&#xA;         [wavefunctions_allowed(:,i) patch_out] = wavefunction_patch(left_allowed_wavefunctions(:,i), right_allowed_wavefunctions(:,i), z_axis, patching(i), E_allowed(i), potential_function);&#xA;         patching_second_pass = [patching_second_pass; patch_out];&#xA;&#xA;    end&#xA;&#xA;end&#xA;&#xA;clearvars MAX_SHRO_LOOP TWO_PASS_PATCHING E_guess_step z_patch_vec z_patch_index init_small...&#xA;    accuracy_to_find num_to_find E_guess_min E_guess_max&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function u = Numerov_Left(init1, init2, T)&#xA;    % init1 = x(1) wavefunction value, must be zero for bound particle&#xA;    % init2 = x(2) wavefunction value, can be anything b/c normalization&#xA;    % T function for Numerov&#xA;&#xA;    u = zeros(numel(T), 1);&#xA;        u(1) = init1; % boundary condition&#xA;        u(2) = init2; % boundary condition&#xA;&#xA;    for i = 2 : numel(T) - 1&#xA;        % Solve for u(x + h)&#xA;        u(i + 1) = (( (2 + 10 * T(i)) * u(i) - (1 - T(i - 1)) * u(i - 1) ) / (1 - T(i + 1)));&#xA;&#xA;    end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [sign_val, val] = make_crossfire_mat(left, right, patch_index, h)&#xA;&#xA;    crossfire = [left(patch_index) right(patch_index);...&#xA;        (left(patch_index) - left(patch_index - 1)) / h ...&#xA;        (right(patch_index) - right(patch_index -1)) / h];&#xA;    val = det(crossfire);&#xA;&#xA;    sign_val = sign(val);&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function u = Numerov_Right(init1, init2, T)&#xA;    % init1 = x(1) wavefunction value, must be zero for bound particle&#xA;    % init2 = x(2) wavefunction value, can be anything b/c normalization&#xA;    % T function for Numerov&#xA;&#xA;    u = zeros(numel(T), 1);&#xA;        u(end) = init1; % boundary condition&#xA;        u(end - 1) = init2; % boundary condition&#xA;&#xA;    for i = numel(T) - 1 : -1 : 2&#xA;        % Solve for u(x - h)&#xA;        u(i - 1) = (( (2 + 10 * T(i)) * u(i) - (1 - T(i + 1)) * u(i + 1) ) / (1 - T(i - 1)));&#xA;&#xA;    end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [out patch] = wavefunction_patch(left, right, z_axis, z_patch, eigenenergy, band_function)&#xA;% patch the left and right wavefunctions together&#xA;&#xA;    % for symmemtric potentials&#xA;    if(sum(band_function - fliplr(flipud(band_function))) == 0)&#xA;        patch = round(numel(band_function) / 2);&#xA;    else&#xA;&#xA;        % patch at location that is closest and nonzero&#xA;        % the division will make it nonzero because zero will go to inf!!&#xA;        left = left * right(z_patch) / left(z_patch);&#xA;        [val patch] = min((abs(left.^2 - right.^2)) ./ (left.^2));&#xA;&#xA;        % patch at a peak&#xA;        temp = find_roots(gradient(left));&#xA;        patch = temp(1); % return the patching point so we can go back and rerun the crossfire with this new patching point&#xA;    end &#xA;&#xA;    out = [left(1 : patch); right(patch + 1 : end)];&#xA;    out = out / sqrt(trapz(z_axis, out.^2));&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21221" LastEditorUserId="21221" LastEditDate="2016-08-11T14:23:46.150" LastActivityDate="2018-05-22T14:28:39.833" Title="Numerical Solution to Schrödinger Equation--Multiple Wells" Tags="|finite-difference|eigenvalues|eigensystem|quantum-mechanics|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24613" PostTypeId="1" CreationDate="2016-08-03T21:16:40.963" Score="3" ViewCount="1289" Body="&lt;p&gt;The scheme is given by&#xA;$$\frac{v_m^{n+1}-v_m^{n-1}}{2k} + b\frac{v_m^{n+1}+v_m^{n-1}-v_{m-1}^n-v_{m+1}^n}{h^2} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $v_m^n$ is the numerical solution at the $m^\text{th}$ spatial coordinate and $n^\text{th}$ time step. It approximates a solution to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t} - b\frac{\partial^2 u}{\partial x^2} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to find a bound for the local truncation error. Is there a relatively &lt;strong&gt;quick&lt;/strong&gt; way of doing this? I need to do this sort of thing on an exam, and I don't want to take too long, or rush it and make a mistake, possibly taking even longer.&lt;/p&gt;&#xA;" OwnerUserId="7322" LastEditorUserId="34160" LastEditDate="2023-05-02T18:13:07.513" LastActivityDate="2023-05-02T18:13:07.513" Title="Local truncation error of Dufort Frankel Scheme" Tags="|finite-difference|numerics|error-estimation|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="24614" PostTypeId="1" CreationDate="2016-08-03T21:33:17.323" Score="1" ViewCount="715" Body="&lt;p&gt;I have 3 variables I am considering: time (t), 1-dimensional space (x), and intensity (I). I would like to plot the intensity in the z-axis as a function of t and x (the latter two variables would form an xy-plane). With the current data I have, these variables themselves are 2-dimensional arrays where each corresponding index from the different arrays match appropriately to a single datapoint (i.e. &lt;code&gt;(x,t,I) = (x[k,i],t[k,i],I[k,i])&lt;/code&gt;). (Ideally this approach would be extended to arrays of higher dimensions than two.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example set of data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I = [[  10.55   0.   0.]&#xA;[  0.   0.   0.01]&#xA;[  0.2   -0.1   3.33]&#xA;[  0.  2.14   0.]&#xA;[  0.   3.80   0.]&#xA;[  9.02   0.   0.]]&#xA;&#xA;t = [[   0.  400.  1000.]&#xA;[   1.  300.  800.]&#xA;[   0.  500.  900.]&#xA;[   200.  400.  0.]&#xA;[   100.  700.  0.]&#xA;[   0.  0.    0.]]&#xA;&#xA;x = [[    0.     0.     0.]&#xA;[ 500.  1500.  1500.]&#xA;[ 3000.  3000.  8000.]&#xA;[ 3300.  4500.  0.]&#xA;[ 6000.  6000.  0.]&#xA;[ 7500.  0.     0.]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just a note: the lengths of each row have been made equivalent in the arrays, but they should ideally be null if &lt;code&gt;z[k,i] = t[k,i] = 0.&lt;/code&gt; if &lt;code&gt;k != 0&lt;/code&gt; and &lt;code&gt;i != 0&lt;/code&gt;. As you can tell, in the example above, there are 4 such elements corresponding to the indices: &lt;code&gt;[k,i] = [3,2], [4,2], [5,1], [5,2]&lt;/code&gt;. Although these would only affect the coordinate &lt;code&gt;(0,0,0)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To restate my goal: I would like to plot t, x, and I as a 3-dimensional surface. In the past, using code such as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from mpl_toolkits.mplot3d import Axes3D&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;&#xA;X, Y = np.meshgrid(t,x)&#xA;ax = fig.add_subplot(1,1,1, projection='3d')&#xA;surf = ax.plot_surface(X,Y,I, rstride=4, cstride=4, alpha=0.1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;worked for cases where x and t were 1-dimensional arrays. But I'm curious as to any good approaches to modelling the example data I've shown above as a 3-dimensional surface where there is a nonuniform grid.&lt;/p&gt;&#xA;" OwnerUserId="20407" LastActivityDate="2016-08-04T03:54:38.187" Title="3-dimensional plotting with nonuniform grids" Tags="|python|numerical-modelling|plotting|grid|3d|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24617" PostTypeId="2" ParentId="24614" CreationDate="2016-08-04T03:54:38.187" Score="1" Body="&lt;p&gt;Your data is already in the right shape, then you don't need to create a &lt;code&gt;meshgrid&lt;/code&gt;. See the code below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;import numpy as np&#xA;&#xA;&#xA;I = np.array([&#xA;    [10.55, 0., 0.],&#xA;    [0., 0., 0.01],&#xA;    [0.2, -0.1, 3.33],&#xA;    [0., 2.14, 0.],&#xA;    [0., 3.80, 0.],&#xA;    [9.02, 0., 0.]])&#xA;&#xA;t = np.array([&#xA;    [0., 400., 1000.],&#xA;    [1., 300., 800.],&#xA;    [0., 500., 900.],&#xA;    [200., 400., 0.],&#xA;    [100., 700., 0.],&#xA;    [0., 0., 0.]])&#xA;&#xA;x = np.array([&#xA;    [ 0.,  0.,  0.],&#xA;    [ 500., 1500., 1500.],&#xA;    [ 3000., 3000., 8000.],&#xA;    [ 3300., 4500., 0.],&#xA;    [ 6000., 6000., 0.],&#xA;    [ 7500., 0.,  0.]])&#xA;&#xA;fig = plt.figure()&#xA;ax = fig.add_subplot(1,1,1, projection='3d')&#xA;surf = ax.plot_surface(x, t, I, rstride=1, cstride=1,&#xA;                       alpha=0.4, cmap=&quot;viridis&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That makes the following plot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BPff9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BPff9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2016-08-04T03:54:38.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24618" PostTypeId="2" ParentId="24613" CreationDate="2016-08-04T10:15:55.840" Score="0" Body="&lt;p&gt;A similar subject &lt;a href=&quot;https://scicomp.stackexchange.com/questions/24580/more-smearing-with-decreasing-timestep-in-advection-problems/24588#24588&quot;&gt;here&lt;/a&gt; where I show the way to calculate the truncation error with the advection equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A search gives me &lt;a href=&quot;http://how.gi.alaska.edu/ao/sim/homework/h1311a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. I think that could answer your question ?&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-08-04T10:15:55.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24621" PostTypeId="1" CreationDate="2016-08-04T13:56:26.983" Score="4" ViewCount="148" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot; rel=&quot;nofollow&quot;&gt;Kahan's summation algorithm&lt;/a&gt; is a method to compute sums:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum x_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with many terms, without significant error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to do this with very large numbers, and instead of the numbers themselves, I am given their logarithms, $\log x_i$ ($x_i &amp;gt; 0$), and instead of the sum, I want to compute the logarithm of the sum,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\log \sum x_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a variation of Kahan's summation for this case?&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2016-08-04T17:17:27.063" LastActivityDate="2016-08-04T17:17:27.063" Title="Stable computation of $\log\sum x_i$ from $\log x_i$, with many terms" Tags="|algorithms|error-estimation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24622" PostTypeId="1" AcceptedAnswerId="24632" CreationDate="2016-08-04T16:50:27.533" Score="4" ViewCount="1115" Body="&lt;p&gt;I've implemented a version of the double-shift QR algorithm featured in this &lt;a href=&quot;http://people.inf.ethz.ch/arbenz/ewp/Lnotes/chapter4.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;report&lt;/a&gt; from ETH Zurich (Begins on page 77). The algorithm takes advantage of the Implicit Q theorem by applying an orthogonal-similarity transformation to our original matrix (in Hessenberg) form. This introduces a 'bulge' which we then chase out of the matrix by applying successive householder reflectors. The end goal is to covert the original matrix to real schur form (Although not truly, since complex eigenvalues form as blocks along the diagonal). The pseudocode for the algorithm is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fZJPo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fZJPo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I've implemented this particular algorithm in C) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when trying to solve companion matrices to retrieve the eigenvalues, I find it often gets things wrong. Even their example results with a $6 \times 6$ couldn't solve for all the eigenvalues: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zDK2om.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zDK2om.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SWwoLl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SWwoLl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare this to my results, which have much the same mistake: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;5.000000 6.000000 0.390635 -1.005053 -7.905002 7.702966 &#xA;-6.000000 5.000000 -10.791799 -2.710116 -14.579765 -19.663045 &#xA;0.000000 0.000000 3.491643 -2.656106 -4.440683 9.373136 &#xA;0.000000 0.000000 3.843327 -1.491643 -10.829108 -5.067750 &#xA;0.000000 0.000000 -0.000000 0.000000 4.000000 4.922432 &#xA;-0.000000 -0.000000 0.000000 0.000000 0.000000 3.000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Even smaller matrices pose problems (though they are close) with very low tolerances:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input:&lt;/strong&gt; (Eigenvalues are: 4, -3, 2, -1)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2.000000 13.000000 -14.000000 -24.000000 &#xA;1.000000 0.000000 0.000000 0.000000 &#xA;0.000000 1.000000 0.000000 0.000000 &#xA;0.000000 0.000000 1.000000 0.000000 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2.968869 11.491388 21.498124 17.801229 &#xA;0.535591 -1.968869 -3.241184 -2.692433 &#xA;-0.000000 0.000000 2.000000 1.078408 &#xA;0.000000 -0.000000 -0.000000 -1.000000 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone suggest what sort of improvements can be made to obtain better results? I've tried implementing another algorithm I found, but it provided worse results. Even when I ran it as-is through Matlab instead of my own C-version of the software. &lt;/p&gt;&#xA;" OwnerUserId="21237" LastActivityDate="2016-08-05T14:42:45.870" Title="How to improve this double-shift QR algorithm for non-symmetric matrices?" Tags="|matrix|eigenvalues|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24623" PostTypeId="1" AcceptedAnswerId="24625" CreationDate="2016-08-04T17:18:11.593" Score="1" ViewCount="535" Body="&lt;p&gt;To make stability proofs simpler, I can consider an explicit scheme written as&#xA;$$V(n+1,i)=aV(n,i-1)+bV(n,i)+cV(n,i+1)$$&#xA;and one can show that if $a,b,c\ge 0$ and $a+b+c\le1$, then the explicit method is stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for implicit, the scheme can be written as&#xA;$$aV(n+1,i-1)+bV(n+1,i)+cV(n+1,i+1)=V(n,i)$$&#xA;and if $a,c \le 0, b\ge 0$, and $a+b+c&amp;gt;0$, then the scheme is stable (This condition can be found in Smith book on ths stability of Thomas algorithm).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, all the finite difference books typically talk about Matrix stability or Fourier stability. But what I have outlined above is not PDE dependent, so why could not I just check the test on coefficients as above and never learn about other stability methods? Does this rule has limitations? &lt;/p&gt;&#xA;" OwnerUserId="1611" LastActivityDate="2016-08-05T05:23:11.093" Title="Stability condition for explicit/implicit via non negative coefficients" Tags="|pde|finite-difference|stability|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24624" PostTypeId="1" CreationDate="2016-08-04T19:38:28.867" Score="3" ViewCount="346" Body="&lt;p&gt;I want to compute an expression of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L = \ln\sum_i e^{x_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that there are many small terms, say $e^{x_i} \approx \epsilon$. If there are $N_\epsilon$ such terms, their contribution to the sum is $\approx \epsilon N_\epsilon$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the usual &lt;em&gt;logsumexp&lt;/em&gt; trick, we let $a=\max x_i$ and compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L = a + \ln \sum_i e^{x_i - a}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to avoid overflows. However, $\epsilon$ could be so small that $\epsilon / e^a$ becomes zero in floating-point (underflow). Even then, if $N_\epsilon$ is large enough, a significant contribution $\epsilon N_\epsilon$ to the sum will be lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to deal with this case? &lt;em&gt;A priori&lt;/em&gt; I may not know about the existence of these very small terms in the sequence. Applying the &lt;em&gt;logsumexp&lt;/em&gt; trick in this case may lead to large errors. Is there a better algorithm?&lt;/p&gt;&#xA;" OwnerUserId="988" LastActivityDate="2016-08-04T19:38:28.867" Title="logsumexp with one very large term and many very small terms" Tags="|algorithms|numerical-analysis|numerical-limitations|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24625" PostTypeId="2" ParentId="24623" CreationDate="2016-08-05T05:23:11.093" Score="3" Body="&lt;p&gt;The fact that positivity of coefficients of explicit and implicit methods &lt;em&gt;usually&lt;/em&gt; leads to sufficient conditions for stability is well known, but rarely discussed in detail. (I'm not aware of any textbooks discussing it in any detail, for example.) I suspect that the method does not receive much attention precisely because it only gives sufficient conditions whereas the von-Neumann method (Fourier analysis) usually gives sufficient and necessary conditions. This paper compares results obtained with the two methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B. J. Noye, &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/cnm.1630060405/abstract&quot; rel=&quot;nofollow&quot;&gt;A new third-order finite-difference method for transient one-dimensional advection—diffusion&lt;/a&gt;, Communications in Applied Numerical Methods, &#xA;6(4):279–288, 1990&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple case in which the positive coefficients do not lead to sufficient stability restrictions is the leapfrog method. This is a simple consequence of the fact that not all oscillations indicate an instability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Positivity of course implies that local extrema do not grow in time, which is why requiring positive coefficients is discussed quite extensively in the literature dealing with the computation of flows with discontinuities, see, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;S. Spekreijse, &lt;a href=&quot;http://www.ams.org/journals/mcom/1987-49-179/S0025-5718-1987-0890258-9/S0025-5718-1987-0890258-9.pdf&quot; rel=&quot;nofollow&quot;&gt;Multigrid solution of monotone second-order discretizations of hyperbolic conservation laws&lt;/a&gt;, Mathematics of Computation, 49(179):135-155, 1987&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Jameson, &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/fld.1650200805/abstract&quot; rel=&quot;nofollow&quot;&gt;Positive schemes and shock modelling for compressible flows&lt;/a&gt;, International Journal for Numerical Methods in Fluids, 20(8-9):743–776, 1995&lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2016-08-05T05:23:11.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24628" PostTypeId="1" CreationDate="2016-08-05T09:31:40.013" Score="4" ViewCount="1182" Body="&lt;p&gt;Assume we have a positive semidefinite matrix $A$ with  $A\in\mathbb{R^{n\times n}}$. Clearly a cholesky decomposition $A=B^TB$ exist with $B\in\mathbb{R^{n\times n}}$. For my research it would be interesting to know, if we can construct a rectangular decompostion so that $A=B^TB$ with $B\in\mathbb{R^{m\times n}}$ and $n\neq m$ where $m$ is arbitrary. However, I have the feeling that such a decompostion only exists, if $m&amp;gt;n$. Has anybody heared of such a decomposition?&lt;/p&gt;&#xA;" OwnerUserId="21244" LastEditorUserId="27" LastEditDate="2016-08-05T13:22:05.457" LastActivityDate="2016-08-07T14:15:34.037" Title="Rectangular Cholesky Decomposition" Tags="|linear-algebra|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24632" PostTypeId="2" ParentId="24622" CreationDate="2016-08-05T14:42:45.870" Score="3" Body="&lt;p&gt;The matrix given as H(11) in your question actually does have the expected eigenvalues of $1 \pm 2i$, $5 \pm 6i$, $3$, and $4$.  In particular, the eigenvalues of the 2x2 block circled in red are $1 \pm 2i$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a matter of pure luck that the 2x2 block in the upper left has the form that it does, making it obvious that its eigenvalues of $5 \pm 6i$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-08-05T14:42:45.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24633" PostTypeId="2" ParentId="24628" CreationDate="2016-08-05T14:51:36.557" Score="3" Body="&lt;p&gt;If $A$ is positive definite, then $\mbox{rank}(A)=n$, and any factorization of the form $A=B^{T}B$ must involve a $B^{T}$ matrix with $m \geq n$ columns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's relatively easy to construct such a factorization from the Cholesky factorization of $A$ (e.g. by adding 0 columns), but what do you want to do with such a $B$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is merely positive semidefinite, then $\mbox{rank}(A)=r&amp;lt;n$, and there is no Cholesky factorization, although you can construct a factorization $A=B^{T}B$ where $B^{T}$ has $r$ columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to do this uses the eigenvalue decomposition of $A$ as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, find the eigenvalue-eigenvector decomposition of $A$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=U \Lambda U^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is $n$ by $n$ and orthogonal and $\Lambda$ is $n$ by $n$ and diagonal with nonnegative elements on the diagonal.  Assume that the eigenvalues in $\Lambda$ are sorted in descending order so that the 0 eigenvalues come last.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By taking the square roots of the eigenvalues, we can write $A$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=U \sqrt{\Lambda} \sqrt{\Lambda} U^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C=\sqrt{\Lambda} U^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=C^{T}C$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $r$ be the number of nonzero eigenvalues in $\Lambda$.  Since the remaining $n-r$ eigenvalues are 0, the last $n-r$ rows of $C$ are 0.  Let $B$ consist of the first $r$ rows of $C$.  Then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=B^{T}B$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $B$ is of size $r$ by $n$ and $B^{T}$ is of size $n$ by $r$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another common way to write this is as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=\sum_{i=1}^{r} \lambda_{i} u_{i}u_{i}^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{i}$ is the $i$th column of $U$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no factorization $A=B^{T}B$ in which $B^{T}$ has fewer than $r$ columns.  There are many factors with more than $r$ columns, such as the $C^{T}$ given above.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-08-07T14:15:34.037" LastActivityDate="2016-08-07T14:15:34.037" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24636" PostTypeId="1" CreationDate="2016-08-05T23:46:03.340" Score="0" ViewCount="284" Body="&lt;p&gt;Can anyone write a Python implementation of a set of  particles interacting in 2D according to a short-range particle-particle force and evolving in time under a Metropolis algorithm, which randomly chooses a particle and proposes a random spatial move? I can write such an implementation using the naive approach of updating all the particle-particle interaction energies associated with a given cell that has been proposed to be moved, but would like to see an implementation using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Verlet_list&quot; rel=&quot;nofollow noreferrer&quot;&gt;Verlet List&lt;/a&gt; or a &lt;a href=&quot;https://en.wikipedia.org/wiki/K-d_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;KD Tree&lt;/a&gt;, which I hope can be at least an order of magnitude faster when the number of particles is large (say 100-1000).&lt;/p&gt;&#xA;&lt;p&gt;An added complication in my particular problem is that the particles replicate and die, but incorporating that into the Verlet/KD Tree approach would be a secondary goal.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any help you can offer.&lt;/p&gt;&#xA;" OwnerDisplayName="user21228" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-10-08T19:59:38.703" Title="Fast Python implementation of short-range interacting particles under Metroplis algorithm" Tags="|performance|monte-carlo|molecular-dynamics|numpy|scipy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24637" PostTypeId="1" CreationDate="2016-08-06T04:17:18.920" Score="1" ViewCount="188" Body="&lt;p&gt;I'm relatively new in numerical methods for PDEs, so I have a question about solving the following PDE numerically for $W(x,p)$ in the phase space:&#xA;$$-p\frac{dW}{dx}+x\frac{dW}{dp}+\frac{d}{dx}\left(xW \right)+2 \frac{d}{dp}\left(pW \right) + \frac{d^{2}W}{dx^{2}} +\frac{d^{2}W}{dp^{2}}-\frac{d^{2}W}{dxdp} = 0.$$&#xA;What is the simplest (yet reasonably accurate) numerical method to solve the equation above? I expect to see a Gaussian in $x,p$ and I wish to visualize the numerical solution in the window $(x,p,W(x,p))$.&lt;/p&gt;&#xA;" OwnerUserId="21254" LastActivityDate="2016-08-06T04:17:18.920" Title="Numerical method to find stationary solution of an PDE in the phase space" Tags="|pde|numerical-analysis|python|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24638" PostTypeId="2" ParentId="24636" CreationDate="2016-08-06T20:21:34.280" Score="1" Body="&lt;p&gt;Scipy offers an implementation of KDTree class:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.KDTree.html&quot; rel=&quot;nofollow&quot;&gt;http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.KDTree.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yu should be able to incorporate this into your code.&lt;/p&gt;&#xA;" OwnerUserId="15014" LastActivityDate="2016-08-06T20:21:34.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24639" PostTypeId="1" AcceptedAnswerId="24640" CreationDate="2016-08-06T21:42:46.667" Score="8" ViewCount="214" Body="&lt;p&gt;I've got two inverse problems,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_1 ~ x = b_1 \qquad A_2 ~ x = b_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I've been solving them independently using Tikhonov Regularization and getting two estimates for $x$. However in my case $x$ represents the same solution in both equations. Is it possible to do a 'simultaneous' solve? Ideally I would be finding the answer for&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min \left( \lVert A_1 x - b_1 \rVert^2 + \lVert A_2 x - b_2 \rVert^2 + \lVert\Gamma x\lVert^2 \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\Gamma = \alpha ~ I$ and $I$ is the identity matrix as in Tikhonov Regularization (aka ridge regression). I suppose I could just take the average of both solutions, wondering if there is a more statistically powerful way of approaching this however.&lt;/p&gt;&#xA;" OwnerUserId="21260" LastEditorUserId="10134" LastEditDate="2016-08-07T23:35:06.773" LastActivityDate="2016-08-07T23:35:06.773" Title="Solving two inverse problems with same solution" Tags="|linear-algebra|numerical-analysis|inverse-problem|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24640" PostTypeId="2" ParentId="24639" CreationDate="2016-08-06T22:29:18.103" Score="9" Body="&lt;p&gt;You can write your problem as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| Fm - g \|_{2}^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F=\left[&#xA;\begin{array}{c}&#xA;A_{1} \\&#xA;A_{2} \\&#xA;\alpha I \\&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g=\left[&#xA;\begin{array}{c}&#xA;b_{1} \\&#xA;b_{2} \\&#xA;0 \\&#xA;\end{array}&#xA;\right].&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use whatever linear least squares solver you want to solve this problem.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-08-06T22:29:18.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24641" PostTypeId="2" ParentId="24501" CreationDate="2016-08-07T19:13:10.623" Score="1" Body="&lt;p&gt;I would suggest OpenSees: &lt;a href=&quot;http://opensees.berkeley.edu/index.php&quot; rel=&quot;nofollow&quot;&gt;http://opensees.berkeley.edu/index.php&lt;/a&gt;. Some colleagues in my department made fairly good use of it a couple of years ago in combination with SAP2000.&lt;/p&gt;&#xA;" OwnerUserId="21268" LastActivityDate="2016-08-07T19:13:10.623" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24642" PostTypeId="2" ParentId="24537" CreationDate="2016-08-07T19:22:44.253" Score="-2" Body="&lt;p&gt;I suggest you to have a visual and intuitive approach using Grasshopper for Rhinoceros3D. Although Rhinoceros is a commercial CAD package and Grasshopper is a plugin for it, you can run plugins for free with no limitations and make your experiments (unlicensed Rhino3D limits only the saving of Rhino files).&#xA;Grasshopper includes a large number of mathematical functions used in a canvas, and 3D Voronoi diagrams is one of them.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Jg4OP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jg4OP.jpg&quot; alt=&quot;a Voronoi cube done with Grasshopper3D in Rhino3D&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21268" LastActivityDate="2016-08-07T19:22:44.253" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24643" PostTypeId="1" CreationDate="2016-08-07T19:25:35.540" Score="0" ViewCount="277" Body="&lt;p&gt;Consider an advection-diffusion problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t+au_{xx}+bu_{x}=0,\quad x&amp;gt;0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to remove the drift and rewrite the problem for the new domain. I use the change of variables $y=x-bt$, and the equation can be rewritten as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$v_t+av_{yy}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but what is the new domain in terms of variables $t$ and $y$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume $b&amp;gt;0$, then $t\in [0,T]$ as before. But for $y$ the domain is not rectangular anymore, because it is like a triangle on one side. That is, for every fixed $t$, my $y$ is between $-bt$ and $\infty$? Did I compute the new domain correctly? If so, how do I solve this numerically? This is not a square box anymore, so do I have to place the grid for finite differences on such domain shape? &lt;/p&gt;&#xA;" OwnerUserId="1611" LastEditorUserId="20688" LastEditDate="2017-04-08T21:15:24.930" LastActivityDate="2018-11-30T08:01:35.280" Title="Cauchy problem with a change of variables" Tags="|finite-difference|parabolic-pde|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24644" PostTypeId="1" AcceptedAnswerId="24651" CreationDate="2016-08-07T19:29:58.483" Score="0" ViewCount="303" Body="&lt;p&gt;I am working on a domain decomposition code in C that uses CHOLMOD to approximate grid values for a PDE in each sub-domain. The issue I have is that the methods use Matrix Market format, which is not an issue in general, but I already have a method that computes a 2D array which is the sparse matrix A in the Ax=b solve. I am not sure how I could either convert that into a sparse matrix using a method in CHOLMOD or if there was a way to save a Matrix Market text file on each loop call and somehow put it into A = cholmod_read_sparse or something else. Any insight would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="20973" LastEditorUserId="20973" LastEditDate="2016-08-08T13:37:52.653" LastActivityDate="2016-08-08T13:43:00.113" Title="CHOLMOD implementation" Tags="|finite-difference|linear-solver|sparse-matrix|c|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24645" PostTypeId="1" CreationDate="2016-08-08T02:40:31.493" Score="2" ViewCount="134" Body="&lt;p&gt;Let $\Sigma$ be positive definite and the $D_i$ positive diagonal.  Let the $X_i$ be unknown square matrices.  Consider the system of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(I+\Sigma D_i)X_i=\Sigma\hspace{5mm}\text{for}\hspace{5mm}i=1,...n.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to numerically solve all of these equations without having to completely recompute the LU factorization of $(I+\Sigma D_i)$ for each $i$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been messing around with the Cholesky and Eigen-decompositions of $\Sigma$, but no luck so far, I think it's probably not possible, but I thought I'd ask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it helps, note that the solutions $X_i$ will all be positive definite, since&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X_i^{-1} = \Sigma^{-1} + D_i.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10533/perturbation-of-cholesky-decomposition-for-matrix-inversion/10535#10535&quot;&gt;this&lt;/a&gt; answer here by the all knowing Brian Borchers means that this is infact impossible.&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-10-07T08:28:27.810" Title="Solving system of related equations without completely recomputing LU decomposition for each equation" Tags="|linear-algebra|linear-system|matrix-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24646" PostTypeId="1" CreationDate="2016-08-08T03:34:52.970" Score="0" ViewCount="1826" Body="&lt;p&gt;I came with a the following code to evaluate a double integral using Gauss Legendre quadrature in MatLab &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m=100;&#xA;&#xA;%generate weights and abscissas&#xA;[wx,xx]=leg(-1,1,m);&#xA;[wtheta,xtheta]=leg(0,2.*pi,m);&#xA;&#xA;%define function&#xA;psi=@(x,theta) hypergeom(-3./4,1./2,x.^2.*exp(1i.*theta));&#xA;&#xA;%integrate with respect to x&#xA;intx=zeros(1,m);&#xA;for num=1:m&#xA;    intx(num)=sum(wx.*psi(xx,xtheta(num)));&#xA;end&#xA;&#xA;sum(wtheta.*intx)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I defined the function leg(x1,x2,m) in a different script to generate the weights and abscissas and I just call it in my code. My MatLab code runs significanlty slowly compared to NIntegrate of Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to make my code faster because my idea of using MatLab is that it is faster than Mathematica. Is there any way I can make my code run faster? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attached is a profile summary when i ran the code &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uKlCw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uKlCw.png&quot; alt=&quot;Profile summary&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21269" LastEditorUserId="21269" LastEditDate="2016-08-08T04:19:06.120" LastActivityDate="2016-12-07T10:10:36.000" Title="Faster code for double integration using Gauss-Legendre quadrature" Tags="|matlab|numerical-analysis|quadrature|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24647" PostTypeId="2" ParentId="24645" CreationDate="2016-08-08T07:47:49.007" Score="1" Body="&lt;p&gt;You can always try a conjugate gradient (&lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Conjugate_gradient_method&lt;/a&gt;). &#xA;Once you have found the solution via LU, if the new changes affect only to a small number of equations (or even a large one), the convergence should be very fast.&#xA;Here it is clearly explained how to do it: &lt;a href=&quot;ftp://ftp.numerical.rl.ac.uk/pub/talks/isd_stanford50.pdf&quot; rel=&quot;nofollow&quot;&gt;ftp://ftp.numerical.rl.ac.uk/pub/talks/isd_stanford50.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21268" LastActivityDate="2016-08-08T07:47:49.007" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24648" PostTypeId="2" ParentId="24644" CreationDate="2016-08-08T08:02:05.480" Score="0" Body="&lt;p&gt;I do hope that your &lt;code&gt;A&lt;/code&gt; that you assemble is a sparse matrix. Otherwise the use of sparse cholesky factorizations would not make sense. Then, if you didn't do this during the assembling process, you should turn it into a sparse matrix format as soon as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then -- and not knowing in what environment you are so that I can only guess -- it will be more efficient to pass the matrix directly to &lt;code&gt;cholmod&lt;/code&gt; which will convert it for you (at least in the python interface) without doing the saving/loading step using the &lt;code&gt;matrix market format&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-08-08T08:02:05.480" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24650" PostTypeId="1" CreationDate="2016-08-08T09:43:28.060" Score="2" ViewCount="477" Body="&lt;p&gt;I need to calculate the inverse CDF for a probability distribution, however there is no closed form approximation available in the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distribution I am working with is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Normal-inverse_Gaussian_distribution&quot; rel=&quot;nofollow&quot;&gt;Normal Inverse Gaussian distribution&lt;/a&gt;, however I would like to make my question more general than this. Of course, mathematical analysis could be used in principle to derive an approximation for any given distribution but I am looking for a more general technique. To give you an idea, the PDF function is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${\frac {\alpha \delta K_{1}\left(\alpha {\sqrt {\delta ^{2}+(x-\mu )^{2}}}\right)}{\pi {\sqrt {\delta ^{2}+(x-\mu )^{2}}}}}\;e^{\delta \gamma +\beta (x-\mu )}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K_1$ is the modified Bessel function of the third kind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementations I have seen for this distribution simply&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use numerical integration to define the CDF; then&lt;/li&gt;&#xA;&lt;li&gt;Use root finding to define the inverse CDF&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There must be a better way to calculate the inverse CDF than this. It's not hard to imagine improvements, for example you could pre-evaluate the PDF at a list of points, use quadrature on this list to evaluate the CDF and then use interpolation to approximate the inverse CDF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any established methods or algorithms for calculating the inverse CDF in this scenario?&lt;/p&gt;&#xA;" OwnerUserId="21270" LastActivityDate="2016-08-08T09:43:28.060" Title="Numerical methods for calculating the inverse CDF when closed form approximation not available" Tags="|numerical-analysis|reference-request|probability|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24651" PostTypeId="2" ParentId="24644" CreationDate="2016-08-08T13:43:00.113" Score="2" Body="&lt;p&gt;The most efficient way will be to directly use the C API of CHOLMOD and call it directly, without saving the matrix to disk, see my answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/24579/linear-solve-using-chlomod-in-c/24607#24607&quot;&gt;this question&lt;/a&gt; for an example (see also CHOLMOD documentation). CHOLMOD uses the standard CRS (compressed row storage) representation. It is explained in many places, see for instance &lt;a href=&quot;https://de.wikipedia.org/wiki/Compressed_Row_Storage&quot; rel=&quot;nofollow noreferrer&quot;&gt;this wikipedia entry&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, it is also possible, as you suggest, to write the matrix to disk using Matrix Market format and call CHOLMOD by spawning the example program provided with it, but this will be less efficient and will require as much programming effort as calling CHOLMOD directly. If you are not programming in C/C++ or in a language that has CHOLMOD bindings, you may need to do that. The easiest solution for you will be to use the coordinate format. If performance is an issue, you may use CRS format (a little bit more programming effort). Both are explained &lt;a href=&quot;http://math.nist.gov/MatrixMarket/formats.html#MMformat&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-08-08T13:43:00.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24653" PostTypeId="2" ParentId="24537" CreationDate="2016-08-08T14:01:22.093" Score="2" Body="&lt;p&gt;To compute the Voronoi diagram of huge (&gt;100 millions) sets of points, you can use the following algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1) create a kd-tree with all the points&#xA;2) for each point p [in parallel optionally]&#xA;     N = 10&#xA;     while not finished&#xA;       compute the N nearest neighbors of the point p&#xA;       compute the intersection of the N half-spaces defined by p &#xA;       and the neighbors&#xA;       if there is a neighbor further away than &#xA;         twice the radius of the ball centered on p and &#xA;         bounding the intersection, finished = true&#xA;       N = N * 1.5&#xA;  // when exiting the loop, the computed intersection &#xA;  // corresponds to the Voronoi cell of p, because no other bisector&#xA;  // can contribute to the Voronoi cell.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The algorithm is explained with further details in &lt;a href=&quot;http://alice.loria.fr/index.php/publications.html?redirect=0&amp;amp;Paper=Vorpaline_IMR@2012&quot; rel=&quot;nofollow&quot;&gt;my article&lt;/a&gt;. It can be trivially parallel-ized (just add &quot;#pragma omp parallel for&quot; before the main loop), since there is no data dependency.  It is implemented in my &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow&quot;&gt;GEOGRAM&lt;/a&gt; C++ programming library (together with a memory efficient Kd-Tree that scales up to more than 100 million points). Note that in GEOGRAM there is also a parallel standard Delaunay/Voronoi implementation that works well with up to 100M sites.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning parallel implementation of the classical (Boywer-Watson) algorithm, the GEOGRAM implementation is documented &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/classGEO_1_1ParallelDelaunay3d.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; (see also the associated c++ source file that has extensive comments). I have no published article about it, I will write one if time permits. The main idea is to use spinlocks associated with the tetrahedra to ensure that only one individual thread can modify a tetrahedron.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2016-08-08T19:53:42.597" LastActivityDate="2016-08-08T19:53:42.597" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24657" PostTypeId="1" CreationDate="2016-08-09T01:59:52.820" Score="1" ViewCount="197" Body="&lt;p&gt;For a graph $G=(V,E)$, recall that the unweighted Laplacian is $L:=D^\top D$, where $D\in\{-1,0,1\}^{|E|\times|V|}$  is the graph &quot;gradient&quot; operator that subtracts adjacent vertex values onto edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm writing an algorithm that adds/removes edges one-at-a-time to a graph.  In each iteration, I need to apply $L^+$ to a few vectors, where $L^+$ is the Moore-Penrose pseudoinverse of $L$ for the current graph.  I could recompute $L$ and its pseudoinverse in each iteration, but this is computationally expensive!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding/removing an edge from $E$ makes a rank-1 change to $L$.  So, it seems like we could update a sparse factorization (Cholesky?  LDLT?) of $L$ and use this to accelerate application of $L^+$.  &lt;strong&gt;Are there easy-to-use algorithms for this?&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;NOTES:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've struggled to apply standard Matlab calls.  Challenges include the fact that $L$ always has a null space (constant functions), $G$ may not be a connected graph, and removing an edge is a rank-1 but subtractive update.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extra credit if you have a pointer to existing library of code!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arxiv.org/pdf/1411.4884.pdf&quot; rel=&quot;nofollow&quot;&gt;This paper&lt;/a&gt; shows how to update the pseudoinverse of the Laplacian when adding an edge.  But it does not show what to do when removing an edge (maybe it's an easy extension, but I didn't see it!), and the pseudoinverse is a dense matrix that I would like to avoid storing.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2016-08-09T01:59:52.820" Title="Updating factorization of Laplacian (add/remove edges)" Tags="|linear-algebra|linear-solver|sparse-matrix|graph-theory|matrix-factorization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24658" PostTypeId="1" CreationDate="2016-08-09T03:56:26.830" Score="1" ViewCount="901" Body="&lt;p&gt;I am modelling a seismic wave equation using FEM. In the few papers that I read, I understand the following: (Kindly correct me if you disagree)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;A shear (secondary wave - no change of volume) is more important to model than the compression (primary wave - no rotation) as it brings more devastation than the latter.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The elastic wave equation is a better representative of a seismic wave than the acoustic wave equation because simulating it, we propagate both 'p' and 's' waves in the mesh instead of just 'p' which we get from simulating acoustic equation. (p - primary; s- secondary)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The elastic wave equation also known as Navier's equation of elastodynamics can be multiplied by curl operator to give the Shear wave equation, or it can be multiplied by divergence operator to give the acoustic wave equation. Both shear and acoustic wave equations are very similar except for the expressions for their speed. This expression is what differentiates both the equations and is responsible for the shape that they take. I say this because I found both expressions to be exactly same except that the acoustic wave also uses Lame's parameter $\lambda$ in addition to shear modulus $\mu$ and mass density $\rho$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My main concern is to know which waves are propagating in my mesh if I am simulating the elastodynamic or elastic wave equation. Is it both 'p' and 's'?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2016-08-09T03:56:26.830" Title="Seismic Wave modelling: Elastic Wave or Acoustic Wave?" Tags="|finite-element|wave-propagation|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24659" PostTypeId="2" ParentId="24612" CreationDate="2016-08-09T06:55:29.577" Score="0" Body="&lt;p&gt;The standard way to find the eigenvalues of the Schrodinger equation is called &quot;imaginary time propagation&quot;.&#xA;You change the coordinates, t=-i\tau, and integrate in the \tau direction.&#xA;Any random initial condition will converge to the lowest energy eigenstate.&#xA;The resulting equation is solved by splitting methods:&#xA;First propagate the kinetic energy using Fast Fourier Transforms and then propagate the potential. This is possible since the solution decays fast (exponentially) as |x| grows and you can effectively replace your boundary conditions by periodic ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some matlab code for a second order method&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% in order to set up the problem, you need a spatial grid and a momentum grid, &#xA;% since you will be using FFTs, the number of grid points should be a power of 2&#xA;Dx  = 0.01; &#xA;x   = [-1:Dx:1-Dx];&#xA;&#xA;% Now you define the eigenvalues in Fourier space&#xA;k           = zeros(1,Len).';&#xA;k(2:Len/2)  = 1:Len/2-1;                %  (x, 1, ..., Len/2-1, x, x, ..., x)&#xA;k((Len/2+1):Len) = -k((Len/2+1):-1:2);  %  (x, ...,x, -Len/2+1, ..., -1)&#xA;k(Len/2+1) = 0;&#xA;&#xA;% from this you derive the momentum operator in Fourier space&#xA;p = -1i k.^2;&#xA;&#xA;% This allows you to define the full problem, kinetic energy and potential, adjust as you please&#xA;E_kin = p.^2 / 2/mass;&#xA;V     = x.^4 - x.^2 ;&#xA;&#xA;% you want to solve e^(h(T+V))Psi&#xA;% you approximate by the Strang splitting e^(h/2 V)e^(h T)e^(h/2 V)&#xA;for j=1:1:n&#xA;        Psi = ifft( Psi);&#xA;        Psi =  exp(- 1i*h* E_Kin ) .* Psi;&#xA;        Psi =  fft( Psi);&#xA;    % Apply potential V&#xA;        Psi = exp(  -1i*h/2* V)&#xA;    % Normalize the result in each step&#xA;         Psi = Psi / ( norm(Psi,2)^2*Dx);&#xA;   end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;More sophisticated (higher order) methods can be found here, as well as many references to the problem: &lt;a href=&quot;http://personales.upv.es/serblaza/2013JChemPhys.pdf&quot; rel=&quot;nofollow&quot;&gt;http://personales.upv.es/serblaza/2013JChemPhys.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you want more eigenstates, there are several procedures:&#xA;1: iteration. compute the first eigenstate. Then, you restart the procedure, but in every step, you subtract the projection to this state.&#xA;E.g. for the second state psi_2, in each step you write  psi_2 = psi_2 - *psi_1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a complete alternative, you can discretize the Laplacian using finite differences (or use again FFTs) and then use the inverse power method.&#xA;given your Hamiltonian H, &#xA;iterate over n:&#xA;    (H-\lambda^(-1)I)^(-1)\psi_(n+1) = \psi_n&#xA;for some guess of the eigenvalue \lambda.&#xA;You can find details on wikipedia.&lt;/p&gt;&#xA;" OwnerUserId="21094" LastActivityDate="2016-08-09T06:55:29.577" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24660" PostTypeId="2" ParentId="24646" CreationDate="2016-08-09T07:20:27.903" Score="-1" Body="&lt;p&gt;The reason might be that your method is not as sophisticated. I would bet that NIntegrate uses an adaptive method and splits the integrals in parts which are then refined according to error estimates. If you do not improve your algorithm, you probably won't beat it.&#xA;As someone pointed out, mupadex is the costly part. This comes from the number of function evaluations (specifically hypergeom). You can reduce them by some divide-and-conquer algorithm. It is not a good idea to use such a high number of Gauss-nodes anyway. A better one is as mention to resort to a lower order integrator and refine the grid.&#xA;If you want code, look at the matlab function integral2Calc.m for the actual algorithm (integral2t) or at its wrapper integral2.m&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case anyone wants to run the code above, here is a working leg function. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [x, w] = leg(a,b,N)&#xA;% Generates the abscissa and weights for a Gauss-Legendre quadrature.&#xA;beta = (1:N-1)./sqrt(4*((1:N-1)).^2-1);&#xA;[w,x] = eig(diag(beta,-1)+diag(beta,1));&#xA;absc = diag(x);&#xA;wght = 2*w(1,:)'.^2;&#xA;&#xA;% Linear map from[-1,1] to [a,b]&#xA;x=(a*(1-absc)+b*(1+absc))/2;      &#xA;w=wght;&#xA;&#xA;x=x';&#xA;w=w';&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21094" LastActivityDate="2016-08-09T07:20:27.903" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24661" PostTypeId="1" CreationDate="2016-08-09T08:55:11.653" Score="4" ViewCount="100" Body="&lt;p&gt;I have the following curve, which is calculated on a large number of points and shows smooth behaviour when viewed from distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oTk3k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oTk3k.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the derivative (shown below) exhibits artificial oscillations which are inevitable due to complications in my method of numerical calculation. I know about the correct result and its derivative (shown below in red), and want to eliminate these sharp peaks and fluctuations with a post-processing (filtering) step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/C9v9P.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C9v9P.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a reasonable filtering process to eliminate the fluctuations in derivative and get the correct result (the red curve)? How can I decide on the details of such filter?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-08-25T09:12:26.863" Title="Post-processing the noisy results of numerical simulation" Tags="|numerics|curve-fitting|oscillations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24663" PostTypeId="1" AcceptedAnswerId="24690" CreationDate="2016-08-09T15:40:23.013" Score="1" ViewCount="155" Body="&lt;p&gt;If $w$ is the vector of conservative variables, $f=f(w)$ the flux function, I think have read somewhere (I can not find it anymore) that the inflow boundary $\Sigma$_ is characterized by: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Sigma_{-} = \left\{x, \nabla_{w}f \cdot  \mathbf{n} &amp;lt; 0 \right\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the exact definition of $\nabla_{w}f$ and is it not $\nabla_{u}f$ instead, where $u$ is the velocity ?  &lt;/p&gt;&#xA;" OwnerUserId="18742" LastEditorUserId="18742" LastEditDate="2016-08-10T06:48:01.547" LastActivityDate="2016-08-11T13:35:13.127" Title="Definition of inflow boundary in CFD" Tags="|finite-element|fluid-dynamics|boundary-conditions|computational-physics|finite-volume|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24664" PostTypeId="2" ParentId="11338" CreationDate="2016-08-09T15:48:11.153" Score="0" Body="&lt;p&gt;I did some research on the topic a few years ago and my conclusion was that iterative methods have a more promising future if the technology keeps drifting towards decentralized computation (as it is doing).&#xA;Apart from direct and iterative methods, there has been some advances in solving very large matrices with Monte Carlo methods, but in my opinion the best bet is to go to Hadoop implementations.&#xA;Below is a link I found to a github project:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JingenXiang/MatrixInversion&quot; rel=&quot;nofollow&quot;&gt;https://github.com/JingenXiang/MatrixInversion&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here you can find more details on how it is done:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cs.uwaterloo.ca/~ashraf/pubs/hpdc14matrix.pdf&quot; rel=&quot;nofollow&quot;&gt;https://cs.uwaterloo.ca/~ashraf/pubs/hpdc14matrix.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21268" LastActivityDate="2016-08-09T15:48:11.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24665" PostTypeId="1" CreationDate="2016-08-09T15:59:07.967" Score="1" ViewCount="107" Body="&lt;p&gt;I'm doing PML simulation, and while I was looking for correct geometry for my layers of propagation, I got a question.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;My device has two vibration sources, and this vibration will propagate through the medium and diminish.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am trying to draw my medium as a cylinder or hemisphere, but since there are two vibration sources, the center of cylinder/hemisphere is an equal distance between two vibrating sources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is the picture of how it looks like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/irHUt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/irHUt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a 2D version, but I am simulating this device in 3D version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vibration source has out of plane waveform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it okay to simulate like this? I'm afraid this will give incorrect result since vibration sources are not at the center of cylinder/hemisphere.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Or, do I have to cut my device in half and put only one vibrating source on the propagating medium? (symmetric condition should be imposed on the half of resonator device)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Trv2B.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Trv2B.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This second method, I think it seems unrealistic because it excludes the presence of another vibrating source which will cause interference, however, I saw a number of papers simulate their device with two vibrating sources such way.&lt;/p&gt;&#xA;" OwnerUserId="19465" LastEditorUserId="20688" LastEditDate="2019-06-22T18:58:04.727" LastActivityDate="2019-06-22T18:58:04.727" Title="Perfectly matched layer simulation with two vibrating sources" Tags="|finite-difference|boundary-conditions|wave-propagation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24667" PostTypeId="1" AcceptedAnswerId="24671" CreationDate="2016-08-09T15:20:16.030" Score="1" ViewCount="2051" Body="&lt;p&gt;For two given vectors, not necessairly the same length, $x=(x_1,\dots,x_N)$ and $s=(s_1,\dots,s_M)$ I would like to compute the following term as efficiently as possible in R. For $j=1, \dots, N$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ H_j:=\sum_{i=1}^Ml_i(x_j)$$ &#xA;where &#xA;$$l_i(x_j)= \prod_{l = 1,l \not= j}^M \frac{x_j-s_l}{s_i-s_l}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;below is some test code which is very inefficient. Howeve, I dont see how I can speed this up using vectorized style or another smart trick.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- rnorm(1000)&#xA;s &amp;lt;- rnorm(10,2,0.2)&#xA;&#xA;l &amp;lt;- rep(0,length(s))&#xA;output &amp;lt;- rep(0,length(x))&#xA;for(j in 1:length(x)){&#xA;  xj &amp;lt;- x[j]&#xA;  for(i in 1:length(s)){&#xA;    s.i.dropped &amp;lt;- s[-i]&#xA;    l[i] &amp;lt;- prod((xj-s.i.dropped)/(s[i]-s.i.dropped))&#xA;  } &#xA;  output[j] &amp;lt;- sum(l)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21291" OwnerDisplayName="math" LastActivityDate="2016-08-15T14:21:44.473" Title="efficient way to compute lagrange polynomials" Tags="|r|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24670" PostTypeId="2" ParentId="24665" CreationDate="2016-08-10T00:50:59.667" Score="2" Body="&lt;p&gt;I believe it is OK to use the PML the first way you described. It's not necessary to put the source at the center, otherwise PML is not better than many other absorbing boundary conditions. The PML even doesn't need to be a spherical shell, which is just a choice of convenience. &lt;/p&gt;&#xA;" OwnerUserId="8265" LastActivityDate="2016-08-10T00:50:59.667" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24671" PostTypeId="2" ParentId="24667" CreationDate="2016-08-10T02:27:16.607" Score="2" Body="&lt;p&gt;Why do it yourself when experts have already done it for you? See &lt;a href=&quot;https://cran.r-project.org/web/packages/polynom/index.html&quot; rel=&quot;nofollow&quot;&gt;polynom&lt;/a&gt; and &lt;a href=&quot;https://cran.r-project.org/web/packages/PolynomF/index.html&quot; rel=&quot;nofollow&quot;&gt;polynomF&lt;/a&gt; for specific implementations in R.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-08-10T02:27:16.607" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24672" PostTypeId="1" CreationDate="2016-08-10T04:30:16.020" Score="3" ViewCount="108" Body="&lt;p&gt;In computational physics, the vast majority of papers has the following structure&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They propose some new algorithm, or improvement to an existing one&lt;/li&gt;&#xA;&lt;li&gt;They give numerical examples, ideally comparing relevant characteristics of their new method with existing methods&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am wondering which journals would publish a theoretical analysis of an existing algorithm, e.g. &quot;Algorithm X was previously hypothesized to have stability condition Y (but this was not proven), and this is indeed true because (math proof)&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a paper would deviate from the structure above in that it neither proposes a new algorithm, nor provides numerical examples. It &quot;merely&quot; proves that an existing algorithm has certain characteristics.&lt;/p&gt;&#xA;" OwnerUserId="21290" LastActivityDate="2016-08-11T00:51:30.617" Title="Journals that publish theoretical analyses of existing algorithms" Tags="|computational-physics|publications|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24673" PostTypeId="1" CreationDate="2016-08-10T05:23:25.913" Score="1" ViewCount="672" Body="&lt;p&gt;From &lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch19.d/IFEM.Ch19.pdf&quot; rel=&quot;nofollow&quot;&gt;section 19.4.1, Rank Sufficiency&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The element stiffness matrix must not possess any zero-energy&#xA;  kinematic mode other than rigid body modes.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This can be mathematically express as follows....&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have no idea what does this section about Rank Sufficiency mean. What is rank sufficiency in physical term, and why is it so important for stiffness matrix? If &lt;em&gt;rank deficiency&lt;/em&gt; occurs, then what will happen to the stiffness matrix and the solution of it? &lt;/p&gt;&#xA;" OwnerUserId="690" LastActivityDate="2016-08-10T05:23:25.913" Title="Rank sufficiency concept in Finite Element" Tags="|finite-element|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24674" PostTypeId="2" ParentId="24406" CreationDate="2016-08-10T07:53:40.623" Score="5" Body="&lt;p&gt;The biggest problem in answering to this question is that it's not as straightforward as you might think to see the physics in the equations until you see them fully developed, and even then you're doing your best to assign a meaning. As a graduate student in mechanical engineering, this was hard for me because I was used to using my physical intuition to guide me through the math forest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a FEM problem applied to modal analysis, you basically start with a strong form equation. For example, a free vibration of an elastic rod is given by:&#xA;$$&#xA;\lambda \rho u + (Eu,_x),_x = 0,\ x \in (0, L)&#xA;$$&#xA;Here, $\rho$ is the density, $E$ is the modulus of elasticity, and $u$ is the displacement. Let's apply a fixed boundary condition ($u(L) = 0$) and a free boundary condition (e.g. $-Eu,_x(0)=0$). You can transform it into the equivalent weak form by using a weighting function $w$:&#xA;$$&#xA;\int_0^Lw,_xEu,_xdx\ -\lambda\int_0^Lw\rho u\, dx = 0&#xA;$$&#xA;This is where the math ends up losing it's physical interpretation. From here (and skipping a lot of rigor for brevity in this answer), we would basically restrict $w$ in a subset of the Sobolev space, apply the Galerkin approximation by discretizing the domain which creates the finite dimensional weighting function $w^h$ and solution $u^h$, make $u^h$ the sum of a Sobolev function $v^h$ and a given function $g^h$, and then define the functions $w^h$ and $v^h$ as a finite sum of linearly independent basis functions (i.e $v^h = \sum_{A=1}^N c_A N_A$ and $w^h = \sum_{B=1}^N d_A N_A$). By applying some linear algebra theorems and rearranging a lot, you eventually get the following equation:&#xA;$$&#xA;(K-\lambda^h_kM)\psi_k = 0&#xA;$$&#xA;Here, $\psi_k$ is the kth eigenvector, which represents the displacement. It's hard to get an appreciation for the math that I've glossed over. The book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0486411818&quot; rel=&quot;nofollow&quot;&gt;The Finite Element Method&lt;/a&gt; has the full explanation for how this is developed (Chapter 1 goes over the basics of the method, and Section 7.3 develops the frequency analysis portion).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to clarify, we've simply called these matrices K and M, and the form of the equation bears a resemblance to a simple modal analysis for a beam, so we simply assign that as the physical interpretation. Really, the K and M matrices are highly dependent on the mesh and how the elements are connected. These large matrices are constructed from each element in the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discretization is really important. Physically for this problem, the nodes represent points at which we're calculating the displacement, and the elements connected to that node restrict its movement under an applied force. You can also think of the mesh for this problem as having $n$ number of rods connected to each other. In that way, each rod piece only really affects those it's connected to. Each element is represented by a small matrix. The element matrices are calculated like this:&#xA;$$&#xA;m_{ab}^e = \int_{\Omega^e} N_a \rho N_b\,d\Omega \\&#xA;k_{ab}^e = \int_{\Omega^e} N_{a,x} E N_{b,x}\,d\Omega&#xA;$$&#xA;Here, $\Omega^e$ is the domain of the element itself. When the element matrices get assembled into the global matrix, some elements in the large matrix will have terms added from several smaller matrices. This is a representation of how the topology from the mesh affects the global matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $N_a$ and $N_b$ symbols are the basis functions that span the subset of the Sobolev space we chose. These are very important because different basis functions changes the solution. We can choose to use linear or parabolic functions, or even different types of functions, like Lagrange or even B-spline basis functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, I don't know how much FEM theory the OP has, and it's been really hard to try to sum up how all this works. This may not be sufficient, but hopefully it serves to highlight some of the theory behind the FE method.&lt;/p&gt;&#xA;" OwnerUserId="21131" LastEditorUserId="21131" LastEditDate="2016-08-10T22:55:46.350" LastActivityDate="2016-08-10T22:55:46.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24675" PostTypeId="1" CreationDate="2016-08-10T16:39:48.587" Score="3" ViewCount="93" Body="&lt;h1&gt;Statement of the problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I need to (numerically) solve an  eigenproblem of the type &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\omega^2\mathcal{D}_1\vec{x}=\mathcal{D}_2\vec{x}$$&#xA;on the interval $[-1,1]$, where $\mathcal{D}_1$ and $\mathcal{D}_2$ are differential operators with a generalised boundary condition of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega^2\mathcal{D}_3\vec{x}(±1)+a~\omega\mathcal{D}_4\vec{x}(±1)+b~\mathcal{D}_5\vec{x}(±1)=0~.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal{D}_3$, $\mathcal{D}_4$ and $\mathcal{D}_5$ are more differential operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The critical feature here being the fact that the eigenvalue appears in the boundary condition. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;My personal take on it&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;SLEPc offers a wide range of method for dealing with generalised eigenvalue problems including the so-called polynomial eigenvalue problem. I used a &lt;a href=&quot;https://arxiv.org/abs/1202.1347&quot; rel=&quot;noreferrer&quot;&gt;spectral representation&lt;/a&gt; to cast my differential problem into a regular quadratic matrix eigenvalue problem of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(A\omega^2+B\omega+C)\vec{x}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used row replacement to include the boundary condition into my matrices. Since the first power of $\omega$ only appears in the boundary condition, the $B$ matrix is mostly empty. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For clarity, here is an idea of what the matrices look like &#xA;&lt;a href=&quot;https://i.stack.imgur.com/UzJmN.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UzJmN.png&quot; alt=&quot;Matrices&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect that, if everything is done correctly, feeding this system into a dedicated solver for quadratic eigenvalue problems should spit out the expected eigenvalues (in some easy case, my particular system has analytical solutions). However, I haven't been able to recover these. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;My question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I have already spent a decent amount of time trying to find bugs in my code and will continue to do so. But in the meantime, I figured I might ask if anyone had a similar problem and had a different approach. I am especially anxious to know if perhaps there are some results unknown to me that might render my whole approach too naive and hopeless. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks lads !&lt;/p&gt;&#xA;" OwnerUserId="19584" LastActivityDate="2016-08-10T16:39:48.587" Title="Differential eigenproblem with eigenvalue in boundary condition" Tags="|boundary-conditions|eigensystem|spectral-method|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24678" PostTypeId="1" CreationDate="2016-08-10T22:00:02.700" Score="2" ViewCount="286" Body="&lt;p&gt;I am trying to answer a question about a finite difference scheme. I need to show that the method is stable in the L1-norm. I can't find a single definition of what that means, so does anyone have a source or know for a fact what the definition is?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm guessing that if we denote the operator corresponding to the scheme as $H$, then we need to show that&#xA;$$||H^n||_1 \le C$$&#xA;where $C&amp;gt;0$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7322" LastEditorUserId="123" LastEditDate="2016-12-13T10:39:34.560" LastActivityDate="2016-12-13T10:39:34.560" Title="What does it mean for a finite difference scheme to be $L^1$-stable?" Tags="|finite-difference|stability|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24679" PostTypeId="2" ParentId="24663" CreationDate="2016-08-10T23:08:16.990" Score="1" Body="&lt;p&gt;I have come across a similar expression in &lt;a href=&quot;https://hal.archives-ouvertes.fr/inria-00603625/document&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; by a group at INRIA. For more general discussion refer to &lt;a href=&quot;http://www.mathematik.uni-dortmund.de/~kuzmin/Transport.pdf&quot; rel=&quot;nofollow&quot;&gt;section 1.2.4&lt;/a&gt; in this notes from Uni Dortmund.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial \Omega_{-} = \left\{x \in \partial d\Omega | \mathbf{n}\cdot\nabla_{u}f    &amp;lt; 0 \right\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my understanding $\nabla_{u} f$ denotes the gradient. It's standard practice to use $u$ for velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Peter Frolkovič's answer is accurate. Corrected my erroneous interpretation.&lt;/p&gt;&#xA;" OwnerUserId="7325" LastEditorUserId="7325" LastEditDate="2016-08-11T13:35:13.127" LastActivityDate="2016-08-11T13:35:13.127" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24681" PostTypeId="2" ParentId="24643" CreationDate="2016-08-11T06:09:53.003" Score="1" Body="&lt;p&gt;Assuming your old domain is $x\in[0,L]$, then $y(t)\in[-bt,L-bt]$.&#xA;In your case, you could either solve it by hand since it is just the heat equation or you directly solve the original equation numerically and avoid the change of variables altogether (it doesn't change the computational cost anyway). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect that rather than your domain, the boundary conditions are of importance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you stay far enough from the boundary until your final time, you could just assume a fixed mesh for $y\in[0,L]$ and shift the solution which would be the case for an open boundary, where both domains are equal $D(x)=D(y)=(-\infty,\infty)$&lt;/p&gt;&#xA;" OwnerUserId="21094" LastActivityDate="2016-08-11T06:09:53.003" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24682" PostTypeId="1" CreationDate="2016-08-11T06:42:15.483" Score="-3" ViewCount="203" Body="&lt;p&gt;--&gt; why computers not used high level language as a first Language what are the main reasons? &lt;/p&gt;&#xA;" OwnerUserId="21306" LastActivityDate="2016-08-11T09:24:19.223" Title="Why Binary language is used as a first language for the Computer Systems not a high level Language?" Tags="|software|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ClosedDate="2016-08-12T13:59:33.117" ContentLicense="CC BY-SA 3.0" />
  <row Id="24684" PostTypeId="1" CreationDate="2016-08-11T07:06:53.273" Score="0" ViewCount="1389" Body="&lt;p&gt;Say for heat equation or Burgers equation with nonlinear boundary condition. Exact solution is unknown. &#xA;So I am taking for small mesh size the discrete solution as exact solution. Then how to write the coding part for $L^2$ and $H^1$ error in Matlab?&lt;/p&gt;&#xA;" OwnerUserId="21307" LastActivityDate="2016-08-11T15:24:41.793" Title="How to find order of convergence in finite element method computationally when exact solution is unknown for time dependent problem" Tags="|finite-element|nonlinear-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24685" PostTypeId="1" AcceptedAnswerId="24686" CreationDate="2016-08-11T08:04:56.633" Score="2" ViewCount="77" Body="&lt;p&gt;I'm reading the following &lt;a href=&quot;http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2529684&quot; rel=&quot;nofollow&quot;&gt;paper&lt;/a&gt; (Grezlak and Oosterlee) and I have a specific question to a sentence on page 5. I quote:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Since the mapping $y=g(x)$ is bijective and $g(x)$ is strictly&#xA;  increasing, so is $g^{-1}(y)$. This implies that the arguments $x$ can&#xA;  be obtained by the inverse interpolation of $g(x)$ against $y$, which&#xA;  can be done at essentially no cost.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For given points $x_i,y_i$ they define $g$ to be the lagrange polynomial:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$g(x) = \sum_{i=1}^Ny_il_i(x)$$&#xA;where $l_i(x)=\prod_{j=1,j\not=i}\frac{x-x_j}{x_i-x_j}$. They cite a paper (E. Michalup. On inverse linear interpolation. Scandinavian Actuarial Journal 1950) about this inverse interpolation which is unfortunatelly not available for free. I'm working with R and was browsing the web about inverse linear interpolation and found the following &lt;a href=&quot;https://stat.ethz.ch/R-manual/R-devel/library/stats/html/NLSstClosestX.html&quot; rel=&quot;nofollow&quot;&gt;function&lt;/a&gt;. However, it seems to be quite costly. Even worse it just takes a single numeric value as input (yval), which means I need to call the function as many time as inverse I want to calculate. Is there another way to do it or a better function in R?&lt;/p&gt;&#xA;" OwnerUserId="21291" LastActivityDate="2016-08-11T08:47:33.527" Title="How to invert a lagrangian polynomial" Tags="|r|inverse-problem|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24686" PostTypeId="2" ParentId="24685" CreationDate="2016-08-11T08:47:33.527" Score="2" Body="&lt;p&gt;Without knowing anything about inverse interpolation, this problem should be a straight-forward numerical task given the information about $g(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, $g(x)$ is said to be monotonously increasing.&#xA;Therefore, given $y$, find two nodes s.t. $g(x_i)\leq y\leq g(x_{i+1})$ [this is simple since all is ordered].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From there, use any root-finder which should converge rapidly.&#xA;This method has the drawback that you have to solve a system for every point $y$.&#xA;What you can do to speed it up is to add any evaluated point of $g(x)$ in the root-finding process to the list which will decrease the number of future iterations assuming you start from a low-degree polynomial, i.e., small $N$.&lt;/p&gt;&#xA;" OwnerUserId="21094" LastActivityDate="2016-08-11T08:47:33.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24687" PostTypeId="2" ParentId="24682" CreationDate="2016-08-11T09:24:19.223" Score="1" Body="&lt;p&gt;In (very) short : Because basically a computer is composed of multiple electronic devices that consistently switch between two states (see transistors). For a human, this can be idealized by a state 0 or a state 1 from which the binary is derived. This is the core language, close to processor instructions, only then appears low-level languages and high level languages that provide more abstraction level.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-08-11T09:24:19.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24688" PostTypeId="2" ParentId="24684" CreationDate="2016-08-11T09:48:46.800" Score="1" Body="&lt;p&gt;If the exact solution is unknown, then yes you have to take the smaller grid as reference. Then you run your simulation with different mesh size, each one varying by a factor 2 and you compute the norm $L^2$ as :&#xA;$$&#xA;\vert \vert u-u_{ref}\vert \vert_{L^2} = \sum_{i=1}^n \sqrt{(u(i)-u_{ref}(i))^2}&#xA;$$&#xA;with $n$ the number of grid nodes for the considered mesh. Finally, you plot this quantity in respect to the factor size in a log-log plot and the slope of the curve will give you the order. I can suggest you to look at this NASA website : &lt;a href=&quot;http://www.grc.nasa.gov/WWW/wind/valid/tutorial/spatconv.html&quot; rel=&quot;nofollow&quot;&gt;http://www.grc.nasa.gov/WWW/wind/valid/tutorial/spatconv.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Don't remember for $H^1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An applied mathematician tolds me once that the order of convergence for a specific method is dependent on the choice of the norm, it may be more relevant for a specific case to compute the order with the $L^{\infty}$ norm or the $L^1$ norm or whatever norm you decide to define. If anyone has an expertise here, it would be great to have a detailed explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; : see also &lt;a href=&quot;https://www.kth.se/social/upload/52ea4f1ff2765454c236fe79/ConvRate.pdf&quot; rel=&quot;nofollow&quot;&gt;https://www.kth.se/social/upload/52ea4f1ff2765454c236fe79/ConvRate.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="20454" LastEditDate="2016-08-11T15:24:41.793" LastActivityDate="2016-08-11T15:24:41.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24689" PostTypeId="2" ParentId="24463" CreationDate="2016-08-11T11:08:08.667" Score="-1" Body="&lt;p&gt;There is a &lt;a href=&quot;http://www.springer.com/de/book/9783319168739&quot; rel=&quot;nofollow&quot;&gt;book&lt;/a&gt; on the finite volume method (FVM), which is widely used to solve flow problems. Aside from presenting the theory behind the method, the authors demonstrate the finite volume method on a Matlab-based FVM code and on OpenFOAM, which is a C++ library implementing the finite volume method.&lt;/p&gt;&#xA;" OwnerUserId="3510" LastActivityDate="2016-08-11T11:08:08.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24690" PostTypeId="2" ParentId="24663" CreationDate="2016-08-11T11:24:55.650" Score="1" Body="&lt;p&gt;If $w=(w_1,w_2,\ldots,w_l)$ represents a vector of $l$ variables and $f$ is a scalar function, then $\nabla_w f$ should stand for the gradient with respect to variables $w$, i.e. it is the vector function $(\partial_{w_1} f,\partial_{w_2} f,\ldots,\partial_{w_l} f)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $l&amp;gt;1$ then typically $f$ is a vector function of $l$ components and one prefers a notation $\bf{f}$ or $\vec{f}$ (me too). Then $\nabla_w \vec{f}$ is a Jacobian (matrix) given by entries $\partial_{w_l} f_k$ where $f_k$ is the $k$-th component of $\vec{f}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are main tools when considering a system of conservation laws like&#xA;$$&#xA;\partial_t w + \nabla \cdot \vec{f} = 0&#xA;$$&#xA;where now $\nabla$ stands for standard divergence operator with respect to space variables $x$, i.e. $w=w(x,t)=(w_1(x,t),w_2(x,t),\ldots,w_l(x,t))$. The system above can be rewritten (if derivatives exist) to the system&#xA;$$&#xA;\partial_t w + \nabla_w \vec{f} \cdot \nabla w = 0&#xA;$$&#xA;that can be viewed as nonlinear advection equation, so $\nabla_w \vec{f}$ plays a role of (solution dependant) velocity. In this sense you might consider the definition of inflow boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system of nonlinear conservation laws is a nontrivial topic, you can learn it e.g. from the book of LeVeque on FVM for Hyperbolic Problems.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-08-11T11:24:55.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24691" PostTypeId="1" CreationDate="2016-08-11T15:13:28.013" Score="1" ViewCount="156" Body="&lt;p&gt;I would like to generate an isotropic gaussian random field  described by a power spectrum $P(k)$ on a 3D grid which represents spherical polar coordinates (i.e., the angular separation between pixels at each slice in the radial direction is equal and each pixel represents a solid angle).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently generating a random field on a cartesian grid by&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Filling a 3D grid in Fourier space $(k_x, k_y, k_z)$ with random numbers drawn from a normal distribution with standard deviation $\sigma = \frac{P(k)}{2}$ where $k = \sqrt{k_x^2 + k_y^2 + k_z^2}$&lt;/li&gt;&#xA;&lt;li&gt;Imposing hermitian symmetry to ensure a real field&lt;/li&gt;&#xA;&lt;li&gt;Fourier transforming into real space&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My only thought is that I could do this process on a high resolution cartesian grid and average into spherical bins, though this seems convoluted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more efficient way that I could generate this directly?&lt;/p&gt;&#xA;" OwnerUserId="21311" LastActivityDate="2016-08-11T15:13:28.013" Title="Generate gaussian random field in spherical polar coordinates" Tags="|random-number-generation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24693" PostTypeId="2" ParentId="23913" CreationDate="2016-08-11T21:27:08.903" Score="1" Body="&lt;p&gt;This isn't an uncommon problem to have. You might consider using different mesh spacing for different variables/derivatives or a staggered method to alleviate the poor conditioning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Better yet, if your eigenvalues are all within four orders of magnitude, the problem really isn't that bad. I suspect the LAPACK solution (if you used double precision) is correct, though you should verify this yourself. Other methods are available for these poorly conditioned problems and the QR methods LAPACK uses aren't especially useful if you're only looking for a few eigenvalues and vectors. Consider ARPACK or SLEPc if this is the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: be wary of spurious eigenvalues in these types of problems! These can often arise during the implementation of boundary conditions.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-08-11T21:27:08.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24694" PostTypeId="1" CreationDate="2016-08-12T01:46:04.660" Score="1" ViewCount="89" Body="&lt;p&gt;What are the implications of applying a direction-splitting within each stage of an SSP-RK scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, given a standard advective transport type equation:&#xA;$$&#xA;\partial_{t}Q + \operatorname{div}(\mathbf{u}Q) = 0&#xA;$$&#xA;it's popular to make use of an explicit SSP-RK style time-stepping scheme, such as the well-known SSP-RK2 method:&#xA;$$&#xA;Q^{(1)} = Q^{(n)} - \Delta t \operatorname{div}(\mathbf{u}Q^{(n)}) &#xA;\\[1ex]&#xA;Q^{(2)} = \tfrac{1}{2}Q^{(n)} + \tfrac{1}{2}Q^{(1)} - \tfrac{1}{2}\Delta t \operatorname{div}(\mathbf{u}Q^{(1)})&#xA;\\[1ex]&#xA;Q^{n+1} = Q^{(2)}&#xA;$$&#xA;But if the flux terms  can only be evaluated via a direction-splitting:&#xA;$$&#xA;\operatorname{div}(\mathbf{u}Q) = \partial_{x}(uQ) + \partial_{y}(vQ)&#xA;$$&#xA;what are the implications of advancing each RK stage according to a simple direction split:&#xA;$$&#xA;Q^{(1a)} = Q^{n}&#xA;\\[1ex]&#xA;Q^{(1b)} = Q^{(1a)} - \Delta t \, \partial_{x}(uQ^{(1a)})&#xA;\\[1ex]&#xA;Q^{(1c)} = Q^{(1b)} - \Delta t \, \partial_{y}(vQ^{(1b)})&#xA;\\[2ex]&#xA;Q^{(2a)} = \tfrac{1}{2}Q^{n} + \tfrac{1}{2}Q^{(1c)}&#xA;\\[1ex]&#xA;Q^{(2b)} = Q^{(2a)} - \tfrac{1}{2}\Delta t \, \partial_{y}(vQ^{(2a)})&#xA;\\[1ex]&#xA;Q^{(2c)} = Q^{(2b)} - \tfrac{1}{2}\Delta t \, \partial_{x}(uQ^{(2b)})&#xA;\\[2ex]&#xA;Q^{n+1} = Q^{(2c)}&#xA;$$&#xA;Is the order of the method reduced? Does it loose its SSP property? Is it possible to combine such an approach with SSP-RK3 to get a third-order accurate method?&lt;/p&gt;&#xA;" OwnerUserId="1139" LastActivityDate="2016-08-12T01:46:04.660" Title="Direction-splitting for SSP-RK schemes" Tags="|pde|advection|operator-splitting|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24695" PostTypeId="1" CreationDate="2016-08-12T04:38:50.507" Score="1" ViewCount="96" Body="&lt;p&gt;I have used &lt;a href=&quot;https://github.com/dchandan/PySHTOOLS&quot; rel=&quot;nofollow&quot;&gt;pyShtools&lt;/a&gt; in the past to expand scalar functions to spherical harmonics and to synthesize band-limited representations of them. However, I am not too sure how to achieve this for a vector field on the surface of a sphere, where $u(\theta, \phi)$ and $v(\theta, \phi)$ are the co-latitudinal and longitudinal components, respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also used &lt;a href=&quot;https://www2.cisl.ucar.edu/resources/legacy/spherepack/documentation&quot; rel=&quot;nofollow&quot;&gt;spherepack&lt;/a&gt; in the past for decomposing vector fields into poloidal/toroidal components; but unfortunately I couldn't find what I am looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would very much appreciate any help in pointing me in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="8245" LastActivityDate="2016-08-12T04:38:50.507" Title="Spherical Harmonics: band-limited representations of a vector field on a sphere" Tags="|software|fourier-analysis|fourier-transform|spherical-harmonics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24696" PostTypeId="1" AcceptedAnswerId="24705" CreationDate="2016-08-12T06:42:02.410" Score="1" ViewCount="70" Body="&lt;p&gt;I am trying to find error estimation based on weighted residual technique:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q(e_h)=\sum_{k \in {\cal T}}\eta_k, $$&#xA;where&#xA;$$\eta_k=\int_kR(z-v_h)d\Omega+\int_{\partial k}J(z-v_h)ds,$$&#xA;or in the weak form&#xA;$$\eta_k=\int_k l(z-v_h)d\Omega+\int_k \nabla u_h.\nabla(z-v_h)d\Omega,$$&#xA;where $u_h$ is the solution of primal problem $a(u,v)=l(v),$ and $z$ is the solution of the dual problem $a(z,v)=Q(v),$ also $Q(v)=|\Omega|^{(-1)}(1,v),$ where $v$ is the test function. Moreover $R$ and $J$ are residual and jump respectively.&#xA;Using approximation  $z_h=\sum_i \phi_i*z_i$ (instead of z) at Gaussian quadrature point leads to a scalar where using $\phi$ as test function at any point gives a vector, but $(z-v_h)$ should be a scalar not a vector. How can I solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2016-08-12T07:03:58.997" LastActivityDate="2016-08-13T03:39:39.197" Title="How can I compute the difference between shape function and dual solution in dwr?" Tags="|finite-element|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24697" PostTypeId="2" ParentId="24406" CreationDate="2016-08-12T12:50:56.230" Score="2" Body="&lt;p&gt;Finite Element Analysis is a mathematical tool very extended among engineers. However, after more than a year researching on the topic of computer simulation, where FEA plays such an important role, I couldn't yet find a satisfactory explanation on how they really really work...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main background of FEM is that of structural engineering in the 60s. Engineers are very practical people, so initially they devised a system which allowed them to set algebraic equations where the relations between different points or &quot;nodes&quot; of their structures could be set.&#xA;These algebraic relations were further proven to be of many different types, not only structural, so also thermal relations could be formulated and many others: all one needed to analyse was a proper discretization of the space in the form of a mesh with nodes to relate to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For our particular case of  structural engineering the procedure can be summarized in its three main steps, so the main ideas come up in a graphic taste:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;STEP 1: Discretization&lt;/strong&gt;&#xA;In this basic yet classic example, I have chosen to divide my sample structure in only 5 elements where nodes are clearly identifiable in the meetings between beams. There are four nodes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pQmPL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pQmPL.jpg&quot; alt=&quot;A sample structure (mesh) and its topology table&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6OxRO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6OxRO.jpg&quot; alt=&quot;Degrees of Freedom&quot;&gt;&lt;/a&gt;&#xA;(Degrees of Freedom)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each node will have 6DOF (Degrees Of Freedom): three for linear displacement on each axis (X,Y,Z) and three for rotational around each axis (X,Y,Z), because we are working on 3D. Many examples available provide the more &quot;simple&quot; 2D situation, but in my opinion this only complicates things further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the nodes are located and there is a network of how they relate to each other, we can consider we have a mesh. In our example, the correlation is depicted in the table: The elements serve to &quot;link&quot; nodes to each other. The table establishes a &quot;topology&quot; for the nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;STEP 2: Element characterization and shape functions&lt;/strong&gt;&#xA;In this step is where FEM formulation and literature get really really awkward and nasty. In fact, this is the core of everything and where FEM differs from other ways of solving PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pDWMf.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pDWMf.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In reality, and despite its mathematical complexity, what we are looking for is a way of characterizing the material properties of the element. For such purpose, the method requires that the behaviour of those links among nodes obeys some formula. This formula is the actual &lt;a href=&quot;http://stochasticandlagrangian.blogspot.lt/2012/02/rayleigh-ritz-method-explained-for.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shape Function&lt;/a&gt;. In fact, the shape function can be any mathematical formula that helps us to interpolate what happens wherever there are no points to define the mesh. This &quot;ghost&quot; entity that appears between nodes is in fact the Finite Element. In practical terms, as engineers we are more interested in the implementation of the FEM, not so much on its formulation, so what is important to understand is that for different shape functions we obtain different element matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zsu0j.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zsu0j.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;2DOF Beam element matrix&#xA;&lt;a href=&quot;https://i.stack.imgur.com/w52vy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/w52vy.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;3DOF Beam Element matrix&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DrvKh.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DrvKh.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;3DOF Timoshenko Beam element matrix&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZdVtP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZdVtP.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;6DOF Timoshenko Beam element matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the chosen formulation we have different degrees of interpolation and hence presumably higher or lower degrees of precision. Also depending on the chosen formulation we might have different ways of locating and relating our nodes to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UsQqi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UsQqi.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For our example we can choose any of the formulations provided in literature (above are the most common used in structural engineering). It is important to note the internal structure of these element matrices, which are symmetrical and clearly divided into parts each corresponding to the nodes that reside on the element's boundaries (2 nodes in the case of a beam - 4 quadrants in the matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;STEP 3: Matrix assembly and solution&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the relations between nodes need to be accomplished all at the same time, we have to set all the equations in such a manner that they compose an algebraic system of equations. The matrix equation we want to solve (at least in statics) is as follows: &#xA;\begin{equation}&#xA;[F]= [Kg]·[u],&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;strong&gt;[F]&lt;/strong&gt; is the vector of applied external forces, &lt;strong&gt;[Kg]&lt;/strong&gt; is the system's global stiffness matrix and &lt;strong&gt;[u]&lt;/strong&gt; is the vector of displacements resulting from the application of the forces. The size of &lt;strong&gt;[Kg]&lt;/strong&gt;, &lt;strong&gt;[F]&lt;/strong&gt; and &lt;strong&gt;[u]&lt;/strong&gt; is that of the number of DOFs times the number of nodes, being &lt;strong&gt;[Kg]&lt;/strong&gt; squared and &lt;strong&gt;[F]&lt;/strong&gt; and &lt;strong&gt;[u]&lt;/strong&gt; unidimensional.&#xA;In order to generate the Force vector, all we need to do is collect the applied forces (linear and moment forces) and sort them according to the index of the node they are applied to.&#xA;For the global stiffness matrix, it is necessary a bit more laborious procedure by means of which we iterate throughout each element's particular stiffness matrix. Out of each one of those, we get only the part that corresponds to the position of the node we are storing in the matrix, and add it to the possible concurrent data that comes from other elements. Warning: before entering in the global stiffness matrix, we must convert local coordinates to global coordinates!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the displacement vector, it is needed to first enforce the constraints and then solve the resulting algebraic system of equations. To do this there are two classical approaches: Penalty Method and Lagrange Multipliers method, but we wont enter into this here...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9hnfb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9hnfb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Global Matrix Assembly&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Afterwards, all we need to do is use any available algebraic equation solver (LU decomposition is one of the most extended), and obtain the solution of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;STEP 4: Computation of the eigenvalues of a matrix&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have a matrix assembled and representing your physical model, you can proceed to make other mathematical operations on it, like for example the computation of its eigenvalues. &#xA;The eigenvalues have the &quot;magical&quot; property of, being the stiffness matrix a form of transformation of the displacement vector, represent that form in which the displacements are the largest. These displacements are directly linked with the physical properties of the structural system and its period of free vibration.&lt;/p&gt;&#xA;" OwnerUserId="21268" LastEditorUserId="21268" LastEditDate="2016-08-12T15:25:33.463" LastActivityDate="2016-08-12T15:25:33.463" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24698" PostTypeId="1" CreationDate="2016-08-12T14:22:51.753" Score="1" ViewCount="1480" Body="&lt;p&gt;I want to use the conjugate gradient method to solve poisson's equation in an electrostatic setup:&#xA;\begin{align}&#xA;\rho=-\nabla^2\phi&#xA;\end{align}&#xA;I am however a little confused when it comes to the boundary conditions.&#xA;For the dirichlet boundaries $\phi(0)=D$ it is pretty easy because i can implement the condition in the initial guess of the solution. However, when using either neumann $\frac{\partial \phi}{\partial x}=C$ or periodic $\phi(0)=\phi(L)$, no values are known which can be implemented in the initial guess. And so i'm not sure where the b.c should be imposed during the iterative process..?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="21323" LastActivityDate="2016-08-13T13:46:03.107" Title="Boundary conditions in conjugate gradient method for poisson's equation" Tags="|boundary-conditions|poisson|conjugate-gradient|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24699" PostTypeId="2" ParentId="24678" CreationDate="2016-08-12T22:09:16.970" Score="2" Body="&lt;p&gt;Consider a general PDE of the form&#xA;\begin{align}&#xA;u_t &amp;amp;= P(x,t,\partial_x)u + F(x,t), \quad a \leq x \leq b, \quad t \geq 0 \\&#xA;u(x,0) &amp;amp;= f(x) \\&#xA;L_a(t,\partial_x) u(a,t) &amp;amp;= g_a(t) \\&#xA;L_b(t,\partial_x) u(b,t) &amp;amp;= g_b(t)&#xA;\end{align}&#xA;where $P$ is a differential operator with smooth matrix coefficients, $F$ is a forcing function, and $L_a$ and $L_b$ are differential operators defining the boundary conditions. Let this PDE be discretised in space by a finite difference scheme as follows:&#xA;\begin{align}&#xA;(v_j)_t &amp;amp;= Q_j(x_j,t) v_j + F_j + S_j, \quad j=0,\dots,N, \quad t \geq 0 \\&#xA;v_j(0) &amp;amp;= f_j&#xA;\end{align}&#xA;where $\mathbf{v}=(v_0, \dots, v_N)^T$ is the grid vector approximating $u$, $Q_j$ is the approximation of $P$ at $x_j$, and $S_j$ contains whatever modification you need of your difference scheme to incorporate the boundary conditions. $\mathbf{f} = (f_0, \dots, f_N)^T$ is the projection of $f$ onto the grid and similarly we define $\mathbf{F}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, consider the PDE above with $F=g_a=g_b=0$. The finite difference scheme is said to be &lt;em&gt;stable&lt;/em&gt; in the norm $\| \cdot \|$ if&#xA;$$&#xA;\| \mathbf{v}(t) \| \leq K e^{\alpha t} \| \mathbf{f} \|&#xA;$$&#xA;holds, and $K$ and $\alpha$ are independent of $\mathbf{f}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For general forcing and boundary conditions, the finite difference scheme is said to be &lt;em&gt;strongly stable&lt;/em&gt; in the norm $\| \cdot \|$ if, instead of the above,&#xA;$$&#xA;\| \mathbf{v}(t) \|^2 \leq K(t) \left( \| \mathbf{f} \|^2 + \max_{\tau \in [0,t]} \| \mathbf{F}(\tau) \|^2 + \max_{\tau \in [0,t]} \| g_a(\tau) \|^2 + \max_{\tau \in [0,t]} \| g_b(\tau) \|^2 \right)&#xA;$$&#xA;holds, and $K(t)$ is bounded for any finite $t$ and independent of $\mathbf{F},g_a,g_b$ and $\mathbf{f}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible way to obtain such estimates is through the &lt;em&gt;energy method&lt;/em&gt;, which is outlined in most text books on the subject. This typically gives estimates in the 2-norm, which may possibly be extended to the 1-norm with the aid of various inequalities.&lt;/p&gt;&#xA;" OwnerUserId="19512" LastActivityDate="2016-08-12T22:09:16.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24700" PostTypeId="2" ParentId="24612" CreationDate="2016-08-12T22:34:32.053" Score="3" Body="&lt;p&gt;While I cannot help you with your specific implementation, I want to point out to an alternative method (as already indicated in a comment to phil's answer) : Marston's &quot;Fourier Grid Hamiltonian&quot; (FGH).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FGH is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Pseudo-spectral_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;pseudo-spectral method&lt;/a&gt; and yields a simple recipe for constructing the discretized Hamiltonian matrix for bound systems. As usual, eigenvectors and eigenvalues of the discrete Hamiltonian represent discrete wavefunctions and corresponding energies. For the FGH, the Hamiltonian $H = T+V$ is split in a kinetic energy part $T$ and the potential part $V$. In position space the matrix elements $\left&amp;lt; x\vert V\vert x'\right&amp;gt;$ are just $V(x)\delta(x- x')$, so the potential energy contributes only the Hamiltonian matrix' diagonal. For the matrix elements of the kinetic energy operator, a band-limited (discrete $x_i$!) plane wave basis is used and the matrix elements $\left&amp;lt; x_i\vert T\vert x_j\right&amp;gt;$ are computed analytically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll find a great discussion of this and various related Fourier methods in David Tannor's book &quot;Introduction to Quantum Mechanics: a time-dependent perspective&quot;. Original references are also given there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll find a Jupyter notebook with a simple Python/Cython implementation that tackles a quartic potential &lt;a href=&quot;https://nbviewer.jupyter.org/github/aeberspaecher/FGH/blob/master/Fourier%20Grid%20Hamiltonian.ipynb?flush_cache=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. To change the potential you work with, find the line with &lt;code&gt;pot = lambda x: x**4 - 20*x**2&lt;/code&gt; and change it to any other potential you're interested in. Make sure to compute only bound states for which $\left&amp;lt;\psi\vert\psi\right&amp;gt;$ has decayed to zero close to the boundary of your $x$ domain.&lt;/p&gt;&#xA;" OwnerUserId="1553" LastEditorUserId="1553" LastEditDate="2018-05-22T14:28:39.833" LastActivityDate="2018-05-22T14:28:39.833" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24701" PostTypeId="1" CreationDate="2016-08-13T00:00:39.233" Score="1" ViewCount="1619" Body="&lt;p&gt;I've been using the &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;Eigen&lt;/a&gt; C++ linear algebra library to solve various eigenvalue problems with complex matrices. I've recently had to use a &lt;em&gt;generalized&lt;/em&gt; eigenvalue solution process, only to be disappointed by the fact that Eigen's &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Eigen::GeneralizedEigenSolver&amp;lt; _MatrixType &amp;gt;&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;routine can't handle complex matrices. My matrices are generally not self-adjoint, so it appears I can't use its &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Eigen::GeneralizedSelfAdjointEigenSolver&amp;lt; _MatrixType &amp;gt;&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;routine, either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some workaround using only C++ and Eigen that would allow me to solve this problem? For example, can I convert a pair of complex, non-Hermitian matrices into a pair of self-adjoint matrices for the generalized problem without any loss of information? &lt;/p&gt;&#xA;" OwnerUserId="17983" LastEditorUserId="20162" LastEditDate="2016-08-13T05:48:49.420" LastActivityDate="2016-08-13T21:24:59.020" Title="Eigen - Solving Complex Generalized Eigenvalue Problem" Tags="|linear-algebra|c++|eigen|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24704" PostTypeId="2" ParentId="24698" CreationDate="2016-08-13T03:33:15.213" Score="3" Body="&lt;p&gt;Neumann boundary conditions will show up in the right hand side of your problem, and so you can just start your iteration with any vector you want -- it doesn't have to satisfy any particular boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Periodic boundary conditions ultimately affect the structure of the matrix. You can choose any value for $\phi(0)$ in your initial starting guess for your solution vector. It is probably useful to choose the same value for $\phi(L)$, but the iterative solver should also figure this equality out just based on the matrix.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-13T03:33:15.213" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24705" PostTypeId="2" ParentId="24696" CreationDate="2016-08-13T03:39:39.197" Score="3" Body="&lt;p&gt;You seem to be confused about what is a vector and what is a scalar. $z_h$ and $\phi_i$ are both scalars if evaluates at quadrature points. The fact that the $z_i$ form a vector really has no relevance since you are computing integrals which you approximate via quadrature for which you need $z_h(x)$, not the individual components $z_i$ of the solution vector $Z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to add something to the answer that you did not ask, but should know anyway. If you approximate $z$ by a finite element solution $z_h$ that uses the &lt;em&gt;same&lt;/em&gt; finite element space as $u_h$, then you will be out of luck because you could then choose $v_h=z_h$ and your entire error estimate would be zero. In other words, approximating $z$ by $z_h$ is not a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are numerous solutions to this problem. I might in fact refer you to &lt;em&gt;the&lt;/em&gt; book about dual weighted residual error estimators, namely W. Bangerth and R. Rannacher, &quot;Adaptive Finite Element Methods for Solving Differential Equations&quot;, Birkhäuser, 2003. In the interest of full disclosure, I will mention the obvious: I'm the first author of this book.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-13T03:39:39.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24706" PostTypeId="1" CreationDate="2016-08-13T06:16:39.583" Score="6" ViewCount="427" Body="&lt;p&gt;In his &lt;a href=&quot;https://people.eecs.berkeley.edu/~jrs/papers/elemtalk.pdf&quot; rel=&quot;noreferrer&quot;&gt;talk&lt;/a&gt; &quot;What is a good linear finite element?&quot;, Shewchuk states that small dihedral angles in linear tetrahedra elements cause ill-conditioning of the stiffness matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do small dihedral angles in cells also cause ill-conditioning of the FVM system of equations? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Shewchuk, J. &lt;a href=&quot;https://people.eecs.berkeley.edu/~jrs/papers/elemj.pdf&quot; rel=&quot;noreferrer&quot;&gt;&quot;What is a good linear finite element? interpolation, conditioning, anisotropy, and quality measures (preprint).&quot;&lt;/a&gt; University of California at Berkeley 73 (2002).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21327" LastEditorUserId="9667" LastEditDate="2016-08-15T18:55:04.093" LastActivityDate="2020-07-22T10:24:19.133" Title="Is there a minimum angle requirement for cells in the finite volume method?" Tags="|finite-element|finite-volume|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24707" PostTypeId="2" ParentId="10678" CreationDate="2016-08-13T08:23:37.640" Score="-1" Body="&lt;p&gt;I also explained the shape functions in more details here: &lt;a href=&quot;http://stochasticandlagrangian.blogspot.lt/2012/02/rayleigh-ritz-method-explained-for.html&quot; rel=&quot;nofollow&quot;&gt;http://stochasticandlagrangian.blogspot.lt/2012/02/rayleigh-ritz-method-explained-for.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It explains step by step in a visual way how the Rayleigh-Ritz method works.&#xA;Writing this article finally helped me to get to understand the shape functions.&lt;/p&gt;&#xA;" OwnerUserId="21268" LastActivityDate="2016-08-13T08:23:37.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24708" PostTypeId="2" ParentId="24706" CreationDate="2016-08-13T09:37:43.720" Score="3" Body="&lt;p&gt;I don't remember exactly the formulation in finite-element but I believe you need to compute the determinant of the jacobian when you define the transformation between some element and the reference element. When your element is very &amp;quot;flat&amp;quot;, the determinant tends to zero and that is a problem in your computation.&lt;/p&gt;&#xA;&lt;p&gt;In finite volume, we define the variables as cell-centered. For instance we define &lt;span class=&quot;math-container&quot;&gt;$\overline{u}$&lt;/span&gt; as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\overline{u} = \frac{1}{V}\int_V u~\mathrm{d}V&#xA;$$&lt;/span&gt;&#xA;and we perform the numerical scheme on this variable.&#xA;If your cell is very &amp;quot;flat&amp;quot;, the volume tends to zero so this quantity is not defined.&#xA;It does not answer your question regarding the minimum angle requirement but in general you should avoid such elements.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="29167" LastEditDate="2020-07-14T15:42:52.657" LastActivityDate="2020-07-14T15:42:52.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24709" PostTypeId="2" ParentId="24502" CreationDate="2016-08-13T10:30:45.420" Score="1" Body="&lt;p&gt;When you mention the convergence of the model, do you mean that you have used some form of iterative method?&#xA;In that case, it is likely that you were using some non-linear model (unadvertely?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is a list of possible non-linearities one can encounter based on the ubiquitous &quot;Governing Equations&quot;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;System's internal balance equations (Statics or dynamics)      &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Nonlinear damping&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Constitutive equations (Material)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Plasticity&#xA;   Viscosity&#xA;   Creeping&#xA;   Hiperelasticity&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Kinematic equations (Geometry)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Large deformations with small strains&#xA;   Large strains&#xA;   Buckling&#xA;   Instability&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Forces&#xA;       Pressure&#xA;       Loads&#xA;       Wind&#xA;       Waves&#xA;       Friction&#xA;   Displacements&#xA;       Contact&#xA;       Link rupture&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As you can see, large strains (the problem that you seem to be studying) is only one of the many cases where the assumption of linear behaviour just limits the validity of a model.&#xA;In the image below, you can see that the linear hypothesis is valid only within the orange triangle. If you were to continue pulling the spring assuming that your real behaviour is on the line, the amount of energy that you would be creating from 'nothing' would grow quadratically once you enter in the non-linear regime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pojN8.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pojN8.gif&quot; alt=&quot;Strain energy of a stress experiment&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FEM is generally formulated on variational principles (balance of energies), so (in my opinion) the error should be accounted for in the energy level. In this case, your error would grow in a quadratic form.&lt;/p&gt;&#xA;" OwnerUserId="21268" LastActivityDate="2016-08-13T10:30:45.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24710" PostTypeId="2" ParentId="24698" CreationDate="2016-08-13T13:46:03.107" Score="0" Body="&lt;p&gt;Depending on the discretization you use and the type of BCs you have, you need to incorporate the BCs into the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can think of it as putting extra rows for the BCs in the matrix, which you can then eliminate by adding them onto another equation. If you do that in a clever way, the matrix remains spd and the rhs changes as Wolfgang pointed out. Note that in case of Neumann BCs you will need to discretize the BCs as well as the interior PDE. For periodic BCs just add an extra set of equations of the form $x_0 = x_L$ (in 1D) to the matrix, which will change its properties. &lt;/p&gt;&#xA;" OwnerUserId="5337" LastActivityDate="2016-08-13T13:46:03.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24711" PostTypeId="2" ParentId="24701" CreationDate="2016-08-13T21:24:59.020" Score="1" Body="&lt;p&gt;If one of the matrices is invertible, you can convert it to a standard eigenvalue problem $(B^{-1}A) x = \lambda x$. With Eigen, you can thus do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EigenSolver&amp;lt;MatrixXcd&amp;gt; eig(B.lu().solve(A));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and, if needed, you can also re-normalize the eigenvectors so that $x^T B x = 1$.&lt;/p&gt;&#xA;" OwnerUserId="4337" LastActivityDate="2016-08-13T21:24:59.020" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24712" PostTypeId="1" AcceptedAnswerId="24718" CreationDate="2016-08-13T22:09:58.310" Score="1" ViewCount="782" Body="&lt;p&gt;For the past few days, I have been writing a numerical solver for the 2D compressible Euler equations for an ideal gas.  My numerical method has been the Local Lax Friedrichs or &quot;Rusanov's method.&quot; Before writing the 2D code I successfully wrote a 1D code using the Local Lax-Friedrichs method, which works. However, my extension to 2D has given me some problems I am struggling with no success.  In terms of the specific error, I keep getting NaNs in my arrays (for &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;). I have attached my code below.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To better understand whats going on in the code the method I am using is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;Q_{i,j}^{n+1} &amp;amp;= Q_{i,j}^{n} - \frac{\Delta t}{\Delta x} \left(F_{i+1/2,j}^{n} - F_{i-1/2,j}^{n} \right) - \frac{\Delta t}{\Delta y} \left(G_{i,j+1/2}^{n} - G_{i,j-1/2}^{n}\right)\\&#xA;F_{i+1/2,j}^{n} &amp;amp;= \frac{1}{2} \left[ \left(F_{i+1,j}^{n} + F_{i,j}^{n}\right) - a_{i+1/2,j} \left(Q_{i+1,j}^{n} - Q_{i,j}^{n} \right)\right]\\&#xA;a_{i+1/2,j} &amp;amp;= \max{\left(\left|u_{i,j}\right| + c_{i,j}, \left|u_{i+1,j}\right| + c_{i+1,j}\right)}\\&#xA;G_{i,j+1/2}^{n} &amp;amp;= \frac{1}{2} \left[ \left(G_{i,j+1}^{n} - G_{i,j}^{n}\right) - b_{i,j+1/2}\left(Q_{i,j+1}^{n} - Q_{i,j}^{n}\right) \right]\\&#xA;b_{i,j+1/2} &amp;amp;= \max{\left(\left|v_{i,j}\right| + c_{i,j}, \left|v_{i,j+1}\right| + c_{i,j+1}\right)}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of &lt;span class=&quot;math-container&quot;&gt;$c_{i,j}$&lt;/span&gt;, this term refers to the speed of sound, i.e. &lt;span class=&quot;math-container&quot;&gt;$c_{i,j} = \sqrt{\frac{\gamma p}{\rho}}$&lt;/span&gt; Our pressure can be defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p = (\gamma - 1)(E - \frac{\rho}{2} \left(u^{2} + v^{2}\right)\\&#xA;E = \frac{p}{\gamma-1} + \frac{\rho}{2} \left(u^{2} + v^{2}\right)\\&#xA;Q = [\rho\;\rho u\;\rho v\;E]^{T}\\&#xA;F = [\rho u\; \rho u^{2} + p\; \rho u v\; u(E+p)]^{T}\\&#xA;G = [\rho v\; \rho u v\; \rho v^{2} + p\; v(E+p)]^{T}&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nx = 101;&#xA;dx = 1/(nx-1);&#xA;&#xA;ny = 101;&#xA;dy = 1/(ny-1);&#xA;&#xA;r = zeros(nx,ny);&#xA;u = zeros(nx,ny);&#xA;v = zeros(nx,ny);&#xA;E = zeros(nx,ny);&#xA;p = zeros(nx,ny);&#xA;&#xA;x = zeros(nx,ny);&#xA;y = zeros(nx,ny);&#xA;&#xA;Q = zeros(nx,ny,4);&#xA;Qnew = zeros(nx,ny,4);&#xA;&#xA;F = zeros(nx,ny,4);&#xA;G = zeros(nx,ny,4);&#xA;&#xA;Fh = zeros(nx-1,ny,4);&#xA;Gh = zeros(nx,ny-1,4);&#xA;&#xA;gamma = 1.4;&#xA;&#xA;c = zeros(nx,ny);&#xA;ah1 = zeros(nx,ny);&#xA;ah2 = zeros(nx,ny);&#xA;&#xA;CFL = 0.5;&#xA;tMax = 0.25;&#xA;&#xA;for j=1:ny&#xA;    for i=1:nx&#xA;        x(i,j) = (i-1)*dx;&#xA;        y(i,j) = (j-1)*dy;&#xA;    end&#xA;end&#xA;&#xA;for j=1:ny&#xA;    for i=1:nx&#xA;        if (x(i,j) &amp;gt;= 0.5 &amp;amp;&amp;amp; y(i,j) &amp;gt;= 0.5)&#xA;            p(i,j) = 0.4; &#xA;            r(i,j) = 0.5313; &#xA;            u(i,j) = 0.0; &#xA;            v(i,j) = 0.0;&#xA;        elseif (x(i,j) &amp;lt; 0.5 &amp;amp;&amp;amp; y(i,j) &amp;gt;= 0.5)&#xA;             p(i,j) = 1.0; &#xA;            r(i,j) = 1.0; &#xA;            u(i,j) = 0.7276; &#xA;            v(i,j) = 0.0;&#xA;        elseif (x(i,j) &amp;lt; 0.5 &amp;amp;&amp;amp; y(i,j) &amp;lt; 0.5)&#xA;            p(i,j) = 1.0; &#xA;            r(i,j) = 0.8; &#xA;            u(i,j) = 0.0; &#xA;            v(i,j) = 0.0;&#xA;        elseif (x(i,j) &amp;gt;= 0.5 &amp;amp;&amp;amp; y(i,j) &amp;lt; 0.5)&#xA;            p(i,j) = 1.0; &#xA;            r(i,j) = 1.0; &#xA;            u(i,j) = 0.0; &#xA;            v(i,j) = 0.7276;&#xA;        end&#xA;&#xA;        c(i,j) = sqrt((gamma*p(i,j))/r(i,j));&#xA;    end&#xA;end&#xA;&#xA;Eig1 = abs(u) + c;&#xA;Eig2 = abs(v) + c;&#xA;&#xA;dt = CFL*dx./max(Eig1(:),Eig2(:));&#xA;nt = floor(tMax/dt);&#xA;&#xA;for k=1:nt&#xA;    for j=1:ny&#xA;        for i=1:nx&#xA;            E(i,j) = p(i,j)./(gamma-1) + (0.5*r(i,j))*(u(i,j).^2 + v(i,j).^2);&#xA;&#xA;            Q(i,j,1) = r(i,j);&#xA;            Q(i,j,2) = r(i,j).*u(i,j);&#xA;            Q(i,j,3) = r(i,j).*v(i,j);&#xA;            Q(i,j,4) = E(i,j);&#xA;&#xA;            F(i,j,1) = r(i,j).*u(i,j);&#xA;            F(i,j,2) = r(i,j).*u(i,j).^2 + p(i,j);&#xA;            F(i,j,3) = r(i,j).*u(i,j).*v(i,j);&#xA;            F(i,j,4) = u(i,j).*(E(i,j) + p(i,j));&#xA;&#xA;            G(i,j,1) = r(i,j).*v(i,j);&#xA;            G(i,j,2) = r(i,j).*v(i,j).*u(i,j);&#xA;            G(i,j,3) = r(i,j).*v(i,j).^2 + p(i,j);&#xA;            G(i,j,4) = v(i,j).*(E(i,j) + p(i,j));&#xA;        end&#xA;    end&#xA;&#xA;    for j=1:ny-1&#xA;        for i=1:nx-1&#xA;            ah1(i,j) = max(abs(u(i,j)) + c(i,j), abs(u(i+1,j)) + c(i+1,j));&#xA;            ah2(i,j) = max(abs(v(i,j)) + c(i,j), abs(v(i,j+1)) + c(i,j+1));&#xA;&#xA;            Fh(i,j,:) = 0.5*(F(i+1,j,:) + F(i,j,:) - ah1(i,j).*(Q(i+1,j,:) - Q(i,j,:)));&#xA;            Gh(i,j,:) = 0.5*(G(i,j,:) + G(i,j+1,:) - ah2(i,j).*(Q(i,j+1,:) - Q(i,j,:)));&#xA;&#xA;        end&#xA;    end&#xA;&#xA;    Qnew(:,1,:)  = Qnew(:,2,:);      &#xA;    Qnew(1,:,:)  = Qnew(2,:,:);&#xA;    Qnew(:,ny,:) = Qnew(:,ny-1,:);    &#xA;    Qnew(nx,:,:) = Qnew(nx-1,:,:);&#xA;&#xA;    for j=2:ny-1&#xA;        for i=2:nx-1&#xA;            Qnew(i,j,:) = Q(i,j,:) - (dt/dx)*(Fh(i,j,:) - Fh(i-1,j,:)) - (dt/dy)*(Gh(i,j,:) - Gh(i,j-1,:));&#xA;        end&#xA;    end  &#xA;&#xA;    Q = Qnew; &#xA;    r = Qnew(:,:,1);&#xA;    u = Qnew(:,:,2)./Qnew(:,:,1);&#xA;    v = Qnew(:,:,3)./Qnew(:,:,1);&#xA;    E = Qnew(:,:,4);&#xA;    p = (gamma-1)*(E - (0.5*Qnew(:,:,1))*(u.^2 + v.^2));&#xA;end&#xA;&#xA;figure(1) &#xA;contour(x,y,r);&#xA;xlabel('x');&#xA;ylabel('y');&#xA;title('Density');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In terms of the initial condition, it divides the discrete domain into 4 regions, i.e. 4 quadrants.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code does &quot;run,&quot; but it only plots the initial condition which means there's something going wrong in my time update (however for my 1D problem it works with no problem.) My density plot &lt;strong&gt;should&lt;/strong&gt; look like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mYzi4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mYzi4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we should expect in terms of (&lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;) is something similar to the picture I uploaded. The problem I am solving is a test problem which tests the accuracy of numerical solvers (Wendroff &amp;amp; Liska 2003) &lt;/p&gt;&#xA;" OwnerUserId="21337" LastEditorUserId="20688" LastEditDate="2019-08-25T23:46:44.943" LastActivityDate="2019-08-25T23:46:44.943" Title="Difficult bug in my 2D Compressible Euler solver" Tags="|finite-difference|fluid-dynamics|computational-physics|numerics|numerical-modelling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="24714" PostTypeId="2" ParentId="24706" CreationDate="2016-08-14T05:54:39.380" Score="0" Body="&lt;p&gt;I point out some references&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dx.doi.org/10.1137/0713021&quot; rel=&quot;nofollow&quot;&gt;http://dx.doi.org/10.1137/0713021&lt;/a&gt;&#xA;Here they show that angles should be bounded below 180 deg.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/article/10.1007/BF03322598&quot; rel=&quot;nofollow&quot;&gt;http://link.springer.com/article/10.1007/BF03322598&lt;/a&gt;&#xA;This gives a more recent survey.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.bcamath.org/documentos_public/archivos/actividades_cientificas/TalkBCAMWSonCM20131018Korotov.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.bcamath.org/documentos_public/archivos/actividades_cientificas/TalkBCAMWSonCM20131018Korotov.pdf&lt;/a&gt;&#xA;Nice brief survey, show that maximum angle condition is also not necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most of this discussion refers to simplicial elements and I am not aware of what happens on quad/hex elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For finite volumes, there is so much variety of schemes (e.g., cell-centered vs cell-vertex) that I don't know any general results. But still small angles should not be a problem and in fact, you benefit from using highly anisotropic triangles when computing discontinuous solutions, or those with steep gradients as in boundary layers. For high Reynolds number flows, you have to use highly stretched triangles in the boundary layers which have small angles, nearly right angled, but they do not have large angles.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2016-08-14T05:54:39.380" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24715" PostTypeId="2" ParentId="24568" CreationDate="2016-08-14T08:20:51.473" Score="-1" Body="&lt;p&gt;Of course you are right, $a_k$ requires N multiplications, it's obvious. It does not involve $d_{jk}$...&lt;/p&gt;&#xA;" OwnerUserId="21341" LastActivityDate="2016-08-14T08:20:51.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24717" PostTypeId="1" CreationDate="2016-08-14T11:29:47.380" Score="3" ViewCount="71" Body="&lt;p&gt;Assume I know all eigenvalues of a matrix $A$ fall into a certain set $\Omega \subset \mathbb{C}$. Is there any way I can exploit this knowledge to design a preconditioner for $A$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some further remarks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The eigenvalues fill $\Omega$ uniformly, i.e. there are no isolated eigenvalues.&lt;/li&gt;&#xA;&lt;li&gt;I know the eigenvalues but not the eigenfunctions. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21342" LastActivityDate="2016-08-14T11:29:47.380" Title="Preconditioning matrix with known spectrum" Tags="|linear-solver|eigenvalues|preconditioning|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24718" PostTypeId="2" ParentId="24712" CreationDate="2016-08-14T18:00:15.700" Score="2" Body="&lt;p&gt;First, your time step &lt;code&gt;dt&lt;/code&gt; is a matrix. That is not possible. Just below your calculate the number of required time step to reach &lt;code&gt;tMax&lt;/code&gt;. You do not know that. You do not know how many time step you will require to complete your simulation, it depends on the velocity field and the speed of sound.&#xA;The time step is linked to the mesh size and the maximum velocity through the CFL condition such that, for the Euler problem :&#xA;$$&#xA;dt = CFL \frac{h}{\vert \vert \mathbf{u} \vert \vert + c}&#xA;$$&#xA;where h is the size of the cell. Notice here that we deal with the norm of the velocity, not the absolute value. In 1D, the absolute value works, you were just lucky.&#xA;To find the largest admissible time step, you need to compute the maximum velocity (that is to say $max(\vert \vert \mathbf{u} \vert \vert + c)$) through your computational domain because each cell will have a different velocity. If your mesh is uniformly spaced, it is the cell with the maximum velocity that will dictate the maximum time step allowed. This process absolutely need to be inside your loop in time at the beginning of each iteration, your &lt;code&gt;dt&lt;/code&gt; will vary through time and it is a scalar, not a matrix that will change at each iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, your boundary conditions are not properly set. Actually, you iterate through all your cells except ones on the boundaries which have a value of zero at the first iteration because of your initialization of $Q_{new}$. Add &lt;code&gt;keyboard&lt;/code&gt; just before the end of the time loop and check your workspace, you will see that u and v are NaN because you have divided by 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BC are difficult to handle, a common solution is to add ghost cells with a virtual state so you don't need to separate the computation of the boundary cells, or directly impose the flux on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following piece of code works for me, however I can't ensure you it is bug-free. I usually work with an isothermal Euler model so I don't know if my energy equation is implemented in the right way. Double-check the pressure law and notation between $E$ and $e$ to be sure.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;&#xA;close all;&#xA;clc; &#xA;&#xA;nx = 101;&#xA;ny = 101;&#xA;&#xA;% Initialization&#xA;r = zeros(nx,ny);&#xA;u = zeros(nx,ny);&#xA;v = zeros(nx,ny);&#xA;E = zeros(nx,ny);&#xA;p = zeros(nx,ny);&#xA;&#xA;x = zeros(nx,ny);&#xA;y = zeros(nx,ny);&#xA;&#xA;Q = zeros(nx,ny,4);&#xA;F = zeros(nx,ny,4);&#xA;G = zeros(nx,ny,4);&#xA;&#xA;Fh = zeros(nx-1,ny,4);&#xA;Gh = zeros(nx,ny-1,4);&#xA;&#xA;Qnew = zeros(nx,ny,4);&#xA;&#xA;c = zeros(nx,ny);&#xA;ah1 = zeros(nx,ny);&#xA;ah2 = zeros(nx,ny);&#xA;&#xA;% Parameters&#xA;gamma = 1.4;&#xA;CFL = 0.5;&#xA;tMax = 0.25;&#xA;dx = 1/(nx-1);&#xA;dy = 1/(ny-1);&#xA;&#xA;% Creating mesh&#xA;for j=1:ny&#xA;    for i=1:nx&#xA;        x(i,j) = (i-1)*dx;&#xA;        y(i,j) = (j-1)*dy;&#xA;    end&#xA;end&#xA;&#xA;% Initial condition&#xA;for j=1:ny&#xA;    for i=1:nx&#xA;        if (x(i,j) &amp;gt;= 0.5 &amp;amp;&amp;amp; y(i,j) &amp;gt;= 0.5)&#xA;            p(i,j) = 0.4; &#xA;            r(i,j) = 0.5313; &#xA;            u(i,j) = 0.0; &#xA;            v(i,j) = 0.0;&#xA;        elseif (x(i,j) &amp;lt; 0.5 &amp;amp;&amp;amp; y(i,j) &amp;gt;= 0.5)&#xA;            p(i,j) = 1.0; &#xA;            r(i,j) = 1.0; &#xA;            u(i,j) = 0.7276; &#xA;            v(i,j) = 0.0;&#xA;        elseif (x(i,j) &amp;lt; 0.5 &amp;amp;&amp;amp; y(i,j) &amp;lt; 0.5)&#xA;            p(i,j) = 1.0; &#xA;            r(i,j) = 0.8; &#xA;            u(i,j) = 0.0; &#xA;            v(i,j) = 0.0;&#xA;        elseif (x(i,j) &amp;gt;= 0.5 &amp;amp;&amp;amp; y(i,j) &amp;lt; 0.5)&#xA;            p(i,j) = 1.0; &#xA;            r(i,j) = 1.0; &#xA;            u(i,j) = 0.0; &#xA;            v(i,j) = 0.7276;&#xA;        end&#xA;        E(i,j) = p(i,j)./(gamma-1) + 0.5*r(i,j)*(u(i,j).^2+v(i,j).^2);&#xA;        c(i,j) = sqrt(gamma*p(i,j)/r(i,j));&#xA;    end&#xA;end&#xA;&#xA;% Q_new will be the solution vector&#xA;Qnew(:,:,1)=r;&#xA;Qnew(:,:,2)=r.*u;&#xA;Qnew(:,:,3)=r.*v;&#xA;Qnew(:,:,4)=E;&#xA;&#xA;t=0.d0;&#xA;% Time loop&#xA;while (t&amp;lt;tMax)&#xA;    dt = CFL*min(dx,dy)./(max(max(c+sqrt(u.*u+v.*v)))); % Compute time step&#xA;    for j=1:ny&#xA;        for i=1:nx&#xA;&#xA;            % Flux calculation&#xA;            Q(i,j,1) = r(i,j);&#xA;            Q(i,j,2) = r(i,j).*u(i,j);&#xA;            Q(i,j,3) = r(i,j).*v(i,j);&#xA;            Q(i,j,4) = E(i,j);&#xA;&#xA;            F(i,j,1) = r(i,j).*u(i,j);&#xA;            F(i,j,2) = r(i,j).*u(i,j).^2 + p(i,j);&#xA;            F(i,j,3) = r(i,j).*u(i,j).*v(i,j);&#xA;            F(i,j,4) = u(i,j).*(E(i,j) + p(i,j));&#xA;&#xA;            G(i,j,1) = r(i,j).*v(i,j);&#xA;            G(i,j,2) = r(i,j).*v(i,j).*u(i,j);&#xA;            G(i,j,3) = r(i,j).*v(i,j).^2 + p(i,j);&#xA;            G(i,j,4) = v(i,j).*(E(i,j) + p(i,j));&#xA;        end&#xA;    end&#xA;&#xA;    % Rusanov method&#xA;    for j=1:ny-1&#xA;        for i=1:nx-1&#xA;            ah1(i,j) = max(abs(u(i,j)) + c(i,j), abs(u(i+1,j)) + c(i+1,j));&#xA;            ah2(i,j) = max(abs(v(i,j)) + c(i,j), abs(v(i,j+1)) + c(i,j+1));&#xA;&#xA;            Fh(i,j,:) = 0.5*(F(i+1,j,:) + F(i,j,:) - ah1(i,j).*(Q(i+1,j,:) - Q(i,j,:)));&#xA;            Gh(i,j,:) = 0.5*(G(i,j,:) + G(i,j+1,:) - ah2(i,j).*(Q(i,j+1,:) - Q(i,j,:)));&#xA;&#xA;        end&#xA;    end&#xA;&#xA;    % Advance the solution in time&#xA;    for j=2:ny-1&#xA;        for i=2:nx-1&#xA;            Qnew(i,j,:) = Qnew(i,j,:) - (dt/dx)*(Fh(i,j,:) - Fh(i-1,j,:)) - (dt/dy)*(Gh(i,j,:) - Gh(i,j-1,:));&#xA;        end&#xA;    end &#xA;&#xA;    % Boundary condition&#xA;    Qnew(:,1,:)  = Qnew(:,2,:);      &#xA;    Qnew(1,:,:)  = Qnew(2,:,:);&#xA;    Qnew(:,ny,:) = Qnew(:,ny-1,:);    &#xA;    Qnew(nx,:,:) = Qnew(nx-1,:,:);&#xA;&#xA;    % Primitive variables&#xA;    r = Qnew(:,:,1);&#xA;    u = Qnew(:,:,2)./r;&#xA;    v = Qnew(:,:,3)./r;&#xA;    E = Qnew(:,:,4);&#xA;    p = (gamma-1)*(E-0.5*r.*(u.*u+v.*v));&#xA;&#xA;   t=t+dt % Advance in time&#xA;&#xA;end&#xA;&#xA;% Plot&#xA;figure(1) &#xA;contour(x(2:nx-1,2:nx-1),y(2:nx-1,2:nx-1),r(2:nx-1,2:nx-1));&#xA;xlabel('x');&#xA;ylabel('y');&#xA;title('Density');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The main changes are the &lt;code&gt;while&lt;/code&gt; time loop and the computation of &lt;code&gt;dt&lt;/code&gt;. Also I have slightly changed your boundary conditions (moving after the update in time). Actually, your mesh now incorporates ghost cells for cells indexes i=1,i=nx,j=1 and j=ny. These cells are not part of the computational mesh (the update of the solution occurs for 2:nx-1 and 2:ny-1) but you need them to compute flux. I just copy the updated value from the neighbouring cells as you do before. That is not perfect but I can't investigate much further, you didn't mention the boundary condition for your problem (Dirichlet, Neumann ?). That's why in the &lt;code&gt;contour&lt;/code&gt; command, I don't plot these cells.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="20454" LastEditDate="2016-08-17T17:07:25.533" LastActivityDate="2016-08-17T17:07:25.533" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="24720" PostTypeId="2" ParentId="24706" CreationDate="2016-08-14T18:52:36.370" Score="1" Body="&lt;p&gt;This is the sort of question that is more difficult to answer in the finite volume context than in the finite element context. So I would not be surprised if there are no papers that &lt;em&gt;prove&lt;/em&gt; that bad triangles (or other kinds of cells) lead to bad solutions, though I would expect that there are plenty of papers that show it experimentally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, many finite volume schemes can be written as variations or re-interpretations of finite element schemes. Consequently, whatever we know about methods in the finite element context must then also be true for the corresponding finite volume methods. I have no doubt that problems with poor cell shapes is one of these items that can be proven in the finite element context and that subsequently carry over to the finite volume context.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-14T18:52:36.370" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24721" PostTypeId="1" AcceptedAnswerId="24729" CreationDate="2016-08-15T00:33:12.043" Score="0" ViewCount="533" Body="&lt;p&gt;My &lt;a href=&quot;https://scicomp.stackexchange.com/q/24706/17671&quot;&gt;last question&lt;/a&gt; was about the condition number of the stiffness matrix.  This question is about discretization error (which is probably more important to most people).  The mesh on the right side of the figure below (&lt;a href=&quot;https://blog.pointwise.com/2012/07/05/accuracy-convergence-and-mesh-quality/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;) produced better discretization error than the &quot;perfect&quot; quality mesh on the left side even though the mesh on the right side has quads with angles close to 180 degrees. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dSl2q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dSl2q.png&quot; alt=&quot;Burgers&amp;#39; Equation on regular and irregular grids&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the creator of the figure (professor Christopher Roy) used a finite volume or finite difference method since he is a fluid dynamics specialist.  Based on your personal experience using FEM, could FEM get a good discretization error using a mesh like the one on the right side of the figure?&lt;/p&gt;&#xA;" OwnerUserId="21327" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-08-21T20:25:26.060" Title="Could the finite element method get a good discretization error using a mesh like this?" Tags="|finite-element|finite-volume|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24724" PostTypeId="2" ParentId="24568" CreationDate="2016-08-15T11:29:07.733" Score="1" Body="&lt;p&gt;[EDITED because of my inattention] I suspect &quot;operations&quot; here means  multiplies and inverses only, neglecting subtractions $x_j-x_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking a look at the paper, I see  Eq. (2.8):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \phi_{N+1}(x) = \prod_{l=0}^N (x-x_l)\,. $$&#xA;Then, [Eq. (2.9)]&#xA;$$ \phi'_{N+1}(x) = \sum_{k=0}^{N}\prod_{l=0,l\neq k}^N (x-x_l) $$&#xA;and so [Eq. (2.10)]&#xA;$$ \phi'_{N+1}(x_k) = a_k \,.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each $a_k$, $N$ multiplies are required. Could one interpret the assertion globally, on all $a_j$? Indeed, the paper evaluate $4N^2$ operations for the whole matrix $D$. What if one only counts multiplies or inverses? Back in 1989, it was not uncommon to forget adds and subtractions in complexity. In Eq. 2.12a, they evaluate off-diagonal elements:&#xA;$$\frac{a_j}{a_k(x_j-x_k)}$$ &#xA;with &quot;another  $2N^2$ operations&quot;. Once the $a_j$ are computed, this seems sound: $1$ product and $1$ divide for each pair of $(j,k)$, $j\neq k$. Then they count $N^2$ for the diagonal elements $$\sum_{l=0,l\neq k}^{N}\frac{1}{x_k-x_l}\,,$$ ie $N$ inverses for each of the diagonal elements, about $N^2$ total.&lt;/p&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="19501" LastEditDate="2016-08-15T14:13:44.920" LastActivityDate="2016-08-15T14:13:44.920" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24726" PostTypeId="2" ParentId="24667" CreationDate="2016-08-15T14:21:44.473" Score="2" Body="&lt;p&gt;If you want to know more about efficient numerical procedures to compute Lagrange polynomials, I recommend the following reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/barycentric.pdf&quot; rel=&quot;nofollow&quot;&gt;J.-P. Berrut and L. N. Trefethen, “Barycentric Lagrange Interpolation,” SIAM Rev., vol. 46, no. 3, pp. 501–517, 2004.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main point of the &quot;barycentric&quot; Lagrange formula is to pre-compute barycentric weights (in $O(n^2$) operations) so that the interpolation can then be subsequently carried out in $O(n)$ operations. The &quot;naive&quot; method you have implemented above takes $O(n^2)$ operations &lt;em&gt;for each interpolation&lt;/em&gt; and is hence slower, as you have noted.  This formula is also numerically stable near the interpolation points, which is a big bonus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not familiar with R, so I don't know if it already implements this routine. However, it may not be hard to implement yourself.&lt;/p&gt;&#xA;" OwnerUserId="9601" LastActivityDate="2016-08-15T14:21:44.473" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24727" PostTypeId="1" CreationDate="2016-08-15T15:40:16.937" Score="1" ViewCount="305" Body="&lt;p&gt;Consider N random variables $\mathbf{x}=[x_1,x_2,...,x_N]^T$ with joint density $p(\mathbf{x})$, and let $\mathbf{R}\in \mathbb{R}^{N\times N}$ denote the covariance matrix. How to derive the mutual information for these multivariate Gaussian distributions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or do you know which literature provide the related derivation?&#xA;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="21347" LastActivityDate="2016-08-15T15:40:16.937" Title="Mutual information of Multivariate Gaussian Distribution" Tags="|matrix|" AnswerCount="0" CommentCount="1" ClosedDate="2016-08-18T23:51:18.460" ContentLicense="CC BY-SA 3.0" />
  <row Id="24728" PostTypeId="1" CreationDate="2016-08-15T15:48:27.963" Score="0" ViewCount="284" Body="&lt;p&gt;I have worked with MATLAB for calculating below equation for any time ($n$) and place($i$). In this equation, flux limiter is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C(i,n+1)=C(i,n)-V\left[1+\frac{1}{2}\psi(r_i)-\frac{1}{2}\frac{1}{(r_i-1)}\psi(r_i-1)\right]\left(C(i,n)-C(i-1,n)\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By writing the equation, the amount of flux limiter in the time zero ($n=0$) faced problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you know, $r_i=\frac{C(i+1,n)-C(i,n)}{C(i,n)-C(i-1,n)}$, which the amount of $C$ in the time zero ($n=0$) for all places ($i$) is the same, then I reached undefined value ($0/0$) for $r$ in time zero($n=0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any equation or any amount for calculating $r$ in the time of zero?&lt;/p&gt;&#xA;" OwnerUserId="21350" LastEditorDisplayName="user20857" LastEditDate="2016-08-15T18:12:22.573" LastActivityDate="2016-10-15T09:18:07.457" Title="flux limiter problem" Tags="|finite-element|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24729" PostTypeId="2" ParentId="24721" CreationDate="2016-08-15T21:19:47.440" Score="0" Body="&lt;p&gt;I think, but I don't have the proof in front of me, that if you use a bilinear, Cartesian product element and the isoparametric map from the master element to the real element, then the Jacobian of the map will be zero at the flat corner on these elements. That might actually be OK if you never evaluate anything there. If you use Gaussian quadrature, then you may get away with only evaluating things in the interior at the element's Gauss points. If you use a Newton-Cotes rule that wants to do integration at the nodes of the element, then your ability to integrate at all is shot. So, the answer is probably &quot;it depends&quot;. In the Gaussian case, you may also lose some accuracy even if you don't evaluate anything.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-08-15T21:19:47.440" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24730" PostTypeId="1" AcceptedAnswerId="24732" CreationDate="2016-08-15T22:23:03.470" Score="0" ViewCount="506" Body="&lt;p&gt;In order to verify a two-phase subsonic compressible isothermal Euler code, I am trying to implement a manufactured solution following what is discussed &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21119/2d-euler-manufactured-solutions&quot;&gt;here&lt;/a&gt; and references therein. Also, as an extension of the question already answered &lt;a href=&quot;https://scicomp.stackexchange.com/questions/12902/guidelines-for-choosing-manufactured-solutions-for-numerical-pde-schemes&quot;&gt;here&lt;/a&gt;, I am looking for a more detailed explanation on the choice of a manufactured solution for a specific case and especially on the choice of the related parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, in &lt;a href=&quot;http://ftp.demec.ufpr.br/CFD/bibliografia/erros_numericos/Roy_et_al_2004.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, the authors suggest for the density a manufactured solution of the form (in 2D):&#xA;$$&#xA;\rho(x,y)=\rho_0 + \rho_x \sin \left( \frac{a_{\rho x}\pi x}{L} \right) + \rho_y \cos \left( \frac{a_{\rho y}\pi y}{L} \right) + \rho_{xy} \cos \left( \frac{a_{\rho xy}\pi xy}{L^2} \right) &#xA;$$&#xA;where $\rho_0,\rho_x,\rho_y,\rho_{xy},a_{\rho x},a_{\rho y},a_{\rho x y} $ are given just like that without any explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find this expression really complicated and my question is why the authors have chosen such a complicated function ? I know that trigonometric functions are good candidate because $C^\infty$ but still why the proposed manufactured solution is such a complex combination of such functions ? Why a simple cosinus cannot be enough ? And above all, how do they choose the value for the seven parameters written above ? What make you choose between setting $\rho_{xy}$ to 2 and 500 for instance ?&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-08-18T18:01:37.140" Title="How to choose a simple manufactured solution for Euler equation?" Tags="|pde|numerical-analysis|testing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24732" PostTypeId="2" ParentId="24730" CreationDate="2016-08-15T23:20:00.650" Score="3" Body="&lt;p&gt;With manufactured solutions, you pick the form of the solutions you want, crank them through the governing equations, and generate right-hand side functions that you can go implement to drive the solution to the solution you manufactured. Some solutions are chosen to satisfy other properties that you want. Spencer's comment suggests that boundary conditions are one set of properties you might desire to also include in your manufactured solution. Periodicity is another one. Also, you need to pick solutions that are smooth enough to have enough derivatives to pass neatly through your operator. There are numerous other conditions you might want to impose on your solution, so the reasons for choices may not ever be guessable from the outside. This paper does not give the authors' reasons for selecting this particular form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transcendental functions are often used since they may excite multiple modes of the operator simultaneously. A sum or product of them is an easy way to combine such functions into something that explores a multi-dimensional space effectively.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-08-15T23:20:00.650" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24733" PostTypeId="1" AcceptedAnswerId="24778" CreationDate="2016-08-16T04:20:52.210" Score="4" ViewCount="286" Body="&lt;p&gt;I've recently been interested in learning about implicit finite element methods. I've found &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7628/what-is-the-difference-between-implicit-fem-and-explicit-fem/7643&quot;&gt;this post&lt;/a&gt;, but I'd like to learn more about them, specifically about how simulations like &lt;a href=&quot;http://www.dynaexamples.com/implicit&quot; rel=&quot;nofollow noreferrer&quot;&gt;these from LS-DYNA&lt;/a&gt; work. Can anyone point me to a good textbook, manual or paper that covers the basics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDITED: Thanks to everyone for answering the question. I think I see now that the question was not posed correctly due to my lack of knowledge about this particular aspect of FEM.&lt;/p&gt;&#xA;" OwnerUserId="21131" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-09-08T22:04:12.307" Title="Textbook/Manual on Implicit FEM Methods" Tags="|finite-element|implicit-methods|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24734" PostTypeId="1" CreationDate="2016-08-16T05:01:25.797" Score="0" ViewCount="372" Body="&lt;p&gt;I have a problem where I am solving a system of linear equations but sometimes the system results in a singular matrix which cannot be easily solved. In this case I would like that those rows for which system is ill-defined would be computed as 0. How can I assure that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{array}{ccccc}&#xA;1 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; -0.3 &amp;amp; -0.7 \\&#xA;-1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;\end{array} \right] \boldsymbol{x} = \left[ \begin{array}{c}&#xA;0 \\&#xA;0 \\&#xA;0 \\&#xA;1 \\&#xA;\end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like that for $\boldsymbol{x}$ the result is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{array}{c}&#xA;0 \\&#xA;0.7 \\&#xA;0 \\&#xA;1 \\&#xA;\end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use least-squares approach but it is not really necessary that I get a solution which has zeros for rows which are ill-defined because it is trying to minimize $|A\boldsymbol{x} - \boldsymbol{b}|$ and not $|\boldsymbol{x}|$ itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some properties of the matrix above: there is always a diagonal with elements $1$, other elements are from $[-1,0]$, sum of each row is always or 1 or 0. The vector on the right can contain only 0 or 1 elements.&lt;/p&gt;&#xA;" OwnerUserId="779" LastEditorUserId="779" LastEditDate="2016-08-18T20:04:37.453" LastActivityDate="2016-08-18T20:04:37.453" Title="Solving a singular system of linear equations with smallest solution?" Tags="|linear-solver|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24735" PostTypeId="2" ParentId="24728" CreationDate="2016-08-16T08:29:39.033" Score="2" Body="&lt;p&gt;You can rewrite the limiter into a form that avoids division by zero, by checking the nominator and denominator with a couple of 'if' statements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can add a small number with the appropriate sign:&#xA;$$r_i = \frac{C(i+1,n)-C(i,n) \pm \epsilon}{C(i,n)-C(i-1,n) \pm \epsilon}$$&#xA;where $\epsilon$ is a small positive constant, that should be added to a positive term and subtracted from a negative term, to prevent $r_i$ from changing sign.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2016-08-16T08:29:39.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24736" PostTypeId="1" CreationDate="2016-08-16T10:15:36.230" Score="0" ViewCount="181" Body="&lt;p&gt;I am looking for a paper or return on experience regarding the use of Machine Learning in predictive maintenance in the context of data center?&lt;/p&gt;&#xA;" OwnerUserId="21355" LastEditorUserId="9667" LastEditDate="2016-08-16T15:21:30.550" LastActivityDate="2021-04-09T15:46:47.670" Title="Machine Learning and predictive maintenance" Tags="|machine-learning|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24737" PostTypeId="1" CreationDate="2016-08-15T21:06:15.110" Score="1" ViewCount="163" Body="&lt;p&gt;Let $|\psi_i\rangle$, $i=1...N+m$, be a set of overcomplete basis vector in a $N$-dim Hilbert space. The following are known: (Einstein's summation convention assumed)&#xA;$$\hat{H}|\psi_i\rangle=H_{ji}|\psi_j\rangle$$,&#xA;$$\lambda^\alpha_j|\psi_j\rangle=0,\alpha=1..m$$ &#xA;$$\langle \psi_i|\psi_j\rangle=S_{ij}$$&#xA;We already know the numerical values of $H_{ij}, S_{ij}, \lambda^\alpha_j$. Then how to find the eigenvalues of $\hat{H}$ using the common Matlab or Mathematica functions?&lt;/p&gt;&#xA;" OwnerDisplayName="Lagrenge" LastActivityDate="2016-08-16T11:46:33.773" Title="How to get the eigenvalues of Hamiltonian in an over complete basis" Tags="|quantum-mechanics|computational-physics|linear-algebra|" AnswerCount="0" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24738" PostTypeId="2" ParentId="24734" CreationDate="2016-08-16T12:24:07.280" Score="2" Body="&lt;p&gt;If you are willing to minimize the 2-norm of the solution rather than minimizing the number of non-zeros, you can use the pseudo-inverse. To solve A*x = b in MATLAB, that would be pinv(A)*b. In your example, this produces 2-norm of solution of 1.2120, vs. 2-norm of your preferred solution of 1.2207.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can come closer to your objective by minimizing the one-norm of x, subject to A*x=b. That is a Linear Programming problem. In this case, it produces your preferred result, but is not guaranteed to minimize the number of non-zeros in general (unless something about the special properties of your A and b wind up doing so?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you really want is to minimize the 0-norm of x, subject to A*x=b. You can solve this as a Mixed Integer Linear Programming problem by using a big M approach to minimize the number of elements of x which are more than some numerical tolerance, say 1e-6, from 0.  This is easy to do in YALMIP under MATLAB using YALMIP&quot;s &quot;implies&quot;, but is more computationally intensive than the previously mentioned approaches.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2016-08-16T12:24:07.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24739" PostTypeId="1" CreationDate="2016-08-16T14:07:40.133" Score="1" ViewCount="43" Body="&lt;p&gt;I'm looking for assistance in understanding a concise portion of a paper that involves a quadratic equation, so that I can properly implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 2 versions of the paper. Each paper utilizes a slightly different quadratic form to achieve the same goal, but I've found success with neither. Here are the papers (along with the corresponding sections) in question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://graphics.cs.uh.edu/website/satelliteLinks/GI2011_formationSketching_preprint.pdf&quot; rel=&quot;nofollow&quot;&gt;http://graphics.cs.uh.edu/website/satelliteLinks/GI2011_formationSketching_preprint.pdf&lt;/a&gt;&lt;br&gt;&#xA;(Section 3.6)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://graphics.cs.uh.edu/website/satelliteLinks/2011_CGA-crowd-formation-generation-preprint.pdf&quot; rel=&quot;nofollow&quot;&gt;http://graphics.cs.uh.edu/website/satelliteLinks/2011_CGA-crowd-formation-generation-preprint.pdf&lt;/a&gt;&lt;br&gt;&#xA;(Section 'Boundary sketches'. The quadratic can be found on page 18)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My math expertise is limited. I am primarily confused when I am asked to solve the quadratic for a single value, since I'm only aware of how to solve quadratic equations in a way that there are 2 possible solutions. How do I solve the quadratic in a way that I would find a single value for &lt;em&gt;b&lt;/em&gt; in the context of the application? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: Here are the equations and a simple description:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given an arbitrary shape and an arbitrary integer &quot;sampling unit&quot;, the sampling unit is used to both place points along the boundary of the shape, as well as to employ a flood fill algorithm to fill the rest of the shape, generating a formation for a group of AI agents $n$. Solving either quadratic below will yield a more optimized/desired boundary point count $b$, which is then used to compute a more optimized sampling unit for a more realistic formation based on the number of agents $n$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I'm looking to understand how to properly solve for&lt;/em&gt; $b$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b$ = Desired number of boundary agents&lt;br&gt;&#xA;$n$ = Number of AI agents&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Version 1&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b{}'$ = Number of points &lt;em&gt;on&lt;/em&gt; the boundary of the initial formation&lt;br&gt;&#xA;$f{}'$ = Number of points &lt;em&gt;inside&lt;/em&gt; the boundary of the initial formation  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b^{2}+(2\times (n+\frac{\sqrt{f{}'}}{b{}'}))\times b +n^{2} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Version 2&lt;/em&gt;:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r$ = Ratio between boundary points and inside points of the initial formation  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b^{2}+r^{^2}\times b-r^{2}\times n=0$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: I'm aware this may not be the right place for this question. I'd appreciate references to any other more appropriate places.&lt;/p&gt;&#xA;" OwnerUserId="21358" LastEditorUserId="21358" LastEditDate="2016-08-17T15:08:54.893" LastActivityDate="2016-08-17T15:08:54.893" Title="Formation sketching; trouble with sampling unit optimization" Tags="|algorithms|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24740" PostTypeId="2" ParentId="8899" CreationDate="2016-08-16T15:36:03.260" Score="3" Body="&lt;p&gt;I have used the description at &lt;a href=&quot;http://www.lucidarme.me/?p=4624&quot; rel=&quot;nofollow&quot;&gt;http://www.lucidarme.me/?p=4624&lt;/a&gt; to create this C++ code. The Matrices are those of the Eigen library, but you can easily create your own data structure from this example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=U\Sigma V^T$&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;Eigen/Core&amp;gt;&#xA;using namespace Eigen;&#xA;&#xA;Matrix2d A;&#xA;// ... fill A&#xA;&#xA;double a = A(0,0);&#xA;double b = A(0,1);&#xA;double c = A(1,0);&#xA;double d = A(1,1);&#xA;&#xA;double Theta = 0.5 * atan2(2*a*c + 2*b*d,&#xA;                           a*a + b*b - c*c - d*d);&#xA;// calculate U&#xA;Matrix2d U;&#xA;U &amp;lt;&amp;lt; cos(Theta), -sin(Theta), sin(Theta), cos(Theta);&#xA;&#xA;double Phi = 0.5 * atan2(2*a*b + 2*c*d,&#xA;                         a*a - b*b + c*c - d*d);&#xA;double s11 = ( a*cos(Theta) + c*sin(Theta))*cos(Phi) +&#xA;             ( b*cos(Theta) + d*sin(Theta))*sin(Phi);&#xA;double s22 = ( a*sin(Theta) - c*cos(Theta))*sin(Phi) +&#xA;             (-b*sin(Theta) + d*cos(Theta))*cos(Phi);&#xA;&#xA;// calculate S&#xA;S1 = a*a + b*b + c*c + d*d;&#xA;S2 = sqrt(pow(a*a + b*b - c*c - d*d, 2) + 4*pow(a*c + b*d, 2));&#xA;&#xA;Matrix2d Sigma;&#xA;Sigma &amp;lt;&amp;lt; sqrt((S1+S2) / 2), 0, 0, sqrt((S1-S2) / 2);&#xA;&#xA;// calculate V&#xA;Matrix2d V;&#xA;V &amp;lt;&amp;lt; signum(s11)*cos(Phi), -signum(s22)*sin(Phi),&#xA;     signum(s11)*sin(Phi),  signum(s22)*cos(Phi);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the standard sign function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double signum(double value)&#xA;{&#xA;    if(value &amp;gt; 0)&#xA;        return 1;&#xA;    else if(value &amp;lt; 0)&#xA;        return -1;&#xA;    else&#xA;        return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This results in exactly the same values as the &lt;code&gt;Eigen::JacobiSVD&lt;/code&gt; (see &lt;a href=&quot;https://eigen.tuxfamily.org/dox-devel/classEigen_1_1JacobiSVD.html&quot; rel=&quot;nofollow&quot;&gt;https://eigen.tuxfamily.org/dox-devel/classEigen_1_1JacobiSVD.html&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="21359" LastEditorUserId="21359" LastEditDate="2016-08-16T16:09:09.613" LastActivityDate="2016-08-16T16:09:09.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24741" PostTypeId="1" AcceptedAnswerId="24747" CreationDate="2016-08-16T20:04:53.557" Score="0" ViewCount="73" Body="&lt;p&gt;I'm no expert on the different types of algorithms to compute eigenvalues and vectors for a real, symmetric matrix (coming from linear mass and stiffness matrices for a frame FEA model). I am looking for a good blend of computational efficiency and ease of coding for a system of ~300 DoFs where I only need the first N (N between 1 and 10) eigenvalues and vectors. I will be programming it in VBA, so the less bundled algorithms (e.g. decompositions and things) that I need to program are a factor, too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was figuring on inverse power iterations with shifting, but am concerned about stability or skipping an eigenvalue if I shift too much or too little. Is this a realistic concern?&lt;/p&gt;&#xA;" OwnerUserId="16541" LastActivityDate="2016-08-17T16:05:49.033" Title="Appropriate algorithm for smallest several eigen(values|vectors) of a ~300 DoF system?" Tags="|eigenvalues|iterative-method|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24743" PostTypeId="1" CreationDate="2016-08-16T22:16:37.783" Score="1" ViewCount="95" Body="&lt;p&gt;Suppose you have the following analytical information about a function $f(x)$ on $[-1,1]$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;f(x) &amp;amp;= \frac{1}{x+3}\\&#xA;f'(x) &amp;amp; = -\frac{1}{(x+3)^2}\\&#xA;f^{''}(x) &amp;amp;= \frac{2}{(x+3)^3}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are asked to find the linear polynomial that is the minimax (best) approximation to $f(x)$ on the interval $[-1,1]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus we must find three points that have equal by alternating sign of the error. Let $q_1(x) = \alpha x + \beta$. Since $f(x)$ is monotonically increasing on $[-1,1]$ i.e. $f'(x) &amp;lt; 0$ we therefore I am told that we need to look for an interior point $c$ such that $$e'(c) = f'(c) - \alpha = 0 = -(c+3)^{-2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been trying to understand why this is the case as in why do we need to find an interior point when $f'(x) &amp;lt; 0$? What would happen if $f(x)$ was monotoncally increasing instead?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions or references are greatly appreciated. I am using the Numerical Analysis by Quarteroni, et al. but this particular section is short and does not go into much detail.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastActivityDate="2016-08-16T22:16:37.783" Title="A linear polynomial that is the minimax (best) approximation to $f(x)$ on the interval $[-1,1]$" Tags="|interpolation|polynomials|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24745" PostTypeId="1" CreationDate="2016-08-17T13:05:00.933" Score="1" ViewCount="38" Body="&lt;p&gt;I have a set of points in a 2D space. I want to connect the outer points so I get the convex hull. The problem here is that there is a limit to the distance between two points. Let me clarify that with a picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q2V0i.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q2V0i.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The green lines are lines that are good. The distance between the two points of the convex hull doesn't exceed the maximum distance (the outer circle of a point). But the red lines aren't correct. They exceed the maximum distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I need is a line, through multiple points, correctly drawn (e.g. all green lines), as an alternative for the red lines, while keeping the path a kind of convex hull.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been looking in concave hulls, and that might do the trick, but I haven't been able to find a good, preferably in Javascript, algorithm.&lt;/p&gt;&#xA;" OwnerUserId="21369" LastEditorUserId="9667" LastEditDate="2016-08-17T14:00:46.763" LastActivityDate="2016-08-17T14:00:46.763" Title="Find alternative path closest to original" Tags="|algorithms|computational-geometry|approximation|convex-hull|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24746" PostTypeId="1" CreationDate="2016-08-17T14:57:17.127" Score="1" ViewCount="62" Body="&lt;p&gt;Suppose we have some function $f(x)$ defined as a Chebyshev series up to order $M$:&#xA;$$ f(x) = \sum_{n=0}^{M} c_n T_n(x). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a given coefficients vector $\mathbf{c}$, and $x \in [-1,1]$ I'm looking to determine the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is $f(x) &amp;gt; 0$ everywhere?&lt;/li&gt;&#xA;&lt;li&gt;The &lt;em&gt;maximum&lt;/em&gt; value of the gradient $f'(x)$&lt;/li&gt;&#xA;&lt;li&gt;The number of stationary points i.e. where $f'(x) = 0$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These properties are of course not difficult to determine, but computational efficiency is an important factor for me in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd love any suggestions of efficient ways to answer these qustions starting only from the coefficients, and no prior calculation of $f(x)$ values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help!&lt;/p&gt;&#xA;" OwnerUserId="6700" LastActivityDate="2016-08-17T14:57:17.127" Title="Efficiently computing the properties of a Chebyshev series" Tags="|linear-algebra|optimization|algorithms|numerical-analysis|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24747" PostTypeId="2" ParentId="24741" CreationDate="2016-08-17T16:05:49.033" Score="3" Body="&lt;p&gt;A couple of options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A matrix of size 300 is very small, and computing this many eigenvalues and vectors shouldn't take more than a couple seconds on any modern computer. Usually the QR algorithm is used when computing all eigenvalues. It is advisable to use LAPACK routines for this, they have great documentation and are widely used. Implementing this should be very easy and quick.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If (1.) doesn't cut it, you can indeed find just the smallest eigenvalues and vectors you need (which should be quicker if you can provide the inverse operator). There are many options; power iteration and the Arnoldi method are often used. The Arnoldi method can be implemented using ARPACK and a whole suite of other methods are included in SLEPc (which gives you some flexibility).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-08-17T16:05:49.033" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24749" PostTypeId="1" CreationDate="2016-08-17T19:14:00.547" Score="3" ViewCount="125" Body="&lt;p&gt;I am trying to implement (in Python for now) low thrust orbit propagation for spacecraft using &lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_variable_formulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal variables&lt;/a&gt;. For a given central body with the gravitational parameter &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and an orbit with the semi-major axis &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and the initial position &lt;span class=&quot;math-container&quot;&gt;$\vec{r}_0$&lt;/span&gt; and velocity &lt;span class=&quot;math-container&quot;&gt;$\vec{v}_0$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$t = t_0$&lt;/span&gt; the position for a given time is given by:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{r} = \vec{r}_0 \ f(s) + \vec{v}_0 \ g(s)&#xA;$$&lt;/span&gt;&#xA;and the velocity by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{v} = \vec{r}_0\ \dot{f}(s) + \vec{v}_0\ \dot{g}(s)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(s) = 1-\left(\frac{\mu}{|\vec{r}_0|}\right) s^2 c_2(\alpha s^2) &#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;g(s) = t-t_0-\mu s^3c_3(\alpha s^2)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;With &lt;span class=&quot;math-container&quot;&gt;$\alpha=\frac{\mu}{a}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$c_n$&lt;/span&gt; the n-th Stumpff function and &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; the solution of&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;t-t_0=|\vec{r}_0| s \ c_1(\alpha s^2) + |\vec{r}_0| s^2 \ c_2(\alpha s^2) + \mu s^3 \ c_3(\alpha s^3)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am calculating &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; by using Newton's Method and everything works fine for scenarios without thrust. The orbit is elliptical and closed, parabolic for higher initial velocities, if I choose a &lt;span class=&quot;math-container&quot;&gt;$\vec{r}_0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\vec{v}_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt; and propagate &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; forward.&lt;/p&gt;&#xA;&lt;p&gt;For cases with thrust where I would change the velocity in every iteration&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{v}_{k} = \vec{v}_{k-1} + \vec{a}_{k} \cdot \Delta t&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;it is necessary to update &lt;span class=&quot;math-container&quot;&gt;$\vec{r}_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\vec{v}_0$&lt;/span&gt; every time, propagate &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; forward and repeat instead of choosing initial conditions and just propagating &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; forward. In preparation for that I chose not to add any velocity so both approaches should yield the same results.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Here is my Problem:&lt;/strong&gt;&#xA;Even after the first iteration the differences are large and the error increases fast&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/opE2X.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/opE2X.png&quot; alt=&quot;blue: fixed initial conditions, propagating &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;; red: propagating initial conditions, fixed &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;blue: fixed initial conditions, propagating &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;red: propagating initial conditions, fixed &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I thought I could solve this by using a more advanced integration algorithm like Runge-Kutta and tried to calculate every iteration in as few steps as possible but I was not able to transform the equations so that I could use Runge-Kutta (since it is not an ODE) and reducing the steps did not help at all.&lt;/p&gt;&#xA;&lt;p&gt;Can someone help to fix this or give any hints to why this error is so large?&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.optimize import newton&#xA;import matplotlib.animation as animation&#xA;import cmath as mth&#xA; &#xA; &#xA;plt.close(&amp;quot;all&amp;quot;)&#xA; &#xA;#defining some constants&#xA;gamma           = 4*np.pi**2*6.6108**3&#xA; &#xA;#stumpff functions&#xA;f_c0            = lambda x: mth.cos(x)&#xA;f_c1            = lambda x: mth.sin(x)/x&#xA;f_c2            = lambda x: (1.-mth.cos(x))/(x*x)&#xA;f_c3            = lambda x: (x-mth.sin(x))/(x*x*x)&#xA; &#xA;#f&#xA;f_uvf           = lambda mu,r0,s,a,sq:     1.-mu/r0*s**2*f_c2(sq)&#xA;#g&#xA;f_uvg           = lambda t,tau,mu,s,a,sq:  t-tau-mu*s**3*f_c3(sq)&#xA;#df/dt&#xA;f_uvdf          = lambda mu,r,r0,s,a,sq:   -(mu/(r*r0))*s*f_c1(sq)&#xA;#dg/dt&#xA;f_uvdg          = lambda mu,r,s,a,sq:      1.-mu/r*s**2*f_c2(sq)&#xA;    &#xA; &#xA;def getPos(t,tau,r0,v0,mu,a):&#xA;    &#xA;    #absolute values of initial position and velocity&#xA;    r0a     = np.sqrt(r0[0]*r0[0]+r0[1]*r0[1])&#xA;    v0a     = np.sqrt(v0[0]*v0[0]+v0[1]*v0[1])&#xA;    &#xA;    #semi-major axis from gravitational paramteter and initial conditions&#xA;    a       = (2./r0a-v0a**2./mu)**-1.&#xA;    &#xA;    #defining function to numericly find s&#xA;    def f_root(s):&#xA;        sqrtx = mth.sqrt(mu/a*s**2)   &#xA;        return r0a*s*f_c1(sqrtx)+r0a*s**2*f_c2(sqrtx)+mu*s**3*f_c3(sqrtx)-(t-tau)&#xA; &#xA;    #using newton algorithm to evaluate the function&#xA;    s       = newton(f_root,2,maxiter=100,tol=1e-10)&#xA;    &#xA;    sq = mth.sqrt(mu/a*s**2)&#xA;    &#xA;    #calculating the functions&#xA;    f       = f_uvf(mu,r0a,s,a,sq)&#xA;    g       = f_uvg(t,tau,mu,s,a,sq)&#xA;    &#xA;    #calculating new position&#xA;    rx          = r0[0]*f + v0[0]*g&#xA;    ry          = r0[1]*f + v0[1]*g&#xA;    &#xA;    #absolute value of new position&#xA;    ra          = np.sqrt(rx**2+ry**2)&#xA;    &#xA;    df          = f_uvdf(mu,ra,r0a,s,a,sq)&#xA;    dg          = f_uvdg(mu,ra,s,a,sq)&#xA;    &#xA;    #calculating new velocity&#xA;    vx          = r0[0]*df + v0[0]*dg&#xA;    vy          = r0[1]*df + v0[1]*dg&#xA; &#xA;    &#xA;    return (rx,ry),(vx,vy)&#xA;    &#xA; &#xA;#setting initial values&#xA;initial_r   = (0.,5.)&#xA;initial_v   = (30.,0.)&#xA; &#xA;#setting simulation parameters&#xA;dt          = .001&#xA;frames      = 1000&#xA; &#xA;#setting up plot&#xA;scale = 5.5&#xA; &#xA;fig = plt.figure()&#xA;ax = fig.add_subplot(111, aspect='equal', autoscale_on=False,xlim=(-scale, scale), ylim=(-scale, scale))&#xA;ax.grid()&#xA; &#xA;#creating list with points in time to propagate t&#xA;t   = np.linspace(0,frames*dt,frames+1)&#xA; &#xA;#creating list for position and velocity for fixed initial conditions while propagating t (method 1)&#xA;data_r = [initial_r]&#xA;data_v = [initial_v]&#xA; &#xA;#same for propagating initial conditions, fixed delta t (method 2)&#xA;data2_r = [initial_r]&#xA;data2_v = [initial_v]&#xA; &#xA;for i in range(len(t)-1):&#xA;    #evaluating with method 1&#xA;    data = getPos(t[i],0,data_r[0],data_v[0],gamma,3.5)&#xA;    #evaluating with method 2&#xA;    data2= getPos(dt,0,data2_r[i],data2_v[i],gamma,3.5)&#xA;    &#xA;    #adding data to lists&#xA;    data_r.append(data[0])&#xA;    data_v.append(data[1])&#xA;    &#xA;    data2_r.append(data2[0])&#xA;    data2_v.append(data2[1])&#xA; &#xA;#formatting data&#xA;data_r = list(zip(*data_r))&#xA;data_v = list(zip(*data_v))&#xA;data2_r = list(zip(*data2_r))&#xA;data2_v = list(zip(*data2_v))&#xA; &#xA;#plot data&#xA;ax.plot(data_r[0],data_r[1],'b')&#xA;ax.plot(data2_r[0],data2_r[1],'r')&#xA; &#xA;#creating line for animation&#xA;line, = ax.plot([0], [0], 'o-', lw=2)&#xA; &#xA;#animation loop&#xA;def animate(i):&#xA;    &#xA;    #accelerate animation&#xA;    di = i*10&#xA;    &#xA;    #animating mehtod 1&#xA;    #line.set_data([0,data_r[0][di]],[0,data_r[1][di]])&#xA;    &#xA;    #animating mehtod 2&#xA;    line.set_data([0,data2_r[0][di]],[0,data2_r[1][di]])&#xA; &#xA;    return line,&#xA; &#xA;#creating animation&#xA;ani = animation.FuncAnimation(fig, animate, interval=500, frames=frames, repeat=False)&#xA; &#xA;plt.show()&#xA;&#xA;RAW Paste Data&#xA;# -*- coding: utf-8 -*-&#xA;&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.optimize import newton&#xA;import matplotlib.animation as animation&#xA;import cmath as mth&#xA;&#xA;&#xA;plt.close(&amp;quot;all&amp;quot;)&#xA;&#xA;#defining some constants&#xA;gamma           = 4*np.pi**2*6.6108**3&#xA;&#xA;#stumpff functions&#xA;f_c0            = lambda x: mth.cos(x)&#xA;f_c1            = lambda x: mth.sin(x)/x&#xA;f_c2            = lambda x: (1.-mth.cos(x))/(x*x)&#xA;f_c3            = lambda x: (x-mth.sin(x))/(x*x*x)&#xA;&#xA;#f&#xA;f_uvf           = lambda mu,r0,s,a,sq:     1.-mu/r0*s**2*f_c2(sq)&#xA;#g&#xA;f_uvg           = lambda t,tau,mu,s,a,sq:  t-tau-mu*s**3*f_c3(sq)&#xA;#df/dt&#xA;f_uvdf          = lambda mu,r,r0,s,a,sq:   -(mu/(r*r0))*s*f_c1(sq)&#xA;#dg/dt&#xA;f_uvdg          = lambda mu,r,s,a,sq:      1.-mu/r*s**2*f_c2(sq)&#xA;    &#xA;&#xA;def getPos(t,tau,r0,v0,mu,a):&#xA;    &#xA;    #absolute values of initial position and velocity&#xA;    r0a     = np.sqrt(r0[0]*r0[0]+r0[1]*r0[1])&#xA;    v0a     = np.sqrt(v0[0]*v0[0]+v0[1]*v0[1])&#xA;    &#xA;    #semi-major axis from gravitational paramteter and initial conditions&#xA;    a       = (2./r0a-v0a**2./mu)**-1.&#xA;    &#xA;    #defining function to numericly find s&#xA;    def f_root(s):&#xA;        sqrtx = mth.sqrt(mu/a*s**2)   &#xA;        return r0a*s*f_c1(sqrtx)+r0a*s**2*f_c2(sqrtx)+mu*s**3*f_c3(sqrtx)-(t-tau)&#xA;&#xA;    #using newton algorithm to evaluate the function&#xA;    s       = newton(f_root,2,maxiter=100,tol=1e-10)&#xA;    &#xA;    sq = mth.sqrt(mu/a*s**2)&#xA;    &#xA;    #calculating the functions&#xA;    f       = f_uvf(mu,r0a,s,a,sq)&#xA;    g       = f_uvg(t,tau,mu,s,a,sq)&#xA;    &#xA;    #calculating new position&#xA;    rx          = r0[0]*f + v0[0]*g&#xA;    ry          = r0[1]*f + v0[1]*g&#xA;    &#xA;    #absolute value of new position&#xA;    ra          = np.sqrt(rx**2+ry**2)&#xA;    &#xA;    df          = f_uvdf(mu,ra,r0a,s,a,sq)&#xA;    dg          = f_uvdg(mu,ra,s,a,sq)&#xA;    &#xA;    #calculating new velocity&#xA;    vx          = r0[0]*df + v0[0]*dg&#xA;    vy          = r0[1]*df + v0[1]*dg&#xA;&#xA;    &#xA;    return (rx,ry),(vx,vy)&#xA;    &#xA;&#xA;#setting initial values&#xA;initial_r   = (0.,5.)&#xA;initial_v   = (30.,0.)&#xA;&#xA;#setting simulation parameters&#xA;dt          = .001&#xA;frames      = 1000&#xA;&#xA;#setting up plot&#xA;scale = 5.5&#xA;&#xA;fig = plt.figure()&#xA;ax = fig.add_subplot(111, aspect='equal', autoscale_on=False,xlim=(-scale, scale), ylim=(-scale, scale))&#xA;ax.grid()&#xA;&#xA;#creating list with points in time to propagate t&#xA;t   = np.linspace(0,frames*dt,frames+1)&#xA;&#xA;#creating list for position and velocity for fixed initial conditions while propagating t (method 1)&#xA;data_r = [initial_r]&#xA;data_v = [initial_v]&#xA;&#xA;#same for propagating initial conditions, fixed delta t (method 2)&#xA;data2_r = [initial_r]&#xA;data2_v = [initial_v]&#xA;&#xA;for i in range(len(t)-1):&#xA;    #evaluating with method 1&#xA;    data = getPos(t[i],0,data_r[0],data_v[0],gamma,3.5)&#xA;    #evaluating with method 2&#xA;    data2= getPos(dt,0,data2_r[i],data2_v[i],gamma,3.5)&#xA;    &#xA;    #adding data to lists&#xA;    data_r.append(data[0])&#xA;    data_v.append(data[1])&#xA;    &#xA;    data2_r.append(data2[0])&#xA;    data2_v.append(data2[1])&#xA;&#xA;#formatting data&#xA;data_r = list(zip(*data_r))&#xA;data_v = list(zip(*data_v))&#xA;data2_r = list(zip(*data2_r))&#xA;data2_v = list(zip(*data2_v))&#xA;&#xA;#plot data&#xA;ax.plot(data_r[0],data_r[1],'b')&#xA;ax.plot(data2_r[0],data2_r[1],'r')&#xA;&#xA;#creating line for animation&#xA;line, = ax.plot([0], [0], 'o-', lw=2)&#xA;&#xA;#animation loop&#xA;def animate(i):&#xA;    &#xA;    #accelerate animation&#xA;    di = i*10&#xA;    &#xA;    #animating mehtod 1&#xA;    #line.set_data([0,data_r[0][di]],[0,data_r[1][di]])&#xA;    &#xA;    #animating mehtod 2&#xA;    line.set_data([0,data2_r[0][di]],[0,data2_r[1][di]])&#xA;&#xA;    return line,&#xA;&#xA;#creating animation&#xA;ani = animation.FuncAnimation(fig, animate, interval=500, frames=frames, repeat=False)&#xA;&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21374" LastEditorUserId="23791" LastEditDate="2022-01-15T18:42:02.313" LastActivityDate="2022-01-15T18:42:02.313" Title="Numerical error in implementation of iterative algorithm" Tags="|computational-physics|iterative-method|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24750" PostTypeId="1" CreationDate="2016-08-17T19:32:06.940" Score="2" ViewCount="239" Body="&lt;p&gt;I'm dealing with a set of nonlinear coupled PDEs that have the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac {\partial y_1}{\partial t} &amp;amp;= y_2y_3 - y_1 \tag{1}\\&#xA;\frac {\partial y_2}{\partial t} &amp;amp;= y_1y_3 - y_2 \tag{2}\\&#xA;\frac {\partial^2 y_3}{\partial t^2} - \frac {\partial y_3}{\partial t} &amp;amp;=  \frac {\partial^2 y_3}{\partial z^2} + \frac {\partial y_3}{\partial z} + \frac {\partial^2 y_2}{\partial z^2} + \frac {\partial y_2}{\partial z} - \frac {\partial^2 y_2}{\partial t^2}  + \frac {\partial y_2}{\partial t} \tag{3}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial/boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;y_1(z,0) &amp;amp;= 10^{-5}\\&#xA;y_2(z,0) &amp;amp;= 10^4\\&#xA;y_3(z,0) &amp;amp;= 0\\&#xA;\frac {\partial y_3}{\partial t} (z,0) &amp;amp;= 0\\&#xA;y_1(0,t) &amp;amp;= 10^{-5}e^{-t}\\&#xA;y_2(0,t) &amp;amp;= 10^{4}e^{-t}\\&#xA;y_3(0,t) &amp;amp;= 0\\&#xA;\frac {\partial y_3}{\partial z} (0,t) &amp;amp;= 0&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An initial approach has been to use a finite difference scheme. One being where I simply use a method of lines and treat it as a set of ODEs. But with this setup, I am having trouble formulating the above equations in such a way to actually use a stiff solver such as Gear's method since I lack an explicit expression for $$\frac {\partial y_3}{\partial t}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By taking the time derivative of $(2)$, I can &quot;simplify&quot; $(3)$ to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\partial^2 y_3}{\partial t^2} + (y_1-1) \frac {\partial y_3}{\partial t} =  \frac {\partial^2 y_3}{\partial z^2} + \frac {\partial y_3}{\partial z} + \frac {\partial^2 y_2}{\partial z^2} + \frac {\partial y_2}{\partial z} + \frac {\partial y_2}{\partial t} - y_3 \frac {\partial y_1}{\partial t}&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;but that still doesn't help too much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on methods/approaches to solve the above equations?&lt;/p&gt;&#xA;" OwnerUserId="20407" LastEditorUserId="20407" LastEditDate="2016-08-18T17:20:12.483" LastActivityDate="2016-08-18T17:20:12.483" Title="Numerical methods for coupled stiff PDEs" Tags="|pde|numerical-analysis|nonlinear-equations|stiffness|coupling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24751" PostTypeId="2" ParentId="24750" CreationDate="2016-08-17T20:23:17.417" Score="1" Body="&lt;p&gt;The first step is to add one more equation defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial y_3}{\partial t} = v$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then if you substitute equations 1 and 2 and the new equation into your modified equation 3 you get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial v}{\partial t}=(1-y_1)v + \frac{\partial^2 y_3}{\partial z^2} + \frac{\partial y_3}{\partial z} +&#xA;\frac{\partial^2 y_2}{\partial z^2} + \frac{\partial y_2}{\partial z} + y_1y_3-y_2 -y_3(y_2y_3-y_1)&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-08-17T20:23:17.417" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24752" PostTypeId="1" CreationDate="2016-08-18T01:14:42.473" Score="2" ViewCount="5136" Body="&lt;p&gt;I am programming cubic element and very confused how to find interpolation function for cubic finite element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have element like this in bi-unit isoparametic system :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-1      \hspace{15pt}    -1/3  \hspace{15pt}        1/3       \hspace{20pt}   1$      &amp;lt;- distance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;|-------------|------------|-----------|&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$1     \hspace{30pt}         2    \hspace{30pt}         3       \hspace{30pt}     4       \hspace{15pt}$ &amp;lt;- Nodes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Physically, the element goes from 1 to 4. Nodes are located at $[1,2,3,4]$. In natural (isoparametric bi-unit system), nodes $[1,2,3,4]$ are located at $[-1,-1/3,1/3,1]$.&#xA;We have $\textbf{x} = N_1x_1 + N_2x_2 + N_3x_3 + N_4x_4 $.&#xA;Where, $N_i$ is shape function in geometric coordinates and $x_i$ are nodal locations. &#xA;Lagrangian interpolation functions for 1D in terms of physical coordinates are:&#xA;$$N_1(x) = \frac{ (x-x_2) (x-x_3) (x-x_4)}&#xA;{(x_1-x_2) (x_1-x_3) (x_1-x_4)}$$&#xA;$$N_2(x) = \frac{ (x-x_1) (x-x_3) (x-x_4)}&#xA;{(x_2-x_1) (x_2-x_3) (x_2-x_4)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;similarly for other two nodes (I am not writing them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of natural coordinates,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$N_1(\xi) = \frac{ (\xi-\xi_2) (\xi-\xi_3) (\xi-\xi_4)}&#xA;{(\xi_1-\xi_2) (\xi_1-\xi_3) (\xi_1-\xi_4)}$$&#xA;$$N_2(\xi) = \frac{ (\xi-\xi_1) (\xi-\xi_3) (\xi-\xi_4)}&#xA;{(\xi_2-\xi_1) (\xi_2-\xi_3) (\xi_2-\xi_4)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\textbf B = \bigg [\frac{\partial N_1}{\partial x}, \frac{\partial N_2}{\partial x},&#xA;\frac{\partial N_3}{\partial x}, \frac{\partial N_4}{\partial x} \bigg ]&#xA;\\&#xA;x(\xi) = N(\xi)x_i&#xA;$$&#xA;Jacobian $J$ is given by &#xA;$$J = \frac{\partial x}{\partial \xi} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we introduce isoparametric coordinates,&#xA;$$\textbf B = \frac{\partial N_i}{\partial x} = \frac{\partial N_i}{\partial \xi} \frac{\partial \xi}{\partial x} = \frac{\partial N_i}{\partial \xi}\frac{1}{J}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stiffness matrtix is given by&#xA;$$\textbf K = \int_{-1}^{1} \textbf{ [B]}^{T} \textbf{ E [B] J} d\xi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone tell me how to calculate $\textbf B$, $\textbf J$ for above given element? Many references only linear elements, in which we can easily get away with constants. I tried looking some codes, but no luck as many of them still had linear elements. Any references are welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After James' answer, I modify my question and reflect findings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is $$\frac{\partial N_i}{\partial \xi} = \begin{bmatrix} &#xA;-27\xi^2/16 + 9\xi/8 + 1/16 &amp;amp; &#xA;81\xi^2/16 - 9\xi/8 - 27/16 &amp;amp;&#xA;-27\xi^2/16 + 9\xi/8 + 1/16 &amp;amp;&#xA;-27\xi^2/16 + 9\xi/8 + 1/16 &amp;amp;&#xA;\end {bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_i= \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end {bmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, $$J(\xi) = \frac{\partial N_i}{\partial \xi} x_i$$ is function of $\xi$. &#xA;$$B(\xi) ^{T} = \frac{1}{J} \begin{bmatrix} &#xA;\frac{\partial N_1}{\partial \xi} &amp;amp; &#xA;\frac{\partial N_2}{\partial \xi} &amp;amp; &#xA;\frac{\partial N_3}{\partial \xi} &amp;amp; &#xA;\frac{\partial N_4}{\partial \xi} &amp;amp; &#xA;\end{bmatrix} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B$ is also a function of $\xi$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ K = \int_{-1}^{1} \begin{pmatrix}&#xA;\frac{1}{J} \begin{bmatrix} &#xA;\frac{\partial N_1}{\partial \xi} &amp;amp; &#xA;\frac{\partial N_2}{\partial \xi} &amp;amp; &#xA;\frac{\partial N_3}{\partial \xi} &amp;amp; &#xA;\frac{\partial N_4}{\partial \xi} &amp;amp; &#xA;\end{bmatrix} &#xA;\end{pmatrix}&#xA;D&#xA;\begin{pmatrix}&#xA;\frac{1}{J} \begin{bmatrix} &#xA;\frac{\partial N_1}{\partial \xi} \\ &#xA;\frac{\partial N_2}{\partial \xi} \\ &#xA;\frac{\partial N_3}{\partial \xi} \\ &#xA;\frac{\partial N_4}{\partial \xi} \\ &#xA;\end{bmatrix} &#xA;\end{pmatrix}&#xA;J d\xi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, we find $K$ by Gauss integral as &#xA;Are these formulas correct, especially $K$? Can someone throw light on how to find Gauss quadrature for $K$? &lt;/p&gt;&#xA;" OwnerUserId="21361" LastEditorUserId="7567" LastEditDate="2016-08-20T16:29:22.857" LastActivityDate="2016-08-25T18:29:17.223" Title="Finding jacobian matrix of cubic finite element" Tags="|finite-element|numerical-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24753" PostTypeId="2" ParentId="24734" CreationDate="2016-08-18T05:11:26.107" Score="0" Body="&lt;p&gt;I think I found a solution. I think the following algorithm can do the trick.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Define a set of restricted indexes $M$.&lt;/li&gt;&#xA;&lt;li&gt;Populate $M$ with indexes of all rows which have $1$ only on a diagonal (equal to: all rows for which the sum of its elements is $1$; or, for which all non-diagonal elements are $0$).&lt;/li&gt;&#xA;&lt;li&gt;Remove all rows and columns corresponding to those indexes.&lt;/li&gt;&#xA;&lt;li&gt;Add (original) index of every row for which the sum (of its elements in this reduced matrix) is $\ne 0$ to $M$.&lt;/li&gt;&#xA;&lt;li&gt;If you added such indexes, go to step 3.&lt;/li&gt;&#xA;&lt;li&gt;Otherwise you are left with rows which are making the matrix ill-defined.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the original matrix we can now change all rows with indexes which are missing in $M$ such that for those rows only diagonal elements are 1 and the rest of elements 0. Then we solve the original problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the above example we start with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{array}{cccc}&#xA;1 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; -0.3 &amp;amp; -0.7 \\&#xA;-1 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;\end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Removing the last row and last column:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{array}{ccc}&#xA;1 &amp;amp; 0 &amp;amp; -1 \\&#xA;0 &amp;amp; 1 &amp;amp; -0.3 \\&#xA;-1 &amp;amp; 0 &amp;amp; 1 \\&#xA;\end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Removing the second row and the second column:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{array}{cc}&#xA;1 &amp;amp; -1 \\&#xA;-1 &amp;amp; 1 \\&#xA;\end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rest are resulting rows. We modify them in the original matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ \begin{array}{cccc}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; -0.3 &amp;amp; -0.7 \\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;\end{array} \right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this we can go and normally solve the matrix equation.&lt;/p&gt;&#xA;" OwnerUserId="779" LastEditorUserId="779" LastEditDate="2016-08-18T19:58:57.940" LastActivityDate="2016-08-18T19:58:57.940" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24754" PostTypeId="2" ParentId="24752" CreationDate="2016-08-18T05:39:40.520" Score="0" Body="&lt;p&gt;From $x(\xi) =N(\xi)x_{i}$ the jacobian is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  J = \frac{\partial{x}}{\partial{\xi}} = \Sigma\frac{\partial{N_{i}}}{\partial{\xi}}x_{i}&#xA;\end{equation} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integration itself is done at quadrature points (and thus the jacobian is found by evaluating $\frac{\partial{N}}{\partial{\xi}}$ at quadrature points)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good reference is &lt;a href=&quot;https://www.google.ca/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=7&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=0ahUKEwi-3-jln8rOAhUC3GMKHVOkAPgQFgg8MAY&amp;amp;url=http%3A%2F%2Fhomepages.cae.wisc.edu%2F~suresh%2FME964Website%2FM964Notes%2FNotes%2Fintrofem.pdf&amp;amp;usg=AFQjCNHznffrxu4qW2NITEokDhj8_wUCxg&amp;amp;sig2=U1yq2Ur5r33Ru6TOGOYpBQ&quot; rel=&quot;nofollow&quot;&gt;Introduction to the Finite Element Method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some more detail using my own code (C++). Consider a shape function for a quadratic 1D line given by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void LineQuad1D::shape(double xi)&#xA;{&#xA;  //shape functions for quadratic 1D lines at node (xi)&#xA;  N[0] = -0.5*xi*(1-xi);&#xA;  N[1] = 0.5*xi*(1+xi);&#xA;  N[2] = 1-xi*xi;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and corresponding derivative:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void LineQuad1D::dshape(double xi)&#xA;{&#xA;  //derivatives of shape functions for quadratic 1D lines at node (xi)&#xA;  dN[0] = -0.5+xi; dN[1] = 0.5+xi; dN[2] = -2.0*xi;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now finding the jacobian at point might look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double Element::jacobian(double xi)&#xA;{&#xA;  Shape *sh = ShapeFactory::NewShape(Type);&#xA;  (*sh).dshape(xi);&#xA;&#xA;  jmat[0][0] = 0.0;&#xA;  for(int k=0;k&amp;lt;Npe;k++){&#xA;    jmat[0][0] += (*sh).dN[k]*xpts[k];&#xA;  }&#xA;&#xA;  //jacobian, jacobian matrix, and inverse jacobian matrix&#xA;  jac = jmat[0][0];  //in your case the dimenion is 1, so you wouldn't need a 2D array for the jacobian&#xA;  imat[0][0] = 1.0/jac;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finding the $B$ matrix at a point looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double Element::tempDiffMatrix(double xi)&#xA;{&#xA;  Shape *sh = ShapeFactory::NewShape(Type);&#xA;&#xA;  (*sh).dshape(xi);&#xA;&#xA;  //set jacobian and inverse jacobian matrix and return jacobian&#xA;  double jac = jacobian(xi);&#xA;&#xA;  //temperature differentiation matrix B&#xA;  for(int i=0;i&amp;lt;Npe;i++){&#xA;    bmat[i] = imat[0][0]*(*sh).dN[i];&#xA;  }&#xA;&#xA;  return jac;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And finally stiffness matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void Element::stiffnessMatrix()&#xA;{&#xA;  for(int i=0;i&amp;lt;Npe;i++){&#xA;    for(int j=0;j&amp;lt;Npe;j++){&#xA;      kmat[i][j] = 0.0; //stiffness matrix&#xA;    }&#xA;  }&#xA;&#xA;  double u=1.0&#xA;&#xA;  QuadratureRule g(nGauss,nDim,Type);&#xA;  for(int ip=0;ip&amp;lt;g.nIntPoints;ip++) //loop through quadrature points&#xA;  {&#xA;    u=g.xii[ip];&#xA;&#xA;    double det = tempDiffMatrix(u);  //creates your B matrix and returns Jacobian, both at integration points xii[ip]&#xA;    double dv = det*g.wi[ip];&#xA;&#xA;    for(int i=0;i&amp;lt;Npe;i++){   // Npe = &quot;Number of Points per Element&#xA;      for(int j=0;j&amp;lt;Npe;j++){&#xA;        double s = 0.0;&#xA;        for(int k=0;k&amp;lt;nDim;k++)  // nDim = 1 in your case&#xA;          s += bmat[k][i]*bmat[k][j];&#xA;        kmat[i][j] += dv*s;&#xA;      }&#xA;    }&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My code actually solves for 1, 2, and 3 dimensions but I have tried to edit out the multi-dimensionality for you since it looks like you are only worried about 1D. Hopefully you can parse the basic idea from these code snippets. Also note that the these code snippets are for solving the heat equation (i.e. no $E$ matrix), but again hopefully you can parse the basic idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok so for more detail on the Gauss quadrature part here is an example found in &quot;Programming Finite Elements in Java&quot; by Gennadiy Nikishkov. Consider trying to integrate the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  I = \int_{-1}^{1}f(\xi)d\xi,&#xA;\end{equation} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;using two integration points. Because we are using two integration points this should integrate to a third degree polynomial. We write the integral as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  I = f(\xi_{1})w_{1} + f(\xi_{2})w_{2}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $w_{i}$ are the weights and $\xi_{1}$ are the undetermined points. This formula should be valid for any third degree polynomial, in particular it should work for $f(\xi)=1$, $f(\xi)=\xi$, $f(\xi)=\xi^{2}$, and $f(\xi)=\xi^{3}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \int_{-1}^{1}d\xi = 2 = w_{1} + w_{2},&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \int_{-1}^{1}\xi d\xi = 0 = \xi_{1}w_{1} + \xi_{2}w_{2},&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \int_{-1}^{1}\xi^{2} d\xi = 2/3 = \xi_{1}^{2}w_{1} + \xi_{2}^{2}w_{2},&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \int_{-1}^{1}\xi^{3} d\xi = 0 = \xi_{1}^{3}w_{1} + \xi_{2}^{3}w_{2},&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving these results in $\xi_{1}=\frac{1}{\sqrt{3}}$, $\xi_{2}=-\frac{1}{\sqrt{3}}$, $w_{1} = w_{2} = 1$. In general for the one dimensional case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  I = \int_{-1}^{1}f(\xi)d\xi = \Sigma_{i=1}^{n}f(\xi_{i})w_{i},&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where n is the number of integration points, $\xi_{i}$ the undetermined points, and $w_{i}$ the weights. So in my &lt;code&gt;stiffnessMatrix()&lt;/code&gt; function/method above you can see that in pseudo code I am doing the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each integration point p:&#xA;    find the Jacobian, dv, at undetermined point xi[p]&#xA;    find weight corresponding to undetermined point xi[p]&#xA;    find det = jacobian*weight corresponding to point xi[p] &#xA;    find B matrix at the undetermined point xi[p]&#xA;    for i = 1:number of ponts in element:&#xA;        for j = number of points in element:&#xA;          K[i][j] += dv*B[i]B[j]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These undetermined points and weights would be stored in an array or table.&lt;/p&gt;&#xA;" OwnerUserId="7567" LastEditorUserId="7567" LastEditDate="2016-08-20T03:21:51.060" LastActivityDate="2016-08-20T03:21:51.060" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24755" PostTypeId="1" CreationDate="2016-08-18T12:36:28.050" Score="1" ViewCount="230" Body="&lt;p&gt;I am working with MATLAB, and I would have the following problem. I have the following matrix of random numbers &lt;code&gt;rand(200,200)&lt;/code&gt; &#xA;and I need to create  a 3D matrix with the following structure &lt;code&gt;3D_mat(r,s,p)&lt;/code&gt; with 15 layers (&lt;code&gt;p=15&lt;/code&gt;), where each layer will contain the  given data &lt;code&gt;rand(200,200)&lt;/code&gt;. Perhaps, it is easy task, but I am MATLAB beginner.&lt;/p&gt;&#xA;" OwnerUserId="21383" LastEditorUserId="19501" LastEditDate="2016-08-18T22:18:15.263" LastActivityDate="2016-10-20T19:49:28.710" Title="Data cloning into 3D matrix" Tags="|matlab|matrix|3d|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24757" PostTypeId="2" ParentId="24755" CreationDate="2016-08-18T13:37:11.107" Score="1" Body="&lt;p&gt;This is probably not the most efficient way to do this in MATLAB, but the following works for me in Octave:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A=rand(200,200);&#xA;for i=1:15&#xA;  B(:,:,i)=A;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And, I think it should work fine in MATLAB as well. If your dimensions get a lot bigger or you have to create the 3D matrix frequently, there are probably much more efficient was to accomplish this.&lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2016-08-18T13:37:11.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24758" PostTypeId="2" ParentId="24730" CreationDate="2016-08-18T18:01:37.140" Score="2" Body="&lt;p&gt;Augmenting what Bill Barth already said: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1/ You don't want to use polynomials because many methods (such as finite element methods) are exact for at least lower order polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2/ You want to choose the solution to be analytic or at least $C^\infty$ so that the solution can be well approximated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These two requirements naturally lead to trig functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the coefficients:&#xA;3/ You want to choose $a_{\rho x}$ large enough so that you get more than, say, half of an oscillation into your domain width.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4/ But you want to choose $a_{\rho x}$ small enough so that you have a chance of resolving the solution with your mesh. For example, even on the coarsest mesh, you probably want at least 2 mesh sizes per period of the (co)sine, and then go to more and more elements per period as you refine the mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These last two points lead to pretty obvious choices for the frequency of the (co)sines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the fore-factors: they typically do not matter. At least for linear problems, they just scale right through (if you choose the solution twice as large, the error will also be twice as large). For nonlinear problems, however, you will want to choose these factors so that the nonlinearity is there and important, but not overwhelming your algorithm.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-18T18:01:37.140" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24759" PostTypeId="2" ParentId="24755" CreationDate="2016-08-18T21:31:30.027" Score="0" Body="&lt;p&gt;Here are three options: with &lt;code&gt;repmat&lt;/code&gt; or &lt;code&gt;bsxfun&lt;/code&gt;, two functions evoked by @Christian Clason. On some versions of Matlab (notably older than 2013, as can be seen from &lt;a href=&quot;https://stackoverflow.com/questions/16746999/most-efficient-way-for-repeating-a-vector-in-matlab&quot;&gt;https://stackoverflow.com/questions/16746999/most-efficient-way-for-repeating-a-vector-in-matlab&lt;/a&gt;), &lt;code&gt;bsxfun&lt;/code&gt; was faster, and if not the case, is believed to use less memory (useful if you get much bigger than $200\times 200\times15$). The last one plays with indexing.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nRow = 200;&#xA;nCol = 200;&#xA;nDep = 15;&#xA;&#xA;r = rand(nRow,nCol);&#xA;o = zeros(nRow,nCol,nDep); % Not bad to initialize&#xA;&#xA;% Choose one&#xA;o = repmat(r,1,1,nDep); % repmat version&#xA;o = bsxfun(@times,ones(1, 1 , nDep),r); % bsxfun version&#xA;o = r(:,:,ones(1, 1 , nDep)); % multiple-indexing version&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my case (Matlab2013b), repmat is $2/3$ faster than the two others, for several matrix sizes and repetition depth (including yours). According to some testimonies, since  a novel coding of &lt;code&gt;repmat&lt;/code&gt; exist in Matlab2013b and above, &lt;code&gt;bsxfun&lt;/code&gt; indeed can be slower.&lt;/p&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-08-19T15:32:40.083" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24760" PostTypeId="1" CreationDate="2016-08-18T21:44:45.893" Score="0" ViewCount="80" Body="&lt;p&gt;Suppose I have a real vector $\bf x$ of length $KN$, where $K&amp;lt;&amp;lt;N$.  Let's say I break this vector $\bf x$ up into $N$ vectors each of length $K$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${\bf x}_1,...{\bf x}_n.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I would like to approximate the sum of their 2-norms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(K, N, {\bf x})\approx \sum_{n=1}^N\left |\right |{\bf x}_n\left |\right |_2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $f$ can be very efficiently computed (when compared to computing the sum of 2-norms).  Is there a standard way to do this?&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="1804" LastEditDate="2016-08-18T22:21:27.350" LastActivityDate="2016-09-24T01:38:14.780" Title="Efficiently approximating sum of 2-norms" Tags="|linear-algebra|vector|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24761" PostTypeId="1" AcceptedAnswerId="24762" CreationDate="2016-08-18T22:46:27.130" Score="1" ViewCount="54" Body="&lt;p&gt;Assume I have a sequence of vectors $v_k \in V$ in some abstract vector space $V$ not necessarily equal to $\mathbb{C}^n$. Can I still use the Householder QR decomposition in this case, even though the concept of &quot;reflecting onto the first coordinate direction&quot; doesn't make sense if $V \neq \mathbb{C}^n$? Or do I have to use the less stable modified Gram-Schmidt process to find an orthogonal basis for $\DeclareMathOperator{Span}{span}\Span\{v_k\}$?&lt;/p&gt;&#xA;" OwnerUserId="21342" LastActivityDate="2016-08-18T23:59:35.380" Title="Householder QR for abstract vectors" Tags="|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24762" PostTypeId="2" ParentId="24761" CreationDate="2016-08-18T23:59:35.380" Score="2" Body="&lt;p&gt;If $V$ is an inner product space, you can define &lt;a href=&quot;https://en.wikipedia.org/wiki/Householder_operator&quot; rel=&quot;nofollow&quot;&gt;Householder operators&lt;/a&gt; that generalize Householder reflectors. If $V$ is finite-dimensional and you have an orthonormal basis with respect to the inner product of $V$, you should be able to compute QR via Householder reflectors. If you only have a basis, you could orthogonalize it via (modified) Gram-Schmidt and then proceed as above.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2016-08-18T23:59:35.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24763" PostTypeId="2" ParentId="24733" CreationDate="2016-08-19T04:01:45.297" Score="1" Body="&lt;p&gt;The implicit/explicit designation refers to the finite difference time stepping scheme used to compute dynamic models.  &lt;em&gt;Nonlinear Finite Element Methods&lt;/em&gt; by Peter Wriggers talks about the two different methods, and virtually any textbook on numerical methods for partial differential equations will discuss the difference as well.  There are a wide range of implicit implementations that provide for convergence checking or automatic scaling of time steps.  Wriggers talks about it a little bit, but you can also learn a lot from the documentation for LS-DYNA or ABAQUS.&lt;/p&gt;&#xA;" OwnerUserId="21391" LastActivityDate="2016-08-19T04:01:45.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24764" PostTypeId="1" CreationDate="2016-08-19T06:04:38.837" Score="1" ViewCount="318" Body="&lt;p&gt;I am simulating acoustic wave equation in which absorbing boundary condition has to be applied. It is applied in two ways &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ist method&lt;/strong&gt; is as mentioned in &lt;a href=&quot;http://web.gps.caltech.edu/~clay/PDF/Clayton1977.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt;. &#xA;Boundary condition at bottom is (equation 9 in paper)&lt;br&gt;&#xA;$$ &#xA;P_{zt} + \frac{1}{v} P_{tt} - \frac{v}{2} P_{xx}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is finite differenced as following (see appendix in above referred paper):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Bottom boundary at k=K' &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;D^{-}_z D_t^0 P_{i,K}^n  &#xA;+ \frac{v}{2} D_t^+ D_{t}^{-}  (P_{i,K}^n  + P_{i,K-1}^n)  &#xA;- \frac{v}{4} D_x^+ D_{x}^{-} (P_{i,K-1}^{n+1} + P_{i,K}^{n-1}) &#xA;= 0 &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here $D$ represent the derivative; subscripts  x,z and t represent the spatial directions (x and z) and time (t); superscripts +,- and 0 represent the forward, backward, and centered approximations for respective derivatives; and  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Notation used above is different from the paper. I am writing FD equations the way it is written generally.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about the last term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What advantage we can get by writing last term in the form &#xA;$&#xA;\frac{v}{4} D_x^+ D_{x}^{-} (P_{i,K-1}^{n+1} + P_{i,K+1}^{n-1}) &#xA;$&#xA; instead of writing like&#xA;$&#xA;\frac{v}{4} D_x^+ D_{x}^{-} (P_{i,K-1}^{n} + P_{i,K}^{n}) &#xA;$.&#xA; Both are centered at &#xA;$&#xA;n\Delta t,(K-\frac{1}{2})\Delta z&#xA;$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2nd way&lt;/strong&gt; is the way defined in the paper &quot;Boundary condition for the numerical solution of wave propagation problem , By Albert C reynolds, 1978, Geophysics(vol.43,No 6)&quot; (I couldn't find its open access)&#xA;The final equation is &#xA;$$&#xA;(\frac{1}{c}\frac{\partial}{\partial t} + \frac{\partial}{\partial x})&#xA;(\frac{p}{c}\frac{\partial}{\partial t} + \frac{\partial}{\partial x}) =0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the author used the first part and written its finite differenced form as&#xA; $$&#xA;u^{j+1}_{m,N+1} = u^j_{m,N+1} + u^j_{m,N} - u^{j-1}_{m,N} - &#xA;      \frac{c \Delta t}{\Delta x}&#xA;       (u^j_{m,N+1} - u^j_{m,N} - (u^{j-1}_{m,N} - u^{j-1}_{m,N-1}))&#xA;$$&#xA;I interpreted above form as &#xA;$$&#xA;D_t^+ u^j_{m,N+1} -D_t^- u^j_{m,N} = D_x^+ u^j_{m,N} - D^-_xu^{j-1}_{m,N}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here &#xA;$$&#xA;u_{m,n}^j &#xA;= u^{\Delta t*j}_{\Delta x *m, \Delta z* n} &#xA;= u^t_{x,z}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here I could not find any cenerting in time or space. Also I found it uses difference insted of averaging...!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How it was formulated?&lt;/p&gt;&#xA;" OwnerUserId="20015" LastEditorUserId="20015" LastEditDate="2016-08-22T13:12:21.917" LastActivityDate="2017-01-19T16:59:06.200" Title="FD implementation of Absorbing Boundary condition for acoustic wave" Tags="|pde|finite-difference|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24765" PostTypeId="1" CreationDate="2016-08-19T08:51:13.950" Score="6" ViewCount="495" Body="&lt;p&gt;In the finite element method, we often construct the constraints of the system by adding penalty-function terms ( which often are many many magnitudes, up to $10^6$ order bigger than the largest stiffness matrix term) to the stiffness matrix by using the &lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch09.d/IFEM.Ch09.pdf&quot; rel=&quot;noreferrer&quot;&gt;so-called Penalty Method (section 9.2)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0i85U.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0i85U.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;On the other hand,&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the eigenvalue analysis, as implemented by &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;noreferrer&quot;&gt;ARPACK&lt;/a&gt;, makes use of &lt;a href=&quot;https://en.wikipedia.org/wiki/Arnoldi_iteration#Implicitly_restarted_Arnoldi_method_.28IRAM.29&quot; rel=&quot;noreferrer&quot;&gt;Arnoldi iteration scheme&lt;/a&gt;, which involves working on the stiffness matrix many times over in the iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My hunch is that, the out-of-magnitude penalty term in stiffness matrix will make the whole Arnoldi iteration unstable, when it is iterated many times over, resulting in inaccuracy when we compute the eigenvalue analysis. I also don't know how to estimate the inaccuracy level .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to lack of competency in rigorous math, I am unable to prove my hunch. Am I right? Can anyone furnish a proof to prove or disprove my hunch?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PS:&lt;/strong&gt; &#xA;My hunch is somewhat validated by a note in the textbook reference above ( section 9.7),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d5W5T.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d5W5T.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But still, I'll be interested to see a rigorous mathematical argument.&lt;/p&gt;&#xA;" OwnerUserId="690" LastEditorUserId="690" LastEditDate="2016-08-19T09:22:46.777" LastActivityDate="2016-08-19T09:22:46.777" Title="Will penalty-augmented stiffness matrix cause numerical issues in eigenvalue analysis?" Tags="|finite-element|eigenvalues|stiffness|penalty-method|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24766" PostTypeId="1" AcceptedAnswerId="24769" CreationDate="2016-08-19T14:51:52.477" Score="2" ViewCount="302" Body="&lt;p&gt;Is there any sufficient and necessary condition to check the feasibility of the linear constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax=b, x\geq 0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;without solving an LP with a constant objective function? $x$ is the variable and $ A\in\mathbb{R}^{m\times n}$ is given and has full row rank. $b$ is also given. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Farka's lemma could be an option. But it is not what I'm looking for since if has the flavor of &quot;if a solution exists for a set of equalities and inequalities&quot; but does not provide a direct way to determine whether the solution exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any more direct method to do the feasibility checking, such as by merely looking at the rank of augmented matrix and coefficient matrix similar to what Rouché–Capelli theorem does?&lt;/p&gt;&#xA;" OwnerUserId="21397" LastEditorUserId="21397" LastEditDate="2016-08-19T15:06:24.497" LastActivityDate="2016-08-19T20:58:25.783" Title="linear programming feasiblity checking" Tags="|linear-algebra|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24767" PostTypeId="1" CreationDate="2016-08-19T17:23:27.087" Score="0" ViewCount="129" Body="&lt;p&gt;In physical problems the infinite force is easily implemented by the mathematical form of that force, take $1/r$ as an example, whenever the distance between two particles becomes small the force goes to infinity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, I want to simulate a system using agent-based modeling which the force between two objects becomes infinity without any mathematical form, my question is: Can the type of force below be implemented in a computer simulation? If so, how is it done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{ij} = \hat{r}_{ij}\begin{cases}&#xA;-\infty &amp;amp;\text{if } r_{ij} &amp;lt; r_\text{hc}\\&#xA;\frac{1}{4}\frac{r_{ij} - r_e}{r_a - r_\text{hc}} &amp;amp;\text{if } r_\text{hc}&amp;lt;r_{ij}&amp;lt;r_a\\&#xA;1 &amp;amp;\text{if } r_a&amp;lt; r_{ij}&amp;lt;r_c&#xA; \end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation is from a paper which has the &lt;a href=&quot;https://arxiv.org/pdf/cond-mat/0401257.pdf&quot; rel=&quot;nofollow&quot;&gt;arxiv version&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18395" LastEditorUserId="9667" LastEditDate="2016-09-26T17:07:09.207" LastActivityDate="2016-09-26T17:07:09.207" Title="Infinite force in simulation" Tags="|simulation|" AnswerCount="1" CommentCount="5" ClosedDate="2016-09-21T20:31:33.880" ContentLicense="CC BY-SA 3.0" />
  <row Id="24769" PostTypeId="2" ParentId="24766" CreationDate="2016-08-19T20:58:25.783" Score="4" Body="&lt;p&gt;The short answer to your question is: kind of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are methods called &quot;preprocessing&quot; or &quot;presolve&quot;, that will take a problem with constraints $b_{l} \leq Ax \leq b_{u}$ and $l \leq x \leq u$ as input. Using tests that are usually linear time, you can check sufficient conditions for infeasibility (but not necessary conditions). These methods are commonly used to transform a problem to reduce the runtime of standard algorithms for linear programming. You should look at &lt;a href=&quot;http://www2.isye.gatech.edu/~ms79/software/ojoc6.pdf&quot; rel=&quot;nofollow&quot;&gt;a 1994 paper by Savelsbergh&lt;/a&gt;, &lt;a href=&quot;http://www.turma-aguia.com/davi/doc/Andersen.pdf&quot; rel=&quot;nofollow&quot;&gt;a paper by Andersen and Andersen&lt;/a&gt;, and &lt;a href=&quot;http://www.mcs.anl.gov/papers/P1752.pdf&quot; rel=&quot;nofollow&quot;&gt;a survey paper by Ashutosh Mahajan&lt;/a&gt;. These papers all cover more general transformations that are also of interest; the Mahajan paper probably has the most modern and readable notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the general case, as Arnold Neumaier points out in the SciComp link from ChristianClason, linear program feasibility checking is polynomial-time reducible to linear programming and vice versa.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2016-08-19T20:58:25.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24770" PostTypeId="1" CreationDate="2016-08-19T23:25:14.350" Score="1" ViewCount="62" Body="&lt;p&gt;$A$ is square and positive definite, and let $r_k = Ax_k - b$. Also let $M = \frac{1}{2}(A+A^T)$. I want to show that&#xA;$$\frac{||r_{k+1}||_2}{||r_k||_2} \le \left(1-\frac{\lambda_\min(M)^2}{\lambda_\max(A^TA)} \right)^{1/2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We define $x_{k+1} = x_k + \alpha_k r_k$, where $\alpha_k$ is chosen to minimize $||r_{k+1}||_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I found $\alpha_k$, to be specific I think that&#xA;$$\alpha_k = -\frac{r_k^TA^TAx_k}{r_k^TA^T Ar_k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An earlier part of the problem has us prove that&#xA;$$\frac{v^TAv}{v^Tv}\ge \lambda_\min(M) &amp;gt; 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having trouble using those two facts to get the desired result. I tried expanding $||r_{k+1}||_2^2 = r_{k+1}^Tr_{k+1}$ by using its definition and the definition of $\alpha$, but I didn't get really anywhere. Any ideas? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7322" LastEditorUserId="20688" LastEditDate="2018-04-08T02:06:59.810" LastActivityDate="2018-04-08T02:06:59.810" Title="bound error for iterative method for solving linear system" Tags="|linear-algebra|linear-solver|iterative-method|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24771" PostTypeId="2" ParentId="24764" CreationDate="2016-08-20T02:35:58.240" Score="1" Body="&lt;p&gt;The spatial indices are $n$ and $k$ (see figure 3 in the paper). $i$ (called $j$ in the paper) seems to be the temporal index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With these definitions, the term about which you inquire seems to be a temporal averaging, not a spatial one.&lt;/p&gt;&#xA;" OwnerUserId="3987" LastActivityDate="2016-08-20T02:35:58.240" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24773" PostTypeId="2" ParentId="24767" CreationDate="2016-08-20T07:59:59.837" Score="1" Body="&lt;p&gt;In the same way you would do to implement a comparison with zero.&#xA;For instance, in numerical solvers, one sometimes needs to check whether a value reaches zero. That is not feasible in a formal way since a real cannot be compared with zero due to the limitation of machine precision. A way to do that is to choose a precision $eps$ and compute :&#xA;$$&#xA;\vert u \vert &amp;lt; eps&#xA;$$&#xA;If you want to check whether $u$ &quot;equals&quot; 2, you do :&#xA;$$&#xA;\vert u-2 \vert &amp;lt; eps&#xA;$$ &#xA;Generally you choose $eps$ to be near the machine precision limit, like $10^{-15}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to &quot;compute&quot; infinity, there is not a formal way to do that too since infinity is not a number so I would say to do the opposite, that is to say to choose a very big number, I often use $10^{20}$, that is what we do for instance in penalization method where we want a term overwhelming the others. If $F$ is the force you want to be infinite, just replace its value with a very big number.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-08-20T07:59:59.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24774" PostTypeId="1" CreationDate="2016-08-20T17:19:12.593" Score="0" ViewCount="41" Body="&lt;p&gt;Edit: figure it out, made an error&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are given that&#xA;$$y' = f(y,t), y(t_0) = y_0$$&#xA;We want to use the following method&#xA;$$&#xA;\begin{align}&#xA;u^1_{n+1/2} &amp;amp;= u_n + \frac{h}{2}f(u_n, t_n)\\&#xA;u^1_{n+1} &amp;amp;= u_{n+1/2} + \frac{h}{2}f(u^1_{n+1/2}, t_n+h/2)\\&#xA;u^2_{n+1} &amp;amp;= u_n + hf(u_n, t_n)\\&#xA;u_{n+1} &amp;amp;= \alpha u^1_{n+1} + \beta u^2_{n+1}&#xA;\end{align}&#xA;$$&#xA;The goal is to choose $\alpha$ and $\beta$ such that the above method is second order. I used Taylor expansions for everything and assumed that $f$ is Lipschitz. I can add more details, but assuming I did it correctly I found that the local truncation error $\tau$ is given by&#xA;$$\tau = (1-\alpha-\beta)u_n + (1-\alpha-\beta)hu_n' +(1-\alpha)\frac{h^2}{2}u_n'' + O(h^3)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: expression for $\tau$ should have&lt;/strong&gt;&#xA;$$(1-\alpha/2)\frac{h^2}{2}u_n''$$&#xA;&lt;strong&gt;Expression above was incorrect.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if we let $\alpha=1, \beta=0$, then we get $\tau= O(h^3)$, thus giving second order convergence. I then implemented this method for two different functions $f$ and only got 1st order convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas? Is those the correct values of $\alpha$ and $\beta$? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7322" LastEditorUserId="7322" LastEditDate="2016-08-20T17:30:07.067" LastActivityDate="2016-09-19T18:21:29.363" Title="choosing parameters for extrapolation method to give second order error" Tags="|numerical-analysis|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24775" PostTypeId="2" ParentId="24774" CreationDate="2016-08-20T17:31:07.687" Score="1" Body="&lt;p&gt;Once I found my mistake (see edit in post), that led to different values of the constants, namely&#xA;$$\alpha = 2, \beta = -1$$&#xA;The posted method then reduces down to the midpoint method, which is second order.&lt;/p&gt;&#xA;" OwnerUserId="7322" LastActivityDate="2016-08-20T17:31:07.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24776" PostTypeId="1" CreationDate="2016-08-21T06:42:08.347" Score="7" ViewCount="1123" Body="&lt;p&gt;I'm wondering specifically in regard to a recursive function such as massive a game tree. I can't specifically say how big yet, but definitely pushing the limits of a given processor or processor array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is it correct to say that passing a variable to a function requires an operation. Certainly there must the the flipping of some bits. Does this reduce efficiency?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="21410" LastEditorUserId="21410" LastEditDate="2016-08-26T08:22:27.337" LastActivityDate="2016-08-26T08:22:27.337" Title="Is there any computational efficiency to global variables?" Tags="|efficiency|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24777" PostTypeId="2" ParentId="24776" CreationDate="2016-08-21T11:29:28.460" Score="1" Body="&lt;p&gt;This is more of a Stack Exchange or Programmer Exchange question, see :&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/2867363/c-performance-of-global-variables&quot;&gt;here for C language&lt;/a&gt; or &lt;a href=&quot;https://stackoverflow.com/questions/2216340/the-advantage-disadvantage-between-global-variables-and-function-parameters-in&quot;&gt;PHP&lt;/a&gt;, &lt;a href=&quot;https://stackoverflow.com/questions/7507110/global-variables-vs-passing-a-value-into-a-function&quot;&gt;Javascript&lt;/a&gt; or &lt;a href=&quot;https://stackoverflow.com/questions/3363714/cost-of-passing-variables-to-functions#3363738&quot;&gt;C++&lt;/a&gt;, and&#xA;&lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/148108/why-is-global-state-so-evil&quot;&gt;more generally&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try to avoid global variables. It's better for maintaining your code. Global variables should be parameter, i.e their value should not change during execution. About efficiency, I think it is a micro-optimization that is not very relevant. I'm not an expert but I think that working with function arguments rather than global variables allows more optimizations from the compiler (something with the stack and the registers...). Passing a variable to a function only requires some pointer references in the stack (assuming your global variable is not a huge object), that is very fast and I am not even sure a profiler can detect such a thing.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-08-21T11:29:28.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24778" PostTypeId="2" ParentId="24733" CreationDate="2016-08-21T13:57:17.473" Score="6" Body="&lt;p&gt;I think you're mixing up ideas. The best thing here is to know the foundations, and it then &quot;implicit FEM&quot; will be a trivial idea (which is why there won't be a book specifically about that).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite Element Methods are a form of spatial discretization where you use some kind of structured or unstructured mesh to re-write the problem as a system of equations using something like a Galarkan approximation. The common way is to take a basis of triangles and use that to get an equation at each node. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Time-Independent&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If the problem was time-independent and linear, this will result in a linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Au = b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is the vector of the values at each node of the triangle. You solve this linear system with any method for linear system, like direct solvers or indirect solvers. If this was a time-dependent problem that was nonlinear, then you have a system of implicit equations at each node like: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Au = f(u)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or more generally (if you let $F(u)= Au -f(u)$ in the above example),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(u)=0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that you have to solve for some function $f$ (or $F$) for some vector $u$ which is the value at the nodes of the triangles. This you have to solve with a root finder method like Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Time-Dependent&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you had started with a time-dependent problem, that at every node of the triangle you get an ordinary differential equation, i.e. instead of the above you have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_t = f(u) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_t = \frac{du}{dt}$ is useful shorthand. This is known as a system of ordinary differential equations because how everything changes in time can be inter-related. At this point, you have to make another choice: how do you discretize in time? Those give you either explicit timestepping methods, or implicit timestepping methods. They each have their pros/cons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explicit timestepping methods are like Euler's method. For example, that would be doing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_{n+1} = u_{n} + \Delta t f(u_{n}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to go from time $t_n = t_0 + n\Delta t$ to $t_{n+1}$. This is simply saying &quot;change $u$ by how much $F(u)$ is now&quot;, almost a direct translation of what the ODE says. This is the simplest approximation and can work in certain cases. However, if your problem is &quot;stiff&quot; (that's tough to define succinctly, I will just say &quot;different parts of the problem move at different speeds&quot;), this will fail to be &quot;stable&quot;. Therefore you will use an implicit timestepping method. For example, this could be the Implicit Euler method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_{n+1} = u_n + \Delta t f(u_{n+1}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is saying that the best guess for how $u$ changes in the time interval is to use $u_{n+1}$ (since the ODE itself doesn't have &quot;timesteps&quot; and takes place with infinitely small steps, they are essentially equivalent in the original formulation, but cause a difference in the approximation for non-infinitesimal $\Delta t$). In a reference on numerical ODEs, it will go into detail on how this will be &quot;more stable. However, this is an implicit equation since $u_{n+1}$ is inside the function $F$, and so we can't just isolate the $u_{n+1}$ on the left hand side of the equation and get a solution for it in terms of $u_n$. Indeed, this is another way of getting an implicit equation since if you let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(u_{n+1})=u_{n+1} - u_n - \Delta t f(u_{n+1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then you'll notice that at each timestep we have to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(u_{n+1}) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note that at time $n$ we already know $u_n$ so it's a constant, which is why the only variable is $u_{n+1}$).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Conclusion&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;As you can see, time dependent or time independent, the problem boils down to solving $F(x)=0$ which is an implicit equation. If you had a time-dependent problem, you have to solve this at each timestep, whereas time-independent problems solve this once. But as you can see, it can arise for very different reasons. However, because all of these problems (and many more) boil down to solving such a simple equation $F(x)=0$, there has been a lot of research into methods for solving this problem. These &quot;root finding&quot; methods (or more generally, this falls under optimization) include Newton, Broyden, trust region, etc. methods, are their own course of study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That should help you find the right reference. Either the knowledge you're missing is in FEM discretizations, methods for numerical ODEs, or in the root finding methods (&quot;Newton methods&quot;) used to solve the resulting equation.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-08-21T13:57:17.473" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24779" PostTypeId="2" ParentId="24776" CreationDate="2016-08-21T14:18:00.780" Score="6" Body="&lt;p&gt;First of all, note that this forum is for Computational Science, not Computer Science. These are different fields, with Computational Science being scientific computing, more like computational mathematics and scientific simulations. That said, even though the example in question is not relevant to this forum, there are things that should be discussed here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inefficiency is not in the passing of values to functions, but in many other places. This is because a global variable can change at anytime. The program can't know if the value will change, and more importantly in a scripting language, they can change types. Because the program doesn't know if the variable changed, the program has to be ready to deal with whatever has happened. Thus all compiler/interpreter optimizations are turned off and any code around the variables has to go into &quot;safe mode&quot;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;A good example is to look at Julia. &lt;a href=&quot;http://julialang.org/&quot;&gt;Julia is a recent programming language&lt;/a&gt; which, although it &quot;looks like a scripting language&quot;, is actually designed to be efficiently compiled at every step of the way. If you write your Julia code correctly (type-stable, etc.), then your code will compile to be the same speed as C/Fortran (and you can use &lt;code&gt;@code_llvm&lt;/code&gt; to see that the machine code is the same as what you'd get from a C/Fortran compiler). However, the language has problems with &lt;a href=&quot;http://docs.julialang.org/en/release-0.4/manual/performance-tips/&quot;&gt;global variables&lt;/a&gt;. A large discussion on why is &lt;a href=&quot;https://github.com/JuliaLang/julia/issues/8870&quot;&gt;found on Github&lt;/a&gt;. The summary is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a (just-in-time) compiled language like Julia, the compiler looks at what it currently has and uses a bunch of tricks to optimize the code. However, if it doesn't necessarily know what type the variable is, it has to carry around a bunch of code for making it switch between integers/floats/etc. If the variable is a global, it also has to deal with problems like if some other thread changed the value. This means that a lot of other compiler tricks like inlining are also not able to be done. Thus the resulting compiled code has a lot more steps to check every little detail since the global can do just about anything. This makes dealing with them really inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you crawl up the ladder to more dynamic typing, these inefficiencies are more baked into the language. At the bottom, all languages are still running compiled code, so something like MATLAB/R/Python always has this extra code around because their numbers can just change types all the time (Javascript does too, though it's odd since &quot;everything is a float&quot; so it has to have parts convert things to integers at the right times). However, smart interpreters (like the Javascript JITs) will sometimes try to still do some optimizations, but for the same reasons as above, they also cannot optimize too much when there are global variables since there's really no way to know in advance what a global variable will be (both in type and value). When all bets are off, they have to play it safe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(For an example in the other direction, if you pass a variable in and it looks like an integer, and all of the code looks like it won't modify it, it can JIT compile the function so that it actually is an integer, resulting in much leaner and more efficient machine code. Note that this requires that the compiler can guarantee this variable isn't changing anywhere else, otherwise it will cause breakage.)&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Conclusion&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This is why the general rule for efficiency is to not use global variables. There's no way for the interpreter/compiler to know what to do with them, so they have to take the safest and least efficient route. Also, since they can change anywhere at any time, they can cause a lot of bugs in programs, and so globals are generally frowned upon for design reasons. In the end, unless you really have to, DO NOT USE GLOBALS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;tldr; passing the variable into the function will cause  a lot less problems than a global variable.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2016-08-21T14:23:30.180" LastActivityDate="2016-08-21T14:23:30.180" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24780" PostTypeId="1" CreationDate="2016-08-21T15:52:05.387" Score="2" ViewCount="1729" Body="&lt;p&gt;I was trying to approximate 1st derivative of a function $\phi$ on a non uniform grids: basically my aim is to do this on a uniform grid on the same domain, so I can calculate it on the &quot;new&quot; grid. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have $x_k$ a non uniform grid, and $\xi_k$ a uniform grid, both in $[0,1]$ and with the same length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now what I have is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- $\phi_k=\phi(x_k)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- $\{x_k\}, \{\xi_k\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I want to know $\phi_x(x_k)$ with an High order approx.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tried to do:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) calculate an &quot;high order approximation&quot; using $\phi(x_k)$ and uniform grid formula (with central finite difference) --&gt; call it $\phi^u_{x_k}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) do $\phi_x=\phi^u_{x_k}*\frac{d\xi}{dx}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but no good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea?&lt;/p&gt;&#xA;" OwnerUserId="21085" LastActivityDate="2016-08-22T19:38:39.423" Title="Derivatives Approximation on non uniform grid" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24782" PostTypeId="2" ParentId="24752" CreationDate="2016-08-21T19:55:29.947" Score="2" Body="&lt;p&gt;I've taken a look on these equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know you have a 1D element, but rewriting the equation would eliminate confusion if you want to write elements in 2D and 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore your stiffness matrix should be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\mathbf{K}_{el} = \sum_{i}^{n_{gauss}} \mathbf{B}^T \mathbf{E} \mathbf{B} w_{\xi,i} det(J).&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the numerical integration with Gauss. Your stiffness matrix formula used $J$ instead of $det(J)$, which is not wrong. The determinant of the Jacobian is only significant for multidimensional cases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula for your Jacobian should be &#xA;\begin{equation}&#xA;J = \sum_{i}^{n} \frac{\partial N_{i}}{\partial \xi}x_{i}.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobian in the one-dimensional case is defined as $J = \left[ \frac{\partial x}{\partial \xi} \right]$ for&#xA;\begin{equation}&#xA;\begin{bmatrix}&#xA;\frac{\partial N}{\partial \xi}&#xA;\end{bmatrix}&#xA;=&#xA;J&#xA;\begin{bmatrix}&#xA;\frac{\partial N}{\partial x}&#xA;\end{bmatrix}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{bmatrix}&#xA;\frac{\partial N}{\partial \xi}&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial x}{\partial \xi}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\frac{\partial N}{\partial x}&#xA;\end{bmatrix}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobian in the isoparametric concept approximates the transformation by the derivatives of the shape function. Here $\frac{\partial x}{\partial \xi}$ has to be approximated. Like the x-coordinate in the element would be calculated by &#xA;\begin{equation}&#xA;x = &#xA;\begin{bmatrix}&#xA;N_1 &amp;amp; N_2 &amp;amp; N_3 &amp;amp; N_4&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\hat{x}_1 \\&#xA;\hat{x}_2 \\&#xA;\hat{x}_3 \\&#xA;\hat{x}_4 \\&#xA;\end{bmatrix}&#xA;\end{equation}&#xA;$\frac{\partial x}{\partial \xi}$ is approximated by&#xA;\begin{equation}&#xA;\frac{\partial x}{\partial \xi}&#xA;=&#xA;\begin{bmatrix}&#xA;\frac{\partial N_1}{\partial \xi} &amp;amp; \frac{\partial N_2}{\partial \xi} &amp;amp; \frac{\partial N_3}{\partial \xi} &amp;amp; \frac{\partial N_4}{\partial \xi}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\hat{x}_1 \\&#xA;\hat{x}_2 \\&#xA;\hat{x}_3 \\&#xA;\hat{x}_4 \\&#xA;\end{bmatrix}.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Important: The definition of the Jacobian in many books covering finite elements is not consistent to the mathematical definition. So be careful when you are doing your transformations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you are doing the transformation of the derivatives of the shape function, you should have&#xA;\begin{equation}&#xA;\begin{bmatrix}&#xA;\frac{\partial N_1}{\partial x_1} &amp;amp; \frac{\partial N_2}{\partial x_2} &amp;amp; \frac{\partial N_3}{\partial x_3} &amp;amp; \frac{\partial N_4}{\partial x_4}&#xA;\end{bmatrix}&#xA;= \mathbf{J}^{-1} \begin{bmatrix}&#xA;\frac{\partial N_1}{\partial \xi_1} &amp;amp; \frac{\partial N_2}{\partial \xi_2} &amp;amp; \frac{\partial N_3}{\partial \xi_3} &amp;amp; \frac{\partial N_4}{\partial \xi_4}&#xA;\end{bmatrix},&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that seems to be ok, except that you have the matrix transposed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your stiffness matrix then is wrong. Note that you have a dyadic product, where in this case you get a $4 \times 4$ matrix out of $1 \times 4$ vectors. That leads us to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\mathbf{K}_{el} = \sum_{i}^{n_{gauss}}&#xA;\begin{bmatrix}&#xA;\frac{\partial N_1}{\partial x_1} \\ &#xA;\frac{\partial N_2}{\partial x_2} \\ &#xA;\frac{\partial N_3}{\partial x_3} \\ &#xA;\frac{\partial N_4}{\partial x_4}&#xA;\end{bmatrix}&#xA;\mathbf{E}&#xA;\begin{bmatrix}&#xA;\frac{\partial N_1}{\partial x_1} &amp;amp; \frac{\partial N_2}{\partial x_2} &amp;amp; \frac{\partial N_3}{\partial x_3} &amp;amp; \frac{\partial N_4}{\partial x_4}&#xA;\end{bmatrix}&#xA;w_{\xi,i}&#xA;det(\mathbf{J})&#xA;\end{equation}&#xA;with $w_{\xi,i}$ being the weight for the gauss integration of the integration point $\xi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gauss quadrature for $\mathbf{K}$ is now really simple. Choose the Gauss points in the isoparametric space and the corresponding weights (just look on wikipedia) and use the formulas given.&lt;/p&gt;&#xA;" OwnerUserId="21418" LastEditorUserId="21418" LastEditDate="2016-08-25T18:29:17.223" LastActivityDate="2016-08-25T18:29:17.223" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24783" PostTypeId="2" ParentId="24721" CreationDate="2016-08-21T20:25:26.060" Score="0" Body="&lt;p&gt;There are multiple problems with the mesh on the right, if it comes to finite elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The elements are quite distorted, which will cause problems. Reason for that is that the derivatives calculated with the help of isoparametric elements will lose accuracy the more the elements are distorted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like it was mentioned before the quadrilateral elements which now have a flat corner will cause the determinant of the Jacobian to be zero in the element. The isoparametric transformation which requires the inverse of the Jacobian would fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's highly unlikely that a finite element calculation with this mesh would be successful. &lt;/p&gt;&#xA;" OwnerUserId="21418" LastActivityDate="2016-08-21T20:25:26.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24784" PostTypeId="2" ParentId="24776" CreationDate="2016-08-21T21:09:06.760" Score="1" Body="&lt;p&gt;From a &lt;em&gt;style&lt;/em&gt; point of view, it is usually best to avoid global variables (as already pointed out). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a performance point of view, the answer depends on your programming language. For example, if your language does not support passing a multidimensional array by reference, then a global/non-local array can help. But I guess that all modern languages used for computing don't have such shortcomings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether it costs more to call a function when it has more arguments is language and compiler-dependent. Often, the extra cost will be zero or negligible, so that a global variable is more likely to hurt than benefit you (as explained in the other answers).&lt;/p&gt;&#xA;" OwnerUserId="18662" LastActivityDate="2016-08-21T21:09:06.760" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24787" PostTypeId="1" AcceptedAnswerId="24788" CreationDate="2016-08-22T10:35:51.313" Score="1" ViewCount="117" Body="&lt;p&gt;How does one write general tensor contractions in the Python-based finite element package &lt;a href=&quot;http://www.nutils.org&quot; rel=&quot;nofollow&quot;&gt;Nutils&lt;/a&gt;? For example, how does one write the contraction of a fourth-order elasticity tensor $\boldsymbol{C}$ with a second-order strain tensor $\boldsymbol{\varepsilon}$, to yield the second-order stress tensor $\boldsymbol{\sigma}=\boldsymbol{C}:\boldsymbol{\varepsilon}$?  &lt;/p&gt;&#xA;" OwnerUserId="21423" LastEditorUserId="21423" LastEditDate="2016-08-22T12:29:19.243" LastActivityDate="2016-08-22T12:29:19.243" Title="Binary tensor operations in Nutils" Tags="|numpy|tensor|nutils|" AnswerCount="1" CommentCount="0" ClosedDate="2016-08-22T19:19:29.670" ContentLicense="CC BY-SA 3.0" />
  <row Id="24788" PostTypeId="2" ParentId="24787" CreationDate="2016-08-22T10:57:58.097" Score="1" Body="&lt;p&gt;For high dimensional tensor manipulations you should avoid using numpy.dot, which is very limited in scope, and instead use a combination of multiplication and summation. Although this would cost performance when used directly on Numpy arrays this is not the case in Nutils which uses a delayed execution model, and in fact uses &lt;code&gt;dot&lt;/code&gt; or &lt;code&gt;einsum&lt;/code&gt; under the hood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following I use &lt;code&gt;_ = numpy.newaxis&lt;/code&gt; as is customary in Nutils:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Inner product of two vectors&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For $\mathbf{a} \in \mathbb{R}^n$ and $\mathbf{b} \in \mathbb{R}^n, c = \mathbf{a} \cdot \mathbf{b} \in \mathbb{R}$ (Einstein notation: $c = a_i b_i$)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;c = numpy.dot(a,b) # numpy only&#xA;c = (a*b).sum(-1) # numpy/nutils&#xA;c = a['i']*b['i'] # nutils only&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Outer product of two vectors&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For $\mathbf{a} \in \mathbb{R}^n$ and $\mathbf{b} \in \mathbb{R}^m$,&#xA;$\mathbf{C} = \mathbf{a} \otimes \mathbf{b} \in \mathbb{R}^{n \times m}$ (Einstein notation:&#xA;$C_{ij} = a_i b_j$)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;C = a[:,_]*b[_,:] # numpy/nutils&#xA;C = a['i']*b['j'] # nutils only&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Matrix-vector product&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For $\mathbf{A} \in \mathbb{R}^{m \times n}$ and $\mathbf{b} \in \mathbb{R}^n$, $\mathbf{c} = \mathbf{A}\mathbf{b}  \in \mathbb{R}^{m}$ (Einstein notation: $c_{i} = A_{ij} b_j$)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;c = numpy.dot(A,b) # numpy only&#xA;c = A[:,:]*b[_,:]).sum(-1) # numpy/nutils&#xA;c = a['ij']*b['j'] # nutils only&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Double contraction of 4th and 2nd dimensional tensors&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For $\mathbf{A} \in \mathbb{R}^{q \times r \times s \times t}$ and $\mathbf{B} \in \mathbb{R}^{s \times t}$, $\mathbf{C} = \mathbf{A}:\mathbf{B}  \in \mathbb{R}^{q \times r}$ (Einstein notation: $C_{ij} = A_{ijkl} B_{kl}$)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;C = A[:,:,:,:]*B[_,_,:,:]).sum([-1,-2]) # numpy/nutils&#xA;C = A['ijkl']*B['kl'] # nutils only&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21422" LastEditorUserId="21422" LastEditDate="2016-08-22T12:04:08.823" LastActivityDate="2016-08-22T12:04:08.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24790" PostTypeId="1" AcceptedAnswerId="24791" CreationDate="2016-08-22T12:02:09.297" Score="0" ViewCount="235" Body="&lt;p&gt;Suppose I am completely new with the Python-based finite element package &lt;a href=&quot;http://www.nutils.org&quot; rel=&quot;nofollow&quot;&gt;Nutils&lt;/a&gt;, what example code would help me to get started?&lt;/p&gt;&#xA;" OwnerUserId="21423" LastEditorUserId="21422" LastEditDate="2016-08-22T12:28:32.227" LastActivityDate="2016-08-22T12:28:32.227" Title="Nutils 'Hello world!'" Tags="|finite-element|nutils|" AnswerCount="1" CommentCount="9" ClosedDate="2016-08-22T19:19:12.870" ContentLicense="CC BY-SA 3.0" />
  <row Id="24791" PostTypeId="2" ParentId="24790" CreationDate="2016-08-22T12:21:16.957" Score="2" Body="&lt;p&gt;The following fully functional script solves the Laplace problem for a scalar valued field $ u $, such that for all test functions $ v $:&#xA;$$ \int_\Omega ∇v·∇u = \int_{\Gamma_N} v f $$&#xA;where $\Omega$ is the unit square domain with a Neumann condition $ f = 1 $ on the right boundary $\Gamma_N$, a homogeneous Dirichlet condition $ u = 0 $ on the lower boundary, and natural boundary conditions on the remaining boundaries. The spaces are discretized using quadratic spline basis functions on an 8x8 computational grid.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from nutils import mesh, function, plot&#xA;&#xA;# prepare domain, geometry, basis&#xA;domain, geom = mesh.rectilinear( [range(9),range(9)] )&#xA;basis = domain.basis( 'spline', degree=2 )&#xA;&#xA;# construct matrix, right hand side, constraints&#xA;matrix = domain.integrate( basis['i,k'] * basis['j,k'], geometry=geom, ischeme='gauss2' )&#xA;rhs = domain.boundary['right'].integrate( basis, geometry=geom, ischeme='gauss2' )&#xA;con = domain.boundary['bottom'].project( 0, onto=basis, geometry=geom, ischeme='gauss2' )&#xA;&#xA;# solve system&#xA;lhs = matrix.solve( rhs, constrain=con )&#xA;sol = basis.dot( lhs )&#xA;&#xA;# plot solution&#xA;points, colors = domain.elem_eval( [geom, sol], ischeme='bezier9', separate=True )&#xA;with plot.PyPlot( 'solution' ) as plt:&#xA;  plt.mesh( points, colors )&#xA;  plt.colorbar()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this code generates the following graph of the solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xPmOw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xPmOw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Where to go from here?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A more elaborate getting started document can be found &lt;a href=&quot;https://joostvanzwieten.github.io/nutils-by-example/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Furthermore, the &lt;a href=&quot;https://github.com/nutils/nutils/tree/master/examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;examples&lt;/a&gt; included in Nutils provide fully functional example code for a wide range of problems.&lt;/p&gt;&#xA;" OwnerUserId="21422" LastEditorUserId="21422" LastEditDate="2016-08-22T12:26:36.470" LastActivityDate="2016-08-22T12:26:36.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24792" PostTypeId="5" CreationDate="2016-08-22T12:37:38.347" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://www.nutils.org&quot; rel=&quot;nofollow&quot;&gt;Nutils&lt;/a&gt; is an open source project aimed at the creation of a modern, general purpose Python programming library for finite element applications. The project is led by Evalf Computing and is being developed in close collaboration with academia, most notably the Technical University of Eindhoven. Nutils is under active development and is presently in use for both industrial applications and academic research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nutils is MIT licenced and is free to use in any application. It is important to understand that Nutils is not a ready to use program. It does not have a user interface and no predetermined functionality. What it is instead is a collection of high level objects that can be mixed, matched and extended to build any specific simulation in the form of a Python script. Knowledge of both computational physics and general programming are required to use Nutils to its potential.&lt;/p&gt;&#xA;" OwnerUserId="21422" LastEditorUserId="21422" LastEditDate="2016-08-22T18:36:47.640" LastActivityDate="2016-08-22T18:36:47.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24793" PostTypeId="4" CreationDate="2016-08-22T12:37:38.347" Score="0" Body="Nutils is an open source project aimed at the creation of a modern, general purpose Python programming library for finite element applications." OwnerUserId="21422" LastEditorUserId="21422" LastEditDate="2016-08-22T18:36:59.223" LastActivityDate="2016-08-22T18:36:59.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24794" PostTypeId="1" CreationDate="2016-08-22T12:47:38.803" Score="1" ViewCount="114" Body="&lt;p&gt;How can one construct non-uniform rational B-splines (NURBS) in &lt;a href=&quot;http://www.nutils.org&quot; rel=&quot;nofollow&quot;&gt;Nutils&lt;/a&gt;, and use them for NURBS-based isogeometric analysis?&lt;/p&gt;&#xA;" OwnerUserId="21423" LastEditorUserId="21423" LastEditDate="2016-08-22T13:42:18.553" LastActivityDate="2016-08-22T13:42:18.553" Title="NURBS-based isogeometric analysis using Nutils" Tags="|finite-element|b-spline|nutils|" AnswerCount="0" CommentCount="0" ClosedDate="2016-08-22T19:19:01.653" ContentLicense="CC BY-SA 3.0" />
  <row Id="24795" PostTypeId="2" ParentId="24780" CreationDate="2016-08-22T19:38:39.423" Score="7" Body="&lt;p&gt;So not quite sure what you're missing, but here's how you go about doing this sort of thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So first, I am assuming this is 1D due to your description. Second, I'm assuming you know the relationship between points in the physical domain, $x$, and the computational domain, $\xi$, something along the lines of $x=x(\xi)$. Given you have this relationship, you can do the following for some function $\phi(\cdot)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\frac{\partial \phi}{\partial x} &amp;amp;= \frac{\partial \phi}{\partial \xi} \frac{\partial \xi}{\partial x}\\&#xA;\frac{\partial \phi}{\partial x} &amp;amp;= \frac{\partial \phi}{\partial \xi} \left(\frac{\partial x}{\partial \xi}\right)^{-1}\\&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this representation, since you know $x(\xi)$, you can produce the second term exactly in the multiplication. So now you just need to approximate the first term, $\frac{\partial \phi}{\partial \xi}$. This term can be approximated using normal finite difference schemes. Using a central difference, for example, you could get the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \phi}{\partial x} = \left(\frac{\phi_{i+1}-\phi_{i-1}}{2\Delta \xi}\right) \left(\frac{\partial x}{\partial \xi}\right)^{-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, $\phi_{i}$ is associated to both $x_i$ and $\xi_i$, where $\phi_i = \phi(x_i)$, and where, in addition, $\phi_i = \phi(x(\xi_i))$. This should help you understand how to compute the necessary quantity you're after.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-08-22T19:38:39.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24797" PostTypeId="1" CreationDate="2016-08-23T05:37:46.980" Score="1" ViewCount="115" Body="&lt;p&gt;Let $\alpha=1.714$ and $\lambda=-v(1-v^{\alpha})/\alpha$, where $0&amp;lt;v&amp;lt;1$. Consider $\phi(v)=\exp\{\int_{v_{0}}^{v}\frac{1}{\lambda(t)} dt \}$, where $0&amp;lt; v_{0} &amp;lt;1$ is an arbitrary chosen constant. How to integrate the given integral numerically and plot $\phi(v)$ vs $v$ using Matlab?&lt;/p&gt;&#xA;" OwnerUserId="21434" LastActivityDate="2016-08-23T10:07:59.227" Title="Numerical Integration" Tags="|matlab|numerical-analysis|" AnswerCount="1" CommentCount="1" ClosedDate="2016-09-06T07:40:11.080" ContentLicense="CC BY-SA 3.0" />
  <row Id="24798" PostTypeId="2" ParentId="24017" CreationDate="2016-08-23T07:10:26.860" Score="-1" Body="&lt;p&gt;How noisy is 'noisy'? ... Is the signal above the noise or vice versa?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FFT -&gt; Average bins over a number of measurements ; threshold ; try narrower f-xlated FFT in the same way, rinse and repeat until you have the f as accurate as you want (limited by the noise power in the final BW of course).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HTH :-) &lt;/p&gt;&#xA;" OwnerUserId="21435" LastActivityDate="2016-08-23T07:10:26.860" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24801" PostTypeId="2" ParentId="24797" CreationDate="2016-08-23T10:07:59.227" Score="6" Body="&lt;p&gt;The &lt;code&gt;integral&lt;/code&gt; function in MATLAB performs numerical integration. See the MATLAB help, it's straightforward to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, an analytical solution of your integral exists. The calculations are a bit tricky but feasible. Let's rewrite your integral in your exponential (assuming $v=v(t)$) :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I = \alpha \int_{v_0}^v{\frac{dt}{v(v^{\alpha}-1)}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We perform a change of variable :&#xA;$$u=v^{\alpha}-1$$&#xA;$$du = \alpha v^{\alpha-1}dt$$&#xA;$$u_0 = v_0^{\alpha}-1$$&#xA;then $I$ becomes :&#xA;$$I = \alpha \int_{u_0}^u{\frac{du}{\alpha~v^{\alpha-1}~v~u}} = \int_{u_0}^u\frac{du}{v^{\alpha}~u} = \int_{u_0}^u\frac{du}{(u+1)u}$$&#xA;Much better to integrate right ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now an element decomposition gives you :&#xA;$$&#xA;I = \int_{u_0}^u\frac{du}{u} - \int_{u_0}^u\frac{du}{u+1}&#xA;$$&#xA;And the &lt;em&gt;final blow&lt;/em&gt;:&#xA;$$&#xA;I = [ln(u)]_{u_0}^u - [ln(u+1)]_{u_0}^u&#xA;$$&#xA;Now going back to $v$ :&#xA;$$&#xA;I = ln \left(\frac{v^\alpha-1}{v_0^\alpha-1}\left(\frac{v_0}{v}\right)^\alpha \right)&#xA;$$&#xA;So taking the exponential gives you the analytical solution of $\phi(v)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\phi(v) = \frac{v^\alpha-1}{v_0^\alpha-1}\left(\frac{v_0}{v}\right)^\alpha&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code below shows the comparison between your numerical integration and the analytical solution.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/qPXqC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qPXqC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc;&#xA;clear all&#xA;close all;&#xA;&#xA;v=[0.1:0.01:0.9]; % Vector v&#xA;v0=0.05; % Initial condition&#xA;a=1.714; % alpha&#xA;&#xA;fun = @(v) 1./(-v.*(1-v.^a)./a); % lambda&#xA;&#xA;for i=1:length(v)&#xA;    phi_num(i) = integral(fun,v0,v(i)); % Numerical integration&#xA;end&#xA;&#xA;phi_ex=((v0./v).^a) .* ((v.^a-1)./(v0.^a-1)); % Analytical solution&#xA;&#xA;plot(v,exp(phi_num),'b.')&#xA;hold on&#xA;plot(v,phi_ex,'r')&#xA;legend('Numerical','Analytical')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-08-23T10:07:59.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24802" PostTypeId="1" CreationDate="2016-08-23T15:13:09.397" Score="1" ViewCount="228" Body="&lt;p&gt;Consider the spherical advection problem: describing the conservation of a property $u$ in a closed spherical domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t}+\frac{1}{r^2}\frac{\partial }{\partial r}\left(r^2u\ \boldsymbol{v} \right)=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the domain $0&amp;lt;r&amp;lt;1,\ t\geq0$, with the following initial/boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left.\frac{\partial u}{\partial r}\right|_{r=a}=0,&#xA;\\&#xA;u(r,t=0)=u_i&#xA;$$&#xA;Where $\boldsymbol{v}=\boldsymbol{v}(r,t)$ and $u_i$ are known. ($\boldsymbol{v}\leq0$). Other posts (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/7267/can-the-advection-equation-with-variable-velocity-be-conservative&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7278/conservative-finite-difference-expression-for-the-advection-equation&quot;&gt;here&lt;/a&gt;) suggested discretizing the conservative form.&#xA;Forward explicit (downwind) discretization leads to:&#xA;$$&#xA;r\to i\Delta r,\ t\to n\Delta t,\ u(r,t)\to u_i^n&#xA;\\&#xA;\frac{\partial u}{\partial t}&#xA;\to&#xA;\frac{u^{n+1}_i-u^{n}_i}{\Delta t}&#xA;\\&#xA;\frac{\partial(r^2u\ \boldsymbol{v})}{\partial r}&#xA;\to&#xA;\frac{&#xA;((i+1)\Delta r)^2 u_{i+1}^n \boldsymbol{v}_{i+1}^n-&#xA;(i\Delta r)^2 u_{i}^n \boldsymbol{v}_{i}^n&#xA;}{\Delta r}&#xA;$$&#xA;And the equation becomes:&#xA;$$&#xA;\frac{u^{n+1}_i-u^{n}_i}{\Delta t}=&#xA;-\frac{1}{(i\Delta r)^2}&#xA;\frac{&#xA;((i+1)\Delta r)^2 u_{i+1}^n \boldsymbol{v}_{i+1}^n-&#xA;(i\Delta r)^2 u_{i}^n \boldsymbol{v}_{i}^n&#xA;}{\Delta r}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problems are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I cannot use this equation at $r=1$ (last node) because i have no boundary condition to fix the ghost node at $r=1+\Delta r$.&lt;/li&gt;&#xA;&lt;li&gt;I'm unsure how to write the equation at $r=0$. I can either use L'Hôpital to write &#xA;$$&#xA;\frac{\partial u}{\partial t}+&#xA;\lim_{r\to0}\left(\frac{1}{r^2}\frac{\partial }{\partial r}\left(r^2u\ \boldsymbol{v} \right)\right)&#xA;=&#xA;\frac{\partial u}{\partial t}+&#xA;\lim_{r\to0}\left(\frac{1}{2r}\frac{\partial^2 }{\partial r^2}\left(r^2u\ \boldsymbol{v} \right)\right)&#xA;=\\&#xA;\frac{\partial u}{\partial t}+&#xA;\frac{1}{2}\frac{\partial^3 }{\partial r^3}\left(r^2u\ \boldsymbol{v} \right)&#xA;=0&#xA;$$&#xA;And discretize the above or I can use the forward derivative and write&#xA;$$&#xA;\frac{u_1^{n}-u_0^{n}}{\Delta r}=0\therefore u_0^{n}=u_1^{n}&#xA;$$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What is the best way to discretize the problem and how do I address the first and last nodes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found another post (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/3043/boundary-conditions-for-the-advection-equation-discretized-by-a-finite-differenc?rq=1&quot;&gt;here&lt;/a&gt;) with almost the same problem. One of the answers suggested an interpolation for the stencil at the last node, which i tried with unsuccessful results. The solution returns values greater than $1$ at $r=1$ and does not converge at all as the nodes increase (unstable).&#xA;The image below shows the solutions for 21, 51 and 101 spatial nodes. The CFL number is below $1$. Solution should stabilize at $0.91$, and be less than $1$ for every instant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iErQW.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iErQW.gif&quot; alt=&quot;results for 21/51/101 nodes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21207" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-08-24T12:10:16.883" Title="Spherical Advection Discretization (boundary nodes)" Tags="|finite-difference|boundary-conditions|hyperbolic-pde|advection|conservation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24804" PostTypeId="1" CreationDate="2016-08-23T17:01:48.620" Score="1" ViewCount="85" Body="&lt;p&gt;I try to simulate thermal version of 1D $(x, t)$ sine-Gordon field model. I am interested in finding thermal static solution that minimizes functional of energy $E$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E = \int dx \left( \frac{1}{2} \phi' ^2 + 1 - \cos \phi \right)  \ ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\phi' = \partial_x \phi $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is very confusing that acceptance ratio of Metropolis algorithm is too high - more than $0.95$, so almost every new proposed configuration is accepted. On each Metropolis step I change field value at one spatial point and calculate difference in energy. To propose new configurations uniform sampling is used with step parameter $\delta = 0.5$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\phi_{new} = \phi_{old} + r \ ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $r$  is random number from $\phi_{old} - \delta$ to $\phi_{old} + \delta$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that if acceptance ratio is too high then algorithm does not work correctly. However, such algorithm is perfectly applied to the ground state harmonic oscillator (via path integral Monte Carlo).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double E_part(double dx, double phi, double phi_plus, double phi_minus) {&#xA;return dx * ( 0.5 * pow ( 0.5 * ( phi - phi_minus ), 2.0 ) + 0.5 * pow ( 0.5 * ( phi_plus - phi ), 2.0 ) + 1 - cos ( 0.5 * ( phi_minus + phi ) ) - cos ( 0.5 * (phi + phi_plus ) ) );&#xA;}&#xA;&#xA;double Metropolis(int N, int Steps, double dx, double Temperature, double* dphi, double* phi, double delta, double* EnergyArray) {&#xA;&#xA;&#xA;double Beta = 1.0 / Temperature;&#xA;&#xA;double Epart;&#xA;&#xA;double Enewpart;&#xA;&#xA;double phinew;&#xA;&#xA;double phi_plus, phi_minus;&#xA;&#xA;int AcceptanceCounter = 0;&#xA;&#xA;&#xA; //Initial energy&#xA;&#xA;&#xA;double r; //random number from 0 to 1;&#xA;&#xA;for (int k=0; k&amp;lt;Steps; k++) {&#xA;&#xA;    for (int j=0; j&amp;lt;N; j++) {&#xA;&#xA;&#xA;&#xA;        int pos = RandomUniformInt(1 , N-2); //random position to change;&#xA;&#xA;        phinew = phi[pos] - delta + RandomUniform() * 2 * delta;&#xA;&#xA;        phi_plus = phi[pos + 1];&#xA;        phi_minus = phi[pos - 1];&#xA;&#xA;        Epart = E_part(dx, phi[pos], phi_plus, phi_minus);&#xA;        Enewpart = E_part(dx, phinew, phi_plus, phi_minus);&#xA;&#xA;&#xA;        r = RandomUniform();&#xA;&#xA;        if ( Enewpart * Beta - Epart * Beta &amp;lt; 0.0 || exp( Epart * Beta - Enewpart * Beta ) &amp;gt;= r ) {&#xA;            phi[pos] = phinew;&#xA;            dphi[pos - 1] = 0.5 * ( phi[pos] - phi[pos - 1] );&#xA;            dphi[pos] = 0.5 * ( phi[pos + 1] - phi[pos] );&#xA;            AcceptanceCounter++;&#xA;&#xA;        }&#xA;    }&#xA;&#xA;   EnergyArray[k] = Energy(N, dx, dphi, phi);&#xA;&#xA;}&#xA;return ( AcceptanceCounter / static_cast&amp;lt;double&amp;gt;(Steps * N) );  //AcceptanceRatio is returned&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know what might be wrong. &lt;/p&gt;&#xA;" OwnerUserId="19192" LastActivityDate="2016-08-23T17:01:48.620" Title="Metropolis algorithm and thermal sine-Gordon model" Tags="|numerical-analysis|computational-physics|monte-carlo|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24806" PostTypeId="1" CreationDate="2016-08-23T23:59:14.950" Score="1" ViewCount="110" Body="&lt;p&gt;How do I compute the adjoint of MATLAB's &lt;code&gt;dwt3&lt;/code&gt; operator?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, how do I compute the adjoint of the linear operator that takes a 3D complex array &lt;code&gt;x&lt;/code&gt; as input and returns the quantity &lt;code&gt;dwt3(x,'db4')&lt;/code&gt; as output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This adjoint computation is an important step in many iterative reconstruction algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might guess that &lt;code&gt;idwt3&lt;/code&gt; is the adjoint of &lt;code&gt;dwt3&lt;/code&gt;, but the following code seems to show that this is not the case (unless my code contains an error):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sz = [384,300,144];&#xA;&#xA;% Randomly generate a 3D complex array&#xA;x1 = randn(sz) + 1i*(randn(sz));&#xA;&#xA;% Compute wavelet transform of x1&#xA;Wx1 = dwt3(x1,'db4');&#xA;&#xA;% Randomly generate x2 which has the same size and shape as Wx1 (which is a Matlab structure)&#xA;x2 = Wx1;&#xA;for i1 = 1:2&#xA;    for i2 = 1:2&#xA;        for i3 = 1:2&#xA;            arrSz = size(x2.dec{i1,i2,i3});&#xA;            x2.dec{i1,i2,i3} = randn(arrSz) + 1i*randn(arrSz);&#xA;        end&#xA;    end&#xA;end&#xA;&#xA;% Compute inverse wavelet transform of w2&#xA;WTransx2 = idwt3(x2);&#xA;&#xA;&#xA;% Check to see if the adjoint property &amp;lt;x1,WTransx2&amp;gt; = &amp;lt;Wx1,x2&amp;gt; is satisfied&#xA;prod1 = x1(:)'*conj(WTransx2(:));&#xA;prod2 = 0;&#xA;for i1 = 1:2&#xA;    for i2 = 1:2&#xA;        for i3 = 1:2&#xA;&#xA;            Wx1_arr = Wx1.dec{i1,i2,i3};&#xA;            x2_arr = x2.dec{i1,i2,i3};&#xA;            prod2 = prod2 + Wx1_arr(:)'*conj(x2_arr(:));&#xA;&#xA;        end&#xA;    end&#xA;end&#xA;&#xA;max(abs(prod1 - prod2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3274" LastEditorUserId="10134" LastEditDate="2016-08-24T08:49:39.117" LastActivityDate="2016-08-24T08:49:39.117" Title="Adjoint of the MATLAB $\tt dwt3$ (3D wavelet transform) operator" Tags="|matlab|wavelet|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24807" PostTypeId="1" AcceptedAnswerId="24815" CreationDate="2016-08-24T05:40:43.030" Score="7" ViewCount="335" Body="&lt;p&gt;For a typical Ridge Regression method for solving an inverse problem&#xA;$$&#xA;\min_x ||A~x - b||^2 + \lambda^2||\Gamma~x||^2&#xA;$$&#xA;Which has an analytical solution of&#xA;$$&#xA;\hat{x}_{est}=(A^TA+\lambda^2 \Gamma^T\Gamma)^{-1} A^T b&#xA;$$&#xA;How should $A, \lambda, x, b$ be 'normalized'? I have a model where $b_{data}$ is a discrete measurement (counts of particles). Ideally the sum of the forward projection of the estimate, $\sum A~\hat{x}_{est}$, would be equal to the sum of the observed data, $\sum A ~ \hat{x}_{est} = \sum b_{data}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I find that when as I increase the tikhonov parameter $\lambda$ from 0 to a non-zero number then the sum $\sum A ~ \hat{x}_{est}$ rapidly decreases. I need the &quot;absolute&quot; scale preserved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should my modelling matrix $A$ be normalized in some fashion? Should the data vector $b_{data}$? Please be as specific as possible.&lt;/p&gt;&#xA;" OwnerUserId="21260" LastActivityDate="2016-08-24T17:16:01.330" Title="Tikhonov (Ridge) Regression and Normalization" Tags="|linear-algebra|matrix|statistics|inverse-problem|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24808" PostTypeId="1" CreationDate="2016-08-24T08:11:11.243" Score="0" ViewCount="1478" Body="&lt;p&gt;I'm trying to use MATLAB to simulate an atom decay process by using Monte Carlo approach. The process is as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that atom 1 decay to atom 2, which, in turn, decay to stable atoms of type 3. The decay constants of 1 and 2 are $\lambda_1$ and $\lambda_2$, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that at $t = 0$, $N_1 = N_0$ and $N_2 = N_3 = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\lambda_1=0.0001$, $\lambda_2=0.00005$,&#xA;$t_{\rm MAX}$ for the process is $150000\rm s$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not really familiar with Monte Carlo method, so I have only managed to simulate the single decay process.When it come to consecutive decay I got completely lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attached my script for single decay below. Hope to get help here.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N0=input('Enter total number of nuclides: '); %Determine N0&#xA;lambda1 = 0.0001; %decays/s&#xA;dt = 100; %time step&#xA;M = 1500; % total number of time steps&#xA;t = 0:dt:(M-1)*dt; %time series&#xA;p = lambda1*dt; %probability of having a decay in the time dt&#xA;&#xA;% Define the array that will be populated with the number of atoms at each time step N = zeros(M,1);&#xA;&#xA;N(1) = N0;&#xA;&#xA;for j=2:M % Loop over time&#xA;    nd = 0; % number of decays&#xA;    for i=1:N(j-1)&#xA;        %generate a single random number  &#xA;        r = rand();&#xA;        %binary value =1 if the condition is True -&amp;gt; Atom has decayed&#xA;        rb = r &amp;lt; dt*lambda1; &#xA;        if rb == 1&#xA;           nd = nd+1;&#xA;        end    &#xA;    end&#xA;    N(j) = N(j-1)-nd; % number of atoms left at the step 2&#xA;end&#xA;&#xA;plot(t, N)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21455" LastEditorUserId="10134" LastEditDate="2016-08-24T09:29:50.423" LastActivityDate="2016-08-24T09:29:50.423" Title="Use Monte Carlo method to simulate consecutive decay in MATLAB" Tags="|matlab|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24809" PostTypeId="2" ParentId="24808" CreationDate="2016-08-24T08:53:15.940" Score="1" Body="&lt;p&gt;The decay of radioisotopes can be described by a set of coupled first order differential equations $$\frac{dN_{i}(t)}{dt} = -\lambda_{i}N_{i}(t) + \sum_{j \neq i}\lambda_{j\to i} N_{j}(t)$$ where $\lambda_{j\to i}$ represents the decay constant associated with the decay of isotope $j$ to isotope $i$, $\lambda_{i}$ is the total decay constant of isotope $i$ and $N_{i}(t)$ is the time-dependent concentration of isotope $i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be easily written in matrix form $$\frac{d\bar{N}(t)}{dt} = \bar{A}\bar{N}(t)$$ where $\bar{A}$ is typically very sparse. The numerical difficulties associated with this set of coupled differential equations are twofold: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The system can be large (in &lt;a href=&quot;http://www.hindawi.com/journals/stni/2012/545103/&quot; rel=&quot;nofollow&quot;&gt;our&lt;/a&gt; applications we track about 3500 isotopes)&lt;/li&gt;&#xA;&lt;li&gt;The system is very stiff due to the big differences between decay constants (some isotopes decay with half-lives in milliseconds, others in millions of years)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Luckily, there exist solvers than can handle these problems: solvers for stiff sets of ODEs like implicit Runge-Kutta methods (&lt;a href=&quot;http://www.unige.ch/~hairer/prog/stiff/radau.f&quot; rel=&quot;nofollow&quot;&gt;RADAU&lt;/a&gt;) or methods based on the matrix exponential method (&lt;a href=&quot;http://lib.tkk.fi/Diss/2013/isbn9789513880002/isbn9789513880002.pdf&quot; rel=&quot;nofollow&quot;&gt;CRAM&lt;/a&gt;). The application of RADAU to this problem is described in &lt;a href=&quot;http://www.hindawi.com/journals/stni/2012/545103/&quot; rel=&quot;nofollow&quot;&gt;Advanced Method for Calculations of Core Burn-Up, Activation of Structural Materials, and Spallation Products Accumulation in Accelerator-Driven Systems&lt;/a&gt; (note that I'm co-author of this paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see no reason nor advantage to use a Monte Carlo method to solve this problem. Do you intend to model the stochastic nature of radioactive decay? Why? &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2016-08-24T09:08:16.640" LastActivityDate="2016-08-24T09:08:16.640" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24811" PostTypeId="1" AcceptedAnswerId="24813" CreationDate="2016-08-24T11:21:17.207" Score="5" ViewCount="707" Body="&lt;p&gt;I am solving a numerical optimization problem with my own L-BFGS (implemented in c++). The problem has $\approx 10^6$ optimization parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the objective function gradient, I am taking a discrete Fourier transform by using FFTW3 library in double precision, all my other calculations/variables are also in double precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the problem is that with the same initial condition and all parameters, the algorithm sometimes converges with $\approx 2000$ iterations, sometimes with $\approx 3000$.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can this non-deterministic behavior be caused by accumulating floating point error (or FFTW error)? &lt;/li&gt;&#xA;&lt;li&gt;OR does it mean that my code has a bug and I am probably somewhere accessing unallocated memory (or some other bug which causes non-deterministic behavior)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19111" LastEditorUserId="10134" LastEditDate="2016-08-25T10:26:36.210" LastActivityDate="2016-08-25T22:17:36.700" Title="Can floating point error (in FFTW3) cause non-deterministic behavior?" Tags="|optimization|floating-point|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24812" PostTypeId="2" ParentId="24811" CreationDate="2016-08-24T13:35:26.233" Score="3" Body="&lt;p&gt;FFTW includes some adaptivity to select the fastest algorithms on your hardware. I'm no expert on the internals of FFTW, but it seems unlikely that this library causes the &lt;em&gt;large differences&lt;/em&gt; you observe, in particular because it is widely used. Therefore I would suggest:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Try to compile with as much run-time error checking as possible. For example, with &lt;code&gt;gfortran&lt;/code&gt; the flag &lt;code&gt;-fcheck=all&lt;/code&gt; can be very helpful when debugging. Unfortunately &lt;code&gt;g++&lt;/code&gt; does not seem to offer such checks, although &lt;code&gt;-Wall -Wextra&lt;/code&gt; could sometimes help. If available, try different compilers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run your program using a tool like &lt;code&gt;valgrind&lt;/code&gt;, perhaps with a reduced problem size, to find out about unallocated, uninitialized or unfreed memory.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You don't mention whether your code runs in parallel, if that is the case, then the parallel load balancing can be a source of non-deterministic behavior.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastEditorUserId="18662" LastEditDate="2016-08-24T14:25:01.360" LastActivityDate="2016-08-24T14:25:01.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24813" PostTypeId="2" ParentId="24811" CreationDate="2016-08-24T13:39:55.590" Score="8" Body="&lt;p&gt;&lt;strong&gt;Non-reproducible&lt;/strong&gt; behaviors in computing amidst different runs can involve several mechanisms, sometimes mixed. They can be  especially sensitive when one iterates calculations on large sets of data, like inverse 3D tomography.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Soft_error&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;soft errors&lt;/strong&gt;&lt;/a&gt;, caused by defects, interaction with high-energy particles in spacecrafts (let's rule them out for now)&lt;/li&gt;&#xA;&lt;li&gt;number representations, different systems and OSes and not sufficiently specified rules for floats like  rounding, overflow,&lt;/li&gt;&#xA;&lt;li&gt;code and compiler options: sometimes &lt;code&gt;x=a+b&lt;/code&gt; differs from &lt;code&gt;x=b+a&lt;/code&gt; (&lt;em&gt;non-commutativity&lt;/em&gt;),&lt;/li&gt;&#xA;&lt;li&gt;hardware management, especially related to forms of parallelization on heterogeneous collections of processors (CPU, GPU): sometimes, &lt;code&gt;x=a+b+c+d&lt;/code&gt; does not give the same results when computed (on two units) either as &lt;code&gt;x=(a+b)+(c+d)&lt;/code&gt; or &lt;code&gt;x=(a+c)+(b+d)&lt;/code&gt; (&lt;em&gt;non-associativity&lt;/em&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;non-deterministic&lt;/strong&gt; aspect might arise, mostly,  from the fourth type, as from one run  to the other, a different set, or a different grouping can be performed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT] Specifically to FFTW3, the &lt;a href=&quot;http://www.fftw.org/faq/section3.html&quot; rel=&quot;nofollow&quot;&gt;documentation&lt;/a&gt; says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Question 3.8. FFTW gives different results between runs&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If you use FFTW_MEASURE or FFTW_PATIENT mode, then the algorithm FFTW&#xA;  employs is not deterministic: it depends on runtime performance&#xA;  measurements. This will cause the results to vary slightly from run to&#xA;  run. However, the differences should be slight, on the order of the&#xA;  floating-point precision, and therefore should have no practical&#xA;  impact on most applications. If you use saved plans (wisdom) or&#xA;  FFTW_ESTIMATE mode, however, then the algorithm is deterministic and&#xA;  the results should be identical between runs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is for instance discussed in &lt;a href=&quot;http://www.hackerfactor.com/blog/index.php?/archives/452-Not-So-Fast.html&quot; rel=&quot;nofollow&quot;&gt;Not So Fast - The Hacker Factor Blog&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Some of the shortcuts are very fast but are non-deterministic&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This could be checked in you case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few quick sources:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repository.library.northeastern.edu/downloads/neu:2743&quot; rel=&quot;nofollow&quot;&gt;Numerical accuracy differences in CPU and GPGPU codes&lt;/a&gt;, D. Yablonski, 2011 &lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://netlib.org/scalapack/slug/node134.html&quot; rel=&quot;nofollow&quot;&gt;New Sources of Error in Parallel Numerical Computations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An important difference between ScaLAPACK and LAPACK is that a&#xA;  parallel computing environment, possibly consisting of a heterogeneous&#xA;  collection of processors, introduces new sources of possible errors&#xA;  not found in the serial environment in which LAPACK runs. These errors&#xA;  could indeed afflict any parallel algorithm that uses floating-point&#xA;  arithmetic.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://people.eecs.berkeley.edu/~hdnguyen/public/papers/ARITH21_Fast_Sum.pdf&quot; rel=&quot;nofollow&quot;&gt;Fast Reproducible Floating-Point Summation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Reproducibility, i.e. getting the bitwise identical floating point&#xA;  results from multiple runs of the same program, is a property that&#xA;  many users depend on either for debugging or correctness checking in&#xA;  many codes &lt;a href=&quot;https://en.wikipedia.org/wiki/Soft_error&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt;. However, the combination of dynamic scheduling of&#xA;  parallel computing resources, and floating point nonassociativity,&#xA;  make attaining reproducibility a challenge even for simple reduction&#xA;  operations like computing the sum of a vector of numbers in parallel&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://randomascii.wordpress.com/2013/07/16/floating-point-determinism/&quot; rel=&quot;nofollow&quot;&gt;Floating-Point Determinism: Is IEEE floating-point math deterministic? Yes and No!&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="19501" LastEditDate="2016-08-25T22:17:36.700" LastActivityDate="2016-08-25T22:17:36.700" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24814" PostTypeId="2" ParentId="24423" CreationDate="2016-08-24T13:57:01.317" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.fftw.org/faq/section3.html#nondeterministic&quot; rel=&quot;nofollow&quot;&gt;The mentioned FFTW library might run in a non-deterministic mode.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're using FFTW_MEASURE or FFTW_PATIENT mode, the programs checks at runtime, which parameter values work the quickest and then will use those parameters throughout the whole program. Because the run time will obviously fluctuate a bit, the parameters will be different and the result of the Fourier transforms will be non-deterministic. If you want deterministic FFTW, use the mode FFTW_ESTIMATE.&lt;/p&gt;&#xA;" OwnerUserId="19111" LastActivityDate="2016-08-24T13:57:01.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24815" PostTypeId="2" ParentId="24807" CreationDate="2016-08-24T17:16:01.330" Score="6" Body="&lt;p&gt;It's a question of how you choose $\lambda$ and $\Gamma$ (of course). Think for a moment about what happens if you choose $\Gamma=I$ and make $\lambda$ large: in that case you say that it is more important to you to minimize the regularization term $\|Ix\|^2=\|x\|^2$ than to minimize the misfit $\|Ax-b\|^2$. Obviously, making the term $\|x\|^2$ small means making $x$ small, and so choosing $\lambda$ large is equivalent to saying: it is more important to me that the vector $x$ be small, than the vector $x$ fitting the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you can also choose a different matrix $\Gamma$. If, for example, $\Gamma$ was a matrix that computes the differences between successive vector elements, i.e.,&#xA;$$&#xA;  \Gamma = \begin{pmatrix} 1 &amp;amp; -1 &amp;amp; 0 &amp;amp; \cdots \\&#xA;                            0 &amp;amp; 1 &amp;amp; -1 &amp;amp; \cdots \\&#xA;                            \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots \end{pmatrix}&#xA;$$&#xA;then even if you make $\lambda$ large, what you are saying now is that you care about &lt;em&gt;successive elements of $x$ being similar&lt;/em&gt;, but you are not trying to minimize the &lt;em&gt;size of $x$&lt;/em&gt; any more. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if you want to &lt;em&gt;preserve&lt;/em&gt; certain features, then they should be in the null-space of $\Gamma$, whereas the features you want to &lt;em&gt;suppress&lt;/em&gt; should not be in the null space.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-24T17:16:01.330" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24817" PostTypeId="1" AcceptedAnswerId="24820" CreationDate="2016-08-24T20:40:05.957" Score="3" ViewCount="3544" Body="&lt;p&gt;I have applied &lt;a href=&quot;https://www.scipy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciPy&lt;/a&gt;'s implementation of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cuthill%E2%80%93McKee_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cuthill-McKee&lt;/a&gt; algorithm to a $48 \times 48$ sparse &lt;strong&gt;non&lt;/strong&gt;-symmetric matrix in Compressed Sparse Row (CSR) format and the output is an array of length $48$ which the documentation calls a &quot;permutation array&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I use this &quot;permutation&quot; array? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the array is in some way representing how my matrix is to be reordered, but a permutation array of length $48$ is only enough to reorder one axis. What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="21459" LastEditorUserId="20417" LastEditDate="2018-03-15T16:41:01.910" LastActivityDate="2021-08-29T01:38:57.607" Title="Applying the result of Cuthill-McKee in SciPy" Tags="|algorithms|python|sparse-matrix|matrix|scipy|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24818" PostTypeId="2" ParentId="18611" CreationDate="2016-08-24T20:52:08.510" Score="1" Body="&lt;p&gt;This question has been open for a long time, but I think it still deserves to be answered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fundamental problem with the use of Krylov-space solvers on individual blocks as inner preconditioners is that they are not linear operators. To understand this, let's denote by $\tilde x = K(A,P,\tau,N; b)$ the vector you get as a solution by running a Krylov space method $K$ on the linear system $Ax=b$ for at most $N$ iterations or until a tolerance $\tau$ is reached, using a preconditioner $P\approx A^{-1}$. In other words, you can think of $K$ as an operator that acts on $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now note that $K(A,P,0,\infty;\cdot)$ is a linear operator: it would require solving $Ax=b$ exactly, i.e., $K(A,P,0,\infty;b)=A^{-1}b$, which is linear in $b$. In many cases, running a Krylov space method for exactly one iteration starting from a zero vector is also a linear operator applied to $b$. But because the sequence of Krylov vectors depends on the starting residual $r^{(0)}=b-Ax^{(0)}$, the operator $K(A,P,\tau,N; \cdot)$ is in general not a linear operator for finite $N$ and $\tau$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What this means is that if you use $K(A,P,\tau,N; \cdot)$ as part of a preconditioner for a linear system in which $A$ is one block, then you end up with a preconditioner that does not act as a linear operator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is in contrast to many other methods that are used to precondition: for example, one SSOR step is a linear operation on the vector to which you apply it, as are all other methods that apply one step of a fixed point iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fundamental problem now is that most Krylov space methods do require that the preconditioner is a linear operator. They will simply not converge if the preconditioner is not linear, explaining your observation. On the other hand, there are variations of some Krylov space methods -- typically prefixed by the word &quot;Flexible&quot;, such as F-GMRES in &quot;Flexible GMRES&quot; -- that work around this and that can deal with preconditioners that are not linear operators. These flexible variants of the original methods will still converge, and are often powerful methods when coupled with good (but nonlinear) preconditioners.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-24T20:52:08.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24819" PostTypeId="1" AcceptedAnswerId="24821" CreationDate="2016-08-24T21:05:42.583" Score="4" ViewCount="227" Body="&lt;p&gt;Consider the following program:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{\pmb a} \sum_i z_i\\&#xA;u.c. \pmb a \pmb P_i\pmb a^\top\geq z_i$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\pmb a \in\mathbb{R}^p$ and the $\pmb P_i$ are all symmetric positive semidefinite matrix (the eigenvalues of $\pmb P_i$ are all $\geq 0$ for all $i$) and $z_i$ is a scalar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if this is indeed a valid semidefinite programming problem (basically I think it can be recast as the standard form but I want to make sure I am not mistaken before I expand more effort into this angle of attack).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Edit:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Actually, to simplify the question I had left some constraints out. The actual problem is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{\pmb a} \sum_i^I z_i - \sum_j^J w_j\\&#xA;\pmb a \pmb P_i\pmb a^\top\geq z_i, i=1,\ldots,I\\&#xA; \quad\pmb a \pmb Q_j\pmb a^\top-w_j= 0, j=1,\ldots,J\\&#xA; \quad w_j\geq 0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where again, the $\pmb Q_j$ are all symmetric non negative definite. I reckon Prof. Borchers's answer below can easily be adapted to the complete program. The added constraints help make the program bounded.&lt;/p&gt;&#xA;" OwnerUserId="21463" LastEditorUserId="20417" LastEditDate="2018-05-12T17:03:06.600" LastActivityDate="2018-05-12T17:03:06.600" Title="Maximization variant of semidefinite programming (SDP)" Tags="|optimization|semidefinite-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24820" PostTypeId="2" ParentId="24817" CreationDate="2016-08-24T21:32:00.980" Score="7" Body="&lt;p&gt;The reverse Cuthill-McKee algorithm produces a reordering that applies to both the rows and columns.  This is because it works by considering matrices as graphs of (undirected) connected nodes.  According to the function's &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csgraph.reverse_cuthill_mckee.html#scipy.sparse.csgraph.reverse_cuthill_mckee&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt; in SciPy, the output array is the permuted row/column indices, so you can simply do the following&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perm = reverse_cuthill_mckee(arr_orig)&#xA;arr_perm = arr_orig[perm, perm]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and numpy's fancy indexing functionality will take care of it.  Of course, this involves making a copy of your array, and I assume that's acceptable.&lt;/p&gt;&#xA;" OwnerUserId="9601" LastActivityDate="2016-08-24T21:32:00.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24821" PostTypeId="2" ParentId="24819" CreationDate="2016-08-24T22:42:28.977" Score="7" Body="&lt;p&gt;You can obtain a semidefinite relaxation of your problem that will provide a bound on the optimal value of your problem, but it will not be an exact semidefinite formulation of your problem.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start with the constraints &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$aP_{i}a^{T} \geq z_{i}$, $i=1, 2, \ldots, n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use the cyclic permutation property of the trace of a product to write this as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(P_{i}a^{T}a) \geq z_{i}$, $i=1, 2, \ldots, n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=a^{T}a$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;then your constraints are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \mbox{tr}(P_{i}A) \geq z_{i}$, $i=1, 2, \ldots, n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;together with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=a^{T}a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then relax the constraint $A=a^{T}a$ to $A \succeq 0$, to get a relaxation of your original problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max_{A,z} \sum_{i} z_{i}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \mbox{tr}(P_{i}A) \geq z_{i}$, $i=1, 2, \ldots, n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A \succeq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an SDP, and its optimal value provides an upper bound on the optimal value of your original formulation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraint $A=a^{T}a$ is a non-convex rank-one constraint that cannot be formulated exactly in SDP.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-08-25T01:32:53.850" LastActivityDate="2016-08-25T01:32:53.850" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24822" PostTypeId="2" ParentId="20474" CreationDate="2016-08-24T22:52:35.883" Score="1" Body="&lt;p&gt;This problem can be solved by the Hungarian algorithm (or any other algorithm for the assignment problem).  Simply make two copies of each element of A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $a$ is connected to $b$, add an edge from each of the two copies of $a$ to  $b$.  Now look for the maximum-weight matching in this bipartite graph.  The result will be the solution to your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This finds the exact global optimum solution to your problem, efficiently.  You don't have to settle for an approximation or local optimum; this finds the best possible solution.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2016-08-24T22:52:35.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24823" PostTypeId="2" ParentId="21935" CreationDate="2016-08-24T23:00:01.280" Score="1" Body="&lt;p&gt;This is an instance of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Maximum_satisfiability_problem&quot; rel=&quot;nofollow&quot;&gt;Weighted Partial MAX-SAT problem&lt;/a&gt;.  You could take a look at MAX-SAT solvers; many of them will support this kind of query.  For instance, the &lt;a href=&quot;http://www.maxsat.udl.cat/&quot; rel=&quot;nofollow&quot;&gt;annual MAX-SAT competition&lt;/a&gt; has Weighted Partial MAX-SAT as one of the categories.  See &lt;a href=&quot;http://www.maxsat.udl.cat/16/results/index.html&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for the results from the 2016 competition.  &lt;a href=&quot;http://www.maxhs.org/&quot; rel=&quot;nofollow&quot;&gt;MaxHS&lt;/a&gt; looks quite good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, formulate this as an integer linear programming problem and then solve it with any off-the-shelf ILP solver: e.g., CPLEX, Gurobi, CBC, SCIP.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2016-08-24T23:00:01.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24824" PostTypeId="2" ParentId="8321" CreationDate="2016-08-24T23:05:42.263" Score="1" Body="&lt;p&gt;As Brian said, &quot;local MC methods&quot; consist in proposing local changes in a configuration in order to minimize some objective functional (usually, the energy of the configuation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Global (or &quot;cluster&quot;) methods are intended to perform the same minimization procedure by making changes over huge subdomains of the configuration space at the same time. This method, called Swendsen-Wang algorithm (&lt;em&gt;PRL, &lt;strong&gt;58&lt;/strong&gt; &lt;strong&gt;(2)&lt;/strong&gt; 1987, p:86&lt;/em&gt;), is based on a transformation found by Fortuin and Kasteleyn (&lt;em&gt;Physica, &lt;strong&gt;57&lt;/strong&gt;, 1972, p:536&lt;/em&gt;) and proves to be extremely efficient in reducing what is called &quot;critical slowing down&quot;, a phenomena occuring in local algorithms due to the fact that the correlation time between configurations can be very large near the critical point (i.e. the point where a phase transition occurs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In their paper, Swendsen and Wang apply their algorithm in the study of the Ising model of statistical mechanics. This cluster method is extremely efficient, widely used, and has allowed numerical simulations of huge systems. One can easily find source codes in several languages. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar cluster method was developed by Ulli Wolff (&lt;em&gt;PRL, &lt;strong&gt;62 (4)&lt;/strong&gt;, 1989, p: 361&lt;/em&gt;).&lt;/p&gt;&#xA;" OwnerUserId="21341" LastActivityDate="2016-08-24T23:05:42.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24825" PostTypeId="2" ParentId="20056" CreationDate="2016-08-24T23:09:22.487" Score="1" Body="&lt;p&gt;This is known as the farthest-point clustering problem.  It is NP-hard, but there is a approximation algorithm: &lt;a href=&quot;https://en.wikipedia.org/wiki/Farthest-first_traversal&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Farthest-first_traversal&lt;/a&gt;.  The problem has normally been studied with the Euclidean distance metric, but you can apply the same greedy algorithm to the Hamming distance, too, to get an approximate (but not necessarily optimal) solution.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2016-08-24T23:09:22.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24826" PostTypeId="2" ParentId="24760" CreationDate="2016-08-24T23:15:33.273" Score="1" Body="&lt;p&gt;If you just want an approximation to the sum of their norms, you can choose a random sample of $m$ of the indices (i.e., a random set $S \subseteq \{1,2,\dots,N\}$ of size $m$) and compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(\textbf{x}) = \sum_{n \in S} \| x_n \|_2.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By a standard central limit theorem argument, $f(\textbf{x})$ will be a reasonable approximation to the sum of the norms; the larger that $m$ is, the better an approximation it will be.  The time to compute $\tilde{f}(\textbf{x})$ might be significantly less than the time to compute the sum of the norms, depending on how $\mathbf{x}$ is stored and depending on the value of $m$.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2016-08-24T23:15:33.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24828" PostTypeId="1" CreationDate="2016-08-25T00:45:43.727" Score="0" ViewCount="334" Body="&lt;p&gt;I am trying to generate initial velocity field which satisfies incompressible flows condition. To formulate this I am using Rogallo's procedure formulation which is given by &#xA;$$&#xA;\tilde{u}(k) = \frac{\alpha k k_2 + \beta k_1 k_3}{k \sqrt {k_1^2 + k_2^2}} \hat{k_1} + \frac{\beta k_2 k_3 - \alpha k_1 k}{k \sqrt {k_1^2 + k_2^2}}\hat{k_2} - \frac{\beta \sqrt {k_1^2 + k_2^2}}{k}\hat{k_3}&#xA;$$&#xA;where $\alpha$ and $\beta$ are given by &#xA;$$&#xA;\alpha = \sqrt{\frac{E_0(k)}{4\pi k^2}} e^{i \theta_1} \cos \phi,\hspace{0.5cm} \beta = \sqrt{\frac{E_0(k)}{4\pi k^2}} e^{i \theta_1} \sin \phi &#xA;$$&#xA;in which $E_0(k)$ is the assumed energy spectrum in my case its &#xA;$$&#xA;E_0(k) = A k^4 e^{-0.14 k^2}, \hspace{1cm} k \in [k_a,k_b]&#xA;$$&#xA;My doubt is what are the terms $k , k_1,k_2,k_3$ in this formulation?&#xA;Secondly, after substituting these terms and obtaining the velocities in fourier space what is the procedure to enforce conjugate symmetry so that real velocities are obtained in the physical space for simulation?&lt;/p&gt;&#xA;" OwnerUserId="20363" LastActivityDate="2017-02-22T03:31:54.027" Title="Generating initial velocity field" Tags="|fourier-transform|fftw|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24829" PostTypeId="1" AcceptedAnswerId="24880" CreationDate="2016-08-25T09:02:15.443" Score="4" ViewCount="475" Body="&lt;p&gt;As my question states, I want to calculate the Fourier transform $F(q)$ of a radial function $f(r)$ (defined on $[0,\infty)$ and which decays like an exponential $\exp(-Ar+b)$ at large $r$) as accurately as possible in Fortran. The function values come from a data file (which I can easily interpolate through cubic interpolation for example and extrapolate since the behavior at large $r$ is known).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the &quot;physics&quot; definition of the Fourier transform in 3D, which gives (because $f$ is radial):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int d\mathbf{r}\,f(\mathbf{r})\exp(i\mathbf{q}\mathbf{r}) = 4\pi\int\limits_0^\infty r^2\frac{\sin(qr)}{qr}f(r)\,dr&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first tried to calculate this integral for some chosen values of $q$ by using Gauss-Legendre quadrature, by generating some 60 or 100 abscissas and weights via the NAG routine D01BCF (&lt;a href=&quot;http://www.nag.com/numeric/fl/manual/pdf/D01/d01bcf.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;D01BCF link)&lt;/a&gt;. In the case of Gauss Legendre quadrature, the problem is to choose the interval $[0,B]$ on which to integrate. While the function $f$ loses 4 to 5 orders of magnitude from $r=10$ to $r=20$ (example), the choice of $B$ as a strong influence on the result of the calculation... When I compared the result I get to a &quot;nearly exact&quot; calculation (made with MATLAB but with a very long computation time), I saw that in fact this was only valid for small values of $q$ (of the order of 5, when I have to deal with values as large as 150). A Gauss-Laguerre quadrature does not give any better result, probably because of the oscillatory part of the integrand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then tried to compute this Fourier transform for some given values of $q$ with the routine D01ASF &lt;a href=&quot;http://www.nag.com/numeric/fl/manual/pdf/D01/d01asf.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(D01ASF link)&lt;/a&gt;. It is a &quot;one-dimensional quadrature, adaptive, semi-infinite interval, weight function $\cos(ωx)$ or $\sin(ωx)$&quot;, which is exactly what I need. The results are quite convincing for $q$ up to 80 or 100 if I input absolute error tolerances of 10E-5. Problems are: I would need to go at larger $q$, and the Fourier transform $F(q)$ oscillates with a magnitude of ~10E-6 at such $q$'s. Lowering the tolerance to 10E-5 already takes some time and even makes the whole thing to output some error message from the subroutine so I don't know if 10E-6 would be feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thus currently wondering if trying to calculate this Fourier transform with FFT wouldn't be a good idea? The problems I face are that I don't know how to calculate radial wave functions with FFT (and also that I don't even know how to use FFT properly either since the definition of the transform is not even the same (exponent sign and argument) and that I never used it before). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you have ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: this comes from &lt;a href=&quot;https://stackoverflow.com/q/39127340/2320757&quot;&gt;https://stackoverflow.com/q/39127340/2320757&lt;/a&gt;, I've been advised to put this here on stackoverflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I tried by FFT (using the routine &lt;a href=&quot;http://www.nag.com/numeric/fl/manual/pdf/C06/c06faf.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;C06FAF&lt;/a&gt; from NAG library). It works quite well up to some large values of $q$. The problem I face is that there is always some constant normalising factor to account for. I don't get why. This normalising factor evolves with the number $N$ of points used in the mesh. It has the for of a power law: Normalising Factor $F = N^(-0.5) \exp(9.9)$ approximately (see figure &lt;a href=&quot;https://i.stack.imgur.com/inlap.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/inlap.jpg&quot; alt=&quot;figure&quot;&gt;&lt;/a&gt; where the black line is the &quot;exact&quot; Fourier Transform and the green, magenta, blue, red and yellow lines are the FFT calculated for different values of $N$).&lt;/p&gt;&#xA;" OwnerUserId="21472" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-09-08T15:58:04.203" Title="How to calculate efficiently and accurately the Fourier transform of a radial function in Fortran" Tags="|fortran|fourier-analysis|fourier-transform|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="24830" PostTypeId="2" ParentId="24661" CreationDate="2016-08-25T09:12:26.863" Score="1" Body="&lt;p&gt;You can use some scheme where value at a point is some type of average of its neighboring values. You will have to decide whether this kind of smoothing is appropriate in your case or not. In MATLAB, smooth3 function is used to smooth data in 3D. Using same principle, you can perform Gaussian smoothing or box smoothing in 1D. This is how smoothing is done. Reliability of smoothed results is what you will have to judge. &lt;/p&gt;&#xA;" OwnerUserId="3888" LastActivityDate="2016-08-25T09:12:26.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24832" PostTypeId="1" CreationDate="2016-08-25T13:27:42.497" Score="1" ViewCount="91" Body="&lt;p&gt;I am interested in working with systems of PDEs in two spatial dimensions, as part of a mathematical modeling project. The systems of PDEs are coupled, and there is no convenient reduction to 1 spatial dimension (due to the particular constraints of the modeling problem). It is my understanding that the built-in MATLAB solvers for PDEs are for problems in 1 spatial dimension. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest a package for MATLAB that can solve coupled PDEs numerically in multiple spatial dimensions? &lt;/p&gt;&#xA;" OwnerUserId="21474" LastEditorUserId="21474" LastEditDate="2016-08-25T15:35:11.603" LastActivityDate="2019-05-16T07:15:22.840" Title="Name of third-party Matlab packages for solving PDEs" Tags="|matlab|pde|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24833" PostTypeId="1" CreationDate="2016-08-25T15:51:58.413" Score="1" ViewCount="273" Body="&lt;p&gt;I need to numerically evaluate 2-D integrals of the form:&#xA;$$&#xA;\mathcal{I}(\theta) = \int_{0}^{1} \int_0^1 \varphi_\theta(x,y) dx dy&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\varphi_\theta$ is a family of smooth functions indexed by $\theta$; and I need to evaluate $\mathcal{I}(\theta)$ for a large number ($&amp;gt; 10^6$) of $\theta_i \in \Theta$, which are given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For how the problem is set up, I can evaluate $\varphi_\theta$ on a fine, equispaced 2-D grid (each evaluation involves a few multiplications), and I cannot evaluate $\varphi_\theta$ on other points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the grid is fine, I could simply compute $\mathcal{I}(\theta_i)$ via trapezoidal or Simpson's rule. However, this becomes extremely computationally expensive and unnecessary. For many (but not all) values of $\theta$, $\varphi_\theta(x,y)$ is $\approx 0$ for large part of the integration domain, and/or slowly-varying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The obvious thing here is to use some sort of adaptive integration method compatible with a fixed equispaced grid.&#xA;The basic idea I have is to start with a very coarse sub-grid, evaluate the integral and an estimate of the error in each sub-cell, and based on that decide which cells to &quot;zoom in&quot; and recompute with a refined grid. Iterate this a couple of times. There are lots of ways to do this naively, but I am interested in state-of-the-art solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What would be the best (fastest and precise) quadrature approach to compute the (sub)integrals? &lt;/li&gt;&#xA;&lt;li&gt;What would be the best method to pick the cells to refine?&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I use a measure of relative or absolute error, what's the best method to compute that?&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Any alternative idea?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the record, I am working with MATLAB but I plan to code this part in C via MEX files since I doubt that the adaptive bit can be efficiently vectorized, and I want it to be as fast as possible.&lt;/p&gt;&#xA;" OwnerUserId="19884" LastEditorUserId="19884" LastEditDate="2016-08-25T21:03:50.020" LastActivityDate="2016-12-23T23:46:28.493" Title="fast adaptive quadrature on equispaced 2-D grid" Tags="|matlab|c|quadrature|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24834" PostTypeId="2" ParentId="24833" CreationDate="2016-08-25T20:46:16.193" Score="1" Body="&lt;p&gt;I don't think it's quite clear to me why an adaptive scheme would be better. Integration with the trapezoidal rule is very cheap: It's in essence just one addition and one multiplication per grid point. Since you already evaluate your integrand $\varphi_\theta$ at each grid point, the summation to the integral would seem to me to be a completely negligible additional step: surely, evaluation of the integrand at these grid points must be far more expensive than the actual summation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may of course be that you have so many grid points that you need to store the values of the integrands at the grid points on disk, or in a large block of memory, and that that requires &lt;em&gt;loading&lt;/em&gt; data back into memory or from memory into the processor cache. The solution in that case is to do the summation in the same loop in which you evaluate the integrand in the first place.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-25T20:46:16.193" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24835" PostTypeId="2" ParentId="24828" CreationDate="2016-08-25T20:53:03.247" Score="1" Body="&lt;p&gt;The method produces a velocity field in Fourier space, where $k_1,k_2,k_3$ are the components of the wave vector $k \in {\mathbb R}^3$. In order to get a velocity field in physical space, you need to do an inverse Fourier transform. This typically done via the inverse Fast Fourier Transform (iFFT).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-25T20:53:03.247" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24836" PostTypeId="1" CreationDate="2016-08-26T08:00:23.323" Score="2" ViewCount="47" Body="&lt;p&gt;I need some kind of inequality constraint in my optimization problems (rude version of SVM for example or skeleton based mesh fitting). However hard constraints is not suitable for me because sometimes small constraint violations are allowed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think about $(max(0, f(x)))^2$ as soft constraint function where $f(x)$ is constraint. I.e. i want to keep $f(x) \le 0$. $f(x)$ is linear function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first glance this function is ok because it provide some kind of barrier for constraint and it can be expressed as sum of squares as Gauss-Newton solvers need.&#xA;But I'm not sure that Gauss-Newton can hold such kind of function. Because expression inside squares (max(0, f(x))) is not smooth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think about $(max(0, exp(f(x))))^2$ as &quot;barrier&quot;. This function is smooth (because $max(0, exp(f(x)))$ is smooth). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is ok to use $(max(0, exp(f(x))))^2$ or $(max(0, f(x)))^2$ within Gauss-Newton for smooth constaint? If not which kind of function i should use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS Sorry for my poor English and math skills.&lt;/p&gt;&#xA;" OwnerUserId="15233" LastActivityDate="2016-08-26T08:00:23.323" Title="$(max(0, f(x)))^2$ or $(max(0, exp(f(x))))^2$ for soft constraints with Gauss-Newton" Tags="|optimization|constrained-optimization|least-squares|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24837" PostTypeId="1" CreationDate="2016-08-26T10:10:48.313" Score="1" ViewCount="347" Body="&lt;p&gt;I am working with matrix operations in MATLAB, and I would have the following problem. I have matrix containing zero elements:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a=zeros(100,100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and another matrix with significantly smaller dimension containing value 6:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;b=6*ones(3,3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I define some coordinates in matrix &lt;code&gt;a&lt;/code&gt;: &lt;code&gt;a(45,50)&lt;/code&gt;. I also consider this point as center of matrix  &lt;code&gt;b(2,2)&lt;/code&gt;. I need to copy to matrix &lt;code&gt;a&lt;/code&gt; the elements from matrix &lt;code&gt;b&lt;/code&gt;, whose center  is defined by &lt;code&gt;b(2,2)&lt;/code&gt;. I just know that it can be done manually by defining coordinates in matrix &lt;code&gt;a&lt;/code&gt;. Please, is there any way, or inbuilt MATLAB function how to do it automatically? I am just MATLAB beginner.&lt;/p&gt;&#xA;" OwnerUserId="21383" LastEditorUserId="19501" LastEditDate="2016-08-28T01:40:04.107" LastActivityDate="2017-05-25T13:25:28.910" Title="Defining a pixel neighborhood in an array in MATLAB" Tags="|matlab|algorithms|matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24838" PostTypeId="2" ParentId="24837" CreationDate="2016-08-26T11:18:48.733" Score="0" Body="&lt;p&gt;You can easily do this by using the sparse function, which allows you to construct matrices by supplying a list of non-zero indices. Here is a short example. Note that I have formatted the local blocks to show the matrix structure, even though they are represented as vectors.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Define i indices in local coordinates&#xA;i = [-1,  0,  1,...&#xA;     -1   0   1,...&#xA;     -1   0   1];&#xA;% Define j indices in local coordinates&#xA;j = [ 1,  1,  1,...&#xA;      0   0   0,...&#xA;     -1  -1  -1];&#xA;% Construct 100 x 100 sparse matrix by using offsets&#xA;a = sparse(i + 45, j + 50, 6, 100, 100);&#xA;% Show subset of matrix&#xA;full(a(43:47, 48:52))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2016-08-26T11:18:48.733" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24839" PostTypeId="1" AcceptedAnswerId="24841" CreationDate="2016-08-26T13:04:28.037" Score="1" ViewCount="202" Body="&lt;p&gt;I was reading about Interpolation error formula and &lt;a href=&quot;https://en.wikipedia.org/wiki/Runge%27s_phenomenon&quot; rel=&quot;nofollow&quot;&gt;Runge's phenomenon&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : Is there any function that gives no error at all? In other words the interpolation function $p$ would equal the actual function $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My answer is : If we choose a function $f(x) = 1$ then we will get no error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my answer correct?&lt;/p&gt;&#xA;" OwnerUserId="21490" LastEditorUserId="9667" LastEditDate="2016-08-26T21:57:15.920" LastActivityDate="2016-08-29T12:22:02.437" Title="Is there any function gives no interpolation error in numerical analysis" Tags="|interpolation|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24840" PostTypeId="2" ParentId="24839" CreationDate="2016-08-26T13:47:11.377" Score="5" Body="&lt;p&gt;Any polynomial interpolation method will include a 0th-order term and thus interpolate a constant exactly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, some methods have the nice property that they can interpolate polynomials up to some degree $N$ exactly, where $N$ is also the max. interpolation degree. Lagrange polynomials are an example of this.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-08-26T13:47:11.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24841" PostTypeId="2" ParentId="24839" CreationDate="2016-08-26T13:48:02.230" Score="-3" Body="&lt;p&gt;I would think that the answer is a linear function f(x)=ax+b, not a constant function because the first order interpolation rule is Euler's method and this has no interpolation error. i.e f(x_1)-f(x_2)/(x_1-x_2) will always be easily determined but there will be an introduction of a rounding error for small values.&lt;/p&gt;&#xA;" OwnerUserId="21161" LastActivityDate="2016-08-26T13:48:02.230" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24842" PostTypeId="1" CreationDate="2016-08-26T14:12:56.997" Score="1" ViewCount="89" Body="&lt;p&gt;I am trying to solve ordinary differential equations with initial conditions using Laplace transform.&#xA;A simple test setup includes &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;an exponential discharge of RC circuit  and &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;an integrator from coil&#xA;and voltage source.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While the result of the first example is as expected, the second one is not.&#xA;What am I missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function ilap_test&#xA;&#xA;syms s RC vc vc0 il il0 L vin&#xA;&#xA;% vc_dot = -1/RC * vc      exponential discharge&#xA;% il_dot = 1/L * vin       current integrator&#xA;&#xA;% s*vc - vc0 = -1/RC * vc  % transform with initial conditions&#xA;% s*il - il0 = 1/L * vin&#xA;&#xA;vc = vc0/(s + 1/RC);&#xA;il = vin/L/s + il0/s;&#xA;&#xA;simplify(ilaplace(vc))&#xA;% vc0*exp(-t/RC)        (ok)&#xA;&#xA;simplify(ilaplace(il))&#xA;% il0 + vin/L           shouldn't this be il0 + vin/L*t ???&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18615" LastActivityDate="2016-12-24T17:02:25.240" Title="Solve ODE with initial values using Laplace transform" Tags="|ode|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24845" PostTypeId="2" ParentId="24842" CreationDate="2016-08-26T15:55:29.520" Score="1" Body="&lt;p&gt;Not all right hand sides are created equal.&#xA;Arranging by degree of derivative:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_dot      x             const&#xA;&#xA;vc_dot     + 1/RC*vc     + 0         = 0&#xA;il_dot                   - 1/L*vin   = 0 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Laplace transform:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s*vc - vc0     + 1/RC*vc                 = 0&#xA;s*il - il0                   -1/L*vin/s  = 0  % const div by s was missing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Solve:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vc = vc0/(s + 1/RC)&#xA;il = il0 + vin/L/s^2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Back-transform:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vc = vc0*exp(-t/RC)&#xA;il = il0 + t*vin/L&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18615" LastActivityDate="2016-08-26T15:55:29.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24846" PostTypeId="1" CreationDate="2016-08-26T19:50:06.233" Score="2" ViewCount="417" Body="&lt;p&gt;I have a quadratic eigenvalue problem of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(A_2 s^2 + A_1 s + A_0)\hat{v} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $s$ is the eigenvalue. The matrices $A_i$ contain derivatives up to order six, and I have six boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\hat{v}(0) = \hat{v}(1) = 0 \\&#xA;\hat{v}^{ii}(0) = \hat{v}^{ii}(1) = 0 \\&#xA;\hat{v}^{iv}(0) = \hat{v}^{iv}(1) = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the eigenvalue problem was of &quot;standard&quot; form $A\hat{v}=\lambda \hat{v}$ then I would simply modify the first and last rows of the matrix $A$ to impose the boundary conditions. But for my problem I don't know if I should impose the boundary conditions for all three matrices, or just for $A_0$ (which is not multiplied by $s$), in which case I would set the corresponding terms of $A_2$ and $A_1$ to zero. Actually I tried to use this last approach but it didn't work (I am calculating the eigenvalues using MATLAB's &lt;code&gt;polyeig&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, the differential equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left [\left(D^2 - K^2 \right)s^2 - 2 \left(D^2 - K^2\right)^2s + \left(D^2 - K^2 \right)^3 + Ra K^2 \right] \hat{v} = 0 &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D = \frac{\partial^2}{\partial y^2}$, $K$ is a constant (so numerically it is a diagonal matrix), and $Ra$ is a constant scalar. The differentiation operator $D$ is implemented using Chebyshev collocation, and the higher-order derivatives can be evaluated as powers of $D$.&lt;/p&gt;&#xA;" OwnerUserId="21493" LastEditorUserId="21493" LastEditDate="2016-08-28T14:01:06.867" LastActivityDate="2016-08-28T14:01:06.867" Title="Imposing boundary conditions for PDE quadratic eigenvalue problem" Tags="|pde|boundary-conditions|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24848" PostTypeId="2" ParentId="24846" CreationDate="2016-08-26T21:08:21.727" Score="3" Body="&lt;p&gt;I think you've already made a mistake in your thinking when you consider the standard EVP $A v = \lambda v$. Let's consider this problem and impose your very first BC, $v(0) = 0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first row of $A$ is then $\{1,0,...,0\}$ as you suggest, and let us not forget that $A v = \lambda v$ is really just $A v = \lambda B v$ in disguise, where $B = I$ the identity matrix. And so your first equation then reads $v(0) = \lambda v(0)$. This is not what you want! To properly impose your first BC you then need to modify $B = I$ such that the first row is a row of zeros, and you will have $v(0) = 0$.  This extends simply to your other Dirichlet condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I assume that your differential operators do not include eigenvalue $\lambda$, though sometimes they might, then this extends to the requirement that the appropriate rows in matrices $A_1$ and $A_2$ need to be modified to enforce them.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-08-26T21:08:21.727" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24849" PostTypeId="1" CreationDate="2016-08-26T22:28:54.147" Score="3" ViewCount="312" Body="&lt;p&gt;My FDM code simulates Backward Facing Step flow when I use conventional BCs such as defining velocity profile at inlet and fully developed condition at outlet. I have validated the results and it seems functioning correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I want to impose a pressure-pressure condition (both static), it only converges when a zero-gradient condition is selected for inlet. &#xA;    $$&#xA;    @inlet:  p=p_{in} \hspace{10 mm} ,\dfrac{du}{dx}=0 \hspace{10 mm}, \dfrac{dv}{dx}=0&#xA;$$&#xA;    $$&#xA;    @outlet: p=p_{out}  \hspace{10 mm},\dfrac{du}{dx}=0 \hspace{10 mm}, \dfrac{dv}{dx}=0&#xA;$$&#xA;When I try to use mass balance to update inlet normal velocity at each iteration, the solution eventually blows up. I am integrating outflow and use the mean velocity as a uniform inlet velocity.&#xA;    $$&#xA;    @inlet:  p=p_{in} \hspace{10 mm} ,u=U_{in} \hspace{10 mm}, v=0&#xA;$$&#xA;    $$&#xA;    @outlet: p=p_{out}  \hspace{10 mm},\dfrac{du}{dx}=0 \hspace{10 mm}, \dfrac{dv}{dx}=0&#xA;$$&#xA;where $U_{in}$ is found from last iteration mass balance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I am interested to know if there is any difference to use momentum balance and mass balance to update inlet normal velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone faced this problem before? I appreciate your help&lt;/p&gt;&#xA;" OwnerUserId="21174" LastEditorUserId="21174" LastEditDate="2016-08-30T02:38:09.487" LastActivityDate="2017-06-26T22:18:36.070" Title="Pressure-Pressure BC" Tags="|fluid-dynamics|boundary-conditions|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24850" PostTypeId="1" CreationDate="2016-08-25T19:38:18.623" Score="3" ViewCount="81" Body="&lt;p&gt;So I have a PDE that I use to describe how material flows through a volume(2D or 3D).&#xA;$$\frac{\partial C}{\partial t} + \vec{u} \cdot \nabla C = (D' + D )\nabla^2C$$&#xA;Now using finite differences I get to the following:&#xA;$$C^{i+1}_{j,k} =C^i_{j,k}+\Delta t \biggr[ (D+D')\biggr[\frac{C^i_{j+1,k}-2C^i_{j,k}+C^i_{j-1,k}}{(\Delta x)^2}+\frac{C^i_{j,k+1}-2C^i_{j,k}+C^i_{j,k-1}}{(\Delta y)^2}\biggr]-u_x \frac{C^i_{j,k} -C^i_{j-1,k}}{\Delta x} - u_y \frac{C^i_{j,k} -C^i_{j,k-1}}{\Delta y}\biggr]$$&#xA;where D is diffusion coefficient and superscript is temporal, subscript is spatial. (Just imagine a grid). We are given $u_x$ and $u_y$ through some already validated code. Now, we use a boundary condition where $C=0$ at the edges. I know this is a silly boundary condition but it's irrelevant because the problem I'm about to show you persists even if we use small times.(where the concentration doesn't reach the edge)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xOahq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xOahq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Look in the comments...It's telling me I can't post 2 links... Where do I have the second?]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is the code. It's relatively simple. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np &#xA;import matplotlib.pyplot as plt&#xA;import time&#xA;%matplotlib inline&#xA;from scipy import ndimage&#xA;import scipy.io as sio&#xA;a =sio.loadmat('C:\\Users\\Tsila elbag\\Documents\\IPythonNotes\\New folder\\Cavity10k_1.mat')&#xA;b = a['VelField']&#xA;N =50&#xA;M =300&#xA;ux = b[0][0][0]&#xA;uy = b[0][0][1]&#xA;#ux = np.zeros((50,50))&#xA;#uy = np.ones((50,50))&#xA;dx = 2/N&#xA;dy =2/N&#xA;D= .001&#xA;Dprime =0&#xA;dt = .0001&#xA;C = np.zeros((N,N))&#xA;#C[np.floor((.25+1)/dx):np.floor((.75+1)/dx),np.floor((.25+1)/dy)\&#xA;#  :np.floor((.75+1)/dy)]=1&#xA;C[np.floor((.25+1)/dy)\&#xA;  :np.floor((.75+1)/dy),.1/dx:np.floor((.25)/dx)]=1&#xA;f, axarr = plt.subplots(2, sharex=True)&#xA;axarr[0].contourf(C)&#xA;Chist= C&#xA;Ctemp = C&#xA;for T in range(M):&#xA;    for j in range(N):&#xA;        for k in range(N):&#xA;            if k==49  or j ==49 or k ==0 or j ==0:&#xA;                Ctemp[j,k]=0&#xA;            else:&#xA;&#xA;                Ctemp[j,k]= C[j,k]+dt*((D+Dprime)*(((C[j+1,k]-2*C[j,k]+\&#xA;                C[j-1,k])/(dx**2))+((C[j,k+1]-2*C[j,k]+C[j,k-1])/(dy**2)))-\&#xA;                ux[j,k]*(C[j,k]-C[j-1,k])/(dx)-uy[j,k]*(C[j,k]-C[j,k-1])/dy)&#xA;    #Chist.append(list(C))&#xA;    C = Ctemp    &#xA;axarr[0].set_title('Original') axarr[2].contourf(C)&#xA;axarr[2].set_title('{} seconds later'.format(dt*M))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So the question is why do I have these waves and if anyone knows effective boundary conditions I would be grateful. Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="21483" OwnerDisplayName="Aviv Moshe Albeg" LastActivityDate="2016-08-27T06:30:00.647" Title="Cavity Flow CFD Boundary conditions and strange waves" Tags="|fluid-dynamics|computational-physics|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="24851" PostTypeId="1" CreationDate="2016-08-27T15:12:43.417" Score="8" ViewCount="1179" Body="&lt;p&gt;I often see neural networks extended to complex-values. Those networks allow complex input, complex parameters, and complex output. My understanding is that the inner products and point nonlinearities are simply extended. I can see it's advantage for signals that are naturally complex (e.g., phase/amplitude decomposition, oscillations, frequency domain processing, ...). However, computationally speaking, is it any different from converting complex signals to real values?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;What is the computational advantage of using the complex field compared to the real field?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;What is the computational advantage of complex vs real vector space?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Akira Hirose. &lt;a href=&quot;http://amzn.to/2c4674S&quot; rel=&quot;nofollow noreferrer&quot;&gt;Complex-Valued Neural Networks&lt;/a&gt;. Springer Science &amp;amp; Business Media. 2006, 2012&lt;/li&gt;&#xA;&lt;li&gt;Danilo P. Mandic, Vanessa Su Lee Goh.&lt;a href=&quot;http://amzn.to/2bYwkhN&quot; rel=&quot;nofollow noreferrer&quot;&gt;Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models&lt;/a&gt;. 2009&lt;/li&gt;&#xA;&lt;li&gt;Tygert, M., Bruna, J., Chintala, S., LeCun, Y., Piantino, S., &amp;amp; Szlam, A. (2016). &lt;a href=&quot;https://direct.mit.edu/neco/article/28/5/815/8157/A-Mathematical-Motivation-for-Complex-Valued&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Mathematical Motivation for Complex-Valued Convolutional Networks&lt;/a&gt;. Neural Computation, 28(5), 815–825.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1128" LastEditorUserId="1128" LastEditDate="2021-05-02T14:00:13.060" LastActivityDate="2021-05-02T14:00:13.060" Title="Power of complex-valued neural network" Tags="|complex-arithmetic|neural-network|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="24853" PostTypeId="1" CreationDate="2016-08-27T19:38:24.600" Score="1" ViewCount="1578" Body="&lt;p&gt;Im currently implementing a Nelder-Mead algorithm in MATLAB that can deal with constraints. In detail, the constraints should be added to the objective function with a Lagrange Multiplier($\lambda$) and the new function is then minimized by Nelder-Mead. Looks something like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$L(x_1, \ldots,x_n,\lambda_1, \ldots,\lambda_n) = f(x_1,\ldots,x_n)+ \lambda\left(c-g(x_1,\ldots,x_n)\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this works for constraints that are equations, $g(x_1,\dots,x_n)= c$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How does it work with constraints that are upper and lower bounds? cant put them into the objective function like that&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are the multipliers to be treated as additional variables by the nelder mead or are they constants you need to set?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thank you guys.&lt;/p&gt;&#xA;" OwnerUserId="21500" LastEditorUserId="10134" LastEditDate="2016-08-28T12:26:45.863" LastActivityDate="2016-08-28T12:26:45.863" Title="Nelder-Mead algorithm with constraints" Tags="|matlab|constrained-optimization|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24854" PostTypeId="2" ParentId="24837" CreationDate="2016-08-27T20:03:39.810" Score="1" Body="&lt;p&gt;Just for the display, the following works as long as you are doing right: you do not try to copy &lt;code&gt;b&lt;/code&gt; values outside of &lt;code&gt;a&lt;/code&gt;, because you did not specify what should be done when you are at &lt;code&gt;a(100,100)&lt;/code&gt;. My choice is a circular shift, with a toroidal topology.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% a = zeros(100); aX = 45 ; aY = 50; &#xA;a = zeros(12); aCenter = [4 6];&#xA;b = 6 * ones(3); bCenter = ceil(size(b)/2);&#xA;&#xA;a(circshift(padarray(true(size(b)),size(a)-size(b),'post'),aCenter-bCenter))=b;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, this creates a 2D logical indexing of &lt;code&gt;b&lt;/code&gt; of the same size as &lt;code&gt;a&lt;/code&gt;, with ones in the top left:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;padarray(true(size(b)),size(a)-size(b),'post')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then &lt;code&gt;circshift&lt;/code&gt; moves this pattern by the vector &lt;code&gt;aCenter-bCenter&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;circshift(...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, the elements of &lt;code&gt;a&lt;/code&gt; with the above logical indexing are affected with the values in &lt;code&gt;b&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19501" LastEditorUserId="19501" LastEditDate="2016-08-28T09:53:48.810" LastActivityDate="2016-08-28T09:53:48.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24855" PostTypeId="2" ParentId="24851" CreationDate="2016-08-27T21:06:38.727" Score="0" Body="&lt;p&gt;From a strict programming point of view, a complex number is composed of a real part and an imaginary part, that is to say, two real values. So I guess that real arithmetic applies here, except for certain operations like &lt;code&gt;sqrt&lt;/code&gt; for a negative number for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A complex number needs twice the storage of a real, because as said before, it is actually two reals. So I assume that a complex-valued neural network will require twice the storage of a real-valued neural network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, since two reals are involved, it is possible that a complex-valued neural network will require twice the arithmetic operations compared to a real-valued neural network, but it may depend on the language, the compiler, some specialists in StackOverflow may give you more details.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-08-27T21:06:38.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24856" PostTypeId="2" ParentId="24839" CreationDate="2016-08-27T22:05:14.550" Score="2" Body="&lt;p&gt;Interpolation generally means estimating values of a function &quot;between&quot; known data. In higher dimensions the &quot;between&quot; concept is not as clear, but we are still concerned with finding an approximation to known data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is often easy to give expressions (such as a polynomial, rational function, or sum of sines and cosines) that fit known data exactly. Asking for an exact fit to &quot;in-between&quot; values amounts to assuming the underlying function belongs to the &quot;space&quot; that furnishes approximations.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2016-08-29T12:22:02.437" LastActivityDate="2016-08-29T12:22:02.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24857" PostTypeId="2" ParentId="24851" CreationDate="2016-08-28T01:31:31.793" Score="1" Body="&lt;p&gt;I don't really see how a complex valued Neural Network would provide anything particularly useful over a real valued Neural Network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole idea of having a Neural Network that operates on complex numbers, uses complex weights, and outputs complex numbers doesn't seem any different than having a real valued Neural Network that has two times as many inputs, outputs, and weights as the complex Neural Network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you want a network that takes some input complex number and outputs another complex number, you could represent this with a real valued Neural Network with two inputs and two outputs.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-08-28T01:31:31.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24858" PostTypeId="1" CreationDate="2016-08-28T15:45:37.410" Score="0" ViewCount="134" Body="&lt;p&gt;I have the following code to execute:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X$ = [0;0;0;0]; $sw$ = 0;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k = 1 : n&#xA;    if (sw(k) == 0 &amp;amp;&amp;amp; X(2,k)&amp;gt; 0.7 ) || (sw(k) == 1 &amp;amp;&amp;amp; X(1,k) &amp;gt;0)&#xA;        sw(k+1) = 1;&#xA;        X(:,k+1) = X(:,k) +h*(A1*X(:,k) + B1*u);&#xA;    else&#xA;        sw(k+1) =0;&#xA;        X(:,k+1) = X(:,k) +h*(A*X(:,k) + B*u);&#xA;    end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is fine. I wanted to solve the same ODE using the bulit-in ode suite. How do I check for $sw$ and $X$ after each iteration to see which one of the two equations to solve.&lt;/p&gt;&#xA;" OwnerUserId="21509" LastActivityDate="2016-08-28T17:11:10.110" Title="Solving ODEs of Switched Systems using MATLAB ODE suite" Tags="|matlab|ode|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24859" PostTypeId="2" ParentId="24858" CreationDate="2016-08-28T17:11:10.110" Score="1" Body="&lt;p&gt;You can use the &quot;event location&quot; feature of MATLAB's ODE suite to integrate the system of equations in one of the two states and stop the integration when the condition has been reached for a state change.  You can then restart the integration in the other state and setup an event trigger to stop the integration at the next state change.  See&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.mathworks.com/help/matlab/math/ode-event-location.html&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/help/matlab/math/ode-event-location.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-08-28T17:11:10.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24860" PostTypeId="1" CreationDate="2016-08-28T17:45:03.877" Score="10" ViewCount="338" Body="&lt;p&gt;Is there now a package for rigorous numerical integration that uses interval arithmetic and has access to a well-developed library of special functions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By &quot;well-developed&quot;, I mean something that, at the very least, includes the error function $\text{erf}$, the incomplete gamma function $\Gamma(s,x)$ and the like. In other words, I would like to integrate expressions involving these functions. VNODE-LP doesn't seem to satisfy this requirement, as it relies on older interval-arithmetic libraries lacking such basic special functions. (Besides, perhaps it is just me, but VNODE-LP actually seems harder to install than three years ago; I am not sure it is being adequately maintained.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. An ideal library would have both arbitrary-precision and double-precision modes, the latter one fast. That isn't really an issue for what I am doing now, but it sounds like a basic desideratum, just as, say, an intuitive interface written in C++ (as opposed to plain C) would be nice.&lt;/p&gt;&#xA;" OwnerUserId="21512" LastEditorUserId="20688" LastEditDate="2018-05-06T04:49:55.150" LastActivityDate="2018-05-06T04:49:55.150" Title="Numerical integration using interval arithmetic, nowadays" Tags="|c++|libraries|integration|special-functions|interval-arithmetic|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="24861" PostTypeId="1" CreationDate="2016-08-29T09:45:33.493" Score="1" ViewCount="147" Body="&lt;p&gt;I need to discretize the surface of prolate spheroid given by the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{x^2}{L^2} + \frac{y^2}{D^2} + \frac{z^2}{D^2} = \frac{1}{4}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The surface has to be divided to 500 equal panels to calculate the velocity potential on it. This means I will need to calculate the normal vector at each panel&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest any methods?&lt;/p&gt;&#xA;" OwnerUserId="21520" LastEditorUserId="21520" LastEditDate="2016-08-29T09:51:00.713" LastActivityDate="2016-08-29T16:45:55.500" Title="How to discretize the surface of a prolate spheroid?" Tags="|numerical-analysis|fluid-dynamics|numerical-modelling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24862" PostTypeId="1" CreationDate="2016-08-29T11:17:49.253" Score="4" ViewCount="263" Body="&lt;p&gt;I have the following setting:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An irregularly-shaped domain, expressed as a mesh of points&lt;/li&gt;&#xA;&lt;li&gt;A Laplacian operator, together with boundary conditions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am looking for the eigenvalues of that operator, i.e. functions (expressed on the mesh, though clearly approximating an underlying continuous function) satisfying the boundary conditions and&#xA;$$&#xA;\triangledown^2 f = \lambda f&#xA;$$&#xA;for some function $f$ and constant $\lambda$. I don't know $\lambda$ a priori, in fact part of the problem is to find the possible values!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I got so far for the special case, where the boundary conditions are a constant (e.g. 0s):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The Laplacian operator, when acting on a discrete mesh of points, becomes just a linear operation - e.g. a matrix transformation&lt;/li&gt;&#xA;&lt;li&gt;So I can just look for the eigenvalues of that&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That works pretty well, but doesn't generalise so well, I think. E.g. now I would like to solve the problem with boundary conditions such that on the boundary&#xA;$$&#xA;\frac{ \partial f }{ \partial n } = 0&#xA;$$&#xA;and now I  don't think I can just express that as a matrix eigenvector problem - or maybe I can?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For context, I am solving the problem of drum vibrations on an irregular-shaped domain. Given a domain, the solutions to the problem above give the shape of vibrations of the drum, along with the associated frequencies (which can be deduced from the eigenvalues). Here the boundary conditions is $f=0$ on boundary, and it solves nicely - I can produce a pretty picture like this, for example (here my domain is a semi-circle):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Gu1KQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gu1KQ.png&quot; alt=&quot;One eigensolution of the vibrating drum problem&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, now I would like to extend this to water waves in a cyllindroid container. The boundary condition becomes that of partial derivatives vanishing at the boundary described above - and I feel out of ideas. All help will be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="21522" LastActivityDate="2016-08-29T11:17:49.253" Title="Eigenvalues of a Laplacian operator on an irregular mesh" Tags="|pde|eigenvalues|eigensystem|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24865" PostTypeId="2" ParentId="24861" CreationDate="2016-08-29T16:45:55.500" Score="6" Body="&lt;p&gt;A spheroid is really just a sphere that has been squashed in the different coordinate directions. So to get a mesh for a spheroid is the same as getting a mesh for a sphere: start with the latter, then do the obvious and trivial transform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how do you get a mesh for a sphere? If you want quadrilaterals, you start with six quads that cover the surface of the sphere and that are arranged like the faces of a cube. Then refine these by recursively replacing each quad by its four children, where you pull the new points at the midpoints of the edges and the face out onto the sphere. This gives you 24 -&gt; 96 -&gt; 384 -&gt; 1536 quadrilaterals that cover the surface of the sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want triangles, then you can either start with the four sides of a tetrahedron, or the 8 sides of an octahedron, or the 20 sides of a icosahedron. You would then recursively refine each triangle into its four children in the same way as above, until you are satisfied that you have enough triangles.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-29T16:45:55.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24870" PostTypeId="1" CreationDate="2016-08-30T07:29:42.033" Score="3" ViewCount="153" Body="&lt;p&gt;In &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999115007135&quot; rel=&quot;nofollow&quot;&gt;this JCP paper&lt;/a&gt;, the authors simultaneously enforce the discrete maximum principles and element-wise mass balance for advection-diffusion equations through convex optimization. The least-squares finite element method is used, so the problem becomes a mixed formulation with a flux variable $\boldsymbol{q}\in\mathbb{R}^{n_qdofs}$ and a scalar concentration $\boldsymbol{c}\in\mathbb{R}^{n_cdofs}$. The constraint optimization problem in the discrete setting is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;\min_{q,c} \ &amp;amp;\tfrac12\langle c, K_{cc}c\rangle + \langle c, K_{cq}q\rangle + \tfrac12\langle q, K_{qq}q\rangle + \langle c,r_c \rangle + \langle q,r_q\rangle\tag{A}\\&#xA;\text{subject to: } &amp;amp;A_c c+ A_q q = b_f\tag{B}\\&#xA;&amp;amp;c_{\text{min}}\preceq c\preceq c_{\text{max}} \tag{C}&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where (B) is the equality constraints for element-wise mass balance and (C) is the bounded constraints to ensure discrete maximum principles. It is known that simply solving unconstrained version of (A) satisfies neither (B) nor (C), and the authors claim that this constrained optimization problem can satisfy both properties using MATLAB's quadprog function (interior-point-convex algorithm).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If only (B) is enforced, there will still be violations in maximum principles, but the solution will be locally conservative.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If only (C) is enforced, there will still be errors in mass balance, but the solution will satisfy the maximum principles.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If both (B) and (C) are enforced, it makes sense to me that maximum principles will be met, but will element-wise mass balance necessarily be met? Based on what I understand about quadratic programming problems, the constraint (B) is associated with a lagrange multiplier $\lambda$ and is added to the objective functional (A), so if I am applying bounds to the new objective functional (A)+$\lambda\cdot$(B), wouldn't I still have some violations in mass balance (B) because my resulting objective functional value will be non-zero?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14576" LastEditorUserId="1804" LastEditDate="2016-09-01T16:04:53.373" LastActivityDate="2016-09-01T16:04:53.373" Title="Enforcing bounds and equality constraints for convex optimization" Tags="|optimization|finite-element|convex-optimization|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24871" PostTypeId="2" ParentId="24849" CreationDate="2016-08-30T11:33:19.177" Score="1" Body="&lt;p&gt;A complication arises when you specify both the mass balance and the pressure differnce for an incompressible fluid because the pressure and velocity are coupled. A certain pressure difference can only have a particular set of velocities in the domain which satisfy the Navier Stokes equations. At the grass root level it can be explained by applying the Bernoulli's equation for a one dimensional problem at the INLET and the OUTLET&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;p_{in}+\dfrac{1}{2}\rho V_{in}^2 = p_{out}+\dfrac{1}{2}\rho V_{out}^2 &#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you were specifying the mass balance $V_{in}$ and pressure differnce $p_{out}-p_{in}$ you would be only left with $V_{out}$ in an incompressible fluid, which sounds reasonable. However the principle of mass conservation also has to be religioulsy followed which is why the solver fails. The mass balance equation is written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;A_{in}V_{in}=A_{out}V_{out}&#xA;\end{equation}&#xA;where $A_{in}$ and $A_{out}$ are the cross-sectional areas of the INLET and the OUTLET. It would be satisfied only if the INLET velocity $V_{in}$ prescribed by you and the OUTLET velocity $V_{out}$ obtained through the Bernouli equation matches with the ratios of the cross sectional areas as well. However this is rarely the case, and thus the equations you solve in the numerical solver, fail the check for conservation of mass. This causes the solver to explode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do note that this would not be the case if the solver was compressible. The density $\rho$ would adjust itself accordingly to satisfy the Navier Stokes equations and also the conservation of mass.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To asnwer your second question, if specifying the mass balance is the same as applying the momentum balance. Again it boils down to the compressibility of the fluid. If the fluid is compressible then the mass balance is not the same as applying the momentum balance. However if you are using an incompressible fluid, the momentum inlet would simply be $\rho $ times the velocity inlet.&lt;/p&gt;&#xA;" OwnerUserId="18870" LastActivityDate="2016-08-30T11:33:19.177" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24872" PostTypeId="2" ParentId="24870" CreationDate="2016-08-30T12:39:11.743" Score="2" Body="&lt;p&gt;This is just a standard convex Quadratic Programming (QP) problem, which quadprog, or any number of other QP solvers, can solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a variety of algorithms for solving QPs, and I think you need not concern yourself at this point with the algorithms. It appears that you are operating under a misunderstanding of the role and impact of Lagrange multipliers in solving the problem.  Upon solution of the QP by the solver, you should have a solution which minimizes the objective function subject to constraints (B) and (C) being satisfied. How the QP solver accomplishes that is &quot;its business&quot;. Don't be distracted by Lagrange multipliers, but yes, there will be Lagrange multipliers associated with each constraint at the solution. At the solution, it, along with the Lagrange multipliers, will satisfy the Karush-Kuhn-Tucker conditions, which are necessary and sufficient for an optimum for a convex QP.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2016-08-30T12:44:52.283" LastActivityDate="2016-08-30T12:44:52.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24873" PostTypeId="1" AcceptedAnswerId="24881" CreationDate="2016-08-30T13:57:50.243" Score="1" ViewCount="355" Body="&lt;p&gt;To get cutoff frequencies and eigenmode field distributions for a waveguide, one uses following equation: &#xA;$$1/\epsilon ∇ \times 1/\mu ∇ \times E = \omega^2 E$$&#xA;With $ \omega^2 $ as eigenvalues. This works fine. But what if I would like to calculate the wave number $k_z$ ?&#xA;If the waveguide is not changing in $z$ direction, then the E-Field can be described as: $E(x,y,z) = E(x,y)e^{jk_zz}$. Following that, the upper equation becomes:&#xA;$$1/\epsilon ∇ \times 1/\mu ∇ \times E(x,y)e^{jk_zz} = \omega^2 E(x,y)e^{jk_zz}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I need an eigenvalue equation whre $k_z$ is my eigenvalue. And in this case it is in the exponential, which is not good for me. &lt;/p&gt;&#xA;" OwnerUserId="20900" LastActivityDate="2018-07-06T14:54:48.177" Title="FEM or FD eigenvalue equation to get wave number instead of cutoff frequency" Tags="|eigenvalues|electromagnetism|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24874" PostTypeId="1" CreationDate="2016-08-30T14:16:01.323" Score="1" ViewCount="691" Body="&lt;p&gt;I am adapting a Java program from C++ and need a C++ function to perform the same task as the Java BOBYQAOptimizer() function. Can anyone recommend a C/C++ library with equivalent or similar functions and good documentation on how to use them? I've found one that might be suitable ( &lt;a href=&quot;http://dlib.net/optimization.html&quot; rel=&quot;nofollow&quot;&gt;http://dlib.net/optimization.html&lt;/a&gt; ), but it's not entirely clear if its optimizers can be used for this task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The program is intended to find the set of parameters that give the maximum statistical likelihood of a phylogenetic tree of HIV virus data across thousands of patients. The parameters are infection rates between demographic groups. (Different versions use either 4 or 8 groups, giving 16 or 64 parameters, though in practice some are held constant at 0.)&lt;/p&gt;&#xA;" OwnerUserId="21537" LastActivityDate="2016-08-30T14:16:01.323" Title="Looking for C++ function for performing optimization of parameters for multivariable function" Tags="|c++|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24875" PostTypeId="1" CreationDate="2016-08-30T14:41:43.477" Score="3" ViewCount="2070" Body="&lt;p&gt;Let $a,b,c,d\in\mathbb{R}^n$. Is it possible to compute&#xA;$$\langle a*b,c*d\rangle$$&#xA;faster than 6 FFTs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can do it with 6 FFTs by doing normal convolutions, 3 FFTs each.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my application I know $b, d$ upfront and can do preprocessing, so the cost is actually 4 FFTs. Can I do better than this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I believe that this might be possible is because FFTs are unitary, so maybe there's a way to omit the inverse FFTs in the convolutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT] where convolution is defined by:&#xA;$$(a*b)_i=\sum_{j=1}^i a_jb_{n+j-i}$$&lt;/p&gt;&#xA;" OwnerUserId="21536" LastEditorUserId="21536" LastEditDate="2016-08-30T15:54:03.487" LastActivityDate="2016-08-30T15:54:03.487" Title="Fast(er) computation of dot product of two convolutions?" Tags="|fourier-transform|computer-arithmetic|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24876" PostTypeId="1" CreationDate="2016-08-30T15:05:38.727" Score="1" ViewCount="1147" Body="&lt;p&gt;My problem is to compute eigenvectors and eigenvalues of a lot of small (n &amp;lt; 30) symetric, positive definite matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I am using LAPACK's DSYEV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The priority is speed more than accuracy. Is there a faster algorithm (maybe not asymptotically, but in practice for small input)?&lt;/p&gt;&#xA;" OwnerUserId="7977" LastEditorUserId="1804" LastEditDate="2016-08-30T19:09:32.163" LastActivityDate="2016-09-01T13:37:34.533" Title="Compute all eigenvectors and eigenvalues of small symmetric matrices" Tags="|linear-algebra|matrix|eigenvalues|lapack|dense-matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24877" PostTypeId="2" ParentId="24875" CreationDate="2016-08-30T15:43:12.490" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Parseval%27s_theorem&quot; rel=&quot;nofollow&quot;&gt;Parseval's theorem&lt;/a&gt; tells you that the dot product between two vectors equals the dot product of the Fourier transforms, possibly up to a constant. Consequently, you do not need to transform back the result of $a\ast b$ and $c \ast d$ and should be able to do the overall operation with just 4 FFTs.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-08-30T15:43:12.490" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24879" PostTypeId="2" ParentId="24876" CreationDate="2016-08-30T17:45:04.913" Score="4" Body="&lt;p&gt;In my experience the answer to this question is not clear-cut; it is dependent on the span of the eigenvalues and relative matrix structure itself. That said, your current approach evokes an implictly shifted QR solver that is essentially the standard for this exact type of problem. With some experimentation, however, you may squeak out a modest performance increase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If your problem is well conditioned, compute using single precision.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;DSYEVR is a LAPACK driver for real symmetric matrices that uses a MRRR algorithm to compute the eigenvalues first, then grabs the requested eigenvectors through an inverse-type problem. It is possible for your particular matrix that it could be quicker.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a parallel library. While your problem is small, I might still expect modest performance gains when using a relatively small number of cores. ScaLAPACK is always a safe bet here, though I would first recommend SLEPc. SLEPc operates through PETSc and serves as a wrapper for several EVP solvers, many in their parallel and thus, usually, scalable form. This will give you the opportunity to try several approaches on the fly and evaluate their relative efficiency for your problem. Iterative algorithms used in SLEPc can take an eigenvalue tolerance parameter that might also improve performance in your case. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note: I realize this answer isn't especially satisfying. If you provide some details on the nature of your matrix and its construction, someone might impart some kind of specialized knowledge on you.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastEditorDisplayName="user20857" LastEditDate="2016-08-30T18:04:49.893" LastActivityDate="2016-08-30T18:04:49.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24880" PostTypeId="2" ParentId="24829" CreationDate="2016-08-30T18:46:51.410" Score="2" Body="&lt;p&gt;I have tried the following code in Julia for $f(r) = exp(-r)$ with 1D FFT, which seems to be working somehow. So, could you compare your code with it and see if there is some difference of normalization? (Please note that &lt;a href=&quot;http://docs.julialang.org/en/latest/stdlib/math/#Base.fft&quot; rel=&quot;nofollow noreferrer&quot;&gt;fft()&lt;/a&gt; below makes no normalization, i.e., $1/sqrt(N)$ is not present, while the FFT routine in NAG library may be multiplying by some normalization factor.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function test( N, rmax )&#xA;&#xA;    dr = rmax / N&#xA;    r = linspace( 0.0, rmax - dr, N )       # r[1:N] = 0, dr, 2dr, ..., (N-1)dr&#xA;&#xA;    rexp = r .* exp( - r )                  # rexp[i] = r[i] * exp(-r[i]), i=1:N&#xA;    integral = - imag( fft( rexp ) ) * dr   # \int_0^rmax dr sin(qr) r exp(-r)&#xA;&#xA;    file = open( &quot;test.dat&quot;, &quot;w&quot; )&#xA;&#xA;    for m = 1 : N&#xA;&#xA;        if m &amp;lt; div( N, 2 )&#xA;            q = (2 * pi / rmax) * (m - 1)&#xA;        else&#xA;            q = -(2 * pi / rmax) * (N - m + 1)&#xA;        end&#xA;&#xA;        H = integral[ m ]                   # obtained from FFT&#xA;        H_exact = 2 * q / ( q^2 + 1 )^2     # analytical solution&#xA;&#xA;        if 0 &amp;lt;= q &amp;lt;= 200.0&#xA;            @printf( file, &quot;%20.10e %20.10e %20.10e\n&quot;,&#xA;                     q, H, H_exact )&#xA;        end&#xA;    end&#xA;&#xA;    close( file )&#xA;end&#xA;&#xA;test( 10^6, 50.0 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bRrBx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bRrBx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16497" LastEditorUserId="16497" LastEditDate="2016-08-31T01:12:38.393" LastActivityDate="2016-08-31T01:12:38.393" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24881" PostTypeId="2" ParentId="24873" CreationDate="2016-08-31T02:19:37.643" Score="2" Body="&lt;p&gt;Assuming that you have scalar permitivity and permeability, you can relate your wavenumber with your angular frequency by&#xA;$$\omega = kc$$&#xA;where $c = 1/\sqrt{\epsilon_0 \mu_0}$ is the speed of the wave in vacuum. Then your equation would read&#xA;$$1/\epsilon_r\nabla \times 1/\mu_r\nabla \times E_n(x,y) = \alpha_n^2 E_n(x,y)$$&#xA;where $\epsilon_r$ and $\mu_r$ are the relative permitivity and permeability, and the wavenumber for your propagating wave is given by&#xA;$$k_n^2 = k^2 - \alpha_n^2 \, .$$&#xA;Then, when $k^2 &amp;gt; \alpha^2$ you have propagating waves. And you need to find the eigenvalues for the transverse section.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-07-06T14:54:48.177" LastActivityDate="2018-07-06T14:54:48.177" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="24883" PostTypeId="1" AcceptedAnswerId="24885" CreationDate="2016-08-31T18:51:18.123" Score="7" ViewCount="669" Body="&lt;p&gt;I would like to solve the linear system $\mathbf{A}\mathbf{x}=\mathbf{b}$, with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{A}=\mathbf{T}+\mathbf{C}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{T}$ is a symmetric tridiagonal matrix and $\mathbf{C}$ is a corner-only matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{C}=\begin{pmatrix}&#xA;0&amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; c\\&#xA;0 &amp;amp; \ddots &amp;amp; &amp;amp; &amp;amp; 0\\&#xA;\vdots &amp;amp;  &amp;amp; \ddots &amp;amp; &amp;amp; \vdots\\&#xA;0 &amp;amp;  &amp;amp; &amp;amp; \ddots &amp;amp; 0\\&#xA;c &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0&amp;amp; 0&#xA;\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some efficient algorithms for solving this system of linear equations (i.e. solving for $\mathbf{x}$)?&lt;/p&gt;&#xA;" OwnerUserId="9562" LastActivityDate="2016-08-31T23:16:14.273" Title="Efficient algorithm for solving linear system with symmetric near-tridiagonal matrix?" Tags="|linear-algebra|sparse-matrix|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24884" PostTypeId="1" CreationDate="2016-08-31T20:43:15.107" Score="1" ViewCount="436" Body="&lt;p&gt;Coldae is a solver writter by U Ascher and R Spiteri, located here &lt;a href=&quot;http://www.cs.ubc.ca/~ascher/coldae.f&quot; rel=&quot;nofollow&quot;&gt;http://www.cs.ubc.ca/~ascher/coldae.f&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can solve upto index-2 DAEs using collocation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to Fortran, and having a hard time writing the driver, which is a collection of subroutines that define the function, jacobian, boundary conditions, guesses etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of what I need: &lt;a href=&quot;https://people.sc.fsu.edu/~jburkardt/f77_src/colnew/colnew_prb.f&quot; rel=&quot;nofollow&quot;&gt;https://people.sc.fsu.edu/~jburkardt/f77_src/colnew/colnew_prb.f&lt;/a&gt; . A solved sample problem for an older solver called &quot;COLNEW&quot; by the same author. &lt;/p&gt;&#xA;" OwnerUserId="6981" LastEditorUserId="6981" LastEditDate="2016-09-02T17:49:27.817" LastActivityDate="2017-04-30T23:42:42.200" Title="Does anyone have a driver for fortran DAE solver - COLDAE?" Tags="|fortran|collocation|dae|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24885" PostTypeId="2" ParentId="24883" CreationDate="2016-08-31T20:51:00.687" Score="5" Body="&lt;p&gt;A variant of Thomas's algorithm for tridiagonal systems of equations handles this problem.  See the Wikipedia page at:&#xA; &lt;a href=&quot;https://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the discussion at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cfd-online.com/Wiki/Tridiagonal_matrix_algorithm_-_TDMA_(Thomas_algorithm)&quot; rel=&quot;noreferrer&quot;&gt;http://www.cfd-online.com/Wiki/Tridiagonal_matrix_algorithm_-&lt;em&gt;TDMA&lt;/em&gt;(Thomas_algorithm)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-08-31T23:16:14.273" LastActivityDate="2016-08-31T23:16:14.273" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24886" PostTypeId="2" ParentId="24819" CreationDate="2016-08-31T21:36:54.470" Score="2" Body="&lt;p&gt;It's possible I'm misunderstanding something (and please let me know if I am), but here's another approach:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on your formulation, it looks like your first group of inequality constraints will always be tight. And since $Q_i \succeq 0$, there's no need for the $w_i \geq 0$ constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would allow you to rewrite the problem as&#xA;$$&#xA;\max_{\pmb a} \sum_i z_i - \sum_j w_j\\&#xA;\pmb a \pmb P_i\pmb a^\top = z_i\\&#xA; \quad\pmb a \pmb Q_j\pmb a^\top = w_j,&#xA;$$&#xA;which can be rewritten again as a problem without constraints:&#xA;$$&#xA;\max_{\pmb a} \sum_i \pmb a \pmb P_i\pmb a^\top - \sum_j \quad\pmb a \pmb Q_j\pmb a^\top.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we let $S = \sum_i P_i - \sum_j Q_j$, then the problem is equivalent to&#xA;$$&#xA;\max_{\pmb a} \pmb a^\top \pmb S \pmb a,&#xA;$$&#xA;which is easily solvable: If $S$ has any positive eigenvalues, the maximization problem is unbounded. Otherwise, the max is achieved at $a = 0$.&lt;/p&gt;&#xA;" OwnerUserId="7318" LastEditorUserId="7318" LastEditDate="2016-09-01T20:48:25.557" LastActivityDate="2016-09-01T20:48:25.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24887" PostTypeId="2" ParentId="24884" CreationDate="2016-09-01T12:20:37.670" Score="1" Body="&lt;p&gt;At line 473 of the source code you provided :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contrl - is the actual driver of the package. This routine contains the strategy for nonlinear equation solving.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code is written in Fortran 77 so you can't take benefit from a module. That means that you probably need to create a main program from scratch, prepare the arguments to call the subroutines you need in a specific order and compiling Coldae with your main together. Consider yourself lucky, the code seems rather well documented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can suggest you, at least, to follow Fortran tutorials (there is a good documentation on StackOverflow) and to carefully read the comments in the source code, the authors have also included their references so you should take a look too, you can't hope to understand the code if you don't know where it comes from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code gives you quite detailed information about the user subroutines needed in the driver. One possible approach is to choose a simple DAE problem, write the corresponding &lt;code&gt;fsub&lt;/code&gt;, &lt;code&gt;dfsub&lt;/code&gt; etc., and then insert them in the main subroutine (your driver). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, have a look at &lt;a href=&quot;http://wwwf.imperial.ac.uk/~jcash/BVP_software/readme.php&quot; rel=&quot;nofollow&quot;&gt;this example&lt;/a&gt; (note that the software they use is different but the user subroutines are very similar to those used by COLDAE).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C     *************    user supplied subroutines   *************&#xA;C&#xA;C&#xA;C     the following subroutines must be declared external in the&#xA;C     main program which calls coldae.&#xA;C&#xA;C&#xA;C     fsub  - name of subroutine for evaluating f(x,z(u(x)),y(x)) =&#xA;C                               t&#xA;C             (f ,...,f        )  at a point x in (aleft,aright). &#xA;C               1      ncomp+ny&#xA;C&#xA;C     dfsub - name of subroutine for evaluating the jacobian of&#xA;C             f(x,z(u),y) at a point x.  &#xA;C&#xA;C     gsub  - name of subroutine for evaluating the i-th component of&#xA;C             g(x,z(u(x))) = g (zeta(i),z(u(zeta(i)))) at a point x =&#xA;C                             i&#xA;C             zeta(i) where 1.le.i.le.mstar. &#xA;C&#xA;C     dgsub - name of subroutine for evaluating the i-th row of&#xA;C             the jacobian of g(x,z(u(x))).  &#xA;C&#xA;C     guess - name of subroutine to evaluate the initial&#xA;C             approximation for  z(u(x)), y(x) and for dmval(u(x))= vector&#xA;C             of the mj-th derivatives of u(x). &#xA;C&#xA;C**********************************************************************&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="17294" LastEditDate="2016-09-02T19:44:54.373" LastActivityDate="2016-09-02T19:44:54.373" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24888" PostTypeId="2" ParentId="24876" CreationDate="2016-09-01T13:37:34.533" Score="3" Body="&lt;p&gt;You might want to try the SelfAdjointEigenSolver class &#xA;&lt;a href=&quot;https://eigen.tuxfamily.org/dox/classEigen_1_1SelfAdjointEigenSolver.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eigen.tuxfamily.org/dox/classEigen_1_1SelfAdjointEigenSolver.html&lt;/a&gt;&#xA;in the Eigen C++ class library&#xA;&lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eigen.tuxfamily.org/index.php?title=Main_Page&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some numerical experiments with a 30x30 SPD matrix with a condition&#xA;number of 1000 constructed &#xA;according to the procedure described by Neumaier here&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/1789/generating-symmetric-positive-definite-matrices-using-indices/1793#1793&quot;&gt;Generating Symmetric Positive Definite Matrices using indices&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used a Windows 7 computer with a 2.8 GHz AMD Phenom II X4 830 CPU.&#xA;The compiler is VS 2013.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the experiments with Lapack ssyev/dsyev, I used OpenBlas&#xA;&lt;a href=&quot;http://www.openblas.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.openblas.net/&lt;/a&gt;.&#xA;In previous experiments I have found the BLAS operations in this library to&#xA;have very good performance. I suspect (but did not verify) that their&#xA;implementation of ssyev/dsyev simply uses the standard Fortran code but that&#xA;the performance of that routine depends significantly on the quality of the&#xA;underlying BLAS functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eigen::SelfAdjointEigenSolver was significantly faster than ssyev/dsyev in this&#xA;case-- ~0.25 ms per call compared with ~2 ms per call. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw only very small reduction of the time when I switched from double to&#xA;single precision. But I noticed large errors in the eigenvalues computed.&#xA;This was true for both Eigen and Lapack.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-09-01T13:37:34.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24889" PostTypeId="1" CreationDate="2016-09-01T14:14:06.680" Score="3" ViewCount="244" Body="&lt;p&gt;I'm currently working with this kind of mesh (AMR) :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/q6y75.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q6y75.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute the gradient at the center of the cell P, I use the following &lt;a href=&quot;http://www.cfd-online.com/Wiki/Gradient_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;formula&lt;/a&gt; :&#xA;$$\nabla \phi_P = \frac{1}{\Omega} \sum_{faces} \phi_f \vec{S_f}$$&#xA;where $\Omega$ is the volume of the cell and $\phi_f$ is the approximation of the flux on the interface between cell P and its neighbor.&#xA;The simplest choice for computing $\phi_f$ is to take the average of the cell-centred variables : $$\phi_f = \frac{1}{2}(\phi_P + \phi_N) ~~~~(1)$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I am wondering if this approximation of $\phi_f$ can be applied in my example above. I read &lt;a href=&quot;http://www.cfd-online.com/Wiki/Gradient_computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; that the arithmetic average works only for constant mesh size. For non conform grid like mine, it suggests to take a weighting factor depending on the geometry :&#xA;$$\phi_f = \alpha \phi_P + (1-\alpha) \phi_N$$&#xA;where :&#xA;$$\alpha = \frac{\vec{r_N}-\vec{r_f}}{\vec{r_N}-\vec{r_P}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it makes sense to me. Then, my question is : do I make a huge mistake by considering an arithmetic average ? If I have correctly understood, the correct approximation would be, in my case ($\alpha = \frac{1}{3}$) :&#xA;$$ \phi_f = \frac{1}{3} \phi_P + \frac{2}{3} \phi_N~~~~(2)$$&#xA;The solver is stable though, but do you think I lose accuracy with (1) instead of (2) ?&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-10-14T07:20:07.187" Title="Gradient computation in AMR framework (Green-Gauss theorem)" Tags="|pde|numerical-analysis|finite-volume|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24890" PostTypeId="5" CreationDate="2016-09-01T14:56:02.893" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2016-09-01T14:56:02.893" LastActivityDate="2016-09-01T14:56:02.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24891" PostTypeId="4" CreationDate="2016-09-01T14:56:02.893" Score="0" Body="A (system of) Differential Algebraic Equations (DAE) is a system of equations with ordinary differential equations and algebraic equations; all the equations should be satisfied simultaneously." OwnerUserId="9667" LastEditorUserId="1804" LastEditDate="2016-09-01T15:07:32.383" LastActivityDate="2016-09-01T15:07:32.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24892" PostTypeId="1" CreationDate="2016-09-01T16:04:12.300" Score="1" ViewCount="418" Body="&lt;p&gt;I am looking for a website/database of computer hardware (CPUs/GPUs) benchmarked with common scientific tools, e.g. *GEMM (see comment below for more info on this tool). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am quite frustrated looking for this information, as it seems scattered across blogs, embedded in manufacturing marketing materials, or shadowed by the plethora of enthusiast gamers posting FPS results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any good databases out there? Or even some good peer-reviewed journals to look out for?&lt;/p&gt;&#xA;" OwnerUserId="20359" LastEditorUserId="20359" LastEditDate="2016-09-01T21:08:31.893" LastActivityDate="2016-11-02T15:43:52.827" Title="Computational Science Hardware Benchmark Database" Tags="|gpu|benchmarking|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24893" PostTypeId="1" CreationDate="2016-09-01T19:33:30.637" Score="2" ViewCount="157" Body="&lt;p&gt;I have a function of many variables (~200-2000) which I am optimizing with some success using L-BFGS. While the function is expensive to evaluate, the gradient can be computed with not much additional effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there are any techniques which could allow this optimization to be sped up by parallel evaluation of the function. On this problem, L-BFGS averages ~1.1-1.3 evaluations per iteration, so I think to a good approximation, I can ignore the line search component of the algorithm. From this perspective then, on every iteration, L-BFGS chooses a tentative input evaluates the function and its gradient, updates its estimation of the inverse Hessian and repeats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagine it is possible to enhance this approach by choosing &lt;em&gt;multiple&lt;/em&gt; tentative inputs and using the multiple resulting gradients to form a better estimation of the Hessian. I imagine there is diminishing returns of some sort, the point selected by L-BFGS probably has the highest expected information, but surely there is some additional information about the Hessian that one can infer by using more function evaluations per iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this type of approach been investigated? I know there are many resources for parallel gradient-free optimization, as well as parallel gradient estimation, but I have not come across anything like this yet.&lt;/p&gt;&#xA;" OwnerUserId="21556" LastActivityDate="2016-09-01T19:33:30.637" Title="Quasi-Newton Optimization with parallel function evaluation" Tags="|optimization|parallel-computing|quasi-newton|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24895" PostTypeId="1" CreationDate="2016-09-01T20:30:11.870" Score="1" ViewCount="53" Body="&lt;p&gt;I am trying to solve convection diffusion type equations by a modification:&#xA;$$-\epsilon \Delta u+ b \nabla u+cu=f_1,$$&#xA;with the following boundary conditions:&#xA;$$u=g~ on ~\Gamma_D,~~~~~~~~~~~~~~ \epsilon \frac{\partial u}{\partial n}=g_N ~~on ~~\Gamma_N.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is  to solve another equation:&#xA;$$-\nu\Delta u+ \nabla u+\frac{c}{b}u=f_2,$$&#xA;$$u=g~ on ~\Gamma_D,~~~~~~~~~~~~~~ \epsilon \frac{\partial u}{\partial n}=g_N ~~on ~~\Gamma_N.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\nu=\frac{\epsilon}{b}$ &#xA;and then use $\nu+\delta$ instead of $\nu.$ &#xA;so we should solve&#xA;$$$$&#xA;$$-(\nu+ \delta) \Delta u+ \nabla u+\frac{c}{b}u=f_3,$$&#xA;$$u=g~ on ~\Gamma_D,~~~~~~~~~~~~~~ \epsilon \frac{\partial u}{\partial n}=g_N ~~on ~~\Gamma_N.$$&#xA;Here&#xA;$(\nu+\delta)=\frac{h}{2}*coth(\frac{h}{2*\nu})$, where $h$ is the mesh size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the question is that what the relations are between $f_1,f_2,f_3.$ I used $f_2=f_3$ and $f_2=\frac{f_1}{b}$. Is it true?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if $b$ be a vector I used $||b||_\infty$, but the results are acceptable just for some examples. Am I wrong in this way?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastActivityDate="2016-09-01T20:30:11.870" Title="Modified convection diffusion reaction equations" Tags="|finite-element|" AnswerCount="0" CommentCount="6" ClosedDate="2019-06-11T23:24:55.153" ContentLicense="CC BY-SA 3.0" />
  <row Id="24896" PostTypeId="1" CreationDate="2016-09-01T20:44:39.777" Score="1" ViewCount="132" Body="&lt;p&gt;So I have a code that I use which solves a 1D variable coefficient diffusion problem in cartesian coordinates:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t}=\frac{\partial}{\partial x}\left(D(x)\frac{\partial u}{\partial x}\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It solves the equation implicitly using a backward euler discretization. it works fine, but I want to convert it into a spherically symmetric code. The problem is that I can't find information on how to do this for variable coefficients. I would appreciate if someone shows me the specific modifications I need to make.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how much different those the solution for this equation looks in a 1D cartesian slab vs a 1D spherically symmetric radial slab? If they look about the same then maybe I shouldn't do the modifications. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="17075" LastActivityDate="2016-09-01T23:30:06.230" Title="Transforming a 1D cartesian variable-coefficient diffusion code into a 1D adially symmetric one" Tags="|finite-difference|numerics|diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24897" PostTypeId="2" ParentId="24896" CreationDate="2016-09-01T23:30:06.230" Score="3" Body="&lt;p&gt;More generally, the equation you are solving is&#xA;\begin{equation}&#xA;\frac{\partial u}{\partial t} = \nabla \cdot \left( D(x) \nabla u \right) \quad ,&#xA;\end{equation}&#xA;where the first nabla operator represents the divergence operator, and the second represents the gradient operator. Your equation is the special case for a one-dimensional cartesian coordinate system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a practical point of view, if you want to move into a cylindrical or spherical coordinate system all you have to do is to apply the respective definitions of the nabla operators.&#xA;For example, you can find them on Wikipedia:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Divergence#Spherical_coordinates&quot; rel=&quot;nofollow&quot;&gt;Divergence#Spherical_coordinates&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient#Cylindrical_and_spherical_coordinates&quot; rel=&quot;nofollow&quot;&gt;Gradient#Cylindrical_and_spherical_coordinates&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you take axial and azimuthal symmetry into consideration, you can derive a similar equation in $r$ that includes metric terms, i.e.:&#xA;\begin{equation}&#xA;\frac{\partial u}{\partial t} = \frac{1}{r} \frac{\partial}{\partial r} \left( D(r) r \frac{\partial u}{\partial r} \right) \quad ,&#xA;\end{equation}&#xA;As you can see, it does not really matter in this approach whether your diffusion coefficient is constant or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure whether telling you the final equation was already spoiling too much.&#xA;But I am sure that you know how to do the implementation from here on.&#xA;;-)&lt;/p&gt;&#xA;" OwnerUserId="20357" LastActivityDate="2016-09-01T23:30:06.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24901" PostTypeId="2" ParentId="21623" CreationDate="2016-09-02T07:37:03.690" Score="1" Body="&lt;p&gt;One way to implement Polylogarithm functions can be found in the book &quot;Numerical Recipes: The Art of Scientific Computing&quot;. The book includes some of the theory and practical aspects of evaluating special functions and finding zeros of equations. If you want, you can also buy the codes at their web page.&lt;/p&gt;&#xA;" OwnerUserId="21561" LastActivityDate="2016-09-02T07:37:03.690" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24902" PostTypeId="2" ParentId="24889" CreationDate="2016-09-02T10:26:22.773" Score="1" Body="&lt;p&gt;For an orthogonal grid, using a proper linear interpolation (2) for the two point flux approximation in this case will result in a theoretical second order accuracy in the local equation truncation error in the calculation of the gradient (the first order error error term arriving from face f ends up cancelling with the error term arising from using the same formula with the face above it). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However you also say that you are implementing a MUSCL scheme, implying that you are performing some flux limiting. This implies that your global error will generally be dominated by the necessity for a first order scheme in the vicinity of shocks to damp spurious oscillations. This makes makes a priori analysis difficult. If you're interested enough and have enough time, you could perform a mesh convergence study for the actual problem you're interested in to see how much difference it actually makes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Editing in some raw mathematics:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define the point $q$ to be the midpoint of the full righthand face of cell P and label the cell above cell $N$ as cell $M$. Then I claim that using eqn (2) to give the face values for the two sub surfaces in your numerical integration is equivalent to taking &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi_q = \frac{1}{3}\phi_P + \frac{2}{3} \frac{\phi_N +\phi_M}{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and integrating over the whole face. Let $\phi$ be a sufficiently smooth exact solution, then applying Taylor series expansions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi(x_P) = \phi(x_q)-\left. h \frac{\partial \phi}{\partial x}\right|_{x_q}+\left.\frac{h^2}{2} \frac{\partial^2 \phi}{\partial x^2}\right|_{x_h}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi(x_M) = \phi(x_q)+\left.\frac{h}{2}\frac{\partial \phi}{\partial x}\right|_{x_q}+\left.\frac{h}{2}\frac{\partial \phi}{\partial y}\right|_{x_q}+\mathcal{O}(h^2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi(x_N) = \phi(x_q)+\left.\frac{h}{2}\frac{\partial \phi}{\partial x}\right|_{x_q}-\left.\frac{h}{2}\frac{\partial \phi}{\partial y}\right|_{x_q}+\mathcal{O}(h^2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And simple substitution shows &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi_q = \phi (x_q) + \mathcal{O}(h^2)$.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="20454" LastEditDate="2016-09-06T13:46:24.913" LastActivityDate="2016-09-06T13:46:24.913" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24904" PostTypeId="1" CreationDate="2016-09-02T14:34:23.747" Score="1" ViewCount="75" Body="&lt;p&gt;I want to numerically solve for electric field of an object in an external electric field. The object has linear displacement field such that its permitivity ($\epsilon$) is constant. The object's shape is not important but I would prefer an ellipse. I do not know how should I impose curl and divergence equations into simulation as well as boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I need is electric field or charge density. Does anyone know how can I calculate charge density of a dielectric ellipse in an external electric field?&lt;/p&gt;&#xA;" OwnerUserId="21569" LastEditorUserId="21569" LastEditDate="2016-09-03T03:47:28.540" LastActivityDate="2016-09-03T03:47:28.540" Title="Dielectric in an external electric field" Tags="|pde|simulation|electromagnetism|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24905" PostTypeId="1" CreationDate="2016-09-02T16:13:47.227" Score="2" ViewCount="409" Body="&lt;p&gt;I want to interpolate a periodic function on a non-equidistant grid and have implemented it the using the Lagrangian formula described in &lt;a href=&quot;https://en.wikipedia.org/wiki/Trigonometric_interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;. For an &lt;strong&gt;odd&lt;/strong&gt; number of data points, this takes the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(x) = \sum_{k=0}^{2K} y_k\,t_k(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$t_k(x) = \prod_{m=0,m\ne k}^{2K} \frac{\sin\frac12(x-x_m)}{\sin\frac12(x_k-x_m)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which bears strong resemblance to the usual Lagrangian interpolation formula, but with sines and a the factors 1/2 appearing. This formula makes perfect sense and in fact, I used the equivalent of the barycentric form described in the wikipedia page of &lt;code&gt;Lagrange polynomials&lt;/code&gt; (I cannot post two links):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ p(x) = \frac{\sum_{k=0}^{2K} \frac{w_k}{\sin{\frac{1}{2}\left(x-x_k\right)}}y_k}{\sum_{k=0}^{2K} \frac{w_k}{\sin{\frac{1}{2}\left(x-x_k\right)}}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w_k = \frac{1}{l'\left(x_k\right)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$l\left(x\right) = \prod_{k=0}^{2K} \sin{\frac{1}{2}\left(x-x_k\right)}$$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works magnificently, as can be seen in the following figure, where I use 15 (&lt;em&gt;red&lt;/em&gt;) points of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f\left(x\right) = \sin{\left(x\right)} + \frac{1}{4}\cos{\left(2x\right)} + 2 \sin{\left(6x\right)} + 3 \cos{\left(7 x\right)}$$ ,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;defined on a non-equidistant grid (in this case with more points on the edges of the region $2\ldots \pi$). As Nyquist states, this is indeed enough information to interpolate $f\left(x\right)$, which I illustrate on 100 (&lt;em&gt;blue&lt;/em&gt;) points, also non-equidistant (but in this case with more points in the middle of the region).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VP7fp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VP7fp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for an &lt;strong&gt;even&lt;/strong&gt; number of points, there is an additional degree of freedom. In the Wikipedia entry for trigonometric interpolation, it is stated that this is usually employed to make sure the highest modal content only has a cosine, and no sine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, they state, is implemented by using additional factors &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha_k=\sum_{m=0,m\ne k}^{2K-1} x_m$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that modify the formulas to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$t_k(x) = \frac{\sin\frac12(x-\alpha_k)}{\sin\frac12(x_k-\alpha_k)}\prod_{m=0,m\ne k}^{2K-1} \frac{\sin\frac12(x-x_m)}{\sin\frac12(x_k-x_m)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, however, I have tried, and I have found it does not work. Not only does the interpolated function not go through the correct points, but also it blows up to high absolute values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I have investigated it in more detail, and have found in fact that this definition of $\alpha_k$ contains singularities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take for example the case with 4 points &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_k = \frac{k \pi}{4} \ , k = 0\ldots 3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;applied to a simple sine. Note that according to Nyquist, 3 points would be enough, so there is indeed a redundancy of 1 point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's again look at the interval $x= 0\ldots 2 \pi$. As the only non-vanishing components of $y_k$ are 1 and 3 (with $y_1 = 1$ and $y_3=-1$), the necessary factors $t_k$ are given by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$t_1 = \sqrt{2} \sin{\frac{x}{2}} \cos{\frac{x}{2}} \left( \cos{\frac{x}{2}} + \sin{\frac{x}{2}} \right ) \frac{\sin{\frac{1}{2}\left(x-\alpha_1\right)}}{\sin{\frac{1}{2}\left(\frac{\pi}{2} - \alpha_1\right)}} \\ &#xA;t_3 = \sqrt{2} \sin{\frac{x}{2}} \cos{\frac{x}{2}} \left( \cos{\frac{x}{2}} - \sin{\frac{x}{2}} \right ) \frac{\sin{\frac{1}{2}\left(x-\alpha_3\right)}}{\sin{\frac{1}{2}\left(\frac{3\pi}{2} - \alpha_3\right)}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where it can be seen that there are extra degrees of freedom $\alpha_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, now using their definition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha_1 = \frac{5}{2} \pi \\&#xA;\alpha_3 = \frac{3}{2} \pi $$ , &lt;/p&gt;&#xA;&#xA;&lt;p&gt;this clearly leads to singularities in the denominators of $t_1$ and $t_3$. Does anybody know why this happens? There must either be something wrong in the definition of $\alpha_k$ or in my understanding of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to find other resources but the only thing I could find limited the discussion to an odd number of points (which indeed works fine.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also tried to follow the arguments in the wikipedia page and I am not able to capture why they reach that expression for $\alpha$, repeated here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha_k=\sum_{m=0,m\ne k}^{2K-1} x_m$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;By the way, as a side note, it can be seen that throwing away the fourth point $\frac{3}{2}\pi$, the remaining three points indeed lead to the correct interpolation:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p\left(x\right) =  \sum_{k=0}^{2} y_k\,t_k(x) = t_1 = \frac{\sin{\left(\frac{x}{2}\right)}}{\sin{\left(\frac{\pi}{4}\right)}}\frac{\sin{\left(\frac{x}{2}-\frac{\pi}{2}\right)}}{\sin{\left(-\frac{\pi}{4}\right)}} = 2 \sin{\left(\frac{x}{2}\right)}\cos{\left(\frac{x}{2}\right)} = \sin{\left(x\right)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, the interpolating formula for $p\left(x\right)$, using 4 points, can be written readily as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ p\left(x\right) = 2 s c \left[ \left(s + c \right) \frac{s c_1 - c s_1}{c_1-s_1} - \left(c-s\right) \frac{s c_3 - c s_3}{s_3 + c_3} \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $c = \cos{\left(\frac{x}{2}\right)}$, $s = \sin{\left(\frac{x}{2}\right)}$, $c_i = \cos{\left(\frac{\alpha_i}{2}\right)}$ and $s_i = \sin{\left(\frac{\alpha_i}{2}\right)}$ as a short-hand notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With basic trigonometric identities $\sin{\left(x\right)} = 2 \sin{\left(\frac{x}{2}\right)} \cos{\left(\frac{x}{2}\right)}$, $\cos{\left(x\right)} = \cos^2{\left(\frac{x}{2}\right)} - \sin^2{\left(\frac{x}{2}\right)}$, this can be rewritten as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p\left(x\right) = \sin{\left(x\right)} \left[  1 + \frac{1}{2} \cos{\left(x\right)} \left( \frac{s_1+c_1}{s_1-c_1} + \frac{s_3-c_3}{s_3+c_3}  \right)\right] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So clearly, to avoid having a term $\propto \sin{\left(2 x \right)}$, the condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{s_1+c_1}{s_1-c_1} + \frac{s_3-c_3}{s_3+c_3} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;needs to hold. But, again, with $\alpha_1 = \frac{5}{2} \pi$ and $\alpha_3 = \frac{3}{2} \pi$, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ s_1 = -\frac{\sqrt{2}}{2} \ , \ c_1 = -\frac{\sqrt{2}}{2}  \\ &#xA;s_3 = \frac{\sqrt{2}}{2} \ , \ c_3 = -\frac{\sqrt{2}}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. singularities in both denominators...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2&lt;/strong&gt;&#xA;Above condition can be rewritten to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\cos{\frac{1}{2}\left( \alpha_1 - \alpha_3 \right)}}{\left(\sin{\left(\frac{\alpha_1}{2}\right)}-\cos{\left(\frac{\alpha_1}{2}\right)}\right) \left(\sin{\left(\frac{\alpha_3}{2}\right)}+\cos{\left(\frac{\alpha_3}{2}\right)}\right)} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is satisfied for $\alpha_1-\alpha_3 = (1+2 r) \pi$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For our simple test case, this is indeed correct, but at the same time they give infinities in the denominator, which becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(\sin{\left(\frac{\alpha_1}{2}\right)}-\cos{\left(\frac{\alpha_1}{2}\right)}\right)^2 = 0^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this is probably not always the case. Indeed, I have run interpolations where there were no problem. Would it just be due to an unfortunate choice of sampling points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;MY OWN, SLIGHTLY UNSATISFYING ANSWER&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some more thought I think that&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;One of the formulas on Wikipedia was slightly wrong. I have corrected it there, but the result is still $\alpha_k = \sum_{j=0, j\ne k}^{2K-1} x_j$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This formula means nothing more than &lt;em&gt;set the phase of the contribution of this point to the highest frequency to zero, i.e. it should be a cosine.&lt;/em&gt; Logically, another option is to set it to $\pi$ in order to get sines at the highest frequency. Or anything else. There is no clear preference&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When forcing the phase to be some value, it can often happen that the denominators in the formulas contain zeros, leading to infinities. This is the case when $\sin{\frac{1}{2}\left(x_k - \alpha_k\right)} = 0$. It is very important, therefore, to have the $\alpha_k$'s satisfy the condition $x_k = \alpha_k \ne m \pi$ with $m$ an integer. This can impede the preference for the phase of the highest frequency.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sometimes it is clear that the phase of the highest frequency does not matter as the amplitude should be zero, as in the case for my simple sine. There is, however, imho no criterion that can be used there, as it requires information about the &lt;em&gt;function values $f_k$&lt;/em&gt; and not only the values of the points $x_k$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In particular, in the case for a simple sine, a phase of 0 is not possible, as this leads to infinities. An other phase has to be used, but it does not matter which: Due to the fact that $\alpha_1 -\alpha_3 = \pi$ and $y_1 = - y_3$, the global contribution to the highest frequency is zero, as required.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It is better just to take an odd number of points.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21549" LastEditorUserId="21549" LastEditDate="2016-09-05T11:29:34.210" LastActivityDate="2016-09-05T11:29:34.210" Title="trigonometric interpolation with non-equidistant sampling" Tags="|interpolation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24906" PostTypeId="2" ParentId="21623" CreationDate="2016-09-02T16:17:14.470" Score="0" Body="&lt;p&gt;A great algorithm for this, including the analytic continuation, is provided &lt;a href=&quot;http://www.wolfgang-ehrhardt.de/Polylog.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;. It is rare that I recommend to write these things yourself, but in this case it is very straightforward.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-09-02T16:17:14.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24908" PostTypeId="1" CreationDate="2016-09-03T02:50:33.273" Score="3" ViewCount="127" Body="&lt;p&gt;Just studying some toy examples of $2\times 2$ and $3 \times 3$ matrices, complex number multiplication already gets a bit messy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a numerical analysis point of view, if one were to try and build very large matrices to model physical behavior, is it much more desirable to build &lt;em&gt;real&lt;/em&gt; matrices over complex matrices, in terms of computational cost?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other advantages are there regarding real vs. complex matrices?&lt;/p&gt;&#xA;" OwnerUserId="21574" LastActivityDate="2016-09-07T20:27:26.370" Title="In constructing matrices to model physical phenomena, are real matrices superior to complex matrices, in terms of computational cost?" Tags="|numerical-analysis|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24909" PostTypeId="1" AcceptedAnswerId="24910" CreationDate="2016-09-03T17:12:36.443" Score="1" ViewCount="755" Body="&lt;ul&gt;&#xA;&lt;li&gt;Could someone explain the link (if any) between the spectral methods (SM), as presented for example &lt;a href=&quot;http://www.scholarpedia.org/article/Spectral_methods&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;&#xA;and the so called spectral volume methods (SV) and spectral difference methods (SD) for CFD ?&lt;/li&gt;&#xA;&lt;li&gt;Are SV and SD the applications of SM to CFD ?&lt;/li&gt;&#xA;&lt;li&gt;If not, is there an application to CFD of SM (that would then have no link with SV and SD) ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21580" LastEditorUserId="10134" LastEditDate="2016-09-04T15:00:34.837" LastActivityDate="2016-09-06T03:17:17.327" Title="Spectral methods, Spectral Volume methods, Spectral Difference methods" Tags="|finite-element|finite-difference|finite-volume|discontinuous-galerkin|spectral-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24910" PostTypeId="2" ParentId="24909" CreationDate="2016-09-04T03:39:48.500" Score="5" Body="&lt;p&gt;&quot;Spectral methods&quot; usually means methods which make use of global basis functions. Fourier spectral methods use sine and cosine, and are used when you have periodic boundary conditions. Chebyshev methods use Chebyshev polynomials and are useful in non-periodic cases. These two methods are used in DNS, see e.g., hit3d which uses fourier and periodic bc, and channelflow which uses fourier in two directions and chebyshev in third direction to model flow through channels. Fourier uses uniform points while Chebyshev uses roots of some polynomials. So these methods are usually restricted to rectagular/cubical domains. If you can map your domain to a cartesian domain, then you can still apply these methods, see e.g., the excellent chebfun software which has recently added support for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For complicated geometries, you will have to use unstructured grids. In these cases, one uses &quot;spectral element&quot; methods, which are essentially finite element methods that use high degree polynomial basis functions with compact support. For examples of such methods used in CFD, see nek5000 which solves incompressible Navier-Stokes equations and nektar++ which has a lot more models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Spectral difference&quot; and &quot;spectral volume&quot; methods are in the same spirit as &quot;spectral element&quot; methods. They use local basis functions but you can achieve &quot;spectral-like&quot; accuracy by using high degree polynomials.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2016-09-06T03:17:17.327" LastActivityDate="2016-09-06T03:17:17.327" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24911" PostTypeId="1" AcceptedAnswerId="24938" CreationDate="2016-09-04T15:04:18.517" Score="1" ViewCount="313" Body="&lt;p&gt;I wrote a code which solves the 2D Poisson equation with homogeneous Dirichlet BC everywhere and a source term of -1. I am using the classical Jacobi iteration method. The grid is $N_x \, \mathrm{x} \, N_y$, and all the arrays are 0 based.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code yields good results with a 5 points stencil (2$^{\mathrm{nd}}$ order finite difference method). With this stencil, the code loops all interior points (i.e. from $1$ to $N_x-2$ in the $x$ direction), while the boundary points are never touched.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the code diverges when using a 9 points stentil, 4$^\mathrm{th}$ order central difference method. Again, only the interior points are iterated on. Since the discretization uses two points on each side, I am extrapolating past the boundary points (ghost points) with a first order approximation, i.e. $U(-1) = U(0), U(Nx) = U(Nx-1)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I have to use special discretization methods near the boundary? I could use non-symmetric finite difference schemes, but I would expect my method (with ghost points) to work. Below is a sample of my code. For this case, the grid is equidistant ($\Delta_x = \Delta_y$, or in the code $h(1) = h(2)$).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~               &#xA;! Jacobi scheme                                                                                          &#xA;if(method_order == second_order) then                                                                    &#xA;    ! 2nd order method: 5 points stencil                                                                 &#xA;    do jj=min_y,max_y                                                                                    &#xA;        do ii=min_x,max_x                                                                                &#xA;            Ujacob(ii,jj) = 0.25_dp                                                     &amp;amp;               &#xA;                &amp;amp;   * (Uold(ii-1,jj) + Uold(ii+1,jj) + Uold(ii,jj-1) + Uold(ii,jj+1)    &amp;amp;               &#xA;                &amp;amp;   - h(1)**2 * Urhs(ii,jj))                                                             &#xA;        end do                                                                                           &#xA;    end do                                                                                               &#xA;else if(method_order == fourth_order) then                                                               &#xA;&#xA;    ! Fill buffer points                                                                                  &#xA;    Ujacob(     -1, 0:grid_ny-1) = -Ujacob(        1, 0:grid_ny-1)                                        &#xA;    Ujacob(grid_nx, 0:grid_ny-1) = -Ujacob(grid_nx-2, 0:grid_ny-1)                                        &#xA;&#xA;    Ujacob(0:grid_nx-1,      -1) = -Ujacob(0:grid_nx-1,         1)                                        &#xA;    Ujacob(0:grid_nx-1, grid_ny) = -Ujacob(0:grid_nx-1, grid_ny-2)  &#xA;&#xA;    ! 4th order method: 9 points stencil                            &#xA;    do jj=min_y,max_y                                                                                    &#xA;        do ii=min_x,max_x                                                                                &#xA;            Ujacob(ii,jj) = 1./60._dp                                                   &amp;amp;               &#xA;        &amp;amp;   *(-1._dp * (Uold(ii-2,jj) + Uold(ii+2,jj) + Uold(ii,jj-2) + Uold(ii,jj+2))  &amp;amp;               &#xA;        &amp;amp;   + 16._dp * (Uold(ii-1,jj) + Uold(ii+1,jj) + Uold(ii,jj-1) + Uold(ii,jj+1))  &amp;amp;               &#xA;        &amp;amp;   - 12._dp * h(1)**2 * Urhs(ii,jj))                                                            &#xA;        end do                                                                                           &#xA;    end do                                                                                               &#xA;end if&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following Wolfgang's advice, I fill my ghost points by mirroring what's happening next to the boundary. However, that does not help and the code still diverges.&lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="16329" LastEditDate="2016-09-06T13:57:22.813" LastActivityDate="2016-11-06T12:44:21.767" Title="Boundary treatment with higher order methods" Tags="|computational-physics|poisson|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24912" PostTypeId="2" ParentId="6701" CreationDate="2016-09-04T16:23:21.933" Score="1" Body="&lt;p&gt;Maybe those subroutines will help you. They are not as efficient as I would like and one of the LAPACK calls simply destroys the solution (the commented one) but as it is below, it works! &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program testando&#xA;&#xA;implicit none&#xA;&#xA;integer(4) :: i,j&#xA;real(8), dimension(2,2,3) :: A&#xA;real(8), dimension(2,2,3) :: B,C&#xA;real(8), dimension(2,3) :: xb&#xA;real(8), dimension(2,3) :: x&#xA;&#xA;do i = 1,3&#xA;    A(1,1,i) = 4.0d0&#xA;    A(1,2,i) = 3.0d0&#xA;    A(2,1,i) = 3.0d0&#xA;    A(2,2,i) = 2.0d0&#xA;end do&#xA;&#xA;do i = 1,3&#xA;&#xA;    B(1,1,i) = 8.0d0&#xA;    B(1,2,i) = 6.0d0&#xA;    B(2,1,i) = 6.0d0&#xA;    B(2,2,i) = 4.0d0&#xA;&#xA;    C(1,1,i) = 16.0d0&#xA;    C(1,2,i) = 9.0d0&#xA;    C(2,1,i) = 9.0d0&#xA;    C(2,2,i) = 6.0d0&#xA;end do&#xA;&#xA;do i = 1, 2&#xA;    do j = 1, 3&#xA;        xb(i,j) = 3.0d0&#xA;    end do&#xA;end do&#xA;&#xA;&#xA;call cc299blktriad(A,B,C,2,3,xb,x)&#xA;&#xA;! answer&#xA;write(*,*) &quot;x(1) = 5.8571429&quot;&#xA;write(*,*) &quot;x(2) = -8.9220779&quot;&#xA;write(*,*) &quot;x(3) = 0.5714286&quot;&#xA;write(*,*) &quot;x(4) = -0.3116883&quot;&#xA;write(*,*) &quot;x(5) = 1.8571429&quot;&#xA;write(*,*) &quot;x(6) = -2.3766234&quot;&#xA;&#xA;write(*,*) &quot;Calculated...&quot; &#xA;&#xA;do j = 1, 3&#xA;    do i = 1, 2&#xA;        write(*,*) &quot;x(&quot;,i,&quot;) = &quot;,x(i,j)&#xA;    end do&#xA;end do&#xA;end program testando&#xA;&#xA;subroutine cc299blktriad(maind,lower,upper,id,md,xb,x)&#xA;&#xA; !|     B(1)    C(1)             | | xb(1)  |&#xA; !| A(2)  B(2)    C(2)           | |        |&#xA; !|   A(3)  B(3)                 | |        |&#xA; !|     .     .                  |*|        | = B[1:mb*3]&#xA; !|       .     .                | |        |&#xA; !|               .       C(mb-1)| |        |&#xA; !|         A(mb)  B(mb)         | |xb(n*id)|&#xA;&#xA;! id = inner matrices dimension.&#xA;! md = number matrices.&#xA;! maind = main diagonal of matrices  format: maind(id,id,md)&#xA;! lower = lower diagonal of matrices format: lower(id,id,2:md)&#xA;! upper = upper diagonal of matrices format: maind(id,id,md-1)&#xA;! xb    = B vector in Ax=B           format: xb(md*id)&#xA;! x     = x vector in Ax=B           format: x(md*id)&#xA;!&#xA;&#xA;implicit none&#xA;&#xA;! +++ Inputs +++&#xA;!&#xA;! Scalar input variables.&#xA;integer(kind=4) :: id,md&#xA;&#xA;! Main diagonal of matrices.&#xA;real(kind=8), dimension(id,id,md) :: maind&#xA;&#xA;! Lower diagonal of matrices.&#xA;real(kind=8), dimension(id,id,md) :: lower&#xA;&#xA;! Upper diagonal of matrices.&#xA;real(kind=8), dimension(id,id,md) :: upper&#xA;&#xA;! Vector of equalties B in Ax=B.&#xA;real(kind=8), dimension(id,md)    :: xb&#xA;&#xA;! Vector of answers x in Ax=B.&#xA;real(kind=8), dimension(id,md)    :: x&#xA;&#xA;! ++ Inside variables ++&#xA;!&#xA;! Scalar variables&#xA;integer(kind=4) :: i,ii,jj&#xA;&#xA;! Array of gamma coefficients.&#xA;real(kind=8), dimension(id,id,md) :: gamm&#xA;&#xA;! Array of beta coefficients.&#xA;real(kind=8), dimension(id,md) :: beta&#xA;&#xA;! Auxiliary arrays.&#xA;real(kind=8), allocatable, dimension(:,:) :: aux_copy&#xA;real(kind=8), allocatable, dimension(:,:) :: aux_mult&#xA;real(kind=8), allocatable, dimension(:,:) :: aux_summ&#xA;real(kind=8), allocatable, dimension(:)   :: aux_dumm&#xA;&#xA;allocate(aux_mult(id,id))&#xA;allocate(aux_summ(id,id))&#xA;allocate(aux_copy(id,id))&#xA;allocate(aux_dumm(id))&#xA;&#xA;&#xA;!--------------------------------------------------------------------------!&#xA;!                  Step 1: BLOCK TRIANGULARIZATION                         !&#xA;!--------------------------------------------------------------------------!&#xA;&#xA;! Lets first get our first gamma.&#xA;&#xA;call dlacpy('A',id,id,maind(:,:,1),id,aux_copy,id)&#xA;&#xA;call inv(aux_copy,aux_copy,id)&#xA;&#xA;call dgemm('N','N',id,id,id,1.0d0,aux_copy,id, &amp;amp; &#xA;    upper(:,:,1),id,1.0d0,gamm(:,:,1),id)&#xA;&#xA;&#xA;! Now that we have our first gamma, let's get the rest of them.&#xA;&#xA;do i = 2, md-1&#xA;&#xA;    call dgemm('N','N',id,id,id,1.0d0,lower(:,:,i),id, &amp;amp;&#xA;        gamm(:,:,i-1),id,1.0d0,aux_mult,id)&#xA;&#xA;    do jj = 1, id&#xA;        do ii = 1, id&#xA;            aux_summ(ii,jj) = maind(ii,jj,i) - aux_mult(ii,jj)&#xA;        end do&#xA;    end do&#xA;&#xA;    call inv(aux_summ,aux_summ,id)&#xA;&#xA;    call dgemm('N','N',id,id,id,1.0d0,aux_summ,id, &amp;amp;&#xA;        upper(:,:,i),id,1.0d0,gamm(:,:,i),id)&#xA;&#xA;end do&#xA;&#xA;&#xA;! Now that we have our gammas, let's get the betas, starting from the first&#xA;! ones. Note that now the calls done by the Lapack library will get a bit&#xA;! more complicated so let's use matmul...&#xA;&#xA;call dlacpy('A',id,id,maind(:,:,1),id,aux_copy,id)&#xA;&#xA;call inv(aux_copy,aux_copy,id)&#xA;&#xA;beta(:,1) = matmul(aux_copy,xb(:,1))&#xA;&#xA;&#xA;! We now have our first beta, lets get the rest.&#xA;&#xA;do i = 2, md&#xA;&#xA;    ! This LAPACK call does not work .... I dont know why.&#xA;    !call dgemm('N','N',id,id,id,1.0d0,lower(:,:,i),id, &amp;amp;&#xA;    !    gamm(:,:,i-1),id,1.0d0,aux_mult(:,:),id)&#xA;&#xA;    aux_mult(:,:) = matmul(lower(:,:,i),gamm(:,:,i-1))&#xA;&#xA;    do jj = 1, id&#xA;        do ii = 1, id&#xA;            aux_summ(ii,jj) = maind(ii,jj,i) - aux_mult(ii,jj)&#xA;        end do&#xA;    end do&#xA;&#xA;    call inv(aux_summ,aux_summ,id)&#xA;&#xA;    aux_dumm(:) = xb(:,i) - matmul(lower(:,:,i),beta(:,i-1))&#xA;&#xA;    beta(:,i) = matmul(aux_summ(:,:),aux_dumm(:))&#xA;&#xA;end do&#xA;&#xA;&#xA;!--------------------------------------------------------------------------!&#xA;!                  Step 2: BACKWARD SWEEP                                  !&#xA;!--------------------------------------------------------------------------!&#xA;&#xA;&#xA;! How cool is that, lets start build our solution vector... iupiiii!&#xA;&#xA;x(:,md) = beta(:,md)&#xA;&#xA;do i = md-1,1,-1&#xA;&#xA;    aux_dumm(:) = matmul(gamm(:,:,i),x(:,i+1))&#xA;&#xA;    do ii = 1, id&#xA;        x(ii,i) = beta(ii,i) - aux_dumm(ii)&#xA;    end do&#xA;&#xA;end do&#xA;&#xA;&#xA;deallocate(aux_mult)&#xA;deallocate(aux_summ)&#xA;deallocate(aux_copy)&#xA;deallocate(aux_dumm)&#xA;&#xA;end subroutine cc299blktriad&#xA;&#xA;&#xA;subroutine inv(A,A_inv,m)&#xA;&#xA; Implicit none&#xA; integer :: m&#xA; real(8), dimension(m,m)::A, A_inv&#xA; real(8),dimension(m)::WORK&#xA; integer,dimension(m)::IPIV&#xA; integer info&#xA;&#xA; A_inv = A&#xA;&#xA; call DGETRF(M,M,A_inv,M,IPIV,info)&#xA;&#xA; if (info /=  0) then&#xA;   write(*,*)&quot;DGETRF: Failed during matrix factorization&quot;&#xA;   stop&#xA; end if&#xA;&#xA; call DGETRI(M,A_inv,M,IPIV,WORK,M,info)&#xA;&#xA; if (info /=  0) then&#xA;  write(*,*)&quot;DGETRI: Failed during matrix inversion.&quot;&#xA;  stop&#xA; end if&#xA;&#xA;end subroutine inv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You have to link LAPACK to run it. With gfortran and a proper LAPACK installation in OpenSUSE the line is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gfortran -g -llapack -lblas &amp;lt;program&amp;gt; -o block&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any tips on how to improve the performance of this subroutine will be welcome as well as why the LAPACK call is not working.&lt;/p&gt;&#xA;" OwnerUserId="21588" LastEditorUserId="20688" LastEditDate="2018-11-08T00:38:54.787" LastActivityDate="2018-11-08T00:38:54.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="24913" PostTypeId="1" AcceptedAnswerId="24926" CreationDate="2016-09-04T18:44:36.437" Score="3" ViewCount="928" Body="&lt;p&gt;What would be the rigorous mathematical expression of the fact that a conservation law discretized with a Finite Element method with Galerkin discretization does not result in a conservative scheme ?&lt;/p&gt;&#xA;" OwnerUserId="21580" LastActivityDate="2016-09-09T07:51:24.637" Title="Discrete conservation and Finite Element methods" Tags="|finite-element|numerical-analysis|fluid-dynamics|conservation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24914" PostTypeId="1" CreationDate="2016-09-05T01:12:52.080" Score="2" ViewCount="181" Body="&lt;p&gt;Our current lab uses SNOPT as an optimizer for aerodynamic shape optimization. I am currently working on building an approximate Hessian for the sake of improving the convergence speed and limits. Unfortunately, SNOPT only allows me to define an initial Hessian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a FORTRAN package that would allow me to input the Hessian at every design iteration?&lt;/p&gt;&#xA;" OwnerUserId="21593" LastActivityDate="2016-09-05T14:34:43.857" Title="Non-linear optimization package that allows an user-defined Hessian" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24916" PostTypeId="1" CreationDate="2016-09-05T10:54:29.110" Score="7" ViewCount="2501" Body="&lt;p&gt;Consider the following scenario: I want to perform a large Monte Carlo simulation across a compute cluster with several nodes. To avoid excessive transmission of data, I am going to generate random data for my simulation on the individual nodes. Using Python, I would like to do this using &lt;code&gt;numpy.random&lt;/code&gt; functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to ensure that the individual nodes generate &lt;em&gt;different&lt;/em&gt; random data (if they generate the same they will just be replicating each other's work). I can do this by selecting different seeds for the RNGs on the individual nodes. But how do I select these seeds to ensure that the resulting random number sequences do not accidentally intersect somewhat across different nodes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I simply generate random seed numbers for my compute nodes on one node submitting the workloads to the compute nodes, I could theoretically end up in the situation where the compute nodes generate random number sequences that intersect somewhat in the RNGs' space of possible random number sequences, attempted illustrated here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;node 1  |-------************----------------------------------------------|&#xA;node 2  |----------------------------------************-------------------|&#xA;node 3  |---************--------------------------------------------------|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;***&lt;/code&gt;s illustrate the sequence of random numbers generated by each node and their starting point is given by the (randomly generated) seed given to each of them at the beginning of their work. I realise that the space of possible random number sequences is probably enormous and the probability of this happening is probably quite low, but how do I ensure that it cannot happen (assuming that the nodes' collective demand for random numbers does not exhaust the space fo possible random sequences)?&lt;/p&gt;&#xA;" OwnerUserId="3724" LastActivityDate="2019-02-26T11:44:26.407" Title="How do I reliably generate random numbers in Python distributed across multiple nodes?" Tags="|python|parallel-computing|random-number-generation|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24917" PostTypeId="2" ParentId="24916" CreationDate="2016-09-05T11:38:12.510" Score="13" Body="&lt;p&gt;To the best of my knowledge, Numpy does not support independent streams. Indeed, getting independent streams from the Mersenne Twister (Pythons RNG) is &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/JUMP/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;notoriously difficult&lt;/a&gt; although it &lt;a href=&quot;http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/DC/dgene.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be done&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider using the &lt;a href=&quot;https://pypi.org/project/randomgen/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;RandomGen&quot;&gt;RandomGen&lt;/a&gt; package. It is fully compatible with Numpy, and provides you with the PCG64 generator, supporting up to &lt;span class=&quot;math-container&quot;&gt;$2^{63}$&lt;/span&gt; independent streams. Arguably enough for most datacenters. See &lt;a href=&quot;https://bashtage.github.io/randomgen/brng/pcg32.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="7522" LastEditorUserId="7522" LastEditDate="2019-02-26T11:44:26.407" LastActivityDate="2019-02-26T11:44:26.407" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="24918" PostTypeId="1" CreationDate="2016-09-05T13:26:13.120" Score="2" ViewCount="393" Body="&lt;p&gt;i'm solving sparse linear equations within &lt;strong&gt;scipy 0.18&lt;/strong&gt; which internally resorts to &lt;strong&gt;SuperLU&lt;/strong&gt; (after umfpack got removed due to license-issues).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current, i'm doing a complete re-factorization in each iteration of my algorithm, but now i &lt;strong&gt;want to re-use the permutation-ordering&lt;/strong&gt; (at least the column-permutation; but maybe even row-permutation) calculated in the first iteration.&#xA;How can i achieve this in a nice way using scipy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got no previous usage-history of SuperLU (and also not much practical experience with sparse-algebra packages), but the &lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/ug.pdf&quot; rel=&quot;nofollow&quot;&gt;user-guide&lt;/a&gt; makes me believe, that this should be done through the &lt;code&gt;options-parameter&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Fact&#xA;  Specifies whether or not the factored form of the matrix A is supplied on entry, and if not,&#xA;  how the matrix A will be factorized base on the previous history, such as factor from scratch,&#xA;  reuse Pc and/or Pr, or reuse the data structures of L and U. fact can be one of:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– DOFACT: the matrix A will be factorized from scratch.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– SamePattern: the matrix A will be factorized assuming that a factorization of a matrix&#xA;  with the same sparsity pattern was performed prior to this one. Therefore, this&#xA;  factorization will reuse column permutation vector perm c.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– SampPattern SameRowPerm: the matrix A will be factorized assuming that a factorization&#xA;  of a matrix with the same sparsity pattern and similar numerical values was performed&#xA;  prior to this one. Therefore, this factorization will reuse both row and column&#xA;  permutation vectors perm r and perm c, both row and column scaling factors Dr and&#xA;  Dc, and the distributed data structure set up from the previous symbolic factorization&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As an alternative, there is also the following within &lt;code&gt;options&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ColPerm&#xA;  Specifies how to permute the columns of the matrix for sparsity preservation.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– NATURAL: natural ordering.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– MMD ATA: minimum degree ordering on the structure of AT A.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– MMD AT PLUS A: minimum degree ordering on the structure of AT + A.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– COLAMD: approximate minimum degree column ordering&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;– MY PERMC: use the ordering given in perm c input by the user.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h2&gt;Questions&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Which part of SuperLU should i use to achieve my goal? One of the above? Something else?&lt;/li&gt;&#xA;&lt;li&gt;How can it be done within scipy?&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;code&gt;ColPerm&lt;/code&gt;-parameter is missing &lt;code&gt;MY_PERMC&lt;/code&gt; in scipy's &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.splu.html#scipy.sparse.linalg.splu&quot; rel=&quot;nofollow&quot;&gt;docs&lt;/a&gt; (maybe it's not usable)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17801" LastEditorUserId="17801" LastEditDate="2016-09-05T13:39:55.157" LastActivityDate="2017-11-22T17:59:24.430" Title="How to reuse permutation-orderings within scipy's SuperLU-wrapper?" Tags="|python|sparse-matrix|scipy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24919" PostTypeId="2" ParentId="24914" CreationDate="2016-09-05T14:14:41.037" Score="4" Body="&lt;p&gt;You are looking for a Newton's method, as opposed to Quasi-Newton used by SNOPT.  You can consider using KNITRO, which can handle any optimization problem type that SNOPT can, but has a greater number of algorithmic options.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among the algorithmic options in KNITRO is Sequential Quadratic Programming (SQP). SQP is used by SNOPT, but only in conjunction with Quasi-Newton. KNITRO can run any of its algorithmic options (SQP and others) with Newton's method, which makes use of user provided Hessian of the Lagrangian, or with Quasi-Newton.  If you wish to use Newton's method and have nonlinear constraints, you will need to not only evaluate the Hessian of the objective function, but also the Hessians of the nonlinear constraints.  For Quasi-Newton, KNITRO provides choice of BFGS or SR1, with SR1 working better than BFGS for some non-convex problems, whereas SNOPT only provides BFGS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;KNITRO also provides interior point algorithm choices which may or may not be better in your problem than SQP.Also of note, KNITRO&quot;s SQP uses trust regions, whereas SNOPT uses line searches. This &quot;allows&quot; use of SR1 as an option in KNITRO, but not in SNOPT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no native FORTRAN interface to KNITRO, but you can call it by use of C wrappers.  See &lt;a href=&quot;https://www.artelys.com/tools/knitro_doc/2_userGuide/otherProgInterfaces.html&quot; rel=&quot;nofollow&quot;&gt;https://www.artelys.com/tools/knitro_doc/2_userGuide/otherProgInterfaces.html&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An open source alternative which is less flexible and may be less robust than KNITRO is IPOPT, which can also be called from FORTRAN via C wrapper &lt;a href=&quot;http://www.coin-or.org/Ipopt/documentation/node25.html&quot; rel=&quot;nofollow&quot;&gt;http://www.coin-or.org/Ipopt/documentation/node25.html&lt;/a&gt; .  It only offers interior point options, with no SQP option. But it does offer choice of Newton (user-provided Hessian) or Quasi-Newton.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2016-09-05T14:34:43.857" LastActivityDate="2016-09-05T14:34:43.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24920" PostTypeId="2" ParentId="24409" CreationDate="2016-09-05T16:50:53.063" Score="0" Body="&lt;p&gt;I think a high level discussion of the terms is more necessary here than a calculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your result is correct. The truncation error is the rough amount of error per step. One equivalent way of defining it is &quot;if I had the exact solution at time $t$, how far off will I be from the exact solution at time $t+\Delta t$?&quot;. The actual amount changes each time due to differences in values/derivatives, but its rough value (the order of magnitude, the order of error), stays about the same. Thus you can think of the local truncation error as &quot;the amount of error I am introducing in any step&quot;, meaning the step you choose the calculate it at is arbitrary (assuming that your function is sufficiently smooth).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might as then, what does this mean for the amount of error I truly have? The error at $t+\Delta t$ is obviously larger than at $t$. For this idea we have the global error. A truly astonishing theorem is the Lax-Equivalence Theorem which says that, when you have stability, the global error is directly related to the local truncation error, giving you a way to know what your total error will be by knowing your truncation error estimates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in conclusion: local truncation error is a rough estimate for the error introduced per step, while the global error is the error of the approximation at a given point.  &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-09-05T16:50:53.063" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24921" PostTypeId="2" ParentId="24913" CreationDate="2016-09-05T17:04:56.533" Score="0" Body="&lt;p&gt;A mathematical expression which shows that a scheme is not conservative is usually an inequality. For example, many methods for the Advection Equation (and generalizations) do not conserve energy (this is known as numerical dissipation). This is usually shown in a theorem like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $u_n$ being the approximation of $u(n\Delta t)$, for method M we have that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Vert u_{n+1} \Vert_{2} &amp;lt; \Vert u_{n} \Vert_{2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this kind of equation, energy is measured as the $L^2$ norm. Real solutions are travelling waves which don't dissipate energy. However, the numerical approximation adds a little bit of diffusion to the wave, and so the diffusing wave loses energy the same way the diffusion equation does (indeed, you can show that the numerical method actually solves a diffusion+advection equation where the diffusion constant is related to the local truncation error). The previous statement captures this fact by basically saying that the peak of the numerical wave is guaranteed to dissipate at least a little bit at each time step.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-09-05T17:04:56.533" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24922" PostTypeId="2" ParentId="24892" CreationDate="2016-09-05T17:08:28.863" Score="1" Body="&lt;p&gt;I don't know if there is one source. Mostly just blogs and manufacturers show this kind of thing. You may want to start a blog on this! The &lt;a href=&quot;https://www.pugetsystems.com/all_hpc.php&quot; rel=&quot;nofollow&quot;&gt;Puget Systems HPC Blog&lt;/a&gt; is probably the most complete source, featuring a lot of benchmarks on Xeon processors, NVIDIA GPUs, and Xeon Phi acceleration cards. It doesn't only focus on BLAS, but the applications are all scientific computing and so it's a good indicator of performance for many people's algorithms (i.e. it uses a lot of BLAS).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-09-05T17:08:28.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24923" PostTypeId="2" ParentId="24140" CreationDate="2016-09-05T17:26:37.207" Score="0" Body="&lt;p&gt;You may want to look into using Julia. Using Julia on multiple nodes is quite trivial using the machine files (which I document &lt;a href=&quot;http://www.stochasticlifestyle.com/multi-node-parallelism-in-julia-on-an-hpc/&quot; rel=&quot;nofollow&quot;&gt;in a blog post&lt;/a&gt;, but essentially you just pass in the list of nodes you're using and now your parallel commands will work on all of them). It's also very easy to get it to work with &lt;a href=&quot;http://www.stochasticlifestyle.com/julia-on-the-hpc-with-gpus/&quot; rel=&quot;nofollow&quot;&gt;GPUs&lt;/a&gt; (which you can do separately on each machine using the parallel functions from the first post) and &lt;a href=&quot;http://www.stochasticlifestyle.com/multiple-gpu-on-the-hpc-with-julia/&quot; rel=&quot;nofollow&quot;&gt;multiple GPUs per node&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, Julia has &lt;a href=&quot;https://github.com/JuliaParallel/DistributedArrays.jl&quot; rel=&quot;nofollow&quot;&gt;Distributed Arrays&lt;/a&gt; which are arrays where only parts of the array are on a single node. Then you can use Julia's fancy iteration syntax &lt;code&gt;for i in eachindex(A)&lt;/code&gt; and it will only iterate over the indices which are contained on the node, for each node. As long as you enforce type-stability it will compile to the same assembly as C/Fortran code (which you can check with &lt;code&gt;@code_llvm&lt;/code&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also work being done to mix Julia's threads with multiprocesses: 1 process per node and the parallel commands automatically parallelize to each node, and each node threads over its cores. This is still experimental, but since it's all written in Julia, if you know Julia then it's easy to contribute (or make a package for it).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-09-05T17:26:37.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24924" PostTypeId="2" ParentId="24908" CreationDate="2016-09-05T21:18:51.000" Score="9" Body="&lt;p&gt;The question is ill-posed -- you don't say what you mean by &quot;desirable&quot; or &quot;superior&quot;. If a phenomenon I try to model involves only real numbers, then clearly using real matrices is the better way. On the other hand, if I try to model something in the Fourier domain, then complex numbers appear and I would try to use a matrix with complex elements, rather than split the whole thing into two matrices. Finally, if I tried to write the fastest code for matrix-matrix multiplication, I may actually try to store the real and imaginary parts separately. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it always comes down to what exactly you're trying to do -- there is no answer so general that it is always true.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-09-07T20:27:26.370" LastActivityDate="2016-09-07T20:27:26.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24925" PostTypeId="2" ParentId="24916" CreationDate="2016-09-05T22:51:53.997" Score="5" Body="&lt;p&gt;It is not a problem if one processor generates a number that has already appeared on a different processor. It would, however, be a problem if the two generated whole &lt;em&gt;sequences&lt;/em&gt; that are similar. This is unlikely, if you start with different random seeds, since the number of numbers you get from a RNG before it starts to repeat itself (the cycle length) is so enormously large that randomly chosen starting points are very unlikely to lead to overlapping sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you wanted to avoid this for sure, then why not the following procedure: let's assume you have two processors, then let every processor generate the same sequence from the same seed, and let processor zero only take the 0th, 2nd, 4th, ... element of this sequence, and processor one the 1st, 3rd, 5th, ... element. If you have $P$ processors, let the $p$th processor take element $x_i$ of the random number sequence if $i = p \mod P$. This guarantees a cycle length for each process that is maximal, and you will for sure not get any overlapping sequences. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-09-05T22:51:53.997" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="24926" PostTypeId="2" ParentId="24913" CreationDate="2016-09-06T03:30:06.383" Score="2" Body="&lt;p&gt;Most pdes coming from physics have a divergence structure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t + \nabla\cdot F =0 \qquad \textrm{in} \quad \Omega&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then for any arbitrary control volume $D \subset \Omega$, we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d}{dt}\int_D u dx + \oint_{\partial D} F \cdot n ds = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e., the total quantity inside $D$ changes due to fluxes on the boundary of $D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A numerical method that has the same property would be called conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here one can distinguish between locally conservative and globally conservative methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A locally conservative method would satisfy this integral equation on every cell in the mesh. Examples are finite volume methods and DG methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A locally conservative method is also globally conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuous Galerkin methods may not be locally conservative but are globally conservative. You can see this if you apply a CG method to heat equation with Neumann boundary conditions. Taking the unit test function, you will see that the integral equation is satisfied if $D$ = the whole domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite volume method: If $F=(f,g)$ then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Delta x \Delta y \frac{d u_{i,j}}{dt} + [f_{i+1/2,j}-f_{i-1/2,j}] \Delta y + [g_{i,j+1/2}-g_{i,j-1/2}]\Delta x = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a discrete analogue of the conservation law.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DGM: Let $V_h$ be space of discontinuous, piecewise polynomials. The DG scheme can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d}{dt}(u_h, \phi_h) - (F(u_h), \nabla\phi_h) + \sum_e \int_e \hat{F}(u_h^-,u_h^+,n) \phi_h ds + \sum_{e\in\partial\Omega}\int_e \hat{F}_b \phi_h ds = 0 \qquad \forall \phi_h \in V_h&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\hat{F}$, $\hat{F}_b$ are numerical flux functions for interior and boundary faces. If $K$ is a cell in the mesh (for simplicity, assume it is away from the boundary), let us take the test function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\phi_h = \begin{cases}&#xA;1 &amp;amp; (x,y) \in K \\&#xA;0 &amp;amp; \textrm{otherwise}&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is in $V_h$. The DGM scheme gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d}{dt}\int_K u_h dx + \sum_{e \in \partial K}\int_e \hat{F}(u_h^-,u_h^+,n) ds =0&#xA;$$&#xA;which is a discrete approximation to the conservation law.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuous Galerkin (CG) method: Let $W_h$ be space of continuous piecewise polynomials. The CG scheme is (we are not concerned with how good this scheme is)&#xA;$$&#xA;\frac{d}{dt}(u_h, \phi_h) - (F(u_h), \nabla\phi_h) + \sum_{e\in\partial\Omega}\int_e \hat{F}_b \phi_h ds = 0 \qquad \forall \phi_h \in W_h&#xA;$$&#xA;We cannot take a test function with support inside one single cell as we did in case of DG scheme. We can take&#xA;$$&#xA;\phi(x,y)=1 \qquad (x,y) \in \Omega&#xA;$$&#xA;and the CG scheme becomes&#xA;$$&#xA;\frac{d}{dt} \int_\Omega u_h dx + \sum_{e\in\partial\Omega}\int_e \hat{F}_b ds = 0 &#xA;$$&#xA;which shows global conservation property.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2016-09-09T07:51:24.637" LastActivityDate="2016-09-09T07:51:24.637" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24927" PostTypeId="1" CreationDate="2016-09-06T03:37:53.553" Score="0" ViewCount="1473" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hfh77.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hfh77.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a system of two bodies, $b^i$ and $b^j$, each in position $\vec{p}^i = (x^i, y^i)$ and $\vec{p}^j = (x^j, y^j)$. The two are connected via spring and I'd like to know, given their states at time $t_n$, their states at time $t_{n+1}$ ($\vec{p}^i_{n+1}$ and $\vec{p}^j_{n+1}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have decided to use the backward Euler method since my spring is going to be stiff. However, I am not sure how to implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am implementing it the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec{p}^i_{{n+1}} = \vec{p}^i_{n} + hf(t_n+h, \vec{p}^i_{n+1}, \vec{p}^j_{n})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec{p}^j_{{n+1}} = \vec{p}^j_{n} + hf(t_n+h, \vec{p}^j_{n+1}, \vec{p}^i_{n})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, I am using the &lt;strong&gt;other&lt;/strong&gt; body's state at each backward Euler step. However, my system won't stabilize sometimes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I supposed to, instead, build a matrix $P$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ P = \begin{bmatrix}&#xA;    x^i &amp;amp; y^i \\&#xA;    x^j &amp;amp; y^j&#xA;\end{bmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;an then use it to build a backward Euler step of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ P_{n+1} = P_n + hf(t_{n}+h, P_{n+1}) $?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, how would I implement Newton's method for this matrix?&#xA;to use Newton's method for a vector of size 2, I build a 2x2 Jacobian matrix, but how do I do it for a matrix of 2x2, do I make some sort of 4x4 Jacobian matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To further elaborate on my function f&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(t_n, \vec{p}^i_{n}, \vec{p}^j_{n}) = K \times (\vec{p}^j_n - \vec{p}^i_n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K$ is an attraction coefficient which is user adjustable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in this case, f requires both the position of $b^i$ and $b^j$ to evaluate a displacement for each of the bodies&lt;/p&gt;&#xA;" OwnerUserId="19906" LastEditorUserId="19906" LastEditDate="2016-09-06T06:16:46.280" LastActivityDate="2016-11-06T04:36:04.293" Title="How to properly implement Backwards Euler on a system of bodies" Tags="|ode|stiffness|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="24928" PostTypeId="2" ParentId="24409" CreationDate="2016-09-06T04:12:52.953" Score="0" Body="&lt;p&gt;Suppose $v$ is exact solution and $u$ is numerical solution. You have a scheme&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;L_j(u^{i+1},u^i) := \frac{u_{j}^{i+1}-u_{j}^{i}}{\delta t} - \frac{u_{j-1}^{i}-2u_{j}^{i}+u_{j+1}^{i}}{\delta x^2} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in general&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;L_j(v^{i+1},v^i) \ne 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we define the above quantity as the &quot;local truncation error&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tau_j^i := L_j(v^{i+1},v^i)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the exact solution $v$ is sufficiently smooth, you can do a Taylor expansion in space and time and show that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tau_j^i := L_j(v^{i+1},v^i) = O(\delta t) + O(\delta x^2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\tau_j^i \to 0$ as $\delta t, \delta x \to 0$, we say that the scheme is consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subtracting the two equations, we get&#xA;$$&#xA;L_j(v^{i+1},v^i) - L_j(u^{i+1},u^i) = L_j(e^{i+1},e^i) = \tau_j^i&#xA;$$&#xA;where &#xA;$$&#xA;e = v - u&#xA;$$&#xA;is the error. The error satisfies the scheme but with a rhs equal to the local truncation error. In one time step, the error changes according to&#xA;$$&#xA;e_j^{i+1} = e_j^i + \frac{\delta t}{\delta x^2}(e_{j-1}^{i}-2e_{j}^{i}+e_{j+1}^{i}) + \delta t \cdot \tau_j^i&#xA;$$&#xA;If further you have some stability, you can use the above error equation and estimate the error in terms of $\delta t, \delta x$ and some other data in the problem.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2016-09-06T04:12:52.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24929" PostTypeId="2" ParentId="24927" CreationDate="2016-09-06T09:00:20.473" Score="1" Body="&lt;p&gt;I assume &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p=[p^i;p^j]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p_i=[x^i, y^i]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p_j=[x^j, y^j]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formula for backward Euler is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p^{n+1}=p^n+h*f(p^{n+1})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here $f_i=K*(p_i-p_j)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_j=K*(p_j-p_i)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting this in backward Euler's formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[p_i^{n+1};p_j^{n+1}] =[p_i;p_j]^n+h*K*[p_i^{n+1}-p^n_j;p_j^{n+1}-p^n_i]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implicated only some part of second term in RHS to enhance stability!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using matrix assoative property, we can write&#xA;$$[p_i^{n+1};p_j^{n+1}] =[p_i;p_j]^n+h*K*[p_i^{n+1};p_j^{n+1}]-h*K*[p_j^{n};p_i^{n}]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[p_i^{n+1};p_j^{n+1}](1-Kh)=[p_i;p_j]^n-Kh*[p_j^{n};p_i^{n}]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This form looks like &lt;a href=&quot;https://en.wikipedia.org/wiki/Explicit_and_implicit_methods&quot; rel=&quot;nofollow&quot;&gt;explicit schemes&lt;/a&gt; and easy to implement. You can directly use this scheme using matrix libraries available in programming languages. For obtaining $p^{n+1} =A*g(p^n)$ foam, you can use symbolic languages like sympy, matlab, mathematica, maple etc, Other wise you can use linear or non-linear root-finding method to solve $p^{n+1}=p^n+h*f(p^{n+1})$ equation iteratively. This is why implicit methods are computationally expensive than explicit method. If we do some little algebra like what we did, you can save computational cost as well as easy to parallelize the code.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Limitations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It is a first order scheme may deviate from actual solution.&lt;/li&gt;&#xA;&lt;li&gt;Need fine mesh to get accurate solution.&lt;/li&gt;&#xA;&lt;li&gt;You may consider higher order time integration.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note:&#xA;Actual spring mass system we balance inertial force by spring force and inertial force is second order term, for that you can split the second order equation into two first order equation and solve it. I'm not sure whether it is completely implicit or semi-implicit scheme. Semi-implicit scheme is conditionally stable for some problems. &lt;/p&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="16930" LastEditDate="2016-09-07T04:09:28.683" LastActivityDate="2016-09-07T04:09:28.683" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24930" PostTypeId="1" AcceptedAnswerId="24956" CreationDate="2016-09-06T13:12:20.687" Score="2" ViewCount="666" Body="&lt;p&gt;I am looking for the subject. The size of matrices will be around 1000x2000 elements with linear amount of ones (say, 6000 ones in the whole matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The operations I will use the most:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;iterating over elements of particular column;&lt;/li&gt;&#xA;&lt;li&gt;iterating over elements of particular row;&lt;/li&gt;&#xA;&lt;li&gt;getting submatrix by columns;&lt;/li&gt;&#xA;&lt;li&gt;solving undetermined linear system (over Galois field $\mathbb F_2$).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Before starting to implement it myself I want to ask the community about the existing libraries. Also if you have some ideas on implementation - feel free to share.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Memory usage is not of much importance for me.&lt;/p&gt;&#xA;" OwnerUserId="21609" LastActivityDate="2016-10-11T07:38:15.960" Title="Fast C++ implementation of sparse binary matrices" Tags="|matrix|c++|sparse-matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24931" PostTypeId="1" AcceptedAnswerId="24944" CreationDate="2016-09-06T13:16:35.697" Score="0" ViewCount="147" Body="&lt;p&gt;In a previous question, &lt;a href=&quot;https://scicomp.stackexchange.com/q/24829/21472&quot;&gt;link&lt;/a&gt;, I asked about how I could most effectively do a Fourier Transform of a radial function given at certain values and which we knew the asymptotical behaviour of. The Fourier transform reading&#xA;$$\frac{4\pi}{q}\int^{\infty}_0 dr\, r \sin(qr) f(r).$$&#xA;I tried several ways and ended up choosing FFT, which approximates it by calculating the DFT over the interval $[0,\Delta r]$ with $N$ points. (I used more particularly the NAG subroutine &lt;code&gt;C06FAF&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still get some issue around $q=0$. Indeed, as can be seen on the figure &lt;a href=&quot;https://i.stack.imgur.com/0w7K2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0w7K2.jpg&quot; alt=&quot;this picture&quot;&gt;&lt;/a&gt;, I have some weird peak at very low frequencies. The black curve that is flat in &lt;code&gt;q=0&lt;/code&gt; is the analytical result while other curves are FFT calculations with increasing $\Delta r$. ($\color{blue}{\Delta r} &amp;gt;\color{red}{\Delta r} &amp;gt; {\bf \color{black}{\Delta r}} &amp;gt; \color{magenta}{\Delta r} $ and the dashed one being the highest $\Delta r$). As can be seen, this peak is narrower when $\Delta r$ gets larger. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is from where this peak does come ? And how could I get rid of it ? I already tried to take the mean of the function and substract it to the function but it does not change anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One subsidiary question is also the following : the subroutine calculates the integral. I then have to divide by $q$ to get this $4\pi/q$ factor. Though, at $q=0$, this can't be done for obvious reasons. So, what can be done instead ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT : the problem of the wide peak was simply due to the fact that I was doing a bad conversion between $q$ and the frequency from the routine. As far as the problem of the division by $q=0$ is concerned, I'm happy with &lt;code&gt;Endulum&lt;/code&gt;'s answer.&lt;/p&gt;&#xA;" OwnerUserId="21472" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-09-09T12:12:19.090" Title="0 Hz (quite sharp) peak in FFT and division by 0" Tags="|fortran|fourier-analysis|fourier-transform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24933" PostTypeId="1" AcceptedAnswerId="24934" CreationDate="2016-09-06T15:39:41.750" Score="0" ViewCount="237" Body="&lt;p&gt;I am trying to use CSDP and am struggling with this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider for example the SDP problem proposed by prof. Borchers &lt;a href=&quot;https://scicomp.stackexchange.com/a/24821/479&quot;&gt;here&lt;/a&gt;. Namely:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{A,z} \sum_{i} z_{i}\quad\text{subject to}\quad\mbox{tr}(P_{i}A) \geq z_{i},\quad i=1, 2, \ldots, n.\quad A \succeq 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering how one represents it in the &lt;a href=&quot;https://cran.r-project.org/web/packages/Rcsdp/Rcsdp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;standard CSDP format&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{X} \mbox{Tr}(CX)\quad\text{subject to}\quad\mbox{tr}(A_{i}X)= b_{i},\quad i=1, 2, \ldots, n.\quad X \succeq 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, I can see how to specify the $A_i$'s. My problem is how to express the $\sum_i z_i$ in the objective function as $\mbox{Tr}(CX)$. &lt;/p&gt;&#xA;" OwnerUserId="479" LastEditorUserId="276" LastEditDate="2017-10-10T00:36:25.493" LastActivityDate="2017-10-10T00:36:25.493" Title="Express SDP problem in CSDP" Tags="|semidefinite-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24934" PostTypeId="2" ParentId="24933" CreationDate="2016-09-06T16:54:18.230" Score="2" Body="&lt;p&gt;Inequality constraints in an SDP can be turned into equality constraints by introducing non-negative slack variables.  If your original problem is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \Sigma_{i=1}^{n} z_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(P_{i}X) \geq z_{i}\;\;$  $i=1, 2, \ldots, n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ X \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can rewrite the constraints as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(P_{i}X) - z_{i} \geq 0\;\;$  $i=1, 2, \ldots, n.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then introduce slack variables $w_{i}$, $i=1, 2, \ldots, n$ and &#xA;write the constraints as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(P_{i}X) - z_{i} -w_{i} = 0\;\;$  $i=1, 2, \ldots, n.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $w_{i} \geq 0$, $i=1, 2, \ldots, n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll assume for this answer that the $z_{i}$ variables are also nonnegative.  If they're free to be negative, you could handle this by &#xA;writing $z_{i}=r_{i}-s_{i}$, where $r, s \geq 0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $Z=\mbox{diag}(z)$ and $W=\mbox{diag}(w)$.  Let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V=\left[&#xA;\begin{array}{ccc}&#xA;X  &amp;amp;  0  &amp;amp; 0 \\&#xA;0  &amp;amp;  Z  &amp;amp; 0 \\&#xA;0  &amp;amp;  0  &amp;amp; W \\&#xA;\end{array}&#xA;\right]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C=\left[&#xA;\begin{array}{ccc}&#xA;0  &amp;amp;  0  &amp;amp; 0 \\&#xA;0  &amp;amp;  I  &amp;amp; 0 \\&#xA;0  &amp;amp;  0  &amp;amp; 0 \\&#xA;\end{array}&#xA;\right]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $\mbox{tr}(CV)=\sum_{i=1}^{n} z_{i}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{i}=\left[&#xA;\begin{array}{ccc}&#xA;P_{i} &amp;amp;  0  &amp;amp; 0 \\&#xA;0  &amp;amp;  -E_{i,i}  &amp;amp; 0 \\&#xA;0  &amp;amp;  0  &amp;amp; -E_{i,i} \\&#xA;\end{array}&#xA;\right]\;\;$ $i=1, 2, \ldots, n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $E_{i,i}$ is the 0 matrix with a single $1$ in the $(i,i)$ position.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the original problem can be written in standard form as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \mbox{tr}(CV)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(A_{i}V)=b_{i}\;\;$ $i=1, 2, \ldots, n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V \succeq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that $V$ is a block diagonal matrix, and that furthermore, $Z$ and $W$ are diagonal blocks.  In the SDPA sparse matrix format it is easy to &#xA;encode $X$ as an $n$ by $n$ positive semidefinite block and $Z$ and $W$ as $n$ by $1$ vectors of non-negative variables.  The additional storage required for $Z$ and $W$ is minimal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraint $V \succeq 0$ ensures that all of the blocks on the diagonal of $V$ are positive semidefinite.  In particular, $X \succeq 0$, &#xA;$z \geq 0$, and $w \geq 0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that you probably have additional linear constraints on $X$ and $z$.  These can easily be added to the formulation.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-09-06T16:54:18.230" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24935" PostTypeId="2" ParentId="24916" CreationDate="2016-09-06T17:57:00.957" Score="1" Body="&lt;p&gt;This exact problem (not generating the same random number sequences) is described in John D. Cook's blog post &lt;a href=&quot;http://www.johndcook.com/blog/2016/01/29/random-number-generator-seed-mistakes/&quot; rel=&quot;nofollow&quot;&gt;Random number generator seed mistakes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is simply to ensure that processes are started with an unique seed. Which you also mention as a possibility in your question. I think your figure might be a bit misleading as most random number generators will not produces random sequences that overlap in the way shown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From John's post:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this case it would have been better to seed each process with &lt;em&gt;sequential&lt;/em&gt; seeds: give the first process seed 1, the second seed 2, etc. The &lt;em&gt;seeds&lt;/em&gt; don’t have to be random; they just have to be unique. If you’re using a good random number generator, the outputs of 1,000 processes seeded with 1, 2, 3, ..., 1000 will be independent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="21614" LastActivityDate="2016-09-06T17:57:00.957" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24937" PostTypeId="1" AcceptedAnswerId="24942" CreationDate="2016-09-06T22:35:59.857" Score="9" ViewCount="874" Body="&lt;p&gt;I have been writing a control system toolbox from scratch and purely in Python3 (shameless plug : &lt;a href=&quot;https://github.com/ilayn/harold&quot;&gt;&lt;code&gt;harold&lt;/code&gt;&lt;/a&gt; ). From my past research, I have always complaints about the Riccati solver &lt;code&gt;care.m&lt;/code&gt; for reasons that are technical/irrelevant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, I've been writing my own set of routines. One thing I can't find a way around is to obtain a high-performance balancing algorithm, at least as good as &lt;code&gt;balance.m&lt;/code&gt;. Before you mention it, &lt;code&gt;xGEBAL&lt;/code&gt; family is exposed in Scipy and you can basically call from Scipy as follows, suppose you have a float type 2D array &lt;code&gt;A&lt;/code&gt;: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy as sp&#xA;gebal = sp.linalg.get_lapack_funcs(('gebal'),(A,)) # this picks up DGEBAL&#xA;Ab, lo, hi, scaling , info = gebal(A, scale=1 , permute=1 , overwrite_a=0 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now if I use the following test matrix &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([[ 6.      ,  0.      ,  0.      ,  0.      ,  0.000002],&#xA;       [ 0.      ,  8.      ,  0.      ,  0.      ,  0.      ],&#xA;       [ 2.      ,  2.      ,  6.      ,  0.      ,  0.      ],&#xA;       [ 2.      ,  2.      ,  0.      ,  8.      ,  0.      ],&#xA;       [ 0.      ,  0.      ,  0.000002,  0.      ,  2.      ]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array([[ 8.      ,  0.      ,  0.      ,  2.      ,  2.      ],&#xA;       [ 0.      ,  2.      ,  0.000002,  0.      ,  0.      ],&#xA;       [ 0.      ,  0.      ,  6.      ,  2.      ,  2.      ],&#xA;       [ 0.      ,  0.000002,  0.      ,  6.      ,  0.      ],&#xA;       [ 0.      ,  0.      ,  0.      ,  0.      ,  8.      ]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, if I pass this to &lt;code&gt;balance.m&lt;/code&gt;, I get  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; balance(A)&#xA;&#xA;ans =&#xA;&#xA;    8.0000         0         0    0.0625    2.0000&#xA;         0    2.0000    0.0001         0         0&#xA;         0         0    6.0000    0.0002    0.0078&#xA;         0    0.0003         0    6.0000         0&#xA;         0         0         0         0    8.0000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you check the permutation patterns, they are the same however the scaling is off. &lt;code&gt;gebal&lt;/code&gt; gives unity scalings whereas matlab gives the following powers of 2: &lt;code&gt;[-5,0,8,0,2]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So apparently, these are not using the same machinery. I've tried various options such as Lemonnier, Van Dooren two sided scaling, original Parlett-Reinsch and also some other less-known methods in the literature such as the dense version of &lt;code&gt;SPBALANCE&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One point maybe I might emphasize is that I am aware of Benner's work; in particular the &lt;em&gt;Symplectic Balancing of Hamiltonian Matrices&lt;/em&gt; specifically for this purpose. However, note that this type of treatment is done within &lt;code&gt;gcare.m&lt;/code&gt; (generalized Riccati solver) and balancing is done directly via &lt;code&gt;balance.m&lt;/code&gt;. Hence, I would appreciate if someone can point me to the actual implementation. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I am really not trying to reverse engineer mathworks code: I actually want to get away from it due to various reasons including the motivation of this question, that is to say, I don't know what it is doing which costed me lots of time back in the day. My intention is to get a satisfactory balancing algorithm that allows me to pass CAREX examples such that I can implement Newton iteration methods on top of the regular solver.&lt;/p&gt;&#xA;" OwnerUserId="21617" LastEditorUserId="21617" LastEditDate="2016-09-06T22:48:40.177" LastActivityDate="2016-10-04T10:00:41.380" Title="Matrix Balancing Algorithm" Tags="|linear-algebra|matlab|lapack|scipy|matrix-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24938" PostTypeId="2" ParentId="24911" CreationDate="2016-09-07T05:20:04.643" Score="2" Body="&lt;p&gt;&lt;strong&gt;Common Ghost cell treatment in CFD&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ghost cell treatment is an art in CFD, we can't arbitrarily choose ghost cell for all the problems. Let's assume $0$ is boundary point and $-1$ is ghost point and $u$ is variable then commonly used ghost cell procedure is, u(-1) =u(0), this is some sense equivalent $\frac{du}{dx}=0$ another procedure is u(-1) = -u(0), this kind of procedure is followed if we want normal velocity is zero at wall ($v_n=0$) commonly used in Euler equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I'm restricting myself to 1D heat equation. From the physics behind it, we can extend that to other elliptic equations.&#xA;All the above B.C described may fail for heat equation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If we impose u(-1) =u(0) that is equivalent to insulated wall B.C, but we not sure whether the wall is insulated or not. Maintaining same temperature at two points without B.C is non-physical&lt;/li&gt;&#xA;&lt;li&gt;If we use u(-1) = -u(0), this also violate physics, lets say u(0)=1000 K, u(-1)=-1000 K. Is it physical? -1000 K doesn't exist. Even if exist we never allow this much jump in numerical scheme unless there is shock. Since it is elliptic solution, expected to be smooth.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Steady state 1-D Heat equation is&#xA;$$\frac{d^2u}{dx^2} = 0$$&#xA;$2^{nd}$ order FD eqivalent is&#xA;$$\frac{d^2u}{dx^2} = (u_{i-1}-2u_i+u_{i+1})/h^2 =0$$ &#xA;then $$u_{i-1}=2*u_i-u_{i+1}$$&#xA;for i=0 and nx this become here $-1$ and $nx+1$ are ghost cell points&#xA; $$u_{-1}=2*u_0-u_{1}$$&#xA; $$u_{nx+1}=2*u_{nx}-u_{nx-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;these equations can be used for ghost cell points, please note that these equations are second order accurate and may have impact on solution.Because interior points are fouth-order scheme. Here I have attached a sample matlab code, you can play with it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc&#xA;clear all&#xA;close all&#xA;nx=101;&#xA;x=linspace(0,1,nx);&#xA;dx=1/(nx-1);&#xA;u=0*x; %I.C&#xA;u_res=u;&#xA;u(2)=10; %bc because x(1) is ghost cell&#xA;u(nx-1)=10; %bc because x(nx) is ghost cell&#xA;itmax=10000;&#xA;for j1=1:1:itmax&#xA;for i1=1:nx&#xA;    if i1==1;&#xA;        u(i1)=u(2)*2-u(i1+2);  % Ghost cell values are framed &#xA;                                    %from GDE in second order&#xA;    end&#xA;&#xA;    if i1==nx;&#xA;        u(i1)=u(nx-1)*2-u(nx-2);&#xA;    end&#xA;    if i1==2&#xA;        u(i1)=10;&#xA;    end&#xA;    if i1==nx-1&#xA;        u(i1)=100;&#xA;    end&#xA;    if i1&amp;gt;2 &amp;amp;&amp;amp; i1&amp;lt;nx-1&#xA;        u(i1)=(-u(i1-2)+16*u(i1-1)+16*u(i1+1)-u(i1+2))/30;       &#xA;    end&#xA;end&#xA;for i1=3:1:nx-2&#xA;    u_res(i1)=(-u(i1-2)+16*u(i1-1)+16*u(i1+1)-u(i1+2))/30-u(i1); %residue calc.&#xA;end&#xA;if max(abs(u_res))&amp;lt;0.0001 %convergence creteria&#xA;    break&#xA;end&#xA;&#xA;end&#xA;plot(x(2:nx-1),u(2:nx-1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I will be happy if you do following things&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You can check the effect of second order ghost cell treatment for this problem on convergence&lt;/li&gt;&#xA;&lt;li&gt;You can set one $4^{th}$ order biased ghost cell interpolation scehme inside the code and check the effect of it.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="16930" LastEditDate="2016-09-07T12:32:01.560" LastActivityDate="2016-09-07T12:32:01.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24939" PostTypeId="1" CreationDate="2016-09-05T21:59:20.960" Score="1" ViewCount="339" Body="&lt;p&gt;Let $f(x,y)$ and $g(x,y)$ be two $\mathbb{R}^2\rightarrow\mathbb{R}$ functions, both strictly increasing in both arguments. Assume that they are well-behaved functions (continuous, differentiable, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given these properties, I want to find a root to the following system of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x,y)=f_0$$&#xA;$$g(x,y)=g_0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;assuming that a solution exists and that it is unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an efficient algorithm to iteratively bracket the root in increasingly smaller regions $\mathbb{R}^2$? That is, I am looking for a derivative free method that is an extension of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bisection_method&quot; rel=&quot;nofollow&quot;&gt;bisection method&lt;/a&gt;, for 2 variables, with convergence guaranteed.&lt;/p&gt;&#xA;" OwnerUserId="988" OwnerDisplayName="becko" LastEditorUserId="988" LastEditDate="2016-09-07T13:55:56.590" LastActivityDate="2016-09-08T19:42:04.997" Title="Iterative root finding of 2-dimensional system of non-linear equations with monotonicity properties" Tags="|algorithms|roots|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24940" PostTypeId="1" AcceptedAnswerId="24962" CreationDate="2016-09-07T16:03:38.447" Score="3" ViewCount="215" Body="&lt;p&gt;I am going to discuss my reasons for wanting this first, as this may in fact not be what I am looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My reason for asking this that I have finished writing a piece of code that solves, $-\nabla \cdot(a \nabla u) = f$ where $a$ is discontinuous, this is the result of the code&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FPF6p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FPF6p.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we can see oscillations about the line of discontinuity, so I have been told that I have to enrich the approximation. After some manipulation, the meat of the problem is dealing with this integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Omega}a\nabla u \cdot \nabla v dx dy$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On enriching my approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x,y) = \sum_{i = 0}^N\sum_{j = 0}^Nu_{ij}h_i(x)h_j(y) + \sum_{k = 1}^N\sum_{l = 1}^N\alpha_{kl}\tilde{h_k(x)}\tilde{h_l(y)}\Phi(x,y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On substituting this into the integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Omega}a\nabla u \cdot \nabla v dx dy $$&#xA;$$\approx w_s\sum_{n = 0}^N \sum_{i = 0}^N w_n h_i'(x_n)h_r'(x_n)u_{is}, \forall s\in[0,N]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$+ w_r\sum_{m = 0}^N \sum_{j = 0}^N w_m h_j'(y_m)h_s'(y_m)u_{rj}, \forall r\in[0,N]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$+ \color{red}{ w_s\sum_{n = 1}^{2N + 2} a_1(x_n,y_s)\sum_{k = 1}^{N-1} w_n \tilde{h_k}'(x_n)h_r'(x_n)\alpha_{ks}\Phi_{1,x}(x_n,y_s), \forall s\in[1,N]}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$+ w_r\sum_{m = 0}^{2N + 2} a_1(x_r,y_m) \sum_{l = 0}^{N-1} w_m \tilde{h_l}'(y_m)h_s'(y_m)\alpha_{rl}\Phi_{1,y}(x_r,y_m), \forall r\in[1,N]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$+ \color{red}{w_s\sum_{n = 1}^{2N + 2} a_2(x_n,y_s)\sum_{k = 1}^{N-1} w_n \tilde{h_k}'(x_n)h_r'(x_n)\alpha_{ks}\Phi_{2,x}(x_n,y_s), \forall s\in[1,N]}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$+ w_r\sum_{m = 0}^{2N + 2} a_2(x_r,y_m) \sum_{l = 0}^{N-1} w_m \tilde{h_l}'(y_m)h_s'(y_m)\alpha_{rl}\Phi_{2,y}(x_r,y_m), \forall r\in[1,N] \textbf{(1)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/24470/21030&quot;&gt;How do I form the Chebyshev differentiation matrix in MATLAB?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first answer in this link deals with the definition of the $h_i'(x_j)$ functions, or the construction of the square Legendre pseudospectral differentiation matrix. The $\tilde{h_i}'(x_j)$ have a slightly different definition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$h_i'(x) = \frac{(1-x^2)(L_N'(x))}{N(N+1)L_N'(x_i)(x - x_i)}$$&#xA;$$\tilde{h_i}'(x) = \frac{(1-x_i^2)(L_N'(x))}{N(N+1)L_N'(x_i)(x - x_i)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The construction of the corresponding square Legendre pseudospectral differentiation matrix for each of the above is given in code as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N1=N+1;&#xA;cheb=cos(pi*(0:N)/N)';&#xA;unif=linspace(-1,1,N1)';&#xA;&#xA;if N&amp;lt;3&#xA;    x=cheb;&#xA;else&#xA;    x=cheb+sin(pi*unif)./(4*N);&#xA;end&#xA;&#xA;P=zeros(N1,N1);&#xA;%N1xN1 zero matrix&#xA;xold=2;&#xA;%eps= epsilon!&#xA;while max(abs(x-xold))&amp;gt;eps&#xA;&#xA;    xold=x;   &#xA;    P(:,1)=1;&#xA;    P(:,2)=x;&#xA;    %set first collumn entries to 1 (P(:,1) = 1);&#xA;    %set second collumn entries to x (P(:,2) = x);&#xA;    % and so on, the collumns of this matrix are the legendre polynomials&#xA;    for k=2:N&#xA;        P(:,k+1)=( (2*k-1)*x.*P(:,k)-(k-1)*P(:,k-1) )/k;&#xA;        %%Bonnets formula ;)&#xA;    end&#xA;    % Roots of (1-x^2)L'_N&#xA;    x=xold-( x.*P(:,N1)-P(:,N) )./( N1*P(:,N1));&#xA;end&#xA;&#xA;%---Legendre pseduspectral differentiation matrix------------page 124-----&#xA;% of the book &#xA;&#xA;x = -x;&#xA;D=zeros(N+1,N+1);&#xA;for k=1:N+1;&#xA;    for j=1:N+1;&#xA;        D(j,k)=P(j,N+1)/(P(k,N+1)*(x(j)-x(k)));&#xA;    end&#xA;    D(k,k)=0;    &#xA;end&#xA;D(1,1)=-N*(N+1)/4 ;           &#xA;D(N+1,N+1)=N*(N+1)/4; &#xA;&#xA;%------------------------------------------------------------------------&#xA;C1 = -(1-x(1:N1).^2);&#xA;C2 = N*N1.*P(:,N1);&#xA;C = C1+C2;&#xA;% L1 = P' L2 = P''...&#xA;% LN(i,N) is the Nth derivative of the Nth legendre polynomial at x(i)&#xA;&#xA;&#xA;% The last collumn in each is only really relevant.&#xA;%The coll's before will give a wierd result, this is easily fixed &#xA;%but we dont use them so....&#xA;&#xA;&#xA;L1 = zeros(N1,N1);&#xA;L2 = zeros(N1,N1);&#xA;L3 = zeros(N1,N1);&#xA;L1(:,1) = 0;&#xA;L1(:,2) = 1;&#xA;&#xA;for m = 2:N;&#xA;    for k = 1:N1;&#xA;        L1(k,m+1) = ((2*m + 1)*(P(k,m) + x(k)*L1(k,m)) - m*L1(k,m-1))/(m+1); &#xA;    end&#xA;end&#xA;L1(1,N1) = N*N1/2;&#xA;L1(N1,N1) = (N*N1/2)*(-1)^(N-1);&#xA;&#xA;L2(:,1) = 0;&#xA;L2(:,2) = 0; &#xA;for m = 2:N;&#xA;    for k = 1:N1;&#xA;        L2(k,m+1) = ((2*m + 1).*(2*L1(k,m) + x(k).*L2(k,m)) - m.*L2(k,m-1))/(m+1); &#xA;    end&#xA;end&#xA;L2(1,N1) = (N*N1 - 2)*(N*N1)^2/8;&#xA;L2(N1,N1) = (N*N1 - 2)*((N*N1)^2/8)*(-1)^N;&#xA;&#xA;L3(:,1) = 0;&#xA;L3(:,2) = 0;&#xA;for m = 2:N;&#xA;    for k = 1:N1;&#xA;        L3(k,m+1) = ((2*m + 1).*(3*L2(k,m) + x(k).*L3(k,m)) - m.*L3(k,m-1))/(m+1); &#xA;    end&#xA;end&#xA;&#xA;L3(1,N1) = (N*N1 - 6)*L2(1,N1)/6;&#xA;L3(N1,N1) = ((N*N1 - 6)*L2(1,N1)/6)*(-1)^N;&#xA;%this is the matrix that corresponds to the derivatives of h tilde.&#xA;DT = zeros(N1,N1);&#xA;for l = 2:N;&#xA;    for k = 2:N;&#xA;        if(k == l)&#xA;            DT(l,k) = C(k)*(L3(k,N1)/2);&#xA;        else    &#xA;            DT(l,k) = C(k)*(L2(l,N1)/(x(l) - x(k)));&#xA;        end&#xA;    end&#xA;end&#xA;for k = 2:N;&#xA;    DT(1,k) = C(1)*((1 - x(k))*(L2(1,N1) - L1(1,N1))/(1-x(k))^2); &#xA;    DT(N1,k) = C(N1)*((1 - x(k))*(L2(N1,N1) - L1(N1,N1))/(1-x(k))^2); &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The reason for the 4 extra terms is that I have split the region into two, either side of the discontinuity. According to my professor I have to use a technique called over-integration. I am going to need to define another D matrix such that is it a $(2N+2) \times (N+1)$ matrix and similarly for DT(ilde).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I form the non-square Legendre pseudospectral differentiation matrix?, if such a thing exists. If not, how do I deal with the over integration parts of the expression?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerDisplayName="user21030" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-09-13T12:40:07.557" Title="How to define a non-square Legendre pseudospectral differentiation matrix?" Tags="|numerical-analysis|spectral-method|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24941" PostTypeId="2" ParentId="24939" CreationDate="2016-09-08T00:23:27.600" Score="3" Body="&lt;p&gt;Because the functions are strictly increasing, this can be reduced to the one-dimensional case. In each one-dimensional problem there are more efficient &lt;em&gt;standard&lt;/em&gt; methods available, more efficient than bisection, and it also means one doesn't have to come up with a non-standard method oneself. After all, one-dimensional root-finding is a &quot;solved&quot; standard problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without loss of generality, assume $f_0=g_0=0$, and define $x_f=x_f(y)$ and $x_g=x_g(y)$ to be the solutions of $f(x_f(y), y) = 0$, and $g(x_g(y), y)=0$, respectively. Both $x_f, x_g$ are well-defined strictly decreasing functions of $y$, and can be evaluated numerically by one-dimensional root-finding, even bisection. Then the common root of $f=g=0$ corresponds to a solution of $x_f(y)-x_g(y) = 0$, which is again a one-dimensional problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial bracket has to be chosen appropriately so that the roots of each of the one-dimensional problems are bracketed correctly. Geometrically speaking, one can choose a rectangle with vertices $(x_-, y_-), (x_+, y_+)$ such that the curves $x=x_f(y)$ and $x=x_g(y)$ cross the rectangle at the $y=\mathrm{const}$ boundaries only (it's easier to draw this), and cross each other inside the rectangle (necessarily only once).&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-09-08T19:42:04.997" LastActivityDate="2016-09-08T19:42:04.997" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24942" PostTypeId="2" ParentId="24937" CreationDate="2016-09-08T13:24:56.917" Score="7" Body="&lt;p&gt;Took me quite a while to figure this out and as usual it becomes obvious &lt;strong&gt;after&lt;/strong&gt; you find the culprit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After checking the problematic cases reported in  &lt;a href=&quot;http://www.math.wsu.edu/faculty/watkins/pdfiles/balbad.pdf&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;David S. Watkins. A case where balancing is harmful. Electron. Trans. Numer. Anal, 23:1–4, 2006&lt;/em&gt;&lt;/a&gt;&#xA;and also the &lt;a href=&quot;http://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=13&amp;amp;t=4270&quot; rel=&quot;nofollow&quot;&gt;discussion here&lt;/a&gt; (both being cited in &lt;a href=&quot;http://arxiv.org/abs/1401.5766v1&quot; rel=&quot;nofollow&quot;&gt;arXiv:1401.5766v1&lt;/a&gt;), it turns out that matlab uses the balancing by separating the diagonal elements first. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial thought was, as per the classical limited documentation on LAPACK functions, GEBAL performed this automatically. However, I guess what the authors mean by &lt;strong&gt;ignoring the diagonal elements&lt;/strong&gt; is not removing them from the row/column sums. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, if I manually remove the diagonal from the array, then both results coincide, that is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy as sp&#xA;gebal = sp.linalg.get_lapack_funcs(('gebal'),(A,)) # this picks up DGEBAL&#xA;Ab, lo, hi, scaling , info = gebal(A - np.diag(np.diag(A)), scale=1 , permute=1 , overwrite_a=0 )  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives the same result as &lt;code&gt;balance.m&lt;/code&gt; (without the diagonal entries of course). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If any Fortran-savy user can confirm this by checking &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/dd/d04/dgebal_8f_source.html&quot; rel=&quot;nofollow&quot;&gt;dgebal.f&lt;/a&gt;, I would be grateful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; Above result does not imply that this is the only difference. I have constructed also different matrices where GEBAL and balance.m produces different results even after the diagonals are separated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am quite curious what the difference might be but it seems there is no way to know since it is a matlab built-in and hence closed code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT2&lt;/strong&gt; : It turns out matlab was using an older version of LAPACK (probably pre 3.5.0) and by 2016b they seem to be upgraded to the newer version. Now results agree as far as I can test. So I think that settles the issue. I should have tested it with older LAPACK versions. &lt;/p&gt;&#xA;" OwnerUserId="21617" LastEditorUserId="21617" LastEditDate="2016-10-04T10:00:41.380" LastActivityDate="2016-10-04T10:00:41.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24943" PostTypeId="2" ParentId="24829" CreationDate="2016-09-08T15:58:04.203" Score="1" Body="&lt;p&gt;I work with these types of transforms all the time. My tool of choice is not the full FFT, but rather a discrete Fourier sine transform (DST). Not all Fourier analysis packages have one, but it looks like NAG's &lt;a href=&quot;http://www.nag.com/numeric/fl/nagdoc_fl25/html/c06/c06raf.html&quot; rel=&quot;nofollow&quot;&gt;C06RAF&lt;/a&gt; will do the trick. The example code supplied by NAG should be a good starting point, but here's the outline I follow:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Make sure my $q$ and $r$ arrays satisfy the Nyquist condition $\Delta q \ge \frac{\pi}{r_{max}}$. (*see footnote)&lt;/li&gt;&#xA;&lt;li&gt;Call the DST routine on $r f(r)$.&lt;/li&gt;&#xA;&lt;li&gt;Scale the result. You will need a least a factor $\frac{\Delta r}{2q}$ and can determine the rest empirically. The $1/2$ is because your integral runs from $0$ to $\infty$, not $-\infty$ to $\infty$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;*There is factor of $2$ difference from the ordinary FFT condition. This is because your actual $r$ domain is $[0, r_{max}]$, but the DST treats the transform as though it were $[-r_{max}, r_{max}]$, so you get double the resolution in $q$ for free.&lt;/p&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2016-09-08T15:58:04.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24944" PostTypeId="2" ParentId="24931" CreationDate="2016-09-08T16:11:49.117" Score="2" Body="&lt;p&gt;I would need additional information about your attempt (e.g., example code) to help with your first question, but here's my input for the division-by-zero question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, do you &lt;em&gt;really&lt;/em&gt; need to divide by $q$? In the problems where these transforms come up, you can often get away with working with $rf(r)$ and $q\hat{f}(q)$ in their own right. Decide if you can put off dividing by $q$ until later on in the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At such a time that you actually do need $q=0$, you can use the fact that&#xA;$$\lim_{q\to 0} {\sin{(q r)} \over {q r}} = 1$$&#xA;for any finite $r$. Thus you have&#xA;$$\hat{f}(0)\approx \lim_{q\to 0} 4\pi\int_0^R dr~{\sin{(q r)} \over {q r}} r^2 f(r) = 4\pi\int_0^R dr~r^2 f(r)$$&#xA;assuming $r^2 f(r)$ is well-behaved on $[0, R]$.&lt;/p&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2016-09-08T16:11:49.117" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24945" PostTypeId="2" ParentId="24939" CreationDate="2016-09-08T16:50:08.280" Score="0" Body="&lt;p&gt;Suppose you can initially &quot;bracket&quot; the root, which means you have four points $(x_i,y_i), i=1,...,4$ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_1,y_1) &amp;lt; f_0 \textrm{ and } g(x_1,y_1) &amp;lt; g_0,$$&#xA;$$f(x_2,y_2) &amp;lt; f_0 \textrm{ and } g(x_2,y_2) &amp;gt; g_0,$$&#xA;$$f(x_3,y_3) &amp;gt; f_0 \textrm{ and } g(x_3,y_3) &amp;lt; g_0,$$&#xA;$$f(x_4,y_4) &amp;gt; f_0 \textrm{ and } g(x_4,y_4) &amp;gt; g_0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we can guarantee that a root of the system of equation is contained in the convex hull of the four points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An algorithm could iterate the following two steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute the centroid of the four points.&lt;/li&gt;&#xA;&lt;li&gt;Replace one of the four points by the centroid. The point replaced is chosen so that the eight inequalities above still hold with the new set of four points.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The area of the convex-hull of the four points will eventually shrink, locating the root to arbitrary precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not been able to give an analysis of the speed at which the area of the convex-hull decreases. Also, a robust way to find the initial four points would be useful. Additionally, I claim that the convex hull of the four points contains a root, but I have not been able to give a rigorous proof of this statement (I think it is true intuitively, but I could be wrong).&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2016-09-08T17:01:15.363" LastActivityDate="2016-09-08T17:01:15.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24947" PostTypeId="2" ParentId="1445" CreationDate="2016-09-09T18:22:10.707" Score="0" Body="&lt;p&gt;I would like to add&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/3642332862&quot; rel=&quot;nofollow&quot;&gt;The Finite Element Method: Theory, Implementation, and Applications by Mats. G. Larson and Fredrik Bengzon&lt;/a&gt;. The main feature of the book is contained in its title. It discusses theory, implementation and application. In contrast with usual finite element theoretical books which require a knowledge of functional analysis this books just keeps the requirements to a minimum. As the authors say in the preface of the book, the material should be accessible to students with only knowledge of calculus of several variables, basic partial differential equations, and linear algebra.&lt;/p&gt;&#xA;" OwnerUserId="19747" LastActivityDate="2016-09-09T18:22:10.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24949" PostTypeId="1" AcceptedAnswerId="25041" CreationDate="2016-09-09T23:02:47.217" Score="7" ViewCount="205" Body="&lt;p&gt;I do computational work as a PhD student and I try to find a correct way to separate code (solver) from the computational experiments based on this solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, each project that I do revolves around one particular solver (MATLAB or Python code), and for each project there is a set of experiments that should be done. Before, I came to the idea to do it in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;project&#xA;    01-2016-01-01-task-1&#xA;    02-2016-01-01-task-2&#xA;    03-2016-01-03-task-3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;such that each subfolder corresponds to one particular task (or experiment) that I do. Each subfolder contains all the output from the solver and the solver code itself as I frequently have to modify the solver for a particular experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already deduced that the best way to modify the solver code is in a backward-compatible way, such that if an experiment requires that I change the code to use, for example, a different time integrator (I solve PDEs), then a new configuration parameter should be added to the solver which defines what time integrator to use in the runtime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have a problem with this approach (keeping local version of solver for each experiment): I have to merge all these different versions together and I do it manually now. Recently, an idea came to me that a better organization is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;project&#xA;    code&#xA;    tasks&#xA;        01-2016-01-01-task-01&#xA;        ...&#xA;        27-2016-03-06-task-27&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;that is, that the code should be centralized and version controlled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine the following situation.&#xA;Experiment $N$ requires that I modify the code and then run solver three times with different parameters and each run takes 20 minutes.&#xA;I do it with a bash script which runs the solver three times sequentially.&#xA;In the meantime, I start experiment $N+1$ which also requires code modification.&#xA;I start to modify the code and it can break the experiment $N$ (for example, between run 1 and run 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you handle such situations?&#xA;Is it better to fork the solver code for each experiment and keep it under the experiment subfolder?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATED 2016-09-28&lt;/strong&gt;: According to the answers and comments that I got, it seems that the question was not written clear enough, particularly the title.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question was about how to conduct experiments based on a simulation code, where the code must evolve simultaneously with the progress of the project. Indeed, usually, the code admits some generality from the very start (for example, all problem parameters can be specified as command-line arguments, as well as some solver parameters). However, it is quite possible that the code will require further generalization after a series of experiments, for example, one finite-difference approximation should be replaced by another. In this case, as many pointed out below, the best solution is to add an option to the solver, which chooses the needed approximation during the runtime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, it is better to version-control it as well as to isolate a copy of the code for each experiment. &lt;strong&gt;This is what my question was about: how to deal with multiple copies of the code for each experiment?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't use any version-control system, you can just copy the code into the subdirectory of each experiment and then manually merge any modifications to the &quot;golden&quot; copy of the code (this is my previous approach).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use version-control system Git, you can make multiple clones of the repository to have isolated copies. However, as Git 2.5+ provides a feature to have multiple working copies of the repository, the answer that I wrote below proposes more elegant approach.&lt;/p&gt;&#xA;" OwnerUserId="3254" LastEditorUserId="3254" LastEditDate="2016-09-28T10:51:45.477" LastActivityDate="2016-09-28T10:51:45.477" Title="How to separate a solver from computational experiments in a correct way?" Tags="|experiment|" AnswerCount="3" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="24950" PostTypeId="1" AcceptedAnswerId="24951" CreationDate="2016-09-10T10:50:59.773" Score="1" ViewCount="363" Body="&lt;p&gt;What is the difference between an explicit and implicit preconditioner?&lt;/p&gt;&#xA;" OwnerUserId="21342" LastActivityDate="2016-09-11T07:40:45.603" Title="Difference between explicit and implicit preconditioning" Tags="|linear-solver|iterative-method|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24951" PostTypeId="2" ParentId="24950" CreationDate="2016-09-10T10:57:17.927" Score="2" Body="&lt;p&gt;An explicit preconditioner $G \approx A^{-1}$ approximates the inverse of the system matrix $A$, an implicit preconditioner $C \approx A$ the system itself. Correspondingly, application of the explicit preconditioner $G$ amounts to multiplication with $G$, while application of the implicit preconditioner $C$ amounts to solving a linear system with coefficient matrix $C$. &lt;/p&gt;&#xA;" OwnerUserId="21342" LastEditorUserId="21342" LastEditDate="2016-09-11T07:40:45.603" LastActivityDate="2016-09-11T07:40:45.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24952" PostTypeId="1" CreationDate="2016-09-10T13:26:37.987" Score="2" ViewCount="253" Body="&lt;p&gt;Suppose we have a $f(x)$ to be minimized (we only know that $f(x)$ is three-differentiable), and a feasible, convex set of $S$ such that all $x$ belong to $S$. Using line search method, how we can &quot;mathematically&quot; prove that a descending direction pointing from previous $x$ towards a new $x$ within the $S$ exists?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, how we can find the best $a\geq 0$ such that $x + ad$ belongs to $S$? I mean if we have a direction, how can we optimize the $d$ to have the least iterations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.I'm neither a beginner nor an expert, somewhere in between.&#xA;2.I need all the iterates to be feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="21654" LastEditorUserId="21654" LastEditDate="2016-09-18T11:49:09.507" LastActivityDate="2016-09-18T11:49:09.507" Title="Using Line Search Method for Constrained Optimization" Tags="|constrained-optimization|nonlinear-programming|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24953" PostTypeId="1" AcceptedAnswerId="24954" CreationDate="2016-09-10T17:45:16.777" Score="1" ViewCount="146" Body="&lt;p&gt;I'd like to take an photograph, subdivide it into a tesselation, either of squares, or (ideally), hexagons, and then find the centre of gravity (or, if you prefer, centre of mass) of each cell of the tesselation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output, for any image, would be a matrix of points. For the attached diagram, something like this (in polar coordinates - $(r,\theta)$ ):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(5,5Π/4) (0,0) (1,7Π/4) (1,5Π/4)&#xA;     (5,3Π/4) (0,0) (5,Π/2) &#xA;(0.0) (0,0) (0.0) (0,0)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've attached an image showing what I mean.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4PEzy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4PEzy.png&quot; alt=&quot;Hexagonal tesselation. The black circles are the centres of gravity of each cell - in clear, or monochrome, cells it is in the centre of the cell. In those with a green splodge, the cog is pulled towards the splodge&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question, to put it simply, is the best method to use to calculate the cog, in this tessellation... For squares, it'd be easy to calculate the weighted mean for each row, but that would be too rough an approximation. What's a good way to iterate either a general tessellation (ideal), or an hexagonal one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have asked this question in the 'Computer Graphics' group, but not had any replies. I thought it might be suited better to the skills here.&lt;/p&gt;&#xA;" OwnerUserId="16820" LastEditorUserId="16820" LastEditDate="2016-09-10T17:57:54.493" LastActivityDate="2016-09-10T21:02:29.773" Title="What is a good algorithm, and framework, to calculate centres of gravity or mass (cog)?" Tags="|algorithms|computational-geometry|computational-physics|image-processing|pattern-recognition|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24954" PostTypeId="2" ParentId="24953" CreationDate="2016-09-10T21:02:29.773" Score="2" Body="&lt;p&gt;There are several alternatives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative #1:&lt;/strong&gt; you need to find for each cell of the tessellation the list of pixels contained in the cell. To do that, you can use a rasterization algorithm. You can use my open-source implementation [1]. If performance is an issue, you can also do that with a GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative #2 (the simplest):&lt;/strong&gt; (if you have many different images to process and if there is a single fixed tessellation): first generate an &quot;image&quot; where each pixel contains the index of the cell that contains the pixel. You need to do that once only (it is completely independent on the input image). &#xA;Then you can compute the pixels mass and centroids using the following algorithm (where cell(i,j) denotes the pre-computed &quot;image&quot;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each i,j&#xA;   k = cell(i,j)&#xA;   mass[k] = mass[k] + image(i,j)&#xA;   centroid[k] = centroid[k] + image(i,j)*(i,j)&#xA;&#xA;for each k&#xA;   centroid[k] = centroid[k] / mass[k]&#xA;   convert centroid[k] in polar coordinates (if needed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where mass[] is an array of scalars and centroid[] an array of 2D points (dimension = number of cells for both).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative #3:&lt;/strong&gt; If your tessellation is a Voronoi diagram (i.e. if the cells are the set of points that are nearer to a &quot;site&quot; (xi,yi) than to all the other &quot;sites&quot; (xj,yj) for a list of sites:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each i,j&#xA;   k = nearest_site(i,j)&#xA;   mass[k] = mass[k] + image(i,j)&#xA;   centroid[k] = centroid[k] + image(i,j)*(i,j)&#xA;&#xA;for each k&#xA;   centroid[k] = centroid[k] / mass[k]&#xA;   convert centroid[k] in polar coordinates (if needed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To implement nearest_site(), you can either put all the sites in a Kd-tree, or exploit the structure of your tessellation (for instance, in your particular case, you can find the nearest site by comparing distances to a few sites based on (i,j))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://alice.loria.fr/publications/papers/2006/EGSR_Ardeco//supplemental/sources.html&quot; rel=&quot;nofollow&quot;&gt;http://alice.loria.fr/publications/papers/2006/EGSR_Ardeco//supplemental/sources.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-09-10T21:02:29.773" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24956" PostTypeId="2" ParentId="24930" CreationDate="2016-09-11T06:57:00.133" Score="2" Body="&lt;p&gt;CADO-NFS [1] (implementation of the Number Field Sieve algorithm for factoring integers) has an implementation of sparse binary matrices and linear solve over specific fields (I am not 100% sure it has the Galois field that you need, but it probably has something at least similar).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://cado-nfs.gforge.inria.fr/&quot; rel=&quot;nofollow&quot;&gt;http://cado-nfs.gforge.inria.fr/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-09-11T06:57:00.133" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24958" PostTypeId="2" ParentId="24501" CreationDate="2016-09-11T22:38:57.307" Score="1" Body="&lt;p&gt;You may do well checking out ElmerFEM (&lt;a href=&quot;https://csc.fi/web/elmer/elmer&quot; rel=&quot;nofollow&quot;&gt;https://csc.fi/web/elmer/elmer&lt;/a&gt;) - it is open-source, fairly powerful and well documented. Both Windows and Linux binaries are available.&lt;/p&gt;&#xA;" OwnerUserId="19452" LastActivityDate="2016-09-11T22:38:57.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24959" PostTypeId="2" ParentId="24369" CreationDate="2016-09-11T22:53:16.227" Score="1" Body="&lt;p&gt;Since you are using simple elements and an explicit solver, my approach would be to keep things as simple as possible - row-summation is the easier approach and induces less overhead in your code. You should remember that there is generally energy dissipation associated with using the lumped mass matrix regardless of the lumping method, so some refinement of timestep/element size might be needed depending on how well you need to conserve energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been using such a setup for solving dynamics problems with MPM (Material Point Method) and it works well Dirichlet Boundaries (i.e. a ball bouncing off of a rigid domain boundary).&lt;/p&gt;&#xA;" OwnerUserId="19452" LastActivityDate="2016-09-11T22:53:16.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24960" PostTypeId="2" ParentId="24293" CreationDate="2016-09-11T23:08:46.940" Score="2" Body="&lt;p&gt;There are a number of justifications for mass lumping from a practical standpoint - getting the inverse of the mass matrix can be a huge overhead as problems increase in size, becoming completely unfeasible. Storage is another key issue for large problems - the difference becomes dramatic for problems with 1e4+ dofs. These issues are further compounded in problems where the mass matrix in not constant. Having the diagonal matrix will allow you to solve problems which you may not be able to otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, if you are using low-order elements (e.g. linear quadrilaterals) - the innertial contributions of the nodes will be equal and thus mass lumping will not affect the kinematics of the problem as much. This may not be so for more complex elements (shells etc.).&lt;/p&gt;&#xA;" OwnerUserId="19452" LastActivityDate="2016-09-11T23:08:46.940" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24962" PostTypeId="2" ParentId="24940" CreationDate="2016-09-12T13:56:39.387" Score="2" Body="&lt;p&gt;For the purpose of solving a differential equation, the differentiation matrix &lt;em&gt;must&lt;/em&gt; be square (and invertible). You have added additional degrees of freedom -- thus increasing the number of columns -- so you also need to add the same number of (independent) conditions by increasing the number of rows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have selected your enriching functions properly, this can be done by using these functions as additional test functions, i.e., inserting&#xA;$$v_{N+k,N+l}(x,y) = \tilde h_k(x)\tilde h_l(y)\Phi(x,y)$$&#xA;and integrating.&#xA;Since LGL-pseudospectral methods use nodal basis functions based on quadrature points, this can be seen as adding more quadrature points for each element; in particular, since the enriching functions do not raise the maximal degree, more than you need for exact integration of each basis function. Hence, this is sometimes referred to as &lt;em&gt;over-integration&lt;/em&gt;; see, e.g., &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999115004301&quot; rel=&quot;nofollow&quot;&gt;this reference&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-09-12T15:26:02.553" LastActivityDate="2016-09-12T15:26:02.553" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="24964" PostTypeId="1" AcceptedAnswerId="24978" CreationDate="2016-09-12T19:59:11.097" Score="2" ViewCount="104" Body="&lt;p&gt;The set of &lt;strong&gt;constructable numbers&lt;/strong&gt; is commonly defined in one of two ways:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;the set of finite numbers reachable &lt;strong&gt;from the rational numbers&lt;/strong&gt; through a finite sequence of &lt;strong&gt;addition/subtraction&lt;/strong&gt;, &lt;strong&gt;multiplication/division&lt;/strong&gt;, and &lt;strong&gt;square root&lt;/strong&gt; operations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;They are also defined as an infinite chain of &quot;field extensions&quot; on the rational numbers which basically boils down to this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Begin by defining the set of constructable numbers as $K = \mathbb Q$. (the rationals)&lt;/li&gt;&#xA;&lt;li&gt;Find an arbitary constructable $q$ such that $\sqrt q$ is not yet known to be constructable.&lt;/li&gt;&#xA;&lt;li&gt;Declare all polynomials over $\sqrt q$ to be constructable; i.e. define $K'\equiv\{a + b\sqrt q \mid a,b\in K\}$.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(note there is special notation for this: $K'\equiv K[\sqrt q]$)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Return to step 2, using $K'$ in place of $K$. (and repeat ad infinitum)&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is not altogether difficult to locate inductive-style proofs of the fact that these two definitions are equivalent.  This is cool and all, but...&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;How on earth do you &lt;em&gt;use&lt;/em&gt; them?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you were to tell a machine to perform computations involving these numbers,&#xA;how would you do it? A naive strategy would be to do this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Any given value in the computation is &lt;strong&gt;stored as a ternary tree of values&#xA;that includes the numbers used to create the field extensions&lt;/strong&gt;;&#xA;E.g. a member of $\mathbb Q[\sqrt a][\sqrt b]$ will store three elements of&#xA;$\mathbb Q[\sqrt a]$ ($b$ and the coefficients), each of which will&#xA;store three rationals ($a$ and the coefficients).&#xA;&lt;em&gt;(let's leave aside the issue of all the unnecessary copies of $a$&#xA;this creates, which is in practice easily addressed)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Given two numbers $x,y \in \mathbb Q[k_1][k_2]\dots[k_n]$,&#xA;&lt;strong&gt;their sum, difference, product, and quotient can all be straightforwardly obtained&lt;/strong&gt; as members of $\mathbb Q[k_1][k_2]\dots[k_n]$.&lt;/li&gt;&#xA;&lt;li&gt;To take a &lt;strong&gt;square root&lt;/strong&gt; of a number $q \in K(=Q[k_1][k_2]\dots[k_n])$,&#xA;first try to find a nonnegative root $x \in K$ of the equation $x^2 - q = 0$.&#xA;If this fails, create $\sqrt q$ as a member of a new field extension&#xA;$K[\sqrt q]$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But this isn't complete; &lt;strong&gt;the second rule above requires the sequences of $k$ values to match!&lt;/strong&gt; If we're generating these field extensions on demand, then how do we define operations when they do not match?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that&#xA;&lt;strong&gt;the same number can have &lt;em&gt;many&lt;/em&gt; vastly different representations!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{alignat}{4}&#xA;(1 + \sqrt 3) + (1 + \sqrt 5)&#xA;&amp;amp;= (2 + \sqrt3) &amp;amp;&amp;amp;+ (1 + 0\sqrt3)\sqrt5&#xA;\quad&amp;amp;&amp;amp;\in \mathbb Q[\sqrt3][\sqrt5] \\&#xA;&amp;amp;= (2 + \sqrt5) &amp;amp;&amp;amp;+ (1 + 0\sqrt5)\sqrt3&#xA;\quad&amp;amp;&amp;amp;\in \mathbb Q[\sqrt5][\sqrt3] \\&#xA;&amp;amp;= (2 + 0\sqrt{15}) &amp;amp;&amp;amp;+ (1 + \frac13\sqrt{15})\sqrt3&#xA;\quad&amp;amp;&amp;amp;\in \mathbb Q[\sqrt{15}][\sqrt3]&#xA;\end{alignat}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we recognize these all as equal?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;They seemed very promising at first, but now I am not so sure...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a workable representation of constructable numbers which&#xA;is actually amenable to computation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Addendum&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;At this point I've decided that &lt;a href=&quot;https://scicomp.stackexchange.com/a/24978/21668&quot;&gt;a CAS was all I really needed to begin with&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my secondary motivation for asking the question remains, which is that &lt;strong&gt;I am still genuinely interested in &lt;em&gt;how&lt;/em&gt; one would perform operations on such numbers.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For point of reference, at some point while I was trying to implement my own, I had in fact begun writing something that vaguely resembled the simple beginnings of some sort of term rewriting system:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Term t where&#xA;    -- Methods that attempty to merge two terms into one.&#xA;    trySimplifySum     :: (t,t) -&amp;gt; Maybe t&#xA;    trySimplifyProduct :: (t,t) -&amp;gt; Maybe t&#xA;    trySimplifyRoot    :: t -&amp;gt; Maybe t&#xA;&#xA;data Expression t = Expr { isSimplified :: Bool&#xA;                         , getRawExpr   :: RawExpression t }&#xA;&#xA;data RawExpression t = SingleTerm t&#xA;                     | Sum [Expression t]&#xA;                     | Product [Expression t]&#xA;                     | Root (Expression t)&#xA;&#xA;simplify :: (Term t) =&amp;gt; Expression t -&amp;gt; Expression t&#xA;expand   :: (Term t) =&amp;gt; Expression t -&amp;gt; Expression t&#xA;factor   :: (Term t) =&amp;gt; Expression t -&amp;gt; Expression t&#xA;&#xA;simplify =&#xA;     -- simplify using a greedy algorithm&#xA;     -- (which I REALLY hope is all I need...)&#xA;     --&#xA;     -- recurse on unsimplified subexpressions;&#xA;     -- attempt to reduce terms by brute force,&#xA;     --    using trySimplify* on all pairs, and starting over&#xA;     --    each time a simplification succeeds;&#xA;     -- replace empty sums/products with zero and one&#xA;&#xA;expand = {- do the obvious thing -}&#xA;factor = error &quot;oh god I hope I never have to write this&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it was around this point when I had to step back and ask myself, &lt;em&gt;is this really necessary?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;well...&lt;strong&gt;is it?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="21668" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-09-14T23:32:24.207" Title="Performing computations over the set of constructable numbers" Tags="|algorithms|computer-arithmetic|symbolic-computation|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="24965" PostTypeId="1" CreationDate="2016-09-12T22:51:13.467" Score="1" ViewCount="66" Body="&lt;p&gt;Consider a real symmetric matrix of dimension N~10^5 and rank m~2000.  What is the most efficient algorithm for determining the top m eigenvectors?  If the answer isn't obvious, are there existing routines for this in languages like MATLAB or Python?&lt;/p&gt;&#xA;" OwnerUserId="21669" LastActivityDate="2016-09-12T22:51:13.467" Title="What's the optimal method to solve for the top eigenvectors of a very large, real, symmetric matrix of limited rank?" Tags="|linear-algebra|eigenvalues|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24967" PostTypeId="1" CreationDate="2016-09-13T06:11:30.787" Score="-2" ViewCount="276" Body="&lt;p&gt;(a) Consider the following differential Equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Y'(t)=\frac{1}{1+t^{2}}-2[Y(t)]^2$$&#xA;$$Y(0)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exact solution is $$Y(t)=\frac{t}{1+t^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Euler method to solve the following differential equation with the help of  matlab (code given below), run the program for $t=10$ and $h=0.1$ compute the error.Then run it for $h=0.05$ compute the error and note how it changes.Then use Richardson Extrapolation to improve the accuracy. Compute the error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function y=Euler(t,h,y0,t0)&#xA;&#xA;  n=floor((t-t0)/h);&#xA;  y=y0;&#xA;  for i=1:n&#xA;      y=y+h*f(t0+h*(i-1),y);&#xA;  end;&#xA;&#xA;&#xA;function f=f(t,y)&#xA;&#xA;f=1/(1+t^2)-2*y^2;&#xA;return;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(b) Now using the Runge-Kutta method to solve the differential equation in part (a) Run the program for $t=10$ and $h=0.1$ The error should be reduced as compared to part(a). Explain the difference. Run it again for $h=0.05$ and notice how the error is changed. Use Richardson extrapolation to improve the accuracy and compute the error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function y=Runge_Kutta(t,h,y0,t0)&#xA;&#xA;  n=floor((t-t0)/h);&#xA;  y=y0;&#xA;  for i=1:n&#xA;          t=t0+h*(i-1);&#xA;          y=y+(h/2)*(f(t,y)+f(t+h,y+h*f(t,y)));&#xA;  end;&#xA;&#xA;&#xA;function f=f(t,y)&#xA;&#xA;f=1/(1+t^2)-2*y^2;&#xA;return;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What i tried&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(a) When i ran the program with a step size of $0.1$ i got an error of $0.0986$ but when i ran  it with a step size of $0.05$, i got an error of $0.0988$ which dosent make sense to me beacuse a smaller step size should give a smaller error but it does not appear so in this case.(There is nothing wrong with the code because it is given by my prof). Could anyone explain this to me. Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b) Again when i run the program for this part i got an error of $0.0990$ for both step size $0.1$ and $0.05$ which is larger than that of part(a) which dosent make sense to me beacuse the Runge-Kutta method is supposed to be more accurate than the Euler method and hence have smaller error, but it dosent appear so in this case. Could anyone explain to me what is going wrong here. Thanks&lt;/p&gt;&#xA;" OwnerDisplayName="ys wong" LastEditorUserId="1804" LastEditDate="2016-09-13T07:58:58.573" LastActivityDate="2016-09-13T20:25:20.023" Title="Solving a differential equation using numerical methods and matlab" Tags="|matlab|ode|time-integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24968" PostTypeId="2" ParentId="24967" CreationDate="2016-09-13T08:28:37.537" Score="1" Body="&lt;p&gt;When you run your program with the parameters you specified, you don't get the error, but the solution at time $t=10$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute an error, you need to compare the numerical solution to the analytical one, that's why the analytical solution $Y_a$ of your EDO is given. You need to compute : &#xA;$$ \vert \vert Y - Y_a\vert \vert _{L^1}$$ &#xA;for instance, and plot or write the results against $h$. I suggest you to read courses about norms and time convergence. You will be able to do that in MATLAB with the &lt;code&gt;norm&lt;/code&gt; command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only then you can conclude and notice that when you divide $h$ by 2, the error is divided by 2 if your scheme is order 1 (Euler), or divided by 4 if your scheme is order 2 (Runge-Kutta).&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="9804" LastEditDate="2016-09-13T20:25:20.023" LastActivityDate="2016-09-13T20:25:20.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24970" PostTypeId="1" CreationDate="2016-09-13T10:31:23.590" Score="3" ViewCount="2033" Body="&lt;p&gt;I have a data file with some points equally spaced. These represent some function. I have to calculate the Fourier transform of this set of points. The thing is, I'm tempted to take a cubic spline of these data points to calculate the Fourier transform by FFT (actually I'm more than tempted, since I have done it already). But I'm wondering, isn't this cubic spline adding noise to the Fourier transform ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested this idea on some particular analytical function. I calculated the FFT in different ways (with and without spline) and compared it to the analytical Fourier transform and it actually added some contribution at large frequency. But is this always the case ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some work that has already been done on this ?&lt;/p&gt;&#xA;" OwnerUserId="21472" LastActivityDate="2016-09-13T20:31:00.680" Title="Fourier transform by FFT : by using cubic splines to interpolate between data points, do we change the frequency content of the Fourier transform?" Tags="|interpolation|fourier-analysis|fourier-transform|fftw|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24972" PostTypeId="2" ParentId="24970" CreationDate="2016-09-13T14:44:05.360" Score="7" Body="&lt;p&gt;The frequency content of the interpolated signal is &lt;strong&gt;significantly&lt;/strong&gt; influenced by the interpolation basis. If you have a band-limited function that you have adequately sampled (i.e. satisfying Nyquist criterion), interpolating with any function that is not band-limited to the same frequency will indeed introduce high-frequency noise. Unfortunately, exact band-limited interpolation (i.e. by sinc filter or Fourier method) is a &lt;em&gt;global&lt;/em&gt; operation, in that the interpolation functions involved have infinite tails as a consequence of being exactly band-limited. The consequence of this is that each sample of the interpolated function necessarily involves &lt;strong&gt;all&lt;/strong&gt; samples of the original function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an interesting sequence of papers by John Knab published in IEEE journals in the late 1970s/early 1980s on &lt;em&gt;local&lt;/em&gt; interpolation using functions that are essentially limited in both time and frequency space. Excellent accuracy with respect to the exact band-limited interpolation scheme and minimal spectral pollution can be achieved using the interpolant Knab describes.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2016-09-13T14:44:05.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24973" PostTypeId="2" ParentId="24970" CreationDate="2016-09-13T20:31:00.680" Score="6" Body="&lt;p&gt;The interpolation indeed affects the Fourier transform. @Steve already gives the correct answer in general, but I want to give you an example that helps the intuition more. Think for example that you have samples a sine function on a set of equally spaced points. If you did a (discrete) FFT on these points alone, you would of course recover a Fourier transform that is non-zero for only one frequency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, imagine you connected these points by piecewise linear functions (the simplest possible interpolation). This is no longer a sine function and consequently if you did a (continuous) Fourier transform, you'd get something different than the Fourier transform of the original sine function. In fact, because the function now has kinks, you will have to expect that the Fourier transform has nonzero coefficients for &lt;em&gt;all&lt;/em&gt; frequencies, up to infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a spline interpolation of course is different from a piecewise linear function, but the principle is the same.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-09-13T20:31:00.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24975" PostTypeId="1" CreationDate="2016-09-14T07:25:00.090" Score="2" ViewCount="443" Body="&lt;p&gt;Consider random variables $X$ and $Y$, their distributions are given. $Z = f_a(X, Y)$ where $f(\cdot, \cdot)$ is a deterministic, not random function $f_a: \mathbb{R}^2 \to \mathbb{R}$ depending on a deterministic real parameter $a$. For example, let  $f_a(X,Y)=\sin X + e^{aXY}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read a lot about Bayesian modelling and fitting with the help of &lt;code&gt;pymc3&lt;/code&gt; Python module. But how can I model $Z$ in a frequentist's manner? There's no prior data, no observations, just random variables. I am to  optimize $a$ over setting $\mathbb{E}Z$ to $0$ etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most convinient way to do this? I haven't found solution neither in &lt;code&gt;scipy.optimize&lt;/code&gt; nor &lt;code&gt;scipy.stats&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21679" LastEditorUserId="21679" LastEditDate="2016-09-14T07:53:46.010" LastActivityDate="2016-09-15T05:51:20.663" Title="Optimize custom probability distribution in Python" Tags="|optimization|python|statistics|probability|" AnswerCount="1" CommentCount="10" ClosedDate="2016-09-16T16:32:08.963" ContentLicense="CC BY-SA 3.0" />
  <row Id="24977" PostTypeId="1" AcceptedAnswerId="24983" CreationDate="2016-09-14T14:43:45.140" Score="0" ViewCount="414" Body="&lt;p&gt;(Note: Corrected code is posted below the original code.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an exercise in optimization, I am interested in coding a simple example from scratch: a Tikhonov denoising routine using gradient descent, that matches exactly in output the LSQ-based Tikhonov denoising result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For regularisation of a noised signal $b$ to recover a denoised version $x$, I apply the typical formulation of Tikhonov&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \|A x - b\|^2 + \|\lambda x\|^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $A$ as the identity matrix and $\lambda x$ the weighted gradient matrix $|\lambda \nabla x|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the LSQ case this can be solved using $\hat{A}$ and $\hat{b}$ where &#xA;$ \hat{A} = \left[ I ; \sqrt{\lambda} \nabla x \right] $ , $b = \left[b ; 0 \right] $ and $x = A \backslash b$. The code below shows this result which smooths the signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the gradient descent version, I now try to minimize $f$ where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = 0.5 \|x-b\|^2 + \|\lambda \nabla x\|^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do this by stepping in the direction that is the negative of the function's gradient, which I believe to be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f'(x) = x - b + 2\lambda \nabla ^2 x $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\nabla ^2$ the Laplacian operator. But this code produces rubbish. Can anyone spot my mistake?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function tik_1d&#xA;&#xA;signal = sin((1:512)/128*2*pi);&#xA;signal = signal + randn(size(signal))*0.05;&#xA;lambda = 15;&#xA;niter = 50;&#xA;signal_lsq = tik_lsq(signal, lambda);&#xA;signal_opt = tik_opt(signal, lambda, niter);&#xA;figure(1); set(gcf, 'color', 'w');&#xA;subplot(3, 1, 1); plot(signal); title('b');&#xA;subplot(3, 1, 2); plot(signal_lsq); title('LSQ x');&#xA;subplot(3, 1, 3); plot(signal_opt); title('GD x');&#xA;&#xA;end&#xA;&#xA;function signal_lsq = tik_lsq(signal, lambda)&#xA;    grad = gradient_matrix(signal)*lambda; %subfunction see below&#xA;    A = [eye(numel(signal)); grad];&#xA;    s = signal';&#xA;    b = [s; zeros(size(s))];&#xA;    signal_lsq = A\b;&#xA;end&#xA;&#xA;function m = gradient_matrix(signal)&#xA;    n = numel(signal);&#xA;    diag1 = ones(n,1);&#xA;    diag2 = -diag1;&#xA;    m = spdiags([diag1 diag2], [0 1], n, n);&#xA;end&#xA;&#xA;function signal_opt = tik_opt(signal, lambda, niter)&#xA;    e = zeros(niter,1);&#xA;    signal_opt = signal;&#xA;    tau = 0.1;&#xA;    for n = 1:niter&#xA;        e(n) = eval_f(signal_opt, signal, lambda); %subfunction see below&#xA;        diff = tau*grad_f(signal_opt, signal, lambda); %subfunction see below&#xA;        signal_opt = signal_opt - diff;&#xA;    end&#xA;    figure(2); plot(e);&#xA;end&#xA;&#xA;function e = eval_f(signal_rec, signal, lambda)&#xA;    signal_grad = conv(signal_rec, [1 -1], 'same');&#xA;    e = 0.5*norm(signal_rec-signal) + norm(lambda*signal_grad);&#xA;end&#xA;&#xA;function g = grad_f(signal_rec, signal, lambda)&#xA;    signal_lap = conv(signal_rec, [1 -2 1], 'same');&#xA;    g = (signal - signal_rec) + lambda*(signal_lap);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Mjgic.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Mjgic.jpg&quot; alt=&quot;Tikhonov denoising results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: Here is the code with Christian Clason's suggestions incorporated:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function tik_1d&#xA;&#xA;signal = sin((1:512)/128*2*pi);&#xA;signal = signal + randn(size(signal))*0.05;&#xA;lambda = 15;&#xA;niter = 500;&#xA;signal_lsq = tik_lsq(signal, lambda);&#xA;signal_opt = tik_opt(signal, lambda, niter);&#xA;figure(1); set(gcf, 'color', 'w');&#xA;subplot(4, 1, 1); plot(signal); title('b'); ylim([-1 1]);&#xA;subplot(4, 1, 2); plot(signal_lsq); title('LSQ x'); ylim([-1 1]);&#xA;subplot(4, 1, 3); plot(signal_opt); title('GD x'); ylim([-1 1]);&#xA;subplot(4, 1, 4); plot(signal_opt - signal_lsq'); title('LSQ x - GD x'); ylim([-1 1]);&#xA;&#xA;end&#xA;&#xA;function signal_lsq = tik_lsq(signal, lambda)&#xA;    grad = gradient_matrix(signal)*sqrt(lambda); %subfunction see below&#xA;    A = [eye(numel(signal)); grad];&#xA;    s = signal';&#xA;    b = [s; zeros(size(s))];&#xA;    signal_lsq = A\b;&#xA;end&#xA;&#xA;function m = gradient_matrix(signal)&#xA;    n = numel(signal);&#xA;    diag1 = ones(n,1);&#xA;    diag2 = -diag1;&#xA;    m = spdiags([diag1 diag2], [0 1], n, n);&#xA;end&#xA;&#xA;function signal_opt = tik_opt(signal, lambda, niter)&#xA;    e = zeros(niter,1);&#xA;    signal_opt = signal;&#xA;    tau = 0.001;&#xA;    for n = 1:niter&#xA;        e(n) = eval_f(signal_opt, signal, lambda); %subfunction see below&#xA;        diff = tau*grad_f(signal_opt, signal, lambda); %subfunction see below&#xA;        signal_opt = signal_opt - diff;&#xA;    end&#xA;    figure(2); plot(e);&#xA;end&#xA;&#xA;function e = eval_f(signal_rec, signal, lambda)&#xA;    signal_grad = conv(signal_rec, [1 -1], 'same');&#xA;    e = 0.5*norm(signal_rec-signal) + norm(lambda*signal_grad);&#xA;end&#xA;&#xA;function g = grad_f(signal_rec, signal, lambda)&#xA;    signal_lap = conv(signal_rec, [1 -2 1], 'same');&#xA;    g = (signal - signal_rec) - lambda*(signal_lap);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ldkz5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ldkz5.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3657" LastEditorUserId="20417" LastEditDate="2016-09-17T13:37:37.350" LastActivityDate="2016-09-17T13:37:37.350" Title="How to code gradient descent-based Tikhonov denoising that exactly matches LSQ Tikhonov denoise?" Tags="|optimization|least-squares|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24978" PostTypeId="2" ParentId="24964" CreationDate="2016-09-14T23:31:24.610" Score="0" Body="&lt;p&gt;I now feel this is an instance of the &lt;a href=&quot;https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem&quot;&gt;XY problem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Needless to say, the reason I asked this question was because I was trying to implement it.  However, &lt;strong&gt;there is no need for me to implement something like this!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As people have noted in the comments, CAS systems such as Mathematica provide a natural way of working with such numbers.  And not just that, but they also provide direct solutions to some of the problems that &lt;em&gt;caused me to think I needed this data type in the first place!&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unsurprisingly, even &lt;a href=&quot;http://www.sympy.org/en/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt; can manipulate these numbers; this is the solution I decided to go with since it is available out-of-the-box in Jupyter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am still open to other answers (see addendum)&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="21668" LastEditorUserId="-1" LastEditDate="2017-03-20T10:29:50.967" LastActivityDate="2016-09-14T23:31:24.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24979" PostTypeId="1" CreationDate="2016-09-15T02:11:27.183" Score="0" ViewCount="38" Body="&lt;p&gt;I'm interested in testing some algorithms on the heat equation, and I'd like to assess their accuracy. When evolving a Hamiltonian system, one has the energy to check the validity/correctness of the algorithm, which turns out to be (arguably) the most important quantity to check. Is there a similar quantity for the heat equation - i.e. a quantity which would very well characterize the error of the algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could compute the time-evolution of the &quot;energy&quot; of the heat equation easily by the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t(x,t)=\hat{H}u(x,t),~~~\hat{H}=\frac{\partial^2}{\partial x^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;energy&quot; would then be the expectation value of the $\hat{H}$ operator, which is simple to evaluate in Fourier space. However, I don't know if this &quot;quantity&quot; would even be useful.&lt;/p&gt;&#xA;" OwnerUserId="9562" LastActivityDate="2016-09-15T02:11:27.183" Title="Useful Quantity for Heat Equation?" Tags="|pde|crank-nicolson|" AnswerCount="0" CommentCount="2" ClosedDate="2016-09-16T16:31:21.797" ContentLicense="CC BY-SA 3.0" />
  <row Id="24980" PostTypeId="2" ParentId="24975" CreationDate="2016-09-15T05:51:20.663" Score="1" Body="&lt;p&gt;Mathematically, the probability density function ($\operatorname{PDF}$) for $Z$ is given by the integral:&#xA;$$\operatorname{PDF}(Z) = \int \delta\left(Z - f_a(X,Y)\right) \operatorname{PDF}(X,Y)\operatorname{d}X \operatorname{d}Y.$$ If the transformation $$\left[\begin{array}{c} X \\ Y \end{array}\right] \rightarrow \left[\begin{array}{c} f_a(X,Y) \\ g_a(X,Y) \end{array}\right] $$ is one to one (if not, break it down into regions where it is, and sum) then:&#xA;$$\operatorname{PDF}(Z) = \int \frac{\operatorname{PDF}(X(f_a, g_a),Y(f_a, g_a))}{\left|\frac{\partial f_a}{\partial X} \frac{\partial g_a}{\partial Y} - \frac{\partial f_a}{\partial Y} \frac{\partial g_a}{\partial X} \right|}\operatorname{d}g_a. $$ Some of the simpler choices for $g_a$ include $g_a= X$ or $g_a = Y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for how I would do this, given the vague setup, I would set up a &lt;A HREF=&quot;https://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm&quot; rel=&quot;nofollow&quot;&gt;Metropolis-Hastings algorithm&lt;/A&gt; to produce a chain of $(Z, g_a)$ pairs from the bivariate density $$\frac{\operatorname{PDF}(X(f_a, g_a),Y(f_a, g_a))}{\left|\frac{\partial f_a}{\partial X} \frac{\partial g_a}{\partial Y} - \frac{\partial f_a}{\partial Y} \frac{\partial g_a}{\partial X} \right|},$$ and just drop the auxiliary variable, $g_a$, when returning results. You may need to implement a &quot;sample every $N^{\mathrm{th}}$ step&quot; limitation to reduce sample correlations. If the numbers don't have to be generated on command, but can be generated as a batch, then you can use the &lt;A HREF=&quot;http://dan.iel.fm/emcee/current/&quot; rel=&quot;nofollow&quot;&gt;emcee MCMC sampling package&lt;/A&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8313" LastActivityDate="2016-09-15T05:51:20.663" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24981" PostTypeId="1" AcceptedAnswerId="24987" CreationDate="2016-09-15T08:29:09.620" Score="2" ViewCount="972" Body="&lt;p&gt;I am doing a little project on solving the heat equation using finite-volume method on a solid cube, I converted the polyhedral mesh of the cube to an OpenFOAM mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a Python code where I parse the points, faces, owner, neighbour and boundary files of the OF case mesh, I've managed to create a hash list (let's call it &lt;code&gt;CellFacesMap&lt;/code&gt;) to map the cell number to the list of its owned faces and faces neighbours to it. Now, I am trying to generate a hash list (let's call it &lt;code&gt;CellAdjacentsMap&lt;/code&gt; that maps each cell with all its adjacent cells, in order to apply the discretized heat equation to each cell (and also for generating the volume weighting factors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way I could think of now to do so is to visit each cell in &lt;code&gt;CellFacesMap&lt;/code&gt; and for the owned faces I get the cells neighbor to this face (adjacent cells) and for faces neighbor the same cell I get the cells that owns theses faces (also adjacent cells), meaning I visit &lt;code&gt;CellFacesMap&lt;/code&gt; over and over for each cell to check its faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this method is costly (time wise, it took 3.5 minutes to generate a map for 4000+ cell mesh). So is there any faster way to generate &lt;code&gt;CellAdjacentsMap&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="13154" LastEditorUserId="13154" LastEditDate="2016-09-15T08:50:38.643" LastActivityDate="2016-09-15T16:56:10.440" Title="Getting adjacent cells map for an unstructured polyhedral mesh" Tags="|finite-volume|mesh|openfoam|unstructured-mesh|mesh-traversal|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24982" PostTypeId="1" CreationDate="2016-09-15T09:36:56.017" Score="4" ViewCount="927" Body="&lt;p&gt;I have crude idea that numerical diffusion arises while using upwind scheme and causes solution to deviate from its original one. But I am unable to understand how numerical diffusion phenomenon is (directly) related to advection phenomenon?&lt;/p&gt;&#xA;" OwnerUserId="20089" LastActivityDate="2023-02-02T19:55:21.527" Title="How numerical diffusion is related to advection term?" Tags="|numerical-modelling|advection-diffusion|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24983" PostTypeId="2" ParentId="24977" CreationDate="2016-09-15T10:03:13.887" Score="2" Body="&lt;p&gt;There are a few problems with your gradient descent:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The gradient of $\|\nabla u\|^2$ is &#xA;$$\nabla^* \nabla u = -\mathrm{div} \nabla u = -\Delta u,$$&#xA;i.e., the &lt;em&gt;negative&lt;/em&gt; Laplacian.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Gradient descent does not converge unconditionally, but only if the step sizes are chosen appropriately (in particular, not too large). Either you reduce the step size manually until you have convergence, or implement a proper line search, e.g., based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Wolfe_conditions#Armijo_rule_and_curvature&quot; rel=&quot;nofollow&quot;&gt;Armijo or Wolfe conditions&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You're not minimizing the same functional as in the LSQR approach, since there's a factor of $1/2$ in front of the first term which isn't there in the LSQR functional (which effectively makes your $\lambda$ twice as large).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Finally, a nitpick: although this is consistent in both approaches, your gradient and matrix $\hat A$ correspond to $\lambda \|\nabla u\|^2$, not $\|\lambda \nabla u\|^2$. (The former is the way it's usually written.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-09-15T10:03:13.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24984" PostTypeId="2" ParentId="24982" CreationDate="2016-09-15T10:03:17.937" Score="1" Body="&lt;p&gt;The numerical diffusion is not related to an equation specifically but is related to &lt;strong&gt;the way you discretize&lt;/strong&gt; this equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the discretization stencil you choose (upwind, downwind, centered, hybrid...), you scheme can be diffusive, compressive or dispersive according to the sign and the order of the derivatives involved in the truncation error. You can understand that by deriving and analysing the truncation error through a Taylor expansion, I did an example &lt;a href=&quot;https://scicomp.stackexchange.com/questions/24580/more-smearing-with-decreasing-timestep-in-advection-problems/24588#24588&quot;&gt;here&lt;/a&gt; where I explain why the advection equation discretized with an upwind scheme produces numerical diffusion.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-09-15T10:03:17.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24985" PostTypeId="2" ParentId="24981" CreationDate="2016-09-15T10:50:51.323" Score="3" Body="&lt;p&gt;Using a hash map adds a log(n) complexity to all accesses (then traversing the whole mesh will cost n log(n) in general), so clearly it is not the best solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now your question is how you can efficiently store and traverse adjacency information, i.e. for each cell the list of adjacent cells. First thing you can do for mapping a cell to its list of neighbors: you can use an array instead of a hash, this will remove the log(n) cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, using an array, this means you need to store an array of lists of different lengths (if your mesh has arbitrary polyhedra), which is in general not very efficient: each list has its own memory overhead to be added to the memory used by the elements it contains.&#xA;A more efficient way of storing this type of information (i.e. an array of lists of different lengths) is the &quot;compressed row storage&quot; (called CRS, CSR or Yale format) representation [1], used to store sparse matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; If adjacency information is not present in your input, then you will need to recreate it. Instead of using a hash or associative map as suggested in the other answer, an alternative is to store all facets in an array then &lt;em&gt;sort&lt;/em&gt; this array. After sorting, the pairs of facets that correspond to adjacent cells are contiguous in the array. I observed that it is significantly more efficient (at least in C++, I did not test that in Python). See also my answer to this question [2]. To adapt my answer to  your polyhedral grid, facet records will have a facet index and as a key, three vertex indices (the lowest vertex id in the facet and the two ones connected to it with an edge, in increasing order). By sorting facet records with the lexicographic order, you will retrieve the pair of adjacent facets contiguously in the array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;For each cell c&#xA;   For each facet f of cell c&#xA;       Find v1 the smallest vertex index in facet f&#xA;       Find v2 and v3, the vertices before and after v1 in facet f&#xA;       If v3 &amp;lt; v2, swap v2 and v3&#xA;       Append a new record [v1, v2, v3, c] to the array&#xA;&#xA;Sort the record array with the lexicographic order on (v1,v2,v3)&#xA;&#xA;In the record array, find (by scanning the array) the contiguous &#xA;pairs of record (v1,v2,v3,c1);(v1,v2,v3,c2) with the same (v1,v2,v3). &#xA;For each such pair, c1 and c2 are adjacent.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If input data has facet ids, then records are simply (f,c) couples, and they are sorted by f (instead of (v1,v2,v3)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://en.wikipedia.org/wiki/Sparse_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Sparse_matrix&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21163/meshing-options-to-generate-number-of-the-sides-of-and-element-tetgen-triangle/21197#21197&quot;&gt;Meshing options to generate number of the sides of and element (tetgen-triangle)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-09-15T16:56:10.440" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24986" PostTypeId="2" ParentId="24501" CreationDate="2016-09-15T13:48:05.500" Score="1" Body="&lt;p&gt;The followings links will help you to find out more.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.dealii.org/&quot; rel=&quot;nofollow&quot;&gt;deal II&lt;/a&gt; is a nice advanced open source FEM code. See their page for more information. It also has the support for windows.&lt;/li&gt;&#xA;&lt;li&gt;Also, &lt;a href=&quot;https://www.researchgate.net/post/What_is_the_best_open_source_finite_element_software_for_mechanical_problems&quot; rel=&quot;nofollow&quot;&gt;this link&lt;/a&gt; provides some experience of other experts in this field.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="19747" LastActivityDate="2016-09-15T13:48:05.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24987" PostTypeId="2" ParentId="24981" CreationDate="2016-09-15T14:14:44.647" Score="2" Body="&lt;p&gt;I'm assuming you are starting with a list of cell definitions, say,&#xA;as a list of vertices defining the cell and a &quot;type&quot; defining the&#xA;topology of each cell. As part of the topology definition for each&#xA;cell, you can easily get the faces for that cell defined, say, as&#xA;a list of vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step in creating the CellAdjacentsMap that you need is to&#xA;first create a map that I'll call FaceCellMap that lists the cells&#xA;(either one or two) that each face belongs to. You do this by&#xA;iterating over all cells, adding each cell's faces to the map, one&#xA;by one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some implementation details in forming this FaceCellMap.&#xA;The first is what data structure do you use? If you have a general&#xA;HashMap capability, the key can be face definition and the value&#xA;can be a length-two array of cell IDs (one of the entries may be&#xA;zero). The second issue is how do you compare faces? You can use&#xA;a list of vertices but you have implement the comparison function&#xA;in such a way that it is insensitive to the &quot;starting&quot; vertex and&#xA;whether you traverse the vertex list in clockwise or counterclockwise&#xA;order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But after you have created FaceCellMap, creating CellAdjacentsMap is&#xA;straightforward. You iterate through all the cells. For each cell, &#xA;you iterate through all its faces. For each face, you check the&#xA;FaceCellMap for that face to see if it has an adjacent cell. If so&#xA;you add it to the list of adjacencies for the current cell.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-09-15T14:14:44.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24988" PostTypeId="1" CreationDate="2016-09-15T14:27:58.147" Score="1" ViewCount="290" Body="&lt;p&gt;I am trying to implement Adaptive Mesh Refinement. I am not a Mathematics/Computational Science person so I will try to write the algorithm in a simpler way. I will be grateful if experts can comment on the correctness of steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Aim&lt;/strong&gt;: To solve Laplace Equation with Dirichlet boundary conditions i.e. &#xA;$\nabla^{2} u = 0 $ and $u = 1$ on $\partial \Omega$. Assume 3 levels of static refinement (say for 2-D mesh on $[0,1]$ I refine cells with $x^{2}+y^{2}&amp;gt;0.7$ and $x^{2}+y^{2}&amp;lt;0.9$, where $x, y$ are coordinates).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialize $u = 0$ on all levels.&lt;/li&gt;&#xA;&lt;li&gt;Solve on coarsest level (level 1) using 5-pt stencil Jacobi iterations. The initial value of $u$ at level 2 is used as Coarse-Fine interface value. &lt;/li&gt;&#xA;&lt;li&gt;Exchange ghost cells for parallel implementation at coarsest level.&lt;/li&gt;&#xA;&lt;li&gt;Interpolate coarse grid cells at level 1 to fill ghost cells at level 2. &lt;/li&gt;&#xA;&lt;li&gt;Solve on level 2. The initial value of $u$ at level 3 is used as Coarse-Fine interface value (and also the interpolated values in ghost cells). &lt;/li&gt;&#xA;&lt;li&gt;Exchange ghost cells for parallel implementation at level 2.&lt;/li&gt;&#xA;&lt;li&gt;Repeat same at level 3.&lt;/li&gt;&#xA;&lt;li&gt;Now restrict from level 3 to 2 and level 2 to 1 (so that coarser grids see some part of the solution at finer grids). &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Repeat these steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Are these steps correct&lt;/em&gt; ?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In 3-D my mesh looks like the following)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hcS2b.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hcS2b.jpg&quot; alt=&quot;3 level mesh in 3-D&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="1280" LastEditDate="2016-09-20T14:50:37.127" LastActivityDate="2016-09-20T14:50:37.127" Title="Algorithm for Adaptive Mesh Refinement" Tags="|pde|finite-difference|multigrid|adaptive-mesh-refinement|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="24989" PostTypeId="1" CreationDate="2016-09-15T18:44:07.500" Score="1" ViewCount="4752" Body="&lt;p&gt;I need to compare two computers and decide which one I want. My goal is to run faster simulations. The current run time is 3.5 hours and I would like to reduce that as much as possible. The code I am running has the capability of running on multiple processors (parallel). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which specifications of the pc should I pay attention to?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The RAM? number of processors? What else? and what are the most important ones?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="1053" LastActivityDate="2016-09-16T14:15:53.757" Title="What makes a computer fast and powerful to run numerical simulations?" Tags="|numerical-analysis|parallel-computing|performance|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="24991" PostTypeId="1" CreationDate="2016-09-15T19:57:02.440" Score="3" ViewCount="223" Body="&lt;p&gt;I have a question regarding the treatment of Robin boundary conditions in a multigrid solver. I am solving the Poisson equation in $\Omega=(0,1)^2$ with Robin boundary conditions on the boundary,&#xA;$$- \Delta u = f \text{ in } \Omega$$&#xA;and&#xA;$$\frac{\partial u}{\partial n} - au= g \text{ on } \partial \Omega.$$&#xA;I use a finite difference discretization with central differences (a standard five point stencil) for the interior points, and eliminate the boundary conditions so that, for example, at a point $x_C$ on the left boundary with east, south and north neighbors $x_E, x_S, x_N$ the resulting equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{h^2}\left( 4u(x_C)-u(x_S)-u(x_N)-2u(x_E)\right) - 2\frac{a}{h}u(x_C) = \frac{2}{h}g(x_C)+f(x_C).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In chapter 5 of the book 'Multigrid' of Trottenberg et. al., it is explained that if this approach is followed for a Neumann problem ($a=0$) and the full-weight restriction operator $R$ is modified on the boundary points, e.g., on a vertical boundary the modified stencil is&#xA;$$ \frac{1}{16}\left[\begin{array}{cc} 2 &amp;amp;2 \\ 4&amp;amp;4 \\2&amp;amp;2 \end{array}\right]_{h}^{2h} $$&#xA;then a relaxation method can be applied directly on the equations for the boundary points with the eliminated boundary condition (no need of transferring the equation and boundary condition separately). I am using the interpolation operator $I=4R^T$ and Galerkin coarse grid systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Is this multigrid approach correct (modified FW operator on the boundaries + relaxation on the equation with eliminated boundary conditions and Galerkin coarse systems) also for more general Robin boundary conditions, provided a good relaxation method is used?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="21695" LastEditorUserId="21695" LastEditDate="2016-09-16T00:43:07.590" LastActivityDate="2016-09-16T00:43:07.590" Title="Multigrid for Robin boundary conditions" Tags="|pde|boundary-conditions|multigrid|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24992" PostTypeId="2" ParentId="24989" CreationDate="2016-09-15T22:00:03.387" Score="1" Body="&lt;p&gt;If you want your PC to perform numerical simulations, the processor is clearly the first component you should paid attention to. For this kind of application, Intel i7 and Intel Xeon are often used in desktop PC, no advertising intended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the frequency of the processor is important but look carefully at the specifications of the L1/L2/L3 caches, especially the size of the cache. A large cache is interesting (and expensive). That is an important information since the cache is a very low latency memory access widely used by the processor that can be used for specific optimizations (called cache optimizations). You should go to specific websites for detailed information, mainly Intel or AMD website's, especially for the ability of the processor to perform specific floating-point arithmetic operations, vectorization, hardware acceleration etc... It is very difficult to say that a processor is faster than another. However, you can have an idea of the overall performance of a CPU by looking at benchmark tests &lt;a href=&quot;http://www.cpubenchmark.net&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For running in parallel, the important is the number of cores. The core is really the component that perform the calculation. For a desktop PC, you could have 1 processor with 8 cores or 2 processors with 4 cores each, I can't explain you the difference. You should also paid attention to the ability of the hyper-threading which can in certain circumstances virtually doubles the number of cores. Remember however that running a code on 16 cores is not always faster than running on 8 cores, it strongly depends on the scalability of your code and the quality of your parallelization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the RAM, it strongly depends on how much data your code needs to store. Besides, don't forget that you should be able to post-process your data and visualization software often require huge amount of RAM. 16Go is a good start for scientific applications but that is very subjective. You may need a decent graphic cards for real-time or high quality rendering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course around all of this, a decent motherboard with high-speed transfer and a (big) &quot;fast read/write&quot; hard-drive are required.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-09-15T22:00:03.387" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="24994" PostTypeId="1" AcceptedAnswerId="24997" CreationDate="2016-09-16T12:56:14.320" Score="2" ViewCount="67" Body="&lt;p&gt;In my optimization problem, the hessian has a structure such that it can be written as the sum of two matrices. Populating the first of the matrices is efficient. Populating the second one is computationally very heavy and practically makes methods like truncated Newton useless. However, the second matrix is block diagonal in nature. I am thinking I will use a Quasi-Newton approximation to the second matrix and compute the first matrix exactly. Hence, get an approximation for the hessian by adding these two matrices. Intuitively, this Hessian approximation should be better than having a Quasi-Newton approximation for the entire Hessian matrix. Is this thinking correct? Have people successfully tried such an approach before and got a better optimization approach than a direct Quasi-Newton based approach?&lt;/p&gt;&#xA;" OwnerUserId="14478" LastEditorUserId="14478" LastEditDate="2016-09-16T13:47:25.180" LastActivityDate="2016-09-16T17:48:33.517" Title="Doubt regarding principled approach towards approximating the Hessian" Tags="|linear-algebra|optimization|convex-optimization|newton-method|quasi-newton|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24996" PostTypeId="2" ParentId="24989" CreationDate="2016-09-16T14:15:53.757" Score="2" Body="&lt;p&gt;Optimisation always depends on the code's &lt;a href=&quot;https://en.wikipedia.org/wiki/Profiling_(computer_programming)&quot; rel=&quot;nofollow&quot;&gt;profile&lt;/a&gt;. You should run the application through an appropriate &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_performance_analysis_tools&quot; rel=&quot;nofollow&quot;&gt;profiling tool&lt;/a&gt; to see where the bottlenecks are - it will typically be :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I/O-bound (spends time writing to disk) - probably not your case.&lt;/li&gt;&#xA;&lt;li&gt;CPU-bound (spends time in CPU operations) which can be both dependent on data throughput or calculation time. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If your data structures and processing strategy is such that the application cannot load the data into RAM all at once, you will suffer. If the computational techniques rely on moving large amounts of data in and out of the memory, you will run into caching issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a perfect world, you would break the computation into as  many independent tasks as possible and send them to as many processing units at once as you can - however, you need to make a trade-off in the real world.&lt;/p&gt;&#xA;" OwnerUserId="21701" LastActivityDate="2016-09-16T14:15:53.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="24997" PostTypeId="2" ParentId="24994" CreationDate="2016-09-16T17:48:33.517" Score="1" Body="&lt;p&gt;You may find &lt;a href=&quot;https://link.springer.com/article/10.1007/BF00962795&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; relevant. They give some BFGS-inspired methods and convergence proofs for cases where the Hessian is the sum of a part you can compute easily and a part you can't (&quot;structured secant&quot; methods), and you want to leverage your knowledge of the computable part as much as possible.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-09-16T17:48:33.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24998" PostTypeId="1" AcceptedAnswerId="25031" CreationDate="2016-09-17T02:52:28.030" Score="13" ViewCount="8116" Body="&lt;p&gt;I simply want to know whether the &lt;a href=&quot;https://en.wikipedia.org/wiki/Dormand-Prince_method&quot; rel=&quot;noreferrer&quot;&gt;Dormand-Prince Numerical Method&lt;/a&gt;&#xA;or the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cash-Karp_method&quot; rel=&quot;noreferrer&quot;&gt;Cash-Karp Numerical Method&lt;/a&gt;&#xA;is more accurate. &lt;/p&gt;&#xA;" OwnerUserId="21712" LastEditorUserId="1804" LastEditDate="2016-09-17T08:59:31.547" LastActivityDate="2022-07-18T18:42:06.577" Title="Which Runge-Kutta method is more accurate: Dormand-Prince or Cash-Karp?" Tags="|numerical-analysis|ode|runge-kutta|integration|accuracy|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="24999" PostTypeId="1" AcceptedAnswerId="25000" CreationDate="2016-09-17T05:12:19.223" Score="16" ViewCount="7173" Body="&lt;p&gt;I have two graphs with nearly n~100000 nodes each. In both graphs, each node is connected to exactly 3 other nodes so the adjacency matrix is symmetric and very sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hard part is I need &lt;strong&gt;all&lt;/strong&gt; eigenvalues of the adjacency matrix but not eigenvectors. To be accurate, this is going to be once in my lifetime (as far as I can see, at least!) so I want to get all eigenvalues and don't mind waiting a few days to get them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried &lt;code&gt;scipy&lt;/code&gt; wrappers around &lt;code&gt;ARPACK&lt;/code&gt;, but it takes way too long. I found multiple libraries but they work best for getting a subset of largest/smallest eigenvalues. Is there any library which works for symmetric sparse matrices with possibly parallel implementation to get all eigenvalues?&lt;/p&gt;&#xA;" OwnerUserId="8343" LastEditorUserId="8343" LastEditDate="2016-09-17T06:45:19.987" LastActivityDate="2017-02-03T20:57:56.850" Title="Compute all eigenvalues of a very big and very sparse adjacency matrix" Tags="|linear-algebra|sparse-matrix|eigenvalues|matrix|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25000" PostTypeId="2" ParentId="24999" CreationDate="2016-09-17T06:38:08.427" Score="12" Body="&lt;p&gt;You can use the shift-invert spectral transform [1] and compute the spectrum band by band.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The technique is also explained in my article [2]. Besides the implementation in [1], an implementation is available in C++ in my Graphite software [3] &lt;em&gt;(&lt;strong&gt;update Jan 17&lt;/strong&gt;: now everything is ported to geogram/graphite version 3&lt;/em&gt;), that I used to compute the eigenfunctions of the Laplace operator for meshes with up to 1 million vertices (a problem that is similar to yours).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How it works:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that if $A$ is invertible, then if $(V,\lambda)$ is an eigenpair of $A$, $(V, 1/\lambda)$ is an eigenpair of $A^{-1}$. The iterative method in ARPACK is very efficient for computing the large eigenvalues (high frequencies), but much less efficient for the small ones (small frequencies). Thus, when one needs to compute small frequencies, it is a good idea to replace $A$ with $A^{-1}$. Now, since ARPACK just needs to compute matrix-vector products, it is not necessary to really invert $A$: one can instead factor it (using for instance a sparse LU or sparse $LL^t$ factorization), then solve $Ax=b$ whenever ARPACK asks for a matrix-vector product. This is the &quot;invert&quot; transform. Now when the number of eigenvalues becomes large, ARPACK becomes to be slow, but there is another trick/transform that can be used, and one computes the eigenvalues of $A - \sigma Id$ where $\sigma$ is a &quot;shift&quot; that determines which part of the spectrum is explored (this is the &quot;shift&quot; transform). Combining both transforms, one computes a certain number of eigenvalues of $(A-\sigma Id)^{-1}$, and then explores the whole spectrum band-by-band, by increasing $\sigma$. The details are in [1],[2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.mcs.anl.gov/uploads/cels/papers/P1263.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.mcs.anl.gov/uploads/cels/papers/P1263.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://alice.loria.fr/index.php/publications.html?redirect=0&amp;amp;Paper=ManifoldHarmonics@2008&quot; rel=&quot;noreferrer&quot;&gt;http://alice.loria.fr/index.php/publications.html?redirect=0&amp;amp;Paper=ManifoldHarmonics@2008&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://alice.loria.fr/software/graphite/doc/html/&quot; rel=&quot;noreferrer&quot;&gt;http://alice.loria.fr/software/graphite/doc/html/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-01-17T09:35:58.470" LastActivityDate="2017-01-17T09:35:58.470" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25001" PostTypeId="1" CreationDate="2016-09-17T09:56:03.277" Score="1" ViewCount="341" Body="&lt;p&gt;I know that, from numerical point of view, computing &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ax = b&#xA;B=inv(A), x= B*b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;are completely different things, and we should factor the matrix using TRF routine then solve with TRS/TRI routine, or in a combined gesv call.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway is there an efficiently routine to specifically deal with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C = A* inv(B)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BTW, I know some templating libraries(e.g. Eigen) might be able to do it well, but I am more interested in a Lapack/MKL solution, that said, correct solution using Eigen is also welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;, as Christian Clason pointed out:&#xA;C = A* inv(B) can be computed via gesv(trans(B),trans(A)), However, in a real implementation with lapack, gesv will overwrite its input, so does it mean we have to introduce temp variable in order to have C = A * inv(B) correctly computed in semantic sense.&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="4532" LastEditDate="2016-09-17T14:14:37.973" LastActivityDate="2016-09-17T14:32:55.850" Title="Appropriate Lapack/MKL routines to efficiently compute C = A* inv(B)" Tags="|linear-algebra|lapack|intel-mkl|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="25002" PostTypeId="2" ParentId="25001" CreationDate="2016-09-17T14:32:55.850" Score="1" Body="&lt;p&gt;Here is the Eigen example you requested. It follows the approach in Christian Clason's comment and also does the same computation using an explicit inverse for comparison.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;using std::cout;&#xA;using std::endl;&#xA;#include &amp;lt;ctime&amp;gt;&#xA;&#xA;#include &amp;lt;Eigen/Core&amp;gt;&#xA;#include &amp;lt;Eigen/LU&amp;gt;&#xA;&#xA;void compareInvAndFactor()&#xA;{&#xA;&#xA;  typedef Eigen::MatrixXd Matrix;&#xA;  const int n = 1000, m = 600;&#xA;  Matrix B = Matrix::Random(n, n);&#xA;  Matrix A = Matrix::Random(m, n);&#xA;&#xA;  std::clock_t start = std::clock();&#xA;  auto Clu = B.transpose().lu().solve(A.transpose()).transpose();&#xA;  cout &amp;lt;&amp;lt; &quot;Elapsed time using LU factorization = &quot; &amp;lt;&amp;lt; (std::clock() - start) / &#xA;          (double)CLOCKS_PER_SEC &amp;lt;&amp;lt; &quot; seconds.&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;  start = std::clock();&#xA;  auto Cinv = A*B.inverse();&#xA;  cout &amp;lt;&amp;lt; &quot;Elapsed time using inverse = &quot; &amp;lt;&amp;lt; (std::clock() - start) / &#xA;          (double)CLOCKS_PER_SEC &amp;lt;&amp;lt; &quot; seconds.&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;  double diff = (Cinv - Clu).cwiseAbs().maxCoeff();&#xA;  cout &amp;lt;&amp;lt; &quot;max difference in C matrices =&quot; &amp;lt;&amp;lt; diff &amp;lt;&amp;lt; endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-09-17T14:32:55.850" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25003" PostTypeId="2" ParentId="21856" CreationDate="2016-09-17T15:36:01.323" Score="2" Body="&lt;p&gt;To expand on Bill Greene's answer, Fortran uses different syntax for single and double precision floating point numbers. 1.e0 is a single precision representation of the number '1.0', whereas 1.d0 is double precision representation of the number '1.0'. If you have a rather straight forward problem, and an associated low condition number, you will not notice a difference because the problem can be solved in single precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As problems become more difficult, they require higher levels of precision - that's why it's good practice to set your variables to double precision values; however, there is a caveat if you mix single and double precision numbers. Fortran will reduce to the lowest common denominator, which means that multiplying a double precision number with a single precision number will result in a single precision number. This won't be an issue for simple problems, but as problems get more complex and require consistent usage of double precision, this will represent itself as a bug in your code.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Copying from &lt;a href=&quot;https://stackoverflow.com/a/8434928/5536001&quot;&gt;this answer&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;4.5.1 Double Precision Exponent.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The form of a double precision exponent is the letter D followed by an optionally signed integer constant. A double precision exponent denotes a power of ten. Note that the form and interpretation of a double precision exponent are identical to those of a real exponent, except that the letter D is used instead of the letter E.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-09-17T15:36:01.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25004" PostTypeId="1" AcceptedAnswerId="25006" CreationDate="2016-09-17T18:24:47.913" Score="0" ViewCount="82" Body="&lt;p&gt;I would like to program the following difference equation. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Find numbers $v_{i,j}$ so that for $1\leq i\leq 4$, $0\leq j\leq 5$: &#xA;$$ v_{i,j+1} = (0.1)v_{i+1,j} + (0.8)v_{i,j} + (0.1)v_{i-1,j} $$&#xA;In addition it satisfies the conditions, for $0\leq j\leq 5$:&#xA;$$ v_{0,j} = (0.2)j \text{ and }v_{5,j} = 25 + (0.2)j$$&#xA;And finally, the condition, for $0\leq i\leq 5$: &#xA;$$ v_{i,0} = i^2 $$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I made a picture of this grid, where $v_{i,j}$ is draw in the standard xy-plane. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VytbX.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VytbX.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red points, is the information which is given to us. The intermediate grid is unknown, but we can determine it from the difference equation. The values of $v_{i,j}$ are draw in black numbers next to red points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tree in the middle is the visualization of the difference equation. The purple point is the weighed average of the three points below it with the numbers indicating how it is being averaged. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I would like to produce a grid $v_{i,j}$ in MatLab. Then plot it using the 'surf' command for the approximating surface. &lt;/p&gt;&#xA;" OwnerUserId="21719" LastActivityDate="2016-09-19T09:55:04.337" Title="Difference Equation PDE in MatLab" Tags="|matlab|pde|finite-difference|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25005" PostTypeId="1" CreationDate="2016-09-17T20:25:21.267" Score="1" ViewCount="559" Body="&lt;p&gt;Dekker's method and in its evolution Brent's method have as design goal to combine the certainty of a root, certified by function values of opposite sign in an increasingly smaller interval, of bracketing methods like bisection and regula falsi with the fast convergence of the secant (and higher degree of (reverse) interpolation) methods. Especially in the regula falsi method without modification one observes with regularity that one border of the bracketing interval becomes fixed, which causes the convergence to be linear, sometimes even slower than the bisection method.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Why experiment with the Dekker's method, not Brent's method? I have implemented Brent's method based on English Wiki but I can't understand two conditionals for &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt;: condition 4 and condition 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; is rather big (for example &lt;span class=&quot;math-container&quot;&gt;$10^{-4}$&lt;/span&gt;) this conditionals are bad, if &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; is small (&lt;span class=&quot;math-container&quot;&gt;$10^{-14}$&lt;/span&gt;) it is no difference in my sample functions.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I try to implement Dekker's method to explore functions with bad behaviour. I have written code based on Wikipedia and [1]. But in both cases is the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do with &lt;span class=&quot;math-container&quot;&gt;$$f(x) = (x + 3)(x − 1)^2~~\text{ and range }~~[−4, 4/3].$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$4/3$&lt;/span&gt; is near the root &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$-4$&lt;/span&gt; is further away from the root &lt;span class=&quot;math-container&quot;&gt;$-3$&lt;/span&gt;. The sequence of iteration points approaches &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; but the left border point of the range is always &lt;span class=&quot;math-container&quot;&gt;$-4$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Based on [1]&#xA;a1=-4.00000000 b1=1.33333333 c1=-4.00000000&#xA;a2= 1.33333333 b2=1.23255814 c2=-4.00000000&#xA;a3= 1.23255814 b3=1.14122330 c3=-4.00000000&#xA;a4= 1.14122330 b4=1.08966720 c4=-4.00000000&#xA;a5= 1.08966720 b5=1.05556489 c5=-4.00000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The contrapoint &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; is always &lt;span class=&quot;math-container&quot;&gt;$-4$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Do I have an error in implementation or has this method this bad behaviour in this function? How to modify point &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Bus, J. C. P., and T. J. Dekker. &lt;a href=&quot;http://oai.cwi.nl/oai/asset/20561/20561A.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Two efficient algorithms with guaranteed convergence for finding a zero of a function.&quot;&lt;/a&gt; ACM Transactions on Mathematical Software (TOMS) 1.4 (1975): 330-345.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21720" LastEditorUserId="6839" LastEditDate="2023-11-28T07:06:31.253" LastActivityDate="2023-11-28T07:20:15.587" Title="Dekker's root-finding method and fixed interval border (should shrink to root)" Tags="|nonlinear-equations|roots|" AnswerCount="2" CommentCount="2" ClosedDate="2018-12-11T16:11:43.480" ContentLicense="CC BY-SA 4.0" />
  <row Id="25006" PostTypeId="2" ParentId="25004" CreationDate="2016-09-19T09:55:04.337" Score="2" Body="&lt;p&gt;According to the way you defined your stencil in your picture, I think you made a mistake in your formula, that should be :&#xA;$$&#xA;v_{i+1,j}=(0.1)v_{i,j-1}+(0.8)v_{i,j}+(0.1)v_{i+1,j}&#xA;$$&#xA;You have switched $i$ and $j$, same thing in your initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation is very straightforward. Define a grid by using &lt;code&gt;meshgrid&lt;/code&gt;. Define &lt;code&gt;v&lt;/code&gt; as a matrix with the same dimension. The first two loops define the boundary conditions. And the last one is the difference formula. Then, all is in place to use &lt;code&gt;surf&lt;/code&gt; :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc;clear all;&#xA;&#xA;N=5;M=5;&#xA;[X,Y]=meshgrid(0:N,0:M);&#xA;v=zeros(N+1,M+1);&#xA;&#xA;for i=1:M+1&#xA;     v(i,1)=0.2*(i-1);&#xA;     v(i,end)=v(i,1)+25;&#xA;end&#xA;for j=1:N+1&#xA;    v(1,j)=(j-1)*(j-1);&#xA;end&#xA;&#xA;for i=1:N&#xA;    for j=2:M&#xA;        v(i+1,j)=0.1*v(i,j-1)+0.8*v(i,j)+0.1*v(i,j+1);&#xA;    end&#xA;end&#xA;&#xA;surf(X,Y,v)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-09-19T09:55:04.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25008" PostTypeId="1" CreationDate="2016-09-19T10:38:24.740" Score="1" ViewCount="813" Body="&lt;p&gt;I have made a 2d slice cut of a 3 dimension figure. When I select &quot;mesh&quot; to display the mesh on the slice, tecplot seems to connect the vertices in a strange way (it seems like two 2d meshes are superimposed). I found a similar question in the forum of tecplot but they did not answer it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;Here is a capture of the 2d slice cut&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cHNty.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cHNty.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21580" LastEditorUserId="21580" LastEditDate="2016-09-19T15:40:31.733" LastActivityDate="2016-09-19T19:16:57.807" Title="2d slice cut of tetrahedral mesh in tecplot 360 ex" Tags="|visualization|plotting|" AnswerCount="1" CommentCount="4" ClosedDate="2016-09-20T14:38:08.567" ContentLicense="CC BY-SA 3.0" />
  <row Id="25009" PostTypeId="2" ParentId="23704" CreationDate="2016-09-19T12:54:16.627" Score="1" Body="&lt;p&gt;Look at the &lt;a href=&quot;https://github.com/jMetal/jMetal&quot; rel=&quot;nofollow&quot;&gt;jMetal&lt;/a&gt; library &lt;a href=&quot;https://www.researchgate.net/post/Is_there_a_fully_functional_NSGA-III_implementation&quot; rel=&quot;nofollow&quot;&gt;1&lt;/a&gt; it is in Java, which is very similar to C++. It is very extensive and has several, and rather recent, multi objective methods. It is probably  the best MO library I have found out there in any language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I remember some professor had a C++ implementation of &lt;a href=&quot;https://www.researchgate.net/post/Is_there_a_fully_functional_NSGA-III_implementation&quot; rel=&quot;nofollow&quot;&gt;NSGA III&lt;/a&gt;. He could not reproduce the paper that proposed the algorithm , but it  may perfectly fit your needs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I added a thread that discusses NSGA III implementations. Also, I would like to comment that I have used the variation of NSGA II implemented in Matlab with many objectives ( &gt;10) and binary variables, and it worked pretty well for my application. So it is always good to try, even if people say that it is limited to a certain number of objectives. You also have to use you parameters intelligently (population size, mutation rate, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;J.J. Durillo, A.J. Nebro jMetal: a Java Framework for Multi-Objective Optimization.  Advances in Engineering Software 42 (2011) 760-771.&lt;/li&gt;&#xA;&lt;li&gt;A.J. Nebro, J.J. Durillo, M. Vergne Redesigning the jMetal Multi-Objective Optimization Framework.   GECCO (Companion) 2015, pp: 1093-1100. July 2015. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21728" LastEditorUserId="21728" LastEditDate="2016-09-19T22:59:30.190" LastActivityDate="2016-09-19T22:59:30.190" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25010" PostTypeId="2" ParentId="25008" CreationDate="2016-09-19T19:16:57.807" Score="1" Body="&lt;p&gt;Seeing the image, what you see is the intersection between a plane and the tetrahedra. Slicing tetrahedra generates new polygons, for instance, it can generate quadrilaterals (there is one on the bottom of your image). See also the right attached image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to visualize the mesh, it may be better to display the tetrahedra that &quot;straddle&quot; the clipping plane, as on the left attached image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The images were generated with geobox/vorpaview, both part of my open-source &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;geogram software http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xZRne.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xZRne.jpg&quot; alt=&quot;two slicing mode for visualizing tetrahedral grids. Left: straddling cells; right: slicing&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-09-19T19:16:57.807" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25012" PostTypeId="1" AcceptedAnswerId="25015" CreationDate="2016-09-20T02:13:49.367" Score="2" ViewCount="10989" Body="&lt;p&gt;I am trying to determine quality of my mesh elements using the Jacobian determinant as the measure. My algorithm takes vertices of nodes in triangular mesh and moves them around so as to form distorted triangles. &#xA;Let us assume my vertex P(x,y,z) moves to P'(x',y',z')&#xA;How to calculate the resulting change in mesh quality? One way to do this is to calculate jacobian of every triangle in mesh, but I cannot understand how to calculate the partial derivatives given only the displacement of nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the existing literature on Jacobian calculation seem to use a function to do so. Can anyone explain in simple terms?&lt;/p&gt;&#xA;" OwnerUserId="21517" LastActivityDate="2020-04-01T17:32:28.520" Title="Calculate Jacobian of triangular element given coordinates of vertices and displacements?" Tags="|finite-element|mesh|jacobian|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25013" PostTypeId="1" CreationDate="2016-09-20T06:57:15.043" Score="2" ViewCount="76" Body="&lt;p&gt;I am playing around with &lt;a href=&quot;http://libelemental.org/&quot; rel=&quot;nofollow&quot;&gt;Elemental's&lt;/a&gt; distributed singular value decomposition and am running into two particular issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Building the test at &lt;a href=&quot;https://github.com/elemental/Elemental/blob/master/tests/lapack_like/SVD.cpp&quot; rel=&quot;nofollow&quot;&gt;tests/lapack_like/SVD.cpp&lt;/a&gt;, and running with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mpirun -np 4 ./SVD --height 3 --width 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SVD residual was unacceptably large&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for the distributed &lt;code&gt;Complex&amp;lt;float&amp;gt;&lt;/code&gt; test (but sequential is OK).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, when I run with &lt;code&gt;--height 50 --width 52&lt;/code&gt;, I run into some problem &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;H and A must have the same height&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Elemental documentation for SVD is outdated, and I don't know what requirements there are on the dimensions of a matrix in order to take its SVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have Elemental built with pulled OpenBLAS, Scalapack, Metis and Parmetis. I could not manage to get libFLAME linked, even using the reference BLAS/LAPACK implementations.&lt;/p&gt;&#xA;" OwnerUserId="21722" LastEditorUserId="9667" LastEditDate="2016-09-20T12:54:14.617" LastActivityDate="2016-09-20T12:54:14.617" Title="Under what circumstances does Elemental's distributed SVD not work?" Tags="|linear-algebra|c++|mpi|lapack|svd|" AnswerCount="0" CommentCount="2" ClosedDate="2016-09-20T14:37:32.253" ContentLicense="CC BY-SA 3.0" />
  <row Id="25014" PostTypeId="1" CreationDate="2016-09-20T11:18:16.730" Score="4" ViewCount="122" Body="&lt;p&gt;In physical problems, it's quite common that we need to solve for specific eigenvalues in the complex plane, e.g. with a positive real part and negative imaginary part. In this case, we are looking for eigenvalues and associated eigenvectors in the second quadrant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am curious if the &lt;a href=&quot;http://www.ecs.umass.edu/~polizzi/feast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEAST algorithm&lt;/a&gt; could deliver faster and more accurate results for this kind of problem (dense as well), with respect to solving the full problem using &lt;code&gt;GEEV&lt;/code&gt; routine from LAPACK and do the sorting explicitly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have experience or suggestion?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="20688" LastEditDate="2018-06-16T09:49:25.453" LastActivityDate="2018-07-16T10:38:35.230" Title="Compute specific eigenvalues in the complex plane with Feast?" Tags="|linear-algebra|eigenvalues|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25015" PostTypeId="2" ParentId="25012" CreationDate="2016-09-20T12:50:23.327" Score="1" Body="&lt;p&gt;You can define multiple measures for the quality of your mesh. For example, &lt;a href=&quot;http://gmsh.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gmsh&lt;/a&gt; uses a criterion that measures the element shape. It is based on the element radii ratio, i.e., the ratio between the inscribed and circumscribed circles (See reference 1). I think that you can use the Jacobian to describe the quality of elements as well, although you might want to check reference 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the Jacobian, the first thing to mention is that is a function of the element coordinates in higher-order elements. Then, for those elements, you would need to compute a figure o merit per element, e.g., the average of the determinant or the maybe the norm. In the case of linear elements, you will have a transformation like the one described in the following image&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LHrqh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LHrqh.png&quot; alt=&quot;Transformation from given triangle to axis-aligned right-angled triangle&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this simple case the transformation is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{pmatrix}x\\ y \end{pmatrix} = \mathbf{T}\begin{pmatrix}r\\ s \end{pmatrix} \equiv [J]\begin{pmatrix}r\\ s \end{pmatrix}  + \begin{pmatrix}x_A\\ y_A \end{pmatrix} \enspace ,$$&lt;/span&gt;&#xA;with&#xA;&lt;span class=&quot;math-container&quot;&gt;$$[J] = \begin{bmatrix} x_B - x_A &amp;amp;x_C - x_A\\ y_B - y_A &amp;amp;y_C - y_A \end{bmatrix} \enspace ,$$&lt;/span&gt;&#xA;and &lt;span class=&quot;math-container&quot;&gt;$\det J = (x_B - x_A)(y_C - y_A) - (x_C - x_A)(y_B - y_A)$&lt;/span&gt;. That is the simplest way to think about the transformation, and you can just find it solving a linear system of equations, and it just depends on the position of your nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your problem, you would like to compute the Jacobian then apply the displacement to each node, and compute it one more time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Geuzaine, Christophe, and Jean‐François Remacle. &lt;a href=&quot;http://gmsh.info/doc/preprints/gmsh_paper_preprint.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Gmsh: A 3‐D finite element mesh generator with built‐in pre‐and post‐processing facilities.&quot;&lt;/a&gt; International Journal for Numerical Methods in Engineering 79.11 (2009): 1309-1331.&lt;/li&gt;&#xA;&lt;li&gt;Pébay, Philippe, and Timothy Baker. &lt;a href=&quot;http://www.ams.org/journals/mcom/2003-72-244/S0025-5718-03-01485-6/S0025-5718-03-01485-6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Analysis of triangle quality measures.&quot;&lt;/a&gt; Mathematics of Computation 72.244 (2003): 1817-1839.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2020-01-23T21:07:49.803" LastActivityDate="2020-01-23T21:07:49.803" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25016" PostTypeId="2" ParentId="25005" CreationDate="2016-09-20T13:46:13.900" Score="0" Body="&lt;p&gt;I think, Dekker method has bad behaviour if is more than one root in given interval. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this method on Cleve’s Corner (&lt;a href=&quot;http://blogs.mathworks.com/cleve/2015/10/12/zeroin-part-1-dekkers-algorithm/&quot; rel=&quot;nofollow&quot;&gt;http://blogs.mathworks.com/cleve/2015/10/12/zeroin-part-1-dekkers-algorithm/&lt;/a&gt;) where method is good enough for one root in interval.&lt;/p&gt;&#xA;" OwnerUserId="21720" LastActivityDate="2016-09-20T13:46:13.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25017" PostTypeId="2" ParentId="24491" CreationDate="2016-09-20T14:51:16.973" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;No, it's not necessary to apply relaxation &lt;em&gt;as long as the process converges to steady-state&lt;/em&gt;. Think of it as a correction:&#xA;\begin{equation*}&#xA;u^n = u^{n-1} + \gamma (\bar{u}^n-u^{n-1})&#xA;\end{equation*}&#xA;As you converge to steady-state where $u^n=u^{n-1}$, the correction $u^\prime=\bar{u}^n-u^{n-1}$ becomes zero and so the dependence on $\gamma$ drops out. Same holds for the pressure, no need to relax for consistency of the converged solution. Of course, with $\gamma=1$ you might not get convergence.&lt;/li&gt;&#xA;&lt;li&gt;There is no obvious relation. You may use a lower value for the relaxation parameter $\gamma_p$ for the pressure to compensate for a higher value of $\gamma_u$ or vice-versa.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2016-09-20T14:51:16.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25018" PostTypeId="1" CreationDate="2016-09-20T15:20:10.677" Score="0" ViewCount="76" Body="&lt;p&gt;From my knowledge if you fit geometrical objects into point clouds you want in general minimize the squared distances of the point cloud to your fitted objects. I do so with the downhill simplex method by minimizing the average squared distance. I fit cylinders. In rare input scenarios I end up having not good fitted cylinders - detected by visual validation. Those cylinders have one thing in common. Their axis is not perpendicular to the points's normals. I can modify my error term by multiplying the squared distance of a point to a cylinder with the cosinus of the angle between the point's normal and the cylinder axis. This cosinus is 1 in the worst case and 0 in the best case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a name of this technique and/or publications. I think the most generic formualtion of what I am doing would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modify the squared euclidean distance between input and output accounting to a change in curvature.&lt;/p&gt;&#xA;" OwnerUserId="20162" LastEditorUserId="549" LastEditDate="2016-09-20T19:56:51.497" LastActivityDate="2016-09-20T21:08:05.247" Title="Loooking for name of this geometrical optimization technique" Tags="|optimization|computational-geometry|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25019" PostTypeId="1" CreationDate="2016-09-20T15:40:23.897" Score="2" ViewCount="133" Body="&lt;p&gt;I heard about &lt;em&gt;Ramdisks&lt;/em&gt; some years ago and even set up one for testing on mymachine. I didn't test it thoroughly so I couldn't really judge the performance improvements (in comparison to plain a SSD).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ramdisks are in short parts of your working memory, which are allocated to a logical(software) disk. Therefore you can access them as an additional drive, put files and programs, as well as run simulations on them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programs cache in RAM anyway,  but a lot of programs write to files, where they store their mesh data, intermediate results etc. Depending on the internals of the program, improvements could be possible. In my case the use would be for Finite Element calculations. The tool I use was created (and is mostly maintained) as scientific project, so many optimizations could be missing. I'm aware of possible data loss, it is acceptable, though it has to be accounted for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any advantage to be gained by using ramdisks? Has anyone tried using ramdisks for their calculations, was there any worthwhile (work/reward) improvement? Are there cases where it makes sense to use them?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="16957" LastActivityDate="2016-09-20T20:49:39.190" Title="RAMdisks and finite element calculations" Tags="|finite-element|performance|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25021" PostTypeId="1" CreationDate="2016-09-20T16:40:26.787" Score="3" ViewCount="139" Body="&lt;p&gt;Recently,I dive into a set of somehow ancient Fortran codes and try to fully understand them. A large fraction of these codes are multiple layers of loops over many state variable dimensions, which are aiming to find policy function(fixed point). That's OK, but what's really annoyed me is the messy expression of the grid node index. For example, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do z=1,notimes&#xA;   do i=1,weignosims&#xA;      do t=1,nt&#xA;        ! main body of operations &#xA;        ! nodes index below&#xA;           node = (i-1)*(nt+ntr) + t                               ! To keep track of an individual's life&#xA;           index = (begnosims -1)*notimes + weignosims*(z-1) + i   ! To keep track of which individual in which economy &#xA;      enddo&#xA;   enddo&#xA;endo &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It seems the author want to stack all index into one array and keep track each of them. However, it's naturally a multiple array. Such expression is both confusing and error-prone. Moreover, in later case, the author also write something like &lt;code&gt;asset(index,age)&lt;/code&gt; or even define three dimension array to represent 1 state/node. So why he bother to write in the above way&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to know is whether is just a matter of programming style or this kind of style can gain some benefit (at least w.r.t Fortran)? If the second reason holds, then can I have some alternative way to &quot;trace node&quot; when I switch to some more modern scientific computation language, like Julia?&lt;/p&gt;&#xA;" OwnerUserId="21737" LastEditorUserId="9667" LastEditDate="2016-09-20T16:55:23.307" LastActivityDate="2016-09-20T17:29:26.623" Title="Alternative to messy grid node indexing within multiple layers of loops" Tags="|fortran|grid|julia|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25022" PostTypeId="2" ParentId="25021" CreationDate="2016-09-20T17:29:26.623" Score="3" Body="&lt;p&gt;This is not a good way to do modern programming for many reasons. First of all, as you pointed out, this kind of code is hard to read and maintain. Secondly, this tends to be done in old versions of Fortran for reasons which are no longer a problem. Namely, old versions of Fortran didn't have ways to &quot;bundle together&quot; objects. If you made different arrays for everything, then you'd have to pass them all together into functions, leading to large maintainable (and prone to error) function signatures. But lastly and most importantly, this can interfere with compiler optimizations. There are a lot of compiler optimizations that occur if you loop over contiguous arrays (loop unrollling, SIMD, etc) in an easy fashion. By using such odd indexing, there's a good chance that this will cause the compiler to not add these (modern) optimizations, leading to worse performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you probably shouldn't design large programs in this style if you are using a modern langaguage like Julia. Instead, you should exploit Julia's type system and multiple dispatch. If you naturally have multiple arrays associated with one item, put them together into a type. Write the &lt;code&gt;assert&lt;/code&gt; function to dispatch on this type. Etc. This will be much easier to read and will could allow you to loop in an easier fashion over your contiguous arrays, which will allow more compiler optimizations and better performance (indeed, I have beat many classic Fortran codes in benchmarks with DifferentialEquations.jl, so properly written Julia code will do this!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in many cases where a Fortran loop is used, Julia's broadcasting &lt;code&gt;.&lt;/code&gt; syntax can be used instead (in any case where you &quot;loop over all indices of an array&quot;). Unlike MATLAB, Julia's broadcast will &lt;a href=&quot;https://github.com/JuliaLang/julia/pull/17300&quot; rel=&quot;nofollow&quot;&gt;fuse loops&lt;/a&gt; and essentially perform the optimizations I mentioned to make the as optimal as the loops (and allow for in-place updates with no temporary arrays unlike is normally seen in MATLAB/NumPy-style vectorized code). This could greatly enhance readability without sacrificing performance.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-09-20T17:29:26.623" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25023" PostTypeId="2" ParentId="25019" CreationDate="2016-09-20T20:36:59.007" Score="2" Body="&lt;p&gt;Truly efficient programs on modern high performance computing architectures require a careful design of algorithm and data structures to take advantage of the full &lt;a href=&quot;https://en.wikipedia.org/wiki/Memory_hierarchy&quot; rel=&quot;nofollow&quot;&gt;memory hierarchy&lt;/a&gt;. (Not to speak about parallel programming issues, in which you may have multiple threads, multiple processes, SMP, NUMA, multiple interconnected nodes...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your question is rephrased as&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Will a legacy program, written when out-of-core storage was a choice driven by main memory constraints, benefit from moving its scratch files to a ramdisk?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would answer that the only way to know is by experiment. But as already pointed out by Kirill, I would not expect dramatic improvements, since almost every modern OS caches disk access.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the *nix world you should check the concept of &lt;a href=&quot;https://en.wikipedia.org/wiki/Tmpfs&quot; rel=&quot;nofollow&quot;&gt;tmpfs&lt;/a&gt; which can be implemented with just a couple of lines in &lt;code&gt;/etc/fstab&lt;/code&gt; and is much more flexible than an ordinary &lt;em&gt;ramdisk&lt;/em&gt;. See for example the discussion in &lt;a href=&quot;https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt&quot; rel=&quot;nofollow&quot;&gt;tmpfs.txt&lt;/a&gt; from the linux kernel.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2016-09-20T20:36:59.007" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25024" PostTypeId="2" ParentId="25019" CreationDate="2016-09-20T20:49:39.190" Score="2" Body="&lt;p&gt;Most modern operating systems already cache reads from and writes to files in main memory, and so you don't gain anything by using a RAM disk -- it's just another layer of data caching, and it is not going to be any faster than the caches the operating system keeps for you. On the other hand, you are giving up a sizable chunk of memory that you can no longer use for computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only place where it makes sense is if the file system you are working on is an network file system (e.g., NFS-mounted). In that case, operations such as linking executables that have a lot of random accesses into files become very slow because they tend to require frequent synchronization between the machine you're on and the file server (which may update the file on disk, or simply keep a copy of it in a cache). This synchronization over the network is expensive and one should try to avoid it. You can either avoid this by working on the local disk (e.g., in a &lt;code&gt;/tmp&lt;/code&gt; file system) or in a RAM disk. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-09-20T20:49:39.190" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25025" PostTypeId="2" ParentId="25018" CreationDate="2016-09-20T21:08:05.247" Score="1" Body="&lt;p&gt;I think what your looking for is a good parameterization of cylinder, as well as a fast scheme for performing gradient descent minimization, similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Iterative_closest_point&quot; rel=&quot;nofollow&quot;&gt;iterative closest point&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While many papers are available, I would redirect you to &lt;a href=&quot;http://ieeexplore.ieee.org/document/7335508/&quot; rel=&quot;nofollow&quot;&gt;this particular one&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Drost, Bertram, and Slobodan Ilic. &quot;Local Hough Transform for 3D Primitive Detection.&quot; &lt;em&gt;3D Vision (3DV), 2015 International Conference on. IEEE, 2015.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There, authors devise a scheme for detection, as well as refinement of the cylinders, with a new parameterization. The approach doesn't use the point normals, however I believe that such a treat would be appropriate. Yet, it is found to be effective enough, when one uses a &lt;em&gt;point-to-primitive&lt;/em&gt; distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure you'll find it useful. &lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2016-09-20T21:08:05.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25027" PostTypeId="1" CreationDate="2016-09-21T07:02:43.647" Score="3" ViewCount="1498" Body="&lt;p&gt;I am a beginner in computational science and programming. I am doing research in non linear solid mechanics analysis and using C++ for coding. I have been exploring various finite element open source programs and saw some nice works like Deal.ii, FEniCS, OOFEM, libmesh. But as a beginner in C++ programming and finite element , documentation of these programs are beyond my comprehension and confusing. I would like to see  some simple object oriented C++ codes for solving finite element problems. My objective is not studying finite element analysis but to understand the implementation of algebra behind it in C++ Object oriented frame work.     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding my work, I am clear about the procedures and concepts in my mind. I have made a prototype in MATLAB. Since MATLAB program takes too much time to get the result I decided to switch to C++ considering the ease of future work. But various data structures in C++ makes me troubled to select appropriate storage and structure for nodes, elements, constraints and loads. In MATLAB I imported prepared data (nodes, element connectivity, loads and boundary constraints) from spread sheet and used in built functions for calculations. In the C++ implementation, I am thinking to use a text-file (like in Abaqus/Calculix) for input. How do I implement it?           &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am searching for a well commented small finite element program source code for solving plane stress problems, like the kind of programs some university is using for teaching. I went through the book of S D Rajan, Numerical methods in C++ and got some insights about C++ data structures but still I need to go through some source codes for developing my own program. In later stages of my research I can make use of existing finite element libraries like Deal.ii and FEniCS.&lt;/p&gt;&#xA;" OwnerUserId="21659" LastEditorUserId="9667" LastEditDate="2016-09-24T16:27:57.860" LastActivityDate="2019-03-09T07:02:31.343" Title="Developing a C++ solid mechanics program" Tags="|finite-element|c++|fenics|deal.ii|solid-mechanics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25029" PostTypeId="1" CreationDate="2016-09-21T17:31:53.273" Score="1" ViewCount="453" Body="&lt;p&gt;I have a problem with the following equation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;V_r\frac{\partial C_A}{\partial r} + \frac{V_\theta}{r}\frac{\partial C_A}{\partial \theta} = \frac{2}{Pe_A} \left[ \frac{\partial ^2 C_A}{\partial r ^2} + \frac{2}{r} \frac{\partial C_A}{\partial r} - k C_A\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to solve this with finite difference technique. My discretization :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;V_r \frac{A_{i+1,j}-A_{i-1,j}}{2\Delta r} + \frac{V_\theta}{r} \frac{A_{i,j+1}-A_{i,j-1}}{2\Delta \theta} - \frac{2}{Pe} \left [ \frac{A_{i+1,j}-2A_{i,j}+A_{i-1,j}}{\Delta r ^2}+\frac{2}{r} \frac{A_{i+1,j}-A_{i-1,j}}{2\Delta r} - kA_{i,j}\right] = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first tried it with this, but because I used central-difference in doesn't work for $Pe(cell)&amp;gt;2$, I had strange oscillations at the boundary. So I thought I use a upwind scheme now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V_r$ and $V_\theta$ are given through polynoms as a function ($r, \theta$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for $\theta&amp;lt;90 °$ I Use second order upwind for the convection term $V_r &amp;gt; 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for $\theta &amp;gt;90 °$ in the other direction.&#xA;$V_\theta$ is always $&amp;lt;0$, so I only use this form for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is, for high Peclet number and low reaction constants, the concentration is rising up where it shouldn't. This happens in a region, where the convection term gets stronger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I supposed to do ? Is there a difference about using upwind in polar coordinates ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want go more in detail. This is my coordinate system :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem appears at the zero degress line. I also have seen that the problem is strongly dependent on $V_\theta$. If I multiply $V_\theta$ with 0.9 for example there is no problem. The concentration is not rising. I've looked at the speed vectors in more detail. The problem is vanishing when the speed vectors near the zero degree line are horizontal or show away from it. I have Neumann boundary condition there ( also at 180°). ($\frac{\partial C_A}{\partial \theta}=0 $). Now I also have seen that for the regions where this &quot;pseudo-source&quot; exist the boundary condition isn't met perfectly. There is a gradient. So I am thinking maybe this acts like a &quot;pseudo-source &quot;. I also have drichilet boundary condition for r=1 and r--&gt;infinite. ( My calculation begins at r=1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My grid number is the following ( for example I have three radial and five angular steps ). My notation is c(i,j), i is the radial direction and j the angular direction. So say for i=1, it is c(1,j) with j=0...4 from phi=0...180°. Let's have a look for zero degress with constants l_i :&#xA;$c10*l1+...c12*l2+c14*l4=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need no ghoistpoints here. I just sad c12=c14 and rearranged the equation to $c10*l1...+c12(l2+l4)$. &#xA;For 180 ° I need ghoistpoints because of second order upwind scheme I have c(i,j-2) which is outside my domain, but at 180 ° there is no problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I implement Neumann on another way ? Or do you have any other suggestions ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3VCYE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3VCYE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21748" LastEditorUserId="21748" LastEditDate="2016-09-26T07:11:49.610" LastActivityDate="2016-09-26T07:11:49.610" Title="Finite Difference advection-reaction-diffusion in spherical coordinates, problem with Diffusion" Tags="|finite-difference|advection-diffusion|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25031" PostTypeId="2" ParentId="24998" CreationDate="2016-09-22T01:06:29.080" Score="44" Body="&lt;p&gt;Since I just finished optimizing a lot of them in software, &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl&quot; rel=&quot;noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt;, I decided to just lay out a comparison of the main Order 4/5 methods. The Fehlberg method was left out because it's commonly known to be less efficient than the DP5 method.&lt;/p&gt;&#xA;&lt;h1&gt;Backstories&lt;/h1&gt;&#xA;&lt;h2&gt;Dormand-Prince 4/5&lt;/h2&gt;&#xA;&lt;p&gt;The Dormand-Prince method was developed to be accurate as a 4/5 pair with local extrapolation usage (i.e. step with the order 5 pair. This is because it was designed to be close to optimal (i.e. minimal) principle truncation error coefficient (under the restraint of also having the minimal number of steps to achieve order 5). It has an order 4 interpolation which is free but needs extra steps for an order 5 interpolation.&lt;/p&gt;&#xA;&lt;h2&gt;Cash-Karp 4/5&lt;/h2&gt;&#xA;&lt;p&gt;The Cash-Karp method was developed to satisfy different constraints, namely to deal with non-smooth problems better. They chose the &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt;, the percentage of the timestep in the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th step (i.e. &lt;span class=&quot;math-container&quot;&gt;$t+c_i \Delta t$&lt;/span&gt; is the time the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th step is calculated at) to be as uniform as possible, yet still achieve order 5. Then it also was derived to have embedded 1st, 2nd, 3rd, and 4th order methods with this uniformity of the &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt;. They are spaced in such a manner that you can find out where a stiff part starts by which difference is large. Moreover, note that the more stiff the equation, the worse a higher order method does (because it needs bounds on higher derivatives). So they develop a strategy that uses the 5 embedded methods to &amp;quot;quit early&amp;quot;: i.e. if you detect stiffness, stop at stage &lt;span class=&quot;math-container&quot;&gt;$i&amp;lt;6$&lt;/span&gt; to decrease the number of function calls and save time. So in the end, this &amp;quot;pair&amp;quot; was developed with a lot of other constraints in mind, and so there's no reason to expect it would be &amp;quot;more accurate&amp;quot;, at least as a 4/5 pair. If you add all of this other machinery then, on (semi-)stiff problems, it will be more accurate (but in that case you may want to use a different method like a W-Rosenbrock method). This is one reason why this pair hasn't become standard over the DP5 pair, but it still can be useful (maybe it would be good for a hybrid method that switches to a stiff solver when stiffness is encountered?).&lt;/p&gt;&#xA;&lt;h2&gt;Bogacki &amp;amp; Shampine 4/5&lt;/h2&gt;&#xA;&lt;p&gt;To round out the answer, let's discuss the Bogacki &amp;amp; Shampine pair that was mentioned in the comment. The BS5 method drops the constraint of &amp;quot;using the least function calls&amp;quot; (it uses 8 instead of 6) in order to do 2 things:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Get really low principle truncation error coefficients.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Produce an order 5 interpolation with lower error coefficients.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These coefficients are so low that for many problems with tolerances that users likely use, it measures as though it's 6th order. Their paper shows that for cheap function calls, this can be more efficient than DP5 by about the same amount as DP5 was over RKF5 (the Fehlberg method).&lt;/p&gt;&#xA;&lt;p&gt;You might put two-and-two together and see: wait a second, Shampine is the same person who developed the MATLAB ODE suite, this was after the BS5 pair paper was published, why doesn't MATLAB's &lt;code&gt;ode45&lt;/code&gt; use the BS5 pair? One reason is, it was mostly done before the BS5 pair was released. The other reason is that the &lt;code&gt;ode45&lt;/code&gt; function was developed to minimize time. While the BS5 pair is more efficient (i.e. gets lower accuracy), the purpose of &lt;code&gt;ode45&lt;/code&gt; is to have a good enough error to make a good enough plot. This means that, in order to deal with the large steps, it also produces two extra interpolated solutions between every step. For the DP5 method, there is a &amp;quot;free&amp;quot; order 4 interpolation, and so this is much faster than using BS5. Since it is also &amp;quot;accurate enough&amp;quot; at moderate tolerances, this method is set as the standard because it gives a better standard user experience than BS5 when doing interactive computing (so this choice was context-specific).&lt;/p&gt;&#xA;&lt;h2&gt;Tsitouras 4/5&lt;/h2&gt;&#xA;&lt;p&gt;Here's one fewer people know about. &lt;a href=&quot;http://users.uoa.gr/%7Etsitourasc/RK54_new_v2.pdf&quot; rel=&quot;noreferrer&quot;&gt;It's derived in this paper&lt;/a&gt;. It's derived using fewer assumptions than the DP5 method and tries to get a pair with lower principle truncation error coefficients. In its tests, it states that it achieves this. It also has a free order 4 interpolation like the DP5 method.&lt;/p&gt;&#xA;&lt;h1&gt;Numerical tests&lt;/h1&gt;&#xA;&lt;p&gt;I wrote the numerical package &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl&quot; rel=&quot;noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt; to be a pretty comprehensive set of solvers for Julia. Along the warpath, I implemented over 100 Runge-Kutta methods, and hand-optimized plenty. Three of the hand-optimized integrators are the DP5, BS5, and Tsit5 methods (I did not do CK5 because, as noted in the backstory, its main case is for problems that are kind of stiff. I think the better way to handle them is to use DP5/BS5 and switch to stiff solvers as necessary in a manner like LSODE, but that's a story for a different time) (one way to see they are close to optimal is that these methods are faster than the Hairer &lt;code&gt;dopri5&lt;/code&gt; implementations, so they are at least decent implementations). Tests between a lot of Runge-Kutta methods on nonstiff equations &lt;a href=&quot;https://benchmarks.sciml.ai/dev/README/#Table-of-Contents&quot; rel=&quot;noreferrer&quot;&gt;can be found on this benchmarks page&lt;/a&gt;. I am adding more as I go along, but you can see from the linear ODE and the Three-Body problem work precision diagrams, I measure the DP5 and Tsit5 methods to have almost identical efficiency, beating out the BS5 method in the linear ODE, while it's DP5 and BS5 that are almost identical on the Three-Body problem with Tsit5 behind. From this information, at least for now, I have settled on the DP5 method as the default, matching previous recommendations. That may change with future tests (or you could add benchmarks! Feel free to contribute, or Star the repo to give this effort more support).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;Conclusion&lt;/h1&gt;&#xA;&lt;p&gt;In conclusion, the Order 5 pairs go like this:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The Dormand-Prince 4/5 pair is a good go-to pair since it's well-optimized in terms of principle truncation error coefficient and has a cheap order 4 interpolation, which makes it fast for producing decent plots.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Cash-Karp pair has more constraints on it to better handle stiff equations. However, to get the full benefit you'll want to use the full algorithm with the 5 embedded methods.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Bogacki &amp;amp; Shampine Order 5 method may be the most efficient in terms of producing error per function calls (it has a double error estimator, so in harder problems, it probably does better), but that allows it to take larger timesteps. However, if you just want to produce a smooth plot, you then have to counter-act this method: use a lower tolerance (so it will take longer than DP5 but with less error) or use more interpolated steps. In the end, this meant that it might not be better for interactive applications, although it might be better for some scientific computing applications.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Tsitouras 4/5. Was developed fairly recently (2011) to beat out the DP5 in a head-to-head comparison. My tests don't give me a reason to believe that it's so much better than DP5 that it should now be considered the new standard method, but future tests may begin to side in its favor.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Edit&lt;/h3&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I did improve the Tsit5 implementation. It now does better than DP5 on most tests, both the DifferentialEquations.jl and the Hairer dopri implementations (though one might be surprised that the DifferentialEquations.jl implementations are actually faster, which of course helps the Tsit5 implementation). I now recommend it as the default order 4/5 method.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="20688" LastEditDate="2022-07-18T18:42:06.577" LastActivityDate="2022-07-18T18:42:06.577" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25032" PostTypeId="1" CreationDate="2016-09-22T07:59:36.343" Score="1" ViewCount="30" Body="&lt;p&gt;I am looking for a PDF describing Chinese character component-based layout engine algorithms. These should allow for recursive encodings of radical and non-radical components, and the encodings should account for the structural layout of characters based on these layouts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="21757" LastActivityDate="2016-09-22T16:33:41.640" Title="Chinese character component-based layout engine algorithms" Tags="|algorithms|graph-theory|languages|" AnswerCount="0" CommentCount="1" ClosedDate="2016-09-22T16:34:50.197" ContentLicense="CC BY-SA 3.0" />
  <row Id="25033" PostTypeId="2" ParentId="20855" CreationDate="2016-09-22T13:10:45.343" Score="2" Body="&lt;p&gt;Unfortunately, the &quot;standard&quot; BLAS implementations does not provide this feature. But due to the goals the scientific community want to achieve with the Exascale Computing serval &quot;reproducible&quot; BLAS implementation are in development. Reproducibility is not the same as minimal error evaluation but it is one necessary part of it. And the reproducible BLAS libraries might provide error bounds for the computed results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two libraries focusing on this are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ExBLAS - &lt;a href=&quot;https://exblas.lip6.fr/&quot; rel=&quot;nofollow&quot;&gt;https://exblas.lip6.fr/&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;ReproBLAS - &lt;a href=&quot;http://bebop.cs.berkeley.edu/reproblas/&quot; rel=&quot;nofollow&quot;&gt;http://bebop.cs.berkeley.edu/reproblas/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="569" LastActivityDate="2016-09-22T13:10:45.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25034" PostTypeId="2" ParentId="25027" CreationDate="2016-09-22T13:11:57.663" Score="1" Body="&lt;p&gt;This is not really a program as much as a big repository of codes. John Burkardt from Florida State University maintains a rich collection of scripts in C++, Matlab and Fortran for a large range of problems. You can check out the &lt;a href=&quot;http://people.sc.fsu.edu/~jburkardt/cpp_src/fem2d_pack/fem2d_pack.html&quot; rel=&quot;nofollow&quot;&gt;FEM 2D library&lt;/a&gt; or go up a level and look at other codes listed there. These have good commenting on them and you can look at the source directly.&lt;/p&gt;&#xA;" OwnerUserId="19452" LastActivityDate="2016-09-22T13:11:57.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25036" PostTypeId="1" CreationDate="2016-09-22T23:00:09.933" Score="0" ViewCount="85" Body="&lt;p&gt;my aim is to interpolate a function like this: I have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      |&amp;lt;--- h --&amp;gt;|&amp;lt;-------- k -----------&amp;gt;|    &#xA;      o----------o------------------------o&#xA;    x(i-1)      x(i)                   x(i+1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with respective values $f(x_i-h), f(x_i), f(x_i+k)$, now I want to approximate $f(x+k/2)$ using centered formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know for example that $f(x+k/2)\approx \frac{f(x_i)+f(x_i+k)}{2}$ but what if I want to use also $f(x-h)?$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks!&lt;/p&gt;&#xA;" OwnerUserId="21085" LastActivityDate="2016-10-23T04:14:17.270" Title="Interpolation over an unequally spaced grid" Tags="|interpolation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25037" PostTypeId="2" ParentId="25027" CreationDate="2016-09-23T01:37:32.730" Score="4" Body="&lt;p&gt;Specific answers to this question are probably time-limited.  However, the following general approach (from the great Eric S. Raymond) works very well:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Rule of Modularity: Write simple parts connected by clean&#xA;interfaces.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Clarity: Clarity is better than cleverness.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Composition: Design programs to be connected to other programs.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Separation: Separate policy from mechanism; separate interfaces   from engines.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Simplicity: Design for simplicity; add complexity only where you must.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Parsimony: Write a big program only when it is clear by   demonstration that nothing else will do.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Transparency: Design for visibility to make inspection and   debugging easier.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Robustness: Robustness is the child of transparency and   simplicity.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Least Surprise: In interface design, always do the least   surprising thing.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Silence: When a program has nothing surprising to say, it should say nothing.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Repair: When you must fail, fail noisily and as soon as possible.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Economy: Programmer time is expensive; conserve it in preference to machine time.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Optimization: Prototype before polishing. Get it working before you optimize it.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Diversity: Distrust all claims for &quot;one true way&quot;.&lt;/li&gt;&#xA;&lt;li&gt;Rule of Extensibility: Design for the future, because it will be here sooner than you think.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2016-09-23T01:37:32.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25038" PostTypeId="2" ParentId="25036" CreationDate="2016-09-23T03:22:51.237" Score="1" Body="&lt;p&gt;With three points, you can either do a linear interpolation which is $\mathcal{O}(h)$ or a quadratic interpolation which is $\mathcal{O}(h^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear interpolation is simple. Let $\theta \in [0,1]$ (the percentage along the interval from $x$ to $x+h$). Then,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x+\theta h) = \theta f(x) + (1-\theta)f(x+h) + \mathcal{O}(h)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's just the weighted average between the two points, putting more weight on the end you're closer to. Notice that this only uses two points. Also notice that if $\theta = \frac{1}{2}$, we recover the midpoint formula you had before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quadratic interpolation is exactly how it sounds: you interpolate by making a polynomial. Recall that in the general form we have that a quadratic is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = ax^2 + bx + c$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus we have 3 unknowns and 3 equations: $f(x) = y_1$, $f(x+h) = y_2$, and $f(x+k) = y_3$. Given that $k$, $h$, and the $y_i$ are known, this uniquely defines the quadratic through these points, and thus gives the coefficients $a$, $b$, and $c$. In this interval, we can then define $\theta \in [0,1] $ (the percentage to $k$) and approximate via&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x+\theta k) = a (x + \theta k)^2 + b (x + \theta k) + c + \mathcal{O}(k^2) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice the order here is 2, so this will be a slightly better approximation than the linear interpolation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let's do the math out for completeness. We get the equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ y_1 = a x^2 + bx + c $$&#xA;$$ y_2 = a x^2 + (b + 2ah)x + (c + bh + a h^2) $$&#xA;$$ y_3 = a x^2 + (b + 2ak)x + (c + bk + a k^2) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easier way to do this is write everything in terms of the coefficent we wish to find:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ y_1 = x^2 a + x b + c $$&#xA;$$ y_2 = (x^2 + 2hx + h^2) a + (x + h) b + c $$&#xA;$$ y_3 = (x^2 + 2kx + k^2) a + (x + k) b + c $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this is a linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left[\begin{array}{c}&#xA;y_{1}\\&#xA;y_{2}\\&#xA;y_{3}&#xA;\end{array}\right]=\left[\begin{array}{ccc}&#xA;x^{2} &amp;amp; x &amp;amp; 1\\&#xA;x^{2}+2hx+h^{2} &amp;amp; x+h &amp;amp; 1\\&#xA;x^{2}+2kx+k^{2} &amp;amp; x+k &amp;amp; 1&#xA;\end{array}\right]\left[\begin{array}{c}&#xA;a\\&#xA;b\\&#xA;c&#xA;\end{array}\right] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left[\begin{array}{c}&#xA;a\\&#xA;b\\&#xA;c&#xA;\end{array}\right]=\left[\begin{array}{ccc}&#xA;x^{2} &amp;amp; x &amp;amp; 1\\&#xA;x^{2}+2hx+h^{2} &amp;amp; x+h &amp;amp; 1\\&#xA;x^{2}+2kx+k^{2} &amp;amp; x+k &amp;amp; 1&#xA;\end{array}\right]^{-1}\left[\begin{array}{c}&#xA;y_{1}\\&#xA;y_{2}\\&#xA;y_{3}&#xA;\end{array}\right] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I threw this into Mathematica to get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left[\begin{array}{c}&#xA;a\\&#xA;b\\&#xA;c&#xA;\end{array}\right]= \left(&#xA;\begin{array}{c}&#xA; \frac{y_1 (h-k)-h y_3+k y_2}{h k (h-k)} \\&#xA; \frac{y_1 (-(h-k)) (h+k+2 x)+h y_3 (h+2 x)-k y_2 (k+2 x)}{h k (h-k)} \\&#xA; \frac{(k+x) \left(y_1 (h-k) (h+x)+k x y_2\right)-h x y_3 (h+x)}{h k (h-k)} \\&#xA;\end{array}&#xA;\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as the explicit equations for the coefficients. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-09-23T03:22:51.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25039" PostTypeId="1" AcceptedAnswerId="25043" CreationDate="2016-09-23T12:03:04.047" Score="2" ViewCount="1297" Body="&lt;p&gt;I am not very familiar with differential equations, nor physics in general.  I am trying to program an object falling with air resistance with the use of a numerical algorithm called Runge-Kutta. The object is going to fall from the same position every-time $x=0$, with the same start speed $v=0$ every time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I then need to work with the equation: $ma=F_d-mg$, where $F_d$ is the drag and it's variables are $\frac {1}{2} pdAv^2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ma=F_d-mg  \rightarrow ma = \frac {1}{2} pdAv^2-mg$, where $A$ is the area, a constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I solve for $a$, the acceleration I get the equation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a = \frac {\frac 12pdav^2}{m}-g$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the differential equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y'' = \frac {\frac 12pda(y')^2}{m}-g$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I am going to solve for this using the Runge-Kutta numerical method I am going to have to rewrite this into two first order differentials equations. ¨&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thus start by saying: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{dx}{dt} = v $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dv}{dt}=\frac {\frac 12pda(v)^2}{m}-g$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that I have my two first order differential equations I should be fine plugging them into the Runge-Kutta method, to obtain the velocity and the position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a picture of what I have programmed. Using the initial values: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$p = 0.5$&lt;/li&gt;&#xA;&lt;li&gt;$d = 1.275$&lt;/li&gt;&#xA;&lt;li&gt;$a = 0.003184$&lt;/li&gt;&#xA;&lt;li&gt;$m = 0.14529$&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$g = 9.81$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using UnityEngine;&#xA;using System.Collections;&#xA;&#xA;public class ODESOLVER1 : MonoBehaviour {}&#xA;&#xA;float t_n = 0; &#xA;float y_n = 0;&#xA;float h = 0.1;&#xA;float x_n = 0;&#xA;float p = 0.5;&#xA;float d = 1.275;&#xA;float a = 0.003184;&#xA;float m = 0.14529; &#xA;float g = 9.81; &#xA;&#xA;void Start () {&#xA;    for (int i = 0; i &amp;lt; 200; i++)&#xA;    {&#xA;        RungaKrutta4(t_n, y_n, x_n, h);&#xA;        Debug.Log(&quot;t_n=&quot; + t_n + &quot;y_n=&quot; + y_n + &quot;x_n=&quot; + x_n);&#xA;    };&#xA;}&#xA;&#xA;void Update () {}&#xA;&#xA;&#xA;private void repeatKrutta() {}&#xA;&#xA;private void RungaKrutta4(float t_n, float y_n, float x_n, float h) {&#xA;&#xA;    float k0 = h * function1(t_n, y_n, x_n);&#xA;    float l0 = h*function2(t_n, y_n, x_n);&#xA;    float k1 = h * function1(t_n + (h / 2), y_n + (k0 / 2), x_n + (l0 / 2));&#xA;    float l1 = h * function2(t_n + (h / 2), y_n + (k0 / 2), x_n + (l0 / 2));&#xA;    float k2 = h * function1(t_n + (h / 2), y_n + (k1 / 2), x_n + (l1 / 2));&#xA;    float l2 = h*function2(t_n + (h/2), y_n + (k1/2), x_n + (l1/2));&#xA;    float k3 = h * function1(t_n + (h / 2), y_n + (k2 / 2), x_n + (l2 / 2));&#xA;    float l3 = h*function2(t_n + (h/2), y_n + (k2/2), x_n + (l2/2));&#xA;&#xA;   this.t_n = t_n + h; &#xA;   this.y_n = y_n + ((h / 6) * (k0 + (2 * k1) + (2 * k2) + k3));&#xA;   this.x_n = x_n + ((h / 6) * (l0 + (2 * l1) + (2 * l2) + l3));&#xA;}&#xA;&#xA;private float function1(float t, float y, float x) {&#xA;    return (((0.5f * p * d * a * y * y)/m) - g);&#xA;}&#xA;&#xA;&#xA;private float function2(float t, float y, float x) {&#xA;    return (y);&#xA;}&#xA;&#xA;private void calculateDecimals(float value) {&#xA;    string s = System.Convert.ToString(value); &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My $y_{n+1}$ almost comes out correctly except for the fact that it becomes a much smaller value. If the speed should have been $0.0981$ with a timestep $h = 0.010$, it comes out as $0.000981.$  in the first iteration for example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My $x_{n+1}$ does not come correctly out at all. The first, second and third values should be as follows: $-0.00049, -0.00196, -0.0441 $ with a timestep of $h=0.010.$&lt;/p&gt;&#xA;" OwnerUserId="21769" OwnerDisplayName="David Lund" LastEditorUserId="9667" LastEditDate="2016-09-23T14:28:22.097" LastActivityDate="2016-09-23T15:36:29.187" Title="Object falling with air resistance using Runge-Kutta" Tags="|ode|computational-physics|runge-kutta|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="25041" PostTypeId="2" ParentId="24949" CreationDate="2016-09-23T14:22:07.623" Score="1" Body="&lt;p&gt;Thanks to everyone for their comments.&#xA;I've found the following good solution to my question.&#xA;Apparently, each task should have its own version of the code to isolate tasks from each other and to ensure reproducibility in a sense that it must be always clear what version of the code was used for the task.&#xA;The remaining question was: how to avoid cloning a git repository multiple times?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Git version 2.5+ allows creating multiple worktrees connected to the same repository.&#xA;Using multiple worktrees, you can avoid cloning the repository multiple times, and at the same time track all versions of the code in separate branches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the structure of the project will be like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;project&#xA;    code&#xA;    tasks&#xA;        01-2016-01-01-task-01&#xA;            code&#xA;        ...&#xA;        27-2016-03-06-task-27&#xA;            code&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where each directory &lt;code&gt;01-2016-01-01-task-01&lt;/code&gt; and similar will have its own copy of the code set to its own branch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To create additional worktrees, the following git command is used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd &amp;lt;project&amp;gt;/code&#xA;git worktree add -b 01-2016-01-01-task-01 \&#xA;    ../tasks/01-2016-01-01-task-01/code master&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which creates a branch &lt;code&gt;01-2016-01-01-task-01&lt;/code&gt; from &lt;code&gt;master&lt;/code&gt; and checks it out into the directory &lt;code&gt;../tasks/01-2016-01-01-task-01/code&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, when the work on the given task is finished, you can merge the corresponding branch into the &lt;code&gt;master&lt;/code&gt; branch.&lt;/p&gt;&#xA;" OwnerUserId="3254" LastActivityDate="2016-09-23T14:22:07.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25043" PostTypeId="2" ParentId="25039" CreationDate="2016-09-23T15:36:29.187" Score="4" Body="&lt;p&gt;I am not going to compile everything to see if this is your only mistake (you can &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl/blob/master/src/ode/ode_integrators.jl#L530&quot; rel=&quot;nofollow&quot;&gt;check here for an RK4 implementation&lt;/a&gt;). However, one clear error is that the fourth step has a stepsize &lt;code&gt;h&lt;/code&gt; and not &lt;code&gt;h/2&lt;/code&gt;. So you should have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float k3 = h * function1(t_n + h , y_n + k2, x_n + l2);&#xA;float l3 = h*function2(t_n + h, y_n + k2, x_n + l2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just for reference, the method with all halves (like you had) is only $\mathcal{O}(h^2)$, but it's more stable than RK4. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(2 * k1) + (2 * k2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is unnecessary,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2 * (k1 * k2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gets rid of an operation. Also, the update step should be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;this.y_n = y_n + ((1 / 6) * (k0 + (2 * k1) + (2 * k2) + k3));&#xA;this.x_n = x_n + ((1 / 6) * (l0 + (2 * l1) + (2 * l2) + l3));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i.e. get rid of the $h$'s there. Most people don't know this, but the RK methods actually respect units / dimensional analysis. Check the dimensions (where $f$ is a rate): &lt;code&gt;function1(t_n, y_n, x_n)&lt;/code&gt; is a rate, so &lt;code&gt;h * function1(t_n, y_n, x_n)&lt;/code&gt; has the same dimensions as &lt;code&gt;y_n&lt;/code&gt;. Since you multiplied the &lt;code&gt;h&lt;/code&gt; already, your &lt;code&gt;k&lt;/code&gt;'s aren't rates, but actually the same units as the &lt;code&gt;y_n&lt;/code&gt; and &lt;code&gt;x_n&lt;/code&gt; and so the last step is just a linear combination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's no reason to &lt;code&gt;t_n + (h / 2)&lt;/code&gt; so many times. Save that value to cut down on operations. &lt;code&gt;RungaKrutta4&lt;/code&gt; should be &lt;code&gt;RungaKutta4&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are all of the mistakes I could find while quickly skimming over your RK4, and RK4 is one of the simplest methods (you have no adaptive timestepping, stabilization, interpolation, or optimizations like within-method multithreading, explicit fused-multiply added, etc.). This is one reason why, for most people, I would recommend that they use tried and tested ODE solver packages.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-09-23T15:36:29.187" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25045" PostTypeId="2" ParentId="24949" CreationDate="2016-09-23T17:15:00.707" Score="0" Body="&lt;p&gt;Part of your question was addressed in a recent publication &quot;Good Enough Practices in Scientific Computing&quot; (reference 1), primarily in point 4. I added the whole list, though.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Data Management&lt;/strong&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Save the raw data.&lt;/li&gt;&#xA;&lt;li&gt;Create the data you wish to see in the world&lt;/li&gt;&#xA;&lt;li&gt;Create analysis-friendly data.&lt;/li&gt;&#xA;&lt;li&gt;Record all the steps used to process data.&lt;/li&gt;&#xA;&lt;li&gt;Anticipate the need to use multiple tables.&lt;/li&gt;&#xA;&lt;li&gt;Submit data to a reputable DOI-issuing repository so that others can access and cite it.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Software&lt;/strong&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Place a brief explanatory comment at the start of every program.&lt;/li&gt;&#xA;&lt;li&gt;Decompose programs into functions.&lt;/li&gt;&#xA;&lt;li&gt;Be ruthless about eliminating duplication.&lt;/li&gt;&#xA;&lt;li&gt;Always search for well-mantained software libraries that do what you need.&lt;/li&gt;&#xA;&lt;li&gt;Test libraries before relying on them.&lt;/li&gt;&#xA;&lt;li&gt;Give functions and variables meaningful names.&lt;/li&gt;&#xA;&lt;li&gt;Make dependecies and requirements explicit.&lt;/li&gt;&#xA;&lt;li&gt;Do not comment and uncomment sections of code to control a program's behavior.&lt;/li&gt;&#xA;&lt;li&gt;Provide a simple example or test data set.&lt;/li&gt;&#xA;&lt;li&gt;Submit code to a reputable DOI-issuing repository.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Collaboration&lt;/strong&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create an overview of your project.&lt;/li&gt;&#xA;&lt;li&gt;Create a shared public &quot;to-do&quot; list.&lt;/li&gt;&#xA;&lt;li&gt;Make the license explicit.&lt;/li&gt;&#xA;&lt;li&gt;Make the project citable.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Project Organization&lt;/strong&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Put each project in its own directory, which is name after the project.&lt;/li&gt;&#xA;&lt;li&gt;Put text documents associated with the project in the doc directory.&lt;/li&gt;&#xA;&lt;li&gt;Put raw data and metadata in a data directory, and files generated during cleanup and analysis in a results directory.&lt;/li&gt;&#xA;&lt;li&gt;Put project source code in the src directory.&lt;/li&gt;&#xA;&lt;li&gt;Put external scripts, or compiled programs in the bin directory.&lt;/li&gt;&#xA;&lt;li&gt;Name files to reflect their content or function.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Keeping Track of Changes&lt;/strong&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Back up (almost) everything created by a human as soon as it is created.&lt;/li&gt;&#xA;&lt;li&gt;Keep changes small.&lt;/li&gt;&#xA;&lt;li&gt;Share changes frequently.&lt;/li&gt;&#xA;&lt;li&gt;Create, mantain, and use a checklist for saving and sharing changes to the project.&lt;/li&gt;&#xA;&lt;li&gt;Store each project in a folder that is mirrored off the the reasearcher's working machine.&lt;/li&gt;&#xA;&lt;li&gt;Add a file called CHANGELOG.txt to the project's docs subfolder.&lt;/li&gt;&#xA;&lt;li&gt;Copy the entire project whenever a significant change has been made.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Manuscripts&lt;/strong&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write manuscripts using online tools with rich formatting, change tracking, and reference management.&lt;/li&gt;&#xA;&lt;li&gt;Include a PUBLICATIONS file in the project's doc directory.&lt;/li&gt;&#xA;&lt;li&gt;Write the manuscript in a plain text format that permits version control.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Wilson, Greg, Jennifer Bryan, Karen Cranston, Justin Kitzes, Lex Nederbragt, and Tracy K. Teal. &lt;a href=&quot;http://arxiv.org/pdf/1609.00037.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;Good Enough Practices in Scientific Computing.&quot;&lt;/a&gt; arXiv preprint arXiv:1609.00037 (2016)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Wilson, Greg, D. A. Aruliah, C. Titus Brown, Neil P. Chue Hong, Matt Davis, Richard T. Guy, Steven HD Haddock et al. &lt;a href=&quot;http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001745&quot; rel=&quot;nofollow&quot;&gt;&quot;Best practices for scientific computing.&quot;&lt;/a&gt; PLoS Biol 12, no. 1 (2014): e1001745.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2016-09-23T17:15:00.707" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25048" PostTypeId="2" ParentId="24949" CreationDate="2016-09-24T01:13:49.793" Score="3" Body="&lt;p&gt;While Greg Wilson's publications in @nicoguaro's answer give great advice, I also think that the situation you describe would best be handled by giving your solver options for runtime configuration. &lt;a href=&quot;http://arxiv.org/abs/1407.2905&quot; rel=&quot;nofollow&quot;&gt;This article by Jed Brown&lt;/a&gt; (also a SciComp user) describes some good high-level practices, and you can also see examples of runtime configuration using PETSc in &lt;a href=&quot;http://www.mcs.anl.gov/uploads/cels/papers/P2010-0112.pdf&quot; rel=&quot;nofollow&quot;&gt;http://www.mcs.anl.gov/uploads/cels/papers/P2010-0112.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that if you're changing parameters, these should all be runtime options, so you can run your code with &lt;code&gt;./mycode -parameter_a 1.7 -parameter_b 2.5&lt;/code&gt; and change the parameters at runtime again with &lt;code&gt;./mycode -parameter_a 2.6 -parameter_b 3.4&lt;/code&gt;. In an ideal world, you should never have to recompile your code to run experiments. In practice, of course, setting up runtime options requires some infrastructure, but if you can do it, the investment is well worth the effort because it will make running concurrent experiments with different parameter values easy.&lt;/p&gt;&#xA;" OwnerUserId="276" LastActivityDate="2016-09-24T01:13:49.793" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25050" PostTypeId="1" CreationDate="2016-09-24T06:17:22.503" Score="1" ViewCount="163" Body="&lt;p&gt;The Dirichlet boundary condition is $n \times \vec{E} = n \times \vec{g}$. I don't know how to calculate the boundary value. Any kind of help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19457" LastActivityDate="2016-09-24T06:17:22.503" Title="How to apply Dirichlet boundary condition for lowest order Nedelec element over tetrahedral domain?" Tags="|finite-element|boundary-conditions|electromagnetism|" AnswerCount="0" CommentCount="2" ClosedDate="2019-06-11T23:26:24.747" ContentLicense="CC BY-SA 3.0" />
  <row Id="25053" PostTypeId="1" AcceptedAnswerId="25059" CreationDate="2016-09-24T17:35:18.593" Score="4" ViewCount="1194" Body="&lt;p&gt;For 1D finite difference, the resulting linear system is tri-diagonal and can be solved in &lt;code&gt;O(n)&lt;/code&gt; using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;nofollow&quot;&gt;Thomas algorithm&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve a finite difference system in 3D. &lt;a href=&quot;http://www.techmat.vgtu.lt/~inga/Files/Quarteroni-SkaitMetod.pdf&quot; rel=&quot;nofollow&quot;&gt;Numerical Methods&lt;/a&gt; by Quarteroni et al goes over some theory behind extending the Thomas algorithm to block linear systems. This method seems to be ideal for me because it is still &lt;code&gt;O(n)&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an algorithmic point of view however, I'm not sure how to proceed. Is there a reference that covers the implementation of this algorithm for a linear system such as the one resulting from 3D finite difference?&lt;/p&gt;&#xA;" OwnerUserId="9728" LastActivityDate="2016-09-25T01:46:59.223" Title="Thomas algorithm for 3D finite difference" Tags="|linear-algebra|numerical-analysis|finite-difference|linear-solver|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25054" PostTypeId="1" AcceptedAnswerId="25057" CreationDate="2016-09-24T18:30:59.627" Score="9" ViewCount="2301" Body="&lt;p&gt;Motivated by &lt;a href=&quot;https://math.stackexchange.com/a/147841/5519&quot;&gt;this top answer&lt;/a&gt; to the question: &lt;a href=&quot;https://math.stackexchange.com/q/146480/11268&quot;&gt;Why is convexity more important than &lt;strong&gt;quasi-convexity&lt;/strong&gt; in optimization?&lt;/a&gt;, I am now hoping to understand why &lt;strong&gt;convex&lt;/strong&gt; problems are easy to optimize (or at least easier than &lt;strong&gt;quasiconvex&lt;/strong&gt; problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some of the most efficient algorithms for convex optimization, and why can't they be used effectively in &lt;strong&gt;quasiconvex&lt;/strong&gt; problems? &lt;/p&gt;&#xA;" OwnerUserId="13" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-09-27T18:39:10.450" Title="Why are convex problems easy to optimize?" Tags="|optimization|convex-optimization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25056" PostTypeId="2" ParentId="25054" CreationDate="2016-09-24T22:06:03.390" Score="7" Body="&lt;p&gt;You can try to apply a convex optimization algorithm to a non-convex optimization problem, and it might even converge to a local minimum, but having only local information about the function, you'll never be able to conclude that you've in fact found the global minimum.  The most important theoretical property of convex optimization problems is that any local minimum (in fact, any stationary point) is also a global minimum.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms for global optimization of non-convex problems must have some sort of global information (e.g. Lipschitz continuity of the function) in order to prove that a solution is a global minimum.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your specific question about why a convex optimization algorithm might fail on a quasi-convex problem, suppose that your convex optimization algorithm happens to be started at a &quot;flat spot&quot; on the graph of the objective function.  There's no local information in the gradient to tell you where to go next.  For a convex problem you could simply stop, knowing that you were already at a local (and thus global) minimum point.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-09-24T22:06:03.390" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="25057" PostTypeId="2" ParentId="25054" CreationDate="2016-09-24T22:38:08.113" Score="5" Body="&lt;p&gt;Most of the best modern methods for large-scale optimization involve making a local quadratic approximation to the objective function, moving towards the critical point of that approximation, then repeating. This includes Newton's method, L-BFGS, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A function can only be locally well-approximated by a quadratic with a minimum if the Hessian at the current point is positive definite. If the Hessian is indefinite, then either&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The local quadratic approximation is a good local approximation to the objective function and is therefore a saddle surface. Then using this quadratic approximation would suggest moving towards a saddle point, which is likely to be in the wrong direction, or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The local quadratic approximation is forced to have a minimum by construction, in which case it is likely to be a poor approximation to the original objective function.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(The same sort of issues arise if the Hessian is negative-definite, in which case it locally looks like an upside-down bowl)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, these methods will work best if the Hessian is positive definite everywhere, which is equivalent to convexity for smooth functions.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Of course, all good modern methods have safeguards in place to ensure convergence when moving through regions where the Hessian is indefinite - E.g., line search, trust regions, stopping a linear solve when a direction of negative curvature is encountered, etc. However, in such indefinite regions the convergence is generally much slower, since full curvature information about the objective function cannot be used.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2016-09-27T18:39:10.450" LastActivityDate="2016-09-27T18:39:10.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25058" PostTypeId="1" CreationDate="2016-09-24T23:09:48.867" Score="1" ViewCount="458" Body="&lt;p&gt;I'm trying to do biologically based locomotion animation.&lt;/p&gt;&#xA;&lt;p&gt;I have a Hill-Type Muscle Model (see reference 1) that I've implemented in code. I can find the initial lengths of the tendons, then do a step and find the new velocities and forces.&lt;/p&gt;&#xA;&lt;p&gt;My question is, how do I use this information to simulate the muscle?&lt;/p&gt;&#xA;&lt;p&gt;Specifically, we will call our muscle a &lt;strong&gt;Muscle Tendon Complex&lt;/strong&gt; (MTC) that has four parts:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;strong&gt;contractile element&lt;/strong&gt; (CE) which models the active force production&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;strong&gt;parallel elastic element&lt;/strong&gt; (PEE) which is in parallel with the CE&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;strong&gt;serial elastic element&lt;/strong&gt; (SEE) which is in series to the CE, and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;strong&gt;serial damping element&lt;/strong&gt; (SDE) which is in parallel to the SEE&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Each of these have a length &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, where&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$l_{SEE}=l_{SDE}, l_{PEE}=l_{CE} \text{ and } l_{MTC}=l_{SEE}+l_{CE}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and each has a velocity &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. We also know a relationship between the force &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; that each tendon produces:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$F_{CE} + F_{PEE} = F_{SEE} + F_{SDE}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the output force &lt;span class=&quot;math-container&quot;&gt;$F_{MTC}$&lt;/span&gt; of the muscle is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$F_{MTC}=F_{SEE}+F_{SDE}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then the muscle has some activation value &lt;span class=&quot;math-container&quot;&gt;$0.001 \leq q \leq 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What I know so far is that you do the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Start the muscle not moving (&lt;span class=&quot;math-container&quot;&gt;$i_{MTC}=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$i_{CE}=0$&lt;/span&gt;), then choose a reasonable starting length for your tendons. They use &lt;span class=&quot;math-container&quot;&gt;$l_MTC=0.264$&lt;/span&gt; so I'll use that too. Then run a solver to find the &lt;span class=&quot;math-container&quot;&gt;$l_{CE}$&lt;/span&gt; such that the force generated by the muscle while not moving is 0 to find the initial &lt;span class=&quot;math-container&quot;&gt;$l_{CE}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Given the current velocities and lengths, calculate the force generated by each element and a new velocity &lt;span class=&quot;math-container&quot;&gt;$i_{CE}$&lt;/span&gt;. if &lt;span class=&quot;math-container&quot;&gt;$i_{CE}&amp;gt;0$&lt;/span&gt;, recalculate it using other equations. This part I can do.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate &lt;span class=&quot;math-container&quot;&gt;$F_{MTC}=F_{SEE}+F_{SDE}$&lt;/span&gt; which is the output of our muscle tendon complex model.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;Integrate &lt;span class=&quot;math-container&quot;&gt;$i_{CE}$&lt;/span&gt; to get &lt;span class=&quot;math-container&quot;&gt;$l_{CE}$&lt;/span&gt; along with the ...&amp;quot;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;Multi-body simulation that calculates the generated motion from &lt;span class=&quot;math-container&quot;&gt;$F_{MTC}$&lt;/span&gt;, which results in new &lt;span class=&quot;math-container&quot;&gt;$l_{MTC}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i_{MTC}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;. Now repeat from step 2&amp;quot;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;4 and 5 are direct quotes from the paper (including the &amp;quot;...&amp;quot;), but I'm not sure what they mean or what I'm supposed to do. Is there something trivial here I am missing?&lt;/p&gt;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Haeufle, D. F. B., et al. &lt;a href=&quot;http://wiki.ifs-tud.de/_media/seminar_3m/3m_2014/haeufle_2014b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Hill-type muscle model with serial damping and eccentric force–velocity relation.&amp;quot;&lt;/a&gt; Journal of biomechanics 47.6 (2014): 1531-1536.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8275" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-09-28T14:16:00.043" Title="Simulating a Hill-Type muscle model" Tags="|simulation|nonlinear-equations|computational-biology|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25059" PostTypeId="2" ParentId="25053" CreationDate="2016-09-25T01:46:59.223" Score="6" Body="&lt;p&gt;The problem is that 2d and 3d discretizations are not block tridiagonal. The only tridiagonal decomposition would be a 2x2 decomposition that encompasses the entire matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this fact is a great pity. If there &lt;em&gt;was&lt;/em&gt; a tridiagonal decomposition, we would have a way of getting an $O(N)$ algorithm from a purely algebraic perspective (i.e., by just looking at the matrix). Instead, the only way to get $O(N)$ algorithms are ones that employ multigrid methods that take into account &lt;em&gt;where the matrix actually comes from&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-09-25T01:46:59.223" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25060" PostTypeId="1" CreationDate="2016-09-25T02:06:11.227" Score="0" ViewCount="185" Body="&lt;p&gt;I have a 2-dimensional system which I would like to solve numerically (I'm using finite difference method right now), and its an eigenvalue problem. I have a term that looks like $H\psi(x,y) = [-\frac{1}{2}(\frac{\partial^2}{\partial x^2} +\frac{\partial^2}{\partial y^2})+ i(y\frac{\partial}{\partial x} - x\frac{\partial}{\partial y}) + (x^2 + y^2)]\psi(x,y)$. Because of the strange middle term that mixes $x$ and $y$, I can't just solve for $x$ and $y$ directions independently like for a 2-dimensional harmonic oscillator, which I'm currently using &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19678/computational-methods-for-finding-the-energy-eigenvalues-of-the-time-independent/19679#19679&quot;&gt;this code&lt;/a&gt; for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is I'm not sure how to &quot;mix&quot; $x$ and $y$ dimensions, how I encode that into the recurrence relation and generate a matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Boundary conditions: Assume an infinite plane, so wavefunction tends to zero.&lt;/p&gt;&#xA;" OwnerUserId="21772" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-10-08T22:44:40.500" Title="How to compute matrix representation of $\hat{y}\frac{\partial}{\partial x}$?" Tags="|linear-algebra|pde|finite-difference|python|quantum-mechanics|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="25063" PostTypeId="1" AcceptedAnswerId="25064" CreationDate="2016-09-25T17:51:40.103" Score="3" ViewCount="1141" Body="&lt;p&gt;I have read &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8470/intuitive-motivation-for-bfgs-update&quot;&gt;this&lt;/a&gt; and other threads on this site on BFGS, but I still don't have a clear understanding of what it's meant by &lt;strong&gt;low-rank&lt;/strong&gt; updates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I read the following in &lt;a href=&quot;http://www.deeplearningbook.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The approach adopted by quasi-Newton methods (of which the BFGS&#xA;  algorithm is the most prominent) is to approximate the inverse of the&#xA;  Hessian $H$, with a matrix $M_t$ that is iteratively refined by &lt;strong&gt;low&#xA;  rank&lt;/strong&gt; updates to become a better approximation of $H^{−1}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What's exactly low-rank update? How is it different from, say, a high-rank update?&lt;/p&gt;&#xA;" OwnerUserId="13" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-09-27T02:39:28.993" Title="Low-rank updates in BFGS" Tags="|optimization|newton-method|quasi-newton|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25064" PostTypeId="2" ParentId="25063" CreationDate="2016-09-25T19:31:00.990" Score="7" Body="&lt;p&gt;A low rank update to an $n$ by $n$ matrix $A$ is an update of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=A+UU^{T}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is matrix with $n$ rows but very few columns (typically just one or two.)  If the matrix $UU^{T}$ has $k$ columns, then its rank is at most $k$, so this is a low-rank update to the $A$ matrix.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Low rank updates are important in computational linear algebra and optimization because you can update the LU or Cholesky factorizaton of a matrix after a low-rank update without having to factor the updated matrix from scratch.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-09-25T20:38:57.857" LastActivityDate="2016-09-25T20:38:57.857" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25067" PostTypeId="1" AcceptedAnswerId="25224" CreationDate="2016-09-26T13:25:12.547" Score="0" ViewCount="543" Body="&lt;p&gt;The last couple of hours I have been looking for an unconditionally stable method to solve the convection-diffusion equation within a 3D inhomogeneous material. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the well known diffusion-convection equation:&#xA;$$\rho(\vec{r})C_p(\vec{r})\frac{\partial T(\vec{r},t)}{\partial t}=\nabla [\kappa(\vec{r})\nabla T(\vec{r},t)]-\rho(\vec{r})C_p(\vec{r})\nabla[\vec{u}T(\vec{r},t)]+g(\vec{r},t)$$ where $\rho$ is the material density, $C_p$ is the heat capacity, $\kappa$ is the thermal conductivity, $\vec{u}$ is the velocity vector and $g$ represents the heat source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I haven't found any method which can be used for that. The articles I came across all made severe simplifications like $\kappa$=const., ignored the convection term (2nd term on the R.H.S.) or reduced the problem to 1 or 2 dimensions.&#xA;So I'm wondering now if this can be solved at all.&lt;/p&gt;&#xA;" OwnerUserId="21797" LastEditorUserId="21797" LastEditDate="2016-09-27T09:09:00.560" LastActivityDate="2016-10-17T14:00:38.453" Title="Stable implicit method to solve convection-heat diffusion in 3D" Tags="|pde|finite-difference|diffusion|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25069" PostTypeId="1" AcceptedAnswerId="25106" CreationDate="2016-09-26T23:41:54.290" Score="0" ViewCount="146" Body="&lt;p&gt;I'm trying to learn SPH by implementing different methods. I am currently following Divergence Free Smoothed Particle Hydrodynamics By Jan Bender [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe I understand the method and the math behind it. However I am not a able to correct the density error by finding the pressure forces because they are very small. It is a predictor corrector algorithm, so I have to use the pressure forces in a Jacobi iterator to find the correct velocities that will make the densities equal the rest density to enforce incompressiblity. However even after 1000 iterations the system doesn’t converge, but it is moving in the right direction. I've had similar problems with PCISPH and IISPH.  I could only get the first one to work by manually scaling the resulting pressure forces.The pressure forces in IISPH were too large, still trying to get that work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can speed up the convergence in DFSPH of the iterator by manually scaling the calculated stiffness values. However this feels wrong. Is it okay to scale values like pressure values to get a SPH simulation working? I think I I am missing something important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.interactive-graphics.de/index.php/research/93-divergence-free-smoothed-particle-hydrodynamics&quot; rel=&quot;nofollow&quot;&gt;http://www.interactive-graphics.de/index.php/research/93-divergence-free-smoothed-particle-hydrodynamics&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21805" LastEditorUserId="16910" LastEditDate="2016-09-27T08:59:07.107" LastActivityDate="2016-10-03T17:40:07.287" Title="Smoothed particle hydrodynamics Implicit methods" Tags="|sph|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25070" PostTypeId="2" ParentId="25063" CreationDate="2016-09-27T02:39:28.993" Score="2" Body="&lt;p&gt;To add on Brian's answer. The idea behind the low rank update is to find what an approximation that is &quot;good enough&quot;, thus saving on computation resources. The BFGS method approximates the hessian with low rank updates by enforcing properties of the real hessian (e.g symmetry, etc.). You can also have a full update every certain number of iterations and a low rank update for the remaining iterations.&lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2016-09-27T02:39:28.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25072" PostTypeId="1" CreationDate="2016-09-27T04:00:05.283" Score="2" ViewCount="199" Body="&lt;p&gt;I'm somewhat struggling with how they are getting this scheme. This is a problem from Morton &amp;amp; Mayers book on numerical pde solutions. I think they are using a forward difference approximation for the 2nd order x-derivative but I just can't seem to figure out how. Sad day for me, but I would appreciate any insight. I think the notation of r,s is hard for me to comprehend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks Greatly&lt;a href=&quot;https://i.stack.imgur.com/mkbAm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mkbAm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21806" LastActivityDate="2016-12-27T20:08:11.510" Title="An interesting numerical pde problem" Tags="|numerical-analysis|finite-difference|parabolic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25073" PostTypeId="1" CreationDate="2016-09-27T05:47:49.670" Score="4" ViewCount="2599" Body="&lt;p&gt;I'm trying to solve the PDE for $c(r,t)$&#xA;$$c_t=(1/r)(rJ)_r$$&#xA;using Crank-Nicolson, and I'm having difficulty with the boundary conditions. $J$ is the flux, the initial condition is $c(0,r)=c_{init}$, and the flux is 0 at both boundaries $r=r_0$ and $r=r_1$.&#xA;The flux is $J=rDc_r-s\omega^2r^2c$; $D$ and $s$ may be functions of $c$, but are often constants, as they are treated in this example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for the Crank-Nicolson scheme I use these approximations:&#xA;$$c_t=(c_{n+1,j}-c_{n,j})/\Delta t$$&#xA;$$c_{rr}=\left({1\over 2}\right)\left[{c_{n,j+1}-2c_{n,j}+c_{n,j-1}\over (\Delta r)^2}+{c_{n+1,j+1}-2c_{n+1,j}+c_{n+1,j-1}\over (\Delta r)^2}\right]$$&#xA;$$c_r=\left({1\over 2}\right)\left[{c_{n,j+1}-c_{n,j-1}\over \Delta r}+{c_{n+1,j+1}-c_{n+1,j-1}\over \Delta r}\right]$$&#xA;$$c=\left({1\over 2}\right)\left[c_{n,j}+c_{n+1,j}\right]$$&#xA;where $n=$ the time step ($t=t_0+n\Delta t$), and $j=$ the position ($r=r_0+j\Delta r$, $j=0,...,J$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the boundary conditions, I want $J(r_0,t)=0$ and $J(r_1,t)=0$ for all $t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first attempt was to use &quot;ghost points&quot; at $j=-1$ and $j=J+1$. I used the centered difference at $j=0$ and $j=J$, respectively, for the derivative $c_r$, and the values $c_{n,0}$ and $c_{n,J}$:&#xA;$$c_r={c_{n,1}-c_{n,-1}\over 2\Delta r}$$&#xA;$$c_r={c_{n,J+1}-c_{n,J-1}\over 2\Delta r}$$&#xA;Substituting these approximations into the expression for the flux $J$ yields $c_{n,-1}$ in terms of $c_{n,0}$ and $c_{n,1}$, and $c_{n,J+1}$ in terms of $c_{n,J-1}$ and $c_{n,J}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the ghost points eliminated, the $J+1 \times J+1$ tridiagonal matrices ${\bf A}$ and ${\bf B}$ are used to solve for the vector ${\bf c}_{n+1}$ from ${\bf c}_n$ for each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet, when the numerical solution is calculated, instead of the expected exponential curve, becoming large near $r=r_1$, $c$ becomes tiny everywhere, and the  values over time make it apparent that there is flux out at both boundaries. That is the issue I am grappling with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about using a Crank-Nicolson average to approximate $c_r$ at the boundaries, but then I get four additional unknowns, and I don't know how to deal with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="21808" LastActivityDate="2016-10-05T06:59:28.467" Title="How to handle boundary conditions in Crank-Nicolson solution of IVP-BVP?" Tags="|boundary-conditions|crank-nicolson|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25074" PostTypeId="1" CreationDate="2016-09-27T12:15:43.080" Score="1" ViewCount="305" Body="&lt;p&gt;I am wondering how to draw a sample complexity plot similar to the following figure which shows the estimated number of samples to incur no more than 10 percent generalisation error on average for the least squares algorithm for classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GEdiQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GEdiQ.png&quot; alt=&quot;Sample Complexity Least Squeres&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem assumes that we have $m$ patterns $\mathbf{x}_1,...,\mathbf{x}_m$ sampled uniformly at random from $\{−1,1\}^n$, where each label is defined as $y_i:=x_{i,1}$ (s.t. the label of pattern $\mathbf{x}$ is just its first coordinate). And the goal is to estimate the sample complexity as a function of the dimension ($n$) of the dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The m (examples) versus n (dimension of dataset) plot also includes error bars indicating the standard deviation for the estimates of m. I'm not interested in that, just the function estimation.&lt;/p&gt;&#xA;" OwnerUserId="21787" LastEditorUserId="21787" LastEditDate="2016-09-27T14:42:10.163" LastActivityDate="2016-09-27T14:42:10.163" Title="Generalization error and Sample Complexity estimation for Least Squares" Tags="|machine-learning|complexity|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25075" PostTypeId="2" ParentId="25072" CreationDate="2016-09-27T14:19:44.827" Score="1" Body="&lt;p&gt;They seem to use the standard explicit scheme to approximate the heat equation:&#xA;$$&#xA;\frac{U_{r,s}^{n+1} - U_{r,s}^n}{\Delta t}=\frac{1}{(\Delta x)^2} \delta^2_x U_{r,s}^n +&#xA;\frac{1}{(\Delta y)^2} \delta^2_y U_{r,s}^n &#xA;$$&#xA;where&#xA;$$&#xA;\delta^2_x U_{r,s}^n := U_{r+1,s}^n - 2 U_{r,s}^n + U_{r-1,s}^n&#xA;$$&#xA;and analogously for $\delta^2_y U_{r,s}^n$. Now boundary conditions $u_x=0$ at points corresponding to $r=0$ are approximated by&#xA;$$&#xA;\frac{U_{1,s}^n - U_{-1,s}^n}{2 \Delta x} = 0&#xA;$$ &#xA;so for the auxilliary value one obtains $U_{-1,s}^n = U_{1,s}^n$. Put this to the scheme (the first formula in this answer) for $r=0$ and you get &#xA;what it is written in the exercise, i.e.&#xA;$$&#xA;\frac{U_{0,s}^{n+1} - U_{0,s}^n}{\Delta t}=\frac{2}{(\Delta x)^2} (U_{1,s}^n- U_{0,s}^n) +&#xA;\frac{1}{(\Delta y)^2} \delta^2_y U_{r,s}^n &#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-11-27T19:35:15.770" LastActivityDate="2016-11-27T19:35:15.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25076" PostTypeId="1" CreationDate="2016-09-27T14:32:54.183" Score="7" ViewCount="454" Body="&lt;p&gt;Is there a faster way to compute the nuclear norm (trace norm, sum of singular values) of a matrix A than computing SVD(A) directly (or diagonalizing A^*A)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am particularly interested in the case where A is square. Assuming that A is real would be OK too. I am thinking in the limit of large matrices.&lt;/p&gt;&#xA;" OwnerUserId="21812" LastEditorUserId="21812" LastEditDate="2016-09-27T14:57:21.023" LastActivityDate="2016-11-26T20:04:18.050" Title="What is the fastest way to compute the sum of the singular values of a matrix?" Tags="|svd|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25077" PostTypeId="2" ParentId="25076" CreationDate="2016-09-27T15:33:04.313" Score="1" Body="&lt;p&gt;Depending on your scenario, perhaps you could make use of the result from this paper by Recht, Fazel and Parrilo, &lt;em&gt;&lt;a href=&quot;http://pages.cs.wisc.edu/~brecht/papers/07.rfp.lowrank.pdf&quot; rel=&quot;nofollow&quot;&gt;&quot;Guaranteed Minimum-Rank Solutions of Linear Matrix Equations&#xA;via Nuclear Norm Minimization&quot;&lt;/a&gt;&lt;/em&gt;, &lt;strong&gt;2007&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the matrix $X$ has rank upper-bounded by $r$, then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left \Vert X \right\Vert_* = \mathrm{inf}_{L,R} \left[\frac{1}{2}\left \Vert L \right\Vert_F^2 + \frac{1}{2}\left \Vert R \right\Vert_F^2 : X = LR^{T}\right ]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://guppy.mpe.nus.edu.sg/~mpexuh/papers/Stochastic_online_pca.pdf&quot; rel=&quot;nofollow&quot;&gt;Feng, Xu and Yan&lt;/a&gt; used this result to derive their online robust PCA algorithm to remove any need for a singular value decomposition.&lt;/p&gt;&#xA;" OwnerUserId="8094" LastActivityDate="2016-09-27T15:33:04.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25078" PostTypeId="2" ParentId="25067" CreationDate="2016-09-27T17:21:56.517" Score="2" Body="&lt;p&gt;It's a standard advection-diffusion equation. As long as your coefficients are bounded away from zero, there is really no difficulty associated with this equation with the possible exception of the fact that you need to stabilize the advection term if it dominates the diffusion term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would use a standard finite element method, plus something like SUPG for the advection term, and a backward Euler or Crank-Nicolson scheme for the time discretization. It's that easy :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-09-27T17:21:56.517" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25079" PostTypeId="1" CreationDate="2016-09-28T08:11:09.030" Score="0" ViewCount="993" Body="&lt;p&gt;I am trying to solve a differential equation system &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x´=Ax\quad \text{with } x(0) = f(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in Python, where $A$ indeed is a complex sparse matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now i have been solving the system using the &lt;code&gt;scipy.integrate.complex_ode&lt;/code&gt; class in the following way.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def to_solver_function(time,vector):&#xA;    sendoff = np.dot(A, np.transpose(vector))&#xA;    return sendoff&#xA;&#xA;solver = complex_ode(to_solver_function)&#xA;solver.set_initial_value(f(x),0)&#xA;&#xA;solution = [f(x)]&#xA;for time in time_grid:&#xA;    next = solver.integrate(time)&#xA;    solution.append(next)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This has been working OK, but I need to &quot;tell the solver&quot; that my matrix is sparse. I figured out that I should use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Asparse = sparse.lil_matrix(A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but how do I change my solver to work with this?&lt;/p&gt;&#xA;" OwnerUserId="21819" LastEditorUserId="9667" LastEditDate="2016-09-28T14:00:25.387" LastActivityDate="2016-10-29T08:08:11.183" Title="Use of scipy sparse in ode solver" Tags="|python|ode|sparse-matrix|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25081" PostTypeId="2" ParentId="20694" CreationDate="2016-09-28T15:59:57.037" Score="2" Body="&lt;p&gt;Let plane triangle $\mathcal T_i$ have vertices $\mathrm a_i, \mathrm b_i, \mathrm c_i \in \mathbb R^2$. Let $\Delta_2$ be the &lt;em&gt;standard&lt;/em&gt; $2$-&lt;a href=&quot;https://en.wikipedia.org/wiki/Simplex&quot; rel=&quot;nofollow noreferrer&quot;&gt;simplex&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If two plane triangles do intersect, i.e., $\mathcal T_1 \cap \mathcal T_2 \neq \emptyset$, then there exist $\eta_1, \eta_2 \in \Delta_2$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix} | &amp;amp; | &amp;amp; |\\ \mathrm a_1 &amp;amp; \mathrm b_1 &amp;amp; \mathrm c_1\\  | &amp;amp; | &amp;amp; |\end{bmatrix} \eta_1 = \begin{bmatrix} | &amp;amp; | &amp;amp; |\\ \mathrm a_2 &amp;amp; \mathrm b_2 &amp;amp; \mathrm c_2\\  | &amp;amp; | &amp;amp; |\end{bmatrix} \eta_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the triangle intersection problem can be reduced to &lt;strong&gt;linear programming&lt;/strong&gt;. Choosing an arbitrary objective function, say, the zero function, we have a linear program in $(\eta_1, \eta_2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \mathrm 0_3^T \eta_1 + \mathrm 0_3^T \eta_2 \\ \text{subject to} &amp;amp; \begin{bmatrix} | &amp;amp; | &amp;amp; |\\ \mathrm a_1 &amp;amp; \mathrm b_1 &amp;amp; \mathrm c_1\\  | &amp;amp; | &amp;amp; |\end{bmatrix} \eta_1 - \begin{bmatrix} | &amp;amp; | &amp;amp; |\\ \mathrm a_2 &amp;amp; \mathrm b_2 &amp;amp; \mathrm c_2\\  | &amp;amp; | &amp;amp; |\end{bmatrix} \eta_2 = \mathrm 0_2\\ &amp;amp; 1_3^T \eta_1 = 1\\ &amp;amp; 1_3^T \eta_2 = 1\\ &amp;amp; \eta_1, \eta_2 \geq 0_3\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the linear program is infeasible, then the intersection of the two triangles is empty.&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2017-03-27T18:18:09.740" LastActivityDate="2017-03-27T18:18:09.740" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25082" PostTypeId="1" AcceptedAnswerId="25084" CreationDate="2016-09-28T16:26:09.277" Score="2" ViewCount="1787" Body="&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a plot of a graph $y$ and its derivative $dy/dx$ calculated using python's &lt;code&gt;numpy.gradient&lt;/code&gt;which approximates the derivative with finite differences using a central scheme. Clearly there is a problem in that unphysical kinks have been introduced in the result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since differentiation will act to amplify any high frequency content in the original data, numerical differentiation is always tricky although here my data is relatively smooth and noise free. Does anyone recognise this type of behaviour and what can be done to get a proper derivative? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some info on the data:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y$ is data that is sampled from a computational mesh and has many more sample points for small $x$, getting coarser as $x$ increases. The data you see plotted is found by linearly interpolating on a uniform number of data points. However because of the non-uniform spacing I have had to use a large number of points so that the data points for small $x$ are effectively the &quot;true&quot; data values, but at high $x$ there are many points in between the &quot;true&quot; ones that are just linear interpolations, adding no real information to the graph. At first I thought maybe there is some floating point rounding problems that are causing   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VhDa8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VhDa8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT: MCVE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate the problem here is a small subset of the data to produce this figure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qs3bC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qs3bC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;# Subset of data such that&#xA;# sub_data = data[50:150,:]&#xA;&#xA;sub_data = np.asarray([[1.700089160294737667e-05, 9.632603526115417480e-01],&#xA;[1.733388671709690243e-05, 9.813255667686462402e-01],&#xA;[1.766688365023583174e-05, 9.993913173675537109e-01],&#xA;[1.799987876438535750e-05, 1.017457604408264160e+00],&#xA;[1.833267015172168612e-05, 1.035513401031494141e+00],&#xA;[1.866566708486061543e-05, 1.053580760955810547e+00],&#xA;[1.899866219901014119e-05, 1.071648478507995605e+00],&#xA;[1.933165913214907050e-05, 1.089716911315917969e+00],&#xA;[1.966444870049599558e-05, 1.107774615287780762e+00],&#xA;[2.000245694944169372e-05, 1.126114964485168457e+00],&#xA;[2.033545388258062303e-05, 1.144184470176696777e+00],&#xA;[2.066844899673014879e-05, 1.162254691123962402e+00],&#xA;[2.100123856507707387e-05, 1.180314183235168457e+00],&#xA;[2.133423549821600318e-05, 1.198385119438171387e+00],&#xA;[2.166586818930227309e-05, 1.216382265090942383e+00],&#xA;[2.199886512244120240e-05, 1.234277606010437012e+00],&#xA;[2.233165650977753103e-05, 1.251089692115783691e+00],&#xA;[2.266465162392705679e-05, 1.267913579940795898e+00],&#xA;[2.299764855706598610e-05, 1.284739017486572266e+00],&#xA;[2.333043812541291118e-05, 1.301555633544921875e+00],&#xA;[2.366844637435860932e-05, 1.318636536598205566e+00],&#xA;[2.400144148850813508e-05, 1.335466384887695312e+00],&#xA;[2.433443842164706439e-05, 1.352297544479370117e+00],&#xA;[2.466722798999398947e-05, 1.369119763374328613e+00],&#xA;[2.500022492313291878e-05, 1.385953545570373535e+00],&#xA;[2.533322003728244454e-05, 1.402788400650024414e+00],&#xA;[2.566621697042137384e-05, 1.419624686241149902e+00],&#xA;[2.599900835775770247e-05, 1.436451792716979980e+00],&#xA;[2.633200347190722823e-05, 1.453290104866027832e+00],&#xA;[2.666500040504615754e-05, 1.470129728317260742e+00],&#xA;[2.699799551919568330e-05, 1.486970305442810059e+00],&#xA;[2.733579822233878076e-05, 1.504054188728332520e+00],&#xA;[2.766743091342505068e-05, 1.520827651023864746e+00],&#xA;[2.800042602757457644e-05, 1.537671327590942383e+00],&#xA;[2.833321741491090506e-05, 1.554505705833435059e+00],&#xA;[2.866621252906043082e-05, 1.571351289749145508e+00],&#xA;[2.899920946219936013e-05, 1.588197588920593262e+00],&#xA;[2.933220639533828944e-05, 1.605044960975646973e+00],&#xA;[2.966499596368521452e-05, 1.621882915496826172e+00],&#xA;[2.999799289682414383e-05, 1.638731956481933594e+00],&#xA;[3.033098801097366959e-05, 1.655581831932067871e+00],&#xA;[3.066398494411259890e-05, 1.672432422637939453e+00],&#xA;[3.100178582826629281e-05, 1.689526319503784180e+00],&#xA;[3.133478094241581857e-05, 1.706378579139709473e+00],&#xA;[3.166777969454415143e-05, 1.723231554031372070e+00],&#xA;[3.200077480869367719e-05, 1.740085244178771973e+00],&#xA;[3.233356619603000581e-05, 1.756929397583007812e+00],&#xA;[3.266656131017953157e-05, 1.773784637451171875e+00],&#xA;[3.299955642432905734e-05, 1.790640354156494141e+00],&#xA;[3.333234781166538596e-05, 1.806897401809692383e+00],&#xA;[3.366534292581491172e-05, 1.822490572929382324e+00],&#xA;[3.399697743589058518e-05, 1.838021636009216309e+00],&#xA;[3.433498568483628333e-05, 1.853852272033691406e+00],&#xA;[3.466777343419380486e-05, 1.869441509246826172e+00],&#xA;[3.500077218632213771e-05, 1.885041832923889160e+00],&#xA;[3.533376730047166348e-05, 1.900644183158874512e+00],&#xA;[3.566676605259999633e-05, 1.916247844696044922e+00],&#xA;[3.599955380195751786e-05, 1.931843996047973633e+00],&#xA;[3.633255255408585072e-05, 1.947450876235961914e+00],&#xA;[3.666554766823537648e-05, 1.963059425354003906e+00],&#xA;[3.699854278238490224e-05, 1.978669643402099609e+00],&#xA;[3.733133416972123086e-05, 1.994271874427795410e+00],&#xA;[3.766432928387075663e-05, 2.009885072708129883e+00],&#xA;[3.800233753281645477e-05, 2.025733470916748047e+00],&#xA;[3.833533264696598053e-05, 2.041349411010742188e+00],&#xA;[3.866812403430230916e-05, 2.056957483291625977e+00],&#xA;[3.900111914845183492e-05, 2.072576045989990234e+00],&#xA;[3.933411426260136068e-05, 2.088196039199829102e+00],&#xA;[3.966690564993768930e-05, 2.103807926177978516e+00],&#xA;[3.999854016001336277e-05, 2.119366645812988281e+00],&#xA;[4.033153527416288853e-05, 2.134990215301513672e+00],&#xA;[4.066453038831241429e-05, 2.150615453720092773e+00],&#xA;[4.099732177564874291e-05, 2.166232347488403320e+00],&#xA;[4.133032052777707577e-05, 2.181859731674194336e+00],&#xA;[4.166832513874396682e-05, 2.197722673416137695e+00],&#xA;[4.200132389087229967e-05, 2.213352441787719727e+00],&#xA;[4.233411164022982121e-05, 2.228973865509033203e+00],&#xA;[4.266711039235815406e-05, 2.244606018066406250e+00],&#xA;[4.300010550650767982e-05, 2.260239124298095703e+00],&#xA;[4.333310062065720558e-05, 2.275873422622680664e+00],&#xA;[4.366589200799353421e-05, 2.291499376296997070e+00],&#xA;[4.399888712214305997e-05, 2.307135581970214844e+00],&#xA;[4.433188223629258573e-05, 2.322772979736328125e+00],&#xA;[4.466467362362891436e-05, 2.338401794433593750e+00],&#xA;[4.499766873777844012e-05, 2.352876663208007812e+00],&#xA;[4.533567698672413826e-05, 2.367459297180175781e+00],&#xA;[4.566867210087366402e-05, 2.381829023361206055e+00],&#xA;[4.600146712618879974e-05, 2.396192073822021484e+00],&#xA;[4.633309799828566611e-05, 2.410506963729858398e+00],&#xA;[4.666609311243519187e-05, 2.424882888793945312e+00],&#xA;[4.699908822658471763e-05, 2.439260959625244141e+00],&#xA;[4.733187961392104626e-05, 2.453632116317749023e+00],&#xA;[4.766487472807057202e-05, 2.468014001846313477e+00],&#xA;[4.799787348019890487e-05, 2.482398033142089844e+00],&#xA;[4.833086859434843063e-05, 2.496783494949340820e+00],&#xA;[4.866365998168475926e-05, 2.511162519454956055e+00],&#xA;[4.900166459265165031e-05, 2.525767564773559570e+00],&#xA;[4.933466334477998316e-05, 2.540158748626708984e+00],&#xA;[4.966766209690831602e-05, 2.554551839828491211e+00],&#xA;[5.000044984626583755e-05, 2.568938016891479492e+00]])&#xA;&#xA;&#xA;x = sub_data[:,0]&#xA;y = sub_data[:,1]&#xA;dx = x[1]-x[0]&#xA;&#xA;dydx = np.gradient(y,dx)&#xA;&#xA;plt.figure()&#xA;fig, ax1 = plt.subplots(figsize=(8,4))&#xA;ax2 = ax1.twinx()&#xA;&#xA;ax1.plot(x,y,'k-')&#xA;ax2.plot(x,dydx, 'r-')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="7864" LastEditDate="2016-09-28T19:21:02.650" LastActivityDate="2016-09-28T20:10:42.667" Title="Data corruption when taking gradient of numerical data in python" Tags="|finite-difference|python|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="25083" PostTypeId="2" ParentId="25082" CreationDate="2016-09-28T20:00:17.087" Score="2" Body="&lt;p&gt;Here's a plot of the derivative evaluated by first performing an spline fit of the data. Compared to the finite difference approach, the result appears more natural.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.interpolate import UnivariateSpline&#xA;&#xA;spl = UnivariateSpline(sub_data[:,0],sub_data[:,1])&#xA;&#xA;x_range = np.linspace(sub_data[0,0], sub_data[-1,0],1000)&#xA;plt.plot(x_range,spl.derivative(1)(x_range),'b', label = 'spline')&#xA;plt.plot(x,dydx, 'r-', label = 'finite diff.')&#xA;plt.legend(loc = 'best')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ylETP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ylETP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2016-09-28T20:00:17.087" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25084" PostTypeId="2" ParentId="25082" CreationDate="2016-09-28T20:05:22.427" Score="6" Body="&lt;p&gt;The $x$-coordinates of your data points are not equally spaced (&lt;code&gt;x[1:]-x[:-1]&lt;/code&gt; is not constant), so &lt;code&gt;numpy.gradient&lt;/code&gt; is not applicable because it assumes that the data is equally spaced (&lt;a href=&quot;http://docs.scipy.org/doc/numpy/reference/generated/numpy.gradient.html&quot;&gt;http://docs.scipy.org/doc/numpy/reference/generated/numpy.gradient.html&lt;/a&gt;). &#xA;Even just forward differences on their own would be better than using an inaccurate value of $\Delta x$ in centered differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With forward differences, the small bumps disappear, but the discontinuities remain, so those probably come from the data itself. You can smooth them by constructing a spline interpolant (&lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/interpolate.html&quot;&gt;http://docs.scipy.org/doc/scipy/reference/interpolate.html&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-09-28T20:10:42.667" LastActivityDate="2016-09-28T20:10:42.667" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25086" PostTypeId="1" AcceptedAnswerId="25087" CreationDate="2016-09-28T23:17:51.840" Score="2" ViewCount="155" Body="&lt;p&gt;Most mesh generation software seems to be aimed at building nicely shaped elements for FEM. I'm curious about a different situation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to numerically integrate over an irregular region. I don't need to solve for any unknowns in this region -- just integrate a known function. And I need to integrate over many such regions as fast as possible. My current approach is to mesh the region (using either Triangle or Tetgen, depending on 2d vs 3d) and then perform Gaussian quadrature over each individual tri/tet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I suspect that these meshing algorithms (which are a bottleneck) are doing far more work than necessary for this use case. Differences:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Small internal angles are fine.&lt;/li&gt;&#xA;&lt;li&gt;The occasional zero volume element would be fine -- integrating over a zero volume element will give just zero.&lt;/li&gt;&#xA;&lt;li&gt;Nonconforming elements are not a problem.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Essentially, anything goes as long as the mesh covers the whole volume and there is no overlap between cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of research that addresses this situation?&lt;/p&gt;&#xA;" OwnerUserId="5516" LastActivityDate="2016-09-29T06:19:07.570" Title="Are there mesh generation methods that allow ugly elements?" Tags="|quadrature|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25087" PostTypeId="2" ParentId="25086" CreationDate="2016-09-29T05:08:01.933" Score="2" Body="&lt;p&gt;Since you don't care about the quality of the tetrahedra (or triangles), you could quickly tetrahedralize your shape with the following 2-step procedure that is commonly used in computer graphics and animation:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Partition the input nonconvex polyhedron into a union of convex polyhedra organized in a binary space partition tree (&lt;a href=&quot;https://en.wikipedia.org/wiki/Binary_space_partitioning&quot; rel=&quot;nofollow noreferrer&quot;&gt;BSP tree&lt;/a&gt;), by means of the standard recursive cut plane algorithm (successively cutting the interior volume into 2 pieces with successive facets).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Further break down each of these convex polyhedron into tetrahedra by choosing any vertex and connecting it with edges to all other vertices.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The ordering of the cut facets in step 1 and the choice of vertex in step 2 are can be chosen however you like (this will affect the quality of the resulting triangulation); random is usually a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once your shape is tetrahedralized, you can do standard quadrature on each tetrahedron in the usual way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 2D example see the picture below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bgbdT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bgbdT.png&quot; alt=&quot;bsp based triangulation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The extension to 3D is straightforward but it's more difficult to draw. In 3D the cut edges in the BSP step become cut planes, but the edges in the tetrahedralizing step are still edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of good tutorials about this online that were written for the Quake engine (A popular 3D computer game from the 1990's on which many other subsequent 3d engines were based).&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2016-09-29T05:08:01.933" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25088" PostTypeId="2" ParentId="25086" CreationDate="2016-09-29T06:19:07.570" Score="3" Body="&lt;p&gt;In 3D, if you are using tetgen, you can deactivate insertion of additional points and optimization of mesh quality. When I want to to that, I am using the following flags:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-p (input data is surfacic)&#xA;-O0 (do not optimize mesh)&#xA;-YY (do not insert Steiner points on boundary)&#xA;-AA (generete region tags for each shell) (if needed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;tetgen command line switches are documented here [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am unsure that Gaussian quadrature on very skinny elements will be stable enough. If you do that, I'd recommend to evaluate the quality by running the algorithm on both an optimized mesh and a non-optimized one and compare the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://wias-berlin.de/software/tetgen/switches.html&quot; rel=&quot;nofollow&quot;&gt;http://wias-berlin.de/software/tetgen/switches.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-09-29T06:19:07.570" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25089" PostTypeId="2" ParentId="25079" CreationDate="2016-09-29T08:07:53.240" Score="1" Body="&lt;p&gt;For Scipy’s ODE module, the function that you feed it (in your case &lt;code&gt;to_solver_function&lt;/code&gt;) as a blackbox that it provides with a state and that returns a vector. It does not care about what happens inside it, in particular it never touches &lt;code&gt;A&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the only thing that you need to do is to ensure that your matrix–vector multiplication inside &lt;code&gt;to_solver_function&lt;/code&gt; is aware of the sparseness of your matrix. I am not very familiar with scipy.sparse, but there is a &lt;a href=&quot;http://docs.scipy.org/doc/scipy/reference/sparse.html#matrix-vector-product&quot; rel=&quot;nofollow&quot;&gt;paragraph of documentation&lt;/a&gt; on this and it tells me that the following should do the job:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sendoff = A.dot(vector)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Sidenote and blatant self-advertising: If you want to speed up things, I wrote &lt;a href=&quot;http://github.com/neurophysik/jitcode&quot; rel=&quot;nofollow&quot;&gt;a wrapper around Scipy’s ODE&lt;/a&gt; that just-in-time-compiles the derivative (in your case &lt;code&gt;to_solver_function&lt;/code&gt;) to speed up things. Given that in your case, the most computation time should be spent on the matrix–vector multiplication and you do that already pretty efficiently, it may give you as much of a boost. Also, it does not handle complex numbers yet, so you would have to do that manually.&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2016-09-29T08:07:53.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25090" PostTypeId="1" AcceptedAnswerId="25092" CreationDate="2016-09-29T10:45:10.493" Score="0" ViewCount="63" Body="&lt;p&gt;For some time I wrote every bit of my code by myself without using external sources. However after my problems became more complicated I decided to use external libraries. For example to save and inspect the solutions of my computations, I chose to use vtk. I understand, that I cannot write everything on my own, but I am worried about new releases of these external libraries that might not be compatible with their previous versions. Should I adapt my code all the time to the newer releases? I am afraid that if I will have to reinstall my system in the future, I won't be able to run my code. &lt;/p&gt;&#xA;" OwnerUserId="14907" LastEditorUserId="14907" LastEditDate="2016-09-29T11:24:57.870" LastActivityDate="2016-09-29T15:01:31.307" Title="New releases of libraries" Tags="|libraries|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25091" PostTypeId="2" ParentId="25073" CreationDate="2016-09-29T12:04:33.967" Score="1" Body="&lt;p&gt;Let me give an answer that is a general comment on prescribed zero flux for advection-diffusion (or convection-diffusion) PDE that is an important topic and it might be (but not necessary) the problem in your situation.&#xA;Zero flux boundary condition is very non-standard in this case, so you might check if there is no misinterpretation of your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your flux $J$ can be split into two parts:&#xA;$$&#xA;J=J_{diff} + J_{adv} = - r D c_r  +  s \omega^2 r^2 c&#xA;$$&#xA;Let, formally, $s \omega^2 = 0$ then $J$ is purely diffusive flux and zero flux boundary conditions at $r_0$ and $r_1$ make sense as it is parabolic PDE. The situation is different if you consider for a moment that $D=0$ and you have only advective flux with &quot;speed&quot; $v(r)=s \omega^2 r^2$ and your PDE would be hyperbolic. For this type of problems one prescribes boundary conditions only for &quot;inflow&quot; boundaries, when the flow enters the domain, in your case when $v(r_0)&amp;gt;0$ or $v(r_1)&amp;lt;0$. In the opposite case for so-called &quot;outflow&quot; boundaries (consider the opposite inequalities) NO boundary conditions are required as the flow simply leaves the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me consider your constants $s$ and $\omega$ being positive. Typical boundary conditions of advection for inflow are Dirichlet boundary conditions, i.e. in your case $c(r_0,t) = c_{given}(t)$. No boundary conditions are required for the advective flux at $r=r_1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if all constants are non-zero, a standard treatment for advection-diffusion flux at boundary is that you consider them separately. For instance prescribe the zero diffusive flux everywhere and prescribe the concentration in the advective flux only at inflow boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you prescribed in your code the zero advective-diffusive flux everywhere it still make sense for the inflow, because it is equivaklent to zero diffusive flux and zero concentration at advective flux. But for the outflow it can lead to non-physical behavior, because in fact what you are prescribing is that the diffusive flux is equal minus advective flux. This is especially serious if you have advection dominated problem when so called Peclet number is large, i.e.  $D&amp;lt;s \omega^2 r_1^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this still does not solve your problem, give us a few pictures of time development for your numerical solution, just one or two time steps from constant initial condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. In general the zero flux B.C. in Crank-Nicolson method are considered for each time point separately, so the ghost values at $n$ and $n+1$ are eliminated separately.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-10-05T06:59:28.467" LastActivityDate="2016-10-05T06:59:28.467" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25092" PostTypeId="2" ParentId="25090" CreationDate="2016-09-29T14:29:10.907" Score="4" Body="&lt;p&gt;Lets assume typical versioning semantic as a.b.c, in that case:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a - MAJOR version, where incompatibilities and API changes can occur, &lt;/li&gt;&#xA;&lt;li&gt;b - MINOR version, where new functionality can be added but in a backwards-compatible manner,&lt;/li&gt;&#xA;&lt;li&gt;c - PATCH version, where backwards-compatible bug fixes are made.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you stick for example with the version a1.b1.c1 you application should compile and work smoothly for any changes of b1 and c1 and constant a1. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Should I adapt my code all the time to the newer releases? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It should not be required to change anything in your application when MINOR or PATCH were changed. In case of MAJOR releases it depends. In case of scientific libraries, major changes of APIs are quite rare and always somehow advertised before release.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional remarks: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You application can require a specific version of external libraries to build. It is fine and it should not be a problem to build it either in huge systems (like clusters or super computers) where solutions like &lt;a href=&quot;http://modules.sourceforge.net/&quot;&gt;Environment Modules&lt;/a&gt; are frequently used to solve multi-lib problems.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You did not mention anything about build system, which you are using. Forcing a specific version of the library can solve your problem and it is very easy to achieve in case of CMake or similar build systems. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can build necessary libraries from sources rather than use those provided by the operating system repositories (yum, apt). Moreover, you can deliver with your software a specific version of external libraries which were tested with your software and works. But in case of huge libraries it does not have any sense. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want to support many versions of external libraries you can always prepare some unit tests to validate the results of important methods which use external APIs to ensure that everything is working properly.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In case of VTK I would not worry. It is very well written library, widely used everywhere. Major changes should not break your application.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1010" LastEditorUserId="1010" LastEditDate="2016-09-29T15:01:31.307" LastActivityDate="2016-09-29T15:01:31.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25094" PostTypeId="2" ParentId="7978" CreationDate="2016-09-30T08:03:05.487" Score="-2" Body="&lt;p&gt;The github project &lt;a href=&quot;https://github.com/haptork/easyLambda&quot; rel=&quot;nofollow noreferrer&quot;&gt;easyLambda&lt;/a&gt; provides a high level interface to MPI with C++14. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the project has similar goals and it will give some idea on things that can be and are being done in this area by using modern C++. Guiding other efforts as well as easyLambda itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial &lt;a href=&quot;https://haptork.github.io/easyLambda/docs/benchmarks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;benchmarks&lt;/a&gt; on performance and lines of code have shown promising results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yJVjt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yJVjt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is a short description of features and interface it provides. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interface is based on data flow programming and functional list operations that provide inherent parallelism. The parallelism is expressed as property of a task. The process allocation and data distribution for the task can be requested with a .prll() property. There are good number of examples in the &lt;a href=&quot;https://haptork.github.io/easyLambda/docs/real-world/&quot; rel=&quot;nofollow noreferrer&quot;&gt;webpage&lt;/a&gt; and &lt;a href=&quot;https://github.com/haptork/easyLambda/tree/master/examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;code-repository&lt;/a&gt; that include LAMMPS molecular dynamics post processing, explicit finite difference solution to heat equation, logistic regression etc. As an example the heat diffusion problem discussed in the article &lt;a href=&quot;http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HPC is dying...&lt;/a&gt; can be expressed in ~20 lines of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope it is fine to give links rather than adding more details and example codes here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclamer: I am the author of the library. I believe I am not doing any harm in hoping to get a constructive feedback on the current interface of easyLambda that might be advantageous to easyLambda and any other project that pursues similar goals.&lt;/p&gt;&#xA;" OwnerUserId="21838" LastEditorUserId="21838" LastEditDate="2016-10-01T11:37:00.940" LastActivityDate="2016-10-01T11:37:00.940" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25095" PostTypeId="1" CreationDate="2016-09-30T18:03:00.390" Score="2" ViewCount="132" Body="&lt;p&gt;I have the following question. I have a code function that computes right-biased and left-biased approximations of the derivative of a function using WENO5 for Hamilton--Jacobi equations as described in&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@Article{Jiang2000,&#xA;  Title                    = {Weighted {ENO} Schemes for {Hamilton--Jacobi} Equations},&#xA;  Author                   = {Jiang, Guang-Shan and Peng, Danping},&#xA;  Journal                  = SIAM Journal of Scientific Computing,&#xA;  Year                     = {2000},&#xA;  Month                    = jan,&#xA;  Number                   = {6},&#xA;  Pages                    = {2126--2143},&#xA;  Volume                   = {21},&#xA;  DOI                      = {10.1137/S106482759732455X},&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I refer to this WENO as WENO5JP below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I unit-test my implementation of WENO5JP by checking the order of convergence while approximating derivatives of simple smooth functions.&#xA;I obtain right- and left-biased approximations of the derivative, average them, and compare this result to the analytic derivative. Then I measure error in $L_{\infty}$-norm, and then I compute the observed order of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I get strange behavior when for one function I get between 5 and 6.5 order of accuracy, while for another function I get between 4 and 6 order of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Computation of the observed order of accuracy&lt;/strong&gt;. I use the following standard formula to estimate the order of accuracy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_k = \frac{\ln \left( \|E_{k-1}\| / \|E_k\| \right)}&#xA;  {\ln \left(N_k / N_{k-1} \right)},&#xA;$$&#xA;where $p_k$ is the observed order for the grid size $N_k$, and here I use $N_k / N_{k-1} = 2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Details&lt;/strong&gt;. Below are the details of convergence behavior that I get.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I apply WENO5JP to a function $y = \sin x$ on domain $[0; 2\pi]$ and obtain the order of accuracy as shown in the table below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|------------|-------|&#xA;| Resolution | Order |&#xA;|         11 |   nan |&#xA;|         41 |  6.68 |&#xA;|         81 |  6.24 |&#xA;|        161 |  6.12 |&#xA;|        321 |  6.15 |&#xA;|        641 |  5.12 |&#xA;|       1281 |  0.60 |&#xA;|       2561 | -1.36 |&#xA;|------------|-------|&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Linear least squares applied to the function $E = p \ln N + \ln C$ gives estimate of the order $p$ as $p = 6.06$. And this is the plot in a log-log scale &quot;Error vs grid size&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lmad9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lmad9.png&quot; alt=&quot;Error vs grid size for sine function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;I apply WENO5JP to the function $ \exp x \, \sin(5x)$ from Trefethen's book on spectral methods (p. 56) and obtain the following behavior&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|------------|-------|&#xA;| Resolution | Order |&#xA;|         11 |   nan |&#xA;|         21 |  4.16 |&#xA;|         41 |  5.64 |&#xA;|         81 |  4.90 |&#xA;|        161 |  4.47 |&#xA;|        321 |  4.11 |&#xA;|        641 |  4.21 |&#xA;|       1281 |  4.45 |&#xA;|       2561 |  5.92 |&#xA;|       5121 | -0.84 |&#xA;|------------|-------|&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Linear least squares applied to the function $E = p \ln N + \ln C$ gives estimate of the order $p$ as $p =4.58$. And this is the plot in a log-log scale &quot;Error vs grid size&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bc3Cg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bc3Cg.png&quot; alt=&quot;Error vs grid size for Trefethen&amp;#39;s smooth function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, WENO5JP exhibits some strange behavior, where for one example it gives more than the 5th order of accuracy, while in another example the order of accuracy drops to 4. Last rows of each table show that the rounding error dominates over the truncation error of the method at these grid step sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is&lt;/strong&gt;: is it normal for WENO5JP to have that volatile order of accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can say from the literature on WENO for conservation laws, people usually don't get strictly 5th order even for simple linear problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is still not clear to me whether I can say that my implementation is correct or not due to this strange convergence behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: you can find the code here: &lt;a href=&quot;https://github.com/dmitry-kabanov/weno5jp-issue&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dmitry-kabanov/weno5jp-issue&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3254" LastEditorUserId="3254" LastEditDate="2016-10-02T20:34:21.847" LastActivityDate="2016-10-02T20:34:21.847" Title="Strange convergence behavior of WENO5 for Hamilton--Jacobi equations" Tags="|convergence|testing|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25096" PostTypeId="1" AcceptedAnswerId="25102" CreationDate="2016-10-01T05:52:23.950" Score="7" ViewCount="241" Body="&lt;h2&gt;Introduction to Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm using a &lt;a href=&quot;http://graphics.stanford.edu/papers/volrange/volrange.pdf&quot; rel=&quot;noreferrer&quot;&gt;Truncated Signed Distance Function to perform 3D reconstruction&lt;/a&gt; from depth images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially I have a large voxel grid where each voxel contains the signed distance to an implicit surface. The surface can be extracted form the Voxel Grid using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Marching_cubes&quot; rel=&quot;noreferrer&quot;&gt;marching cubes algorithm&lt;/a&gt; to generate a mesh. The vertices of this generated mesh lie on the edges of the cubes formed by joining the central points of 8 adjacent voxels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the volume that I'm reconstructing is moving in a non-rigid way. I am able to track the motion of points on the surface of the object. In this way I can determine how a particular point on the mesh moved between one frame and another. I want to use this information to generate the effective transformation (translation) of each voxel in the voxel grid which would lead to the deformation of the implicit mesh which most closely matches the observed scene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the situation described below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MtAYt.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MtAYt.png&quot; alt=&quot;https://i.stack.imgur.com/MtAYt.png&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_0, v_1, \cdots, v_m$  are points in 3D space. They initially lie on a regular lattice with width $w$, height $h$ and depth $d$ so that $m = whd$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than at the boundaries of the grid, each $v_x$ is connected to 6 neighbouring points:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_{x-1}, v_{x+1}, v_{x-wh}, v_{x+wh}, v_{x+h}$ and $v_{x-w}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The set of points $p_{a,b}$ lie on the edges connecting pairs of vertices $v_a$  and $v_b$. I have at most one point per edge that I care about but for some edges, I have no point. For example, in the image, the edges connecting $v_{wh}$ to $v_{wh+1}$ and $v_{wh}$ to $v_{2wh}$ have no point defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each defined point, we can define $\alpha$ (green in the image), which is the proportion of the way along the edge from the first vertex to the second. So for example, $p_{0,1}$ has $\alpha$ equal to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha = \frac{|v_0 p_{0,1}|}{|v_0 v_1|}$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The grid as shown represents the situation at time 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At time 1, I require each of the each of the points $p_{a,b}$ to be in a new location,  $p'_{a,b}$ (known) which requires each of these points to undergo a translation. The overall transformation of the point cloud is not rigid, i.e. each point will be translated by a different amount.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can only translate the points $p_{a,b}$ by moving the vertices $v_x$.  When I do this, the distances between neighbouring vertices can change arbitrarily however the values of $\alpha$ computed after remain constant, i.e. the point $p_{a,b}$ still divides the edge $v_{a} v_{b}$ in the ratio $\alpha:(1 - \alpha)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to work out the ‘best' translations for $v_0, \cdots, v_m$  which will :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Get each of the points $p$ as close as possible to its desired position; and&lt;/li&gt;&#xA;&lt;li&gt;Minimise the differences between the translations applied to neighbouring $v$s&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;What I think is true&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I need to construct an energy equation comprising a data term and a regularisation term and then minimise this equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data term, $E_\text{data}$ minimises the squared difference between the desired coordinates of $p'_{a,b}$ and the computed values of $p'_{a,b}$ which are given by $(1 - \alpha) v_a + \alpha v_b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E_\text{data} = \sum_{a, b} (((1 - \alpha) v_a + \alpha v_b) - p'_{a,b})^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sum is across all edges and so each $v'_a$ and $v'&#xA;_b$   may be involved in multiple terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The regularisation term $E_\text{reg}$ will aim to minimise the differences between vertex transforms and will be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E_\text{reg} = \sum_{i=0}^m \sum_{j\in N(i)} (|t_i| - |t_j|)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $N(i)$ is the neighbourhood of i and comprises all of the adjacent vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note that there are more regularisation terms ($m$) than there are data terms (because there’s not one $p$ per $v$).&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;My questions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I’m thinking of using non-linear least squares to solve this. I’ve written code which does this for a simpler problem in MATLAB and it works fine. The simpler problem uses :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Only 2 dimensions&lt;/li&gt;&#xA;&lt;li&gt;A very small grid&lt;/li&gt;&#xA;&lt;li&gt;The built in MATLAB lsqnonlin function.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now I want to build my own code which can solve this for the full scale problem in 3D but I have some concerns:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It seems like using Newton’s method should work however to compute the gradient vector Hessian matrix it feels like I need the same number of data and regularisation terms so I need to express $E_\text{data}$ in terms of $v_0, \cdots, v_m$  but am unsure how to do this. Is this required ?&lt;/li&gt;&#xA;&lt;li&gt;Given that $m$ is of the order of tens, if not hundreds of thousands, computing the Hessian and it’s inverse seems like it will take a very long time. The Hessian should be sparse but is there a better way to solve this ? &lt;/li&gt;&#xA;&lt;li&gt;It feels like avoiding constructing matrices altogether would be beneficial but is there a simple (to understand) way of achieving this ?&lt;/li&gt;&#xA;&lt;li&gt;I feel that using CUDA to parallelize the problem would help with performance but again, I'm not sure how I would go about crafting the algorithm to use.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can anyone point me at appropriate literature or suggest a good starting point for research ?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Curless, Brian, and Marc Levoy. &lt;a href=&quot;http://graphics.stanford.edu/papers/volrange/volrange.pdf&quot; rel=&quot;noreferrer&quot;&gt;&quot;A volumetric method for building complex models from range images.&quot;&lt;/a&gt; Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 1996.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21849" LastEditorUserId="21849" LastEditDate="2016-10-01T20:41:14.147" LastActivityDate="2016-10-06T00:59:52.377" Title="What would be a good approach to solving this large data non-linear least squares optimisation" Tags="|optimization|nonlinear-equations|newton-method|cuda|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25097" PostTypeId="2" ParentId="1321" CreationDate="2016-10-01T13:55:15.150" Score="3" Body="&lt;p&gt;I know there has been a significant delay here. I also hope that it is not inappropriate to advertise personal work in this situation. I am making an exception to my usual policy because of the time frame. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short answer to your question is: &quot;yes&quot; if $a_i$'s and $b_i$'s are real. (For complex $a_i$'s and $b_i$'s it is still true in practice, but more thinking is required) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bernhard Beckermann and I just submitted a paper &lt;a href=&quot;http://www.math.cornell.edu/~ajt/papers/PosDefHankel.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt; that shows that if $AX-XB = F$ with $A$ and $B$ normal matrices, then the singular values of $X$ can be bounded as&#xA;$$&#xA;\sigma_{1+\nu k}(X) \leq Z_k(\sigma(A),\sigma(B))\|X\|_2, \qquad \nu = {\rm rank}(F),&#xA;$$ &#xA;where $\sigma(A)$ and $\sigma(B)$ are the spectra of $A$ and $B$, respectively, and $Z_k(E,F)$ is the so-called Zolotarev number. More precisely, &#xA;$$&#xA;Z_k(E,F) = \inf_{r\in\mathcal{R}_{k,k}} \frac{\sup_{z\in E}|r(z)|}{\inf_{z\in F}|r(z)|},&#xA;$$&#xA;where $\mathcal{R}_{k,k}$ is the space of degree $(k,k)$ rational functions. This is called the third Zolotarev problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the Cauchy matrix, let $A = {\rm diag}(a_1,\ldots,a_n)$ and $B = {\rm diag}(b_1,\ldots,b_n)$ then &#xA;$$&#xA;AC - CB = \begin{bmatrix}1\\\vdots\\1 \end{bmatrix}\begin{bmatrix}1&amp;amp;\cdots&amp;amp;1 \end{bmatrix}.&#xA;$$&#xA;Hence, $\nu=1$ and &#xA;$$&#xA;\sigma_{1+k}(C) \leq Z_k([a_1,\ldots,a_n],[b_1,\ldots,b_n])\|C\|_2.&#xA;$$&#xA;Unfortunately, discrete Zolotarev number (like above) are very difficult to bound.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we make the &lt;strong&gt;additional assumption&lt;/strong&gt; (which is appropriate for off-diagonal blocks) that $[a_1,\ldots,a_n]\subseteq [a,b]$ and $[b_1,\ldots,b_n]\subseteq [c,d]$, where $[a,b]$ and $[c,d]$ are disjoint intervals, then &#xA;$$&#xA;\sigma_{1+k}(C) \leq Z_k([a,b],[c,d])\|C\|_2.&#xA;$$&#xA;In Corollary 7 of &lt;a href=&quot;http://www.math.cornell.edu/~ajt/papers/PosDefHankel.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;, we showed that this leads to the singular value bounds &#xA;$$&#xA;\sigma_{1+k}(C) \leq 4\left[{\rm exp}\left(\frac{\pi^2}{2\log(16\gamma)}\right)\right]^{-2k}\|C\|_2,&#xA;$$&#xA;where $\gamma$ is the absolute value of the cross-ratio of $a$, $b$, $c$, and $d$. Namely, &#xA;$$&#xA;\gamma = \frac{|c-a|\,|d-b|}{|c-b|\,|d-a|}.&#xA;$$&#xA;This gives you a bound on the numerical rank of $C$ as &#xA;$$&#xA;{\rm rank}_\epsilon(C) \leq \Bigg\lceil \frac{\log(16\gamma)\log(4/\epsilon)}{\pi^2}\Bigg\rceil = \mathcal{O}(\log\gamma \log(1/\epsilon)).&#xA;$$&#xA;For a case where $\gamma = \mathcal{O}(n)$ and a $\log(n)$ term appears see the Hilbert matrix: &lt;a href=&quot;https://mathoverflow.net/questions/137059/the-singular-values-of-the-hilbert-matrix/251187&quot;&gt;https://mathoverflow.net/questions/137059/the-singular-values-of-the-hilbert-matrix/251187&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; In your case the $a_i$'s and $b_i$'s are complex valued. This is more tricky and more related to Section 5.1 of &lt;a href=&quot;http://www.math.cornell.edu/~ajt/papers/PosDefHankel.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="20812" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2016-10-01T14:06:03.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25099" PostTypeId="1" CreationDate="2016-10-02T15:52:23.383" Score="1" ViewCount="461" Body="&lt;p&gt;I have two computers, both with 4-cores and I am running some heavy computational stuff in one of them using Intel Parallel Studio Cluster edition. I am coding in fortran and making extensive use of OpenMP to parallelize the loops. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read that it should be possible to run this code using the two computers at the same time, but I have no clue on how to set it up. Is it worth it, meaning that, if my code once paralellized on a single machine decreases computational time from 9 hours to 1.37 hours. I guess that if I manage to cluster two machines it will get even faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two machines in which I want to set this up are exactly identical and are connected to the same router via wi fi. My exact question is whether anyone is aware of any good tutorial on how to connect the two machines using MPI? Also, does the coding need to be changed a lot? Is there any good resource on the MPI syntax?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="21861" LastEditorUserId="21861" LastEditDate="2016-10-03T18:06:23.980" LastActivityDate="2016-10-03T18:06:23.980" Title="Using two computers to run one parallelized program with intel fortran" Tags="|fortran|hpc|intel-mkl|" AnswerCount="2" CommentCount="0" ClosedDate="2016-10-03T17:38:15.620" ContentLicense="CC BY-SA 3.0" />
  <row Id="25100" PostTypeId="2" ParentId="25099" CreationDate="2016-10-03T01:32:51.970" Score="2" Body="&lt;p&gt;Being a big believer in parallel computation (the chips are not getting any faster), the short answer is yes. You should look into MPI (Message Passing Interface), the defecto standard for passing data between different machines when it comes to cluster computing. See this link &lt;a href=&quot;https://en.wikipedia.org/wiki/Message_Passing_Interface&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Message_Passing_Interface&lt;/a&gt; for an intro to MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assessing whether it is worth it for you to invest time into making the code cluster-safe, to borrow after the thread-safe idiom, depends on how parallelizable your code is, its memory requirements, whether the data is partitioned on a per thread-basis or not, and how much shared data is accessed by each compute node and CPU/core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, think of MPI as a layer above OpenMP at the level of different computers in your cluster, just as OpenMP which works at the level of different threads (or cores, if you will).&lt;/p&gt;&#xA;" OwnerUserId="20098" LastEditorUserId="20098" LastEditDate="2016-10-03T04:30:10.693" LastActivityDate="2016-10-03T04:30:10.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25102" PostTypeId="2" ParentId="25096" CreationDate="2016-10-03T09:46:59.980" Score="5" Body="&lt;p&gt;I don't know much about tracking implicit surfaces, so I'm just going to start with the optimization problem and go from there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimization problem is, at the core, nonlinear least squares, and can therefore be solved efficiently by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gauss%E2%80%93Newton_algorithm&quot; rel=&quot;nofollow&quot;&gt;Gauss-Newton method&lt;/a&gt;, wherein at each step one linearizes the problem at the current guess, solves the &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)&quot; rel=&quot;nofollow&quot;&gt;linear least squares&lt;/a&gt; problem for this linearization, then uses the solution as the initial guess for the next step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each step, the linear least squares problem can be recast as solving one large sparse linear system. Figuring out what the coefficient matrix is for this linear system is a little mindbending, but doable with some matrix vectorization tricks (explanation below). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the matrix is constructed, the linear system can be solved efficiently with any good sparse-direct factorization technique up to quite large problem sizes. Furthermore, this matrix is physically analogous to the matrices that arise when trying to find the minimum energy configuration of a spring network, and so if the problem is extremely huge (too big for a sparse-direct factorization to fit in memory) it should still be amenable to multigrid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This optimization problem has the interesting property that the data misfit term is linear (and therefore stays the same regardless of the iteration), whereas the regularization term is nonlinear (and therefore changes from iteration to iteration). Usually it is exactly the opposite, where the data misfit is nonlinear but the regularization is linear.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Optimization problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In what follows, let $N$ be the total number of grid vertices, and let $M$ be the total number of target points $p'$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I read everything correctly, the optimization problem can be recast as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{V'} \frac{1}{2}||\hat{Q}'(V') - P'||^2_\text{fro} + \frac{1}{2}||D~m(V'-V)||^2_\text{fro}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this formula, &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$||\cdot||_\text{fro}$ is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_norm#Frobenius_norm&quot; rel=&quot;nofollow&quot;&gt;Frobenius matrix norm&lt;/a&gt; (square root of sum of squares of all the entries of the matrix), &lt;/li&gt;&#xA;&lt;li&gt;$V$ is the $3$-by-$N$ matrix where each column is one of the vertices $v$ in the initial configuration.&lt;/li&gt;&#xA;&lt;li&gt;$V'$ is the $3$-by-$N$ matrix where each column is one of the vertices $v'$ of the moved configuration.&lt;/li&gt;&#xA;&lt;li&gt;$P'$ is the $3$-by-$M$ matrix where each column is one of the target points $p'$. &lt;/li&gt;&#xA;&lt;li&gt;$\hat{Q}'(V')$ is the $3$-by-$M$ matrix of approximate target points, $q' \approx p'$, based on a given choice of moved vertices $V'$.&lt;/li&gt;&#xA;&lt;li&gt;$D$ is the $M$-by-$N$ graph finite difference matrix that takes in a vector of values on the nodes of a graph, and outputs a list of differences across the edges. The $j$'th entry of the output is the difference between the value at the head vertex and the value at the tail vertex for edge $e_j$. I.e., the $i$'th row of this matrix has $1$ in the column corresponding to the &quot;head&quot; vertex for edge $e_i$, $-1$ in the column corresponding to the &quot;tail&quot; vertex for edge $e_i$, and zeros elsewhere.&lt;/li&gt;&#xA;&lt;li&gt;$m:\mathbb{R}^{3\times N}\rightarrow \mathbb{R^N}$ is the columnwise length measuring function. I.e, if $X$ is a matrix with columns $x_i$, then the $i$'th element of $m(X)$ is $||x_i||$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Data misfit term&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Suppose the $j$'th approximate vertex, $q_j$, lies on an edge, where the &quot;tail&quot; of the edge is a moved vertex $v'_\text{tail}$, and the &quot;head&quot; of the edge is a moved vertex $v'_\text{head}$. Then from the problem statement,&#xA;$$q = (1-\alpha_j)v'_\text{head} + \alpha_j v'_\text{tail}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, applying this transformation to all vertices at once, we have:&#xA;$$Q'(V') = (1 - A).*(V' E_\text{heads}) + A.*(V' E_\text{tails}),$$&#xA;where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;$.*$&quot; is MATLAB-style notation for the elementwise (Hadamard) matrix product.&lt;/li&gt;&#xA;&lt;li&gt;$A$ is the matrix where each column is constant, and the $j$'th column contains copies of $\alpha_j$ repeated vertically,&lt;/li&gt;&#xA;&lt;li&gt;$1$ represents the matrix of all ones, of the same size as $A$ (not the identity matrix),&lt;/li&gt;&#xA;&lt;li&gt;$E_\text{heads}$ and $E_\text{tails}$ are the column selection matrices, that, when acting on the matrix $V'$ from the right, generate the matrices whose $j$'th columns are the &quot;head&quot; and &quot;tail&quot; vectors for $q_j$, respectively. More details on these column selection matrices later..&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Element-wise multiplication of matrices is the same as multipying the vectorized form of one of the matrices by a big diagonal matrix whose diagonal entries come from the other matrix. Thus if we &lt;a href=&quot;https://en.wikipedia.org/wiki/Vectorization_(mathematics)&quot; rel=&quot;nofollow&quot;&gt;vectorizes&lt;/a&gt; (unrolls) the matrix $Q'(V')$ into one long vector, &quot;$\text{vec}(Q'(V')$&quot;, we can write it as follows:&#xA;$$\text{vec}(Q'(V')) = (I_{3M}-D)~\text{vec}(V' E_\text{heads}) + D~\text{vec}(V' E_\text{tails}),$$&#xA;where $D:=\text{diag}(\text{vec}(A))$ is a very big diagonal matrix whose diagonal entries are the entries of $A$, and the notation $I_{k}$ denotes a $k$-by-$k$ identity matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, recall the following relationship between vectorization $\text{vec}(\cdot)$ and &lt;a href=&quot;https://en.wikipedia.org/wiki/Kronecker_product&quot; rel=&quot;nofollow&quot;&gt;Kronecker products&lt;/a&gt; $\otimes$:&#xA;$$\text{vec}(ABC) = (C^T \otimes A)\text{vec}(B).$$&#xA;Applying this identity to our expression above, we have:&#xA;$$\text{vec}(Q'(V')) = (I_{3M}-D)~(E_\text{heads}^T \otimes I_3)~\text{vec}(V') + D~(E_\text{tails}^T \otimes I_3)~\text{vec}(V'),$$&#xA;or &#xA;$$\text{vec}(Q'(V')) = J~ \text{vec}(V'),$$ &#xA;where&#xA;$$\boxed{J:=(I_{3M}-D)~(E_\text{heads}^T \otimes I_3) + D~(E_\text{tails}^T \otimes I_3)}$$&#xA;is the matrix that describes the vectorized transformation from all moved $v'$ vertices to all approximate target points $q'$. This matrix is extremely large, but also extremely sparse. (Also very easy to construct in MATLAB from built-in functions)&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Regularization term&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Recall that the regularization term is:&#xA;$$\frac{1}{2}||D~m(V'-V)||^2_\text{fro},$$&#xA;where, $m(\cdot)$ is the function that takes in a $3$-by-$N$ matrix, and outputs the vector containing the lengths of each column of the input matrix. I.e.,&#xA;$$m(X) = \begin{bmatrix}&#xA;||x_1|| \\&#xA;||x_2|| \\&#xA;\vdots \\&#xA;||x_N|| \\&#xA;\end{bmatrix}$$&#xA;We seek to linearize this term about some initial guess, $V'_0$, for the moved point $V'$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the formula for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Norm_(mathematics)#p-norm&quot; rel=&quot;nofollow&quot;&gt;derivative of the norm&lt;/a&gt;, we can compute the the derivative of $m$ at point $Y$ in direction $\delta X$ as follows:&#xA;\begin{align}&#xA;\left.\frac{dm}{dX}\right|_{Y}~\delta X &amp;amp;= \begin{bmatrix}&#xA;\frac{y_1^T \delta x_1}{||y_1||} \\&#xA;\frac{y_2^T \delta x_2}{||y_2||} \\&#xA;\vdots \\&#xA;\frac{y_N^T \delta x_N}{||y_N||} \\&#xA;\end{bmatrix} \\&#xA;&amp;amp;= \begin{bmatrix}&#xA;\frac{y_1^T}{||y_1||} \\&#xA;&amp;amp; \frac{y_2^T}{||y_2||} \\&#xA;&amp;amp; &amp;amp; \ddots \\&#xA;&amp;amp; &amp;amp; &amp;amp; \frac{y_N^T}{||y_N||}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\delta x_1 \\&#xA;\delta x_2 \\&#xA;\vdots \\&#xA;\delta x_N&#xA;\end{bmatrix} \\&#xA;&amp;amp;= M(Y)~\text{vec}(\delta X),&#xA;\end{align}&#xA;where $M(Y)$ is the big $N$-by-$3N$ matrix shown in the middle formula. I.e., &#xA;$$M(Y)=\text{blockdiag}\left(\frac{y_1^T}{||y_1||}, \frac{y_2^T}{||y_2||}, \dots, \frac{y_N^T}{||y_N||}\right).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, the regularization function can be linearized about an initial point $V'_0 \approx V'$ as follows,&#xA;$$D~m(V'-V) = D~m(V_0' - V) + D~M(V_0' - V)~\text{vec}(V' - V'_0) + O(||V' - V'_0||^2),$$&#xA;Defining&#xA;$$\boxed{R(V, V_0') := D~M(V_0' - V)}$$&#xA;and&#xA;$$\boxed{r_0(V, V'_0) := D~m(V_0' - V) - R(V, V_0')~\text{vec}(V'_0),}$$&#xA;the linearization of the regularization term can be written as follows:&#xA;$$D~m(V'-V) = r_0(V, V_0') + R(V, V_0')~\text{vec}(V') + O(||V' - V'_0||^2).$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Linearized optimization problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Hence, the original optimization problem, when the regularization is linearized about a point $V'_0$, can be recast in vectorized form as follows:&#xA;$$\min_{\text{vec}(V')} \frac{1}{2}||J~\text{vec}(V') - \text{vec}(P')||^2 + \frac{1}{2}||R~\text{vec}(V') + r_0||^2,$$&#xA;where we abbreviate $R:=R(V, V_0')$, and $r_0:=r_0(V, V_0')$. This linear least squares problem can be solved by solving the normal equations system:&#xA;$$\boxed{(J^TJ + R^TR)~\text{vec}(V') = J^T~\text{vec}(P') - R^T~r_0.}$$&#xA;So, you can form solve this sparse system for $\text{vec}(V')$, then reshape the result from a vector into a matrix, and then extract the columns of this matrix as the desired vertices $v'$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give you the optimal moved vertices based on the linearization of the regularization about point $V'_0$. To solve the problem with general nonlinear regularization, you can iterate: linearize about an initial guess, solve the linearized problem, use the solution as the next point to linearize about, solve that problem, and repeat until convergence. This is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gauss%E2%80%93Newton_algorithm&quot; rel=&quot;nofollow&quot;&gt;Gauss-Newton method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A side note of interest is that the graph Laplacian $D^TD$ appears within the core of the $R^TR$, so this regularization could be interpreted as some form of grouped discrete smoothing regularization.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Heads and tails selection matrices&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Above we glossed over the construction of the &quot;heads&quot; and &quot;tails&quot; selection matrices $E_\text{heads}$ and $E_\text{tails}$, so more details are provided here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $e_\text{heads}$ and $e_\text{tails}$ be length-$M$ vectors representing the &quot;head-for-$p$&quot; map and &quot;tail-for-$p$&quot; maps. That is, if $p_j$ lies on the edge going from vertex $v_a$ to vertex $v_b$, then the $j$'th entry of $e_\text{heads}$ is the index $b$, and the $j$'th entry of $e_\text{tails}$ is the index $a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose the first point lies on the edge going from vertex $v_3$ to $v_8$, the second point lies on the edge going from vertex $v_5$ to $v_6$, and the third point lies on the edge going from $v_9$ to $v_5$ (this is a made up example to illustrate the point). Schematically,&#xA;\begin{align}&#xA;v_3 &amp;amp;\overset{p_1}{\rightarrow} v_8 \\&#xA;v_5 &amp;amp;\overset{p_2}{\rightarrow} v_6 \\&#xA;v_9 &amp;amp;\overset{p_3}{\rightarrow} v_5.&#xA;\end{align}&#xA;Then the &quot;edge-to-head&quot; and &quot;edge-to-tail&quot; vectors would start:&#xA;\begin{align}&#xA;e_\text{tails}=&amp;amp;\begin{bmatrix}8, &amp;amp; 6, &amp;amp; 5, &amp;amp; \dots\end{bmatrix} \\&#xA;e_\text{heads}=&amp;amp;\begin{bmatrix}3, &amp;amp; 5, &amp;amp; 9, &amp;amp; \dots\end{bmatrix}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to find the selection matrices $E_\text{heads}$ and $E_\text{tails}$ such that:&#xA;$$\begin{bmatrix}v_1 &amp;amp; v_2 &amp;amp; \dots &amp;amp; v_N\end{bmatrix} E_\text{heads} &#xA;= &#xA;\begin{bmatrix}v_{e_\text{heads}(1)} &amp;amp; v_{e_\text{heads}(2)} &amp;amp; \dots &amp;amp; v_{e_\text{heads}(M)}\end{bmatrix},$$&#xA;and&#xA;$$\begin{bmatrix}v_1 &amp;amp; v_2 &amp;amp; \dots &amp;amp; v_N\end{bmatrix} E_\text{tails} &#xA;= &#xA;\begin{bmatrix}v_{e_\text{tails}(1)} &amp;amp; v_{e_\text{tails}(2)} &amp;amp; \dots &amp;amp; v_{e_\text{tails}(M)}\end{bmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $E_\text{heads}$ that does this is the sparse $N$-by-$M$ matrix, where the $j$'th column has a $1$ in row $e_\text{heads}(j)$ (and likewise for $E_\text{tails}$, except with the row indices drawn from $e_\text{tails}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the following product of a matrix whose columns are the vectors $x_1$, $x_2$, and $x_3$, with a selection matrix:&#xA;$$\begin{bmatrix}&#xA;x_1 &amp;amp; x_2 &amp;amp; x_3&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;&amp;amp; &amp;amp; 1\\&#xA;&amp;amp; &amp;amp; &amp;amp; 1 \\&#xA;1 &amp;amp; 1 &amp;amp; &amp;amp;&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;x_3 &amp;amp; x_3 &amp;amp; x_1 &amp;amp; x_2&#xA;\end{bmatrix}.$$&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2016-10-06T00:59:52.377" LastActivityDate="2016-10-06T00:59:52.377" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25104" PostTypeId="2" ParentId="25099" CreationDate="2016-10-03T12:46:56.083" Score="0" Body="&lt;p&gt;What you mention is called hybrid parallelism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A cluster is composed of several nodes. A node is a group of sockets that share the same physical memory and a socket is a group of cores that we refer as a processor. The nodes are stored in racks and communicate with each other thanks to a very high-speed connection through a switch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To simplify, you perform hybrid parallelism when you affect a number of OpenMP threads equal to the number of cores per socket, and a number of MPI processes equal to the number of nodes you want to use on the cluster. For instance, if I have a cluster with 4 nodes and 12 cores by sockets/processors, I launch my program to run with 4 MPI processes (1 by node) and 12 OpenMP threads by MPI process (one by core on each socket). In your example, you have 2 nodes (2 computers, 2 different entities with different memory), and 4 cores by processor. So run with 2 MPI processes, each one dealing with 4 OpenMP threads on each computer. NameRakes gives a good word : MPI is a &lt;strong&gt;layer&lt;/strong&gt; above OpenMP. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative is to only use MPI with 8 processes since MPI works both for shared and distributed memory architecture, you cannot do that with OpenMP which is restricted to shared-memory computers. However, in that case, 8 MPI communications are required against 2 for a hybrid parallelism, which is less efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seem to have a pretty decent speed-up with OpenMP. Hybrid parallelism makes sense for clusters with dozens of sockets and highly scalable numerical codes with thousand of degrees of freedom. As said by NameRakes, not sure that the time investment will be rewarding since MPI and OpenMP are very different in their concept. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also be aware that your code will run according to the slower processor so if your two computers have different processors, it may deteriorate your speed-up so you want to make sure to distribute the load correctly. You can surely gain time, but not by a factor 2, it depends on the architecture and the scalability of your code.&lt;/p&gt;&#xA;" OwnerUserId="20454" LastActivityDate="2016-10-03T12:46:56.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25105" PostTypeId="1" AcceptedAnswerId="25107" CreationDate="2016-09-24T16:02:52.110" Score="4" ViewCount="250" Body="&lt;p&gt;I am trying to evaluate the Exponential Integral $Ei(x)=-\int^{\infty}_{-x}\frac{e^{-t}}{t}dt$ for $x&amp;gt;0$ (interpreted as the Cauchy principal value) by using rational Chebyshev approximations, which can be found in a paper by Cody &amp;amp; Thacher &quot;Chebyshev Approximations for the Exponential Integral $Ei(x)$&quot;. The paper is available online at &lt;a href=&quot;http://www.ams.org/mcom/1969-23-106/S0025-5718-1969-0242349-2/S0025-5718-1969-0242349-2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ams.org&lt;/a&gt;. I am trying to use the equation on page 292 for the interval $0&amp;lt;x\le6$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ei(x)\simeq\log(x/x_0)+(x-x_0)\frac{\sum_{j=0}^{n}p_jT_j^{*}(x/6)}{\sum_{j=0}^{n}q_jT_j^{*}(x/6)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $x_0\approx{}0.37$ is the zero of $Ei(x)$, $p_j$ and $q_j$ are the coefficients found in table II of the paper (I am using $n=9$) and $T_j^{*}(x)=T_j(2x-1)$ are the shifted Chebyshev polynomials. So I implemented the equation in a &lt;a href=&quot;http://pastebin.com/FwfFJbb7&quot; rel=&quot;nofollow noreferrer&quot;&gt;naive C program&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;&#xA;#define N_1 10&#xA;&#xA;double chebyshev(int n, double x){&#xA;    switch(n){&#xA;        case 0:&#xA;            return 1.0;&#xA;            break;&#xA;        case 1:&#xA;            return x;&#xA;            break;&#xA;        default:&#xA;            return 2.0*x*chebyshev(n-1, x)-chebyshev(n-2, x);&#xA;            break;&#xA;    }&#xA;}&#xA;&#xA;double shifted_chebyshev(int n, double x){&#xA;    return chebyshev(n, 2.0*x-1.0);&#xA;}&#xA;&#xA;double interval1(double x){&#xA;    static const double pj_1[N_1] = {&#xA;                            -4.1658081333604994241879E11,&#xA;                            +1.2177698136199594677580E10,&#xA;                            -2.5301823984599019348858E10,&#xA;                            +3.1984354235237738511048E8,&#xA;                            -3.5377809694431133484800E8,&#xA;                            -3.1398660864247265862050E5,&#xA;                            -1.4299841572091610380064E6,&#xA;                            -1.4287072500197005777376E4,&#xA;                            -1.2831220659262000678155E3,&#xA;                            -1.2963702602474830028590E1&#xA;    };&#xA;    static const double qj_1[N_1] = {&#xA;                            -1.7934749837151009723371E11,&#xA;                            +9.8900934262481749439886E10,&#xA;                            -2.8986272696554495342658E10,&#xA;                            +5.4229617984472955011862E9,&#xA;                            -7.0108568774215954065376E8,&#xA;                            +6.4698830956576428587653E7,&#xA;                            -4.2648434812177161405483E6,&#xA;                            +1.9418469440759880361415E5,&#xA;                            -5.5648470543369082846819E3,&#xA;                            +7.6886718750000000000000E1&#xA;    };&#xA;    static const double x0 = 0.372507410781366634461991866580;&#xA;&#xA;    int j;&#xA;    double result, numerator, denominator, sum;&#xA;&#xA;    result = log(x/x0);&#xA;    numerator = 0.0;&#xA;    denominator = 0.0;&#xA;    sum = 0.0;&#xA;&#xA;    for(j=0; j&amp;lt;N_1; j++){&#xA;        numerator += pj_1[j] * shifted_chebyshev(j, x/6.0);&#xA;        denominator += qj_1[j] * shifted_chebyshev(j, x/6.0);&#xA;    }&#xA;    sum = numerator / denominator;&#xA;    sum *= (x-x0);&#xA;    result += sum;&#xA;&#xA;    return result;&#xA;}&#xA;&#xA;double ei(double x){&#xA;    if( x &amp;lt; 0.0 ){&#xA;        printf(&quot;Argument of ei(x) must be positive\n&quot;);&#xA;        exit(1);&#xA;    }&#xA;&#xA;    if( x &amp;lt;= 6.0 ){&#xA;        return interval1(x);&#xA;    }&#xA;&#xA;    printf(&quot;Argument range not yet implemented for ei(x).\n&quot;);&#xA;    exit(1);&#xA;}&#xA;&#xA;int main(void){&#xA;    double x = 5.0;&#xA;    printf(&quot;ei(%g)=%g\n&quot;, x, ei(x));&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But for the interval in question the results are completely off. For example I want to evaluate $Ei(5)\approx{}40.2$, but i am getting $\approx{}19.0654$. The paper claims a maximal relative error of $8\cdot10^{-19}$, so I guess I must be doing something wrong. I double checked the coefficients and my implementation of the equation but I don't see any mistake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I plot the relative error of the C calculated value compared to the real/expected value (as returned by the ExpIntegralEi function of Mathematica) I can see that only for the first interval ($0&amp;lt;x\le6$) the error is unreasonably high:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/44QPM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/44QPM.jpg&quot; alt=&quot;Relative error, compared to Mathematica&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotted is $y(x)=\left|\frac{C\left(x\right)-M\left(x\right)}{M\left(x\right)}\right|$, where $C(x)$ is the value as returned by my C program and M(x) the same value as returned by Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Cody, W. J., and Henry C. Thacher. &lt;a href=&quot;http://www.ams.org/mcom/1969-23-106/S0025-5718-1969-0242349-2/S0025-5718-1969-0242349-2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Chebyshev approximations for the exponential integral  ().&quot;&lt;/a&gt; Mathematics of Computation 23.106 (1969): 289-303.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21811" OwnerDisplayName="jitter" LastEditorUserId="9667" LastEditDate="2016-10-03T18:37:38.600" LastActivityDate="2016-10-04T18:44:36.007" Title="Numerical evaluation of the Exponential Integral Ei by rational Chebyshev approximations fails" Tags="|c|approximation|special-functions|numerical-limitations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25106" PostTypeId="2" ParentId="25069" CreationDate="2016-10-03T17:40:07.287" Score="0" Body="&lt;p&gt;Finally got this to work. So the answer is no, you shouldn't have to scale anything in the simulation to get the it to work. The problem was much simpler than that and not related to the paper at all. It was a coding error, I wasn't paying attention to the order of operations in one line of code, explaining the weak forces. The second thing is getting the initial settings correct for the simulation, which happened through experimentation. &lt;/p&gt;&#xA;" OwnerUserId="21805" LastActivityDate="2016-10-03T17:40:07.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25107" PostTypeId="2" ParentId="25105" CreationDate="2016-10-03T18:13:59.050" Score="10" Body="&lt;p&gt;There is a prime missing both in your code and in the expression in your question. In the original paper, the expression is:&#xA;$$ \log(x/x_0) + (x-x_0) \frac{\sum^\prime_{0\leq j\leq n}p_j T_j^*(x/6)}{\sum^\prime_{0\leq j\leq n}q_j T_j^*(x/6)}. $$&#xA;This &lt;em&gt;primed sum&lt;/em&gt; is very common and standard when dealing with Chebyshev series: it means that the first term of the sum, corresponding to the zeroth Chebyshev polynomial (the constant term), must be &lt;em&gt;divided by two&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The origin of this modification is the identity (&lt;a href=&quot;http://mathworld.wolfram.com/ChebyshevPolynomialoftheFirstKind.html&quot; rel=&quot;noreferrer&quot;&gt;(25) on mathworld&lt;/a&gt;)&#xA;$$ \sum_{1\leq k\leq m} T_i(x_k) T_j(x_k) = \frac{[i=j]m}{1 + [i=j\neq0]}  $$&#xA;where $x_k$ are the roots of $T_m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After dividing $p_0$ and $q_0$ by $2$, I think it seems to work fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I would like to note that the standard, and really quite easy, way to evaluate Chebyshev series is with the Clenshaw recurrence, which is both much faster than your own implementation of Chebyshev polynomials (which takes time exponential in $n$ to evaluate $T_n$), and more accurate. In cases where coefficients have very large magnitudes, it can matter quite a bit, even though for some reason that doesn't seem to be the case here.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-10-04T18:44:36.007" LastActivityDate="2016-10-04T18:44:36.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25108" PostTypeId="2" ParentId="25073" CreationDate="2016-10-03T20:31:52.803" Score="0" Body="&lt;p&gt;The main reason my solution did not work was that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$c_r=\left({1\over 2}\right)\left[{c_{n,j+1}-c_{n,j-1}\over \Delta r}+{c_{n+1,j+1}-c_{n+1,j-1}\over \Delta r}\right]$$&#xA;is incorrect. For the centered-difference approximation for $c_r$, the denominator should be $2\Delta r$, not $\Delta r$. There is an additional factor of $1\over 2$ from the Crank-Nicolson stencil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Making this correction fixed most, but not all, of the mass loss. There is still some problem, most likely in the application of the boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="21808" LastEditorUserId="21808" LastEditDate="2016-10-03T20:45:29.930" LastActivityDate="2016-10-03T20:45:29.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25110" PostTypeId="1" AcceptedAnswerId="25151" CreationDate="2016-10-04T03:25:45.530" Score="10" ViewCount="17197" Body="&lt;p&gt;Now I have been &lt;em&gt;believing&lt;/em&gt; that FEM/CFD is supposed to be faster on a GPU unit -  here I am using CUDA as solid example.  However, I have not been able to find a convincing paper where the benchmark actually appear to me that 'Yes, this is true!'. Can I please be pointed to one? Or if not, is there any reasons why GPU unit can suck when compared to CPU for CFD/FEM? Could it have anything to do with sparse matrices structure? In terms of performance index like speed/degree of parallelism etc.&lt;/p&gt;&#xA;" OwnerUserId="21881" LastEditorUserId="1280" LastEditDate="2016-10-13T15:43:44.077" LastActivityDate="2022-06-25T09:13:02.753" Title="The real myth of GPU (specifically CUDA) really speed up FEM/CFD" Tags="|finite-element|fluid-dynamics|performance|cuda|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25111" PostTypeId="1" CreationDate="2016-10-04T09:08:13.603" Score="1" ViewCount="89" Body="&lt;p&gt;It is my understanding that the multigrid solution techniques are generally the preferable method to solve large Poisson problems. Now assume I have written a multigrid solver that is tailored to my special needs that actually works quite well, i.e. the residual norm decreases by a factor ~10 with every successive V-cycle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I actually show/prove/test for textbook efficiency? Can optimal scalability only be shown for a fixed continuous problem that is discretized onto increasingly finer grids? If so, are there any special properties such a problem should have?&lt;/p&gt;&#xA;" OwnerUserId="20867" LastActivityDate="2016-10-04T09:08:13.603" Title="What does it take to prove that a multigrid algorithm scales linearly with system size?" Tags="|linear-solver|multigrid|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25112" PostTypeId="1" CreationDate="2016-10-04T11:52:51.190" Score="0" ViewCount="403" Body="&lt;p&gt;I am applying a 4th order Runge-Kutta code, using the method of lines, to solve the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation} \frac {\partial y_1}{\partial t} = y_2 y_3  - C_1 y_1 \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation} \frac {\partial y_2}{\partial t} = y_3y_1 - C_2 y_2 \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation} \frac {\partial y_3}{\partial z} = y_4 \end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation} \frac {\partial y_4}{\partial z} = \frac {\partial ^2 y_2}{\partial t^2} + \frac {\partial ^2 y_3}{\partial t^2} + y_1y_3 - y_2y_3 + y_3^2y_2 - y_4\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(where $ y_n$ are the solutions I am looking for, $ C_n $ are real constants)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code appears to work and solve the system when I just remove the $ y_4 $ term from equation (4), but otherwise, the solution goes to infinity. I have tried rescaling parameters/variables, decreasing the spatial intervals, and changing the boundary/initial conditions (except trivial ones), but these seem not to work. For testing purposes, I also changed equation (4) to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\partial y_4}{\partial z} = \frac {\partial ^2 y_2}{\partial t^2} + \frac {\partial ^2 y_3}{\partial t^2} + y_1y_3 - y_2y_3 + y_3^2y_2 - \text {SCALE}*y_4 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where SCALE = $10^{-5}$. With this change, the output did not go to infinity, although it had an unexpected output (namely there were frequent large oscillations). I could not change SCALE to values larger than $10^{-5}$, otherwise the output would go to infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RK4 seems to work well for almost everything except this one term in the 4th equation, and I was just wondering if you had any ideas? I was considering possibly applying an implicit method on only the 3rd and 4th equations, although due to the nonlinearity, I haven't been able to isolate for the $y_3$ when using higher order BDF methods.&lt;/p&gt;&#xA;" OwnerUserId="20407" LastActivityDate="2016-12-08T03:23:12.420" Title="Applying Runge-Kutta to nonlinear system of PDEs" Tags="|pde|nonlinear-equations|runge-kutta|stiffness|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25113" PostTypeId="1" CreationDate="2016-10-04T12:03:50.970" Score="2" ViewCount="99" Body="&lt;p&gt;There is a huge literature on efficient preconditioners for saddle-point problems. In computational physics, the case where the Lagrange multipliers enjoy a weak formulation (say, the Stokes equations) is almost always considered. In such cases, very fast solvers can be designed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, in structural mechanics, the Lagrange multipliers are often introduced on the discrete level. Consider for instance multi-points constraints between degrees of freedom (this feature is very common in industrial software). I know this is not the good way of doing things, but this is how it is often done. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know about the ancient work of St-Georges, Notay, Warzée, or the recent work of Rees and Greif, Gould, Hribar and Nocedal. &#xA;But I wonder if someone has experienced other strategies (such as the ones available in the FieldSplit preconditioners of PETSc for instance) for preconditioning saddle point systems with those horrible discrete Lagrange multipliers? I am indded seeking a way of using efficient preconditioners (say multigrid) on these saddle-point systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I wonder if the best way to use efficient preconditioners with Lagrange multipliers wouldn't be the domain decomposition methods, such as FETI or BDD, that can naturally handle multipliers.&lt;/p&gt;&#xA;" OwnerUserId="21884" LastActivityDate="2016-10-04T12:03:50.970" Title="Preconditioners and discrete Lagrange multipliers" Tags="|preconditioning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25114" PostTypeId="1" CreationDate="2016-10-04T12:25:05.817" Score="2" ViewCount="849" Body="&lt;p&gt;I was trying to understand how to use the inverse interation method to compute the page rank as an exercise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://au.mathworks.com/moler/exm/chapters/pagerank.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this chapter (page 4) about page rank&lt;/a&gt; (by Cleve Moler), the author suggests to use the following statements to compute the page rank using the &lt;em&gt;inverse iteration&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c = sum(G, 1);&#xA;k = find(c ~= 0);&#xA;D = sparse(k, k, 1 ./ c(k), n, n);&#xA;e = ones(n, 1);&#xA;I = speye(n, n);&#xA;...&#xA;&#xA;A = p*G*D + delta&#xA;x = (I - A) \ e&#xA;x = x / sum(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;p&lt;/code&gt; should be the probability that a random walk follows a link and &lt;code&gt;delta&lt;/code&gt; should be &lt;code&gt;(1 - p) / n&lt;/code&gt; (as far I've understood), i.e. the probability that a particular random page is chosen. Note that &lt;code&gt;1 - p&lt;/code&gt; is the probability that some arbitrary page is chosen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in the chapter 8 of the book &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/9780898719987&quot; rel=&quot;nofollow noreferrer&quot;&gt;A First Course in Numerical Methods&lt;/a&gt; (about Eigenvalues and Singular Values), at page 228 (at least in this version that I've), there's the following &lt;em&gt;inverse iteration&lt;/em&gt; method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KhNJc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KhNJc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Main Questions&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Excluding the calculation of $\lambda$ (the eigenvalue), what's the relation between this inverse iteration and the previous? How does one reach from this last exposure of the algorithm (in the picture) to the Matlab implementation above? &#xA;A step by step procedure would really be appreciated!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Notes (so that you don't have to answer to these):&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I understood that the shift inverse method consists in finding the eigenven values of $(A - \alpha I)^{-1}$ instead of $A$. Even though I can't visualize really how this would be helpful.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I understood that $(A - \alpha I)\tilde{v} = v_{k - 1}$ comes from $\tilde{v} = (A - \alpha I)^{-1}v_{k - 1}$, but we don't want to compute the inverse of $(A - \alpha I)$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I understood that we can choose $\alpha$ (so that it's close to the wanted eigenvalue). Since we know that the sought eigenvalue is $1$, we the statement $(A - \alpha I)$ may simplify to $(A - I)$...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;Other Minor Questions&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why are we using &lt;code&gt;delta&lt;/code&gt; in &lt;code&gt;A = p*G*D + delta&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What's &lt;code&gt;A&lt;/code&gt; in the &lt;em&gt;inverse iteration&lt;/em&gt; of the picture?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Again in &lt;a href=&quot;http://au.mathworks.com/moler/exm/chapters/pagerank.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this chapter (page 4) about page rank&lt;/a&gt; (by Cleve Moler), the author says (after those statements above) that (I - A) is theoretically singular. Why is that? And how could we avoid it?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user9925" LastEditorDisplayName="user9925" LastEditDate="2017-01-17T14:36:25.690" LastActivityDate="2017-01-18T03:02:31.753" Title="PageRank using Inverse Iteration Method by Cleve Moler" Tags="|matlab|eigenvalues|iterative-method|graph-theory|numerics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25118" PostTypeId="1" AcceptedAnswerId="25119" CreationDate="2016-10-05T08:15:13.463" Score="2" ViewCount="3821" Body="&lt;p&gt;I have a discrete contour represented by a set of points. The contour looks like a polygon but if you zoom you see that the edges are rugged (that's because it was obtained while working on a finite differences grid)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to fit a polygon to this contour (which is closest in the least-squares sense). Are there any simple known algorithms to do this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xwMHp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xwMHp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="549" LastEditDate="2016-10-07T14:15:21.737" LastActivityDate="2020-10-08T18:10:26.213" Title="Fit best polygon to a discrete contour" Tags="|computational-geometry|curve-fitting|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25119" PostTypeId="2" ParentId="25118" CreationDate="2016-10-05T11:50:50.027" Score="2" Body="&lt;p&gt;The Hough Transform is an image processing algorithm for extracting features for an image. The classical version of the algorithm is designed to extract lines from a binary image (such as this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the ability to do this, you can make a script fairly easily. Here's one using the hough transform (and associated utilities) in Matlab. It probably requires the image processing toolbox, but hopefully you can track that down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only shortcut present in this code is that I told the &quot;houghpeaks&quot; function to search for 4 lines. You can be smarter than this by doing other peak detection methods in the Hough space (read the wiki if you're not familiar with the output of the Hough transform, &lt;a href=&quot;https://en.wikipedia.org/wiki/Hough_transform&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Hough_transform&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One caveat is that the Hough transform will only give you whole lines. You'll have to post-process them to get line segments. This example works particularly well without any of that since the input was a convex polygon whose vertices were at/near the edge of the image.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all&#xA;close all&#xA;clc&#xA;&#xA;% Load image (I suspect your input may be different, but the same ideas hold)&#xA;img = imread('polygon.png');&#xA;&#xA;% Make a black/white image using the &quot;blue&quot; as the new white values&#xA;bw = img(:,:,1) ~= 255;&#xA;&#xA;% Use hough transform to find lines&#xA;[H,theta,rho] = hough(bw);&#xA;P = houghpeaks(H,4);&#xA;lines = houghlines(bw,theta,rho,P);&#xA;&#xA;% Visualization&#xA;imagesc(bw); colormap gray;&#xA;hold on;&#xA;for l = 1:length(lines)&#xA;    plot([lines(l).point1(1) lines(l).point2(1)], ...&#xA;         [lines(l).point1(2) lines(l).point2(2)],'r');&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2016-10-05T11:50:50.027" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25120" PostTypeId="1" CreationDate="2016-10-05T12:03:34.173" Score="1" ViewCount="127" Body="&lt;p&gt;I have a working SPH simulation in 2D. When the particles are in motion everything works fine. But once the simulation comes to a rest particles at the bottom begin to oscillate violently and eventually penetrate the boundary particles. I have tried dampening the simulation with two types of viscosity, XSPH and Artificial viscosity. They only helped slightly. Smaller time steps don't help either. Obviously pressures are building at the bottom. Is there another way to do fix this apart from viscosity and smaller timesteps?&lt;/p&gt;&#xA;" OwnerUserId="21805" LastActivityDate="2017-11-27T13:38:48.880" Title="Smoothed particle Hydrodynamics: Resting fluid oscillations" Tags="|sph|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25121" PostTypeId="1" AcceptedAnswerId="25140" CreationDate="2016-10-05T14:01:04.283" Score="1" ViewCount="1858" Body="&lt;p&gt;I am interested in writing a simple, cell-centered, 2D FVM code for the unsteady, compressible Navier-Stokes equations (including shocks).  Most of my experience is with finite difference and finite element methods.  I did develop a finite volume code for Sod's problem as a learning exercise a while back.  I've been searching through books and the literature for a simple algorithm which I can implement as a starting point, without much luck.  Most of the more recent papers on finite volume methods are focused on higher-order accurate schemes, etc.  I only really want second-order accuracy in space.  Does anyone know of a good description of an algorithm for the compressible Navier-Stokes equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to avoid DG for now as shock capturing is still an active research area.  Any insights or thoughts would be greatly appreciated.  &lt;/p&gt;&#xA;" OwnerUserId="19799" LastActivityDate="2018-05-14T21:08:39.810" Title="Simple finite volume method for compressible Navier-Stokes equations" Tags="|algorithms|finite-volume|navier-stokes|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25122" PostTypeId="1" CreationDate="2016-10-05T14:04:21.390" Score="1" ViewCount="139" Body="&lt;p&gt;The equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial f/\partial t + \nabla \cdot (v f) = 0$    $, \;\; f \in [0,1] $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $v$ is a velocity known at every grid cell. A more precise constraint is that $f$ is either 0 or 1 but I can also live with it being in the domain [0,1]. The geometry is 2D cylindrical R-Z coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Initial condition, $f(R, Z)$ is known for all zones in the $R-Z$ domain and is not a simple function. Note that $v$ is evolving in time and space i.e. $v$ is really $v(R, Z, t)$. I do not want the solution to be diffusive. &lt;/p&gt;&#xA;" OwnerUserId="21896" LastEditorUserId="21896" LastEditDate="2016-10-05T21:05:43.643" LastActivityDate="2016-10-05T21:05:43.643" Title="What is a good algorithm to solve a discrete continuity equation in Cylindrical coordinates?" Tags="|pde|fluid-dynamics|hyperbolic-pde|numerical-modelling|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25123" PostTypeId="1" CreationDate="2016-10-05T16:52:05.760" Score="1" ViewCount="1818" Body="&lt;p&gt;I am running a PCA analysis on a data set using Python's (v2.7.10) NumPy.  I validated that my program works by running the PCA analysis on a smaller dataset and then confirming that I get similar results using &lt;code&gt;sklearn.decomposition&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run my full dataset (with 60000 variables and 93 samples), I experience some strangeness.  After I construct my covariance matrix (which is 60000 x 60000), I compute the eigenvalues and eigenvectors using &lt;code&gt;numpy.linalg.eig()&lt;/code&gt;.  When I inspect the eigenvalues and eigenvectors, all the entries are exactly 0.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads me to believe that there is something strange with the behavior of &lt;code&gt;numpy.linalg.eig()&lt;/code&gt; due to the large matrix size.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;QUESTION&lt;/strong&gt; : What is going on?  What assumptions are breaking down? How do I fix it? &lt;/p&gt;&#xA;" OwnerUserId="17874" LastEditorUserId="9667" LastEditDate="2016-10-05T20:33:58.410" LastActivityDate="2019-12-26T00:52:34.953" Title="Finding eigenvectors and eigenvalues of large matrices in Python's numpy" Tags="|python|numpy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25125" PostTypeId="1" AcceptedAnswerId="25139" CreationDate="2016-10-06T03:04:35.507" Score="1" ViewCount="3636" Body="&lt;p&gt;I am solving the simple 2nd-order wave equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\partial ^2 E}{\partial t^2} = c^2 \frac {\partial ^2 E}{\partial z^2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over a domain of (in SI units):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ z = [0,L=5]$m, $t = [0,t_{max} = 5]$s, $ c = 1$ m/s&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and boundary/initial conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ E(z=0) = E(z=L) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ E(t=0) = -E(t=t_{max}) = sin(\frac{\pi z}{L}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the analytic solution, but am trying to solve it numerically. (The numerical and analytic solutions are compared to test accuracy.) I just had a couple questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) When I solve the wave equation by applying a central difference approximation of order 2 on the second derivative of time, the code works perfectly fine. Although when I apply &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_coefficient#Forward_and_backward_finite_difference&quot; rel=&quot;nofollow noreferrer&quot;&gt;backward finite difference&lt;/a&gt; approximations of higher orders for the second derivative in time, my solution diverges. Is there any particular reason why using a backward difference approximation would be worse than a central difference approximation of the same order? I was under the impression using a higher order backward difference method would give me higher accuracy, but it appears to not work at all. I can supply the code if needed, although it's a fairly basic implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) I am able to solve the equation above when using a central difference approximation and setting $ c = 1$ m/s, but if I use $c = 3 \times 10^8$ m/s, the solution diverges. This makes sense as the Courant number ($ C_o = c \frac {\Delta t}{\Delta x}$) is much greater than one. But I am curious: are there any possible finite difference schemes that can be applied to numerically solve the equation if $C_o &amp;gt;&amp;gt; 1$? I have read a bit about BDF methods to help solve stiff equations, but in this case, the only problem is that a more precise grid is needed, correct? I had also heard about spectral methods, but am still trying to learn how to implement them and if they would be applicable/accurate enough if other (nonlinear) terms were included in the wave equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After further testing, it appears the output is actually slightly off even without the backward differences. The code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib&#xA;matplotlib.use('pdf')&#xA;import os &#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;from tqdm import tqdm&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;from matplotlib import cm&#xA;import matplotlib.pyplot as plt&#xA;from matplotlib.colors import Normalize&#xA;&#xA;c = 1.#3.*(10.**8.)&#xA;&#xA;#z space&#xA;Ngridz = 400 #number of intervals in z-axis&#xA;zmax = L = 5.&#xA;dz = zmax/Ngridz &#xA;z = np.linspace(0.0,zmax,Ngridz+1) &#xA;&#xA;#time&#xA;Ngridt = 400 #number of intervals in t-axis&#xA;tmax = L/c&#xA;dt = tmax/Ngridt&#xA;t = np.linspace(0.0,tmax,Ngridt+1)  &#xA;&#xA;#def dt2(X,k,i,z,t,kX,TYPE): &#xA;#    &quot;&quot;&quot;Approximation for second time derivative&quot;&quot;&quot; - BACKWARD DIFFERENCE&#xA;#    ht = t[1]-t[0]&#xA;#    if TYPE == 'CONJ':&#xA;#        if i == 0:&#xA;#            kTT = np.conj(X[k,i]-2.*X[k,i-1]+X[k,i-2])/(ht**2.)&#xA;#        elif i == 1:&#xA;#            kTT = np.conj(X[k,i-1]-2.*X[k,i]-X[k,i+1])/(ht**2.)&#xA;#        elif i == 2:&#xA;#            kTT = np.conj(-(X[k,i])+2.*(X[k,i-1])-(X[k,i-2]))/(ht**2.)&#xA;#        elif i == 3:&#xA;#            kTT = np.conj(-2.*(X[k,i])+5.*(X[k,i-1])-&#xA;#            4.*(X[k,i-2])+(X[k,i-3]))/(ht**2.)&#xA;#        elif i == 4:&#xA;#            kTT = np.conj((-35/12.)*(X[k,i])+(26./3.)*(X[k,i-1])-&#xA;#            (19./2.)*(X[k,i-2])+(14./3.)*(X[k,i-3])-&#xA;#            (11./12.)*(X[k,i-4]))/(ht**2.)&#xA;#        elif i == 5:&#xA;#            kTT = np.conj((-15./4.)*(X[k,i])+(77./6.)*(X[k,i-1])-&#xA;#            (107./6.)*(X[k,i-2])+13.*(X[k,i-3])-&#xA;#            (61./12.)*(X[k,i-4])+(5./6.)*(X[k,i-5]))/(ht**2.)&#xA;#        else:&#xA;#            kTT = np.conj((-203./45.)*(X[k,i])+(87./5.)*(X[k,i-1])-&#xA;#            (117./4.)*(X[k,i-2])+(254./9.)*(X[k,i-3])-&#xA;#            (33./2.)*(X[k,i-4])+(27./5.)*(X[k,i-5])-&#xA;#            (137./180.)*(X[k,i-6]))/(ht**2.)       &#xA;#    else: &#xA;#        if i == 0:&#xA;#            kTT = (X[k,i]-2.*X[k,i-1]+X[k,i-2])/(ht**2.)&#xA;#        elif i == 1:&#xA;#            kTT = (X[k,i-1]-2.*X[k,i]-X[k,i+1])/(ht**2.)&#xA;#        elif i == 2:&#xA;#            kTT = (-(X[k,i])+2.*(X[k,i-1])-(X[k,i-2]))/(ht**2.)&#xA;#        elif i == 3:&#xA;#            kTT = (-2.*(X[k,i])+5.*(X[k,i-1])-&#xA;#            4.*(X[k,i-2])+(X[k,i-3]))/(ht**2.)&#xA;#        elif i == 4:&#xA;#            kTT = ((-35/12.)*(X[k,i])+(26./3.)*(X[k,i-1])-&#xA;#            (19./2.)*(X[k,i-2])+(14./3.)*(X[k,i-3])-&#xA;#            (11./12.)*(X[k,i-4]))/(ht**2.)&#xA;#        elif i == 5:&#xA;#            kTT = ((-15./4.)*(X[k,i])+(77./6.)*(X[k,i-1])-&#xA;#            (107./6.)*(X[k,i-2])+13.*(X[k,i-3])-&#xA;#            (61./12.)*(X[k,i-4])+(5./6.)*(X[k,i-5]))/(ht**2.)&#xA;#        else:&#xA;#            kTT = ((-203./45.)*(X[k,i])+(87./5.)*(X[k,i-1])-&#xA;#            (117./4.)*(X[k,i-2])+(254./9.)*(X[k,i-3])-&#xA;#            (33./2.)*(X[k,i-4])+(27./5.)*(X[k,i-5])-&#xA;#            (137./180.)*(X[k,i-6]))/(ht**2.)      &#xA;#    return kTT&#xA;&#xA;def dt2(X,k,i,z,t,kX,TYPE): &#xA;    &quot;&quot;&quot;Approximation for second time derivative&quot;&quot;&quot; - CENTRAL DIFFERENCE&#xA;    ht = t[1]-t[0]&#xA;    if TYPE == 'CONJ':&#xA;        if i == 0:&#xA;            kTT = np.conj(X[k,i]-2.*X[k,i+1]+X[k,i+2])/(ht**2.)&#xA;        else:&#xA;            kTT = np.conj(X[k,i-1]-2.*X[k,i]-X[k,i+1])/(ht**2.) &#xA;    else: &#xA;        if i == 0:&#xA;            kTT = (X[k,i]-2.*X[k,i+1]+X[k,i+2])/(ht**2.) &#xA;        else: &#xA;            kTT = (X[k,i-1]-2.*X[k,i]+X[k,i+1])/(ht**2.) &#xA;    return kTT&#xA;&#xA;Ep = np.zeros((len(z),len(t)),dtype=np.complex_)  &#xA;TEST = np.zeros((len(z),len(t)),dtype=np.complex_)  &#xA;&#xA;progress = tqdm(total=100.) #this provides a progress bar&#xA;total = 100.&#xA;progressz = (total)/(len(z))&#xA;&#xA;k = 0&#xA;while k &amp;lt; (len(z) - 1):&#xA;    progress.update(progressz)&#xA;&#xA;    EpM = Ep&#xA;&#xA;    hz = z[k+1]-z[k] #hz is positive&#xA;&#xA;    i = 0&#xA;    while i &amp;lt; (len(t) - 1):&#xA;        ht = t[i+1] - t[i]&#xA;&#xA;        EpM[0,i] = 0. #at z = 0 &#xA;#        EpM[Ngridz-1,:] = 0. &#xA;        EpM[k,0] = np.sin(np.pi*z[k]) &#xA;#        EpM[k+1,0] = np.sin(np.pi*z[k+1])&#xA;        EpM[k,Ngridt-1] = -np.sin(np.pi*z[k])    &#xA;&#xA;        EpM[k+1,i] = (-EpM[k,i-1] + 2.*EpM[k,i] + ((hz/(c))**2.)*dt2(EpM,k,i,z,t,0.,'x'))&#xA;#((hz/(c*ht))**2.)*(EpM[k,i+1] - 2.*EpM[k,i] + EpM[k,i-1]))&#xA;&#xA;&#xA;#        print 'k='+str(k)+',i='+str(i)+',dE='+str(EpM[k+1,i]-EpM[k,i])&#xA;&#xA;        EpM[0,i] = 0. #at z = 0 &#xA;#        EpM[Ngridz-1,:] = 0. &#xA;        EpM[k,0] = np.sin(np.pi*z[k]) &#xA;#        EpM[k+1,0] = np.sin(np.pi*z[k+1])&#xA;        EpM[k,Ngridt-1] = -np.sin(np.pi*z[k])   &#xA;&#xA;        TEST[k,i] = np.sin(np.pi*z[k])*np.cos(np.pi*t[i])&#xA;&#xA;        i = i + 1 &#xA;&#xA;        Ep[k,:] = EpM[k,:]&#xA;&#xA;    k = k + 1&#xA;&#xA;    if k == (len(z)-1):  &#xA;        progress.update(progressz) &#xA;&#xA;Ereal = (Ep).real&#xA;&#xA;newpath = r'test_wave_equation'&#xA;if not os.path.exists(newpath):&#xA;    os.makedirs(newpath)&#xA;&#xA;plt.figure(1)&#xA;fig, ax = plt.subplots(figsize=(20, 20))&#xA;plt.subplot(221)&#xA;plt.plot(t,Ereal[0,:],'k:',linewidth = 1.5)&#xA;plt.ylabel('Normalized E.real')&#xA;plt.subplot(222)&#xA;plt.plot(t,Ereal[int(Ngridz*0.33),:],'k:',linewidth = 1.5)&#xA;plt.subplot(223)&#xA;plt.plot(t,Ereal[int(Ngridz*0.66),:],'k:',linewidth = 1.5)&#xA;plt.subplot(224)&#xA;plt.plot(t,Ereal[int(Ngridz),:],'k:',linewidth = 1.5)&#xA;plt.xlabel(r&quot; t (s)&quot;)&#xA;plt.savefig(str(newpath)+'/E.real_vs_t.pdf')&#xA;#plt.show()&#xA;&#xA;plt.figure(2)&#xA;fig, ax = plt.subplots(figsize=(20, 20))&#xA;plt.subplot(221)&#xA;plt.plot(z,Ereal[:,0],'k:',linewidth = 1.5)&#xA;plt.ylabel('Normalized E.real')&#xA;plt.subplot(222)&#xA;plt.plot(z,Ereal[:,int(Ngridt*0.33)],'k:',linewidth = 1.5)&#xA;plt.subplot(223)&#xA;plt.plot(z,Ereal[:,int(Ngridt*0.66)],'k:',linewidth = 1.5)&#xA;plt.subplot(224)&#xA;plt.plot(z,Ereal[:,Ngridt],'k:',linewidth = 1.5)&#xA;plt.xlabel(r&quot; z (m)&quot;)&#xA;plt.savefig(str(newpath)+'/E.real_vs_z.pdf') &#xA;&#xA;plt.figure(3)&#xA;fig, ax = plt.subplots(figsize=(20, 20))&#xA;plt.subplot(221)&#xA;plt.plot(t,TEST[0,:],'k:',linewidth = 1.5)&#xA;plt.ylabel('True E')&#xA;plt.subplot(222)&#xA;plt.plot(t,TEST[int(Ngridz*0.33),:],'k:',linewidth = 1.5)&#xA;plt.subplot(223)&#xA;plt.plot(t,TEST[int(Ngridz*0.66),:],'k:',linewidth = 1.5)&#xA;plt.subplot(224)&#xA;plt.plot(t,TEST[int(Ngridz),:],'k:',linewidth = 1.5)&#xA;plt.xlabel(r&quot; t (s)&quot;)&#xA;plt.savefig(str(newpath)+'/E.real_vs_t.pdf')&#xA;#plt.show()&#xA;&#xA;plt.figure(4)&#xA;fig, ax = plt.subplots(figsize=(20, 20))&#xA;plt.subplot(221)&#xA;plt.plot(z,TEST[:,0],'k:',linewidth = 1.5)&#xA;plt.ylabel('Normalized E.real')&#xA;plt.subplot(222)&#xA;plt.plot(z,TEST[:,int(Ngridt*0.33)],'k:',linewidth = 1.5)&#xA;plt.subplot(223)&#xA;plt.plot(z,TEST[:,int(Ngridt*0.66)],'k:',linewidth = 1.5)&#xA;plt.subplot(224)&#xA;plt.plot(z,TEST[:,Ngridt],'k:',linewidth = 1.5)&#xA;plt.xlabel(r&quot; z (m)&quot;)&#xA;plt.savefig(str(newpath)+'/E.real_vs_z.pdf') &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output: &lt;a href=&quot;https://i.stack.imgur.com/NianM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/NianM.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output of backward difference approximation: &lt;a href=&quot;https://i.stack.imgur.com/Lvmxv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/Lvmxv.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you may see, my output does not seem to match the analytic solution, and this discrepancy only appears to be present initially when using the central difference approximation. As the time- and z-components advance, it appears my solution does converge to the right answer. I have tried increasing the number of grid points (i.e. decreasing interval size) and debugging but I don't seem to see any obvious errors. Any ideas on what may be causing the problem? I have commented out the function for the backward difference approximation of the second time derivative, but when I uncomment it and use it instead of the central difference function for dt2, the solution diverges and the original problem still remains.&lt;/p&gt;&#xA;" OwnerUserId="20407" LastEditorUserId="20407" LastEditDate="2016-10-06T19:19:58.323" LastActivityDate="2016-10-07T00:39:13.197" Title="Using backward vs central finite difference approximation" Tags="|finite-difference|python|wave-propagation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25126" PostTypeId="2" ParentId="20152" CreationDate="2016-10-06T06:03:52.490" Score="10" Body="&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/a/20164/20458&quot;&gt;Kirill's answer&lt;/a&gt; already touched on a large number of relevant issues. I would like to expand on some of them based on practical math library design experience. A note up front: math library designers tend to use every published algorithmic optimization, as well as many machine specific optimizations, not all of which will be published. The code frequently will be written in assembly language, rather than use compiled code. It is therefore unlikely that a straightforward and compiled implementation will achieve more than 75% of the performance of an existing high-quality math library implementation, assuming identical feature sets (accuracy, special cases handling, error reporting, rounding-mode support).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of accuracy, for elementary functions (e.g. &lt;span class=&quot;math-container&quot;&gt;$exp$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$log$&lt;/span&gt;) as well as simple special functions (e.g. &lt;span class=&quot;math-container&quot;&gt;$erfc$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\Gamma$&lt;/span&gt;), &lt;a href=&quot;https://en.wikipedia.org/wiki/Unit_in_the_last_place&quot; rel=&quot;noreferrer&quot;&gt;ulp&lt;/a&gt; error has replaced relative error as the relevant error metric. A common design goal for elementary functions is a maximum error of less than 1 ulp, resulting in a faithfully-rounded function. A faithfully rounded function returns either the floating-point number closest to the infinitely precise result, or an immediately adjacent floating-point number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accuracy is typically assessed by comparison with a (third-party) higher-precision reference. Single-argument single-precision functions can easily be tested exhaustively, other functions require testing with (directed) random test vectors. Clearly one cannot compute infinitely precise reference results, but research into the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rounding#Table-maker.27s_dilemma&quot; rel=&quot;noreferrer&quot;&gt;Table-Maker's Dilemma&lt;/a&gt; suggests that for many simple functions it suffices to compute a reference with a precision of about three times the target precision. See for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vincent Lefèvre, Jean-Michel Muller, &quot;Worst Cases for Correct Rounding of the Elementary Functions in Double Precision&quot;. In &lt;em&gt;Proceedings 15th IEEE Symposium on Computer Arithmetic&lt;/em&gt;, 2001,111-118).&lt;a href=&quot;https://hal.archives-ouvertes.fr/file/index/docid/72594/filename/RR-4044.pdf&quot; rel=&quot;noreferrer&quot;&gt;(preprint online)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of performance, one has to distinguish between optimizing for latency (important when one looks at the execution time of dependent operations), versus optimizing for throughput (relevant when considering execution time of independent operations). Over the past twenty years, the proliferation of hardware parallelization techniques such as instruction level parallelism (e.g. superscalar, out-of-order processors), data-level parallelism (e.g. SIMD instructions), and thread-level parallelism (e.g. hyper-threading, multi-core processors) has lead to an emphasis on computational throughput as the more relevant metric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Core approximations for elementary functions are almost exclusively &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax_approximation_algorithm&quot; rel=&quot;noreferrer&quot;&gt;minimax approximations&lt;/a&gt;. Due to the relatively high cost of division, polynomial minimax approximations. Tools such as Mathematica or Maple have built-in facilities for generating these; there are also specialized tools such as &lt;a href=&quot;http://sollya.gforge.inria.fr/&quot; rel=&quot;noreferrer&quot;&gt;Sollya&lt;/a&gt;. For the logarithm, the basic core approximation choices, after argument reduction to values close to unity, are &lt;span class=&quot;math-container&quot;&gt;$log (1+x) = p(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$log (x) = 2 \cdot atanh ((x-1)/ (x+1)) = p(((x-1)/ (x+1))^{2})$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a polynomial minimax approximation. Performance-wise, the former is usually preferred for single-precision implementations (see &lt;a href=&quot;https://stackoverflow.com/questions/39821367/very-fast-logarithm-natural-log-function-in-c/39822314#39822314&quot;&gt;this answer&lt;/a&gt; for a worked example), while the latter is preferred for double-precision implementations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fused multiply-add operation (&lt;a href=&quot;https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation#Fused_multiply.E2.80.93add&quot; rel=&quot;noreferrer&quot;&gt;FMA&lt;/a&gt;), first introduced by IBM 25 years ago, and now available on all major processor architectures, is a crucial building block of modern math library implementations. It provides rounding error reduction, gives limited protection against &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_of_significance&quot; rel=&quot;noreferrer&quot;&gt;subtractive cancellation&lt;/a&gt;, and vastly simplifies &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation#Fused_multiply.E2.80.93add&quot; rel=&quot;noreferrer&quot;&gt;double-double arithmetic&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exemplary IEEE-754 double-precision &lt;code&gt;C99&lt;/code&gt; implementation of &lt;code&gt;log()&lt;/code&gt; below demonstrates the use of FMA (exposed in &lt;code&gt;C99&lt;/code&gt; as the &lt;code&gt;fma()&lt;/code&gt; standard math function), along with very limited use of double-double techniques to improve the accuracy of products with transcendental constants. Two different core approximations are provided, both delivering faithfully-rounded results, as demonstrated by testing with &lt;span class=&quot;math-container&quot;&gt;$2^{32}$&lt;/span&gt; random test vectors. The minimax approximations used were computed with my own tools based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Remez_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Remez algorithm&lt;/a&gt;. A second-order Horner scheme is used to evaluate most of the straight polynomial approximation (&lt;code&gt;USE_ATANH = 0&lt;/code&gt;) to maximize instruction level parallelism.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;math.h&amp;gt;&#xA;&#xA;/* compute natural logarithm&#xA;&#xA;   USE_ATANH == 1: maximum error found: 0.84555 ulp&#xA;   USE_ATANH == 0: maximum error found: 0.84995 ulp&#xA;*/&#xA;double my_log (double a)&#xA;{&#xA;    const double LOG2_HI = 0x1.62e42fefa39efp-01; // 6.9314718055994529e-01&#xA;    const double LOG2_LO = 0x1.abc9e3b39803fp-56; // 2.3190468138462996e-17&#xA;    double m, r, i, s, t, p, f, q;&#xA;    int e;&#xA;&#xA;    m = frexp (a, &amp;amp;e);&#xA;    if (m &amp;lt; 0.70703125) { // 181/256&#xA;        m = m + m;&#xA;        e = e - 1;&#xA;    }&#xA;    i = (double)e;&#xA;&#xA;    /* m in [181/256, 362/256] */&#xA;&#xA;#if USE_ATANH&#xA;&#xA;    /* Compute q = (m-1) / (m+1) */&#xA;    p = m + 1.0;&#xA;    m = m - 1.0;&#xA;    q = m / p;&#xA;&#xA;    /* Compute (2*atanh(q)/q-2*q) as p(q**2), q in [-75/437, 53/309] */&#xA;    s = q * q;&#xA;    r =            0x1.2f1da230fc404p-3;  // 1.4800574028006974e-1&#xA;    r = fma (r, s, 0x1.399f73f9346b4p-3); // 1.5313616375219896e-1&#xA;    r = fma (r, s, 0x1.746654253068cp-3); // 1.8183580149166223e-1&#xA;    r = fma (r, s, 0x1.c71c51a8bf1eep-3); // 2.2222198291991851e-1&#xA;    r = fma (r, s, 0x1.249249425f16bp-2); // 2.8571428744887467e-1&#xA;    r = fma (r, s, 0x1.999999997f6b1p-2); // 3.9999999999404695e-1&#xA;    r = fma (r, s, 0x1.5555555555593p-1); // 6.6666666666667351e-1&#xA;    r = r * s;&#xA;&#xA;    /* log(a) = 2*atanh(q) + i*log(2) = LOG2_LO*i + p(q**2)*q + 2q + LOG2_HI*i.&#xA;       Use K.C. Ng's trick to improve the accuracy of the computation, like so:&#xA;       p(q**2)*q + 2q = p(q**2)*q + q*t - t + m, where t = m**2/2.&#xA;    */&#xA;    t = m * m * 0.5;&#xA;    r = fma (q, t, fma (q, r, LOG2_LO * i)) - t + m;&#xA;    r = fma (LOG2_HI, i, r);&#xA;&#xA;#else // USE_ATANH&#xA;&#xA;    /* Compute f = m-1 */&#xA;    f = m - 1.0;&#xA;    s = f * f;&#xA;&#xA;    /* Approximate log1p (f), f in [-75/256, 106/256] */&#xA;    r =            -0x1.961d64dc5342dp-6;  // -2.4787281510089954e-2&#xA;    t =             0x1.d35fd59a178ebp-5;  //  5.7052533332113096e-2&#xA;    r = fma (r, s, -0x1.fcf513884b8f8p-5); // -6.2128580235695396e-2&#xA;    t = fma (t, s,  0x1.b9711474aee8ep-5); //  5.3886928513262253e-2&#xA;    r = fma (r, s, -0x1.b5b5054106abcp-5); // -5.3431043874487355e-2&#xA;    t = fma (t, s,  0x1.dd660c0bc8ec6p-5); //  5.8276198890124628e-2&#xA;    r = fma (r, s, -0x1.00bda5ecd0075p-4); // -6.2680862564777076e-2&#xA;    t = fma (t, s,  0x1.1159b2e3be946p-4); //  6.6735934054951235e-2&#xA;    r = fma (r, s, -0x1.2489f14dd7c1cp-4); // -7.1420614809071414e-2&#xA;    t = fma (t, s,  0x1.3b0ee248a04fbp-4); //  7.6918491287887678e-2&#xA;    r = fma (r, s, -0x1.55557d3b4941fp-4); // -8.3333481965909048e-2&#xA;    t = fma (t, s,  0x1.745d4666f7eecp-4); //  9.0909266480135364e-2&#xA;    r = fma (r, s, -0x1.999999d9593d1p-4); // -1.0000000092766405e-1&#xA;    t = fma (t, s,  0x1.c71c70bbce7b6p-4); //  1.1111110722131809e-1&#xA;    r = fma (r, s, -0x1.fffffffa6167cp-4); // -1.2499999991822536e-1&#xA;    t = fma (t, s,  0x1.249249262c6b7p-3); //  1.4285714290376969e-1&#xA;    r = fma (r, s, -0x1.555555555f02ap-3); // -1.6666666666776681e-1&#xA;    t = fma (t, s,  0x1.99999999975b5p-3); //  1.9999999999974497e-1&#xA;    r = fma (r, f, t);&#xA;    r = fma (r, f, -0x1.fffffffffff54p-3); // -2.4999999999999523e-1&#xA;    r = fma (r, f,  0x1.555555555555bp-2); //  3.3333333333333365e-1&#xA;    r = fma (r, f, -0x1.0000000000000p-1); // -5.0000000000000000e-1&#xA;&#xA;    /* log(a) = log1p (f) + i * log(2) */&#xA;    p = fma ( LOG2_HI, i, f);&#xA;    t = fma (-LOG2_HI, i, p);&#xA;    f = fma ( LOG2_LO, i, f - t);&#xA;    r = fma (r, s, f);&#xA;    r = r + p;&#xA;&#xA;#endif // USE_ATANH&#xA;&#xA;    /* Handle special cases */&#xA;    if (!((a &amp;gt; 0.0) &amp;amp;&amp;amp; (a &amp;lt;= 0x1.fffffffffffffp1023))) {&#xA;        r = a + a;  // handle inputs of NaN, +Inf&#xA;        if (a  &amp;lt; 0.0) r =  0.0 / 0.0; //  NaN&#xA;        if (a == 0.0) r = -1.0 / 0.0; // -Inf&#xA;    }&#xA;    return r;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20458" LastEditorUserId="20458" LastEditDate="2020-05-24T01:28:37.600" LastActivityDate="2020-05-24T01:28:37.600" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25127" PostTypeId="1" CreationDate="2016-10-06T12:53:31.520" Score="5" ViewCount="257" Body="&lt;p&gt;I am currently trying to get through Pattern Classification by Duda et al (for a course). However, the book seems too dense for me. Pattern recognition seems like a topic that could be better learned through guided discovery. That made me think... There is a a &lt;a href=&quot;https://math.stackexchange.com/questions/1104996/big-list-of-guided-discovery-books&quot; title=&quot;thread&quot;&gt;thread&lt;/a&gt; over at mathematics SE about books that teach math by guiding you from exercise to exercise; So, essentially you prove every result yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any similar books on the mathematics used in computational science? It seems to me that computational science is something that can be learned by doing.&lt;/p&gt;&#xA;" OwnerUserId="15047" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2021-07-28T07:27:53.543" Title="Learning computational science through guided discovery" Tags="|numerical-analysis|machine-learning|numerics|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25128" PostTypeId="2" ParentId="24140" CreationDate="2016-10-06T14:35:18.543" Score="1" Body="&lt;p&gt;I'm working on a project related to your question, trying to take one implementation of a parallel process and make it 'future proof' by abstracting away as much as possible the specifics of particular parallel architectures and setups. From my research it appears there's no fully-automated parallelism that will efficiently fit any arbitrary algorithm or data type, but there are some tools that can help parallelize existing serial code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For purely Python solutions, there is the &lt;strong&gt;Anaconda Accelerate&lt;/strong&gt; package, which includes Python bindings to CUDA libraries e.g. cuBLAS, cuFFT, cuSPARSE, cuRAND. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;Kokkos&lt;/strong&gt; package within Trilinos provides something like that, though it's a set of C++ libraries so maybe not what you're looking for. As long as you're careful about how to define memory and execution spaces it's possible to quickly compile parallelized code for various architectures without needing to specifically target a particular architecture. So long as your functors and arrays are sensibly defined it's trivial to switch from CUDA to, say, Pthreads, though your performance will vary depending on how cleverly you've defined your memory and execution spaces. This package is particularly well-suited to sparse matrix operations and is most often built into molecular dynamics simulations (so far as I can tell).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your arrays are dense you might consider looking at the &lt;strong&gt;Halide&lt;/strong&gt; language project. The goal of that language is to separate the algorithm from the scheduling, so you can write the algorithm once then focus on tuning the schedule to optimize for various parallel architectures. It is primarily used for image analysis but is appropriate for many other dense array operations. &lt;/p&gt;&#xA;" OwnerUserId="21868" LastEditorUserId="21868" LastEditDate="2016-10-06T17:24:55.633" LastActivityDate="2016-10-06T17:24:55.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25131" PostTypeId="2" ParentId="25060" CreationDate="2016-10-06T19:13:29.777" Score="2" Body="&lt;p&gt;I am not sure if I understand the question completely, but I will try to answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In brackets you have an operator in an infinite dimensional space, although the first one is a derivative and the second one a multiplicative operator. Under some representations they would admit an (infinite) matrix form. And, when you approximate the solution you would end up with a finite dimensional approximation, where the operators are represented by matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I mentioned in the answer you link to, Finite Differences would be the easiest approach, but no the only one. You could try with Finite Element Methods, but the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19455/finite-element-method-for-odd-order-de/19459#19459&quot;&gt;second term makes me wonder&lt;/a&gt;. In the case of Finite Differences, you would end up with similar matrices, but now you have a problem that would not appear in 1D, ordering the cells. In 1D, they are &quot;naturally&quot; ordered from left to right. But in 2D you would need to decide how. The most straightforward would be to consider that you have a rectangular grid and you number by rows first and then by columns (or the opposite way). Nothing says that you cannot have a different order, but it is easier this way. You might found references 1 and 2 useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, taking as a base my code from the previous answer, let's consider the quantum harmonic oscillator first (in normalized units)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[-\frac{1}{2}\nabla^2 + \frac{1}{2}(x^2 + y^2)\right] \psi = E \psi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that has as eigenvalues $E_{m,n} =[(m + n) + 1]$ where $m, n$ are non-negative integers. And, let's assume that we have equal partitions in $x$ and $y$, in number and size $N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Laplace operator would have 5 diagonals, below an example for $N=3$. Three diagonals are consecutive, since the cells are contiguous, and the other one have a spacing of $N$ from the center (as cells in the grid).&#xA;$$T = -\frac{1}{2 \Delta x}\begin{bmatrix}&#xA;    -4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;    1 &amp;amp; -4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;    0 &amp;amp; 1 &amp;amp; -4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;    0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\&#xA;    1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;    0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -4 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;    0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -4 &amp;amp; 1 &amp;amp; 0\\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -4 &amp;amp; 1\\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -4&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other term is the multiplicative operator, that can be represented as a diagonal matrix with the values for $1/2(x^2 + y^2)$ in each cell, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U = \frac{1}{2}\begin{bmatrix}&#xA;    x_0^2 + y_0^2 &amp;amp; 0 &amp;amp;\cdots &amp;amp;0\\&#xA;    0 &amp;amp;x_1^2 + y_1^2 &amp;amp;\cdots &amp;amp;0\\&#xA;    \vdots &amp;amp;\vdots &amp;amp;\ddots  &amp;amp;\vdots\\&#xA;    0 &amp;amp;0 &amp;amp;\cdots &amp;amp;x_{N^2}^2 + y_{N^2}^2&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code below solves this problem, with the first six eigenvalues being&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ 0.9999  1.9997  1.9997  2.9993  2.9993  2.9995]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;that is close to the theoretical values &lt;code&gt;[1.0, 2.0, 2.0, 3.0, 3.0, 3.0]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;import numpy as np&#xA;from scipy.sparse import diags&#xA;from scipy.sparse.linalg import eigsh&#xA;import matplotlib.pyplot as plt&#xA;&#xA;L = 20&#xA;N = 501&#xA;dx = L/(N - 1)&#xA;X, Y = np.mgrid[-L/2:L/2:1j*N,-L/2:L/2:1j*N]&#xA;&#xA;T = -0.5*diags([-4., 1., 1., 1., 1.], [0,-1, 1, -N, N],&#xA;               shape=(N**2, N**2))/dx**2&#xA;U_vec = 0.5*(X**2 + Y**2)&#xA;U_vec.shape = N**2&#xA;U = diags([U_vec], [0])&#xA;&#xA;H = T + U&#xA;&#xA;vals, vecs = eigsh(H, which='SM')&#xA;order = np.argsort(vals)&#xA;vals = vals[order]&#xA;vecs = vecs[:, order]&#xA;&#xA;#%%&#xA;print np.round(vals, 6)&#xA;plt.figure(figsize=(12, 8))&#xA;for k in range(6):&#xA;    plt.subplot(2, 3, k+1)&#xA;    vec = vecs[:, k]&#xA;    mag = np.sqrt(np.dot(vecs[:, k],vecs[:, k]))&#xA;    vec = vec/mag&#xA;    vec.shape = X.shape&#xA;    plt.contourf(X, Y, np.abs(vec), cmap=&quot;viridis&quot;)&#xA;&#xA;plt.savefig(&quot;quantum_harmonic.png&quot;, dpi=600)    &#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YH4Ww.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YH4Ww.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if we add the other term we would have a new operator. I would show it for the first term in parenthesis. The derivative with respect to $x$ (and imaginary unite) can be represented as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D_x = \frac{i}{\Delta x}\begin{bmatrix}&#xA;0 &amp;amp; 1 &amp;amp;\cdots &amp;amp;0 &amp;amp;0\\&#xA;-1 &amp;amp;0 &amp;amp;\cdots &amp;amp;0 &amp;amp;0\\&#xA;\vdots &amp;amp;\vdots &amp;amp;\ddots  &amp;amp;\vdots &amp;amp;\vdots\\&#xA;0 &amp;amp;0 &amp;amp;\cdots  &amp;amp;0 &amp;amp;1\\&#xA;0 &amp;amp;0 &amp;amp;\cdots &amp;amp;-1 &amp;amp;0&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the multiplication by $y$ would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Y = \begin{bmatrix}&#xA;y_0 &amp;amp; 0 &amp;amp;\cdots &amp;amp;0\\&#xA;0 &amp;amp;y_1 &amp;amp;\cdots &amp;amp;0\\&#xA;\vdots &amp;amp;\vdots &amp;amp;\ddots  &amp;amp;\vdots\\&#xA;0 &amp;amp;0 &amp;amp;\cdots &amp;amp;y_{N^2}&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other term is similar, the $D_y$ would have two non-zero diagonals in the $N$ and $-N$ diagonals. The final matrix would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H = T + 2U + YD_x - XD_y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using $2U$ because of the $1/2$ that I used before. The following code presents this approach&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;import numpy as np&#xA;from scipy.sparse import diags&#xA;from scipy.sparse.linalg import eigsh&#xA;import matplotlib.pyplot as plt&#xA;&#xA;L = 20&#xA;N = 501&#xA;dx = L/(N - 1)&#xA;X, Y = np.mgrid[-L/2:L/2:1j*N,-L/2:L/2:1j*N]&#xA;&#xA;T = -0.5*diags([-4., 1., 1., 1., 1.], [0,-1, 1, -N, N],&#xA;               shape=(N**2, N**2), dtype=np.complex)/dx**2&#xA;U_vec = X**2 + Y**2&#xA;U_vec.shape = N**2&#xA;DX = diags([-0.5j, 0.5j], [-1, 1], shape=(N**2, N**2), dtype=np.complex)/dx&#xA;DY = diags([-0.5j, 0.5j], [-N, N], shape=(N**2, N**2), dtype=np.complex)/dx&#xA;Xmat = diags([np.reshape(X, (N**2))], [0], dtype=np.complex)&#xA;Ymat = diags([np.reshape(Y, (N**2))], [0], dtype=np.complex)&#xA;U2 = np.dot(Ymat, DX) - np.dot(Xmat, DY)&#xA;U = diags([U_vec], [0]) +  U2&#xA;&#xA;H = T + U&#xA;&#xA;vals, vecs = eigsh(H, which='SM')&#xA;order = np.argsort(vals)&#xA;vals = vals[order]&#xA;vecs = vecs[:, order]&#xA;&#xA;#%%&#xA;print np.round(vals, 6)&#xA;plt.figure(figsize=(12, 8))&#xA;for k in range(6):&#xA;    plt.subplot(2, 3, k+1)&#xA;    vec = vecs[:, k]&#xA;    mag = np.sqrt(np.dot(vecs[:, k],vecs[:, k]))&#xA;    vec = vec/mag&#xA;    vec.shape = X.shape&#xA;    plt.contourf(X, Y, np.abs(vec), cmap=&quot;viridis&quot;)&#xA;&#xA;&#xA;plt.savefig(&quot;eigenvectors.png&quot;, dpi=600)    &#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The eigenvalues are &lt;code&gt;[ 1.000399  2.001196  2.001196  3.001993  3.002786  3.002786]&lt;/code&gt;, that seem slightly perturbed over the harmonic oscillator... but I don't really know if they are correct or not. The eigenvectors are presented below, the first three make sense to me, but one more time, I don't know about the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NaC53.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NaC53.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I am using &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_coefficient&quot; rel=&quot;nofollow noreferrer&quot;&gt;second order finite differences&lt;/a&gt;, but you could pick other orders.&lt;/li&gt;&#xA;&lt;li&gt;The domain used in the codes is finite. That is one easy way of approaching the infinite domain, since outside that region the probability density is &lt;strong&gt;small enough&lt;/strong&gt;. A better approach is to use special boundary conditions that mimics the infinite region outside.&lt;/li&gt;&#xA;&lt;li&gt;Even for the finite domain, it would be a better choice to have small cell in the center and bigger ones in the outside. This is because most of the &quot;interesting&quot; things would happen in the center, where the potential is smaller and hence the probability density higher.&lt;/li&gt;&#xA;&lt;li&gt;I am not considering boundary conditions in this case, although they are somehow implicitly taken into account. In general, you should consider those, specially if your domain is finite.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Korsch, H. J., and M. Glück. &lt;a href=&quot;http://kluedo.ub.uni-kl.de/files/1171/no_series_145.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Computing quantum eigenvalues made easy.&quot;&lt;/a&gt; European journal of physics 23.4 (2002): 413.&lt;/li&gt;&#xA;&lt;li&gt;Garcia, Rafael, Alex Zozulya, and James Stickney. &lt;a href=&quot;https://arxiv.org/pdf/0704.1622v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;MATLAB codes for teaching quantum physics: Part 1.&quot;&lt;/a&gt; arXiv preprint arXiv:0704.1622 (2007).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-10-08T22:44:40.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25132" PostTypeId="1" AcceptedAnswerId="25145" CreationDate="2016-10-06T21:52:41.393" Score="0" ViewCount="119" Body="&lt;p&gt;I am using an open source FEM platform, which requires you to convert your equation system to non-dimensional form. So, there are no units specified for the parameters in the problem. If you use compatible input units, you should expect the output in matching unit - just like one would do in a normal python or C++ code for solving physical problems. I have an elasto-plastic dynamic problem to solve, and these are the parameters for the Drucker-Prager Soil Model for soft clay.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;E = 5000000;                 // Young's Modulus in N/m^2 or Pa&#xA;nu = 0.35;                   // Poisson's Ratio Unitless&#xA;cohesion = 20000.0;          // N/m^2 or Pa&#xA;dilatancy_angle = 25.0;      // Degrees&#xA;hardening_parameter = 0.1;   // Unitless&#xA;friction_angle = 25.0;       // Degrees&#xA;double rho = 1750.0;         // Mass density in kg/m^3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The mesh is a (1 x 1) square mesh. So when I see the displacement caused by wave propagating on this mesh using the above parameters, in Paraview, is it meaningful to assume that the output shall be in metres (m). Also, in such platforms, how does one compare the mesh to a realistic scenario? For example, can I assume that my mesh is a 1km x 1km square ground surface and the displacement I am viewing is in metre? What if I take a (10 x 10) mesh? Can anyone please explain how this non-dimensional logic works?&lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="16705" LastEditDate="2016-10-07T00:47:03.707" LastActivityDate="2016-10-07T14:09:58.537" Title="How to use non-dimensional form in open source codes instead of Units" Tags="|finite-element|nonlinear-equations|wave-propagation|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25134" PostTypeId="1" AcceptedAnswerId="25137" CreationDate="2016-10-06T23:03:26.487" Score="7" ViewCount="736" Body="&lt;p&gt;I am currently working on my bachelor’s diploma. The research concerns mixed finite element method for the 2D Stokes system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;- \Delta \boldsymbol u + \nabla p = \boldsymbol f, \quad \boldsymbol x \in \Omega \subset \mathbb R^2, \\&#xA;\nabla \cdot \boldsymbol u = 0, \quad \boldsymbol x \in \Omega,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with homogeneous Dirichlet (no–slip) BCs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\boldsymbol u = \boldsymbol 0, \quad \boldsymbol x \in \partial \Omega.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obtaining weak form and choosing appropriate spaces $\mathbb U_h \times \mathbb P_h \ni (\boldsymbol u_h, \, p_h)$ for the velocity field and pressure distribution, one reduces the problem to a saddle–point linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;	\begin{bmatrix}&#xA;		\mathbf A &amp;amp; \mathbf B^T \\&#xA;		\mathbf B &amp;amp; \mathbf 0           &#xA;	\end{bmatrix}&#xA;	\begin{bmatrix}&#xA;		\boldsymbol \xi \\&#xA;		\boldsymbol \psi         &#xA;	\end{bmatrix}&#xA;    =&#xA;	\begin{bmatrix}&#xA;		\boldsymbol b \\&#xA;		\boldsymbol 0         &#xA;	\end{bmatrix},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from which one then gets FE–solutions (using natural isomorphism $\mathcal P$ between vectors and FE–interpolants) for pressure $p = \mathcal P \, \boldsymbol \psi $ and ditto for velocity components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I focus on the analysis and implementation of (geometric) multigrid solving techniques for this system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an &lt;a href=&quot;https://github.com/CATSPDEs/papers/blob/master/FEM%20(and%20FDM)%20for%20Elliptic%20(Parabolic%2C%20Hyperbolic)%202D%20Problems/main.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;experience&lt;/a&gt; [Russian text] in solving elliptic and time–dependent parabolic and hyperbolic PDEs, yet I’ve never dealt with saddle–point problems before. So I want to clarify several aspects.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It is well–known that one should carefully choose FE–spaces in order to obtain a well–posed problem. I focus on two LBB–stable finite element pairs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i. $\Delta \, \boldsymbol P^1 \, CR — \Delta \, P^0 \, L$ finite element pair (non–conform space for the velocity components),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ii. $\Delta \, \boldsymbol P^2 \, L — \Delta \, P^1 \, L$ finite element pair (also known as Taylor–Hood).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me clarify these notations. I follow Ciarlet’s definition of FE here: $“\Delta”$ defines geometry (a triangle in my case), $“P^i”$ defines a polynomial space for shape functions, and the last component defines DOFs (which one uses to obtain shape functions and their cross–element behavior)—$“L”$ for “Lagrange” and $“CR”$ for “Crouzeix–Raviart.” I have some visualizations for (i) &lt;a href=&quot;https://github.com/CATSPDEs/CATSPDEs/tree/master/sln/%CE%94%20P1%20CR%20%E2%80%94%20%CE%94%20P0%20L%20Interpolant%20Visualization&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I concerned, inf–sup stability of these elements should guarantee existence of the unique solution of the above linear system. However, since the momentum equation involves only pressure gradient, it is determined up to a constant; so one usually requires $p$ to have zero mean value: $\int_\Omega p \, d \boldsymbol x = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is, &lt;strong&gt;should I enforce this constraint explicitly in the system&lt;/strong&gt;? Authors of [1, p. 309] suggest the following modification (they use (i)&lt;sup&gt;st&lt;/sup&gt; FE pair):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;	\begin{bmatrix}&#xA;		\mathbf A &amp;amp; \mathbf B^T &amp;amp; \boldsymbol 0 \\&#xA;		\mathbf B &amp;amp; \mathbf 0 &amp;amp; \boldsymbol a \\&#xA;        \boldsymbol 0 &amp;amp; \boldsymbol a^T &amp;amp; 0&#xA;	\end{bmatrix}&#xA;	\begin{bmatrix}&#xA;		\boldsymbol \xi \\&#xA;		\boldsymbol \psi \\&#xA;        \mu&#xA;	\end{bmatrix}&#xA;    =&#xA;	\begin{bmatrix}&#xA;		\boldsymbol b \\&#xA;		\boldsymbol 0 \\&#xA;         0&#xA;	\end{bmatrix}.&#xA;$$Other publications I’ve seen so far do not mention such modifications. Moreover, since I am interested in implementation of geometric multigrid, I have to assemble prolongation / restriction operators (sparse matrices); this is pretty straightforward for velocity components and pressure vectors (however, non–conform case will require some special handling), yet it is not for $\mu$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPD&lt;/strong&gt;: I noticed &lt;a href=&quot;https://scicomp.stackexchange.com/a/3295/21916&quot;&gt;this&lt;/a&gt; answer. It seems pretty reasonable that, since the “discrete version” of $\nabla p$ is $\mathbf B^T \boldsymbol \psi$, it should imply that operator $\mathbf B^T$ has a kernel consisting of the identity vector (I should check this by direct computations (for not–too–big matrices, of course) when I’m done with assembling routine).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also implies that the final system actually has an infinite number of solutions (&lt;strong&gt;doesn’t it?&lt;/strong&gt;). However, this should not cause any problems for a Krylov solver to converge, and one can enforce the desired constraints on the pressure at the post–processing step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now I got a bit confused. Does LBB–stability of a finite element pair really imply existence and &lt;em&gt;uniqueness&lt;/em&gt;? Isn’t it really about existence? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;According, for one, to [2, p. 99] it is reasonable to use multigrid as a preconditioner for a Krylov solver for div–grad elliptic problems. For the Stokes problem, &lt;strong&gt;is it reasonable to use multigrid (for one, with the Vanka smoother) as a preconditioner for MINRES&lt;/strong&gt;? MINRES seems reasonable since the system is symmetric and sign–indefinite. I’ve seen articles suggesting using multigrid as a stand–alone solver.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I also want to compare my future implementation with “black–box” Krylov solvers (for one, in terms of residuals’ history). And I do not want to implement them all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’ve already implemented data structures for CSC and symmetric CSlC sparse matrices (these data structures are used for $B$ and $A$ blocks of the system, respectively). I’ve also implemented import / export in Harwell–Boeing format for these data structures (HB–format is pretty popular and supported, for example, by Mathematica).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am wondering which tools I can use. It would be nice if the solver routine provided residuals’ history (not only the solution vector). &lt;strong&gt;Any suggestions&lt;/strong&gt;? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Mats G. Larson, Fredrik Bengzon&lt;br&gt;&#xA;&lt;em&gt;The Finite Element Method: Theory, Implementation, and Applications&lt;/em&gt;&lt;br&gt;&#xA;2013&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Maxim A. Olshanskii, Eugene E. Tyrtyshnikov&lt;br&gt;&#xA;&lt;em&gt;Iterative Methods for Linear Systems&lt;/em&gt;&lt;br&gt;&#xA;2014&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21916" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-04-14T10:05:28.477" Title="Mixed Finite Element Method for the Stokes System—Some Implementation Details" Tags="|finite-element|navier-stokes|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25135" PostTypeId="1" AcceptedAnswerId="25170" CreationDate="2016-10-06T23:17:36.107" Score="3" ViewCount="114" Body="&lt;p&gt;I'm looking to numerically evaluate $\log f_p(z)$ and its derivative $f^\prime_p(z)/f_p(z)$ accurately and efficiently in floating-point, where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_p(z)=\int_0^\infty r^{p-1} \exp\left(-\tfrac{1}{2} (r-z)^2\right)\,dr&#xA;\ .&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\phi(x)$ and $\Phi(x)$ be the PDF and CDF of the standard normal distribution $x \sim \mathcal{N}(0,1)$ respectively. Then the integral $f_p(z)$ can be expressed as the recurrence relation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;f_1(z) &amp;amp;= \sqrt{2\pi}\,\Phi(z) \\&#xA;f_2(z) &amp;amp;= \sqrt{2\pi}\,\phi(z) + z\,f_1(z) \\&#xA;f_p(z) &amp;amp;= (p-2)\,f_{p-2}(z) + z\,f_{p-1}(z)\ .&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I evaluate $\log f_p(z)$ and its derivative $f^\prime_p(z)/f_p(z)$ in floating point efficiently and accurately? The parameter $p$ can be assumed to be fixed such that extra pre-computation can be performed for each $p$. If needed, I can also restrict $z$ to an interval $z \in [c_1,c_2]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A direct evaluation by precomputing the polynomial coefficients of $\phi(z)$ and $\Phi(z)$ is feasible, but the coefficients involved quickly become unmanageably huge and would require extended-precision intermediate values even though the integrand and result are both well-behaved.&lt;/p&gt;&#xA;" OwnerUserId="11510" LastEditorUserId="414" LastEditDate="2016-10-11T17:50:06.793" LastActivityDate="2016-10-11T17:50:06.793" Title="Numerical evaluation of gaussian-like integral expressible as a recurrence relation" Tags="|numerical-analysis|quadrature|floating-point|special-functions|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25137" PostTypeId="2" ParentId="25134" CreationDate="2016-10-07T00:31:40.580" Score="5" Body="&lt;p&gt;The topic of linear solvers for the Stokes equation is pretty well discussed in the literature, and the common consensus is that (i) using GMRES as the outer solver, (ii) the Silvester-Wathen approach to forming a preconditioner for the outer solver, (iii) using multigrid as the inner solver or preconditioner for the elliptic  block is the way to go. This is not to say that there are not other methods that can &lt;em&gt;also&lt;/em&gt; be used, but this is what people consider state of the art.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own take at implementing this is given in &lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/step_22.html&quot; rel=&quot;noreferrer&quot;&gt;https://www.dealii.org/developer/doxygen/deal.II/step_22.html&lt;/a&gt; . The solver/preconditioner mentioned above is discussed in the &quot;Possibilities for extensions section&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for implementation: You are wasting your time if you try to implement &lt;em&gt;everything&lt;/em&gt; yourself. Use PETSc or Trilinos to represent your matrices, vectors, and linear solvers. Use any of the finite element libraries that are out there to represent the mesh and everything finite element related. For example, you can write a simple Stokes solver with not very much more than 100 or 150 lines of code in deal.II (a project I am heavily involved with, &lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;noreferrer&quot;&gt;see here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-07T00:31:40.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25138" PostTypeId="2" ParentId="25127" CreationDate="2016-10-07T00:39:00.237" Score="3" Body="&lt;p&gt;I learn computation science through Practical Numerical Methods with Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/numerical-mooc/numerical-mooc/wiki&quot; rel=&quot;nofollow&quot;&gt;https://github.com/numerical-mooc/numerical-mooc/wiki&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It covers finite differencing and many other numerical algorithms. For me the most interesting part is that it shows how various factors impact the stability, accuracy, and performance through small working examples.&lt;/p&gt;&#xA;" OwnerUserId="7258" LastActivityDate="2016-10-07T00:39:00.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25139" PostTypeId="2" ParentId="25125" CreationDate="2016-10-07T00:39:13.197" Score="6" Body="&lt;p&gt;Higher order methods often have a smaller radius of convergence, i.e., they require smaller time steps. In your context, this means that they require a smaller CFL number, often significantly smaller than 1. Did you take this into account?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the question of whether there are methods that can deal with CFL numbers much larger than one: This really doesn't make any sense. The CFL number is defined as the ratio of the time step divided by the time it takes information to cross one cell. If you wanted to have CFL $\gg$ 1, that would mean that within one time step, information travels far more than one cell. Because within this time interval, the signal may interact with boundaries or with itself, you can not expect that you can accurately resolve the solution if your CFL number is large. In other words, there are of course methods that remain &lt;em&gt;stable&lt;/em&gt;, but they will not be &lt;em&gt;accurate&lt;/em&gt; if the CFL number is much greater than one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put things differently: If you increase your wave speed by a factor of 10, then everything happens ten times as fast, and it only makes sense that you then have to choose a time step ten times smaller than before.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-07T00:39:13.197" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25140" PostTypeId="2" ParentId="25121" CreationDate="2016-10-07T03:20:26.400" Score="1" Body="&lt;p&gt;There are many ways to go about solving the compressible navier stokes equations.  One approach is to solve all of the equations (continuity, momentum, &amp;amp; energy) together as one system of coupled equations.  If your flow is inviscid, this is a fairly simple system of equations to solve.  But if you want to model viscous flow, your resulting system of non-linear equations will be extremely large and may not even fit into your machine's memory.  Furthermore, solving a system of non-linear equations presents its own set of challenges unto itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative approach to resolving viscous flow is to decouple the equations and solve each, in a one-at-a-time fashion over an iterative loop within each timestep.  This allows to you work more flexibly with limited computer memory constraints.  It also linearizes the advection term in the momentum equation (and the kinetic energy term in the energy equation) so that you don't need to solve a non-linear system for velocity or energy.  The PISO (pressure implicit splitting of operators) and SIMPLE (Semi-Implicit Method for Pressure Linked Equations) are both based upon this principle.   Though PISO &amp;amp; SIMPLE were originally formulated to solve the incompressible navier stokes equations, they can also be used to solve the compressible version as well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A hybrid between these two algorithms is sometimes referred to by the as the &quot;PIMPLE&quot; scheme (a portmanteau of piso and simple).  This hybrid algorithm works as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each timestep&#xA;    for a fixed number of iterations N1&#xA;        solve continuity equation (for density)&#xA;        solve momentum predictor equation (for velocity)&#xA;        solve energy equation (for energy)&#xA;        for some fixed number of iterations N2&#xA;             solve pressure equation&#xA;             update velocity&#xA;             update density (using the equation of state)&#xA;        end&#xA;        solve turbulence equations&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At any one given step, you use the most recently resolved quantities.  For example, after solving the continuity equation for density, you use this density solution to solve the momentum predictor equation for velocity, then use this velocity to solve the energy equation, etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will notice that this algorithm has both an outer loop (inspired by the SIMPLE algorithm) and an inner loop (inspired by the PISO algorithm).   This is how the compressible navier stokes equations are resolved in OpenFOAM's &lt;a href=&quot;https://github.com/OpenFOAM/OpenFOAM-2.2.x/blob/master/applications/solvers/compressible/rhoPimpleFoam/rhoPimpleFoam.C&quot; rel=&quot;nofollow&quot;&gt;rhoPimpleFoam&lt;/a&gt; solver.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accuracy depends on your mesh/timestep sizes and on how you discretize the operators in each equation.  For time derivatives, you can use a crank nicholson scheme to acheive second order accuracy in time.  Using centered difference discretization for laplacian operators (which appear in the momentum predictor, energy, and pressure equations) will result in second order accuracy.  However, the continuity, momentum predictor, and energy equations all contain 1st order divergence operators (think of them like first derivatives in space).  A centered difference on a divergence operator is unstable.  You might be able to achieve second order accuracy by using a second order upwind scheme for your divergence operators.  Depending on the complexity of your flow (e.g. shock waves and other high mach compressible flow effects), a second order scheme may not result in second order accuracy and/or may not even be stable.  The safest thing to do in such cases is use a first order upwind scheme (even though it lowers your overall accuracy, it generally leads to more stable time-stepping and/or shock capturing).&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2016-10-07T03:38:28.650" LastActivityDate="2016-10-07T03:38:28.650" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25141" PostTypeId="1" AcceptedAnswerId="25144" CreationDate="2016-10-07T07:49:47.877" Score="3" ViewCount="1434" Body="&lt;p&gt;I am solving Biot equation with sparse matrix in MATLAB. I have no problem with global sparse matrices assembly, but when I assign Dirichlet boundary condition, it is so slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this topic, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/20515/how-to-efficiently-implement-dirichlet-boundary-conditions-in-global-sparse-fini&quot;&gt;How to efficiently implement Dirichlet boundary conditions in global sparse finite element stiffnes matrices&lt;/a&gt;, @James also mentioned that it is so slow if global matrix is modified. My matrix is 25000x25000, and it took around 100s to assign boundary conditon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB, many authors state that, by using &lt;code&gt;setdiff&lt;/code&gt; function, only free nodes are considered when solving the equation. Only right hand vector is modified when applying dirichlet boundary condition and global matrix is kept unchanged.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hier is the code in MATLAB: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Dirichlet conditions &#xA;u = sparse(size(coordinates,1),1);&#xA;u(unique(dirichlet)) = u_d(coordinates(unique(dirichlet),:));&#xA;b = b - A * u;&#xA;&#xA;% Computation of the solution&#xA;u(FreeNodes) = A(FreeNodes,FreeNodes) \ b(FreeNodes);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the difference between two approaches? And is there any similar function as &lt;code&gt;setdiff&lt;/code&gt; in MATLAB in C++?&lt;/p&gt;&#xA;" OwnerUserId="21912" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-09-20T11:27:44.487" Title="Dirichlet boundary condition for sparse matrix - Solving Ax=b only for free nodes?" Tags="|finite-element|boundary-conditions|sparse-matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25143" PostTypeId="2" ParentId="25118" CreationDate="2016-10-07T08:10:12.900" Score="2" Body="&lt;p&gt;There is a approach that uses notions from &lt;em&gt;Discrete Geometry&lt;/em&gt;: Discrete Geometry is a discipline that works with objects defined as sets of pixels that try to mimic their standard counterparts. It defines discrete segments, discrete circles, discrete planes etc... In your case, there is an algorithm [1] that has a definition of what a discrete segment is, and that reconstructs in the input image the set of &lt;em&gt;maximal segments&lt;/em&gt;, i.e. segments that cannot be further extended by adding new pixels to them. See also the extensions to fuzzy data [2,3]. The approach was successfully applied to 3D reconstruction from multiview images [4]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] I. DEBLED-RENNESSON, J.-P. REVEILLES, A linear algorithm for segmentation of digital curves, International Journal of Pattern Recognition and Artificial Intelligence, Volume 9, N. 6, December 1995.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] I. DEBLED-RENNESSON, J.-L. REMY and J. ROUYER-DEGLI, Linear segmentation of discrete curves into fuzzy segments, Discrete Applied Mathematics, 151:122-137, October 2005.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] I. DEBLED-RENNESSON, F. FESCHET and J. ROUYER-DEGLI, Optimal blurred segments decomposition of noisy shapes in linear time, Computers &amp;amp; Graphics, 30(1), 2006.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;https://hal.inria.fr/inria-00349084&quot; rel=&quot;nofollow&quot;&gt;https://hal.inria.fr/inria-00349084&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-10-07T08:10:12.900" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25144" PostTypeId="2" ParentId="25141" CreationDate="2016-10-07T11:50:01.680" Score="3" Body="&lt;p&gt;I frequently solve simple FEM problems with 100 000 to 1 000 000 unknowns in Matlab and setting Dirichlet boundary conditions is blazing fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a code (although you cannot run it without the related functions but you get the idea):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% get mesh&#xA;mesh=squaremesh(9);&#xA;&#xA;% build stiffness matrix&#xA;K=bilin_assembly(@(u,v,ux,uy,vx,vy,x,y) ux.*vx+uy.*vy,mesh);&#xA;&#xA;% build load vector&#xA;f=lin_assembly(@(v,vx,vy,x,y) 1.*v,mesh);&#xA;&#xA;% number of nodes&#xA;n=size(mesh.p,2);&#xA;&#xA;% nodes on the dirichlet boundary&#xA;D=boundary_dofs(mesh);&#xA;&#xA;% free nodes are all nodes except dirichlet boundary nodes&#xA;I=setdiff(1:n,D);&#xA;&#xA;% initialize u as zero&#xA;u=zeros(n,1);&#xA;&#xA;% set dirichlet condition&#xA;X=mesh.p(1,D);&#xA;u(D)=sin(10*X);&#xA;&#xA;% solve the problem in the free nodes&#xA;u(I)=K(I,I)\(f(I)-K(I,D)*u(D));&#xA;&#xA;% plot the solution&#xA;figure(1);&#xA;trisurf(mesh.t',mesh.p(1,:),mesh.p(2,:),u);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The whole thing took me 3.5 seconds to run with n=263169. The result looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/J10ry.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/J10ry.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One cannot really see what's happening without removing the element edge lines:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OXyPQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OXyPQ.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u(I)=K(I,I)\(f(I)-K(I,D)*u(D));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;actually does is that it forms a smaller sparse matrix by including only the columns and rows defined in set I. In fact, running&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tic;K(I,I);toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;prints&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Elapsed time is 0.018481 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so this is a very fast operation indeed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that in C++ you store your sparse matrix in compressed row storage format. Then you can easily drop rows by looking at the contents of your row index array. Next perform a transpose and drop rows again. This way you end up with a smaller sparse matrix that you can use in your computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Forgot to mention, that in C++ standard library you have &lt;a href=&quot;http://en.cppreference.com/w/cpp/algorithm/set_difference&quot; rel=&quot;nofollow noreferrer&quot;&gt;set_difference&lt;/a&gt; as well.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastEditorUserId="10160" LastEditDate="2016-10-07T11:55:48.727" LastActivityDate="2016-10-07T11:55:48.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25145" PostTypeId="2" ParentId="25132" CreationDate="2016-10-07T13:53:02.950" Score="2" Body="&lt;p&gt;The answer is very simple: you provide the code with &lt;em&gt;geometric&lt;/em&gt; information, i.e. nodal coordinates (which in your case are expressed in metre), and not only topological information, i.e. how the mesh is laid out and connected to the nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So depending on the nodal coordinates you provide, you can have different domain &quot;sizes&quot;. E.g. if you have four nodes with coordinates $(0,0,0)$, $(100,0,0)$, $(100,100,0)$, $(0,100,0)$ then you are modelling a 100m square.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course all quantities (geometric, material properties, boundary conditions) shall be expressed in consistent units. As a general rule, if you use only SI units (kg, m, s, and their derivatives) you are on the &quot;safe&quot; side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A final comment about the term &quot;non-dimensional form&quot;. The way in which I like to understand what is going on is that I'm feeding the FE code with the numerical values of the physical quantities in a given consistent system of units, say SI. The output of the code will be the numerical values of the computed physical quantities in the same system of units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A true non-dimensional form is a more general concept, in which you rewrite the equations so that the variables are pure numbers and not physical quantities.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2016-10-07T14:09:58.537" LastActivityDate="2016-10-07T14:09:58.537" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25146" PostTypeId="1" CreationDate="2016-10-07T20:48:34.510" Score="0" ViewCount="863" Body="&lt;p&gt;Consider the equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_0^L \mathbf W(\mathbf u, s) \, \mathrm ds = \mathbf 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $0 \leq s \leq L$ and $\mathbf u$ is a vector of constants. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically, what is the best way to determine $\mathbf u$ that satisfy the equations? &lt;/p&gt;&#xA;" OwnerUserId="17368" LastEditorUserId="17368" LastEditDate="2016-10-07T22:53:42.363" LastActivityDate="2016-10-09T02:16:31.307" Title="Solve integral equation for unknown constant" Tags="|integral-equations|constraints|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25147" PostTypeId="2" ParentId="24636" CreationDate="2016-10-08T19:59:38.703" Score="0" Body="&lt;p&gt;I have never done this myself, I do mostly MD. Still, your energy is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E = \frac{1}{2} \sum_i e_i = \frac{1}{2} \sum_i \sum_{j\ne i} V_{ij}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the second sum is performed on neighbors only. If you store for every particle $i$ the sum $e_i = \sum_{j\ne i} V_{ij}$&#xA;you can just recompute it for the MC update. Explicitly, you compute the difference in energy for the particle that has moved. This idea extends to particle removal and addition easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The average cost is $O(N_\textrm{neigh})$ where $N_\textrm{neigh}$ is the average number of neighbors for a particle.&lt;/p&gt;&#xA;" OwnerUserId="16340" LastActivityDate="2016-10-08T19:59:38.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25149" PostTypeId="2" ParentId="25146" CreationDate="2016-10-09T02:16:31.307" Score="2" Body="&lt;p&gt;Like what @Kirill says, write a script that defines $f(u)$ to be a function that approximates the integral. Thus all that you need to is solve $f(u)=0$ which can use standard root finding tools like Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're best choice for this is probably Julia. Using the built in &lt;code&gt;quadgk&lt;/code&gt; function for Gauss-Kronrad quadrature, you can define the &lt;code&gt;f&lt;/code&gt; function via:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(u) = quadgk((x)-&amp;gt;W(u,x),0,L)[1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(the &lt;code&gt;[1]&lt;/code&gt; is because &lt;code&gt;quadgk&lt;/code&gt; returns a tuple where the first part is the integral and the second part is the error estimate, so this function ignores the error estimate and just returns the integral approximation). Note that &lt;code&gt;(x)-&amp;gt;W(u,x)&lt;/code&gt; is using what's known as a closure: it makes an anonymous function &lt;code&gt;g(x) = W(u,x)&lt;/code&gt; where it sticks in the value of &lt;code&gt;u&lt;/code&gt;, allowing you to easily integrate among the &lt;code&gt;x&lt;/code&gt;. Since Julia v0.5 compiles the anonymous functions as generic functions, this will be very fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you just have to solve the root finding problem. I particularly like the library NLSolve. Since this function is not in place, we would use the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using NLsolve&#xA;nlsolve(not_in_place(f),initial_u)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;initial_x&lt;/code&gt; is a guess for the final value of &lt;code&gt;u&lt;/code&gt;. This will use finite differencing. We can many times make it faster and more accurate by enabling auto-differentiation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nlsolve(not_in_place(f),initial_u,autodiff = true)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-10-09T02:16:31.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25150" PostTypeId="2" ParentId="25112" CreationDate="2016-10-09T02:26:50.073" Score="-2" Body="&lt;p&gt;This is definitely a stiffness issue. Instead of using RK4, what about using a reasonably well-optimized adaptive timestepping method? In many cases that will work well enough. If not, the places to go are Rosenbrock methods are BDF methods. You don't need to isolate anything to use BDF methods: you know the second time derivatives (since you know the first derivatives), so you can write down the vector function for how everything changes and just plop that into  an ODE solver (with all 4 equations). The ODE solver will solve the root finding problem for you.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-10-09T02:26:50.073" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25151" PostTypeId="2" ParentId="25110" CreationDate="2016-10-09T02:38:15.290" Score="21" Body="&lt;p&gt;Here's the deal with GPUs. On a GPU, every single core is slow. Really slow. However, you have thousands of cores. If you can effectively use the thousands of cores at a time, then your algorithm will run better on the GPU. If you cannot, then it will run much slower on the GPU. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linear algebra is one domain where parallelism is really well established. Thus the best way to write for a GPU is to essentially have the GPU do all of the linear algebra: it essentially becomes a card to compute &lt;code&gt;Ax=b&lt;/code&gt; and &lt;code&gt;A*B&lt;/code&gt; much faster than the CPU (this fact is pretty easy to check, look at the numerous benchmarks or even just open up MATLAB and type in &lt;code&gt;A*B&lt;/code&gt; for both matrices and GPU matrices). But there's a caveat: data transfer to GPUs is really slow. Also, memory allocation on GPUs is really slow. So while the linear algebra is fast, you have to deal with the fact that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Serial performance is awful.&lt;/li&gt;&#xA;&lt;li&gt;Allocating memory dynamically on the GPU destroys performance.&lt;/li&gt;&#xA;&lt;li&gt;Transferring back and forth between the CPU and GPU is slow.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This puts constraints on your algorithm: you need to try to leave as much on the GPU as possible, transferring back and forth the minimum amount, while trying to avoid serial parts from running on the GPU. Likewise, the GPU can easily do the linear algebra 1000x faster than the CPU (which is usually the performance bottleneck) so in many cases you can effectively manage this dilemma and end up with large performance gains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting alternative are the Xeon Phi. These cards have much faster data transfer, much better serial performance, and can allocate memory much better. However, the tradeoff is it's less specialized to be a &quot;dumb linear algebra solver&quot; and you thus have to pay a heftier price, and in return its linear algebra performance is about half that of a GPU. However, this can be much easier to develop code for (OpenMP parallel codes will use it automatically, and you can use a Xeon Phi card as another node via MPI, so if you've already parallelized a code you can use the same code with the Phi) and, by allowing you to more effectively keep data on the acclerator or using the increased data transfer speeds, can be much faster than a GPU in real-world (S)PDE solving. Of course, it depends greatly on the implementation.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-10-09T02:38:15.290" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25152" PostTypeId="1" AcceptedAnswerId="25154" CreationDate="2016-10-09T04:30:08.453" Score="2" ViewCount="4332" Body="&lt;p&gt;I have two large-ish matrices (~100K cols x ~100K rows).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are sparse and symmetrical (about 0.1% of them values are non-zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to do element-wise multiplication between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I plan to perform this operation about 1,000,000 times, so speed is is definitely going to be an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of a good library/software that could help me accomplish this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I've stumbled unto SuiteSparse. If anyone knows more about it, do you think I could use it for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best.&lt;/p&gt;&#xA;" OwnerUserId="21931" LastEditorUserId="21931" LastEditDate="2016-10-09T08:18:19.560" LastActivityDate="2016-10-10T02:27:54.107" Title="Fastest way to perform element-wise multiplication on a sparse matrix" Tags="|linear-algebra|matrix|lapack|blas|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25153" PostTypeId="2" ParentId="25134" CreationDate="2016-10-09T13:24:50.753" Score="1" Body="&lt;p&gt;I happen to work on the same mixed finite element method as well, but in a different context. (So I don't know if my answer can fully apply to your problem)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my work I'm enforcing the compability constraint directly. I've seen the extended system of equations you have posted numerous times. In solid mechanics, it will be used as a kind of stabilization, but is not really needed, at least to my experience. A big downfall (of mixed methods in general) which might be able to overcome with the extension of the system of equations is that the mixed stiffnessmatrix (or however you want to call it...) is ill-conditioned. It resulted in a poor convergence in my case. &lt;/p&gt;&#xA;" OwnerUserId="21418" LastActivityDate="2016-10-09T13:24:50.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25154" PostTypeId="2" ParentId="25152" CreationDate="2016-10-09T13:34:24.793" Score="2" Body="&lt;p&gt;I suggest you take a look at the Eigen C++ matrix class library,&#xA;&lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;http://eigen.tuxfamily.org/index.php?title=Main_Page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a small test with sparse matrices of the size and sparsity you state and&#xA;it takes about 1 ms per matrix multiplication on my moderate-power windows&#xA;machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code for my experiment is below. As you can see, most of the code is for&#xA;setting up the test matrices. The actual matrix multiply is a simple one-liner.&#xA;(I multiplied the matrices 500 times to make the timing more reliable.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;using std::cout;&#xA;using std::endl;&#xA;#include &amp;lt;random&amp;gt;&#xA;&#xA;#include &amp;lt;Eigen/Core&amp;gt;&#xA;#include &amp;lt;Eigen/Sparse&amp;gt;&#xA;&#xA;#include &amp;lt;boost/timer.hpp&amp;gt;&#xA;&#xA;int main()&#xA;{&#xA;  const int n = 100000;&#xA;  const int nnz = static_cast&amp;lt;int&amp;gt;(.001*n);&#xA;  typedef Eigen::Triplet&amp;lt;double&amp;gt; Triplet;&#xA;  std::vector&amp;lt;Triplet&amp;gt; at(nnz), bt(nnz);&#xA;  std::random_device rd; // obtain a random number from hardware&#xA;  std::mt19937 eng(rd()); // seed the generator&#xA;  std::uniform_int_distribution&amp;lt;&amp;gt; distr(0, nnz-1); // define the range&#xA;  for (int i = 0; i &amp;lt; nnz; i++) {&#xA;    int ii = distr(eng), jj = distr(eng);&#xA;    at[i] = Triplet(ii, jj, 1);&#xA;    bt[i] = Triplet(jj, ii, 3);&#xA;  }&#xA;  Eigen::SparseMatrix&amp;lt;double&amp;gt; as(n, n), bs(n, n), cs(n, n);&#xA;  as.setFromTriplets(at.begin(), at.end());&#xA;  bs.setFromTriplets(bt.begin(), bt.end());&#xA;  boost::timer timer;&#xA;  const int nt = 500;&#xA;  for (int i = 0; i &amp;lt; nt; i++)&#xA;     cs = as.cwiseProduct(bs);&#xA;  cout &amp;lt;&amp;lt; &quot;elapsed time=&quot; &amp;lt;&amp;lt; timer.elapsed()/(double) nt  &amp;lt;&amp;lt; endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-10-09T13:34:24.793" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25157" PostTypeId="1" CreationDate="2016-10-09T18:15:18.893" Score="1" ViewCount="345" Body="&lt;p&gt;I'm kind new in the Computed Tomography field and I'm trying to understand and implement ART technique. Said it, I started to read the book &lt;em&gt;The Mathematics of Medical Imaging - A Beginners Guide&lt;/em&gt; by Timothy G. Feeman. In the chapter 9, author describes the ART method. Well, If I understood fine (correct me If I'm wrong, please), in ART we are just trying to solve this system: &lt;strong&gt;Ax=p&lt;/strong&gt;. One approach to solve it is to use Kaczmarz method (I read it and it is ok to understand it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's understand each variable in the equation above: x is the image matrix that I want to found. A is the projection matrix and p is the sinogram. Is this reasoning right? If yes, how I obtain A from p? Can anyone show me an example with some numbers? (Ex: let A = [...], p = [....], etc.) A very simple example for the sake of simplicity would be fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance, guys!&lt;/p&gt;&#xA;" OwnerUserId="21935" LastActivityDate="2016-10-09T18:15:18.893" Title="How to obtain projections from sinogram in ART reconstruction technique?" Tags="|linear-algebra|python|image-processing|projection|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25158" PostTypeId="1" CreationDate="2016-10-09T20:40:49.793" Score="2" ViewCount="934" Body="&lt;p&gt;I have an objective function that I can write either in quadratic programming (QP) such as &#xA;$$\sum_{i=1}^N \sum_{j=1}^N C_{ij}^2$$&#xA;or as an LP problem&#xA;$$\sum_{i=1}^N \sum_{j=1}^N |C_{ij}|$$&#xA;which can be done by splitting $C_{ij}$ into two parts $C_{ij} = C_{ij}^+ + C_{ij}^-$ and imposing $C_{ij}^{+}&amp;gt;0$ and $C_{ij}^{-}&amp;gt;0$. This doubles the number of optimisation variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both approaches will have additional linear constraints which will be the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is the following: Is it possible to argue that the LP algorithm will always be faster than the QP algorithm given that we use the state of the art algorithms for each ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I realise that the solutions will not be the same. But assuming that I want to compare both approaches, I simply want to know which one will in general be faster. In practice I find LP to be faster. But I would like to know if this is a hard rule or does it simply come down to which algorithm I am using and the nature of the problem and constraints.&lt;/p&gt;&#xA;" OwnerUserId="7997" LastEditorUserId="7997" LastEditDate="2016-10-10T19:08:21.637" LastActivityDate="2016-10-10T19:08:21.637" Title="Are linear programming algorithms faster than quadratic programming algorithms?" Tags="|optimization|convex-optimization|constrained-optimization|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="25160" PostTypeId="1" CreationDate="2016-10-09T22:28:28.227" Score="1" ViewCount="1454" Body="&lt;p&gt;I want to simulate a molecular system at a certain temperature. It would be best if it could be implemented by NVT molecular dynamics. However, if one were to use a NVE simulation (like velocity Verlet), how could I fix the temperature, or let the temperature only fluctuate a little bit around the desired value?&lt;/p&gt;&#xA;" OwnerUserId="21938" LastActivityDate="2016-10-10T20:19:35.667" Title="How to control temperature in NVE molecular dynamics" Tags="|molecular-dynamics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25161" PostTypeId="2" ParentId="25152" CreationDate="2016-10-10T02:27:54.107" Score="2" Body="&lt;p&gt;If there is a choice in programming language, one option can be to use &lt;a href=&quot;http://julialang.org&quot; rel=&quot;nofollow&quot;&gt;Julia&lt;/a&gt;, which has built in support for sparse matrices (via Suitsparse). The timing come out to about one and a half milliseconds on my laptop, and you get to use an interactive, dynamic language, which may be useful in certain situations. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; a=sprand(1000, 1000, 0.1)&#xA;1000x1000 sparse matrix with 99749 Float64 entries:&#xA;    [5   ,    1]  =  0.725824&#xA;    [17  ,    1]  =  0.420022&#xA;    [19  ,    1]  =  0.404282&#xA;    [21  ,    1]  =  0.0307138&#xA;    [52  ,    1]  =  0.453376&#xA;    [55  ,    1]  =  0.30054&#xA;    [69  ,    1]  =  0.360203&#xA;    [74  ,    1]  =  0.346881&#xA;    [94  ,    1]  =  0.312849&#xA;    ⋮&#xA;    [932 , 1000]  =  0.978966&#xA;    [933 , 1000]  =  0.149551&#xA;    [954 , 1000]  =  0.417852&#xA;    [959 , 1000]  =  0.722707&#xA;    [964 , 1000]  =  0.519931&#xA;    [967 , 1000]  =  0.567152&#xA;    [971 , 1000]  =  0.964192&#xA;    [979 , 1000]  =  0.88494&#xA;    [987 , 1000]  =  0.286723&#xA;    [988 , 1000]  =  0.24282&#xA;&#xA;julia&amp;gt; b=sprand(1000, 1000, 0.1)&#xA;1000x1000 sparse matrix with 99998 Float64 entries:&#xA;    [1   ,    1]  =  0.920533&#xA;    [3   ,    1]  =  0.879179&#xA;    [7   ,    1]  =  0.267203&#xA;    [25  ,    1]  =  0.522407&#xA;    [34  ,    1]  =  0.656031&#xA;    [41  ,    1]  =  0.280885&#xA;    [44  ,    1]  =  0.735824&#xA;    [68  ,    1]  =  0.433098&#xA;    [69  ,    1]  =  0.124862&#xA;    ⋮&#xA;    [932 , 1000]  =  0.505959&#xA;    [939 , 1000]  =  0.983413&#xA;    [947 , 1000]  =  0.418157&#xA;    [949 , 1000]  =  0.884657&#xA;    [963 , 1000]  =  0.412645&#xA;    [964 , 1000]  =  0.544348&#xA;    [966 , 1000]  =  0.709398&#xA;    [983 , 1000]  =  0.260483&#xA;    [989 , 1000]  =  0.1218&#xA;    [1000, 1000]  =  0.468975&#xA;&#xA;julia&amp;gt; a.*b&#xA;1000x1000 sparse matrix with 9876 Float64 entries:&#xA;    [69  ,    1]  =  0.0449757&#xA;    [102 ,    1]  =  0.0340867&#xA;    [137 ,    1]  =  0.0794594&#xA;    [247 ,    1]  =  0.108002&#xA;    [376 ,    1]  =  0.248346&#xA;    [609 ,    1]  =  0.241789&#xA;    [633 ,    1]  =  0.224115&#xA;    [658 ,    1]  =  0.379804&#xA;    [754 ,    1]  =  0.272618&#xA;    ⋮&#xA;    [224 , 1000]  =  0.0122434&#xA;    [301 , 1000]  =  0.163899&#xA;    [309 , 1000]  =  0.0972784&#xA;    [403 , 1000]  =  0.0245688&#xA;    [659 , 1000]  =  0.0801249&#xA;    [700 , 1000]  =  0.158823&#xA;    [760 , 1000]  =  0.388442&#xA;    [926 , 1000]  =  0.193808&#xA;    [932 , 1000]  =  0.495317&#xA;    [964 , 1000]  =  0.283024&#xA;&#xA;julia&amp;gt; @time a.*b&#xA;  0.001649 seconds (25 allocations: 3.056 MB)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1119" LastActivityDate="2016-10-10T02:27:54.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25162" PostTypeId="1" AcceptedAnswerId="25165" CreationDate="2016-10-10T09:53:06.413" Score="7" ViewCount="943" Body="&lt;p&gt;I'm comparing some of the different ODE integrators in &lt;code&gt;scipy.integrate.ode&lt;/code&gt; on solving the logistic function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x(t) = \frac{1}{1+e^{-rt}}$$&#xA;$$\dot{x} = rx(1-x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard that LSODA should be very good, so I was a bit surprised to find that it fails completely for $r&amp;gt;4$. The &lt;code&gt;dopri5&lt;/code&gt; integrator, on the other hand, seems to have no problem for very large values of $r$ (see plots below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does LSODA fail here? Do I need to tune some parameters, or is it simply a poor choice for this particular problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy import integrate&#xA;&#xA;# Logistic function&#xA;r = 5&#xA;def func(t):&#xA;    return 1. / (1 + np.exp(-r * t))&#xA;&#xA;def dfunc(t, X):&#xA;    x, = X&#xA;    return [r * x * (1 - x)]&#xA;&#xA;t0 = -10&#xA;t1 = 10&#xA;dt = 0.01&#xA;X0 = func(t0)&#xA;&#xA;integrator = 'lsoda'&#xA;&#xA;t = [t0]&#xA;Xi = [func(t0)]&#xA;ode = integrate.ode(dfunc)&#xA;ode.set_integrator(integrator)&#xA;ode.set_initial_value(X0, t0)&#xA;&#xA;while ode.successful() and ode.t &amp;lt; t1:&#xA;    t.append(ode.t + dt)&#xA;    Xi.append(ode.integrate(ode.t + dt))&#xA;&#xA;t = np.array(t)     # Time&#xA;X = func(t)         # Solution&#xA;Xi = np.array(Xi)   # Numerical&#xA;&#xA;# Plot analytical and numerical solutions&#xA;X = func(t)&#xA;&#xA;plt.subplot(211)&#xA;plt.title(&quot;Solution&quot;)&#xA;plt.plot(t, X, label=&quot;analytical&quot;, color='g')&#xA;plt.plot(t, Xi, label=integrator)&#xA;plt.xlabel(&quot;t&quot;)&#xA;plt.ylabel(&quot;x&quot;)&#xA;plt.legend(loc='upper left', bbox_to_anchor=(1.0, 1.05))&#xA;&#xA;# Plot numerical integration errors&#xA;err = np.abs(X - Xi.flatten())&#xA;print(&quot;{}: mean error: {}&quot;.format(integrator, np.mean(err)))&#xA;&#xA;plt.subplot(212)&#xA;plt.title(&quot;Error&quot;)&#xA;plt.plot(t, err, label=integrator)&#xA;plt.xlabel(&quot;t&quot;)&#xA;plt.ylabel(&quot;error&quot;)&#xA;plt.legend(loc='upper left', bbox_to_anchor=(1.0, 1.05))&#xA;&#xA;plt.tight_layout()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result for &lt;code&gt;integrator = 'lsoda'&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;lsoda: mean error: 0.500249750249742&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZpClh.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZpClh.png&quot; alt=&quot;result with lsoda&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Result for &lt;code&gt;integrator = 'dopri5'&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;dopri5: mean error: 3.7564128626655536e-11&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZAtEq.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZAtEq.png&quot; alt=&quot;result with dopri5&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21942" LastActivityDate="2016-10-11T19:54:51.810" Title="Why does LSODA fail to integrate the logistic function?" Tags="|python|ode|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25163" PostTypeId="2" ParentId="25160" CreationDate="2016-10-10T13:59:51.657" Score="0" Body="&lt;p&gt;It depends on what you mean by $NVE$; if you want to do dynamics in a constant energy ensemble, and you consider the total system energy to be composed of the sum of kinetic and potential energies of your system, one can contemplate three ensembles: $NVE_{tot}$, $NVE_{kin}$ and $NVE_{pot}$, where the total, kinetic and potential energies are conserved, respectively, for all time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Temperature, in the MD sense, corresponds to the instantaneous kinetic energy of a system. However, there are no constraints placed on the kinetic energy of the system in the $NVE_{tot}$ ensemble (other than summing with the potential energy gives a constant value for all times).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, the isokinetic ensemble, $NVE_{kin}$, requires that the kinetic energy, and therefore the instantaneous temperature, remains constant for all time, which appears to be what you want. Tuckerman discusses the various schemes one can use in his book quite comprehensively. &lt;/p&gt;&#xA;" OwnerUserId="21943" LastEditorUserId="21943" LastEditDate="2016-10-10T14:21:22.377" LastActivityDate="2016-10-10T14:21:22.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25164" PostTypeId="1" CreationDate="2016-10-10T14:57:29.050" Score="1" ViewCount="502" Body="&lt;p&gt;I have two identical computers, both with 4 dual cores and MATLAB installed on them. They are connected to the same network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anyway I can run a parforloop using the computation power of both machines without relying on MATLAB distributed computing server? I do not have a license for this, and seems to expensive to connect just two computers.&lt;/p&gt;&#xA;" OwnerUserId="21861" LastActivityDate="2018-11-20T21:15:22.937" Title="Pool two computers for MATLAB parfor without MDCS" Tags="|matlab|hpc|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25165" PostTypeId="2" ParentId="25162" CreationDate="2016-10-10T19:16:13.513" Score="8" Body="&lt;p&gt;When you use $r=5$, the initial condition is&#xA;$$ x(-10) \approx e^{-50} \approx 1.9\times 10^{-22}. $$&#xA;This is much smaller than the machine epsilon, $2\times 10^{-16}$, and it is very likely that LSODA just concludes that the solution is identically zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check this idea, I replaced $t_0$ with $-5$ instead of $-10$, so that $x(-5)\approx 1.4\times 10^{-11}$, and then it seems to work. The solution isn't very accurate, the error rises to $0.01$, but this is likely because the numerical error places the location of the bump in slightly the wrong place, while the shape of the solution is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way LSODA seems to decide that the solution is identically zero is that it uses its absolute tolerance, and presumably $x(-10)$ is smaller than that, causing the problem. So to fix that, instead of changing $t_0$, one can also replace the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ode.set_integrator(integrator)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ode.set_integrator(integrator, atol=0.0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then it also seems to work, but the error is $10^{-5}$. To get the error down to the same level as DOPRI5, I can set &lt;code&gt;rtol=1e-10&lt;/code&gt;, but I'm not quite sure why DOPRI5 manages with its default parameters, or whether those defaults are different. In any case, I think it's the default tolerances at work here.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-10-11T19:54:51.810" LastActivityDate="2016-10-11T19:54:51.810" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25166" PostTypeId="2" ParentId="25160" CreationDate="2016-10-10T20:19:35.667" Score="1" Body="&lt;p&gt;In my quite limited knowledge, there is no straightforward way to simulate a system with $NVE$-ensemble and get a (nearly) constant temperature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could approach this problem by starting with an $NVT$-ensemble. Set up the system, run energy minimization, generate velocities and use the velocity rescaling method until you are close to the wanted temperature. Once you are close to $T_{wanted}$, switch to a more gentle method, like Berendsen weak coupling or use a thermostat, such as Nose-Hoover.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the system has thermalized, you can switch to $NVE$ and run the production simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have the privilege to post any links, but a simple google search with the keywords 'NVE MD simulation temperature control' will provide a good amount of references.&lt;/p&gt;&#xA;" OwnerUserId="19186" LastActivityDate="2016-10-10T20:19:35.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25167" PostTypeId="1" CreationDate="2016-10-10T20:36:44.297" Score="0" ViewCount="60" Body="&lt;p&gt;I am preparing to do a MD-simulation of a system which consists of two phases. The gaseous phase consists of $H_{2}$, $Ar$, $He$, $N_{2}$, $CH_{4}$ and $CO_{2}$. In this system, $p=0.355 ~bar$ and $T\approx 350 K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In direct contact with this phase is an aqueous phase, consisting of the same components and water.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've spent some time looking in to suitable forcefields for this system, but to avoid making amateurish mistakes, I decided to ask this community for advice. Are you aware of any forcefield which would be suitable for this kind of simulation? I'm currently looking into Dreiding FF, and considering ReaxFF (although this is not the first choice, since it needs &lt;em&gt;ab initio&lt;/em&gt; calculations to support it, and I don't really need data about reactions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using DL_POLY Classic, which has no forcefield of its own.&lt;/p&gt;&#xA;" OwnerUserId="19186" LastActivityDate="2017-06-27T10:20:44.790" Title="Forcefield for studying a two phase system, consisting of noble gases and gaseous organic components" Tags="|molecular-dynamics|computational-chemistry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25169" PostTypeId="1" CreationDate="2016-10-11T11:03:57.950" Score="2" ViewCount="2151" Body="&lt;p&gt;I need to build a new desktop PC, where ab-initio DFT calculation going to be performed. I am searching for a CPU in value range 600 - 1000€. I was thinking about six-core Intel Core i7-6850K or 8-core Intel Core i7-6900K. However Intel Core ix processors are desktop processors so they are optimized for multimedia application and games, I think there are some features I don't need. In case of DFT calculation the most important stuff are matrix-matrix multiplication and FFT. So I was thinking also about Intel Xeon processors. In the same price I can buy 8-12 - core CPU, but with lower frequency (Intel Xeon Processor E5-2630 v4). Now I dont know what to buy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot find any DFT CPU benchmarks on the web. Can you help me please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make it more clear: In value range 600 - 1000€ is better to buy intel i7 or intel xeon for DFT calculations. I am aware of GPU computing and that there are also server solutions, but I am not asking about that.&lt;/p&gt;&#xA;" OwnerUserId="21957" LastEditorUserId="21957" LastEditDate="2016-10-13T09:14:07.260" LastActivityDate="2017-01-03T19:00:11.800" Title="CPU for ab-initio DFT calculations" Tags="|performance|density-functional-theory|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25170" PostTypeId="2" ParentId="25135" CreationDate="2016-10-11T13:39:24.397" Score="5" Body="&lt;h2&gt;Step 1: analyze the recurrence&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I implemented the recurrence in Python, using basic numpy and scipy.special for the &lt;code&gt;erf&lt;/code&gt; function. Why? Because it is simple and cheap (for reasonable $p$). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.special as sp&#xA;def F1(z):&#xA;    return np.sqrt(np.pi/2)*(1.0+sp.erf(z/np.sqrt(2)))&#xA;&#xA;def F2(z):&#xA;    return F1(z)*z + np.exp(-z**2/2.0)&#xA;&#xA;def F(p,z):&#xA;    if (p==1):&#xA;        return F1(z)&#xA;    elif (p==2):&#xA;        return F2(z)&#xA;    else:&#xA;        fpm2 = F1(z)&#xA;        fpm1 = F2(z)&#xA;&#xA;        for pi in range(3,p+1):&#xA;            fp = (pi-2)*fpm2 + z* fpm1&#xA;            fpm2 = fpm1&#xA;            fpm1 = fp&#xA;        return fp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I compared results from this to fixed precision results using Maple 2016 (Digits set to 500). The following graph shows the relative error between the recurrence results and Maple results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LPUiX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LPUiX.png&quot; alt=&quot;Simple implementation of recurrence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can clearly see that the recurrence is forward-stable (stable in increasing $p$) for positive $z$. For negative $z$, things are different. For $z&amp;lt;0$, $f_p$ as a function of the parameter $p$ for a fixed $z$ has a bump (it is not monotonous). See next graph.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CZBQi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CZBQi.png&quot; alt=&quot;Behaviour of $f_p$ as a function of the parameter $p$ for fixed $z=-4$&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotting the relative error gives:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mDl4z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mDl4z.png&quot; alt=&quot;Relative error for $f_p$ using the recurrence for fixed $z=-4$&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One observes a loss of accuracy for increasing $p$. For $p$ above 27, you can see that the log plot of $f_p$ does not continue, simply because the recurrence gives negative values. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Step 2: can we do black-box numerical integration?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Again, for $z&amp;gt;0$, this would be meaningless since the recurrence works so well. We could try it for negative values of $z$. So let's have a look at the integrand:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MmTOm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MmTOm.png&quot; alt=&quot;Integrand defining $f_p$ for $p=11$&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Positive point: the integrand decreases very rapidly (after a peak). So one could approximate the infinite range integration to a finite range based on the value of the integrand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problematic point: the function is enormously peaked. Let's see if QUADPACK (used as back-end in &lt;code&gt;scipy.integrate.quad&lt;/code&gt;) can handle this. Note that I put the right point of integration arbitrarily at $r=200$. This could be optimized (if need be). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.integrate import quad&#xA;def F_neg(p,z):&#xA;    Qres = np.zeros_like(z)&#xA;    for i,zz in enumerate(z):&#xA;        Q,err = quad(integrand,0,200,args=(float(p),float(zz)),epsrel=1e-12,epsabs=1e-12)&#xA;        Qres[i] = Q&#xA;    return Qres&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, a plot of the relative error comparing this Python code to Maple results: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/TQRtg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TQRtg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perfect!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Preliminary conclusion: use the recurrence for $z&amp;gt;0$, use the numerical quadrature for $z&amp;lt;0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think that using closed form solutions using hypergeometric functions (or LaguerreL functions) will help much since you will have linear combinations of these, possibly leading to cancellation problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Work to be done: check range of $z$ and $p$ for which the above holds. But that is for the poster of the question...&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2016-10-11T13:39:24.397" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25171" PostTypeId="1" CreationDate="2016-10-11T14:26:01.863" Score="1" ViewCount="214" Body="&lt;p&gt;I am looking for an open source solver to solve the a quadratic programming problem with an additional piecewise linear objective, as show below. The problem is fairly small ($\mathbf{x}$ is a vector of dimension 120).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA; \max_{x}: \mathbf{x}^{T} \mathbf{q_0}&#xA; - \mathbf{x^{T}} P_0 \mathbf{x} - f(\mathbf{x})&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that above $P_0$ is positive semidefinite and $f$ is piecewise linear for each variable in $\mathbf{x}$ but not convex. For each $x_i$, $i \in \left\{1, ..., 120\right\}$, $f(x)$ would be a vector of coordinate points, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[(x_0, y_0), (x_1, y_1), ..., (x_N, y_N)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For any given $x_i$, $f(x_i)$ looks something like the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I7Z7K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I7Z7K.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $f$ does not depend on any cross terms in $\mathbf{x}$, the above optimization could also be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA; \max_{x}: \mathbf{x}^{T} \mathbf{q_0}&#xA; - \mathbf{x^{T}} P_0 \mathbf{x} - (f(x_1) + ... + f(x_{120}))&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The are also linear constraints of the form $A\mathbf{x} &amp;lt; \mathbf{b}$ are added.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have formulated the problem in Gurobi which is fairly straightforward but was hoping to compare this to an open source solver. Looking around it seems like GLPK would work for this, but I have no experience using this library so was hoping for a bit of info related to this or alternative solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sample problem for gurobi using the matlab API is shown below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 2;&#xA;&#xA;q_0 = [-3.31e3, -5.07e3];&#xA;P_0 = [-0.90e-04  -0.63e-04;&#xA;       -0.63e-04  -0.90e-04];&#xA;&#xA;x = [-2.0000e9, -1.0000e9, -0.8000e9, -0.6000e9, -0.4000e9, -0.2500e9,...&#xA;    -0.1000e9, -0.0500e9, -0.0250e9, -0.0100e9, -0.0010e9, 0, 0.0010e9,...&#xA;    0.0100e9, 0.0250e9, 0.0500e9, 0.1000e9, 0.2500e9, 0.4000e9,...&#xA;    0.6000e9, 0.8000e9, 1.0000e9, 2.0000e9];&#xA;&#xA;y = [2.938964e6, 0.753364e6, 0.488104e6, 0.280144e6, 0.129472e6,...&#xA;    0.053766e6, 0.011007e6, 0.003751e6, 0.001435e6, 0.000465e6,...&#xA;    0.000037e6, 0, 0.000037e6, 0.000465e6, 0.001435e6, 0.003751e6,...&#xA;    0.011007e6, 0.053766e6, 0.129472e6, 0.280144e6, 0.488104e6,...&#xA;    0.753364e6, 2.938964e6];&#xA;&#xA;params.outputflag = 0;&#xA;&#xA;%formulate model&#xA;model.obj = zeros(N, 1);&#xA;model.Q = sparse(P_0);&#xA;model.A = sparse(zeros(1, N));&#xA;model.sense = '=';&#xA;model.rhs = 0;&#xA;model.ub = repmat(1e9, N, 1);&#xA;model.lb = repmat(-1e9, N, 1);&#xA;model.modelsense = 'max';&#xA;&#xA;% piecewise tcosts&#xA;for i = 1:2&#xA;    model.pwlobj(i).var = i;&#xA;    model.pwlobj(i).x   = x;&#xA;    model.pwlobj(i).y   = x * q_0(i) - y;&#xA;end&#xA;&#xA;result = gurobi(model, params);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21960" LastEditorUserId="21960" LastEditDate="2016-10-11T19:06:20.477" LastActivityDate="2016-10-11T19:06:20.477" Title="Open Source Quadratic Programming with Piecewise Linear Objective" Tags="|optimization|convex-optimization|open-source|" AnswerCount="0" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="25172" PostTypeId="2" ParentId="25169" CreationDate="2016-10-11T15:29:38.333" Score="2" Body="&lt;p&gt;Not aware of any CPU benchmarks for DFT. This might be due to different programs being built to take advantage of different architectures etc. If you are truly looking for pure computational power and do not intend to use the computer in any other way (gaming, movies etc.) then it might be worthwhile to look into buying a server computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick search in ebay will give you a good idea about the hardware on the market. For example, your price range can get you a HP ProLiant DL585 G7 Server, which has 64-cores and 192 GB of RAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the down side, setting up a computer like this is not a trivial thing, and you will probably not be able to convert it to a gaming computer if you change your mind. It is most likely also going to be loud and produce more heat than a normal PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also want to check whether or not the program you are using is capable to fully use all the hardware you are about to purchase. 12 core CPU is not going to give you much edge if the program you use can only utilize a couple of them.&lt;/p&gt;&#xA;" OwnerUserId="19186" LastActivityDate="2016-10-11T15:29:38.333" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25174" PostTypeId="1" AcceptedAnswerId="25182" CreationDate="2016-10-12T02:56:07.747" Score="1" ViewCount="71" Body="&lt;p&gt;I wonder how one manages to use BLAS level 3 operations in the multifrontal sparse decomposition algorithm. As far as I understand, the algorithm proceeds as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For each row/column pair, assemble the dense matrix corresponding to&#xA;its nonzero entries. Write this matrix as &#xA;$$ &#xA;M =&#xA;\begin{pmatrix} &#xA;     \alpha &amp;amp; a^T \\ &#xA;     a &amp;amp; A &#xA;\end{pmatrix}. &#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Decompose the first row/column of this matrix,&#xA;$$&#xA;M = &#xA;\begin{pmatrix}&#xA;1 &amp;amp; \\&#xA;a/\alpha &amp;amp; \mathbb{I}&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;\alpha &amp;amp; \\&#xA;&amp;amp; A - a a^T/\alpha&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;1 &amp;amp; a^T/\alpha \\&#xA; &amp;amp; \mathbb{I}&#xA;\end{pmatrix}&#xA;.&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Add the resulting $A - aa^T/\alpha$ into the $M$s for row/column pairs later in the decomposition. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Each of these steps performs $\mathcal{O}(n^2)$ operations on $\mathcal{O}(n^2)$ data, so cannot be done using BLAS3 operations. In order to fix this, one would have to block several decomposition steps (step 2.) together, but then I wonder how such a method would differ from the supernodal approach. &lt;/p&gt;&#xA;" OwnerUserId="21342" LastActivityDate="2017-04-14T16:31:22.470" Title="BLAS3 in Multifrontal Sparse Decomposition" Tags="|sparse-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25175" PostTypeId="1" AcceptedAnswerId="25190" CreationDate="2016-10-12T03:54:10.443" Score="0" ViewCount="569" Body="&lt;p&gt;After trying to use RK4 to solve the below system of equations, it appears the output had large and fast oscillations even with an adaptive time step I incorporated using the Cash-Karp method. I am now looking to existing solvers with more sophisticated tools (e.g. BDF methods) to solve PDEs, and came across &lt;a href=&quot;http://hplgit.github.io/odespy/doc/pub/tutorial/html/main_odespy.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;odespy&lt;/a&gt; which I hope will work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wyRD0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wyRD0.png&quot; alt=&quot;system of PDEs with domain of interest&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YuKEq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YuKEq.png&quot; alt=&quot;Boundary/initial conditions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hplgit.github.io/odespy/doc/pub/tutorial/html/main_odespy.html#ode-sec-exgr-farg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tutorials for odespy are fairly detailed.&lt;/a&gt; Although helpful, an example of how to implement a system of PDEs with both time and spatial derivatives is missing, and I am wondering if odespy can actually solve such a problem. I am mainly curious since using the method of lines reduces this case to solving a system of a system of ODEs. Here is my code thus far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import odespy&#xA;import matplotlib&#xA;matplotlib.use('pdf')&#xA;import os &#xA;import matplotlib.pyplot as plt &#xA;from pylab import *&#xA;import numpy as np &#xA;&#xA;c = 3.*(10.**8.)  &#xA;N = 500&#xA;zmax = 1000.&#xA;z = np.linspace(0.,zmax,N+1)&#xA;f_kwargs = dict(L=zmax, z=z)&#xA;y_1 = np.zeros((N+1),dtype=np.complex_) #should be [Nz x Nt] array&#xA;y_2 = np.zeros((N+1),dtype=np.complex_)&#xA;y_3 = np.zeros((N+1),dtype=np.complex_)&#xA;y_4 = np.zeros((N+1),dtype=np.complex_)&#xA;u = y_1,y_2,y_3,y_4&#xA;&#xA;U_0 = np.zeros((4,N+1),dtype=np.complex_) &#xA;U_0[0][:] = np.cos(0.01)*(np.e**(z[:]/c))&#xA;U_0[1][:] = np.sin(0.01)*(np.e**(z[:]/c))&#xA;U_0[2][:] = 0. &#xA;U_0[3][:] = 0. &#xA;&#xA;def rhs(u, t, L=None, beta=None, z=None): &#xA;    N = len((u[0])[:]) - 1 #the PDE example specifies end condition, uses -1&#xA;    dz = z[1] - z[0] &#xA;    for i in range(0, N):&#xA;        if i == 0:&#xA;            OUT = [np.cos(0.01)*np.e**(-t),&#xA;                   np.sin(0.01)*np.e**(-t),&#xA;                    0.,&#xA;                    0.] &#xA;&#xA;        else:&#xA;            OUT = [(y_2[i]*np.conj(y_3[i]) - y_1[i]),&#xA;                   (np.conj(y_3[i]*y_1[i]) - y_2[i]),&#xA;                    y_4[i],&#xA;                (y_4[i] - ((y_3[i])**2.)*(1./dz)*np.conj(y_2[i+1] - y_2[i]))]&#xA;    return OUT&#xA;&#xA;solver = odespy.ForwardEuler(rhs, f_kwargs=f_kwargs,complex_valued=True)&#xA;solver.set_initial_condition(U_0)#solver.set_initial_condition(U_0)&#xA;&#xA;dz = z[1] - z[0]&#xA;time_points = np.linspace(0.,600.,600) #t' = t/TR&#xA;dt = time_points[1] - time_points[0]&#xA;&#xA;u, t = solver.solve(time_points)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code does not work, and my main concerns are that I am:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Unable to properly formulate the equations in such a way odespy could handle a time derivative in my PDE (the 4th equation), and that the 3rd and 4th equations were derivatives in z as opposed to t (which is the derivative of the first two equations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Unable to provide input to odespy such that it could take a system of PDEs, and then apply the &lt;a href=&quot;https://github.com/hplgit/odespy/blob/master/doc/src/tutorial/src-odespy/pde_diffusion.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;method of lines approach outlined here&lt;/a&gt;. For example, one error already visible is my initial/boundary condition implementation. I used a 4 x (N+1) array for my output and initial/boundary conditions, but the solver is not expecting such an input from what I've gathered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm still going through the documentation, but was wondering if anyone had any input for the above? Any suggestions at all on how to solve the above problem and comments on using odespy/numerical methods for a system of PDEs would be greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="20407" LastActivityDate="2016-10-14T06:05:48.073" Title="Implementing odespy for system of PDEs" Tags="|pde|python|ode|stiffness|coupling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25176" PostTypeId="2" ParentId="24066" CreationDate="2016-10-12T08:35:57.480" Score="3" Body="&lt;p&gt;First of all, there seems to be an inconsistency between the equation you wrote for $r_w$ and the code (a minus sign). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, there are two issues with your problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;As indicated in the comments, you're system is very stiff, so you should use the vode or lsoda integrator using the 'bdf' option or use the Radau method implemented in the &lt;a href=&quot;http://www.jmodelica.org/assimulo&quot; rel=&quot;nofollow&quot;&gt;Assimulo package&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Second, the model is ill-behaved. If you use the &quot;poor mans debugging method&quot; (i.e. print statements), you will observe that the calculated rates make the concentrations go negative. This will lead to problems when evaluating the rates in the next step since there are powers $&amp;lt; 1$ involved. This results in &lt;code&gt;NaN&lt;/code&gt;s appearing and the integrator doesn't know what to do with them. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The solution is to double-check your rate equations, double-check your initial conditions. Maybe &quot;capping&quot; the rate to the current value of the concentration (to allow it to reach 0.0 but not go below which is unphysical).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will also help the numerical routine if you provide it with the analytical Jacobian (which for this $3x3$ is not too hard. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your convenience, the code for Assimulo:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def dCdt(t,c):&#xA;       blahblah&#xA;&#xA;t0 = 0.0&#xA;y0 = np.array([1.0,0.0,0.0])&#xA;tmax = 25.0&#xA;&#xA;from assimulo.solvers import Radau5ODE&#xA;from assimulo.problem import Explicit_Problem&#xA;&#xA;prb = Explicit_Problem(dCdt,y0)&#xA;sim = Radau5ODE(prb)&#xA;sim.reset()&#xA;t, y = sim.simulate(tmax)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2016-10-12T15:26:13.467" LastActivityDate="2016-10-12T15:26:13.467" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25178" PostTypeId="2" ParentId="23560" CreationDate="2016-10-12T12:19:40.483" Score="1" Body="&lt;p&gt;I think you are trying to do something inherently impossible (or at least inefficient): how can you be &quot;adaptive&quot; if you want to integrate different functions at the same time? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The essence of adaptive quadrature is that your algorithm tries to detect the hard parts of the integrand and subdivides the integration interval accordingly. How can the algorithm do this efficiently if you ask it to do this for different functions at the same time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking for a less restricting library, why not use the mother of all quadrature libraries: &lt;a href=&quot;http://www.netlib.org/quadpack/&quot; rel=&quot;nofollow&quot;&gt;Quadpack&lt;/a&gt; by Piessens et al.? It's written in FORTRAN77 but you can link to it in C++. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2016-10-12T12:19:40.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25179" PostTypeId="1" CreationDate="2016-10-12T17:03:47.340" Score="1" ViewCount="80" Body="&lt;p&gt;I am trying to reproduce this figure of nodal lines of a wavefunction from this work of &lt;a href=&quot;https://michaelberryphysics.files.wordpress.com/2013/07/berry115.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Berry&lt;/a&gt; $$\psi = \sin 2r\,x \sin y + (1 + \epsilon) \sin x \sin 2r\,y$$&#xA;Here the image.  The first is $\epsilon = 0$ and the second is $\epsilon &amp;gt; 0$:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7M5AZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7M5AZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I produced a script to evaluate the value of sine at many points using numpy.  Here is the iPython script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;%matplotlib inline&#xA;&#xA;dt = 0.005&#xA;t  = 2*np.pi*np.arange(-1,1+dt,dt)&#xA;&#xA;r = 5&#xA;eps = 0&#xA;&#xA;w = np.sin(2*r*t[...,None])*np.sin(t[None,...]) + (1+eps)*np.sin(t[...,None])*np.sin(2*r*t[None,...])&#xA;&#xA;z = t[...,None] + t[None,...]&#xA;&#xA;x, y = np.where(np.abs(w)&amp;lt; 0.05)&#xA;&#xA;plt.plot(t[x], t[y], '.', markersize=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The outcome looks more like image &lt;strong&gt;(a)&lt;/strong&gt; - in fact many copies of it - instead of &lt;strong&gt;(b)&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XkxBc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XkxBc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, the paper says $r = 12$ and I get this fascinating but incorrect image.  While these graphics were dazzling in 1982 - any ordinary laptop computer should be able to re-create these and much more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DoDrU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DoDrU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is the closes I could obtain setting $r = 6$ and changing the window.  I get a very different pattern than is displayed, and my $\epsilon = 0.5$ which is quite large.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dt = 0.0005&#xA;t  = 2*np.pi*np.arange(0,0.5+dt,dt)&#xA;&#xA;r = 6&#xA;&#xA;eps = 0.5&#xA;&#xA;w = np.sin(2*r*t[...,None])*np.sin(t[None,...]) + (1+eps)*np.sin(t[...,None])*np.sin(2*r*t[None,...])&#xA;&#xA;z = t[...,None] + t[None,...]&#xA;&#xA;x, y = np.where(np.abs(w)&amp;lt; 0.05)&#xA;&#xA;plt.plot(t[x], t[y], '.', markersize=1)&#xA;plt.xlim([-0.25,3.25])&#xA;plt.ylim([-0.25,3.25])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and here is the result -- a different pattern than &lt;strong&gt;33b&lt;/strong&gt; above&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eEH2Z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eEH2Z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="4006" LastEditDate="2016-10-12T17:11:37.190" LastActivityDate="2016-10-12T19:08:00.120" Title="nodal lines of wave-function $\psi(x,y) = \sin 12x \sin y + (1 + \epsilon) \sin x \sin 12y$" Tags="|numerical-analysis|computational-physics|fourier-analysis|numpy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25181" PostTypeId="2" ParentId="25179" CreationDate="2016-10-12T19:08:00.120" Score="2" Body="&lt;p&gt;Source [61] is the book &lt;a href=&quot;https://archive.org/details/MethodsOfMathematicalPhysicsVolume1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Methods of Mathematical Physics Vol 1&lt;/a&gt; by Courant and Hilbert.  The preface is signed &lt;em&gt;R. Courant, New Rochelle, New York, 1953&lt;/em&gt; -- and in the preface we find the original German edition goes back to &lt;strong&gt;1924&lt;/strong&gt; so he might no have had a computer available to him yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed if I set &lt;strong&gt;r=6&lt;/strong&gt; or $2r=12$ and $\epsilon = 0.01$ I do get the image they produced by hand. With much narrower gaps&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5Ef0k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Ef0k.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2016-10-12T19:08:00.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25182" PostTypeId="2" ParentId="25174" CreationDate="2016-10-12T20:04:49.637" Score="2" Body="&lt;p&gt;You have essentially answered your own question. To exploit BLAS3 routines (e.g. dgemm) in sparse matrix factorization, you have to identify the supernodes and&#xA;then operate on those as dense blocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There really is no single factorization approach that can be called &lt;em&gt;the&lt;/em&gt; supernodal method. Both modern left looking codes (e.g. SuperLU) and&#xA;multifrontal codes (e.g. UMFPACK) operate on supernodes.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-10-12T20:04:49.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25183" PostTypeId="1" AcceptedAnswerId="25186" CreationDate="2016-10-12T23:02:50.903" Score="0" ViewCount="151" Body="&lt;p&gt;I came across this form of damping implemented in an elastodynamics problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wAsjO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wAsjO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stress tensor without the damping would look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \sigma = 2 \mu \epsilon + (\lambda \, \text{tr} (\epsilon)) I $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, $\mu$ and $\lambda$ are Lame' parameters dependent on the Young's Modulus and the Poisson's ratio. Trace is represented by '$\text{tr}$', and $\epsilon = (\nabla u + {(\nabla u)}^T)$ is the strain tensor. Here, $I$ is the second order identity tensor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is this type of damping called?&lt;/li&gt;&#xA;&lt;li&gt;Does $\eta$ control the damping %? For eg. should $\eta = 0.1$ mean 10% damping? I don't think it works like that though.&lt;/li&gt;&#xA;&lt;li&gt;Any other comments on this type of damping?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I ran some wave propagation simulations based on this type of damping and observed the following in Paraview for $\eta=0$ and $\eta=1$. I don't see substantial difference in the displacement values. It is 17% for the highlighted point. Any observations/comments from experts will be useful. Please click on the picture to enlarge it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Aaj3M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Aaj3M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="16705" LastEditDate="2016-10-12T23:16:47.527" LastActivityDate="2016-10-13T03:25:32.843" Title="Damping for Dynamic Problem using FEM" Tags="|finite-element|wave-propagation|solid-mechanics|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="25185" PostTypeId="2" ParentId="20403" CreationDate="2016-10-12T23:12:59.893" Score="-5" Body="&lt;p&gt;A few more pointers on how to start with GPUs:&#xA;1. the new free online course that covers libraries and OpenACC directives: &lt;a href=&quot;https://developer.nvidia.com/intro-to-openacc-course-2016&quot; rel=&quot;nofollow&quot;&gt;https://developer.nvidia.com/intro-to-openacc-course-2016&lt;/a&gt;&#xA;2. the list of libraries already accelerated with GPUs: &lt;a href=&quot;https://developer.nvidia.com/gpu-accelerated-libraries&quot; rel=&quot;nofollow&quot;&gt;https://developer.nvidia.com/gpu-accelerated-libraries&lt;/a&gt;&#xA;3. openacc.org - also has a lot of resources &lt;/p&gt;&#xA;" OwnerUserId="21976" LastActivityDate="2016-10-12T23:12:59.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25186" PostTypeId="2" ParentId="25183" CreationDate="2016-10-13T03:25:32.843" Score="1" Body="&lt;p&gt;The constitutive model is &#xA;$$&#xA;  \boldsymbol{\sigma}  = 2\mu \boldsymbol{\varepsilon} + \lambda \text{tr}(\boldsymbol{\varepsilon})\,\mathbf{I} + \eta\, \text{tr}(\dot{\boldsymbol{\varepsilon}})\,\mathbf{I} \,.&#xA;$$&#xA;Consider a hydrostatic creep test where $\boldsymbol{\sigma}$ is kept fixed.  We want to find the evolution of $\boldsymbol{\varepsilon}$ under that condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have &#xA;$$&#xA; \text{tr}[ \eta\, \text{tr}(\dot{\boldsymbol{\varepsilon}})\,\mathbf{I}] = \text{tr}[\boldsymbol{\sigma} - 2\mu \boldsymbol{\varepsilon} - \lambda \text{tr}(\boldsymbol{\varepsilon})\,\mathbf{I}]&#xA;$$&#xA;or&#xA;$$&#xA;  3\eta\, \text{tr}(\dot{\boldsymbol{\varepsilon}}) = \text{tr}(\boldsymbol{\sigma}) - 2\mu\,\text{tr}(\boldsymbol{\varepsilon}) - 3\lambda\, \text{tr}(\boldsymbol{\varepsilon})\,.&#xA;$$&#xA;Define $\varepsilon_v := \text{tr}(\boldsymbol{\varepsilon})$ and $\sigma_m := \text{tr}(\boldsymbol{\sigma})$.  Then,&#xA;$$&#xA;  \dot{\varepsilon_v} = \frac{\sigma_m}{\eta} - \frac{2\mu + 3\lambda}{3\eta}\,\varepsilon_v = \frac{\sigma_m}{\eta} - \frac{K}{\eta} \,\varepsilon_v \,.&#xA;$$&#xA;Solving this ODE gives&#xA;$$&#xA;  \varepsilon_v(t) = \frac{\sigma_m}{K} + C \exp\left(-\frac{K t}{\eta}\right)&#xA;$$&#xA;where $C$ is determined from the initial conditions. For zero initial strain (is this physically possible in this case?), &#xA;$$&#xA;  \varepsilon_v(t) = \frac{\sigma_m}{K}\left[1 - \exp\left(-\frac{K t}{\eta}\right)\right] \,.&#xA;$$&#xA;This is a creep model. But the effect of the value of $\eta$ will not be observed except at small values of $t$ and large values ($O(K)$) of $\eta$. This why this is not a good model for damping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can interpret the parameter $\eta$ as a damping parameter.&#xA;To see why you have to repeat the above exercise with&#xA;$$&#xA;  \sigma_m = \sigma_m \exp(-i \omega t) &#xA;$$&#xA;and look at the imaginary part of the strain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the deviatoric stresses/strains, the model has no viscoelastic effect.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2016-10-13T03:25:32.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25187" PostTypeId="1" AcceptedAnswerId="25188" CreationDate="2016-10-13T10:44:36.750" Score="4" ViewCount="242" Body="&lt;p&gt;I'm sorry for this silly question. Several times I faced with optimization problems which can be expressed as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \mathrm x^T \mathrm A \mathrm x\\ \text{subject to} &amp;amp; \|\mathrm x\| = 1\end{array}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathrm A$ is square matrix and $\mathrm x$ is vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This kind of problem arises in PCA or rigid alignment of 3D point sets)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that solution of this problem is eigenvalue decomposition of $\mathrm A$. Smallest eigenvalue should be minimum of optimization problem and corresponding eigenvector should be solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know is there exist special name for such optimization problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: Sorry for my poor English.&lt;/p&gt;&#xA;" OwnerUserId="15233" LastEditorUserId="20417" LastEditDate="2016-10-13T13:40:38.137" LastActivityDate="2016-11-01T18:29:31.427" Title="Looking for name of optimization problem in form $\min \mathrm x^T \mathrm A \mathrm x$ subject to $\|\mathrm x\| = 1$" Tags="|optimization|constrained-optimization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25188" PostTypeId="2" ParentId="25187" CreationDate="2016-10-13T16:18:23.507" Score="10" Body="&lt;p&gt;Since the optimization problem has a well-known closed-form solution, it is rarely used in itself and hence usually not given a name. The objective function $x^TAx$ (using $\|x\|=1$), however, is known under the name &lt;em&gt;Rayleigh quotient&lt;/em&gt;; the set of its values is called &lt;em&gt;field of values&lt;/em&gt;, or &lt;em&gt;numerical range&lt;/em&gt; (of $A$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, this is a textbook example of a quadratic programming problem with a single differentiable equality constraints (if written as $\|x\|^2-1=0$).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2016-11-01T18:29:31.427" LastActivityDate="2016-11-01T18:29:31.427" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25189" PostTypeId="2" ParentId="25187" CreationDate="2016-10-13T17:58:22.330" Score="7" Body="&lt;p&gt;I know this should be a comment but I do not have enough reputation to comment. I just want to point out that for an arbitrary $A$ with real entries the solution (minimum) to this problem is the smallest eigenvalue of the symmetric part of $A$, namely $S(A)=\frac{(A+A^T)}{2}$, and in the special case when $A$ is symmetric this indeed (as in the original question) gives the smallest eigenvalue of $A$.&lt;/p&gt;&#xA;" OwnerUserId="21695" LastActivityDate="2016-10-13T17:58:22.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25190" PostTypeId="2" ParentId="25175" CreationDate="2016-10-13T19:19:48.373" Score="2" Body="&lt;p&gt;&lt;strong&gt;Odespy&lt;/strong&gt; needs from you a 1st order system of ODEs, namely something of the form&#xA;\begin{align}&#xA;\frac{d\mathbf{y}}{dt} = \mathbf{f}(\mathbf{y},t), \tag{1}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, for your particular problem, $y_i$ are single-variate complex-valued functions that depend solely on time (that means you need to arrive at the semi-discrete form first by discretizing the spatial dimension, this is what is meant by the method of lines), and $i=1 \ldots3 $. Note that $y_4$ is just a placeholder variable defined (for the original space-time problem) by&#xA;$y_4 := \partial y_3 / \partial z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if we get rid of $y_4$ in your original formulation, we get the space-time system:&#xA;\begin{eqnarray}&#xA;\frac{\partial y_1}{\partial t} &amp;amp;=&amp;amp; \mbox{(as before)} \\&#xA;\frac{\partial y_2}{\partial t} &amp;amp;=&amp;amp; \mbox{(as before)} \\&#xA;\frac{\partial y_3}{\partial t} &amp;amp;=&amp;amp; \frac{\partial^2 y_3}{\partial z^2} - \frac{\partial y_3}{\partial z} + y_3^2 \frac{\partial y_2^*}{\partial z}.&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's left to do is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;discretize the RHS in space ($z$-terms) using suitable finite-difference formulas for the second and first partial derivatives in $z$, to arrive at a semi-discrete system like (1) above, and&lt;/li&gt;&#xA;&lt;li&gt;pass the semi-discrete system to Odespy (make sure you get the boundary conditions right).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Make sure to choose a time integration method that can handle the time scales and other properties of your system (for example, forward Euler does not work that well for stiff systems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Later edit&lt;/em&gt;: had a closer look at the system and the $y_i$ look to be real-valued functions of real variables (initial/boundary conditions and right-hand side are all real). So if you know your solutions have nonzero imaginary parts, you should double-check your PDE system.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2016-10-14T06:05:48.073" LastActivityDate="2016-10-14T06:05:48.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25193" PostTypeId="2" ParentId="24889" CreationDate="2016-10-14T07:14:17.523" Score="0" Body="&lt;p&gt;You can use area averaging (where weight is cell area) technique to calculate the variable values at the cell vertices. The variable value at the edge center can be calculated just by taking simple average of the values at the nodes of that edge. We in our group commonly use this technique for gradient calculation using Green-Gauss approach. Refer section 3.4 of &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045793010003336&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; for more information. All the best.&lt;/p&gt;&#xA;" OwnerUserId="3549" LastEditorUserId="3549" LastEditDate="2016-10-14T07:20:07.187" LastActivityDate="2016-10-14T07:20:07.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25194" PostTypeId="1" CreationDate="2016-10-14T07:32:44.210" Score="3" ViewCount="145" Body="&lt;p&gt;If I take the step too large I will get error, while if I take the step too small I also get an error. In my case, instead of seeing the function decreasing, i have it increasing if I take the step too small.&lt;/p&gt;&#xA;" OwnerUserId="20760" LastEditorUserId="20417" LastEditDate="2016-10-14T17:16:16.277" LastActivityDate="2016-10-14T17:16:16.277" Title="Will the numerical solving of the differential equation be wrong if I take the step too small?" Tags="|numerical-analysis|ode|" AnswerCount="1" CommentCount="1" ClosedDate="2016-10-15T09:11:43.197" ContentLicense="CC BY-SA 3.0" />
  <row Id="25196" PostTypeId="1" AcceptedAnswerId="25197" CreationDate="2016-10-14T10:24:39.293" Score="7" ViewCount="3215" Body="&lt;p&gt;I am implementing higher order derivatives for FEM. Example, to solve a Poisson problem, biharmonic or triharmonic PDE one needs first, second or third order derivatives respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As usually done in FEM, there is a mapping from a reference element to a physical element e.g. $F : \hat{K} \rightarrow K$ with the associated coordinates $(\hat{x}, \hat{y})$ and $(x,y).$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we assume a linear affine map on a quadrilateral mesh, then for a function $u,$ the derivative is easily computed and given by $\nabla u = J^{-T}  \nabla \hat{u}.$  Now, my aim is to compute $\nabla^2 u, \nabla^3 u,...$ possibly $\nabla^n u.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will appreciate any ideas on how to compute these derivatives. &lt;/p&gt;&#xA;" OwnerUserId="7984" LastEditorUserId="7984" LastEditDate="2016-10-25T11:39:11.220" LastActivityDate="2019-12-24T15:31:37.977" Title="implementing higher order derivatives for finite element" Tags="|finite-element|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="25197" PostTypeId="2" ParentId="25196" CreationDate="2016-10-14T11:03:59.460" Score="5" Body="&lt;p&gt;You appear to have pretty much answered your own question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing notation slightly to use $\bf x$ for physical coordinates and $\bf \xi$ for the reference coordinate (because I have a nasty habit of leaving tildes off variables where I shouldn't) you have in component form that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial}{\partial x_{i}}u\left(\bf{x}\right)=\sum_{j}\frac{\partial\xi_{j}}{\partial x_{i}}\frac{\partial}{\partial\xi_{j}}u\left(\bf{\xi}\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so using the definition &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 f}{\partial x_i^2} := \frac{\partial}{\partial x_i}\frac{\partial f}{\partial x_i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial x_i^2} = \frac{\partial}{\partial x_i}\frac{\partial u}{\partial x_i} = \frac{\partial}{\partial x_i} \frac{\partial}{\partial x_{i}}u\left(\bf{x}\right)= \frac{\partial}{\partial x_i} \sum_{j}\frac{\partial\xi_{j}}{\partial x_{i}}\frac{\partial}{\partial\xi_{j}}u\left(\bf{\xi}\right) = \sum_k \frac{\partial\xi_{k}}{\partial x_{i}}\frac{\partial}{\partial\xi_{k}}\sum_{j}\frac{\partial\xi_{j}}{\partial x_{i}}\frac{\partial}{\partial\xi_{j}}u\left(\bf{\xi}\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming your mapping is linear, we can take the derivative through the summation and write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial x_i^2} = \sum_{j,k} \frac{\partial\xi_{k}}{\partial x_{i}}\frac{\partial\xi_{j}}{\partial x_{i}}\frac{\partial^2 }{\partial \xi_k \partial\xi_{j}}u\left(\bf{\xi}\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully you see the connection to what you're already doing with first derivatives.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-10-14T11:03:59.460" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25199" PostTypeId="1" CreationDate="2016-10-14T14:10:50.893" Score="1" ViewCount="1782" Body="&lt;p&gt;In NVT molecular dynamics, Nosé Hoover thermostat is a method defining an extended system. I can understand perfectly how the Nosé Hoover differential equations are derived (Frenkel&amp;amp;Smit's book). However, when it comes to practice, I have no idea how to solve it numerically. I tried a method similar to velocity Verlet to integrate the full equations, but it does not really work. What is mysterious is that all the textbooks I have seen so far all stop at the differential equations and do not explain how to numerically integrate it, and this is the hard part. Could anyone suggest any references that clearly explain how to discretize Nosé Hoover equations, or any written codes to look at?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="21938" LastActivityDate="2020-07-19T18:06:50.330" Title="How to integrate numerically Nosé Hoover equation?" Tags="|molecular-dynamics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25200" PostTypeId="2" ParentId="25164" CreationDate="2016-10-14T14:48:57.573" Score="1" Body="&lt;p&gt;The short answer is &quot;no&quot;, you cannot run a &lt;code&gt;parfor&lt;/code&gt; loop without the distributed computing toolbox. You would have to do it manually, by setting up a system for communicating between the two matlab sessions, for sharing data, and for remotely executing commands. All of which is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a small number of packages that will take care of this for you:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;a href=&quot;http://www.mathworks.com/matlabcentral/fileexchange/44077-batch-job&quot; rel=&quot;nofollow&quot;&gt;http://www.mathworks.com/matlabcentral/fileexchange/44077-batch-job&lt;/a&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;a href=&quot;https://github.com/kinnala/matlab-partool&quot; rel=&quot;nofollow&quot;&gt;https://github.com/kinnala/matlab-partool&lt;/a&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have not used either, so I cannot make a recommendation.&lt;/p&gt;&#xA;" OwnerUserId="9810" LastActivityDate="2016-10-14T14:48:57.573" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25201" PostTypeId="2" ParentId="25194" CreationDate="2016-10-14T16:43:39.480" Score="8" Body="&lt;p&gt;Using too small of a timestep can lead to accuracy issues due to finite precision. Since you can use derivative approximations to derive time integration schemes, it is fair to look at how derivative approximations fare as you change stepsize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IjaQd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IjaQd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see above, where I used a second order central difference approximation, the error for derivative step sizes between $1$ and $10^{-5}$ produce the expected second order convergence rate. However, past that span of step sizes, the error starts to grow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a problem with finite precision and so it's feasible that too small a step size in your integration schemes could result in a similar issue.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-10-14T16:53:57.927" LastActivityDate="2016-10-14T16:53:57.927" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25202" PostTypeId="2" ParentId="25199" CreationDate="2016-10-14T20:59:33.270" Score="4" Body="&lt;p&gt;You can find explicit pseudo-code for Nosé-Hoover Chains in Section 4 of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Martyna, G. J., Tuckerman, M. E., Tobias, D. J., &amp;amp; Klein, M. L. (1996). Explicit reversible integrators for extended systems dynamics. Molecular Physics, 87(5), 1117–1157. Retrieved from &lt;a href=&quot;http://www.tandfonline.com/doi/abs/10.1080/00268979600100761&quot; rel=&quot;nofollow&quot;&gt;http://www.tandfonline.com/doi/abs/10.1080/00268979600100761&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a short example of the Nosé-Hoover-Langevin thermostat on a 1D harmonic oscillator implemented in MATLAB ($q$ is position, $p$ is momentum, $\xi$ is the degree of freedom associated to the thermostat, $\gamma$ is the friction, $kT$ is the temperature times the Boltzmann constant, and $dt$ is the time-step length):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [q p xi] = nhl(q, p, xi, gamma, kT, dt)&#xA;&#xA;mass = 1.0;&#xA;&#xA;kinetic = @(p) 1/(2*mass) * p^2;&#xA;&#xA;force = @(q) -q;&#xA;&#xA;N = 1;                                  % Number of dimensions&#xA;&#xA;xi = exp(-gamma*dt/2) * xi + sqrt(kT/mass*(1 - exp(-gamma*dt))) * randn;&#xA;&#xA;p = p + dt/2 * force(q);&#xA;&#xA;p = p * exp(-xi*dt/4);&#xA;xi = xi + dt/2 * (2*kinetic(p) - N*kT)/mass;&#xA;p = p * exp(-xi*dt/4);&#xA;&#xA;q = q + dt/mass * p;&#xA;&#xA;p = p * exp(-xi*dt/4);&#xA;xi = xi + dt/2 * (2*kinetic(p) - N*kT)/mass;&#xA;p = p * exp(-xi*dt/4);&#xA;&#xA;p = p + dt/2 * force(q);&#xA;&#xA;xi = exp(-gamma*dt/2) * xi + sqrt(kT/mass*(1 - exp(-gamma*dt))) * randn;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can find the general Nosé-Hoover-Langevin method described explicitly in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Leimkuhler, B., Noorizadeh, E., &amp;amp; Theil, F. (2009). A Gentle Stochastic Thermostat for Molecular Dynamics. Journal of Statistical Physics, 135(2), 261–277. &lt;a href=&quot;http://doi.org/10.1007/s10955-009-9734-0&quot; rel=&quot;nofollow&quot;&gt;http://doi.org/10.1007/s10955-009-9734-0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since these two methods (NHC and NHL) are closely related to the original NH method, understanding how to get from the ODEs to the actual integrator in any of these methods will lead you to understand how it's done for NH.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2016-10-14T21:06:20.217" LastActivityDate="2016-10-14T21:06:20.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25203" PostTypeId="1" CreationDate="2016-10-14T21:16:38.130" Score="22" ViewCount="8499" Body="&lt;p&gt;I've been using HDF5 for years, but as the size of the dataset grows I'm starting to experience the same problems listed here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cyrille.rossant.net/moving-away-hdf5/&quot;&gt;http://cyrille.rossant.net/moving-away-hdf5/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you point me to a format like HDF5 with&#xA;- Good support for parallel writes&#xA;- Support for chunked access of huge matrices&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My typical use case is a 100k x 100k integer matrix. I'd like to have it as a whole file from a logical perspective, but I need to write it chunk by chunk with parallel workers.&lt;/p&gt;&#xA;" OwnerUserId="21998" LastEditorUserId="21998" LastEditDate="2016-10-15T07:36:57.730" LastActivityDate="2019-02-04T13:07:26.647" Title="Alternatives to hdf5" Tags="|data-storage|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25204" PostTypeId="1" CreationDate="2016-10-15T10:50:17.530" Score="1" ViewCount="767" Body="&lt;p&gt;I've the following Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r = symrcm(A(2:end, 2:end)); &#xA;prcm = [1 r + 1]; &#xA;spy(A(prcm, prcm));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;A&lt;/code&gt; should be sparse &lt;em&gt;connectivity&lt;/em&gt; matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understood what it does:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Finds a permutation vector &lt;code&gt;r&lt;/code&gt; of the submatrix of &lt;code&gt;A&lt;/code&gt; &lt;code&gt;A(2:end, 2:end)&lt;/code&gt; (produced by the reverse Cuthill-McKee algorithm)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Creates a vector &lt;code&gt;prcm&lt;/code&gt; which is basically a vector with a $1$ in the first position and all other elements of &lt;code&gt;r&lt;/code&gt; increased by $1$. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This &lt;code&gt;prcm&lt;/code&gt; vector applied to &lt;code&gt;A&lt;/code&gt; as &lt;code&gt;A(prcm, prcm)&lt;/code&gt;  logically means that we're going to permutate all rows and columns of &lt;code&gt;A&lt;/code&gt; according to the reverse Cuthill-McKee algorithm except the first row and the first column. So the resulting matrix would look something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8ka14.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8ka14.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ignore the specific numbers that you see in the plot.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why would one want such a permutation of the rows and columns of a matrix?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I've been reading and I've observed applying, for example, the guassian elimination to this matrix would produce a disastrous fill-in after trying to remove all entries of the first row (Check chapter 5.7 from &quot;A first course in numerical methods&quot; by Ascher and Greif). So, who wrote this code definitely didn't want to find a permutation of $A$ to apply the guassian elimination...&lt;/p&gt;&#xA;" OwnerDisplayName="user9925" LastActivityDate="2017-02-13T09:38:08.667" Title="Find a permutation matrix (using the Matlab's function $symrcm$) of a matrix $A(2:end, 2:end)$" Tags="|matlab|sparse-matrix|vector|banded-matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25205" PostTypeId="2" ParentId="25204" CreationDate="2016-10-15T13:43:43.087" Score="1" Body="&lt;p&gt;From your comment &quot;The plot emphasises expectedly the connections of node 1&quot;, I guess that maybe the idea is showing that node 1 is connected not only to a set of nodes that are &quot;one close to the other&quot;, but its interconnections are spread somewhat evenly across the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reverse Cuthill-McKee in theory reorders the nodes so that clusters are mapped into nearby positions. Or at least &lt;em&gt;some&lt;/em&gt; clusters: those which include the starting vertex. Or something morally similar, at least. So the fact that the nonzeros in row 1 are not an almost adjacent set of entries, (say, $\{2,3,\dots,20\}$), but  are spread uniformly across the matrix somehow shows that 1 is a 'well-connected node'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is really the answer, then it seems quite an odd way to measure 'well-connectedness'. There is a lot of academic literature on measures of &lt;a href=&quot;https://en.wikipedia.org/wiki/Centrality&quot; rel=&quot;nofollow&quot;&gt;centrality&lt;/a&gt; of vertices in a graph, from &lt;a href=&quot;https://en.wikipedia.org/wiki/PageRank&quot; rel=&quot;nofollow&quot;&gt;Pagerank&lt;/a&gt; to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Estrada_index&quot; rel=&quot;nofollow&quot;&gt;Estrada index&lt;/a&gt;, and I have never seen RCM used for this purpose.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2016-10-15T13:43:43.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25208" PostTypeId="1" CreationDate="2016-10-15T17:51:56.013" Score="2" ViewCount="70" Body="&lt;p&gt;I am solving a quadratic program with regularization parameter $\alpha\geq0$ to get the solution to a problem of the form&#xA;$$&#xA;p(\alpha):=&#xA;\arg\min_{p\in\mathbb{R}^n}\ [\alpha(v^\top p)+f(p)],&#xA;$$&#xA;where $f(\cdot)$ is a strictly convex, continuous, and piecewise quadratic function of $p\in\mathbb{R}^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm hoping to say something about the behavior of $p$ and/or the optimal objective value as $\alpha$ changes, in the style of solution path papers like &lt;a href=&quot;http://projecteuclid.org/euclid.aos/1185303996&quot; rel=&quot;nofollow&quot;&gt;this one&lt;/a&gt;.  Sadly, the paper I linked does not seem to apply since the function next to $\alpha$ is not bounded below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have pointers for theorems that characterize the &quot;solution path&quot; ($p$ as a function of the regularizer $\alpha$) of problems in this form?&lt;/p&gt;&#xA;" OwnerUserId="5129" LastEditorUserId="5129" LastEditDate="2016-10-15T18:20:20.110" LastActivityDate="2016-10-15T18:20:20.110" Title="&quot;Solution path&quot; for quadratic program as regularizer changes" Tags="|optimization|convex-optimization|quadratic-programming|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25209" PostTypeId="1" AcceptedAnswerId="25238" CreationDate="2016-10-15T22:25:07.520" Score="1" ViewCount="138" Body="&lt;p&gt;Assume we use FEM with piecewise linear finite elements to discretize the BVP over $\omega = (0,1)$: $-u''+ bu' + u = 2x$, $u(0) = u(1) = 0$ for parameter $b\in R$. Given a mesh $T = \left\{x_i\right\}_{i=0}^{N}$ where $x_i = 1 - (1-\frac{i}{N})^{\beta}$ for some $\beta &amp;gt; 1$. Using principle of error equilibration, find a value of $\beta$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempt:&lt;/strong&gt; I tried to look up the literature for the mathematical definition of &quot;error equilibrium in FEM,&quot; but I failed to find it. Could someone please help with a useful reference? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I have a question on my discretization method: as we know, multiplying both sides of the PDE by a test function $v\in H_{0}^{1}(w)$. Then we would have the following variational formulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{0}^{1} (u'v' - buv' + uv) = \int_{0}^{1} 2xv$ for every $v\in H_{0}^{1}(w)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, define $B[u,v] = \int_{0}^{1} (u'v' - buv' + uv)$ and $F(v) = 2xv$. Let $\left\{\phi_{i}\right\}_{i=0}^{N-2}$ be the basis of the finite-dimensional subspace $V_h\subset H_{0}^{1}(w)$ over the given mesh $T$. Now, we can express $u(x) = \sum_{i=0}^{N-2} \phi_{i}(x)u_i$ where $\phi_{i}(x)$ are piecewise linear functions defined over the mesh $T$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Due to the boundary conditions: $u(0) = u(1) = 0$, would the first basis function (i.e, $\phi_0(x)$) be a piecewise hat function with vertices $(x_0, 0), (x_1, 1)$ and $(x_2,0)$, and the last basis function (i.e, $\phi_N(x)$)  are piecewise hat functions with vertices $(x_{N-2}, 0), (x_{N-1}, 1)$ and $(x_N,0)$? Usually, for Dirichlet condition, $\phi_0(x)$ are with vertices $(x_0, 1), (x_1,0)$ and $(x_0, 0)$, but then in that case, $u(x) = \sum_{i=1}^{N} \phi_{i}(x)u_i + \alpha \phi_{0}(x)$ where $\alpha = u(0)$, and $u(1) = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Is working with $v\in H_{0}^{1}(w)$ the right setting in this case? Usually, for one-dimensional problem like this, we are only allowed to let either $v(0) = 0$ or $v(1) = 0$, but in this case as I could see that the information from the given data at $x=1$ and $x=0$ are &quot;useless,&quot; I decided to set $v(1) = v(0) = 0$. This resulted in the slight change in the set of basis functions $\left\{\phi_{i}\right\}_{i=0}^{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Assume the BCs now are $u'(0) = 1$ and $u(1) = 1$, could I use the same set of test function as described in (1) above, but adding two elements: $\phi_{0}(x) = $  linear function between 2 points $(0,1)$ and $(x_1, 0)$, while $\phi_{N}(x) = $ linear function connecting $(x_{N-1},0)$ and $(x_N, 1).$ &lt;/p&gt;&#xA;" OwnerUserId="18465" LastEditorUserId="18465" LastEditDate="2016-10-18T05:33:27.890" LastActivityDate="2016-10-18T18:05:01.670" Title="3 questions on FEM to solve elliptic PDE with homogeneous and mixed boundary conditions" Tags="|finite-element|pde|elliptic-pde|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25210" PostTypeId="1" CreationDate="2016-10-16T01:23:54.547" Score="1" ViewCount="177" Body="&lt;p&gt;The Schrodinger equation for time-dependent Hamiltonian is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$i\hbar\frac{d}{dt}\psi(t) = H(t)\psi(t) \, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I knew $\psi(t)$, I want to know $\psi(t+\Delta t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I take exponential or use ode solver?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the mathematical solution after one timestep should be (or not?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(t+\Delta t) = e^{-\frac{i}{\hbar}H(t)\Delta t}\psi(t) \, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It gives a different result from using ode solver unfortunately. Which is the most correct one?&lt;/p&gt;&#xA;" OwnerUserId="20759" LastActivityDate="2016-10-16T01:23:54.547" Title="The Schrodinger equation for time-dependent Hamiltonian after one timestep, taking exponential or use ode solver?" Tags="|ode|computational-physics|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="25212" PostTypeId="1" CreationDate="2016-10-16T02:27:16.910" Score="1" ViewCount="92" Body="&lt;p&gt;If this isn't the right SE site, please suggest the right one? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can software remove all distortion from a 180-degree lens? In other words, if the goal is to optimize for distortion-free images, would a stereo configuration with two 100-degree lenses and two sensors (overlapped to provide 180-deg coverage) be better than a single sensor with a 180-deg lens?&lt;/p&gt;&#xA;" OwnerUserId="22007" LastEditorUserId="9667" LastEditDate="2016-10-16T23:32:37.577" LastActivityDate="2016-10-17T20:35:09.863" Title="Can software remove all distortion from image taken with 180-degree lens?" Tags="|optimization|computer-vision|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25213" PostTypeId="1" AcceptedAnswerId="25214" CreationDate="2016-10-16T04:21:55.153" Score="16" ViewCount="2746" Body="&lt;p&gt;In a system where energy theoretically should be conserved, the most accurate simulation would conserve energy (as well as giving accurate positions, velocities and etc). RK4 is more accurate than leapfrog, yet leapfrog conserves energy and RK4 doesn’t. Why is this?&lt;/p&gt;&#xA;" OwnerUserId="21658" LastActivityDate="2016-10-16T18:14:38.733" Title="Why is leapfrog integration symplectic and RK4 not, if the latter is more accurate?" Tags="|computational-physics|time-integration|molecular-dynamics|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25214" PostTypeId="2" ParentId="25213" CreationDate="2016-10-16T06:47:51.410" Score="15" Body="&lt;p&gt;&lt;strong&gt;TL;DR: It depends on what kind of accuracy you need.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Energy conservation does not automatically equal accuracy. Suppose, you want to simulate the solar system, and you are using a solver that – to use an extreme example – just rotates the entire system by some angle every second. These solutions obviously conserve energy, but they are blatantly incorrect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you want to predict celestial motions for a sufficiently short time span, then the effects of a Runge–Kutta method not preserving energy are negligible. Rather, this takes its toll on long-term simulations. On short time scales, a Runge–Kutta method will give you more accurate results than leapfrog – at least for a comparable computational effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, on long time scales, neither method yields very accurate results in the sense of predicting the precise future of some initial condition (which may also get difficult due to the butterfly effect). However, the leapfrog method at least yields some plausible solution, as energy is preserved. This is sufficient for many simulations where the qualitative behaviour of the investigated systems is of interest.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2016-10-16T18:14:38.733" LastActivityDate="2016-10-16T18:14:38.733" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25216" PostTypeId="1" CreationDate="2016-10-16T19:06:33.090" Score="2" ViewCount="120" Body="&lt;p&gt;The main motivation behind my next question is that I think I derived a higher order numerical scheme for linear advection equation that is unconditionally stable using Von Neumann stability analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the stability analysis of the numerical scheme I have to find the maximal value of so called amplification factor $|S|$ where $S$ depends on 5 parameters, it means  $S=S(x,y,f,c,d)$ and&#xA;$$&#xA;S=&#xA;\left(2 \left(-c^2+c \cos (x) (c-d (2 f+1) \cos (y)+2 d f+d-1)+d&#xA;   \cos (y) (2 c f+c+d-1)-2 c d f-c d \sin (x) \sin (y)-c d-3 i c&#xA;   \sin (x)+c-d^2-3 i d \sin (y)+d+6\right)\right) \cdot \left(2 i c^2 \sin (x)-i&#xA;   c^2 \sin (2 x)+c^2-2 i c d \sin (x+y)+2 c d \cos (x+y)+2 i c d&#xA;   \sin (x)-2 c (c+d+5) \cos (x)+2 i c d \sin (y)-2 c d \cos (y)+2&#xA;   c d+10 i c \sin (x)-2 i c \sin (2 x)+(c+2) c \cos (2 x)+8 c+2 i&#xA;   d^2 \sin (y)-i d^2 \sin (2 y)-2 d^2 \cos (y)+d^2 \cos (2&#xA;   y)+d^2+10 i d \sin (y)-2 i d \sin (2 y)-10 d \cos (y)+2 d \cos&#xA;   (2 y)+8 d+12\right)^{-1}&#xA;$$&#xA;Note that $i$ is imaginary number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My hypothesis is that $|S| \le 1$ for $-\pi \le x \le \pi$,  $-\pi \le y \le \pi$, $-1 \le f \le 0$ and $c \ge 0$ and $d \ge 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to now I checked the values of $|S|$ for many tabulated values of input variables e.g. for $c \le 100$ and $d \le 100$. Moreover I used Mathematica to compute numerically the maximal values of $|S|$ with many starting values and several built-in optimization methods, I got always the maximal value of $|S|$ being $1$. Of course I used the numerical scheme to compute several examples of linear advetion equation with very large time steps and observed no instabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any useful thoughts, helps or even your tries ;-) how to find the values of $(x,y,f,c,d)$ bounded by above constraints for which $|S|&amp;gt;1$. Or from the other view - what kind of methods and arguments would you use to support the hypothesis that $|S| \le 1$ ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Here is the input for Mathematica&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S=(2*(6 + c - c^2 + d - c*d - d^2 - 2*c*d*f + &#xA;     d*(-1 + c + d + 2*c*f)*Cos[y] + &#xA;     c*Cos[x]*(-1 + c + d + 2*d*f - d*(1 + 2*f)*Cos[y]) - &#xA;     3*I*c*Sin[x] - 3*I*d*Sin[y] - c*d*Sin[x]*Sin[y]))/(12 + 8*c + c^2 + 8*d + 2*c*d + d^2 - 2*c*(5 + c + d)*Cos[x] + &#xA;  c*(2 + c)*Cos[2*x] - 10*d*Cos[y] - 2*c*d*Cos[y] - &#xA;  2*d^2*Cos[y] + 2*d*Cos[2*y] + d^2*Cos[2*y] + &#xA;  2*c*d*Cos[x + y] + 10*I*c*Sin[x] + 2*I*c^2*Sin[x] + &#xA;  2*I*c*d*Sin[x] - 2*I*c*Sin[2*x] - I*c^2*Sin[2*x] + &#xA;  10*I*d*Sin[y] + 2*I*c*d*Sin[y] + 2*I*d^2*Sin[y] - &#xA;  2*I*d*Sin[2*y] - I*d^2*Sin[2*y] - 2*I*c*d*Sin[x + y])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;P.S.S. If for any reasons you need more insights, see my paper at&#xA;&lt;a href=&quot;http://dl.dropboxusercontent.com/u/386482/articles/frolkovic-algoritmy-corr.pdf&quot; rel=&quot;nofollow&quot;&gt;http://dl.dropboxusercontent.com/u/386482/articles/frolkovic-algoritmy-corr.pdf&lt;/a&gt;&#xA;where the scheme is given in (3.10) for $f=0$.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-10-16T19:23:43.723" LastActivityDate="2017-02-12T19:44:34.287" Title="How to support or contradict a hypothesis on unconditional stability using numerical optimization" Tags="|optimization|finite-difference|stability|advection|implicit-methods|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25217" PostTypeId="2" ParentId="20786" CreationDate="2016-10-16T21:28:20.520" Score="1" Body="&lt;p&gt;Check out &lt;a href=&quot;https://github.com/tbs1980/NumericalIntegration&quot; rel=&quot;nofollow&quot;&gt;https://github.com/tbs1980/NumericalIntegration&lt;/a&gt;. It is based on QUADPACK (which GSL is also based on), and has some neat modern features, e.g. based on Eigen, multiprecision support.&lt;/p&gt;&#xA;" OwnerUserId="1190" LastActivityDate="2016-10-16T21:28:20.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25218" PostTypeId="1" CreationDate="2016-10-16T23:14:36.490" Score="-2" ViewCount="221" Body="&lt;p&gt;Is there any method to deal with this problem? I am using Mathematica to solve the differential equations, but the calculating time is so long because of the large domain $x\in[0,10^{6})$. In fact I don't need the valves of y at x=0,0.01,0.02,0.03....I just need the values at x=0,1000,2000... or even less dilute. But to get the value at x=1000, the program must go through x=0.01,0.02...this the reason why it cost so much time. I am wondering whether this a clever method could facilitate this process (like, the computer doesn't need to store the values at x=0.01,0.02..., but just use these numbers as a bridge to calculate the following points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="22013" LastEditorUserId="1280" LastEditDate="2016-10-17T16:22:29.577" LastActivityDate="2016-10-17T19:57:22.397" Title="Numerically solving differential equations, the domain is very long, [0, +10^6), so the calculating time is very long" Tags="|ode|mathematica|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25219" PostTypeId="1" AcceptedAnswerId="25220" CreationDate="2016-10-17T05:41:14.577" Score="1" ViewCount="162" Body="&lt;p&gt;In an N-body simulation where forces between particles are attractive and particles do not lose energy on colliding with walls or each other, should energy be conserved? How could it be, with total kinetic energy increasing to unlimited values and total potential energy capped by the number of particles and their closest allowed proximity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm not sure if this is the correct forum to ask this, as it doesn't question the implementation mechanism itself. Feel free to tell me to repost it elsewhere.)&lt;/p&gt;&#xA;" OwnerUserId="21658" LastActivityDate="2016-10-17T18:55:41.643" Title="Should energy be conserved in an N-body simulation where particles don't lose energy in collisions?" Tags="|simulation|computational-physics|conservation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25220" PostTypeId="2" ParentId="25219" CreationDate="2016-10-17T11:39:33.887" Score="1" Body="&lt;p&gt;Think of the collisions between particles or walls as being modeled by a potential energy term of the form:&#xA;$$U(r) = \begin{cases}&#xA;  0, &amp;amp; \text{if $r &amp;gt; r_c$}, \\&#xA;  +\infty, &amp;amp; \text{if $r \le r_c$},&#xA;\end{cases}$$&#xA;where $r_c$ is the radius of a particle (or wall). You can see that the energy goes to infinity, compensating the attractive forces. Another way of looking at the problem is to consider a sequence $U_n$ of smooth approximations to $U$ and to see that conservation of energy holds for every one of those approximations.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2016-10-17T11:39:33.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25221" PostTypeId="2" ParentId="25218" CreationDate="2016-10-17T11:43:35.190" Score="6" Body="&lt;p&gt;Actuallly you can scale the variable of the problem into a different interval. Transform $ x \in [0,a]$ where ($a=10^6$) to $\xi \in [0,1]$ by the formula $ \xi = \frac{x}{a} $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, using the chain rule we have $\frac{d y }{ d x } = \frac{d y}{ d \xi } \frac{d \xi }{ d x } = \frac{ 1}{ a } \frac{d y}{ d \xi } $. Hence, replacing $\frac{d y }{ d x }$ by $\frac{ 1}{ a } \frac{d y}{ d \xi } $ in your equation, you can solve a new prolem of $y = y(\xi)$ in a shorter duration, say $[0,1]$.&lt;/p&gt;&#xA;" OwnerUserId="15175" LastEditorUserId="9667" LastEditDate="2016-10-17T15:18:37.333" LastActivityDate="2016-10-17T15:18:37.333" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25222" PostTypeId="1" CreationDate="2016-10-17T12:37:06.970" Score="10" ViewCount="2475" Body="&lt;p&gt;Basically, FEM seems to be a problem that is pretty much &quot;solved&quot;. There are numerous powerful frameworks existing, like Trilinos, PETSc, FEniCS, Libmesh or MOOSE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing they have in common: They are extremely &quot;heavy-weight&quot;. First, the installation normally is super painful. Second, their interface/API is thick and heavy - you have to translate your whole idea into the thinking of the respective library. That also means, interoperability and extendability for special requirements or existing code is difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other projects like (random examples) Boost, LibIGL, Aztec (linear solver), Eigen or CGAL demonstrate that it's absolutely possible to write powerful libraries that seamlessly integrate into C++ or Python code, with a very lean and clean interface, without need of installation of a super heavy framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a really lightweight package for FEM?&lt;/strong&gt; I'm not looking for the easy, automagic solver - I'm looking for a library that offers powerful functions while maintaining a lean interface, interoperability with common datastructures (C++ STL for example) and lightweight installation (header only for example).&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2018-07-15T14:08:10.060" Title="Are there any &quot;light-weight&quot; FEM packages around?" Tags="|finite-element|python|c++|petsc|fenics|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="25223" PostTypeId="2" ParentId="20786" CreationDate="2016-10-17T13:52:02.030" Score="2" Body="&lt;p&gt;MFEM [1] has easy-to-use quadrature functions (both for surfacic and volumetric elements). We were able to use them for various tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://mfem.org/&quot; rel=&quot;nofollow&quot;&gt;http://mfem.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-10-17T13:52:02.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25224" PostTypeId="2" ParentId="25067" CreationDate="2016-10-17T14:00:38.453" Score="0" Body="&lt;p&gt;You would be satisfied by the finite volume method (FVM) with three-time level scheme for unsteady term, which has second order accuracy and does not require any condition for stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solving procedure should go with the deferred correction method, which allows us to correct solution step by step inside an iteration. Upwind scheme blending with the central difference can be applied to the convection term; it also has second order accuracy. The diffusion term with a non-constant $\kappa (\vec{r})$ can be handled by &lt;a href=&quot;http://powerlab.fsb.hr/ped/kturbo/OpenFOAM/docs/HrvojeJasakPhD.pdf&quot; rel=&quot;nofollow&quot;&gt;Hrvoje Jasak PhD Thesis, p.83&lt;/a&gt;. For a generic FVM, we can find more useful things within this thesis.&lt;/p&gt;&#xA;" OwnerUserId="15175" LastActivityDate="2016-10-17T14:00:38.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25225" PostTypeId="2" ParentId="25219" CreationDate="2016-10-17T18:55:41.643" Score="1" Body="&lt;p&gt;The numerical integration scheme might not be energy-preserving.  The error term is often strictly positive, allowing for non-secular growth of the energy.  There are some ways around this, such as conservative Runge-Kutta integrators.&lt;/p&gt;&#xA;" OwnerUserId="3182" LastActivityDate="2016-10-17T18:55:41.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25226" PostTypeId="2" ParentId="25218" CreationDate="2016-10-17T19:57:22.397" Score="4" Body="&lt;p&gt;@tqviet already says this in a some way, but let me phrase it differently: $10^6$ is not a large end time by itself. It seems like a large number because it has 6 zeros at the end, but it all depends on (i) which units you compute it, (ii) how fast the effects are that you are trying to model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give examples: If I told you that you had to compute to $10^6$ picoseconds, you would probably not think of that as a long time -- it is, after all, just $10^{-6}$ seconds. But what really matters is how long that is &lt;em&gt;in relation to the things you try to describe&lt;/em&gt;. Even $10^6$ years is not very much if you try to simulate the evolution of a galaxy -- these things happen on time scales of billions of years, so $10^6$ could probably be done in one or just a few time steps. On the other hand, if you try to simulate the weather -- which works on time scales of hours -- then $10^6$ years clearly is a very very long time. Indeed, if your goal were to simulate the motion of electrons in semiconductors -- which works on time scales of nanoseconds or even less -- then $10^6$ years is an &lt;em&gt;even longer&lt;/em&gt; time interval. In other words, it really depends on what you are looking at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads to the last observation: You are using time steps of size $0.01$. But why? Because you think that time steps need to be small (so significantly smaller than 1, in whatever unit you are using), or because the dynamics of your system happen on time scales of 0.01 time units? In the latter case, the choice is clearly valid. But if the dynamics of your system happen on time scales of ~100 or ~1000 or even more time units, then there is no need for such small time steps: you should be able to use $\Delta t=10, 100$, or even more. In that case, you will be at your end time very quickly indeed.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-17T19:57:22.397" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25228" PostTypeId="1" AcceptedAnswerId="25262" CreationDate="2016-10-17T20:27:25.103" Score="2" ViewCount="76" Body="&lt;p&gt;I want to vary the input parameter of a physical dynamic mechanics problem, as a Gaussian Random variable and view the resulting Probability Density Function (PDF). I used the Finite Element Method to create deterministic codes in C++ and Python language. However, before I go there, I want to begin by applying this technique to a simple Poisson problem deterministic FEM code written in python or C++. I have mentioned my thoughts below. If you have done this before, please, help me through your own approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a Poisson problem, I need to vary &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, i.e., the &lt;em&gt;spatial variability&lt;/em&gt; parameter, as an RV with some statistical distribution. Let's assume a mean of 1 and a std. deviation of 0.3, or any other convenient value. Now, I want to run this code multiple times for different values of the parameter - &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. So, let's say, I generate an array of 100 values using &lt;code&gt;randn()&lt;/code&gt; function of Matlab/python. Now, maybe a shell script or python script is needed to use this array with a &lt;code&gt;for&lt;/code&gt;-loop and run the FEM code 100 times for each value of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. How will the deterministic FEM code be mentioned in the loop? How will it accept those values of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;? Does anyone have a generic script that can help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also really appreciate some suggestions/ comments on post-processing. I believe the output &lt;code&gt;.vtk&lt;/code&gt; or &lt;code&gt;.pvd&lt;/code&gt; format files can be used to create PDF in Paraview software. I am also a little confused about the generation of PDF. I believe I'll have to choose a single point on the 2D mesh and for each point, I have output values in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; direction, since it is a 2D problem. Therefore, I'll have to choose 1 direction too. So, the PDF of output on 1 point, in 1 direction, is what I am looking for. Is that correct? I am not sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I basically want to do a non-intrusive polynomial chaos expansion using quadrature or sampling. However, as stated above, I am starting with Monte Carlo, taking a step at a time, so I understand how to do pre and post-processing.&lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="20688" LastEditDate="2019-07-05T06:09:40.017" LastActivityDate="2019-07-05T06:09:40.017" Title="Using physical parameter as a Gaussian random variable in a simple Poisson problem" Tags="|finite-element|statistics|random-number-generation|probability|paraview|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25229" PostTypeId="2" ParentId="25212" CreationDate="2016-10-17T20:35:09.863" Score="1" Body="&lt;p&gt;Software could never get rid of all the effects of distortion. The reason is that, if you do not operate in the hardware level, you could only make approximations about the real effects. Distortion models such as pin-cushion, barrel and etc are only the effects which we could model.&#xA;The distortion removal techniques, which are based on some kind of supervised learning might do a better job, but even what they learn is an approximating function and not the actual physical model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said those, software could do a pretty descent job of handling distortion. Yet, I'm afraid that 180 is significant. Of course, on the other side, you would have the stereo stitching artifacts. The choice is dependent on which one you care about more. Stereo would probably have less distortion related issues, but the calibration errors and seams will degrade the image quality. It is best to try out for the particular choice of a wide angle and stereo cameras.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2016-10-17T20:35:09.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25230" PostTypeId="1" CreationDate="2016-10-17T23:48:40.447" Score="1" ViewCount="52" Body="&lt;p&gt;So I'm working with a rather complicated dynamical system. Instead of writing it all out. It's probably easier if you just clone my git repository.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;git clone https://gitlab.com/mdornfe1/vortex_acoustic&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The dynamics of the system is encoded in the function flow inside vortex_acoustic.py. If I integrate the system from rest I find oscillations for certain values of the parameters. The system is 13 dimensional, but here is a plot of the first variable vs. time&#xA;&lt;a href=&quot;https://i.stack.imgur.com/pyvjj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pyvjj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it seems like the system is approaching a limit cycle attractor. From my understanding of dynamical systems there should be an unstable fixed point inside that limit cycle. I want to try to calculate it so I can analyze its stability. My idea is that the fixed point should be close to the mean of the oscillating quantities, so I use that as a starting value for a numerical root solver. Here's the code for finding the fixed point, but you can also just run stack_exchange.py.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pickle, numpy as np, matplotlib.pyplot as plt&#xA;import vortex_acoustic as va&#xA;from scipy import optimize&#xA;from constants import *&#xA;&#xA;T = 5000&#xA;dt = 0.05&#xA;tn_transients = int(0.9 * T / dt)&#xA;&#xA;def calc_fixed_point(seed, params):&#xA;    return optimize.fsolve(va.flow_star, seed, params)&#xA;&#xA;def calc_closest_fixed_point(y, params):&#xA;    seed = np.mean(y[tn_transients:, :], 0)&#xA;    seed[Nq+1:2*Nq+1] = 0&#xA;    y_star = calc_fixed_point(seed, params)&#xA;&#xA;    return y_star, seed&#xA;&#xA;y = np.load('simulation.npy')&#xA;with open('params.p', 'r') as f:&#xA;    params = pickle.load(f)&#xA;&#xA;plt.plot(y[:,0])&#xA;&#xA;y_star, seed = calc_closest_fixed_point(y, params)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The function calc_closest_fixed_point takes the output of the simulation and the params passed to the simulation. It calculates the mean of the fluctuating quantities after the transients has died out. It sets the derivatives to 0 and uses that quantity seed as a starting point for fsolve. The function calc_fixed_point then finds the fixed point of the system y_star. I thought this approach would show that seed and y_star are very close to each other. But for this example I'm seeing they're quite different. Furthermore the actual value isn't inside the limit cycle.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(seed)&#xA;&amp;gt;&amp;gt;&amp;gt;array([ 0.18307736, -0.11207637,  0.00382286,  0.        ,  0.        ,&#xA;       -0.02016129,  0.08881281, -0.07852504,  0.07439292,  0.0109214 ,&#xA;       -0.00894938, -0.02896082,  0.00833759])&#xA;print(y_star)&#xA;&amp;gt;&amp;gt;&amp;gt;array([  1.06466138e+00,  -6.65605305e-01,   2.15920825e-02,&#xA;        -5.18569988e-25,   1.44087666e-24,   7.55288134e-03,&#xA;         5.17875853e-01,  -4.41152924e-01,   4.72080951e-01,&#xA;         4.86813549e-02,  -6.04678149e-02,  -1.42533599e-01,&#xA;         5.90028044e-02])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also seed is pretty obviously not a fixed point. If you calculate the function flow at seed the derivatives are pretty far from 0.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;va.flow(seed,0,*params)&#xA;&amp;gt;&amp;gt;&amp;gt;array([-0.00543102,  0.        ,  0.        , -0.00113827, -0.00042674,&#xA;       -0.00973662, -0.00775962,  0.01449168,  0.01369892,  0.00331889,&#xA;        0.00502761, -0.00026469, -0.00076228])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Anyone know what's going? I'm correct in assuming there is a fixed point inside the limit cycle right?&lt;/p&gt;&#xA;" OwnerUserId="18815" LastEditorUserId="18815" LastEditDate="2016-10-18T00:32:57.823" LastActivityDate="2016-10-18T00:32:57.823" Title="Calculating fixed point inside limit cycle" Tags="|roots|fixed-point|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25231" PostTypeId="1" CreationDate="2016-10-18T00:01:21.870" Score="0" ViewCount="248" Body="&lt;p&gt;I am trying to estimate a few unknown parameters of my continuous non-linear PDAE model (simulated through finite-volume method spatial discretisation, and time-stepping through method-of-lines). I am planning to do this by minimising the least-squares error of the model output to experimental data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read about the various optimisation solvers that can be used. However, I am having a weird situation. I do not have the exact analytical form of my function (since its a system of PDAES, which after discretisation yields a lot of DAEs).  While, this is fairly common and the recommended method is the method of lines, I have a slightly different situation. I do have the Jacobian matrix (evaluated through Automatic Differentiation package 'CASADI').  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to really take advantage of the power of the jacobian matrix, i.e. directional information towards solution. Furthermore, my function evaluation is very cheap, since my simulation model is hand-optimised and takes advantage of various intrinsic CPU instructions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the recommended software package for implementing a global optimisation for parameter estimation, given this situation ?&lt;/p&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2016-10-18T15:18:21.280" Title="Choice of solver/software for global optimisation of cheap black-box function with known derivatives" Tags="|optimization|pde|constrained-optimization|curve-fitting|dae|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25232" PostTypeId="2" ParentId="25231" CreationDate="2016-10-18T00:17:56.670" Score="0" Body="&lt;p&gt;You can either use some derivative free optimization, for which there are several packages available, or: If in addition to the jacobian you can obtain the hessian then there are many methods that you can write up. If you don't have constraints the newton method and steepest descent are very easy to write, and work rather well if you have a good method of estimating step size ( armijo conditions etc), constrained methods take more work to program. I recommend you the reference Numerical Optimization by J Nocedal, it contains pseudo code and references to software that would help you with your problem.&lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2016-10-18T00:17:56.670" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25233" PostTypeId="2" ParentId="25231" CreationDate="2016-10-18T01:11:24.610" Score="0" Body="&lt;p&gt;Let me clarify from my previous comment that I had linear least squares problems in mind, non linear least squares need to meet several conditions in order to be convex. I would look for some simple code for non linear least square problems (which are basically non linear methods that take advantage of the structure of the problem to reduce in computations, e.g.  Gauss-newton and levenberg-marquardt) and modify it to add the jacobian and hessian from your program.  You may need some understanding of the methods to do this. Most packages will expect that you provide the objective function.&lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2016-10-18T01:11:24.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25235" PostTypeId="1" AcceptedAnswerId="25240" CreationDate="2016-10-18T11:29:03.207" Score="2" ViewCount="360" Body="&lt;p&gt;I am trying to solve an elliptic PDE in 2-D: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\nabla^{2} u = 20tanh(10x-5)(10-10tanh(10x-5)^2) = f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the solution is $u = tanh(10x-5)$ but I am unable to get $O(h^2)$ solution with a cell-centered, finite-difference on a uniform mesh with Dirichlet boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discretization&lt;/strong&gt; : central difference &#xA;$$-[\frac{u_{i+1,j}-2u_{i,j}+u_{i-1,j}+u_{i,j+1}-2u_{i,j}+u_{i,j-1}}{h^{2}}] = f_{i,j}$$&#xA;$$u_{i,j} = \frac{u_{i+1,j}+u_{i-1,j}+u_{i,j+1}+u_{i,j-1}+f_{i,j}h^{2}}{4.0} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Boundary conditions&lt;/strong&gt; : X-direction is from top to bottom, Y-direction is from left to right. Since I know the analytical solution : $u = tanh(10x-5)$, I substitute the value of $u$ for various discretised points at the boundary. Then at each update of $u$, I re-adjust the ghost cell as : $$\frac{u_{0,j}+u_{1,j}}{2.0} = tanh(10*0-5)$$ at the lower-X boundary (and similar expressions for high-X,low-Y, high-Y).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Plot&lt;/strong&gt;: In 2-D/3-D I get a plot of the solution &lt;em&gt;similar&lt;/em&gt; to the figure shown below. &lt;em&gt;Please ignore the Adaptive Mesh&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u6B2g.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u6B2g.jpg&quot; alt=&quot;Solution Plot is *similar* to this for above PDE &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Code&lt;/strong&gt;: using &lt;em&gt;Gauss-Seidel&lt;/em&gt; iterative scheme. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double update(double phi[][N+2],double rhs[][N+2], int Nx, int Ny, double dx, double kp )&#xA;{&#xA;int i,j;&#xA;double abs_err = 0.0, err, x; &#xA;&#xA;for(i = 1; i &amp;lt;= Nx-2; i++)&#xA;{&#xA;    x = (i-0.5)*dx ; &#xA;    for(j = 1; j &amp;lt;= Ny-2; j++ )&#xA;    {&#xA;        phi[i][j] = (phi[i+1][j] + phi[i-1][j] + phi[i][j-1] + phi[i][j+1] + dx*dx*rhs[i][j])/4.0 ; &#xA;        err = tanh(kp*(x-0.5)) - phi[i][j];&#xA;        if(fabs(err) &amp;gt; abs_err)&#xA;            abs_err = fabs(err); &#xA;    }&#xA;}       &#xA;return abs_err ; &#xA;}&#xA;&#xA;void boundary(double phi[][N+2],int Nx,int Ny,double dx, double kp)&#xA;{&#xA;int i,j;&#xA;double x,y; &#xA;&#xA;//low-y boundary i.e. y = 0     &#xA;for(i = 1; i &amp;lt;= N+2-2; i++)&#xA;{&#xA;      x = (i-0.5)*dx ; &#xA;      phi[i][0]      = 2 * tanh(kp*(x-0.5)) - phi[i][1]; &#xA;}   &#xA;//high-y boundary i.e. y = 1    &#xA;for(i = 1; i &amp;lt;= N+2-2; i++)&#xA;{&#xA;    x = (i-0.5)*dx ; &#xA;      phi[i][N+1]    = 2 * tanh(kp*(x-0.5)) - phi[i][N];        &#xA;}   &#xA;//low-x boundary i.e. x = 0&#xA;for(j = 1; j &amp;lt;= N+2-2; j++)&#xA;{        &#xA;      phi[0][j]    =     2 * tanh(kp*(0.0-0.5)) - phi[1][j];&#xA;&#xA;}   &#xA;//high-x boundary i.e. x = 1    &#xA;for(j = 1; j &amp;lt;= N+2-2; j++)&#xA;{       &#xA;      phi[N+1][j]    =   2 * tanh(kp*(1.0-0.5)) - phi[N][j]; &#xA;&#xA;}   &#xA;}&#xA;&#xA;void main()&#xA;{&#xA;double phi[N+2][N+2] ;                  //solution array with ghost boundaries&#xA;double rhs[N+2][N+2];                   //RHS i.e. -del^{2}u = RHS&#xA;&#xA;double prob_x[2] = {0.0, 1.0};                  //X-direction problem domain&#xA;double prob_y[2] = {0.0, 1.0};                  //Y-axis problem domain&#xA;double x,y, kp = 10.0, abs_err; &#xA;&#xA;double dx, dy ;                                 //Mesh spacing&#xA;dx = dy = (prob_x[1] - prob_x[0])/N ; &#xA;&#xA;int i,j,ctr; &#xA;&#xA;for(i = 0; i &amp;lt;= N+2-1; i++)&#xA;    for(j=0; j &amp;lt;= N+2-1; j++)&#xA;        {&#xA;            phi[i][j] = 0.0;&#xA;            rhs[i][j] = 0.0; &#xA;        }&#xA;&#xA;for(i = 1; i &amp;lt;= N; i++)&#xA;{&#xA;    x = (i-0.5)*dx ; &#xA;    for(j = 1; j&amp;lt;= N; j++)&#xA;    rhs[i][j] = 20 * tanh(10.0*x-5.0) * (10.0 - 10.0 * tanh( (10.0*x-5.0) * (10.0*x-5.0) ) ) ; &#xA;}&#xA;&#xA;boundary(phi,N+2,N+2,dx,kp);&#xA;&#xA;ctr = 1; &#xA;abs_err = 1.0; &#xA;&#xA;//while(abs_err &amp;gt; dx*dx)&#xA;for(ctr = 1; ctr &amp;lt;= 50000; ctr++)&#xA;{&#xA;&#xA;    abs_err = update(phi,rhs,N+2,N+2,dx,kp);                //Updates phi &#xA;    boundary(phi,N+2,N+2,dx,kp); &#xA;    //ctr = ctr+1; &#xA;}   &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Convergence&lt;/strong&gt; : I don't know the order of convergence I am getting. Sorry I don't know how to calculate that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do get solutions close to the analytic solutions but not the order $O(h^2)$ ($h$ = mesh points spacing) convergence. Can anyone shed some light on this ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDITS again&lt;/strong&gt;&#xA;&lt;strong&gt;Convergence Rate&lt;/strong&gt;: Thanks for all the answers and helpful comments. I defined $r_{0} = ||\bar{x} - x^{(0)}||_{2}$ and $r_{k} = ||\bar{x} - x^{(k)}||_{2}$ where $\bar{x}$ is the actual analytical solution and $x^{k}$ is calculated solution at the $k^{th}$ iteration. I agree that properly the stopping criterion should be the relative norm of residuals but here I took the norm of error measures from the analytical solution. I am finally getting these values.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mesh size : 4x4, $\frac{r_{k}}{r_{0}} = 0.31$&lt;/li&gt;&#xA;&lt;li&gt;Mesh size : 8x8, $\frac{r_{k}}{r_{0}} = 0.092459$&lt;/li&gt;&#xA;&lt;li&gt;Mesh size : 16x16, $\frac{r_{k}}{r_{0}} = 0.01122066$&lt;/li&gt;&#xA;&lt;li&gt;Mesh size : 32x32, $\frac{r_{k}}{r_{0}} = 0.0026179$&lt;/li&gt;&#xA;&lt;li&gt;Mesh size : 64x64, $\frac{r_{k}}{r_{0}} = 0.000646645$&lt;/li&gt;&#xA;&lt;li&gt;Mesh size : 128x128, $\frac{r_{k}}{r_{0}} = 0.00016118577$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think this qualifies for a quadratic convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mistake&lt;/strong&gt; : I made a mistake in the expression for RHS which I had obtained from doubly differentiating $u = tanh(10x-5)$. I used &lt;em&gt;Maple&lt;/em&gt; for it (first time user) and I did not know that Maple writes $tanh^{2}(10x-5)$ as $tanh(10x-5)^{2}$. Hence the correct PDE is :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\nabla^{2} u = 20tanh(10x-5)(10-10tanh(10x-5)tanh(10x-5)) = f$$&lt;/p&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="14303" LastEditDate="2016-10-22T10:43:04.633" LastActivityDate="2016-10-22T10:43:04.633" Title="$O(h^2)$ convergence for Elliptic PDE" Tags="|finite-difference|boundary-conditions|convergence|elliptic-pde|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25236" PostTypeId="2" ParentId="25231" CreationDate="2016-10-18T15:18:21.280" Score="2" Body="&lt;p&gt;Since you have the Jacobian matrix, you can apply it within a Gauss-Newton or Levenberg-Marquardt method to effectively approximate the Hessian and gradient of your least squares objective function (the gradient is $J^{T}f$, and the Hessian is to first order $J^{T}J$.)   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also use the Jacobian to compute the gradient of your least squares objective functions and then solve the problem with any gradient based scheme (e.g. BFGS or a limited memory BFGS method.)  This might be a better approach if the number of parameters to be estimated is very large and you don't have sufficient space to deal with solving a system of equations involving $J^{T}J$.  An alternative is to use an iterative method to solve the normal equations that arise in the GN or LM methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of these methods can (with appropriate attention to detail) be guaranteed to converge to a local minimum of the least squares objective from any starting solution.  This is sometimes confusingly called &quot;global convergence&quot;, even though it doesn't mean convergence to a global minimum.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that your least squares objective is non-convex (most likely), then you'll want to consider combining a local nonlinear least squares solver with some kind of stochastic search in hopes of finding a global minimum.  The simplest approach would be to use &quot;multi-start&quot;, in which you run a local search from lots of randomly chosen solutions, and pick the best local minimum that you see.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-10-18T15:18:21.280" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25238" PostTypeId="2" ParentId="25209" CreationDate="2016-10-18T18:05:01.670" Score="0" Body="&lt;p&gt;You need to look at this a bit more systematically. In essence, you're not wrong in your approach, but it gets more complicated if you want to do this in 2d/3d, or if you have nonzero boundary values. My take on these issues is in lectures 21.5 and following here: &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-18T18:05:01.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25239" PostTypeId="2" ParentId="25235" CreationDate="2016-10-18T21:02:35.747" Score="2" Body="&lt;p&gt;I think if you are interested to find the order of convergence then you should vary the mesh spacing ($h$). Now as you vary the mesh spacing you can obtain the error of the numerical solution. Then if you do a log plot of the error and $h$, you should get a straight line with slope $2$ I guess. And that is how you can estimate the order of convergence, quadratic in this case I presume.&lt;/p&gt;&#xA;" OwnerUserId="18005" LastActivityDate="2016-10-18T21:02:35.747" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25240" PostTypeId="2" ParentId="25235" CreationDate="2016-10-18T21:57:55.867" Score="4" Body="&lt;p&gt;You appear (and correct me if I am wrong, i do not wish to presume) to be confused on the notion of what is an order of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The order of convergence is the order at which your approximate solution (in this case, your simulation) approaches the theoretical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the theoretical order of convergence of the centered scheme is 2. This means that if you refine your mesh by a factor of two, the error on the approximate solution, which is O(h^2) will decrease by a factor of 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, defining the error is not trivial. Generally we use the L2 norm for the error (&lt;a href=&quot;http://mathworld.wolfram.com/L2-Norm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mathworld.wolfram.com/L2-Norm.html&lt;/a&gt;) for continous problems (for a discontinous problem such as a shock wave, the L1 or Linf norm are also often used).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concretly, to show that you get second order convergence, you need to calculate the L2 error between your numerical solution and the analytical solution for a number of meshes (say 10x10x10, 20x20x20, 30x30x30, etc.) and then carry out a linear regression such that (log(error) = slope * log (dx)) where dx is the mesh spacing. The slope will be the order of convergence of your implemented scheme, and in your case should be 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a graph should look like this (taken from a paper by me):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/f0CLD.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f0CLD.jpg&quot; alt=&quot;Order of convergence analysis&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2016-10-19T14:38:24.207" LastActivityDate="2016-10-19T14:38:24.207" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25241" PostTypeId="2" ParentId="12967" CreationDate="2016-10-19T00:38:00.960" Score="1" Body="&lt;p&gt;You can formulate this at the purely linear algebra level in terms of csr matrix multplications and a matrix addition, which are fast. I have found this to be the fastest and most memory efficient method in scipy on my laptop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, let $M$ be the original matrix, and $M'$ be the modified version with rows and columns associated with boundary nodes zeroed out, except for the diagonal entry in which the value of 1 is placed. Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M' = I_\text{interior} ~ M ~ I_\text{interior} + I_\text{boundary},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $I_\text{interior}$ is the diagonal matrix with ones in locations corresponding to interior degrees of freedom and zeros elsewhere, and contrarily $I_\text{boundary}$ is the diagonal matrix with ones in the locations corresponding to the boundary degrees of freedom and zeros elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the overall strategy is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Form the diagonal matrices $I_\text{interior}$ and $I_\text{boundary}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Convert $I_\text{interior}$ and $I_\text{boundary}$ to CSR format.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Form the modified matrix $M' = I_\text{interior} ~ M ~ I_\text{interior} + I_\text{boundary},$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is the code that I use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def dirichlet_zero_matrix_modification(matrix, dirichlet_zero_dofs):&#xA;    &quot;&quot;&quot;Enforces dirichlet zero B.C.'s by zeroing-out rows and columns &#xA;    associated with dirichlet dofs, and putting 1 on the diagonal there.&quot;&quot;&quot;&#xA;    N = matrix.shape[1]&#xA;&#xA;    chi_interior = np.ones(N)&#xA;    chi_interior[dirichlet_zero_dofs] = 0.0&#xA;    I_interior = sps.spdiags(chi_interior, [0], N, N).tocsr()&#xA;&#xA;    chi_boundary = np.zeros(N)&#xA;    chi_boundary[dirichlet_zero_dofs] = 1.0&#xA;    I_boundary = sps.spdiags(chi_boundary, [0], N, N).tocsr()&#xA;&#xA;    matrix_modified = I_interior * matrix * I_interior + I_boundary&#xA;    return matrix_modified&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my experience with scipy, expressing matrix manipulations in terms of higher-level linear algebraic operations (where possible) almost always yields better performance, compared to other methods that actually manipulate matrix entries or slices. In other languages and frameworks this may not be the case.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2016-10-19T00:38:00.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25243" PostTypeId="1" CreationDate="2016-10-19T03:50:18.410" Score="1" ViewCount="368" Body="&lt;p&gt;I am trying to numerically solve a PDE, and just had a question as to the validity of a certain approach.  For example, given the PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\partial ^2 E}{\partial t^2} = - k\frac {\partial E}{\partial t} + c^2 \frac {\partial ^2 E}{\partial z^2} -  c\frac {\partial E}{\partial z}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I was to apply an explicit Runge-Kutta method, I could make a substitution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u = \frac {\partial E}{\partial t} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and solve the following instead:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\partial u}{\partial t} = - ku + c^2 \frac {\partial ^2 E}{\partial z^2} -  c\frac {\partial E}{\partial z}&#xA;\\ \\&#xA;u = \frac {\partial E}{\partial t} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the spatial derivatives are computed by a finite difference approximation (e.g. central difference). I was just wondering: is solving the following a valid approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\partial E}{\partial t} = \frac{1}{k}(-\frac {\partial ^2 E}{\partial t^2}  + c^2 \frac {\partial ^2 E}{\partial z^2} -  c\frac {\partial E}{\partial z})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where now both the spatial derivatives and second time derivative are approximated by central difference schemes, while I apply the Runge-Kutta method to the first time derivative of $E$. I have not seen this done anywhere, but was just curious as to why (or why not) this is an unacceptable method? Mathematically, it seems odd that my first order derivative depends on higher order derivatives of itself, although I don't immediately see any reason why this is wrong numerically. &lt;/p&gt;&#xA;" OwnerUserId="20407" LastActivityDate="2017-04-23T15:50:48.717" Title="Solving first versus second order PDE" Tags="|pde|finite-difference|runge-kutta|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25244" PostTypeId="1" AcceptedAnswerId="25259" CreationDate="2016-10-19T05:21:27.067" Score="0" ViewCount="115" Body="&lt;p&gt;I have a convex optimization problem as follows: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;maximize_{x\in R^n}   &amp;amp;\sum_{i=1}^n a_i log(x_i)\\&#xA;  st\quad &amp;amp; \sum_{i=1}^n p_{ij} (x_i-1)  = 0 \quad \forall j\\&#xA;               &amp;amp; x_i \geq 0          &#xA;\end{align*}&#xA;where, $a_i \geq 0$ and $P$ is an $n\times m$ probability matrix with rows sum up to 1.&#xA;Is there any efficient and fast algorithm to solve it? (I suppose it doesn't have a closed-form solution)&lt;/p&gt;&#xA;" OwnerUserId="22036" LastActivityDate="2016-10-19T22:35:59.590" Title="An efficient algorithm to solve this convex optimization problem" Tags="|convex-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25245" PostTypeId="2" ParentId="25110" CreationDate="2016-10-19T09:36:37.030" Score="5" Body="&lt;p&gt;To extend Chris Rackauckas's exhaustive answer with a reference try to look &lt;a href=&quot;http://www.infor.uva.es/~diego/docs/torres11.pdf&quot; rel=&quot;noreferrer&quot;&gt;pdf&lt;/a&gt; by Torres, Gonzalez-Escribano, Llanos. It is about the tuning of a gpu, that is an important aspect for performance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Bill Greene's comment remembers the most relevant part of the computation work is about solve linear system, but however the assembly&#xA;part can take a discrete amount of time. In this side gpu is a good&#xA;help because this part has got a naturally high parallelism. You can&#xA;calculate every element independent and at the end built the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(For my master thesis in mathematics I developed a parallel version with CUDA of a CFD solver write in matlab)&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2016-10-19T09:36:37.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25246" PostTypeId="1" AcceptedAnswerId="25248" CreationDate="2016-10-19T09:36:37.740" Score="1" ViewCount="355" Body="&lt;p&gt;I am reading the paper, &lt;a href=&quot;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&quot; rel=&quot;nofollow&quot;&gt;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says applying the divergence to both sides of this equation&#xA;$$\frac{1}{\Delta t} U^{n+1} - \frac{1}{\Delta t} U^{n} =  - \nabla P^{n+1}  $$&#xA;yields the linear system,&#xA;$$-\Delta P^{n+1} = - \frac{1}{\Delta t} U^{n} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does the $\frac{1}{\Delta t} U^{n+1}$ term vanish by taking the divergence?&lt;/p&gt;&#xA;" OwnerUserId="14387" LastEditorUserId="549" LastEditDate="2016-10-19T17:57:53.530" LastActivityDate="2016-10-19T17:57:53.530" Title="The pressure correction equation in Chorin's Projection Method for the Navier-Stokes equation" Tags="|navier-stokes|incompressible|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25247" PostTypeId="2" ParentId="25222" CreationDate="2016-10-19T10:00:25.807" Score="4" Body="&lt;p&gt;I think you have some confusion. PETSc is not in the same league as Fenics, Libmesh, Moose etc. In fact, all of these (heavyweight) packages use PETSc for linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMHO PETSc is as lightweight as you can get. It just requires C/Fortran compilers and Python (used only for configuration) and you can build the library in under 5 minutes on your laptop. Also, the most complicated part of a FE code is parallel assembly and solve and PETSc takes care of both. The rest (e.g., element level calculations) is rather straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trillinos, OTOH is much more than a linear algebra framework, e.g., Aztec (linear solver) which you mention is part of it. In some ways Aztec in Trillinos can be compared to PETSc.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2016-10-19T10:00:25.807" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25248" PostTypeId="2" ParentId="25246" CreationDate="2016-10-19T11:12:33.147" Score="2" Body="&lt;p&gt;It is a consequence of the continuity condition ($\nabla \cdot u = 0$), condition (3) at page 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At chapter 4 at the beginning there is the assumption about the condition&#xA;(3) is valid. You are at the $\textit{n+1}$, step and so &#xA;$$\nabla \cdot U^{n+1} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This method is in the family of projection method. In this class of methods the pressure and velocity problems are decouple. &#xA;The intermediate velocity does not respect the continuity condition.&#xA;The method use the pressure to impose the continuity condition,&#xA;i.e. the pressure is used to project the intermediate velocity onto a space of divergence-free velocity field. So when you calculate the pressure use $\nabla \cdot U^{n+1} = 0$.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="22025" LastEditDate="2016-10-19T13:39:49.677" LastActivityDate="2016-10-19T13:39:49.677" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25249" PostTypeId="1" CreationDate="2016-10-19T11:38:22.173" Score="2" ViewCount="467" Body="&lt;p&gt;I have the following problem, there is an objective function f() depending on 7 variables x=(x1,x2,...x7), so f(x)=f((x1,x2,...x7)) and I want to find the combination of variables that minimize the objective function (x* for which f(x*) is minimum). The formula of the objective function is unknown. The problem is that the evaluation of objective function for a given point x, f(x), is difficult and requires running of a simulation software that can take time and computational resources so I want to do as few estimations of f() as possible. Which method/algorithm do you think would be most suitable here? What about Monte Carlo stochastic optimization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will add here more details about the objective function - this is a result of a simulation software called MIKE11 that models flow of water in rivers. The model of the river is 1D and the user's manual says that the equations used for the hydrodynamic part are Saint Venant equations. Now the objective function represents the maximum flow (in cubic meters per second) on a point on the river. And I want to minimize the objetive function (so I want the minimum of the maximum flow values). So to estimate the objetive function for the x vector of parameters I have to run one MIKE simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="21936" LastEditorUserId="6910" LastEditDate="2016-10-20T13:06:38.703" LastActivityDate="2016-10-20T13:06:38.703" Title="optimization problem. Monte Carlo stochastic method or another one?" Tags="|optimization|monte-carlo|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25250" PostTypeId="1" AcceptedAnswerId="25258" CreationDate="2016-10-19T11:39:00.037" Score="1" ViewCount="2129" Body="&lt;p&gt;I am reading the paper, &lt;a href=&quot;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&quot; rel=&quot;nofollow&quot;&gt;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could not really understand the description. Could someone explain a little bit more?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says,&#xA;&quot;For the lid driven cavity problem this means that homogeneous Neumann boundary conditions are prescribe everywhere. This implies in particular that the pressure $P$ is only defined up to a constant, which is fine, since only the gradient of $P$ enters the momentum equation.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does it mean &quot;only defined up to a constant&quot;? By solving the Poisson equation, you obtain the values of $P$ and can calculate its gradient. The constant means this pressure value? And if it's not &quot;only up to constant&quot;, what do we get? derivative or integral?&lt;/p&gt;&#xA;" OwnerUserId="14387" LastActivityDate="2016-10-19T22:24:02.497" Title="Boundary condition for Pressure in Navier-Stokes equation" Tags="|boundary-conditions|navier-stokes|incompressible|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25251" PostTypeId="2" ParentId="25222" CreationDate="2016-10-19T12:07:34.110" Score="10" Body="&lt;p&gt;I've been developing a lightweight finite element library in Python 2.7 harnessing the power of NumPy arrays and SciPy sparse matrices. The general idea is that given a mesh and a finite element, you have more-or-less one-to-one correspondence between the bilinear form and a (sparse) matrix. The user can then use the resulting matrix as he or she sees fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me present a canonical example where we solve the Poisson equation in a unit square with an unit loading.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from spfem.mesh import MeshTri&#xA;from spfem.asm import AssemblerElement&#xA;from spfem.element import ElementTriP1&#xA;from spfem.utils import direct&#xA;&#xA;# Create a triangular mesh. By default, the unit square is meshed.&#xA;m=MeshTri()&#xA;&#xA;# Refine the mesh six times by splitting each triangle into four&#xA;# subtriangles repeatedly.&#xA;m.refine(6)&#xA;&#xA;# Combine the mesh and a type of finite element to create&#xA;# an assembler. By default, an affine mapping is used.&#xA;a=AssemblerElement(m,ElementTriP1())&#xA;&#xA;# Assemble the bilinear and linear forms. The former outputs&#xA;# a SciPy csr_matrix and the latter outputs linear NumPy array.&#xA;A=a.iasm(lambda du,dv: du[0]*dv[0]+du[1]*dv[1])&#xA;b=a.iasm(lambda v: 1.0*v)&#xA;&#xA;# Solve the linear system in interior nodes using&#xA;# a direct solution method provided by SciPy.&#xA;x=direct(A,b,I=m.interior_nodes())&#xA;&#xA;# Visualize the solution using Matplotlib.&#xA;m.plot3(x)&#xA;m.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Other comments:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My goal is to write rigorous convergence unit tests checking e.g. that theoretical convergence rates in the respective norms are obtained. Tests are run automatically on each change.&lt;/li&gt;&#xA;&lt;li&gt;Implementing new elements is quite easy.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can find the project &lt;a href=&quot;https://github.com/kinnala/sp.fem&quot; rel=&quot;nofollow noreferrer&quot;&gt;in GitHub&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python 3 version of the code can be found &lt;a href=&quot;https://github.com/kinnala/scikit-fem&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastEditorUserId="10160" LastEditDate="2018-07-15T14:08:10.060" LastActivityDate="2018-07-15T14:08:10.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25252" PostTypeId="2" ParentId="25222" CreationDate="2016-10-19T12:08:25.817" Score="4" Body="&lt;p&gt;I can recommend &lt;a href=&quot;http://www.nutils.org/about.html&quot; rel=&quot;nofollow&quot;&gt;&lt;em&gt;nutils&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;nutils&lt;/em&gt; meets at least a few your &quot;light-weight&quot; requirements.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it is pure python and easy to install since it only depends on standard Python libraries &lt;em&gt;numpy&lt;/em&gt;, &lt;em&gt;scipy&lt;/em&gt;, and &lt;em&gt;matplotlib&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;and, thus, it is well suited for interoperations. At least the developers claim that &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Exposed objects are of native python type or allow for easy conversion to leverage third party tools.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-10-19T12:08:25.817" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25253" PostTypeId="2" ParentId="25249" CreationDate="2016-10-19T12:18:45.413" Score="0" Body="&lt;p&gt;Gradient-based optimization is very simple to use (look at gradient descent and it's variations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Newton-type methods generally have faster convergence. Using approximations of second-order derivatives (Hesse matrix) results in quasi-Newton methods (for example BFGS) which aren't necessarily more expensive in terms of the number of function evaluations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have more details about the character of your function? People in PDE optimization for example often use the &quot;adjoint method&quot; to compute a gradient of the objective function at a cost that is only a constant factor of the evaluation of the cost of the original function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have the source code of your function available you can look into (adjoint) algorithmic differentiation or &quot;automatic&quot; differentiation to compute cheap gradients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do least squares regression (like machine learning models) these usually also use steepest descent based on gradients computed through adjoint models (called backpropagation in machine learning).&lt;/p&gt;&#xA;" OwnerUserId="22041" LastActivityDate="2016-10-19T12:18:45.413" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25255" PostTypeId="2" ParentId="25250" CreationDate="2016-10-19T13:25:21.437" Score="1" Body="&lt;p&gt;For the lid driven cavity problem, we apply the Dirichlet boundary condition (or, no-slip boundary condition) to the velocity field $\tt U$, that is, $\tt U = U_{\tt Lid}$ on the moving Lid and $\tt U=0$ on the rest of the boundary. Based on this, we should have ${\tt U}^{n+1} = {\tt U}^{n} $ on the boundaries. Hence, if we have $\tt U^{n+1} = U^{n} - \Delta t \nabla P ^{n+1}$ everywhere (as shown in the article) then &#xA;$$\tt \nabla P ^{n+1} = 0 $$ &#xA;must be hold on the boundaries (this is so-called the homogeneous Neumann boundary conditions). In addition, if a pressure field $\tt P$ satisfies the momentum equations then $\tt P + const$ also satisfies.&lt;/p&gt;&#xA;" OwnerUserId="15175" LastEditorUserId="15175" LastEditDate="2016-10-19T13:35:01.460" LastActivityDate="2016-10-19T13:35:01.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25256" PostTypeId="2" ParentId="25243" CreationDate="2016-10-19T13:39:34.563" Score="1" Body="&lt;p&gt;You can not approximate in your last equation the second time derivative by central difference, because you introduce the unknown discrete values, say $E^{n+1}$, that are not available when you compute $E^n$ in the $n$-th time step if you want to apply &lt;strong&gt;explicit&lt;/strong&gt; Runge-Kutta solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formally, you might use a second order accurate backward difference for the approximation of $\frac{\partial^2E}{\partial t^2}$, when you use the discrete values of $E$ at two previous time levels. Nevertheless, higher order Runge-Kutta methods require to evaluate the right hand side of your system at some intermediate time levels with variable (adaptive) time step, so then you must be able to approximate $\frac{\partial^2E}{\partial t^2}$ in such time points and I do not see a way how to do it with the required accuracy of higher, &lt;strong&gt;i.e. greater than 2&lt;/strong&gt;, order Runge-Kutta methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard approach as you described at the beginning of your question is appropriate, just write the second equation in suitable order, i.e.&#xA;$$&#xA;\frac{\partial E}{\partial t} = u &#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2017-03-24T15:35:53.837" LastActivityDate="2017-03-24T15:35:53.837" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25257" PostTypeId="2" ParentId="25249" CreationDate="2016-10-19T18:00:18.883" Score="1" Body="&lt;p&gt;You haven't mentioned this, but if your function evaluations are the results of a complicated simulation, is the simulation a Monte Carlo simulation or deterministic?  If the simulation is a Monte Carlo simulation, then your function values will be &quot;noisy&quot;, making the problem even more complicated.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For problems with a small number of parameters, expensive function evaluations, and particularly if there are noisy function values, methods that fit a surrogate model (aka a &quot;response surface&quot;) to the function are often your best bet.  You then minimize over the surrogate model using a conventional minimization algorithm.  You might also look into latin hypercube sampling.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-10-19T18:00:18.883" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25258" PostTypeId="2" ParentId="25250" CreationDate="2016-10-19T22:24:02.497" Score="6" Body="&lt;p&gt;When we say that the pressure is only defined &quot;up to a constant&quot;, what we mean is this: If $u,p$ is a solution to the Navier-Stokes equations, then $u,p+c$ for &lt;em&gt;any&lt;/em&gt; constant $c$ is &lt;em&gt;also&lt;/em&gt; a solution. In other words, the solution is not unique, but the non-uniqueness has a very particular structure: two solutions $u,p$ and $u,p'$ can only differ in a way where $p-p'=const$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-19T22:24:02.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25259" PostTypeId="2" ParentId="25244" CreationDate="2016-10-19T22:35:59.590" Score="1" Body="&lt;p&gt;If you flip the sign of the objective function and find the &lt;em&gt;minimum&lt;/em&gt;, you have a smooth, linearly constrained, convex optimization problem. Newton's method will rapidly converge to the solution for this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the equality constraints, you can either work with Lagrange multipliers, or you can eliminate the constraint by only working on the null space of the matrix $P$, assuming that it is easy to compute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the inequality constraints, you will have to use something like an active set method, or a penalty approach. Since these are only linear box constraints, there should be no difficulty here.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-19T22:35:59.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25260" PostTypeId="2" ParentId="12967" CreationDate="2016-10-20T08:32:24.630" Score="1" Body="&lt;p&gt;Although this does not directly answer to your question, let me provide a yet another simple approach to impose the essential boundary conditions that I use in &lt;a href=&quot;https://github.com/kinnala/sp.fem/blob/master/spfem/utils.py&quot; rel=&quot;nofollow&quot;&gt;my own FEM library&lt;/a&gt;. The following code is adapted to your use case. In the following &lt;em&gt;A&lt;/em&gt; is a csr_matrix and &lt;em&gt;b&lt;/em&gt; is the right hand side NumPy array.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.sparse.linalg as spl&#xA;import numpy as np&#xA;D=boundary&#xA;I=np.setdiff1d(np.arange(A.shape[0]),D)&#xA;x[I]=spl.spsolve(A[I].T[I].T,b[I]-A[I].T[D].T.dot(x[D]),use_umfpack=use_umfpack)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The advantage is that you do not have to modify the matrix. You simply drop unnecessary rows and columns and modify the right hand side which &lt;em&gt;seems&lt;/em&gt; to be rather quick.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2016-10-20T08:32:24.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25261" PostTypeId="2" ParentId="10940" CreationDate="2016-10-20T10:45:51.743" Score="4" Body="&lt;p&gt;I had the same problem and spoke with my supervisor. Here is his solution:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Use -B instead of B:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In my case, matrix B was negative definite (all eigenvalues are negative).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;λ(-B)x=(-A)x&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which will give the same result, but -B is positive definite and you can use scipy.sparse.linalg.eigs().&#xA;By trying this solution, an error in ARPACK arose for some matrix sizes :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;scipy.sparse.linalg.eigen.arpack.arpack.ArpackError: ARPACK error&#xA;  -9999: Could not build an Arnoldi factorization. IPARAM(5) returns the size of the current Arnoldi factorization. The user is advised to&#xA;  check that enough workspace and array storage has been allocated&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I dont know if that error is bound to my particular configuration and code or something else.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Other alternative for LinearOperator:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In my cases, I didnt have have to matrices A and B but only to the associate LinearOperators. I upgraded the solution of sebas for LinearOperators and it worked well, but it is slower than the first solution as it needs to invert B.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.sparse.linalg import LinearOperator&#xA;&#xA;N = your_size&#xA;&#xA;def A_fct(v):&#xA;    # write the function you need or just define matrix A&#xA;    return np.dot(A, v)&#xA;linop_A = LinearOperator( (N,N), matvec = A_fct)&#xA;&#xA;def B_fct(v):&#xA;    # write the function you need or just define matrix B&#xA;    return np.dot(B, v)&#xA;linop_B = LinearOperator( (N, N), matvec = B_fct)&#xA;&#xA;def Binv_A(v):&#xA;    retour = linop_A.matvec(v)&#xA;    sol, info = scipy.sparse.linalg.gmres(linop_B, retour)&#xA;    # or similar to gmres, it calculate inv(B) * A * v&#xA;    return sol&#xA;linop_Binv_A = LinearOperator( (N, N), matvec = Binv_A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22026" LastActivityDate="2016-10-20T10:45:51.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25262" PostTypeId="2" ParentId="25228" CreationDate="2016-10-20T18:15:00.483" Score="1" Body="&lt;p&gt;In the loop over your instances of 100 random values for $k$, you could write the value for $k$ into a file, and call your finite element code; the finite element code could then open the file, read the value for $k$ from it and then use it for whatever computation you may want to do with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for your second question, visualizing uncertainty is difficult. Why don't you just output the solution at a bunch of points for each value of $k$, and then create histograms from this for individual points by hand? This should be easy enough within matlab.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-20T18:15:00.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25263" PostTypeId="1" AcceptedAnswerId="25303" CreationDate="2016-10-20T22:33:31.873" Score="4" ViewCount="431" Body="&lt;p&gt;I am looking for a good numerical method to solve a linear system (of small/moderate size) with a matrix of the form&#xA;$$&#xA;\begin{bmatrix}&#xA;a_1 &amp;amp; b_1\\&#xA;&amp;amp; a_2 &amp;amp; \ddots \\&#xA;&amp;amp; &amp;amp; \ddots &amp;amp; b_{n-1}\\&#xA;b_n &amp;amp; &amp;amp; &amp;amp; a_n&#xA;\end{bmatrix}.&#xA;$$&#xA;What would you suggest?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Entries not displayed are zero. The $a_i$ and $b_i$ entries can be arbitrary, so I would not trust the stability of unpivoted Gaussian elimination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related question: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10137/solving-system-of-linear-equations-with-cyclic-tridiagonal-matrix&quot;&gt;solving system of linear equations with cyclic tridiagonal matrix&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2016-10-25T10:58:50.640" Title="Solving systems of linear equations with cyclic bidiagonal matrix" Tags="|linear-solver|banded-matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25264" PostTypeId="2" ParentId="25263" CreationDate="2016-10-20T23:49:53.703" Score="0" Body="&lt;p&gt;How small is &quot;small/moderate&quot;?  N=10? N=10000? N=1000000?  How many of these systems of equations do you have to solve (one?  a few hundred?  a few million?)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If N is small and the total number of these systems that you have to solve is small, then there's no point in doing anything more sophisticated than using a dense LU factorization ala LAPACK/BLAS.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If N is really big and and the total number of these that you have to solve is small, then use a sparse LU factorization routine to do the job. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only if N is really big and you have lots of these to solve would there be any point in trying to develop a custom solution to the problem.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-10-20T23:49:53.703" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25265" PostTypeId="1" CreationDate="2016-10-21T00:23:23.003" Score="1" ViewCount="62" Body="&lt;p&gt;Let $\bar{x}$ denote the analytical solution of a PDE. Let $x^{(k)}$ be the solution at the $k^{th}$ iteration. The initial guess for the solution is $x^{(0)} = 0$. Let $r_{0} = ||\bar{x}-x^{(0)}||_{2}$ i.e. the $l_{2}$ norm of the error. Let $r_{k} = ||\bar{x}-x^{(k)}||_{2}$ be the $l_{2}$ norm at the $k^{th}$ iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Is there a possibility that $\frac{r_{k}}{r_{0}}$ first decreases then increases and then stabilises to a particular value ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Additional info&lt;/em&gt;: When I examine this &lt;em&gt;stable&lt;/em&gt; value of $\frac{r_{k}}{r_{0}}$ for increasing mesh sizes, it &lt;em&gt;does&lt;/em&gt; give me $O(h^{2})$ convergence for a uniform cell-centered grid with Dirichlet boundaries solved using Finite Difference Method (using Jacobi iterations).   &lt;/p&gt;&#xA;" OwnerUserId="14303" LastActivityDate="2016-10-21T00:23:23.003" Title="Decreasing - increasing - stabilising $l_{2}$ norm" Tags="|finite-difference|convergence|elliptic-pde|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25267" PostTypeId="1" CreationDate="2016-10-21T10:13:40.897" Score="5" ViewCount="1333" Body="&lt;p&gt;I have a quantum mechanics simulation where I need to multiply three matrices that look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho(t_1)=U^\dagger \rho(t_0) \, U$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U^\dagger$ is the hermitian conjugate of $U$. This evolves the density matrix $\rho$ from one time point to another time point. $U$ is called the evolution operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Is there a smart way to reduce the time required to do this matrix multiplication in C++? Is there anything better than using BLAS's &lt;code&gt;zgemm&lt;/code&gt; 2 times (or &lt;code&gt;zhemm&lt;/code&gt; 2 times, since the matrices are Hermitian)?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Additional information:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean with smart is not only using a different library, but also finding a way to mathematically or computationally reduce the number of multiplications that have to be done in that operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sizes of the matrices I deal with range from $2^7$ to $2^{15}$ in side-length (All these matrices are square matrices).&lt;/p&gt;&#xA;" OwnerUserId="5410" LastEditorUserId="5410" LastEditDate="2016-10-21T19:24:22.233" LastActivityDate="2016-10-24T08:51:29.453" Title="Smart way to multiply 3 matrices" Tags="|linear-algebra|matrix|blas|dense-matrix|" AnswerCount="2" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="25268" PostTypeId="1" CreationDate="2016-10-21T13:25:57.843" Score="1" ViewCount="49" Body="&lt;p&gt;Hi I want to approximate $u(x,t)$ solution of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{cases}u_t+H(u_x)=0 \\u(x,0)=u_0(x)= e^{−(x−3)^2}&#xA;\end{cases}$,  where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H(u_x)=\frac{u_x^2}{2}$ and my spatial domain is $\Omega=[0;10]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now i want to apply Hopf-Lax formula (Evans' chapter 3 theorem 4), who says&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x_j,t_n+\Delta t)=\min_{a\in \mathbb{R}} \left\{ \Delta t H^*(a)+u(x_j-a\Delta t,t_n)\right\} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in my case $H^*(p)=H(p)=\frac{p^2}{2}$ and $u(x_j-a\Delta t,t_n)$ needs to be interpolated but... which different values I can assign to $a$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tested it with $a=\left\{ \min_j u(x_j,t_n),\max_j u(x_j,t_n)\right\}$ but it seems only partially working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help?&lt;/p&gt;&#xA;" OwnerUserId="21085" LastActivityDate="2016-10-21T13:25:57.843" Title="Solving HJ using Hopf-Lax formula (Burgers case)" Tags="|matlab|pde|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25269" PostTypeId="2" ParentId="25267" CreationDate="2016-10-21T13:54:30.810" Score="0" Body="&lt;p&gt;Considering only a computational point of view you could think of using  &lt;a href=&quot;http://docs.nvidia.com/cuda/cublas/#axzz4Nj04jj18&quot; rel=&quot;nofollow&quot;&gt;cuBLAS&lt;/a&gt;&#xA;the CUDA version of BLAS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the &lt;a href=&quot;https://solarianprogrammer.com/2012/05/31/matrix-multiplication-cuda-cublas-curand-thrust/&quot; rel=&quot;nofollow&quot;&gt;link&lt;/a&gt; is an example of use.&#xA;In the last part there is a example in C++, with &lt;a href=&quot;http://docs.nvidia.com/cuda/thrust/#axzz4Nj04jj18&quot; rel=&quot;nofollow&quot;&gt;thrust&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2016-10-21T13:54:30.810" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="25271" PostTypeId="1" AcceptedAnswerId="25272" CreationDate="2016-10-21T16:39:09.157" Score="1" ViewCount="615" Body="&lt;p&gt;Hi I am trying to take a derivative of an array but am having trouble.  The array is two dimensional, $x$ and $y$ directions.  I would like to take a derivative along $x$ and along $y$ using central difference discretization.  The array has random values of numbers, no values are NaN.  I will provide a basic portion of the code below to illustrate my point (assume the array $u$ is defined and has some initial values already inputted into it)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integer :: i,j&#xA;integer, parameter :: nx=10, ny=10&#xA;real, dimension(-nx:nx, -ny:ny) :: u,v,w&#xA;real, parameter :: h&#xA;&#xA;do i=-nx,nx&#xA;    do j=-ny,ny&#xA;&#xA;        v = (u(i+1,j)-u(i-1,j))/(2*h)&#xA;        w = (u(i,j+1)-u(i,j-1))/(2*h)&#xA;&#xA;    end do &#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note, assume the array $u$ is defined and filled up before I find $v$ and $w$. $v$, $w$ are supposed to be derivatives of the array $u$ along $x$ and along $y$, respectively. Is this the correct way to take a derivative of an array?&lt;/p&gt;&#xA;" OwnerUserId="19571" LastEditorUserId="9667" LastEditDate="2016-10-21T17:30:00.023" LastActivityDate="2016-10-23T16:06:21.057" Title="derivative of an array" Tags="|pde|finite-difference|fortran|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25272" PostTypeId="2" ParentId="25271" CreationDate="2016-10-21T18:15:22.473" Score="5" Body="&lt;p&gt;Purely in terms of terminology, it's probably better to talk about taking discrete partial derivatives of variable fields stored in an array, rather than differentiating an array itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the code itself, you appear to have dropped the element assignment inside your loop,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;v(i,j) = (u(i+1,j)-u(i-1,j))/(2*h)&lt;br&gt;&#xA;    w(i,j) = (u(i,j+1)-u(i,j-1))/(2*h)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;although that may be a typo. Your version is also bugged and unsafe at the end points of the ranges (i.e &lt;code&gt;i=-nx&lt;/code&gt;, &lt;code&gt;i=nx&lt;/code&gt;, &lt;code&gt;j=ny&lt;/code&gt;, &lt;code&gt;j=-ny&lt;/code&gt;) since in these places you are accessing elements of the u array which do not exist. At these locations you either need to use a non centred derivative, something like &#xA;&lt;code&gt;v(-nx,j) = (u(-nx+1,j) - u(-nx,j)/h&lt;/code&gt;, or use any knowledge of the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side note, if you are writing Fortran code frequently, then in terms of code optimisations, it is useful to remember that the index furthest to right should be outermost in loops for the code to have the best chance of running fast, which implies swapping your i and j loops. In many simple cases it's possible the compiler will fix this for you, and it's certainly less important than having code which works, but it's probably a good habit to get into. &lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-10-21T18:15:22.473" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25277" PostTypeId="1" AcceptedAnswerId="25281" CreationDate="2016-10-23T12:29:33.893" Score="1" ViewCount="595" Body="&lt;p&gt;I am reading the paper, &lt;a href=&quot;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&quot; rel=&quot;nofollow&quot;&gt;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper, the nonlinear term is treated as mix of central central difference and upwind difference using a transition parameter $\gamma \in [0,1]$, and it's defined as &#xA;$$\frac{U^*-U}{\Delta t} = &#xA;((\bar{U}^h)^2 - \gamma \lvert \bar{U}^h \rvert \tilde{U}^h )_x&#xA;- ((\bar{U}^v \bar{V}^h) - \gamma \lvert \bar{V}^h \rvert \tilde{U}^v )_y&#xA;$$&#xA;The variables $\bar{U}^h$ and $\tilde{U}^h$ are defiend as,&#xA;$$&#xA;(\bar{U}^h)_{i+\frac{1}{2},j} = \frac{U_{i+1,j}+U_{i,j}}{2}\\&#xA;(\bar{U}^v)_{i,j+\frac{1}{2}} = \frac{U_{i,j+1}+U_{i,j}}{2}\\&#xA;(\tilde{U}^h)_{i+\frac{1}{2},j} = \frac{U_{i+1,j}-U_{i,j}}{2}\\&#xA;(\tilde{U}^v)_{i,j+\frac{1}{2}} = \frac{U_{i,j+1}-U_{i,j}}{2}\\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U^2$ term is &#xA;$$ ((\bar{U}^h)^2 - \gamma \lvert \bar{U}^h \rvert \tilde{U}^h )_{i+\frac{1}{2}, j}&#xA;= \bar{U}^h&#xA;\begin{cases}&#xA;\left(\frac{1-\gamma}{2}\right) U_{i+1,j}&#xA;+ \left(\frac{1+\gamma}{2}\right) U_{i,j} &#xA;\;\;\; \text{if} \;\; \bar{U}^h\ge0\\&#xA;\left(\frac{1+\gamma}{2}\right) U_{i+1,j}&#xA;+ \left(\frac{1-\gamma}{2}\right) U_{i,j}&#xA;\;\;\; \text{if} \;\; \bar{U}^h\lt 0\\&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it says, &#xA;&quot;One can easily see that this becomes averaged central differencing for $\gamma=0$ and conservative upwinding for $\gamma=1$.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I cannot see this becomes the conservative upwind difference when $\gamma =1$. I thought the conservative upwinding is &#xA;$$&#xA;\begin{cases}&#xA;(U_{i,j}^2-U_{i-1,j}^2)/\Delta x&#xA;\;\;\; \text{if}\;\;U_{i,j}\ge0&#xA;\\&#xA;(U_{i+1,j}^2-U_{i,j}^2)/\Delta x&#xA;\;\;\; \text{if}\;\;U_{i,j}\lt0&#xA;\\&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I plug $U_{i,j}$ in $\bar{U}$ and $\tilde{U}$ with the condition $\bar{U}^h \ge 0$ and $\gamma=1$, the first term becomes&#xA;$$ ((\bar{U}^h_{i+\frac{1}{2}} U_{i,j}) - (\bar{U}^h_{i-\frac{1}{2}} U_{i-1,j}))/{\Delta x}\\&#xA;=((\frac{U_{i+1,j}+U_{i,j}}{2}U_{i,j}) - (\frac{U_{i,j}+U_{i-1,j}}{2} U_{i-1,j}))/{\Delta x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not the conservative upwinding difference I thought. I am confused because it depends on $\bar{U}^h$, not $U_{i,j}$. Can someone explain and show the right conservative upwinding difference form?&lt;/p&gt;&#xA;" OwnerUserId="14387" LastActivityDate="2016-10-24T12:03:19.213" Title="Upwind difference for velocity in staggered grid" Tags="|finite-difference|navier-stokes|advection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25279" PostTypeId="1" CreationDate="2016-10-23T15:43:12.633" Score="2" ViewCount="696" Body="&lt;p&gt;I am new to MPI and am trying to figure out the subtleties in it, and for that reason I am just playing around with an easy example. The question I have is that when I run the code below with 2 processors, even though the printf statement is BEFORE the MPI_Init both processors print it. Is there anyway to not have this happen, and only use the multiple processors when the environment is initialized? Below is my minimum working example.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;mpi.h&amp;gt;&#xA;#include&amp;lt;stdio.h&amp;gt;&#xA;&#xA;#define send_data_tag 2001&#xA;#define return_data_tag 2002&#xA;&#xA;double add(int a);&#xA;&#xA;int main(int argc, char ** argv)&#xA;{&#xA;  int rank, size, len;&#xA;  char host[MPI_MAX_PROCESSOR_NAME];&#xA;&#xA;  printf(&quot;Will all processors print this?&quot;);&#xA;  MPI_Status status;&#xA;  MPI_Init(&amp;amp;argc, &amp;amp;argv);&#xA;&#xA;  MPI_Comm_rank(MPI_COMM_WORLD, &amp;amp;rank);&#xA;  MPI_Comm_size(MPI_COMM_WORLD, &amp;amp;size);&#xA;  MPI_Get_processor_name(host, &amp;amp;len);&#xA;&#xA;  if(rank == 0)&#xA;    {&#xA;        int test = 1;&#xA;        int final_test;&#xA;        MPI_Send(&amp;amp;test, 1 , MPI_INT, 1, send_data_tag, MPI_COMM_WORLD); &#xA;&#xA;        MPI_Recv(&amp;amp;final_test, 1, MPI_INT, 1, return_data_tag, MPI_COMM_WORLD, &amp;amp;status);&#xA;        printf(&quot;%d\n&quot;, final_test);&#xA;    }&#xA;  else&#xA;    {&#xA;        int test_recv , final_test;&#xA;        MPI_Recv( &amp;amp;test_recv, 1, MPI_INT, 0, send_data_tag, MPI_COMM_WORLD, &amp;amp;status);&#xA;        final_test = test_recv + 1;&#xA;        MPI_Send(&amp;amp;final_test, 1, MPI_INT, 0, return_data_tag, MPI_COMM_WORLD);&#xA;    }&#xA;&#xA;&#xA;MPI_Finalize();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20973" LastActivityDate="2017-01-23T20:12:05.980" Title="MPI in C, all processor run outside of the MPI envir created" Tags="|parallel-computing|c|mpi|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25280" PostTypeId="2" ParentId="25271" CreationDate="2016-10-23T16:06:21.057" Score="2" Body="&lt;p&gt;It's called column major due to the Fortran array model. Brieftly saying about this: Let $a$ be a matrix of size $m \times n$. Let $a_{ij}$ be  element of $a$ for $i=\overline{1,m}$ and $j=\overline{1,n}$, then $a_{ij} $ is stored at the $(i+(j-1)m)$-th position of one-dimenstional storage $A(:)$, i.e. $ a_{ij} = A(i+(j-1)m )$. Writting the loop as the orginal post, the storage $A(:)$ is processed with $m$-increment. However, swapping loop order, the storage $A(:)$ is processed continuously. Hence this approach is performed faster than the first one.&lt;/p&gt;&#xA;" OwnerUserId="15175" LastActivityDate="2016-10-23T16:06:21.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25281" PostTypeId="2" ParentId="25277" CreationDate="2016-10-23T17:10:30.243" Score="1" Body="&lt;p&gt;Your last term&#xA;$$ ((\frac{U_{i+1,j}+U_{i,j}}{2}U_{i,j}) - (\frac{U_{i,j}+U_{i-1,j}}{2} U_{i-1,j}))/{\Delta x}&#xA;$$&#xA;is a conservative approximation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea of conservative upwind scheme for $(u^2)_x$ is to consider it as a difference of fluxes $u u$ at $i+1/2$ and $i-1/2$. The  arithmetic averages in this formula are velocities at these points, say $U_{i+1/2,j}$ and $U_{i-1/2,j}$. The values $U_{i,j}$ and $U_{i-1,j}$ are upwinded values of $u$ at these points. In fact this formula shall be used when the averaged velocities are positive, i.e. $U_{i+1/2,j}&amp;gt;0$ and $U_{i-1/2,j}&amp;gt;0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. here I add the derivation of such conservative upwind approximation (where I skip the index $j$):&#xA;$$&#xA;(u u)_x(x_i) \approx \frac{(u u)(x_{i+1/2})-(u u)(x_{i-1/2})}{\Delta x} \approx \frac{U_{i+1/2} U_i -U_{i-1/2}U_{i-1}}{\Delta x}&#xA;$$&#xA;if&#xA;$$&#xA;U_{i+1/2}=0.5 (U_i+U_{i+1}) &amp;gt; 0 \hbox{   and   } U_{i-1/2}=0.5 (U_{i-1}+U_{i}) &amp;gt; 0&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-10-24T12:03:19.213" LastActivityDate="2016-10-24T12:03:19.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25282" PostTypeId="2" ParentId="25279" CreationDate="2016-10-23T19:12:35.083" Score="2" Body="&lt;p&gt;I fear you're confusing your parallel paradigms. MPI supports parallelism with multiple processes (think instances of your program) which communicate with each other across a network. It's possible the &quot;network&quot; can be within one box, or it can be on multiple machines miles away from each other. With this in mind, it's easy to see why your print command is being run multiple times. Each separate process starts up, prints the string, then actually gets round to setting up communication with the other processes. Thus the only way to limit IO to one process is with conditionals like your &lt;code&gt;if (rank==0)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another parallel model, threading, works on a single box in a single process with threads sharing access to the same memory. This model fits better with the viewpoint you describe in which threads are only created when there is work to be done in parallel. If you're interested in this, then look up some tutorials on &lt;a href=&quot;https://en.wikipedia.org/wiki/OpenMP&quot; rel=&quot;nofollow&quot;&gt;OpenMP&lt;/a&gt;, (not to be confused with openMPI, a well known implementation of MPI) which seems to be the popular way to thread a program with minimum effort at the moment.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2016-10-23T19:12:35.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25283" PostTypeId="1" AcceptedAnswerId="25296" CreationDate="2016-10-23T20:12:20.763" Score="6" ViewCount="7328" Body="&lt;p&gt;A dull matrix multiplication algorithm where we use the formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C_{ij}=\sum_{k}A_{ik}B_{kj}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By literally following this in 3 loops we'll get a very slow program, because we don't utilize &lt;a href=&quot;https://en.wikipedia.org/wiki/Vector_processor&quot; rel=&quot;noreferrer&quot;&gt;processor's vectorization&lt;/a&gt; power. An example of this slow implementation is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T comp;&#xA;for (int i = 0; i &amp;lt; lhs.rows(); i++)&#xA;{&#xA;    for (int j = 0; j &amp;lt; rhs.columns(); j++)&#xA;    {&#xA;        comp = 0;&#xA;        for (int k = 0; k &amp;lt; lhs.columns(); k++)&#xA;        {&#xA;            comp += lhs.at(i,k)*rhs.at(k,j);&#xA;        }&#xA;        result.at(i,j) = comp;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is: If we consider vectorization, are we going to beat typical libraries like OpenBLAS?&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why?&lt;/strong&gt; Because I want to specialize my matrix multiplications so much that BLAS doesn't provide what I need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we assume that these matrices are square matrices of side-length $n$, and are stored in &lt;a href=&quot;https://en.wikipedia.org/wiki/Row-major_order&quot; rel=&quot;noreferrer&quot;&gt;column-major order&lt;/a&gt; and contiguous in memory (e.g., std::vector, with $n^2$ elements). Consider the following code snippet:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;transpose(lhs); //apply in-place transpose&#xA;for (int i = 0; i &amp;lt; lhs.columns(); i++)&#xA;{&#xA;    for (int j = 0; j &amp;lt; rhs.columns(); j++)&#xA;    {&#xA;        result.at(i,j) = &#xA;            std::transform(&#xA;            lhs.begin()+i*lhs.rows(),&#xA;            lhs.begin()+(i+1)*lhs.rows(),&#xA;            rhs.begin()+j*rhs.rows(),&#xA;            std::multiplies&amp;lt;T&amp;gt;);&#xA;    }&#xA;}&#xA;transpose(lhs);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now this doesn't only consider vectorization, but even uses standard algorithms to give the compiler full freedom to do all the optimizations it wants. So my question here is: Will this beat good BLAS libraries? What else should I do to make such code super-efficient?&lt;/p&gt;&#xA;" OwnerUserId="5410" LastActivityDate="2021-02-15T21:49:39.077" Title="Beating typical BLAS libraries matrix multiplication performance" Tags="|optimization|c++|matrix|performance|blas|" AnswerCount="4" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25284" PostTypeId="2" ParentId="24017" CreationDate="2016-10-23T21:11:37.657" Score="1" Body="&lt;p&gt;If you want the amplitude at a single frequency $f_0$ then you can multiply your signal by $e^{-j 2\pi f_0}$ and then use a simple moving average filter to remove other frequency components. This is probably the most efficient way, computationaly. Especially if you store  $\sin$ and $\cos$ values in a look up table.&lt;/p&gt;&#xA;" OwnerUserId="10169" LastActivityDate="2016-10-23T21:11:37.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25285" PostTypeId="2" ParentId="24530" CreationDate="2016-10-24T00:15:01.780" Score="0" Body="&lt;p&gt;I use matVTK to do this type of volume plotting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cir.meduniwien.ac.at/assets/people/birngruber/publications/matVTK_miccai_2009.pdf&quot; rel=&quot;nofollow&quot;&gt;https://www.cir.meduniwien.ac.at/assets/people/birngruber/publications/matVTK_miccai_2009.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7257" LastActivityDate="2016-10-24T00:15:01.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25287" PostTypeId="1" AcceptedAnswerId="25291" CreationDate="2016-10-24T03:32:48.057" Score="3" ViewCount="314" Body="&lt;p&gt;Consider the 1D Poisson equation&#xA;$$\nabla^2 u = f.$$&#xA;Using finite difference method on cell corner data and a uniform grid with ghost points, I think we can write the system of equations with Neumann BCs as:&#xA;$$Au = f-Au_{BC},$$&#xA;where&#xA;$$&#xA;\begin{aligned}&#xA;A &amp;amp;= \frac{1}{\Delta x^2} \left[\begin{array}{ccccccccc}&#xA;-2    &amp;amp; 2         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;     \\&#xA;1     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;     \\&#xA;      &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;     \\&#xA;      &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; 1   \\&#xA;      &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  2        &amp;amp; -2  \\&#xA;\end{array} \right]&#xA;\\&#xA;Ax_{BC} &amp;amp;= \frac{1}{\Delta x^2} \left[\begin{array}{ccccccccc}&#xA;    2 \hat{n} \theta \Delta x   \\&#xA;    0  \\&#xA;       \\&#xA;\vdots \\&#xA;       \\&#xA;    0  \\&#xA;    2 \hat{n} \theta \Delta x   \\&#xA;\end{array} \right]&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And $\hat{n},\theta$ is the outward facing normal and the prescribed slope of the derivative at the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notably, the system is singular, which can be addressed by removing the mean of the RHS.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I know that $A$ must be symmetric, positive definite. I've done some tests without multiplying the first and last rows by 0.5 and they seem to work fine, so my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Must the first and last rows of the left and right hand side be multiplied by 0.5? In other words, are there cases where it will not work without multiplication of 0.5?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Notes&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I imagine that both sides of the equation would be balanced the same with and without this multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final note, I know that the code / algorithm may be more clear with the multiplication, since the equation explicitly satisfies symmetry, but I'm more interested in whether the multiplication is necessary or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="713" LastEditDate="2016-10-24T04:01:00.367" LastActivityDate="2016-10-24T12:48:07.823" Title="CG question: is symmetry always necessary?" Tags="|finite-difference|numerical-analysis|conjugate-gradient|symmetry|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25288" PostTypeId="1" CreationDate="2016-10-24T05:40:52.613" Score="1" ViewCount="1028" Body="&lt;p&gt;Here is the flow past a square cylinder configuration. The problem is a viscous and incompressible with parabolic velocity profile using freestream velocity &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; across a single cylinder. I use the single relaxation time lattice Boltzmann method. For the two-dimensional nine-velocity lattice(D2Q9), the directions of the particles are shown in the picture.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GDL4O.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GDL4O.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The parameters: &lt;span class=&quot;math-container&quot;&gt;$H=160$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$L=780$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$I=170$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$D=20$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$U_\max=0.0438$&lt;/span&gt;,&#xA;&lt;span class=&quot;math-container&quot;&gt;$\mathrm{density}=1.0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dy=dx=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dt=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathrm{Re}=160$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The boundary conditions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Inlet: He-Zou boundary condition&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Wall: bounce-back conditions&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Outlet: interpolation (Here is the code for outlet BCs)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;f(1,n,j)=2*f(1,n-1,j)-f(1,n-2,j)&#xA;f(5,n,j)=2*f(5,n-1,j)-f(5,n-2,j)&#xA;f(8,n,j)=2*f(8,n-1,j)-f(8,n-2,j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My problem is whether the outlet boundary conditions are right or not. If it is right, why it's about &lt;code&gt;f1&lt;/code&gt;, &lt;code&gt;f5&lt;/code&gt;, &lt;code&gt;f8&lt;/code&gt; rather than &lt;code&gt;f3&lt;/code&gt;, &lt;code&gt;f6&lt;/code&gt;, &lt;code&gt;f7&lt;/code&gt;? In my opinion, it seems that we can't get &lt;code&gt;f3&lt;/code&gt;, &lt;code&gt;f6&lt;/code&gt;, &lt;code&gt;f7&lt;/code&gt; from streaming on the outlet.&lt;/p&gt;&#xA;" OwnerUserId="21055" LastEditorUserId="20688" LastEditDate="2022-01-19T20:40:29.000" LastActivityDate="2022-01-19T20:40:29.000" Title="Outlet boundary conditions in lattice Boltzmann method" Tags="|fluid-dynamics|boundary-conditions|lattice-boltzmann-methods|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25290" PostTypeId="2" ParentId="25267" CreationDate="2016-10-24T06:57:51.647" Score="3" Body="&lt;p&gt;Have you considered working with the Cholesky (or low-rank) factor of $\rho(t_0)$ rather than with the matrix itself? This might reduce the number of products that you need to make, and it has the additional benefit of preserving positive semidefiniteness across your computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is already cheaper to use Cholesky factors for this computation alone, if I am not making mistakes with the costs: let $n$ be the side of each matrix appearing here; computing the upper triangular Cholesky factor $R$ such that $\rho(t_0)=R^\dagger R$ costs $\frac13 n^3$ (counting with the traditional model addition=multiplication=1); computing $RU$ costs $n^3$ (because $R$ is triangular), and computing $(RU)^\dagger (RU)$ costs $n^3$ (because you only need to compute half of the entries). OTOH, computing $U^\dagger \rho(t_0)$ costs $2n^3$, and then computing half of the entries of $(U^\dagger \rho(t_0))U$ costs $n^3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\rho(t_0)$ has low rank (which in the examples I saw when I studied quantum mechanics happened quite often for initial states), then one can start from a rectangular $R$ coming from the QR of its low-rank factor rather than a Cholesky decomposition, and this method is even cheaper. If you have to make more computations with your matrices after this triple product, then there may be additional savings: for instance, to compute products and solve linear system you can work with the low-rank factors directly rather than forming the last product.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2016-10-24T08:51:29.453" LastActivityDate="2016-10-24T08:51:29.453" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25291" PostTypeId="2" ParentId="25287" CreationDate="2016-10-24T12:48:07.823" Score="6" Body="&lt;p&gt;The CG method works by producing in every iteration a vector $x^k\in\mathbb{R}^n$ that solves the minimization problem (assuming $x^0=0$ for simplicity)&#xA;$$\min_{x\in \mathcal{K}_k} \frac12(x-x^*)^TA(x-x^*),\tag{1}\label{cg1}$$&#xA;where $x^*=A^{-1}b$ and $\mathcal{K}_k$ is a &lt;em&gt;Krylov space&lt;/em&gt; of (apart from exceptional circumstances) dimension $k$. (The CG algorithm is just a clever way of computing this minimizer efficiently by reusing as much information from the previous iteration as possible.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for $k=n$ (at the latest), $x^n$ minimizes the function $f(x) = \frac12(x-x^*)^TA(x-x^*)$ over $\mathbb{R}^n$, which means that the gradient has to vanish. After expanding the product and using the product rule, this yields&#xA;$$0 = \nabla f(x^n) = \frac12(A+A^T)x^n - \frac12(A+A^T)x^*.\tag{2}\label{cg2}$$&#xA;If $A$ is symmetric ($A^T=A$) and $Ax^*=b$, this of course implies that $Ax^n=b$; otherwise you only get a solution to $(A+A^T)x = (A+A^T)A^{-1}b$. In your case, $A$ is nearly symmetric and fairly well-conditioned, so the difference between the solutions to these two equations is probably so small that you don't see it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question is whether the numerical algorithm breaks down if $A$ is not symmetric. Looking at the algorithm, this can only happen if $x^TA x =0$ for some $x\neq 0$ (which of course is impossible for positive definite matrices by definition). Now note that you can write any matrix $A$ as&#xA;$$A = \frac12 (A+A^T) + \frac12(A-A^T) =: A_{\text{sym}} + A_{\text{skew}},$$&#xA;where $A_{\text{sym}}$ is the &lt;em&gt;symmetric part&lt;/em&gt; and $A_{\text{skew}}$ the &lt;em&gt;skew-symmetric part&lt;/em&gt; of $A$. Furthermore, $x^TA_{\text{skew}}x=0$ and hence $x^TAx = x^TA_{\text{sym}}x$ for all $x\in\mathbb{R}^n$. So the algorithm works as long as the symmetric part $A_{\text{sym}}$ of $A$ is positive definite. Conveniently, this implies that the minimization problem \eqref{cg1} is strictly convex and hence has a unique minimizer, which is the only solution to \eqref{cg2}.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR: &lt;strong&gt;CG works for nonsymmetric matrices as long as the symmetric part is positive definite, and will give a reasonable approximation to the solution if the skew-symmetric part is &quot;negligible&quot;.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-10-24T12:48:07.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25292" PostTypeId="1" CreationDate="2016-10-24T13:05:02.697" Score="0" ViewCount="576" Body="&lt;p&gt;I wanted to implement a basic example from the book &quot;The Art of molecular dynamics simulation&quot; in LAMMPS. It describes the 2d-movement of molecules in a Lennard-Jones-potential, defined as &#xA;$$u(r_{ij})=\left\{\begin{matrix}4\varepsilon\left[\left(\frac{\sigma}{r_{ij}}\right)^{12}-\left(\frac{\sigma}{r_{ij}}\right)^{6}\right]+\varepsilon&amp;amp;r_{ij}&amp;lt;r_c=2^{1/6}\sigma\\0&amp;amp;r_{ij}\geq r_c\end{matrix}\right. $$&#xA;using temperature and density as parameters. The dimensions of the 2d-area are $\left(20, 20\right)$, having a molecule sitting at each of the nodes, resulting in 400 molecules. The values measured are the total kinetic energy, the total potential energy and the total pressure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My implementation in lammps thus is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#First test, if I am able to write a lammps script on my own&#xA;#------------------Init-----------------&#xA;clear&#xA;units metal&#xA;dimension 2&#xA;boundary p p p&#xA;atom_style body nparticle 2 6&#xA;atom_modify map array#Array or Hash&#xA;&#xA;#------------Create Atoms----------------&#xA;lattice sq 1&#xA;region box block 0 20 0 20 0 1 units lattice&#xA;create_box 1 box&#xA;&#xA;lattice sq 1 orient x 1 0 0 orient y 0 1 0 orient z 0 0 1&#xA;create_atoms 1 box&#xA;replicate 1 1 1&#xA;&#xA;#-----------Interatomic potential---------&#xA;pair_style lj/cut 1.122466&#xA;pair_coeff * * 1 1&#xA;neighbor 0.4 bin&#xA;velocity all create 1.44 87287 loop geom&#xA;&#xA;#------------Define Settings---------------&#xA;compute eng all pe/atom&#xA;compute keng all ke/atom&#xA;compute eatoms all reduce sum c_eng&#xA;compute keatoms all reduce sum c_keng&#xA;#compute peng all pressure&#xA;&#xA;#-------------Run minimization--------------&#xA;reset_timestep 0&#xA;fix 1 all box/relax iso 0.0 vmax 0.001&#xA;thermo 10&#xA;thermo_style custom step pe lx ly lz press pxx pyy pzz c_eatoms&#xA;min_style cg&#xA;minimize 1e-25 1e-25 5000 10000&#xA;&#xA;variable natoms equal &quot;count(all)&quot;&#xA;variable teng equals &quot;c_eatoms&quot;&#xA;variable length equals &quot;lx&quot;&#xA;variable ecoh equal &quot;v_teng/v_natoms&quot;&#xA;&#xA;print &quot;Total energy (eV) = ${teng};&quot;&#xA;print &quot;Number of atoms = ${natoms};&quot;&#xA;print &quot;Lattice constant (Angstoms) = ${length};&quot;&#xA;print &quot;Cohesive energy (eV) = ${ecoh};&quot;&#xA;print &quot;All done!&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is that implementation approximately correct, or is there something completely wrong?&lt;br&gt;&#xA;When executing the file, the execution stops with the error &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ERROR: Illegal variable command (../variable.cpp:512)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I debug that?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2016-10-24T13:18:07.480" Title="Implementation of basic example in LAMMPS" Tags="|lammps|" AnswerCount="1" CommentCount="2" ClosedDate="2016-10-24T16:04:09.693" ContentLicense="CC BY-SA 3.0" />
  <row Id="25293" PostTypeId="2" ParentId="25292" CreationDate="2016-10-24T13:18:07.480" Score="2" Body="&lt;p&gt;It seems your questions is more on &quot;How to debug that error&quot; than on the error itself. If you wish me to look into the error specifically, I will do this, I promise :).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, it is great to learn from your own mistakes and to learn how to debug yourself.&#xA;The best way to debug a lammps script is to use the command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;echo both&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;at the beggining of the script. This command will output each line processed by lammps one by one to the terminal.&#xA;You must understand that the script is read and interpreted line by line. So if you put echo both, each line that is interpreted will be displayed as it is interpreted. This way, when the code crash, the line above it will be the last one that has been processed and the error will most likely reside within that line. In your case, I believe your error is most likely related to one of your compute command.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2016-10-24T13:18:07.480" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25294" PostTypeId="1" CreationDate="2016-10-24T14:23:40.713" Score="1" ViewCount="178" Body="&lt;p&gt;I want some scalar spline function defined on regular 2D grid $F(x,y)$ with &lt;strong&gt;continuous first derivative&lt;/strong&gt; which is easy to intersect with arbitrary ray/line ${\vec l}(t) = (c_x t,c_y t,c_z t)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding the intersection means finding roots of equation $f_{c_x,c_y}(t)-c_zt=0$ where  $f_{c_x,c_y}(t)=F(c_x t, c_y t )$ are values of the spline along the line ${\vec l}(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is easy to find roots of quadratic polynominal I want $f_{c_x,c_y}(t)$ to be &lt;strong&gt;piecewise quadratic polynominal&lt;/strong&gt; for any ${c_x,c_y}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is the problem / what is not a solution:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;consider &lt;strong&gt;bi-quadratic B-spline on rectangular&lt;/strong&gt; grid created by tensor-product of 1D quadratic B-splines. The result is bi-quadratic which means it is $f_{c_x,c_y}(t)$ is &lt;strong&gt;4-th order polynominal&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;function composed of &lt;a href=&quot;https://en.wikipedia.org/wiki/B%C3%A9zier_triangle&quot; rel=&quot;nofollow&quot;&gt;&lt;strong&gt;Quadratic Bezier-triangles&lt;/strong&gt;&lt;/a&gt;. While it is certainly just quadratic function along any direction, it is &lt;strong&gt;hard ensure continuous derivatives&lt;/strong&gt; at the boundary between triangles.  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4696" LastActivityDate="2016-10-24T14:23:40.713" Title="C1 continuous spline on regular 2D-grid with quadratic 1D cuts" Tags="|interpolation|b-spline|grid|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25295" PostTypeId="2" ParentId="25283" CreationDate="2016-10-24T14:33:18.217" Score="0" Body="&lt;p&gt;This is not an answer, but a reference to explore the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.maths.manchester.ac.uk/~higham/papers/high90s.pdf&quot; rel=&quot;nofollow&quot;&gt;Here&lt;/a&gt; there is an article by Higham about the matrix multiplication (~ 1990).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Title:  Exploiting Fast Matrix Multiplication Within the Level 3 BLAS&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2016-10-24T14:33:18.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25296" PostTypeId="2" ParentId="25283" CreationDate="2016-10-24T16:11:58.503" Score="14" Body="&lt;p&gt;Consolidating the comments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;No&lt;/strong&gt;, you are very unlikely to beat a typical BLAS library such as Intel's MKL, AMD's Math Core Library, or OpenBLAS.&lt;sup&gt;1&lt;/sup&gt;&#xA;These not only use vectorization, but also (at least for the major functions) use kernels that are hand-written in architecture-specific assembly language in order to optimally exploit available vector extensions (SSE, AVX), multiple cores, and cache reuse. For example, OpenBLAS has &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dgemm_kernel_16x2_haswell.S     &#xA;dgemm_kernel_4x4_haswell.S  &#xA;dgemm_kernel_4x8_haswell.S  &#xA;dgemm_kernel_4x8_sandy.S    &#xA;dgemm_kernel_6x4_piledriver.S   &#xA;dgemm_kernel_8x2_bulldozer.S    &#xA;dgemm_kernel_8x2_piledriver.S&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and that's only for the &lt;code&gt;x86_64&lt;/code&gt; architecture -- not only a different implementation for each instruction set, but also for different register blocks (meaning that different kernels can be used depending on the matrix size). Of course, where possible, they make use of (also optimized) BLAS2 and BLAS1 operations. That's not something a compiler could do automatically.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1. The LAPACK implementation of BLAS available from &lt;code&gt;netlib&lt;/code&gt; does not really count -- it's a reference implementation and should rather be considered as a (functional) specification of the &lt;em&gt;interface&lt;/em&gt;. (See: &lt;a href=&quot;http://dx.doi.org/10.1145/355841.355847&quot; rel=&quot;noreferrer&quot;&gt;http://dx.doi.org/10.1145/355841.355847&lt;/a&gt;, &lt;a href=&quot;http://dx.doi.org/10.1145/42288.42291&quot; rel=&quot;noreferrer&quot;&gt;http://dx.doi.org/10.1145/42288.42291&lt;/a&gt; and &lt;a href=&quot;http://dx.doi.org/10.1145/77626.79170&quot; rel=&quot;noreferrer&quot;&gt;http://dx.doi.org/10.1145/77626.79170&lt;/a&gt;) &#xA;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="569" LastEditDate="2016-10-25T09:06:38.167" LastActivityDate="2016-10-25T09:06:38.167" CommentCount="0" CommunityOwnedDate="2016-10-24T16:11:58.503" ContentLicense="CC BY-SA 3.0" />
  <row Id="25298" PostTypeId="2" ParentId="24828" CreationDate="2016-10-25T00:50:05.003" Score="0" Body="&lt;p&gt;FFTW has real-to-complex, real-to-real, and complex-to-real plans. For example, in 1d to take a complex fft result and get a real data result you call the function &quot;fftw_plan_dft_c2r_1d&quot; where the &lt;em&gt;c2r&lt;/em&gt; means complex-to-real.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 3d the function is &quot;fftw_plan_dft_c2r_3d&quot; which does the same thing I describe above but in 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out the documentation &lt;a href=&quot;http://www.fftw.org/fftw3_doc/Multi_002dDimensional-DFTs-of-Real-Data.html#Multi_002dDimensional-DFTs-of-Real-Data&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="7257" LastEditorUserId="7257" LastEditDate="2016-10-25T01:02:54.303" LastActivityDate="2016-10-25T01:02:54.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25299" PostTypeId="1" CreationDate="2016-10-25T04:18:36.197" Score="2" ViewCount="428" Body="&lt;p&gt;Suppose we have equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H \Psi = E \Psi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H$ is Dirac Hamiltonian (also my question can be answered by people who are not familiar with Dirac Hamiltonian but familiar with numerical solution of different eigenvalue problems):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ H = \vec{\alpha} \vec{\nabla} + V(r) = H_0 + V(r)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\vec{\alpha}$ and $\beta$ are $4 \times 4$ Dirac matrices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{\alpha} = \left( \begin{matrix} 0  &amp;amp; \vec{\sigma} \\ -\vec{\sigma} &amp;amp; 0 \end{matrix} \right) \ , \ \ \beta = \left( \begin{matrix} 1 &amp;amp; 0 \\ 0 &amp;amp; -1 \end{matrix} \right)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec{\sigma}$ are $2 \times 2$ Pauli matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $H_0$ is free Dirac Hamiltonian (without potential, only kinetic term is preserved).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the best way to start solving such an equation numerically is to introduce plane-wave basis (which diagonalizes free Hamiltonian $H_0$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Psi = \sum_n c_n \psi_n $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we have a problem like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H \sum_n c_n \psi_n = E \sum_n c_n \psi_n $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: what are the most commonly used ways to solve this equation numerically for arbitrary potential? Just to compute matrix elements $H_{nm}$ in this plane-wave basis, to find eigenvalue $E$ and coefficients $C_n$ from this computation? Or maybe there are some better methods?&lt;/p&gt;&#xA;" OwnerUserId="19192" LastEditorUserId="9667" LastEditDate="2016-10-27T15:58:40.697" LastActivityDate="2016-10-27T15:58:40.697" Title="Numerical solution of Dirac equation (eigenvalue problem)" Tags="|numerical-analysis|eigenvalues|eigensystem|quantum-mechanics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25300" PostTypeId="2" ParentId="25279" CreationDate="2016-10-25T08:44:36.403" Score="2" Body="&lt;p&gt;Usually, in MPI programs, the &lt;code&gt;MPI_Init()&lt;/code&gt;, &lt;code&gt;MPI_Comm_rank&lt;/code&gt; and &lt;code&gt;MPI_Comm_size&lt;/code&gt; are used only once in the very beginning. They initialize the environment, assign the &quot;rank&quot; and get the communicator size in the &lt;code&gt;MPI_COMM_WORLD&lt;/code&gt; communicator (one may set multiple communicators for convenience in some scenarios). But your question is still interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compile and run this small code with MPI:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include&amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;unistd.h&amp;gt;&#xA;&#xA;int main(int argc, char ** argv)&#xA;{&#xA;  printf(&quot;Will all processors print this?\n&quot;&#xA;          &quot;My process ID : %d\n&quot;&#xA;          &quot;My parent's ID: %d\n&quot;,&#xA;          getpid(),getppid());&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will print the ID of each process you start with the mpirun, as well as the parent process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that not even the &lt;code&gt;mpi.h&lt;/code&gt; is included here. What you see is because you start several processes with mpirun. Each of them gets the same binary, but can follow a different branch because MPI assigns a different &lt;code&gt;rank&lt;/code&gt; to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to do something only with one process, you have to put it in an &lt;code&gt;if (rank==0)&lt;/code&gt; (or a similar statement), as origimbo mentioned. Ofcourse, this needs to be after the &lt;code&gt;MPI_Init()&lt;/code&gt; and the &lt;code&gt;MPI_Comm_rank()&lt;/code&gt; calls.&lt;/p&gt;&#xA;" OwnerUserId="17650" LastActivityDate="2016-10-25T08:44:36.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25301" PostTypeId="1" CreationDate="2016-10-25T09:24:16.633" Score="1" ViewCount="349" Body="&lt;p&gt;How to normalize the CPU time for comparing algorithm execution time on systems with different hardware configurations?&lt;/p&gt;&#xA;" OwnerUserId="22105" LastActivityDate="2016-10-25T10:45:44.833" Title="comparison of CPU time and normalization" Tags="|algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25302" PostTypeId="2" ParentId="25301" CreationDate="2016-10-25T10:45:44.833" Score="1" Body="&lt;p&gt;Comparing algorithms in terms of having comparable, reliable, and reproducible results is only possible on the same hardware. Especially the runtime ( wall- or cpu-time) depend strongly on nearly all aspects of your hardware like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;clock speed of the CPU &lt;/li&gt;&#xA;&lt;li&gt;clock speed of the memory &lt;/li&gt;&#xA;&lt;li&gt;cycle length of the memory &lt;/li&gt;&#xA;&lt;li&gt;bandwidth of the memory connection &lt;/li&gt;&#xA;&lt;li&gt;ECC mode for the memory (enabled/disabled) &lt;/li&gt;&#xA;&lt;li&gt;interconnect between CPU sockets&lt;/li&gt;&#xA;&lt;li&gt;power consumption ( regarding AVX turbo mode or slowing down the system due to thermal reasons) &lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So there are too many factors that influence the runtime if there are different hardware configurations and in this way no normalization which takes all of them into account can be found. &lt;/p&gt;&#xA;" OwnerUserId="569" LastActivityDate="2016-10-25T10:45:44.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25303" PostTypeId="2" ParentId="25263" CreationDate="2016-10-25T10:53:30.913" Score="2" Body="&lt;p&gt;The special structure of your matrix is easily exploited by a custom made&#xA;$QR$ factorization based on Givens rotations. This leads to a method of $O(n)$ complexity. Below is a Octave/Matlab code, which solves $Mx=rhs$ for cyclic bidiagonal matrices $M$ of size $n \ge 2$ .&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Create a cyclic bidiagonal testmatrix&#xA;a=[2 -3 -1 5 3]';&#xA;b=[7 2 -4 -1 6]';&#xA;rhs=[-1 -3 2 6 2]';&#xA;n=5;&#xA;&#xA;% Full matrix solution&#xA;M=diag(a)+diag(b(1:n-1),1);  M(n,1)=b(n);&#xA;x_full_matrix=M\rhs;&#xA;res_nrn_full_matrix=norm(M*x_full_matrix-rhs)&#xA;&#xA;% QR factorization using Givens rotations (see Golub and Van Loan), without &#xA;% forming the full Matrix M. M is transformed to an upper triangular &#xA;% matrix R (which is also not formed as a full matrix)&#xA;% a and b are overwritten. d stores fill-in entries of R&#xA;c=zeros(n-1,1);&#xA;s=zeros(n-1,1);&#xA;d=zeros(n-2,1);&#xA;&#xA;z=b(n);&#xA;for i=1:n-1,&#xA;  % Compute Givens rotation&#xA;  if z==0, c(i)=1; s(i)=0;&#xA;  else&#xA;    if abs(z)&amp;gt;abs(a(i)), tau=-a(i)/z; s(i)=1/sqrt(1+tau*tau); c(i)=s(i)*tau;&#xA;    else                 tau=-z/a(i); c(i)=1/sqrt(1+tau*tau); s(i)=c(i)*tau;&#xA;    end&#xA;  end&#xA;  % Apply Givens rotation&#xA;  a(i)=c(i)*a(i)-s(i)*z;&#xA;  a_n=a(n);&#xA;  if i==n-1, break; end&#xA;  z   =s(i)*b(i);&#xA;  b(i)=c(i)*b(i);&#xA;  a(n)=c(i)*a_n;&#xA;  d(i)=-s(i)*a_n;&#xA;end&#xA;a(n)  =s(n-1)*b(n-1)+c(n-1)*a_n;&#xA;b(n-1)=c(n-1)*b(n-1)-s(n-1)*a_n;&#xA;&#xA;% Apply Givens rotations to rhs&#xA;x=rhs;&#xA;for i=1:n-1,&#xA;  x_i=x(i);&#xA;  x(i)=c(i)*x_i-s(i)*x(n);&#xA;  x(n)=s(i)*x_i+c(i)*x(n);&#xA;end&#xA;&#xA;% Triangular solve&#xA;x(n)=a(n)\x(n);&#xA;x(n-1)=a(n-1)\(x(n-1)-b(n-1)*x(n));&#xA;for i=n-2:-1:1,&#xA;  x(i)=a(i)\(x(i)-d(i)*x(n)-b(i)*x(i+1));&#xA;end&#xA;&#xA;% Check the answer&#xA;res_nrm_cycl_bidiag_QR=norm(M*x-rhs)&#xA;diff=norm(x_full_matrix-x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obviously, one should translate this Octave/Matlab code &#xA;into Fortran or C etcetera, if computational efficiency is important.&#xA;Even for small $n$, for example, $n=100$, such an implementation of this method maybe much faster than a general sparse $LU$ factorization or a dense LAPACK/BLAS $LU$ factorization.&lt;/p&gt;&#xA;" OwnerUserId="5286" LastEditorUserId="5286" LastEditDate="2016-10-25T10:58:50.640" LastActivityDate="2016-10-25T10:58:50.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25304" PostTypeId="1" CreationDate="2016-10-25T11:25:26.237" Score="5" ViewCount="2211" Body="&lt;p&gt;I have to solve a least squares problem:&#xA;$$&#xA;x=\arg \min\|Ax-b\|&#xA;$$&#xA;where $A$ is a $m\times n$ matrix, $m&amp;gt;n$, and $b\in\mathbb{R}^m$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I always thought that doing this via QR factorization is better than solving the normal equations &#xA;$$&#xA;A^*Ax=A^*b&#xA;$$&#xA;directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am now in the situation where the columns of $A$ are close to orthogonal and my code is significantly faster (factor 10) when using normal equation. Just to be sure I am doing nothing wrong, is this a reasonable result? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;($A$ has size $20000\times 2000$. Computing the Gramian matrix takes longer than solving the normal equation)&lt;/p&gt;&#xA;" OwnerUserId="7789" LastActivityDate="2016-10-26T16:13:04.647" Title="Least Squares: Numerically, is solving normal equations okay for nice matrices?" Tags="|linear-algebra|least-squares|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25305" PostTypeId="2" ParentId="12867" CreationDate="2016-10-25T11:55:57.987" Score="2" Body="&lt;p&gt;The variational or weak formulation of the biharmonic problem reads as : &#xA;find $u \in V_D$ such that&#xA;$$ a(u,v) = \ell(v), \quad \forall v \in V_0,$$&#xA;where the bilinear and linear forms are given by &#xA; $$a(u,v) = \int_\Omega \Delta u \Delta v \,d\Omega \quad  \text{and} \quad  \ell(v) = \int_\Omega f v\, d\Omega,$$&#xA;and the hyperplane and test spaces are given by $V_D :=\{ v \in H^2(\Omega), v = g_1, \frac{\partial v}{\partial n} = g_2 \quad on \quad \partial \Omega\}$ and &#xA;$V_0 :=\{ v \in H^2(\Omega), v = 0,   \frac{\partial v}{\partial n} = 0 \quad on \quad \partial \Omega\}.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The higher order derivatives on the $\partial \Omega$ are actually the Neumann boundary conditions. These are not given hence your choice of test function is enough. This is a biharmonic problem with Dirichlet boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="7984" LastEditorUserId="7984" LastEditDate="2016-10-26T21:15:49.377" LastActivityDate="2016-10-26T21:15:49.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25306" PostTypeId="2" ParentId="25304" CreationDate="2016-10-25T13:48:36.540" Score="7" Body="&lt;p&gt;For solving the least squares problem in general as principal methods there are (matrix $A$ with full rank):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;solve the system of normal equations $A^{T}Ax = A^{T}b$&lt;/li&gt;&#xA;&lt;li&gt;use QR factorization&lt;/li&gt;&#xA;&lt;li&gt;use SVD decomposition&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Generally speaking the QR factorization is a method with a good balancing between accuracy and computational cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the normal equations is possible, but it is often avoided due to condition number. The matrix of the system is $A^{T}A$, and it has got &#xA;$$ K_{A^{T}A} = (K_{A})^{2} $$&#xA;where $K$ is the condition number.&#xA;So if your matrix has got a good $K$, the normal equations are OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the matrix $A^{T}A$ is symmetric and definite positive (the eigenvalues are $\sigma^{2}_{i} &amp;gt; 0$ with  $\sigma_{i}$ singular value of $A$) so to solve directly the system you can use the Cholesky factorization.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="4405" LastEditDate="2016-10-26T08:38:54.307" LastActivityDate="2016-10-26T08:38:54.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25308" PostTypeId="2" ParentId="12867" CreationDate="2016-10-25T17:24:40.077" Score="5" Body="&lt;p&gt;I decided to expand my earlier comment into an answer. I'd suggest using the Morley element that uses $P_2$ basis in each element and the degrees of freedom are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;values at vertices&lt;/li&gt;&#xA;&lt;li&gt;normal derivatives at edge midpoints&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It is probably the easiest element for biharmonic problem implementation wise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example using &lt;a href=&quot;http://github.com/kinnala/sp.fem&quot; rel=&quot;nofollow noreferrer&quot;&gt;sp.fem library&lt;/a&gt; (version 8019aa7):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from spfem.mesh import MeshTri                                                                                                                                                                                                           &#xA;from spfem.asm import AssemblerGlobal                                                                                                                                                                                                    &#xA;from spfem.element import ElementGlobalMorley                                                                                                                                                                                            &#xA;from spfem.utils import direct,const_cell                                                                                                                                                                                                &#xA;import numpy as np                                                                                                                                                                                                                       &#xA;&#xA;# define mesh                                                                                                                                                                                                                            &#xA;m=MeshTri()                                                                                                                                                                                                                              &#xA;m.refine(5)                                                                                                                                                                                                                              &#xA;&#xA;# define element and assemble the stiffness matrix and load vector                                                                                                                                                                       &#xA;e1=AbstractElementMorley()                                                                                                                                                                                                                 &#xA;a=AssemblerAbstract(m,e1)                                                                                                                                                                                                                  &#xA;&#xA;def bilinear(ddu,ddv):                                                                                                                                                                                                                   &#xA;    return ddu[0][0]*ddv[0][0]+\                                                                                                                                                                                                         &#xA;           ddu[0][1]*ddv[0][1]+\                                                                                                                                                                                                         &#xA;           ddu[1][0]*ddv[1][0]+\                                                                                                                                                                                                         &#xA;           ddu[1][1]*ddv[1][1]                                                                                                                                                                                                           &#xA;&#xA;def F(x):                                                                                                                                                                                                                                &#xA;    return 50                                                                                                                                                                                                                            &#xA;&#xA;A=a.iasm(bilinear)                                                                                                                                                                                                                       &#xA;f=a.iasm(lambda v,x: F(x)*v)                                                                                                                                                                                                             &#xA;&#xA;# find boundary and interior DOFs                                                                                                                                                                                                        &#xA;bnd_nodes=m.boundary_nodes()                                                                                                                                                                                                             &#xA;bnd_facets=m.boundary_facets()                                                                                                                                                                                                           &#xA;bnd_facets_mid=0.5*(m.p[:,m.facets[0,bnd_facets]]+\                                                                                                                                                                                      &#xA;                    m.p[:,m.facets[1,bnd_facets]])                                                                                                                                                                                       &#xA;&#xA;d1=a.dofnum_u.getdofs(N=bnd_nodes)                                                                                                                                                                                                       &#xA;d2=a.dofnum_u.getdofs(F=bnd_facets)                                                                                                                                                                                                      &#xA;&#xA;I=np.setdiff1d(np.arange(a.dofnum_u.N),np.union1d(d1,d2))                                                                                                                                                                                &#xA;&#xA;# set boundary conditions                                                                                                                                                                                                                &#xA;x=np.zeros(A.shape[0])                                                                                                                                                                                                                   &#xA;&#xA;def G1(x,y):                                                                                                                                                                                                                             &#xA;    return 0.05*np.sin(4*np.pi*y)                                                                                                                                                                                                        &#xA;&#xA;def G2(x,y):                                                                                                                                                                                                                             &#xA;    return np.sin(4*np.pi*x)                                                                                                                                                                                                             &#xA;&#xA;# check orientation of boundary normals from sign of jacobian determinant                                                                                                                                                                &#xA;tris=np.ones(m.t.shape[1])                                                                                                                                                                                                               &#xA;flip=np.nonzero((a.mapping.detDF(np.array([[0]]))&amp;lt;0).flatten())[0]                                                                                                                                                                       &#xA;tris[flip]=-1.0                                                                                                                                                                                                                          &#xA;ori=tris[m.f2t[0,bnd_facets]]                                                                                                                                                                                                            &#xA;&#xA;x[d1]=G1(m.p[0,bnd_nodes],m.p[1,bnd_nodes])                                                                                                                                                                                              &#xA;x[d2]=ori*G2(bnd_facets_mid[0,:],bnd_facets_mid[1,:])                                                                                                                                                                                    &#xA;&#xA;# solve the system                                                                                                                                                                                                                       &#xA;x=direct(A,f,I=I,x=x)                                                                                                                                                                                                                    &#xA;&#xA;# draw the solution                                                                                                                                                                                                                      &#xA;xverts=x[a.dofnum_u.n_dof[0,:]]                                                                                                                                                                                                          &#xA;m.plot3(xverts)                                                                                                                                                                                                                          &#xA;m.show()    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The solution looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zlKFO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zlKFO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that both $g_1$ and $g_2$ are easily defined as Dirichlet-type boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastEditorUserId="10160" LastEditDate="2016-11-07T08:51:33.350" LastActivityDate="2016-11-07T08:51:33.350" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25309" PostTypeId="1" AcceptedAnswerId="25599" CreationDate="2016-10-25T18:36:07.697" Score="0" ViewCount="1111" Body="&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; (&lt;em&gt;copying from my comment&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's consider the inverse problem when I need to transfer velocities from particles to the grid (inverse bilinear interpolation). How'd I transfer a particle's x-velocity to the velocities located at cell's face in the Q region? Would the particle's velocity be distributed among the 2 face-velocities (as opposed to 4 face velocities in non-boundary locations)?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I'm implementing staggered grid for a &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle-in-cell&quot; rel=&quot;nofollow noreferrer&quot;&gt;PIC&lt;/a&gt; simulation (pressures in the center of a cell, velocities on faces) and I'm trying to find out how to interpolate the velocities. To simplify my problem, let's consider the 2D case;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/i4jJC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i4jJC.png&quot; alt=&quot;PIC staggered grid&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;red dot = pressure (irrelevant to my question)&lt;/li&gt;&#xA;&lt;li&gt;blue bar = the x-axis component of velocity&lt;/li&gt;&#xA;&lt;li&gt;green bar = the y-axis component of velocity&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If I was to implement the grid based on the above image, I'd store 3 arrays:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;3x4&lt;/code&gt; array for the (blue) x-velocities&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;4x3&lt;/code&gt; array for the (green) y-velocities&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;3x3&lt;/code&gt; array for the (red) pressures&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem with these array's sizes is interpolation of velocities at the grid boundaries – imagine you'd want to interpolate the (blue) x-velocity (from the &lt;code&gt;3x4&lt;/code&gt; array) for particles that are located in the lower half of the bottom cells (the lower half of a cell is the part of the cell, that is below the blue bar representing x-velocities - &lt;em&gt;take a look at the second image below – it's the area marked as &lt;code&gt;Q&lt;/code&gt;&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I'd want to interpolate the x-velocity in the center (non-boundary) cells, everything would be OK – I'd choose 4 nearest (blue) velocities for a given position in the grid and based on the 4 velocities, I'd bilinearly interpolate the velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However, when I'd try to interpolate in the lower-half of the bottom cells, I'd no longer have 4 velocities to interpolate from – only the 2. The interpolation would degrade from bilinear into linear!&lt;/strong&gt; (And I suppose that's incorrect implementation of staggered grid interpolation.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The obvious fix would be to store the velocities on grid's &lt;strong&gt;verticies&lt;/strong&gt; instead of on faces. The velocity for a face could then be linearly interpolated from 2 neighboring verticies when needed. &lt;strong&gt;Is this the conventional/preferred/best way how to solve interpolation of velocities on staggered grid?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KYlJn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KYlJn.jpg&quot; alt=&quot;Velocities stored on verticies, the face velocities are interpolated when needed.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;green bars = x-velocities stored on grid's verticies&lt;/li&gt;&#xA;&lt;li&gt;orange bars = linearly interpolated x-velocities from 2 green bars&lt;/li&gt;&#xA;&lt;li&gt;yellow points = the sample point for which velocity needs to be interpolated&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Q&lt;/code&gt; = the lower(upper)-half of the bottom(top) cells&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;A&lt;/code&gt; = the sample point in the &lt;code&gt;Q&lt;/code&gt; region&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21183" LastEditorUserId="21183" LastEditDate="2016-10-25T20:00:59.093" LastActivityDate="2016-11-18T12:55:57.533" Title="Interpolation of velocities on staggered grid (in PIC)" Tags="|finite-difference|interpolation|navier-stokes|grid|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25310" PostTypeId="2" ParentId="25279" CreationDate="2016-10-25T19:24:11.887" Score="1" Body="&lt;p&gt;If that is what you &lt;em&gt;really&lt;/em&gt; want, there is MPI_COMM_SPAWN(...)( and the more general MPI_COMM_SPAWN_MULTIPLE(...)). It can be used to start and  communicate with a second (separate) MPI program. &#xA;You would have to split your code into two sections, the initial serial part and then the subsequent parallel part that will be started by MPI_COMM_SPAWN(...)  &lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2016-10-25T19:24:11.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25311" PostTypeId="1" CreationDate="2016-10-25T20:26:01.937" Score="1" ViewCount="2175" Body="&lt;p&gt;I'm trying to take data in numpy arrays, interpolate them and retrieve this function to differentiate/integrate at different stages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how I got here, data included:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;x = np.array([  50.,   55.,   60.,   65.,   70.,   75.,   80.,   85.,   90.,&#xA;         95.,  100.,  105.,  110.,  115.,  120.,  125.,  130.,  135.,&#xA;        140.,  145.,  150.,  155.,  160.,  165.,  170.,  173.,  174.,&#xA;        175.,  176.,  177.,  178.,  179.,  180.,  181.,  182.,  183.,&#xA;        184.,  185.,  186.,  187.,  188.,  189.,  190.,  191.,  192.,&#xA;        193.,  194.,  195.,  196.,  197.,  198.,  199.,  200.,  201.,&#xA;        202.,  203.,  204.,  205.,  206.,  207.,  208.,  209.,  210.,&#xA;        211.,  212.,  213.,  214.,  215.,  216.,  217.,  218.,  219.,&#xA;        220.,  221.,  222.,  223.,  224.,  225.,  226.,  227.,  228.,&#xA;        229.,  230.,  231.,  232.,  233.,  234.,  235.,  236.,  237.,&#xA;        238.,  239.,  240.,  245.,  250.,  255.,  260.,  265.,  270.,&#xA;        275.,  280.,  285.,  290.,  295.,  300.,  305.,  310.,  315.,&#xA;        320.,  325.])&#xA;&#xA;y = np.array([  1.00000000e-02,   1.00000000e-02,   1.00000000e-02,&#xA;         1.00000000e-02,   1.00000000e-02,   1.00000000e-02,&#xA;         1.00000000e-02,   1.50000000e-02,   2.00000000e-02,&#xA;         2.50000000e-02,   3.50000000e-02,   4.00000000e-02,&#xA;         5.00000000e-02,   6.00000000e-02,   8.00000000e-02,&#xA;         1.05000000e-01,   1.30000000e-01,   1.70000000e-01,&#xA;         2.20000000e-01,   2.75000000e-01,   3.50000000e-01,&#xA;         4.50000000e-01,   5.75000000e-01,   7.30000000e-01,&#xA;         9.40000000e-01,   1.08500000e+00,   1.14000000e+00,&#xA;         1.19500000e+00,   1.25500000e+00,   1.32000000e+00,&#xA;         1.38500000e+00,   1.45000000e+00,   1.52500000e+00,&#xA;         1.60000000e+00,   1.68000000e+00,   1.76500000e+00,&#xA;         1.84500000e+00,   1.94000000e+00,   2.03500000e+00,&#xA;         2.13500000e+00,   2.24000000e+00,   2.35000000e+00,&#xA;         2.47500000e+00,   2.59000000e+00,   2.72500000e+00,&#xA;         2.85000000e+00,   2.99000000e+00,   3.15000000e+00,&#xA;         3.29500000e+00,   3.46500000e+00,   3.63000000e+00,&#xA;         3.81500000e+00,   4.00500000e+00,   4.20500000e+00,&#xA;         4.41500000e+00,   4.62500000e+00,   4.86500000e+00,&#xA;         5.10500000e+00,   5.36000000e+00,   5.62000000e+00,&#xA;         5.89500000e+00,   6.18000000e+00,   6.49000000e+00,&#xA;         6.80500000e+00,   7.14000000e+00,   7.49500000e+00,&#xA;         7.87000000e+00,   8.26000000e+00,   8.68500000e+00,&#xA;         9.12500000e+00,   9.58500000e+00,   1.00800000e+01,&#xA;         1.06000000e+01,   1.11500000e+01,   1.17300000e+01,&#xA;         1.23400000e+01,   1.29850000e+01,   1.36650000e+01,&#xA;         1.43700000e+01,   1.51150000e+01,   1.58900000e+01,&#xA;         1.67000000e+01,   1.75450000e+01,   1.84200000e+01,&#xA;         1.93150000e+01,   2.02200000e+01,   2.11600000e+01,&#xA;         2.20800000e+01,   2.30300000e+01,   2.39950000e+01,&#xA;         2.49600000e+01,   2.59400000e+01,   2.69150000e+01,&#xA;         3.18400000e+01,   3.68050000e+01,   4.17550000e+01,&#xA;         4.67450000e+01,   5.17150000e+01,   5.66950000e+01,&#xA;         6.16750000e+01,   6.66600000e+01,   7.16350000e+01,&#xA;         7.66150000e+01,   8.15900000e+01,   8.65700000e+01,&#xA;         9.15500000e+01,   9.65300000e+01,   1.01520000e+02,&#xA;         1.06490000e+02,   1.11470000e+02])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With some assistance, I got to this point.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;from scipy.interpolate import UnivariateSpline&#xA;&#xA;y_spl = UnivariateSpline(x,y,s=0,k=4)&#xA;x_range = np.linspace(x[0],x[-1],1000)&#xA;&#xA;plt.plot(x,y,'ro',label = 'data')&#xA;plt.plot(x_range,y_spl(x_range))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YlMEA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YlMEA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems like a reasonably good fit. I want to take this polynomial and retrieve its results symbolically, because my numerical derivatives appear unnatural, even for this spline.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y_spl_2d = y_spl.derivative(n=2)&#xA;plt.plot(x_range,y_spl_2d(x_range))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UhZNS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UhZNS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think even visually, this second derivative just doesn't cut it. The initial function appears pretty &quot;smooth&quot; in a math-sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What are my options from here?&lt;/strong&gt; Can I retrieve the initial function symbolically and perform the derivative this way (analytically?). What other numerical procedures can I implement to smooth my second derivative? It doesn't appear to me at any point that my second derivative should be negative (and in a modeling sense with what I'm physically modeling, it should never be negative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I retrieve the polynomial from the fit, or can I make numerical modifications to smooth my second derivative?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: the goal is to retrieve a probability density function. I would also like to restrict this final derivative to be strictly positive and integrate to 1, but I think my problems are bigger than this right now. Those are secondary issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the first derivative, which looks like a very fine-looking &lt;a href=&quot;https://en.wikipedia.org/wiki/Cumulative_distribution_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;CDF&lt;/a&gt;, so I'm becoming more and more sure that my problems are numerical. Should I interpolate this result, as well, and differentiate? I'm sure these problems are very common - what are my next steps.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y_spl_2d = y_spl.derivative(n=1)&#xA;plt.plot(x_range,y_spl_2d(x_range), 'ro')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CM5yk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CM5yk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22112" LastActivityDate="2016-11-24T22:02:43.207" Title="Python - Differentiating Cubic Spline numerically or analytically" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25312" PostTypeId="2" ParentId="25311" CreationDate="2016-10-25T22:01:44.283" Score="2" Body="&lt;p&gt;The scipy spline interpolation routine can create a smoothed spline that doesn't exactly interpolate the given points but which trades off smoothness against how closely it interpolates noisy points.  You could turn up the smoothing to get a more stable result.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-10-25T22:01:44.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25313" PostTypeId="1" CreationDate="2016-10-26T05:35:00.877" Score="1" ViewCount="126" Body="&lt;p&gt;I am trying to parallelize a program with CUDA that has irregular memory access. To explain what I am talking about, I have attached two figures. In the first figure, I have plotted the indices of the output and input arrays that needs to be accessed during each loop iteration. Obviously I have plotted only a few iterations (the total number is actually in millions). For the loop iterations in between the black horizontal lines, I assign to the same element in the output array. It can be seen that the number of times this happens for different elements in the output array is varying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JHsIZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JHsIZ.png&quot; alt=&quot;Memory access patterns with iterations&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BuYrE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BuYrE.png&quot; alt=&quot;Memory access patters (zoomed)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the next picture which is a zoom in of the first one, it can be seen that I need to access four elements of the input array for each iteration. These can be quite close together or quite far apart. I have only plotted the case when the number of elements in the input array are in the hundreds. This can be in tens of thousands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a first naiive approach for parallelising this, I can read all the elements in the input array into the shared memory. Then I can distribute the iterations over the threads. This way I only need to to reads/writes from the global memory and if I am careful in how I assign the work, these can be coalesced too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the problem comes when the input array don't fit into the shared memory. As you can see, for some of the iterations, I need to deal with elements quite far apart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is this, for general, large sized (can't fit all into the shared memory) problems, what sort of algorithms or techniques can I apply to make this efficient? I have read around a bit regarding irregular programs but didn't find much that is directly applicable to my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I already have this working very well with MPI. Doing this on CUDA is a bit of a curiosity thing.&lt;/p&gt;&#xA;" OwnerUserId="22115" LastActivityDate="2016-10-26T05:35:00.877" Title="Irregular data access on CUDA" Tags="|parallel-computing|cuda|" AnswerCount="0" CommentCount="3" ClosedDate="2019-06-11T23:28:33.123" ContentLicense="CC BY-SA 3.0" />
  <row Id="25314" PostTypeId="1" AcceptedAnswerId="25342" CreationDate="2016-10-26T07:03:04.847" Score="1" ViewCount="3465" Body="&lt;p&gt;The title says is all. This question is in the contest of an incompressible Navier-Stokes solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I am currently working on a new solver while referring myself to an older code for guidance. The use of high-order methods - WENO5 for convection and 4th order finite difference other derivatives - is compulsory. My advisor wants me to use a staggered grid, which complicates the problem but more importantly, will add an additional ~15% computing for each time step (e.g. $\dfrac{\partial u}{\partial x}$ needs to be computed on the $u, v, w$ and pressure points instead of just one grid point for a regular grid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I remember the first advantage of the staggered grid was to avoid checkerboarding oscillatory solutions and pressure-velocity coupling. But when using higher order methods, doesn't that emilinate the problem automatically? Also, I think I am willing to use staggered grids if they provide some real numerical advantage over collocated grids.&lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="16329" LastEditDate="2016-10-26T07:09:35.370" LastActivityDate="2016-10-27T11:43:32.870" Title="Is there an advantage of using a staggered grid over a regular one when combined with high order methods?" Tags="|fluid-dynamics|computational-physics|grid|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25315" PostTypeId="1" CreationDate="2016-10-26T09:07:52.057" Score="3" ViewCount="595" Body="&lt;p&gt;I am solving a first-order ODE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \rho }{\partial t} = -a \rho^2 + b |A(t)|^2 \rho +c|A(t)|^{2m}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the evolution of the plasma density in the presence of a laser pulse (complex amplitude A, which is normalized). The coefficients a, b and c depend on the material and the initial laser intensity. Typically, these coefficients are very large (for example a = 1e13, b=2e16 c=1e7). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I solve this equation (in Python, and I also played with C) with the conventional Runge-Kutta 4th order method (also tried 6th order); here is a little bit of code (quite standard, nothing fancy):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for l in range(Nr):&#xA;    mag= np.abs(Uo[:,l] )**2 &#xA;&#xA;    for k in range(1,Nt):  &#xA;        k1 = dT*(b*rho[k-1,l]*mag[k-1] + c*mag[k-1]**m - a*rho[k-1,l **2.0 )   &#xA;&#xA;        tmp1=0.5  * ( mag[k]+mag[k-1]) &#xA;        tmp2=rho[k-1,l] + 0.5*k1      &#xA;        k2 = dT * (b*tmp2 * tmp1 + c*tmp1**m- a* tmp2**2.0  )          &#xA;&#xA;        tmp2=rho[k-1,l] + 0.5*k2    &#xA;        k3 = dT * (b*tmp2 * tmp1 + c*tmp1**m- a* tmp2**2.0 )           &#xA;&#xA;        tmp2=rho[k-1,l] + k3        &#xA;        k4 = dT * (b*tmp2 * mag[k] + c*mag[k]**m- a* tmp2**2.0 )   &#xA;&#xA;        rho[k,l]=   rho[k-1,l] + (k1/6. + k2/3. + k3/3. + k4/6.)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also loop over space (Nr) because I have a time and space dependent plasma density. Since I discretized my time axis (with a total of Nt points, dT is usually 1e-16s), I do not use adaptive step size control (besides, I want this code to be fast).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if I do this integration with low power, everything works fine, and I get a nice result (image showing the temporal evolution of plasma density $\rho$ for low power, at one space point):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qPuhJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qPuhJ.png&quot; alt=&quot;temporal evolution of plasma density $\rho$ for low power&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I increase the pulse power, my Runge Kutta solution blows up, and I get NaNs. Reason is, that for high powers (say 1e9 Watts), the coefficients become huge (for example a = 1e13, b=2e52,  c=1e13).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&#xA;Do you have an idea how I can still get a result, without using step size control or using a finer grid? I wouldn't mind using approximations.&#xA;I am really thankful for any help:)&lt;/p&gt;&#xA;" OwnerUserId="22121" LastActivityDate="2016-10-26T12:43:17.770" Title="Runge Kutta solution blows up for a first order ODE with very large coefficients" Tags="|ode|runge-kutta|numerical-modelling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25317" PostTypeId="2" ParentId="25315" CreationDate="2016-10-26T10:58:02.827" Score="5" Body="&lt;p&gt;Though the following will not necessarily solve your problem, it is highly recommended to implement the ode dimensionless such that all relevant quantities are around 1 to stabilize solvers. The result is to be scaled back accordingly. This will also give you a hint what goes wrong in terms of the size of the coefficients or step size in the discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in your case, try:&#xA;$\rho = \hat{\rho} \rho_0, t = \hat{t} t_0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \hat{\rho}}{\partial \hat{t}}=-\hat{a}\hat{\rho}^2+\hat{b}|A(t_0\hat{t})|^2 \hat{\rho}+\hat{c}|A(t_0\hat{t})|^{2m}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\hat{a}=at_0\rho_0,\hat{b}=b t_0, \hat{c}=c \frac{t_0}{\rho_0}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way, when your expected plasma density increases because of larger laser powers, you pick larger $\rho_0$ and scale down $\hat{a}, \hat{b}, \hat{c}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g.&#xA;You can always pick $\hat{b}=1\Rightarrow t_0=\frac{1}{b}, \hat{a}=1 \Rightarrow \rho_0=\frac{b}{a}, \hat{c}=\frac{c a}{b^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for your given values (a = 1e13, b=2e52, c=1e13), you will probably need very small time steps, but all $\hat{a}=\hat{b}=1,\hat{c}=25\cdot 10^{-80}$, in the scaled version. So this should not blow up and the c term can be neglected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of the limit $c\rightarrow 0$, you can even find a analytical solution for verification of your solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\hat{\rho}(\hat{t})=\frac{e^{\int_1^\hat{t} |A(t_0 \xi)|d\xi}}{C_1 -\int_1^\hat{t}-e^{\int_1^\zeta |A(t_0 \xi)|d\xi}d\zeta}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $C_1$ is the integration constant.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2016-10-26T12:43:17.770" LastActivityDate="2016-10-26T12:43:17.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25320" PostTypeId="1" AcceptedAnswerId="25330" CreationDate="2016-10-26T12:29:43.003" Score="0" ViewCount="124" Body="&lt;p&gt;I have solved a PDE with an analytical equation. Through operator splitting I divided the PDE into one PDE and one ODE, using a sequential approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally for different $dt$, I got euclidian norm of the difference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Euclidian norm&lt;/strong&gt;&lt;br&gt;&#xA;$dt_1$  $error_1$&lt;br&gt;&#xA;$dt_2$  $error_2$&lt;br&gt;&#xA;$dt_3$  $error_3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know the order of my numerical scheme, is this possible?&#xA;I have seen the following formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (p+1)_\text{estimated} = \frac{\log_{10}(error_1) - \log_{10}(error_2) }{ \log_{10}(dt_1) - \log_{10}(dt_2) } $$&lt;/p&gt;&#xA;" OwnerUserId="22125" LastEditorUserId="22125" LastEditDate="2016-10-26T17:11:43.013" LastActivityDate="2016-10-26T18:16:02.153" Title="how to estimate convergence orders? what about this formula?" Tags="|numerical-analysis|error-estimation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25321" PostTypeId="2" ParentId="25304" CreationDate="2016-10-26T13:02:17.040" Score="2" Body="&lt;p&gt;In a comment you state that the condition number of the system of the normal &#xA;equentions is as small as $\kappa=5$. For such a well condtioned problem it might be a good idea to try the conjugate gradient &#xA;iterative method. This method should converge fast, according to the well known conjugate gradient error&#xA;estimate $$||x_*-x_m||_A \le 2 \left[ \frac{\sqrt \kappa -1}{\sqrt \kappa +1} \right]^m ||x_*-x_0||_A \, ,$$ &#xA;where $A$ is not your $A$, but the matrix $A^*A$ from the normal equations (the one with condition number 5).&#xA;Note that $\left[ \frac{\sqrt 5 -1}{\sqrt 5 +1} \right]^{40}=$1.9e-17 .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An advantage of the conjugate gradient method is, that there is no need to compute $A^* A$ explicitly, &#xA;only matrix vector products of the form $q_i=A^* Ap_i$ are needed. These are computed as &#xA;$q_i=A^* (Ap_i)$ .  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also Saad's book: &lt;em&gt;Iterative methods for sparse linear systems&lt;/em&gt;, &#xA;chapter 6 and 8.&lt;/p&gt;&#xA;" OwnerUserId="5286" LastActivityDate="2016-10-26T13:02:17.040" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25322" PostTypeId="1" AcceptedAnswerId="25332" CreationDate="2016-10-26T13:05:11.157" Score="1" ViewCount="112" Body="&lt;p&gt;If I have a system of equations as follows (where $i = \sqrt{-1}$): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac {\partial A}{\partial t} = iA^*B - A \tag{1} \\&#xA;$$&#xA;$$&#xA;\frac {\partial B}{\partial z} = AB^* - B \tag{2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the method of lines, in general, I discretize space and solve the system for all time for a particular $z_k$—this transforms the PDEs into a system of ODEs. But in the above case, I have a spatial derivative and also a time derivative that determines the evolution of my system. Is the particular method of discretizing both time and space and solving over such a grid with different partial derivatives (e.g. $t, z$) given a particular name? Is such a method invalid for any particular reason when solving PDEs? I realize I will not have any intermediate values for $B$ when advancing in $t$, and lacking intermediate values for $A$ when advancing in $z$ when using a method such as a 4th order Runge-Kutta. This, I assume, will result in a poor approximation, but is such a method invalid for any reason? Does such a method have a particular name?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://imgur.com/a/qjFSg&quot; rel=&quot;nofollow noreferrer&quot;&gt;For further clarification, I have tried to draw a single iteration of what I am attempting to describe.&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="20407" LastActivityDate="2016-10-26T19:43:30.850" Title="Classification of method for solving PDEs" Tags="|pde|numerical-analysis|method-of-lines|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25323" PostTypeId="1" CreationDate="2016-10-26T13:41:10.613" Score="5" ViewCount="647" Body="&lt;p&gt;I'm using the finite element method to obtain the time response of a structure harmonically excited. I'm using a linear displacement function to obtain the stiffness matrix and the consistent mass matrix, i.e., I have this set of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[M]\{\ddot x\} + [K]\{x\} = \{F(t)\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to apply the boundary conditions on the $[M]$ and $[K]$ matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, I have found two ways:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Remove the rows/columns whose degree of freedoms are constrained&lt;/li&gt;&#xA;&lt;li&gt;Modify the matrices setting to zero the rows and columns and placing ones on the diagonal to maintain non-singularity.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I prefer the second way, since we can adjust it to account for non zero displacement constrains. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem here is that I'm using the Newmark method to integrate the system and the biggest difficult is that we are dealing with geometrical nonlinearities, which means $[K]$ is the tangent stiffness matrix. Since this is a nonlinear problem, we modify the Newmark method to account for the nonlinearities and use the Newton-Raphson method to find the equilibrium position at time $t(i+1)$ when marching forward in time. Usually, the Newton-Raphson method is fast enough to find the equilibrium position. However, if the initial estimate is not good enough, it will not converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The biggest problem here is that, when modifying the matrices, an error residue on any degree of freedom found by the Newton-Raphson is propagated and the final answer diverges. Any ideas on how to go around this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to keep it simple, without too much details on the Newmark method or the FEM method. If necessary, however, we can discuss it. &lt;/p&gt;&#xA;" OwnerUserId="22126" LastEditorUserId="9667" LastEditDate="2016-10-27T15:47:00.247" LastActivityDate="2024-02-11T15:28:23.130" Title="Boundary condtions on nonlinear FEM time integration" Tags="|finite-element|time-integration|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25325" PostTypeId="1" CreationDate="2016-10-26T16:01:42.747" Score="1" ViewCount="583" Body="&lt;p&gt;I'm trying to solve exercises of the book  &quot;Computational Geometry in C&quot; by O'Rourke. Could you please help me with this one?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Design an algorithm to find a line $L$ that:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;has all the points of a given set to one side&lt;/li&gt;&#xA;  &lt;li&gt;minimizes the sum of the perpendicular distances of the points to $L$   Assume a hull algorithm is available.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22130" LastEditorUserId="20417" LastEditDate="2016-10-26T23:26:50.363" LastActivityDate="2017-05-25T15:29:47.480" Title="Min supporting line for a set of points" Tags="|algorithms|computational-geometry|convex-hull|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25326" PostTypeId="2" ParentId="25304" CreationDate="2016-10-26T16:07:35.990" Score="1" Body="&lt;p&gt;The other answers already give good advice, so I want to offer a guess as to the cause of the unexpected speedup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving normal equations should take $\sim mn^2+13n^3$ flops (forming $A^*A$, then solving it with Cholesky factorization), and doing QR should take $2mn^2$ flops (for example: &lt;a href=&quot;https://seas.ucla.edu/~vandenbe/103/lectures/qr.pdf&quot; rel=&quot;nofollow&quot;&gt;https://seas.ucla.edu/~vandenbe/103/lectures/qr.pdf&lt;/a&gt;), which makes us expect that normal equations would be 2 times faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you say that you use &lt;code&gt;numpy.dot&lt;/code&gt;, and &lt;code&gt;numpy.dot&lt;/code&gt; is automatically parallelized (&lt;a href=&quot;https://scipy.github.io/old-wiki/pages/ParallelProgramming&quot; rel=&quot;nofollow&quot;&gt;https://scipy.github.io/old-wiki/pages/ParallelProgramming&lt;/a&gt;), it seems possible that if QR is &lt;em&gt;not&lt;/em&gt; parallelized, you would see a speedup of $2\times \text{number-of-threads}$, so a speedup of 10 might then be reasonable. But this multithreading speedup is usually not mentioned because the issue with squaring condition number is more important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To check, you'd need to find out what BLAS/LAPACK library your numpy is using (&lt;code&gt;numpy.show_config&lt;/code&gt;), and, to make sure, set the appropriate environment variable (like &lt;code&gt;OMP_NUM_THREADS&lt;/code&gt;) to 1 to check if it's really multithreading that causes this. There are already a number of questions on StackOverflow about how to do this.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-10-26T16:13:04.647" LastActivityDate="2016-10-26T16:13:04.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25327" PostTypeId="1" CreationDate="2016-10-26T16:26:55.133" Score="6" ViewCount="2866" Body="&lt;p&gt;I have a Triangular Plane T given by three points in 3D space $$P_1 = (x_1,y_1,z_1)\\ P_2 = (x_2,y_2,z_2) \\ P_3 = (x_3,y_3,z_3)$$ &#xA;I want to find the Shape Functions on this plane as  $N_1(x,y,z),N_2(x,y,z),N_3(x,y,z)$.&#xA;I am totally aware of finding the basis function for&lt;br&gt;&#xA;&lt;a href=&quot;http://kratos-wiki.cimne.upc.edu/index.php/Two-dimensional_Shape_Functions&quot; rel=&quot;noreferrer&quot;&gt;2D&lt;/a&gt; but I have no clue on how to do it for the triangular plane in 3D.&#xA;I hope I made my question clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="22110" LastEditorUserId="22110" LastEditDate="2016-10-26T16:51:37.493" LastActivityDate="2019-07-02T14:37:55.817" Title="Finding Shape Functions for a Triangle in 3D coordinate space" Tags="|finite-element|computational-geometry|basis-set|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25328" PostTypeId="2" ParentId="25323" CreationDate="2016-10-26T16:58:50.607" Score="2" Body="&lt;p&gt;Have you tried using SOR (successive Over Relaxation scheme). Newton Raphson will face sever issue if one of the following is occuring in your case. &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow&quot;&gt;Practicle Considerations for Newton Raphson&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22110" LastActivityDate="2016-10-26T16:58:50.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25329" PostTypeId="2" ParentId="25320" CreationDate="2016-10-26T17:42:01.897" Score="0" Body="&lt;p&gt;I am not 100% sure, though I would say that the answer to the question can be founded in the following &lt;a href=&quot;http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471929905.html&quot; rel=&quot;nofollow&quot;&gt;book&lt;/a&gt; (I have not found it), where apparently the definition of local truncation error is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ p =sup \{ q \in N : \lim_{\tau\rightarrow 0}\frac{E(\tau)}{\tau^{q+1}} = c &amp;lt; +inf\}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p is the order, E the error that depends on $$\tau$$ q a natural (I guess) and c the constant. to be consistent p&gt;0. Then you approach tau to the machine zero, and see the limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the reference for this formula? Also, I think this approach can be use online for local truncation error, so one step. My results are at the end of the simulation. So I would like to ask, what others ways do I have to found the order from my global errors? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks&lt;/p&gt;&#xA;" OwnerUserId="22125" LastActivityDate="2016-10-26T17:42:01.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25330" PostTypeId="2" ParentId="25320" CreationDate="2016-10-26T18:16:02.153" Score="1" Body="&lt;p&gt;The idea of estimating the order of numerical scheme is following. If you have only one discretization parameter, say $dt$, you start with an assumption that the error have the following character $e = C (dt)^p$ where $e$ is your preferred ``global'' error when comparing numerical solution with exact solution e.g. at a fixed time $T$ (i.e. you need different number of time steps with different $dt$). If you expect such behavior of your error then $p$ expresses the order of method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently&#xA;$$&#xA;e_1 = C (dt_1)^p \,, \quad e_2 = C (dt_2)^p \,.&#xA;$$&#xA;Then you make a simplification (an approximation) that $C$ is identical in both cases that is not true in general, but for enough small $dt$ this approximation is reasonable. Doing simple algebraic computations you obtain&#xA;$$&#xA;\frac{e_2}{e_1} = \left(\frac{dt_2}{dt_1}\right)^p&#xA;$$&#xA;that you can express as you wrote. &#xA;The simplest way to do this is to use $dt_2=dt_1/2$ when for small enough $dt_1$ you can estimate that $p=1$ if $e_2=e_1/2$ or $p=2$ if $e_2=e_1/4$ and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First remark - you typically do not (and do not want to) estimate local truncation error that in many cases is of order $p+1$. Secondly, you mention PDEs, so you typically have also another discretization step, e.g. $dx$ (or you solve one equation in splitting method with analytical methods?). Then your error will be dominated by lower order accuracy, so with no prior knowledge of order it is not easy to estimate it.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-10-26T18:16:02.153" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25331" PostTypeId="1" CreationDate="2016-10-26T18:16:31.340" Score="1" ViewCount="599" Body="&lt;p&gt;Why, you may ask?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Python sympy symbolic library provides solutions to single, linear Diophantine equations in terms of parametric variables. For instance,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{(t_0, -9*t_0 - 5*t_1 + 154, -5*t_0 - 3*t_1 + 77)}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Without worrying about coding details, how can one solve such a system of parametric expressions for values of the original variables constrained to be, for instance, greater than zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any opensource products?&lt;/p&gt;&#xA;" OwnerUserId="22133" LastActivityDate="2016-11-27T20:21:57.773" Title="What are the prominent algorithms for solving systems of linear inequalities?" Tags="|linear-algebra|algorithms|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25332" PostTypeId="2" ParentId="25322" CreationDate="2016-10-26T19:43:30.850" Score="4" Body="&lt;p&gt;It is easier to see what is happening if you put the $\partial B/\partial z$ term also on the right hand side: Then, you have one equation with a time derivative and one that does not. This is what we typically call a &quot;Differential Algebraic Equation&quot; (DAE), and it is the same as what is happening, for example, in the time dependent Stokes and Navier-Stokes equations. There are a good number of methods that implement efficient schemes for DAEs, such as projection methods.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-26T19:43:30.850" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25333" PostTypeId="2" ParentId="25327" CreationDate="2016-10-26T19:50:06.713" Score="6" Body="&lt;p&gt;This is actually quite simple. Let's say you have your (two-dimensional) reference triangle $\hat K=\left\{(\xi,\eta)\in {\mathbb R}^2: 0\le \xi \le 1, 0\le \eta\le 1, \xi \le (1-\eta)\right\}$. Then you know how to define the three basis functions on this triangle. They are:&#xA;$$&#xA;\hat N_1(\xi,\eta) = 1-\xi-\eta,\\&#xA;\hat N_2(\xi,\eta) = \xi,\\&#xA;\hat N_3(\xi,\eta) = \eta.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if you take the &lt;em&gt;mapping&lt;/em&gt; $(x,y,z)=\Phi(\xi,\eta)$ from the reference triangle $\hat K$ to the real triangle $K$, then you simply define&#xA;$$&#xA;  N_i(x,y,z) = \hat N_i(\Phi^{-1}(x,y,z)),&#xA;$$&#xA;or, equivalently but easier to compute,&#xA;$$&#xA;  N_i(\Phi(\xi,\eta)) = \hat N_i(\xi,\eta).&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-26T19:50:06.713" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25334" PostTypeId="1" AcceptedAnswerId="25345" CreationDate="2016-10-27T02:59:50.197" Score="1" ViewCount="2146" Body="&lt;p&gt;I have a usual vtk file with a mesh of triangles and a scalar velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to obtain this type of plots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kjNsb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kjNsb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;something like an &quot;elevation&quot;, but in paraview I just had obtained plane plots like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ydcaO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ydcaO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I obtain plots like the first one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&#xA;I Edited my message with an example of my vtk file. Note that the points of the mesh are in the plane XY (for that reason its third coordinate is zero) and I want to plot the first and second component of the velocity and pressuse, each one like an &quot;elevation&quot; (as the first image).&#xA;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# vtk DataFile Version 2.0&#xA;Stokes Solution 2D&#xA;ASCII&#xA;&#xA;DATASET UNSTRUCTURED_GRID&#xA;POINTS           13  DOUBLE&#xA;0.000000000000000E+000   0.000000000000000E+000  0.000000000000000E+000&#xA;1.00000000000000         0.000000000000000E+000  0.000000000000000E+000&#xA;1.00000000000000         1.00000000000000        0.000000000000000E+000&#xA;0.000000000000000E+000   1.00000000000000        0.000000000000000E+000&#xA;0.500000000000000        0.500000000000000       0.000000000000000E+000&#xA;0.000000000000000E+000   0.500000000000000       0.000000000000000E+000&#xA;0.500000000000000        0.000000000000000E+000  0.000000000000000E+000&#xA;1.00000000000000         0.500000000000000       0.000000000000000E+000&#xA;0.500000000000000        1.00000000000000        0.000000000000000E+000&#xA;0.250000000000000        0.250000000000000       0.000000000000000E+000&#xA;0.250000000000000        0.750000000000000       0.000000000000000E+000&#xA;0.750000000000000        0.250000000000000       0.000000000000000E+000&#xA;0.750000000000000        0.750000000000000       0.000000000000000E+000&#xA;&#xA;CELLS           16          64&#xA;   3       7      12       4&#xA;   3       5       9       4&#xA;   3      10       8       3&#xA;   3      10       3       5&#xA;   3       4       9       6&#xA;   3       6      11       4&#xA;   3       4      11       7&#xA;   3       4      12       8&#xA;   3       0       6       9&#xA;   3       5       0       9&#xA;   3       4       8      10&#xA;   3       5       4      10&#xA;   3       1       7      11&#xA;   3       6       1      11&#xA;   3       2       8      12&#xA;   3       7       2      12&#xA;&#xA; CELL_TYPES           16&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;&#xA;POINT_DATA           13&#xA;SCALARS presion double&#xA;LOOKUP_TABLE default&#xA;-4.59703889304213     &#xA; 3.82888363610970     &#xA; 3.81756522383386     &#xA;-4.60835730531561     &#xA; 0.197145644934616     &#xA;-4.59535605348466     &#xA; 0.195500198716605     &#xA; 3.82752829972580     &#xA; 0.187138058219474     &#xA;-2.12293352900423     &#xA;-2.12078991814064     &#xA; 2.21724715421349     &#xA; 2.21939076507648     &#xA;&#xA;VECTORS velocidad double&#xA;0.000000000000000E+000   0.000000000000000E+000  0.000000000000000E+000&#xA;8.41470984807897         0.000000000000000E+000  0.000000000000000E+000&#xA;4.54648713412841        -4.54648713412841        0.000000000000000E+000&#xA;0.000000000000000E+000  -8.41470984807897        0.000000000000000E+000&#xA;4.20725870743923        -4.20725889966961        0.000000000000000E+000&#xA;0.000000000000000E+000  -4.79425538604203        0.000000000000000E+000&#xA;4.79425538604203         0.000000000000000E+000  0.000000000000000E+000&#xA;7.38460262604129        -2.59034723999926        0.000000000000000E+000&#xA;2.59034723999926        -7.38460262604129        0.000000000000000E+000&#xA;2.39717406023838        -2.39717405652409        0.000000000000000E+000&#xA;1.81026401191967        -6.60460085064085        0.000000000000000E+000&#xA;6.60460083608821        -1.81026405008086        0.000000000000000E+000&#xA;4.98756220389105        -4.98756217656822        0.000000000000000E+000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17630" LastEditorUserId="17630" LastEditDate="2016-10-27T12:35:43.343" LastActivityDate="2016-10-27T15:52:54.540" Title="&quot;Elevation&quot; plot of a 2D plot - paraview vtk" Tags="|paraview|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25335" PostTypeId="1" CreationDate="2016-10-27T03:04:26.433" Score="1" ViewCount="341" Body="&lt;p&gt;I need to implement the following in python:&lt;/p&gt;&#xA;&lt;p&gt;For a given discrete time series &lt;span class=&quot;math-container&quot;&gt;$Z_t$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$t={0...T}$&lt;/span&gt;), find the smallest &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; such that:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$c\sum_{s=0}^t e^{[k(Z_t-Z_s)+m(t-s)]}  \geq \frac{p^*}{1-p^*} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$c,k,m$&lt;/span&gt; are constants and&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P^*=\int_0^{0.5} \frac{(1-2y)e^{-d/y}}{(1-y)^{1+d}y^{1-d}}dy=\int_{0.5}^{p^*} \frac{(2y-1)e^{-d/y}}{(1-y)^{1+d}y^{1-d}}dy $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(d is another constant)&lt;/p&gt;&#xA;&lt;p&gt;I need to implement this for each row of an &lt;span class=&quot;math-container&quot;&gt;$N\times w$&lt;/span&gt; array &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I implemented the sum in equation 1 above utilizing the fact&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ e^{[k(Zt-Zs)+m(t-s)]} = \frac{e^{kZt+mt}}{e^{kZs+ms}} $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;hence&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{s=0}^t e^{[k(Z_t-Z_s)+m(t-s)]} =e^{kZ_t+m_t}*\sum_{s=0}^t\frac{1}{e^{kZ_s+m_s}} $$&lt;/span&gt; which is reflected in use of cumsum in code below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def f(z):&#xA;    return ((1-2*z)*np.exp(-d/z))/(((1-z)**(1+d))*(z**(1-d)))&#xA;lhs=integrate.quad(f,0,0.5)&#xA;&#xA;def rhs(p):&#xA;    return integrate.quad(-f,0.5, p)&#xA;p_star= fsolve(rhs-lhs,0.75) # will depend on time series only indrectly when d will be optimized&#xA;&#xA;for i in np.arange(N):&#xA;    z=Z[i,:]  &#xA;    main = np.exp (kz+m*np.arange(w)) &#xA;    cumsum_t=np.cumsum(1/main)&#xA;    final_sum= main*cumsum_t &#xA;    t_solution= # index i where final_sum[i]&amp;gt; p_star/(1-p_star)) # not implemented yet&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any way I can vectorize this for &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt;? In this case &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is ~400,000 so vectorization would really help. &lt;code&gt;f(z)&lt;/code&gt; will be fine with vector inputs but I don't think &lt;code&gt;rhs(p)&lt;/code&gt; will be fine as it uses integration.&lt;/p&gt;&#xA;" OwnerUserId="22137" LastEditorUserId="23791" LastEditDate="2022-01-10T20:52:44.380" LastActivityDate="2022-01-10T20:52:44.380" Title="Vectorize function integration" Tags="|python|scipy|integration|vectorization|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="25337" PostTypeId="2" ParentId="25334" CreationDate="2016-10-27T06:52:55.713" Score="2" Body="&lt;p&gt;We can use the UNSTRUCTURED_GRID dataset attribute format in VTK. Here I'm going to provide you a very simple example. Suppose that you had a set of vertices of the triangle mesh and a scalar field valued at the vertices. For example, we firstly generate vertices like this: $(x_0, y_0) = (0,0)$ and &#xA;$$&#xA;(x_{j+1}, ~ y_{j+1}) = \Big( \cos(\frac{j\pi}{4}), ~ \sin(\frac{j\pi}{4}) \Big), ~~&#xA;j=\overline{0,7},&#xA;$$&#xA;which can be done by the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -le '$pi = 3.14159265358979323846 ; for ( $i=0; $i&amp;lt;8 ; $i++ ){ print cos($pi*$i/4).&quot;\t&quot;. sin($pi*$i/4)}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Secondly, adding values of $z_j=z(x_j,y_j)$, for instance, $z_0 = 1$ and $z_j = 0$ for $j \not = 0$, we now had the set of 9 points in 3D, where $z(x,y)$ is the scalar that we want to plot, and the mesh now is with 8 trianlges surrounding $(0,0)$. Lastly, make a file, say triangle_surf.vtk, with the following contents:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# vtk DataFile Version 2.0&#xA;Simple surface on a triangle mesh with vertices (xj,yj,zj) for j=0,8.&#xA;ASCII&#xA;DATASET UNSTRUCTURED_GRID&#xA;POINTS   9  double &#xA;   0                             0                       1.0&#xA;   1                             0                       0.0&#xA;   0.707106781186548             0.707106781186547       0.0&#xA;   6.12323399573677e-17          1                       0.0&#xA;   -0.707106781186547            0.707106781186548       0.0&#xA;   -1                            1.22464679914735e-16    0.0&#xA;   -0.707106781186548            -0.707106781186547      0.0&#xA;   -1.83697019872103e-16         -1                      0.0&#xA;   0.707106781186547             -0.707106781186548      0.0&#xA;&#xA;CELLS 8 32&#xA;   3  0  1  2&#xA;   3  0  2  3&#xA;   3  0  3  4&#xA;   3  0  4  5&#xA;   3  0  5  6&#xA;   3  0  6  7&#xA;   3  0  7  8&#xA;   3  0  8  1&#xA;&#xA;CELL_TYPES 8&#xA;   5 &#xA;   5 &#xA;   5 &#xA;   5 &#xA;   5 &#xA;   5 &#xA;   5&#xA;   5&#xA;&#xA;POINT_DATA 9&#xA;SCALARS z_values double&#xA;LOOKUP_TABLE default&#xA;   1.0&#xA;   0.0&#xA;   0.0&#xA;   0.0&#xA;   0.0&#xA;   0.0&#xA;   0.0&#xA;   0.0&#xA;   0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now openning the vtk file by paraview, you will get something like this&#xA;&lt;a href=&quot;https://i.stack.imgur.com/oFd7p.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oFd7p.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Since I am quite lazy to explain as well as I do not know what you had in hand and want to perform, any discussion in details is welcomed.&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;@yemino: sure. you just simply replace the 3rd column of coordinates, i.e. the z components, by the scalar values. For instance, to plot your presion data (presure?), we make this file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# vtk DataFile Version 2.0&#xA;Stokes Solution 2D&#xA;ASCII&#xA;&#xA;DATASET UNSTRUCTURED_GRID&#xA;POINTS           13  DOUBLE&#xA;0.000000000000000E+000   0.000000000000000E+000  -4.59703889304213     &#xA;1.00000000000000         0.000000000000000E+000   3.82888363610970     &#xA;1.00000000000000         1.00000000000000         3.81756522383386     &#xA;0.000000000000000E+000   1.00000000000000        -4.60835730531561     &#xA;0.500000000000000        0.500000000000000        0.197145644934616     &#xA;0.000000000000000E+000   0.500000000000000       -4.59535605348466     &#xA;0.500000000000000        0.000000000000000E+000   0.195500198716605     &#xA;1.00000000000000         0.500000000000000        3.82752829972580     &#xA;0.500000000000000        1.00000000000000         0.187138058219474     &#xA;0.250000000000000        0.250000000000000       -2.12293352900423     &#xA;0.250000000000000        0.750000000000000       -2.12078991814064     &#xA;0.750000000000000        0.250000000000000        2.21724715421349     &#xA;0.750000000000000        0.750000000000000        2.21939076507648     &#xA;&#xA;CELLS           16          64&#xA;   3       7      12       4&#xA;   3       5       9       4&#xA;   3      10       8       3&#xA;   3      10       3       5&#xA;   3       4       9       6&#xA;   3       6      11       4&#xA;   3       4      11       7&#xA;   3       4      12       8&#xA;   3       0       6       9&#xA;   3       5       0       9&#xA;   3       4       8      10&#xA;   3       5       4      10&#xA;   3       1       7      11&#xA;   3       6       1      11&#xA;   3       2       8      12&#xA;   3       7       2      12&#xA;&#xA; CELL_TYPES           16&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;       5&#xA;&#xA;POINT_DATA           13&#xA;SCALARS presion double&#xA;LOOKUP_TABLE default&#xA;-4.59703889304213     &#xA; 3.82888363610970     &#xA; 3.81756522383386     &#xA;-4.60835730531561     &#xA; 0.197145644934616     &#xA;-4.59535605348466     &#xA; 0.195500198716605     &#xA; 3.82752829972580     &#xA; 0.187138058219474     &#xA;-2.12293352900423     &#xA;-2.12078991814064     &#xA; 2.21724715421349     &#xA; 2.21939076507648 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you will get something like a 3D surface, here is it:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WWph3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WWph3.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Treat other scalar fields as the same way above.&lt;/p&gt;&#xA;" OwnerUserId="15175" LastEditorUserId="15175" LastEditDate="2016-10-27T15:52:54.540" LastActivityDate="2016-10-27T15:52:54.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25338" PostTypeId="2" ParentId="20927" CreationDate="2016-10-27T07:37:03.593" Score="2" Body="&lt;p&gt;The following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// g++ hg.cc -o hg -L/usr/local/lib -lgsl -lgslcblas -lm &amp;amp;&amp;amp; ./hg&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_complex.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_complex_math.h&amp;gt;&#xA;&#xA;// ----------------------------------------------------&#xA;gsl_complex operator/(   gsl_complex  z1 , const gsl_complex &amp;amp; z2) { &#xA;    return gsl_complex_div(z1, z2); &#xA;}&#xA;gsl_complex operator*(   gsl_complex  z1 , const gsl_complex &amp;amp; z2) { &#xA;    return gsl_complex_mul(z1, z2); &#xA;}&#xA;gsl_complex operator+(   gsl_complex  z1 , const gsl_complex &amp;amp; z2) { &#xA;    return gsl_complex_add(z1, z2); &#xA;}&#xA;gsl_complex operator+(   gsl_complex z2  , const double  &amp;amp; x ) { &#xA;    return gsl_complex_add( z2,  gsl_complex_rect(x,0)); &#xA;}&#xA;gsl_complex operator/(   gsl_complex z2  , const double  &amp;amp; x ) { &#xA;    return gsl_complex_div( z2,  gsl_complex_rect(x,0)); &#xA;}&#xA;// ----------------------------------------------------&#xA;&#xA;gsl_complex hy_ge( const gsl_complex &amp;amp; a &#xA;,                  const gsl_complex &amp;amp; b &#xA;,                  const gsl_complex &amp;amp; c &#xA;,                  const gsl_complex &amp;amp; z&#xA;,           unsigned int n = 100                   // accuracy&#xA;,           unsigned int i = 0                     // itteration step&#xA;,            gsl_complex t = gsl_complex_rect(1,0) // coefficient t for internal calculations &#xA;) {&#xA;&#xA;    gsl_complex  t_next = (a+i)*(b+i)/(c+i)/(i+1) * t;&#xA;&#xA;    return (i==n+1) ? gsl_complex_rect(0,0) : t*gsl_complex_pow_real(z,i)+hy_ge(a,b,c,z,n,i+1,t_next); &#xA;}&#xA;&#xA;// ----------------------------------------------------&#xA;int main (void)&#xA;{&#xA;&#xA;    gsl_complex a = gsl_complex_rect(  1,  0);&#xA;    gsl_complex b = gsl_complex_rect(  2,  0);&#xA;    gsl_complex c = gsl_complex_rect(  2,  2);&#xA;    gsl_complex z = gsl_complex_rect(0.2,0.1);&#xA;&#xA;    gsl_complex z_out = hy_ge(a,b,c,z);&#xA;&#xA;    printf( &quot; %.10f%+.10f i \n&quot;,  GSL_REAL( z_out ), GSL_IMAG( z_out ) ) ;&#xA;&#xA;    return 0;&#xA;}&#xA;// ----------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;calculates:&#xA;    1.1833113578-0.0657194591 i&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in Your case, You may set a=1: &lt;strong&gt;hy_ge(1,b,c,z);&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="22123" LastEditorUserId="22123" LastEditDate="2016-10-27T07:47:45.707" LastActivityDate="2016-10-27T07:47:45.707" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25339" PostTypeId="1" AcceptedAnswerId="25364" CreationDate="2016-10-27T08:49:08.967" Score="1" ViewCount="610" Body="&lt;p&gt;For the heat equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; u_t(t,x) = \nu u_{xx}(t,x)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $x \in [0,1]$ with boundary conditions $u(t,0) = u(t,1) = 0$ and initial value $u(0,x) = u_0(x)$ it is easy to show that the &quot;energy&quot; defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;E(t) = \frac{1}{2} \int_{0}^{1} u^2(t,x)~dx&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;decays over time, that is $E(t) \leq E(0)$. I wonder if there is any physical interpretation of the quantity $E$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of units, $u$ is temperature in Kelvin while the thermal diffusivity $\nu$ has units $m^2/s$ and is composed from&#xA;\begin{equation}&#xA;  \nu = \frac{k}{\rho c_p}&#xA;\end{equation}&#xA;where $k$ is the thermal conductivity in $W/(m*K)$, $\rho$ the density in $kg/m^3$ and $c_p$ the specific heat capacity in $J/(kg*K)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I figured out that $u$ is related to the internal energy in Joule per volume via&#xA;\begin{equation}&#xA;    I = c_p \rho u.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what is the interpretation of $E$, which is related to $u^2$?&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="16921" LastEditDate="2016-10-28T07:14:03.580" LastActivityDate="2016-10-28T16:41:57.653" Title="Physical interpretation of L2 norm of heat equation solution" Tags="|heat-transfer|l2-norm|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25340" PostTypeId="1" CreationDate="2016-10-27T10:34:37.777" Score="1" ViewCount="231" Body="&lt;p&gt;Do you know some interesting literature concerning the comparison of Simplex and Interior-Point-Methods referring to linear optimization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read about the possibility of combining both of them but couldn't find much literature on that either.  &lt;/p&gt;&#xA;" OwnerUserId="22143" LastEditorUserId="22143" LastEditDate="2016-10-27T19:27:59.037" LastActivityDate="2016-10-27T20:08:52.873" Title="Literature on comparing Simplex and Interior-Point-Methods (or combining both of them)" Tags="|algorithms|reference-request|constrained-optimization|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25341" PostTypeId="1" CreationDate="2016-10-27T11:19:05.490" Score="1" ViewCount="7571" Body="&lt;p&gt;I implemented the Jacobi iteration using Matlab based &lt;a href=&quot;http://homepages.dcc.ufmg.br/~assuncao/an/LAB-scilab05.pdf&quot; rel=&quot;nofollow&quot;&gt;on this paper&lt;/a&gt;, and the code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function x = jacobi(A, b)&#xA;% Executes iterations of Jacobi's method to solve Ax = b.&#xA;&#xA;assert(size(A, 1) == size(A, 2))&#xA;assert(size(A, 1) == size(b, 1))&#xA;&#xA;% matrix with all zeros except the diagonal elements,&#xA;% which are those of A.&#xA;D = diag(diag(A));&#xA;&#xA;% matrix A but with its diagonal elements equal zero&#xA;R = D - A;&#xA;&#xA;tol = 1.e-8; &#xA;max_num_of_iter = 1000;&#xA;&#xA;% initialize relative error to large value&#xA;relative_error = inf; &#xA;num_of_iter = 1;&#xA;&#xA;% initial guess&#xA;x_prev = zeros(size(b, 1), 1);&#xA;&#xA;% iterate until convergence or maximum number of iterations is reached&#xA;while relative_error &amp;gt; tol &amp;amp;&amp;amp; num_of_iter &amp;lt; max_num_of_iter&#xA;    x = D \ (b + R * x_prev);&#xA;&#xA;    relative_error = norm(x - x_prev, inf) / norm(x, inf);&#xA;&#xA;    x_prev = x;&#xA;&#xA;    num_of_iter = num_of_iter + 1;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now what I don't understand is why the iteration &lt;code&gt;x&lt;/code&gt; is found using &lt;code&gt;D \ (b + R * x_prev)&lt;/code&gt;, more specifically why do we have a &lt;code&gt;+&lt;/code&gt; sign instead of a &lt;code&gt;-&lt;/code&gt; sign in &lt;code&gt;D \ (b + R * x_prev)&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to [Wikipedia](&lt;a href=&quot;https://en.wikipedia.org/wiki/Jacobi_method&quot; rel=&quot;nofollow&quot;&gt;https://en.wikipedia.org/wiki/Jacobi_method&lt;/a&gt;, the Jacobi method is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf {x} ^{(k+1)}=D^{-1}(\mathbf {b} -R\mathbf {x} ^{(k)})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ is an $n \times n$ matrix whose entries are all zeros except the diagonal entries which are the diagonal entries of $A$. $R = D - A$, i.e. a matrix whose diagonal elements are zeros and all other are the same as the corresponding elements in $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, if we want to solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax = b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $$A = L + D + U =\\ D + L + U = \\ D + R \Rightarrow \\ A - D = R = \\ L + U $$&lt;br&gt;&#xA;then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(D + L + U) x = b \\ Dx + Lx + Ux = b \Rightarrow\\ Dx = b - Lx - Ux \Rightarrow \\ Dx = b - (L + U) x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;since we have defined $L + U = R$, then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Dx = b - (L + U) x \Rightarrow \\ Dx = b - Rx \Rightarrow \\ x = D^{-1} \times (b - Rx)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the original formula of the iterative method $\mathbf {x} ^{(k+1)}=D^{-1}(\mathbf {b} -R\mathbf {x} ^{(k)})$ provided by Wikipedia seems to be consistent with my calculations. The problem is that if I use a &lt;code&gt;-&lt;/code&gt; instead of &lt;code&gt;+&lt;/code&gt; I don't find the right solutions. How do I know that? I'm using &lt;code&gt;x = A \ b&lt;/code&gt; (i.e. guassian elimination) to check for the solution.&lt;/p&gt;&#xA;" OwnerDisplayName="user9925" LastActivityDate="2016-10-27T11:56:59.593" Title="Implementation of the Jacobi iteration to find the solution to $Ax = b$" Tags="|matlab|matrix|iterative-method|jacobian|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25342" PostTypeId="2" ParentId="25314" CreationDate="2016-10-27T11:43:32.870" Score="4" Body="&lt;p&gt;Let me formulate  my remark as an answer (and make it more precise):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My experience and knowledge is that to use collocated grids with any standard (lower or higher order) methods for this problem, you must modify (stabilize) your Navier-Stokes equation. it means with some additional effort. This needs not to be necessary when using staggered grids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The references and some summary I am familiar with are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;S. Patankar: Numerical Heat Transfer and Fluid Flow. 1980&lt;/strong&gt; - see chapter 6 &quot;Calculation of flow field&quot; where the problem of checkerboard pattern of pressure values is mentioned for &quot;collocated&quot; grids (i.e. all unknowns at the same grid points) and where only single remedy is proposed - staggered grids, and no modifications of equations or discretization methods is necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;J.H. Ferziger, M. Peric: Computational Methods for Fluid Dynamics. 2002&lt;/strong&gt; (3rd edition) - it can be seen as a continuation of the book of Patankar. See Chapter 7 on Solution of the Navier-Stokes equation and sections 7.2 on Choice of Variable Arrangement on the Gird, and sections afterwards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is noted there that between 1960 and 1980 after introducing staggered grids very few works used the collocated grids. The collocated grids began to be again popular with &quot;projection methods&quot; that exist in many variants and that are well discussed in the book of Ferziger and Peric. If I simplify it the basic idea is to derive and use a Poisson equation to compute the pressure and to update afterwards the velocity field to be conservative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An introduction of stabilizations (sometimes compared to artificial compressibility) on collocated grids is known to me from papers like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Schneider, G.E., Raw, M.J.: Control volume finite-element method&#xA;for heat transfer and fluid flow using collocated variables — 1. Computational&#xA;procedure. Numer. Heat Transf. 11, 363–390 (1987)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Karimian, S.M., Schneider, G.E.: Pressure-based control-volume&#xA;finite element method for flow at all speeds. AIAA J. 33(9), 1611–&#xA;1618 (1995)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nägele, S., Wittum, G.: On the influence of different stabilisation&#xA;methods for the incompressible Navier–Stokes equations. J. Comput.&#xA;Phys. 224, 100–116 (2007)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my quite subjective summary might be that when using staggered grids, no additional modification to some &quot;standard&quot; numerical methods is required. Opposite to that for collocated grids some additional efforts is necessary.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-10-27T11:43:32.870" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25343" PostTypeId="2" ParentId="25341" CreationDate="2016-10-27T11:51:41.710" Score="3" Body="&lt;p&gt;In your case (line 12 in your code)&#xA;$$A = D-R.$$&#xA;Then&#xA;$$&#xA;\begin{aligned}&#xA;A\mathbf{x} {}&amp;amp;{}= D\mathbf{x}-R\mathbf{x}\\&#xA;             {}&amp;amp;{}= \mathbf{b}\,,&#xA;\end{aligned}$$&#xA;which rearranges to&#xA;$$\mathbf{x} = D^{-1}\left( \mathbf{b} + R\mathbf{x} \right)\,,$$&#xA;which leads to the iteration which is the the first line in your while loop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This only differ's from the Wikipedia version by the definition, i.e. Wikipedia's&#xA;$$R' = A - D$$&#xA;to your&#xA;$$R = D - A\,.$$&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="17671" LastEditDate="2016-10-27T11:56:59.593" LastActivityDate="2016-10-27T11:56:59.593" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25344" PostTypeId="2" ParentId="25325" CreationDate="2016-10-27T13:36:06.557" Score="2" Body="&lt;p&gt;You're allowed an algorithm that computes the hull. If this means an algorithm that computes the convex polygon then I would say consider the lines defined by adjacent points on that polygon. I think it's possible to prove that one of those lines is the one required. Therefore, the required algorithm is simply to iterate through them, calculate sums of distances and select 'best'.&lt;/p&gt;&#xA;" OwnerUserId="22133" LastActivityDate="2016-10-27T13:36:06.557" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25345" PostTypeId="2" ParentId="25334" CreationDate="2016-10-27T14:10:06.813" Score="5" Body="&lt;p&gt;A good way to achieve this is to use the Warp By Vector filter to warp your field in the Z direction using the value of your scalar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to give a brief overview,&#xA;The warp by vector warps your field in a direction using a vector. If you wish to warp your field in the Z direction perpendicular to your XY plane, then you need to create a warp vector in the Z direction. This is very easy to achieve using the calculator and the kHat vector.&#xA;Then you can rescale the value of the vector to obtain a warping that is pleasant to the eye.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the easiest way to achieve this type of graph using your (X Y 0) type of vtk files as far as I know.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2016-10-27T14:10:06.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25346" PostTypeId="1" AcceptedAnswerId="25347" CreationDate="2016-10-27T14:14:45.183" Score="0" ViewCount="1369" Body="&lt;p&gt;I have a pretty basic question about Lattice-Boltzmann methods. I heard that this is a meshless simulation method. How do you account for obstacles and boundaries in a Lattice-Boltzmann code. Does it work like a $\Delta x$, $\Delta y$ grid ?&lt;/p&gt;&#xA;" OwnerUserId="22128" LastActivityDate="2017-05-11T18:51:04.573" Title="Mesh and Lattice-Boltzmann methods" Tags="|lattice-boltzmann-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25347" PostTypeId="2" ParentId="25346" CreationDate="2016-10-27T14:48:09.043" Score="2" Body="&lt;p&gt;The Lattice Boltzmann is not a meshless method. Actually, when looking at it, it is a finite difference method on an homogenous structured Cartesian grid (dx = dy = dz). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the variables solved for are not the primary variables (U and P), but are pseudo-populations $f_i$ where $i\in [1,n_p]$ and where $n_p$ is the number of population in the lattices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lattices in the Lattice Boltzmann method are structures like this one :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Z3smC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z3smC.jpg&quot; alt=&quot;D2Q9 Lattice for Navier-Stokes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These populations undergo advection (streaming) and collisions (using a collision operator like the BGK operator) and &quot;move from node to node&quot;. This allows them to reproduce the pseudo-incompressible Navier-Stokes equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How are boundary conditions specified? Well there are numerous ways. One of the simple way to apply no-slip Dirichlet Boundary condition is the Bounce-Back rule. This rule makes it so that the populations literally bounce back on the obstacle, leading to a 0 velocity at the position of the interface. There are numerous ways to specify the boundary conditions (extrapolation methods, immersed boundary methods, on-lattice and off-lattice boundary conditions, etc.), but you always need to specify them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall, if  you wish to get a very good overview of LBM, I lead you to the Wikipedia article on the matter:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Lattice_Boltzmann_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Lattice_Boltzmann_methods&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or to the book by Guo:&#xA;&lt;a href=&quot;http://www.worldscientific.com/worldscibooks/10.1142/8806&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.worldscientific.com/worldscibooks/10.1142/8806&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="24110" LastEditDate="2017-05-11T18:51:04.573" LastActivityDate="2017-05-11T18:51:04.573" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25350" PostTypeId="2" ParentId="25340" CreationDate="2016-10-27T19:38:46.120" Score="2" Body="&lt;p&gt;I am not sure what you mean by comparing and combining. But a general textbook on linear and, more generally, convex optimization that I can recommend is the one by Stephen Boyd:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Boyd, Stephen, and Lieven Vandenberghe. &lt;a href=&quot;http://stanford.edu/~boyd/cvxbook/&quot; rel=&quot;nofollow&quot;&gt;Convex optimization&lt;/a&gt;. Cambridge university press, 2004. &lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It covers both simplex and interior point methods, and how they are related to each other. And, if you ask me, he does so quite well. And, have I already mentioned that you can access it online (legally)? ;-)&lt;/p&gt;&#xA;" OwnerUserId="20357" LastEditorUserId="9667" LastEditDate="2016-10-27T20:08:52.873" LastActivityDate="2016-10-27T20:08:52.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25352" PostTypeId="2" ParentId="10676" CreationDate="2016-10-27T19:53:04.193" Score="1" Body="&lt;p&gt;Not only is this a non-convex programming problem, it is actually a concave programming problem, i.e., the minimization of a concave function subject to convex constraints.  I will assume that you have reformulated to use &amp;lt;= rather than &amp;lt; for the bound constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because your concave programming problems has a compact constraint region, there must exist at least one global minimum at an extreme point of the constraint region. See, for example, theorem 1.1 on p. 10 of &lt;a href=&quot;https://books.google.com/books?id=Pe_1CAAAQBAJ&amp;amp;pg=PA10&amp;amp;lpg=PA10&amp;amp;dq=concave%20programming%20extreme%20point%20compact%20theorem%20global&amp;amp;source=bl&amp;amp;ots=cvyi_FErL1&amp;amp;sig=v7NLeAzoQ-CjEWSHqmd236yqJaM&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwjrmuzrs_7PAhWc2YMKHV4bC1wQ6AEIPTAE#v=onepage&amp;amp;q=concave%20programming%20extreme%20point%20compact%20theorem%20global&amp;amp;f=false&quot; rel=&quot;nofollow&quot;&gt;&quot;Global Optimization: Deterministic Approaches&quot; by Reiner Horst, Hoang Tuy&lt;/a&gt;.  So there must be a global minimum at one of the $2^v$ vertices of the box constituting the constraints, where $n$ is the dimension of $x$..  Therefore, you can solve the problem to global optimality by brute force evaluation of the objective function at all $2^n$ vertices, and choosing the vertex (or vertices) having the lowest value of the objective function.  This approach will work fine if $n$ is not too big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally you can solve this to global optimality using CPLEX&quot;s QP solver, with solutiontarget set to 3 (solve to global optimality).  If you are content with a locally optimal solution, you can set solutiontarget to 2.  BARON can also be usesd to solve this to global optimality.  As well, there are many local QP solvers (must accept non-convex problems) which can as well, in addition to some other global solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if you use a general purpose local nonlinear solver to solve this concave QP using a Quasi-Newton Method, and you use BFGS as the choice of Quasi-Newton method, performance may be quite poor, and it may not even succeed. That is because BFGS maintains a positive semi-definite Hessian approximation, and therefore objective function approximation, and that is a quite poor representation, for optimization purposes, of a concave function. However, the SR1 Quasi-Newton method might succeed, because its definiteness can adapt to what it &quot;sees&quot; based on the gradient differences.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2016-10-28T21:19:14.457" LastActivityDate="2016-10-28T21:19:14.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25353" PostTypeId="1" AcceptedAnswerId="25354" CreationDate="2016-10-27T20:39:12.993" Score="5" ViewCount="641" Body="&lt;p&gt;In the finite element method, in order to use MPI, we need to decompose the domain into sub-domains first. Then my question is whether we can solve each sub-domain using a direct solver?&lt;/p&gt;&#xA;&lt;p&gt;Of course, unlike solving the entire domain as a whole, we don't get the right solution in one iteration. Will the solution converge after several iterations with the direct solver? If it converges, how does it compare to an iterative solver with domain decomposition: which one is better?&lt;/p&gt;&#xA;" OwnerUserId="7905" LastEditorUserId="20688" LastEditDate="2021-12-28T03:26:13.603" LastActivityDate="2021-12-28T03:26:13.603" Title="Direct solvers and domain decomposition for FEM" Tags="|finite-element|parallel-computing|mpi|domain-decomposition|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25354" PostTypeId="2" ParentId="25353" CreationDate="2016-10-28T02:12:43.270" Score="3" Body="&lt;p&gt;As I understand, this is a fairly popular approach. Direct solvers are usually more efficient than iterative solvers for &amp;lt; 100,000 unknowns, so you partition the problem into subproblems of roughly this size, use direct solvers on each subproblem, and combine them globally with some method like alternating Schwarz. It's also common use the domain decomposition method as a preconditioner for a global CG/GMRES solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're correct that domain decomposition methods don't get the right answer in a single iteration. The alternating Schwarz method can be thought of as overlapping block Jacobi, just with very big blocks. The simple Jacobi iteration converges for strongly diagonally-dominant matrices; it's a similar argument to show that the alternating Schwarz method is convergent (for certain nice systems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are of course many variations on a theme. If you can make the subdomains structured grids, you might be able to use geometric multigrid on the subproblems rather than a direct solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also worth noting is that some folks think of multigrid and domain decomposition methods as the endpoints of a continuum of algebraic multi-level methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in these sorts of things, I highly recommend &lt;a href=&quot;https://books.google.com/books?isbn=3540206965&quot; rel=&quot;nofollow&quot;&gt;this book&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-10-28T02:12:43.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25356" PostTypeId="1" AcceptedAnswerId="25359" CreationDate="2016-10-28T10:37:54.797" Score="6" ViewCount="255" Body="&lt;p&gt;I am trying to approximate the exponential of a matrix. I want to use a tolerance but I am confused as to how to compute the error. Any ideas or hints?&lt;/p&gt;&#xA;" OwnerUserId="22153" LastEditorUserId="9667" LastEditDate="2016-10-28T20:21:18.503" LastActivityDate="2016-10-28T20:21:18.503" Title="Approximating the exponent of a matrix $\exp(A)$ using Taylor series" Tags="|matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25357" PostTypeId="1" AcceptedAnswerId="25369" CreationDate="2016-10-28T10:59:06.377" Score="2" ViewCount="681" Body="&lt;p&gt;For the &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%E2%80%93Cotes_formulas#Closed_Newton.E2.80.93Cotes_formulae&quot; rel=&quot;nofollow&quot;&gt;closed Newton-Cotes quadrature&lt;/a&gt; over $[x_1, x_n]$, the coefficients $H_{n,i}$ for&#xA;$$&#xA;\int_{x_1}^{x_n} f(x)\:\text{d}x = h \sum_{i=1}^n H_{n,i} \; f(x_i)&#xA;$$&#xA;are given explicitly by&#xA;$$&#xA;H_{n,r+1} =\frac{(-1)^{n-r}}{r!(n-r)!}\int_0^n \frac{\prod_{k=0}^n (t-k)}{t-r}\:\text{d}t;&#xA;$$&#xA;see &lt;a href=&quot;http://mathworld.wolfram.com/Newton-CotesFormulas.html&quot; rel=&quot;nofollow&quot;&gt;http://mathworld.wolfram.com/Newton-CotesFormulas.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute those values numerically, one could go ahead and evaluate the integral, but perhaps that's not the most efficient thing to do. &lt;a href=&quot;https://github.com/scipy/scipy/blob/v0.18.0/scipy/integrate/quadrature.py#L774-L855&quot; rel=&quot;nofollow&quot;&gt;SciPy does something else&lt;/a&gt;, but I don't quite get it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hints?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2016-10-28T12:00:27.330" LastActivityDate="2016-10-28T20:22:30.740" Title="computing Newton-Cotes weights" Tags="|quadrature|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25358" PostTypeId="2" ParentId="25356" CreationDate="2016-10-28T12:59:51.047" Score="0" Body="&lt;p&gt;I have not entirely thought this through, but have you have thought about using the largest eigenvalue?&#xA;If $A$ is your matrix, and $Q \Lambda Q^{-1}$ its eigendecomposition, then you can use the following identity for each term in your Taylor series:&#xA;\begin{equation}&#xA;A^n = Q \Lambda^n Q^{-1} \quad .&#xA;\end{equation}&#xA;Then, you take the largest eigenvalue, and compute from it an upper bound for the Lagrange remainder.&#xA;Obviously, this works best for symmetric matrices where Q is orthogonal.&#xA;But I still think this may be a good starting point.&lt;/p&gt;&#xA;" OwnerUserId="20357" LastActivityDate="2016-10-28T12:59:51.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25359" PostTypeId="2" ParentId="25356" CreationDate="2016-10-28T13:00:55.083" Score="11" Body="&lt;p&gt;First: there is a must read on this topic&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Moler, Cleve, and Charles Van Loan. &quot;&lt;a href=&quot;http://www.cs.cornell.edu/cv/researchpdf/19ways+.pdf&quot;&gt;Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later&lt;/a&gt;.&quot; SIAM review 45.1 (2003): 3-49.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(in case you wonder, the original paper is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Moler, Cleve, and Charles Van Loan. &quot;Nineteen dubious ways to compute the exponential of a matrix.&quot; SIAM review 20.4 (1978): 801-836.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;More to the question: The following error bound from &lt;a href=&quot;http://www.maths.manchester.ac.uk/~higham/talks/exp10.pdf&quot;&gt;this slides&lt;/a&gt; will be helpful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a complex  $n\times n$ matrix $A$ let&#xA;$$&#xA;T_{r,s} = \Big(\sum_{i=0}^r \frac{1}{i!}\big(\tfrac{A}{s}\big)^i\Big)^s&#xA;$$&#xA;then&#xA;$$&#xA;\|e^A - T_{r,s}\| \leq \frac{\|A\|^{r+1}}{s^r(r+1)!}e^{\|A\|}.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2016-10-28T13:00:55.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25360" PostTypeId="1" AcceptedAnswerId="25362" CreationDate="2016-10-28T13:19:49.457" Score="1" ViewCount="47" Body="&lt;p&gt;Consider the following regression model :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Y = AX + BU&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the size of Y is $N \times n$, A is $N \times n$, X is $n \times n$, B is $N \times n$ and U is $n \times 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrices X,Y and U are kown and the matrices A and U have to be estimated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an optimal method (in the sense of minimal residual error between Y and its estimate) for estimating A and U simultaneously ?&lt;/p&gt;&#xA;" OwnerUserId="20482" LastActivityDate="2016-10-28T15:18:02.960" Title="Parameter identification for regression model" Tags="|optimization|statistics|regression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25362" PostTypeId="2" ParentId="25360" CreationDate="2016-10-28T15:18:02.960" Score="1" Body="&lt;p&gt;You can just write it as a least squares minimization problem. That leads to a quadratic problem that is easy to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, in general, underdetermined since you only have $Nn$ equations, but $Nn+n$ variables to determine. But that's not a problem for the least squares method -- it will simply find &lt;em&gt;one&lt;/em&gt; solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Of course, it is conceivable that $Y$ does not lie in the range of $A$-applied-to-X or $B$, in which case the residual of the minimizer will be minimal, but not zero.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-28T15:18:02.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25363" PostTypeId="1" AcceptedAnswerId="25367" CreationDate="2016-10-28T16:06:37.853" Score="5" ViewCount="2973" Body="&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; Thank you very much for all of you who answered below. I'm studying each answer now. In the long term, I'm more interested in solutions that work for &lt;strong&gt;sparse tensors&lt;/strong&gt; (sorry I should have mentioned this earlier), but I can use your ideas as a temporary solution (I will need to add more RAM for larger size problems.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $T=(T_{ijk})$ be an $n\times n\times n$ tensor and $x=(x_1,x_2,\ldots,x_n)$, $y=(y_1,y_2,\ldots,y_n)$, $z=(z_1,z_2,\ldots,z_n)$  be $n$-dimensional vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;Edit:&lt;/strong&gt; as suggested by @WolfgangBangerth I should not use the $\otimes$ symbol that's usually used for outer product. I replaced it by $\otimes_i$, which means now the product of a tensor and a vector at mode $i$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The products&#xA;\begin{align}&#xA;a &amp;amp;= T\otimes_2 y\otimes_3 z\\&#xA;b &amp;amp;=T\otimes_3 z\otimes_1 x \\&#xA;c &amp;amp;=T\otimes_1 x\otimes_2 y&#xA;\end{align} &#xA;are $n$-dimensional vectors defined by:&#xA;\begin{align}&#xA;a_i &amp;amp;= \sum_{1\le j\le n}\sum_{1\le k\le n} T_{ijk}y_jz_k,\quad i=1,\ldots,n \\&#xA;b_j &amp;amp;= \sum_{1\le i\le n}\sum_{1\le k\le n} T_{ijk}x_iz_k,\quad i=1,\ldots,n \\&#xA;c_k &amp;amp;= \sum_{1\le i\le n}\sum_{1\le j\le n} T_{ijk}x_iy_j,\quad i=1,\ldots,n.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an algorithm doing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Repeat:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute $a$, update $x=f(a)$.&lt;/li&gt;&#xA;&lt;li&gt;Compute $b$, update $y=g(b)$.&lt;/li&gt;&#xA;&lt;li&gt;Compute $c$, update $z=h(c)$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would like to ask for a way (or toolboxes/libraries that help me) to efficiently compute $a,b,c$ at each iterations, without doing loops. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Language: C++ is preferred but Matlab is also acceptable if easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much in advance for your help!&lt;/p&gt;&#xA;" OwnerUserId="7695" LastEditorUserId="7695" LastEditDate="2016-10-31T13:29:25.353" LastActivityDate="2016-10-31T13:29:25.353" Title="Efficiently computing the product of a multi-dimensional matrix (or tensor) and vectors" Tags="|matlab|c++|matrix|tensor|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25364" PostTypeId="2" ParentId="25339" CreationDate="2016-10-28T16:41:57.653" Score="3" Body="&lt;p&gt;In these &lt;a href=&quot;http://www.mn.uio.no/ifi/english/services/knowledge/cs/forelesn.kap8.pdf&quot; rel=&quot;nofollow&quot;&gt;slide&lt;/a&gt;  there are some comments about the energy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At pag 4 it focus on the fact that this energy is not a physical energy, but it is a mathematical tool. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At pag 8 it observes that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From a physical point of view it seems reasonable that a the energy will decrease in a system without any heat source &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And after this the author defines another kind of energy based on $u_x$.&#xA;Also this energy is decreasing (with the same boundary conditions).&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2016-10-28T16:41:57.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25365" PostTypeId="2" ParentId="10078" CreationDate="2016-10-28T17:18:34.977" Score="0" Body="&lt;p&gt;Did you try to sort the $\tt x$ array before invoking L2DINTERPOL3 ? I mean only the $\tt x$ array. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am thinking about is based on the fact that two following arrays are neighbor of each other with $M$- length of increment (i.e. column-major in Fortran array model),&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Image( * , x1(jj) ) and Image( * , x2(jj) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;due to $ {\tt x2(jj)} = {\tt x1(jj)} +1$, since ${\tt x1(jj)} = [x_{\rm jj}] $ and ${\tt x2(jj)} = [x_{\rm jj}] +1$. Sorting $\tt x$ before invoking L2DINTERPOL3 may help $\tt x1(jj+1)$ to be near $\tt x1(jj)$ when we proceed $\tt jj = jj+1$ in the loop. Hence, it may optimize access to the storage of $\tt Image(:,:)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea is simply described as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   call SORT( NPts, x, iorder )&#xA;&#xA;   do j = 1, NPts&#xA;       x(j) = x( iorder(j) )&#xA;       y(j) = y( iorder(j) )&#xA;   enddo&#xA;&#xA;   call L2DINTERPOL3(IntIm,Image,x,y,NPts,M,N)&#xA;&#xA;   do j = 1, NPts&#xA;       IntIm( iorder(j) ) = IntIm(j)&#xA;   enddo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the last loop is to recover $\tt IntIm(:)$ based on  $\tt iorder(:)$. Can you implement this idea and let me know its effect?&lt;/p&gt;&#xA;" OwnerUserId="15175" LastEditorUserId="15175" LastEditDate="2016-10-28T17:30:02.803" LastActivityDate="2016-10-28T17:30:02.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25366" PostTypeId="2" ParentId="25357" CreationDate="2016-10-28T18:45:30.600" Score="0" Body="&lt;p&gt;In the pass I did something similarly. At the time I applied Gauss-Legendre quadrature to evaluate $H_{n,r+1}$ intergals; this calculation adopted (1) Gauss-Legendre weight and points calculated in advance (then, change the variable $t$ in the integrand to $\xi \in [-1,1]$), and (2) multi-precision floating point library &lt;a href=&quot;http://myweb.lmu.edu/dmsmith/fmlib.html&quot; rel=&quot;nofollow&quot;&gt;fmlib&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess that you obtain $H_{n,i} $ from Lagrange Interpolation Formulae. Let me review some basic thing so that you may recognize the same derivation. From the Lagrange Interpolation, we have &#xA;\begin{align}&#xA;	f(x)&#xA;=&#xA;	\sum_{i=1}^{n}&#xA;	l_i (x) &#xA;	f_i&#xA;+&#xA;	R_n (x)&#xA;\end{align}&#xA;where&#xA;\begin{align}&#xA;	l_i (x) &#xA;&amp;amp;=&#xA;	\frac{&#xA;		\Pi_n( x) &#xA;	}{&#xA;		(x-x_i)&#xA;	} &#xA;	.&#xA;	\frac{&#xA;		1&#xA;	}{&#xA;		\Pi _n ' (x_i) &#xA;	} &#xA;=&#xA;	\frac{&#xA;		( x - x_1 ) &#xA;		\ldots &#xA;		( x - x_{i-1} )&#xA;		( x - x_{i+1} )&#xA;		\ldots &#xA;		( x - x_{n} )&#xA;	}{&#xA;		( x_i - x_1 ) &#xA;		\ldots &#xA;		( x_i - x_{i-1} )&#xA;		( x_i - x_{i+1} )&#xA;		\ldots &#xA;		( x_i - x_{n} )&#xA;	} ,&#xA;\\&#xA;	R_n (x) &#xA;&amp;amp;=&#xA;	\frac{ f^{(n)} (\xi) }{ n! } \cdot &#xA;	\Pi _n (x)  &#xA;   ,&#xA;\end{align}&#xA;for some $\xi = \xi(x) \in (x_1, x_n )$. &#xA;Note that &#xA;we defined&#xA;$ &#xA;	\Pi_n (x)&#xA;=&#xA;	\prod_{i=1}^{n}&#xA;	(x-x_i)&#xA;$, &#xA;and we also have &#xA;\begin{align}&#xA;	\left| R_n (x) \right| &#xA;&amp;amp;\le &#xA;	\frac{ ( x_n - x_1 )^{n} }{ n! }&#xA;	\max_{ x_1 \le x \le x_n } \left| f^{(n)} (x) \right|&#xA;\end{align}&#xA;which can be proved using the Role’s theorem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we take &#xA;\begin{align}&#xA;	\int_{x_1}^{x_n}&#xA;	f(x) &#xA;	d x &#xA;&amp;amp;=&#xA;	\sum_{i=1}^{n}&#xA;   \underbrace{&#xA;	\left( &#xA;		\int_{x_1}^{x_n}&#xA;			l_i (x) &#xA;		d x &#xA;	\right) &#xA;   }_{ A_ i }&#xA;	f_i&#xA;+&#xA;   \underbrace{&#xA;	\int_{x_1}^{x_n}&#xA;		R_n (x)&#xA;	d x &#xA;   }_{ B }&#xA;\end{align}&#xA;Define &#xA;\begin{align}&#xA;   A_ i &#xA;&amp;amp;=   &#xA;   \int_{x_1}^{x_n}&#xA;	   l_i (x) &#xA;   d x ~~~(\equiv H_{n,i} ? )&#xA;\\&#xA;   B&#xA;&amp;amp;=&#xA;	\int_{x_1}^{x_n}&#xA;		R_n (x)&#xA;	d x &#xA;\end{align}&#xA;We are going to estimate those integrals. Change variable,&#xA;\begin{align}&#xA;   h &#xA;&amp;amp;=&#xA;   \frac{ x_n  - x_1 }{ n - 1 }&#xA;,~~~~~&#xA;   x = x_1 + (t-1) h&#xA;,~~&#xA;   \forall t \in [1,n]&#xA;\end{align}&#xA;Using the latter one, $x - x_j = x_1 + (t-1)h - x_j $ and &#xA;$ x_j = x_1 + (j-1) h $, &#xA;we have &#xA;$$&#xA;   x - x_j &#xA;   = &#xA;   x_1 + (t-1)h - &#xA;   \left[ x_1 + (j-1)h \right] &#xA;   =&#xA;   (t-j)h &#xA;$$&#xA;Thus &#xA;\begin{align}&#xA;   A_ i&#xA;&amp;amp;=   &#xA;   \int_{x_1}^{x_n}&#xA;	   l_i (x) &#xA;   d x &#xA;=&#xA;	\int_{x_1}^{x_n}&#xA;	\frac{&#xA;		( x - x_1 ) &#xA;		\ldots &#xA;		( x - x_{i-1} )&#xA;		( x - x_{i+1} )&#xA;		\ldots &#xA;		( x - x_{n} )&#xA;	}{&#xA;		( x_i - x_1 ) &#xA;		\ldots &#xA;		( x_i - x_{i-1} )&#xA;		( x_i - x_{i+1} )&#xA;		\ldots &#xA;		( x_i - x_{n} )&#xA;	} &#xA;	d x &#xA;\\&#xA;&amp;amp;=&#xA;	\int_{1}^{n}&#xA;   \frac{&#xA;   h ( t - 1 ) &#xA;   \cdot  &#xA;   h ( t - 2 )&#xA;   \cdots&#xA;   h ( t-i+1 )&#xA;   \cdot &#xA;   h (t-i-1)&#xA;   \cdots&#xA;   h(t-n)&#xA;   }{ &#xA;    h ( i - 1 ) &#xA;   \cdot  &#xA;   h ( i - 2 )&#xA;   \cdots&#xA;   h ( i-i+1 )&#xA;   \cdot &#xA;   h (i-i-1)&#xA;   \cdots&#xA;   h(i-n)&#xA;   }&#xA;   \cdot &#xA;   h d t &#xA;\end{align}&#xA;Or&#xA;\begin{align}&#xA;   A_ i&#xA;&amp;amp;=&#xA;   h &#xA;	\int_{1}^{n}&#xA;   \frac{&#xA;   ( t - 1 ) &#xA;   \cdot  &#xA;   ( t - 2 )&#xA;   \cdots&#xA;   ( t-i+1 )&#xA;   \cdot &#xA;   (t-i-1)&#xA;   \cdots&#xA;   (t-n)&#xA;   }{ &#xA;   ( i - 1 ) &#xA;   \cdot  &#xA;   ( i - 2 )&#xA;   \cdots&#xA;   ( i-i+1 )&#xA;   \cdot &#xA;   (i-i-1)&#xA;   \cdots&#xA;   (i-n)&#xA;   }&#xA;   d t &#xA;\end{align}&#xA;And &#xA;\begin{align}&#xA;   B&#xA;&amp;amp;=   &#xA;   \int_{x_1}^{x_n}&#xA;	   R_n (x) &#xA;   d x &#xA;=&#xA;	\int_{x_1}^{x_n}&#xA;		( x - x_1 ) &#xA;		\ldots &#xA;      ( x - x_{i} )&#xA;		\ldots &#xA;		( x - x_{n} )&#xA;      \cdot &#xA;      \frac{ f^{(n)} (\xi (x) ) }{ n! }  &#xA;	d x &#xA;\\&#xA;&amp;amp;=&#xA;	\int_{1}^{n}&#xA;   h ( t - 1 ) &#xA;   \cdot  &#xA;   h ( t - 2 )&#xA;   \cdots&#xA;   h ( t - i)&#xA;   \cdots&#xA;   h(t-n)&#xA;   \cdot &#xA;   \frac{ f^{(n)} (\xi (x(t)) ) }{ n! }  &#xA;   \cdot &#xA;   h d t &#xA;\end{align}&#xA;We estimate&#xA;\begin{align}&#xA;   | B | &#xA;&amp;amp;\le&#xA;   M &#xA;   h^{n+1}&#xA;\end{align}&#xA;where $M = |C_n | \sup_{ \xi \in [x_1,x_n]  } \left| f^{(n)} (\xi) \right| $&#xA;and &#xA;\begin{align}&#xA;   C_n&#xA;&amp;amp;=&#xA;   \frac{1}{ n! }&#xA;	\int_{1}^{n}&#xA;   ( t - 1 ) &#xA;   \cdots&#xA;   ( t - i)&#xA;   \cdots&#xA;   ( t - n )&#xA;   d t &#xA;\label{c}&#xA;\end{align}&#xA;Here integrals $A_i$ and $C_n$ can be estimated EXACTLY using&#xA;Gauss-Legendre formulas with $n/2$ points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I could see in numerical result at the time, we have (a) $A_i $ are increased to very large values as $n$ increasing, for example $\max A_i \sim 10^{0}, 10^{2}, 10^{4}, 10^{7}, 10^{10}$ and $10^{24}$ when $n=10,20,30,40,50$, and $100$, respectively; (b) $A_i$ is symmetry with repect to $i$, i.e. $A_{i} = A_{n-i+1}$; (c) $\sum_{i=1}^{n} A_{i} = n-1$; and (c) $C_n = 0$ for $n$ odd, and $C_n$ is bounded by a small value when $n$ even. &lt;/p&gt;&#xA;" OwnerUserId="15175" LastEditorUserId="15175" LastEditDate="2016-10-28T19:02:52.643" LastActivityDate="2016-10-28T19:02:52.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25367" PostTypeId="2" ParentId="25363" CreationDate="2016-10-28T19:21:27.490" Score="4" Body="&lt;p&gt;I did a little test with the Tensor package in Eigen C++ (release candidate 1&#xA;of version 3.3) &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow&quot;&gt;http://eigen.tuxfamily.org/index.php?title=Main_Page&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The path to the Tensor include file shows this package as &quot;unsupported&quot; and,&#xA;presumably, this will still be the case in version 3.3. I do know that this&#xA;package is rapidly reaching the &quot;supported&quot; stage. For more information on this,&#xA;you could contact the Eigen developers directly if that is a concern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My test code is pasted below. On my system, the Eigen versions of the tensor products&#xA;you show below are around 3X faster than the brute force versions. The syntax&#xA;of the operation seems fairly convenient to me.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;boost/timer.hpp&amp;gt;&#xA;#include &amp;lt;unsupported/Eigen/CXX11/Tensor&amp;gt;&#xA;&#xA;  const int n = 300;&#xA;  Eigen::Tensor&amp;lt;double, 3&amp;gt; T(n, n, n);&#xA;  Eigen::Tensor&amp;lt;double, 1&amp;gt; y(n), z(n);&#xA;  int ii = 1;&#xA;  for (int i = 0; i &amp;lt; n; i++) {&#xA;    y(i) = i+1;&#xA;    z(i) = i + 3;&#xA;    for (int j = 0; j &amp;lt; n; j++) {&#xA;      for (int k = 0; k &amp;lt; n; k++) {&#xA;        T(i, j, k) = ii++;&#xA;      }&#xA;    }&#xA;}&#xA;  //cout &amp;lt;&amp;lt; T &amp;lt;&amp;lt; endl;&#xA;  typedef Eigen::IndexPair&amp;lt;int&amp;gt; IP;&#xA;  Eigen::array&amp;lt;IP, 1&amp;gt; d10 = { IP(1, 0) }; // second index of T&#xA;  boost::timer timer;&#xA;  Eigen::Tensor&amp;lt;double, 1&amp;gt;  Tyz = T.contract(y, d10).contract(z, d10);&#xA;  //cout &amp;lt;&amp;lt; Tyz &amp;lt;&amp;lt; endl;&#xA;  printf(&quot;elapsed time=%8.3f\n&quot;, timer.elapsed());&#xA;&#xA;  // brute force&#xA;  Eigen::VectorXd v(n);&#xA;  v.setZero();&#xA;  timer.restart();&#xA;  for (int i = 0; i &amp;lt; n; i++) {&#xA;    for (int j = 0; j &amp;lt; n; j++) {&#xA;      for (int k = 0; k &amp;lt; n; k++) {&#xA;        v(i) += T(i,j,k)*y(j)*z(k);&#xA;      }&#xA;    }&#xA;  }&#xA;  //cout &amp;lt;&amp;lt; v.transpose() &amp;lt;&amp;lt; endl;&#xA;  printf(&quot;elapsed time=%8.3f\n&quot;, timer.elapsed());&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2016-10-29T13:52:22.160" LastActivityDate="2016-10-29T13:52:22.160" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25368" PostTypeId="2" ParentId="25331" CreationDate="2016-10-28T19:45:37.747" Score="1" Body="&lt;p&gt;If you are looking for a numerical answer, LP and MILP solvers can efficiently (not always for MIP) provide you with a feasible solution by optimizing an objective function of zeroes. &lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2016-10-28T19:45:37.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25369" PostTypeId="2" ParentId="25357" CreationDate="2016-10-28T19:56:01.463" Score="2" Body="&lt;p&gt;I can explain what the scipy code is doing (&lt;a href=&quot;https://github.com/scipy/scipy/blob/v0.18.0/scipy/integrate/quadrature.py#L833-L842&quot; rel=&quot;nofollow&quot;&gt;https://github.com/scipy/scipy/blob/v0.18.0/scipy/integrate/quadrature.py#L833-L842&lt;/a&gt;). I haven't checked this directly, but this should be correct in principle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The line &lt;code&gt;ti = 2 * yi - 1&lt;/code&gt; remaps the interval $(0,1)$ to $(-1,1)$, so now the weights will be the integrals of Lagrange elementary interpolants over $(-1,1)$ with nodes $(-1:2/N:1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The line &lt;code&gt;C = ti ** nvec[:, np.newaxis]&lt;/code&gt; constructs the (transpose of) the Vandermonde matrix, with entries $C_{ij} = t_i ^ j$. This matrix has the property that when multiplied on the left by a vector $\alpha$, it produces&#xA;$$\alpha^\top C = (p_\alpha(t_0), \ldots, p_\alpha(t_N))$$&#xA;which is the vector of $p_\alpha$ (polynomial with coefficients being the entries of $\alpha$) evaluated at each $t_0,\ldots,t_N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inverse of this, $C^{-1}$, will then consist of vectors $\alpha_k$ that give the coefficients of Lagrange interpolants, because $\alpha_k^\top C = e_k$, and having $p_{\alpha_k}(t_l) = \delta_{kl}$ is the definition of Lagrange interpolants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Vandermonde matrix is very ill-conditioned, so scipy applies twice the Newton-Raphson iteration for the (matrix) reciprocal: &lt;code&gt;Cinv = 2*Cinv - Cinv.dot(C).dot(Cinv)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the coefficients of Lagrange interpolants, $C^{-1}$, are known, it is straightforward to integrate them, by multiplying $C^{-1}$ on the right with $(2/1, 0, 2/3, 0, 2/5, \ldots)^\top$ (the integrals of $x^m$ for odd $m$ are zero—the integral is over $(-1,1)$ because of &lt;code&gt;ti&lt;/code&gt; above), which gives the Newton-Cotes weights: &lt;code&gt;ai = Cinv[:, ::2].dot(vec) * (N / 2.)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-10-28T20:22:30.740" LastActivityDate="2016-10-28T20:22:30.740" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25370" PostTypeId="2" ParentId="25363" CreationDate="2016-10-29T02:01:17.870" Score="4" Body="&lt;p&gt;I think you should look at Tammy Kolda's &quot;Tensor Toolbox&quot; for matlab. It has many of the kind of operations you are looking for implemented in efficient ways.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-29T02:01:17.870" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25371" PostTypeId="2" ParentId="25363" CreationDate="2016-10-29T08:00:44.577" Score="6" Body="&lt;p&gt;In Matlab you can do these operations in a vectorized way using the commands &lt;code&gt;reshape&lt;/code&gt;, &lt;code&gt;shiftdim&lt;/code&gt;, and &lt;code&gt;permute&lt;/code&gt;. The essential idea is that contraction of a tensor with a vector is equivalent to matrix multiplication of that vector with an &lt;em&gt;unfolded&lt;/em&gt; version of the tensor. For the first example in the question, the command is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = z*reshape(y*reshape(shiftdim(T,1), n, n*n), n, n);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $y$, $z$, and $c$ are row vectors. This computes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_i = \sum_{j=1}^n \sum_{k=1}^n T_{ijk} y_j z_k.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for using row vectors and multiplying them from the right (rather than using column vectors and multiplying them from the left) is that MATLAB stores tensors in column-first order, so when reshaping and multiplying, you access elements of the tensor in the same order as it is stored in memory. This way it is more efficient in terms of cache usage and so on.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Explanation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's break it down as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(5) a = z*                                              ;&#xA;(4)       reshape(                                 n, n)&#xA;(3)               y*             &#xA;(2)                 reshape(               n, n*n)&#xA;(1)                         shiftdim(T,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(1) The &lt;code&gt;shiftdim(T,1)&lt;/code&gt; shifts all the modes to the left, wrapping around. That is, &#xA;$$T_{ijk} \rightarrow T_{jki}.$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) The innermost &lt;code&gt;reshape&lt;/code&gt; unfolds the 3-tensor into a $n$-by-$n^2$ matrix where the $j$ mode is exposed on the left:&#xA;$$\underbrace{T_{jki}}_{3-\text{tensor}} \rightarrow \underbrace{T_{(j)(ki)}}_{\text{matrix}}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Multiplying on the left by $y$ contracts $y$ with the $j$ mode, creating a temporary row vector of length $n^2$, lets call it $p$, which is a vectorized version of the tensor after contracting it with the $y$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p_{ki} = \sum_j y_j T_{jki}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(4) The second reshape refolds $p$ into a matrix to expose the $k$ mode:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\underbrace{p_{(ki)}}_{\text{row vector}} \rightarrow \underbrace{p_{(k)(i)}}_{\text{matrix}}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(5) The final multiplication by $z$ contracts $v$ with the $k$'th mode:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_k = \sum_k z_k p_{ki}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you can perform any sequence of tensor-vector or tensor-tensor contractions by unfolding the tensors into matrices that expose the modes being contracted over, then performing matrix multiplication, then folding the result back up into a tensor again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vectorized operations in Matlab that allow you to do this are &lt;code&gt;shiftdim&lt;/code&gt; and &lt;code&gt;permute&lt;/code&gt; to shift or permute the modes of the tensor (to bring the desired modes to the front or the back, or do the reverse later), and &lt;code&gt;reshape&lt;/code&gt; to unfold and fold the tensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Python with numpy, you can use the built in commands &lt;code&gt;tensordot&lt;/code&gt; or &lt;code&gt;einsum&lt;/code&gt;, which do this under the hood.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Timing example:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a timing example you can run to get you an idea of how much faster this is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 250;&#xA;T = randn(n,n,n);&#xA;y = randn(1,n);&#xA;z = randn(1,n);&#xA;disp('Vectorized:')&#xA;tic&#xA;a1 = z*reshape(y*reshape(shiftdim(T,1), n, n*n), n, n);&#xA;toc&#xA;&#xA;a2 = zeros(1,n);&#xA;disp('Unvectorized:')&#xA;tic&#xA;for ii=1:n&#xA;    for jj=1:n&#xA;        for kk=1:n&#xA;            a2(ii) = a2(ii) + T(ii,jj,kk)*y(jj)*z(kk);&#xA;        end&#xA;    end&#xA;end&#xA;toc&#xA;&#xA;disp('Error:')&#xA;disp(norm(a2-a1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On my laptop this displays the following results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Vectorized:&#xA;Elapsed time is 0.056645 seconds.&#xA;Unvectorized:&#xA;Elapsed time is 1.781048 seconds.&#xA;Error:&#xA;   3.1182e-11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2016-10-29T20:35:56.243" LastActivityDate="2016-10-29T20:35:56.243" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25372" PostTypeId="2" ParentId="14667" CreationDate="2016-10-29T13:11:14.780" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/nschloe/quadpy&quot; rel=&quot;nofollow noreferrer&quot;&gt;quadpy&lt;/a&gt; (a small project of mine) might be helpful. For quadrature of line segments, it features&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chebyshev-Gauß,&lt;/li&gt;&#xA;&lt;li&gt;Clenshaw-Curtis,&lt;/li&gt;&#xA;&lt;li&gt;Fejér (both types),&lt;/li&gt;&#xA;&lt;li&gt;Gauß-Hermite,&lt;/li&gt;&#xA;&lt;li&gt;Gauß-Laguerre,&lt;/li&gt;&#xA;&lt;li&gt;Gauß-Legendre,&lt;/li&gt;&#xA;&lt;li&gt;Gauß-Lobatto,&lt;/li&gt;&#xA;&lt;li&gt;Gauß-Patterson (7 schemes up to degree 191),&lt;/li&gt;&#xA;&lt;li&gt;Gauß-Radau,&lt;/li&gt;&#xA;&lt;li&gt;open and closed Newton-Cotes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2017-03-09T11:42:03.670" LastActivityDate="2017-03-09T11:42:03.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25373" PostTypeId="1" CreationDate="2016-10-29T18:08:12.943" Score="2" ViewCount="195" Body="&lt;p&gt;I'm working on a certain problem of slow, non-Newtonian, thin-film flow. This problem can be modelled with the incompressible Stokes equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla\cdot 2\mu(\dot\varepsilon)\dot\varepsilon - \nabla p + \rho g = 0, \qquad \nabla\cdot u = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is the velocity, $\mu$ is the (nonlinear) viscosity, $\dot\varepsilon$ is the strain rate tensor, $p$ the pressure, $\rho$ the fluid density and $g$ the gravitational acceleration vector. In my field, there's a good approximation for when the fluid doesn't slip at all along the lower boundary, and a different approximation for when the fluid is in free slip along the lower boundary. The Stokes equations are costly to solve in the whole domain, so some folks have tried to come up with approximations to it which work in both the no-slip and free-slip regimes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea to approach this is to use semi-discretization in the $z$-direction, but with spectral methods. From looking at experimental evidence, I have a strong hunch that most of the vertical variation could be captured with the Legendre polynomials up to degree 4. The velocity field would be approximated as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x, y, z) = \sum_lp_l(z)u_l(x, y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to derive a coupled system of PDE for the fields $u_l$. This coupled system for $\{u_l\}$ can then be solved using, e.g. Galerkin finite element methods, finite difference methods, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for any references or case studies where this approach has been used. I googled and found &lt;a href=&quot;https://link.springer.com/article/10.1007/BF01410105&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; on solving the Navier-Stokes equations in domains with only one direction of periodicity. There's also &lt;a href=&quot;https://hal-enpc.archives-ouvertes.fr/file/index/docid/377492/filename/epvmms.pdf&quot; rel=&quot;nofollow&quot;&gt;this paper&lt;/a&gt; which is more from a model reduction point of view. If there's a buzzword for this kind of approach I'd like to know what it is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, any software implementations would be helpful. I can write this from the ground up using deal.II myself. I'm less familiar with FEniCS, Firedrake, or LifeV; do any of them have a way to create Frankenstein finite element spaces like this?&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-10-31T19:52:22.360" Title="Finite element method in $x$, $y$, spectral method in $z$" Tags="|finite-element|spectral-method|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25375" PostTypeId="1" CreationDate="2016-10-29T21:48:49.980" Score="2" ViewCount="880" Body="&lt;p&gt;&lt;em&gt;This is a follow-up to &lt;a href=&quot;https://scicomp.stackexchange.com/a/25369/713&quot;&gt;this answer&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you have a possibly very ill-conditioned matrix $A$, and you compute its inverse with LU/GE to get $X_{\text{lu}}\approx A^{-1}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Newton-Raphson iteration&#xA;$$ f: X \mapsto 2X-XAX $$&#xA;generally converges to the matrix inverse of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does computing applying the Newton-Raphson iteration to $X_{\text{lu}}$, that is, computing $f(X_{\text{lu}})$, $f(f(X_{\text{lu}}))$ make the resulting matrix inverse more accurate, or less accurate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I was unable to find a reference directly, so I'm asking if someone knows for sure.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The absolute error will be proportional to $\|(f')^{-1}\|$ (e.g., &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;, Chapter 25), but $f'$ vanishes at $X=A^{-1}$. So presumably this means that applying $f$ could only help when the errors came from the growth factors in the LU factorization step? But doesn't that mean that applying $f$ hurts at other times since it relies on computing a root of a function with a vanishing Jacobian, which would make it less accurate than LU? For example, in the one-dimensional case with a root of multiplicity 2, Newton's method would have error $O(\sqrt{\epsilon})$, but LU/GE always has error $O(\rho \epsilon)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried it on all $16\times 16$ matrices in &lt;code&gt;MatrixDepot.jl&lt;/code&gt;, and it seems like it almost always hurts accuracy, never helps by more than $\frac13$, and often fails. (It's not very clear what matrices to test.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;module MatrixInverseNewtonRaphson&#xA;&#xA;using MatrixDepot&#xA;&#xA;F(A, X) = 2X - X*A*X&#xA;R(A, X) = norm(A*X - eye(A), 1)&#xA;&#xA;function go(name, m)&#xA;    A = full(matrixdepot(name, m))&#xA;    B = inv(A)&#xA;    B1 = F(A, B)&#xA;    X = 0.5A' / norm(A*A', 2)&#xA;    for i in 1:2m; X = F(A, X); end&#xA;    (R(A, B1) / max(eps(eltype(A)), R(A, B)), R(A, B), R(A, X), name)&#xA;end&#xA;&#xA;function main(m=16)&#xA;    e = []&#xA;    for name in matrixdepot(&quot;all&quot;)&#xA;        try&#xA;            push!(e, go(name, m))&#xA;        catch err&#xA;            println(err)&#xA;        end&#xA;    end&#xA;    sort(e)&#xA;end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-10-31T17:51:03.447" Title="Does applying the Newton-Raphson iteration for matrix reciprocal refine a matrix inverse from LU/GE?" Tags="|linear-algebra|numerical-analysis|convergence|newton-method|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25377" PostTypeId="1" CreationDate="2016-10-30T10:24:23.893" Score="1" ViewCount="477" Body="&lt;p&gt;I have been trying to solve a nonlinear PDE related to structural mechanics (nonlinear Timoshenko beam to be precise). I am doing both &lt;em&gt;h&lt;/em&gt;-refinement and &lt;em&gt;p&lt;/em&gt;-refinement to reach the solution. The nonlinear equations were solved using Newton-Raphson method and I am comparing my solution (transverse displacement at a specified load) with the benchmark solution (the benchmark solution was generated using a highly &lt;em&gt;p&lt;/em&gt;-refined model and this benchmark solution was interpolated to the solution nodes using spline interpolation). The relative error was computed using both $L_2$-norm and $L_{\infty}$-norm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{Relative error}= \frac{||U_\text{FEM}-U_\text{benchmark}||}{||U_\text{benchmark}||}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The codes were written in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said all these, my problem is, the convergence of the solution (with benchmark) stops after a limit and start oscillating around that limit (See the figure attached for reference- LFEM is FEM with linear shape function, QFEM is FEM with quadratic shape function, PFEMs are different &lt;em&gt;p&lt;/em&gt;-refined finite elements). I have verified the solution with commercial FEM package and it matches perfectly with it. I have combed the whole code for error with no avail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ScvaI.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ScvaI.jpg&quot; alt=&quot;Convergence plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the sources of error? I have observed that the solutions are accurate till some significant digits, and that doesn't change even after the refinements. The behavior is as expected till the relative error reaches $10^{-6}$, and the problem starts afterward.&lt;/p&gt;&#xA;" OwnerUserId="22164" LastEditorUserId="9667" LastEditDate="2016-10-31T01:45:30.053" LastActivityDate="2016-11-01T23:59:41.153" Title="Does the convergence of finite element have limit?" Tags="|finite-element|convergence|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25378" PostTypeId="2" ParentId="25373" CreationDate="2016-10-30T10:54:31.890" Score="1" Body="&lt;p&gt;If I understood your problem correctly, this paper may help you. In this paper, they are trying to solve a thermo-mechanical coupled problem. The structural equations in the domain are modeled using  1-D models and thermal equations in the domain are modeled using 2-D. Therefore they have consistently reduced the dimension of thermal equations in 2 D by using a selective weighted residual technique in one direction which render it 1-D.&#xA;They approximate it as&#xA;$T(x,z,t)=g(z)\bar{T}(x,t)$. please refer to the paper for detailed implementation. Hope this will help you&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0020740315000600&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to the paper http://www.sciencedirect.com/science/article/pii/S0020740315000600&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22164" LastActivityDate="2016-10-30T10:54:31.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25381" PostTypeId="2" ParentId="2249" CreationDate="2016-10-30T20:35:58.727" Score="0" Body="&lt;p&gt;Since your evaluations are expensive, you need to take advantage of running sevaral function evaluations in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend you to take a look at this &lt;a href=&quot;https://github.com/paulknysh/blackbox&quot; rel=&quot;nofollow&quot;&gt;code&lt;/a&gt;. The math behind is described &lt;a href=&quot;https://arxiv.org/pdf/1605.00998.pdf&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22170" LastActivityDate="2016-10-30T20:35:58.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25382" PostTypeId="1" CreationDate="2016-10-31T04:42:50.627" Score="3" ViewCount="2017" Body="&lt;p&gt;I need to calculate the integral of this function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(z):&#xA;    return ((1-2*z)*np.exp(-d/z))/(((1-z)**(2+d))*(z**(2-d)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here d is a constant. I am using this function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; import scipy.integrate as integrate&#xA; return integrate.quad(f,0, 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I need to calculate this for many different d values. I tried putting all d values in a numpy array but it didn't work. Any idea how to vectorize this?&lt;/p&gt;&#xA;" OwnerUserId="22137" LastActivityDate="2016-10-31T20:57:37.720" Title="integral including a vector" Tags="|python|scipy|numpy|integration|vectorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25384" PostTypeId="2" ParentId="943" CreationDate="2016-10-31T08:13:14.617" Score="0" Body="&lt;p&gt;Imagine a problem that can be solved by lots of brute force, like Travelling Salesman.  Then imagine you've got racks of servers with 8 spanky video cards each, and each card has 3000 CUDA cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply solve ALL the possible salesman's routes and then sort for time/distance/some metric.  Sure you're throwing away almost 100% of your work, but brute force is a viable solution sometimes.&lt;/p&gt;&#xA;" OwnerUserId="22175" LastActivityDate="2016-10-31T08:13:14.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25385" PostTypeId="1" CreationDate="2016-10-31T08:49:47.050" Score="1" ViewCount="209" Body="&lt;p&gt;This example is a dummy example, but it describes exactly whats goes on in my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result from some sympy calculation and manipulation i got two arrays&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;array_1 = ([f_1(x,y)      0         f_3(x,y) ])&#xA;array_2 = ([g_1(x,y)   g_2(x,y)     g_3(x,y) ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where f_n(x,y) and g_n(x,y) are complex functions. When I print them to se what kind of creature they are, they both are on the form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Matrix([[#, #, #]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as expected from a sympy matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore I use lambdify to convert the arrays by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lambda_array_1 = lambdify([x, y], array_1, 'numpy' )&#xA;lambda_array_2 = lambdify([x, y], array_2, 'numpy' )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I let&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = 1&#xA;y = 1&#xA;x_array = np.array([x], dtype = complex)&#xA;y_array = np.array([y], dtype = complex)&#xA;&#xA;pythonarray_1 = lambda_array_1(x_array , y_array )&#xA;pythonarray_1 = np.reshape(pythonarray_1, 3) &#xA;&#xA;pythonarray_2 = lambda_array_2(x_array , y_array )&#xA;pythonarray_2 = np.reshape(pythonarray_2, 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I then print the two pythonarrays I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pythonarray_1 = array([array([# + #j]), array([# + #j]), array([# + #j])], dtype=object)&#xA;&#xA;pythonarray_2 = array([# + #j, # + #j, # + #j ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where I need the arrays on the form as in pythonarray_2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is wrong with pythonarray_1, how can I fix it, and why does it happen?&lt;/p&gt;&#xA;" OwnerUserId="21819" LastActivityDate="2016-10-31T08:49:47.050" Title="Lambdify error when sympy array contains number" Tags="|python|symbolic-computation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25386" PostTypeId="1" CreationDate="2016-10-31T10:44:05.223" Score="1" ViewCount="103" Body="&lt;p&gt;I have a loop that has a similar structure as give below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DO WHILE (t &amp;lt; tend)&#xA;  t = t + dt&#xA;  DO j = 1,dim&#xA;&#xA;    var2 = 0&#xA;    CALL func_a(param1,var2)&#xA;    CALL MPI_COMM_RANK(comm(j),rank,ierr)&#xA;    CALL func_b(param1, var2,rank)&#xA;&#xA;    var2 = var2 + 0.5d0*var3(j)&#xA;    CALL func_c(var3,param1)&#xA;    var2 = var2 - 0.5d0*var3&#xA;&#xA;    CALL func_d(var2,params)&#xA;    DO ii = 1,dim&#xA;        CALL func_e(var2,param)&#xA;        CALL func_f(var2,param)&#xA;        CALL func_g(var1,var2,comm(j),t)&#xA;        var1 = var2   &#xA;    END DO&#xA;  END DO&#xA;END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This section is a time loop that I want to port to the GPU. All functions are parallel enough to be ported to the GPU individually. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the variables named var1, var2, var3 are on the GPU memory copied to the GPU memory(devvar1, devvar2, devvar3) before the time loop starts. If I do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DO WHILE (t &amp;lt; tend)&#xA;  t = t + dt&#xA;  DO j = 1,dim&#xA;&#xA;    devvar2 = 0&#xA;    CALL cu_func_a&amp;lt;&amp;lt;&amp;lt;block,grid&amp;gt;&amp;gt;&amp;gt;(param1,devvar2)&#xA;    CALL cu_func_b&amp;lt;&amp;lt;&amp;lt;block,grid&amp;gt;&amp;gt;&amp;gt;(param1, devvar2)&#xA;&#xA;    devvar2 = devvar2 + 0.5d0*devvar3(j)&#xA;    CALL cu_func_c&amp;lt;&amp;lt;&amp;lt;block,grid&amp;gt;&amp;gt;&amp;gt;(devvar3,param1)&#xA;    devvar2 = devvar2 - 0.5d0*devvar3&#xA;&#xA;    CALL cu_func_d&amp;lt;&amp;lt;&amp;lt;block,grid&amp;gt;&amp;gt;&amp;gt;(devvar2,params)&#xA;    DO ii = 1,dim&#xA;        CALL cu_func_e&amp;lt;&amp;lt;&amp;lt;block,grid&amp;gt;&amp;gt;&amp;gt;(devvar2,param)&#xA;        CALL cu_func_f&amp;lt;&amp;lt;&amp;lt;block,grid&amp;gt;&amp;gt;&amp;gt;(devvar2,param)&#xA;        CALL cu_func_g&amp;lt;&amp;lt;&amp;lt;block,grid&amp;gt;&amp;gt;&amp;gt;(devvar1,devvar2,t)&#xA;        devvar1 = devvar2   &#xA;    END DO&#xA;  END DO&#xA;END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;would it copy from the GPU memory to the GPU memory (in the same GPU) itself ? Or is there some unexpected behaviour possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I create another kernel (specifically to multiply, add and copy the vectors) and launch it within the kernels (cu_func_a, cu_func_b,...) so that this copying outside is not necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I have another question. I am using multiple K20x GPU's, which have about 6 GB global memory. When I copy from the CPU to the GPU before the time loop, what happens if the overall memory I copy exceeds 6 GB ? Does it give me an error or I just get unexpected behaviour ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if anything above is unclear. I am happy to clarify and discuss ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16601" LastActivityDate="2016-11-04T19:03:02.603" Title="Time loop for multiple GPU's" Tags="|parallel-computing|gpu|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25387" PostTypeId="1" CreationDate="2016-10-31T11:30:35.887" Score="1" ViewCount="137" Body="&lt;p&gt;Assuming that the underlying algorithm can be ported to multi GPU, what aspects should one consider while porting from MPI (on multiple nodes) to multi GPU (again on multiple nodes)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Making use of Hyper Q ? Of MPS ? Peer to peer communication ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we consider an MPI code that solves for the Navier Stokes equations using ADI, on multiple nodes with domain decomposition, and solves for the pressure and velocity by solving the schur complement matrices and interface unknowns induced by the domain decomposition. We could solve for internal unknowns by a simple tridiagonal solver (say, Thomas algorithm) which is highly parallel and use the schur complement method to solve for the interface unknowns and repeat this throughout the time stepping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The schur complement would involve solving directional systems (MPI ranks in depending on the cartesian decomposition in one direction) and hence would involve communication between the ranks in one direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue of communication arises when porting to the GPU, as we would need to communicate between the GPU's on different nodes which are logically in the same direction (considering 1:1 ratio of MPI ranks: GPU). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions or ideas are welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank You. &lt;/p&gt;&#xA;" OwnerUserId="16601" LastActivityDate="2016-10-31T11:30:35.887" Title="Porting from MPI to GPU" Tags="|parallel-computing|navier-stokes|gpu|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25388" PostTypeId="2" ParentId="25386" CreationDate="2016-10-31T11:48:20.210" Score="1" Body="&lt;p&gt;I divide the answer for the different question. (I answer to some)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can I create another kernel (specifically to multiply, add and copy the vectors) and launch it within the kernels (cu_func_a, cu_func_b,...) so that this copying outside is not necessary.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes you can, this is a feature called dynamic parallelism. It was introduced in CUDA 5. In these links, &lt;a href=&quot;https://devblogs.nvidia.com/parallelforall/introduction-cuda-dynamic-parallelism/&quot; rel=&quot;nofollow noreferrer&quot;&gt;part1&lt;/a&gt; and &lt;a href=&quot;https://devblogs.nvidia.com/parallelforall/cuda-dynamic-parallelism-api-principles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;part2&lt;/a&gt; there is the explanation how to use with examples.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;would it copy from the GPU memory to the GPU memory (in the same GPU) itself ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure I understand, if you question id if you can modify/copy a vector in the GPU memory yes you can. For example&#xA;with &lt;a href=&quot;http://docs.nvidia.com/cuda/thrust/#axzz4OevOxbvp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thrust&lt;/a&gt; you can copy (using '=') device vector in device vector or modify (see saxpy example) device vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; there are different libraries for cuda for specific arguments,&#xA;for optimum performance it is important to use them (see for example &lt;a href=&quot;https://developer.nvidia.com/gpu-accelerated-libraries&quot; rel=&quot;nofollow noreferrer&quot;&gt;list&lt;/a&gt;, &lt;a href=&quot;https://developer.nvidia.com/cusp&quot; rel=&quot;nofollow noreferrer&quot;&gt;cups&lt;/a&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When I copy from the CPU to the GPU before the time loop, what happens if the overall memory I copy exceeds 6 GB ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This depends by the GPU. From the CUDA 8 &lt;a href=&quot;http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-oversubscription&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;, section &lt;em&gt;J.1.3. GPU Memory Oversubscription&lt;/em&gt;, using the unified memory with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GPU &amp;lt; 6.x cite: &quot;cannot allocate more managed memory than the physical size of GPU memory&quot;&lt;/li&gt;&#xA;&lt;li&gt;GPU &gt; 6.x cite: &quot;in other words they can allocate, access, and share arrays larger than the total physical capacity of the system, enabling out-of-core processing of very large datasets. cudaMallocManaged will not run out of memory as long as there is enough system memory available for the allocation.&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="22025" LastEditDate="2016-11-04T19:03:02.603" LastActivityDate="2016-11-04T19:03:02.603" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25389" PostTypeId="1" CreationDate="2016-10-31T13:03:01.063" Score="4" ViewCount="688" Body="&lt;p&gt;I am trying to solve a LP in which one of my constraints is of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{A(x)}{B(x)} = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I transform this constraint into a linear one&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A(x) - B(x) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when CPLEX solves the problem, this constraint is of the order of ~1.e-5, but the ratio is of the order of ~1.4, which is a large deviation for my needs. Unfortunately the kappa value (condition number of the optimal basis) of the problem is kappa ~10^10, and this doesn't make things easier. I suspect this is the source of the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question&lt;/strong&gt;: is it possible (given the value of kappa) to increase the precision of this constraint such as the ratio to be accurate within 2 decimal digits (at least) ~ 1.00something ? &lt;/p&gt;&#xA;" OwnerUserId="14280" LastEditorUserId="20417" LastEditDate="2016-10-31T20:22:54.953" LastActivityDate="2016-10-31T20:22:54.953" Title="Precision of ratio constraint in a linear program" Tags="|optimization|convex-optimization|constrained-optimization|linear-programming|" AnswerCount="0" CommentCount="6" ClosedDate="2018-01-07T17:44:56.707" ContentLicense="CC BY-SA 3.0" />
  <row Id="25390" PostTypeId="1" AcceptedAnswerId="25392" CreationDate="2016-10-26T07:25:42.073" Score="14" ViewCount="449" Body="&lt;p&gt;I'm doing some simulations of the Langevin equation, for various external forces. Being told that C's &lt;code&gt;rand()&lt;/code&gt; from &lt;code&gt;stdlib.h&lt;/code&gt; can introduce bias in my results, I'm using a Mersenne Twister.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, I would like to know (and see) exactly what kind of errors a linear congruential generator can introduce in my simulation. These are things that I have tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generating 3D tuples of randoms to try to see hyperplanes. I can't see anything.&lt;/li&gt;&#xA;&lt;li&gt;Doing the FFT of a large vector of random numbers. It's almost the same for both the Mersenne Twister and &lt;code&gt;rand()&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Checking the equipartition principle for a particle in Brownian motion. Both integrators agree in the expected value of $\langle \text{KE}\rangle=\frac{1}{2}k_BT$ with the same number of significant digits.&lt;/li&gt;&#xA;&lt;li&gt;Seeing how well they bin in a number of bins that is not a power two. Both give the same qualitative results, no one being better.&lt;/li&gt;&#xA;&lt;li&gt;Looking at Brownian paths to see clear divergences from $\langle x\rangle = 0$. Again, no luck.&lt;/li&gt;&#xA;&lt;li&gt;Distribution of points in a circle. Filled, and only in the perimeter. Between all of them and between nearest neighbours (Shor's answer, below in the comments). Available in &lt;a href=&quot;https://gist.github.com/RedPointyJackson/59ac3e4bdafa781ffec00713804fe5c6&quot; rel=&quot;nofollow&quot;&gt;this gist&lt;/a&gt;, just run it with Julia 0.5.0 after installing the needed libraries (see the gist for instructions).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would like to emphasize that I am looking for introduced bias in the context of physical simulations. For example, I have seen how &lt;code&gt;rand()&lt;/code&gt; fails miserably the dieharder tests while the Mersenne Twister doesn't, but for the moment that doesn't mean too much for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any physical, concrete, examples on how a bad random number generator wrecks a Montecarlo simulation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note: I have seen how PRNG's like &lt;code&gt;RANDU&lt;/code&gt; can be awful. I'm interested on not obvious examples, of generators that look innocent but ultimately introduce bias.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="20251" OwnerDisplayName="RedPointyJackson" LastActivityDate="2016-11-02T22:26:48.497" Title="Quality of linear congruential generators for random numbers" Tags="|computational-physics|" AnswerCount="2" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="25392" PostTypeId="2" ParentId="25390" CreationDate="2016-10-31T18:43:42.477" Score="3" Body="&lt;p&gt;One interesting reference that describes a failure of a Monte Carlo simulation of a physical system due to inadequate RNG (although they didn't use an LCG) is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Ferrenberg and D. P. Landau.  Monte Carlo Simulations: Hidden Errors from &quot;Good&quot; Random Number Generators.  Physical Review Letters 63(23):3382-3384, 1992.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Ising models that Ferrenberg and Landua studied are good tests of RNG's because you can compare with an exact solution (for the 2-D problem) and find errors way out in the digits.  These models should show the faults in an old fashioned 32 bit arithmetic PMMLCG without too much difficulty.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting reference is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;H. Bauke and Stephan Mertens.  Pseudo Random Coins Show More Heads Than Tails.  arXiv:cond-mat/0307138 [cond-mat.stat-mech]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bauke and Mertens make a strong case against binary linear feedback shift register style random number generators.  Bauke and Mertens have some other papers related to this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be difficult to find the Marsaglia planes in a 3D scatter plot.  You can try to rotate the plot to get a better view and sometimes they'll just pop out at you.  You can also do 3D tests of statistical uniformity- for the older 32 bit LCG's, these will fail at quite small numbers of bins.  e.g. a uniformity test with a 20x20x20 grid of bins in 3 dimensions is sufficient to detect lack of uniformity for the widely used (and previously well regarded) PMMLCG with m=2^31-1, a=7^5.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-11-01T15:17:12.070" LastActivityDate="2016-11-01T15:17:12.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25393" PostTypeId="2" ParentId="25382" CreationDate="2016-10-31T18:48:07.167" Score="2" Body="&lt;p&gt;The function &lt;code&gt;integrate.quad&lt;/code&gt; is a python wrapper to the DQAGSE function from QUADPACK. This function uses adaptive quadrature, i.e. it will apply a fixed rule (in this case Gauss-Kronrod) on intervals that it will adaptively refine trying to reach the absolute and/or relative tolerance you requested. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the parameter &lt;code&gt;d&lt;/code&gt; alters the behaviour of the function, it will also change the iterative interval refinement. Hence, it doesn't make sense to &quot;vectorize&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you could do is use the Pool class from the &lt;a href=&quot;https://docs.python.org/3/library/multiprocessing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiprocessing&lt;/a&gt; module in the standard Python library. Distribute your values of &lt;code&gt;d&lt;/code&gt; over multiple cores. Wrap your integral in a function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.integrate as integrate&#xA;from multiprocessing import Pool&#xA;&#xA;def f(z,d):&#xA;    return ((1-2*z)*np.exp(-d/z))/(((1-z)**(2+d))*(z**(2-d)))&#xA;&#xA;def F_int(d):&#xA;    return integrate.quad(f,0.0,0.5,args=(d))&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    d_array = np.array([1, 0.1, 2.0])&#xA;    pool = Pool(processes=3)&#xA;    print(pool.map(F_int, d_array))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The only restriction with this solution is that it doesn't work in interactive modes (so not in Jupyter notebooks nor Qtconsole). See &lt;a href=&quot;https://stackoverflow.com/questions/24466804/multiprocessing-breaks-in-interactive-mode&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-10-31T20:57:37.720" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25395" PostTypeId="2" ParentId="25377" CreationDate="2016-10-31T19:47:00.127" Score="4" Body="&lt;p&gt;The overall error of a finite element solution often has many components. For example, these may include (i) the discretization error due to a finite mesh and/or finite polynomial degree, (ii) the error incurred in solving the linear system that defines the nodal values of your finite element solution, (iii) if your problem is nonlinear, the error incurred by terminating a nonlinear iteration (such as a Newton scheme) after only finitely many iterations, (iv) the error incurred by numerical round-off; and possibly others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order for your error truly to go to zero, you need to make sure that &lt;em&gt;every single one&lt;/em&gt; of these error contributions goes to zero. In other words, as you refine the mesh, you also need to solve the linear system more accurately, run more nonlinear iterations, and possibly switch from single to double precision. If you don't control &lt;em&gt;each one&lt;/em&gt; of these error sources, eventually one of them will come to dominate all of the others. For example, if you only do 3 Newton iterations, then it doesn't matter how accurately you solve each of them: the overall error will simply be dominated by the fact that you stop after 3 Newton iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, there is also the possibility that your &quot;reference&quot; solution is not completely accurate. For example, if you read it from a file into which you stored the nodal values of the reference solution with 6 digits of accuracy, you can not expect that the error between your numerical solution and this &quot;reference&quot; solution will be less than $10^{-6}$ simply because the last digit of the reference solution is no longer accurate.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-31T19:47:00.127" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25396" PostTypeId="2" ParentId="25373" CreationDate="2016-10-31T19:52:22.360" Score="1" Body="&lt;p&gt;Not a Franken-element at all :-) If you know the number $N$ of modes you want to consider in z-direction, then you simply get a coupled system of $N$ dim-1 dimensional PDEs, all of which happen to have fundamentally the same structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The place where this kind of discretization is most often used is if you have a cylindrical coordinate system. There, people often use a Fourier transform in the angle $\varphi$. If you keep the first $N$ modes, then you end up with a system of $N$ couples PDEs in $r$-$z$ only. I've seen this many times, but can't recall any concrete papers other than the ones by J.-L. Guermond on magnetohydrodynamics. His whole Sfemans code is based on this concept.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-10-31T19:52:22.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25397" PostTypeId="1" AcceptedAnswerId="25408" CreationDate="2016-10-31T20:23:33.913" Score="0" ViewCount="102" Body="&lt;p&gt;I am simulating the wave equation using FEM. For a 2D wave equation, when I visualise my output in Paraview, I see a separate solution in 'x' and 'y' direction for each node on the mesh. Therefore, if I freeze the time, I will have a solution vector of the size: (No. of nodes x 2). This makes sense to me because when a wave strikes a surface, we represent the time history graphs in all 3 directions (for 3D). So, it is obvious that I should have a displacement solution vector in 2 directions for 2D. However, is it possible to substitute the 2 solution values obtained in 'x' and 'y' direction on each node, by a single value? I believe they are just the projection of a single solution and therefore, is it reasonable to use Pythagoras theorem on the 'x' and 'y' solution value to obtain a single solution value at each node?&lt;/p&gt;&#xA;" OwnerUserId="16705" LastEditorUserId="16705" LastEditDate="2016-11-01T03:21:03.970" LastActivityDate="2016-11-01T21:34:34.013" Title="Regarding solution vector of the wave equation" Tags="|finite-element|wave-propagation|projection|vector|paraview|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25398" PostTypeId="1" AcceptedAnswerId="25439" CreationDate="2016-11-01T03:36:01.770" Score="12" ViewCount="12145" Body="&lt;p&gt;I understand these equations in particular can be solved easily without use of computational methods. Although right now I am concerned with trying to solve these equations using numerical integration strictly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\partial ^2 E}{\partial t ^2} = c^2 \frac {\partial ^2 E}{\partial z ^2} \tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E(t=0) = \sin(z), \frac {\partial E}{\partial t}(t=0) = -c \cos(z)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where c $= 3 \times 10^8 \frac{\text{m}}{\text{s}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the method of lines, I am trying to reduce this to a system of first-order differential equations and then apply a 4th-order Runge-Kutta method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although a necessary condition is the CFL condition which states (for explicit methods):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\text{c} \Delta t}{\Delta z} &amp;lt; \text{C}_{max} = 1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It makes sense that if I'm considering a domain of $ 0 &amp;lt; z &amp;lt; 100 \text{ m}$ where $\Delta z \sim 1 \text{m}$, then $\Delta t$ should be on the order of $10^{-9} \text{s} $. Although is the CFL condition assuming my  $\Delta z$ is at an appropriate spatial width to account for changes in $E$ over space? Since if I made $\Delta z$ arbitrarily large (e.g. $10^9 \text{m}$) to consider a larger domain in $z$ now, then I could set $\Delta t$ to $1\text{s}$ and satisfy the CFL condition. Although based on (1) and the initial conditions, we know $E = \sin(z-ct)$, and would need to be sampled at a spatial frequency of $1\text{m}^{-1}$ and time frequency of GHz to be evaluated correctly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To raise my question: it appears the CFL condition is only a necessary but not sufficient condition. But in a more general case where one does not necessarily know how rapidly $E$ will vary with $z$ (e.g. a more complicated hyperbolic PDE where $E = \text{A}(z,t)\text{B}(z,t)\text{C}(z,t)$ where $\text{A, B,}$ and $\text{C}$ are variable coefficients that vary with $t$ and $z$ on unknown scales scales (e.g. stiff)), how can we define $\Delta z$ a priori when applying the method of lines? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I was to advance in $z$ as opposed to $t$ (i.e. discretizing $t$) which is the contrary to the typical method of lines, I would have a condition reading: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\Delta z}{\text{c} \Delta t} &amp;lt; \text{C}^{z}_{max} = 1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which would thus place a limit on how small $\Delta z$ should be based on $\Delta t$ (which I define ahead of time) if I want to model the spatial frequency of $E$ correctly. But once again, this assumes I have chosen an appropriate $\Delta t$ for the problem—it is the discretized variable I am not quite understanding how to define its step since we can always implement an adaptive step for the other variable. But how can one implement an adaptive step for both independent variables when these equations involve derivatives of the other variable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is there a well-defined value for $\text{C}_{max}$ when using implicit methods (e.g. BDF)?&lt;/p&gt;&#xA;" OwnerUserId="20407" LastEditorUserId="20407" LastEditDate="2016-11-01T04:03:02.747" LastActivityDate="2016-11-07T19:46:13.567" Title="Understanding the Courant–Friedrichs–Lewy condition" Tags="|pde|hyperbolic-pde|stiffness|method-of-lines|cfl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25399" PostTypeId="1" AcceptedAnswerId="28360" CreationDate="2016-11-01T04:00:16.250" Score="2" ViewCount="94" Body="&lt;p&gt;I have a linear system of equations $Ax = b$ where the number of unknowns $N$ is intractably large but the right-hand side has only small support and the unknown $x$ is known to decay exponentially. I hence want to find a sequence of reduced linear systems $A_n x_n = b_n$ with only $n \leq N$ unknowns such that $x_n \to x$ as $n \to N$. Is there any theory available on how to find suitable $A_n, b_n$ such that I can guarantee well-posedness of the reduced problems and convergence, possibly under additional assumptions on $A$ and $b$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a fairly limited understanding of what wavelets are and how they work, but as far as I see we should face the same problem there: we have an in principle infinite expansion of the solution in terms of the wavelet basis functions, but the smoothness of the solution implies that the corresponding coefficients decay and hence we can truncate. This makes me wonder how the above problems of well-posedness and convergence for truncated systems are addressed there.&lt;/p&gt;&#xA;" OwnerUserId="21342" LastActivityDate="2018-03-13T22:52:31.400" Title="Reduction of linear system with decaying unknown" Tags="|wavelet|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25400" PostTypeId="1" CreationDate="2016-11-01T04:00:35.233" Score="1" ViewCount="208" Body="&lt;p&gt;I need to find the roots of a function. I am currently using scipy.optimize.fsolve&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def f(z): &#xA;        return ((1-2*z)*np.exp(-d/z))/(((1-z)**(2+d))*(z**(2-d)))&#xA;        # d is a constant ndarray of shape(300000,)&#xA;        # hence f(z) is an ndarray of shape (300000,)&#xA;&#xA;    def trap(func,a,b,num): #trapezoidal approach to integration&#xA;        xlinear=np.linspace(a,b,num)&#xA;        dx=np.diff(xlinear)&#xA;        fx=func(xlinear[:,None])&#xA;        f12=fx[:-1,:]+fx[1:,:]&#xA;        return np.dot(dx,f12)/2.0&#xA;        #also returns an array of shape (300000,)&#xA;&#xA;    def integral(p):&#xA;        return trap(f,0,p,1000)&#xA;&#xA;    root= fsolve(f,0.75) # 0.75 because the answer should lie in (0.5,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The issue is that I was expecting fsolve to return an array in line with returns of functions f and trap. However, somehow I am only getting a array of shape (1,)&lt;/p&gt;&#xA;" OwnerUserId="22137" LastEditorUserId="22137" LastEditDate="2016-11-01T04:18:16.520" LastActivityDate="2016-11-01T04:18:16.520" Title="vectorizing optimization or root finding" Tags="|optimization|python|scipy|roots|" AnswerCount="0" CommentCount="1" ClosedDate="2016-11-24T19:47:44.683" ContentLicense="CC BY-SA 3.0" />
  <row Id="25401" PostTypeId="1" AcceptedAnswerId="25414" CreationDate="2016-11-01T09:22:18.483" Score="2" ViewCount="207" Body="&lt;p&gt;I am reading the paper, &lt;a href=&quot;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve an optimization problem using this Navier-Stokes code as constraining PDE. Now I am trying to add an external force parameterized some input parameter to the method. The method solves the Navier-Stokes equation in 3 steps, 1) nonlinear term, 2) linear term, 3) pressure correction.&#xA;$$\text{(1)} \quad \frac{U^*-U^n}{\Delta t} = -((U^n)^2)_x - (U^nV^n)_y$$&#xA;$$\text{(2)} \quad \frac{U^**-U^*}{\Delta t} = \frac{1}{Re}(U^{**}_{xx}+U^{**}_{yy})$$&#xA;$$\text{(3)} \quad U{n+1} = U^{**} - \Delta t \nabla P^{n+1}$$&#xA;(written only for U)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I add the external force term to the step 1 like this?&#xA;$$\text{(1)} \quad \frac{U^*-U^n}{\Delta t} = -((U^n)^2)_x - (U^nV^n)_y +f $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then, will the external force corresponds to the input be modified to satisfy the incompressible condition at step 3?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I give some physically impossible external force term (like too much force), the method will automatically decrease it to satisfy the incompressible condition?&lt;/p&gt;&#xA;" OwnerUserId="14387" LastEditorUserId="14387" LastEditDate="2016-11-01T13:32:22.647" LastActivityDate="2016-11-02T09:16:06.813" Title="Adding an external force to Chorin's projection method for the Navier-Stokes equation" Tags="|navier-stokes|incompressible|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25402" PostTypeId="1" CreationDate="2016-11-01T12:58:54.333" Score="1" ViewCount="591" Body="&lt;p&gt;I have to solve the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-u_{xx}=1$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $x\in(0,1)$ and $u(0)=0,u(1)=0$. I have to solve it with the following numerical scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{h_k^2}(-\frac{1}{2}u_{k-1}+u_k-\frac{1}{2}u_{k+1})=1$. So I have to use a non-uniform grid. I have done this so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 4; %Number points&#xA;k = 0:n;&#xA;x = 0.5 - 0.5*cos(pi*k/n); %Function to generate points&#xA;&#xA;h = diff(x);&#xA;h = h(1:n-1);&#xA;h = (h.^2)'; %Difference from point 1 to n-1&#xA;&#xA;b = ones(n-1,1);&#xA;b = h.*b; %Solve&#xA;&#xA;A=sparse(diag(2*ones(n-1,1))+diag((-1)*ones(n-2,1),1)+diag((-1)*ones(n-             2,1),-1));&#xA;&#xA;u=A\b;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see I was trying to define everything I need to use a for loop and also trying to do that using just matrix multiplications. The problem is that with for loop I am not able to understand what should be the value at $u(1)$ since I only have $u(0)$ and the method requires three points. With the &quot;matrix version&quot; I am not able to understand how to compute the vector containing the differences between the pints.&lt;/p&gt;&#xA;" OwnerUserId="22185" LastEditorUserId="22185" LastEditDate="2016-11-03T08:32:27.483" LastActivityDate="2016-11-03T08:32:27.483" Title="Solving PDE in 1D with FD and MATLAB" Tags="|matlab|pde|finite-difference|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25403" PostTypeId="2" ParentId="25402" CreationDate="2016-11-01T13:38:55.910" Score="1" Body="&lt;p&gt;A few points:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Your points &lt;code&gt;x_k&lt;/code&gt; aren't in your domain $(0,1)$. Check &lt;code&gt;plot(x_k)&lt;/code&gt;: try instead &lt;code&gt;x_k = 0.5 - 0.5*cos(k*pi/n)&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Your vector &lt;code&gt;h&lt;/code&gt; could also include the distance to the endpoints &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;1&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;You can get the distance between points with &lt;code&gt;diff(x_k)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;I think your finite difference operator my be out by a factor of 2. You might want to check that, also what the error is on an irregular mesh&lt;/li&gt;&#xA;&lt;li&gt;Check the sizes of all your vectors and matrices: &lt;code&gt;h&lt;/code&gt; is 1-by-2, &lt;code&gt;m&lt;/code&gt; is 3-by-3 and &lt;code&gt;u&lt;/code&gt; is 5-by-1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have previously answered a question on &lt;a href=&quot;https://scicomp.stackexchange.com/a/21513/17671&quot;&gt;Finite-difference approximation of the 2nd derivative operator matrix for a staggered grid&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You've said in a comment that you don't know how to check if your solution on a uniform grid is correct: You can investigate this by using the exact solution&#xA;$$u(x) = \frac{x(1-x)}{2}\,,$$&#xA;(found by integrating your ODE twice and imposing the boundary conditions to find the constants of integration), and comparing the numerical solution with the exact solution as $n$ increases, something like&#xA;$$E_n:= \max_{j\in[1,\dotsc,n]}\left| U_j - u(x_j) \right|\,,$$&#xA;where $U_j$ is your numerical solution at the point $x_j$.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-11-02T12:50:34.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25405" PostTypeId="2" ParentId="25398" CreationDate="2016-11-01T17:16:52.173" Score="9" Body="&lt;p&gt;You are correct: If you satisfy the CFL condition, then all that guarantees is that your scheme is &lt;em&gt;stable&lt;/em&gt;, i.e., the numerical solution does not go to infinity. But the CFL condition says nothing about how &lt;em&gt;accurate&lt;/em&gt; the numerical solution is. For that, indeed $\Delta z$ and $\Delta t$ must also be &lt;em&gt;small enough&lt;/em&gt; compared to the features of the exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to note that $C_\text{max}$ may not be 1, but can also be smaller (sometimes much smaller) depending on what time stepping scheme you use. Generally, higher order schemes will yield smaller values for $C_\text{max}$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-01T17:16:52.173" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25406" PostTypeId="2" ParentId="25399" CreationDate="2016-11-01T17:43:18.490" Score="0" Body="&lt;p&gt;This is akin to having a partial differential equation that is posed to an infinite domain. But we can't solve on infinite domains, so we need to truncate it somewhere, and we can typically do that because the source is localized in a small area of the domain and the solution decays away from the source. One then needs to pose an &quot;artificial boundary condition&quot;, for example by only solving in a ball of radius $R$ around the center of the source, and prove that the solution converges as $R\rightarrow \infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing more about the problem you have, there is little I can offer in terms of concrete suggestions. But you could look into the literature on artificial boundary conditions, truncated domains, and exterior problems, to see how this is done there and how that may apply to your own problem.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-01T17:43:18.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25408" PostTypeId="2" ParentId="25397" CreationDate="2016-11-01T21:34:34.013" Score="0" Body="&lt;p&gt;From what you are saying, you are probably visualising the intensity of the electric (?) field inside of your domain. Since you are in 2-D, your electric field $\vec{E}(x,y,t)$ has two components: $\vec{E}(x,y,t)=E_x(x,y,t)\hat{x}+E_y(x,y,t)\hat{y}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you evaluate the field values at the FEM grid nodes (probably a setting in your solver), you have a following choice:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Visualize your $E_x(x,y,t)$ and $E_y(x,y,t)$ separately. If you fix your time $t$, you are going to have a plot with values of the intensity at $\hat{x}$ or $\hat{y}$ direction at the grid points, and you can make it a surface plot by using some kind of interpolation rule in between.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the magnitude of the vector $|\vec{E}(x,y,t)|=E_x^2+E_y^2$ and visualize just one quantity as you mentioned in your question. Such visualization is still useful, but you are certainly loosing a sense of direction here.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Plot a vector field using Glyph (or streamlines). By doing that, at each of your 2-D grid points you would see a vector that directed according to the values $E_x(x,y)$ and $E_y(x,y)$. This is the most natural way to visualize a vector quantity. And if you play an animation in time, the vectors would even rotate, showing you the field evolution. You can read about vector field visualization in Paraview on their website.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From my experience, all three representations are useful, but 1 and 3 are usually the most informative ones.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2016-11-01T21:34:34.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25409" PostTypeId="2" ParentId="25402" CreationDate="2016-11-01T21:44:41.550" Score="1" Body="&lt;p&gt;First of all, your finite difference is wrong, as Steve pointed out. It has to be &#xA;\begin{equation}&#xA;\frac{1}{h^2} \left( -u_{i-1} + 2 u_{i} - u_{i+1} \right) = 1.&#xA;\end{equation}&#xA;This applies only to a structured grid (with constant space between the points)!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you want to do the calculation with finite differences, you don't need three points as you dont advance in time. Keep in mind that this is a boundary value problem! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need to do is to write down the finite difference equation at every point of your field from 0 to 1.&#xA;\begin{array}&#xA;-\frac{1}{h^2} u_0 +\frac{2}{h^2} u_{1} - \frac{1}{h^2} u_{2} = 1 \\&#xA;... \\&#xA;-\frac{1}{h^2} u_{n-2} +\frac{2}{h^2} u_{n-1} - \frac{1}{h^2} u_{n} = 1&#xA;\end{array}&#xA;$u_0$ und $u_{n}$ are your boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can create a system of equations like $Au=r$ with $u$ being your solution vector and $r$ a vector containing only ones. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have MATLAB, so I cant help you with the code, but here's an example with Python.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    import numpy as np&#xA;    import numpy.linalg as LA&#xA;&#xA;    n = 21 # number of points&#xA;    x = np.linspace(0,1,n) # positions of the nodes between 0 and 1&#xA;    h = (x[-1]-x[0])/(n-1) # distance between two points&#xA;    A = np.zeros((n-2,n-2)) &#xA;&#xA;    for iter in range(n-2):&#xA;        if iter == 0:&#xA;            A[iter,iter] = 2/h**2&#xA;            A[iter,iter+1] = -1/h**2&#xA;        elif iter == n-3:&#xA;            A[iter,iter] = 2/h**2&#xA;            A[iter,iter-1] = -1/h**2&#xA;        else:&#xA;            A[iter,iter] = 2/h**2&#xA;            A[iter,iter-1] = -1/h**2&#xA;            A[iter,iter+1] = -1/h**2&#xA;&#xA;    rhs = np.ones((n-2,1))&#xA;    A_inv = LA.inv(A)&#xA;    u = np.dot(A_inv,rhs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Before you use a non-uniform grid, you might try it with a uniform-grid. Don't make it to hard in the beginning.&lt;/p&gt;&#xA;" OwnerUserId="21418" LastActivityDate="2016-11-01T21:44:41.550" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25410" PostTypeId="2" ParentId="25377" CreationDate="2016-11-01T23:59:41.153" Score="1" Body="&lt;p&gt;I don't really have a definitive answer to your question.&#xA;However, after some experimentation, I believe your conclusion&#xA;regarding the condition number of the stiffness matrix is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also believe you can improve the accuracy with an iterative solver.&#xA;A very good preconditioner is to use the factors from the direct solution.&#xA;In Octave/MATLAB, you can use the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lt = chol(K);&#xA;u = pcg(K, rhs, 1e-16, 10, lt', lt);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is basically a form of iterative refinement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't the final word on this question but it points in a direction&#xA;for more experiments.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-11-01T23:59:41.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25412" PostTypeId="2" ParentId="25399" CreationDate="2016-11-02T00:22:04.837" Score="1" Body="&lt;p&gt;Your question is phrased in a very general way and so it is hard to recommend a specific way to go. I will give a recommendation that might help you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is well known that every matrix $A\in\mathbf{R}^{n\times n}$ can be factored using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular_value_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Singular Value Decomposition&lt;/a&gt; so that $A=U\Sigma V^T$ where $U$ and $V$ are orthogonal matrices and $\Sigma$ is diagonal. There are standard computational methods for the SVD. Since $U$ and $V$ are orthogonal, the equation $Ax=b$ can be written $x = V\Sigma^{-1}U^T b$. So far, nothing is easier since all we have done is find another way of finding the inverse of $A$. The goal is to approximate $V\Sigma^{-1}U^T$ in an efficient way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Luckily there are methods to compute the truncated SVD where only the first $k$ columns of $U$ and rows of $V$ are computed. It can be shown that, at least in some sense, that is the best approximation that you can find. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might search for 'model reduction' or 'model reduction svd' to get more of a view of how the SVD has been used by others to construct a reduced model. &lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2016-11-02T00:22:04.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25413" PostTypeId="2" ParentId="2535" CreationDate="2016-11-02T01:44:43.183" Score="0" Body="&lt;p&gt;It's an old question, but hey, 25+ years ago somebody could already solve a 1.1M constraint, 2.6M variable problem in 3h on a PC cluster. &lt;a href=&quot;http://www.maths.ed.ac.uk/~gondzio/CV/finance.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.maths.ed.ac.uk/~gondzio/CV/finance.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to see the generating equations, perhaps it's smart to do some serious decomposition before you throw this problem to the algorithms, and I would like to say as a practitioner that maybe it is smart to chew on it before feeding the hardware. Also sounds like the size that would induce to numerical errors in the formulation given limited computer memory and precision.&lt;/p&gt;&#xA;" OwnerUserId="21302" LastActivityDate="2016-11-02T01:44:43.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25414" PostTypeId="2" ParentId="25401" CreationDate="2016-11-02T09:16:06.813" Score="2" Body="&lt;p&gt;I would not bother too much. Chorin's method is known to be a consistent scheme, which means, that for smaller time-steps it will approximate the actual solution. Checkout Gresho's/Sani's book &lt;em&gt;Incompressible Flow and the Finite Element Method. Vol. 2: Isothermal Laminar Flow&lt;/em&gt; there is the convergence theory and also Chorin's method with a forcing term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, just put your force into $(1)$ or $(2)$ -- there is no difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last step $(3)$ is a projection step that projects your tentative velocity $U^{**}$ to a divergence-free corrected velocity. It will remove certain parts of $f$, namely those that entered the &lt;em&gt;gradient&lt;/em&gt;-part of $U^{**}$. In fact, if $f=\nabla \rho$, where $\rho$ is suitable scalar function, then (in theory) $f$ should never appear in the velocity and then, at best, also not in $U^{**}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to address your last points... It's not about &lt;em&gt;too much&lt;/em&gt; force, it is about the direction of the force (whether it is a gradient for example). And Chorin's method will project out the gradient parts up to the consistency error. &lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-11-02T09:16:06.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25415" PostTypeId="2" ParentId="24892" CreationDate="2016-11-02T15:43:52.827" Score="1" Body="&lt;p&gt;There's two recent papers you should check:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GPU Performance Modeling and Optimization:&#xA;&lt;a href=&quot;https://pure.tue.nl/ws/files/39759895/20161018_Li.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pure.tue.nl/ws/files/39759895/20161018_Li.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accelerating BLAS on Custom Architecture thru Algo/Arch codesign&#xA;&lt;a href=&quot;https://arxiv.org/abs/1610.06385&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1610.06385&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22205" LastActivityDate="2016-11-02T15:43:52.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25416" PostTypeId="2" ParentId="25390" CreationDate="2016-11-02T21:15:02.923" Score="1" Body="&lt;p&gt;It is &#xA;possible to use &lt;a href=&quot;http://simul.iro.umontreal.ca/testu01/tu01.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the TestU01&#xA;suite&lt;/a&gt; of PRNG tests&#xA;in order to find out &lt;em&gt;which&lt;/em&gt; of those tests &lt;code&gt;rand&lt;/code&gt; fails. (See &lt;a href=&quot;http://www.iro.umontreal.ca/~lecuyer/myftp/papers/testu01.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TestU01: A C Library for Empirical Testing of&#xA;Random Number Generators&lt;/a&gt; for an overview of the test suite.) That's easier than coming up with Monte Carlo simulations of one's own. In a way, it's also a question of software composability (and software correctness): given a PRNG that appears to work okay on small, simple tests, how do you know its pathological behaviours won't be triggered by a larger program?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;TestU01.h&quot;&#xA;&#xA;int main() {&#xA;  // Same as rand() on my machine&#xA;  unif01_Gen* gen = ulcg_CreateLCG(2147483647, 16807, 0, 12345);&#xA;&#xA;  bbattery_SmallCrush(gen);&#xA;  bbattery_Crush(gen);&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the &lt;em&gt;SmallCrush&lt;/em&gt; suite, there are 3 tests failing out of 15 (see&#xA;&lt;em&gt;guidelongtestu01.pdf&lt;/em&gt; in TestU01 for long descriptions and all the&#xA;references; these are 15 statistics from 10 tests).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;BirthdaySpacings&lt;/em&gt;: bin $n$ $t$-dimensional vectors into $d^t$ bins,&#xA;generating $d^t$ bin counts, $I_1,\ldots$, then the number of&#xA;collisions among $\{I_{j+1}-I_j\}$ follows approximately a known&#xA;distribution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Collision&lt;/em&gt;: generate $n$ $t$-dimensional vectors in $[0,1)^t$, bins&#xA;them into $d^t$ equal hypercubes, and count the number of collisions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;MaxOft&lt;/em&gt;: generate $n$ groups of $t$ values in $[0,1)$, compute the&#xA;maximum $X$ in each group, and compare the distribution of the $n$&#xA;maxima with the theoretical distribution $\mathbb{P}(X&amp;lt;x) = x^t$;&#xA;the values are $n=2\times 10^6$, $t=6$. The comparison is done&#xA;via a chi-square test and an Anderson-Darling test: the $\chi^2$ test fails with p-value $&amp;lt;10^{-300}$, while the AD test passes (this AD test fails in some of the larger similar tests in Crush).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Assuming these are all &quot;typical&quot; Monte Carlo simulations (though they might&#xA;not be like the problems you had in mind), the conclusion is that&#xA;&lt;code&gt;rand&lt;/code&gt; fails some unknown subset of them. I don't know why it is&#xA;specifically that subset, though, so it's impossible for me to say&#xA;whether it will work on your own problem or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;MaxOft&lt;/em&gt; seems particularly suspicious, given how straightforward the description is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among the tests in the &lt;em&gt;Crush&lt;/em&gt; suite, &lt;code&gt;rand&lt;/code&gt; fails 51 out of 140 (140 statistics in 96 tests). Some of the failed tests (like &lt;em&gt;Fourier3&lt;/em&gt;) are done on bit strings, so perhaps it's possible they wouldn't be relevant to you. Another curious test that fails is &lt;em&gt;GCD&lt;/em&gt;, which tests the distribution of the GCD of two random integers. (Again, I don't know why this particular test fails or whether your simulation will suffer from this.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;P.S.&lt;/em&gt;: Yet another thing to note is that &lt;code&gt;rand()&lt;/code&gt; is actually slower than some PRNGs that successfully pass all the &lt;em&gt;SmallCrush&lt;/em&gt;, &lt;em&gt;Crush&lt;/em&gt;, &lt;em&gt;BigCrush&lt;/em&gt; tests, such as &lt;em&gt;MRG32k3a&lt;/em&gt; (see the L'Ecuyer &amp;amp; Simard paper above).&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-11-02T22:26:48.497" LastActivityDate="2016-11-02T22:26:48.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25418" PostTypeId="2" ParentId="23852" CreationDate="2016-11-03T12:03:06.330" Score="1" Body="&lt;p&gt;From my experience with Navier-Stokes equations, one can do very well without fully implicit schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you just want a fast numerical scheme for the solution of the time evolution, take a look at IMEX (implicit-explicit) schemes; see e.g. this paper by Ascher, Ruuth, Spiteri &lt;a href=&quot;http://www.cs.ubc.ca/~ascher/papers/ars.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Implicit-Explicit Runge-Kutta Methods for Time-Dependent Partial Differential Equations&lt;/em&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may even try to just use an explicit high-order time integration scheme with step size control (like Matlab's &lt;code&gt;ODE45&lt;/code&gt;). However, you may run into trouble because of the stiffness of your system, which comes from the diffusive part. Luckily, the diffusive part is linear so that it can be treated implicitly (which is the idea of the IMEX schemes).&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-11-03T12:03:06.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25419" PostTypeId="1" AcceptedAnswerId="29258" CreationDate="2016-11-03T13:42:31.320" Score="2" ViewCount="95" Body="&lt;p&gt;Is there a single noun that indicates unambiguously (to a computational scientist, at least) &quot;the ordering (&lt;a href=&quot;https://en.wikipedia.org/wiki/Row-major_order&quot; rel=&quot;nofollow noreferrer&quot;&gt;row-major or column major&lt;/a&gt;) used in an array&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, the term &lt;a href=&quot;https://en.wikipedia.org/wiki/Endianness&quot; rel=&quot;nofollow noreferrer&quot;&gt;endianness&lt;/a&gt; (or sometimes even the odd-sounding &lt;a href=&quot;http://www.catb.org/jargon/html/B/byte-sex.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;byte sex&lt;/a&gt;) refers to whether a word is stored in little-endian or big-endian order, and one single English word is enough to convey this meaning. I would not consider &quot;the ordering of an array&quot; clear enough by itself without the parenthetical, by contrast.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2018-04-08T01:12:23.200" Title="Is there a single noun to indicate &quot;the ordering (row-major or column major) used in an array&quot;?" Tags="|terminology|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25420" PostTypeId="1" CreationDate="2016-11-03T14:01:50.980" Score="0" ViewCount="178" Body="&lt;p&gt;I would like to solve in Matlab the following optimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{maximize} &amp;amp; \bigg\| \displaystyle\sum_{l=1}^{2}\alpha_l \int_{\tau_{m+l-1}}^{\alpha_1\tau_m+\alpha_2\tau_{m+1}}e^{J(h-s)} \mathrm{d}s \bigg\|\\ \text{subject to} &amp;amp; \alpha_1+\alpha_2=1\\ &amp;amp; \alpha_1, \alpha_2 \geq 0\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to solve this problem with Yalmip in Matlab? (Maybe solving the integral first)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the tips!&lt;/p&gt;&#xA;" OwnerUserId="6703" LastEditorUserId="6703" LastEditDate="2016-11-03T14:12:34.963" LastActivityDate="2016-11-04T19:55:32.507" Title="Optimization with Yalmip" Tags="|optimization|matlab|linear-solver|quadratic-programming|" AnswerCount="1" CommentCount="0" ClosedDate="2016-11-06T01:58:25.600" ContentLicense="CC BY-SA 3.0" />
  <row Id="25421" PostTypeId="1" AcceptedAnswerId="25425" CreationDate="2016-11-03T14:32:36.087" Score="0" ViewCount="91" Body="&lt;p&gt;Given a distributed Epetra CrsMatrix $A$, along with a a list of rows, a list of columns and a processor ID, I would like to extract a sub-matrix $A_{sub}$, all the entries of which should be stored on the specified processor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this may be done with Import/Export object with suitably defined maps but I cannot seem to find an example.  &lt;/p&gt;&#xA;" OwnerUserId="16379" LastEditorUserId="16379" LastEditDate="2016-11-03T14:38:32.557" LastActivityDate="2016-11-03T19:14:21.530" Title="Trilinos: Extracting a submatrix of an Epetra CrsMatrix" Tags="|trilinos|" AnswerCount="1" CommentCount="4" ClosedDate="2016-11-04T12:37:32.070" ContentLicense="CC BY-SA 3.0" />
  <row Id="25423" PostTypeId="1" CreationDate="2016-11-03T16:52:03.993" Score="0" ViewCount="325" Body="&lt;p&gt;In the following staggered grid setting,&lt;a href=&quot;https://i.stack.imgur.com/z1Ca5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z1Ca5.png&quot; alt=&quot;Staggered grid&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve diffusion equation as a linear system.&#xA;$$\frac{\partial u}{\partial t} = \frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For discretization, use central difference for space and backward euler for time, you can obtain&#xA;$$\frac{u^{n+1}-u^{n}}{\Delta t} = A u^{n+1}\\&#xA;(I-A\Delta t)u^{n+1} = u^{n}\\&#xA;\tilde{A}u^{n+1} = u^{n}$$ &#xA;Where $A$ is the differential operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, solving a linear system give the velocity $u$ at the next time step $n+1$. However, to incorporate the boundary condition, I have to modify the&#xA;matrix $A$ and the right hand side. the boundary conditions are given as Dirichlet condition at the boundaries, $u_N$, $u_S$, $u_W$ and $u_E$, respectively. Since the points $u_1$~$u_4$ and $u_{17}$~$u_{20}$ are located outside the domain, the velocity at those points are given as &#xA;$$\frac{u_i+u_{i+4}}{2} = u_N \quad \text{and} \frac{u_i+u_{i+4}}{2} = u_S $$&#xA;Then the linear system is the following,&#xA;$$&#xA;\begin{pmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots  &amp;amp; \cdots &amp;amp; 0\\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots  &amp;amp; \cdots&amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots  &amp;amp; \cdots&amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots  &amp;amp; \cdots&amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots  &amp;amp; \cdots&amp;amp; 0\\&#xA;0 &amp;amp; \frac{-1\Delta t}{dy^2} &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; 1+\frac{2\Delta t}{dx^2}+\frac{2\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{-1\Delta t}{dy^2} &amp;amp; \cdots  &amp;amp; \cdots &amp;amp; 0\\&#xA; &amp;amp; &amp;amp; &amp;amp; &amp;amp; \vdots &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp;\\&#xA;0 &amp;amp; \cdots &amp;amp; \frac{-1\Delta t}{dy^2} &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; 1+\frac{2\Delta t}{dx^2}+\frac{2\Delta t}{dy^2} &amp;amp; \frac{-1 \Delta t}{dx^2} &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{-1\Delta t}{dy^2} &amp;amp; \cdots &amp;amp; 0\\&#xA; &amp;amp; &amp;amp; &amp;amp; &amp;amp; \vdots &amp;amp; &amp;amp; &amp;amp; &amp;amp; &amp;amp;\\&#xA;0 &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \cdots &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;u_1^{n+1}\\&#xA;u_2^{n+1}\\&#xA;u_3^{n+1}\\&#xA;u_4^{n+1}\\&#xA;u_5^{n+1}\\&#xA;u_6^{n+1}\\&#xA;\vdots \\&#xA;u^{n+1}\\&#xA;\vdots \\&#xA;u_{20}^{n+1}\\&#xA;\end{pmatrix}&#xA;=&#xA;\begin{pmatrix}&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;u_{W}\\&#xA;u_6^{n}\\&#xA;\vdots \\&#xA;u_i^{n}\\&#xA;\vdots \\&#xA;2u_{N}\\&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to write this system as $$\tilde{A}u^{n+1}=u^n+f$$ so that I can modify the right hand side by adding something to the velocity vector at the previous time step. Is it possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand it's possible if the system is for only the internal points&#xA;$[u_6,u_7,u_{10},u_{11},u_{14},u_{15}]$,&#xA;$$&#xA;\begin{pmatrix}&#xA;1+\frac{2\Delta t}{dx^2}+\frac{3\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\&#xA;\frac{-1\Delta t}{dx^2} &amp;amp; 1+\frac{2\Delta t}{dx^2}+\frac{3\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\&#xA;\frac{-1\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; 1+\frac{2\Delta t}{dx^2}+\frac{2\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; \frac{-1\Delta t}{dy^2} &amp;amp; 0\\&#xA;0 &amp;amp; \frac{-1\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; 1+\frac{2\Delta t}{dx^2}+\frac{2\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; \frac{-1\Delta t}{dy^2}\\&#xA;0 &amp;amp; 0 &amp;amp; \frac{-1\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp; 1+\frac{2\Delta t}{dx^2}+\frac{3\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2}\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{-1\Delta t}{dy^2} &amp;amp; \frac{-1\Delta t}{dx^2} &amp;amp;  1+\frac{2\Delta t}{dx^2}+\frac{3\Delta t}{dy^2}\\&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;u_6^{n+1}\\&#xA;u_7^{n+1}\\&#xA;u_{10}^{n+1}\\&#xA;u_{11}^{n+1}\\&#xA;u_{14}^{n+1}\\&#xA;u_{15}^{n+1}\\&#xA;\end{pmatrix}&#xA;=&#xA;\begin{pmatrix}&#xA;u_6^{n}\\&#xA;u_7^{n}\\&#xA;u_{10}^{n}\\&#xA;u_{11}^{n}\\&#xA;u_{14}^{n}\\&#xA;u_{15}^{n}\\&#xA;\end{pmatrix}&#xA;+&#xA;\begin{pmatrix}&#xA;2\frac{\Delta t}{dy^2}u_S+\frac{\Delta t}{dx^2}u_W\\&#xA;2\frac{\Delta t}{dy^2}u_S+\frac{\Delta t}{dx^2}u_E\\&#xA;0\\&#xA;0\\&#xA;2\frac{\Delta t}{dy^2}u_{N}+\frac{\Delta t}{dx^2}u_W\\&#xA;2\frac{\Delta t}{dy^2}u_{N}+\frac{\Delta t}{dx^2}u_E\\&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="14387" LastEditorUserId="14387" LastEditDate="2016-11-03T18:48:20.733" LastActivityDate="2018-03-29T19:01:46.153" Title="The system matrix and the right hand side for diffusion equation with staggered grid" Tags="|finite-difference|boundary-conditions|diffusion|linear-system|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25425" PostTypeId="2" ParentId="25421" CreationDate="2016-11-03T19:14:21.530" Score="0" Body="&lt;p&gt;After some trial and error I have found a solution. Here is a minimal working example in Python.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from PyTrilinos import Epetra, Galeri&#xA;comm = Epetra.PyComm()&#xA;&#xA;my_id = comm.MyPID()&#xA;&#xA;if my_id ==0:&#xA;    my_range = [0, 1, 5 , 7]&#xA;else:&#xA;    my_range = []&#xA;&#xA;subset_map = Epetra.Map(-1, my_range , 0, comm)&#xA;&#xA;nx, ny = 4&#xA;&#xA;GaleriList = {&quot;n&quot;  : nx * ny,&#xA;              &quot;nx&quot; : nx,&#xA;              &quot;ny&quot; : ny&#xA;              }&#xA;Map    = Galeri.CreateMap(&quot;Linear&quot;, comm, GaleriList)&#xA;Matrix = Galeri.CreateCrsMatrix(&quot;Recirc2D&quot;, Map, GaleriList)&#xA;&#xA;importer = Epetra.Import(subset_map, Matrix.Map())&#xA;&#xA;Submatrix = Epetra.CrsMatrix(Epetra.Copy, subset_map,subset_map, 5)&#xA;Submatrix.Import(Matrix, importer, Epetra.Insert)&#xA;Submatrix.FillComplete()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16379" LastActivityDate="2016-11-03T19:14:21.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25427" PostTypeId="1" CreationDate="2016-11-03T23:16:50.813" Score="1" ViewCount="67" Body="&lt;p&gt;I have an application in sensitivity analysis of complex system models with moderately nonlinear interactions between arguments&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Arguments potentially dozens or hundreds in number&lt;/li&gt;&#xA;&lt;li&gt;Arguments mostly real-valued but some categorical with a handful of levels (&lt;em&gt;unordered&lt;/em&gt; settings) per variable&lt;/li&gt;&#xA;&lt;li&gt;The outputs of interest are real-valued and highly interdependent, and respond well to the usual dimensionality reduction techniques&lt;/li&gt;&#xA;&lt;li&gt;Assume for present purposes that the model output is a single real number.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have the compute resource to evaluate the model several hundred or a few thousand times, but not millions of times, and I will typically do this on random or Sobol'-gridded points in the design space, i.e. I will have the output value at a few thousand points randomly scattered in a few dozen dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to do the usual thing of assigning importance rankings to inputs and combinations of inputs using techniques from Saltelli &lt;em&gt;et al&lt;/em&gt; -- see &lt;a href=&quot;http://www.andreasaltelli.eu/file/repository/Homma_Saltelli_RESS.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; to get a flavor. I want to estimate the &lt;em&gt;total variation&lt;/em&gt; of the &lt;em&gt;expectation&lt;/em&gt; of the model output conditional on one or a few inputs at a time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I plan to do this by fitting a low-order smoothing spline through the space of one input, or a few possibly interacting inputs, plus the model output, and calculating the total variation of the smoother by numerical integration. The interacting sets will typically be three or four variables, so sparsity in high dimensions will never be an issue. I also know how to cross-validate and regularise the splines for optimal fit, and correct for randomness in the total-variation measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I &lt;em&gt;don't&lt;/em&gt; know is how to proceed when one or more of the inputs of interest is categorical. I can see what to do for a single binary-valued input, but anything more complex defeats me. I've wiki'ed and googled this up and down without success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to summarise the question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Is there an extension of the concept of total variation to scalar real-valued functions with (possibly multiple) (more than binary) unordered categorical arguments?&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Is there an explanation digestible by a mere engineer?&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance, I hope, but I'm actively looking in multiple places and I'll update this post if I find anything useable elsewhere.&lt;/p&gt;&#xA;" OwnerUserId="22223" LastActivityDate="2016-11-03T23:16:50.813" Title="Sensitivity Analysis -- Total Variation for a function with categorical arguments?" Tags="|computational-physics|interpolation|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25429" PostTypeId="2" ParentId="25423" CreationDate="2016-11-04T09:47:34.637" Score="1" Body="&lt;p&gt;I found a way to do it. I can modify the RHS as following,&#xA;\begin{pmatrix}&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;u_{W}\\&#xA;u_6^{n}\\&#xA;\vdots \\&#xA;u_i^{n}\\&#xA;\vdots \\&#xA;2u_{N}\\&#xA;\end{pmatrix}&#xA;into &#xA;$$&#xA;\begin{pmatrix}&#xA;u_{1}\\&#xA;u_{2}\\&#xA;u_{3}\\&#xA;u_{4}\\&#xA;u_{5}\\&#xA;u_6^{n}\\&#xA;\vdots \\&#xA;u_i^{n}\\&#xA;\vdots \\&#xA;u_{20}\\&#xA;\end{pmatrix}&#xA;-&#xA;\begin{pmatrix}&#xA;u_{1}\\&#xA;u_{2}\\&#xA;u_{3}\\&#xA;u_{4}\\&#xA;u_{5}\\&#xA;0\\&#xA;\vdots \\&#xA;0\\&#xA;\vdots \\&#xA;u_{20}\\&#xA;\end{pmatrix}&#xA;+&#xA;\begin{pmatrix}&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;2u_{S}\\&#xA;u_{W}\\&#xA;0\\&#xA;\vdots \\&#xA;0\\&#xA;\vdots \\&#xA;2u_{N}\\&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="14387" LastActivityDate="2016-11-04T09:47:34.637" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25430" PostTypeId="1" CreationDate="2016-11-04T12:36:25.950" Score="0" ViewCount="335" Body="&lt;p&gt;I'm trying to perform inference over a subset of the latent variables of a hierarchical hidden Markov model I built.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've derived the relevant optimization problem, but it's a pretty nasty piece of work, I'm wondering if I have any hope of optimizing the following expression:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\max_{z}\frac{\pi_{z_1}}{(\sum_{n=1}^N(x_n-z_n)^2)^\frac{N}{2}}\prod_{i=0}^K\Big[\frac{\prod_{j=0}^K\Gamma(P_{ij}+f_{ij}(z)+1)}{\Gamma(K+2+\sum_{j=0}^Kf_{ij}(z))}\Big]\\\\\\&#xA;&amp;amp;\text{Subject to}\hspace{5mm}0\leq z_n\leq K\\\\&#xA;&amp;amp;\hspace{24.5mm}z_n\in{\bf Z}\\\\&#xA;&amp;amp;\hspace{24.5mm}x_n\in{\bf R}\backslash{\bf Z}.&#xA;\end{align}&#xA;$$$$&#xA;Where $P$ is a right stochastic matrix, $\pi$ is a non-negative vector which sums to $1$, and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{ij}(z)=\#\{(n,n+1):z_n=i,\; z_{n+1}=j,\; 1\leq n&amp;lt;N\}.$$&#xA;$$$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I can't get the global optimum, then I'm open to all types and manners of heuristics and approximations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mostly I just don't have the first clue where to begin when it comes to integer programming except to try for a linear relaxation, and in this case that approach seems completely unworkable.&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="6849" LastEditDate="2016-11-04T12:55:23.393" LastActivityDate="2019-09-01T18:02:15.353" Title="Nonlinear integer program with linear constraints" Tags="|optimization|mixed-integer-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25431" PostTypeId="1" CreationDate="2016-11-04T12:51:52.430" Score="4" ViewCount="419" Body="&lt;p&gt;Good morning&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to develop an unstructured CFD code to solve Euler equations in a finite-volume (cell-centered) context (learning purposes). I was able to build from a cgns file some basic information for the code. What I have now are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Node index and its 'x' 'y' coordinates.&lt;/li&gt;&#xA;&lt;li&gt;Element index and nodes (indexes) that build these elements.&lt;/li&gt;&#xA;&lt;li&gt;Boundary conditions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since the code is 2D every element is actually a cell and every face is actually a line forming this cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic data-structures I build right now are:&#xA;+ Node array where I put the node index and choose between the x and y coordinates.&#xA;+ Element/node connectivity (anti-clockwise manner) where (for Quad):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;connec(1,nelem) = node 1&#xA;connec(2,nelem) = node 2&#xA;connec(3,nelem) = node 3&#xA;connec(4,nelem) = node 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These data-structures were very simple to build but the next one is very obscure for me. I need a face based data-structure where each face &quot;knows&quot; the nodes that forms it the left element index and the right element index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The end result would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;faces(1,face_index) = node 1 of the face.&#xA;faces(2,face_index) = node 2 of the face.&#xA;faces(3,face_index) = left element.&#xA;faces(4,face_index) = right element.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I searched a lot but could not find a reference that explains me the algorithms to build this face data-structure. Lohner's book about Finite-elements have very nice explanations about node-based datas-tructures but not about face-based ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone has a good reference that explains these algorithms or even the algorithm itself ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you !&#xA;cheers &lt;/p&gt;&#xA;" OwnerUserId="21588" LastEditorUserId="21588" LastEditDate="2016-11-04T12:57:27.770" LastActivityDate="2016-11-04T12:57:27.770" Title="Algorithm for face based data-structure - CFD" Tags="|pde|fluid-dynamics|fortran|finite-volume|data-structures|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25432" PostTypeId="1" AcceptedAnswerId="25445" CreationDate="2016-11-04T14:51:06.837" Score="0" ViewCount="109" Body="&lt;p&gt;I'm aware that a 2nd order accurate finite-difference method using variable properties for central differencing can be written in a finite-volume type way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla \bullet (k \nabla f) = \frac{\frac{f_{i+1}-f_{i}}{\Delta x_{i+1/2}}k_{i+1/2}-\frac{f_{i}-f_{i-1}}{\Delta x_{i-1/2}} k_{i-1/2}}{\Delta x_{i}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This discretization is 2nd order accurate for interior points, but what formula is typically used near boundaries? I'm curious about 2 cases: where $f$ is located at (1) cell center and (2) cell corner locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-11-06T19:31:20.957" Title="2nd order accurate finite difference method variable material properties near boundary" Tags="|finite-element|finite-difference|finite-volume|convergence|accuracy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25433" PostTypeId="1" CreationDate="2016-11-04T15:14:25.673" Score="2" ViewCount="2101" Body="&lt;p&gt;I want to compute $\Delta f/f$ (as explained in &lt;a href=&quot;http://www.nature.com/nprot/journal/v6/n1/box/nprot.2010.169_BX1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;, reference 1), but I am not able to do integration as I have distinct values and no function to generate the continuous time based values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a $1\times N$ time series data where $N$ is the total number of neuron response for one neuron, one for each millisecond. How do I calculate $\Delta f/f$ for this single neuron? If I do $(f_{i+1} - f_{i})/f_{i}$, it gives me very small values.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Jia, Hongbo, et al. &lt;a href=&quot;http://www.nature.com/nprot/journal/v6/n1/full/nprot.2010.169.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;In vivo two-photon imaging of sensory-evoked dendritic calcium signals in cortical neurons.&quot;&lt;/a&gt; Nature protocols 6.1 (2011): 28-35. doi:10.1038/nprot.2010.169&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22232" LastEditorUserId="9667" LastEditDate="2016-11-04T19:29:43.067" LastActivityDate="2016-11-04T19:29:43.067" Title="How to compute $\Delta f/ f$ in computational neuroscience?" Tags="|computational-biology|" AnswerCount="0" CommentCount="4" ClosedDate="2016-12-04T17:19:02.290" ContentLicense="CC BY-SA 3.0" />
  <row Id="25435" PostTypeId="2" ParentId="25169" CreationDate="2016-11-04T17:51:52.197" Score="1" Body="&lt;p&gt;I don't know how complex your calculations going to be. If you are planning to use any of the highly parallelized scripts like VASP, I would prefer Xeon processors. Most of the Intel based High Performance Computing (HPC) systems in the world employs Xeon. First task is to confirm the paralleization capabilities of your DFT package.&#xA;In any case, I recommend you to contact Intel directly. Since it is not so trivial like buying a commercial system for general use, they have a separate section dedicated to handle HPC concerns ( ITP-HPC@intel.com ). I hope this helps. &lt;/p&gt;&#xA;" OwnerUserId="21775" LastActivityDate="2016-11-04T17:51:52.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25436" PostTypeId="2" ParentId="25420" CreationDate="2016-11-04T19:46:53.863" Score="1" Body="&lt;p&gt;The integral does not depend on the variable $\alpha_l$, so the objective is weird as the integral can be moved outside the sum, which then is equal to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyhow, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tau1 = 1;&#xA;tau2 = 2;&#xA;J = 3;&#xA;h = 4;&#xA;alpha = sdpvar(2,1);&#xA;from = tau1;&#xA;to = alpha(1)*tau1 + alpha(2)*tau2;&#xA;&#xA;Objective = 0;&#xA;for i = 1:2&#xA;    Objective = Objective + exp(J*h)*(-1)*[exp(to)-exp(from)]);&#xA;end&#xA;optimize([alpha &amp;gt;= 0, sum(alpha) == 1], abs(Objective))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2016-11-04T19:55:32.507" LastActivityDate="2016-11-04T19:55:32.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25437" PostTypeId="1" CreationDate="2016-11-05T19:35:32.687" Score="5" ViewCount="180" Body="&lt;p&gt;First of all, let me set up some notations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have a hierarchy of meshes $\mathcal{T}_0 \subset \mathcal{T}_1 \subset \dots \subset \mathcal{T}_n$. For simplicity I restrict myself here to $\mathcal{T}_H := \mathcal{T}_0 \subset \mathcal{T}_1 =: \mathcal{T}_h$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ps1zc.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ps1zc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbb{U}_H$ and $\mathbb{U}_h$ are spaces of FE–interpolants defined on the coarse and fine mesh, respectively. $\mathbb{R}_H$ and $\mathbb{R}_h$ are corresponding spaces of vectors of coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal{P} \; : \; \mathbb{R} \longleftrightarrow \mathbb{U}$ is a natural isomorphism. For one, for linear Lagrange shapes we have $\mathcal{P}_H \; : \; \mathbb{R}_H \ni \langle \, \xi_0, \, \xi_1, \, \dots, \, \xi_{16} \, \rangle \, \mapsto \, \sum_{i = 0}^{16} \xi_i \, S^H_i(x, y) \in \mathbb{U}_H$, where $S^H_i$ is a usual “hat–function.” $S^H_{15}$ looks like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3YE96.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3YE96.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;i. Prolongation, $\mathbb{U}_H \subset \mathbb{U}_h$&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;We want to define a prolongation operator $P \, : \, \mathbb{R}_H \longrightarrow \mathbb{R}_h$ in order to travel between meshes in MG–routines. It is easy in this case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;P \, : \, \mathbb{R}_H \stackrel{\mathcal{P}_H}{\longrightarrow} \mathbb{U}_H \subset \mathbb{U}_h \stackrel{\mathcal{P}_h^{-1}}{\longrightarrow} \mathbb{R}_h,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so for linear Lagrange elements $P$ is a sparse matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fIQ7Z.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fIQ7Z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that coarse DOFs, $i = 0, 1, \dots, 16$, are multiplied by unity (i.e. they remain the same), and fine DOFs are just means of DOFs associated with the ribs they lie on (since the shapes are linear functions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let’s fix some $\xi \in \mathbb{R}_H$. I obtain the following results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5rr4g.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5rr4g.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;$\mathcal{P}_H \, \xi$ (green) and $\mathcal{P}_h \, P \, \xi$ (blue)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can see that everything works fine.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;ii. Prolongation, $\mathbb{U}_H \not\subset \mathbb{U}_h$&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Let’s stick to Crouzeix–Raviart elements. In this case DOFs are associated with ribs, so we should enumerate them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fPgdf.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fPgdf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/KSeEV.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KSeEV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one may notice, we do not have $\mathbb{U}_H \not\subset \mathbb{U}_h$ since $u_H \in \mathbb{U}_H$ may be discontinuous at midpoints of fine ribs.&#xA;So we have to use some tricks in order to define $P$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Namely, we take a space $\mathbb{V} := \{ \, \text{functions that are linear on each fine triangle of } \mathcal{T}_h \, \}$, $\mathbb{U}_H, \mathbb{U}_h \subset \mathbb{V}$; then we define a mapping $\phi \, : \, \mathbb{V} \longrightarrow \mathbb{U}_h$ as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;(\phi \, v)(x) = \frac{1}{2} \left( v|_{t_1}(x) + v|_{t_2}(x) \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for each pair of fine triangles $t_1$ and $t_2$ that share a rib ($x$ is a midpoint of the rib). Now we are able to define our operator:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;P \, : \, \mathbb{R}_H \stackrel{\mathcal{P}_H}{\longrightarrow} \mathbb{U}_H \subset \mathbb{V} \stackrel{\phi}{\longrightarrow} \mathbb{U}_h \stackrel{\mathcal{P}_h^{-1}}{\longrightarrow} \mathbb{R}_h.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MuUAO.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MuUAO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wFRGu.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wFRGu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Pattern of sparse matrix $P$,&lt;br&gt; $\mathcal{P}_H \, \xi$ (red), and $\mathcal{P}_h \, P \, \xi$ (yellow)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can see that everything works fine here, too.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;iii. Restriction Operators: Here Comes a Problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In the corresponding literature it is common to see that the rule of thumb of defining a restriction operator is that it should be “somewhat transpose of $P$.” And by “somewhat” they usually mean “multiply by a scalar.”&lt;br&gt;&#xA;For one, in [1, p. 436] the prolongation / restriction operators are defined via interpolation / weighting (without information extraction about FE–spaces at hand). In this sense I prefer approaches of [2] and [3] more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is, &lt;strong&gt;given $P$ (from (i) and (ii)), how can one &lt;em&gt;canonically&lt;/em&gt; define the corresponding restriction $R$&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case (i), if I fix some vector $\xi \in \mathbb{R}_h$ and blindly multiply it by $P^T$, I will get the following (I used 3 mesh levels here for the sake of clarity):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1Vt5N.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1Vt5N.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BT4rJ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BT4rJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;em&gt;$\mathcal{P}_2 \, \xi$ (green), $\mathcal{P}_1 \, P_{1 \rightarrow 2}^T \, \xi$ (blue), and $\mathcal{P}_0 \, P_{0 \rightarrow 1}^T \, P_{1 \rightarrow 2}^T \, \xi$ (red)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can see that such “restrictions” are not good at all. And I doubt that multiplication by a scalar will do here. For the case (ii) I have similar problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I bet the solution is “hidden” in definitions of the following inner products:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AZQew.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AZQew.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[3, p.160]&lt;/em&gt;&#xA;&lt;hr&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lBMEp.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lBMEp.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[2, p.110]&lt;/em&gt;  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Please help me to clarify this so I will be able to implement restrictions correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Yousef Saad&lt;br&gt;&#xA;&lt;em&gt;Iterative Methods for Sparse Linear Systems (2&lt;sup&gt;nd&lt;/sup&gt; Edition)&lt;/em&gt;&lt;br&gt;&#xA;2003&lt;/li&gt;&#xA;&lt;li&gt;Maxim A. Olshanskii, Eugene E. Tyrtyshnikov&lt;br&gt;&#xA;&lt;em&gt;Iterative Methods for Linear Systems&lt;/em&gt;&lt;br&gt;&#xA;2014&lt;/li&gt;&#xA;&lt;li&gt;Susanne C. Brenner, L. Ridgway Scott&lt;br&gt;&#xA;&lt;em&gt;The Mathematical Theory of Finite Element Methods (3&lt;sup&gt;rd&lt;/sup&gt; Edition)&lt;/em&gt;&lt;br&gt;&#xA;2008&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2016-11-05T19:35:32.687" Title="Geometric Multigrid for Conform and Non–″ Elements: Restriction Operators" Tags="|finite-element|multigrid|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25439" PostTypeId="2" ParentId="25398" CreationDate="2016-11-05T21:53:16.923" Score="12" Body="&lt;p&gt;I have two extra points I would like to add to Wolfgang's answer. A formulation&#xA;of the CFL condition that I find more useful than the classic formula is this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A necessary condition for the stability of a numerical scheme is that the&#xA;    numerical domain of dependence bounds the physical domain of dependence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is exactly what good old&#xA;$$&#xA;\dfrac{\Delta t}{c \Delta x} \leq 1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;says for simple schemes. The quote I wrote above has a bit more nuance, though,&#xA;that can be used to answer the questions about implicit methods and higher order&#xA;algorithms (my two points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My points below are for the first order wave equation just to make things easy:&#xA;I hope that everyone can see how the same ideas apply to the second order&#xA;equation.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Higher order algorithms&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It is true that for most time stepping methods increasing the order imposes&#xA;stricter stability constraints but a notable exception to this is explicit Runge&#xA;Kutta methods (which include some part of the imaginary axis in their stability&#xA;regions; that is a separate stability concern). Consider the first order wave&#xA;equation $$ u_t + c u_x = 0 $$ with periodic boundary conditions and second&#xA;order finite differences $$ u_x(x_j) = \dfrac{u(x_j + \Delta x) - u(x_j - \Delta&#xA;x)}{2 \Delta x} + O(\Delta x^2) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(to make things as easy as possible) in space. One can write the semidiscrete&#xA;system as&#xA;$$&#xA;\dfrac{d\vec{U}}{dt} = A \vec{U}.&#xA;$$&#xA;If we use RK3 then we have&#xA;$$&#xA;\begin{align}&#xA;    k_1 &amp;amp;= A \vec{U}\,^n \\&#xA;    k_2 &amp;amp;= A \left(\vec{U}\,^n + \dfrac{\Delta t}{2} k_1\right) \\&#xA;    k_3 &amp;amp;= A \left(\vec{U}\,^n + \dfrac{3 \Delta t}{4} k_2\right) \\&#xA;    \vec{U}\,^{n + 1} &amp;amp;= \vec{U}\,^n + \dfrac{\Delta t}{9} (2 k_1 + 3 k_2 + 4&#xA;    k_3) \\&#xA;\end{align}&#xA;$$&#xA;so, by definition of (A):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the $j$th entry of $k_1$ uses $U_{j - 1}^n$ and $U_{j + 1}^n$&lt;/li&gt;&#xA;&lt;li&gt;the $j$th entry of $k_2$ uses $U_{j - 2}^n$, $U_{j - 1}^n$, $U_j^n$, $U_{j + 1}^n$, and $U_{j + 2}^n$&lt;/li&gt;&#xA;&lt;li&gt;the $j$th entry of $k_3$ uses $U_{j - 3}^n$ through $U_{j + 3}^n$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;since for each stage we use the same space operator on the current solution&#xA;which effectively widens the stencil. Hence doing one step with this method has&#xA;the CFL constraint&#xA;$$&#xA;\dfrac{c \Delta t}{\Delta x} \leq 3&#xA;$$&#xA;since calculating $U_j^{n + 1}$ requires data from three grid points to the left&#xA;and three grid points to the right. The same argument holds for higher order&#xA;methods: a method with more stages tends to have a wider stencil and therefore a&#xA;larger Courant number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Warning&lt;/em&gt;: due to the spatial operator we used &lt;em&gt;this scheme is not stable, even&#xA;if it satisfies the CFL constraint&lt;/em&gt;. The correct Courant number to use (which&#xA;comes from the stability region of RK3) is (roughly)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\dfrac{c \Delta t}{\Delta x} \leq 1.75.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Implicit methods&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The answer to your question about ''well-defined values for $C_{max}$ when using&#xA;implicit methods'' is that $C_{max} = \infty$ by the same plain-text version of&#xA;the CFL condition. To see why this is true consider the same ODE system with&#xA;backward Euler:&#xA;$$&#xA;\vec{U}\,^{n + 1} = \vec{U}\,^{n} + \Delta t A \vec{U}\,^{n + 1}&#xA;\Rightarrow&#xA;(I - \Delta t A) \vec{U}\,^{n + 1} = \vec{U}\,^{n}&#xA;$$&#xA;so solving for the solution at $t^{n + 1}$ requires knowledge of all values at&#xA;$t^n$, meaning that no matter what time step we use we are pulling in the&#xA;entire physical domain of dependence. Implicit methods for wave equations are an&#xA;unusual choice for a variety of (unrelated to the original question) reasons but&#xA;they do satisfy the CFL condition for any choice of $\Delta t &amp;gt; 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@Mathews24 asked some more closely related questions whose answers won't&#xA;really fit in a comment so I am appending them to my answer here. I hope that&#xA;that is an acceptable way to answer questions (please correct me if there is a&#xA;better place for this).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Why is the centered difference scheme unstable past a Courant number of $1.75$?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;As mentioned previously the CFL condition is a necessary, but not sufficient,&#xA;condition for the stability of a numerical scheme. The other usual stability&#xA;condition comes from the ODE solution method (which depends on the eigenvalues&#xA;of $A$, which in turn depends on the spatial discretization: this is why I say&#xA;that the choice of spatial discretization impacts the stability of the time&#xA;discretization). Ignoring the physical meaning of things for a moment, we say&#xA;that an ODE solver for a scalar problem&#xA;$$&#xA;y' = \lambda y + f&#xA;$$&#xA;is &lt;em&gt;stable&lt;/em&gt; if the numerical approximation of the &lt;em&gt;unforced&lt;/em&gt; problem, $y' =&#xA;\lambda y$, remains bounded for all time. I like to think of this&#xA;thermodynamically: if one considers an unforced wave in a closed system then&#xA;that wave will not grow in energy over time (but we allow it to shrink for the&#xA;purposes of our numerical scheme, since that is better than exponential&#xA;growth). An easy example of this is forward Euler: one can check that if we do&#xA;$$&#xA;w^n \approx y(t^n), \,w^{n + 1} = w^n + \lambda \Delta t w^n&#xA;\Rightarrow w^{n + 1} = (1 + \lambda \Delta t) w^n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that the scheme is stable when (for complex $\lambda$) $|\lambda \Delta t + 1|&#xA;\leq 1$. The value on the left is called the &lt;em&gt;amplification factor&lt;/em&gt;: the&#xA;solution will grow exponentially if it is higher than one. The same idea holds&#xA;for all other ODE schemes (in particular Runge-Kutta methods): one can calculate&#xA;a stability region for the three stage scheme I presented above with a&#xA;restriction on $\lambda \Delta t$ that guarantees that the solution does not&#xA;grow. The stability plot is the blue curve under ''Bogacki-Shampine-4-2-3'' at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://runge.math.smu.edu/arkode_dev/doc/guide/build/html/Butcher.html&quot; rel=&quot;noreferrer&quot;&gt;http://runge.math.smu.edu/arkode_dev/doc/guide/build/html/Butcher.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if anyone wants to see it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rule of thumb that people use (that works for all non-pathological cases;&#xA;there is a nice book by Trefethen and Reddy about how to do this more generally)&#xA;to examine stability of a system of ODEs $y' = A y$ is to do 1D stability&#xA;analysis with all of the eigenvalues of $A$. This is rigorous if the matrix is&#xA;diagonalizable and usually works even if it is not. If one looks at the&#xA;stability region of the RK3 scheme then one will see that it stays (barely!) to&#xA;the right of the imaginary axis and intersects at about $1.75$. For the matrix I&#xA;gave above the eigenvalues are purely imaginary and are bounded (in modulus) by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;|\text{eig}(A)| \leq \dfrac{c}{\Delta x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is exactly why things blow up for a Courant number greater than $1.75$.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Why are implicit methods unusual for wave equations?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Running this problem with RK3 near the stability limit is nice because the&#xA;energy in the system is very nearly conserved: things decay very slightly over&#xA;time since all of the amplification factors are very close to $1$ (the product&#xA;$\lambda_i \Delta t$ is always near the boundary of the stable region). If we&#xA;used an implicit method then we won't have to worry about the CFL constraint&#xA;(or, for the right choice of implicit method, the amplification factors) but one&#xA;can check that as a result of this the amplification factors will be&#xA;significantly less than one, leading to a lot of extra dissipation in the&#xA;scheme: this is usually not what you want with waves (usually some sort of&#xA;conservation is desired, even if it is only approximate). I recommend&#xA;calculating the amplification factors of backward Euler along the imaginary axis&#xA;to convince oneself of why this is a concern.&lt;/p&gt;&#xA;" OwnerUserId="1891" LastEditorUserId="1891" LastEditDate="2016-11-07T19:46:13.567" LastActivityDate="2016-11-07T19:46:13.567" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25441" PostTypeId="1" AcceptedAnswerId="25465" CreationDate="2016-11-06T04:24:27.977" Score="3" ViewCount="717" Body="&lt;p&gt;My goal is to create a numerical solution of 1D-solute transport (Convective-dispersion equation, CDE) to match it's analytical solution based on experimental data. The CDE can be written as (where, C= solute concentration, D= dispersion coefficient, v=velocity, t=time):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-v\frac{\partial C}{\partial x} + D\frac{\partial^ 2C}{\partial^ 2x} = \frac{\partial C}{\partial t} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fully explicit finite-difference approximation for the CDE is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C_{x,t} = C_{x,t-\Delta t} + \frac{D \Delta t}{\Delta x^2}(C|_{x+\Delta x, t-\Delta t}- 2C|_{x,t-\Delta t} + C|_{x-\Delta x,t-\Delta t}) - \frac{v\Delta t}{2 \Delta x}(C|_{x+\Delta x,t-\Delta t} - C|_{x-\Delta x, t-\Delta t}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when, the initial condition: $$C(x,0) = 0$$&#xA;Left boundary condition: $$C(0,t) = C_{0}$$&#xA;Right boundary condition at infinite distance: $$\frac{\partial C}{\partial x} (\infty, t) = 0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the analytical solution can be given by:&#xA;$$\frac{C}{C_{0}} = \frac{1}{2}\operatorname{erfc}\Bigg[\frac{Rx-vt}{2(DRt)^\frac{1}{2}}\Bigg]+ \frac{1}{2}\exp\Bigg(\frac{vx}{D}\Bigg)\operatorname{erfc}\Bigg[\frac{Rx+vt}{2(DRt)^\frac{1}{2}}\Bigg]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the analytical expression, breakthrough $(C=C_{0})$ occurs at a distance of $x=0.075$ $m$ at $0.61$ $days$ (see black line in the attcahed figure) given a $D = 0.0033$ $m^2/day$, $v =0.66$ $m/day$, a retardation, $R = 1$ (dimensionless), and a constant input concentration $C_{0} = 320$ $ppt$. The numerical solution plotted for the same distance, $x$, was simulated with the same $D$ and $v$ values with a $\Delta t = 0.005$ $days$ and $\Delta x = 0.009375$ $m$ (magenta). The parameters used for numerical solution were well within the prescribed stability limits: $\frac{v\Delta t}{2D}=0.00001 &amp;lt;&amp;lt;1$; $\frac{D\Delta t}{\Delta x^2}=0.19 &amp;lt;0.5$; $P_{e}=$$\frac{v\Delta x}{2D}=0.94 &amp;lt;=1$; and $1-\Delta t\Big(\frac{2D}{\Delta x^2}+\frac{v}{2\Delta x}\Big)=0.45&amp;gt;=0$ along with a Reynold's number, $R_{L}=15$. The numerical solution was set up in an excel spreadsheet with the same boundary conditions described above. Since it is not possible to set up the right boundary at &quot;$\infty$&quot; distance, I set it up at an arbitrary distance of $x=1.74 $ $m$, but extending this distance futhur doesn't change the outcome (e.g. early breakthrough) in any significant way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/alPwb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/alPwb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is the numerical approximation good enough? Is there a better way of implementing it especially when retardation and reaction terms needs to be incorporated?&lt;/p&gt;&#xA;" OwnerUserId="22244" LastEditorUserId="9667" LastEditDate="2016-11-06T23:49:49.307" LastActivityDate="2016-11-08T13:22:10.910" Title="Numerical approximation for a known exact solution of advection-dispersion equation" Tags="|finite-difference|advection-diffusion|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25442" PostTypeId="2" ParentId="25441" CreationDate="2016-11-06T09:20:42.457" Score="1" Body="&lt;p&gt;Before comparing your particular numerical solution with the analytical one, in fact you should compare it with another numerical solution to be sure your approximation is satisfactory. If your choice of discretization steps $\Delta t$ and $\Delta x$ fulfill some stability conditions (I will comment later), it does not mean that it shall reproduce the analytical solution well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If possible, compute another numerical solution with the space discretization step being $0.5 \Delta x$ and the time discretization steps being $0.25 \Delta t$ and compare it with the previous one. If you see no important differences, it is an indication that your numerical solution has low enough approximation error and then you can compare it with the exact solution. If you see a difference, you must continue this refinement of steps up to &quot;convergent&quot; numerical solution is obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning the stability restriction, in your case you should condider the following two ones. Firstly, if you choose the central difference for the advection, you obtain a restriction on $\Delta x$ as you should require that the (grid) Peclet number is smaller than $1$ that is in your case&#xA;$$&#xA;Pe = \frac{v \Delta x}{2 D} \le 1&#xA;$$&#xA;It seems you have $Pe=1.875$. I would guess that if you plot your numerical solution after first or second time step as a function of $x$ you might see some oscillations near the left boundary, but this might be not the case and it needs not to be critical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second restriction is on $\Delta t$ due to the explicit in time discretization, in your case it takes the form&#xA;$$&#xA;1 - \Delta t \frac{2 D}{\Delta x^2} \ge 0&#xA;$$&#xA;In fact if you fulfill the both restrictions then all your coefficients on the right hand sides of your numerical scheme (the terms before $C_{x, t-\Delta t}$ and so on) are nonnegative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning your question about better numerical approximation, it depends what do you want to solve next (as for this equation you have the analytical solution). The drawback of your scheme is your restriction on the time step due to the choice of explicit in time discretization, but if you can leave with the two above restrictions and if your research is not on numerical methods, you might use this method  that can be applied also for more general 1D equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.&#xA;This part of answer was added after some issues were cleared in comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should compare not only the breakthrough curve, it means $C$ at $x=0.075$ as a function of time, but also compare numerical and analytical solution at some fixed time, e.g. $t=0.15$, as a function of $x$. I suppose you might observe a difference due to the boundary condition at the right point that might explain behavior of your numerical breakthrough curve and that might suggest you where to place the right point of your interval (now 1.74).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a problematic issues with this boundary condition:&#xA;$$&#xA;\frac{\partial C}{\partial x}(L,t) = 0&#xA;$$&#xA;If you define it at finite distance, say $L=1.74$, then your analytical solution is not valid and for some time interval the difference might be large. Simply plot the exact solution and observe for which time interval the condition&#xA;is not fulfilled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another point is how did you implement this boundary condition for advection-dispersion equation, but this might be now not so important.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-11-07T18:40:47.873" LastActivityDate="2016-11-07T18:40:47.873" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25445" PostTypeId="2" ParentId="25432" CreationDate="2016-11-06T19:31:20.957" Score="1" Body="&lt;p&gt;This answer is on how to implement boundary conditions (B.C.) for variable material property. My experience is that they are 2nd order accurate, but I can not find now references where such statement is proved. I hope it helps anyway, if not, it might be useful for other readers. The most closest reference is the book of Patankar on Numerical heat and mass transfer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.)&#xA;Let for instance $x_{i-1/2}$ is located at boundary.&#xA;If $f_i$ is located at cell center then the flux based boundary conditions are straightforward to implement, replace in your scheme the term&#xA;$$&#xA;k_{i-1/2} \frac{f_i - f_{i-1}}{\Delta x_{i-1/2}}&#xA;$$ &#xA;by the term prescribed for $k \partial_x f$ in flux based boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have Dirichlet B.C., then more effort is necessary. Typically you extrapolate linearly the missing (outer) value $f_{i-1}$ from given value $f_{i-1/2}$ from B.C. and from $f_i$ and then plug such $f_{i-1}$ to your scheme. In fact you might use a quadratic extrapolation depending also on $f_{i+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.)&#xA;If $f_i$ is located in corner locations and if $x_i$ is located at boundary and you have Dirichlet B.C., you need not to apply the scheme here, the value $f_i$ is given by B.C. If you have flux-based B.C. then one typically use a &quot;half&quot; finite volume, so your scheme is then &#xA;$$ \frac{\frac{f_{i+1}-f_{i}}{\Delta x_{i+1/2}}k_{i+1/2}-Flux_i}{\Delta x_{i}/2}&#xA;$$&#xA;where the value $Flux_i$ is obtained from B.C.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-11-06T19:31:20.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25448" PostTypeId="1" CreationDate="2016-11-07T02:41:22.810" Score="-1" ViewCount="409" Body="&lt;p&gt;[t,y] = odeXX(odefun,tspan,y0)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a solver odeXX, and the &lt;code&gt;tspan = [0 0.0001]&lt;/code&gt;. It seems that for any ode solver in MATLAB, they integrate by breaking &lt;code&gt;tspan&lt;/code&gt; into multiple steps (some adaptive like ode45 and some non-adaptive). But I am looking for the most primitive one--just integrate from &lt;code&gt;0&lt;/code&gt; to &lt;code&gt;0.0001&lt;/code&gt; in one step. Which ode solvers allow me to do this and how to specify this?&lt;/p&gt;&#xA;" OwnerUserId="20759" LastActivityDate="2016-11-07T05:10:49.610" Title="Which ODE solver in Matlab allows me to advance in just one timestep only" Tags="|matlab|ode|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25449" PostTypeId="1" AcceptedAnswerId="25456" CreationDate="2016-11-07T02:44:59.453" Score="0" ViewCount="274" Body="&lt;p&gt;I have a PDE that I solve in &lt;code&gt;MATLAB&lt;/code&gt; over a 2D domain using the &quot;recommended workflow&quot;, i.e. the functions &lt;code&gt;descg&lt;/code&gt;, &lt;code&gt;createpde&lt;/code&gt; and so on. The solver (&lt;code&gt;solvepde&lt;/code&gt;) works and return a solution which looks reasonable. So far so good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to take the result and calculate various integrals on it over the whole domain. Mainly, I would like to calculate &#xA;$$\int_\Omega \Big|\vec\nabla u\Big|^2 d^2x$$&#xA;where $\Omega$ is the 2D PDE domain and $u$ is the solution of the PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best way to achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;---- &lt;strong&gt;&lt;em&gt;More details&lt;/em&gt;&lt;/strong&gt; ----&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it makes any difference, the equation is &#xA;$$\nabla^2 u -a u=0$$&#xA;with Neumann boundary conditions. $a$ is some small number that I put in just to make the solution unique (since all my boundary conditions are Neumann the solution is defined only up to an additive constant. If you have a better way to solve the non-uniqueness then I'll be glad to hear about it too). The domain is a circle with two circular holes. Here's the code that generates the geometry and the solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gd=[1 0 0 1; 1 .8 0 .1; 1 -.4 0 .1]';&#xA;ns=char('bound','p1','p2')';&#xA;sf='(bound-p1)-p2';&#xA;[dl, bt]=decsg(gd,sf,ns);&#xA;model=createpde();&#xA;geometryFromEdges(model,dl);&#xA;applyBoundaryCondition(model,'neumann','Edge',1:12,'q',0,'g',1);&#xA;specifyCoefficients(model,'m',0,'d',0,'a',1e-6,'f',0,'c',1);&#xA;generateMesh(model,'Hmax',.06);&#xA;u=model.solvepde;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="753" LastActivityDate="2016-11-07T17:52:56.640" Title="Integrate result of finite element calculation in MATLAB" Tags="|finite-element|matlab|interpolation|integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25450" PostTypeId="1" AcceptedAnswerId="25505" CreationDate="2016-11-07T03:40:11.930" Score="1" ViewCount="527" Body="&lt;p&gt;I am using CVX to solve an optimization problem. One of my constraints in the problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M \succeq \eta {\eta}^T$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $M$ is a square matrix and $\eta$ is a column vector (both $M$ and $\eta$ are variables). CVX issues the error&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Only scalar quadratic forms can be specified in CVX. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think the error results from the quadratic form of the constraint. I am wondering whether I can convert the constraint into another equivalent form?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Update) Here is the optimization problem I am trying to solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;\label{svm_new_obj_semi}&#xA;&amp;amp; {\min}_{\boldsymbol{\eta}, M, \boldsymbol{\nu}, \boldsymbol{w}, \delta} \,\,\delta \\&#xA; \text{s.t.} &amp;amp; \quad   \boldsymbol{\nu} \succeq 0  \nonumber \\&#xA; &amp;amp; \quad  \boldsymbol{w} \succeq 0 \nonumber \\&#xA; &amp;amp; \quad 0 \preceq \boldsymbol{\eta} \preceq 1,  \nonumber \\&#xA; &amp;amp; \quad M \succeq  \boldsymbol{\eta} {\boldsymbol{\eta}}^T \nonumber \\&#xA; &amp;amp; \quad&#xA; \begin{bmatrix}&#xA;G\circ M &amp;amp; \boldsymbol{\eta} + \boldsymbol{\nu} - \boldsymbol{w} \\&#xA;{(\boldsymbol{\eta} + \boldsymbol{\nu} - \boldsymbol{w}   )}^T &amp;amp; \frac{2}{\beta}( \delta - {\boldsymbol{w}}^T\boldsymbol{e} + {\boldsymbol{\eta}}^T \boldsymbol{e} )&#xA; \end{bmatrix} \succeq 0&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to use: $0 \preceq M \preceq \boldsymbol{A}$, $diag(M) \preceq \boldsymbol{\eta}$, where $\boldsymbol{A}$ is a all ones matrix, to approximate $M \succeq  \boldsymbol{\eta} {\boldsymbol{\eta}}^T$. Is this reasonable?&lt;/p&gt;&#xA;" OwnerUserId="22267" LastEditorUserId="22267" LastEditDate="2016-11-08T22:55:49.283" LastActivityDate="2016-11-11T08:47:25.543" Title="How to deal with quadratic constrain in semidefinite programming" Tags="|convex-optimization|semidefinite-programming|cvx|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25451" PostTypeId="2" ParentId="25448" CreationDate="2016-11-07T05:10:49.610" Score="2" Body="&lt;p&gt;Are you asking for a method that solves $x'(t)=f(t,x(t))$, $x(0)=x_{0}$ with only one evaluation of $f$ or are you willing to evaluate $f$ at multiple points?  How accurate a solution do you need?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although MATLAB's built-in solvers with adaptive step sizes won't do this for you, it's trivial to implement your own method.  e.g. Euler's method is simply $x_{1}=x_{0}+hf(t_{0},x_{0})$.  Implementing RK4 isn't much harder.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-11-07T05:10:49.610" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25452" PostTypeId="2" ParentId="25449" CreationDate="2016-11-07T07:34:20.400" Score="1" Body="&lt;p&gt;I am sure that &lt;code&gt;matlab&lt;/code&gt; either has a function in this toolbox, to compute this term (it is part of the $H^1$ norm) or a function that will give you the &lt;em&gt;stiffness matrix&lt;/em&gt; &lt;code&gt;A&lt;/code&gt; of your FEM discretization. Then you compute this integral as &lt;code&gt;u'*A*u&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2016-11-07T07:34:20.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25454" PostTypeId="2" ParentId="25441" CreationDate="2016-11-07T11:58:39.740" Score="2" Body="&lt;p&gt;A couple of points:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Let me start by clearing up a persistent mistake relating to the stability condition of the FTCS method you are using. The correct stability condition, derived using the von Neumann method, is $c^2 \leq 2d \leq 1$, where $c=u\Delta t/\Delta x$ and $d=\nu\Delta t/\Delta x^2$. See, for example, B.P. Leonard, &lt;em&gt;Note on the von Neumann stability of the explicit FTCS convective diffusion equation&lt;/em&gt;, Appl. Num. Modelling, Vol. 4, pp. 401-402, 1980. The stability restrictions that must be satisfied by the time step are therefore $\Delta t\leq \frac{\Delta x^2}{2\nu}$ and $\Delta t \leq \frac{2\nu}{u^2}$. So the Peclet number does not play a role in the numerical solution of the &lt;em&gt;transient&lt;/em&gt; convection-diffusion problem. &lt;/li&gt;&#xA;&lt;li&gt;Now regarding the discrepancy in your results. I suspect that your grid spacing and time step might not be fine enough. The reason is that you have steep gradients in your solute concentration. Have you done a systematic refinement study in both space and time? If you have not done it such a study, your result does not mean anything. (Note that you can even calculate the order of accuracy since you have the exact solution.) I assume that the position of the right boundary does not influence the results because you mention having checked this.&lt;/li&gt;&#xA;&lt;li&gt;There's something I don't understand about your IC and BC. You say that the IC is $C=0$ everywhere. Then you have a left BC of $C_0=320$, which does not look to be consistent with your computed solution, which is 0 near the left boundary. Did you mean to say that the right BC is $C_0=320$? If so, how did you implement the left BC?  &lt;/li&gt;&#xA;&lt;li&gt;You ask whether the numerical approximation is good enough. I think the answer is &quot;it depends&quot;. It depends on whether the result is good enough for &lt;em&gt;your&lt;/em&gt; purposes. Why did you do the simulation? What quantity are you trying to predict? How accurate does that prediction have to be? &lt;/li&gt;&#xA;&lt;li&gt;You also ask whether there are better methods into which you can implement additional physical mechanisms (at least this is how I interpret your question). Yes, there are (higher order of accuracy in space (e.g., 4) and time (e.g., 2)), but implementing them requires more effort, in particular near boundaries. Depending on how many simulations you need to do and how accurate they must be, it may or may not be worth the effort.   &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2016-11-07T11:58:39.740" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25456" PostTypeId="2" ParentId="25449" CreationDate="2016-11-07T17:52:56.640" Score="1" Body="&lt;p&gt;If the domain $\Omega$ is nice, the solution to the pure Neumann problem is defined up to an additive constant. And any conjugated gradient method for solving $Ax=b$ with such singular A will converge if you make the righthand side orthogonal to the kernel which is the constant vector (i.e., subtract from b the average value of its elements). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the only thing to do then is to make the solution obtained from the linear solver also orthogonal to constant vector.&#xA;For me this way seems to bit a little bit simpler than adding small perturbation to the problem via $au$.&lt;/p&gt;&#xA;" OwnerUserId="21958" LastActivityDate="2016-11-07T17:52:56.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25457" PostTypeId="1" AcceptedAnswerId="25767" CreationDate="2016-11-07T20:00:37.373" Score="2" ViewCount="189" Body="&lt;p&gt;Rather than&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;gesv -&gt; solve $AX = B$&lt;/li&gt;&#xA;&lt;li&gt;gemm -&gt; compute $M = BX$,&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;somehow I feel there are better ways to compute $M$ with lapack/mkl?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="123" LastEditDate="2016-12-13T10:37:20.107" LastActivityDate="2016-12-13T10:37:20.107" Title="numerically stable routines to compute $M = B A^{-1} B$" Tags="|matrix|lapack|matrix-factorization|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25458" PostTypeId="1" CreationDate="2016-11-08T03:20:51.940" Score="1" ViewCount="264" Body="&lt;p&gt;Unlike finite element (FEM) or finite difference methods (FDM), where the original PDE is transformed into a sparse linear system, spectral methods return a dense linear system. For a large system, it's not practical to use the a direct elimination method, so a natural choice is iterative methods. As far as I know, most are targeted to sparse matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how this problem treated in the scientific computing world? In the case of Chebyshev polynomials, an FFT might give some help. What about other polynomials, such as Legendre or Lobatto? Is this difficulty the reason why this method is not so widely used as FEM or FDM?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorDisplayName="user20857" LastEditDate="2016-11-08T09:28:32.393" LastActivityDate="2016-11-08T09:28:32.393" Title="How is the dense system usually dealt with in spectral method?" Tags="|linear-solver|sparse-matrix|spectral-method|dense-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25459" PostTypeId="1" CreationDate="2016-11-08T04:28:32.250" Score="3" ViewCount="166" Body="&lt;p&gt;In Hamiltonian Monte Carlo, Hamiltonian dynamics are used to generate new proposals from the current state.  I understand why these dynamics are used as opposed to random walk behavior to generate proposals, but I don't understand why Hamiltonian dynamics are used as opposed to any other dynamics.  Could someone please explain this to me?  Also, this seems to be a difficult thing to search for as well, so any pointers to references would be appreciated too.&lt;/p&gt;&#xA;" OwnerUserId="22280" LastActivityDate="2018-03-23T23:27:33.913" Title="Why are Hamiltonian dynamics used in MCMC?" Tags="|optimization|molecular-dynamics|monte-carlo|random-sampling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25460" PostTypeId="2" ParentId="25458" CreationDate="2016-11-08T04:41:23.023" Score="0" Body="&lt;p&gt;Even with spectral methods, the matrix is still sparse: if you fix the polynomial degree (even if to a large number) and refine the mesh, the number of nonzeros per row of the matrix stays constant. That's the definition of a sparse matrix. It's also independent of the kind of basis you use on each cell -- Legendre, Gauss-Lobatto, Chebyshev, all only couple locally on a cell and consequently lead to a sparse matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What best to use is a separate question. First, the usual iterative solvers that work for low-order polynomial degrees still work. Matrix-vector products just cost more because the matrix is less sparse. In return, you can efficiently implement spectral methods matrix-free, i.e., where the matrix is not actually formed and only its action is implemented when multiplying with a vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bigger question is how best to precondition the linear system. I like the experience to comment on that in sufficient detail.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-08T04:41:23.023" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25461" PostTypeId="2" ParentId="25458" CreationDate="2016-11-08T04:45:57.000" Score="2" Body="&lt;p&gt;There are a few important points to be made here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Accuracy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Spectral methods exhibit, appropriately, spectral (super-algebraic) convergence. As a result, much smaller matrix sizes are required to produce the same accuracy as a FEM or FDM. Of course, the specific size and relative advantages depend on the specific scheme chosen and stability and resolution desired. An excellent example of this is shown below (S.W. Armeld et al. JCAM, 2009), where the eigenspectra of the Orr-Sommerfeld operator is computed and accuracy compared between a standard FDM and various spectral discretizations.&#xA;Indeed this is a principal example of how spectral methods became ubiquitous in computation science, illustrated first by the hallmark paper Orszag JFM, 1971. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uhKVY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uhKVY.jpg&quot; alt=&quot;convergence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Matrix structure and solve&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you rightly point out, spectral methods yield dense and typically unstructured difference matrices. The expense of computing the eigensystem or a linear solve of such a matrix is no doubt more expensive than a symmetric, sparse system from a FDM scheme, were they both the same size. The exact expense, again, depends on the specific scheme and solution method. Methods for solving both such matrix systems are extensively studied and well documented in the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conditioning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a subtle but very important point. Spectral methods are typically &lt;em&gt;horribly&lt;/em&gt; conditioned. Specifically, polynomial methods have condition number&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa \equiv \frac{\sigma_{max}}{\sigma_{min}} = \mathcal{O}(N^{2p})$$&#xA;where $p$ is the order of the highest derivative and $N$ is the number of &#xA;collocation points. Accurately evaluating solutions to poorly conditioned matrices requires care and sometimes extended precision, making such operations more costly. This also places a limit on the accuracy you are able to obtain for a given precision (conveniently you can see this behavior in the above figure, also).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, the above expression for $\kappa$ has yielded methods designed to split operators such that you have two coupled order $p/2$ equations, as opposed to a single order $p$ equation. There are other specialty methods to finagle your problem to be less computationally horrible, though I won't document them all here.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2016-11-08T04:45:57.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25462" PostTypeId="1" CreationDate="2016-11-08T06:41:28.050" Score="1" ViewCount="264" Body="&lt;p&gt;The partial differential equation is a combination of the diffusion plus convective trans­port equations and an adsorption sink. The equation for one-dimensional solute transport model is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial C}{\partial t} = D\frac{\partial^ 2C}{\partial^ 2x}-v\frac{\partial C}{\partial x} - \frac{\rho}{\theta} \frac{\partial S}{\partial t}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, C = solute concentration, D = dispersion coefficient, v = average pore-water velocity, x = distance from the inflow position, and t = time. Assuming the adsorption process is a first order reversible reaction, the rate of mass transfer to the adsorbed phase, $\frac{\partial S}{\partial t} = \frac{k_{A}\theta C}{\rho}-k_{D}S$; where, $k_{A}$ and $k_{D}$ are the adsorption (forward) and desorption (backward) rate coefficients (unit: 1/time), $\theta$ is the soil-water content by volume, and $\rho$ is the bulk density of the soil system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fully explicit finite-difference approximation for all except for the first order reversible reaction term can be written simply as (also, tested to work fine against &lt;a href=&quot;https://scicomp.stackexchange.com/questions/25441/numerical-approximation-for-a-known-exact-solution-of-advection-dispersion-equat&quot;&gt;exact solution&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C_{x,t} = C_{x,t-\Delta t} + \frac{D \Delta t}{\Delta x^2}(C|_{x+\Delta x, t-\Delta t}- 2C|_{x,t-\Delta t} + C|_{x-\Delta x,t-\Delta t}) - \frac{v\Delta t}{2 \Delta x}(C|_{x+\Delta x,t-\Delta t} - C|_{x-\Delta x, t-\Delta t}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot seem to figure out how the above finite-difference approximation could be modified to incorporate the reaction-term defined above. &lt;strong&gt;Hint:&lt;/strong&gt; &lt;em&gt;Page#96-99 of the &lt;a href=&quot;https://books.google.com/books?id=IYThyxERqMgC&amp;amp;pg=PA94&amp;amp;lpg=PA94&amp;amp;dq=finite%20difference%20form%20of%20first%20order%20reactions%20soil%20column&amp;amp;source=bl&amp;amp;ots=M_0iOGLH_y&amp;amp;sig=-EIXYY2lfyilWddtkvx2QENhkpg&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=0ahUKEwjLh-z1ipjQAhVHxGMKHXjJAgc4ChDoAQgdMAE#v=onepage&amp;amp;q=finite%20difference%20form%20of%20first%20order%20reactions%20soil%20column&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt; does provide a solution&lt;/em&gt; but I just cannot get my head around it. I'm supplying the best known articles for the &lt;a href=&quot;https://drive.google.com/file/d/0B6GUNg-8d30vOE12SURKb0FMUE0/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;analytical solution&lt;/a&gt; and &lt;a href=&quot;https://drive.google.com/file/d/0B6GUNg-8d30vWlRDLXRJS0ZLb1U/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;numerical solution&lt;/a&gt; that I could find. Any help with reproducible example codes would be hihgly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="22244" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-12-13T21:17:54.530" Title="Finite-difference form of the reaction-term in the solute transport equation" Tags="|finite-difference|advection-diffusion|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25463" PostTypeId="1" CreationDate="2016-11-08T09:25:23.980" Score="2" ViewCount="53" Body="&lt;p&gt;I am looking for a kind of modern test set of large nonlinear problems. The only option I managed to find so far is rather dated: &lt;a href=&quot;http://folk.uib.no/ssu029/Pdf_file/Testproblems/testprobRheinboldt03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://folk.uib.no/ssu029/Pdf_file/Testproblems/testprobRheinboldt03.pdf&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="720" LastActivityDate="2016-11-08T09:25:23.980" Title="A test suite of large systems of nonlinear equations" Tags="|nonlinear-equations|reference-request|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25464" PostTypeId="1" AcceptedAnswerId="29068" CreationDate="2016-11-08T10:55:00.177" Score="1" ViewCount="439" Body="&lt;p&gt;What will be the complexity of finding Gini Index of a sorted vector of $N$ values, which is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Gini(\mathbf{x})=1-2\sum_{k=1}^N \frac{\mathbf{x}(k)}{\Vert\mathbf{x}\Vert_1}(\frac{N-k+.5}{N})$&lt;/p&gt;&#xA;" OwnerUserId="17949" LastEditorUserId="17949" LastEditDate="2016-11-08T11:30:56.987" LastActivityDate="2018-03-17T08:11:20.503" Title="Big-O Complexity of Gini Index" Tags="|complexity|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25465" PostTypeId="2" ParentId="25441" CreationDate="2016-11-08T13:22:10.910" Score="1" Body="&lt;p&gt;I solved this equation using &lt;a href=&quot;https://www.gnu.org/software/octave/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Octave&lt;/a&gt; and the &lt;a href=&quot;https://github.com/wgreene310/pde1d&quot; rel=&quot;nofollow noreferrer&quot;&gt;pde1d&lt;/a&gt; function.&#xA;pde1d uses a two-node, linear finite element for spatial discretization and&#xA;solution of the time-dependent ODE is by the implicit BDF solver, IDA,&#xA;from &lt;a href=&quot;http://computation.llnl.gov/projects/sundials&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sundials&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that the spatial discretization is basically equivalent to the FD&#xA;approach in the post. Solution of the system of ODE using IDA is done to&#xA;a relatively high accuracy, however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a mesh with 300 points ($\Delta x = .009m$) the agreement between the&#xA;numerical and analytical solutions is relatively good as seen in the figure&#xA;below. The Octave script is also shown below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wj86t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wj86t.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the mesh is refined to 500 points the numerical and analytical solutions&#xA;are almost indistinguishable on a plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;pde1d is similar to the MATLAB function pdepe so I expect that pdepe would&#xA;produce similar results using the script shown below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function cse_11_06_16&#xA;n=300;&#xA;L=2.7;&#xA;x = linspace(0,L,n);&#xA;fprintf('dx=%6.3f\n', L/n);&#xA;[dum,nb] = min(abs(x-.075));&#xA;tFinal = .6;&#xA;nt = 100;&#xA;t = linspace(0,tFinal,nt);&#xA;C0=300;&#xA;D=.0033;&#xA;v=.66;&#xA;cAnal=uAnal(x,t,D, v, C0);&#xA;if(0)&#xA;figure; plot(t, cAnal(:,nb), 'o'); grid on;&#xA;return;&#xA;end&#xA;&#xA;pdeFunc = @(x,t,u,DuDx) pde(x,t,u,DuDx, D, v);&#xA;icFunc = @(x) ic(x, C0);&#xA;bcFunc = @(xl,ul,xr,ur,t) bc(xl,ul,xr,ur,t, C0);&#xA;&#xA;m=0;&#xA;opts.abstol=1e-8;&#xA;opts.vectorized='on';&#xA;tic&#xA;u = pde1d(m, pdeFunc,icFunc,bcFunc,x,t,opts);&#xA;toc&#xA;&#xA;figure; plot(t, u(:,nb), t, cAnal(:,nb)); grid on;&#xA;xlabel('Time'); ylabel('C');&#xA;title('C at x=.075 as a function of time.');&#xA;legend('numerical', 'analytical');&#xA;&#xA;timeToPlot=.1;&#xA;[dum,ti] = min(abs(x-timeToPlot));&#xA;figure; plot(x, u(ti,:), x, cAnal(ti,:), 'o'); grid on;&#xA;xlabel('x'); ylabel('C');&#xA;title(sprintf('C as a function of x at t=%g', timeToPlot));&#xA;&#xA;end&#xA;&#xA;function [c,f,s] = pde(x,t,u,DuDx, D, v)&#xA;nx=length(x);&#xA;c = ones(1,nx);;&#xA;f = D*DuDx;&#xA;s = -v*DuDx;&#xA;end&#xA;&#xA;function u0 = ic(x,C0)&#xA;% force the IC at x=0 to be consistent with the BC&#xA;if(x==0)&#xA;u0 = C0;&#xA;else&#xA;u0=0;&#xA;end&#xA;end&#xA;&#xA;function [pl,ql,pr,qr] = bc(xl,ul,xr,ur,t,C0)&#xA;pl = ul-C0;&#xA;ql = 0;&#xA;pr = 0;&#xA;qr = 1;&#xA;end&#xA;&#xA;function c=uAnal(x,t,D, v, C0)&#xA;R=1;&#xA;d = 2*sqrt(D*R*t)';&#xA;c=C0/2*(erfc((R*x - v*t')./d) + exp(v*x/D).*erfc((R*x + v*t')./d));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(In the interest of full disclosure, I should acknowledge that I am&#xA;one of the developers of pde1d.)&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-11-08T13:22:10.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25466" PostTypeId="1" CreationDate="2016-11-08T15:35:22.887" Score="1" ViewCount="65" Body="&lt;p&gt;I have to simulate a cylinder moving in transverse direction due to an external force (no damping). The equation of motion is: $M\frac{d^2Y}{dt^2}+kY = F_Y$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My confusion is regarding $kY$, how to get displacement when the cylinder is not at $(0,0)$ but at an arbitrary position say, $x_c=0.5, y_c=0.5$. The cylinder should oscillate freely about the ($0.5,0.5$) point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to solve this equation using Euler method, what will be the $Y$ in this scenario. My version is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y = Y0 + dt * V0;&#xA;V = V0 + dt * (-K * Y0 + FY) / M;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;V0&lt;/code&gt; and &lt;code&gt;Y0&lt;/code&gt; are the values at old/previous time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xSqEk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xSqEk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18144" LastEditorUserId="18144" LastEditDate="2016-11-08T16:11:37.467" LastActivityDate="2016-11-08T16:11:37.467" Title="Equation of Motion Moving Cylinder, Displacement due to spring force" Tags="|finite-difference|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25467" PostTypeId="2" ParentId="25462" CreationDate="2016-11-08T15:43:22.920" Score="1" Body="&lt;p&gt;Typically you rewrite the first equation for $C$ to&#xA;$$\frac{\partial C}{\partial t} = D\frac{\partial^ 2C}{\partial^ 2x}-v\frac{\partial C}{\partial x} - k_A C + \frac{\rho}{\theta} k_D S$$&#xA;using the second equation for $S$ that I repeat here&#xA;$$&#xA;\frac{\partial S}{\partial t} = \frac{k_{A}\theta}{\rho} C - k_{D} S \,.&#xA;$$&#xA;If you consider the simplest (first order accurate in time) explicit numerical scheme for these two equations then using your notation it takes the form&#xA;$$&#xA;C_{x,t} = C_{x,t-\Delta t} + \frac{D \Delta t}{\Delta x^2}(C|_{x+\Delta x, t-\Delta t}- 2C|_{x,t-\Delta t} + C|_{x-\Delta x,t-\Delta t}) - \frac{v\Delta t}{2 \Delta x}(C|_{x+\Delta x,t-\Delta t} - C|_{x-\Delta x, t-\Delta t}) - \Delta t  k_A C_{x,t-\Delta t} + \Delta t  \frac{\rho}{\theta} k_D S_{x,t-\Delta t}&#xA;$$&#xA;and &#xA;$$&#xA;S_{x,t}=S_{x,t-\Delta t} + \Delta t \frac{k_{A}\theta}{\rho} C_{x,t-\Delta t} - \Delta t k_{D} S_{x,t-\Delta t} \,.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Analogous equations are studied here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;De Smedt, F., Brevis, W., &amp;amp; Debels, P. (2005). Analytical solution for solute transport resulting from instantaneous injection in streams with transient storage. Journal of Hydrology, 315, 25–39. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://users.clas.ufl.edu/jbmartin/website/Classes/Surface_Groundwater/Class%202/Smedt%20et%20al%20J%20Hydro%202005.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://users.clas.ufl.edu/jbmartin/website/Classes/Surface_Groundwater/Class%202/Smedt%20et%20al%20J%20Hydro%202005.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that analytical solution is quite complicated, it involves so called special Bessel function and infinite sums. It should be analogous to the paper you quoted having analytical solution. If you plan only to compare numerical solution to available analytical solution, you might prefer only to compare published plots of analytical solution like Figure 1 in the above paper.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2016-11-13T20:21:56.583" LastActivityDate="2016-11-13T20:21:56.583" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25468" PostTypeId="1" CreationDate="2016-11-08T17:17:33.090" Score="0" ViewCount="617" Body="&lt;p&gt;I have to solve this equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot{x}x = -\frac{3}{2}\dot{x}^2 + \frac{\dot{x}}{h} + \sin(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h$ is defined by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$h = \left(\frac{\dot{x}}{h}\right)^{1/3} + \frac{\dot{x}}{h}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to initialize $h=1$, solve the first equation, calculate a new value for h:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$h = \frac{\dot{x}}{x\ddot{x} + (3/2)\dot{x}^2 - \sin(t)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and do this &lt;em&gt;many times&lt;/em&gt; because it guarantees that h has converged.&#xA;In the following code I do not know how to express $\ddot{x}$, in fact MATLAB says that &lt;code&gt;xp&lt;/code&gt; ($\dot{x}$) is undefined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plesset.m file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function xp = Plesset(t, x)&#xA;xp = zeros(2, 1);&#xA;xp(1) = x(2);&#xA;h = 1;&#xA;h = (x(2)/h)^(1/3) + x(2)/h;&#xA;xp(2) = 1/x(1)*(-1.5*(x(2))^2 + x(2)/h+sin(t));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;run.m file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i = 1:100&#xA;    [t, x] = ode45('Plesset', [0,5], [0,0]);&#xA;    h = x(2)/(x(1)*xp(2) + 3/2*(x(2))^2 - sin(t));&#xA;end&#xA;[t, x(:,1)]&#xA;plot(t, x(:,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22288" LastEditorUserId="9667" LastEditDate="2016-11-08T18:14:25.170" LastActivityDate="2016-11-08T20:33:52.867" Title="Matlab: ode45 loop" Tags="|matlab|ode|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25469" PostTypeId="2" ParentId="25468" CreationDate="2016-11-08T17:31:22.383" Score="3" Body="&lt;p&gt;You should adopt a different approach here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Express your second order equation as a system of two first order equations in&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$ u(t)=\left[&#xA;\begin{array}{c}&#xA;x(t) \\&#xA;\dot{x}(t)&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Your system of equatrions will be &quot;implicit.&quot;  That is, you can write it as &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$f(\dot{u}(t), u(t), t)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB has the ability to work directly with implicit systems of differential equations in this form.  See the ode15i solver.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-11-08T17:31:22.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25470" PostTypeId="1" CreationDate="2016-11-08T18:27:45.937" Score="0" ViewCount="1407" Body="&lt;p&gt;How can I determine a matrix $R$ in matlab such that, given a known matrix of coefficients $A$ gives me back its row reduced echelon form? Obviously I need an algorithm/function that works also with matrices that are not fully ranked.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;RA=\text{rref}A&#xA;$$&#xA;I don't know how to handle this... please can anybody help me? Thanks a lot! :)&lt;/p&gt;&#xA;" OwnerUserId="22289" LastEditorUserId="22289" LastEditDate="2016-11-09T10:42:36.010" LastActivityDate="2016-11-10T23:34:14.827" Title="Matrix of linear transformation in MATLAB" Tags="|matlab|matrix|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25471" PostTypeId="1" CreationDate="2016-11-08T19:24:00.463" Score="2" ViewCount="784" Body="&lt;p&gt;I'm developing my own generic Runge-Kutta solver, and I'm currently implementing the adaptive step-size routine. I say generic because I want to be able to test different RK implementations by only passing the solver a Butcher tableau, such as the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array}{c|cccc}&#xA;c_{1}&amp;amp;a_{11}&amp;amp;a_{12}&amp;amp;\dots &amp;amp;a_{1s}\\&#xA;c_{2}&amp;amp;a_{21}&amp;amp;a_{22}&amp;amp;\dots &amp;amp;a_{2s}\\&#xA;\vdots &amp;amp;\vdots &amp;amp;\vdots &amp;amp;\ddots &amp;amp;\vdots&#xA;\\c_{s}&amp;amp;a_{s1}&amp;amp;a_{s2}&amp;amp;\dots &amp;amp;a_{ss}\\&#xA;\hline &#xA;&amp;amp;b_{1}&amp;amp;b_{2}&amp;amp;\dots &amp;amp;b_{s}\\&#xA;&amp;amp;b_{1}^*&amp;amp;b_{2}^*&amp;amp;\dots &amp;amp;b_{s}^*\\&#xA;\end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I'm only focused on adaptive-explicit methods. As far as I understand it, a non-adaptive method would only have only the first row of $b$, and an explicit method would be zero on the diagonal and upper triangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially what I'm doing within a single iteration of my solver is passing a timestep span [$t_0, t_1$], and initial guess of the step ($h$), and an initial value of $y_0$, which could be a vector, and the function $\frac{dy}{dt} = f(t, y)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I calculate $y(t_0 + h)$ and $y^*(t_0 + h)$ using coefficients of the Butcher tableau, calculate the element-wise error between them, and then calculate a scaling factor of the step using the maximum absolute error. The scaling factor is used to determine if $h$ is too small, a good size, or too big, and is adjusted accordingly. Currently, I'm calculating the scaling factor like this (described in &lt;a href=&quot;https://youtu.be/Vs2usBZjUO8?t=33m&quot; rel=&quot;nofollow noreferrer&quot;&gt;this youtube video&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ s = \frac{\epsilon \cdot h}{2 \cdot (t_1 - t_0) \cdot max(|y-y^*|)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $t_0$ and $t_1$ are the beginning and end of the time step of interest, and epsilon is some error parameter set previously (e.g. 1e-8).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting value of $h$ would be scaled using $s$:&#xA;$$h = sh$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If s is larger than 2, then I assume the error between $y(t_0 + h)$ and $y^*(t_0 + h)$ is small and I can continue with the next step. If s is less than 1, then I adjust h without moving forward and try the same timestep again. If $s$ is between 1 and 2, then I move on as well, but I don't adjust h because it's good enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What as not readily obvious to me at the start, but now becomes so as the $tspan$ increases ($t_1-t_0$ goes up), is that the acceptable h is kept really far down. This also seems to be greatly influenced by the RK method used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the following system takes about 2 seconds on my laptop with 100 pts between [0, $4\pi$]:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{array} {rl}&#xA;y_1'(t) &amp;amp;= y_2 \\&#xA;y_2'(t) &amp;amp;= -y_1 \\&#xA;y_1(0) &amp;amp;= 0 \\&#xA;y_2(0) &amp;amp;= 0.1 \\&#xA;\end{array}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RRHYq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RRHYq.png&quot; alt=&quot;Using 100 points&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Decreasing the number of sampling points to 10 within the same range ([0, $4\pi$]) now takes 16 seconds because I think the value of s is too small, due to the $t_1-t_0$ term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be that I'm only interested in a very few number of points (or even the final value), but this is causing the solution time to increase, even for this relatively small problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x5FDF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x5FDF.png&quot; alt=&quot;Using 10 points&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone point me to a reference that goes into more detail about this scaling ($s$) parameter? Or is there another way to solve this approach this problem?&lt;/p&gt;&#xA;" OwnerUserId="19791" LastActivityDate="2016-11-08T19:24:00.463" Title="Scaling step size in adaptive runge-kutta method" Tags="|ode|runge-kutta|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="25472" PostTypeId="2" ParentId="25468" CreationDate="2016-11-08T20:22:42.913" Score="3" Body="&lt;p&gt;Let's first take your equation for $h$, which is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$h = \left(\frac{\dot{x}}{h}\right)^{1/3} + \frac{\dot{x}}{h}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we define $u=\frac{\dot{x}}{h}$, we can transform the equation to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 0 = u^{2} + u^{4/3} - \dot{x} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can notice, this equation can be seen as close to a parabola with respect to $u$, where the difference is the $u^{4/3}$ grows a little faster than linearly. To find the approximate parabolic form of this equation, we can use the Taylor Series of $u^{4/3}$ about $u=1$, which gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u^{4/3} \approx 1 + \frac{4}{3}(u-1) + \frac{2}{9}(u-1)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substitute this quantity into the transformed equation and simplify and you'll get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 11u^2 + 8u - (9\dot{x}+1) \approx 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this equation for $u$, we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u \approx \frac{1}{11} \left(-4 \pm \sqrt{16 + 11(9\dot{x}+1)}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approximate solution for $u$ should be fairly close to the actual solution, though you could refine it using one or two Newton iterations. The recursive equation for a single Newton Iteration is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u = u - \frac{u^2 + u^{4/3} - \dot{x}}{2u + \frac{4}{3}u^{1/3}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I use the most positive root the approximation produces. In your dynamics, you could then actually use $u$ directly using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot{x} = \frac{1}{x}\left(\sin(t) + u - \frac{3}{2}\dot{x}^2\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the approximation for $u$, without using any Newton iterations, and the initial conditions $[x,\dot{x}]^{T} = [1,0]^{T}$ produces the following results using &lt;code&gt;ode45(...)&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YS2KR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YS2KR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-11-08T20:33:52.867" LastActivityDate="2016-11-08T20:33:52.867" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25476" PostTypeId="2" ParentId="25470" CreationDate="2016-11-09T09:40:07.647" Score="-2" Body="&lt;p&gt;You can solve the equation for $R$, then use the resulting term:&#xA;$$R = rref(A)*A^{-1}$$&lt;/p&gt;&#xA;" OwnerUserId="22293" LastEditorDisplayName="user20857" LastEditDate="2016-11-09T20:41:17.367" LastActivityDate="2016-11-09T20:41:17.367" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25477" PostTypeId="1" CreationDate="2016-11-09T09:53:00.697" Score="3" ViewCount="3227" Body="&lt;p&gt;I am trying to solve the Poisson equation in a rectangular domain using a finite difference scheme with a rectangular mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have happily generated the matrix system of equations Ax = b which is required to be solved, but when I try to impose Neumann boundary conditions (using ghost cells) of zero gradient I am running in to problems, I believe because the matrix is singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there are several similar threads on this topic: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14246/discrete-poisson-equation-with-pure-neumann-boundary-conditions&quot;&gt;Discrete Poisson Equation with Pure Neumann Boundary Conditions&lt;/a&gt; , &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5355/writing-the-poisson-equation-finite-difference-matrix-with-neumann-boundary-cond&quot;&gt;Writing the Poisson equation finite-difference matrix with Neumann boundary conditions&lt;/a&gt; , &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19807/solving-a-linear-equation-system-with-pure-neumann-condition&quot;&gt;Solving a linear equation system with pure Neumann condition&lt;/a&gt; but I cannot find how to fix my problem. Some people suggest that I should impose a Dirichlet condition at one point on the boundary, but when I try this I am getting a spike at this point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was using a Gauss-Seidel procedure to solve the system, and at each iteration I would re-set the solution to have an average of 0. However, as this is very slow I moved to using the Bicongugate Gradient Stabilised Method which is much quicker. However, I cannot see a way in which to do something similar in this case, so any help or suggestions would be welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One reason I think this may be the case is that the test problems I am using may be unphysical. For the case where I had b being a sinusoidal forcing with an average of zero over the domain (-2 pi^2 cos(pi x)cos(pi y) from 0-1 in x and y), setting one corner to a Dirichlet condition appeared to work fine.&#xA;However, for a point source in the centre of the domain, setting a Dirichlet condition at one corner caused a spike there. In my head/reasoning physically a point source in the centre of the domain with Neumann conditions is just going to keep 'pushing' and the solution will float off to infinity with an ever growing arbitrary constant and cannot be anchored. So there is some kind of condition that says I need zero net forcing by b on my closed domain must be zero i.e. the integral of the source term across the domain must be zero. Is this correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it helps, the application of this is solving a Poisson equation for pressure in a fluid dynamics pressure correction scheme. Here I cannot guarantee that b will have  an average of zero, though it should hopefully be close.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am happy to provide more information as necessary.&lt;/p&gt;&#xA;" OwnerUserId="22294" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2017-01-13T15:12:47.457" Title="Solving the Poisson equation with Neumann Boundary Conditions - Finite Difference, BiCGSTAB" Tags="|finite-difference|boundary-conditions|poisson|conjugate-gradient|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25478" PostTypeId="1" AcceptedAnswerId="25485" CreationDate="2016-11-09T10:43:28.470" Score="1" ViewCount="96" Body="&lt;p&gt;Using the code from &lt;a href=&quot;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&lt;/a&gt;, I simulated cavity flow with an external force with the boundary conditions on the all 4 sides are $u=v=0$ and $\frac{\partial p}{\partial n}=0$. I just added external force to u direction in the step that calculates nonlinear term&#xA;$$\text{(1)} \quad \frac{U^*-U^n}{\Delta t} = -((U^n)^2)_x - (U^nV^n)_y $$&#xA;to&#xA;$$\text{(1)} \quad \frac{U^*-U^n}{\Delta t} = -((U^n)^2)_x - (U^nV^n)_y +f $$&#xA;. So the code was changed only 2 lines, from&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;uN = x*0+1&#xA;U = U-dt*(UVy(2:end-1,:)+U2x);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;uN = x*0&#xA;U = U-dt*(UVy(2:end-1,:)+U2x+10);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;, and I got this result.&lt;a href=&quot;https://i.stack.imgur.com/LrxNo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LrxNo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused with this result. This external force is like gravity, so I thought, when applying only gravity the fluid inside a cavity does not circulate. It is like water in a glass. Why does it circulate? Is this physically correct or because of the numerical simulation method?&lt;/p&gt;&#xA;" OwnerUserId="14387" LastActivityDate="2016-11-09T23:45:37.767" Title="cavity flow with only an external force. Why does it circulate?" Tags="|navier-stokes|incompressible|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25479" PostTypeId="1" AcceptedAnswerId="25487" CreationDate="2016-11-09T14:09:05.270" Score="1" ViewCount="85" Body="&lt;p&gt;I have a set of data, $x, y$ and $ z$, each with length n:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x \rightarrow \{x_{1}...x_{n}\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y \rightarrow \{y_{1}...y_{n}\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z \rightarrow \{z_{1}...z_{n}\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y$ and $z$ are parameterised by $x$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y = y(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z = z(x)$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to find derivatives $\Large \frac{dy}{dx}$ and $\Large \frac{dz}{dx}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'second-order error' numerical scheme:&#xA;$\Large \frac{dy}{dx} = \frac{y(x+dx)-y(x-dx)}{2dx} + \mathcal{O}({dx}^{2})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;woud usually work, but in my case, my 'x' data is not uniformly spaced (it is nonlinear) i.e. '$dx$' is not constant (hence my problem turns into a first order error one). The problem applies for the derivative of z.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest a good method to find these derivatives? (for anyone interested in meteorology, $x$ are pressure coordinates in atmosphere, $y$ is the potential temperature, $z$ is the geopotential height).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One person suggested that I interpolate $x$, $y$ and $z$ onto a new $x$ whereby this new list of $x$ values are uniformly spaced. However, I also want to see how the error introduced here propagates to the end result.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to find the derivative: $\Large \frac{dy}{dz}$. However, as my $z$ are not necessarily spaced evenly between each other, my numerical scheme is not second order. Hence, what scheme should I use and what is my resulting error, preferably as a function of $x$. I guess it's a similar problem to above.&lt;/p&gt;&#xA;" OwnerUserId="18797" LastActivityDate="2016-11-10T07:53:36.170" Title="Best numerical scheme for this problem" Tags="|finite-difference|numerical-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25480" PostTypeId="1" AcceptedAnswerId="26118" CreationDate="2016-11-09T15:19:01.417" Score="2" ViewCount="128" Body="&lt;p&gt;Suppose that we have to find an optimal solution $x^*$ to an optimization problem involving some function $f$, such that $0\in\partial f(x^*)$ where $\partial$ denotes the subdifferential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $(x_n)_{n\ge 0}$ be a sequence generated by some algorithm, satisfying $\lim\limits_{n \to \infty} (x_{n+1} -x_{n}) = 0$, and that $(x_n)_{n\ge 0}$ has an accumulation point $\bar{x}$ (e.g. when the sequence is bounded).&#xA;Suppose further that this sequence has the following property:&#xA;$$x_{n+1} -x_{n} \in \partial f(x_n).$$&#xA;My question is: Under which conditions can we conclude that $0\in \partial f(\bar{x})$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for your discussions!&lt;/p&gt;&#xA;" OwnerUserId="7695" LastEditorUserId="7695" LastEditDate="2016-11-09T15:51:03.097" LastActivityDate="2017-02-07T17:43:29.017" Title="Question concerning accumulation point" Tags="|optimization|convex-optimization|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25481" PostTypeId="2" ParentId="25480" CreationDate="2016-11-09T15:34:01.090" Score="0" Body="&lt;p&gt;The condition &#xA;$$x_{n+1} -x_{n} \in \partial f(x_n)$$&#xA;does not make any sense because the units do not match. Think of $x$ having units seconds and $f$ having units meters. Then the left hand side has units seconds and the right meters per second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reasonable algorithms do not tolerate such mismatches because they are not invariant under rescaling (e.g., using km instead of m as units).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-09T15:34:01.090" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25482" PostTypeId="2" ParentId="25479" CreationDate="2016-11-09T15:39:39.957" Score="3" Body="&lt;p&gt;&lt;em&gt;For Problem 1:&lt;/em&gt; You could use the forward-looking one-sided finite difference scheme,&#xA;$$&#xA;  \frac{dy(x_k)}{dx} \approx \frac{y_{k+1}-y_k}{x_{k+1}-x_k}.&#xA;$$&#xA;Of course, you could equally well have used the backward looking scheme:&#xA;$$&#xA;  \frac{dy(x_k)}{dx} \approx \frac{y_k-y_{k-1}}{x_k-x_{k-1}}.&#xA;$$&#xA;To make it more accurate, you can also average them:&#xA;$$&#xA;  \frac{dy(x_k)}{dx} \approx &#xA;  \frac 12 \frac{y_{k+1}-y_k}{x_{k+1}-x_k}&#xA;  +&#xA;  \frac 12 \frac{y_k-y_{k-1}}{x_k-x_{k-1}}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;For Problem 2:&lt;/em&gt; Use the chain rule:&#xA;$$&#xA;  \frac{dz}{dy}=\frac{\frac{dz}{dx}}{\frac{dy}{dx}}.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-09T15:39:39.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25485" PostTypeId="2" ParentId="25478" CreationDate="2016-11-09T23:45:37.767" Score="1" Body="&lt;p&gt;@Wolfgang Bangerth has a very important point about what $f$ is. The constant external force will not affect your solution unless your BCs allow for motion. In your case, they do not. An easy conceptual proof of this is to look at the vorticity equation (by taking the curl of your equations). The source term disappears, which means no motion will ensue. Therefore, the motion you're see is from an non-physical setup.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-11-09T23:45:37.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25487" PostTypeId="2" ParentId="25479" CreationDate="2016-11-10T07:31:49.353" Score="1" Body="&lt;p&gt;This is not the best numerical scheme, however, it should work well with second order accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem 1&lt;/strong&gt;: you can apply the following second order approximation&#xA;\begin{align}&#xA;\left( &#xA;\frac{dy}{ dx}&#xA;\right)_i&#xA;&amp;amp;=&#xA;\frac{ &#xA;y_{i+1} (\Delta x_{i} )^2  -&#xA;y_{i-1} ( \Delta x_{i+1}  )^2  +&#xA;y_{i} \left( (\Delta x_{i+1})^2 - (\Delta x_{i})^2  \right)&#xA;}{ \Delta x_{i} \Delta x_{i+1}  \left( \Delta x_{i} + \Delta x_{i+1} \right) } ~~+&#xA;\\&#xA;&amp;amp;~~~~&#xA;-  \frac{\Delta x_{i+1}  \Delta x_{i} }{ 6}&#xA;\left( \frac{d^3 y }{ dx ^3} \right)_i +&#xA;\textrm{Higher-Order-Terms}&#xA;\\&#xA;&amp;amp;\simeq &#xA;\frac{ &#xA;y_{i+1} (\Delta x_{i} )^2  -&#xA;y_{i-1} ( \Delta x_{i+1}  )^2  +&#xA;y_{i} \left( (\Delta x_{i+1})^2 - (\Delta x_{i})^2  \right)&#xA;}{ \Delta x_{i} \Delta x_{i+1}  \left( \Delta x_{i} + \Delta x_{i+1} \right) }&#xA;\end{align}&#xA;for $2 \le i \le n-1$, where $\Delta x_{i} = x_{i}  - x_{i-1}$. For the end-points, i.e. $i=1$ and $i=n$, one can adopt the one-side approximations&#xA;\begin{align}&#xA;\left( &#xA;\frac{dy}{ dx}&#xA;\right)_1&#xA;&amp;amp;\simeq &#xA;\frac{ &#xA;- y_{3} ( x_2 -x_1 )^2  +&#xA;y_{2} ( x_3 - x_1 )^2  -&#xA;y_{1} \left( (x_3 -x_1)^2 - (x_2 - x_1 )^2 \right)&#xA;}{ (x_2-x_1)(x_3-x_1)(x_3-x_2) }, &#xA;\\&#xA;\left( &#xA;\frac{dy}{ dx}&#xA;\right)_n&#xA;&amp;amp;\simeq &#xA;\frac{ &#xA;y_{n-2} ( x_n -x_{n-1} )^2  -&#xA;y_{n-1} ( x_{n} - x_{n-2} )^2  +&#xA;y_{n} \left( ( x_{n} - x_{n-2} )^2 - ( x_{n} - x_{n-1} )^2 \right)&#xA;}{ ( x_n -x_{n-1} ) ( x_n -x_{n-2} ) ( x_{n-1} -x_{n-2} ) } ,&#xA;\end{align}&#xA;which also have second order accuracy on any grid. For further reading, you can read &lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-3-642-56026-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;the excellent book of Ferziger and Peric, Chapter 3&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding &lt;strong&gt;Problem 2&lt;/strong&gt;, you can apply the idea from Wolfgang Bangerth's post, i.e. $$ \left( \frac{ d y }{ d z } \right)_i = \frac{ \left( \frac{dy }{dx} \right)_i }{ \left( \frac{ dz}{ dx} \right)_i } $$ for any $i=1,2,..,n$.&lt;/p&gt;&#xA;" OwnerUserId="15175" LastEditorUserId="15175" LastEditDate="2016-11-10T07:53:36.170" LastActivityDate="2016-11-10T07:53:36.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25489" PostTypeId="1" AcceptedAnswerId="25497" CreationDate="2016-11-10T09:01:35.987" Score="0" ViewCount="417" Body="&lt;p&gt;I have to solve this problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\{\begin{matrix}&#xA;y''=4y'+\frac{1}{h}-7y\\ &#xA;y(1)=1\\ &#xA;y'(1)=1&#xA;\end{matrix}\right. \text{where} \; h=5y'-2y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as it were a DAE (I know I could just substitute $h$ into the equation, but this is just an example, because in reality the problem I have to solve is a DAE and more complex than this).&#xA;When I use &lt;code&gt;ode45&lt;/code&gt; and treat the problem as a second order differential equation, the graph $t$ Vs. $y$ is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oZ4Ef.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oZ4Ef.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but when I treat it as a DAE, the graph is completely different and I do not understand why. Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;ode45&lt;/code&gt; second order differential equation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function yp = dae_normale(t,y)&#xA;yp = zeros(2,1);&#xA;yp(1) = y(2);&#xA;yp(2) = 4*y(2) + 1/(5*y(2) - 2*y(1) ) - 7*y(1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;ode45&lt;/code&gt; second order differential equation &lt;strong&gt;run&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[t,y] = ode45('dae_normale',[1,5],[1,1]);&#xA;[t,y(:,1)]&#xA;plot(t,y(:,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;DAE &lt;code&gt;ode15s&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function out = dae(t,y)&#xA;out = [y(2)&#xA;   4*y(2) + 1/y(3) - 7*y(1)&#xA;   y(3) - 5*y(2) + 2*y(1) ];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;DAE &lt;code&gt;ode15s&lt;/code&gt; &lt;strong&gt;run&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y0 = [1; 1; 3];&#xA;M = [1 0 0; 0 1 0; 0 0 0];&#xA;options = odeset('Mass',M);&#xA;[t,y] = ode15s(@dae,[1 5],y0,options);&#xA;[t,y(:,1)]&#xA;plot(t,y(:,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22288" LastEditorUserId="22288" LastEditDate="2016-11-10T10:43:20.050" LastActivityDate="2016-11-10T19:24:11.720" Title="DAE in Matlab: ode15s" Tags="|dae|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25493" PostTypeId="2" ParentId="25470" CreationDate="2016-11-10T15:58:07.347" Score="2" Body="&lt;p&gt;Try this! Example here has rank 2 for a 3x3 matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = [1 1 1; 2 2 2; 5 7 6];&#xA;rrefA = rref(A);&#xA;&#xA;disp(rrefA) % rref of A&#xA;&#xA;C=[A eye(size(A))];&#xA;&#xA;rrefC = rref(C);&#xA;&#xA;R=rrefC(:,4:6); &#xA;disp(R*A) % should get rref of A&#xA;disp(R)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18998" LastActivityDate="2016-11-10T15:58:07.347" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25494" PostTypeId="1" AcceptedAnswerId="25495" CreationDate="2016-11-10T16:36:48.270" Score="0" ViewCount="183" Body="&lt;p&gt;Let say I have N computers and 2 N-by-N matrices distributed across them called A and B. Matrix A is row distributed while B is column distributed. Therefore, processor i knows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A(i,1:N) and B(1:N,i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do you know an efficient way to determine if A and B are identical that scales well to many processors? (some kind of checksum function for distributed variables that does not depend on how the variable is scattered)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Background is I am working with a fortran code based on MPI. &lt;/p&gt;&#xA;" OwnerUserId="9815" LastActivityDate="2016-11-10T17:32:07.033" Title="checksum of a distributed variable" Tags="|parallel-computing|fortran|mpi|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25495" PostTypeId="2" ParentId="25494" CreationDate="2016-11-10T17:32:07.033" Score="0" Body="&lt;p&gt;You can be &quot;pretty sure&quot; that the two matrices are the same if their hash values are the same. What you will have to do is compute a hash that is &lt;em&gt;additive&lt;/em&gt; (possibly including modulo operations) so that it can be computed efficiently in parallel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I may for a moment assume that the matrices have integer entries, then the (not particularly good) hash function&#xA;$$&#xA;  h(A) = \left(\sum_{i,j} (i^2+j)A_{ij} \right) \text{mod} P&#xA;$$&#xA;with some large integer P is one such function because the modulo operation can be done in parallel by each processor based on their respective parts of the matrix. You then just have to do a global sum reduction module $P$ at the end.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-10T17:32:07.033" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25496" PostTypeId="2" ParentId="25477" CreationDate="2016-11-10T19:02:15.450" Score="4" Body="&lt;p&gt;One of the standard ways to handle singular problems like Poisson with Neumann boundary conditions which lead to a sinuglar problem $Ax=b$ is to make the obtained solution orthogonal to the kernel (which is for simple domains only the constant vector) if you use something like a Krylov subspace method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, you can solve your system with conjugated gradients (CG) and then make the solution orthogonal to the kernel, i.e., subtract the average from your solution.&#xA;But before solving the linear system make sure that your righthand side vector is also orthogonal to the kernel (again, subtract the average of its values), which is the compatibility condition for singular systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach with 1-point Dirichlet condition is known to be bad because you spoil the underlying boundary value problem (which is singular, but nice) and one should usually avoid this way.&lt;/p&gt;&#xA;" OwnerUserId="21958" LastActivityDate="2016-11-10T19:02:15.450" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25497" PostTypeId="2" ParentId="25489" CreationDate="2016-11-10T19:24:11.720" Score="1" Body="&lt;p&gt;ode45 is an explicit (Runge-Kutta) ODE solver and, therefore, is only conditionally&#xA;stable. Normally, ode45 will automatically reduce the step size to maintain&#xA;stability. But, as discussed in this article, &lt;a href=&quot;http://www.radford.edu/~thompson/vodef90web/ErrorControlMatters.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Error Control Matters&lt;/a&gt;, sometimes it does not because the error tolerances are too large. Try modifying your code as follows to reduce the error tolerances:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;opts=odeset('RelTol', 1e-4, 'AbsTol', 1e-7);&#xA;[t,y] = ode45('dae_normale',[1,5],[1,1], opts);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-11-10T19:24:11.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25498" PostTypeId="1" CreationDate="2016-11-10T19:52:28.433" Score="0" ViewCount="80" Body="&lt;p&gt;As far as I know, precision errors become larger as the condition number of a matrix increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider a matrix-based operator:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A = \nabla \bullet k \nabla $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And a matrix-free operator:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\tilde{A} = B_{1} + B_{2} + B_{3}, \qquad B_{i} = \partial_{x_i} k \partial_{x_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One difference between the matrix-free operator and matrix-based operator is that the diagonal of $A$ combines derivatives along all 3 directions, whereas the matrix-free operator must add the diagonal for all 3 directions each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried comparing these two approaches numerically, and I see a difference between them on the order of machine accuracy. It seems that the difference between the operators also increases where $k$ is large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are my questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Am I just visualizing the local condition number where $k$ is large?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is one of these methods superior being that the difference becomes larger? Or is this just lack of precision becoming more apparent from the increasing condition number?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Am I just completely wrong? (this should not happen)&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-11-11T01:22:08.150" Title="Accuracy between ill-conditioned matrix-free vs. matrix-based operators" Tags="|numerical-analysis|matrix|precision|accuracy|operator-splitting|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25499" PostTypeId="1" CreationDate="2016-11-10T20:49:33.540" Score="2" ViewCount="246" Body="&lt;p&gt;Bengt Fornberg derived a general way to compute the weights for arbitrary finite difference schemes in two papers: his &lt;a href=&quot;http://amath.colorado.edu/faculty/fornberg/Docs/MathComp_88_FD_formulas.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1988 paper&lt;/a&gt; and (better) his &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/S0036144596322507&quot; rel=&quot;nofollow noreferrer&quot;&gt;1998 paper&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the numerical errors associated with his approach as codified in the 1998 paper? Can the truncation and round-off errors be calculated for double precision arithmetic, or even estimated, for this particular algorithm -- and if so, how? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand how it is done with standard finite difference schemes that are derived from Taylor expansions, but this seems a bit more tricky, what with Lagrange polynomials and all. &lt;/p&gt;&#xA;" OwnerUserId="9451" LastActivityDate="2016-11-11T11:26:05.517" Title="What is the error associated with Fornberg's algorithm?" Tags="|finite-difference|numerical-analysis|error-estimation|precision|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25500" PostTypeId="2" ParentId="25498" CreationDate="2016-11-10T22:08:32.010" Score="2" Body="&lt;p&gt;When solving a linear system $Ax=b$, you suffer from two sources of error:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Round-off error&lt;/li&gt;&#xA;&lt;li&gt;The ill-conditionedness of the matrix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The condition number of a computational representation of a matrix is, for all practical purposes equal to the condition number of the &lt;em&gt;exact&lt;/em&gt; matrix (which you can't represent computationally), so it does not matter whether you represent it as a split, matrix-free form, or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, whether you add the three matrices together or not does not make any difference for the difference between your computational representation of the matrix and the &quot;exact&quot; matrix. That is, because for all practical purposes, the addition of 3 terms is as accurate as the computation of each of these terms -- 3 times round off is, in practice, equal to round off, namely very small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, there will be no practical difference between the two representations of your matrix, unless your condition number is on the order of one over the level of round-off. That is hopefully not the case.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-10T22:08:32.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25501" PostTypeId="2" ParentId="25470" CreationDate="2016-11-10T23:34:14.827" Score="0" Body="&lt;p&gt;I still cannot understand what this $R$ is good for, but in matlab you can simply (ab)use the slash operator: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; [rrefA, jb] = rref(A);&#xA;&amp;gt;&amp;gt; R = rrefA(:,jb)/A(:,jb);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here an example, showing that this method works also for non square matrices:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; A = randn(10,3)*randn(3,4);&#xA;&amp;gt;&amp;gt; rank(A)&#xA;&#xA;ans =&#xA;&#xA;     3&#xA;&#xA;&amp;gt;&amp;gt; size(A)&#xA;&#xA;ans =&#xA;&#xA;    10     4&#xA;&#xA;&amp;gt;&amp;gt; [rrefA, jb] = rref(A);&#xA;&amp;gt;&amp;gt; R = rrefA(:,jb)/A(:,jb);&#xA;&amp;gt;&amp;gt; rank(R)&#xA;&#xA;ans =&#xA;&#xA;     3&#xA;&#xA;&amp;gt;&amp;gt; norm(R*A-rrefA)/norm(A)&#xA;&#xA;ans =&#xA;&#xA;   1.7351e-15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please note that this answer is just &lt;a href=&quot;https://en.wikipedia.org/wiki/Code_golf&quot; rel=&quot;nofollow noreferrer&quot;&gt;code golf&lt;/a&gt;: reduced row echelon form is not the most stable rank revealing factorisation, as noted above by Christian Clason.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2016-11-10T23:34:14.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25502" PostTypeId="2" ParentId="25498" CreationDate="2016-11-11T01:22:08.150" Score="0" Body="&lt;p&gt;Your &quot;two approaches&quot; are actually the same thing. You know already that they are mathematically the same. But they are also computationally the same thing, except for a possible change in the order of operations. A change in the order of operations will give an error of order of machine epsilon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're right of the condition number, in the following sense. As $k$ increases, the effect of roundoff error is amplified proportionately. So if $k$ is much larger than the quantity your operator is applied to, you'll notice more roundoff error.&lt;/p&gt;&#xA;" OwnerUserId="5541" LastActivityDate="2016-11-11T01:22:08.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25505" PostTypeId="2" ParentId="25450" CreationDate="2016-11-11T08:47:25.543" Score="3" Body="&lt;p&gt;Apply a Schur complement on the constraint $\begin{pmatrix} M &amp;amp; \eta\\\eta^T &amp;amp; 1\end{pmatrix} \succeq 0$&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2016-11-11T08:47:25.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25506" PostTypeId="2" ParentId="25499" CreationDate="2016-11-11T11:26:05.517" Score="2" Body="&lt;p&gt;It seems to me that ultimately you do not care about the numerical error in the finite-difference weights themselves, but in the numerical error you incur when you use these weights. Assessing the latter error is very simple. Just plug into your approximation constant function values. Assuming your weights approximate a derivative, you should obtain identically zero. Most likely you will not obtain exactly zero, but a value on the order of the machine precision. If your result is much larger than that, the cause might be numerical errors in the computed weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can carry out this test also for higher-order functions. Assuming you have a second-order accurate approximation of the second derivative, you will have a TE that is proportional to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$h^2\dfrac{d^4 u}{dx^4}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the fourth derivative of a cubic polynomial is zero, so your approximation should give you the exact second derivative if you plug in the function values of a cubic polynomial. The deviation between the exact derivative and your computed derivative is again an indication of the numerical error in your weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my use of Fornberg's algorithm, I did not see any indication that errors in the numerically computed weights were significant. &lt;/p&gt;&#xA;" OwnerUserId="1792" LastActivityDate="2016-11-11T11:26:05.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25507" PostTypeId="1" AcceptedAnswerId="25509" CreationDate="2016-11-11T15:45:45.383" Score="1" ViewCount="460" Body="&lt;p&gt;There are a number of packages available for generating Continuous Galerkin (CG) FEM meshes (gmsh, tetgen, netgen, etc.), but I have been unable to find one that generates Discontinuous Galerkin (DG) FEM meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently interested in focusing on the solver part of DG, not the mesh construction, so a package that creates a simple DG quadrilateral/triangle mesh would be fine. I'm looking for an array of coordinates, node numbers, and element/node connectivity. Converting a CG mesh file (such as Gmsh) into  a discontinuous one would also be fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has this been done before - and if yes, what is available for personal use?&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="4244" LastEditDate="2016-11-15T12:02:32.230" LastActivityDate="2016-11-15T12:02:32.230" Title="Library for generating Discontinuous Galerkin FEM mesh" Tags="|mesh-generation|discontinuous-galerkin|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25508" PostTypeId="2" ParentId="25457" CreationDate="2016-11-11T17:58:23.337" Score="1" Body="&lt;p&gt;Without special properties of matrixes you can not have a smart decompositions or think of using special methods of resolution for the linear system (for example iterative methods with preconditioning techniques to mitigate the ill-conditioning, always with in mind Wolfgang Bangerth's comment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering Toepliz’s matrix you meet sometimes.&#xA;From a more linked to numerical libraries perspective for what concerns lapack certainly until early 2013, it was not available a special function for Toepliz’s matrix, see &lt;a href=&quot;http://icl.cs.utk.edu/lapack-forum/archives/lapack/msg01415.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;lapack archives&lt;/a&gt;. Also now I don't found dedicate functions. About mkl I don't know special function for Toepliz’s matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps it may be useful &lt;a href=&quot;http://www.netlib.no/netlib/toeplitz/&quot; rel=&quot;nofollow noreferrer&quot;&gt;these functions&lt;/a&gt; of Netlib (so we remain in an environment similar to lapack/mkl) for Toepliz’s matrix.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="22025" LastEditDate="2016-11-11T18:22:22.167" LastActivityDate="2016-11-11T18:22:22.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25509" PostTypeId="2" ParentId="25507" CreationDate="2016-11-11T18:20:26.073" Score="9" Body="&lt;p&gt;You are confused about different concepts. A &lt;em&gt;mesh&lt;/em&gt; is really just a collection of &lt;em&gt;cells&lt;/em&gt; defined by the &lt;em&gt;vertices&lt;/em&gt; of the mesh and which vertices together form each cell. Consequently, a &lt;em&gt;mesh&lt;/em&gt; is an entirely geometric object. It knows nothing about finite element spaces one may define on it -- that's for a later step in your workflow. It's also important that mesh generators really only work on these geometric objects: They know nothing about CG, or DG, or any other function spaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Entirely separate from the mesh are the &lt;em&gt;function spaces that you define on the mesh&lt;/em&gt;. Oftentimes, though not always, they are built by interpolating between &lt;em&gt;nodes&lt;/em&gt;. If you have lowest order, continuous elements, these nodes &lt;em&gt;happen&lt;/em&gt; to be located at vertices, but they are separate logical concepts. Likewise, if you use DG methods, the nodes are duplicated -- i.e., in 1d, you will have 2 nodes on each vertex, but there is still only one vertex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite element spaces are defined in finite element codes, which are traditionally separate from mesh generators. As a consequence, you will just read a regular mesh, and then in your own code assign multiple degrees of freedom (&quot;nodes&quot;) to each vertex.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-11T18:20:26.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25510" PostTypeId="1" AcceptedAnswerId="25609" CreationDate="2016-11-11T23:56:40.023" Score="1" ViewCount="86" Body="&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Fractal_dimension_on_networks&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; from Wikipedia, the box-covering algorithm calculates the fractal dimension of a graph. The algorithm is based on the concept of distance between nodes; see for example the sentence:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A box consists of nodes separated by a distance $l &amp;lt; l_B$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The distance between nodes can be defined also for directed graphs, so I think the algorithm should work also in that case. However, on the Internet, I cannot find any explicit statement about the possibility to use this algorithm for directed graphs.&lt;/p&gt;&#xA;" OwnerUserId="22319" LastEditorUserId="7574" LastEditDate="2016-11-19T16:38:39.683" LastActivityDate="2016-11-19T16:38:39.683" Title="Does the box-covering algorithm work also for directed graphs?" Tags="|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25511" PostTypeId="1" CreationDate="2016-11-12T05:12:04.940" Score="3" ViewCount="212" Body="&lt;p&gt;I have a generalized eigen-problem: $A\psi = \lambda B \psi$ with $A$ and $B$ are large (&gt;1000) complex non-Hermitian matrices. I know that eigenvalues with largest and smallest absolute values can be found using iterative approaches like ARPACK. But I am only interested in those complex eigenvalues (not all of them) with absolute values closest to 1. It there any fast and parallel approach to do this?&lt;/p&gt;&#xA;" OwnerUserId="22320" LastActivityDate="2016-11-12T05:12:04.940" Title="Eigenvalues with absolute values close to 1" Tags="|linear-algebra|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25516" PostTypeId="1" CreationDate="2016-11-10T03:22:30.553" Score="8" ViewCount="3531" Body="&lt;p&gt;When does it typically make sense in programming to be testing the equality of two floating point numbers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a == b &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where both a &amp;amp; b are floats. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My naive impression is that one would always test the difference against some tolerance epsilon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I wrong? Can testing the equality of floats be meaningful in certain contexts? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any examples from the wild? i.e. From real codebases or applications out there on git etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. I'm implicit assuming that using the equality operator on floats is indeed meaningful in &lt;em&gt;some&lt;/em&gt; contexts; otherwise why would most programming languages allow it. &lt;/p&gt;&#xA;" OwnerUserId="3795" OwnerDisplayName="curious_cat" LastActivityDate="2016-11-13T11:40:48.497" Title="Testing equality of two floats: Realistic example" Tags="|floating-point|" AnswerCount="6" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25517" PostTypeId="2" ParentId="25516" CreationDate="2016-11-10T07:27:10.350" Score="3" Body="&lt;p&gt;An obvious example where &lt;code&gt;==&lt;/code&gt; is ok, is when &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; is the same number i.e. &lt;code&gt;a=c; b=c&lt;/code&gt;, for example to check if &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; were initialized the same way. Of course, &lt;code&gt;|a-b| &amp;lt; epsilon&lt;/code&gt; would also work here. The only problem is, how small is &lt;code&gt;epsilon&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, &lt;code&gt;a == b&lt;/code&gt; would compile into one instruction while &lt;code&gt;|a-b| &amp;lt; epsilon&lt;/code&gt; would take quite a few.&lt;/p&gt;&#xA;" OwnerUserId="22332" OwnerDisplayName="Karolis Juodelė" LastActivityDate="2016-11-10T07:27:10.350" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25518" PostTypeId="2" ParentId="25516" CreationDate="2016-11-10T09:12:26.750" Score="0" Body="&lt;p&gt;Your &quot;naive impression&quot; is not a &quot;naive impression&quot; - it is the result of reading about floating point arithmetic and only half understanding it. The &quot;naive impression&quot; would be the obvious impression that to find out whether a and b are equal, you ask if they are equal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are plenty of situations where all you need to know is whether two floating point numbers are equal or not. There are plenty of situations where you know that there are either no rounding errors, or no variations due to rounding errors. Like converting decimal numbers to floating point, which will be deterministic in any sane implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a good one: Someone claimed that for any two floating point numbers a, b the result of (b + a + b) and (b + b + a) are the same and you want to test that claim. Try doing that without comparing two floating point numbers for equality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a better one: Try to create a set of floating point numbers. &lt;/p&gt;&#xA;" OwnerUserId="15239" OwnerDisplayName="gnasher729" LastActivityDate="2016-11-10T09:12:26.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25519" PostTypeId="2" ParentId="25516" CreationDate="2016-11-10T09:16:55.243" Score="2" Body="&lt;p&gt;An extreme example: IBM was the first company building processors with a fused multiply-add instruction. Using that instruction, they created a very fast method for calculating square roots according to the IEEE-754 standard. This method fails for one single input value 1 ≤ x &amp;lt; 4: If x is the largest number representable as a floating point number that is less than 4, then the result would be rounded incorrectly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So somewhere in their implementation, they check whether x equals that one specific value. They want to recognise that value, and not any others. &lt;/p&gt;&#xA;" OwnerUserId="15239" OwnerDisplayName="gnasher729" LastActivityDate="2016-11-10T09:16:55.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25520" PostTypeId="2" ParentId="25516" CreationDate="2016-11-12T15:22:03.597" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My naive impression is that one would always test the difference against some tolerance epsilon.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A not-naive implementation of this idea should probably take advantage of the equality comparison operator to handle the important special cases that the &lt;a href=&quot;http://en.wikipedia.org/wiki/IEEE_754-2008&quot; rel=&quot;nofollow noreferrer&quot;&gt;IEEE 754&lt;/a&gt; standard contemplates (infinities, denormalized numbers...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at &lt;a href=&quot;https://stackoverflow.com/a/4915891/3235496&quot;&gt;How should I do floating point comparison?&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (a == b)  // shortcut, handles infinities&#xA;  return true;&#xA;&#xA;if (a == 0 || b == 0 || diff &amp;lt; Float.MIN_NORMAL) {&#xA;  // a or b is zero or both are extremely close to it&#xA;  // relative error is less meaningful here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Sometimes there really is &lt;strong&gt;one&lt;/strong&gt; answer that is correct and you want exact equality. Testing the correctness of an implementation is a good example (i.e. &lt;a href=&quot;https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-floatsso-test-them-all/&quot; rel=&quot;nofollow noreferrer&quot;&gt;There are Only Four Billion Floats–So Test Them All!&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="22422" OwnerDisplayName="manlio" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-11-12T15:28:09.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25521" PostTypeId="2" ParentId="25516" CreationDate="2016-11-12T20:18:22.557" Score="-1" Body="&lt;p&gt;See this question: &lt;a href=&quot;https://cs.stackexchange.com/questions/19013/is-transitivity-required-for-a-sorting-algorithm?rq=1&quot;&gt;https://cs.stackexchange.com/questions/19013/is-transitivity-required-for-a-sorting-algorithm?rq=1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It explains very nicely how floating point comparison with some epsilon can lead to a complete and total failure of the quicksort algorithm. &lt;/p&gt;&#xA;" OwnerUserId="15239" LastEditorUserId="-1" LastEditDate="2017-04-13T12:48:31.170" LastActivityDate="2016-11-12T20:18:22.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25522" PostTypeId="2" ParentId="25516" CreationDate="2016-11-12T20:27:52.567" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;My naive impression is that one would always test the difference against some tolerance epsilon.&#xA;  Am I wrong? Can testing the equality of floats be meaningful in certain contexts?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is not a unique recipes.&#xA;In &lt;a href=&quot;https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; there is an exhaustive treatment, where you can found a complete answer with technical and code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, there are mainly 3 cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;comparing against zero&lt;/li&gt;&#xA;&lt;li&gt;comparing against a non-zero&lt;/li&gt;&#xA;&lt;li&gt;comparing two arbitrary numbers&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Your idea to use a comparison against a tolerance is good for some cases, but there is also a technic based on Unit in the last place (&lt;a href=&quot;https://en.wikipedia.org/wiki/Unit_in_the_last_place&quot; rel=&quot;nofollow noreferrer&quot;&gt;ULP&lt;/a&gt;),&#xA;described inside the article&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm implicit assuming that using the equality operator on floats is indeed meaningful in some contexts; otherwise why would most programming languages allow it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As above, there are situations where you can use it, but be warned. For example the gcc compiler has a warning:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;warning: comparing floating point with == or != is unsafe&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I add some considerations above this argument also they are not strictly related to the case &lt;code&gt;a == b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Equality with expression&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering the case:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a + b == c &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with &lt;code&gt;a b c&lt;/code&gt; floats. This is what we are coding, but with more pedantic from a math point of view we are doing:&#xA;$$ &#xA;\begin{aligned}&#xA;a \oplus b &amp;amp;== c \\&#xA;\mathbf{fl}(\mathbf{fl}(a) + \mathbf{fl}(b)) &amp;amp;== \mathbf{fl}(c)\\&#xA;\end{aligned}&#xA;$$&#xA;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\oplus$ machine add&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{fl}(x)$ is the floating point representation of the number $x$, i.e. the real machine number.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With this operation we are introducing a possible estimated error (give the bound) from:&#xA;$$ &#xA;\left| \frac{a}{a+b}\right|\mathit{err}_a + &#xA;\left| \frac{b}{a+b}\right|\mathit{err}_b&#xA;$$&#xA;where&#xA;$$\mathit{err}_x = \frac{\lvert x - \mathbf{fl}(x) \rvert}{\lvert x \rvert}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in this case there the use of &lt;code&gt;==&lt;/code&gt; is more delicate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Porting in different environments&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we port a code in different environments (different machine) we can obtain some different result (for example try to think to unit test). Also in the case the use of &lt;code&gt;==&lt;/code&gt; is delicate.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="22025" LastEditDate="2016-11-13T11:40:48.497" LastActivityDate="2016-11-13T11:40:48.497" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25524" PostTypeId="1" CreationDate="2016-11-13T00:12:22.197" Score="1" ViewCount="164" Body="&lt;p&gt;I want to solve a small (20 $\times$ 20 up to 30$\times$30) system which is not symmetric, not diagonally dominant, and not sparse. Each row contains a modified form of the Legendre coefficient of a set of user specified functions. The difference between between each row is that the functions are raised to a different power (row $i$ has $[f]^i$ and row $i+1$ has $[f]^{i+1}$). Currently the system has Cond(A) ~ $10^{19}$ and for a 20$\times$20 system Rank(A) =15.  &lt;/p&gt;&#xA;" OwnerUserId="22326" LastActivityDate="2016-11-13T23:59:41.433" Title="Solving a small non-symmetric, non-diagonally dominant, and non-sparse system" Tags="|linear-algebra|computational-physics|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25525" PostTypeId="1" CreationDate="2016-11-13T02:33:51.380" Score="1" ViewCount="95" Body="&lt;p&gt;I have the following PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial c}{\partial t} + v\frac{\partial c}{\partial x} - D\frac{\partial^2 c}{\partial x^2} = 0 &#xA;\\$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have discretized it such that i now have&#xA;$\frac{dC}{dt} =  aC_{i+1} +bC_{i} +cC_{i-1} \\$    (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been given the following initial conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\\C=C_0\quad\forall  \ 0&amp;lt;t&amp;lt;\tau\\$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\\C=0\quad\forall  \ t&amp;gt;\tau\\$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i have been asked how i would go about: ' describe how to implement the initial conditions in terms of the solution domain and in terms of any extra vectors'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question is from a degree course and is about injecting a trace of dye.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have discretised the convective component using simple upwind and the diffusive component using central differences. I'm struggling to understand what the question is asking me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that evaluation at i=1 poses a problem in that it requires C at $\ C_0\\$. This is outside of the solution domain and would require a ghost node, but this is outside the solution domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are peoples thoughts? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the 1st initial condition i was thinking of ignoring the diffusive component and for the 2nd initial condition ignoring both the diffusive and convective terms since there is no dye left. Then i would have two equations that describe the initial conditions.&lt;/p&gt;&#xA;" OwnerUserId="22328" LastActivityDate="2016-11-13T19:33:08.113" Title="Implementing initial conditions into the solution domain of a 1-D advection-diffusion equation" Tags="|pde|finite-difference|parabolic-pde|advection-diffusion|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25527" PostTypeId="1" AcceptedAnswerId="25536" CreationDate="2016-11-13T04:09:28.833" Score="-2" ViewCount="539" Body="&lt;p&gt;How can the non-smooth functions be integrated using Matlab? Can the Cash-Karp Method be used? The Dormand-Prince method seems to give an error while integrating a nonsmooth solution.&lt;/p&gt;&#xA;&lt;p&gt;I am using MATLAB ODE45 for the integration of my ordinary differential equation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5EQAx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5EQAx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\{q\}$&lt;/span&gt; is a Column Vector of length 14. The excitation force on the right-hand side varies in the following form&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UwfiG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UwfiG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\{h_p\}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\{g_p\}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\{h_g\}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\{g_g\}$&lt;/span&gt; are column vectors of length 6&lt;/p&gt;&#xA;&lt;p&gt;I call it nonsmooth because &lt;span class=&quot;math-container&quot;&gt;$\{F_\text{excit}\}$&lt;/span&gt; varies with &lt;span class=&quot;math-container&quot;&gt;$F_\text{mf}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The system of equations can be solved when &lt;span class=&quot;math-container&quot;&gt;$F_\text{mf}=0$&lt;/span&gt;. When &lt;span class=&quot;math-container&quot;&gt;$F_\text{mf}$&lt;/span&gt; is nonzero, I am getting a warning which actually indicates that integration cannot be performed.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning/Error Message in Matlab&#xA;&#xA;Warning: Failure at t=1.821477e+00.  Unable to meet&#xA;integration tolerances without reducing the step &#xA;size below the smallest value&#xA;allowed (3.552714e-15) at time t.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I did try ODE15S but the error appeared much earlier than ODE45. Both gave me the same error&lt;/p&gt;&#xA;&lt;p&gt;I use square brackets to represent matrices and curly braces to represent vectors.&lt;/p&gt;&#xA;" OwnerUserId="22331" LastEditorUserId="20688" LastEditDate="2022-01-10T06:32:26.477" LastActivityDate="2022-10-19T08:52:42.643" Title="Non-smooth integration in Matlab: Cash-Karp vs Dormand-Prince" Tags="|ode|time-integration|runge-kutta|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25528" PostTypeId="1" AcceptedAnswerId="25540" CreationDate="2016-11-13T08:04:17.277" Score="0" ViewCount="195" Body="&lt;p&gt;Concerning edge finite element (not nodal element), is there any explicit function (e.g. polynomial) to describe the interpolation within the element? &#xA;I've read numerous literatures such as &quot;Mixed finite element methods and application&quot; by Boffi et al, and &quot;FENICS manual (&lt;a href=&quot;https://femtable.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://femtable.org/&lt;/a&gt;)&quot;. I could follow the concepts of mixed variational principles and Hilbert spaces: H(Div) and H(curl). However, I'm having a hard time implementing the element because the interpolatation functions and spaces are written in mathematical terms, which are rather jargons for me. For instance, $P_1\Lambda^1(\Delta_2)$, and $P_1\Lambda^2(\Delta_2)$ in exterior calculus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be greatly appreciated if anyone suggest me a way to interprete these jargons, or some literatures to understand it. Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="17204" LastActivityDate="2016-11-14T01:55:26.070" Title="Interpolation function for vector finite element" Tags="|finite-element|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25529" PostTypeId="2" ParentId="25524" CreationDate="2016-11-13T11:52:42.993" Score="1" Body="&lt;p&gt;Considering that the matrixes are not huge, but ill-conditionated you can try to use a regularization technique:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;truncated SVD (for example see &lt;a href=&quot;http://infolab.stanford.edu/pub/cstr/reports/na/m/86/36/NA-M-86-36.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tikhonov_regularization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tikhonov regularization&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Unlike the preconditioning techniques, where you have an equivalent system but with a better spectral properties for the matrix, here you approximate the problem, i.e. the original system.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2016-11-13T11:52:42.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25530" PostTypeId="2" ParentId="25524" CreationDate="2016-11-13T14:25:25.157" Score="3" Body="&lt;p&gt;As I understand your question you are given a set of values $f,y\in\mathbb{R}^n$ and have to solve a system&#xA;\begin{align*}&#xA;\begin{pmatrix}&#xA;f_1&amp;amp;\ldots&amp;amp; f_n\\&#xA;\vdots&amp;amp;&amp;amp;\vdots\\&#xA;f_1^n&amp;amp;\ldots&amp;amp; f_n^n&#xA;\end{pmatrix}\cdot&#xA;x&#xA;&amp;amp;=&#xA;y&#xA;\end{align*}&#xA;for $x\in\mathbb{R}^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the values $f_i$ are nonzero. Otherwise your matrix would be singular and you were out of luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After scaling the arguments $z_i := f_i \cdot x_i$ you obtain a Vandermonde-matrix as system matrix:&#xA;\begin{align*}&#xA;\begin{pmatrix}&#xA;1&amp;amp;\ldots&amp;amp; 1\\&#xA;f_1&amp;amp;\ldots&amp;amp; f_n\\&#xA;\vdots&amp;amp;&amp;amp;\vdots\\&#xA;f_1^{n-1}&amp;amp;\ldots&amp;amp; f_n^{n-1}&#xA;\end{pmatrix}\cdot&#xA;z&#xA;&amp;amp;=&#xA;y&#xA;\end{align*}&#xA;Note that matrices of this kind are known to be very ill conditioned in the general case.&#xA;There are &lt;a href=&quot;http://www.ams.org/journals/mcom/1970-24-112/S0025-5718-1970-0290541-1/S0025-5718-1970-0290541-1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;special algorithms&lt;/a&gt; for such systems that exploit divided differences. Maybe you find even more recent variants of that algorithm by an internet search.&#xA;There is also a &lt;a href=&quot;https://people.sc.fsu.edu/~jburkardt/m_src/vandermonde/vandermonde.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;matlab code VANDERMONDE under LGPL license&lt;/a&gt; that implements this method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the similar question &lt;a href=&quot;https://math.stackexchange.com/questions/384126/solving-linear-equations-with-vandermonde&quot;&gt;https://math.stackexchange.com/questions/384126/solving-linear-equations-with-vandermonde&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After solving that system you can scale back the solution via $x_i = \frac{z_i}{f_i}$.&lt;/p&gt;&#xA;" OwnerUserId="7652" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-11-13T15:50:39.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25532" PostTypeId="2" ParentId="25216" CreationDate="2016-11-13T19:02:18.653" Score="0" Body="&lt;p&gt;Here is an example how I have tested the hypothesis that $|S| \le 1$ for given constraints. I have run the following code in Mathematica with many many values of variables &lt;em&gt;nstart&lt;/em&gt; and &lt;em&gt;nend&lt;/em&gt; and the method $M$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(* amplification parameter *)&#xA;S=&#xA;(2*(6+c-c^2+d-c*d-d^2-2*c*d*f+d*(-1+c+d+2*c*f)*Cos[y]+c*Cos[x]*(-1+c+d+2*d*f-d*(1+2*f)*Cos[y])-3*I*c*Sin[x]-3*I*d*Sin[y]-c*d*Sin[x]*Sin[y]))&#xA;/&#xA;(12+8*c+c^2+8*d+2*c*d+d^2-2*c*(5+c+d)*Cos[x]+c*(2+c)*Cos[2*x]-10*d*Cos[y]-2*c*d*Cos[y]-2*d^2*Cos[y]+2*d*Cos[2*y]+d^2*Cos[2*y]+2*c*d*Cos[x+y]+10*I*c*Sin[x]+2*I*c^2*Sin[x]+2*I*c*d*Sin[x]-2*I*c*Sin[2*x]-I*c^2*Sin[2*x]+10*I*d*Sin[y]+2*I*c*d*Sin[y]+2*I*d^2*Sin[y]-2*I*d*Sin[2*y]-I*d^2*Sin[2*y]-2*I*c*d*Sin[x+y]);&#xA;&#xA;(* numerical global maximum search *)&#xA;M=&quot;RandomSearch&quot;;(* RandomSearch NelderMead DifferentialEvolution SimulatedAnnealing*)&#xA;nstart=1;nend=10;&#xA;&#xA;For[n=nstart,n&amp;lt;=nend,n++,&#xA;Print[NMaximize[{Abs[S],-Pi&amp;lt;=x&amp;lt;=Pi, -Pi&amp;lt;= y&amp;lt;= Pi, 0&amp;lt;= c &amp;lt;=n, 0&amp;lt;=d&amp;lt;=n, -1&amp;lt;=f&amp;lt;=0},&#xA;{x,y,c,d,f}, Method-&amp;gt;{M,&quot;SearchPoints&quot;-&amp;gt;20,RandomSeed-&amp;gt;n}]]]&#xA;&#xA;(* example of output *)&#xA;{1.,{x-&amp;gt;3.87721*10^-7,y-&amp;gt;-3.14159,c-&amp;gt;0.996877,d-&amp;gt;0.,f-&amp;gt;-0.998816}}&#xA;{1.,{x-&amp;gt;-3.14159,y-&amp;gt;2.19618*10^-8,c-&amp;gt;0.,d-&amp;gt;1.71028,f-&amp;gt;-0.941516}}&#xA;{1.,{x-&amp;gt;-3.14159,y-&amp;gt;-3.14159,c-&amp;gt;0.,d-&amp;gt;0.,f-&amp;gt;-1.}}&#xA;{1.,{x-&amp;gt;3.14159,y-&amp;gt;3.14159,c-&amp;gt;0.,d-&amp;gt;0.,f-&amp;gt;-1.}}&#xA;{1.,{x-&amp;gt;4.71742*10^-7,y-&amp;gt;3.14159,c-&amp;gt;3.3804,d-&amp;gt;0.,f-&amp;gt;-0.997559}}&#xA;{1.,{x-&amp;gt;-3.14159,y-&amp;gt;-3.14159,c-&amp;gt;0.,d-&amp;gt;0.,f-&amp;gt;-1.}}&#xA;{1.,{x-&amp;gt;-3.14159,y-&amp;gt;-2.4762*10^-7,c-&amp;gt;0.,d-&amp;gt;1.34533,f-&amp;gt;-0.922121}}&#xA;{1.,{x-&amp;gt;-0.000101745,y-&amp;gt;-0.0000189795,c-&amp;gt;3.48753,d-&amp;gt;4.20828,f-&amp;gt;-0.84976}&#xA;{1.,{x-&amp;gt;3.14159,y-&amp;gt;1.0698*10^-6,c-&amp;gt;0.,d-&amp;gt;8.98382,f-&amp;gt;-0.63621}}&#xA;{1.,{x-&amp;gt;0.000268716,y-&amp;gt;-0.0000310003,c-&amp;gt;0.593874,d-&amp;gt;3.95438,f-&amp;gt;-0.832499}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2016-11-13T19:02:18.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25534" PostTypeId="1" AcceptedAnswerId="25535" CreationDate="2016-11-13T20:05:13.443" Score="2" ViewCount="270" Body="&lt;p&gt;Suppose I wish to solve the following optimization problem:&#xA;$$&#xA;\begin{array}{rl}&#xA;\min_{\mu\in\mathbb{R}^n} &amp;amp;\mu^\top c\\&#xA;\textrm{subject to} &amp;amp; \mu\in C_1\cap C_2\cap\cdots\cap C_k,&#xA;\end{array}&#xA;$$&#xA;where each $C_i\subseteq\mathbb{R}^n$ is a convex set.  Furthermore, I have access to projection operators&#xA;$$&#xA;p_i(x):=\left\{&#xA;\begin{array}{rl}&#xA;\arg\min_y &amp;amp; \|y-x\|_2\\&#xA;\textrm{subject to} &amp;amp; y\in C_i.&#xA;\end{array}&#xA;\right.&#xA;$$&#xA;&lt;strong&gt;Is there an optimization algorithm that wraps around the $p_i(\cdot)$'s that can find $\mu$?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Notes:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The number of sets $k$ is potentially large.  When I tried to apply standard ADMM tricks to this problem, I ended up needing $O(nk)$ space, which is too much.&lt;/li&gt;&#xA;&lt;li&gt;If I add a regularizer $\varepsilon \|\mu\|_2^2$ to the problem, then it looks like projection and I can use a cyclic method like &lt;a href=&quot;https://en.wikipedia.org/wiki/Dykstra%27s_projection_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dykstra's algorithm&lt;/a&gt;. But I really would like to solve this problem without regularization.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2016-11-14T00:07:10.547" Title="Minimizing linear objective on intersection of convex sets" Tags="|optimization|convex-optimization|constrained-optimization|nonlinear-programming|projection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25535" PostTypeId="2" ParentId="25534" CreationDate="2016-11-13T21:47:00.967" Score="3" Body="&lt;p&gt;See this recent paper on an extension of stochastic gradient descent that could be used on your problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1511.03760&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1511.03760&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also apply Dykstra's algorithm (or any other algorithm that does alternating projections on convex sets) by setting a target value for $\mu^{T}c \leq \gamma $ and reducing it once feasibility has been achieved.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-11-13T21:47:00.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25536" PostTypeId="2" ParentId="25527" CreationDate="2016-11-13T22:31:48.703" Score="3" Body="&lt;p&gt;You really have not provided enough information to diagnose the cause of&#xA;your problem (e.g. I don't know if the dimension of your q-vector is two or two million) but guessing, based on your error message, I doubt it has&#xA;anything to do with &quot;smoothness&quot; of the forcing function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, the way to debug these MATLAB ODE solvers when you get the error&#xA;message you show is to repeat the analysis, stopping just before the failure time&#xA;(e.g. 1.8 seconds). You want to save the solution at enough time steps &#xA;so you can plot some of the key dependent variables as a function of time.&#xA;I think it is very likely that one or more of these is going to plus or&#xA;minus infinity near this final time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for this is most likely an error in your formulation or&#xA;input to ode45.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second possibility is that your system is inherently unstable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A third possibility is that your system of ODE is stiff and that is&#xA;causing problems for ode45. ode45 is not designed to solve stiff ODE&#xA;systems. Changing to an ode solver with a different Runge-Kutta pair or&#xA;some other explicit solver will not solve this problem. The MATLAB &#xA;solver-of-choice for stiff systems of the form you show is ode15s.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2016-11-14T13:17:32.283" LastActivityDate="2016-11-14T13:17:32.283" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25538" PostTypeId="2" ParentId="25524" CreationDate="2016-11-13T23:59:41.433" Score="1" Body="&lt;p&gt;Another option here would be to use high precision arithmetic to solve your system of equations.   Quadruple precision floating point arithmetic should be enough to deal with your condition number of $10^{19}$ and give you an answer good to about 10 digits, assuming that all of the coefficients in your problem are accurate to 30 digits.  Beyond that, there are libraries for extended precision floating point arithmetic that could be used to obtain even more digits.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-11-13T23:59:41.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25540" PostTypeId="2" ParentId="25528" CreationDate="2016-11-14T00:47:19.960" Score="1" Body="&lt;p&gt;There are a lot of less abstract frameworks for defining the finite element basis functions, and classical books among them like Ciarlet, Brezzi, Fortin, etc. As far as I know, the differential forms formalism is quite a new thing from the viewpoint of FEM discretizations and was developed not many years ago. Are you looking for something like in &lt;a href=&quot;http://www.math.clemson.edu/~vjervin/papers/erv112.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.clemson.edu/~vjervin/papers/erv112.pdf&lt;/a&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the formalism, most people refer to the works by Douglas Arnold with keywords &quot;finite element exterior calculus&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this could help&lt;/p&gt;&#xA;" OwnerUserId="21958" LastActivityDate="2016-11-14T00:47:19.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25541" PostTypeId="2" ParentId="25528" CreationDate="2016-11-14T01:55:26.070" Score="0" Body="&lt;p&gt;If you take the lowest-order 2d Raviart-Thomas element as an example, the degrees of freedom are located at the mid-point of edges, and represent the normal component of the vector field. As a consequence, if you want to define an interpolation operator $\vec\psi_h = I_h \vec f$ for some vector-valued (and continuous) function $\vec f$, then you need to evaluate $\vec f$ at the mid-points of edges, take the component of this vector normal to the edge, and that is the value of $\vec n \cdot \vec \psi_h$ at that mid-points of edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, if you had lowest-order Nedelec elements, it would be the &lt;em&gt;tangential&lt;/em&gt; component that you had to evaluate.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-14T01:55:26.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25542" PostTypeId="1" CreationDate="2016-11-14T02:21:37.280" Score="1" ViewCount="172" Body="&lt;p&gt;Consider the problem $-u''=f$, $u(a)=A$, $u(b)=B$. Everywhere, FEM seems to be introduced by using the case $(a,b,A,B) = (0,1,0,0)$, in which case the term $u'(a)v(a)-u'(b)v(b)$ resulting from integration by parts vanishes. If we divide $[a,b]$ into $a=x_0 &amp;lt; ... &amp;lt; x_n = b$, we'll end up solving an $(n-1)\times (n-1)$ system, fine and dandy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you solve the above problem without convenient boundary conditions? I have tried searching and the few results I find are for higher dimensions, or I simply don't understand the answers. I have looked up several books on the matter and I can't find an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively I would form a stiffness matrix $K=(K_{ij})_{i,j=0}^n$ and the load vector $b$ per the &quot;usual&quot; way, then add... something, somehow. In a case with BCs $au'(0) = \kappa_0(u(0)-g_0)$, $-au'(L) = \kappa_L(u(L)-g_L)$ the solution was to add $\kappa_0$ to $K_{00}$, $\kappa_L$ to $K_{nn}$, $\kappa_0 g_0$ to $b_0$ and $\kappa_L g_L$ to $b_n$. I don't know if that's at all applicable (I don't have any information about first derivatives) but those are the only special cases I see covered. On the other hand, $(n+1)\times (n+1)$ doesn't sound right since we know the values at the boundary and so we shouldn't want degrees of freedom there...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really need help with this, I'd appreciate any advice, thanks.&lt;/p&gt;&#xA;" OwnerUserId="22341" LastActivityDate="2016-11-14T02:21:37.280" Title="1D FEM with nonhomogeneous Dirichlet BC" Tags="|finite-element|" AnswerCount="0" CommentCount="5" ClosedDate="2018-11-19T15:15:10.853" ContentLicense="CC BY-SA 3.0" />
  <row Id="25543" PostTypeId="1" CreationDate="2016-11-14T09:06:44.443" Score="2" ViewCount="227" Body="&lt;p&gt;I'm doing some reasearch on electromagnetic nanostructures and I have to solve this differential equation (the exact values of the constants don't matter, I just want &lt;strong&gt;all&lt;/strong&gt; the possible solutions of y(x) given some values to these constants).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d^2 y}{dx^2}=-\frac{1}{x}\frac{dy}{dx}+\frac{\sin(2y)}{2} (\frac{1}{x^2}+\frac{K}{A})-\frac{D}{A}\frac{\sin(y)}{x}+\frac{\mu HM}{2A}\sin(y)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from x=0 till x=R, with the boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;y(0)=0,\ \frac{dy}{dx}(R)=\frac{-D}{2A}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe you can not find an analitic solution to this equation, so I've been trying to use numerical methods like the shooting method (given the boundary conditions, I found it appropiate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that the singularity on x=0 doesn't let me find the solutions. I obtain different results depending on how many steps I take in the method. Do you have any ideas on how I can proceed? Another numerical method maybe?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also posted this on &lt;a href=&quot;https://physics.stackexchange.com/questions/292320/skyrmions-solving-a-second-order-nonlinear-ode-with-a-singularity-on-x-0&quot;&gt;Physics&lt;/a&gt; and &lt;a href=&quot;https://math.stackexchange.com/questions/2010468/solving-a-second-order-nonlinear-ode-with-a-singularity-on-x-0&quot;&gt;Mathematics&lt;/a&gt; StackExchange, but for now I couldn't fix it.&lt;/p&gt;&#xA;" OwnerUserId="22342" LastEditorUserId="-1" LastEditDate="2017-04-13T12:40:12.913" LastActivityDate="2017-02-13T14:41:44.373" Title="Solving a second-order nonlinear ODE with a singularity on x=0" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25547" PostTypeId="1" CreationDate="2016-11-14T14:24:20.363" Score="1" ViewCount="1055" Body="&lt;p&gt;Is there any way to numerically solve the following two-dimensional equation:&#xA;\begin{equation}&#xA;\nabla_{xy} \cdot \vec{f}(x,y) = a(x,y)&#xA;\end{equation}&#xA;on a rectangular grid, knowing that $\vec{f}(x,y)$ is zero on the boundaries?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastActivityDate="2016-11-15T04:11:25.290" Title="Numerical solution to 2D divergence equation" Tags="|pde|numerics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25548" PostTypeId="2" ParentId="25547" CreationDate="2016-11-14T14:43:55.107" Score="0" Body="&lt;p&gt;There are many ways to do this, either finite difference methods, finite volume methods, or finite element methods can be applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above equation often known as the poison equation is often the first presented equation in a new topic in a numerical methods course.&lt;/p&gt;&#xA;" OwnerUserId="17691" LastActivityDate="2016-11-14T14:43:55.107" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25549" PostTypeId="2" ParentId="25547" CreationDate="2016-11-14T17:04:42.297" Score="0" Body="&lt;p&gt;I have constructed the matrix associated with the finite difference discretisation of $u_{xy}$ in an answer to a stack overflow question &lt;a href=&quot;https://stackoverflow.com/a/36015353/5358968&quot;&gt;Matrix to generate finite difference&lt;/a&gt; (the matrix is in the MATLAB code as &lt;code&gt;A&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, mentioned on &lt;a href=&quot;https://en.wikipedia.org/wiki/Hyperbolic_partial_differential_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt; and referencing &lt;a href=&quot;https://books.google.co.uk/books?isbn=0821849743&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evans (2010)&lt;/a&gt; (p. 400), you can transform $u_{xy}$ equation into the one dimensional wave equation. I've attempted this below. Boundary conditions could get trickyer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to get rid of your mixed derivatives you can use a transformation of coordinates&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;  \hat{x} &amp;amp;= x+y\,, &amp;amp; x = \frac{\hat{x}+\hat{y}}2\,, \\&#xA;  \hat{y} &amp;amp;= x-y\,, &amp;amp; y = \frac{\hat{x}-\hat{y}}2\,.&#xA;\end{aligned}&#xA;$$&#xA;You can show that&#xA;$$&#xA;\begin{gathered}&#xA;  u_{\hat{x}\hat{x}} = u_{xx} + 2 u_{xy} + u_{yy}\,,\\&#xA;  u_{\hat{y}\hat{y}} = u_{xx} - 2 u_{xy} + u_{yy}\,,&#xA;\end{gathered}&#xA;$$&#xA;then&#xA;$$&#xA;\begin{aligned}&#xA;u_{xy} {}&amp;amp;{}=  &#xA;\frac{&#xA;u_{\hat{x}\hat{x}} -   u_{\hat{y}\hat{y}}&#xA;}4&#xA;\\&#xA; {}&amp;amp;{}= a\left(\frac{\hat{x}+\hat{y}}2,\frac{\hat{x}-\hat{y}}2\right)\,.&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-11-14T17:04:42.297" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25550" PostTypeId="1" AcceptedAnswerId="25553" CreationDate="2016-11-14T19:05:05.597" Score="1" ViewCount="167" Body="&lt;p&gt;Consider the following second-order cone program (SOCP):&#xA;$$&#xA;\begin{array}{rl}&#xA;\min_x &amp;amp; c^\top x\\&#xA;\mathrm{s.t.} &amp;amp; \|A_ix+b_i\|_2 \leq c_i^\top x+d_i \ \forall i&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I solve the dual of a second-order cone program (SOCP) for the dual variables $\lambda_i,u_i$:&#xA;$$&#xA;\begin{array}{rl}&#xA;\max_{\lambda,u} &amp;amp; -(\lambda^\top d + \sum_i u_i^\top b_i)\\&#xA;\mathrm{s.t.} &amp;amp; \sum_i A_i^\top u_i -\lambda_i c_i = 0\ \forall i\\&#xA;&amp;amp;\|u_i\|_2\leq\lambda_i \ \forall i&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a standard technique for recovering the primal variable $x$ from $(\lambda,u)$, assuming the SOCP exhibits strong duality?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;  I am using the notation of &lt;a href=&quot;https://people.eecs.berkeley.edu/~elghaoui/Teaching/EE227A/lecture10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I attempt to plug $\lambda$ back into the Lagrange multiplier expression, everything seems to just cancel out!  I managed to solve the dual of my SOCP using an elegant optimization technique but am struggling to translate to the original primal problem.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastEditorUserId="11716" LastEditDate="2016-11-16T06:50:57.107" LastActivityDate="2016-11-16T06:50:57.107" Title="SOCP: Recovering primal from dual" Tags="|optimization|convex-optimization|constrained-optimization|nonlinear-programming|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25551" PostTypeId="2" ParentId="25547" CreationDate="2016-11-14T20:03:33.100" Score="5" Body="&lt;p&gt;The problem you have is not well-posed. In essence, you are looking for two functions $f_1(x,y),f_2(x,y)$ that should solve only one equation. That's not enough information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you an idea why this can't work, assume that you had found one set of solutions $\vec f(x,y)=(f_1(x,y),f_2(x,y))^T$. Then it is not difficult to verify that that $\tilde {\vec f}(x,y) =(f_1(x,y)+g(y),f_2(x,y)+h(x))^T$ is &lt;em&gt;also&lt;/em&gt; a solution for &lt;em&gt;any&lt;/em&gt; functions $g(y),h(x)$ you may want to choose. With one stroke of a pen, I have therefore already found an infinity of solutions -- too many to do anything useful with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, it is a moot point whether there are numerical methods that can solve your problem: There are fundamental mathematical problems you first need to address, and then we can talk about numerical approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(You may want to object that the boundary conditions require me to choose $g(y)=h(x)=0$. But that only shifts the problem: I can still construct a $\tilde{\vec f}(x,y)=\vec f(x,y) + \vec \varphi(x,y)$ for any $\vec\varphi$ that satisfies $\text{div}\vec\varphi=0$ and $\vec\varphi=0$ on the boundary. There are infinitely many such functions, for example all solutions of the Stokes equation with zero boundary values and arbitrary right hand sides.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-14T20:03:33.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25553" PostTypeId="2" ParentId="25550" CreationDate="2016-11-15T00:13:38.157" Score="3" Body="&lt;p&gt;The standard technique is to solve the optimality conditions as a&#xA;linear system of equations. Let us rewrite your problem into standard&#xA;form&#xA;$$&#xA;\begin{align*}&#xA;\text{maximize } &amp;amp; -c^{T}x\text{ s.t. }\begin{bmatrix}-c_{i}^{T}\\&#xA;-A_{i}&#xA;\end{bmatrix}x+\begin{bmatrix}t_{i}\\&#xA;z_{i}&#xA;\end{bmatrix}=\begin{bmatrix}d_{i}\\&#xA;b_{i}&#xA;\end{bmatrix},\;\begin{bmatrix}t_{i}\\&#xA;z_{i}&#xA;\end{bmatrix}\in\mathrm{SOC},\\&#xA;\text{minimize } &amp;amp; \sum_{i=1}^{m}\begin{bmatrix}d_{i}\\&#xA;b_{i}&#xA;\end{bmatrix}^{T}\begin{bmatrix}\lambda_{i}\\&#xA;u_{i}&#xA;\end{bmatrix}\text{ s.t. }\sum_{i=1}^{m}\begin{bmatrix}-c_{i}^{T}\\&#xA;-A_{i}&#xA;\end{bmatrix}^{T}\begin{bmatrix}\lambda_{i}\\&#xA;u_{i}&#xA;\end{bmatrix}=-c,\;\begin{bmatrix}\lambda_{i}\\&#xA;u_{i}&#xA;\end{bmatrix}\in\mathrm{SOC}.&#xA;\end{align*}&#xA;$$&#xA;The classical optimality conditions are written&#xA;$$&#xA;\begin{align*}&#xA;\sum_{i=1}^{m}\begin{bmatrix}-c_{i}^{T}\\&#xA;-A_{i}&#xA;\end{bmatrix}^{T}\begin{bmatrix}\lambda_{i}\\&#xA;u_{i}&#xA;\end{bmatrix} &amp;amp; =-c,\\&#xA;\begin{bmatrix}-c_{i}^{T}\\&#xA;-A_{i}&#xA;\end{bmatrix}x+\begin{bmatrix}t_{i}\\&#xA;z_{i}&#xA;\end{bmatrix} &amp;amp; =\begin{bmatrix}b_{i}\\&#xA;d_{i}&#xA;\end{bmatrix}\;\forall i\in\{1,\ldots,m\},\\&#xA;\begin{bmatrix}\lambda_{i} &amp;amp; u_{i}^{T}\\&#xA;u_{i} &amp;amp; \lambda_{i}I&#xA;\end{bmatrix}\begin{bmatrix}t_{i}\\&#xA;z_{i}&#xA;\end{bmatrix} &amp;amp; =0\qquad\forall i\in\{1,\ldots,m\},&#xA;\end{align*}&#xA;$$&#xA;see Theorem 16, Alizadeh &amp;amp; Goldfarb, &lt;strong&gt;Second-order cone programming&lt;/strong&gt;,&#xA;&lt;em&gt;Math. Program.&lt;/em&gt;, 2001. Notice that the system of equations&#xA;is &lt;em&gt;linear&lt;/em&gt; over $x,t_{i},z_{i}$ with $\lambda_{i},u_{i}$ fixed. Substituting&#xA;$\lambda_{i}\gets\lambda_{i}^{\star}$ and $u_{i}\gets u_{i}^{\star}$&#xA;and solving the linear equations &#xA;$$&#xA;\begin{bmatrix}\lambda_{i} &amp;amp; u_{i}^{T}\\&#xA;u_{i} &amp;amp; \lambda_{i}I&#xA;\end{bmatrix}\left(\begin{bmatrix}c_{i}^{T}\\&#xA;A_{i}&#xA;\end{bmatrix}x+\begin{bmatrix}b_{i}\\&#xA;d_{i}&#xA;\end{bmatrix}\right)=0 \qquad \forall i\in\{1,\ldots,m\}&#xA;$$&#xA;yields the primal-optimal solution $x^{\star},t_{i}^{\star},z_{i}^{\star}$, assuming that the equations are full-rank or overdetermined (given strong duality, a zero-residual solution must exist). If the equations are underdetermined, then we would have to reimpose the constraint $[t_i^\star, z_i^\star]\in \mathrm{SOC}$ and solve a SOCP feasibility problem.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2016-11-15T22:19:26.400" LastActivityDate="2016-11-15T22:19:26.400" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="25554" PostTypeId="2" ParentId="25547" CreationDate="2016-11-15T04:11:25.290" Score="0" Body="&lt;p&gt;As Wolfgang explained, the issue is not to &lt;strong&gt;find&lt;/strong&gt; a solution, but to &lt;strong&gt;pick one&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If just some solution is fine for you, go ahead and take $f_2\equiv 0$ (for example) and solve $\partial_x f_1(x,y) = a(x,y)$ for every $y$ (which means to integrate up, e.g. $f_1(x,y) = \int_{x_0}^x a(t,y)dt$). But if you are not happy with this solution for some reason, you need to think about what other conditions you have on a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just two examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You may want to look for solutions with minimal $L^2$-norm. This would lead to a quadratic optimization problem with linear equality constraints, namely&#xA;$$&#xA;\min \int f_1(x,y)^2 + f_2(x,y)^2 dxdy\quad\text{s.t.}\quad \operatorname{div} f = a.&#xA;$$&#xA;The optimality system for this question is another system of PDEs…&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You may be interested in a solution that is curl-free. Then make the ansatz $f = \nabla u$ (which implies $\operatorname{curl}f = \operatorname{curl}\nabla u = 0$) and solve&#xA;$$&#xA;\operatorname{div}\nabla u = \Delta u = a&#xA;$$&#xA;which is indeed a Poisson equation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="302" LastActivityDate="2016-11-15T04:11:25.290" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25555" PostTypeId="2" ParentId="25430" CreationDate="2016-11-15T04:39:47.373" Score="1" Body="&lt;p&gt;There are solvers for non linear mixed integer programming that can solve problems to optimality under certain conditions, or provide good quality solutions otherwise. These are based in generalized benders decomposition and outer approximation. You also have heuristics, which pretty much let you do whatever you want. You can try genetic algorithms which are relatively well studied. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now a few suggestions to help you linearize your problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The product of two continuous variables cannot be linearized. So if this is entirely unavoidable to represent your problem, don't worry about linearizing.  Also remember that quadratic programs are not that bad and even non convex MIQP can be solved to global optimality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often you can create new variables to help in the linearization process, as well as express part of the objective function as constraints.&lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2016-11-15T04:39:47.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25556" PostTypeId="1" CreationDate="2016-11-15T05:51:53.250" Score="3" ViewCount="243" Body="&lt;p&gt;I'm trying to benchmark a new numerical method for the incompressible Navier-Stokes equations.  I'd like to use the most widely-accepted test cases possible.  In particular, I'm looking for a reference setup for the 3-dimensional Rayleigh-Taylor instability in a periodic domain.  It would be especially helpful if:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The initial conditions are described precisely&lt;/li&gt;&#xA;&lt;li&gt;The diagnostics (measures of correctness) are described precisely&lt;/li&gt;&#xA;&lt;li&gt;Code and data are available&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If necessary, I will settle for a reference for which code is not available or the data is only available in figures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd prefer single-mode RTI, but a benchmark setup for multi-mode RTI that is widely accepted would be okay.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2016-11-16T12:04:18.923" LastActivityDate="2016-11-16T13:40:41.683" Title="Benchmark setup for incompressible Rayleigh-Taylor flow" Tags="|fluid-dynamics|benchmarking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25557" PostTypeId="2" ParentId="25204" CreationDate="2016-11-15T07:49:39.480" Score="0" Body="&lt;p&gt;A very simple hypothesis is that a problem has to be solved in which the &quot;degree of freedom&quot; of node 1 is &quot;constrained&quot;. Suppose that the degree of freedom at node $i$ is $u_i$ and that the problem to be solved is &#xA;$$K\,u=f,$$&#xA;where matrix $K$ is symmetric and singular, and we now that $f$ belongs to the range of $K$. A common approach in such cases, if the null space of $K$ has dimension 1, is to impose $u_1 = 0$ and simply solve &#xA;$$K_{{2\dots N},{2\dots N}} \; u_{2\dots N} = f_{2\dots N}$$&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2016-11-15T07:49:39.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25558" PostTypeId="2" ParentId="25543" CreationDate="2016-11-15T11:56:18.597" Score="1" Body="&lt;p&gt;If you are content with a numerical solution, then I would suggest using an implicit integration scheme. There are three types of implicit integration schemes, classified by whether or not they require evaluation of the integrand at the endpoints of domain.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lobatto:  integrand is evaluated at both endpoints&lt;/li&gt;&#xA;&lt;li&gt;Radau:  integrand is evaluated at a single endpoint&lt;/li&gt;&#xA;&lt;li&gt;Gauss:  integrand is evaluated at neither endpoint&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The simplest Radau scheme is backwards Euler integration:&#xA;$$x_{k+1} = x_k + (t_{k+1}-t_k) \cdot f(t_{k+1}, x_{k+1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each integration step requires a non-linear solve, but it does not require evaluation of the integrand at the initial point. Note that you will need to &lt;a href=&quot;https://math.stackexchange.com/questions/2015096/how-to-write-a-second-order-dynamical-system-in-first-order-form/2015102#2015102&quot;&gt;put your dynamics into first-order form&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For higher-order schemes, checkout Wolfram MathWorld:&#xA;&lt;a href=&quot;http://mathworld.wolfram.com/RadauQuadrature.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Radau Quadrature&lt;/a&gt;&#xA;&lt;a href=&quot;http://mathworld.wolfram.com/GaussianQuadrature.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian Quadrature&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These will both work well, provided that your problem is only singular at the initial point. If there are singularies at other times in the solution, then you need to do something more sophisticated. If this occurs, then the rough approach is to identify when singularities occur and break the problem into continuous sub-problems, where the singularities only occur on the edges of the domain of each sub-problem.&lt;/p&gt;&#xA;" OwnerUserId="22344" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2016-11-15T13:23:58.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25559" PostTypeId="1" AcceptedAnswerId="25567" CreationDate="2016-11-15T14:34:12.970" Score="1" ViewCount="87" Body="&lt;p&gt;I don't really know where to ask this one...&#xA;In fact, I am not sure I can define it properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here goes...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I take measurements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to &quot;normalize&quot; these measurements, I divide their values by a reference measurement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if my reference measurement has value &lt;code&gt;100&lt;/code&gt;, and one measurement has value &lt;code&gt;150&lt;/code&gt;, I rescale it as follows: &lt;code&gt;150 / 100 = 1.5&lt;/code&gt;. That particular measurement's value is &lt;code&gt;1.5&lt;/code&gt; time the reference value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, time for a picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lf3uv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lf3uv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The triangles are measurement values. &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; are reference values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The domain the measurements are taken in indicates that the closer a measurement value is to &lt;code&gt;A&lt;/code&gt;, the more &lt;code&gt;A&lt;/code&gt; should be the reference value to rescale it. Similarly, the closer a measurement value is to &lt;code&gt;B&lt;/code&gt;, the more B should be the reference value to rescale it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, values &lt;code&gt;1&lt;/code&gt; and &lt;code&gt;2&lt;/code&gt; should be restated in terms of &lt;code&gt;A&lt;/code&gt;, while values &lt;code&gt;4&lt;/code&gt; and &lt;code&gt;5&lt;/code&gt; should be restated in terms of &lt;code&gt;B&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, in general, all values should probably be restated in function of &lt;code&gt;A&lt;/code&gt; &lt;em&gt;and&lt;/em&gt; &lt;code&gt;B&lt;/code&gt;. This is particular true for value &lt;code&gt;3&lt;/code&gt; in the picture, as it is almost equidistant from &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, there must be a function &lt;code&gt;f(x){A, B}&lt;/code&gt;, that will allow to order all measurements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you call that type of problem? Any reference or pointer I could use to restate what I mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update 1:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a metaphor that just came to me...&#xA;Let's say I must define a scale that measure the total cost of moving an object at a certain speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; become two reference &quot;cost per unit of speed&quot;s. For speeds close to &lt;code&gt;B&lt;/code&gt;, the total cost can be accurately estimated in terms of &lt;code&gt;B&lt;/code&gt;. For speeds close to &lt;code&gt;A&lt;/code&gt;, the total cost can be accurately estimated in terms of &lt;code&gt;A&lt;/code&gt;.&#xA;For speeds in between &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt;, the total cost must be a function of &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The farther away a speed is from &lt;code&gt;A&lt;/code&gt;/&lt;code&gt;B&lt;/code&gt;, the less &lt;code&gt;A&lt;/code&gt;/&lt;code&gt;B&lt;/code&gt; is a factor in the cost of moving the object at that speed.&lt;/p&gt;&#xA;" OwnerUserId="22351" LastEditorUserId="17850" LastEditDate="2016-11-18T19:53:04.730" LastActivityDate="2016-11-18T19:53:04.730" Title="Using two reference values for a scalar variable: What's the name of this type of problem?" Tags="|constrained-optimization|interpolation|optimal-control|extrapolation|distance-measure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25560" PostTypeId="1" AcceptedAnswerId="25579" CreationDate="2016-11-15T15:28:21.827" Score="2" ViewCount="135" Body="&lt;p&gt;Suppose that $\Omega\subset\mathbb{R}^2$ and we define the triangulation $\mathcal{T}_h$ where,&#xA;$$\mathcal{T}_h=\bigcup_{i=1}^nK_i$$,&#xA;with $K_i$ are triangles. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have the explicit a posteriori estimator for elliptic problems,&#xA;$$\eta_K(u_h)=h_K\|f+\Delta u_h\|_{L_{2}(K)}+\frac{1}{2}h_K^{1/2}\|[n\cdot\nabla u_h]\|_{L_{2}(\partial K\backslash\partial\Omega)}$$&#xA;Here $[n\cdot\nabla u_h]$ denotes the jump in the normal derivative of $u_h$ on the interior edges of the element $K.$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_h$ is the finite element solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to numerically approximate $[n\cdot\nabla u_h],$ without using Matlab function &lt;a href=&quot;https://www.mathworks.com/help/pde/ug/pdejmps.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdejmps&lt;/a&gt;   &lt;/p&gt;&#xA;" OwnerUserId="22334" LastActivityDate="2016-11-16T22:53:55.103" Title="Numerically Approximate $[n\cdot\nabla u_h],$ without using Matlab function" Tags="|finite-element|matlab|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25561" PostTypeId="1" AcceptedAnswerId="25618" CreationDate="2016-11-15T17:40:49.757" Score="2" ViewCount="518" Body="&lt;p&gt;I'm trying to make a 2D contour plot of a function with a complex input and a real output, namely: $f(z=x+iy)= |z^2-3z|$, on MATLAB. I want to make a contour plot of f, such that the axes are $(x,y)$, i.e. $(\Re(z),\Im(z))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this done in MATLAB? The problem I'm having is the fact the input is complex.  &lt;/p&gt;&#xA;" OwnerUserId="22087" LastEditorUserId="9810" LastEditDate="2016-11-24T19:10:45.370" LastActivityDate="2016-11-24T19:10:45.370" Title="Contour plot of a scalar function over the complex domain in MATLAB" Tags="|matlab|plotting|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25562" PostTypeId="1" CreationDate="2016-11-15T18:28:15.007" Score="1" ViewCount="255" Body="&lt;p&gt;I have a real sparse matrix of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left( \begin{array}{ccc}&#xA;h_{11} &amp;amp; h_{12} &amp;amp; 0 &amp;amp; h_{14} &amp;amp;  &amp;amp;  &amp;amp; \\&#xA;h_{21} &amp;amp; h_{22} &amp;amp; h_{23} &amp;amp; 0 &amp;amp; h_{25} &amp;amp;  &amp;amp; \\&#xA;0 &amp;amp; h_{32} &amp;amp; h_{33} &amp;amp; h_{34} &amp;amp; 0 &amp;amp; h_{36} &amp;amp; \\&#xA;h_{41} &amp;amp; 0 &amp;amp; h_{43} &amp;amp; h_{44} &amp;amp; h_{45} &amp;amp; 0 &amp;amp; \dots \\&#xA; &amp;amp; h_{52} &amp;amp; 0 &amp;amp; h_{54} &amp;amp; h_{55} &amp;amp; h_{56} &amp;amp; \\&#xA; &amp;amp;  &amp;amp; h_{63} &amp;amp; 0 &amp;amp; h_{65} &amp;amp; h_{66} &amp;amp; \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; \vdots  &amp;amp;  &amp;amp;  &amp;amp; \ddots\\&#xA;\end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It typically has dimensions $2500\times 2500$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've written a small routine using the LAPACK routine &lt;code&gt;dsyev&lt;/code&gt;, which is fine for small matrices but takes a while for the above dimensions I am interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anyone suggest a more appropriate routine? I am only interested in the lower eigenvalues, so an iterative procedure would probably be suitable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or are there any (ready-made) routines that exploit sparse matrices? LAPACK seems to be primarily for dense matrices.&lt;/p&gt;&#xA;" OwnerUserId="7808" OwnerDisplayName="ThomasKelly" LastEditorUserId="20688" LastEditDate="2018-04-08T00:40:27.143" LastActivityDate="2018-04-08T00:40:27.143" Title="Finding the lowest $n$ eigenvalues of a band-diagonal Matrix" Tags="|linear-algebra|computational-physics|eigenvalues|software|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25563" PostTypeId="2" ParentId="25556" CreationDate="2016-11-15T22:10:44.377" Score="2" Body="&lt;p&gt;&lt;em&gt;EDIT&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had forgotten about this, but a while ago a friend of mine did his whole thesis work on the Rayleigh-Taylor instability in the context of statistical analysis of turbulence. You can find it enclosed here :&#xA;&lt;a href=&quot;http://www.theses.fr/2011DENS0035&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.theses.fr/2011DENS0035&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thesis is in French, but the work is extremely thorough concerning the RT instability. You might find the quantitative information (growth rate, turbulent fluctuations, etc.) that you are interested in as well as the problem setup. If you need help with some parts, I can help with the translation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;EDIT&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On another note, if you wish to study the order of convergence of your method (if you wish to study stability, ignore what I am about to say) the best approach to test your convergence is to design test cases using the method of manufactured solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference book : &lt;a href=&quot;https://www.amazon.ca/Verification-Validation-Scientific-Computing-Oberkampf/dp/0521113601&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.amazon.ca/Verification-Validation-Scientific-Computing-Oberkampf/dp/0521113601&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of a recent article using this (this one is by me, but there are many others) :&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045793015000675&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0045793015000675&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concretly, the method of manufactured solution allows you to create analytical solutions to the incompressible Navier-Stokes equations that are infinitely continuous, that stimulate every term in the Navier-Stokes equation equally and that may even possibly be unsteady. This allows you to fully assess the order of convergence (or the run time, etc.) on fully complex test cases that are representative of the real applications of your CFD code. Furthermore, they can be combined with any type of boundary condition (Dirichlet, Periodic, Neumann, Robin).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is a good way to benchmark a numerical method and may be even used to evaluate RANS models (with or without wall models). For the latter, see the work of Pr. Pelletier's group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not what you are looking for, then the Rayleigh-Taylor flow can be good example of an unsteady test case. I will try to look-up a reference.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2016-11-16T13:40:41.683" LastActivityDate="2016-11-16T13:40:41.683" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25564" PostTypeId="2" ParentId="25556" CreationDate="2016-11-16T00:29:29.153" Score="2" Body="&lt;p&gt;It's not quite the problem for which you're looking for solutions, but if you're willing to consider something slightly out of the box, there are numerous benchmarks for the R-T instability in the case of the Stokes equation without time derivative -- i.e., the Boussinesq approximation. Rather than list a bunch of papers, I'm simply going to reference the ASPECT manual that reproduces many of them, with appropriate references:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://aspect.dealii.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://aspect.dealii.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Go to Info &gt; Manual.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-16T00:29:29.153" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25565" PostTypeId="1" CreationDate="2016-11-16T04:27:52.180" Score="0" ViewCount="646" Body="&lt;p&gt;Given the following optimization problem: Given positive integers $n_1, n_2, \dots, n_m$ and real numbers $P_1, P_2, \dots P_m$. Besides these, given positive real numbers $a_{i,j,l}$ for $1 \leq i \leq n_l$, $1 \leq l\leq m$ and $0 \leq j \leq k$, and we try to find real numbers $(\beta_1, \beta_2, \dots, \beta_k)$ that minimize&#xA;$$  \sum_{l=1}^m (\sum_{i=1}^{n_l} a_{i,0,l} a_{i,1,l}^{\beta_1} a_{i,2,l}^{\beta_2} \cdots a_{i,k,l}^{\beta_k} -P_l )^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which method you think would be efficient to be implemented in MATLAB/Python?&lt;/p&gt;&#xA;" OwnerUserId="22355" LastEditorUserId="22355" LastEditDate="2016-11-16T04:38:12.113" LastActivityDate="2016-12-16T12:45:46.070" Title="Optimization using Python/MATLAB" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25566" PostTypeId="2" ParentId="25565" CreationDate="2016-11-16T05:45:21.003" Score="0" Body="&lt;p&gt;If you want the global optimum, try using the matbar (under MATLAB) interface to BARON &lt;a href=&quot;http://www.minlp.com/baron&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.minlp.com/baron&lt;/a&gt;.  Or you can use the YALMIP interface (under MATLAB) to call BARON, which might be simpler. But if you do use YALMIP, follow the advice in &lt;a href=&quot;https://groups.google.com/forum/#!topic/yalmip/Z1-oYiknijU&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/forum/#!topic/yalmip/Z1-oYiknijU&lt;/a&gt; to avoid expansion of squares.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2016-11-16T05:45:21.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25567" PostTypeId="2" ParentId="25559" CreationDate="2016-11-16T09:46:38.327" Score="1" Body="&lt;p&gt;Have you thought of &lt;em&gt;barycentric coordinates&lt;/em&gt;? There is a unique way to write $x=\alpha A + \beta B$ with $\alpha+\beta=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Barycentric coordinates are usually employer in larger dimensions, but seem to be the concept you are looking for. You can sort your observations according to either $\alpha$ or $\beta$, this will give you their &quot;closeness&quot; to either. If $A$ and $B$ are well chosen, values should cluster around $[0,1]$, or even around $\{0,1\}$. See e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Barycentric_coordinate_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Barycentric_coordinate_system&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2016-11-16T09:46:38.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25568" PostTypeId="2" ParentId="23250" CreationDate="2016-11-16T10:18:14.310" Score="1" Body="&lt;p&gt;This problem also arises in 2D, when all the vertices of a triangle lie on the boundary. And Stokes problem is not the only problem which may fail with such meshes, a $p$-Laplacian problem for some values of $p$ will find a zero gradient in the element and fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mesh generators generally have an option to handle this, e.g. the 2D mesh generator &lt;code&gt;bamg&lt;/code&gt; of &lt;code&gt;freefem++&lt;/code&gt; has a &lt;code&gt;-splitpbedge&lt;/code&gt; option that adds a node in the middle of any edge having both ends on the boundary. According to &lt;code&gt;bamg&lt;/code&gt; documentation, unstructured mesh generation &lt;em&gt;can&lt;/em&gt; return such triangles.&lt;/p&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2016-11-16T10:18:14.310" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25569" PostTypeId="1" CreationDate="2016-11-16T10:28:17.760" Score="1" ViewCount="366" Body="&lt;p&gt;I was just wondering, in case of computing &lt;code&gt;B=inv(A)&lt;/code&gt;, suppose &lt;code&gt;I&lt;/code&gt; is the identity matrix (diagonal),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After obtaining the factorization &lt;code&gt;A_factorization&lt;/code&gt; by computing &lt;code&gt;getrf(A)&lt;/code&gt;, is it numerically the same to do the following solving steps?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getrs(A_factorization, piv, I)&#xA;getri(A_factorization, piv)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="22380" LastEditDate="2016-11-18T09:01:04.953" LastActivityDate="2016-11-18T17:02:14.387" Title="Is lapack getri numerically the same as getrs with identity matrix as RHS?" Tags="|numerical-analysis|linear-solver|lapack|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25570" PostTypeId="2" ParentId="25569" CreationDate="2016-11-16T11:02:50.310" Score="1" Body="&lt;p&gt;Basically I LAPACK &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/dd/d9a/group__double_g_ecomputational_ga56d9c860ce4ce42ded7f914fdb0683ff.html#ga56d9c860ce4ce42ded7f914fdb0683ff&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; it states for &lt;code&gt;getri&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This method inverts U and then computes inv(A) by solving the system&#xA;   inv(A)*L = inv(U) for inv(A).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems that numerically is a different procedure than solving &lt;code&gt;L*inv(L)=I&lt;/code&gt; and then &lt;code&gt;U*inv(A)=inv(L)&lt;/code&gt;. My understanding is that &lt;code&gt;getri&lt;/code&gt; should be faster than &lt;code&gt;getrs&lt;/code&gt;. Otherwise it would be no reason of creating this, except maybe of saving some memory work space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course result is same in both cases.&lt;/p&gt;&#xA;" OwnerUserId="5098" LastEditorUserId="5098" LastEditDate="2016-11-18T17:02:14.387" LastActivityDate="2016-11-18T17:02:14.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25571" PostTypeId="2" ParentId="25565" CreationDate="2016-11-16T12:01:38.753" Score="1" Body="&lt;p&gt;It looks like your problem is an unconstrained non-linear optimization problem. If you have the &lt;a href=&quot;https://www.mathworks.com/products/optimization/features.html#nonlinear-optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matlab optimization toolbox&lt;/a&gt;, then I would suggest using that to solve your problem. It is easy to try multiple algorithms with the same interface. Just make sure to vectorize your code to have it run quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Matlab global optimization toolbox in would also work, although it would likely take a bit longer to find a solution. Optimization using BARON and / or the YALMIP interface, as suggested by @Mark L. Stone would work as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're looking for a simple way to get an answer in Matlab and do not have the optimization toolbox, then you could try using &lt;a href=&quot;https://www.lri.fr/~hansen/cmaes_inmatlab.html#matlab&quot; rel=&quot;nofollow noreferrer&quot;&gt;CMA-ES&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as python goes, you might look at the list of non-linear solvers that are included in the SciPy package.&lt;/p&gt;&#xA;" OwnerUserId="22344" LastActivityDate="2016-11-16T12:01:38.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25572" PostTypeId="1" CreationDate="2016-11-16T16:19:17.637" Score="-2" ViewCount="249" Body="&lt;p&gt;I've just been introduced to the Poisson's equation. I've never had the need to dealt with PDE, so I'm a bit lost.&lt;/p&gt;&#xA;&lt;p&gt;Apparently we can compute an approximate solution of the Poisson's equation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} u(x, y) = f(x, y)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;by &lt;a href=&quot;https://people.eecs.berkeley.edu/%7Edemmel/cs267/lecture24/lecture24.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;discretizing the 2D Poisson's equation&lt;/a&gt; using finite differences. Here's a picture of that discretization taken from the website I've just linked you to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BFaYi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BFaYi.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;According to that website, we have:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The above linear equation relating &lt;span class=&quot;math-container&quot;&gt;$U(i,j)$&lt;/span&gt; and the value at its neighbors (indicated by the blue stencil) must hold for &lt;span class=&quot;math-container&quot;&gt;$1 &amp;lt;= i,j &amp;lt;= n$&lt;/span&gt;, giving us &lt;span class=&quot;math-container&quot;&gt;$N=n^2$&lt;/span&gt; equations in &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; unknowns.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;where &amp;quot;by above linear equation&amp;quot; I guess they are referring to&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$−4u_{i,j}+u_{i+1,j}+u_{i−1,j}+u_{i,j+1}+u_{i,j−1} = b_{i,j}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$1 \leq i, j \leq n$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When &lt;span class=&quot;math-container&quot;&gt;$(i,j)$&lt;/span&gt; is adjacent to a boundary (&lt;span class=&quot;math-container&quot;&gt;$i=1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$j=1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$i=n$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$j=n$&lt;/span&gt;), one or more of the &lt;span class=&quot;math-container&quot;&gt;$U(i+-1, j+-1)$&lt;/span&gt; values is on the boundary and therefore &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$$b(i,j) = -f(i*h,j*h)*h^2$$&lt;/span&gt; the scaled value of the right-hand-side function &lt;span class=&quot;math-container&quot;&gt;$f(x,y)$&lt;/span&gt; at the corresponding grid point &lt;span class=&quot;math-container&quot;&gt;$(i,j)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h2&gt;Questions&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Where does the &lt;span class=&quot;math-container&quot;&gt;$-4$&lt;/span&gt; in front of &lt;span class=&quot;math-container&quot;&gt;$u_{i,j}$&lt;/span&gt; in the equation above comes from?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What's &lt;span class=&quot;math-container&quot;&gt;$b(i, j)$&lt;/span&gt;? Why is it equal to &lt;span class=&quot;math-container&quot;&gt;$−4u_{i,j}+u_{i+1,j}+u_{i−1,j}+u_{i,j+1}+u_{i,j−1}$&lt;/span&gt;? I mean, I don't understand where does it come from. Usually &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; refers to a right-hand side, but...&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why do we have &lt;span class=&quot;math-container&quot;&gt;$b(i,j) = -f(i*h,j*h)*h^2$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user9925" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-11-17T14:17:56.173" Title="Use Finite Difference Discretization to find approximate solution to the Poisson's equation" Tags="|finite-difference|numerical-analysis|discretization|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25573" PostTypeId="1" CreationDate="2016-11-16T16:25:13.697" Score="1" ViewCount="103" Body="&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;: Is there an efficient out-of-place GPU tensor transpose operation that scales as $O(n)$ for tensors with $n$ total elements, regardless of the rank $d$?  The naive algorithm costs $O(dn)$, since it requires $O(d)$ worth of index manipulation per entry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the current implementation of high rank tensor transpose in TensorFlow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/transpose_functor_gpu.cu.cc#L27&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/transpose_functor_gpu.cu.cc#L27&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does a flat 1-D loop, deconstructs the index using $O(d)$ divisions by strides, and reconstructs the transposed index.  Is there any index trickery / precomputation that would avoid the $O(d)$ cost per entry?&lt;/p&gt;&#xA;" OwnerUserId="447" LastActivityDate="2016-11-16T16:25:13.697" Title="Efficient out-of-place arbitrary rank GPU transpose" Tags="|tensor|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25575" PostTypeId="2" ParentId="25572" CreationDate="2016-11-16T18:04:52.517" Score="1" Body="&lt;p&gt;The answers are all there in the link&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://people.eecs.berkeley.edu/~demmel/cs267/lecture17/lecture17.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://people.eecs.berkeley.edu/~demmel/cs267/lecture17/lecture17.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so this is only a short guide to what you should be looking for. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The derivatives have been discretized as &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial x^2} = \frac{u(i+1,j) - 2u(i,j) + u(i-1, j)}{h^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial y^2} = \frac{u(i,j+1) - 2u(i,j) + u(i, j-1)}{h^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, add the two together and you get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} = \frac{u(i+1,j) - 2u(i,j) + u(i-1, j)}{h^2} + \frac{u(i,j+1) - 2u(i,j) + u(i, j-1)}{h^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rearrange&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} = \frac{u(i+1,j) + u(i-1, j) + u(i,j+1) + u(i, j-1) - 4u(i,j)}{h^2} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives you the -4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{u(i+1,j) + u(i-1, j) + u(i,j+1) + u(i, j-1) - 4u(i,j)}{h^2} = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now just take the $h^2$ to the other side and you get b which is also your third question&lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2016-11-16T18:04:52.517" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25576" PostTypeId="1" CreationDate="2016-11-16T18:57:49.677" Score="2" ViewCount="209" Body="&lt;p&gt;Define $S\in\mathbb{R}^{n\times n}$ as&#xA;$$S:=H+Q^\top V^{-1} Q.$$&#xA;$H,V$ are positive semidefinite.  Here, $H$, $Q$, and $V$ are large, dense matrices but they are &lt;em&gt;structured&lt;/em&gt;:  I can write code for matrix-vector products (e.g. $x\mapsto Hx$) but computing $V^{-1}$ or the map $x\mapsto V^{-1}x$ would require iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Given a sparse, symmetric, positive definite matrix $M$, what is the best way to find the $k$ smallest eigenvalues of $S$ satisfying $Sx=\lambda Mx$?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that this problem is equivalent to the expanded generalized eigenvalue problem&#xA;$$&#xA;\begin{pmatrix}&#xA;H &amp;amp;Q^\top\\Q  &amp;amp; -V&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;x \\ z&#xA;\end{pmatrix}&#xA;=&#xA;\lambda&#xA;\begin{pmatrix}&#xA;M &amp;amp; 0 \\ 0 &amp;amp; 0&#xA;\end{pmatrix}&#xA;\begin{pmatrix}&#xA;x \\ z&#xA;\end{pmatrix},&#xA;$$&#xA;where $z$ is an auxiliary variable.  This is nice because it's symmetric and does not contain $V^{-1}$, so I was hoping to use machinery for generalized eigenvalues.  But the right-hand side of this generalized problem is not invertible, which seems to prevent the use of usual eigenvalue iteration.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2016-11-16T18:57:49.677" Title="Generalized eigenvalue with null space" Tags="|eigenvalues|eigensystem|iterative-method|svd|krylov-method|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25577" PostTypeId="1" AcceptedAnswerId="25584" CreationDate="2016-11-16T20:22:46.763" Score="0" ViewCount="806" Body="&lt;p&gt;I am looking at numerical solutions to the heat equation with Dirichlet and Neumann conditions on the same boundary. That is $u(x,t)$ satisfying&#xA;$$&#xA;  u_t = u_{xx}\,, \quad x \in[0,1]\,, \quad t&amp;gt;0\,,&#xA;$$&#xA;with&#xA;$$&#xA;  \begin{gathered}&#xA;    u(x,0) = u_0(x)\,,\\&#xA;    u(0,t) = g(t) \quad \forall \, t&amp;gt;0\,,\\&#xA;    u_x(0,t) = h(t) \quad \forall \, t&amp;gt;0.&#xA;  \end{gathered}&#xA;$$&#xA;This is with a view to solving an advection-diffusion problem with the same BCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I discretise on $N+1$ points,&#xA;$$&#xA;  U^n_j \approx u(j \Delta x, n \Delta t)&#xA;$$&#xA;with $\Delta x = 1/N$ then I would expect to discretise as&#xA;$$&#xA;  (U_j)_t = \frac{1}{\Delta x^2}\left[ U_{j-1} - 2 U_j + U_{j+1} \right]\,,&#xA;$$&#xA;but then I can't see how to discretise both the boundary conditions. If I let&#xA;$$&#xA;  U_0(t) = g(t)\,,&#xA;$$&#xA;and&#xA;$$&#xA;  \frac{U_1(t) - U_0(t)}{\Delta x} = h(t)&#xA;$$&#xA;then I have an issue determining $(U_N)_t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Resentfully I tried discretising with backwards difference twice&#xA;$$&#xA;  (U_j)_t = \frac{1}{\Delta x^2}\left[ U_{j-2} - 2 U_{j-1} + U_{j} \right]\,,&#xA;$$&#xA;which allows me to impose both boundary conditions on $(U_2)_t$, but I think this is unconditionally unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this even make sense? I was thinking about a shooting method using a shooting parameter $s$ with $u(1,t)=s$ to try to match $u_x(0,t;s)=h(t;s)$ at each time step, but this feels inefficient.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastEditorUserId="17671" LastEditDate="2016-11-16T21:12:22.850" LastActivityDate="2016-11-17T11:38:54.850" Title="Heat equation with Neumann and Dirichlet conditions on same boundary" Tags="|pde|finite-difference|heat-transfer|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25578" PostTypeId="1" CreationDate="2016-11-16T22:42:15.920" Score="0" ViewCount="485" Body="&lt;p&gt;Based on this &lt;a href=&quot;http://www.scholarpedia.org/article/Runge-Kutta_methods#Stability&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;, in particular Figure 1, what is the exact meaning of the plot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my understanding, it implies that for a given differential equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {dy}{dt} = \lambda y $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that the value $\lambda \Delta t$ has to be within the complex region shown in Figure 1 corresponding to which order of Runge-Kutta you used. To ensure I am understanding this plot correctly, must $\lambda \Delta t &amp;lt; 0$ if both $\lambda$ and $\Delta t$ are purely real? If $\Delta t$ is purely positive and real, wouldn't this restrict you to only considering equations where $\lambda &amp;lt; 0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we now consider a case where $\lambda$ is complex-valued and $\Delta t$ purely positive and real, what exactly is the intuition behind the value $\lambda \Delta t = 0.01 + 2i$ being stable (based on Figure 1) yet $\lambda \Delta t = 0.01$ not being stable? Why does removing/decreasing the magnitude of the imaginary part in $\lambda$ induce instability when using a Runge-Kutta method of order 4? &lt;/p&gt;&#xA;" OwnerUserId="20407" LastActivityDate="2016-11-21T18:07:56.817" Title="Runge-Kutta Stability Regions" Tags="|finite-difference|stability|runge-kutta|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25579" PostTypeId="2" ParentId="25560" CreationDate="2016-11-16T22:53:55.103" Score="2" Body="&lt;p&gt;Since you mention that your elements are linear, the gradient of $u_h$ on each cell is constant. As a consequence, if you have an edge $e$ between cells $K$ and $K'$, with $\vec n$ being the normal to the edge pointing from $K$ to $K'$, then&#xA;$$&#xA;  [\vec n \cdot \nabla u_h]&#xA;  =&#xA;  \vec n \cdot \left(\nabla u_h|_{K'} - \nabla u_h|_K\right).&#xA;$$&#xA;Furthermore,&#xA;$$&#xA;  \|[\vec n \cdot \nabla u_h]\|_e^2&#xA;  =&#xA;  \left|\vec n \cdot \left(\nabla u_h|_{K'} - \nabla u_h|_K\right)\right|^2 h_e,&#xA;$$&#xA;where $h_e$ is the length of edge $e$. Equivalent formulas are true for faces in 3d.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have higher order elements, then the situation is not as trivial. Rather, you need to typically compute the integral via quadrature, which requires evaluating the jump term at the quadrature points along the edge. To do this, you need to evaluate both $\nabla u_h|_K$ and $\nabla u_h|_{K'}$ at the quadrature points along the edge.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-16T22:53:55.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25580" PostTypeId="1" CreationDate="2016-11-17T01:13:29.240" Score="1" ViewCount="49" Body="&lt;p&gt;Imagine a bus serving a line with N stations. Each station, $i, i=1,…N$,  has $s_{ij}$ passengers that want to board the bus to go to $j$, $\forall j \neq i$. (one direction). So there are $\sum_j s_{ij}$ passengers waiting at station $i$ to board the bus. Now, suppose that station $m$, is a strategically important  station and we want to make sure there will be enough room on the bus to board waiting passengers. Assume the bus can choose how many people to board at each station, regardless of their destination. We now want to determine how many passengers at each of the stations (prior to station m) should board the train to ensure everyone at station $m$ can board the bus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What type of optimization problem is this? Typical network optimization problems involve maximizing flow or capacity, but not this sort of problem. How would one go about modeling this? Are there any examples of problems similar to this? &lt;/p&gt;&#xA;" OwnerUserId="8109" LastActivityDate="2016-11-17T18:04:26.320" Title="Help formulating/finding the general class of this problem" Tags="|linear-programming|mixed-integer-programming|optimal-control|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25581" PostTypeId="1" AcceptedAnswerId="25588" CreationDate="2016-11-17T01:18:43.287" Score="25" ViewCount="14364" Body="&lt;p&gt;I was just curious as to why high-order (i.e. greater than 4) Runge–Kutta methods are almost never discussed/employed (at least to my knowledge). I understand it requires greater computational time per step (e.g. &lt;a href=&quot;http://sce.uhcl.edu/rungekutta/rk1412.txt&quot; rel=&quot;noreferrer&quot;&gt;RK14 with 12th-order embedded step&lt;/a&gt;), but are there any other downsides of using higher order Runge–Kutta methods (e.g. stability issues)? When applied to equations with highly oscillating solutions on extreme time scales, wouldn't such higher-order methods be typically preferred?&lt;/p&gt;&#xA;" OwnerUserId="20407" LastEditorUserId="7574" LastEditDate="2016-11-17T11:22:50.767" LastActivityDate="2022-07-21T20:06:50.873" Title="Why are higher-order Runge–Kutta methods not used more often?" Tags="|ode|runge-kutta|" AnswerCount="5" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25584" PostTypeId="2" ParentId="25577" CreationDate="2016-11-17T11:38:54.850" Score="2" Body="&lt;p&gt;In the comments Christian directed me towards &lt;em&gt;lateral Cauchy problems&lt;/em&gt; and the fact that this is a textbook example of an ill-posed problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following this lead, I found that this is more specifically know as the &lt;em&gt;sideways heat equation&lt;/em&gt;. As an alternative to the suggested &lt;em&gt;quasireversibility method&lt;/em&gt; (again Christian), there is a proposed sequential solution in &lt;a href=&quot;http://dx.doi.org/10.1080/1068276021000048564&quot; rel=&quot;nofollow noreferrer&quot;&gt;Berntsson (2003)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Berntsson (2003)] Fredrik Berntsson , Sequential solution of the sideways heat equation by windowing of the data, 2003, Inverse Problems in Engineering, (11), 2, 91-103.&lt;/p&gt;&#xA;" OwnerUserId="17671" LastActivityDate="2016-11-17T11:38:54.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25586" PostTypeId="1" CreationDate="2016-11-17T14:34:12.123" Score="0" ViewCount="160" Body="&lt;p&gt;For a 1D grid I can calculate a Lagrange polynomial through an arbitrary set of points for the reconstruction of a polynomial function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 2D I have an unstructured grid and want to interpolate the value on one node from the values on neighboring nodes. A difference from the 1D case is that a well-defined Lagrange polynomial has fixed stencil sizes, like 3 points for a 1&lt;sup&gt;st&lt;/sup&gt; order reconstruction, 6 for a 2&lt;sup&gt;nd&lt;/sup&gt; order reconstruction and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best approach for a reconstruction from the neighboring points?&#xA;My application is a reconstruction inside a finite volumes scheme with Voronoi cells, which unlike triangles have an arbitrary (but in many meshes roughly constant) number of neighbors.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="22380" LastEditDate="2016-11-18T11:17:23.287" LastActivityDate="2016-11-18T12:29:14.727" Title="Polynomial reconstruction on unstructured grids" Tags="|finite-volume|interpolation|voronoi-diagrams|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25587" PostTypeId="2" ParentId="10938" CreationDate="2016-11-17T14:40:23.633" Score="1" Body="&lt;p&gt;Your proposed corner shape function is the same as for the Q9 element. So your&#xA;question is why isn't it allowed to simply &quot;resuse&quot; these shape functions for the Q8?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the several criteria for shape functions is that at any point $\xi$ and $\eta$ in the element, the sum of all shape functions evaluated at that point must equal one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the center point, $\xi=0, \eta=0$. The Q9 corner point shape functions are all equal zero there. The Q8 edge shape functions (e.g. your first equation) all equal $1/2$ so their sum is two, violating this criterion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The shape functions for the Q8 element can be found in many FE texts, of course.&#xA;I find this set of notes by Felippa (&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch18.d/IFEM.Ch18.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shape Function Magic&lt;/a&gt;) that describes a&#xA;systematic process for deriving shape functions in general and those for the Q8,&#xA;in particular, to be very readable.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2016-11-18T12:38:00.123" LastActivityDate="2016-11-18T12:38:00.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25588" PostTypeId="2" ParentId="25581" CreationDate="2016-11-17T14:48:01.897" Score="22" Body="&lt;p&gt;There are thousands of papers and hundreds of codes out there using Runge-Kutta methods of fifth order or higher.  Note that &lt;strong&gt;the most commonly used explicit integrator in MATLAB&lt;/strong&gt; is ODE45, which advances the solution using a 5th-order Runge-Kutta method.&lt;/p&gt;&#xA;&lt;h1&gt;Examples of widely-used high-order Runge-Kutta methods&lt;/h1&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0771050X80900133&quot; rel=&quot;noreferrer&quot;&gt;paper of Dormand &amp;amp; Prince giving a 5th-order method&lt;/a&gt; has over 1700 citations &lt;a href=&quot;https://scholar.google.com/scholar?cluster=9166409146951279414&amp;amp;hl=en&amp;amp;as_sdt=0,5&quot; rel=&quot;noreferrer&quot;&gt;according to Google Scholar&lt;/a&gt;.  Most of those are papers using their method to solve some problem.  &lt;a href=&quot;https://scholar.google.com/scholar?cluster=1105973012203467107&amp;amp;hl=en&amp;amp;as_sdt=0,5&quot; rel=&quot;noreferrer&quot;&gt;The Cash-Karp method paper has over 400 citations&lt;/a&gt;.  Perhaps the most widely-used method of order higher than 5 is the 8th-order method of Prince-Dormand &lt;a href=&quot;https://scholar.google.com/scholar?cluster=1105973012203467107&amp;amp;hl=en&amp;amp;as_sdt=0,5&quot; rel=&quot;noreferrer&quot;&gt;which has over 400 citations on Google Scholar&lt;/a&gt;.  I could give many other examples; and keep in mind that many (if not most) of the people using these methods never cite the papers.&lt;/p&gt;&#xA;&lt;p&gt;Note also that &lt;a href=&quot;http://www.davidketcheson.info/assets/papers/2014_hork.pdf&quot; rel=&quot;noreferrer&quot;&gt;high-order extrapolation and deferred correction methods are Runge-Kutta methods&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h1&gt;High-order methods and rounding error&lt;/h1&gt;&#xA;&lt;p&gt;If your accuracy is limited by rounding errors then &lt;strong&gt;you should use a higher-order method&lt;/strong&gt;.  This is because higher-order methods require fewer steps (and fewer function evaluations, even though there are more evaluations per step), so they commit fewer rounding errors.  You can easily verify this yourself with simple experiments; it is a good homework problem for a first course in numerical analysis.&lt;/p&gt;&#xA;&lt;p&gt;Tenth-order methods are extremely useful in double-precision arithmetic.  On the contrary, if all we had was Euler's method, then rounding error would be a major issue and we would need very high-precision floating point numbers for many problems where high-order solvers do just fine.&lt;/p&gt;&#xA;&lt;h1&gt;High order methods can be just as stable&lt;/h1&gt;&#xA;&lt;p&gt;@RichardZhang has referenced the second Dahlquist barrier, but that applies only to multistep methods.  The question posted here is about Runge-Kutta methods, and &lt;strong&gt;there are Runge-Kutta methods of every order that are not only &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;-stable, but also &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;-stable&lt;/strong&gt; (a stability property useful for some nonlinear problems).  To learn about these methods, see for instance the text of Hairer &amp;amp; Wanner.&lt;/p&gt;&#xA;&lt;h1&gt;High-order methods in celestial mechanics&lt;/h1&gt;&#xA;&lt;p&gt;You ask&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When applied to equations with highly oscillating solutions on extreme time scales, wouldn't such higher-order methods be typically preferred?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You're exactly right!  A prime example of this is celestial mechanics.  I'm not an expert in that area.  But &lt;a href=&quot;http://link.springer.com/article/10.1007/BF00049361&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt;, for instance, compares methods for celestial mechanics and doesn't even consider order lower than 5.  It concludes that methods of order 11 or 12 are often the most efficient (with the Prince-Dormand method of order 8 also often very efficient).&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-05-07T18:02:19.520" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="25589" PostTypeId="2" ParentId="25586" CreationDate="2016-11-17T15:59:00.220" Score="2" Body="&lt;p&gt;You could use a Least Square fit approach to fit some polynomial via neighboring nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could even make the Least Square fit weighted based on distances (potentially passed through something like a Gaussian function) from neighboring nodes to the location you want to evaluate the polynomial at. The latter approach, which can be viewed as local regression, can in turn tailor a local model about the point you want to evaluate, potentially leading to a superior result.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-11-17T15:59:00.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25590" PostTypeId="2" ParentId="25581" CreationDate="2016-11-17T17:39:58.193" Score="13" Body="&lt;p&gt;As long as you're using standard double precision floating point arithmetic, very high order methods aren't needed to get a solution with high accuracy in a reasonable number of steps.  In practice I find that the accuracy of the solution is normally limited to a relative error of 1.0e-16 by the double precision floating point representation rather than the number/length of the steps that are taken with RKF45.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you switch to some higher than double precision floating point arithmetic scheme, then using a 10th order method might well be worth while.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2016-11-18T05:47:07.107" LastActivityDate="2016-11-18T05:47:07.107" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25592" PostTypeId="2" ParentId="25586" CreationDate="2016-11-17T20:03:34.737" Score="1" Body="&lt;p&gt;It is not polynomial, but you may be interested in the &lt;strong&gt;Natural Neighbors interpolation&lt;/strong&gt; (it fits well with Voronoi diagram). To evaluate the interpolant at a given point, insert the point into the diagram, and compute the volumes of the intersections between the new Vornoi cell (of the added point) and the Voronoi cells in the diagram before inserting the points.This gives the interpolation weights. The interpolant is $C^2$ almost everywhere (on the points it is $C^1$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_neighbor&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Natural_neighbor&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2016-11-17T20:03:34.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25593" PostTypeId="2" ParentId="8591" CreationDate="2016-11-17T22:58:15.300" Score="3" Body="&lt;p&gt;This response is a few years late, but I feel those questions are still relevant today. In the recent years new applications of DEC appeared in fields such as computer graphics, geometry processing, Navier-Stokes equations and Darcy flow. In the introduction of the paper suggested below, you will find a quick overview of the fields (including linear elasticity, electrodynamics and variational integrators) in which DEC has been used (some of the authors cited have been quite active in the DEC literature).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As timur said in an answer on the mathoverflow blog, convergence can be obtained in special cases by relating DEC with other methods known to converge. However, serious attempts at developing a general framework to tackle convergence problems were undertaken. Recently, we proved convergence of the DEC solutions for the Poisson problem (on functions, i.e. 0-forms) in arbitrary dimension in the discrete L2-norm. Many problems and questions related to the asymptotic behaviours of the discrete solutions in other norms remain open, but the following is a welcomed step toward a better understanding of the theory: &lt;a href=&quot;https://arxiv.org/abs/1611.03955&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1611.03955&lt;/a&gt; (Convergence of Discrete Exterior Calculus Approximations for Poisson Problems, Erick Schulz and Gantumur Tsogtgerel, 2016).&lt;/p&gt;&#xA;" OwnerUserId="22377" LastEditorUserId="22377" LastEditDate="2016-11-23T21:02:08.560" LastActivityDate="2016-11-23T21:02:08.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25595" PostTypeId="2" ParentId="8591" CreationDate="2016-11-18T08:22:27.740" Score="3" Body="&lt;p&gt;Discrete Exterior Calculus (DEC) has pros and cons:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Ease of &quot;use&quot;&lt;/em&gt; For a student, it is quite easy to assemble a discretization and a solver for a simple PDE, e.g. Laplace/Poisson on curved surface (Laplace Beltrami). It made the method very popular in Computer Graphics, after the Caltech geometry lab. made some courses at the main graphics conference (SIGGRAPH), see reference in other answer. It is especially the case in Computer Graphics, where students know matrices well, but are less familiar with integrals. Using DEC, they can &quot;play lego&quot; and solve simple PDEs without suffering too much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Making some computations simpler&lt;/em&gt; DEC is an emanation of EC (Exterior Calculus, invented by Elie Cartan from 1899 to 1945). Central in EC, there is the notion of forms (&quot;things to be integrated&quot;) and chains (&quot;integration domains&quot;), and a duality between them. Several theorems (Stokes, Green-Gauss, Ostrogradsky, and the fundamental theorem of analysis) are a special case of this duality. Not only this is elegant, but also this makes computations simpler in some cases (like in electromagnetics), and avoids referring to the parameterization of the objects in many cases (e.g. when manipulating vector fields over surfaces, or in the curved space-time setting of relativity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Exhibiting untrivial degrees of freedom&lt;/em&gt; By explaining the relation between forms (&quot;things to be integrated&quot;) and chains (&quot;integration domains&quot;), EC can exhibit untrivial parameterization of objects, such as vector fields on surfaces, and explain the relations between the topology of the vector field and the underlying surface (homology: topology of curves traced on the surface, co-homology: topology of vector fields), see [1] for a deep study. We used it in [2,3] to study the topological degrees of freedom of discrete vector fields. An impressive example of the power of this type of reasoning is Gortler et.al's proof of Tutte's planar embedding theorem [4]. Previous proofs of this theorem (by Tutte, and later by Colin de Verdière) require a certain background on graph theory to be understood. The proof by Gortler et. al is much more accessible, it introduces some discrete differential forms defined on the edges of the graph, and then it obtains the result by an easy-to-understand counting argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplified version of DEC promoted by the Caltech geometry lab. hides many details under the hood. While it is OK for deriving Laplace and Poisson equations on both the Euclidean and curved settings, some problems are quickly encountered when discretizing more complicated equations, because it does not incitate to ask questions concerning the convergence to the continuous setting and/or the properties that are preserved by the discretization (identities with div/grad/curl, referred to as the Hodge complex, studied by mathematicians such as Jenny Harisson and Robert Kotiuga). The way it is used in Computer Graphics (mainly for the Laplace equation) does not bring more in most cases than the classical P1 FEM Laplacian. I tend to prefer the classical P1 FEM Laplacian, because it both gives you the formula of the discretization and explains you &lt;strong&gt;why&lt;/strong&gt;. Another aspect is the way the form of DEC promoted by Caltech mixes the Hodge star and the inner product. While it makes it easy to assemble a discrete Laplacian as a single matrix, it does not tell how to project a function onto the so-defined function space, and makes it difficult also to understand how orthogonality acts: you obtain a matrix equal to $B^{-1}A$ where $B$ is the condensed matrix of the inner product and $A$ the stiffness matrix, and you no longer see that the eigenvectors are orthogonal with respect to the inner product discretized in $B$, because you don't see $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion/Summary:&lt;/strong&gt; EC and DEC is a powerful theory for studying complicated problems (electromagnetics, vector fields on surfaces of arbitrary topology). The way it is used in Computer Graphics makes it simple for students who do not know integrals to do simple things. For simple things I tend to prefer the classical FEM formulation, where the complete deduction path is easier to follow from the theory down to the discretization together with the theoretical guarantees. For complicated things it can be very elegant and efficient (provided that all the reasoning path is preserved rather than just &quot;playing lego&quot; with some forms of the discrete Hodge star and the discrete exterior derivative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Douglas Arnold, Finite Element Exterior Calculus, 2006&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] N-Symmetry direction field design, ACM Trans. Graph., 2008&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Geometry-aware direction field processing, ACM Trans. Graph., 2009&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] An elementary proof of Tutte's Planar Embedding Theorem, Gortler, Gotsmann, Thurston, 2006, Computer Aided Geometric Design&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2016-11-24T13:11:32.390" LastActivityDate="2016-11-24T13:11:32.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25596" PostTypeId="5" CreationDate="2016-11-18T09:05:56.147" Score="0" Body="&lt;p&gt;Much like &lt;a href=&quot;/questions/tagged/interpolation&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;interpolation&amp;#39;&quot; rel=&quot;tag&quot;&gt;interpolation&lt;/a&gt;, which produces estimates &lt;strong&gt;between&lt;/strong&gt; known observations, extrapolation produces estimates &lt;strong&gt;beyond&lt;/strong&gt; the original observation range.&lt;/p&gt;&#xA;" OwnerUserId="22380" LastEditorUserId="22380" LastEditDate="2016-11-18T19:53:13.497" LastActivityDate="2016-11-18T19:53:13.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25597" PostTypeId="4" CreationDate="2016-11-18T09:05:56.147" Score="0" Body="Extrapolation is a method of constructing new data points outside the range of a discrete set of known data points." OwnerUserId="22380" LastEditorUserId="22380" LastEditDate="2016-11-18T19:52:49.270" LastActivityDate="2016-11-18T19:52:49.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25598" PostTypeId="2" ParentId="25586" CreationDate="2016-11-18T12:29:14.727" Score="0" Body="&lt;p&gt;For estimates on data points within 1D, 2D or 3D unstructured grids &lt;a href=&quot;https://en.wikipedia.org/wiki/Kriging&quot; rel=&quot;nofollow noreferrer&quot;&gt;kriging&lt;/a&gt; is a well established technique. It's a statistical method (Bayesian inference done on the function space) to estimate the value of the unknown given the data points. Smoothness can be controlled by the width of the underlying Gaussian process.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2016-11-18T12:29:14.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25599" PostTypeId="2" ParentId="25309" CreationDate="2016-11-18T12:55:57.533" Score="0" Body="&lt;p&gt;For the fluid-&gt;particle problem, @AbhilashReddyM gave you the answer, the boundary condition will give you the value of $v_x$ at the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the particle-&gt;fluid problem, yes, it is &lt;em&gt;a&lt;/em&gt; boundary condition that you need to use, but not necessarily the one of your fluid problem: e.g. if particles are allowed to slide/roll against the boundary while the fluid obeys no-slip condition, then you have a different BC for each.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that for clarity, you should have asked questions separately.&lt;/p&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2016-11-18T12:55:57.533" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="25600" PostTypeId="1" CreationDate="2016-11-18T14:31:34.517" Score="2" ViewCount="411" Body="&lt;p&gt;I'm trying to solve the following integral numerically.&#xA;$$H(y) = \int dx \, f(x) \, \delta(g(x,y)).$$&#xA;For this I chose a representation of the delta-function and employ&#xA;convergence with respect to $\eta$&#xA;$$\sum_\limits{x_i} \, f(x_i) \, \frac{\eta}{g(x_i,y)^2+\eta^2}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I calculate this for every $y$ such that I can plot $H(y)$ in the end.&#xA;The problem is, that I would expect for $\eta \rightarrow 0$ the best results, but there seems to be a line (approx. $\eta=10^{-4}$) upon which I get a bunch of delta-like peaks in $H(y)$ instead of a smooth curve (for e.g. $\eta=10^{-2}$) which I want and would expect.&lt;/p&gt;&#xA;" OwnerUserId="21813" LastEditorUserId="713" LastEditDate="2016-11-18T16:48:38.113" LastActivityDate="2016-11-18T16:48:38.113" Title="Numeric integration over Dirac delta" Tags="|numerical-analysis|convergence|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25601" PostTypeId="1" CreationDate="2016-11-18T15:40:06.413" Score="2" ViewCount="130" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; be two rigid transformations in 3D space that transform things from global to local coordinates. Let their relative transformation be expressed by &lt;span class=&quot;math-container&quot;&gt;$W=AB^{-1}$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; can also be expressed by an element of &lt;a href=&quot;https://en.wikipedia.org/wiki/Euclidean_group&quot; rel=&quot;nofollow noreferrer&quot;&gt;SE(3)&lt;/a&gt;: &lt;span class=&quot;math-container&quot;&gt;$\omega=\ln(W)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\ln()$&lt;/span&gt; is defined to be the inverse of the exponential map.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to know the closed form solutions for computing the 6x6 Jacobians&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\dfrac{\partial\omega}{\partial\alpha}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\dfrac{\partial\omega}{\partial\beta}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; are the six degrees of freedom of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; respectively. In plain English: How does &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; change when I wiggle &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;? It might be that there's a very simple solution to this and I'm just not seeing it.&lt;/p&gt;&#xA;" OwnerUserId="740" LastEditorUserId="23791" LastEditDate="2022-02-25T20:17:22.353" LastActivityDate="2022-02-25T20:17:22.353" Title="Closed-form Jacobian of SE(3) element with 6-degrees of freedom" Tags="|linear-algebra|matrix|computational-geometry|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25602" PostTypeId="1" AcceptedAnswerId="25613" CreationDate="2016-11-18T16:37:34.547" Score="2" ViewCount="756" Body="&lt;p&gt;I have a legacy DG finite element code for which I would like to write some unit tests. As it is written, for 2D problems, each degree of freedom has associated with it a Jacobian matrix (for later transformation to the master element). All elements are straight-sided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good way of sanity checking that these Jacobians are correct? The previous author used some nasty meta-programming to generate a file thousands of lines long which is parsed by python and used to compute the Jacobians. Therefore I prefer to treat it as a black box and test the output.&lt;/p&gt;&#xA;" OwnerUserId="20148" LastActivityDate="2016-11-20T11:59:59.963" Title="Sanity checking jacobians for Finite Element code" Tags="|finite-element|jacobian|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25604" PostTypeId="2" ParentId="25602" CreationDate="2016-11-18T19:31:53.390" Score="5" Body="&lt;p&gt;Well, the first obvious test is to check if the determinant of the Jacobian matrix is positive. If it not, it means that your element has inverted and your answer is going to be invalid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, I am not sure of any other test that would be good... For a unit test you could rotate a triangle 360 degrees by increment of 1 degree and measure that the jacobian is always of determinant 1... &lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2016-11-18T19:38:50.547" LastActivityDate="2016-11-18T19:38:50.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25608" PostTypeId="1" CreationDate="2016-11-19T12:31:10.143" Score="0" ViewCount="268" Body="&lt;p&gt;How can a student with zero computing or programming language knowledge, few engineering mathematics knowledge, understand computational science especially Finite Element Modelling (FEM) from engineering analysis point of view. I found out that there are so many branches or approaches to FEM (computational science) such as linear FEM, advanced FEM, Non-linear FEM, codes and algorithm etc. that are brought together under most good books and therefore render such books 'useless' or incomprehensible to a child. Some even approach problems from mathematical perspective. In fact you are happy as you begin reading the beginning chapters of some books but such happiness is often short-lived and begin to wonder when will the circle end in a good understanding and grasp of FEM.&#xA;My question is who can really 'explain to me like I am five' the step by step action(s) I need to take to master FEM from A to Z (assuming one has a project in mind that is related to the modelling and simulation of thermo-mechanical stresses in an Internal Combustion Engine (I.C.E.) piston using a particulate material).&lt;/p&gt;&#xA;" OwnerUserId="20572" LastEditorUserId="20572" LastEditDate="2016-11-19T14:28:12.357" LastActivityDate="2016-11-19T15:23:39.650" Title="How can an engineering student become a computational scinece expert in a short time" Tags="|finite-element|numerical-analysis|fluid-dynamics|boundary-conditions|computational-physics|" AnswerCount="1" CommentCount="5" ClosedDate="2016-11-21T16:03:10.160" ContentLicense="CC BY-SA 3.0" />
  <row Id="25609" PostTypeId="2" ParentId="25510" CreationDate="2016-11-19T13:49:14.867" Score="1" Body="&lt;p&gt;The algorithm in question does not impose any constraints on the graphs it can be applied to, so it plainly does not care how whether your graph is directed or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when applying the algorithm to directed graph, you imply a definition of dimension (which also depend on how exactly you adapt it to directed graphs). The more crucial question is whether this algorithm makes any sense for your application. Unfortunately, only you can answer that question.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2016-11-19T13:49:14.867" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25610" PostTypeId="2" ParentId="25608" CreationDate="2016-11-19T15:23:39.650" Score="7" Body="&lt;p&gt;There is no shortcut. Just like there is no shortcut to becoming an &quot;engineering expert in a short time&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that to be an expert in civil engineering, you need to understand load analysis, use cases of buildings and bridges, materials, designs, and regulatory issues. For computational science, you need to understand the mathematical background, programming, some computer software and hardware design, nonlinear and linear solvers, and a few other things. If it was easy, &lt;em&gt;everyone&lt;/em&gt; would be an expert. But it isn't. You can't teach finite element methods to students &quot;as if they were five&quot;. The closest you can come is if you used commercial codes that hide a lot of the material from you, but even then you will be more productive and accurate if you understand what you are doing.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-19T15:23:39.650" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25612" PostTypeId="1" CreationDate="2016-11-20T08:21:17.977" Score="2" ViewCount="193" Body="&lt;p&gt;Since the computational complexity of direct elimilation methods for solving linear systems is $O(n^3)$, it's not practical when the number of dofs is large. But how large would you call it a &lt;strong&gt;large system&lt;/strong&gt;?  I saw some material saying that 100,000 dofs would be the watershed when you should consider iterative methods. Wouldn't that be too large? I mean when dofs reach just less than 100,000, the complexity would be $O(100,000^3)$ with direct methods, it's hard to imagine that we can afford such a huge comlexity? Could anyone give some concrete examples or intuitive sugestions indicating the usual upper bound when direct elimilation methods are treated as impractical? Literatures would also be fine. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="18224" LastActivityDate="2016-11-20T17:10:01.710" Title="Regarding impractical usage of direct solvers of linear systems" Tags="|linear-solver|iterative-method|complexity|" AnswerCount="1" CommentCount="11" ClosedDate="2016-11-21T16:06:43.953" ContentLicense="CC BY-SA 3.0" />
  <row Id="25613" PostTypeId="2" ParentId="25602" CreationDate="2016-11-20T11:59:59.963" Score="7" Body="&lt;p&gt;Like with any other thing you want to test, you need to come up with a list of situations where you know that something is true, and then check this. In many cases, knowing that something is true does not require being able to actually compute the numerically correct answer, which simplifies things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As @BlaisB suggestes, if you rotate the reference cell, the determinant of the Jacobian needs to be one. You don't need to know the exact Jacobian matrix.&lt;/li&gt;&#xA;&lt;li&gt;The same is true if you &lt;em&gt;translate&lt;/em&gt; the cell.&lt;/li&gt;&#xA;&lt;li&gt;If you scale the cell by a factor, the Jacobian needs to be a multiple of the identity matrix. The multiplication factor is simply the scaling factor.&lt;/li&gt;&#xA;&lt;li&gt;If you randomly generate cells that are (i) convex, and (ii) whose vertices are oriented in the correct way, the determinant needs to have a positive determinant.&lt;/li&gt;&#xA;&lt;li&gt;If you evaluate the determinant at multiple points of an affine triangle, it needs to be the same everywhere.&lt;/li&gt;&#xA;&lt;li&gt;If you compute $\int_T 1 \; dx$ via quadrature, then the result of the quadrature needs to be equal to the area/volume of the cell. Because the area is something that's easy to compute, it's easy to assess that the answer you get is correct up to round-off.&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of these make for good, individual testcases that are relatively simple to write. Within an hour, I could probably come up with a dozen or more such testcases.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-20T11:59:59.963" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25615" PostTypeId="2" ParentId="25612" CreationDate="2016-11-20T17:10:01.710" Score="5" Body="&lt;p&gt;The &quot;100,000 unknowns&quot; rule-of-thumb applies to sparse matrices rather than dense ones. A naive direct solver, which doesn't take advantage of sparsity at all, could in principle have $O(n^3)$ complexity for some matrices. In practice, a good direct solver will have much lower computational complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better starting point would be to look at banded matrices. Factoring a matrix of bandwidth $d$ requires $O(d^2n)$ time, and the factors have bandwidth $d$ also. If you can reorder a sparse matrix so that it has small bandwidth, then you can make direct factorization much more manageable. Unfortunately, computing the optimal reordering is NP-complete. Fortunately, there are some good heuristics, i.e. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cuthill%E2%80%93McKee_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cuthill-McKee algorithm&lt;/a&gt;, for reducing the bandwidth of a graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now most FEM matrices, especially in 3D, actually have pretty high bandwidth. Rather than take this indirect approach of reducing fill-in by reducing bandwidth, you could ask which ordering, among the $n!$ possibilities, gives the least fill-in for a direct matrix factorization. This problem is also NP-complete, but there are also some good heuristics. These heuristics are the basis for approximate minimum degree orderings. A good reference is &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/S0895479894278952&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amestoy's original paper&lt;/a&gt; on AMD ordering and &lt;a href=&quot;https://books.google.com/books?isbn=0898716136&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tim Davis's book&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Bill Greene suggests, you should try it for yourself as well. &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.sparse.linalg.splu.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy&lt;/a&gt; has a sparse LU factorization routine where you can specify to either leave the matrix in its natural ordering, or to use AMD. You can always try it out on the &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/matrices/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SuiteSparse matrix collection&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2016-11-20T17:10:01.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25616" PostTypeId="1" CreationDate="2016-11-21T07:18:42.063" Score="1" ViewCount="950" Body="&lt;p&gt;I implemented a finite difference scheme to solve Poisson's equation in a 2D grid in C. I solve the system by using Jacobi iteration. Everything works fine until I use a while loop to check whether it is time to stop iterating or not (with for loops is easy). On the notes I am following there is written that I have to compute the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta = max ||uNew - u||,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $1 &amp;lt;= i,j &amp;lt;= n$. With $uNew$ being the current solution and $u$ the previous iteration. Obviously they are 2D arrays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Iterations&#xA;    double delta=(tau+1), temp_delta;&#xA;    #ifdef _OPENMP&#xA;            wt1=omp_get_wtime();&#xA;    #endif&#xA;    do {&#xA;&#xA;            for(i=1;i&amp;lt;y-1;i++) {&#xA;                    for(j=1;j&amp;lt;x-1;j++) {&#xA;                            uNew[i*x+j] = 0.25 * (u[(i-1)*x+j] + u[i*x+(j+1)] + u[i*x+(j-1)] + u[(i+1)*x+j] - dx*dy*func(i,j,dx,dy));&#xA;                    }&#xA;            }&#xA;&#xA;            // Boundary conditions using g(x,y)&#xA;            for(j=0;j&amp;lt;x;j++) {&#xA;                    uNew[j] = gunc(0,j,dx,dy);&#xA;                    uNew[(y-1)*x+j] = gunc(y-1,j,dx,dy);&#xA;            }&#xA;            for(i=0;i&amp;lt;y;i++) {&#xA;                    uNew[i*x] = gunc(i,0,dx,dy);&#xA;                    uNew[i*x+(x-1)] = gunc(i,x-1,dx,dy);&#xA;            }&#xA;&#xA;            // Check if to terminate Jacobi iteration&#xA;            for(i=1;i&amp;lt;y-1;i++) {&#xA;                    for(j=1;j&amp;lt;x-1;j++) {&#xA;                            temp_delta = abs(uNew[i*x+j]-u[i*x+j]);&#xA;                            printf(&quot;%f &quot;, temp_delta);&#xA;                            if (delta &amp;lt;= temp_delta) {&#xA;                                    delta = temp_delta;&#xA;                            }&#xA;                    }&#xA;            }&#xA;&#xA;            // Update solution&#xA;            for(i=0;i&amp;lt;y;i++) {&#xA;                    for(j=0;j&amp;lt;x;j++) {&#xA;                            u[i*x+j] = uNew[i*x+j];&#xA;                    }&#xA;            }&#xA;    } while(delta &amp;gt; tau);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $\tau$ is the tolerance, $\delta$ is the result of the above formula, $temp\_delta$ is used to find the maximum and $uNew$ and $u$ are just the matrices containing the solutions at the grid points. The problem is that it's not working. Can somebody give me a hint, please?&lt;/p&gt;&#xA;" OwnerUserId="22185" LastActivityDate="2016-11-21T10:22:38.917" Title="Jacobi iteration for finite difference: when to stop?" Tags="|finite-difference|iterative-method|c|poisson|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25617" PostTypeId="2" ParentId="25616" CreationDate="2016-11-21T10:22:38.917" Score="4" Body="&lt;p&gt;For solving systems you shouldn't be comparing the results between each iteration  but rather computing the residual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you consider the matrix representation to be in the standard form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A\cdot y = b $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can define the residual at some iteration ($y_{i}$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ r =b -  A\cdot y_i  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you 'just' need to determine a stopping method based on the residual vector, some choices are &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\delta &amp;lt; max(|r|)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\delta &amp;lt; sum(|r|)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Residual_(numerical_analysis)&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Residual&quot;&gt;Wikipedia article on the residual&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20410" LastActivityDate="2016-11-21T10:22:38.917" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25618" PostTypeId="2" ParentId="25561" CreationDate="2016-11-21T13:50:06.790" Score="1" Body="&lt;p&gt;You need to define a mesh over which your function will be evaluated&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[mfRe, mfIm] = ndgrid(-1, 1, 100);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then evaluate your function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fh = @(z)abs(z.^2 - 3 .* z);&#xA;mfF = fh(complex(mfRe, mfIm));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now make the contour plot and beautify&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contour(mfRe, mfIm, mfF);&#xA;axis equal;&#xA;xlabel('Re(z)');&#xA;ylabel('Im(z)');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u21yD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u21yD.png&quot; alt=&quot;Contour plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9810" LastActivityDate="2016-11-21T13:50:06.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25619" PostTypeId="2" ParentId="25581" CreationDate="2016-11-21T17:45:09.117" Score="18" Body="&lt;h2&gt;The Benchmark Setup&lt;/h2&gt;&#xA;&lt;p&gt;In the Julia software &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt; we implemented plenty of higher order methods, including the Feagin methods. You can see it in our &lt;a href=&quot;https://juliadiffeq.github.io/DiffEqDocs.jl/latest/solvers/ode_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;list of methods&lt;/a&gt;, and then there are tons of others you can use &lt;a href=&quot;https://juliadiffeq.github.io/DiffEqDevDocs.jl/latest/internals/tableaus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;as supplied tableaus&lt;/a&gt;. Because all of these methods are put together, you can easily benchmark between them. &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;You can see the benchmarks I have online here&lt;/a&gt;, and see that it's very simple to benchmark many different algorithms. So if you want to take a few minutes to run the benchmarks, go for it. Here's a summary of what comes out.&lt;/p&gt;&#xA;&lt;p&gt;First off it's important to note that, if you look at each of the benchmarks, you will see that our &lt;code&gt;DP5&lt;/code&gt; (Dormand-Prince Order 5) and &lt;code&gt;DP8&lt;/code&gt; methods are faster than the Hairer Fortran codes (&lt;code&gt;dopri5&lt;/code&gt; and &lt;code&gt;dop853&lt;/code&gt;), and so these implementations are very well optimized. These show that &lt;a href=&quot;https://scicomp.stackexchange.com/questions/24998/which-runge-kutta-method-is-more-accurate-dormand-prince-or-cash-karp/25031#25031&quot;&gt;as noted in another thread&lt;/a&gt; the over-use of the Dormand-Prince methods is because the methods are already written, not because they are still the best. So the real comparison between the most optimized implementations is between the Tsitouras methods, the Verner methods, and the Feagin methods from DifferentialEquations.jl.&lt;/p&gt;&#xA;&lt;h2&gt;The Results&lt;/h2&gt;&#xA;&lt;p&gt;In general, the methods of an order higher than 7 have an added computational cost which is usually not outweighed by order, given the tolerances chosen. One reason for this is that the coefficient choices for lower order methods are more optimized (they have small &amp;quot;principle truncation error coefficients&amp;quot;, which matter more when you're not asymtopically small). You can see that in many problems &lt;a href=&quot;https://benchmarks.juliadiffeq.org/html/NonStiffODE/linear_wpd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;like here&lt;/a&gt; the Verner Efficient 7 and 8 methods do extremely well, but methods like the Verner Efficient 9 can have a lower slope. This is because the &amp;quot;gains&amp;quot; of higher order are compounding at lower tolerances, so there is always a tolerance where the higher order methods will be more efficient.&lt;/p&gt;&#xA;&lt;p&gt;However, the question is then, how low? In a well-optimized implementation, that gets pretty low for two reasons. The first reason is because lower order methods implement something called FSAL (first same as last). This property means that the lower order methods re-use a function evaluation from the previous step in the next step, and thus have effectively one less function evaluation. If this is used properly, then something like a 5th order method (Tsitouras or Dormand-Prince) is actually taking 5 function evaluations instead of the 6 that the tableaus would suggest. This is also true for the Verner 6 method.&lt;/p&gt;&#xA;&lt;p&gt;The other reason is due to interpolations. One reason to use a very high order method is to take less steps and simply interpolate intermediate values. However, in order to get the intermediate values, the interpolating function may need more function evaluations than were used for taking the step. &lt;a href=&quot;http://people.math.sfu.ca/%7Ejverner/&quot; rel=&quot;nofollow noreferrer&quot;&gt;If you look at the Verner methods&lt;/a&gt;, it takes 8 extra function evaluations for the Order 8 method to get an Order 8 interpolant. Many times the low order methods provide a &amp;quot;free&amp;quot; interpolant, for example most 5th order methods have a free 4th order interpolation (no extra function evaluations). So this means that if you need intermediate values (which you will need for a good plot if you're using a high order method), there are some extra hidden costs. Factor in the fact that these interpolated values are really important for event handling and solving delay differential equations and you see why the extra interpolation cost factors in.&lt;/p&gt;&#xA;&lt;h2&gt;So What About the Feagin Methods?&lt;/h2&gt;&#xA;&lt;p&gt;So you will see that the Feagin methods are suspiciously missing from the benchmarks. They are fine, convergence tests work on arbitrary precision numbers, etc., but to actually get them to do well you need to be asking for some pretty absurdly low tolerances. For example, I found in unpublished benchmarks that the &lt;code&gt;Feagin14&lt;/code&gt; outperforms &lt;code&gt;Vern9&lt;/code&gt; (the 9th order Verner Efficient Method) at tolerances like &lt;code&gt;1e-30&lt;/code&gt;. For applications with chaotic dynamics (like in the Pleides or 3-body astrophysics problems), you may want this amount of accuracy due to sensitive dependence (errors in chaotic systems compound fast). However, Most people are probably computing with double-precision floating point numbers, and I haven't found a benchmark where they outperform in this domain of tolerance.&lt;/p&gt;&#xA;&lt;p&gt;Here's a work-precision efficiency plot &lt;a href=&quot;https://benchmarks.juliadiffeq.org/html/NonStiffODE/linear_wpd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;on this 100 dimensional ODE&lt;/a&gt; which showcases this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CunQP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CunQP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And here's a work-precision efficiency plot &lt;a href=&quot;https://benchmarks.juliadiffeq.org/html/NonStiffODE/Pleiades_wpd.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;for the Pleiades ODE&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W7Rhk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W7Rhk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;With these we do not find good results when testing the Feagin methods with adaptivity in tolerances within the Float64-range. Possibly the issue could be the stability of the error estimate. Note that this is done with PI-adaptivity for stabilizing the stepping of each of the methods.&lt;/p&gt;&#xA;&lt;p&gt;In addition, there is no interpolant to go along with the Feagin methods (Edit 2020: See Feagin's remark that one now exists, but is not available online. But note that this has no effect on the benchmarks since the benchmarks do not use the interpolant). So what I do is simply put a third order Hermite interpolation on them so that way one exists (and it works surprisingly well). However, if there's no standard interpolating function, you can do the recursive Hermite method (use this interpolation to get the midpoint, then do a 5th order interpolation, etc.) to get a high order interpolation, but this is very costly and the resulting interpolation doesn't necessarily have a low principle truncation error term (so it's only good when &lt;code&gt;dt&lt;/code&gt; is really small, which is the exact opposite of the case we want!). So if you ever need a really good interpolation to match your accuracy, you need to at least go back to something like &lt;code&gt;Vern9&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Note About Extrapolation&lt;/h2&gt;&#xA;&lt;p&gt;Note that extrapolation methods are simply algorithms for generating arbitrary order Runge-Kutta methods. However, for their order they take more steps than necessary and have high principle truncation error coefficients, and so they are not as efficient as a well-optimized RK method at a given order. But given the previous analysis, this means that there is a domain of extremely low tolerance where these methods will do better than the &amp;quot;known&amp;quot; RK methods. But in every benchmark I've ran, it seems I haven't gotten that low.&lt;/p&gt;&#xA;&lt;h2&gt;Note About Stability&lt;/h2&gt;&#xA;&lt;p&gt;The choice really has nothing to do with stability issues. In fact, if you go through the DifferentialEquations.jl tableaus (you can just &lt;code&gt;plot(tab)&lt;/code&gt; for the stability regions) you will see that most of the methods have suspiciously similar stability regions. This is actually a choice. Usually when deriving the methods, the author usually does the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find the lowest principle truncation error coefficients (that is, the coefficients for the next order terms)&lt;/li&gt;&#xA;&lt;li&gt;Subject to the order constraints&lt;/li&gt;&#xA;&lt;li&gt;And make the stability region close to that of the Dormand-Prince Order 5 method.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Why the last condition? Well, because that method tends to be always stable with the way PI-controlled adaptive stepsize choices are done, so it's a good bar for &amp;quot;good enough&amp;quot; stability regions. So it's no coincidence that the stability regions all tend to be similar.&lt;/p&gt;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&lt;p&gt;There's tradeoffs in every choice of method. The highest order RK methods are simply not as efficient at lower tolerances both because it's harder to optimize the choice of coefficients, and because the number of function evaluations compounds (and grows even faster when interpolations are involved). However, if the tolerance gets low enough they win out, but the tolerances that are required can be far below &amp;quot;standard&amp;quot; applications (i.e. really only applicable to chaotic systems).&lt;/p&gt;&#xA;&lt;h3&gt;Appendix: Benchmarking Code&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;using OrdinaryDiffEq, DiffEqDevTools, Plots&#xA;using Random&#xA;Random.seed!(123)&#xA;gr()&#xA;# 2D Linear ODE&#xA;function f(du,u,p,t)&#xA;  @inbounds for i in eachindex(u)&#xA;    du[i] = 1.01*u[i]&#xA;  end&#xA;end&#xA;function f_analytic(u₀,p,t)&#xA;  u₀*exp(1.01*t)&#xA;end&#xA;tspan = (0.0,10.0)&#xA;prob = ODEProblem(ODEFunction(f,analytic=f_analytic),rand(100,100),tspan)&#xA;&#xA;abstols = 1.0 ./ 10.0 .^ (3:13)&#xA;reltols = 1.0 ./ 10.0 .^ (0:10);&#xA;&#xA;setups = [Dict(:alg=&amp;gt;Tsit5())&#xA;          Dict(:alg=&amp;gt;Vern7())&#xA;          Dict(:alg=&amp;gt;Vern8())&#xA;          Dict(:alg=&amp;gt;Feagin10())&#xA;          Dict(:alg=&amp;gt;Feagin12())&#xA;          Dict(:alg=&amp;gt;Feagin14())&#xA;          Dict(:alg=&amp;gt;Vern9())]&#xA;wp = WorkPrecisionSet(prob,abstols,reltols,setups;save_everystep=false,numruns=100)&#xA;plot(wp)&#xA;savefig(&amp;quot;feagin1.png&amp;quot;)&#xA;&#xA;f = (du,u,p,t) -&amp;gt; begin&#xA;  @inbounds begin&#xA;  x = view(u,1:7)   # x&#xA;  y = view(u,8:14)  # y&#xA;  v = view(u,15:21) # x′&#xA;  w = view(u,22:28) # y′&#xA;  du[1:7] .= v&#xA;  du[8:14].= w&#xA;  for i in 14:28&#xA;    du[i] = zero(u[1])&#xA;  end&#xA;  for i=1:7,j=1:7&#xA;    if i != j&#xA;      r = ((x[i]-x[j])^2 + (y[i] - y[j])^2)^(3/2)&#xA;      du[14+i] += j*(x[j] - x[i])/r&#xA;      du[21+i] += j*(y[j] - y[i])/r&#xA;    end&#xA;  end&#xA;  end&#xA;end&#xA;&#xA;prob = ODEProblem(f,[3.0,3.0,-1.0,-3.0,2.0,-2.0,2.0,3.0,-3.0,2.0,0,0,-4.0,4.0,0,0,0,0,0,1.75,-1.5,0,0,0,-1.25,1,0,0],(0.0,3.0))&#xA;&#xA;abstols = 1.0 ./ 10.0 .^ (9:12)&#xA;reltols = 1.0 ./ 10.0 .^ (6:9);&#xA;&#xA;sol = solve(prob,Vern8(),abstol=1/10^12,reltol=1/10^10,maxiters=1000000)&#xA;test_sol = TestSolution(sol);&#xA;&#xA;setups = [Dict(:alg=&amp;gt;Tsit5())&#xA;          Dict(:alg=&amp;gt;Vern7())&#xA;          Dict(:alg=&amp;gt;Vern8())&#xA;          Dict(:alg=&amp;gt;Feagin10())&#xA;          Dict(:alg=&amp;gt;Feagin12())&#xA;          Dict(:alg=&amp;gt;Feagin14())&#xA;          Dict(:alg=&amp;gt;Vern9())]&#xA;wp = WorkPrecisionSet(prob,abstols,reltols,setups;appxsol=test_sol,save_everystep=false,numruns=100,maxiters=1000)&#xA;plot(wp)&#xA;savefig(&amp;quot;feagin2.png&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="25589" LastEditDate="2022-07-21T20:06:50.873" LastActivityDate="2022-07-21T20:06:50.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25620" PostTypeId="2" ParentId="25578" CreationDate="2016-11-21T18:07:56.817" Score="3" Body="&lt;p&gt;The plot basically say that it's the region where the linear differential equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{dy}{dt} = \lambda y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\lambda &amp;lt; 0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;will actually go to zero. If you play around with the Euler method for a bit, you will see that if you make the stepsize too large it will explode. What happens is it will make a big step with low accuracy, end up negative, then flip back to positive, and these oscillations will continue to explode and the solution will go to infinity. So stability is essentially a maximum stepsize for which a method will accurately solve the linear differential equation without exploding ($\lambda &amp;lt; 0 $ because of course if $ \lambda &amp;gt; 0$ the solution and the thus numerical method will always explode if you take enough steps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is very much related to oscillations (since it seems to come from over-stepping a boundary which it shouldn't) and so complex $\lambda$ is important since a pure complex $\lambda$ is only oscillations. So the real part measures the speed to zero, the imaginary part measures the oscillations, and the stability plot shows you how well a method can handle the combinations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This maps to &quot;real&quot; problems because every smooth enough problem is locally linear. Thus you can instead think of $\lambda$ as an eigenvalue of the localized Jacobian. For a nonlinear problem, you will have stability if you always have that the &quot;instantaneous speed and oscillations&quot;, $\lambda $, are within the stability region. This may change throughout the problem, which is why adaptive timestepping methods are crucial parts of efficient solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When your problem has very large $\lambda$, either complex or real, that's when people begin to say your problem is stiff (though there isn't truly one definition around. Another way is to say if your highest eigenvalue is &quot;large&quot; and your smallest eigenvalue is &quot;small&quot;, since this separation causes many numerical problems). In that case, you can see from the stability plots that most explicit methods will require a very small timesteps in order to be stable. This is where &quot;stiffly stable&quot;, A-stable, etc. methods come in since they can be used in these cases with &quot;normal&quot; timesteps, but with added costs somewhere else (no free lunch!). &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2016-11-21T18:07:56.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25627" PostTypeId="1" CreationDate="2016-11-22T15:49:47.297" Score="1" ViewCount="264" Body="&lt;p&gt;I have to solve&#xA;$$\left\{\begin{matrix}&#xA;\ddot{R}=-\frac{3}{2}\frac{\dot{R}^{2}}{R}+\frac{1}{R\rho}\left [P_{v}(T_{\infty})-P_{0}+\frac{(L\rho_{g})^{2}}{hT_{\infty}}\dot{R}-4\mu \frac{\dot{R}}{R}-\frac{2S}{R}  \right ]\\ &#xA;h=\frac{\lambda}{2R}\left [ 2+\left ( \frac{-6\rho c_{p}\dot{R}}{\pi h}\right )^{1/3}-\left ( \frac{12\rho c_{p}\dot{R}}{\pi h} \right ) \right   ]\\ &#xA;\ln\left ( \frac{\rho_{g}}{\rho_{c}}\right )=c_{1}\theta^{1/3}+c_{2}\theta^{2/3}+c_{3}\theta^{4/3}+c_{4}\theta^{3}+c_{5}\theta^{37/6}+c_{6}\theta^{71/6}&#xA;\end{matrix}\right.$$&#xA;where $\theta=1-\frac{1}{T_{c}}\left ( T_{\infty}+\frac{L\rho_{g}}{h}\dot{R} \right )$ and $R(0)=R_{0}$, $\dot{R}(0)=0$.&#xA;If a use a pressure $P_{0}=2100$, the code works; if I set $P_{0}=2000$, MATLAB returns this error: &quot;Warning: Imaginary parts of complex X and/or Y arguments ignored.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function out = densita_20_latente(t,y,R0)&#xA;S = 0.073; %N/m&#xA;rho = 998;    &#xA;mi = 1.005e-3;&#xA;P0 = 2100; %&amp;lt;-----------------------------------------&#xA;L = 2454000;&#xA;Tinf = 293;   &#xA;lambda = 0.5982;&#xA;cp = 4184;    &#xA;Pv_Tinf = 2329.6;&#xA;c1 = -2.03150240;&#xA;c2 = -2.6830294;&#xA;c3 = -5.38626492;&#xA;c4 = -17.2991605;&#xA;c5 = -44.7586581;&#xA;c6 = -63.9201063;&#xA;Tc = 647.096;&#xA;rhoc = 322;&#xA;%&#xA;out=[y(2)&#xA;    -1.5 * y(2)^2 / y(1) + 1 / rho / y(1) * (Pv_Tinf - P0 +...&#xA;    (L * y(4))^2 / Tinf * y(2) / y(3) -...&#xA;    4 * mi * y(2) / y(1) - 2 * S / y(1))&#xA;    -y(3) + lambda * 0.5 / y(1) * (2 + (-6 * rho * cp * y(2) / pi / y(3))^(1/3)...&#xA;    - 12 * rho * cp * y(2) / pi / y(3))&#xA;    -y(4) + rhoc * exp(c1*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(1/3)...&#xA;    + c2*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(2/3)...&#xA;    + c3*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(4/3)...&#xA;    + c4*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(3)...&#xA;    + c5*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(37/6)...&#xA;    + c6*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(71/6)) ];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;run file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R0=510e-6;&#xA;y0=[510e-6; 0; 1172.94; 0.01716];&#xA;M = [1 0 0 0; 0 1 0 0; 0 0 0 0; 0 0 0 0];&#xA;options = odeset('Mass',M);&#xA;[t,y] = ode15s(@(t,y) densita_20_latente(t,y,R0),[0 46.75e-6],y0,options);&#xA;t = t*1000000;&#xA;y = y*1000000;&#xA;[t,y(:,1)];&#xA;plot(t,y(:,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: I am going to simplify the problem I have to solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\{\begin{matrix}&#xA;\ddot{R}=-\frac{3}{2}\frac{\dot{R}^{2}}{R}+\frac{1}{R\rho}\left [P_{v}(T_{\infty})-P_{0}+\frac{(L\rho_{g})^{2}}{hT_{\infty}}\dot{R}-4\mu \frac{\dot{R}}{R}-\frac{2S}{R}  \right ]\\ &#xA;h=\frac{\lambda}{2R}\left [ 2+\left ( \frac{-6\rho c_{p}\dot{R}}{\pi h}\right )^{1/3}-\left ( \frac{12\rho c_{p}\dot{R}}{\pi h} \right ) \right   ]\\&#xA;\end{matrix}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function out = convezione_20(t,y,R0)&#xA;S = 0.073;&#xA;rho = 998;    &#xA;mi = 1.005e-3;&#xA;P0 = 2000; %&amp;lt;-------------------------------------------&#xA;L = 2454000;&#xA;Tinf = 293;   &#xA;rho_g = 1/57.79;&#xA;&#xA;lambda = 0.5982;&#xA;cp = 4184;    &#xA;Pv_Tinf = 2329.6;&#xA;%&#xA;out=[y(2)&#xA;    -1.5 * y(2)^2 / y(1) + 1 / rho / y(1) * (Pv_Tinf - P0 +...&#xA;    (L * rho_g)^2 / Tinf * y(2) / y(3) -...&#xA;    4 * mi * y(2) / y(1) - 2 * S / y(1))&#xA;    - y(3) + lambda * 0.5 / y(1) * (2 + (-6 * rho * cp * y(2) / pi / y(3))^(1/3)...&#xA;    - 12 * rho * cp * y(2) / pi / y(3)) ];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;run file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R0 = 510e-6;&#xA;y0 = [R0; 0; 1172.94];&#xA;M = [1 0 0; 0 1 0; 0 0 0];&#xA;options = odeset('Mass',M);&#xA;[t,y] = ode15s(@(t,y) convezione_20(t,y,R0),[0 40e-6],y0,options);&#xA;t = t*1000000;&#xA;y = y*1000000;&#xA;[t,y(:,1)];&#xA;plot(t,y(:,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Same problem as before. With 2100 the code works; with 2000 MATLAB reads &quot;Warning: Imaginary parts of complex X and/or Y arguments ignored.&quot; I tried using as exponent $1$ instead of $1/3$ in the expression of $h$, but the warning now becomes &quot;Warning: Failure at t=2.874099e-007.  Unable to meet integration tolerances without reducing the step size below the smallest value allowed (1.021085e-021) at time t.&quot;&lt;/p&gt;&#xA;" OwnerUserId="22288" LastEditorUserId="22288" LastEditDate="2016-11-22T17:16:09.807" LastActivityDate="2016-11-22T17:16:09.807" Title="MATLAB ODE15s: imaginary parts" Tags="|matlab|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25628" PostTypeId="1" AcceptedAnswerId="25631" CreationDate="2016-11-22T16:00:16.913" Score="3" ViewCount="217" Body="&lt;p&gt;I have a data set, composed of points $(x_i, y_i)$ for $i=1,N$. I also have a known function $F$, which maps these points $x_i$ to $y_i$ as such $F(x_i, a(x_i),b(x_i)) = y_i$, where $a(x_i)$ and $b(x_i)$ are unknown functions of $x_i$. What I'd like to do is run an optimization (ideally using Python packages) in order to determine the functions $a$ and $b$ such that the error is minimized &lt;em&gt;and&lt;/em&gt; also such that the functions $a$ and $b$ are smooth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One, possibly naive way of doing this is to bin my data with respect to the $x_i$ variable, and then solve the optimization problem $F(x_i, a,b) = y_i$ for the single values $a,b$ for all points within the bin. Then, using this discretized solution for $a,b$, perform a numerical fit which results in a smooth curve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm wondering if this is the best way to go about it. Is there a method that I could use which would determine the $a,b$ values simultaneously while trying to create a smooth curve from them? Or even if there were, would the effect be the same as if I perform the naive binned-optimization as I've described above? &lt;/p&gt;&#xA;" OwnerUserId="5196" LastActivityDate="2016-11-22T21:10:53.097" Title="Optimization of known function with respect to two unknown function arguments" Tags="|optimization|python|numerics|curve-fitting|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25630" PostTypeId="1" CreationDate="2016-11-22T19:14:25.697" Score="2" ViewCount="160" Body="&lt;p&gt;I recently scaled my dynamic model based on an open source FEM solver, to run a mesh containing nearly 34 million cells successfully on 800 cores. I have very limited experience using commercial software like ABAQUS, ANSYS, FLAC, PLAXIS, etc. So, I was wondering if this can also be easily achieved using such commercial software? In a nutshell, I want to know if scaling your model to run for large scale problems is easy in commercial software as well. The idea is to highlight the advantages of open source over commercial software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for sharing your thoughts and experience.&lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2018-03-23T17:31:29.357" Title="Parallel processing ability of popular commercial software" Tags="|parallel-computing|open-source|scaling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25631" PostTypeId="2" ParentId="25628" CreationDate="2016-11-22T19:21:42.453" Score="4" Body="&lt;p&gt;Your problem is ill posed in the sense that if a solution exists, it is not unique. To see this, let ($a_i^*$, $b_i^*$), denote a solution of the equation $F(x_i, a_i,b_i)=y_i$, $i\in\{1,\ldots,N\}$ (or the minimizers of an appropriate cost function if equality cannot be achieved). Then, &lt;em&gt;any&lt;/em&gt; pair of functions $\{a^*(x)$, $b^*(x)\}$ such that $a^*(x_i)=a_i^*$, $b^*(x_i)=b_i^*$, for all $i\in\{1,\ldots,N\}$, is a solution to your problem. Clearly, the set of all possible solutions is uncountable, even with the additional requirement that they are 'smooth'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to avoid this ambiguity in the solution, one approach is to impose additional conditions. For example, you could restrict the solution functions to belong to a finite dimensional (and 'smooth') space (e.g., the space of polynomials of order $n$). In that case, the (difficult) functional optimization problem is converted to the (much simpler) problem of determination of a finite set of variables.&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2016-11-22T19:21:42.453" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25632" PostTypeId="2" ParentId="25628" CreationDate="2016-11-22T21:10:53.097" Score="3" Body="&lt;p&gt;You could discretize the unknown functions $a$ and $b$, then formulate this as a least squares optimization problem with smoothing regularization. Specifically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{\mathbf{a}, \mathbf{b}} \frac{1}{2}\sum_{i=1}^N\left( f_i - F(x_i, \mathbf{m}_i^T \mathbf{a}, \mathbf{m}_i^T \mathbf{b}) \right)^2 + \frac{\alpha}{2} \mathbf{a}^T R \mathbf{a} + \frac{\alpha}{2} \mathbf{b}^T R \mathbf{b}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\mathbf{a}$ and $\mathbf{b}$ are coefficients of $a$ and $b$ with respect to some discrete set of basis functions $\{\phi_j\}_{j=1}^m$ (say, a piecewise linear basis of &quot;hat&quot; functions),&#xA;$$a(x) = \sum_{j=1}^m \mathbf{a}_j \phi_j (x)$$ &lt;/li&gt;&#xA;&lt;li&gt;$f_i$ is the $i$'th piece of data you are trying to fit, &lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{m}_i^T := \begin{bmatrix}\phi(x_1) &amp;amp; \phi(x_2) &amp;amp; \dots &amp;amp; \phi(x_N)\end{bmatrix}$ is the &quot;measure at $x_i$ vector&quot; such that&#xA;$$\mathbf{m}_i^T \mathbf{a} = a(x_i)$$&lt;/li&gt;&#xA;&lt;li&gt;R is a discrete form of the Laplacian with Neumann boundary conditions, plus some small multiple $\tau$ of the identity:&#xA;$$R := \Delta_N + \tau I.$$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This could then be solved with the Gauss-Newton method, or L-BFGS, or something like that, (it is straightforward enough to figure out the derivative of the objective function by hand). In Python you could do the discretization with FEniCS, and the optimization with Scipy, though it will involve knowing a little about finite elements and a little about optimization.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2016-11-22T21:10:53.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25633" PostTypeId="1" CreationDate="2016-11-22T21:54:47.267" Score="2" ViewCount="192" Body="&lt;p&gt;My phd project is heavily related to numerical modeling of planetary atmospheres. In particular now I am dealing with a particular expression of the continuity equation, involving a thermodynamic flux. Equations read&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial n}{\partial t} = -\frac{\partial \Phi}{\partial z}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Phi = -nD\left(\frac{1}{n}\frac{\partial n}{\partial z} + \frac{1+\alpha}{T}\frac{\partial T}{\partial z} + \frac{1}{h}\right) -nK\left(\frac{1}{n}\frac{\partial n}{\partial z} + \frac{1}{T}\frac{\partial T}{\partial z} + \frac{1}{H}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where of course $n = n\left(z,t\right)$ refers to the particular chemical species density the equation solves. $h$ is the atmospheric scale height for the particular molecule while H is the average atmospheric scale height corresponding to the average atmospheric molecule. $D$ and $K$ are the molecular and eddy diffusion coefficients respectively. $T$ is the temperature. $\alpha$ is a parameter related to heat diffusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a standard implicit method to find a system of linear equations so that its solution yields the density at a new time step. I solve a tridagonal matrix in Fortran basically. For 1º order derivatives I take a 1º order central finite differences and 2º order central finite differences for second order derivatives. At the boundaries I use forward and backward approaches when necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I stated the solution seems correct when studying the density profiles. But there is something very annoying when at the boundaries fixed flux conditions are applied. Let's see by integrating the equation over all the domain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{z_{bot}}^{z_{top}}dz\left(z,t\right) = \rho \left(t\right) = - \int_{z_{bot}}^{z_{top}}dz\frac{\partial \Phi}{\partial z}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\implies \rho \left(t\right) = - \Phi_{top} + \Phi_{bottom}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Being $\rho\left(t\right)$ the magnitude known as column density. After this long introduction here my doubt comes. I fix $\Phi_{bottom} = \Phi_{top} = 0$ so that we should expect $\rho\left(t\right) \equiv constant $. Nevertheless this is not the case at all, mass is not conserved in my numerical system by very big numbers, integrating for each molecule. The processing of results is undertaken by Python scripts using numpy when necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been struggling with this for a month and I draw the following few guesses;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Could be that some parameters are not well treated. For instance there is a point where I need to take the numerical derivative of a big set of magnitudes. When studying the outcome it is very very fluctuating for a smooth curve.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Since the system deals with an atmosphere, the background profile used is a hydrostatic kind one, with densities ranging from $n = 10^{20} cm^{-3}$ to $n = 10^{8} cm^{-3}$. Using double precision numbers in fortran could make that during computation numerical fluctuations at the bottom part of the atmosphere may &quot;mask&quot; or screen what happens at the top, yielding a fake production of particles out of the blue.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This method does not conserve mass.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Am I on the correct way to find an answer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time and attention.&lt;/p&gt;&#xA;" OwnerUserId="22428" LastActivityDate="2016-11-22T21:54:47.267" Title="Mass conservation in atmospheric continuity equation numerical solution" Tags="|finite-difference|boundary-conditions|computational-physics|fortran|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25634" PostTypeId="1" CreationDate="2016-11-23T06:53:15.270" Score="0" ViewCount="827" Body="&lt;p&gt;I am a PhD student and my field of study is Quantum Information and Computation in the theoretical aspect. Actually I write the computer codes/program in FORTRAN 90 which are serial codes but now I have to run my codes in supercomputer so I have to learn parallel FORTRAN programming. Is there anybody to guide me from where I can start as beginner like the best book or notes.&lt;/p&gt;&#xA;" OwnerUserId="22433" LastActivityDate="2016-11-23T12:27:42.717" Title="Best books and notes for beginners in the parallel FORTRAN 90 programming" Tags="|parallel-computing|fortran|quantum-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25636" PostTypeId="1" CreationDate="2016-11-23T12:08:09.820" Score="0" ViewCount="180" Body="&lt;p&gt;What is the role of non-symmetric coefficient matrices in the solution of partial differential equations with self-adjoint operators?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Particularly, I'm thinking about time-propagation of a linear equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$i\partial_t v =M v$$&#xA;where $M$ is self-adjoint. (Or equivalently, without the complex unit $i$ and then $M$ is skew-hermitian, or in the real case, symmetric or skew-symmetric),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discretization of this equation on an equidistant grid, together with the usual finite-difference formulas for derivatives, leads to a hermitian coefficient matrix. The same is true for a spectral expansion. The arising matrices have nice properties: one can diagonalize them and get real eigenvalues and orthogonal eigenvectors, one can apply linear algebra routines tailored to symmetric matrices, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, there are several successful methods (finite differences on non-equally spaced grids, one-sided finite differences, Chebyshev and other pseudospectral methods, etc.) which allow for a more accurate expansion (e.g. finer grid in important regions), but usually lead to non-hermitian coefficient matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is preferable? Should I keep the symmetry as long as possible? Or can one rely as well on non-symmetric matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a question which is puzzling me for a long time and where I have not found a concrete analysis yet. However, almost any reference seems to give statements such as &quot;symmetry in the coefficient matrix is preferable&quot; due to &quot;stability&quot; or whatever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions (more detailed):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;What is the theoretical impact of this symmetry breaking?&lt;/strong&gt; Problems I see: eigenvalues can become complex, so there is no guarantee that variational optimization still holds. This also implies that conservation laws do not hold anymore. Is this a problem in practice, or does a good choice of non-symmetric discretization (numerically) avoids these problems?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;What is the drawback in using non-symmetric coefficient matrices?&lt;/strong&gt; (Besides the obvious one that no Linear Algebra routines specifically designed to symmetric matrices can be used).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If there should be a disadvantage by using non-symmetric matrices: Is it reasonable to introduce symmetry artificially? E.g. by symmetrizing $[\tilde M_{ij} = 1/2(M_{ij} + M_{ji})]$, setting up the second derivative matrix (which is often the source of asymmetry) in terms of the first $(D_2 = D_1^T D_1)$ or by more sophisticated constructions like variable changes, etc.?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Summarizing the previous points: are there any references which consider this topic (under whatever aspect)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2016-11-23T17:14:55.117" LastActivityDate="2016-11-23T18:12:25.757" Title="Role of non-hermitian coefficient matrices in the discretization of self-adjoint operators" Tags="|linear-algebra|pde|time-integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25637" PostTypeId="2" ParentId="25634" CreationDate="2016-11-23T12:27:42.717" Score="3" Body="&lt;p&gt;I'm assuming by 'supercomputer' you're talking about HPC and distributed memory systems, as opposed to shared memory workstations, in which case you have a couple of options when it comes to Fortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are restricted to Fortran 90, then your options basically come down to MPI. MPI is not a Fortran-only solution, and if you understand some C/C++, you should be able to learn from various resources. One introduction site I found concise was this one, and it shows examples in both C and Fortran:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hamilton.nuigalway.ie/teaching/AOS/NINE/mpi-first-examples.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hamilton.nuigalway.ie/teaching/AOS/NINE/mpi-first-examples.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have the option of using a later version of Fortran (2008), then you have the option of using a Fortran-specific feature called coarrays. These are relatively new to the Fortran standard, and have been included in versions of gfortran since about 2015. Coarray Fortran (CAF) isn't as widespread in use as MPI is, but that may change in the future. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S002199911500354X&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some researchers&lt;/a&gt; have noted that since CAF is a Fortran-specific feature as opposed to MPI, &quot;[it] is of course much easier to write and maintain, and the simpler syntax makes the parallelism easier to understand.&quot;&lt;/p&gt;&#xA;" OwnerUserId="19791" LastActivityDate="2016-11-23T12:27:42.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25638" PostTypeId="2" ParentId="25636" CreationDate="2016-11-23T14:58:20.077" Score="1" Body="&lt;p&gt;I'll answer why skew-symmetric matrices are sometimes preferable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing to note is that this is usually problem dependent. But for a general 1D PDE like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} + \frac{\partial f}{\partial x} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;skew-symmetry has a distinct advantage. Suppose we discretize this equation, then we'll get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_t + Du = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose that D is skew-symmetric, then we have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D^T = -D$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's now find out what the derivative of $u^2$ is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(u^Tu)_t = (u^T)_tu + u^Tu_t  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting from our expression for $u_t$ we get,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(u^Tu)_t = -(Du)^Tu - u^TDu  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(u^Tu)_t = -u^TD^Tu - u^TDu  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(u^Tu)_t = -u^T(D^T + D)u  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall the skew-symmetry matrix property, $D^T = -D$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(u^Tu)_t = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of things to note. First $(u^Tu)_t $ is what we usually call kinetic energy (with a factor of 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the original equation had a divergence term,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial f}{\partial x} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that means that if we do a volume integral, any energy can only come in through the boundaries. So, $(u^Tu)_t = 0$ should be satisfied in general unless there's a dissipation term in the PDE. And that is exactly what the skew-symmetry operator is doing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll link to a presentation that I found useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://calcul.math.cnrs.fr/Documents/Ecoles/CEMRACS2012/Julius_Reiss.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://calcul.math.cnrs.fr/Documents/Ecoles/CEMRACS2012/Julius_Reiss.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, all this depends on individual problems. A different PDE may not like this method. A different operator matrix may have some other property that is preferable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1605.09763v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1605.09763v1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The convection operator that is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u.\nabla u $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is transformed to the skew-symmetric form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{1}{2}(u.\nabla u + (\nabla.u)u) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is in fact a lot of literature in which a similar transformation is done. The reason for doing it is because often a uniform mesh is not available so there is less control available over the discretized operator, so instead the PDE term is itself transformed to enforce skew-symmetry. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another common technique is to make the finite difference operator into a summation by parts operator which is essentially converting it into skew-symmetric. This is a paper that shows the technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/article/10.1007/s10543-015-0599-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://link.springer.com/article/10.1007/s10543-015-0599-0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But obviously this is only useful when finite difference is an option. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in essence, more and more studies are being done to create schemes so that desired properties be satisfied. &lt;/p&gt;&#xA;" OwnerUserId="16735" LastEditorUserId="16735" LastEditDate="2016-11-23T18:12:25.757" LastActivityDate="2016-11-23T18:12:25.757" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25639" PostTypeId="1" CreationDate="2016-11-23T15:57:13.583" Score="-1" ViewCount="247" Body="&lt;p&gt;I have to solve&#xA;$$\left\{\begin{matrix}&#xA;\ddot{R}=-\frac{3}{2}\frac{\dot{R}^{2}}{R}+\frac{1}{R\rho}\left [P_{v}(T_{\infty})-P_{0}-a_{1}t^{2}-a_{2}t+\frac{(L\rho_{g})^{2}}{hT_{\infty}}\dot{R}-4\mu \frac{\dot{R}}{R}-\frac{2S}{R}  \right ]\\ &#xA;h=\frac{2\hat{\sigma}}{2-\hat{\sigma}}\frac{L^{2}\rho_{g}}{T_{B}}\left ( \frac{\bar{M}}{2\pi \bar{R}T_{B}} \right )^{1/2}\left [ 1-\frac{1}{2L\rho_{g}}\left ( P_{v}(T_{\infty})+\frac{(L\rho_{g})^{2}}{hT_{\infty}}\dot{R} \right ) \right ]\\ &#xA;\ln\left ( \frac{\rho_{g}}{\rho_{c}}\right )=c_{1}\theta^{1/3}+c_{2}\theta^{2/3}+c_{3}\theta^{4/3}+c_{4}\theta^{3}+c_{5}\theta^{37/6}+c_{6}\theta^{71/6}&#xA;\end{matrix}\right.$$&#xA;where $$\left\{\begin{matrix}&#xA;R(0)=R_{0}\\ &#xA;\dot{R}(0)=0\\ &#xA;T_{B}=T_{\infty}+\frac{L\rho_{g}}{h}\dot{R}&#xA;\end{matrix}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I do not know if this may be helpful, but note the order of magnitude of $a_{1}$ ($10^{14}$) and of time span ($10^{-6}$).&#xA;The problem is that I receive a warning: &quot;Warning: Failure at t=5.142410e-005.  Unable to meet integration tolerances without reducing the step size below the smallest value allowed (1.826951e-019) at time t.&quot; How can I fix it? I tried switching to ode23t but that did not solve the problem.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fPK9j.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fPK9j.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function out=densita_20_latente_christ(t,y,R0)&#xA;S = 0.073; %N/m&#xA;rho = 998;    &#xA;mi = 1.005e-3;&#xA;P0 = 101325;&#xA;L = 2454000;&#xA;Tinf = 293;   &#xA;lambda = 0.5982;&#xA;cp = 4184;   &#xA;Pv_Tinf = 2329.6;&#xA;sigma_hat=0.03;&#xA;M_bar=18;&#xA;R_bar=8314;&#xA;c1 = -2.03150240;&#xA;c2 = -2.6830294;&#xA;c3 = -5.38626492;&#xA;c4 = -17.2991605;&#xA;c5 = -44.7586581;&#xA;c6 = -63.9201063;&#xA;Tc = 647.096;&#xA;rhoc = 322;&#xA;a1 = 2.488125e14;&#xA;a2 = -9.9525e9;&#xA;%&#xA;out=[y(2)&#xA;    -1.5 * y(2)^2 / y(1) + 1 / rho / y(1) * (Pv_Tinf - a1 * t^2 - a2 * t - P0 +...&#xA;    (L * y(4))^2 / Tinf * y(2) / y(3) -...&#xA;    4 * mi * y(2) / y(1) - 2 * S / y(1))&#xA;     -y(3) + 2 * sigma_hat / (2 - sigma_hat) *...&#xA;     L^2 * y(4) / (Tinf + L * y(4) * y(2) / y(3))*...&#xA;    (M_bar / (2 * pi * R_bar * (Tinf+ L * y(4) * y(2) / y(3))))^0.5 *...&#xA;    (1 - 1 / (2 * L * y(4)) *...&#xA;    (Pv_Tinf + (L * y(4))^2 * y(2) / y(3) / Tinf))&#xA;    -y(4) + rhoc * exp(c1*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(1/3)...&#xA;    + c2*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(2/3)...&#xA;    + c3*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(4/3)...&#xA;    + c4*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(3)...&#xA;    + c5*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(37/6)...&#xA;    + c6*(1-1/Tc*(Tinf+y(4)*L/y(3)*y(2)))^(71/6)) ];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;run file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R0 = 510e-6;&#xA;y0 = [510e-6; 0; 11326.8525; 0.01716];&#xA;M = [1 0 0 0; 0 1 0 0; 0 0 0 0; 0 0 0 0];&#xA;options = odeset('Mass',M);&#xA;[t,y] = ode15s(@(t,y) densita_20_latente_christ(t,y,R0),[0 52.424e-6],y0,options);&#xA;t=t*1000000;&#xA;y=y*1000000;&#xA;[t,y(:,1)];&#xA;plot(t,y(:,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22288" LastActivityDate="2016-11-24T07:56:59.203" Title="ODE15s: unable to meet integration tolerances" Tags="|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25640" PostTypeId="1" AcceptedAnswerId="25642" CreationDate="2016-11-23T23:05:37.490" Score="1" ViewCount="25" Body="&lt;p&gt;I have a set of numbers, all positive, integers obviously, and what I want to do is divide them in the least possible groups with each group having a GCD bigger than 1. The biggest possible number is 2^64-1. What I also need to do is keep the complexity as low as possible. I am aware that there are a lot of prime numbers from 0 to 2^64-1 but each of those will be in its own group containing only one number. Any thoughts?&lt;/p&gt;&#xA;" OwnerDisplayName="user20656" LastActivityDate="2016-11-24T01:33:10.600" Title="Break a set of numbers to minimum groups with each having a common divider" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25642" PostTypeId="2" ParentId="25640" CreationDate="2016-11-24T01:33:10.600" Score="1" Body="&lt;p&gt;If your set of numbers contains any number from 1 to $N$, then the answer is pretty obvious: You will need $\pi(N)+1$ groups where $\pi(N)$ is the number of primes less than or equal to $N$. That is because each of the $\pi(N)$ primes as well as 1 can not go into the same group as any of the others as they have no common divider. The remaining numbers, however, are obviously multiples of one of the primes, and you can for example put it into the group of its smallest divider. The optimal complexity algorithm to group everything is to use the sieve of Eratosthenes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Put 1 in its own group&lt;/li&gt;&#xA;&lt;li&gt;Put all multiples of 2 from 2 to $N$ into one group&lt;/li&gt;&#xA;&lt;li&gt;Put all multiples of 3 from 3 to $N$ that have not previously been put into a group, into their own group&lt;/li&gt;&#xA;&lt;li&gt;Since you have already handled 4, skip it&lt;/li&gt;&#xA;&lt;li&gt;Put all multiples of 5 from 5 to $N$ that have not previously been put into a group, into their own group&lt;/li&gt;&#xA;&lt;li&gt;Since you have already handled 6, skip it&lt;/li&gt;&#xA;&lt;li&gt;Put all multiples of 7 from 7 to $N$ that have not previously been put into a group, into their own group&lt;/li&gt;&#xA;&lt;li&gt;Etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The situation is more complicated if you have a set of numbers that is not complete. But a variation of the sieve algorithm above should still be able to do the job.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-11-24T01:33:10.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25643" PostTypeId="2" ParentId="25639" CreationDate="2016-11-24T07:56:59.203" Score="0" Body="&lt;p&gt;Did you try scaling your problem? If $a_1$ is very large and $t$ is very small, why not scale $t$ (and all the other parameters) accordingly? If $t$ is now expressed in seconds, try to write down the problem in microseconds... $a_1$ and $a_2$ are both large so for $t$ it will work out. I don't know how it will go with the other parameters.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2016-11-24T07:56:59.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25644" PostTypeId="1" AcceptedAnswerId="25647" CreationDate="2016-11-24T14:30:16.450" Score="2" ViewCount="2564" Body="&lt;p&gt;So this semester, I'm taking a Finite Element Method course at my graduate school. We started out making codes for 1D bars and came all the way to 8 node solid elements. However, I seem to have run into a wall, as I have made my code and combed through it for the past week, making sure all the shape functions and mathematics were correct. In the end, everything looks correct. However, while I am supposed to get deflections in the z direction around the ballpark of 0.3 m, I keep getting deflections in the magnitude of E-04. I am very much at my wit's end. Also, because I genuinely don't know which StackExchange community to post this question to, I have tried Mathematics and Overflow. I would like to emphasize that I'm not trying to just spam my question. If you kindly explain to me why my question doesn't belong here, I will delete it quickly. But I have done my research and I have made my own code, so I'm not trying to something out of nothing. Any and all help is appreciated. I will detail the problem below:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There is a bar that is fully clamped at one end. &lt;/li&gt;&#xA;&lt;li&gt;It is 1[m] long in    the x-dir and 0.1[m] long in the y-dir and&#xA;z-dir. &lt;/li&gt;&#xA;&lt;li&gt;A point force of 100[kN] is applied in an upward direction at    the&#xA;free end of the bar. The number of elements in the y-dir and z-dir is&#xA;to always be 1 element only, but the I'm supposed to increase the&#xA;number of x-dir elements to show that the displacement values&#xA;converge. Currently, not only do they not converge, but the values&#xA;are much too small.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once again, thank you for any and all help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/drive/folders/0BzaeyQa0ePwvU2lLTmZQSUZFLTg?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/drive/folders/0BzaeyQa0ePwvU2lLTmZQSUZFLTg?usp=sharing&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22445" LastActivityDate="2016-11-25T15:36:48.667" Title="3D Solid 8 Node FEM Matlab Code" Tags="|finite-element|matlab|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25647" PostTypeId="2" ParentId="25644" CreationDate="2016-11-25T15:36:48.667" Score="5" Body="&lt;p&gt;After some investigation, I believe your FE code is correct. Congratulations!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From beam theory, deflection of a cantilever beam with a point load at the tip is&#xA;$$u_{tip} = PL^3/(3EI)$$&#xA;For the parameters of your model, $u_{tip}=.03333m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a single element, your code gives $u_{tip}=8.4697 \times 10^{-4}m$. For 30 elements,&#xA;$u_{tip}=0.027862m$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why isn't the FE solution closer to the beam solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These fully-integrated eight-node hex elements (as well as the corresponding&#xA;2D, four-node quadrilateral) are notoriously bad when modeling bending behavior.&#xA;Remedies for this problem have been studied for more than 30 years. One&#xA;crude solution is to simply under-integrate the terms when forming the&#xA;stiffness matrix (e.g. a single gauss point). But this causes new problems.&#xA;A better approach is to use a more sophisticated approach than the&#xA;simple displacement formulation. You can find a wealth of information if&#xA;you search for &quot;shear locking in finite elements.&quot;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-11-25T15:36:48.667" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25648" PostTypeId="1" CreationDate="2016-11-25T17:45:59.163" Score="5" ViewCount="246" Body="&lt;p&gt;I am attempting to solve a computational acoustics problem that involves solving an underlying sparse matrix. The size of the problem varies with grid size (3D) and fill-in's obviously make direct solution impractical. Important features of the matrix are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It is non-Hermitian and particularly NOT diagonally dominant.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Regardless of the size of the problem, there is always one row that consists of only one element, far away from the diagonal (and a zero on the diagonal). This is part of the problem's closure condition.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have attempted to solve this problem using iterative techniques with preconditioning to speed up the solution. Iterative solution &lt;em&gt;without&lt;/em&gt; preconditioning yields fairly inaccurate results (by comparison with directly computed results for a small grid). ILU preconditioning works for coarse grids (small matrices ~ 42000 x 42000) but fails for anything over 100,000 x 100,000; keeping a low fill-in factor ends in &quot;Factor is exactly singular&quot; and large fill-in factor ends in one of the sub-matrices being singular during pivoting. I also tried enforcing the one zero diagonal element with machine epsilon before attempting ILU but it still fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relevant problem parameters are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There are 8 PDE's: conservation of mass, conservation of energy, three momentum equations for interior points of computation (full staggered grid with velocity on faces and pressure, temperature &amp;amp; density on cell centers), and three problem specific momentum equations on certain boundaries. &lt;/li&gt;&#xA;&lt;li&gt;There are suitable closure conditions applied to the problem to obtain a full ranked system, one of which has been described above (zero diagonal row).&lt;/li&gt;&#xA;&lt;li&gt;I am using a hybrid scheme, with divergences and certain gradients being constructed by using a finite volume approach, and the rest of the operators being constructed using a finite difference approach utilizing polynomial fitting.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If it helps, I am using scipy sparse libraries to carry out my computations. The iterative method I am using is LGMRES, which works without any problems when provided a suitable preconditioner. Please suggest some viable preconditioning techniques for this kind of problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The sparsity pattern of the matrix is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oIXNx.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oIXNx.jpg&quot; alt=&quot;Fine Grid&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22450" LastEditorUserId="22450" LastEditDate="2016-11-25T19:45:11.040" LastActivityDate="2016-11-25T19:45:11.040" Title="Preconditioning technique for large sparse non-hermitian matrix" Tags="|sparse-matrix|iterative-method|preconditioning|gmres|linear-algebra|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25649" PostTypeId="1" AcceptedAnswerId="25669" CreationDate="2016-11-26T05:26:10.040" Score="3" ViewCount="16576" Body="&lt;p&gt;The composite simpson's rule subdivides the interval into n equal subintervals, with n even.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then &#xA;$$\int_a^b f(x) dx \approx \frac{h}{3}[f(x_0) + 4f(x_1) + 2f(x_2) + 4f(x_3) + ... + 2f(x_{n-2}) + 4f(x_{n-1}) + f(x_n)] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(where $x_0 = a$ and $x_n=b$, $h=(b-a)/n$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Simpson&amp;#39;s_rule&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what to do if given n equal intervals, with n odd?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution I implemented is to calculate using Simpson's rule for all but the last interval and use the trapezoid rule for the last interval. This seems to work well enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I started thinking, how would I calculate the last interval using Simpson's rule as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{aligned}&#xA;I_1 &amp;amp;= \frac{h}{3}[f(x_{n-3}) + 4f(x_{n-2}) + f(x_{n-1})] \\&#xA;I_2 &amp;amp;= \frac{h}{3}[f(x_{n-2}) + 4f(x_{n-1}) + f(x_n)] \\&#xA;\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_n$ is the end of the integral interval. $I_1$ would be the sum corresponding to the last subinterval such that the total number of subintervals considered is even. $I_2$ is Simpson's rule applied to the final, odd subinterval, but half of the subinterval overlaps with $I_1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to identify what's common to $I_1$ and $I_2$ and subtract it out? Just staring at it, I can't easily see what is the common part. Is it as simple as $(1/2)(h/3)[5f(x_{n-2}) + 5f(x_{n-1})]$?&lt;/p&gt;&#xA;" OwnerUserId="21132" LastActivityDate="2017-10-23T03:55:31.203" Title="Composite simpson's rule with odd intervals" Tags="|integration|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25653" PostTypeId="1" AcceptedAnswerId="25655" CreationDate="2016-11-26T22:00:59.887" Score="0" ViewCount="89" Body="&lt;p&gt;I am trying to put the function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(\mu,\nu) = i^{\nu-\mu} \sum_{0}^{19} H_{\mu-\nu}(7j) + \delta_{\mu,\nu}\ ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mu, \nu =-3,-2,...2,3$ into a 7x7 matrix, where $H$ is the Hankel function of the first kind, and the $\delta$ represents the Kronecker Delta. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this be done, on MATLAB?&lt;/p&gt;&#xA;" OwnerUserId="22087" LastEditorUserId="9667" LastEditDate="2017-07-22T16:12:33.447" LastActivityDate="2017-07-22T16:12:33.447" Title="MATLAB: Matrix whose elements depend on its indicies" Tags="|matlab|matrix|linear-system|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25655" PostTypeId="2" ParentId="25653" CreationDate="2016-11-27T18:00:35.260" Score="3" Body="&lt;p&gt;A simple approach:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F = eye(7); % gives you the Kronecker delta part&#xA;for mu = -3:3&#xA;  for nu = -3:3&#xA;    isum = 0; % temporary variable for the sum&#xA;    for j = 0:19&#xA;      isum = isum + besselh(mu-nu,7.0*j);&#xA;    end&#xA;    F(4+mu,4+nu) = F(4+mu,4+nu) + (1i)^(nu-mu) * isum;&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For systems larger than 7x7, it might be wise to precompute the bessel function sums for each possible value of $\mu-\nu$ and look up the desired value.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2016-11-27T18:00:35.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25658" PostTypeId="1" CreationDate="2016-11-27T21:07:31.180" Score="5" ViewCount="350" Body="&lt;p&gt;I have an iterative finite difference scheme for the Poisson equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^2 u=-\rho&#xA;$$&#xA;It's the Jacobi method, which has the form (for 1D systems)&#xA;$$&#xA;u^{n+1}_{i} = \frac{1}{2}(u^n_{i+1} + u^n_{i-1} + \rho_i~h^2)&#xA;$$&#xA;Is there a similar scheme for the nonlinear equation&#xA;$$&#xA;\nabla^2u = -\rho ~e^{u-v}&#xA;$$&#xA;I tried the following&#xA;$$&#xA;u^{n+1}_{i} = \frac{1}{2}(u^n_{i+1} + u^n_{i-1} + \rho_i~e^{u^n_i-v_i}~h^2)&#xA;$$&#xA;but couldn't get a converged answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Above, I have given a 1D example. I am solving the 3D version, over a cube. The front and back of the cube have Von Neumann boundary conditions ( equal to 0), and the top, bottom, left, and right have Dirichlet boundary conditions ( also equal to 0 ). My initial guess is $u = 0$ everywhere.&lt;/p&gt;&#xA;" OwnerUserId="7808" OwnerDisplayName="ThomasKelly" LastEditorUserId="1804" LastEditDate="2016-11-28T11:54:18.147" LastActivityDate="2016-11-29T13:57:09.070" Title="Convergence of Jacobi's method for a semilinear elliptic PDE" Tags="|finite-difference|nonlinear-equations|iterative-method|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="25660" PostTypeId="1" AcceptedAnswerId="25661" CreationDate="2016-11-28T13:38:55.573" Score="3" ViewCount="399" Body="&lt;p&gt;I have to solve&#xA;$$\left\{\begin{matrix}&#xA;x\ddot{x}+\frac{3}{2}\dot{x}^2=\frac{1}{\rho}\left (P_v-a_1t^2-a_2t-P_0  \right )\\ &#xA;x(0)=5\cdot10^{-6}\\ &#xA;\dot{x}(0)=0&#xA;\end{matrix}\right.$$&#xA;I do not know if it may be helpful, but note that $a_1=2.488125\cdot 10^{14}$ is very big while the timespan is very small ($t_f=70\cdot 10^{-6}$).&#xA;I get the result without MATLAB returning any error or warning, but this result does not convince me (I expected the peak to be around $t=4\cdot 10^{-5}$). I have already tried switching to ode15s, but the output is the same. What else can I do to see if the output changes or not and then be sure that this result is correct?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function vdot = no_thermal_effect_98(t,v)&#xA;rho = 959.78;&#xA;P0 = 101325;&#xA;Pvap = 94285.313;&#xA;a1 = (P0 - 1800) / (20e-6)^2;    %nondimensional&#xA;a2 = 2 * (1800 - P0) / (20e-6);  %nondimensional&#xA;%                                       &#xA;vdot = zeros(2,1);&#xA;vdot(1) = v(2);&#xA;vdot(2) = -1.5 * v(2) * v(2) / v(1) + 1 / (v(1) * rho) *...&#xA;    (Pvap - P0 - a1 * t^2 - a2 * t);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;run&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x0 = 5e-6; %meters&#xA;tf = 70e-6; %seconds&#xA;[t,v] = ode45(@no_thermal_effect_98,[0,tf],[x0,0]);&#xA;[t,v(:,1)];&#xA;plot(t,v(:,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NEoRi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NEoRi.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us first consider the case where the pressure outside the bubble is constant: $x\ddot{x}+\frac{3}{2}\dot{x}^2=\frac{1}{\rho}\left (P_v-P_0\right )$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $P_0&amp;gt;P_v$ the bubble collapses (i.e. the radius decreases in a monotonic fashion); conversely, if $P_0&amp;lt;P_v$ the bubble grows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let us consider a pressure variation outside the bubble described by the equation $P(t)=a_1t^2+a_2t+P_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5pBdm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5pBdm.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At $40 \mu s$ the pressure outside the bubble is greater than the pressure inside the bubble (here in the picture, indicated by the orange line and equal to $Pv=94285.313 Pa$). That is why I expected the peak, that is, the moment where the bubble started collpasing, would be $40 \mu s$.&lt;/p&gt;&#xA;" OwnerUserId="22288" LastEditorUserId="22288" LastEditDate="2016-11-28T20:03:06.427" LastActivityDate="2016-11-28T20:03:06.427" Title="ODE45: doubts about the result. Correct or not?" Tags="|matlab|ode|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25661" PostTypeId="2" ParentId="25660" CreationDate="2016-11-28T16:29:42.987" Score="8" Body="&lt;p&gt;I ran your &lt;em&gt;same code&lt;/em&gt; in Python&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.integrate import odeint&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;def func(v, t):&#xA;rho = 959.78&#xA;P0 = 101325&#xA;Pvap = 94285.313&#xA;a1 = (P0 - 1800) / (20e-6)**2&#xA;a2 = 2 * (1800 - P0) / (20e-6) &#xA;v1 = v[1];&#xA;v2 = -1.5 * v[1] * v[1] / v[0] + 1 / (v[0] * rho) *\&#xA;(Pvap - P0 - a1 * t**2 - a2 * t)&#xA;return [v1, v2]&#xA;&#xA;&#xA;y0 = 5e-6&#xA;tf = 70e-6&#xA;t = np.linspace(0, tf, 1000)&#xA;y = odeint(func, [y0, 0], t)&#xA;plt.plot(1e5*t, 1e5*y[:, 0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I obtained the same result&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SsMCb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SsMCb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are worried about the numerical values of your constants, maybe you can change the &lt;em&gt;units&lt;/em&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Nondimensionalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-dimensionalize&lt;/a&gt; your equation. For example you can change the time for $\tau = t/t_0$, with $t_0 = 20\times 10^{-6}$. And use the chain rule to make the changes to the derivatives in the left—hand side to obtain something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y y''+\frac{3}{2}y'^2 = \frac{1}{\rho}\left [(P_v - P_0) - (P_0 - P_1)\tau^2 - 2(P_1 - P_0)\tau\right] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $y = x/t_0$, $y' = dy/d\tau$, and $P_1 = 1800$. I tried this version and gave me the &lt;em&gt;same&lt;/em&gt; answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In both cases, I scaled the axes by $10^5$.&lt;/li&gt;&#xA;&lt;li&gt;It might be better to non-dimensionalize your equation using some knowledge of the physics in your problem. You can check reference 1 to get some ideas.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UJJx5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UJJx5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Kudryashov, Nikolay A., and Dmitry I. Sinelshchikov. &lt;a href=&quot;https://arxiv.org/pdf/1409.6699v1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Analytical solutions of the Rayleigh equation for empty and gas-filled bubble.&quot;&lt;/a&gt; Journal of Physics A: Mathematical and Theoretical 47.40 (2014): 405202.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2016-11-28T19:22:08.057" LastActivityDate="2016-11-28T19:22:08.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25662" PostTypeId="1" AcceptedAnswerId="25681" CreationDate="2016-11-28T17:03:11.550" Score="7" ViewCount="142" Body="&lt;p&gt;I'm working on an inverse problem for my Ph.D. research, for which I'll write the objective functional as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J(\theta) = E(G(\theta) - u^o)$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\theta$ are the parameters, $G$ is the forward map from parameters to observations, $u^o$ are the observations, and $E$ is some metric of the error. The forward map $G$ comes from solving an elliptic PDE in which $\theta$ is a coefficient, which for definiteness' sake we'll say is the Poisson equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-\nabla\cdot ke^\theta\nabla u = f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general I'll say that the forward map from $\theta$ to $u$ amounts to solving the nonlinear system $F(u, \theta) = 0$. I've been solving this using the adjoint method so far. This can also be written as a constrained optimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J(\theta; u, \lambda) = E(u - u^o) + \langle F(u, \theta), \lambda\rangle$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but computationally all the procedures are the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What both formulations do &lt;em&gt;not&lt;/em&gt; exploit is the fact that the forward map is, itself, another optimization problem, i.e. the solution $u$ of the Poisson equation is the minimizer of the Dirichlet energy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D(u, \theta) = \int_\Omega\left(\frac{1}{2}ke^\theta|\nabla u|^2 - fu\right)dx$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can this inverse problem be reformulated as a multi-objective optimization problem for the pair of objectives $\{E(u - u^o)$, $D(u, \theta)\}$?&lt;/strong&gt; If so, is this useful, either for computational purposes or for proving things?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know a lot about multi-objective optimization, so references to any useful literature would be helpful. All I could turn up on a google search for multi-objective optimization and inverse problems were a bunch of papers about genetic algorithms.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="9667" LastEditDate="2016-11-28T18:22:16.340" LastActivityDate="2016-11-30T15:29:13.553" Title="reformulating inverse problem as multi-objective optimization" Tags="|optimization|pde|inverse-problem|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25663" PostTypeId="1" CreationDate="2016-11-28T23:31:11.483" Score="1" ViewCount="80" Body="&lt;p&gt;Scipy integration allows us to do ode integration one adaptive timestep at a time and do something to it. However, matlab ode needs us to specify a &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/ode45.html#inputarg_tspan&quot; rel=&quot;nofollow noreferrer&quot;&gt;timespan&lt;/a&gt; , and determine the adaptive timestep within, but cannot do things within.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From scipy &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ODE.set_initial_value(y0, tlist[0])&#xA;numoftimes = len(tlist)&#xA;for ii in range(1, numoftimes):&#xA;    # while loop up to tlist[ii]&#xA;    while ODE.t &amp;lt; tlist[ii]:&#xA;        t_prev = ODE.t&#xA;        y_prev = ODE.y            &#xA;        # integrate up to tlist[ii], one step at a time.&#xA;        ODE.integrate(tlist[ii], step=1)&#xA;        #do something to the output&#xA;        ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, in matlab, I failed to do this, mainly &lt;code&gt;step=1&lt;/code&gt; is not available.&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/25448/which-ode-solver-in-matlab-allows-me-to-advance-in-just-one-timestep-only&quot;&gt;Which ODE solver in Matlab allows me to advance in just one timestep only&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attempt is to make the tspan extremely small in matlab ode suite. But this is then not adaptive. In some region the original ode timestep is very big (non-stiff), but at some regions the ode timestep is very small (stiff).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tlist = [0 tf]    &#xA;numoftimes = len(tlist)&#xA;for ii = 1:numoftimes&#xA;     # while loop up to tlist(ii)&#xA;     while t_prev &amp;lt; tlist(ii)&#xA;            t_prev = T(end,:)&#xA;            y_prev = Y(end,:)           &#xA;            # integrate up to tlist(ii), one step at a time.&#xA;            t    = [t_prev,t_prev+0.0000000000001];&#xA;            [T, Y] = ode45(dydt, t, y);&#xA;            #do something to the output&#xA;            ...&#xA;     end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This then is not good as I have, even if the ode45 does integrate with only 1 step&#xA;1. force the timestep to be fixed interval&#xA;2. time-consuming as in some non-stiff region the &quot;original adaptive&quot; timestep needs not be that small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I solve this with matlab ode suite?&lt;/p&gt;&#xA;" OwnerUserId="20759" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2016-11-28T23:31:11.483" Title="Convert scipy integration with one step to matlab integration" Tags="|matlab|ode|scipy|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25666" PostTypeId="2" ParentId="23255" CreationDate="2016-11-29T11:19:30.083" Score="1" Body="&lt;p&gt;On a side note perhaps, I think it's funny how the Ghia paper is still used as the benchmark 35 years later on. It had indeed produced great results for its time, but this being a computational problem means their accuracy was limited by the available computer power of the time and today they seem under-resolved. Actually I don't think Ghia et al. even tried to get more accuracy, since they report running times of minutes, aiming to showcase the efficiency of the multigrid method more than to produce benchmarks I think. Little did they know people would still be comparing against their numbers decades later! But any budget laptop now is more than a million times faster than the mainframe Ghia used and multigrid or not, one can easily do a lot better with even a basic implementation. And since we are using this as a benchmark then why not use the more accurate available solutions now? For up to Re 1000 one can look at &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.454.2675&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; 1998 Botella and Peyret paper and for Re up to 30000 &lt;a href=&quot;http://www.acenumerics.com/the-benchmarks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Botella, O., and R. Peyret. &quot;Benchmark spectral results on the lid-driven cavity flow.&quot; Computers &amp;amp; Fluids 27.4 (1998): 421-433.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22491" LastEditorUserId="9667" LastEditDate="2016-11-29T14:33:23.793" LastActivityDate="2016-11-29T14:33:23.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25668" PostTypeId="2" ParentId="25658" CreationDate="2016-11-29T13:50:17.467" Score="2" Body="&lt;p&gt;The Jaocbi iteration assumes that the right hand size is a constant with iteration. Here's how the error analysis is done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax = b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A = A_1 - A_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_1 x^{k+1} = A_2x^k +b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$[A_1 x^{k+1} = A_2x^k +b] - [Ax = b]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_1\epsilon^{k+1} = A_2\epsilon^k$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\epsilon^{k+1} = (A_1^{-1}A_2)\epsilon^k$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\epsilon^{k+1} = (A_1^{-1}A_2)^k\epsilon^0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as you can see that b has to be constant between iterations for the error analysis to be valid. In addition Jacobi is a slow method because the max eigenvalue for a central scheme like yours is close to 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, Jacobi MAY or MAY NOT work for a right hand side that depends on the solution as in your case and worse will converge very slowly, if it does at all. Your best bet right now, I think, is to use a method with better convergence. Maybe try SOR or even the newer Krylov subspace methods such as Bi Conjugate Gradient Stabilized. &lt;/p&gt;&#xA;" OwnerUserId="16735" LastEditorUserId="16735" LastEditDate="2016-11-29T13:57:09.070" LastActivityDate="2016-11-29T13:57:09.070" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25669" PostTypeId="2" ParentId="25649" CreationDate="2016-11-29T20:05:30.670" Score="5" Body="&lt;p&gt;A simple solution is to apply Simpson's (standard) rule to the first $n-3$ grid points, where $n-3$ is even for $n$ odd, and cover the remaining three gridpoints via the &lt;a href=&quot;https://en.wikipedia.org/wiki/Simpson&amp;#39;s_rule#Simpson.27s_3.2F8_rule&quot; rel=&quot;noreferrer&quot;&gt;Simpson 3/8 formula&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I_{3/8} = \frac{3h}{8}[f(x_{n-3}) + 3f(x_{n-2}) + 3f(x_{n-1}) + f(x_n)].$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both have remainder terms of order $\mathcal O(h^5)$, so it keeps the order of the Simpson integration.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2016-11-29T20:05:30.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25670" PostTypeId="1" AcceptedAnswerId="25675" CreationDate="2016-11-29T20:45:44.943" Score="2" ViewCount="234" Body="&lt;p&gt;A Comsol study using frequency sweep on electric current physics yields only current density as accessible variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the underlying equation used is Ohm's law, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{J} = \sigma  \mathbf{E}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{E}$ is the electric field, $\mathbf{J}$ is the current density, and $\sigma$ is the conductivity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is it that they do not implement $V = R I$ directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the computational difficulty in its implementation ?&lt;/p&gt;&#xA;" OwnerUserId="22424" LastEditorUserId="9667" LastEditDate="2016-11-29T21:39:09.017" LastActivityDate="2016-12-30T11:46:55.190" Title="Why FEM electric analysis gives only access to current density?" Tags="|computational-physics|comsol|electromagnetism|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25671" PostTypeId="1" CreationDate="2016-11-29T21:17:17.453" Score="1" ViewCount="100" Body="&lt;p&gt;I'm a probabilist and need to do a few computations for a rather big linear least squares problem, so I'm trying to optimize the computation as far as is feasible to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In computing the QR factorization of the matrix representation $A$ of an overdetermined system $Ax=b$ using the Householder reflections, the commonly stated algorithms suggest that you compute the outer product $uu^\top$ for some $u \in U^\perp$ and a hyperplane $U$, following a geometric motivation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This yields the Householder reflector $H = I - \frac{2}{\langle u, u\rangle } uu^\top$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in order to eliminate subcolumns of the columns of $A$, multiply $A$ from left with $H$. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The computation of $uu^\top$ requires $m^2$ multiplcations.&lt;/li&gt;&#xA;&lt;li&gt;The direct computation of the product $HA$ requires $m^2n$ multiplications.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The product $AH$ is equivalent (or expands) to $A-\frac{2}{\langle u, u\rangle} u(u^\top A)$ which requires $2mn$ multiplications -- by associativity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might be missing something rather obvious. Still, is there a way to avoid this computation on every iteration and reduce the complexity from $2mn$ flops?&lt;/p&gt;&#xA;" OwnerUserId="22498" LastEditorUserId="22498" LastEditDate="2016-11-29T22:06:43.533" LastActivityDate="2016-11-29T22:06:43.533" Title="QR via Householder: less computationally complex variants?" Tags="|linear-algebra|algorithms|linear-solver|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25675" PostTypeId="2" ParentId="25670" CreationDate="2016-11-30T11:07:50.430" Score="2" Body="&lt;p&gt;They are saying the same thing, at a fundamental level - it's just that the implementation is a bit different. The first equation you noted is valid throughout the system because it solves for the &lt;em&gt;intrinsic&lt;/em&gt; values throughout the system, such as the local current density and local electric potential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second equation you mentioned is a lumped-value equation, that is technically valid throughout the system, but by the way you wrote it, I'm guessing that $\mathbf{V}$, $\mathbf{R}$, and $\mathbf{I}$ have units of voltage ($V$), resistance ($\Omega$), and current ($I$), respectively. The question is then, voltage where? Current where? Resistance where?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When modeling a continuum physics problem it's beneficial to reduce everything into its intrinsic properties instead of extrinsic properties, which means that you end up working with scale-independent quantities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now back to your first equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{J} = \sigma \mathbf{E} $$ is the same as $$ \mathbf{E} = \frac{1}{\sigma} \mathbf{J} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's good to remember that conductivity ($\sigma$, with units Siemens/meter) is just the reciprocal of resistivity ($\rho$), which has units of $\Omega \cdot m$. Thus&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{E} = \rho \mathbf{J} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is more or less the valid &lt;em&gt;local&lt;/em&gt; equation that an FEM solver would solve at each node in the problem domain, and which most resembles your &lt;em&gt;global&lt;/em&gt; equation of Ohm's law.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more comprehensive answer to how resistivity relates to conductivity, resistance, and conductance, check out the wiki article on exactly this topic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Electrical_resistivity_and_conductivity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Electrical_resistivity_and_conductivity&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19791" LastActivityDate="2016-11-30T11:07:50.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25677" PostTypeId="1" AcceptedAnswerId="25684" CreationDate="2016-11-30T12:09:10.057" Score="0" ViewCount="3476" Body="&lt;p&gt;I am making a code for an 18-node (3x3x2) 3D element FEM. However, even though I am (pretty) sure that all the shape functions are correct and whatnot, whenever I try and invert the stiffness matrix to solve for displacement, I get the warning message from Matlab telling me that my matrix is close to singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a checklist of things I should test for to fix this problem? What kind of error commonly causes these kind of errors? Or is it just too broad to even answer meaningfully?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Thanks for all your input. Sorry for responding so late, but here is the link to my &lt;a href=&quot;https://drive.google.com/drive/folders/0BzaeyQa0ePwvdWxlaFJLOGZqS1E?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;. I originally didn't want to post it, it seemed a bit much for me to ask strangers to subject themselves to such annoying debugging.&lt;/p&gt;&#xA;" OwnerUserId="22445" LastEditorUserId="22445" LastEditDate="2016-12-05T03:29:34.823" LastActivityDate="2021-03-24T23:18:12.330" Title="FEM Stiffness Matrix is always close to Singular or Badly Scaled" Tags="|finite-element|matlab|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25680" PostTypeId="1" CreationDate="2016-11-30T14:26:23.620" Score="1" ViewCount="314" Body="&lt;p&gt;I'm looking for a benchmark for the flow past square cylinder problem in 2D to compare my results. I have found some papers, but it seems that there are different results in different papers. I want to see the streamline and drag coefficient and lift coefficient . So do you know a famous paper which could be seen as a benchmark for others?&lt;/p&gt;&#xA;" OwnerUserId="21055" LastEditorUserId="21055" LastEditDate="2016-12-01T14:45:41.453" LastActivityDate="2017-05-05T12:57:01.767" Title="Flow past square cylinder benchmark in 2D. Famous papers to compare" Tags="|fluid-dynamics|reference-request|benchmarking|lattice-boltzmann-methods|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="25681" PostTypeId="2" ParentId="25662" CreationDate="2016-11-30T15:29:13.553" Score="4" Body="&lt;p&gt;&lt;strong&gt;No.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multi-objective optimization is concerned with the simultaneous optimization of two (or more) competing objectives where you do not wish to decide on the trade-off between the two beforehand. However, this is not the situation you have: Only the exact minimizer of the Dirichlet energy $D$ is a solution to your forward mapping; the actual value of the functional is irrelevant (in particular, any $u$ that does not attain the minimal value is completely worthless). So the only &quot;trade-off&quot; that makes sense is to ensure that $D$ is minimized, and use the remaining freedom (in choosing $\theta$) to minimize $E$. This would lead to a &lt;em&gt;bilevel&lt;/em&gt; optimization problem (&quot;minimize the minimizer&quot;), but since $D$ is strictly convex and thus admits a unique minimizer, you can equivalently replace the optimization problem for $D$ with the first-order necessary condition $F(u)=0$. This is exactly what you are doing with your Lagrangian $J$ -- by taking the supremum over all $\lambda$, you are enforcing minimality of $D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention that you would like to eliminate the adjoint state. In principle, you need &lt;em&gt;some&lt;/em&gt; way of &quot;backpropagating&quot; the difference between state and data back to the parameter space. This is usually provided by the adjoint state $\lambda$; however, if the equation is sufficiently linear in parameter and/or state separately, you might be able to replace the adjoint state with a suitable surrogate. In this context, you might find the following paper useful: &lt;a href=&quot;http://dx.doi.org/10.1088/0266-5611/19/6/010&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1088/0266-5611/19/6/010&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2016-11-30T15:29:13.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25682" PostTypeId="1" AcceptedAnswerId="25690" CreationDate="2016-11-30T17:52:48.407" Score="5" ViewCount="749" Body="&lt;p&gt;I have a number of vectors $v_1, …, v_n$ and another vector $w$, that all are linearly independent, but not orthogonal. Let $V := \mathop{\text{span}}(v_1, …, v_n)$. I need to remove $w$’s projection to $V$ from $w$, i.e., I need to find the vector $\tilde{w}$ such that $\langle \tilde{w},v \rangle = 0 ~∀~ v∈ V$ and $w-\tilde{w} ∈ V$. I need to do this exactly once for a given $V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the scalar product is not the canonical scalar product and has a higher cost than $n$. Touching the scalar product would be extremely tedious, so for the purpose of this question you can consider it a blackbox.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best solution for this so far is to apply Gram–Schmidt orthonormalisation to the collection $v_1, …, v_n, w$, without normalising the last vector, which then is $\tilde{w}$. With other words, I first orthonormalise $v_1, …, v_n$ to $\hat{v}_1, …, \hat{v}_n$ and then separately remove $w$’s projections to $\hat{v}_1, …, \hat{v}_n$ to obtain $\tilde{w}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this, I need to calculate $\frac{n(n-1)}{2}$ scalar products (or norms) just to orthonormalise $V$ as a prerequisite for the last step, which only involves $n$ scalar products. This feels like there could be some more efficient way to do this, but I cannot find one.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2016-12-01T10:26:12.083" LastActivityDate="2016-12-01T16:49:09.110" Title="Efficiently removing projection to subspace without having an orthogonal basis" Tags="|linear-algebra|projection|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="25683" PostTypeId="1" CreationDate="2016-11-30T19:57:51.527" Score="1" ViewCount="66" Body="&lt;p&gt;Most Adaptive Mesh Refinement softwares/logic have cubic blocks. I have explored the BoxLib library and it ONLY supports cubic blocks. To be more precise it supports non-cubic blocks at the coarsest grid level but fails to maintain this non-cubic size when the boxes are refined. &lt;strong&gt;&lt;em&gt;I wanted to know if there are block structured AMR packages/softwares which support non-cubic blocks&lt;/em&gt;&lt;/strong&gt; ?  &lt;/p&gt;&#xA;" OwnerUserId="14303" LastActivityDate="2016-11-30T19:57:51.527" Title="Non-cubic blocks in Adaptive Mesh Refinement" Tags="|finite-difference|adaptive-mesh-refinement|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25684" PostTypeId="2" ParentId="25677" CreationDate="2016-11-30T20:39:51.460" Score="7" Body="&lt;p&gt;My comments apply specifically to a 3D structural finite element but the&#xA;principles are applicable to more general elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A test you will want to execute as part of element development is to calculate&#xA;the eigenvalues and eigenvectors for the element stiffness matrix. For a 3D&#xA;structural element, you should get exactly six eigenvalues that are zero (or very&#xA;close to zero). The eigenvectors for these should be rigid body displacements &#xA;in the three directions and rigid body rotations about the three axes (or linear combinations&#xA;of these motions). If you get more than six, your stiffness matrix doesn't have &#xA;the necessary rank and is therefore defective. It sounds like this might be your&#xA;problem. (However, in general, if you get less than six zero-eigenvalues, your&#xA;formulation is also defective.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what can cause this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most likely cause is that the shape functions are invalid in some way.&#xA;A second possibility is that you are not accurately integrating the terms&#xA;in the stiffness matrix. In my previous post on a similar topic&#xA;(&lt;a href=&quot;https://scicomp.stackexchange.com/questions/25644/3d-solid-8-node-fem-matlab-code&quot;&gt;3D Solid 8 Node FEM Matlab Code&lt;/a&gt;),&#xA;I alluded to the problems with using too few integration points. Specifically, the&#xA;problem is that this can cause zero-frequency (zero eigenvalue) displacement modes,&#xA;often referred to as &quot;hourglass&quot; modes due to the shape of the eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple solution is to use enough integration points to integrate the stiffness&#xA;matrix terms exactly. There are also clever numerical tricks that can be employed&#xA;to remove these hourglass modes but still allow the under-integration. Also, interestingly,&#xA;sometimes a mesh of defective elements with appropriate boundary conditions will stabilize&#xA;each other so that a solution of the complete model is possible. But, in general, this&#xA;is not something you want to rely on.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-11-30T20:39:51.460" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="25686" PostTypeId="2" ParentId="25682" CreationDate="2016-12-01T10:17:46.483" Score="4" Body="&lt;p&gt;What about simply computing the projection of $w$ along the complement of $V$?&#xA;$$&#xA;\tilde w = [I - V(VV^T)^{-1}V^T]w,&#xA;$$&#xA;where $V$ is the matrix that has the basis vectors of $V$ as columns. This $\tilde w$ has all the properties that you look for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are in the non standard scalar product, then there is a symmetric positive matrix $M$ that induces this scalar product. In this case the projection reads&#xA;$$&#xA;\tilde w = [I - V(V^TMV)^{-1}V^TM]w.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is OK, if the scalar product is a black box. You only need it's realization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The costs are mainly in the solve with $(V^TMV)$. If CG is applied, there will be a number of scalar product evaluations needed that scales with $n^2$: $\mathcal O(n)$ evaluations times $\mathcal O(n)$ iterations. (read the comments) &lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2016-12-01T14:35:39.067" LastActivityDate="2016-12-01T14:35:39.067" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="25689" PostTypeId="1" AcceptedAnswerId="25697" CreationDate="2016-12-01T13:06:58.217" Score="1" ViewCount="285" Body="&lt;p&gt;I am interested with &lt;code&gt;Map&lt;/code&gt; and &lt;code&gt;Reduce&lt;/code&gt; operations in computer science. I would like to do analogies with Mathematics and especially with Linear Algebra. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Reduce&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;First, could we say that &quot;&lt;code&gt;Reduce&lt;/code&gt;&quot; operation can be translated as a linear form, ie I start from a vector and I get a scalar from the coordinates of the initial vector ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But a linear form is by definition linear, is it always the case for Reduction in computer science ?&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Map&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Secondly, could we say that &quot;&lt;code&gt;Map&lt;/code&gt;&quot; can be translated as an endomorphism, i.e we start from an initial vector (like a list in computer science) and we get also a vector as output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I take &quot;&lt;code&gt;Map&lt;/code&gt;&quot; operation, and &lt;code&gt;f&lt;/code&gt; a function, I have :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f[x_{1}, x_{2} ..., x_{n}] = [ f(x_{1}), f(x_{2}), ..., f(x_{n})]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could write it like this :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f[\vec{X}] = f [\sum_{i=1}^{n}\,x_{i}\,\vec{e_{i}}]= \sum_{i=1}^{n}\,x_{i}\,\vec{f(e_{i})}=\sum_{i=1}^{n}\,x_{i}\,F_{ij}\,\vec{e_{j}}=[f(x_{1}), f(x_{2}), ..., f(x_{n})]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I get : $f(x_{j})=\sum_{i=1}^{n}\,F_{ij}\,x_{i}$ with $F_{ij}$ called the matrix of change basis of the &lt;code&gt;endomorphism &quot;f&quot;&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't get this relation above, well known between a change of basis (Matrix $F_{ij}$) and coordinates into 2 basis :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{j} = F_{ij}\,x'_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to make correct connections between Linear algebra and &quot;Map,Reduce&quot; operations ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If someone could help to Conceptualize these operations, this would be nice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: don't hesitate to transfer this question to mathematics exchange community if necessary.&lt;/p&gt;&#xA;" OwnerDisplayName="user14831" LastActivityDate="2016-12-02T15:51:12.950" Title="Connections between Map Reduce operations and Linear Algebra" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ClosedDate="2016-12-07T08:45:33.887" ContentLicense="CC BY-SA 3.0" />
  <row Id="25690" PostTypeId="2" ParentId="25682" CreationDate="2016-12-01T16:43:35.053" Score="2" Body="&lt;p&gt;&lt;em&gt;Converted to an answer from my comments to Jan's answer.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fix dimensions and notation, let us say that $V$ is a $m\times n$ matrix with the vectors $v_i$ as columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Jan notes, we have to compute&#xA;$$&#xA;\tilde w = [I - V(V^TMV)^{-1}V^TM]w,&#xA;$$&#xA;where $M$ is the symmetric $m\times m$ matrix associated to the scalar product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it were possible to solve this problem with $O(n)$ black-box calls to the scalar product $M$, like the OP asks, then it would mean that we can solve the version of the problem in which $M=I$ in $O(mn)$, since now each scalar product costs $O(m)$. This seems too much to ask for, since the usual algorithms to compute projections (QR, SVD, solving the normal equations $(V^TV)^{-1}V^Tw$ explicitly...) all cost $O(mn^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This answer is an amended and corrected version of the comments: in the comments I claim that computing the scalar product costs $O(m)$ for each matrix $M$, but on second thought this is not true in general. For $M=I$ it's fine, though.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2016-12-01T16:49:09.110" LastActivityDate="2016-12-01T16:49:09.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25691" PostTypeId="2" ParentId="25680" CreationDate="2016-12-01T21:06:02.407" Score="1" Body="&lt;p&gt;I believe this paper by Breuer et al. which uses both the lattice Boltzmann and the finite volume method could be of interest for you. There is tremendous information therein and I have used it before as a benchmark. I am not sure if it is THE reference, but it is a good one (and cited above 300 times)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0142727X99000818&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0142727X99000818&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2016-12-01T21:06:02.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25692" PostTypeId="1" CreationDate="2016-12-01T23:45:31.893" Score="3" ViewCount="492" Body="&lt;p&gt;I'm working on a project where I'll need to be able to interpolate scalar potential values at arbitrary points in a 3-D box from a large (potentially millions to billions of points) collection of values that were calculated at points of convenience (roughly randomly and uniformly distributed) that are not on a regular grid.  I'll be computing a trajectory and interpolating values of this potential at points along along the trajectory, so my queries will consistently be at points near the most recently used point.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without much background in this area, my initial idea would be to put the known points into a k-d tree and access the tree each time I need to interpolate a value.  An alternative would be to sort and bin the points and then use them to interpolate values onto a rectangular grid, and then interpolate from the regular grid as needed.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other more specialized data structures and interpolation methods that might be useful for this task?&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-12-08T17:49:02.480" Title="Methods for interpolating from points that are not on a regular grid?" Tags="|interpolation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25693" PostTypeId="1" AcceptedAnswerId="25694" CreationDate="2016-12-02T03:18:10.467" Score="6" ViewCount="262" Body="&lt;h1&gt;The Task&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Let $z_1, z_2, z_3$ be positive real numbers and define&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r(\mathbf{z}):= \int_0 ^\infty (t+z_1)^{-3/2}(t+z_2)^{-3/2}(t+z_3)^{-1/2}\text{d}t.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The task is to compute $r$ numerically in an efficient way. Absolute error on the order of $10^{-8}$ is acceptable. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Attempts at a Solution&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In the notation of DLMF 19.16, $r$ can be related to the general &lt;a href=&quot;http://dlmf.nist.gov/19.16.E9&quot; rel=&quot;nofollow noreferrer&quot;&gt;multivariate hypergeometric function&lt;/a&gt;&#xA;$$&#xA;R_{-a}(\mathbf{b};\mathbf{z} ) := \frac{1}{B(a,a')}\int_0^\infty  t^{a'-1} \prod_{j=1}^n (t+z_j)^{-b_j}\ \text{d}t,&#xA;$$&#xA;where $B$ is the &lt;a href=&quot;http://dlmf.nist.gov/5.12&quot; rel=&quot;nofollow noreferrer&quot;&gt;beta function&lt;/a&gt; and $a'=-a+\sum_{j=1}^nb_j$, under the restrictions that $a,a' &amp;gt; 0$ and $z_j \in \mathbb{C}\setminus (-\infty,0]$, by&#xA;$$&#xA;r(\mathbf{z})= \frac{2}{5}R_{-\frac{5}{2}}\left( \frac{3}{2}, \frac{3}{2}, \frac{1}{2} ; \mathbf{z} \right).&#xA;$$&#xA;There is a generalized &lt;a href=&quot;http://dlmf.nist.gov/19.19.E3&quot; rel=&quot;nofollow noreferrer&quot;&gt;series representation&lt;/a&gt; of $R_{-a}$, but I would guess that it's more expensive to compute than using a canned routine on the indefinite integral above (which is currently too expensive). The DLMF lists several special cases of $R_{-a}$, many of which permit inexpensive recursive evaluations through &lt;a href=&quot;http://dlmf.nist.gov/19.26&quot; rel=&quot;nofollow noreferrer&quot;&gt;addition theorems&lt;/a&gt;, but the integral here is not one of them. I can, however, rearrange $r$ into two forms that are special cases, when $z_1 = z_2$ and when $z_1 \ne z_2$, as follows.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Case 1: $z_1 = z_2$&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Assuming we can pass the limit through the integral,&#xA;$$&#xA;\lim_{z_2 \to z_1} r(\mathbf{z})= \int_0 ^\infty (t+z_1)^{-3}(t+z_3)^{-1/2}\text{d}t&#xA;$$&#xA;which we can rewrite in DLMF 19 notation as $R_{-\frac{5}{2}}\left( 3,\frac{1}{2};z_1,z_3\right)$ and &lt;a href=&quot;http://dlmf.nist.gov/19.25.E43&quot; rel=&quot;nofollow noreferrer&quot;&gt;identify&lt;/a&gt; with the &lt;a href=&quot;http://dlmf.nist.gov/15.2.E2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gauss series&lt;/a&gt; $\,_2F_1(a,b;c;z)$:&#xA;$$&#xA;R_{-a}\left(b_1,b_3;z_1,z_3\right) = z_3^{-a}\ _2F_1\left(a,b_1;b_1+b_3;1-(z_1/z_3)\right)&#xA;$$&#xA;for which there exist &lt;a href=&quot;http://dlmf.nist.gov/15.5.ii&quot; rel=&quot;nofollow noreferrer&quot;&gt;recurrence relations&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Case 2: $z_1 \ne z_2$&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When $z_1 \ne z_2$ we can express $r$ as&#xA;$$&#xA;r=\frac{\chi_1-\chi_2}{z_2-z_1}&#xA;$$&#xA;where&#xA;\begin{eqnarray}&#xA;\chi_1&amp;amp;:=&amp;amp;\int_0^\infty \frac{1}{(t+z_1)\sqrt{(t+z_2)(t+z_3)}}dt \\&#xA;\chi_2&amp;amp;:=&amp;amp;\int_0^\infty \frac{1}{(t+z_2)\sqrt{(t+z_1)(t+z_3)}}dt&#xA;\end{eqnarray}&#xA;are elliptic integrals corresponding to the &lt;a href=&quot;http://dlmf.nist.gov/19.16.E5&quot; rel=&quot;nofollow noreferrer&quot;&gt;special case&lt;/a&gt; $R_D(x,y,z)$ in DLMF 19 notation (up to a constant), for which there is another &lt;a href=&quot;http://dlmf.nist.gov/19.26.E20&quot; rel=&quot;nofollow noreferrer&quot;&gt;recurrence relation&lt;/a&gt;. (See &lt;a href=&quot;https://scicomp.stackexchange.com/questions/15961/numerical-evaluation-of-an-elliptic-integral-in-python&quot;&gt;this question&lt;/a&gt; for details). &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Splitting the Cases Is Unstable&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In principle one could use the first method when $z_1 = z_2$ and the second when $z_1 \ne z_2$; however, in practice I need to compute the integral over a discretization of $z_2$ in which $z_2 \to z_1$, and the method above becomes unstable when $z_2 \simeq z_1$. Is there a way to reduce the integral to a form for which there exists a relatively inexpensive recurrence, as there is for each of the two cases? &lt;/p&gt;&#xA;" OwnerUserId="9599" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2017-04-01T17:28:15.513" Title="Numerical integration of a hypergeometric function" Tags="|quadrature|integration|special-functions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25694" PostTypeId="2" ParentId="25693" CreationDate="2016-12-02T07:27:05.583" Score="7" Body="&lt;p&gt;First of all, from the first paragraph of your attempts at a solution, I assume that the $z_j$ are non-negative? In that case, the integrand has no real problematic points (it's monotonous, decreasing). The only difficulty is the infinite integration range. And then it all depends on the accuracy you want... Do you want machine epsilon precision (or there about) or is it ok for you to aim for $10^{-8}$ or ... ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm impressed by the work you did on finding hypergeometric representations of the integral and trying to find a recurrence for cheap evaluation. However, in this case with such a simple integrand, I would first check if brute force quadrature isn't fast enough. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you to try the so-called double-exponential quadrature for an infinite integral. You can find an overview paper of the double-exponential method and its applications in the paper &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S037704270000501X&quot; rel=&quot;noreferrer&quot;&gt;The double-exponential transformation in numerical analysis &lt;/a&gt;. This paper contains references to the original papers in which the method was developed. A very nice and free (as in free beer) implementation can be found on &lt;a href=&quot;http://www.kurims.kyoto-u.ac.jp/~ooura/intde.html&quot; rel=&quot;noreferrer&quot;&gt;professor Ooura's webpage&lt;/a&gt;. He has &quot;didactic&quot; implementations (so you can map the implementation to the algorithm described in the papers) and a fast implementation (a bit more obscure code). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I quickly (and dirtily) implemented the function above and linked to the C version of the fast implementation, using the &lt;code&gt;intdei&lt;/code&gt; function for infinite range integration. For a result with a relative error tolerance of $10^{-15}$, the code needs typically 100 function evaluations (if $z_j &amp;lt; 1000$, more if $z_j$ are larger) , which in your case are extremely cheap.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;time.h&amp;gt;&#xA;&#xA;#include &quot;intde2.h&quot;&#xA;&#xA;int nfunc;&#xA;double z1, z2, z3;&#xA;&#xA;double f(double x)&#xA;{&#xA;    extern int nfunc;&#xA;    extern double z1,z2,z3;&#xA;&#xA;    nfunc++;&#xA;&#xA;    const double p1 = -3.0/2;&#xA;    const double p2 = -1.0/2;&#xA;    const double f1 = pow(x+z1,p1);&#xA;    const double f2 = pow(x+z2,p1);&#xA;    const double f3 = pow(x+z3,p2);&#xA;&#xA;    return f1*f2*f3;&#xA;}&#xA;&#xA;void main()&#xA;{&#xA;    extern double z1,z2,z3;&#xA;    extern int nfunc;&#xA;    double tiny, aw[8000], i, err;&#xA;&#xA;    const double a = 10000.0;&#xA;&#xA;    srand(time(NULL));&#xA;&#xA;    z1 = (double)rand()/(double)(RAND_MAX/a);&#xA;    z2 = (double)rand()/(double)(RAND_MAX/a);&#xA;    z3 = (double)rand()/(double)(RAND_MAX/a);&#xA;    lenaw = 8000;&#xA;    tiny = 1.0e-307;&#xA;&#xA;    intdeiini(lenaw, tiny, 1.0e-15, aw);&#xA;    nfunc = 0;&#xA;    intdei(f, 0.0, aw, &amp;amp;i, &amp;amp;err);&#xA;    printf(&quot;z1 = %20.12e\nz2 = %20.12e\nz3 = %20.12e\n&quot;,z1,z2,z3);&#xA;    printf(&quot; I =  %20.12e\nerr= %20.12e\nN= %d\n&quot;, i, err, nfunc);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2016-12-02T12:25:35.503" LastActivityDate="2016-12-02T12:25:35.503" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25695" PostTypeId="2" ParentId="1749" CreationDate="2016-12-02T11:19:31.680" Score="1" Body="&lt;p&gt;(Years later) two solvers that handle box constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scipy least_squares&lt;/a&gt;&#xA;has 3 methods, with extensive doc:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;'trf’: Trust Region Reflective&lt;/li&gt;&#xA;&lt;li&gt;'dogbox'&lt;/li&gt;&#xA;&lt;li&gt;'lm': a legacy wrapper for MINPACK, without box constraints.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ceres-solver.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ceres&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2018-02-17T14:01:43.677" LastActivityDate="2018-02-17T14:01:43.677" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25696" PostTypeId="2" ParentId="1159" CreationDate="2016-12-02T11:29:53.207" Score="0" Body="&#xA;&#xA;&lt;p&gt;(Years later) as you know, Levenberg-Marquardt optimizers take a whole vector of functions&#xA;$[f_i]$ and do the summing and squaring inside. Thus&lt;br&gt;&#xA;$\qquad levmar(\ max(\ [f_i ...],\ 0\ )]$&lt;br&gt;&#xA;will see only the $f_i &amp;gt; 0$, which is what you want, is that right ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linear constraints $lin_i(x) \equiv A_i \cdot x - b_i \leq 0$&#xA;have the same property as your $f_i$,&#xA;that only the positive terms matter.&#xA;Thus you can just levmar the lot:&lt;br&gt;&#xA;$\qquad levmar(\ max(\ [f_i ...\ lin_i ...],\ 0\ )]$&lt;br&gt;&#xA;(with $lin_i$ multiplied by 1000 or 1000000.)&#xA;Of course the $Jacobians_i(x)$ have to look at the signs of $f_i$ and $lin_i$,&#xA;but that's easy.&#xA;Also I'd add a term $\|x\|$ to keep $x$ from wandering off to infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Both &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scipy least_squares&lt;/a&gt;&#xA;and &lt;a href=&quot;http://ceres-solver.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ceres&lt;/a&gt;&#xA;do box constraints,&#xA;but I don't see any way of mapping regions $Ax\leq b$ to boxes.)&lt;/p&gt;&#xA;" OwnerUserId="1841" LastActivityDate="2016-12-02T11:29:53.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25697" PostTypeId="2" ParentId="25689" CreationDate="2016-12-02T15:51:12.950" Score="3" Body="&lt;p&gt;There really is no obvious connection between MapReduce and linear algebra. It is true that &lt;em&gt;some&lt;/em&gt; linear algebra operations could be written as MapReduce operations, but the converse is not true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it's useful to give an example of what MapReduce is intended to do. Let's assume you are looking for the closest coffee shop and that you have a database of all businesses in your town. Then the &quot;Map&quot; operation would extract from the database all businesses that are coffee shops, extract their location, and compute the distance to your current location. Then, the &quot;Reduce&quot; operation would take the minimum of these to get you the &lt;em&gt;closest&lt;/em&gt; coffee shop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the &quot;Map&quot; phase is not a linear transformation from database entries to distances; nor is the &quot;Reduce&quot; operation (taking the minimum) a linear operation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-12-02T15:51:12.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25698" PostTypeId="2" ParentId="25692" CreationDate="2016-12-02T16:17:20.977" Score="2" Body="&lt;p&gt;I have some experience working in particle trajectory simulations using interpolated flow field information.&#xA;In our case, we needed to simulate the motion of numerous particle using a single time dependent flow. Consequently, we had to carry out a complex interpolation (position and time) during the trajectories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our conclusion ended up being that it was much more cost effective to map the flow information on a Cartesian grid. Then, using integer arithmetic and the particle position, we could quickly find the position of the particle within the grid. Afterward, we would use trilinear interpolation in order to obtain the flow properties at the position of the particle. This was by far the most robust and efficient approach. Furthermore (and this is important I find) it could be parallelized in distributed memory environment (MPI) quite easily since you only have to decompose the Cartesian grid (and ensure the correct transfer of information for the particles). Since we had to load a considerable amount of data, distributed memory parallelism was essential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary of why we went to a Cartesian mapping:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mapping is expensive, but you only need to do it once per data&lt;/li&gt;&#xA;&lt;li&gt;Allows for easy distributed memory parallelism&lt;/li&gt;&#xA;&lt;li&gt;Allowed us to use trivial trilinear interpolation on the grid&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some issues you might encounter:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Your data structure could become very sparse if you have a complex 3D geometry.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2016-12-08T17:49:02.480" LastActivityDate="2016-12-08T17:49:02.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25699" PostTypeId="1" CreationDate="2016-12-02T17:05:27.310" Score="1" ViewCount="114" Body="&lt;p&gt;In my current problem, I am looking for an algorithm to reconstruct the position of multiple points in the 2D euclidean plane with incomplete distance information, &lt;span class=&quot;math-container&quot;&gt;$d_{ij}=||x_j-x_i||\in\text{dom}_a$&lt;/span&gt;.&#xA;E.g. Information is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\{d_{ii},d_{ij}, d_{ik}, d_{il}\}\in \text{dom}_a$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\{d_{ji}, d_{jj}, d_{jk}\} \in \text{dom}_a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\{d_{jl}\} \in  \text{dom}_b$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;there are more than 2 domains and all domains are non overlapping.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This &amp;quot;binned distance&amp;quot; information is exact and is available of all points.&#xA;I like to find the positions or rather areas with high likelihood, where the points can be found.&lt;/p&gt;&#xA;&lt;p&gt;Currently I think, I have too less information to solve the nonlinear constrained optimization problem. On the other hand, I believe it should be enough to make a Bayesian approach work.&lt;/p&gt;&#xA;&lt;p&gt;Information, I also have but currently don't make use of:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the binned distance information are from time series and &lt;span class=&quot;math-container&quot;&gt;$\max(dx/dt)$&lt;/span&gt; is known. This might be useful for finding the next solution of a given configuration in time.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any suggestions how to obtain these regions of possible positions are appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/SDbYM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SDbYM.png&quot; alt=&quot;Example of forward Problem&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To answer the questions I added a picture of the forward problem. Black dots represents all positions I like to reconstruct. For two points (green and blue) I exemplary plotted their bins (circle) for which I get the information.&lt;/p&gt;&#xA;&lt;p&gt;E.g. For the two points close to the blue center, blue would return:&#xA;&lt;span class=&quot;math-container&quot;&gt;$d_{\text{blue},1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d_{\text{blue},2}$&lt;/span&gt; are within bin 2 where as &lt;span class=&quot;math-container&quot;&gt;$d_{\text{green},1}$&lt;/span&gt; reports within bin 5 and &lt;span class=&quot;math-container&quot;&gt;$d_{\text{green},2}$&lt;/span&gt; within bin 6. I do have the radius information for each bin.&lt;/p&gt;&#xA;&lt;p&gt;All bins are a subset of the real line. Their union represents the real line. Their intersection is empty.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2016-12-05T11:26:40.373" Title="2D Multilateration with constraints as distance information" Tags="|nonlinear-programming|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25702" PostTypeId="1" AcceptedAnswerId="25703" CreationDate="2016-12-02T20:14:44.877" Score="1" ViewCount="178" Body="&lt;p&gt;Consider the 1D poisson equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d^2 u}{dx^2} = -\rho&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with Dirichlet boundary conditions $u(0) = u(l) = g$. Using a finite difference scheme, with a 5-point grid $u_1,u_2,u_3,u_4,u_5$ (excluding boundary points $u_0$ and $u_l$), we get the set of linear equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left( \begin{array}{ccc}&#xA;2 &amp;amp; -1 &amp;amp;  &amp;amp;  &amp;amp;   \\&#xA;-1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  &amp;amp;    \\&#xA; &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;   \\&#xA; &amp;amp;  &amp;amp; -1 &amp;amp; 2 &amp;amp; -1    \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; -1 &amp;amp; 2    \\&#xA;\end{array} \right)\left( \begin{array}{c} u_1 \\ u_2 \\ u_3 \\ u_4 \\ u_5\end{array} \right) = \left( \begin{array}{c} \rho_1+g \\ \rho_2 \\ \rho_3 \\ \rho_4 \\ \rho_5+g\end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: What would the matrix look like if I made $u_3$ a boundary point too?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left( \begin{array}{ccc}&#xA;2 &amp;amp; -1 &amp;amp;  &amp;amp;  &amp;amp;   \\&#xA;-1 &amp;amp; 2 &amp;amp; 0 &amp;amp;  &amp;amp;    \\&#xA; &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp;   \\&#xA; &amp;amp;  &amp;amp; 0 &amp;amp; 2 &amp;amp; -1    \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; -1 &amp;amp; 2   \\&#xA;\end{array} \right)\left( \begin{array}{c} u_1 \\ u_2 \\ u_3 \\ u_4 \\ u_5\end{array} \right) = \left( \begin{array}{c} \rho_1+g \\ \rho_2+g \\ g \\ \rho_4+g \\ \rho_5+g\end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask because I have a 3d system with small but irregular internal boundary regions, and it would be currently more convenient for my purposes to leave them in the matrix (even if it means extra computational cost).&lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2016-12-05T11:43:17.657" Title="Explicitly including boundary points in a set of finite-difference equations" Tags="|finite-difference|poisson|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25703" PostTypeId="2" ParentId="25702" CreationDate="2016-12-02T20:47:50.107" Score="2" Body="&lt;p&gt;If you include the boundary condition directly in the matrix, you will only get the g value at the points where the boundary is prescribed. If we use 5 nodes with the following BCS:&#xA;$$u_1=g_1$$ and $$u_5=g_5$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the matrix resulting from the finite difference will be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left( \begin{array}{ccc}&#xA;1 &amp;amp; 0 &amp;amp;  &amp;amp;  &amp;amp;   \\&#xA;-1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  &amp;amp;    \\&#xA; &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;   \\&#xA; &amp;amp;  &amp;amp; -1 &amp;amp; 2 &amp;amp; -1    \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; 0 &amp;amp; 1   \\&#xA;\end{array} \right)\left( \begin{array}{c} u_1 \\ u_2 \\ u_3 \\ u_4 \\ u_5\end{array} \right) = \left( \begin{array}{c} g_1 \\ \rho_2 \\ \rho_3 \\ \rho_4 \\ g_5\end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, the value of $$u_1$$ and $$u_5$$ follow the Dirichlet BCs and within the domain, the Poisson equation applies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have Neumann boundary conditions or Robin Boundary condition, than line 1 and 5 will have to change to respect the application of these boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the third point is a boundary point and following my example, then the resulting matrix should be  :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left( \begin{array}{ccc}&#xA;1 &amp;amp; 0 &amp;amp;  &amp;amp;  &amp;amp;   \\&#xA;-1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  &amp;amp;    \\&#xA; &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp;   \\&#xA; &amp;amp;  &amp;amp; -1 &amp;amp; 2 &amp;amp; -1    \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; 0 &amp;amp; 1   \\&#xA;\end{array} \right)\left( \begin{array}{c} u_1 \\ u_2 \\ u_3 \\ u_4 \\ u_5\end{array} \right) = \left( \begin{array}{c} g_1 \\ \rho_2 \\ g_3 \\ \rho_4 \\ g_5\end{array} \right)&#xA;$$&#xA;But this is impossible in our case, since this would imply prescribing 3 conditions on a problem which requires 2.&#xA;In 2D, however, you will get lines with strictly diagonal term that appear within the matrix due to Dirichlet BCs.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2016-12-02T21:15:03.453" LastActivityDate="2016-12-02T21:15:03.453" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25704" PostTypeId="1" AcceptedAnswerId="25709" CreationDate="2016-12-03T03:04:09.700" Score="1" ViewCount="160" Body="&lt;p&gt;I &lt;a href=&quot;https://scicomp.stackexchange.com/questions/25658/convergence-of-jacobis-method-for-a-semilinear-elliptic-pde&quot;&gt;recently asked a question&lt;/a&gt; pertaining to the appliciation of Jacobi's method to a semilinear elliptic PDE (Poisson's equation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^2u = -\rho~e^{-u}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more efficient method like the Bi conjugate gradient stabilised method was recommended. I have tested this method out and it is indeed much faster. But I am unsure of what the matrix representation of a semilinear system would look like. For an ordinary linear PDE like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^2u=-\rho&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it looks like&#xA;$$&#xA;\frac{1}{h^2}\left( \begin{array}{ccc}&#xA;2 &amp;amp; -1 &amp;amp;  &amp;amp;  &amp;amp;   \\&#xA;-1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  &amp;amp;    \\&#xA; &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;   \\&#xA; &amp;amp;  &amp;amp; -1 &amp;amp; 2 &amp;amp; -1    \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; -1 &amp;amp; 2    \\&#xA;\end{array} \right)\left( \begin{array}{c} u_1 \\ u_2 \\ u_3 \\ u_4 \\ u_5\end{array} \right) = \left( \begin{array}{c} \rho_1+g \\ \rho_2 \\ \rho_3 \\ \rho_4 \\ \rho_5+g\end{array} \right)&#xA;$$&#xA;where $g$ is the Dirichlet boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: What would the corresponding matrix representation of the set of simultaneous equations for the semilinear case look like? I'm guessing something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{h^2}\left( \begin{array}{ccc}&#xA;2 &amp;amp; -1 &amp;amp;  &amp;amp;  &amp;amp;   \\&#xA;-1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  &amp;amp;    \\&#xA; &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;   \\&#xA; &amp;amp;  &amp;amp; -1 &amp;amp; 2 &amp;amp; -1    \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; -1 &amp;amp; 2    \\&#xA;\end{array} \right)\left( \begin{array}{c} u_1 \\ u_2 \\ u_3 \\ u_4 \\ u_5\end{array} \right) = \left( \begin{array}{c} \rho_1e^{-u_1}+g \\ \rho_2e^{-u_2} \\ \rho_3e^{-u_3} \\ \rho_4e^{-u_4} \\ \rho_5e^{-u_5}+g\end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this doesn't leave me with all $u$ values in a single vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it make sense to do something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Solve the linear case&#xA;$$&#xA;\nabla^2u = -\rho&#xA;$$&#xA;2) Use the resultant $u$ to construct a new linear case&#xA;$$&#xA;\nabla^2u_i = -\rho~C&#xA;$$&#xA;where&#xA;$$&#xA;C = e^{-u_{old}}&#xA;$$&#xA;3) Repeat step 2 until self-consistency is reached.&lt;/p&gt;&#xA;" OwnerUserId="7808" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2016-12-04T01:27:37.017" Title="Finite difference equations representing semilinear elliptic PDE" Tags="|finite-difference|poisson|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25705" PostTypeId="1" CreationDate="2016-12-03T06:14:21.997" Score="1" ViewCount="342" Body="&lt;p&gt;Even though i know that i cannot advance the solver ode23tb, which uses TR-BDF2 itself, in just one timestep, i want to set an upper bound on the integration period after which I do something to output and feed the modified output into ode solver again. The upper bounds serve for limiting the timesteps taken in the integration interval &lt;code&gt;tspan = [t_prev ti[j]]&lt;/code&gt;, (see below), to be as low as possible, hopefully just one. To implement it, i try to do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    y_prev = rho0;&#xA;    t_prev = 0&#xA;    ti = [list of timestep in  TR-BDF2]&#xA;    j = 1;&#xA;    while t_prev &amp;lt; tf          &#xA;         #do something to the output&#xA;          ...&#xA;         # integrate up to adaptive ti in  TR-BDF2 even though inside the interval ode45 has its own timestep&#xA;         tspan = [t_prev ti[j]]&#xA;         j = j + 1;&#xA;         [T, Y] = ode23tb(dydt, tspan, y_prev);&#xA;         t_prev = T(end,:)&#xA;         y_prev = Y(end,:) &#xA;         ....&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But, how can I know what &lt;code&gt;ti&lt;/code&gt; is beforehand? I look at &lt;a href=&quot;https://math.la.asu.edu/~gardner/TRBDF.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://math.la.asu.edu/~gardner/TRBDF.pdf&lt;/a&gt;&#xA;, but do not have an idea on how to do this.&lt;/p&gt;&#xA;" OwnerUserId="20760" LastEditorUserId="20760" LastEditDate="2016-12-03T06:19:48.663" LastActivityDate="2016-12-03T06:19:48.663" Title="How to implement the timestepping in TR-BDF2 using matlab ode solver as end-points (upper bound)?" Tags="|matlab|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25708" PostTypeId="1" AcceptedAnswerId="25725" CreationDate="2016-12-03T15:46:08.843" Score="0" ViewCount="158" Body="&lt;p&gt;I have solved a second order differential equation $\ddot R=f(t,R,\dot R)$ with a piecewise function $$P(t)=\begin{cases}&#xA;P_0 &amp;amp; \text{ if } 0&amp;lt;t\leq40\times10^{-6} \\ &#xA;\frac{(P_0+P_{min})}{2}+\frac{(P_0-P_{min})}{2}\cos(2\pi ft) &amp;amp; \text{ if } 40\times10^{-6}&amp;lt;t\leq60\times10^{-6} \\ &#xA;P_{min} &amp;amp; \text{ if } 60\times10^{-6}&amp;lt;t\leq100\times10^{-6} \\ &#xA;\frac{(P_0+P_{min})}{2}+\frac{(P_0-P_{min})}{2}\cos(2\pi ft) &amp;amp; \text{ if } 100\times10^{-6}&amp;lt;t\leq120\times10^{-6} \\ &#xA;P_0 &amp;amp; \text{ if } t&amp;gt;120\times10^{-6} &#xA;\end{cases}$$&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XR8zY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XR8zY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to plot, on the same graph with two $y$ axes (I guess with the &lt;code&gt;plotyy&lt;/code&gt; command) both $t$ versus $R$ and $t$ versus $P(t)$. At the moment I am only able to plot $t$ versus $R$ &lt;code&gt;plot(t,R(:,1))&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function overshoot()&#xA;R0 = 15e-6;&#xA;tspan = [0 160e-6];&#xA;[t,R] = ode45(@(t,R) DE(t,R,R0), tspan, [R0,0]);&#xA;t = t * 1e6;&#xA;R = R * 1e6;&#xA;plot(t,R(:,1))&#xA;end&#xA;%&#xA;function Rdot = DE(t,R,R0)&#xA;S = 0.073;&#xA;rho = 998;&#xA;mi = 1.005e-3; % kg / (m * s)&#xA;P0 = 101325;&#xA;Pvap = 2329.6;&#xA;Pmin = 1800;&#xA;f = 1e6 / 40;&#xA;%&#xA;Rdot = zeros(2,1);&#xA;Rdot(1) = R(2);&#xA;Rdot(2) = -1.5 * R(2) * R(2) / R(1) + 1 / (R(1) * rho) *...&#xA;    (Pvap - P(t,f,P0,Pmin) + (P0 - Pvap + 2 * S / R0) *...&#xA;    (R0 / R(1))^3 - 2 * S / R(1) - 4 * mi * R(2) / R(1));&#xA;end&#xA;%&#xA;function fval = P(t,f,P0,Pmin)&#xA;if (t &amp;lt;= 40e-6)&#xA;    fval = 101325;&#xA;elseif (t &amp;gt; 40e-6) &amp;amp;&amp;amp; (t &amp;lt;= 60e-6)&#xA;    fval = (P0 + Pmin) / 2 + (P0 - Pmin) / 2 * cos(2 * pi * f * t);&#xA;elseif (t &amp;gt; 60e-6) &amp;amp;&amp;amp; (t &amp;lt;= 100e-6)&#xA;    fval = 1800;&#xA;elseif (t &amp;gt; 100e-6) &amp;amp;&amp;amp; (t &amp;lt;= 120e-6)&#xA;    fval = (P0 + Pmin) / 2 + (P0 - Pmin) / 2 * cos(2 * pi * f * t);&#xA;else&#xA;    fval = 101325;&#xA;end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22288" LastActivityDate="2016-12-05T14:21:13.780" Title="ODE in Matlab: how to plot result and piecewise function" Tags="|matlab|" AnswerCount="1" CommentCount="3" ClosedDate="2016-12-05T20:43:53.750" ContentLicense="CC BY-SA 3.0" />
  <row Id="25709" PostTypeId="2" ParentId="25704" CreationDate="2016-12-03T20:34:50.630" Score="2" Body="&lt;p&gt;Since you don't end up with a linear system, you have to use a nonlinear root finder. The most common choice is Newton's method, because it converges quite quickly (if your initial guess is good enough). You've probably seen Newton's method for a single variable in a single equation. There is an analogous method for systems of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a vector function $f(\mathbf{u}^*) = 0$, and an initial guess $\mathbf{u}^0$, we can approximate the root $\mathbf{u}^*$ by solving the sequence of &lt;em&gt;linear&lt;/em&gt; systems&#xA;$$ J(\mathbf{u}^k)\Delta\mathbf{u}^k = -f(\mathbf{u}^k),$$&#xA;where $\Delta \mathbf{u}^k = \mathbf{u}^{k+1} - \mathbf{u}^{k}$ and $J(\mathbf{u}^k)$, called the &lt;em&gt;Jacobian&lt;/em&gt;, is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ J(\mathbf{u}^k) = \begin{pmatrix} \frac{\partial f_1}{\partial u_1}(\mathbf{u}^k) &amp;amp; \cdots &amp;amp; \frac{\partial f_1}{\partial u_n}(\mathbf{u}^k)\\ \vdots &amp;amp; \ddots &amp;amp; \vdots\\ \frac{\partial f_n}{\partial u_1}(\mathbf{u}^k) &amp;amp; \cdots &amp;amp;\frac{\partial f_n}{\partial u_n}(\mathbf{u}^k)\end{pmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case $f(\mathbf{u})$ is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(\mathbf{u}) = \frac{1}{h^2}A\mathbf{u} - \mathbf{b}(\mathbf{u})$$&#xA;where $A$ is your finite difference matrix and $\mathbf{b}(\mathbf{u})$ is your right hand side. The Jacobian is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ J(\mathbf{u}) = \frac{1}{h^2}A + \begin{pmatrix} \rho_1 e^{-u_1}\\  \mathbf{0} &amp;amp; \ddots &amp;amp; \mathbf{0} \\ &amp;amp; &amp;amp; \rho_n e^{-u_n}\end{pmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your initial guess could come from solving the homogeneous system for example. Then given $\mathbf{u}^0$, we can evaluate $J(\mathbf{u}^0)$ and $f(\mathbf{u}^0)$ and solve for $\Delta \mathbf{u}^0$ using &#xA;$$ J(\mathbf{u}^0)\Delta\mathbf{u}^0 = -f(\mathbf{u}^0).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then find $\mathbf{u}^1$ from the definition of $\Delta \mathbf{u}^0$. We can repeat the process to find $\mathbf{u}^2, \mathbf{u}^3, \cdots$, stopping once $|\Delta \mathbf{u}^k|$ is below some tolerance.&lt;/p&gt;&#xA;" OwnerUserId="9728" LastActivityDate="2016-12-03T20:34:50.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25711" PostTypeId="2" ParentId="25704" CreationDate="2016-12-04T01:27:37.017" Score="1" Body="&lt;p&gt;@H.H. already gives a good answer on how to implement things for the Newton iteration. If you want to get an overview of different approaches to address nonlinear problems, take a look at lectures 31.5 and following here:&#xA;&lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-12-04T01:27:37.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25714" PostTypeId="1" AcceptedAnswerId="25731" CreationDate="2016-12-04T15:35:09.873" Score="3" ViewCount="379" Body="&lt;p&gt;I have implemented a saddle point optimization problem based on the algorithm by Prof. Nesterov, primal-dual[1]. &#xA;Unfortunately, it doesn't work. It seems it is converging. But unfortunately, not to the optimal solution which I get with some out of the box solver. I am confident that out of the box solver's result is correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, based on the arguments by paper, algorithm must work correctly. But somethings gets wrong. I want to check inequalities and conditions in proofs of the paper, But I don't know how to do this?especially because many of the inequalities involved unknown optimal solution ($x^*$).&#xA;Is there any way to check inequalities in the paper, and understand what is wrong which cause my implementation doesn't work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I solved inner optimization problems in algorithm, using the same solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1], Yurii Nesterov, Primal-dual subgradient methods for convex problems , Math. Program. 2009&lt;/p&gt;&#xA;" OwnerUserId="8223" LastEditorUserId="549" LastEditDate="2016-12-06T22:22:10.957" LastActivityDate="2016-12-07T00:39:05.280" Title="How to debug a constrained optimization algorithm?" Tags="|optimization|convex-optimization|constrained-optimization|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25715" PostTypeId="1" CreationDate="2016-12-04T16:24:25.087" Score="1" ViewCount="172" Body="&lt;p&gt;I want to minimize an energy of the form&#xA;$$V_1(\mathbf{x}) + V_2(\mathbf{x})$$&#xA;where $V_1$ is much stiffer than $V_2$. When I try to use LBFGS, convergence is extremely slow, as the solution oscillates wildly in the &quot;stiff directions&quot; while searching for the optimum in the non-stiff directions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way I can modify how I am formulating the problem to avoid this problem? $V_1$ is nonlinear so the stiff directions change during optimization, though from the structure of the problem I can identify them easily at any given iteration.&lt;/p&gt;&#xA;" OwnerUserId="3612" LastActivityDate="2016-12-04T16:24:25.087" Title="preconditioning LBFGS?" Tags="|optimization|nonconvex|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25716" PostTypeId="2" ParentId="25714" CreationDate="2016-12-04T17:10:08.093" Score="7" Body="&lt;p&gt;Writing an optimization routine is like writing any other not too simple program and involves a fair amount of debugging. All things you mention in your question are good checks (i.e. taking care that all the things that theoretically should hold do indeed hold during iterations). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more general checks are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Are you sure that solutions are unique? If not, two solvers may return different optimal solutions. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Primal values. See if the values of the primal objective values converge. If all primal values are finite (i.e. feasible) then the sequence of primal objectives should converge to the minimum of the sequence (not necessarily monotone). You can also compare the primal values of the outcome of your algorithm with the primal value of the black box solver. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Similarly with dual values. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some standard errors:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Errors in the stepsizes. Some times a wrong stepsizes still leads to a convergent method but and in fact solving a different problem. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Wrong order of updates. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Signs, especially in front of data terms. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;All kinds of bugs, e. g. using the name of the iteration counter for some other purpose, accidentally overwriting somethings... &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="302" LastActivityDate="2016-12-04T17:10:08.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25717" PostTypeId="2" ParentId="25714" CreationDate="2016-12-04T17:20:05.820" Score="7" Body="&lt;p&gt;In addition to the things that @Dirk already states, run your algorithm on a problem that is simple enough that you can do each step of the algorithm exactly on a piece of paper. Then compare what your algorithm actually does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, run it on a problem where you try to minimize a low-degree polynomial with linear constraints.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2016-12-07T00:39:05.280" LastActivityDate="2016-12-07T00:39:05.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25719" PostTypeId="1" CreationDate="2016-12-05T05:26:16.417" Score="1" ViewCount="78" Body="&lt;p&gt;Consider a simplex $T$ in $R^d$ with $N_1(T) = \left\{N_i\right\}_{i=0}^{d}\subset P_1^{*}(T)$ be the Lagrange nodal variables (or nodal evaluation). By the Riesz representation theorem, there exist functions $\lambda_{i}^{*}\in P_1(T)$ (which is the representation of $N_i\in P_1(T)$) such that $N_j(\lambda_{i}) = \int_{T} \lambda_{i}\lambda_{j}^{*} = \delta_{ij}$ for each $0\leq i\leq d$. Show that $\lambda_{i}^{*} = \frac{(1+d)^2}{|T|}\lambda_{i} - \frac{1+d}{|T|}\sum_{j\neq i} \lambda_j$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempt:&lt;/strong&gt; I was thinking that $\lambda_i^{*}$ plays a role like a Fourier coefficient, so I multiply both sides by $\lambda_j$ and integrate over $T$. Then LHS would be $\delta_{ji}$, while RHS $= \frac{(1+d)^2}{|T|}\int_{T} \lambda_{i}\lambda_{j} - \frac{1+d}{|T|}\sum_{j\neq i}\int_{T} \lambda_{i}\lambda_{j}$. Now, for $i=j$, then $LHS=1$, and I was thinking of $\int_{T} \lambda_{i}\lambda_{i} = $  the area of region $T$, but then if that's the case, the RHS would be $(1+d)^2,$ which is certainly not equal to $1$ (contradiction). So I don't think my interpretation is correct, but I could not see how to proceed further without knowing the explicit formula for $\lambda_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; Could anyone please help me with this problem? It seems to be such a beautiful identity, yet quite hard to prove. Any thoughts would really be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="18465" LastEditorUserId="18465" LastEditDate="2016-12-06T03:45:27.017" LastActivityDate="2016-12-06T03:45:27.017" Title="Determine Lagrange nodal variables of a simplex $T$" Tags="|finite-element|pde|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25720" PostTypeId="1" AcceptedAnswerId="25734" CreationDate="2016-12-05T10:49:25.513" Score="1" ViewCount="340" Body="&lt;p&gt;The term divergence-free sounds more general and appears particularly in wavelet-related approaches to the Navier-Stokes equations. However I have yet to find a discussion focusing on the distinction, either mathematical or intuition-based, between curl fields and divergence-free fields. Can anyone clarify this distinction?&lt;/p&gt;&#xA;" OwnerUserId="3657" LastActivityDate="2017-02-02T15:03:30.993" Title="What is the difference between the curl component, and the divergence-free component, of a vector field?" Tags="|navier-stokes|wavelet|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25721" PostTypeId="2" ParentId="25702" CreationDate="2016-12-05T11:43:17.657" Score="1" Body="&lt;p&gt;Another option to the answer @BlaisB is to incorporate the constraints via Lagrange multipliers, which basically leads to a linear system which is augmented by two variables,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left( \begin{array}{ccc}&#xA;2 &amp;amp; -1 &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp; 1 &amp;amp;\\&#xA;-1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  &amp;amp;    \\&#xA; &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;   \\&#xA; &amp;amp;  &amp;amp; -1 &amp;amp; 2 &amp;amp; -1    \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; -1 &amp;amp; 2 &amp;amp; &amp;amp; 1  \\&#xA;1 &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp; \\&#xA; &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp;  1 &amp;amp;\\&#xA;\end{array} \right)\left( \begin{array}{c} u_1 \\ u_2 \\ u_3 \\ u_4 \\ u_5 \\ \lambda_1 \\ \lambda_2 \\ \end{array}  \right) = \left( \begin{array}{c} \rho_1 \\ \rho_2 \\ \rho_3 \\ \rho_4 \\ \rho_5 \\ g_1\\ g_2 \end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The found Lagrange multipliers $\lambda_i$ itself are not important and can be discarded after solution. Their only purpose is to keep up the boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of this method is that you don't need to bother with adjustments of the original system. Moreover, it's straightforward to incorporate also more sophisticated boundary conditions, e.g. of Neumann-type (--try that with the explicit method!) &lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2016-12-05T11:43:17.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25723" PostTypeId="2" ParentId="25720" CreationDate="2016-12-05T12:27:31.117" Score="3" Body="&lt;p&gt;For a vector field $\vec{F}$ that satisfies a couple of conditions (smooth, decaying rapidly enough) &lt;a href=&quot;https://en.wikipedia.org/wiki/Helmholtz_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Helmholtz's theorem&lt;/a&gt; states that you can write $\vec{F}$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{F} = \nabla \times \vec{A} - \nabla \phi.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'curl field' $\nabla \times \vec{A}$ is equal to the divergence-free part of $\vec{F}$, since we have:&#xA;$$\nabla \cdot (\nabla \times \vec{A}) = 0,$$&#xA;and conversely, if $\nabla \cdot \vec{F} = -\nabla^2 \phi = 0$, we have $\nabla \phi = 0$ (when $\phi \to 0$ at infinity), so that&#xA;$$\vec{F} = \nabla \times \vec{A}$$&lt;/p&gt;&#xA;" OwnerUserId="18662" LastEditorUserId="18662" LastEditDate="2016-12-06T12:33:44.000" LastActivityDate="2016-12-06T12:33:44.000" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25724" PostTypeId="2" ParentId="25288" CreationDate="2016-12-05T13:19:57.430" Score="1" Body="&lt;p&gt;Your outlet boundary condition is not correct. In your outlet BCs, you have to impose the value of $f_6$, $f_3$ and $f_7$ since these populations are not updated by the streaming step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This paper presents a good example on how to do an outflow BC in LBM :&#xA;&lt;a href=&quot;https://www.researchgate.net/publication/40536594_Outflow_boundary_conditions_for_the_lattice_Boltzmann_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/40536594_Outflow_boundary_conditions_for_the_lattice_Boltzmann_method&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2016-12-05T13:19:57.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25725" PostTypeId="2" ParentId="25708" CreationDate="2016-12-05T14:21:13.780" Score="1" Body="&lt;p&gt;This can achieved by making the parameters &lt;code&gt;P0 Pmin f&lt;/code&gt; global.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function overshoot()&#xA;global P0 Pmin f&#xA;R0 = 15e-6;&#xA;&#xA;tspan = [0 160e-6]&#xA;[t,R] = ode45(@(t,R) DE(t,R,R0), tspan, [R0,0]);&#xA;%t = t * 1e6;&#xA;%R = R * 1e6;&#xA;figure(1)&#xA;plot(t,R(:,1),'r--')&#xA;hold on&#xA;plot(t,P(t,f,P0,Pmin),'o');&#xA;end  &#xA;&#xA;function Rdot = DE(t,R,R0)&#xA;global P0 Pmin f&#xA;S = 0.073;&#xA;rho = 998;&#xA;mi = 1.005e-3; % kg / (m * s)&#xA;P0 = 101325;&#xA;Pvap = 2329.6;&#xA;Pmin = 1800;&#xA;f = 1e6 / 40;&#xA;%&#xA;Rdot = zeros(2,1);&#xA;Rdot(1) = R(2);&#xA;Rdot(2) = -1.5 * R(2) * R(2) / R(1) + 1 / (R(1) * rho) *...&#xA;    (Pvap - P(t,f,P0,Pmin) + (P0 - Pvap + 2 * S / R0) *...&#xA;    (R0 / R(1))^3 - 2 * S / R(1) - 4 * mi * R(2) / R(1));&#xA;end&#xA;&#xA;function fval = P(t,f,P0,Pmin)&#xA;for i=1:length(t)&#xA;if (t(i) &amp;lt;= 40e-6)&#xA;    fval(i) = 101325;&#xA;elseif (t(i) &amp;gt; 40e-6) &amp;amp;&amp;amp; (t(i) &amp;lt;= 60e-6)&#xA;    fval(i) = (P0 + Pmin) / 2 + (P0 - Pmin) / 2 * cos(2 * pi * f * t(i));&#xA;elseif (t(i) &amp;gt; 60e-6) &amp;amp;&amp;amp; (t(i) &amp;lt;= 100e-6)&#xA;    fval(i) = 1800;&#xA;elseif (t(i) &amp;gt; 100e-6) &amp;amp;&amp;amp; (t(i) &amp;lt;= 120e-6)&#xA;    fval(i) = (P0 + Pmin) / 2 + (P0 - Pmin) / 2 * cos(2 * pi * f * t(i));&#xA;else&#xA;    fval(i) = 101325;&#xA;end&#xA;end    &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output plot which now has two $y$ axes, i.e, $R(t)$ and $P(t)$. But the plot doesn't make any sense because the maximum for $R(t)$ is approximately  $8\times10^{-5}$ and for $P(t)$ is $\approx 10\times10^{4}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways the desired combined plot looks like this&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GgTa5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GgTa5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="20267" LastActivityDate="2016-12-05T14:21:13.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25726" PostTypeId="1" CreationDate="2016-12-05T15:05:35.310" Score="1" ViewCount="135" Body="&lt;p&gt;In finite element, I can calculate the Lagrangian interpolation shape functions for each degree of freedom in an element, from the the number of nodal degrees of freedom and the number of nodes required in the element, if I know the data points ($x$,$y$ coordinates) required to generate linear algebraic conditions, which is in number of total degrees of freedom in the element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my problem, for a 2D element I have 6 degree of freedom in each node, namely $\phi, \phi _x, \phi _y, \phi _{xx}, \phi _{yy}$ and $\phi _{xy}$. As the name indicate, the five degrees of freedom are derivative of the $\phi$ degree of freedom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to calculate, shape functions for degree of freedom in a quadrilateral of four nodes, I would select a incomplete polynomial of degree five and $24$ terms. But the complexity will increase, if the number of nodes are more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know that is there any programming library available which can be used for generation of these shape functions for higher nodes?&lt;/p&gt;&#xA;" OwnerUserId="21659" LastEditorUserId="22185" LastEditDate="2016-12-05T19:23:37.237" LastActivityDate="2016-12-05T19:23:37.237" Title="computational tool for higher order Lagrangian interpolation for finite element" Tags="|finite-element|interpolation|fenics|deal.ii|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25727" PostTypeId="1" AcceptedAnswerId="25736" CreationDate="2016-12-05T22:22:47.083" Score="0" ViewCount="244" Body="&lt;p&gt;I am trying to implement a stiff ODE solver that uses step doubling with the backward Euler method but I want to make a modification and I am unsure how to incorporate it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve $y'=f(y), y(a)=d$ and I am using an approximation to $f$ such that $y'\approx F(y),\;t\in (T,T+dt), y(T)=Y$ where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(y)=f(y)+Df(Y)(y(t) - Y).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand how to implement the backward Euler method when we simply have $y'=f(t,y)$ but I am having trouble writing a program to solve the problem here because $F(y)$ is dependent on $y(t)$, which is obviously unknown so the usual formulation of $\Delta x=Df^{-1}(x)f(x)$ doesn't seem to work. I was wondering how I could go about writing a method for this? &lt;/p&gt;&#xA;" OwnerUserId="22570" LastEditorUserId="9667" LastEditDate="2016-12-05T22:41:51.233" LastActivityDate="2016-12-07T14:45:01.763" Title="Slight Modification to Backward Euler Stiff ODE Solver" Tags="|ode|nonlinear-equations|implicit-methods|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25728" PostTypeId="1" CreationDate="2016-12-06T00:40:31.253" Score="0" ViewCount="180" Body="&lt;p&gt;I have a heat transfer equation in a cube in $R^{100}$: $[0,1]\times[0,1]\times[0,1]\dots$:&#xA;$$&#xA;\nabla^2 \varphi = f,&#xA;$$&#xA;with boundary conditions set in a form that in the number of points $p_i$, temperature field should least deviate from observed values $o_i$, or in other words that solution of heat equation should minimise:&#xA;$$&#xA;\sum_{k=0}^{m}|\varphi(p_i) - o_i|^2.&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be pretty straightforward problem in 2-3 dimensional case (assuming problem is well-posed), I've solved it with FEM successfully, but for high dimensional case I cannot even build the grid, let alone do any calculations. (I don't store $f$, I can easily calculate it in any point).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems, I need to employ some grid-less method. I've skimmed google briefly and found two possible venues: to use radial basis functions or use particle methods. Are they applicable in my case? Do my problem feasible at all? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've never worked with high dimensional problems before, so I would like to hear all suggestions and references to the relevant and possibly relevant literature.&lt;/p&gt;&#xA;" OwnerUserId="16620" LastActivityDate="2016-12-06T16:16:44.040" Title="Solving Poisson equation while suffering from the curse of dimensionality" Tags="|reference-request|poisson|high-dimensional|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="25729" PostTypeId="1" CreationDate="2016-12-06T01:08:14.710" Score="1" ViewCount="232" Body="&lt;p&gt;I have a 2D (x,y) scalar advection problem that describes net blowing snow ($q$) transport at a point. This takes the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q = A - F*\nabla\cdot(q {\bf \hat u}),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where A ($kg\cdot m^{-2}\cdot s^{-1}$) is a spatially variable scalar source term, $F$ is fetch (m) and $\hat u$ unit normal parallel to to the wind direction (unitless); $\hat u$ is spatially variable. Initial conditions are [ I believe] $q(x,y)=0$ (no blowing snow, develop steady state) and 0 flux boundary condition. This equation is from the literature, but feels a bit wonky so I'm trying to piece it together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to solve this via an upwind-scheme -- blowing snow transport moves down wind and should only transport snow to cells downwind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, I would like to solve this on a unstructured triangular mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My hang up (and it's likely a dumb one):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;do I discretize this via FVM and then, for each edge, apply an unwind scheme? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ah the moment I'd like to keep it simple (can move to a better upwind method later), and just find the 'upwind' triangle (via dot-product signs at the edge?) and compute the difference with the current cell. Will that work? If not what is the best way to piece this together? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate any help, I feel like this is a silly question, but I can't quite wrap my head around the order in which to do this.&lt;/p&gt;&#xA;" OwnerUserId="12027" LastEditorUserId="12027" LastEditDate="2016-12-07T19:46:17.397" LastActivityDate="2016-12-07T19:46:17.397" Title="Simple steady-state advection problem: do I need FVM with upwind scheme?" Tags="|finite-volume|advection|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="25730" PostTypeId="1" CreationDate="2016-12-06T04:22:28.287" Score="3" ViewCount="132" Body="&lt;p&gt;In my Robust model control, I have got a couple of quadratic Riccati inequalities which need to be solved numerically on MATLAB. My question, Is there function on MATLAB can solve quadratic Riccati inequality or I have to transform it to Linear matrix inequality(LMI) using Schur complement approach and then apply it on MATLAB?. Thank you a lot. &lt;/p&gt;&#xA;" OwnerUserId="22555" LastEditorUserId="21617" LastEditDate="2016-12-07T00:51:14.047" LastActivityDate="2016-12-07T00:51:14.047" Title="Computing Algebraic Riccati inequality" Tags="|linear-algebra|algorithms|matrix-equations|" AnswerCount="0" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25731" PostTypeId="2" ParentId="25714" CreationDate="2016-12-06T08:38:02.253" Score="6" Body="&lt;p&gt;You can use a Test Driven Development (&lt;a href=&quot;https://en.wikipedia.org/wiki/Test-driven_development&quot; rel=&quot;noreferrer&quot;&gt;TDD&lt;/a&gt;), the ideas behind are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;before write code you write the test with  the expected result;&lt;/li&gt;&#xA;&lt;li&gt;every line of code (almost all lines) is under test so when you modify a line you can check if there is the correct behavior or side effects.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With this methodology you can find and prevent some erros described by Dirk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To write significative test from a math point of you follow the advice of Wolfgang Bangerth. You keep a simple case that you can do by hand and where you now the exact solution (in your case the optimal solution $x^*$). I this way you can put under tests the inequalities in the paper,&#xA;and check the individual steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To keep you life easy try to use a unit-test framework (many languages have got it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantages of using TDD emerge in subsequent amendments to the first draft, and you will save time in the future operations to your code.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2016-12-06T08:38:02.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25733" PostTypeId="2" ParentId="25728" CreationDate="2016-12-06T16:16:44.040" Score="2" Body="&lt;p&gt;The short answer is that you can't do this -- it's outside our computational power today. To explain why, think of just building the box itself, where you have one degree of freedom on each vertex. In 100 dimensions, there are $2^{100}\approx 10^{10}$ vertices. That's not far from the size of the biggest finite element computations, and you'll need on the order of $10^5-10^6$ processors to do that. At the same time, all you could do on this one cube is represent the solution as some kind of linear function in each direction -- that's not going to tell you anything about what is really going on; in fact, all it really does is interpolate the boundary values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, try smaller problems. You probably want to look into sparse grids in that case.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-12-06T16:16:44.040" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25734" PostTypeId="2" ParentId="25720" CreationDate="2016-12-06T16:27:05.823" Score="4" Body="&lt;p&gt;@JannisTeunissen gave an answer for infinite domains. However, for finite domains, the Helmholtz decomposition is not unique, and the scalar potential $\phi$ can take many forms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, it is true that the curl part $\nabla\times\vec A$ is of course divergence free. But if $\vec F$ is divergence free, all you know is that $\phi$ has to satisfy $-\Delta \phi=0$, but you don't know anything about its boundary values. In other words, &lt;em&gt;any&lt;/em&gt; choice of boundary values for $\phi$, together with $-\Delta \phi=0$ yields a divergence free $\vec F$; in particular, $\phi$ is not necessarily zero. As a consequence, on finite domains, the set of divergence free vector fields is &lt;em&gt;not&lt;/em&gt; just the set of curls of vector potentials, but substantially larger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, on infinite domains you have that&#xA;$$&#xA;  \{ \vec F \in H_\text{div}: \nabla\cdot\vec F=0 \}&#xA;  =&#xA;  \{ \nabla\times\vec A: \vec A \in H_\text{curl} \}.&#xA;$$&#xA;On the other hand, on bounded domains, you have that&#xA;$$&#xA;  \{ \vec F \in H_\text{div}: \nabla\cdot\vec F=0 \}&#xA;  =&#xA;  \{ \nabla\times\vec A: \vec A \in H_\text{curl} \}&#xA;  \cup&#xA;  \{ \nabla\phi: \phi \in H^1, -\Delta\phi=0 \}.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-12-06T16:27:05.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25735" PostTypeId="1" CreationDate="2016-12-06T21:34:38.933" Score="1" ViewCount="10995" Body="&lt;p&gt;I have used &lt;code&gt;ode45&lt;/code&gt; in Matlab. And, it is my understanding that the 4 and the 5 are for the order of the global and local error, respectively.  So, the global error is on the order of $h^4$ and the local error is on the order of $h^5$, where $h$ is the step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, then there's an option called &lt;code&gt;RelTol&lt;/code&gt;, which you can set by: &lt;code&gt;odeset('RelTol',1e-6)&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is it possible to set the relative tolerance lower than the global or local error in this case?  Or is the name &lt;code&gt;ode45&lt;/code&gt; a bit of a misnomer?&lt;/p&gt;&#xA;" OwnerUserId="27864" LastEditorUserId="7574" LastEditDate="2016-12-07T17:41:58.850" LastActivityDate="2016-12-07T22:45:17.443" Title="What is “tolerance” in ODE45 in Matlab?" Tags="|matlab|ode|error-estimation|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="25736" PostTypeId="2" ParentId="25727" CreationDate="2016-12-07T04:18:21.503" Score="0" Body="&lt;p&gt;Let's assume you have some differential equation that can be written as the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{dy}{dt} = f(t,y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some initial condition $y(a) = d$. We then want to integrate this quantity using Backward Euler, with the fundamental formula being derived in the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align}&#xA;\frac{dy}{dt} &amp;amp;= f(t_{n+1},y_{n+1})\\&#xA;\frac{y_{n+1} - y_{n}}{\Delta t} &amp;amp;= f(t_{n+1},y_{n+1})\\&#xA;y_{n+1} &amp;amp;= y_{n} + \Delta t f(t_{n+1},y_{n+1})&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we might want to make things a little easier than solving nonlinear equations for $y_{n+1}$, if we're willing to add in a little bit of error, by linearizing $f(t_{n+1},y_{n+1})$ around $y_{n}$. We can show the linearization for $f(t_{n+1},y_{n+1})$ about $y_{n}$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(t_{n+1},y_{n+1}) = f(t_{n+1},y_{n} + (y_{n+1} - y_{n})) \approx f(t_{n+1},y_{n}) + \left.\frac{\partial f}{\partial y}\right|_{(t_{n+1},y_{n})}\left(y_{n+1} - y_{n}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we substitute this into the Backward Euler equation, and set $A_n = \left.\frac{\partial f}{\partial y}\right|_{(t_{n+1},y_{n})}$, we can work out the following:&#xA;$$ \begin{align}&#xA;y_{n+1} &amp;amp;= y_{n} + \Delta t f(t_{n+1},y_{n}) + \Delta tA_n\left(y_{n+1} - y_{n}\right) \\&#xA;(I + \Delta t A_n)y_{n+1} &amp;amp;= (I - \Delta t A_n)y_{n} + \Delta t f(t_{n+1},y_{n}) \\&#xA;y_{n+1} &amp;amp;= y_{n} + \Delta t (I - \Delta t A_n)^{-1}f(t_{n+1},y_{n})&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final formula above should be all you need, assuming you can compute $A_n$.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-12-07T14:45:01.763" LastActivityDate="2016-12-07T14:45:01.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25737" PostTypeId="2" ParentId="25735" CreationDate="2016-12-07T08:25:40.947" Score="11" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;And, it is my understanding that the 4 and the 5 are for the order of the global and local error, respectively.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Your understanding is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The local error of a Runge–Kutta method of order $n$ is proportional to $h^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What &lt;code&gt;ode45&lt;/code&gt; does is to estimate the solution (of one step) with two Runge–Kutta methods with local orders of 4 and 5, respectively (hence those numbers). It uses the solution of the 5&lt;sup&gt;th&lt;/sup&gt;-order method to estimate the solution of the ODE and the difference between the solutions from the two methods to estimate the error of the integration. Mind you that this is only an estimate – if you knew the precise error, you would also have already solved the problem precisely.&#xA;To save computation time, the two Runge–Kutta methods are designed such that they use common intermediate results. Other embedded Runge–Kutta methods (i.e., Runge–Kutta methods with two numbers to indicate order) work similarly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error estimate is then used to optimise the step size: If the error is above a given tolerance (for each component), the step size is decreased until the error is below that tolerance. If the error is far below that tolerance, the step size in increased to save time. &lt;a href=&quot;//www.mathworks.com/help/pdf_doc/otherdocs/ode_suite.pdf&quot; rel=&quot;noreferrer&quot;&gt;This tolerance is&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \text{AbsTol}_i + \text{RelTol}·|\hat{y}_i|,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $|\hat{y}_i|$ is the estimated solution.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2016-12-07T22:45:17.443" LastActivityDate="2016-12-07T22:45:17.443" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25738" PostTypeId="1" AcceptedAnswerId="25740" CreationDate="2016-12-07T19:51:45.350" Score="0" ViewCount="2683" Body="&lt;p&gt;I am trying to implement a two level iterator as I will now describe. Basically I have a &quot;vector of vectors&quot; and I want a single iterator that will loop over all the objects in the vector of vectors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code below does this but it is a little ugly and I can't declare the iterator inside the for loop. Thus I declare the iterator before the for loop, then iterate, then delete the iterator, which is klugy. I don't know C++ so I'm a little out of my league here so don't hesitate to offer C++ programming advice. How can this code be fixed so that the iterator can be declared inside the for loop? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// iterator_2.cpp&#xA;//to compile:   g++ iterator_2.cpp&#xA;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;typedef double Object;&#xA;&#xA;typedef vector&amp;lt;Object&amp;gt; ObjectVector;&#xA;vector&amp;lt;ObjectVector*&amp;gt; types;&#xA;&#xA;class ObjectIterator{&#xA; protected:&#xA;&#xA; public:&#xA;&#xA; Object *object;&#xA; ObjectIterator operator++()&#xA;  {object++; //increment object&#xA;    // when current object is at the end for type[s] increment s and set object to the first entry in the next type&#xA;    if(object==&amp;amp;(*(types[s])-&amp;gt;end())){&#xA;      s++;&#xA;      object=&amp;amp;((*types[s])[0]);&#xA;    }&#xA;    return (*this);&#xA;  }&#xA;&#xA; ObjectIterator():null(0){s=0;object= &amp;amp;((*types[0])[0]);};&#xA;&#xA;  int s;&#xA;  ObjectIterator *null;&#xA;&#xA;};&#xA;&#xA;int num_types=2;&#xA;&#xA;int main(){&#xA;&#xA;  int num_objects[num_types];&#xA;  num_objects[0]=3;&#xA;  num_objects[1]=5;&#xA;&#xA;  // initialize objects for each type&#xA;&#xA;  for(int s=0;s&amp;lt;num_types;s++){&#xA;    ObjectVector *objectlist = new ObjectVector;&#xA;&#xA;    for(int i=0;i&amp;lt;num_objects[s];i++){&#xA;      objectlist-&amp;gt;push_back((double)2*i*(s+1));&#xA;    }&#xA;    types.push_back(objectlist);&#xA;  }&#xA;&#xA;&#xA;// use the two-level iterator&#xA;&#xA;  ObjectIterator *OI;&#xA;  OI = new ObjectIterator;&#xA;&#xA;  for(; OI-&amp;gt;s &amp;lt; num_types; ++(*OI))&#xA;    {&#xA;      cout &amp;lt;&amp;lt; *(OI-&amp;gt;object)&amp;lt;&amp;lt;endl;&#xA;    }&#xA;  delete OI;  &#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2119" LastEditorUserId="2119" LastEditDate="2016-12-07T19:59:54.933" LastActivityDate="2016-12-07T23:19:15.907" Title="two level iterators C++" Tags="|c++|vector|" AnswerCount="1" CommentCount="6" ClosedDate="2016-12-08T14:00:20.787" ContentLicense="CC BY-SA 3.0" />
  <row Id="25739" PostTypeId="1" CreationDate="2016-12-07T22:12:17.843" Score="1" ViewCount="152" Body="&lt;p&gt;Consider this BVP with variable coefficients:&#xA;$$-\nabla\cdot(a\nabla u) = f\ in\ \Omega$$&#xA;$$-n\cdot(a\nabla u) = \kappa(u-g_D)-g_N\ on\ \partial \Omega$$&#xA;where $a&amp;gt;0,f,\kappa&amp;gt;0,g_D,g_N$ are given functions. To obtain the variational formulation of this problem, multiply $f=-\nabla\cdot(a\nabla u)$ by a test function $v$ and integrate over the domain $\Omega$:&#xA;$$\int_{\Omega} fv\ dx = \int_{\Omega}-\nabla\cdot(a\nabla u)v\ dx$$&#xA;$$= \int_{\Omega} a\nabla u\cdot \nabla v\ dx-\int_{\partial \Omega}n\cdot (a\nabla u)v\ ds$$&#xA;$$= \int_{\Omega} a\nabla u\cdot \nabla v\ dx-\int_{\partial \Omega} (\kappa(u-g_D)-g_N)v\ ds$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Green's thm was used in the second-to-last equality, and for the final equality we just used the BC. The steps and calculations following this are not a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider the BVP&#xA;$$-\Delta u=f\ in\ \Omega$$&#xA;$$u=\sin{2\pi x_1}\cdot \sin{2\pi x_2}\ on\ \partial \Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following the same idea as above we end up with&#xA;$$\int_{\Omega} fv\ dx = \int_{\Omega} \nabla u\cdot \nabla v\ dx-\int_{\partial \Omega} n\cdot \nabla uv\ ds$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point I am stuck. What should I do about $n$? I can't find any clues on this particular problem anywhere. Should I attempt to &quot;eliminate&quot; $n$ as in the first example (if so, how?), or should I use $n$ directly in further calculations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm just looking for hints on how to handle $n$, no one seems to have had this same problem though.&lt;/p&gt;&#xA;" OwnerUserId="22341" LastActivityDate="2016-12-08T03:24:20.900" Title="Surface Normal for 2D Finite Element Method" Tags="|finite-element|poisson|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25740" PostTypeId="2" ParentId="25738" CreationDate="2016-12-07T22:14:40.027" Score="1" Body="&lt;p&gt;While I don't state this to be an optimal implementation or one taking into account the all edge cases, below is a sample implementation I put together for your task that can give you an idea of how one might approach your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Header File&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#ifndef _container_hpp_&#xA;#define _container_hpp_&#xA;&#xA;#include &amp;lt;vector&amp;gt;&#xA;&#xA;class container {&#xA;public:&#xA;   class iterator {&#xA;   public:&#xA;       iterator();&#xA;       void operator++();&#xA;       double &amp;amp; operator*();&#xA;       bool operator==(const iterator &amp;amp; it);&#xA;       bool operator!=(const iterator &amp;amp; it);&#xA;   private:&#xA;       friend container;&#xA;       container * parent;&#xA;       int count;&#xA;       int r, c;&#xA;   };&#xA;&#xA;   container();&#xA;   container(int rows, int cols);&#xA;   void resize(int rows, int cols);&#xA;   double &amp;amp; operator()(int r, int c);&#xA;   iterator begin();&#xA;   iterator end();&#xA;&#xA;   int nrows() const;&#xA;   int ncols() const;&#xA;&#xA;private:&#xA;    std::vector&amp;lt;std::vector&amp;lt;double&amp;gt;&amp;gt; data;&#xA;    int totalElements;&#xA;    int rows, cols;&#xA;};&#xA;&#xA;&#xA;&#xA;#endif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Implementation File&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;Container.hpp&quot;&#xA;&#xA;container::iterator::iterator():parent(0),r(0),c(0) {&#xA;&#xA;}&#xA;void container::iterator::operator++() {&#xA;    int totRows = parent-&amp;gt;nrows();&#xA;    if( r != totRows ){&#xA;        int totCols = parent-&amp;gt;ncols();&#xA;        if( c == (totCols-1) ){ c = 0; ++r; }&#xA;        else{ ++c; }&#xA;        ++count;&#xA;    }&#xA;}&#xA;double &amp;amp; container::iterator::operator*() {&#xA;    return parent-&amp;gt;data[r][c];&#xA;}&#xA;&#xA;bool container::iterator::operator==(const iterator &amp;amp; it) {&#xA;    return (parent == it.parent &amp;amp;&amp;amp; count == it.count);&#xA;}&#xA;bool container::iterator::operator!=(const iterator &amp;amp; it) {&#xA;    return !this-&amp;gt;operator==(it);&#xA;}&#xA;&#xA;container::container():data(0),totalElements(0),rows(0),cols(0) {&#xA;&#xA;}&#xA;container::container(int rows_, int cols_):data(rows_),totalElements(rows_*cols_),rows(rows_),cols(cols_) {&#xA;    for(int i = 0; i &amp;lt; rows; ++i ){&#xA;        data[i].resize(cols);&#xA;    }&#xA;}&#xA;&#xA;void container::resize(int rows_, int cols_) {&#xA;    rows = rows_; cols = cols_;&#xA;    data.resize(rows);&#xA;    for(int i = 0; i &amp;lt; rows; ++i ){&#xA;        data[i].resize(cols);&#xA;    }&#xA;}&#xA;double &amp;amp; container::operator()(int r, int c) {&#xA;    return data[r][c];&#xA;}&#xA;&#xA;container::iterator container::begin() {&#xA;    iterator it;&#xA;    it.parent = this;&#xA;    it.count = 0;&#xA;    it.r = 0;&#xA;    it.c = 0;&#xA;    return it;&#xA;}&#xA;container::iterator container::end() {&#xA;    iterator it;&#xA;    it.parent = this;&#xA;    it.count = totalElements;&#xA;    it.r = -1;&#xA;    it.c = -1;&#xA;    return it;&#xA;}&#xA;&#xA;int container::nrows() const {&#xA;    return rows;&#xA;}&#xA;int container::ncols() const {&#xA;    return cols;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A sample &lt;code&gt;main.cpp&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;Container.hpp&quot;&#xA;&#xA;int main( int argc, char** argv ){&#xA;    container test(2,2);&#xA;    for(int i = 0; i &amp;lt; test.nrows(); ++i ){&#xA;        for(int j = 0; j &amp;lt; test.ncols(); ++j ){&#xA;            test(i,j) = i + j;&#xA;        }&#xA;    }&#xA;&#xA;    container::iterator it;&#xA;    for( it = test.begin(); it != test.end(); ++it ){&#xA;        printf(&quot;Value = %lf\n&quot;,*it);&#xA;    }&#xA;&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This outputs, as expected, the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Value = 0.000000&#xA;Value = 1.000000&#xA;Value = 1.000000&#xA;Value = 2.000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you might be able to tell, it is implemented to follow a similar convention to the STL C++ library with respect to iterators (at least at a basic level).&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2016-12-07T23:19:15.907" LastActivityDate="2016-12-07T23:19:15.907" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25741" PostTypeId="2" ParentId="25739" CreationDate="2016-12-08T03:24:20.900" Score="4" Body="&lt;p&gt;As @BillGreen already said in one of the comments, the test function $v$ is actually zero on the boundary, so the whole boundary integral simply disappears.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to understand &lt;em&gt;why&lt;/em&gt; $v$ is zero, take a look at lecture 21.5 of mine:&#xA;&lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt; .&#xA;The point is that $v$ is a &lt;em&gt;variation&lt;/em&gt; of the solution $u$; so if $u=g$ on the boundary, then any variation $u+\epsilon v$ must also be equal to $g$ on the boundary, which can only be the case if $v$ is zero on the boundary.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2016-12-08T03:24:20.900" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25746" PostTypeId="1" CreationDate="2016-12-08T16:45:53.113" Score="2" ViewCount="3708" Body="&lt;p&gt;My fortran code contains lines like the following&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integer, parameter :: dbp = kind(1.0d0)&#xA;integer, parameter :: n = 1 000 000&#xA;&#xA;real(dbp) :: x(n), y(n), z(n)&#xA;&#xA;y(:) = x(:) * z(:)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to take advantage (if possible) of some optimised maths libraries to carry out this operation. I have found a lapack routine dgbmv which multiplies a matrix by a vector. This would suit my needs if I create a diagonal matrix such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left( \begin{array}{c} y_1 \\ y_2 \\ \vdots \\ y_n \end{array} \right) = \left( \begin{array}{ccc}&#xA;x_1 &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp;   \\&#xA; &amp;amp; x_2 &amp;amp;  &amp;amp;  &amp;amp;    \\&#xA; &amp;amp;  &amp;amp; \ddots &amp;amp;  &amp;amp;   \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; x_n &amp;amp;     \\&#xA;\end{array} \right)\left( \begin{array}{c} z_1 \\ z_2 \\ \vdots \\ z_n\end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't know if this is the best way to go about calculating x(:)*z(:). Is there a more appropriate way?&lt;/p&gt;&#xA;" OwnerUserId="7808" LastEditorUserId="123" LastEditDate="2016-12-10T18:18:54.547" LastActivityDate="2016-12-10T18:22:05.313" Title="What's the fastest implementation of elementwise vector multiplication in Fortran?" Tags="|linear-algebra|lapack|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25748" PostTypeId="2" ParentId="25746" CreationDate="2016-12-08T18:47:57.723" Score="2" Body="&lt;p&gt;You are doing an element wise operation between two vectors, so if is possible is better use functions designed for vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The library &lt;a href=&quot;https://en.wikipedia.org/wiki/LAPACK&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lapack&lt;/a&gt; is for linear algebra, but it is made for operations or methods with an high level respect this basic operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an optimized form of your code you can try to use some compiler option as BlaisB suggested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An other way can be to use more basic level library, similar &lt;a href=&quot;https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blas&lt;/a&gt; (note the Lapack is builded over Blas).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example MKL has got &lt;a href=&quot;https://software.intel.com/en-us/node/470448&quot; rel=&quot;nofollow noreferrer&quot;&gt;v?mul&lt;/a&gt; that performs a vector-vector element wise multiplication.&#xA;See also this &lt;a href=&quot;https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/288531&quot; rel=&quot;nofollow noreferrer&quot;&gt;question&lt;/a&gt; in MKL forum.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2016-12-08T18:47:57.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25750" PostTypeId="1" CreationDate="2016-12-09T03:10:45.533" Score="1" ViewCount="147" Body="&lt;p&gt;I wrote a hybridizable Discontinuous Galerkin code for transient Navier-Stokes flow,&#xA;with thanks to Martin Kronbichler and Scott Miller's step-51 code in Deal.II. The main algorithm &#xA;is from Nguyen and Peraire's JCP paper(2012).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{\nu}q_{ij}+\frac{\partial u_j}{\partial x_i}=0\;\;in \;\Omega$$&#xA;$$\frac{\partial u_j}{\partial t}+\frac{\partial}{\partial x_i}(q_{ij}+p\delta_{ij}+u_i u_j)=f_{j}\;\;in \;\Omega$$&#xA;$$\frac{\partial u_j}{\partial x_j}=0\;\;in \;\Omega$$&#xA;$$n_i(q_{ij}+p\delta_{ij})|_{\Gamma_N}=g^N_j$$&#xA;$$u_j|_{\Gamma_D}=g^D_j$$&#xA;$$u_j|_{t=0}=u^0_j$$&#xA;The weak form I've taken is as follows,&#xA;$$(e_{ij},\frac{1}{\nu}q_{ij})_{T_h}-(\frac{\partial e_{ij}}{\partial x_i},u_j)_{T_h}+&amp;lt;n_ie_{ij},\hat{u}_j&amp;gt;_{\partial T_h}=0$$&#xA;$$(v_j,\frac{\partial u_j}{\partial t})_{T_h}+(v_j,\frac{\partial}{\partial x_i}(q_{ij}+p\delta_{ij}))_{T_h}-(\frac{\partial v_j}{\partial x_i},u_i u_j)_{T_h}+&amp;lt;v_j,n_i\hat{u}_i \hat{u}_j&amp;gt;_{\partial T_h}+&amp;lt;v_j,S(u_j-\hat{u}_j)&amp;gt;_{\partial T_h}=(v_j,f_j)_{T_h}$$&#xA;$$-(\frac{\partial r}{\partial x_j},u_j)_{T_h}+&amp;lt;r,n_j \hat{u}_j&amp;gt;_{\partial T_h}=0$$&#xA;$$(1,p)_{K}=\bar{p}A_K,\;\;\forall K\in T_h$$&#xA;and globally,&#xA;$$&amp;lt;\mu_j,n_iq_{ij}+pn_j+S(u_j-\hat{u}_j)&amp;gt;_{\partial T_h}=&amp;lt;\mu_j,g^N_j&amp;gt;_{\Gamma_N}$$&#xA;$$&amp;lt;1,\hat{u}_jn_j&amp;gt;_{\partial T_h}=0$$&#xA;$$(1,\bar{p})_{T_h}=0\;\;(if\; \Gamma_N=\emptyset)$$&#xA;Now my code was tested with the 2D lid-driven cavity case, up to Re=1000. All&#xA;tests passed with steady mode. But once the time derivatives are turned on,&#xA;the code only converged to the steady result with large time step,e.g. dt=100.&#xA;If the time step is small, such as 1.e-2, the solution diverged. In all cases,&#xA;the stablization coefficient tau is kept constant as 5.0 .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, the less the time step, the more unstable the code. This really contradicts &#xA;againt our common understanding of instability and convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, the time marching was done by backward Euler scheme. The Picard&#xA;iteration was used for linearization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am just wondering if there's any other aspect in the stability of HDG method I should&#xA;pay attention to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any possible suggestion will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="22604" LastEditorUserId="22604" LastEditDate="2016-12-09T04:01:45.943" LastActivityDate="2016-12-09T04:01:45.943" Title="Diverged HDG solution for 2D incompressible Navier-Stokes test case at SMALL time step. Why?" Tags="|finite-element|convergence|incompressible|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25751" PostTypeId="2" ParentId="25746" CreationDate="2016-12-09T04:04:51.627" Score="0" Body="&lt;p&gt;Rephrasing what other people have said, you can treat diagonal matrices as vectors, which will reduce the memory required to store them, and the number of computations unless you are using sparse routines. You can follow similar approaches with other structured matrices like tridiagonal matrices.&lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2016-12-09T04:04:51.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25752" PostTypeId="2" ParentId="19065" CreationDate="2016-12-09T04:44:22.987" Score="2" Body="&lt;p&gt;You may be able to formulate your problem as a binary integer program, which even though in general they are NP hard, some instances can be solved very quickly through branch and cut algorithms (if the problem is linear). If your problem is non linear there are MINLP solvers that can give you an optimal answer. Nonetheless I must say that if your problem is particularly hard the most appropriate way to proceed will be metaheuristic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest that you attempt a linear or quadratic formulation of you problem and try some of the MILP/MIQP solvers available out there. If you are having issues formulating your problem in this framework check some linear programming/integer programming modeling books or post a question here.&lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2016-12-09T04:44:22.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25753" PostTypeId="2" ParentId="24242" CreationDate="2016-12-09T05:19:16.070" Score="2" Body="&lt;p&gt;As I know, a natural setup for FEM solution of incompressible flow is :&#xA;velocity specified at inlet and homogeneous condition at outlet for &#xA;$$\vec{n}\cdot(-\nu\nabla u+pI)|_{\Gamma_N}=0$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the latter can be applied for facets of outlet as so-called natural BCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps you.&lt;/p&gt;&#xA;" OwnerUserId="22604" LastActivityDate="2016-12-09T05:19:16.070" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25754" PostTypeId="1" CreationDate="2016-12-09T11:19:53.917" Score="1" ViewCount="408" Body="&lt;p&gt;I am trying to solve numerically the following 1D EBM:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C\frac{\partial T[x,t] }{\partial t} - \frac{\partial }{\partial x}\left ( D(1-x^2)\frac{\partial T[x,t] }{\partial x} \right ) + I[T] = S[x,t](1-A[T])$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $C$ and $D$ are constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use the Crank-Nicolson method to solve it but I am unsure how to implement it with all of the non-linear terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would the Crank-Nicolson discretization of this equation be?&lt;/p&gt;&#xA;" OwnerUserId="22606" LastEditorUserId="4405" LastEditDate="2021-08-23T13:04:05.873" LastActivityDate="2021-08-23T15:15:10.583" Title="Numerical solution of non-linear heat-diffusion PDE using the Crank-Nicolson Method" Tags="|pde|finite-difference|nonlinear-equations|crank-nicolson|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25755" PostTypeId="1" CreationDate="2016-12-09T14:31:01.873" Score="3" ViewCount="145" Body="&lt;p&gt;Can the Multigrid method be applied to solve a linear system of equations in the form $Ax=b$, that is not necessarily related to differential equations?&lt;/p&gt;&#xA;" OwnerUserId="22609" LastEditorUserId="123" LastEditDate="2016-12-13T10:43:26.533" LastActivityDate="2016-12-13T10:43:26.533" Title="Can I use multigrid to solve linear algebra problems that do not arise from a differential equation?" Tags="|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25756" PostTypeId="2" ParentId="25755" CreationDate="2016-12-09T15:02:22.517" Score="6" Body="&lt;p&gt;There are &quot;Algebraic Multigrid Methods&quot; that can be applied to general systems $Ax=b$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2016-12-09T15:02:22.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25757" PostTypeId="1" CreationDate="2016-12-09T17:58:54.650" Score="0" ViewCount="265" Body="&lt;p&gt;I implemented a FEM solver in MATLAB for Poisson's equation in 3D, using hexahedron and sparse matrix for the Laplacian. I was using the backslash but now I have to use a few iterative methods (GMRES with &lt;em&gt;ilu&lt;/em&gt; for example). With $N_x$, $N_y$ and $N_z$ (in general, $N$) being the number of elements (50, 100 and 200 as test cases) I have some problems. For $N=50$ it takes more than $600$ seconds using &lt;strong&gt;cgs&lt;/strong&gt; with incomplete &lt;strong&gt;LU&lt;/strong&gt; preconditioner. For $N=100$ it takes more than 60 minutes and it is still running. I did not set the tolerance or the maximum number of iterations since it does not seem to affect the timing problem. Is this possible or it should not take that long? Is there a way to speed it up?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[L,U] = ilu(Kb);&#xA;[u_h,flag,relres,iter,resvec] = cgs(Kb,fb,[],[],L,U);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;em&gt;Kb&lt;/em&gt; is the Laplacian and &lt;em&gt;fb&lt;/em&gt; the right-hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="22185" LastActivityDate="2016-12-10T13:01:21.113" Title="Iterative methods to solve linear system in 3D FEM" Tags="|finite-element|matlab|iterative-method|preconditioning|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25760" PostTypeId="1" CreationDate="2016-12-10T00:34:05.743" Score="1" ViewCount="210" Body="&lt;p&gt;I am solving turbulent pipe flow with an explicit MacCormack scheme and Wilcox k-omega model. The laminar version of the code had three distinct stability criteria which worked fine after implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the turbulent version seems to have stability problem. I tried to upgrade the stability criteria for the new set of equations, but it seems that solution eventually blows up. I have noticed the omega values grow very large through all the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know if there are new stability conditions with k-omega model, or I should merely extend MacCormack stability to the new equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I appreciate it if anyone can introduce a paper or book that discusses two-equation models stability.&lt;/p&gt;&#xA;" OwnerUserId="21174" LastEditorUserId="549" LastEditDate="2016-12-10T03:25:09.670" LastActivityDate="2016-12-10T03:25:09.670" Title="Stability of the explicit MacCormack Scheme to solve the Navier Stokes equations with Wilcox's K-Omega Turbulence Model" Tags="|fluid-dynamics|stability|numerical-modelling|explicit-methods|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25761" PostTypeId="2" ParentId="25757" CreationDate="2016-12-10T13:01:21.113" Score="2" Body="&lt;p&gt;Your stiffness matrix is symmetric, positive definite so the Krylov-based solver of choice for this problem is conjugate gradients. You want code similar to this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;L = ichol(K);&#xA;u = pcg(K, b, tol, maxIter, L, L');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;assuming you want to use an incomplete factorization as a preconditioner. The stiffness matrix for Poisson's equation on a cube is relatively well-conditioned so you might want to experiment with no preconditioner or a very simple preconditioner like diagonal scaling. But, since you say the MATLAB implementation of ichol is quite fast, that is likely to be best choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other Krylov-based iterative algorithms, like cgs and GMRES that you mention, are primarily for unsymmetric or non-positive definite systems.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-12-10T13:01:21.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25762" PostTypeId="1" CreationDate="2016-12-10T15:40:27.953" Score="1" ViewCount="1450" Body="&lt;p&gt;I have been exploring finite differences and heat transfer using the 2D heat equation to further expand my knowledge. So far I think it is going well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am running into some confusion around grid spacing for the finite difference method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically if I have a metal plate measuring 100mm x 100mm it would seem natural to establish a grid spacing of 1mm which would yield 100 x 100 nodes or so (I haven't fully explored the different grid spaces yet to see what works best). If I run the simulation I get results I would expect after a certain time given the initial conditions. At time, t, the system has evolved to a certain state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to try a larger metal plate made of the exact same material with the exact same properties. The only difference is that it is larger, 1m x 1m. For this plate I would establish the grid spacing at 1cm yielding the same number of notes, 100 x 100. I might be naive here but I can't see how the scale would affect the finite differences. I would expect the plate to take longer to reach the same state (or pretty close, provided the initial conditions and boundary conditions were the same) as the smaller plate. But from what I see in my calculations is that it doesn't. I might be missing something and would appreciate any pointers to the relevant literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the purposes of this question there are constant heat sources defined at the boundaries, nothing fancy. It really is a very basic problem definition. I had set the $\alpha$ value to 1 do that I could investigate the algorithm. I didn't pay attention to the units that it was defined with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think my problem is in regards to the unit that are used to define the spacing. In the initial problem I had the units in my mind as millimeter. I really should have paid closer attention to the units used in defining the heat capacity, material density and thermal conductivity. They were defined with meters. To me this would imply that I need to define the grid spacing in terms of meters. This would give me the difference that I am looking for. For example the first plate of 100mm x 100mm would have a spacing of 1mm = 0.001m. The second plate of 1m x 1m would have a spacing of 1 cm = 0.01m. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this logic correct?&lt;/p&gt;&#xA;" OwnerUserId="22615" LastEditorUserId="22615" LastEditDate="2016-12-11T14:07:06.087" LastActivityDate="2016-12-16T18:42:46.830" Title="Finite Difference Grid Spacing and Scaling" Tags="|finite-difference|grid|heat-transfer|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25763" PostTypeId="2" ParentId="25762" CreationDate="2016-12-10T16:16:54.763" Score="2" Body="&lt;p&gt;Let's assume the following heat transient heat transfer equation in 1D :&#xA;$$&#xA; \frac{\partial T} {\partial t} =  \alpha \frac{\partial^2 T}{\partial x^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take its finite difference approximation  using an explicit time scheme we obtain :&#xA;$$&#xA; \frac{T_i^{t+\Delta t} - T_i^{t}}{\Delta t} =  \frac{\alpha}{(\Delta x)^2} (T_{i-1}^{t}-2T_i^{t}+T_{i+1}^{t})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So as you see, the diffusion you will obtain in a time step will be proportional to : $ \frac{\Delta t\alpha}{(\Delta x)^2}$. As such, if you increase the size of your domain, then $\Delta x$ will be larger (if you use the same number of points). Therefore, in a single time step, the diffusion of the temperature will be less pronounced because the diffusion term of your finite difference scheme will be smaller.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2016-12-10T16:16:54.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25765" PostTypeId="2" ParentId="25746" CreationDate="2016-12-10T18:22:05.313" Score="7" Body="&lt;p&gt;&lt;strong&gt;The cost of the multiplication is almost insignificant compared to the cost of loading the data from memory (and writing it back)&lt;/strong&gt;.  If you're worried about performance, you should be thinking about data locality.  Perform more flops with each of your $x_i$ and $z_i$ values (if that's possible) when you load them from memory and before proceeding on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Setting up a diagonal matrix will at best make no difference, but is more likely to disastrously degrade performance.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-12-10T18:22:05.313" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25766" PostTypeId="1" CreationDate="2016-12-11T08:01:31.010" Score="1" ViewCount="117" Body="&lt;p&gt;Is there routine in standard BLAS or LAPACK to set strictly-upper triangular part (the part above the diagonal) of a matrix to alpha? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not want to change diagonal elements so &lt;code&gt;laset&lt;/code&gt; is not a good candidate.&lt;/p&gt;&#xA;" OwnerUserId="3978" LastActivityDate="2017-02-06T07:37:24.183" Title="Set strictly upper triangular part of a matrix to alpha using BLAS or LAPACK" Tags="|lapack|blas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25767" PostTypeId="2" ParentId="25457" CreationDate="2016-12-11T20:43:43.920" Score="3" Body="&lt;p&gt;If numerical stability and robustness is your &lt;em&gt;only&lt;/em&gt; concern, you might try computing the SVD of $A$, and using that to compute the product. Specifically, let the SVD of $A$ be:&#xA;$$A = U \Sigma V^T.$$&#xA;Then&#xA;$$B A^{-1}B = (BV)\Sigma^{-1}(U^TB).$$&#xA;This may even work in cases where $A$ is ill-conditioned, but $B A^{-1} B$ is not, due to the spaces on which $B$ and $B^T$ span, and their interaction with the dominant subspaces of $A$. And if it fails, you can look and see how exactly it is failing based on which modes that are being amplified, then decide what to do from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing the SVD is, of course, very expensive. In this case avoid divide and conquer SVD algorithm implementations since those, although often faster to compute, tend to be less accurate for the very small singular values.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2016-12-11T20:43:43.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25768" PostTypeId="1" AcceptedAnswerId="25772" CreationDate="2016-12-12T02:42:13.763" Score="1" ViewCount="1713" Body="&lt;p&gt;In continuum mechanics, we define the Right-Cauchy-Green Deformation Tensor as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\boldsymbol{C}=\boldsymbol{F}^T\boldsymbol{F}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to compute $\frac{\partial \boldsymbol{C}^{-1}}{\partial \boldsymbol{C}}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Wikipedia Article covering &lt;a href=&quot;https://en.wikipedia.org/wiki/Tensor_derivative_(continuum_mechanics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tensor Derivatives&lt;/a&gt;, it says that in index form &#xA;$&#xA;\left(\frac{\partial \boldsymbol{C}^{-1}}{\partial \boldsymbol{C}}\right)_{IJKL} = -C^{-1}_{IK}C^{-1}_{LJ}$. (Correct?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm confused is this part here found in a paper regarding continuum mechanics (but also appears in many other places) : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nmcNP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nmcNP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I am trying to figure out the step where we go from the Second-Piola-Kirchhoff to the Material Tangent Tensor. The Second Piola-Kirchhoff Stress $\boldsymbol{S}$ is related to the Material Tangent Tensor $\mathbb{C}$ by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbb{C} = 2\frac{\partial \boldsymbol{S}}{\partial \boldsymbol{C}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we should be able to use the tensor derivative given in the wikipedia article to get what we need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't get is how the derivative $\left(\frac{\partial \boldsymbol{C}^{-1}}{\partial \boldsymbol{C}}\right)_{IJKL}$ got splitted into two parts, $(IKLJ)$ and $(ILJK)$. There is only one term in the Wikipedia article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody correct my misunderstanding?&lt;/p&gt;&#xA;" OwnerUserId="22629" LastActivityDate="2017-01-11T06:54:26.273" Title="Derivative of the inverse of the Right Cauchy-Green Deformation Tensor wrt itself" Tags="|linear-algebra|tensor|solid-mechanics|calculus|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25769" PostTypeId="2" ParentId="25768" CreationDate="2016-12-12T06:45:39.057" Score="2" Body="&lt;p&gt;We know that&#xA;$$C C^{-1} = I$$&#xA;If we perturb C a little bit, by a matrix $X$, and recompute the inverse, the result should stay roughly the same. In symbols:&#xA;$$(C + X) \left(C^{-1} + \underbrace{\frac{\partial C^{-1}}{\partial C}\cdot X}_{\text{matrix}}\right) \approx I,$$&#xA;where &quot;$\frac{\partial C^{-1}}{\partial C}\cdot X$&quot; is the first order correction to $C^{-1}$ due to the perturbation by $X$ (we want to determine this).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiply things out:&#xA;$$I + C \frac{\partial C^{-1}}{\partial C}\cdot X + X C^{-1} +  \underbrace{X \frac{\partial C^{-1}}{\partial C}\cdot X}_{O(X^2) \approx 0} = I.$$&#xA;Then drop the term that is quadratic in $X$ (since tends to zero much faster than terms that are linear in $X$), and solve the resulting equation for $\frac{\partial C^{-1}}{\partial C}\cdot X$. You get the following formula:&#xA;$$\frac{\partial C^{-1}}{\partial C}\cdot X = -C^{-1} X C^{-1}.$$&#xA;This completely describes how to compute the directional derivative of of the map $C \mapsto C^{-1}$ in direction $X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Viewing $X$ as a free variable, this formula describes a 4'th order multilinear map. When written out in components, it becomes:&#xA;\begin{align}&#xA;\overbrace{\frac{\partial C^{-1}}{\partial C}}^{4'\text{th order tensor}} : (u,X ,v) \mapsto &amp;amp; -u^T C^{-1} X C^{-1} v \\&#xA;=&amp;amp; -\sum_{IJKL} \left( C^{-1} \right)_{IK} \left( C^{-1} \right)_{LJ} u_I X_{KL} v_J,&#xA;\end{align}&#xA;and hence the tensor $-\left( C^{-1} \right)_{IK} \left( C^{-1} \right)_{LJ}$ represents $\frac{\partial C^{-1}}{\partial C}$ when all quantities are expressed in a given basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can make everything in this post rigorous by keeping track of the error committed at each step, then using Taylor's theorem and making a limiting argument.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2017-01-11T06:54:26.273" LastActivityDate="2017-01-11T06:54:26.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25771" PostTypeId="1" CreationDate="2016-12-12T22:04:09.390" Score="5" ViewCount="195" Body="&lt;p&gt;I have a multi-variate, continuous function $f$ from $R^n$ to $R$, which I can query for its output for any input. &#xA;I would like to create interpolation polynomial for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one-dimensional case choosing points is relatively straight forward. &#xA;I have error estimation:&#xA;$$&#xA;f(x)-H(x)={\frac  {f^{{(K)}}(c)}{K!}}\prod _{{i}}(x-x_{i})^{{k_{i}}},&#xA;$$&#xA;and to choose the interpolation points, I need to solve corresponding minmax problem, as I'm looking to minimize the maximum error for a given number of interpolation points. I'm willing to go with any class of  multivariate polynomials. I'm even willing to go with something other that polynomials if helps me interpolate my function, takes into account derivatives and provides theoretical guarantees (for example, in form of error bound).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In multi-variate case I cannot find such error bounds in the literature nor I unable to derive them myself. Any advice on how should I pick the interpolation points for this function?  &lt;/p&gt;&#xA;" OwnerUserId="16620" LastEditorUserId="16620" LastEditDate="2016-12-13T17:58:21.930" LastActivityDate="2016-12-13T17:58:21.930" Title="How to optimally choose points for multivariable Hermite interpolation?" Tags="|optimization|interpolation|error-estimation|polynomials|grid|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25772" PostTypeId="2" ParentId="25768" CreationDate="2016-12-13T00:16:22.857" Score="7" Body="&lt;p&gt;Nick Alger gives a nice explanation. Here is another one, possibly slightly simpler because it avoids the &quot;should stay roughly the same&quot; part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you want to compute the derivative of any matrix function $X=X(C)$ with regard to entry $C_{ij}$:&#xA;$$&#xA;  \frac{\partial X}{\partial C_{ij}}.&#xA;$$&#xA;In other words, you ask how the matrix $X$ changes as you change $C_{ij}$ a bit. This can easily be answered by considering that $XX^{-1}=I$, independent of any of the elements of $C$. In other words,&#xA;$$&#xA;  \frac{\partial XX^{-1}}{\partial C_{ij}}=0&#xA;$$&#xA;because $XX^{-1}=I$ does not change if you modify $C_{ij}$. But by the product rule, you have&#xA;$$&#xA;  \frac{\partial XX^{-1}}{\partial C_{ij}}=&#xA;  X\frac{\partial X^{-1}}{\partial C_{ij}}&#xA;  +&#xA;  \frac{\partial X}{\partial C_{ij}}X^{-1}&#xA;  = 0&#xA;$$&#xA;and consequently&#xA;$$&#xA;  X\frac{\partial X^{-1}}{\partial C_{ij}}&#xA;  = -&#xA;  \frac{\partial X}{\partial C_{ij}}X^{-1}&#xA;$$&#xA;or equivalently, assuming that the matrix $X$ is invertible:&#xA;$$&#xA;  \frac{\partial X^{-1}}{\partial C_{ij}}&#xA;  = -&#xA;  X^{-1} \frac{\partial X}{\partial C_{ij}}X^{-1}.&#xA;$$&#xA;This is true for &lt;em&gt;any&lt;/em&gt; function $X=X(C)$. In particular, it is true if you consider $X(C)=C$, in which case you get&#xA;$$&#xA;  \frac{\partial C^{-1}}{\partial C_{ij}}&#xA;  = -&#xA;  C^{-1} \frac{\partial C}{\partial C_{ij}}C^{-1}.&#xA;$$&#xA;In component notation, this can be written as&#xA;$$&#xA;  \frac{\partial (C^{-1})_{kl}}{\partial C_{ij}}&#xA;  = -&#xA;  C^{-1}_{km} \frac{\partial C_{mn}}{\partial C_{ij}}C^{-1}_{nl}.&#xA;$$&#xA;Now, the derivative of $C_{mn}$ with regard to $C_{ij}$ is only nonzero if $m=i,n=j$, and in that case it is in fact one. So&#xA;$$&#xA;  \frac{\partial C_{mn}}{\partial C_{ij}}&#xA;  = \delta_{mi}\delta_{nj},&#xA;$$&#xA;with the Kronecker delta symbol, which then finally yields this:&#xA;$$&#xA;  \frac{\partial (C^{-1})_{kl}}{\partial C_{ij}}&#xA;  = -&#xA;  C^{-1}_{ki} C^{-1}_{jl}.&#xA;$$&#xA;This also explains what exactly the various indices in your formula correspond to.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="1744" LastEditDate="2016-12-14T10:24:26.317" LastActivityDate="2016-12-14T10:24:26.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25774" PostTypeId="2" ParentId="23560" CreationDate="2016-12-13T17:11:46.157" Score="0" Body="&lt;p&gt;Since your problem can be expressed as a set of independent ordinary differential equations with $x$ being the time variable, you could use the &lt;a href=&quot;http://www.boost.org/doc/libs/1_62_0/libs/numeric/odeint/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boost Odeint&lt;/a&gt; library. You should find in easy to modify the examples given in the tutorial for the purposes of your problem.&lt;/p&gt;&#xA;" OwnerUserId="22217" LastActivityDate="2016-12-13T17:11:46.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25777" PostTypeId="1" AcceptedAnswerId="30775" CreationDate="2016-12-14T19:57:36.313" Score="0" ViewCount="317" Body="&lt;p&gt;Once again, I am not entirely sure how to describe what I am looking for, hence I have a hard time finding answers using Google or any other literal search method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have a time series:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(x_1, y_1), (x_2, y_2), (x_3, y_3), (x_4, y_4),\ldots, (x_n, y_n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I also have 2 key values: $V_1$ and $V_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find a continuous decomposition of the time series such that each segment's average of its $y$ values is either closest to $V_1$ or to $V_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a trivial example to illustrate the point:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Time series: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(0, 1), (1, 1), (2, 1), (3, 4), (4, 4), (5, 4), (6, 4)\\&#xA;V_1 = 1,\quad V_2 = 4$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;best&lt;/em&gt; continuous segmentation of the time series is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$s_1=\begin{bmatrix}(0, 1) &amp;amp; (1, 1) &amp;amp; (2, 1)\end{bmatrix}\\&#xA;s_2=\begin{bmatrix}(3, 4) &amp;amp; (4, 4) &amp;amp; (5, 4)&amp;amp;(6, 4)\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the average value of $s_1$'s $y$ values is $1$ and the &quot;distance&quot; between $1$ and $V_1$ is zero.&#xA;Similarly, the average value of $s_2$'s $y$ values is $4$ and the &quot;distance&quot; between $4$ and $V_2$ is zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generalizing, each segment would either be determined to be closer to $V_1$ or closer to $V_2$, the algorithm would minimize the sum of distances (between each segment's average $y$ values and either $V_1$ or $V_2$) and the sequence of segments would equal the time series without gaps or overlaps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ouch!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that even comprehensible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you think of an algorithm making that possible? (regardless of complexity for now)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Updates based on initial answers:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Segmentation could include as many segments as necessary to minimize the overall distance.&#xA;Another trivial example to illustrate this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (0, 1), (1, 1), (2, 1), (3, 4), (4, 4), (5, 4), (6, 4), (7, 1), (8, 1) $$&#xA;$$ V_1 = 1 \quad V_2 = 4 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the following segmentation is perfect:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$s_1=\begin{bmatrix}(0, 1) &amp;amp; (1, 1) &amp;amp; (2, 1)\end{bmatrix}\\&#xA;s_2=\begin{bmatrix}(3, 4) &amp;amp; (4, 4) &amp;amp; (5, 4) &amp;amp; (6, 4)\end{bmatrix}\\&#xA;s_3=\begin{bmatrix}(7, 1) &amp;amp; (8, 1)\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overall distance between each segment's average value and its proximal key is zero.&lt;/p&gt;&#xA;" OwnerUserId="22351" LastEditorUserId="20688" LastEditDate="2017-04-02T17:48:38.307" LastActivityDate="2019-01-24T23:02:10.423" Title="Full segmentation of a linear space using a clustering algorithm" Tags="|algorithms|hpc|clustering|" AnswerCount="3" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="25778" PostTypeId="2" ParentId="25777" CreationDate="2016-12-15T00:01:40.577" Score="0" Body="&lt;p&gt;This sounds hard when you ask it like that, but it actually isn't. Due to the continuity, one could always do a linear search over the whole array. At each iteration a new value from the sequence is picked and the means of both segments are updated. Because the mean could drastically change by inclusion/exclusion of one element, we cannot early stop and have to traverse the whole array. Therefore, we exhaustively iterate over all the elements (all possible continuous segmentations) and compute the mean and the distance from $V_1$ and $V_2$. At the end, we take the one which minimizes the sum of these distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because running mean could be computed in $O(1)$ time, the computational cost of this is $O(N)$. Please correct me if I'm wrong.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2016-12-15T00:01:40.577" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25780" PostTypeId="1" AcceptedAnswerId="25785" CreationDate="2016-12-15T02:59:15.177" Score="0" ViewCount="1918" Body="&lt;p&gt;We know that the relaxation time is very important in LBM. I have searched lost of papers, but can't find some systematic introductions about the choice of relaxation time in SRT LBM. Could you give some ways or papers to get a suitable relaxation time?&lt;/p&gt;&#xA;" OwnerUserId="21055" LastActivityDate="2016-12-15T13:02:48.887" Title="How to choose the relaxation time in the Lattice Boltzmann Method?" Tags="|fluid-dynamics|stability|lattice-boltzmann-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25781" PostTypeId="1" AcceptedAnswerId="25784" CreationDate="2016-12-15T07:49:24.563" Score="4" ViewCount="1007" Body="&lt;p&gt;I have the following problem. The domain is $(0,1)$ and we consider a uniform triangulation on $\hat{\Omega}$ with elements $K_i = [i/N,(i+1)/N]$ and $X_h^1$ the linear finite element space. I wrote the following affine map to the reference element $\hat{K} = [0,1]$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x = x_i + \gamma (x_{i+1} - x_i),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the following basis function $\phi(x)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{cases}&#xA;  \frac{x-x_{i-1}}{x_i-x_{i-1}} &amp;amp; x_{i-1} \leq x \leq x_i \\&#xA;  \frac{x_{i+1}-x}{x_{i+1}-x_i} &amp;amp; x_i \leq x \leq x_{i+1} \\&#xA;  0,&#xA; \end{cases}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;end the following expression for the interpolant:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Pi_h^1 f(x) = \sum_{i=0}^{N+1} f(x_i) \phi_i(x). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I interpolated the function $f(x) = cos(2 \pi x)$ with a small program in Python and now I have a problem with the $L^2$-error defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ ||f - \Pi_h^1 f(x)||^2 = \int_0^1 \left (f - \Pi_h^1 (f) \right )^2 dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be decomposed into the sum of the integrals over each element of the mesh and each integral is computed on the reference element using the affine map. I have a problem with the part. I did something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum_{i=0}^{N_e} \left (\int_0^1 (f_e - \Pi_h^1 f_e) \right )^2 dx, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and even if this is correct, I do not know how to compute each integral on the reference element. Can somebody give me an hint, especially on how to compute $f$ on the reference element?&#xA;In particular, I tried to wrote the integrand in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def func_ref(z, x, a, b):&#xA;    return np.power((np.cos(x[a]) - np.cos(x[a]) * (1 - z) +&#xA;    np.cos(x[b]) * z) * (x[b] - x[a]), 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;z&lt;/code&gt; is the variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="22185" LastEditorUserId="22185" LastEditDate="2016-12-16T18:52:35.357" LastActivityDate="2016-12-16T18:52:35.357" Title="$L^2$-error in FEM: how to compute integral over reference element?" Tags="|finite-element|error-estimation|l2-norm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25783" PostTypeId="1" CreationDate="2016-12-15T10:38:34.447" Score="0" ViewCount="104" Body="&lt;p&gt;I have a nonlinear system of algebraic equations with a nonlinear boundary condition of the form &#xA;\begin{equation}&#xA;f'''(x,t) - (f'(x,t))^3 - \alpha(t) f'(x,t) = 0,&#xA;\end{equation}&#xA;at $x =0$, where $\alpha(t)$ is constant in space but evolves in time. I use sided finite differences to approximate it and Newton's method to solve the nonlinear system approximated using finite differences. The solution does not converge when the mesh size decreases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you suggest any other method to approximate the bc?&lt;/p&gt;&#xA;" OwnerUserId="18560" LastEditorUserId="9667" LastEditDate="2016-12-15T23:44:27.850" LastActivityDate="2016-12-15T23:44:27.850" Title="nonlinear boundary condition" Tags="|ode|nonlinear-equations|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25784" PostTypeId="2" ParentId="25781" CreationDate="2016-12-15T10:42:17.450" Score="4" Body="&lt;p&gt;You can write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{0}^1 (f - \Pi_h^1(f))^2 dx = \sum_{i=0}^{N_e} \int_{e} (f - \Pi_h^1(f))^2 dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e = [x_k, x_{k+1}]$ is an interval (triangle as you say). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method A&lt;/strong&gt; (simple, no reference element)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now on each $e$, you know what is $\Pi_h^1(f)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Pi_h^1(f)(x) = f(x_k)\frac{x_{k+1}-x}{x_{k+1}-x_{k}} + f(x_{k+1})\frac{x-x_k}{x_{k+1}-x_{k}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and you can integrate using you favorite numerical quadrature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method B&lt;/strong&gt; (reference element)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the reference element will lead you to this procedure:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You start by defining the quadrature (nodes and weights) on the reference element.&lt;/li&gt;&#xA;&lt;li&gt;Now you want to evaluate $(f - \Pi_h^1(f))^2$ on the reference element. $\Pi_h^1(f)$ is quite easy, because it is a combination of basis functions (whose values you know in the reference element). For $f$, you have no choice but to map the quadrature point to the element $e$ (because this is where $f$ is defined).&lt;/li&gt;&#xA;&lt;li&gt;You gain one term which is the determinant of the inverse of the jacobian of the mapping (because of the coordinate change).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Finally, you can realize that you have evaluated the function in the same points, and the additional term from the jacobian of the mapping in method (B) is the one you use to adapt the weights of the quadrature rule in method (A). So, actually, it's just two ways of seeing exactly the same stuff.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="3684" LastEditDate="2016-12-16T07:35:24.720" LastActivityDate="2016-12-16T07:35:24.720" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25785" PostTypeId="2" ParentId="25780" CreationDate="2016-12-15T13:02:48.887" Score="0" Body="&lt;p&gt;The dimensionless relaxation time $\bar{\tau}$ should be chosen as to obtain the correct fluid viscosity. That is :&#xA;$$\bar{\tau} = \frac{\mu}{\rho c^2 \Delta t} + \frac{1}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c$ is the lattice celerity, $\Delta t$ the time step and $\rho$ the density. Generally $\rho$ is chosen as unity in LBM&#xA;Therefore, generally, you know the physical property you want to simulate ($\mu$ and $\rho$) and you use a value of $\tau$ that you choose. Then, with this choice of value of $\tau$ (which is generally 1 or lower), you obtain the value of the time step $\Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find some explanation on this issue in this paper :&#xA;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999115001916&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0021999115001916&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, many books also treat the subject, like this one :&#xA;&lt;a href=&quot;http://www.worldscientific.com/worldscibooks/10.1142/8806&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.worldscientific.com/worldscibooks/10.1142/8806&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2016-12-15T13:02:48.887" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25787" PostTypeId="1" CreationDate="2016-12-15T21:15:41.250" Score="1" ViewCount="51" Body="&lt;p&gt;For a computer with 8 GB of memory, which is the order of magnitude for the number of particles that could be used in a simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the answer $N=8/6\times 10^9$? or do the particles take up more memory than 6N(3N position +3N velocity)?&lt;/p&gt;&#xA;" OwnerUserId="22664" LastEditorUserId="9667" LastEditDate="2016-12-15T23:25:41.993" LastActivityDate="2016-12-15T23:25:41.993" Title="Number of particles that could be used in a simulation" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25790" PostTypeId="2" ParentId="23258" CreationDate="2016-12-16T18:09:20.197" Score="3" Body="&lt;p&gt;You could try my MATLAB solver &lt;a href=&quot;http://stanford.edu/group/SOL/software/pdco/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDCO&lt;/a&gt; which uses an interior method and will be happy that your n &amp;lt; m.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options.Method = 1 % the default&#xA;d1 = 1e-4&#xA;d2 = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22670" LastEditorUserId="9667" LastEditDate="2016-12-16T20:23:38.660" LastActivityDate="2016-12-16T20:23:38.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25791" PostTypeId="2" ParentId="25762" CreationDate="2016-12-16T18:42:46.830" Score="0" Body="&lt;p&gt;A very good understanding about functioning of $dt$, $dx$ and $\alpha$ in discretized PDE can be developed if you look into the stability analysis for the heat equation. Here is the &lt;a href=&quot;http://www.cems.uvm.edu/~tlakoba/math337/notes_12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; for one such tutorial about that. I hope it will help.&lt;/p&gt;&#xA;" OwnerUserId="20015" LastActivityDate="2016-12-16T18:42:46.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25792" PostTypeId="1" CreationDate="2016-12-16T18:43:08.537" Score="3" ViewCount="135" Body="&lt;p&gt;I'm solving coupled PDEs using finite difference method: Incompressible Navier-Stokes and the divergence-free induction equation (Maxwell's equations) with non-uniform electrical conductivity. The dimensionless equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$1) \partial_t \mathbf{u} + \nabla \bullet (\mathbf{u}\mathbf{u}) = -\nabla p + Re^{-1}\nabla^2 \mathbf{u} + N \mathbf{j} \times \mathbf{B}, \qquad \mathbf{j} = Re_m^{-1} \nabla \times \mathbf{B}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$2) \partial_t \mathbf{B} + Re_m^{-1} \nabla \times (\bar{\sigma}^{-1} \nabla \times \mathbf{B}) = \nabla \times (\mathbf{u} \times \mathbf{B})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$3) \nabla \bullet \mathbf{u} = 0, \qquad \nabla \bullet \mathbf{B} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $Re,N,Re_m$ are the Reynolds, Interaction and magnetic Reynolds numbers. Also, $\bar{\sigma}$ is the dimensionless electrical conductivity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The induction equation becomes stiff / ill-conditioned as $\bar{\sigma}$ approaches zero. In my case, $\bar{\sigma}$ is very small and, as a result, I resort to treating $Re_m^{-1} \nabla \times (\bar{\sigma}^{-1} \nabla \times \mathbf{B})$ implicitly in time using diagonally Preconditioned Conjugate Gradient Method (PCG). I use the limiting stopping criteria among 3 choices:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$N_{\text{max-iterations}}$&lt;/li&gt;&#xA;&lt;li&gt;$\frac{||Ax_k-b||}{||Ax_0-b||} \le \varepsilon_{\text{relative}}$&lt;/li&gt;&#xA;&lt;li&gt;$||Ax_k-b|| \le \varepsilon_{\text{absolute}}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here, $A=I + \Delta t Re_m^{-1} \nabla \times (\bar{\sigma}^{-1} \nabla \times)$. $b = \Delta t (\mathbf{B}^n + \nabla \times (\mathbf{u}^n \times \mathbf{B}^n))$. $x_k$ is the $k$th estimated solution of $\mathbf{B}^{n+1}$ using PCG. Finally, $N,\varepsilon$ are the maximum allowable PCG iterations and tolerances (relative to initial guess and absolute respectively).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Experimenting with stopping criteria and different time steps ($\Delta t$), it seems that there is a non-trivial trade-off in picking $\Delta t$ and stopping criteria to optimize performance, which I'll define as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal P = \frac{\text{convective time per step}}{\text{wall clock time per step}} = \frac{\text{convective time}}{\text{wall clock time}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, I'm trying to reach a steady solution, however, the coupled equations are non-linear, stiff and can (depending on $Re,N,Re_m$) have seemingly long time scales. If I choose $\Delta t$ too large, or perform too few iterations of PCG on Eq. 2), the solution diverges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Are there any methods to optimize the choice of stopping criteria or $\Delta t$ in order to optimize $\mathcal P$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Are there any other tricks I can use to reach steady state faster?&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2016-12-16T18:43:08.537" Title="Finite difference method for coupled PDEs: optimizing performance (time step, iterations per step)" Tags="|optimization|performance|convergence|navier-stokes|electromagnetism|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25793" PostTypeId="2" ParentId="19586" CreationDate="2016-12-16T19:14:43.280" Score="11" Body="&lt;p&gt;Without assuming something special on &lt;code&gt;my_function&lt;/code&gt; choosing &lt;code&gt;multiprocessing.Pool().map()&lt;/code&gt; is a good guess for parallelizing such simple loops. &lt;code&gt;joblib&lt;/code&gt;, &lt;code&gt;dask&lt;/code&gt;, &lt;code&gt;mpi&lt;/code&gt; computations or &lt;code&gt;numba&lt;/code&gt; like proposed in other answers looks not bringing any advantage for such use cases and add useless dependencies (to sum up they are overkill). Using threading as proposed in another answer is unlikely to be a good solution, because you have to be intimate to the GIL interaction of your code or your code should do mainly input/output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said &lt;code&gt;numba&lt;/code&gt; might be a good idea to speed up sequential pure python code, but I feel this  is outside  of the scope of the question.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import multiprocessing&#xA;import numpy as np&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;   #the previous line is necessary under windows to not execute &#xA;   # main module on each child under windows&#xA;&#xA;   X = np.random.normal(size=(10, 3))&#xA;   F = np.zeros((10, ))&#xA;&#xA;   pool = multiprocessing.Pool(processes=16)&#xA;   # if number of processes is not specified, it uses the number of core&#xA;   F[:] = pool.map(my_function, (X[i,:] for i in range(10)) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is some caveats however (but which should not impact most of applications):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;under windows there is no fork support, so an interpreter with main module is launched at the startup of each child, so it might have an overhead (ad it is the reason for the &lt;code&gt;if __name__ == &quot;__main__&quot;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;The arguments and the results of my_function are pickled and unpickled, it might be a too big overhead, see this answer for reducing it &lt;a href=&quot;https://stackoverflow.com/a/37072511/128629&quot;&gt;https://stackoverflow.com/a/37072511/128629&lt;/a&gt; . It also makes  non pickable objects unusables&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;my_function&lt;/code&gt; should not depend on shared states like communicating with global variables because states are not shared between process. pure functions (functions in the mathematical senses) are example of functions which not share states&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18631" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-12-16T19:14:43.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25794" PostTypeId="1" CreationDate="2016-12-17T09:39:47.517" Score="1" ViewCount="130" Body="&lt;p&gt;The following is related to a question a asked a few days back &lt;a href=&quot;https://scicomp.stackexchange.com/questions/25781/l2-error-in-fem-how-to-compute-integral-over-reference-element&quot;&gt;1&lt;/a&gt;, but now I would like to focus on just one part of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have problems computing the integral over the reference element:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{0}^1 (f - \Pi_h^1(f))^2 dx. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I understood how to use the basis functions on the reference element and what the values of $f$ should be for the interpolant. The problem is with the &lt;em&gt;first&lt;/em&gt; $f$ in the integrand. How can I map it? If the affine map is $x = x_j + \gamma (x_{j+1} - x_j)$ I can substitute it into $f$ and obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x(\gamma)) = cos(2 \pi (x_j + \gamma (x_{j+1} - x_j))). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I continue? I was following these beautiful &lt;a href=&quot;http://hplgit.github.io/INF5620/doc/pub/main_fem.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;notes&lt;/a&gt; on page $49$ but that is very simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Python to do this I ended up with this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def func_ref(z, x, a, b):&#xA;    cos_a = np.cos(2*np.pi*x[a])&#xA;    cos_b = np.cos(2*np.pi*x[b])&#xA;    return np.power((cos_a - cos_a * (1 - z) + cos_b * z) * (x[b] - x[a]), 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;strong&gt;x&lt;/strong&gt; is simply the vector containing the nodes, &lt;strong&gt;a&lt;/strong&gt; and &lt;strong&gt;b&lt;/strong&gt; would be the left and right neighbor, &lt;strong&gt;z&lt;/strong&gt; the variable of integration. The first term, $\mathbf{cos_a}$ should be the &lt;em&gt;first&lt;/em&gt; $f$ of the integrand but I do not know/understand how to do this. Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Possible solution&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it correct to write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{0}^1 (cos(2\pi (x_i + \gamma (x_{i+1} - x_i)) - cos(x_i)(1-\gamma) + cos(x_{j+1})\gamma)^2 (x_{i+1} - x_i) d \gamma $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and integrate with respect to $\gamma$?&lt;/p&gt;&#xA;" OwnerUserId="22185" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2016-12-17T10:56:38.097" Title="Integral over reference element in $1$D FEM: how to map the quadrature points?" Tags="|finite-element|python|integration|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25795" PostTypeId="1" AcceptedAnswerId="25796" CreationDate="2016-12-14T17:50:17.367" Score="4" ViewCount="610" Body="&lt;p&gt;In my efforts to write a code for a calculation I have encountered a problem of numerically differentiating a non-linear function at different points on a grid. I used the simple forward finite difference method to differentiate the function but the errors were huge in case when I know the analytical solution of the calculation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to better understand the problem I numerically differentiated $f(x)=1/x$ since the analytical solution is known to be $f'(x)/=-1/x^2$. So I tried to calculate the error between numerical and analytical derivative of $f(x)$ and plot the error against the step  size for various values of $x$. The result is attached as a picture(Both axes are logarithmic!). &lt;a href=&quot;https://i.stack.imgur.com/ectVr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ectVr.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears from the graph the error is significant event for the smallest value of stepsize when $x$ is small. That makes sense to me now because a straight line approximation would fail as the function becomes more and more non-linear with decreasing $x$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, is there a better way of numerically differentiating a non-linear function on the domain of the function other than the finite difference method? The function(s) that I need to numerically differentiate are multivariable functions but the method  can be easily extended from univariable functions to multivariable function I suppose. &lt;/p&gt;&#xA;" OwnerDisplayName="Shaz" LastEditorUserId="123" LastEditDate="2016-12-18T06:57:49.743" LastActivityDate="2016-12-18T06:57:49.743" Title="Numerical differentiation of non-linear functions" Tags="|finite-difference|accuracy|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="25796" PostTypeId="2" ParentId="25795" CreationDate="2016-12-14T21:27:42.020" Score="8" Body="&lt;p&gt;You are missing a minus sign in the derivative, $f'(x)=-\frac{1}{x^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error to first order in $h$ is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{f(x+h)-f(x)}{h} - f'(x)\approx \frac{h}{x^3}$$ which  fits with your data as far as I can see. If $x$ is small, this can be a very big number for fixed $h$. So your problem is purely mathematical and not due to mistakes in your implementation or to numerical rounding problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is of course that $f(x)$ is diverging as $x\to 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can deal with problems of this type by (suggestions are not limited to your specific case of $1/x$ or to numerical differentiation, but generally to dealing with &quot;ugly&quot; not well-behaved functions numerically):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Transforming the function to a nicer looking function (without divergences, etc); In this specific case you could multiply with $x$ to get $g(x)=f(x)x$, differentiate $g(x)$ numerically and use that $f'(x)=(g'(x)-f(x))/x$. Another possible transform could be, that if you are dealing with a complicated function g(x) of which you know the asymptotic behavior at $x=0$ to be like $1/x$ (or any other simple function you can handle analytically), you could instead work with g(x) - 1/x which would not have the singularity.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pick a sufficiently small $h$, perhaps adaptively (smaller as you get closer to the divergence); Eventually, if you are getting too close to 0, you will run into problems with precision or rounding, so this method alone will not get you all the way to the singularity, but if you are fine with values a bit away from the singularity it could suffice.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If it is a real world problem (from physics,...), analyze it from that perspective. Usually things don't go to infinity in the real world, so perhaps the quantity you are dealing with is not the right quantity to analyze.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If this is part of a function, perhaps another part cancels this singularity!?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;See if shifting the singularity with transforms $x\to x_0 - x$ or similar helps.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;See whether other numerical methods are more suited to your task. (E.g. in your case you could try the central difference instead of forward difference).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerDisplayName="user1583209" LastActivityDate="2016-12-16T10:39:49.657" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="25798" PostTypeId="2" ParentId="25795" CreationDate="2016-12-18T06:57:01.603" Score="1" Body="&lt;p&gt;The answer to this question depends on what kind of functions you are actually interested in differentiating.  That said, I will give a basic, general answer here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already pointed out, you are using a finite difference formula &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f'(x) \approx \frac{f(x+h)-f(x)}{h}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that is accurate to first order (i.e., the error is proportional to $h$).  There are two simple ways to improve the accuracy.  The first, as you know, is to decrease $h$.  The only issue with this is that for very small $h$, roundoff error will become significant (you have not come close to such small values in your plot).  The other general way to get more accurate approximations is to use a formula with a higher order of accuracy.  For instance, the centered formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f'(x) \approx  \frac{f(x+h)-f(x-h)}{2h}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is accurate to second order (i.e., the error is proportional to $h^2$).  For a better understanding of finite difference approximations of derivatives, and their accuracy, read any introductory numerical analysis book, or even just &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;.  I particularly like chapter one of &lt;a href=&quot;https://faculty.washington.edu/rjl/fdmbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LeVeque's text&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-12-18T06:57:01.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25802" PostTypeId="2" ParentId="10678" CreationDate="2016-12-19T03:18:53.167" Score="-3" Body="&lt;p&gt;Shape functions are the functions that relate the displacement at any point on the element to the displacement of the nodes of the element. A graph of the shape function vs points on the element shows the deformed &quot;shape&quot; of the element and hence the name shape function.&lt;/p&gt;&#xA;" OwnerUserId="22686" LastActivityDate="2016-12-19T03:18:53.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25804" PostTypeId="1" CreationDate="2016-12-20T08:42:07.960" Score="-1" ViewCount="166" Body="&lt;p&gt;I am working on a flow physics problem (2D), which ultimately boils down to solving Eigen value problem. Even for a grid size of 60 x 60, the matrix in the Eigen value problem end up with 1260 million elements (This matrix has to be solved iteratively!). Sometimes the matrix is sparse and other times it is not depending on the nature of problem to be solved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARPACK&lt;/a&gt; written in Fortran to solve the problem. Surprisingly, MATLAB solves the same Eigen value problems at least 5 times faster than the Fortran counterpart. (And I wonder how!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : Are there any techniques that is known in the computation community to speed up the Eigen solver?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parallel-ized ARPACK is one solution that I tried, but still not up to the level of MATLAB &lt;/p&gt;&#xA;" OwnerUserId="20670" LastEditorUserId="20670" LastEditDate="2016-12-20T15:14:55.600" LastActivityDate="2016-12-22T02:28:33.237" Title="Efficient Eigen Solver" Tags="|eigensystem|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="25805" PostTypeId="1" CreationDate="2016-12-20T15:36:39.703" Score="1" ViewCount="486" Body="&lt;p&gt;$$\frac{d^2 u}{dx^2} +A \frac{d^2}{dx^2}\left(\frac{du}{dt}\right)=B $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve the equation given above. I need to first discretize it by the Method of Lines and then evolve the resulting ODE using a Runge-Kutta method. My question is, how can I reduce this special equation to two different ODEs?&lt;/p&gt;&#xA;" OwnerUserId="22700" LastEditorUserId="9804" LastEditDate="2016-12-20T17:29:33.360" LastActivityDate="2016-12-20T18:23:22.083" Title="implementation of method of line and Runge-Kutta to the given equation" Tags="|pde|finite-difference|runge-kutta|method-of-lines|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25806" PostTypeId="1" CreationDate="2016-12-20T15:51:24.423" Score="1" ViewCount="283" Body="&lt;h2&gt;Disclaimer : This question is related to computational science, but it is a purely a post-processing question so if this is on the wrong board, please tell me&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I would like to postprocess simulation results in an automatic way. Our current simulation software generates output in an ensight gold format. This means that for each simulation a name.case file is generated, which we can then open in paraview and post-process.&#xA;However, the post-processing I have to carry out is very simple, yet will have to be done numerous time. Therefore, I want to carry it without launching the paraview gui in a fully automatic fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am looking for is a way to launch a paraview script (maybe via pvpython?), do the rendering, output a .png and save it in the folder. All of this automatically without needed to launch a GUI or to interact with it. This aspect will eventually be residing inside a bash script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried looking into using pvpython which I could launch from the shell, but I have not found much documentation (is there any unified clear documentation on it?) on even opening ensight case files and post-processing images... &lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2016-12-21T13:23:57.840" Title="Generate paraview plot of an ensight .case without launching the paraview gui" Tags="|python|paraview|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25807" PostTypeId="2" ParentId="25805" CreationDate="2016-12-20T17:47:39.227" Score="0" Body="&lt;p&gt;As an example, you should be able to build up a system of equations to solve for the time derivatives if you use something like the following scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;A \frac{\partial ^2}{\partial x^2}\left(\frac{\partial u_i}{\partial t}\right) &amp;amp;= B - \frac{\partial^2 u_i}{\partial x^2}\\&#xA;%&#xA;\frac{A}{\Delta x^2} \frac{\partial u_{i-1}}{\partial t} - \frac{2A}{\Delta x^2}\frac{\partial u_i}{\partial t} + \frac{A}{\Delta x^2} \frac{\partial u_{i+1}}{\partial t} &amp;amp;\approx B - \frac{u_{i-1} - 2u_{i} + u_{i+1}}{\Delta x^2}\\&#xA;%&#xA;A \frac{\partial u_{i-1}}{\partial t} - 2A\frac{\partial u_i}{\partial t} + A \frac{\partial u_{i+1}}{\partial t} &amp;amp;\approx B \Delta x^2 - \left(u_{i-1} - 2u_{i} + u_{i+1}\right)\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_i$ is the solution at the $i^{th}$ spatially discretized location at some time and where this equation holds for all discretized locations. With the final equation written, you should then be able to put together what should be a tridiagonal system of equations so you can obtain the values for the time derivatives.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-12-20T17:47:39.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25808" PostTypeId="2" ParentId="25805" CreationDate="2016-12-20T18:23:22.083" Score="3" Body="&lt;p&gt;You can convert your equation into the following system of two PDE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} = v&#xA;$$&#xA;$$&#xA;0= \frac{\partial^2 u}{\partial x^2} + A \frac{\partial^2 v}{\partial x^2} - B&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are now two dependent variables, $u$ and $v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discretization in the spatial dimension (i.e. method of lines (MOL)) is&#xA;straightforward; central difference will work. The resulting set&#xA;of equations is a system of differential algebraic equations (DAE) rather&#xA;than an ODE. There are many available methods for solving DAE but&#xA;explicit methods, like classical Runge-Kutta, are not typically used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A widely used DAE solver is &lt;a href=&quot;http://computation.llnl.gov/projects/sundials/ida&quot; rel=&quot;nofollow noreferrer&quot;&gt;IDA&lt;/a&gt; from the Sundials suite.&#xA;Another well-regarded DAE solver is an implicit Runge-Kutta solver,&#xA;&lt;a href=&quot;https://www.unige.ch/~hairer/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RADAU5&lt;/a&gt;. If you are determined to code your own solution, the most basic&#xA;implicit ODE solver, backward Euler, will also likely solve this DAE system but will, of course, require a relatively small time step for an accurate solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you have access to Matlab or Octave, there are 1D PDE solvers&#xA;that will solve these two equations and let you avoid the difficulties of coding your own MOL solution.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-12-20T18:23:22.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25810" PostTypeId="1" CreationDate="2016-12-21T01:02:32.133" Score="1" ViewCount="41" Body="&lt;p&gt;I've been trying to find a way to correlate data sets from two sensor types. In theory these measure the same signal. One is in a known format expressed in meters while the other is in numeric form read from a 24 bit value. If anyone could help me find a way to find the approximate meter value of these integer values I would be so appreciative for the assistance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These sensors may have different offsets but the difference between each set of two measurements should be close. I am listing the known values first then the integer representation from the unknown sensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.3940 meters = 38884&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.3860 meters = -348332&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.2125 meters = 32230&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.2055 meters = -97977&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.6510 meters = 21108&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0.7495 meters = -363721&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have many more sample sets. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="22706" LastActivityDate="2016-12-21T01:02:32.133" Title="Trying to find a correlation in data from 2 sensors" Tags="|numerical-analysis|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25811" PostTypeId="1" CreationDate="2016-12-21T06:00:22.280" Score="0" ViewCount="291" Body="&lt;p&gt;I am wondering if this is the right place to ask this question. I found a related question on stack overflow &lt;a href=&quot;https://stackoverflow.com/questions/814312/physics-in-computer-science&quot;&gt;https://stackoverflow.com/questions/814312/physics-in-computer-science&lt;/a&gt; but it was closed. So I am asking my question on SE Physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am planning to study graduate level computational physics study. For example, it involves programming to solve problems in linear algebra, PDEs, ODEs (more specific examples are finite volume, finite element, iterative methods to solve nonlinear equations). I am mechanical engineering major student with basic programming knowledge. During my ME coursework, I didn't program heavily. I just took a look at the undergraduate CS degree curriculum, and found that I do not know most of the CS stuff - algorithms, theory of computer science, parallel computing, etc. This knowledge may be vital in programming efficiently - in terms of human time spent on programming. I am feeling that a CS major student can far easily program and debug; something that can take hours for a beginner physics student without CS or computer engineering knowledge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused how a ME major student can undertake computational physics study without CS or computer engineering background. Is knowledge of a programming language enough?  Can anyone comment here? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="22707" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2016-12-23T11:28:28.000" Title="What type of computer science/engineering knowledge someone need to have to study graduate level course in compurational physics?" Tags="|numerical-analysis|computational-physics|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25812" PostTypeId="1" CreationDate="2016-12-21T08:04:49.913" Score="1" ViewCount="286" Body="&lt;p&gt;I need to integrate a function along the edge of a quadrilateral (boundary integral). For example, the function is &lt;span class=&quot;math-container&quot;&gt;$f(x,y)=x^3+y^3$&lt;/span&gt;, the quadrilateral coordinates are &lt;span class=&quot;math-container&quot;&gt;$(0,0),(2,-1),(3,2),(1,3)$&lt;/span&gt; and the concerned edge is the line joining &lt;span class=&quot;math-container&quot;&gt;$(2,-1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(3,2)$&lt;/span&gt;. So, the required integral is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$I=\int_{(2,-1)}^{(3,2)}x^3+y^3\ dl\ \ ;\ \ dl=\sqrt{dx^2+dy^2}$$&lt;/span&gt;&#xA;In another procedure I mapped the quadrilateral into a standard square in &lt;span class=&quot;math-container&quot;&gt;$s,t $&lt;/span&gt; axes.&lt;br&gt;&#xA; By using Lagrange interpolation, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$X(s,t)=\ \frac{3(s + 1)(t + 1)}{4} - \frac{(s + 1)(t - 1)}{2} - \frac{(s - 1)(t + 1)}{4}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Y(s,t)=\ \frac{(s + 1)(t - 1)}{4} - \frac{3(s - 1)(t + 1)}{4}+ \frac{(s + 1)(t + 1)}{2}$$&lt;/span&gt;&#xA;Since integration is from &lt;span class=&quot;math-container&quot;&gt;$t=-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dl=J dt$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$J=\sqrt{\left(\frac{\partial X}{\partial t}\right)^2+\left(\frac{\partial Y}{\partial t}\right)^2}$&lt;/span&gt;. So the required integral is,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$I=\int_{-1}^1X^3+Y^3\ Jdt$$&lt;/span&gt;&#xA;Unfortunately, the second method gives the wrong result. I got &lt;code&gt;66.4078&lt;/code&gt; while the actual result is &lt;code&gt;55.3399&lt;/code&gt;.&lt;br&gt;&#xA;Where did I make mistake here?&lt;/p&gt;&#xA;" OwnerUserId="21659" LastEditorUserId="20688" LastEditDate="2018-11-08T00:27:20.563" LastActivityDate="2019-01-07T01:01:30.313" Title="Line integral along the edge of an isoparametrically mapped quadrilateral" Tags="|finite-element|interpolation|integration|numerics|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="25813" PostTypeId="2" ParentId="25806" CreationDate="2016-12-21T13:23:57.840" Score="1" Body="&lt;p&gt;Following the comments from origimbo, the solution is actually very simple.&#xA;You just need to generate a sufficient script using the trace option of paraview, if this script is functional (load your file, generate your image, save it to the format you wish), then you can save it as an independent script. &#xA;Then, if you run this script using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pvpython SCRIPT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the script will run completly as if it were running inside paraview, but without launching the GUI.&#xA;Therefore it is pretty simple...&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2016-12-21T13:23:57.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25814" PostTypeId="2" ParentId="25811" CreationDate="2016-12-21T16:02:18.343" Score="2" Body="&lt;p&gt;You are correct that with respect to the software side of things, it's possible a typical CS student who has programmed a lot in their undergrad could perform much quicker than someone with basic programming experience. They may not have the solid physics background you could, though, so obviously both sides have their weaknesses with respect to computational physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing about getting involved in computational science is not only do you need to have a strong footing in the actual science, but you ideally would have sufficiently strong software and algorithmic abilities. As Wolfgang mentions in the comments, a big part of this is just practice. Things like data structures is a useful topic to study as well provided you have the time, but you could get away with a basic understanding of what each data structure is best used for and just use the data structure implementations that come along with language libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of the day, the most beneficial thing you could probably start doing is trying to get a lot of practice programming with whatever language(s) you expect you might use. I would probably recommend looking into Fortran or C, maybe C++ after you understand the basics of C. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also recommend getting a book or two to not just help with learning the languages, but learning to build good software. Some books that come to mind are &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0123850037&quot; rel=&quot;nofollow noreferrer&quot;&gt;API Design in C++&lt;/a&gt; and &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0132350882&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clean Code&lt;/a&gt;. Note that API Design in C++ actually teaches a lot of language independent keys to building good software, so this is useful whether you use C++ or not. I stress developing this background because without it, you may end up building codes that are hard to maintain, that are buggy, and in turn result in lots of lost time in the long run. Taking the effort to build something clean and modular can really save time in the long run even if you put up a little more effort up front.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you have these fundamentals down, then you should investigate more advanced topics like parallel computing.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2016-12-21T16:02:18.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25815" PostTypeId="2" ParentId="25804" CreationDate="2016-12-21T18:27:50.353" Score="3" Body="&lt;p&gt;MATLAB's &lt;code&gt;eigs&lt;/code&gt; function simply calls ARPACK.  So if you are using that, there is no difference and they should be equally fast.  If they are not, the difference is in your own code.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2016-12-21T18:27:50.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25816" PostTypeId="2" ParentId="25804" CreationDate="2016-12-22T02:19:14.220" Score="0" Body="&lt;p&gt;I'm not an expert on this, but from what I can tell all of your matrix operations essentially reduce to polynomial time algorithms at best. MatLab, being purpose-specific software might be using some tricks to get results faster (heuristics, caching or look-up tables). People might look at you funny if you tell them math is hard, but most of it is NP-hard or NPC. ColPack is another related software library that does a great job with graph coloring.&lt;/p&gt;&#xA;" OwnerUserId="7593" LastEditorUserId="7593" LastEditDate="2016-12-22T02:28:33.237" LastActivityDate="2016-12-22T02:28:33.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25817" PostTypeId="2" ParentId="25811" CreationDate="2016-12-22T12:30:02.287" Score="0" Body="&lt;p&gt;A typical computational physics class will have a modest expectation on the students' abilities to program. The physics curriculum often covers only basic programming. From the &quot;coding&quot; point of view, you should not worry that much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are worried anyway, consider following a tutorial online where the language is the one used in your planned class.&lt;/p&gt;&#xA;" OwnerUserId="16340" LastActivityDate="2016-12-22T12:30:02.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25823" PostTypeId="2" ParentId="25811" CreationDate="2016-12-23T11:28:28.000" Score="3" Body="&lt;p&gt;I had basically no programming experience (except for a one semester class I took in first year - Octave) when I started solving problems in computational physics (it was basic quantum mechanics), as part of my Honours year coursework.  In the first week of learning how to code in FORTRAN, we did trivial things, like formatting text for printing, and compiling source code contained in one file.  In the second week, we solved the 1D Schrodinger equation using a shooting method.  By the fourth week, we were coding the Numerov-Cooley method, and it was a few weeks later (this was a half-semester course), after a few more assignments, that we were propagating a Gaussian wave packet.  The math involved was a little tricky at first, but it was the programming which made life very difficult for the uninitiated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was about the fifth week when I learnt about options to see helpful error messages when compiling my programs!  You mean you know which array reference is out of bounds, and you haven't been telling me?  I was livid that this had not been considered important enough for the lecture slides. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyhow, enough about me, and my hopelessly inefficient 80 hour coding weeks which were required to ace that course... the point is that you don't want to go through what I went through, and you don't have to.  Here are the lessons, in order of importance:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)  Don't bother with FORTRAN or C as your first language, if you're trying to solve mathematical problems.  These languages (especially C) will only slow you down in reaching your short term goal of completing a scientific computing course.  Instead, learn a language which is designed to help you write useful code faster (Python is ideal, but Octave and Matlab have their advantages too).  With Python and the appropriate modules installed (mainly just numpy), you'll have no problems performing &quot;simple&quot; tasks like reading data files into arrays, sorting the data, and subjecting it to standard linear algebra operations.  In FORTRAN, on the contrary, you're likely to give up trying to write your own parser for input files, or smash the keyboard after getting every one of the ten thousand arguments to a LAPACK routine incorrect.  C is better to learn than FORTRAN, but in your case, it will only make writing numerical routines more frustrating, as well as looking much uglier in your text editor (C is a horrendously ugly language - at least FORTRAN code looks a bit like math on the page).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2)  Ask for help early, regularly, and online.  Don't allow simple, idiosyncratic tricks of the trade to waste all your free time.  It's OK that you don't know that stuff; it's unintuitive.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3)  If you can't debug the code in time, don't destroy it with hacks and arbitrary &quot;bug fixes&quot;.  Submit your concise pseudo code, along with the source code, and provide a clear writeup of the problems.  Sometimes, only time and a clear head will help, while at other times (equally often when you're first learning) you just need the help of someone more experienced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4)  Get your math right before you start planning your code.  Algorithms are understood on paper by humans, and then taught to computers via programming languages.  If you get the algorithm wrong, you'll never be able to find the bug, because it doesn't actually exist (you just coded the wrong algorithm, correctly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) Forget about parallel computing, until you've &quot;mastered&quot; writing programs for one processor.  Most computational scientists just lean on the expertise of colleagues here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary: learn Python as quickly as possible, and try using numpy to solve linear algebra problems.  Use of a language like Python will make your life a million times easier.  &lt;/p&gt;&#xA;" OwnerUserId="22728" LastActivityDate="2016-12-23T11:28:28.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25824" PostTypeId="1" AcceptedAnswerId="25825" CreationDate="2016-12-23T14:27:01.003" Score="1" ViewCount="483" Body="&lt;p&gt;Let me first be clear that I'm still new to Julia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to plot time series data contained in an Excel file using Julia. So far I couldn't find any packages for that. The only workaround I found was to save the Excel as a CSV file and then plot the data using DataFrames and Gadfly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that this can be done easily with Pyplot in Python and that a version is also available for Julia, but still I couldn't figure how to do it in Julia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would really be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="22729" LastActivityDate="2016-12-23T14:33:45.973" Title="Plotting Excel data with Julia" Tags="|plotting|julia|" AnswerCount="1" CommentCount="0" ClosedDate="2016-12-27T20:51:42.107" ContentLicense="CC BY-SA 3.0" />
  <row Id="25825" PostTypeId="2" ParentId="25824" CreationDate="2016-12-23T14:33:45.973" Score="1" Body="&lt;p&gt;I have to options to extract data from Excel documents:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/davidanthoff/ExcelReaders.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;ExcelReaders&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/aviks/Taro.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Taro&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Once you have the data you can use Gadfly.&lt;/p&gt;&#xA;" OwnerUserId="22185" LastActivityDate="2016-12-23T14:33:45.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25829" PostTypeId="1" AcceptedAnswerId="25839" CreationDate="2016-12-23T20:18:21.977" Score="0" ViewCount="292" Body="&lt;p&gt;I've written a simple legendre quadrature in MAPLE to compute the integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{-1}^1 r^2 dr = \frac{r^3}{3} |_{-1}^1=\frac{2}{3}$ &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;restart; with(orthopoly): with(LinearAlgebra):&#xA;legendre_quad := proc (n, digits) &#xA;    local i, s, w, location; &#xA;    w := Vector(n, fill = 0); &#xA;    location := Vector(sort([evalf[digits](solve(P(n, x) = 0, x))], `&amp;lt;`));&#xA;    for i to n do &#xA;        w[i] := eval[digits](2/((1-location[i]^2)*(diff(P(n, x), x))^2), [x = location[i]]) &#xA;    end do; &#xA;    s := (w, location) &#xA;end proc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;,such that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(weights, location) := legendre_quad(10, 50);&#xA;int := DotProduct(location^~2, weights)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;,which results in &lt;code&gt;0.666666635631640303&lt;/code&gt;. But this is only accurate up to 7 digits, even doubling the number of &lt;code&gt;digits&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I increase the accuracy of this quadrature?&lt;/p&gt;&#xA;" OwnerUserId="11656" LastActivityDate="2016-12-24T22:09:12.177" Title="How to increase precision of Gauss-Legendre Quadrature in MAPLE?" Tags="|quadrature|maple|" AnswerCount="1" CommentCount="1" ClosedDate="2016-12-29T19:36:22.477" ContentLicense="CC BY-SA 3.0" />
  <row Id="25830" PostTypeId="2" ParentId="83" CreationDate="2016-12-24T01:49:33.550" Score="2" Body="&lt;p&gt;KNITRO has Python and MATLAB interfaces, among others. Think of it as an FMINCON replacement, but much better performing, and more expensive.  &lt;a href=&quot;https://www.artelys.com/en/optimization-tools/knitro#doc-tab&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.artelys.com/en/optimization-tools/knitro#doc-tab&lt;/a&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a user of KNITRO, but not otherwise affiliated with the product.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2017-11-29T02:32:55.320" LastActivityDate="2017-11-29T02:32:55.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25831" PostTypeId="1" AcceptedAnswerId="25832" CreationDate="2016-12-24T11:43:34.790" Score="0" ViewCount="664" Body="&lt;p&gt;I have the &lt;code&gt;x&lt;/code&gt; and the &lt;code&gt;f(x)&lt;/code&gt; for a set of &lt;code&gt;x&lt;/code&gt;. I don't know the function, actually. This is sufficient to integrate it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;x&lt;/code&gt; value might span from &lt;code&gt;1&lt;/code&gt; to no more than &lt;code&gt;50&lt;/code&gt;, usually around &lt;code&gt;10&lt;/code&gt;. While &lt;code&gt;y&lt;/code&gt; is always negative and it's peak might pass &lt;code&gt;-100&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the simplest approach?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x 0  1  2  3  4  5  6  7  8 9&#xA;y 0 -3 -5 -7 -5 -3 -2 -1 -1 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;May I ask the Python code com compute that?&lt;/p&gt;&#xA;" OwnerUserId="22682" LastActivityDate="2016-12-24T12:10:56.130" Title="Numerical integration of given points, simple/easy way" Tags="|numerical-analysis|numerics|integration|integral-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25832" PostTypeId="2" ParentId="25831" CreationDate="2016-12-24T12:10:56.130" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;scipy.integrate&lt;/code&gt; with a given method (Simpson's rule or composite trapezoidal rule for instance):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy import integrate&#xA;x = array1&#xA;y = array2&#xA;int = integrate.cumtrapz(y, x, initial = 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.cumtrapz.html#scipy.integrate.cumtrapz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for more.&lt;/p&gt;&#xA;" OwnerUserId="22185" LastActivityDate="2016-12-24T12:10:56.130" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25833" PostTypeId="1" CreationDate="2016-12-24T14:44:14.903" Score="2" ViewCount="601" Body="&lt;p&gt;I have a quadratic constraint of the form&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ x_1 x_2 \leq x_3 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x_1, x_2, x_3 \geq 0$&lt;/span&gt;. The objective is linear, and all other constraints are linear, too. I know that I can transform the product as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ y_1=\frac{1}{2}(x_1+x_2) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ y_2=\frac{1}{2}(x_1−x_2) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ y_1^2−y_2^2\leq x_3 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But when it comes to the last constraint, it is not convex (matrix is not PSD), and thus not suitable for commercial solvers like CPLEX and Gurobi (as far as I know). Moreover, they are not conic quadratic representable. At least I don't know how to reformulate them, or to find a suitable approximation (&amp;quot;good for practical purposes&amp;quot;). Now, my question.&lt;/p&gt;&#xA;&lt;p&gt;Is there an efficient (to some degree) approach to deal with these kind of constraints?&lt;/p&gt;&#xA;&lt;p&gt;I am asking this because they look quite simple and the constraint expressions are the difference of convex functions &lt;span class=&quot;math-container&quot;&gt;$ y_1^2−(y_2^2+x_3) $&lt;/span&gt; (the sum of two convex functions on different domains is convex).&lt;/p&gt;&#xA;&lt;p&gt;Maybe some relaxation technique has been proven to be useful? Convexification? In other words, how to circumvent this?&lt;/p&gt;&#xA;" OwnerUserId="20239" LastEditorUserId="20417" LastEditDate="2023-03-05T23:07:35.080" LastActivityDate="2023-03-05T23:07:35.080" Title="How to handle the quadratic constraint $x y \leq z$?" Tags="|optimization|constraints|" AnswerCount="3" CommentCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="25834" PostTypeId="2" ParentId="21771" CreationDate="2016-12-24T15:17:07.257" Score="0" Body="&lt;p&gt;This can be done fairly easy in maple. For this I have choose the pde from your other question because that pde has an exact solution.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;restart:with(plots):&#xA;pde:= diff(u(x, t), t) = diff(u(x, t), x, x)-sin(x+t)+cos(x+t);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First we will check whether your exact solution satisfy the pde or not. For this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ansol:=u(x, t)=cos(x+t);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The zero output means that the exact solution satisfies the pde. Now moving on to the numerical solution,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ic:={u(x,0)=cos(x)};&#xA;bcs:={ D[1](u)(0,t)=-sin(t),D[1](u)(1,t)=-sin(1+t)};&#xA;&#xA;pds:= pdsolve({pde},ic union bcs,numeric,time=t,range=0..1,errorest=true,timestep=1/16,spacestep=1/16);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;pdsolve&lt;/code&gt; we have utilized the option &lt;code&gt;errorest=true&lt;/code&gt; to compute error estimates.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The error estimators used, in both the visual error estimates and the error control, are simply local truncation error estimates for the PDE or PDE system.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;First, lets find out the effect of the &lt;code&gt;timestep&lt;/code&gt; and &lt;code&gt;spacestep&lt;/code&gt; on the error,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pds:-settings(timestep=1/8,spacestep=1/8);&#xA;pds:-plot([u(x,t),[u(x,t)+err(u(x,t)),color=blue,linestyle=2],[u(x,t)-err(u(x,t)),color=green,linestyle=3]],t=5,axes=boxed);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3B7jW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3B7jW.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we reduce the &lt;code&gt;timestep&lt;/code&gt; and &lt;code&gt;spacestep&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pds:-settings(timestep=1/64,spacestep=1/64);&#xA;pds:-plot([u(x,t),[u(x,t)+err(u(x,t)),color=blue,linestyle=2],[u(x,t)-err(u(x,t)),color=green,linestyle=2]],t=5,axes=boxed);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uj0at.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uj0at.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, we can see from the above plot that the error in the pde solution is acceptable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we will plot the error in the solution along the space&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5Bq7r.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Bq7r.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotting error in the solution with time,&#xA;&lt;a href=&quot;https://i.stack.imgur.com/tGywB.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tGywB.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, visualizing the absolute error,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pds:-plot([[(abs(u(x,t)-(cos(x+t)))^2),color=red]],t=5,axes=boxed);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UmJ5f.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UmJ5f.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pds:-plot([[(abs(u(x,t)-(cos(x+t)))^2),color=red]],t=0..5,x=0.5,axes=boxed);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TmlC2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TmlC2.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20267" LastEditorUserId="20267" LastEditDate="2016-12-24T15:28:40.003" LastActivityDate="2016-12-24T15:28:40.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25836" PostTypeId="2" ParentId="24271" CreationDate="2016-12-24T16:13:43.883" Score="1" Body="&lt;p&gt;You can do it in maple like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First you need to specifiy the lower and upper limits. &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;b&lt;/code&gt;, and &lt;code&gt;c&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For normal/primitive integration, we use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int(1/x, x = a .. c);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now to find the Cauchy Principal Value, there is built-in option in maple &lt;code&gt;int&lt;/code&gt;,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int(1/x, x = a .. c, 'CauchyPrincipalValue');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But if you actually want type in yourself then&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Limit(Int(1/x, x = a .. b-epsilon)+Int(1/x, x = b+epsilon .. c), epsilon = 0, right);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20267" LastActivityDate="2016-12-24T16:13:43.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25837" PostTypeId="1" AcceptedAnswerId="25843" CreationDate="2016-12-24T21:13:00.700" Score="3" ViewCount="370" Body="&lt;p&gt;I am trying out the arpack driver &lt;a href=&quot;https://github.com/pv/arpack-ng/blob/master/EXAMPLES/SYM/dsdrv1.f&quot; rel=&quot;nofollow noreferrer&quot;&gt;dsdrv1&lt;/a&gt;, which is used to iteratively obtain the first m eigenvectors from the eigenvalue problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\hat{A}\mathbf{x} = \lambda\mathbf{x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it is an iterative procedure, a good initial guess will reduce the number of iterations needed for convergence. One of the best initial guesses is presumably the correct eigenvectors (obtained, for example, from a previous run).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With arpack's dsdrv1 driver, You can supply an initial guess through the NxN &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/UG/node29.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;residual array&lt;/a&gt; passed to dsaupd, which is fine if you are only computing one eigenvector. However, I am computing m eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I.e. Say I have m correct eigenvectors obtained from a previous run of dsdrv1. How would I utilise these vectors to minimise the number of iterations in a new call to dsdrv1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried feeding the lowest converged eigenvector back into the procedure by assigning it to the residual array and setting info to 1. Only a few iterations are required if I'm only calculating the first eigenvector. But when I calculate more, the initial guess is worse than a random initial vector. I've also tried feeding the average of all eigenvectors as an initial guess, but no luck either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, if I have all the data from a successful dsdrv1 calculation, can I use it to minimise the number of iterations in a 2nd dsdrv1 run applied to the same eigenvalue problem?&lt;/p&gt;&#xA;" OwnerUserId="7808" LastEditorUserId="7808" LastEditDate="2016-12-24T21:52:13.783" LastActivityDate="2016-12-26T13:08:40.377" Title="Iteratively obtaining m eigenvectors using arpack: If I have a good initial guess, how do I use it?" Tags="|eigenvalues|arpack|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25839" PostTypeId="2" ParentId="25829" CreationDate="2016-12-24T22:09:12.177" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.maplesoft.com/support/help/Maple/view.aspx?path=evalf&quot; rel=&quot;nofollow noreferrer&quot;&gt;evalf&lt;/a&gt; evaluates using floating-point arithmetic. Increase the floating-point precision by modifying the built-in parameter &lt;a href=&quot;http://www.maplesoft.com/support/help/Maple/view.aspx?path=Digits&quot; rel=&quot;nofollow noreferrer&quot;&gt;Digits&lt;/a&gt; using the command &lt;code&gt;Digits:=n&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22734" LastActivityDate="2016-12-24T22:09:12.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25842" PostTypeId="1" CreationDate="2016-12-25T20:14:27.597" Score="2" ViewCount="89" Body="&lt;p&gt;I'm solving simple single-objective multidimensional global optimization problem using various stochastic algorithms like Monte-Carlo, GA and other evolutionary approaches. The task is formulated as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;search space $X \subseteq R^n$&lt;/li&gt;&#xA;&lt;li&gt;objective function $f: R^n \rightarrow R$&lt;/li&gt;&#xA;&lt;li&gt;must find $\textbf{x}^*=\{x_1^*, ..., x_n^* \} \in X: f(\textbf{x}^*)=\min_{\textbf{x} \in X} f(\textbf{x})$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, I also have some preferences of $x_i \in [a;b]$ which are presented in set of normalized values (from $0$ to $1$). This set can be interpolated (here no matter how) to probability distributions. So, for given $x_i$ exists function $g_i(x)$. Figure below shows typical distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XIxR0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XIxR0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't mean that $x_i$ always possess preferable values and in peaks exists smaller value of objective. However, better values of $f$ tends to be near peaks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is how to use this information about $x_i$ preferable values during optimization process and acheive better results (e.g. rate of convergence)? Any recommendations/books/articles with similar situations will be appreciated. I even don't know in what math area I should look for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to use simulated annealing — just added $g_i$ (multiplied by $-1$) to objective function with a high weight at the beginning and then gradually reduce it to zero. With this simple scheme I didn't acheive much better results. &lt;/p&gt;&#xA;" OwnerUserId="22741" LastActivityDate="2016-12-25T20:14:27.597" Title="Global optimization with known distributions of some variables" Tags="|optimization|nonlinear-programming|stochastic|nonconvex|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25843" PostTypeId="2" ParentId="25837" CreationDate="2016-12-26T01:17:30.373" Score="3" Body="&lt;p&gt;Since no one else has responded to this, I'll take a shot at it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doubtful you will find an option like you are looking for in Arpack.&#xA;Here is why I think that. The algorithms in Arpack are based on constructing&#xA;a vector subspace called a Krylov sequence as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;K = \{x, Ax, A^2x, A^3x... A^nx\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x$ is the single vector you are allowed to specify if you wish.&#xA;If you could set more than a single vector, the subspace would not&#xA;necessarily be a Krylov sequence that the algorithm requires.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say that you have had poor experience choosing the starting $x$ as&#xA;the first eigenvector from a previous analysis.&#xA;There is a very good reason why this is&#xA;particularly bad choice when you want to calculate several eigenvalues.&#xA;If you look closely at the Krylov sequence you will see that it simply&#xA;implements the power method of calculating the eigenvector corresponding&#xA;to the largest eigenvalue of A. Since $x$ is a relatively good approximation to the eigenvector with largest eigenvalue, all of the remaining vectors also are increasingly good approximations to this first eigenvector.  The complete subspace has almost no components&#xA;of the other eigenvectors you want to calculate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are willing to consider alternative eigenvalue solvers and algorithms,&#xA;the subspace iteration algorithm might be more efficient in your particular&#xA;case. It has some similarities to the Krylov-based solvers like Arpack but&#xA;it is generally less efficient.&#xA;However it allows you to specify an arbitrary vector subspace like your&#xA;previous set of eigenvectors and that can significantly improve its efficiency.&#xA;The software package &lt;a href=&quot;http://slepc.upv.es/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SLEPc&lt;/a&gt; includes a&#xA;subspace iteration algorithm among its many options.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2016-12-26T13:08:40.377" LastActivityDate="2016-12-26T13:08:40.377" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25845" PostTypeId="2" ParentId="23271" CreationDate="2016-12-26T15:49:55.933" Score="3" Body="&lt;p&gt;To elaborate on the linearization argument, in uu_x you want to assume u is locally constant, not u_x, for two reasons: a) u varies more slowly than its derivative, and b) in this particular case, if you assume u_x is locally constant, by definition you also assume u is locally linear, which means higher space derivatives are zero, and this not only introduces additional approximation error, but it may imply you may be throwing out the baby with the bathwater, depending on your equation.&lt;/p&gt;&#xA;" OwnerUserId="22753" LastActivityDate="2016-12-26T15:49:55.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25846" PostTypeId="1" CreationDate="2016-12-27T01:07:27.100" Score="1" ViewCount="244" Body="&lt;p&gt;I have a question as I am starting my dive into computational analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a large set of data (~2 months) which includes the room temperature, HVAC status (heat/cool/off) and the location of this room (I can find detailed information on outdoor temperature). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to do some analysis to find the following information....&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Unit Capability (or efficiency).&lt;/li&gt;&#xA;&lt;li&gt;Loss of energy through the room. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Preferably for each heat/cool cycle i would be able to calculate the efficiency and be able to see a distribution of the results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best set of mathematical tools (type of functions/ approaches)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I can attempt regression over each period, however it seems in-efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="22756" LastActivityDate="2018-09-20T03:02:01.987" Title="Data Analysis - Cooling Efficiency" Tags="|statistics|regression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25847" PostTypeId="2" ParentId="25833" CreationDate="2016-12-27T17:06:22.990" Score="0" Body="&lt;p&gt;As I said in the comment, I got a new hint: &lt;a href=&quot;https://www.or-exchange.org/questions/14687/approach-to-resolve-the-issue-with-a-non-linear-constraint-xy-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.or-exchange.org/questions/14687/approach-to-resolve-the-issue-with-a-non-linear-constraint-xy-z&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bi-linear constraints can be attacked with the logarithm function. Therefore, we can introduce new variables $ z_i = \ln(x_i) $ and our constraints will become&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ z_1 + z_2 \leq z_3 $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, we have to include the additional constraints, too. Particularly, those that are representing the connection of $ z_i $ and $ x_i $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \exp(z_i) = x_i $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the exponential function is conic quadratic representable for practical purposes.&lt;/p&gt;&#xA;" OwnerUserId="20239" LastEditorUserId="20239" LastEditDate="2016-12-30T01:54:22.847" LastActivityDate="2016-12-30T01:54:22.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25849" PostTypeId="2" ParentId="25846" CreationDate="2016-12-28T10:19:50.447" Score="1" Body="&lt;h3&gt;An efficiency metric&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You could use,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eta = p_{\text{out}}/p_{\text{in}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the power out ($p_{\text{out}}$) is defined as the heat per second from the room to the environment, and, power in ($p_{\text{in}}$) is the heat per second entering the room from the heat source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it might be more fundamental to compare the thermal conductivity $k$ to characterise heat loss, where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p_{\text{out}} = -\frac{k A}{d} \left(T_2 - T_1\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is the surface area of the wall, $d$ is the thickness, and $\left(T_2 - T_1\right)$ is the temperature gradient between inside and outside.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could compare the heat capacity $C$ of the room,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C = p_{\text{in}} / (T_2 - T_1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How far you can get depends on how much data you have and exactly what it is. It's not so clear from your question.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Energy loss&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I think a useful metric here would be power dissipated per unit area (closely related to the thermal conductivity). This would allow you to compare different sized rooms. So calculate the power out per second and divide by the area of the floor the ceiling and the walls (NB, some walls, ceiling and floors might be externally facing so you will find they loose more heat. You may wish to create subcategories at this point for internal and external surfaces).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some ideas to get you stated!&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2016-12-28T14:07:50.897" LastActivityDate="2016-12-28T14:07:50.897" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25850" PostTypeId="1" AcceptedAnswerId="27852" CreationDate="2016-12-28T15:28:44.643" Score="1" ViewCount="2462" Body="&lt;p&gt;I am using OpenFoam to do an LES simulation. I am using mapped boundary condition on the inlet plane. I know it maps from a source plane to the target plane(here the inlet plane). In &quot;Pitzdaily-mapped&quot; tutorial, we only have &quot;set average&quot; and &quot;average&quot; options and there's no option for selecting the source plane. Can anyone please explain how we can specify the plane from which we want to map?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="19836" LastActivityDate="2017-09-14T15:29:17.107" Title="OpenFoam Mapped Boundary condition" Tags="|boundary-conditions|openfoam|" AnswerCount="1" CommentCount="0" ClosedDate="2017-09-18T15:36:34.140" ContentLicense="CC BY-SA 3.0" />
  <row Id="25852" PostTypeId="1" AcceptedAnswerId="25853" CreationDate="2016-12-29T17:03:53.483" Score="7" ViewCount="478" Body="&lt;p&gt;I was wondering if there is any known way to compute the Charactaristic Polynomial P of a matrix A numerically stable in the sense of realizing the Cayley-Hamilton Theorem, i.e. that P(A)=0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found and implemented a couple of ways, but when the dimension of the matrix is even a few hunderds, P(A) is not even close to the zero matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'&lt;a href=&quot;http://perso.uclouvain.be/paul.vandooren/publications/MisraQV95.pdf&quot; rel=&quot;noreferrer&quot;&gt;Numerically Reliable Computation of Characteristic Polynomials&lt;/a&gt;' by Pradeep Misra, Enrique S. Quintana, Paul M. Van Dooren claims that reduction to Frobenius canonical form, use of Hyman's method for computing the determinant&#xA;of a Hessenberg matrix, Faddeev-LeVerrier recursion, finding the polynomial by first computing the eigenvalues of the matrix are not proved to be numerically stable, and presents an algorithm which is numerically stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented the algorithm on MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function p = characteristic(A)&#xA;&#xA;n = size(A,1);&#xA;[AA,BB,~,~] = hess(A, eye(n,n));&#xA;F = zeros(n,n); G = zeros(n,n);&#xA;&#xA;F(:, 2:n) = AA(:,1:n-1); F(1,1) = -1;&#xA;fn = AA(:,n);&#xA;&#xA;G(:, 2:n) = BB(:,1:n-1);&#xA;gn = BB(:,n);&#xA;&#xA;res = zeros(n, n+1);&#xA;for i=0:n&#xA;    if i == 0&#xA;        res(:,n+1) = F\(-fn);&#xA;    elseif i == 1&#xA;        res(:,n) = F\(G*res(:,n+1) + gn);&#xA;    else&#xA;        res(:,n+1-i) = F\(G*res(:,n+1-i+1));&#xA;    end&#xA;end&#xA;&#xA;p = res(1,:);&#xA;for i=1:n-1&#xA;    p = p*AA(i+1,i);&#xA;end&#xA;p = p*(-1)^(n-1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which does work well for small dimenstions (P(A) is close to the zero matrix), but for larger dimenstions, it's better than MATLAB's 'poly' but still very far from the zero matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;A = rand(10,10); p = characteristic(A); polyvalm(p,A); ans(1,1)&#xA;ans =&#xA;  -2.1472e-10  &#xA;&amp;gt;&amp;gt;A = rand(100,100); p = characteristic(A); polyvalm(p,A); ans(1,1)&#xA;ans = &#xA;  9.8824e+152&#xA;&#xA;&amp;gt;&amp;gt;A = rand(20,20); p = characteristic(A); polyvalm(p,A); ans(1,1)&#xA;ans =&#xA; -581.7226&#xA;&amp;gt;&amp;gt;A = rand(20,20); p = poly(A); polyvalm(p,A); ans(1,1)&#xA;ans =&#xA;   3.9789e+03&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I still didn't find an algorithm to calculate the characteristic polynomial in a way that P(A) will be close to the zeros matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll much appreciate any answers,&#xA;Niv &lt;/p&gt;&#xA;" OwnerUserId="22775" LastActivityDate="2016-12-29T19:31:03.023" Title="Numerically stable computation of the Characteristic Polynomial of a matrix for Cayley-Hamilton Theorem" Tags="|linear-algebra|numerical-analysis|matrix|polynomials|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25853" PostTypeId="2" ParentId="25852" CreationDate="2016-12-29T18:25:36.100" Score="6" Body="&lt;p&gt;Generally speaking, $P(A)$ would &lt;em&gt;not&lt;/em&gt; be close to the zero matrix: if you compute the polynomial $P$ to a relative accuracy of $\epsilon$ (which is $10^{-16}$ in double-precision), then $P(A)$ would be something on the order of $\epsilon \|A^n\|$ ($n$—the size of the matrix), which would be huge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can only ask that $P(A)$ be as small as the typical perturbation due to roundoff error. That would be as good as an exact zero in floating-point arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think it matters much how you compute the characteristic polynomial itself, so the exact algorithm probably doesn't matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not specific to matrices either: even for a real roots of $p(x)=0$, you can't really ask that $p(x)$ be exactly zero in floating-point arithmetic. So to determine if $x$ is a root of $p$, you compare $p(x)$ not to zero, but to some error bound derived from the coefficients of $p$ and machine epsilon. There is a clear, advanced-level discussion of these kinds of issues in Higham's &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; (see Section 5.1 about Horner's method and its error bounds).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, the roots of polynomials are notoriously sensitive to the coefficients of those polynomials expressed in the monomial basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $P(A)=0$ is an unreachable goal, you have to consider what you're trying to achieve, because whatever it is, it can probably be done without evaluating the coefficients of the characteristic polynomial explicitly or requiring that $P(A)=0$. If you absolutely &lt;em&gt;must&lt;/em&gt; have $P(A)=0$, I'd recommend simply using arbitrary-precision floating-point arithmetic with number of digits proportional to the size of the matrix.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="127" LastEditDate="2016-12-29T19:31:03.023" LastActivityDate="2016-12-29T19:31:03.023" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25854" PostTypeId="1" CreationDate="2016-12-29T21:16:45.767" Score="4" ViewCount="157" Body="&lt;p&gt;I'm trying to solve a nonlinear elliptic equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$(n(u)u')' = f(u)$$&lt;/span&gt;&#xA;and have a crucial misunderstanding.&lt;/p&gt;&#xA;&lt;p&gt;I suppose the procedure of solving some nonlinear equation consists of:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choosing a proper approximation method (FEM, FVM, FDM etc.).&lt;/li&gt;&#xA;&lt;li&gt;Linearization of the resulting system of nonlinear algebraic equations.&lt;/li&gt;&#xA;&lt;li&gt;Choosing a proper method for solving the algebraic system.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is it possible to invert steps 1 and 2, i.e. can I linearize the PDE first, and only after that perform some discretization? Are all of these steps strictly necessary and do they need to be in this order?&lt;/p&gt;&#xA;" OwnerUserId="22777" LastEditorUserId="23791" LastEditDate="2021-12-20T21:20:18.920" LastActivityDate="2021-12-21T22:30:53.860" Title="Three steps of pde numerical solution and nonlinear equation" Tags="|pde|nonlinear-equations|discretization|elliptic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25855" PostTypeId="2" ParentId="25833" CreationDate="2016-12-30T00:36:40.353" Score="1" Body="&lt;p&gt;If this is the only relevant part of your problem, you can write this as a semidefinite program. Since $x_1, x_2$ do not appear individually you can treat it as a square of a positive number then your problem is the feasibility of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{pmatrix} &#xA;x_3 &amp;amp; y  &amp;amp;  \\ &#xA;y &amp;amp; 1 &amp;amp;   \\&#xA;&amp;amp;&amp;amp; y \\&#xA;\end{pmatrix} \succeq 0&#xA;$$&#xA;where $y = \sqrt{x_1x_2}$.&lt;/p&gt;&#xA;" OwnerUserId="21617" LastActivityDate="2016-12-30T00:36:40.353" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25859" PostTypeId="1" AcceptedAnswerId="25866" CreationDate="2016-12-30T23:35:40.043" Score="0" ViewCount="223" Body="&lt;p&gt;I am implementing an algorithm which produces a 4d mesh for a cylinder with a given 3d base. This means, I have a 3d mesh and I want to generate a 4d mesh for the corresponding space-time cylinder.&#xA;Currently I am using netgen for 3d mesh.&#xA;I am able to generate the 4d vertices and the new 4d elements (pentatopes).&#xA;This is done by applying a sort of the element-wise procedure for each 3d element.&#xA;However, I also need boundary elements (which are tetrahedrons in 4d case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the boundary elements of the 3d mesh as produced by netgen are not connected to the 3d elements. I just have a set of triangle indices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what I need is to &lt;strong&gt;know whether a given 3d element has some of its faces at the boundary&lt;/strong&gt; (and which one). If I know this, I could use it in the element-wise procedure and create a list of 4d boundary elements as well.&#xA;But by know I only have a list of 3d boundary elements which contains vertex indices for each boundary triangle of the 3d mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give a hint of how one can do that, probably with another meshing software, or somehow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or probably there is a software which can generate 4d meshes for a space-time cylinders with, e.g., 3d sphere as a base?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any comments.&lt;/p&gt;&#xA;" OwnerUserId="21958" LastActivityDate="2017-01-01T08:41:36.743" Title="Meshing software: connectivity between elements and boundary" Tags="|mesh-generation|high-dimensional|unstructured-mesh|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25860" PostTypeId="1" AcceptedAnswerId="25861" CreationDate="2016-12-31T02:18:54.353" Score="3" ViewCount="337" Body="&lt;p&gt;I'm using an iterative subspace algorithm (&lt;a href=&quot;http://drum.lib.umd.edu/bitstream/handle/1903/572/CS-TR-2908.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;dsrrit&lt;/a&gt;) to obtain the eigenvalues of an eigenvector equation&#xA;$$&#xA;-\nabla^2 \mathbf{x} = \lambda\mathbf{x}&#xA;$$&#xA;where $\nabla^2$ is the usual Laplacian operator. The algorithm returns a set of eigenvalues $\lambda$ and a subspace $Q$ that spans the eigenvectors of $\lambda$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The literature states&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;the programs do not produce a set of eigenvectors corresponding to the eigenvalues computed&quot; ... &quot;If explicit eigenvectors are desired, they may be obtained by evaluating the eigenvectors of $T$ and applying (2).&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I have plotted the vectors in the returned subspace $Q$ and they do indeed match the expected eigenvectors of $\lambda$. Are there particular conditions that guarantee this? (My eigenvectors are real, and my operator is represented by a symmetric matrix)  &lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2016-12-31T22:53:41.643" Title="Using a subspace iteration method to obtain eigenvalues. Getting eigenvectors too but I don't understand why" Tags="|eigenvalues|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25861" PostTypeId="2" ParentId="25860" CreationDate="2016-12-31T18:15:24.757" Score="3" Body="&lt;p&gt;I haven't read it in detail, but from your linked paper (p. 2):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;SRRIT attempts&#xA;  to compute a nested sequence of orthonormal bases of $Q_1, Q_2, \ldots, Q_m$. Specifically, if all goes well, the subroutine produces a matrix Q with orthonormal columns having the property that if $|\lambda_i| &amp;gt; |\lambda_{i+1}|$ then $q_1,\ldots,q_i$ span $Q_i$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What this means is that if your matrix has only real eigenvalues, with $|\lambda_1|\geq|\lambda_2|\geq\cdots\geq|\lambda_n|$, then $q_i$ will be an eigenvector for $\lambda_i$, using the fact that the matrix is symmetric and the eigenvectors will be orthogonal to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\lambda_i, \lambda_{i+1}$ are two complex conjugate eigenvalues, then the corresponding columns of $Q$ wouldn't be the eigenvectors because they would be real.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2016-12-31T22:53:41.643" LastActivityDate="2016-12-31T22:53:41.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25863" PostTypeId="1" CreationDate="2016-12-31T20:29:35.610" Score="1" ViewCount="62" Body="&lt;p&gt;I'm trying to solve the following equation&#xA;$$\dfrac{\partial}{\partial x}\left(e^{au}\dfrac{\partial u}{\partial x}\right) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this equation can be solved analytically. I am trying to understand the way of working with exponential coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the approximation, I use the &lt;strong&gt;finite volume method&lt;/strong&gt; which leads  to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F_{i+1/2} - F_{i-1/2} = 0,\qquad F_{i-1/2} = \left.e^{au}\dfrac{\partial u}{\partial x}\right|_{i+1/2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the further linearization, I'm trying the following two methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;simple iteration, where $e^u$ is calculated based on the previous iteration&lt;/li&gt;&#xA;&lt;li&gt;Newton's method (with the series expansion, dropping terms that are proportional to $\delta u^2$ and smaller)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$u^{k+1} = u^k + \delta u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one worked well until some value of $a$, and the second one didn't work at all (I have not found the mistake yet)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any way to deal with exponential coefficient?&lt;/strong&gt;  Will be glad for any advice. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="22777" LastEditorUserId="20688" LastEditDate="2017-04-02T17:47:54.903" LastActivityDate="2017-04-02T17:47:54.903" Title="elliptic equation with exponential coefficient" Tags="|nonlinear-equations|numerical-modelling|diffusion|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25864" PostTypeId="2" ParentId="25860" CreationDate="2016-12-31T22:50:05.830" Score="2" Body="&lt;p&gt;The reason that the $Q_i$ are eigenvectors of your original matrix is actually due to the symmetry of that matrix. Yes, it is true that because of the symmetry the eigenvalues are real. But also due to the symmetry, the $T$ matrix returned from &lt;code&gt;dsrrit&lt;/code&gt; is not only upper triangular but also symmetric, itself. Thus it must be a diagonal matrix. The $T$ matrix can be thought of as representing the eigenvectors of the reduced eigen-system. In general, the eigenvectors of the original matrix are computed as $QT$. In this case, because $T$ is diagonal, it is simply selecting columns from the $Q$ matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That the eigenvalues are all real is not enough for the eigenvectors to be columns of $Q$. It is easy to construct an unsymmetric matrix that has real eigenvalues. In this case, $T$ will not be diagonal so the eigenvectors must be computed as $QT$.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2016-12-31T22:50:05.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25865" PostTypeId="1" AcceptedAnswerId="25868" CreationDate="2016-12-31T23:28:42.403" Score="2" ViewCount="109" Body="&lt;p&gt;I don't think this is a pure math problem, so I post it here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we have two regions in $\mathbb R^n$: &#xA;$$&#xA;\lbrace x : a \leq x \leq b \rbrace\\&#xA;$$&#xA;and&#xA;$$&#xA;\left\lbrace x : \sum_{i=1}^n |x_i | \leq c \right\rbrace&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is to find the smallest possible cube that contains the intersection between the two regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $x \in \mathbb R^2$, then the first region is a square and the second a diamond-shape, and the problem looks something like this (the red square is the solution to the problem):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QD9xD.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QD9xD.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is reasonably easy if one takes a particular case like the one on the picture. We can find the equation for the line that crosses the square and find the $x$ and $y$ values where the line is equal to the constants signifying the sides of the square. However, which sides of the square we use depends on the position of the square (it can even be crossed by several lines). In general, it seems to be a lot of special cases to deal with by going that route.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there are general procedures that can be applied to this problem, or if anyone knows an elegant solution to this particular one. &lt;/p&gt;&#xA;" OwnerUserId="17170" LastEditorUserId="17170" LastEditDate="2016-12-31T23:36:29.697" LastActivityDate="2017-01-01T14:27:52.557" Title="Finding smallest cube in $\mathbb R^n$ that contains intersection between two regions" Tags="|geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25866" PostTypeId="2" ParentId="25859" CreationDate="2017-01-01T08:41:36.743" Score="1" Body="&lt;p&gt;Given for each element the indices of its vertices, you can efficiently retrieve face adjacencies by sorting the faces lexicographically, then the pairs of elements that share a face are contiguous in the sorted facet list. The boundary elements are those that have a facet that appears only once in the list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more detailed procedure is given in my answer to this question [1] (for surfacic meshes, can be easily adapted to volumetric meshes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21163/meshing-options-to-generate-number-of-the-sides-of-and-element-tetgen-triangle/21197#21197&quot;&gt;Meshing options to generate number of the sides of and element (tetgen-triangle)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-01-01T08:41:36.743" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25867" PostTypeId="1" CreationDate="2017-01-01T12:34:36.023" Score="1" ViewCount="44" Body="&lt;p&gt;I'd like to simulate the following problem: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Define some square, horizontal and hard area onto which cylinders&#xA;(aspect ratio R) sediment.&lt;/li&gt;&#xA;&lt;li&gt;Preferably, the source of cylinders&#xA;sequentially generates N of these cylinders from a large distance&#xA;above the plane. There is gravity.&lt;/li&gt;&#xA;&lt;li&gt;The cylinders are hard, if possible deformable (bending), non-interpenetrable, and friction / damping forces are large to stabilize the system (avoid infinite bouncing).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Essentially I need a physics simulation software which handles collisions of many-body systems. The number of cylinders would be around 100'000. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am in fine interested in the stacking dynamics of the cylinders. A measure of this would be the z-position of the center of mass of the sedimented mesh of cylinders in function of N/A (aerial density) and the cylinder aspect ratio. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have knowledge of Python, and can work with scripting Blender, and should be able to find my around making scripts (C,Python,...) work on GPUs.&#xA;Blender apparently incorporates Bulletphysics. The Molecular Add-on for Blender that I tried however doesn't have S.I values of physical units (Gravity is mapped between 0 and 1, I don't know what this is), and I couldn't get the simulation to converge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion would be appreciated !&lt;/p&gt;&#xA;" OwnerUserId="22793" LastActivityDate="2017-01-01T12:34:36.023" Title="Software suggestion for simulating the stacking of sedimenting rods" Tags="|computational-physics|software|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25868" PostTypeId="2" ParentId="25865" CreationDate="2017-01-01T13:40:06.767" Score="3" Body="&lt;p&gt;Here is a suggestion.&#xA;Partition your problem into two parts: (1) Construct the intersection, (2) Find the smallest cube.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) The intersection is a polytope defined by the union of your inequalities for both shapes. This is known as an &lt;a href=&quot;https://en.wikipedia.org/wiki/Convex_polytope#Construction_of_representations&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;H&lt;/em&gt;-representation&lt;/a&gt; of the polytope. There is software for constructing a polytope from its &lt;em&gt;H&lt;/em&gt;-representation. &lt;a href=&quot;https://polymake.org/doku.php/tutorial/apps_polytope&quot; rel=&quot;nofollow noreferrer&quot;&gt;polymake&lt;/a&gt; is one option. Essentially you want to convert your &lt;em&gt;H&lt;/em&gt;-representation to a&#xA;&lt;a href=&quot;https://mathoverflow.net/a/178894/6094&quot;&gt;&lt;em&gt;V&lt;/em&gt;-representation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) This is the easier half. Find the smallest bounding box from the extreme vertex coordinates. Then fatten the box to a cube.&lt;/p&gt;&#xA;" OwnerUserId="673" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2017-01-01T14:27:52.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25870" PostTypeId="1" CreationDate="2017-01-02T08:47:15.273" Score="10" ViewCount="664" Body="&lt;p&gt;I have a system of $n$ non-linear equations that I want to solve numerically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{f}(\mathbf{x})=\mathbf{a}$$&#xA;$$\mathbf{f}=(f_1,\dots,f_n)\quad\mathbf{x}=(x_1,\dots,x_n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This system has a number of characteristics that makes it particularly difficult to handle. I am looking for ideas on how to deal with the system more effectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is the system difficult?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The functions are similar to this one (but of course in multiple dimensions):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tF2g4.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have flat plateaus separated by a region of smooth change.   In 2D, you can imagine something like this for one $f_i$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/QSF2F.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, each $f_i$ has two plateaus separated by smooth change around an $n-1$ dimensional hyperplane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Functions like this are difficult to handle with Newton-like methods because the derivative is effectively zero on the plateaus.  In multiple dimensions I cannot easily find a region where &lt;em&gt;none of the $f_i$&lt;/em&gt; have a plateau—if I could that would solve the problem.  The bisection method works well for $n=1$, but it does not generalize well to multiple dimensions. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The functions are very slow to compute. I am looking for a method that can get a reasonable approximation of the root in as few iterations as possible.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The functions are calculated with a Monte Carlo method. This means that each time they are calculated, I get a slightly different random value. Derivatives are difficult to estimate.  Once we are close enough to the root, the noise will start to dominate, and it is necessary to use averaging to increase precision.  Ideally it should be possible to generalize the method to an equivalent &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic approximation&lt;/a&gt; version (e.g., Newton → Robbins-Monro).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The system is high-dimensional. $n$ can be as large as 10-20. When $n=2$, an effective method would probably be the following: try to follow the contours defined by $f_1(x_1,x_2) = 0$ and $f_2(x_1,x_2)=0$ and see where they intersect. It is not clear how this would generalize to high dimensions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What else do I know about the system?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is precisely one root (from theoretical results).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I know the value of $f_i$ on the plateaus (let's say it's 0 and 1 for any $i$).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$f_i$ has a special relationship to $x_i$:&amp;nbsp; $f_i(\dots, x_i, \dots)$ changes monotonically from 1 to 0 as $x_i$ goes from $-\infty$ to $\infty$. This is true for any fixed value of the other $x_{j\ne i}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="605" LastEditorUserId="605" LastEditDate="2017-01-02T10:44:18.263" LastActivityDate="2017-01-03T03:09:53.627" Title="Solving a difficult system of equations numerically" Tags="|numerical-analysis|nonlinear-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25871" PostTypeId="2" ParentId="20834" CreationDate="2017-01-02T15:21:38.123" Score="5" Body="&lt;p&gt;The National Institute of Standards and Technology (NIST) proposes different inputs and outputs to test your implementation. Have a look there:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nist.gov/mml/csd/chemical-informatics-research-group/spce-water-reference-calculations-10%C3%A5-cutoff&quot; rel=&quot;noreferrer&quot;&gt;https://www.nist.gov/mml/csd/chemical-informatics-research-group/spce-water-reference-calculations-10%C3%A5-cutoff&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See paragraph &lt;em&gt;6. Reference Calculations of Intermolecular Energy for SPC/E&lt;/em&gt;&#xA;that includes the usual contributions $E_{real}$, $E_{fourier}$ etc. with everything detailed (cutoffs, numerical precision, etc).&lt;/p&gt;&#xA;" OwnerUserId="9846" LastActivityDate="2017-01-02T15:21:38.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25874" PostTypeId="2" ParentId="25870" CreationDate="2017-01-03T03:09:53.627" Score="1" Body="&lt;p&gt;Since there is a single root and there are no constraints, you might have luck posing it as an optimization problem: minimize the sum (along each dimension) of the squares of your original function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Classical&#xA;Optimization methods will likely fail, but heuristic methods such as genetic algorithms or CME-ES (covariant etc matrix adaptation - evolutionary strategy) might work.&lt;/p&gt;&#xA;" OwnerUserId="22344" LastActivityDate="2017-01-03T03:09:53.627" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25875" PostTypeId="1" AcceptedAnswerId="25886" CreationDate="2017-01-03T10:08:02.327" Score="6" ViewCount="710" Body="&lt;p&gt;Is the &lt;a href=&quot;http://fab.cba.mit.edu/classes/S62.12/docs/Hilton_marching_triangles.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;marching triangles&lt;/a&gt; algorithm guaranteed to terminate (sucessfully)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm does roughly speaking:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;iterate over all boundary edges&lt;/li&gt;&#xA;&lt;li&gt;project a new point, add a new triangle from the edge and the point, if it's admissible (the new point does not intersect the circumsphere of any existing triangle)&lt;/li&gt;&#xA;&lt;li&gt;if it's not admissible, try do connect the edge with the next or previous point (closing non-convex parts of the boundary), if they are admissible&lt;/li&gt;&#xA;&lt;li&gt;if it's still not admissible, try to connect a point from the intersected triangle with the edge&lt;/li&gt;&#xA;&lt;li&gt;else skip the edge in this iteration&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now i get the following case&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3puDe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3puDe.png&quot; alt=&quot;problem case in the mesh generation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm correctly generated a mesh with triangles, by projecting the new points. Now there is a triangular crack, which is not local delaunay.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;projecting a point by a constant distance clearly fails&lt;/li&gt;&#xA;&lt;li&gt;using the next, previous or intersection point doesn't generate a admissible triangle&lt;/li&gt;&#xA;&lt;li&gt;ignoring the edge leads to a non-closed mesh&lt;/li&gt;&#xA;&lt;li&gt;adding the edge for the next iteration leads to a non-terminating algorithm, as it cannot be processed in the next iteration either and no other processed edge will close the hole&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The example is a simple implicit circle function as used in the paper, the seed triangle is an arbitrary edge on the implicit function connected with a point projected by the fixed distance and then projected back on the surface as defined in the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When i give the next / previous points preference (choose the next / previous point if it is nearer than $\epsilon$, even when a new triangle could be created), i can reduce the number of tight cracks, but it doesn't cover all problem cases either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite the obvious problem with my implementation, i do not see if the algorithm as described in the paper is guaranteed to always find a admissible triangle in an iteration until the edge list is empty.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="14700" LastEditDate="2017-01-05T09:05:12.133" LastActivityDate="2017-01-05T10:44:40.443" Title="Is the marching triangles algorithm guaranteed to terminate?" Tags="|delaunay-triangulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25876" PostTypeId="1" CreationDate="2017-01-03T16:04:18.827" Score="3" ViewCount="3438" Body="&lt;p&gt;I'm researching numerical optimization. Recently I've come across a variant of a conjugate gradient method named &lt;code&gt;fmincg&lt;/code&gt;. &lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function is written in MATLAB and is used in the famous Andrew Ng's course on Machine Learning on Coursera. According to the copyright notice, &lt;code&gt;fmincg&lt;/code&gt; was written by &lt;a href=&quot;http://learning.eng.cam.ac.uk/carl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Carl Edward Rasmussen&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find the full text of &lt;code&gt;fmincg&lt;/code&gt; on the MATLAB &lt;a href=&quot;https://it.mathworks.com/matlabcentral/fileexchange/42770-logistic-regression-with-regularization-used-to-classify-hand-written-digits/content/Logistic%20Regression%20with%20regularisation/fmincg.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;file exchange&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to figure out the algorithm. The structure is quite straightforward as it closely follows the reference text I'm currently using (Numerical Optimization, Nocedal &amp;amp; Wright). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I'm having a harder time figuring out the details of the line search procedure.&#xA;Specifically, here's the code for the line search loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while 1&#xA;while ((f2 &amp;gt; f1+z1*RHO*d1) | (d2 &amp;gt; -SIG*d1)) &amp;amp; (M &amp;gt; 0) &#xA;  limit = z1;                                         % tighten the bracket&#xA;  if f2 &amp;gt; f1&#xA;    z2 = z3 - (0.5*d3*z3*z3)/(d3*z3+f2-f3);                 % quadratic fit&#xA;  else&#xA;    A = 6*(f2-f3)/z3+3*(d2+d3);                                 % cubic fit&#xA;    B = 3*(f3-f2)-z3*(d3+2*d2);&#xA;    z2 = (sqrt(B*B-A*d2*z3*z3)-B)/A;       % numerical error possible - ok!&#xA;  end&#xA;  if isnan(z2) | isinf(z2)&#xA;    z2 = z3/2;                  % if we had a numerical problem then bisect&#xA;  end&#xA;  z2 = max(min(z2, INT*z3),(1-INT)*z3);  % don't accept too close to limits&#xA;  z1 = z1 + z2;                                           % update the step&#xA;  X = X + z2*s;&#xA;  [f2 df2] = eval(argstr);&#xA;  M = M - 1; i = i + (length&amp;lt;0);                           % count epochs?!&#xA;  d2 = df2'*s;&#xA;  z3 = z3-z2;                    % z3 is now relative to the location of z2&#xA;end&#xA;if f2 &amp;gt; f1+z1*RHO*d1 | d2 &amp;gt; -SIG*d1&#xA;  break;                                                % this is a failure&#xA;elseif d2 &amp;gt; SIG*d1&#xA;  success = 1; break;                                             % success&#xA;elseif M == 0&#xA;  break;                                                          % failure&#xA;end&#xA;A = 6*(f2-f3)/z3+3*(d2+d3);                      % make cubic extrapolation&#xA;B = 3*(f3-f2)-z3*(d3+2*d2);&#xA;z2 = -d2*z3*z3/(B+sqrt(B*B-A*d2*z3*z3));        % num. error possible - ok!&#xA;if ~isreal(z2) | isnan(z2) | isinf(z2) | z2 &amp;lt; 0   % num prob or wrong sign?&#xA;  if limit &amp;lt; -0.5                               % if we have no upper limit&#xA;    z2 = z1 * (EXT-1);                 % the extrapolate the maximum amount&#xA;  else&#xA;    z2 = (limit-z1)/2;                                   % otherwise bisect&#xA;  end&#xA;elseif (limit &amp;gt; -0.5) &amp;amp; (z2+z1 &amp;gt; limit)          % extraplation beyond max?&#xA;  z2 = (limit-z1)/2;                                               % bisect&#xA;elseif (limit &amp;lt; -0.5) &amp;amp; (z2+z1 &amp;gt; z1*EXT)       % extrapolation beyond limit&#xA;  z2 = z1*(EXT-1.0);                           % set to extrapolation limit&#xA;elseif z2 &amp;lt; -z3*INT&#xA;  z2 = -z3*INT;&#xA;elseif (limit &amp;gt; -0.5) &amp;amp; (z2 &amp;lt; (limit-z1)*(1.0-INT))   % too close to limit?&#xA;  z2 = (limit-z1)*(1.0-INT);&#xA;end&#xA;f3 = f2; d3 = d2; z3 = -z2;                  % set point 3 equal to point 2&#xA;z1 = z1 + z2; X = X + z2*s;                      % update current estimates&#xA;[f2 df2] = eval(argstr);&#xA;M = M - 1; i = i + (length&amp;lt;0);                             % count epochs?!&#xA;d2 = df2'*s; &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to understand the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;before the loop, the point 3 is initialized as: &lt;code&gt;f3 = f1; d3 = d1; z3 = -z1;&lt;/code&gt; and the comment says &lt;code&gt;Save point 0&lt;/code&gt;. But why does he set &lt;code&gt;z3 = -z1&lt;/code&gt;?&lt;br&gt; (in case someone's wondering, no, it's - apparently - not a mistake because 1. the algo works and 2. I've tried running it with &lt;code&gt;z3 = z1&lt;/code&gt; and it breaks)&lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;how the author is using the Wolfe conditions and its constants?&lt;br&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;interpolation / extrapolation. Where do the equations he's using come from? E.g.:&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;A = 6*(f2-f3)/z3+3*(d2+d3); &#xA; B = 3*(f3-f2)-z3*(d3+2*d2);&#xA; z2 = -d2*z3*z3/(B+sqrt(B*B-A*d2*z3*z3));&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;what's going on in the if-else checks that involve the &lt;code&gt;limit&lt;/code&gt; variable? Note that before entering the &lt;code&gt;while 1&lt;/code&gt;, he set &lt;code&gt;limit = -1&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;b&gt;EDIT:&lt;/b&gt;&lt;br&gt;&#xA;added one more question to the bullet list&lt;/p&gt;&#xA;" OwnerUserId="22812" LastEditorUserId="22812" LastEditDate="2017-01-03T17:46:20.227" LastActivityDate="2017-01-11T16:57:03.830" Title="Understanding MATLAB's `fmincg` optimization function" Tags="|optimization|conjugate-gradient|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="25878" PostTypeId="1" CreationDate="2017-01-03T19:55:42.550" Score="1" ViewCount="241" Body="&lt;p&gt;In the finite element lecture we learned, that it is impractical to use centre of gravity of a triangle as a degree of freedom. So we defined a subspace $P_3'$, where $P_2\subseteq P_3'\subseteq P_3$&#xA;$$P_3'=\{p\in P_3;\ p(a_{123})=-\frac{1}{6}\sum_{l=1,2,3}p(a_l)+\frac{1}{4}\sum_{l,m=1,2,3,\ l\neq m}p(a_{llm})\}$$&#xA;where $a_1$, $a_2$, $a_3$ form a triangle in 2 dimensions, $a_{llm}=\frac{1}{3}(2a_l+a_m)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the reason for this? Why don't we use the centre of gravity?&lt;/p&gt;&#xA;" OwnerUserId="18910" LastActivityDate="2017-05-24T18:06:15.873" Title="Why do we eliminate centre of gravity as degree of freedom in FEM" Tags="|finite-element|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25879" PostTypeId="1" CreationDate="2017-01-04T03:26:56.720" Score="2" ViewCount="551" Body="&lt;p&gt;Let $\Omega$ be a bounded open set in $\mathbb{R}^d$ where $d \geq 1$ is a positive integer, with Lipschitz boundary. Let $k,l$ be non-negative integers and $1 \leq p &amp;lt; \infty $ then if $k &amp;gt; l$ and $\frac{k-l}{d} &amp;gt; \frac{1}{p} - \frac{1}{q}$ then the Sobolev embedding &#xA;$$ W^{k,p}(\Omega) \subset W^{l,q}(\Omega) $$&#xA;is compact. As an example, for $k = 1+s, p=2$ and $l = 2,$ we have&lt;br&gt;&#xA;$$ W^{1+s,2}(\Omega) \subset W^{2,q}(\Omega), \quad s &amp;gt; 2 -2/q. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what does this mean for solving PDEs? and what does it mean for other choices of $k$ as in $k= 2+s,$  i.e.,&#xA;$$ W^{2+s,2}(\Omega) \subset W^{2,q}(\Omega), \quad s &amp;gt; 1 -2/q. $$&#xA;and $k= 3+s,$ i.e.,&#xA;$$ W^{3+s,2}(\Omega) \subset W^{2,q}(\Omega), \quad s &amp;gt; -2/q. $$&lt;/p&gt;&#xA;" OwnerUserId="7984" LastActivityDate="2017-01-05T11:53:58.290" Title="On the Rellich-Kondrachov embedding theorem" Tags="|pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25881" PostTypeId="1" CreationDate="2017-01-04T17:43:13.120" Score="1" ViewCount="238" Body="&lt;p&gt;I have problem with volume loss in FLIP simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately it's necessary to obey the CFL condition when using explicit integration methods (RK2 in my case) to advance particle positions using the Fluid Implicit Particle method, otherwise fluid volume gets lost over time (simulation becomes unstable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turns out that in my case its very expensive to take several sub-iterations per frame and obey the CFL condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought the problem with volume loss could be solved by using an implicit integration method, that'd make the fluid unconditionally stable and would allow me to take larger time steps without obeying the CFL condition and performing the costly sub-iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does such implicit formulation of FLIP exist? If not, is there another way how to alleviate (or completely remove) volume loss in FLIP (other than CFL)?&lt;/p&gt;&#xA;" OwnerUserId="21183" LastActivityDate="2018-05-30T06:57:16.820" Title="Implicit integration for FLIP?" Tags="|fluid-dynamics|implicit-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25882" PostTypeId="2" ParentId="25881" CreationDate="2017-01-04T20:58:10.007" Score="1" Body="&lt;p&gt;In the computer graphics community there exist many approaches that approximate the advection operator of the Navier-Stokes or Euler equations with unconditional stability by particle tracing that are somehow implicit in time. The basic idea is to use a negative time steps and therefore trace particles backwards in time. At the grid points, where velocity samples are located, the particles are integrated to find the position where the particle originated and use and interpolated velocity for the update. Stam &lt;a href=&quot;https://doi.org/10.1145/311535.311548&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; introduced this concept into the computer graphics community, although the concept might have appeared somewhere else before. Zhu and Bridson &lt;a href=&quot;https://doi.org/10.1145/1186822.1073298&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt; used FLIP for incompresible flow, but as far as I see use explicit integration for the particle tracing (positive time step). Maybe the methods in &lt;a href=&quot;https://doi.org/10.1145/311535.311548&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; and &lt;a href=&quot;https://doi.org/10.1145/1186822.1073298&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt; can be combined to get an unconditionally stable FLIP integrator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't know if this will lead to an improved volume conservation. &#xA;Please also note that I made the experience that higher CFL numbers imply a significant reduction in accurary (for the velocity field) and maybe also result in volume loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1145/311535.311548&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1] Stam, J. Stable fluids. SIGGRAPH Proceedings, 1999, 121-128&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1145/1186822.1073298&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2] Zhu, Y.; Bridson, R. Animating Sand As a Fluid ACM Trans. Graph., ACM, 2005&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19766" LastEditorUserId="20688" LastEditDate="2018-03-31T05:11:57.060" LastActivityDate="2018-03-31T05:11:57.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25884" PostTypeId="1" CreationDate="2017-01-05T08:36:45.627" Score="2" ViewCount="1584" Body="&lt;p&gt;I am trying to solve a standard Poisson equation on image with Neumann boundary condition. The Poisson equation has the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Δp=b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $b$ is the divergence of the input image,which is as shown:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/a8suO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a8suO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to write it in Matlab with centred finite difference method and Gauss-Seidel iterative method but the result turns out to be very weird, which is as shown:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JeI7A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JeI7A.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can any kind soul help to spot errors in my code below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{%Specifying parameters&#xA;nx=rows;                           %Number of steps in space(x)(vertical)&#xA;ny=columns;                        %Number of steps in space(y)(horizontal)&#xA;dx=1;&#xA;dy=1;&#xA;h=dx;&#xA;&#xA;&#xA;&#xA;&#xA;%Solving (Dxx+Dyy)p(i,j)=b(i,j) (eqn 3.16) using centred FDM (5-points difference)&#xA;%Gauss-Seidel iterative method&#xA;pn=zeros(nx,ny);                 %Preallocating pn&#xA;p=zeros(nx,ny);                  %Preallocating p&#xA;b=zeros(nx,ny);&#xA;ax=0;                            %counter &#xA;&#xA;for j=2:ny-1&#xA;   for i=2:nx-1&#xA;   b(i,j)=0.5*(grayImage1(i+1,j)-grayImage1(i-1,j)+grayImage(i,j+1)-grayImage(i,j-1))./h;&#xA;&#xA;   end&#xA;end &#xA;&#xA;i=2:nx-1; &#xA;j=2:ny-1;&#xA;for it=1:200000&#xA;        p(1,1)=0;        &#xA;        pn=p;   &#xA;        p(i,j)=0.25*(((pn(i+1,j)+p(i-1,j)))+((pn(i,j+1)+p(i,j-1)))-h^2*(b(i,j)));&#xA;&#xA;    %%boundary conditions &#xA;    %The four corners&#xA;    %p(1,1)=1/2*pn(2,1)+1/2*pn(1,2)-1/4*dx^2*b(1,1);            &#xA;    p(1,ny)=1/2*pn(2,ny)+1/2*pn(1,ny-1)-1/4*dx^2*b(1,ny);            &#xA;    p(nx,1)=1/2*pn(nx,2)+1/2*pn(nx-1,1)-1/4*dx^2*b(nx,1);          &#xA;    p(nx,ny)=1/2*pn(nx-1,ny)+1/2*pn(nx,ny-1)-1/4*dx^2*b(nx,ny);&#xA;&#xA;    %The four sides&#xA;    p(1,j)=1/4*(2*pn(2,j)+pn(1,j+1)+pn(1,j-1)-dx^2*b(1,j));            &#xA;    p(nx,j)=1/4*(2*pn(nx-1,j)+pn(nx,j+1)+pn(nx,j-1)-dx^2*b(nx,j));            &#xA;    p(i,1)=1/4*(2*pn(i,2)+pn(i+1,1)+pn(i-1,1)-dy^2*b(i,1));&#xA;    p(i,ny)=1/4*(2*pn(i,ny-1)+pn(i+1,ny)+pn(i-1,ny)-dy^2*b(i,ny));&#xA;&#xA;    if abs(p(i,j)-pn(i,j))&amp;lt;=zeros(nx-2,ny-2)+0.0001&#xA;        break&#xA;    end&#xA;    ax=ax+1&#xA;end }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks very much!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p/s: I do notice that pure Neumann boundary condition will give rise to non-unique solutions. Hence I followed one of the advice from other answers to set one of the corner to zero. Unfortunately it seems to have no effect on the results.&lt;/p&gt;&#xA;" OwnerUserId="22831" LastEditorUserId="22848" LastEditDate="2017-01-06T08:10:15.177" LastActivityDate="2017-01-14T13:07:17.110" Title="Solving poisson equation on image with neumann boundary condition" Tags="|matlab|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25885" PostTypeId="1" CreationDate="2017-01-05T09:19:06.363" Score="1" ViewCount="240" Body="&lt;p&gt;I am trying to solve an equation by a meshfree method. Shape functions (and their derivatives) have a compact support domain. So, over a given integration cell used for numerical integration, only a small number of shape functions will have nonzero values. At the same time, for each integration point belonging to a cell, the number and indexes of the nonzero shape functions may be different. Hence, I could not construct local stiffness matrices as in typical FEM codes, because the size of them are unknown. And it is not possible to construct some arrays and then use sparse command for assembling. I used sparse command to define stiffness matrix $K$, as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K=sparse(NDOF,NDOF)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then tried to find each element $K_{ij}$ during of computation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for j=1:ngauss&#xA;    //subroutint to &quot;find v&quot; here&#xA;    [fi, fix, fiy]=shape(xg,yg,v);&#xA;    for i=1:length(fi)&#xA;        sub_K=fiy*fiy(i)+fix*fix(i));                 &#xA;        K(v(i),v(:))=K(v(i),v(:))+sparse(weight(j)*areaj*sub_K);&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;v&lt;/code&gt; is the array of indices of nonzero shape functions. But this kind of using sparse command is too slow. Are there any suggestions or any other way for that?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="20688" LastEditDate="2017-04-16T16:40:26.177" LastActivityDate="2017-04-16T16:40:26.177" Title="How to complete sparse matrix?" Tags="|matlab|sparse-matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25886" PostTypeId="2" ParentId="25875" CreationDate="2017-01-05T10:44:40.443" Score="4" Body="&lt;p&gt;According to the following paper, the algorithm creates cracks like you have (see figure 1 and surrounding discussion).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Fournier, Marc. &amp;quot;Surface Reconstruction: An Improved Marching Triangle&#xA;Algorithm for Scalar and Vector Implicit Field Representations.&amp;quot; 2009&#xA;XXII Brazilian Symposium on Computer Graphics and Image Processing.&#xA;IEEE, 2009.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.graphicon.ru/html/2009/conference/se1/14/14_Paper.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.graphicon.ru/html/2009/conference/se1/14/14_Paper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This paper claims that their new algorithm is able to &amp;quot;reduce the cracks in the final resulting mesh&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-01-05T10:44:40.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25887" PostTypeId="2" ParentId="25879" CreationDate="2017-01-05T11:53:58.290" Score="1" Body="&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; this is more like a long comment with some example without proofs of application about the Rellich-Kondrakov theorem, it is not intended as complete answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I report two example of the use of the Rellich-Kondrakov theorem related to the PDE word.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start with note that in particular the Rellich-Kondrakov theorem says that the embedding $W^{l,p}(\Omega) \longrightarrow  L^p(\Omega)$ is compact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example 1: Helmholtz equation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Helmholtz_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Helmholtz&lt;/a&gt; equation is an eigenvalue problem with this form:&#xA;$$&#xA;\left\{&#xA;\begin{aligned}&#xA;- \Delta u= \lambda u \quad \text{in} \quad\Omega\\&#xA;u=0 \quad \text{in}   \quad \partial\Omega&#xA;\end{aligned}&#xA;\right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value $\lambda$ is the eigenvalue of the $-\Delta$ operator with the boundary conditions.&#xA;Considering the 2 dimensional case this problem is related to the membrane vibration ($\lambda_n=$ harmonic) and the minimum eigenvalue, $\lambda_1$, is the fundamental tone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can obtain the weak formulation:&#xA;$$&#xA;\left\{&#xA;\begin{aligned}&#xA; \int_{\Omega} \nabla u \nabla \varphi = \lambda \int_{\Omega} u \varphi \, dx \quad \forall \varphi \in W^{1,2}_0(\Omega) \\&#xA;u \in W^{1,2}_0(\Omega) \setminus \{0\}&#xA;\end{aligned}&#xA;\right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can demonstrate that, under some hypothesis on $\Omega$, the set&#xA;$$&#xA;\{ \lambda \in \mathbb{R} \, : \text{weak formulation problem has a solution } u \neq 0 \} &#xA;$$&#xA;has a positive minimum. In this proof is used the Rellich-Kondrakov theorem to obtain a subsequence the converge to a function in $L^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example 2 Poisson Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the Sobolev's theory is possible show that the weak Poisson problem is a well posed in Hadamard sense, i.e. exist a unique and stable solution. For this proof is necessary the &lt;a href=&quot;https://en.wikipedia.org/wiki/Poincar%C3%A9_inequality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Poincaré inequality&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now according with &lt;a href=&quot;https://en.wikipedia.org/wiki/Rellich%E2%80%93Kondrachov_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Rellich–Kondrachov theorem may be used to prove the Poincaré inequality&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think this can be see as an indirect use of the theorem Rellich-Kondrakov.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2017-01-05T11:53:58.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25891" PostTypeId="1" CreationDate="2017-01-05T21:41:49.303" Score="1" ViewCount="106" Body="&lt;p&gt;Trying to find a way to solve exponentially nonlinear elliptic equation with complex source term i manage to know about such schemes like Godunov, Lax-Friedrich, MUSCL. I still seraching for literature about them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main question is could this schemes do something with&#xA;$$ \dfrac{\partial}{\partial x}\left(ce^{\varphi_n-\varphi_c}\dfrac{\partial\varphi_n}{\partial x}\right) = R$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This schemes traditionally for parabolic/hyperbolic equation, but i could add some time derivative (i erase it because i need stationary solution). Or it is possible to use them with elliptic one? (why not)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know that it could possible if only the scheme with it initial condition will be stable. For example, MUSCL scheme deal with discontinueties, large gradient etc. (from Wiki)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible when $ce^{\varphi_n-\varphi_c}$ change very fast (1e18 in a small area)? Could you adviced me some literature?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. sorry for my english, i working on it.&lt;/p&gt;&#xA;" OwnerUserId="22777" LastEditorUserId="22777" LastEditDate="2017-01-05T21:49:12.560" LastActivityDate="2017-01-05T21:49:12.560" Title="Which numerical scheme should be used?" Tags="|pde|nonlinear-equations|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="25892" PostTypeId="1" CreationDate="2017-01-05T22:40:38.310" Score="3" ViewCount="83" Body="&lt;p&gt;I have this MIP optimization problem, with couple of binary variables; however when I relax one of the binary variables the optimal solution of the objective does not change. But the solving time increase drastically. How could this happen? I noticed that this only happens for just when only I relax that specific variable, not for the rest. Does it like this that when you relax a variable the solution space does not change? is the some MIP optimization logic or property behind this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help.&lt;/p&gt;&#xA;" OwnerUserId="22841" LastActivityDate="2017-01-05T22:40:38.310" Title="Relaxing a variable in MIP" Tags="|optimization|mixed-integer-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25893" PostTypeId="2" ParentId="926" CreationDate="2017-01-06T11:38:55.463" Score="0" Body="&lt;p&gt;Yes, you can learn Matlab using Octave.  Of course there are some limitations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Octave and Matlab share a much of their syntax.  The two are interchangeable in that respect.  The experience in Matlab is more rich and user friendly, particularly when working with graphics, though Octave has a new graphical user interface (GUI) that is in beta.  The two systems will likely converge further as the graphical environment for Octave improves.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of free courses on Coursera that give you a free temporary Matlab student license for the duration of the course.  Search for machine learning courses.&lt;/p&gt;&#xA;" OwnerUserId="22850" LastEditorUserId="22850" LastEditDate="2017-01-06T21:53:10.683" LastActivityDate="2017-01-06T21:53:10.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25899" PostTypeId="1" AcceptedAnswerId="25902" CreationDate="2017-01-08T10:56:58.747" Score="3" ViewCount="496" Body="&lt;p&gt;I have a ''cosmetic'' problem with a singular term in my Matlab script. I am trying to solve the following system of differential equations:&#xA;$$ \begin{aligned}&#xA;y_1' &amp;amp;= y_2,\\&#xA;y_2' &amp;amp;= \frac{2y_1}{x^2}+k(y_1^3 - y_1) - \frac{2y_2}{x}.&#xA;\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where x is the independent variable of y and k is a constant. This system is to be solved with the boundary conditions:&#xA;$$\begin{aligned}&#xA;y_1(0)&amp;amp;=0,\\&#xA;y_1(10) &amp;amp;= 1.&#xA;\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a working script for this with Matlab's bvp4c function. However, the solution for $y_2$ appears to go off to infinity near zero. I've read from Matlab's documentation that bvp4c can solve problems with singular boundary values if they are of the form:&#xA;$$y' = S \cdot \frac{y}{x} + F(x,y,p).$$&#xA;But as you can see there is a $\frac{1}{x^2}$-term in addition to a $\frac{1}{x}$-term. I've been trying to modify the form of the differential equation such that I could write it in the form Matlab wants it to be. However, whatever I do I can't seem to be able to modify it to the desired form. I was hoping that maybe someone might have some insight to this problem. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="22872" LastEditorUserId="713" LastEditDate="2017-01-08T22:05:52.793" LastActivityDate="2017-01-08T22:05:52.793" Title="How to reformulate a 1/x^2 singular term to 1/x so that bvp4c can solve it?" Tags="|matlab|ode|boundary-conditions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25900" PostTypeId="1" CreationDate="2017-01-08T18:18:06.107" Score="4" ViewCount="1000" Body="&lt;p&gt;Suppose we want to approximate a function $f: [a, b] \rightarrow \Re$ with a Chebyshev series:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) \approx \sum_{k=0}^n c_k \, T_k\left( \frac{2x-b-a}{b-a} \right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $T_k(x) = \cos(k\, \cos^{-1}x)$ are Chebyshev polynomials of the first kind. There are two methods of finding the coefficients $c_k$ that make this approximation good.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Projection / Truncation method&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This uses the fact that Chebyshev polynomials are orthogonal. Let [A] = 1 if statement A is true and [A] = 0 otherwise:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align}&#xA;\frac{\pi}{1 + [k \neq 0]} \, c_k &amp;amp;= \int_0^\pi f\left( \frac{b-a}{2} \cos\theta + \frac{b+a}{2} \right) \cos(k\theta)  \, d\theta \\[1ex]&#xA;&amp;amp;= \int_{-1}^1 f\left( \frac{b-a}{2}x + \frac{b+a}{2} \right) T_k(x) \, \frac{1}{\sqrt{1-x^2}} \, dx \\[1ex]&#xA;&amp;amp;= \int_a^b f(x) \; T_k\left( \frac{2x-b-a}{b-a} \right) \frac{2}{\sqrt{(b-a)^2 - (2x-b-a)^2}} \, dx&#xA;\end{align} $$&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Interpolation / Collocation method&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The Chebyshev nodes in [–1, 1] are $x_k = \cos \displaystyle{\frac{k\pi}{n}}$ where k = 0, 1, ..., n. We can force the Chebyshev series to cut $f(x)$ at those nodes by solving the linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f\left( \frac{b-a}{2} \cos \left( \frac{i\pi}{n} \right) + \frac{b+a}{2} \right)  = \sum_{j=0}^n c_j \, \cos \left( \frac{ij\pi}{n} \right) \qquad \forall \, i = 0, \dotsm, n $$&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Questions&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Which method is better? In terms of accuracy, speed or any relevant criteria.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Which method is implemented in NumPy and Chebfun?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22877" LastEditorUserId="22877" LastEditDate="2017-01-15T13:06:38.970" LastActivityDate="2017-01-15T13:06:38.970" Title="Chebyshev approximation by projection vs interpolation" Tags="|numerical-analysis|interpolation|numpy|approximation|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="25901" PostTypeId="1" CreationDate="2017-01-08T21:04:20.083" Score="1" ViewCount="70" Body="&lt;p&gt;I am solving a elastodynamics (vector valued elastic wave) equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I create the 2D mesh in Gmsh discretised into triangular elements of second order. Therefore, it is my understanding that the element is a triangle with total six nodes, i.e., 3 vertices and 1 node on each of the 3 edges. I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q.1: The dof for a simple scalar problem is equal to the no. of nodes on the mesh. Therefore, the dof for my problem should be equal to twice the no. of nodes on the mesh right? This is because each node can only experience displacement in 'x' or 'y' direction as a result of wave propagation. Is this correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q.2: Each triangular element on the mesh will have 12 dof. Right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read somewhere that the velocity of wave in both directions will also constitute 2 dof, but that doesn't seem to make sense. &lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2017-01-08T21:04:20.083" Title="Degree of freedom for elastic wave propagation problem" Tags="|finite-element|pde|hyperbolic-pde|wave-propagation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25902" PostTypeId="2" ParentId="25899" CreationDate="2017-01-08T21:37:53.863" Score="2" Body="&lt;p&gt;Your equation, written as a 2nd-order ODE, is&#xA;$$ y'' = -\frac{2}{x}y' + \frac{2}{x^2}y + ky(y^2-1). $$&#xA;Substituting&#xA;$$ y(x) = x w(x), $$&#xA;it becomes&#xA;$$ w'' = -\frac{4}{x}w' + k w(1 - x^2 w^2), $$&#xA;which matches the form accepted by bvp4c with the matrix&#xA;$$ S = \begin{pmatrix} 0 &amp;amp; 0 \\ -4 &amp;amp; 0 \end{pmatrix}. $$&#xA;Other transformations of the form $y=(x+\alpha x^{-2})w$ also get it to this kind of form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is, in fact, a well-known theory of changes of variables in&#xA;linear ODEs (see, for example, this theorem due to Liouville:&#xA;&lt;a href=&quot;https://mathoverflow.net/a/187499&quot;&gt;https://mathoverflow.net/a/187499&lt;/a&gt;), and finding a transformation like&#xA;$y=xw$ doesn't actually require any guesswork. Applying the transformations to the first-order form of the ODE in $(y_1,y_2)$ makes this unnecessarily unwieldy.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2017-01-08T21:37:53.863" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25903" PostTypeId="2" ParentId="22022" CreationDate="2017-01-09T03:18:00.813" Score="11" Body="&lt;p&gt;I've implemented this recently, basically it counts how many times each specific colour borders another colour to make up a frequency table. To generate an image, a random colour and position are selected and the rest of the image is built up from there. Results aren't very coherent, but they match the colour palette of the original.&lt;/p&gt;&#xA;&lt;p&gt;In order to make sure each colour has a neighbour, the colours are bucketed. You can modify this degree of information loss. You can also use 8 neighbours or 4 neighbours and also learn the direction of colours (top, left, right, bottom etc.).&lt;/p&gt;&#xA;&lt;p&gt;At this time it only generates random images after learning from input images, the code is here: &lt;a href=&quot;https://gist.github.com/JonnoFTW/5b036797593c5c40aabdf2d6c1399387&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/JonnoFTW/5b036797593c5c40aabdf2d6c1399387&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: Full repository is here: &lt;a href=&quot;https://github.com/JonnoFTW/markov-img-gen&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JonnoFTW/markov-img-gen&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It should be simple enough to try and make it fill in gaps in images, though I suspect the result would be nonsensical.&lt;/p&gt;&#xA;&lt;p&gt;Here's some sample output: &lt;a href=&quot;https://i.stack.imgur.com/uZeWn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uZeWn.png&quot; alt=&quot;Sample Markov Based image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22879" LastEditorUserId="22879" LastEditDate="2021-09-15T05:59:07.993" LastActivityDate="2021-09-15T05:59:07.993" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25904" PostTypeId="1" AcceptedAnswerId="25905" CreationDate="2017-01-09T09:02:01.107" Score="1" ViewCount="74" Body="&lt;p&gt;I looking for literature  for $\mathbb{P}_k$ bubble enriched for finite elements. &lt;/p&gt;&#xA;" OwnerUserId="22334" LastActivityDate="2017-01-09T10:04:19.103" Title="Literature for $\mathbb{P}_k$ bubble enriched for finite elements" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25905" PostTypeId="2" ParentId="25904" CreationDate="2017-01-09T10:04:19.103" Score="2" Body="&lt;p&gt;In our paper &lt;a href=&quot;http://dx.doi.org/10.1051/m2an/2015029&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite element decomposition ... flow equations&lt;/a&gt; -- see my webpage &lt;a href=&quot;http://janheiland.de/publications.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://janheiland.de/publications.html&lt;/a&gt; for a link to the postprint -- we have touched on such &quot;bubble enriched&quot; Finite Element spaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sources we based on were&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;V. Girault and P.-A. Raviart, &lt;em&gt;Finite Element Methods for Navier-Stokes Equations&lt;/em&gt;. Springer-Verlag. Berlin (1986)&lt;/li&gt;&#xA;&lt;li&gt;C. Bernardi and G. Raugel, &lt;em&gt;Analysis of some finite elements for the Stokes problem&lt;/em&gt;. Math. Comput. 44(1985) 71–79.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2017-01-09T10:04:19.103" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25907" PostTypeId="2" ParentId="926" CreationDate="2017-01-09T19:18:17.623" Score="2" Body="&lt;p&gt;The best free language which has MATLAB-like syntax is Julia. It's also faster and has a more extensive package system (among other reasons why it's better...), but the linear algebra syntax is almost exactly the same (many algorithms you can translate to MATLAB by changing &lt;code&gt;A[i]&lt;/code&gt; for indexing to &lt;code&gt;A(i)&lt;/code&gt;). I believe it's the best language to learn right now, and you'll &quot;accidentally know MATLAB&quot; just by using it.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-01-09T19:18:17.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25909" PostTypeId="2" ParentId="15891" CreationDate="2017-01-10T02:14:26.783" Score="3" Body="&lt;p&gt;Perhaps surrogate-based optimization algorithms are what you are looking for. These algorithms use surrogate models to replace the real computationally expensive models during the optimization process and try to get a suitable solution using as few evaluations of the computationally expensive models as possiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the &lt;a href=&quot;http://www.sfu.ca/~gwa5/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mode Pursuing Sampling method&lt;/a&gt; may be used to solve your problem. This algorithm uses the RBF surrogate model to approximate the expensive objective function and can handle nonlinear constraints. More importantly, it selects multiple candidates to do the expensive function evaluations so that you can distribute these candidates for parallel computing to further speed up the search process. The code is open-source and written in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wang, L., Shan, S., &amp;amp; Wang, G. G. (2004). Mode-pursuing sampling method for global optimization on expensive black-box functions. Engineering Optimization, 36(4), 419-438. &lt;/p&gt;&#xA;" OwnerUserId="19983" LastActivityDate="2017-01-10T02:14:26.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25910" PostTypeId="2" ParentId="15891" CreationDate="2017-01-10T04:16:07.210" Score="2" Body="&lt;p&gt;I am not sure a parallel algorithm is truly what you're looking for. It's your function evaluations which are very costly. What you want to do is parallelize the function itself, not necessarily the optimization algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can't do that, then there's a middle ground between exhaustive search and Newton algorithm, it's Monte Carlo methods. You can, on a bunch of different cores/nodes, start the same algorithm which is prone to fall to local optima (say quasi-Newton algorithms), but all with random initial conditions. Your best guess then for the true optima is the minimum of the minimums. This is trivial to parallelize and can be used to extend any method. While not perfectly efficient, if you have enough computing power at your disposal it can definite win the programming productivity vs algorithm performance battle (if you have lots of computing power, this can finish before you'd ever be finished making a fancier algorithm).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-01-10T04:16:07.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25912" PostTypeId="2" ParentId="24199" CreationDate="2017-01-10T20:13:20.923" Score="6" Body="&lt;p&gt;There as a general &lt;a href=&quot;http://pywavelets.readthedocs.io/en/latest/ref/swt-stationary-wavelet-transform.html#multilevel-n-dimensional-swtn&quot; rel=&quot;noreferrer&quot;&gt;n-dimensional SWT&lt;/a&gt; for Python in the PyWavelets package as of the 0.5.0 release.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basic usage with &lt;code&gt;data&lt;/code&gt; stored in a NumPy array would be as follows (shown here for a 4-level decomposition and Debauchies 'db2' wavelet).  PyWavelets uses the same wavelet naming conventions as the Matlab Wavelet Toolbox.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import pywt&#xA;coeffs = pywt.swtn(data, wavelet='db2', level=4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The example above takes &amp;lt;2 seconds for a 128x128x128 data array on my system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are willing to &lt;a href=&quot;http://pywavelets.readthedocs.io/en/latest/#install&quot; rel=&quot;noreferrer&quot;&gt;build from source&lt;/a&gt;, there is also an n-dimensional inverse SWT available in the master branch.  Note that the inverse stationary wavelet transform is not currently implemented in a very efficient manner (particular for larger number of levels of decomposition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 3D, the SWT is redundant by a factor of &lt;code&gt;(1 + 7*L)&lt;/code&gt; for an &lt;code&gt;L&lt;/code&gt;-level decomposition (Although the implementation in PyWavelets currently returns the approximation coefficients at every level, not just the final one, leading to a redundancy of &lt;code&gt;8*L&lt;/code&gt; in 3D).&lt;/p&gt;&#xA;" OwnerUserId="22891" LastActivityDate="2017-01-10T20:13:20.923" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25914" PostTypeId="2" ParentId="19693" CreationDate="2017-01-11T08:55:17.927" Score="0" Body="&lt;p&gt;Here's an implementation of restricted cubic spline (based on &lt;a href=&quot;https://uk.mathworks.com/matlabcentral/fileexchange/41241-restricted-cubic-spline&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; matlab code)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.linalg as lin&#xA;&#xA;def rcs(x,y,knots):&#xA;    n = len(y)&#xA;    k = knots&#xA;    X1 = x&#xA;    q = len(k)-1&#xA;    myX=np.zeros((n,len(knots)-2))&#xA;&#xA;    for j in range(q-1):&#xA;        tmp1 = (x-k[j])**3 * (x&amp;gt;k[j])&#xA;    tmp2 = (x-k[q-1])**3 * (x&amp;gt;k[q-1])*(k[q]-k[j])&#xA;    XX= tmp1-tmp2/(k[q]-k[q-1])&#xA;        tmp1 = (x-k[q])**3 * (x&amp;gt;k[q])&#xA;        tmp2 = (k[q-1]-k[j])&#xA;    XX = XX+tmp1*tmp2/(k[q]-k[q-1])&#xA;    myX[:,j]=XX&#xA;&#xA;    X = np.hstack( (np.ones((n,1)),np.reshape(X1,(n,1)),myX) )&#xA;    bhat = np.linalg.lstsq(X,y)[0]&#xA;    bhatt = np.zeros(len(knots)+1)&#xA;    bhatt[len(bhat)] = (bhat[2:]*(k[0:-2]-k[-1])).sum()&#xA;    bhatt[len(bhat)] = bhatt[len(bhat)] / (k[-1]-k[-2])&#xA;    bhatt = np.hstack([bhatt, 0])    &#xA;    bhatt[-1] = (bhat[2:]*(k[0:-2]-k[-2])).sum()&#xA;    bhatt[-1] = bhatt[-1] / (k[-2]-k[-1])&#xA;    bhat = np.hstack((bhat, bhatt[-2:]))&#xA;    return bhat&#xA;&#xA;def speval(x,coefs,knots):&#xA;    tmp = coefs[0] + coefs[1]*x&#xA;    for k in range(len(knots)): tmp = tmp + coefs[k+2]*((x-knots[k])**3)*(x&amp;gt;knots[k])&#xA;    return tmp&#xA;&#xA;&#xA;&#xA;import pandas as pd&#xA;x = np.random.randn(300)*np.sqrt(2)&#xA;e = np.random.randn(300)*np.sqrt(0.5)&#xA;y = np.sin(x)+e&#xA;df = pd.DataFrame([x,y]).T&#xA;df.columns = ['x','y']&#xA;df = df.sort_index(by='x')&#xA;print df.head()&#xA;knots=np.array([-5.5938, -3.7732, -1.9526, -0.1320, 1.6886, 3.5092, 5.3298]);&#xA;bhat = rcs(df.x,df.y,knots)&#xA;print bhat&#xA;df['spline'] = speval(df.x, bhat, knots)&#xA;df2 = df.set_index('x')&#xA;df2[['y','spline']].plot()&#xA;plt.hold(True)&#xA;for k in knots: plt.plot(k,speval(k,bhat,knots),'rd')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8e9Gz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8e9Gz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="621" LastActivityDate="2017-01-11T08:55:17.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25915" PostTypeId="1" CreationDate="2017-01-11T13:38:56.263" Score="2" ViewCount="134" Body="&lt;p&gt;Burgers' equation is a fundamental PDE used in various fields such as number theory, gas dynamics, heat conduction, elasticity, etc. It is crucial especially for developing numerical models for turbulence since nonlinear viscous Burgers' equation can be considered as an approach to the Navier-Stokes equations. It can be stated as &#xA;$$\frac{\partial u}{\partial t} + \frac{\partial}{\partial x} \left(\frac{u^2}{2}\right) = \nu \frac{\partial^2 u}{\partial x^2}$$ where $a&amp;lt;x&amp;lt;b$ , $t&amp;gt;0$ with appropriate initial and boundary conditions given. (it can be determined later)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let assume that the finite difference numerical method are considered to obtain approximate solution of this viscous equation. In my perspective, there are two decisions need to be made to have stable solutions  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What kind of finite difference method can be considered for inviscid and viscous part (or both of them) i.e., Lax-Wendroff, Lax Ricthmyer, Crank-Nicholson, Lax-Friedrich's, Roe's scheme ?&lt;/li&gt;&#xA;&lt;li&gt;How one can decide the value of the artificial diffusion number ($\nu$) ? Should we consider its relation with Reynolds number since it is related with the Navier-Stokes equations ? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note : This viscous Burgers' equation is not going to be considered as linearized. It will be considered in a nonlinear form.  &lt;/p&gt;&#xA;" OwnerUserId="11495" LastEditorUserId="7452" LastEditDate="2017-01-11T22:04:08.827" LastActivityDate="2017-01-11T22:04:08.827" Title="How one could choose the value of viscous coefficient for obtaining stable solution of Burgers' equation?" Tags="|pde|finite-difference|numerical-analysis|fluid-dynamics|parabolic-pde|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25916" PostTypeId="1" CreationDate="2017-01-11T15:42:07.693" Score="2" ViewCount="354" Body="&lt;p&gt;I have a linear system of the form $\mathbf{A}\mathbf{x} = \mathbf{f}$. If the length of the vector $\mathbf{x}$ is $N$, meaning that there are $N$ unknowns, then the matrix $\mathbf{A}$ has seven bands and the matrix is sparse. The matrix is not symmetric. It looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\pmatrix{a_1&amp;amp;a_2&amp;amp;a_3&amp;amp;a_4&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;\\&#xA;b_1&amp;amp;b_2&amp;amp;b_3&amp;amp;b_4&amp;amp;b_5&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;\\&#xA;c_1&amp;amp;c_2&amp;amp;c_3&amp;amp;c_4&amp;amp;c_5&amp;amp;c_6&amp;amp;&amp;amp;&amp;amp;&amp;amp;\\&#xA;d_1&amp;amp;d_2&amp;amp;d_3&amp;amp;d_4&amp;amp;d_5&amp;amp;d_6&amp;amp;d_7&amp;amp;&amp;amp;&amp;amp;\\&#xA;&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;&amp;amp;\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;\\&#xA;&amp;amp;&amp;amp;&amp;amp;k_1&amp;amp;k_2&amp;amp;k_3&amp;amp;k_4&amp;amp;k_5&amp;amp;k_6&amp;amp;k_7&amp;amp;&amp;amp;&amp;amp;\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp;l_1&amp;amp;l_2&amp;amp;l_3&amp;amp;l_4&amp;amp;l_5&amp;amp;l_6&amp;amp;l_7&amp;amp;&amp;amp;&amp;amp;\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp;&amp;amp;\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;w_1&amp;amp;w_2&amp;amp;w_3&amp;amp;w_4&amp;amp;w_5&amp;amp;w_6&amp;amp;w_7\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;x_1&amp;amp;x_2&amp;amp;x_3&amp;amp;x_4&amp;amp;x_5&amp;amp;x_6\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;y_1&amp;amp;y_2&amp;amp;y_3&amp;amp;y_4&amp;amp;y_5\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;z_1&amp;amp;z_2&amp;amp;z_3&amp;amp;z_4\\&#xA;}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the vector $\mathbf{f}$ looks like this:&#xA;$\pmatrix{f_1\\f_2\\f_3\\f_4\\\vdots\\\\f_i\\f_{i+1}\\\vdots\\\\f_{N-3}\\f_{N-2}\\f_{N-1}\\f_{N}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the general method is to use a LU decomposition and as the matrix is constant for me, I can pre-compute its decomposition and store it and use it to solve for the unknowns. But I would like to know if there is any method to solve such systems faster than the LU decomposition. Methods may not be restricted to direct methods only.&lt;/p&gt;&#xA;" OwnerUserId="16601" LastActivityDate="2017-01-11T15:42:07.693" Title="Fast solution of a heptadiagonal linear system" Tags="|matrix|linear-solver|banded-matrix|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25917" PostTypeId="2" ParentId="19591" CreationDate="2017-01-11T15:49:18.020" Score="2" Body="&lt;p&gt;You specify an appropriate boundary condition for all flow variables including velocity over the whole boundary of the computational domain, which essentially means you know the face velocity on the boundary and you do not need to construct a Rhie-Chow face velocity.&lt;/p&gt;&#xA;" OwnerUserId="8166" LastActivityDate="2017-01-11T15:49:18.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25919" PostTypeId="2" ParentId="23782" CreationDate="2017-01-11T22:41:09.560" Score="2" Body="&lt;p&gt;The basic issue is that generally $\exp(\ln(x)) \neq x$ when using a fixed-precision floating-point format. This is true &lt;em&gt;independent&lt;/em&gt; of the accuracy with which the exponential and logarithm functions are being computed, due to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Pigeonhole_principle&quot; rel=&quot;nofollow noreferrer&quot;&gt;pidgeonhole principle&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without loss of generality, this effect is most easily demonstrated by using a binary logarithm &lt;code&gt;log2&lt;/code&gt; in conjunction with one of the IEEE-754 binary floating-point formats. With such a format, each &lt;a href=&quot;https://en.wikipedia.org/wiki/Binade&quot; rel=&quot;nofollow noreferrer&quot;&gt;binade&lt;/a&gt; comprises an identical number of encodings. If we compute &lt;code&gt;log2&lt;/code&gt; of &lt;em&gt;two&lt;/em&gt; consecutive binades $[4,8)$ and $[8,16)$ all results will all fall into the &lt;em&gt;single&lt;/em&gt; binade $[2,4)$. Since there are twice as many source encodings than target encodings are available, we must have plenty of collisions by the pidgeonhole principle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All logarithms are contracting operations. This means that when using a fixed-precision floating-point computation, multiple inputs can map to the same output. Information is lost due to this compression, and as a consequence, when we expand the numbers in $[2,4)$ back to $[4,16)$ with &lt;code&gt;exp2&lt;/code&gt;, for many $x$ we cannot get back the starting value even if both functions use a correctly rounded implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reversing the order of operations, that is, use of an expanding operation followed by the contracting inverse operation, we can &lt;em&gt;sometimes&lt;/em&gt; guarantee that every operand $x$ round-trips successfully, as long as there is no intermediate overflow or underflow, and assuming correctly-rounded operations. I do not know (off-hand) whether this guarantees $\ln(\exp(x)) = x$, but the graph in &lt;a href=&quot;https://scicomp.stackexchange.com/a/23790/20458&quot;&gt;Laurent Duval's answer&lt;/a&gt; seems to suggest it. &lt;a href=&quot;https://hal.inria.fr/hal-01148409/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; shows that the square root of the square of a floating-point number $x$ is exactly $\lvert x \lvert$, under the restrictions stated.&lt;/p&gt;&#xA;" OwnerUserId="20458" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2017-01-11T22:41:09.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25921" PostTypeId="1" CreationDate="2017-01-12T09:39:37.527" Score="1" ViewCount="63" Body="&lt;p&gt;I worked on a method based on this paper, &quot;Gradient methods for minimizing composite functions&quot;. But unfortunately, every time I code it doesn't work. Even I investigate inequalities of the paper numerically, but I couldn't figure out what's the problem. Is there any matlab implementation of the paper available? I search the internet but I didn't find any thing? In my work, I don't need to find Lipschitz constant as it did in paper. &lt;/p&gt;&#xA;" OwnerUserId="8223" LastEditorUserId="8223" LastEditDate="2017-01-12T14:24:24.347" LastActivityDate="2017-01-12T14:24:24.347" Title="Is there any implementation of Nesterov's paper: Gradient methods for minimizing composite functions" Tags="|convex-optimization|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25922" PostTypeId="1" AcceptedAnswerId="25924" CreationDate="2017-01-12T16:45:21.473" Score="0" ViewCount="2583" Body="&lt;p&gt;Is there a way in which Depth First Traversal will put a node into the stack more than once using the general algorithm as shown &lt;a href=&quot;https://www.tutorialspoint.com/data_structures_algorithms/depth_first_traversal.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is it compulsory for all nodes to be entered into the stack atleast once? Is there any case in which a node is not entered into the stack?&lt;/p&gt;&#xA;" OwnerUserId="22917" LastActivityDate="2017-01-12T17:12:50.050" Title="Can a Depth first traversal of a graph visit node more than once?" Tags="|algorithms|graph-theory|" AnswerCount="1" CommentCount="3" ClosedDate="2017-01-13T22:42:00.990" ContentLicense="CC BY-SA 3.0" />
  <row Id="25924" PostTypeId="2" ParentId="25922" CreationDate="2017-01-12T17:12:50.050" Score="3" Body="&lt;p&gt;Depth first search will put a node into the stack only once. The usual way to perform DFS involves marking a vertex as marked while pushing it into the stack and not pushing an already marked vertex again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A node will not enter the stack if and only if it is not part of the connected component involving the DFS start vertex. This only matters for graphs made out of disjoint connected components and not for connected graphs.&lt;/p&gt;&#xA;" OwnerUserId="22802" LastActivityDate="2017-01-12T17:12:50.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25925" PostTypeId="2" ParentId="1671" CreationDate="2017-01-12T18:36:26.197" Score="1" Body="&lt;p&gt;SZ (developed by Argonne in 2016) could be a good choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SZ: Fast Error-Bounded Floating-point Data Compressor for Scientific Applications&#xA;&lt;a href=&quot;https://collab.cels.anl.gov/display/ESR/SZ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://collab.cels.anl.gov/display/ESR/SZ&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22920" LastActivityDate="2017-01-12T18:36:26.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25927" PostTypeId="1" CreationDate="2017-01-13T01:25:03.893" Score="2" ViewCount="1266" Body="&lt;p&gt;I'm trying to make a CFD model where I can place a source and a sink anywhere in a grid and get the fluid flow rate across each cell boundary between those locations.  I'm starting simple with a 3x3 grid and solving continuity for each grid element, but that leaves me a few equations short (9 equations, 12 unknowns).  In general I'd like to be able to raise that to a 64x64 grid with multiple flow inputs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to constrain the system such that I can solve it with continuity equations or is this more difficult than I had initially thought?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a simple diagram of my 3x3 grid (o = source, x = drain):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-------&#xA;|o| | |&#xA;-------&#xA;| | | |&#xA;-------&#xA;| | |x|&#xA;-------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;within the above grid is 9 cells and 12 cell boundaries.  How do I fully constrain the system to solve for the flow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: doing some reading on CFD, it would seem that storing the velocity for the CENTRE of each cell is advantageous and the boundary velocities would be calculated based on the centre values for surrounding cells...  Although I still don't entirely know how my equations would look with that.&lt;/p&gt;&#xA;" OwnerUserId="22924" LastEditorUserId="22925" LastEditDate="2017-01-13T18:11:23.517" LastActivityDate="2017-01-25T09:05:55.297" Title="Simple methods for solving 2D steady incompressible flow?" Tags="|fluid-dynamics|constraints|incompressible|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25928" PostTypeId="2" ParentId="25927" CreationDate="2017-01-13T07:21:31.277" Score="1" Body="&lt;p&gt;With the continuity eqn only, you are missing all the mechanical balance: viscous and/or inertial effects will decide of the streamlines of such a flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your major aim is to keep it as simple as possible, I would go for a lubrication approximation. If you consider that your fluid is seeping between two flat plates with a very narrow gap, in the end the equations will be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ c v = -\nabla p $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \nabla \cdot v = 0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c$ is a friction coefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't say how you currently solve for continuity equation, so it's a bit difficult to say how you should extend the method for this system. Let's just say that quite likely you'll want a finite volume type of grid, with pressures in cell centers and velocities in each direction at midpoint of cell edges orthogonal to this direction.&lt;/p&gt;&#xA;" OwnerUserId="17850" LastActivityDate="2017-01-13T07:21:31.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25930" PostTypeId="2" ParentId="25927" CreationDate="2017-01-13T12:51:10.607" Score="2" Body="&lt;p&gt;I cannot comment, so as an answer may I recommend looking at something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/&lt;/a&gt;&#xA;form start to finish?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This takes the reader from basic discretisation of convective and diffusive terms in 1d and 2d, over a way of dealing with incompressibility via an equation for pressure derived from the continuity equation, to finally a basic lid-driven cavity code. All is done in the context of finite differences with easy to interpret code samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed the above course myself and found if very helpful in getting started.&lt;/p&gt;&#xA;" OwnerUserId="16739" LastActivityDate="2017-01-13T12:51:10.607" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25931" PostTypeId="2" ParentId="21709" CreationDate="2017-01-13T13:15:13.423" Score="4" Body="&lt;p&gt;Continuity equation is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{D \rho}{ D t}+\rho \nabla \cdot \vec{U}=0$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if it is incompressible then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \cdot \vec{U}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just check if the velocity is solenoidal.If it is, then it is incompressible.&lt;/p&gt;&#xA;" OwnerUserId="8166" LastActivityDate="2017-01-13T13:15:13.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25932" PostTypeId="2" ParentId="23551" CreationDate="2017-01-13T14:12:05.753" Score="1" Body="&lt;p&gt;The most efficient way would likely be to perform one step of Gauss-Elimination on A to eliminate the sub-diagonal and then store the inverse of the new main diagonal, the new modified super-diagonal and the constants to preform forward substitution on &lt;strong&gt;b&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way each rhs will require one forward substitution to modify &lt;strong&gt;b&lt;/strong&gt; followed by one back-solve. The forward operations consist of N additions and N multiplications and the backward operations consist of 2N multiplications and N additions. This adds up to 5N operations and no divisions (which are slower).&lt;/p&gt;&#xA;" OwnerUserId="16739" LastActivityDate="2017-01-13T14:12:05.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25933" PostTypeId="2" ParentId="19807" CreationDate="2017-01-13T15:04:19.410" Score="4" Body="&lt;p&gt;Including even one Dirichelt condition changes the problem you are trying to solve and will not give you the correct solution! You must fulfil the Discrete Compatibility Criteria, see e.g. first and second pages of:&#xA;&lt;a href=&quot;http://eprints.ma.man.ac.uk/894/2/covered/MIMS_ep2007_156_Sample_Chapter.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eprints.ma.man.ac.uk/894/2/covered/MIMS_ep2007_156_Sample_Chapter.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It basically states that the summation of each element of &lt;strong&gt;b&lt;/strong&gt; must be equal to zero for all-Neumann problems, otherwise your solution will drift as there is nothing holding it at the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You simply need to add a line stating that &lt;strong&gt;b&lt;/strong&gt; = &lt;strong&gt;b&lt;/strong&gt; - mean(&lt;strong&gt;b&lt;/strong&gt;) in your code before solving, if you are solving in double precision. In single precision it might also be necessary to ensure, at every other iteration, that the residual in biCGstab also meets the Discrete Compatibility Criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attempted to solve an all-Neumann problem using CG by imposing a Dirichlet condition at a single point, as one is usually told to do anecdotally by e.g. some Prof., and then compared it to using a Discrete Cosine Transform approach. It does not yield the same result, using the Discrete Compatibility Criteria instead does.&lt;/p&gt;&#xA;" OwnerUserId="16739" LastEditorUserId="16739" LastEditDate="2018-05-27T22:11:13.077" LastActivityDate="2018-05-27T22:11:13.077" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="25935" PostTypeId="2" ParentId="25884" CreationDate="2017-01-13T16:05:03.147" Score="3" Body="&lt;p&gt;I cannot recommend fixing a corner to zero, that changes the problem you are trying to solve, instead just add before you iterate a line with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;b = b - mean(b);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is called the Discrete Compatibility Criteria. May I refer to my own answer to the following: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19807/solving-a-linear-equation-system-with-pure-neumann-condition/25933#25933&quot;&gt;Solving a linear equation system with pure Neumann condition&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an effort to make this an answer, the following code will generate the matrix version of your code and let you use e.g. conjugate gradient as a solver. I grabbed it out of some of the code on: &lt;a href=&quot;http://math.mit.edu/classes/18.086/2008/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://math.mit.edu/classes/18.086/2008/&lt;/a&gt;&#xA;It combines two 1d matrices for the all-Neumann Poisson problem into a large matrix for the 2d problem using the Kronecker Product:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;maxit = 1000;&#xA;tol = 1e-6;&#xA;&#xA;e = ones(nx-4,1);&#xA;A1= spdiags([1 -1 1; e*[1 -2 1]; 1 -1 1],-1:1,nx-2,nx-2); %/dx^2;&#xA;e = ones(ny-4,1);&#xA;A2= spdiags([1 -1 1; e*[1 -2 1]; 1 -1 1],-1:1,ny-2,ny-2); %/dy^2;&#xA;A = -( kron(speye(ny-2),A1)+kron(A2,speye(nx-2)) ); clear A1 A2&#xA;b = reshape(b,(nx-2)*(ny-2),1);&#xA;b = b - mean(b);&#xA;&#xA;x = pcg(A,b,tol,maxit);&#xA;&#xA;x = reshape(x,nx-2,ny-2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16739" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-01-14T13:07:17.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25936" PostTypeId="1" CreationDate="2017-01-13T16:33:29.127" Score="4" ViewCount="375" Body="&lt;p&gt;I am looking find a way of efficiently generating a random, undirected subgraph $G_s$ with $N$ vertices, using a subset of edges from an exisiting undirected graph $G$, also of size $N$, where the subgraph has only nodes with degree $k \leq k_{max}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, I want to create a new random graph with the same number of vertices, but with a random set of the original edges from $G$ removed such that the maximum degree of the new graph is less than some threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already developed a working algorithm (MATLAB) which randomly prunes edges from nodes with degree $k &amp;gt; k_{max}$ until this condition is met - however I feel it is far from optimal. It is important that the new random subgraph has the same number of vertices and remains undirected (symmetric). Disconnected vertices (with no in/out edges are allowed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally both original graph $G$ and new random subgraph $G_s$ should result in sparse matrices (i.e. an edge list), however a full adjacency matrix will suffice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!!&lt;/p&gt;&#xA;" OwnerUserId="22930" LastActivityDate="2020-04-28T12:01:55.163" Title="Efficiently generate a random subgraph (Gs) with maximum degree K, using only edges from an existing graph G" Tags="|matlab|python|graph-theory|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25938" PostTypeId="2" ParentId="25936" CreationDate="2017-01-13T20:44:33.247" Score="1" Body="&lt;p&gt;Okay, here is an efficient but partial solution:&#xA;Randomly select a vertex, and from that create the &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimum_spanning_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimum spanning tree&lt;/a&gt;. This will make sure that you select each vertex and an associated edge, with minimal degree. Of course this graph is unlikely to include any disconnected vertices. So just include them if they exist in the original graph. If you would like to impose a degree constraint on the MST, you could &lt;a href=&quot;https://cs.stackexchange.com/questions/22775/minimal-spanning-tree-with-degree-constraint&quot;&gt;check here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this stage, a minimal graph is established. You could then select some random edges from the original graph and add them until your degree criteria is satisfied. I call this a partial solution because the graph will be pseudo-random, i.e. the only randomness source stem from the selection of initial vertex, and the random edges added afterwards. But that might still work for you.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="-1" LastEditDate="2017-04-13T12:48:31.170" LastActivityDate="2017-01-13T20:44:33.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25939" PostTypeId="2" ParentId="25927" CreationDate="2017-01-13T22:58:46.777" Score="1" Body="&lt;p&gt;Ironically, there is an iterative method called &quot;SIMPLE&quot; (semi-implicit method for pressure-linked equations) designed to resolve the steady state navier-stokes equations based on a predictor-corrector scheme.  It works by solving a linearized form of the momentum equation (predictor step) which produces a velocity field which generally does not satisfy continuity.  The predicted velocity is used to obtain an equation for pressure which is constrained by the continuity equation.  The result of this equation is a pressure field  which roughly satisfies the continuity equation (corrector step).  Using the updated pressure field, one can easily update the velocity field again so that it satisfies continuity with less error.  One repeats this process iteratively until a steady state is reached.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example of its implementation, you may want to look at OpenFOAM's simpleFoam solver for an example of such an implementation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/OpenFOAM/OpenFOAM-2.3.x/tree/master/applications/solvers/incompressible/simpleFoam&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/OpenFOAM/OpenFOAM-2.3.x/tree/master/applications/solvers/incompressible/simpleFoam&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Admittedly, OpenFOAM is not the easiest software to read and understand, but it shows all of the aforentioned steps (predictor in UEqn.H and corrector in pEqn.H).  It may help you as a guide to developing your own implementation.  For a more detailed explanation of the terms in the software, you may consider looking at some of the tutorials from Chalmers.  In particular, I recommend &quot;A look inside icoFoam (and pisoFoam)&quot;:&#xA;&lt;a href=&quot;http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2013/aLookInsideIcoFoam.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2013/aLookInsideIcoFoam.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2017-01-13T23:07:50.553" LastActivityDate="2017-01-13T23:07:50.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25940" PostTypeId="2" ParentId="25900" CreationDate="2017-01-14T00:41:37.663" Score="2" Body="&lt;p&gt;Please &lt;strong&gt;don't downvote this answer just because it's incomplete&lt;/strong&gt;. My intention is to let whoever answering my question build on it, rather than write from scratch. If your answer is more comprehensive than mine, then I'll mark yours as the answer.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Before answering your question, be aware of two kinds of Chebyshev nodes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Roots of $T_{n+1}(x)$ are $\displaystyle{x_k = \cos \left( \frac{k+1/2}{n+1}\pi \right)}$ where k = 0, 1, ..., n&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extrema of $T_n(x)$ in [–1, 1] are $\displaystyle{x_k = \cos \left( \frac{k\pi}{n} \right)}$ where k = 0, 1, ..., n&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The former may be necessary when the endpoints are problematic, such as when integrating a function that goes to infinity.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;1. Which method is better?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The projection method can be made faster by using the fact that roots of $T_{n+1}(x)$ satisfy discrete orthogonality relations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum_{k=0}^n T_r(x_k) T_s(x_k) = \left\{ \begin{array}{ll}&#xA;0 &amp;amp; \text{if } \, r \neq s \\&#xA;(n+1)/2 &amp;amp; \text{if } \, r = s \neq 0 \\&#xA;n+1 &amp;amp; \text{if } \, r = s = 0&#xA;\end{array} \right. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be proven by considering&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align}&#xA;S_n(\theta) &amp;amp;= \sum_{k=0}^n \cos (k+1/2) \theta = \frac{1}{2} \csc \frac{\theta}{2} \sin (n+1) \theta \\&#xA;S_n(0) &amp;amp;= n+1 \\&#xA;S_n \left( \frac{k\pi}{n+1} \right) &amp;amp;= 0 \qquad \qquad \forall \, k = \pm 1, \dotsm, \pm (2n+1) \\&#xA;S_n(\pm 2\pi) &amp;amp;= -(n+1)&#xA;\end{align} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{align}&#xA;\sum_{k=0}^n T_r(x_k) T_s(x_k) &amp;amp;= \sum_{k=0}^n \cos \left( \frac{k+1/2}{n+1} r\pi \right) \cos \left( \frac{k+1/2}{n+1} s\pi \right) \\&#xA;&amp;amp;= \frac{1}{2} \sum_{k=0}^n \left[ \cos \left( \frac{k+1/2}{n+1} (r+s) \pi \right) + \cos \left( \frac{k+1/2}{n+1} (r-s) \pi \right) \right] \\&#xA;&amp;amp;= \frac{1}{2} \left[ S_n \left( \frac{r+s}{n+1} \pi \right) + S_n \left( \frac{r-s}{n+1} \pi \right) \right]&#xA;\end{align} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to answer your question directly, computing the coefficients $c_r$ as follows will be faster than solving a linear system as in the interpolation method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{n+1}{1 + [r\neq 0]} \, c_r = \sum_{k=0}^n f\left( \frac{b-a}{2}x_k + \frac{b+a}{2} \right) T_r(x_k) $$&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;2. Which method is implemented in NumPy and Chebfun?&lt;/h3&gt;&#xA;" OwnerUserId="22877" LastEditorUserId="22877" LastEditDate="2017-01-14T02:18:04.093" LastActivityDate="2017-01-14T02:18:04.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25943" PostTypeId="1" AcceptedAnswerId="41916" CreationDate="2017-01-14T09:29:47.223" Score="2" ViewCount="329" Body="&lt;p&gt;In advance I am sorry for my noobish question. I am a physics PHD student&#xA;and basically I use python for my math/physics problems. &#xA;But now I have a problem which requires more computing capacity and &#xA;I intend to use Fortran or C and supercomputer.&#xA;Those things are very new to me :). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my problem is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have a system of 17576 atoms and &lt;strong&gt;real symmetric&lt;/strong&gt; weighted hessian   m&#xA;atrix which describes interatomic interactions and has 52728x52728=2780241984 elements.&lt;/li&gt;&#xA;&lt;li&gt;This matrix is very &lt;strong&gt;sparse&lt;/strong&gt;: 1.56% of elements are nonzero&lt;/li&gt;&#xA;&lt;li&gt;The minimum and maximum of eigenvalues and density of states are known&lt;/li&gt;&#xA;&lt;li&gt;I need to diagonalise this matrix and find &lt;strong&gt;all eigenvalues and eigenvectors&lt;/strong&gt;  with high precision&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I heard that there is &lt;a href=&quot;http://www.ecs.umass.edu/~polizzi/feast/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;FEAST&quot;&gt;FEAST&lt;/a&gt; and &lt;a href=&quot;http://slepc.upv.es/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;SLEPc&quot;&gt;SLEPc&lt;/a&gt;. However SLEPc is not&#xA;suitable for whole eigenspectrum calculations. Is it ok to use FEAST&#xA;for this kind of problem or I should use something different?&#xA;In FEAST you can define search interval and in documentation it is written &#xA;that high accuracy can be obtained only for up to 1000 eigenpairs. &#xA;So my idea is to calculate whole eigenspectrum by splitting this problem &#xA;into multiple intervals with less than 1000 eigenvalues. &#xA;Or my intuition is wrong about how should I approach this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in summary my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it OK to use FEAST for this kind of problem or I should use something different?&#xA;Any other recommendations are very acceptable&lt;/li&gt;&#xA;&lt;li&gt;If FEAST is suitable for this problem, how it scales and how should I distribute resources?&lt;/li&gt;&#xA;&lt;li&gt;Also suggestions about what should I read or learn is very appreciated, &#xA;because I don't know from where to start and I don't have anyone to consult&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks for Your attention &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lukas&lt;/p&gt;&#xA;" OwnerUserId="21771" LastActivityDate="2022-09-24T15:23:15.173" Title="All eigenpairs of large sparse symmetric matrix" Tags="|linear-algebra|sparse-matrix|eigenvalues|eigensystem|mpi|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25945" PostTypeId="1" AcceptedAnswerId="25947" CreationDate="2017-01-15T13:05:04.397" Score="3" ViewCount="324" Body="&lt;p&gt;I am considering finite difference methods and their error analysis for solving HJB equation of the following form:&#xA;$$&#xA;v_t=|\sigma(x)v_x|,\quad x\in \mathbb{R},&#xA;$$&#xA;where $\sigma$ is a given function on $\mathbb{R}$ (with possible regularity assumptions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you give me a simple (perhaps first-order) FD scheme and its error analysis (stability and convergence in spatial discretization) for solving the above equation? I will also appreciate if you could give me some references(textbook/paper) on this subject. I just want to study the common techniques to study such numerical methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there are a lot of literature for solving HJB equations in quite general form. But I hope to start with this concrete example.&lt;/p&gt;&#xA;" OwnerUserId="5527" LastActivityDate="2017-01-15T19:23:49.050" Title="Stable method for solving a HJB equation" Tags="|pde|finite-difference|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25947" PostTypeId="2" ParentId="25945" CreationDate="2017-01-15T17:52:41.153" Score="4" Body="&lt;p&gt;One may prefer to write your equation in the form&#xA;$$&#xA;v_t = \sigma(x) |v_x| &#xA;$$&#xA;where $\sigma(x) \ge 0$ for $x \in R$. You need to define an initial condition e.g.&#xA;$$&#xA;v(x,0) = v^0(x) .&#xA;$$&#xA;Let me skip a treatment of boundary conditions (you hardly can solve the equation numerically on an unbounded domain). Note that the exact solution $v(x,t)$ for a fixed $x$ is non-decreasing function in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest (first order accurate) numerical scheme is based on Rouy-Tourin method. You aim to find the values $v_i^n$ that approximate the unknown exact values $v(x_i,t^n)$ for discrete points $x_i \in R$ and $t^n &amp;gt; 0$. Let me suppose uniform steps $h=x_{i+1}-x_i$ and $\tau=t^{n+1}-t^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, $v_i^0 = v^0(x_i)$. Furthermore, the scheme can be formally written as follows&#xA;$$&#xA;v_i^{n+1} = v_i^n + \tau \sigma(x_i) |\partial_x v_i^n|&#xA;$$&#xA;where a proper finite difference $\partial_x v_i^n$ (backward or forward one sided) shall be used to approximate $\partial_x v(x_i,t^n)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make the story short such appropriate approximation is based on the fact that the solution $v$ for a fixed $x$ is non-decreasing, therefore one has to choose one sided finite difference depending which of three values, $v_{i-1}^n$, $v_i^n$, $v_{i+1}^n$ is maximal. Namely&#xA;$$&#xA;\partial_x v_i^n = (v_{i+1}^n - v_i^n)/h \,\,\hbox{  if   } \,\, v_{i+1} \ge \max\{v_i^n,v_{i-1}^n \}&#xA;$$&#xA;or&#xA;$$&#xA;\partial_x v_i^n = (v_{i}^n - v_{i-1}^n)/h \,\,\hbox{  if   } \,\, v_{i-1} \ge \max\{v_i^n,v_{i+1}^n \}&#xA;$$&#xA;or&#xA;$$&#xA;\partial_x v_i^n = 0 \,\,\hbox{  if   } \,\, v_{i} \ge \max\{v_{i-1}^n,v_{i+1}^n \}&#xA;$$&#xA;There are other first order accurate methods, e.g. Osher-Sethian scheme, but this one is considered ``less diffusive''.&#xA;The stability of scheme can be proved if all local Courant numbers&#xA;$$&#xA;C_i = \tau \sigma(x_i) / h&#xA;$$&#xA;are less or equal to one. In fact in such case the discrete minimum and maximum principle is fulfilled for numerical solution (e.g. no unphysical oscillations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a reference I would recommend the book of Sethian or/and the book of Osher-Fedkiw, it is easy to find the exact references (P.S. see level set methods).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I was asked about stability in comments. In fact, as I mentioned above, one can prove even more. The scheme can be written equivalently as&#xA;$$&#xA;v_i^{n+1} = (1 - C_i) v_i^n + C_i \max \{ v_{i-1}^n, v_i^n, v_{i+1}^n\}&#xA;$$&#xA;so for $C_i \le 1$ the value $v_i^{n+1}$ is the convex combination of two values, consequently&#xA;$$&#xA;v_i^{n+1} \le \max \{v_{i-1}^n,v_i^n, v_{i+1}^n\} \,.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2017-01-15T19:23:49.050" LastActivityDate="2017-01-15T19:23:49.050" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25948" PostTypeId="1" CreationDate="2017-01-15T20:17:36.277" Score="2" ViewCount="99" Body="&lt;p&gt;Let $V_1,\ldots,V_n$ be $n$ vector subspaces of a Hilbert space, $y_i\in V_i$ for each $i$ and $\overline{x}$ be a fixed vector. I want to solve the optimization problem: &#xA;\begin{equation*}&#xA;\begin{aligned}&#xA;&amp;amp; \underset{x_1,\ldots,x_n}{\min}&#xA;&amp;amp; &amp;amp; \sum_{i=1}^n \| x_i - y_i\|_2^2 \\&#xA;&amp;amp; \text{subject to}&#xA;&amp;amp; &amp;amp; \frac{1}{n}\sum_{i=1}^n x_i = \overline{x},\\&#xA;&amp;amp; &amp;amp;&amp;amp; \ x_i \in V_i \text{ for }i=1,...n.&#xA;\end{aligned}&#xA;\end{equation*}&#xA;I don't have a basis for the $V_i$ a priori but I can use the projection operators $P_i$ as I please. The setting is supposed to be an infinite dimensional Hilbert space but any finite dimensional help is appreciated too. The $V_i$ are not necessarily orthogonal nor does their direct sum equal the whole space necessarily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the $V_i$ are orthogonal and direct sum into the whole space, I believe the solution is $x_i = y_i + N*P_i(\overline{x} - \overline{y})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feel free to add in any assumptions you might need. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asked on MSE originally, sorry if reposts are frowned upon.&lt;/p&gt;&#xA;" OwnerUserId="22944" LastActivityDate="2017-01-15T20:17:36.277" Title="A least square problem with a fixed mean constraint and a subspace constraint" Tags="|optimization|constrained-optimization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25950" PostTypeId="1" CreationDate="2017-01-16T03:58:17.890" Score="2" ViewCount="2811" Body="&lt;p&gt;I try to convolve a rectangle function in [-1/2, 1/2] with itself using fft. The convolution should be a tent shaped function, see figure below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TKiIX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TKiIX.png&quot; alt=&quot;rect wave and its convolution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is below. In the 3rd to last line I add /50 so it appears correct - I have no idea what the normalization factor should be. And my question is how to normalize the power of a fft result, so the subsequent application of ifft gives the correct result?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from numpy.fft import fft, ifft, fftshift, ifftshift&#xA;N = 1000&#xA;t=np.linspace(-10, 10, N)&#xA;dt = t[1]-t[0]&#xA;&#xA;data = 1.0*(np.abs(t)&amp;lt;0.5)&#xA;data_fft = np.fft.fft(data)&#xA;&#xA;plt.hold(1)&#xA;plt.plot(t, (ifft(data_fft**1)), 'r')&#xA;plt.plot(t, ifftshift(ifft(data_fft**2)) / 50.0, 'g')&#xA;plt.xlim(-4,4)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8203" LastActivityDate="2019-07-02T12:09:55.823" Title="Wrong amplitude of convolution using numpy fft" Tags="|python|numpy|fftw|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="25951" PostTypeId="1" CreationDate="2017-01-16T05:34:45.663" Score="1" ViewCount="53" Body="&lt;p&gt;I am considering solving HJB equation of the form&#xA;$$&#xA;v_t=g(a(x)v_x),\quad x\in \mathbb{R}, t&amp;gt;0,&#xA;$$&#xA;with initial condition $v(0)=v_0$. Here $g:\mathbb{R}\to \mathbb{R}$ is &lt;strong&gt;Lipschitz and monotone increasing&lt;/strong&gt;. I hope to construct a scheme stable in $L^\infty$ norm.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is what I tried. For the linear $g$, i.e. $g(p)=cp$ with possible space dependent $c&amp;gt;0$, I consider the upwind and downwinding scheme；&#xA;$$&#xA;D_iv^{n}=\begin{cases}\frac{v_{i+1}^{n}-v_{i}^{n}}{h},&amp;amp; a_i&amp;gt;0,\\&#xA;\frac{v_{i}^{n}-v_{i-1}^{n}}{h},&amp;amp; a_i&amp;lt;0 \end{cases},&#xA;$$&#xA;and update the solution by &#xA;$$v^{n+1}_i=v^{n}_i+\tau c_ia_iD_iv^n.$$&#xA;Then for $a_i&amp;gt;0$, we have&#xA;$$v^{n+1}_i=v^{n}_i+\frac{\tau c_ia_i}{h}(v_{i+1}^{n}-v_{i}^{n})=(1-C_i)v_i^n+C_iv^{n}_{i+1}\le |v^n|_\infty,$$&#xA;with $C_i=\frac{\tau c_ia_i}{h}&amp;gt;0$, while for $a_i&amp;lt;0$, we have &#xA;$$v^{n+1}_i=v^{n}_i+\frac{\tau c_ia_i}{h}(v_{i}^{n}-v_{i-1}^{n})=(1-C_i)v_i^n+C_iv^{n}_{i+1}\le |v^n|_\infty,$$&#xA;with $C_i=-\frac{\tau c_ia_i}{h}&amp;gt;0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for Lipschitz and monotone increasing $g$, the scheme becomes &#xA;$$v^{n+1}_i=v^{n}_i+\tau g(a_iD_iv^n).$$&#xA;To perform similar analysis, I write &#xA;$$g(a_iD_iv^n)=g(a_iD_iv^n)-g(0)+g(0),$$ thus&#xA;$$|v_i^{n+1}|=|v_i^n+\tau (g(a_iD_iv^n)-g(0))|+\tau|g(0)|,$$&#xA;then using Lipschitz $g$ I have&#xA;$$|v_i^{n+1}|=|v_i^n|+L\tau |a_iD_iv^n|+\tau|g(0)|.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the similar analysis can't process since I don't know how to write $|v^{n}_i|+C_i|v^{n}_{i+1}-v^{n}_{i}|$ as a convex combination. Also I haven't used the &lt;strong&gt;monotone increasing&lt;/strong&gt; assumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also tried the scheme discussed &lt;a href=&quot;https://scicomp.stackexchange.com/questions/25945/stable-method-for-solving-a-hjb-equation&quot;&gt;here&lt;/a&gt;, but I still can't demonstrate the stability due to a similar reason.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Is my simple upwinding scheme a stable scheme to solve this problem? I am not sure whether the proof fails due to my poor techniques or the scheme is not stable in general for this type of $g$. Is there any other scheme commonly used for this problem?&lt;/p&gt;&#xA;" OwnerUserId="5527" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-01-16T05:40:47.153" Title="A stable method for solving monontoe HJB equation" Tags="|finite-difference|numerical-analysis|fluid-dynamics|optimal-control|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25953" PostTypeId="1" AcceptedAnswerId="25956" CreationDate="2017-01-16T14:44:23.340" Score="4" ViewCount="1675" Body="&lt;p&gt;Say that N is the number of nodes, E the number of elements and S the number of sides in a triangular 2D mesh. Is there a relationship that links these quantities, possibly taking into account that some nodes are boundary nodes (Nb) and some are interior nodes (Ni)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer can be restricted to Delaunay triangulation.&lt;/p&gt;&#xA;" OwnerUserId="22948" LastActivityDate="2017-01-20T17:34:42.203" Title="Relationship between number of nodes, elements and sides in a triangular 2D mesh" Tags="|mesh-generation|delaunay-triangulation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25954" PostTypeId="1" AcceptedAnswerId="25963" CreationDate="2017-01-16T17:21:58.650" Score="0" ViewCount="849" Body="&lt;p&gt;I'm testing a Finite Elements code. To do this, i have created a simple one element quadrilateral mesh to verify the solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/McsGb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/McsGb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a plane stress elasticity problem, and the local stiffness is computed through the following weak formulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YFti7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YFti7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem data is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nk4lI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nk4lI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the resultant stiffness matrix and load vector:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JioYF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JioYF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/phGB8.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/phGB8.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After applying a penalty boundary conditions (multiply key boundary points in the matrix by a very large number) the final stiffness matrix is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xe9Wx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xe9Wx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and after solving the linear system above, the solution is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rebtL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rebtL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, i'm trying to solve the same problem in a rotated mesh (45 degrees counter clockwise), and i'm facing some problems to impose the dirichlet boundary conditions, i.e. to impose the null displacement in the specified nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the new nodes coordnates:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DN3hD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DN3hD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the new mesh is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zjQm8.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zjQm8.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the rotated stiffness matrix is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TVut2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TVut2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the rotated load vector is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/svkzJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/svkzJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution must be the same in both meshes, but i'm facing a problem to impose the new boundary conditions. How can i impose the natural boundary conditions in the rotated mesh?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="22951" LastEditorUserId="22951" LastEditDate="2017-01-17T01:21:40.007" LastActivityDate="2017-01-22T20:29:16.297" Title="Applying boundary conditions in a simple one element rotated mesh" Tags="|finite-element|boundary-conditions|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25955" PostTypeId="1" CreationDate="2017-01-16T19:55:58.233" Score="3" ViewCount="273" Body="&lt;p&gt;Suppose I have a mesh consisting of a set $M$ of conformal elements that fill the region $R=[0,1]\times[0,1]$. Suppose that I also have a 2D shape $S\subset R$  whose boundary $\partial S$ is piecewise linear, but which is not necessarily convex nor necessarily simply connected.  Without making any assumptions on the particular shape of the elements in the mesh, how can I efficiently determine the smallest set of elements whose union contains all of the shape $S$?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though my question is posed in 2D, I'm interested in an approach that would extend to 3D as well.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="20688" LastEditDate="2017-04-17T08:39:31.630" LastActivityDate="2018-06-11T17:12:37.003" Title="How do I find the smallest set of elements that covers a given shape?" Tags="|computational-geometry|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25956" PostTypeId="2" ParentId="25953" CreationDate="2017-01-16T21:03:43.717" Score="5" Body="&lt;p&gt;Yes there is a relationship, the Euler characteristic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 2-dimensional orientable manifold with boundaries embedded in $\mathbb{R}^3$, the Euler characteristic is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\chi = V - E + F = 2 - 2g - b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $V$ is the number of vertices, $E$ is the number of edges, $F$ is the number of faces, $g$ is the genus of the manifold, and $b$ is the number of borders of the manifold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a disk has no holes, and one border, so its Euler characteristic is $2 - 2(0) - 1 = 1$, thus we know that $V - E + F = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a torus, $g = 1$ and $b = 0$, thus $\chi = 2 - 2(1) - 0 = 0$, so $V - E + F = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point being that this number is independent of the exact triangulation, it only depends on the topology of the manifold.&lt;/p&gt;&#xA;" OwnerUserId="22189" LastEditorUserId="22189" LastEditDate="2017-01-20T17:34:42.203" LastActivityDate="2017-01-20T17:34:42.203" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25957" PostTypeId="2" ParentId="25955" CreationDate="2017-01-17T04:07:30.780" Score="0" Body="&lt;p&gt;Let's first also assume you have some way to validate that a point is within the shape $S$. Given you have that implemented, one could use the vertices of some given element to check if it is within the shape $S$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step would be to find an element that does lie within $S$, which could be done by finding an element that contains/shares one of the vertices used to create the piecewise shape $S$. This could be efficiently done using something like a KD Tree, Spatial Hashtable, or similar data structure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have found a single element that is within the shape $S$, you can then use a Breadth/Depth First Search based on neighboring elements and their vertices to find the elements within $S$.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-01-17T04:07:30.780" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25960" PostTypeId="1" CreationDate="2017-01-17T13:05:06.567" Score="1" ViewCount="254" Body="&lt;p&gt;I have a linear program of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\underset{P,\;g}{\text{Minimize}}\hspace{3mm}c^Tg$$&#xA;\begin{align}&#xA;\hspace{17mm}\text{Subject to}\hspace{3mm}AP_{\cdot,j}&amp;amp;=\begin{bmatrix}&#xA;-g\\ &#xA;d&#xA;\end{bmatrix}\\&#xA;P_{ii}&amp;amp;=0\\&#xA;|P_{\cdot,j}|&amp;amp;\leq P_{\max}\\&#xA;g&amp;amp;\leq G_{\max}\\&#xA;g&amp;amp;\geq 0&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with variables: $g\in{\bf R}^k$, $P\in{\bf R}^{n\times n}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and problem data: $c\in{\bf R}^k$, $A\in{\bf R}^{m\times n}$, $d\in{\bf R}^{m-k}$, $P_{\max}\in{\bf R}^n$, $G_{\max}\in{\bf R}^k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm only interested in $g$, and thus I was wondering if there was a way to somehow eliminate the variable $P$, thereby significantly reducing the dimension of the search space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ is invertible it's almost certainly doable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ has a left-inverse it might be doable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ has more columns than rows and full row-rank it might be doable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ has more columns than rows without full row-rank then I don't think it's doable, however this is the case I'm most interested in.&lt;/p&gt;&#xA;" OwnerUserId="6849" LastEditorUserId="6849" LastEditDate="2017-01-17T15:53:10.477" LastActivityDate="2017-01-17T15:53:10.477" Title="Variable elimination in linear programming" Tags="|optimization|convex-optimization|linear-programming|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25961" PostTypeId="2" ParentId="25114" CreationDate="2017-01-17T14:29:28.327" Score="0" Body="&lt;p&gt;I haven't found all answers to my questions, but in the meantime I found some answers to a few questions.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Why are we using delta in &lt;code&gt;A = p*G*D + delta&lt;/code&gt;?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;delta&lt;/code&gt; = $\delta = \frac{1 - p}{n}$, as defined in page 2 of &lt;a href=&quot;https://au.mathworks.com/content/dam/mathworks/mathworks-dot-com/moler/exm/chapters/pagerank.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;C. Moler's article&lt;/a&gt;, is the probability that a particular random page is chosen. Note that $1 - p$ is the probability that some arbitrary page is chosen and $p$ is the probability that a random walk from a user follows a specific link. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why would this be helpful to answer my question?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's helpful because we're defining A we're assuming that there are always outlinks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why is that the case?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same paper, $A$ is defined for the power method as $A = pGD + ez^T$, which is very similar to the formula for the &lt;em&gt;inverse iteration&lt;/em&gt;, but we use $ez^T$ instead of delta. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;z^T=&#xA;\begin{cases}&#xA;\delta = \frac{1 - p}{n}, \text{ if } c_j \neq 0 \\&#xA;\frac{1}{n}, \text{ if } c_j = 0&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as we can see from the definition of $z$, by using just $\delta$ in the inverse iteration method I assume that we're assuming that all nodes have an outgoing link or, equivalently, $c_j \neq 0$. Note that $n$ is the number of nodes in the graph.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Again in this chapter (page 4) about page rank (by Cleve Moler), the author says (after those statements above) that (I - A) is theoretically singular. Why is that? ...&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We want to solve the equation $Ax = x$. We know that our sought eigenvalue is $1$, and this is why we're omitting $\lambda$ in $Ax = \lambda x$. Given this, we can manipulate the equation as follows &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;Ax &amp;amp;= x \\&#xA;Ax &amp;amp;= Ix \\&#xA;Ax - Ix &amp;amp;= \vec{0} \\&#xA;(A - I)x &amp;amp;= \vec{0}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, from the beginning, we're actually assuming $x$ is &lt;strong&gt;not&lt;/strong&gt; the zero vector because it represents the &lt;em&gt;state vector&lt;/em&gt; or &lt;em&gt;PageRank vector&lt;/em&gt;. Hence, knowing that $x$ can't be $\vec{0}$, we conclude that $(A - I)$ has a &lt;strong&gt;non-trivial null-space (or kernel)&lt;/strong&gt; or, equivalently, $(A - I)$ is singular, or has determinant zero, or $(A - I)$ is not invertible.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... And how could we avoid it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's explained in page $4$ of the &lt;a href=&quot;https://au.mathworks.com/content/dam/mathworks/mathworks-dot-com/moler/exm/chapters/pagerank.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;previously mentioned article&lt;/a&gt;. Apparently, using Guassian elimination (i.e. &lt;a href=&quot;https://ch.mathworks.com/help/matlab/ref/mldivide.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;\&lt;/code&gt; operator&lt;/a&gt; in Matlab, which is a robust operation that takes into account many possibilities regarding the operands), roundoff errors seem to make $(A - I)$ not singular and the computation not fail. From the article:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Because $I − A$ is theoretically singular, with exact computation some diagonal element of the &lt;em&gt;upper triangular factor&lt;/em&gt; of $I − A$ should be &lt;em&gt;zero&lt;/em&gt; and this computation should fail. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;But &lt;strong&gt;with roundoff error&lt;/strong&gt;, the computed matrix $I - A$ is &lt;strong&gt;probably not exactly singular&lt;/strong&gt;. Even if it is singular, roundoff during Gaussian elimination will most likely prevent any exact zero diagonal elements. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;We know that Gaussian elimination &lt;em&gt;with partial pivoting&lt;/em&gt; always produces a solution with a small residual, relative to the computed solution, even if the matrix is &lt;em&gt;badly conditioned&lt;/em&gt;. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The vector obtained with the backslash operation, &lt;code&gt;(I - A)\e&lt;/code&gt;, usually has very large components. If it is rescaled by its sum, the residual is scaled by the same factor and becomes very small. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Consequently, the two vectors $x$ and $A*x$ equal each other to within roundoff error. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In this setting, solving the singular system with Gaussian elimination blows&#xA;  up, but it blows up in exactly the right direction.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Note&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Questions still not answered&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Main question&lt;/li&gt;&#xA;&lt;li&gt;Minor question number 2&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerDisplayName="user9925" LastEditorDisplayName="user9925" LastEditDate="2017-01-17T14:35:08.733" LastActivityDate="2017-01-17T14:35:08.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25963" PostTypeId="2" ParentId="25954" CreationDate="2017-01-17T15:10:11.723" Score="1" Body="&lt;p&gt;The large-number approach you are using to impose constraints is generally referred to as a&#xA;penalty method. One upside of penalty methods is that they are generally easy to apply. A downside&#xA;is that adding large numbers to the equations makes them ill-conditioned and can cause errors in the&#xA;solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to look at penalty methods is to write the total potential energy of your system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E_{TP} = \frac{1}{2}U^TKU - U^TF&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is the vector of nodal displacments, $K$ is the global stiffness matrix, and $F$ is the load vector. Constraints on the&#xA;values of $U$ can be prescribed by including an additional term in the potential energy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E_{TP} = \frac{1}{2}U^TKU - U^TF + \frac{r}{2}\sum_{i=1}^{n_c} g_i(U)^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the $g_i(U)$ are constaint equations that are functions of $U$, $r$ is a large number, and $n_c$ is the number of constraints.&#xA;The equations, $g_i$, equal zero when the constraint is satisfied.&#xA;In your un-rotated case, the $g_i$ are simply the components of $U_i$ you want to constrain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the rotated case, you want to impose constraints on the displacements in the rotated coordinate system. At a particular node $j$ these can&#xA;be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\{\begin{array}{c}u^\prime_j \\ v^\prime_j\end{array}\right\} = &#xA;\left[\begin{array}{cc}cos\theta &amp;amp; sin\theta \\ -sin\theta &amp;amp; cos\theta\end{array}\right]&#xA;\left\{\begin{array}{c}u_j \\ v_j\end{array}\right\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a typical $g_i$ might be $cos\theta u_j + sin\theta v_j$ if you wanted to constrain $u^\prime_j$ to zero.&#xA;By minimizing the potential energy with respect to the $U$ components, just like you&#xA;did to find the stiffness matrix and load vector, there will also be a set of $2 \times 2$ constraint matrices, one for each displacement constraint&#xA;you want to apply. These are added to the global stiffness matrix just like the large diagonal term you added in the un-rotated problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find more information on applying constraints in FEM in these sets of notes from Carlos Felippa&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch08.d/IFEM.Ch08.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MultiFreedom Constraints I&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch09.d/IFEM.Ch09.index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MultiFreedom Constraints II&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He refers to these types of constraints as &quot;multifreedom&quot; because they are a linear combination of more than a single degree-of-freedom in the FE model. He also discusses the alternate, Lagrange multiplier approach which is a better approach, numerically, to apply these types of constraints.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-01-18T17:19:10.500" LastActivityDate="2017-01-18T17:19:10.500" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25964" PostTypeId="1" CreationDate="2017-01-17T16:35:03.983" Score="1" ViewCount="1070" Body="&lt;p&gt;I have a pretty naive question, though important to me.&#xA;Usually when solving the following PDE in solute transport:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{{\partial C}}{\partial t } = \nabla. (D\nabla C -vC )=0,$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;one can be asked to use a third-type boundary condition (called flux in software like Comsol or Phreeqc):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(D\nabla C -vC )=vCo,$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the left-hand side is assumed to be evaluated at the center of the cell, and the right-hand side represents the inflow or outflow flux (Am I right about this last sentence?)&lt;/p&gt;&#xA;&lt;p&gt;In some software (Comsol) you do not have to specify Co, therefore I assume that in such cases the boundary condition is reduced to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$D\nabla C = 0.$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is that true? Can such boundary condition be considered constant flux boundary conditon?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="22125" LastEditorUserId="23747" LastEditDate="2020-06-30T18:04:51.767" LastActivityDate="2020-06-30T18:04:51.767" Title="Flux boundary condition in solute transport" Tags="|boundary-conditions|comsol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="25965" PostTypeId="2" ParentId="25964" CreationDate="2017-01-17T22:20:18.353" Score="2" Body="&lt;p&gt;The correct boundary condition is in fact &#xA;$$&#xA;  (D\nabla C - vC) \cdot n = g.&#xA;$$&#xA;That is, it is not the vector-valued quantity in parentheses that you can describe, but only the &lt;em&gt;normal&lt;/em&gt; component, where $n$ is the normal vector to the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term in the parentheses is called the &lt;em&gt;flux&lt;/em&gt;. It describes the amount of material (solute) that moves around, and the entire left side of the equation above is the amount of material that flows &lt;em&gt;across the boundary&lt;/em&gt;. Specifically, if you at an &lt;em&gt;inflow&lt;/em&gt; boundary condition, and the concentration of your solute on the &lt;em&gt;outside&lt;/em&gt; of the domain is $C_0$, then $g=(vC_0)\cdot n$ is the correct description.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, in Comsol, you can't specify a right hand side $g=vC_0$, then I my interpretation would be that that implies that $g=0$, which would mean&#xA;$$&#xA;  (D\nabla C - vC) \cdot n = 0.&#xA;$$&#xA;In other words, no material flows across the boundary.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-01-17T22:20:18.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25966" PostTypeId="1" AcceptedAnswerId="25981" CreationDate="2017-01-18T01:39:29.587" Score="2" ViewCount="212" Body="&lt;p&gt;I am attempting to model the following SDOF system with a variable spring and having a sinusoidal input,&#xA;$$m\ddot{x}+(k_0+k_1)x = m(2\pi f)^2\sin(2\pi ft)$$&#xA;where $m$ is the mass, $k_0$ the original spring and $k_1 = k_0$ for $x\dot{x}&amp;gt;0$ and $k_1 = 0$ otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was able to code this in Matlab, but for low frequencies, say $f=0.05$, a chattering effect would happen in the plot of $x\dot{x}$ vs. $t$. The following is my code and plots of the result for low frequencies. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = 0.05;&#xA;t_f = 10;&#xA;tspan = [0 t_f];&#xA;z0 = [0 ; 0];&#xA;&#xA;options = odeset( 'RelTol', 1e-13, 'AbsTol', 1e-13, 'Stats','on' );&#xA;&#xA;&#xA;[t, z] = ode23s(@(t,z) sys_13(t, z, f), tspan, z0, options);&#xA;&#xA;figure(1)&#xA;set(gcf,'units','normalized','outerposition',[0 0 1 1])&#xA;&#xA;subplot(2,1,1);&#xA;plot(t,z(:,1),'b');&#xA;title(['Time history of $y(t)\,\,(\ddot{u}_{gx}=-(2\pi f)^2\sin(2\pi f t)) \,\, f=\, $' num2str(f) ' $ $'],'interpreter','latex','fontsize',20)&#xA;ylabel('$y$','interpreter','latex','fontsize',15)&#xA;xlabel('$t$','interpreter','latex','fontsize',15)&#xA;xlim(tspan)&#xA;&#xA;grid on&#xA;hold on&#xA;&#xA;subplot(2,1,2);&#xA;plot(t,z(:,2),'b')&#xA;ylabel('$\dot{y}$','interpreter','latex','fontsize',15)&#xA;xlabel('$t$','interpreter','latex','fontsize',15)&#xA;xlim(tspan)&#xA;&#xA;grid on&#xA;hold on&#xA;&#xA;figure(2)&#xA;plot(t,z(:,1).*z(:,2),'b')&#xA;&#xA;grid on&#xA;grid minor&#xA;hold on&#xA;&#xA;k0 = 4;&#xA;k1 = 4;&#xA;ktot = k0+k1&#xA;k = calcKSmooth(z(:,1).*z(:,2), k0, ktot )*1e-5;&#xA;&#xA;plot(t,k)&#xA;&#xA;&#xA;function dz = sys_13(t, z , f)&#xA;dz = zeros(2,1);&#xA;&#xA;m = 1;&#xA;k0 = 4;&#xA;k1 = 4&#xA;% tol = 1e-6;&#xA;% if abs(z(1)*z(2)) &amp;gt; tol &amp;amp;&amp;amp; z(1)*z(2) &amp;gt; 0&#xA;%     k1 = 4;&#xA;% else&#xA;%     k1 = 0;&#xA;% end&#xA;&#xA;% if z(1)*z(2) &amp;gt; 0&#xA;%    k1 = 4;&#xA;% else&#xA;%    k1 = 0;&#xA;% end&#xA;ktot = k0 + k1;&#xA;k = calcKSmooth(z(1)*z(2) , k0 , ktot);&#xA;&#xA;finput = ( -(2*pi*f)^2 )*sin( 2*pi*f*t );&#xA;&#xA;dz(1) = z(2);&#xA;% dz(2) = - finput - ((k0 + k1)/m)*z(1);&#xA;dz(2) = - finput - (( k )/m)*z(1);&#xA;end&#xA;&#xA;function k = calcKSmooth(xxdot, k0, ktot)&#xA;  k1 = k0; k2 = ktot; c = 0;&#xA;  r = 1e6;&#xA;  ecr = exp(c*r);&#xA;  erx = exp(r*xxdot);&#xA;  k = (k1*ecr + k2*erx)./(ecr+erx);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The plot of the time history of the position and velocity are&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zDhvP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zDhvP.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plot of $x\dot{x}$ vs. $t$ is&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0hofi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0hofi.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I zoom in close to where the plot is zero I see the following behavior&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DB0jE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DB0jE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried applying a tolerance so to avoid the constantly switching on and off and instead the chattering occurs about the tolerance I set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'm not sure what is causing these issues, but my best guess would be that the discontinuity in the state-space matrix&#xA;$$ A=\begin{bmatrix}0&amp;amp;&amp;amp;1\\-\frac{k_0+k_1}{m}&amp;amp;&amp;amp;0\end{bmatrix}$$&#xA;when $k_1$ switches from $0$ to $k_0$ or vice versa is causing some sort of numerical errors in the solver. I was thinking about using an event location to find when $x\dot{x}$ passes through zero and start/stop the solver to avoid the discontinuity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help or advice!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Using the answer provided by Bill Greene, the chattering was removed, but the way stiffness is added to the system is gradual, yet physically the switching off or on should be more sudden, much like a step function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plot of $x\dot{x}$ and with $k=k_0+k_1$ superimposed is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JnA5K.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JnA5K.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22543" LastEditorUserId="22543" LastEditDate="2017-01-22T11:36:18.597" LastActivityDate="2017-01-22T11:36:18.597" Title="Chattering effect using ode23s -- SDOF with variable spring and periodic input force" Tags="|matlab|numerical-modelling|modeling|numerical-limitations|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25967" PostTypeId="2" ParentId="25114" CreationDate="2017-01-18T03:02:31.753" Score="2" Body="&lt;p&gt;The MATLAB code that you've computed finds the eigenvector of $A$ associated with the eigenvalue 1.  We know for this particular problem that $A$ has 1 as an eigenvalue- this can be shown using the Perron-Frobenius Theorem.  This code isn't using the inverse iteration algorithm.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inverse iteration algorithm can be used to find an eigenvalue and corresponding eigenvector of $A$ starting with a value of $\alpha$ that is close to an eigenvalue.  The algorithm could be used starting with $\alpha=1$ to compute the eigenvector of $A$ associated with the eigenvalue 1.  However, this is overkill since we already know that the eigenvalue is exactly 1.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can easily implement the inverse iteration and start it with $\alpha=1$ to verify this.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-01-18T03:02:31.753" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25968" PostTypeId="1" CreationDate="2017-01-18T12:02:20.427" Score="1" ViewCount="1585" Body="&lt;p&gt;I have to solve the following equation for $x(i), 0 \leq i \leq 1$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ y(i) = x(i)^{-a} \int_0^1 y(j)x(j) dj \left(\int_0^1 \mathcal A(j) x(j)^{1-a} dj\right)^{-1} \int_i^1 \left( \int_0^x x(j)^{1-a} dj \right)^\frac{a+b}{a-1} A_i^{b - 1} dx $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_i$ and $y(i)$ are known and I use the short cut &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathcal A(j) = \int_j^1 A_z^{1 - b} dz$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess there is no point in trying to solve it symbolically with Sympy. How would I go on solving this numerically using Python? Do I discretize $x(i)$ into 100 or 1000 points and hope to brute force find a solution to the system of equations? Or is there a more clever approach?&lt;/p&gt;&#xA;" OwnerUserId="9694" LastActivityDate="2017-01-18T12:02:20.427" Title="Solving an integral equation in Python" Tags="|python|integral-equations|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="25969" PostTypeId="1" CreationDate="2017-01-18T16:10:52.877" Score="4" ViewCount="1346" Body="&lt;p&gt;I'm implementing a Finite Difference WENO5 with Lax-Friedrich flux splitting on a uniform, structured grid to solve the 2D Euler equations of fluid dynamics on a rectangular domain in cartesian coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial U}{\partial t}+\frac{\partial F}{\partial x}+\frac{\partial G}{\partial y}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U = (\rho,\rho{u},\rho{v},\rho E)^t$$ &#xA;$$F = (\rho{u},\rho{u^2}+p,\rho u v, \rho uH)^t$$&#xA;$$G = (\rho{v},\rho u v,\rho{v^2}+p, \rho vH)^t$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E = \frac{p }{\rho (\gamma-1)} + \frac{1}{2}(u^2+v^2)$$&#xA;$$H = \frac{\gamma p }{\rho (\gamma-1)} + \frac{1}{2}(u^2+v^2)$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Although the interior scheme seems to work fine, I am having trouble with the numeric implementation of boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The WENO5 interpolation is done on the fluxes, to approximate the derivative on the node $i$ as: $$\left( \frac{\partial F}{\partial x}\right) _{i}=\frac{f^{WENO}_{i+1/2}-f^{WENO}_{i-1/2}}{\Delta x}$$ and then update the conservative variables' vector via a Runge-Kutta method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use ghost nodes, and I know there has to be 3 ghost nodes on each end of the domain (because of the stencil of the WENO5) but I do not know what values I have to assign to the ghost nodes, or what kind of extrapolation I have to use to find those values. In my case the physical boundaries coincide with the leftmost and rightmost (lowermost and uppermost for the other direction) points of the grid, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the three types of boundary conditions I'm working with at the moment, namely:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Reflecting wall (x-direction): $$\begin{bmatrix}\frac{\partial \rho }{\partial x}=0 \\ u=0\\ \frac{\partial v }{\partial x}=0 \\ \frac{\partial p }{\partial x}=0 \end{bmatrix} $$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reflecting wall (y-direction): $$\begin{bmatrix}\frac{\partial \rho }{\partial y}=0 \\ \frac{\partial u }{\partial y}=0 \\ v=0\\ \frac{\partial p }{\partial y}=0 \end{bmatrix} $$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Dirichlet condition with fixed value on all primitive variables: $$\begin{bmatrix} \rho =\rho_0 \\ u =u_0 \\ v =v_0 \\ p =p_0 \end{bmatrix} $$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for your answers.&lt;/p&gt;&#xA;" OwnerUserId="22962" LastActivityDate="2017-01-18T16:10:52.877" Title="How to implement boundary conditions on Finite Difference WENO5 scheme for the Euler equations" Tags="|finite-difference|fluid-dynamics|boundary-conditions|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25970" PostTypeId="1" CreationDate="2017-01-18T23:05:05.410" Score="2" ViewCount="2522" Body="&lt;p&gt;I'm using Python scipy.integrate.ode, and I want to stop my integration at a certain condition. So I use the integrator &quot;dopri5&quot; and use the method &quot;set_solout&quot; to specify a function for my stop condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, in some cases, the program says my problem is stiff, and exits. (The message is &quot;UserWarning: dopri5: problem is probably stiff (interrupted)&quot;). So I try another integrator. The documentation mentions 3 other integrators (&quot;vode&quot;, &quot;zvode&quot; and &quot;lsoda&quot;) which might be suitable for stiff problems. (There is also &quot;dop853&quot;, but that exits in the same way as &quot;dopri5&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my problem: none of these integrators support the &quot;set_selout&quot; method. So: is there another way to stop any of these integrators on a condition?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT (a few hours later): I got around this with repeated calls of the &quot;integrate&quot; method (with integrator set to &quot;lsoda&quot;) with a short time step; and checking my end condition after each call. Ugly, but gets the job done. I'd still like to know if there's a better way to do this.&lt;/p&gt;&#xA;" OwnerUserId="22965" LastEditorUserId="22965" LastEditDate="2017-01-19T04:38:54.710" LastActivityDate="2019-07-10T10:50:47.900" Title="stop condition in scipy.integrate.ode for stiff system" Tags="|python|scipy|stiffness|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25972" PostTypeId="1" CreationDate="2017-01-19T08:21:42.910" Score="5" ViewCount="569" Body="&lt;p&gt;I'm wondering what alternatives there are to a grid convergence study to judge solution accuracy for a given grid resolution when doing steady-state RANS simulations on an automatically generated complex unstructured co-located grid (using OpenFOAM). Ideally, such a criterion would lend itself to automatic grid refinement.&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="549" LastEditDate="2017-01-19T14:39:52.210" LastActivityDate="2020-12-20T02:03:45.440" Title="Finite volume a posteriori error estimation" Tags="|fluid-dynamics|openfoam|unstructured-mesh|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25974" PostTypeId="1" CreationDate="2017-01-20T02:34:11.403" Score="0" ViewCount="175" Body="&lt;p&gt;Given a feasibility problem with both inequality and equality constraints, I'm interested in the sensitivity of the bounds of the region to changes in the constraints. To help with answering the rather general question, I'm interested in particular for linear equality constraints with perturbations for&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Linear&lt;/li&gt;&#xA;&lt;li&gt;Non-linear&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;inequality constraints. As a sample problem, consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align} &#xA;min \qquad 1 \\&#xA;\text{subject to} \hspace{1ex} b-\epsilon \leq \hspace{1ex}&amp;amp;a^{T}x \leq b + \epsilon \\&#xA;\mathbf{1}^{T}x &amp;amp;= 1&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do the limits of each co-ordinate in the feasible region $x_{i}$ vary as a function of $\epsilon$? Is there a common name for this problem?&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="15120" LastEditDate="2017-01-20T09:12:43.717" LastActivityDate="2017-01-20T09:12:43.717" Title="Perturbation in bounds given the perturbation to constraints" Tags="|convex-optimization|constrained-optimization|linear-programming|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25976" PostTypeId="1" CreationDate="2017-01-19T21:50:46.457" Score="1" ViewCount="1729" Body="&lt;p&gt;I am trying to solve the equation below numerically for various functions $I(x)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial^2}{\partial x^2} G(x) + I(x)G(x) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subject to the boundary contitions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial}{\partial x} G(x)\Big|_{x=0} = C_1 $$&#xA;$$ \frac{\partial}{\partial x} G(x)\Big|_{x=L} = C_2 $$&#xA;$$ \frac{\partial}{\partial x} G(x)\Big|_{x=x_0+\epsilon} - \frac{\partial}{\partial x} G(x)\Big|_{x=x_0-\epsilon} = 1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $x_0 \in [0,L]$, $\epsilon\to 0$, and $C_1$ and $C_2$ are given.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, if there were no boundary conditions, I would discretize $[0,L]$ into $N$ individual units, then I would approximate $\frac{\partial^2}{\partial x^2}$ using the finite difference method yielding a $N\times N$ matrix. I would then add the discretized version of $I(x)$ as a matrix by multiplying by the identity $N\times N$ matrix, yielding:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;\left(&#xA;\begin{bmatrix}&#xA;    -1       &amp;amp; 1 &amp;amp; 0 &amp;amp; \dots &amp;amp; 0 &amp;amp; 0 \\&#xA;    1       &amp;amp; -2 &amp;amp; 1 &amp;amp; \dots &amp;amp; 0 &amp;amp; 0 \\&#xA;    \vdots &amp;amp; \vdots &amp;amp; \vdots  \\&#xA;    0       &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots &amp;amp; -2 &amp;amp; 1 \\&#xA;    0       &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots &amp;amp; 1 &amp;amp; -1&#xA;\end{bmatrix}&#xA;+&#xA;\mathbb{1} \cdot&#xA;\begin{bmatrix}&#xA;I(x_1) \\&#xA;I(x_2) \\&#xA;\vdots \\&#xA;I(x_N) \\&#xA;\end{bmatrix}&#xA;\right)&#xA;\begin{bmatrix}&#xA;G_0 \\&#xA;G_1 \\&#xA;\vdots \\&#xA;G_{N-1} \\&#xA;G_N \\&#xA;\end{bmatrix}&#xA;=&#xA;0&#xA;$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then finding $G$ is a matter of inverting the matrix on the left hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question&lt;/strong&gt;: how do I incorporate the boundary conditions into he expression above? Should the derivative matrix (first matrix) change, or should the right hand side become a vector other than $0$? Is there a good reference for learning about how to numerically solve PDEs with weird boundary conditions? &lt;/p&gt;&#xA;" OwnerUserId="22988" OwnerDisplayName="alexvas" LastEditorUserId="276" LastEditDate="2017-10-10T00:36:05.257" LastActivityDate="2017-10-10T00:36:05.257" Title="Discretization of Laplacian with boundary conditions" Tags="|finite-difference|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25977" PostTypeId="2" ParentId="25976" CreationDate="2017-01-20T01:16:49.510" Score="2" Body="&lt;p&gt;You are correct that you should augment the matrix to satisfy the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have the book infront of me, but I recall that in the first chapter of the book &lt;em&gt;Fundamental Algorithms in Computational Fluid Dynamics&lt;/em&gt; by Thomas H. Pullman and David W. Zingg they discuss in detail the matrix formulation of finite difference methods. In particular they discuss how different boundary conditions change the form of the matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience I have always used periodic boundary conditions. In this case, when you write down the difference equations at the left boundary of domain, you will get a difference approximation of your differential operator of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{N-1}-2u_{i}+u_{i+1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a similar form can be found for the right boundary of the domain. What this does it it changes the form of your matrix slightly so that it  becomes somewhat like the matrix below up to some constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YGA4W.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YGA4W.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="arie64" LastActivityDate="2017-01-20T01:16:49.510" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25978" PostTypeId="2" ParentId="25974" CreationDate="2017-01-20T06:10:29.390" Score="2" Body="&lt;p&gt;I assume that you're interested in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b-\epsilon \leq a^{T}x \leq b +\epsilon$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;rather than&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b+\epsilon \leq a^{T}x \leq b +\epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;right?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in more complicated parameterizations of the constraints, what would those look like?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this linear case, for any fixed $\epsilon$ (including 0), you can minimize (or maximize) $x_{i}$ subject to the constraints.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a^{T}x \geq b-\epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a^{T}x \leq b+\epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$1^{T}x=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a linear programming problem.  Once you have an optimal solution, you can find the sensitivity of the optimal value to changes in the right-hand sides of the constraints from the dual solution.  This is called &quot;sensitivity analysis&quot; in linear programming.  You can also use &quot;parametric linear programming&quot; techniques to find the optimal value as a function of $\epsilon$.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parametric linear programming techniques are discussed in many textbooks on linear programming.  I believe for example that you'll find it in Vanderbei's text (I'm not in the office at the moment so I can't immediately verify this.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convex nonlinear problems you can also use the Lagrangian dual to perform sensitivity analysis.  There are also parametric nonlinear programming techniques that have been discussed in a few papers, but this isn't very well developed.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-01-20T06:10:29.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25979" PostTypeId="2" ParentId="25970" CreationDate="2017-01-20T11:54:06.390" Score="2" Body="&lt;p&gt;I don't know of a better way to handle this with SciPy since I don't think it has event handling. But if you're willing to venture beyond SciPy, the following software have the capability, either documented as event handling or rootfinding:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sundials' CVODE&lt;/li&gt;&#xA;&lt;li&gt;MATLAB's &lt;code&gt;ode23&lt;/code&gt; and &lt;code&gt;ode15s&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Julia's DifferentialEquations.jl Sundials wrapper and Rosenbrock methods&lt;/li&gt;&#xA;&lt;li&gt;Mathematica's NDSolve&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-01-20T11:54:06.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25980" PostTypeId="1" CreationDate="2017-01-20T13:51:08.303" Score="5" ViewCount="442" Body="&lt;p&gt;I would like to know which is the best way to integrate numerically Legendre polynomials. I am building up a Discontinuous Galerkin CFD code for which Legendre polynomials are used as basis functions in the spectral projection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In previous codes, I was used to employ Gauss-Legendre numerical quadrature, where only $k$ points allowed to obtain the exact integral of a polynomial of degree $2k−1$. Now, as such quadrature points are the roots of Legendre polynomials, I need use $k+1$ points to integrate the $k$-th Legendre polynomial (otherwise the polynomials are evaluated at their roots and all higher order ingegrals are zero). Is there any other efficient way to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for your help.&lt;/p&gt;&#xA;" OwnerUserId="22984" LastActivityDate="2017-01-20T19:50:29.033" Title="Numerical quadrature in Discontinuous Galerkin" Tags="|quadrature|hyperbolic-pde|discontinuous-galerkin|integration|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25981" PostTypeId="2" ParentId="25966" CreationDate="2017-01-20T15:33:14.477" Score="2" Body="&lt;p&gt;This a followup to my comment about smoothing the expression for $k$ that is a function of $x$ and $\dot x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I converted your expression for $k$ into the following function to make experimentation easier. The input variable &lt;code&gt;xxdot&lt;/code&gt; is just $x\dot x$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function k=calcK(xxdot)&#xA;  k = 4;&#xA;  if(xxdot&amp;gt;0)&#xA;   k = k + 4;&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I wrote a second version of this function that closely approximates the function above but is continuous and differentiable.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function k=calcKSmooth(xxdot)&#xA;  k1 = 4; k2 = 8; c = 0;&#xA;  r = 1e6;&#xA;  ecr = exp(c*r);&#xA;  erx = exp(r*xxdot);&#xA;  k = (k1*ecr + k2*erx)./(ecr+erx);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you plot these two functions in the vicinity of $x\dot x=0$ they are almost indistinguishable. Using &lt;code&gt;calcKSmooth&lt;/code&gt; in the ode calculation results in a solution that is very close to the one with &lt;code&gt;calcK&lt;/code&gt; but doesn't exhibit the chattering. In addition, the execution time is much less because the algorithm doesn't need to cut the step size so drastically to maintain accuracy.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-01-20T15:33:14.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25982" PostTypeId="2" ParentId="25980" CreationDate="2017-01-20T19:50:29.033" Score="3" Body="&lt;p&gt;Which basis you use for your finite element space does not matter for quadrature in general. If you use, for example, polynomials of degree $k$ (whether the Lagrange basis, or the Legendre basis, or monomials, or anything else) then the integrand of the mass matrix has polynomial degree $2k$, and you need a Gauss quadrature with $k+1$ points to integrate that correctly. That is because such Gauss quadratures can integrate &lt;em&gt;any&lt;/em&gt; polynomial of degree $2k+1$ exactly, regardless of how you actually &lt;em&gt;represent&lt;/em&gt; that polynomial in terms of bases.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-01-20T19:50:29.033" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25983" PostTypeId="2" ParentId="21483" CreationDate="2017-01-21T15:45:26.960" Score="-4" Body="&lt;p&gt;Rounding errors is due to finite representation of real numbers in the computer.&#xA;Computer systems can not work with real numbers accurate, but only with rational approximations&#xA;thereof. Consequently, the actual numbers can not be represented in the computer than with&#xA;a finite number of significant figures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In what follows, it is useful to represent real numbers in base 10, with a&#xA;f fractional part and an exponent n.&#xA;x = f • 10n. (2.8)&#xA;Moreover, for any number outside of 0 through convenient choice of n, the&#xA;Fractional satisfy 0.1 ≤ | f | &amp;lt;1.&#xA;For example 3.14 = 0,314 * 10 ^ 1, -0.007856 =&#xA;• 10 ^ 2 -0.7856. In terms of the convention forms, figures are called fractional part&#xA;significant numbers. &lt;/p&gt;&#xA;" OwnerUserId="22978" LastActivityDate="2017-01-21T15:45:26.960" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25986" PostTypeId="2" ParentId="16051" CreationDate="2017-01-22T15:13:56.973" Score="1" Body="&lt;p&gt;To answer your question, the most popular right now is phonopy. If you are having a technical problem you can post it on GitHub, usage issues are better suited for the mailing list. My opinion is that, if possible, using phonopy and getting it working would probably be best for the community if it is a technical problem in the code or best for you if it's just a usage issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you must use another software, &lt;a href=&quot;http://dx.doi.org/10.1016/j.cpc.2014.06.023&quot; rel=&quot;nofollow noreferrer&quot;&gt;YPHON&lt;/a&gt; was developed in my research group and the same formalism is used by phonopy and is cited as such. &lt;/p&gt;&#xA;" OwnerUserId="23005" LastActivityDate="2017-01-22T15:13:56.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25987" PostTypeId="2" ParentId="25954" CreationDate="2017-01-22T19:18:09.617" Score="0" Body="&lt;p&gt;if u impose a boundary condition u1=v1=0, v2=0, u4=0, the single rectangular mesh shouldn't rotate at all because node one is locked into place, node two can stretch horizontally while node 4 can stretch vertically leaving node three free to mode.  applying a stretching force of 6000lbs in the horizontal direction at node two and at node three will deform the elements from a rectangular shape to some unknown shape but not rotate the rectangular mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when the rectangular mesh is rotated, the u and v boundary conditions have changed even though you have rotated the input forces. the stiffness matrix should remain the same only the boundary conditions have changed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[K]=[J]*integral [B]^t[D][B]dV where [F]=[K][x].  rotating [x] whould change the integral by a factor equals to the jacobian matrix but not the stiffness matrix.  [F]=[K][x], [e]=[Q][u], [u]=[N][d], [e]=[B][d],where Q is the differential operator.&lt;/p&gt;&#xA;" OwnerUserId="23008" LastEditorUserId="23011" LastEditDate="2017-01-22T20:29:16.297" LastActivityDate="2017-01-22T20:29:16.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25989" PostTypeId="1" CreationDate="2017-01-22T20:46:43.133" Score="0" ViewCount="60" Body="&lt;p&gt;I am using this matrix class &lt;a href=&quot;https://www.quantstart.com/articles/Matrix-Classes-in-C-The-Source-File&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to assist me in converting this Least Squares algorithm that prices American options that is written in Matlab in C++. My question is pretty simple. I want to pass in a vector into a function but it has to be with this matrix class. For some reason, I get an error stating:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  cannot convert 'QSMatrix&amp;lt;double&amp;gt;' to 'double' for argument '2' to &#xA;'QSMatrix&amp;lt;double&amp;gt; randn_vec2(int, double)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My code has many comments in it so it should be rather easy to follow. I am using the matrix class linked in the above so I will just post my main file for sake of completion. Any comments or suggestions are greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; #include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;limits&amp;gt;&#xA;#include &amp;lt;algorithm&amp;gt;&#xA;#include &quot;matrix.h&quot;&#xA;&#xA;using namespace std;&#xA;&#xA;&#xA;double LaguerreExplicit(int R, double x); // Generates the (weighted) laguerre value&#xA;double payoff_Call(double S, double K); // Pay off of a call option&#xA;double LSM(int T, double r, double sigma , double K, double S0, int N, int M, int R);&#xA;double generateGaussianNoise(double mu, double sigma); // Generates Normally distributed random numbers&#xA;// Notation:&#xA;// T        expiration time&#xA;// r        riskless interest rate&#xA;// sigma    volatility&#xA;// K        strike price&#xA;// S0       initial asset price&#xA;// N        number of time steps&#xA;// M        number of paths&#xA;// R        number of basis functions&#xA;QSMatrix&amp;lt;double&amp;gt; randn_vec(int n, int M);&#xA;QSMatrix&amp;lt;double&amp;gt; range(double min, double max, double N); // Creates a vector of evenly spaced numbers&#xA;QSMatrix&amp;lt;double&amp;gt; rand_vec(int n, int k); // k = 1 to make this into a vector&#xA;QSMatrix&amp;lt;double&amp;gt; randn_vec2(int n, int M); // this is to replace the compand [z;z] in Matlab&#xA;&#xA;int main(){&#xA;    int M = 4;&#xA;    QSMatrix&amp;lt;double&amp;gt; z = randn_vec2(M/2,1);&#xA;    z.print();&#xA;&#xA;    QSMatrix&amp;lt;double&amp;gt; B = range(0,1,6);&#xA;    //B.print();&#xA;&#xA;    LSM(1, .06, .25 , 10.5, 12.5, 2, 4, 4);&#xA;&#xA;&#xA;&#xA;      return 0;&#xA;}&#xA;&#xA;&#xA;double payoff_Call(double S, double K){&#xA;    double payoff;&#xA;    if((S - K) &amp;gt; 0)&#xA;    {&#xA;        payoff = S - K;&#xA;    }else&#xA;    {&#xA;        payoff = 0.0;&#xA;    }&#xA;    return payoff;&#xA;}&#xA;&#xA;double LaguerreExplicit(int R, double x){&#xA;    double value;&#xA;    if(R==0)&#xA;    {&#xA;        value = 1;&#xA;    }&#xA;    else if(R==1)&#xA;    {&#xA;        value = 0.5*(pow(x,2) - 4.0*x + 2);&#xA;    }&#xA;    else if(R==3)&#xA;    {&#xA;        value = (1.0/6.0)*(-1*pow(x,3) + 9*pow(x,2) - 18*x + 6);&#xA;    }&#xA;    else if(R==4)&#xA;    {&#xA;        value = (1.0/24.0)*(pow(x,4) - 16*pow(x,3) + 72*pow(x,2) - 96*x + 24);&#xA;    }&#xA;    else if(R==5)&#xA;    {&#xA;        value = (1.0/120.0)*(-1*pow(x,5) + 25*pow(x,4) - 200*pow(x,3) + 600*pow(x,2) - 600*x + 120);&#xA;    }&#xA;    else if (R==6)&#xA;    {&#xA;        value = (1.0/720.0)*(pow(x,6) - 36*pow(x,5) + 450*pow(x,4) - 2400*pow(x,3) + 5400*pow(x,2) - 4320*x + 720);&#xA;    }&#xA;    else{&#xA;        cout &amp;lt;&amp;lt; &quot;Error!, R is out of range&quot; &amp;lt;&amp;lt; endl;&#xA;        value  = 0;&#xA;    }&#xA;    value = exp(-0.5*x)*value; // Weighted used in Longstaff-Scwartz&#xA;    return value;&#xA;}&#xA;&#xA;double generateGaussianNoise(double mu, double sigma)&#xA;{&#xA;    const double epsilon = std::numeric_limits&amp;lt;double&amp;gt;::min();&#xA;    const double two_pi = 2.0*M_PI;&#xA;&#xA;    static double z0, z1;&#xA;    static bool generate;&#xA;    generate = !generate;&#xA;&#xA;    if (!generate)&#xA;       return z1 * sigma + mu;&#xA;&#xA;    double u1, u2;&#xA;    do&#xA;     {&#xA;       u1 = rand() * (1.0 / RAND_MAX);&#xA;       u2 = rand() * (1.0 / RAND_MAX);&#xA;     }&#xA;    while ( u1 &amp;lt;= epsilon );&#xA;&#xA;    z0 = sqrt(-2.0 * log(u1)) * cos(two_pi * u2);&#xA;    z1 = sqrt(-2.0 * log(u1)) * sin(two_pi * u2);&#xA;    return z0 * sigma + mu;&#xA;}&#xA;&#xA;&#xA;QSMatrix&amp;lt;double&amp;gt; range(double min, double max, double N){&#xA;    QSMatrix&amp;lt;double&amp;gt; mat(N+1,1,0);&#xA;    double delta = (max-min)/N;&#xA;    for(int i = 0; i &amp;lt; mat.get_rows();i++){&#xA;        for(int j = 0; j &amp;lt; mat.get_cols();j++){&#xA;            mat(i,j) = min + i*delta;&#xA;        }&#xA;    }&#xA;    return mat;&#xA;}&#xA;&#xA;QSMatrix&amp;lt;double&amp;gt; randn_vec(int n, int M){&#xA;    QSMatrix&amp;lt;double&amp;gt; mat(n,1,0);&#xA;    for(int i = 0; i &amp;lt; mat.get_rows();i++){&#xA;        for(int j = 0; j &amp;lt; mat.get_cols();j++){&#xA;            mat(i,j) = generateGaussianNoise(M/2,1);&#xA;        }&#xA;    }&#xA;    return mat;&#xA;}&#xA;&#xA;QSMatrix&amp;lt;double&amp;gt; randn_vec2(int n, int M){&#xA;    QSMatrix&amp;lt;double&amp;gt; mat = randn_vec(n,M);&#xA;    QSMatrix&amp;lt;double&amp;gt; mat1(2*n,1,0);&#xA;    for(int i = 0; i &amp;lt; mat1.get_rows();i++){&#xA;            for(int j = 0; j &amp;lt; mat1.get_cols();j++){&#xA;                mat1(i,j) = mat(0,j);&#xA;            }&#xA;        }&#xA;        return mat1;&#xA;}&#xA;&#xA;QSMatrix&amp;lt;double&amp;gt; rand_vec(int n, int k){&#xA;    QSMatrix&amp;lt;double&amp;gt; mat(n,k,0);&#xA;    for(int i = 0; i &amp;lt; mat.get_rows(); i++){&#xA;        for(int j = 0; j &amp;lt; mat.get_cols(); j++){&#xA;            mat(i,j) = 0.0;&#xA;        }&#xA;    }&#xA;    return mat;&#xA;}&#xA;&#xA;&#xA;&#xA;double LSM(int T, double r, double sigma , double K, double S0, int N, int M, int R){&#xA;    double dt = T/N;                                // Time steps&#xA;    QSMatrix&amp;lt;double&amp;gt; t = range(0,T,dt);             // Time vector&#xA;    //QSMatrix&amp;lt;double&amp;gt; z = randn_vec(M/2,1);&#xA;    //QSMatrix&amp;lt;double&amp;gt; temp = rand_vec(4,1);&#xA;    //QSMatrix&amp;lt;double&amp;gt; w = temp + (r-pow(sigma,2))*T + sigma*sqrt(T);&#xA;    //z.print();&#xA;    //cout &amp;lt;&amp;lt; &quot; &quot; &amp;lt;&amp;lt; endl;&#xA;    //w.print();&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please ignore the last part above I am sort of re-editing this question as so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, so I decided instead of passing in a vector into a function I will make a new function that calls on the previous function randn_vec(). Please turn your attention to this block of code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;QSMatrix&amp;lt;double&amp;gt; randn_vec2(int n, int M){&#xA;    QSMatrix&amp;lt;double&amp;gt; mat = randn_vec(n,M);&#xA;    QSMatrix&amp;lt;double&amp;gt; mat1(2*n,1,0);&#xA;    for(int i = 0; i &amp;lt; mat1.get_rows();i++){&#xA;            for(int j = 0; j &amp;lt; mat1.get_cols();j++){&#xA;                mat1(i,j) = mat(0,j);&#xA;            }&#xA;        }&#xA;        return mat1;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I am trying to do here is make a function that basically is going to copy the Matlab function $[z;z]$ where z = randn(2,1) the randn function in Matlab generates an array or vector of Normally distributed random numbers. so for example $$z = \begin{pmatrix}&#xA;0.3188\\&#xA;-1.3077&#xA;\end{pmatrix}$$&#xA;and when you write $[z;z]$ in Matlab it produces this:&#xA;$$[z;z] = \begin{pmatrix}&#xA;0.3188\\&#xA;-1.3077\\&#xA;0.3188\\&#xA;-1.3077&#xA;\end{pmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in the latter block of code I am referring to that is what I am trying to achieve but I cannot seem to get it correctly. &lt;strong&gt;This is my new updated question&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="17826" LastEditDate="2017-01-22T21:33:26.803" LastActivityDate="2017-01-22T21:33:26.803" Title="Passing in a vector from this particular matrix class floating around online" Tags="|matlab|algorithms|c++|" AnswerCount="1" CommentCount="8" ClosedDate="2017-01-22T22:30:00.423" ContentLicense="CC BY-SA 3.0" />
  <row Id="25990" PostTypeId="2" ParentId="25989" CreationDate="2017-01-22T21:17:19.163" Score="1" Body="&lt;pre&gt;&lt;code&gt;QSMatrix&amp;lt;double&amp;gt; randn_vec2(int n,double vec){&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the culprit. You want vec to be a vector, so make it such&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;QSMatrix&amp;lt;double&amp;gt; randn_vec2(int n,QSMatrix&amp;lt;double&amp;gt; vec){&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the code has too many other bugs. For example, you have missed the loop variable in the condition in &lt;code&gt;for(int j = 0; 2*vec.length();j++){&lt;/code&gt;. You need a thorough review to make sure that everything is written as you intend it to work.&lt;/p&gt;&#xA;" OwnerUserId="22802" LastActivityDate="2017-01-22T21:17:19.163" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="25991" PostTypeId="1" CreationDate="2017-01-22T21:48:23.130" Score="4" ViewCount="469" Body="&lt;p&gt;looking to introduce a mathematics program in my work, I would have 2 uses:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In a classroom (high school level), providing good visualisation of physics ideas, and making my lecture more interactive. An example is the Mathematica demonstrations project, or simple interactive plots.&lt;/li&gt;&#xA;&lt;li&gt;Symbolic computation, I work with noncommutative geometry, algebras, etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have very limited experience with mathematics programs and zero with programming. I use Ubuntu at work and a Mac at home. I avoid the terminal most of the time, but I use it when it seems the only solution (with a lot of copy pasting). I use Latex to write mathematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica seems amazing and easy, but if the school license seems a bit expensive with the current exchange rate, the one for research is out of the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jupyter notebook and the Sagemath alternatives seems nice, but I don't know Python, not sure how complicated it's going to be implementing them, and how much time consuming could be producing simple presentations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few questions then:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are those my best alternatives?&lt;/li&gt;&#xA;&lt;li&gt;Are they good for my classroom like I explained?&lt;/li&gt;&#xA;&lt;li&gt;How difficult would be implementing them compared with Mathematica?&lt;/li&gt;&#xA;&lt;li&gt;Sagemath seems to have a bunch of different languages, should I focus in a specific one (like Maxima, or R)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23012" LastActivityDate="2017-01-23T11:16:50.427" Title="Alternatives to Mathematica" Tags="|visualization|symbolic-computation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25992" PostTypeId="2" ParentId="25991" CreationDate="2017-01-22T22:38:20.903" Score="3" Body="&lt;p&gt;Sage would be perfect for classroom use. I once assisted a professor with classroom demonstrations of Physics, and it worked out pretty well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Working with Sage does not require advanced programming knowledge. Students would be able to do it without breaking their heads like in a usual programming class. You can get started with &lt;a href=&quot;https://doc.sagemath.org/html/en/tutorial/tour.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; or &lt;a href=&quot;https://drive.google.com/file/d/0B92yb82dH00zVVRzVHJEeDVaSHM/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt;. The only thing in my opinion that makes it &quot;harder&quot; than Mathematica is the text-based input. In Mathematica you have a toolbox from which you can click integral signs and so on, and you can pretty much give input the way written equations look. In Sage, everything has to be done with text. $\int\limits_{-\infty}^\infty e^{-x^2} dx$ might be more intuitive than &lt;code&gt;integral(e^(-x^2), x,-oo,oo)&lt;/code&gt;. Otherwise programming with Sage is no more difficult than doing it in Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of focusing on Maxima or R or the other possible extensions, I suggest that you work with the Sage language itself. These extensions can only do one thing each, and you will only need to go for these extensions in advanced cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I don't know how good Sage is going to be for research level work. I made figures for my first paper in Sage, but have found that there are integrals that Mathematica can do but Sage cannot.&lt;/p&gt;&#xA;" OwnerUserId="22802" LastEditorUserId="22802" LastEditDate="2017-01-23T11:16:50.427" LastActivityDate="2017-01-23T11:16:50.427" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="25995" PostTypeId="2" ParentId="25878" CreationDate="2017-01-23T00:25:34.530" Score="1" Body="&lt;p&gt;No you can't use the center of an element as the center of gravity.  The mass of the element at the center of gravity must be distributed to the corner nodes.  $[ m]=r \int{ n^Tndv}$ where $[m]$ is the mass matrix, $r$ the density, and $n$ the shape function. $v$ is a volume or area depending whether it is a 3d or 2d problem.&lt;/p&gt;&#xA;" OwnerUserId="23011" LastEditorUserId="20526" LastEditDate="2017-01-24T14:32:42.217" LastActivityDate="2017-01-24T14:32:42.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25997" PostTypeId="2" ParentId="25991" CreationDate="2017-01-23T07:21:05.357" Score="3" Body="&lt;p&gt;Having used Mathematica, then trying Sage, and now SymPy/SymEngine in Julia, the clear winner is SymEngine.jl. Sage was hard to get working, was very slow, and I found it very hard to develop my own algorithms (which would get decent performance. That's the key: you can write algorithms for it, but they won't necessary be speedy!). The syntax can be really hard to learn too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I still enjoy Mathematica's notebook, SymEngine.jl has a lot going for it. Even SymPy.jl used from Julia is a very good experience (better than using in Python). [SymEngine.jl is the C++ re-write of SymPy]. First of all, they're free. But most of all, it works as part of Julia, and works naturally through dispatch. If you have expressions &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;, then to make the expression &lt;code&gt;a+b&lt;/code&gt;, the code is... &lt;code&gt;a+b&lt;/code&gt;. Everything just works via proper dispatches. I mean, everything. You don't even need to really know much about the packages themselves. Julia provides a generic fallback for &lt;code&gt;inv(A)&lt;/code&gt; which takes an inverse of a matrix. If you have a matrix of symbolic expressions, then because symbolic expressions has &lt;code&gt;+&lt;/code&gt; and &lt;code&gt;/&lt;/code&gt; defined, by extension &lt;code&gt;inv(A)&lt;/code&gt; just works. You can even do absurd things with them like throw them in a numerical ODE solver and it'll just work like it's a number, and give out the symbolic expression for the solution after &lt;code&gt;n&lt;/code&gt; steps. So if you know Julia and its Base library, then you already know all of the commands for SymEngine.jl/SymPy.jl. Also, every type-stable Julia function you write compiles down to something with C performance, so you don't have to worry there either. So you tend to just use them like normal Julia numbers with a few special functions (like &lt;code&gt;differentiate&lt;/code&gt;) and everything is fast: makes it really easy to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing that's missing is the beautiful notebook. You can use a Jupyter notebook, but it's still not the same because the Mathematica one where you code actually has fractions and all of that. But the performance and productivity increase definitely makes up for the lack of notebook.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-01-23T07:21:05.357" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="25998" PostTypeId="1" CreationDate="2017-01-23T11:58:51.147" Score="1" ViewCount="367" Body="&lt;p&gt;Consider $f(x)= \sum\limits_{n=0}^N a_n p_n (x)$, where $p_n$ are the Legendre polynomials. If one wants to differentiate $f'$ symbolically, i.e. to compute $$f'(x) = \sum\limits_{n=0}^{N-1} b_n p_n (x)= \sum\limits_{n=0}^N a_n p_n '(x)$$, he should be able to use the fact that the $p_n'(x)$ are well known for the Legendre polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Question:&lt;/strong&gt; Given a vector &lt;code&gt;a_n&lt;/code&gt; that represents the coefficients $a_n$, how can I get, in Matlab, the coefficients $b_n$ of the above derivative? Is there a built in function for that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My Second question&lt;/strong&gt; Can I compute symbolically the following coefficients: $$f'(x) =\sum\limits_{n=0}^N c_n x^n$$&lt;/p&gt;&#xA;" OwnerUserId="15188" LastEditorUserId="15188" LastEditDate="2017-01-23T14:57:44.670" LastActivityDate="2017-01-24T12:56:28.477" Title="Matlab symbolic differentiation of Legendre polynomials" Tags="|matlab|polynomials|special-functions|symbolic-computation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="25999" PostTypeId="2" ParentId="25998" CreationDate="2017-01-23T13:30:30.243" Score="4" Body="&lt;p&gt;Philips in 1988  proved the following relationship:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $f(x)$ is an infinitely differentiable function defined on the interval $[-1,+1]$ and its Legendre expansion is given by $f(x) = \sum_{n=0}^{\infty}a_{n}P_{n}(x)$, then the Legendre coefficients $a_{n}^{(q)}$ of the $q$-th derivative of $f(x)$ are given by $$a_{n}^{(q)} = \frac{(2n+1)}{2^{q-2}(q-1)!} \sum_{k=1}^{\infty} \frac{(k+q-2)!}{(k-1)!}\frac{(2n+2k+2q-3)!(n+k)!}{(2n+2k)!(n+q+k-2)!} a_{n+2k+q-2}$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simplifying this for $q=1$, one finds&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a_{n}^{(1)} = (2n+1) \sum_{k=1}^{\infty} a_{n+2k-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You need a large number of coefficients (in theory an infinite amount) to calculate the coefficients of the derivative. This means that the order at which you cut-off the series will have an impact on the accuracy. If the original approximation is cut-off at order $M$, then clearly all $a_n$ for $n&amp;gt;M$ are identical to zero and the summation is finite. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Care  must be taken in the summation if the $a_n$ have different signs. You'll quickly loose correct significant digits in IEEE double precision arithmetic.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The evaluation of $\sum_{n=0}^{N}a_{n}^{(q)}P_{n}(x)$ is best done using the Clenshaw-Smith algorithm (see &lt;a href=&quot;http://www.ams.org/journals/mcom/1965-19-089/S0025-5718-1965-0172445-6/S0025-5718-1965-0172445-6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Smith's paper&lt;/a&gt;). This is much more stable than converting to the monomial base and applying Horner's rule. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Reference&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;T.N. Philips, &quot;On the Legendre Coefficients of a General-Order Derivative of an Infinitely Differential Function&quot;, IMA Journal of Numerical Analysis, 1988, Volume 8, p. 455--459&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2017-01-24T12:56:28.477" LastActivityDate="2017-01-24T12:56:28.477" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26000" PostTypeId="2" ParentId="21483" CreationDate="2017-01-23T15:46:58.247" Score="6" Body="&lt;p&gt;Below are some tips to reduce the effect of round off errors.&#xA;A short method is to increment the floating point precision, for example from &lt;em&gt;float&lt;/em&gt; to &lt;em&gt;double&lt;/em&gt;, but many times this is too expensive or not possible.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Kahan summation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Kahan summation&lt;/a&gt; the idea is to make up for the mistake made in the previous step.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function KahanSum(input)&#xA;var sum = 0.0&#xA;var c = 0.0                  // A running compensation for lost low-order bits.&#xA;for i = 1 to input.length do&#xA;    var y = input[i] - c         // So far, so good: c is zero.&#xA;    var t = sum + y              // Alas, sum is big, y small, so low-order digits of y are lost.&#xA;    c = (t - sum) - y        // (t - sum) cancels the high-order part of y; subtracting y recovers negative (low part of y)&#xA;    sum = t                  // Algebraically, c should always be zero. Beware overly-aggressive optimizing compilers!&#xA;next i                       // Next time around, the lost low part will be added to y in a fresh attempt.&#xA;return sum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The variable &lt;em&gt;c&lt;/em&gt; is the correction to use, step by step, with the new sum element &lt;em&gt;input[i]&lt;/em&gt;. The error during &lt;em&gt;c&lt;/em&gt; evaluation is more small, because the numbers in game are in the same order of magnitude. The important step is that the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c = (t - sum) - y &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is explicit evaluated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implemented code can not be effective due to compiler, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm#Possible_invalidation_by_compiler_optimization&quot; rel=&quot;noreferrer&quot;&gt;wiki&lt;/a&gt;, i.e. the compiler puts all in a unique and direct command so the previous line are not evaluated alone.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Sort&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is a trick quite simple to use, before the sum sort the array and start from the smallest value. The idea is that small sum grows and became more large so the round off errors are mitigate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not guaranteed to work, but it is a good practice to use.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Subtraction, array with mixed signed values&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Floating point subtraction is not a stable operation.&#xA;In the case the array contains mixed signed values, need some extra attentions to avoid that the two numbers in absolute value are very close. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Reduce Operations&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;This is a general advise, not strictly related to a generic sum of values.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try to reduce the operation to obtain the result. An example is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Horner&amp;#39;s_method&quot; rel=&quot;noreferrer&quot;&gt;Horner's method&lt;/a&gt; where a polynomial can be rewrite in the form&#xA;$$&#xA;P_N(x) = a_N + x ( a_{N-1} + x ( a_{N-2} + \ldots + x (a_1 + a_0 x) \ldots))&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this form the polynomial can be evaluate in $N$ additions and $N$ multiplications, against the $N$ additions and $\frac{N(N+1)}{N}$ multiplications of the normal form. (more details, &lt;a href=&quot;https://en.wikipedia.org/wiki/Horner%27s_method#Efficiency&quot; rel=&quot;noreferrer&quot;&gt;wiki&lt;/a&gt; )&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="1804" LastEditDate="2017-01-23T15:53:36.270" LastActivityDate="2017-01-23T15:53:36.270" CommentCount="1" CommunityOwnedDate="2017-01-23T15:46:58.247" ContentLicense="CC BY-SA 3.0" />
  <row Id="26003" PostTypeId="1" CreationDate="2017-01-24T08:31:49.487" Score="1" ViewCount="119" Body="&lt;p&gt;Consider $[0,1]$ with the Lebesgue measure $m$ and $f:[0,1]\to \mathbb{R}$, and $x$ a uniformly distributed random variable in $[0,1]$. Then, $f(x)$ itself define a new random variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can then define the Cumulative density function of $f$ as $$P_f (y) :\,= m(f^{-1} (-\infty, y)) \, ,$$ and its PDF as $$\rho _f (y) :\,= \frac{dP_f (y)}{dy} \, , $$ for all $y\in {\rm range} (f)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A straightforward numerical computation of the probability density function (PDF) of $f$ involves inverse derivatives, or, conversely, $1/f'$. As numerical differentiation is numerically ill-conditioned, when $f'$ becomes very low, this is numerically problematic. I'm looking for one of two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;An out of the box&lt;/strong&gt; function in Matlab that computes a PDF for a given function $f$.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;A numerical Algorithm&lt;/strong&gt; for this task.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="15188" LastEditorUserId="15188" LastEditDate="2017-01-24T21:25:19.027" LastActivityDate="2017-01-24T21:25:19.027" Title="Numerically compute PDF given a function" Tags="|matlab|numerical-analysis|probability|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26004" PostTypeId="1" CreationDate="2017-01-24T12:26:59.733" Score="1" ViewCount="1219" Body="&lt;p&gt;I wanted to generate LU decomposition of large size dense matrices ($N&amp;gt;10^7$), the LU decomposition I'm currently using is based on &lt;a href=&quot;https://drive.google.com/file/d/0B4tReu45goapY2g4VG5IRlM3aWM/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adaptive Cross Approximation&lt;/a&gt; and is taking very long time to execute for larger $N$. Can anybody suggest a few LU decomposition techniques that can be very well parallelised (using OpenMP) and take a shorter period of time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I write the code in C++ and make use of Xeon Processor(128 threads)&#xA;and Eigen library.&lt;/li&gt;&#xA;&lt;li&gt;The entries in the matrix are filled through a kernel function of&#xA;form $e^{-(x_1-x_2)^2}$.&lt;/li&gt;&#xA;&lt;li&gt;Storage of matrix is not a problem, I'm working on Xeon processor and has enough memory and moreover, I'm not storing the full matrix and whenever  I need to find an entry in matrix, I use the kernel function and generate type-double number for that cell.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23023" LastEditorUserId="20688" LastEditDate="2017-01-25T03:02:29.367" LastActivityDate="2017-01-25T03:02:29.367" Title="LU decomposition of large dense matrices" Tags="|optimization|algorithms|c++|matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26005" PostTypeId="2" ParentId="26004" CreationDate="2017-01-24T13:55:14.797" Score="1" Body="&lt;p&gt;You can't. Unless you have some special knowledge, your $L$ and $U$ factors are going to be dense, and will have $N^2 = {\cal O}(10^{14})$ entries -- more than you can store on any reasonable machine (about a million GB). Furthermore, it takes ${\cal O}(N^3)={\cal O}(10^{21})$ operations to compute these factors, i.e., $3\cdot 10^4$ CPU years -- longer than you want to wait. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem you have can simply not be solved via LU decomposition. You need to come up with other strategies for doing what you want to do -- e.g., solving linear systems with Krylov space methods, computing explicit Green's functions, or using knowledge about sparsity.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-01-24T13:55:14.797" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26006" PostTypeId="1" CreationDate="2017-01-24T15:56:47.487" Score="2" ViewCount="136" Body="&lt;p&gt;Can the order of the local truncation error be higher than the order of global truncation error in a finite volume method?&lt;/p&gt;&#xA;" OwnerUserId="8166" LastActivityDate="2017-01-24T15:56:47.487" Title="Order of local and global truncation error in a finite volume scheme" Tags="|numerical-analysis|fluid-dynamics|finite-volume|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26007" PostTypeId="1" CreationDate="2017-01-24T17:29:21.143" Score="1" ViewCount="1837" Body="&lt;p&gt;I have a large dense matrix $A$. For simplicity in simulation purposes and application demand, I induced sparsity by replacing lower/insignificant values by zero and reordering it in block diagonal form e.g $\hat{A}$. How can I measure how much information I lost in this process? What property of matrix represents it's index of information. (It can not be distance as I reorder the elements ) &#xA;Thank you very much in advance. &lt;/p&gt;&#xA;" OwnerUserId="23029" LastActivityDate="2017-01-25T14:03:01.027" Title="Comparison between two matrices" Tags="|linear-algebra|matrix|dense-matrix|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26008" PostTypeId="1" AcceptedAnswerId="26010" CreationDate="2017-01-24T18:11:29.657" Score="1" ViewCount="743" Body="&lt;p&gt;I am testing the ability of my integration method to reproduce results obtained analytically (for various values of the parameters), but the results turn out to be completely wrong and I can't find what's wrong with my code.&lt;/p&gt;&#xA;&lt;p&gt;As an example, for the parameter values below, &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; is supposed to tend towards &lt;span class=&quot;math-container&quot;&gt;$40$&lt;/span&gt; with time (t).&#xA;However, when I run this code in PyCharm, &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; stops at &lt;span class=&quot;math-container&quot;&gt;$0.1227$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The dynamic equation used is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{dx_{1}}{dt} = (N - x_1) \mu  - \frac{\theta x_1}{k + x_1^2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated, thanks !&lt;/p&gt;&#xA;&lt;h1&gt;Code:&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;#Parameters:&#xA;N = 50  &#xA;theta= 0.6  &#xA;µ = 0.6/412.5  &#xA;k = 1  &#xA;n = 2  &#xA;x1 = 0   #x1(0)  &#xA;listx1 = []  #vector of all x1-values  &#xA;dt = 0.01    #delta t  &#xA;&#xA;from math import pow  &#xA;import matplotlib.pyplot as plt  &#xA;&#xA;t= 3600*24  # time is 1 day  &#xA;while t&amp;gt;0:  &#xA;    listx1.append(x1)  &#xA;    dx1 = (µ * (N-x1) - (theta * x1) / (k + pow(x1, 2))) * dt  &#xA;    x1 = x1 + dx1  &#xA;    t -= dt  &#xA;    &#xA;for i in range(500):&#xA;    print(&amp;quot;x1=&amp;quot;, listx1[i])&#xA;&#xA;print(&amp;quot;x1 final:&amp;quot;, listx1[-1])&#xA;&#xA;t = []&#xA;for i in range(len(listx1)): &#xA;    t.extend([i])&#xA;&#xA;plt.plot(t, listx1, 'r')&#xA;plt.autoscale(enable=True, axis='both', tight=False)&#xA;v = [0, 3600 * 24, 0, 60] &#xA;plt.axis(v)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23026" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-01-24T20:09:48.607" Title="What is wrong with this Euler method code in python?" Tags="|python|integration|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26010" PostTypeId="2" ParentId="26008" CreationDate="2017-01-24T19:53:37.677" Score="7" Body="&lt;p&gt;I did not check your code, however, the result you are getting is also verified by &lt;code&gt;scipy.integrate.odeint&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.integrate import odeint&#xA;&#xA;def ode(x, t, N, theta, mu, k):&#xA;    dxdt = (N - x) * mu - theta * x / (k + x**2)&#xA;&#xA;    return dxdt&#xA;&#xA;N = 50  &#xA;theta= 0.6  &#xA;mu = 0.6/412.5  &#xA;k = 1  &#xA;&#xA;x0 = 0&#xA;t = np.linspace(0, 30, 101)&#xA;sol = odeint(ode, x0, t, args=(N, theta, mu, k))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the plot of the solution (note that a steady steady has already been reached for $t=30$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iyIzi.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iyIzi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The steady state value is &lt;code&gt;sol[-1]&lt;/code&gt;, which equals $0.1227$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you say you expected a value of around $40$. Let's see what happens. At the steady state, it must hold (with $x_{ss}\triangleq \lim_{t\rightarrow \infty} x_1(t)$)&#xA;$$ 0 = (N - x_{ss}) \mu - \frac{\theta x_{ss}}{k+x_{ss}^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the steady state solution is the root of 3rd order polynomial, i.e., there should be three possible steady state values for the system. These can be found using sympy as follows.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sympy as sp&#xA;&#xA;t, N, theta, mu, k = sp.symbols('t, N, theta, mu, k', real = True)&#xA;x_ss = sp.symbols('x_ss', real = True)&#xA;eq = sp.Eq(0, (N - x_ss) * mu - theta * x_ss / (k + x_ss**2))&#xA;&#xA;sol_sym = sp.solve(eq, x_ss)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sol_sym&lt;/code&gt; is a list containing the three roots in a symbolic form. Substituting the numerical values for the parameters it evaluates as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[s.subs({N: 50, theta: 0.6, mu: 0.6/412.5, k:1}).n() for s in sol_sym]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; [0.12273605326634 - 2.29e-16*I, 10.2908641151715 + 3.09e-16*I,&#xA;&amp;gt; 39.5863998315621 - 7.96e-17*I]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Ignoring the essentially zero imaginary components, it follows that the steady state value can indeed take the value $0.1227$ obtained by the numerical integration, as well as $39.586$ and $10.29$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, with the initial condition $x_1(0)=0$, the system reaches the first steady state value. You need to consider a different initial value to reach $40$. For example, running the numerical code with $x_1(0) = 35$, the system reaches this value at around $t=10000$&lt;/p&gt;&#xA;" OwnerUserId="21460" LastEditorUserId="21460" LastEditDate="2017-01-24T20:09:48.607" LastActivityDate="2017-01-24T20:09:48.607" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26011" PostTypeId="2" ParentId="26004" CreationDate="2017-01-24T20:42:06.663" Score="1" Body="&lt;p&gt;I'm not exactly sure what type of framework for LU are you using, as one can apply ACA to various different setups. And to make it explicit, the approach you are working on right now, and I am going to propose does not give you an LU-decomposition. It is &lt;b&gt;an approximation of an LU-decomposition&lt;/b&gt; in some form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are using a non-singular kernel, I guess you can try to compress the whole matrix. Usually, those methods are applied to singular kernels and then you have a hierarchical pattern, when you subdivide you matrix into blocks in a multilevel fashion and most of them are compressible. Those blocks would be computed using some fast technique, like ACA. It's worth mentioning, that ACA is less than ideal, has troubles with the controlled accuracy and etc. Then, you can factorize the matrix. The approach I described is pretty much Hierarchical matrix ($\mathcal H$-matrix) approach, developed by Dr. W. Hackbusch &lt;a href=&quot;http://www.hmatrix.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.hmatrix.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To cut along story short, that approach allows for $\mathcal O(k_\text{max}^3N\log^2N)$ LU decomposition and $\mathcal O(k_\text{max}^3N\log N)$ for back substitution and pretty well parallelizable via OpenMP. Notice, here $k_\text{max}$ is the maximum rank, that for ACA would mean the number of skeletons. Of course, that assumes that you have a nice balanced partitioning. Details and limitations are available in numerous papers on $\mathcal H$-matrix papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at this approach, and their available $\mathcal H$lib library, that is available to the public. Other suggestion might include several other frameworks, like HSS, that I know about (but much less familiar), and more from computational electromagnetics:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;J. Shaeffer, &quot;Million plus unknown MOM LU factorization on a PC,&quot; 2015 International Conference on Electromagnetics in Advanced Applications (ICEAA), Turin, 2015, pp. 62-65. doi: 10.1109/ICEAA.2015.7297075&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;S. Kapur and D. E. Long, &quot;N-body problems: IES3: Efficient electrostatic and electromagnetic simulation,&quot; in IEEE Computational Science and Engineering, vol. 5, no. 4, pp. 60-67, Oct.-Dec. 1998. doi: 10.1109/MCSE.1998.7102081&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-01-24T20:42:06.663" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26012" PostTypeId="2" ParentId="26007" CreationDate="2017-01-24T21:01:17.347" Score="1" Body="&lt;p&gt;There are multiple ways to judge the information lost. The easiest way would be to calculate Frobenius norm of the, so called, &lt;i&gt;sparsified&lt;/i&gt; matrix $\hat A$ and an original dense matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$||A||_F=\sqrt{\sum\limits_{i}\sum\limits_{j}|a_{ij}|^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relative error in the Frobenius norm can tell you something about how much you have dropped relatively:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{Error}_F=\frac{||A-\hat A||_F}{||A||_F}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, Frobenius norm has nice connection to the spectral properties of the matrix which might be useful: $||A||_2\leq||A||_F$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, judging just &quot;element-wise&quot; is not very good, as well as just simply zeroing out &lt;i&gt;allegedly&lt;/i&gt; small and insignificant blocks. I would start from plotting the singular values of the original matrix and compared them to the ones of the approximated one. That would allow you to judge how did you change the spectrum properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the conclusion, I would suggest you take a look into various techniques that allow for fast matrix algebra for similar approximations, like $\mathcal H$-matrices (&lt;a href=&quot;http://www.hmatrix.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.hmatrix.org/&lt;/a&gt;), HSS (hierarchical semi-separable), and others, rather than just dropping the matrix entries below a certain threshold. However, for your application your approach might be accurate enough.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2017-01-25T03:36:51.987" LastActivityDate="2017-01-25T03:36:51.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26013" PostTypeId="1" CreationDate="2017-01-25T03:14:35.913" Score="2" ViewCount="323" Body="&lt;p&gt;Going off of the answer here:&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/10936/sorting-adjacency-matrix-by-the-fiedler-vector/10937#10937?newreg=ca06ec045dde4c48b62c60d45a4aacc8&quot;&gt;sorting adjacency matrix by the Fiedler vector&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here, Jesse the answerer plotted the first 3 eigenvectors associated with non-zero eigenvalues of the Laplacian against each other in order to see more distinguishable clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My 2 questions about this are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Why did he choose specifically the first 3 eigenvectors to plot, as opposed to for example, just the first 2?  Like what determines how many eigenvectors you should compare?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is there a way to quantify connectivity between clusters?  For example, when looking at the graph Jesse originally made, is there a way to tell from plotting the eigenvector values that the top cluster is more closely connected to the second top most cluster than the bottom most cluster?&lt;/p&gt;&#xA;" OwnerUserId="23036" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-01-25T03:14:35.913" Title="Connectivity and Clustering using Eigenvectors and the Fiedler Vector" Tags="|linear-algebra|clustering|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26014" PostTypeId="2" ParentId="14527" CreationDate="2017-01-25T08:19:38.127" Score="1" Body="&lt;p&gt;I usually use &lt;em&gt;gradient averaging&lt;/em&gt; since it has some superconvergence properties under certain regularity assumptions and I always tend to have some sort of finite element code lying around when I want to do this. Let $V_h$ be for example the piecewise linear finite element space defined on the mesh of the domain $\Omega$. The idea is to consider the projection problem: find $w \in V_h$ such that&#xA;$$\int_\Omega w v \,\mathrm{d}x = \int_\Omega \frac{\partial f}{\partial x} v\,\mathrm{d}x, \quad \forall v \in V_h.$$&#xA;The matrix formulation reads&#xA;$$M\boldsymbol{w}=C\boldsymbol{f},$$&#xA;where $M$ is the typical mass matrix, $\boldsymbol{f}$ is the vector of nodal values and $C_{ij}=\int_\Omega \frac{\partial \varphi_j}{\partial x} \varphi_i \,\mathrm{d}x$ where $\varphi_i$ is the basis of $V_h$. Then you simply solve&#xA;$$\boldsymbol{w}=M^{-1}C\boldsymbol{f}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you  don't have a finite element code lying around, you can implement the equivalent projection by computing the piecewise constant derivative in each neighboring element of a node, taking their weighted average with the areas of the triangles as weights and let that average value represent the derivative at the node.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2017-01-25T08:19:38.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26015" PostTypeId="2" ParentId="25927" CreationDate="2017-01-25T08:28:20.023" Score="0" Body="&lt;p&gt;You can only have 9 unknowns as you have 9 equations. Use of a staggered grid introduces more unknowns then equations, hence you have to use a collocated grid. Hence applying the continuity equation for a cell having center $(i,j)$ you get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u|_{i-\frac{1}{2},j}^{i+\frac{1}{2},j}+v|_{i,j-\frac{1}{2}}^{i,j+\frac{1}{2}}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The edge velocities can be approximated by geometric avergae of values in adjacent cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;eg.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{i+\frac{1}{2},j}=\frac{u_{i+1,j}+u_{i,j}}{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying in a such a manner, should enable you to discretize with 9 unknowns, 9 equations. &#xA;Add a source term wherever needed.&lt;/p&gt;&#xA;" OwnerUserId="8166" LastEditorUserId="8166" LastEditDate="2017-01-25T09:05:55.297" LastActivityDate="2017-01-25T09:05:55.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26016" PostTypeId="1" AcceptedAnswerId="26019" CreationDate="2017-01-25T09:57:37.367" Score="1" ViewCount="605" Body="&lt;p&gt;I want to numerically solve the advection-diffusion equation:&#xA;\begin{equation}&#xA;u_t(x,t) + cu_x(x,t) = v u_{xx}(x,t) &#xA;\end{equation}&#xA;for $x \in [0,1]$ and $t \geq 0$ subject to the boundary conditions  $u(0,t) = u(1,t) = 0 $ and $u(x,0) = f(x)$. To compare the accuracy of the numerical solution, I will at first derive the analytical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analytical Solution&lt;/strong&gt; The solution satisfies &#xA;\begin{equation}&#xA; u(x,t) = h(x,t) \exp \left( \alpha x + \beta t \right) &#xA;\end{equation} &#xA;If one sets $\alpha = c /2v$ and $\beta = - c^{2} / 4v$ the function $h(x,t)$ adheres to the conventional heat equation: $h_t(x,t) = vh_{xx}(x,t)$ suject to $h(0,t) = h(1,t) = 0$ and $h(x,0) = g(x)$. The solution is particularly inexpensive to calculate if one sets $g(x) = \sin ( 2\pi x)$. Then, the function $u$ is: &#xA;\begin{equation}&#xA;u(x,t) =  \sin (2 \pi x) \exp  \left( -4 \pi^2 v t - \dfrac{c^2 t}{4v} + \dfrac{cx}{2v} \right )&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Numerical Solution with BCTS and Crank-Nicolson&lt;/strong&gt;&#xA;To solve the advection-diffusion equation numerically, I use the BTCS and Crank-Nicolson algorithms. Define first &#xA;\begin{align*}&#xA;r &amp;amp;=  \dfrac{v \Delta t}{\Delta x^{2}} \\&#xA;R &amp;amp;=  \dfrac{c \Delta t}{\Delta x} &#xA;\end{align*} &#xA;The discretized version of the advection-diffusion equation adherring to the BTCS algorithm is: &#xA;\begin{equation}&#xA;u_{k}^{n}(1 + 2r) + u_{k-1}^{n} ( -R/2 - r) + u_{k+1}^{n} (R/2 -r) = u_{k}^{n-1}&#xA;\end{equation}&#xA;The differential equation for the Crank-Nicolson algorithm can be written as:&#xA;\begin{align*}&#xA;&amp;amp;u_{k}^{n} ( 1 + r) + u_{k+1}^{n} ( R/4 - r/2)  + u_{k-1}^{n} ( -R/4 - r/2) = \\&#xA;&amp;amp; u_{k}^{n-1} ( 1 - r) + u_{k+1}^{n} ( -R/4 + r/2) + u_{k-1}^{n} ( R/4 + r/2) &#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Simulations&lt;/strong&gt; When I solve the advection-diffusion algorithm with $c=1, v=2$ and $\Delta t = 0.001$ and $\Delta x \approx 0.0416 $ the solution looks as follows:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/clwJb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/clwJb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analytical solution is both positive and negative, the numerical solution however is an inverted parabola. With  a reduced strength of the diffusion term $v=1/6$ the solution is very accurate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SVe5Z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SVe5Z.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I compute the numerical solution accurately over a large range of values for $v$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am particularly concerned about this question, as I solved the advection-diffusion equation to understand the algorithms and want to apply them to a non-linear PDE of which I don't know the analytical solution. The computer code used for the example is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy import linalg&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;class ConvectionDiffusion(object):&#xA;    &quot;&quot;&quot;&#xA;    Class to construct solutions to the convection-diffusion equation&#xA;    &quot;&quot;&quot;&#xA;&#xA;    def __init__(self, DELTA_T, M, V, C):&#xA;        &quot;&quot;&quot;&#xA;        Parameters:&#xA;        -----------&#xA;        DELTA_T: scalar(float):&#xA;            The time step Delta_t&#xA;&#xA;        M: scalar(float):&#xA;            The number of grid-points for x&#xA;&#xA;        V: scalar(float):&#xA;            The constant for the diffusion term&#xA;&#xA;        C: scalar(float):&#xA;            The constant for the advection term&#xA;        &quot;&quot;&quot;&#xA;        self.DELTA_T = DELTA_T&#xA;        self.M = M&#xA;        self.DELTA_X = 1 / (M - 1)&#xA;        self.xVec = np.linspace(0, 1., num=M)&#xA;        self.C = C&#xA;        self.V = V&#xA;        self.R = C * DELTA_T / self.DELTA_X&#xA;        self.r = V * DELTA_T / self.DELTA_X**2&#xA;        self.u0 = np.sin(2 * np.pi * self.xVec) * \&#xA;            np.exp(self.C * self.xVec / (2 * self.V))&#xA;&#xA;    def Implicit(self, stencil, RHS, Sol, l_and_u, T):&#xA;        &quot;&quot;&quot;&#xA;        Solving a PDE with implicit algorithms through (banded) matrix inversion&#xA;&#xA;        Parameters:&#xA;        -----------&#xA;        stencil: fun&#xA;            Function defining for each value u^{n-1} the stencil such that&#xA;            the discretized PDE can be iterated forward&#xA;&#xA;        RHS: fun&#xA;            Constructs the RHS of A x = b.&#xA;&#xA;        l_and_u: tupel(int)&#xA;            The number of lower and upper diagonal elements used in stencil to&#xA;            use linalg.solve_banded&#xA;&#xA;        T: scalar(float):&#xA;            The time at which the solution to the PDE is expressed.&#xA;&#xA;        Sol: fun&#xA;            Function generating next periods u^{n} from the solution in&#xA;            the interior `x` and the boundary values&#xA;&#xA;        Return:&#xA;        -------&#xA;        initial: array_like(float):&#xA;            The solution to the PDE at time T&#xA;        &quot;&quot;&quot;&#xA;        DELTA_T, initial = self.DELTA_T, self.u0&#xA;        t = 0.0&#xA;        while t &amp;lt; T:&#xA;            A = stencil(initial)&#xA;            b = RHS(initial)&#xA;            x = linalg.solve_banded(l_and_u, A, b)&#xA;            initial = Sol(x)&#xA;            t = t + DELTA_T&#xA;        return initial&#xA;&#xA;    def stencilCN(self, initial):&#xA;        &quot;&quot;&quot;&#xA;        Stencil for the Crank-Nicolson algorithm; Stencil * u^n = b&#xA;&#xA;        Parameters:&#xA;        ----------&#xA;        initial: array_like(float):&#xA;            The previous solution&#xA;&#xA;        Returns:&#xA;        --------&#xA;        A: banded_matrix(float):&#xA;            The matrx in baned for to pass it to scipy.linalg.banded_solve&#xA;        &quot;&quot;&quot;&#xA;        A = np.zeros((3, len(initial) - 2))&#xA;        R, r = self.R, self.r&#xA;        A[0, 1:] = R / 4 - r / 2&#xA;        A[1, :] = 1 + r&#xA;        A[2, :-1] = -R / 4 - r / 2&#xA;        return A&#xA;&#xA;    def stencil(self, initial):&#xA;        &quot;&quot;&quot;&#xA;        Stencil for the BCTS algorithm, ; Stencil * u^n = u^{n-1}&#xA;&#xA;        Parameters:&#xA;        ----------&#xA;        initial: array_like(float):&#xA;            The previous solution&#xA;&#xA;        Returns:&#xA;        --------&#xA;        A: banded_matrix(float):&#xA;            The matrx in baned for to pass it to scipy.linalg.banded_solve&#xA;        &quot;&quot;&quot;&#xA;        A = np.zeros((3, len(initial) - 2))&#xA;        R, r = self.R, self.r&#xA;        A[0, 1:] = R / 2 - r  # uper diagonal&#xA;        A[1, :] = 1 + 2 * r&#xA;        A[2, :-1] = -R / 2 - r  # lower diagonal&#xA;        return A&#xA;&#xA;    def RHS(self, initial):&#xA;        &quot;&quot;&quot;&#xA;        Pepare the RHS for the BCTS algorithm.&#xA;&#xA;        Parameters:&#xA;        -----------&#xA;        initial: array_like(float):&#xA;            The previous solution u^{n-1}&#xA;&#xA;        Returns:&#xA;        --------&#xA;        b: array_like(float):&#xA;            Only the interior values of u^{n-1}&#xA;        &quot;&quot;&quot;&#xA;        return initial[1: -1]&#xA;&#xA;    def RHSCN(self, initial):&#xA;        &quot;&quot;&quot;&#xA;        Pepare the RHS for the Crank-Nicolson algorithm.&#xA;&#xA;        Parameters:&#xA;        -----------&#xA;        initial: array_like(float):&#xA;            The previous solution u^{n-1}&#xA;&#xA;        Returns:&#xA;        --------&#xA;        b: array_like(float):&#xA;            A weighted sum of previous u^{n-1} values&#xA;        &quot;&quot;&quot;&#xA;        R, r = self.R, self.r&#xA;        present = initial[1:-1] * (1 - r)&#xA;        past =  initial[:-2] * (R / 4 + r / 2)&#xA;        future = initial[2:] *(-R / 4 + r / 2)&#xA;        b =  present + past + future&#xA;        return b&#xA;&#xA;    def Sol(self, x):&#xA;        &quot;&quot;&quot;&#xA;        Add the boundary condition&#xA;&#xA;        Parameters:&#xA;        x: array_like(float):&#xA;            The solution in the interior&#xA;&#xA;        Returns:&#xA;        --------&#xA;        uNew: array_like(float):&#xA;            The entire solution&#xA;        &quot;&quot;&quot;&#xA;        uNew = np.zeros(len(x) + 2)&#xA;        uNew[1:-1] = x&#xA;        return uNew&#xA;&#xA;    def analyticalSol(self, x, T):&#xA;        &quot;&quot;&quot;&#xA;        Analytical Solution of the advection-difussion equation&#xA;&#xA;        Parameters:&#xA;        -----------&#xA;        x   array_like(float):&#xA;            The state space&#xA;        T:  scalar(float):&#xA;            The time at which the solution is evaluated&#xA;&#xA;        Returns:&#xA;        v:  array_like(float):&#xA;            The solution for the PDE&#xA;        &quot;&quot;&quot;&#xA;        C, V = self.C, self.V&#xA;        exponential = (- 4 * np.pi**2 * V * T&#xA;                       - C**2 * T / (4 * V) + C * x[1:-1] / (2 * V) )&#xA;        interior = np.sin(2 * np.pi * x[1:-1]) * np.exp(exponential)&#xA;        v = self.Sol(interior)&#xA;        return v&#xA;&#xA;    def ComparisonSol(self, T):&#xA;        &quot;&quot;&quot;&#xA;        Calculates the analytical solution and the its numercial approximation&#xA;        according to the BCTS, C-N and Explicit algorithm at time T&#xA;&#xA;        Parameters:&#xA;        T:  scalar(float):&#xA;            The time value&#xA;&#xA;        Returns:&#xA;        --------&#xA;        sol: array_like(float):&#xA;            The array with all solutions&#xA;&#xA;        e:  array_like(float):&#xA;            The normalized second error norm of each approximation&#xA;        &quot;&quot;&quot;&#xA;        xVec, DELTA_T, DELTA_X = self.xVec, self.DELTA_T, self.DELTA_X&#xA;        sol, E = np.empty((len(self.u0), 3)), np.zeros(2)&#xA;&#xA;        ## == Anaytical == ##&#xA;        sol[:, 0] = self.analyticalSol(xVec, T)&#xA;&#xA;        ## == BTCS == ##&#xA;        sol[:, 1] = self.Implicit(self.stencil, self.RHS, self.Sol, (1, 1), T)&#xA;        E[0] = np.linalg.norm(sol[:, 0] - sol[:, 1]) / np.linalg.norm(sol[:, 0])&#xA;&#xA;        ## == CN == ##&#xA;        sol[:, 2] = self.Implicit(&#xA;            self.stencilCN, self.RHSCN, self.Sol, (1, 1), T)&#xA;        E[1] = np.linalg.norm(sol[:, 0] - sol[:, 2]) /np.linalg.norm(sol[:, 0])&#xA;        return sol, E&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    Simulation = ConvectionDiffusion(DELTA_T=0.001, M=50, V=1/6, C=1.)&#xA;    y, e = Simulation.ComparisonSol(0.5)&#xA;    x = Simulation.xVec&#xA;    fig, ax1 = plt.subplots()&#xA;    ax1.plot(x, y[:, 0], 'b-', label='Anaytical Solution')&#xA;    ax1.set_xlabel('x')&#xA;    ax1.set_ylabel('Axis Anaytical Solution', color='b')&#xA;    ax1.tick_params('y', colors='b')&#xA;    ax1.legend(loc=2)&#xA;&#xA;    ax2 = ax1.twinx()&#xA;    ax2.plot(x, y[:, 1], 'r--', label='BTCS ')&#xA;    ax2.plot(x, y[:, 2], 'r:', label='C-N ')&#xA;    ax2.set_ylabel('Axis Numerical Solutions', color='r')&#xA;    ax2.tick_params('y', colors='r')&#xA;    ax2.legend()&#xA;    fig.tight_layout()&#xA;    plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14651" LastEditorUserId="14651" LastEditDate="2017-01-25T11:29:51.960" LastActivityDate="2017-01-25T17:43:28.910" Title="How can I numericaly solve a convection-diffusion equation with a large diffusion term?" Tags="|pde|numerical-analysis|advection-diffusion|crank-nicolson|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26017" PostTypeId="2" ParentId="26007" CreationDate="2017-01-25T10:31:43.157" Score="0" Body="&lt;p&gt;You are perturbing the initial system.&#xA;If you wish to solve the system $Au=f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By perturbing the system you solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(A+\Delta A)(u+\Delta u)=f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta A \Delta u= f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta u= \Delta A^{-1}  f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a submultiplicative norm $||$ $||$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|| \Delta u||=||\Delta A^{-1} f||$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$||\Delta u||&amp;lt;||\Delta A^{-1}||$ $|| f||$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The norm can gives the upper limit of the effect of the perturbation.&lt;/p&gt;&#xA;" OwnerUserId="8166" LastEditorUserId="8166" LastEditDate="2017-01-25T14:03:01.027" LastActivityDate="2017-01-25T14:03:01.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26018" PostTypeId="1" AcceptedAnswerId="26024" CreationDate="2017-01-25T12:26:10.980" Score="10" ViewCount="1002" Body="&lt;p&gt;I have implemented an ADER-Discontinuous Galerkin scheme for the resolution of linear systems of conservation laws of the type of $\partial_t U + A \partial_x U + B \partial_y U=0 $ and observed that the CFL condition is very restrictive. In the bibliography, an upper bound for the time step $\Delta t \leq \frac{h}{d(2N+1)\lambda_{max}}$ can be found, where $h$ is the cell size, $d$ is the number of dimensions and $N$ is the max degree of the polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to circumvent this issue? I had been working with WENO-ADER Finite volume schemes and the CFL restrictions were much more relaxed. For instance, for a 5-th order scheme, a CFL lower than 0.04 must be imposed when using DG while CFL=0.4 can still be used in a WENO-ADER FV scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why using DG schemes rather than ADER-FV, for instance, in computational aeroacustics (linearized Euler equations) or similar applications (gas dynamics, shallow water, magnetohydrodynamics)? Is the overall computational cost of the scheme similar than that of the ADER-FV, in spite of the much lower time step?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thoughts and suggestions for this are welcome. &lt;/p&gt;&#xA;" OwnerUserId="22984" LastActivityDate="2023-04-07T13:26:39.330" Title="CFL condition in Discontinuous Galerkin schemes" Tags="|fluid-dynamics|finite-volume|discontinuous-galerkin|numerical-modelling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26019" PostTypeId="2" ParentId="26016" CreationDate="2017-01-25T17:43:28.910" Score="4" Body="&lt;p&gt;The solution you believe to be inaccurate is actually by far the more accurate one; you've simply plotted it in a very deceptive way.  For $\nu=2$, the exact solution is actually no bigger than about $10^{-35}$ everywhere -- it's zero for all intents and purposes.  Therefore the numerical solution is correct to 10 digits -- far better than the accuracy of your solution when $\nu=1/6$.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-01-25T17:43:28.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26020" PostTypeId="1" AcceptedAnswerId="26023" CreationDate="2017-01-25T18:10:03.943" Score="1" ViewCount="115" Body="&lt;h1&gt;Frame of the question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I am currently editing an add-on module to an ocean/circulation model, which is written in Fortran. The code of the main model is quite optimized with respect to short run time (not with respect to readability of the code). There are some code parts in the add-on module that do not seem to be optimized - at least from my point of view.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;At three occasion a temporary variable is introduced to substitute some calculations as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;! comment&#xA;a2 = a * a&#xA;b2 = b * b&#xA;&#xA;! comment&#xA;c = a / (a2 + b2)&#xA;&#xA;! comment&#xA;f = c * d&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This seems reasonable to me if (1) I want to make the code better human-readable or (2) the &lt;code&gt;a * a&lt;/code&gt; is to be substituted several times in the code. However, (1) and (2) are not the case: &lt;code&gt;a2&lt;/code&gt; and &lt;code&gt;b2&lt;/code&gt; are only needed once. In order to reduce the run time this formulation seems to be better:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = d * a / (a * a + b * b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The number of calculation steps remains the same. The declaration of three variables is removed. We loose some explanations in the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that in one previous version of the code &lt;code&gt;a * a&lt;/code&gt; was present several times which is why it was substituted by &lt;code&gt;a2&lt;/code&gt;. However, it was not removed.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Questions&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is the a good reason with respect to &quot;optimal&quot; run time to keep the code as it is posted in the first code snipped?&lt;/li&gt;&#xA;&lt;li&gt;Does a state-of-the-art compiler (i.e. ifort) detect it and remove the unnecessary variables? Thus, both formulations do not make a difference?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22408" LastActivityDate="2017-01-26T00:16:09.710" Title="Declare variable to substitute one calculate only once" Tags="|fortran|performance|numerical-modelling|efficiency|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26021" PostTypeId="1" CreationDate="2017-01-25T18:21:13.350" Score="2" ViewCount="190" Body="&lt;p&gt;I'm not sure that this *overflow is right place to ask.... Sorry if it is off topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know a citation (scientific article or book) for a numerical trick (method), when we tabulate a right-hand side of given differential equation(s) and use an interpolation in between rows of the lookup table when we numerically integrate PDEs/ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This trick speeds up computations but, of course, reduces an accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method is well known, but I couldn't find any references in scientific literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also any estimation of how to select correct number of points in lookup table to reach required accuracy of solution will be very helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I have to clarify the question due to a several answer which aren't answer for my question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have &#xA;$\frac{dx}{dt} = f(x)$, I can create a lookup table of $y_i=f(x_i)$ for some range of $x_i$. This table is in a memory. I can use interpolation between rows in this table instead of evaluation function  $f(x)\approx y_i+\frac{y_{i+1}-y_{i}}{x_{i+1}-x_{i}}(x-x_i)$, which is faster than a direct calculation of $f(x)$.&lt;/p&gt;&#xA;" OwnerUserId="23043" LastEditorUserId="23043" LastEditDate="2017-01-31T19:20:41.073" LastActivityDate="2017-01-31T19:20:41.073" Title="citations for numerical lookup table interpolation of P/ODE(s) RHS" Tags="|pde|ode|interpolation|" AnswerCount="2" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="26022" PostTypeId="1" AcceptedAnswerId="26033" CreationDate="2017-01-25T19:27:03.183" Score="3" ViewCount="6773" Body="&lt;p&gt;I just searched around a bit for BLAS implementations and was amazed by the sheer amount of libraries around. Does someone know of a benchmark or otherwise rating of the various libraries?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How easy they are to install.&lt;/li&gt;&#xA;&lt;li&gt;Performance.&lt;/li&gt;&#xA;&lt;li&gt;How easy they are to get to work with Octave or Matlab.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I realize all of these would depend on OS and hardware and version of the softwares. But any input would be welcome at this point.&lt;/p&gt;&#xA;" OwnerUserId="18579" LastActivityDate="2017-01-27T17:17:06.577" Title="BLAS libraries for Octave or Matlab, preferrably with GPU support?" Tags="|linear-algebra|matlab|reference-request|blas|octave|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26023" PostTypeId="2" ParentId="26020" CreationDate="2017-01-26T00:16:09.710" Score="3" Body="&lt;p&gt;Common subexpression elimination and inlining code are like the bread and butter of compiler optimizations, the first and easiest optimizations that a compiler can do. There is every reason to think the performance of the code will be exactly the same (while losing the comments), and there is no particular reason to think that a compiler would have trouble performing this kind of code transformation if it made sense. This is exactly the sort of thing that should be left to the compiler to decide. This is even more true if the comments were meaningful.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The declaration of three variables is removed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is misguided: since the code will be compiled to assembly, and both variables and subexpressions will be stored in whatever registers the compiler wants to use, there shouldn't be a reason to think that this will affect performance. There are sites like godbolt.org (for C++; &lt;a href=&quot;https://godbolt.org/g/HlSpqa&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;), where you can see just how code looks after compilation: the variables-subexpressions distinction is basically meaningless performance-wise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, unfortunately, optimization tends to be a little bit of an art, especially now that optimizing compilers are powerful and widely available—I don't know what reading to recommend. Based on my understanding, the exact thing you are describing is never going to do anything helpful. So since you want to optimize the code anyway, the first thing you are going to need is &lt;em&gt;not&lt;/em&gt; a list of optimizations to try, but &lt;em&gt;a method&lt;/em&gt; to check if a given optimization does something. I don't know specifically about Fortran, but the first step is &lt;em&gt;always&lt;/em&gt; to get a good profiler that can tell you which parts of your code are actually expensive, and you can use it to measure performance of modified code. It is much too easy to spend a lot of time for no gain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second major observation is that famous principle that the most helpful optimizations are those that &lt;em&gt;avoid&lt;/em&gt; work, rather than those that do the same work but faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really do end up doing these micro-optimizations, one always starts with eliminating memory accesses and ensuring that memory access patterns are cache-friendly. These types of things compilers generally can't handle on their own.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-01-26T00:16:09.710" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26024" PostTypeId="2" ParentId="26018" CreationDate="2017-01-26T05:43:02.617" Score="8" Body="&lt;p&gt;The restrictive CFL of DG schemes typically comes from the combination of high order accuracy and a compact stencil (see &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/060672601&quot;&gt;this reference&lt;/a&gt; for example).  The CFL depends on bounding the variational form in terms of the $L^2$ norm of the solution, which depends on derivative and traces of polynomials.  Bounds for each of these quantities (using Bernstein or Markov brothers inequalities and discrete trace inequalities) give constants which depend inversely on $h$ and quadratically on the order $N$, resulting in an overall CFL of $O(h/N^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYI - I've seen the CFL you mention referenced before, but I can't recall where it's proven. I'd like to know how they avoid quadratic dependence on $N$ in their bound. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite difference and WENO schemes (as well as B-spline based finite element methods on periodic meshes) have looser CFL conditions because the constants in the analogous bounds grow more slowly in $N$. This is in turn because the stencil size tends to increases with order $N$, which reduces some of these issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;DG methods are more expensive, but they can deal easily with unstructured meshes and can be implemented efficiently. There are high order versions of WENO (or similar reconstructions) for unstructured grids, though these can introduce additional mathematical or implementational complications.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2017-01-26T05:43:02.617" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26026" PostTypeId="1" AcceptedAnswerId="26036" CreationDate="2017-01-26T13:09:01.900" Score="0" ViewCount="358" Body="&lt;p&gt;I have just read yet another introduction to image processing, which describes Gaussian smoothing from a convolution perspective and least-squares smoothing from an optimization perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would help me knit this knowledge together if I could imagine how I could also use optimization to obtain a Gaussian-smoothed signal as well, but I never see this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that a least squares smoothing is obtained from&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;argmin_{x} \| Ax-b \|^{2}_2 + \lambda \| \nabla x \|_2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What corresponding penalty function would produce a Gaussian smoothing?&lt;/p&gt;&#xA;" OwnerUserId="3657" LastEditorUserId="3657" LastEditDate="2017-01-26T13:18:23.057" LastActivityDate="2017-01-27T17:56:32.703" Title="What penalty function produces optimization-based Gaussian smoothing?" Tags="|optimization|smoothing|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26027" PostTypeId="2" ParentId="25970" CreationDate="2017-01-26T13:36:02.980" Score="1" Body="&lt;p&gt;If you want to stick to the Python scientific family, you could opt for &lt;a href=&quot;http://www.jmodelica.org/assimulo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Assimulo&lt;/a&gt;. Assimulo is a wrapper around a lot of ODE integrators, providing a common interface. If you happen to be running on Windows, you can find wheels at &lt;a href=&quot;http://www.lfd.uci.edu/~gohlke/pythonlibs/#assimulo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Christoph Gohlke's page&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assimulo allows to handle state events to allow for discontinuities in the RHS function of your ODE but you can use them also to stop the iteration when a certain condition is fulfilled. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The procedure is to define a function 'state_events' that analyzes whether an event took place and a second function 'handle_event' to act accordingly (changing the state equations, changing the variables, stopping the iteration, ...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example in the Assimulo package is a pendulum that hits a wall. When it hits the wall, the state changes in the sense that the direction should be reversed and some energy should be lost. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I modified the example that comes with Assimulo, using a simple exponential decay problem where I asked to stop the integration when the concentration of the species drops below a threshold value. See the code below. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python &#xA;# -*- coding: utf-8 -*-&#xA;import numpy as N&#xA;import pylab as P&#xA;from assimulo.problem import Explicit_Problem&#xA;from assimulo.solvers.sundials import CVode&#xA;from assimulo.exception import TerminateSimulation&#xA;&#xA;&#xA;def run_example():&#xA;&#xA;    def decay(t,y):&#xA;&#xA;        yd_0 = -0.5*y[0]&#xA;&#xA;        return N.array([yd_0])&#xA;&#xA;    def state_events(t,y,sw):&#xA;        &quot;&quot;&quot;&#xA;        This is our function that keep track of our events, when the sign&#xA;        of any of the events has changed, we have an event.&#xA;        &quot;&quot;&quot;&#xA;&#xA;        return N.array([y[0]-50.0])&#xA;&#xA;&#xA;    def handle_event(solver, event_info):&#xA;        &quot;&quot;&quot;&#xA;        Event handling. This functions is called when Assimulo finds an event as&#xA;        specified by the event functions.&#xA;        &quot;&quot;&quot;&#xA;        state_info = event_info[0] #We are only interested in state events info&#xA;&#xA;        if state_info[0] != 0: #Check if the first event function have been triggered&#xA;            raise TerminateSimulation&#xA;&#xA;&#xA;    #Initial values&#xA;    y0 = [100.0] #Initial states&#xA;    t0 = 0.0             #Initial time&#xA;&#xA;    #Create an Assimulo Problem&#xA;    mod = Explicit_Problem(decay, y0, t0)&#xA;&#xA;    mod.state_events = state_events #Sets the state events to the problem&#xA;    mod.handle_event = handle_event #Sets the event handling to the problem&#xA;    mod.name = 'Simple decay'   #Sets the name of the problem&#xA;&#xA;    #Create an Assimulo solver (CVode)&#xA;    sim = CVode(mod)&#xA;&#xA;    #Specifies options &#xA;    sim.discr = 'BDF'     #Sets the discretization method&#xA;    sim.iter = 'Newton' #Sets the iteration method&#xA;    sim.rtol = 1.e-8        #Sets the relative tolerance&#xA;    sim.atol = 1.e-6        #Sets the absolute tolerance&#xA;&#xA;    #Simulation&#xA;    ncp = 200     #Number of communication points&#xA;    tfinal = 10.0 #Final time&#xA;&#xA;    t,y = sim.simulate(tfinal, ncp) #Simulate&#xA;&#xA;    #Print event information&#xA;    sim.print_event_data()&#xA;&#xA;    #Plot&#xA;    P.plot(t,y)&#xA;    P.show()&#xA;&#xA;if __name__=='__main__':&#xA;    run_example()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastActivityDate="2017-01-26T13:36:02.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26028" PostTypeId="2" ParentId="26021" CreationDate="2017-01-26T17:04:48.097" Score="2" Body="&lt;p&gt;I wouldn't say that it is a trick (interpolation is one of the most basic numerical techniques) and I don't see why it would be &quot;dirty&quot; (though it may be inaccurate).  I don't think there is a special name attached to interpolation between solutions of a differential equation.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-01-26T17:04:48.097" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26030" PostTypeId="1" AcceptedAnswerId="26035" CreationDate="2017-01-27T13:12:30.620" Score="1" ViewCount="667" Body="&lt;p&gt;The lsqcurvefit solution in matlab converges at different solutions depending upon the initial guess: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4ntI6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4ntI6.png&quot; alt=&quot;Parameter Space&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Surface represents the error (SSE) between model and data at various combinations of parameters p1 and p2. Red lines represent the path lsqcurvefit takes from initial guess of parameters to the solution marked with red ball.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 95% confidence in the parameter estimates at the solutions do not overlap...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does lsqcurvefit estimate error at each iteration of parameter estimation? Why does lsqcurvefit not find the same solution each time? What approaches should be taken to choose a solution?&lt;/p&gt;&#xA;" OwnerUserId="23069" LastActivityDate="2017-01-27T17:28:40.667" Title="matlab lsqcurvefit parameter estimation journey" Tags="|matlab|error-estimation|curve-fitting|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26033" PostTypeId="2" ParentId="26022" CreationDate="2017-01-27T16:58:41.400" Score="4" Body="&lt;p&gt;Among open-source BLAS, as far as I know, OpenBLAS (&lt;a href=&quot;http://www.openblas.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.openblas.net/&lt;/a&gt;) is the best option. The website has a DGEMM benchmark, comparing against MKL (see below) and the reference Fortran BLAS. The library is threaded and written in C and assembly. For GPUs, there's clBLAS (&lt;a href=&quot;https://github.com/clMathLibraries/clBLAS&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/clMathLibraries/clBLAS&lt;/a&gt;) that implements BLAS using OpenCL. Unfortunately I don't know of benchmarks for clBLAS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If closed-source proprietary BLAS implementations are okay, Intel MKL is a good option for use on multi-core CPUs and Xeon-Phi accelerators. It has more than just BLAS - it also includes many LAPACK functions and FFT, for instance. NVIDIA's CuBLAS is an option for CUDA-enabled GPUs. cuBLAS is compared against MKL here &lt;a href=&quot;http://developer.download.nvidia.com/compute/cuda/6_5/rel/docs/CUDA_6.5_Performance_Report.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://developer.download.nvidia.com/compute/cuda/6_5/rel/docs/CUDA_6.5_Performance_Report.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know about ClBLAS, but the others are pretty easy to install on Linux; the documentation is clear enough. As for interfacing with Octave/MATLAB, I don't know because I don't use them, but hopefully someone else can answer about that.&lt;/p&gt;&#xA;" OwnerUserId="5541" LastActivityDate="2017-01-27T16:58:41.400" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26034" PostTypeId="2" ParentId="26022" CreationDate="2017-01-27T17:17:06.577" Score="3" Body="&lt;p&gt;MATLAB already comes with Intel MKL for its BLAS implementation. There's no reason to replace that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for using GPUs, if you make your array a &lt;code&gt;gpuArray&lt;/code&gt; (to do that, just do &lt;code&gt;gpuArray(A)&lt;/code&gt;), then &lt;a href=&quot;https://www.mathworks.com/help/distcomp/run-built-in-functions-on-a-gpu.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;you can use MATLAB's matrix multiplication and it will use optimized kernals from MAGMA to perform the computation&lt;/a&gt;. You can Google around to reason some people saying this outperforms CUBLAS by like 10%, but the comments are usually old (2013) and blablabla: it's fast enough that it's likely the best option if you're in MATLAB (though if you really want performance, you should look at Julia with CUBLAS, which will have a lower interop overhead and faster user-compiled kernals).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why you won't find a BLAS implementation that &quot;has both&quot; is because the implementations have to be completely different to fully leverage the GPU, and so at that point they might as well be different libraries since the reason to bundle code is usually for some form of code reuse. You can try swapping out backend BLAS implementations in MATLAB to learn, but it likely won't cause a performance change it may give difficulties because it's very undocumented. That's just one problem among many with closed-source software. If you want to be able to modify every detail, swap BLAS libraries and libm implementations, etc., you may want to look into using an open-source software instead, which as above I'd recommend Julia (or you may be able to do this with Octave, though I don't know and will refer to whatever documentation they have).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-01-27T17:17:06.577" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26035" PostTypeId="2" ParentId="26030" CreationDate="2017-01-27T17:28:40.667" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How does lsqcurvefit estimate error at each iteration of parameter estimation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It both checks how far it has moved (i.e. am I not moving very much anymore? Then stop), and the norm of the gradient (minima are when the gradient is zero). I think there's a third tolerance, &quot;function tolerance&quot;, which I don't quite know. It's all &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/lsqcurvefit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here in the docs&lt;/a&gt; (search for tolerance).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why does lsqcurvefit not find the same solution each time?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Levenberg-Marquardt is a local nonlinear least squares optimizer, so it's dependent on the initial condition (hence the term local). The reason is because it uses a Newton method (which is clearly local: it falls to the closest minima) mixed with gradient decent (it always goes downhill, hence going to the local minima). The result it that it finds minima closest to where you start, so it has initial condition dependence.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What approaches should be taken to choose a solution?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You should take a look at global optimization algorithms. MATLAB's Global Optimization Toolbox is okay, but I find the litany of methods provided by Julia's JuMP and Optim.jl to be better suited for most problems. These give you access to a whole range of packages, and you can use the same code to call all of the different solvers. I have tended to have most success with some methods from NLopt, though IPOPT can do well too.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-01-27T17:28:40.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26036" PostTypeId="2" ParentId="26026" CreationDate="2017-01-27T17:56:32.703" Score="2" Body="&lt;p&gt;The least squares optimization problem in your question has a penalty term that only penalizes 1st derivatives. However, for a Gaussian kernel, the corresponding penalty term would have to penalize all derivatives because Gaussians are infinitely smooth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A mathematically precise way to link various penalty terms to smoothing kernel functions would require the theory of reproducing kernel Hilbert spaces (RKHS). The penalty term for a kernel $K$ is the corresponding RKHS norm $|| \cdot ||_K$. For a Gaussian kernel this is given by:&#xA;$$&#xA;||f||_K = \sum_{m=0}^\infty \frac{\sigma^{2m}}{2^m m!} \int_{-\infty}^{\infty} [f^{(m)}(t)]^2 dt&#xA;$$&#xA;where $f^{(m)}$ is the $m^{th}$ derivative and $\sigma$ is the standard deviation of the Gaussian kernel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Page 212 here: &lt;a href=&quot;https://alex.smola.org/papers/2004/SmoSch04.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://alex.smola.org/papers/2004/SmoSch04.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other references:&#xA;&lt;a href=&quot;http://www.mit.edu/~9.520/spring07/Classes/class03_rkhs.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mit.edu/~9.520/spring07/Classes/class03_rkhs.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;https://people.eecs.berkeley.edu/~jordan/courses/281B-spring01/lectures/regularization.ps&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://people.eecs.berkeley.edu/~jordan/courses/281B-spring01/lectures/regularization.ps&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23074" LastActivityDate="2017-01-27T17:56:32.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26038" PostTypeId="1" CreationDate="2017-01-28T00:14:24.053" Score="2" ViewCount="111" Body="&lt;p&gt;I'm considering implementing (just for simplicity) the unconstrained implicit optimization based integration for Material Point Method as described in &lt;a href=&quot;https://www.math.ucla.edu/~jteran/student_thesis/jiang.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chenfanfu Jiang's thesis on MPM&lt;/a&gt; (the minimization algorithm starts on page 101 – &quot;&lt;em&gt;A.3.1 Unconstrained minimization&lt;/em&gt;&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have difficulties understanding how would be the algorithm implemented.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Are the statements below (in)correct?&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The goal of the algorithm is obtaining new grid velocities in response to external and internal forces of the material represented by material points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function to be minimized is $E(\mathbf{v}) = \sum_i\frac{1}{2}m_{\mathbf{i}} \lVert \mathbf v_i - \mathbf {v_i}^n \rVert + \Phi(\mathbf {x_i}^n + \Delta t\mathbf v_i)$. (from Jiang's thesis)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's derivative is $\mathbf h(\mathbf v) = \mathbf M\mathbf v - \Delta t \mathbf f(\mathbf x^n + \Delta t \mathbf v) - \mathbf M\mathbf v^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the thesis I assume the approach to minimizing $E$ is by the Newton method. The Newton method requires inverse of hessian $\mathbf{\mathit{H}}$. However we're in fact interested in $\Delta \mathbf v$ that would minimize (the linearization of) $E$, so we can instead solve $\mathbf{\mathit{H}}\Delta \mathbf v=\mathbf h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When solving the system via Conjugate gradient, we only need to know the product $\mathbf{\mathit{H}}\Delta \mathbf v$, which can be thought of as differential $\delta \mathbf h = \mathbf M\Delta \mathbf v - \Delta t^2 \delta \mathbf f(\mathbf x^n + \Delta t \mathbf v) = \mathbf{\mathit{H}}\Delta \mathbf v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://www.math.ucla.edu/~jteran/papers/SSCTS13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[Stomakhin et al. MPM snow paper]&lt;/a&gt; we know how to compute $\delta\mathbf{f_i}$ (page 6 – &quot;&lt;em&gt;6 Stress-based forces and linearization&lt;/em&gt;&quot;) – i.e. force differential for the grid velocity at node $\mathbf i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My assumptions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When solving $\mathbf{\mathit{H}}\Delta \mathbf v=\mathbf h$ via Conjugate Gradient (step 4. in the algorithm)&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;${\Delta \mathbf v}$ is a column vector consisting of component-wise grid velocity differentials – $[{\Delta v}_{1x}, {\Delta v}_{1y}, {\Delta  v}_{1z},\,\,\, {\Delta v}_{2x}, {\Delta  v}_{2y}, {\Delta v}_{2z}, \,\,\,\ldots]$ (length of $\Delta \mathbf v$ is therefore $3n$, where $n$ is the number of grid nodes (containing velocity))&lt;/li&gt;&#xA;&lt;li&gt;When multiplying $\mathbf{\mathit{H}}\Delta \mathbf v$, we're in fact always processing 3 rows at the same time - i.e. we process one grid node at time (which consists of x,y,z velocity components).&lt;/li&gt;&#xA;&lt;li&gt;When computing residual $\mathbf r = \mathbf h - \mathbf{\mathit{H}}\Delta \mathbf v$, we can compute its consecutive 3 components corresponding to grid node $\mathbf i$ (like in assumption 1.1.) by writing $\mathbf{r_i} = \left(m_{\mathbf i}\mathbf v_{\mathbf i} - \Delta t \mathbf f_{\mathbf i}(\mathbf x^n_{\mathbf i} + \Delta t \mathbf v_{\mathbf i}) - m_{\mathbf i}\mathbf {v_{\mathbf i}}^n\right) - \left(m_{\mathbf i}\Delta\mathbf v_{\mathbf i} - \Delta t^2 \delta \mathbf {f_i}(\mathbf x^n_{\mathbf i} + \Delta t \mathbf v_{\mathbf i})\right)$.&lt;/li&gt;&#xA;&lt;li&gt;When computing $\mathbf r \cdot \mathbf r$, I ignore the fact that each 3 consecutive elements of vector $\mathbf r$ correspond to 1 grid node, and just compute the dot product like with any other vector.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is the line search (step 7 of the algorithm) necessary?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21183" LastEditorUserId="21183" LastEditDate="2017-01-28T00:26:04.710" LastActivityDate="2017-01-28T00:26:04.710" Title="Optimization based integration for MPM" Tags="|optimization|algorithms|linear-solver|newton-method|conjugate-gradient|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26039" PostTypeId="2" ParentId="3202" CreationDate="2017-01-28T00:49:54.143" Score="1" Body="&lt;p&gt;Numerical recipes in c provides closed form expression for real roots of quadratic and cubic which presumably have decent precision. Since the algebraic solution of the quartic involves solving a cubic and then solving two quadratics maybe a closed form quartic w good precision isn't out of the question. &lt;/p&gt;&#xA;" OwnerUserId="23076" LastActivityDate="2017-01-28T00:49:54.143" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26042" PostTypeId="1" AcceptedAnswerId="26077" CreationDate="2017-01-30T02:14:57.970" Score="4" ViewCount="565" Body="&lt;p&gt;In all the sources I have that discuss the elasticity equations, they start by saying that the strain tensor, $\epsilon$, is related to the displacement gradient, $\nabla u$, where $u$ is the (vector valued) displacement as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\epsilon_{ij} = \frac{1}{2}\left(\frac{\partial u_j}{\partial x_i} + \frac{\partial u_i}{\partial x_j}  + \frac{\partial u_k}{\partial x_i}  \frac{\partial u_k}{\partial x_j}\right).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then they say something like &quot;if $\left\|u\right\|$ is small and $\left\|\nabla u\right\|$ is small we ignore the third term in the parentheses and we get the infinitesimal strain elasticity equations...&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*}&#xA;\sigma_{kl} &amp;amp;= C_{ijkl}\frac{1}{2}\left(\frac{\partial u_j}{\partial x_i} + \frac{\partial u_i}{\partial x_j}\right)\\ &#xA;-g_i &amp;amp;= \frac{\partial}{\partial x_j}\sigma_{ij}&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: I understand why $\left\|\nabla u\right\|$ has to be small, but where in the theory does it require that $u$ is small?  I can't find where that is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reason for the question: If we consider a cantilevered shaft and apply a torque at the unsupported end, the resulting displacement isn't necessarily small compared to shaft's cross section.  Does this mean that this very simple case has to be analyzed with the finite strain model?&lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="9667" LastEditDate="2017-03-05T18:24:22.687" LastActivityDate="2020-06-11T20:06:39.483" Title="Why does the displacement have to be small to use the infintesimal strain elasticity equations?" Tags="|finite-element|pde|solid-mechanics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26043" PostTypeId="1" CreationDate="2017-01-30T14:18:36.407" Score="0" ViewCount="178" Body="&lt;p&gt;I'm trying to solve the phase problem from single intensity measurement. I implemented &lt;code&gt;error reduction algorithm&lt;/code&gt; in &lt;code&gt;Matlab&lt;/code&gt; for my problem but it does not give any result. Is there anybody uses this algorithm for phase problem?&lt;/p&gt;&#xA;" OwnerUserId="23095" LastActivityDate="2017-02-04T16:19:53.713" Title="Error Reduction algorithm for single measurement intensity" Tags="|matlab|algorithms|" AnswerCount="1" CommentCount="1" ClosedDate="2017-02-06T16:43:27.883" ContentLicense="CC BY-SA 3.0" />
  <row Id="26045" PostTypeId="2" ParentId="23562" CreationDate="2017-01-30T16:06:19.243" Score="0" Body="&lt;p&gt;No, it is not always necessary. The so-called central schemes (like &lt;a href=&quot;https://en.wikipedia.org/wiki/MUSCL_scheme#Kurganov_and_Tadmor_central_scheme&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/MUSCL_scheme#Kurganov_and_Tadmor_central_scheme&lt;/a&gt;) don't use Riemann solvers. They keyword to search is &lt;em&gt;Riemann solver free&lt;/em&gt; schemes.&lt;/p&gt;&#xA;" OwnerUserId="1244" LastActivityDate="2017-01-30T16:06:19.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26046" PostTypeId="2" ParentId="26021" CreationDate="2017-01-30T19:02:08.540" Score="1" Body="&lt;p&gt;If I understand the question, the name I have always known for this kind of tabular integration is &quot;numerical quadrature&quot;.  In the days of calculating machines this was the only way of going about solving differential equations.  It was important to minimise the number of points at which you have to calculate the right hand side of $y' = f(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One very effective method of achieving this was referred to as &quot;&lt;em&gt;Richardson's deferred approach to the limit&lt;/em&gt;&quot; in which you can choose two different step sizes, $h_1$ and $h_2$, to define points at which you evaluate $f(x)$.  If using these two step sizes results in estimates $f_1$ and $f_2$ for the value at a specific point, you can combine these to produce an answer which is better than if you had picked a finer resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Famously, if $h_1 = 2 h_2$, then $f = f_2 + (f_2 - f_1)/3$ is $O(h^3)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In effect, a &quot;free lunch&quot;. If the accuracy was still not good enough, you would choose a smaller $h_3$, and so on.  There were many papers on the choice of sequence $\{h_i\}$.  The method was also adapted for extrapolation and for summing slowly convergent series. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was first discussed by Richardson and Gaunt, 1926, &lt;em&gt;Phil. Trans. A&lt;/em&gt;, &lt;strong&gt;226&lt;/strong&gt;, 299-361.  When solving ODE's we refer to this as Richardson Extrapolation, see Press et al. &lt;em&gt;Numerical Recipes&lt;/em&gt; Section 16.4 et. seq. in the context of the important Bulirsch-Stoer method for solving ODE's.&lt;/p&gt;&#xA;" OwnerUserId="21129" LastActivityDate="2017-01-30T19:02:08.540" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26047" PostTypeId="2" ParentId="23562" CreationDate="2017-01-30T19:47:29.383" Score="2" Body="&lt;p&gt;When you say &quot;characteristic decomposition&quot;, I take it to mean finding the eigenvalues and eigenvectors of the Euler flux Jacobian. This is needed in approximate Riemann solvers such as that of Roe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are several Riemann solvers that do not explicitly need the eigen-decomposition of the flux Jacobian. These include Van Leer's flux vector splitting, the AUSM family of schemes and several others. Some 1D versions are, for instance, described here in brief: &lt;a href=&quot;http://chimeracfd.com/programming/gryphon/fluxmain.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://chimeracfd.com/programming/gryphon/fluxmain.html&lt;/a&gt;. Many newer flux-vector splitting schemes, such as AUSM+, are known to resolve shocks etc very well, provided you use a good limiter. Since you're using WENO, you should be fine. You might need to adjust the WENO parameters a bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 2D and 3D problems, there is hardly any difference in the Riemann solver itself. Essentially the 1D Riemann solver is applied in the direction normal to the face between two cells. For example, if the 1D scheme requires the left cell Mach number $ M_L = u_L/c_L$, the 2D scheme needs the left cell face-normal Mach number $M_{nL} = \vec{u}_L \cdot \vec{n} / c_L$ where $\vec{n}$ is the normal to the face across which flux is being computed. The mass flux, instead of being $f_{mass} = \rho u$, becomes $f_{mass} = \rho \vec{u}\cdot\vec{n}$, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this with any of the common Riemann solvers (Roe, HLLC etc), not just the ones mentioned before. See the last section in &lt;a href=&quot;https://web.stanford.edu/group/frg/course_work/AA214B/CA-AA214B-Ch7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/group/frg/course_work/AA214B/CA-AA214B-Ch7.pdf&lt;/a&gt; , for instance. They are also described in some books such as J. Blazek, &quot;Computational Fluid Dynamics: Principles and Applications&quot;.&lt;/p&gt;&#xA;" OwnerUserId="5541" LastActivityDate="2017-01-30T19:47:29.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26048" PostTypeId="1" CreationDate="2017-01-30T20:52:43.087" Score="2" ViewCount="267" Body="&lt;p&gt;I am searching for a good algorithm to solve problem 487 on project-euler. I dont want code or something like that, I only want the name of the algorithm thats best suited. Till now I think of either using Faulhabers Formula or using the stirling numbers of second kind to compute the sum. &#xA;&lt;a href=&quot;https://projecteuler.net/problem=487&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://projecteuler.net/problem=487&lt;/a&gt;&#xA;If there is a better algorithm please let me know.&lt;/p&gt;&#xA;" OwnerUserId="23101" LastActivityDate="2017-01-30T20:52:43.087" Title="sum of n numbers with the power k" Tags="|algorithms|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26049" PostTypeId="1" CreationDate="2017-01-31T02:27:29.910" Score="0" ViewCount="333" Body="&lt;p&gt;I have quadratic finite element - shape function  is quadratic. Element spans from 0 to 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Body force is given by (in physical coordinates)&#xA;$$f_b = \int_0^5 N(x)^T b(x) dx \approx \sum_{i=1}^3 N(\xi_i)b_i(x)w_i $$ &#xA;The last term is obtained from Gauss quadrature. $\xi_i$ are Gauss points and $w_i$ are corresponding weights. Say, $b=x^3$&#xA;$$&#xA;N_1(x) = ((x-x_2)(x-x_3))/( (x_1-x_2)  (x_1-x_3) ) \\&#xA;N_2 (x)= ((x-x_1)(x-x_3))/( (x_2-x_1)  (x_2-x_3) ) \\&#xA;N_3 (x)= ((x-x_1)(x-x_2))/( (x_3-x_1)  (x_3-x_2) )&#xA;$$&#xA;When I transform the problem into isoparametric coordinates, I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_b = \sum_{i=1}^3 N(\xi_i)b_i(x)w_iJ$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA; N_1 = -0.5 \xi (1-\xi)\\&#xA;  N_2 = (1-\xi^2)\\&#xA;  N_3 = 0.5\xi(1+\xi)\\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is when I evaluate integrals - in physical and isoparametric coordinates - they do not match. I suspect that I am using $b(x)=x^3$ in isoparametric without converting it. But, even if I multiply $b$ by $J$, problem does not solve. I do not have much idea what is going wrong. Can someone help me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;As per comments, I checked that when $b$ is a constant, $f_b$ in both coordinates match. So problem comes when $b$ is a function of $x$. I am wondering how to convert $b$ from physical to isoparametric. To be specific, say $b=x^3$. How can I transform and integrate this in isoparametric coordinates?&lt;/p&gt;&#xA;" OwnerUserId="23051" LastEditorUserId="23051" LastEditDate="2017-02-01T02:18:20.233" LastActivityDate="2017-02-05T13:43:59.193" Title="Finite element method applied to 1D structural problem - what is wrong with body force?" Tags="|finite-element|pde|integration|" AnswerCount="2" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26050" PostTypeId="1" AcceptedAnswerId="26051" CreationDate="2017-01-31T10:35:19.043" Score="1" ViewCount="213" Body="&lt;p&gt;I am trying to constrain an eigenvalue problem. I am aware of the method utilizing the nullspace of the constraint vectors but I was wondering if it would be to use a penalty method for the same purpose. Trying the following at MATLAB/Octave&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r = rand(4,4);&#xA;r=r' + r;&#xA;p=[1, 0,  0 , -1]';&#xA;w=10000;&#xA;[V, D] = eig(r + w*p*p');&#xA;V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;does not enforce $p={[1 , 0, 0,-1}]$ for all eigenvectors (components 1 and 4 of the system  should have the same value). There seems to be always 1 eigenvector where the 1st and last vector component are the same in value and have opposite signs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any obvious reason why this is happening and I miss it? Sorry if this is obvious, my linear algebra is a bit rusty.&lt;/p&gt;&#xA;" OwnerUserId="22986" LastEditorUserId="9667" LastEditDate="2017-01-31T12:56:24.023" LastActivityDate="2017-01-31T12:58:18.030" Title="Eigenvalue problem constrained with a penalty method" Tags="|linear-algebra|eigenvalues|constraints|penalty-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26051" PostTypeId="2" ParentId="26050" CreationDate="2017-01-31T11:16:12.630" Score="1" Body="&lt;p&gt;I found the error, I just need to constrain the implied identity matrix on the right-hand-side too:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r = rand(4, 4);&#xA;r = r' + r;&#xA;p=[1 0 0  -1]';&#xA;M = eye(4);&#xA;w = 1000; &#xA;[V,D ] = eig(r + w*p*p', M + w*p*p');&#xA;V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22986" LastEditorUserId="9667" LastEditDate="2017-01-31T12:58:18.030" LastActivityDate="2017-01-31T12:58:18.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26053" PostTypeId="2" ParentId="26049" CreationDate="2017-01-31T13:40:46.873" Score="4" Body="&lt;p&gt;Instead of writing $b_i(x)$, I suggest writing it as $b(x(\xi_i))$. This is more indicative of how to evaluate your integral.&#xA;$x(\xi_i)$ is the value of $x$ at the ith integration point which you can compute from the simple relationship between&#xA;$x$ and $\xi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x = \sum_{i=1}^3 N_i(\xi) x_i $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the defintion of the mapping from real to parametric coordinates assumed in an isoparametric element formulation.&#xA;For each integration point, $\xi_i$, you evaluate the shape functions at that point and compute the sum above to determine $x$ of the point.&#xA;Then simply evaluate your $b$ function at $x$ computed from this expression.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-02-01T12:16:32.147" LastActivityDate="2017-02-01T12:16:32.147" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26054" PostTypeId="2" ParentId="23562" CreationDate="2017-01-31T14:26:45.390" Score="1" Body="&lt;p&gt;You will get better results with characteristic decomposition. This brings in upwind character to your scheme which is most crucial thing for hyperbolic problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use an upwind flux like Riemann solver based, then upwinding is already built into the scheme. So doing WENO on conserved variables may be ok. But if you are using a central flux like Lax-Friedrich, then it is better to do WENO reconstruction of characteristic variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;WENO is a sort of limiter which is trying to enforce some maximum principle or monotonicity condition. Conserved variables dont have a maximum principle but characteristic variables do, at least in a local, linearized sense. So I would recommend applying WENO on characteristic variables all the time, if you dont mind the additional expense. Or atleast have both options in your code, so you can try out both and see which one works best for your problem.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2017-01-31T14:26:45.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26055" PostTypeId="1" CreationDate="2017-01-31T15:27:18.303" Score="9" ViewCount="2286" Body="&lt;p&gt;I'm a bit confused about the concepts of convergence rate and convergence order. Let me first give you the definitions we use: [sorry for the English, it's all self translated]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $x^{*}$ be our solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Definition 1:&lt;/strong&gt; The sequence $x^{(k)}$ is called linearly converging towards $x^{*}$, if&#xA;$$\exists L&amp;lt;1 \text{ so that } \|x^{(k+1)}-x^{*}\|\leq L\| x^{(k)}-x^{*}\|,\quad \forall k\geq k_0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We call the constant $L$ rate of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Definition 2:&lt;/strong&gt; The order of convergence of a numerical method is $p$, if:&#xA;$$\exists C &amp;gt; 0 \text{ so that }\|x^{(k+1)}-x^{*}\|\leq C\| x^{(k)}-x^{*}\|^p,\quad \forall k\in \mathbb N \quad\text{with $C&amp;lt;1$ for $p=1$}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: We assumed, that chose the starting value so that we get an converging sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Question 1:&lt;/em&gt; Can someone explain me the difference between $C$ and $L$ here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Question 2:&lt;/em&gt; Can someone explain me the concept/idea behind the rate/order of convergence? [Just so I heard it from another perspective]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Question 3:&lt;/em&gt; Also, I often see that we use linglog and loglog plots, but I don't really get why we do that. E.g., if we have linear convergence, we can see a linear function if we lin-log plot our errors. (Why do we need the lin-log plot here)&lt;/p&gt;&#xA;" OwnerUserId="22921" LastEditorUserId="713" LastEditDate="2017-01-31T17:57:53.923" LastActivityDate="2017-01-31T18:22:15.340" Title="Convergence rate vs convergence order" Tags="|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26056" PostTypeId="2" ParentId="26055" CreationDate="2017-01-31T16:27:42.500" Score="11" Body="&lt;p&gt;$\|x^{(k)}-x^*\|$ is the error in the $k$th term, call it $E_k$. For a &quot;good&quot; numerical method, we want the approximation to get closer and closer to the desired result so $E_k$ has to decrease to zero. If the error is guaranteed to reduce to at least a certain fraction $L$ of the previous step, you have linear convergence: $$E_{k+1} \le L E_k.$$ This essentially guarantees that the error drops at least as fast as a geometric series. An example would be the sequence $0.9, 0.99, 0.999 \ldots$ which converges to 1. The error at the $k$th step is $10^{-k}$ so that $L=0.1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the error could fall faster than linear. Consider the sequence $$0.9, 0.99, 0.9999, 0.99999999\ldots$$ This converges to 1 too, but at each step the error reduces as the square of the previous step so that $p=2$. When you have a faster than linear convergence order ($p&amp;gt;1$), you are not really worried about $C$: it could even be greater than 1. But when $p=1$, $C$ becomes the same as $L$ and you want it to be less than 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now why the log-linear plot? Assuming linear convergence with geometrically reducing error, we would have $$E_k = L^{-k} E_0$$&#xA;replacing $\le$ with equality. Taking the log of this, $$\log(E_k) = \log(E_0) - k \log(L)$$&#xA;which is a linear curve.&lt;/p&gt;&#xA;" OwnerUserId="22802" LastEditorUserId="713" LastEditDate="2017-01-31T18:22:15.340" LastActivityDate="2017-01-31T18:22:15.340" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26058" PostTypeId="1" CreationDate="2017-02-01T03:01:51.283" Score="1" ViewCount="100" Body="&lt;p&gt;I don't fully understand the subsection &lt;em&gt;3.2 Constrained minimization&lt;/em&gt; of &lt;a href=&quot;https://www.math.ucla.edu/~jteran/papers/GSSJT15.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. In particular, I don't understand the first step &quot;&lt;strong&gt;Register active set&lt;/strong&gt;&quot; and the definition of projection $P(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simplified algorithm (stripped-down version of the paper's one):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Select initial guess and project it: $x^{(0)} = P(x'^{(0)})$&lt;/li&gt;&#xA;&lt;li&gt;Loop until precise enough&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Register active set&lt;/li&gt;&#xA;&lt;li&gt;Solve $\nabla^2E \cdot \Delta v = \nabla E$ via Conjugate Gradient to obtain $\Delta v$&lt;/li&gt;&#xA;&lt;li&gt;Use line search to find step $\alpha$&lt;/li&gt;&#xA;&lt;li&gt;Take step: $v^{(n+1)} = v^{(n)} + \alpha\Delta v$&lt;/li&gt;&#xA;&lt;li&gt;Project $v^{(n+1)}$&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Projections.&lt;/em&gt; Let $P(x)$ be the projection that applies $P_{bp}$ to&#xA;  $x_p$ &lt;strong&gt;for all body-particle pairs $(b, p)$ that are labeled as active&lt;/strong&gt;&#xA;  or are violated ($\phi_b(x_p) &amp;lt; 0$).&#xA;  Note that pairs such that&#xA;  $\phi_b(x_p) = 0$ (as would be the case once projected) are considered&#xA;  to be touching but not violated. The iterates $x^{(i)}$&#xA;  obtained at the end of each Newton step, as well as the&#xA;  initial guess, are projected with $P$.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&#xA;  We begin by representing our collision objects (indexed&#xA;  with $b$) by a level set, which we denote $\phi_b$ to avoid confusion&#xA;  with potential energy. By convention, $\phi_b(x) &amp;lt; 0$ for points $x$&#xA;  in the interior of the collision object $b$. Our collision constraint&#xA;  is simply that $\phi_b\left(x_p^{(n+1)}\right) \gt 0$ for each simulation&#xA;  particle $p$ and every constraint collision object $b$. With such a&#xA;  formulation, we can project a particle at $x_p$ to the closest&#xA;  point $x'_p$ on the constraint manifold using $x'_p = P_{bp}(x_p) = x_p - \phi_b(x_p)\cdot\nabla\phi_b(x_p)$. ...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;em&gt;Register active set.&lt;/em&gt; Let $E'$ be the objective that would be computed in the unconstrained case. The objective function for constrained optimization is $E(x) = E'(P(x))$.&lt;br&gt;&#xA;  Compute the gradient $\nabla E'$. &lt;strong&gt;Constraints that are touching ($\phi(x_p) = 0$) and for which $\nabla E'\cdot\nabla\phi \ge 0$ are labeled as active for the remainder of the Newton step.&lt;/strong&gt; All others are labeled as inactive. No constraint should be violated at this stage. Note that $E'(x^{(i)}) = E(x^{(i)})$ is true before and after every Newton&#xA;  step, since constraints are never violated there.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The emphasized sections are the ones I don't understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the above I assume that the projection function $P(x)$ is defined as&#xA;  $$\begin{align}&#xA;\phi_b \left( x_i \right) &amp;amp;= \text{the shortest distance from point $x_i$ to the surface/boundary of $b$ } \\&#xA;P \left(x_i\right) &amp;amp;=&#xA;\begin{cases}&#xA;  x_i - \phi_b \left( x_i \right) \cdot \nabla\phi_b \left( x_i \right)&#xA;    &amp;amp; \,\,\, \text{if $\phi_b \left( x_i \right) &amp;lt; 0$ or when particle $i$ is active} \\&#xA;   x_i&#xA;    &amp;amp; \,\,\, \text{otherwise}\\&#xA;\end{cases}&#xA;\end{align}&#xA;  $$&#xA;  As can be seen from the algorithm, and from the fact $E(x) = E'(P(x))$, the projection function has tangible effect only on such particles that are &quot;active&quot; (since no constraint is violated inside of the loop). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the purpose of labeling a pair $(b, p)$ as &quot;active&quot;? And why are labeled only those pairs for which $\nabla E'\cdot\nabla\phi \ge 0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I do currently understand this as selecting those particles $p$ that violated the constraints and were thus projected on the level-set manifold (or just happened to be at the precise boundary of the manifold) &lt;strong&gt;and&lt;/strong&gt; at the same time the direction of force (represented by $-\nabla E'$) is trying to push the particle outside the manifold ($\nabla\phi$ is the surface normal of the level-set pointing inside the manifold at the position of particle $p$).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I understand it correctly that the projection function has effect &lt;strong&gt;only&lt;/strong&gt; on the derivatives of the objective function $E'(x^{(i)}) = E(x^{(i)})$ (because otherwise it degrades to identity)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why exactly is $P = x_i - \phi_b \left( x_i \right) \cdot \nabla\phi_b \left( x_i \right)$ for the &quot;active&quot; particles? By definition, for the &quot;active&quot; particles $\phi_b \left( x_i \right) = 0$, so the value of $P$ is $x_i$ – the same as for the inactive particles.&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="21183" LastActivityDate="2017-02-01T03:01:51.283" Title="How does this Constrained Minimization algorithm work?" Tags="|optimization|algorithms|constrained-optimization|projection|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26059" PostTypeId="1" CreationDate="2017-02-01T09:33:07.737" Score="2" ViewCount="2688" Body="&lt;p&gt;this is a follow up question to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26055/convergence-rate-vs-convergence-order/26056?noredirect=1#comment44667_26056&quot;&gt;Convergence rate vs convergence order&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess the whole confusion about how rate vs. order of convergence also came from the implementations I saw.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for fixpoint iteration, I saw this: [was part ofa fixpoint iteration]&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Function to calculate order of convergence  &#xA;def konvord(x,e):&#xA;    p = np.log(e[2:]/e[1:-1]) / np.log(e[1:-1]/e[:-2])&#xA;    mw = p.sum()/len(p)&#xA;    return mw #&#xA;&#xA;#Function to calculate rate of convergence (for linear convergence)&#xA;def konvrate(x,e):&#xA;    n = len(e)&#xA;    k = np.arange(0,n) #array mit 0,1,2,3,...,n-1 (Iterationsschritte)&#xA;    fit = np.polyfit(k,np.log(e),1)&#xA;    L = np.exp(fit[0])&#xA;    return L&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But then I look at code which does some quadratur and there we calculate the order of convergence using this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p = - polyfit(log(n), log(err), 1)[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These implementations are confusing me. I think, that the last one makes sense. But I can't really see, how konvord() gives us an order of convergence and how konvrate() gives us a rate of convergence. To me, konvord() rather looks like it gives the rate of convergence for $p=1$. The function konvrate() should probably be to get an exact value of the order of convergence, but it only works for $p$ very close to $1$ [hence only the error is in a log in the polyfit]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What don't I get? Can someone explain me all three implementations please? What's their idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I tred to play around with it using a fixpointiteration: &lt;a href=&quot;https://hastebin.com/zevuhiqaya.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hastebin.com/zevuhiqaya.py&lt;/a&gt; (with output) and all three give different results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I also wanted to say, how I would do it, ignoring all the above:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the order of convergence, I have $p\approx\frac{\log(e_{k+1})-\log(e_{k})}{\log(e_k)-\log(e_{k-1})}$ I'd implement this using polyfit. I'd get a linear graph and it's slope would be my $p$.(This is the last from the above implementations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For getting the rate of convergence for $p=1$ I'd just do the ratio test. I think, that's what he tried at konvrate(). For an arbritary $p$ I'd first calculate p and then basically do the ratio test again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 1: Aren't konvord() and the last implementation the same? If so, why do they result in different results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 2: Is konvrate() correctly implemented?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 3: I'm sure, that the function konvord() and the last implementation should result in the same result. What exactly should n be there? I think I didn't quiet get the idea behind it.&lt;/p&gt;&#xA;" OwnerUserId="22921" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-02-01T13:22:32.480" Title="Computing rate and order of convergence" Tags="|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26060" PostTypeId="1" CreationDate="2017-02-01T10:03:23.987" Score="2" ViewCount="140" Body="&lt;p&gt;I'm looking for the SCAT (Symbolic convex analysis toolkit) for Maple written by Hamilton and Borwein.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen some references in the specialized literature than mention some urls. However, the links are dead.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Borwein, J.M. &amp;amp; Hamilton, C.H.: &lt;a href=&quot;http://link.springer.com/article/10.1007/s10107-007-0134-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symbolic Fenchel Conjugation&lt;/a&gt;.  Math. Program. (2009) 116: 17. doi:10.1007/s10107-007-0134-4&lt;/li&gt;&#xA;&lt;li&gt;Hamilton, C.H.: &lt;a href=&quot;http://docserver.carma.newcastle.edu.au/296/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symbolic convex analysis&lt;/a&gt;. Masters thesis, Simon Fraser University, (2005)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9749" LastEditorUserId="9667" LastEditDate="2017-03-28T05:05:55.550" LastActivityDate="2017-03-28T05:05:55.550" Title="Symbolic convex analysis toolkit for Maple" Tags="|symbolic-computation|maple|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26061" PostTypeId="2" ParentId="26059" CreationDate="2017-02-01T13:22:32.480" Score="3" Body="&lt;p&gt;Assuming that the error progression is exactly of the form&#xA;$$e_{k+1} = C e_k^p,$$&#xA;We have&#xA;$$e_{k+2} = C e_{k+1}^p$$&#xA;$$\frac{e_{k+2}}{e_{k+1}} = \left(\frac{e_{k+1}}{e_{k}}\right)^p$$&#xA;$$\log\frac{e_{k+2}}{e_{k+1}} = p\log\frac{e_{k+1}}{e_{k}}$$&#xA;$$\frac{\log\frac{e_{k+2}}{e_{k+1}}}{\log\frac{e_{k+1}}{e_{k}}} = p$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what &lt;code&gt;konvord()&lt;/code&gt; finds. By averaging over all possible values of $k$ from $0$ to $k_{max}-2$, the function attempts to average out inexact behaviour at low or high $k$. If there is too much offset at either end, the averaging might not work and the log-log fit might give a better result as you propose. Perhaps it is better to find a median value of p instead of the average in &lt;code&gt;mw&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;code&gt;konvrate()&lt;/code&gt;, assume a linear convergence&#xA;$$e_{k+1} = Le_k$$&#xA;$$e_{k} = L^k e_0$$&#xA;$$\log e_k = \log e_0 - k \log L$$&#xA;Fitting $\log e_k$ vs $k$ gives $\log L$ as the slope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think you can find $p$ from a log-log plot. $p$ is more like a proportionality between $\log\log e_k$ and $k$ rather than between $\log e_k$ and $\log k$.&lt;/p&gt;&#xA;" OwnerUserId="22802" LastActivityDate="2017-02-01T13:22:32.480" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26062" PostTypeId="1" CreationDate="2017-02-01T13:27:39.617" Score="2" ViewCount="2328" Body="&lt;p&gt;i have to decide if the following differential equation is stiff:&#xA;$$y''(t)=-201y'-200y^2 + 2, \quad t\in[0,20].$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sadly, I don't have any solutions. So, what I did was implementing explicit and implicit euler and look at the result for various step sizes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import *&#xA;from scipy import optimize&#xA;&#xA;rhs = lambda z: array([ z[1], -201*z[1] - 200*z[0]**2 + 2 ])&#xA;&#xA;def explicit_euler(y0,t0,tEnd,N):&#xA;    y = zeros((2,N+1))&#xA;    t = zeros(N+1)&#xA;&#xA;    y[:,0] = y0&#xA;    t[0] = t0&#xA;&#xA;    h = float(tEnd-t0)/N&#xA;&#xA;    for k in xrange(N):&#xA;        y[:,k+1] = y[:,k] + h*rhs(y[:,k])&#xA;        t[k+1] = t[k] + h&#xA;&#xA;    return y, t&#xA;&#xA;&#xA;def implicit_euler(y0,t0,tEnd,N):&#xA;    y = zeros((2,N+1))&#xA;    t = zeros(N+1)&#xA;&#xA;    y[:,0] = y0&#xA;    t[0] = t0&#xA;&#xA;    h = float(tEnd-t0)/N&#xA;&#xA;    for k in xrange(N):&#xA;        F = lambda a: -a + y[:,k] + h*rhs(0.5*(y[:,k]+a))&#xA;        startvalue = y[:,k] + h*rhs(y[:,k])&#xA;        y[:,k+1] = optimize.fsolve(F, startvalue)&#xA;        t[k+1] = t[k] + h&#xA;&#xA;    return y, t&#xA;&#xA;&#xA;t0 = 0.&#xA;tEnd = 20.&#xA;y0 = array([1,0])&#xA;N = 100&#xA;&#xA;&#xA;ye, te = explicit_euler(y0,t0,tEnd,N)&#xA;print ye[0]&#xA;yi, ti= implicit_euler(y0,t0,tEnd,N)&#xA;print yi[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For $N=100$ steps, like in the code above, the result output &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;is:&#xA;&#xA;[  1.00000000e+000   1.00000000e+000  -6.92000000e+000   2.95624000e+002&#xA;  -1.19471120e+004  -2.31180176e+005  -1.13350513e+009  -3.84263356e+011&#xA;  ...&#xA;               nan               nan               nan               nan&#xA;               nan               nan               nan               nan&#xA;               nan]&#xA;[ 1.          0.84122674  0.7135129   0.63248909  0.55673461  0.50831417&#xA;  0.45792912  0.42604819  0.39009643  0.36765118  0.34076357  0.32415616&#xA;  ....&#xA;  0.10461852  0.10443526  0.10425726  0.1040885   0.10392479  0.10376937&#xA;  0.10361875  0.10347557  0.10333695  0.10320504  0.10307743]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So we see, that the explicit euler diverges. (all the nans) but the implicit doesn't. By setting $N=100000$ we also see that both get the same result. [We expect a stiff differential equation to be solvable using an explicit method when the stepsize is small enough]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that seems a little to easy here. Would you say that's fine? How would you do it? &lt;/p&gt;&#xA;" OwnerUserId="22921" LastEditorUserId="713" LastEditDate="2017-02-01T17:07:23.910" LastActivityDate="2017-02-01T22:25:47.280" Title="Checking if a given differential equation is stiff" Tags="|stiffness|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26063" PostTypeId="1" AcceptedAnswerId="26068" CreationDate="2017-02-01T16:05:36.067" Score="0" ViewCount="201" Body="&lt;p&gt;I'm writing a code in C++ to parse abaqus/calculix input file for 2D plane stress problems. I'm not a user of abaqus/calculix but I noticed that the input file doesn't ask complete boundary details. If there is a loading/displacement boundary condition then we need to specify the corresponding boundary element otherwise no need to specify boundary element/nodes. How can we recognize boundary elements and edges from a mere element node connectivity data? &lt;/p&gt;&#xA;" OwnerUserId="21659" LastActivityDate="2017-02-01T22:29:28.617" Title="How to recognize boundary nodes and sides from the given element node connectivity data" Tags="|finite-element|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26064" PostTypeId="1" AcceptedAnswerId="26094" CreationDate="2017-02-01T16:43:04.680" Score="3" ViewCount="1054" Body="&lt;p&gt;I am new to the field of computational physics and have a couple of questions regarding solving the non-linear Schrödinger equation using Operator splitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) If the hamiltonian is of the form $H=\frac{\partial^{2}}{\partial x^{2}}+\gamma|\psi|^{2}$ then the standard procedure I understand is to exponentiate $-i(\gamma |\psi|^{2})\Delta t/\hbar$ and operate it on the initial value of $\psi$, then take a fourier transform to convert it to momentum space and operate it with exponential of $-ip^{2}\Delta t/\hbar$ and convert the resultant back to position space. We repeat this for each time interval $\Delta t$.&#xA;Instead, why can't we do everything in the momentum space to begin with? Why this back and forth shifting from position to momentum space?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Suppose now I have an additional term of $\frac{\partial^{2}}{\partial x^{2}}|\psi|^{2}$ in the Hamiltonian, then how do I accomodate this term in the scheme of split operator method?&lt;/p&gt;&#xA;" OwnerUserId="23125" LastEditorUserId="23125" LastEditDate="2017-02-02T05:17:04.250" LastActivityDate="2017-02-04T22:03:57.297" Title="Split operator method" Tags="|nonlinear-equations|fourier-analysis|operator-splitting|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26065" PostTypeId="2" ParentId="26062" CreationDate="2017-02-01T18:37:29.257" Score="5" Body="&lt;p&gt;There is no solid definition for stiff equations. I like Shampine's working definition the best: a differential equation is stiff if explicit methods are less computationally efficient than implicit methods. All of the other definitions about eigenvalues are purposefully vague because they cannot capture the fact that the &quot;eigenvalues which matter&quot; is dependent on how fast the methods are, and so in some cases what may seem &quot;stiff&quot; by eigenvalues is something you don't actually want to treat as &quot;stiff&quot; because explicit methods may step well enough that they work better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that then has a lot to do with the program you are using. Since you are using looping with NumPy, this is going to a be a very inefficient implementation. Because your implementation is inefficient, the fact that the kernal in &lt;code&gt;fsolve&lt;/code&gt; is efficient may mean that, for this comparison, the implicit method will have less of a disadvantage due to implementation, and thus your threshold for stiffness would be lower than if you were using a heavily optimized code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I encourage you to adopt this working definition and use the idea of stiffness and the eigenvalues estimates as a guideline for choosing methods, but double-check intuition with benchmarks.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-02-01T18:43:07.460" LastActivityDate="2017-02-01T18:43:07.460" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26067" PostTypeId="2" ParentId="26062" CreationDate="2017-02-01T22:25:47.280" Score="1" Body="&lt;p&gt;The term &quot;stiff&quot; has many definitions, but a commonly understood one would be &quot;there are two or more timescales associated with the process being modeled, and these timescales are very different&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you have a single ODE whose solution is a single function $y(t)$. As a consequence, it has only a single timescale, and so one would not consider the ODE stiff.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-02-01T22:25:47.280" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26068" PostTypeId="2" ParentId="26063" CreationDate="2017-02-01T22:29:28.617" Score="4" Body="&lt;p&gt;You need to build a set of faces and see how many neighbors each has. You build the set of faces by looping over the cells. Each triangular cell has 3 faces and so you have to look up if the face already exists and add it to the list of faces if it doesn't. After that, you know that the face exists in the set and you add the current cell as one of possibly two neighbors of the face. At the end, you go through all faces and those that have only one neighbor will be at the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, the nodes that belong to boundary faces are at the boundary.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-02-01T22:29:28.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26071" PostTypeId="1" AcceptedAnswerId="26072" CreationDate="2017-02-02T11:37:43.243" Score="4" ViewCount="432" Body="&lt;p&gt;I'm working on some c++ code to solve (conjugate gradient, least squares conjugate gradient, LSQR,..) large sparse overdetermined systems of linear equations. There is a twist to my matrices and the system they're running on, that's why I code the solver myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For testing, debugging and benchmarking purposes I'm looking for some test matrix-vector-problems with well known solutions. I would like to be able to generate such systems for varying matrix size &lt;code&gt;N x M&lt;/code&gt;. (Typically I deal with matrices of the size &lt;code&gt;10^7 x 10^5&lt;/code&gt; with &lt;code&gt;10^9&lt;/code&gt; non-zero elements.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know any such matrix-vector-systems what can easily be generated?&lt;/p&gt;&#xA;" OwnerUserId="23122" LastActivityDate="2017-03-04T14:15:25.840" Title="Test matrices for large sparse overdetermined system of linear equations" Tags="|linear-solver|sparse-matrix|testing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26072" PostTypeId="2" ParentId="26071" CreationDate="2017-02-02T13:55:19.280" Score="4" Body="&lt;p&gt;I use the following website &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/matrices/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cise.ufl.edu/research/sparse/matrices/&lt;/a&gt; and, in particular, this collection &lt;a href=&quot;http://yifanhu.net/GALLERY/GRAPHS/search.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://yifanhu.net/GALLERY/GRAPHS/search.html&lt;/a&gt; when I need to find certain kind of matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another large collection of matrices is &lt;a href=&quot;http://math.nist.gov/MatrixMarket/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MatrixMarket&lt;/a&gt;, though it is a little bit outdated in my opinion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you search for rectangular matrices, you would be able to find matrices you are interested in. However, there are now sample RHS's and solutions. I guess you might use an SVD-based solver to get a reference solution for your iterative solver. &lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-02-02T13:55:19.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26073" PostTypeId="2" ParentId="25720" CreationDate="2017-02-02T15:03:30.993" Score="0" Body="&lt;p&gt;For physical intuition you can consider the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)For a fluid flow divergence of velocity calculates the the total flux leaving and entering the control volume. Hence in absence of a mass source-the velocity has to be divergence free, that is mass leaving a fixed control volume should be equal to that entering it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Curl calculates the 'magnitude' of rotation of the field. So a non-zero curl for velocity implies a rotational motion corresponding to a vortex. Then, curl free fields can be described by a potential function&lt;/p&gt;&#xA;" OwnerUserId="8166" LastActivityDate="2017-02-02T15:03:30.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26075" PostTypeId="1" CreationDate="2017-02-02T20:48:05.323" Score="2" ViewCount="586" Body="&lt;p&gt;I am trying to numerically evaluate a Greens function for this equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left[\frac{\partial^2}{\partial x^2} + f(x) \right] G(x) = \delta (x-x_0) $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Neumann boundary conditions. Here, the function $f(x)$ is known. The difficulty in solving this lies in the numerical approximation of the $\delta$ function. Rather than approximate $\delta \sim \frac{1}{x}$ I attempted to resolve the $\delta$ by integrating both sides in a small interval around $x_0$, but this leaves me with conditions on the derivative of $G$ at $x_0$ that I don't know how to implement in a finite elements method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would really appreciate a reference for solving such equations (preferably with a finite elements method), or if someone could lend a guiding hand for solving such an equation.&lt;/p&gt;&#xA;" OwnerUserId="22988" LastEditorUserId="9667" LastEditDate="2017-02-02T21:10:38.160" LastActivityDate="2017-02-03T22:53:27.463" Title="Numerically find Greens function" Tags="|finite-element|finite-difference|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26077" PostTypeId="2" ParentId="26042" CreationDate="2017-02-03T04:00:45.370" Score="3" Body="&lt;p&gt;In general, in the linearized theory of elasticity, it is only the displacement gradients that need to be small compared to unity. The displacements don't need to be small. (Note that when we say displacements are small, we usually mean displacements nondimensionalized with some quantity such as the characteristic length of the structure. The length of a beam in a beam problem would be a candidate for nondimensionalizing, for example.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the context of bars subjected to bending moments and torsion (Saint Venant's Problem), the assumption of small displacements (nondimensionalized appropriately) is made in addition to small displacement gradients. Without this assumption, the boundary conditions would involve terms of the form $\int_\mathcal{B}\mathbf{u} \times \mathbf{t} dS $ where $\mathbf{u}$ is the displacement vector $\mathbf{t}$ is the traction vector on one face of the bar. This is not an easy term to deal with as a boundary condition and therefore, we require $\mathbf{u}$ to be small so that the integral is of the same order as the order of the terms neglected in the linearized theory of elasticity. &lt;/p&gt;&#xA;" OwnerUserId="1156" LastActivityDate="2017-02-03T04:00:45.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26078" PostTypeId="1" AcceptedAnswerId="26079" CreationDate="2017-02-03T11:16:37.873" Score="10" ViewCount="1436" Body="&lt;p&gt;I know that the piecewise linear finite element approximation $u_h$ of &#xA;$$&#xA;\Delta u(x)=f(x)\quad\text{in }U\\&#xA;u(x)=0\quad\text{on }\partial U&#xA;$$&#xA;satisfies&#xA;$$&#xA;\|u-u_h\|_{H^1_0(U)}\leq Ch\|f\|_{L^2(U)}&#xA;$$&#xA;provided that $U$ is smooth enough and $f\in L^2(U)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; If $f\in H^{-1}(U)\setminus L^2(U)$, do we have the following analogous estimate, in which one derivative is taken away on both sides:&#xA;$$&#xA;\|u-u_{h}\|_{L^2(U)}\leq Ch\|f\|_{H^{-1}(U)}\quad?&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you provide references?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Thoughts:&lt;/strong&gt; Since we still have $u\in H^1_0(U)$, it should be possible to obtain convergence in $L^2(U)$. Intuitively, this should even be possible with piecewise constant functions.&lt;/p&gt;&#xA;" OwnerUserId="7789" LastEditorUserId="7789" LastEditDate="2017-02-03T12:20:14.673" LastActivityDate="2017-02-03T15:21:17.630" Title="$L^2$-convergence of finite element method when right hand side is only in $H^{-1}$ (Poisson eqn)" Tags="|finite-element|pde|reference-request|convergence|elliptic-pde|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26079" PostTypeId="2" ParentId="26078" CreationDate="2017-02-03T12:53:28.997" Score="14" Body="&lt;p&gt;&lt;strong&gt;Yes&lt;/strong&gt;, this is the standard &lt;em&gt;Aubin-Nitsche&lt;/em&gt; (or &lt;em&gt;duality&lt;/em&gt;) trick. The idea is to use the fact that $L^2$ is its own dual space to write the $L^2$-norm as an operator norm&#xA;$$\|u\|_{L^2} = \sup_{\phi\in L^2\setminus\{0\}} \frac{(u,\phi)}{\|\phi\|_{L^2}}.$$&#xA;We thus have to estimate $(u-u_h,\phi)$ for arbitrary $\phi\in L^2$. To do that, we &quot;lift&quot; $u-u_h$ to $H^1_0$ by considering first for arbitrary $\phi\in L^2$ the solution $w_\phi\in H^1_0$ of the dual problem&#xA;$$\label{eq1}\tag{1}&#xA; (\nabla w_\phi,\nabla v) =(\phi,v) \qquad\text{for all }v\in H^1_0.&#xA;$$&#xA;Using the standard regularity of the Poisson equation, we know that&#xA;$$ \|w_\phi\|_{H^2}\leq C \|\phi\|_{L^2}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inserting $v=u-u_h\in H^1_0$ in \eqref{eq1} and using Galerkin orthogonality for any finite element (in your case, piecewise linear) function $w_h$ yields the estimate&#xA;$$&#xA;\begin{aligned}&#xA;(\phi,u-u_h) &amp;amp;= (\nabla w_\phi,\nabla (u-u_h))\\&#xA; &amp;amp;= (\nabla w_\phi-\nabla w_h,\nabla (u-u_h))\\&#xA; &amp;amp;\leq C \|u-u_h\|_{H^1}\|w_\phi-w_h\|_{H^1}.&#xA;\end{aligned}&#xA;$$&#xA;Since this holds for all $w_h$, the inequality is still true if we take the infimum over all piecewise linear $w_h$.&#xA;We therefore obtain&#xA;$$\label{eq2}\tag{2}&#xA;\|u-u_h\|_{L^2} = \sup_{\phi\in L^2\setminus\{0\}} \frac{(u-u_h,\phi)}{\|\phi\|_{L^2}} \leq C \|u-u_h\|_{H^1} \sup_{\phi\in L^2\setminus\{0\}} \frac{\inf_{w_h}\|w_\phi-w_h\|_{H^1}}{\|\phi\|_{L^2}}.&#xA;$$&#xA;This is the &lt;em&gt;Aubin-Nitsche-Lemma&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step is now to use standard error estimates for the best finite element approximation of solutions to the Poisson equation. Since $u$ is only in $H^1$, we don't get a better estimate than&#xA;$$\label{eq3}\tag{3}&#xA;\|u-u_h\|_{H^1} \leq \inf_{v_h}\|u-v_h\|_{H^1} \leq c \|u\|_{H^1} \leq C \|f\|_{H^{-1}}.&#xA;$$&#xA;But fortunately, we can use the fact that $w_\phi$ has higher regularity since the right-hand side $\phi\in L^2$ instead of $H^{-1}$. In this case, we have&#xA;$$\label{eq4}\tag{4}&#xA;\inf_{w_h}\|w_\phi-w_h\|_{H^1} \leq c h \|w_\phi\|_{H^2} \leq C h \|\phi\|_{L^2}&#xA;$$&#xA;Inserting \eqref{eq3} and \eqref{eq4} into \eqref{eq2} now yields the desired estimate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note that the standard estimates require that the polynomial degree $k$ of the finite element approximation and the Sobolev exponent $m$ of the true solution satisfy $m&amp;lt;k+1$, so this argument doesn't work for piecewise constant ($k=0$) approximation. We also have used that $u-u_h\in H^1_0$ -- i.e., that we have a &lt;em&gt;conforming&lt;/em&gt; approximation -- which is not true for piecewise constants.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you asked for a reference: You can find a statement (even for negative Sobolev spaces $H^{-s}$ instead of $L^2$) in Theorem 5.8.3 (together with Theorem 5.4.8) in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Susanne C. Brenner and L. Ridgway Scott&lt;/em&gt;, MR 2373954 &lt;a href=&quot;http://dx.doi.org/10.1007/978-0-387-75934-0&quot;&gt;&lt;strong&gt;The mathematical theory of finite element methods&lt;/strong&gt;&lt;/a&gt;, &lt;em&gt;Texts in Applied Mathematics&lt;/em&gt; ISBN: 978-0-387-75933-3.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2017-02-03T15:21:17.630" LastActivityDate="2017-02-03T15:21:17.630" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26080" PostTypeId="1" CreationDate="2017-02-03T15:20:10.957" Score="1" ViewCount="319" Body="&lt;p&gt;In computation of the solution I achieve a solution with nice precision and small number of nodes but the condition number of the matrix is very large. I am confused. Is it possible to have a large condition number with a good precision? The condition number is $10^4$ and it increases as the number of nodes increses and also the precision increases. How do you explain this condition?&lt;/p&gt;&#xA;" OwnerUserId="23152" LastEditorUserId="9667" LastEditDate="2017-02-03T15:37:08.640" LastActivityDate="2017-02-03T21:50:07.830" Title="Large condition number with good accuracy" Tags="|finite-element|condition-number|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26081" PostTypeId="1" AcceptedAnswerId="26084" CreationDate="2017-02-03T16:56:47.040" Score="0" ViewCount="52" Body="&lt;p&gt;I need to write a report about some image processing topic. I want to describe some kind of density, e.g:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho = \frac{\text{voxels with greyvalue 128}}{\text{number of voxels}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a common way how to describe this in a more mathematical sense? I would use for example the cardinality symbol and define some function for the voxels:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho = \frac{|V_{128}|}{|V|}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;denoting $V$ as the Voxels.&#xA;But in my opinion the $V_{128}$ looks terrible and unintuitive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one publication I read for example $o(128)$ where $o$ is a function of greyvalues.&#xA;But I think this is also not commonly used?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This question might be off-topic here. I'm not sure :/)&lt;/p&gt;&#xA;" OwnerUserId="23155" LastActivityDate="2017-02-05T01:31:58.740" Title="Is there a common way to decribe voxels and greyvalues in formulars" Tags="|image-processing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26082" PostTypeId="2" ParentId="24999" CreationDate="2017-02-03T20:57:56.850" Score="1" Body="&lt;p&gt;Another option would be using Jacobi rotations. Since your matrix is already almost diagonal, it shouldn't take much time to converge. Generally it converges in linear rate, but after enough iterations the convergence rate becomes quadratic. &lt;/p&gt;&#xA;" OwnerUserId="3761" LastActivityDate="2017-02-03T20:57:56.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26083" PostTypeId="2" ParentId="26080" CreationDate="2017-02-03T21:50:07.830" Score="5" Body="&lt;p&gt;Of course you can. The condition number gives you an upper bound for the error. Suppose you want to solve $Ax=b$ where $A$ is a matrix with large condition number. The error depends on the structure of $A$ and on $b$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, if $A$ is a $2 \times 2$ diagonal matrix, with elements $1$ and $10^{-20}$, solving $Ax=b$ will still be accurate, despite having a huge condition number of $10^{20}$.&lt;/p&gt;&#xA;" OwnerUserId="3761" LastActivityDate="2017-02-03T21:50:07.830" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26084" PostTypeId="2" ParentId="26081" CreationDate="2017-02-03T22:36:05.413" Score="0" Body="&lt;p&gt;It's simply a question of notation. Let's say you have previously defined $V_{128}$ as the set&#xA;$$&#xA;  V_i = \{k : v_k=i\}&#xA;$$&#xA;(in other words, as the set of &lt;em&gt;indices&lt;/em&gt; $k$ for which the $k$th voxel value $v_k$ equals $i$), then $|V_i|$ is clearly the size of the set -- in other words, the number of voxels with the desired grayscale value. The point is that you must make clear beforehand that $V_i$ is a set, and then writing $|V_i|$ makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other notations would use $\#V_i$ to denote the number of elements of a set, which is of course the same as the size of the set.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2017-02-05T01:31:58.740" LastActivityDate="2017-02-05T01:31:58.740" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26085" PostTypeId="1" AcceptedAnswerId="26087" CreationDate="2017-02-03T22:41:04.187" Score="0" ViewCount="438" Body="&lt;p&gt;I am trying to teach myself the basics of FEM but am having trouble with the the manipulations involved. For instance, Example 5.5 of Hutton's &quot;Fundamentals of Finite Element Analysis&quot; provides a worked solution of&lt;br&gt;&#xA; $$x{d^2y\over dx^2}+{dy\over dx} -4x=0 \qquad 1\le x \le 2 \qquad y(1)=y(2)=0$$&#xA;using a hat-type trial function: $y(x)=y_1.N_1+y_2.N_2=y_1{x_2-x\over x_2-x_1}+y_2{x-x_1\over x_2-x_1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The working given, relies on the insight that the equation is equivalent to&#xA;$${d\over dx}\left(x{dy\over dx}\right)-4x=0$$&#xA;giving the element equation&#xA;$$\int_{x_1}^{x_2}N_i\left[{d\over dx}\left(x{dy\over dx}\right)-4x\right]=0 \qquad i=1, 2$$&#xA;Integration by parts, substitution and rearrangement then gives&#xA;$$N_ix{dy\over dx}\biggl|_{x_1}^{x_2}-\int_{x_1}^{x_2}x{dN_i\over dx}.{dy \over dx}dx-4\int_{x_1}^{x_2}x.N_idx=0$$&#xA;All this I can follow; my problem comes if I miss that insight that leads to the second form of the subject DE above. Then working from the original form of the DE, I get an element equation like&#xA;$$ x.N_i{dy\over dx}\biggl|_{x_1}^{x_2}-\int_{x_1}^{x_2}{d(N_i.x)\over dx}{dy\over dx}dx+\int_{x_1}^{x_2}N_i.{dy\over dx}dx-4\int_{x_1}^{x_2}x.N_idx=0$$&#xA;and try as I may, I cannot get to a form equivalent to the first approach above. I suspect I am making a basic error in the integration or other part of the manipulation. Can someone please guide me out of my confusion?&lt;/p&gt;&#xA;" OwnerUserId="4700" LastActivityDate="2017-02-03T23:33:57.890" Title="Integration by parts for FEM" Tags="|finite-element|integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26086" PostTypeId="2" ParentId="26075" CreationDate="2017-02-03T22:53:27.463" Score="2" Body="&lt;p&gt;Why approximate the delta function? In the finite element method, you need to evaluate the weak form of the equation, for which the right hand side will look like this:&#xA;$$&#xA;  \int \varphi_i(x) \delta(x-x_0),&#xA;$$&#xA;where $\varphi_i$ is the $i$th shape function. But you &lt;em&gt;know&lt;/em&gt; what this integral is:&#xA;$$&#xA;  \int \varphi_i(x) \delta(x-x_0) = \varphi_i(x_0).&#xA;$$&#xA;Of course, most shape functions will be zero at $x_0$, but the shape functions defined on the cell that contains $x_0$ will not, and those are going to be the only nonzero entries in your right hand side vector.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-02-03T22:53:27.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26087" PostTypeId="2" ParentId="26085" CreationDate="2017-02-03T23:33:57.890" Score="3" Body="&lt;p&gt;If you use product rule of differentiation on $N_i \cdot x$, you can simplify the second and third terms in the last equation of your post to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ -\int_{x_1}^{x_2}{d(N_i.x)\over dx}{dy\over dx}dx+\int_{x_1}^{x_2}N_i {dy\over dx}dx = - \int_{x_1}^{x_2}x{dN_i\over dx}{dy\over dx}dx  - \int_{x_1}^{x_2} N_i {dy\over dx}dx+\int_{x_1}^{x_2}N_i{dy\over dx}dx&#xA;= - \int_{x_1}^{x_2}x{dN_i\over dx}{dy\over dx}dx .&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="1156" LastActivityDate="2017-02-03T23:33:57.890" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26088" PostTypeId="1" CreationDate="2017-02-04T05:12:15.650" Score="1" ViewCount="48" Body="&lt;p&gt;Assume a function $M(t)$ strictly increasing, essentially growing exponentially, and asymptoptically  growing at a known rate $\bar{g}$, i.e. $\lim_{t\to\infty}M'(t)/M(t) = \bar{g}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a set of awful integro-differential equations, I need to use the inverse of $M(t)$, denoted by $q(\cdot)$ for simplicity, i.e. $t = M^{-1}(M) \equiv q(M)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $M(t)$ will be solved numerically with some sort of spectral collocation method with a polynomial basis (see below).  My general problem is: with a guess on $M(t)$, how can I find $q(M)$?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Since this equation is growing exponentially, I don't think I can use a polynomial basis for $M(t)$ directly.  Instead, I suspect that I should guess a Chebyshev basis for $g(t)$ constraining for some large $T$ that $g(T) = \bar{g}$.  If so, then&#xA;$$M(t) \equiv \exp\left(\int_0^t g(\tau) d \tau \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which works fine for my computations.  In particular, it is easy with a Chebyshev basis to find the partial integrals $\int_0^t g(\tau) d \tau$ for $t$ nodes of the Chebyshev basis (through &lt;a href=&quot;https://scicomp.stackexchange.com/questions/23398/calculating-integrals-for-a-function-approximated-by-chebyshev-polynomials&quot;&gt;Calculating integrals for a function approximated by Chebyshev polynomials&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if so, how can I find the $q(M)$ function, or a basis of it in $M$ space?  Obviously, I could just evaluate $M(t)$ at the Chebyshev nodes of $t$ to find the appropriate $M$, and then use some sort of interpolation to find the approximating $q(M)$, but I would prefer an approach that has a chance of using auto-differentiation by throwing it into my collocation method.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For example, is there a way to translate this into a nonlinear ODE using the inverse-function theorem, and if so does this simplify substantially given that this is a Chebyshev basis?&lt;/p&gt;&#xA;" OwnerUserId="9692" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-02-04T05:12:15.650" Title="Numerically inverting an exponentially growing function (defined by Chebyshev polynomials)" Tags="|inverse|collocation|chebyshev|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26089" PostTypeId="1" AcceptedAnswerId="26097" CreationDate="2017-02-04T08:05:19.427" Score="2" ViewCount="265" Body="&lt;p&gt;&lt;strong&gt;Problem description:&lt;/strong&gt;&#xA;I want to numerically solve system of two time-independent partial differential equations (pde) of the following simple form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u(x,y)}{\partial x} = f_1(x,y,u),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u(x,y)}{\partial y} = f_2(x,y,u),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u(x,y)$ is the unknown we want to solve for, $x$ and $y$ are the spatial variables and $f_1$,$f_2$ are some smooth (or at least continuously differentiable) and possibly nonlinear functions satisfying $\frac{\partial f_1}{\partial y}=\frac{\partial f_2}{\partial x}$. More compactly&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla u(x,y) = f(x,y,u),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\nabla:=[\frac{\partial~ \cdot}{\partial x}, \frac{\partial~ \cdot}{\partial y}]^T$ and $f:=[f_1,f_2]^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve it on a rectangular domain (defined let’s say by $x$,$y$ for which $W&amp;gt;x&amp;gt;0$ and $H&amp;gt;y&amp;gt;0,~ W,H \in \mathbb R$) for an initial given boundary condition of $u(0,0)=c,~c \in \mathbb R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I have tried so far:&lt;/strong&gt;&#xA;$u(x,y)$ can be imagined like a surface. We stand at the initial point and the partial derivatives tell us how much the surface goes upwards or downwards in the corresponding directions.&#xA;In 1D case, when we have $\frac{du}{dx}=f(x,u)$, an ODE solver (like &lt;em&gt;ode45&lt;/em&gt; in Matlab) can be used (we can interpret the time $t$ as the spatial variable $x$ without no harm here). I thought, whether there is some way how to extend this approach to the 2 dimensional case. The solution that works to some extent is to solve for $u$ along one of the edges of the domain (e.g. find $u$ on $W&amp;gt;x&amp;gt;0,~y=0$ using ode45 as mentioned above) and then to use points of this solution as starting values for other set of ode's (now going in the perpendicular direction). In other words, if the goal is to find the values of $u$ on a regular grid, then we first find its values along the first row of the grid and then use the result as a starting point for ode's going along each of the columns and thus getting values for the whole grid. Although such a solution gives an idea how the surface $u(x,y)$ looks like, it is, however, not very precise as the individual solutions for rows/columns of the grid get apart from each other as we are moving away from the edge used as a set of initial values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I have also tried &lt;em&gt;chebfun&lt;/em&gt; toolbox for Matlab and pde toolbox, but it seems that they only support time-dependent equations.&lt;/p&gt;&#xA;" OwnerUserId="23148" LastEditorUserId="23148" LastEditDate="2017-02-04T12:44:01.343" LastActivityDate="2017-02-06T01:24:43.270" Title="Numerically solving $\nabla u(x,y) = f(x,y,u)$ on a rectangular domain having initial value of $u$ at some point" Tags="|pde|numerics|" AnswerCount="3" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26090" PostTypeId="2" ParentId="26043" CreationDate="2017-02-04T16:19:53.713" Score="1" Body="&lt;p&gt;I assume you refer to what's become known as the error reduction algorithm for phase retrieval (e.g. as defined in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Phase_retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article on Phase retrieval&lt;/a&gt;). I've used ER in several contexts, but since this algorithm relies on a very simpler alternating projections scheme between the two spaces, convergence tends to be slow. It also tends to get stuck in local minima, where the most direct developments (HIO is mentioned in the article, in my group we have been using RAAR quite a bit) can perform much better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would also look seriously into whether the problem, as you specify it, is expected to be solvable at all. You say very little on the structure of your problem, and it is very easy to specify something which is seriously underdetermined, at least when you take experimental noise into account. That is, there are a plethora of feasible &quot;solution&quot; which are nothing alike a proper recovery, and it can even be that the very best global optimum, would one find it, is too far removed from the true object to be of relevance. Having a proper support constraint, and possibly constraints on positivity/reality (no imaginary part) in the object space can make a significant difference, in practice.&lt;/p&gt;&#xA;" OwnerUserId="23161" LastActivityDate="2017-02-04T16:19:53.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26091" PostTypeId="1" AcceptedAnswerId="26093" CreationDate="2017-02-04T18:21:15.753" Score="1" ViewCount="179" Body="&lt;p&gt;I have a system where $A$ is a large $n\times n$ marix with fast MVMs. It may have many nonzero entries (albeit in a structured way so as to allow fast MVMs), and is not necessarily diagonally dominant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, $A$ is weakly positive definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Y$ is a $n\times m$ dense matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand iterative Krylov-subspace-based methods exist for finding $X=A^{-1}Y$, and they perform well. Are there any optimizations that can be implemented if I am only interested in finding:&#xA;$$&#xA;\textbf{x}=\text{diag }Y^\top A^{-1}Y&#xA;$$&#xA;In other words, I only want to solve for the $i$-th entry of the solution $Y^\top\textbf{x}_i$ where $A\textbf{x}_i=\textbf{y}_i$ for $Y=[\textbf{y}_i]_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MVM stands for matrix-vector multiplication. In my application this takes only ~$n$ runtime compared to $n^2$ when dense.&lt;/p&gt;&#xA;" OwnerUserId="23164" LastEditorUserId="23164" LastEditDate="2017-02-05T13:12:38.990" LastActivityDate="2022-11-08T03:31:26.753" Title="Iterative single variable solutions in large linear systems" Tags="|linear-solver|sparse-matrix|krylov-method|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26093" PostTypeId="2" ParentId="26091" CreationDate="2017-02-04T20:07:48.090" Score="2" Body="&lt;p&gt;An approach by &lt;a href=&quot;https://ttic.uchicago.edu/%7Egpapan/pubs/confr/PapandreouYuille_VariationalBayesCompressedSensing_ieee-c-iccvw11.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Papandreou and Yuille&lt;/a&gt; for diagonal &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; relates variance estimation to the expectation of a quadratic form. The logic follows more generally: since &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is PD so is its inverse. Then &lt;span class=&quot;math-container&quot;&gt;$Z\sim N(0, A^{-1})$&lt;/span&gt; is well-formed, and &lt;span class=&quot;math-container&quot;&gt;$\textbf{x}=\mathbb{E}[(Y^\top Z)^2]$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If we can sample &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; then an approximation is possible (and converges). The linked paper has success when we can find the square root of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;: solving &lt;span class=&quot;math-container&quot;&gt;$AZ=A^{1/2}G$&lt;/span&gt; for samples &lt;span class=&quot;math-container&quot;&gt;$G\sim N(0,I)$&lt;/span&gt; offers a way of sampling &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; since &lt;span class=&quot;math-container&quot;&gt;$A^{-1/2}G\sim N(0,A^{-1})$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When an operator &lt;span class=&quot;math-container&quot;&gt;$A^{1/2}$&lt;/span&gt; is available, this provides an estimation opportunity for variance.&lt;/p&gt;&#xA;&lt;p&gt;Edit (5 years later):&lt;/p&gt;&#xA;&lt;p&gt;Given only fast MVMs for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; we can actually achieve this. Using &lt;a href=&quot;https://arxiv.org/abs/2006.11267&quot; rel=&quot;nofollow noreferrer&quot;&gt;a matrix countour integral&lt;/a&gt; we can approximate MVMs of &lt;span class=&quot;math-container&quot;&gt;$A^{-1/2}$&lt;/span&gt; using essentially a single iterative solve of a linear systems in &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (it's actuall &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt; concurrent solves, where this &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt; depends on quadrature error from the integral, and here concurrent solves means that Krylov iterates are shifts of one another, so we don't require additional applications of the operator &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;). As a result, we can directly sample &lt;span class=&quot;math-container&quot;&gt;$Z=A^{-1/2}G$&lt;/span&gt; via this iterative routine.&lt;/p&gt;&#xA;&lt;p&gt;The question then becomes, how many such solves do we need for our particular variance terms of interest &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[(Y^\top Z)^2]$&lt;/span&gt;? At the end of the day, this is essentially asking what's the efficiency of estimating the covariance matrix for some &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$W\sim N(0,Y^\top A^{-1} Y)$&lt;/span&gt;, which admits &lt;a href=&quot;https://en.wikipedia.org/wiki/Wishart_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wishart&lt;/a&gt; statistics.&lt;/p&gt;&#xA;&lt;p&gt;Edit edit:&lt;/p&gt;&#xA;&lt;p&gt;Thanks to @YaroslavBulatov we can add some meat to the last statement, which is that worst-case relative &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; error in expectation in estimating &lt;span class=&quot;math-container&quot;&gt;$\mathrm{cov}\ W$&lt;/span&gt; demands &lt;span class=&quot;math-container&quot;&gt;$O(\epsilon^{-2} r \log m)$&lt;/span&gt; samples, where &lt;span class=&quot;math-container&quot;&gt;$r=\mathrm{tr}\ \Sigma/\lambda_{\max}(\Sigma)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\Sigma = Y^\top A^{-1} Y$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;While &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is unknown, it may be practically feasible to estimate using Hutchinson's trick. Sampling new &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; and independent Rademacher vectors &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbb{E}[\rho^\top Y^\top Z Z^\top Y \rho]=\mathrm{tr}\ \Sigma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\lambda_{\max}$&lt;/span&gt; we can estimate with inverse power iteration. The composition &lt;span class=&quot;math-container&quot;&gt;$Y\rho$&lt;/span&gt; is a single vector, so this second expectation is (conditionally on &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt;) a chi-squared statistic with degrees of freedom equal to the number of samples &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; we use for this error estimation, which is sub-exponential and concentrates.&lt;/p&gt;&#xA;" OwnerUserId="23164" LastEditorUserId="23164" LastEditDate="2022-11-08T03:31:26.753" LastActivityDate="2022-11-08T03:31:26.753" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="26094" PostTypeId="2" ParentId="26064" CreationDate="2017-02-04T20:49:21.190" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Strang_splitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;Strang splitting method&lt;/a&gt; goes like this. You start with the PDE&#xA;$$ u_t = (L+B)u, \qquad L = \partial_x^2, $$&#xA;and you notice that when $L$ and $B$ are independent of $x$, the exact solution after time $\delta t$ to this is&#xA;$$ u = e^{(L+B)\delta t}u_0 \approx e^{\frac12 L\delta t}e^{B\delta t}e^{\frac12 L\delta t}u_0. $$&#xA;Because $B$ is just a function of $x$, the operator $e^{B\delta t}$ just multiplies by $e^{B(x)\delta t}$.&#xA;Because $L = F \hat L F^{-1}$ is diagonalizable by the Fourier transform (assuming the right boundary conditions),&#xA;$$ e^{L\delta t} = F e^{\hat L \delta t} F^{-1},$$&#xA;where $e^{\hat L\delta t}$ is the operator that multiplies each Fourier mode by $e^{-k^2\delta t}$ (this depends on choice of normalization).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why Fourier transforms are done at each step: in the Fourier basis, and only in that basis, is $L$ diagonal, which makes it trivial to compute its exponential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your equation, to get what $e^{B\delta t}$ would look like, you write out the relevant portion of the equation, with only $B$ present:&#xA;$$ i\hbar u_t = (\gamma |u|^2 + \alpha (|u|^2)_{xx})u. $$&#xA;One thing you could do is to approximate $g(x) \approx (|u_0|^2)_{xx}$, so that&#xA;$$ u(t,x) \approx u(0,x)\exp\left(\frac{\gamma|u(0,x)|^2 + \alpha g(x)}{i\hbar}\,\delta t\right). $$&#xA;Because of the nonlinearity, it might work, but I think there isn't a guarantee that it will—I haven't tried it. But the idea is still the same: split the r.h.s. into two operators, and for each operator solve the corresponding PDE, choosing the operators in a way that makes this step easy.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-02-04T22:03:57.297" LastActivityDate="2017-02-04T22:03:57.297" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26095" PostTypeId="2" ParentId="26089" CreationDate="2017-02-04T22:07:43.447" Score="0" Body="&lt;p&gt;u have a system of differential equations to be solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;first u are trying to map a surface into a cube.  u can break the equation into delta u(x,y)=f(x,y)=phi(u(x,y).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the set of equations are delta u(x,y)=f(x,y)....................equation 1..&#xA;.........................................delta u(x,y)=phi (u(x,y))=f(x,y).......equation 2.............................................with the condition .........f(x,y) where x=f1, y=f2.............. means..................(partial x)/(partial y)=(partial y)/(partial x)&#xA;...................................................................equation 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the system of equations will be&#xA;{                          partial u/partial x .{f                    ={0&#xA;                           partial u/partial y . f                    = 0 &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                       partial u/partial x . f                  = 0&#xA;                       partial u/partial y . f                  = 0&#xA;                       partial x/partial y . partial y/partial x  = 0 &#xA;                       partial y/partial x . partial x/partial y} = 0}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;that's {delta u(x,y), partial (x,y)}^T dot {f, f, partial (y,x))^T={0}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;u just have to solve the equations simultaneously.                &lt;/p&gt;&#xA;" OwnerUserId="23011" LastActivityDate="2017-02-04T22:07:43.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26097" PostTypeId="2" ParentId="26089" CreationDate="2017-02-05T07:40:51.550" Score="1" Body="&lt;p&gt;The approach you describe should work, and if you let the step size of your ODE solvers go to zero, you will recover the exact solution of the problem. You can of course also start by first going in $y$ direction instead of the $x$ direction. Or, you can use a marching front algorithm in which you compute, for example, $u(0,h)$ and $u(h,0)$ using one-dimensional steps, and then $u(h,h)$ by &lt;em&gt;averaging&lt;/em&gt; the solution you would get by starting from either $u(0,h)$ or $u(h,0)$ and then going in the perpendicular solution. This is likely going to be more accurate. You would then move out from the points you have to the next lattice points on a mesh with points spaces by distances $h$ in each direction, averaging whenever you can.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-02-05T07:40:51.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26098" PostTypeId="2" ParentId="26049" CreationDate="2017-02-05T13:43:59.193" Score="1" Body="&lt;p&gt;An alternate approach to going from ${b}(x)$ to ${b}(x(\xi))$ by direct substitution of $x(\xi)$ in $b(x)$ is to interpolate $b(x)$ over the element using $b_i^e$ and element shape functions (quadratic) or even linear shape functions. So, one would write&#xA;$$&#xA;b(x(\xi)) \approx \sum_{i=1}^3 b_i^e N_i(\xi) \text{ on } \Omega^e&#xA;$$&#xA;with $b_i^e$ being the nodal value of the body force at node i on element $e$. Of course, when $b(x)$ is constant or linear, this would be exact.&lt;/p&gt;&#xA;" OwnerUserId="1156" LastActivityDate="2017-02-05T13:43:59.193" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26101" PostTypeId="2" ParentId="26089" CreationDate="2017-02-06T00:44:31.667" Score="0" Body="&lt;p&gt;How about the following approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 1: Determine the values on the $x=0$ and $y=0$ lines:&#xA;$$&#xA;u_{i+1,0} = u_{i,0} + f_{i,0}^1\Delta x + \mathcal{O}(\Delta x^2) \\&#xA;u_{0,j+1} = u_{0,j} + f_{0,j}^2\Delta y + \mathcal{O}(\Delta y^2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step 2: Use Taylor series expansion for all the other points:&#xA;$$&#xA;u_{i+1,j+1} = u_{i,j} + f_{i,j}^1\Delta x + f_{i,j}^2 \Delta y + \mathcal{O}( \Delta x^2, \Delta x\Delta y, \Delta y^2)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that for all the &quot;interior&quot; points, this involves both $f^1$ and $f^2$ contributing to $u_{i,j}$ thus removing directional bias.&lt;/p&gt;&#xA;" OwnerUserId="1156" LastEditorUserId="1156" LastEditDate="2017-02-06T01:24:43.270" LastActivityDate="2017-02-06T01:24:43.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26102" PostTypeId="2" ParentId="25766" CreationDate="2017-02-06T07:37:24.183" Score="1" Body="&lt;p&gt;There are a bunch of useful functions missing in BLAS/LAPACK, for example the $ax+by \rightarrow y$ operation. The easiest way to get this functionality is to search for a similar BLAS/LAPACK routine, pick its source code, rename the function and extend its functionality to your needs. &lt;/p&gt;&#xA;" OwnerUserId="569" LastActivityDate="2017-02-06T07:37:24.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26105" PostTypeId="1" AcceptedAnswerId="31123" CreationDate="2017-02-06T18:30:17.890" Score="15" ViewCount="1441" Body="&lt;p&gt;I have a question regarding quadric fit to a set of points and corresponding normals (or equivalently, tangents). Fitting quadric surfaces to point data is well explored. Some works are as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://people.eecs.berkeley.edu/%7Esequin/PAPERS/2013_CAD&amp;amp;A_quadrics.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Type-Constrained Direct Fitting of Quadric Surfaces&lt;/a&gt;, &lt;em&gt;James Andrews, Carlo H. Sequin&lt;/em&gt;  Computer-Aided Design &amp;amp; Applications, 10(a), 2013, bbb-ccc&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.maths.dundee.ac.uk/%7Egawatson/algebraic.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algebraic fitting of quadric surfaces to data&lt;/a&gt;, &lt;em&gt;I. Al-Subaihi and G. A. Watson&lt;/em&gt;, University of Dundee&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Fitting to projective contours is also covered by some works, &lt;a href=&quot;http://www.cc.gatech.edu/%7Edellaert/workshop/final/kang.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;such as this one&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From all these works, I think Taubin's method for Quadric fitting is pretty popular:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;G. Taubin, &lt;a href=&quot;http://mesh.brown.edu/taubin/pdfs/Taubin-pami91.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Estimation of Planar Curves, Surfaces and Nonplanar Space Curves Defined by Implicit Equations, with Applications to Edge and Range Image Segmentation&lt;/a&gt;&amp;quot;, IEEE Trans. PAMI, Vol. 13, 1991, pp1115-1138.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Let me briefly summarize. A Quadric &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; can be written in the algebraic form:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(\mathbf{c},\mathbf{x}) = A x^2 + By^2 + Cz^2 + 2Dxy + 2Exz + 2Fyz + 2Gx + 2Hy + 2Iz + J&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{c}$&lt;/span&gt; is the coefficient vector and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; are the 3D coordinates. Any point &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt; lies on the quadric &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$\begin{pmatrix}\mathbf{x}^T &amp;amp; 1\end{pmatrix}Q\begin{pmatrix}\mathbf{x} \\ 1\end{pmatrix}=0$&lt;/span&gt;, where:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Q = &#xA;\begin{bmatrix}&#xA;A &amp;amp; D &amp;amp; E &amp;amp; G \\&#xA;D &amp;amp; B &amp;amp; F &amp;amp; H \\&#xA;E &amp;amp; F &amp;amp; C &amp;amp; I \\&#xA;G &amp;amp; H &amp;amp; I &amp;amp; J \\&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Algebraic Fit&lt;/strong&gt; In principle, we would like to solve for the parameters that&#xA;minimize the sum of squared geometric distances between the points and the quadratic surface. Unfortunately, it turns out that this is a non-convex optimization problem with no known analytical solutions. Instead, a standard approach is to solve for an algebraic fit, that is to solve for the parameters&#xA;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{c}$&lt;/span&gt; that minimize:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum\limits_{i=1}^{n} f(\mathbf{c},\mathbf{x}^i)^2 = \mathbf{c}^T  M \mathbf{c}&#xA;$$&lt;/span&gt;&#xA;with&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;M = \sum\limits_{i=1}^{n} l(\mathbf{x}^i)l(\mathbf{x}^i)^T&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\{\mathbf{x}^i\}$&lt;/span&gt; are the points in the point cloud and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;l = [x^2, y^2, z^2, xy, xz, yz, x,y,z, 1]^T&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Notice that such direct minimization would yield the trivial solution with &lt;span class=&quot;math-container&quot;&gt;$\mathbf{c}$&lt;/span&gt; at the origin. This question has been studied extensively in the&#xA;literature. One resolution that has been found to work well in practice is Taubin’s method (cited above), introducing the constraint:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\| \nabla_x f(\mathbf{c},\mathbf{x}^i) \|^2 = 1&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This can be solved as follows: Let :&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;N = \sum\limits_{i=1}^n l_x(\mathbf{x}^i)l_x(\mathbf{x}^i)^T+ l_y(\mathbf{x}^i)l_y(\mathbf{x}^i)^T + l_z(\mathbf{x}^i)l_z(\mathbf{x}^i)^T&#xA;$$&lt;/span&gt;&#xA;where subscripts denote the derivatives. The solution is given by the generalized Eigen decomposition, &lt;span class=&quot;math-container&quot;&gt;$(M −\lambda N) \mathbf{c} = 0$&lt;/span&gt;. The best-fit parameter vector is equal to the Eigenvector corresponding to the smallest Eigenvalue.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Main Question&lt;/strong&gt;&#xA;In many applications, the normals of the point cloud are available (or computed). The normals of the quadric &lt;span class=&quot;math-container&quot;&gt;$\mathbf{N}(x)$&lt;/span&gt; can also be calculated by differentiating and normalizing the implicit surface:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{N}(x) = \frac{\nabla f(\mathbf{c},\mathbf{x})}{\|\nabla f(\mathbf{c},\mathbf{x})\|}&#xA;$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla f(\mathbf{c},\mathbf{x}) = 2&#xA;\begin{bmatrix}&#xA;Ax + Dy + Fz + G \\ &#xA;By + Dx + Ez + H \\ &#xA;Cz + Ey + Fx + I \\ &#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, Taubin's method utilizes only the point geometry, and not the tangent space. And I am not aware of many methods, which are suitable for fitting quadrics such that the tangents of the quadric also match the tangents of underlying point cloud. I am looking for potential extensions of the method above, or any other to cover these first order derivatives.&lt;/p&gt;&#xA;&lt;p&gt;What I would like to achieve is maybe addressed partially in lower dimensional spaces, with more primitive surface (curve) types. For example, fitting lines to image edges, taking into consideration the gradient information is covered &lt;a href=&quot;http://ieeexplore.ieee.org/document/6816847/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Fitting planes (a simple type of quadric) to 3D clouds is very common (&lt;a href=&quot;https://hal-mines-paristech.archives-ouvertes.fr/hal-01097361/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;link 1&lt;/a&gt;) or fitting spheres or cylinders can be fit to oriented point sets (&lt;a href=&quot;http://ieeexplore.ieee.org/document/7335508/?reload=true&amp;amp;arnumber=7335508&quot; rel=&quot;nofollow noreferrer&quot;&gt;link 2&lt;/a&gt;). So what I'm wondering is something similar, but the fitted primitive is a quadric.&lt;/p&gt;&#xA;&lt;p&gt;I would also welcome the analysis of the proposed method such as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What is the minimum number of oriented points required?&lt;/li&gt;&#xA;&lt;li&gt;What are the degenerate cases ?&lt;/li&gt;&#xA;&lt;li&gt;Can anything be said about robustness?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;:&#xA;I would like to present a direction to follow. Formally, what I desire to achieve:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\| \nabla f - \mathbf{n} \| = 0&#xA;$$&lt;/span&gt;&#xA;at the point &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt;. Maybe it might be possible to fuse it with Taubin's method to come up with an additional constraint and minimize using Lagrange multipliers?&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="11650" LastEditDate="2020-10-04T09:54:43.133" LastActivityDate="2020-10-04T09:54:43.133" Title="Fitting Implicit Surfaces to Oriented Point Sets" Tags="|computational-geometry|regression|geometry|curve-fitting|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="26106" PostTypeId="2" ParentId="25118" CreationDate="2017-02-06T20:54:08.757" Score="3" Body="&lt;p&gt;I am quite surprised why no-one mentioned the famous &lt;a href=&quot;https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Douglas Peucker Algorithm&lt;/a&gt; for polyline simplification. Since you have contour points in hand, you could benefit from it directly. &lt;a href=&quot;http://docs.opencv.org/3.1.0/dd/d49/tutorial_py_contour_features.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contour approximation in OpenCV&lt;/a&gt; uses this method. See &lt;a href=&quot;https://github.com/opencv/opencv/blob/master/samples/cpp/contours2.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; for usage and you could also find a MATLAB implementation &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/21132-line-simplification&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/41986-ramer-douglas-peucker-algorithm-demo&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2020-10-08T18:10:26.213" LastActivityDate="2020-10-08T18:10:26.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="26107" PostTypeId="1" AcceptedAnswerId="26138" CreationDate="2017-02-06T21:46:28.053" Score="1" ViewCount="1162" Body="&lt;p&gt;What is the difference between edge finite elements and nodal finite elements?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This for use in modeling the eddy current problem in classical electromagnetism. I am attempting to convert MATLAB code written for Electrical Impedance Tomography (EIT) into a code for Magnetic Induction Tomography (MIT). EIT involves electrodes with applied alternating currents and requires nodal FEM. Whereas MIT involves coils and induction of eddy currents and requires edge FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code for both EIT and MIT involves solving the forward and inverse problems in a simulation for tomographic image reconstruction of conductivity maps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MATLAB code for conversion is found at EIDORS:&lt;br&gt;&#xA;&lt;a href=&quot;http://eidors3d.sourceforge.net/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eidors3d.sourceforge.net/index.shtml&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23181" LastEditorUserId="9667" LastEditDate="2017-02-06T22:41:28.340" LastActivityDate="2017-06-19T09:56:36.970" Title="Edge and Nodal finite element methods in MATLAB for Magnetic induction tomography" Tags="|finite-element|matlab|inverse-problem|electromagnetism|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26108" PostTypeId="2" ParentId="26105" CreationDate="2017-02-06T23:13:18.690" Score="1" Body="&lt;p&gt;I know of one example where the normals have been included in the fitting procedure. It is not a direct quadric fitting though. A locally parametrized patch is fitted to the points and normals. Using normals gives more equations in the fitting problem, allowing higher order polynomials to be used.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dl.acm.org/citation.cfm?id=966134&quot; rel=&quot;nofollow noreferrer&quot;&gt;A novel cubic-order algorithm for approximating principal direction vectors&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16934" LastEditorUserId="16934" LastEditDate="2017-02-07T01:10:43.290" LastActivityDate="2017-02-07T01:10:43.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26109" PostTypeId="1" AcceptedAnswerId="26112" CreationDate="2017-02-07T01:29:12.933" Score="2" ViewCount="102" Body="&lt;p&gt;Since learning about point charges in my physics II class this semester, I want to be able to investigate not only the static force and field distributions but the actual trajectories of movement of electrically charged particles. The first stage in doing this is to build a naive engine for simulating the dynamics of $n$ individual point particles. I've implemented the solution using matrices in python and was hoping someone could comment on whether I've done so correctly. As I don't know what kind of dynamics to expect, I can't tell directly from the videos that my implementation of my equations is correct. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;My Particular Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In particular, I cannot tell if in my calculation of Force magnitude I am computing the $\frac{1}{r^{3/2}}$ factor correctly. Why? because when I simulate a dipole and use $2/2$ as an exponent the particles start going in an elliptical orbit. &lt;a href=&quot;https://i.stack.imgur.com/pV9Xn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pV9Xn.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; which is what I would expect. However, when I use the correct exponent, I get this: &lt;a href=&quot;https://i.stack.imgur.com/K2Gqu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K2Gqu.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; Where is my code going wrong? What am I supposed to expect&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll first write down the equations I'm using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $n$ charges $q_1, q_2, ..., q_n$, with masses $m_1, m_2, ..., m_n$ located at initial positions $\vec{r}_1, \vec{r}_2, ..., \vec{r}_n$, with velocities $\dot{\vec{r}}_1, \dot{\vec{r}}_2, ..., \dot{\vec{r}}_n$ the force induced on $q_i$ by $q_j$ is given by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{F}_{j \to i} = k\frac{q_iq_j}{\left|\vec{r}_i-\vec{r}_j\right|^2}\hat{r}_{ij}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat{r}_{ij} =\frac{ \vec{r}_i-\vec{r}_j}{\left|r_i-r_j\right|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the net &lt;em&gt;marginal&lt;/em&gt; force on particle $q_i$ is given as the sum of the pairwise forces&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{F}_{N, i} = \sum_{j \ne i}{\vec{F}_{j \to i}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then the net acceleration of particle $q_i$ just normalizes the force by the mass of the particle:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot{\vec{r}}_i = \frac{\vec{F}_{N, i}}{m_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In total, for $n$ particles, we have an n-th order system of differential equations. We will also need to specify $n$ initial particle velocities and $n$ initial positions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To implement this in python, I need to be able to compute pairwise point distances and pairwise charge multiples. To do this I tile the $\mathbf{q}$ vector of charges and the $\mathbf{r}$ vector of positions and take, respectively, their product and difference with their transpose. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;!-- language: python --&amp;gt;&#xA;&#xA;def integrator_func(y, t, q, m, n, d, k):&#xA;    y = np.copy(y.reshape((n*2,d)))&#xA;&#xA;    # rj across, ri down&#xA;    rs_from = np.tile(y[:n], (n,1,1))&#xA;&#xA;    # ri across, rj down&#xA;    rs_to = np.transpose(rs_from, axes=(1,0,2))&#xA;&#xA;    # directional distance between each r_i and r_j&#xA;    # dr_ij is the force from j onto i, i.e. r_i - r_j&#xA;    dr = rs_to - rs_from&#xA;&#xA;    # Used as a mask to ignore divides by zero between r_i and r_i&#xA;    nd_identity = np.eye(n).reshape((n,n,1))&#xA;&#xA;    # WHAT I AM UNSURE ABOUT&#xA;    drmag = ma.array(&#xA;        np.power(&#xA;            np.sum(np.power(dr, 2), 2)&#xA;        ,3./2)&#xA;        ,mask=nd_identity)&#xA;&#xA;    # Pairwise q_i*q_j for force equation&#xA;    qsa = np.tile(q, (n,1))&#xA;    qsb = np.tile(q, (n,1)).T&#xA;    qs = qsa*qsb&#xA;&#xA;    # Directional forces&#xA;    Fs = (k*qs/drmag).reshape((n,n,1))&#xA;&#xA;    # Dividing by m to obtain acceleration vectors&#xA;    a = np.sum(Fs*dr, 1)&#xA;&#xA;    # Setting velocities&#xA;    y[:n] = np.copy(y[n:])&#xA;&#xA;    # Entering the acceleration into the velocity slot&#xA;    y[n:] = np.copy(a)&#xA;&#xA;    # Flattening it out for scipy.odeint to work properly&#xA;    return np.array(y).reshape(n*2*d)&#xA;&#xA;def sim_particles(t, r, v, q, m, k=1.):&#xA;    &quot;&quot;&quot;&#xA;    With n particles in d dimensions:&#xA;&#xA;    t: timepoints to integrate over&#xA;    r: n*d matrix. The d-dimensional initial positions of n particles&#xA;    v: n*d matrix of initial particle velocities&#xA;    q: n*1 matrix of particle charges&#xA;    m: n*1 matrix of particle masses&#xA;    k: electric constant.&#xA;    &quot;&quot;&quot;&#xA;&#xA;    d = r.shape[-1]&#xA;    n = r.shape[0]&#xA;    y0 = np.zeros((n*2,d))&#xA;    y0[:n] = r&#xA;    y0[n:] = v&#xA;    y0 = y0.reshape(n*2*d)&#xA;    yf = odeint(&#xA;        integrator_func,&#xA;        y0,&#xA;        t,&#xA;        args=(q,m,n,d,k)).reshape(t.shape[0],n*2,d)&#xA;    return yf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19718" LastActivityDate="2017-02-07T09:12:41.237" Title="Is this a proper implementation of point charge dynamics with ODEs" Tags="|computational-physics|integration|electromagnetism|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26112" PostTypeId="2" ParentId="26109" CreationDate="2017-02-07T09:12:41.237" Score="3" Body="&lt;p&gt;It's an interesting question - how does a dipole actually move? It seems to me you're not entirely sure what to expect, so we need to get a solid test case where we understand what's actually going on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a related note, do try to add your imports and initial conditions the next time :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, let's assume both masses equal and opposite charges. We can then test out circular motion (think two point bodies acting on each other by gravity, I'm sure you can see the similarity):&#xA;$$&#xA;F = \frac{mv_{orbital}^2}{r} = \frac{k q^2}{r^2} \implies v_{orbital} = \sqrt{\frac{k q^2}{m r}}&#xA;$$&#xA;So it's easy to see that if we see all the variables on the right equal to 1 (charges equal to $\pm 1$), the orbital velocity should be 1. So if I run these initial conditions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t = np.linspace(0, 5, 500)&#xA;N = 2 #particles&#xA;d = 2 #dimensions&#xA;&#xA;q = np.array([-1.,1.])&#xA;m = np.ones_like(q)&#xA;k = 1.&#xA;r = np.array([[0.5,0],[-0.5,0]]) # initial distance equal to 1&#xA;distance = np.linalg.norm(r[0] - r[1])&#xA;print(distance) # should return 1&#xA;v_orbital_value = (k*q**2/m/distance)**0.5&#xA;print(v_orbital_value) #should also return 1&#xA;v = np.array([[0.,1.],[0.,-1.]]) * v_orbital_value&#xA;&#xA;y = sim_particles(t, r, v, q, m, k)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get, seemingly disappointingly...&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rt06F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rt06F.png&quot; alt=&quot;Whoops.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then you realize that neither particle is stationary, and $v_{orbital}$ should refer to their &lt;em&gt;relative&lt;/em&gt; velocity. Upon dividing it by 2, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gWTno.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gWTno.png&quot; alt=&quot;Circular like something quite circular.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both particles exhibit circular motion, as it should be if we're to have a stationary center of mass (which you can check easily, I'll just leave this line here):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plt.plot(0.5*(x1+x2), 0.5*(y1+y2), &quot;go&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To sum up, your code is quite fine (and written pretty neatly, I must add)! Although the ellipsoids you get from the $\frac{1}{r}$ force law look pretty, they're not what we were expecting. Always have some preexisting idea of how to test your code's results.&lt;/p&gt;&#xA;" OwnerUserId="22644" LastActivityDate="2017-02-07T09:12:41.237" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26113" PostTypeId="1" CreationDate="2017-02-07T10:13:22.277" Score="2" ViewCount="72" Body="&lt;p&gt;Suppose I have a multi-objective optimization problem, and I wish to find solutions using two different methods/algorithms. The result of each algorithms is a Pareto front. Comparing two different methods for a single-objective optimization problem is simple: just see which one produces the better result for the single objective. However, for a multi-objective problem, one would need to compare the resulting Pareto fronts, supposedly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question then is: is it possible to compare two Pareto fronts? How would you say that one algorithm performs &quot;better&quot; than the other? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks, and any insight is valued.&lt;/p&gt;&#xA;" OwnerUserId="18208" LastActivityDate="2017-02-07T10:13:22.277" Title="Comparing the solutions to a multi-objective optimization problem" Tags="|optimization|algorithms|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26115" PostTypeId="1" CreationDate="2017-02-07T11:31:19.123" Score="10" ViewCount="1775" Body="&lt;p&gt;I am looking for methods that allow estimating the information entropy of a distribution when the only practical ways of sampling from that distribution are Monte Carlo methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is not unlike the standard Ising model that is typically used as the introductory example for Metropolis–Hastings sampling.  I have a probability distribution over a set $A$, i.e. I have $p(a)$ for each $a \in A$.  The elements $a \in A$ are of a combinatorial nature, like Ising states, and there is a very high number of them. This means that in practice I never get the same sample twice when sampling from this distribution on a computer. $p(a)$ cannot be directly computed (due to not knowing the normalization factor), but the ratio $p(a_1)/p(a_2)$ is easy to calculate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to estimate the information entropy of this distribution,&#xA;$$&#xA;S = -\sum_{a \in A} p(a) \ln p(a).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, I want to estimate the entropy difference between this distribution and one obtained by restricting it to a subset $a\in A_1 \subset A$ (and of course re-normalizing).&lt;/p&gt;&#xA;" OwnerUserId="23191" LastEditorUserId="549" LastEditDate="2017-02-08T16:27:19.917" LastActivityDate="2017-02-11T15:34:00.527" Title="Estimate information entropy through Monte Carlo sampling" Tags="|monte-carlo|random-sampling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26116" PostTypeId="1" AcceptedAnswerId="26117" CreationDate="2017-02-07T13:52:28.607" Score="4" ViewCount="178" Body="&lt;p&gt;What would be the numerical method of choice to find minima in a non-smooth, non-convex, locally Lipschitz function $f: \mathbb{R}^n\rightarrow \mathbb{R}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $f$ is mostly smooth but contains three-dimensional cusps of the following form:&lt;br&gt;&#xA;$$&#xA;g: \mathbb{R}^3\rightarrow \mathbb{R}\\&#xA;g(\boldsymbol{x})= -\exp(-{\lVert \boldsymbol x \rVert}_2)&#xA;$$&#xA;with $\lVert \cdot \rVert_2$ being the Euclidean Norm.&lt;/p&gt;&#xA;" OwnerUserId="23195" LastEditorUserId="9667" LastEditDate="2017-02-07T16:56:56.697" LastActivityDate="2017-02-07T16:56:56.697" Title="Optimization of non-smooth, non-convex, locally Lipschitz functions of type exp(-abs(x))" Tags="|optimization|nonconvex|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26117" PostTypeId="2" ParentId="26116" CreationDate="2017-02-07T14:59:03.937" Score="3" Body="&lt;p&gt;I think I found the right article myself:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://link.springer.com/article/10.1007%2Fs10107-012-0514-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lewis, A.S. &amp;amp; Overton, M.L. Math. Program. (2013) 141: 135. doi:10.1007/s10107-012-0514-2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23195" LastActivityDate="2017-02-07T14:59:03.937" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26118" PostTypeId="2" ParentId="25480" CreationDate="2017-02-07T17:43:29.017" Score="1" Body="&lt;p&gt;This holds if &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$f$ is &lt;strong&gt;convex, proper and lower semi-continuous&lt;/strong&gt; (in which case the subdifferential is weakly-strongly closed) and&lt;/li&gt;&#xA;&lt;li&gt;$x_n\to \bar x$ &lt;strong&gt;strongly&lt;/strong&gt; (in particular, if $\{x_n\}\subset \mathbb{R}^N$).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(If $\bar x$ is just an accumulation point, you can apply this argument to the subsequence converging to it.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2017-02-07T17:43:29.017" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26119" PostTypeId="1" AcceptedAnswerId="26124" CreationDate="2017-02-07T19:29:59.187" Score="6" ViewCount="9426" Body="&lt;p&gt;My question may be so general and simple but I'm really confused about the meaning of the &quot;&lt;em&gt;stability&lt;/em&gt;&quot;. I look that up in the Internet but there was no general answer to this question. Can anyone help me to understand its meaning or definition and its types or references about the subject? And how is it determined?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example what is &quot;&lt;em&gt;stability&lt;/em&gt;&quot; in spectral methods where we use a set of functions to approximate the solution or in other methods?&lt;/p&gt;&#xA;" OwnerUserId="22763" LastEditorUserId="9667" LastEditDate="2017-02-07T22:29:16.083" LastActivityDate="2019-04-21T15:00:14.310" Title="What is the meaning of stability in numerical analysis? How to deterimne the stability of a numerical method?" Tags="|stability|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26120" PostTypeId="1" AcceptedAnswerId="26121" CreationDate="2017-02-07T20:05:03.550" Score="1" ViewCount="94" Body="&lt;p&gt;I had a similar question to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26049/finite-element-method-applied-to-1d-structural-problem-what-is-wrong-with-body&quot;&gt;Finite element method applied to 1D structural problem - what is wrong with body force?&lt;/a&gt; I did some work on it as per answers, but it seems that is not working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quadratic 1D element runs from 1 to 5. Nodes are $x_1=1, x_2=3,x_3=5$. $x_{global} = [1,3,5]^T$ Element length $h=4$, Jacobian for given element $J=h/2=2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$N_1(x) = \frac{(x-x_2)(x-x_3)}{(x_1-x_2)(x_1-x_3)}$$&#xA;$$N_2(x) = \frac{(x-x_1)(x-x_3)}{(x_2-x_1)(x_2-x_3)}$$&#xA;$$N_3(x) = \frac{(x-x_1)(x-x_2)}{(x_3-x_1)(x_3-x_2)}$$&#xA;$$N_1(\xi)= -\frac{1}{2}\xi(1-\xi)$$&#xA;$$N_2(\xi) = (1-\xi^2)$$&#xA;$$N_3(\xi) = \frac{1}{2}\xi(1+\xi)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N^T = [N_1 \hspace{6pt} N_2 \hspace{6pt} N_3]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Body force term $b=5x^3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = \int_1^5 N(x)^T \times(5x^3)dx  $$&#xA;I get $f(x) = [34.0 \hspace{6pt}  456.0 \hspace{6pt} 358.0 ]^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I writee $x=\sum N(\xi)x_{global}$, after simplification, I get $x=x_1 + (h/2)(1+\xi)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, if I integrate problem in isoparametric coordinates,&#xA;$$f(\xi) = \int_{-1}^1 N(\xi)^T \times 5 \times \big ( x_1 + (h/2)(1+\xi) \big )^3 \times J d\xi  $$&#xA;I get $f(\xi) = [-18.0 \hspace{6pt}  184.0 \hspace{6pt} 174.0 ]^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, $f(x),f(\xi)$ are not equal.  If I make $b$ constant, then both force terms are equal. But not when $b$ is nonlinear. Can someone comment what went wrong?&lt;/p&gt;&#xA;" OwnerUserId="23200" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-02-07T22:32:31.893" Title="Integrating nonlinear force term in FEM" Tags="|finite-element|integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26121" PostTypeId="2" ParentId="26120" CreationDate="2017-02-07T22:32:31.893" Score="3" Body="&lt;p&gt;I get the same for the force vector with either approach. Check your calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{f}^e = [-34 \quad 456 \quad 358]^T&#xA;$$&#xA;The sum of the three components is 780 which is also the integral $\int_1^5 f(x) dx$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other comments: note that $f$ is neither a function of $x$ nor of $\xi$ in the integrals in your post. The first component of the force vector is -34 and not +34.&lt;/p&gt;&#xA;" OwnerUserId="1156" LastActivityDate="2017-02-07T22:32:31.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26122" PostTypeId="1" CreationDate="2017-02-08T09:19:57.580" Score="1" ViewCount="579" Body="&lt;p&gt;In a collocated grid, which velocity is used in the convective term in the momentum equation? Is the Rhie and Chow constructed face velocity or an average of the adjacent cell center values(in a structured grid) used?&lt;/p&gt;&#xA;" OwnerUserId="8166" LastActivityDate="2017-02-08T09:19:57.580" Title="Rhie and Chow Pressure Velocity Coupling" Tags="|fluid-dynamics|finite-volume|discretization|navier-stokes|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26123" PostTypeId="1" CreationDate="2017-02-08T11:40:18.343" Score="1" ViewCount="48" Body="&lt;p&gt;I was wondering if there are any shallow water equation solvers out there that can include rigid body motion for the surface the water flows on?&lt;/p&gt;&#xA;" OwnerUserId="23206" LastActivityDate="2017-02-08T11:40:18.343" Title="Shallow water equations with moving body" Tags="|numerical-analysis|fluid-dynamics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26124" PostTypeId="2" ParentId="26119" CreationDate="2017-02-08T11:42:19.223" Score="6" Body="&lt;p&gt;You can define a method/algorithm  stable if during the various steps it not amplifies excessively the errors on the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you introduce some bounded estimate.&#xA;Note that this kind of errors, monitored by the stability check, are different from an error occurred by the ill-condition of a problem,&#xA;i.e. in the stability you check the errors propagated by the algorithm itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing to note is that stability is connected to the problem, for example the same method is stable for a problem and unstable for a different problem,&#xA;see chapter 1.16 of [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As reference can be [1]. The book of Kendal, [2], cover different arguments and you can find the relative stability proof (I suggest it because I like it). In general books over specific argument have got the stability proof for the methods covered.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;em&gt;Nicholas J. Higham&lt;/em&gt;, MR 1927606 &lt;a href=&quot;http://dx.doi.org/10.1137/1.9780898718027&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Accuracy and stability of numerical algorithms&lt;/strong&gt;&lt;/a&gt;, ISBN: 0-89871-521-0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Atkinson, Kendall, and Weimin Han. Theoretical numerical analysis. Vol. 39. Berlin: Springer, 2005.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2017-02-08T11:42:19.223" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26126" PostTypeId="1" AcceptedAnswerId="26127" CreationDate="2017-02-08T18:37:02.983" Score="3" ViewCount="1413" Body="&lt;p&gt;I am currently following J.Anderson Jr.'s CFD with basic application and I came into some troubles while coding for my very first CFD problem. As the title suggests I am solving an incompressible Couette flow using an explicit finite difference approach. How can I increase the steps without it blowing up? I tried to change the delta_t but it does not go any greater than what I've prescribed in the code without it going haywire any  advice will be appreciated.&#xA;Oh and the B.c for the upper plate is 2 unit/t&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the governing equation. After nondimensionalizing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cfrac%7B%5Cpartial&amp;space;u&amp;space;%7D%7B%5Cpartial&amp;space;t%7D&amp;space;=&amp;space;%5Cfrac%7B1%7D%7BRe%7D&amp;space;%5Cfrac%7B%5Cpartial&amp;space;u%5E2&amp;space;%7D%7B%5Cpartial&amp;space;y%5E2%7D&quot; title=&quot;\frac{\partial u }{\partial t} = \frac{1}{Re} \frac{\partial u^2 }{\partial y^2}&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And using second order central difference &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?u_%7Bi%7D%5E%7Bn&amp;plus;1%7D=%5Cfrac%7B%5CDelta&amp;space;t%7D%7BRe%7D%5Cleft&amp;space;(&amp;space;%5Cfrac%7Bu_%7Bi&amp;plus;1%7D%5E%7Bn%7D-2u_%7Bi%7D%5E%7Bn%7D&amp;plus;u_%7Bi-1%7D%5E%7Bn%7D%7D%7B(%5CDelta&amp;space;y)%5E%7B2%7D%7D&amp;space;%5Cright&amp;space;)&amp;plus;u_%7Bi%7D%5E%7Bn%7D&quot; title=&quot;u_{i}^{n+1}=\frac{\Delta t}{Re}\left ( \frac{u_{i+1}^{n}-2u_{i}^{n}+u_{i-1}^{n}}{(\Delta y)^{2}} \right )+u_{i}^{n}&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stability criterion for parabolic equation like this is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5CDelta&amp;space;t&amp;space;%5Cleq&amp;space;%5Cfrac%7B1%7D%7B2%7D&amp;space;Re&amp;space;%5Cleft&amp;space;(&amp;space;%5CDelta&amp;space;y&amp;space;%5Cright&amp;space;)%5E%7B2%7D&quot; title=&quot;\Delta t \leq \frac{1}{2} Re \left ( \Delta y \right )^{2}&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here's the code I have written for the central difference method referencing 12 steps to CFD by Lorena Barb&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from matplotlib import pyplot as py&#xA;D=1&#xA;N=21&#xA;Rey=5000&#xA;dy=D/(N-1)&#xA;dt=Rey*(dy)**2&#xA;&#xA;u= np.zeros(N)&#xA;un=np.zeros(N)&#xA;un[20]=2&#xA;u[20]=2&#xA;for n in range(N-1):&#xA;    un = u.copy()&#xA;    for i in range(1, N-1):&#xA;       u[i] = un[i] + dt/Rey/dy**2 * (un[i+1] - 2 * un[i] + un[i-1])&#xA;&#xA;&#xA;py.plot(u/2, np.linspace(0, 2, N)/2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the graph I get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5ZHd6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5ZHd6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution graph is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/curXu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/curXu.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realized where I went wrong... my matrix was all messed up I was using forward time central space scheme and I did not account for my time only my space... anyway this seems to work fine&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from matplotlib import pyplot as plt&#xA;&#xA;y_max=1&#xA;t_max=1&#xA;N=101&#xA;Rey=5&#xA;dy=y_max/(N-1)&#xA;dt=0.5*Rey*(dy)**2&#xA;u0=2&#xA;&#xA;def Couette_Flow(dt,dy,y_max,t_max,u0,Rey):&#xA;    c=dt/Rey/dy**2&#xA;    y=np.arange(0, y_max+dy,dy)&#xA;    t=np.arange(0, t_max+dt,dt)&#xA;    r=len(y)&#xA;    s=len(t)&#xA;    u=np.zeros([s,r])&#xA;    u[:,0]=u0     &#xA;    for n in range(0,s-1):&#xA;         for j in range(1, r-1):&#xA;             u[n+1,j] = u[n,j] + c*(u[n,j+1]-2*u[n,j]+u[n,j-1])&#xA;    return y,u,r,s&#xA;&#xA;&#xA;&#xA;&#xA;y,u,r,s = Couette_Flow(dt,dy,y_max,t_max,u0,Rey)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is not perfect still though I think the fact that I nondimensionalized my equation is messing with my solution, but nonetheless for small value of reynolds number it seems to generate the graph I want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ipzfz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ipzfz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23212" LastEditorUserId="-1" LastEditDate="2017-03-10T09:42:38.190" LastActivityDate="2017-02-12T03:17:58.497" Title="1-D incompressible unsteady Couette Flow Explicit finite differece CFD" Tags="|finite-difference|fluid-dynamics|python|numerical-modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26127" PostTypeId="2" ParentId="26126" CreationDate="2017-02-08T18:48:05.620" Score="2" Body="&lt;p&gt;The answer is that you cannot increase the time step past the limit:&#xA;$$ \Delta t &amp;lt; \frac{1}{2Re} \Delta y^2$$&#xA;This is because you are using an explicit scheme. Explicit scheme have a stability condition, which requires that the amplification factor of the whole scheme be below 1 in order to ensure stability of the scheme. In this case, this leads to the criterion you have on $\Delta t$. If you wish to use a larger time step, you need to use an implicit scheme. This will lead to the resolution of a linear system of equations.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2017-02-08T18:48:05.620" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26128" PostTypeId="1" CreationDate="2017-02-08T20:58:39.273" Score="13" ViewCount="378" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am solving a variant of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ornstein%E2%80%93Zernike_equation&quot;&gt;Ornstein-Zernike&lt;/a&gt; equation from liquid theory. Abstractly, the problem can be represented as solving the fixed point problem $A c(r)=c(r)$, where $A$ is an integro-algebraic operator and $c(r)$ is the solution function (the OZ direct correlation function). I am solving by Picard iteration, where I provide an initial trial solution $c_0(r)$ and generate new trial solutions by the scheme&#xA;$$&#xA;c_{j+1} = \alpha (A c_j) + (1-\alpha)c_j~,&#xA;$$&#xA;where $\alpha$ is an adjustable parameter that controls the mix of $c$ and $Ac$ used in the next trial solution. For this discussion, let's assume that the value of $\alpha$ is unimportant.&#xA;I repeat until the iteration converges to within a desired tolerance, $\epsilon$:&#xA;$$&#xA; \Delta_{j+1} \equiv \int d\vec r |c_{j+1}(r)-c_j(r)| &amp;lt; \epsilon~.&#xA;$$&#xA;In my variant of the problem, $A$ depends on a parameter $\lambda$, and my question is about how the convergence of $Ac=c$ depends on this parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a wide range of values for $\lambda$, the iteration scheme above converges exponentially quickly. However, as I decrease $\lambda$, I eventually reach a regime in which the convergence is non-monotonic, pictured below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lnuMD.png&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lnuMD.png&quot; alt=&quot;onset of non-monotone convergence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Key Questions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In iterative solutions to fixed-point problems, does non-monotonic convergence have any special significance? Does it signal that my iterative scheme is on the verge of instability? &lt;strong&gt;Most importantly&lt;/strong&gt;, should non-monotone convergence make me suspicious that the &quot;converged&quot; solution is not a good solution to the fixed-point problem?&lt;/p&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2018-01-05T13:35:15.917" Title="Non-monotonic convergence in fixed-point problem" Tags="|iterative-method|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26129" PostTypeId="1" AcceptedAnswerId="26131" CreationDate="2017-02-08T23:07:05.003" Score="3" ViewCount="81" Body="&lt;p&gt;I have to minimize a function $f(\mathbf{x})$, where the vector $\mathbf{x}\in\mathbb{R}^n$ satisfies $|\mathbf{x}|=1$. So I tweaked the code of $f$ so that it renormalizes $\mathbf{x}$ as the first step, and this allows me to &lt;em&gt;avoid&lt;/em&gt; adding the constraint to the minimization algorithm. At the end of the optimization I can simply renormalize the result. This is working well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'd like to help the algorithm further by supplying the gradients of $f$. I calculated them naively by hand and I'm happy that they turned out to be somewhat simple functions, but I'm having troubles because my &quot;code for $f$&quot; effectively computes $g(\mathbf{x}) = f(\mathbf{x}/|\mathbf{x}|)$, while the gradient that I calculated is actually the gradient of $f(\mathbf{x})$. Can I get around this problem without having to calculate the gradient of $g$?&lt;/p&gt;&#xA;" OwnerUserId="22811" LastEditorUserId="22811" LastEditDate="2017-02-08T23:55:34.147" LastActivityDate="2017-02-09T00:29:54.697" Title="Gradient of function after renormalization of variables" Tags="|nonlinear-programming|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26130" PostTypeId="2" ParentId="26129" CreationDate="2017-02-08T23:47:54.557" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Does your optimization guarantee that $|\mathbf{x}|=1$ all the time? If not you have to renormalize at each step and actually your function changes.&lt;/li&gt;&#xA;&lt;li&gt;If you are always on the L1-ball, then simply treat $\mathbf{x}_{new}=\mathbf{x}/|\mathbf{x}|$ and optimize $f(\mathbf{x}_{new})$. This will not change your optimization. You could scale back at the end of optimization by the L1-norm to get back to the scale of the data.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2017-02-08T23:47:54.557" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26131" PostTypeId="2" ParentId="26129" CreationDate="2017-02-09T00:29:54.697" Score="5" Body="&lt;p&gt;Based on your current optimization strategy, it is likely you cannot get away with just the gradient of $f(\cdot)$ evaluated at $\frac{\boldsymbol{x}}{\left| \boldsymbol{x}\right|}$ since normalization isn't linear wrt. $\boldsymbol{x}$. However, you could try using chain rule by first assuming that $f(\boldsymbol{u}(\boldsymbol{x}))$ where $\boldsymbol{u}(\boldsymbol{x}) = \frac{\boldsymbol{x}}{\left| \boldsymbol{x}\right|}$. Then the gradient result you say you derived would actually represent $\vec{\nabla}_{u} f$, compared to $\vec{\nabla}_{x} f$ which you want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using chain rule, we can show the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial f}{\partial x_{p}} &amp;amp;= \frac{\partial f}{\partial u_{k}} \frac{\partial u_{k}}{\partial x_{p}}  \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can find $\frac{\partial u_{k}}{\partial x_{p}} \;\forall k,p$, since $u_k = x_{k} \left(x_l x_l\right)^{-1/2} \; \forall k$, doing the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial u_{k}}{\partial x_{p}} &amp;amp;= \frac{\partial}{\partial x_{p}} \left( x_{k} \left(x_l x_l\right)^{-1/2} \right) \\&#xA;&amp;amp;=  \delta_{kp} \left(x_l x_l\right)^{-1/2} - x_{k} x_{p} \left(x_l x_l\right)^{-3/2} \\&#xA;&amp;amp;=  \left(\delta_{kp} - \frac{x_{k} x_{p}}{\left(x_l x_l\right)} \right) \left(x_l x_l\right)^{-1/2}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This expression can be simplified into the following in matrix form:&#xA;\begin{align}&#xA;\frac{\partial \boldsymbol{u}}{\partial \boldsymbol{x} } &amp;amp;= \frac{1}{\left| \boldsymbol{x}\right|} \left( I - \frac{\boldsymbol{x} \boldsymbol{x}^{T}}{\left| \boldsymbol{x}\right|^2}\right)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, assuming you are defining the gradients as column vectors, you get the following relationship to compute what you want:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\vec{\nabla}_{x} f &amp;amp;= \frac{1}{\left| \boldsymbol{x}\right|} \left( I - \frac{\boldsymbol{x} \boldsymbol{x}^{T}}{\left| \boldsymbol{x}\right|^2}\right)&#xA; \vec{\nabla}_{u} f&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-02-09T00:29:54.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26132" PostTypeId="1" CreationDate="2017-02-09T03:57:14.430" Score="1" ViewCount="177" Body="&lt;p&gt;In the DEAP algorithms (&lt;a href=&quot;http://deap.readthedocs.io/en/master/api/algo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;see documentation here&lt;/a&gt;), I notice that we need to specify the the number of generations (NGEN). I was advised that convergence has been achieved if the Pareto curve is smooth. How can I monitor for convergence in DEAP? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update:&#xA;It is possible to monitor for convergence by specifying a &quot;smoothness&quot; value in the stats. However, I'm still confused about how to define &quot;smoothness&quot;. For example, consider the &lt;a href=&quot;http://deap.readthedocs.io/en/master/examples/ga_knapsack.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knapsack problem specified here&lt;/a&gt;. How do we monitor for smoothness in this example?&lt;/p&gt;&#xA;" OwnerUserId="18208" LastActivityDate="2017-02-09T03:57:14.430" Title="How to test for convergence (smoothness of Pareto front) in DEAP" Tags="|optimization|python|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26133" PostTypeId="2" ParentId="26128" CreationDate="2017-02-09T07:53:33.533" Score="1" Body="&lt;p&gt;Suppose $x$ is the unknown independent variable in the solution of $x=f(x)$, then fixed point method will converge from a point $x^*$ provided that the Jacobian $\frac{\partial f}{\partial x}(x^*) \le \alpha$, where $\alpha$ is a constant $&amp;lt;1$. In general $x^*$ is not a single point, but the domain traversed by the iterative scheme.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your solution &lt;em&gt;is&lt;/em&gt; converging, albeit non-monotonically. Check your Jacobian for various values of $\lambda$ and the solution variable to see if you go from satisfying convergence criteria to not satisfying it, which might explain what you are seeing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If your solution has converged within a properly established relative tolerance which also accounts for small numbers, then it has.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20098" LastActivityDate="2017-02-09T07:53:33.533" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26134" PostTypeId="1" CreationDate="2017-02-09T08:33:15.140" Score="3" ViewCount="143" Body="&lt;p&gt;Consider a random variable which is given by an orthogonal polynomial expansion in one parameter (or polynomial chaos expansion PCE), i.e. , &#xA;$$ f(\alpha) = \sum\limits_{n=0}^{\infty} \hat{f} (n) \psi _n (\alpha) \, , $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha$ is uniformly distributed in $[-1,1]$,  $\psi _n (\alpha)$ are the orthogonal Legendre polynomials $$\hat{f} = \frac{1}{2}(f,\psi _n )_{L^2[-1,1]}  \, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; is there a closed form formula or an approximation for the cumulative distribution function (CDF) or the probability distribution function (PDF) of $f$ in a closed form, using only $\psi _n$ and $\hat{f} (n)$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is somewhat related to &lt;a href=&quot;https://scicomp.stackexchange.com/q/26003/15188&quot;&gt;this post&lt;/a&gt;, and was asked by me before on MO &lt;a href=&quot;https://mathoverflow.net/q/248496/42864&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15188" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2017-02-09T08:33:15.140" Title="Closed form PDF/CDF using Orthogonal Polynomial Expansion (gPC)" Tags="|numerical-analysis|polynomials|spectral-method|probability|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26135" PostTypeId="1" AcceptedAnswerId="26143" CreationDate="2017-02-09T09:49:52.577" Score="1" ViewCount="278" Body="&lt;p&gt;I am writing a GAMS program where I am interested in using the value of a variable as a condition inside an another equation. Let's say I have two equations with two variables, $g_1(t)$ and $g_2(t)$, and they are defined over a set of time $t$. The sample equations are written below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;g_1(t) &amp;amp;= g_1(t-1)+1 \\&#xA;g_2(t) &amp;amp;= g_2(t-1)+20 \;\$\left(g_1(t) \gt 20\right)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge, I do not think I can use any variable (endogenous) inside $\$(\cdot)$ operator but if my algorithm demands the way as I mentioned above, is there any way proceed? Or are there any other ideas how to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="23220" LastEditorUserId="9804" LastEditDate="2017-02-09T16:46:51.503" LastActivityDate="2017-02-09T16:56:04.740" Title="GAMS Optimization" Tags="|optimization|simulation|optimal-control|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26136" PostTypeId="2" ParentId="26119" CreationDate="2017-02-09T10:22:36.200" Score="2" Body="&lt;p&gt;If in a numerical method, the solution $\phi$ at an iteration level $n+1$ is given by,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B_1 \phi^{n+1}=B_2 \phi^n +f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then for an initial hypothetical perturbation $\delta^0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B_1 \delta^{n+1}=B_2 \delta^n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the numerical method is stable if for all $\delta^0$, $\delta^n$ remains bounded as $n\rightarrow \infty$&lt;/p&gt;&#xA;" OwnerUserId="8166" LastEditorUserId="8166" LastEditDate="2017-02-24T07:09:33.540" LastActivityDate="2017-02-24T07:09:33.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26137" PostTypeId="1" AcceptedAnswerId="26139" CreationDate="2017-02-09T11:51:41.347" Score="17" ViewCount="1999" Body="&lt;p&gt;Given two different BLAS implementations, can we expect that they make the exact same floating point computations and return the same results? Or can it happen, for instance, that one computes a scalar product as&#xA;$$&#xA;((x_1y_1 + x_2y_2) + x_3y_3) + x_4y_4&#xA;$$&#xA;and one as&#xA;$$&#xA;(x_1y_1 + x_2y_2) + (x_3y_3 + x_4y_4),&#xA;$$&#xA;so possibly giving different result in IEEE floating point arithmetic?&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-02-09T17:09:23.853" Title="Are BLAS implementations guaranteed to give the exact same result?" Tags="|floating-point|blas|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26138" PostTypeId="2" ParentId="26107" CreationDate="2017-02-09T12:55:53.433" Score="3" Body="&lt;p&gt;The paper: I. Anjam, J. Valdman, &quot;Fast MATLAB assembly of FEM matrices in 2D and 3D: Edge elements&quot;, Applied Mathematics and Computation, 267, (2015), 252–263; states the following.&#xA;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0096300315004191&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0096300315004191&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;A finite element discretization is done in terms of edge elements, typically Raviart–Thomas elements [12] for H(div) problems and Nédélec elements [9] for H(curl) problems. &lt;strong&gt;Edge element basis functions are not defined on the nodes of 2D triangular or 3D tetrahedral meshes, but on edges and faces&lt;/strong&gt;. Edge elements provide only partial continuity over element boundaries: continuity of normal vector component for H(div) problems and continuity of tangential vector component for H(curl) problems.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They give a figure for this representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lw8vd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lw8vd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edge elements are derived from nodal shape functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://www.iue.tuwien.ac.at/phd/orio/node48.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;4.1.3 Shape Function&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;In Edge FEM on a tetrahedral mesh, a vector field is&#xA;represented using a basis vector function&quot; (1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N_{ij} = L_i\bigtriangledown L_j - L_j\bigtriangledown L_i$ ____________ (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear shape function of the node $i$ has the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L_i (x,y,z) = a_i + b_i x + c_i y + d_i z$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HWFKs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HWFKs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) &lt;a href=&quot;http://opus.bath.ac.uk/43926/&quot; rel=&quot;nofollow noreferrer&quot;&gt;L. Ma, “Magnetic Induction Tomography for Non-destructive Evaluation and Process Tomography,” University of Bath, 2014, pp.12.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23181" LastEditorUserId="23181" LastEditDate="2017-06-19T09:56:36.970" LastActivityDate="2017-06-19T09:56:36.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26139" PostTypeId="2" ParentId="26137" CreationDate="2017-02-09T15:01:39.753" Score="15" Body="&lt;p&gt;No, that is not guaranteed. If you are using a NETLIB BLAS without any optimizations, it it mostly true that the results are the same. But for any practical usage of BLAS and LAPACK one uses a highly optimized an parallel BLAS. The parallelization causes, even if it only works in parallel inside the vector registers of a CPU, that the order how the single terms are evaluated changes and the order of the summation changes too. Now it follows form the missing associative property in the IEEE standard that the results are not the same. So exactly the thing you mentioned can happen. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the NETLIB BLAS the scalar product is only a for loop unrolled by a factor 5:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DO I = MP1,N,5&#xA;          DTEMP = DTEMP + DX(I)*DY(I) + DX(I+1)*DY(I+1) +&#xA;     $            DX(I+2)*DY(I+2) + DX(I+3)*DY(I+3) + DX(I+4)*DY(I+4)&#xA;END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it is up to the compiler if each multiplication is added to DTEMP immediately or if all 5 components are summed up first and than added to DTEMP.&#xA;In OpenBLAS it is depending on the architecture a more complicated kernel: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; __asm__  __volatile__&#xA;    (&#xA;    &quot;vxorpd     %%ymm4, %%ymm4, %%ymm4               \n\t&quot;&#xA;    &quot;vxorpd     %%ymm5, %%ymm5, %%ymm5               \n\t&quot;&#xA;    &quot;vxorpd     %%ymm6, %%ymm6, %%ymm6               \n\t&quot;&#xA;    &quot;vxorpd     %%ymm7, %%ymm7, %%ymm7               \n\t&quot;&#xA;&#xA;    &quot;.align 16                           \n\t&quot;&#xA;    &quot;1:                          \n\t&quot;&#xA;        &quot;vmovups                  (%2,%0,8), %%ymm12         \n\t&quot;  // 2 * x&#xA;        &quot;vmovups                32(%2,%0,8), %%ymm13         \n\t&quot;  // 2 * x&#xA;        &quot;vmovups                64(%2,%0,8), %%ymm14         \n\t&quot;  // 2 * x&#xA;        &quot;vmovups                96(%2,%0,8), %%ymm15         \n\t&quot;  // 2 * x&#xA;&#xA;    &quot;vmulpd      (%3,%0,8), %%ymm12, %%ymm12 \n\t&quot;  // 2 * y&#xA;    &quot;vmulpd    32(%3,%0,8), %%ymm13, %%ymm13 \n\t&quot;  // 2 * y&#xA;    &quot;vmulpd    64(%3,%0,8), %%ymm14, %%ymm14 \n\t&quot;  // 2 * y&#xA;    &quot;vmulpd    96(%3,%0,8), %%ymm15, %%ymm15 \n\t&quot;  // 2 * y&#xA;&#xA;    &quot;vaddpd      %%ymm4 , %%ymm12, %%ymm4 \n\t&quot;  // 2 * y&#xA;    &quot;vaddpd      %%ymm5 , %%ymm13, %%ymm5 \n\t&quot;  // 2 * y&#xA;    &quot;vaddpd      %%ymm6 , %%ymm14, %%ymm6 \n\t&quot;  // 2 * y&#xA;    &quot;vaddpd      %%ymm7 , %%ymm15, %%ymm7 \n\t&quot;  // 2 * y&#xA;&#xA;    &quot;addq       $16 , %0	  	     \n\t&quot;&#xA;	&quot;subq	        $16 , %1            \n\t&quot;      &#xA;    &quot;jnz        1b                   \n\t&quot;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which splits the scalar product in small scalar products of length 4 and sum them up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the other typical BLAS implementations like ATLAS, MKL, ESSL,... this problem stays the same because each BLAS implementation uses different optimizations to get fast code. &#xA;But as far as I know one need an artificial example to cause really faulty results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is necessary that the BLAS library returns for the same results ( bit-wise the same) one have to use a reproducible BLAS library such as: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ReproBLAS &lt;a href=&quot;http://bebop.cs.berkeley.edu/reproblas/&quot;&gt;http://bebop.cs.berkeley.edu/reproblas/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;exBLAS &lt;a href=&quot;https://exblas.lip6.fr/&quot;&gt;https://exblas.lip6.fr/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="569" LastEditorUserId="23226" LastEditDate="2017-02-09T17:09:23.853" LastActivityDate="2017-02-09T17:09:23.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26140" PostTypeId="2" ParentId="26137" CreationDate="2017-02-09T15:03:04.827" Score="4" Body="&lt;p&gt;In general, no. Leaving associativity aside, the choice of compiler flags (for example, SIMD instructions being enabled, usage of &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation#Fused_multiply.E2.80.93add&quot; rel=&quot;nofollow noreferrer&quot;&gt;fused multiply add&lt;/a&gt;, etc.) or the hardware (e.g., whether &lt;a href=&quot;https://en.wikipedia.org/wiki/Extended_precision&quot; rel=&quot;nofollow noreferrer&quot;&gt;extended precision&lt;/a&gt; is being used) may produce different results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some efforts to get reproducible BLAS implementations. See &lt;a href=&quot;https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-229.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReproBLAS&lt;/a&gt; and &lt;a href=&quot;https://www.nist.gov/sites/default/files/documents/itl/ssd/is/NRE-2015-04-iakymchuk.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ExBLAS&lt;/a&gt; for more information.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-02-09T15:03:04.827" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26141" PostTypeId="2" ParentId="26137" CreationDate="2017-02-09T15:05:10.493" Score="8" Body="&lt;p&gt;&lt;strong&gt;The Short Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the two BLAS implementations are written to carry out the operations in the exact same order, and the libraries were compiled using the same compiler flags and with the same compiler, then they'll give you the same result. Floating point arithmetic is &lt;em&gt;not&lt;/em&gt; random, so two identical implementations will give identical results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are a variety of things that can break this behavior for the sake of performance...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Longer Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IEEE also specifies the &lt;em&gt;order&lt;/em&gt; in which these operations are carried out, in addition to how each operation should behave. However, if you compile your BLAS implementation with options like &quot;-ffast-math&quot;, the compiler can perform transformations that would be true in exact arithmetic but not &quot;correct&quot; in IEEE floating point. The canonical example is the non-associativity of floating point addition, as you pointed out. With the more aggressive optimization settings, associativity will be assumed, and the processor will do as much of that in parallel as possible by re-ordering the operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other standard-breaking behavior comes via the use of FMA (fused multiply-add) instructions. These are prominent in operations like matrix multiplication, and they have the potential to &lt;em&gt;double&lt;/em&gt; the throughput of your routine. However, they perform the operation $a+b*c$ in a single operation, and it only incurs a &lt;strong&gt;single&lt;/strong&gt; floating point rounding step. This deviates from the IEEE standard, which requires that this operation have two rounding steps. This makes the FMA result actually &lt;em&gt;more&lt;/em&gt; accurate than the IEEE one, but it is technically standard-breaking behavior.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-02-09T15:05:10.493" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26142" PostTypeId="2" ParentId="18850" CreationDate="2017-02-09T15:12:45.550" Score="3" Body="&lt;p&gt;According to the paper &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Nicholas J. Higham&lt;/em&gt;, MR 943997 &lt;a href=&quot;http://dx.doi.org/10.1016/0024-3795(88)90223-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Computing a nearest symmetric positive semidefinite matrix&lt;/strong&gt;&lt;/a&gt;, &lt;em&gt;Linear Algebra Appl.&lt;/em&gt; &lt;strong&gt;103&lt;/strong&gt; (1988), 103--118,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one of the solution is in the following form, &#xA;$$A_{opt} = S + |\lambda| I$$&#xA;where $I$ is the identity matrix, $\lambda$ is the smallest negative eigenvalue of $S$ &lt;/p&gt;&#xA;" OwnerUserId="23223" LastEditorUserId="1804" LastEditDate="2017-02-09T17:13:41.617" LastActivityDate="2017-02-09T17:13:41.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26143" PostTypeId="2" ParentId="26135" CreationDate="2017-02-09T16:56:04.740" Score="0" Body="&lt;p&gt;You could replace the expression $g_1(t) \gt 20$ with $H \left(g_1(t) - 20\right)$ where $H(\cdot)$ is a step function and can be defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;H(x) = \begin{cases}&#xA;1 &amp;amp; \text{if } x \gt 0 \\&#xA;0 &amp;amp; \text{otherwise}&#xA;\end{cases}&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-02-09T16:56:04.740" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26144" PostTypeId="1" CreationDate="2017-02-09T18:07:08.507" Score="1" ViewCount="182" Body="&lt;p&gt;I have the function $c(t) = A \cdot \cos \left(\dfrac{2\pi}{\tau} \cdot t + \phi \right) $&#xA;which is used to define&#xA;$	T(t) =&#xA;	\begin{cases}&#xA;		M + c(t), &amp;amp; c(t) &amp;gt; 0 \\&#xA;		M, &amp;amp;c(t) \leq 0.&#xA;	\end{cases}&#xA;$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T(t)$ is used to fit in the least square sense to measurements $y(t)$. Now, the question is how to get the partial derivative of this piece wise defined function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function is not continuous, therefore I thought to approximate the rect function through a Fourier series in order to calculate the derivative required by the least square procedure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;f(t_i, \gamma) = M + \left[ \dfrac{4h}{\pi} \sum_{k = 1}^{n}{\dfrac{\sin \left( (2k-1)w(t+\Delta t) \right)}{2k-1}} \right] \cdot A \cdot \cos (\omega t+\phi)&#xA;$&#xA;$&#xA;f(t_i, \gamma) = M + \left[ \dfrac{4h}{\pi} \sum_{k = 1}^{n}{x_k\alpha_k+z_k\beta_k} \right] \cdot A \cdot \cos (\omega t+\phi)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &#xA;$&#xA;\alpha_k = A_k\cos(\phi_k),\ \beta_k=A_k\sin(\phi_k),\ x_k = \sin(\omega_kt),\ z_k=\cos(\omega_kt)&#xA;$ and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;S = \sum r_i^2&#xA;$&#xA;with&#xA;$&#xA;r_i = T_i - f(t_i, \gamma),\&#xA;$&#xA;where the adjustable parameters are held in the vector $\gamma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the right way or should I continue with the first try with the piecewise defined function and how to proceed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@Choward: You're right, it was an idea how to avoid the piecewise derivative.&#xA;Let's rewrite the equation above to the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA; T(t) = M + \alpha\cdot x + \beta\cdot z + e\left(t\right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\alpha = A \cdot \cos \left(\phi \right)$, $\beta = -A \cdot \sin \left(\phi \right)$, $x = \cos \left(2 \cdot \pi \cdot \frac{t}{\tau}\right)$ and $z = \sin \left(2 \cdot \pi \cdot \frac{t}{\tau}\right)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For all partial derivatives then I would get: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial f(t_i, \gamma)}{\partial M} = 1$ for $c(t) &amp;gt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial f(t_i, \gamma)}{\partial (A\cos(\phi))} = \cos(\frac{2\pi t}{\tau})$ for $c(t) &amp;gt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial f(t_i, \gamma)}{\partial (-A\sin(\phi))} = \sin(\frac{2\pi t}{\tau})$ for $c(t) &amp;gt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial f(t_i, \gamma)}{\partial M} = 1$ for $ c(t) &amp;lt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial f(t_i, \gamma)}{\partial (A\cos(\phi))} = 0$ for $c(t) &amp;lt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial f(t_i, \gamma)}{\partial (-A\sin(\phi))} = 0$ for $c(t) &amp;lt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I have two sets of equations, one for $c(t) &amp;gt;0$ and $c(t) &amp;lt; 0$. To proceed I continue as following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA; \dfrac{\partial S}{\partial \gamma_i} = 2 \cdot \sum{r_i \dfrac{\partial r_i}{\partial \gamma _j}} \ \forall j \\&#xA;\rightarrow 0 = -2 \cdot \sum{r_i \dfrac{\partial f(T_i, \gamma)}{\partial \gamma _j}} \ \forall j &#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and get for $c(t) &amp;gt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;0 = -2 \cdot \sum_{i}^{N} \left(T_i - \alpha x_i - \beta z_i \right) \cdot \frac{\partial f(t_i, \gamma)}{\partial M} \\&#xA;	\Leftrightarrow\sum_{i}^{N}{T_i}  = M \cdot N + \alpha \cdot \sum_{i}^{N}{x_i} + \beta \cdot \sum_{i}^{N}{z_i}\\&#xA;	 0 = -2 \cdot \sum_{i}^{N} \left(T_i - \alpha x_i - \beta z_i \right) \cdot \frac{\partial f(t_i, \gamma)}{\partial \alpha} \\&#xA;	\Leftrightarrow\sum_{i}^{N}{T_i\cdot x_i} = M \cdot \sum_{i}^{N}{x_i} + \alpha \cdot \sum_{i}^{N}{x_i^2} + \beta \cdot \sum_{i}^{N}{x_i\cdot z_i} \\&#xA;	 0 = -2 \cdot \sum_{i}^{N} \left(T_i - \alpha x_i - \beta z_i \right) \cdot \frac{\partial f(t_i, \gamma)}{\partial \beta} \\&#xA;	\Leftrightarrow \sum_{i}^{N}{T_i\cdot z_i} = M \cdot \sum_{i}^{N}{z_i} + \alpha \cdot \sum_{i}^{N}{x_i\cdot z_i} + \beta \cdot \sum_{i}^{N}{z_i^2}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and for $c(t) &amp;lt; 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;0 = -2 \cdot \sum_{i}^{N} \left(T_i - \alpha x_i - \beta z_i \right) \cdot \frac{\partial f(t_i, \gamma)}{\partial M} \\&#xA;	\Leftrightarrow\sum_{i}^{N}{T_i}  = M \cdot N + \alpha \cdot \sum_{i}^{N}{x_i} + \beta \cdot \sum_{i}^{N}{z_i}\\&#xA;	 0 = -2 \cdot \sum_{i}^{N} \left(T_i - \alpha x_i - \beta z_i \right) \cdot \frac{\partial f(t_i, \gamma)}{\partial \alpha} \\&#xA;	\Leftrightarrow = 0 \\&#xA;	 0 = -2 \cdot \sum_{i}^{N} \left(T_i - \alpha x_i - \beta z_i \right) \cdot \frac{\partial f(t_i, \gamma)}{\partial \beta} \\&#xA;	\Leftrightarrow = 0.&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $c(t) &amp;gt; 0$ I can express it using $d = B \cdot u$&#xA;$&#xA;\begin{pmatrix}&#xA;		\sum_{i}^{N}{T_i} \\ \sum_{i}^{N}{T_i\cdot x_i} \\ \sum_{i}^{N}{T_i\cdot z_i}&#xA;	\end{pmatrix}&#xA;	=&#xA;	\begin{pmatrix}&#xA;		N &amp;amp; \sum_{i}^{N}{x_i} &amp;amp; \sum_{i}^{N}{z_i} \\&#xA;		\sum_{i}^{N}{x_i} &amp;amp; \sum_{i}^{N}{x_i^2} &amp;amp; \sum_{i}^{N}{x_i \cdot z_i} \\&#xA;		\sum_{i}^{N}{z_i} &amp;amp; \sum_{i}^{N}{x_i \cdot z_i} &amp;amp; \sum_{i}^{N}{z_i ^2} \\&#xA;	\end{pmatrix}&#xA;	\cdot&#xA;	\begin{pmatrix}&#xA;		M \\ \alpha \\ \beta&#xA;	\end{pmatrix}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and solve it easily with $u = B^{-1} \cdot d$. For $c(t) &amp;lt; 0$ this does not make sense... How to take this into account?&lt;/p&gt;&#xA;" OwnerUserId="23228" LastEditorUserId="23228" LastEditDate="2017-02-12T20:29:11.530" LastActivityDate="2017-02-12T20:29:11.530" Title="Least square with rectangular function" Tags="|least-squares|curve-fitting|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26145" PostTypeId="1" AcceptedAnswerId="30500" CreationDate="2017-02-09T18:55:06.027" Score="1" ViewCount="413" Body="&lt;p&gt;For a large parallel sparse matrix (mpiaij type matrix) in my code, I was experimenting with various preconditioners to see which one would do best with GMRES/BiCGSTAB.  I tried the &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCMG.html#PCMG&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCMG preconditioner&lt;/a&gt; without specifying any grid/interpolation/restriction information, and this actually seemed to do pretty well for my problem.  At first, I was confused about what this actually did since I didn't give PETSc any information about the grid and whatnot, but I &lt;em&gt;think&lt;/em&gt; that all it is doing is performing a single smoothing step.  Thus, I wanted to know what exactly this &quot;smoother&quot; is since it seems to work fairly well as a preconditioner.  I tried looking through the PCMG documentation but I couldn't figure it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone tell me what the default smoother is for the PCMG preconditioner, or at least offer some advice on how I could figure out what it is?  I've tried scouring through the PETSc documentation without much luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: As noted in the comments below, it seems that it is the Chebyshev smoother.  However, PETSc documentation says &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPCHEBYSHEV.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chebyshev only works for symmetric positive (semi) definite matrices&lt;/a&gt;.  So, how can this be?  Can someone explain this?&lt;/p&gt;&#xA;" OwnerUserId="20583" LastEditorUserId="20583" LastEditDate="2017-03-07T19:00:00.993" LastActivityDate="2018-11-07T22:03:12.493" Title="What is the default smoother for the &quot;PCMG&quot; preconditioner in PETSc?" Tags="|iterative-method|petsc|multigrid|smoothing|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26146" PostTypeId="2" ParentId="26144" CreationDate="2017-02-09T19:04:08.953" Score="1" Body="&lt;p&gt;Is there a reason you considered the Fourier series?  It actually seems fairly simple to compute $\frac{\partial T}{\partial t}$ as a piecewise function of $t$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial T}{\partial t} = \left\{ \begin{matrix} \frac{\partial c}{\partial t} &amp;amp; c(t) &amp;gt; 0 \\ 0 &amp;amp; c(t) &amp;lt; 0 \end{matrix} \right. = \left\{ \begin{matrix} -\frac{2\pi}{\tau} A \sin( \frac{2\pi}{\tau} t + \phi) &amp;amp; c(t) &amp;gt; 0 \\ 0 &amp;amp; c(t) &amp;lt; 0 \end{matrix} \right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, from here, you just need to figure out the values of $t$ for which $c(t)$ is positive/negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Perhaps I am misunderstanding your question?)&lt;/p&gt;&#xA;" OwnerUserId="20583" LastActivityDate="2017-02-09T19:04:08.953" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26151" PostTypeId="2" ParentId="26115" CreationDate="2017-02-10T05:29:44.257" Score="3" Body="&lt;p&gt;If I understand what information you have available, what you want is not possible: the information available to you is not enough to determine the entropy.  It's not even enough to approximate the entropy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It sounds like you have a way to sample from the distribution $p(\cdot)$, and you have a way to compute the ratio $p(a_1)/p(a_2)$ for any pair of elements $a_1,a_2$ that you obtained via sampling, but you have no other information.  If so, your problem is not solvable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, we can find a pair of distributions that have different entropies, but that cannot be distinguished using the information available to you.  Consider first the uniform distribution on a (random) set of size $2^{200}$.  Consider next the uniform distribution on a (random) set of size $2^{300}$.  These have different entropies (200 bits vs 300 bits).  However, given the information available to you, you have no way of knowing which of those two distributions you are working with.  In particular, in both cases, the ratio $p(a_1)/p(a_2)$ will always be exactly 1, so the ratios won't help you distinguish between the two distributions.  And due to the birthday paradox, you can sample as much as you like, but you'll never get the same value twice (not within your lifetime, except with exponentially small probability), so the values you get from sampling will look like just random points and contain no useful information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to solve your problem, you'll need to know something more.  For instance, if you know something about the structure of the distribution $p(\cdot)$, that might make it possible to solve your problem.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2017-02-10T05:29:44.257" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26152" PostTypeId="1" CreationDate="2017-02-10T10:34:42.233" Score="6" ViewCount="159" Body="&lt;p&gt;I need to solve the  &lt;a href=&quot;https://en.wikipedia.org/wiki/Kadomtsev%E2%80%93Petviashvili_equation&quot;&gt;Kadomtsev Petviashvili (KP) equations&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_x(\partial_t u+u \partial_x u+\epsilon^2\partial_{xxx}u)+\lambda\partial_{yy}u=0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $$\lambda=\pm 1 \;.$$&#xA;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Which is the best numerical method to solve them? Or maybe a good method that is simple?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) What does wikipidia mean with: &quot;the KP equation is completely integrable&quot;? I don't think it means KP has a general analytical solution, otherwise why are people still coming out with numerical methods for solving it? See for example &lt;a href=&quot;http://math.u-bourgogne.fr/IMB/klein/Hosdina/Publications_files/kpdsnumart4.pdf&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10032" LastEditorUserId="10032" LastEditDate="2017-02-10T14:27:45.800" LastActivityDate="2017-02-10T14:27:45.800" Title="Good numerical method for solving the Kadomtsev Petviashvili equations. Is there an analytical solution?" Tags="|hyperbolic-pde|numerical-modelling|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26154" PostTypeId="2" ParentId="26115" CreationDate="2017-02-10T15:35:08.160" Score="2" Body="&lt;p&gt;For the second part of your question (estimation of entropy &lt;em&gt;difference&lt;/em&gt; between distributions) you may be able to use the identity $$F = \langle E \rangle - T S,$$ where $\langle E \rangle$ is the average energy, $T$ is the temperature (it is proportional to $\theta$ in $p \propto \mathrm{e}^{\theta E}$), and $S$ is the entropy. For details, see: Jaynes, E. (1957). Information Theory and Statistical Mechanics. Physical Review, 106(4), 620–630. &lt;a href=&quot;http://doi.org/10.1103/PhysRev.106.620&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doi.org/10.1103/PhysRev.106.620&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea then would be to use one of the &lt;a href=&quot;http://www.alchemistry.org/wiki/Free_Energy_Fundamentals&quot; rel=&quot;nofollow noreferrer&quot;&gt;methods available in the Computational Statistical Physics literature&lt;/a&gt; (see the links in the sidebar of that page) to find free energy &lt;em&gt;differences&lt;/em&gt; $\Delta F$ and then find $\Delta S$ as a function of $\Delta F$ and $\Delta \langle E \rangle$ using the above formula (keep in mind that you can think of the restriction to a subset $A_1$ of $A$ as being equivalent to modifying the energy function $E$ so that it becomes infinite in the complement of $A_1$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are two additional references on algorithms for computing free energy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lelièvre, T., Rousset, M., &amp;amp; Stoltz, G. (2010). Free Energy Computations. Imperial College Press. &lt;a href=&quot;http://doi.org/10.1142/9781848162488&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doi.org/10.1142/9781848162488&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chipot, C., &amp;amp; Pohorille, A. (2007). Free Energy Calculations. (C. Chipot &amp;amp; A. Pohorille, Eds.) (Vol. 86). Berlin, Heidelberg: Springer Berlin Heidelberg. &lt;a href=&quot;http://doi.org/10.1007/978-3-540-38448-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doi.org/10.1007/978-3-540-38448-9&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2017-02-11T15:34:00.527" LastActivityDate="2017-02-11T15:34:00.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26155" PostTypeId="1" CreationDate="2017-02-10T22:47:02.420" Score="1" ViewCount="418" Body="&lt;p&gt;I'm trying to solve the Travelling Salesman Problem in a non-complete graph $(G,E)$ using genetic algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I can't find a good first approximation by the usual greedy algorithms, as long as I can't assure an arbitrary edge $(u,v)$ to be in $E$. I tried to add to $E$ new edges (with almost infinity weight) in order to make it complete, bit then it turns to be way more inefficient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find a good solution on internet. So can anyone help me?&lt;/p&gt;&#xA;" OwnerUserId="23244" LastEditorUserId="9667" LastEditDate="2017-02-11T16:21:04.803" LastActivityDate="2017-02-11T16:21:04.803" Title="First approximation to the TSP in a non-complete Graph" Tags="|optimization|graph-theory|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26156" PostTypeId="1" CreationDate="2017-02-10T23:56:10.047" Score="1" ViewCount="220" Body="&lt;p&gt;At each iteration $i$ of the GMRES method, is calculated a single new orthonormal vector of the existing Krylov subspace. If the norm of that vector is 0 (or close to 0), then the subspace is &quot;complete&quot;, and we should be able to calculate the converged solution. In theory, this occurs when we have $N$ orthonormal vectors (where $N$ is the size of $A$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution can then be computed using &#xA;$$x=x_0+Vy$$&#xA; where $V$ is the matrix of orthonormal vectors computed above. [$x_0 (N \times 1),V(N \times N)$ and $y(N \times 1)$].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: What happens if the algorithm finds only $P &amp;lt; N$ orthonormal vectors? How can I complete the Krylov-subspace and how can be computed $x=x_0+Vy$ if $V(N \times P)$ is not a square matrix?&lt;/p&gt;&#xA;" OwnerUserId="23245" LastEditorUserId="22025" LastEditDate="2017-02-11T15:45:29.413" LastActivityDate="2020-02-28T14:54:17.460" Title="GMRES : incomplete Krylov-subspace" Tags="|krylov-method|gmres|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26159" PostTypeId="2" ParentId="26156" CreationDate="2017-02-11T16:09:03.793" Score="2" Body="&lt;p&gt;I'm not sure I understand the question, but in GMRES you built a orthonormal base for the space $K_m \; \text{ for } m= 1 \dots N$.&#xA;In this space the generic vector can be write as:&#xA;$$ x = x_0 + V_m y$$&#xA;where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x_0 \in \mathbb{R}^N $&lt;/li&gt;&#xA;&lt;li&gt;$\text{dim}(V_m) = N \times m$&lt;/li&gt;&#xA;&lt;li&gt;$y \in \mathbb{R}^m$, $\text{dim}(y) = m \times 1$ &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;During the various steps the algorithm computes the unique vector of&#xA;$x_0 + K_m$ such that minimize&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ || b -Ax ||_2 = || b - A(x_0 + V_m y) ||_2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the idea, as in other Krilov methods, is to obtain a good approximation after a value $m &amp;lt;&amp;lt; N$, if $m=N$ then $K_m = \mathbb{R}^N$ the whole space&#xA; and you have got the exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; $y, V$ have got a different dimension respect your question and this is the key, i.e. there is no problem if $P&amp;lt;N$, it is the normal case, because the dimension of matrix and vector are different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good reference for more detail is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Saad, Yousef. Iterative methods for sparse linear systems. Society for Industrial and Applied Mathematics, 2003.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2017-02-11T16:09:03.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26160" PostTypeId="1" CreationDate="2017-02-11T22:21:34.757" Score="3" ViewCount="141" Body="&lt;p&gt;The Wikipedia page for SGD describes optimizing a function $f = \sum f_i(\theta;x_i)$ by successively approximating gradients from random subsets of the data, while most literature poses the problem as optimizing $f = \text{E}[F(\theta,\zeta)]$ for some random variable $\zeta$. I can see how these are asymptotically equivalent when $\zeta$ is a sample from all available observations and $F(\theta; \zeta) = \sum_{x_i\in \zeta}f_i(\theta;x_i)$, and intuitively, it seems that the later formulation is more descriptive of the actual optimization task. However, it isn't apparent to me that these problems necessarily have the same solution. Under what conditions are the solutions equal (convexity, etc.)? Are there any theoretical guarantees that a stationary point of the stochastic approximation be in some $\epsilon$-neighborhood of a stationary point of the deterministic function when these assumptions are not met?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Wikipedia says the method converges to a global optima for convex and pseudo convex functions. I'm curious if there is any established error analysis between a deterministic solution and it's stochastic approximation, or are these algorithms used because they make solutions for normally intractable problems possible, and so error analysis is generally impossible/unimportant? I'm trying to decide if an SGD, or one of several online L-BFGS methods, is a flexible enough framework for a tool where I expect a large range of dataset sizes (from 10 to 10s of thousands) and dimensions, but is still generally too many dimensions for most second order methods, without losing accuracy or significant speed when the problem is small enough for other deterministic methods.&lt;/p&gt;&#xA;" OwnerUserId="23072" LastEditorUserId="23072" LastEditDate="2017-02-11T23:16:30.793" LastActivityDate="2017-02-11T23:16:30.793" Title="Stochastic gradient descent for large deterministic optimization problems" Tags="|optimization|stochastic|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26161" PostTypeId="1" AcceptedAnswerId="26162" CreationDate="2017-02-12T02:04:43.260" Score="14" ViewCount="7387" Body="&lt;p&gt;I'm a double major in computer science and mathematics. I love both subjects. I'm thinking in taking a graduate career, perhaps in scientific computing. What's the real difference between scientific computing and numerical analysis? Are they studied as careers? &lt;/p&gt;&#xA;" OwnerUserId="23250" LastEditorUserId="20688" LastEditDate="2019-06-22T20:22:44.187" LastActivityDate="2022-01-23T20:28:57.187" Title="Scientific computing vs numerical analysis" Tags="|numerics|education|terminology|career-development|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26162" PostTypeId="2" ParentId="26161" CreationDate="2017-02-12T03:59:17.773" Score="24" Body="&lt;p&gt;Wikipedia gives a good definition&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Numerical analysis is the study of algorithms that use numerical&#xA;  approximation (as opposed to general symbolic manipulations) for the&#xA;  problems of mathematical analysis (as distinguished from discrete&#xA;  mathematics).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Numerical analysts are typically interested in proving mathematical results about their algorithms, including error bounds (how large can the error in the approximation be), convergence of iterative schemes (does the approximation approach the correct limit), order and rate of convergence (how fast does the algorithm converge), and computational complexity (bounding the number of operations required by an algorithm.)  It's possible to do research in these areas without ever using a computer, and some important results even predate the development of digital computers in the 1950's.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia also has a definition for &quot;Scientific Computing&quot;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Computational science (also scientific computing or scientific&#xA;  computation (SC)) is a rapidly growing multidisciplinary field that&#xA;  uses advanced computing capabilities to understand and solve complex&#xA;  problems. Computational science fuses three distinct elements:[1]&#xA;  Algorithms (numerical and non-numerical) and modeling and simulation&#xA;  software developed to solve science (e.g., biological, physical, and&#xA;  social), engineering, and humanities problems Computer and information&#xA;  science that develops and optimizes the advanced system hardware,&#xA;  software, networking, and data management components needed to solve&#xA;  computationally demanding problems The computing infrastructure that&#xA;  supports both the science and engineering problem solving and the&#xA;  developmental computer and information science.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Scientific computing is much more about practical aspects of getting accurate solutions out of computers. This obviously builds on the results of numerical analysis, but it also draws heavily on computer architecture and software engineering.  Although research in scientific computing is often done for its own sake and to develop hardware and software that will be of use in many applications, there is also a lot of scientific computing research that is driven by the need to solve particular science and engineering problems.  For example, the development of global climate models to study climate change has also moved scientific computing forward. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical analysis is most commonly found in mathematics and applied mathematics departments, while scientific computing is an interdisciplinary field that can be found in computer science departments, mathematics departments, and in the various engineering and science disciplines.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-02-12T03:59:17.773" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26164" PostTypeId="2" ParentId="26161" CreationDate="2017-02-12T15:40:15.013" Score="8" Body="&lt;p&gt;As someone who moved from Engineering to Scientific Computing during Grad school as an incidental need of the kind of work I was doing here are my two cents:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Numerical analysis would focus on the math and algorithms side of things. Figuring out what techniques to use to solve a particular mathamatical problem that does not have an analytical solution e.g. ODE's PDE's Matrix Manipulations Optimization etc. &lt;/li&gt;&#xA;&lt;li&gt;Numerical Analysis would these days often involve significant amounts of programming but still it is pretty much translating the mathematical ideas of an efficient algorithm into computer code. &lt;/li&gt;&#xA;&lt;li&gt;Traditionally FORTRAN was the mainstay. But you could also expect to work with C / C++ and these days Python. Some stuff might also involve packages like Mathematica or MATLAB&lt;/li&gt;&#xA;&lt;li&gt;Coming to Scientific Computing that's more an applied area where one tries to use computing resources to solve some scientific problem. This may involve a lot of the nuts n bolts work. e.g. Compiling codes, installing operating systems and libraries, setting up options to make scientific code work etc.&lt;/li&gt;&#xA;&lt;li&gt;Since a fair bit of scientific computing these days involves parallel computations you will most likely have some exposure to computing clusters, supercomputers, cloud computing etc&lt;/li&gt;&#xA;&lt;li&gt;In scientific computing while you may work with programming languages like C / FORTRAN etc. expect to work a lot with the &quot;gluing&quot; / scripting languages like bash / perl   etc. &lt;/li&gt;&#xA;&lt;li&gt;You'll probably work a lot with Linux-ey systems and end up fairly proficient working on the command line &amp;amp; with tools like sed / awk etc. Some people end up being sys admins. &lt;/li&gt;&#xA;&lt;li&gt;Lot of scientific computing involves visualization and data storage / data retrival. Many people end up becoming experts in Big Data / Hadoop / Map Reduce etc. &lt;/li&gt;&#xA;&lt;li&gt;Numerical Analyisis is essentially a specialist job. You get good at math and coding and solve a specific problem very efficiently. Sometimes inventing an algorithm or two along the way.  Scientific Computing is, in some sense, a generalist job. Relatively speaking. You are often using diverse tools together to solve a specific applied problem. &lt;/li&gt;&#xA;&lt;li&gt;A lot of scientific computing can involve working at the interfaces. e.g. Interfaces between two programs. Where you pipe data from one tool to another for processing. With some format manipulation along the way. i.e. You are trying to get diverse tools to talk to each other where the tools were not really designed to talk to each other. &lt;/li&gt;&#xA;&lt;li&gt;A scientific computing guy often will have to master various data formats. Many instruments will have their own proprietary formats and someone has to decode the data into a format that the numerical algorithm likes. &lt;/li&gt;&#xA;&lt;li&gt;Some Scientific Computing guys end up manning &quot;helpdesks&quot; of a very specialized nature (well paying too) where one essentially helps a generic researcher / student / Prof. use the computing resources at an institution to solve whatever problem might pop up. i.e. The scientific computing guy is the one familiar with a variety of codes and packages and able to advise a user on what tool to use to best solve the problem computationally. &lt;/li&gt;&#xA;&lt;li&gt;You can end up porting codes to other hardware. Or parallelizing legacy codes that were written in serial mode. Or optimizing codes to run faster. Some guys will convert codes to run in GPUs / CUDA etc. to make them run faster. &lt;/li&gt;&#xA;&lt;li&gt;A fair bit of scientific computing involves troubleshooting. Often codes that other people have written. To figure out why they crash on certain hardware etc. &lt;/li&gt;&#xA;&lt;li&gt;Often you are the middleman liasoning between specialists. e.g. I've had to work on teams with hardcore programmers, biologists that need computation but cannot code much themselves, sys admins, network gurus, technicians manning data centers etc. &lt;/li&gt;&#xA;&lt;li&gt;Scientific Computing guys can be asked to give significant inputs when new hardware is purchased or the architecture of a computing system is decided. On those assignments you end up working very closely with sales engineers and technologists from  Dell / Cray / IBM / Infiniband / Cisco etc. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hope this gives you some idea about the fields!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One final bit of advice (take it with a huge pinch of salt!): If you are good with math, like precision and detail and reading papers and figuring out details after significant, focused effort where individual intelligence matters and involves long periods of sustained effort then go for Numerical Analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you like being a generalist, switch areas, compensate for genius with hard work, be a jack of all trades, be willing to work with fuzzy and vague often conflicting recommendations, like to work with teams and deal with conflict, tight deadlines, deal with MBAs etc. then go become a Scientific Computing guy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again take this with a huge pinch of salt. Every person's circumstance is special. And most of us landed where we did out of sheer coincidence and not because we planned it that way. :)&lt;/p&gt;&#xA;" OwnerUserId="3795" LastEditorUserId="3795" LastEditDate="2017-02-12T15:52:28.767" LastActivityDate="2017-02-12T15:52:28.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26165" PostTypeId="2" ParentId="8371" CreationDate="2017-02-12T16:18:05.133" Score="0" Body="&lt;p&gt;To answer the first question and provide a (probably numerically inefficient) method for the second, note that this is the inverse of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bernoulli_number#Generating_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;generating function of the Bernoulli numbers&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="440" LastActivityDate="2017-02-12T16:18:05.133" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26166" PostTypeId="1" CreationDate="2017-02-12T19:15:44.450" Score="5" ViewCount="630" Body="&lt;p&gt;If $A, B, C$ are $n \times n$ matrices, where both $B$ and $C$ are nonsingular, and $b$ is a vector of length $n$, how would you compute the following &lt;em&gt;without&lt;/em&gt; computing any inverses?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x = B^{-1}(2A+I)(C^{-1}+A)b$$&lt;/p&gt;&#xA;" OwnerUserId="23257" LastEditorUserId="20417" LastEditDate="2017-03-24T13:45:19.173" LastActivityDate="2017-03-24T14:02:14.533" Title="Compute $x = B^{-1}(2A+I)(C^{-1}+A)b$ without calculating matrix inverses" Tags="|linear-algebra|matrix|matrix-equations|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26167" PostTypeId="2" ParentId="25216" CreationDate="2017-02-12T19:44:34.287" Score="1" Body="&lt;p&gt;In your expression for $S$, is $x = k_x \Delta x$ and $y = k_y \Delta y$ for some &quot;wave number&quot; vector $\mathbf{k} = [k_x \quad k_y]^T$ with $\Delta x $ and $\Delta y$ being grid spacings in $x$ and $y$ directions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assumed that $x = \pi$ and $y = \pi$ and obtained the following for $S$:&#xA;$$&#xA;-{\frac {2\,cdp+{c}^{2}+{d}^{2}-c-d-3}{{c}^{2}+2\,cd+{d}^{2}+5\,c+5\,d&#xA;+3}}&#xA;$$&#xA;which satisfies $ |S| \le 1$. The quantity $p$ is equal to $2f +1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values $x=\pi$ and $y=\pi$, of course, correspond to the minimum resolvable wavelengths in each direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I played with a few other values of $x$ and $y$ as well and in all those cases, $|S| \le 1$.&lt;/p&gt;&#xA;" OwnerUserId="1156" LastActivityDate="2017-02-12T19:44:34.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26168" PostTypeId="1" CreationDate="2017-02-12T20:01:12.033" Score="1" ViewCount="863" Body="&lt;p&gt;I am trying to integrate the following function in Python,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{0}^{\infty}\int_{0}^{\infty} \dfrac{e^{-x-y}}{B(x,y)}dx dy$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $B(x,y)$ is the beta function - $B(x,y) = \int_0^{1}a^{x-1}(1-a)^{y-1}da$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the above function was not directly integrable in &lt;code&gt;scipy.integrate&lt;/code&gt;, I did a change of variable, $x\rightarrow e^{-x},y\rightarrow e^{-y}$ to have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{0}^{1}\int_{0}^{1} \dfrac{1}{B(-\log (x),-\log (y))}dx dy$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;scipy.integrate.dblquad(lambda y,x: 1/ scipy.special.beta(-math.log(x),-math.log(y) ),  0, 1, lambda x: 0, lambda x: 1)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;returns $6.81\ldots$ with error in order of $10^{-6}$. But  integrating the above functions (with and without change of variable) in Mathematica (using &lt;code&gt;NIntegrate&lt;/code&gt;) returns $2.0505\ldots$. I do not understand this huge difference in answer when python returns an answer with $10^{-6}$ accuracy. Whats happening? Is there a better way to integrate it in python? I plan to perform the integration many number of times as part of an algorithm written in python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{B(-\log (x),-\log (y))}$ looks like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YBhKu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YBhKu.png&quot; alt=&quot;singularity at (0,0)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone with Windows 10 and Anaconda reproduce the results I get? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the incorrect answer using Windows 10, Python 3.5.2 |Anaconda 4.3.0 (64-bit), scipy version 0.18.1. I am not sure about the BLAS linkage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the correct answer in Linux using Python 2.7.12, scipy version 0.18.1&lt;/p&gt;&#xA;" OwnerUserId="23258" LastEditorUserId="23258" LastEditDate="2017-02-14T01:32:56.957" LastActivityDate="2017-09-03T17:50:12.107" Title="Numerical Double integration with endpoint singularity in scipy Python gives incorrect answer" Tags="|quadrature|scipy|mathematica|" AnswerCount="0" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="26169" PostTypeId="2" ParentId="26166" CreationDate="2017-02-12T20:51:06.847" Score="8" Body="&lt;p&gt;As was mentioned in the comment, calculating $x=M^{-1}y$ is equivalent to solving $Mx=y$. Here is the full solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you can reformulate the equation to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Bx=(2A+I)(C^{-1}+A)b$, and by defining $\tilde{b}=C^{-1}b$, the equation can be rewritten as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Bx=(2A+I)(I+AC)\tilde{b}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, compute $\tilde{b}$ by solving (using, for example, LU decomposition for $C$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C\tilde{b}=b$. Now, once $\tilde{b}$ is computed, you can define $h=(2A+I)(I+AC)\tilde{b}$ and solve, using LU decomposition,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Bx=h$&lt;/p&gt;&#xA;" OwnerUserId="3761" LastActivityDate="2017-02-12T20:51:06.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26171" PostTypeId="1" CreationDate="2017-02-13T12:33:58.063" Score="3" ViewCount="81" Body="&lt;p&gt;I'm reading an article lately, and there is one point which confuses me. So, we have the following constrained binary quadratic problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;min $x^{T}Qx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the constraints that $Ax≤b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $x\in {0,1}^{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be relaxed into the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;min $\sum_{1\leq i &amp;lt;j \leq n}2q_{ij}y_{ij} + \sum_{i=1}^{n}q_{ii}x_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the constraints that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Ax\leq b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y_{ij}\geq x_i+x_j-1 \text{ for any } i,j \text{ with } i&amp;lt;j \text{ and } q_{ij}&amp;gt;0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y_{ij}\leq x_i \text{ for any } i,j \text{ with } i&amp;lt;j \text{ and } q_{ij}&amp;lt;0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y_{ij} \leq x_j \text{ for any } i,j \text{ with } i&amp;lt;j \text{ and } q_{ij}&amp;lt;0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0\leq x_i \leq 1\text{ for any } i=1,2,...n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y\geq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the solution of the relaxed optimization problem would serve as a lower bound of the optimal solution for the original optimization problem. Could anyone help me explain the relaxation part? I was thinking to prove&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sum_{1\leq i &amp;lt;j\leq n}2q_{ij}y_{ij} + \sum_{i=1}^{n}q_{ii}x_i \leq \sum_{j=1}^{n}\sum_{i=1}^{n}x_iq_{ij}x_j$ where $x_i$ is either 0 or 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the term $\sum_{i=1}^{n}q_{ii}x_i$ of the left part would cancel out the term $\sum_{i=1}^{n}x_i^{2}q_{ii}$ of the right part. &#xA;Then, what shall I do next? Any comments would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="23260" LastActivityDate="2017-02-13T12:33:58.063" Title="linear relaxation of an optimization problem" Tags="|optimization|convex-optimization|constrained-optimization|linear-programming|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="26173" PostTypeId="1" AcceptedAnswerId="26176" CreationDate="2017-02-13T17:05:08.577" Score="0" ViewCount="222" Body="&lt;p&gt;I'm going through an article lately and there is one point which is very confusing. So, we have the following original constrained binary quadratic problem as the following. The pre-assumption of certain parameters are $Q\in Z^{n*n}$, $A\in Z^{n*n}$, and $b\in Z^{n*1}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{min}$ $x^{T}Qx$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s.t.,$ $Ax\leq b$&#xA;$\text{and}$ $x\in \{0,1\}^{n}$. Let's call this optimization problem the problem $P$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, by the classical lagrangian reduction, this can be relaxed  as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{min}$ $x^{T}Qx+\lambda^{T}(Ax-b)$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s.t.,$ $x\in \{0,1\}^{n}$ and $\lambda &amp;gt;0$. Let's call this optimization problem the problem $L_{\lambda}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the relaxation serves as a lower bound, we ideally plan to make it as maximum as possible. So, let $d(\lambda) = x^{T}Qx + \lambda^{T}(Ax-b)$, we have the other optimization,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{max}$ $d(\lambda)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s.t.,$ $\lambda \geq 0$.  Let's call this optimization problem the problem $L$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far so good. Now, here is the confusing point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article mentioned that the problem $L$ can be rewritten as the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$max$ $\mu$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s.t.,$ $\mu \leq x^{T}Qx + \lambda^{T}(Ax-b)  $&#xA;for any $x\in \{0,1\}^{n}$ and $\lambda \geq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's call the last optimization problem the problem $W$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not quite understand why this is a rewritten. So, suppose $\mu_{0}$ is the optimal solution of problem $W$, then it follows that $\mu_{0}\leq x^{T}Qx+\lambda^{T}(Ax-b)$ for any $x\in \{0,1\}^{n}$ and $\lambda \geq 0$. While on the other hand, for the problem $L$, what I understand it is like this: for each given $\lambda$, there is one optimization problem $L_{\lambda}$, and one optimal solution (which we take a minimum by the definition of $L_{\lambda}$). Then, finally, among all the optimum solutions, one maximum value is chosen. Somehow, I can not see that in the formulation of the optimization problem $W$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or did I overthink? Any comments would be greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="23260" LastEditorUserId="23260" LastEditDate="2017-02-13T17:16:35.793" LastActivityDate="2017-02-13T22:22:42.300" Title="lagrangian dual and linear programming" Tags="|optimization|convex-optimization|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26174" PostTypeId="1" CreationDate="2017-02-13T17:53:31.840" Score="1" ViewCount="723" Body="&lt;p&gt;I am currently trying to plot a function which describes linear perturbation growth in cosmology for different world models. I would like to be able to have all the curves on the same set of axes, but am struggling with setting it up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D(z) = \frac{5 \Omega_M H_0^2}{2}  \int_z^{\infty} \frac{1+z}{H^3(z)}dz $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H(z) = H_0 [\Omega_m(1+z)^3+\Omega_{\Lambda}]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My aim is to plot this function D, with respect to z, but have multiple plots with varying density parameters ($\Omega$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have managed two solutions but both aren't working perfectly, the first is very inefficient (adding new functions for each set of parameters):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z = np.arange(0.0,10,0.1)&#xA;&#xA;MOm = 1.0&#xA;MOv = 0.0&#xA;&#xA;COm = 0.3&#xA;COv = 0.7&#xA;&#xA;H0 = 70&#xA;&#xA;def Mf(z):&#xA;    A = (5/2)*MOm*(H0**2)&#xA;    H = H0 * np.sqrt( MOm*((1+z)**3) + MOv )&#xA;    return A * ((1+z)/(H**3))&#xA;&#xA;def MF(z):&#xA;    res = np.zeros_like(z)&#xA;    for i,val in enumerate(z):&#xA;        y,err = integrate.quad(Mf,val,np.inf)&#xA;        res[i] = y&#xA;    return res&#xA;&#xA;def MD(z):&#xA;    return (H0 * np.sqrt( MOm*((1+z)**3) + MOv )) * MF(z)&#xA;&#xA;def Cf(z):&#xA;    A = (5/2)*COm*(H0**2)&#xA;    H = H0 * np.sqrt( COm*((1+z)**3) + COv )&#xA;    return A * ((1+z)/(H**3))&#xA;&#xA;def CF(z):&#xA;    res = np.zeros_like(z)&#xA;    for i,val in enumerate(z):&#xA;        y,err = integrate.quad(Cf,val,np.inf)&#xA;        res[i] = y&#xA;    return res&#xA;&#xA;def CD(z):&#xA;    return (H0 * np.sqrt( COm*((1+z)**3) + COv )) * CF(z) &#xA;&#xA;&#xA;plt.plot(z,MD(z),label=&quot;Matter Dominated&quot;)&#xA;plt.plot(z,CD(z),label=&quot;Current Epoch&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I tried to make it simpler with a for loop, but have been unable to work out how to add labels to each plot inside the loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Om = (1.0,0.3)&#xA;Ov = (0.0,0.7)&#xA;&#xA;for param1,param2 in zip(Om,Ov):&#xA;    def f(z):&#xA;        A = (5/2)*param1*(H0**2)&#xA;        H = H0 * np.sqrt( param1*((1+z)**3) + param2 )&#xA;        return A * ((1+z)/(H**3))&#xA;&#xA;    def F(z):&#xA;        res = np.zeros_like(z)&#xA;        for i,val in enumerate(z):&#xA;            y,err = integrate.quad(f,val,np.inf)&#xA;            res[i] = y&#xA;        return res&#xA;&#xA;    def D(z):&#xA;        return (H0 * np.sqrt( param1*((1+z)**3) + param2 )) * F(z)&#xA;&#xA;    plt.plot(z,D(z))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could someone please help explain an efficient method of doing so? Or how to add labels to plots on the fly with a for loop. Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="23269" LastActivityDate="2017-02-13T21:56:30.807" Title="Plotting the same function multiple times on the same set of axes but with different parameters [Python]" Tags="|python|plotting|efficiency|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26175" PostTypeId="2" ParentId="26174" CreationDate="2017-02-13T21:56:30.807" Score="1" Body="&lt;p&gt;The following code gives the plot in less than a sec on my pc.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.integrate import quad&#xA;&#xA;def H(z, H0, MOm, MOv):&#xA;    return H0 * np.sqrt( MOm*((1+z)**3) + MOv )&#xA;&#xA;# fixed values&#xA;COm = 0.3&#xA;COv = 0.7&#xA;H0 = 70&#xA;&#xA;z_range = np.linspace(0, 1, 100)&#xA;Om_range = (1.0,0.3)&#xA;Ov_range = (0.0,0.7)&#xA;&#xA;for (MOm, MOv) in zip(Om_range, Ov_range):&#xA;    d_range = [(5/2)*MOm*(H0**2) * &#xA;               quad(lambda x: (1 + x)/H(z, H0, MOm, MOv)**3, z, np.inf)[0] for z in z_range]&#xA;    plt.plot(z_range, d_range, &#xA;             label = '$\\Omega_m = ' + str(MOm)+ ', \\Omega_\\Lambda = ' + str(MOv)+'$')&#xA;plt.legend(loc = 0)&#xA;plt.xlabel('$z$')&#xA;plt.ylabel('$D(z)$')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eKX1J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eKX1J.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2017-02-13T21:56:30.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26176" PostTypeId="2" ParentId="26173" CreationDate="2017-02-13T22:22:42.300" Score="1" Body="&lt;p&gt;You have not defined $d(\lambda)$ (and, therefore, problem $L$) correctly. It should be &#xA;$$ &#xA;d(\lambda) = \min_{x \in \{0,1\}^n} \left(x^TQx+\lambda^T(Ax-b)\right).&#xA;$$&#xA;Now note that any problem of the form&#xA;$$&#xA;\begin{align}&#xA;\text{maximize }&amp;amp;\min_zf(z)\\&#xA;\text{subject to }&amp;amp; \mathcal{C}, &#xA;\end{align}&#xA;$$&#xA;where $z$ is the (vector) optimization variable and $\mathcal{C}$ is a set of constraints, can be equivalently written as &#xA;$$&#xA;\begin{align}&#xA;\text{maximize }&amp;amp;t\\&#xA;\text{subject to }&amp;amp; t\leq f(z) \text{ for all } z, \\&#xA; &amp;amp; \mathcal{C}. &#xA;\end{align}&#xA;$$&#xA;where the optimaziation variables are the vector $z$ and the scalar $t$.&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2017-02-13T22:22:42.300" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26177" PostTypeId="1" CreationDate="2017-02-14T01:02:11.450" Score="1" ViewCount="48" Body="&lt;p&gt;While reading &lt;a href=&quot;http://web.cs.ucla.edu/~cffjiang/research/mpmcourse/mpmcourse.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper on MPM&lt;/a&gt; (page 48), the &lt;em&gt;definition&lt;/em&gt; of the objective function $E(\boldsymbol v_{\boldsymbol i})$ didn't make sense to me, although I completely understand the definition of (what I suppose to be) it's derivative $\boldsymbol h(\boldsymbol v^{n+1})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Excerpt from the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We will show that solving  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$\boldsymbol h(\boldsymbol v^{n+1}) = \boldsymbol M \boldsymbol v^{n+1}  -\Delta t \boldsymbol f(\boldsymbol x^n + \Delta\boldsymbol v^{n+1}) - \boldsymbol M \boldsymbol v^n = \boldsymbol 0$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;is equivalent to minimizing the following objective function:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$E(\boldsymbol v_{\boldsymbol i}) = \sum_{\boldsymbol i} \frac{1}{2} m_{\boldsymbol i}\lVert \boldsymbol v_{\boldsymbol i} - \boldsymbol v_{\boldsymbol i}^n \rVert^2 + e(\boldsymbol x_{\boldsymbol i}^n + \Delta t \boldsymbol v_{\boldsymbol i}).$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I believe $\boldsymbol v^{n+1}$ and $\boldsymbol v^{n}$ are velocities of all grid node&lt;strong&gt;s&lt;/strong&gt; of cartesian grid, whereas $\boldsymbol v^{n+1}_{\boldsymbol i}$ and $\boldsymbol v^{n}_{\boldsymbol i}$ are the velocities of one particular grid node.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;The questions:&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why is $E(\boldsymbol v_{\boldsymbol i})$ taking only single velocity vector $\boldsymbol v_{\boldsymbol i}$ as its parameter ($\boldsymbol v_{\boldsymbol i}$ = the velocity of the grid node at position $\boldsymbol i$) while it's summing over all grid nodes $\boldsymbol i$ (the sum $\sum_{\boldsymbol i}$) – isn't that nonsense? Shouldn't it take all grid nodes' velocities $\boldsymbol v$ instead?&lt;/li&gt;&#xA;&lt;li&gt;Isn't it nonsense to sum $e(\boldsymbol{\hat x})$ $i$-times, given that:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$e(\boldsymbol{\hat x}) = \sum_p V^0_p \Psi(\boldsymbol{\hat F}_p(\boldsymbol{\hat x}) )$ - i.e. sum over all particles $p$ (page 45/equation 191)&lt;/li&gt;&#xA;&lt;li&gt;supposing $E(\boldsymbol v_{\boldsymbol i}) =   \sum_{\boldsymbol i} \left(\frac{1}{2} m_{\boldsymbol i}\lVert \boldsymbol v_{\boldsymbol i} - \boldsymbol v_{\boldsymbol i}^n \rVert^2 + e(\boldsymbol x_{\boldsymbol i}^n + \Delta t \boldsymbol v_{\boldsymbol i})\right)$ (&lt;em&gt;note the parentheses&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;I'd suggest to define $E$ instead as&#xA;$$E(\boldsymbol v) = e(\boldsymbol x^n + \Delta t \boldsymbol v) +  \sum_{\boldsymbol i} \frac{1}{2} m_{\boldsymbol i}\lVert \boldsymbol v_{\boldsymbol i} - \boldsymbol v_{\boldsymbol i}^n \rVert^2$$&#xA;Is this redefinition of $E$ correct? If not, why is the original formulation correct?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21183" LastActivityDate="2017-02-14T01:02:11.450" Title="Unclear definition of objective function for MPM" Tags="|optimization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26178" PostTypeId="1" CreationDate="2017-02-14T01:02:56.233" Score="1" ViewCount="17" Body="&lt;p&gt;Note: &lt;a href=&quot;https://cs.stackexchange.com/questions/70272/optimizing-estimator-of-composed-functions-when-function-is-known&quot;&gt;This was cross-posted from comp-sci&lt;/a&gt;, as I didn't know this community existed!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a problem which I'm looking to see if there is literature on:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider three types of actors, a Director, Aggregator, and Follower. The Director talks to multiple Aggregators, the Aggregator talks to multiple Followers. The director gives simple commands to the Aggregator, who then can portion out the commands to each Follower as she sees fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question then is, how can the Director know whether or not an Aggregator can complete a given request at time step T? T+1? If T is now, then this is relatively easy as the Aggregator can simply sum up the capabilities of each of the Followers and return it, saying &quot;this is what I can do in total.&quot; The problem then is, if the Director is doing her own optimization problem in deciding how to direct two different Aggregators, given each command she must be able to predict how the Aggregator will react. I.e., what are the Aggregator's capabilities in time step T+1 given a direction D in time step T?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the complexity of the Aggregator, I think it's likely best to create an estimator across the current state of the Aggregator (or all of their followers). Further, the Aggregator can give whatever knowledge it wants to the Director, though ideally this would be of constant size no matter the size of the set of Followers. This is therefor an optimization problem on the Aggregator's side for creating an optimal estimator, which is well studied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to know: Is there any literature about creating an estimator when you know the exact function you are estimating? So if I am the aggregator, how can I tell the Director, in a constant space for the amount of information I give no matter the number of followers, the information require to best predict I would react to a given direction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put another way: How can I create an optimal estimator in which I have full knowledge of the shape and composition of the functions I am estimating?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of a number of places where this might occur, potentially in networking, data compression, etc., though I think I don't have the necessary vocabulary. I've tried searching through aggregation, optimization, estimation, forecasting, prediction, which a number of different combinations of keywords relating to these. There is a ton of generalized work on making estimators and predictors, though none which target the assumption that they already have perfect knowledge of that which they're estimating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Post note after finding this stack exchange]&#xA;I read a bit, finding &lt;a href=&quot;https://scicomp.stackexchange.com/questions/25628/optimization-of-known-function-with-respect-to-two-unknown-function-arguments&quot;&gt;this&lt;/a&gt; for instance, and felt that my question was still unique. In this case all functions are known beforehand.&lt;/p&gt;&#xA;" OwnerUserId="23275" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-02-14T01:02:56.233" Title="Optimizing estimator of composed functions when function is known" Tags="|optimization|reference-request|machine-learning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26179" PostTypeId="1" CreationDate="2017-02-14T04:16:43.860" Score="1" ViewCount="95" Body="&lt;p&gt;I'm going through an article with title &quot;Solving constrained quadratic binary problems via quantum adiabatic evolution&quot; (reference 1). And there are several points confusing me a lot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This article is aimed to solve the CBQP (constrained binary quadratic programming) with the following format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\min &amp;amp;x^{T}Qx \cr&#xA;&amp;amp;\text{subject to} &amp;amp;Ax\leq b&#xA;\end{align}&#xA;and $x\in \lbrace0,1\rbrace^{n}$, where $Q\in Z^{n\times n}$ and $A\in Z^{m\times n}$. Let's call this optimization problem the problem $P$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The outline is like this. Suppose there is a UBQP (Unconstrained binary quadratic programming) oracle, and with the successive application of LP(linear programming), the lagrangian dual of $P$ (or lower bound of $P$ can be provided) can be solved. And then with the branch-bound-approach, the problem $P$ can be solved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can understand almost of it until the section 5 counting the solution density on page 9. I'm not exactly sure how the branch-bound process is combined with the optimization process to finally tackle this problem. Could anyone point a direction or share some thoughts? Any comments would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Ronagh, P., Woods, B., &amp;amp; Iranmanesh, E. (2015). &lt;a href=&quot;https://arxiv.org/pdf/1509.05001.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solving constrained quadratic binary problems via quantum adiabatic evolution.&lt;/a&gt; arXiv preprint:1509.05001.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23260" LastEditorUserId="9667" LastEditDate="2017-04-15T17:21:04.823" LastActivityDate="2017-04-15T17:21:04.823" Title="constrained quadratic binary problems and quantum adiabatic evolution" Tags="|optimization|constrained-optimization|quantum-mechanics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26180" PostTypeId="1" CreationDate="2017-02-14T04:21:23.520" Score="2" ViewCount="66" Body="&lt;p&gt;I'm a New Yorker and take the subways every day. I have a growing interest in understanding the distribution of paths people take on the subways to work every day. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I.e. if there are $n$ subway stations, I want a temporal distribution over the paths $\omega = (n_i, n_j, t)$ where $n_i$ and $n_j$ are nodes representing subway stations and $t$ is a continuous variable representing time. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am given the expected arrival and departure times for each subway station as well as the number of people who enter and exit each station at discrete moments in time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am hoping someone can give me ideas on how to translate my deterministic model into one I can fit statistically, as well as reason about theoretically&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Ideal Theoretical Model&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Let $N$ be the total population in the subways at any given moment in time and let $N_i$ be the number of individuals in station $i$, and $N_ij$ be the number of individuals on the subway line heading from station $i$ to station $j$. The total number of individuals in transit &#xA;$$T = \sum{N_{ij}}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the total number of individuals in standby is &#xA;$$S = \sum{N_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then $$N = T+S$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 1.5 undergrad physics classes under my belt so I've been thinking about the system with the following analogy. Imagine the subway station to be a system of $n$ point masses(each point mass is a subway station) suspended in space, where $N_i$ represents the height of each point mass.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each adjacent subway station exerts a downwards and upwards force on its neighbors. In particular $F_{ij}$ is the force exerted by $i$ on $j$ such that $$F_{ij} = g(N_{ij})$$ where $g$ is some unknown function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this light, the net force on station $i$ is defined as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F_{\text{net}, i} =  \sum{ F_{ji} } - \sum{ F_{ij} }$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, at each moment in time, every station exerts a force on its neighbors. Actually, to be more precise, this force based model implies there are actually $n^2+n$ particles between stations and $n$ stations, therefore a total of $n^2 + 2n$ particles. The velocity of a particle is the rate at which its height decreases and the mass of a particle is a coefficient $m_{ij}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having interpreted the subway station as a group of point particles with pairwise interactions under an unknown force mechanic, we can also define a potential energy configuration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U(N)$ defines the potential energy configuration of the system and $U_{ij}(N)$ defines the potential energy of a particle $ij$, such that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U(T(t)) + K(S(t)) = E(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$K_{ij} = \frac{1}{2}m_{ij}v_{ij}^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U_{ij} = -\int^{N_{ij}}_{0}{F_{ij}} = -N_{ij}m_{ij}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fluctuations in $T$ and $S$ represent changes in the internal potential and kinetic energies of the system. Fluctuations in $N$ represents net work being done on the system, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$W = \Delta E$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Issues in the Model&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I still haven't defined $g$, nor defined the potential and kinetic energy functions. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Statistical Modelling and my Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;At this point, I have a cute little physical interpretation of the subway system. But I am having trouble translating my dynamical model into one I can statistically fit to find a distribution over the ultimate paths taken by individuals over the course of a day. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In practice, I don't know what &quot;&lt;em&gt;modelling the subway dynamics&lt;/em&gt;&quot; will actually mean? &lt;/li&gt;&#xA;&lt;li&gt;Ought I to fit coefficients to $g$ , $U$, and $K$? &lt;/li&gt;&#xA;&lt;li&gt;How to formulate a rigorous statistical question given this abstract physical model.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Or&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alternative ways of finding the distribtion of trips, including links to literature that has already solved this problem. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19718" LastActivityDate="2017-02-14T04:21:23.520" Title="How to model pedestrian flow through subway systems?" Tags="|computational-physics|computational-geometry|constrained-optimization|neural-network|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26181" PostTypeId="1" CreationDate="2017-02-14T10:01:15.297" Score="1" ViewCount="159" Body="&lt;p&gt;Given that there are eigensolvers in Scalapack that use a divide and conquer method, is there any way we can use Scalapack functions to only compute the first $k$ dominant eigenvalues and corresponding eigenvectors? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is for a spectral clustering code.&lt;/p&gt;&#xA;" OwnerUserId="23224" LastEditorUserId="17294" LastEditDate="2017-02-14T12:31:41.927" LastActivityDate="2017-02-14T12:31:41.927" Title="Computing only the $k$ biggest eigenvalues and eigenvectors with Scalapack" Tags="|eigenvalues|fortran|clustering|scalapack|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26182" PostTypeId="1" CreationDate="2017-02-14T14:42:53.513" Score="3" ViewCount="293" Body="&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are trying to solve some models of emission of substances to the air. In these models, emission stops at a certain point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are using the &lt;a href=&quot;http://www.dotnumerics.com/NumericalLibraries/DifferentialEquations/Default.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;DotNumerics implementation&lt;/a&gt; of &lt;a href=&quot;http://www.unige.ch/~hairer/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;radau5&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We had issue finding correct solutions. If emission stops, the air concentration should drop, but this was not always the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We were able to reproduce this issue with a simple model for which we could also find the analytic solution. Also, we inspected the radau5 source code and found a test that looks like an optimization. Due to this test many evaluations of the Jacobian are skipped. If we remove this test, the results are a lot better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have also used the implicit fourth order Runge Kutta by &lt;a href=&quot;http://numerical.recipes/nrcontents.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Press et all., Numerical Recipes 2nd edition&lt;/a&gt; for our original problems and this method did not show this issue. We didn't find the time to test it on this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We wonder:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is this test? Is there some documentation on it?&lt;/li&gt;&#xA;&lt;li&gt;Can we safely remove the test to skip evaluation of the Jacobian?&lt;/li&gt;&#xA;&lt;li&gt;Can we expect radau5 to cope with discontinuities, or should we integrate the two trajectories separately?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Detailed information&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model consists of two compartments. Initially, compartment I contains the substance that is emitted to compartment II. From compartment II, product is removed by ventilation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cyWVd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cyWVd.png&quot; alt=&quot;The model with two compartments&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{rclr}&#xA;\frac{dC_I}{dt}&amp;amp;=&amp;amp;-k_{12}(C_I- C_{II}) \\&#xA;\frac{dC_{II}}{dt}&amp;amp;=&amp;amp;k_{12}(C_I- C_{II})-k_{02}C_{II} &amp;amp; t &amp;lt; T \\&#xA;\frac{dC_{II}}{dt}&amp;amp;=&amp;amp;-k_{02}C_{II} &amp;amp; t &amp;gt; T \\&#xA;C_I (0)&amp;amp;=&amp;amp;1 \\&#xA;C_{II}(0)&amp;amp;=&amp;amp;0&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We found these analytic solutions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{rclr}&#xA;C_{II}(t)&amp;amp;=&amp;amp;C_{II}(T)e^{-k_{02}(t-T)} \\&#xA;&amp;amp;=&amp;amp;\frac{C_0}{\sqrt{D}}k_{12}(e^{λ_+ T}-e^{λ_-T})e^{-k_{02}(t-T)} &amp;amp; t &amp;lt; T \\&#xA;C_{II}(t)&amp;amp;=&amp;amp;\frac{C_0}{\sqrt{D}} k_{12} (e^{λ_+t}-e^{λ_- t}) &amp;amp; t &amp;gt; T \\&#xA;D&amp;amp;=&amp;amp;4k_{12}^2+k_{02}^2 \\&#xA;λ_±&amp;amp;=&amp;amp;-k_{12}-\frac{k_{02}}{2}±\frac{\sqrt{D}}{2}&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We implemented this model in C#.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numeric solution with a tolerance of 0.01 is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1VOUB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1VOUB.png&quot; alt=&quot;Original radau5 and relative tolerance: 0.01&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fiddle.jshell.net/RSchreurs/yeagtgpn/show/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jsfiddle for original radau5 and relative tolerance: 0.01&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the Jacobian is evaluated only twice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking through the radau5 code, we found a test:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (THETA &amp;lt;= THET) goto LABEL20;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;on line 1663 in .\DotNumerics\ODE\Radau5\radau5.cs, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IF (THETA.LE.THET) GOTO 20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;on line 1073 in &lt;a href=&quot;http://www.unige.ch/~hairer/prog/stiff/radau5.f&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.unige.ch/~hairer/prog/stiff/radau5.f&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we apply the fix we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gXDE1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gXDE1.png&quot; alt=&quot;Original radau5 and relative tolerance: 0.01&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://fiddle.jshell.net/RSchreurs/zeczntro/show/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jsfiddle for original radau5 and relative tolerance: 0.01&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reducing the tolerance to 0.001 yields better result, with or without fix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fq3q6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fq3q6.png&quot; alt=&quot;Original radau5 and relative tolerance: 0.001&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jsfiddle.net/RSchreurs/82fsLxqL/show/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jsfiddle for original radau5 and relative tolerance: 0.001&lt;/a&gt;&#xA;&lt;a href=&quot;http://jsfiddle.net/RSchreurs/71sfLjq9/show/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jsfiddle for modified radau5 and relative tolerance: 0.001&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, without fix, the Jacobian is still evaluated only three times, so that could be a coincidence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;a href=&quot;https://github.com/git-hub-user/radau5-discontinuities-demo&quot; rel=&quot;nofollow noreferrer&quot;&gt;c# solution that demonstrates this is on GitHub&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7313" LastActivityDate="2017-10-03T00:03:45.880" Title="Failing integration with the radau5-implementation in DotNumerics, over a discontinuity" Tags="|numerical-analysis|ode|stiffness|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26183" PostTypeId="1" AcceptedAnswerId="26187" CreationDate="2017-02-14T15:58:22.573" Score="4" ViewCount="3561" Body="&lt;p&gt;I have written a function that sets up a sparse matrix &lt;em&gt;&lt;strong&gt;A&lt;/strong&gt;&lt;/em&gt; and RHS &lt;em&gt;&lt;strong&gt;b&lt;/strong&gt;&lt;/em&gt; for the 3D Poisson equation in a relatively efficient way. The set-up is nothing fancy: I have extended the 2D 5-point stencil to an equivalent 7-point stencil for 3D. Conductors are (at this moment) simply blocks of Dirichlet BCs and I am not (yet) taking dielectrics into account. Therefore, the constructed &lt;em&gt;&lt;strong&gt;A&lt;/strong&gt;&lt;/em&gt; is quite straightforward:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Maximum number of non-zero terms is &lt;code&gt;Nx*Ny*Nz*7&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;In practice, my actual number of non-zero terms is about 90% of the maximum&lt;/li&gt;&#xA;&lt;li&gt;Due to the BCs, &lt;em&gt;&lt;strong&gt;A&lt;/strong&gt;&lt;/em&gt; is non-symmetric (&lt;em&gt;I should have mentioned this from the start&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The correctness of my arrays has been verified by running some very simple problems with Nx, Ny and Nz between 50-100 and then using &lt;code&gt;mldivide&lt;/code&gt;. However, my peak memory usage is already hitting ~20GB for these test problems, while I'll be using significantly larger arrays for my actual problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now I'm looking into iterative approaches, but my knowledge currently does not extend beyond Gauss-Seidel, Jacobi and SOR. When I attempt to use any of the built-in iterative solvers out of the box, I typically end up with the following message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;METHOD stopped at iteration 2 without converging to the desired tolerance 1e-06&#xA;because a scalar quantity became too small or too large to continue computing.&#xA;The iterate returned (number 1) has relative residual 0.97.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As this is quite a common system to solve, I was expecting to easily find some resources on this, but I am finding significantly less useful information than I expected. Some advice and pointers would be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="23280" LastEditorUserId="23280" LastEditDate="2017-02-16T09:19:28.187" LastActivityDate="2017-02-17T19:10:53.720" Title="Iteratively solving 3D Poisson equation in MATLAB" Tags="|finite-difference|linear-solver|sparse-matrix|iterative-method|poisson|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26184" PostTypeId="2" ParentId="26182" CreationDate="2017-02-14T16:54:38.480" Score="3" Body="&lt;p&gt;First of all, I cannot judge the quality of the DotNumerics implementation of Radau5, I can only assume it is a direct port of the Fortran original from Hairer and Wanner. A full description of their Radau5 implementation is in &lt;a href=&quot;http://link.springer.com/book/10.1007%2F978-3-642-05221-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;their book on solving stiff differential equations&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The test you refer to is a check to see if the Jacobian should be recomputed. If you don't provide an explicit Jacobian, Radau will try to estimate this Jacobian by finite differences. The Radau code computes, based on the Newton iterations a parameter &quot;theta&quot;. If this value is larger than a certain threshold (&quot;THET&quot;), the Jacobian is recomputed. The user has the option to set the value of THET (at least in the Fortran implementation) by means of the WORK(3) variable. So don't change anything in the code! Use this option to find good values for THET. According to the Fortran implementation, the default for WORK(3) is 0.001. You could check if in the DotNumerics implementation this default value is kept. Again, in the FORTRAN implementation, setting WORK(3) to a negative value will skip the test and re-evaluate the Jacobian at every time step (probably more accurate, but also more expensive). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, I ran your problem as you describe using Assimulo in Python. See the code below. The solution gets rougher, but I don't observe the jump&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, if I understand it correctly, you know your &quot;cross-over&quot; point $T$ explicitly, i.e. it is not problem/simulation dependent. So why make life hard on Radau by having a discontinuity in the right-hand-side of the ODE? Why not simply integrate up to $t = T$ and then use those values as initial values for the next problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if further on you wish to have &quot;problem-dependent&quot; discontinuities, I suggest to use the CVODE code which implements a very nice way to handle state changes. See for example my answer to a recent question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/25970/stop-condition-in-scipy-integrate-ode-for-stiff-system/26027&quot;&gt;on how to stop a simulation if a certain condition has been met &lt;/a&gt; and the example on the Assimulo website &lt;a href=&quot;http://www.jmodelica.org/assimulo_home/tutorial_disc.html#example&quot; rel=&quot;nofollow noreferrer&quot;&gt;describing a pendulum problem where the pendulum hits a wall&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Code example&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from assimulo.problem import Explicit_Problem&#xA;from assimulo.solvers import Radau5ODE&#xA;import matplotlib.pyplot as plt&#xA;%matplotlib inline&#xA;&#xA;C0 = 1.0&#xA;T = 20.0&#xA;tf = 60.0&#xA;k02 = 0.005&#xA;k12 = 0.1&#xA;&#xA;def rhs(t,y):&#xA;&#xA;    ydot = np.zeros_like(y)&#xA;&#xA;    if t &amp;lt;= T:&#xA;        ydot[0] = k12 * (y[1] - y[0]) - k02 * y[0]&#xA;        ydot[1] = -k12 * (y[1] - y[0])&#xA;    else:&#xA;        ydot[0] = -k02 * y[0]&#xA;        ydot[1] = 0.0&#xA;&#xA;    return ydot&#xA;&#xA;y0 = np.array([C0,0.0])&#xA;&#xA;model = Explicit_Problem(rhs,y0)&#xA;simulation = Radau5ODE(model)&#xA;simulation.rtol = 1e-2&#xA;&#xA;t, y = simulation.simulate(60.)&#xA;simulation.reset()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this gives the following output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Final Run Statistics: --- &#xA;&#xA; Number of steps                                 : 9&#xA; Number of function evaluations                  : 48&#xA; Number of Jacobian evaluations                  : 2&#xA; Number of function eval. due to Jacobian eval.  : 4&#xA; Number of error test failures                   : 0&#xA; Number of LU decompositions                     : 9&#xA;&#xA;Solver options:&#xA;&#xA; Solver                  : Radau5 (explicit)&#xA; Tolerances (absolute)   : 0.01&#xA; Tolerances (relative)   : 0.01&#xA;&#xA;Simulation interval    : 0.0 - 60.0 seconds.&#xA;Elapsed simulation time: 0.0007725144052983524 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and plotting both concentrations gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4w4Ze.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4w4Ze.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness, this is the plot for a requested absolute and relative tolerance of 1e-12.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gszuT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gszuT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is different from what you observe: in this simulation there is no decrease (I took the values for the coefficients from your Github code).&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2017-02-14T17:17:59.637" LastActivityDate="2017-02-14T17:17:59.637" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26185" PostTypeId="1" AcceptedAnswerId="26186" CreationDate="2017-02-15T04:27:08.527" Score="1" ViewCount="36" Body="&lt;p&gt;I'm going through the article in the following link lately and one point confuses me a lot. &lt;a href=&quot;https://arxiv.org/pdf/1509.05001.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1509.05001.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the goal of this paper is to solve the following constrained binary quadratic programming. Here the parameters $A$, $Q$ and $b$ are all of integer values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;max $x^{T}Qx$&#xA;$s.t.,$ $Ax\leq b$ $\text{and}$ $x\in \{0,1\}^{n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors consider the lagrangian relaxation of this problem in the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, we have the following optimization problem $L_{\lambda}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$d(\lambda) = \text{min}_{x} x^{T}Qx+\lambda^{T}(Ax-b) $&#xA;$s.t., $  $x\in \{0,1\}^{n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the further optimization problem $L$ in the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L:\text{max}_{\lambda \in R_{+}^{m}}$ $d(\lambda)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, a branch-and-bound tree is created (which I do not quite understand why to create the tree). In each node $u$ of the branch-and-bound tree, a lower bound is computed by solving the problem $L$ and the primal-dual pair $(x^{u},\lambda^{u})$ is obtained. we define the slack of constraint $i$ at a point $x$ to be $s_{i}$ = $b_{i} −a_{i}^{T}x$, where $a_{i}$ is the $i$-th row of $A$. Then the set of violated constraints at $x$ is the set $V = {i : si &amp;lt; 0}$. If $x^{u}$ is infeasible for the original problem, it must violate one or more constraints. Additionally, we define the change in slack for constraint $i$ resulting from flipping variable $j$ in $x^{u}$ to be $$\delta_{ij}=a_{ij}(2x_{j}^{u}-1).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not understand why $\delta_{ij}$ is defined in this way. For my understanding, $x^{u}\in \{0,1\}^{n}$, so when you flip the variable $j$ of $x^{u}$, you change it either from 0 to 1 or from 1 to 0. So, in either case, the change in slack for constraint $i$ can be calculated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did I miss something here ? Could anyone shed some light on what shall I do here? Many thanks for your time and attention.&lt;/p&gt;&#xA;" OwnerUserId="23260" LastActivityDate="2017-02-15T04:36:26.667" Title="constraint satisfaction via an LD solution" Tags="|optimization|convex-optimization|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26186" PostTypeId="2" ParentId="26185" CreationDate="2017-02-15T04:36:26.667" Score="0" Body="&lt;p&gt;$2x_{j}^{u}-1$ is either $+1$ if $x_{j}^{u}$ is 1 or $-1$ if $x_{j}^{u}$ is 0.  Flipping $x_{j}^{u}$ from 0 to 1 or 1 to 0 will always change the contribution of $a_{ij}x_{j}$ to the left hand side of constraint $i$ (either by $+a_{ij}$ or $-a_{ij}$ This formula makes explicit that change in the slack for each constraint $i$.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-02-15T04:36:26.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26187" PostTypeId="2" ParentId="26183" CreationDate="2017-02-15T12:31:52.853" Score="4" Body="&lt;p&gt;The finite difference matrix for the Poisson equation is symmetric and positive definite.&#xA;So the preconditioned conjugate gradient algorithm is the iterative solver of choice for&#xA;this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The choice of preconditioner has a big effect on the convergence of the method. Incomplete&#xA;Cholesky factorization is known to work well for this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following MATLAB code constructs the finite difference matrix for the 3D Poisson problem and&#xA;solves the equation  for a right-hand-side of all ones.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function laplaceEqnTestCSE&#xA;n = 50;&#xA;L=1;&#xA;h = L/(n+1);&#xA;dim=3;&#xA;K=laplaceEqn(dim, n);&#xA;neq = rows(K);&#xA;b = ones(neq,1);&#xA;ne2 = ceil(neq/2);&#xA;tic&#xA;disp('Begin solve.');&#xA;if(0)&#xA;u = K\b;&#xA;else&#xA;tol=1e-8;&#xA;maxIter=100;&#xA;L = ichol(K);&#xA;u = pcg(K, b, tol, maxIter, L, L');&#xA;end&#xA;toc&#xA;end&#xA;&#xA;function K=laplaceEqn(dim, n)&#xA;h = 1/(n+1);&#xA;&#xA;K1D = spdiags(ones(n,1)*[-1 2 -1],-1:1,n,n);  % 1d Poisson matrix&#xA;%subplot(2,3,4), spy(K1D)&#xA;if(dim==1)&#xA;K = K1D/h^2;&#xA;return;&#xA;end&#xA;&#xA;I1D = speye(size(K1D));                       % 1d identity matrix&#xA;K2D = kron(K1D,I1D)+kron(I1D,K1D);            % 2d Poisson matrix&#xA;%subplot(2,3,5), spy(K2D)&#xA;if(dim==2)&#xA;K = K2D/h^2;&#xA;return;&#xA;end&#xA;&#xA;I2D = speye(size(K2D));                       % 2d identity matrix&#xA;K3D = kron(K2D,I1D)+kron(I2D,K1D);            % 3d Poisson matrix&#xA;if(dim==3)&#xA;K = K3D/h^2;&#xA;return;&#xA;end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I ran this code in Octave and got a converged solution in 54 iterations. It took&#xA;about a second on my desktop Windows PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several of the comments pointed out that some methods for discretizing the boundary conditions&#xA;or the introduction of a spatially-varying coefficient can result in an unsymmetric finite&#xA;difference matrix. But fundamentally, the Laplace operator is self-adjoint so, ideally, we would&#xA;expect the numerical coefficient matrix to also be self-adjoint (i.e. symmetric). If that is not&#xA;the case, the discretization approach is probably not the best one. Leveque's introductory book&#xA;on finite difference methods (&lt;a href=&quot;http://faculty.washington.edu/rjl/fdmbook/%20Leveque&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leveque&lt;/a&gt;) has been previously recommended here. &#xA;It addresses both the issue of how to apply boundary conditions and, in example 2.1, how to handle&#xA;spatially-varying coefficients to maintain symmetry of the coefficient matrix.&#xA;To paraphrase Leveque, the most &quot;obvious&quot; way to deal with&#xA;spatially-varying coefficients by taking the derivative and then approximating it with FD is not&#xA;the best way to discretize this case. He discusses in more detail why the important fundamental&#xA;properties of the original PDE should be reflected in the numerical approximation.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-02-17T19:10:53.720" LastActivityDate="2017-02-17T19:10:53.720" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26188" PostTypeId="1" AcceptedAnswerId="26196" CreationDate="2017-02-15T17:57:22.480" Score="7" ViewCount="666" Body="&lt;p&gt;Suppose that I'm solving the poisson equation by the finite element method by lagrange elements.  I know that the error can be measured in a variety of ways, depending on which norm you choose.  For a given lagrange element order p and a symmetric bilinear form (e.g. galerkin's finite element method), the errors in the $L^2$ and $H^1$ norms are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$||u-u_h||_{L^2}\le Ch^{p+1} |u|_{H^{p+1}}$$&#xA;and&#xA;$$||u-u_h||_{H^1}\le Ch^p|u|_{H^{p+1}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $|u|_{H^{p+1}}$ is the seminorm&#xA;$$|u|^2_{H^{p+1}} = \sum_{i+j=p+1}\int_\Omega |\frac{\partial^{p+1}u}{\partial x^i\partial y^j}|^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I before I begin delving into reading the full details of the proof of this result (e.g. Brenner &amp;amp; Scott (1994) The Mathematical Theory of Finite Element Methods, Chapter 4), I want to get a &quot;big picture&quot; sketch of the derivation of the bounds.  What are the key ideas that go into the proof and how do they fit together to bound the error?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My hope is to gain insight into the overall strategy of deriving error bounds on other (perhaps more exotic) finite elements by first understanding how to derive them for a well-studied case such as this.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2017-02-16T21:26:52.153" Title="How can I derive the a priori error estimate for a symmetric bilinear form using lagrange finite elements?" Tags="|finite-element|error-estimation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26189" PostTypeId="2" ParentId="26105" CreationDate="2017-02-15T18:57:23.690" Score="1" Body="&lt;p&gt;See also this paper&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://ieeexplore.ieee.org/document/5395267/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hermite Interpolation of Implicit Surfaces with Radial Basis Functions&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and ist extension&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8659.2010.01785.x/abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hermite Radial Basis Functions Implicits&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="640" LastActivityDate="2017-02-15T18:57:23.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26190" PostTypeId="2" ParentId="26188" CreationDate="2017-02-15T22:38:00.707" Score="3" Body="&lt;p&gt;I am not pretending to give the best point of view, but I think of the following general lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain the error estimates in classical cases, you use as ingredients:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;interpolation error (which is purely the property of the chosen finite element space and deals with interpolation of one class of functions by another)&#xA;$$&#xA;\parallel u - \pi_h u \parallel \leq Ch^k |u|_{k+1}&#xA;$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Céa's lemma, which says that the approximate solution is the projection in energy norm of the exact solution onto the finite dimensional subspace.&#xA;$$&#xA;\parallel u - u_h \parallel_A = \inf_{v_h \in V_h} \parallel u - v_h \parallel_A&#xA;$$&#xA;where $V_h$ is the finite element space and $\parallel \cdot \parallel_A$ is the energy norm produced by the main symmetric bilinear form of the problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Equivalence of the norms in the energy space and appropriate Sobolev space (for the simplest case of Dirichlet problem, it is $H^1$).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This way one can get an estimate in $H^1$. For $L_2$ Nitsche's trick is usually used.&lt;/p&gt;&#xA;" OwnerUserId="21958" LastEditorUserId="9667" LastEditDate="2017-02-16T14:29:07.563" LastActivityDate="2017-02-16T14:29:07.563" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26191" PostTypeId="1" CreationDate="2017-02-15T23:34:27.540" Score="5" ViewCount="530" Body="&lt;p&gt;I have a 7000x7000 sparse matrix (scipy), which I want to exponentiate. I've tried using scipy.sparse.linalg.expm, which works quite well for smaller matrices (takes a few seconds for a 1000x1000 matrix) but it takes too long time to compute for the matrix in question. Is there a way to optimise this or work around the time complexity of this algorithm (apparently O(n^2))?&#xA;For background, I've tried the same with R package {expm} with similar results, so it's apparently more about the algorithmic approach to the problem rather than the specific software used. Do you have any ideas?&lt;/p&gt;&#xA;" OwnerUserId="23290" LastActivityDate="2017-02-15T23:34:27.540" Title="Optimisation of matrix exponential" Tags="|optimization|sparse-matrix|matrix|complexity|scipy|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26192" PostTypeId="1" AcceptedAnswerId="26193" CreationDate="2017-02-16T03:28:45.683" Score="2" ViewCount="450" Body="&lt;p&gt;Given the one dimensional equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\epsilon\frac{\partial^2u}{\partial x^2} +\frac{\partial u}{\partial x} = 0 $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $0\le\epsilon \ll1$ with boundary conditions $u(0) = 0$ and $u(1) = 2$, we can't neglect the diffusive term because of the boundary conditions. In which situations (with very small $\epsilon$ could we? What's the mathematical theory behind that could explain it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if we had a time-dependent equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t} = \epsilon\frac{\partial^2u}{\partial x^2} +\frac{\partial u}{\partial x}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;how would the situation change? what if the convective term were nonlinear, such as in the Burgers equation? I haven't been able to find references on this topic, I would appreciate any. &lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2017-02-16T18:29:15.520" Title="When is it safe to ignore the diffusion term in an advection-diffusion equation?" Tags="|pde|hyperbolic-pde|advection-diffusion|advection|singular-perturbation|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26193" PostTypeId="2" ParentId="26192" CreationDate="2017-02-16T04:36:17.660" Score="6" Body="&lt;p&gt;The stationary equation you show transports information from the right to the left via the advection term; it also diffuses slightly. If you switch off the diffusion term altogether, then you only have transport from the right to the left, and you need to also drop the boundary condition at the left: because information is from the right to the left, nothing that happens at the left end of the domain has any effect on the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar argument can be made for the time dependent equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, these equations are examples of &quot;singularly perturbed problems&quot;. You will be able to find a lot of literature on the subject.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-02-16T04:36:17.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26194" PostTypeId="1" CreationDate="2017-02-16T06:43:12.607" Score="1" ViewCount="52" Body="&lt;p&gt;I'm trying to come up with the right algorithm for a system in which the user enters a few symptoms and the system has to predict or determine the likelihood that a few selected symptoms are associated with those existing in the system. Then after associating them, the result or output should be a specific disease for the symptoms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system is comprised of a series of diseases with each assigned to specific symptoms, which also exist in the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that the user entered the following input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A, B, C, and D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first thing the system should do is check and associate each symptom (in this case represented by alphabetical letters) individually against a data-table of symptoms that already exist. And in cases where the input doesn't exist, the system should report or send feedback about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And also, let's say that A and B was in the data-table, so we are 100% sure that they're valid or exist and the system is able to give out the disease based on the input. Then let's say that the input now is C and D where C doesn't exist in the data-table, but there is a possibility that D exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We don't give D a score of 100%, but maybe something lower (let's say 90%). Then C just doesn't exist at all in the data-table. So, C gets a score of 0%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, the system should have some kind of association and prediction techniques or rules to output the result by judging the user's input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary of generating the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;If A and B were entered and exist, then output = 100%&#xA;If D was entered and existed but C was not, then output = 90%&#xA;If all entered don't exist, then output = 0%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What algorithm would be used to produce or come up with the above system concept?&lt;/p&gt;&#xA;" OwnerUserId="23159" LastActivityDate="2017-02-16T06:43:12.607" Title="The right algorithm to predict from selected data samples" Tags="|algorithms|machine-learning|data-sets|" AnswerCount="0" CommentCount="2" ClosedDate="2018-06-02T18:01:25.813" ContentLicense="CC BY-SA 3.0" />
  <row Id="26196" PostTypeId="2" ParentId="26188" CreationDate="2017-02-16T09:52:42.387" Score="9" Body="&lt;p&gt;To extend VorKir's basically correct answer, to derive a priori error estimate for finite element approximations, you need to stack the following three Lego blocks:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The &lt;strong&gt;well-posedness of the weak formulation&lt;/strong&gt; of the PDE.&lt;/li&gt;&#xA;&lt;li&gt;The (global) &lt;strong&gt;conformity of the Galerkin approximation&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The (local) &lt;strong&gt;approximation properties of your finite element space&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(Only the last one is specific to finite elements.) Of course, you need to stack them correctly, so let's look at these in a bit more detail. I'll state them abstractly since you're interested in the blueprint and not the specifics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume your problem in weak formulation is to find $u\in V$ such that &#xA;$$a(u,v) = (f,v) \qquad\text{for all }v\in V \label{weak}\tag{1}$$&#xA;for some (not necessarily symmetric) bilinear form $a:V\times V\to\mathbb{R}$.&#xA;&lt;strong&gt;Well-posedness&lt;/strong&gt; here means that there exist constants $c,C&amp;gt;0$ such that for all $u,v\in V$,&#xA;$$\begin{aligned} c\|u\|_V^2 &amp;amp;\leq a(u,u), \\ a(u,v)&amp;amp;\leq C\|u\|_V\|v\|_V.\end{aligned}$$&#xA;The first inequality is called &lt;em&gt;coercivity&lt;/em&gt;, the second &lt;em&gt;continuity&lt;/em&gt;. &lt;strong&gt;The space $V$ in which you can prove &lt;em&gt;both&lt;/em&gt; is the natural space for the problem and determines the rest of the analysis.&lt;/strong&gt; &#xA;Usually $V$ is a Sobolev space such as $H^1(\Omega)$ (which is the standard space for elliptic second order equations).&#xA;From this, existence, uniqueness and stability of the solution follows using the Lax-Milgram Lemma, which is basically a nonsymmetric variant of the Riesz representation theorem. (If your test function needs to be taken  in a different space $W\neq V$ (because you can't show continuity otherwise), the coercivity is replaced by the so-called &lt;em&gt;inf-sup inequality&lt;/em&gt;, and the Lax-Milgram Lemma by the Banach-Nečas-Babuška Theorem.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you pick a finite-dimensional subspace $V_h\subset V$ (this is the key property!) and look for $u_h\in V_h$ such that &#xA;$$a(u_h,v_h) = (f,v_h) \qquad\text{for all }v_h\in V_h \label{galerkin}\tag{2}.$$&#xA;Since $v_h\in V_h\subset V$, you can insert $v_h$ in \eqref{weak} as well and subtract the two to obtain the &lt;em&gt;Galerkin orthogonality&lt;/em&gt;&#xA;$$a(u-u_h,v_h) = 0 \qquad\text{for any }v_h\in V_h.$$&#xA; Hence, for arbitrary $v_h\in V_h$, you have $v_h-u_h\in V_h$ and therefore $a(u-u_h,v_h-u_h)=0$. Using coercivity and continuity of the bilinear form, you obtain&#xA;$$&#xA;\begin{aligned}&#xA;    c \|{u-u_h}\|_V^2&amp;amp; \leq a(u-u_h,u-u_h)\\&#xA;    &amp;amp;=a(u-u_h,u-v_h) + a(u-u_h,v_h-u_h)\\&#xA;    &amp;amp;\leq C \|{u-u_h}\|_V\|{u-v_h}\|_V.&#xA;\end{aligned}&#xA;$$&#xA;Dividing by $\|{u-u_h}\|_V$, rearranging, and taking the infimum over all $v_h\in V_h$ yields&#xA;$$ \|{u-u_h}\|_V \leq \frac{C}{c} \inf_{v_h\in V_h}\|{u-v_h}\|_V.\label{cea}\tag{3}$$&#xA;This is &lt;em&gt;Céa's lemma&lt;/em&gt;, which says that &lt;strong&gt;the error of any (conforming) Galerkin approach is determined by the approximation error of the exact solution in $V_h$&lt;/strong&gt;. (If the discrete space is not conforming, i.e., $V_h\not\subset V$, you get extra terms you need to estimate and end up with &lt;em&gt;Strang's Lemmas&lt;/em&gt; instead.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step is fairly straightforward: By Céa's Lemma, the discretization error is bounded by the best approximation error in $V$, which in turn can be bounded by &lt;em&gt;any&lt;/em&gt; approximation error -- for example, the interpolation error in $V$, which as pointed out above is usually a Sobolev space. The &lt;strong&gt;interpolation error of polynomials in Sobolev spaces can be estimated by the &lt;em&gt;Bramble-Hilbert lemma&lt;/em&gt;&lt;/strong&gt;, which yields the error estimate&#xA;$$\|u-\mathcal{I}_h u\|_{H^m} \leq C h^{k-m} \|u\|_{H^k}\label{poly}\tag{4}$$&#xA;provided $V_h$ contains piecewise polynomials of degree $k-1$ (and of course $u\in H^k$, which is an assumption on the solution of \eqref{weak}). (Basically, this follows from the fact that the interpolation error is bounded on the reference element, combined with a simple scaling using transformation of integrals. &#xA;Summing over all elements gives a global interpolation estimate.)&#xA;Note that the choice of $m$ in \eqref{poly} is determined by the condition that $H^m\hookrightarrow V$ so that you can chain \eqref{cea} and \eqref{poly} to obtain the standard a priori error estimate in the $V$ norm (I'm being free here with the constants, which can change from one occurrence to the next):&#xA;$$\label{apriori}\tag{5}&#xA; \|{u-u_h}\|_V \leq C \inf_{v_h\in V_h}\|{u-v_h}\|_V \leq C \|u-\mathcal{I}_h u\|_{V} \leq C \|u-\mathcal{I}_h u\|_{H^m} \leq C h^{k-m} \|u\|_{H^k}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the $L^2$ estimate from this requires the &lt;strong&gt;Nitsche trick&lt;/strong&gt;. Assume you have a Hilbert space $H$ with $V\hookrightarrow H$ (such as $H=L^2(\Omega)$ for $V=H^m(\Omega)$). Using the fact that you can identify the Hilbert space $H$ with its own dual space and thus write the $H$ norm as an operator norm, you can derive the inequality&#xA;$$\label{aubin}\tag{6}&#xA;\|u-u_h\|_{H} = \sup_{\varphi\in H\setminus\{0\}} \frac{(u-u_h,\varphi)_H}{\|\varphi\|_{H}} \leq C \|u-u_h\|_{V} \sup_{\varphi\in H\setminus\{0\}} \frac{\inf_{w_h\in V_h}\|w_\phi-w_h\|_{V}}{\|\varphi\|_{H}},&#xA;$$&#xA;where $w_\varphi$ is the solution of the &lt;em&gt;dual&lt;/em&gt; (or &lt;em&gt;adjoint&lt;/em&gt;) problem&#xA;$$\label{adjoint}\tag{7}&#xA;a(v,w_\varphi) = (\varphi,v)_H \qquad\text{for all }v\in V&#xA;$$&#xA;(note that the test function $v$ is now the &lt;em&gt;first&lt;/em&gt; argument of $a$), which is assumed to be well-posed, too. This is the &lt;em&gt;Aubin-Nitsche Lemma&lt;/em&gt; (see &#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/a/26079&quot;&gt;my previous answer&lt;/a&gt; for a derivation in a specific case).&#xA;The idea is now to use the same interpolation error estimate for the &lt;em&gt;dual&lt;/em&gt; problem as well to get an additional power of $h$. First, you have as before&#xA;$$&#xA;    \inf_{w_h\in V_h}\|{w_\varphi-w_h}\|_V \leq \|{w_\varphi - \mathcal{I}_h w_\varphi}_V \| \leq C h\|w_\varphi\|_{{H}^2} \leq Ch \|{\varphi}\|_{H}.&#xA;$$&#xA;(In the last step, we have used the well-posedness of the dual problem to estimate the $H^2$ norm of the solution $w_\varphi$ by the $H=L^2$ norm of the right-hand side $\varphi$, which requires some assumptions on the PDE.)&#xA;Combining this with \eqref{aubin} and \eqref{apriori} gives the estimate&#xA;$$\|u-u_h\|_{H} \leq C h \|u-u_h\|_{V} \leq C h^{k-m+1}  \|u\|_{H^k}.$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So, to summarize, if you want to create your own exotic finite element spaces and show error estimates, you need to &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Find the correct energy space&lt;/strong&gt; $V$. (For the Poisson equation with homogeneous Dirichlet conditions, $V=H^1_0(\Omega)$.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Choose the discrete approximation space $V_h$ such that &lt;strong&gt;discrete functions in $V_h$ have the right global continuity properties to be in the $V$&lt;/strong&gt; (or you can estimate the additional terms from Strang's Lemma).  (For Poisson's equation, having $v_h$ continuously differentiable on each element and continuous across element boundaries to ensure $v_h\in V$.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Decide on the polynomial degree you want to represent to get the correct power of $h$, and then &lt;strong&gt;choose your degrees of freedom to a) have a unique solution of the interpolation problem in each element and b) ensure the global continuity properties&lt;/strong&gt; from step 2. (For the Poisson equation and the estimate you gave, this leads to piecewise linear polynomials, and the Lagrange elements correspond to choosing the degrees of freedom at the vertices of the element. Taking $m=1$ and $k=2$ in the interpolation error gives the estimate you stated.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Nitsche trick then gives you a free additional power of $h$ &lt;strong&gt;if you can show that the dual problem is well-posed&lt;/strong&gt; and you only measure the error in a weaker norm. (For the Poisson equation which is symmetric, the dual problem is of course another Poisson equation, and that step is a freebie.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;One final remark: In your case, I would recommend Braess's book over Brenner and Scott's; it's just as mathematically rigorous, but it's meant as a textbook instead of a monograph and thus concentrates on the fundamental ideas rather than the more general and technical results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Dietrich Braess&lt;/em&gt;, MR 2322235 &lt;a href=&quot;http://dx.doi.org/10.1017/CBO9780511618635&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Finite elements&lt;/strong&gt;&lt;/a&gt;, ISBN: 978-0-521-70518-9.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;(I also can't resist mentioning &lt;a href=&quot;https://www.uni-due.de/~adf040p/skripte/NumPDENotes13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;my lecture notes&lt;/a&gt;.)&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-02-16T21:26:52.153" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26197" PostTypeId="1" CreationDate="2017-02-16T12:35:53.213" Score="1" ViewCount="244" Body="&lt;p&gt;I have written a code which solves the incompressible formulation of the Navier-Stokes equations. It uses high-order methods both for time and spatial derivatives. I have been conducting convergence rate analysis to validate the high-order aspect of my solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For space convergence, the problem was relatively simple. For example, I choose a timestep of $\mathrm{dt} = 10^{-4}$, started with a 32-points ($\sqrt{n_x n_y}$) grid and refined my grid by factors of 2. At the 1024-grid, the $L_2$-error reached a plateau, which corresponds to the point where the leading error term comes from the time discretization scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The time convergence study was not as easy. I am using an explicit low-storage RK3 method. The problem is that I need to have a grid fine enough so that the leading error term is that of the time discretization. Based on the CFL condition, this yields a very small $\mathrm{dt}$ for the simulation to be stable. When reducing $\mathrm{dt}$ to compute the rate of convergence, I very quickly reach plateau, which is indicative that the leading error term comes from the spatial discretization. Therefore, I need to further refine my grid, which would in turn lead to a smaller time-step and so on. This feels like the snake biting its own tail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's say I start my convergence analysis in time for a 4096-points grid and a time-step of $\mathrm{dt} = 10^{-4}$. I obtain the $L_2$-error for $\mathrm{dt} = 5 \mathrm{x} 10^{-5}, 2.5 \mathrm{x} 10^{-5}, 1.25 \mathrm{x} 10^{-5}$ and observe an order of convergence of $2.99, 2.70$ and $0.26$, respectively, with that last value indicating that the leading error term has now becomes that of the spatial discretization. In the end, there isn't much convergence to show, as opposed to that for the spatial schemes mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now I am wondering, &lt;strong&gt;am I taking the right approach&lt;/strong&gt;? This seems overly complicated and time-consuming for a seemingly simple task. &lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="16329" LastEditDate="2017-02-16T19:07:06.463" LastActivityDate="2017-04-17T20:23:29.287" Title="Converge rate analysis: issue with time convergence" Tags="|pde|numerical-analysis|convergence|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26198" PostTypeId="2" ParentId="26192" CreationDate="2017-02-16T13:40:18.390" Score="0" Body="&lt;p&gt;In addition to AJK's recommendation, here are a couple more recommendations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Perturbation methods in fluid mechanics by Milton Van Dyke&lt;/li&gt;&#xA;&lt;li&gt;Introduction to Perturbation Methods by M.H. Holmes, (2013).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I recommend the second to get started on perturbation methods. The first one can be read after going through the first 2-3 chapters of the second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question about when the diffusion term can be ignored (in the first equation), it can be ignored when $u(0) \approx u(1)$. In this case, $u(x)$ will be more or less constant. This can of course be quantified. In your case with $u(0)=0$ and $u(1) = 2$, the diffusion term cannot be ignored. There is a boundary layer near $x=0$. See a plot of the exact solution below for $\epsilon = 0.01$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ylvkw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ylvkw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1156" LastActivityDate="2017-02-16T13:40:18.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26199" PostTypeId="1" CreationDate="2017-02-16T14:28:17.080" Score="4" ViewCount="790" Body="&lt;p&gt;I've recently written a code that solves the incompressible/low-Mach number formulation of the Navier-Stokes equation with high-order methods for both time and space. My advisor insisted that I use third order Runge-Kutta method. However, I do not find it to be adapted to our current problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In incompressible solvers, a poisson equation for the pressure must be solved, which is very CPU-time consuming and often the bottlenecks of such solvers (for low-Mach number problems, combustion might also be a bottleneck). When using a RK method, the poisson equation has to be solved at every RK-step, which is not efficient. Therefore, I think using a standard $3^{rd}$ order backward difference method might be more appropriate for our case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge, the main advantages of Runge-Kutta methods are that they are relatively &lt;strong&gt;simple to implement, self-starting and very stable&lt;/strong&gt;. In my case, only the last of these of importance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other advantages of the RK methods I am not aware of? Will I lose much accuracy/stability by switching to, say a $3^{rd}$ order backward difference method? Also, as mentioned in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26197/converge-rate-analysis-issue-with-time-convergence&quot;&gt;this post&lt;/a&gt;, I've proven than the spatial discretization error might be the leading error in my sims and therefore dictate the global accuracy of my code.&lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-02-21T20:41:51.220" Title="Time discretization: Runge-Kutta methods vs. standard backward difference" Tags="|finite-difference|fluid-dynamics|time-integration|runge-kutta|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26200" PostTypeId="1" AcceptedAnswerId="26201" CreationDate="2017-02-16T15:48:10.497" Score="6" ViewCount="4557" Body="&lt;p&gt;I think it's already been asked, but I still can't figure out a way to do it computationally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had to check for positive definiteness of an $n \times n$ matrix $A$. I know that for any nonzero column vector $x$, we have $x^{t}A x &amp;gt; 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to implement it in &lt;strong&gt;C++&lt;/strong&gt; to use it with the Cholesky method, in which I need:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;symmetric matrix; and&lt;/li&gt;&#xA;&lt;li&gt;positive definite matrix.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How do I approach this situation? If I go for the definition then it can be computationally inefficient as I have to check for each vector $x$. Is there any other method that I can use to test for positive definiteness?&lt;/p&gt;&#xA;" OwnerUserId="23297" LastEditorUserId="23297" LastEditDate="2017-03-01T13:45:25.200" LastActivityDate="2017-03-02T16:29:31.437" Title="Method to check for positive definite matrices" Tags="|linear-algebra|matrix|c++|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26201" PostTypeId="2" ParentId="26200" CreationDate="2017-02-16T17:41:07.760" Score="11" Body="&lt;p&gt;The standard way of approaching this is really to attempt a Cholesky factorization and check to see if such a factorization exists. This is both fast and realiable. Here it is in MATLAB notation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A = zeros(3); % some matrix&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[~,p] = chol(A)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the input matrix is not positive definite, then $p$ will be a positive integer, e.g. $p=1$ and MATLAB will not generate an error. $p$ could also be positive for some positive semi-definite matrices. Of course 'chol' only considers the upper (or lower) triangular part of the matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way is to check the following condition:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;p == 0 &amp;amp;&amp;amp; rank(A) == size(A,1)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;this involves @Bill Greene's suggestion. Likewise, you might use the Eigen decomposition:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;all(eig(A) &gt; eps)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's it to the best of my knowledge. Note that MATLAB uses a Lapack library internally (such as MKL) and you should be able to get similar functionality out of any Lapack library in C++.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2017-02-16T17:56:24.683" LastActivityDate="2017-02-16T17:56:24.683" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26202" PostTypeId="2" ParentId="26192" CreationDate="2017-02-16T18:29:15.520" Score="3" Body="&lt;p&gt;In the time-dependent nonlinear case, if you drop the diffusive term then you have a nonlinear hyperbolic problem.  Solutions will naturally generate singularities (discontinuities) in finite time.  To extend the solution beyond that time, one must consider weak solutions, and uniqueness is lost.  To specify a unique, physically relevant solution one typically invokes an entropy condition, which is equivalent to enforcing that the weak solution is the &quot;vanishing-viscosity limit&quot; of a diffusive equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The canonical reference in this area is Whitham.  I am partial to the two texts by LeVeque, although they focus on numerics more than theory.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-02-16T18:29:15.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26203" PostTypeId="2" ParentId="26197" CreationDate="2017-02-16T19:29:25.500" Score="2" Body="&lt;p&gt;It depends on exactly what you want to validate. Usually one wishes to confirm convergence of the full discretization, which requires simultaneous refinement in space and time. I would recommend doing your convergence study that way.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do want to investigate purely convergence of the time discretization and you are using the method of lines, I think it's correct to think about convergence to the exact solution of the semi-discrete system, rather than of the PDE. This is often done in the literature on numerical time discretization. &lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-02-16T19:29:25.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26204" PostTypeId="2" ParentId="26200" CreationDate="2017-02-16T20:09:46.813" Score="1" Body="&lt;p&gt;If your idea is to use this info for the Cholesky decomposition I assume that you are working with symmetric matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of $A \in M(\mathbb{R}^n)$ and a, i.e. a real matrix, we can add to @Tolga_Birdal's answer the consideration that all the  eigenvalues are real. With this you can check only the smallest eigenvalue, if it is $\lambda_{min} &amp;gt; 0$, not all.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="3441" LastEditDate="2017-02-16T21:26:57.110" LastActivityDate="2017-02-16T21:26:57.110" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26205" PostTypeId="1" AcceptedAnswerId="26206" CreationDate="2017-02-16T23:16:53.973" Score="0" ViewCount="115" Body="&lt;p&gt;I have spend the last few days working my way &lt;a href=&quot;http://www.ijmse.net/uploadfile/2014/0619/20140619114601783.pdf&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Enhanced Calibration of Robot Tool Centre Point Using Analytical Algorithm&quot;&gt;through an interesting paper&lt;/a&gt; and I'm building a numerical model so I can apply the method. However, I am getting stuck at an &lt;em&gt;&quot;it can be shown&quot;&lt;/em&gt; step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to convert the following system into the form $A\cdot{x}=b$. Have I missed something obvious, I really don't know how to go about this step. The paper just says &quot;can be simplified to the general form&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rcdMx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rcdMx.png&quot; alt=&quot;Equation 19 from doi: 10.12720/ijmse.3.1.12-18&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\boldsymbol{I}$ is a 3x3 identity matrix, $\boldsymbol{A}$, $\boldsymbol{B}$, $\boldsymbol{C}$ and $\boldsymbol{D}$ are 3x3 matrices and $\boldsymbol{t_A}$, $\boldsymbol{t_B}$, $\boldsymbol{t_C}$ and $\boldsymbol{t_D}$ are 3x1 vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The the size of the matrix is 24x12, the size of the unknown matrix and the right-hand side is 12x1.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastActivityDate="2017-02-17T06:52:58.730" Title="Writing a non-square linear system in standard form $A\cdot{x}=b$" Tags="|linear-algebra|linear-solver|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26206" PostTypeId="2" ParentId="26205" CreationDate="2017-02-17T06:52:58.730" Score="8" Body="&lt;p&gt;The matrix math in this paper is terribly hard to follow, but here's what equation (19) &lt;em&gt;should&lt;/em&gt; look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( &#xA; \begin{array}{c|c|c|c}&#xA;   c_{11} \mathbf{I} - \mathbf{A}^T &amp;amp; c_{12} \mathbf{I} &amp;amp; c_{12} \mathbf{I} &amp;amp; \mathbf{0}^{3\times 3} \\&#xA;\hline&#xA;   c_{21} \mathbf{I} &amp;amp; c_{22} \mathbf{I} - \mathbf{A}^T  &amp;amp; c_{23} \mathbf{I} &amp;amp; \mathbf{0}^{3\times 3} \\&#xA;\hline&#xA;   c_{31} \mathbf{I} &amp;amp; c_{32} \mathbf{I}  &amp;amp; c_{33} \mathbf{I} - \mathbf{A}^T &amp;amp; \mathbf{0}^{3\times 3} \\&#xA;\hline&#xA;   d_{11} \mathbf{I} - \mathbf{B}^T &amp;amp; d_{12} \mathbf{I} &amp;amp; d_{12} \mathbf{I} &amp;amp; \mathbf{0}^{3\times 3} \\&#xA;\hline&#xA;   d_{21} \mathbf{I} &amp;amp; d_{22} \mathbf{I} - \mathbf{B}^T  &amp;amp; d_{23} \mathbf{I} &amp;amp; \mathbf{0}^{3\times 3} \\&#xA;\hline&#xA;   d_{31} \mathbf{I} &amp;amp; d_{32} \mathbf{I}  &amp;amp; d_{33} \mathbf{I} - \mathbf{B}^T &amp;amp; \mathbf{0}^{3\times 3} \\&#xA;\hline&#xA;   \mathbf{t}_A^T &amp;amp; \mathbf{0}^{1\times 3} &amp;amp; \mathbf{0}^{1\times 3} &amp;amp; \\&#xA;   \mathbf{0}^{1\times 3} &amp;amp; \mathbf{t}_A^T &amp;amp; \mathbf{0}^{1\times 3} &amp;amp; \mathbf{I}-\mathbf{C}\\&#xA;    \mathbf{0}^{1\times 3} &amp;amp; \mathbf{0}^{1\times 3} &amp;amp; \mathbf{t}_A^T &amp;amp; \\&#xA;\hline&#xA;   \mathbf{t}_B^T &amp;amp; \mathbf{0}^{1\times 3} &amp;amp; \mathbf{0}^{1\times 3} &amp;amp; \\&#xA;   \mathbf{0}^{1\times 3} &amp;amp; \mathbf{t}_B^T &amp;amp; \mathbf{0}^{1\times 3} &amp;amp; \mathbf{I}-\mathbf{D}\\&#xA;    \mathbf{0}^{1\times 3} &amp;amp; \mathbf{0}^{1\times 3} &amp;amp; \mathbf{t}_B^T &amp;amp; \\&#xA; \end{array} &#xA; \right)&#xA;\left(&#xA;\begin{array}{c}&#xA;\mathbf{x}_1 \\&#xA;\hline&#xA;\mathbf{x}_2 \\&#xA;\hline&#xA;\mathbf{x}_3 \\&#xA;\hline&#xA;\mathbf{t}_x \\&#xA;\end{array}&#xA;\right)&#xA;=&#xA;\left(&#xA;\begin{array}{c}&#xA;\mathbf{0}^{3\times 1}  \\&#xA;\hline&#xA;\mathbf{0}^{3\times 1} \\&#xA;\hline&#xA;\mathbf{0}^{3\times 1} \\&#xA;\hline&#xA;\mathbf{0}^{3\times 1} \\&#xA;\hline&#xA;\mathbf{0}^{3\times 1} \\&#xA;\hline&#xA;\mathbf{0}^{3\times 1} \\&#xA;\hline&#xA;\\&#xA;\mathbf{t}_C \\&#xA;\\&#xA;\hline&#xA;\\&#xA;\mathbf{t}_D \\&#xA;\\&#xA;\end{array}&#xA;\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used dividing lines to help with alignment. Each block within lines in the matrix is $3 \times 3$. I've marked zero blocks with superscripts to indicate their size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following matrices are dimension $3 \times 3$: $\mathbf{A}$, $\mathbf{B}$, $\mathbf{C}$, $\mathbf{D}$, and $\mathbf{I}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following vectors are dimension $3 \times 1$: $\mathbf{x}_1$, $\mathbf{x}_2$, $\mathbf{x}_3$, $\mathbf{t}_x$,  $\mathbf{t}_A$,  $\mathbf{t}_B$, $\mathbf{t}_C$, and $\mathbf{t}_D$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following are scalars: $c_{ij}$, $d_{ij}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes the whole system &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{N}^{24 \times 12} \mathbf{x}^{12 \times 1} = \mathbf{b}^{24 \times 1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most obvious way to solve this in the least squares sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{N}\mathbf{x} = \mathbf{b}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{N}^T\mathbf{N}\mathbf{x} = \mathbf{N}^T\mathbf{b}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{x} = \left( \mathbf{N}^T\mathbf{N}\right)^{-1}\mathbf{N}^T\mathbf{b}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not explicitly said in the paper that this is what they did, but it's possible. &lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2017-02-17T06:52:58.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26208" PostTypeId="1" CreationDate="2017-02-17T15:34:03.027" Score="5" ViewCount="725" Body="&lt;p&gt;I've recently got interested in CFD and started a small project by solving the radial Reynoldsequation. Why the Reynoldsequation? I recently encountered it through my studies and somehow got stuck :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would greatly appreciate it, if someone were so kind to have a read through my steps and point out my thought mistakes or any other errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a first approach, I would like to solve it without the source term, which reduces it basically to a two dimensional diffusive process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial}{\partial r}(rh^3\frac{\partial p}{\partial r}) + \frac{1}{r}\frac{\partial}{\partial \theta}(h^3\frac{\partial p}{\partial \theta}) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one were to imaging a ring, with two Diriclet Boundary Conditions:&#xA;$p(r_I) = 2$ and $p(r_O) = 250$, so I don't need to worry about the singularity at $r=0$. I also have a periodic condition along the angle at $\theta = 0$ and $\theta = 2\pi$&#xA;The actual pressure values don't matter, as long as they are larger as zero. Negative pressures don't really work :D In the end, I shall have a unsymmetrical problem with several boundary conditions on the ring, that's why I don't wish to reduce this to a one dimensional one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following the approach by Patanker or any other book on Finite Volume Methods (such as The Finite Volume Method in Computational Fluid Dynamics by F. Moukalled), the semi discretization with the midpoint rule will give me a second order accurate approximation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_eh_e^3\Delta \theta(\frac{\partial p}{\partial r})_e - r_wh_w^3\Delta \theta(\frac{\partial p}{\partial r})_w + \frac{\Delta r}{r_i}(h_s^3(\frac{\partial p}{\partial \theta})_s - h_n^3(\frac{\partial p}{\partial \theta})_n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That can be confirmed using the Taylor analysis for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convenience, I wrote the code in Matlab, so the East direction is $j+1$ and the South direction is $i+1$ for a matrix given by $M(i,j)$ Lowerbound letters, such as e and s denote the faces midway between two points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a structured grid and equidistant in r and $\theta$ ($\Delta r \neq \Delta \theta$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm getting to the part, where I've been scratching my head for some time now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one were to approximate the gradient at the faces of the control volume like:&#xA;$(\frac{\partial p}{\partial r})_e = \frac{p_E - p_i}{\Delta r}$ and $(\frac{\partial p}{\partial r})_w = \frac{p_i - p_W}{\Delta r}$ and similar for the south and north faces, the formal order of accuracy is what?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taylor analysis yields a formal order of accuracy of $1$ for the gradient approximation as above, while my books and other sources (internet etc.) refer to the combined discretization (see below) as of order $2$ on a uniform grid and others as of order $1$. They contradict themselves, which is kind of annoying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{r_eh_e^3(p_E-p_i)\Delta \theta}{\Delta r} - \frac{r_wh_w^3(p_i - p_W)\Delta \theta}{\Delta r}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand, that when I choose $d_1 = r_eh_e^3\Delta \theta$ and $d_2 = r_wh_w^3\Delta \theta$ (similar for South and North) and in the special case of $d_1 = d_2$, the above discretization would become a second order central difference scheme look a like. But I don't have that case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{d (p_E - 2p_i + p_W)}{\Delta r}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented the discretization and performed an order of accuracy study yielding an order of accuracy of $1$ by comparing the solution of the scheme in radial coordinates to the exact solution. I did that by integrating the resulting pressure field to obtain the hydrostatic lift force. I can now compare the numerical one to the exact one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is this actually allowed? The order of accuracy I get is basically 1 (+/- 0.003) for fine grids and starts to oscillate for coarser grids as expected. Would it be better to directly compare the resultant pressure fields?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's basically it. I sifted through so many texts that I'm now confused to a point, where every explanation I try to come up with is clouded in self doubt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I would like to find out, which formal order of accuracy I should obtain and  how I can adapt the discretization to achieve an order of accuracy of $2$.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above discretization's of the gradients are basically a linear profile between two points, but I'm not sure how a higher order profile would look like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a staggered grid, where the pressure is stored at the centre of the cell and the height at the cell faces. The radii, for example $r_e$, are not approximated but direclty computed at the position of the faces by $(j+c \pm 0.5)*\Delta r$, where c is a constant to account for the inner radius, as my domain does not extend to $r=0$ but starts at $r_I$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be able to use the exact solution, the height is currently uniform and could be taken out of the equation, but will be later evaluted through the interpolation of a matrix. Additionally, the density and viscosity could also be added, currently assumed constant, and would also be evaluated at the cell faces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also updated the above equations, such that capital letters denote a cell centre and the small letters the cell faces, where &lt;strong&gt;i&lt;/strong&gt; is also at the centre of a cell and conincides with the centre of my stencil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently using these three books:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: Patanker: Numerical Heat Transfer and Fluid Flow &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0891165223&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2: F. Moukalled: The Finite Volume Method in Computational Fluid Dynamics &lt;a href=&quot;http://www.springer.com/de/book/9783319168739&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3: J.H. Ferziger: Computational Methods for Fluid Dynamics &lt;a href=&quot;http://www.springer.com/de/book/9783540420743&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Patanker doesn't go much into detail regarding the order of accuracy, while in the second book they note (page: 216), that the semi discretization is of order two, but the linear profile used to approximate the gradient is of first order, which would make the complete approximation be of order $1$ from my point of view.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The third book goes more into detail regarding the linear profile used for the gradient on pages: 72-74. They note, that the linear gradient approximation is of formal order $1$, but behaves as a second order approximation on a uniform grid, when one chooses the faces midway between two neighbouring points, as I have. On Page 75 it states, that the scheme converges asymptotically in a second order manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Several lecture notes on the internet just mentioned, it would be second order &#xA;accurate without going much into detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\enclose{horizontalstrike}{&lt;em&gt;After going through the texts again, from my understanding, I should obtain a order of accuracy of one, which I do. Yet, I would like to be certain of it^^&lt;/em&gt;} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Answer&lt;/strong&gt;&#xA;May I answer my own question? Well, anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've now been able to obtain an order of accuracy of $2$. I had the error norm computed wrong or in other words, I did not scale the error of the obtained lift force to the grid.&lt;/p&gt;&#xA;" OwnerUserId="23063" LastEditorUserId="23063" LastEditDate="2017-02-24T08:19:29.373" LastActivityDate="2017-02-24T08:19:29.373" Title="Order of accuracy of FVM discretization" Tags="|finite-volume|discretization|accuracy|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26210" PostTypeId="1" AcceptedAnswerId="26212" CreationDate="2017-02-18T15:17:50.513" Score="2" ViewCount="370" Body="&lt;p&gt;When simulating a partial differential equation describing a physical phenomenon like vibrations on a string, fluid flow in a chamber or quantum wave functions, the most straight-forward way is to divide your space into small regions / cells, set up the relevant values at each cell according to the initial condition, and simulate one time step at a time using your favourite partial differential equation algorithm, making sure the boundary conditions are upheld at all times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the simplest example, say we have vibrations in a tight, uniform string. If the vibrations are small enough, we can assume that each part of the string is only moving up and down. Each cell is then a miniscule part of the string, and to each cell belongs a position and velocity. The model is that each cell wants to be directly in-between the two cells on either side, and it feels an accelration proportional to how far from that position it is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If both ends of the string are fastened, then the boundary condition is simple; the first and last cells are special and do not feel any accelration. However, say I just want to study what happens near a single end as, say, a wave hits it and reflects back. Then it's not so simple to see how to treat the opposite end. I can't just let it hang free (saying it wants to be level with the cell before), because that would cause reflections to come back. &lt;a href=&quot;https://phet.colorado.edu/sims/html/wave-on-a-string/latest/wave-on-a-string_en.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This simulation&lt;/a&gt; shows more or less what I'm talking about (&quot;Loose end&quot; vs. &quot;No end&quot;). How is that done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarily, say I want to simulate air flow over an airfoil (say using a two-dimensional Navier-Stokes model). Uniform wind comes in on one end of my simulated chamber, and that's simple enough, conceptually. But how do I treat the end where the wind leaves the chamber in a correct way? or the top and bottom? I assume it goes more or less along the same lines, at least intuitively, as the string. If not, the string is the main part of my question.&lt;/p&gt;&#xA;" OwnerUserId="23318" LastEditorUserId="9667" LastEditDate="2017-02-18T15:42:04.097" LastActivityDate="2017-02-18T16:36:40.597" Title="How do I simulate an open end?" Tags="|pde|boundary-conditions|simulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26211" PostTypeId="1" AcceptedAnswerId="26215" CreationDate="2017-02-18T15:56:25.473" Score="0" ViewCount="170" Body="&lt;p&gt;In a initial value problem does the initial condition has to satisfy the boundary condition and the governing equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: If a non-homogeneous Neumann boundary condition for the pressure in a fluid dynamics problem is applied on a boundary, can one initialize a zero pressure field? &lt;/p&gt;&#xA;" OwnerUserId="8166" LastEditorUserId="8166" LastEditDate="2017-02-18T16:11:51.640" LastActivityDate="2017-02-19T04:11:30.403" Title="Initial Condition in a Numerical Problem" Tags="|finite-difference|numerical-analysis|boundary-conditions|finite-volume|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26212" PostTypeId="2" ParentId="26210" CreationDate="2017-02-18T16:36:40.597" Score="10" Body="&lt;p&gt;The problem you describe, how to prescribe non-reflecting&#xA;or absorbing boundary conditions when solving partial differential&#xA;equations (PDE) has been extensively studied. For complex (e.g. nonlinear)&#xA;systems of PDE and general boundaries, it can be quite challenging and&#xA;is something of an ongoing research problem. You can find many references&#xA;on the topic, for example &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.557.9856&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;noreferrer&quot;&gt;Huan and Thompson&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for dynamics of a one-dimensional string, there is a simple solution.&#xA;As you may already know, this problem is described by the one-dimensional wave&#xA;equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial^2 u}{\partial x^2} = \frac{1}{c^2} \frac{\partial^2 u}{\partial t^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c^2=T/\rho$, $T$ is the tension in the string, and $\rho$ is the density per&#xA;unit length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the right end of the string, to make the wave not reflect back,&#xA;it is necessary to impose the following boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial x} = -\frac{1}{c}\frac{\partial u}{\partial t}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see how this condition is derived, you can take a look at page 28 in this set&#xA;of class notes &lt;a href=&quot;http://gwu.geverstine.com/pdenum.pdf&quot; rel=&quot;noreferrer&quot;&gt;Everstine&lt;/a&gt;&#xA;which also show how to prescribe this condition in a finite difference implementation.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-02-18T16:36:40.597" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26215" PostTypeId="2" ParentId="26211" CreationDate="2017-02-19T04:11:30.403" Score="4" Body="&lt;p&gt;That depends on the equation you have, and on the situation you want to model. Imagine, for example, that you are considering the advection equation&#xA;$$&#xA;  \partial_t u + c \partial_x u = 0,&#xA;$$&#xA;i.e., you are transporting a concentration of a substance along with velocity $c$ from the left to the right. It does make sense to say that at time $t=0$, the concentration everywhere is zero, i.e. $u(x,0)=0$ but that the &quot;reservoir&quot; that lies beyond the left edge has a concentration 1, i.e. $u(0,t)=1$, so that a fluid with a high concentration flows in from the left, replacing the fluid with low concentration that was there before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, if you consider the heat equation&#xA;$$&#xA;  \partial_t u - k \partial_{xx} u = 0,&#xA;$$&#xA;then you can think of a situation where you model a solid that has initial temperature $u(x,0)=0$ but that starting at time $t=0$, you put a heat bath with temperature 1 in contact with the left or right end of the solid, i.e., $u(0,t)=1$, so that it heats the solid from the left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you had the Stokes equation&#xA;$$&#xA;  \partial_t u - \eta \Delta u + \nabla p= 0,&#xA;  \\&#xA;  \nabla\cdot u = 0,&#xA;$$&#xA;then you also have to prescribe initial and boundary values for the velocity. Here, I cannot immediately conjure a situation where the initial velocity is zero (which implies that up until $t=0$, the walls that enclose the fluid are also at rest), but that starting at $t=0$ the walls move at a certain velocity, thereby imparting a nonzero boundary value on the velocity. That is because you would have to accelerate the walls from rest to a nonzero velocity instantaneously -- something that require infinite acceleration and consequently infinite forces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The considerations above are all from a physical perspective. Mathematically, such non-matching initial/boundary conditions introduce a discontinuity in the solution at the initial time and the boundary. Consequently, you will get a solution that lacks regularity at that location. That is not, a priori, a problem. However, it requires careful thought about the appropriate function spaces for your solution and, in the weak form, the test functions. You will also in general have solutions that only satisfy the weak form, not the strong form of the equation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-02-19T04:11:30.403" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26217" PostTypeId="2" ParentId="26199" CreationDate="2017-02-19T12:49:41.943" Score="0" Body="&lt;p&gt;If you discretize the problem in to ODEs, then you shouldn't need to worry about this because you likely shouldn't be writing the ODE solver anyways. At that point, plug the discretized ODEs into whatever solver is available in your programming language and swap around, try BDF methods or Runge-Kutta methods, and see what's faster. You're not going to beat that by writing a special purpose method, and using an ODE package will have a lot of stuff already built into it (like adaptive timestepping) which will be very helpful for solving the problem.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-02-19T12:49:41.943" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26218" PostTypeId="1" AcceptedAnswerId="26225" CreationDate="2017-02-19T20:43:45.253" Score="1" ViewCount="282" Body="&lt;p&gt;How do we correctly define the flux in a finite volume method applied to Poisson's equation where we have a piecewise constant material?  Specifically, say we have the equation&#xA;\begin{align*}&#xA;-\nabla\cdot(k\nabla u) =&amp;amp; f\textrm{ on }\Omega\\&#xA;u =&amp;amp; 0\textrm{ on }\partial\Omega&#xA;\end{align*}&#xA;where $\Omega$ is some kind of easy, square domain in $\mathbb{R}^2$.  Then, let's mesh $\Omega$ using square elements where one of the cells is $K$.  Then, on the interior, we can integrate over $K$, which gives&#xA;$$&#xA;\int_{K}-\nabla\cdot(k\nabla u) = \int_{K}f&#xA;$$&#xA;After using the divergence theorem, we have&#xA;$$&#xA;\int_{\partial K}-k(\nabla u\cdot n) = \int_{K}f&#xA;$$&#xA;Now, we have to discretize the quantity on the left, which can be divided into four separate integrals.  Assume that center of cell $K$ is $(x_i,y_j)$ and let $u_{ij}=u(x_i,y_j)$ and $k_{ij}=k(x_i,y_j)$.  Then, we can discretize $\nabla u \cdot n$ into four different pieces, depending on what side of the square we're on, as&#xA;\begin{align*}&#xA;&amp;amp;\frac{u_{ij}-u_{{i-1},j}}{\Delta x}\\&#xA;&amp;amp;\frac{u_{ij}-u_{{i+1},j}}{\Delta x}\\&#xA;&amp;amp;\frac{u_{ij}-u_{i,{j-1}}}{\Delta y}\\&#xA;&amp;amp;\frac{u_{ij}-u_{i,{j+1}}}{\Delta y}&#xA;\end{align*}&#xA;What I'm not clear about is what to do about $k$ if we assume that $k$ is piecewise constant and defined on the same mesh as $u$.  We could just multiply the fluxes by $k_{ij}$.  For example, discretize $-k(\nabla u\cdot n)$ along the left face as&#xA;$$&#xA;k_{ij}\left(\frac{u_{ij}-u_{{i-1},j}}{\Delta x}\right)&#xA;$$&#xA;However, in this case, the flux on the other side would be&#xA;$$&#xA;k_{{i-1},j}\left(\frac{u_{{i-1},j}-u_{i,j}}{\Delta x}\right)&#xA;$$&#xA;so they wouldn't match and be conservative because they're using two different material values.  The best I can think of is to average the material and use&#xA;$$&#xA;\left(\frac{k_{ij}+k_{{i-1},j}}{2}\right)\left(\frac{u_{ij}-u_{{i-1},j}}{\Delta x}\right)&#xA;$$&#xA;but I'm not sure if this is correct or the implications of this choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, how do we discretize the flux when we have a piecewise constant material, $k$, defined on the same mesh as our material $u$.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2017-02-20T10:40:06.627" Title="How to correctly define the flux in a finite volume method for Poisson's equation with a piecewise constant material" Tags="|pde|numerical-analysis|finite-volume|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26220" PostTypeId="2" ParentId="20387" CreationDate="2017-02-19T21:42:14.350" Score="2" Body="&lt;p&gt;I suppose there could be some difference between how line-search and trust-region methods handle scaling, but I really don't see it bear out in practice as long as we're aware of the scaling.  And, to be clear, the Nocedal and Wright book was talking about affine scaling.  Nonlinear scaling is somewhat trickier to quantify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why, say we want to minimize $f:X\rightarrow \mathbb{R}$, but we want to scale the variables by some kind of nonsingular, self-adjoint operator $A\in\mathscr{L}(X)$.  Define $J:X\rightarrow \mathbb{R}$ as the scaled objective function.  Then,&#xA;\begin{align*}&#xA;J(x) =&amp;amp; f(Ax)\\&#xA;\nabla J(x) =&amp;amp; A\nabla f(Ax)\\&#xA;\nabla^2 J(x) =&amp;amp; A\nabla^2 f(Ax) A&#xA;\end{align*}&#xA;The real difference in algorithms is what happens to the scaling $A$.  In Newton's method, we solve&#xA;$$&#xA;\nabla^2 J(x) \delta x = -\nabla J(x)&#xA;$$&#xA;or&#xA;$$&#xA;A\nabla^2 f(Ax) A \delta x = -A\nabla f(Ax)&#xA;$$&#xA;Assuming the Hessian is nonsingular, we have&#xA;$$&#xA;A \delta x = -\nabla^2 f(Ax)^{-1} \nabla f(Ax)&#xA;$$&#xA;Basically, the scaling cancels out and disappears, so it doesn't affect the direction.  That's why we say Newton's method is affine scale invariant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alright, so now let's say that we don't have the Hessian.  Really, at the end of the day, trust-region methods rely on solving the system&#xA;$$&#xA;H \delta x = -\nabla J(x)&#xA;$$&#xA;for some kind of Hessian approximation $H$.  Most of the time, we're going to use Steihaug-Toint truncated-CG because it works well.  If we plug back in our scaling, we have&#xA;$$&#xA;H \delta x = -A \nabla f(Ax)&#xA;$$&#xA;If we're throwing CG at this system, that basically means that we have one tool for dealing with the scaling $A$ and that's the Hessian or its approximation $H$.  Theoretically, we could change the shape of trust-region, but all that really means is cutting off our step earlier or later.  This does affect the step, but I've always found it a pain to control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a line-search method, we can view our iterate as applying some kind of magic function $\phi$ to our gradient.  Hence, for the scaled direction:&#xA;$$&#xA;\delta x = \phi(-A\nabla f(Ax))&#xA;$$&#xA;Maybe $\phi$ computes the Newton step.  Maybe it $\phi$ computes the BFGS step.  Whatever.  Certainly, we have some restrictions such as we likely need $\phi$ to produce a descent direction, but it does highlight that there's a great amount of flexibility here.  That means that we have a greater amount of tools at our disposal to handle the scaling $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what are these tools and should we be using them?  Personally, I think the answer is no.  Unless you really know your application and have a specialized algorithm to find the solution, inexact Newton methods work really, really well.  By inexact Newton, I mean solving the system&#xA;$$&#xA;\nabla^2 J(x) \delta x = -\nabla J(x)&#xA;$$&#xA;inexactly using CG.  This is precisely using Steihaug-Toint in the trust-region setting (p. 171 in Nocedal and Wright) or Newton-CG for line-search (p. 169 in Nocedal and Wright).  They work pretty close to the same and they don't care about affine scaling.  They also don't require storing the Hessian, only Hessian-vector products are required.  Really, these algorithms should be the workhorses for most problems and they don't care about affine scaling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the preconditioner for the trust-region problem, I don't think there's any easy way to tell apriori if you're going to improve the number of overall optimization iterations or not.  Really, at the end of the day, optimization methods operate in two modes.  In mode one, we're too far from the Newton's method convergence radius, so we globalize and just force the iterates to insure that the objective goes down.  Trust-region is one way.  Line-search is another.  In mode two, we're in the Newton's method convergence radius, so we try not to mess with it and let Newton's method do it's job.  In fact, we can see this in the convergence proofs of things like trust-region methods.  For example, look at Theorem 4.9 (p.93 in Nocedal and Wright).  Very explicitly, they state how the trust-region becomes inactive.  In this context, what's the utility of the preconditioner?  Certainly, when we're in the Newton's method convergence radius, we do far less work and the number of CG iterations goes down.  What happens when we're outside this radius?  It sort of depends.  If we calculate the full-Newton step, then the benefit is that we did less work.  If we cut off our step early due to truncation from truncated-CG, then our direction will be in the Krylov subspace&#xA;$$&#xA;\{-P\nabla J(x),-(PH)(P\nabla J(x)),\dots,-(PH)^k(P\nabla J(x))\}&#xA;$$&#xA;where $P$ is the preconditioner and $H$ is our Hessian approximation.  Is this a better subspace to find a direction in than&#xA;$$&#xA;\{-\nabla J(x),-(H)(\nabla J(x)),\dots,-(H)^k(\nabla J(x))\}?&#xA;$$&#xA;It's hard to tell.  Maybe.  Maybe not.  The theory just tells us that we converge in a finite number of steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't mean that there's not value in defining a good preconditioner.  However, I'm not sure how someone defines a preconditioner to assist in the optimization for points away the Newton's method convergence radius.  Typically, we design a preconditioner to cluster the eigenvalues of the Hessian approximation, which is a tangible, measurable goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;tldr; Practically speaking, there are a greater variety of ways for a line-search method to generate an iterate than a trust-region method, so it's possible there's an amazing way to handle affine scaling.  However, just use an inexact Newton method and it doesn't matter.  A preconditioner does affect the performance of an algorithm away from the Newton's method convergence radius, but it's hard to quantify how, so just design a preconditioner to cluster the eigenvalues of the Hessiasn approximation.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2017-02-19T21:42:14.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26221" PostTypeId="2" ParentId="19826" CreationDate="2017-02-19T21:57:09.397" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.optimojoe.com/products/optizelle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optizelle&lt;/a&gt; should be able to handle this.  It's open-source and BSD licensed.  Specifically, it will solve a trust-region subproblem of the form&#xA;$$\begin{array}{rcl}&#xA;\min\limits_{\delta x \in X} &amp;amp;&amp;amp; \frac{1}{2}\langle H\delta x,\delta x\rangle + \langle \nabla f(x),\delta x\rangle\\&#xA;\textrm{st} &amp;amp;&amp;amp; \langle \delta x,\delta x\rangle \leq \Delta^2&#xA;\end{array}$$&#xA;where the user can define an arbitrary preconditioner $P$ and inner product $\langle \cdot,\cdot\rangle$.  As such, if you want a nonspherical shaped trust-region, redefine the inner product into something more oblique.  Be careful, because the this also affects the definition of the gradient.  If you want a preconditioner that changes optimization iteration to optimization iteration, that's fine.  The CG system solved will be&#xA;$$&#xA;PH \delta x = -P\nabla f(x).&#xA;$$&#xA;Technically, if want to change the inner product every iteration, that's also possible, but needs to be done very carefully.  There's a mechanism called the state manipulator that can inject arbitrary code.  At the end of the optimization iteration, you could inject some code to update your inner product, but you'd also need to update the cached gradient since this would also be affected.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2017-02-19T21:57:09.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26222" PostTypeId="1" CreationDate="2017-02-20T01:34:40.420" Score="3" ViewCount="193" Body="&lt;p&gt;In a book that our course is following this semester, the theorem given is only in one direction:  if the scheme is both consistent and stable, then the scheme is convergent.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since this isn't a double implication theorem, then are there convergent schemes that are either consistent but not stable, or stable but not consistent?  &lt;/p&gt;&#xA;" OwnerUserId="23333" LastEditorUserId="9667" LastEditDate="2017-02-20T04:18:14.103" LastActivityDate="2017-02-21T01:50:12.277" Title="In numerical methods, eg, finite differencing approaches, does there exist convergent schemes that are not both consistent and stable?" Tags="|finite-difference|numerical-analysis|convergence|stability|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26223" PostTypeId="2" ParentId="26200" CreationDate="2017-02-20T05:41:57.963" Score="10" Body="&lt;p&gt;Mostly, I'm leaving this answer here as a cautionary tale to not use a Choleski factorization.  Most of the time, this is a fine answer.  However, very specifically, it can and will fail, so if this is in a process where people can be hurt, please do not use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, a Choleski factorization exists for matrices that have zero eigenvalues.  This is a property that exists without round-off error.  To see this, we can look at a $2\times 2$ matrix.  Here, we have&#xA;\begin{align*}&#xA;L_{11} =&amp;amp; \sqrt{A_{11}}\\&#xA;L_{21} =&amp;amp; \frac{A_{21}}{\sqrt{A_{11}}}\\&#xA;L_{22} =&amp;amp; \sqrt{A_{22}-A_{21}^2/A_{11}}&#xA;\end{align*}&#xA;Then, consider the matrix&#xA;$$&#xA;\begin{bmatrix}2 &amp;amp; 2\\2 &amp;amp; 2\end{bmatrix}&#xA;$$&#xA;The Choleski factorization is&#xA;$$&#xA;\begin{bmatrix}\sqrt{2} &amp;amp; 0\\\sqrt{2} &amp;amp; 0\end{bmatrix}&#xA;$$&#xA;Clearly, this matrix is rank-1 and has a zero eigenvalue.  Further, MATLAB is perfectly willing to compute it&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; A=ones(2)*2; chol(A)    &#xA;&#xA;ans =&#xA;&#xA;    1.4142    1.4142&#xA;         0    0.0000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Long story short, Choleski works fine both theoretically and practically for some matrices at have zero eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, the question becomes: With round off error, can we Choleski factor a matrix with negative eigenvalues?  The answer is also yes.  Here's a code to find one for you:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:100&#xA;    try&#xA;        A11=rand*1e5;&#xA;        A22=rand;&#xA;        A21=sqrt(A11*A22)+1e-14;&#xA;        A=[A11 A21;A21 A22];&#xA;        U=chol(A);&#xA;        [v d]=eig(A);&#xA;        if d(1) &amp;lt; 0&#xA;            break;&#xA;        end &#xA;    catch myerr&#xA;        i&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I just ran it and found:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   7.89575133795518e+04   2.47689038064322e+02&#xA;   2.47689038064322e+02   7.76998374838857e-01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note, the matrix is design precisely to be indefinite with a negative eigenvalue.  With round off, sometimes even &lt;code&gt;eig&lt;/code&gt; gives the wrong answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where does this leave us?  Unfortunately, not in a good spot.  Most of the time, trying a Choleski factorization is the way to go.  It's fast and is mostly reliable.  However, as I said at the top, this can and will fail.  I had no idea until it broke one of my codes.  Now, I have safety margins.  Is there a more reliable way?  Sometimes.  If you use something like an implicitly restarted Arnoldi method, such as with ARPACK, and you ask for the smallest eigenvalue with an error tolerance on the computation.  If the smallest eigenvalue minus this error tolerance is greater than zero, then you're good to go.  If not, there can be trouble, which requires additional investigation.  Beyond that, I don't know of any perfectly reliable way to check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, let me state one more time because it's important:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;With round off, a Choleski factorization can be taken of a matrix with negative eigenvalues.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Edit 1&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There was a follow-up question on how to accomplish this in C++.  Unfortunately, I don't know a fully straightforward way, but I can offer a few suggestions.  Note, each of these methods involves calculating the smallest eigenvalue of the symmetric matrix to a specified tolerance.  If the smallest eigenvalue minus the tolerance is greater than or equal to zero, then we know we're positive definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Frankly, your best bet is to use ARPACK.  This is a Fortran code, but if you compile it with something like gfortran and write your own headers, it will work just fine.  Using Fortran is a little tricky in C++.  You'll have to turn off C++ name mangling with &lt;code&gt;extern &quot;C&quot; {}&lt;/code&gt; and determine the Fortran name mangling scheme which can be &lt;code&gt;FOO&lt;/code&gt;, &lt;code&gt;FOO_&lt;/code&gt;, &lt;code&gt;foo&lt;/code&gt;, or &lt;code&gt;foo_&lt;/code&gt;.  To determine what your compiler is doing, use the utility &lt;code&gt;nm&lt;/code&gt; on Linux/Unix.  Alternatively, both autotools and CMake have routines to determine the Fortran name mangling scheme.  Anyway, ARPACK is good, really good.  Just ask it for the smallest algebraic eigenvalue and specify some kind of tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, use LAPACK and the routine &lt;code&gt;syevr&lt;/code&gt;.  Make sure to specify that you only want one eigenvalue, set &lt;code&gt;RANGE&lt;/code&gt; to &lt;code&gt;I&lt;/code&gt; and set &lt;code&gt;IL&lt;/code&gt; and &lt;code&gt;IU&lt;/code&gt;.  Like ARPACK, LAPACK is in Fortran, so make sure to correctly deal with name mangling.  There will be fewer routines that you need to write a header for, namely only one whereas ARPACK will require more, so it's simpler to use.  That said, for this case, ARPACK will work better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, the paper that ARPACK is based on is called &quot;Implicit Application of Polynomial Filters in a k-Step Arnoldi Method&quot; by Dan Sorensen (&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0613025&quot; rel=&quot;noreferrer&quot;&gt;link&lt;/a&gt;).  It's a good paper and worth a read.  That said, really what you're doing is using the Arnoldi method to reduce your matrix to a tridiagonal form and then looking at the eigenvalues of this form.  The Arnoldi method really is just Gram-Schmidt applied to the Krylov subspace.  Basically, we get a relationship where $AV = VH + \textrm{stuff}$.  The structure of $H$ is&#xA;$$&#xA;H=\begin{bmatrix}&#xA;\langle \cdot,\cdot \rangle &amp;amp; \langle \cdot,\cdot \rangle &amp;amp; \dots &amp;amp; \langle \cdot,\cdot \rangle\\&#xA;\|\cdot\| &amp;amp; \langle \cdot,\cdot \rangle &amp;amp; \dots &amp;amp; \langle \cdot,\cdot \rangle\\&#xA;0 &amp;amp; \|\cdot\| &amp;amp; \dots &amp;amp; \langle \cdot,\cdot \rangle\\&#xA;\vdots &amp;amp; 0 &amp;amp; \ddots &amp;amp; \vdots\\&#xA;0 &amp;amp; 0 &amp;amp; \dots &amp;amp; \langle \cdot,\cdot \rangle&#xA;\end{bmatrix}&#xA;$$&#xA;Here, the $\langle \cdot,\cdot \rangle$ are the Gram-Schmidt coefficients and the $\|\cdot\|$ are the normalization of the Krylov vectors after normalization.  For a symmetric matrix, $H$ should be tridiagonal, but it really never will be due to round-off error.  Anyway, the eigenvalues of $H$ are called Ritz values and they approximate the eigenvalues of $A$.  If I recall correctly, the Ritz values will approach the eigenvalues of $A$ from the inside and find the outermost eigenvalues first.  How does this help you?  You can just write your own eigenvalue routine.  Basically, use the method from Sorensen's paper, but you can really skip the implicit QR iterations if you only want one eigenvalue.  Literally, what you'll be doing is coding the Arnoldi method and then finding the eigenvalues of $H$.  Restarting will be helpful and the paper explains that, but, again, you can probably skip the implicit QR iterations.  Once the outermost eigenvalues converge to a certain tolerance, pick the smallest one and that's your smallest eigenvalue to the tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, yeah, this is kind of a pain and I'm sorry about that.  Really, ARPACK is the right tool even though there will be some pain using it.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastEditorUserId="10094" LastEditDate="2017-03-02T16:29:31.437" LastActivityDate="2017-03-02T16:29:31.437" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="26224" PostTypeId="1" CreationDate="2017-02-20T07:06:19.583" Score="1" ViewCount="60" Body="&lt;p&gt;I have a function $\mu(x)$ that I need to convolve with several functions $\nu_n(x)$. I'm currently taking the DFT of $\mu$ and multiplying it individually with the N DFTs of the functions $\nu_n$ and then taking the inverse DFT to obtain the convolution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I instead define&#xA;$m(x, y) = \mu(x), n(x, y) = \nu_y(x)$ for discrete $y$, take the 2D DFT to obtain $M(u, v)$ and $N(u, v)$, which I then multiply element wise as $C(u, v) = M(u, v) N(u, v)$ and then take the inverse 2D DFT to obtain the same result $c(x, y)$ as I did elementwise in the above method?&lt;/p&gt;&#xA;" OwnerUserId="23337" LastActivityDate="2017-02-20T07:06:19.583" Title="Can convolution be generalized to 2D from several 1D convolutions?" Tags="|fourier-analysis|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26225" PostTypeId="2" ParentId="26218" CreationDate="2017-02-20T10:40:06.627" Score="5" Body="&lt;p&gt;Taking the average will certainly work, but is recommended to take the harmonic mean to account for the changing conductivity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k = \Big( \frac{1-f}{k_i} + \frac{f}{k_{i+1}} \Big)^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you choose your faces directly midway between your two cells, f = 0.5:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ k = \frac{2k_ik_{i+1}}{k_i + k_{i+1}} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be derived by looking at the physical flux, which you want to approximate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest, you take a look at the book by Patankar: Numercial Heat Transfer and Fluid Flow Chapter 3. It can be found in any decent science library and I guess, the internet counts as well.&lt;/p&gt;&#xA;" OwnerUserId="23063" LastActivityDate="2017-02-20T10:40:06.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26226" PostTypeId="2" ParentId="26161" CreationDate="2017-02-20T13:39:11.170" Score="0" Body="&lt;p&gt;You want to be an applied mathematician. This is a major available at many universities in their engineering school also as a PhD.  Though in my experience, mathematics is the more important skill, some technical background in formal computer science can be helpful and you may not get it elsewhere. Just be careful not to go to far down the CS rabbit hole or you may wind up doing CS research which tends to pull you away from practical applications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your specific question, an applied mathematician will do research in numerical methods and analysis and possible partner with scientific researchers to do scientific computing in several areas. If you want to pursue scientific computing specifically, you may want to enter a scientific research discipline such as computational chemistry, biophysics, climate, bioinformatics, etc.&lt;/p&gt;&#xA;" OwnerUserId="3450" LastActivityDate="2017-02-20T13:39:11.170" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26228" PostTypeId="2" ParentId="26222" CreationDate="2017-02-20T22:03:02.463" Score="2" Body="&lt;p&gt;What you're thinking about is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lax_equivalence_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lax Equivalence Theorem&lt;/a&gt;. It is a double implication (if and only if). You can prove generally using a Functional Analysis approach, though that might be more in depth and &quot;further from practical methods&quot; than an approach most classes would take, which is probably why only the forward implication was proved.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-02-20T22:03:02.463" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26229" PostTypeId="1" AcceptedAnswerId="26244" CreationDate="2017-02-20T23:09:06.643" Score="7" ViewCount="126" Body="&lt;p&gt;I'm writing a library that involves some approximations of variational calculus problems. Whenever I implement routines to evaluate the derivative or Hessian of an action functional $A$, I write a test to check that these are working correctly. To do this, I evaluate the error in using the first and second derivatives of the action to compute a local quadratic approximation to the objective functional:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E = A(u + \delta v) - \left(A(u) + \delta\cdot dA(u)\cdot v + \frac{1}{2}\delta^2(d^2A(u)v)\cdot v\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and check that $E/\delta^3$ doesn't grow for $\delta = 2^{-1},\ldots,2^{-N}$ for some big enough $N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works well for moderately sized $N$ (up to 16 or so), but for $N$ too large, truncation error begins to dominate and the error in the approximation gets worse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is a good way to test derivative approximations in the face of truncation error?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I see it, I have only bad options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write an initial version of the test that prints the approximation errors, find the $N$ for which truncation error takes over, then use this $N$ as the cutoff point in the final version of the unit test.&lt;/li&gt;&#xA;&lt;li&gt;Have some way of automatically diagnosing where truncation error takes over and discard all the results thereafter.&lt;/li&gt;&#xA;&lt;li&gt;Don't make my unit tests return failure unless it's really obvious. Graph the results and let the user decide.&lt;/li&gt;&#xA;&lt;li&gt;Determine analytically where truncation error is likely to occur.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I don't like option 1 because it feels like cherry-picking my tests so that they will pass. I don't like option 2 because a genuinely failing implementation's inherent mathematical errors might be erroneously assessed as truncation error, so that automated regression testing would miss the bug. At that rate, I'd have to check the results manually, in which case I might as well go with option 3. But I don't like option 3 because I want automated testing. I think option 4 is practical for functions of a single variable -- evaluating $\sqrt{\epsilon|f(x)/f''(x)|}$ where $\epsilon$ is the machine epsilon gives you a pretty good upper bound -- but for big PDE problems where the unknown is a field with thousands of variables, not so much.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-04-23T03:39:16.440" Title="testing derivative approximations" Tags="|testing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26230" PostTypeId="1" AcceptedAnswerId="26242" CreationDate="2017-02-21T00:09:06.750" Score="0" ViewCount="958" Body="&lt;p&gt;The &lt;strong&gt;question&lt;/strong&gt; is exactly as the title: &#xA;Which 2D (triangular) mesh generator software can be used which has a set of geometric primitives, controlled mesh size and standard output (.vtk or something similar)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For testing my code I need a &lt;strong&gt;triangular&lt;/strong&gt; mesh for a circle and I am looking for the easiest way to do that. E.g., in 3D case for a sphere I am happy to use NETGEN with several lines of code to get meshes with controlled mesh size and standard output format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give me a recommendation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="21958" LastActivityDate="2017-02-21T21:01:10.860" Title="2D mesh generator with geometric primitives" Tags="|software|mesh-generation|unstructured-mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26231" PostTypeId="1" CreationDate="2017-02-21T00:32:06.477" Score="1" ViewCount="150" Body="&lt;p&gt;I have two vectors which specify the bounds $x_{min}$ and $x_{max}$ of the sample space. Also, it has to satisfy the linear constraint $Ax \leq b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to generate an evenly spaced set of points, with some control over grid spacing, which satisfy these constraints and this includes both the boundaries and the interior? In other words, how do I generate a 'mesh' for a feasibility region? The feasibility problem in this case&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;\begin{aligned}&#xA;&amp;amp; \underset{x}{\text{minimize}}&#xA;&amp;amp; &amp;amp; 1 \\&#xA;&amp;amp; \text{subject to}&#xA;&amp;amp; &amp;amp; Ax \leq b \\&#xA;&amp;amp;&amp;amp;&amp;amp; x_{min} \leq x \leq x_{max}&#xA;\end{aligned}&#xA;\end{equation*}&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="15120" LastEditDate="2017-02-21T06:26:10.200" LastActivityDate="2017-02-21T06:26:10.200" Title="Generate discrete set of points in a feasible region" Tags="|convex-optimization|linear-programming|mesh-generation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26232" PostTypeId="2" ParentId="26222" CreationDate="2017-02-21T01:50:12.277" Score="0" Body="&lt;p&gt;In general, I agree with the previous answer by Chris Rackauckas. However, to be more precise, the answer depends on the definition of stability and convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Stetter &quot;Analysis of discretization methods for ordinary differential equations&quot;, in chapter 1, there are simple examples for schemes which are convergent, consistent but not stable or stable but not convergent. They are in a sense degenerate because a bad choice for the spaces is made.&lt;/p&gt;&#xA;" OwnerUserId="21958" LastActivityDate="2017-02-21T01:50:12.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26234" PostTypeId="1" CreationDate="2017-02-21T11:58:03.250" Score="2" ViewCount="2276" Body="&lt;p&gt;I have a optimization problem in which the optimal objective value occurs at multiple point in the feasible space. If I run my problem in LINGO software then it gives me the optimal objective value at a point in the feasible space but how to get the all points in which optimal solution occurs.&lt;/p&gt;&#xA;" OwnerUserId="23351" LastActivityDate="2017-02-23T00:28:55.690" Title="How to get multiple solutions for a optimization problem using any kind of software" Tags="|optimization|matlab|linear-solver|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26235" PostTypeId="2" ParentId="26234" CreationDate="2017-02-21T12:39:58.500" Score="3" Body="&lt;p&gt;In general this is difficult, however, usually you can get a decent solution by initializing at different locations and collecting the solutions. This can be quite time consuming (computationally), and suboptimal (piggybacking on an optimizer), but easy to try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To choose different locations, you can do either by (1) predefining a grid-like structure, or (2) sampling randomly from a distribution. Prior knowledge such as the smoothness of the objective function, or expected regions where optima are located will help you determine your strategy.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2017-02-21T12:39:58.500" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26237" PostTypeId="1" AcceptedAnswerId="26245" CreationDate="2017-02-21T14:52:40.670" Score="5" ViewCount="244" Body="&lt;p&gt;There are several functions of two or three variables that I am working with. For this question I have made a small set showing the resistivity, $\rho$, in n$\Omega$m, of copper as a function of its purity ($RRR$ value) and its temperature, $T$, in Kelvin.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RRR\Temp [K]    20    90    160    230    300&#xA;          60    0.27  3.1   8.15   12.9   17.5&#xA;         300    0.06  2.81  7.84   12.6   17.2&#xA;         500    0.04  2.78  7.81   12.5   17.2&#xA;        1000    0.02  2.76  7.79   12.5   17.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So this 'function' has the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho (T, RRR)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I do know how to make an interpolating function for two variables in my language of choice but I don't know how to change the variables, say to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$RRR (T, \rho)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be very beneficial as the purity of the sample is only known approximately while the temperature and resistivity are known to great accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen how to do this where an analytic representation is know by using a transform of independent variables. But when I thought about how I apply that to this to this problem I ran into a wall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;To clarify what I mean when I say that the purity is known approximately: a particular sample may claim that its RRR is 100. If I check a sample of 6 AWG (1.33e-5 $m^2$), length 0.1 m at 293 K the calculation says I should see a resistance of 127.07 $\mu\Omega$. There will be a difference from the measured value but I should be able to distinguish the source of error, i.e. incorrect purity, instrumentation error, etc... Taking a data point at several temperatures and converting resistance back in to resistivity will be a good plan but this would necessitate the use of a function of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$RRR(T, \rho)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to whether it is invertible, I think it is. I will always know the temperature. See contour plot below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/UFYIY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UFYIY.png&quot; alt=&quot;Resistivity of Copper by Temperature and Purity(RRR)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23353" LastEditorUserId="23353" LastEditDate="2017-02-22T12:19:48.640" LastActivityDate="2017-02-22T14:50:24.430" Title="Computing inverse functions of functions of two variables" Tags="|interpolation|inverse-problem|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26238" PostTypeId="2" ParentId="26199" CreationDate="2017-02-21T15:58:10.603" Score="2" Body="&lt;p&gt;The key issue in this discussion is explicit versus implicit ODE solvers.&#xA;The traditional Runge-Kutta (RK) methods are explicit and so the time step must&#xA;be small enough to satisfy the stability requirement. The backward difference&#xA;methods (BDF) are implicit so the time step can be chosen based, simply, on what&#xA;is needed for an accurate solution; typically much larger than for an&#xA;explicit method. I assume that is basically the point of your question&#xA;about RK versus BDF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The incompressible Navier-Stokes (NS) equations present a very interesting&#xA;challenge to both classes of methods. This is discussed in many books&#xA;on computational fluid dynamics. I found this particular survey, &lt;a href=&quot;https://pdfs.semanticscholar.org/082a/f81cd12213a1bbcbae2647c7297a29e40615.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Langtangen&lt;/a&gt;,&#xA;to be an especially nice introduction to the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, for low-Reynolds number flow, the viscous terms are important&#xA;which makes implicit methods attractive because these terms dictate a very&#xA;small time step in explicit solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the convective terms can be integrated with an explicit solver&#xA;and a time step that is much larger. Furthermore, these terms are nonlinear&#xA;which makes an implicit solver much more unattractive because a set of nonlinear&#xA;algebraic equations must be solved at every time step. This system includes&#xA;all the velocity degrees of freedom so can be quite computationally&#xA;costly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as discussed in, for example, the referenced paper, state of the art&#xA;algorithms for the incompressible NS equations often use an explicit&#xA;solver where it works best-- integrating the convective terms-- and an&#xA;implicit solver where it works best-- integrating the viscous terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally I would suggest discretizing the equations in space and&#xA;then using an off-the-shelf ODE solver. But since most standard ODE solvers&#xA;implement just an explicit &lt;strong&gt;or&lt;/strong&gt; an implicit algorithm, using one of these&#xA;would not be the optimal approach for this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instructional purposes, Benjamin Seibold, wrote a NS solver based on&#xA;these principles, &lt;a href=&quot;http://math.mit.edu/~gs/cse/codes/mit18086_navierstokes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seibold&lt;/a&gt;,&#xA;in, remarkably, a little over 100 lines of MATLAB code! It is well-worth the time studying and&#xA;experimenting with it.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-02-21T20:41:51.220" LastActivityDate="2017-02-21T20:41:51.220" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26239" PostTypeId="2" ParentId="19082" CreationDate="2017-02-21T16:07:46.070" Score="0" Body="&lt;p&gt;This is a short answer and mostly a workaround. This could hit you back again later on, but can provide you with a temporary solution.&#xA;You can try to relax the solution to your Newton-Raphson solver.&#xA;What I mean by this is do not take the full solution of your non-linear system, but just take a factor of the previous solution and of the new solution (say 0.1 new solution, 0.9 old solution), and use this intermediate solution in your other solver. This means that you would be relaxing your global system of equation (and is similar conceptually to the above answer of using a single Newton-Raphson iteration).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This usually leads to more robust convergence, but it can turn out to be extremely slow.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2017-02-21T16:07:46.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26240" PostTypeId="1" AcceptedAnswerId="26253" CreationDate="2017-02-21T18:46:51.977" Score="1" ViewCount="126" Body="&lt;p&gt;I am attempting to solve a nonlinear advection diffusion equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t} = \frac{\partial}{\partial x}(\frac{\partial u}{\partial x} + u^2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with Robin boundary conditions via the finite volume method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the WENO method to reconstruct the function $u$ and its derivative $u_x$ as well as the numerical flux $\hat{f}$ all from the average values $\bar{u}$ and so my discretization currently looks like: &#xA;$$ \frac{\bar{u}^{n+1} - \bar{u}^{n}}{h} = \hat{f}_{i+\frac{1}{2}} - \hat{f}_{i-\frac{1}{2}}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use an implicit time discretization, do I need to use Newton's method (or any other known method to solve nonlinear equations) to advance a time step?&lt;/p&gt;&#xA;" OwnerUserId="23363" LastEditorUserId="123" LastEditDate="2017-02-22T23:44:17.073" LastActivityDate="2017-02-22T23:48:35.633" Title="If I discretize a PDE in space with WENO and in time with an implicit method, do I need to solve a nonlinear algebraic system at each time step?" Tags="|pde|numerical-analysis|finite-volume|hyperbolic-pde|numerical-modelling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26241" PostTypeId="2" ParentId="26230" CreationDate="2017-02-21T19:19:39.660" Score="1" Body="&lt;p&gt;You can use Jonathan Shewchuk's &lt;em&gt;triangle&lt;/em&gt; mesh generator [1]. It has its own file format, but it can be easily translated. If you plan to embed it in another software, it can be also compiled as a library with an easy to use API.&#xA;You can also use CGAL [2], a large piece of software with many functionalities (including 2D mesh generation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.cmu.edu/~quake/triangle.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://www.cgal.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cgal.org&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2017-02-21T19:19:39.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26242" PostTypeId="2" ParentId="26230" CreationDate="2017-02-21T21:01:10.860" Score="2" Body="&lt;p&gt;Shewchuk's triangle mesh generator produces high quality meshes and is quite&#xA;robust. However, the boundary definition is a series of straight lines. So to&#xA;produce a sequence of refined meshes over a circle you would also have to refine&#xA;the definition of the boundary for each mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is Persson's &lt;a href=&quot;http://persson.berkeley.edu/distmesh/&quot; rel=&quot;nofollow noreferrer&quot;&gt;distmesh&lt;/a&gt; triangle mesher,&#xA;As shown on this page, only two lines of MATLAB (or Octave) code are required&#xA;to mesh a circle&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fd=@(p) sqrt(sum(p.^2,2))-1;&#xA;[p,t]=distmesh2d(fd,@huniform,0.2,[-1,-1;1,1],[]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the mesh can be refined by changing a single &quot;mesh density&quot; parameter&#xA;with no changes to the geometry definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can conveniently use MATLAB to write the coordinate matrix, &lt;code&gt;p&lt;/code&gt;, and the&#xA;connectivity matrix, &lt;code&gt;t&lt;/code&gt;, in whatever format you choose. And, as I've verified,&#xA;it also runs in Octave if you don't have access to MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-02-21T21:01:10.860" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26244" PostTypeId="2" ParentId="26229" CreationDate="2017-02-22T02:15:45.630" Score="2" Body="&lt;p&gt;Well, I can tell you what I do, but it's somewhat unsatisfactory.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Check $dA(u)$ using $A(u)$.  I prefer a fourth-order finite difference scheme, but I check $dA(u)\cdot \delta u$ against the approximation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;d A(u)\cdot \delta u \approx \frac{A(u - 2\epsilon \delta u)-8A(u - \epsilon \delta u)+8A(u +\epsilon \delta u)-A(u + 2\epsilon \delta u)}{12}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Once we know that $dA(u)$ is accurate, check $d^2A(u)\delta u$ versus the approximation&#xA;$$&#xA;d^2 A(u)\delta u \approx \frac{dA(u - 2\epsilon \delta u)-8dA(u - \epsilon \delta u)+8dA(u +\epsilon \delta u)-dA(u + 2\epsilon \delta u)}{12}&#xA;$$&#xA;This is not that different from above, except we now have a vector instead of a scalar, so the error is calculated with a norm and not an absolute value.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As you mentioned above, it's hard to know when there's a switch between truncation and round-off error.  As such, I run this for several values of $\epsilon$ and look for two things&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The minimum error in the approximation over all attempted $\epsilon$ reaches an acceptable level&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Typically, there's a curve where the error goes high-low-high as the value of $\epsilon$ is reduced&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In a unit test, I typically only use 1.  When validating a function by hand, I look for 2.  Technically, 1 could give an erroneous pass in case we get lucky and everything cancels.  I've never seen that happen, so I don't worry about it.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2017-02-22T02:15:45.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26245" PostTypeId="2" ParentId="26237" CreationDate="2017-02-22T09:31:12.903" Score="2" Body="&lt;p&gt;In principle, you should find another table, similar to the one you posted, this time with independent variables $T$, $\rho$, and dependent variable $RRR$. Then, you will find the $RRR$ function by interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to populate this table, for each point in the $(T, \rho)$-grid, say $(T_k, \rho_k)$, the corresponding value of $RRR$, say, $RRR_k$, will be numerically obtained as the root $RRR_k$ of the equation &#xA;$$&#xA;\rho_k = \rho(T_k, RRR_k),&#xA;$$&#xA;where $\rho(\cdot)$ is the interpolating function you obtained above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above procedure works if the map $\rho(T, RRR)$ is invertible, i.e., the above equation has a unique solution.&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2017-02-22T09:31:12.903" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26246" PostTypeId="2" ParentId="26237" CreationDate="2017-02-22T11:39:34.727" Score="1" Body="&lt;p&gt;The idea is to reverse your table so the variable $RRR$ will dependent by $T, \rho$, until here is similar to Stelios's answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now with this new table you can find $RRR(T, \rho)$, and you have different way. I suggest to use some different methods respect to the interpolation for a couple of reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;according with your question the values of $RRR$ are affect by error, so forcing the interpolation have got not much meaning&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;This reason depending to the type of interpolation use but generally:&lt;/em&gt; good result of interpolation is depending by the nodes distribution, it is involved the Lebesgue's constant.  By the table you have got a fixed set of nodes that you can not change, i.e. you have not much choice. Furthermore find good distribution of nodes for the multivariate case (your case id $2D$) that you can make of use it is not a trivial fact.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For this reason maybe you can think about an approximation approach (to clarify for example Least square). Always in the approximation environment there is &lt;a href=&quot;http://www.scholarpedia.org/article/Radial_basis_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;RBF&lt;/a&gt;, this method can works on scattered data (no problem about nodes distribution) &lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="22025" LastEditDate="2017-02-22T14:50:24.430" LastActivityDate="2017-02-22T14:50:24.430" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26248" PostTypeId="2" ParentId="26234" CreationDate="2017-02-22T16:35:01.583" Score="0" Body="&lt;p&gt;Depends on the type of problem you are dealing with. For non convex see the other answer about multistart. For convex problems, iteratively adding previous solutions as constraints will return alternative optima. Any convex combination of optimal solutions is optimal as well, so in something like an LP there might be variables that can take a range of values at the optimum. For problems with integral variables, again, iteratively add solutions as constraints (these are referred to as integer cuts)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding non convex problems there are methods that fully explore the solution space (spatial branch and bound). These may be capable of finding alternative optima in the rare (at least for problems that only have continuous variables) instance of them existing.&lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2017-02-22T16:35:01.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26249" PostTypeId="1" AcceptedAnswerId="26251" CreationDate="2017-02-22T17:11:20.097" Score="0" ViewCount="139" Body="&lt;p&gt;I have worked on mathematical modeling based on differential equations, and now I want to simulate a cellular automaton based on a &lt;code&gt;system of mixed ode-pde coupled first order differential equations&lt;/code&gt; &#xA;\begin{eqnarray}&#xA;\dfrac{dR}{dt}(t)&amp;amp;=&amp;amp;v_r(R,t),\\&#xA;\dfrac{\partial v_r}{\partial r}(r,t)&amp;amp;=&amp;amp;\dfrac{R\sinh(r)}{r\sinh(R)}[1-\zeta_1\sqrt{\sigma_r^2(r,t)+2(\sigma_r(r,t)-\beta(r,t))^2}]-\epsilon[1+\zeta_2\sqrt{\sigma_r^2(r,t)+2(\sigma_r(r,t)-\beta(r,t))^2}]-2\dfrac{v_r(r,t)}{r},\\&#xA;\dfrac{\partial\sigma_r}{\partial r}(r,t)&amp;amp;=&amp;amp;-\dfrac{2\beta}{r},\\&#xA;\gamma(r,t)&amp;amp;=&amp;amp;\left(\dfrac{\partial}{\partial t}+v_r(r,t)\dfrac{\partial}{\partial r}\right)\beta(r,t),\end{eqnarray}&#xA;where $\gamma$ is a given function depending also of $R,\;\sigma_r,\;v_r,\;\beta$ corresponding to a model with domain $\Omega\subset\mathbb{R}^2$ compact (and of course with the necessary initial and boundary conditions). But I am new in this and I have no idea of a programming platform or software (I'm not really sure how to say that) to achieve my goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone recommend a good program to simulate cellular automaton?&lt;/p&gt;&#xA;" OwnerUserId="23378" LastEditorUserId="23378" LastEditDate="2017-02-22T19:13:19.140" LastActivityDate="2017-02-22T19:13:19.140" Title="A program to simulate cellular automaton model" Tags="|pde|ode|simulation|reference-request|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26250" PostTypeId="2" ParentId="26249" CreationDate="2017-02-22T18:25:39.783" Score="1" Body="&lt;p&gt;I believe Mathematica has some functionality for basic automatons. See for instance this reference:&#xA;&lt;a href=&quot;http://reference.wolfram.com/language/tutorial/CellularAutomata.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://reference.wolfram.com/language/tutorial/CellularAutomata.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you wish to implement a complex cellular automaton, then you might have to program it yourself. If performance is not an issue, then a scripting language like Python can be ideal. If performance is an issue, than a C++/C/Fortran approach can extract the best performance out of your machine. However, this will all require certain programming skill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the PDE you intend to solve with your cellular automaton?&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2017-02-22T18:25:39.783" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26251" PostTypeId="2" ParentId="26249" CreationDate="2017-02-22T18:37:02.170" Score="1" Body="&lt;p&gt;As @BlaisB mentioned, Mathematica has quite a bit of stuff built-in to do cellular automata. You might want to check that out if you have a license. I find Mathematica to be very nice when something has library functions you can use, but if you can't solve the problem by easily stringing together library functions then I find Mathematica much tougher to use than other languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you will need good performance, the ideal is to use Julia, C/C++, or FORTRAN. These days I would usually recommend Julia, but other scripting languages which can do this are Python, MATLAB, and even R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what language you use really doesn't matter all that much (except for performance. There is a difference between the languages in terms of performance which might matter if your PDEs are difficult to solve).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-02-22T18:37:02.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26252" PostTypeId="2" ParentId="20242" CreationDate="2017-02-22T19:18:54.370" Score="1" Body="&lt;p&gt;I will answer your points 1 (still trying to find an answer for 2) since these seem to be the key of your question&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1)&lt;/strong&gt; This depends. If your analytical solution is a steady solution, then when you reach that solution, your time derivative term should be close to zero (or close to the machine error at least). Thus keeping a constant CFL condition is not necessary. Since your scheme is implicit and unconditionally stable, you do not need to worry about the influence of the time step of the error.&#xA;For some solution we investigate, you will sometimes have an analytical solution that depends on time. In this case, your error, if you have a second order time discretization, will be:&#xA;$$ e \propto O(\Delta x^2) + O(\Delta t^2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus if you keep the CFL constant&#xA;$$ e \propto O(\Delta x^2) + O (CFL \Delta x^2) \propto O(\Delta x^2) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, you only need to keep the CFL constant for unsteady analytical solution, not for steady one. An example is the Taylor-Green vortex...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2)&lt;/strong&gt; Regretfully, I am not sure I understand the question exactly at the moment. Let me think about it and I will try to edit it later...&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2017-02-23T15:19:18.947" LastActivityDate="2017-02-23T15:19:18.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26253" PostTypeId="2" ParentId="26240" CreationDate="2017-02-22T23:48:35.633" Score="2" Body="&lt;p&gt;Generically, yes.  Since you have a nonlinear PDE you will end up with a nonlinear algebraic system no matter what spatial discretization you use.  With WENO you will have a more strongly nonlinear system and (at high Reynolds number) it will be hard to find a convergent solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common approach for this kind of equation is to use an IMplicit-EXplicit (IMEX) additive Runge-Kutta method, in which the explicit part is applied to the nonlinear convection term (avoiding the need to solve a nonlinear algebraic system) and the implicit part is applied to the linear diffusion term (avoiding the need for an excessively small timestep).&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-02-22T23:48:35.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26254" PostTypeId="2" ParentId="26234" CreationDate="2017-02-23T00:28:55.690" Score="2" Body="&lt;p&gt;Your comment on the question indicates that you're in the special case of a binary integer linear programming problem.  For these problems, a standard approach is to find an optimal solution, add a constraint to eliminate that particular solution, and then reoptimize to find another optimal solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if your first optimal solution has binary variables with values $x_{1}=1$, $x_{2}=0$, $x_{3}=1$, then you can add the constraint&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(1-x_{1}) + x_{2} + (1-x_{3}) \geq 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to eliminate the solution $x_{1}=1$, $x_{2}=0$, $x_{3}=1$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-02-23T00:28:55.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26257" PostTypeId="1" AcceptedAnswerId="26366" CreationDate="2017-02-23T12:03:47.117" Score="3" ViewCount="331" Body="&lt;p&gt;Can somebody point me to software/code to extract a solenoidal (a.k.a. divergence-free) field from a 2D vector field numerically?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a plethora of papers and documents describing how to do this, but for some reason none of the authors (or anybody else for that matter) puts a simple piece of source code online implementing that functionality. All I found are rather huge packages or very cryptic pieces of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update. More concretely for my situation:&#xA;I have two 2D arrays of size 64x64 with double-precision floating point numbers. These two arrays represent the x- and y-velocities of a fluid flow. Periodic boundaries are employed. I would like to extract the solenoidal part of the velocities into two new 2D arrays.&lt;/p&gt;&#xA;" OwnerUserId="23386" LastEditorUserId="23386" LastEditDate="2017-02-24T19:16:16.407" LastActivityDate="2017-03-10T19:36:10.047" Title="Software/code to extract a solenoidal (a.k.a. divergence-free) field from a 2D vector field numerically" Tags="|numerical-analysis|fluid-dynamics|software|vector|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="26258" PostTypeId="1" AcceptedAnswerId="26260" CreationDate="2017-02-23T14:06:28.760" Score="28" ViewCount="50265" Body="&lt;p&gt;Right now I stuck with a problem. It seems to be really trivial one, but still it is hard for me to find an appropriate solution. The problem is:&#xA;One has two intervals and are to find the intersection of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Intersection of &lt;strong&gt;[0, 3]&amp;amp;[2, 4]&lt;/strong&gt; is &lt;em&gt;[2, 3]&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Intersection of &lt;strong&gt;[-1, 34]&amp;amp;[0, 4]&lt;/strong&gt; is &lt;em&gt;[0, 4]&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Intersection of &lt;strong&gt;[0, 3]&amp;amp;[4, 4]&lt;/strong&gt; is &lt;em&gt;empty set&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is pretty clear that the problem can be solved by using tests of all possible cases, but it will take a lot of time and is very prone to mistakes. Are there any easier way to tackle the problem? If you know the solution help me, please. Will be very grateful.&lt;/p&gt;&#xA;" OwnerUserId="23387" LastActivityDate="2022-09-16T15:47:18.327" Title="The easiest way to find intersection of two intervals" Tags="|algorithms|interval-arithmetic|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26259" PostTypeId="1" AcceptedAnswerId="26263" CreationDate="2017-02-23T14:54:58.717" Score="0" ViewCount="637" Body="&lt;p&gt;I have a MATLAB code (see below) that employs 'fsolve' from the optimization toolbox for a root finding problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottleneck is that, within the objective function calculation, there is a computationally expensive function (denoted as 'expensive_function' in the below code, and is located in the matlabpath). The variable 'x_sol' holds solution returned by 'fsolve'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that, within the main code, I need access to the 'other_results' struct computed by the expensive function for further calculations. It is wasteful to recompute the 'other_results' struct using the solution value (such as by using the following line of code within 'main' )&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;other_results = expensive_function(x_sol,param);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a way to get access to this struct without re-evaluating the expensive_function? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I have been cautioned throughout my career against using global/persistent variables, and I have thus far stayed clear of them. I don't know if this is the only recourse to solving this issue, or there exists a better/clever way ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% main code&#xA;&#xA;x_sol = fsolve(@(x) compute_residual(x,param) , x0);&#xA;&#xA;% code below needs to use various values from the &quot;other_results&quot; struct computed in the obj_fun  function&#xA;&#xA;line with some computation on 'other_results' result&#xA;line with some other computation using 'other_results' struct&#xA;....&#xA;...&#xA;and so on.&#xA;&#xA;&#xA;________________________________________________________&#xA;&#xA;function residual = compute_residual(x, param)&#xA;     other_results = expensive_function(x,param);  % other_results is a struct returned by a very computationally expensive function&#xA;     residual = abs(other_results.a * other_results.b - x)   % the abs(diff) &#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2017-02-23T21:07:39.857" Title="Obtaining extra output argument(s) from the objective function used by fsolve in MATLAB" Tags="|optimization|matlab|nonlinear-equations|iterative-method|roots|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26260" PostTypeId="2" ParentId="26258" CreationDate="2017-02-23T15:02:32.613" Score="49" Body="&lt;p&gt;We can define a solution to this problem in the following way. Assume the input intervals can be defined as $I_{a} = [a_s, a_e]$ and $I_{b} = [b_s, b_e]$, while the output interval is defined as $I_{o} = [o_s, o_e]$. We can find the intersection $I_{o} = I_{a} \bigcap I_{b}$ doing the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;if&lt;/strong&gt; ( $b_s \gt a_e$ or $a_s \gt b_e$ ) {&#xA;&lt;strong&gt;return&lt;/strong&gt; $\emptyset$ }&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;else&lt;/strong&gt; {&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$o_s = \max (a_s,b_s)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$o_e = \min (a_e,b_e)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;return&lt;/strong&gt; $[o_s,o_e]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2017-02-23T20:39:31.687" LastActivityDate="2017-02-23T20:39:31.687" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26261" PostTypeId="1" CreationDate="2017-02-23T15:33:36.133" Score="1" ViewCount="147" Body="&lt;p&gt;Suppose I have a energy functional E depending on X, where X is a N-dimensional real value vector and N could be very large (~=2000). I assume that there exists (at least) a (local) minimizer for E. There are several problems I am dealing with now:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Firstly, the functional E is extremely 'complicated', it is nearly impossible to compute its exact gradient and its Hessian as well. So the 'input' data are always approximated. (approximated Gradient by finite difference formula, approximate Hessian, ...) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Secondly, N is very large: N ~= 2000. So I guess this can be called a Large Scaled Problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Thirdly, in my code, I need to repeat this optimization problem many times (approx. 100 iterations), so a fast and efficient optimization solver is very necessary.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there any good algorithm that can satisfies all the requirement above?&lt;/p&gt;&#xA;" OwnerUserId="23389" LastEditorUserId="9667" LastEditDate="2017-02-23T16:00:27.927" LastActivityDate="2018-04-24T16:48:34.020" Title="Numerical optimization algorithm with approximated derivatives" Tags="|optimization|numerics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26262" PostTypeId="2" ParentId="24179" CreationDate="2017-02-23T19:29:01.353" Score="3" Body="&lt;p&gt;Surprisingly, &lt;a href=&quot;https://en.wikipedia.org/wiki/Lloyd&amp;#39;s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lloyd smoothing&lt;/a&gt; hasn't come up here yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/S0036144599352836?journalCode=siread&quot; rel=&quot;nofollow noreferrer&quot;&gt;Du, Qiang; Faber, Vance; Gunzburger, Max (1999), &quot;Centroidal Voronoi tessellations: applications and algorithms&quot;, SIAM Review, 41 (4): 637–676&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(and perhaps &lt;a href=&quot;https://github.com/nschloe/voropy&quot; rel=&quot;nofollow noreferrer&quot;&gt;voropy&lt;/a&gt;, a small project of mine, if you're interested to see Lloyd smoothing in action).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hd2W1.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hd2W1.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2017-02-27T11:40:44.343" LastActivityDate="2017-02-27T11:40:44.343" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26263" PostTypeId="2" ParentId="26259" CreationDate="2017-02-23T21:07:39.857" Score="1" Body="&lt;p&gt;As I said in my comment, I don't think you can necessarily count on the value of&#xA;&lt;code&gt;x&lt;/code&gt; in the last call to your function being the value that is returned as the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, that being said, here is a way to do what you want. The solution in the code I show below relies on two components. First you define a MATLAB class that derives from &lt;code&gt;handle&lt;/code&gt;. This makes it a so-called handle class so that when it is passed to a function as an argument, it is passed by reference, and therefore its contents can be modified by that function. Second, an anonymous function definition is used to pass an instance of this class into your &lt;code&gt;fsolve&lt;/code&gt; function so you can store your solution there. The example code below shows this explicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The class definition is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;classdef SolutionSaver &amp;lt; handle&#xA;  properties&#xA;    x, expensiveResult;&#xA;  end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the test case that uses this class is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function solutionSaveTest&#xA;   ss = SolutionSaver;&#xA;   func = @(x) calcF(x, ss);&#xA;   x = fsolve(func, 1);&#xA;   fprintf('x=%18.10e, ss.x=%18.10e\n', x, ss.x);&#xA;   ss.expensiveResult&#xA;end&#xA;&#xA;function f=calcF(x, ss)&#xA;  e=expensive_calc(x);&#xA;  f = e - 5;&#xA;  ss.x = x;&#xA;  ss.expensiveResult = e;&#xA;end&#xA;&#xA;function e=expensive_calc(x)&#xA;  e = x^2;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I ran this example in Octave and, in this case, the saved &lt;code&gt;x&lt;/code&gt; &lt;strong&gt;was&lt;/strong&gt; equal to the returned &lt;code&gt;x&lt;/code&gt;. But, in general, it is probably worth testing that fact before using the saved solution data.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-02-23T21:07:39.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26264" PostTypeId="1" CreationDate="2017-02-24T03:45:10.147" Score="6" ViewCount="389" Body="&lt;p&gt;I want to numerically solve the damped oscillator equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2x}{dt^2}+\eta\frac{dx}{dt}+x=0 .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually it's very easy to solve this kind of ODE analytically and numerically. But now the damping coefficient is extremely small, $\eta=10^{-10}$. To get the obvious damping pattern, the value of oscillating time $t$ should be bigger than $10^{10}$. I use Mathematica to solve it.  It can solve from $t=0$ to $t=10^6$ easily, but it's still far from what I want to get. Calculating for a large domain may be just impossible in principle. (I think this difficulty is universal for other languages and softwares like Fortran, C, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to rescale the domain, like set $t=e^\tau$ and then modify the ODE correspondingly, so the domain becomes $\tau=[0,10]$ rather than $t=[0,10^{10}]$. But this method fails. It turns out that it will cost the same amount of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have some good ideas or similar experience to handle this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: Actually the ODE I want to calculate is more complicated than this damped oscillator, and it cannot be solved analytically, so I have to numerically solve it. But these two ODEs have the same structure. &lt;/p&gt;&#xA;" OwnerUserId="23397" LastEditorUserId="9667" LastEditDate="2017-02-24T04:51:50.193" LastActivityDate="2017-03-03T22:09:02.203" Title="ODE $x''(t)+\eta x'(t)+x(t)=0$ with the $\eta$ extremely small" Tags="|finite-difference|ode|numerics|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26265" PostTypeId="1" AcceptedAnswerId="26299" CreationDate="2017-02-24T06:06:42.723" Score="0" ViewCount="269" Body="&lt;p&gt;As the title says, I'm trying to make my code include a uniform pressure/force to the surface of a plate for analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Structural/Element Modeling&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am modeling a 2D 9 node Mindlin plate that has a clamped boundary condition on one side. It is 0.2 [m] by 0.2 [m]. Currently, I am using 6x6 elements. The plate has 6 layers, consisting of, in order from top to bottom,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0.1  [mm] thick piezoelectric layer at a 0 degree angle&lt;/li&gt;&#xA;&lt;li&gt;0.25 [mm] thick composite material layer at a -45 degree angle&lt;/li&gt;&#xA;&lt;li&gt;0.25 [mm] thick composite material layer at a  45 degree angle&lt;/li&gt;&#xA;&lt;li&gt;0.25 [mm] thick composite material layer at a  45 degree angle&lt;/li&gt;&#xA;&lt;li&gt;0.25 [mm] thick composite material layer at a -45 degree angle&lt;/li&gt;&#xA;&lt;li&gt;0.1  [mm] thick piezoelectric layer at a 0 degree angle&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as calculating the electric load, I am 100% sure that it is correct. Same with calculating the various effects of the different orientations of the plates. The code works perfectly fine when I only include the electrical load, so that's not the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some tinkering and some astute remarks, I have realized that the issue is not with the construction of the Force Matrix, but rather, the way I implemented the uniform load. I had originally just made a Force Matrix and multiplied the entire thing with the load I wanted to apply. This was a fundamental flaw. So I started my studies again and found that the force matrix must be calculated as shown in the image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D0WuX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D0WuX.png&quot; alt=&quot;Force Vector equation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since in my case, the fz was a constant, I integrated the nine-node shape functions w.r.t to the area and got the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;H1 =  (1/4) * (1 - xi) * (1 - eta) * xi * eta --&amp;gt; 1/9&#xA;H2 = -(1/4) * (1 + xi) * (1 - eta) * xi * eta --&amp;gt; 1/9&#xA;H3 =  (1/4) * (1 + xi) * (1 + eta) * xi * eta --&amp;gt; 1/9&#xA;H4 = -(1/4) * (1 - xi) * (1 + eta) * xi * eta --&amp;gt; 1/9&#xA;H5 = -(1/2) * (1 - xi^2) * (1 - eta) * eta    --&amp;gt; 4/9&#xA;H6 =  (1/2) * (1 + xi) * (1 - eta^2) * xi     --&amp;gt; 4/9&#xA;H7 =  (1/2) * (1 - xi^2) * (1 + eta) * eta    --&amp;gt; 4/9&#xA;H8 = -(1/2) * (1 - xi) * (1 - eta^2) * xi     --&amp;gt; 4/9&#xA;H9 =  (1 - xi^2) * (1 - eta^2)                --&amp;gt; 16/9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Code I Implemented&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, instead of making my Force Vector with the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fef = [ H1 H2 H3 H4 H5 H6 H7 H8 H9 ] * P;&#xA;&#xA;for i = 1:length(nodes)&#xA;&#xA;    Fe(i,1) = 0;&#xA;    Fe(i+num_node_ele,1) = 0;&#xA;    Fe(i+num_node_ele*2,1) = 0;&#xA;    Fe(i+num_node_ele*3,1) = 0;&#xA;    Fe(i+(num_node_ele*4),1) = fef(i);&#xA;&#xA;end&#xA;&#xA;f = f + Fe * (detjacob);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I changed it by making P into a matrix like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = 1/9;&#xA;b = 4/9;&#xA;c = 16/9;&#xA;&#xA;dist_load = P * [ a a a a b b b b c ];&#xA;&#xA;fef = H .* dist_load;&#xA;% the rest is identical&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Doing so gives me deflections of reasonable ranges and the deflection converges as I increase the number of elements. So now, I must ask, did I do this correctly? Is there something I missed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help!&lt;/p&gt;&#xA;" OwnerUserId="22445" LastEditorUserId="22445" LastEditDate="2017-03-02T04:42:33.557" LastActivityDate="2017-03-02T15:10:22.393" Title="How to Apply a Uniform Load to a Laminate Plate in FEM" Tags="|finite-element|matlab|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="26266" PostTypeId="2" ParentId="24239" CreationDate="2017-02-24T10:36:22.513" Score="1" Body="&lt;p&gt;The algorithm described in your scan looks like it could be implemented using a library that can do the two things: 1) solution of a linear system (in particular, steps (3) and (5)) and 2) matrix-vector products (steps (4), (6) and (7)). This library could be, e.g., SciPy, cholmod, SuperLU, etc. depending on the properties of the matrix $K$. &lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2017-02-24T10:36:22.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26267" PostTypeId="2" ParentId="26264" CreationDate="2017-02-24T16:35:18.430" Score="8" Body="&lt;p&gt;The problem you have is what is called &quot;stiff&quot;: it has two time scales, namely one for the oscillation (which is ${\cal O}(1)$) and one for the damping (which is ${\cal O}(\eta^{-1})$). If $\eta$ is very small, then these time scales are very different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stiff ODEs are difficult to solve because in order to solve them accurately, you have to resolve the short time scale with the time step, which yields very large number of time steps to resolve the long time scale. You can, however, use an implicit solver with long time steps (sufficient to resolve the long time scale) and get a qualitatively correct answer (although it will no longer show the oscillatory behavior).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a large literature on stiff ODEs and solution approaches. You will find much information if you look for it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-02-24T16:35:18.430" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26269" PostTypeId="1" CreationDate="2017-02-25T10:07:37.027" Score="8" ViewCount="641" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;I've asked &lt;a href=&quot;https://mathoverflow.net/questions/263008/time-discretization-of-the-variational-formulation-of-the-navier-stokes-equation&quot;&gt;this question on mathoverflow&lt;/a&gt; too&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$T&amp;gt;0$&lt;/li&gt;&#xA;&lt;li&gt;$I:=(0,T]$&lt;/li&gt;&#xA;&lt;li&gt;$d\in\mathbb N$&lt;/li&gt;&#xA;&lt;li&gt;$\Lambda\subseteq\mathbb R^d$ be nonempty and open, $$\mathcal V:=\left\{\phi\in C_c^\infty(\Lambda,\mathbb R^d):\nabla\cdot\phi=0\right\}$$ and $$V:=\overline{\mathcal V}^{\left\|\;\cdot\;\right\|_{H^1(\Lambda,\:\mathbb R^d)}}\;,\;\;\;H:=\overline{\mathcal V}^{\left\|\;\cdot\;\right\|_{L^2(\Lambda,\:\mathbb R^d)}}$$&lt;/li&gt;&#xA;&lt;li&gt;$\operatorname P_H$ denote the orthogonal projection from&#xA;$L^2(\Lambda,\mathbb R^d)$ onto $H$&lt;/li&gt;&#xA;&lt;li&gt;$A_0u:=-\Delta u$ for $u\in\mathcal D(A_0):=H_0^1(\Lambda,\mathbb R^d)\cap H^2(\Lambda,\mathbb R^d)$, $$Au:=\operatorname P_HA_0u\;\;\;\text{for }u\in\mathcal D(A):=\mathcal D(A_0)\cap V$$ and $$B(u,v):=(u\cdot\nabla)v\;\;\;\text{for }u\in L^2(\Lambda,\mathbb R^d)\text{ and }v\in H^1(\Lambda,\mathbb R^d)$$&lt;/li&gt;&#xA;&lt;li&gt;$f:I\to H$&lt;/li&gt;&#xA;&lt;li&gt;$u\in L^2(I,\mathcal D(A))$ with $u'\in L^2(I,H)$ and $$u'(t)+A_0u(t)+B(u(t),u(t))+\nabla p(t)=f(t)\;\;\;\text{for all }t\in I\tag1$$ for some $p:I\to H^1(\Lambda)$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Assuming that $\Lambda$ is sufficiently regular such that $(1)$ is well-defined, it can be shown that $(1)$ is equivalent to $$u'(t)+Au(t)+\operatorname P_HB(u(t),u(t))=f(t)\;\;\;\text{for all }t\in I\;.\tag2$$ I want to solve $(2)$ numerically and I'm only interested in $u$ (and not in $p$).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I know that there are many references for the numerical study of $(1)$. However, it seems to me that all the considered schemes don't use $(2)$. They only use $(2)$ for theoretical results like existence and uniqueness of solutions. Maybe I'm wrong and I just don't see that these schemes use $(2)$.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In any case, my question is: Are we able to provide a numerical scheme which solves $(2)$ directly?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Or is there something which prevents us from doing that? My idea is to apply, for example, a semi-implicit Oseen discretization in time, i.e. consider $$\frac{u(t_n)-u(t_{n-1})}h+Au(t_n)+\operatorname P_HB(u(t_{n-1}),u(t_n))=f(t_n)\;\;\;\text{for all }n\in\left\{1,\ldots,N\right\}\tag3$$ with $$t_n:=nh\;\;\;\text{for }n\in\left\{0,\ldots,N\right\}$$ and $h:=T/N$ for some $N\in\mathbb N$. After that for each $n\in\left\{1,\ldots,N\right\}$ $(3)$ should be solvable by a finite element method (or is there some problem that I don't see?).&lt;/p&gt;&#xA;" OwnerUserId="23411" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2020-04-15T15:28:14.090" Title="Time discretization of the variational formulation of the Navier-Stokes equation" Tags="|finite-element|pde|numerical-analysis|fluid-dynamics|navier-stokes|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26270" PostTypeId="2" ParentId="26261" CreationDate="2017-02-26T14:55:58.357" Score="3" Body="&lt;p&gt;Consider &lt;a href=&quot;//en.wikipedia.org/wiki/Simulated_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;simulated annealing&lt;/strong&gt;&lt;/a&gt;  or similar algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very briefly, such algorithms perform a random walk in your $N$-dimensional space, but whether a new step is taken depends on the energy lost¹ by this step:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the step decreases $E$, it is made.&lt;/li&gt;&#xA;&lt;li&gt;If the step increases $E$, the step is made with a probability depending on the energy lost by this step. This probability is decreased over time (that’s the annealing).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The advantages are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You do not need to compute the derivatives at all (unless you opt for a mixed technique).&lt;/li&gt;&#xA;&lt;li&gt;You can also find global minima.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The downsides are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You need to evaluate $E$ a lot. &lt;/li&gt;&#xA;&lt;li&gt;It’s a Monte Carlo algorithm, so it may not always find the answer you desire (in particular if you have to cut down on the computation).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Whether this is actually the best solution for you is obviously something only you can decide as it depends on a lot of factors.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;¹ or gained, depending on your perspective and signs&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2017-02-26T14:55:58.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26272" PostTypeId="2" ParentId="2669" CreationDate="2017-02-26T16:18:01.393" Score="2" Body="&lt;p&gt;I would recommend using &lt;a href=&quot;https://jump.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;JuMP&lt;/a&gt;. You can see &lt;a href=&quot;https://jump.readthedocs.io/en/latest/refexpr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; that expressions and constraints can be written directly in terms of summations. The objective function can be written in terms of sums using standard Julia (i.e. use the &lt;code&gt;sum&lt;/code&gt; function on arrays and things like that which make sense and is easily readable). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what's even better than the fact that you can easily write the objective and constraints directly in the &lt;code&gt;sum&lt;/code&gt; notation, but you can then use that model to solve the equation &lt;a href=&quot;https://jump.readthedocs.io/en/latest/installation.html#getting-solvers&quot; rel=&quot;nofollow noreferrer&quot;&gt;using a ton of other packages, standard commercial and open source solvers&lt;/a&gt;, all directly from within JuMP. This list right here is a list of the most widely used LP solvers, and you can just test them all in one go without changing your syntax using JuMP.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-02-26T16:18:01.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26275" PostTypeId="2" ParentId="26269" CreationDate="2017-02-27T07:53:18.740" Score="0" Body="&lt;p&gt;The &lt;em&gt;something&lt;/em&gt; that prevents me from using such a scheme with projections is the numerical realization of the projector $P_H\colon L^2 \to H$.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't know of a formulation that is better accessible to numerical algorithms than expression of the projection via&#xA;$$&#xA;v_0 = P_Hv \leftrightarrow &#xA;\begin{bmatrix}I &amp;amp; \nabla \\ div &amp;amp; 0\end{bmatrix}&#xA;\begin{bmatrix}v_0 \\ *\end{bmatrix} =&#xA;\begin{bmatrix}v \\ 0\end{bmatrix},&#xA;$$&#xA;where the $*$ is a dummy variable,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;which I would approximate by means of &lt;em&gt;mixed finite elements&lt;/em&gt;,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;which (probably) would realize $(3)$ as a standard &lt;em&gt;mixed FEM&lt;/em&gt; of the Navier-Stokes in the $(v,p)$-formulation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So... &lt;strong&gt;If you have a reasonable numerical realization of $P_H$, there is nothing wrong with the scheme sketched in $(3)$.&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2017-02-27T07:53:18.740" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="26277" PostTypeId="1" CreationDate="2017-02-27T16:50:17.143" Score="6" ViewCount="280" Body="&lt;p&gt;I am working on some medium to large scale finite element codes. By using established and available tools I am able to have an algorithm that scales well up to about 10,000 cores. Investigating scaling beyond this requires investigating larger meshes, which leads to my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once meshes start getting large (in range of 100s of gigabytes to terabyte ranges), simply getting them to a cluster environment can completely overwhelm any cost of solving the resulting system. 100GB-1TB mesh sizes aren't especially large sizes by today's standard of nodes as well, which can have upwards of 64GB of memory each (and in many cases more than that)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SO how is this commonly handled? Are there common ways to improve the bandwidth in transferring data to a cluster? Do you just need to be on an incredibly high bandwidth connection, or physically ship a drive containing all the data you want?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a followup question: if I could re-engineer this, would it solve the problem to rely more on automatic mesh refinement so that we start with a smaller starting mesh and refine at need in-memory?&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2017-02-28T09:53:22.590" Title="Working with large mesh files" Tags="|finite-element|hpc|unstructured-mesh|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26279" PostTypeId="1" CreationDate="2017-02-27T19:44:30.707" Score="3" ViewCount="190" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I've asked &lt;a href=&quot;https://math.stackexchange.com/questions/2164214/numerical-computation-of-the-velocity-in-the-steady-navier-stokes-equation&quot;&gt;this question on Math.SE&lt;/a&gt; too.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$d\in\left\{1,\ldots,4\right\}$&lt;/li&gt;&#xA;&lt;li&gt;$\Lambda\subseteq\mathbb R^d$ be bounded, nonempty and open and $\partial\Lambda$ be Lipschitz&lt;/li&gt;&#xA;&lt;li&gt;$V:=\left\{u\in H_0^1(\Lambda,\mathbb R^d):\nabla\cdot u=0\right\}$&lt;/li&gt;&#xA;&lt;li&gt;$W:=\left\{p\in L^2(\Lambda):\int_\Lambda p=0\right\}$&lt;/li&gt;&#xA;&lt;li&gt;$\mathfrak a(u,v):=\sum_{i=1}^d\langle\nabla u_i,\nabla v_i\rangle_{L^2}$ for $u,v\in H^1(\Lambda,\mathbb R^d)$&lt;/li&gt;&#xA;&lt;li&gt;$\mathfrak b(p,v):=\langle p,\nabla\cdot v\rangle_{L^2}$ for $p\in L^2(\Lambda)$ and $v\in H^1(\Lambda,\mathbb R^d)$&lt;/li&gt;&#xA;&lt;li&gt;$\mathfrak c(u,v,w):=\langle((u\cdot\nabla)v,w\rangle_{L^2}$ for $u,v,w\in H^1(\Lambda,\mathbb R^d)$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The usually studied variational formulation of the steady Navier-Stokes equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}\left\{&#xA;\begin{split}&#xA;\mathfrak a(u,v)+\mathfrak b(p,v)+\mathfrak c(u,u,v)&amp;amp;=0\;\;\;\text{for all }v\in H_0^1(\Lambda,\mathbb R^d)\\&#xA;\mathfrak b(u,q)&amp;amp;=0\;\;\;\text{for all }q\in W&#xA;\end{split}\tag1\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $(u,p)\in H_0^1(\Lambda,\mathbb R^d)\times W$ is the searched solution.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to solve $(1)$ numerically. Usually, $(1)$ is linearized in some way and then a mixed finite element method is used to approximate a solution.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;However, I'm not interested in $p$. Now, $$\mathfrak a(u,v)+\mathfrak c(u,u,v)=0\;\;\;\text{for all }v\in V\tag2$$ (where $u\in V$ is the searched solution) is a variational formulation of the steady Navier-Stokes equation which is equivalent to $(1)$ and which doesn't contain $p$.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;I wonder if it might be better for me to find a numeric scheme which solves $(2)$. A linearization, e.g. an Oseen iteration, is possible for $(2)$ too. I think the crucial point is the choice of a (conforming) finite element.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, the question if I can benefit from the fact that I'm not interested in $p$ and hence don't need to care about its approximation. I've read many papers, but I couldn't find any which tries to solve the steady Navier-Stokes equation for the velocity only.&lt;/p&gt;&#xA;" OwnerUserId="23411" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2017-02-28T03:05:12.913" Title="Numerical computation of the velocity in the steady Navier-Stokes equation" Tags="|finite-element|pde|numerical-analysis|fluid-dynamics|variational-calculus|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="26280" PostTypeId="2" ParentId="26277" CreationDate="2017-02-27T20:42:16.540" Score="4" Body="&lt;p&gt;My answer is primarily opinion-based, given my experience. In my work, I haven't (yet) dealt with meshes quite as large as what you're describing. However, I've seen large enough meshes to hint that your problems might be my problems in the future, so I've pondered the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some comments/suggestions ordered from obvious/easy-to-implement to more involved.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Make sure you're using binary mesh files, not ASCII. Depending on exact implementation, ASCII files can be significantly larger and also take significantly more time to read, due to the cost of string parsing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Parallel mesh file formats such as &lt;a href=&quot;https://trac.mcs.anl.gov/projects/parallel-netcdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parallel NetCDF&lt;/a&gt; can help spread storage and I/O of the mesh on separate compute nodes. Of course, that doesn't help you get the data &lt;em&gt;to&lt;/em&gt; the cluster.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could transfer a smaller initial mesh and do some command line/script-based refinement on the remote cluster. Of course, you still need to generate a single huge mesh on the cluster, whether that mesh is stored on disk or in memory, as I know of no parallel mesh refinement tools.&#xA;(&lt;strong&gt;Edit:&lt;/strong&gt; @VorKir brought my attention to &lt;a href=&quot;http://mfem.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mfem&lt;/a&gt;, which appears to include a very nice parallel meshing library made by the good people at &lt;a href=&quot;https://www.llnl.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lawrence Livermore National Laboratory&lt;/a&gt;.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In my opinion, &lt;a href=&quot;https://en.wikipedia.org/wiki/Domain_decomposition_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;Domain Decomposition Methods&lt;/a&gt; present a critical mathematical framework for implementing massively parallel finite element methods. There's more than one way to skin this cat, since DDMs give you the flexibility to do a lot of different things. However, one of the most important features of DDMs is that they can be formulated to allow non-matching meshes at interfaces between domains. In the context of mesh refinement, this allows the mesh for each domain to refined independently of all other domains, avoiding the need for a truly parallel mesher. The final solution could look something like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Generate coarse mesh from original geometry.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Partition coarse mesh into domains and distribute to compute nodes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perform mesh refinement independently on each domain.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run simulation using DDM-enabled finite element method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23262" LastEditorUserId="23262" LastEditDate="2017-02-28T09:53:22.590" LastActivityDate="2017-02-28T09:53:22.590" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26281" PostTypeId="1" AcceptedAnswerId="26289" CreationDate="2017-02-27T23:56:19.023" Score="3" ViewCount="91" Body="&lt;p&gt;As a toy problem, I was looking at a classical second order system of the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\ddot{x}(t) + c \dot{x}(t) + k x(t) = 0 &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of the basic system above, I want to replace $x(t)$ with a Low-Pass filtered version of $x(t)$. This can change the equation to the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\ddot{x}(t) + c \dot{x}(t) + \frac{k}{\tau} \int_{0}^{t} \exp\left(\frac{-(t-\hat{t})}{\tau}\right)x(\hat{t}) d\hat{t} = 0 &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this form, how might I be able to find numerical stability analytically? In the original system, I could break it into a system of two first order ODEs and analyze the eigenvalues of the matrix relating independent variables to their derivatives. In this latter case, I don't see a strategy like that working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is motivated by an empirical observation myself and a colleague noted when we were simulating our missile system and found it would develop large errors for Explicit Euler, no matter the step size, but would work fine for 4th Order Runge Kutta. We reproduced it on two separate simulation codebases of our weapon, so it seems to be more than potentially bugs in the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since our system can be approximately viewed as a second order system with a Low-Pass Filter (due to filtering sensors for state estimations), I am interested in trying to understand the second equation from an accuracy and stability perspective. Any hints or insights about how to analyze this scenario would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2017-02-28T14:03:06.597" LastActivityDate="2017-03-01T01:52:22.697" Title="Finding Numerical Stability of Simple System with Integral Term Due to Low Pass Filter" Tags="|numerical-analysis|stability|differential-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26282" PostTypeId="2" ParentId="26264" CreationDate="2017-02-28T02:20:55.820" Score="7" Body="&lt;p&gt;Let us write the equation in state-space form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{d}{dt}\begin{bmatrix}x_{1}(t)\\&#xA;x_{2}(t)&#xA;\end{bmatrix}+\begin{bmatrix}\eta/2 &amp;amp; -\omega\\&#xA;\omega &amp;amp; \eta/2&#xA;\end{bmatrix}\begin{bmatrix}x_{1}(t)\\&#xA;x_{2}(t)&#xA;\end{bmatrix}=0 \tag1&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the natural frequency is $\omega \triangleq \sqrt{1-\eta^2/4}$. It is easy to verify that the characteristic polynomial is indeed $p(\lambda) = \lambda^2 + \eta \lambda + 1$ by construction. When $\eta \ll 1$, we have $\omega \approx 1$, so the solution can always be approximated to great accuracy as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{bmatrix}x_{1}(t)\\&#xA;x_{2}(t)&#xA;\end{bmatrix}=\exp\left(-t\begin{bmatrix}0 &amp;amp; -1\\&#xA;1 &amp;amp; 0&#xA;\end{bmatrix}\right)\begin{bmatrix}y_{1}\\&#xA;y_{2}&#xA;\end{bmatrix}\tag2&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $y_{1}=x_{1}(0), y_{2}=x_{2}(0)$. Indeed this is the exact solution when $\eta=0$. Now, with a finite $\eta$, this solution is not completely correct, so we let $y_{1},y_{2}$ vary with time to yield&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{bmatrix}x_{1}(t)\\&#xA;x_{2}(t)&#xA;\end{bmatrix}=\exp\left(-t\begin{bmatrix}0 &amp;amp; -1\\&#xA;1 &amp;amp; 0&#xA;\end{bmatrix}\right)\begin{bmatrix}y_{1}(t)\\&#xA;y_{2}(t)&#xA;\end{bmatrix},\tag3&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $y_{1}(0)=x_{1}(0)$, $y_{2}(0)=x_{2}(0)$. These variables can be intuitively understood as “corrections” to the initial guess above. Then using the laws of matrix exponentials, we derive an ODE for the correction terms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{d}{dt}\begin{bmatrix}y_{1}(t)\\&#xA;y_{2}(t)&#xA;\end{bmatrix}+\begin{bmatrix}\eta/2 &amp;amp; -(\omega-1)\\&#xA;(\omega-1) &amp;amp; \eta/2&#xA;\end{bmatrix}\begin{bmatrix}y_{1}(t)\\&#xA;y_{2}(t)&#xA;\end{bmatrix}=0.\tag4&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This latter ODE is non-stiff, and can be solved using any simple integration method. With $y_{1}(t)$ and $y_{2}(t)$ known, $x(t)$ is easily recovered. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize, the principal idea is to start with an excellent initial guess, and to write the true solution as a time-dependent correction multiplied by this initial guess. Solving for the correction is then a far easier problem (read: less stiff) than solving for the solution directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The particularly strategy above is known as an “exponential integrator” (more specifically, the Lawson type), and assumes that the nonlinear dynamics can be well approximated by a linear one. For this very specific problem of a damped oscillation, it is also known as a phasor transform.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2017-03-03T20:06:09.093" LastActivityDate="2017-03-03T20:06:09.093" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26283" PostTypeId="2" ParentId="26279" CreationDate="2017-02-28T03:05:12.913" Score="4" Body="&lt;p&gt;It is difficult, in practice, to construct finite dimensional subspaces of $V$ that are conforming and that still satisfy appropriate approximation properties. For example, finite element spaces lack appropriate continuity properties across faces between cells. But, it is not difficult to use Fourier approximations that are divergence free and that allow you what you want to do. Of course, Fourier approximations require you to work in a box geometry -- and that is their main practical drawback, because the Navier-Stokes equations are just not very interesting from an applied perspective in a box geometry.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-02-28T03:05:12.913" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26284" PostTypeId="1" AcceptedAnswerId="26295" CreationDate="2017-02-28T11:11:49.860" Score="2" ViewCount="677" Body="&lt;p&gt;I am working on a generalized eigenvalue problem of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\boldsymbol{A}\cdot\boldsymbol{x}=\lambda\boldsymbol{B}\cdot\boldsymbol{x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\boldsymbol{B}$ is &lt;strong&gt;not&lt;/strong&gt; symmetric positive. Therefore I recast the problem to&#xA;$$&#xA;\boldsymbol{B}^{-1}\cdot(\boldsymbol{A}\cdot\boldsymbol{x})=\lambda\boldsymbol{x}\ .&#xA;$$&#xA;I am &lt;em&gt;lucky&lt;/em&gt;: the matrix $\boldsymbol{B}$ is block diagonal. Therefore I can compute the inverse by an LU-factorization of the diagonal blocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This eigenvalue problem is then solved with the ARPACK solver. ARPACK is used because of its feature to extract specific parts of the spectrum like eigenvalues with the largest real part. The method works well for small problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I increase the problem size, the performance of the ARPACK algorithmen in terms of iterations required increases. The condition number of $\boldsymbol{B}^{-1}\cdot\boldsymbol{A}$ increases heavily up to the order $10^6$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way of preconditioning the ARPACK algorithmen to accelerate the convergence?&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2017-03-01T19:04:00.140" Title="Preconditioning ARPACK eigenvalue solver" Tags="|linear-algebra|eigensystem|arpack|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="26285" PostTypeId="2" ParentId="23860" CreationDate="2017-02-28T11:12:03.493" Score="0" Body="&lt;p&gt;Just a guess, but with a 1d minimization of 2 parameters, the best fit is likely a single global minimum in k1,k2-space shaped like a 2d bowl (depending on the function). If true, that is your only local min. (It may be helpful to look up convex vs concave functions; maxima are negative minima too..) So, guess initial values of the parameters so that you can keep one parameter constant while trying 4 indexed values of the other parameter - one in each cardinal direction. Then vary the next parameter while keeping the former constant in the same way. If the optimization error decreases in any direction, then you know that is the direction of your minima. You can throw this in an iterator with if conditions (if error increases then pass) and let it run; you only care about a subinterval of minimized values, in which the global minima exists. You can even use the change in optimization error per parameter fluctuation to get an idea of a cutoff range. (Though I imagine scipy optimization routines in other answers are more convenient with know how.)&lt;/p&gt;&#xA;" OwnerDisplayName="user23442" LastActivityDate="2017-02-28T11:12:03.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26286" PostTypeId="1" CreationDate="2017-02-28T14:44:17.307" Score="2" ViewCount="174" Body="&lt;p&gt;I was wondering if there is a simple or known way to minimize the number of unique elements in a decision variable (vector). &#xA;Note that I'm not asking for minimization of nonzero elements (rank constraint). &#xA;In particular I'm searching for a penalization (soft constraint) or hard constraint in the form &#xA;$$f(x) &amp;lt;= n_{max} $$ &#xA;where f() is what I'm searching for, x is the decision vector and n_max is the maximum number of unique elements that x is allowed to have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: I am not trying to minimize the number of unique elements in a vector &quot; per se&quot;, which of course is trivial. The question is related to minimizing the unique element in a vector of decision variables that is the solution to another optimization problem. Eg:&#xA;$$argmin_{x} g(x) $$ s.t. $$x \in \mathbb{X} $$ $$\sum(unique(x))&amp;lt;n_{max }$$ &#xA;Thank you for your time, &#xA;Lorenzo&lt;/p&gt;&#xA;" OwnerUserId="23451" LastEditorUserId="23451" LastEditDate="2017-03-01T14:07:30.790" LastActivityDate="2017-03-02T15:59:42.070" Title="Minimize the number of unique elements in a vector" Tags="|optimization|convex-optimization|rank|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26287" PostTypeId="1" AcceptedAnswerId="26288" CreationDate="2017-02-28T15:10:05.617" Score="1" ViewCount="432" Body="&lt;p&gt;I am trying to calculate real speed and time in free fall of a body. I wrote a code in Fortran and I am trying to improve it by using RK4 method&lt;a href=&quot;https://i.stack.imgur.com/hZ1gQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hZ1gQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;x=time y=total free fall&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Purple line using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DO WHILE ((K1-K2) &amp;gt; y)&#xA;             y = y + t*((G*m*t/(2*((r-y)**2))) + sqrt(2*(G*m*y)/(r*(r-y)))) &#xA;             time = time + t&#xA;            END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Green line using(RK4 method):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DO WHILE ((D1-D2) &amp;gt; y)&#xA;           k1 = t*((G*m*(0)/(2*((r-y)**2))) + sqrt(2*(G*m*y)/(r*(r-y))))&#xA;           k2 = (t/2)*((G*m*(t/2)/(2*((r-(y+k1))**2))) + sqrt(2*(G*m*(y+k1)/(r*(r-(y+k1))))))&#xA;           k3 = (t/2)*((G*m*(t/2)/(2*((r-(y+k2))**2))) + sqrt(2*(G*m*(y+k2)/(r*(r-(y+k2))))))&#xA;           k4 = t*((G*m*t/(2*((r-(y+k3))**2))) + sqrt(2*(G*m*(y+k3))/(r*(r-(y+k3)))))&#xA;             y = y + (1.0/6.0)*(k1+2*k2+2*k3+k4)&#xA;             time = time + t&#xA;            END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All of other codes are the same (stepsize, for example) and I am getting better results without RK4. I guess I didn't understand RK4 right and I write it wrong but I cant see my mistake. What should I change to get better results with RK4 or did I do it wrong ?&lt;/p&gt;&#xA;" OwnerUserId="23448" LastEditorUserId="9667" LastEditDate="2017-02-28T18:12:22.417" LastActivityDate="2017-02-28T19:08:14.660" Title="Numerical Free Fall Analysis with RK4" Tags="|numerical-analysis|computational-physics|fortran|runge-kutta|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26288" PostTypeId="2" ParentId="26287" CreationDate="2017-02-28T18:48:02.590" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;You are using &lt;code&gt;t&lt;/code&gt; for the time step. It would be clearer to use &lt;code&gt;dt&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;k1&lt;/code&gt;,... should be the derivatives, not the steps. So you should use&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;k1 = ((G*m*t/(2*((r-y)**2))) + sqrt(2*(G*m*y)/(r*(r-y))))&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to calculate the first derivative. And also for &lt;code&gt;k2&lt;/code&gt;,... don't multiply by time either.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then when calculating &lt;code&gt;k2&lt;/code&gt; (and &lt;code&gt;k3&lt;/code&gt;) don't use &lt;code&gt;y+k1&lt;/code&gt; but &lt;code&gt;y + 0.5*dt*k1&lt;/code&gt; inside the formula. The first two tentative steps in RK4 are only half step long.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To make the code look nicer, you should write a function that calculates the acceleration, instead of pasting the same formula 4 times. Less visual clutter also helps thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also your equations are wrong, but that's a separate issue. With these corrections at least you should be able to get the same result from your Euler and RK versions.&lt;/p&gt;&#xA;" OwnerUserId="4780" LastEditorUserId="4780" LastEditDate="2017-02-28T19:08:14.660" LastActivityDate="2017-02-28T19:08:14.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26289" PostTypeId="2" ParentId="26281" CreationDate="2017-03-01T01:52:22.697" Score="4" Body="&lt;p&gt;You can introduce an auxiliary variable &#xA;$$&#xA;  y(t) = \int_0^t \exp\left(\frac{-(t-\hat t)}{\tau}\right) x(\hat t) \; d\hat t,&#xA;$$&#xA;which you can differentiate to get on ODE for $y(t)$ that depends on $x(t)$, but not any of the times before (i.e., no longer contains an integral). In other words, you end up with a system of two ODEs that you can analyze as you always do.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-03-01T01:52:22.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26290" PostTypeId="1" CreationDate="2017-03-01T05:04:08.033" Score="2" ViewCount="84" Body="&lt;p&gt;Consider solving a differential equation system for 1D, 2D, or 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It involves various input and output &quot;field&quot; variables, which, more often than not, correspond to various physical quantities relevant to the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical discretization step is to agree on a mesh, and then solve the system. We might refine the mesh in some areas of the spatial-domain (1D, 2D, or 3D) to increase resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, each &quot;field&quot; has its own high-resolution requiring regions, which are, in general, different from every other field. E.g., considering 1D only, a sin(x) function is relatively flat around pi/2, but steep around 0, whereas cos(x) is flat around 0, but step around pi/2. That means a non-uniform mesh optimized for a sin 1D field is not optimal for a cos field, and vice versa. This means we need a mesh for sin, and another mesh for cos, if we are to do things in the most optimized way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite spending many years in the area of scientific computing, I'm surprised that I've never come across such a discussion that highlights the need for &quot;multi-mesh numerics&quot;. Granted it has its own problems, e.g., the proliferation of meshes for intermediate results, computational cost of conversion, and so on, but academically, it's a direction worth pursuing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, can anyone help me point to a body of literature that covers this topic? Or is there a different term used for this that I'm not familiar with?&lt;/p&gt;&#xA;" OwnerUserId="15198" LastActivityDate="2017-03-01T05:04:08.033" Title="Literature on numerical solving based on multiple meshes?" Tags="|optimization|discretization|numerics|mesh|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26291" PostTypeId="1" AcceptedAnswerId="26296" CreationDate="2017-03-01T07:30:21.870" Score="3" ViewCount="212" Body="&lt;p&gt;As the title mentioned, some suggestions and references are needed to help me to construct a function in $H^3(\Omega)$, but not in $H^4(\Omega)$？The domain $\Omega=[0,1]\times[0,1]$ and has zero trace. I'm very glad to receive some suggestions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, I want to construct a function satisfying&#xA;\begin{align}&#xA;&amp;amp;-\Delta u=f\quad in~~\Omega=[0,1]\times[0,1],\\&#xA;&amp;amp;~~u=0,~~~~~~~~~on ~~~~\partial\Omega.&#xA;\end{align}&#xA;It may be like $r^{2/3}\sin(2/3\theta)$ on a L-shape domain $[-1,1]\times [-1,1]\backslash[0,1]\times[-1,0]$, which has singularity in $(0,0)$.&lt;/p&gt;&#xA;" OwnerUserId="7494" LastEditorUserId="7494" LastEditDate="2017-03-02T01:24:18.450" LastActivityDate="2017-03-02T04:36:16.467" Title="How to construct a function in $H^3(\Omega)$, but not in $H^4(\Omega)$?" Tags="|finite-element|pde|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26293" PostTypeId="1" CreationDate="2017-03-01T16:22:43.473" Score="1" ViewCount="61" Body="&lt;p&gt;my problem is a ecuation i don't understand, I have no idea how to solve that, is a classic minimisation programme of quadratic form with linear constraints, here is the ecuation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{i,j}^n[P(i/j)P(j) - \sum_{k=1}^rt(ijk)\prod_k]^2 + \sum_{i,j}^n[P(i/ \bar j)P(\bar j) - \sum_{k=1}^rs(ijk)\prod_k]^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to the contrainsts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{k=1}^r \prod_k =1, \prod_k \ge 0 , \text{ For all k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{minimize }[(0.30 -  \prod_1, -  \prod_5)^2  + (0.15 -  \prod_3, -  \prod_7)^2&#xA; + (0.30 -  \prod_1, -  \prod_3)^2  + (0.04 -  \prod_5, -  \prod_7)^2 &#xA; + (0.24 -  \prod_1, -  \prod_5)^2  + (0.04 -  \prod_2, -  \prod_6)^2&#xA; + (0.24 -  \prod_1, -  \prod_2)^2 + (0.24 -  \prod_5, -  \prod_6)^2&#xA; + (0.27 -  \prod_1, -  \prod_3)^2 + (0.35 -  \prod_2, -  \prod_4)^2&#xA; + (0.35 -  \prod_1, -  \prod_2)^2 + (0.20 -  \prod_3, -  \prod_4)^2]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All these equations I got from this &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/0016328775900488$&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; called &quot;SMIC 74—A method for constructing and ranking scenarios&quot;, i must applicate this in a programming languaje (preferably web), can help me with this forecasting method SMIC?&lt;/p&gt;&#xA;" OwnerUserId="23462" LastActivityDate="2017-03-01T16:22:43.473" Title="minimization a quadratic form with linear constraints (Prospective method SMIC 74 method)" Tags="|quadratic-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26295" PostTypeId="2" ParentId="26284" CreationDate="2017-03-01T18:49:41.047" Score="3" Body="&lt;p&gt;I've summarized the comments thread of your original question into an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few things that you can try:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Increase the number of your Arnoldi vectors (&lt;strong&gt;NCV&lt;/strong&gt;) generated at each iteration. Here's what the ARPACK documentation for &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/UG/node136.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSAUPD&lt;/a&gt; says about this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;At present there is no a-priori analysis to guide the selection of NCV relative to NEV (the number of eigenvalues you're looking for). The only formal requirement is that NCV &amp;gt; NEV. However, it is recommended that NCV &amp;gt;= 2*NEV.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;If many problems of the same type are to be solved, one should experiment with increasing NCV while keeping NEV fixed for a given test problem. This will usually decrease the required number of OP*x operations but it also increases the work and storage required to maintain the orthogonal basis vectors. The optimal &quot;cross-over&quot; with respect to CPU time is problem dependent and must be determined empirically.&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Revisit your implementation of the linear operator. In your case, it should read something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(a) $ z \leftarrow A v$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b) Solve $M w = z$ for $w$, using the pre-computed factorization of $M$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you're looking for the smallest eigenvalues - or eigenvalues close to a given real value, it is advisable to use the &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/arpack.html#shift-invert-mode&quot; rel=&quot;nofollow noreferrer&quot;&gt;shift-and-invert mode of ARPACK&lt;/a&gt;. The shift $\sigma$ can be zero (see the example they give &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/arpack.html#shift-invert-mode&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) or any other real/complex value. Your linear operator becomes $(A - \sigma M)^{-1} M $ (for $M$ symmetric and indefinite).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If SciPy-ARPACK works well enough for your problem, then PETSc/SLEPc are probably overkill.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-03-01T19:04:00.140" LastActivityDate="2017-03-01T19:04:00.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26296" PostTypeId="2" ParentId="26291" CreationDate="2017-03-02T04:36:16.467" Score="3" Body="&lt;p&gt;There are fundamentally two options you have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You construct a function based on the radial distance $r=\sqrt{x^2+y^2}$ and make the &lt;em&gt;ansatz&lt;/em&gt; $w(x,y)=r^\alpha$ as suggested by Praveen Chandrashekar in the comments. Then compute the $H^s$ norm of this function on the unit disk via&#xA;$$&#xA;  \|w\|^2_{H^s} = 2\pi \int_0^1 |\partial^s_r w(r)|^2 \, r\, dr, &#xA;$$&#xA;which yields, up to constants&#xA;$$&#xA;  \|w\|^2_{H^s} \propto \int_0^1 r^{2(\alpha-s)} \, r\, dr. &#xA;$$&#xA;This integral is only finite if $2(\alpha-s)+1 &amp;gt; -1$, or in other words if $\alpha &amp;gt; -1+s$. That is, the function $w=r^\alpha$ is in $H^3$ if $\alpha&amp;gt;2$ but it is not in $H^4$ if $\alpha\le 3$. Praveen C.'s choice of $\alpha=\frac 52$ is a valid choice then.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You construct a one dimensional function $w(x,y)=|x|^\beta$ that simply doesn't depend on the $y$ coordinate at all. Consequently, all two-dimensional norms really just come down to one dimensional ones. By a similar argument as above, the norm of $w$ is given by &#xA;$$&#xA;  \|w\|^2_{H^s} = \int_{-1}^1 |\partial^s_x w(x)|^2\, dr, &#xA;$$&#xA;which yields again, up to constants&#xA;$$&#xA;  \|w\|^2_{H^s} \propto \int_0^1 |x|^{2(\beta-s)}\, dr. &#xA;$$&#xA;This is again finite if $2(\beta-s)&amp;gt;-1$, or $\beta&amp;gt;-\frac 12 + s$. So $w\in H^3$ but $w\not\in H^4$ if $\frac 52 &amp;lt; \beta \le \frac 72$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In both cases, $w$ does not satisfy the boundary conditions you ask for. But that can be addressed in the following way: Let $u_0$ satisfy the equation&#xA;$$&#xA;  -\Delta u_0 = 0 \qquad \text{on $\Omega=(-1,1)^2$},&#xA;$$&#xA;with boundary values $u_0|_{\partial\Omega}=-w|_{\partial\Omega}$, then $u=w+u_0$ does satisfy the equation you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is, of course, not a priori clear that $u_0$ actually has at least $H^3$ regularity, to ensure that $u$ is also in $H^3$. But a bit of functional analysis shows that since $w\in H^3$ by construction, $w|_{\partial\Omega}\in H^{5/2}(\partial\Omega)$, which in turn again implies that $u_0\in H^3$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-03-02T04:36:16.467" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26297" PostTypeId="1" CreationDate="2017-03-02T06:49:30.993" Score="2" ViewCount="144" Body="&lt;p&gt;I have read some papers about time-fractional PDEs solved by finite element method. The time-fractional derivative is the Caputo derivative defined by &#xA;$$&#xA;\frac{\partial^{\alpha}u}{\partial t^{\alpha}}=\frac{1}{\Gamma(1-\alpha)}\int_{0}^t\frac{\partial u}{\partial s}\frac{ds}{(t-s)^{\alpha}},~~~~0&amp;lt;\alpha&amp;lt;1.&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we consider the following time-fractional diffusion equation&#xA;\begin{align}&#xA;&amp;amp;\frac{\partial^{\alpha}u}{\partial t^{\alpha}}-\Delta u=f,~~in~~\Omega,\\&#xA;&amp;amp;u=0,~~~~~~~~~on~~~\partial\Omega,\\&#xA;&amp;amp;u(0)=u_0,&#xA;\end{align}&#xA;where $\Omega$ is a convex and bounded domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.What does mean the singularity of the time-fractional derivative?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.In what cases, the singularity of the time-fractional derivative can lead to low smoothness of solution $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.What does mean the regularity of a function is not smooth about time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone give me some explanations?&lt;/p&gt;&#xA;" OwnerUserId="7494" LastActivityDate="2017-03-02T06:49:30.993" Title="Some questions about fractional derivative" Tags="|finite-element|pde|space-time-galerkin|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26299" PostTypeId="2" ParentId="26265" CreationDate="2017-03-02T15:10:22.393" Score="0" Body="&lt;p&gt;For a plate element lying in the x-y plane, the consistent nodal loads corresponding to a pressure load on the&#xA;face are given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f_i = \int_{A_e} N_i P(x,y) dx dy $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N_i$ is the shape function at the ith node and $P$ is the pressure. Of course, if $P$ is constant, as it is in your&#xA;case, it can  be moved outside the integral so the nodal loads are simply the integral of the shape functions multiplied&#xA;by the pressure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f_i = P\int_{A_e} N_i dx dy $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only remaining step is to sum these elemental load components into the appropriate locations in the global load vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's all there is to it.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-03-02T15:10:22.393" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26300" PostTypeId="2" ParentId="26286" CreationDate="2017-03-02T15:59:42.070" Score="3" Body="&lt;p&gt;Inspired by the well-known approach of employing a regularization (penalty) term $\gamma ||\mathbf{x}||_1$ as a means of relaxing the requirement of minimum non-zero elements of vector $\mathbf{x}$ (e.g., see &lt;a href=&quot;https://scicomp.stackexchange.com/a/21864/21460&quot;&gt;here&lt;/a&gt;), you could try using a regularization of the from &#xA;$$&#xA;\gamma ||\mathbf{x} - c\mathbf{1}||_1,&#xA;$$&#xA;where $\mathbf{1}$ is the all-ones vector and $c\in \mathbb{R}$ (assuming the domain is the set of real-valued vectors). You would then solve the resulting problem with respect to both $\mathbf{x}$ and $c$.&lt;/p&gt;&#xA;" OwnerUserId="21460" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2017-03-02T15:59:42.070" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26302" PostTypeId="1" AcceptedAnswerId="26303" CreationDate="2017-03-03T10:44:36.880" Score="4" ViewCount="3360" Body="&lt;p&gt;I have $m$ inequalities involving $n$ variables as follows&#xA;$$a_{1,j} x_1 +a_{2,j} x_2 +\dots +a_{n,j} x_n&amp;gt;0 \quad \text{for} \quad 1 \leq j \leq m$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I check if a solution exists (with the possible help of computer)?&lt;/p&gt;&#xA;" OwnerUserId="23486" LastEditorUserId="713" LastEditDate="2017-03-03T19:58:05.580" LastActivityDate="2017-03-04T02:01:45.320" Title="Checking the feasibility of a system of inequalities" Tags="|constrained-optimization|linear-programming|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26303" PostTypeId="2" ParentId="26302" CreationDate="2017-03-03T12:40:30.093" Score="5" Body="&lt;p&gt;You can check feasibility of a set of linear inequalities by constructing a linear programming (LP) problem with a &quot;dummy&quot; objective, e.g., &#xA;$$&#xA;\begin{align}&#xA;\max_{\{x_i\}_{i=1}^n} &amp;amp;\;0\\&#xA;\text {subject to } &amp;amp; \sum_{i=1}^n a_{i,j} x_i \geq0 \text{ for all } 1 \leq j \leq m.&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any reasonable LP solver will either return an &quot;optimal&quot; $\{x_i\}$ (implying feasibility of the inequalities) or will return a proof of infeasibility (via Farkas' Lemma). &lt;/p&gt;&#xA;" OwnerUserId="21460" LastEditorUserId="11716" LastEditDate="2017-03-04T02:01:45.320" LastActivityDate="2017-03-04T02:01:45.320" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26304" PostTypeId="1" AcceptedAnswerId="26306" CreationDate="2017-03-03T14:28:10.067" Score="1" ViewCount="51" Body="&lt;p&gt;I have a &lt;strong&gt;computationally expensive&lt;/strong&gt; objective function, whose analytical form is not available. The only input argument to the objective function is an &lt;em&gt;integer&lt;/em&gt; variable. The goal is to compute the integer variable that minimizes the objective function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_{sol} = argmin_{x \epsilon [x_{min},x_{max}]}  f(x)$$&#xA;where $x$ is &lt;em&gt;integer&lt;/em&gt;. $f(x)$ returns a scalar value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The objective function is computed by solving a complicated system of non-linear coupled PDEs). The goal is to hopefully use some form of data-driven black-box optimisation which can solve over &lt;em&gt;integer&lt;/em&gt; values for $x$ without resorting to a time-consuming/slow for-loop, since the bounds are very large.  A MATLAB interface to a suitable solver is desired.&lt;/p&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2017-03-03T19:43:01.230" Title="Optimisation of purely integer quantity with bound-constraints for a 1D expensive function whose analytical form is not available" Tags="|optimization|matlab|constrained-optimization|nonlinear-programming|least-squares|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26305" PostTypeId="2" ParentId="26304" CreationDate="2017-03-03T15:56:39.953" Score="2" Body="&lt;p&gt;Without any information/insights about $f(\cdot)$, your only option to find the maximum is by brute force, i.e., evaluate $f$ for all integers. Now, if this is is practically impossible to do, your only hope is to identify properties of $f$ and/or its maximum that will allow to obtain the maximum with fewer evaluations of $f$. For example, if it makes sense to define $f$ over $\mathbb{R}$ (set of reals) and $f$ is also continuous, you could perform an optimization over $\mathbb{R}\cap [x_\text{min}, x_\text{max}]$ (which is, in principle, much more efficient) in order to narrow the range of integers where you should look for the optimum.&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2017-03-03T15:56:39.953" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26306" PostTypeId="2" ParentId="26304" CreationDate="2017-03-03T19:43:01.230" Score="4" Body="&lt;p&gt;@Stellos already gave the correct answer, but let me try to back it up with a bit of intuition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think of your function $f(x)$ as a function that would actually make sense for any real-valued argument $x$. Then, if that function happened to be of the form $f(x)=\sin(1000x)$, you would have lots of maxima and minima between each integer point and, in essence, which &lt;em&gt;integer&lt;/em&gt; value happens to minimize $f(x)$ comes down to chance: where the integers happen to line up with the minima of the high frequency sine function. In other words, just because you know $f(15)$ and $f(17)$ tells you nothing about $f(16)$, and consequently the only way to find which integer $x$ minimizes $f(x)$ is to try them all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, imagine that you knew that $f(x)$ only varies slowly, on lengthscales much larger than the distance between integers. Then if you know that $f'(16)\approx\frac{f(17)-f(16)}{1}$, and you can run a derivative based search to find the minimum of the function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, if you knew that the function only changes slowly, you could use something like bisection search to find where $f'(x)\approx 0$, and the minimum is likely going to be one of the adjacent integers to all of these critical points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even better, if you knew that your function is, for example, convex or concave, then you would know that there is only one minimum or maximum, plus ones possibly located at the end points of your interval, and you can again find them through bisection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, it is often very valuable to establish such &lt;em&gt;qualitative&lt;/em&gt; properties of your objective function because it allows you to choose more efficient algorithms than if you knew nothing at all about your $f(x)$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-03-03T19:43:01.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26307" PostTypeId="2" ParentId="26302" CreationDate="2017-03-03T19:48:42.383" Score="1" Body="&lt;p&gt;In general, finding a feasible point -- more specifically a vertex of the feasible region -- is a non-trivial task. This is the first step of any implementation of the vertex algorithm to solve linear problems, and it is typically done by solving a &lt;em&gt;dual&lt;/em&gt; problem. In this dual problem, we know that the origin is a feasible vertex and that the &lt;em&gt;solution&lt;/em&gt; of the dual problem (which we can again find through the vertex algorithm) is a feasible vertex of the original problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, this means that you can test for feasibility by writing down a &quot;fake&quot; linear problem (as suggested by @Stellos in his/her post), constructing the dual problem, and solving it. If it succeeds, then you have a feasible vertex of your original set of constraints; there is then no need to solve the fake problem, of course. Any commercial LP solver will also be able to do that for you, though it may nevertheless try to solve for the fake problem.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-03-03T19:48:42.383" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26308" PostTypeId="2" ParentId="26302" CreationDate="2017-03-03T20:13:34.000" Score="4" Body="&lt;p&gt;I agree with Stelios' answer, but it could use some fleshing out. This problem is called the &quot;Phase One&quot; problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, convert your problem into &quot;standard form&quot;.&#xA;\begin{align*}&#xA;\min_x 0 \\&#xA;s.t. \;\;Ax =b \\&#xA;x \ge 0&#xA;\end{align*}&#xA;This can always be done for problems of your type, though it may require introducing new variables. Next, solve the Phase One problem for the new problem.&#xA;You can solve it by solving the following LP:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\min \sum_i z_i \\&#xA;Ax + z = b \\&#xA;z \ge 0 \\&#xA;x \ge 0&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A feasible initial point of this new problem is $x=0$, $z=b$ (assuming $b\ge 0$. If not, just flip the right signs in $A$ to make it so).&#xA;Once you reach the minimum of this linear program, you will be at a corner of the polytope that you actually care about.&#xA;If the minimum is nonzero, then you know that no such point exists.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastEditorUserId="7378" LastEditDate="2017-03-03T20:21:02.653" LastActivityDate="2017-03-03T20:21:02.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26309" PostTypeId="2" ParentId="26264" CreationDate="2017-03-03T22:09:02.203" Score="1" Body="&lt;p&gt;I would give spectral methods a trial. For your one-dimensional equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2x}{dt^2}+ \eta \frac{dx}{dt} + x =0\,,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this basically means the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choose some basis, e.g. some Lagrange polynomials over a grid.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the matrix elements of the derivatives (and possibly other terms).&lt;/li&gt;&#xA;&lt;li&gt;Diagonalize the matrix (which is possible for the function above, as it's symmetric), and use the eigenvalues and eigenvectors to construct the matrix exponential, with which one can propagate the solution to very long times. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Points 1. and 2. are identical as if you would use an explicit or implicit method. However, point 3. gives you the &lt;strong&gt;exact result&lt;/strong&gt; for the chosen discretization, by which you can take very large steps. This is in sharp contrast to the mentioned explicit and implicit methods (which so to say only approximate the matrix exponential).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using spectral methods, the stiffness then appears through the fact that the eigenvalues differ by some orders of magnitude.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2017-03-03T22:09:02.203" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26310" PostTypeId="2" ParentId="25353" CreationDate="2017-03-04T02:01:58.340" Score="0" Body="&lt;p&gt;which one is better?... Is a wide question. It depends, what is your main goal.&#xA;Direct solvers usually require considerable RAM, even for a sub-domain solution.&#xA;One can use iterative solvers for sub-domains, and an iterative solver for global solution/convergence. The memory footprint is minimal, but this will be slower (according to my experience) than using direct solvers or iterative/direct hybrid methods. Check &lt;a href=&quot;http://www.domain-decomposition.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.domain-decomposition.com&lt;/a&gt; for coverage recent advances in DD.&lt;/p&gt;&#xA;" OwnerUserId="23491" LastActivityDate="2017-03-04T02:01:58.340" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26312" PostTypeId="1" CreationDate="2017-03-04T04:36:03.207" Score="1" ViewCount="129" Body="&lt;p&gt;Trotter expansions say:&#xA;$$ e^{A+B} = \lim_{P\to\infty} \big(e^{A/2P} e^{B/P} e^{A/2P} \big)^P. $$&#xA;With $P = 2$, it becomes (with high accuracy)&#xA;$$ e^{A/4} e^{B/2} e^{A/2} e^{B/2} e^{A/4}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I want to solve&#xA;$\frac{dy}{dt} = Cy$, where $C = A + B$. $C$ can be time dependent. How can I solve this equation by splitting $C$ into $A$ and $B$, before using any ode solving scheme. Will it yield a gain in performance by doing so, like in matrix exponential?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Background information: &#xA;&lt;a href=&quot;https://www.quora.com/What-is-the-significance-of-the-Trotter-product-formula-in-physics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quora.com/What-is-the-significance-of-the-Trotter-product-formula-in-physics&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20760" LastEditorUserId="9667" LastEditDate="2017-03-05T00:00:48.263" LastActivityDate="2017-03-05T00:00:48.263" Title="Trotter expansions in ode solver?" Tags="|numerical-analysis|ode|computational-physics|operator-splitting|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26313" PostTypeId="1" AcceptedAnswerId="26322" CreationDate="2017-03-04T17:34:21.103" Score="9" ViewCount="867" Body="&lt;p&gt;In this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/20279/how-can-i-solve-wave-equation-for-circular-membrane-in-polar-coordinates/20354#20354&quot;&gt;question&lt;/a&gt;, I suggested that the &lt;a href=&quot;https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Couran-Friedrichs-Lewy (CFL) condition&lt;/a&gt; for the wave equation in polar coordinates reads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C = 2c\frac{\Delta t}{\Delta r \Delta \phi} \leq C_\max \enspace ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c$ is the phase speed. I suggested this from an intuitive point of view, and it worked in that example. Nevertheless, This is probably not right, And I could not find an expression for this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; What is the CFL condition in polar coordinates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question was asked before in Math.SE: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Harley (&lt;a href=&quot;https://math.stackexchange.com/users/357049/harley&quot;&gt;https://math.stackexchange.com/users/357049/harley&lt;/a&gt;), CFL condition in polar coordinates., URL (version: 2016-07-31): &lt;a href=&quot;https://math.stackexchange.com/q/1874881&quot;&gt;https://math.stackexchange.com/q/1874881&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-03-05T20:30:54.257" Title="CFL condition in polar coordinates" Tags="|finite-difference|stability|wave-propagation|cfl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26314" PostTypeId="2" ParentId="26312" CreationDate="2017-03-04T21:21:19.087" Score="0" Body="&lt;p&gt;Your equation $\frac{dy}{dt} = C(t)y$ has the exact solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;y(t) = \mathcal T exp\left(\int_0^t C(s) ds\right) y(0)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal T$ is the time-ordering operator which orders operators at later time to the left. Approximate the integral by a discrete sum ($t=N\Delta t$),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y(t) = \mathcal T exp\left(\sum_{i=1}^N H(t_i) \Delta t\right) y(0)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and split the terms (*),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y(t) = \mathcal T \prod_{i=1}^N exp\left(H(t_i) \Delta t\right) y(0).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can trivially apply the time-ordering operator, leading to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y(t) = \prod_{i=1}^N exp\left(H(t_i) \Delta t\right) y(0).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The latter equation says that one can repeatedly apply matrix exponentials to obtain the solution. This is a fact on which almost any numerical integrator relies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now note the step marked by (*). Here, the exponential of a sum of several operators is decomposed into a product of these exponentials, that is, the elementary splitting formula $exp(H_1+H_2) = exp(H_1) exp(H_2)$ is applied several times (the formula is, of course, only an approximation when the two operators do not commute).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does this help you in solving your equation? In no way. But it's good to know. For the practical part of your question, please read the first answer in your linked question on quora, it basically says it all.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2017-03-04T21:21:19.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26315" PostTypeId="1" CreationDate="2017-03-04T21:32:14.040" Score="2" ViewCount="278" Body="&lt;p&gt;I'm trying to find an approach for integral image resizing.&#xA;I found out that I can do it with a bilinear interpolation method, but with this approach I can only resizing by the factor which is a power of two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a paper &lt;a href=&quot;http://publications.dustsigns.de/Gschwandtner14a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Speeding Up Object Detection&#xA;Fast Resizing in the Integral Image Domain)&lt;/a&gt; which describes a formula which allows to resize arbitrarily:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$II_r(x,y) \approx \frac{1}{4a^2}\space \cdot bilinear(II, (2ax + b, 2ay+b))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$2a \space - resizing\space factor$$ &#xA;$$b \space - resizing\space filter\space offset$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to ask about explanation of this formula. I assume that I have to find a correct value of resizing filter offset but I don't know how to do this.&lt;/p&gt;&#xA;" OwnerUserId="18200" LastEditorUserId="18200" LastEditDate="2017-03-04T22:16:31.887" LastActivityDate="2023-03-25T23:00:40.063" Title="Integral image resizing" Tags="|interpolation|image-processing|computer-vision|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26316" PostTypeId="2" ParentId="26312" CreationDate="2017-03-04T22:02:05.103" Score="4" Body="&lt;p&gt;What you're proposing is widely referred to as &lt;a href=&quot;https://en.wikipedia.org/wiki/Strang_splitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;Strang splitting&lt;/a&gt;.  There is a huge literature on similar methods.  There are also many &lt;a href=&quot;https://scicomp.stackexchange.com/search?q=splitting+method&quot;&gt;questions on this site on the same topic&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/tagged/operator-splitting&quot;&gt;a tag for them&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question seems to suggest that in your case the ODE is linear (but possibly time dependent), and you don't assume any special structure of $C$.  In this case there is (generically) nothing to be gained by using a splitting method.  Splitting methods are useful when your right-hand-side can be decomposed into two additive parts, each of which is much easier to integrate than the combination.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2017-03-04T22:02:05.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26317" PostTypeId="1" CreationDate="2017-03-05T10:10:55.803" Score="0" ViewCount="1199" Body="&lt;p&gt;When looking at the output file of my solver, I have been told that the time-step taken by the solver depends on parameters like the total number of elements and their relative size in my geometry, or the material properties. A material with a high Young modulus could decrease the time step taken by the solver, as well as high number of elements, or elements with great difference in size. I am solving under explicit solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, the time step appears to be &lt;em&gt;the time needed for the information to go through one element to another&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gets me quite confused. How is time step related with these parameters ? Is there any formal physical relationship ? What about this time to go through an element, does it have mathematical foundation? Is there any advice to get it fit correctly ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I am running a time-dependent linear elastic simulation in solid mechanics indeed.&lt;/p&gt;&#xA;" OwnerUserId="20191" LastEditorUserId="9667" LastEditDate="2017-03-05T18:22:31.907" LastActivityDate="2017-03-05T18:22:31.907" Title="Time step relationship with number of elements or material properties" Tags="|finite-element|computational-physics|wave-propagation|cfl|solid-mechanics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26318" PostTypeId="2" ParentId="26317" CreationDate="2017-03-05T13:52:22.173" Score="5" Body="&lt;p&gt;It sounds as if you're running a time-dependent linear elasticity simulation, right? Most likely, you're running an &quot;Explicit&quot; time-stepping scheme, which means that all of your information at time $t_{i+1}$ is computed entirely using information from time $t_i$ (or $t_{i-1}$...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One consequence of such a scheme is that in order to be a stable time stepping scheme, it must be able to propagate a wave through the material without the wave moving more than a single element distance in a single time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This imposes a relationship between your Young's Modulus $E$, density $\rho$, your mesh sizing $\Delta x$, and $\Delta t$.&#xA;Imprecisely, you need something like $\sqrt{\frac{E}{\rho}}\frac{\Delta t}{\Delta x}$ to be small in order for most schemes to be stable. The exact size of this ratio will depend on the time stepping scheme used (and the ratio is usually expressed using other elastic constants, but this scales in the same way).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This condition is called the &quot;CFL&quot; condition. It tells us that large $E$, small $\rho$, and small $\Delta x$ all force the solver to use a smaller $\Delta t$. In addition, this is a &lt;em&gt;global&lt;/em&gt; restriction on the time step. It will pick the &lt;strong&gt;smallest&lt;/strong&gt; element in your mesh to compute the stable time step. This is why a large difference in element sizes will result in a seemingly small time step compared to a mesh with uniform elements (that all happen to be larger).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a way to check if this is what is going on, try increasing your density by a factor of 100. The solver should pick a time step that is ~10x larger than the one it's using now.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastEditorUserId="7378" LastEditDate="2017-03-05T14:32:26.257" LastActivityDate="2017-03-05T14:32:26.257" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26319" PostTypeId="2" ParentId="26317" CreationDate="2017-03-05T14:26:32.633" Score="4" Body="&lt;p&gt;The most commonly used explicit ODE solver in structural analysis is the central difference method.&#xA;Because it is explicit, the solution becomes unstable if the time step is larger than a so-called critical&#xA;time step. The calculation of this critical time step is straightforward and can be found in many references&#xA;(e.g. page 808 in this text by &lt;a href=&quot;http://web.mit.edu/kjb/www/Books/FEP_2nd_Edition_4th_Printing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bathe &lt;/a&gt;) and is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta t = \frac{2}{\omega_{max}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\omega_{max}$ is the maximum vibration frequency of the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is generally too computationally costly to calculate $\omega_{max}$ exactly so, it is often assumed that the largest&#xA;frequency in the model is approximately that of the ``smallest'' element. This frequency depends on the characteristics&#xA;of the model and the specific finite elements used but is given by a formula in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega_e = \frac{c_e}{L_e}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L_e$ is a characteristic element length and $c_e$ is the wave speed in the material (e.g $\sqrt{E/\rho}$).&#xA;The wave speed is the speed of a stress wave propagating through the material (or elements in the mesh).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Robust explicit ODE solvers estimate this critical time step during the calculation and adjust the numerical time step&#xA;as necessary so that it is smaller than critical.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-03-05T14:32:30.920" LastActivityDate="2017-03-05T14:32:30.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26320" PostTypeId="5" CreationDate="2017-03-05T18:29:29.607" Score="0" Body="&lt;p&gt;Solid mechanics is the branch of continuum mechanics that studies the behavior of solid materials, and deformation under the action of forces, temperature changes, phase changes, and other external or internal agents.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-03-06T03:38:35.960" LastActivityDate="2017-03-06T03:38:35.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26321" PostTypeId="4" CreationDate="2017-03-05T18:29:29.607" Score="0" Body=" Studies the behavior of solid materials, and deformation under the action of forces" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-03-06T03:38:31.367" LastActivityDate="2017-03-06T03:38:31.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26322" PostTypeId="2" ParentId="26313" CreationDate="2017-03-05T20:30:54.257" Score="4" Body="&lt;p&gt;I was going to write a comment, but the equation seems to view better in answers..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume Von Neumann analysis is the proper approach to derive this equation, but a coordinate transformation from the cartesian CFL condition (I took from wikipedia) is not somehow equivalent? Specifically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \Delta t \sum_{i=1}^3 \frac{u_i}{\Delta x_i} =&#xA;  \Delta t \left( \frac{u_r}{\Delta r} +&#xA;                  \frac{u_{\phi}}{r \Delta \phi} +&#xA;                  \frac{u_z}{\Delta z} \right) &amp;lt; C_{max} \\&#xA;\end{equation}&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2017-03-05T20:30:54.257" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26324" PostTypeId="2" ParentId="26042" CreationDate="2017-03-05T20:36:51.267" Score="1" Body="&lt;p&gt;Let us consider the uncoupled mechanical problem. Hence, we have 10 unknowns &lt;span class=&quot;math-container&quot;&gt;$\{\rho, \mathbf{v}, \underline{\underline{\sigma}}\}$&lt;/span&gt;, assuming that the stress and strains are symmetric (this might not be true in Cosserat solids, for example). And we have 10 equations, namely:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Continuity;&lt;/li&gt;&#xA;&lt;li&gt;Conservation of density of momentum; and&lt;/li&gt;&#xA;&lt;li&gt;Constitutive equations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That are given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;  &amp;amp;\frac{\partial \rho}{\partial t} +  \nabla \cdot \mathbf{\rho v} = 0 \tag1\\&#xA;  &amp;amp;\nabla \cdot \underline{\underline{\sigma}} + \rho \mathbf{b} = \rho \frac{\partial \mathbf{v}}{\partial t} \tag2\\&#xA;  &amp;amp;f_i(\underline{\underline{\sigma}}, \underline{\underline{\epsilon}}) = 0 \quad i=1, \cdots 6 \tag3&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you enforce that displacements are small you are saying that the material and spatial references of your problem are the same, i.e.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{x} = \mathbf{X} + \underbrace{\mathbf{u}}_{\approx 0} \Rightarrow  \mathbf{x} \approx \mathbf{X}\, , \tag4$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thus&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\underline{\underline{F}} = \frac{\partial \mathbf{x}}{\partial \mathbf{X}} \approx \underline{\underline{I}} \Rightarrow |\underline{\underline{F}}| \approx 1\, . \tag5$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a consequence of (5) and (1), you obtain that the density in the initial configuration and the final one is the same, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rho_0 = \rho_t |\underline{\underline{F}}| \approx \rho_t\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the density is not an unknown anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, regarding a Finite Element formulation, when you have an integral of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_{\Omega_t} \underline{\underline{\sigma}}: \underline{\underline{\epsilon}}\, \mathrm{d}\Omega_t\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you face the problem of not-knowing &lt;span class=&quot;math-container&quot;&gt;$\Omega_t$&lt;/span&gt;. When you assume that the displacements are small you are also saying that your initial configuration/domain is the same through your analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This discussion should be present in several continuum mechanics books. That I remember it is presented in&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Olivella, Xavier Oliver, and Carlos Agelet de Saracíbar Bosch. Mecánica de medios continuos para ingenieros. Vol. 92. Univ. Politèc. de Catalunya, 2002.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2020-06-11T20:06:39.483" LastActivityDate="2020-06-11T20:06:39.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="26325" PostTypeId="2" ParentId="23852" CreationDate="2017-03-05T22:05:18.447" Score="4" Body="&lt;p&gt;I'm assuming the limitation in 2D and 3D is storing the Jacobian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option is to retain the time derivatives and use an explicit &quot;pseudo&quot; time-stepping to iterate to steady state. Normally the CFL number you need for diffusive and reactive systems might get prohibitively small. You could try nonlinear multigrid (it's also called Full Approximation Storage multigrid) and local time-stepping to speed up convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other option is to use a fully implicit scheme as you're doing now, but not store the global Jacobian. You could use a matrix-free implicit scheme.&#xA;$$&#xA;DF(u^n)\, \delta u^n = -F(u^n)&#xA;$$&#xA;(where $DF$ is the Jacobian) can be solved with a Krylov subspace solver like GMRES and BiCGStab using the fact that&#xA;$$&#xA;DF(u^n)\, \delta u \approx \frac{F(u^n+\epsilon\frac{\delta u}{\Vert\delta u\Vert}) - F(u^n)}{\epsilon}.&#xA;$$&#xA;This is because GMRES and BiCGStab don't require a LHS matrix $A$, they only need to be able to compute its product $Ax$ given a vector $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now with a proper value of $\epsilon$ (usually about $10^{-7}$ for double-precision floats) you can execute a Newton loop without ever computing or storing a Jacobian. I know for a fact that this technique is used to solve some non-trivial cases in computational fluid dynamics. Note, however, that the number of evaluations of the function $F$ will be more than in a matrix-storage technique, instead of requiring a matrix-vector product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing to note is that if your system is such that a powerful preconditioner is needed (ie. Jacobi or block-Jacobi will not suffice), you might want to try using the above-mentioned method as a smoother in a multigrid scheme. If you want to try a point- or block-Jacobi preconditioner, you could compute and store only the diagonal elements or diagonal blocks of the Jacobian, which is not much. I would also mention that a Gauss-Seidel or SSOR preconditioner may be possible to implement without explicitly storing a Jacobian. &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0021999198960764&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; describes the implementation of a matrix-free GMRES preconditioned with matrix-free symmetric Gauss-Seidel in the context of computational fluid dynamics.&lt;/p&gt;&#xA;" OwnerUserId="5541" LastEditorUserId="5541" LastEditDate="2018-06-16T21:16:57.473" LastActivityDate="2018-06-16T21:16:57.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="26326" PostTypeId="1" CreationDate="2017-03-06T09:00:33.697" Score="1" ViewCount="85" Body="&lt;p&gt;Just for my interest I'm thinking how to make some cheap-and-dirty simulation of processes such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;penetration of metal plate by projectile&lt;/li&gt;&#xA;&lt;li&gt;explosion of &lt;a href=&quot;https://en.wikipedia.org/wiki/Shaped_charge&quot; rel=&quot;nofollow noreferrer&quot;&gt;shaped charge&lt;/a&gt;  (for example &lt;a href=&quot;https://www.youtube.com/watch?v=qqMoFx0uwpo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=qqMoFx0uwpo&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Nuclear_weapon_design#/media/File:Implosion_bomb_animated.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;implosion of atomic bomb core&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I can imagine how to do it in Lagraginan frame ( something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle-in-cell&quot; rel=&quot;nofollow noreferrer&quot;&gt;particle-in-cell&lt;/a&gt; ). But I have feeling that eulerian simulation (just grid, no particles) would be much faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem I don't see how to avoid is &lt;strong&gt;mixing of components&lt;/strong&gt;. When some amount of steel influx into cell full of air, I loose information where was the steel and where was the air (it instantly spread homogeneously over whole cell). It will therefore spread to next cell much faster than if it would be simulated as particles with finite velocity. It means there will be fast diffusion of the steel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;are there any tricks how to avoid this mixing ?&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="4696" LastEditDate="2017-03-07T06:29:29.760" LastActivityDate="2017-03-08T16:27:19.790" Title="Avoid component mixing in eulerian simulation of plastic deformation of solid" Tags="|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26327" PostTypeId="2" ParentId="11217" CreationDate="2017-03-06T09:04:08.477" Score="0" Body="&lt;p&gt;To do it on a plan aligned triangle, you just have to make a loop that fulfils three conditions:&#xA;for i... i++&#xA;if current tested point is on left of line ab, right of line bc and above line ac, point is within triangle. then you just need to use the xy plane  that the triangle is pointing at, which is described by the smallest x,y,z value of the ortho axis, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;use a square containing all 3 points, loop xy points, move y one higher if x goes from true to false if you want to optimize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;its a lot easier to find fairly &lt;strong&gt;random points on a triangle.&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="23512" LastActivityDate="2017-03-06T09:04:08.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26329" PostTypeId="2" ParentId="23536" CreationDate="2017-03-06T16:46:02.237" Score="14" Body="&lt;p&gt;The convergence behavior you are seeing is actually expected. One of &#xA;things that makes the Lanczos method so interesting is that it does&#xA;a good job of simultaneously converging eigenvalues at both ends of &#xA;the spectrum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume your expectation of converging only the largest eigenvalues is&#xA;based on the fact that, as expected from the Power iteration method, the&#xA;last Lanczos vector is getting closer and closer to the eigenvector&#xA;corresponding to the largest eigenvalue. But remember, each computed eigenvector&#xA;is actually the &quot;best&quot; linear combination of all the computed Lanczos vectors.&#xA;And, if you start the Lanczos process with a random vector, this vector is&#xA;as good an approximation to the lowest eigenvector as the largest. That the&#xA;largest eigenvalues do converge somewhat faster than the smallest, however, is&#xA;consistent with what you would expect from Power iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Demmel's book, &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898713897&quot; rel=&quot;noreferrer&quot;&gt;Applied Numerical Linear Algebra&lt;/a&gt;,&#xA;has a nice discussion of&#xA;the convergence properties of the Lanczos algorithm in chapter 7.&#xA;The plot below is computed with Demmel's Lanczos code and reproduces&#xA;figure 7.2 in the book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bEX8B.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bEX8B.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He computes approximate eigenvalues to a 1000 x 1000 matrix using&#xA;increasing numbers of Lanczos vectors. The last column of $+$ signs shows all&#xA;1000 eigenvalues from the full matrix. (the last two columns in this&#xA;plot are essentially the same as your plot above rotated 90 degrees).&#xA;But this plot also shows that the distribution of the eigenvalues across&#xA;the spectrum is not accidental; the eigenvalues at both extremes are&#xA;converging monotonically to the extreme eigenvalues of the full matrix.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-03-06T16:46:02.237" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="26330" PostTypeId="1" AcceptedAnswerId="26344" CreationDate="2017-03-06T22:56:35.047" Score="0" ViewCount="8839" Body="&lt;p&gt;EDIT based on comments below:&#xA;I add the mathematical formulation of my problem below. I am trying to solve an equation of the form&#xA;$$&#xA;\partial_t f(x,y,t)= (\partial^2_x +\partial^2_y) f(x,y,t) \equiv G(x,y,t),&#xA;$$&#xA;discretizing this equation we have&#xA;$$&#xA;f^{k+1}_{i,j}= f^k_{i,j} + \Delta t G^k_{i,j}&#xA;$$&#xA;where $i,j$ refer to discretized spatial coordinates $x,y$ and $k$  corresponds to the iteration step.&lt;br&gt;&#xA;However here $\Delta t $ is a fixed step size, I want to use a line search to find an optimal step size.  Defining $\Delta t \equiv \alpha_k$, I want to find $\alpha_k $ such that $f^{k+1}(i,j) &amp;lt; f^{k}(i,j)-c\alpha_k  G^\top G$ which is a backtracking Armijo line search.  So the equation I am trying to solve is :&#xA;$$&#xA;f^{k+1}_{i,j}= f^k_{i,j}+\alpha^k G^k (i,j)&#xA;$$&#xA;Below is a back tracking line search algorithm to find $\alpha_k$ but it is not being computed correctly I realize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I updated my algorithm based on the comment below however it still seems my stepsize at each iteration, $\alpha_k$ is not being updated properly.  When I print it out it just prints the initial value I inputted for it.  Is this algorithm not updating $\alpha_k$ correctly?  I thought the point of the backtracking line search was to find me an optimal value $\alpha_k$ such that I get to the minimum. How can I fix this?  thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to code the backtracking-Armijo line search algorithm on page 10 here &lt;a href=&quot;https://people.maths.ox.ac.uk/hauser/hauser_lecture2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://people.maths.ox.ac.uk/hauser/hauser_lecture2.pdf&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a sample code for a back tracking line search algorithm .  I can check that the algorithm is not correct but I am not sure where I am going wrong.  A few errors i've realized are possibly the if statement condition and updating alphak at the very end ; lastly I'm not sure what else is wrong  (I am unsure how to fix these problems).  I tried to follow the algorithm in the book but it is not too detailed.  It is very clear there is a problem with the algorithm but I am not sure what.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, the search direction  I choose given by Pk is in the direction of the negative gradient, which I call -g.  Assume below that g(i,j) and fk(i,j) are given at the first iteration, and are 2D arrays since they depend on spatial positions i,j.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integer, parameter :: nx=10,ny=10, k=10&#xA;real, dimension(-nx:nx,-ny:ny) :: fk,fk1,g,gt,Pk&#xA;integer :: i,j,m&#xA;real :: alphak,c,rho !step size at iteration k&#xA;&#xA;c=0.0001&#xA;rho=0.5&#xA;&#xA;do m=1,k&#xA;&#xA;&#xA;        alphak = 1.0&#xA;        Pk(i,j) = -g(i,j) !search direction = -gradient&#xA;        fk1(i,j) = fk(i,j) + alphak*Pk(i,j)&#xA;        gt(i,j) = g(j,i) !transpose of g&#xA;&#xA;           if (fk1(i,j) &amp;gt; fk(i,j)-c*alphak*gt*g) then&#xA;&#xA;                alphak = rho*alphak&#xA;&#xA;                   do j=-ny+1,ny-1&#xA;                   do i=-nx+1,nx-1&#xA;                      fk1(i,j) = fk1(i,j)-alphak*g(i,j)&#xA;                   end do&#xA;                   end do&#xA;&#xA;           end if&#xA;&#xA;    print*, &quot;print out alphak for m=', m&#xA;    print &quot;(//(5(5x,e22.14)))&quot;, alphak&#xA;&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19571" LastEditorUserId="19571" LastEditDate="2017-03-07T20:11:20.923" LastActivityDate="2017-03-09T22:03:53.960" Title="Backtracking-Armijo Line Search Algorithm" Tags="|optimization|algorithms|nonlinear-programming|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26331" PostTypeId="2" ParentId="26326" CreationDate="2017-03-07T03:18:45.143" Score="2" Body="&lt;p&gt;One potential option is the Material Point Method (MPM). It is a successor of the PIC methods that you mentioned, but I do not believe it suffers from the same problems. Implementation is non-trivial, but it might do what you need. It can be described as a &quot;mesh-free&quot; finite element method that stores its information at Lagrangian markers, but updates its motion over the course of a single time step using an ephemeral background finite element grid. I like to plug my friends' work, so I recommend you check out an article from my lab-mate on using MPM for granular flows. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0022509616306901&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0022509616306901&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another potential approach would be to use a level-set function to track your interfaces. My advisor came up with an interesting method a few years back to do nonlinear solid mechanics and fluid-structure interaction on an Eulerian grid. He calls this the &quot;Reference Map&quot; method. It may not be exactly the right thing for you, since the particular way it handles multi-body contact makes it difficult to have topology changes, but it is an interesting take on thermodynamically-consistent nonlinear solid mechanics in an Eulerian frame.&#xA;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0022509612001135&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0022509612001135&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-03-07T03:18:45.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26332" PostTypeId="1" AcceptedAnswerId="26337" CreationDate="2017-03-07T06:56:24.307" Score="4" ViewCount="689" Body="&lt;p&gt;I am solving a flow physics problem, in which I encounter a symmetric boundary. So I set the boundary conditions to be $$\frac{\partial u}{\partial r} =  \frac{\partial v}{\partial r}=\frac{\partial w}{\partial r}=\frac{\partial T}{\partial r} =\frac{\partial \rho}{\partial r}$$ where $u,v,w$ are the 3 components of velocity and $T, \rho$ are temperature and density of the fluid respectively. $r$ is the direction perpendicular to the plane of symmetry. $v$ is the velocity in the $r$ direction. &#xA;Since there will no flow across plane of symmetry, we also get another boundary condition $$v=0$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : Should I take $v=0$ and $\frac{\partial v}{\partial r}=0$  together for $v$ at the boundary or anyone one those will be sufficient?&lt;/p&gt;&#xA;" OwnerUserId="20670" LastActivityDate="2017-03-07T22:03:05.630" Title="Symmetric boundary condition" Tags="|boundary-conditions|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26333" PostTypeId="1" AcceptedAnswerId="26334" CreationDate="2017-03-07T08:00:58.480" Score="1" ViewCount="326" Body="&lt;p&gt;Consider two-body central force problem in polar co-ordinates $r,θ$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Corresponding 2nd order differential equation is obtained by using conservation of angular momentum. This equation is :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ d^2r/dt^2=l^2/(m^2r^3)−GM/r^2 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r(t)$&#xA;is the radial position of particle (of mass $m$) as a function of time $t$. $l$ is angular momentum which is constant. $G$ is gravitational constant and $M$ is mass of the heavier body, assumed to be at rest at the origin of co-ordinate system i.e. at $(r,θ)=(0,0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve above non-linear differential equation; it is non-linear since dependent variable $r$ &#xA;has powers -3 and -2 on RHS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I use 4-order Runge-Kutta method to solve this equation ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extra Note: Actually we have two different 2-order differential equations (coupled) : one for $r$ and another for $θ$. Conservation of angular momentum de-couples them and reduces to one equation given above. Also if we try to solve the above 1-Dim equation analytically, we end up with a solution of the form $t(r)$ i.e. time is function of $r$. So we have to invert that into $r(t)$. This inversion process can be extremely difficult in practice. Please see standard textbook on classical mechanics e.g. by Goldstein (Chapter 3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here Initial conditions should be on $r$ and $dr/dt$, if we want to solve numerically. But if we want to solve analytically we need initial conditions as Total Energy and Angular momentum of the mass $m$. I am confused here: do I need use all initial conditions i.e. energy, angular momentum, $r(t=0)$ and $dr/dt$ at $t=0$ ?&lt;/p&gt;&#xA;" OwnerUserId="16599" LastActivityDate="2017-03-08T04:41:59.570" Title="Can Runga-Kutta method be used to solve non-linear differential equations?" Tags="|computational-physics|differential-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26334" PostTypeId="2" ParentId="26333" CreationDate="2017-03-07T11:31:48.357" Score="5" Body="&lt;p&gt;Runga-Kutta schemes are multistage recipes for numerical discretizations of temporal derivatives. That is they tell to how to solve equations in the form $\dot{\mathbf{y}} = \mathbf{f}(\mathbf(y,t)$.  In order to achieve the advertised accuracy the right hand side should be a sufficiently smooth function of $y$, and nonlinearity can complicate the question of stability for explicit methods like RK4, but otherwise I see no reason RK4 can't be used with a sufficiently small tilmestep. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, as you've described, this is a problem with a Hamiltonian formulation, so there are more appropriate symplectic methods like Stormer-Verlet that are worth looking in to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the question of initial conditions, you should be able to calculate the energy in the system given $r(0)$ and $\dot{r}(0)$, so specifying all three would lead to an over constrained system (i.e. you only need two, but are free to pick any pair). &lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2017-03-07T11:31:48.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26335" PostTypeId="2" ParentId="11217" CreationDate="2017-03-07T13:54:02.343" Score="1" Body="&lt;p&gt;To iterate over the triangle points you could follow this algorithm: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;find the longest edge and use it as direction for the inner loop, in the following it is assumed that the longest edge is (p1,p2) with the third point is p3&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For the inner loop always iterate along the direction D:(p1-&gt;p2) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Iterate along this line starting at point A=p1 with a given step&#xA;length L until you reach B=p2.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;move into direction of point p3 by updating A=p1 + (p3-p1)*L13, and B=p2+(p3-p2)*L23. L13, L23 are evaluated from the step length for the second dimension that is picked based on the distance of p3 to the line (p1,p2). It should be trivial to get the right formula. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;repeat the iteration from A to B by using the same direction vector D&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;continue until all the triangle was covered. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some C++ code that implements this can be found &lt;a href=&quot;https://github.com/gerddie/mia/blob/master/mia/3d/imagedraw.cc&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, look for &lt;em&gt;draw_triangle&lt;/em&gt;, but note that the code also checks whether the points of the line are within a given bounding box, and you would replace &lt;em&gt;draw_point&lt;/em&gt; with whatever action you want to run on the triangle point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Since I am the author of this GPLed code, I hereby grant the license to freely use the parts of filling a triangle in 3D space to be used as blueprint for one's own implementation). &lt;/p&gt;&#xA;" OwnerUserId="23526" LastEditorUserId="23526" LastEditDate="2017-03-08T12:49:16.740" LastActivityDate="2017-03-08T12:49:16.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26336" PostTypeId="1" CreationDate="2017-03-07T20:30:23.863" Score="1" ViewCount="427" Body="&lt;p&gt;I need to use variable precision arithmetic in MATLAB for an expensive set of computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vpa function provided by the symbolic math toolbox is very slow. I found a non-free alternative toolbox known as &quot;Advanpix&quot; (&lt;a href=&quot;http://www.advanpix.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.advanpix.com/&lt;/a&gt;) ,which sped up things by an order of magnitude, using a trial version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my funding body does not allow purchase of any additional software for this project.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a viable/fast alternative to advanpix/vpa for MATLAB ?&lt;/p&gt;&#xA;" OwnerUserId="16900" LastEditorUserId="16900" LastEditDate="2017-03-07T21:54:48.663" LastActivityDate="2017-03-08T04:40:12.613" Title="Any FOSS MATLAB/Octave toolbox for high-speed variable precision arithmetic?" Tags="|matlab|software|floating-point|precision|symbolic-computation|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26337" PostTypeId="2" ParentId="26332" CreationDate="2017-03-07T22:03:05.630" Score="8" Body="&lt;p&gt;For the sake of simplicity let me slightly change your notation. Let $u, v, w$ be the components in the $x, y, z$ directions of a &lt;a href=&quot;http://mathworld.wolfram.com/PolarVector.html&quot; rel=&quot;noreferrer&quot;&gt;true (polar) vector&lt;/a&gt;, and $y=0$ the symmetry plane. For a true vector, symmetry conditions across the $xz$ plane are:&#xA;\begin{eqnarray}&#xA;u(x,y,z) &amp;amp;= u(x, -y, z) \\&#xA;-v(x,y,z) &amp;amp;= v(x, -y, z) \\&#xA;w(x,y,z) &amp;amp;= w(x,-y,z)&#xA;\end{eqnarray}&#xA;which imply&#xA;\begin{eqnarray}&#xA;-\frac{\partial}{\partial y}u(x,y,z) &amp;amp;= \frac{\partial}{\partial y} u(x, -y, z) \\&#xA;\frac{\partial}{\partial y}v(x,y,z) &amp;amp;= \frac{\partial}{\partial y}v(x, -y, z) \\&#xA;-\frac{\partial}{\partial y}w(x,y,z) &amp;amp;= \frac{\partial}{\partial y}w(x,-y,z)&#xA;\end{eqnarray}&#xA;Under a continuity hypothesis up to the first partial derivative, you obtain&#xA;\begin{equation}&#xA;v(x,0,z) = &#xA;\frac{\partial}{\partial y}u(x,0,z) =&#xA;\frac{\partial}{\partial y}w(x,y,z) =&#xA;0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With reference to your problem you should not impose $\frac{\partial v}{\partial r} = 0$ across the symmetry plane, since velocity is a polar vector.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2017-03-07T22:03:05.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26339" PostTypeId="2" ParentId="26336" CreationDate="2017-03-08T04:40:12.613" Score="3" Body="&lt;p&gt;While I do not know of a toolbox which fits the bill, a good open source alternative to MATLAB and Octave which does have a good solution to this problem is Julia. The linear algebra syntax from MATLAB/Octave almost transfers over to Julia directly, though you need to swap indexing like &lt;code&gt;A(i)&lt;/code&gt; to &lt;code&gt;A[i]&lt;/code&gt;. But after a quick translation, you can use Julia's &lt;code&gt;BigFloats&lt;/code&gt; for variable precision numbers. That's a wrapper of the MPFR library which is pretty standard and does okay. A faster implementation for bitsizes &amp;lt;500 is given in ArbFloats.jl which is based on the Arb library. Lastly, if you just want 128-bit numbers, DecFP.jl gives some good implementations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's nice about going this route is that Julia's dispatch system makes these just stand-in for standard numbers, so you can use them in most well written libraries. So you can take these high-speed variable precision numbers and just call generic matrix multiplication, SVD factorization, &lt;code&gt;\&lt;/code&gt;, differential equation solvers, optimization routines, etc. and expect them to work in place of &lt;code&gt;Float64&lt;/code&gt;s. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-03-08T04:40:12.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26340" PostTypeId="2" ParentId="26333" CreationDate="2017-03-08T04:41:59.570" Score="0" Body="&lt;p&gt;Yes. Runge-Kutta techniques like the Dormand-Prince 4/5 method (&lt;code&gt;ode45&lt;/code&gt;) are the standard numerical techniques for solving nonlinear differential equations. Of course there are many more methods available, but all of them are designed to be used on nonlinear equations where no analytical solution exists.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-03-08T04:41:59.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26341" PostTypeId="1" AcceptedAnswerId="26399" CreationDate="2017-03-08T05:37:43.313" Score="1" ViewCount="466" Body="&lt;p&gt;I have a real signal in time given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B0BuZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B0BuZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I am simply trying to compute its power spectrum, &lt;a href=&quot;https://personal.egr.uri.edu/chelidz/courses/mce567/handouts/psdtheory.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;which is the Fourier transform of the autocorrelation of the signal, and is also a purely real and positive quantity in this case&lt;/a&gt;. To do this, I simply write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.fftpack import fft, arange, rfftfreq, rfft &#xA;from pylab import *&#xA;&#xA;lags1, c1, line1, b1 = acorr(((Y_DATA)), usevlines=False, normed=True, maxlags=3998, lw=2)&#xA;Power_spectrum = (fft(np.real(c1)))&#xA;freqs = np.fft.fftfreq(len(c1), dx)&#xA;plt.plot(freqs,Power_spectrum)&#xA;plt.xlabel('f (Hz)')&#xA;plt.xlim([-20000,20000])&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the output gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3HYb0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3HYb0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which has negative-valued output. Although if I simply take the absolute value of the data on the y-axis and plot it (i.e. &lt;code&gt;np.abs(Power_spectrum)&lt;/code&gt;), then the output is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0wbW2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0wbW2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is exactly what I expect. Although why is this only fixed by taking the absolute value of my power spectrum? I checked my autocorrelation and plotted it—it seems to be working as expected and matches what others have computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vV6Zs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vV6Zs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although what appears odd is the next step when I take the FFT. The FFT function outputs negative values which is contrary to the theory discussed in the link above and I don't quite understand why. Any thoughts on what is going wrong?&lt;/p&gt;&#xA;" OwnerUserId="20407" LastActivityDate="2017-03-14T17:45:09.827" Title="Power spectrum incorrectly yielding negative values" Tags="|scipy|numpy|fourier-transform|signal-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26342" PostTypeId="1" CreationDate="2017-03-08T05:39:06.090" Score="3" ViewCount="388" Body="&lt;p&gt;I am leveraging the SLEPc library for solving the first $k$ (where $k = 3$ or $4$) eigenvalues and their corresponding vectors for a matrix of size 200,000. The matrix is sparse and symmetric. &lt;strong&gt;I&lt;/strong&gt; &lt;strong&gt;intend to compute the eigenvalues without using MPI&lt;/strong&gt;, as a result of which the entire computation takes more than 2 minutes. I have tried many eigensolvers like Krylov-Schur, Jacobi-Davidson, &#xA;and Rayleigh Quotient CG but it takes an &lt;strong&gt;&lt;em&gt;awful amount of time to finish computation for each of them.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Lanczos/Krylov-Schur returns the extreme eigenvalues of the spectrum quite fast, on the contrary. It would be awesome if I could replicate such behavior for the interior values of the spectrum. But so far I have not met with any success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any way to accelerate the convergence using a single MPI process?&lt;/strong&gt; Tweaking the tolerance and maximum iterations does not help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far these are what I have tried for a sparse symmetric matrix of size 51309 x 51309: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./RunPart -eps_type gd -st_type precond -st_pc_type ilu -eps_target 0.01 -eps_tol 1e-6 -eps_conv_abs -eps_nev 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This takes 42.76 seconds to produce the 4 eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./RunPart -st_type sinvert -eps_target 0.001 -st_pc_type ilu -eps_tol 1e-5 -eps_conv_abs -eps_nev 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This takes around 13.49 seconds but the eigenvalues contain a significant amount of error. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       k       ||Ax-kx||/||kx||&#xA; -------------- ----------------&#xA;     0.087226       0.0797854&#xA;     0.088502       0.0947138&#xA;     0.089405       0.0702111&#xA;     0.091540       0.0483211&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can some please provide some guidance with this issue? &lt;/p&gt;&#xA;" OwnerUserId="23533" LastEditorDisplayName="user20857" LastEditDate="2017-03-08T20:48:07.520" LastActivityDate="2017-04-07T20:57:10.917" Title="SLEPc eigensolvers take long time to converge for large sparse symmetric matrices" Tags="|linear-algebra|c++|eigenvalues|petsc|preconditioning|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26343" PostTypeId="1" CreationDate="2017-03-08T09:17:29.807" Score="2" ViewCount="566" Body="&lt;p&gt;I am applying a 6th order Finite-Difference differentiation scheme as seen in &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines/example_implementation/dss006&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.scholarpedia.org/article/Method_of_lines/example_implementation/dss006&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to be severe numerical/floating-point issues with this scheme. The result is very sensitive to the values in the vector with which the FD matrix is pre-multiplied with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the gradient of a 10*1 vector with identical values is all zeros. For example, applying the following 10*10 matrix&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   -1764        4320       -5400        4800       -2700         864        -120           0           0           0&#xA;    -120        -924        1800       -1200         600        -180          24           0           0           0&#xA;      24        -288        -420         960        -360          96         -12           0           0           0&#xA;     -12         108        -540           0         540        -108          12           0           0           0&#xA;       0         -12         108        -540           0         540        -108          12           0           0&#xA;       0           0         -12         108        -540           0         540        -108          12           0&#xA;       0           0           0         -12         108        -540           0         540        -108          12&#xA;       0           0           0          12         -96         360        -960         420         288         -24&#xA;       0           0           0         -24         180        -600        1200       -1800         924         120&#xA;       0           0           0         120        -864        2700       -4800        5400       -4320        1764&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to a 10x1 vector &lt;code&gt;25545.007*ones(10,1)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;results in [6.1118e-10,   -2.3574e-09,   -1.266e-09, -3.638e-10, -3.638e-10,    -3.638e-10, -3.638e-10,    2.0082e-09,  -6.1118e-10,  -9.8225e-09]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;whereas the correct answer is a 10x1 zero vector. The result produced by the second derivative matrix on the same set of vectors is even worse, with numbers of the order of 1e3 instead of zeros!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try this same first-derivative FD matrix on a slightly different vector &lt;code&gt;25545.007*ones(10,1)&lt;/code&gt;, this works and produces the right answer (i.e. zero vector). These results have been double-checked on MATLAB, Python(Numpy) and Mathematica using standard double-precision arithmetic. The exact numerical values differ in each of the software, but the order of magnitude remains similiar to the above results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is the scheme so sensitive to a vector that differs only by 1e-3? I really need high accuracy and such numerical issues mean that my solution values in downstream computations become unreliable and doesn't lend confidence in the overall solutions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to implement this? Perhaps there exists an alternative recommended scheme for such scenarios? I am unable to introduce variable-precision arithmetic libraries, due to the project's complexities and their inherent slowness.&lt;/p&gt;&#xA;" OwnerUserId="16900" LastEditorUserId="16900" LastEditDate="2017-03-08T11:37:59.740" LastActivityDate="2017-03-12T05:12:39.533" Title="Precision loss in Matrix-Vector product when applying Finite-Difference scheme" Tags="|linear-algebra|finite-difference|numerical-analysis|precision|accuracy|" AnswerCount="2" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="26344" PostTypeId="2" ParentId="26330" CreationDate="2017-03-08T10:10:52.617" Score="1" Body="&lt;p&gt;[The following references &lt;a href=&quot;http://www.optimization-online.org/DB_FILE/2008/07/2045.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper by Ascher et al.&lt;/a&gt; ...]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You want to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\partial_t f(x,y,t)= (\partial^2_x +\partial^2_y) f(x,y,t),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to some initial and boundary conditions. However, as I understood from your comments, you are in fact interested in the steady state solution (and not in the transient behavior of $f$). This steady state is of course the $f$ that satisfies&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial f}{\partial t} = 0.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You start by choosing a semi-discrete form of your PDE, namely:&#xA;$$&#xA;\partial_t f^h(t) = - \left( A^h f^h - b^h \right),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, if your discrete computational grid is $N \times N$, then $f^h \in R^{N^2}$ is your state vector (one DOF per grid point but reshaped into a column vector), $A^h$ is the discrete Laplacian operator (symmetric and positive definite), and $b^h$ includes forcing terms and boundary conditions (if any). See equations (2a) - (2b) in the paper I linked to for an example finite volume discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, an obvious second step would be to use Euler's (explicit) method to march in time towards the steady state, i.e. go from $t^n$ to $t^{n+1}$ using&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f^{n+1,h}  = f^{n,h} + \alpha \left( b^h - A^h f^{n,h} \right).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially what you're doing here is to take a step $\alpha$ &lt;strong&gt;along the direction of the residual&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r^{n,h} := b^h - A^h f^{n,h}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trouble is, a &quot;default&quot; Euler time step $\alpha$ will be small (it is constrained by numerical stability) so converging to the steady state will take a while with this approach. That's why you want to use a higher time step if possible, and this is where the line search algorithm comes into play. You will essentially try to take the &lt;em&gt;best&lt;/em&gt; step along the residual direction at each iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do that, observe that the Euler discretization above can be interpreted as a gradient descent method for &lt;em&gt;a convex numerical optimization problem&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\min_{f^h} J(f^h) := \frac{1}{2} \left( (f^h)^T A^h (f^h) - (b^h)^T f^h \right).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the cost function $J$ is a &lt;em&gt;scalar&lt;/em&gt;. The necessary condition for a minimum is&#xA;$\nabla_f{^h} J = 0$ i.e. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A^h f^{h} = b^h \leftrightarrow r^h = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence the necessary (and sufficient, since the cost function is convex) conditions give you the steady state of the discrete PDE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Returning to the line search, a natural choice for the $n$-th time step is that who minimizes the &lt;em&gt;cost function $J$&lt;/em&gt; along the residual line, i.e., the solution to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\min_\alpha J(f^{n,h} + \alpha r^{n,h})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives you the &lt;em&gt;steepest descent step&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\alpha = \frac{(r^{n,h})^T(r^{n,h})}{(r^{n,h})^T A^h (r^{n,h})}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what's called an &lt;em&gt;exact line search&lt;/em&gt;. However, minimizing $J$ may not be cost effective for more complicated cost functions. Instead, people have come up with Armijo-type backtracking searches that do not look for the exact minimizer of $J$ along the search direction, but only require sufficient decrease in $J$: you iterate over $\alpha$ until&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;J(f^{n,h} + \alpha r^{n,h}) &amp;lt; J(f^{n,h}) + \alpha \beta (r^{n,h})^T r^{n,h}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some fixed $\beta \in (0,1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other step choices are of course possible (see Ascher's paper). If you're looking for an implementation of steepest descent with Armijo, check out &lt;a href=&quot;http://www4.ncsu.edu/~ctk/darts/steep.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;this MATLAB code&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pseudocode:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$n \leftarrow 0$, $t^n \leftarrow t^0, f^h = f^{0,h} $ the initial conditions, $f$ is a 1D array&lt;/li&gt;&#xA;&lt;li&gt;$r^h = b^h - A^h f^h$ (the residual, also a 1D array)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;while&lt;/strong&gt; $r^h$ not small enough&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(a) run the Armijo iteration using $J$ (as shown in the slides you've referenced); this gives you the time step $\alpha$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b) update the PDE state using the Euler formula $f^{n+1,h} \leftarrow f^{n,h} + \alpha r^{n,h}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(c) update the residual $r^{n+1,h} = b^h - A^h f^{n+1,h}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(d) check if the new residual is small enough i.e. $\| r^{n+1,h} \| &amp;lt; \epsilon$ (for small $\epsilon &amp;gt; 0$, the exact choice of norm and $\epsilon$ is up to you); if so &lt;strong&gt;break&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(e) $t^{n+1} \leftarrow t^n + \alpha$; $t^n \leftarrow t^{n+1}$; $n \leftarrow n+1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;end while&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'd suggest you try implementing this in the programming language you're most familiar with, then port that working implementation to Fortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative technique is &lt;a href=&quot;http://www.cs.odu.edu/~keyes/papers/sinum98.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pseudo-transient continuation.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-03-09T22:03:53.960" LastActivityDate="2017-03-09T22:03:53.960" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26345" PostTypeId="1" AcceptedAnswerId="26346" CreationDate="2017-03-08T12:44:55.690" Score="1" ViewCount="163" Body="&lt;p&gt;In some cases I have seen that if the local error is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Err = O (\Delta t^{p+1})$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the global error is p. So if local error is 3, global will be 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does somebody know where it comes? For instance in the case of multistep solver you can say that regarding &lt;a href=&quot;https://en.wikipedia.org/wiki/Truncation_error_(numerical_integration)&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;, but are there other cases. Is there a simple way to explain such thing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the truncation error is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Err = \Delta t f(t) + O (\Delta t^{3})$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;might I say that my error is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Err =  f(t) + O (\Delta t^{2})$ &lt;/p&gt;&#xA;" OwnerUserId="22125" LastActivityDate="2017-03-09T14:35:09.193" Title="Relationship between global and local error?" Tags="|convergence|error-estimation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26346" PostTypeId="2" ParentId="26345" CreationDate="2017-03-08T15:48:38.793" Score="3" Body="&lt;p&gt;This is not a formal proof, but intuitively you're making a truncation error at each time step of $O(\Delta t^k)$. Your global error will be the error at the time horizon $T = N\Delta t$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means you're taking $N=T/\Delta t$ time steps and making an $O(\Delta t^k)$ error at each of those steps. So global error will be on the order of $(T/\Delta t) O(\Delta t^k) \Rightarrow O(\Delta t^{k-1})$.&lt;/p&gt;&#xA;" OwnerUserId="9728" LastEditorUserId="9728" LastEditDate="2017-03-09T14:35:09.193" LastActivityDate="2017-03-09T14:35:09.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26347" PostTypeId="2" ParentId="26326" CreationDate="2017-03-08T16:27:19.790" Score="1" Body="&lt;p&gt;In terms of purely Eulerian methods for interface capturing multi-material simulations, two current techniques are the level-set approach Tyler Olsen refers to in his answer and Piecewise Linear Interface Capturing (PLIC) for volume of fluid methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the level set approach a scalar field is created which has its zero contour surface lying on the interface. This implicit interface is then displaced by modifying the scalar field under the imposed motion, with material properties at a given point defined by which side of the contour line they lie.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the PLIC method a linear interface is reconstructed in the cells from the available volume fraction data, so that if, for example, a half filled cell lies next to a full cell, the interface passes through the centre of the cell. This reconstruction relies on assumptions about the smoothness of the interface to use information from neighbouring cells in estimating the interface location in a given cell.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2017-03-08T16:27:19.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26350" PostTypeId="1" AcceptedAnswerId="26372" CreationDate="2017-03-08T19:34:13.757" Score="2" ViewCount="689" Body="&lt;p&gt;For a CFD panel-based potential method, I'm trying to reduce the time to solve the linear system. The matrix has the larger values on the diagonal, since the influence of a panel on itself is maximum, and that the panel's influence on other panels decreases at least as the inverse of their distance. The matrix has no reason to be diagonally dominant however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GMRES converges nicely, perhaps due to the diagonal having the larger values. IS there a  preconditioner that would make it even faster? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks, &lt;/p&gt;&#xA;" OwnerUserId="23545" LastActivityDate="2017-03-11T20:31:15.090" Title="Preconditioner for dense matrix &quot;with diagonal predominance&quot;" Tags="|matrix|preconditioning|gmres|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26351" PostTypeId="2" ParentId="26342" CreationDate="2017-03-08T20:29:03.763" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have tried many eigensolvers like Krylov-Schur, Jacobi-Davidson, and Rayleigh Quotient CG but it takes an awful amount of time to finish computation for each of them.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Consider using &lt;a href=&quot;http://slepc.upv.es/documentation/reports/str4.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arnoldi iteration&lt;/a&gt;. It should be included with your SLEPc distribution, and is typically used when only the largest eigenvalues of a sparse matrix are required (or the smallest, if you have an inverse operator available). Also realize that your matrix is somewhat large by most metrics.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Lanczos/Krylov-Schur returns the extreme eigenvalues of the spectrum quite fast, on the contrary. It would be awesome if I could replicate such behavior for the interior values of the spectrum. But so far I have not met with any success.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think this problem might be spurious. If your matrix is symmetric, then computing the second largest eigenvalue (after computing the first) is not, in general, more expensive than computing the first largest. This is because you can simply &lt;a href=&quot;https://math.stackexchange.com/questions/1464162/transforming-matrix-to-remove-a-eigenvalue&quot;&gt;remove any eigenmode&lt;/a&gt; from a symmetric matrix once you have computed it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible that some of your interior eigenvalues are poorly conditioned/highly sensitive. Some discussion on this is detailed &lt;a href=&quot;http://www.math.usm.edu/lambers/mat610/sum10/lecture13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and might prove helpful.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastEditorUserId="-1" LastEditDate="2017-04-13T12:19:23.050" LastActivityDate="2017-03-08T20:29:03.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26352" PostTypeId="2" ParentId="26345" CreationDate="2017-03-08T22:45:33.597" Score="0" Body="&lt;p&gt;This link comes from the Lax Equivalence Theorem and requires that the numerical algorithm is stable. Essentially the Lax Equivalence Theorem is a proof that if the numerical method is stable, then the link between local truncation error and global error that is in the OP must follow. The last step of the proof is just, because you now have a bound on the global error, this implies convergence as $\Delta t \rightarrow 0 $. So along the way it establishes the estimate you're looking for in a pretty general manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way you prove this estimate is to think of the system as a discrete dynamical system, and do substitution $N$ times and collect the error terms, and do some bounds estimating. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-03-08T22:45:33.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26354" PostTypeId="1" AcceptedAnswerId="26357" CreationDate="2017-03-09T08:37:43.760" Score="1" ViewCount="4850" Body="&lt;p&gt;I have a function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;delt=1 #trial&#xA;def f(z):&#xA;    return ((1-2*z)*np.exp(-delt/z))/(((1-z)**(2+delt))*(z**(2-delt)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also have a variable:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.integrate as integrate&#xA;var=integrate.quad(f,0,0.5)[0]  # equals 0.040353419593637516&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I am trying to find the value p such that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;integrate.quad(f,0.5,p)= var&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and manually I can check that it is around 0.605&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I define the following function to be used in optimization:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def integral(p):&#xA;    return integrate.quad(f,0.5, p)[0]-var&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, i get the following results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.optimize as op&#xA;In[26]: op.root(integral,0.61)&#xA;Out[26]: &#xA;    fjac: array([[-1.]])&#xA;     fun: -0.040353420516861596&#xA; message: 'The iteration is not making good progress, as measured by the \n  improvement from the last ten iterations.'&#xA;    nfev: 18&#xA;     qtf: array([ 0.04035342])&#xA;       r: array([ 0.00072888])&#xA;  status: 5&#xA; success: False&#xA;       x: array([ 0.50002065])&#xA;In[27]: op.fsolve(integral,0.61)&#xA;Out[27]: array([ 0.50002065])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any idea why both root and fsolve might be failing?&lt;/p&gt;&#xA;" OwnerUserId="22137" LastActivityDate="2017-03-10T02:59:14.973" Title="scipy optimize fsolve or root" Tags="|optimization|scipy|integration|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26355" PostTypeId="1" CreationDate="2017-03-09T11:46:12.460" Score="3" ViewCount="266" Body="&lt;p&gt;I have successfully solved the multi-species diffusion-reaction equation&#xA;\begin{equation}&#xA;\frac{\partial c_i}{\partial t} = \nabla \cdot (d_i(x)\nabla c_i) + s_i(x,t), \quad \quad (1)&#xA;\end{equation}&#xA;with dicontinuous source term&#xA;\begin{equation}&#xA; s(x,t) =  \left\{ \begin{array}{rcl} s_1(x,t) &amp;amp; \text{for} &amp;amp; 0&amp;lt;x\le x' , \\ s_2(x,t) &amp;amp;\text{for} &amp;amp; x'&amp;lt;x\le 1. \end{array}\right.&#xA;\end{equation}&#xA;In fact, the variable diffusion coefficient also takes slightly different forms in these two region. The equation is discretized with the conservative central scheme&#xA;\begin{equation}&#xA;c_j'(t) = \frac{1}{h^2}\biggl( d(x_{j+1/2})(c_{j+1}(t)-c_j(t))- d(x_{j-1/2})(c_{j}(t)-c_{j-1}(t))\biggr) + s(x_j,t).&#xA;\end{equation}&#xA;I used SUNDIALS package for time integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm trying to solve the problem in a  concentrated solution&#xA;\begin{equation}&#xA;\frac{\partial c_i}{\partial t} = \nabla \cdot \sum_k  D_{ik}(x)\nabla c_k + s(x,t),&#xA;\end{equation}&#xA;where $D_{ik}$ may be negative. In dilute solution approximation (solvent concentration $c_0 \rightarrow \infty$), we have $D_{ik} \rightarrow d_{i}\delta_{ik}$. However, the solution exhibit spurious oscillations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The blue line is sloved with $c_0=10^{20}$ or equation (1); purple and green lines with $c_0=10^{10}$. The difference between purple and green line is the average scheme used for the diffusion coefficient at the cell interface, the former is with simple average and the latter with harmonic average. As shown in the figures below, there are oscillations near the boundaries and the point $x'$. (I should mention that the vary first osscilations began at the left boundary before they built up at the right boundary and the point $x'$).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WvIef.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WvIef.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is odd that even with the oscillations, the scheme with harmonic average never fails to converge at each timestep and follows closely the result of analytical dilution scheme (equation (1)); where as the simple average scheme fails at the one tenth of the time domain. At much later time the solutions look like&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yVvq6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yVvq6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there discretizatons within the finite volume scheme that avoid such oscillations?&lt;/p&gt;&#xA;" OwnerUserId="16998" LastEditorUserId="16998" LastEditDate="2017-03-09T12:01:28.387" LastActivityDate="2017-03-09T12:01:28.387" Title="Spurious oscillations in diffusion-reaction problems with finite volume" Tags="|finite-volume|diffusion|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26356" PostTypeId="1" CreationDate="2017-03-09T12:19:43.900" Score="1" ViewCount="698" Body="&lt;p&gt;I'm trying to better understand finite volume methods and have started coding up a basic script to solve the diffusion equation $$u_t = u_{xx}$$ which has the finite volume form: $$\frac{\bar{u}^{n+1}_i\ - \bar{u}^n_i }{\Delta t} = \hat{F}_{i+\frac{1}{2}}^n - \hat{F}_{i-\frac{1}{2}}^n$$ where $\bar{u_i}$ is the average value of $u$ over cell $i$ and $\hat{F}$ is the numerical approximation to the flux at a cell interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using an ENO approximation in order to reproduce the point values $u$ and the derivative values $u_x$ as well. I'm aware of the ENO method to reproduce the numerical flux at each cell, however, I'm confused on how I can implement no flux boundary conditions using this method. Instead, here's what I've tried:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using my array of known averages, $\bar{u}_i^n$, construct two coefficient matrices. One that approximates the value of the flux, $u_x$, at the left cell interface (which I'll call $L$) and one approximates it at the right cell interface (which I'll call $R$), giving me an equation that looks like $$\bar{u}^{n+1}_i = \bar{u}^n_i - \frac{\Delta t}{\Delta x}(R-L)\bar{u}^n_i$$. This allows me to just make the coefficients of the first row of $L$ and the last row of $R$ all zeros in order to implement no flux boundary conditions. However, when doing it this way, I seem to get something that is obviously non-conservative after I advance one time step:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9AAaR.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9AAaR.jpg&quot; alt=&quot;$u$ after a timestep of .01 with initial condition $u(x,0) = erfc(x)$&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is clearly due to my implementation of the flux conditions, as the interior nodes all give correct values within the prescribed error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="23363" LastEditorUserId="23363" LastEditDate="2017-03-09T12:25:09.233" LastActivityDate="2017-03-09T12:25:09.233" Title="How to impose Neumann boundary conditions in finite volume problems?" Tags="|pde|numerical-analysis|finite-volume|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26357" PostTypeId="2" ParentId="26354" CreationDate="2017-03-09T15:16:22.237" Score="2" Body="&lt;p&gt;As @Stelios mentioned, It seems like the integral from 0.5 to ~0.605 is close to 0, and then it turns negative.The antiderivative of your function is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int f(z; t=1)\, dz = \frac{e^{-\frac{1}{z} - 1}\left[e\, (1-2z)z - e^{1/z}(z-1)^2\operatorname{Ei}\left(\frac{z-1}{z}\right)\right]}{2(z-1)^2}\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;keep in mind that it does not work for $z=0$, indeed, your original function is not defined at $z=0$, and the limit does not even exist. If you make the plot of the antiderivative you can see these features&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3OYk0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3OYk0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-03-10T02:59:14.973" LastActivityDate="2017-03-10T02:59:14.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26358" PostTypeId="2" ParentId="26343" CreationDate="2017-03-09T22:26:24.553" Score="2" Body="&lt;p&gt;If I understood you correctly, you are using the &lt;a href=&quot;http://computation.llnl.gov/projects/sundials/ida&quot; rel=&quot;nofollow noreferrer&quot;&gt;IDA&lt;/a&gt; solver from the LLNL SUNDIALS suite. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have access to the source code that computes the residual of your DAE system, you may then consider automatic differentiation and &lt;a href=&quot;http://computation.llnl.gov/projects/sundials/idas&quot; rel=&quot;nofollow noreferrer&quot;&gt;IDAS&lt;/a&gt;, which is basically IDA with sensitivity analysis capabilities, both forward and adjoint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a &lt;a href=&quot;http://computation.llnl.gov/sites/default/files/public/idas_guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;user's manual&lt;/a&gt; and a comprehensive list of &lt;a href=&quot;http://computation.llnl.gov/sites/default/files/public/idas_examples.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;examples&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to stick with numerical differentiation, i think &lt;a href=&quot;https://mathoverflow.net/questions/64302/numerical-differentiation-what-is-the-best-method&quot;&gt;this na.stackexchange post&lt;/a&gt; is a good starting point.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="-1" LastEditDate="2017-04-13T12:57:55.033" LastActivityDate="2017-03-09T22:39:57.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26359" PostTypeId="1" CreationDate="2017-03-10T03:01:33.753" Score="2" ViewCount="557" Body="&lt;p&gt;I actually find the formulae for peak theoretical performance:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Node performance in GFlops = (CPU speed in GHz) x (number of CPU cores) x (CPU instruction per cycle) x (number of CPUs per node)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CPU Speed and CPU cores are easy. but how can i know the cpu operations per cycle?&#xA;For example, my qualcomm processor specs are as following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CPU Clock Speed Up to 1.2 GHz&#xA;CPU Cores:Quad-core CPU /4x ARM Cortex A7&#xA;CPU Bit Architecture:32-bit&#xA;Can someone please help me with this. &lt;/p&gt;&#xA;" OwnerUserId="23555" LastActivityDate="2017-03-10T03:01:33.753" Title="How To Calculate Theoretical CPU FLOPS?" Tags="|floating-point|" AnswerCount="0" CommentCount="8" ClosedDate="2018-03-20T16:43:45.877" ContentLicense="CC BY-SA 3.0" />
  <row Id="26360" PostTypeId="2" ParentId="26343" CreationDate="2017-03-10T03:20:41.557" Score="0" Body="&lt;p&gt;This could be just a roundoff error issue. Each row of a differentiation matrix must sum to zero. Subtracting mean value of vector gives better results&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = 25545.007*numpy.ones((10,1))&#xA;A.dot(x)&#xA;array([[  0.00000000e+00],&#xA;       [ -7.45058060e-09],&#xA;       [  0.00000000e+00],&#xA;       [ -1.01863407e-10],&#xA;       [ -1.01863407e-10],&#xA;       [ -8.14907253e-10],&#xA;       [ -9.31322575e-10],&#xA;       [ -3.72529030e-09],&#xA;       [  3.72529030e-09],&#xA;       [  1.49011612e-08]])&#xA;xa = x.mean()&#xA;y = x - xa&#xA;A.dot(y)&#xA;array([[ 0.],&#xA;       [ 0.],&#xA;       [ 0.],&#xA;       [ 0.],&#xA;       [ 0.],&#xA;       [ 0.],&#xA;       [ 0.],&#xA;       [ 0.],&#xA;       [ 0.],&#xA;       [ 0.]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another option would be this. If the derivative is given by&#xA;$$&#xA;v = Du, \qquad v_i = \sum_{j=1}^N D_{ij} u_j&#xA;$$&#xA;then compute as&#xA;$$&#xA;v_i = \sum_{j=1}^N D_{ij} (u_j - u_i), \qquad 1 \le i \le N&#xA;$$&#xA;I expect this to reduce round-off errors.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2017-03-12T05:12:39.533" LastActivityDate="2017-03-12T05:12:39.533" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26361" PostTypeId="1" CreationDate="2017-03-10T08:32:57.677" Score="2" ViewCount="123" Body="&lt;p&gt;I am currently working on a project where a two-phase flow is considered. The phases are described using a level set approach and a signed distance function from the interface between the phases where positive values are liquid and negative values are gas. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I consider a Stoke's flow which gives me a velocity field $v$ and a pressure field $p$. Then I use $v$ to advect the level set using the advection equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial \phi}{\partial t} + v \cdot\nabla\phi=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to this type of problem but I think I have some basic understanding on the numerical (diffusion) problems arising from this equation due to discretization and I know that I need some kind of stabilization. My research concerns the modelling of two-phase flow in a porous material and the results needs to be quite accurate, hence the popular SUPG scheme does not quite cut it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is simply, what schemes (for Finite Elements) exists, that gives a better solution than the SUPG scheme? And furthermore, is the any good literature on the topic that discuss, and preferably, compares any stabilization schemes?&lt;/p&gt;&#xA;" OwnerUserId="6646" LastActivityDate="2017-03-10T08:32:57.677" Title="Numerical diffusion in during advection of a free surface in an FE context" Tags="|finite-element|advection|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26362" PostTypeId="2" ParentId="21343" CreationDate="2017-03-10T11:08:24.410" Score="3" Body="&lt;pre&gt;&lt;code&gt;Eigen::SparseLU&amp;lt;Eigen::SparseMatrix&amp;lt;double&amp;gt; &amp;gt; solverA;&#xA;A.makeCompressed();&#xA;solverA.analyzePattern(A);&#xA;solverA.factorize(A);&#xA;&#xA;if(solverA.info()!=Eigen::Success) {&#xA;    std::cout &amp;lt;&amp;lt; &quot;Oh: Very bad&quot; &amp;lt;&amp;lt;&quot;\n&quot;;&#xA; }&#xA;else{&#xA;     std::cout&amp;lt;&amp;lt;&quot;okay computed&quot;&amp;lt;&amp;lt;&quot;\n&quot;;&#xA; }&#xA;Eigen::VectorXd solnew = solverA.solve(b);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You don't need to mention the COLAMDOrdering but if you want you can declare the solver as :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigen::SparseLU&amp;lt;Eigen::SparseMatrix&amp;lt;double&amp;gt;, Eigen::COLAMDOrdering&amp;lt;int&amp;gt; &amp;gt; solver;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;PS: More info about COLAMDOrdring can be found &lt;a href=&quot;http://eigen.tuxfamily.org/dox-devel/classEigen_1_1COLAMDOrdering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23023" LastEditorUserId="1804" LastEditDate="2017-03-10T11:35:24.173" LastActivityDate="2017-03-10T11:35:24.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26363" PostTypeId="1" CreationDate="2017-03-10T14:38:44.720" Score="1" ViewCount="222" Body="&lt;p&gt;I am referring &lt;a href=&quot;http://www.rle.mit.edu/dspg/documents/SynchroofLorenz.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;  paper, and trying to implement chaotic sender and receiver, to decode message, as given in section $V^{th}$ Chaotic signal masking.&#xA;The process that I want to implement is figure 6 of the same paper, where we have Lorenz system given by equations:&#xA;:  \begin{align}&#xA;\frac{\mathrm{d}x}{\mathrm{d}t} &amp;amp;= \sigma (y - x), \\&#xA;\frac{\mathrm{d}y}{\mathrm{d}t} &amp;amp;= x (\rho - z) - y, \\&#xA;\frac{\mathrm{d}z}{\mathrm{d}t} &amp;amp;= x y - \beta z.&#xA;\end{align}  [from Wikipedia]and $\sigma=16; \rho=45.6; \beta=8$ for real applications the parameters have been scaled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My MATLAB code is given here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;&#xA;close all;&#xA;clc;&#xA;%Solution for the Lorenz equations in the time interval [0,100] with initial conditions [1,1,1].&#xA;sigma=16;&#xA;beta=4;&#xA;rho=45.6;&#xA;f = @(t,a) [-sigma*a(1)/10 + sigma*a(2)/10; rho*a(1)/10 - a(2)/10 - (a(1)*a(3))/200; -beta*a(3)/20 + (a(1)*a(2))/100];&#xA;%'f' is the set of differential equations and 'a' is an array containing values of x,y, and z variables.&#xA;%'t' is the time variable&#xA;[t,a] = ode45(f,[0 10],[1 1 1]);%'ode45' uses adaptive Runge-Kutta method of 4th and 5th order to solve differential equations&#xA;% plot3(a(:,1),a(:,2),a(:,3))&#xA;m_t= rand(1,size(a,1))&amp;lt;=0.5;&#xA;s_t=a(:,1)+ m_t';&#xA;f = @(t,a_r) [-sigma*a_r(1)/10 + sigma*a(1)/10; rho*a_r(1)/10 - a(1)/10 - (a_r(1)*a(1))/200; -beta*a(1)/20 + (a_r(1)*a(1))/100];&#xA;[t,a_r] = ode45(f,[0 10],[1 1 1]);&#xA;m_t_est=s_t(1:size(a_r,1),:)-a_r(:,1);&#xA;l=m_t(:,1:size(a_r,1))'-m_t_est;&#xA;plot(l)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;part of this code has been taken from Wikipedia: Lorenz system, and modified suitably, my problem is twofold:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I am not sure as how this ODE is working as integrating from 1 to 10 the size I am getting is like size(a_r(:,1))=61. How to decide the size of given function?&lt;/li&gt;&#xA;&lt;li&gt;The plot of difference between received waveform and transmitted is like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/qppva.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qppva.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;which is quite fine, for some samples but increases dramatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I doing this the right way or something is really wrong?&lt;/p&gt;&#xA;" OwnerUserId="23559" LastEditorUserId="23559" LastEditDate="2017-03-10T15:24:00.813" LastActivityDate="2017-03-10T15:24:00.813" Title="How to implement chaotic sender and receiver with ordinary differential equations?" Tags="|matlab|ode|chaotic-systems|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26365" PostTypeId="1" CreationDate="2017-03-10T18:49:51.203" Score="4" ViewCount="2149" Body="&lt;p&gt;There seem to be many books and papers that explain various CFD methods in great detail, but unfortunately I have not been able to find many good resources of such methods implemented in codes such as MATLAB or Python (besides the most basic of problems such as the linear advection equation). As such, I was wondering if there are any books out there that I could purchase that come with many coded examples, or if there are any free online resources that show one how to solve various CFD problems. I realise that CFD can be a very difficult field, especially when dealing with complex geometries. However, if I could find a resource that, for example, gives step-by-step instructions on how to solve the 2D Euler equations (with very simple geometry), which seems to be the logical step up when moving from scalar to vector equations, this would be a great starting point on how to learn more advanced topics. Unfortunately, I learn by example, and can get bogged down by too much theory. As such, having code examples to work through and understand alongside the theory would be a big help. Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="21148" LastActivityDate="2017-03-17T01:09:31.377" Title="Practical coding resources for new CFD practicioners" Tags="|matlab|fluid-dynamics|python|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26366" PostTypeId="2" ParentId="26257" CreationDate="2017-03-10T19:36:10.047" Score="1" Body="&lt;p&gt;After some searching I found a C/C++ function called project() in the article&lt;br&gt;&#xA;'Real-Time Fluid Dynamics for Games' by Jos Stam&lt;br&gt;&#xA;&lt;a href=&quot;http://www.dgp.toronto.edu/people/stam/reality/Research/pdf/GDC03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dgp.toronto.edu/people/stam/reality/Research/pdf/GDC03.pdf&lt;/a&gt;&lt;br&gt;&#xA;I had to do a little bit of editing, such as increasing iteration variable &lt;code&gt;k&lt;/code&gt; (dependent on how divergent your input field is) and employing periodic boundaries. After that it worked for me and solved my problem. The article also contains a fairly intuitive description of what the function outputs without going into the details of how it does this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found the reasonably small Python software package windspharm can extract a solenoidal field&#xA;&lt;a href=&quot;http://ajdawson.github.io/windspharm/api/windspharm.cdms.html#windspharm.cdms.VectorWind.nondivergentcomponent&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ajdawson.github.io/windspharm/api/windspharm.cdms.html#windspharm.cdms.VectorWind.nondivergentcomponent&lt;/a&gt;&lt;br&gt;&#xA;I did not test this one though.&lt;/p&gt;&#xA;" OwnerUserId="23386" LastActivityDate="2017-03-10T19:36:10.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26367" PostTypeId="1" AcceptedAnswerId="26378" CreationDate="2017-03-11T07:02:13.850" Score="2" ViewCount="64" Body="&lt;p&gt;I recently coded a 1 stage and 3 stage optimal TVD-RK explicit scheme using eqn 3.3 here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ams.org/journals/mcom/1998-67-221/S0025-5718-98-00913-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ams.org/journals/mcom/1998-67-221/S0025-5718-98-00913-2/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;on the equation Ux+Uy=0, where x and y are the independent variables with which U velocity is differentiated. It is a 1D problem. The space is discretized using the upwind scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am running it for 100 mesh size, and a constant declared CFL of 1. I see that as the solution progresses to t=1s for TVD-3, the solution disperses in both value and shape very fast, while for TVD-1 it gives perfect shift.&#xA;My question is-is this supposed to happen? I do not think changing the time accuracy to 3 stages should necessarily affect the shape/values of the function to such a huge degree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I not, then I think I do not understand the implementation of TVD. I did it exactly as described in the paper. Except I am not sure if I should have CFL different for each stage, or if the CFL should be multiplied by (1/(2p+1)) where p is the order of the polynomial.&#xA;Can you guys help me?&lt;/p&gt;&#xA;" OwnerUserId="23565" LastActivityDate="2017-03-12T15:25:51.160" Title="Can 3rd order TVD admit perfect shift for Upwind 1D Advection equation?" Tags="|fluid-dynamics|computational-physics|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26368" PostTypeId="2" ParentId="26365" CreationDate="2017-03-11T08:27:50.720" Score="3" Body="&lt;p&gt;SUPER LONG POST HERE, I WISH I COULD TL;DR IT, BUT IT WOULD BE PRETTY USELESS&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming you are an undergrad, in engineering, because if you are a grad, you should have taken the CFD course in the respective department first. If you are in math, I don't know how to relate to your knowledge of things in classical fluid dynamics.Moving on...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BACKGROUND FOR ME&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the position you are in quite well, I was myself stuck in the same bog for quite some time and things never really came to me theoretically, and as far as I have experienced, reading a book like J.D. Anderson or Versteeg doesn't really help much. And, I feel it is rather insensitive to ask someone to read J.D Anderson cover to cover, even the first few chapters, and only then allow him to start real CFD coding. During my time as an eager undergrad trying to learn CFD I spent several dorm nights scanning the internet to find decent tutorials on CFD, scouring CFD online and reddit. Some even wanted me to read &lt;em&gt;the entire&lt;/em&gt; JDA &lt;strong&gt;and&lt;/strong&gt; Versteeg before starting coding(which is ridiculous), and I couldn't make jack of it, and got really frustrated. I still couldn't easily make jack of half the things in those books, and I am in my 2nd advanced CFD course in grad school. I hope my answer here outlines a better way for the next generation to start coding in CFD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;SCREW 2D!!!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First things first, if you are an undergrad student and not something of a genius, it will be very hard to battle CFD and the regular coursework together. &lt;em&gt;I suggest you forget 2D and stick to 1D codes for now&lt;/em&gt;. CFD is pretty hard and you definitely need a grad-level course to start understanding the nuances of finite volume method and finite element method in 2D. I suggest firmly, firmly sticking to 1D, and learning the different numerical methods in Finite Differencing, like Lax schemes(normal, Friedrich, Wendroff, etc.), central, upwind differencing schemes and time discretizing schemes. If you need examples, just google something like &quot;Lax Friedrich 1D Advection C++ code simple&quot; and try and match the theory in those books with the code, understand how they work, and why they work the way they do. Do error analysis(Von Neumann, and multi-mode too). There is a world of things to be learnt in 1D, you will never run out of things. Most importantly, to an undergrad, they are easy, logical and indispensable while building concepts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest trying the inviscid, incompressible, linear transient advection equation, and then the transient Heat equation, playing around with different boundary conditions(dirichlet, neumann, periodic, Robin) and ghost cells. If that is done, you can try to move on to compressible 1D Euler, but the above two should be done perfectly and rigorously, including the theory. This will help you build up a base perfectly for a grad school course, where you can then relax and focus on getting your 2D concepts right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are not sure about anything, just Google. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BACKGROUND FOR ME #2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you an idea, I did two 2D Navier-Stokes projects which basically included like 8 codes last semester. I then thought I was sorted for CFD, and then half-way into the advanced CFD course this sem I am still doing 1D codes. This is how important and emphasized getting a firm hand on 1D is. Don't be disappointed by the lack of lovely colors and shapes, they will come. :D&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;SCREW 2D #2!!!!!!!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you did all of this to your satisfaction, you can try constructing the 2D Poisson solver. You can usually find sample codes and a simple sample grid on the internet. If not, you can always make your own square grid uniform and solve on it. It is THE starting point for coding any NS equations. You can learn the different iterative schemes like ILU, Point Jacobi and stuff. You can find easy samples of &lt;em&gt;everything&lt;/em&gt; on the internet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forget about 2D Euler for now, as it's main flavor comes when you attempt compressible flows, which have some tricky physics in them. Learning numerical schemes now will make it much easier for you to understand 2D Euler in the future. You can do 2D with Finite differences, but that would be pretty simplistic, useless and cumbersome for the most part if you are not a master at 1D. Most modern 2D codes use Finite Volume or Discontinuous Galerkin methods today, which are very advanced in terms of formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;POINTS TO CONSIDER&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to conclude by giving a few more pointers which might help you realistically start CFD&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) Try learning commercial codes first and doing internships in places which use them. They will give you an idea of what to expect, and form an excellent educational tool for understanding the elements and workflow that go into a CFD problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) Switch to C++ or Fortran ASAP if you are even dreaming about 2D codes. My professor last sem told the class about a student senior to me who wrote his code in Python, and it took a week for him to converge the final project which took just a handful of minutes for us in C++ and Fortran. He couldn't turn his whole project in and got a bad grade. Matlab is equally bad at scicomp. Learn to  output your data in a .dat file and use Tecplot or Origin to plot graphs and contours. Between C++ and Fortran, doesn't really matter much at your stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.) Again, I cannot stress this enough, DO NOT GO FOR 2D. It is cumbersome and irrelevant to your growth in CFD at this stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4.) If you do not plan on doing grad studies, forget about CFD coding. It will take you years to master a CFD sub-area, and doing it on a commercial code will take you a day or two tops. The codes are designed to be idiot-proof, and the results are pretty good too. It will be much more useful in non-fundamental-scicomp-research engineering teams if you understand the basics of FM and HT properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5.) CFD is going to kick your ass, and you will love it. I haven't slept more than a total of 4 hours in 3 consecutive days, and this is in the middle of my spring break(no shit). Be prepared for it's learning curve, but it is guaranteed to be worth it when you generate your own colorful pile of garbage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ON THE 12 STEPS COURSE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can try doing the 12 steps thing as a hobby, but in hindsight it was a useless, self-congratulatory, overrated endeavor in terms of trying to gain insight into real CFD, since most of it is just too well-structured and simplistic. It's a great course, I admit, but it is almost too good to the point of being ultimately useless, if you know what I mean. The professor is very cogent and smart, but goes too easy on the topics. I know a lot of people here swear by her course and might get pretty offended with me, but I think quite a few who have taken a grad course in CFD might agree with me too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;LEARN THIS BY HEART!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An Ode to CFD&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By Steve Bova and Alfred Lorber&#xA;University of Texas CFD Laboratory&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Old codes never die&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They get handed down to you and I&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If yours does not have much robosity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need only add more viscosity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many a knob and several a switch&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are required to run it without a hitch&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If higher flop rates you must realize&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply generate a larger grid size&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a trivial modification your manager seeks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My estimation is at least ten weeks.&lt;/p&gt;&#xA;" OwnerUserId="23565" LastEditorUserId="23565" LastEditDate="2017-03-11T09:01:02.540" LastActivityDate="2017-03-11T09:01:02.540" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26369" PostTypeId="1" CreationDate="2017-03-11T11:43:39.703" Score="5" ViewCount="2056" Body="&lt;p&gt;I am having some serious difficulties trying to understand how to use (apply) CG to minimize a function. In all the textbooks and notes, the step size $\alpha$ is give by the following expression&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \alpha _k = - \frac{r_k ^T p_k}{p_k ^T A p_k}, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $r$ is the residual and $p$ the search direction. The problem is that I am not trying to find the solution of a linear system, so I do not have a matrix $A$. How can I use CG to minimize a function $f(x,y)$? Can I use a fixed $\alpha$ or implement the &lt;em&gt;Wolfe sufficient condition&lt;/em&gt;? I used a fixed step size to minimize the following function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x,y) := -\cos(x) \cos(y) \exp(-(x-\pi)^2 - (y-\pi)^2)) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but CG does dot converge to one of the minimizers even if the starting point is close &lt;em&gt;enough&lt;/em&gt; and using a step size of $1.0$. Is this possible even with CG?&lt;/p&gt;&#xA;" OwnerUserId="22185" LastEditorUserId="9667" LastEditDate="2017-08-31T18:46:20.413" LastActivityDate="2017-10-02T07:01:33.203" Title="Conjugate gradient method to minimize a function" Tags="|optimization|conjugate-gradient|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26370" PostTypeId="1" AcceptedAnswerId="26376" CreationDate="2017-03-11T13:23:50.367" Score="1" ViewCount="1511" Body="&lt;p&gt;I'm writing a matrix library in c++. After some debugging I found that a simple double difference is not zero for two &quot;equals&quot; numbers. This is due how double are represented in a computer of course. So my question is, what I could use to improve the numeric stability about subtracting matrix (or vectors) and possibly other basic operations? I tried to use long double but I have the same problem. The same algorithm in MATLAB works so MATLAB must use some trick about internal number representation.&lt;/p&gt;&#xA;" OwnerUserId="23567" LastEditorUserId="1804" LastEditDate="2017-03-12T12:34:04.647" LastActivityDate="2017-03-13T08:57:50.597" Title="Improve numeric stability of subtraction in C++" Tags="|c++|stability|floating-point|" AnswerCount="1" CommentCount="4" ClosedDate="2017-03-12T17:13:20.970" ContentLicense="CC BY-SA 3.0" />
  <row Id="26371" PostTypeId="1" AcceptedAnswerId="26640" CreationDate="2017-03-11T16:41:54.940" Score="6" ViewCount="227" Body="&lt;p&gt;I wish to compute the derivative of a vector through numerical differentiation. Let's say, we use a standard 2nd order central difference scheme, to arrive at a differentiation matrix, and apply it on a numerical vector, $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u' = D u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the top and bottom few elements of the resulting derivative vector, I get spurious numerical round-off even by using complicated stencil. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if I remove the mean from every element of my vector, and then pre-multiply it with my differentiation matrix $D$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\tilde u' = D (u - mean(u))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it seems to get better results (i.e. better accuracy), although not fully accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did I happen to get lucky by chance for this specific vector I am working with, or is there a fundamental mathematical reason why the same differentiation matrix performed remarkably on the mean-removed input vector ?&lt;/p&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2017-04-30T12:10:58.690" Title="Does mean removal increase accuracy of numerical differentiation?" Tags="|linear-algebra|finite-difference|numerical-analysis|floating-point|numerical-limitations|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="26372" PostTypeId="2" ParentId="26350" CreationDate="2017-03-11T20:31:15.090" Score="2" Body="&lt;p&gt;In case it may be of help to others, here's the result of the testing on a 30x30 influence matrix.It was made using non-restarted GMRES.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4T2s1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4T2s1.png&quot; alt=&quot;GMRES convergence with preconditioners&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GMRES without preconditioner (red curve) converges in 13 iterations, with a sudden convergence at iteration 12.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GMRES with symmetric Gauss-Seidel (dashed curve) converges more smoothly, but in a little more iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ILU preconditioners are incomplete LU factorizations of the matrix, where only a band of variable width below and above the diagonal has been kept, i.e. Abs(i-j)&#xA;&#xA;&lt;p&gt;Bandwidth 2, black curve : smooth convergence, in roughly the same number of iterations as the non-preconditioned GMRES&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bandwidth 5 and 10 are the yellow and green curves : convergence increases with the bandwith.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The curves with icons are ILU with bandwidth 10, where the preconditiong is only applied on one side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all, the ILU applied on the left and right sides seem to be the most promising preconditioners. It remains to be tested at full scale for large matrices.&lt;/p&gt;&#xA;" OwnerUserId="23545" LastActivityDate="2017-03-11T20:31:15.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26376" PostTypeId="2" ParentId="26370" CreationDate="2017-03-12T10:43:31.503" Score="5" Body="&lt;p&gt;&lt;em&gt;This is comment, but too long for comment so I write here. I think is better you edit the post with more information, now I explain why.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/21483/how-to-avoid-the-round-off-errors-in-the-larger-calculations/26000#26000&quot;&gt;Here&lt;/a&gt; you can find some general advice to improve precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem you find I think it's possible that it come from the floating point representation and the relative effect in the machine operation.&#xA;When you do an add or a subtraction the machine operation are these:&#xA;$$&#xA; x \oplus y = \mathbf{fl}(\mathbf{fl}(x) + \mathbf{fl}(y))\\&#xA; x \ominus y = \mathbf{fl}(\mathbf{fl}(x) - \mathbf{fl}(y))&#xA;$$&#xA;where $\mathbf{fl}(x)$ is the floating point number of $x$.&#xA;You find that the error estimates are:&#xA;$$&#xA;\text{err}_{x,y}^{\oplus} \quad \alpha \quad \left| \frac{x}{x+y} \right|\text{err}_x + \left| \frac{y}{x+y} \right|\text{err}_y \\&#xA;\text{err}_{x,y}^{\ominus} \quad \alpha \quad \left| \frac{x}{x-y} \right|\text{err}_x + \left| \frac{y}{x-y} \right|\text{err}_y\\&#xA;\text{whit} \quad \text{err}_x = \frac{| x - \mathbf{fl}(x) |}{|x|}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that if $x+y \approx 0$ in the sum or  $x-y \approx 0$ in the subtraction you can have got a accuracy loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principal trick is to avoid potential dangerous subtraction/sum and reformulate the algorithm in numerical stable version.&#xA;The typical example, that you can find in many textbook, is about the roots of a second degree polynomial.&#xA;You can write the polynomial $ax^2 + bx + c$ as $x^2 + 2px - q$.&#xA;In this form, if $\sqrt{p^2 + q} \geq 0$, the roots are:&#xA;$$&#xA;y = -p \pm \sqrt{p^2 + q}&#xA;$$&#xA;Now if $p &amp;gt;&amp;gt;q$ one of the root is potentially dangerous because&#xA;$\sqrt{p^2 + q} \approx p$ and you obtain a dangerous subtraction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is to use a different formulation:&#xA;$$&#xA;y = -p + \sqrt{p^2 + q} =&#xA;\frac{(-p + \sqrt{p^2 + q}) (p + \sqrt{p^2 + q})}{(p + \sqrt{p^2 + q})}  = &#xA;\frac{q}{(p + \sqrt{p^2 + q})}&#xA;$$&#xA;This is a stable version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Maybe the matlab code uses a more stable version of the algorithm. Try to see if the problem might be due to this (of course, each case must be treated separately) and edit the question. Anyway as @Wrzlprmft suggested edit the question.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Reference&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I report a couple of references. In [1] there is chapter 1 about floating point with error propagation, there is also the above example.&#xA;The book [2] is a good and complete reference about accuracy and stability for many intentions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;em&gt;Stoer, J.; Bulirsch, R.&lt;/em&gt;, Introduction to numerical analysis. Translated from the German by R. Bartels, W. Gautschi and C. Witzgall., Texts in Applied Mathematics. 12. New York: Springer-Verlag. xiii, 660 p. (1993). &lt;a href=&quot;https://zbmath.org/?q=an:0771.65002&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL0771.65002&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;em&gt;Higham, Nicholas J.&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1137/1.9780898718027&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Accuracy and stability of numerical algorithms.&lt;/strong&gt;&lt;/a&gt;, Philadelphia, PA: SIAM. xxx, 680 p. (2002). &lt;a href=&quot;https://zbmath.org/?q=an:1011.65010&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL1011.65010&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.797" LastActivityDate="2017-03-13T08:57:50.597" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26378" PostTypeId="2" ParentId="26367" CreationDate="2017-03-12T15:25:51.160" Score="3" Body="&lt;p&gt;Yes, this is the expected behavior.  If you plug CFL=1 into most 1st-order schemes, you'll see that they give the exact solution.  If you do the same for most higher-order schemes, you'll see that they don't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the purpose of these schemes is not to solve the linear, scalar advection equation in 1D.  For more complicated situations (systems, multi-D, nonlinear), you can't impose a CFL number of 1 for all waves everywhere, so the fact above is irrelevant.  If you run both schemes for the advection equation at a CFL number significantly smaller than 1, you'll see that the high-order scheme is more accurate.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-03-12T15:25:51.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26380" PostTypeId="1" CreationDate="2017-03-12T19:07:45.933" Score="1" ViewCount="60" Body="&lt;p&gt;Both algebraically or with software (numerically/computationally, for example) is acceptable. Here is a how one of the polynomials looks like ($L&amp;lt;1$ is a constant).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have attached a text file with all the 13 polynomials that I need to sort in ascending order - more precisely, I have a target order I need to achieve, so I need to define the ranges of the variables $h_0$ - $h_4$ where that order would be valid. Here is one representative polynomial sample:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;7 h_0 h_1^2 h_2^2 h_3 L^3 + &#xA;29 h_0 h_1^2 h_2^2 h_3 h_4^2 L^4 + &#xA;3 h_0 h_2^2 h_4 L^2 + &#xA;5 h_0 h_2^2 h_4^3 L^3 + &#xA;\ldots&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each polynomial has approximately 200 terms(!). &lt;/p&gt;&#xA;" OwnerUserId="23578" LastEditorUserId="17294" LastEditDate="2017-03-12T19:29:18.973" LastActivityDate="2017-03-12T19:29:18.973" Title="How to sort 13 enormous polynomials, each having terms with 5 variables ($1&gt;h_0&gt;h_1&gt;h_2&gt;h_3&gt;h_4$)?" Tags="|polynomials|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26383" PostTypeId="1" CreationDate="2017-03-13T06:32:35.337" Score="3" ViewCount="1159" Body="&lt;p&gt;Consider the 1D advection equation in its strong and weak forms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ u_t + a u_x = 0 $$&#xA;$$\  \int_{x_{j-0.5}}^{x_{j+0.5}}   w u_t \ dx  - a \int_{x_{j-0.5}}^{x_{j+0.5}} w_x u \ dx + a [w(x)\hat{u}]_{x_{j-0.5}} ^{x_{j+0.5}} = 0  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having introduced the piecewise constant Lagrange polynomials or the first two Legendre Polynomials as our test functions, the system can be written as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{bmatrix} \int (\phi_1)^2 dx \ \int \phi_1 \phi_2 dx \\ \int \phi_2 \phi_1 dx \ \int (\phi_2)^2 dx  \end{bmatrix} \begin{bmatrix} u_1 \\ u_2 \end{bmatrix}_t  - a \begin{bmatrix} \int (\phi_1)_x \phi_1 dx \ \int (\phi_1)_x \phi_2 dx \\ \int (\phi_2)_x \phi_1 dx \ \int (\phi_2)_x \phi_2 dx  \end{bmatrix} \begin{bmatrix} u_1 \\ u_2 \end{bmatrix} + a \begin{bmatrix} \phi_1(x_{j-0.5}) \ \phi_1(x_{j+0.5}) \\ \phi_2(x_{j-0.5}) \ \phi_2(x_{j+0.5})  \end{bmatrix}  \begin{bmatrix} -\hat{u}_{x_{j-0.5}} \\ \hat{u}_{x_{j+0.5}} \end{bmatrix} = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When solving through the use of the Lagrange polynomial basis, the values $ u_1 $ and $ u_2 $ that are being solved for are the values of the function $ u(x,t) $ at the nodes.&#xA;In the case of the Legendre Polynomials, what are we solving for? What is the physical significance of $ u_1 $ and $ u_2 $ and how do they relate to the function values at the nodes, cell average and flux ?&lt;/p&gt;&#xA;" OwnerUserId="23585" LastEditorUserId="9667" LastEditDate="2017-03-13T17:12:29.000" LastActivityDate="2022-10-26T18:34:47.807" Title="1D Discontinuous Galerkin - Lagrange vs Legendre Basis" Tags="|finite-element|discontinuous-galerkin|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26385" PostTypeId="1" CreationDate="2017-03-13T10:46:11.523" Score="2" ViewCount="447" Body="&lt;p&gt;I'm trying -to no avail- to &lt;a href=&quot;https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;broadcast&lt;/a&gt; the following piece of code in &lt;code&gt;Python&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.linalg import expm&#xA;&#xA;def ExactSol(c, omega0, x0, v0, T, p):&#xA;    # Docstring&#xA;    &quot;&quot;&quot;&#xA;    Returns the solution of&#xA;    x'' + c x' + w0^2 x = 0&#xA;    embodied with initial condition x(0) = x0 and x'(0) = v0&#xA;    i.e. the function X(t) = exp(tA) * X0&#xA;    with X = (x, v), X0 = (x0, v0), A = (0       1)&#xA;                                        (-w0^2  -c)&#xA;    over [0,T] cut in p sub-intervals&#xA;    Positional arguments:&#xA;    c      -- damping parameter&#xA;    omega0 -- pulsation&#xA;    x0     -- initial position&#xA;    v0     -- initial speed&#xA;    T      -- final time&#xA;    p      -- the number of samples in [0,T]&#xA;    Keyword arguments:&#xA;    &quot;&quot;&quot;&#xA;    # Body&#xA;    time = np.linspace(0, T, p + 1)&#xA;    time = time[:, np.newaxis, np.newaxis]&#xA;    X0   = np.array([x0, v0])&#xA;    A    = np.array([[0, 1], [- omega0**2, - c]])&#xA;    B    = time * A&#xA;    E    = np.zeros(B.shape)&#xA;    for i in range(0, B.shape[0]):&#xA;        E[i,:,:] = expm(B[i,:,:])&#xA;    return time[:,0,0], E@X0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to avoid the use of a &lt;code&gt;for&lt;/code&gt; loop.&lt;/p&gt;&#xA;" OwnerUserId="7784" LastActivityDate="2017-03-14T10:46:14.523" Title="How to broadcast my matrix exponentiation" Tags="|python|scipy|numpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26386" PostTypeId="2" ParentId="26365" CreationDate="2017-03-13T11:26:16.163" Score="7" Body="&lt;p&gt;Depends on what you mean by CFD. I would differentiate between academic and applied approaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For academic purposes -- say, for acquisition of skills in numerical analysis and scientific computing or a playground for your newly developed preconditioners -- I can recommend the book by Griebel, Dornseifer, and Neunhoeffer &lt;em&gt;Numerical Simulations in Fluid Dynamics&lt;/em&gt;. It covers the whole range: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Statement of the physics and the PDEs, boundary conditions, and the expected difficulties&lt;/li&gt;&#xA;&lt;li&gt;Discretization technique (here, &lt;em&gt;finite differences&lt;/em&gt;)&lt;/li&gt;&#xA;&lt;li&gt;pseudocode&lt;/li&gt;&#xA;&lt;li&gt;ready to use C-code&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you want to become a CFD-engineer for a company, you should get yourself some theoretical background about modelling and approximation of flows and turbulence. (Maybe find a lecture note on &lt;em&gt;turbulent flows&lt;/em&gt;) And then just go and do the tutorials with your favorite commercial CFD tool. (Ansys, Fluent, Star...) &lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2017-03-13T12:03:38.007" LastActivityDate="2017-03-13T12:03:38.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26387" PostTypeId="1" AcceptedAnswerId="26390" CreationDate="2017-03-13T12:27:14.240" Score="1" ViewCount="589" Body="&lt;p&gt;Given a uniform polar grid, as in the figure below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gK96U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gK96U.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a FV discretization of a gradient for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial p}{\partial \varphi} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta r \frac{p_e - p_w}{\Delta \varphi} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My Question revolves around the value of $\Delta \varphi$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the Grid is uniform, $\Delta r$ does not change across the grid. But what about $\Delta \varphi$? From an angle point of view, it does not change either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta \varphi = 2\pi / NCells = $ constant&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an arc point of view, the length between two cells will change for different radii.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$arc = \Delta \varphi \cdot r_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which one is correct? Textbooks rarely concern themselves with anything other than Cartesian grids or jump directly into more complex ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual PDE is more complicated. I compared the results of the simulation against the exact solution for a constant $\Delta\varphi$, but which only exists for a simplified problem. They match pretty well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I was moving onto more complex situations, I was starting to doubt that assumption and before jumping into the code, I would like to clarify the situation.&lt;/p&gt;&#xA;" OwnerUserId="23063" LastEditorUserId="23063" LastEditDate="2017-03-14T14:12:04.103" LastActivityDate="2017-03-14T14:12:04.103" Title="Finite Volume Polar Discretization: Lengths" Tags="|finite-volume|discretization|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26390" PostTypeId="2" ParentId="26387" CreationDate="2017-03-13T20:48:27.660" Score="1" Body="&lt;p&gt;Your cells are not infinitesimally small so it will be a bit more complicated than either formulation you have there. (The ``width'' of your finite volume cell varies nontrivially over the cell, so we cannot just approximate it as $\Delta s \Delta r$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the piece you're integrating is defined by&#xA;$$ r_1 &amp;lt; r &amp;lt; r_2 $$&#xA;$$ \varphi_1 &amp;lt; \varphi &amp;lt; \varphi_2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We begin with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial p}{\partial \varphi} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we integrate in polar coordinates, we need a factor of $r$ there. &#xA;$$ \int_{\varphi_1}^{\varphi_2}\int_{r_1}^{r_2} \frac{\partial p}{\partial \varphi}   r dr   d\varphi  = 0 \\&#xA;\int_{r_1}^{r_2} \left[\int_{\varphi_1}^{\varphi_2} \frac{\partial p}{\partial \varphi}  d\varphi \right]  r dr    = 0  \\&#xA;\int_{r_1}^{r_2} \left[p(r,\varphi_2) - p(r,\varphi_1) \right]  r dr    = 0  \\&#xA; (rp)_e - (rp)_w = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the last line, I have tried to express the integral result in the notation you used, but I'm not exactly sure if that's what you intended with your notation.  Note that the function $p$ must be weighted by $r$ before integrating over the east and west borders of your finite volume cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $p$ is constant with respect to $r$, the previous result simplifies to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \left[p(\varphi_2) - p(\varphi_1) \right] \int_{r_1}^{r_2}   r dr    = 0  \\ \left[p(\varphi_2) - p(\varphi_1) \right] \frac{r_2^2-r_1^2}{2}    = 0  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiplying it by $\Delta r$ and $\Delta \varphi$ is unnecessary/arbitrary unless you have other terms in your differential equation.  When you do have other terms, I recommend you do the integration as I have done to see what the coefficients are.&lt;/p&gt;&#xA;" OwnerUserId="20583" LastEditorUserId="20583" LastEditDate="2017-03-13T20:54:25.243" LastActivityDate="2017-03-13T20:54:25.243" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26391" PostTypeId="2" ParentId="26385" CreationDate="2017-03-14T01:36:04.930" Score="2" Body="&lt;p&gt;Since $A$ is only a 2x2 matrix, you could just hard-code the matrix exponentiation. Note that if we can write $A=UDU^{-1}$, where $D$ is a diagonal matrix of eigenvalues of $A$ and the columns of $U$ are the associated eigenvectors, then $e^A = Ue^DU^{-1}$ and $e^{tA} = Ue^{tD}U^{-1}$, where $t$ is a scalar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that remains is to calculate the eigenvalues and eigenvectors. It's a little ugly, but it's probably faster than your code. Based on &lt;a href=&quot;http://matriisi.ee.tut.fi/~piche/ode/expm2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. Note that I didn't address the case of a defective matrix. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from numpy.linalg import eig,inv&#xA;&#xA;def ExactSol(c, omega0, x0, v0, T, p):&#xA;    time = np.linspace(0, T, p + 1)&#xA;    A = np.array([[0, 1], [- omega0**2, - c]])&#xA;    e,U = eig(A)&#xA;    tol = 1.e-6&#xA;    E = np.zeros( (p+1,2,2),dtype=float)&#xA;    if abs(e[0]-e[1]) &amp;lt; tol:&#xA;        # this is a defective matrix, which must be handled differently&#xA;        raise UserWarning(&quot;Defective matrix not implemented&quot;)&#xA;    else:&#xA;        # the eigenvalues are distinct -- possibly complex, but &#xA;        # E will always be real&#xA;        Uinv=inv(U)&#xA;        for k in range(2):&#xA;            E += np.real(np.exp(time*e[k])[:,np.newaxis,np.newaxis] *&#xA;                         np.dot(U[:,k,np.newaxis], Uinv[np.newaxis,k,:])[np.newaxis,:,:] )&#xA;&#xA;    X0 = np.array([x0, v0])&#xA;    X = np.dot(E,X0[:,np.newaxis])&#xA;    return time, X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit: Some quick timing information. I set &lt;code&gt;omega0=1.0&lt;/code&gt;, &lt;code&gt;c=0.1&lt;/code&gt;, &lt;code&gt;x0=0.0&lt;/code&gt;, &lt;code&gt;v0=1.0&lt;/code&gt;, &lt;code&gt;T=100.0&lt;/code&gt;, and &lt;code&gt;p=10000&lt;/code&gt;. Using your code, this function takes 2.95 seconds, while using my code, it takes 0.0025 seconds on my machine. So I've improved efficiency by a factor of over 1000. &lt;/p&gt;&#xA;" OwnerUserId="23262" LastEditorUserId="23262" LastEditDate="2017-03-14T10:46:14.523" LastActivityDate="2017-03-14T10:46:14.523" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26392" PostTypeId="1" CreationDate="2017-03-14T11:38:50.987" Score="2" ViewCount="239" Body="&lt;p&gt;I am trying to calculate the smoothness indicators for the WENO methods using the method given by &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999196901308&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jiang and Shu&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\beta_k = \sum_{l=1}^k \Delta x^{2l-1} \int_{x_{i-1/2}}^{x_{i+1/2}} \Big(\frac{\partial^l}{\partial x^l} p_j(x)\Big)^2 dx$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p_j(x)$&lt;/span&gt; is the interpolation polynomial in the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-th sub-stencil. I believe he uses the &lt;a href=&quot;http://mathworld.wolfram.com/LagrangeInterpolatingPolynomial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lagrangian&lt;/a&gt; interpolation, but I am not getting the exact coefficients that he gets &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/070679065&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; which is a review, and the smoothness indicators are given explicitly in equation (2.9) of the &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/070679065&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, I would like to ask if anyone knows of a reference that shows the calculation or if anyone has explicitly calculated the smoothness indicators from the polynomials.&lt;/p&gt;&#xA;" OwnerUserId="16601" LastEditorUserId="20688" LastEditDate="2023-01-19T13:12:23.240" LastActivityDate="2023-01-20T14:18:20.170" Title="Smoothness indicator calculation for WENO methods" Tags="|finite-difference|numerics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="26393" PostTypeId="1" CreationDate="2017-03-14T12:21:59.993" Score="3" ViewCount="182" Body="&lt;p&gt;I work in the field of Finite Element Methods, specifically, developing algorithms to speed up the computational time in structural mechanics problems. Till now I have been developing my code in Octave/Fortran and testing out my algorithms on small/simple problems (number of degrees of freedom less than 1000). Now I want to test out the algorithms on relatively large problems (number of degrees of freedom &gt; 10000). To implement my algorithms, I need access to the system matrices: Mass, tangent Stiffness, Damping matrices, so that I can construct my reduced order model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know which opensource FEA software (e.g., Calculix, Elmer, Tahoe or any other software) allows easy access and exchange of such information. &lt;/p&gt;&#xA;" OwnerUserId="23598" LastEditorUserId="9667" LastEditDate="2017-03-14T19:02:51.680" LastActivityDate="2017-03-14T19:02:51.680" Title="Extracting system matrices from FEM software" Tags="|finite-element|software|solid-mechanics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26394" PostTypeId="2" ParentId="26393" CreationDate="2017-03-14T12:48:47.393" Score="6" Body="&lt;p&gt;All of the major finite element libraries (such as libMesh; FEniCS; or the project I run, deal.II) provide you with ready access to the system matrix and or any other matrices you need. They typically also have tutorials and examples from a wide variety of areas (e.g., structures, fluids, etc) that you can use to generate examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe a simpler first step, however, is to take a look at Tim Davis's collection of matrices: &lt;a href=&quot;http://www.cise.ufl.edu/research/sparse/matrices/list_by_id.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.cise.ufl.edu/research/sparse/matrices/list_by_id.html&lt;/a&gt; . These contain examples from a number of areas and you can search the collection for specific examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to end on a side remark: &quot;relatively large problems&quot; would today have far more than the 10,000 degrees of freedom you quote in your post. I would say that everything under 100,000 is small today, and everything over 1,000,000 is large if you think of solving things on a single computer.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-03-14T12:48:47.393" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26395" PostTypeId="1" AcceptedAnswerId="26406" CreationDate="2017-03-14T13:36:49.970" Score="15" ViewCount="30060" Body="&lt;p&gt;I'm new to computational science and I already have learned basic methods for integration, interpolation, methods like RK4, Numerov etc on c++ but recently my professor asked me to learn how to use LAPACK for solving problems related to matrices. Like for example finding eigenvalues of a complex matrix. I have never used third-party libraries and I almost always write my own functions. I have been searching around for several days but can't find any amateur-friendly guide to lapack. All of them are written in words I don't understand and I don't know why using already written functions should be this complicated. They are full of words like zgeev, dtrsv, etc. and I'm frustrated. I just want to code something like this pseudo-code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;lapack:matrix&amp;gt;&#xA;int main(){&#xA;  LapackComplexMatrix A(n,n);&#xA;  for...&#xA;   for...&#xA;    cin&amp;gt;&amp;gt;A(i,j);&#xA;  cout&amp;lt;&amp;lt;LapackEigenValues(A);&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know if I'm being silly or amateur. But again, this shouldn't be that hard should it?&#xA;I don't even know should I use LAPACK or LAPACK++. (I write codes in c++ and have no knowledge of Python or FORTRAN) and how to install them.&lt;/p&gt;&#xA;" OwnerUserId="23599" LastActivityDate="2022-08-01T15:38:38.610" Title="How to start using LAPACK in c++?" Tags="|lapack|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26396" PostTypeId="2" ParentId="26395" CreationDate="2017-03-14T14:03:09.747" Score="10" Body="&lt;p&gt;I usually resist telling people what I think they should do rather&#xA;than answering their question but in this case I'm going to make an&#xA;exception.&lt;/p&gt;&#xA;&lt;p&gt;Lapack is written in FORTRAN and the API is very FORTRAN-like.&#xA;There is a C API to Lapack that makes the interface slightly&#xA;less painful but it will never be a pleasant experience to use&#xA;Lapack from C++.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, there is a C++ matrix class library called &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigen&lt;/a&gt; that&#xA;has many of the capabilities of Lapack, provides computational performance&#xA;comparable to the better Lapack implementations, and is very convenient to&#xA;use from C++. In particular, here is how your example code might be&#xA;written using Eigen&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;using std::cout;&#xA;using std::endl;&#xA;&#xA;#include &amp;lt;Eigen/Eigenvalues&amp;gt;&#xA;&#xA;int main()&#xA;{&#xA;  const int n = 4;&#xA;  Eigen::MatrixXd a(n, n); &#xA;  a &amp;lt;&amp;lt;&#xA;    0.35, 0.45, -0.14, -0.17,&#xA;    0.09, 0.07, -0.54, 0.35,&#xA;    -0.44, -0.33, -0.03, 0.17,&#xA;    0.25, -0.32, -0.13, 0.11;&#xA;  Eigen::EigenSolver&amp;lt;Eigen::MatrixXd&amp;gt; es; &#xA;  es.compute(a);&#xA;  Eigen::VectorXcd ev = es.eigenvalues();&#xA;  cout &amp;lt;&amp;lt; ev &amp;lt;&amp;lt; endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This example eigenvalue problem is a test case for the Lapack function&#xA;&lt;code&gt;dgeev&lt;/code&gt;. You can view the FORTRAN code and results for this problem&#xA;&lt;a href=&quot;https://www.nag.com/lapack-ex/node87.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dgeev example&lt;/a&gt; and make your own comparisons.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="-1" LastEditDate="2021-01-06T04:57:35.863" LastActivityDate="2021-01-06T04:57:35.863" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="26397" PostTypeId="2" ParentId="26395" CreationDate="2017-03-14T14:59:15.700" Score="9" Body="&lt;p&gt;Here's another answer in the same vein as the above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should look into the &lt;a href=&quot;http://arma.sourceforge.net/docs.html&quot; rel=&quot;noreferrer&quot;&gt;Armadillo C++ linear algebra library&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The function syntax is high-level (similar to that of MATLAB). So no &lt;code&gt;DGESV&lt;/code&gt; mumbo-jumbo, just &lt;code&gt;X = solve( A, B )&lt;/code&gt; (although there is &lt;a href=&quot;http://www.netlib.org/lapack/lug/node24.html&quot; rel=&quot;noreferrer&quot;&gt;a reason behind those oddly-looking LAPACK function names&lt;/a&gt;...).&lt;/li&gt;&#xA;&lt;li&gt;Implements various matrix decompositions (LU, QR, eigenvalues, SVD, Cholesky, etc.)&lt;/li&gt;&#xA;&lt;li&gt;It is &lt;a href=&quot;http://arma.sourceforge.net/speed.html&quot; rel=&quot;noreferrer&quot;&gt;fast&lt;/a&gt; when used properly.&lt;/li&gt;&#xA;&lt;li&gt;It is well &lt;a href=&quot;http://arma.sourceforge.net/docs.html&quot; rel=&quot;noreferrer&quot;&gt;documented&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Has support for sparse matrices (you will want to look into these later).&lt;/li&gt;&#xA;&lt;li&gt;You can link it against your super-optimized BLAS/LAPACK libraries for optimal performance.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here's how @BillGreene's code would look like with Armadillo:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;armadillo&amp;gt;&#xA;&#xA;using namespace std;&#xA;using namespace arma;&#xA;&#xA;int main()&#xA;{&#xA;   const int k = 4;&#xA;   mat A = zeros&amp;lt;mat&amp;gt;(k,k) // mat == Mat&amp;lt;double&amp;gt;&#xA;&#xA;   // with the &amp;lt;&amp;lt; operator...&#xA;   A &amp;lt;&amp;lt;&#xA;    0.35 &amp;lt;&amp;lt; 0.45 &amp;lt;&amp;lt; -0.14 &amp;lt;&amp;lt; -0.17 &amp;lt;&amp;lt; endr&#xA;    0.09 &amp;lt;&amp;lt; 0.07 &amp;lt;&amp;lt; -0.54 &amp;lt;&amp;lt; 0.35  &amp;lt;&amp;lt; endr&#xA;    -0.44 &amp;lt;&amp;lt; -0.33 &amp;lt;&amp;lt; -0.03 &amp;lt;&amp;lt; 0.17 &amp;lt;&amp;lt; endr&#xA;    0.25 &amp;lt;&amp;lt; -0.32 &amp;lt;&amp;lt; -0.13 &amp;lt;&amp;lt; 0.11 &amp;lt;&amp;lt; endr;&#xA;&#xA;   // but using an initializer list is faster&#xA;   A = { {0.35, 0.45, -0.14, -0.17}, &#xA;         {0.09, 0.07, -0.54, 0.35}, &#xA;         {-0.44, -0.33, -0.03, 0.17}, &#xA;         {0.25, -0.32, -0.13, 0.11} };&#xA;&#xA;   cx_vec eigval; // eigenvalues may well be complex&#xA;   cx_mat eigvec;&#xA;&#xA;   // eigenvalue decomposition for general dense matrices&#xA;   eig_gen(eigval, eigvec, A);&#xA;&#xA;   std::cout &amp;lt;&amp;lt; eigval &amp;lt;&amp;lt; std::endl;&#xA;&#xA;   return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-03-16T16:41:43.540" LastActivityDate="2017-03-16T16:41:43.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26398" PostTypeId="1" CreationDate="2017-03-14T16:22:05.463" Score="1" ViewCount="255" Body="&lt;p&gt;I was trying to optimise CdSe (Cysteine) molecule using a semi-empirical method in Gaussian 09 (and gaussView) for a preliminary study of quantum dots. But it seems as the number of iterations advances, resultant optimised molecule is getting  dissociated into different sub group which has no direct bond/connection with other fragment. Is there any workaround to deal with this?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thi is the initial molecule and one of the optimised molecular conformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kSTu4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kSTu4.png&quot; alt=&quot;fragmented molecule after optimization&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23602" LastEditorUserId="9667" LastEditDate="2017-03-14T18:55:19.040" LastActivityDate="2017-03-17T19:54:58.487" Title="Gaussian geometry optimisation: molecule is getting dissociated into sub group?" Tags="|optimization|geometry|density-functional-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26399" PostTypeId="2" ParentId="26341" CreationDate="2017-03-14T17:45:09.827" Score="2" Body="&lt;p&gt;This is due to an implicit time shift, which corresponds to a phase shift, perceived here as a sign inversion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The signal you are Fourier transforming is symmetric around t=0 and this is why you should expect a positive power spectrum.&#xA;However, the discrete Fourier transform (DFT) of a time-series $x_n$, which is what the FFT alorithm implements, is defined as &#xA;$$&#xA;X_k = \sum_{n=0}^{N-1}x_n e^{-2\pi i k n},&#xA;$$&#xA;where $N$ is the number of data points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ignoring some normalization issues, this can be interpreted as an approximation of the Fourier transform of a sampled signal, where $$\omega_k = 2\pi k$$ and $$t_n = \frac{n}{T},$$ $T$ being the width of the time window. This is the source of the implicit time shift. The &quot;correct&quot; time sample values are $$\hat t_n = t_n - T/2,$$ so that the phase corrected power spectrum, which has real and non-negative values, is given by &#xA;$$&#xA;\hat X_k = \sum_{n=0}^{N-1}x_n e^{-2\pi i k (n-1/2)} = X_k \cdot e^{ i \pi  k},$$&#xA;or&#xA;$$X_k=(-1)^k \hat X_k$$,&#xA;which explains the fast sign change that you observe.&lt;/p&gt;&#xA;" OwnerUserId="22217" LastActivityDate="2017-03-14T17:45:09.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26400" PostTypeId="1" CreationDate="2017-03-14T18:24:55.003" Score="1" ViewCount="400" Body="&lt;pre&gt;&lt;code&gt;#f,g are the dH/dq and -dH/dp respectively&#xA;&#xA;def f(v1,v2):&#xA;&#xA;    alpha = (a*v1 - b*v2)/k &#xA;    beta =  (a*v2 - c*v1)/k&#xA;&#xA;    dot1 = (c-a)/(3*k) +  (a*np.exp(alpha)/k - c*np.exp(beta)/k)/(1 + np.exp(alpha) + np.exp(beta))&#xA;    dot2 = (b-a)/(3*k) +  (a*np.exp(beta)/k - b*np.exp(alpha)/k)/(1 + np.exp(alpha) + np.exp(beta))&#xA;    return dot1, dot2&#xA;&#xA;def g(u1,u2):&#xA;&#xA;    dot1 = -1/3 + np.exp(u1)/(1 + np.exp(u1)+ np.exp(u2))&#xA;    dot2 = -1/3 + np.exp(u2)/(1 + np.exp(u1)+ np.exp(u2))&#xA;    return -dot1, -dot2&#xA;&#xA;&#xA;&#xA;#these are the constants for 4th order symplectic rk method where c1 is for canonical position and c2 for canonical momentum cordinates. (As given by Ruth and Forest)&#xA;&#xA;c1 = np.array((1.0/(2.0*(2.0-2.0**(1.0/3.0))), (1.0-2.0**(1.0/3.0))/(2.0*(2.0-2.0**(1.0/3.0))),(1.0-2.0**(1.0/3.0))/(2.0*(2.0-2.0**(1.0/3.0))), 1.0/(2.0*(2.0-2.0**(1.0/3.0)))))&#xA;&#xA;c2 = np.array((1.0/(2.0-2.0**(1.0/3.0)), -2.0**(1.0/3.0)/(2.0-2.0**(1.0/3.0)),1.0/(2.0-2.0**(1.0/3.0)), 0.0))&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;#this is the function where the integration is happening and returns the 4 arrays which are time series of the phase-space variables&#xA;&#xA;def sol(x0):&#xA;&#xA;    u1 = np.zeros(n+1)&#xA;    u2 = np.zeros(n+1)&#xA;    v1 = np.zeros(n+1)&#xA;    v2 = np.zeros(n+1)&#xA;&#xA;    u1[0] = x0[0]&#xA;    u2[0] = x0[1]&#xA;    v1[0] = x0[2]&#xA;    v2[0] = x0[3]&#xA;    for k in xrange (n):&#xA;            u10 = u1[k]&#xA;            u20 = u2[k]&#xA;            v10 = v1[k]&#xA;            v20 = v2[k]&#xA;            for i in xrange(len(c2)):&#xA;                    U1 = u10 + c1[i]*h*f(v10,v20)[0]&#xA;                    U2 = u20 + c1[i]*h*f(v10,v20)[1]&#xA;                    V1 = v10 + c2[i]*h*g(U1,U2)[0]&#xA;                    V2 = v20 + c2[i]*h*g(U1,U2)[1]&#xA;                    u10 = U1&#xA;                    u20 = U2&#xA;                    v10 = V1&#xA;                    v20 = V2&#xA;&#xA;            U1 = u10 + h*c1[-1]*f(v10,v20)[0]&#xA;            U2 = u20 + h*c1[-1]*f(v10,v20)[1]&#xA;            u10 = U1&#xA;            u20 = U2&#xA;            u1[k+1] = U1&#xA;            u2[k+1] = U2&#xA;            v1[k+1] = V1&#xA;            v2[k+1] = V2&#xA;&#xA;    return u1,u2,v1,v2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run the code for some values of parameters (like a,b,c, etc) I get an overflow error in dot2. For other parameters I am getting a phase space which is not bounded, but a Hamiltonian system is supposed to have a bounded trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is something wrong with my algorithm? Kindly point the error. I can provide the whole code and the paper, whose results I am trying to reproduce.&lt;/p&gt;&#xA;" OwnerUserId="23575" LastActivityDate="2017-03-14T18:24:55.003" Title="4th order Symplectic integration of a 2 degree of freedom Hamiltonian system" Tags="|python|nonlinear-equations|time-integration|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26401" PostTypeId="2" ParentId="26365" CreationDate="2017-03-14T19:37:45.370" Score="4" Body="&lt;p&gt;The CFD book by &lt;a href=&quot;http://www.sciencedirect.com/science/book/9780080999951&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blazek&lt;/a&gt; helped me more than any other.  He explains finite volume methods very well.  Using this book, I wrote a 2D Euler solver and a 2D compressible Navier--Stokes solver.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also suggest that you watch &lt;a href=&quot;https://www.youtube.com/channel/UCPVvF1gF2NMNWV-GQCCrpKQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Qiqi Wang's lecture videos&lt;/a&gt;.  Not the aerodynamics of viscous fluids course -- scroll further down and you should see videos with Lecture 20 ... Lecture 1.  These videos are from a numerical methods course at MIT.  This is where I really learned how to write finite volume codes.  He explains the theory/how the methods work and then develops examples in MATLAB so you can follow along.  He teaches very well and is easy to follow.&lt;/p&gt;&#xA;" OwnerUserId="19799" LastActivityDate="2017-03-14T19:37:45.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26402" PostTypeId="1" CreationDate="2017-03-14T22:08:54.307" Score="1" ViewCount="26" Body="&lt;p&gt;Is there an algorithm on python to analyze swarming behavior or model swarming behavior for predictions?&lt;/p&gt;&#xA;" OwnerUserId="23606" LastActivityDate="2017-03-14T22:08:54.307" Title="Is there an algorithm on python for analyzing swarming behavior or a modeling algorithm to predict swarming behavior?" Tags="|algorithms|" AnswerCount="0" CommentCount="1" ClosedDate="2017-03-15T08:23:07.640" ContentLicense="CC BY-SA 3.0" />
  <row Id="26403" PostTypeId="1" CreationDate="2017-03-15T00:32:35.673" Score="1" ViewCount="294" Body="&lt;p&gt;I'm attempting to simulate a standing breather of the form&#xA;$$ u(x,t)=4\tan^{-1}\left(\sqrt{3}\cos\left(\frac{t}{2}\right)sech\left(\frac{\sqrt{3}x}{2}\right)\right)$$&#xA;for the Sine-Gordon equation&#xA;$$u_{tt}-u_{xx}+\sin u  =0 $$&#xA;on the real line by using centered differences in both time and space. Because of reflections and the use of zero Dirichlet conditions, I get something like this after just a bit of time:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dc0nT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dc0nT.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does one avoid this and make it seem like the simulation is on the whole of $\mathbb{R}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code in MATLAB for reference&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;l=40;delta=0.08;T=80;h=0.1;&#xA;t=linspace(0,T,T/delta);&#xA;x=linspace(-l,l,l/h);&#xA;u=zeros(length(x),length(t));&#xA;u0=4*atan(sqrt(3)*sech(sqrt(3)*x/2));&#xA;ut0=0*x;&#xA;for n=2:length(x)-1&#xA;        u(n,1)=u0(n);&#xA;        u(n,2)=(delta/h)^2/2*(u0(n+1)+u0(n-1))-(delta)^2/2*sin(u0(n))+(1-(delta/h)^2)*u0(n)+delta*ut0(n);&#xA;end&#xA;for m=2:length(t)-1&#xA;    for n=2:length(x)-1&#xA;        u(n,m+1)=(delta/h)^2*(u(n+1,m)+u(n-1,m))-delta^2*sin(u(n,m))+2*(1-(delta/h)^2)*u(n,m)-u(n,m-1);       &#xA;    end&#xA;    plot(x,u(:,m+1));&#xA;    ylim([-4 4]);&#xA;    xlabel('x');&#xA;    ylabel('u(x,t)');&#xA;    pause(0.001);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18825" LastEditorUserId="18825" LastEditDate="2017-03-15T05:32:47.097" LastActivityDate="2017-03-15T05:32:47.097" Title="Breather solutions of Sine-Gordon Using Finite Differences" Tags="|pde|finite-difference|boundary-conditions|nonlinear-equations|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26404" PostTypeId="1" AcceptedAnswerId="26405" CreationDate="2017-03-15T06:20:10.797" Score="3" ViewCount="242" Body="&lt;p&gt;I have recently learned about Discontinuous Galerkin method to solve differential equations and I was trying to implement it to solve Euler equation. For now, consider the standard Sod Shock Tube Case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without using any limiter I am getting an unstable scheme (even with CFL 0.01) and my solution blows up. Presently I am using 1st order polynomials in each cell using Lobatto nodes (in case someone needs to know). Also, I am calculating the interface flux using AUSM method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am not sure whether my code is wrong or does it behave like this because I haven't used any flux limiter?&lt;/p&gt;&#xA;" OwnerUserId="11977" LastActivityDate="2017-03-15T06:32:25.817" Title="Is it possible to solve Euler equation numerically without using any flux limiter (in DG scheme)?" Tags="|numerical-analysis|fluid-dynamics|discontinuous-galerkin|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26405" PostTypeId="2" ParentId="26404" CreationDate="2017-03-15T06:32:25.817" Score="1" Body="&lt;p&gt;A finite element scheme (CG or DG) without stabilization is expected to result in a solution that violates the maximum principle, even for linear scalar problems. That is why sufficient dissipation/viscosity is needed in addition to the dissipation on the approximate Riemann solver on the interface. See the papers by Cockburn and Shu for a detailed discussion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Various approaches exist to deal with this issue such as entropy viscosity, SUPG stabilization etc...&lt;/p&gt;&#xA;" OwnerDisplayName="user814" LastActivityDate="2017-03-15T06:32:25.817" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26406" PostTypeId="2" ParentId="26395" CreationDate="2017-03-15T08:23:49.137" Score="34" Body="&lt;p&gt;I'm going to disagree with some of the other answers and say that I believe that figuring out how to use LAPACK &lt;em&gt;is&lt;/em&gt; important in the field of scientific computing.&lt;/p&gt;&#xA;&lt;p&gt;However, there is a large learning curve to using LAPACK. This is because it is written at a very low level. The disadvantage of that is that it seems very cryptic, and not pleasant to the senses. The advantage of it is that the interface is unambiguous and basically never changes. Additionally, implementations of LAPACK, such as the &lt;a href=&quot;https://software.intel.com/en-us/intel-mkl&quot; rel=&quot;noreferrer&quot;&gt;Intel Math Kernel Library&lt;/a&gt; are really fast.&lt;/p&gt;&#xA;&lt;p&gt;For my own purposes, I have my own higher level C++ classes which wrap around LAPACK subroutines. Many scientific libraries also use LAPACK underneath. Sometimes it's easier to just use them, but in my opinion there's a lot of value in understanding the tool underneath. To that end, I've provided a small working example written in C++ using LAPACK to get you started. This works in Ubuntu, with the &lt;code&gt;liblapack3&lt;/code&gt; package installed, and other necessary packages for building. It can probably be used in most Linux distributions, but installation of LAPACK and linking against it can vary.&lt;/p&gt;&#xA;&lt;p&gt;Here's the file &lt;code&gt;test_lapack.cpp&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;    #include &amp;lt;iostream&amp;gt;&#xA;    #include &amp;lt;fstream&amp;gt;&#xA;&#xA;&#xA;    using namespace std;&#xA;&#xA;    // dgeev_ is a symbol in the LAPACK library files&#xA;    extern &amp;quot;C&amp;quot; {&#xA;    extern int dgeev_(char*,char*,int*,double*,int*,double*, double*, double*, int*, double*, int*, double*, int*, int*);&#xA;    }&#xA;&#xA;    int main(int argc, char** argv){&#xA;&#xA;      // check for an argument&#xA;      if (argc&amp;lt;2){&#xA;        cout &amp;lt;&amp;lt; &amp;quot;Usage: &amp;quot; &amp;lt;&amp;lt; argv[0] &amp;lt;&amp;lt; &amp;quot; &amp;quot; &amp;lt;&amp;lt; &amp;quot; filename&amp;quot; &amp;lt;&amp;lt; endl;&#xA;        return -1;&#xA;      }&#xA;&#xA;      int n,m;&#xA;      double *data;&#xA;&#xA;      // read in a text file that contains a real matrix stored in column major format&#xA;      // but read it into row major format&#xA;      ifstream fin(argv[1]);&#xA;      if (!fin.is_open()){&#xA;        cout &amp;lt;&amp;lt; &amp;quot;Failed to open &amp;quot; &amp;lt;&amp;lt; argv[1] &amp;lt;&amp;lt; endl;&#xA;        return -1;&#xA;      }&#xA;      fin &amp;gt;&amp;gt; n &amp;gt;&amp;gt; m;  // n is the number of rows, m the number of columns&#xA;      data = new double[n*m];&#xA;      for (int i=0;i&amp;lt;n;i++){&#xA;        for (int j=0;j&amp;lt;m;j++){&#xA;          fin &amp;gt;&amp;gt; data[j*n+i];&#xA;        }&#xA;      }&#xA;      if (fin.fail() || fin.eof()){&#xA;        cout &amp;lt;&amp;lt; &amp;quot;Error while reading &amp;quot; &amp;lt;&amp;lt; argv[1] &amp;lt;&amp;lt; endl;&#xA;        return -1;&#xA;      }&#xA;      fin.close();&#xA;&#xA;      // check that matrix is square&#xA;      if (n != m){&#xA;        cout &amp;lt;&amp;lt; &amp;quot;Matrix is not square&amp;quot; &amp;lt;&amp;lt;endl;&#xA;        return -1;&#xA;      }&#xA;&#xA;      // allocate data&#xA;      char Nchar='N';&#xA;      double *eigReal=new double[n];&#xA;      double *eigImag=new double[n];&#xA;      double *vl,*vr;&#xA;      int one=1;&#xA;      int lwork=6*n;&#xA;      double *work=new double[lwork];&#xA;      int info;&#xA;&#xA;      // calculate eigenvalues using the DGEEV subroutine&#xA;      dgeev_(&amp;amp;Nchar,&amp;amp;Nchar,&amp;amp;n,data,&amp;amp;n,eigReal,eigImag,&#xA;            vl,&amp;amp;one,vr,&amp;amp;one,&#xA;            work,&amp;amp;lwork,&amp;amp;info);&#xA;&#xA;&#xA;      // check for errors&#xA;      if (info!=0){&#xA;        cout &amp;lt;&amp;lt; &amp;quot;Error: dgeev returned error code &amp;quot; &amp;lt;&amp;lt; info &amp;lt;&amp;lt; endl;&#xA;        return -1;&#xA;      }&#xA;&#xA;      // output eigenvalues to stdout&#xA;      cout &amp;lt;&amp;lt; &amp;quot;--- Eigenvalues ---&amp;quot; &amp;lt;&amp;lt; endl;&#xA;      for (int i=0;i&amp;lt;n;i++){&#xA;        cout &amp;lt;&amp;lt; &amp;quot;( &amp;quot; &amp;lt;&amp;lt; eigReal[i] &amp;lt;&amp;lt; &amp;quot; , &amp;quot; &amp;lt;&amp;lt; eigImag[i] &amp;lt;&amp;lt; &amp;quot; )\n&amp;quot;;&#xA;      }&#xA;      cout &amp;lt;&amp;lt; endl;&#xA;&#xA;      // deallocate&#xA;      delete [] data;&#xA;      delete [] eigReal;&#xA;      delete [] eigImag;&#xA;      delete [] work;&#xA;&#xA;&#xA;      return 0;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This can be built using the command line&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;    g++ -o test_lapack test_lapack.cpp -llapack&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will produce an executable named &lt;code&gt;test_lapack&lt;/code&gt;. I've set this up to read in a text input file. Here's a file named &lt;code&gt;matrix.txt&lt;/code&gt; containing a 3x3 matrix.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3 3&#xA;-1.0 -8.0  0.0&#xA;-1.0  1.0 -5.0&#xA; 3.0  0.0  2.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To run the program simply type&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;    ./test_lapack matrix.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;at the command line, and the output should be&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--- Eigenvalues ---&#xA;( 6.15484 , 0 )&#xA;( -2.07742 , 3.50095 )&#xA;( -2.07742 , -3.50095 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Comments:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You seem thrown off by the naming scheme for LAPACK. A short description is &lt;a href=&quot;https://en.wikipedia.org/wiki/LAPACK&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The interface for the DGEEV subroutine is &lt;a href=&quot;http://www.netlib.org/lapack/lapack-3.1.1/html/dgeev.f.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. You should be able to compare the description of the arguments there to what I've done here.&lt;/li&gt;&#xA;&lt;li&gt;Note the &lt;code&gt;extern &amp;quot;C&amp;quot;&lt;/code&gt; section at the top, and that I've added an underscore to &lt;code&gt;dgeev_&lt;/code&gt;. That's because the library was written and built in Fortran, so this is necessary to make the symbols match when linking. This is compiler and system dependent, so if you use this on Windows, it will all have to change.&lt;/li&gt;&#xA;&lt;li&gt;Some people might suggest using the &lt;a href=&quot;http://www.netlib.org/lapack/lapacke.html&quot; rel=&quot;noreferrer&quot;&gt;C interface to LAPACK&lt;/a&gt;. They might be right, but I've always done it this way.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23262" LastEditorUserId="-1" LastEditDate="2021-01-06T04:58:44.710" LastActivityDate="2021-01-06T04:58:44.710" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="26407" PostTypeId="1" CreationDate="2017-03-15T10:00:28.713" Score="1" ViewCount="97" Body="&lt;p&gt;I am new to COMSOL and I want to model fluid flow in stress field. As starting point I want a simple 2D box having a fracture from right to left while pressure gradient vertically. I want to know the threshold value of that pressure from top which allows flow of fluid in fracture. Can anyone please suggest me which modules of COMSOL should I be using. I see simple fluid flow in fracture examples but not where impact of pressure controls the flow!&lt;/p&gt;&#xA;" OwnerUserId="20089" LastEditorUserId="9667" LastEditDate="2017-03-15T16:55:56.663" LastActivityDate="2017-03-15T16:55:56.663" Title="Modelling fracture flow in stress field in COMSOL" Tags="|fluid-dynamics|comsol|solid-mechanics|" AnswerCount="0" CommentCount="3" ClosedDate="2017-03-20T18:33:30.243" ContentLicense="CC BY-SA 3.0" />
  <row Id="26409" PostTypeId="1" CreationDate="2017-03-15T17:44:13.007" Score="1" ViewCount="30" Body="&lt;p&gt;Suppose you have spheres (e.g. cells). You add particles to the suspended cells which are smaller but do have a size distribution rather than a single diameter. The particles stick to the cells. I would like to model how much of the cell surface is covered by particles and how much of the covered fraction is covered by which particle size fraction. I have no idea how to start (probably with a pen and a paper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any computer packages which can do similar calculations. Is there any Literature I havent found because I do not know what terms to look for? How would you approach such a task ?&lt;/p&gt;&#xA;" OwnerUserId="3957" LastActivityDate="2017-03-15T17:44:13.007" Title="modelling surface coverage of spheres with spheres of varying size distribution" Tags="|modeling|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26410" PostTypeId="2" ParentId="7357" CreationDate="2017-03-15T18:23:21.793" Score="1" Body="&lt;p&gt;There are, to my knowledge, two codes available:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/ghb24/NECI_STABLE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ghb24/NECI_STABLE&lt;/a&gt;, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/hande-qmc/hande&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hande-qmc/hande&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both are developed by the authors involved in the&#xA;design of FCIQMC. The second one has a better documentation&#xA;and also includes other recent methods like i-FCIQMC, CCMC, &#xA;DMMC or semi-stochastic MC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods are intended to compete with wavefunction methods on&#xA;accuracy but with a lower computational complexity. For example, i-FCIQMC&#xA;has been reported for ethene and butadiene molecules &#xA;(&lt;a href=&quot;http://doi:10.1021/ct300486d&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doi:10.1021/ct300486d&lt;/a&gt;), that otherwise are impossible to treat with Full Configuration Interaction. In fact the later cannot go beyond dinitrogen ($N_2$) with a decent basis set. On the other hand this is not always possible. For strongly correlated systems you will have to explore almost all the Hilbert space, then you gain only a slight or no benefit from using them.&#xA;Improvements are part of current research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the authors claim that their calculations were done with these codes&#xA;I would expect the results reported in the literature to be reproducible.&lt;/p&gt;&#xA;" OwnerUserId="21198" LastActivityDate="2017-03-15T18:23:21.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26411" PostTypeId="2" ParentId="24851" CreationDate="2017-03-15T19:30:34.820" Score="1" Body="&lt;p&gt;I also found that book written by Akira Hirose in the library. I have the same question as well, and I found this paper: &lt;a href=&quot;https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2011-42.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2011-42.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, we are not even in the &quot;real&quot; world. The input spaces of any neural networks implemented on modern computers are not just countable, but finite. Also the cardinality of real numbers is the same as complex numbers. So I think for the same structure NN, the real-valued NN can solve all the questions that can be solved by complex-valued NN.&lt;/p&gt;&#xA;" OwnerUserId="23611" LastActivityDate="2017-03-15T19:30:34.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26413" PostTypeId="1" CreationDate="2017-03-15T22:26:05.493" Score="2" ViewCount="730" Body="&lt;p&gt;I am using spilu (incomplete LU decomposition for sparse matrix) in scipy: &#xA;&lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.sparse.linalg.spilu.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.sparse.linalg.spilu.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However there is an option call &quot;fill_factor&quot; which is quite unclear to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If fill_factor=100, does it mean the matrix is full; and fill_factor=50 means the matrix is half-full?&lt;/p&gt;&#xA;" OwnerUserId="23481" LastEditorUserId="17294" LastEditDate="2017-03-17T07:56:48.980" LastActivityDate="2017-03-17T07:56:48.980" Title="How to use &quot;fill_factor&quot; in spilu (scipy)" Tags="|numerical-analysis|python|scipy|preconditioning|" AnswerCount="0" CommentCount="4" ClosedDate="2017-03-20T00:16:25.653" ContentLicense="CC BY-SA 3.0" />
  <row Id="26416" PostTypeId="1" CreationDate="2017-03-16T15:39:54.350" Score="1" ViewCount="75" Body="&lt;p&gt;MATLAB/Octave is a pretty useful language for prototyping scientific computation use cases, however I would like to know if the sheer nature of being a high level language bottlenecks how fast the performance can get. (I'm not talking about writing optimal code like vectorization, but a general impact on CPU Time, like how C++ can be made faster than equivalent Python programs by really large magnitudes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My use case involves running a lot of statistical computations (like Sample Entropy for a 1D temporal time series) and I don't have equivalent code ready in C right now to test my hypothesis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we may agree upon as a rule of thumb, lower level languages outperform higher level languages at the cost of code readability and learning curves, but I had also heard that MATLAB interfaces super fast Fortran libraries for many of its numeric computations so I'm unsure how to proceed without unnecessarily wasting my time.&lt;/p&gt;&#xA;" OwnerUserId="23626" LastEditorUserId="9667" LastEditDate="2017-03-16T15:44:46.230" LastActivityDate="2017-03-16T15:44:46.230" Title="Is MATLAB feasible for app deployment versus lower level languages" Tags="|matlab|c++|fortran|c|octave|" AnswerCount="0" CommentCount="2" ClosedDate="2017-04-17T07:24:53.227" ContentLicense="CC BY-SA 3.0" />
  <row Id="26417" PostTypeId="1" CreationDate="2017-03-16T16:09:46.537" Score="1" ViewCount="158" Body="&lt;p&gt;I am studying the discretization of Poisson's equation in $1D$. In Matlab I created different discretization matrices (Laplace operator) according to different sizes of the mesh:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Parameters&#xA;N = [40, 80, 160, 1280]; % Dimension 1-D grid&#xA;&#xA;% Laplace operators&#xA;L1 = -1/((1/N(1))^2) * full( ( spdiags(repmat([1,-2,1], N(1)-1, 1), -1:1, N(1)-1, N(1)-1) ) );&#xA;L2 = -1/((1/N(2))^2) * full( ( spdiags(repmat([1,-2,1], N(2)-1, 1), -1:1, N(2)-1, N(2)-1) ) );&#xA;L3 = -1/((1/N(3))^2) * full( ( spdiags(repmat([1,-2,1], N(3)-1, 1), -1:1, N(3)-1, N(3)-1) ) );&#xA;L4 = -1/((1/N(4))^2) * full( ( spdiags(repmat([1,-2,1], N(4)-1, 1), -1:1, N(4)-1, N(4)-1) ) );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then plotted the eigenvalues of the matrices using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[V1, D1] = eig(L1);&#xA;plot(diag(D1) / norm(diag(D1),2), '--');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I see is that as I increase $N$ (so the step size goes to zero), the difference between the biggest and the smallest eigenvalues decreases. Therefore the condition number is getting better. Is there anything else I can see from such a plot?&lt;/p&gt;&#xA;" OwnerUserId="22185" LastEditorUserId="22185" LastEditDate="2017-03-16T16:39:34.467" LastActivityDate="2017-03-16T16:39:34.467" Title="Spectrum of the Laplace operator" Tags="|matlab|eigenvalues|poisson|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26418" PostTypeId="1" AcceptedAnswerId="34060" CreationDate="2017-03-16T17:57:15.630" Score="10" ViewCount="533" Body="&lt;p&gt;Given a set of points $\{x_j\}_{j=1}^n$ in $[-1, 1]$, I would like to compute&#xA;$$&#xA;\int_{-1}^{1} L_i(x)\,\text{d} x&#xA;$$&#xA;exactly. $L_i$ is the Lagrange polynomial with respect to the points $x_j$ with $x_i$ as node, i.e.,&#xA;$$&#xA;L_i(x) = \prod_{j\neq i} \frac{x - x_j}{x_i - x_j}.&#xA;$$&#xA;Since this is a polynomial of degree $n$, I could use any old Gaussian quadrature of sufficient degree. This works well if $n$ isn't too large, but leads to results flawed by round-off errors for large $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea how to avoid those?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2017-03-18T19:59:17.367" LastActivityDate="2019-12-19T16:25:05.080" Title="Integrating Lagrange polynomials with many nodes, round-off" Tags="|quadrature|integration|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26419" PostTypeId="2" ParentId="26418" CreationDate="2017-03-16T19:06:36.503" Score="2" Body="&lt;p&gt;Calculate the products of the nominators and the denominators first and then divide once. The two products should be of the same order of magnitude, so there should be no significant round-off errors. Also you get the added benefit of increased speed, due to the reduced number of floating point calculations.&lt;/p&gt;&#xA;" OwnerUserId="22217" LastActivityDate="2017-03-16T19:06:36.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26420" PostTypeId="2" ParentId="26365" CreationDate="2017-03-17T01:09:31.377" Score="1" Body="&lt;p&gt;This question is extremely broad. As Jan stated above, CFD could be industry related or based on a research topic. Furthermore there are big difference between compressible and incompressible flow problems. If you are focusing on compressible flow, then familiarizing yourself with hyperbolic conservation laws is a good start. And as James above has said, implementing a finite volume solver in a simple 2D Cartesian grid is another good step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend books such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Finite Volume Methods for Hyperbolic Problems by Randall J. LeVeque&lt;/li&gt;&#xA;&lt;li&gt;Nodal Discontinuous Galerkin Methods by Jan S. Hesthaven and Tim Warburton&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hesthaven and Warburton have both matlab and C++ codes for implementing DG schemes for Euler equations in 1D and 2D. Finite volume methods are DG with piecewise constant approximations.&lt;/p&gt;&#xA;" OwnerDisplayName="user814" LastActivityDate="2017-03-17T01:09:31.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26422" PostTypeId="1" CreationDate="2017-03-17T10:12:40.560" Score="1" ViewCount="386" Body="&lt;p&gt;I'm currently developing a program to solve 2D transient state heat conduction on a square plate using the V-cycle multigrid. Althought my program is able to reach the steady state solution, it's computational time is longer then just running the problem using Gauss-seidel method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem case: 0.1 m by 0.1 m square plate with fixed temperatures. Top: 20°C and the other three sides to be 40°C. Assuming constant material properties, no internal heat generation, and equal grid length $\Delta x=\Delta y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parameters:  thermal diffusion ($\alpha=23.1\times 10^{-6}$) (using steel for now as a guide), $\Delta t=0.01$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Methodology : implicit finite difference method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_2(i,j)=\frac{T_1 + F_O (T_2 (i-1, j) + T_2(i+1, j) + T_2(i, j+1) + T_2(i,j-1)}{1 + 4 F_O}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $F_O$ is the fourier number $\alpha \Delta t/ \Delta x^2$.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Step 1: Pre-smoothing- using the implicit finitie difference method shown above. The problem is smoothen by 2 cycles of red-balck gauss seidel.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Step 2: compute residual- the residual is computed using&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$\operatorname{res}=\frac{T_1 + F_O [T_2(i-1,j) + T_2(i+1,j) + T_2(i,j+1) + T_2(i, j-1)]}{1 + 4 F_O} - T_2(i,j)$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Step 3: The residual is restricted using full weightage&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Step 4: The residual equation is solved with the intial guess of the error to be zero. I solve it by using&#xA;$$\operatorname{error}(i,j)= \frac{\operatorname{res}(i,j) + F_O [\operatorname{error} (i-1,j) + \operatorname{error}(i+1,j) + \operatorname{error}(i,j-1)+ \operatorname{error}(i,j+1)]}{1 + 4 F_O}$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if I used the correct equation to solve the residual equation and as the gird size decrease $\Delta x$ will increase therefore the Fourier number ($F_O$) will change . So I believe this maybe the problem to my program but i'm not sure as there are not alot of information regarding this. (I have been looking for a few weeks). I will really appreciate the help from you guys. If anyone needs any other information, please comment.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mMO3i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mMO3i.png&quot; alt=&quot;Error reduction rate&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23632" LastEditorUserId="23632" LastEditDate="2017-03-18T09:50:26.993" LastActivityDate="2017-05-30T01:47:52.553" Title="V-cycle Multigrid for 2D transient heat transfer on a square plate using finite difference" Tags="|finite-difference|multigrid|heat-transfer|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26423" PostTypeId="1" AcceptedAnswerId="26428" CreationDate="2017-03-17T11:02:07.353" Score="20" ViewCount="3416" Body="&lt;p&gt;I am aware about that inverting a matrix to solve a linear system is not a good idea, since it is not as accurate and as efficient as directly solving the system or using LU, Cholesky or QR decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have not been able to check this with a practical example. I have tried this code (in MATLAB)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M   = 500;    &#xA;A   = rand(M,M);&#xA;A   = real(expm(1i*(A+A.')));&#xA;b   = rand(M,1);&#xA;&#xA;x1  = A\b;&#xA;x2  = inv(A)*b;&#xA;&#xA;disp(norm(b-A*x1))&#xA;disp(norm(b-A*x2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the residuals are always of the same order (10^-13).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone provide a practical example in which inv(A)*b is much less inaccurate than A\b? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;------Question update------&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your answers. However, suppose that we have to solve $n$ times a system $Ax = b$, where $A$ is always the same matrix. Consider that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-$A$ is full, and thus $A^{-1}$ requires the same memory storage than $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-The condition number of $A$ is small, hence $A^{-1}$ can be computed with accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case, would not it be more efficient to compute $A^{-1}$ rather than to use a LU decomposition? For example, I have tried this Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%Set A and b:&#xA;M           = 1000; &#xA;A           = rand(M,M);&#xA;A           = real(expm(1i*(A+A.')));&#xA;b           = rand(M,1);&#xA;&#xA;%Times we solve the system:&#xA;n           = 3000;&#xA;&#xA;%Performing LU decomposition:&#xA;disp('Performing LU decomposition')&#xA;tic&#xA;[L,U,P]     = lu(A);&#xA;toc&#xA;fprintf('\n')&#xA;&#xA;%Solving the system n times with LU decomposition:&#xA;optsL.LT    = true;   %Options for linsolve&#xA;optsU.UT    = true;&#xA;disp('Solving the system n times using LU decomposition')&#xA;tic&#xA;for ii=1:n&#xA;    x1      = linsolve(U, linsolve(L,P*b,optsL) , optsU);&#xA;end&#xA;toc&#xA;fprintf('\n')&#xA;&#xA;%Computing inverse of A:&#xA;disp('Computing inverse of A')&#xA;tic&#xA;Ainv        = inv(A);&#xA;toc&#xA;fprintf('\n')&#xA;&#xA;%Solving the system n times with Ainv:&#xA;disp('Solving the system n times with A inv')&#xA;tic&#xA;for ii=1:n&#xA;    x2  = Ainv*b;&#xA;end&#xA;toc&#xA;fprintf('\n')&#xA;&#xA;disp('Residuals')&#xA;disp(norm(b-A*x1))&#xA;disp(norm(b-A*x2))&#xA;&#xA;disp('Condition number of A')&#xA;disp(cond(A))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a matrix with condition number about 450, the residuals are $O(10^{-11})$ in both cases, but it takes 19 seconds for solving the system n times using the LU decomposition, whereas using the inverse of A it only takes 9 seconds. &lt;/p&gt;&#xA;" OwnerUserId="23633" LastEditorUserId="23633" LastEditDate="2017-03-18T12:39:29.187" LastActivityDate="2017-03-18T20:16:45.697" Title="Practical example of why it is not good to invert a matrix" Tags="|matrix|accuracy|inverse|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="26424" PostTypeId="2" ParentId="26423" CreationDate="2017-03-17T16:26:31.350" Score="15" Body="&lt;p&gt;Here is a quick example which is very practical related to memory usage in PDEs. When one discretizes the Laplace operator, $\Delta u$, for example, in the Heat Equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_t = \Delta u + f(t,u) .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve it numerically, one ends up with sparse matrices $A$, and a method of lines discretization then solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_t = Au + f(t,u) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The canonical 1D example is the Strang matrix. Implicit methods will need to invert $A$, or some form of $I-\gamma A$. For a discretization with 5 points in space, let's see what happens to this operator. We can generate it easily in Julia using &lt;code&gt;SpecialMatrices.jl&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; using SpecialMatrices&#xA;julia&amp;gt; Strang(5)&#xA;5×5 SpecialMatrices.Strang{Float64}:&#xA; 2.0  -1.0   0.0   0.0   0.0&#xA;-1.0   2.0  -1.0   0.0   0.0&#xA; 0.0  -1.0   2.0  -1.0   0.0&#xA; 0.0   0.0  -1.0   2.0  -1.0&#xA; 0.0   0.0   0.0  -1.0   2.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This special matrix is tridiagonal (many other discretizations are banded, and thus still sparse). This means that you can store it by only storing 3 arrays, or in this case, it can be &quot;lazy&quot; (i.e., no array is needed and you can have a &quot;pseudo-array&quot; type which generates the values on demand). This means that even for very large spatial discretizations with $n$ points, sparse matrix formats can store this in $\mathcal{O}(3n)$ memory (and lazy formats can do this in $\mathcal{O}(1)$!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, let's say we want to invert the matrix. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; inv(collect(Strang(5)))&#xA;5×5 Array{Float64,2}:&#xA; 0.833333  0.666667  0.5  0.333333  0.166667&#xA; 0.666667  1.33333   1.0  0.666667  0.333333&#xA; 0.5       1.0       1.5  1.0       0.5&#xA; 0.333333  0.666667  1.0  1.33333   0.666667&#xA; 0.166667  0.333333  0.5  0.666667  0.833333&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that the inverse of this sparse matrix is dense. Thus, if we do not know the analytical solution of the inverse (which is true for most sparse matrices arising from PDE discretizations), the amount of memory that will be required for the inverse is $\mathcal{O}(n^2)$. This means that the inverse will take up a massive amount of extra memory, and your memory limit will thus be determined by the size of the dense inverse matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, direct solver methods of &lt;code&gt;\&lt;/code&gt; and iterative solvers like those in &lt;code&gt;IterativeSolvers.jl&lt;/code&gt; will solve $Ax=b$ without ever computing $A^{-1}$, and thus only have the memory requirement of having to store $A$. This can greatly expand the size of the PDEs you can solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others have mentioned, condition number and numerical error is another reason, but the fact that the inverse of a sparse matrix is dense gives a very clear &quot;this is a bad idea&quot;.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="6775" LastEditDate="2017-03-17T21:20:46.727" LastActivityDate="2017-03-17T21:20:46.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26425" PostTypeId="2" ParentId="26398" CreationDate="2017-03-17T19:54:58.487" Score="2" Body="&lt;p&gt;I don't fully understand where the problem is. Where the bonds are supposed to be, S-Se bonds? Like this, those distances don't seem much longer than Cd-S.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, you should not be troubled by the &quot;sticks&quot; connecting atoms in the visual representation: there are thresholds telling the program when to show them, and when not to show them. Actually, in GaussView, you can add sticks between the atoms you want, even if they are at 10 Ang distance.&#xA;Additionally, if you are using a Quantum Mechanical method (even if semi-empirical) sticks have no meaning at all, as they are only a representation for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you should check is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the actual Se-S distance, and how it compares to literature values.&lt;/li&gt;&#xA;&lt;li&gt;whether the optimisation is proceeding correctly, and it is not oscillating between the same structures.&lt;/li&gt;&#xA;&lt;li&gt;possibly increase the level of theory if 1 is not satisfactory.&lt;/li&gt;&#xA;&lt;li&gt;if 3, use a suitable basis set for the metal, as probably &quot;regular&quot; basis sets are going to describe it poorly.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23566" LastActivityDate="2017-03-17T19:54:58.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26426" PostTypeId="2" ParentId="26418" CreationDate="2017-03-18T00:38:00.150" Score="10" Body="&lt;p&gt;You can evaluate this using the Björck-Pereyra algorithm for solving Vandermonde systems, because you are evaluating $b^\top V^{-1}$ with $b=(2,0,\frac23,0,\frac25,0,\ldots)$, and the algorithm is known to be forward-stable (see &lt;em&gt;Error analysis of the Björck-Pereyra algorithms for solving Vandermonde systems&lt;/em&gt; by Nick Higham,  &lt;a href=&quot;http://www.maths.manchester.ac.uk/~higham/narep/narep108.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.maths.manchester.ac.uk/~higham/narep/narep108.pdf&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: it seems that this analysis relies on the property that $0\leq x_1&amp;lt;x_2&amp;lt;\cdots &amp;lt;x_n$ (which is equivalent to $V$ being totally positive), as well as the elements of $b$ having alternating signs (which ensures there's no catastrophic cancellation in the subtractions below), in which case the errors are independent of the condition number, and it will still work in the more general case, without $0\leq x_1$, but the error bounds will be different. In any case, it takes $O(n^2)$ time, and avoids the problem with evaluating/integrating $L_i$'s, so it might be worth it even then, but I hadn't realized this point when I started writing this answer. You might be able to just map $x\mapsto \frac12(x+1)$ if that works for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a small Julia program to check that this actually works, and it gives $O(\epsilon_{\mathrm{mach}})$ relative errors.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;module VandermondeInverse&#xA;&#xA;using SpecialMatrices&#xA;&#xA;function main(n=8)&#xA;  X = Rational{BigInt}[k//(n-1) for k=0:n-1]&#xA;  # X = convert(Vector{Rational{BigInt}}, linspace(-1, 1, n))&#xA;  x = convert(Vector{Float64}, X)&#xA;&#xA;  A = convert(Matrix{Rational{BigInt}}, Vandermonde(X))&#xA;  b = [i%2==0 ? 2//(i+1) : 0 for i=0:n-1]&#xA;  println(&quot;Norm: &quot;, norm(A, Inf))&#xA;  println(&quot;Norm of inverse: &quot;, norm(inv(A), Inf))&#xA;  println(&quot;Condition number: &quot;, cond(convert(Matrix{Float64}, A)))&#xA;  ans = A'\b&#xA;  println(&quot;True answer: &quot;, ans)&#xA;&#xA;  B = convert(Matrix{Float64}, A)&#xA;  c = convert(Vector{Float64}, b)&#xA;&#xA;  println(&quot;Linear solve: &quot;, norm((B'\c - ans)./ans, Inf))&#xA;&#xA;  d = vec(c')&#xA;  for k=1:n, l=n:-1:k+1&#xA;    d[l] -= x[k]*d[l-1]&#xA;  end&#xA;&#xA;  for k=n-1:-1:1, l=k:n&#xA;    if l &amp;gt; k&#xA;      d[l] /= x[l]-x[l-k]&#xA;    end&#xA;    if l &amp;lt; n&#xA;      d[l] -= d[l+1]/(x[l+1] - x[l-k+1])&#xA;    end&#xA;  end&#xA;  println(&quot;Neville elimination: &quot;, norm((d-ans)./ans, Inf))&#xA;&#xA;  nothing&#xA;end&#xA;&#xA;end&#xA;&#xA;V = VandermondeInverse&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; V.main(14)&#xA;Norm: 14.0&#xA;Norm of inverse: 1.4285962612120493e10&#xA;Condition number: 5.2214922998851654e10&#xA;True answer: Rational{Int64}[3202439130233//2916000,-688553801328731//52390800,19139253128382829//261954000,-196146528919726853//785862000,6800579086408939//11642400,-43149880138884259//43659000,32567483200938127//26195400,-7339312362348889//6237000,48767438804485271//58212000,-69618881108680969//157172400,44275410625421677//261954000,-2308743351566483//52390800,11057243346333379//1571724000,-209920276397//404250]&#xA;Linear solve: 1.5714609387747318e-8&#xA;Neville elimination: 1.3238218572356314e-15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If &lt;code&gt;X&lt;/code&gt; isn't positive like in this test, then it seems the relative errors are of the same order as with a regular linear solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Why $b^\top V^{-1}$?&lt;/em&gt; It's actually a very useful common trick for working with polynomials of all types, but especially the Lagrange interpolating polynomials, converting the problem to a matrix form. The condition that defines $L_i$ is that $L_i(x_j)=\delta_{ij}$. Let $\alpha_{jk}$ be the coefficients of $L_k$, i.e., &#xA;$$ L_k(x) = \sum_{j,k} \alpha_{j,k}x^j = (1,x,x^2,\ldots,x^n)^\top (\alpha_{0k},\ldots,\alpha_{nk}),&#xA;$$&#xA; and $L$ be the whole matrix of coefficients, arranged by columns:&#xA;$$ L = \begin{pmatrix} \alpha_{00}&amp;amp; \cdots &amp;amp; \alpha_{0n}\\\vdots &amp;amp;&amp;amp;\vdots\\ \alpha_{n0}&amp;amp;\cdots&amp;amp;\alpha_{nn} \end{pmatrix}. $$&#xA;Because of the definition of $L_k$ above as a vector product, multiplying $L$ on the left by $(1,x,\ldots,x^n)$ yields&#xA;$$ (1,x,x^2,\ldots,x^n)L = (L_0(x),L_1(x),\ldots,L_n(x)). $$&#xA;Using the condition $L_k(x_j)=\delta_{jk}$, this means that&#xA;$$ \begin{pmatrix} 1&amp;amp;x_0&amp;amp;x_0^2&amp;amp;\cdots&amp;amp;x_0^n\\ \vdots\\ 1&amp;amp;x_n&amp;amp;x_n^2&amp;amp;\cdots&amp;amp;x_n^n \end{pmatrix} L = I, $$&#xA;so $L = V^{-1}$, where $V$ is the Vandermonde matrix of $x_j$'s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, since $\int_{-1}^{1}x^k\,\mathrm{d}x = \frac{1+(-1)^k}{k+1}$, we have&#xA;$$ \int_{-1}^{1}L_k(x)\,\mathrm{d}x = \sum_j \alpha_{jk}\frac{1+(-1)^k}{k+1} = (2,0,\tfrac23,0,\tfrac25,0,\ldots)^\top (\alpha_{0k},\ldots,\alpha_{nk}).$$&#xA;So the $n+1$ numbers you are looking for, for $k=0\ldots n$, are given by $(2,0,\tfrac23,0,\ldots)^\top L$, where $L=V^{-1}$ is the inverse of the Vandermonde matrix.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-03-20T22:42:05.260" LastActivityDate="2017-03-20T22:42:05.260" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26427" PostTypeId="1" CreationDate="2017-03-18T12:32:25.223" Score="1" ViewCount="154" Body="&lt;p&gt;If we take a 1st order polynomial approximation in each cell, we can find the (2x2) -mass matrix, differentiation matrix, and flux matrix through the integration of Lagrange polynomials. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am not able to comprehend the simplified equation which we obtain on using centered numerical flux. It seems that we get the result same as up-winding/ down-winding if we ensure continuity of solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to me, if we multiply the element equation by its mass inverse we should be getting the final form, but it doesn't seem to be the case. I would be glad if someone points out my mistake in understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am attaching the content, which I am referring to: - &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yiVUg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yiVUg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LnKt7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LnKt7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JP4dX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JP4dX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11977" LastEditorUserId="11977" LastEditDate="2017-03-18T12:45:02.823" LastActivityDate="2017-03-18T12:45:02.823" Title="How do we derive the elemental equation for Discontinuous Galerkin method using Centered Numerical Flux?" Tags="|numerical-analysis|fluid-dynamics|discontinuous-galerkin|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26428" PostTypeId="2" ParentId="26423" CreationDate="2017-03-18T15:59:57.790" Score="13" Body="&lt;p&gt;Normally there are some principal reasons to prefer solve a linear system respect to use the inverse. Briefly:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;problem with the conditional number (@GoHokies comment)&lt;/li&gt;&#xA;&lt;li&gt;problem in the sparse case (@ChrisRackauckas answer)&lt;/li&gt;&#xA;&lt;li&gt;efficiency (@Kirill comment)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Anyway, as @ChristianClason remarked in comments, can be some cases where the use of the inverse is a good option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the note/article by Alex Druinsky, Sivan Toledo, &lt;em&gt;&lt;a href=&quot;https://arxiv.org/abs/1201.6035&quot; rel=&quot;noreferrer&quot;&gt;How Accurate is inv(A)*b?&lt;/a&gt;&lt;/em&gt; there are some consideration about this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According with the paper the principal reason for the general preference to use solve the linear system is inside these two estimates ($x$ is the true solution):&#xA;$$&#xA;\text{inverse}  \quad  || x_V - x || \leq O(\kappa^2(A) \epsilon_{machine})\\&#xA;\text{ backward stable (LU, QR,...)}   \quad  || x_{backward-stable} - x || \leq O(\kappa(A) \epsilon_{machine})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the estimate for the inverse can be improve, under some condition over the inverse, see theorem 1 in the paper, but $x_V$ can be conditionally accurate and not backward stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper show the cases when this happens ($V$ is the inverse)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(1) $V$ is not a good right inverse, or&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;(2) $V$ is such a poor left inverse that $||x_V||$ is much smaller than $||x||$, or&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;(3)  the projection of $b$ on the left singular vectors of $A$ associated with small singular values is small.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So the opportunity to use or not the inverse depend by the application, may you can check the article an see if you case satisfies the condition to obtain the&#xA;backward stability or if you don't need of it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, in my opinion, is more safe solve the linear system.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="713" LastEditDate="2017-03-18T20:16:45.697" LastActivityDate="2017-03-18T20:16:45.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26429" PostTypeId="1" AcceptedAnswerId="26430" CreationDate="2017-03-18T21:06:54.700" Score="1" ViewCount="590" Body="&lt;p&gt;I'm using a simple version of Verlet integration for a particle–particle interaction system with collisions. At the end of each iteration, I integrate like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;verletX = ( 1.92*x[i] - 0.92*x_old[i] ) + fx[i]*0.002;&#xA;verletY = ( 1.92*y[i] - 0.92*y_old[i] ) + fy[i]*0.002;&#xA;x_old[i] = x[i];&#xA;y_old[i] = y[i];&#xA;x[i] = verletX;&#xA;y[i] = verletY;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can I increase its order of error (to have more precision for decreasing timesteps) using even more old data of X and Y? For now it should be less precise than a RK4, but I’m sure it’s much better than a simple Euler integration. For example can I use the data from the last eight iterations to reach RK4 levels?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I’m not trying RK4 is that the calculation of forces takes too much time. However, having a copy of the old data and updating it does not. Also I don’t need exactness; I just need the system stability such that particles stay in fixed points rather than having explosive close range leaps.&lt;/p&gt;&#xA;" OwnerUserId="2014" LastEditorUserId="7574" LastEditDate="2017-03-19T08:19:48.553" LastActivityDate="2017-03-19T08:19:48.553" Title="Higher-order Verlet integration" Tags="|computational-physics|time-integration|runge-kutta|particle|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26430" PostTypeId="2" ParentId="26429" CreationDate="2017-03-19T04:57:47.547" Score="2" Body="&lt;p&gt;Use multistep methods in this case. See the Adams-Bashforth method if it's nonstiff, or Adams-Bashforth-Moulton methods, or if its stiff BDF methods. These use past timepoints like you want in order to increase the order of accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would highly recommend checking out something like Sundials which has variable timestep plus variable order methods. This helps a lot with the stability of these methods and lets them get a lot more efficiency out.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-03-19T04:57:47.547" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26433" PostTypeId="1" CreationDate="2017-03-19T16:11:18.743" Score="2" ViewCount="574" Body="&lt;p&gt;I am looking to learn Fortran for Scientific Computing, which IDE setup for this exists that supports debugging, code completion capabilities and offers granular Running time Analytics. I am interested in a free software that accomplishes the above requirements and is available on Mac.&lt;/p&gt;&#xA;" OwnerUserId="23645" LastEditorUserId="23645" LastEditDate="2017-03-21T23:19:31.010" LastActivityDate="2017-03-21T23:19:31.010" Title="IDE for Scientific computing using Fortran" Tags="|fortran|" AnswerCount="0" CommentCount="9" ClosedDate="2017-03-20T18:34:54.293" ContentLicense="CC BY-SA 3.0" />
  <row Id="26435" PostTypeId="1" CreationDate="2017-03-19T21:54:07.907" Score="1" ViewCount="371" Body="&lt;p&gt;My goal is to numerically solve the convection-diffusion equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial C}{\partial t} = \nabla (D \nabla C) - \nabla (v C)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C$ is concentration and $D$ is the diffusivity constant.&#xA;I am simulating this on a small flat plate model of porous media.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V66DL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V66DL.jpg&quot; alt=&quot;Picture of flat plate model&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming that fluid flow goes across the plates through the porous media from right to left and gas flow occurs from top to bottom inside the porous media and mass transport occurs between the gas and the fluid. I am modeling this in a commercial solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assigned a mass flow to the fluid inlet and a constant pressure to the fluid outlet to model the fluid flow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I keep having trouble modeling the gas flow. I tried setting a gradient by assigning a constant pressure at the inlet and outlet of the porous media but that does not seem correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm beginning to think that there is a more fundamental problem with my approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; is it correct to assign a gas pressure or a mass flux at the inlet and outlet to the porous domain? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would a mass flux be required at the interfaces to the fluid regions as well? Is there a specific approach for modeling gas flux?&lt;/p&gt;&#xA;" OwnerUserId="23654" LastEditorUserId="9667" LastEditDate="2017-03-19T22:23:47.167" LastActivityDate="2017-10-07T12:35:00.293" Title="Mass transport in porous media" Tags="|pde|fluid-dynamics|boundary-conditions|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26436" PostTypeId="2" ParentId="14828" CreationDate="2017-03-20T00:28:55.153" Score="0" Body="&lt;p&gt;You can try fitting your data by a 10-dimensional B-spline hypersurface by penalized least-squares fitting as suggested in this article: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0010465513001434&quot; rel=&quot;nofollow noreferrer&quot;&gt;N. Whitehorn, J. van Santen, S. Lafebre, Computer Physics Communications (2013) 184: 9. doi:10.1016/j.cpc.2013.04.008&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This yields a smooth function that is very cheap to evaluate and provides you an inexpensive gradient. This could bring you close to your global minimum where a local search shouldn't be a problem.&lt;/p&gt;&#xA;" OwnerUserId="23195" LastActivityDate="2017-03-20T00:28:55.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26440" PostTypeId="1" AcceptedAnswerId="27529" CreationDate="2017-03-20T19:25:52.177" Score="0" ViewCount="728" Body="&lt;p&gt;An element $a \in \mathbb{Z}_n$ is a quadratic residue in $\mathbb{Z}_n$ if it's congruent to some perfect square modulo $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an efficient algorithm to find all quadratic residues in $\mathbb{Z}_n$?&lt;br&gt;&#xA;$n$ is composite and we know all it's factors if that helps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have one more restriction: $n$ = $p_1 p_2 \dots p_k$, where $p_i$ are distinct odd primes and $p_i \equiv 3 \pmod 4$. Can we get something in this case?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I use the following approach at the moment:&lt;br&gt;&#xA;Iterate over $\left\lfloor\frac{n}{2}\right\rfloor + 1$ perfect squares starting from $0$ and store them as we go. The problem is that it becomes slow quickly as $n$ grows. Here's the code example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;&#xA;int main() {&#xA;    int n = 7 * 11;&#xA;    int qr = 0;&#xA;    int step = 1;&#xA;&#xA;    for (int i = 0;i &amp;lt;= n / 2;i++) {&#xA;        printf(&quot;qr: %i\n&quot;, qr);&#xA;&#xA;        // perform some operation on qr here&#xA;        // e.g. store it somewhere to access later&#xA;&#xA;        qr = (qr + step) % n;&#xA;        step += 2;&#xA;    }&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23528" LastEditorUserId="23528" LastEditDate="2017-03-24T08:52:27.747" LastActivityDate="2017-07-28T17:16:51.187" Title="Algorithm to find all quadratic residues modulo $n$" Tags="|algorithms|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26443" PostTypeId="1" CreationDate="2017-03-20T23:35:46.090" Score="0" ViewCount="368" Body="&lt;p&gt;I would like to create a parallel solver using petsc4py as one component of a larger application. I can easily get PETSc.COMM_WORLD.getSize() to be 4 by calling &lt;/p&gt;&#xA;&#xA;&lt;p&gt;mpiexec -n 4 python my_prog.py&lt;/p&gt;&#xA;&#xA;&lt;p&gt;however, this creates 4 applications, whereas I only want the PETSc solve in parallel. Currently it is only using one process, and provides performance essentially the same as scipy's spsolve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can refactor such that the KSP solve is its own program and spawn that from the main program, using IO between them to send the matrices etc, but that seems less than ideal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible for petsc4py to create its own mpi procs via petsc4py.init or somesuch, or specify the number of partitions programatically? Or am I missing something more fundamental to the whole question?&lt;/p&gt;&#xA;" OwnerUserId="23669" LastActivityDate="2018-03-16T06:12:01.967" Title="setting COMM_WORLD size from petsc4py without mpiexec command line" Tags="|petsc|mpi|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26444" PostTypeId="2" ParentId="26443" CreationDate="2017-03-20T23:56:28.863" Score="1" Body="&lt;p&gt;Starting with all the data on one process is not scalable, though it may be practical if you only want a modest amount of parallelism.  It is possible to use mpi4py spawn, but if you really want the rest of your program to use only one process and you don't want the extra processes to exist except during your solve, I would just launch an independent parallel job for the PETSc solve.  If you don't mind having the (idle) processes around when you're not using them, you can just have all your logic on rank=0; PETSc will communicate values assembled there to an appropriate process.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2017-03-20T23:56:28.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26445" PostTypeId="1" CreationDate="2017-03-21T00:57:14.073" Score="6" ViewCount="647" Body="&lt;p&gt;I have good background of finite element methods and continuum mechanics and I am familiar with fluid mechanics. My aim is to understand the required theory and to write my own simple codes using finite element, finite volume or other methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for references for computational fluid dynamics. I am beginner in this field, so I am looking for 'beginner references.' The reference should also outline necessary background in an accessible manner for computational fluid dynamics, such as Navier-Stokes equations, finite volume method derivation etc. (I personally think many books outline prerequisites before going for advance topics.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did search and found out two books:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;T. J. Chung, computational fluid dynamics&lt;/li&gt;&#xA;&lt;li&gt;Versteeg and Malalasekera, An Introduction to Computational Fluid &#xA;Dynamics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Chung's book seems to give a lot of background information on FEM, FDM, FVM. But I am not sure how the book will turn out after investing time and efforts. Suggestions are welcome.&lt;/p&gt;&#xA;" OwnerUserId="23671" LastEditorUserId="9667" LastEditDate="2017-03-21T01:13:35.430" LastActivityDate="2018-02-15T03:07:32.393" Title="Reference request for computational fluid dynamics" Tags="|fluid-dynamics|reference-request|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26446" PostTypeId="2" ParentId="26445" CreationDate="2017-03-21T09:41:01.573" Score="1" Body="&lt;p&gt;Unfortunately this is a vast question and can have many responses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most CFD books tend to focus more on the author's interests. In addition most books would only do either finite volume or finite difference, and that too on (mostly) Cartesian grids. There are a few reasons for this but probably the most obvious reason would be that they are conceptually the simplest, and probably easiest to implement (for Cartesian grids). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of personal preferences, for finite volume I would recommend Computational Fluid Dynamics: Principles and Applications by Blazek since it helped me when I needed to implement a finite volume method. I am making a generalization here, but Blazek, and other books will only get you started. When I needed to implement WENO schemes, which are really not that complicated, I had to go straight to papers. To go further, at least for compressible flows you would definitely have to go through Riemann Solvers and Numerical Methods for Fluid Dynamics by Toro. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For finite difference and more generally as a reference I would recommend Numerical Computation of Internal and External Flows: The Fundamentals of Computational Fluid Dynamics - Hirsch since it covers most topics in a reasonable amount of detail. Also CFD by J. D Anderson would be good as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know about finite element methods, but if you want to use Discontinuous Galerkin methods, Nodal Discontinuous Galerkin Methods by Hesthaven, Warburton is a great book that includes sections on how to write the code, although its in Matlab. &lt;/p&gt;&#xA;" OwnerUserId="16735" LastActivityDate="2017-03-21T09:41:01.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26447" PostTypeId="1" CreationDate="2017-03-21T09:53:18.120" Score="1" ViewCount="318" Body="&lt;p&gt;I would like to use MKL to solve a sparse linear system. I chose the &lt;a href=&quot;https://software.intel.com/en-us/node/470302&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSS (Direct Sparse Solver)&lt;/a&gt; interface, which implements the following steps:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//(1).define the non-zero structure of the matrix&#xA;dss_define_structure(handle, sym, rowIndex, nRows, nCols, columns, nNonZeros);&#xA;//(2).reorder the matrix&#xA;dss_reorder(handle, opt, 0);&#xA;//(3).factor the matrix&#xA;dss_factor_real(handle, type, values);&#xA;//(4).get the solution vector&#xA;dss_solve_real(handle, opt, rhs, nRhs, solValues);&#xA;//(5).deallocate solver storage&#xA;dss_delete(handle, opt);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;According to my test, the DSS uses a column-major ordering. That means&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//column = 3&#xA;{ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is equivalent to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{{1,  6,  11},&#xA; {2,  7,  12},&#xA; {3,  8,  13},&#xA; {4,  9,  14}&#xA; {5, 10,  15}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For instance, there is a sparase linear system $\mathbf A_{5\times 5}\mathbf X_{5\times 3}=\mathbf B_{5\times 3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf A$ is symmetric sparse array&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/O5cgq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//A stored with CSR3 format&#xA;rowIndex = { 0, 5, 6, 7, 8, 9 };&#xA;columns = { 0, 1, 2, 3, 4, 1, 2, 3, 4 };&#xA;values = { 9, 1.5, 6, .75, 3, 0.5, 12, .625, 16 };&#xA;//B&#xA;rhs[5*3] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}&#xA;//X&#xA;solValues[15]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By calling the above DSS interface, the &lt;code&gt;solValues&lt;/code&gt; is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{-326.333, 983, 163.417, 398, 61.5, &#xA; -844.667, 2548, 423, 1028, 159, &#xA;-1363, 4113, 682.583, 1658, 256.5}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Namely&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf X_{5\times 3}=\left(&#xA;\begin{array}{ccc}&#xA; -326.333 &amp;amp; -844.667 &amp;amp; -1363 \\&#xA; 983 &amp;amp; 2548 &amp;amp; 4113 \\&#xA; 163.417 &amp;amp; 423 &amp;amp; 682.583 \\&#xA; 398 &amp;amp; 1028 &amp;amp; 1658 \\&#xA; 61.5 &amp;amp; 159 &amp;amp; 256.5 \\&#xA;\end{array}&#xA;\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my application, the matrix is row-major. How to deal with this problem?&lt;/p&gt;&#xA;" OwnerUserId="14845" LastEditorUserId="14845" LastEditDate="2017-03-22T07:38:19.370" LastActivityDate="2017-03-22T07:38:19.370" Title="How to reshape matrix into row-major order for MKL DSS?" Tags="|linear-solver|sparse-matrix|intel-mkl|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26448" PostTypeId="1" CreationDate="2017-03-21T10:55:01.440" Score="1" ViewCount="133" Body="&lt;p&gt;I had to code a program which calculates Inverse of a matrix by Gauss-Jordan Inverse method , I was trying to analyse the program and then code it myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the link &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hullooo.blogspot.in/2011/02/matrix-inversion-by-gauss-jordan.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hullooo.blogspot.in/2011/02/matrix-inversion-by-gauss-jordan.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand it all except two things -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I don't understand how it does the row transformations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) While doing the Jordan method why the $j$ loop starts over and ends till last implying we are always pivoting that is moving the largest element up and hence I am unable to see a stable way of doing the left side part of the Augmented matrix to be an Identity matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference to Jordan method I am using this link -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mathsisfun.com/algebra/matrix-inverse-row-operations-gauss-jordan.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathsisfun.com/algebra/matrix-inverse-row-operations-gauss-jordan.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is great!&#xA;Am I leaving something or interpreting in a wrong way?&lt;/p&gt;&#xA;" OwnerUserId="23297" LastEditorUserId="23297" LastEditDate="2017-03-21T11:11:50.833" LastActivityDate="2017-03-21T11:11:50.833" Title="Problem in analyzing the program of Gauss Jordan Inverse problem" Tags="|numerical-analysis|c++|matrix|c|inverse|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26450" PostTypeId="1" AcceptedAnswerId="26451" CreationDate="2017-03-21T18:27:26.720" Score="2" ViewCount="827" Body="&lt;p&gt;I have to solve a system $Ax^{(n)} = b^{(n)}$ many times, $A$ being a sparse (pentadiagonal in most part of its structure), unsymmetric, constant matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am performing the LU factorization with rows and columns permutations, i.e., $PAQ = LU$, and then I am using this factorization to compute $x^{(k)}$ in the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x^{(n)} = Q*(U \backslash L \backslash (P*b^{(n)}))$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read recently about iterative methods for sparse systems, but I am not very familiar with them. Could you please tell if there is any iterative method that could perform faster and if it is already implemented in Matlab?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks, Manu&lt;/p&gt;&#xA;" OwnerUserId="23633" LastEditorUserId="23633" LastEditDate="2017-03-21T18:33:57.607" LastActivityDate="2017-03-21T19:48:11.763" Title="Fastest way to solve a sparse unsymmetric system many times" Tags="|sparse-matrix|iterative-method|linear-system|matrix-factorization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26451" PostTypeId="2" ParentId="26450" CreationDate="2017-03-21T18:58:38.560" Score="6" Body="&lt;p&gt;Generally speaking, for many right-hand side (RHS) problems, a direct solver is a more feasible solution for several reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Major computations are performed during the factorization step (which is done only once for all RHS), and the solution find (for each RHS) is much cheaper.&lt;/li&gt;&#xA;&lt;li&gt;Direct solvers do not suffer from poor conditioning of the matrix or, in other words, you don't have to look for a nice preconditioner (required for iterative solvers) ensuring $\mathcal O(1)$ iterations for every RHS (relatively easy) and every matrix (much harder) resulting from a numerical simulation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;NB. Sure, if the matrix is extremely ill-conditioned, the direct solver would also suffer, however, direct solvers extend &quot;the dynamic range&quot; of problems that can be solved without re-formulating a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of things that I should mention:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Usually, it is ill-advised to use an inverse based on LU-decomposition to find a solution. You might want to use a back-substitution LU subroutine that would be more efficient. From your formula, it is unclear if you are doing it in the right way.&lt;/li&gt;&#xA;&lt;li&gt;Since your matrix is sparse and has a stable structure, I would look into sparse direct solvers as opposed to full direct solver. It should speed up the computations significantly.&lt;/li&gt;&#xA;&lt;li&gt;&lt;s&gt;In your case, I would consider doing partial pivoting ($PA=LU$) only, as opposed to full-pivoting ($PAQ=LU$). That might speed up the computations.&lt;/s&gt;&lt;/li&gt;&#xA;&lt;li&gt;Connecting a high-performance LAPACK library to MATLAB is also an option.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Regarding sparse iterative solvers:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You can try different iterative solvers, including GMRES, BiCGStab, TFQMR, etc. Your task would be to provide a fast sparse matrix-vector product to them. Most of them are already included in MATLAB. And if you are using sparse matrix storage there, using them should not be too hard.&lt;/li&gt;&#xA;&lt;li&gt;Repeating an item from above about advantages of a direct solver: you would have to implement a decent preconditioner for all those methods that would reduce the number of iterations they require to converge. One choice would be using incomplete LU (iLU) as a preconditioner. There are more sophisticated and easier options available.&lt;/li&gt;&#xA;&lt;li&gt;Using iterative solvers is a bit tricky. You have to figure out how well is you system conditioned, what is a reasonable choice for a preconditioner, what is a good tolerance to use with an iterative solver (accuracy of the obtained solution), have a feel what kind of problems (in the subset of problems you are solving using numerical simulations) cause non- or slow-convergence of your iterative algorithm.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2017-03-21T19:48:11.763" LastActivityDate="2017-03-21T19:48:11.763" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26453" PostTypeId="1" CreationDate="2017-03-22T12:36:03.957" Score="1" ViewCount="134" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Lenstra%E2%80%93Lenstra%E2%80%93Lov%C3%A1sz_lattice_basis_reduction_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lenstra–Lenstra–Lovász (LLL)&lt;/a&gt; algorithm is a famous one for lattice reduction. But concerning its application, I am always baffled by one point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm applies equally well to irrational and rational numbers, right? But, it seems that in the literature, people always confine themselves to the rational case. That is, the initial basis vectors are always like $[101, 394, 5677]$, but never $[\sqrt{2}, \sqrt{345}, \sqrt{13}]$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One should note that the irrational case is the most generic case, as in the Diophantine approximation problem.   &lt;/p&gt;&#xA;" OwnerUserId="23683" LastEditorUserId="9667" LastEditDate="2017-03-22T19:48:54.777" LastActivityDate="2017-03-22T20:17:20.257" Title="Why rational numbers in the Lenstra–Lenstra–Lovász algorithm?" Tags="|algorithms|computational-geometry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26455" PostTypeId="2" ParentId="26453" CreationDate="2017-03-22T20:17:20.257" Score="3" Body="&lt;p&gt;&lt;em&gt;I am by no means an expert on LLL, but I have worked with it before. Please correct me if this answer is in some way incorrect.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define a basis $\beta = \{v_1,v_2,\ldots,v_n\}$ for $\mathbb{R}^n$. Then the lattice $L$ generated by $\beta$ is the set of &lt;em&gt;integer&lt;/em&gt; linear combinations of $\beta$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;L = \{ m_1v_1 + \cdots + m_nv_n : m_i \in \mathbb{Z} \} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means the $\beta$-coordinate representation of vectors in $L$ are entirely integers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basis $B$ is a set of vectors in $L$ that spans $L$ by integer linear combinations of the vectors in $B$. Since each of the vectors in $B$ are in $L$, they must have &lt;em&gt;integer coordinates&lt;/em&gt; with respect to $\beta$, but they may not have integer entries as vectors in $\mathbb{R}^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make this concrete, consider the lattice $L$ spanned by $\beta = \{ (\sqrt{2},0), (0,\sqrt{3}) \}$. Then $B = \{(\sqrt{2},0),(\sqrt{2},\sqrt{3}) \}$ is a basis for $L$. Note that the vectors in $B$ have irrational entries. The coordinates of $(\sqrt{2},0)$ in the basis $\beta$ is $(1,0)$ and the coordinates of $(\sqrt{2},\sqrt{3})$ in $\beta$ is $(1,1)$. So while the vectors in $B$ do not have integer values, they do have integer coordinates with respect to the basis $\beta$.&lt;/p&gt;&#xA;" OwnerUserId="16757" LastActivityDate="2017-03-22T20:17:20.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26456" PostTypeId="1" AcceptedAnswerId="26470" CreationDate="2017-03-22T22:01:26.867" Score="1" ViewCount="214" Body="&lt;p&gt;Suppose we have a straight line in Cartesian space such that&#xA;$$ x_k = x_0 + k \delta x, \quad \quad y_k = y_0 + k \delta y, \quad \quad z_k = z_0 + k \delta z $$&#xA;where $k$ can take any real value. If we project that line into cylindrical polar coords then&#xA;$$ R_k = \sqrt{ x_{k}^{2} + y_{k}^{2} }. $$&#xA;Now suppose there exists some rectangle in the $(R,z)$ plane having a centre at $(R_c, z_c)$ and sides of length $L_R$ and $L_z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rectangle is aligned with the $R$ and $z$ axes such that a point on the curve is inside the rectangle if $|R_c - R_k| &amp;lt; L_R / 2$ &lt;em&gt;and&lt;/em&gt; $|z_c - z_k| &amp;lt; L_z / 2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can anyone suggest an efficient way to compute whether the projection of the line onto the $(R,z)$ plane intersects the rectangle?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6700" LastEditorUserId="6700" LastEditDate="2017-03-22T23:07:38.123" LastActivityDate="2017-03-24T23:20:49.290" Title="Efficiently determine whether a curve intersects a given rectangle?" Tags="|numerical-analysis|algorithms|computational-geometry|performance|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26459" PostTypeId="1" AcceptedAnswerId="26465" CreationDate="2017-03-23T03:23:01.730" Score="5" ViewCount="196" Body="&lt;p&gt;Given a polyhedron $\mathbf{Ax} \leq \mathbf{b}$, how to find the largest hypercuboid, with unknown center $\mathbf{x_{0}}$ and side lengths $2\epsilon_{i}$, which are aligned along the co-ordinate axes that fit inside?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By 'largest', I'm flexible with the definition. It could be $\prod_{i} \epsilon_{i}$ or $\sum_{i} \epsilon_{i}$, the latter being a linear objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is quite well known for ellipsoids and hypercubes, but the fact that there are unequal side lengths, which themselves are unknowns, seems to make it a tricky one.&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="20417" LastEditDate="2017-03-24T17:17:18.833" LastActivityDate="2017-03-25T17:59:16.560" Title="Largest hypercuboid inside a polyhedron" Tags="|convex-optimization|linear-programming|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26461" PostTypeId="1" AcceptedAnswerId="26462" CreationDate="2017-03-23T14:58:47.510" Score="8" ViewCount="293" Body="&lt;p&gt;What is a simple way to compute &lt;span class=&quot;math-container&quot;&gt;$10^x - 1$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is close to zero?&lt;/p&gt;&#xA;&lt;p&gt;Using exponentiation and then subtraction isn't good enough, because the fractional part is very small compared to the one that we subtract away.&lt;/p&gt;&#xA;" OwnerUserId="23700" LastEditorUserId="127" LastEditDate="2020-07-10T04:58:32.787" LastActivityDate="2020-07-10T04:58:32.787" Title="Compute powers close to zero" Tags="|numerics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="26462" PostTypeId="2" ParentId="26461" CreationDate="2017-03-23T15:07:54.963" Score="14" Body="&lt;p&gt;Use the Taylor series expansion of $a^x$ about $x=0$ and evaluate a small number of terms for $a=10$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a^x - 1 \approx x\log(a) + \frac{1}{2}x^2\log^2(a)+\cdots.$$&lt;/p&gt;&#xA;" OwnerUserId="7241" LastEditorUserId="7241" LastEditDate="2017-04-01T00:36:19.263" LastActivityDate="2017-04-01T00:36:19.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26465" PostTypeId="2" ParentId="26459" CreationDate="2017-03-24T03:35:04.607" Score="4" Body="&lt;p&gt;It turns out that the problem has quite an elegant solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the hypercuboid be defined by $\mathbf{l} \leq \mathbf{x} \leq \mathbf{u}$ instead of using the more cumbersome $\mathbf{x_{0}}$ and $\epsilon_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a hypercuboid to lie within a polyhedron, each of its vertices must lie inside, that is, satisfy the inequality $\mathbf{Ax} \leq \mathbf{b}$. Note that every vertex consists of co-ordinates either from $\mathbf{l}$ or $\mathbf{u}$. Thus, there is one vertex which is $\mathbf{u}$ for all the positive entries of $\mathbf{A}$ and $\mathbf{l}$ for all negative entries. This can be written more succinctly as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define $\mathbf{A^{+}_{ij}} = max(0,\mathbf{A}_{ij})$ and $\mathbf{A^{-}_{ij}} = max(0,-\mathbf{A}_{ij})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hypercuboid constraint can thus be written as &#xA;$\mathbf{A^{+}u - A^{-}l} \leq \mathbf{b}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For maximum volume, one could use the concave function, geometric mean($\mathbf{u-l}$), as the objective function. This can be easily solved in CVX, for example.&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="15120" LastEditDate="2017-03-25T17:59:16.560" LastActivityDate="2017-03-25T17:59:16.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26466" PostTypeId="2" ParentId="26166" CreationDate="2017-03-24T13:14:41.423" Score="6" Body="&lt;p&gt;$$\rm x = B^{-1} (2A + I) (C^{-1} + A) b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Left-multiplying both sides by $\rm B$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rm B x = (2A + I) (C^{-1} + A) b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\rm y:= C^{-1} b$. Hence,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rm B x = (2A + I) (C^{-1} b + A b) = (2A + I) (y + A b) = (2A + I) y + (2A + I) A b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and, thus, we obtain a linear system of $2n$ equations in $2n$ unknowns&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix} \mathrm B &amp;amp; -(2 \mathrm A + \mathrm I)\\ \mathrm O&amp;amp; \mathrm C\end{bmatrix} \begin{bmatrix} \mathrm x\\ \mathrm y  \end{bmatrix} = \begin{bmatrix} (2 \mathrm A + \mathrm I) \mathrm A \mathrm b\\ \mathrm b\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be solved using Gaussian elimination.&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2017-03-24T14:02:14.533" LastActivityDate="2017-03-24T14:02:14.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26467" PostTypeId="1" AcceptedAnswerId="26474" CreationDate="2017-03-24T16:06:30.553" Score="2" ViewCount="1039" Body="&lt;p&gt;For a flow simulation I am trying to reproduce a specific screw element design for an intermeshing co-rotating twin-screw extruder. I am using gmsh (v2.16) where the element is made from a 2D surface with a combined translational and rotational &lt;code&gt;extrude&lt;/code&gt; command. The 2D sufrace is a simple shape as seen on the left in the picture below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7HzCV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7HzCV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shape is extruded translationally by 1 unit and rotationally by 360 degrees simulateneously to get the shape on the right which results in a screw thread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the resulting mesh contains 'distortions' which show up as holes along the extruded lines running across the length of the screw. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried refining the mesh, but the holes do not disappear. How do i get a smooth surface using this extrusion process? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R50LH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R50LH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The geometry file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// inputs&#xA;length = 1;&#xA;pitch = 1;&#xA;alpha_i = Pi/3; // intermesh angle [0,Pi/2]&#xA;alpha_f = alpha_i; // flight angle&#xA;alpha_t = Pi/2 - alpha_f; // tip angle&#xA;alpha_r = alpha_t; // root angle&#xA;&#xA;// based on unit screw diameter&#xA;Cd = Cos(alpha_i/2);  // centerline distance&#xA;Dr = 2*Cd-1; // root diameter&#xA;Hc = 1-Cd; // channel depth&#xA;Ih = Sin(alpha_i/2)/2; // intermesh height&#xA;&#xA;lc = 0.1;&#xA;offset_x = Cd/2; // from x=0&#xA;offset_y = 0; // from y=0&#xA;&#xA;// first screw&#xA;&#xA;c1p0 = newp; // center point &#xA;Point(c1p0) = {-offset_x+0, offset_y+0, 0, lc};&#xA;&#xA;angle = alpha_t/2;&#xA;xt1 = 1/2*Cos(angle);&#xA;yt1 = 1/2*Sin(angle);&#xA;&#xA;angle = angle + alpha_f;&#xA;xr1 = Dr/2*Cos(angle);&#xA;yr1 = Dr/2*Sin(angle);&#xA;&#xA;angle = angle + alpha_r;&#xA;xr2 = Dr/2*Cos(angle);&#xA;yr2 = Dr/2*Sin(angle);&#xA;&#xA;angle = angle + alpha_f;&#xA;xt2 = 1/2*Cos(angle);&#xA;yt2 = 1/2*Sin(angle);&#xA;&#xA;angle = angle + alpha_t;&#xA;xt3 = 1/2*Cos(angle);&#xA;yt3 = 1/2*Sin(angle);&#xA;&#xA;angle = angle + alpha_f;&#xA;xr3 = Dr/2*Cos(angle);&#xA;yr3 = Dr/2*Sin(angle);&#xA;&#xA;angle = angle + alpha_r;&#xA;xr4 = Dr/2*Cos(angle);&#xA;yr4 = Dr/2*Sin(angle);&#xA;&#xA;angle = angle + alpha_f;&#xA;xt4 = 1/2*Cos(angle);&#xA;yt4 = 1/2*Sin(angle);&#xA;&#xA;c1p1 = newp; Point(c1p1) = {-offset_x+xt1, offset_y+yt1, 0, lc};&#xA;c1p2 = newp; Point(c1p2) = {-offset_x+xr1, offset_y+yr1, 0, lc};&#xA;c1p3 = newp; Point(c1p3) = {-offset_x+xr2, offset_y+yr2, 0, lc};&#xA;c1p4 = newp; Point(c1p4) = {-offset_x+xt2, offset_y+yt2, 0, lc};&#xA;c1p5 = newp; Point(c1p5) = {-offset_x+xt3, offset_y+yt3, 0, lc};&#xA;c1p6 = newp; Point(c1p6) = {-offset_x+xr3, offset_y+yr3, 0, lc};&#xA;c1p7 = newp; Point(c1p7) = {-offset_x+xr4, offset_y+yr4, 0, lc};&#xA;c1p8 = newp; Point(c1p8) = {-offset_x+xt4, offset_y+yt4, 0, lc};&#xA;&#xA;c1a = newc; Circle(c1a) = {c1p2,c1p0,c1p3};&#xA;c1b = newc; Circle(c1b) = {c1p4,c1p0,c1p5};&#xA;c1c = newc; Circle(c1c) = {c1p6,c1p0,c1p7};&#xA;c1d = newc; Circle(c1d) = {c1p8,c1p0,c1p1};&#xA;&#xA;// flank areas&#xA;alpha = -1/2*((xt1-xr1)*(xt1-xr1)+(yt1-yr1)*(yt1-yr1))/(yt1-yr1);&#xA;beta = -(xt1-xr1)/(yt1-yr1);&#xA;x0 = xr1 + alpha*beta/(1+beta*beta)*(1-Sqrt(1-(1+1/(beta*beta))*(1-Cd*Cd/(alpha*alpha))));&#xA;y0 = yr1 - alpha - beta*(xr1-x0);&#xA;&#xA;p = newp; Point(p) = {-offset_x+x0, offset_y+y0, 0, lc};&#xA;c1e = newc; Circle(c1e) = {c1p1,p,c1p2};&#xA;&#xA;p = newp; Point(p) = {-offset_x-x0, offset_y+y0, 0, lc};&#xA;c1f = newc; Circle(c1f) = {c1p3,p,c1p4};&#xA;&#xA;p = newp; Point(p) = {-offset_x-x0, offset_y-y0, 0, lc};&#xA;c1g = newc; Circle(c1g) = {c1p5,p,c1p6};&#xA;&#xA;p = newp; Point(p) = {-offset_x+x0, offset_y-y0, 0, lc};&#xA;c1h = newc; Circle(c1h) = {c1p7,p,c1p8};&#xA;&#xA;ll = newl; Line Loop(ll) = {c1f, c1b, c1g, c1c, c1h, c1d, c1e, c1a};&#xA;s = newl; Plane Surface(s) = {ll};&#xA;&#xA;Extrude {{0,0,length}, {0,0,1}, {-offset_x, -offset_y, 0}, 2*Pi*pitch*length} {&#xA;  Surface{s};&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: Using @nicoguaro answer I was able to produce the following geometry in FREECAD:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y73Qn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y73Qn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as shown the ellipses are joined by curves which really should be straight lines. Refining by adding more slices improves this but also increases the computational cost significantly. This is only one screw element of a screw containing approximatly 30 of these elements. This quickly becomes too much to render. Anyway to connect with straight lines rather than curves?&lt;/p&gt;&#xA;" OwnerUserId="18017" LastEditorUserId="18017" LastEditDate="2017-03-29T11:10:39.910" LastActivityDate="2020-07-30T17:42:19.627" Title="Combined translational and rotational meshing in gmsh" Tags="|mesh-generation|mesh|gmsh|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26468" PostTypeId="1" CreationDate="2017-03-24T16:07:57.683" Score="1" ViewCount="537" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;I've asked &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26468/global-numbering-of-the-vertices-in-a-triangulation-consisting-of-quadratic-lagr&quot;&gt;this question on Computational Science SE&lt;/a&gt; too&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Consider the following triangulation of a rectangle $(0,a)\times(0,b)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gMpR2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gMpR2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have two types of triangles:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I5lpR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I5lpR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I enumerate the rectangles(consisting of two triangles) starting at the bottom left corner from left to right and bottom to top. The triangles are enumerated in the following way: Each triangle of type A has the same index as its corresponding cell and each index of a triangle of type B is the sum of the index of its corresponding cell &quot;$+$ the total number of cells&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the triangulation depicted above, the cell at the bottom left corner has index $1$ and the contained triangles of type A and B have the index 1 and 17, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The local numbering of the vertices is shown the second figure. Starting from the bottom left corner, from left to right and bottom to top, I build the global indexing of the vertices. For example, in the triangulation depicted above, the vertex at the bottom left corner has index $1$, its horizontal neighbor has index $2$ and its vertical neighbor has index $6$.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This indexing is suitable if each triangle is considered to be a linear Lagrange element (evaluation at the vertices). How do I need to modify my indexing, if I want to use quadratic Lagrange elements instead (evaluation at the vertices plus evaluation at the midpoints of the edges)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would like to use the following local numbering for the vertices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/buxpO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/buxpO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not clear to me how I should build the global numbering of the additional nodes (i.e. the midpoints of the edges). I'm not sure, but I could imagine that the choice of the numbering influences the linear system of equations in a crucial way (I've read, for example, that a tuple of increasing local vertex numbers should correspond to a tuple of global vertex numbers).&lt;/p&gt;&#xA;" OwnerUserId="23411" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2017-05-26T16:51:48.213" Title="Global numbering of the vertices in a triangulation consisting of quadratic Lagrange elements" Tags="|finite-element|mesh-generation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26469" PostTypeId="1" AcceptedAnswerId="26473" CreationDate="2017-03-24T18:36:03.587" Score="4" ViewCount="356" Body="&lt;p&gt;I submit a &lt;strong&gt;Reduce&lt;/strong&gt; command and get results like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Reduce[t0 &amp;gt;= 0 &amp;amp;&amp;amp; -9 + t0 - 5 t1 + 154 &amp;gt;= 0 &amp;amp;&amp;amp; -5 + t0 - 3 t1 + 77 &amp;gt;= &#xA;   0, {t0, t1}, integers]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Reduce::bdomv: Warning: integers is not a valid domain specification. Assuming it is a variable to eliminate. &#xA;(0 &amp;lt;= t0 &amp;lt;= 75/2 &amp;amp;&amp;amp; t1 &amp;lt;= (72 + t0)/3) || (t0 &amp;gt; 75/2 &amp;amp;&amp;amp; &#xA;   t1 &amp;lt;= (145 + t0)/5)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put simply, what algorithm is being used to obtain these results?&lt;/p&gt;&#xA;" OwnerUserId="22133" LastEditorDisplayName="user20857" LastEditDate="2017-03-25T08:44:08.310" LastActivityDate="2017-03-25T08:44:08.310" Title="How does Mathematica compute 'Reduce'?" Tags="|algorithms|mathematica|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26470" PostTypeId="2" ParentId="26456" CreationDate="2017-03-24T23:15:48.057" Score="1" Body="&lt;p&gt;Given a &lt;em&gt;line&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal L := \left\{ \begin{bmatrix} x_0\\ y_0\\ z_0\end{bmatrix} + t \begin{bmatrix} v_x\\ v_y\\ v_z\end{bmatrix} : t \in \mathbb R \right\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a &lt;em&gt;cylindrical shell&lt;/em&gt; of thickness $r_2 - r_1$ and height $z_2 - z_1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal S := \left\{ \begin{bmatrix} x\\ y\\ z\end{bmatrix} : (r_1^2 \leq x^2 + y^2 \leq r_2^2) \land (z_1 \leq z \leq z_2) \right\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we would like to determine whether $\mathcal L \cap \mathcal S \neq \emptyset$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intersecting line $\mathcal L$ with the cylindrical shell $\mathcal S$, we obtain the following system of $4$ inequalities in $t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{rl} (x_0 + v_x t)^2 + (y_0 + v_y t)^2 &amp;amp;\geq r_1^2\\ (x_0 + v_x t)^2 + (y_0 + v_y t)^2 &amp;amp;\leq r_2^2\\ z_0 + v_z t &amp;amp;\geq z_1\\ z_0 + v_z t &amp;amp;\leq z_2\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can decide the feasibility of this system of linear and quadratic inequalities using &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantifier_elimination&quot; rel=&quot;nofollow noreferrer&quot;&gt;quantifier elimination&lt;/a&gt;. &lt;a href=&quot;https://reference.wolfram.com/language/tutorial/Quantifiers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematica&lt;/a&gt; has the function &lt;code&gt;Resolve&lt;/code&gt;. There are also &lt;a href=&quot;https://www.usna.edu/CS/qepcadweb/B/QEPCAD.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;QEPCAD&lt;/a&gt; and &lt;a href=&quot;http://www.redlog.eu&quot; rel=&quot;nofollow noreferrer&quot;&gt;Redlog&lt;/a&gt;. Of course, we can also do quantifier elimination by hand, but it is rather cumbersome.&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2017-03-24T23:20:49.290" LastActivityDate="2017-03-24T23:20:49.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26472" PostTypeId="1" CreationDate="2017-03-25T01:03:22.317" Score="1" ViewCount="65" Body="&lt;p&gt;I'm dealing a lot with large sparse linear operators these days and I'm quite new to them. A lot of the matrices I deal with originate with only a few unique integers, however, there are lots of them. As a result I have tons of memory locked up for the same values and fetching them from RAM seems like quite the bottle neck. It seems like a well designed for loop could replace these matrices for matrix multiplication, but writing custom loops for that seems like a chore. Are there tools or techniques to do this?&lt;/p&gt;&#xA;" OwnerUserId="23647" LastActivityDate="2017-03-25T01:03:22.317" Title="Reduce large sparse linear operators to memory efficient loops?" Tags="|linear-algebra|sparse-matrix|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26473" PostTypeId="2" ParentId="26469" CreationDate="2017-03-25T01:48:12.673" Score="8" Body="&lt;p&gt;Preamble: &quot;integers&quot; -&gt; Integers in your problem statement, Mathematica uses CamelType.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method depends on your problem, naturally. In your case it appears to be the Contejean–Devie method. A [buried] excerpt from the &lt;a href=&quot;https://reference.wolfram.com/language/tutorial/SomeNotesOnInternalImplementation.html&quot; rel=&quot;noreferrer&quot;&gt;Mathematica documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For &lt;strong&gt;polynomial systems&lt;/strong&gt;, Reduce uses cylindrical algebraic decomposition for real domains and Gröbner basis methods for complex domains.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;With &lt;strong&gt;algebraic functions&lt;/strong&gt;, Reduce constructs equivalent purely polynomial systems. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;With &lt;strong&gt;transcendental functions&lt;/strong&gt;, Reduce generates polynomial systems composed with transcendental conditions, then reduces these using functional relations and a database of inverse image information. With piecewise functions, Reduce does symbolic expansion to construct a collection of continuous systems.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For &lt;strong&gt;univariate transcendental equations&lt;/strong&gt;, Reduce represents solutions as transcendental Root objects. For exp-log function equations over the reals, Reduce uses an exact exp-log root isolation algorithm. For analytic function equations over bounded real or complex domains, Reduce finds roots using validated numerical methods. For elementary functions, both the number and the location of roots are fully validated using interval arithmetic methods. For non-elementary functions, validation of the number of roots depends on correctness of numeric integration and validation of the location of roots depends on correctness of accuracy estimates provided by the Wolfram Language's significance arithmetic.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For &lt;strong&gt;Diophantine systems&lt;/strong&gt;, Reduce solves linear equations using Hermite normal form, and linear inequalities using Contejean–Devie methods. For univariate polynomial equations, it uses an improved Cucker–Koiran–Smale method, while for bivariate quadratic equations, it uses Hardy–Muskat–Williams methods for ellipses and classical techniques for Pell and other cases. Reduce includes specialized methods for about 25 classes of Diophantine equations, including the Tzanakis–de Weger algorithm for Thue equations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's also worth mentioning that the original problem is rather straightforward to reduce by hand:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$t_0 \ge 0 \quad \text{and} \quad t_1 \le \cases{ \frac{t_0}{3} + 24 \quad \text{if } \, t_0 \le 32, \\&#xA; \frac{t_0}{5} + 29 \quad \text{otherwise}}$$&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastEditorDisplayName="user20857" LastEditDate="2017-03-25T02:11:42.900" LastActivityDate="2017-03-25T02:11:42.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26474" PostTypeId="2" ParentId="26467" CreationDate="2017-03-25T01:59:12.717" Score="4" Body="&lt;p&gt;I cannot visualize your geometry properly using Gmsh, or export it. I generated something similar using FreeCAD. Maybe you can modify this script for your purposes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division, print_function&#xA;import FreeCAD as FC&#xA;import Draft&#xA;from numpy import sin, cos, pi&#xA;&#xA;nturns = 1&#xA;nslices = 20&#xA;length = 10&#xA;width = 20&#xA;height = 60&#xA;dz = height/nslices&#xA;place = FC.Placement()&#xA;dang = 2*pi*nturns/(nslices - 1)&#xA;slices = []&#xA;for k in range(nslices):&#xA;    ang = k*dang&#xA;    place.Rotation = (0, 0, sin(ang/2), cos(ang/2))&#xA;    place.Base = FC.Vector(-length/2, -width/2, k*dz)&#xA;    slices.append(Draft.makeEllipse(10, 20, place))&#xA;&#xA;doc = FC.ActiveDocument&#xA;loft = doc.addObject('Part::Loft','Loft')&#xA;loft.Sections = slices&#xA;loft.Solid = True&#xA;loft.Ruled = True&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2020-07-30T17:42:19.627" LastActivityDate="2020-07-30T17:42:19.627" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="26476" PostTypeId="1" CreationDate="2017-03-25T19:46:17.453" Score="1" ViewCount="68" Body="&lt;p&gt;I'm trying to do discretization to the following system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{{\partial \rho }}{{\partial t}} =  - \frac{{\partial \rho }}{{\partial x}}u - \rho \frac{{\partial u}}{{\partial x}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{{\partial u}}{{\partial t}} =  - u\frac{{\partial u}}{{\partial x}} - \frac{\partial }{{\partial x}}\left[- {\frac{1}{{\sqrt \rho  }}\frac{{{\partial ^2}(\sqrt \rho  )}}{{\partial {x^2}}}} \right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the boundary conditions: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho (x,t = 0),\,\,\,\,u(x,t = 0)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first equation everything is good:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{{\rho _i^{n + 1} - \rho _i^n}}{{\Delta t}} =  - \frac{{\rho _{i + 1}^n - \rho _i^n}}{{\Delta x}}u_i^n - \rho _i^n\frac{{u_{i + 1}^n - u_i^n}}{{\Delta x}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I can explicitly solve for $\rho _i^{n + 1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for the second equation. i'm struggling with the term:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial }{{\partial x}}\left[ { - \frac{1}{{\sqrt \rho  }}\frac{{{\partial ^2}}}{{\partial {x^2}}}\left( {\sqrt \rho  } \right)} \right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So my questions are as follows:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should I use the chain rule in a straight forward manner like this:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial }{{\partial x}}\left[ { - \frac{1}{{\sqrt \rho  }}\frac{{{\partial ^2}}}{{\partial {x^2}}}\left( {\sqrt \rho  } \right)} \right] = \frac{1}{\rho }\left[ { - \frac{1}{{2{\rho ^2}}}{{\left( {\frac{{\partial \rho }}{{\partial x}}} \right)}^3} + \frac{1}{\rho }\left( {\frac{{\partial \rho }}{{\partial x}}} \right)\left( {\frac{{{\partial ^2}\rho }}{{\partial {x^2}}}} \right) - \frac{1}{{2\rho }}\left( {\frac{{{\partial ^3}\rho }}{{\partial {x^3}}}} \right)} \right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and afterwards discretize the derivatives using forward Euler like in the first equation ?&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;If I'm using the approach above, how should I calculate the third derivative ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should I use some other clever way to discretize the term ? maybe calculating the square root of $\rho$ before hand ? I don't know if it's more efficient that way&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23717" LastEditorUserId="9667" LastEditDate="2017-03-26T02:23:49.780" LastActivityDate="2017-03-26T02:23:49.780" Title="Discretization of a multi-function term" Tags="|numerical-analysis|finite-difference|hyperbolic-pde|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26477" PostTypeId="2" ParentId="26468" CreationDate="2017-03-25T21:40:30.350" Score="2" Body="&lt;h1&gt;First approach: using pattern of sparse matrix&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;You are talking about triangular Lagrange FEs of second order, so I suppose you are familiar with ″ of first order—you know how to assemble system resulting from Laplace or Poisson equation using linear elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the resulting matrix we typically use &lt;em&gt;compressed sparse (lower triangular) column&lt;/em&gt; format, CSlC (see &lt;a href=&quot;http://www-users.cs.umn.edu/%7Esaad/software/SPARSKIT/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPARSKIT documentation&lt;/a&gt; or &lt;a href=&quot;http://math.nist.gov/MatrixMarket/formats.html#hb&quot; rel=&quot;nofollow noreferrer&quot;&gt;User’s Guide for the Harwell–Boeing Sparse Matrix Collection&lt;/a&gt; for details).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First we build the pattern of our matrix (vectors &lt;code&gt;colptr&lt;/code&gt; and &lt;code&gt;rowind&lt;/code&gt;), and then we assemble our system. When one has matrix pattern, one also has &lt;strong&gt;numeration for middle nodes (or ribs)&lt;/strong&gt;!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the following mesh and resulting matrix pattern:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EvGtM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EvGtM.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have&#xA;$$&#xA;\text{colptr} = \left\{ 1, 3, 6, 10, 11, 11, 12, 12 \right\}, \\&#xA;\text{rowind} = \left\{ 1, 7, 3, 4, 7, 4, 5, 6, 7, 5, 7 \right\}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to get element $(i, j)$ of CSlC matrix we do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k = colptr[j], colptr[j] + 1, ..., colptr[j+1] - 1&#xA;    if (i == rowind[k]) return values[k]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Typically any vertex in triangulation has degree ~6, so this operation is $O(1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you want to get a number of a rib connecting vertices $i$ and $j$, simply use $k$ from the algorithm above.&#xA;In my picture, for one, a rib connecting vertices 3 and 7 has number 9.  &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Second approach: using elements’ neighbors&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;You are talking about rather structured mesh. In reality, in order to handle more complex geometries, we typically represent a mesh as vector of vertices + vector of elements (triplets of indices of vertices).&#xA;Sometimes it is also convenient to store vector of neighbors (triplets of indices of neighboring elements). For one, &lt;a href=&quot;https://reference.wolfram.com/language/FEMDocumentation/ref/ElementMesh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathematica does so in its mesh abstractions&lt;/a&gt; (see Scope &gt; &quot;ElementConnectivity&quot;). I bet that all popular software tools for mesh generation such as &lt;a href=&quot;http://gmsh.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gmsh&lt;/a&gt; can do this as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, if you have neighbors, it is easy to enumerate ribs iterating over elements. I tend to use this approach in my projects since I have access to neighbors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPD&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;but I could imagine that the choice of the numbering influences the linear system of equations in a crucial way&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Indeed, different numeration of DOFs corresponds to different pattern of matrix. However, one should not care about this at all if one wants to use iterative solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, if one wants to use some kind of sparse direct solver based on LU decomposition, one should care about DOFs numeration since it affects bandwidth (hence, memory) crucially: LU decomposition of CSC-type matrix may be dense; LU decomposition of Skyline matrix preserves pattern of sparsity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as @BillGreene and @anton-menshov mentioned, there are algorithms that minimize bandwidth of your matrix (i.e. renumber DOFs). You should not care about this while assembling / choosing DOFs numeration.   &lt;/p&gt;&#xA;" OwnerUserId="21916" LastEditorUserId="21916" LastEditDate="2017-03-27T15:29:57.680" LastActivityDate="2017-03-27T15:29:57.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26478" PostTypeId="1" CreationDate="2017-03-26T00:18:13.687" Score="3" ViewCount="277" Body="&lt;p&gt;So the problem I am working on is as such. Given the x,y,z data of a periodic object over time (from the origin in 3d space) (need not be uniform), calculate the period of the object (if the data given is periodic and the calculation is possible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the best idea I have been able to come up with so far, is to use some form of interpolation to get a function for the data, but I am not sure how I would go about using the function to calculate the period. Also, if interpolation is the best way forward, would using cubic spline interpolation work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, since the data given is 3d, and so far all the experience I have is working with 2d data for interpolation, if I was to use cubic splines to interpolate, how would the actual implementation work?&lt;/p&gt;&#xA;" OwnerUserId="23257" LastActivityDate="2018-10-17T15:00:26.900" Title="Given x,y,z data of a periodic object, calculate the period of the object (if possible)" Tags="|matlab|numerical-analysis|interpolation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26479" PostTypeId="2" ParentId="26478" CreationDate="2017-03-26T01:41:58.423" Score="1" Body="&lt;p&gt;I actually have to do this for very large data sets on occasion. Since my data are usually somewhat uniformly distributed, I've found the following procedure simple and sufficient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define: $p \equiv data(t=0)$, $\delta(t) \equiv \lVert data(t) - p \rVert_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then find the inflection points of $\delta(t)$ and measure the $\Delta t$ between them. If $\delta(t)$ at the inflection points is sufficiently small (and the variation of $\Delta t$ between multiple neighboring inflection points) then you can be confident you have a good measurement of the period.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a very non-uniform distribution, where the 2-norm doesn't seem like a good metric, you might interpolate the data as you mention and then follow a similar procedure. Something like a spectral representation, maybe spherical harmonics? Though what works best will depend on the object itself (what is it?). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For a spline interpolation you could use a built-in Matlab call, then uniformly sample that interpolation and use the method above. &lt;/li&gt;&#xA;&lt;li&gt;For a spectral representation you might simply use the normalized basis coefficients in the procedure above.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2017-03-26T01:41:58.423" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26481" PostTypeId="1" AcceptedAnswerId="26649" CreationDate="2017-03-26T09:47:52.617" Score="2" ViewCount="1263" Body="&lt;p&gt;exchangers,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have run into a bit of a puzzling problem. To solve an complex eigenvalue-problem, I make use of the LAPACK library function ZHEEV. To test the implementation I used a real symmetric matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\left[&#xA;\begin{array}{c c c c}&#xA;   2 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 \\  &#xA;   0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 2 \\&#xA;   2 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 \\&#xA;   0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 2 \\&#xA;\end{array}&#xA;\right]&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which should have eigenvalues: 2x0, 2x4&#xA;Using ZHEEV I get however 2x -2 and 2x 2.&#xA;Modifying the implementation to use DSYEV on the other hand gives the correct results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am missing something very trivial here, but I do not see what it is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the subroutine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine solvearray(m)                                                                                         &#xA;complex(16), dimension(:,:),allocatable :: dm&#xA;complex(16), dimension(:), allocatable :: freq, work&#xA;complex(16) :: One&#xA;doubleprecision, dimension(:),allocatable :: w, rwork&#xA;integer,dimension(:,:),allocatable, intent(inout) :: m &#xA;integer :: n, k, h, l, info&#xA;&#xA;One = cmplx(1.0,0.0,kind=SELECTED_REAL_KIND(15,300))&#xA;write(*,'(A)',advance='no') 'order of matrix = '   &#xA;read(*,*) n  &#xA;l = 2*n - 1 &#xA;allocate(work(2*l), rwork(3*n-2), w(n))  &#xA;w = 0.0  &#xA;work = 0.0 *One &#xA;rwork = 0.0  &#xA;info = 0&#xA;allocate(dm(1:n,1:n))&#xA;dm(1:n,1:n) = m(1:n,1:n)*One&#xA;write(*,*) 'Setup for lapack workspace has been completed.'&#xA;call ZHEEV('V','U', n, dm, n, w, work, 2*l, rwork, info)&#xA;&#xA;write(*,*) 'This is the eigenvector matrix:'&#xA;do k=1,n  &#xA;    write(*,'(255F8.4)') dm(k,1:n)&#xA;end do &#xA;write(*,*) 'These are the eigenvalues :'&#xA;write(*,'(255F8.4)') w  &#xA;deallocate(m)           &#xA;deallocate(W,Work,rwork)&#xA;end subroutine   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23727" LastEditorUserId="17294" LastEditDate="2017-03-28T18:04:57.443" LastActivityDate="2017-04-17T12:04:01.883" Title="LAPACK: ZHEEV and DSYEV give different eigenvalues for real symmetric matrix" Tags="|eigenvalues|lapack|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26483" PostTypeId="1" AcceptedAnswerId="26505" CreationDate="2017-03-26T17:10:43.820" Score="2" ViewCount="959" Body="&lt;p&gt;I am trying to implement the Laplacian operator using Fourier Transform differentiation (&lt;a href=&quot;https://en.wikibooks.org/wiki/Parallel_Spectral_Numerical_Methods/Finding_Derivatives_using_Fourier_Spectral_Methods#Taking_a_Derivative_in_Fourier_Space&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikibooks.org/wiki/Parallel_Spectral_Numerical_Methods/Finding_Derivatives_using_Fourier_Spectral_Methods#Taking_a_Derivative_in_Fourier_Space&lt;/a&gt;). I am comparing my Fourier differentiation with a periodic Laplacian in real space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When my function is cos(x)*cos(y), the two results (real and Fourier space) agree with each other:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rfCC0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rfCC0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, when my function is sin(x)*sin(y), the Fourier space Laplacian has strange oscillations in it (the general shape remains the same). In the figure below, you can see that the left figure (the Fourier space differentiation) is a little fuzzy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7LkXt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7LkXt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example cross section through the above left figure looks like:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cPWmf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cPWmf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am very confused why these oscillations are appearing, and I would appreciate a solution. I suspect it may have something to do with aliasing, but I have been unable to pinpoint the issue or a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please find my code for implementing the Fourier space Laplacian here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Nx = 512;&#xA;Ny = 256;&#xA;&#xA;Lx = 2 * pi;&#xA;Ly = 4 * pi;&#xA;&#xA;x = linspace(0, Lx, Nx)';&#xA;y = linspace(0, Ly, Ny)';&#xA;&#xA;dx = x(2) - x(1);&#xA;dy = y(2) - y(1);&#xA;&#xA;% Fourier space vectors&#xA;kx = 2*pi/Lx*[0:Nx/2-1 0 -Nx/2+1:-1]'; &#xA;ky = 2*pi/Ly*[0:Ny/2-1 0 -Ny/2+1:-1]';&#xA;&#xA;[x_grid, y_grid] = meshgrid(x, y);&#xA;[kx_grid, ky_grid] = meshgrid(kx, ky);&#xA;&#xA;v = sin( 2*pi*x_grid ./ Lx ) .* sin( 2*pi*y_grid ./ Ly ); % Test Function&#xA;v_hat = fft2(v); % FFT of test function&#xA;&#xA;% Fourier Space Laplacian&#xA;w_L = -( kx_grid.^2 + ky_grid.^2 ) .* v_hat;&#xA;w_L = real(ifft2(w_L));&#xA;&#xA;% Periodic Real Space Laplacian&#xA;L = Laplacian_2D(v, dx, dy);&#xA;&#xA;c = 10; % Because the edges get very weird&#xA;figure; mesh(x(c + 1:end-c),y(c + 1:end-c),w_L(c + 1:end-c,c+1:end-c)); title('Fourier');&#xA;figure; mesh(x(c + 1:end-c),y(c + 1:end-c),L(c + 1:end-c,c+1:end-c)); title('Real Space');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also please find my 2D periodic Laplacian here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function out = Laplacian_2D( f, hx, hy )&#xA;&#xA;    f = f';&#xA;    [r c] = size(f);&#xA;    out = zeros(r, c);&#xA;&#xA;    out( 2 : end - 1, 2 : end - 1 ) = ...&#xA;        ( f( 3 : end, 2 : end - 1 ) - ...&#xA;        2 * f( 2 : end - 1, 2 : end - 1 ) + ...&#xA;        f( 1 : end - 2, 2 : end - 1 ) ) / hx^2 + ...&#xA;        ( f( 2 : end - 1, 3 : end ) - ...&#xA;        2 * f( 2 : end - 1, 2 : end - 1 ) + ...&#xA;        f( 2 : end - 1, 1 : end - 2 ) ) / hy^2 ;&#xA;&#xA;    % Remember (i, j)&#xA;    %  i = row, j = column&#xA;    %  i = y, j = x&#xA;    % but we transposed so we can say i = x, j = y&#xA;&#xA;    % Periodic boundary conditions&#xA;&#xA;    % 4 corners&#xA;    out( 1, 1 ) = ...&#xA;        ( f(2, 1) - 2 * f(1, 1) + f(end, 1) ) / hx^2 + ...&#xA;        ( f(1, 2) - 2 * f(1, 1) + f(1, end) ) / hy^2;&#xA;    out(1, end) = ...&#xA;        ( f(2, end) - 2 * f(1, end) + f(end, end) ) / hx^2 + ...&#xA;        ( f(1, 1) - 2 * f(1, end) + f(1, end - 1) )/ hy^2;&#xA;    out(end, 1) = ...&#xA;        ( f(1, 1) - 2 * f(end, 1) + f(end - 1, 1) ) / hx^2 + ...&#xA;        ( f(end, 2) - 2 * f(end, 1) + f(end, end) ) / hy^2;&#xA;    out(end, end) = ...&#xA;        ( f(1, end) - 2 * f(end, end) + f(end - 1, end) ) / hx^2 + ...&#xA;        ( f(end, 1) - 2 * f(end, end) + f(end, end - 1) ) / hy^2;&#xA;&#xA;    % edges&#xA;    out( 1, 2 : end - 1 ) = ...&#xA;        ( f(2, 2 : end - 1) - 2 * f(1, 2 : end - 1) + f(end, 2 : end - 1) ) / hx^2 + ...&#xA;        ( f(1, 3 : end) - 2 * f(1, 2 : end - 1) + f(1, 1 : end - 2) ) / hy^2;&#xA;    out( end, 2 : end - 1 ) = ...&#xA;        ( f(1, 2 : end - 1) - 2 * f(end, 2 : end - 1) + f(end - 1, 2 : end - 1) ) / hx^2 + ...&#xA;        ( f(end, 3 : end) - 2 * f(end, 2 : end - 1) + f(end, 1 : end - 2) ) / hy^2;&#xA;&#xA;    out( 2 : end - 1, 1) = ...&#xA;        ( f(3 : end, 1) - 2 * f(2 : end - 1, 1) + f(1 : end - 2, 1) ) / hx^2 + ...&#xA;        ( f(2 : end - 1, 2) - 2 * f(2 : end - 1, 1) + f(2 : end - 1, end) ) / hy^2;&#xA;    out( 2 : end - 1, end) = ...&#xA;        ( f(3 : end, end) - 2 * f(2 : end - 1, end) + f(1 : end - 2, end) ) / hx^2 + ...&#xA;        ( f(2 : end - 1, 1) - 2 * f(2 : end - 1, end) + f(2 : end - 1, end - 1) ) / hy^2;&#xA;&#xA;    out = out';&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21221" LastEditorUserId="21221" LastEditDate="2017-03-26T17:19:42.327" LastActivityDate="2017-03-29T17:56:03.157" Title="MATLAB FFT Differentiation" Tags="|matlab|finite-difference|fourier-analysis|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26484" PostTypeId="1" AcceptedAnswerId="29405" CreationDate="2017-03-26T18:04:29.803" Score="2" ViewCount="133" Body="&lt;p&gt;I was thinking about the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose there is a positive semidefinite matrix $X$ of size $n$ (for example, a  kernel). Suppose $X$ can be approximated as a low rank matrix, $X\approx GG^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to find the $k$-largest/smallest values of each row of $X$ without computing it explicitly and going over all the $n^2$ entries? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the solution can approximated or probabilistic... any ideas?&lt;/p&gt;&#xA;" OwnerUserId="3761" LastActivityDate="2018-04-30T05:01:53.793" Title="Using low rank property for maximal/minimal value search (or sorting)" Tags="|linear-algebra|matrix|machine-learning|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26485" PostTypeId="1" CreationDate="2017-03-27T04:04:47.140" Score="1" ViewCount="704" Body="&lt;p&gt;So I have been working on a Smooth Particle Hydrodynamics (SPH) simulation and I am trying to implement the Marching Cubes (MC) algorithm to visualize my SPH output. I understand the MC algorithm well enough from reading about it, however usually it is discussed assuming that you already have some scalar field which you can than use to classify your voxels as being either above, below, or on the boundary surface you are trying to create (in my case this amounts to checking if the voxel is above, below, or on the fluid surface). The key part here is that it is easy to check the value of the scalar field at any cube vertex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, with my SPH output data I don't have some nice scalar field to use. Instead I have a lot of particles distributed (randomly or haphazardly) in my domain. I know each particles density at these random points, but I don't know the density at the MC grid vertices. For example consider the simple 2d example: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R1NE1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R1NE1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of the blue points represent a particle. After performing my SPH solve, I know the position, velocity, and density of each particle at these points. Now, in order to classify each voxel as being above, below, or on the fluid surface I need to know the density at the voxel vertices (i.e. the red points). How is this typically done given you know the density at the blue particles?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial thought was to calculate the density at the red points in the same way as is done in the SPH simulation. In this case I would use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho_{i}(r) = \Sigma_{j}m_{j}W(r -r_{j}, h)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;at each voxel vertex point where $W=0$ when $r &amp;gt; h$. In the picture below this means the green particles would be used in calculating the voxel vertex density value:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JaoK4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JaoK4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my concerns/doubts with this approach are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The $h$ value we use here is probably smaller than the $h$ value we used in the actual SPH simulation because we want a refined voxel grid to allow for refined surface reconstruction.&lt;/li&gt;&#xA;&lt;li&gt;With the above point in mind, even with a modest $64\times{64}\times{64}$ voxel grid in 3D, this results in over 2 million vertex points (64*64*64*8).&lt;/li&gt;&#xA;&lt;li&gt;Calculating the density in the actual SPH simulation this way is somewhat expensive, but at least we might only be doing it for O(100k) particles. Here though as just mentioned above we would need to do it for more than 2 million. This I fear would surpass the actual simulation time.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So my questions are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this approach reasonable/correct? If not how should I classify my voxels?&lt;/li&gt;&#xA;&lt;li&gt;If it is correct, than would you use just a large array of size 64*64*64*8 to store each density value at every voxel vertex?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note: As mentioned in the title I am using CUDA so keep that in mind. &lt;/p&gt;&#xA;" OwnerUserId="7567" LastActivityDate="2017-12-24T01:09:44.343" Title="Classifying Marching Cube voxels from SPH output data using CUDA" Tags="|cuda|sph|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26486" PostTypeId="1" CreationDate="2017-03-27T08:15:48.180" Score="0" ViewCount="55" Body="&lt;p&gt;Consider this basic scheme for particle in cell simulations ( with just short-range interactions ):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;assign particles to disjunct cells&lt;/li&gt;&#xA;&lt;li&gt;for cell $A$ go over neighboring cells $B$ &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;for each particle $a_i$ in $A$ interact with all $b_i$ in $B$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;move all particles&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For GPU is very important memory locality. Therefore it make sense to assign each cell-cell interaction $(A,B)$ to one work-group, &lt;a href=&quot;http://www.browndeertechnology.com/docs/BDT_OpenCL_Tutorial_NBody-rev3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;which can share&lt;/a&gt; &lt;code&gt;__local&lt;/code&gt; buffer of $a_i$s. But it may very well happen that some cells are empty and other are filled with very varying numbers of particles $n$. =&gt; each work group would have to process  very different number of interaction $N = n_A . n_B$ between particle pairs $(a_i,b_i)$. They will have problem to synchronize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess this is some commonplace problem in PIC, GPGPU and parallel computing. But I have seen just introductory tutorials and codes, without much care of optimizations. I would be happy for reference to good and concise learning resources.&lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="4696" LastEditDate="2017-03-27T08:39:29.220" LastActivityDate="2017-03-28T15:43:49.923" Title="Balancing core load when number of particles in cells vary (PIC on GPU)" Tags="|parallel-computing|grid|opencl|particle|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26487" PostTypeId="1" CreationDate="2017-03-27T08:35:22.260" Score="3" ViewCount="234" Body="&lt;p&gt;The Rhie--Chow [&lt;a href=&quot;https://arc.aiaa.org/doi/abs/10.2514/3.8284&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] interpolation seems to be a standard tool in the Finite-Volume discretization of incompressible flows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is commonly defined on the discrete level [&lt;a href=&quot;http://www.tfd.chalmers.se/~lada/comp_fluid_dynamics/postscript_files/chapter_rhie.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the lecture notes [&lt;a href=&quot;http://www.tfd.chalmers.se/~lada/comp_fluid_dynamics/lecture_notes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;] -- Chapter 6, page 56 [&lt;a href=&quot;http://www.tfd.chalmers.se/~lada/comp_fluid_dynamics/postscript_files/chapter_rhie.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;] -- it says that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We find that the Rhie-Chow interpolation is the same as&#xA;  adding a pressure term, which is proportional to a third [spatial] derivative. [...] In the continuity equation [...] the term is proportional to a fourth-order derivative term [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is there any work on how this &lt;em&gt;third or fourth order term&lt;/em&gt; actually looks like? Or is there an approximation of the NSE on the PDE level such that a, say, 1st order FVM discretization would directly lead to the Rhie--Chow interpolated scheme?&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2017-03-27T08:35:22.260" Title="Rhie--Chow interpolation on PDE level" Tags="|fluid-dynamics|reference-request|navier-stokes|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26490" PostTypeId="2" ParentId="26483" CreationDate="2017-03-27T21:43:14.617" Score="1" Body="&lt;p&gt;I just tried running this, and it seems to remove the oscillations, so I'll add this as an answer. I suggest you change your code as follows, before your linspace definition for coordinates, add and modify the lines to read:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; x = linspace(0, Lx, Nx+1)'; x = x(1:Nx);&#xA; y = linspace(0, Ly, Ny+1)'; y = y(1:Ny);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suggest you write out the discrete form of the equations to see why the last point should not be included (it is repeated if you do include it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've updated my answer, thanks to @Richard Zhang, because my original suggestion was susceptible to precision errors.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastEditorUserId="8174" LastEditDate="2017-03-28T16:26:54.550" LastActivityDate="2017-03-28T16:26:54.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26491" PostTypeId="1" CreationDate="2017-03-28T00:18:59.790" Score="7" ViewCount="2538" Body="&lt;p&gt;I was reading about the Crank-Nicolson method, and it is often said that it can produce &quot;spurious oscillations&quot; or that this method is prone to &quot;ringing&quot;, especially for large time step and stiff initial value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious about that phenomena but I can't find any satisfying answer. I read on this website that this could be because the Crank-Nicolson scheme was not L-stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the definition of L-stable ? How this is related to the production of oscillations ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found in &lt;a href=&quot;https://www.researchgate.net/profile/Jorg_Strutwolf/publication/10607072_Damping_of_Crank-Nicolson_error_oscillations/links/53d143cd0cf220632f392c3b/Damping-of-Crank-Nicolson-error-oscillations.pdf&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt; a beginning of an explanation thanks to a Fourier transform (see eq. 9, p. 255), but I really can't understand why the frequency $\varphi$ is only between $-\pi$ and $\pi$, and why $\varphi \approx \pm \pi$ should correspond to &quot;high frequencies&quot;. Can you help me with that, too ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I would like to know if this spurious oscillations problem always appear with a Crank-Nicolson scheme, or does it depend on the equation you discretize ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum up, my questions are :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is the origin of the spurious oscillations when using a Crank-Nicolson scheme ?&lt;/li&gt;&#xA;&lt;li&gt;Does it depend on the equation we consider ?&lt;/li&gt;&#xA;&lt;li&gt;What is the L-stability and in what it is related with the oscillations ? (if it is)&lt;/li&gt;&#xA;&lt;li&gt;Can you help me to understand &lt;a href=&quot;https://www.researchgate.net/profile/Jorg_Strutwolf/publication/10607072_Damping_of_Crank-Nicolson_error_oscillations/links/53d143cd0cf220632f392c3b/Damping-of-Crank-Nicolson-error-oscillations.pdf&quot; rel=&quot;noreferrer&quot;&gt;eq. 9 of this paper&lt;/a&gt;, why $-\pi &amp;lt; \varphi &amp;lt; \pi$ and not $\varphi \in \mathbb{R}$ ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23744" LastEditorUserId="9667" LastEditDate="2017-03-28T01:22:52.167" LastActivityDate="2017-04-07T22:51:05.030" Title="What is the origin of the spurious oscillations in the Crank-Nicolson scheme?" Tags="|stability|crank-nicolson|oscillations|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26492" PostTypeId="2" ParentId="26060" CreationDate="2017-03-28T02:10:57.513" Score="2" Body="&lt;p&gt;A working link is provided in this site: &lt;a href=&quot;http://vaopt.math.uni-goettingen.de/software.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://vaopt.math.uni-goettingen.de/software.php&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21159" LastActivityDate="2017-03-28T02:10:57.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26493" PostTypeId="2" ParentId="26467" CreationDate="2017-03-28T13:16:51.097" Score="-1" Body="&lt;p&gt;I would really use a commercial tool for your complicated geometry. I suggest Comsol.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this does not directly answer your question, but I have recently been in your shoes (I needed to create a 3D mesh for a non-trivial geometry), and I wasted so much time writing the appropriate .geo file. At the end, Gmsh was not even able to recombine the 3D mesh into tetrahedrons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I have been able to produce the desired mesh with Comsol almost immediately.&lt;/p&gt;&#xA;" OwnerUserId="4826" LastActivityDate="2017-03-28T13:16:51.097" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26494" PostTypeId="2" ParentId="26486" CreationDate="2017-03-28T15:43:49.923" Score="1" Body="&lt;p&gt;You ought to be interested in this preprint in which we discuss exactly the sort of questions you seem to be having: &lt;a href=&quot;https://arxiv.org/abs/1612.03369&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1612.03369&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-03-28T15:43:49.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26495" PostTypeId="1" AcceptedAnswerId="26497" CreationDate="2017-03-28T16:49:31.773" Score="0" ViewCount="346" Body="&lt;p&gt;For geotechnical engineering problems, it is common to fix a single component of displacement along a boundary as a Dirichlet boundary condition (roller boundary condition). However, I'm having trouble seeing why this leads to a well-posed problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of unknowns in an elastic problem is equal to the dimension of the problem (i.e. one unknown for each displacement component for the Navier equations). I was under the impression you needed to specify a boundary condition for each unknown in a boundary value problem. Why can we get away with a single boundary condition for a single component of displacement? Is there an implicit stress boundary condition implied when we do this? I've included an example sketch with a common configuration. My intuition says this should have a unique solution but I can't see why mathematically.&lt;a href=&quot;https://i.stack.imgur.com/11wUt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/11wUt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20526" LastEditorUserId="9667" LastEditDate="2017-03-29T04:34:15.637" LastActivityDate="2017-03-29T04:34:15.637" Title="Well-posedness of Elasticity Boundary Conditions" Tags="|pde|boundary-conditions|solid-mechanics|well-posedness|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26496" PostTypeId="2" ParentId="26485" CreationDate="2017-03-28T19:20:49.747" Score="1" Body="&lt;p&gt;It is a different approach but maybe you could do a point cloud surface reconstruction using total variation denoising as in &lt;a href=&quot;https://link.springer.com/article/10.1007%2Fs10915-012-9674-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. They have a demo code that you could use as a base for your implementation,  you can get it &lt;a href=&quot;https://sites.google.com/site/pcsurfacereconstruction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The paper also shows other alternatives, the Poisson surface reconstruction also shows nice results. The poisson surface reconstruction algorithm is implemented in Meshlab.&lt;/p&gt;&#xA;" OwnerUserId="23664" LastEditorUserId="23664" LastEditDate="2017-03-28T19:53:33.010" LastActivityDate="2017-03-28T19:53:33.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26497" PostTypeId="2" ParentId="26495" CreationDate="2017-03-28T19:31:15.953" Score="3" Body="&lt;p&gt;For this problem you have &lt;a href=&quot;https://en.wikipedia.org/wiki/Mixed_boundary_condition&quot; rel=&quot;nofollow noreferrer&quot;&gt;mixed boundary conditions&lt;/a&gt;. Then, you really have 8 boundary conditions for the problem that you present in your sketch. Although, 4 of them are not explicitly written.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Top side:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Non-homogeneous Neumann BC (normal traction): $\sigma_n=C$&lt;/li&gt;&#xA;&lt;li&gt;Homogeneous Neumann BC (tangent traction): $\sigma_t = 0$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Left side:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Homogeneous Dirichlet BC: $u_x=0$&lt;/li&gt;&#xA;&lt;li&gt;Homogeneous Neumann BC (tangent traction): $\sigma_t = 0$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Right side:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Homogeneous Dirichlet BC: $u_x=0$&lt;/li&gt;&#xA;&lt;li&gt;Homogeneous Neumann BC (tangent traction): $\sigma_t = 0$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Bottom side:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Homogeneous Dirichlet BC: $u_y=0$&lt;/li&gt;&#xA;&lt;li&gt;Homogeneous Neumann BC (tangent traction): $\sigma_t = 0$&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think that this might answer your question regarding the number of boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2017-03-28T19:31:15.953" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26499" PostTypeId="1" CreationDate="2017-03-28T21:27:01.730" Score="1" ViewCount="201" Body="&lt;p&gt;I'm trying find the eigenvalues of a matrix A using QR iteration with Householder. I used this code which I found from Cornell University that decomposes QR with Householder.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; A=  [3.56829648331616, -2.19026841424281,0.557474633283884,-0.126072015319561;&#xA;    -3.25393787896303, 4.81285873937612,-2.79568990543933,0.632240894103258&#xA;    1.14081451177546,   -4.13904528762030,  6.19190034744431,-2.98172499459682&#xA;    -0.639981731971678, 2.32194922534595,   -7.91889162828103,6.11802605281534]&#xA;&#xA;&#xA;[m,n] = size(A);&#xA;Q = eye(m); % Orthogonal transform so far&#xA;R = A; % Transformed matrix so far&#xA;&#xA;&#xA;for j = 1:n&#xA;% -- Find H = I-tau*w*w’ to put zeros below R(j,j)&#xA;normx = norm(R(j:end,j));&#xA;s = -sign(R(j,j));&#xA;u1 = R(j,j) - s*normx;&#xA;w = R(j:end,j)/u1;&#xA;w(1) = 1;&#xA;tau = -s*u1/normx;&#xA;% -- R := HR, Q := QH&#xA;R(j:end,:) = R(j:end,:)-(tau*w)*(w'*R(j:end,:));&#xA;Q(:,j:end) = Q(:,j:end)-(Q(:,j:end)*w)*(tau*w)';&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now that I have the correct  R and Q verified through MATLAB, I should be able to find the eigenvalues by multiplying R and Q. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $E=RQ$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the diagonal elements of E must contain the eigenvalues and is symmetric. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, upon doing so its not giving me the correct eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I have to write another iteration to find the eigenvalues with Q and R?&lt;/p&gt;&#xA;" OwnerUserId="23146" LastActivityDate="2017-03-28T21:27:01.730" Title="Eigenvalues using QR iteration" Tags="|linear-algebra|matlab|numerical-analysis|eigenvalues|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26501" PostTypeId="1" CreationDate="2017-03-29T10:37:17.677" Score="1" ViewCount="123" Body="&lt;p&gt;If $I_h$ is the bilinear interpolation operator, i.e.&#xA;$I_h: H^2(\Omega)\rightarrow V_h$, where $V_h=\{v_h\in L^2(\Omega)|v_h|_K\in \mathrm{span}\{1,x,y,xy\},~~K&#xA;\in\mathcal{T}_h\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i know that $\|I_hu\|_{0,\infty}\leq C(u)$ for $u\in H^2(\Omega)$. However, i dont know how to prove it. Can any one give me some suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, if $I_h$ is a nonconforming interpolation operator(such as Crouzeix-Raviart element, the degree of freedom is $\int_l uds$). More precisely,&#xA;$I_h: H^1(\Omega)\rightarrow V_h$, where $V_h=\{v_h\in L^2(\Omega)|v_h|_K\in \mathrm{span}\{1,x,y\},~~K&#xA;\in\mathcal{T}_h\}$. Does the above relationship still hold? If so, how to prove it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the $\|\nabla I_hu\|_{0,\infty}\leq C(u)$ holds, $u$ should belong to $H^{?}(\Omega)$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm very glad to receive some suggestions.&lt;/p&gt;&#xA;" OwnerUserId="7494" LastEditorUserId="7494" LastEditDate="2017-03-31T01:15:03.037" LastActivityDate="2017-03-31T01:15:03.037" Title="How to prove the relationship $\|I_h u\|_{0,\infty}\leq C$ for $u\in H^2(\Omega)$?" Tags="|finite-element|pde|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26502" PostTypeId="2" ParentId="7525" CreationDate="2017-03-29T12:12:34.993" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://www.openscad.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenSCAD&lt;/a&gt; fits all points except volume meshing. It is based on input scripts and generates STL surface meshes.&lt;/p&gt;&#xA;" OwnerUserId="3510" LastActivityDate="2017-03-29T12:12:34.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26504" PostTypeId="1" CreationDate="2017-03-29T14:08:47.657" Score="1" ViewCount="73" Body="&lt;p&gt;I'm studying the Fisker-KPP equation on the line (and in &lt;span class=&quot;math-container&quot;&gt;$]0, 100[$&lt;/span&gt; numerically):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\partial_t u = \Delta_{xx} u + u(1-u)&#xA;$$&lt;/span&gt;&#xA;I notice a behavior I don't understand with a smooth initial condition &lt;span class=&quot;math-container&quot;&gt;$u_0$&lt;/span&gt; that has the following form:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_0(x) =&#xA;\left\{&#xA;\begin{aligned}&#xA;&amp;amp;1 \quad \mbox{if} \quad |x-50| &amp;lt; 10 \\&#xA;&amp;amp;\exp{\left(\frac{1}{3^2} - \frac{1}{||x-50|-13|^2}\right)}&#xA;\left(1 - \exp{\left( -\frac{1}{||x-50|-10|^2}\right)}\right) \quad \mbox{if} \quad 10 &amp;lt; |x-50| &amp;lt; 13 \\&#xA;&amp;amp;0 \quad \mbox{if} \quad |x-50| &amp;gt; 13&#xA;\end{aligned}&#xA;\right.&#xA;$$&lt;/span&gt;&#xA;I'm using as a numerical scheme the Strang splitting, and a code taken from &lt;a href=&quot;https://github.com/zingale/hydro_examples/blob/master/multiphysics/diffusion-reaction.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;&quot;&quot;&#xA;solve a scalar diffusion-reaction equation:&#xA;&#xA; phi_t = kappa phi_{xx} + (1/tau) R(phi)&#xA;&#xA;using operator splitting, with implicit diffusion&#xA;&#xA;M. Zingale&#xA;&quot;&quot;&quot;&#xA;&#xA;#from __future__ import print_function&#xA;&#xA;import numpy as np&#xA;from scipy import linalg&#xA;from scipy.integrate import ode&#xA;#import sys&#xA;import matplotlib.pyplot as plt&#xA;&#xA;def frhs(t, phi, tau):&#xA;    &quot;&quot;&quot; reaction ODE righthand side &quot;&quot;&quot;&#xA;    return 0.25*phi*(1.0 - phi)/tau&#xA;&#xA;def jac(t, phi):&#xA;    return None&#xA;&#xA;def react(gr, phi, tau, dt):&#xA;    &quot;&quot;&quot; react phi through timestep dt &quot;&quot;&quot;&#xA;&#xA;    phinew = gr.scratch_array()&#xA;&#xA;    for i in range(gr.ilo, gr.ihi+1):&#xA;        r = ode(frhs,jac).set_integrator(&quot;vode&quot;, method=&quot;adams&quot;,&#xA;                                         with_jacobian=False)&#xA;        r.set_initial_value(phi[i], 0.0).set_f_params(tau)&#xA;        r.integrate(r.t+dt)&#xA;        phinew[i] = r.y[0]&#xA;&#xA;    return phinew&#xA;&#xA;def diffuse(gr, phi, kappa, dt):&#xA;    &quot;&quot;&quot; diffuse phi implicitly (C-N) through timestep dt &quot;&quot;&quot;&#xA;&#xA;    phinew = gr.scratch_array()&#xA;&#xA;    alpha = kappa*dt/gr.dx**2&#xA;&#xA;    # create the RHS of the matrix&#xA;    R = phi[gr.ilo:gr.ihi+1] + \&#xA;        0.5*alpha*(    phi[gr.ilo-1:gr.ihi] -&#xA;                   2.0*phi[gr.ilo  :gr.ihi+1] +&#xA;                       phi[gr.ilo+1:gr.ihi+2])&#xA;&#xA;&#xA;    # create the diagonal, d+1 and d-1 parts of the matrix&#xA;    d = (1.0 + alpha)*np.ones(gr.nx)&#xA;    u = -0.5*alpha*np.ones(gr.nx)&#xA;    u[0] = 0.0&#xA;&#xA;    l = -0.5*alpha*np.ones(gr.nx)&#xA;    l[gr.nx-1] = 0.0&#xA;&#xA;    # set the boundary conditions by changing the matrix elements&#xA;&#xA;    # homogeneous neumann&#xA;    d[0] = 1.0 + 0.5*alpha&#xA;    d[gr.nx-1] = 1.0 + 0.5*alpha&#xA;&#xA;    # dirichlet&#xA;    #d[0] = 1.0 + 1.5*alpha&#xA;    #R[0] += alpha*0.0&#xA;&#xA;    #d[gr.nx-1] = 1.0 + 1.5*alpha&#xA;    #R[gr.nx-1] += alpha*0.0&#xA;&#xA;    # solve&#xA;    A = np.matrix([u,d,l])&#xA;    phinew[gr.ilo:gr.ihi+1] = linalg.solve_banded((1,1), A, R)&#xA;&#xA;    return phinew&#xA;&#xA;def est_dt(gr, kappa, tau):&#xA;    &quot;&quot;&quot; estimate the timestep &quot;&quot;&quot;&#xA;&#xA;    # use the proported flame speed&#xA;    s = np.sqrt(kappa/tau)&#xA;    dt = gr.dx/s&#xA;    return dt&#xA;&#xA;&#xA;class Grid(object):&#xA;&#xA;    def __init__(self, nx, ng=1, xmin=0.0, xmax=1.0, vars=None):&#xA;        &quot;&quot;&quot; grid class initialization &quot;&quot;&quot;&#xA;&#xA;        self.nx = nx&#xA;        self.ng = ng&#xA;&#xA;        self.xmin = xmin&#xA;        self.xmax = xmax&#xA;&#xA;        self.dx = (xmax - xmin)/nx&#xA;        self.x = (np.arange(nx+2*ng) + 0.5 - ng)*self.dx + xmin&#xA;&#xA;        self.ilo = ng&#xA;        self.ihi = ng+nx-1&#xA;&#xA;        self.data = {}&#xA;&#xA;        for v in vars:&#xA;            self.data[v] = np.zeros((2*ng+nx), dtype=np.float64)&#xA;&#xA;    def fillBC(self, var):&#xA;&#xA;        if not var in self.data.keys():&#xA;            sys.exit(&quot;invalid variable&quot;)&#xA;&#xA;        vp = self.data[var]&#xA;&#xA;        # Neumann BCs&#xA;        vp[0:self.ilo+1] = vp[self.ilo]&#xA;        vp[self.ihi+1:] = vp[self.ihi]&#xA;&#xA;    def scratch_array(self):&#xA;        return np.zeros((2*self.ng+self.nx), dtype=np.float64)&#xA;&#xA;    def initialize(self):&#xA;        &quot;&quot;&quot; initial condition &quot;&quot;&quot;&#xA;&#xA;        phi = self.data[&quot;phi&quot;]&#xA;        length1 = 10.&#xA;        length2 = 13.&#xA;        epsilon   = length2 - length1&#xA;        phi[:] = np.maximum( \&#xA;                 np.exp( 1./epsilon**2 - 1./(np.abs(np.abs(self.x-50.)-length2))**2) * \&#xA;                 ( 1. - np.exp( -1./(np.abs(np.abs(self.x-50.)-length1))**2)) * \&#xA;                 ( self.x &amp;gt;  50.-length2 ) * \&#xA;                 ( self.x &amp;lt;  50.+length2 ) \&#xA;                 , \&#xA;                 ( self.x &amp;gt;= 50.-length1 ) * \&#xA;                 ( self.x &amp;lt;= 50.+length1 ) \&#xA;                 )&#xA;&#xA;def interpolate(x, phi, phipt):&#xA;    &quot;&quot;&quot; find the x position corresponding to phipt &quot;&quot;&quot;&#xA;&#xA;    idx = (np.where(phi &amp;gt;= 0.5))[0][0]&#xA;    xs   = np.array([x[idx-1],   x[idx],   x[idx+1]])&#xA;    phis = np.array([phi[idx-1], phi[idx], phi[idx+1]])&#xA;&#xA;    xpos = 0.0&#xA;&#xA;    for m in range(len(phis)):&#xA;        # create Lagrange basis polynomial for point m&#xA;        l = None&#xA;        n = 0&#xA;        for n in range(len(phis)):&#xA;            if n == m:&#xA;                continue&#xA;&#xA;            if l == None:&#xA;                l = (phipt - phis[n])/(phis[m] - phis[n])&#xA;            else:&#xA;                l *= (phipt - phis[n])/(phis[m] - phis[n])&#xA;&#xA;        xpos += xs[m]*l&#xA;&#xA;    return xpos&#xA;&#xA;def evolve(nx, kappa, tau, tmax, dovis=1, return_initial=0):&#xA;    &quot;&quot;&quot;&#xA;    the main evolution loop.  Evolve&#xA;&#xA;     phi_t = kappa phi_{xx} + (1/tau) R(phi)&#xA;&#xA;    from t = 0 to tmax&#xA;    &quot;&quot;&quot;&#xA;&#xA;    # create the grid&#xA;    gr = Grid(nx, ng=1, xmin = 0.0, xmax=100.0,&#xA;              vars=[&quot;phi&quot;, &quot;phi1&quot;, &quot;phi2&quot;])&#xA;&#xA;    # pointers to the data at various stages&#xA;    phi  = gr.data[&quot;phi&quot;]&#xA;    phi1 = gr.data[&quot;phi1&quot;]&#xA;    phi2 = gr.data[&quot;phi2&quot;]&#xA;&#xA;    # initialize&#xA;    gr.initialize()&#xA;&#xA;    phi_init = phi.copy()&#xA;&#xA;    # runtime plotting&#xA;    if dovis == 1: plt.ion()&#xA;&#xA;    t = 0.0&#xA;    while t &amp;lt; tmax:&#xA;&#xA;        dt = est_dt(gr, kappa, tau)&#xA;&#xA;        if t + dt &amp;gt; tmax:&#xA;            dt = tmax - t&#xA;&#xA;        # react for dt/2&#xA;        phi1[:] = react(gr, phi, tau, dt/2)&#xA;        gr.fillBC(&quot;phi1&quot;)&#xA;&#xA;        # diffuse for dt&#xA;        phi2[:] = diffuse(gr, phi1, kappa, dt)&#xA;        gr.fillBC(&quot;phi2&quot;)&#xA;&#xA;        # react for dt/2 -- this is the updated solution&#xA;        phi[:] = react(gr, phi2, tau, dt/2)&#xA;        gr.fillBC(&quot;phi&quot;)&#xA;&#xA;        t += dt&#xA;&#xA;        if dovis == 1:&#xA;            plt.clf()&#xA;            plt.plot(gr.x, phi)&#xA;            plt.grid()&#xA;            plt.xlim(gr.xmin,gr.xmax)&#xA;            plt.ylim(0.0,1.0)&#xA;            plt.title(&quot;Reaction-Diffusion, &lt;span class=&quot;math-container&quot;&gt;$t = {:3.2f}$&lt;/span&gt;&quot;.format(t))&#xA;            plt.draw()&#xA;            plt.pause(0.1)&#xA;&#xA;    if return_initial == 1:&#xA;        return phi, gr.x, phi_init&#xA;    else:&#xA;        return phi, gr.x&#xA;&#xA;&#xA;kappa = 1.0&#xA;tau = 0.25&#xA;nx = 256&#xA;tmax1 = 1.0&#xA;phi1, x1 = evolve(nx, kappa, tau, tmax1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell, the initial condition being of class &lt;span class=&quot;math-container&quot;&gt;$\cal{C}^{\infty}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; being stable, the solution should remain &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; where the initial condition is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. But this is not what I observe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a numerical artifact?&lt;/p&gt;&#xA;" OwnerUserId="7784" LastEditorUserId="20688" LastEditDate="2019-05-18T21:54:26.723" LastActivityDate="2019-05-18T21:54:26.723" Title="Growing error from a smooth initial condition for Fisher KPP equation" Tags="|numerics|parabolic-pde|operator-splitting|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="26505" PostTypeId="2" ParentId="26483" CreationDate="2017-03-29T14:53:13.083" Score="5" Body="&lt;p&gt;1) You have misinterpreted the order the fft2 output elements are stored. This is a very common mistake, since it is very easy to get confused. Thankfully, there is an easy way to avoid such bugs. Arrange the Fourier space vectors the &quot;natural way&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;% Fourier space vectors&#xA;kx = 2*pi/Lx*[-Nx/2:Nx/2-1]'; &#xA;ky = 2*pi/Ly*[-Ny/2:Ny/2-1]';&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, make sure the fft2 output has the same arrangement by using the ifft function. This hardly adds any computation load to your code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v_hat = fftshift(fft2(v)); % FFT of test function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Perform the algebra in the Fourier space as before and then make the inverse shift to the input of ifft2, by using the ifftshift function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w_L = real(ifft2(ifftshift(w_L)));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) As you noticed, things get &quot;very weird&quot; at the edges. This is another issue, which has not to do with bugs or numerical accuracy, but with the convergence properties of the Fourier series transform. When the function to be transformed is continuous, the Fourier series has absolute convergence. But, when there are discontinuities, the Fourier series converges by norm, which means that only the measure of the difference between the truncated series and its limit goes to infinity, hence these &quot;weird&quot; oscillations at the edges, called &quot;Gibbs oscillations&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it may not be clear what the source of the discontinuity is. After all, v and its derivatives are well behaved functions. This has to do with the discrepancies with the Fourier series transform (FST) and the discrete Fourier transform(DFT). FST acts on periodic functions, whereas DFT on discrete signals of finite length. In order to interpret DFT as a numerical approximation to FFT, we implicitly assume that the input of DFT represents a sample of some periodic signal, whose half-period is represented by this window and the other half by its mirror image. This is, of course, easier to visualize in the 1D case. Therefore, every function whose sample is not zero at the window edges is implicitly interpreted as discontinuous. v may be zero at the edges, but its Laplacian is not and this makes these spooky Gibbs oscillations appear as if from nowhere.&lt;/p&gt;&#xA;" OwnerUserId="22217" LastEditorUserId="22217" LastEditDate="2017-03-29T17:56:03.157" LastActivityDate="2017-03-29T17:56:03.157" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26509" PostTypeId="2" ParentId="7702" CreationDate="2017-03-29T16:52:33.010" Score="-1" Body="&lt;p&gt;It is really important! If you do not have an appropriate time step, you can not achieve mesh independence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use my personal experience: I investigate CFD of fluid-structure interaction using FEM. I was trying to do the mesh independence study to make sure that mesh does not affect the accuracy of my simulations. However, when I refine the mesh, the simulation results even diverges further. At last, I figured out that I forgot to adjust my time step accordingly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you reduce the size of element, you are recommended to reduce the time step correspondingly. Otherwise, you may have trouble. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you got time and computational resources, you may just run some test cases to see how the $\Delta t$ affects your specific cases, which I think should be the most reliable method.&lt;/p&gt;&#xA;" OwnerUserId="23732" LastEditorUserId="23732" LastEditDate="2017-03-30T16:25:37.643" LastActivityDate="2017-03-30T16:25:37.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26510" PostTypeId="1" CreationDate="2017-03-29T18:39:06.203" Score="0" ViewCount="322" Body="&lt;p&gt;I have a $9 \times 9$ square lattice. In each site of the lattice, there are $N$ molecules, where $N$ is a large random number and it is different for different sites.&#xA;N equal to 100000000+-100 in beginning and changes by passing time.&#xA;I want each molecule to do a random walk in each time step. $N$ is very large and it is not possible to apply random walk for each molecule in each site. Could anyone help? It is a part of other code, actually molecules f cites are consumed by other particles and they diffuse in time, but the part I have problem in is diffusion of molcules.&#xA;I don't have enough score to answer the comments, so I answer here. Yes lattice is 2d.&#xA;Yes, Exactly! I want number of particles in each site.But I don't underestand what do you mean. Could you provide an answer?&lt;/p&gt;&#xA;" OwnerUserId="23766" LastEditorUserId="23768" LastEditDate="2017-03-29T19:55:25.200" LastActivityDate="2017-03-30T12:01:15.013" Title="Random walk on lattice" Tags="|matlab|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26512" PostTypeId="2" ParentId="26510" CreationDate="2017-03-29T20:36:56.107" Score="1" Body="&lt;p&gt;In the comments, @WolfgangBangert mentions the first part, the representation: instead of storing for $10^8$ particles a lattice location -- a two dimensional vector $(x,y)$ with 9 entries in each dimension -- you choose the dual representation by storing for each site $(x,y)$ the number of particles which occupy the site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second component is the transition. Straightforwardly, you would have to draw two times $10^8$ random numbers in order to check whether each one of the particles undergoes a transition. By this, however, you would have won nothing (regardless of the reduced representation of the system state in step one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore you approximate the distribution of the particles which undergo a transition by a normal distribution, which should be a very good approximation as your $N$ is quite large. The theoretical basis for this is the central limit theorem, and the parameters of the normal distribution depend on your microscopic transition law. You then draw a few random numbers from the normal -- which might even not be necessary as the normal should be really sharp, so it's possible you can just pick the mean. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the above procedure is similar to what is done in classical statistical physics, when going from the microscopic description to the canonical ensemble.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2017-03-29T20:36:56.107" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26513" PostTypeId="2" ParentId="21241" CreationDate="2017-03-29T20:45:12.290" Score="2" Body="&lt;p&gt;This one seems to work great for me:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mcximing/hungarian-algorithm-cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mcximing/hungarian-algorithm-cpp&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23773" LastActivityDate="2017-03-29T20:45:12.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26515" PostTypeId="1" CreationDate="2017-03-30T00:42:28.127" Score="0" ViewCount="1457" Body="&lt;p&gt;&lt;strong&gt;Some context:&lt;/strong&gt; I am working with the Black-Scholes model.. I have an explicit (Black-Scholes) formula which is the exact solution to my problem. I have written code which implements a finite-difference scheme (explicit Forward-Euler, i.e: forward march-in-time iterative scheme) to approximate the solution to a partial differential equation (PDE). I have discretized the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to compare these two and verify that the error between the exact solution and my approximation to the solution is indeed small. I have stored the values of the exact solution and the approximation in two &lt;code&gt;(NxM)&lt;/code&gt; arrays, for each time-step and node in space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some &lt;strong&gt;pseudocode&lt;/strong&gt; for what I wish to do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;E = 0&#xA;for i = 1,...,N:&#xA;    E_new = | BS(x_i) - u_i |&#xA;    if E_new &amp;gt;= E&#xA;        E = E_new&#xA;    else:&#xA;        E = E&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is &lt;strong&gt;the code I have written&lt;/strong&gt; to calculate the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Compute the Error&#xA;E = np.zeros((N+1, M+1)) # so that E has NxM entries--one for each time-step and node in space&#xA;for m in range(1, M+1):&#xA;    for j in range(0, N+1):&#xA;        E_new = np.absolute(BS[j,m] - u[j,m])&#xA;        if (np.any(E_new &amp;gt;= E)): &#xA;            E = E_new&#xA;        else:&#xA;            E = E&#xA;&#xA;print 'Error = ', E&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, I am taking &lt;code&gt;N = 100, M = 10,000&lt;/code&gt;. If necessary, I can post the segments of my code where I calculate the arrays &lt;code&gt;BS&lt;/code&gt; and &lt;code&gt;u&lt;/code&gt;. Both are defined as arrays of the same size, and when I plot them, I am receiving an output which agrees with what the solution looks like. So I believe I am on the right track and that the rest of my code is working as it is supposed to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I run this particular segment in an attempt to calculate the error, I get that the error is roughly 29. I know this is wrong, the error should be small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempt to fix the problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the problem possibly lies in the line containing &lt;code&gt;if (np.any(E_new &amp;gt;= E)):&lt;/code&gt;. I want to check if any entry of the array &lt;code&gt;E_new&lt;/code&gt; is greater than or equal to &lt;code&gt;E&lt;/code&gt;. If I just run it with &lt;code&gt;if (E_new &amp;gt;= E):&lt;/code&gt;, I get this ValueError: &quot;The truth value of an array with more than one element is ambiguous. Use a.any() or a.all().&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried running this: &lt;code&gt;if (np.any([E_new, E])):&lt;/code&gt;, but again no luck. I received this error message, &quot;ValueError: setting an array element with a sequence&quot;. I have also tried using &lt;code&gt;np.all&lt;/code&gt; in place of &lt;code&gt;np.any&lt;/code&gt; in each of my attempts and this also does not seem to fix the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any and all help is very greatly appreciated. Thank you in advance! I am really stuck here and I do not know how to move forward. I don't need a direct answer necessarily, but some insight as to what is causing the problem and a push in the right direction would be immensely helpful. Thanks again!!&lt;/p&gt;&#xA;" OwnerUserId="20906" LastActivityDate="2017-03-30T02:46:01.483" Title="Trying to compute the error from comparing two arrays" Tags="|pde|finite-difference|python|numpy|parabolic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26516" PostTypeId="2" ParentId="26515" CreationDate="2017-03-30T02:30:38.227" Score="2" Body="&lt;p&gt;You can just do something to the effect of &quot;$\mathtt{np.max(np.abs( BS - u ) )}$&quot;. This computes the $L_\infty$ norm of the error, which seems to be what you're after. It'll save you the heartache/headache of writing loops, and it also happens to execute about 100x faster in python, since you're offloading the work to functions written in C. This assumes that &quot;BS&quot; is your exact solution, &quot;u&quot; is your numerical solution, and that they're both stored in numpy arrays.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-03-30T02:30:38.227" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26517" PostTypeId="2" ParentId="26515" CreationDate="2017-03-30T02:46:01.483" Score="2" Body="&lt;p&gt;I would highly suggest that you use the &lt;code&gt;norm&lt;/code&gt; functions in &lt;code&gt;numpy&lt;/code&gt; via &lt;code&gt;numpy.linalg.norm(BS - u, ord=2)&lt;/code&gt; which would give you the $\ell_2$ norm of your error or if you set &lt;code&gt;ord=numpy.infty&lt;/code&gt; you can do the $\ell_\infty$.&lt;/p&gt;&#xA;" OwnerUserId="567" LastActivityDate="2017-03-30T02:46:01.483" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26518" PostTypeId="1" CreationDate="2017-03-30T08:23:05.073" Score="0" ViewCount="564" Body="&lt;p&gt;Maybe my question is obvious but i cannot find any good source which answers it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I trying to learn about proximal gradient. One thing which is not clear for me is particular algorithm for line search.&#xA;According to &lt;a href=&quot;https://pdfs.semanticscholar.org/c924/20f001e023c693db762758f9590571256e35.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this presentation&lt;/a&gt; (slide 20) and some other sources backtracking line search with previous step length as initialization is good choice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented simple version of LASSO with proximal gradient and it looks like that increasing step from time to time can make convergence significantly faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: Is it good idea to use bracketing in proximal gradient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Accoring to &quot;Numerical optimization by Nocedal and Wright&quot; i use term &quot;bracketing&quot; in meaning &quot;finding initial interval for backtracking&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My second question is: Which is best line search algorithm for LASSO/general proximal gradient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for my English&lt;/p&gt;&#xA;" OwnerUserId="15233" LastActivityDate="2017-08-22T18:11:43.847" Title="Line search bracketing for proximal gradient. Is it good idea?" Tags="|optimization|convex-optimization|convergence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26519" PostTypeId="2" ParentId="26510" CreationDate="2017-03-30T12:01:15.013" Score="1" Body="&lt;p&gt;For such large numbers the diffusion process becomes essentially 'deterministic' (&lt;a href=&quot;https://en.wikipedia.org/wiki/Law_of_large_numbers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Law_of_large_numbers&lt;/a&gt;) so the classic approach is to implement a finite differencing method for a diffusion equation, such as: &lt;a href=&quot;https://nl.mathworks.com/matlabcentral/fileexchange/38088-diffusion-in-1d-and-2d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nl.mathworks.com/matlabcentral/fileexchange/38088-diffusion-in-1d-and-2d&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="23781" LastActivityDate="2017-03-30T12:01:15.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26520" PostTypeId="1" AcceptedAnswerId="26609" CreationDate="2017-03-30T09:50:34.150" Score="2" ViewCount="200" Body="&lt;p&gt;I'm currently trying to solve a system of (3) nonlinear equations of (3) variables which are the baryonic density, the isospin asymmetry and and the density of a fluid with the Broyden's method (a quasi-Newton method) included in the GSL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I don't know how to avoid negative roots. I don't want them because it's obviously nonphysical. I guess I should use some conditions but I don't know which ones... I'm programming in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you help please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="23799" OwnerDisplayName="T. Auerrac" LastActivityDate="2017-04-12T13:39:30.537" Title="How to avoid negative roots with a Newton's method?" Tags="|computational-physics|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26522" PostTypeId="1" AcceptedAnswerId="26524" CreationDate="2017-03-30T16:33:43.897" Score="7" ViewCount="226" Body="&lt;p&gt;I am trying to familiarise myself with &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic_differential_equation&quot; rel=&quot;noreferrer&quot;&gt;SDE&lt;/a&gt;s and have been reading a few review papers on the topic. They leave the impression that a great deal of work has been put into solvers that are derivative-free. To my understanding this means that for a DDE like&#xA;$$\newcommand\diff{\mathop{}\!\mathrm{d}}&#xA;\diff X = f(X)\diff t + g(X) \diff W,&#xA;$$&#xA;the derivatives of $f$ and $g$ are not required for the method (correct me if I am wrong).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can understand that this property is useful in some applications where the derivative is difficult or computationally infeasible to obtain or does not exist. However, I would not expect such problems to be very relevant in application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This suggests to me that at least one of the following applies:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is some further relevant advantage to derivative-free solvers that I am missing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Problems where derivative-free solvers are required (due to the above reason) are more relevant than I think they are.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The demand for derivative-free solvers is lower than the “supply”, i.e., the attention given to them by those who develop solvers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Which is it?&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="9667" LastEditDate="2017-03-30T18:04:51.697" LastActivityDate="2017-03-30T18:04:51.697" Title="What’s so great about derivative-free solvers for SDEs?" Tags="|stochastic|stochastic-ode|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26523" PostTypeId="2" ParentId="26522" CreationDate="2017-03-30T17:26:05.043" Score="4" Body="&lt;p&gt;I am not an expert in specifically stochastic differential equations, but I would assume that my answer will still be of some value.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Computation of the derivative can be challenging, as you mentioned in your question. However, this would be even more pronounced in a multidimensional case, as one would have to calculate Jacobian matrices ($n^2$ entries). So, non-derivative-free solvers will suffer from the curse of dimensionality. The situation becomes even worse when higher-order derivatives are required for a scheme.&lt;/li&gt;&#xA;&lt;li&gt;Computation of a derivative by itself generally amplifies numerical noise. So, for example, if the underlying function ($f$ or $g$) is not analytical, the error in the derivative might completely distort the solution. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2017-03-30T17:31:25.413" LastActivityDate="2017-03-30T17:31:25.413" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26524" PostTypeId="2" ParentId="26522" CreationDate="2017-03-30T17:33:58.317" Score="7" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I can understand that this property is useful in some applications where the derivative is difficult or computationally infeasible to obtain or does not exist. However, I would not expect such problems to be very relevant in application.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If an analytical solution for the derivative is not known, it's very costly and error prone. Calculating the Jacobian is $n^2$ entries, but numerical differentiation techniques will need to do multiple function calls per entry. Then, to get this right, numerical differentiation techniques have to divide by a small number when computing the derivative, which causes a lot of numerical issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With autodifferentiation tools this cost is reduced, but it can still be significant. So when analytical Jacobians are not prescribed, it's usually good to stay away from methods which require derivatives.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I would not expect such problems to be very relevant in application.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For most things like nonlinear SPDEs or large systems of SDEs (1000's) coming from biology, getting the Jacobian written out can be nearly impossible and error prone. I would say that it's the other way around: expecting an analytical Jacobian to be provided is not a good idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some further advantages as well. Runge-Kutta methods are derivative-free methods, and they can do a lot of coefficient optimization. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The demand for derivative-free solvers is lower than the “supply”, i.e., the attention given to them by those who develop solvers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is not the case. In &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl&quot; rel=&quot;noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt; derivative-free methods were implemented before the KPS Stochastic Taylor Series methods because, for most users, it will lead to ease-of-use and increased performance. That said, in the field of differential equations, you can always find a counter example where that's not the case, so I do plan on implementing some methods which explicitly use derivatives. However, I am sure most users will probably just default to the derivative-free methods because the cognitive load on their end is much lower.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-03-30T17:39:37.057" LastActivityDate="2017-03-30T17:39:37.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26525" PostTypeId="2" ParentId="26422" CreationDate="2017-03-30T21:09:26.040" Score="1" Body="&lt;p&gt;Regarding your issue of multigrid taking longer than Gauss-Seidel:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It could be that you didn't code the interpolation/restriction operations very efficiently.  Do you make sure to take full advantage of the sparsity of the matrices?  We write the coarse grid operator as something like RAI, but you shouldn't multiply out the matrices directly.  All three are sparse matrices and you can save a lot of multiplications by being smart about it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In your figure, multigrid converges in fewer iterations than G-S, so it is doing something -- in most cases, you need less than half as many iterations with multigrid.  And it converges to the right answer, so that leads me to believe you've probably coded it correctly.  However, it doesn't seem like G-S is taking very many iterations at all.  20 iterations isn't a lot for G-S.  The benefit you get from multigrid will be minimal at best since G-S isn't really being all that slow for this problem.  The point of using multigrid is to accelerate smoothers like G-S in cases where they converge really slowly (i.e., in cases with low frequency error modes).  Since there isn't much to accelerate, the overhead of setting up the multigrid stuff is probably comparable to, if not greater than, the speedup you get from multigrid...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One big benefit with multigrid is that the number of iterations is usually independent of (or at least really insensitive) the problem size.  The same cannot be said for G-S.  So you could try bigger problems to see if you get a speedup, and you could try seeding the problem with something that you know will give a low frequency error mode (which G-S should struggle with).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, a side comment: unless you're actually computing with 43 digits of precision, your results are meaningless below a tolerance of $10^{-15}$ or so.  I suggest cutting that part out of your plot.  In all practical applications, I don't think you'd need to go much beyond $10^{-12}$.&lt;/p&gt;&#xA;" OwnerUserId="20583" LastActivityDate="2017-03-30T21:09:26.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26527" PostTypeId="2" ParentId="7525" CreationDate="2017-03-31T00:19:41.790" Score="1" Body="&lt;p&gt;Why not &lt;strong&gt;Blender&lt;/strong&gt;? It is a powerful, free and open-source software with python scripting support too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;em&gt;Blender 2.79&lt;/em&gt; you need to choose &lt;code&gt;Scripting&lt;/code&gt; as &lt;code&gt;Screen layout&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the top-left &lt;code&gt;Text Editor&lt;/code&gt; press + to create a new text data-block; start it with &lt;code&gt;import bpy&lt;/code&gt; and then insert the python commands relative to each operation that you can do from the GUI, reading the relative command putting your mouse over the GUI item.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the attached screenshot are shown the commands to triangulate a circle, starting from the default scene with a cube:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9kDNZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9kDNZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way, you can use Blender in scripting mode, without its GUI to export, for example, an output file, running your python script in a terminal:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blender --background --python myscript.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=K0yb4sZ7B4g&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; a videotutorial on YouTube.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;" OwnerUserId="16695" LastEditorUserId="16695" LastEditDate="2018-08-13T16:15:09.397" LastActivityDate="2018-08-13T16:15:09.397" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="26528" PostTypeId="2" ParentId="26422" CreationDate="2017-03-31T00:57:06.197" Score="0" Body="&lt;p&gt;Your residual calculation is incorrect.&#xA;If $[A]\{u\}=\{b\} $ is your system of linear equations, the residual is $res=\{b\}-[A]\{u_k\}$,where $u_k$ is the approximation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case it should be calculated as:&#xA;$$\operatorname{res}=T_1 + F_O [T_2(i-1,j) + T_2(i+1,j) + T_2(i,j+1) + T_2(i, j-1)] - (1 + 4 F_O) T_2(i,j)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, it just needed to be multiplied by $(1 + 4 F_O)$&lt;/p&gt;&#xA;" OwnerUserId="16934" LastActivityDate="2017-03-31T00:57:06.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26529" PostTypeId="1" AcceptedAnswerId="26532" CreationDate="2017-03-31T18:19:50.123" Score="6" ViewCount="513" Body="&lt;p&gt;Gaussian quadrature improves on Newton-Cotes formulas by allowing the abscissas to vary along with the weights in order to integrate higher order polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this idea be extended to numerical differentiation? To wit, can I choose a set $\{h_{i}\}$ such that $f'$ is much better approximated by a weighted sum of evaluations at $x+h_{i}$ than it could be at equally spaced data point? For example, maybe the following relation could hold for some $j$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;f'(x) = \sum_{i=1}^{n}\frac{w_i}{h_i}f(x+h_{i}) + \mathcal{O}(\left\| hf^{(n+j)}\right\|^{n+j})&#xA;\end{align*}&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2017-04-01T11:52:51.560" Title="Gaussian Numerical Differentiation" Tags="|finite-difference|quadrature|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26530" PostTypeId="1" CreationDate="2017-04-01T08:33:07.397" Score="1" ViewCount="342" Body="&lt;p&gt;Suppose we have a very efficient way to perform $\ell_0$ &quot;norm&quot; compressed vs $\ell_1$ norm compressed sensing. Specifically, $\ell_0$ &quot;norm&quot; compressed sensing is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eqalign{&#xA;  &amp;amp; \min \quad {x^T}Qx + {b^T}x + \mu {\left\| x \right\|_0}  \cr &#xA;  &amp;amp; \text{s.t}:\quad Ax \le b \cr} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;while $\ell_1$ norm compressed sensing is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eqalign{&#xA;  &amp;amp; \min \quad {x^T}Qx + {b^T}x + \mu {\left\| x \right\|_1}  \cr &#xA;  &amp;amp; \text{s.t}:\quad Ax \le b \cr} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should we almost always use the $\ell_0$ norm formulation? Is there any paper discussing the benefits of $\ell_0$ (disregard the computational perspectives) over $\ell_1$ norm compressed sensing?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: The actual application we are interested in is about avoiding overfitting formulated below. For example $x$ is a very long feature vector (say length $1000$); however we only $100$ data, i.e., ${E_\text{feature}}$ is $100 \times 1000$, and ${E_\text{input}}$ is $100 \times 1$. The question we are interested in is whether the $\ell_0$ norm formulation (if can be computed practically) can offer a significant advantage over the $\ell_1$ norm formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\eqalign{&#xA;  &amp;amp; \min \quad \left\| {{E_\text{feature}}x - {E_\text{input}}} \right\|_2^2 + \mu {\left\| x \right\|_{0,1}}  \cr &#xA;  &amp;amp; \text{s.t}:\quad Ax \le b \cr} $$&lt;/p&gt;&#xA;" OwnerUserId="7891" LastEditorUserId="20417" LastEditDate="2018-07-11T15:26:01.043" LastActivityDate="2018-08-10T16:01:05.803" Title="Compressed sensing: $\ell_0$ &quot;norm&quot; vs $\ell_1$ norm" Tags="|optimization|sparse-matrix|regression|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="26531" PostTypeId="2" ParentId="26461" CreationDate="2017-04-01T09:44:03.013" Score="13" Body="&lt;p&gt;Personally, in the absence of a special function like &lt;a href=&quot;http://www.cplusplus.com/reference/cmath/expm1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;expm1&lt;/code&gt;&lt;/a&gt; (see also &lt;a href=&quot;https://scicomp.stackexchange.com/questions/20629&quot;&gt;this scicomp.SE question&lt;/a&gt;), I'd use a Padé approximant instead of a Taylor/Maclaurin series; usually, those have a slightly wider applicability for the same amount of computational effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll be discussing $\exp x-1$ for the rest of the answer, since $10^x-1=\exp(x\log 10)-1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate, here's a plot comparing the relative error of the $7$-th order Taylor polynomial&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x+\frac{x^2}{2}+\frac{x^3}{6}+\frac{x^4}{24}+\frac{x^5}{120}+\frac{x^6}{720}+\frac{x^7}{5040}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the $(4,3)$ Padé approximant&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{x+\frac{x^2}{14}+\frac{x^3}{42}+\frac{x^4}{840}}{1-\frac{3 x}{7}+\frac{x^2}{14}-\frac{x^3}{210}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/t2mQq.png&quot; alt=&quot;Taylor vs. Padé&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar plots can be seen for order $2p+1$ Taylor polynomials vs. $(p+1,p)$ Padé approximants.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Another alternative presents itself, if your system has the $\sinh$ function: use the relation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\exp x-1=2\exp\left(\frac{x}{2}\right)\sinh\left(\frac{x}{2}\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, &lt;a href=&quot;https://books.google.com/books?hl=en&amp;amp;id=5tv3HdF-0N8C&amp;amp;pg=PA19&quot; rel=&quot;nofollow noreferrer&quot;&gt;Higham&lt;/a&gt; presents a neat trick in &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; (second edition, p. 19), and originally attributed to Kahan. Look there for more details.&lt;/p&gt;&#xA;" OwnerUserId="127" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.590" LastActivityDate="2017-04-04T13:01:36.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26532" PostTypeId="2" ParentId="26529" CreationDate="2017-04-01T11:52:51.560" Score="6" Body="&lt;p&gt;&lt;strong&gt;Yes.&lt;/strong&gt; As you may know, numerical differentiation and integration is closely related to (polynomial) interpolation: The idea to approximately differentiate or integrate a given function is to approximate it with a function (often an interpolating polynomial) that can be differentiated or integrated exactly. For example, the standard central difference quotient formulas  for $f(x)$ can be derived from differentiating a quadratic  interpolating polynomial through the points $x-h, x, x+h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The benefit ist that the error in approximating the derivative or integral is only determined by the error in approximating the function -- which is well understood for polynomial interpolation. In particular, it turns out that uniform interpolation points are a poor choice in general, and interpolation points based on roots of orthogonal polynomials are much better. In the context of quadrature, this leads to the different variants of Gaussian quadrature (Legendre, Chebyshev, Jacobi, Laguerre, Hermite...); in the context of differentiation, this is referred to as &lt;em&gt;spectral collocation&lt;/em&gt;. Since it is the basis of spectral methods for solving differential equations, rather than a finite-difference quotient this is usually realized by a &lt;em&gt;differentiation matrix&lt;/em&gt; that maps the values of $f$ at a selection of collocation points to the corresponding values of $f'$. (In contrast to the standard finite-difference matrices, these are usually dense; spectral methods are therefore &lt;em&gt;global&lt;/em&gt; methods.) The deep relations between these concepts are explained in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Trefethen, Lloyd N.&lt;/em&gt;, Approximation theory and approximation practice, Other Titles in Applied Mathematics 128. Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM) (ISBN 978-1-611972-39-9/pbk). 305~p. (2013). &lt;a href=&quot;https://zbmath.org/?q=an:1264.41001&quot; rel=&quot;noreferrer&quot;&gt;ZBL1264.41001&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, chapter 21 is concerned with spectral collocation, and Theorem 21.1 is precisely the kind of error estimate you wrote.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard choice for Gaussian quadrature are the roots of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Legendre_polynomials&quot; rel=&quot;noreferrer&quot;&gt;Legendre polynomials&lt;/a&gt;, and you can in fact use the same points for differentiation as well (aptly called &lt;em&gt;Legendre collocation&lt;/em&gt;); here's a &lt;a href=&quot;https://mathworks.com/matlabcentral/fileexchange/5015-legendre-collocation-differentiation&quot; rel=&quot;noreferrer&quot;&gt;Matlab script&lt;/a&gt; that sets up the corresponding differentiation matrix. More commonly used are the roots of &lt;a href=&quot;https://en.wikipedia.org/wiki/Chebyshev_polynomials&quot; rel=&quot;noreferrer&quot;&gt;Chebyshev polynomials&lt;/a&gt; leading to &lt;em&gt;Chebyshev collocation&lt;/em&gt;, which is the basis of the remarkable &lt;a href=&quot;https://www.chebfun.org&quot; rel=&quot;noreferrer&quot;&gt;Chebfun&lt;/a&gt; Matlab package (and its Julia child, &lt;a href=&quot;https://github.com/JuliaApproximation/ApproxFun.jl&quot; rel=&quot;noreferrer&quot;&gt;ApproxFun.jl&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2017-04-01T11:52:51.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26534" PostTypeId="1" AcceptedAnswerId="26535" CreationDate="2017-04-01T13:14:23.337" Score="1" ViewCount="1485" Body="&lt;p&gt;I have to create a &lt;strong&gt;strong scalability plot&lt;/strong&gt;, which measures the &lt;strong&gt;execution time&lt;/strong&gt; for a &lt;code&gt;numberOfThreads =1,2,4,8,16&lt;/code&gt; with &lt;code&gt;size=1024&lt;/code&gt;. Here is the table with the results after execution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QUjzx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QUjzx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is the plot based on the table: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TG8VZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TG8VZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know exactly when is a plot strong scalable and if this plot is a &lt;strong&gt;strong scalability plot&lt;/strong&gt; ? &lt;/p&gt;&#xA;" OwnerUserId="23813" LastActivityDate="2017-04-01T18:50:54.103" Title="Strong scalability plot, HPC" Tags="|hpc|plotting|scaling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26535" PostTypeId="2" ParentId="26534" CreationDate="2017-04-01T18:50:54.103" Score="3" Body="&lt;p&gt;As sensitive_scientist mentioned, &lt;a href=&quot;https://www.sharcnet.ca/help/index.php/Measuring_Parallel_Scaling_Performance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Measuring Parallel Scaling Performance&lt;/a&gt; provides the information you want on how to calculate strong\weak scaling. I've plotted your data in a way I consider it the most informative: both execution timing and strong scaling.&#xA;Note:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In my opinion, it is preferred to plot the execution timing on a log-log plot. The dashed line shows ideal parallelization ($t$/$T$). The data suggests that you might want to add one more datapoint ($T=32$ threads) to show that the &quot;Multiply&quot; parallelization also breaks down (as well as &quot;Block Multiply&quot;).&lt;/li&gt;&#xA;&lt;li&gt;In a way, both of those plots serve the same purpose, but if they are detached it is hard to see the advantage of &quot;Block Multiply&quot; in an absolute sense - it is faster, though less parallel efficient.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0MdiP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0MdiP.png&quot; alt=&quot;Execution timing&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lAs8p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lAs8p.png&quot; alt=&quot;Strong Scaling&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness, strong scaling as a function of # Threads ($T$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\text{strong scaling}(T)=\frac{t_1}{t_T*T}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $t_1$ is the execution time for $1$ thread and $t_T$ is the execution time for $T$ threads.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-04-01T18:50:54.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26537" PostTypeId="1" AcceptedAnswerId="26541" CreationDate="2017-04-02T04:12:13.247" Score="2" ViewCount="238" Body="&lt;p&gt;&lt;em&gt;Appendix A&lt;/em&gt; of Liu, Baoyin, and Ma (2011) &lt;a href=&quot;https://arxiv.org/abs/1108.4636&quot; rel=&quot;nofollow noreferrer&quot;&gt;Equilibria, periodic orbits around equilibria, and heteroclinic connections in the gravity field of a rotating homogeneous cube&lt;/a&gt; shows an analytic expression for the gravitational potential of a uniform cube. I'd like to reproduce the orbit calculation using python, which will require  evaluating the gradient of the reduced potential, i.e. acceleration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given sufficient coffee I can probably type that into Wolfram alpha, evaluate the gradient, and script as python. Alternately I could use a numerical gradient which would require four potential evaluations and be faster and easier for me to reliably script. I just want to make plots of single orbits, so I do not require very high accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there another way to get a fairly good approximation to the gravitational force from a uniform cube of side $a$ with similar (or less) effort? Fairly good might be say 1E-06 error at distances &gt; $0.1a$ from a face, possibly need to stay farther from the corner for similar error, but it seems that most of the stable orbits tend to do that anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3D direct integration at each time step is extremely slow with scipy's 'triple quadratic' method, not that I'd ever admit trying it. There may be an amazingly clever integration algorithm, and it could compete in speed considering I will be writing python without any of its numerical acceleration options at this point in time.&lt;/p&gt;&#xA;" OwnerUserId="17869" LastActivityDate="2021-06-21T19:27:51.753" Title="Another way to evaluate the gravitational force from a uniform cube?" Tags="|numerical-analysis|python|integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26539" PostTypeId="1" AcceptedAnswerId="26548" CreationDate="2017-04-02T07:26:28.700" Score="2" ViewCount="132" Body="&lt;p&gt;Assume I have the following system of equations. I am trying to figure out if it's better for me to be solving this as a system of ODEs or as a system of DAEs. The real code can have up to a dozen or so equations, and the system is stiff (I am dealing with chemical kinetics, where the rate constants can vary by many orders of magnitude).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The governing ODEs are:&#xA;$$\frac{d\theta_{A}}{dt}=2R_{1}(\theta_{A},\theta_{*})-R_{2}(\theta_{A},\theta_{*})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d\theta{*}}{dt}=-2R_{1}(\theta_{A},\theta_{*})+R_{1}(\theta_{A},\theta_{*})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can solve for $\theta_{A}$ and $\theta_{*}$ by solving this system. However, it also happens that there is a conservation law such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\theta_{A}+\theta_{*}=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means I can solve the system of equations consisting of, for example, the first ODE and the algebraic conservation law to get $\theta_{A}$ and $\theta_{*}$ as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computationally, which is easier to solve numerically? I am more concerned with accuracy than speed but am curious either way. Is it easier to solve the system of two ODEs or one of the ODEs and the algebraic conservation law?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it's relevant, I'm currently thinking about using &lt;code&gt;ODE15s&lt;/code&gt; in MATLAB, which can solve a stiff system of ODEs or DAEs. &lt;/p&gt;&#xA;" OwnerUserId="22245" LastEditorUserId="22245" LastEditDate="2017-04-02T07:38:29.770" LastActivityDate="2017-04-03T07:55:25.350" Title="Solving a system of DAEs versus ODEs (which is preferable)" Tags="|numerical-analysis|ode|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26541" PostTypeId="2" ParentId="26537" CreationDate="2017-04-02T14:38:19.667" Score="2" Body="&lt;p&gt;I am rewriting my comment as an answer.&lt;/p&gt;&#xA;&lt;p&gt;I think that you have several options. Some of them are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use &lt;a href=&quot;http://docs.sympy.org/latest/tutorial/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SymPy&lt;/a&gt; to find the gravitational field from the potential. Then you can generate your &lt;a href=&quot;http://docs.sympy.org/latest/modules/printing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python code from it&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could create some kind of mesh and then compute the gradient numerically using finite differences, piecewise polynomials (FEM-like), or Chebyshev polynomials (see &lt;a href=&quot;https://github.com/pychebfun/pychebfun&quot; rel=&quot;nofollow noreferrer&quot;&gt;pychebfun&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use a Boundary Integral Representation for the gravitational field of your problem.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can represent your cube as a set of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; discrete particles and consider that the overall field is just the superposition of these &amp;quot;little&amp;quot; contributions. When &lt;span class=&quot;math-container&quot;&gt;$n\rightarrow\infty$&lt;/span&gt;, the field should approach the real distribution.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2021-06-21T19:27:51.753" LastActivityDate="2021-06-21T19:27:51.753" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="26543" PostTypeId="1" CreationDate="2017-04-02T15:59:10.763" Score="0" ViewCount="552" Body="&lt;p&gt;I have this method which computes the Fibonacci function: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;long optimized_p_fib(long n)&#xA;{&#xA;    long i, j;&#xA;    if(n &amp;lt; 2)&#xA;        return n;&#xA;&#xA;    #pragma omp task shared (i)&#xA;    {&#xA;        i = p_fib(n-1);&#xA;    }&#xA;    #pragma omp task shared (j)&#xA;    {&#xA;        j = p_fib(n-2);&#xA;    }        &#xA;    #pragma omp taskwait&#xA;&#xA;    return (i+j);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The execution time of this method is &lt;strong&gt;2.527737&lt;/strong&gt;.&lt;br&gt;&#xA;I want to use tasks in OpenMP so that the execution time will be smaller, below is what I did so far, but the execution time got bigger (&lt;strong&gt;4.039427&lt;/strong&gt;) compared to the method above.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;long optimized_p_fib(int n)&#xA;{&#xA;    long i, j;&#xA;    if(n &amp;lt; 2)&#xA;        return n;&#xA;&#xA;    #pragma omp single nowait&#xA;    {&#xA;        #pragma omp task shared (i) firstprivate(n)&#xA;        {&#xA;            i = optimized_p_fib(n-1);&#xA;        }&#xA;        #pragma omp task shared (j) firstprivate(n)&#xA;        {&#xA;            j = optimized_p_fib(n-2);&#xA;        }&#xA;    }&#xA;    #pragma omp taskwait&#xA;&#xA;    return (i+j);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What could I possibly optimize my code, so that the execution time would get small ?&lt;/p&gt;&#xA;" OwnerUserId="23813" LastEditorUserId="20688" LastEditDate="2017-04-02T16:20:43.933" LastActivityDate="2017-04-02T23:02:21.207" Title="Decrease execution time using openMP" Tags="|optimization|c|hpc|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26544" PostTypeId="1" CreationDate="2017-04-02T21:33:24.683" Score="0" ViewCount="1043" Body="&lt;p&gt;I have a code I'm working on that involves solving a 1D Schrodinger equation using a Crank-Nicolson time step.  The code is written in NumPy/SciPy, and I was doing a bit of profiling and discovered that the bulk of the simulation time is taken up with calls to the linear solver (spsolve in this case).  I'm using a simple discretization such that my matrices are symmetric and tridiagonal, and I was wondering if anyone had tried doing things like implementing the Thomas algorithm directly and if this was able to beat spsolve.&lt;/p&gt;&#xA;" OwnerUserId="14672" LastActivityDate="2017-04-02T21:33:24.683" Title="Tridiagonal Solver in Python" Tags="|python|linear-solver|scipy|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26545" PostTypeId="2" ParentId="26543" CreationDate="2017-04-02T23:02:21.207" Score="3" Body="&lt;p&gt;The algorithm you are using spawns extremely cheap tasks, especially at small $n$. As you don't have a luxury of changing the algorithm, the only reasonable (relatively) way to avoid the overhead of creating cheap tasks would be to introduce another &lt;code&gt;if&lt;/code&gt;-condition near the bottom of the recursion tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, now you have a baseline case at $n&amp;lt;2$ (simply &lt;code&gt;return n&lt;/code&gt;). I suggest you add another &lt;code&gt;if&lt;/code&gt;-condition for $n&amp;lt;k$, at first trying $k&amp;lt;\lfloor\frac{n}{2}\rfloor$, for which the value of the function will be calculated without spawning OpenMP tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One other thing is that you forgot to add the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma omp parallel&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;before the section&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#pragma omp single nowait&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Such &lt;code&gt;#pragma&lt;/code&gt; causes all of the threads in the pool to execute the next block of code. This might shave off a little bit of the timing in the second version code and make the execution time closer to the first one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also suggest the following webpage about basic &lt;a href=&quot;https://en.wikibooks.org/wiki/OpenMP/Tasks&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenMP+Tasks&lt;/a&gt; where you might find an appropriate balance between &lt;em&gt;still following the assignment&lt;/em&gt; and &lt;em&gt;making the most out of the computation&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-04-02T23:02:21.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26546" PostTypeId="2" ParentId="20277" CreationDate="2017-04-02T23:48:40.240" Score="1" Body="&lt;p&gt;I think that this book is perfect for you, as it did for me:&#xA;Discovering Modern C++: An Intensive Course for Scientists, Engineers, and Programmers (C++ In-Depth) by Peter Gottschling especially if used in conjunction with Programming Principles and Practice Using C++ 2nd Edition Bjarne Stroustrup. The inventor of C++ himself. Both should provide a solid ground to stand on.&lt;/p&gt;&#xA;" OwnerUserId="23828" LastActivityDate="2017-04-02T23:48:40.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26548" PostTypeId="2" ParentId="26539" CreationDate="2017-04-03T07:55:25.350" Score="1" Body="&lt;p&gt;Depends on the level from where you consider your problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the mathematical equations are concerned, the ODE formulation is preferable, since there are a lot of fine-tuned codes out there, that solve ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have concerns about the validity of your solutions, you should step back and check where the equations came from. As you said, often there are (algebraic) conservation laws in the background. In theory, these laws are implicitly satisfied. In practice, however, there are deviations. From this point of view, it is preferable to have the constraints explicitly in the equations that are solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One more general remark. It is valid to say that under certain continuity assumptions, a sufficiently accurate approximation of the ODEs will satisfy the implicit constraints with arbitrary accuracy. However,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;in applications, the constraints (think of the train that should stay on the rails) need to be fulfilled exactly&lt;/li&gt;&#xA;&lt;li&gt;in theory, sometimes the ODEs are derived with the assumption that the conservation law is exactly fulfilled (e.g. the Navier--Stokes equations in the standard formulation of the convection term). Then an approximate solution may render the model itself invalid.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2017-04-03T07:55:25.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26554" PostTypeId="1" AcceptedAnswerId="26566" CreationDate="2017-04-04T20:17:16.193" Score="0" ViewCount="174" Body="&lt;p&gt;I have to find and plot a numerical solution for the following equation (I have to write a solver):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{t} = (u^2 u_x)_x$$ with the following conditions $u(0,t) =0, u(1,t) = \sqrt{\frac{2c-2}{t}}, u(x,0) = 0$, where $c$ is a speed of the wave&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to use the following method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u_{m}^{n+1} - u_{m}^n}{\tau} = \frac{F_{m +1/2}^n - F_{m-1/2}^n}{h}$$&#xA;where $$F_{m+1/2}^n = (u_{m+1/2}^n)^2(\frac{u_{m+1}^n - u_{m-1}^n}{h})$$&#xA;where $$u_{m+1/2}^n = \frac{u_{m+1}^n + u_{m}^n}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tt = 0.1;&#xA;&#xA;tnn = 10000;&#xA;&#xA;xnn = 20;&#xA;&#xA;tau = tt/tnn;&#xA;&#xA;h = 1/xnn;&#xA;&#xA;u0[x_] := x;&#xA;&#xA;Do[v[m, 0] = u0[h*m], {m, 0, xnn}]&#xA;&#xA;Do[v[0, n] = 0; v[xnn, n] = 0;&#xA; Do[v[m, n + 1] = &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;v[m, n] + &#xA;    tau/h*(     ((v[m + 1, n] - v[m, n])/&#xA;          h)&lt;em&gt;(((v[m + 1, n])^2 + (v[m, n])^2)/&#xA;          2) - ((v[m, n] - v[m - 1, n])/&#xA;          h)&lt;/em&gt;(((v[m, n])^2 + (v[m - 1, n])^2)/2) ), {m, 1, &#xA;   xnn - 1}], {n, 0, tnn - 1}]&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u = ListInterpolation[&#xA;   Table[v[i, j], {i, 0, xnn}, {j, 0, tnn}], {{0, 1}, {0, tt}}];&#xA;&#xA;Plot3D[u[x, t], {x, 0, 1}, {t, 0, tt}, PlotRange -&amp;gt; All]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have changed the code and have got the correct solvation, but for the u(0,t)=u(1,t) = 0, u(x,0) = f(x). I don't actually understand how should I write the conditions like u(1,t) = f(t).&lt;/p&gt;&#xA;" OwnerUserId="23846" LastEditorUserId="23846" LastEditDate="2017-04-06T20:49:47.947" LastActivityDate="2017-04-06T20:49:47.947" Title="Numerical solution of nonlinear thermoconductivity equation" Tags="|finite-difference|nonlinear-equations|mathematica|differential-equations|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="26557" PostTypeId="1" AcceptedAnswerId="26589" CreationDate="2017-04-05T08:06:57.830" Score="6" ViewCount="190" Body="&lt;p&gt;I have been pondering about this issue for some time...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say, I want to minimize a costfunctional&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tilde J(u) = J(v(u),u) = \frac 12 \int_0^T (v-v_0)^2 + \alpha u^2 dt&#xA;$$&lt;/span&gt;&#xA;subject to&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\dot v = v^2 + u, \quad v(0)=0.&#xA;$$&lt;/span&gt;&#xA;Then, from the first order necessary optimality conditions it follows that at an optimal point &lt;span class=&quot;math-container&quot;&gt;$(v^*,u^*)$&lt;/span&gt;, there is a &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; that solves&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;-\dot \lambda = 2v^*\lambda - (v^*-v_0), \quad \lambda(T)=0, \tag{1}&#xA;$$&lt;/span&gt;&#xA;such, that that the gradient of &lt;span class=&quot;math-container&quot;&gt;$\tilde J$&lt;/span&gt; is given as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D_u\tilde J(u^*) = \lambda + \alpha u^*=0. \tag{2}&#xA;$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;From &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt;, I infer that &#xA;&lt;span class=&quot;math-container&quot;&gt;$$u^*(T) = 0.$$&lt;/span&gt; &#xA;My Question: &lt;strong&gt;Is this right?&lt;/strong&gt; Or is there something wrong in the argumentation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Two more remarks:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I don't think that the answer lies in the &lt;em&gt;right&lt;/em&gt; functional analytic formulation. (I have had a very close look at this). &lt;/li&gt;&#xA;&lt;li&gt;If one uses gradient based methods, then &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt; implies that the update is zero at &lt;span class=&quot;math-container&quot;&gt;$t=T$&lt;/span&gt;. Which means that the terminal value of the converged control will equal the terminal value of the initial guess. (I have seen this in a master thesis, where a fluid/structure interaction was successfully controlled by the adjoint based approach; see the screenshot)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kvKMA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kvKMA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: The conclusion that &lt;span class=&quot;math-container&quot;&gt;$u^*(T) = 0$&lt;/span&gt; (as well as the update) is &lt;strong&gt;not&lt;/strong&gt; true. (Thanks to L.P. for pointing this out) However, in practice, in a gradient descent method, one updates an initial guess &lt;span class=&quot;math-container&quot;&gt;$u_0$&lt;/span&gt; via &lt;span class=&quot;math-container&quot;&gt;$u_1 = u_0 - s D_u\tilde J(u_0)$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$\lambda(T) = 0$&lt;/span&gt; implies that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_1(T) = (1-s\alpha)u_0(T)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that the step size &lt;span class=&quot;math-container&quot;&gt;$s=\mathcal O(1)$&lt;/span&gt; whereas &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; can be like &lt;span class=&quot;math-container&quot;&gt;$10^{-5}$&lt;/span&gt; so that a gradient iteration, in fact, hardly affects the endpoint of the control.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2019-05-14T07:08:21.243" LastActivityDate="2019-05-14T12:22:57.323" Title="zero terminal value of the adjoint based optimal control" Tags="|optimization|optimal-control|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="26558" PostTypeId="1" CreationDate="2017-04-05T12:40:51.500" Score="1" ViewCount="324" Body="&lt;p&gt;I try to solve the following mixed second order elliptic PDE in the domain $D=[0, 1]^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray*}&#xA;    v+\nabla p=&amp;amp;0 \quad &amp;amp;\text{in} \quad D,\\&#xA;    \text{div}(v)=&amp;amp;1/2 \quad &amp;amp;\text{in} \quad  D,\\&#xA;    v\cdot n =&amp;amp;0 \quad &amp;amp;\text{on} \quad \partial D,\\&#xA;\end{eqnarray*}&#xA;where $n$ is the unit outward normal vector for $D$. Divided the domain into four sub-squares: $[0,0.5]^2$, $[0.5,1] \times [0, 0.5]$, $[0,0.5] \times [0.5,1]$ and $[0.5,1]^2$, denote the partition as $\mathcal{T}^h$. We want to solve the above problem using (rectangular) &lt;em&gt;lowest-order Raviart Thomas element&lt;/em&gt; $RT_0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take the reference square to be $[0,1]^2$, then we can compute four $RT$ basis functions as follow&#xA;$$ \psi_1(x_1,x_2) = (x_1,0)^T $$&#xA;$$ \psi_2(x_1,x_2) = (0, x_2-1)^T $$&#xA;$$ \psi_3(x_1,x_2) = (x_1-1, 0)^T $$&#xA;$$ \psi_4(x_1,x_2) = (0,x_2)^T.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can transform these basis functions to our sub-squares defined above. For example, in $[0,0.5]^2$, &#xA;$$\tilde{\psi_1} (x_1,x_2) = (2x_1,0)^T.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weak formulation of the above PDE is: find $(v,p)\in V_h \times Q$ such that&#xA;\begin{eqnarray*}&#xA; \int_D v \cdot w - \int_D p \text{div}(w)= &amp;amp; 0 \quad &amp;amp; \forall w \in V_h,\\&#xA; \int_D \text{div}(v) q = &amp;amp;\frac{1}{2}\int_D q \quad &amp;amp; \forall q \in Q,&#xA;\end{eqnarray*}&#xA;where $V_h = \{ w\in \big(L^2(D)\big)^2: w\cdot n =0\}$ and $Q = \{ q: q|_K = const, K\in \mathcal{T}^h\}.$&#xA;After we derive the weak formulation, the finite element system will look like the following&#xA;$$ \left [ &#xA;\begin{array}{cc}&#xA;B &amp;amp; C \\&#xA;C^T &amp;amp; 0 &#xA;\end{array} \right ] \left [ \begin{array}{c}&#xA;\tilde{v} \\&#xA;\tilde{p}&#xA;\end{array} \right ] = \left [ \begin{array}{c}&#xA;f_v \\&#xA;f_p&#xA;\end{array} \right ].&#xA;$$&#xA;&lt;strong&gt;Here is my question:&lt;/strong&gt; In our special case, it should be $B,C \in \mathbb{R}^{4\times 4}$ and $\tilde{f_v}$ zero vector. I am not clear that which $RT$ basis functions I should consider when forming the finite element matrix. I think there is one basis function per interior edge for $RK_0$. Also, I want to know how the matrices $B$ and $C$ and the vector $\tilde{f_p}$ exactly look like. (Both the numeric and analytical expression)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Another question is:&lt;/strong&gt; What is the compact support of these $RT$ basis functions from the corresponding sub-squares? Say $\tilde{\psi_1}$ in the sub-square $[0,0.5]^2$, what is the compact support of $\tilde{\psi_1}$? How about the other $\tilde{\psi_i}$ in the same sub-squares $[0,0.5]^2$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remark: To approximate $v$, we use $RT_0$ basis functions. For $p$, we use the piecewise constant function.&lt;/p&gt;&#xA;" OwnerUserId="17653" LastEditorUserId="17653" LastEditDate="2017-04-05T16:25:55.470" LastActivityDate="2017-05-06T01:42:03.207" Title="An example of mixed elliptic problem using lowest-order Raviart Thomas element" Tags="|finite-element|pde|elliptic-pde|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26559" PostTypeId="1" CreationDate="2017-04-05T16:25:37.717" Score="4" ViewCount="756" Body="&lt;p&gt;Given a first order ODE $y'(x)=f(x,y)$ with the initial condition $y(x_0)=x_0$ such that it satisfies Picard thoerem of existence and uniquness, one can compute the solution by Picard iterations :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ y_0 (x) \equiv y_0 \, ,$$ and&#xA;$$ y_{n+1}(x) = y_0 + \int\limits_{x_0}^x f(t,y_n (t)) \, dt \, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Computationally speaking,&lt;/strong&gt; &#xA;how fast is this method? Do we have a theorem about its convergence rate? Do we have particulary &quot;bad&quot; examples for its convergence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, I know this method is not the usual go-to for numerical solutions of ODE's, but why?&lt;/p&gt;&#xA;" OwnerUserId="15188" LastEditorUserId="9667" LastEditDate="2017-04-08T21:45:32.573" LastActivityDate="2017-04-08T21:45:32.573" Title="Convergence rate of Picard iterations" Tags="|numerical-analysis|ode|iterative-method|convergence|fixed-point|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26560" PostTypeId="1" CreationDate="2017-04-05T20:21:49.757" Score="1" ViewCount="334" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a stiff system of 6 ODEs, represented in MATLAB as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;system = @(t,x)[x(1,:).*x(2,:).*(-5.726882618492327e8)-x(1,:).*x(3,:).*1.467710449114545e10-x(1,:).*x(4,:).*3.012162507288153e12+x(3,:).*x(6,:).*1.137674873581712e12+x(4,:).*x(6,:).*5.703626484852603e12+x(5,:).*x(6,:).*1.656793950858653e9-x(1,:).^2.*2.980001443206009e8+x(6,:).^2.*8.25e9;x(1,:).*x(2,:).*(-5.726882618492327e8)+x(3,:).*x(6,:).*1.137674873581712e12-x(2,:).^2.*4.476510081133096e2+x(6,:).^2.*3.403491297089207e3;x(1,:).*x(2,:).*5.726882618492327e8-x(1,:).*x(3,:).*1.467710449114545e10-x(3,:).*x(6,:).*1.137674873581712e12+x(4,:).*x(6,:).*5.703626484852603e12;x(1,:).*x(3,:).*1.467710449114545e10-x(1,:).*x(4,:).*3.012162507288153e12-x(4,:).*x(6,:).*5.703626484852603e12+x(5,:).*x(6,:).*1.656793950858653e9;x(5,:).*(-8.186979330234089e8)+x(6,:).*2.1e9+x(1,:).*x(4,:).*3.012162507288153e12-x(5,:).*x(6,:).*1.656793950858653e9;x(5,:).*8.186979330234089e8-x(6,:).*2.1e9+x(1,:).*x(2,:).*5.726882618492327e8+x(1,:).*x(3,:).*1.467710449114545e10+x(1,:).*x(4,:).*3.012162507288153e12-x(3,:).*x(6,:).*1.137674873581712e12-x(4,:).*x(6,:).*5.703626484852603e12-x(5,:).*x(6,:).*1.656793950858653e9+x(1,:).^2.*2.980001443206009e8+x(2,:).^2.*4.476510081133096e2-x(6,:).^2.*8.250003403491297e9];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I use an ODE solver, I can solve the ODE (shown &lt;a href=&quot;https://i.stack.imgur.com/hggLs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) and use it to approximate the steady-state solution. As an example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[t,x] = ode15s(system,[0 1e-6],[0 0 0 0 0 1]);&#xA;plot(t,x)&#xA;ss_sol = x(end,:);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For reference, the steady-state solution obtained in this way is approximately &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ss_sol = [0.322330352943109, 0.458043435766086, 0.001213186698607, 0.000016426443105, 0.157136142002196, 0.061260531336451];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Even if I change the initial conditions a bit, I come back to this solution. Note that &lt;code&gt;sum(ss_sol)&lt;/code&gt; is approximately 1 (the ODEs were derived under this assumption). Also note that due to physical constraints, each variable must be between 0 and 1, which the solution satisfies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the steady-state solution, I'd like to set the derivatives equal to zero and solve the analogous, nonlinear algebraic system for its roots. The system is slightly rewritten in MATLAB as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;system = @(x)[x(1).*x(2).*(-5.726882618492327e8)-x(1).*x(3).*1.467710449114545e10-x(1).*x(4).*3.012162507288153e12+x(3).*x(6).*1.137674873581712e12+x(4).*x(6).*5.703626484852603e12+x(5).*x(6).*1.656793950858653e9-x(1).^2.*2.980001443206009e8+x(6).^2.*8.25e9,x(1).*x(2).*(-5.726882618492327e8)+x(3).*x(6).*1.137674873581712e12-x(2).^2.*4.476510081133096e2+x(6).^2.*3.403491297089207e3,x(1).*x(2).*5.726882618492327e8-x(1).*x(3).*1.467710449114545e10-x(3).*x(6).*1.137674873581712e12+x(4).*x(6).*5.703626484852603e12,x(1).*x(3).*1.467710449114545e10-x(1).*x(4).*3.012162507288153e12-x(4).*x(6).*5.703626484852603e12+x(5).*x(6).*1.656793950858653e9,x(5).*(-8.186979330234089e8)+x(6).*2.1e9+x(1).*x(4).*3.012162507288153e12-x(5).*x(6).*1.656793950858653e9,x(5).*8.186979330234089e8-x(6).*2.1e9+x(1).*x(2).*5.726882618492327e8+x(1).*x(3).*1.467710449114545e10+x(1).*x(4).*3.012162507288153e12-x(3).*x(6).*1.137674873581712e12-x(4).*x(6).*5.703626484852603e12-x(5).*x(6).*1.656793950858653e9+x(1).^2.*2.980001443206009e8+x(2).^2.*4.476510081133096e2-x(6).^2.*8.250003403491297e9];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I have trouble getting the same values of &lt;code&gt;ss_sol&lt;/code&gt; mentioned earlier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;vpasolve&lt;/code&gt;, even when I increase the precision to 100+ digits, does not lead to any solution. Using &lt;code&gt;fsolve&lt;/code&gt; or &lt;code&gt;lsqnonlin&lt;/code&gt; (the latter of which lets me set the lower and upper bounds of 0 and 1, respectively) converge to values that are not the steady-state solution, unless I supply &lt;code&gt;ss_sol&lt;/code&gt; as the initial guess. I try decreasing the tolerances and increasing the number of iterations but still no luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions? Is the best way really to just integrate the ODEs to steady-state? I keep reading that reducing the stiff system of ODEs to an algebraic equation should be easier to solve, but I just can't find the steady-state solutions this way. I need a general way to solve for the steady-state values for many different systems, so the problem extends beyond the aforementioned specific example.&lt;/p&gt;&#xA;" OwnerUserId="22245" LastEditorUserId="22245" LastEditDate="2017-04-06T00:31:26.723" LastActivityDate="2017-04-06T00:31:26.723" Title="Trouble getting steady-state solution by solving system of nonlinear algebraic equations in MATLAB" Tags="|optimization|matlab|numerical-analysis|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26561" PostTypeId="2" ParentId="26560" CreationDate="2017-04-05T23:21:50.710" Score="2" Body="&lt;p&gt;The problems you are encountering are not unusual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonlinear systems of equations, as you have already&#xA;discovered, often have multiple solutions. It can be&#xA;difficult to find the specific one you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basic Newton's method is not guaranteed to converge to&#xA;&lt;em&gt;any&lt;/em&gt; solution given a poor initial guess. (I had thought&#xA;that the MATLAB/fsolve algorithm would be able to find&#xA;&lt;em&gt;some&lt;/em&gt; solution in this case, however.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a huge body of literature on techniques for solving&#xA;nonlinear algebraic equations. A large subset of this literature&#xA;deals with algorithms called &quot;continuation methods&quot;  designed&#xA;to deal with particularly challenging problems. Many of these&#xA;continuation methods are very similar to ODE solution methods!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, I'll assume you are more interested in simply&#xA;solving your equations than researching numerical methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I suggest:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a good initial guess, basic Newton's methods will&#xA;converge. In your case, it looks like your &lt;code&gt;ode15s&lt;/code&gt; solution is&#xA;approximately steady-state at t=2e-7 which is significantly&#xA;before your actual termination time of t=1e-6. Since you only&#xA;need an approximate guess, you can probably also reduce the&#xA;accuracy of &lt;code&gt;ode15s&lt;/code&gt; (by setting the options &lt;code&gt;AbsTol&lt;/code&gt; and &lt;code&gt;RelTol&lt;/code&gt;)&#xA;by at least an order of magnitude. Try calculating an initial&#xA;guess by this approach and then passing it to &lt;code&gt;fsolve&lt;/code&gt; to&#xA;calculate the final steady-state solution.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-04-05T23:21:50.710" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26562" PostTypeId="1" AcceptedAnswerId="26572" CreationDate="2017-04-05T23:36:17.333" Score="6" ViewCount="6863" Body="&lt;p&gt;Assume, we have an $m\times n$ block matrix $M=\left[\begin{array}{c c}A&amp;amp;C\\B&amp;amp;D \end{array}\right]$, where&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$A$ is an $m_1 \times n_1$ matrix of rank $k_A$.&lt;/li&gt;&#xA;&lt;li&gt;$B$ is an $m_2 \times n_1$ matrix of rank $k_B$.&lt;/li&gt;&#xA;&lt;li&gt;$C$ is an $m_1 \times n_2$ matrix of rank $k_C$.&lt;/li&gt;&#xA;&lt;li&gt;$D$ is an $m_2 \times n_2$ matrix of rank $k_D$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Obviously, $m_1+m_2=m$ and $n_1+n_2=n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper by &lt;a href=&quot;https://www.jstor.org/stable/2100027&quot; rel=&quot;noreferrer&quot;&gt;Carl D. Meyer &quot;Generalized inverses and ranks of block matrices&quot;, &lt;em&gt;SIAM J. Appl. Math&lt;/em&gt;, vol. 25, no. 4, pp. 597-602, Dec. 1973&lt;/a&gt;, I found a way to bound the rank of a block matrix $M$ by the rank of its subblocks, as follows (Corollary 4.1):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{equation}&#xA;\newcommand{\rank}[1]{\text{rank}(#1)}&#xA;\rank{M}\leq\rank{A}+\rank{B}+\rank{C}+\rank{Z},&#xA;\tag{1}\label{1}&#xA;\end{equation}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $Z=D-BA^-C$, thus leading to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{equation}&#xA;\rank{M}\leq k_A+k_B+k_C+\underbrace{\big(k_D+\min(k_A,k_B,k_C)\big)}_{\rank{Z}},&#xA;\tag{2}\label{2}&#xA;\end{equation}$$&#xA;as $\rank{Z}\leq \rank{D}+\rank{BA^-C}=k_D+\min(k_A,k_B,k_C)$ by the properties of matrix ranks: &lt;a href=&quot;https://en.wikipedia.org/wiki/Rank_(linear_algebra)#Properties&quot; rel=&quot;noreferrer&quot;&gt;rank of a sum and rank of a product&lt;/a&gt;.&#xA;So, we can convert an original result $\eqref{1}$ to &#xA;$$&#xA;\begin{equation}&#xA;\rank{M}\leq k_A+k_B+k_C+k_D+\min(k_A,k_B,k_C)\leq 5\underbrace{\max{(k_A,k_B,k_C,k_D)}}_{k_\max}.&#xA;\tag{3}\label{3}&#xA;\end{equation}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is anybody aware of any way to come up with a tighter (than $\eqref{1}$ or $\eqref{3}$) bound to a rank of a $2\times 2$ block matrix in terms of the ranks matrices of its subblocks? The reason is that in my numerical experiments (matrices arise from boundary value problem) I cannot come up even close to this bound (usually staying safely within $2k_\max$) - which proves nothing, but certainly makes me think.&lt;/li&gt;&#xA;&lt;li&gt;The original paper talks about exact ranks. Would this or any other bound change significantly if we start talking about the numerical rank $\tilde{k}_M(\epsilon)$ in terms of numerical ranks of its subblocks $\tilde{k}_{A,B,C,D}(\epsilon)$? Here $\epsilon$ is the relative tolerance for a truncated SVD. In other words, if $\tilde{k}_{M}(\epsilon)=5$, then $\sigma_6/\sigma_1&amp;lt;\epsilon$. ($\sigma_{1},\ldots,\sigma_{\min(m,n)}$ being the singular values of $M$)&lt;/li&gt;&#xA;&lt;li&gt;Is there an efficient way to construct a truncated SVD of $M$ from already computed $\epsilon$-truncated SVD's of subblocks $A,B,C,D$? and how to estimate the error in the resultant truncated SVD of the original matrix $M$? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2017-04-06T07:40:58.440" LastActivityDate="2017-04-06T22:58:42.027" Title="Block-matrix SVD and rank bounds" Tags="|linear-algebra|matrix|svd|block-decomposition|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26563" PostTypeId="2" ParentId="26558" CreationDate="2017-04-06T00:54:30.797" Score="1" Body="&lt;p&gt;The size of the system depends on whether you eliminate the degrees of freedom from your system or not. If you eliminate them, then $B$ will be a 4x4 matrix (one unknown per interior edge), $C$ will be also 4x4 (but in general case it will be rectangular).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that you enumerate first dofs for $x$-components and then for $y$-components in a standard way, $B$ will be a block diagonal matrix with blocks corresponding to $x$ and $y$-directions. Each block is  (before elimination of boundary dofs) a tridiagonal matrix with a row &#xA;$$&#xA;\left\{ h_x h_y \frac{1}{6}, h_x h_y \frac{4}{6}, h_x h_y \frac{1}{6} \right\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you again use the standard enumeration for dofs for $p$, matrix $C$ before elimination will be a block matrix which basically gives you simply the two-point approximation of the $- \nabla$ operator, with entries $\pm h_x$ and $\pm h_y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The support of each basis function for $\mathbf{v}$ are the two rectangles which shared the edge with the corresponding degree of freedom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Vector $\tilde{f_p}$ has the elements $\frac{1}{2}\int_D \chi_j$ where $\chi_j$ is the characteristic function of a mesh cell, that is, it will be just the area which equals $h_x h_y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you first write down the local $B_{loc}$ and $C_{loc}$ matrices and then look at the assembling procedure which will produce the global matrix in a straight-forward way.&lt;/p&gt;&#xA;" OwnerUserId="21958" LastEditorUserId="21958" LastEditDate="2017-04-06T01:00:13.473" LastActivityDate="2017-04-06T01:00:13.473" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26564" PostTypeId="2" ParentId="26435" CreationDate="2017-04-06T03:56:41.980" Score="0" Body="&lt;p&gt;Since you are trying to solve a scalar field $C$, you should be worry about boundary conditions for your primal variable in the case $C$, injecting water or gas will describe the velocity field $v$ to be incompressible ($\nabla \cdot v = 0$) or not ($\nabla \cdot v \neq 0$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the other hand if you are solving the velocity field $v$ with a CFD solver, then you are simulating the mixing of two flows, in that case imposing velocity at both inlets and pressure at outlets may be a good form to tackle your problem.&lt;/p&gt;&#xA;" OwnerUserId="23860" LastActivityDate="2017-04-06T03:56:41.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26565" PostTypeId="1" CreationDate="2017-04-06T06:51:53.097" Score="5" ViewCount="972" Body="&lt;p&gt;In an optical wavefront propagation problem, I need to do excessive Fourier-type computations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;N=10000&#xA;input=np.random.random(N)&#xA;&#xA;x=np.linspace(-1,1,N)&#xA;y=np.linspace(-2,2,N)&#xA;X,Y=np.meshgrid(x,y,sparse=True)&#xA;&#xA;output=np.dot(np.exp(1j*2*np.pi*X*Y),input)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the true kernel (here, the complex exponential) is more complicated. In particular, it is not periodic, so no FFT-type reduction is possible! So, briefly, I need to evaluate trigonometric functions on a very large grid and then do a matrix-vector multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this with pure &lt;strong&gt;numpy&lt;/strong&gt; takes more than a week for one execution of my computations on a middle class desktop machine (with eight cores). Using &lt;strong&gt;numexpr&lt;/strong&gt; reduces the runtime by a factor of roughly 1/3. Employing &lt;strong&gt;parallel Cython&lt;/strong&gt; instead further reduces runtime by a factor of about 8/10. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, this is still too slow. I'm not an expert on scientific computing at all. So, for me, it is hard to judge how to speed this up further. Hence, my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do there exist other software frameworks (numba, weave, OpenCL, ...) that are better suited to this problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What kind of hardware upgrade would make sense? (Obviously, the main computational costs are the function evaluations.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;The continuous problem is a generalized one-dimensional Fresnel propagator&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u(x)= \int_{-\infty}^\infty \exp\left(i\pi\frac{(x-y)^2}{\lambda z(x,y)}\right) \frac{u_0(y)}{\sqrt{i\lambda z(x,y)}} dy,$$&#xA;where $\lambda$ is the wavelength of light and z is a positive function which is smooth, but more or less arbitrary&lt;/p&gt;&#xA;" OwnerUserId="23861" LastEditorUserId="23861" LastEditDate="2017-04-07T13:41:03.390" LastActivityDate="2017-04-08T11:10:20.187" Title="Speeding up a linear transform using Python" Tags="|python|parallel-computing|software|numpy|multicore|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26566" PostTypeId="2" ParentId="26554" CreationDate="2017-04-06T07:01:06.380" Score="1" Body="&lt;p&gt;You got good advices in comments, but as it is only 1D problem, you can manage it using fully explicit method with very small time steps. I checked your code in Mathematica and there are several misunderstandings there. After correcting them, the code works for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, your description in the question contains, I suppose, some mistakes. As I guess that this is some student project, so let me only indicate what shall be corrected ;-).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your method in the question can be viewed as finite volume method for computational cells $(x_{m-1/2},x_{m+1/2})$. The quantity $F_{m +1/2}^n$ approximates the flux at the right point, the approximation in your question is inappropriate, check it, it shall use so-called central difference at point $x_{m+1/2}$. Analogously the nonlinear conductivity coefficient $u_{m+1/2}^n$ at $x_{m+1/2}$ is approximated inproperly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your code anyway does not implement what you describe in your question. So, check the scheme and implement it exactly and I suppose your code can work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be careful about a correct input data for your model. In your implementation you use different boundary and initial conditions as you describe in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope after these indications you manage to finish the task, good luck!&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2017-04-06T07:01:06.380" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26569" PostTypeId="1" CreationDate="2017-04-06T14:32:03.033" Score="1" ViewCount="97" Body="&lt;p&gt;Let $\Delta\subseteq\mathbb R^2$ denote the triangle spanned by $(0,0)$, $(1,0)$ and $(0,1)$ and $$\mathbb P_r(\Delta):=\left\{p:\Delta\to\mathbb R\mid p(x)=\sum_{|\alpha|\le r}\lambda_\alpha x^\alpha\text{ for all }x\in\Delta\text{ for some }(\lambda_\alpha)_{|\alpha|\le r}\subseteq\mathbb R\right\}$$ for $r\in\mathbb N_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm numerically solving a PDE on a rectangle $\Lambda=(0,a)\times(0,b)$ triangulated using &lt;em&gt;quadratic Lagrange&lt;/em&gt; finite elements as depicted in the following figure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GlCEH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GlCEH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the assembly of the linear system, I need to compute integrals of the form $$\int_\Delta(u^0\circ F_{\tilde\Delta})\psi\;,\tag1$$ where $u^0$ is the solution of the previous time step, $F_{\tilde\Delta}$ is the transformation from a finite element $\tilde\Delta$ to $\Delta$ and $\psi\in\mathbb P_2(\Delta)$.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How should I compute these integrals?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By definition of the finte element space, $\left.u^0\right|_{\tilde\Delta}\in\mathbb P_2(\tilde\Delta)$. So, the integrand in $(1)$ belongs to $\mathbb P_4(\Delta)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's why I guess I should use a quadrature scheme which exactly integrations $\mathbb P_4(\Delta)$-functions. However, since I only now the values of $u^0\circ F_{\tilde\Delta}$ at $a^0:=(0,0)$, $a^1:=(1,0)$, $a^2:=(0,1)$, $a^3:=(1/2,1/2)$, $a^4:=(0,1/2)$ and $a^5:=(1/2,0)$, the Gaussian quadrature ansatz $$\int_\Delta f\approx\frac12\sum_{i=0}^5w_if(a^i)\tag2$$ is somehow limited. In fact, with these $a^i$ it's only possible to exactly integrate $\mathbb P_2(\Delta)$-functions (and we have $w_0=w_1=w_2=0$, $w_3=w_4=w_5=1/3$).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So, is there no possibility to do better?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Please suppose that we have a fixed mesh (i.e. no adaptive refinement) we need to deal with.&lt;/p&gt;&#xA;" OwnerUserId="23411" LastActivityDate="2017-04-06T14:32:03.033" Title="Quadrature in finite element methods | How should I compute integrals involving the solution of the last time step?" Tags="|finite-element|numerical-analysis|quadrature|polynomials|integration|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26570" PostTypeId="1" CreationDate="2017-04-06T17:10:39.013" Score="2" ViewCount="760" Body="&lt;p&gt;I perform matrix matrix multiplications (between rank-3 and rank-2 arrays) in fortran using following subroutine, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine ab_c_dir1(n1,n2,n3,a,b,c)&#xA;&#xA;  implicit none&#xA;  integer :: n1,n2,n3&#xA;  real(rk) :: a(n1,n1),b(n1,n2,n3),c(n1,n2,n3)&#xA;&#xA;  call dgemm('n','n',n1,n2*n3,n1,1.d0,a,n1,b,n1,0.d0,c,n1)&#xA;&#xA;end subroutine ab_c_dir1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I like to have MPI version of it. &lt;em&gt;I understand that it can became less efficient.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How this can be done in an easy and clean way?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think about the options using &lt;code&gt;PDGEMM&lt;/code&gt; or &lt;code&gt;PETSc&lt;/code&gt;'s &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMatMult.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;matmatmult&lt;/code&gt;&lt;/a&gt;(may be it is using &lt;code&gt;PDGEMM&lt;/code&gt;as well) function. I am already using PETSC in the same code to solve a linear system, also I find &lt;code&gt;PDGEMM&lt;/code&gt; harder to grasp at first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other (better) approach? &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using OpenMP accelerated BLAS is working quite well. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Alternatively following subroutine can be used as well, please see the warning in the comment section from &lt;a href=&quot;https://scicomp.stackexchange.com/users/569/m-k-aka-grisu&quot;&gt;M.K. aka Grisu&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine ab_c_dir1_openMP(n1,n2,n3,a,b,c)&#xA;  !$ use OMP_LIB&#xA;  implicit none&#xA;  integer :: n1,n2,n3&#xA;  real(rk) :: a(n1,n1),b(n1,n2,n3),c(n1,n2,n3)&#xA;  integer :: k&#xA;&#xA;  !$OMP PARALLEL PRIVATE(k)&#xA;  !$OMP DO SCHEDULE(RUNTIME)&#xA;  do k=1,n3&#xA;    call dgemm('n','n',n1,n2,n1,1.d0,a,n1,b(1,1,k),n1,0.d0,c(1,1,k),n1)&#xA;  enddo&#xA;  !$OMP END DO&#xA;  !$OMP END PARALLEL&#xA;&#xA;end subroutine ab_c_dir1_openMP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10303" LastEditorUserId="-1" LastEditDate="2017-04-13T12:53:54.790" LastActivityDate="2018-06-06T02:07:41.073" Title="Distributed (MPI) matrix matrix multiplication" Tags="|linear-algebra|matrix|petsc|mpi|blas|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26572" PostTypeId="2" ParentId="26562" CreationDate="2017-04-06T18:36:54.720" Score="8" Body="&lt;p&gt;Let us begin with the exact&#xA;singular value decompositions $A=U_{A}S_{A}V_{A}^{T}$, $B=U_{B}S_{B}V_{B}^{T}$,&#xA;$C=U_{C}S_{C}V_{C}^{T}$, $D=U_{D}S_{D}V_{D}^{T}$. Then&#xA;$$&#xA;M=\underbrace{\begin{bmatrix}U_{A} &amp;amp; 0 &amp;amp; U_{C} &amp;amp; 0\\&#xA;0 &amp;amp; U_{B} &amp;amp; 0 &amp;amp; U_{D}&#xA;\end{bmatrix}}_{X}\underbrace{\begin{bmatrix}S_{A} &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; S_{B} &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; S_{C} &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; S_{D}&#xA;\end{bmatrix}}_{S}\underbrace{\begin{bmatrix}V_{A}^{T} &amp;amp; 0\\&#xA;V_{B}^{T} &amp;amp; 0\\&#xA;0 &amp;amp; V_{C}^{T}\\&#xA;0 &amp;amp; V_{D}^{T}&#xA;\end{bmatrix}}_{Y^{T}}\tag{*}&#xA;$$&#xA;is a low-rank decomposition of $M$. This immediately proves&#xA;$$&#xA;\mathrm{rank}(M)\le k_{1}+k_{2}+k_{3}+k_{4}.\tag{**}&#xA;$$&#xA;Note that the bound is not always tight, because $X$, $Y$ are not&#xA;completely orthogonal. What we can do is compute the QR factors&#xA;$$&#xA;X=Q_{X}R_{X},\qquad Y=Q_{Y}R_{Y},&#xA;$$&#xA;form the inner matrix $W=R_{X}SR_{Y}^{T}$, and compute its SVD $W=U_{W}\Sigma V_{W}^{T}$.&#xA;Then&#xA;$$&#xA;M=Q_{X}WQ_{Y}^{T}=\underbrace{Q_{X}U_{W}}_{U}\Sigma\underbrace{V_{W}^{T}Q_{Y}^{T}}_{V^{T}}=U\Sigma V^{T}&#xA;$$&#xA;is the exact singular value decomposition for $M$, and $\mathrm{rank}(M)=\mathrm{rank}(\Sigma)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For general rectangular matrices, (**) is the tightest bound&#xA;we can form without needing to perform linear algebra. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are many different definitions of numerical ranks. Assuming&#xA;that you are talking about the truncated SVD rank, i.e. the rank needed&#xA;for an $\epsilon$ approximation, then the same bound holds due to&#xA;the result below.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now suppose that each SVD of $A$, $B$, $C$, $D$ has been truncated&#xA;to achieve a Frobenius norm bound of $\epsilon$ (that is much greater than machine precision). Then the decomposition&#xA;(*) has a Frobenius norm bound of $2\epsilon$. If we perform the&#xA;next two steps to machine precision (which is easy, because the ranks&#xA;are small), then our resulting SVD for $M$ is also accurate to a&#xA;Frobenius norm of $2\epsilon$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2017-04-06T22:58:42.027" LastActivityDate="2017-04-06T22:58:42.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26573" PostTypeId="1" AcceptedAnswerId="26574" CreationDate="2017-04-07T00:22:24.240" Score="3" ViewCount="1163" Body="&lt;p&gt;Let's suppose I'm numerically solving the Poisson equation for a delta function source:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla^2 f(x) = \delta(x-x') $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can represent the Laplacian $\nabla^2$ using the finite difference method as a tri-diagonal matrix $A$. I can represent the delta function as a vector $b$ that is $0$ everywhere except at one point where it is $\frac{1}{dx}$ (here $dx$ denotes the length of the mesh).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, my resulting $Ax=b$ system of equations is very sparse -- the matrix has only $3N$ nonzero elements and the $b$ vector has only $1$ nonzero element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a numerical method that is optimized for solving $Ax=b$ for this rather specific case? I have seen some solvers that utilize the sparseness of the matrix, but I haven't found one that also utilizes the sparseness of the $b$ vector. I might be naive but I feel like a sparse $b$ vector should simplify the solving a lot.&lt;/p&gt;&#xA;" OwnerUserId="22988" LastEditorUserId="127" LastEditDate="2017-04-08T14:45:25.953" LastActivityDate="2017-04-08T14:45:25.953" Title="Solving Ax = b with sparse A and sparse b" Tags="|linear-algebra|finite-difference|sparse-matrix|poisson|laplacian|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26574" PostTypeId="2" ParentId="26573" CreationDate="2017-04-07T08:57:03.090" Score="8" Body="&lt;p&gt;When looking at the solution of your system, you will find that almost all entries of $x$ are nonzero although the right-hand side is &quot;sparse&quot;. Hence, whatever algorithm you use, it'll have to visit each and every entry at least once, so one wouldn't expect that you can save a lot of time using the sparsity of $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right-hand sides where you can save a lot of computation are, for example, multiples of eigenvectors of $A$ since the solution of&#xA;$$&#xA;Ax = \alpha v_i&#xA;$$&#xA;is clearly $\frac{\alpha}{\lambda_i} v_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, every full-rank tridiagonal equation system can be solved in $O(n)$. Using multigrid, the same goes for the Poisson equation in any dimension.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2017-04-07T08:57:03.090" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26575" PostTypeId="2" ParentId="26565" CreationDate="2017-04-07T11:22:47.953" Score="7" Body="&lt;p&gt;Assuming that your kernel is somewhat smooth, &lt;strong&gt;use low-rank approximation&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a naive example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;N=2000&#xA;input=np.random.random(N)&#xA;&#xA;x=np.linspace(-1,1,N)&#xA;y=np.linspace(-2,2,N)&#xA;X,Y=np.meshgrid(x,y,sparse=True)&#xA;A = np.exp(1j*2*np.pi*X*Y)&#xA;&#xA;output = np.dot(A, input)&#xA;&#xA;U,S,V = np.linalg.svd(A)&#xA;&#xA;# find truncation rank for given tolerance&#xA;k = np.nonzero(S &amp;lt; 1e-12)[0][0]&#xA;print('Truncation rank:', k)&#xA;Xk = U[:,:k].dot(np.diag(S[:k]))&#xA;Yk = V[:k,:]&#xA;&#xA;output2 = Xk.dot(Yk.dot(input))&#xA;&#xA;# check quality of approximation&#xA;print(np.linalg.norm(output - output2, np.inf))&#xA;&#xA;%timeit np.dot(A, input)&#xA;%timeit Xk.dot(Yk.dot(input))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Truncation rank: 32&#xA;6.3535593536e-12&#xA;100 loops, best of 3: 3.9 ms per loop&#xA;10000 loops, best of 3: 61.4 µs per loop&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What the above does is compute a &lt;a href=&quot;http://langvillea.people.cofc.edu/DISSECTION-LAB/Emmie&amp;#39;sLSI-SVDModule/p5module.html&quot; rel=&quot;noreferrer&quot;&gt;truncated singular value decomposition&lt;/a&gt; of your matrix and use the resulting low-rank approximation of A to speed up the matrix-vector product. So, instead of using $A$, you use an approximation&#xA;$$&#xA; A \approx X_k Y_k^T,&#xA; \qquad X_k,Y_k \in \mathbb R^{N\times k}&#xA;$$&#xA;with $k \ll N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example, this leads to a speedup of more than 60x, and this will only get better as N gets larger. That's because the full matrix-vector product has complexity $\mathcal O(N^2)$, whereas the low-rank matrix-vector product has complexity $\mathcal O(Nk)$.&#xA;The result is not exact, but you can influence the accuracy by playing with the truncation rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, computing the SVD is very slow. Instead, there are faster alternatives for computing low-rank approximations to a matrix, for instance: &lt;a href=&quot;https://research.fb.com/fast-randomized-svd/&quot; rel=&quot;noreferrer&quot;&gt;fast randomized SVD&lt;/a&gt; or, what I would recommend, Adaptive Cross Approximation (ACA). The latter is a black-box algorithm for computing a low-rank approximation without ever even computing the entire matrix A and thus is very well suited for very large problems. It's also easy to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original literature on ACA (in particular, by M. Bebendorf from 2000 on) is somewhat inaccessible. Instead, here's a random paper which describes ACA in a basic form: &lt;a href=&quot;https://lirias.kuleuven.be/bitstream/123456789/137926/2/TW513.pdf&quot; rel=&quot;noreferrer&quot;&gt;(link)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your kernel is relatively simple, it might even be possible to derive a separable approximation to it analytically, which directly leads to a low-rank approximation instead of having to compute it algorithmically. Check the literature on &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Multipole_expansion&quot; rel=&quot;noreferrer&quot;&gt;multipole expansion&lt;/a&gt;&quot; and &quot;fast multipole method&quot; for many examples of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the bottleneck in your original program was actually computing A, not the matrix-vector product. ACA will eliminate that, since it doesn't need the full matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;UPDATE:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just realized that scipy already includes a fast method for approximating the SVD in the little-known &lt;code&gt;scipy.linalg.interpolative&lt;/code&gt; module. Here's an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.linalg.interpolative as sli&#xA;&#xA;U,S,V = sli.svd(A, 1e-12)&#xA;print('Rank:', U.shape[1])&#xA;&#xA;Xk = U.dot(np.diag(S))&#xA;Yk = V.conj().T&#xA;&#xA;output3 = Xk.dot(Yk.dot(input))&#xA;&#xA;# check quality of approximation&#xA;print(np.linalg.norm(output - output3, np.inf))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Rank: 37&#xA;2.62815545927e-10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The difference in timing between exact SVD and approximate SVD (again, this will get even better with larger N due to the involved asymptotics):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%timeit np.linalg.svd(A)&#xA;%timeit sli.svd(A, 1e-12)&#xA;&#xA;1 loop, best of 3: 11.4 s per loop&#xA;1 loop, best of 3: 297 ms per loop&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, this still requires you to compute the entire matrix A first and doesn't get rid of that bottleneck. Nevertheless, if you need to compute A only once and then have many matrix-vector products, it may be a suitable option.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastEditorUserId="14598" LastEditDate="2017-04-08T10:26:41.547" LastActivityDate="2017-04-08T10:26:41.547" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26577" PostTypeId="2" ParentId="26573" CreationDate="2017-04-07T15:49:42.147" Score="3" Body="&lt;p&gt;It is unclear to me from your question whether the answer is of theoretical&#xA;or practical interest. I'll address both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this to be of practical importance, either your system would need&#xA;to have a very large number of equations or you need to solve it many&#xA;times. I suspect that simply calling the Lapack function &lt;code&gt;dptsv&lt;/code&gt;, which&#xA;is specific to solving a symmetric tridiagonal system will be significantly&#xA;faster than invoking MATLAB backslash on a sparse matrix. And it will be&#xA;close enough to the optimal-time approach that spending further effort&#xA;would not be worthwhile. It does not exploit a sparse right-hand-side, however. This algorithm is described in section 4.3.6 of &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/1421407949&quot; rel=&quot;nofollow noreferrer&quot;&gt;Golub and Van Loan&lt;/a&gt;. A total of $8n$ floating point operations (flops) are required; $3n$ flops for the factorization of the matrix and $5n$ flops to compute the solution given the right-hand-side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chapter 3 of Davis' book &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898716136&quot; rel=&quot;nofollow noreferrer&quot;&gt;Direct Methods for Sparse Linear Systems&lt;/a&gt;&#xA;deals with solving a linear system with a triangular matrix on the&#xA;left-hand side. He spends considerable time on this topic because most&#xA;of the methods for general sparse matrices in this book rely on &#xA;solving these triangular systems $n$ times. He states that these methods&#xA;would be impractical if each triangular solve assumed that the right-hand-side&#xA;vector was dense. So he describes a linear solve algorithm that allows&#xA;for a general, sparse right-hand-side vector and implements this in function&#xA;&lt;code&gt;cs_spsolve&lt;/code&gt;. To solve a symmetric positive definite system with the software&#xA;described in this book would require a call to &lt;code&gt;cs_chol&lt;/code&gt; to factor the&#xA;sparse matrix followed by two calls to &lt;code&gt;cs_spsolve&lt;/code&gt;. This is interesting&#xA;theoretically but I'm doubtful it would result in less computational&#xA;time than the Lapack approach unless you are solving for many &#xA;right-hand-sides (e.g. many cases with a delta function at a different point in the mesh).&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-04-07T21:06:55.573" LastActivityDate="2017-04-07T21:06:55.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26578" PostTypeId="2" ParentId="26565" CreationDate="2017-04-07T17:59:37.340" Score="1" Body="&lt;p&gt;Matrix-vector multiplication is RAM-bound. You make two arithmetic operations for each float that you read. So the major cost is going to be storing the whole matrix in slow, non-cached core memory and reading it back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should consider evaluating your &quot;complicated kernel&quot; on the fly instead of saving it to memory:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;output = np.zeros(n)&#xA;for j in range(n):&#xA;  for i in range(n):&#xA;    output[i] += kernel(i, j) * input[j]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Warning: this loop is going to be extremely slow until you cythonize it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have to make a single matrix-vector product with each matrix, like in your example, this is likely going to be a major speedup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have to make many products with the same matrix, then it depends: try both approaches and compare them. (Also, if you have all the vectors available at the same time, store them in a matrix and turn this operation into a matrix-matrix multiplication). In the hardware department, the first thing to check is to make sure that you have enough RAM to store the whole matrix -- if you have to hit the disk, then things are going to get s..l..o..w. &lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2017-04-08T11:10:20.187" LastActivityDate="2017-04-08T11:10:20.187" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26579" PostTypeId="2" ParentId="26491" CreationDate="2017-04-07T22:51:05.030" Score="8" Body="&lt;p&gt;There is something very basic that you should know about hyperbolic problems. Consider the most basic example $\partial_tu+a\partial_xu=0$&#xA;with a numerical marching scheme of the form $$u_j^{n+1}=\sum_kc_ku_{j+k}^n$$.&#xA;This covers all explicit schemes, and all implicit schemes like Crank Nicolson also, if you begin by solving the tridiagonal system. It actually covers ALL numerical methods, of any kind, that are linear under addition and multiplication. Godunov's theorem, known since 1959, asserts that any method of that form that is better than first-order accurate will produce spurious oscillations. LeVeque's book will fill you in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important thing to stress is that this has little to do with stability, but is absolutely pervasive in CFD because the F stands for Fluid and fluids can distort a lot, so advection terms are very important, and there things that cannot be done, even in the most trivial cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rather amazing conclusion is that even linear hyperbolic problem must be solved by nonlinear numerical methods to achieve reasonable accuracy. All successful methods involve an element of feedback between the developing solution and the numerical scheme.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-04-07T22:51:05.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26581" PostTypeId="2" ParentId="7030" CreationDate="2017-04-08T00:17:09.853" Score="0" Body="&lt;pre&gt;&lt;code&gt;from mpl_toolkits.mplot3d import axes3d&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import matplotlib.animation as animation&#xA;&#xA;def generate(X, Y, phi):&#xA;    R = 1 - np.sqrt(X**2 + Y**2)&#xA;    return np.cos(2 * np.pi * X + phi) * R&#xA;&#xA;fig = plt.figure()&#xA;ax = axes3d.Axes3D(fig)&#xA;&#xA;xs = np.linspace(-1, 1, 50)&#xA;ys = np.linspace(-1, 1, 50)&#xA;X, Y = np.meshgrid(xs, ys)&#xA;Z = generate(X, Y, 0.0)&#xA;wframe = ax.plot_wireframe(X, Y, Z, rstride=2, cstride=2)&#xA;ax.set_zlim(-1,1)&#xA;&#xA;def update(i, ax, fig):&#xA;    ax.cla()&#xA;    phi = i * 360 / 2 / np.pi / 100&#xA;    Z = generate(X, Y, phi)&#xA;    wframe = ax.plot_wireframe(X, Y, Z, rstride=2, cstride=2)&#xA;    ax.set_zlim(-1,1)&#xA;    return wframe,&#xA;&#xA;def xrange(x):&#xA;    return iter(range(x))&#xA;&#xA;ani = animation.FuncAnimation(fig, update, frames=xrange(100), &#xA;    fargs=(ax, fig), interval=100)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23874" LastEditorUserId="20688" LastEditDate="2017-04-08T21:05:22.487" LastActivityDate="2017-04-08T21:05:22.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26582" PostTypeId="1" CreationDate="2017-04-08T08:49:20.183" Score="1" ViewCount="186" Body="&lt;p&gt;I am studying optimization methods and I was able to understand and derive the search direction&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ p_k = r_{k-1} + \beta p_{k-1} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for Conjugate Gradient Method, with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \beta = -\frac{p_{k-1}^TAr_{k-1}}{p_{k-1}^TAp_{k-1}}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this expression $p_k$ is the search direction and $r_k$ the residual. Nevertheless, in a paper [1] I found the following expression ($d$ is the direction, $g$ is the gradient and $y_{k-1} = (g_k-g_{k-1}))$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ d_1 = -Hg_1,\\ d_k = -Hg_k + \frac{y_{k-1}^THg_k}{y_{k-1}^Td_{k-1}}d_{k-1} .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not able to understand if this is the same as $p_k$. In the paper the author writes that the basic CG is with $H$ being the identity matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is $\frac{y_{k-1}^THg_k}{y_{k-1}^Td_{k-1}} = \beta$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;strong&gt;A Relationship between the BFGS and Conjugate Gradient Algorithms and Its Implications for New Algorithms&lt;/strong&gt;, L. Nazareth.&lt;/p&gt;&#xA;" OwnerUserId="22185" LastEditorUserId="22185" LastEditDate="2017-04-08T09:15:11.990" LastActivityDate="2017-04-08T09:15:11.990" Title="Search direction for CG method" Tags="|optimization|conjugate-gradient|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26583" PostTypeId="2" ParentId="26435" CreationDate="2017-04-08T10:02:03.197" Score="0" Body="&lt;p&gt;Since you are asking quite a general question, I will respond with a general answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In describing incompressible flows we use velocity inlet boundary condition (e.g. uniform, parabolic, etc) and a constant pressure outlet boundary condition (e.g. atmospheric pressure).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the membrane side specify a mass flux boundary condition which depends on the gradient in $C$ across the membrane:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-D\nabla C = k\left(C-C_0\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $D$ and $k$ are the diffusion coefficient and mass transfer coefficient respectively. Usually, these are related through so-called Sherwood relations where the Sherwood number $Sh=kL/D$ and $L$ is a characteristic length scale (e.g. membrane thickness). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These Sherwood relations are a function of the Reynolds number $Re=UL/\nu$ where $\nu$ is the kinematic viscosity and $U$ is the average fluid speed. In this way the mass transfer across the membrane is coupled to the fluid flow.&lt;/p&gt;&#xA;" OwnerUserId="18017" LastEditorUserId="18017" LastEditDate="2017-04-08T10:10:20.937" LastActivityDate="2017-04-08T10:10:20.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26588" PostTypeId="1" CreationDate="2017-04-09T11:43:36.837" Score="2" ViewCount="1714" Body="&lt;p&gt;I am trying to solve the system of linear equations: $AX=B$. For this currently I am using Intel MKL Pardiso solver. It works well when the order of $A$ is around $13500\times13500$ and below. Above that the solver gives wrong answer (perhaps I am giving wrong values of parameters required to be set before calling solver). Dimensions of $A$ and $B$ are needed to be as high as ($15\cdot 10^6\times15\cdot 10^6$) and ($15\cdot 10^6 \times 6$), respectively. My program is in Fortran fixed form and I am using Intel Visual Fortran Compiler. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reason to use Sparse linear solver: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) The $A$ matrix is primarily sparse. I have attached morphology of my $A$ matrix for visualization, where yellow pixels represents non-zero values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) I was previously using &lt;code&gt;DGELSD&lt;/code&gt; LAPACK subroutine, but it is taking so much time. Sparse solvers are about 180 times faster than non-sparse solvers. Also, memory consumption with Pardiso solver is very less as it works on CSR format of the matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how to set parameters for Pardiso solver, or is there any other solver with less presetting requirements that I can use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently I working on Windows 10. System specification:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intel Xeon E series processor 32 cores, 64 GB RAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Image information: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A$ matrix morphology, with a total of $135\times135$ elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yellow pixels: 666 in number. Matrix dimensions are $135\times135$ (easiest case). &#xA;NOTE THAT MATRIX STRUCTURE REMAINS EXACTLY IN THE SAME FASHION WITH INCREASE IN ITS DIMENSIONS, also the A matrix is non symmetric.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TmMs1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TmMs1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="23886" LastEditorUserId="20688" LastEditDate="2017-04-09T15:06:05.147" LastActivityDate="2017-04-10T13:31:29.063" Title="Which is the best subroutine available for solving sparse linear system of equations" Tags="|linear-solver|sparse-matrix|intel-mkl|" AnswerCount="1" CommentCount="4" ClosedDate="2017-04-10T15:44:15.193" ContentLicense="CC BY-SA 3.0" />
  <row Id="26589" PostTypeId="2" ParentId="26557" CreationDate="2017-04-09T18:17:16.700" Score="6" Body="&lt;p&gt;You won't like the answer, but it is in fact &lt;em&gt;&quot;This lies in the right functional-analytic framework&quot;&lt;/em&gt; — which you have not given! In particular, you did not specify in which function space you are looking for &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; in. If &lt;span class=&quot;math-container&quot;&gt;$u\in L^2$&lt;/span&gt; as the formulation suggests, then asking for the value of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; at a single point makes no sense a priori. (Even if the ODE is based on classical derivatives, the optimal control will be &lt;span class=&quot;math-container&quot;&gt;$\bar u = \alpha^{-1}\bar \lambda$&lt;/span&gt; &lt;em&gt;in the sense of &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt;&lt;/em&gt;. Hence even though the latter is indeed continuous and will have zero value at &lt;span class=&quot;math-container&quot;&gt;$t=T$&lt;/span&gt;, if you change &lt;span class=&quot;math-container&quot;&gt;$\bar u$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$t=T$&lt;/span&gt; arbitrarily, you will still get the same &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; function (or rather, equivalence class of functions), so asking for the value of &lt;span class=&quot;math-container&quot;&gt;$\bar u(T)$&lt;/span&gt; is not meaningful.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you do a naive discretization of &lt;span class=&quot;math-container&quot;&gt;$u,v,\lambda$&lt;/span&gt; by finite differences, identifying their function with their values at the nodes, you get exactly the behavior that you describe. If you do not want that, the usual procedure is to discretize a control in &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; as piecewise constant on each control interval, in which case the optimal control &lt;span class=&quot;math-container&quot;&gt;$\bar u$&lt;/span&gt; will not be &lt;span class=&quot;math-container&quot;&gt;$\alpha^{-1}\bar \lambda$&lt;/span&gt; but its &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; projection — i.e., the average over each interval, which can be nonzero on the last interval even for &lt;span class=&quot;math-container&quot;&gt;$\bar \lambda(T)=0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also look for &lt;span class=&quot;math-container&quot;&gt;$u\in H^1$&lt;/span&gt; (not &lt;span class=&quot;math-container&quot;&gt;$H^1_0$&lt;/span&gt;!) — adding an appropriate penalty to make the problem well-posed. In this case, &lt;span class=&quot;math-container&quot;&gt;$\bar u = (\alpha I+\beta \Delta)^{-1} \bar \lambda$&lt;/span&gt; (i.e., the solution to &lt;span class=&quot;math-container&quot;&gt;$\beta\Delta u + \alpha u =\bar \lambda$&lt;/span&gt;  subject to homogeneous Neumann boundary conditions) will be continuous due to the Sobolev embedding and have a well-defined — not necessarily zero — value at &lt;span class=&quot;math-container&quot;&gt;$t=T$&lt;/span&gt; even if &lt;span class=&quot;math-container&quot;&gt;$\bar \lambda(T)=0$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="9667" LastEditDate="2019-05-14T12:22:57.323" LastActivityDate="2019-05-14T12:22:57.323" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="26590" PostTypeId="1" CreationDate="2017-04-09T21:51:37.347" Score="4" ViewCount="92" Body="&lt;p&gt;I wish to compute a certain function &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; (in my case an eigenvalue of the Laplacian) to a certain accuracy, which I wish to guarantee, if possible. I started with a finite element method. I know that in 2D there are way better methods, but my goal is to work on general surfaces, so that's why I work with finite elements for now. Let &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; be the size of the elements.&lt;/p&gt;&#xA;&lt;p&gt;I start from some &lt;span class=&quot;math-container&quot;&gt;$h_0$&lt;/span&gt; and keep refining using the midpoint refinement procedure (replace each triangle with four triangle constructed using midpoints). This procedure gives me an approximation of &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; depending on &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\lambda_h,\lambda_{h/2},...,\lambda_{h/2^n}...$&lt;/span&gt; Due to the size of the computations involved I can only do &lt;span class=&quot;math-container&quot;&gt;$9$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; refinements which seem to get me &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; digits of precision.&lt;/p&gt;&#xA;&lt;p&gt;After reading a few chapters from the extremely useful book &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898717969&quot; rel=&quot;noreferrer&quot;&gt;SIAM 100 digit challenge&lt;/a&gt; I decided to try applying a basic extrapolation technique to the sequence of approximations &lt;span class=&quot;math-container&quot;&gt;$\lambda_h,\lambda_{2h},...,\lambda_{h/2^n}$&lt;/span&gt;. To my surprise the extrapolation seems to double the number of correct digits.&lt;/p&gt;&#xA;&lt;p&gt;I searched a bit and did not found references justifying such a behavior. So my question is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Do you know any papers dealing with this kind of approximations:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;compute some finite element approximations for successive mesh refinements&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;apply an extrapolation procedure&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;guarantee the extra precision of the extrapolated result ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-04-09T21:51:37.347" Title="Extrapolation after successive finite element refinement" Tags="|finite-element|approximation|extrapolation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26592" PostTypeId="1" AcceptedAnswerId="26615" CreationDate="2017-04-10T08:43:43.817" Score="2" ViewCount="975" Body="&lt;p&gt;Is it a good criterion to stop iterative methods for solving non-linear equations, such as Newton-Raphson and good Broyden's methods, when $|x_k-x_{k-1}|&amp;lt;|x_k|\,reltol + abstol$ &lt;strong&gt;OR&lt;/strong&gt; when $|f_k|&amp;lt;tolfun$, or should it be when $|x_k-x_{k-1}|&amp;lt;|x_k|\,reltol + abstol$ &lt;strong&gt;AND&lt;/strong&gt; when $|f_k|&amp;lt;tolfun$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Until now, I was using the first criterion, but recently I had a problem using the Broyden's method in a continuation algorithm: when trying to obtain the next point of the curve, $\{x_k\}$ converged to a value $x^*$ although the residual was already too big. Is this because the stopping criterion I was using was not good enough, or is it just because there might be a bifurcation at that point?&lt;/p&gt;&#xA;" OwnerUserId="23633" LastEditorUserId="9667" LastEditDate="2017-04-10T22:39:06.087" LastActivityDate="2017-04-18T17:09:29.647" Title="Stopping criteria in iterative methods for solving nonlinear equations" Tags="|nonlinear-equations|iterative-method|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26593" PostTypeId="1" AcceptedAnswerId="26784" CreationDate="2017-04-10T10:53:26.940" Score="0" ViewCount="156" Body="&lt;p&gt;I implemented an algorithm for solving the Hankel transformation based on a paper. My problem is: It works very good for the functions suggested in the paper (as test functions), it works pretty ok for certain other test functions, but for other functions with known result it just produces garbage (result is far away from the expected result).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does that mean for my implementation?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My implementation is garbage, and I have to redo it.  &lt;/li&gt;&#xA;&lt;li&gt;I choose a wrong data input for the function, and I have to change the parameters for it  &lt;/li&gt;&#xA;&lt;li&gt;The algorithm is garbage, and I have to choose another one  &lt;/li&gt;&#xA;&lt;li&gt;My programming is bad, and I should check that  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What would be the best, i.e. most successfull approach for that situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something to note: The algorithm provides a transformation method ($g=HT(f)$) and a backtransformation ($f2=IHT(g)$) method. When comparing $f$ and $f2$, they are equal. But $g$ and the expected function from theory are not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the following paper as source: &lt;a href=&quot;https://www.osapublishing.org/josaa/abstract.cfm?uri=josaa-21-1-53&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.osapublishing.org/josaa/abstract.cfm?uri=josaa-21-1-53&lt;/a&gt; (unfortunately behind paywall), but calculate the Hankel transformation in the zeroth order. As languages I implemented the method both in python and in C/C++. The examples from the paper work, also such examples as $f=1/r, HT(f)=1/k$, but for example $f=r, HT(f)=1/(k^3)$ does not (produces oscillations).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is one of the implementations of it (same limitations): &lt;a href=&quot;https://se.mathworks.com/matlabcentral/fileexchange/15623-hankel-transform?requestedDomain=www.mathworks.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://se.mathworks.com/matlabcentral/fileexchange/15623-hankel-transform?requestedDomain=www.mathworks.com&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20545" LastEditorUserId="20545" LastEditDate="2017-04-11T05:35:22.230" LastActivityDate="2017-07-03T13:07:55.863" Title="Hankel transform from paper works only for certain functions" Tags="|algorithms|fourier-transform|signal-processing|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="26594" PostTypeId="1" AcceptedAnswerId="26596" CreationDate="2017-04-10T13:22:29.810" Score="0" ViewCount="192" Body="&lt;p&gt;In Hesthaven book (Nodal Discontinuous Galerkin Methods) he uses SSP Runge-Kutta time method which is explicit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I change the explicit RK to an implicit one?&lt;/p&gt;&#xA;" OwnerUserId="23896" LastEditorUserId="23896" LastEditDate="2017-08-18T12:16:17.010" LastActivityDate="2017-08-18T12:16:17.010" Title="From explicit to implicit SSP Runge-Kutta time discretization for DG" Tags="|implicit-methods|explicit-methods|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26595" PostTypeId="2" ParentId="26588" CreationDate="2017-04-10T13:31:29.063" Score="2" Body="&lt;p&gt;A matrix of size 15M x 15M is likely too big for a (sparse) direct solver on a single machine -- it is going to take too much time and memory. If you wanted to use a direct solver, you could try parallel sparse direct solvers such as MUMPS or SuperLU-dist, both of which are conveniently called via PETSc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also the option of using iterative solvers. This is particularly true if you have knowledge where these linear systems come from, and can come up with a good preconditioner. My thoughts on how to choose solvers and preconditioners can -- in significant detail -- be found in lectures 34-38 here: &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-04-10T13:31:29.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26596" PostTypeId="2" ParentId="26594" CreationDate="2017-04-10T14:43:46.363" Score="2" Body="&lt;p&gt;You cannot merely adjust an explicit RK scheme into an implicit one, implicit routines are much more involved because each intermediate slope can depend upon slopes 'in the future'. This introduces a (possibly nonlinear) system of equations that needs to be solved within each RK step. That is nowhere to be found in the explicit method you posted above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If by 'adjustment' to this routine, you are willing to consider adding a solver for the resulting system of equations required by implicit RK methods, then I would say yes it's possible to adjust your routine. That being said, implicit methods are much more involved than explicit methods for this very reason, and I would caution you that it is not a small adjustment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good starting point would be looking at how stiff ODEs are solved. I would advise you to implement your own version of a simple implicit method to solve a test problem, and then see for yourself how the explicit method implementated by Hesthaven can be extended to be solved implicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I found the textbooks by Hairer to be very helpful when I was learning about explicit and implicit RK methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hairer, Solving Ordinary Differential Equations I and II&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first goes over explicit methods and the second goes over implicit ones - both contain examples to compare your own codes to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, if you are want to see how I implemented a few explicit and implicit RK methods, take a look at my github project written in Fortran: github.com/cbcoutinho/generic_rk&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="19791" LastEditDate="2017-04-10T14:49:16.340" LastActivityDate="2017-04-10T14:49:16.340" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26597" PostTypeId="1" CreationDate="2017-04-10T15:56:08.227" Score="3" ViewCount="378" Body="&lt;p&gt;I have to solve many times a nonlinear system of the form&#xA;$$f(x) = b^{(n)}$$&#xA;inside a loop.&#xA;The function $f$ is expensive to compute and I do not have its jacobian, so I have tried the good Broyden's method. As the initial guesses of the solution, $x_0^{(n+1)}$, and the jacobian, $J_0^{(n+1)}$, at the (n+1)th iteration of the loop, I am using the solution of the previous one, $x^{(n)}$, and the approximation to the jacobian at $x^{(n)}$ provided by Broyden's method, $B^{(n)}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have observed that, along the simulation, the Broyden's method needs more and more iterations to find the solutions. I guess this happens because the approximation $B^{(n)}$ to the jacobian that the method uses becomes poorer and poorer (sometimes it even becomes singular). Is there any way to avoid this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PD: In this link &lt;a href=&quot;http://www.math.hkbu.edu.hk/~zeng/Teaching/math3620/Broyden.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.hkbu.edu.hk/~zeng/Teaching/math3620/Broyden.pdf&lt;/a&gt; it is said that Powell (&quot;A Hybrid Method for Nonlinear Equations&quot;) presented a modification to Broyden's jacobian update so that $B$ converges to the jacobian as $x$ converges to the solution. I cannot find any paper in which this is explained. Does anyone know where to find this modification explained?&lt;/p&gt;&#xA;" OwnerUserId="23633" LastEditorUserId="9667" LastEditDate="2017-04-10T22:38:36.453" LastActivityDate="2017-04-10T22:38:36.453" Title="How to avoid the Broyden's jacobian approximation becoming poorer with the number of iterations?" Tags="|nonlinear-equations|iterative-method|quasi-newton|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26598" PostTypeId="1" CreationDate="2017-04-10T17:06:19.077" Score="6" ViewCount="2112" Body="&lt;p&gt;I have to use an iterative method (Newton-Raphson, modified Newton and Broyden) to solve a system of nonlinear equations $f(x)=0$. Every unknown $x_i$ is bounded between $l_i$ and $u_i$, i.e., $l_i&amp;lt;x_i&amp;lt;u_i$, and evaluating $f$ if any of the unknowns goes out of its bounds would give an error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the best technique to avoid this during the iteration? Currently, I am using a damping parameter $t\leq 1$ so that, in Newton-Raphson, for example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x^{(n+1)} = x^{(n)} - t \, J^{(n)}\backslash f^{(n)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is always inside the corresponding hypercube. However, this is an &lt;em&gt;ad hoc&lt;/em&gt;, nonrigorous solution, and I have found papers about reflective techniques (although they were only applied for optimization problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the above solution a good one or should I try something different?&lt;/p&gt;&#xA;" OwnerUserId="23633" LastEditorUserId="9667" LastEditDate="2017-04-10T22:36:16.380" LastActivityDate="2017-07-29T15:42:28.003" Title="Newton's method with box-constraints" Tags="|iterative-method|newton-method|constraints|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26600" PostTypeId="1" CreationDate="2017-04-10T18:55:59.087" Score="3" ViewCount="900" Body="&lt;p&gt;I am trying to solve the Lamm eqn numerically, using 1-D FEM. What I observe is spurious oscillations at one boundary ($r_b$), which quickly leads to unreasonable values over the entire range. I am looking for advice on how to modify the FE formulation to eliminate these oscillations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Lamm eqn is&#xA;$${\partial C \over \partial t} +  {1\over r}{\partial (r J) \over \partial r}=Q, \ \ r_a\leq r\leq r_b,$$&#xA;and&#xA;$$J=s \omega^2 r C - D {\partial C \over \partial r}$$&#xA;where $s$ is the sedimentation coefficient, $D$ is the diffusion coefficient, and $\omega$ the angular velocity. The boundary&#xA;conditions are that the fluxes $J(r_a,t) = J(r_b, t)= 0$. For simple cases, $s$ and $D$ are constants, but they may also be functions of $C$. $Q$ is a &quot;source function&quot;, but it is 0 for our case. The initial condition is typically $C$=constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current FEM formulation uses piecewise linear (&quot;hat&quot;) functions for discretization in space, and an implicit method to compute $C(t+\Delta t)$ from $C(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Typical values are $r_a$=5.9, $r_b$=7.2, $C_0$=1, $s=4\times 10^{-13}$, $\omega^2=2.7\times 10^7$, $D=3.6\times 10^{-7}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At long times, the solution is proportional to $\exp(r^2)$.&lt;/p&gt;&#xA;" OwnerUserId="21808" LastEditorUserId="21808" LastEditDate="2017-04-11T02:43:14.480" LastActivityDate="2017-04-11T16:16:39.260" Title="How to avoid spurious oscillations 1D FEM Neumann BC" Tags="|finite-element|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26601" PostTypeId="2" ParentId="24005" CreationDate="2017-04-11T03:54:21.183" Score="0" Body="&lt;p&gt;When you choose the test function equal to the trial function in the DG method you are creating an optimization problem. That is, you have a Galerkin rather than a Petrov-Galerkin method. You are seeking the time derivatives of the trial function amplitudes that will minimise the element residual in the L2 norm, and you make this miminimisation on the assumption of a given flux function at inflow.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-04-11T03:54:21.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26604" PostTypeId="2" ParentId="26600" CreationDate="2017-04-11T16:16:39.260" Score="3" Body="&lt;p&gt;The Lamm equation is a specific example of a convection-diffusion PDE. You can find many references on the difficulties&#xA;of numerical solution of this equation (e.g. &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0898716292&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leveque&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oscillations in the solution near the boundaries, due to characteristics of the numerical method are a common problem. A key parameter in&#xA;understanding the likelyhood of oscillations is the mesh Peclet number given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;P = \frac{s\omega^2r\Delta x}{2D}&#xA;$$&#xA;where $\Delta x$ is the mesh spacing. Oscillations almost always occur when this number is $&amp;gt;1$. For the Lamm equation and the&#xA;constants you provided, for a reasonable mesh, this number is $&amp;lt;1$. However, due to the boundary conditions, particularly at the right&#xA;end, there is a very steep gradient in the solution. This, along with even the moderate Peclet number is responsible for the oscillations there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The obvious solution is to refine the mesh to reduce the Peclet number. This can be done non-uniformly so that, for example, the mesh&#xA;is much more refined at the right end. Often, this produces a model that too computationally-costly to solve, but that does not appear to be the case here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another alternative is to artificially increase the value of $D$. One of the comments suggested to instead use a forward-difference (upwind)&#xA;finite difference method. This has the unfortunate side-effect of significantly increasing $D$ so that the solution is very &quot;smeared&quot; compared&#xA;with exact. In the finite element method, you can achieve the same effect by slightly increasing $D$ directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By doing a simple web search, I was able to find a number of papers related to numerical solution of the Lamm equation for the particular&#xA;application you are interested in. Many of them describe the difficulties of using basic FE methods for this problem and propose more&#xA;sophisticated alternatives, such a moving-grid methods, to overcome these problems. I think this one, by &lt;a href=&quot;http://www.math.colostate.edu/~liu/PUB/2009_NMPDE_LiuChen.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Liu and Chen&lt;/a&gt;, does a particularly nice job in this regard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, using the parameters you provided, I computed the following solution to your equation using $500$ two-node finite elements and&#xA;an implicit time integration algorithm. As you can see, for these parameters and this level of mesh refinement, there is no noticeable&#xA;oscillation at the right end.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1louA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1louA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-04-11T16:16:39.260" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="26605" PostTypeId="2" ParentId="19742" CreationDate="2017-04-11T18:53:18.333" Score="9" Body="&lt;p&gt;Let's denote by $\otimes,\oplus,\ominus$ (I was lazy trying to get circled version of division operator) the floating-point analogs of exact multiplication ($\times$), addition ($+$), and subtraction ($-$), respectively. We'll assume (IEEE-754) that for all of them &#xA;$$&#xA;[x\oplus y]=(x+ y)(1+\delta_\oplus),\quad |\delta_\oplus|\le\epsilon_\mathrm{mach},&#xA;$$&#xA;where $\epsilon_\mathrm{mach}$ is the machine epsilon giving an upper bound on the relative error due to rounding off.&#xA;We will also use the following lemma (assuming all $|\delta_i|\le\epsilon_\mathrm{mach}$, and $m$ is not too large) that can be easily proven:&#xA;$$&#xA;\prod\limits_{i=1}^{m}(1+\delta_i)=1+\theta(m),\quad |\theta(m)|\le\frac{m\epsilon_\mathrm{mach}}{1-m\epsilon_\mathrm{mach}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's define the true function $f$ that operates on real numbers $x,y,z$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(x,y,z)=(x\times z)-(y\times z)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and two versions of the function implementation in IEEE-compliant floating-point arithmetic as $\tilde{f_1}$ and $\tilde{f_2}$ that operate on floating-point representations $\tilde{x}=x(1+\delta_x),\tilde{y},\tilde{z}$, as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde{f_1}(\tilde{x},\tilde{y},\tilde{z})=(\tilde{x}\otimes\tilde{z})\ominus(\tilde{y}\otimes\tilde{z}),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde{f_2}(\tilde{x},\tilde{y},\tilde{z})=(\tilde{x}\ominus\tilde{y})\otimes\tilde{z}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error analysis for $\tilde{f_1}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;\tilde{f_1}&amp;amp;=\Big(\underbrace{\big(x(1+\delta_x)\times z(1+\delta_z)\big)\big(1+\delta_{\otimes_{xz}}\big)}_{(\tilde{x}\otimes\tilde{z})}-\underbrace{\big(y(1+\delta_y)\times z(1+\delta_z)\big)\big(1+\delta_{\otimes_{yz}}\big)}_{(\tilde{y}\otimes\tilde{z})}\Big)\Big(1+\delta_{\ominus}\Big)\\&#xA;&amp;amp;=xz(1+\delta_x)(1+\delta_z)(1+\delta_{\otimes_{xz}})(1+\delta_\ominus)-yz(1+\delta_y)(1+\delta_z)(1+\delta_{\otimes_{yz}})(1+\delta_\ominus)\\&#xA;&amp;amp;=xz(1+\theta_{xz,1})-yz(1+\theta_{yz,1}).&#xA;\end{aligned}&#xA;$$&#xA;Here, $|\theta_{xz,1}|,|\theta_{yz,1}|\le\frac{4\epsilon_\mathrm{mach}}{1-4\epsilon_\mathrm{mach}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, for $\tilde{f_2}$&#xA;$$&#xA;\begin{aligned}&#xA;\tilde{f_2}&amp;amp;=\Bigg(\Big(\big( x(1+\delta_x)-y(1+\delta_y \big)\big(1+\delta_{\ominus_{xy}}\big)\Big)\times \Big(z(1+\delta_z)\Big)\Bigg)\Bigg(1+\delta_{\otimes}\Bigg)\\&#xA;&amp;amp;=xz(1+\delta_x)(1+\delta_z)(1+\delta_{\ominus_{xy}})(1+\delta_\otimes)-yz(1+\delta_y)(1+\delta_z)(1+\delta_{\ominus_{xy}})(1+\delta_\otimes)\\&#xA;&amp;amp;=xz(1+\theta_{x,2})-yz(1+\theta_{y,2}).&#xA;\end{aligned}&#xA;$$&#xA;Here, $|\theta_{x,2}|,|\theta_{y,2}|\le\frac{4\epsilon_\mathrm{mach}}{1-4\epsilon_\mathrm{mach}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for both $\tilde{f_1}$ and $\tilde{f_2}$ we got expressions of the same type, thus I do not see why one implementation would be preferred to another from a numerical point of view (except the fact that $\tilde{f_2}$ performs only 2 floating-point operations compared to $\tilde{f_1}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing the relative error will show, that the problem comes from the fact that  $x$ and $y$ can be very close (&lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_of_significance&quot; rel=&quot;noreferrer&quot;&gt;cancellation&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;\frac{|\tilde{f_1}-f|}{|f|}&amp;amp;=\frac{|xz+xz\theta_{xz,1}-yz-yz\theta_{yz,1}-(xz-yz)|}{|xz-yz|}=\frac{|x\theta_{xz,1}-y\theta_{yz,1}|}{|x-y|}\\&#xA;&amp;amp;\le\frac{|x|+|y|}{|x-y|}\frac{4\epsilon_\mathrm{mach}}{1-4\epsilon_\mathrm{mach}},&#xA;\end{aligned}&#xA;$$&#xA;$$&#xA;\begin{aligned}&#xA;\frac{|\tilde{f_2}-f|}{|f|}&amp;amp;=\frac{|xz+xz\theta_{x,2}-yz-yz\theta_{y,2}-(xz-yz)|}{|xz-yz|}=\frac{|x\theta_{x,2}-y\theta_{y,2}|}{|x-y|}\\&#xA;&amp;amp;\le\frac{|x|+|y|}{|x-y|}\frac{4\epsilon_\mathrm{mach}}{1-4\epsilon_\mathrm{mach}}.&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Slight differences between $\theta$'s might make one of the two numerical implementations marginally better or worse depending on $x,y,z$. However, I doubt it can be of any significance.&#xA;The result totally makes sense, because no matter what, if you have to compute $(x-y)$, when $x$ and $y$ are close enough in values (for the precision you work with) using floating-point arithmetic, no scaling will significantly help you: you are already in trouble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: All discussion above assumes no overflow or underflow, i.e. $x,y,z,f(x,y,z)\in\mathbb F_0$, $\mathbb F_0$ being the set of all normal floating-point numbers.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-04-11T18:53:18.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26609" PostTypeId="2" ParentId="26520" CreationDate="2017-04-12T13:39:30.537" Score="0" Body="&lt;p&gt;I finally solved my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still use the GSL but I changed the Broyden's method for the classical discrete Newton's method because I saw that Broyden's method &quot;is not recommended for serious use&quot; (source: &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Algorithms-without-Derivatives.html#Algorithms-without-Derivatives&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gnu.org/software/gsl/manual/html_node/Algorithms-without-Derivatives.html#Algorithms-without-Derivatives&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also programmed &quot;mirrors&quot; to avoid nonphysical values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the assistance.&lt;/p&gt;&#xA;" OwnerUserId="23799" LastActivityDate="2017-04-12T13:39:30.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26610" PostTypeId="2" ParentId="26598" CreationDate="2017-04-12T13:48:44.317" Score="1" Body="&lt;p&gt;You should put &quot;mirrors&quot; to bracket your domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in a one-dimensional case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I know that my unknown $x$ is bounded between $0$ and $a$. At each iteration of my method, i will check if $0 &amp;lt; x &amp;lt; a$. If not, while $x &amp;lt; 0$ or $x &amp;gt; a$, if $x &amp;lt; 0$ I will change $x$ to $-x$, else if $x &amp;gt; a$ I will change $x$ to $a-(x-a)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another method that might be more complex to program depending what language you use but that is often more successful is to modify your Jacobian by dividing the step by 2 while you are not in your domain of interest.&lt;/p&gt;&#xA;" OwnerUserId="23799" LastEditorUserId="23633" LastEditDate="2017-07-29T15:42:28.003" LastActivityDate="2017-07-29T15:42:28.003" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26611" PostTypeId="1" AcceptedAnswerId="26616" CreationDate="2017-04-12T14:32:40.610" Score="0" ViewCount="209" Body="&lt;p&gt;I have to work with values such as 1e-15 in my code but I can't. Indeed, because of low precision these values are equivalent to 0. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="23799" LastActivityDate="2017-04-13T01:06:28.293" Title="How to deal with very low numerical values in C?" Tags="|c|precision|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26613" PostTypeId="1" CreationDate="2017-04-12T16:42:34.443" Score="1" ViewCount="184" Body="&lt;p&gt;I'm working on a 2-D planar finite difference code. My differencing scheme at the boundary nodes involves introducing ghost nodes in the computation. &#xA;My code also involves a multi-dimensional derivative:&#xA;$$&#xA;\frac{d^2u}{dxdy} \approx \frac{u_{i+1,j+1} - u_{i+1,j-1} - u_{i-1, j+1} + u_{i-1,j-1}}{4\Delta x\Delta y}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This scheme becomes an issue at a corner node. Let me illustrate:&#xA;$$&#xA;g1\  \ g2 \ \ g3 \\&#xA;g4\ \  d1 \ \  d2 \\&#xA;g5\ \  d3 \ \ d4 \\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the g's represent the ghost codes and the d's represent the real domain nodes that I'm solving for. x-direction is horizontal; y-direction is vertical. Dirichlet boundary conditions are specified for both the top and left boundaries. Let's say $u=u_1$ at the top boundary and $u=u_2$ at the left boundary. What should be the boundary condition at ghost node g1, which lies along both the x and y boundaries?&lt;/p&gt;&#xA;" OwnerUserId="23915" LastActivityDate="2017-04-12T16:42:34.443" Title="boundary condition in 2-D planar finite difference problem" Tags="|finite-difference|boundary-conditions|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26615" PostTypeId="2" ParentId="26592" CreationDate="2017-04-12T19:12:56.760" Score="5" Body="&lt;p&gt;It can happen in principle that the condition on $f$ will never be satisfied, if, for example, $|f'(x_*)|u&amp;gt;\mathit{tolfun}$, where $u$ is the unit roundoff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I tried your example with $K=10^8,y_0=0.1$, and the root $x_0=1.1051709180756475$ (in double precision) has the property that $f(x_0)≈−1.25×10^{−8}$, but $f(x_0+\epsilon)≈6.94×10^{−9}$, where $x_0+\epsilon$ is the next double-precision floating-point number immediately after $x_0$. So there are no such numbers for which $|f|&amp;lt;10^{-9}$. I think it's also possible to come up with easy polynomial examples by, e.g., taking characteristic polynomials of random integer matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the relationship between the residual of the equation $f(x)=0$ and the error $x-x_*$ depends on the function itself. You know that asymptotically both $f(x)$ and $x-x_*$ will converge to zero, but to derive a bound of the true error $x-x_*$ from the residual $f(x)$ requires making assumptions on $f$. If you know the value of $|f'(x_*)|$, then you know that $|x-x_*|\approx |f(x)|/|f'(x_*)|$. In the multivariate case, the inequality instead says that &#xA;$$\|f(x)\|\|J\|^{-1} \lesssim \|x-x_*\| \lesssim \|f(x)\|\|J^{-1}\|,$$&#xA;so you need at least to know the condition of the Jacobian to convert the residual to an error estimate (see Higham's &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;, 25.5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of the above is &lt;em&gt;before&lt;/em&gt; you start analyzing the numerical stability of evaluating $f$ near a root. You gave an example of essentially a linear function, but for a typical more complicated function, you wouldn't be able to evaluate $f$ accurately to full machine precision near a root at all, which will also limit how small $f$ can possibly get.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;" OwnerUserId="713" LastEditorUserId="23633" LastEditDate="2017-04-18T17:09:29.647" LastActivityDate="2017-04-18T17:09:29.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26616" PostTypeId="2" ParentId="26611" CreationDate="2017-04-13T01:06:28.293" Score="3" Body="&lt;p&gt;The best thing to do would be to re-scale your problem so that way you don't have such small values. If you can, do this and call it a day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If every number is small, then you can still be fine because of the subnormal floating point numbers. People mentioned this in the comments, and its addressed here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/8341395/what-is-a-subnormal-floating-point-number&quot;&gt;https://stackoverflow.com/questions/8341395/what-is-a-subnormal-floating-point-number&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, subnormal numbers have slower calculations, so it's normal to try and avoid them. Also, because they are less than machine epsilon, they do not behave well (the easiest example is that if you add 1 to any of them, you get 1 back). The will multiply and add in ways that round the gaps, and you pay and extra cost for calculating with them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So rescale your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or use quads, BigFloats, Arbs, 128-bit decimal floating point numbers. If you're using Julia, BigFloats are in Base, Arbs are in ArbFloats.jl, and decimals are in DecFP.jl. Due to type-genericity, you can generally replace any algorithm with these number types and it should just work (even in code from packages, if the author didn't overtype their functions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But first, try rescaling your problem.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="-1" LastEditDate="2017-05-23T12:35:01.797" LastActivityDate="2017-04-13T01:06:28.293" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26617" PostTypeId="1" AcceptedAnswerId="26618" CreationDate="2017-04-13T04:11:15.673" Score="0" ViewCount="451" Body="&lt;p&gt;So our team has been able to code up a BiCGStab implementation for a class project, and we'd like a potential example problem to try it out on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, we've talked about a 1D Laplacian with Neumann boundaries (so the matrix becomes nonsymmetric), but I am strongly inclined to demonstrate at least a very slightly more exotic problem that can be shown using the BiCGStab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the time and skill constraints, a steady-state problem description would be idea, since no one in the team has a decent idea about time propagation, and our implementation of the BiCGStab is fairly naive (slow).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions for a demo problem for our solver would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17947" LastActivityDate="2017-04-13T10:37:31.537" Title="Example Problem to Demonstrate BiCGStab" Tags="|numerical-analysis|iterative-method|nonlinear-programming|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26618" PostTypeId="2" ParentId="26617" CreationDate="2017-04-13T10:37:31.537" Score="4" Body="&lt;p&gt;Check out &lt;a href=&quot;http://math.nist.gov/MatrixMarket/&quot; rel=&quot;noreferrer&quot;&gt;Matrix Market&lt;/a&gt;. It has a nice collection of matrices from different areas (fluid dynamics, elasticity, acoustics etc.)&#xA;So you do not need to assemble systems yourself in order to provide decent test cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrices are provided in ASCII–format (Matrix Market and Harwell–Boeing). I do not know what language your team uses, but you can easily find I/O routines for C/C++, Matlab, Python, and FORTRAN on the website. Mathematica has these routines built in (&lt;a href=&quot;https://reference.wolfram.com/language/ref/format/MTX.html&quot; rel=&quot;noreferrer&quot;&gt;MTX&lt;/a&gt;, &lt;a href=&quot;https://reference.wolfram.com/language/ref/format/HarwellBoeing.html&quot; rel=&quot;noreferrer&quot;&gt;HB&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-04-13T10:37:31.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26619" PostTypeId="1" CreationDate="2017-04-13T12:05:14.317" Score="1" ViewCount="254" Body="&lt;p&gt;I have a binary full-rank matrix of size, say, $25 \times 50$. I need to count how many subsets of its columns form matrices with a full column rank, i.e. all the columns in the subset are linearly independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Straightforward approach would be to iterate over all subsets of columns of size up to $25$, and then check corresponding submatrix if it has full column rank. This way one needs to test&#xA;$$&#xA;\binom{50}{1} + \binom{50}{2} + \dotsb + \binom{50}{25} = 626\,155\,256\,640\,187 \approx 6.3 \times 10^{14}&#xA;$$&#xA;matrices. Hence one needs a really fast algorithm to test if a particular submatrix has a full column rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, assume I have 500 cores and I want to calculate the subject in 24h. Then I need to test $1.4 \times 10^7$ submatrices per second on one core. Old good Gaussian elimination fails with this task. Can I do something much faster than it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach might be some optimised method like branch and bounds, so that one does not need to check all the submatrices - but only a small portion of them. However, I don't see at the moment what can be done in this direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. All operations are over Galois field $\mathbb F_2$.&lt;/p&gt;&#xA;" OwnerUserId="21609" LastEditorUserId="21609" LastEditDate="2017-04-15T10:38:27.950" LastActivityDate="2017-04-15T13:26:24.817" Title="Fast counting of all submatrices of a binary matrix with a full column rank" Tags="|linear-algebra|matrix|linear-solver|rank|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26620" PostTypeId="1" AcceptedAnswerId="26625" CreationDate="2017-04-13T10:22:15.747" Score="3" ViewCount="4261" Body="&lt;p&gt;I'm trying to solve the ODE of motion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;    x''=&amp;amp;\ -myu \frac{x}{r^3} \left(1+\frac{3}{2} {J_2} \left(\frac{{r_{\text{eq}}}}{r}\right)^2 \left(1-\frac{5z^2}{r^2}\right)\right) \\&#xA;    y''=&amp;amp;\ -myu \frac{y}{r^3} \left(1+\frac{3}{2} {J_2} \left(\frac{{r_{\text{eq}}}}{r}\right)^2 \left(1-\frac{5z^2}{r^2}\right)\right) \\&#xA;    z''=&amp;amp;\ -myu \frac{z}{r^3} \left(1+\frac{3}{2} {J_2} \left(\frac{{r_{\text{eq}}}}{r}\right)^2 \left(1-\frac{5z^2}{r^2}\right)\right) \\&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;r^2=&amp;amp;\ x^2+y^2+z^2 \\&#xA;{J_2},{r_{\text{eq}}},myu &amp;amp;\ \textrm{constants}&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used &lt;code&gt;RK4&lt;/code&gt;, &lt;code&gt;RKF5&lt;/code&gt; methods, programmed in Python, also used &lt;code&gt;odeint&lt;/code&gt; function, &lt;code&gt;integrate.ode&lt;/code&gt; function with &lt;code&gt;lsode&lt;/code&gt; and &lt;code&gt;vode&lt;/code&gt; methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All they give small errors, which increase in long time interval. Therefore, in long time interval I get wrong results!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code of &lt;code&gt;odeint&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from pylab import *&#xA;from scipy.integrate import *&#xA;&#xA;myu=398600.4418E+9&#xA;J2=1.08262668E-3&#xA;req=6378137&#xA;&#xA;t0=86400*2.3567000000000000E+04&#xA;tN= 86400*2.3567250000000000E+04&#xA;&#xA;y0 = np.array([-9.0656779992979474E+05, -4.8397431127596954E+06, -5.0408120071376814E+06, -7.6805804020022015E+02, 5.4710987127502622E+03, -5.1022193482389539E+03])&#xA;&#xA;def f(y,t):&#xA;    r2=(y[0]**2 + y[1]**2 + y[2]**2)&#xA;    r3=r2**(3/2)&#xA;&#xA;    w=1+1.5*J2*(req*req/r2)*(1-5*y[2]*y[2]/r2)&#xA;&#xA;    return np.array([&#xA;        y[3],&#xA;        y[4],&#xA;        y[5],&#xA;        -myu*y[0]*w/r3,&#xA;        -myu*y[1]*w/r3,&#xA;        -myu*y[2]*w/r3,&#xA;    ])&#xA;&#xA;&#xA;t = linspace(t0,tN)&#xA;&#xA;result = odeint(f,y0,t)&#xA;&#xA;for i in range(0,6):&#xA;    print(result[-1][i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Julia code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Pkg.add(&quot;DifferentialEquations&quot;)&#xA;Pkg.add(&quot;ODEInterfaceDiffEq&quot;)&#xA;&#xA;using DifferentialEquations&#xA;using ODEInterfaceDiffEq&#xA;&#xA;myu=398600.4418E+9&#xA;J2=1.08262668E-3&#xA;req=6378137&#xA;&#xA;t0=86400*2.3567000000000000E+04&#xA;tN= 86400*2.3567250000000000E+04&#xA;&#xA;y0 =[-9.0656779992979474E+05, -4.8397431127596954E+06, -5.0408120071376814E+06, -7.6805804020022015E+02, 5.4710987127502622E+03, -5.1022193482389539E+03]&#xA;&#xA;function f(t,y,dy)&#xA;  r2=(y[1]^2 + y[2]^2 + y[3]^2)&#xA;  r3=r2^(3/2)&#xA;  w=1+1.5J2*(req*req/r2)*(1-5y[3]*y[3]/r2)&#xA;  dy[1] = y[4]&#xA;  dy[2] = y[5]&#xA;  dy[3] = y[6]&#xA;  dy[4] = -myu*y[1]*w/r3&#xA;  dy[5] = -myu*y[2]*w/r3&#xA;  dy[6] = -myu*y[3]*w/r3&#xA;end&#xA;&#xA;t = linspace(t0,tN)&#xA;&#xA;prob = ODEProblem(f,y0,(t0,tN))&#xA;&#xA;@time sol = solve(prob,Vern7(),abstol=1e-13)&#xA;&#xA;println(sol[end])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="23929" OwnerDisplayName="Sonya Seyrios" LastEditorUserId="23929" LastEditDate="2017-04-14T07:15:50.303" LastActivityDate="2017-04-14T20:30:04.530" Title="Small errors accumulate while solving ODE of motion" Tags="|python|scipy|differential-equations|" AnswerCount="2" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="26621" PostTypeId="1" AcceptedAnswerId="26624" CreationDate="2017-04-13T18:34:23.090" Score="0" ViewCount="202" Body="&lt;p&gt;I am working on mesh generation and optimization and encountered this problem. For example, in the figure below, we have the edge-vertex connectivity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EEgUs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EEgUs.png&quot; alt=&quot;simple mesh&quot;&gt;&lt;/a&gt;&#xA;0-1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1-4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4-5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2-5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3-2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;0-3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1-2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1-6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7-6&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4-7&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to generate the faces in terms of face-vertex connectivity? For this case,&#xA; the faces can be &quot;generated&quot; by visual inspection (they are 0-3-2-1, 1-2-5-4, and 1-4-7-6). But is there an algorithm to automate this?&lt;/p&gt;&#xA;" OwnerUserId="8203" LastEditorUserId="8203" LastEditDate="2017-04-13T19:01:36.320" LastActivityDate="2017-04-13T22:45:06.283" Title="From edge-vertex connectivty to face-vertex connectivity?" Tags="|finite-element|computational-geometry|mesh-generation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26622" PostTypeId="1" CreationDate="2017-04-13T21:31:08.443" Score="2" ViewCount="157" Body="&lt;p&gt;I am trying to code a scalable parallel AMR for unstructured grid. There seems to be three approaches for this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Store some global grid info on each processor and partition with parmetis (The easiest approach I am pursuing). I have never used ParMetis before (just metis) so I do not know how one would be able to read a given mesh file in parallel for partitioning, and then later on store the file in a format readable in parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Store global grid on rank=0, use Metis and then scatter. I don't like this approach because it is more involved than (a), and you still need one node that is able to store the whole grid anyway. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;c) Use an approach like p4est, which only works for structured quads/hexes. I am not sure how much global information is stored on each processor but it seems pretty scalable. Can the techniques used for the mesh encoding in p4est be used for unstructured grids as well. It seems to me even if one stores only one integer, say to indicate to which processor an element belongs, sooner or later scalability will be limited due to not being able to store that information one a node. What I think is done in p4est is to partition a coarse grid (able to be stored on each node), and do refinement after partitioning, but this approach can not be used for unstructured AMR, right ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any input.&lt;/p&gt;&#xA;" OwnerUserId="18132" LastEditorUserId="18132" LastEditDate="2017-04-13T21:57:11.927" LastActivityDate="2017-04-13T21:57:11.927" Title="scalable parallel mesh/amr on unstructured grid" Tags="|parallel-computing|mesh-generation|adaptive-mesh-refinement|unstructured-mesh|domain-decomposition|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26624" PostTypeId="2" ParentId="26621" CreationDate="2017-04-13T22:45:06.283" Score="4" Body="&lt;p&gt;As noted in the comments, your problem doesn't seem quite solvable as stated. However, if you include the assumption that &lt;strong&gt;each node has a 2D coordinate&lt;/strong&gt; associated with it, then it is solvable. With the coordinates, the embedding has been chosen for you, so you don't have to worry about uniqueness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing to do is to name each edge and build the node-edge adjacency list:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edge Names:&#xA;\begin{align*}&#xA;a=\{0,1\} &amp;amp;&amp;amp;&#xA;b=\{1,4\}\\&#xA;c=\{4,5\} &amp;amp;&amp;amp; d = \{2,5\}\\&#xA;e=\{3,2\} &amp;amp;&amp;amp; f = \{0,3\}\\&#xA;g=\{1,2\} &amp;amp;&amp;amp; h = \{1,6\}\\&#xA;i=\{7,6\} &amp;amp;&amp;amp; j = \{4,7\}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Node-edge adjacency sorted &quot;clockwise&quot;:&#xA;\begin{align*}&#xA;0 &amp;amp;= \{a,f\}\\&#xA;1 &amp;amp;= \{b,g,a,h\}\\&#xA;2 &amp;amp;= \{d,e,g\}\\&#xA;3 &amp;amp;= \{e,f\}\\&#xA;4 &amp;amp;= \{c,b,j\}\\&#xA;5 &amp;amp;= \{c,d\}\\&#xA;6 &amp;amp;= \{i,h\}\\&#xA;7 &amp;amp;= \{j,i\}\\&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I made no distinction about inbound vs outbound edges. You'll have to take care of this at some point, but you can decide where to do your bookkeeping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, set up two arrays of booleans. The first array, $F$, indicates whether an edge has been touched in the &quot;forward&quot; direction (ie, in the direction listed above, like $0 \rightarrow 1$). A similar array, $B$, keeps track of the edges that you've used in the reverse direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the part that actually does some work: Pick an edge and do a depth-first search. When picking the next edge to visit, be sure to turn &quot;left&quot;. (Pick the next edge in the clockwise adjacency list for the node that you're arriving at.) Mark each edge as &quot;visited&quot; in the direction that you are travelling. Once you encounter an edge that you have already used, pop everything off of your stack, and you have your cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a worked example starting at edge $g$.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Path $P = \{\}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Select edge: $g$ (forward). Push onto path: $P=\{g\}$. &#xA;Mark $F[g] = visited$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Arriving at node 2. Next edge after $g$ is edge $d$ (forward).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Push $d$ into $P$. $P = \{g,d\}$. Mark $F[d]=visited$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Arriving at node 5. Next edge after $d$ is edge $c$ (backward).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Push $-c$ into $P$. $P = \{g,d,-c\}$. Mark $B[c]=visited$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Arriving at node 4. Next edge after $c$ is edge $b$ (backward).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Push $-b$ onto $P$. $P=\{g,d,-c,-b\}$. Mark $B[b]=visited$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Arriving at node 1. Next after $b$ is edge $g$ (forward). &lt;strong&gt;$F[g]$ is already visited&lt;/strong&gt;, so stop.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Record your path $P$ as it now holds the edges that comprise a face in your graph.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Clear $P$, pick a new (unvisited) edge, and start again. Repeat this process until you have used every edge in both directions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note, this algorithm will terminate after every edge has been visited precisely once in each direction. This means that your &quot;boundary&quot; edges will all be linked together in a single &quot;face&quot;, but this face will have all of its edges pointing with clockwise circulation, while the interior faces will have counterclockwise circulation. This can be useful for identifying which edges make up the boundary of your domain. &lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-04-13T22:45:06.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26625" PostTypeId="2" ParentId="26620" CreationDate="2017-04-14T05:02:13.743" Score="4" Body="&lt;p&gt;As noted in the comments, when I ran your code (using LSODE in SciPy) I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; result = odeint(f,y0,t)&#xA; lsoda--  warning..internal t (=r1) and h (=r2) are&#xA;       such that in the machine, t + h = t on the next step  &#xA;       (h = step size). solver will continue anyway&#xA;      in above,  r1 =  0.2036188800000D+10   r2 =  0.1292689215500D-07&#xA; lsoda--  warning..internal t (=r1) and h (=r2) are&#xA;       such that in the machine, t + h = t on the next step  &#xA;       (h = step size). solver will continue anyway&#xA;      in above,  r1 =  0.2036188800000D+10   r2 =  0.1292689215500D-07&#xA; lsoda--  at current t (=r1), mxstep (=i1) steps   &#xA;       taken on this call before reaching tout     &#xA;      in above message,  i1 =       500&#xA;      in above message,  r1 =  0.2036188801501D+10&#xA;/usr/lib64/python2.7/site-packages/scipy/integrate/odepack.py:218: ODEintWarning: Excess work done on this call (perhaps wrong Dfun type). Run with full_output = 1 to get quantitative information.&#xA;  warnings.warn(warning_msg, ODEintWarning)&#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&amp;gt;&amp;gt;&amp;gt; for i in range(0,6):&#xA;...     print(result[-1][i])&#xA;... &#xA;9.07638086329e+223&#xA;9.48797814966e-81&#xA;1.04657495534e-312&#xA;2.52059148034e-306&#xA;9.94646703738e+86&#xA;8.48798317088e-313&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is what I assume you must mean by the errors. I am not sure why it would do this, but there must be some bad default values for the tolerances, causing the timesteps to go really low with lsode. Multistep methods have to kick start using a low order method like Euler, and so their error estimators are really bad in the first few steps. I think it's going awry there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I investigated this a bit using DifferentialEquations.jl. It has a bunch of algorithms ready, so it makes it a easier to do this kind of analysis. Setting up the problem is very similar to your Python code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myu=398600.4418E+9&#xA;J2=1.08262668E-3&#xA;req=6378137&#xA;&#xA;t0=86400*2.3567000000000000E+04&#xA;tN= 86400*2.3567250000000000E+04&#xA;&#xA;y0 =[-9.0656779992979474E+05, -4.8397431127596954E+06, -5.0408120071376814E+06, -7.6805804020022015E+02, 5.4710987127502622E+03, -5.1022193482389539E+03]&#xA;&#xA;function f(t,y,dy)&#xA;  r2=(y[1]^2 + y[2]^2 + y[3]^2)&#xA;  r3=r2^(3/2)&#xA;  w=1+1.5J2*(req*req/r2)*(1-5y[3]*y[3]/r2)&#xA;  dy[1] = y[4]&#xA;  dy[2] = y[5]&#xA;  dy[3] = y[6]&#xA;  dy[4] = -myu*y[1]*w/r3&#xA;  dy[5] = -myu*y[2]*w/r3&#xA;  dy[6] = -myu*y[3]*w/r3&#xA;end&#xA;&#xA;t = linspace(t0,tN)&#xA;&#xA;using DifferentialEquations&#xA;prob = ODEProblem(f,y0,(t0,tN))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't think much of a comment needs to be said there, since it's almost a straight copy plus changing 0-based indexing to 1-based indexing. First I tried this with LSODA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using LSODA&#xA;@time sol = solve(prob,lsoda(),abstol=1e-12)&#xA;#0.000690 seconds (17.74 k allocations: 373.578 KB)&#xA;println(sol[end])&#xA;#[1.20141e6,1.14517e6,7.13797e6,74.8469,-7231.03,1079.12]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run this with Sundials:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@time sol = solve(prob,CVODE_Adams(),abstol=1e-14)&#xA;#0.000730 seconds (8.87 k allocations: 187.641 KB)&#xA;println(sol[end])&#xA;#[1.1016e6,3.68894e6,6.3157e6,513.044,-6351.17,3642.44]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It seems like it works out alright. When I use the FORTRAN &lt;code&gt;dopri&lt;/code&gt; method (this is just a wrapper to the Hairer method):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using ODEInterfaceDiffEq&#xA;@time sol = solve(prob,dopri5(),abstol=1e-12)&#xA;#0.000885 seconds (7.83 k allocations: 180.469 KB)&#xA;println(sol[end])&#xA;[-965604.0,-1.84395e6,-5.65673e6,-292.419,7785.53,-2437.98]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can see that errors build up more. I checked using &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/NonStiffODE/Linear%20Work-Precision%20Diagrams.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;a more modern Julia re-implementation&lt;/a&gt; of the Dormand-Prince 4/5 algorithm&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@time sol = solve(prob,DP5(),abstol=1e-12)&#xA;#0.000467 seconds (6.82 k allocations: 155.828 KB)&#xA;println(sol[end])&#xA;[-9.60193e5,-1.96886e6,-5.61328e6,-319.081,7737.2,-2594.4]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It seems like this method is just very error prone. I would suggest using newer algorithms, like &lt;code&gt;Tsit5()&lt;/code&gt; and &lt;code&gt;Vern7()&lt;/code&gt;. You can see from the benchmarks that I posted that they are more efficient than the old Fortran codes, and I discuss this more &lt;a href=&quot;https://scicomp.stackexchange.com/questions/24998/which-runge-kutta-method-is-more-accurate-dormand-prince-or-cash-karp/25031#25031&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@time sol = solve(prob,Tsit5(),abstol=1e-12)&#xA;#0.000485 seconds (8.97 k allocations: 218.261 KB)&#xA;println(sol[end])&#xA;[1.03967e6,-2.41448e6,6.47286e6,-527.843,-7082.73,-2564.45]&#xA;&#xA;@time sol = solve(prob,Vern7(),abstol=1e-13)&#xA;#0.001131 seconds (11.19 k allocations: 266.861 KB)&#xA;println(sol[end])&#xA;[1.10658e6,-1.5227e6,6.79846e6,-369.784,-7333.39,-1590.5]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So it looks like the higher order methods (&lt;code&gt;CVODE_Adams&lt;/code&gt; and &lt;code&gt;Vern7&lt;/code&gt;) are in the same direction. To see what the correct solution should be, we can re-solve it using higher-precision numbers. Here we will use &lt;code&gt;BigFloat&lt;/code&gt;s in the algorithms. We can do this just by changing the number types that we build the problem type with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y0_big =big([-9.0656779992979474E+05, -4.8397431127596954E+06, -5.0408120071376814E+06, -7.6805804020022015E+02, 5.4710987127502622E+03, -5.1022193482389539E+03])&#xA;t0_big=big(86400*2.3567000000000000E+04)&#xA;tN_big=big(86400*2.3567250000000000E+04)&#xA;prob_big = ODEProblem(f,y0_big,(t0_big,tN_big))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then solve the problem with higher precision numbers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@time sol = solve(prob_big,Vern7(),abstol=1e-20,reltol=1e-20)&#xA;#5.124532 seconds (20.51 M allocations: 1020.807 MB, 29.95% gc time)&#xA;println(sol[end]) &#xA;&#xA;BigFloat[1.1053381662331820146336046671754933670595930944811486000087489128455927171218624720058718205588449974168956065779438050345e+06,-1.5416202537237067975875829305475944291202837487809289389890156042765665624065710316274224354945044382194625246683296005503e+06,6.7926473294913369302731863142467582412609188710382594274952998303466256241086526296652443665837323225497701006408603155527e+06,-3.7312828821058534895294734492513244053231049180643905427777841548594598586886829969951338572602652254542799098196952577885e+02,-7.3297961245134371967675775894582329860535459110035746501350371674509262650960485899126786425646757794442551813929572586948e+03,-1.61096155619704279518364795928351631960055375601824673571833246176656122912319418152730921855088538873761614569864638284e+03]&#xA;&#xA;@time sol = solve(prob_big,Tsit5(),abstol=1e-20,reltol=1e-20)&#xA;#46.843028 seconds (202.64 M allocations: 9.848 GB, 21.61% gc time)&#xA;println(sol[end])&#xA;&#xA;BigFloat[1.1053381662331820147067348726438400371709496335752162175134992231756774664804231225139143546847966065233311732755080517428e+06,-1.5416202537237067964033929090489735675629358124880169057038770565130982115037191409720907051171224745683301005104028510218e+06,6.7926473294913369306108500165460363424229267867186882151308990443549948472674738460107391298061753478752197008344571867852e+06,-3.7312828821058534874482439660747198554520591273413619386987734456375288506664244304076979140861220572371658224120877522505e+02,-7.3297961245134371970128642262106938950880875625957763378149307834830636800982964288295248480875424454937258559641758284114e+03,-1.6109615561970427939085757874332499247300266093609788206242850106668114350752364680610553537125616823780476995508356156203e+03]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is confirmed by two algorithms at high precision, so this must be the value.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Recap&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;So let's recap a little bit. The timings are really to quick to say much, but at least in Julia there isn't a problem with getting high accuracy. The older methods like &lt;code&gt;lsoda&lt;/code&gt; and &lt;code&gt;dopri5&lt;/code&gt; have higher errors. Even &lt;code&gt;DP5&lt;/code&gt;. Methods and coefficients have been refined since the time these algorithms were made, which is why I wouldn't really recommend them anymore. Sundials and the newer Runge-Kutta methods in DifferentialEquations.jl (OrdinaryDiffEq.jl) are a lot more accurate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few sources of error that this shows. For one, each method has discretization error, which causes a difference between them. But that should go away with a small enough tolerance. This isn't necessarily the case because of floating point issues, the second source of error. It's a very long timespan, and so you need very low tolerances to not have a substantial buildup of error. This is confirmed by the &lt;code&gt;BigFloat&lt;/code&gt; results which show that we can only get the first few digits right if we go to much higher precision numbers (tolerances of about &lt;code&gt;1e-13&lt;/code&gt; without getting junk, and we needed to go lower!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But lastly, even after going to arbitrary precision numbers, putting a low tolerance, and essentially having two algorithms converge to the same result for the first 10 or so decimal places, we are still off from your proposed &quot;true result&quot;. This is likely due to the fact that the initial condition was slightly inaccurate as well, which builds up over time to cause an error in the first digit. The error from the initial condition can grow very fast, so you need to make sure that is very accurate!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(However, I was not able to re-create the SciPy problems here... I am not sure why)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you need to be careful about all 3 sources of errors, as demonstrated here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding a plot for good measure.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot(sol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6wPIQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6wPIQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know your problem, but you should always double-check the plot!&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Can we get around the floating point errors?&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The next question to address is, can we get around the floating point errors? Yes, we can rescale the problem. There is no reason to have such large numbers for every number! We can instead move every number down by something like 10^3 by changing units. I would suggest doing a change of units like that in both the dependent variables and time, and see if it can get better results with just &lt;code&gt;Float64&lt;/code&gt;'s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But most importantly, I hope this shows a good way to investigate the results.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Edit: More Verification&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Because it was mentioned in the comments, I ran this with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@time sol = solve(prob_big,Vern9(),abstol=1e-25,reltol=1e-25)&#xA;println(sol[end])&#xA;&#xA;BigFloat[1.1053381662331822596169776849641239592819519884844738459016726262029644062128757651516073415939693720405423642205909058064e+06,-1.5416202537237025194193402796238592848398830150314947716991093949659223769788623589030795132080240499488785780018224381598e+06,6.7926473294913380453538931349439465820220085181755818261687088166926884907951721678701419759674780893612240340931187099615e+06,-3.7312828821058461338614861968657625567908036695262270829525438975640521184553269025483381748908274236026477510068778050456e+02,-7.3297961245134380914206076512518039416894076161354630688132265052545303764963243830516528946565213622293265857749536691384e+03,-1.6109615561970383265735660519068721854582438082504069296606070807069042413835561362598331583440076713255077056006112545402e+03]&#xA;&#xA;@time sol = solve(prob_big,Feagin14(),abstol=1e-25,reltol=1e-25)&#xA;println(sol[end])&#xA;BigFloat[1.1053381662331822596169777141870567234610511965557086835077483653533437799111462386523440877691562284278052381367094500505e+06,-1.5416202537237025194193397999916272100650863657026496400673463469591815591302649671026419236684086191564320676752769500241e+06,6.7926473294913380453538932693163486282736145849617871379561032756359556363271724242234264204395645390979862595520421289454e+06,-3.7312828821058461338614853616747368391649624257444346156270400632544621072526444156009615705008474094021665763109262437141e+02,-7.3297961245134380914206077511660934608408482056577325775582350370158926687300526721134971356964860345847978865296553096107e+03,-1.6109615561970383265735655400955954012952369987169718357788394884494500522058249689736956400471634474979232165731626140887e+03]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So many different algorithms are giving, at very high precision and accuracy, the same result to the first 14 decimal places. Thus for these inputs, this is a value you can trust. If this value is off, then it's the input numbers (the initial conditions, the constants, or the times) which are off.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Another Edit&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;We worked it out in the JuliaDiffEq Gitter chat channel. There was an error in the z-acceleration component. The new script is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myu=398600.4418E+9&#xA;J2=1.08262668E-3&#xA;req=6378137&#xA;&#xA;t0=86400*2.3567000000000000E+04&#xA;tN= 86400*2.3567250000000000E+04&#xA;&#xA;y0 =[-9.0656779992979474E+05, -4.8397431127596954E+06, -5.0408120071376814E+06, -7.6805804020022015E+02, 5.4710987127502622E+03, -5.1022193482389539E+03]&#xA;&#xA;function f(t,y,dy)&#xA;  r2=(y[1]^2 + y[2]^2 + y[3]^2)&#xA;  r3=r2^(3/2)&#xA;  w=1+1.5J2*(req*req/r2)*(1-5y[3]*y[3]/r2)&#xA;  w2=1+1.5J2*(req*req/r2)*(3-5y[3]*y[3]/r2)&#xA;  dy[1] = y[4]&#xA;  dy[2] = y[5]&#xA;  dy[3] = y[6]&#xA;  dy[4] = -myu*y[1]*w/r3&#xA;  dy[5] = -myu*y[2]*w/r3&#xA;  dy[6] = -myu*y[3]*w2/r3&#xA;end&#xA;&#xA;t = linspace(t0,tN)&#xA;&#xA;using DifferentialEquations&#xA;prob = ODEProblem(f,y0,(t0,tN))&#xA;sol = solve(prob,DP5(),abstol=1e-13,reltol=1e-13)&#xA;println(sol[end])&#xA;&#xA;[1.09206e6,-1.97193e6,6.68529e6,-408.338,-7208.31,-2057.99]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which hits the true solution dead on. Credit to @crbinz&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-04-14T20:30:04.530" LastActivityDate="2017-04-14T20:30:04.530" CommentCount="32" ContentLicense="CC BY-SA 3.0" />
  <row Id="26627" PostTypeId="2" ParentId="25134" CreationDate="2017-04-14T10:05:28.477" Score="2" Body="&lt;p&gt;Thanks a lot for the answers. I would like to summarize what have been said before and add some more details in this post. I hope it will help if anyone faces similar problems.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;(1)&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First note&lt;/strong&gt;: indeed, inf–sup stability guarantees existence and uniqueness. Yet I missed a crucial point: it also guarantees optimal error estimates (thanks &lt;a href=&quot;https://scicomp.stackexchange.com/users/10160/knl&quot;&gt;@knl&lt;/a&gt;). In particular, if your FE–pair is not stable, you may get nonphysical solutions, e.g. spurious modes of pressure (nonphysical oscillations) or locking phenomenon. Nice explanation and examples are given in [3, p. 62]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second note&lt;/strong&gt;: if we work w/ essential + natural BCs, we may choose spaces $\mathbb H^1(\Omega) \times \mathbb L^2(\Omega)$ and and we are fine. If we work w/ essential BCs only (as in my post), we may choose spaces $\mathbb H^1(\Omega) \times \mathbb L^2_0(\Omega)$ and we are fine again, at least mathematically. However, working w/ a discrete analogue of $\mathbb L^2_0(\Omega)$ is somewhat inconvenient in practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way is to use approach from [1, p. 309] which I mentioned in my question. Another way is to fix one of the pressure DOFs (just like we usually handle essential BCs). A very good (and fun) explanation is given in [4, p. 293]:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KuDmM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KuDmM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for my experience, I tried to solve the Stokes problem w/ essential BCs without any constraints on the pressure space. I used equivalent formulation where block (2, 1) equals $-\mathbf B$ instead of $\mathbf B$ (so the resulting matrix is not symmetric but sign–definite). I started with a random guess and used BiCGStab and PBiCGStab w/ block diagonal preconditioner. &#xA;Convergence history in terms of number of iterations and working time were similar to the case w/ essential + natural BCs.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;(2)&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;It is much more efficient and smart to use MG–iterations to approximate actions of inverses of diagonal blocks of $\mathbf A = \text{diag}(\mathbf A_{11}, \mathbf A_{22})$ since they are elliptic. This is a part in building physics–based / block preconditioners which are considered to be the most robust choice for “complex” problems if one wants to solve the system in $O(n)$ time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/users/393/wolfgang-bangerth&quot;&gt;Wolfgang Bangerth&lt;/a&gt; mentioned this approach. It was extremely helpful for me to watch his lectures on preconditioners (especially &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.676.38.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; which covers block preconditioners), so I marked his answer.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I decided not to implement Vanka’s approach—I implemented block diagonal preconditioner for the problem in my question and AL–preconditioner for the Oseen problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Volker John&lt;br&gt;&#xA;&lt;em&gt;Finite Element Methods for Incompressible Flow Problems&lt;/em&gt;&lt;br&gt;&#xA;2016&lt;/li&gt;&#xA;&lt;li&gt;Daniele Boffi, Franco Brezzi, Michel Fortin&lt;br&gt;&#xA;&lt;em&gt;Mixed Finite Element&lt;br&gt;&#xA;Methods and Applications&lt;/em&gt;&lt;br&gt;&#xA;2013&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-04-14T10:05:28.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26628" PostTypeId="1" AcceptedAnswerId="26630" CreationDate="2017-04-14T10:55:05.657" Score="8" ViewCount="7838" Body="&lt;p&gt;In our university some Ph.D students for computational methods prefer FORTRAN over MATLAB. I can't understand why? What is the difference between them when are used in computational methods like spectral methods, FEM or BEM? Please tell me which one do you use and why? and what language do you recommend me to learn?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to work on computational methods in Fluid Dynamics.&lt;/p&gt;&#xA;" OwnerUserId="22763" LastActivityDate="2022-12-14T18:08:58.637" Title="What is the difference between MATLAB and FORTRAN?" Tags="|matlab|fluid-dynamics|fortran|languages|" AnswerCount="7" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26629" PostTypeId="2" ParentId="26628" CreationDate="2017-04-14T12:18:39.490" Score="3" Body="&lt;p&gt;&lt;em&gt;This question can be considered opinion based, but can be useful for who starts in the field. In the answer I try to do not use personal opinion but suggest some criteria of choice&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well to begin Fortran is program language with a long story, it was born to do computations. Matlab is more like to an environment, it is built over (you can read make in) Fortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab is implemented to provide a user friendly environment in which to develop numerical code (in particular for handling matrices) quickly and simplicity. It comes with many features and methods already available to be immediately productive. Obviously, these advantages are offset by some compromises, such as the speed of execution (here I am a bit 'simplifying, but I think the idea is quite clear).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reasons &lt;em&gt;Matlab is mainly directed to the development prototype&lt;/em&gt;,&#xA;abusing a computer terminology is not for production use.&#xA;Conversely Fortran is a compiled language in which the development of the code is slower, but has some advantage such as performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus an answer to the question is depend. Depends on what? Your goals.&#xA;I try to explain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do CFD with Matlab.&#xA;In this case I expect that your work is more focused on the development of new numerical methods and not on the application itself.&#xA;This mainly because of the ease of development (read implementation) new ideas / algorithms and because would you use problems of &quot;small&quot; size in which the appearance of computations performance is less important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if your interest is the simulation results, perhaps using real problems, then you most likely are better Fortran or Domain specific languages/programs (commercial or otherwise).&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2017-04-14T12:18:39.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26630" PostTypeId="2" ParentId="26628" CreationDate="2017-04-14T12:37:40.637" Score="11" Body="&lt;p&gt;When one uses a low–level programming language, e.g. C++ or FORTRAN, one essentially controls lots of things: how parameters are passed, how data structures are aligned in memory, what is the most efficient way to loop over elements of a big sparse matrix (see cache thrashing) when one multiples it, and so on. In contrast, when one uses high–level software, lots of things are “black box”—it is usually unknown what happens underhood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, there is much more to this question, but this is the basic difference. I am not a confident Matlab user, but I can give you an example concerning Wolfram Language: notstandingwith it is very high–level, it &lt;a href=&quot;https://reference.wolfram.com/language/tutorial/LinearAlgebraMatrixComputations.html&quot; rel=&quot;noreferrer&quot;&gt;uses&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/LAPACK&quot; rel=&quot;noreferrer&quot;&gt;LAPACK&lt;/a&gt; for dense matrix computations (which is a standard software library for this purpose written in FORTRAN). It also relies on &lt;a href=&quot;https://en.wikipedia.org/wiki/Math_Kernel_Library&quot; rel=&quot;noreferrer&quot;&gt;Intel MKL&lt;/a&gt; which is again written in FORTRAN / C. (So high–level software relies heavily on legacy code.)&#xA;It also has &lt;a href=&quot;http://blog.wolfram.com/2011/12/07/10-tips-for-writing-fast-mathematica-code/&quot; rel=&quot;noreferrer&quot;&gt;Compile[]&lt;/a&gt; function which may speed things up dramatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So these may be reasons why some students of your university prefer FORTRAN: it has low–level features, it is convenient for matrix / vector manipulations, it is pretty old and has a huge legacy (Intel MKL is a good example), it is compilable and fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at modern open source FEM software libraries (deal.II, mfem, FEniCS), you will notice that they are developed using C++. FEniCS also has a high–level Python interface which is usually &lt;a href=&quot;https://fenicsproject.org/qa/5323/difference-between-application-fenics-python-application&quot; rel=&quot;noreferrer&quot;&gt;slower&lt;/a&gt; than C++’s one.&lt;br&gt;&#xA;I believe they tend to use C++ for the reasons I mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for me, I use C++ for large computations. I also use Mathematica for symbolic computations (which is extremely useful) or if I want to code something fast. So my suggestion would be this: learn at least one high–level and at least one low–level language. &lt;/p&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-04-14T12:37:40.637" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26631" PostTypeId="2" ParentId="26628" CreationDate="2017-04-14T13:20:27.550" Score="8" Body="&lt;p&gt;In general, you will be much more productive writing software in a higher-level&#xA;language (e.g MATLAB) that has features useful in describing problems in your particular&#xA;domain (e.g. matrices in computational science). Often much more time is spent&#xA;&lt;em&gt;writing&lt;/em&gt; the software rather than acutally &lt;em&gt;running&lt;/em&gt; it so reducing programming&#xA;time by using the right higher level language as opposed to reducing runtime&#xA;by using a lower-level language is the right tradeoff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your question is not so much MATLAB vs FORTRAN as it is high-level versus&#xA;low-level languages. Even the assumption that I made about a high-level language&#xA;leading to slower running code is not necessarily true. One of the goals of the &lt;a href=&quot;https://julialang.org/&quot; rel=&quot;noreferrer&quot;&gt;Julia&lt;/a&gt; programming&#xA;language project is to challenge that assumption. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get a much better understanding of this low-level versus high-level language&#xA;comparison, I strongly suggest taking a look at some of the documents written&#xA;by the Julia developers (e.g. &lt;a href=&quot;https://julialang.org/publications/julia-fresh-approach-BEKS.pdf&quot; rel=&quot;noreferrer&quot;&gt;Julia: A Fresh Approach to Numerical Computing&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding these broader issues will allow you to make an informed decision about the specific question, MATLAB versus FORTRAN.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-04-14T13:20:27.550" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26632" PostTypeId="2" ParentId="26620" CreationDate="2017-04-14T13:46:25.233" Score="1" Body="&lt;p&gt;Your dynamics is Hamiltonian. Therefore unlike for dissipative systems, solutions are dense and a numerical error will most likely can kick you from one energy shell to another. If this happens systematically, your solution may slowly drift to a higher or lower energy. While this does not seem to be a big problem here (the solutions I get are pretty periodic on the timescale in question), it may lead to some undesired inaccuracy. If you want to avoid this, you need a symplectic integrator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, if you precisely want to predict a periodic dynamics, a long-term integration does not seem like the best approach to me, as a slight error in the period length may destroy everything. You are probably better off determining the period length as exactly as you can (either from experimental data or by finding it with a collocation method or similar) and go from there.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2017-04-14T13:46:25.233" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26633" PostTypeId="2" ParentId="6770" CreationDate="2017-04-14T15:16:51.403" Score="1" Body="&lt;p&gt;Summarizing the answer to the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;zgetrf&lt;/code&gt; (LU factorization of a complex matrix $A$) returns &lt;code&gt;info=i&amp;gt;0&lt;/code&gt; if $U_{ii}=0$; thus in your case $U_{44}=0$ during the attempted LU-decomposition by MKL. That happens (excluding very exotic cases) when your original matrix $A$ is singular or numerically singular. If &lt;code&gt;zgetrf&lt;/code&gt; returned an error there is no point trying to execute back-substitution &lt;code&gt;zgetrs&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&quot;Why is my matrix singular?&quot;&lt;/em&gt; — is a totally different question that requires the information about where $A$ comes from in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;zgesvd&lt;/code&gt; will try to compute the singular value decomposition (SVD) of your matrix $A$. Note, every matrix (full-rank, singular, numerically singular - you name it) has an SVD. In your question, you mentioned that SVD has no troubles and $A$ is regular. I guess, you mean that &lt;code&gt;zgesvd&lt;/code&gt; does not return any error message, which is an expected behavior. &lt;code&gt;zgesvd&lt;/code&gt; returns &lt;code&gt;info&lt;/code&gt;$\neq$&lt;code&gt;0&lt;/code&gt; only if some input parameter to it was wrong or the computation of SVD did not converge for some reason (see &lt;a href=&quot;https://software.intel.com/en-us/node/469236&quot; rel=&quot;nofollow noreferrer&quot;&gt;zgesvd&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum it up: for SVD-purposes, your matrix is fine, and in the comments, you listed the singular values that confirms the hypothesis of matrix being singular. Thus, LU-decomposition &lt;code&gt;zgetrf&lt;/code&gt; will fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One may have the reasons to use &lt;code&gt;zgetrf&lt;/code&gt; for factorizing followed by &lt;code&gt;zgetrs&lt;/code&gt;; however, you might find useful &lt;code&gt;zgesv&lt;/code&gt;, that is technically the wrapper around &lt;code&gt;zgetrf&lt;/code&gt;/&lt;code&gt;zgetrs&lt;/code&gt; (if you don't use its iterative refinement options). Its error messages (&lt;code&gt;info&lt;/code&gt;) are the same as &lt;code&gt;zgetrf&lt;/code&gt;. This might simplify your life in catching errors during the factorization and not executing unnecessary subroutines.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-04-14T15:16:51.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26634" PostTypeId="2" ParentId="25174" CreationDate="2017-04-14T16:31:22.470" Score="1" Body="&lt;p&gt;The nomenclature used in the literature is misleading. High-performance implementations of both the supernodal as well as the multifrontal methods operate on &lt;em&gt;supernodes&lt;/em&gt;, i.e. multiple columns rather than single columns. The difference between the two methods lies in how they propagate the Schur complement: when the supernodal method tackles column (or supernode) $k$, it takes the $k$th column of $A$ and updates it with the Schur complements of the columns $1:k-1$. A more accurate name for the supernodal method would hence be &quot;left-looking&quot;. The multifrontal method, on the other hand, does some more intricate bookkeeping for the Schur complements which resembles (but is distinct from) a right-looking factorisation method.&lt;/p&gt;&#xA;" OwnerUserId="21342" LastActivityDate="2017-04-14T16:31:22.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26636" PostTypeId="1" CreationDate="2017-04-14T18:43:00.123" Score="1" ViewCount="105" Body="&lt;p&gt;Kahan summation applies to summation problems, but not to three-term recurrence relations. However, a three-term recurrence shares many of the features of a summation-albeit with a rescaling step at each iteration. Hence there seems to be hope for a Kahan summation for three-term recurrences. Has this idea been worked out in the literature or is the idea flawed in some way?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2017-04-16T01:43:30.937" LastActivityDate="2017-04-16T01:43:30.937" Title="Kahan Summation for Three-Term Recurrences" Tags="|reference-request|stability|floating-point|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26638" PostTypeId="2" ParentId="10692" CreationDate="2017-04-15T03:07:26.640" Score="2" Body="&lt;p&gt;After much deliberation, I tracked down the problem to a few software bugs:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;An array index is not a field value. &lt;/li&gt;&#xA;&lt;li&gt;Floating point precision is a factor when squaring a &lt;em&gt;VERY&lt;/em&gt; small number.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;NaN&lt;/code&gt; really isn't a number, and trying to normalize a zero-length vector requires careful thought. &lt;/li&gt;&#xA;&lt;li&gt;The really big misconception on my part: since the power flows between adjacent E points via connecting H points, and the source is a sinusoid, the magnitude (and Poynting vector) will, of course, oscillate. I've added a pair of RMS calculations that looks at the average and peak values over a period, and I'm getting something more like what I expected. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Ultimately, my math was correct, but my implementation had a few issues. Thankfully it was on the Poynting vector calculation side rather than the FDTD side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/8kyhg.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="7246" LastActivityDate="2017-04-15T03:07:26.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26640" PostTypeId="2" ParentId="26371" CreationDate="2017-04-15T04:31:13.630" Score="4" Body="&lt;p&gt;I took an example from &lt;a href=&quot;http://scicomp.stackexchange.com/a/26358/16900&quot;&gt;your other question&lt;/a&gt; and tried to demonstrate here what is going on. So, for completeness:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are applying the 6th order finite-difference scheme for numerical differentiation using the &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines/example_implementation/dss006&quot; rel=&quot;nofollow noreferrer&quot;&gt;following code&lt;/a&gt;. First, let's apply it on the constant function of different scale ($C_1, C_2, C_3$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;U(x)=C,\quad x\in[0,1]&#xA;$$&#xA;Since $\frac{\text{d}U(x)}{\text{d}x}=0$, the expected result of differentiation is a zero-vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used $N$=100 discretization points for &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$C_1=2.5545007\cdot10^2$, shown in blue&lt;/li&gt;&#xA;&lt;li&gt;$C_2=2.5545007\cdot10^4$, shown in red (original scale of the problem you described)&lt;/li&gt;&#xA;&lt;li&gt;$C_3=2.5545007\cdot10^6$, shown in black&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The function (solid lines), as well as it's derivative computed without mean subtraction (circle markers) and with mean subtraction (cross markers) is in the figure below. All crosses are exactly at $0$ (I had to assign a dummy value to display them on the semilog plot), which is an expected behavior for a constant function, as Kirill mentioned in a comment. Because you are passing to the differentiation subroutine (or $U-\text{mean}(U)=0$) vector of zeros.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/krxz8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/krxz8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, consider a tiny bit more complicated function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;U(x)=C\sin(x),\quad x\in[0,1]&#xA;$$&#xA;Since $\frac{\text{d}U(x)}{\text{d}x}=C\cos(x)$, you would also expect to get a zero-vector for $\frac{\text{d}U(x)}{\text{d}x}-C\cos(x)$, and the scales of the problem are similar to the ones above with a constant function.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bo20G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bo20G.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $\sin$, the mean subtraction did nothing for improving the accuracy of the computed derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in general, subtracting the mean value (i.e. a &lt;em&gt;constant&lt;/em&gt;) does not help too much for functions more complicated than a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the problem with numerical differentiation being sensitive to round off, you already got several good answers in the comments to both questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab code for completeness. The function &lt;code&gt;dss006&lt;/code&gt; is available &lt;a href=&quot;http://www.scholarpedia.org/article/Method_of_lines/example_implementation/dss006&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xmin=0;    xmax=1;&#xA;N=100;&#xA;x=linspace(xmin,xmax,N);&#xA;&#xA;fudge=1E-17;&#xA;cU1=2.5545007E+2;    cU2=2.5545007E+4;    cU3=2.5545007E+6;&#xA;U1=cU1*ones(N,1);    U2=cU2*ones(N,1);    U3=cU3*ones(N,1);&#xA;W1=cU1*sin(x);    W2=cU2*sin(x);    W3=cU3*sin(x);&#xA;dUx1=dss006(xmin,xmax,N,U1);&#xA;dUx2=dss006(xmin,xmax,N,U2);&#xA;dUx3=dss006(xmin,xmax,N,U3);&#xA;dUx1p=dss006(xmin,xmax,N,U1-mean(U1))+fudge;&#xA;dUx2p=dss006(xmin,xmax,N,U2-mean(U2))+fudge;&#xA;dUx3p=dss006(xmin,xmax,N,U3-mean(U3))+fudge;&#xA;dWx1=dss006(xmin,xmax,N,W1);&#xA;dWx2=dss006(xmin,xmax,N,W2);&#xA;dWx3=dss006(xmin,xmax,N,W3);&#xA;dWx1p=dss006(xmin,xmax,N,W1-mean(W1))+fudge;&#xA;dWx2p=dss006(xmin,xmax,N,W2-mean(W2))+fudge;&#xA;dWx3p=dss006(xmin,xmax,N,W3-mean(W3))+fudge;&#xA;&#xA;%% ONLY PLOTTING AFTER THIS POINT&#xA;MarkerSize=7;    LineWid=2.5;    ff=22;  MyLineWidth=1.0;&#xA;set(groot,'DefaultTextInterpreter','latex');&#xA;set(groot,'defaultLegendInterpreter','latex');&#xA;set(groot,'defaultAxesTickLabelInterpreter','latex'); &#xA;&#xA;figure(1)&#xA;ph(1)=semilogy(x,abs(U1),'b-','LineWidth',MyLineWidth);hold on;&#xA;p1_str='$|U_1(x)|,\;C_1=2.5545007\cdot 10^2$';&#xA;ph(2)=semilogy(x,abs(U2),'r-','LineWidth',MyLineWidth);hold on;&#xA;p2_str='$|U_2(x)|,\;C_2=2.5545007\cdot 10^4$';&#xA;ph(3)=semilogy(x,abs(U3),'k-','LineWidth',MyLineWidth);hold on;&#xA;p3_str='$|U_3(x)|,\;C_3=2.5545007\cdot 10^6$';&#xA;ph(4)=semilogy(x,abs(dUx1),'bo','LineWidth',MyLineWidth);hold on;&#xA;p4_str='$|\widetilde{dU_1/dx}(x)-0|$';&#xA;ph(5)=semilogy(x,abs(dUx2),'ro','LineWidth',MyLineWidth);hold on;&#xA;p5_str='$|\widetilde{dU_2/dx}(x)-0|$';&#xA;ph(6)=semilogy(x,abs(dUx3),'ko','LineWidth',MyLineWidth);hold on;&#xA;p6_str='$|\widetilde{dU_3/dx}(x)-0|$';&#xA;ph(7)=semilogy(x,abs(dUx1p),'bx','LineWidth',MyLineWidth);hold on;&#xA;p7_str='$|\widetilde{dU_1/dx}(x)-0|$,$-$mean($U_1$)';&#xA;ph(8)=semilogy(x,abs(dUx2p),'rx','LineWidth',MyLineWidth);hold on;&#xA;p8_str='$|\widetilde{dU_2/dx}(x)-0|$,$-$mean($U_2$)';&#xA;ph(9)=semilogy(x,abs(dUx3p),'kx','LineWidth',MyLineWidth);hold on;&#xA;p9_str='$|\widetilde{dU_3/dx}(x)-0|$,$-$mean($U_3$)';&#xA;title('$U(x)=C$');    grid on; set(gca,'FontSize',ff-2);&#xA;xlabel('$x$','FontSize',ff); &#xA;ylabel('$|U(x)|, |\widetilde{\frac{dU}{dx}}(x)-\frac{dU}{dx}(x)|$','FontSize',ff);&#xA;h=legend([ph(1) ph(2) ph(3) ph(4) ph(5) ph(6) ph(7) ph(8) ph(9)],p1_str,p2_str,p3_str,p4_str,p5_str,p6_str,p7_str,p8_str,p9_str,'Location','Best');&#xA;h.FontSize=ff-6;    h.EdgeColor=[1. 1. 1.];&#xA;ylim([1E-17 1E+9]);&#xA;set(gca,'YTick',[1E-15 1E-12 1E-9 1E-6 1E-3 1E-0 1E+3 1E+6 1E+9]);&#xA;&#xA;figure(2)&#xA;ph(1)=semilogy(x,abs(W1),'b-','LineWidth',MyLineWidth);hold on;&#xA;p1_str='$|U_1(x)=C_1\sin(x)|,\;C_1=2.5545007\cdot 10^2$';&#xA;ph(2)=semilogy(x,abs(W2),'r-','LineWidth',MyLineWidth);hold on;&#xA;p2_str='$|U_2(x)=C_2\sin(x)|,\;C_2=2.5545007\cdot 10^4$';&#xA;ph(3)=semilogy(x,abs(W3),'k-','LineWidth',MyLineWidth);hold on;&#xA;p3_str='$|U_3(x)=C_3\sin(x)|,\;C_3=2.5545007\cdot 10^6$';&#xA;ph(4)=semilogy(x,abs(dWx1-cU1*cos(x)),'bo','LineWidth',MyLineWidth);hold on;&#xA;p4_str='$|\widetilde{dU_1/dx}(x)-C_1\cos(x)|$';&#xA;ph(5)=semilogy(x,abs(dWx2-cU2*cos(x)),'ro','LineWidth',MyLineWidth);hold on;&#xA;p5_str='$|\widetilde{dU_2/dx}(x)-C_2\cos(x)|$';&#xA;ph(6)=semilogy(x,abs(dWx3-cU3*cos(x)),'ko','LineWidth',MyLineWidth);hold on;&#xA;p6_str='$|\widetilde{dU_3/dx}(x)-C_3\cos(x)|$';&#xA;ph(7)=semilogy(x,abs(dWx1p-cU1*cos(x)),'bx','LineWidth',MyLineWidth);hold on;&#xA;p7_str='$|\widetilde{dU_1/dx}(x)-C_1\cos(x)|$,$-$mean($U_1$)';&#xA;ph(8)=semilogy(x,abs(dWx2p-cU2*cos(x)),'rx','LineWidth',MyLineWidth);hold on;&#xA;p8_str='$|\widetilde{dU_2/dx}(x)-C_2\cos(x)|$,$-$mean($U_2$)';&#xA;ph(9)=semilogy(x,abs(dWx3p-cU3*cos(x)),'kx','LineWidth',MyLineWidth);hold on;&#xA;p9_str='$|\widetilde{dU_3/dx}(x)-C_3\cos(x)|$,$-$mean($U_3$)';&#xA;title('$U(x)=C\sin(x)$');    grid on; set(gca,'FontSize',ff-2);&#xA;xlabel('$x$','FontSize',ff);&#xA;ylabel('$|U(x)|, |\widetilde{\frac{dU}{dx}}(x)-\frac{dU}{dx}(x)|$','FontSize',ff);    &#xA;h=legend([ph(1) ph(2) ph(3) ph(4) ph(5) ph(6) ph(7) ph(8) ph(9)],p1_str,p2_str,p3_str,p4_str,p5_str,p6_str,p7_str,p8_str,p9_str,'Location','Best');&#xA;h.FontSize=ff-6;    h.EdgeColor=[1. 1. 1.];&#xA;ylim([1E-17 1E+9]);&#xA;set(gca,'YTick',[1E-15 1E-12 1E-9 1E-6 1E-3 1E-0 1E+3 1E+6 1E+9]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="24110" LastEditDate="2017-04-30T12:10:58.690" LastActivityDate="2017-04-30T12:10:58.690" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26641" PostTypeId="2" ParentId="26619" CreationDate="2017-04-15T13:26:24.817" Score="3" Body="&lt;p&gt;Interesting question. To me this looks like a modified column subset selection problem. The problem concerns the determination of a permutation matrix $P$ so that :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;AP = (A_1 \quad A_2)&#xA;$$&#xA;for the real or complex matrix $A$. Columns of $A_1$ is supposed to be &lt;strong&gt;&lt;em&gt;very&lt;/em&gt;&lt;/strong&gt; linearly independent, while redundant columns of $A$ are stored in $A_2$. You could probably decompose your problem into multiple subset selection problems, like a tree search (don't know how to do this yet). Here are some references on the topic:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Avron, Haim, and Christos Boutsidis. &quot;Faster subset selection for&#xA;  matrices and applications.&quot; SIAM Journal on Matrix Analysis and&#xA;  Applications 34.4 (2013): 1464-1499.&#xA;  &lt;a href=&quot;http://www.boutsidis.org/Boutsidis_SIMAX_13a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.boutsidis.org/Boutsidis_SIMAX_13a.pdf&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Civril, Ali, and Malik Magdon-Ismail. &quot;Column subset selection via&#xA;  sparse approximation of SVD.&quot; Theoretical Computer Science 421 (2012):&#xA;  1-14.&#xA;  &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0304397511009388&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0304397511009388&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Arai, Hiromasa, Crystal Maung, and Haim Schweitzer. &quot;Optimal Column&#xA;  Subset Selection by A-Star Search.&quot; AAAI. 2015.&#xA;  &lt;a href=&quot;https://pdfs.semanticscholar.org/abda/eb770487773d20ee887107569d4e36278972.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pdfs.semanticscholar.org/abda/eb770487773d20ee887107569d4e36278972.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2017-04-15T13:26:24.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26642" PostTypeId="1" CreationDate="2017-04-14T00:44:23.200" Score="2" ViewCount="1247" Body="&lt;p&gt;I'm trying to solve an iterative problem that includes an implicit (backwards) Euler method to find successive time values for a given function.  The numerical problem is shown here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;\frac{u_{r,s+1}-u_{r,s}}{\Delta t}&amp;amp;+u_{r,s}\frac{u_{r+1,s+1}-u_{r-1,s+1}+u_{r+1,s}-u_{r-1,s}}{4\Delta x}\\&#xA;&amp;amp;\quad+\frac{\eta^*_{r+1,s+1}-\eta^*_{r-1,s+1}+\eta_{r+1,s}-\eta_{r-1,s}}{4\Delta x}\\&#xA;&amp;amp;=\frac{1}{3}(\alpha x)^2\frac{u_{r+1,s+1}-2u_{r,s+1}+u_{r-1,s+1}-u_{r+1,s}+2u_{r,s}-u_{r-1,s}}{\Delta x^2 \Delta t}\\&#xA;&amp;amp;\quad+\alpha^2x\frac{u_{r+1,s+1}-u_{r-1,s+1}-u_{r+1,s}+u_{r-1,s}}{\Delta x\Delta t},&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $s$ is the time iteration and $r$ is the location iteration, $x$ is the true location ($x=r\,\Delta x$), and $\alpha$, $\Delta x$, and $\Delta t$ are constants.  The $\eta$ values are already solved, so I need to solve for $u(r,s+1)$.  However, the solution includes values for $u(r-1,s+1)$ and $u(r+1,s+1)$, which requires this to be solved as a system of equations over $r$ for each $s$.  I have boundary conditions of $u$ for all $r$ when $s=1$, as well as u at the first and last $r$ values for all $s$ (i.e. $u(1,s)$ and $u(r_\max,s)$).  I'm not sure what to do from here though to solve for the &quot;interior&quot; $u$ values at $s+1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, the original, non-discretized equation is here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial\overline{u}}{\partial t}+\overline{u}\frac{\partial\overline{u}}{\partial x}+\frac{\partial\eta}{\partial x}=\frac{1}{3}(\alpha x)^2 \frac{\partial^3\overline{u}}{\partial x^2\partial t}+\alpha^2 x\frac{\partial^2 \overline{u}}{\partial x \partial t}.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="23954" OwnerDisplayName="Jonathan Lala" LastEditorUserId="23954" LastEditDate="2017-04-16T19:00:08.810" LastActivityDate="2018-05-04T09:20:02.173" Title="Solving an iterative, implicit Euler method in MATLAB" Tags="|pde|finite-difference|differential-equations|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26644" PostTypeId="2" ParentId="24572" CreationDate="2017-04-16T02:40:28.010" Score="4" Body="&lt;p&gt;I assume you are interested in how the &lt;code&gt;surf&lt;/code&gt; function controls the coloring of the surfaces. Most of the information is readily available at the Matlab help pages for &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/surf.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;surf&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/shading.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;shading&lt;/code&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default, &lt;code&gt;surf&lt;/code&gt; uses &lt;code&gt;faceted&lt;/code&gt; shading, so &lt;em&gt;&quot;each mesh line segment and face has a constant color determined by the color value at the endpoint of the segment or the corner of the face that has the &lt;strong&gt;smallest index&lt;/strong&gt; or &lt;strong&gt;indices&lt;/strong&gt;&quot;&lt;/em&gt;. So, in your code — no explicit interpolation has been performed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By introducing a fine grid and using &lt;code&gt;spline&lt;/code&gt;, you performed interpolation by yourself, and now &lt;code&gt;surf&lt;/code&gt; function for $N_x=1500$ just uses the result you fed it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the picture below, I demonstrated the work of &lt;code&gt;surf&lt;/code&gt; function with different &lt;code&gt;shading&lt;/code&gt; options. It is clear, that &lt;code&gt;surf&lt;/code&gt; with &lt;code&gt;shading interp&lt;/code&gt; for $N_x=150$ (bottom-left) pretty much resembles the figure obtained with a fine ($10\times$) grid $N_x=1500$ (bottom-right). &lt;code&gt;shading flat&lt;/code&gt; and &lt;code&gt;shading faceted&lt;/code&gt; look the same because you manually disabled lines for the surface plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D75Xf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D75Xf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matlab code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sharp_gausian = @(y,sigma) exp((-y.^2).*sigma);&#xA;&#xA;Nz=4001;&#xA;Nx=150;&#xA;Nx_refine=Nx*10;&#xA;z=linspace(-1,1,Nz);&#xA;x=linspace(-2*pi,2*pi,Nx);&#xA;x_fine=linspace(-2*pi,2*pi,Nx_refine);&#xA;&#xA;% initializing low-resolution surf&#xA;psi_grid=zeros(Nz,Nx);&#xA;for j=1:length(z)&#xA;    psi_grid(j,:)=sharp_gausian(x+sin(5*z(j)),10);&#xA;end&#xA;&#xA;% using higher resolution grid and interpolating the function using spline.&#xA;psi_grid2=zeros(Nz,Nx_refine);&#xA;for j=1:length(z)&#xA;    psi_grid2(j,:)=spline(x,psi_grid(j,:),x_fine);&#xA;end&#xA;&#xA;%% ONLY VISUALIZATION AFTER THIS POINT&#xA;set(groot,'DefaultTextInterpreter','latex');&#xA;set(groot,'defaultLegendInterpreter','latex');&#xA;set(groot,'defaultAxesTickLabelInterpreter','latex'); &#xA;ff=20;&#xA;&#xA;figure(1)&#xA;set(gcf,'Units','inches');&#xA;set(gcf,'Position',[2 2 13 12]);&#xA;% original x, flat shading&#xA;subplot(2,2,1)&#xA;s1=surf(x,z, psi_grid);&#xA;shading flat;&#xA;set(s1,'linestyle','none')&#xA;xlabel('$x$','FontSize', ff, 'Interpreter', 'latex');&#xA;ylabel('$z$','FontSize', ff, 'interpreter', 'latex','rot',0);&#xA;title('$N_x = 150$, shading flat');&#xA;set(gca,'FontSize',ff-2);&#xA;% original x, faceted (default) shading&#xA;subplot(2,2,2)&#xA;s2=surf(x,z, psi_grid);&#xA;shading faceted;&#xA;set(s2,'linestyle','none')&#xA;xlabel('$x$' , 'FontSize', ff, 'Interpreter', 'latex');&#xA;ylabel('$z$', 'FontSize', ff, 'interpreter', 'latex','rot',0);&#xA;title('$N_x = 150$, shading faceted');&#xA;set(gca,'FontSize',ff-2);&#xA;% original x, inerpolated shading&#xA;subplot(2,2,3)&#xA;s3=surf(x,z, psi_grid);&#xA;shading interp;&#xA;set(s3,'linestyle','none')&#xA;xlabel('$x$' , 'FontSize', ff, 'Interpreter','latex');&#xA;ylabel('$z$', 'FontSize', ff, 'interpreter','latex','rot',0);&#xA;title('$N_x = 150$, shading interp');&#xA;set(gca,'FontSize',ff-2);&#xA;% 10* refinement in x, faceted (default) shading&#xA;subplot(2,2,4)&#xA;s4=surf(x_fine,z, psi_grid2);&#xA;shading faceted;&#xA;set(s4,'linestyle','none')&#xA;xlabel('$x$','FontSize',ff,'Interpreter','latex');&#xA;ylabel('$z$','FontSize',ff,'interpreter','latex','rot',0);&#xA;title('$N_x = 1500$, shading default (faceted)');&#xA;set(gca,'FontSize',ff-2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-04-16T02:40:28.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26645" PostTypeId="2" ParentId="25885" CreationDate="2017-04-16T06:21:38.477" Score="2" Body="&lt;p&gt;There are two simple solutions to speed up your matrix assembly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Kirill mentioned, you might want to use the following form &lt;code&gt;sparse(i,j,v)&lt;/code&gt;, where &lt;code&gt;i&lt;/code&gt;,&lt;code&gt;j&lt;/code&gt;,and &lt;code&gt;v&lt;/code&gt; are vectors defining &lt;code&gt;K(i(k),j(k)) = v(k)&lt;/code&gt;. That will require you to rewrite a portion of your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach would be to preallocate the sparse matrix storage. What happens, is that when you use &lt;code&gt;K=sparse(NDOF,NDOF)&lt;/code&gt;, no memory is reserved for your matrix $K$, so, every time a new element is added to the sparse matrix - reallocation happens (not entirely true, but pretty close).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to use &lt;code&gt;K = spalloc(NDOF,NDOF,NNZ)&lt;/code&gt; (See, &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/spalloc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matlab help &lt;code&gt;spalloc&lt;/code&gt;&lt;/a&gt;) . That will allocate the storage right from the start and avoid unnecessary reallocations inside the loop. However, you have to find or estimate &lt;code&gt;NNZ&lt;/code&gt; (number of non-zero entries). If you are fine using a little bit extra memory, you can find $\max$ (upper bound — no reallocations will occur) or avarage (safe choice) #non-zero shape functions for every point and use it. Or you can find the exact number of NNZ elements by running your code without evaluation of the shape functions and assigning values to matrix elements.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-04-16T06:21:38.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26646" PostTypeId="1" CreationDate="2017-04-16T11:12:35.887" Score="1" ViewCount="473" Body="&lt;p&gt;I have the elliptic problem $$-\Delta u = 1,\,\,\Omega\subset\mathbb{R}^2$$&#xA;with $u=0$ on $\partial\Omega,$ with $\Omega=[-1,1]^2\backslash([0,1]\times[-1,0])$ and I want to estimate the $L_2$ error of linear finite element method on triangular mesh, if $u_h$ is the finite element solution and $u$ is the exact solution (unknown), &#xA;$$\|u-u_h\|^2_{L_{2}(\Omega)}=\int_{\Omega}|u-u_h|^2\,dx.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ι though to solve this problem with quadratic finite elements and set $u=\tilde{u}_h,$ where $\tilde{u}_h$ is the solution of quadratic finite elements.&lt;/p&gt;&#xA;" OwnerUserId="22334" LastActivityDate="2017-04-16T11:12:35.887" Title="Estimate $L_2$ norm of a elliptic problem with unknown exact solution on finite element method" Tags="|finite-element|matlab|error-estimation|l2-norm|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26647" PostTypeId="1" CreationDate="2017-04-16T12:45:14.193" Score="1" ViewCount="380" Body="&lt;p&gt;I have encountered the term cell &quot;&lt;strong&gt;Cell averaged derivatives&lt;/strong&gt;&quot; and &lt;strong&gt;&quot;Taylor series expansion using cell averaged derivatives about centroid&lt;/strong&gt;&quot; in the context of polynomial representation in any cell (computational element with some higher order interpolation polynomial nodes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the link to the below paper:&#xA;&lt;a href=&quot;http://dept.ku.edu/~cfdku/papers/AIAA-2009-605.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dept.ku.edu/~cfdku/papers/AIAA-2009-605.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SGFow.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SGFow.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also read the reference of this reference paper &#xA;&lt;a href=&quot;http://people.math.gatech.edu/~yingjie/publications/centr_DG_ovlpcell_new_iii.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://people.math.gatech.edu/~yingjie/publications/centr_DG_ovlpcell_new_iii.pdf&lt;/a&gt;&#xA;&lt;em&gt;Section 4.1&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to find alternate references and easier sources of explanation, but everywhere it has been mentioned that the coefficients of the above Taylor series expansion can be found easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am still unable to grasp the procedure properly. It would be nice if someone could help me understand which seems to be trivial for everyone.&lt;/p&gt;&#xA;" OwnerUserId="11977" LastActivityDate="2017-04-16T12:45:14.193" Title="How to perform Taylor series expansion consisting of cell averaged derivatives in a computational element?" Tags="|fluid-dynamics|interpolation|discontinuous-galerkin|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26649" PostTypeId="2" ParentId="26481" CreationDate="2017-04-17T12:04:01.883" Score="2" Body="&lt;p&gt;Following a similar problem answered by Ian Bush , the problem is solved by knowing that complex(16) is not the same as complex*16. In the specific case of the 64bit compiler used complex*16 needed to be translated to complex(8)...or even better use SELECT_REAL_KIND to define a compiler independent definition:&#xA;C_double=SELECTED_REAL_KIND(15,307).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you all for the help and useful remarks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Danny&lt;/p&gt;&#xA;" OwnerUserId="23727" LastActivityDate="2017-04-17T12:04:01.883" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26651" PostTypeId="1" CreationDate="2017-04-17T21:09:44.633" Score="1" ViewCount="602" Body="&lt;p&gt;This question was moved from Cross-Validated: &lt;a href=&quot;https://stats.stackexchange.com/questions/274042/pseudoinverse-of-large-sparse-matrix-in-r&quot;&gt;https://stats.stackexchange.com/questions/274042/pseudoinverse-of-large-sparse-matrix-in-r&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to calculate the pseudoinverse of a large sparse matrix in R using the singular value decomposition. The matrix is roughly 240,000 x 240,000, and I have it stored as type &lt;code&gt;dgCMatrix&lt;/code&gt;. I have tried using &lt;code&gt;pinv&lt;/code&gt;, &lt;code&gt;ginv&lt;/code&gt;, and other standard pseudoinverse functions, but they error out due to memory constraints. I then tried to opt for the sparse matrix svd provided by the package &lt;code&gt;irlba&lt;/code&gt;, which I was then going to use to compute the pseudoinverse using the standard formula after converting all outputs to sparse matrices. My code is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lim = 40&#xA;digits = 4&#xA;SVD =irlba(L,lim)&#xA;tU = round(SVD$u,digits)&#xA;nonZeroU = which(abs(U)&amp;gt;0,arr.ind = T)&#xA;sparseU = sparseMatrix(i=nonZeroU[,2],j=nonZeroU[,1],x = U[nonZeroU])&#xA;V = round(SVD$v,digits)&#xA;nonZeroV = which(abs(V)&amp;gt;0,arr.ind = T)&#xA;sparseV = sparseMatrix(i=nonZeroV[,1],j=nonZeroV[,2],x = U[nonZeroV])&#xA;D = as(Diagonal(x=1/SVD$d),&quot;sparseMatrx&quot;)&#xA;pL =D%*%sparseU&#xA;pL = sparseV%*%pL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am able to get to the last line without an issue, but then I get an error due to memory constraints that says &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in sparseV %*% pL : &#xA;  Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 105&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course I could piece together the pseudoinverse entry by entry using a for loop and vector multiplications, but I would like to be able to calculate it using a simple function that takes advantage of the sparsity of the resultant pseudoinverse matrix. Due to the nature of the original matrix (it is a graph laplacian), I know that the pseudoinverse should also be a sparse matrix. Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="23963" LastActivityDate="2017-04-18T13:20:18.767" Title="Pseudoinverse of a large sparse matrix in r" Tags="|sparse-matrix|svd|r|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26652" PostTypeId="1" AcceptedAnswerId="26683" CreationDate="2017-04-18T07:17:29.263" Score="0" ViewCount="83" Body="&lt;p&gt;I'm learning to use OpenCL to optimize some of my simulations. I realized that I need some sort of Graph-clustering or graph-partitioning to exploit efficiently local memory for un-ordered meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;example:&lt;/strong&gt; I implemented &lt;a href=&quot;https://github.com/ProkopHapala/SimpleSimulationEngine/blob/master/cpp/sketches_OCL/Grids/test_clSpaceFrame2D.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;elastic cloth simulation&lt;/a&gt; with regular mesh of 4-bonded vertexes, which I can separate manually to localized batches. Now I would like to move to general irregular mesh where each node can have any number of edges.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From quick search I found &lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/130840/efficient-graph-clustering-algorithm&quot;&gt;some resources&lt;/a&gt; but I'm not sure if it adress very well my needs (I have no experience in area of discrete math and graph theory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;split mesh to batches such that&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;all batches has approximately same size (number of nodes and edges) e.g. 16 nodes each correspoding to &lt;code&gt;local_size&lt;/code&gt; of my OpenCL kernell &lt;/li&gt;&#xA;&lt;li&gt;the number of edges between different batches is minimal - to minimize overlap between nodes loaded by each workgroup&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Algorithm which is concise and easy to implement by myself - for me this is just side issue not main topic. I do not what to create dependence on some 3rd party software or library even if it is open-source.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it does not have to lead to optimal solution, it can be stochastic and rough heuristic &lt;/li&gt;&#xA;&lt;li&gt;it should be fast - $O(n)$ with small prefactor&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4696" LastActivityDate="2017-04-21T10:03:29.193" Title="simple and fast graph-clustering for paralelization of finite element simulations" Tags="|parallel-computing|graph-theory|unstructured-mesh|partitioning|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26653" PostTypeId="2" ParentId="26642" CreationDate="2017-04-18T09:53:33.187" Score="2" Body="&lt;p&gt;Thanks to TroyHaskin, I've realized that this is in reality a linear problem, since the u(s) values are known from the initial conditions, and is therefore easy to solve in Matlab. Thanks to everyone else for help as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: To be more specific, the system can be solved linearly by separating the u(s+1) terms and their coefficients from everything else. The solution takes the form [Aw,Ap,Ae]u = Q, where u = [u(r-1,s+1),u(r,s+1),u(r+1,s+1)]^T.  Because this is a tridiagonal matrix, it can be solved with minimum storage (note that I have changed location r to i and time s to n in my code, just for personal consistency):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    A_E = zeros(N,1);&#xA;    A_W = zeros(N,1);&#xA;    A_P = zeros(N,1);&#xA;    Q = zeros(N+1,1);&#xA;    Q(1) = u(1,n+1);&#xA;    A_P(1) = 1;&#xA;&#xA;    for i = 2:N&#xA;        A_W(i) = (-u(i,n)/(4*dx))-(((alpha*x(i))^2)/(3*(dx^2)*dt))+((alpha^2)*x(i)/(dx*dt));&#xA;        A_P(i) = (1/dt)+((2*((alpha*x(i))^2))/(3*(dx^2)*dt));&#xA;        A_E(i) = (u(i,n)/(4*dx)) - (((alpha*x(i))^2)/(3*(dx^2)*dt)) - ((alpha^2)*x(i)/(dx*dt));&#xA;        Q(i) = u(i,n)/dt - (u(i,n)/(4*dx))*(u(i+1,n)-u(i-1,n)) ...&#xA;            - (eta(i+1,n+1)-eta(i-1,n+1)+eta(i+1,n)-eta(i-1,n))/(4*dx) ...&#xA;            + (((alpha*x(i))^2)/(3*(dx^2)*dt))*(-u(i+1,n)+(2*u(i,n))-u(i-1,n)) ...&#xA;            + ((alpha^2)*x(i)/(dx*dt))*(-u(i+1,n)+u(i-1,n));&#xA;    end&#xA;&#xA;        %Gaussian Elimination&#xA;        for i = 2:N&#xA;            A_P(i) = A_P(i) - (A_W(i)*A_E(i-1)/A_P(i-1));&#xA;            Q(i) = Q(i) - (A_W(i)*Q(i-1)/A_P(i-1));&#xA;        end&#xA;&#xA;        %Back Substitution&#xA;        for i = N:-1:2&#xA;            u(i,n+1) = (Q(i) - (A_E(i)*u(i+1,n+1)))/A_P(i);&#xA;        end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23954" LastEditorUserId="23954" LastEditDate="2017-04-22T12:34:40.280" LastActivityDate="2017-04-22T12:34:40.280" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26654" PostTypeId="1" CreationDate="2017-04-18T09:55:43.377" Score="2" ViewCount="260" Body="&lt;p&gt;Consider a fluid flow simulation in a pipe. At the outflow, instead of explicitly imposing a boundary condition, I linearly extrapolate information from the interior (for velocity components). This will translate to $$\frac{\partial ^2 u}{\partial x^2} =\frac{\partial ^2 v}{\partial x^2}=\frac{\partial ^2 w}{\partial x^2}= 0$$ at the boundary($x$ being axial direction). This kind of numerical condition has been used successfully throughout the literature (see page 262 of &lt;a href=&quot;https://www.researchgate.net/profile/V_Theofilis/publication/228464691_Advances_in_global_linear_instability_analysis_of_nonparallel_and_three-dimensional_flows/links/00b7d52aac68806e31000000.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am interested is - what does this condition physically mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any discussion would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20670" LastEditorUserId="20670" LastEditDate="2017-04-20T09:42:01.803" LastActivityDate="2017-04-20T09:42:01.803" Title="Outflow boundary condition - second derivative of velocity" Tags="|fluid-dynamics|boundary-conditions|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26658" PostTypeId="2" ParentId="17504" CreationDate="2017-04-19T03:39:12.127" Score="5" Body="&lt;p&gt;Another option if you want to stay with Octave is &lt;a href=&quot;https://www.featool.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEATool Multiphysics&lt;/a&gt; which is a Finite Element FEM GUI simulation toolbox that supports both Octave and Matlab (FEATool is free for personal use with a limited grid size). In your case there are tutorials of coupling fluid flow with heat transfer, for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.featool.com/doc/quickstart.html#ex_he1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.featool.com/doc/quickstart.html#ex_he1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, FEATool also includes easy to use GUI interfaces for external solvers, including for example &lt;a href=&quot;https://www.featool.com/tutorial/2017/06/16/Python-FEM-and-Multiphysics-Simulations-with-Fenics-and-FEATool.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEniCS&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user23977" LastEditorDisplayName="user23977" LastEditDate="2017-08-28T07:02:16.360" LastActivityDate="2017-08-28T07:02:16.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26659" PostTypeId="2" ParentId="8098" CreationDate="2017-04-19T04:52:25.860" Score="2" Body="&lt;p&gt;I found a quite extensive technical report by &lt;a href=&quot;https://web.stanford.edu/group/SOL/reports/SOL-2011-2R.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;David Chin-Lung Fong and Michael Saunders, &quot;CG versus MINRES: an empirical comparison&quot;&lt;/a&gt;. In this report, they say (Sec. 2.2) that for symmetric positive definite matrices, MINRES generates the same iterates as CR, and a large portion of the report is devoted to CR and analyses of its properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The report also mentions several another references for CR - which is in Deutch:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;E. Stiefel, &quot;Relaxationsmethoden bester strategie zur lösung linearer gleichungssysteme,&quot; &lt;em&gt;Comm. Math. Helv.&lt;/em&gt;, 29 (1955), pp. 157–179. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and some of CR properties are discussed and proven in &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0117118&quot; rel=&quot;nofollow noreferrer&quot;&gt;D. G. Luenberger, &quot;Hyperbolic pairs in the method of conjugate gradients,&quot; &lt;em&gt;SIAM J. Appl. Math.&lt;/em&gt;, 17 (1969), pp. 1263–1267.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0707032&quot; rel=&quot;nofollow noreferrer&quot;&gt;D. G. Luenberger, &quot;The conjugate residual method for constrained minimization problems&quot;, &lt;em&gt;SIAM J. Numer. Anal.&lt;/em&gt;, 7 (1970), pp. 390–398.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;where the extension of CR to indefinite systems is also discussed.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-04-19T04:52:25.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26660" PostTypeId="2" ParentId="1346" CreationDate="2017-04-19T10:08:16.213" Score="4" Body="&lt;p&gt;Here is a method known as &lt;em&gt;basis recombination&lt;/em&gt;, which has not been mentioned in the present thread. I'm citing from the book of J.P. Boyd, &quot;Chebyshev and Fourier Spectral Methods&quot;, 2nd Ed., Chapter 6.5.:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the problem&#xA;  $$L u = f$$&#xA;  has inhomogeneous boundary conditions, then it may always be replaced by an equivalent&#xA;  problem with homogeneous boundary conditions, so long as the boundary conditions are&#xA;  linear. The first step is to choose a simple function $B(x)$ which satisfies the inhomogeneous&#xA;  boundary conditions. One may then define a new variable $v(x)$ and new forcing function&#xA;  $g(x)$ via&#xA;  $$u(x) ≡ v(x) + B(x)\\&#xA;g(x) ≡ f(x) − L B(x)$$&#xA;  so that the modified problem is&#xA;  $$L v = g$$&#xA;  where $v(x)$ satisfies homogeneous boundary conditions.&#xA;  ...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The shift function $B(x)$ is arbitrary except for the constraint that it must satisfy all the&#xA;  inhomogeneous boundary conditions. However, the simplest choice is the best: polynomial&#xA;  interpolation of the lowest order that works.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Next comes my own explanation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;Inhomogeneous boundary condition&quot; means a condition which contains a constant, e.g. $$\partial_x u(x,y) |_{x=x_0} = 1.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the above program, by choosing a convenient function $B(x)$, you get that down to&#xA;$$\partial_x u(x,y) |_{x=x_0} = 0.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once you made all boundary conditions homogeneous in this way, you can turn to your basis expansion (which I assume is done in terms of a product basis):&#xA;$$&#xA;u(x,y) \ =\ \sum_{ij} a_{ij} \, \phi_{i}(x) \, \varphi_{j}(y)&#xA;$$&#xA;By applying the corresponding BC operator, one obtains $$&#xA;\partial_x u(x,y) \ =\ \sum_{ij} a_{ij} \, \phi^\prime_{i}(x) \, \varphi_{j}(y) $$ and this should be zero for $x=x_0$ according to the above example.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now comes the &lt;strong&gt;crucial step&lt;/strong&gt;: by using a basis $\phi_i(x)$ which already satisfies the BC by itself, i.e. $\phi^\prime_i(x)|_{x=x_0} = 0$ for all $i$, the BC of the (transformed) two-dimensional problem are &lt;strong&gt;satisfied automatically&lt;/strong&gt;! Basis sets of this and similar kind can be found, e.g., by a procedure called &quot;basis recombination&quot; (that is often used in combination with collocation methods).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Note that this is the point where &lt;em&gt;homogeneous&lt;/em&gt; boundary conditions really matter, because otherwise one would need to impose further constraints. For example, suppose we would be working with the &quot;$=1$&quot; condition above, and, correspondingly, let's try to use a basis with $\phi^\prime_i(x)|_{x=x_0} = 1$. Then&#xA;$$&#xA;\partial_x u(x,y)|_{x=x_0} \ =\ \sum_{ij} a_{ij} \, \varphi_{j}(y)\,&#xA;$$&#xA;and in order to make this expression equal to $1$ for all $y$, one would have to  constrain the expansion coeffcients $a_{ij}$ as well. Thus, for inhomogeneous BCs, there is a general way to apply the constraints to the one-dimensional parts but use it for to the full problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The nice thing about this whole approach is that it is working on a relatively abstract level. Necessary ingredients are only linearity of the BC operator and an ansatz in terms of product basis functions. As such, it is also applicable to approximate methods.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2018-02-19T21:46:53.290" LastActivityDate="2018-02-19T21:46:53.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26662" PostTypeId="2" ParentId="21612" CreationDate="2017-04-19T14:14:30.730" Score="1" Body="&lt;p&gt;As stated by Peter the matrix of the all Neumann boundary Poisson problem is indeed singular and therefore the Matlab Backslash operator, which attempts to use direct solvers, cannot deal with it. Conjugate Gradient on the other hand can, you just need to, as you have, ensure the mean of f equals 0 before handing it to e.g. PCG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to stress that arbitrarily replacing a Neumann boundary with a fixed (arbitrary) valued Dirichelt boundary is not the correct way of dealing with the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To demonstrate this, here is some code for anyone to verify in Matlab or Octave:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear;&#xA;&#xA;n = 64;&#xA;lx = 1.0;&#xA;BCA = 0; %-0.1533;&#xA;&#xA;% problem setup&#xA;dx=lx/(n-1);&#xA;x=0:dx:1;&#xA;&#xA;% set up A's to compute Laplacian on internal points&#xA;nx=n-2;&#xA;e=ones(nx-2,1);&#xA;% left Dirichelt, right Neumann&#xA;AD = - spdiags([1 -2 1; e*[1 -2 1]; 1 -1 1],-1:1,nx,nx)/dx^2;&#xA;% both sides Neumann&#xA;AN = - spdiags([1 -1 1; e*[1 -2 1]; 1 -1 1],-1:1,nx,nx)/dx^2;&#xA;&#xA;% set up RHS for internal points&#xA;b=zeros([nx 1]);&#xA;% b(1:nx)=4*sin(33/pi.*x(2:nx+1));&#xA;b(1:nx)=-2*pi*cos(2*pi.*x(2:nx+1));&#xA;&#xA;% incorporate left Dirichelt BC conditon into RHS&#xA;% e.g. p.2 of https://web.stanford.edu/class/cs205b/lectures/lecture16.pdf&#xA;bD=b;&#xA;bD(1)=-AD(2,1)*BCA; % fix xD(1) to random value BCA by adjusting rhs&#xA;&#xA;% ensure Discrete Compatibility for Neumann case, i.e. mean(bN)=0&#xA;bN=b-mean(b);&#xA;&#xA;% solve both with CG&#xA;xD=AD\bD; %pcg(AD,bD,1e-6,2*n);&#xA;xN=pcg(AN,bN,1e-6,2*n);&#xA;&#xA;% solutions are obtained on internal points only and boundary conditions&#xA;% are incorporated into the RHS. For plotting values need to be appended.&#xA;xD=[ BCA ; xD; xD(end)];&#xA;xN=[xN(1); xN; xN(end)];&#xA;&#xA;% compute first derivatives, frequently of interest for potential problems&#xA;% and also to see the Neumann boundary conditions&#xA;dxDdx=gradient(xD,dx);&#xA;dxNdx=gradient(xN,dx);&#xA;&#xA;% plot solutions&#xA;f=figure(1); set(f,'Position', [200, 300, 720, 310]);&#xA;subplot(1,2,1);&#xA;plot(x,xD,'r-'); hold on&#xA;plot(x,xN,'b--'); hold off&#xA;title('Solutions'); grid on;&#xA;&#xA;subplot(1,2,2);&#xA;plot(x,dxDdx,'r-'); hold on&#xA;plot(x,dxNdx,'b--'); hold off&#xA;title('First Derivatives of Solutions'); grid on;&#xA;% legend('left Dirichelt','both Neumann','location','SouthEast');&#xA;legend('left Dirichelt','both Neumann','location','NorthEast');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will produce the following output:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yERd3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yERd3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might be inclined to think that fixing the left boundary and thus being able to use the Backslash operator has done the job. The agreement between the two is good enough, however for a different rhs the results can differ significantly. Changing the rhs to e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;b(1:nx)=4*sin(33/pi.*x(2:nx+1));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Produces the following output:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9LXts.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9LXts.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here both the solutions and their first derivatives are very different, especially near where the boundary condition of the problem has been changed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many different ways of dealing with singular matrices, but using an arbitrary Dirichlet boundary condition is not one. For the problem at hand it is easiest to switch to an iterative solver like PCG and ensure  Discrete Compatibility.&lt;/p&gt;&#xA;" OwnerUserId="16739" LastEditorUserId="16739" LastEditDate="2017-04-19T18:27:40.830" LastActivityDate="2017-04-19T18:27:40.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26665" PostTypeId="1" CreationDate="2017-04-19T15:56:46.133" Score="0" ViewCount="1118" Body="&lt;p&gt;I am trying to solve a coupled system of ODE's using MATLAB's &lt;code&gt;bvp4c&lt;/code&gt; function. I want to impose the condition that&#xA;$$\int_{0}^{\pi} y_{1}(t) y_{1}(t) dt = 1,$$&#xA;where $y_{1}$ is the first state variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another post suggested adding a additional equation to the system of ODE's. Namely,&#xA;$$I(t) = \int_{0}^{t} y_{1}(s) y_{1}(s) ds \, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the additional boundary condition $I(\pi)-1=0$ would enforce the integral condition I want. However, I am not sure how to code this in MATLAB. Can anyone help me with this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per &lt;a href=&quot;/questions/tagged/nicoguaro&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;nicoguaro&amp;#39;&quot; rel=&quot;tag&quot;&gt;nicoguaro&lt;/a&gt;'s comment, the integral equation can be written as &#xA;$$\frac{d}{dt}I(t) = y_{1}(t) y_{1}(t) \, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Updating the code accordingly, I have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the whole code I am using with &lt;code&gt;bvp4c&lt;/code&gt; (as &lt;a href=&quot;/questions/tagged/kirill&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;kirill&amp;#39;&quot; rel=&quot;tag&quot;&gt;kirill&lt;/a&gt; said, it would be useful to include the whole code):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s = 100;&#xA;solinit = bvpinit(linspace(0,pi,1001),@mat4init,s);&#xA;sol = bvp4c(@odeproblem,@eightbc,solinit);&#xA;&#xA;x = linspace(0,pi,1001);&#xA;y = deval(sol,x);&#xA;trapz(y(1,:).*y(1,:))&#xA;y_bar = y(1,:)./norm(y(1,:),2);&#xA;plot(x,y(1,:))&#xA;&#xA;fprintf('The intial guess for s was %d. The value solved for was %d.\n',s,sol.parameters)&#xA;&#xA;function dydt = odeproblem(t,y,s)&#xA;b = pi;&#xA;dydt(1) = y(2);&#xA;dydt(2) = y(3);&#xA;dydt(3) = y(4);&#xA;dydt(4) = y(5);&#xA;dydt(5) = y(6);&#xA;dydt(6) = y(7);&#xA;dydt(7) = y(8);&#xA;dydt(8) = (-b^4*y(1) + 2*b^2*y(3) - (b^4-2)*y(5) - 2*s^2*b^2*y(7))/s^2;&#xA;dydt(9) = y(1)*y(1);&#xA;end&#xA;&#xA;function res = eightbc(ya,yb,s)&#xA;res = [ya(1); yb(1); ya(3); yb(3); ya(5); yb(5); ya(7); yb(7); ya(9); yb(9)-1];&#xA;end&#xA;&#xA;function yinit = mat4init(x)&#xA;yinit = [  cos(2*x), -2*sin(2*x), -4*cos(2*x), 8*sin(2*x),... &#xA;    16*cos(2*x),  -32*sin(2*x), -64*cos(2*x), 128*sin(2*x), 1];&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is a plot of the solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wk4Xt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wk4Xt.jpg&quot; alt=&quot;Solution from bvp4c&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All other boundary conditions appear to be met (I didn't include the plots showing that). But the integral condition is still not.&lt;/p&gt;&#xA;" OwnerUserId="23984" LastEditorUserId="23984" LastEditDate="2017-04-20T02:22:52.990" LastActivityDate="2020-07-19T18:06:04.517" Title="How to implement an integral condition when solving a BVP in MatLab" Tags="|matlab|ode|boundary-conditions|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="26666" PostTypeId="2" ParentId="19441" CreationDate="2017-04-19T16:32:20.063" Score="1" Body="&lt;p&gt;Some brief comments, since this is an old question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Some results on theoretical examination of eigenspectra of random matrices for neural networks: (&lt;a href=&quot;https://doi.org/10.1103/PhysRevLett.97.188104&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Rajan &amp;amp; Abbott 2006&quot;&gt;Rajan &amp;amp; Abbott 2006&lt;/a&gt;, &lt;a href=&quot;https://doi.org/10.1103/PhysRevE.91.042808&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Muir &amp;amp; Mrsic-Flogel 2015&quot;&gt;Muir &amp;amp; Mrsic-Flogel 2015&lt;/a&gt;). Much of random matrix theory doesn't apply however, since graphs of realistic networks are non-Hermitian and can be quite constrained.&lt;/li&gt;&#xA;&lt;li&gt;In general you can certainly use results from linear systems analysis to examine stability of the network. However, the results will only hold while the system is operating in a linear way — while the activity of any neuron in the active partition is far from saturation, and while the active partition does not change (i.e. no active neuron drops below threshold).&lt;/li&gt;&#xA;&lt;li&gt;However, you can analyse each active partition separately (see &lt;a href=&quot;https://doi.org/10.1016/S0893-6080(98)00012-4&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Hahnloser 1998&quot;&gt;Hahnloser 1998&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Instability will lead to saturation of firing rates, since you are using a saturating transfer function. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9810" LastActivityDate="2017-04-19T16:32:20.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26667" PostTypeId="1" AcceptedAnswerId="26675" CreationDate="2017-04-19T18:02:46.187" Score="2" ViewCount="113" Body="&lt;p&gt;I have a chamber full with a fluid flowing horizontally in laminar regime from one side to the other. It carries a suspension with concentration $c$. This suspension also falls to the bottom of the chamber with settling velocity $\mathbf{v_s}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it reaches a critical concentration $c_{max}$ at the bottom, it starts building up as a deposition. This blocks the influx of particles, hence the function $\phi(c)$, this nonlinear term is similar to the one in the Burgers equation for the traffic problem. Now the region with concentration $c_{max}$ can grow upwards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This deposition is big enough that it can disturb the flow. To account for this, I added the term $\psi(c)\mathbf{u}$ in the Navier-Stokes equations below. This term stops the flow in regions where the concentration is $c_{max}$. The equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial c}{\partial t} = \nabla (\phi(c,c_{max})(\mathbf{v_s}  +  \mathbf{v}))$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac {\partial \mathbf {u} }{\partial t}+(\mathbf {u} \cdot \nabla )\mathbf {u} -\nu \nabla ^{2}\mathbf{u} + \nabla p + \psi(c,c_{max})\mathbf{u}= \mathbf{f}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \cdot \mathbf{u} = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\psi(c, c_{max}) = &#xA;\left\{&#xA;\begin{align}&#xA;0 &amp;amp;&amp;amp; c &amp;lt; c_{init} \\&#xA;10000 (c - c_{init})^3 &amp;amp;&amp;amp; else&#xA;\end{align}&#xA;\right. &#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c_{init}$ is the initial concentration in the entire chamber, it only increases at the bottom, when it starts building up. Therefore $\psi&amp;gt;0$ only at the bottom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not 100% confident with this model, and hence, my question. I believe that the penalization term $\psi$ can restrict my step size. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a better approach to model a flow with a growing deposition that can disturb it? &lt;/li&gt;&#xA;&lt;li&gt;Is it admissible to decouple the equations by treating $\mathbf{u}$ in the first equation and $c$ in the NS equations explicitly?&lt;/li&gt;&#xA;&lt;li&gt;I am solving both equations on the same mesh. So far I haven't done a proper error estimation, but I believe that I would need a high refinement near the deposition layer. Can I model this phenomena with level set methods and save costs in refinement?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have not been able to find literature on similar physics, I would appreciate any reference.&lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2017-04-24T12:02:19.780" Title="Deposition model in laminar flow" Tags="|fluid-dynamics|navier-stokes|advection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26668" PostTypeId="1" AcceptedAnswerId="26696" CreationDate="2017-04-19T19:26:33.563" Score="1" ViewCount="168" Body="&lt;p&gt;I have an optimization problem that I'm trying to cast as a linear program. However, I have an objective function of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{maximize} &amp;amp; a_1 x_1 - a_2 \lvert x_1\rvert\\ \text{subject to} &amp;amp; \color{gray}{\text{(constraints)}}\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried doing the classic linear reformulation using a replacement variable $x_1'$ and adding the constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_1 - x'_1 \le 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-x_1 - x'_1 \le 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem that I ran into is that the solutions seem to be constraining $x_1$ to positive values, even when they obviously should be negative. Looking at the docs I have on this, it seems that this is because the assumption is that all values of it will be maximized or minimized in the same direction and if you break this assumption then you're forced to utilize the M/ILP method with binary determiners in order to solve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't found any discussion on the limitation of using the variables $x_1$ and $x'_1$ in the objective function. Is it possible to do what I want without resorting to ILP?&lt;/p&gt;&#xA;" OwnerUserId="23275" LastEditorUserId="20417" LastEditDate="2017-04-22T14:11:31.367" LastActivityDate="2017-04-23T16:11:23.183" Title="Is it possible to use both the absolute value and the actual value of a variable in a linear objective function?" Tags="|optimization|linear-programming|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26669" PostTypeId="1" AcceptedAnswerId="26671" CreationDate="2017-04-19T20:15:37.893" Score="2" ViewCount="342" Body="&lt;p&gt;How can I estimate the error of a result obtained by using the trapezoidal rule if I don't have the function that describes my problem? The only thing I have is discrete points.&lt;/p&gt;&#xA;" OwnerUserId="23082" LastEditorUserId="127" LastEditDate="2017-04-21T02:41:05.333" LastActivityDate="2017-04-21T02:41:05.333" Title="How to estimate the error of trapezoidal rule using discrete data?" Tags="|quadrature|integration|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26671" PostTypeId="2" ParentId="26669" CreationDate="2017-04-20T07:50:33.920" Score="7" Body="&lt;p&gt;You may use the ideas of error extrapolation as one uses it to construct high-order Runge Kutta methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the function that you interpolate, the interpolation error $I - I_h$, where $I$ is the actual integral value and $I_h$ the value obtained by the piecewise trapezoidal rule, may have a smooth asymptotic expansion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;I - I_h = C_p h^p + C_{p+1}h^{p+1} + ... &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the ODE context &lt;em&gt;smooth&lt;/em&gt; means that the constants $C_p$, $C_{p+1}$, ... are smooth functions of time. I don't know how this translates to integration, but let's assume that they are independent of $h$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, if you have enough samples and they are evenly spaced, you can assume that $h^{p+1} $ and higher order terms are negligible and that you can compute the approximation $I_h$ also for $h\leftarrow h/2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you set up the system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;I - I_h = C_p h^p \\&#xA;I - I_{h/2} = C_p \frac{h^p}{2^p},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which you can solve for $C_p$ and $I$ which gives you the error $I-I_{h}$ up to all the assumptions we made before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: In a former version, I assumed that the piecewise trapezoidal rule is convergent with order $3$. Actually, it is of order $2$. Anyway, I have put down the formulas for an arbitrary convergence order $p$.&lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2017-04-20T16:23:58.717" LastActivityDate="2017-04-20T16:23:58.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26672" PostTypeId="1" CreationDate="2017-04-20T14:10:29.517" Score="0" ViewCount="220" Body="&lt;p&gt;I wanted to know if practically the inverse radon transform operation is considered linear and would be a good candidate for the application of compressed sensing. To my understanding it should be linear since it is just an integral transform, but I have only basic knowledge about compressed sensing and the inverse radon transform. Any knowledge or resources would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="23999" LastActivityDate="2017-04-21T02:32:15.943" Title="Is the inverse radon transform considered a linear operation?" Tags="|image-processing|computer-vision|signal-processing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26674" PostTypeId="2" ParentId="26669" CreationDate="2017-04-20T17:55:47.667" Score="1" Body="&lt;p&gt;There is a graphical version of this method (Richardson extrapolation) that can be very insightful. Use least three values of $h$, not necessarily different by factors of two or any other simple relationship. Plot $I_h$ versus $h^p$ where $p$ is the assumed order of accuracy. The result will be a straight line if your data is so smooth, and the spacing so small, that the asymptotic error estimate is valid (so that higher-order terms are indeed negligible) The intercept at $h=0$ of the best line through your points will be an improved estimate of the exact result. If you do not get a straight line then are you sure about $p$; should you try a different value?; is your spacing small enough? If the data are experimental, are they accurate enough?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not essential that the data be evenly spaced, but the refined data must be in some sense a systematic refinement. If the coarse data points were distributed according to some rule, like geometrical progression, then the finer data points should be distributed according to the same rule. &lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-04-20T17:55:47.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26675" PostTypeId="2" ParentId="26667" CreationDate="2017-04-20T18:50:01.780" Score="2" Body="&lt;p&gt;I will try to answer as best as I can your three questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Your approach is quite classical in that you are considering the particles to be an active scalar. What most people would usually do is to consider that the particle concentration affects the viscosity (via a model such as the Kreiger-Dougherty model for particle suspension) instead of penalizing the velocity via a Darcy term like you are doing. Generally this gives a more appropriate result for the velocity gradient and models the viscous dissipation that occurs at the particle scale. Furthermore, your model does not consider the particle-particle interaction and the diffusive fluxes that occur due to the particle-particle interaction. You might want to look at models like the Phillips model ( see &lt;a href=&quot;http://aip.scitation.org/doi/abs/10.1063/1.858498&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://aip.scitation.org/doi/abs/10.1063/1.858498&lt;/a&gt;) which consider a diffusive flux due to particle-particle and particle-fluid interaction. The authors of that paper also implement a viscosity model instead of a Darcy term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) it is admissible to decouple the equations, but you might want to solve them iteratively inside a single time step. For instance, solve for velocity, then solve concentration, recalculate viscosity and solve again for velocity, etc. until convergence. It is best not to put the concentration and the (velocity,pressure) in the same system from my experience. Be careful about the equation for the particle concentration, as you will most likely need to do some form of upwinding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) You can model this phenomena using concentration values at the nodes (or cell centers) depending on the numerical model you are using. There is no need for multiple meshes or for a level set approach.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2017-04-24T12:02:19.780" LastActivityDate="2017-04-24T12:02:19.780" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26676" PostTypeId="2" ParentId="26668" CreationDate="2017-04-20T19:20:49.123" Score="3" Body="&lt;p&gt;You could split the original problem into two linear programming problems, the first with objective  $f(x_1)=a_1 x_1 -a_2 x_1$ and (additional) constraint $x_1\geq 0$, and the second with objective $f(x_1)=a_1 x_1 + a_2 x_1$ and (additional) constraint $x_1 &amp;lt;0$. Then you would select the best of the two solutions as the solution of the original problem. &lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2017-04-20T19:20:49.123" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26678" PostTypeId="2" ParentId="6738" CreationDate="2017-04-20T23:16:25.497" Score="1" Body="&lt;p&gt;The sparsity sparsity structure helps only in avoiding repeating the fill reducing reordering heuristic and symbolic factorization step for each iteration. Two points to take note of are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;These are graph processing (read pointer jumping or List Processing) intensive which results in low cash hit ratio, and not that easily parallelizable.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Memory requirements tend to be far lower in the reordering phase. There are ways to reduce the memory footprint during the symbolic factoring phase.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In essence, sometimes it is best to run these two phases sequentially or on a small number of cores (4 or 8) in an SMP mode.&lt;/p&gt;&#xA;" OwnerUserId="9394" LastActivityDate="2017-04-20T23:16:25.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26680" PostTypeId="2" ParentId="26672" CreationDate="2017-04-21T02:32:15.943" Score="2" Body="&lt;p&gt;For an operator $R$ to be linear, it has to satisfy two conditions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$R(f+g) = Rf + Rg$ for any two operands $f,g$;&lt;/li&gt;&#xA;&lt;li&gt;$R(\alpha f) = \alpha Rf$ for any operand $f$ and (real or complex) number $\alpha$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is true for the Radon transform, as one easily verifies. Whether compressed sensing can be applied to it is something beyond my realm of knowledge.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-04-21T02:32:15.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26681" PostTypeId="1" AcceptedAnswerId="26706" CreationDate="2017-04-21T05:14:27.703" Score="4" ViewCount="2946" Body="&lt;p&gt;I am a Mechanical Engineering grad student, currently working on a project which will be scaled up in the new future to require quite some processing power. I am using C++ for the code that I have, and later plan on writing a parallel version of the code for running on distributed memory HPC systems. I do not possess a deep knowledge of the theoretical side of computing, and only know the parallel side of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to ask the community for suggestions on good, easy-to-read-and-understand books or non-video internet resources, which would help me start on parallel programming. Preferably a book which works with plenty of sample C++ codes for each concept that it dictates. I can definitely find more advanced books for myself later, but I would like to get started on my project work with minimum inertia, &quot;in parallel&quot; with my learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do let me know if you know of such resources.&lt;/p&gt;&#xA;" OwnerUserId="23565" LastActivityDate="2017-04-24T14:20:05.110" Title="A good, simple book/resource on Parallel Programming in C++ for scientific computing" Tags="|parallel-computing|c++|hpc|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26683" PostTypeId="2" ParentId="26652" CreationDate="2017-04-21T07:34:25.320" Score="1" Body="&lt;p&gt;For a simple (yet not optimal, see below) mesh partitioning algorithm, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1) sort all the cells of the mesh using Hilbert sort&#xA;2) partition the sorted list of cells into chunks of the desired size&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Spatial Hilbert sorting is implemented in my GEOGRAM library [1,2] and in CGAL [3]. It is reasonably easy to implement, using the &lt;code&gt;std::nth_element()&lt;/code&gt; function of the STL. See also [3] for a nice explanation of spatial sorting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is not optimal in the sense that it completely ignores the connections between the chunks and thus does not minimize communication between processes, but I think it is an interesting alternative because it is extremely fast (but it is O(n log(n)) rather than O(n) as requested). It can be implemented in a handful of lines, whereas METIS uses a certain amount of memory and has a processing time that sometimes cancels the gain it has on the communications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To complete this answer, I mention also SCOTCH [4], it has interesting alternative graph partitioning algorithms (but it is not fully satisfactory for you if you do not want to have an external dependency as you said in the question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://alice.loria.fr/software/graphite/doc/html/mesh__reorder_8h.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/graphite/doc/html/mesh__reorder_8h.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://doc.cgal.org/latest/Spatial_sorting/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doc.cgal.org/latest/Spatial_sorting/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;http://www.labri.fr/perso/pelegrin/scotch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.labri.fr/perso/pelegrin/scotch/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-04-21T10:03:29.193" LastActivityDate="2017-04-21T10:03:29.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26685" PostTypeId="1" CreationDate="2017-04-21T11:08:19.243" Score="1" ViewCount="197" Body="&lt;p&gt;I am implementing a Trust-Region method using Dogleg for the search direction defined as $\tau p^U$ for $0 \leq \tau \leq 1$ and $p^U + (\tau -1)(p^B-p^U)$ if $1 \leq \tau \leq 2$. To compute $\tau$ we have to solve the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\| p^U + (\tau - 1)(p^B-p^U) \|^2 = \Delta ^2.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to expand it in order to arrive at an equation of the form $ax^2+bx+c=0$. I obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (\tau -1)^2\|p^B-p^U\|^2 + 2(\tau -1)(p^U,p^B-p^U) + \|p^U\|^2 - \Delta ^2 = 0 .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not not if it is correct or not and how to proceed. Any suggestion?&lt;/p&gt;&#xA;" OwnerUserId="22185" LastEditorUserId="127" LastEditDate="2017-04-22T17:05:48.223" LastActivityDate="2017-04-22T17:05:48.223" Title="Solve scalar quadratic equation" Tags="|optimization|trust-regions|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26686" PostTypeId="1" AcceptedAnswerId="26690" CreationDate="2017-04-21T11:57:11.477" Score="0" ViewCount="89" Body="&lt;p&gt;I am trying to make the jump from a FEM plot of a static load applied to Mindlin laminate plate to a surface visualization of various Mode Shapes of the Laminate Plate. For that, I understand that the two big steps are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;developing a Mass Matrix $[M]$; and&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;getting the eigenvectors and eigenvalues $[V]$ and $[D]$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;While all my textbooks tell me how to do this for a simple Mindlin plate, I don't know how to do it for a laminate Mindlin plate where the very top and bottom layers are of different thickness. All the resources I have found only deal with the most generic of situations. If anyone of you could explain to me what changes (if any) I need to make to the equations, that would be great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After assembling my mass and stiffness matrices I solve the generalized eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;D, V = eig(stiffness(activeDof, activeDof), M_g(activeDof, activeDof)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But then I do not fully understand what I do with the results. Also, I don't get how to represent the modes in graph form. I know that I'm asking a lot, but if you could at least point me toward a good resource.&lt;/p&gt;&#xA;" OwnerUserId="22445" LastEditorUserId="9667" LastEditDate="2017-04-22T01:32:29.373" LastActivityDate="2017-04-22T01:32:29.373" Title="Dynamic Analysis and Visualization of Laminate Mindlin Plate" Tags="|finite-element|visualization|solid-mechanics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26688" PostTypeId="1" CreationDate="2017-04-21T14:52:14.140" Score="1" ViewCount="330" Body="&lt;p&gt;I read  ADER WENO Finite Volume scheme for hyperbolic conservation laws with source term. I want to implement this method in Matlab coding. I got some problems in coding during to computation of &#xA;numerical flux. Please help me the Matlab coding&#xA;$$u_t+f(u)_x=S(u,x,t),\; u(0,x)=u_0(x),\; x\in \mathbb{R}, \;t&amp;gt;0,\;u(t,-\infty)=u(t,\infty)=0.$$&lt;/p&gt;&#xA;" OwnerUserId="24022" LastEditorUserId="24022" LastEditDate="2017-04-25T03:43:59.500" LastActivityDate="2017-04-25T03:43:59.500" Title="Matlab CODE of ADER scheme" Tags="|finite-volume|" AnswerCount="0" CommentCount="5" ClosedDate="2017-04-24T07:41:14.457" ContentLicense="CC BY-SA 3.0" />
  <row Id="26689" PostTypeId="2" ParentId="26668" CreationDate="2017-04-21T17:28:21.290" Score="0" Body="&lt;p&gt;Splitting the variable into positive and negative components seems to have worked for us. I'll include my solution, though there are likely limitations to the scope of the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, for each variable $x_1$, we split the variable into two: $x_{1+}$ and $x_{1-}$. We then added the constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-x_{1+}\le 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{1-}\le 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We were then able to apply our value function for coefficients to each part individually. For instance,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x_{1+},x_{1-}) = a_1 x_{1+} - a_1 x_{1-}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the final solution for the variable $x_1$ being:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_1 = x_{1+} + x_{1-}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After we attempted this and proved it out, we found discussion on the method &lt;a href=&quot;https://www.cs.duke.edu/courses/spring09/cps111/notes/LP.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="23275" LastEditorUserId="23275" LastEditDate="2017-04-21T20:11:52.250" LastActivityDate="2017-04-21T20:11:52.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26690" PostTypeId="2" ParentId="26686" CreationDate="2017-04-21T23:17:04.857" Score="0" Body="&lt;p&gt;Regarding your first question. One option would be to try to use what is &lt;a href=&quot;https://en.wikipedia.org/wiki/Composite_laminate&quot; rel=&quot;nofollow noreferrer&quot;&gt;commonly done in laminates&lt;/a&gt; for Kirchoff-Love plates, I would not know how good is that approximation, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option, is to consider some effective properties and just replace your constitutive tensor in your equations. For example, for the orthotropic case you would have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix} C_{11} &amp;amp; C_{12} &amp;amp; C_{13} \\ C_{12} &amp;amp; C_{22} &amp;amp; C_{23} \\&#xA;                   C_{13} &amp;amp; C_{23} &amp;amp; C_{33} \end{bmatrix}&#xA;   = \frac{1}{1-\nu_{12}\nu_{21}}&#xA;   \begin{bmatrix} E_1 &amp;amp; \nu_{12}E_2 &amp;amp; 0 \\&#xA;                   \nu_{21}E_1 &amp;amp; E_2 &amp;amp; 0 \\&#xA;                   0 &amp;amp; 0 &amp;amp; 2G_{12}(1-\nu_{12}\nu_{21}) \end{bmatrix}&#xA;    \, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and in the case of transverse loading in a Kirchoff plate this would lead to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D_x \frac{\partial^4 w}{\partial x^4} + 2 D_{xy}\frac{\partial^4 w}{\partial x^2 \partial y^2}  + D_y \frac{\partial^4 w}{\partial y^4} = q - \rho h \frac{\partial^2 w}{\partial t^2}$$&#xA;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;     D_x &amp;amp; = \frac{2h^3 E_1}{3(1 - \nu_{12}\nu_{21})} \\&#xA;     D_y &amp;amp; =\frac{2h^3 E_2}{3(1 - \nu_{12}\nu_{21})} \\&#xA;     D_{xy} &amp;amp; = \frac{4h^3 G_{12}}{3} + \frac{2h^3 \nu_{21} E_1}{3(1 - \nu_{12}\nu_{21})} \, ,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for sure it would be more complex in the case of Mindlin-Reissner plates. You could probably check the references suggested below.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Leissa, A. W., &amp;amp; Qatu, M. S. (2011). Vibrations of Continuous Systems.McGraw-Hill.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reddy, J. N. (2004). Mechanics of laminated composite plates and shells: theory and analysis. CRC press.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2017-04-21T23:17:04.857" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26691" PostTypeId="1" CreationDate="2017-04-22T03:30:18.130" Score="1" ViewCount="724" Body="&lt;p&gt;For the equation $\frac{\partial u}{\partial t}=-(-\Delta)u$ with zero boundary condition  using finite element method. Applying the corresponding weak formulation and taking $v=\phi_{j},j=1,2,...,N$, We obtain the following equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum\limits_{i=1}^{N} \frac{du_{i}}{dt}\int _{\Omega} \phi_{i}\phi_{j}d\Omega=-\sum\limits_{i=1}^{N}u_{i} \int _{\Omega} \nabla\phi_{i}.\nabla\phi_{j}d\Omega\,. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introducing the mass $\mathcal {M}_{i,j}=\int _{\Omega} \phi_{i}\phi_{j}d\Omega$ and stiffness matrices $\mathcal{K}_{i,j}=\int _{\Omega} \nabla\phi_{i}.\nabla\phi_{j}d\Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we have &#xA;$$\frac{d\bf{u}}{dt}=-\mathcal{M}^{-1}\mathcal{K}\bf{u}\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so far, we have derived an approximation matrix representation of the Diffusion equation for the finite element method that.is $A=\mathcal{M}^{-1}\mathcal{K}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is why $A$ is not only nonsymmetric, also dense.&#xA;Although both $\mathcal{M}$ and $\mathcal{K}$ are positive-semidefinite and sparse.&lt;/p&gt;&#xA;" OwnerUserId="23763" LastEditorUserId="9667" LastEditDate="2017-04-22T14:16:24.603" LastActivityDate="2017-04-22T14:16:24.603" Title="Why the product of symmetric-sparse matrices is not symmetric, or dense" Tags="|linear-algebra|finite-element|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26692" PostTypeId="2" ParentId="26691" CreationDate="2017-04-22T07:26:36.213" Score="8" Body="&lt;p&gt;You seem to think that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The product of two sparse matrices is sparse;&lt;/li&gt;&#xA;&lt;li&gt;The inverse of a sparse matrix is sparse;&lt;/li&gt;&#xA;&lt;li&gt;The product of two symmetric matrices is symmetric.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;None of these facts is true&lt;/strong&gt;, in general. When it happens, it's the exception, not the rule. Try yourself on some random examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's like believing that $(a+b)^2=a^2+b^2$: that would be nice, and a student could intuitively expect it, but unfortunately it's true only in very special cases.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-04-22T07:26:36.213" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26693" PostTypeId="1" CreationDate="2017-04-21T16:17:57.677" Score="3" ViewCount="555" Body="&lt;p&gt;I'm attempting to solve a convex optimization problem where one of the constraints is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\|M\|_* \leq a$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\|M\|_*$ denotes the nuclear norm of matrix $M$. I'm using &lt;a href=&quot;http://cvxopt.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVXOPT&lt;/a&gt; in Python to solve this as a linear program (LP), but the nuclear norm (sum of singular values) isn't a nice linear constraint. How can I formulate this?&lt;/p&gt;&#xA;" OwnerUserId="24060" OwnerDisplayName="Pterosaur" LastEditorUserId="20417" LastEditDate="2017-04-22T21:52:37.017" LastActivityDate="2017-04-22T21:52:37.017" Title="Translating a nuclear norm constraint to an LMI constraint" Tags="|python|convex-optimization|semidefinite-programming|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26695" PostTypeId="2" ParentId="26693" CreationDate="2017-04-22T18:57:06.193" Score="3" Body="&lt;p&gt;Quoting Michael Grant [MG'14]:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The nuclear norm inequality &lt;span class=&quot;math-container&quot;&gt;$\|X\|_*\leq y$&lt;/span&gt; is satisfied if and only if there exist symmetric matrices &lt;span class=&quot;math-container&quot;&gt;$W_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W_2$&lt;/span&gt; satisfying&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix} W_1 &amp;amp; X \\ X^T &amp;amp; W_2 \end{bmatrix} \succeq 0, ~ \mathop{\textrm{Tr}}W_1 + \mathop{\textrm{Tr}}W_2 \leq 2 y$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here, &lt;span class=&quot;math-container&quot;&gt;$\succeq 0$&lt;/span&gt; should be interpreted to mean that the &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt; block matrix is positive semidefinite. Because of this transformation, you can handle nuclear norm minimization or upper bounds on the nuclear norm in any semidefinite programming setting.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Note that &lt;a href=&quot;http://cvxopt.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVXOPT&lt;/a&gt; has a semidefinite programming (SDP) solver.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;[MG'14] Michael Grant, &lt;a href=&quot;https://math.stackexchange.com/q/704271&quot;&gt;Derivative of the nuclear norm with respect to its argument&lt;/a&gt;, Mathematics Stack Exchange, March 2014.&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-04-22T19:02:22.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26696" PostTypeId="2" ParentId="26668" CreationDate="2017-04-22T20:49:06.180" Score="3" Body="&lt;p&gt;$$\begin{array}{ll} \text{maximize} &amp;amp; a_1 x - a_2 | x |\\ \text{subject to} &amp;amp; \color{gray}{\text{(linear constraints)}}\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a_1, a_2 \in \mathbb R$ are given. Writing as a &lt;em&gt;minimization&lt;/em&gt; problem,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{minimize} &amp;amp;  a_2 | x | - a_1 x\\ \text{subject to} &amp;amp; \color{gray}{\text{(linear constraints)}}\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function to be minimized is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f (x) := a_2 |x| - a_1 x = \begin{cases} -(a_1 + a_2) \, x &amp;amp; \text{if } x \leq 0\\ \,\,\,\,(a_2 - a_1) \, x &amp;amp; \text{if } x \geq 0\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $a_1, a_2 \geq 0$, then $f$ is &lt;em&gt;convex&lt;/em&gt;. In that case, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Epigraph_(mathematics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;epigraph&lt;/a&gt; of $f$ is also convex and can be defined by the &lt;em&gt;intersection&lt;/em&gt; of half-spaces, as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\{ (x,y) \in \mathbb R^2 \mid y \geq (a_2 - a_1) \, x \land y \geq -(a_1 + a_2) \, x \right\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we have the following linear program in $x, y \in \mathbb R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \,\,\,\, y\\ \text{subject to} &amp;amp; \,\,\,\,(a_2 - a_1) \, x - y \leq 0\\ &amp;amp; -(a_1 + a_2) \, x - y \leq 0\\ &amp;amp; \,\,\,\,\,\color{gray}{\text{(linear constraints on } x)}\end{array}$$&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2017-04-23T16:11:23.183" LastActivityDate="2017-04-23T16:11:23.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26697" PostTypeId="2" ParentId="26681" CreationDate="2017-04-23T00:09:15.970" Score="3" Body="&lt;p&gt;Here's a couple of resources for MPI for C language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://mpitutorial.com/tutorials/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mpitutorial.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://moss.csc.ncsu.edu/~mueller/cluster/mpi.guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A User's Guide to MPI&lt;/a&gt; by Peter Pacheco. This is accessible from the page that @BillGreene referenced in his comment, but this is a convenient PDF while that is a rather inconvenient zipped PostScript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think both of these are good resources for beginners.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Regarding C++, it has indeed been deprecated. &lt;a href=&quot;https://blogs.cisco.com/performance/the-mpi-c-bindings-what-happened-and-why&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a discussion. Just stick with the regular C bindings. There's no reason you can't use the regular C calls in a C++ code. If you really want, you can write your own C++ wrapper around the C interface or use &lt;a href=&quot;http://www.boost.org/libs/mpi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boost MPI&lt;/a&gt;, which is a C++ wrapper around MPI.&lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2017-04-23T00:09:15.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26698" PostTypeId="1" CreationDate="2017-04-23T01:12:18.150" Score="3" ViewCount="512" Body="&lt;p&gt;I have a system of parameterized ODEs that I would like to solve using MATLAB and its &lt;code&gt;ode45&lt;/code&gt; solver, and was wondering if it is possible to perform such a task in parallel. For example, I have a simple model of a rocket falling in gravity, where I would like to try different parameter values of &lt;code&gt;tburnstart&lt;/code&gt; and &lt;code&gt;tburnend&lt;/code&gt;, which tell the program when to turn a rocket's engines on and off:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function xdot = equations_of_motion_1D_landing(t, x, G, g, M, Isp, T, tburnstart, tburnend)&#xA;&#xA;    if t &amp;lt; tburnstart&#xA;        xdot = [&#xA;                x(3); %xdot1: x-velocity&#xA;                x(4); %xdot2: y-velocity&#xA;                -(G*M*x(1))/((x(1)^2+x(2)^2)^(3/2)); %xdot3: x-acceleration&#xA;                -(G*M*x(2))/((x(1)^2+x(2)^2)^(3/2)); %xdot4: y-acceleration&#xA;                0 %xdot5: engine mass flow rate&#xA;                ];&#xA;    else if  t &amp;gt;= tburnstart &amp;amp;&amp;amp; t &amp;lt;= tburnend&#xA;        xdot = [&#xA;                x(3); %xdot1: x-velocity&#xA;                x(4); %xdot2: y-velocity&#xA;                -(G*M*x(1))/((x(1)^2+x(2)^2)^(3/2)); %xdot3: x-acceleration&#xA;                -(G*M*x(2))/((x(1)^2+x(2)^2)^(3/2))+(T/x(5)); %xdot4: y-acceleration&#xA;                -T/(g*Isp) %xdot5: engine mass flow rate&#xA;                ]; &#xA;    else&#xA;        xdot = [&#xA;                x(3); %xdot1: x-velocity&#xA;                x(4); %xdot2: y-velocity&#xA;                -(G*M*x(1))/((x(1)^2+x(2)^2)^(3/2)); %xdot3: x-acceleration&#xA;                -(G*M*x(2))/((x(1)^2+x(2)^2)^(3/2)); %xdot4: y-acceleration&#xA;                0 %xdot5: engine mass flow rate&#xA;               ];&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Currently I test different values for &lt;code&gt;tburnstart&lt;/code&gt; and &lt;code&gt;tburnend&lt;/code&gt; using the following code in my main file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%Setting up burn start and end times&#xA;tmin = 0;&#xA;tmax = 200;&#xA;tburnstart = (tmax-tmin).*rand(pop_size,1)+tmin; %burn start times&#xA;tburnend = (tmax-tburnstart).*rand(pop_size,1) + tburnstart + 10; %burn end times (must be larger than start times) (+10 makes sure tspan has more than 1 element)&#xA;pop_init = [tburnstart, tburnend]; %creating intial population&#xA;&#xA;&#xA;for i = 1:length(pop_init)&#xA;    tburnstart = pop_init(i,1);&#xA;    tburnend = pop_init(i,2);&#xA;    %tspan = [0:tburnend];&#xA;    tspan = 0:0.1:tburnend;&#xA;    [t,x] = ode45(@(t,x) equations_of_motion_1D_landing(t, x, G, g, M, Isp, T, tburnstart, tburnend), tspan, xinit);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, using a for loop to evaluate &lt;code&gt;ode45&lt;/code&gt; is extremely slow, and I was wondering if there is a way to evaluate all parameters simultaneously to save on computing time.&lt;/p&gt;&#xA;" OwnerUserId="21148" LastActivityDate="2017-04-23T01:12:18.150" Title="MATLAB: solving multiple ODE systems in parallel" Tags="|matlab|performance|differential-equations|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26701" PostTypeId="1" CreationDate="2017-04-23T07:39:27.490" Score="6" ViewCount="4351" Body="&lt;p&gt;I have a sample of 100 particles with their 3D positions and velocities. I want to decompose the velocity vector field to its &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Helmholtz_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;curl free and divergence free components&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it a feasible to do &lt;a href=&quot;http://faculty.uml.edu/cbaird/95.657(2012)/Helmholtz_Decomposition.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;so&lt;/a&gt; for the sampled data using Python, Matlab or Mathematica? &lt;/p&gt;&#xA;" OwnerUserId="23048" LastEditorUserId="23048" LastEditDate="2018-09-15T15:01:21.443" LastActivityDate="2018-09-15T15:01:21.443" Title="How to compute the Helmholtz decomposition of 2D and 3D vector fields?" Tags="|matlab|mathematica|vector|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="26702" PostTypeId="1" CreationDate="2017-04-23T10:47:21.320" Score="0" ViewCount="482" Body="&lt;p&gt;I want to find an expression for the stability of the nonlinear Poisson equation. I know about von Neumann stability analysis which applies to linear equations as far as I know. Any suggestion how to analyse the stability?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The  equation I am trying to solve is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\frac{\partial ^{2}\phi}{\partial \tau^{2}}+\alpha\frac{1}{r'^{2}}\frac{\partial}{\partial r'}\left(r'^{2}\frac{\partial\phi}{\partial r'}\right) = \left(\frac{\gamma}{\phi^{n+1}}+\beta\right)\,. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my discretisation looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ a\phi_{i-2,j}-2a\phi_{i-1,j}+b\phi_{i,j+1}+ c\phi_{i,j-1}+d= e \phi_{i,j}+\frac{h}{\phi_{i,j}^{n+1}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a, b, c, d, e, h$ are constants, and $i$ and $j$ indices stand for time and radius.&lt;/p&gt;&#xA;" OwnerUserId="24045" LastEditorUserId="9667" LastEditDate="2017-04-24T18:04:00.000" LastActivityDate="2017-04-24T18:04:00.000" Title="Stability of nonlinear partial differential equation" Tags="|pde|numerical-analysis|nonlinear-equations|stability|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26703" PostTypeId="2" ParentId="26681" CreationDate="2017-04-23T13:20:22.033" Score="3" Body="&lt;p&gt;I quite liked &lt;a href=&quot;https://www.amazon.co.uk/Parallel-Scientific-Computing-MPI-Implementation-x/dp/0521520800/ref=sr_1_1?s=books&amp;amp;ie=UTF8&amp;amp;qid=1492953457&amp;amp;sr=1-1&amp;amp;keywords=Parallel%20Programming%20in%20C%2B%2B%20for%20scientific%20computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parallel Scientific Computing in C++ and MPI&lt;/a&gt; when I first used it few years ago.  I've a feeling it's C++ is quite dated now and, as has already been observed, MPI and C++ don't play nicely together any more.&lt;/p&gt;&#xA;" OwnerUserId="1003" LastActivityDate="2017-04-23T13:20:22.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26704" PostTypeId="1" CreationDate="2017-04-23T14:33:31.417" Score="3" ViewCount="726" Body="&lt;p&gt;I am trying to estimate the position of a point $P$ in Matlab. I have $n$ access points (AP) at known positions ($n&amp;gt;2$) as well as the distances to the point $P$ from each AP. These distances all have a distance error. I know how to solve this using multilateration if the circles from the known nodes would intersect, but they don't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distances are computed from RSSI measurements (received signal strength indication) and might be weighted (larger weight if RSSI is high).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See code below. Note that I'm not weighting the RSSI measurements. I'm creating an error function, where $xp$ and $yp$ are the point's unknown coordinates. $x$ and $y$ are known coordinates of the APs. I want to sum all the errors and find the $xp$ and $yp$ that minimize the error function, but not sure how to do this in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    for n=1:N           % N - number of APs&#xA;        d(n)=getDistance(rssi(n),tx,2);&#xA;        err=@(px,py) sqrt((x(n)-px).^2+(y(n)-py).^2)-d(n);&#xA;    end&#xA;    err=sum(err);    % this doesn't work obviously&#xA;    x0 = [xp_true(k),yp_true(k)];&#xA;    out= fminunc(err,x0);&#xA;    xp(k,sample)=out(1);&#xA;    yp(k,sample)=out(2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24047" LastEditorUserId="24047" LastEditDate="2017-04-23T15:27:12.587" LastActivityDate="2017-04-23T18:34:45.480" Title="Position Estimation using 2D multilateration for non-intersecting distances" Tags="|matlab|optimal-control|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26705" PostTypeId="2" ParentId="26701" CreationDate="2017-04-23T14:55:14.563" Score="4" Body="&lt;p&gt;The short answer is &quot;It depends&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certainly, you can try to find the Helmholtz decomposition on your sampled data, and find your irrotational and solenoidal components. However, there are certain requirements on your original vector field you started with. In general, it is that the vector field you are trying to decompose has to be &lt;em&gt;sufficiently smooth&lt;/em&gt; and &lt;em&gt;decay rapidly&lt;/em&gt;. Commonly, that corresponds to the vector field $\mathbf F$ being twice continuously differentiable on some bounded domain $V$, and $\mathbf F$ decay faster than $1/r$, leading to&#xA;$$&#xA;\mathbf F=-\nabla\Phi+\nabla\times\mathbf A&#xA;$$&#xA;decomposition with scalar potential $\Phi$ (for the irrotational component) and vector potential $\mathbf A$ (for the solenoidal component).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you have to know, where your vector field comes from (what physics) to answer &lt;strong&gt;the question of applicability of Helmholtz decomposition in the first place for the continuous problem&lt;/strong&gt;. Then, if positive, you must ensure your discretization (sampling) is sufficient enough to capture the behaviour of the original field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also consider Hodge-Helmholtz decomposition that in addition to irrotational and solenoidal components would allow extracting the harmonic component. A recent paper, &lt;a href=&quot;http://pcmap.unizar.es/~jaca2016/PDFXII/Ahusborde.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;E. Ahusborde &lt;em&gt;et al&lt;/em&gt;, &quot;Discrete Hodge Helmholtz decomposition,&quot; &lt;em&gt;Monografías Matemáticas&lt;/em&gt;, vol. 39, pp. 1–10, 2014&lt;/a&gt; might be useful.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2017-04-23T14:55:14.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26706" PostTypeId="2" ParentId="26681" CreationDate="2017-04-23T16:13:19.083" Score="7" Body="&lt;p&gt;One of the first things that you need to understand about parallel programming is the difference between shared memory multiprocessor computer systems and distributed memory clusters.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A shared memory multiprocessor system is a computer in which several processor cores (which might be on one, two, or more integrated circuits) share the same memory.  From the programmer's point of view, multiple processes created by the programmer can run on the different cores and share access to the memory.  Parallel programs for scientific computing on shared memory systems are typically written using specialized language extensions such as OpenMP (which has C, C++, and Fortran versions.) The OpenMP language extensions make it easy to describe operations on arrays that are to be performed in parallel- the compiler takes care of distributing the work to the multiple processors.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A distributed memory cluster can be thought of as a large collection of computers, each of which has its own memory.  In order to interact, processes running on these computers must explicitly send messages to each other.  Parallel programs for scientific computing on distributed memory clusters are most commonly written using the message passing interface (MPI) library.  Programming with MPI is more difficult than programming with OpenNMP because of the difficulty of deciding how to distribute the work and how processes will communicate by message passing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that a program written using MPI can be run on a shared memory system using a version of the MPI library that simply passes messages between processes through the shared memory.  In that sense, an MPI program is much more flexible- it can run on either type of system.  In comparison, you can't effectively run an OpenMP program on a distributed memory cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shared memory systems are typically limited in the number of processor cores and the amount of storage that can be used.  In practice, you'll seldom find more than 64 processor cores or about 128 gigabytes of RAM in a shared memory system, while distributed memory clusters might have tens of thousands of processor cores and terabytes of memory.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For truly large scale parallel computing you will need to learn MPI.  If you are willing and able to work within the limitations of a smaller shared memory system, than learning OpenMP will be an easier way to get started in parallel computing.  For example, if you're using a desktop computer with 4 or 8 cores and you want to take advantage of those cores, then OpenMP is probably the best way to get started.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other answers have already mentioned some books to get started with MPI.  For OpenMP, I'd recommend that you start with the list of resources at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.openmp.org/resources/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.openmp.org/resources/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2017-04-24T14:20:05.110" LastActivityDate="2017-04-24T14:20:05.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26707" PostTypeId="2" ParentId="26704" CreationDate="2017-04-23T18:27:38.853" Score="3" Body="&lt;p&gt;The problem you describe is an example of a nonlinear least squares problem.&#xA;You can find documentation for the MATLAB function to solve such a problem&#xA;along with several references describing the mathematics and numerical methods&#xA;for solution here: &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/lsqnonlin.html#inputarg_fun&quot; rel=&quot;nofollow noreferrer&quot;&gt;lsqnonlin&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea behind least squares problems is that you have more data points (access point locations and distances in your case) than unknowns (x and y locations of the unknown point in your case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple MATLAB code to solve this problem is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  % function to calculate the error in distance between point and access point&#xA;  % x, y, and d are known vectors of length &amp;gt; 2&#xA;  fun = @(xyp) distResidual(xyp, x, y, d);&#xA;  xy0 = [0 0]; % initial guess at the point's x and y location&#xA;  xy = lsqnonlin(fun, xy0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;x, y&lt;/code&gt; and &lt;code&gt;d&lt;/code&gt; are the known x, y, and distance vectors to the point for the set of access points. The definition of the &lt;code&gt;distResidual&lt;/code&gt; function is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function F=distResidual(xy, x, y, d)&#xA;    F = (x-xy(1)).^2 + (y-xy(2)).^2 - d.^2;&#xA;  end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-04-23T18:34:45.480" LastActivityDate="2017-04-23T18:34:45.480" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26710" PostTypeId="1" AcceptedAnswerId="26711" CreationDate="2017-04-24T00:03:21.510" Score="9" ViewCount="650" Body="&lt;p&gt;I have a dataset that is slowly changing, and I need to keep track of eigenvectors/eigenvalues of its covariance matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been using &lt;code&gt;scipy.linalg.eigh&lt;/code&gt;, but it's too expensive, and it doesn't use the fact that I already have a decomposition that's only slightly incorrect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest a better approach to deal with this problem?&lt;/p&gt;&#xA;" OwnerUserId="18786" LastActivityDate="2017-04-24T09:06:48.953" Title="Eigenvectors of a small norm adjustment" Tags="|linear-algebra|optimization|python|eigenvalues|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26711" PostTypeId="2" ParentId="26710" CreationDate="2017-04-24T05:27:16.757" Score="5" Body="&lt;p&gt;A naive approach is to use the eigenvalue solution of your matrix $A(t)$ as the initial guess of an iterative eigensolver for matrix $A(t  + \delta t)$. You might use QR if you need the full spectrum, or the power method otherwise. This isn't an entirely robust approach, however, as the eigenvalues of a matrix aren't necessarily close to a nearly neighboring matrix &lt;a href=&quot;https://arxiv.org/pdf/1401.0482.pdf&quot; rel=&quot;noreferrer&quot;&gt;(1)&lt;/a&gt;, especially if its poorly conditioned &lt;a href=&quot;http://wmatem.eis.uva.es/~marsan/publications/preprints/ident3/node7.html&quot; rel=&quot;noreferrer&quot;&gt;(2)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A subspace tracking method is apparently more useful &lt;a href=&quot;http://www.ee.ucr.edu/~yhua/HuaXiangChenAbedMiao1999.pdf&quot; rel=&quot;noreferrer&quot;&gt;(3)&lt;/a&gt;. An excerpt from &lt;a href=&quot;http://www.springer.com/us/book/9789811029134&quot; rel=&quot;noreferrer&quot;&gt;(4)&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The iterative computation of an extreme (maximal or minimum) eigen&#xA;  pair (eigenvalue and eigenvector) can date back to 1966 [72]. In 1980,&#xA;  Thompson proposed a LMS-type adaptive algorithm for estimating&#xA;  eigenvector, which correspond to the smallest eigenvalue of sample&#xA;  covariance matrix, and provided the adaptive tracking algorithm of the&#xA;  angle/frequency combing with Pisarenko’s harmonic estimator [14].&#xA;  Sarkar et al. [73] used the conjugate gradient algorithm to track the&#xA;  variation of the extreme eigenvector which corresponds to the smallest&#xA;  eigenvalue of the covariance matrix of the slowly changing signal and&#xA;  proved its much faster convergence than Thompson’s LMS-type algorithm.&#xA;  These methods were only used to track single extreme value and&#xA;  eigenvector with limited application, but later they were extended for&#xA;  the eigen-subspace tracking and updating methods. In 1990, Comon and&#xA;  Golub [6] proposed the Lanczos method for tracking the extreme&#xA;  singular value and singular vector, which is a common method designed&#xA;  originally for determining some big and sparse symmetrical eigen&#xA;  problem $Ax = kx$ [74].&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[6]: Comon, P., &amp;amp; Golub, G. H. (1990). Tracking a few extreme singular values and vectors in signal processing. In Processing of the IEEE (pp. 1327–1343).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[14]: Thompson, P. A. (1980). An adaptive spectral analysis technique&#xA;  for unbiased frequency &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[72]: Bradbury, W. W., &amp;amp; Fletcher, R. (1966).&#xA;  New iterative methods for solutions of the eigenproblem. Numerical&#xA;  Mathematics, 9(9), 259–266. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[73]: Sarkar, T. K., Dianat, S. A., Chen,&#xA;  H., &amp;amp; Brule, J. D. (1986). Adaptive spectral estimation by the&#xA;  conjugate gradient method. IEEE Transactions on Acoustic, Speech, and&#xA;  Signal Processing, 34(2), 272–284.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[74]: Golub, G. H., &amp;amp; Van Load, C. F. (1989). Matrix computation (2nd ed.). Baltimore: The John Hopkins University Press.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I should also mention that solutions to symmetric matrices, such as what you must be solving given your use of &lt;code&gt;scipy.linalg.eigh&lt;/code&gt;, are somewhat cheap. If you are only interested in a few eigenvalues, you might find speed improvements in your method as well. The Arnoldi method is often used in such situations.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastEditorDisplayName="user20857" LastEditDate="2017-04-24T05:32:43.493" LastActivityDate="2017-04-24T05:32:43.493" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26712" PostTypeId="2" ParentId="26710" CreationDate="2017-04-24T09:00:13.140" Score="7" Body="&lt;p&gt;There exist special techniques for updating the eigen-decomposition of time-dependent covariance matrices. Given a &quot;prior&quot; eigenvalue decomposition (say at some initial time $t^0$), these recursive algorithms lower the complexity of the spectrum update from $\mathcal{O}(N^3)$ (essentially the cost of a new eigendecomposition) to $\mathcal{O}(k N^2)$ where $N$ is the size of your matrix and $k$ is the rank of your update.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a couple of relevant references:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ece.mcgill.ca/~bchamp/Papers/Jounal/IEEE-SP94.pdf&quot; rel=&quot;noreferrer&quot;&gt;Adaptive Eigendecomposition of Data Covariance Matrices Based on First-Order Perturbations &#xA;&lt;/a&gt; (Champagne, IEEE TSP 42(10) 1994)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=80968&quot; rel=&quot;noreferrer&quot;&gt;Recursive updating the eigenvalue decomposition of a covariance matrix&lt;/a&gt; (Yu, IEEE TSP, 39(5) 1991)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1511.03688.pdf&quot; rel=&quot;noreferrer&quot;&gt;Online Principal Component Analysis in High Dimension:&#xA;Which Algorithm to Choose?&lt;/a&gt; (Cardot and Degras)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.yale.edu/publications/techreports/tr966.pdf&quot; rel=&quot;noreferrer&quot;&gt;A Stable And Fast Algorithm For Updating The Singular Value Decomposition&lt;/a&gt; (Gu and Eisenstadt, 1994)&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-04-24T09:06:48.953" LastActivityDate="2017-04-24T09:06:48.953" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26714" PostTypeId="1" CreationDate="2017-04-24T22:15:59.527" Score="1" ViewCount="107" Body="&lt;p&gt;I'm currently trying to get into this topic.  I've learned that the basic scheme for the advection problem ($D_{x}u+a*D_{t}u=0$) can be solved in a scheme like $$ M^{k}\frac{d}{dt}u^{k}_{h}-(S^{k})^{T}au^{k}_{h}=-(au_{h})^{*}\psi(x^{k}_{r})+(au_{h})^{*}\psi(x^{k}_{l})$$Here, we have $$ M^{k}_{ij}=(\psi_{i},\psi_{j}),  S^{k}_{ij}=(\psi_{i},\frac{d\psi_{j}}{dx})$$ $\psi_{i}$ are the basis functions.&lt;br/&gt; I think I understand this scheme but I'm not quite how to deal with the case of an inhomogeneous PDE. &lt;br/&gt; How has the scheme to be modified if I've got $f(x,t)$ instead of 0 above?&lt;/p&gt;&#xA;" OwnerUserId="24061" LastActivityDate="2017-04-24T22:15:59.527" Title="Discontinuos Galerkin Method - inhomogeneous advection problem" Tags="|pde|boundary-conditions|discontinuous-galerkin|advection|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26715" PostTypeId="1" CreationDate="2017-04-25T08:41:37.163" Score="1" ViewCount="168" Body="&lt;p&gt;We are using Kitware VTK to visualize our models. When we display streamlines, it sometimes happens that a single streamline is left out. This can be especially seen when duplicating a model several times, so that there is a model that has several components that look alike with the same distance between each other. Usually in one of the components close to the middle, a single streamline is missing. I suspect that it is due to an accumulated error in the integration VTK performs. Can it be solved in some way, e.g. by changing the Runge-Kutta method?&lt;/p&gt;&#xA;" OwnerUserId="24064" LastActivityDate="2017-04-25T09:11:53.503" Title="VTK: Missing Streamlines due to error in Runge-Kutta method?" Tags="|ode|runge-kutta|visualization|vtk|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26716" PostTypeId="2" ParentId="26715" CreationDate="2017-04-25T09:11:53.503" Score="1" Body="&lt;p&gt;Are you using a program that implements VTK (i.e. ParaView), or are you implementing the RK method yourself? I know that in the ParaView GUI, it's possible to adjust the relative error between the two RK methods, assuming it's some kind of adaptive method like 4/5 Cash-Karp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are implementing your own RK method, I would adjust the error or tolerance. If you are using VTK directly, but not through a GUI like ParaView, you should investigate if the RK method has some API that addresses this issue. For very sensitive systems, adaptive RK methods are very dependent on this tolerance/error value.&lt;/p&gt;&#xA;" OwnerUserId="19791" LastActivityDate="2017-04-25T09:11:53.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26717" PostTypeId="1" AcceptedAnswerId="33846" CreationDate="2017-04-25T13:20:03.507" Score="4" ViewCount="214" Body="&lt;p&gt;Given $\omega$ I want to compute the vector $q=(u,v)$ such that&#xA;$$&#xA;v_x - u_y = \omega, \qquad u_x + v_y = 0, \qquad \textrm{in } \Omega&#xA;$$&#xA;and&#xA;$$&#xA;u n_x + v n_y = 0, \qquad \textrm{on } \partial\Omega&#xA;$$&#xA;I am looking for a finite element method to find such a vector field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can introduce a stream function $\psi$ such that&#xA;$$&#xA;\Delta\psi = -\omega&#xA;$$&#xA;and then $(u,v) = (\psi_y, -\psi_x)$. Solving this numerically I would obtain $\psi^h \in P_k$ but then $(u^h, v^h) = (\psi_y^h, -\psi_x^h)$ will not be globally divergence-free, i.e., normal component of velocity will not be continuous across the elements when using $C^0$ elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I want to obtain a globally divergence-free vector field with specified curl.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Projection to Raviart-Thomas space&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I obtain $\psi^h \in P_k$ and then project to $RT_k$: find $q^h \in RT_k$ such that&#xA;$$&#xA;\int_\Omega q^h \cdot w dx = \int_\Omega (\psi^h_y, -\psi^h_x) \cdot w dx, \qquad \forall w \in RT_k&#xA;$$&#xA;I can check that the resulting $q^h$ indeed has zero divergence and converges with error $O(h^k)$ which seems less than optimal. I don't know how to prove these things but I guess this must already be published somewhere. I am looking for some reference on this sort of projection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am also interested to know if we can compute $q$ without using a stream-function.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2017-04-26T08:56:49.063" LastActivityDate="2019-11-24T06:40:27.873" Title="Compute 2d velocity field with given curl and zero divergence" Tags="|vorticity|divergence-free|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26718" PostTypeId="1" AcceptedAnswerId="26720" CreationDate="2017-04-25T13:47:14.393" Score="5" ViewCount="332" Body="&lt;p&gt;I'm trying to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}\left\{&#xA;\begin{split}&#xA;\frac{\partial u}{\partial t}+(u\cdot\nabla)u-\nu\Delta u+\frac1\rho\nabla p&amp;amp;=f\;\;\;\text{in }\Lambda\\&#xA;u&amp;amp;=0\;\;\;\text{on }\partial\Lambda\\&#xA;\nabla\cdot u&amp;amp;=0\;\;\;\text{in }\Lambda&#xA;\end{split}\right.\tag1&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on $\Lambda=(0,a)\times(0,b)$. Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{split}\mathfrak a(u,v)&amp;amp;:=\sum_{i=1}^d\langle\nabla u_i,\nabla v_i\rangle_{L^2}\\&#xA;\mathfrak b(u,q)&amp;amp;:=-\langle\nabla\cdot u,q\rangle_{L^2}\\&#xA;\mathfrak c(u,v,w)&amp;amp;:=\langle(u\cdot\nabla)v,\cdot w\rangle_{L^2}&#xA;\end{split}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $u,v\in V:=H_0^1(\Lambda,\mathbb R^2)$ and $q\in\Pi:=\left\{p\in L^2(\Lambda):\int p=0\right\}$. Now, let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{split}\tilde{\mathfrak a}(u,v)&amp;amp;:=\langle u,v\rangle_{L^2}+\Delta t\nu\mathfrak a(u,v)+\Delta t\mathfrak c(u^0,u,v)\\&#xA;\tilde{\mathfrak b}(v,q)&amp;amp;:=\frac{\Delta t}\rho\mathfrak b(v,q)&#xA;\end{split}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $u,v\in V$ and $q\in\Pi$. $u^0$ is the approximate solution of the previous time step and $\Delta t$ is the elappsed time. I've chosen a Taylor-Hood pair for the finite element discretization. Now, I'm left with a system $$\left(\begin{matrix}A&amp;amp;B^T\\B&amp;amp;0\end{matrix}\right)\left(\begin{matrix}u\\p\end{matrix}\right)=\left(\begin{matrix}f\\0\end{matrix}\right)\tag2$$ where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{split}&#xA;a_{ij}&amp;amp;=\tilde{\mathfrak a}(\phi_i,\phi_l)\\&#xA;b_{ij}&amp;amp;=\tilde{\mathfrak b}(\phi_j,\psi_i)&#xA;\end{split}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $(\phi_i)$ and $(\psi_i)$ are bases of the finite dimensional subspaces of $V$ and $\Pi$, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea was to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\left\{\begin{split}&#xA;BA^{-1}B^Tp&amp;amp;=BA^{-1}f\\&#xA;Au&amp;amp;=f-B^Tp\;.&#xA;\end{split}\right.\tag3&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I've got several problems with $(3)$:&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;I'm trying to solve the linear equations using the GMRES method. As a first step, the right-hand side of the first equation, $BA^{-1}f$, has to be computed. I've assembled $A$ and $B$; hence I'm trying to find a solution $Ax=f$ (using the GMRES method) and then compute $Bx$. However, the GMRES method for $Ax=f$ converges extremely slowly (most probably due to the undesirable eigenvalue distribution of $A$). Can we accelerate the convergence by a preconditioner? If so, which one should I use?&lt;/li&gt;&#xA;  &lt;li&gt;The same question applies to the invocation of the GMRES method for $BA^{-1}B^Tp=BA^{-1}f$. Which preconditioner can I use here?&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="23411" LastActivityDate="2017-04-25T21:36:52.733" Title="Preconditioner for the GMRES method in the Uzawa algorithm" Tags="|finite-element|fluid-dynamics|iterative-method|preconditioning|gmres|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26719" PostTypeId="1" CreationDate="2017-04-25T15:35:10.730" Score="1" ViewCount="106" Body="&lt;p&gt;So, as the title of the post implies, I'm making an FEM code for a composite laminate plate using the Mindlin plate theory. And, as I am trying to get the mode shapes for the plate, I needed to make a mass matrix. Following some other examples and some past code someone gave me, I managed to make one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem is, when I change the number of elements, like for example, from 4x4 to 5x5 elements, some of the mode shapes, the 4th one specifically, flips upside down. And that is the case when I change to number of elements. Sometimes the 1st and 2nd mode shapes are flipped. Sometimes all of them are. But I still can't find what is wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be causing this? Is there some checklist of simple things I should check to correct my mass matrix? If there is an issue you think that might be a problem, please be as specific as possible in coding terms. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="22445" LastActivityDate="2017-04-25T15:35:10.730" Title="Composite Laminate Mass Matrix Help" Tags="|finite-element|matlab|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26720" PostTypeId="2" ParentId="26718" CreationDate="2017-04-25T16:13:44.330" Score="3" Body="&lt;p&gt;Please check &lt;a href=&quot;http://page.math.tu-berlin.de/~liesen/Publicat/BenGolLie05.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper by Benzi et al. They address this issue and give corresponding references on p. 45.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shortcut: for the Stokes problem $A = \text{diag}(A_{11},A_{11},\dots,A_{11})$ is just a collection of discrete Laplace operators, so it is natural to approximate their inverses using multigrid. However, things get much more complicated for Oseen type problems as in your case (especially if convection is high, $\nu \ll 1$); Uzawa converges rather slowly in this case. They give references to papers which propose preconditioning techniques for Uzawa.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond Uzawa: it may be useful to look at different solving techniques, e.g. block preconditioners such as PCD preconditioner by Kay et al. or AL preconditioner by Benzi and Olshanskii. &#xA;A nice overview is &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.373.2908&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;given&lt;/a&gt; by Rehman et al. A recent step of deal.II tutorial &lt;a href=&quot;http://www.dealii.org/developer/doxygen/deal.II/step_57.html#TheSolverandPreconditioner&quot; rel=&quot;nofollow noreferrer&quot;&gt;implements&lt;/a&gt; AL approach (yet they use a direct solver for $A$ for simplicity of implementation).&lt;/p&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-04-25T16:13:44.330" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26721" PostTypeId="2" ParentId="26718" CreationDate="2017-04-25T21:36:52.733" Score="1" Body="&lt;p&gt;As already pointed out in the other answer, deal.II has a number of tutorial programs that deal with exactly this problem. In particular, &#xA;&lt;a href=&quot;https://www.dealii.org/8.5.0/doxygen/deal.II/step_22.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;step-22&lt;/a&gt; uses essentially the approach you want to use, but concludes that this is among the least efficient ways to do things. In the appendix it points out a better way to do it, and provides a bit of code as well. It also has links to other tutorials such as step-57 that describe better and more efficient approaches.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-04-25T21:36:52.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26724" PostTypeId="1" AcceptedAnswerId="26726" CreationDate="2017-04-26T15:02:46.040" Score="2" ViewCount="447" Body="&lt;p&gt;I am modelling a soil medium in plane strain and would like to determine if paraview supports the 15-noded triangular element as shown on &lt;a href=&quot;https://dianafea.com/manuals/d944/ElmLib/node149.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the documentation I am only seeing elements up to second order for an unstructured grid (please see &lt;a href=&quot;http://www.vtk.org/vtk-users-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VTK users guide&lt;/a&gt; pages 480 and 481. I also found some indication on this link that other element types might be supported, but nothing specifically regarding the element type I am after.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore the model I am using has additional stress points (9 of them) in addition to the geometric nodes on the model. I am not terribly worried about these for the time being although I will eventually have to include them in the rendition.&lt;/p&gt;&#xA;" OwnerUserId="22736" LastEditorUserId="9667" LastEditDate="2017-04-26T19:10:12.147" LastActivityDate="2017-04-26T19:10:12.147" Title="15 noded triangular elements in paraview" Tags="|visualization|paraview|vtk|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26725" PostTypeId="1" CreationDate="2017-04-26T16:02:10.857" Score="7" ViewCount="673" Body="&lt;p&gt;I wrote a simple 2D wave simulation using the following equations:&#xA;$$\frac{\partial^2 u}{\partial t^2}=c^2\nabla^2u$$&#xA;Where $\nabla^2$ is the discrete laplace operator using a Von Neumann neighborhood lattice.&lt;br&gt;&#xA;$$\mathbf {D}^2_{xy} = \begin{bmatrix}&#xA;    0       &amp;amp; 1 &amp;amp; 0 \\&#xA;    1       &amp;amp; -4 &amp;amp; 1 \\&#xA;    0       &amp;amp; 1 &amp;amp; 0 &#xA;\end{bmatrix}$$&#xA;This gives the discrete formula:&#xA;$$\Delta{}^t_{t+1}u_{x,y} - \Delta^{t}_{t-1}u_{x,y} = c^2(u_{x+1,y,t}+u_{x-1,y,t}+u_{x,y+1,t}+u_{x,y-1,t}-4u_{x,y,t})$$&#xA;Rearranging the terms gives the solution for the next timestep:&#xA;$$u_{x,y,t+1} = c^2(u_{x+1,y,t}+u_{x-1,y,t}+u_{x,y+1,t}+u_{x,y-1,t}-4u_{x,y,t}) + \Delta^{t}_{t-1}u_{x,y} + u_{x,y,t}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to work well, but I am currently taking $u=0$ when out of boundary. (which is apparently called the Dirichlet condition)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{m\times n}\\ &#xA;    \begin{cases}&#xA;      0, &amp;amp; \text{if}\ (i&amp;lt;0) \lor (j&amp;lt;0) \lor (i\geq m) \lor (j\geq n) \\&#xA;      A_{ij}, &amp;amp; \text{otherwise}&#xA;    \end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But according to the wave equation, that will make the boundaries act like walls, which waves can bounce on, like this:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aRGJV.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aRGJV.gif&quot; alt=&quot;Animated Example&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;Animated Example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I cannot figure out how to make the boundaries &quot;&lt;strong&gt;pass through&lt;/strong&gt;&quot; as if the grid were infinite. Other simulators do it, but I could not figure out the equations for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kpUw8.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kpUw8.gif&quot; alt=&quot;Wavelet&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;Example of a simulation that has absorbing boundaries, the waves do not reflect on the borders.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simple method that I can apply to my currently used equation? I don't need anything fancy, only something simple to apply and fast to compute.&lt;/p&gt;&#xA;" OwnerUserId="24083" LastEditorUserId="24083" LastEditDate="2017-04-27T13:10:41.933" LastActivityDate="2018-02-26T02:08:28.147" Title="Discrete wave simulation - absorbing boundaries?" Tags="|boundary-conditions|discretization|numerical-modelling|wave-propagation|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="26726" PostTypeId="2" ParentId="26724" CreationDate="2017-04-26T16:04:33.030" Score="3" Body="&lt;p&gt;Unfortunately (unless things have changed very recently) VTK doesn't fully support anything higher than second order cells in terms of mesh visualisation and filtering. Various common workarounds based on upscaling have been attempted, see for example &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7468/fenics-visualizing-high-order-elements&quot;&gt;this related question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2017-04-26T16:04:33.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26727" PostTypeId="1" CreationDate="2017-04-26T21:53:21.820" Score="1" ViewCount="123" Body="&lt;p&gt;I have to write a solver for 2D equation:&#xA;$$\partial_t u = u^2(\partial_x ^2 u + \partial_y ^2 u)$$&#xA;I try to use explicit method:&#xA;$$\partial_t u = \frac{u_{i,j}^{k+1} - u_{i,j}^k}{\tau}$$ and $$\partial_x ^2 u = \frac{u_{i-1,j}^k - 2u_{i,j}^k + u_{i+1,j}^k}{h^2}$$ and tha same for $\partial_y ^2 u$ with points $j-1, j, j+1$. Also I use: $$ u_{i,j}^k = \frac{u_{i+1,j}^k + u_{i,j}^k}{2}$$ for the coefficient before $\partial_x ^2$ and the same for $y$ with points $j+1, j$.&#xA;I wrote the following code in Mathematica:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nn = 6;&#xA;mm = 200;&#xA;h = 1./nn;&#xA;tau = 1./mm;&#xA;v0[x_, y_] := 1.;&#xA;&#xA;&#xA;Table[{x[i] = i*h, y[j] = j*h}, {i, 0, nn}, {j, 0, nn}];&#xA;Table[t[i] = i*t, {i, 0, mm}];&#xA;Table[u[i, j, 0] = v0[x[i], y[j]], {i, 0, nn}, {j, 0, nn}];&#xA;Table[u[0, j, k] = 0, {j, 0, nn - 1}, {k, 0, mm}];&#xA;Table[u[nn, j, k] = 0, {j, 0, nn - 1}, {k, 0, mm}];&#xA;Table[u[i, 0, k] = 0, {i, 0, nn - 1}, {k, 0, mm}];&#xA;Table[u[i, nn, k] = 0, {i, 0, nn - 1}, {k, 0, mm}];&#xA;&#xA;Do[ &#xA;  u[i, j, k + 1] = &#xA;   u[i, j, k] + &#xA;    tau*(((u[i - 1, j, k] - 2*u[i, j, k] + u[i + 1, j, k])/&#xA;          h^2)*(( u[i, j, k] + u[i - 1, j, k])/&#xA;           2)^2 + ((u[i, j - 1, k] - 2*u[i, j, k] + u[i, j + 1, k])/&#xA;          h^2)*((u[i, j, k] + u[i, j - 1, k])/2)^2), {k, 0, mm}, {i, &#xA;   1, nn - 1}, {j, 1, nn - 1}];&#xA;&#xA;&#xA;&#xA;g[k_] := Table[u[i, j, k], {i, 0, nn}, {j, 0, nn}];&#xA;p = Table[&#xA;  ListPlot3D[g[k], PlotRange -&amp;gt; {{0, nn}, {0, nn}, {0, 6}}], {k, 0, &#xA;   mm, 20}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I use the simpliest conditions:&#xA;$u(0,0,t) = u(10,10,t) = 0$ and $u(x,y,0) = 1$&lt;/p&gt;&#xA;" OwnerUserId="23846" LastEditorUserId="23846" LastEditDate="2017-05-15T19:20:02.793" LastActivityDate="2017-05-15T19:20:02.793" Title="Nonlinear 2D thermoconductivity equation(numerical solution)" Tags="|nonlinear-equations|mathematica|differential-equations|" AnswerCount="0" CommentCount="11" ClosedDate="2017-05-20T21:26:46.697" ContentLicense="CC BY-SA 3.0" />
  <row Id="26728" PostTypeId="2" ParentId="20641" CreationDate="2017-04-27T03:34:35.370" Score="0" Body="&lt;p&gt;You can use the Grimme's suggested functional instead of B3LYP to correct the the &lt;strong&gt;interaction&lt;/strong&gt; between molecules. In Gaussian program, have a look at empiricaldispersion keyword. Read more details &lt;a href=&quot;http://gaussian.com/dft/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22108" LastActivityDate="2017-04-27T03:34:35.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26729" PostTypeId="1" CreationDate="2017-04-27T06:51:00.807" Score="4" ViewCount="534" Body="&lt;p&gt;Given two (here and below: single-precision, IEEE 32-bit floats) normalized floating-point numbers $x, y$ (perhaps of reasonable range: my counterexamples don't have unusual magnitudes), and another floating-point number $0\leq t\leq 1$, I would like to evaluate the linear interpolant&#xA;$$ z_t = x(1-t) + yt $$&#xA;in such a way that:&#xA;$$ \begin{gathered}&#xA;x \leq z_t \leq y,\\&#xA;z_0 = x, \qquad z_1 = y,&#xA;\end{gathered} $$&#xA;as well as&#xA;$$ &#xA;z_t \leq z_s \quad\text{whenever}\quad 0\leq t \leq s \leq 1. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The direct formula, $z_t = x(1-t)+yt$ fails. For example, with $x=2.0000076$, $y=2.9007149$, we have in single-precision arithmetic&#xA;$$ \mathrm{fl}(z_t) &amp;lt; x, \qquad \text{for}\quad2.9802326\times 10^{-8}\leq t&amp;lt;4.109652\times 10^{-8}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The obvious other formula $z_t = x + (y-x)t$ satisfies $z_t \geq x$, but sometimes $z_1 &amp;gt; y$ for&#xA;$$ x=1.0000609, \quad y=3.9999998, \quad z_1 = 4.0. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I considered&#xA;$$ z_t = \mathrm{fma}(x, 1-t, y t), $$&#xA;and it has counterexamples &lt;code&gt;x=y=1.9210567f0, t=0.10707888f0, z_t = 1.9210566f0 &amp;lt; x&lt;/code&gt; and &lt;code&gt;x=-1.4928777f-36, y=-1.4825014f-36, t=2.9802322f-8, z_t=-1.4928778f-36 &amp;lt; x&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I allow using if-statements, then simply evaluating one of&#xA;$$ x+(y-x)t\quad(t\leq\tfrac12), \qquad y - (y-x)(1-t)\quad(t\geq\tfrac12) $$&#xA;works as defined above, but this is quite inefficient because of the branching on $t$. There is also sometimes an unexpected discontinuity at $t=\frac12$:&#xA;$$ \begin{gathered}&#xA;x = 0.9921611, \quad y = 4.0\\&#xA;x+\tfrac12(y-x) = 2.4960806, \qquad y-\tfrac12 (y-x) = 2.4960804.&#xA;\end{gathered} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Does there exist an efficient simple formula that gives the linear interpolant with the standard mathematical properties?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-08-01T19:28:17.817" LastActivityDate="2017-08-01T19:28:17.817" Title="How can I interpolate $z_t = x(1-t)+y t$ with single-precision floats so that it satisfies $x\leq z\leq y$, $z_0=x$, $z_1=y$?" Tags="|numerical-analysis|floating-point|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26730" PostTypeId="2" ParentId="26729" CreationDate="2017-04-27T09:49:46.497" Score="3" Body="&lt;p&gt;The following link might give some insight &lt;a href=&quot;https://fgiesen.wordpress.com/2012/08/15/linear-interpolation-past-present-and-future/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fgiesen.wordpress.com/2012/08/15/linear-interpolation-past-present-and-future/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The suggested variant is:&#xA;&lt;code&gt;lerp_3(t, a, b) = fma(t, b, fnms(t, a, a))&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(fnms is fused negative multiply subract, so it can be reexpressed as &lt;code&gt;fma(-t, a, a)&lt;/code&gt;, which should be equivalent from an accuracy, but not a performance, standpoint)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is more accurate than ever including the $1-t$ explicitly, where I would expect most of the practical precision loss, so &lt;code&gt;fma(t, y, fma(-t, x, x))&lt;/code&gt; in your variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have a proof for the $x \leq y \leq z$ inequality, though.&lt;/p&gt;&#xA;" OwnerUserId="23161" LastActivityDate="2017-04-27T09:49:46.497" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26731" PostTypeId="2" ParentId="20641" CreationDate="2017-04-27T12:43:45.290" Score="1" Body="&lt;p&gt;I assume you are asking about the total energy which is printed as the final result of your DFT calculation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This energy represents the kinetic energy of all your particles (atoms and electrons) and their coulomb interaction. It does not have much physical meaning by itself. Usually, the more particles and the larger your basis set, the smaller (more negative) the total energy. This is true for all methods (DFT, CC, CI ...), although the actual numbers will be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get something meaningful out of it, you need to compare with something on the same (or similar) level of theory. For example you can calculate a single nickel atom and subtract twice its total energy to get the dimers binding energy. This is what you expect going to 0 for large  distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You always need a reference to compare your value with. Just as with voltage, pressure or the height of a mountain.&lt;/p&gt;&#xA;" OwnerUserId="24086" LastActivityDate="2017-04-27T12:43:45.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26732" PostTypeId="2" ParentId="11414" CreationDate="2017-04-27T15:36:45.107" Score="5" Body="&lt;p&gt;When analyzing the matrix structure, one can see that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;column &lt;code&gt;i&lt;/code&gt; is connected to column &lt;code&gt;j&lt;/code&gt; iff &lt;code&gt;i == j (mod k)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;row &lt;code&gt;i&lt;/code&gt; is connected to row &lt;code&gt;j&lt;/code&gt; iff &lt;code&gt;i == j (mod k)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;there are at most 2 non-zero elements per row&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That leads to the idea of sorting your row and column indices by cosets (w.r.t. &quot;modulo k&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible way of sorting by &quot;modulo k&quot; cosets is given by the following permutation &lt;code&gt;myIndex&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = 7; % example&#xA;k = 4;&#xA;&#xA;A = -2*diag(ones(n,1)) + diag(ones(n-k,1),k) + diag(ones(n-k,1),-k);&#xA;&#xA;myIndex = [];&#xA;ctr = 1;&#xA;subsystemCount = 1;&#xA;for p=1:n&#xA;  myIndex = [myIndex ctr];&#xA;  ctr = ctr + k;&#xA;  if ctr &amp;gt; n&#xA;    subsystemCount = subsystemCount + 1;&#xA;    if mod(n,k) &amp;gt; 0&#xA;      ctr = ctr - n;&#xA;    else&#xA;      ctr = ctr - n + 1;&#xA;    end&#xA;  end&#xA;end&#xA;&#xA;B = A(myIndex,myIndex) % tridiagonal system&#xA;subsystemCount % number of independent sub-systems&#xA;&#xA;&#xA;A =&#xA;-2     0     0     0     1     0     0&#xA; 0    -2     0     0     0     1     0&#xA; 0     0    -2     0     0     0     1&#xA; 0     0     0    -2     0     0     0&#xA; 1     0     0     0    -2     0     0&#xA; 0     1     0     0     0    -2     0&#xA; 0     0     1     0     0     0    -2&#xA;&#xA;B =&#xA;-2     1     0     0     0     0     0&#xA; 1    -2     0     0     0     0     0&#xA; 0     0    -2     1     0     0     0&#xA; 0     0     1    -2     0     0     0&#xA; 0     0     0     0    -2     1     0&#xA; 0     0     0     0     1    -2     0&#xA; 0     0     0     0     0     0    -2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The resulting matrix is tridiagonal and for each coset you get an independent system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then solve all sub systems independently, using an algorithm for solving (symmetric) tridiagonal systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the systems independently shouldn’t really be necessary, but it’s interesting to note that you don’t have a single linear system but multiple independent ones for a matrix like that.&lt;/p&gt;&#xA;" OwnerUserId="24087" LastEditorUserId="24087" LastEditDate="2017-04-30T22:22:28.293" LastActivityDate="2017-04-30T22:22:28.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26733" PostTypeId="1" CreationDate="2017-04-27T18:32:44.607" Score="1" ViewCount="58" Body="&lt;p&gt;During the past few years several important areas of image processing and image classification or generation became dominated by convolutional neural networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested if there are any methods coming from mathematical physics context (like methods designed for solving ill-posed problems, spectral analysis, image deblurring and deringing), that outperform neural network-based approaches in 2017 for some common computer vision or image processing problem. Or methods that don't have any neural network-based rivals. Maybe in the field of biomedical image analysis (just a guess)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The deeper and more specific the answer is, the better.&lt;/p&gt;&#xA;" OwnerUserId="24088" LastActivityDate="2017-04-27T18:32:44.607" Title="Mathematical physics applications in present-day image processing" Tags="|image-processing|computer-vision|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26734" PostTypeId="1" CreationDate="2017-04-27T20:55:12.323" Score="5" ViewCount="137" Body="&lt;p&gt;Are there sites for discussing more informally about computational modelling?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forum-style, rather than QA.&lt;/p&gt;&#xA;" OwnerUserId="17316" LastActivityDate="2018-05-04T06:26:56.320" Title="Are there sites for discussing more informally about computational modelling?" Tags="|education|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26735" PostTypeId="2" ParentId="11249" CreationDate="2017-04-27T23:19:24.140" Score="10" Body="&lt;h1&gt; Overview &lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Good question. There is a paper entitled &quot;Improving the accuracy of the matrix differentiation method&#xA;for arbitrary collocation points&quot; by R. Baltensperger. It's no big deal in my opinion, but it has a point (that already was known before the appearance in 2000): it stresses the importance of an accurate representation of the fact that the derivative of the constant function &lt;span class=&quot;math-container&quot;&gt;$f(x)=1$&lt;/span&gt; should be zero (this holds exactly in the mathematical sense, but not necessarily in the numerical representation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is simple to see that this requires the row sums of the n-th derivative matrices &lt;span class=&quot;math-container&quot;&gt;$D^{(n)}$&lt;/span&gt; to be zero. It is common to enforce this constraint by adjusting the diagonal entry, i.e. by setting &lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tag 1 D^{(n)}_{jj} := -\sum_{\substack{i=1\\i \neq j }}^N D_{ij}\,.&#xA;$$&lt;/span&gt; It's clear that this feature does not hold exactly when working on a computer due to roundoff errors in floating point calculations. What is more surprising is that these errors are even more severe when using the analytical formulas for the derivative matrix (which are available for many classical collocation points, e.g. Gauss-Lobatto).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the paper (and references therein) states that the error of the derivative is in the order of the deviation of the row sums from zero. The goal is therefore to make these numerically as small as possible.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt; Numerical tests &lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The good point is that the Fornberg procedure seems to be quite good in this regard. In the picture below I've compared the behaviour of the exact, i.e. analytical, first derivative matrix and the one derived by the Fornberg algorithm, for varying number of Chebyshev-Lobatto collocation points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/zLe3p.png&quot; width=&quot;512&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, believing the statement in the cited paper, this implies that the Fornberg algorithm will yield more accurate results for the derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to prove that, I'll use the same function as in the paper,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\tag 2 f(x) = \frac{1}{1+x^2}\,.$$&lt;/span&gt; and evaluate the error &lt;span class=&quot;math-container&quot;&gt;$$\tag 3  E_n = \max_{i\in \{0,\ldots,n\}} \bigg| f^\prime(x_i) - \sum_{j=1}^n D_{ij} f(x_j) \bigg|\,.$$&lt;/span&gt; This is done for (i) the analytically obtained derivative matrix, (ii) the Fornberg derivative matrix, and (iii) an adjusted version of the Fornberg matrix, where the above Eq. (1) is enforced quite straightforwardly via &lt;span class=&quot;math-container&quot;&gt;$$\tag 4 \tilde D_{jj} = D_{jj} - \left(\sum_{i=1}^n D_{ji} \right), \qquad \text{for all}\ j\,.&#xA;$$&lt;/span&gt; Here is what I get (again for the example of Gauss-Lobatto abscissas):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/kvNKW.png&quot; width=&quot;512&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt; Conclusion &lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In conclusion, Fornberg's method seems to be quite accurate, in the case &lt;span class=&quot;math-container&quot;&gt;$N=512$&lt;/span&gt; even by about 3 orders of magnitude more accurate than results from the analytical formulas. This should be accurate enough for most applications. Moreover, this is remarkable because Fornberg does not seem to explicitly include this fact in his method (at least there is no mention in the two Fornberg papers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another order of magnitude can be gained for this example through a straightforward inclusion of Eq.(4). As this is a quite simple approach and applied only once for each derivative, I see no reason in not using it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method from the Baltensperger paper -- which uses a more sophisticated approach for evaluating the sum in Eq.(1) in order to reduce roundoff errors -- yields about the same order of magnitude for the error. So, at least for this example, it's roughly equivalent to the &quot;Adjusted Fornberg&quot; method above.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2019-04-26T05:44:24.730" LastActivityDate="2019-04-26T05:44:24.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="26736" PostTypeId="1" CreationDate="2017-04-28T00:20:59.827" Score="1" ViewCount="60" Body="&lt;p&gt;I'm attempting to solve a problem of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{a}^{(n+1)}(t) = \int_{0}^{t}d\tau e^{i\mathbf{H}\tau} \mathbf{D}(\tau)e^{-i\mathbf{H}\tau}\mathbf{a}^{(n)}(\tau)&#xA;$$&#xA;Where $\mathbf{D}(\tau)$ is a oscillatory time varying matrix ($\mathbf{D}\cdot E(\tau)$), $\mathbf{H}$ is a diagonal matrix, and $\mathbf{a}^{(n)}(t)$ is a vector function of time.  I would like to obtain $\mathbf{a}^{(n+1)}(t)$. $\mathbf{D}$ oscillates  with frequency significantly less than the oscillations due to the exponents, in general, though later iterations will change that.  The frequencies for different terms of the vector are likely to be very different (by a factor of $10$ is likely).  In later iterations $\mathbf{a}^{(n)}(t)$ will also oscillate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likely important to note:  the first step in this ($a^{(0)}$) is constant, and thus the integrand of the first part is known for all $t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you might be able to guess by the $(n+1)$ and $(n)$, this is an iterative process, and I would like to know both $\mathbf{a}^{(n)}(t)$ and $\mathbf{a}^{(n+1)}(t)$ for a number of evenly spaced points per period of $\mathbf{D}(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how I should be going about doing this.  Since I would like to find the integral for a number of different values of $t$, something like a Riemann sum (like the trapezoid rule or similar) seems like a better idea than something like Gaussian quadrature, however, I'm not sure what is a good method when dealing with complex oscillatory functions.  Is an adaptive method a good option?  If so, what are good methods for finding the adaptive step for vectors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vector nature of this is only moderately important:  the vectors are approximately 3000 values long, so I would rather not do a different calculation for each vector, instead preferring to find them all at once.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2017-04-28T00:35:59.193" LastActivityDate="2017-04-28T01:25:15.967" Title="Methods for integration of oscillatory complex vectors as a function of time" Tags="|quadrature|oscillations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26737" PostTypeId="2" ParentId="26736" CreationDate="2017-04-28T01:01:01.057" Score="1" Body="&lt;p&gt;That is a Volterra integral equation in principle, and there are several methods to solve it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would first differentiate the whole expression to obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tag 1 \partial_t \mathbf{a}^{(n+1)}(t) = e^{i\mathbf{H}t} \mathbf{D}(t)e^{-i\mathbf{H}t}\mathbf{a}^{(n)}(\tau)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define $\mathbf b^{(i)}(t) = e^{-i\mathbf{H}t} \mathbf a^{(i)}$, then&#xA;$$&#xA;\tag 2 i\partial_t \mathbf{b}^{(n+1)}(t) = \mathbf{H}(t) \mathbf{b}^{(n)}(t) + \mathbf{D}(t) \mathbf{b}^{(n)}(t),\\&#xA;\mathbf{b}^{(n+1)}(0) = 0\,.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, this might look to you like going back, as this is probably where you came from: Schrödinger equation, electric field in dipole approximation, interaction picture, and so on -- you know the stuff. But the point is, you can solve that latter differential equation much better than the integral equation above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So drop the upper iteration index,&#xA;$$&#xA;\tag 3 i\partial_t \mathbf{b}(t) = \Big(\mathbf{H}(t) + E(t)\mathbf{D} \Big)\mathbf{b}(t)&#xA;$$&#xA;and just apply some basic Runge-Kutta for the first. Thereafter, I'd look at Crank-Nicolson, Lanczos schemes, and so on.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; ok, you want the iteration summands by itself. For that, one can use the boundary condition $\mathbf b^{(0)}(t) = 1$ (the first term in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Dyson_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dyson series&lt;/a&gt;) and starting from this again solve the differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The underlying idea here is that the indexed version, Eq.(2), is some kind of fix-point problem (starting with a more or less irrelevant initial value), which, when it hopefully converges, also satisfies the Schrödinger equation, Eq.(3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know, this is probably again not what you've expected, but your integral and Eq.(2) are mathematically equivalent, and the latter is simply easier to solve, so I'd use it.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2017-04-28T01:25:15.967" LastActivityDate="2017-04-28T01:25:15.967" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26738" PostTypeId="1" CreationDate="2017-04-28T02:02:19.120" Score="3" ViewCount="70" Body="&lt;p&gt;Id like to know methods for  numerical computation of $L^2$ norm of a two dimensional trigonometric polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the coefficients. If I want to compute the L^1 norm, I can do so by sampling in time at a rate greater than twice of the order of the trig polynomial (Shannon sampling theorem) and then doing summation of samples which can approximate the L^1 norm integral quite well. But for L^2 norm I need to square in time domain and cannot use this procedure as squaring is non linear?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS : I dont want to use Parseval's theorem, as it wont help if I want to compute L^3 norm of a 3 dimensional trigonometric polynomial. I want a general procedure for $L^p$ norm of a $p$ dimensional trig polynomial.&lt;/p&gt;&#xA;" OwnerUserId="13214" LastActivityDate="2017-04-28T02:02:19.120" Title="How to do numerical computation of $L^p$ norm of a $p$ dimensional trigonometric polynomial" Tags="|numerical-analysis|fourier-analysis|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26739" PostTypeId="1" AcceptedAnswerId="26741" CreationDate="2017-04-28T03:30:20.887" Score="2" ViewCount="1023" Body="&lt;p&gt;I need to generate a discontinuous plot (piecewise in each triangle) in matlab, something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NymYO.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NymYO.gif&quot; alt=&quot;Discontinuous plot, solution of finite elements method&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This plot is from &lt;a href=&quot;http://www.alecjacobson.com/weblog/?p=3616&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.alecjacobson.com/weblog/?p=3616&lt;/a&gt;, but I don't understand how generate it.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I have two matrices corresponding to a triangular mesh of a domain (typical of finite elements method, that is, no overlaped triangles, the union of all triangles is equal to the domain...):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coord=[x1,y1;x2,y2;...;xnod,ynod]      % vertices of the mesh&#xA;ele  =[n1,n2,n3;n1,n2,n3;...;n1,n2,n3] % mesh conectivity&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coord(1,:)=[x1,y1] are the coordinates of vertex 1 of the mesh (vertex of some triangle)&#xA;coord(2,:)=[x2,y2] are the coordinates of vertex 2 of the mesh (vertex of some triangle)&#xA;etc,&#xA;&#xA;&#xA;ele(1,:)=[n1,n2,n3] is the number of the vertices of triangle 1&#xA;ele(2,:)=[n1,n2,n3] is the number of the vertices of triangle 2&#xA;etc.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This way, for example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coord(ele(4,:),:)=[x1,y1;x2,y2;x3,y3]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;are the coordinates (x1,y1), (x2,y2) and (x3,y3) of the vertices of triangle 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The solution that I need to plot is a vector &quot;u&quot;&lt;/strong&gt; which size is the number of triangles of the mesh (=number of rows of &quot;elem&quot;), because this solution is &lt;strong&gt;one constant&lt;/strong&gt; over each triangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I plot this discontinuous solution?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="17630" LastActivityDate="2023-05-27T17:14:40.917" Title="Plot 2D piecewise constant in matlab in a finite elements mesh" Tags="|finite-element|matlab|discontinuous-galerkin|octave|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26740" PostTypeId="1" AcceptedAnswerId="26750" CreationDate="2017-04-28T06:09:31.020" Score="3" ViewCount="457" Body="&lt;p&gt;A common approach says an incompressible flow has velocity divergence of 0; use this to solve for pressure in the Navier Stokes momentum equation. Or, using the Helmholtz Decomposition &quot;project&quot; the zero divergence component of the velocity field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, can we treat the positive/negative divergence as expansion/compression, and obtain pressure from the gas equation? This seems straightforward, copying how nature does it. (Similar to the shallow water equations, where the depth of water gives hydrostatic pressure directly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it simply naive? And worse than solving in terms of efficiency/accuracy/stability? But I haven't seen it  discussed at all - and it might work well for some cases (though not the engineering/heliophysics applications where much of cfd seems to originate). I'm curious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding so far:&#xA;an incompressible &lt;em&gt;flow&lt;/em&gt; doesn't require that the fluid is incompressible. e.g. air is compressible, yet can be treated as an incompressible flow. This requires velocities in the fluid to have a magnitude much less than the speed of sound (pressure waves) in the fluid. The speed of sound doesn't directly cause the incompressibility, but indicates how stiff the fluid is; the strength of the elastic reaction to pressure. The ratio of fluid velocity to speed of sound is the &lt;em&gt;Mach number&lt;/em&gt; $M$; the &lt;em&gt;Cauchy number&lt;/em&gt; is $M^2$, and is the &lt;a href=&quot;https://wikipedia.org/wiki/Cauchy_number&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;ratio between inertial and the compressibility force (elastic force) in a flow&quot;&lt;/a&gt;. So I think that: if the pressure is transmitted more quickly than the fluid itself, we get an &quot;incompressible flow&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW: I've mainly looked at cfd for computer animation, so I'm probably missing a lot.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-05-08T06:04:28.703" LastActivityDate="2017-05-08T06:04:28.703" Title="Can we simulate incompressible flows using the (slight) density changes to give pressure?" Tags="|fluid-dynamics|navier-stokes|incompressible|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26741" PostTypeId="2" ParentId="26739" CreationDate="2017-04-28T07:22:08.960" Score="3" Body="&lt;p&gt;I use typically the following approach. The idea is to make a new mesh where every vertex is duplicated so that each triangle has its own copy. Then you can use standard &lt;em&gt;trisurf&lt;/em&gt; command to the resulting mesh structure.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p=coord';&#xA;t=ele';&#xA;x=p(1,:);&#xA;y=p(2,:);&#xA;P=[x(t(:));y(t(:))];&#xA;T=reshape(1:size(P,2),[3 size(P,2)/3]);&#xA;% create random u for testing&#xA;u=rand(size(P,2)/3,1);&#xA;tmp=[u';u';u'];&#xA;trisurf(T',P(1,:),P(2,:),tmp(:))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fnjN0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fnjN0.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have PDE toolbox there is also a built in command for this called &lt;em&gt;pdesurf&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2017-04-28T07:22:08.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26742" PostTypeId="1" AcceptedAnswerId="26743" CreationDate="2017-04-28T11:52:41.010" Score="2" ViewCount="2855" Body="&lt;p&gt;I've been trying to plot the following function in Python:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H(a,u) = \frac{a}{\pi} \int_{-\infty}^{\infty}\frac{exp(-y^2)}{a^2 + (u - y)^2}dy $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I keep receiving the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;File &quot;//anaconda/lib/python3.5/site-packages/scipy/integrate/quadpack.py&quot;, line 383, in _quad&#xA;return _quadpack._qagie(func,bound,infbounds,args,full_output,epsabs,epsrel,limit)&#xA;&#xA;error: Supplied function does not return a valid float.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the code I am using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy import integrate&#xA;import matplotlib.pyplot as plt&#xA;&#xA;a = 0.1&#xA;pi = np.pi&#xA;u = np.linspace(-100,100,200)&#xA;&#xA;&#xA;def integrand(a,u,y):&#xA;    top = np.exp(-y**2)&#xA;    bottom = a**2 + (u - y)**2&#xA;    return top/bottom&#xA;&#xA;def func(u):&#xA;    res = np.zeros_like(u)&#xA;    for i,val in enumerate(u):&#xA;        y,err = integrate.quad(integrand, -np.inf, np.inf, args=(a,u))&#xA;        res[i] = y&#xA;    return (a/pi)*res&#xA;&#xA;&#xA;plt.plot(u,func(u))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;u is a series of values which represent a range of frequencies. I've always struggled to get integration to work in python. Is there a good direction anyone could recommend me to go in to get it to work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="23269" LastActivityDate="2018-01-28T12:58:38.703" Title="Plotting Voigt Function in Python" Tags="|python|scipy|integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26743" PostTypeId="2" ParentId="26742" CreationDate="2017-04-28T12:17:03.597" Score="3" Body="&lt;p&gt;Have you seen the documentation of &lt;code&gt;scipy.integrate.quad&lt;/code&gt;? I see at least two issues with your code:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the first argument in the definition of &lt;code&gt;integrand&lt;/code&gt; should be the integration variable &lt;code&gt;y&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;the argument &lt;code&gt;u&lt;/code&gt; in &lt;code&gt;integrand&lt;/code&gt; is a scalar, yet you are providing a vector in the &lt;code&gt;args&lt;/code&gt; option of &lt;code&gt;integrate.quad&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If all you want to do is plot your function, the following snippet is compact and clean&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy import integrate&#xA;import matplotlib.pyplot as plt&#xA;&#xA;def Voigt(a, u):&#xA;    I = integrate.quad(lambda y: np.exp(-y**2)/(a**2 + (u - y)**2),-np.inf, np.inf)[0]&#xA;&#xA;    return (a/np.pi)*I&#xA;&#xA;a = 0.1&#xA;u_range = np.linspace(-100,100,200)&#xA;&#xA;plt.plot(u_range, [Voigt(a, u) for u in u_range])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AVCf3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AVCf3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21460" LastEditorUserId="26689" LastEditDate="2018-01-27T22:01:32.547" LastActivityDate="2018-01-27T22:01:32.547" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26744" PostTypeId="2" ParentId="8570" CreationDate="2017-04-28T20:09:03.767" Score="0" Body="&lt;p&gt;Check out the following paper: &lt;a href=&quot;http://iopscience.iop.org/article/10.1088/1748-3190/aa5e08/meta&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://iopscience.iop.org/article/10.1088/1748-3190/aa5e08/meta&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's our overview of Peskin's Immersed Boundary Method with open source (full) implementations in both MATLAB and Python 3, here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nickabattista/IB2d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nickabattista/IB2d&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24101" LastActivityDate="2017-04-28T20:09:03.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26745" PostTypeId="1" CreationDate="2017-04-29T10:52:41.177" Score="1" ViewCount="2777" Body="&lt;p&gt;I am working with VTK legacy files containing vector information. The files have incremental extension representing a file series as defined by &lt;a href=&quot;http://www.paraview.org/Wiki/Animating_legacy_VTK_file_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When applying the filter &quot;Plot Selection over Time&quot;, I get a nice curve as follows: &lt;a href=&quot;https://i.stack.imgur.com/saXgx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/saXgx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the x-axis does not show the realtime, rather it shows the actual time step. I would like to preserve the same data but have it show the x axis from 0 to 1 seconds rather than from 0 to 100 time steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've searched up and down for a further filter that I can apply to the plot that can do this but haven't found one. Also the &quot;Bottom axis custom scale&quot; property does not help since it truncates the data after 1 second rather than rescaling it. I feel like there should be a relatively straightforward way to accomplish this on Paraview. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="22736" LastEditorUserId="9667" LastEditDate="2017-05-01T20:34:09.907" LastActivityDate="2017-06-01T01:22:39.310" Title="Rescaling x-axis data in Paraview" Tags="|visualization|paraview|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26746" PostTypeId="1" CreationDate="2017-04-29T12:19:01.137" Score="1" ViewCount="53" Body="&lt;p&gt;I am referring to this paper:- &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;A Parameter-Free Generalized Moment Limiter for High-&#xA;Order Methods on Unstructured Grids &quot; by Michael Yang and Z.J. Wang.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dept.ku.edu/~cfdku/papers/AIAA-2009-605.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dept.ku.edu/~cfdku/papers/AIAA-2009-605.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have implemented a general limiter which doesn't require any user input. In the process, they have represented the solution in the cell in terms of the average of derivatives (Eqn. 28 of paper). &lt;a href=&quot;https://i.stack.imgur.com/mxaQI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mxaQI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please explain how was this representation attained. The understanding of this is a quite high priority for me. Please help.&lt;/p&gt;&#xA;" OwnerUserId="11977" LastActivityDate="2017-04-29T12:19:01.137" Title="How do we implement Parameter free generalised Moment limiter in 1D Case in Discontinuous Galerkin methods?" Tags="|fluid-dynamics|discontinuous-galerkin|wave-propagation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26747" PostTypeId="2" ParentId="19586" CreationDate="2017-04-29T13:11:27.753" Score="0" Body="&lt;p&gt;i recommend to use joblib library's Parallel and delayed functions&#xA;use &quot;tempfile&quot; module to create temp shared memory for huge arrays, the examples and usage can be found in here &lt;a href=&quot;https://pythonhosted.org/joblib/parallel.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pythonhosted.org/joblib/parallel.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24106" LastActivityDate="2017-04-29T13:11:27.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26748" PostTypeId="2" ParentId="19586" CreationDate="2017-04-29T14:43:56.537" Score="1" Body="&lt;p&gt;You can try Julia. It's pretty close to Python, and has a lot of MATLAB constructs. The translation here is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F = @parallel (vcat) for i in 1:10&#xA;    my_function(randn(3))&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This makes the random numbers in parallel too, and just concatenates the results in the end during the reduction. That uses multiprocessing (so you need to do &lt;code&gt;addprocs(N)&lt;/code&gt; to add processes before using, and this is also works on multiple nodes on an HPC &lt;a href=&quot;http://www.stochasticlifestyle.com/multi-node-parallelism-in-julia-on-an-hpc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;as shown in this blog post&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use &lt;code&gt;pmap&lt;/code&gt; instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;F = pmap((i)-&amp;gt;my_function(randn(3)),1:10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want thread parallelism, you can use &lt;code&gt;Threads.@threads&lt;/code&gt; (though make sure you make the algorithm thread-safe). Before opening Julia, set the environment variable JULIA_NUM_THREADS, then it's:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ftmp = [Float64[] for i in Threads.nthreads()]&#xA;Threads.@threads for i in 1:10&#xA;    push!(Ftmp[Threads.threadid()],my_function(randn(3)))&#xA;end&#xA;F = vcat(Ftmp...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here I make a separate array for each thread, so that way they don't clash when adding to the array, then just concatenate the arrays afterwards. Threading is pretty new so right now there's just the straight use of threads, but I'm sure threaded reductions and maps will be added just like it was for multiprocessing.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-04-29T14:50:54.980" LastActivityDate="2017-04-29T14:50:54.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26749" PostTypeId="2" ParentId="6978" CreationDate="2017-04-29T19:45:06.770" Score="1" Body="&lt;p&gt;In addition to preconditioners mentioned by Jed Brown, there are adaptive strategies for selecting the restarting parameter, which also have the potential to speed up convergence. See for example &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0377042709000132&quot; rel=&quot;nofollow noreferrer&quot;&gt;A.H. Baker, E.R. Jessup, Tz.V. Kolev — A simple strategy for varying the restart parameter in GMRES(m)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-04-29T19:45:06.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26750" PostTypeId="2" ParentId="26740" CreationDate="2017-04-29T20:28:03.163" Score="3" Body="&lt;p&gt;Water (and other fluids that are considered incompressible) are of course in reality compressible -- just not very much, at the pressures and speeds involved in the flow we are modeling. Within the ranges of pressures we consider, a good approximation is that&#xA;$$&#xA;  \rho = \rho_0 (1+\alpha p)&#xA;$$&#xA;where $\rho_0$ is the reference density at reference pressure, $p$ the dynamic pressure induces by the flow, and $\alpha$ the compressibility. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we say that a fluid is incompressible, what we mean in practice is that $\alpha p$ (for typical pressures) is just much smaller than one, and so can be neglected. In other words, the mass conservation equation&#xA;$$&#xA;  \partial_t \rho + \nabla \cdot (\rho \mathbf u) = 0&#xA;$$&#xA;is well approximated by &#xA;$$&#xA;  \nabla \cdot \mathbf u = 0.&#xA;$$&#xA;But, because within the range we consider, $\rho$ and $p$ are linearly related, we can choose one or the other in the equations. In other words, if you really wanted to, you could write the Navier-Stokes equations as&#xA;$$&#xA;  \partial_t \mathbf u + \mathbf u \cdot \nabla \mathbf u&#xA;  - \nu \Delta \mathbf u + \frac{1}{\alpha\rho_0} \nabla \rho = 0,&#xA;  \\&#xA;  \nabla \cdot \mathbf u = 0,&#xA;$$&#xA;and so work with velocity and density as primary variables. That would be uncommon, but equally valid. You would then compute the pressure from the density, rather than the other way around. We just choose the pressure in the usual formulation of the Navier-Stokes equation for convenience, and because it makes it clear that the pressure -- not a multiple of the density -- is a Lagrange multiplier for the incompressibility. But both formulations are reasonable.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-04-29T20:28:03.163" CommentCount="20" ContentLicense="CC BY-SA 3.0" />
  <row Id="26751" PostTypeId="2" ParentId="12960" CreationDate="2017-04-29T21:37:58.793" Score="0" Body="&lt;p&gt;As an important factor for scientific computing which has not been mentioned in @LKevin's answer, let me point out the difference in double-precision performance between the Kepler and Maxwell architectures. As &lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Maxwell_(microarchitecture)&amp;amp;oldid=762794666#Performance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia says&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The theoretical double-precision processing power of a Maxwell GPU is 1/32 of the single precision performance (which has been noted as being very low compared to the previous generation Kepler).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-04-29T21:37:58.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26752" PostTypeId="2" ParentId="23631" CreationDate="2017-04-29T22:23:12.530" Score="0" Body="&lt;p&gt;If we assume CUDA, the answer is indeed simple: run &lt;em&gt;everything&lt;/em&gt; in parallel. It doesn't matter that the problem size is larger than the number of computational units, in fact it is very desirable and necessary to fully utilize the GPU. See the &lt;a href=&quot;http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#simt-architecture&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUDA Programming Guide&lt;/a&gt; for details about the execution model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make the answer more general, I don't think it makes sense to deal with problems where the number or parallelizable tasks is close to the number of computational units: if it is, you haven't revealed enough parallelism yet or the problem is too small. If there is enough parallelism, the &quot;tail effect&quot; is negligible assuming that all tasks take about the same time to process. For CPUs, where the number of cores is much lower compared to GPUs, the general strategy is to split the problem into small blocks of fixed size (say, &lt;code&gt;n*m&lt;/code&gt;) and then process the blocks in parallel -- this preserves data locality and avoids the need to decide along which axis the problem should be parallelized. For distributed systems, first split the tasks into big blocks equal to the number of nodes and then split each block into smaller blocks as before. For other GPU frameworks you will probably need to adapt to their execution model, but I think there will always be some block structure.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-04-29T22:23:12.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26753" PostTypeId="2" ParentId="26725" CreationDate="2017-04-29T22:34:26.257" Score="1" Body="&lt;h2&gt;Overview&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;First a few comments on absorbing boundaries: your second picture seems to imply that it's really the edges of your square which absorb the wave, but this is not quite true. Rather, one can consider your picture as in the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/aXz15.png&quot; width=&quot;256&quot;/&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, you see only the excerpt of the full space in which the wave is considered as undisturbed. And outside of what you see is something happening. Most straightforwardly, that would be a grid large enough so that no reflections can occur. But depending on the parameters, this is numerically too costly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this case, one can apply &quot;absorbing&quot; techniques whose general idea is to make the solution vanish outside the region of interest, without changing the behavior inside (the outside boundary is here sketched as a circle, which might be more appropriate due to symmetry, but depending on your wave this doesn't have to be). In order to do so, a balance must be found between (i) having as little impact on the inner region as possible and (ii) an as small outer region as possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many different techniques to implement the absorbing which try to find such an appropriate balance, among them (complex) absorbing potential, complex exterior scaling, masking functions, etc. (see also the references in the comments).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Masking functions / absorbing potentials&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As the OP asked for a simple method, I would suggest to try a masking function:&#xA;for this you apply a given function $\text{mask}(x,y)$ to your solution at each time step which is smaller than in the outside region $R_\text{out}$. A suitable function could be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_\text{mask}(x,y) = \tanh^2\left(\lambda (R - \sqrt{x^2+y^2})\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $R$ denotes the maximum radius on the grid and the parameter $\lambda$ determines the width of the mask which is to be tuned appropriately. Many other choices are possible here as well, e.g. a suitably streched quarter wavelength of a cosine, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are basically two alternatives how the masking function is applied. The simple version is to simply multiply the solution $u(x,y)$ by $\text{mask}(x,y)$ at each timestep. More sophisticated, one can include the logarithm of the masking function to the partial differential equation,&#xA;$$\frac{\partial u}{\partial t}=v + \log(f_\text{mask}(x,y)) u\\&#xA;\frac{\partial v}{\partial t} = c^2\nabla^2u $$&#xA;Why the log? Because in the operator one needs something which is zero inside, and smaller than zero outside. The latter version is very similar to the &lt;em&gt;(complex) absorbing potential&lt;/em&gt; technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although there are more efficient methods available, imo nothing beats the conceptual simplicity of the masking approach.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: the previous applies to first-order-in-time PDEs, the kind of which I'm usually dealing with. For second order equations, represented as two first-order equations for the function and the temporal derivative, the absorbing potential must be applied only to the first equation corresponding to the function. The reason is that one does not want the derivative to be damped out, but the function value.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2017-05-01T19:27:33.493" LastActivityDate="2017-05-01T19:27:33.493" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26754" PostTypeId="2" ParentId="11555" CreationDate="2017-04-30T11:53:41.567" Score="2" Body="&lt;p&gt;Note that Armadillo's &lt;code&gt;cx_double&lt;/code&gt; (which is used by &lt;code&gt;cx_vec&lt;/code&gt;) is just an alias for &lt;code&gt;std::complex&amp;lt;double&amp;gt;&lt;/code&gt;, so in fact you're converting between &lt;code&gt;std::complex&amp;lt;double&amp;gt;&lt;/code&gt; and &lt;code&gt;MKL_Complex16&lt;/code&gt;. The only requirement for the &lt;code&gt;reinterpret_cast&lt;/code&gt; to behave as expected is the compatibility of memory layouts of the involved structures -- in this case they are compatible. The &lt;a href=&quot;http://scc.ustc.edu.cn/zlsc/sugon/intel/mkl/mkl_userguide/GUID-9A806754-1453-4433-9254-6FEA2C66B21B.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;MKL User Guide&lt;/a&gt; even has an example where &lt;code&gt;MKL_Complex16&lt;/code&gt; is redefined to &lt;code&gt;std::complex&amp;lt;double&amp;gt;&lt;/code&gt;, in which case there is no need to do the casts manually and you can just pass the Armadillo pointers directly.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-04-30T11:53:41.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26755" PostTypeId="1" AcceptedAnswerId="26757" CreationDate="2017-04-30T16:59:27.050" Score="6" ViewCount="107" Body="&lt;p&gt;Let $\mathbf{x}, \mathbf{y} \in \mathbb{R}^n$ and $\text{supp}(\mathbf{x}) \subset \{1,2,...,n\}$ denote the set of indices such that $\mathbf{x}$ is non-zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What type of optimization problem can model the following constraint?&#xA;$\text{supp}(\mathbf{x}) \ \cap \ \text{supp}(\mathbf{y}) = \emptyset$&lt;/p&gt;&#xA;" OwnerUserId="24121" LastEditorUserId="-1" LastEditDate="2017-05-04T06:00:54.387" LastActivityDate="2017-05-04T06:00:54.387" Title="Intersections of supports constraint" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26757" PostTypeId="2" ParentId="26755" CreationDate="2017-04-30T19:03:58.643" Score="6" Body="&lt;p&gt;Suppose that $\mathbf{x}\ge0$ and $\mathbf{y}\ge0$. Then a necessary&#xA;and sufficient condition for $\mathrm{supp}\{\mathbf{x}\}\cap\mathrm{supp}\{\mathbf{y}\}=\emptyset$&#xA;is for the two vectors to be orthogonal, $\mathbf{x}^{T}\mathbf{y}=0$.&#xA;Indeed, this is the familiar strong duality condition for linear programming:&#xA;$$&#xA;\mathbf{x}_{i}\mathbf{y}_{i}=0\;\forall i\in\{1,\ldots,n\}\overset{\mathbf{x}\ge0,\;\mathbf{y}\ge0}{\iff}\mathbf{x}^{T}\mathbf{y}=0.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the general case, we may define $\mathbf{u}$ and $\mathbf{v}$&#xA;to bound the element-wise absolute values of $\mathbf{x}$ and $\mathbf{y}$,&#xA;as in&#xA;$$&#xA;-\mathbf{u}\le\mathbf{x}\le\mathbf{u},\quad-\mathbf{v}\le\mathbf{y}\le\mathbf{v}.\tag{*}&#xA;$$&#xA;It is clear that $\mathbf{u}\ge0$ and $\mathbf{v}\ge0$ are implicitly&#xA;enforced. Now, combining this with the orthogonality condition, the following&#xA;is a necessary and sufficient statement for $\mathrm{supp}\{\mathbf{x}\}\cap\mathrm{supp}\{\mathbf{y}\}=\emptyset$:&#xA;$$&#xA;\text{There exists }\mathbf{u},\mathbf{v} \text{ satisfying (*) and } \mathbf{u}^{T}\mathbf{v}=0.&#xA;$$&#xA;Note that the condition is generally nonconvex, and problems involving&#xA;such constraints are usually NP-hard. Some special cases, however,&#xA;can be solved in polynomial time using geometric programming or semidefinite&#xA;programming techniques.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="24121" LastEditDate="2017-05-01T19:13:49.037" LastActivityDate="2017-05-01T19:13:49.037" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26758" PostTypeId="1" AcceptedAnswerId="26790" CreationDate="2017-04-30T19:45:29.787" Score="7" ViewCount="745" Body="&lt;p&gt;Ordering of elements in an unstructured mesh is undoubtedly very important for the performance of computations. For example, it determines the structure of sparse matrices arising from PDE discretizations, which affects the performance of most linear algebra operations like matrix-vector product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently we used an unstructured tetrahedral mesh generated by COMSOL Multiphysics. For given mesh, we construct a matrix with the following structure: each row corresponds to some facet $F$ of the mesh and the non-zero elements correspond to the facets of the one or two cells adjacent to the facet $F$, i.e. for tetrahedral mesh each row has at most 7 non-zero elements. The matrix structure for the mesh generated by COMSOL is shown on the following figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xn1dX.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xn1dX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simultaneously, we experiment with other mesh generators, but we currently don't have a good way to order the mesh. Using a simple approach of an in-order traversal of an octree of the cell centers, we get the following structure of the same matrix as above, which is not as &quot;nice&quot; as the original:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rPOxO.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rPOxO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately the COMSOL documentation does not say anything about mesh ordering algorithms. Does anybody have an idea which algorithm it might be using or which algorithms might generate similar (or better) structure?&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-05-04T19:56:30.740" Title="Mesh ordering algorithms used by COMSOL Multiphysics" Tags="|mesh|comsol|unstructured-mesh|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26759" PostTypeId="2" ParentId="26758" CreationDate="2017-04-30T23:38:43.257" Score="8" Body="&lt;p&gt;The idea of &quot;ordering the nodes&quot; in a finite element mesh to improve the&#xA;computational time of the sparse solver originated in the large structural&#xA;analysis FE codes of the 70's. Those codes typically used banded or variable-band&#xA;storage schemes for the sparse matrices so reducing the bandwidth was the main&#xA;criterion. That is the origin of the old Cuthill-McKee and Reverse-Cuthill-McKee&#xA;bandwidth reduction algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those concepts are essentially obsolete today in all FE codes, such as COMSOL,&#xA;that use modern sparse solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say that your second ordering is &quot;not as nice&quot; as your first, presumably&#xA;because the bandwidth is larger than the first. Bandwidth is a completely&#xA;irrelevant metric in all modern sparse solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a specific example that you may find surprising. In a classic paper from&#xA;1973, &lt;a href=&quot;https://math.berkeley.edu/~mgu/MA228A/Nested_Dissection.pdf&quot; rel=&quot;noreferrer&quot;&gt;nested dissection&lt;/a&gt;,&#xA;Alan George showed that the optimum numbering for the Laplace equation &#xA;discretized with a simple square finite element mesh was obtained by what&#xA;he called &quot;nested dissection.&quot; The non-zeros for a $7\times7$ mesh with equations ordered according to the nested dissection algorithm are shown in the&#xA;figure below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CpU4A.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CpU4A.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the bandwidth of the matrix is nearly equal to the number of &#xA;equations. This is the equation ordering that minimizes the number of floating point&#xA;operations during factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All modern sparse solvers that I am aware of include algorithms for reordering&#xA;the equations for computational efficiency and they are invoked by default. Many include multiple algorithms and&#xA;will try two or more to find the one which minimizes the factorization time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The latest COMSOL documentation I have found shows that the direct sparse solver&#xA;options are MUMPS, PARDISO, and SPOOLES. All three of these have excellent algorithms&#xA;for reordering equations. They include algorithms that attempt to find an ordering&#xA;similar to George's nested dissection on general meshes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your specific problem, it is worthwhile to review COMSOL's guidelines regarding&#xA;choice of direct solver. And you might want to try one of the iterative solver&#xA;options where the equation ordering is irrelevant. But, in either case, the&#xA;numbers you use to label the nodes in your mesh are not going to have any significant&#xA;effect on the computational time. So pick ones that are most convenient for you.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-04-30T23:38:43.257" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26760" PostTypeId="1" CreationDate="2017-05-01T00:03:50.037" Score="1" ViewCount="234" Body="&lt;p&gt;Orthogonal polynomials are often preferred as basis functions. Recently I learned selecting orthonormal basis further simplifies the mass matrix from diagonal to simply the identity matrix when used as modal basis. But I am confused how the normalizing factors are obtained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say the orthonormal Legendre polynomials in each direction of a reference hexahedron are $P_1$, $P_2$, $P_3$. Then using tensor-product approach, is the orthonormal basis for hex simply $P_1\times P_2\times P_3$? What about for quads, simply $P_1\times P_2$, or do we need to multiply by some constant to keep the orthonormality of the basis? I learned from the text book (Hesthaven &amp;amp; Warburton) that orthonormal basis for triangles and tetrahedron have different normalizing factors, i.e. simply cancelling one of the directions of an orthonormal basis in tetrahedron does not result in a basis for the triangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Triangle: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sqrt{2} P_i^{(0,0)}(a) P_j^{(2i+1,0)}(b) (1-b)^i $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tetrahedron: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 2\sqrt{2} P_i^{(0,0)}(a) P_j^{(2i+1,0)}(b) P_k^{(2i+2j+2,0)}(c) (1-b)^i (1-c)^{(i+j)} $$&lt;/p&gt;&#xA;" OwnerUserId="18132" LastEditorUserId="18132" LastEditDate="2017-05-01T14:08:31.053" LastActivityDate="2017-05-01T14:08:31.053" Title="Orthonormal basis for hexahedron" Tags="|finite-element|discontinuous-galerkin|basis-set|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26761" PostTypeId="2" ParentId="26760" CreationDate="2017-05-01T04:22:12.807" Score="3" Body="&lt;p&gt;On quad/hex you can use tensor product polynomials. For example in 2d, you map the cell to a reference cell $[-1,1] \times [-1,1]$ and if $N$ is the degree, you would use&#xA;$$&#xA;P_i(\xi) P_j(\eta), \qquad 0 \le i,j \le N&#xA;$$&#xA;as the basis functions, where $\xi,\eta$ are coordinates on the reference cell and $P_i$ is the Legendre polynomial of degree $i$. These are orthogonal. To make it orthonormal, you just calculate&#xA;$$&#xA;m_i = \int_{-1}^{+1} P_i^2(\xi) d\xi&#xA;$$&#xA;and redefine&#xA;$$&#xA;P_i \rightarrow \frac{1}{\sqrt{m_i}} P_i&#xA;$$&#xA;You do the same thing in 3-d.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2017-05-01T04:22:12.807" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26762" PostTypeId="1" CreationDate="2017-05-01T12:21:20.843" Score="2" ViewCount="273" Body="&lt;p&gt;I have some &quot;flat&quot; triangles in my FEM mesh (angles for example 30°, 30°, 120°). I wonder whether this just decreases the mesh quality, or whether it will really cause problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do I ask? When computing the matrix for the Laplacian for linear elements, the off-diagonal elements are positive, and the diagonal elements are negative. In the case of &quot;flat&quot; triangles, this might change. Seems to be a bit suspicious to me...&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-05-02T00:04:09.983" Title="FEM with flat triangles" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26763" PostTypeId="1" AcceptedAnswerId="28865" CreationDate="2017-05-01T16:22:03.590" Score="10" ViewCount="1294" Body="&lt;p&gt;I've been using Intel MKL's SVD (&lt;code&gt;dgesvd&lt;/code&gt; through SciPy) and noticed that results are are significantly different when I change precision between &lt;code&gt;float32&lt;/code&gt; and &lt;code&gt;float64&lt;/code&gt; when my matrix is badly conditioned/not full rank. Is there a guide on the minimum amount of regularization I should add to make results insensitive to &lt;code&gt;float32&lt;/code&gt;-&gt;&lt;code&gt;float64&lt;/code&gt; change?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, doing $A=UDV^{T}$, I see that $L_\infty$ norm of $V^{T}X$ moves by about 1 when I change precision between &lt;code&gt;float32&lt;/code&gt; and &lt;code&gt;float64&lt;/code&gt;. $L_2$ norm of $A$ is $10^5$ and it has about 200 zero eigenvalues out of 784 total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing SVD on $\lambda I + A$ with $\lambda=10^{-3}$ made the difference vanish.&lt;/p&gt;&#xA;" OwnerUserId="18786" LastEditorUserId="11716" LastEditDate="2017-05-02T22:42:00.153" LastActivityDate="2018-02-27T17:23:33.287" Title="How much regularization to add to make SVD stable?" Tags="|numerical-analysis|stability|svd|intel-mkl|numerical-limitations|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="26764" PostTypeId="2" ParentId="26745" CreationDate="2017-05-01T20:33:50.880" Score="1" Body="&lt;p&gt;As @TylerOlsen suggests, it is probably a good idea to do this particular plot using another tool. But if you insist on doing it using Paraview, you can&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Apply a Temporal Shift Scale Filter; and&lt;/li&gt;&#xA;&lt;li&gt;Then plot over the shifted/scaled time.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In my example below I scaled my time by 0.1, turning the maximum to 25 (it has 250 timesteps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bZG8E.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bZG8E.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-05-02T00:52:42.513" LastActivityDate="2017-05-02T00:52:42.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26765" PostTypeId="2" ParentId="26762" CreationDate="2017-05-01T22:34:23.997" Score="3" Body="&lt;p&gt;A good resource if you're interested in this sort of thing would be &lt;a href=&quot;https://people.eecs.berkeley.edu/~jrs/papers/elemj.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;What is a Good Linear Finite Element?&lt;/a&gt; by Johnathan Richard Shewchuk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Large angles tend to degrade the quality of finite element approximations and the condition number of the finite-dimensional linear system that discretizes the PDE. Ivo Babuška has a &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/0713021&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; from way back showing an example where a sequence of finite element approximations to the solution of Laplace's equation in a rectangle fails to converge when the maximum angle of the triangles increases to $180^\circ$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this is a pretty pathological way to refine a mesh in practice; you would never do this on purpose. It would still be nice to have some provable upper bound, which is strictly less than $180^\circ$, on the maximum angle of a triangle in the mesh. &lt;a href=&quot;https://en.wikipedia.org/wiki/Ruppert%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ruppert's algorithm&lt;/a&gt; for 2D mesh generation is provably good in this respect; no angle is less than $20^\circ$, which gives the trivial upper bound of $140^\circ$ since the sum of the angles of a triangle must be $180^\circ$. This isn't exactly ideal, but nonetheless it means things won't go horribly wrong and people have improved upon it substantially since then. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, there's a lot of research on &lt;em&gt;mesh improvement&lt;/em&gt; -- taking an existing mesh and moving the vertices around a bit, possibly with a few topological modifications, to get a mesh of substantially higher quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't said where you're getting your meshes from in the first place, but &lt;a href=&quot;https://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Triangle&lt;/a&gt; and &lt;a href=&quot;http://gmsh.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gmsh&lt;/a&gt; are both very good mesh generators.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2017-05-02T00:04:09.983" LastActivityDate="2017-05-02T00:04:09.983" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26766" PostTypeId="1" AcceptedAnswerId="26771" CreationDate="2017-05-01T22:48:58.350" Score="2" ViewCount="3205" Body="&lt;p&gt;Although I've been looking everywhere, I have been unable to find an answer to my question so here it is.  For a driven damped pendulum the equation of motion in dimensionless units is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha(\omega,\theta,t)=-c\ \omega -\sin \theta +F(t).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is obtaining my next step $\omega(t + \Delta t)$.  I know what to do for time $t$ and $\omega$, but not for $\theta$.  The fourth order RK method can be found pretty much anywhere&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\omega(t+ \Delta t) = \omega(t) + (k_1 +2k_2 +2k_3 + k_4)*\Delta t/6, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I am specifically confused on what to enter for $\theta$ in the $k_1, k_2$. . . calculations.  The first one is easy but I'm unsure on the rest:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$k_1 = \alpha(\omega(t),\ \theta(t),\ t) \\ k_2=\alpha(\omega(t) + k_1\Delta t/2,\ \theta(t) + \ ???,\  t +\Delta t/2) \\ k_3=\alpha(\omega(t) + k_2\Delta t/2,\ \theta(t) + \ ???,\  t +\Delta t/2) \\ k_4=\alpha(\omega(t) + k_3\Delta t,\ \theta(t) + \ ???,\  t +\Delta t).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here solving for the next $\theta(t+\Delta t)$ is simple enough I just need help knowing what to do the above equation.  It seems like all the examples that I've seen show examples for only two variables and not three.&lt;/p&gt;&#xA;" OwnerUserId="24134" LastActivityDate="2017-05-02T17:27:18.507" Title="4th order Runge-Kutta Method for Driven Damped Pendulum" Tags="|runge-kutta|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26767" PostTypeId="1" AcceptedAnswerId="26768" CreationDate="2017-05-02T05:48:23.583" Score="2" ViewCount="82" Body="&lt;p&gt;I am working on a method based on moving least square approximation where shape functions  do not satisfy Kroneker Delta property. So Dirichlet boundary condition should be enforced.&#xA;I usually used penalty method for that. In the error estimation based on residual I used the following for Poisson equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\eta^2=h_T^2\Vert  \Delta u+f \Vert^2+h_T^{-1} \Vert u-g_D\Vert^2_{\partial \Omega}.&#xA;\end{align} &#xA;where&#xA;\begin{align}&#xA;-\Delta u=f, \quad u_{|\partial \Omega}=g_D.&#xA;\end{align}&#xA;Also, it should be noted that, because of the shape functions there is no jump between the internal elements.&#xA;The problem is that penalty parameter $\gamma$ should be bigger than $1e3$ to get good result. But for large $\gamma$ the effect of adaptive technique is not really good. All the time results by adaptive refinement is better than uniform. But when $\gamma$ is smaller for example $\gamma&amp;lt; 1e2$ the difference is remarkable. By these results I guess the term relates to Dirichlet boundary should be change by $\gamma$. But how? &lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2017-05-02T05:57:35.980" LastActivityDate="2017-05-02T09:15:24.367" Title="Dirichlet term in error estimations" Tags="|finite-element|error-estimation|meshless-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26768" PostTypeId="2" ParentId="26767" CreationDate="2017-05-02T09:15:24.367" Score="2" Body="&lt;p&gt;It is quite well known in finite element research community that penalty methods easily lead to under/over refinement when used with adaptive methods, see for example the numerical examples in the end of &lt;a href=&quot;http://www.ams.org/journals/mcom/2009-78-267/S0025-5718-08-02183-2/S0025-5718-08-02183-2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Juntunen, Stenberg: Nitsche's method for general boundary conditions&lt;/a&gt; (page 1373). If you want to enforce the boundary conditions weakly, I would advise you to use Nitsche's method instead.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2017-05-02T09:15:24.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26770" PostTypeId="2" ParentId="26763" CreationDate="2017-05-02T17:09:02.080" Score="13" Body="&lt;p&gt;The singular value decomposition for a symmetric matrix $A=A^{T}$&#xA;is one and the same as its canonical eigendecomposition (i.e. with an orthonormal matrix-of-eigenvectors), while the&#xA;same thing for a nonsymmetric matrix $M=U \Sigma V^T$ is just the canonical eigenvalue&#xA;decomposition for the symmetric matrix &#xA;$$&#xA;H=\begin{bmatrix}0 &amp;amp; M\\&#xA;M^{T} &amp;amp; 0&#xA;\end{bmatrix}=\begin{bmatrix}U &amp;amp; 0\\&#xA;0 &amp;amp; V&#xA;\end{bmatrix}\begin{bmatrix}0 &amp;amp; \Sigma\\&#xA;\Sigma &amp;amp; 0&#xA;\end{bmatrix}\begin{bmatrix}U &amp;amp; 0\\&#xA;0 &amp;amp; V&#xA;\end{bmatrix}^{T}&#xA;$$&#xA;Hence, without loss of generality, let us consider a closely related&#xA;question: &lt;strong&gt;If two symmetric matrices are approximately the&#xA;same, then should we expect their canonical eigendecompositions to also be approximately&#xA;the same?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is a surprising no. Let $\epsilon&amp;gt;0$ be small, and consider the two matrices&#xA;$$&#xA;A_{\epsilon}=\begin{bmatrix}1 &amp;amp; \epsilon\\&#xA;\epsilon &amp;amp; 1&#xA;\end{bmatrix}=V\Lambda_{\epsilon}V^{T},\qquad B_{\epsilon}=\begin{bmatrix}1+\epsilon &amp;amp; 0\\&#xA;0 &amp;amp; 1-\epsilon&#xA;\end{bmatrix}=U\Lambda_{\epsilon}U^{T}&#xA;$$&#xA;both of which have eigenvalues $\Lambda_{\epsilon}=\mathrm{diag}(1+\epsilon,1-\epsilon)$,&#xA;but whose eigenvectors are&#xA;$$&#xA;V=\frac{1}{\sqrt{2}}\begin{bmatrix}1 &amp;amp; 1\\&#xA;1 &amp;amp; -1&#xA;\end{bmatrix},\qquad U=\begin{bmatrix}1 &amp;amp; 0\\&#xA;0 &amp;amp; 1&#xA;\end{bmatrix}.&#xA;$$&#xA;While the matrices $A_{\epsilon} \approx B_{\epsilon}$ are approximately the same, their matrix-of-eigenvectors $V$ and $U$ are very different. Indeed, since the eigendecompositions are unique for $\epsilon&amp;gt;0$, there really exists no choice of $U,V$ such that $U\approx V$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, applying this insight back to the SVD under finite precision,&#xA;let us write $M_{0}=U_{0}\Sigma_{0}V_{0}^{T}$ as your matrix in &lt;code&gt;float64&lt;/code&gt;&#xA;precision, and $M_{\epsilon}=U_{\epsilon}\Sigma_{\epsilon}V_{\epsilon}^{T}$&#xA;as the same matrix in &lt;code&gt;float32&lt;/code&gt; precision. If we assume that the SVDs&#xA;themselves are exact, then the singular values $\Sigma_{0},\Sigma_{\epsilon}$&#xA;must differ by no more than a small constant factor of $\epsilon\approx10^{-7}$,&#xA;but the singular vectors $U_{0},U_{\epsilon}$ and $V_{0},V_{\epsilon}$&#xA;can differ by an arbitrarily large quantity. &lt;strong&gt;Hence, as shown, there is no way to make the SVD &quot;stable&quot; in the sense of the singular vectors.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2017-05-02T22:37:25.143" LastActivityDate="2017-05-02T22:37:25.143" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26771" PostTypeId="2" ParentId="26766" CreationDate="2017-05-02T17:27:18.507" Score="3" Body="&lt;p&gt;Let's define $\boldsymbol{u} = [\theta, \omega ]^{T} = [u_1, u_2]^{T}$. The system of differential equations to solve can then be represented as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot{\boldsymbol{u}} = f(t,\boldsymbol{u}) = \begin{bmatrix} u_2 \\ -c u_2 - \sin(u_1) + F(t)\end{bmatrix}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the resulting Runge-Kutta formulation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\boldsymbol{k}_1 &amp;amp;= f\left(t_k, \boldsymbol{u}_k\right) \\&#xA;%&#xA;\boldsymbol{k}_2 &amp;amp;= f\left(t_k + \frac{\Delta t}{2}, \boldsymbol{u}_k + \frac{\Delta t}{2} \boldsymbol{k}_1\right) \\&#xA;%&#xA;\boldsymbol{k}_3 &amp;amp;= f\left(t_k + \frac{\Delta t}{2}, \boldsymbol{u}_k + \frac{\Delta t}{2} \boldsymbol{k}_2\right) \\&#xA;%&#xA;\boldsymbol{k}_4 &amp;amp;= f\left(t_k + \Delta t, \boldsymbol{u}_k + \Delta t \boldsymbol{k}_3\right) \\&#xA;%&#xA;\boldsymbol{u}_{k+1} &amp;amp;= \boldsymbol{u}_k + \frac{\Delta t}{6}\left(\boldsymbol{k}_1 + 2\boldsymbol{k}_2 + 2\boldsymbol{k}_3 + \boldsymbol{k}_4\right)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $t_j = t_{0} + j\Delta t$ and $\boldsymbol{u}_j = \boldsymbol{u}(t_{j})$. As noted by &lt;strong&gt;nicoguaro&lt;/strong&gt;, this is just the vector form of the Runge-Kutta equations. All it assumes is that your function $f(\cdot,\cdot)$ takes a vector input for the state you are integrating.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-05-02T17:27:18.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26773" PostTypeId="1" CreationDate="2017-05-02T22:42:45.360" Score="1" ViewCount="1085" Body="&lt;p&gt;I am solving the one dimensional wave equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0=\Box\phi = -\partial_t^2\phi + \partial_r^2\phi ,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using a Crank-Nicolson finite difference scheme, in the domain $r\in[0,R]$. First, I define $\xi\equiv\partial_r\phi$ and $\Pi\equiv\partial_t\phi$, so that the equations of motion can be written as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ &#xA;0 = \partial_t\xi - \partial_r\Pi, \\ &#xA;0 = \partial_t\Pi - \partial_r\xi,&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, the stencils I'm using for the interior grid points are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;(\partial_t u)^{n+1/2}_j = \frac{u^{n+1}_j - u^n_j}{\Delta t} + \mathcal{O}\left((\Delta t)^2\right) \\&#xA;(\partial_r u)^{n+1/2}_j = \frac{1}{4\Delta r}\left(u^{n+1}_{j+1} - u^{n+1}_{j-1} + u^n_{j+1} - u^n_{j-1}\right) + \mathcal{O}\left((\Delta t)^2, (\Delta r)^2\right)&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Crank-Nicolson scheme is implicit but I am solving the implicit equations using Gauss-Seidel iteration to solve for $\xi^{n+1}_j$ and $\Pi^{n+1}_j$. I'm having no trouble evolving my initial data (a Gaussian profile for the $\phi$ field, moving the the left) in the interior grid points. I'm only running into trouble at the boundaries of my simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to implement outgoing wave boundary conditions at the boundaries of my simulation, i.e. at $r=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;0 = \partial_t\xi - \partial_r\xi, \\&#xA;0 = \partial_t\Pi - \partial_r\Pi,&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and at $r=R$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;0 = \partial_t\xi + \partial_r\xi \\&#xA;0 = \partial_t\Pi + \partial_r\Pi&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some reason I am having a hard time numerically implementing these boundary conditions. I have tried several approaches (including first and second order upwind schemes) to implement the boundary conditions but when I run my code either I get what appears to be a numerical instability or the the wave solution 'bounces back' into my domain; i.e. it doesn't exit my domain contrary to the boundary condition!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been having difficulty finding a good reference/discussion on outgoing wave boundary conditions. Could someone provide some insight on how to implement outgoing wave boundary conditions using an implicit method? Thank you in advance for the help. &lt;/p&gt;&#xA;" OwnerUserId="23388" LastEditorUserId="23388" LastEditDate="2017-05-03T13:56:59.307" LastActivityDate="2017-05-03T13:56:59.307" Title="How to implement outgoing wave boundary condition" Tags="|boundary-conditions|hyperbolic-pde|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26774" PostTypeId="1" AcceptedAnswerId="26779" CreationDate="2017-05-03T09:09:15.880" Score="5" ViewCount="1325" Body="&lt;p&gt;I have solved Poisson equation on a 3d domain with neumann and dirichlet boundary condition. I get the potential, take the gradient for each element and integrate on a surface of an element, I do this for all elements on a surface enclosing an electrode I am interested in knowing the current through. This does not give very accurate results, upto 50% error. And commercial FEM software gets very accurate fluxes.&lt;br&gt;&#xA; Does anyone know how they do it or know a better way to compute flux? I am unable to find any references on this&lt;/p&gt;&#xA;" OwnerUserId="23558" LastEditorUserId="6775" LastEditDate="2017-05-03T16:20:44.377" LastActivityDate="2017-05-07T17:51:25.300" Title="Computing accurate fluxes with FEM" Tags="|finite-element|boundary-conditions|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26775" PostTypeId="2" ParentId="26774" CreationDate="2017-05-03T16:04:52.453" Score="2" Body="&lt;p&gt;One simple way to improve the accuracy of the fluxes is to average the&#xA;discontinuous values of the element fluxes at each node. This gives you&#xA;a continuous, nodal flux function over the surface that can be integrated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all the elements connected to a node are (roughly) the same size, &#xA;simply averaging the element fluxes at the node is sufficiently&#xA;accurate. If the element sizes are significantly different, the&#xA;contribution of each element flux to the nodal value should be weighted by&#xA;the size of the element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more sophisticated methods for improving the values of fluxes&#xA;at the nodes but the fundamental idea is to &quot;smooth&quot; the discontinuous&#xA;values across inter-element boundaries. In structural analysis, this is&#xA;commonly referred to as &quot;stress smoothing.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the fluxes are going to be less accurate than the fundamental&#xA;nodal unknowns because they are derivatives of these quantities.&#xA;If you are using the simple 4-node tetrahedron, the flux is constant in&#xA;the element. Particularly, when flux calculations are important, it is&#xA;common to use the 10-node tetrahedron element instead.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-05-03T16:04:52.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26776" PostTypeId="2" ParentId="26774" CreationDate="2017-05-03T17:12:46.310" Score="6" Body="&lt;p&gt;If $\nabla u$ is of more interest than $u$ itself, it is reasonable to set $\mathbb v := -\sigma \, \nabla u$ and convert your Poisson problem&#xA;$$&#xA;-\nabla \cdot (\sigma \, \nabla u) = f \quad \text{in }\Omega, \\&#xA;u = g_E \quad \text{on }\partial\Omega_E, \\&#xA;\sigma \, \nabla u \cdot \hat{\mathbb n} = g_N \quad \text{on }\partial\Omega_N&#xA;$$&#xA;to a mixed one:&#xA;$$&#xA;\frac{1}{\sigma} \, \mathbb v + \nabla u = 0 \quad \text{in }\Omega, \\&#xA;\nabla \cdot \mathbb v = f \quad \text{in }\Omega, \\&#xA;u = g_E \quad \text{on }\partial\Omega_E, \\&#xA;\mathbb v \cdot \hat{\mathbb n} = -g_N \quad \text{on }\partial\Omega_N.&#xA;$$&#xA;(Note that for the mixed problem essential BCs are natural and natural BCs are essential.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you build an approximation for $\mathbb v$ directly since it is one of the unknowns. Hence you do not lose accuracy due to numerical differentiation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This mixed Poisson formulation is pretty well discussed in literature. &lt;/p&gt;&#xA;" OwnerUserId="21916" LastEditorUserId="21916" LastEditDate="2017-05-04T04:53:36.150" LastActivityDate="2017-05-04T04:53:36.150" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26777" PostTypeId="1" CreationDate="2017-05-03T18:12:43.027" Score="2" ViewCount="104" Body="&lt;p&gt;I'm struggling now for several weeks with a very bizarre problem with a system of delay differential equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, here the system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot a = 1 - \Theta(b(t-\tau)-\kappa) \,- a(t)&#xA;\\ \dot b = \,\,\,\,\,\,\,\,\,\,\,\Theta(a(t-\tau)-\kappa) - b(t),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $$\Theta: \Bbb R \rightarrow \{ 0,1 \},\,\,x \mapsto \begin{cases} 1,  x \ge 0 \\  0, x &amp;lt; 0 \end{cases}$$ is the Heaviside function, $\tau$ a delay and $0 \le \kappa \le 1$ a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here a visualisation of the numerically calculated periodic dynamics of $a$ and $b$ (please ignore the variable $c$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7cECV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7cECV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The horizontal dashed line marks the value of $\kappa$, here $\kappa \approx 0.364$ was used. Furthermore, $\tau = 0.89$ was used as well as the initial conditions $a(0) = 0$ and $b(0) = 0$ (the plot shows the dynamics after the transient oscillations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to solve the system of delay differential equations given above analytically by parting the interval $[0, T]$ into appropriate sections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, let's have a look at the dynamics of $a$:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$0 \leq t \leq t_a^{max}$:&#xA;\begin{align}&#xA;\partial_t a_1\left(t\right) &amp;amp;= 1 - a_1\left(t\right) \nonumber \\&#xA; \Rightarrow a_1\left(t\right) &amp;amp;= \left(a_{min} - 1\right) \mathrm{e}^{-t} + 1&#xA;\end{align}&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$t_a^{max}\leq t \leq T$:&#xA;\begin{align}&#xA;\partial_t a_2\left(t\right) &amp;amp;= - a_2\left(t\right) \nonumber \\&#xA; \Rightarrow a_2\left(t\right) &amp;amp;= a_{max} \mathrm{e}^{-\left(t-t_a^{max}\right)}&#xA;\end{align}&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The continuity conditions are given by&#xA;\begin{align}&#xA;a_1 \left(0\right) &amp;amp;= a_2\left(T\right) \nonumber \\&#xA;a_1\left(t_a^{max}\right) &amp;amp;= a_2\left(t_a^{max}\right).&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We get&#xA;\begin{align}&#xA;a_{min} &amp;amp;= \frac{\mathrm{e}^{t_a^{max}-T} - \mathrm{e}^{-T}}{1-\mathrm{e}^{-T}} \nonumber \\&#xA;a_{max} &amp;amp;= \frac{1- \mathrm{e}^{-t_a^{max}}}{1-\mathrm{e}^{-T}}.&#xA;\end{align} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's have a look at the dynamics of $b$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Analogous to the dynamics of $a$, we get&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$0 \le t \le t_b^{min}$:&#xA;\begin{align}&#xA;b_0(t) = b_r e^{-t}&#xA;\end{align}&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$t_b^{min} \le t \le t_b^{max}$:&#xA;\begin{align}&#xA;b_1\left(t\right) = \left(b_{min} - 1\right) \mathrm{e}^{-\left(t-t_b^{min}\right)} + 1 \nonumber&#xA;\end{align}&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$t_b^{max} \le t \le T$:&#xA;\begin{align}&#xA;b_2\left(t\right) = b_{max} \mathrm{e}^{-\left(t-t_b^{max}\right)}&#xA;\end{align}&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and with the continuity conditions&#xA;\begin{align}&#xA;b_0(0) &amp;amp;= b_2(T) \\&#xA;b_1 \left(t_b^{min}\right) &amp;amp;= b_2\left(t_b^{min}\right) \nonumber \\&#xA;b_1\left(t_b^{max}\right) &amp;amp;= b_2\left(t_b^{max}\right) &#xA;\end{align}&#xA;we conclude&#xA;\begin{align}&#xA;b_{min} &amp;amp;= \frac{\mathrm{e}^{t_b^{max} - t_b^{min} -T} - \mathrm{e}^{-T}}{1-\mathrm{e}^{-T}} \nonumber \\&#xA;b_{max} &amp;amp;= \frac{1- \mathrm{e}^{t_b^{min}-t_b^{max}}}{1-\mathrm{e}^{-T}}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we calculate now the times $t_a^{max}$ and $T$ for a given delay $\tau$, we see something very bizarre (the calculation is not shown here because it's very long, but we are very confident, really very confident, that there was no error made in this calculation).&#xA;We get the following dynamics (a in blue, b is not shown):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7yTqq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7yTqq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the curve of $a(t)$ is horizontally mirrored in compare to the curve of $a(t)$ shown in the image above which we get from the numerical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is it possible that we now get this curve instead of a similar curve to the one in the first image? Did we make any wrong assumptions? Can anybody solve this mystery?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other hints that may help to solve the mystery:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the analytical solution it holds $a_{max} &amp;lt; a_{min}$ and $b_{min} &amp;gt; b_{max}$. This comes along with the fact that the times $t_a^{max}$ and $T$ are negative (there is no positive solution for them).&lt;/p&gt;&#xA;" OwnerUserId="24151" OwnerDisplayName="Peter123" LastEditorUserId="7574" LastEditDate="2018-03-27T11:12:37.890" LastActivityDate="2018-03-27T11:12:37.890" Title="Mysterious Mirroring in Analytical Solution of a delay differential equation (DDE)" Tags="|computational-physics|differential-equations|dde|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26778" PostTypeId="1" CreationDate="2017-05-03T20:38:45.377" Score="2" ViewCount="2255" Body="&lt;p&gt;I am trying to find the intersection points in images like the one posted below. The intersections I'm trying to find are circled in red in the image below it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://res.cloudinary.com/turnipentropy/image/upload/v1493842185/curves_wxxjyo.jpg&quot; alt=&quot;binary curves&quot;&gt;&#xA;&lt;img src=&quot;https://res.cloudinary.com/turnipentropy/image/upload/v1493842186/with_intersections_ctujgz.jpg&quot; alt=&quot;intersections marked&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My most succesful approach:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Thin the image&lt;/li&gt;&#xA;&lt;li&gt;Perform a Hough transform, peak identify, and back project&lt;/li&gt;&#xA;&lt;li&gt;Identify peaks in the back projection and select those as intersection points&lt;/li&gt;&#xA;&lt;li&gt;If there are too many peaks(I say 4 is too many), go to a smaller section of the image and repeat the above steps&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Obviously a large problem here is that the Hough transform for detecting lines isn't supposed to detect curves. I thought zooming in on points of interest might fix this, but it has not. Some curves are just still too sharp, and if you zoom in too far the transform doesn't have enough data to make accurate calculations of where lines are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried some local feature detection, such as corner detection, to find the intersections. Local features fail miserably, and there are some images to explain why that is as well (although I am currently toying with another approach using gradients; will update post after I implement it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://res.cloudinary.com/turnipentropy/image/upload/v1493842851/localbad_c2vvc3.jpg&quot; alt=&quot;local doesn&amp;#39;t work&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The white curve isn't an intersection, but the blue and yellow curves obviously form an intersection. The curves, however, have a section of nearly identical local features when converted into a binary image. Any suggestions how to find these intersecting points would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="24153" LastActivityDate="2017-10-02T20:51:29.110" Title="Finding intersections of curves in a binary image" Tags="|image-processing|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="26779" PostTypeId="2" ParentId="26774" CreationDate="2017-05-04T06:34:55.150" Score="4" Body="&lt;p&gt;If averaging the element fluxes, as suggested by @Bill Greene, does not produce accurate enough fluxes, there are ways to improve the accuracy of the fluxes by post-processing. A standard and quite straightforward method, which is mentioned as an exercise in the book by Hughes T.J.R., &quot;The Finite Element Method&quot;, is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem is:&#xA;\begin{align*}&#xA;&amp;amp; \nabla^2 u = f \quad \text{in} \quad \Omega \\&#xA;&amp;amp; \frac{\partial u}{\partial n} = T \quad \text{on} \quad \Gamma_T &amp;amp;&amp;amp; \text {Neumann data} \\&#xA;&amp;amp; \frac{\partial u}{\partial n} = T^* \quad \text{on} \quad \Gamma_g &amp;amp;&amp;amp; \text {sought flux on the Dirichlet boundary} \\&#xA;&amp;amp; u = g \quad \text{on} \quad \Gamma_g &amp;amp;&amp;amp; \text {Dirichlet data}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you find the unknown fluxes by solving the corresponding weak problem:&#xA;Find $T^*\in L_2(\Gamma_g)$ such that for all $w\in H^{\frac{1}{2}}(\Gamma_g)$ there holds:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;\int_{\Gamma_g} wT^* d\Gamma = \int_\Omega w_{,i}u_{,i} d\Omega - &#xA;\int_{\Omega} wf d\Omega - \int_{\Gamma_T} wT d\Gamma,&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the comma followed by index $i$ denotes differentiation with respect to the coordinates (the summation convention is implied). Note that the Dirichlet data are not used here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You proceed to discretize this additional weak problem as you did for the main problem. &lt;/p&gt;&#xA;" OwnerUserId="6775" LastEditorUserId="6775" LastEditDate="2017-05-07T17:51:25.300" LastActivityDate="2017-05-07T17:51:25.300" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26781" PostTypeId="1" AcceptedAnswerId="26812" CreationDate="2017-05-04T07:52:56.523" Score="2" ViewCount="146" Body="&lt;p&gt;In case B(size ~ 2k, complex double) is block-diagonal, where block size is small(e.g. 2), is there any more efficient way to compute this other than Lapack gesv?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastActivityDate="2017-05-07T22:40:51.670" Title="Efficient computation of AX=B where B has special structure (block-diagonal)" Tags="|linear-algebra|lapack|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26782" PostTypeId="1" CreationDate="2017-05-04T12:38:57.953" Score="0" ViewCount="579" Body="&lt;p&gt;Assume I have the following equation (light propagating in $z$-direction through the matter):&#xA;$$id_zu+d^2_ru=0$$&#xA;with $u(z, r)$ being a complex wave. The time scale in this equation is &#xA;$$t\equiv t_\text{lab}-zn_0/c$$&#xA;I know how to solve that equation using a Crank-Nicolson-algorithm: I can reformulate it with &#xA;$$\partial_zu_m^{n+\frac{1}{2}}=\frac{u^{n+1}_m-u_m^n}{dz}$$&#xA;and &#xA;$$\partial_r^2u_m^{n+\frac{1}{2}}=\frac{\partial_r^2u^{n+1}_m+\partial_r^2u^n_m}{2}$$&#xA;resulting in the equation&#xA;$$\left[1-i\cdot\frac{dz}{2}\partial_r^2\right]u_m^{n+1}=\left[1+i\cdot\frac{dz}{2}\partial_r^2\right]u_m^{n}$$&#xA;which can be solved using linear algebra. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what happens if I add a time dependence, such that the first equation changes to &#xA;$$id_zu+d_r^2u+d_t^2u=0$$&#xA;with $u(z, r, t)$? How do I then have to reformulate my approach from above, after $dt=dz\cdot n_0/c$? Or do I have to use a completely different approach?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastEditorUserId="20688" LastEditDate="2017-05-04T17:53:57.270" LastActivityDate="2018-08-03T19:01:17.053" Title="Solving an equation in space and time using the Crank-Nicolson approach" Tags="|pde|finite-difference|algorithms|crank-nicolson|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26784" PostTypeId="2" ParentId="26593" CreationDate="2017-05-04T13:02:37.147" Score="0" Body="&lt;p&gt;My main problem in the implementation was that I did not do the boundary check carefully enough, resulting in either way to large regimes, or too small ones.&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-05-04T13:02:37.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26785" PostTypeId="1" AcceptedAnswerId="26792" CreationDate="2017-05-04T13:04:03.357" Score="2" ViewCount="3012" Body="&lt;p&gt;I tried some python optimization functions and some of them needed Jacobian matrix prior for faster convergence. I understand Jacobians are basically transformation matrices that data from one space to another or carrying gradients information. Can someone explain me with some literature references, how the speed up is achieved?&lt;/p&gt;&#xA;" OwnerUserId="20723" LastActivityDate="2023-08-23T06:45:04.250" Title="How Jacobian matrix helps optimization faster?" Tags="|linear-algebra|optimization|jacobian|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26786" PostTypeId="1" AcceptedAnswerId="26789" CreationDate="2017-05-04T13:54:40.917" Score="5" ViewCount="5690" Body="&lt;p&gt;I am working with an Eigen::SparseMatrix matrix of type double. I would like to find the largest and the smallest eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A solution of the problem is to convert it to dense and then find its eigenvalues. Finally the maximum and the minimum in the spectrum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better efficient way to do it? Let say&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Possibly without converting it to dense&lt;/li&gt;&#xA;&lt;li&gt;Possibly without computing all the spectrum&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="24163" LastActivityDate="2017-05-04T23:16:48.170" Title="Eigen - Max and minimum eigenvalues of a sparse matrix" Tags="|sparse-matrix|eigenvalues|eigen|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26788" PostTypeId="2" ParentId="26786" CreationDate="2017-05-04T18:32:07.303" Score="1" Body="&lt;p&gt;You can use arpack [1], that implements the Arnoldi algorithm for computing eigenpairs. Since arpack communicates with client code only through matrix vector product, you can use your own matrix type (including eigen sparse matrix). There is also a version with C++ bindings [2] that may be easier to use with eigen. To compute the smallest eigenvalue, it may be interesting to factorize the matrix using a sparse factorization algorithm (SuperLU for non-symmetric, CHOLDMOD for symmetric), and use the factorization to compute the largest eigenvalues of M^-1 instead of the smallest eigenvalue of M (a technique known as spectral transform, that I used a while ago in [3], can have a dramatic impact on performances in some cases).&#xA;The whole algorithm (including coupling with SUPERLU) is implemented in my OpenNL library [4]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.caam.rice.edu/software/ARPACK/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://www.ime.unicamp.br/~chico/arpack++/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ime.unicamp.br/~chico/arpack++/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://alice.loria.fr/index.php/publications.html?Paper=ManifoldHarmonics@2008&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/index.php/publications.html?Paper=ManifoldHarmonics@2008&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] OpenNL:&lt;a href=&quot;http://alice.loria.fr/index.php/software/4-library/23-opennl.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/index.php/software/4-library/23-opennl.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2017-05-04T18:32:07.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26789" PostTypeId="2" ParentId="26786" CreationDate="2017-05-04T19:25:46.833" Score="8" Body="&lt;p&gt;There are two relatively convenient options for calculating&#xA;selected (e.g. a few largest or smallest) eigenvalues using Eigen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first is &lt;a href=&quot;https://spectralib.org/index.html&quot; rel=&quot;noreferrer&quot;&gt;Spectra&lt;/a&gt;, a header-only C++ library based on Eigen&#xA;that uses algorithms similar to ARPACK (implicitly-restarted &#xA;Arnoldi) to calculate a few eigensolutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is header-only, you simply download and include the&#xA;appropriate header files. The site contains several example problems to&#xA;get you started.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second option, if your matrix is symmetric,  is an&#xA;interface to a subset of ARPACK, that is part of the standard Eigen distribution, and is very easy to use. Here is a small snippet of&#xA;code to give you the basic idea of how to use this interface&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;unsupported/Eigen/ArpackSupport&amp;gt;&#xA;typedef Eigen::SparseMatrix&amp;lt;double&amp;gt; SparseMat;&#xA;typedef Eigen::SimplicialLDLT&amp;lt;SparseMat&amp;gt; SparseChol;&#xA;typedef Eigen::ArpackGeneralizedSelfAdjointEigenSolver &amp;lt;SparseMat, SparseChol&amp;gt; Arpack;&#xA;Arpack arpack;&#xA;// define sparse matrix A&#xA;SparseMat A;&#xA;//...&#xA;// calculate the two smallest eigenvalues&#xA;int nbrEigenvalues = 2;&#xA;arpack.compute(A, nbrEigenvalues, &quot;SM&quot;);&#xA;cout &amp;lt;&amp;lt; &quot;arpack eigenvalues\n&quot; &amp;lt;&amp;lt; arpack.eigenvalues().transpose() &amp;lt;&amp;lt; endl;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The downside of this approach is that you need to have an ARPACK library&#xA;built for your particular system. If you need to build ARPACK yourself, I&#xA;suggest this version, &lt;a href=&quot;https://github.com/opencollab/arpack-ng&quot; rel=&quot;noreferrer&quot;&gt;ARPACK-NG&lt;/a&gt;, because it has many bug fixes compared with the&#xA;original and more support for building on different platforms.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-05-04T23:16:48.170" LastActivityDate="2017-05-04T23:16:48.170" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26790" PostTypeId="2" ParentId="26758" CreationDate="2017-05-04T19:56:30.740" Score="1" Body="&lt;p&gt;Based on the comment by @TylerOlsen, I've applied the &lt;a href=&quot;http://www.boost.org/doc/libs/1_64_0/libs/graph/doc/cuthill_mckee_ordering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;boost implementation&lt;/a&gt; of the reverse Cuthill-McKee ordering on the same matrix from the original question and here is the result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2md19.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2md19.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The structure looks very similar, but the bandwidth is significantly smaller compared to the original matrix with COMSOL ordering. Also changing the starting vertex in the Cuthill-McKee algorithm (e.g. enforcing to use the same starting vertex as COMSOL) does not have a significant effect on the bandwidth for this particular mesh. I consider this to be even better result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I can't formally accept comments, I'm going to self-accept this answer, but of course the credit goes to @TylerOlsen.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-05-04T19:56:30.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26791" PostTypeId="1" AcceptedAnswerId="26813" CreationDate="2017-05-04T20:54:51.913" Score="0" ViewCount="79" Body="&lt;p&gt;I am computing some partitions in my work and would like to be able to extract informations in an automatic way when given a certain partition structure. To see what I have in mind, look at the photos attached below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HmdJ4s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HmdJ4s.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rm79Cs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rm79Cs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the partition I can easily extract contours in 2D or triangulations of the surface of the cells in 3D. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there any &quot;classic&quot;, not too complicated to implement methods which can put such objects (contours, surface triangulations) into classes which are similar modulo euclidean transformations?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In 2D the cells are close to being polygons, so extracting information in the number of neighbors can give a rough classification (pentagons, hexagons,...). In 3D, however things get more complicated and that is what interests me most. &lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2017-08-05T23:02:38.117" Title="Automatic classification of a family of 2D contours or closed surfaces in 3D" Tags="|partitioning|pattern-recognition|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26792" PostTypeId="2" ParentId="26785" CreationDate="2017-05-04T23:21:34.803" Score="13" Body="&lt;p&gt;You haven't told us exactly what optimization routine you're using, so it's difficult to provide a very specific answer to your question.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you don't supply your own Jacobian function then the optimization routine that you're using is presumably approximating the derivatives using a finite difference approximation scheme.  Computing finite difference approximations to the derivative requires many function evaluations and this slows down the optimization process.  Furthermore, the inaccuracy of such approximate derivatives can cause the algorithm to require more steps to converge to an optimal solution and thus run more slowly.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantages of using exact analytical derivatives rather than finite difference approximations are discussed in most textbooks on nonlinear programming.  See for example &lt;em&gt;Practical Optimization&lt;/em&gt; by Gill, Murray, and Wright.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2017-05-05T01:42:28.363" LastActivityDate="2017-05-05T01:42:28.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26794" PostTypeId="1" AcceptedAnswerId="26799" CreationDate="2017-05-05T06:56:04.607" Score="1" ViewCount="81" Body="&lt;p&gt;First, $AX=B$ is solved, so I have the LU factorization of $A$ computed already. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I need to solve $BX=A$. Is there any way to reuse this information (LAPACK &lt;code&gt;gesv&lt;/code&gt; was used to compute LU of $A$)? &lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="20688" LastEditDate="2017-05-05T07:31:13.377" LastActivityDate="2017-05-05T14:00:28.750" Title="Efficient computation of $BX=A$ when LU factorization of $A$ is given" Tags="|linear-algebra|lapack|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26797" PostTypeId="2" ParentId="26777" CreationDate="2017-05-05T11:48:29.600" Score="2" Body="&lt;p&gt;First of all, you have no exact mirroring, which can be clearly seen when superimposing your plots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/06vgY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/06vgY.png&quot; alt=&quot;superimposed plots&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest to consider the following to get a better idea of the possible mistake:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider different sets of parameters, in particular ones with a less symmetric dynamics, e.g., $κ=0.2$ and $τ=0.1$. If there is a disagreement between numerical and analytical results, it should be much clearer then.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I do not know what integrator you used, but if it relies on interpolating and adapts step sizes, the discontinuity of the Heaviside function may cause problems. I had to replace it with a steep sigmoidal function to numerically integrate the system with adaptive step sizes. (Whether my results agree with yours, I cannot say as you do not give any initial conditions or $τ$.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2017-05-05T11:48:29.600" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26798" PostTypeId="2" ParentId="25680" CreationDate="2017-05-05T12:57:01.767" Score="0" Body="&lt;p&gt;This is the work and data from Turek and group&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.featflow.de/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark2_re100.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.featflow.de/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark2_re100.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9781" LastActivityDate="2017-05-05T12:57:01.767" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26799" PostTypeId="2" ParentId="26794" CreationDate="2017-05-05T14:00:28.750" Score="2" Body="&lt;p&gt;Your question amounts to asking which is the optimal solution strategy for computing $X,Y$ such that &#xA;$$&#xA;\left\{&#xA;\begin{aligned}&#xA;AX &amp;amp;= B\\&#xA;BY &amp;amp;= A&#xA;\end{aligned}&#xA;\right.&#xA;$$&#xA;where $A, B$ are generic square dense matrices.&#xA;Apart from the observation that if $A$ and $B$ are non singular, then $Y=X^{-1}$, I do not see any particular reason for which the knowledge of $X$ should help in the computation of $Y$. But I'd love to be contradicted on this.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2017-05-05T14:00:28.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26800" PostTypeId="1" CreationDate="2017-05-05T19:47:17.167" Score="1" ViewCount="98" Body="&lt;p&gt;I'm trying to use newton-raphson method for nonlinear systems of equations as described in 'Numerical recipies' book in chapter 9.6 to solve the hydrostatic equation for a polytropic star.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each iteration, I change the radius vector as described in the book like x→x+Δx(Δ can be calculated for each iteration) but I can't ignore the fact that it can do shell crossing - meaning the i shell moves to a higher shell then i+1i+1, this cause a negative volume. I know there is a method to avoid shell crossing by multiplying delta with a factor but I don't know how to calculate it. There is no description in the book about shell crossing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;let me be more specific:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve $F_i(x_1,....x_N)=0$ set of equations,( when i goes from 1 to N).&#xA;the method is to expand $F$ to a taylor series:&#xA;$\vec{F}(\vec{x}+\vec{\delta x})=\vec{F}+J\vec{\delta x} +O(x^2)$&#xA;, where $J$ is the partial deriviatives of $F$. by setting $\vec{F}(\vec{x}+\vec{\delta x})=0 $  we can get N sets of equations : $J \cdot \vec{\delta x}=-F$ that can be solved by LU decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the method starts from initial point $x_{old}$. for each step we calculate $\delta x$ and add it to $x_{old}$ , meaning : $x_{new}=x_{old}+\delta x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if the newton step $\delta x$ minimizes $f=0.5\cdot F\cdot F$ we can say this step is acceptible. By using line searches and backtracking we can guaranty to minimize $f$. Be aware that every solution to $F_i(x_1,....x_N)=0$  is a minimum of $f$ but not the oposite, so we must check if $f$ goes to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My set of equations are:&#xA;$F_i=\frac{dP_i/dm_i}{Gm_i/4\pi r_i^4}+1=0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\delta P_i=P_{i-1}-P_i$ , $\delta m_i =0.5\cdot (m_{i+1}-m_{i-1})$ , and from the polytropic equetion: $P_i=K(\frac{m_i-m_{i-1}}{4\pi / 3 (r_i^3-r_{i-1}^3)})^\gamma$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where do i stuck? in the first iteration. and it's suppose to be bad. the problem is shell crossing. $\delta x$ changes $r$ so much that it changes the order of the cells. meaning, $r_i$ the radius of the star (from zero) is suddenly bigger then $r_{i+1}$.  I know there is a method to correct this by multiplying $\delta$ by some factor. How to calculate it? Is there any other methods to correct it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do anyone know how to help?&lt;/p&gt;&#xA;" OwnerUserId="24175" LastEditorUserId="24175" LastEditDate="2017-05-06T06:37:20.620" LastActivityDate="2017-05-06T06:37:20.620" Title="Using Newton-Raphson method to solve the hydrostatic equation" Tags="|newton-method|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26802" PostTypeId="2" ParentId="26778" CreationDate="2017-05-05T21:00:32.760" Score="0" Body="&lt;p&gt;I am still looking for a better answer, but I found a solution for smooth lines with little to no noise (so still fairly useless). Turns out that when a binary image such as the ones above is thinned, one of two things occurs at intersections: either the connected component is broken, or a pixel in the general area is connected to 2 pixels instead of 1. Solution: modified connected components&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//PLEASE NOTE: THE IMAGE MUST BE THINNED FIRST!! Read above paragraph&#xA;int width = img.length;&#xA;int height = img[0].length;&#xA;int intersections = 0;&#xA;for (int i = 1; i &amp;lt; width - 2; i++){&#xA;  for (int j = 1; j &amp;lt; height - 2; j++){&#xA;    if (img[i,j] &amp;gt; 50){&#xA;      //initialize a queue for this connected component&#xA;      ArrayList&amp;lt;Integer[]&amp;gt; queue = new ArrayList&amp;lt;&amp;gt;();&#xA;      queue.add({i, j});&#xA;      img[i, j] = 0;&#xA;      while (!queue.isEmpty()){&#xA;        Integer[] c = queue.remove(0);&#xA;        int count = 0;&#xA;        for (int x = -1; x &amp;lt; 2; x++){&#xA;          for (int y = -1; y &amp;lt; 2; y++){&#xA;            if (img[c[0]+ x, c[1] + y] &amp;gt; 50){&#xA;              queue.add({c[0] + x, c[1] + y});&#xA;              img[c[0] + x, c[1] + y] = 0&#xA;              //count the number of pixels connected to this one&#xA;              count += 1;&#xA;            }&#xA;          }&#xA;        }&#xA;        if (count &amp;gt; 1){&#xA;          //if more than 1 pixel was connected, it's at an intersection&#xA;          intersections += 1;&#xA;        }&#xA;      }&#xA;    }&#xA;    //add 1 intersection for every connected component found&#xA;    intersections += 1;&#xA;  }&#xA;}&#xA;//correct for the original number of connected components in the image&#xA;//minus 1; helps offset against tendency for false negatives&#xA;intersections -= original_number_of_connected_components - 1&#xA;return intersections / 2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above is written in Java to expose the logic, but I implemented it in IDL, so I haven't debugged this (i.e. no promises this code compiles when put inside an actual method. In fact it most definitely won't if you don't pass in an array as an image and the &lt;code&gt;original_number_of_connected_components&lt;/code&gt; parameter). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: This does a horrendous job, as one might expect, on jagged or noisy images. Still looking for a better solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I have a solution that does an OK job. It will be on GitHub on October 19th, when I become much less busy. It's off on some images that don't thin very well, but is usually in the same ball park, and does not work on self intersecting images at the moment. This answer will completely change on the 19th, and unless I receive a fairly fool proof solution by then, I will revamp this answer and accept it as correct.&lt;/p&gt;&#xA;" OwnerUserId="24153" LastEditorUserId="24153" LastEditDate="2017-09-27T20:17:26.787" LastActivityDate="2017-09-27T20:17:26.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26803" PostTypeId="1" CreationDate="2017-05-06T18:17:01.107" Score="2" ViewCount="202" Body="&lt;p&gt;Based on page $35$ of the book:&lt;br&gt;&#xA;(W. Bangerth and R. Rannacher, &quot;Adaptive Finite Element Methods for Solving Differential Equations&quot;, Birkhäuser, 2003,)&lt;br&gt;&#xA;for computing the error in  dual weighted residual method we can use a higher order interpolation  $Iu^{(2)}_{2h}$ of $u$, i.e.&lt;br&gt;&#xA;\begin{align}&#xA;e\approx Iu^{(2)}_{2h}-u_h, &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does higher order interpolation mean? Should we use fe approximation with higher order elements? The approximation in the method that I use is cost (moving least squares approximation),so is there any other way to find $e$ for example using interpolation commands in MATLAB?&lt;/p&gt;&#xA;" OwnerUserId="10403" LastEditorUserId="10403" LastEditDate="2017-05-06T18:55:53.273" LastActivityDate="2017-11-30T21:10:45.823" Title="Higher order interpolation in DWR method" Tags="|finite-element|interpolation|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26805" PostTypeId="1" CreationDate="2017-05-06T18:26:33.457" Score="3" ViewCount="89" Body="&lt;p&gt;I have a non-negative function $f(x) \ge 0$ defined on the interval $[a,b]$. I would like to have a finite-dimensional approximation to this function that is guaranteed to be non-negative on $[a,b]$. What would be a good way to do this (say assuming $f$ is a smooth function)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I want to do this is that I would like to solve a functional equation $T(f)=0$ by using a collocation method and $T$ is defined only for non-negative $f$.&lt;/p&gt;&#xA;" OwnerUserId="24179" LastEditorUserId="9667" LastEditDate="2017-05-06T19:44:04.013" LastActivityDate="2017-05-07T07:13:22.003" Title="Bounded approximation to a bounded function" Tags="|numerical-analysis|interpolation|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26806" PostTypeId="2" ParentId="26781" CreationDate="2017-05-06T19:43:24.213" Score="0" Body="&lt;p&gt;When solving $AX = B$ with dense $A$, the fact that $B$ is sparse &lt;em&gt;does not&lt;/em&gt; imply that $X$ is sparse: this means that when computing $X$ little can be gained from the structure/sparsity of $B$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true (as noted by Kirill) that &lt;del&gt;with a custom implementation of the back solve step&lt;/del&gt; in the forward substitution step some floating point operations can be saved (since you know that some entries in the intermediate result are zero) but this implies introducing a &lt;code&gt;for&lt;/code&gt; loop and multiple calls to &lt;code&gt;trsm&lt;/code&gt;. I'm still convinced that a single call to &lt;code&gt;trsm&lt;/code&gt; on the whole $L$ and $B$ compares favourably to multiple calls to &lt;code&gt;trsm&lt;/code&gt; on sub-blocks of $L$ and $B$, unless you carefully optimise the block size of the custom implementation. (And of course, you have to take into account pivoting...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottom line: you can save some operations, but to gain efficiency the implementation is still (in my opinion) non-trivial.&lt;/p&gt;&#xA;" OwnerUserId="1744" LastEditorUserId="1744" LastEditDate="2017-05-07T22:40:51.670" LastActivityDate="2017-05-07T22:40:51.670" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26808" PostTypeId="2" ParentId="26805" CreationDate="2017-05-06T23:37:42.750" Score="0" Body="&lt;p&gt;My first thought is to use a logarithmic change of variables, $g(x) = \ln f(x)$, then rewrite your functional equation in terms of $g$. Of course, this can be problematic if $f$ is zero at many points. If it's acceptable for your use case, you can put a lower bound on the log-transformed function,&#xA;$$g(x) = \ln \max\bigl(f(x), C\bigr)$$&#xA;where $C$ is some small positive constant like $10^{-100}$ or $10^{-40}$. This means that $g$ will no longer be smooth, but hopefully you can still come up with a smooth approximation to it that will be always nonnegative.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2017-05-06T23:37:42.750" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26809" PostTypeId="2" ParentId="26805" CreationDate="2017-05-07T07:13:22.003" Score="1" Body="&lt;p&gt;You could try approximation by Bernstein polynomials. For the interval $[0,1]$, they are defined by $$B_n(f)(x)=\sum^n_{k=0}\binom{n}{k}\,f\left(\frac{k}{n}\right)\,x^k\,(1-x)^{n-k}.$$ Those are naturally non-negative for non-negative $f$. Approximation order is not exciting, though, and the resulting system of equations may have bad condition.&lt;/p&gt;&#xA;" OwnerUserId="24183" LastActivityDate="2017-05-07T07:13:22.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26812" PostTypeId="2" ParentId="26781" CreationDate="2017-05-07T17:21:59.440" Score="4" Body="&lt;p&gt;Supposing you already have an LU factorization, you can save a half of a forward substitution step. In the system $Lx=b$, you would have&#xA;$$ \begin{pmatrix}&#xA;l_{11}&amp;amp;0&amp;amp;&amp;amp;&amp;amp;&amp;amp;\cdots&amp;amp;0\\&#xA;\vdots&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;0\\ l_{k-1,1}&amp;amp;\cdots&amp;amp;l_{k-1,k-1}&amp;amp;0&amp;amp;\cdots&amp;amp;\cdots&amp;amp;0\\&#xA;l_{k,1}&amp;amp;\cdots&amp;amp;l_{k,k-1}&amp;amp;l_{k,k}&amp;amp;0&amp;amp;\cdots&amp;amp;0\\&#xA;\vdots&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;\vdots\\&#xA;l_{n1}&amp;amp;\cdots&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;l_{nn}&#xA;\end{pmatrix}&#xA;\begin{pmatrix} x_1\\\vdots\\x_{k-1}\\x_k\\\vdots\\x_n\end{pmatrix}&#xA;= &#xA;\begin{pmatrix} 0\\\vdots\\0\\b_k\\\vdots\\b_n \end{pmatrix}&#xA;$$&#xA;and the observation is that only the submatrix $L[k:n,k:n]$ matters whenever the rhs vector $b$ has $b[1:k-1]=0$. (It makes no difference if only $b_k$ is nonzero, or if $b_{k:n}$ are nonzero.) The elements of $x$ corresponding to the first $k-1$ zero elements of $b$ all vanish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So instead of invoking the usual procedure (in blas, this is trsm—you shouldn't have to write this yourself, and if you do it may well be slower) on the whole matrix, you invoke it only on the bottom-right submatrix of the right size, because you know that $x_1=\cdots=x_{k-1}=0$. &#xA;For a $2\times2$ block-diagonal matrix, you can solve for two rhs $b$ vectors at a time with the same position of the first nonzero. This saves on average $\frac12$ of the work, averaged over all the $b$'s, and I believe this is the best you can do.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-05-07T17:21:59.440" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26813" PostTypeId="2" ParentId="26791" CreationDate="2017-05-07T22:31:06.973" Score="0" Body="&lt;p&gt;I found a way to do the classification in 3D. It is not very rigorous, but it works very nice and it is not too expensive (less than one minute for 120 cells, which could probably be optimized further). The main idea is to use the &quot;heuristic&quot; that similar surfaces have similar Laplace-Beltrami spectrum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The procedure is as follows: for each cell&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;compute a triangulation of the surface&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;compute the first 10 (for example) Laplace-Beltrami eigenvalues on this triangulation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Normalize the vector of eigenvalues (since dilated surfaces have rescaled eigenvalues)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once I have these vectors for each cell, I compute the norms of the pairwise differences and I look for all norm differences below a certain treshold (in my case 0.01 seems to work fine). I applied this algorithm successfully on the 3D partition in the question and I obtained as expected four classes containing: 4 elements (corners), 36 elements (cells on the sides), 60 elements (cells in the interior of faces), 20 elements (cells in the interior of the pyramid).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it seems like the spectrum is a nice indicator of likeliness.  &lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2017-05-07T22:31:06.973" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26814" PostTypeId="1" AcceptedAnswerId="26830" CreationDate="2017-05-08T06:02:44.420" Score="2" ViewCount="374" Body="&lt;p&gt;Is this is plausible at all? It seems the most obvious/naive approach, so there's probably good reasons why it's not used - what are they?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Viscosity is not important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with inviscid Navier Stokes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \frac{\partial \mathbf u}{\partial t} +&#xA;    \mathbf u \cdot \nabla \mathbf u +&#xA;    \frac{1}{\rho} \nabla p = \mathbf f&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conserve mass by tracking density as a state variable. We advect it and update it with the divergence of the velocity:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \frac{\partial \rho}{\partial t} =&#xA;    - \mathbf u \cdot \nabla \rho&#xA;    - \rho ( \nabla \cdot \mathbf u )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, calculate pressure from density:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;    p = \frac{1}{\alpha} ( \frac{\rho}{\rho_0} - 1 )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we have pressure $p$ and we can use its gradient to calculate the change in $\mathbf u$ from the first Navier Stokes equation (rearranged from above)&#xA;$$&#xA;  \frac{\partial \mathbf u}{\partial t} =&#xA;    - \mathbf u \cdot \nabla \mathbf u&#xA;    - \frac{1}{\rho} \nabla p&#xA;   + \mathbf f&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea is similar to simulating the Shallow Water equations, where the velocity divergence is used to update the height of the water. The height is like a 2d density (mass per area). The weight of the fluid causes pressure. Differences in pressure (gradient) cause accelerations in the velocity field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't seem to get any eddies/vorticity from simulating the Shallow Water equations, so I wonder if there's some fundamental problem with this idea... &lt;/p&gt;&#xA;" OwnerUserId="24091" LastActivityDate="2017-05-11T18:12:15.507" Title="Can we simulate compressible flows by simple direct explicit calculation, without solving systems of linear equations (such as Poisson eq)?" Tags="|fluid-dynamics|navier-stokes|vorticity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26815" PostTypeId="1" CreationDate="2017-05-08T06:21:17.330" Score="0" ViewCount="409" Body="&lt;p&gt;I am trying to implement a fem code on tet10 elements. I do not prefer to use open source at the moment as I would like to have the basic feel over the algorithm. I closely follow the lecture notes for tet10 implementation at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch10.d/AFEM.Ch10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch10.d/AFEM.Ch10.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when I test my code on the example tetrahedron given above I get a very different stiffness matrix than given in the document.  I however get a match on the trend of the Eigen values for my stiffness matrix. I have included a test code here to just compute the basic stiffness matrix. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #include&amp;lt;cstdlib&amp;gt;&#xA;    #include&amp;lt;iostream&amp;gt;&#xA;    #include&amp;lt;fstream&amp;gt;&#xA;    #include&amp;lt;ctime&amp;gt;&#xA;    #include&amp;lt;cmath&amp;gt;&#xA;    #include&amp;lt;string&amp;gt;&#xA;    #include&amp;lt;vector&amp;gt;&#xA;    #include&amp;lt;unistd.h&amp;gt;&#xA;    #include &amp;lt;iterator&amp;gt;&#xA;    #include &amp;lt;fstream&amp;gt;&#xA;    using namespace std;&#xA;    #include&amp;lt;Eigen/Core&amp;gt;&#xA;    #include&amp;lt;Eigen/Dense&amp;gt;&#xA;    #include&amp;lt;Eigen/Eigenvalues&amp;gt; &#xA;    #include&amp;lt;Eigen/SVD&amp;gt;&#xA;    #include&amp;lt;Eigen/Sparse&amp;gt;&#xA;    #include&amp;lt;Eigen/SparseCholesky&amp;gt;&#xA;    #include&amp;lt;Eigen/SparseLU&amp;gt;&#xA;    using namespace Eigen;&#xA;    typedef Eigen::SparseMatrix&amp;lt;double&amp;gt; SpMat;&#xA;    struct NodeInfo&#xA;     {&#xA;      double x,y,z;&#xA;     };&#xA;    struct ElementInfo&#xA;    {&#xA;      int node[10];&#xA;     double volume, area[4],nx[4],ny[4],nz[4];&#xA;    };&#xA;   struct constants&#xA;   {&#xA;    double YoungsModulus, PoissonRatio, Density, gravity;&#xA;   };&#xA;   std::istream&amp;amp; operator&amp;gt;&amp;gt;(std::istream&amp;amp; is, ElementInfo&amp;amp; element)&#xA;   {&#xA;    is &amp;gt;&amp;gt; element.node[0] &amp;gt;&amp;gt; element.node[1] &amp;gt;&amp;gt; element.node[2] &amp;gt;&amp;gt;      element.node[3] &#xA;   &amp;gt;&amp;gt; element.node[4] &amp;gt;&amp;gt; element.node[5] &amp;gt;&amp;gt; element.node[6] &amp;gt;&amp;gt; element.node[7]&#xA;   &amp;gt;&amp;gt; element.node[8] &amp;gt;&amp;gt; element.node[9] &amp;gt;&amp;gt; element.volume&#xA;   &amp;gt;&amp;gt; element.nx[0] &amp;gt;&amp;gt; element.ny[0] &amp;gt;&amp;gt;element.nz[0] &amp;gt;&amp;gt; element.area[0] &#xA;   &amp;gt;&amp;gt; element.nx[1] &amp;gt;&amp;gt; element.ny[1] &amp;gt;&amp;gt;element.nz[1] &amp;gt;&amp;gt; element.area[1]&#xA;   &amp;gt;&amp;gt; element.nx[2] &amp;gt;&amp;gt; element.ny[2] &amp;gt;&amp;gt;element.nz[2] &amp;gt;&amp;gt; element.area[2]&#xA;   &amp;gt;&amp;gt; element.nx[3] &amp;gt;&amp;gt; element.ny[3] &amp;gt;&amp;gt;element.nz[3] &amp;gt;&amp;gt; element.area[3];&#xA;  return is;&#xA;  }&#xA;   std::istream&amp;amp; operator&amp;gt;&amp;gt;(std::istream&amp;amp; is, NodeInfo&amp;amp; node)&#xA;  {&#xA;    is &amp;gt;&amp;gt; node.x &amp;gt;&amp;gt;node.y &amp;gt;&amp;gt; node.z ;&#xA;    return is;&#xA;  }&#xA;&#xA;  void ComputeElasticityMat(Eigen::MatrixXd&amp;amp; El, constants&amp;amp; M);&#xA;  void ComputeElementStiffness(ElementInfo&amp;amp; ithEle, vector&amp;lt;NodeInfo&amp;gt;&amp;amp; node,&#xA;  Eigen::MatrixXd&amp;amp; K, Eigen::MatrixXd&amp;amp; E, double (*eta)[4], double weight);&#xA;&#xA;int main()&#xA; {&#xA;char filename[] = &quot;elements.dat&quot;;&#xA;char filename1[] =&quot;nodes.dat&quot;;&#xA;int nelem, nnodes, TotalDOF;&#xA;int ndf=3;//number of degrees of freedom at each node&#xA;int npe=10;//number of points per element&#xA;&#xA;// Gauss Integration points&#xA;double alfa=(5.0+3.0*sqrt(5.0))/20;&#xA;double beta =(5.0-sqrt(5.0))/20;&#xA;double weight=1.0/4.0;&#xA;double GaussPoints[4][4]={{alfa,beta,beta,beta},{beta,alfa,beta,beta},  {beta,beta,alfa,beta},{beta,beta,beta,alfa}};&#xA;&#xA;// To define the element properties&#xA;constants MaterialProperties={.YoungsModulus=480.0,.PoissonRatio=1.0/3.0, .Density=2830, .gravity=9.81}; &#xA;// Read the data file for the elements and the nodes&#xA;std::vector&amp;lt;ElementInfo&amp;gt; Elem;&#xA;std::vector&amp;lt;NodeInfo&amp;gt; Node;&#xA;std::ifstream ifs(filename), ifs1(filename1);&#xA;ifs&amp;gt;&amp;gt;nelem;&#xA;cout&amp;lt;&amp;lt;&quot;The total number of elements is=&quot;&amp;lt;&amp;lt;nelem&amp;lt;&amp;lt;endl;&#xA;if (ifs) {&#xA;    std::copy(std::istream_iterator&amp;lt;ElementInfo&amp;gt;(ifs), &#xA;            std::istream_iterator&amp;lt;ElementInfo&amp;gt;(),&#xA;            std::back_inserter(Elem));&#xA;}&#xA;else {&#xA;    std::cerr &amp;lt;&amp;lt; &quot;Couldn't open &quot; &amp;lt;&amp;lt; filename &amp;lt;&amp;lt; &quot; for reading\n&quot;;&#xA;}&#xA;cout&amp;lt;&amp;lt;&quot;Element file read sucessfuly  &quot;&amp;lt;&amp;lt;endl;&#xA;&#xA;  if (ifs1) {&#xA;    std::copy(std::istream_iterator&amp;lt;NodeInfo&amp;gt;(ifs1), &#xA;            std::istream_iterator&amp;lt;NodeInfo&amp;gt;(),&#xA;            std::back_inserter(Node));&#xA;}&#xA;else {&#xA;    std::cerr &amp;lt;&amp;lt; &quot;Couldn't open &quot; &amp;lt;&amp;lt; filename1 &amp;lt;&amp;lt; &quot; for reading\n&quot;;&#xA;}&#xA;    cout&amp;lt;&amp;lt;&quot;Node file read sucessfuly  &quot;&amp;lt;&amp;lt;endl;&#xA;&#xA;  cout&amp;lt;&amp;lt;&quot;The total number of Nodes is=&quot;&amp;lt;&amp;lt;Node.size() &amp;lt;&amp;lt;endl;&#xA;  nnodes=Node.size();&#xA;&#xA;  //To form the Elasticity Matrix&#xA;  TotalDOF=nnodes*ndf;&#xA;  cout&amp;lt;&amp;lt;&quot;The Total number of Degrees of freedom=:\n&quot;&amp;lt;&amp;lt;TotalDOF&amp;lt;&amp;lt;endl;&#xA;&#xA;&#xA;   MatrixXd EelemMat(6,6), KelemMat(30,30),  KGl(TotalDOF, TotalDOF);&#xA;   VectorXd BodyForce(30), BFGl(TotalDOF), SurfaceForce(30);&#xA;   EelemMat.setZero(6,6);&#xA;&#xA;   ComputeElasticityMat(EelemMat,MaterialProperties);&#xA;   cout&amp;lt;&amp;lt;&quot;Elasticity Matrix=:\n&quot;&amp;lt;&amp;lt;EelemMat&amp;lt;&amp;lt;endl;&#xA;&#xA;   //To find the Stiffness Matrix&#xA;   if(nelem!=Elem.size()){&#xA;   cout&amp;lt;&amp;lt;&quot;Something wrong in element file...please check!&quot;&amp;lt;&amp;lt;endl;&#xA;   exit(0);&#xA;  }&#xA;&#xA; for(int i=0;i&amp;lt;nelem;i++)&#xA;   {&#xA;    KelemMat.setZero(30,30);&#xA;       ComputeElementStiffness(Elem[i],Node,KelemMat,&#xA;       EelemMat,GaussPoints,weight);&#xA;&#xA;      cout&amp;lt;&amp;lt;&quot;Stiffness Matrix=&quot;&amp;lt;&amp;lt;KelemMat&amp;lt;&amp;lt;endl;&#xA;   }&#xA;   JacobiSVD&amp;lt;MatrixXd&amp;gt; svd(KelemMat, ComputeThinU | ComputeThinV);&#xA;   return 0;    &#xA; }   &#xA;&#xA; void ComputeElasticityMat(Eigen::MatrixXd&amp;amp; El, constants&amp;amp; M)&#xA;  {&#xA;   double K=M.YoungsModulus/((1.0+M.PoissonRatio)*  (1.0-2.0*M.PoissonRatio));&#xA;    El(0,0)=1.0-M.PoissonRatio;&#xA;    El(0,1)=M.PoissonRatio;&#xA;    El(0,2)=M.PoissonRatio;&#xA;&#xA;    El(1,0)=M.PoissonRatio;&#xA;    El(1,1)=1.0-M.PoissonRatio;&#xA;    El(1,2)=M.PoissonRatio;&#xA;&#xA;    El(2,0)=M.PoissonRatio;&#xA;    El(2,1)=M.PoissonRatio;&#xA;    El(2,2)=1.0-M.PoissonRatio;&#xA;&#xA;    El(3,3)=0.5-M.PoissonRatio;&#xA;    El(4,4)=El(3,3);&#xA;    El(5,5)=El(3,3);&#xA;&#xA;    El=El*K;&#xA;&#xA;    return;&#xA;  }&#xA;&#xA;&#xA;  void ComputeElementStiffness(ElementInfo&amp;amp; ithEle, vector&amp;lt;NodeInfo&amp;gt;&amp;amp; node, Eigen::MatrixXd&amp;amp; K,&#xA;  Eigen::MatrixXd&amp;amp; E, double (*eta)[4], double weight)&#xA; {&#xA;  double jx1,jx2,jx3,jx4,jy1,jy2,jy3,jy4,jz1,jz2,jz3,jz4,&#xA;       a1,a2,a3,a4,b1,b2,b3,b4,c1,c2,c3,c4,Jdet;&#xA;  VectorXd Nfx(10),Nfy(10),Nfz(10),xi(4);&#xA;  MatrixXd B(6,30), Bt; &#xA;  MatrixXd J(4,4), P, Iaug(4,3),Jinv(4,4);&#xA;  double v01,v02,v03,v04,V;&#xA; for(int intPoints=0;intPoints&amp;lt;4;intPoints++)&#xA;  {&#xA;    for(int j=0;j&amp;lt;4;j++)&#xA;       xi(j) = eta[intPoints][j];&#xA;&#xA;&#xA;       jx1=4.0*(node[ithEle.node[0]].x*(xi(0)-0.25)+node[ithEle.node[4]].x*xi(1)+node[ithEle.node[6]].x*xi(2)+node[ithEle.node[7]].x*xi(3));&#xA;       jy1=4.0*(node[ithEle.node[0]].y*(xi(0)-0.25)+node[ithEle.node[4]].y*xi(1)+node[ithEle.node[6]].y*xi(2)+node[ithEle.node[7]].y*xi(3));&#xA;       jz1=4.0*(node[ithEle.node[0]].z*(xi(0)-0.25)+node[ithEle.node[4]].z*xi(1)+node[ithEle.node[6]].z*xi(2)+node[ithEle.node[7]].z*xi(3));&#xA;&#xA;       jx2=4.0*(node[ithEle.node[4]].x*xi(0)+node[ithEle.node[1]].x*(xi(1)-0.25)+node[ithEle.node[5]].x*xi(2)+node[ithEle.node[8]].x*xi(3));&#xA;       jy2=4.0*(node[ithEle.node[4]].y*xi(0)+node[ithEle.node[1]].y*(xi(1)-0.25)+node[ithEle.node[5]].y*xi(2)+node[ithEle.node[8]].y*xi(3));&#xA;       jz2=4.0*(node[ithEle.node[4]].z*xi(0)+node[ithEle.node[1]].z*(xi(1)-0.25)+node[ithEle.node[5]].z*xi(2)+node[ithEle.node[8]].z*xi(3));&#xA;&#xA;       jx3=4.0*(node[ithEle.node[6]].x*xi(0)+node[ithEle.node[5]].x*xi(1)+node[ithEle.node[2]].x*(xi(2)-0.25)+node[ithEle.node[9]].x*xi(3));&#xA;       jy3=4.0*(node[ithEle.node[6]].y*xi(0)+node[ithEle.node[5]].y*xi(1)+node[ithEle.node[2]].y*(xi(2)-0.25)+node[ithEle.node[9]].y*xi(3));&#xA;       jz3=4.0*(node[ithEle.node[6]].z*xi(0)+node[ithEle.node[5]].z*xi(1)+node[ithEle.node[2]].z*(xi(2)-0.25)+node[ithEle.node[9]].z*xi(3));&#xA;&#xA;       jx4=4.0*(node[ithEle.node[7]].x*xi(0)+node[ithEle.node[8]].x*xi(1)+node[ithEle.node[9]].x*xi(2)+node[ithEle.node[3]].x*(xi(3)-0.25));&#xA;       jy4=4.0*(node[ithEle.node[7]].y*xi(0)+node[ithEle.node[8]].y*xi(1)+node[ithEle.node[9]].y*xi(2)+node[ithEle.node[3]].y*(xi(3)-0.25));&#xA;       jz4=4.0*(node[ithEle.node[7]].z*xi(0)+node[ithEle.node[8]].z*xi(1)+node[ithEle.node[9]].z*xi(2)+node[ithEle.node[3]].z*(xi(3)-0.25));&#xA;&#xA;       J.row(0)&amp;lt;&amp;lt;1,1,1,1;&#xA;       J.row(1)&amp;lt;&amp;lt;jx1,jx2,jx3,jx4;&#xA;       J.row(2)&amp;lt;&amp;lt;jy1,jy2,jy3,jy4;&#xA;       J.row(3)&amp;lt;&amp;lt;jz1,jz2,jz3,jz4;&#xA;&#xA;       Jdet=J.determinant();&#xA;       Jinv=J.inverse();&#xA;&#xA;       Iaug&amp;lt;&amp;lt;0,0,0,&#xA;             1,0,0,&#xA;             0,1,0,&#xA;             0,0,1;&#xA;&#xA;       P=Jinv*Iaug;&#xA;&#xA;       a1=P(0,0);&#xA;       a2=P(1,0);&#xA;       a3=P(2,0);&#xA;       a4=P(3,0);&#xA;&#xA;       b1=P(0,1);&#xA;       b2=P(1,1);&#xA;       b3=P(2,1);&#xA;       b4=P(3,1);&#xA;&#xA;       c1=P(0,2);&#xA;       c2=P(1,2);&#xA;       c3=P(2,2);&#xA;       c4=P(3,2);&#xA;&#xA;       Nfx &amp;lt;&amp;lt; (4.0*xi(0)-1)*a1, (4.0*xi(1)-1)*a2, (4.0*xi(2)-1)*a3, (4.0*xi(3)-1)*a4,&#xA;              4.0*(a1*xi(1)+a2*xi(0)), 4.0*(a2*xi(2)+a3*xi(1)), 4.0*(a1*xi(2)+a3*xi(0)),&#xA;              4.0*(a1*xi(3)+a4*xi(0)), 4.0*(a2*xi(3)+a4*xi(1)), 4.0*(a3*xi(3)+a4*xi(2));&#xA;&#xA;       Nfy &amp;lt;&amp;lt; (4.0*xi(0)-1)*b1, (4.0*xi(1)-1)*b2, (4.0*xi(2)-1)*b3, (4.0*xi(3)-1)*b4,&#xA;              4.0*(b1*xi(1)+b2*xi(0)), 4.0*(b2*xi(2)+b3*xi(1)), 4.0*(b1*xi(2)+b3*xi(0)),&#xA;              4.0*(b1*xi(3)+b4*xi(0)), 4.0*(b2*xi(3)+b4*xi(1)), 4.0*(b3*xi(3)+b4*xi(2));&#xA;&#xA;       Nfz &amp;lt;&amp;lt; (4.0*xi(0)-1)*c1, (4.0*xi(1)-1)*c2, (4.0*xi(2)-1)*c3, (4.0*xi(3)-1)*c4,&#xA;              4.0*(c1*xi(1)+c2*xi(0)), 4.0*(c2*xi(2)+c3*xi(1)), 4.0*(c1*xi(2)+c3*xi(0)),&#xA;              4.0*(c1*xi(3)+c4*xi(0)), 4.0*(c2*xi(3)+c4*xi(1)), 4.0*(c3*xi(3)+c4*xi(2));          &#xA;&#xA;       B.row(0) &amp;lt;&amp;lt; Nfx(0),0,0, Nfx(1),0,0, Nfx(2),0,0, Nfx(3),0,0, Nfx(4),0,0, Nfx(5),0,0, Nfx(6),0,0, Nfx(7),0,0, Nfx(8),0,0, Nfx(9),0,0;&#xA;       B.row(1) &amp;lt;&amp;lt; 0,Nfy(0),0, 0,Nfy(1),0, 0,Nfy(2),0, 0,Nfy(3),0, 0,Nfy(4),0, 0,Nfy(5),0, 0,Nfy(6),0, 0,Nfy(7),0, 0,Nfy(8),0, 0,Nfy(9),0;&#xA;       B.row(2) &amp;lt;&amp;lt; 0,0,Nfz(0), 0,0,Nfz(1), 0,0,Nfz(2), 0,0,Nfz(3), 0,0,Nfz(4), 0,0,Nfz(5), 0,0,Nfz(6), 0,0,Nfz(7), 0,0,Nfz(8), 0,0,Nfz(9);&#xA;       B.row(3) &amp;lt;&amp;lt; Nfy(0),Nfx(0),0,  Nfy(1),Nfx(1),0, Nfy(2),Nfx(2),0, Nfy(3),Nfx(3),0, Nfy(4),Nfx(4),0, Nfy(5),Nfx(5),0, Nfy(6),Nfx(6),0,&#xA;                   Nfy(7),Nfx(7),0,Nfy(8),Nfx(8),0,  Nfy(9),Nfx(9),0;&#xA;       B.row(4) &amp;lt;&amp;lt; 0,Nfz(0),Nfy(0), 0,Nfz(1),Nfy(1), 0,Nfz(2),Nfy(2), 0,Nfz(3),Nfy(3), 0,Nfz(4),Nfy(4), 0,Nfz(5),Nfy(5), 0,Nfz(6),Nfy(6),&#xA;                   0,Nfz(7),Nfy(7), 0,Nfz(8),Nfy(8), 0,Nfz(9),Nfy(9);&#xA;       B.row(5) &amp;lt;&amp;lt; Nfz(0),0,Nfx(0), Nfz(1),0,Nfx(1), Nfz(2),0,Nfx(2), Nfz(3),0,Nfx(3), Nfz(4),0,Nfx(4), Nfz(5),0,Nfx(5), Nfz(6),0,Nfx(6),&#xA;                   Nfz(7),0,Nfx(7), Nfz(8),0,Nfx(8), Nfz(9),0,Nfx(9);&#xA;&#xA;       Bt = B.transpose();&#xA;       K += weight*(Bt*E*B*Jdet/6.0);&#xA;       cout&amp;lt;&amp;lt;&quot;Jdet=&quot;&amp;lt;&amp;lt;Jdet&amp;lt;&amp;lt;endl;&#xA; }&#xA;   return;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The element file (elements.dat) for the problem is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    1&#xA;    0     1     2    3    4    5    6    7    8    9      &#xA;    24.0  -0.6295797707 -0.7709500359   -0.0962567111   0.0018562616&#xA;   -0.5930703716    0.2172077372    -0.7752988670   0.0051374961&#xA;    0.9488273288    -0.1721170560   0.2647686142    0.0037463536&#xA;    0.1956924749    0.2842456298    0.9385674601    0.0033773339&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The node file (nodes.dat)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 2.0000000000   3.0000000000    4.0000000000    &#xA; 6.0000000000   3.0000000000    2.0000000000    &#xA; 2.0000000000   5.0000000000    1.0000000000    &#xA; 4.0000000000   3.0000000000    6.0000000000    &#xA; 4.0000000000   3.0000000000    3.0000000000    &#xA; 4.0000000000   4.0000000000    1.5000000000    &#xA; 2.0000000000   4.0000000000    2.5000000000    &#xA; 3.0000000000   3.0000000000    5.0000000000    &#xA; 5.0000000000   3.0000000000    4.0000000000    &#xA; 3.0000000000   4.0000000000    3.5000000000    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The correct stiffness matrix as given in figure 10.8 is just for the first three values &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   KelemMat=[447, 324,72.....]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;whereas I get &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   KelemMat=[ 70.617   134.209  -108.193 .... ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One more thing is it OK to get a negative determinant at a Gaussian point..I do get one&lt;/p&gt;&#xA;" OwnerUserId="23149" LastEditorUserId="23149" LastEditDate="2017-05-16T07:23:40.433" LastActivityDate="2017-06-22T09:37:24.327" Title="FEM implementation on tet10 element stiffness matrix:(code snippet provided)" Tags="|finite-element|" AnswerCount="1" CommentCount="5" ClosedDate="2017-06-27T15:33:06.903" ContentLicense="CC BY-SA 3.0" />
  <row Id="26818" PostTypeId="2" ParentId="26814" CreationDate="2017-05-08T16:20:31.113" Score="1" Body="&lt;p&gt;Of course you can apply some explicit scheme to discretize the equations. The explicit approach is just impractical for stiff problems due to stability constraints on the time step. See the &lt;a href=&quot;https://en.wikipedia.org/wiki/Explicit_and_implicit_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;Explicit and implicit methods&lt;/a&gt; page on Wikipedia for more details.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-05-08T16:20:31.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26819" PostTypeId="1" AcceptedAnswerId="26839" CreationDate="2017-05-08T17:56:26.917" Score="0" ViewCount="139" Body="&lt;p&gt;This question is based upon a &lt;a href=&quot;https://arxiv.org/abs/cond-mat/9811408&quot; rel=&quot;nofollow noreferrer&quot;&gt;research article&lt;/a&gt; which I am trying to reproduce. One of the main result of this paper is the condition on transverse confinement of the Bose-Einstein Condensate(BEC) to make the black soliton solution stable. The equation governing BEC is the Gross-Pitaevskii(GP) equation, given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$i\hbar \frac{\partial\psi}{\partial t}=-\frac{\hbar^{2}}{2m}\frac{\partial^{2}\psi}{\partial x^{2}}+g\psi|\psi|^{2}+V_{ext}\: \psi$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $|\psi|^{2}$ gives the density of the condensate. We can see when $V_{ext}=0$, the above equation has a solution, in one dimension(say z), of the form $\psi(z,t)=\tanh{cz} \:e^{-i\mu t}$, where $c$ accounts for the constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose now that we are working in a cylindrical geometry, such that the $V_{ext}=\omega_{z}z^{2}+\omega_{r}r^{2}$ and $\omega_{z}&amp;lt;\omega_{r}$, meaning the radial confinement is stronger than the axial confinement. In such a case, one can obtain a solitonic solution with a nodal plane perpendicular to the axial direction. This can be done by using the split operator method and imaginary time evolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, comes the question of stability of the solitonic solution. One can perturb $\psi$ with a perturbation of the form $\psi\rightarrow\psi+\delta\psi$, where $\delta\psi = u(z)e^{iq.r-i\epsilon t}+v(z)e^{-iq.r+i\epsilon t}$. So, basically, we are looking for small amplitude oscillations where the soliton, whose nodal plane is perpendicular to the axial(z) direction, gives out energy in the radial direction. Putting the form of $\delta\psi$ in the GP equation, we get the following set of equations for $f_{\pm}(z)=u(z)\pm v(z)$, where $\psi_{0}(z)$ is the density profile of the soliton in the z-direction(axial direction) in the presence of the trap($V_{ext}$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\epsilon f_{-}(z)=\Big[-\frac{\hbar^{2}}{2m}\big(\frac{\partial^{2}}{\partial z^{2}}-q^{2}\big)-\mu+V_{ext}+3g\psi_{0}^{2}(z) \Big]f_{+}(z)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\epsilon f_{+}(z)=\Big[-\frac{\hbar^{2}}{2m}\big(\frac{\partial^{2}}{\partial z^{2}}-q^{2}\big)-\mu+V_{ext}+g\psi_{0}^{2}(z) \Big]f_{-}(z)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain a stability condition, we need a dispersion relation between $\epsilon$ and $q$. However, as you can see in the above set of equations, there are a lot of terms with $z$ dependences, including a derivative in the $z$ direction. The authors of the paper say that they have numerically solved these set of equations to obtain a dispersion relation. How does one do that?&lt;/p&gt;&#xA;" OwnerUserId="23125" LastEditorUserId="9667" LastEditDate="2017-05-10T21:35:11.910" LastActivityDate="2017-05-10T21:35:11.910" Title="Stability of dark solitons in a harmonic trap" Tags="|ode|eigensystem|quantum-mechanics|operator-splitting|differential-equations|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26821" PostTypeId="1" AcceptedAnswerId="26822" CreationDate="2017-05-08T19:42:54.817" Score="0" ViewCount="102" Body="&lt;p&gt;So I am trying a molecular dynamics simulation, and trying to populate a 3D rectangular volume with random particles which are uniformly distributed throughout the volume. Each molecule has a fixed radius r(sphere), where r can be different for each molecule. So I would want the sphere to be generated in the volume, and then no other sphere should exist within ( 2(r+tolerance) ) distance from that point(the tolerance will be really small, like 10^-6).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the channel will have a specified length, breadth and width, so the randomization should be done within those limits in a single direction with a similar condition above i.e. the molecule should not be generated if the molecule is closer to the wall than its radius+tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was initially trying a structured lattice, but that would mean I the number of molecules would be correlated to the dimensions, which is does not work for me. So I wrote the following algorithm(I wanted to paste my code, but it doesn't work and its a big, exaggerated mess right now, and has almost crashed my computer once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the logic is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) For each particle, make max_X=X-(radius+tol). Do it same for Y and Z.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) For each particle, generate random number between 0 and max_X.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.) calculate distance between each particle and create a list of particles which violate any of the above conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4.) Go over the list and re-generate those particles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5.) Create another list of violating particle pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6.) Go over. Rinse and repeat until the size of the list is zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, it doesn't work. And I need this code to perform eventually for a large number of particles, so I need it as efficient and OpenMP paralleizable as possible. I am using the standard cliched #pragma omp parallel for method in front of the for loop for parallelizing the calculations, but it doesn't work with or without the pragma, at runtime, after like 5-10 minutes of a superpowered fan noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am someone from a non-coding background who recently started learning writing complex code in C++, so I cannot do something fancy like structures or classes or pointers right now. I am working with std::vectors though. Would love it if you guys could show me a way out, and if you are in and around Raleigh, I will personally drive over and hand you an ice cold beer. Have been trying to do this for days, and is the only major non-functional thing in my code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Help?!!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: Just to be clear, this isn't homework, else I would have asked the professor by now. Its part of an independent project which I will be freely distributing after completion.&lt;/p&gt;&#xA;" OwnerUserId="23565" LastActivityDate="2017-05-09T00:04:02.323" Title="Generate a set of random x,y,z numbers, with a minimum difference between them, between defined limits" Tags="|parallel-computing|molecular-dynamics|random-number-generation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26822" PostTypeId="2" ParentId="26821" CreationDate="2017-05-09T00:04:02.323" Score="1" Body="&lt;p&gt;So one thing you can do is implicitly represent a structured region, like a 3D rectangular volume, as a set of chopped up sub-volumes where each sub-volume automatically enforces the spatial constraints you require. You will do this implicitly by just storing how many pieces to slice up each dimension into individually and referencing each sub-volume using a unique tuple of indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then compute how many slices you should have for each dimension, such that your spatial constraints are met, using your values for a particle's radius and the tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then from here you can iteratively Monte Carlo the next sub-volume to stick a particle in. To efficiently check if you Monte Carlo a sub-volume you have used already, you can compute a unique hash, given a sub-volume's tuple of indices, and store it inside an efficient data structure, like &lt;code&gt;std::map&lt;/code&gt; or &lt;code&gt;std::unordered_map&lt;/code&gt; in C++, to be able to check for existence against quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sample code I wrote in C++ for this is below:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;particle_cloud.hpp&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#ifndef _particle_cloud_&#xA;#define _particle_cloud_&#xA;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;cstdint&amp;gt;&#xA;&#xA;namespace particle {&#xA;&#xA;    class cloud {&#xA;    public:&#xA;&#xA;        // useful typedefs&#xA;        typedef std::vector&amp;lt;double&amp;gt;     position;&#xA;        typedef std::vector&amp;lt;position&amp;gt;   positions;&#xA;&#xA;        // ctors / initializers&#xA;        cloud();&#xA;        cloud(int num_particles, double xrng[2], double yrng[2], double zrng[2], double radius, double tolerance);&#xA;        void init(int num_particles, double xrng[2], double yrng[2], double zrng[2], double radius, double tolerance);&#xA;&#xA;        // getter for particle position&#xA;        const position &amp;amp; getParticlePositionAt(int idx) const;&#xA;&#xA;        // get number of particles in particle cloud&#xA;        int numParticles() const;&#xA;&#xA;    private:&#xA;        typedef uint64_t h_int;&#xA;        enum Bounds { Low = 0, High = 1};&#xA;        positions ps;&#xA;&#xA;        // unique hash function&#xA;        static h_int hash(h_int ix, h_int iy, h_int iz, h_int Nx, h_int Ny);&#xA;    };&#xA;&#xA;}&#xA;&#xA;&#xA;#endif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;particle_cloud.cpp&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &quot;particle_cloud.hpp&quot;&#xA;#include &amp;lt;map&amp;gt;&#xA;#include &amp;lt;random&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;&#xA;namespace particle {&#xA;&#xA;&#xA;    cloud::cloud()&#xA;    {&#xA;    }&#xA;&#xA;    cloud::cloud(int num_particles, double xrng[2], double yrng[2], double zrng[2], double radius, double tolerance)&#xA;    {&#xA;        init(num_particles, xrng, yrng, zrng, radius, tolerance);&#xA;    }&#xA;&#xA;    void cloud::init(int num_particles, double xrng[2], double yrng[2], double zrng[2], double radius, double tolerance)&#xA;    {&#xA;        // init constant quantities&#xA;        const double r = radius;&#xA;        const double e = tolerance;&#xA;        const double lb[3]      = { xrng[Low], yrng[Low], zrng[Low] };&#xA;        const double ub[3]      = { xrng[High], yrng[High], zrng[High] };&#xA;        const double span[3]    = { ub[0]- lb[0], ub[1]- lb[1], ub[2]- lb[2] };&#xA;        const h_int N[3]        = { std::floor(0.5*span[0]/(r+e)), std::floor(0.5*span[1] / (r + e)), std::floor(0.5*span[2] / (r + e)) };&#xA;        const double del[3]     = { span[0] / static_cast&amp;lt;double&amp;gt;(N[0]), span[1] / static_cast&amp;lt;double&amp;gt;(N[1]) , span[2] / static_cast&amp;lt;double&amp;gt;(N[2]) };&#xA;&#xA;        // init array of temp map indices&#xA;        h_int idx[3]            = { 0 };&#xA;&#xA;        // init temp position vector&#xA;        position tmp(3, 0.0);&#xA;&#xA;        // size up the positions container correctly&#xA;        ps.resize(num_particles, tmp);&#xA;&#xA;        // init the uniform distribution random number generator&#xA;        std::default_random_engine generator;&#xA;        std::uniform_real_distribution&amp;lt;double&amp;gt; U(0, 1);&#xA;&#xA;        // create a map container to check if a given hash&#xA;        // has already been created&#xA;        std::map&amp;lt;h_int, bool&amp;gt; exist_map;&#xA;&#xA;        // init variable that will say if the new position generated&#xA;        // is valid or not&#xA;        bool noGoodPos = false;&#xA;&#xA;        // loop and generate the particle positions&#xA;        for (unsigned int i = 0; i &amp;lt; num_particles; ++i) {&#xA;            noGoodPos = true;&#xA;&#xA;            // try to generate a valid position via Monte Carlo sampling until one has been made&#xA;            while (noGoodPos) {&#xA;                for (int d = 0; d &amp;lt; 3; ++d) { idx[d] = U(generator)*N[d]; }&#xA;                h_int h_ = hash(idx[0], idx[1], idx[2], N[0], N[1]);&#xA;&#xA;                if (exist_map.count(h_) != 0) { }&#xA;                else { &#xA;                    exist_map[h_] = true;&#xA;                    noGoodPos = false; &#xA;                }&#xA;            }&#xA;&#xA;            // compute each component of position&#xA;            // based on indices and dimensions of container&#xA;            for (int d = 0; d &amp;lt; 3; ++d) {&#xA;                tmp[d] = lb[d] + (del[d] * idx[d]) + (del[d] * 0.5);&#xA;            }&#xA;&#xA;            // copy temp position to ith position in positions container&#xA;            ps[i] = tmp;&#xA;        }&#xA;&#xA;    }&#xA;&#xA;    const typename cloud::position &amp;amp; cloud::getParticlePositionAt(int idx) const&#xA;    {&#xA;        return ps[idx];&#xA;    }&#xA;&#xA;    int cloud::numParticles() const&#xA;    {&#xA;        return (int)ps.size();&#xA;    }&#xA;&#xA;    typename cloud::h_int cloud::hash(h_int ix, h_int iy, h_int iz, h_int Nx, h_int Ny)&#xA;    {&#xA;        return ix + Nx*(iy + Ny*iz); // unique hash based on spatial indices&#xA;    }&#xA;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;main.cpp&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &quot;particle_cloud.hpp&quot;&#xA;&#xA;int main(int argc, char** argv) {&#xA;&#xA;    particle::cloud pc;&#xA;    double v[2] = { 0, 1 };&#xA;    pc.init(100000, v, v, v, 1e-4, 1e-8);&#xA;&#xA;    FILE * file_ = fopen(&quot;points.csv&quot;, &quot;w&quot;);&#xA;    if (file_) {&#xA;        for (int i = 0; i &amp;lt; pc.numParticles(); ++i) {&#xA;            const particle::cloud::position &amp;amp; p = pc.getParticlePositionAt(i);&#xA;            fprintf(file_, &quot;%0.8e, %0.8e, %0.8e\n&quot;, p[0], p[1], p[2]);&#xA;        }&#xA;&#xA;        fclose(file_); file_ = nullptr;&#xA;    }&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A sample graphic based on generating 1000 particle positions using the above code is shown below as well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7A3Hh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7A3Hh.png&quot; alt=&quot;Sample Particle Distribution using Algorithm&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-05-09T00:04:02.323" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26823" PostTypeId="1" CreationDate="2017-05-09T06:11:32.203" Score="1" ViewCount="171" Body="&lt;p&gt;Consider the following steps for the $i$-th non-linear conjugate gradient iteration, in the context of 3D electromagnetic inversion, and as discussed in (Newman and Boggs, 2004):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) set $i = 1$, choose initial model $\mathbf{m}_i$ and compute $\mathbf{r}_i = -\nabla\phi(\mathbf{m}_i)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) set $\mathbf{u}_i = \mathbf{M}^{-1}\mathbf{r}_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $\phi$ is a misfit functional and &quot;the matrix operator $\mathbf{M}$ in the algorithm is a preconditioner, which steers and scales the conjugate search direction $\mathbf{u}_i$ such that it more closely approximates the Newton direction&quot; (Newman and Boggs, 2004).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in implementing the construction of $\mathbf{M}$ using MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The context for the 3D EM problem is as follows: We are interested in minimizing $\phi(\mathbf{m}) = \frac{1}{2}||\mathbf{d}^{obs} - \mathbf{d}^{pred}||_2^2 + \frac{1}{2}\lambda||\mathbf{W}(\mathbf{m}_i)||_2^2$, where $\mathbf{m}$ is the model of Earth's electrical conductivity, $\mathbf{d}^{obs}$ and $\mathbf{d}^{pred}$ are the observed and predicted data vectors, respectively, $\lambda$ is the regularization parameter, $\mathbf{W}$ is the regularization matrix, based upon a finite-difference approximation of the Laplacian, and $\mathbf{D}$ is a data weighting matrix, which for the purpose of this question can be assumed to be the identity matrix. The predicted data arises from the forward problem solution, which yields the electric field $\mathbf{E}$ all over the control volume. This forward problem is no other that the Helmholtz equation for a secondary electric field $\mathbf{E}^S$: $\nabla\times\nabla\times\mathbf{E}^S + j\omega\mu_0\sigma^*\mathbf{E}^S = -j\omega\mu_0(\sigma^* - \sigma^{P*}\mathbf{E}^P)$, where $\mathbf{E} = \mathbf{E}^P + \mathbf{E}^S$ (scattered field formulation), $j$ is the imaginary unit, $\omega$ is the angular frequency, $\mu_0$ is the free space magnetic permeability constant, $\sigma^* = \sigma + j\omega\epsilon$ is the complex conductivity including real conductivity $\sigma$ and permittivity $\epsilon$, and $\sigma^{P*}$ is the conductivity of a layered background model (Grayver, Streich, and Ritter, 2013).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to (Newman and Boggs, 2004): $\mathbf{M} = diag[(\mathbf{D}\mathbf{J})^H(\mathbf{D}\mathbf{J})] + \lambda\mathbf{W}^T\mathbf{W}$, where the operator $diag$  extracts the diagonal of the given matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ignoring the contributions from the term $\lambda\mathbf{W}^T\mathbf{W}$ (for now), it would seem like one can compute $\mathbf{J}^H\mathbf{J}$ by computing $\mathbf{E}^H\mathbf{E}$, since $\mathbf{J}^H\mathbf{J} = -(\omega \mu_0)^2\mathbf{E}^H\mathbf{E}$. Does this make sense? Th articles I have consulted seem to disregard detailed explanation about the construction of this preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Newman, Gregory A.; Boggs, Paul T.&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1088/0266-5611/20/6/S10&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Solution accelerators for large-scale three-dimensional electromagnetic inverse problems&lt;/strong&gt;&lt;/a&gt;, Inverse Probl. 20, No. 6, S151-S170 (2004). &lt;a href=&quot;https://zbmath.org/?q=an:1077.78012&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL1077.78012&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24203" LastActivityDate="2017-12-18T13:22:26.643" Title="Computing preconditioner for a non-linear conjugate gradient implementation" Tags="|optimization|preconditioning|conjugate-gradient|inverse-problem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26824" PostTypeId="2" ParentId="26815" CreationDate="2017-05-09T07:44:35.100" Score="0" Body="&lt;p&gt;Here is the matlab snippet for the same. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc;&#xA;node = [2 3 4&#xA;        6 3 2 &#xA;        2 5 1&#xA;        4 3 6&#xA;        4 3 3&#xA;        4 4 1.5&#xA;        2 4 2.5&#xA;        3 3 5&#xA;        5 3 4&#xA;        3 4 3.5]; &#xA;Y=480; %Youngs Modulus&#xA;nu=1/3; % Poisson ratio&#xA;alfa=(5.0+3.0*sqrt(5.0))/20; %Gauss Points&#xA;beta =(5.0-sqrt(5.0))/20;    %Gauss Points&#xA;weight=0.25; %weights for integration points&#xA;GaussPoints=[alfa beta beta beta;beta alfa beta beta;beta beta alfa  beta;beta beta beta alfa];&#xA;%Elasticity Matrix&#xA;E=[1-nu nu nu 0 0 0;nu 1-nu nu 0 0 0;nu nu 1-nu 0 0 0;0 0 0 0.5-nu 0 0;0 0 0 0 0.5-nu 0;0 0 0 0 0 0.5-nu];&#xA;E=Y/((1+nu)*(1-2*nu))*E;&#xA;%Stiffness Matrix&#xA;K=zeros(30,30);&#xA;&#xA;for j=1:4&#xA;xi=GaussPoints(:,j);&#xA;jx1=4.0*(node(1,1)*(xi(1)-0.25)+node(5,1)*xi(2)+node(7,1)*xi(3)+node(8,1)*xi(4));&#xA;jy1=4.0*(node(1,2)*(xi(1)-0.25)+node(5,2)*xi(2)+node(7,2)*xi(3)+node(8,2)*xi(4));&#xA;jz1=4.0*(node(1,3)*(xi(1)-0.25)+node(5,3)*xi(2)+node(7,3)*xi(3)+node(8,3)*xi(4));&#xA;&#xA;jx2=4.0*(node(5,1)*xi(1)+node(2,1)*(xi(2)-0.25)+node(6,1)*xi(3)+node(9,1)*xi(4));&#xA;jy2=4.0*(node(5,2)*xi(1)+node(2,2)*(xi(2)-0.25)+node(6,2)*xi(3)+node(9,2)*xi(4));&#xA;jz2=4.0*(node(5,3)*xi(1)+node(2,3)*(xi(2)-0.25)+node(6,3)*xi(3)+node(9,3)*xi(4));&#xA;&#xA;jx3=4.0*(node(7,1)*xi(1)+node(6,1)*xi(2)+node(3,1)*(xi(3)-0.25)+node(10,1)*xi(4));&#xA;jy3=4.0*(node(7,2)*xi(1)+node(6,2)*xi(2)+node(3,2)*(xi(3)-0.25)+node(10,2)*xi(4));&#xA;jz3=4.0*(node(7,3)*xi(1)+node(6,3)*xi(2)+node(3,3)*(xi(3)-0.25)+node(10,3)*xi(4));&#xA;&#xA;jx4=4.0*(node(8,1)*xi(1)+node(9,1)*xi(2)+node(10,1)*xi(3)+node(4,1)*(xi(4)-0.25));&#xA;jy4=4.0*(node(8,2)*xi(1)+node(9,2)*xi(2)+node(10,2)*xi(3)+node(4,2)*(xi(4)-0.25));&#xA;jz4=4.0*(node(8,3)*xi(1)+node(9,3)*xi(2)+node(10,3)*xi(3)+node(4,3)*(xi(4)-0.25));&#xA;&#xA;&#xA;J=[1 1 1 1;jx1 jx2 jx3 jx4;jy1 jy2 jy3 jy4;jz1 jz2 jz3 jz4];&#xA;&#xA;Jdet=det(J);&#xA;Jinv=inv(J);&#xA;Iaug=[0 0 0;1 0 0;0 1 0;0 0 1];&#xA;P=Jinv*Iaug;&#xA;&#xA;       a1=P(1,1);&#xA;       a2=P(2,1);&#xA;       a3=P(3,1);&#xA;       a4=P(4,1);&#xA;&#xA;       b1=P(1,2);&#xA;       b2=P(2,2);&#xA;       b3=P(3,2);&#xA;       b4=P(4,2);&#xA;&#xA;       c1=P(1,3);&#xA;       c2=P(2,3);&#xA;       c3=P(3,3);&#xA;       c4=P(4,3);&#xA;&#xA;Nfx=[(4.0*xi(1)-1)*a1 (4.0*xi(2)-1)*a2  (4.0*xi(3)-1)*a3 (4.0*xi(4)-1)*a4 ...&#xA;   4.0*(a1*xi(2)+a2*xi(1)) 4.0*(a2*xi(3)+a3*xi(2)) 4.0*(a1*xi(3)+a3*xi(1)) ...&#xA;   4.0*(a1*xi(4)+a4*xi(1)) 4.0*(a2*xi(4)+a4*xi(2)) 4.0*(a3*xi(4)+a4*xi(3))];&#xA;&#xA;Nfy=[(4.0*xi(1)-1)*b1 (4.0*xi(2)-1)*b2 (4.0*xi(3)-1)*b3 (4.0*xi(4)-1)*b4 ...&#xA;  4.0*(b1*xi(2)+b2*xi(1)) 4.0*(b2*xi(3)+b3*xi(2)) 4.0*(b1*xi(3)+b3*xi(1)) ...&#xA;  4.0*(b1*xi(4)+b4*xi(1)) 4.0*(b2*xi(4)+b4*xi(2)) 4.0*(b3*xi(4)+b4*xi(3))];&#xA;&#xA;Nfz=[(4.0*xi(1)-1)*c1 (4.0*xi(2)-1)*c2 (4.0*xi(3)-1)*c3 (4.0*xi(4)-1)*c4 ...&#xA;  4.0*(c1*xi(2)+c2*xi(1)) 4.0*(c2*xi(3)+c3*xi(2)) 4.0*(c1*xi(3)+c3*xi(1)) ...&#xA;  4.0*(c1*xi(4)+c4*xi(1)) 4.0*(c2*xi(4)+c4*xi(2)) 4.0*(c3*xi(4)+c4*xi(3))];&#xA;&#xA;B=[Nfx(1) 0 0 Nfx(2) 0 0 Nfx(3) 0 0 Nfx(4) 0 0 Nfx(5) 0 0 Nfx(6) 0 0 Nfx(7) 0 0 Nfx(8) 0 0 Nfx(9) 0 0 Nfx(10) 0 0;&#xA;  0 Nfy(1) 0 0 Nfy(2) 0 0 Nfy(3) 0 0 Nfy(4) 0 0 Nfy(5) 0 0 Nfy(6) 0 0 Nfy(7) 0 0 Nfy(8) 0 0 Nfy(9) 0 0 Nfy(10) 0;&#xA;  0 0 Nfz(1) 0 0 Nfz(2) 0 0 Nfz(3) 0 0 Nfz(4) 0 0 Nfz(5) 0 0 Nfz(6) 0 0 Nfz(7) 0 0 Nfz(8) 0 0 Nfz(9) 0 0 Nfz(10);&#xA;  Nfy(1) Nfx(1) 0 Nfy(2) Nfx(2) 0 Nfy(3) Nfx(3) 0 Nfy(4) Nfx(4) 0 Nfy(5) Nfx(5) 0 Nfy(6) Nfx(6) 0 Nfy(7) Nfx(7) 0 Nfy(8) Nfx(8) 0 Nfy(9) Nfx(9) 0 Nfy(10) Nfx(10) 0;&#xA;  0 Nfz(1) Nfy(1) 0 Nfz(2) Nfy(2) 0 Nfz(3) Nfy(3) 0 Nfz(4) Nfy(4) 0 Nfz(5) Nfy(5) 0 Nfz(6) Nfy(6) 0 Nfz(7) Nfy(7) 0 Nfz(8) Nfy(8) 0 Nfz(9) Nfy(9) 0 Nfz(10) Nfy(10);&#xA;  Nfz(1) 0 Nfx(1) Nfz(2) 0 Nfx(2) Nfz(3) 0 Nfx(3) Nfz(4) 0 Nfx(4) Nfz(5) 0 Nfx(5) Nfz(6) 0 Nfx(6) Nfz(7) 0 Nfx(7) Nfz(8) 0 Nfx(8) Nfz(9) 0 Nfx(9) Nfz(10) 0 Nfx(10)];  &#xA;&#xA;K=K+weight*(B'*E*B*Jdet/6.0);&#xA;Jdet&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23149" LastEditorUserId="23149" LastEditDate="2017-05-23T09:01:46.017" LastActivityDate="2017-05-23T09:01:46.017" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26825" PostTypeId="1" AcceptedAnswerId="26845" CreationDate="2017-05-09T08:45:50.780" Score="2" ViewCount="815" Body="&lt;p&gt;I am using a parametrized natural spline to interpolate 2D curves. For example something like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lMNRU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lMNRU.png&quot; alt=&quot;Airfoil&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where I parametrize both the x and the y coordinate with a parameter that increases by one between each of the given points.&#xA;The interpolation looks very good, here the x coordinate as a function of the parameter t:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5vTsj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5vTsj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first derivative with respect to t already shows a few wiggles:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TWSVs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TWSVs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the second derivative is oscillating heavily:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LOMPb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LOMPb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One interesting fact: The oscillations appear exactly where the spacing of the points changes from uniform to a cosine spacing.&#xA;When I apply the spline to a circle or ellipse I obtain perfectly smooth first and second derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anybody has an idea why the derivatives start to oscillate and how to prevent this?&lt;/p&gt;&#xA;" OwnerUserId="23586" LastEditorUserId="9667" LastEditDate="2017-05-09T23:03:20.417" LastActivityDate="2017-05-11T06:15:51.063" Title="Parametrized spline - oscilating second derivative" Tags="|interpolation|b-spline|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26827" PostTypeId="1" CreationDate="2017-05-09T11:56:17.860" Score="1" ViewCount="113" Body="&lt;p&gt;I have the following function&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x,y) =\left(\frac{1}{\exp(x-E_f)+1}-\frac{1}{\exp(x-y-E_f)+1} \right)\frac{1}{\sqrt{4-xy}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to integrate it using Python 3. The domain of the $y$ variable is from sampled data hence I cannot used something like &lt;code&gt;quad&lt;/code&gt;. The $x$ variable comes from analytical formula so I can generate whatever value I want for it. My question is how can use &lt;code&gt;simps&lt;/code&gt; and &lt;code&gt;quad&lt;/code&gt; combined to integrate this formula.&lt;/p&gt;&#xA;" OwnerUserId="24208" LastEditorUserId="9667" LastEditDate="2017-05-09T15:41:41.223" LastActivityDate="2017-05-09T15:41:41.223" Title="solving numerically a 2D integral by using simps and quad combined" Tags="|quadrature|scipy|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26828" PostTypeId="1" CreationDate="2017-05-09T13:25:46.567" Score="0" ViewCount="119" Body="&lt;p&gt;Let $d_1=1,d_2=2,a_{11}=\frac{5}{13},a_{12}=\frac{22}3,a_{21}=-2,a_{22}=\frac{6}7,\tau=\frac{5}7$, $\psi(t,x)=\cos^42x,\phi(t,x)=\frac{3}{13}x^4\sin^2 3x$, $\Omega=[0,200]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\{&#xA;\begin{array}{lc}&#xA; \dfrac{\partial u(t,x)}{\partial t}=d_1\triangle u(t,x)+u(t,x)\left(r_1-a_{11}u(t-\tau,x)-a_{12}v(t,x)\right),&amp;amp; t&amp;gt;0,x\in\Omega \\&#xA;\dfrac{\partial v(t,x)}{\partial t}=d_2\triangle v(t,x)+v(t,x)\left(-r_2+a_{21}u(t,x)-a_{22}v(t,x)\right),&amp;amp; t&amp;gt;0,x\in\Omega\\&#xA;\dfrac{\partial u}{\partial n}=\dfrac{\partial v}{\partial n}=0,\quad t\ge0,x\in\partial\Omega \quad(\text{Neumann conditions})\\&#xA;u(t,x)=\phi(t,x)\ge 0,\qquad v(t,x)=\psi(t,x)\ge 0, &amp;amp;(t,x)\in[-\tau,0]\times\Omega&#xA;\end{array}&#xA;\right.$$&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="7574" LastEditDate="2018-03-27T11:12:23.260" LastActivityDate="2020-07-19T18:10:56.253" Title="Parabolic differential equations with time delay" Tags="|finite-element|pde|finite-difference|parabolic-pde|dde|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26830" PostTypeId="2" ParentId="26814" CreationDate="2017-05-09T13:44:14.847" Score="5" Body="&lt;p&gt;Physically, vorticity can only be created (as opposed to transported, stretched or intensified &lt;strong&gt;after&lt;/strong&gt; being created) either by the appearance of a boundary layer on a solid surface, or through the pressure gradient not being parallel to the density gradient. For the shallow water equation without viscosity neither mechanism is available, so congratulations, your code is making a correct prediction. Quite often though vorticity does appear on account of numerical viscosity. It appears in more or less the same place that vorticity would appear with real viscosity. It might or might not be relevant to the true situation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT The non creation of vorticity is revealed by the evolution equation of vorticity for an inviscid compressible gas in two dimensions;&#xA;$$\partial_t\omega+\mathbf{u}\cdot\nabla\omega+\omega\nabla\cdot\mathbf{u}+\frac{\nabla{p}\times\nabla{\rho}}{\rho^2}=0$$&lt;/p&gt;&#xA;" OwnerUserId="23873" LastEditorUserId="23873" LastEditDate="2017-05-11T18:12:15.507" LastActivityDate="2017-05-11T18:12:15.507" CommentCount="29" ContentLicense="CC BY-SA 3.0" />
  <row Id="26831" PostTypeId="1" CreationDate="2017-05-09T15:20:41.697" Score="1" ViewCount="66" Body="&lt;p&gt;There is a &lt;a href=&quot;http://www-personal.umich.edu/~mejn/papers/geocomp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; called &quot;Density-equalizing map projections: Diffusion-based algorithm and applications&quot; by Michael T. Gastner and M. E. J. Newman, which explains their algorithm (which is based in diffusion equations) for generating value-by-area cartograms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it explains the theoretical side of the mathematics involved with their algorithm, it doesn't explain how they actually implemented it. I tried to piece it together by looking at the source code from &lt;a href=&quot;http://www-personal.umich.edu/~mejn/cart/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cart&lt;/a&gt;, but I don't have the programming knowledge (it's written in c, which I don't know) required to understand it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has at least a decent understanding of it and can explain the steps needed to create a cartogram using their algorithm, that would be greatly appreciated. Otherwise, if you have other helpful resources on the topic, those would be good too.&lt;/p&gt;&#xA;" OwnerUserId="24202" LastActivityDate="2017-05-09T15:20:41.697" Title="How is the Gastner-Newman equation implemented to create value-by-area cartograms?" Tags="|linear-algebra|fluid-dynamics|nonlinear-programming|diffusion|differential-equations|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26832" PostTypeId="1" CreationDate="2017-05-09T18:54:07.337" Score="1" ViewCount="291" Body="&lt;p&gt;&lt;strong&gt;1s question: definition of stationary point for constrained optimization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, a stationary point of a constrained optimization problem is a stationary point of the Lagrangian (that has to be differentiable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider the problem&#xA;\begin{equation}&#xA;\min_{x\in X} F(x)&#xA;\end{equation}&#xA;where $F:\mathbb{R}^p\to \mathbb{R}$ is differentiable but possibly non-convex, $X\subset \mathbb{R}^p$ is a closed convex set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is easy to obtain the following first-order optimality condition, where $x^*$ is a local minimizer:&#xA;$$\nabla F(x^*)^\top(x-x^*) \ge 0\quad\forall x\in X. \qquad (*)$$&#xA;In several references that I read (such as &lt;a href=&quot;https://arxiv.org/abs/1607.00345&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;), $x^*$ is called a &lt;em&gt;stationary point&lt;/em&gt; if and only if it satisfies the above condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Is this definition of &lt;em&gt;stationary point&lt;/em&gt; widely accepted? Is there any reliable reference on that? And, is there a similar definition if $f$ is non-differentiable? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2nd question: Stationary point vs. Nash (equilibrium) point&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider now that $F(x)$ takes the form&#xA;$$F(x_1,...,x_n)= f(x_1,...,x_n) + \sum_{i=1}^n r_i(x_i)$$&#xA;where $f:\mathbb{R}^p\to \mathbb{R}$ is a differentiable multi-convex function, $r_i:\mathbb{R}^{p_i}\to \mathbb{R}$ are extended-value convex functions. Also, $X$ is a closed multi-convex set. (Roughly speaking, the problem of minimizing over one block while the others are fixed is a convex problem.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is considered in &lt;a href=&quot;http://www.math.ucla.edu/~wotaoyin/papers/pdf/Xu%20and%20Yin%20-%202013%20-%20A%20Block%20Coordinate%20Descent%20Method%20for%20Regularized.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nash equilibrium (eq. (2.3) in the paper, reformulated):&#xA;$$x_i^* = \arg\min_{x_i} F(x_1^*,.\ldots.,x_{i-1}^*,x_i,x_{i+1}^*,\ldots,x_n^*) \qquad (2.3),$$&#xA;which is equivalent to (eq. (2.4) in the paper):&#xA;$$\left(\nabla_{x_i}f(x^*) + p_i^*\right)^\top (x_i - x_i^*) \ge 0 \quad\forall x_i \qquad (2.4),$$&#xA;where $p_i^*\in\partial r_i(x_i^*)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Remark 2.2 (right after equation (2.4)), the authors stated that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In general, the condition (2.4) is weaker than the first-order&#xA;  optimality condition. For our problem, a critical point must be a&#xA;  Nash point, but a Nash point is not necessarily a critical point.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not sure how the authors defined 'critical point' (= 'stationary point') in this case, because if it's the same as in my first question, then I think for this problem, a Nash point must be a stationary point, given that $r_i$ are differentiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, assume that $x^*$ is a Nash point, we have&#xA;$$\nabla F(x^*)^\top (x - x^*) = \sum_{i=1}^n \left(\nabla_{x_i}f(x^*) + \nabla r_i(x_i^*)\right)^\top (x_i - x_i^*).$$&#xA;Since each term in the sum is non-negative according to $(2.4)$, we must have $\nabla F(x^*)^\top (x - x^*) \ge 0$, i.e., $x^*$ is a stationary point according to the definition $(*)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot for your discussion.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastActivityDate="2017-05-09T18:54:07.337" Title="Constrained optimization: Stationary point vs. Nash point" Tags="|optimization|nonlinear-programming|nonconvex|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26833" PostTypeId="2" ParentId="26828" CreationDate="2017-05-09T21:48:48.837" Score="1" Body="&lt;p&gt;Do the same discretization that you normally do for the nonlinear Heat Equation to turn &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, the Strang matrix second order discretization (the &lt;code&gt;[1 -2 1]&lt;/code&gt; tridiagonal matrix). Now you have a system of DDEs. Use a DDE solver on this. &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/dde23.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MATLAB's DDE23&lt;/a&gt;, &lt;a href=&quot;http://diffeq.sciml.ai/latest/tutorials/dde_example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Julia's DDE solvers&lt;/a&gt;, etc.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:10:56.253" LastActivityDate="2020-07-19T18:10:56.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="26834" PostTypeId="2" ParentId="26435" CreationDate="2017-05-10T07:57:44.253" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Since you are asking quite a general question, I will respond with a&#xA;  general answer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I like this. I will address the following question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Would a mass flux be required at the interfaces to the fluid regions&#xA;  as well? Is there a specific approach for modeling gas flux?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Boundary conditions (BCs) are prescribed behavior involving the variables of the problem. The more you add, the more your problem will be described; at least from the computer code standpoint. Since you are solving for a scalar-valued concentration $C$, then one common BC you can prescribe is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Robin_boundary_condition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robin boundary condition&lt;/a&gt; (see previous link for details):&#xA;$$&#xA;a C + b \frac{\partial C}{\partial n} = g.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;a href=&quot;https://en.wikipedia.org/wiki/Neumann_boundary_condition&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Neumann component&lt;/a&gt; on the aforementioned equation; i.e. $b \frac{\partial C}{\partial n}$ is a good start to study the normal concentration flux at the boundary. In general, these are very useful for these type of problem in the case you need to prescribe an insulation condition on your variable of interest; i.e. from Wikipedia:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Robin boundary condition is a general form of the insulating&#xA;  boundary condition for convection–diffusion equations. Here, the&#xA;  convective and diffusive fluxes at the boundary sum to zero: $$&#xA;u_x(0)\,c(0) -D \frac{\partial c(0)}{\partial x}=0 $$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Therefore, I would advise to further study Robin BCs in this context, in order to learn the advantages these may bring to the table, in terms of imposing a mass flux at the interfaces, which are indeed boundaries to each individual sub-volume.&lt;/p&gt;&#xA;" OwnerUserId="24203" LastActivityDate="2017-05-10T07:57:44.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26835" PostTypeId="2" ParentId="26782" CreationDate="2017-05-10T08:12:47.893" Score="1" Body="&lt;p&gt;This question is confusing. At first you are speaking of a steady-state equation, and suddenly you speak of a time scale...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will try to clarify the following. From the numerical PDEs standpoint, you could classify physical phenomena as &lt;strong&gt;time-dependent&lt;/strong&gt; or &lt;strong&gt;time-independent&lt;/strong&gt;. &lt;a href=&quot;http://bookstore.siam.org/ot98/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LeVeque's book&lt;/a&gt; is a solid reference for this. Usually, time-independent phenomena are described via elliptic partial differential equations (PDEs), which can be discretized thus yielding a system of equations, whose solution represents an approximation for the state of the simulated system in a snapshot of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two notes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A time-periodic dependence, yields time-independence when formulated in the frequency domain.&lt;/li&gt;&#xA;&lt;li&gt;If the PDE is linear/non-linear then the arising system of equations will be linear/non-linear... In the latter, clearly &quot;linear algebra&quot; methods won't suffice.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Time-dependent problems require time-wise integration. The Crank-Nicolson-method is usually used for time-dependent problems. In time-dependent problems, snapshots of the system behavior in time are solved progressively, and their temporal evolution is guided by the time integration method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question seems to be mixing temporal dependence concepts. If you have an elliptic PDE (let us say it is linear), then you must discretize, using any space-oriented discretization methods. This will yield a linear system of equations with properties you can leverage in order to choose your method for solving it. Non-periodical time-dependent PDEs, e.g. parabolic PDEs, will require a time integration method. However, time integration methods can be &lt;strong&gt;explicit&lt;/strong&gt; or &lt;strong&gt;implicit,&lt;/strong&gt; and their numerical stability must be ensured. The choice of explicit vs. implicit, depends on how hard it is to ensure this numerical stability. Other aspects matter as well: Dispersion, physics-dependent grid refinement conditions, and numerical accuracy, among others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your situation, I would focus on solving the elliptic problem first. Once you understand this, proceed to introduce time dependence, and study which time integration method will yield the best results.&lt;/p&gt;&#xA;" OwnerUserId="24203" LastActivityDate="2017-05-10T08:12:47.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26836" PostTypeId="2" ParentId="23882" CreationDate="2017-05-10T12:47:07.490" Score="3" Body="&lt;p&gt;The number of file formats for FEM is ridiculous, partly due to the fact that every software package implemented its own format in the past.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7OALZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7OALZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(From &lt;a href=&quot;https://xkcd.com/927/&quot; rel=&quot;nofollow noreferrer&quot;&gt;xkcd&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've created &lt;a href=&quot;https://github.com/nschloe/meshio&quot; rel=&quot;nofollow noreferrer&quot;&gt;meshio&lt;/a&gt; to alleviate the pain of converting between formats, so if you use any format supported by meshio, you should be able to easily make a switch in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Out of all formats I know, VTU and XDMF are the ones I personally use most.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2017-05-10T12:47:07.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26837" PostTypeId="2" ParentId="23882" CreationDate="2017-05-10T14:34:29.223" Score="0" Body="&lt;p&gt;I second the use of VTK for file output.&#xA;If you wish to use a very generic mesh structure, I would point you in the direction of the CGNS library (CFD General Notation System) which is an opensource mesh format with its own library that is maintained by a consortium (including Boeing and NASA). Reading a list of nodes, properties, boundary conditions, etc. is extremely useful with this library and the library itself is very easy to compile. The library is in C++, but it can be linked with Fortran.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use it in our in-house code and I have found it is a very efficient format. It uses HDF5 under the hood for binary files. Furthermore, ANSYS, Salome, GMSH all support the exportation to this format at varying degree.&#xA;Here is the github:&#xA;&lt;a href=&quot;https://cgns.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cgns.github.io/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2017-05-10T14:41:10.017" LastActivityDate="2017-05-10T14:41:10.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26838" PostTypeId="1" AcceptedAnswerId="26840" CreationDate="2017-05-10T14:47:44.437" Score="4" ViewCount="1107" Body="&lt;p&gt;I have a question concerning the von Neumann stability analysis of finite difference approximations of PDEs. There seem to be a wealth of online source explaining the application of this stability analysis to a few example cases, most commonly the heat equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${\frac {\partial u}{\partial t}} = \alpha {\frac {\partial^2 u}{\partial x^2}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The procedure is also elaborated in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&quot; rel=&quot;noreferrer&quot;&gt;wikipedia article&lt;/a&gt;, so please refer to the linked source for a detailed stability analysis on this equation. Now my question: Is the Von Neumann stability analysis applicable if my PDE involves a constant term $C$, i.e. some sort of source term:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${\frac {\partial u}{\partial t}} - \alpha {\frac {\partial^2 u}{\partial x^2}}= C$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wouldn't introducing a constant prohibit one from eliminating the variables introduced in the Fourier expansion? Can the von Neumann stability analysis applied here, and if yes, how?&lt;/p&gt;&#xA;" OwnerUserId="24220" LastEditorUserId="9667" LastEditDate="2017-05-10T14:53:49.313" LastActivityDate="2017-05-10T17:01:11.447" Title="Von Neumann stability analysis with a constant term" Tags="|pde|finite-difference|stability|fourier-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26839" PostTypeId="2" ParentId="26819" CreationDate="2017-05-10T16:35:37.547" Score="0" Body="&lt;p&gt;As short answer, you need to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Sweep over your wavenumber $q\in [q_\min, q_\max]$;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Approximate your differential equation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{L}_q \mathbf{f} = \epsilon \mathbf{f}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to obtain a discrete problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[H(k)]\lbrace U \rbrace = E[S]\lbrace U \rbrace\, ;$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Solve the generalized eigenvalue problem to obtain the admissible frequencies $\epsilon$ for that particular $q$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Regarding point 2 and 3, there are several methods. Some of the most popular ones are discussed in this answer &lt;a href=&quot;https://scicomp.stackexchange.com/a/19679/9667&quot;&gt;[1]&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Finite Difference Methods; &lt;/li&gt;&#xA;&lt;li&gt;Finite Element Methods; and&lt;/li&gt;&#xA;&lt;li&gt;Direct Variational methods.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Let's rewrite the equation as above, $\mathcal{L}_q \mathbf{f} = \epsilon \mathbf{f}$, with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{L}_q =&#xA;\begin{bmatrix}&#xA;0 &amp;amp;-\frac{1}{2}\nabla^2 + \hat{V}(q) + 3\psi^2_0\\&#xA;-\frac{1}{2}\nabla^2 + \hat{V}(q) + \psi^2_0 &amp;amp;0\&#xA;\end{bmatrix}\, ,\quad&#xA;\mathbf{f} =&#xA;\begin{Bmatrix} f_{-}\\ f_{+}\end{Bmatrix}\, ;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\hat{V(q)} = V_\text{ext} - \mu - q^2$, where I re-scaled some variables for convenience. The operator $\mathcal{L}$ does not seem to be self-adjoint, if that's the case, the resulting matrices might not be Hermitian (depending on the scheme).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one uses a weighted residual approach, the resulting functional is of the form (I would double-check, it's easy to make silly mistakes)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Pi_q[\mathbf{f}, \mathbf{w}] =&#xA;\frac{1}{2}\int_\mathbb{R} \nabla w_1 \nabla f_{+}\, dz +&#xA;\frac{1}{2}\int_\mathbb{R} \nabla w_2 \nabla f_{-}\, dz +\\&#xA;\int_\mathbb{R} |\psi_0|^2[3 w_1 f_{+} + w_2 f_{-}]\, dz +&#xA;\int_\mathbb{R} \hat{V}(q)[w_1 f_{+} + w_2 f_{-}]\, dz -&#xA;\epsilon\left[\int_\mathbb{R}  w_1 f_{-}\, dz +&#xA;\int_\mathbb{R}  w_2 f_{+}\, dz\right]&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-05-10T18:21:15.417" LastActivityDate="2017-05-10T18:21:15.417" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26840" PostTypeId="2" ParentId="26838" CreationDate="2017-05-10T17:01:11.447" Score="6" Body="&lt;p&gt;From your link, consider the definition of the round off error and the statement &quot;Since the exact solution must satisfy the discretized equation exactly, the error must also satisfy the discretized equation.&quot; This is actually only true if the PDE is homogeneous, that is, if we can write it in the form&#xA;$\mathcal{L}(u;u_t,u_x,u_{xx},\ldots)=0$, with all terms involving the dependent variable or its derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case however, we have $\mathcal{L}(u)=C$. Keeping the notation from your link have assumed both the numerical solution, $N_i^{n}$ and the true trajectory ${u^n_i}$ satisfy the discretised equation $\mathcal{L}^*$, hence using the linearity of $\mathcal{L}^*$ the rounding error satisfies $\mathcal{L}^*(\epsilon)$= $\mathcal{L}^*(N)-\mathcal{L}^*(u) = C-C=0$, or in other words the homogeneous version of the operator. This means that the von Neumann analysis on conditions for the stability of the round-off error remains the same.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2017-05-10T17:01:11.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26841" PostTypeId="1" CreationDate="2017-05-10T19:04:41.890" Score="5" ViewCount="146" Body="&lt;p&gt;What's the best practice for naming output files from scripts that you're experimenting with? I'm trying different parameters in my data generation script and I'd like to keep track of which files correspond to which parameters. If I use the standard way of naming with timestamps I soon lose track of how the files were created.&lt;/p&gt;&#xA;" OwnerUserId="24224" LastActivityDate="2017-05-11T07:11:21.293" Title="Naming scientific output files" Tags="|testing|programming-paradigms|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26842" PostTypeId="2" ParentId="26825" CreationDate="2017-05-10T19:45:15.570" Score="0" Body="&lt;p&gt;I found a more or less satisfying solution to the problem. The profile above was just something i tried after having the same problems with other &quot;real&quot; profiles (they were given to me in a csv file -&gt; no analytic formula to reconstruct or redistribute the points).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is as follows: I created a spline with the given points, then i redistributed the points with uniform distance (arc length). With the redistributed points I created a new spline. In the new the spline, the oscillations in the second derivative were almost gone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the second derivative of the spline to determine the curvature and the normal vector.&lt;/p&gt;&#xA;" OwnerUserId="23586" LastActivityDate="2017-05-10T19:45:15.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26843" PostTypeId="2" ParentId="26841" CreationDate="2017-05-10T20:53:39.380" Score="2" Body="&lt;p&gt;This question might get closed for being opinion-based, but I think it's an important question ... so here's my opinion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, there's no real industry best practice for this kind of thing -- a lot of &quot;don't&quot;s, but not a lot of &quot;do&quot;s. For example, &lt;a href=&quot;https://library.stanford.edu/research/data-management-services/data-best-practices/best-practices-file-naming&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford&lt;/a&gt; has a web page dedicated to file naming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's worked well for me is to create a series of folders whose names are numerical keys (e.g. &quot;7001&quot;, &quot;7002&quot;, &quot;7003&quot;, ...), as well as a text file which gives a description of each folder (e.g. date, numerical parameters, motivation, brief summary of results, etc.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I like this scheme because the folder names are short which makes it easy to find the one I want. Having a description of each folder in one text file makes it nice for finding results months later, and remembering why I did what I did. &lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2017-05-10T20:53:39.380" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26844" PostTypeId="2" ParentId="26841" CreationDate="2017-05-11T04:10:35.427" Score="1" Body="&lt;p&gt;The best approach may depend on the problem at hand. Just make sure that: 1) it is well automated and 2) every detail is recorded. I have been naming files with date and most important parameters (or the parameters that are being changed).&lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2017-05-11T04:10:35.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26845" PostTypeId="2" ParentId="26825" CreationDate="2017-05-11T06:07:32.767" Score="6" Body="&lt;p&gt;Your choice of parameterization is creating problems. Instead of spanning one in $t$ between points, span an amount proportional to the line segment between the two points in $(x,y)$ space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've created a Python example that demonstrates the issue. It compares a uniform parameterization $(x_t(t),y_t(t))$ (like yours, but mine is scaled so that $t \in [0,1]$) with a length-based parameterization $(x_u(u),y_u(u))$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my example airfoil:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rBaqG.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rBaqG.jpg&quot; alt=&quot;example airfoil x,y points&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the $x$ and $y$ coordinates as functions of parameters $t$ and $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6fBJb.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6fBJb.jpg&quot; alt=&quot;parameterization w.r.t t and u&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are the second derivatives. Note that the uniform parameterization shows these oscillations in the second derivative, just like you've shown, but the length-based parameterization looks much smoother.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZnhWb.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZnhWb.jpg&quot; alt=&quot;second derivatives w.r.t. t and u&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I calculated the curvature using the formula&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\kappa = |x' y'' - y' x''| / (x'^2+y'^2)^{3/2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from &lt;a href=&quot;https://en.wikipedia.org/wiki/Curvature#Local_expressions&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, where&#xA;the derivatives are taken w.r.t. the parameter $t$ or $u$. Note that in this curve I've calculated $\kappa_t(t)$ using the formula above with derivatives w.r.t the parameter $t$, but I've plotted against $u$ so that we can see the comparison between the two calculated curvatures,which should be similar. As we can see, the uniform parameterization introduces some artifacts, but the length-based parameterization is pretty smooth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JfRdd.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JfRdd.jpg&quot; alt=&quot;curvature&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, here's the Python code I used to generate these plots&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from numpy import *&#xA;from matplotlib.pyplot import *&#xA;from scipy import interpolate&#xA;&#xA;# first create a list of x,y points based on an airfoil design from&#xA;# https://en.wikipedia.org/wiki/NACA_airfoil&#xA;# created so that points are not uniformly distributed&#xA;c=1.0&#xA;th=0.15&#xA;x=concatenate( (linspace(1.0,0.1,40),linspace(0.95,0.05,20)**3.0 *0.1 ) )&#xA;y=5.0*th*(0.2969*np.sqrt(x/c) - 0.1260*(x/c) - 0.3516*(x/c)**2 +0.2843*(x/c)**3 - 0.1015*(x/c)**4)&#xA;&#xA;x=concatenate( (x,flipud(x)))&#xA;y=concatenate( (y,-flipud(y)))&#xA;&#xA;&#xA;# create parameters. t is uniformly spaced on the interval 0 to 1. &#xA;# u is spaced proportional to edge lengths&#xA;t=arange( len(x),dtype=float) / (len(x)-1)&#xA;u=zeros( (len(x)),dtype=float)&#xA;L=0.0&#xA;u[0]=0.0&#xA;for ii in range(1,len(x)):&#xA;  L += sqrt( (x[ii]-x[ii-1])**2 + (y[ii]-y[ii-1])**2 )&#xA;  u[ii]=L&#xA;u /= L&#xA;&#xA;# create two cubic splines for each x and y&#xA;csxt=interpolate.splrep(t,x,k=3)&#xA;csxu=interpolate.splrep(u,x,k=3)&#xA;csyt=interpolate.splrep(t,y,k=3)&#xA;csyu=interpolate.splrep(u,y,k=3)&#xA;&#xA;# tt and uu are fine samples used for plotting&#xA;tt=linspace(0.0,1.0,10000)&#xA;uu=linspace(0.0,1.0,10000)&#xA;&#xA;&#xA;# evaluate x(t),y(t),x(u),y(u) and their derivatives&#xA;xt0=interpolate.splev(tt,csxt)&#xA;xt1=interpolate.splev(tt,csxt,der=1)&#xA;xt2=interpolate.splev(tt,csxt,der=2)&#xA;yt0=interpolate.splev(tt,csyt)&#xA;yt1=interpolate.splev(tt,csyt,der=1)&#xA;yt2=interpolate.splev(tt,csyt,der=2)&#xA;&#xA;xu0=interpolate.splev(uu,csxu)&#xA;xu1=interpolate.splev(uu,csxu,der=1)&#xA;xu2=interpolate.splev(uu,csxu,der=2)&#xA;yu0=interpolate.splev(uu,csyu)&#xA;yu1=interpolate.splev(uu,csyu,der=1)&#xA;yu2=interpolate.splev(uu,csyu,der=2)&#xA;&#xA;# calculate curvature by the formula&#xA;# |x' y'' - y' x'' | / |x'^2 + y'^2|^3/2&#xA;Kt=abs( xt1*yt2 - yt1*xt2) / sqrt(xt1*xt1 + yt1*yt1)**3&#xA;Ku=abs( xu1*yu2 - yu1*xu2) / sqrt(xu1*xu1 + yu1*yu1)**3&#xA;&#xA;&#xA;# interpolate between t and u, so that we can plot&#xA;# apples to apples&#xA;cstu=interpolate.splrep(t,u,k=1)&#xA;uu2=interpolate.splev(tt,cstu)&#xA;&#xA;# plots&#xA;ff=figure(1)&#xA;ff.clf()&#xA;plot(x,y,'.',markersize=6)&#xA;gca().set_aspect('equal')&#xA;grid('on')&#xA;xlabel('x')&#xA;ylabel('y')&#xA;axis([-0.05,1.05,-0.2,0.2])&#xA;&#xA;ff=figure(2)&#xA;ff.clf()&#xA;subplot(211)&#xA;plot(tt,xt0,'b-',label=r&quot;$x_t(t)$&quot;)&#xA;plot(tt,yt0,'r-',label=r&quot;$y_t(t)$&quot;)&#xA;grid('on')&#xA;xlabel('t')&#xA;legend(loc='upper right',fontsize=20)&#xA;subplot(212)&#xA;plot(uu,xu0,'b-',label=r&quot;$x_u(u)$&quot;)&#xA;plot(uu,yu0,'r-',label=r&quot;$y_u(u)$&quot;)&#xA;grid('on')&#xA;xlabel('u')&#xA;legend(loc='upper right',fontsize=20)&#xA;&#xA;&#xA;&#xA;ff=figure(3)&#xA;ff.clf()&#xA;subplot(211)&#xA;plot(tt,xt2,'b-',label=r&quot;$x_t''(t)$&quot;)&#xA;plot(tt,yt2,'r-',label=r&quot;$y_t''(t)$&quot;)&#xA;grid('on')&#xA;xlabel('t')&#xA;legend(loc='upper right',fontsize=20)&#xA;subplot(212)&#xA;plot(uu,xu2,'b-',label=r&quot;$x_u''(u)$&quot;)&#xA;plot(uu,yu2,'r-',label=r&quot;$y_u''(u)$&quot;)&#xA;grid('on')&#xA;xlabel('u')&#xA;legend(loc='upper right',fontsize=20)&#xA;&#xA;ff=figure(4)&#xA;ff.clf()&#xA;plot(uu2,Kt,'b-',label=r&quot;$\kappa_t(u)$&quot;)&#xA;plot(uu,Ku,'r-',label=r&quot;$\kappa_u(u)$&quot;)&#xA;grid('on')&#xA;xlabel('u')&#xA;gca().set_xlim([0.4,0.6])&#xA;legend(loc='upper right',fontsize=20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23262" LastEditorUserId="23262" LastEditDate="2017-05-11T06:15:51.063" LastActivityDate="2017-05-11T06:15:51.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26846" PostTypeId="2" ParentId="26841" CreationDate="2017-05-11T07:11:21.293" Score="6" Body="&lt;p&gt;The problem of recording how a particular piece of scientific data was generated has come to be known as &quot;provenance tracking&quot;. Here is an article on it which also includes the description of a software tool, Sumatra: &lt;a href=&quot;http://rrcns.readthedocs.io/en/latest/provenance_tracking.html&quot; rel=&quot;noreferrer&quot;&gt;http://rrcns.readthedocs.io/en/latest/provenance_tracking.html&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another, simpler Python tool to do something similar is &lt;a href=&quot;http://blog.rtwilson.com/introducing-recipy-effortless-provenance-tracking-with-python/&quot; rel=&quot;noreferrer&quot;&gt;recipy&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2017-05-11T07:11:21.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26847" PostTypeId="1" AcceptedAnswerId="26863" CreationDate="2017-05-11T10:00:06.103" Score="4" ViewCount="2825" Body="&lt;p&gt;I know lot of density functional packages in fortran, including one which we are developing in our group  (&lt;a href=&quot;https://github.com/fireball-dft&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fireball-DFT&lt;/a&gt;) but I don't like fortran very much and I would like something which is easier to integrate to other software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I have idea to create some simple interactive tool with GUI based on DFT or semi-empirical quantum chemistry method which can provide rough results (e.g. molecular orbitals) in almost real time (for small molecules). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although some fortran based DFT packages can be interfaced into python interface e.g. in &lt;a href=&quot;https://wiki.fysik.dtu.dk/ase/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ASE&lt;/a&gt; it is quite slow and cumbersome way. Also I search for solver which is minimalistic and intendend for small systems, therefeore e.g. &lt;a href=&quot;https://wiki.fysik.dtu.dk/gpaw/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPAW&lt;/a&gt; which is largely in C/C++ is not very good for me.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know some simple quantum solver written in C/C++ or even better OpenCL? In case, I consider start to develop something similar by myself.&lt;/p&gt;&#xA;" OwnerUserId="4696" LastActivityDate="2017-05-12T13:23:43.773" Title="Simple open-source Quantum chemistry or DFT code in C/C++" Tags="|c++|c|quantum-mechanics|density-functional-theory|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26849" PostTypeId="1" AcceptedAnswerId="26853" CreationDate="2017-05-11T11:05:38.083" Score="3" ViewCount="12767" Body="&lt;p&gt;I am working on a project with electrical circuits, where I am trying to compute the voltages at all the nodes of an electrical circuit. I know that the electrical circuit is a perfect grid, so each node only touches at most 8 other nodes. Which means that I endup trying to solve a system:&#xA;&lt;code&gt;L v = i&lt;/code&gt;&#xA;where &lt;code&gt;L&lt;/code&gt; is at least a 1000x1000 matrix where each row has only 9 non-zeros. So, it's really really sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried solving it using &lt;a href=&quot;http://soplex.zib.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SoPlex&lt;/a&gt;, &lt;a href=&quot;http://www.netlib.org/lapack/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LaPACK&lt;/a&gt; and &lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SuperLU&lt;/a&gt; (these last 2 through &lt;a href=&quot;http://arma.sourceforge.net/docs.html#solve&quot; rel=&quot;nofollow noreferrer&quot;&gt;Armadillo&lt;/a&gt;). But all are too slow. On a 10000x10000 the best I have is 18s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there is another software that does the same task as mine, written in Python that uses Scipy (&lt;code&gt;scipy.sparse.linalg&lt;/code&gt; to be precise) that is ridiculously fast (can solve those systems in less than one second). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a library that is equivalent to Scipy or a way of porting Scipy to C++? I need to write the software in C++, for other reasons...&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: My code to call SuperLU/LaPACK through Armadillo is simply:&#xA;&lt;code&gt;voltages[i] = spsolve(laplacians[i],iflow[i],&quot;lapack&quot;);&lt;/code&gt;&#xA;Or &#xA;&lt;code&gt;voltages[i] = spsolve(laplacians[i],iflow[i],&quot;superlu&quot;);&lt;/code&gt;&#xA;No options have been given before.&lt;/p&gt;&#xA;" OwnerUserId="24229" LastEditorUserId="24229" LastEditDate="2017-05-12T00:42:33.867" LastActivityDate="2017-05-17T14:53:23.893" Title="Fast c++ library to solve very big sparse systems" Tags="|linear-solver|sparse-matrix|scipy|lapack|" AnswerCount="5" CommentCount="21" ContentLicense="CC BY-SA 3.0" />
  <row Id="26852" PostTypeId="2" ParentId="26849" CreationDate="2017-05-11T11:55:33.803" Score="0" Body="&lt;p&gt;Try &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigen&lt;/a&gt;? ... basic advice, I know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eigen has added several sparse solvers in the last time, and also offers &lt;a href=&quot;https://eigen.tuxfamily.org/dox-devel/group__TopicSparseSystems.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;benchmarks&lt;/a&gt; with established methods such as UMFPACK or SUPERLU against which it seems to perform quite well.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2017-05-11T11:55:33.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26853" PostTypeId="2" ParentId="26849" CreationDate="2017-05-11T12:00:12.877" Score="6" Body="&lt;p&gt;I second the idea of using Eigen, which is pretty efficient, but also very simple to include.&#xA;If you need a lot more performance, you could try to use &lt;a href=&quot;https://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc&lt;/a&gt; or  &lt;a href=&quot;https://trilinos.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trilinos&lt;/a&gt;. They are very powerful libraries to store and solve sparse systems, they allow for  a large number of iterative or direct solvers and are compatible with MPI for added performance. However, I think that they would be highly efficient even in serial.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2017-05-13T12:45:24.573" LastActivityDate="2017-05-13T12:45:24.573" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26855" PostTypeId="2" ParentId="26849" CreationDate="2017-05-11T13:58:06.760" Score="0" Body="&lt;p&gt;10000x10000 isn't that large for such a sparse matrix, so direct solvers should still work fine. In addition to the Eigen recommendation already made, here's another option: download &lt;a href=&quot;https://software.intel.com/en-us/articles/free-mkl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel MKL&lt;/a&gt; and then use the &lt;a href=&quot;https://software.intel.com/en-us/node/470282&quot; rel=&quot;nofollow noreferrer&quot;&gt;MKL PARDISO&lt;/a&gt; direct solver, which is in my experience much, much faster than SuperLU.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2017-05-11T13:58:06.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26856" PostTypeId="2" ParentId="26849" CreationDate="2017-05-11T14:53:58.477" Score="2" Body="&lt;p&gt;I would recommend &lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SuiteSparse&lt;/a&gt; or &lt;a href=&quot;http://docs.nvidia.com/cuda/cusparse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuSPARSE&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-05-11T14:53:58.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26858" PostTypeId="1" CreationDate="2017-05-11T18:32:04.643" Score="0" ViewCount="482" Body="&lt;p&gt;I am working on a trajectory analysis project using python and its data science related libraries. I am planning to implement Quadtrees for for trajectory analysis, each trajectory has sequence of (x,y) coordinates along with timestamp, speed, dist between consecutive points, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would kindly request if anyone who has experience with quadtrees and trajectory analysis, can provide with some useful links for Python implementation of quadtree. I would not ask that implementation should be focussed on trajectory analysis but should be able to provide an idea for the same. My ultimate goal is to cluster the similar trajectories (location and direction wise).&lt;/p&gt;&#xA;" OwnerUserId="24236" LastEditorUserId="9667" LastEditDate="2017-05-12T00:27:17.710" LastActivityDate="2017-05-12T00:27:17.710" Title="Python implementation of Quadtree for trajectory analysis" Tags="|python|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26859" PostTypeId="2" ParentId="26858" CreationDate="2017-05-11T18:43:50.363" Score="2" Body="&lt;p&gt;Look at &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.18.1/reference/spatial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scipy.spatial&lt;/code&gt;&lt;/a&gt;. It has an implementation of a &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.spatial.KDTree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;KD-tree&lt;/a&gt;, which should work fine for your application.&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2017-05-11T18:43:50.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26862" PostTypeId="1" AcceptedAnswerId="26872" CreationDate="2017-05-12T12:12:10.310" Score="1" ViewCount="1424" Body="&lt;p&gt;I was reading &lt;a href=&quot;http://ta.twi.tudelft.nl/nw/users/gijzen/CURSUS_DTU/LES5/TRANSPARANTEN/les5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these slides&lt;/a&gt; about preconditioners. I believe I grasp the idea of how they work but there is something that is still not making sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have the system $Ax=b$ and use a preconditioner we end up with a system $M^{-1}Ax = M^{-1}b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now say I decide to use an $LU$ factorization, it means that I need to compute $M = LU$ and to use it as preconditioner I need to invert it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is where I am a little bit missing, does this mean that we are actually computing the inverse for $M$? I guess there is some property of lower/upper matrices that makes the inverse easy but I can't seem to find it.&#xA;If we are actually computing the inverse and no property of lower/upper matrices makes the task easy, why do we bother to compute the inverse of the preconditioner instead of computing the inverse (or approximate the inverse) of $A$?&lt;/p&gt;&#xA;" OwnerUserId="3304" LastActivityDate="2022-08-03T07:29:17.427" Title="Is it necessary to invert precondition matrix for iterative solver?" Tags="|iterative-method|preconditioning|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26863" PostTypeId="2" ParentId="26847" CreationDate="2017-05-12T12:56:09.777" Score="3" Body="&lt;p&gt;Quantum Chemistry codes can get very complicated very fast. Even if you limit yourself to DFT, there are many functionals to support. There's going to be a trade off here for you. You can get a simple, smaller code base, but you'll end up with very few features. Or you can go with a larger, more mature code base, but it will be more complicated for you to package it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speaking of simple, consider the example code from LIBINT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/evaleev/libint/wiki/using-modern-CPlusPlus--interface-v22&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/evaleev/libint/wiki/using-modern-CPlusPlus--interface-v22&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This builds a fairly functional Hartree-Fock code in modern C++ using only a few hundred lines of code. Why not reproduce that in your new package? Of course, implementing density functionals, post Hartree-Fock methods, forces, etc is going to make your code grow quickly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: the full Hartree-Fock example code example including the 2 electron integrals is in the source code under tests/hartree-fock/hartree-fock.cc . &lt;/p&gt;&#xA;" OwnerUserId="23968" LastEditorUserId="23968" LastEditDate="2017-05-12T13:23:43.773" LastActivityDate="2017-05-12T13:23:43.773" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26864" PostTypeId="1" AcceptedAnswerId="26865" CreationDate="2017-05-12T14:10:56.700" Score="-2" ViewCount="7895" Body="&lt;p&gt;I am new to C++.&#xA;I am trying to set up a very basic TCG game (with only minion with no text).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think my issue might be linked with pointer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are the functions and attributes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Player *Game::Turn&#xA;Board Player::getBoard()&#xA;Card Card_collection::get(int i)&#xA;int Card::m_has_been_played_this_turn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Board inherits from Card_collection&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(*Turn).getBoard().get(i).m_has_been_played_this_turn = 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I have as a compiler error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Expression must be modifiable lvalue '=' : left operand must be l-value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please, where do you think is the error?&lt;/p&gt;&#xA;" OwnerUserId="24244" LastEditorUserId="24244" LastEditDate="2017-05-12T14:42:35.387" LastActivityDate="2017-05-12T14:50:56.997" Title="C++ - Expression must be modifiable lvalue" Tags="|c++|" AnswerCount="1" CommentCount="0" ClosedDate="2017-05-12T15:52:50.190" ContentLicense="CC BY-SA 3.0" />
  <row Id="26865" PostTypeId="2" ParentId="26864" CreationDate="2017-05-12T14:47:08.617" Score="0" Body="&lt;p&gt;To this isn't the right place to ask this question, so for future reference ask these types of questions on Stack Overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't really give enough code to be totally sure what's wrong, but one thing that comes to mind given the error is the following. First, note that your &lt;code&gt;get(int i)&lt;/code&gt; method is returning a copy of a &lt;code&gt;Card&lt;/code&gt;, not the actual ith &lt;code&gt;Card&lt;/code&gt; from the Card Collection. This means when you do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(*Turn).getBoard().get(i).m_has_been_played_this_turn = 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You are actually setting the &lt;code&gt;m_has_been_played_this_turn&lt;/code&gt; variable for a temporary &lt;code&gt;Card&lt;/code&gt; instance, which may be where the error is coming from since this temporary &lt;code&gt;Card&lt;/code&gt; instance should be an &lt;code&gt;rvalue&lt;/code&gt;, not a &lt;code&gt;lvalue&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to change this might be to change your code to the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Card card_ = (*Turn).getBoard().get(i);&#xA;card_.m_has_been_played_this_turn = 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this is still not going to accomplish what I believe you want, since &lt;code&gt;card_&lt;/code&gt; is just a copy of the ith &lt;code&gt;Card&lt;/code&gt; in your Card Collection. If you really want to modify the ith &lt;code&gt;Card&lt;/code&gt; in your Card Collection, change the &lt;code&gt;get(int i)&lt;/code&gt; definition to the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Card &amp;amp;  Card_collection::get(int i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That &lt;code&gt;&amp;amp;&lt;/code&gt; means you will pass a &lt;code&gt;Card&lt;/code&gt; out of &lt;code&gt;get(int i)&lt;/code&gt; by reference, allowing you to modify the &lt;code&gt;Card&lt;/code&gt; within the Card Collection directly. Then the original code you wrote to set &lt;code&gt;m_has_been_played_this_turn&lt;/code&gt; should be fine.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-05-12T14:47:08.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26867" PostTypeId="2" ParentId="26862" CreationDate="2017-05-12T16:26:20.947" Score="5" Body="&lt;p&gt;Based on your comments, it looks like your main difficulty is how having an LU factorisation for a given matrix $M$ makes it easy to find a vector $\mathbf{y}$ such that $M\mathbf{y}=\mathbf{b}$ for a given $\mathbf{b}$. Hopefully you're comfortable that if we have an easy method to do that, then this bit of the problem is solved, since we can treat $M^{-1}$ as a black box, where we're given an input and the method gives back an output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose for a second that we (somehow) know how to find a $\mathbf{z}$ such that $L\mathbf{z} = \mathbf{b}$ for a given $\mathbf{b}$. Then providing we can find&#xA;a $\mathbf{y}$ which gives $U\mathbf{y} = \mathbf{z}$, then &#xA;$$LU\mathbf{y} = L\mathbf{z} = \mathbf{b}, $$&#xA;and we're done. But how to find $\mathbf{y}$?&#xA;Since $U$ is upper triangular, then writing things as simultaneous equations, the last equation is just&#xA;$$ U_{n,n}y_n = z_n $$&#xA;This is easy to solve in one go as $y_n = \frac{z_n}{U_{n,n}}$. But now the equation one back from the last can be written as &#xA;$$ U_{n-1,n-1}y_{n-1} = - U_{n-1,n}y_n + z_{n-1},$$&#xA;which again is a one liner to solve, given that we've just found $y_n$ (this method is called backwards substitution if you want to look it up). Since getting the i-th component  means doing $n-i$ multiplications, and one division, and there are $n$ lines, we're doing about $n^2$ operations, which is better than general matrix inversion. Even better, if $U$ actually has fewer entries than a general upper triangular matrix, then we have even less maths to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now we need to go back to finding $\mathbf{z}$. But the &lt;em&gt;first&lt;/em&gt; equation in $L\mathbf{z}=\mathbf{b}$ is just&#xA;$$ L_{1,1} z_1 = b_1, $$&#xA;so we can play exactly the same game we played before, except using &lt;em&gt;forward&lt;/em&gt; substitution.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2017-05-12T16:26:20.947" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26868" PostTypeId="1" CreationDate="2017-05-12T20:34:45.873" Score="3" ViewCount="132" Body="&lt;p&gt;I am making an n-body simulation in python. There are many different methods to numerically solve the system of differential equations governing the gravitational interactions between the $n$ particles; the challenge of n-body simulations is to find an integration scheme which is both accurate and fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the moment, I am working with n small (say $n \le 5$). I began with the most naive method of Euler integration, which is first order. Even with two bodies, this method can produce pretty bad errors if the bodies come very close (since then the force between them becomes very large, so their accelerations become very large). Of course, one can always reduce the time-step to get more accuracy, but at the expense of computation time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next I would like to implement a higher order method which has better stability properties etc like leapfrog integration or Runge-Kutta. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect these methods to give much better results, but the problem is &lt;strong&gt;I am unsure how to measure quantitatively the accuracy of my simulation. What metric should I use?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the issues is that I can't tell if the simulation is giving realistic results, because I haven't been using the real value of the gravitational constant $G$ and none of the distances/times/masses in my simulation have units. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why I haven't been paying attention to units is because, say if I wanted to do a scale sun-earth-moon simulation; since the sun-earth distance is about 370 times the earth-moon distance, it's not possible for everything to fit inside the display window and also be able to distinguish the earth and the moon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other problem is using the real value of G and real masses etc leads to computations involving big numbers. But, keeping everything else fixed, choosing lower values of $G$ reduces the errors (because the forces are then smaller). So I have no real way of telling if the simulation is accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So how can I rigorously test the accuracy of the simulation, and what can I do concerning units?&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="24246" LastActivityDate="2017-05-12T20:34:45.873" Title="How can I evaluate the accuracy of my n-body simulation?" Tags="|algorithms|computational-physics|accuracy|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26869" PostTypeId="1" CreationDate="2017-05-12T21:24:59.697" Score="1" ViewCount="85" Body="&lt;p&gt;This will be a vague question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I was writing a finite element matrix assembly routine, a colleague noticed that I had a bug in my code because the sparsity pattern of the one of the blocks didn't look correct to him.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He explained to me that the block operator is a Laplacian operator, which is the discrete analogue of the continuous second derivative operator, hence there should be no coupling between the off diagonal block terms of the matrix and therefore that I had a bug in my code, since the sparsity pattern did not match. I was blown away by the alternative way of viewing the problem, and I want to learn more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He didn't know of any reference materials that teach things like that; he just claimed to have learned it by experience. Does anyone know of a textbook or  resource on discrete operators which can help train my brain to think about numerics in this sort of sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I come from a mainly computer science background, and have more coding intuition than this sort of mathematical intuition.&lt;/p&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="18009" LastEditDate="2017-05-13T09:56:05.560" LastActivityDate="2017-05-13T09:56:05.560" Title="Discrete operator textbooks" Tags="|finite-element|reference-request|discretization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26870" PostTypeId="2" ParentId="4993" CreationDate="2017-05-13T13:50:43.610" Score="1" Body="&lt;p&gt;Use $\partial \rho \phi = \rho \partial \phi + \phi \partial \rho$ in the equation for the component fraction, then apply the continuity equation to obtain&#xA;\begin{equation}&#xA;\rho \frac{\partial \phi} {\partial t} + \frac1\epsilon \rho u \frac{\partial \phi}{\partial x} = 0.&#xA;\end{equation}&#xA;Assuming $\rho &amp;gt; 0$ in the whole domain, it can be canceled in this equation. Substitute $u$ with Darcy's equation as before and solve this equation for $\phi$ using the approach you suggested and then use $\rho = f(\phi)$ to get $\rho$. Alternatively, if $f$ can be inverted, you might as well solve the original continuity equation for $\rho$ and get $\phi$ as $\phi = f^{-1}(\rho)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the boundary condition for $u$, substitute  the equation of state and Darcy's equation into $u(0) = u_{inlet}$ and discretize the spatial derivative. Though I'm not sure if the boundary conditions described in your question are consistent with the problem you're trying to solve.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-05-13T13:50:43.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26871" PostTypeId="1" AcceptedAnswerId="26873" CreationDate="2017-05-13T14:00:08.487" Score="7" ViewCount="8185" Body="&lt;p&gt;I wonder if there is any study that compares the performance of kd-tree vs brute-force nearest neighbor search on GPU. &lt;a href=&quot;https://forums.khronos.org/showthread.php/7402-kd-tree-nearest-neighbour&quot; rel=&quot;noreferrer&quot;&gt;Post #4 on this page&lt;/a&gt; suggests that kd-tree may not be the optimal algorithm for GPU but I wonder if there is any data supporting this claim?&lt;/p&gt;&#xA;" OwnerUserId="17178" LastActivityDate="2020-01-02T17:00:06.580" Title="Performance of kd-tree vs brute-force nearest neighbor search on GPU?" Tags="|gpu|nearest-neighbors|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26872" PostTypeId="2" ParentId="26862" CreationDate="2017-05-13T14:46:08.330" Score="5" Body="&lt;p&gt;OK - so your original equation is $$Ax = b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you've come up with a good preconditioner for $A$, call this $M$. Also, say you have pre-computed an LU-decomposition for this $M$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M = L_m U_m$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I've used the subscript $m$ to indicate that it is the LU decomposition of $M$ that we're talking about here (and not that of $A$). Then, assuming left preconditioning, you will be solving the preconditioned linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ M^{-1} A x = M^{-1}b,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or, equivalently&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (L_m U_m)^{-1} A x = (L_m U_m)^{-1} b,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ U_m^{-1} L_m^{-1}  A x = U_m^{-1} L_m^{-1} b.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the preconditioned system matrix is now $U_m^{-1} L_m^{-1}A$, which is (if you have chosen a good preconditioner) much better conditioned than $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you can&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(a)&lt;/strong&gt; (as a purely academic exercise,) solve the system with a non-preconditioned Krylov method (say GMRES). Because the system matrix is well conditioned, you can expect quick convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Krylov algorithm will need to multiply this matrix to an arbitrary (known) vector, call that $r$. In other words, what the Krylov procedure needs from you is a routine that, given $r$, computes $$z = U_m^{-1} L_m^{-1} A r$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is equivalent to solving the linear system $L_m U_m z = A r$ (for $z$). This system is easy (=cheap) to solve because the matrices $L_m$ and $U_m$ are triangular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;or&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(b)&lt;/strong&gt; (what you'd do in practice) run preconditioned CG on the preconditioned system. CG will apply the inverse preconditioner to the residual vector $r := b - Ax$. Again, this just means it solves the linear system $L_m U_m z = r$ (for $z$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to summarize: &lt;strong&gt;never&lt;/strong&gt; invert matrices &lt;em&gt;explicitly&lt;/em&gt; (you can't do that in practice for large matrices anyway), instead solve the corresponding linear system (which is &lt;em&gt;mathematically equivalent to applying the inverse matrix operator to a known vector&lt;/em&gt;).&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-05-16T18:38:56.907" LastActivityDate="2017-05-16T18:38:56.907" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26873" PostTypeId="2" ParentId="26871" CreationDate="2017-05-13T15:47:58.153" Score="14" Body="&lt;p&gt;Ultimately, naive brute-force KNN is an $O(n^2)$ algorithm, while kd-tree is $O(n \log n)$, so at least in theory, kd-tree will eventually win out for a large enough $n$. In practice, the leading constants for a GPU implementation may be vastly different --- we may be comparing $0.0001n^2$ vs $1000n\log n$ --- so it may indeed be the case that the former wins out for &lt;em&gt;practical problem sizes&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the reason for the difference in leading constants come down to parallelizability and memory access patterns. Naive KNN is &lt;em&gt;embarrassingly&lt;/em&gt; parallelizable, and can be implemented entirely using vectorized sequential memory access. By contrast, kd-tree is naturally serial, and requires extensive conditional statements and random memory access. Algorithms like the former enjoy a massive speed-up on a GPU, while those like the latter often run slower on a GPU than a CPU.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2017-05-13T16:24:36.027" LastActivityDate="2017-05-13T16:24:36.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26875" PostTypeId="1" CreationDate="2017-05-13T18:39:27.323" Score="1" ViewCount="73" Body="&lt;p&gt;I need help programming a script, in MATLAB, that for assigned values compute the optimal transmission angle of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Four-bar_linkage#Classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;crank-rocker mechanism&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameters are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$L$: crank's angle of oscillation, for example from -20° to 150°&lt;/li&gt;&#xA;&lt;li&gt;$m$: rocker's angle of oscillation, it's a fixed value.&lt;/li&gt;&#xA;&lt;li&gt;$t$:  $\tan(L/2)$&lt;/li&gt;&#xA;&lt;li&gt;$u$: $\tan((L-m)/2)$&lt;/li&gt;&#xA;&lt;li&gt;$v$: $\tan(m/2)$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is a condition on $L$, $L&amp;lt;(90+(m/2))$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is a cubic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Q^3 + 2Q^2 - t^2 Q- \frac{t^2(1+t^2)}{u^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $Q=t^2/x^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The roots of $Q$ must be real and positive.&#xA;$x$ is the variable and it should satisfy the following inequalities:&#xA;$x^2 \geq 1$ and $x^2 \leq u^2 t^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the lengths of the 4-bar are&#xA;\begin{align}&#xA;a &amp;amp;= \sqrt{\frac{v^2}{1 + v^2}}\cr&#xA;b &amp;amp;= \sqrt{\frac{x^2 v^2}{1 + v^2}}\cr&#xA;c &amp;amp;= \sqrt{\frac{x^2 + t^2}{1 + t^2}}\cr&#xA;d &amp;amp;= \sqrt{\frac{u^2 + x^2}{1 + u^2}}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solve the equation for $Q$ then calculate $x$. Now the transmission angle (max and min) depends on $a(x)$, $b(x)$, $c(x)$ and $d(x)$. I have a different transmission angle (max and min) for all the different values of crank's angle of oscillation. For example as, Istated previously from $-20$ to $(90+(m/2))$ (the lower bound is discretionary)&#xA;$$\begin{aligned}&#xA; \mathit{angtrasm}_{\min}&amp;amp;= \arccos\left({\frac{b^2 + c^2-(d-a)^2}{2bc}}\right)&#xA;\\&#xA;\mathit{angtrasm}_{\max} &amp;amp;= \arccos\left({\frac{b^2 + c^2-(d + a)^2}{2bc}}\right)&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is the following&#xA;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Compute transmission angle and lengths of bars&#xA;clear all&#xA;&#xA;m = 32;   %rocker’s oscillation&#xA;i = 1;&#xA;f = 1;&#xA;&#xA;for l = 1: (90 + m/2 );  % crank’s oscillation -pi (there is a problem here i can’t start from -20 for example)&#xA;  L(i) = l + 180 ;       %crank’s oscillation&#xA;  t = tand(L(i)/2);&#xA;  u = tand((L(i)-m)/2);&#xA;  v = tand(m/2);&#xA;  p = [1 2 -(t^2) -((t^2 + (1 + t^2))/(u^2))];&#xA;  if L(i) = = m&#xA;    fprintf('Value of crank’s oscillation is not acceptable \n' , L(l));&#xA;  else&#xA;    Q = roots(p);&#xA;    for j = 1:3&#xA;      if isreal(Q(j)) &amp;amp;&amp;amp; Q(j)&amp;gt;0&#xA;        x = abs(t/sqrt(Q(j)));&#xA;        if x^2 &amp;gt; = 1 &amp;amp;&amp;amp; x^2 &amp;lt; = u^2*t^2&#xA;          a = sqrt(v^2/(1 + v^2));&#xA;          b = sqrt(x^2*v^2/(1 + v^2));&#xA;          c = sqrt((x^2 + t^2)/(1 + t^2));&#xA;          d = sqrt((u^2 + x^2)/(1 + u^2));&#xA;          angtrasm_min(f) = acosd((b^2 + c^2-(d-a)^2)/(2*b*c));&#xA;          angtrasm_max(f) = acosd((b^2 + c^2-(d + a)^2)/(2*b*c));&#xA;          scarto_min(f) = abs(90-angtrasm_min(f));&#xA;          scarto_max(f) = abs(90-angtrasm_max(f));&#xA;&#xA;          theta(f) = L(i);&#xA;          r1(f) = a;&#xA;          r2(f) = b;&#xA;          r3(f) = c;&#xA;          r4(f) = d;&#xA;          fprintf('The max transmision angle is %d when theta is %d\n',angtrasm_max(f),L(i));&#xA;          fprintf('The max transmision angle is %d when theta is %d\n',angtrasm_min(f),L(i));&#xA;&#xA;          fprintf('.................................\n');&#xA;          f = f + 1;&#xA;        end&#xA;      end&#xA;    end&#xA;  end&#xA;  i = i + 1;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24250" LastEditorUserId="9667" LastEditDate="2017-05-15T02:38:44.460" LastActivityDate="2017-05-15T02:38:44.460" Title="Optimal synthesis of crank–rocker linkages" Tags="|optimization|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26876" PostTypeId="1" CreationDate="2017-05-14T00:47:19.377" Score="1" ViewCount="31" Body="&lt;p&gt;I'm about to get started trying to use &lt;a href=&quot;https://github.com/appliedtopology/javaplex&quot; rel=&quot;nofollow noreferrer&quot;&gt;Javaplex&lt;/a&gt; for the first time for some application in topological data analysis (TDA) and I want to use a custom defined metric. From looking over the documents, I can do this if I build the distance matrix for all of my points. My question is twofold:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can I do it by just defining the metric between two points (it's a simple function to define, just the Euclidean cross-product metric between a Euclidean space and a circle) &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;and if so,&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Would it make sense computationally, or should I just build the distance matrix for each of my datasets? For the context, I have 800 datasets, each with about 40-50 points on a torus in $(x,y,\theta)$-coordinates, so I need to define my torus metric.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="24254" LastEditorUserId="20688" LastEditDate="2017-05-14T08:23:09.653" LastActivityDate="2017-05-14T08:23:09.653" Title="Use custom defined metric(not matrix) in Javaplex" Tags="|computational-geometry|data-analysis|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26877" PostTypeId="1" CreationDate="2017-05-14T01:24:11.883" Score="1" ViewCount="2794" Body="&lt;p&gt;Being new to numerical analysis techniques, in particular RK2, I decided the best way to jump in is by using python to solve the well known mass-spring oscillator using RK2 techniques. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that the step size seems to influence the period of my solution and I do know why and would appreciate it if someone could help me better understand what I am doing wrong. Commenting on the code is welcomed! Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is as follows (this is not a textbook or homework problem!): A mass is attached to a spring with a linear spring constant. At time $t=0$ the mass is displaced from equilibrium, $x=0$, to the position $x(t=0) = 1 cm$. The mass is released and left to oscillate about equilibrium. Use the method of RK2 to plot the evolution of the masses position as a function of time. The required first order differential equations for the problem are:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot{x} = -\omega x(t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dx}{dt} = \dot{x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the RK2 functions I am using are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v(t_{n+1}) = v(t_n) -\omega x(t_n)\delta t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x(t_{n+1}) = x(t_n) + v(t_n)\delta t $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ k2_{x} = \delta t (v(t_n) - \omega x(t_n)\frac{\delta t}{2}) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ k2_{v} = \delta t (\omega x(t_n) + v(t_n) \frac{\delta t}{2}) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last two function are the midpoint values that I think are used for the RK2 approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along with the implemented code : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;print &quot;RK2 Method for Oscillating Spring&quot;&#xA;&#xA;#Spring Constant&#xA;k = 1.0&#xA;#Mass&#xA;m = 0.02&#xA;#Angular frequency squared&#xA;w = k/m&#xA;&#xA;period = 2*np.pi*(np.sqrt(1/w))&#xA;&#xA;#Initial Conditions&#xA;x_start = 1.0&#xA;v_start = 0.0&#xA;&#xA;#Set loop variables&#xA;x_position = x_start&#xA;velocity = v_start&#xA;&#xA;#Step Size&#xA;dt = 0.001&#xA;&#xA;#Arrays to store variable values&#xA;x_plot = []&#xA;v_plot = []&#xA;time = []&#xA;&#xA;#Plot the numerically determined position over&#xA;# the time interval 1 to 10 seconds in steps of 0.1&#xA;for t in np.arange(1,10,0.1):&#xA;&#xA;    #The two equations below are used in the Euler Method approach&#xA;    #Their solution's period, which can be plotted, varies depending on the step size&#xA;    # and I know it shouldn't&#xA;    #x_position = x_position + dt*velocity&#xA;    #velocity = velocity - dt*w*x_position&#xA;&#xA;    #The equations below are used for RK2 method. I don't know why I cant get it to work.&#xA;velocity = velocity + dt*(-w*x_position)&#xA;    k2_x = dt*(velocity + (-w*x_position*dt/2.0))&#xA;    x_position = x_position + k2_x&#xA;&#xA;    k2_v = -dt*w*(x_position + velocity*dt/2.0)&#xA;    velocity = velocity + k2_v&#xA;&#xA;&#xA;    x_plot.append(x_position)&#xA;    v_plot.append(velocity)&#xA;    time.append(t)&#xA;&#xA;&#xA;plt.subplot(3,1,1)&#xA;plt.plot(time,x_plot)&#xA;plt.subplot(3,1,2)&#xA;plt.plot(time,v_plot)&#xA;plt.subplot(3,1,3)&#xA;plt.plot(x_plot,v_plot) # illustrate divergence over number of steps&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My problem is that I don't understand why varying the step size, dt, that the period of the solution, a sinusoidal function, changes. &lt;/p&gt;&#xA;" OwnerUserId="24256" LastEditorUserId="24256" LastEditDate="2017-05-15T19:37:32.393" LastActivityDate="2017-05-15T19:37:32.393" Title="Using RK2 Method to solve the simple harmonic oscillator of a horizontal mass on a spring (1D)" Tags="|numerical-analysis|python|ode|computational-physics|runge-kutta|" AnswerCount="0" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26878" PostTypeId="1" AcceptedAnswerId="26880" CreationDate="2017-05-14T05:13:23.777" Score="1" ViewCount="279" Body="&lt;p&gt;I am using &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; to simulate the reaction of a system with known input signals via integration. The simplified code below illustrates what I'm doing. It simulates the response of a number of first order systems to sinusoidal inputs by first generating the data and interpolating to the data in the integration function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to speed up the simulations by exploiting the fact that the time points for the data table are regularly spaced and that we are doing interpolation between the same rows of the table for all the input variables. Profiling has shown that the actual code I'm working on is indeed spending a significant amount of time in &lt;code&gt;numpy.interp&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exact question then is if I should just roll my own interpolation function which handles this special case or if there exists an off-the shelf solution which handles this case efficiently. I have searched for this but come up empty handed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;import scipy.integrate&#xA;&#xA;import matplotlib.pyplot as plt&#xA;&#xA;Ntime = 400&#xA;Ninputs = 4&#xA;time = numpy.arange(Ntime)&#xA;&#xA;# Generate some out-of-phase sinusoids for input data&#xA;DATA = numpy.array([numpy.sin(time/10 - i) for i in range(Ninputs)]).T&#xA;&#xA;# Time constants&#xA;taus = numpy.ones(Ninputs)*10&#xA;&#xA;def intfun(x, t):&#xA;    # I would like to speed up this lookup:&#xA;    inputs = numpy.array([numpy.interp(t, time, DATA[:, i]) &#xA;                          for i in range(Ninputs)])&#xA;&#xA;    return -(1/taus)*x + inputs  # first order&#xA;&#xA;x0 = numpy.ones(Ninputs)&#xA;&#xA;x = scipy.integrate.odeint(intfun, x0, time)&#xA;&#xA;plt.plot(time, x)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16654" LastEditorUserId="7574" LastEditDate="2017-05-14T12:37:22.623" LastActivityDate="2017-05-14T12:37:22.623" Title="Fast table interpolation on regular time data for ODE" Tags="|ode|interpolation|numpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26879" PostTypeId="1" CreationDate="2017-05-14T06:41:39.413" Score="1" ViewCount="535" Body="&lt;p&gt;I have  a PDE equation with Robin Boundary condition in an annulus system and I should solve it by finite volume method:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/C1Rgw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C1Rgw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;        &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}     &#xA;\frac{\partial T_f}{\partial t} - k \left(\frac{\partial^2 T_f}{\partial r^2} + \frac{1}{r} \frac{\partial T_f}{\partial r} \right) = -Q                     &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have any problem in the discretization of inner points by finite volume method,&#xA;but in $r_{ID}$ and $r_{OD}$, it has robin boundary condition which I do not know how to discretize it.                    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\left.\frac{\partial T_f}{\partial r}\right|_{r=r_{ID}} &amp;amp;= h(T_f-T_w ) \\&#xA;\left.\frac{\partial T_f}{\partial r}\right|_{r=r_{OD}} &amp;amp;= h(T_f-T_g )&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(h&amp;amp;k are constant)&lt;br&gt;&#xA;I would appreciate it if you could help me to discretize this boundary condition.&lt;/p&gt;&#xA;" OwnerUserId="17594" LastEditorUserId="17594" LastEditDate="2017-05-14T16:19:44.753" LastActivityDate="2017-05-14T16:19:44.753" Title="Implement Robin boundary condition (finite volume)" Tags="|pde|boundary-conditions|finite-volume|discretization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26880" PostTypeId="2" ParentId="26878" CreationDate="2017-05-14T09:41:56.237" Score="2" Body="&lt;p&gt;I see the following inefficiencies in your approach:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For every call of &lt;code&gt;intfun&lt;/code&gt;, &lt;code&gt;numpy.interp&lt;/code&gt; has to start from zero searching the point from &lt;code&gt;DATA&lt;/code&gt; used for interpolation every time it is called. Though this uses a compiled binary search, it takes some time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For every call of &lt;code&gt;intfun&lt;/code&gt;, you have a considerable overhead from &lt;code&gt;numpy.interp&lt;/code&gt; which was made for interpolating more than one data point.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You do not exploit the data being equidistant at all.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You have to loop over the input dimension (&lt;code&gt;Ninputs&lt;/code&gt;), as &lt;code&gt;numpy.interp&lt;/code&gt; only works with one-dimensional data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, what can you do?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;To avoid the search, you can exploit the regular sampling and only use the relevant part of the array for interpolation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def intfun(x,t):&#xA;    j = int(t)&#xA;    inputs = numpy.array([&#xA;                numpy.interp( t, time[j:j+1], DATA[j:j+1,i] )&#xA;                for i in range(Ninputs)&#xA;                ])&#xA;    return -(1/taus)*x + inputs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Empirically, this gives a slight speed boost.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can exploit your knowledge of the regular sampling &lt;strong&gt;and&lt;/strong&gt; do the interpolation yourself. This way you get rid of the overhead from &lt;code&gt;numpy.interp&lt;/code&gt; and you can do a multidimensional interpolation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def intfun(x,t):&#xA;    j = int(t)&#xA;    t -= j&#xA;    inputs = DATA[j]*(1-t) + DATA[j+1]*t&#xA;    return -(1/taus)*x + inputs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Empirically, this gives a better speed boost than the above.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You use a fixed step size for the integration (instead of an adaptive integrator like &lt;code&gt;odeint&lt;/code&gt;). This allows you to perform the interpolation beforehand in a vectorised (and hence efficient) manner. However, you may need more steps and lose time this way. Also you have to go through the trouble of finding a good step size and you have to find or implement a fixed-step-size integrator for Python.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You switch to a compiled language with static types. This way, you get rid of the Python overhead from the interpolation operations in suggestion 2 and you can make use of an adaptive step size at the same time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2017-05-14T09:47:12.187" LastActivityDate="2017-05-14T09:47:12.187" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26884" PostTypeId="1" CreationDate="2017-05-14T16:01:57.883" Score="0" ViewCount="856" Body="&lt;p&gt;From an old, wise engineering book Peterson's Stress Concentration Factors (&lt;a href=&quot;http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470048247.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470048247.html&lt;/a&gt;  page 324) I've got the following problem: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/AVaCi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AVaCi.png&quot; alt=&quot;Periodic structure with holes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is 2D infinite periodic structure with round holes of radius r with tension applied to it. The relation of maximum stress in the body to the applied one should follow nice 1/x like curve, see the picture. While I'm trying to reproduce this example by numerical simulations, I'm getting that max stress is almost independent of hole size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use displacement based weak formulation for elasticity with periodic boundary conditions remaping top to bottom and left to right, with exception of corners (0,0), (0,1), (1,0), (1,1).&#xA;For problem to be well-posed I've fixed displacement vector in one point (0,0) to be equal (0,0).&#xA;Am I right? Is fixing one point enough? I still can rotate the unit square though. Do I need to fix an additional point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tension is applied via Neumann boundary conditions with tension applied in the following way. First, I define vector $g = (1.0, 0)$ and then I create right hand side for the weak form of equation:&#xA;$$&#xA;\int C\varepsilon(u):\varepsilon(v)dx = \int_{left border} (-g,v)ds +  \int_{right border} (g,v)ds, &#xA;$$&#xA;where the differential&#xA;operator $\varepsilon$ is the symmetric part of the gradient and $v$ is a test function obeying periodic boundary conditions. &#xA;This is the place I'm most doubted about applying tension correctly. Is it correct? Is my problem well-posed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. I'm solving it with Fenics and code can be found here &lt;a href=&quot;https://fenicsproject.org/qa/13464/verifying-elasticity-benchmark-structure-boundary-conditions&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fenicsproject.org/qa/13464/verifying-elasticity-benchmark-structure-boundary-conditions&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="16620" LastEditorUserId="16620" LastEditDate="2017-05-18T16:02:39.520" LastActivityDate="2017-05-21T18:45:48.290" Title="Correctly setting boundary condition for periodic linear elasticity problem" Tags="|finite-element|pde|boundary-conditions|elliptic-pde|well-posedness|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="26886" PostTypeId="1" AcceptedAnswerId="26889" CreationDate="2017-05-15T09:55:44.113" Score="5" ViewCount="3282" Body="&lt;p&gt;I want to evaluate the Surface force integral in an FEM procedure.&lt;a href=&quot;https://i.stack.imgur.com/esQM9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/esQM9.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic reference tet is shown in the figure. The faces are numbered corresponding to the node opposite to them. For example the nodes 2-3-4 which forms a face opposite to node 1 is labeled as 1, and  so on... I now transform the tet into the volume coordinate denoted by&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;$(\xi_1, \xi_2, \xi_3, \xi_4 )$ which are also the shape functions of the linear element. The surface force vector on a face is denoted by $f_e$.  I now need to evaluate the integral $$\int_{\Gamma_e} N^T f_e \ d\Gamma_e $$ where $\Gamma_e$ represent any face. $N^T$ represent the shape function matrix given by $$\begin{bmatrix}&#xA;  \xi_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \xi_4 &amp;amp; 0 &amp;amp; 0 \\&#xA;  0 &amp;amp; \xi_1 &amp;amp; 0 &amp;amp; \cdots &amp;amp;  0 &amp;amp; \xi_4 &amp;amp; 0 \\&#xA;  0 &amp;amp; 0 &amp;amp; \xi_1 &amp;amp; \cdots &amp;amp;  0 &amp;amp; 0 &amp;amp; \xi_4 &#xA; \end{bmatrix} $$ &#xA;To evaluate the surface integral one takes the trace of the matrix on the particular face. For example on face one,  $\xi_1=0$.  I now need to use a Gaussian Quadrature and  not sure how to evaluate the transformed Jacobians. I am actually implementing the above procedure on a 10 node element. So actually confused on how to implement the surface quadratures. Another source of doubt is that in case of the linear element one can do a node lumping i,e distributing the surface load equally among all the nodes and that gives a very simple expression for the surface vector. Can this be also be done for a higher element?&lt;br&gt;&#xA;&#xA;&lt;/p&gt;&#xA;" OwnerUserId="23149" LastEditorUserId="24203" LastEditDate="2017-05-17T16:17:40.147" LastActivityDate="2021-09-22T11:08:07.840" Title="Evaluating the surface integral in an FEM (Finite Elements Method) procedure" Tags="|finite-element|integration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26889" PostTypeId="2" ParentId="26886" CreationDate="2017-05-15T21:16:49.570" Score="7" Body="&lt;p&gt;The particular surface integral you want to calculate is basically a specific case of integrating a function over a surface defined&#xA;in terms of two parametric coordinates. Lets first consider this general case. A fundamental relation from differential geometry is&#xA;(using your notation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;d{\bf \Gamma_e} = &#xA;\left\{\begin{array}{c}&#xA;\frac{\partial x}{\partial\xi_1} \\&#xA;\frac{\partial y}{\partial\xi_1} \\&#xA;\frac{\partial z}{\partial\xi_1} &#xA;\end{array}\right\} &#xA;\times &#xA;\left\{\begin{array}{c}&#xA;\frac{\partial x}{\partial\xi_2} \\&#xA;\frac{\partial y}{\partial\xi_2} \\&#xA;\frac{\partial z}{\partial\xi_2} &#xA;\end{array}\right\} &#xA;$$&#xA;where $\xi_1$ and $\xi_2$ are the two parametric coordinates of the surface. An integral of a function, $f$ over the surface would be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int f d\Gamma_e = \int f(\xi_1, \xi_2) ||d{\bf \Gamma_e}||_2 d\xi_1 d\xi_2$$&#xA;where $||d{\bf \Gamma_e}||_2$ is just the magnitude of the vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now lets consider the specific case of faces of the ten-node tetrahedron. In an isoparametric formulation, $x, y,$ and $z$ are written in terms of&#xA;the shape functions, $N_I$, e.g.&#xA;$$ x = N_I(\xi_1, \xi_2) x_I $$&#xA;where $ x_I$ are the values of $x$ at the nodes of the triangular face.&#xA;The expressions for $N_I$ on a face can be obtained from the element shape functions by&#xA;specializing them for a particular face, or equivalently, by using the shape functions for the six-node triangle. From these equations, you can easily evaluate the vector cross product above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most general way to actually perform the integration is by using the appropriate Gauss-Legendre rule for the triangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, you asked about the difference between formally evaluating the consistent load integral and simply lumping equal values at the&#xA;nodes. The equivalent nodal loads are quite different from the simple lumped values. For a simple, straight-sided element, the equivalent nodal loads are non-zero only at the mid-side nodes. If you have a fine enough mesh, the simple lumped approach will give you the correct answer but it is not optimum in the Galerkin-FEM sense.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-05-16T19:51:43.490" LastActivityDate="2017-05-16T19:51:43.490" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26893" PostTypeId="1" AcceptedAnswerId="26899" CreationDate="2017-05-16T09:49:28.487" Score="0" ViewCount="2221" Body="&lt;p&gt;I am working on numerical simulation of breast model to evaluate the use of thermal imaging for breast cancer detection. To do this I need to construct an ellipsoid rotated 30 degrees around y-axis and cut from x–y plane. The outer major and minor semi-axis lengths of the ellipsoid are set to 0.08 m and 0.05 m, respectively. I am working on Fluent for the numerical simulation and using Design Modeller to generate the model. I need help on how to create an ellipsoid using any 3D CAD software (after which it can be imported into design modeller) &lt;/p&gt;&#xA;" OwnerUserId="24266" LastActivityDate="2017-05-17T05:14:59.217" Title="How to construct an ellipsoid using Ansys design modeller (or any other 3D CAD software)" Tags="|numerical-analysis|numerical-modelling|" AnswerCount="1" CommentCount="1" ClosedDate="2017-05-20T21:30:37.913" ContentLicense="CC BY-SA 3.0" />
  <row Id="26894" PostTypeId="1" CreationDate="2017-05-16T11:51:34.017" Score="2" ViewCount="207" Body="&lt;p&gt;We know that &#xA;\begin{align*}&#xA;\|u\|_{0,\infty,\Omega}={\text{ess} \sup}_{x\in \Omega}|u|.&#xA;\end{align*}&#xA;Moreover, the nonconforming Crouzeix-Raviart finite element space is&#xA;\begin{align*}&#xA;V_h=\{ v|_K\in P_1(K),\ v ~\text{is continuous at the midpoints of edges and } v=0 \text{ at the midpoints on boundary edges}\}.&#xA;\end{align*}&#xA;For a function $w\in V_h$, we know that $w\notin H_0^1(\Omega)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is what is the definition of $\|w\|_{0,\infty,\Omega}$ and $\|\nabla w\|_{0,\infty,\Omega}$?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, does the inversive inequality&#xA;\begin{align*}&#xA;\|\nabla w\|_{0,\infty,\Omega}\leq Ch^{-1}\|w\|_{0,2,\Omega}, w\in V_h&#xA;\end{align*}&#xA;hold?&lt;/p&gt;&#xA;" OwnerUserId="7494" LastEditorUserId="20688" LastEditDate="2017-05-16T18:05:01.490" LastActivityDate="2017-05-21T11:16:28.990" Title="What's the definition of $L^{\infty}$-norm for nonconforming finite element?" Tags="|finite-element|pde|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26895" PostTypeId="1" AcceptedAnswerId="26896" CreationDate="2017-05-16T14:12:16.760" Score="1" ViewCount="142" Body="&lt;p&gt;My coworkers and I have developed a c++ library implementing a &quot;3D Geological Model&quot; class. Goal of this library is to provide a data structure that is used in computational codes (such as Finite Element codes etc...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This model class contains &quot;Entities&quot; (i.e. Corner, Line, Surface, Volume). Each Entities own a Mesh, which contains vertices in a 3D space and the connectivity informations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would like to handle also &quot;2D Geological Model&quot;, which has the same structure (i.e. it contains set of Corner, Line, Surface). But this time, these Entities own a Mesh which contains vertices in a 2D space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One objective is to give the user the ability to run their computational codes with the same API, even if it is on 3D or 2D &quot;Geological Model&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does someone here have faced the same problem ? One solution is to template all the classes by the space dimension, but it seems a bit heavy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers,&lt;/p&gt;&#xA;" OwnerUserId="17731" LastActivityDate="2017-05-23T10:15:32.790" Title="How to handle 2D and 3D models efficiently" Tags="|mesh|data-structures|3d|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26896" PostTypeId="2" ParentId="26895" CreationDate="2017-05-16T21:33:47.347" Score="2" Body="&lt;p&gt;I use C++ template to parameterize dimension in a fluid simulation project.&#xA;It is not directly related to your application, but I think some general ideas would be the same.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Vectorize operations for vector/array/point/vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe you already have this implemented.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Provide iterators to access topology in a model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way you can abstract(hide) dimensional details, such as &quot;how many adjacent faces/edges do I have for a given element?&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Prefer the Structure-Of-Array(SOA) memory layout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In SOA layout you put the same attributes from different objects in an array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this is a common way to define vertices:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;struct Vertex {&#xA;    VECTOR&amp;lt;D&amp;gt; position;&#xA;    double weight;&#xA;    // other attributes&#xA;};&#xA;std::vector&amp;lt;Vertex&amp;gt; vertices;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is called Array-Of-Structure. One potential issue of this is that the layout could vary for different dimensions. As a result you might have alignment problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, I would suggest the SOA way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  struct Vertices {&#xA;      std::vector&amp;lt;VECTOR&amp;lt;D&amp;gt;&amp;gt; positions;&#xA;      // or&#xA;      // std::vector&amp;lt;double&amp;gt; px;&#xA;      // std::vector&amp;lt;double&amp;gt; py;&#xA;      // ...&#xA;      std::vector&amp;lt;double&amp;gt; weights;&#xA;      // other attribute arrays&#xA;  };&#xA;  Vertices vertices;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should have less alignment issues when all attributes of same type are stored together.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instantiate the template class in .cpp files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In C++ a template class will not be instantiated when the compiler does not see a use of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your lib and your user code are compiled separately, your lib will not include your class implementation because it does not see its use, and the compiler will throw undefined reference errors in your user project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So remember to instantiate your class for each dimension you care(usually 2 and 3) manually in .cpp files.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One solution is to template all the classes by the space dimension, but it seems a bit heavy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not sure what do you mean by &quot;heavy&quot;.&#xA;Indeed, instantiating the template classes for each dimension will increase the code size. But I have not seen this is a problem in practice.&lt;/p&gt;&#xA;" OwnerUserId="7258" LastActivityDate="2017-05-16T21:33:47.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26898" PostTypeId="1" AcceptedAnswerId="27434" CreationDate="2017-05-17T04:44:26.953" Score="0" ViewCount="2665" Body="&lt;p&gt;I am trying to solve a 1D transient heat conduction problem using the finite volume method (FVM), with a fully implicit scheme, in polar coordinates. The governing equation is written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \frac{\partial \rho C_p T}{\partial t} = \frac{1}{r}\frac{\partial}{\partial r}\left(kr\frac{\partial T}{\partial r}\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, the problem is heat conduction in an infinitely long cylinder. The cylinder is made of two solid concentric cylinders (they have the same thermal properties). The body is heated by convection. The schematic heat flow diagram is shown in the figure below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ob3FV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ob3FV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the standard notation of P, as in the figure below, could you please help me to discretize the equation near the interface and at the boundary?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hek9O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hek9O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please kindly confirm that the discretization for the first control volume (CV) and the middle CVs is correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;1st CV&lt;/em&gt;: $(\rho C_p \frac{\Delta r r_P}{\Delta t} + \frac{k_w r_w}{\delta r_w})T_P = \frac{k_w r_w}{\delta r_w}T_W + \rho C_p \frac{\Delta r r_P}{\Delta t} T_P^o$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Middle CVs&lt;/em&gt;: $(\rho C_p \frac{\Delta r r_P}{\Delta t} + \frac{k_e r_e}{\delta r_e} + \frac{k_w r_w}{\delta r_w})T_P = \frac{k_e r_e}{\delta r_e}T_E + \frac{k_w r_w}{\delta r_w}T_W + \rho C_p \frac{\Delta r r_P}{\Delta t} T_P^o$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $r_P$ is the distance from the center of the cylinder to the node P, $r_e$ is the distance from the center of the cylinder to the $e$ face of the CF, $r_w$ is the distance from the center of the cylinder to the $w$ face of the CF, $\Delta r$ is the width of the CV, $\delta r_e$ is the distance from the node P to the node E, and $\delta r_w$ is the distance from the node P to the node W.&lt;/p&gt;&#xA;" OwnerUserId="10123" LastEditorUserId="9667" LastEditDate="2017-05-17T15:34:14.103" LastActivityDate="2017-07-18T08:42:59.110" Title="1D heat conduction using FVM in polar coordinates" Tags="|finite-volume|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26899" PostTypeId="2" ParentId="26893" CreationDate="2017-05-17T05:14:59.217" Score="0" Body="&lt;p&gt;You can draw an ellipsoid in ANSYS DM by following the below steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Select a plane and draw an ellipse. I selected the XYPlane.;&lt;/li&gt;&#xA;&lt;li&gt;Using 'trim' (available in 'Sketching'&gt;'Modify') remove half of the ellipse, as shown in the figure below. Note that 'trim' tool is shown on the left side of the figure.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/q3j1z.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q3j1z.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Thereafter, change the plane. In my case, my ellipse was drawn on the YXplane, and I created a new plane on the XZplane. Draw a circle (figure below).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yXRjf.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yXRjf.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Finally, in the Modelling mode, select 'Sweep'. In 'Sweep', specify your half of the ellipse as 'Profile' and your circle as 'Path'. Done!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="10123" LastActivityDate="2017-05-17T05:14:59.217" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26900" PostTypeId="1" CreationDate="2017-05-17T08:23:48.990" Score="2" ViewCount="92" Body="&lt;p&gt;I want to simulate a system of random walkers (called A) with diffusion coefficient equal to D and other systems of random walkers (called type B) with diffusion coefficient equal to 1000 D. Second type of walkers are smaller than the first ones and are eaten by the first type. As diffusion coefficient of B walkers are much bigger than the first type, I don't know how to choose the time step and space step of the simulation. Could anyone help? &#xA;The small walkers perform a discrete random walk on a lattice and the big ones are described by a Wiener process. Also, big walkers move to the direction of the gradient of small walkers and they have a velocity which pushes them in a preferred direction of their body&lt;/p&gt;&#xA;" OwnerUserId="24280" LastEditorUserId="20688" LastEditDate="2019-06-22T22:50:06.313" LastActivityDate="2019-06-22T22:50:06.313" Title="Multiscale Simulation of random walker" Tags="|simulation|randomized-algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26901" PostTypeId="1" AcceptedAnswerId="26906" CreationDate="2017-05-17T09:29:37.950" Score="2" ViewCount="68" Body="&lt;h3&gt;Background&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I have a huge dataset consisting of points (on a plane) together with a timestamp for each point. This is a collection of car GPS measures, giving us the location (latitude/longitude) of some car and the time at which that car was there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to store the time-series’ information on a grid.&#xA;That is, I have a rectangular grid, and for each cell, I would like to have a list of the dates at which there was a GPS signal in the cell.&#xA;So, in the end I would like the data to be a 2 by 2 matrix of lists. Each element of the matrix representing a cell of the grid and the attached list would contain the dates at which there was a signal in that cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be the ideal data representation for the time-series analysis I will have to perform.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Problem&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The dataset is huge. So I would like to be more memory-efficient. The main point to consider is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Since I’m using a regular rectangular grid and GPS signal is confined to cells containing roads, most of the cells will be “unactivated”, i.e most of the elements of the matrix will store an empty list (no signal).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Attempt&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The above remark suggests that the spatial information could be stored into a list, instead of a matrix.&lt;br&gt;&#xA;We would iterate over the dataset, for each measure look if the corresponding cell already exists in the list, and if not append a new cell to the list of activated cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach has two drawbacks compared to the matrix-based one:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Computational overhead&lt;/strong&gt;: For each datapoint, we have to search a whole list to determine if the cell it belongs to is already in the list. In the matrix approach, we can easily determine the index of the corresponding element, given the coordinates of the point and the resolution of the grid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Loss of spatial information&lt;/strong&gt;: In the matrix approach, it is simple to change the resolution of the grid, or perform nearest neighbor based operations. The list aproach completely dumps all spatial information.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In both cases, the matrix approach has $\mathcal{O}(1)$ complexity, whereas the list one has $\mathcal{O}(n)$ complexity.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I would thus like to be able to find a representation which doesn’t need to store empty cells, while at the same time retaining the advantages of the matrix representation. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;P.S.&lt;/strong&gt;: I know &lt;em&gt;kdTrees&lt;/em&gt; would be good for nearest-neighbor search, I'm just a little concerned about the time needed to build it. Comments?&lt;/p&gt;&#xA;" OwnerUserId="24281" LastEditorUserId="7574" LastEditDate="2017-05-17T16:18:07.897" LastActivityDate="2017-05-19T06:27:30.167" Title="Efficient Representation of (spatially sparse) spatial time series" Tags="|complexity|data-sets|grid|data-structures|nearest-neighbors|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26902" PostTypeId="1" CreationDate="2017-05-17T09:31:02.023" Score="2" ViewCount="309" Body="&lt;p&gt;I would like some classical molecular mechanics forcefield library or super-simple program which does not have any external dependences, so that I can plug it into my software without worry of &quot;dependency hell&quot;. Also I would like to modify it, so I would prefer something with simple and compact source code. It should do just one thing - in summary it should follow &lt;a href=&quot;https://en.wikipedia.org/wiki/Unix_philosophy&quot; rel=&quot;nofollow noreferrer&quot;&gt;unix philosophy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example there is &lt;a href=&quot;https://github.com/rdkit/rdkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;rdkit&lt;/a&gt; but it seems to be too ambitius (lot of functionality I don't need, lot of source code and dependencies to care about). Also many molecular editors like &lt;a href=&quot;https://avogadro.cc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Avogadro&lt;/a&gt; and &lt;a href=&quot;http://open-babel.readthedocs.io/en/latest/Forcefields/Overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenBable&lt;/a&gt; have some molecular mechanics implemented ... but it would be much more convenient to have clean simple library than try to extract this functionality out of big software package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have much requirements on precision of force-field, but I would prefer something which is as general as possible (e.g. UFF) for all elements and exotic bonding topologies although organic chemistry is main target, not something specialized to e.g. proteins like GROMOS, AMBER, CHARM etc.&lt;/p&gt;&#xA;" OwnerUserId="4696" LastActivityDate="2017-05-17T09:31:02.023" Title="minimalistic library for classical molecular mechanics force-field in C/C++/Python" Tags="|python|c|libraries|molecular-mechanics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26903" PostTypeId="2" ParentId="83" CreationDate="2017-05-17T10:18:14.180" Score="3" Body="&lt;p&gt;Is &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.optimize.basinhopping.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;basin hopping via scipy&lt;/a&gt; sufficient for your needs? If it returns a local min and not a global min, you can change the number of iterations and/or apply bounds. &lt;/p&gt;&#xA;" OwnerDisplayName="user23442" LastActivityDate="2017-05-17T10:18:14.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26904" PostTypeId="1" CreationDate="2017-05-17T10:40:42.847" Score="2" ViewCount="54" Body="&lt;p&gt;I need to to find a quick and dirty way to estimate the absolute error introduced by a series of agebraic operations of IEEE single precision floating point numbers, a pessimistic result is ok.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I am doing it, is to evaluate the biggest possible value that can be obtained during the chain of operations, find the power of two immediately greater than that number, and multiply such power of two by the number of algebraic operations performed and by the value of the least significant bit in the significand assuming exponent 1 (which is 0.00000011920928955078125)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it look like a sensible approach, or am I missing something?&lt;/p&gt;&#xA;" OwnerUserId="24283" LastActivityDate="2017-05-17T10:40:42.847" Title="Quick evaluation of floating point Absolute Error" Tags="|error-estimation|floating-point|accuracy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26905" PostTypeId="1" AcceptedAnswerId="26922" CreationDate="2017-05-17T13:05:22.803" Score="2" ViewCount="95" Body="&lt;p&gt;Let $C(u)$ be a B-spline curve depending on the parameter $u\in[0,1]$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C(u) = \sum_{i=0}^n c_i \, B_i(u)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $n+1$ coefficients $\{c_i\}$ and B-spline basis functions $\{B_i(u)\}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edited based on origimbo's comment:&lt;/strong&gt; We define the knots $\{\bar u_i\}$ to be fixed (and equidistantly spaced).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I would like to know if a B-spline curve is uniquely defined by its coefficients.&#xA;To rephrase the question: Can two non-identical sets of coefficients result in the exact same curve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, if I lower this restriction, how likely is it that two very different sets of coefficients $ \{c'_i\}\neq\{c_i\}$ yield a curve $C'(u) \approx C(u) $.&#xA;Does anyone know some literature about this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My motivation is something unusual: I want to minimize a cost function that depends on the curve and use the coefficients $c_i$ as the optimization parameters. Is this a suitable approach? I know that for large numbers of coefficients $n+1$, the B-spline curve representation gets very flexible and I assume lots of local minima in which I can run into. Therefore, it is definitely not a convex problem.&lt;/p&gt;&#xA;" OwnerUserId="23195" LastEditorUserId="23195" LastEditDate="2017-05-17T21:34:21.577" LastActivityDate="2017-05-18T23:11:04.610" Title="Is a B-spline curve uniquely defined by one set of coefficients?" Tags="|optimization|b-spline|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26906" PostTypeId="2" ParentId="26901" CreationDate="2017-05-17T14:43:50.520" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The dataset is huge. So I would like to be more memory-efficient. The main point to consider is the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[…] most of the cells will be “unactivated”, i.e most of the elements of the matrix will store an empty list (no signal).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This sounds as if an array of lists (or variably sized arrays) is just fine. In this case, the crucial factor for memory storage is not the size of your dataset but the size of the grid: Not all your lists need to have the same length. In fact, all you have to store for an empty grid point is a null pointer.&lt;/p&gt;&#xA;&lt;p&gt;Of course, if your grid is so large that you cannot even store the respective number of null pointers, you do indeed have a problem.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-05-17T14:43:50.520" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26907" PostTypeId="2" ParentId="26849" CreationDate="2017-05-17T14:53:23.893" Score="3" Body="&lt;p&gt;the size is 1k, it is not large-scale. I think many sparse direct solvers can be used. First of all, you need to make sure that your matrices are symmetric (or not)? , then you can choose suitable packages for handling them, such as PARDISO, MUMPS, UMFPACK, SuperLU and so on. If you also want to try the iterative solvers with preconditioning, you can try to use ILUPACK, ARMS, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, can you share your matrices with us? Recently, we collect some test matrices arsing from real applications for studying the numerical algorithms. We plan to establish a database of test matrices.&lt;/p&gt;&#xA;" OwnerUserId="10290" LastActivityDate="2017-05-17T14:53:23.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26908" PostTypeId="2" ParentId="26900" CreationDate="2017-05-17T19:05:56.330" Score="1" Body="&lt;p&gt;In general, the only harm a small step size causes is that it costs runtime. Therefore the step size imposed by your small random walkers is a reasonable choice. The only reason why it wouldn’t be is that you need an even smaller step size, but I can only see that happening if:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The velocity of your big walkers is not much smaller than the velocity of your small walkers.&lt;/li&gt;&#xA;&lt;li&gt;The concentration of your small walkers is inhomogeneous on scales of the movement of a big walker during one step (which should not happen unless it is imposed as an initial condition).&lt;/li&gt;&#xA;&lt;li&gt;The eating rate of your big walkers is so large that they eat a relevant portion of small walkers during one time step.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, with other words, you should ensure that no considerable changes happen during one time step.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2017-05-17T19:05:56.330" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="26909" PostTypeId="2" ParentId="26884" CreationDate="2017-05-17T20:18:28.563" Score="1" Body="&lt;p&gt;As a first approach, you could do as @BillGreene suggests, that is increase the  size of your model, increasing the number of cells each time. I guess that around 10 cells in each direction you should have a value that won't change much for further increasing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is to constrain the displacements at opposite sides of the cell, for example, in the horizontal direction, you would have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(\text{left}, y) = u(\text{right}, y) + g(y)\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a common choice is to have $g(y) = u_0$, constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you combine both approaches, you would have better results, I guess.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2017-05-17T20:18:28.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26910" PostTypeId="1" AcceptedAnswerId="26911" CreationDate="2017-05-17T20:50:05.637" Score="1" ViewCount="470" Body="&lt;p&gt;&lt;strong&gt;Context&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Portable, Extensible Toolkit for Scientific Computing (PETSc), the user often creates matrices and vectors. These objects are then used as input for other routines like iterative solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc offers a &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;routine&lt;/a&gt; for preallocating matrix memory by providing the number of nonzero entries in the diagonal submatrix and the off diagonal submatrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB has some useful articles about why preallocation helps -- for example, it eliminates the need to reallocate memory every time an element is added to a vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does memory preallocation in PETSc help so much? What exactly is happening when I tell PETSc the number of nonzeros in my matrix, and how does it compare to when I do not preallocate? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a partial answer in &lt;a href=&quot;http://www.training.prace-ri.eu/uploads/tx_pracetmo/petscIntroHNDSCi15.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this powerpoint&lt;/a&gt; about how the preallocation is related to the AIJ format. It says that PETSc sparse matrices are dynamic objects and can have nonzeros added to them on the fly. But this &quot;on the fly&quot; approach can lead to extra copying and reallocating, etc. I suppose this is the answer I wanted, but more details would be great.&lt;/p&gt;&#xA;" OwnerUserId="22717" LastEditorUserId="22717" LastEditDate="2017-05-17T20:58:32.817" LastActivityDate="2017-05-18T00:10:05.080" Title="Why does PETSc matrix memory allocation improve performance so much?" Tags="|petsc|efficiency|memory-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26911" PostTypeId="2" ParentId="26910" CreationDate="2017-05-17T23:08:49.983" Score="2" Body="&lt;p&gt;Warning: this answer is only going to give a brief overview, for the real details, the one source that won't be wrong is the &lt;a href=&quot;https://bitbucket.org/petsc/petsc&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The core matrix AIJ format is basically the same as the one known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_.28CSR.2C_CRS_or_Yale_format.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;compressed sparse row&lt;/a&gt; (CSR) or Yale format. This stores a sparse matrix as a list (normally concretely implemented as an array), $A$, of the nonzero entries, ordered by a key like (row index)$\times$(number of columns)+(column index), along with a list, $I$, of the index in $A$ at which which each row starts, and another list, $J$, containing the column indices of each of the entries in $J$. This format makes it quick to pull out the individual rows of $A$, and is good for right multiplication by column vectors. On the other hand, this makes adding a random new entry potentially expensive, since it means inserting an entry into the middle of a list, which potentially means reallocating a new block of memory and copying the existing values on either side of the one you're inserting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the code, it looks like when used without preallocation PETSc inserts some padding space into its guesses for the size of $A$ and $J$, and stores an additional list, $Ilen$, containing the number of genuine entries in each row. However, if a number of nonzero entries in a row exceeds the guess, you still need to reallocate and copy.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="19263" LastEditDate="2017-05-18T00:10:05.080" LastActivityDate="2017-05-18T00:10:05.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26912" PostTypeId="1" CreationDate="2017-05-18T03:04:02.960" Score="0" ViewCount="97" Body="&lt;p&gt;can any one show the steps how I convert the summation to norm as below:&#xA;$$ \sum_{j=1}^{M'} \big(\lambda_j (\mathbf{L}\mathbf{x})^2_j\big) = \|\mathbf{L}_\lambda \mathbf{x}\|^2, \qquad\text{where } \mathbf{L}_\lambda = \mathbf{\Lambda}^{1/2}\mathbf{L} \text{ and } \mathbf{\Lambda} = \mathrm{diag}(\lambda). $$&#xA;where $L$ is a matrix, and $x$ is a vector.&lt;/p&gt;&#xA;" OwnerUserId="20025" LastEditorUserId="713" LastEditDate="2017-05-18T18:40:56.247" LastActivityDate="2017-05-18T18:40:56.247" Title="simple matrix multiplication?" Tags="|linear-algebra|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26913" PostTypeId="2" ParentId="26898" CreationDate="2017-05-18T05:36:49.930" Score="0" Body="&lt;p&gt;I found the problem in my discretization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I integrated the PDE over a control volume and time step, I obtained the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho C_p (T_P -T_P^o) \Delta V = \left(k_e A_e \frac{T_E - T_P}{\delta r_e} - k_w A_w \frac{T_P - T_W}{\delta r_w} \right)\Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My mistake was to equate $\Delta V$ and $A$ to $\Delta r$ and $1$,respectively; as that was 1D in the Cartesian coordinates. However, in 1D of the cylindrical coordinates, $\Delta V = \pi (r_e^2 -r_w^2)$ and $A = 2 \pi r$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, the general solution in 1D cylinrical coordinates can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho C_P (T_P - T_P^o)(r_e^2 - r_w^2) = \left(2 k_e r_e \frac{T_E - T_P}{\delta r_e} - 2 k_e r_e \frac{T_E - T_P}{\delta r_e} \right) \Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;em&gt;1st CV&lt;/em&gt;: $\left(\rho C_P \frac{r_e^2}{\Delta t} + \frac{2 k_e r_e}{\delta r_e} \right) T_P = \frac{2 k_e r_e}{\delta r_e} T_E + \rho C_P \frac{r_e^2}{\Delta t}T_P^o$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;em&gt;middle CVs&lt;/em&gt;: $\left(\rho C_P \frac{r_e^2 - r_w^2}{\Delta t} + \frac{2 k_e r_e}{\delta r_e} + \frac{2 k_w r_w}{\delta r_w} \right) T_P = \frac{2 k_e r_e}{\delta r_e} T_E + \frac{2 k_w r_w}{\delta r_w} T_W + \rho C_P \frac{r_e^2 - r_w^2}{\Delta t}T_P^o$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;em&gt;last CV&lt;/em&gt; at the boundary: $\left(\rho C_P \frac{r^2 - r_w^2}{\Delta t} + \frac{2 k_w r_w}{\delta r_w} - ( -2 rh) \right) T_P = \frac{2 k_w r_w}{\delta r_w} T_W + \rho C_P \frac{r^2 - r_w^2}{\Delta t}T_P^o + 2rhT_a$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To discretize the equation at an interface use a harmonic mean, thouroughly described by Patankar (1980).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Patankar, Suhas V.&lt;/em&gt;, Numerical heat transfer and fluid flow, Series in Computational Methods in Mechanics and Thermal Sciences. Washington - New York - London: Hemisphere Publishing Corporation; New York etc.: McGraw-Hill Book Company. XIII, 197 p. \$ 37.50 (1980). &lt;a href=&quot;https://zbmath.org/?q=an:0521.76003&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL0521.76003&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10123" LastActivityDate="2017-05-18T05:36:49.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26916" PostTypeId="2" ParentId="26912" CreationDate="2017-05-18T16:59:23.477" Score="1" Body="&lt;p&gt;Define $u_j = (L\boldsymbol{x})_j$. Then we can do the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\sum_{j=1}^{n} \lambda_j u_j^2 &amp;amp;= \sum_{j=1}^{n} (\lambda_j^{1/2} u_j)^2 \\&#xA;&amp;amp;= (\Lambda^{1/2}\boldsymbol{u})^{T}(\Lambda^{1/2}\boldsymbol{u}) \\&#xA;&amp;amp;= \left\lVert \Lambda^{1/2}\boldsymbol{u} \right\rVert^2 \\&#xA;&amp;amp;= \left\lVert \Lambda^{1/2}L\boldsymbol{x} \right\rVert^2 \\&#xA;&amp;amp;= \left\lVert L_{\lambda}\boldsymbol{x} \right\rVert^2&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, as you put it, $L_{\lambda} = \Lambda^{1/2}L$.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-05-18T16:59:23.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26917" PostTypeId="2" ParentId="26884" CreationDate="2017-05-18T18:41:15.560" Score="3" Body="&lt;p&gt;Here is a description of a small FE model that might approximate the&#xA;case of an infinite number of holes in an infinite plate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a model of a single repeating element with $1/4$ of a&#xA;hole centered at $x=0, y=0$. Along the boundaries at $x=0$ and $y=0$&#xA;apply classical symmetry boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the two remaining edges, apply constraints on the displacements&#xA;simply to keep the edges straight and parallel to either the x-axis&#xA;or y-axis, as appropriate. In other words, all nodes along the&#xA;boundary $x=l/2$ would have the same $u$ displacement and all nodes along&#xA;the boundary $y=l/2$ would have the same $v$ displacement.&#xA;This type of boundary condition&#xA;is often referred to as an MPC (multi-point constraint) (e.g.&#xA;NASTRAN, ABAQUS). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either or both of these edges would also have an applied stress&#xA;loading normal to the edge of $\sigma_1$ or $\sigma_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that the straight-edge constraints would enforce&#xA;compatibility with the adjoining cells but not restrict the&#xA;motion unnecessarily.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-05-21T18:45:48.290" LastActivityDate="2017-05-21T18:45:48.290" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26918" PostTypeId="1" AcceptedAnswerId="26921" CreationDate="2017-05-18T18:49:33.997" Score="0" ViewCount="75" Body="&lt;p&gt;(I resume my question with a simple example)&#xA;I want to create a struct in matlab/octave as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c(1).x   = [0  1];&#xA;c(2).x   = [0  1];&#xA;...&#xA;c(100).x = [0  1];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One way is to do an for cycle:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:100&#xA;   c(i).x = [0 1];&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is, how can avoid the cycle for? (The reason is that everybody knows that we need to avoid the for cycles on matlab).&lt;/p&gt;&#xA;" OwnerUserId="17630" LastActivityDate="2017-05-18T22:56:44.513" Title="Matlab struct vector" Tags="|matlab|octave|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26920" PostTypeId="1" AcceptedAnswerId="27654" CreationDate="2017-05-18T20:41:46.560" Score="1" ViewCount="145" Body="&lt;p&gt;I am currently writing my masters thesis and my topic also touches on Stochastic Tomography for volume reconstruction presented in this&#xA;&lt;a href=&quot;http://www.cs.ubc.ca/labs/imager/tr/2012/StochasticTomography/files/StochasticTomography-Gregson2012.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. Now i understand most of the process described, but i just don't understand one aspect of calculating the objective function. On page 4 under &quot;Evaluating the Objective Function&quot; the author mentions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is essentially a density estimation&#xA;  problem in 2D, where new samples need to be added on the&#xA;  fly. A suitable algorithm is kernel density estimation [Parzen 1962],&#xA;  also known as splatting&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How is the KDE used for calculating the new residual value here? Are the input values for the KDE calculation actually all sample emissitvities which were already incorporated into the residual image at the pixel position the sample gets projected to? Does someone know how it is used here? Sadly the author does not get any more specific about that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="24299" LastActivityDate="2017-08-17T21:22:23.350" Title="How is KDE used in stochastic tomography" Tags="|stochastic|probability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26921" PostTypeId="2" ParentId="26918" CreationDate="2017-05-18T22:56:44.513" Score="3" Body="&lt;p&gt;You can do something like this in Matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 1e4;&#xA;c = []; c.x = [0 1]; &#xA;c(1:N) = c;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-05-18T22:56:44.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26922" PostTypeId="2" ParentId="26905" CreationDate="2017-05-18T23:11:04.610" Score="3" Body="&lt;p&gt;No, it's not completely unique. Like @origimbo said, a straight line can have many different representations that describe the same curve. Less trivially, splines can be degree-elevated and have repeated knots inserted and produce an identical curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most unconstrained optimization methods are gradient-based. You were right that if you try to find a location along that spline that is a minimum, you will have potentially a few local minima and which one that it converges to is highly dependent on the initial point. However, it seems that you want to minimize the curve in some way. You might be able to do that if you define an acceptable objective function, like the area under the curve or something.&lt;/p&gt;&#xA;" OwnerUserId="21131" LastActivityDate="2017-05-18T23:11:04.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26923" PostTypeId="2" ParentId="26901" CreationDate="2017-05-19T04:35:11.307" Score="0" Body="&lt;p&gt;You should define what nearest neighbor operations you are looking to perform and give a ballpark estimate of how big the dataset is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of data structures, you could also consider using a hash table with a spatial hash function. Given you are representing things with a regular grid, you can easily create a hash function and develop an efficient $O(1)$ search structure while also representing things in a sparse fashion. Using this data structure, it should take one pass through your $n$ pieces of data and place them into the search structure in a way you can easily search them up based on their grid cell. Depending on what nearest neighbor operations you are doing, this hash table approach could still be sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to KD Trees, they will take some time to build but they usually are pretty quick (though obviously the size of your dataset does influence this). Assuming your dataset doesn't change, you could also store the sorted KD Tree into memory and just read it in directly. Also, assuming you build a roughly balanced KD Tree, the search for nearest neighbors can be quite efficient being on the order of $O(log(n))$.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2017-05-19T06:27:30.167" LastActivityDate="2017-05-19T06:27:30.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26926" PostTypeId="2" ParentId="23200" CreationDate="2017-05-19T07:18:05.590" Score="4" Body="&lt;p&gt;In short, orthogonalization of the Krylov vectors occurs with respect to the operator, but not with respect to the preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alright, so say we want to solve $Ax=b$ with preconditioner $B$.  the preconditioned-CG iteration is basically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\hat{v}_1=\tilde{v}_1 =&amp;amp; Bb\\&#xA;v_1 =&amp;amp; \tilde{v}_1 / c_1\\&#xA;\\&#xA;\hat{v}_i =&amp;amp; BAv_{i-1}\\&#xA;\tilde{v}_i=&amp;amp;\hat{v}_i-\sum\limits_{j=1}^{i-1} \langle Av_k,\hat{v}_i\rangle v_k\\&#xA;v_i =&amp;amp; \tilde{v}_i/c_i\\&#xA;c_i =&amp;amp; \sqrt{\langle \tilde{v}_i,\tilde{v}_i\rangle}&#xA;\end{align*}&#xA;In other words&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\hat{v}_i$ - New Krylov vector&lt;/li&gt;&#xA;&lt;li&gt;$\tilde{v}_i$ - Orthgonalized Krylov vector&lt;/li&gt;&#xA;&lt;li&gt;$v_i$ - Normalized Krylov vector&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, what's interesting from this is that we get a number of properties.  The two that really matter for you are&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$\textrm{span}\{v_i\}_{i=1}^m = \textrm{span}\{(BA)^{i-1}(Bb)\}_{i=1}^m$&lt;/li&gt;&#xA;&lt;li&gt;$V^TAV = I$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The first property follows from how $\hat{v}_i$ is generated.  We're subtracting things from it and normalizing, but we're basically just getting a bunch of $BAv_{i-1}$ where $v_1$ is a scaling of $Bb$.  The second property follows from $\tilde{v}_i$.  Here, we do &lt;em&gt;not&lt;/em&gt; use $B$ in the orthogonalization.  Orthogonalization depends purely on $A$ being symmetric positive definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, say we want to solve the system $Ax=b$.  If we choose $x$ in our Krylov space, we solve for $\beta$ in&#xA;$$&#xA;AV\beta = b&#xA;$$&#xA;However, this is probably overdetermined.  Hence, we solve the normal equations, so that&#xA;$$&#xA;V^TAV\beta = V^Tb&#xA;$$&#xA;From above, we know that $V^TAV=I$, so we really have that&#xA;$$&#xA;\beta = V^Tb&#xA;$$&#xA;or that&#xA;$$&#xA;x = VV^Tb.&#xA;$$&#xA;Note, this holds regardless of the preconditioner $B$.  In fact, we could take random vectors and $A$-orthogonalize them against each other and the same would hold.  Really, the only thing that $B$ affects is the Krylov space.  As long as $B$ helps cluster the eigenvalues of $A$, the algorithm performs well.  Though, if $B$ is nonsymmetric, we may have to look at something like the pseudospectra to figure that out.  I'm not entirely sure.  Mostly, cluster the spectra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we lose anything with a nonsymmetric $B$?  Probably.  Normally, in CG, we have properties like $\langle Br_i,r_j\rangle = 0$ for all $i\neq j$.  I'm pretty sure that no longer holds.  Also, we normally have $\langle BAv_i,Av_j\rangle=0$ for $|i-j|\geq 2$.  That may or may not hold.  Really, any result that has $B$ I'd be suspicious of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, I'm pretty sure it's possible to code CG in a way that'd break everything with a nonsymmetric $B$.  However, if we code it using the equations above, there's nothing that really prohibits it from working with a nonsymmetric $B$.  I do it all the time and it seems to work fine.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2017-05-19T07:18:05.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26929" PostTypeId="1" CreationDate="2017-05-21T01:55:23.593" Score="1" ViewCount="56" Body="&lt;p&gt;Suppose we know the SVD of matrix $\bf A$, and $\bf B$ is a slight perturbation of $A$ (e.g. $\|{\bf B}-{\bf A}\|_{\text F}$ is relatively small), then is there any method that can efficiently compute the SVD of $\bf B$? That is, can the knowledge of SVD of $\bf A$ be helpful for SVD of $\bf B$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched a little bit and found there are some papers on the bound of perturbation, e.g. &lt;a href=&quot;http://users.math.msu.edu/users/markiwen/Teaching/MTH995/Papers/SVD_Stewart.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perturbation Theory for the Singular Value&#xA;Decomposition&lt;/a&gt;, but I currently have no luck in finding a method to compute SVD of perturbation taking advantage of the SVD of the original matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or reference will be very much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="24316" LastEditorUserId="24316" LastEditDate="2017-05-21T02:05:03.943" LastActivityDate="2017-05-21T02:05:03.943" Title="Any method to efficiently compute SVD of a perturbation of matrix $\bf A$ if the SVD of $\bf A$ is already known?" Tags="|linear-algebra|matrix|" AnswerCount="0" CommentCount="2" ClosedDate="2017-05-23T10:31:35.273" ContentLicense="CC BY-SA 3.0" />
  <row Id="26932" PostTypeId="2" ParentId="26894" CreationDate="2017-05-21T11:16:28.990" Score="4" Body="&lt;p&gt;As @VorKir mentions, for nonconforming spaces one has to use &lt;em&gt;broken Sobolev norms&lt;/em&gt; that are defined elementwise (in your notation) as&#xA;$$\begin{aligned}&#xA;\|v\|_{k,p,\Omega} &amp;amp;= \left(\sum_{K} \|v\|_{k,p,K}^p\right)^{1/p},\quad 1\leq p&amp;lt;\infty,\\&#xA;\|v\|_{k,\infty,\Omega} &amp;amp;= \max_K \|v\|_{k,\infty,K}&#xA;\end{aligned}$$&#xA;for $v\in V_h$, where $\|v\|_{k,p,K}$ is the usual $W^{k,p}(K)$ norm of $v|_K$ (which by definition is a polynomial and hence arbitrarily often differentiable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are inverse estimates for broken norms, but you lose additional powers of $h$ by having a larger value of $p$ on the left-hand side. If $\Omega\subset \mathbb{R}^n$, there exists a $C$ independent of $h$ such that&#xA;$$&#xA;\|v\|_{1,\infty,\Omega} \leq C h^{-1-n/2} \|v\|_{0,2,\Omega} \quad\text{for all }v\in V_h.$$&#xA;(This is a special case of Theorem 4.5.11 in &#xA;&lt;em&gt;Brenner, Susanne C.; Scott, L.Ridgway&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1007/978-0-387-75934-0&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;The mathematical theory of finite element methods&lt;/strong&gt;&lt;/a&gt;, Texts in Applied Mathematics 15. New York, NY: Springer (ISBN 978-0-387-75933-3/hbk). xvii, 397~p. (2008). &lt;a href=&quot;https://zbmath.org/?q=an:1135.65042&quot; rel=&quot;noreferrer&quot;&gt;ZBL1135.65042&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2017-05-21T11:16:28.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26934" PostTypeId="2" ParentId="11626" CreationDate="2017-05-21T17:38:22.900" Score="2" Body="&lt;p&gt;Linear system solvers are generally limited by memory access, so in parallel systems communication becomes the bottleneck. Good scaling can be usually achieved only for very large systems (millions of unknowns), for smaller systems you might be better off with a single-node or single-GPU solver. The libraries with GPU support that I know are listed below, for the &quot;usual&quot; CPU solvers see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/104/libraries-for-solving-sparse-linear-systems&quot;&gt;Libraries for solving sparse linear systems&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the custom matrix-vector product, one of the main issues with these libraries is that each uses their own representation of sparse matrices. Most of the time they are based on a standard format like CSR, CSC or Ellpack, so an efficient conversion might be possible, but there is no chance for bitwise compatibility. The problem is that it is not sufficient to optimize the &lt;em&gt;operations&lt;/em&gt;, which could be generalized by user-defined operators, but also the memory access pattern for each operation has to be considered. If you can think of a solution that is both general and efficient, please let me know (you should probably write a paper first).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CUDA-only libraries:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.nvidia.com/cuda/cusolver/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuSOLVER&lt;/a&gt; provides several direct methods for both dense and sparse systems. It is closed-source, bundled with the CUDA toolkit.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://docs.nvidia.com/cuda/cusparse/index.html#cusparse-preconditioners-reference&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuSPARSE&lt;/a&gt; provides incomplete factorizations, which can be used as preconditioners for iterative methods. It is closed-source, bundled with the CUDA toolkit.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://cusplibrary.github.io/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUSP&lt;/a&gt; provides iterative methods and multiple preconditioners, including a smoothed-aggregation algebraic multigrid. It is free and open-source.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hybrid or CUDA-accelerated libraries:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SuiteSparse&lt;/a&gt; has a CUDA-accelerated Cholesky factorization. It is free and open-source.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SuperLU_DIST&lt;/a&gt; has a CUDA-accelerated LU factorization. It is free and open-source.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://viennacl.sourceforge.net/viennacl-about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ViennaCL&lt;/a&gt; is a library using CUDA, OpenCL and OpenMP for parallelization, but I don't know if one computation can use multiple backends in parallel. It provides multiple iterative solvers and preconditioners. It is free and open-source.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://developer.nvidia.com/amgx&quot; rel=&quot;nofollow noreferrer&quot;&gt;AmgX&lt;/a&gt; provides algebraic multigrid and preconditioned iterative methods. It uses CUDA, MPI and OpenMP for parallelization. It is available with a commercial and a free license, the latter is limited to registered developers and non-commercial use.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-05-21T17:38:22.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26935" PostTypeId="1" AcceptedAnswerId="26943" CreationDate="2017-05-21T18:21:21.183" Score="1" ViewCount="77" Body="&lt;p&gt;I am facing the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given is a line of length $L$ which I want to split into $N$ segments. The lengths of the first $(s_1)$ and last segment $(s_N)$ are given. &#xA;You can assume that the first and the last segment are shorter than the average length $s_{ave} = \frac{L}{N}$ but they are not required to be equal. There should be a smooth transition, no sudden jump. The segment length should therefore increase at the beginning and decrease again towards the end of the line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a simple algorithm that could do this for me.&#xA;I was trying to use (one or multiple) geometric series: $s_i = \alpha  s_{i-1}$. If you specify the growth factor $\alpha$, the length (partial sum) is given by $L = s_1 \frac{\alpha^N -1}{\alpha -1}$ and if you specify the first and last segment length, $\alpha$ must satisfy $\alpha = (\frac{s_N}{s_1})^{\frac{1}{N}}$. Thus the system is overspecified if you fix $N, L, s_1, s_N$. Even if you split the problem, say into an increasing section, one where the length is constant and another one wehere the length decreases. Eventually you have to specify everything in one section.&lt;/p&gt;&#xA;" OwnerUserId="23586" LastActivityDate="2017-05-22T15:00:28.060" Title="Distirbution of Points along a Line" Tags="|computational-geometry|geometry|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26936" PostTypeId="1" AcceptedAnswerId="26937" CreationDate="2017-05-21T22:13:37.753" Score="3" ViewCount="67" Body="&lt;p&gt;Suppose I have implemented an LP, where some constraint coefficients are implemented as the mean of some probability distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I would like to solve the same problem but with stochasticity reintroduced based on the probability distributions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this still be solved using LP techniques?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, what are the natural alternatives?&lt;/p&gt;&#xA;" OwnerUserId="24323" LastEditorUserId="2150" LastEditDate="2017-05-22T00:20:21.373" LastActivityDate="2017-05-22T00:20:21.373" Title="Linear programming with stochasticity?" Tags="|optimization|convex-optimization|constrained-optimization|linear-programming|stochastic|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26937" PostTypeId="2" ParentId="26936" CreationDate="2017-05-22T00:09:27.863" Score="3" Body="&lt;p&gt;There is a huge literature on &quot;stochastic programming&quot;, but you're probably interested in what is called &quot;chance constrained programming&quot;, in which the constraint coefficients are random variables, and you want to find a solution $x$ such that each constraint is individually satisfied with probability $\eta$ ($\eta$ might 0.95, 0.99, etc.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These problems cannot typically be reformulated as LP's, but they can sometimes be formulated and solved as other types of optimization problems.  For example, if the constraint coefficients of your LP have a multivariate normal distribution, then the chance constrained version of the LP can be formulated and solved as a second order cone program.  You can find a discussion of this in section 4.4 of &lt;a href=&quot;https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convex Optimization by Boyd and Vandenberghe.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also look up the broader topic of &quot;Robust Optimization&quot; for methods that work with various kinds of uncertain coefficients.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-05-22T00:09:27.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26938" PostTypeId="1" AcceptedAnswerId="26992" CreationDate="2017-05-22T05:10:23.533" Score="1" ViewCount="363" Body="&lt;p&gt;I was reading about ellipse-ellipse intersection and I came across this article: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/679622/intersection-between-conic-and-line-in-homogeneous-space/867428#867428&quot;&gt;https://math.stackexchange.com/questions/679622/intersection-between-conic-and-line-in-homogeneous-space/867428#867428&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a program that implements this method and it works, but now I'm working to understand why. I understand how $D = L^T C L$ is constructed, but I don't understand how/why the decomposition $P = D + λ L$ works. It seems that the author seems to be solving for one of the eigenvalues of $D$, but I don't immediately recognize if this is a standard method or not (i.e. if it only works for conics, or only when decomposing from rank 2 to rank 1 matrices, etc). Any reference material or help understanding the theory here is greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I realized I mentioned ellipse-ellipse intersections, and the article is about conic-line intersections. To erase some of the confusion, the method of determining ellipse-ellipse intersection basically comes down to constructing a pencil of conics and with that pencil finding a pair of lines that intersect the ellipses at the same points that the ellipses intersect each other. That &lt;a href=&quot;https://math.stackexchange.com/questions/425366/finding-intersection-of-an-ellipse-with-another-ellipse-when-both-are-rotated&quot;&gt;article can be found here&lt;/a&gt; for those interested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For clarity, here is some explanation of the conic-line problem and some of the equations from the line-conic article.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general Cartesian equation for a conic section $Ax^2+Bxy+Cy^2+Dx+Ey+F=0$ can be rewritten as $p^TQp = 0$, where:&#xA;$$p = \left [ \begin{matrix} x \\ y \\ 1 \\ \end{matrix} \right ],\,\, Q = \left [ \begin{matrix} A &amp;amp; B/2 &amp;amp; D/2 \\ B/2 &amp;amp; C &amp;amp; E/2 \\ D/2 &amp;amp; E/2 &amp;amp; F\end{matrix} \right ]$$&#xA;The point $p$ is in homogeneous coordinates. Here I use the letter Q to differentiate from C which is used as a coefficient in the conic equation (the article calls this matrix C). The implicit formula for a line $ax+by+c=0$ can be rewritten as expressed as $l\cdot p = 0$ where $l = [a,b,c]$ and $p=[x,y,1]^T$ as above. Additionally, the place where a line $l=(a_1,b_1,c_1)$ so defined intersects another line $g=(a_2,b_2,c_2)$ can be calculated using the cross product $l \times g = q$, where $q$ is expressed in homogeneous coordinates. The method for determining the intersection of the line and general conic is given in the article mentioned originally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the article, the matrix $L$ is the skew-symmetric matrix such that $Lg=l\times g$. If we define $R=L^TQL$ (the article calls this matrix D), then $g^TRg=0$ is the equation of all lines that intersect $l$ at points on the conic, since $(Lg)^TQ(Lg) = 0$ (but the article says the lines are tangent to the conic). The article also says that $R$ is a degenerate conic that consists of the 2 points where the line and conic intersect (which I also don't understand).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is the decomposition of the matrix $R$, which should have rank 2. We construct a matrix $P=R+\lambda L$ and find a $\lambda$ to make $P$ rank 1. Then, $P=uv^T$, where $u$ and $v$ are the intersection points in homogeneous coordinates. I just don't understand how the decomposition works, why it works, and if that's a general method I can use (range of applicability).&lt;/p&gt;&#xA;" OwnerUserId="21131" LastEditorUserId="21131" LastEditDate="2017-05-23T18:38:26.277" LastActivityDate="2017-05-27T06:01:19.263" Title="Matrix Decomposition of Conics" Tags="|linear-algebra|geometry|rank|matrix-factorization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26939" PostTypeId="1" CreationDate="2017-05-22T09:26:20.360" Score="2" ViewCount="501" Body="&lt;p&gt;My simulation creates a 2d grid of vectors and scalars (EDIT of velocity, depth etc), at 60 frames per second. Is it correct? It looks sort of right... but who knows?&#xA;How can I tell what's happening - especially for debugging?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thoughts so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Arrows for velocity field is difficult to see what's happening in small areas/fine detail. And if using length for magnitude, you have to get the scale right, or the arrows disappear (too small) or become lines (too big).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Or, can use colours for magnitude (standard rainbow key, to map colours to numbers). Still has the scaling problem.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;particles (dots), evenly placed and moved by the velocity field. Density of dots then implies divergence; and cirlcular motion implies vorticity.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Green/red for curl (vorticity), clockwise/counter-clockwise&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Grey-scale for divergence (compression)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Height field as a 3d isometric surface, with labelled axes and scales&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is it better to dump all the data to a file/s, and then visualize it later, with a separate tool? You could then move forward and backward in time, like a movie, and switch between quantities to visualize, and methods of visualizing. Maybe, in practice, just hacking the code to display different things is the best way to make progress?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll have to create the visualization code myself, and it seems a bit of a project in itself, so I'm hesitant. I'd like to get some guidance from those more experienced, as to what is actually most useful, before diving in.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;what quantities are most useful to visualize?    &lt;/li&gt;&#xA;&lt;li&gt;how to display them?    &lt;/li&gt;&#xA;&lt;li&gt;how (and when) to select which one to display (a UI issue)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; I should add, I'm developing on android (on the device itself, not on a PC), so many popular tools/apps aren't available. And, since my main project is to display real-time simulation, also visualizing some data isn't that different a task.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-06-01T06:38:45.590" LastActivityDate="2017-11-13T13:37:55.063" Title="In practice, what are the most useful ways to visualize 2d fluid flow, to tell what is happening in the simulation? Esp for verification and debugging" Tags="|fluid-dynamics|visualization|verification|" AnswerCount="2" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26940" PostTypeId="2" ParentId="26823" CreationDate="2017-05-22T10:04:34.227" Score="1" Body="&lt;p&gt;The core of the question is whether computing $\mathbf{M}$ via the assumption that $\mathbf{J}^H\mathbf{J} = -(\omega\mu_0)^2(\mathbf{E}^H\mathbf{E})$ makes sense or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mathematics do not support said identity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\mathbf{d}^{pred}$ is the array of predicted data, then there must exists a prediction operator $\mathbf{f}(\mathbf{m}) = \mathbf{Q}\mathbf{E}$, $\mathbf{Q}$ is an interpolation operator placing data from the grid points at the receivers locations. If we assume (for now) , that all receivers are located on grid points, then $\mathbf{Q} = \mathbf{I}$, and&#xA;$$\mathbf{J} = \frac{\partial \mathbf{f}(\mathbf{m})}{\partial \mathbf{m}} = \frac{\partial \mathbf{E}}{\partial \mathbf{m}}.$$&#xA;The solution to the discrete counterpart of the forward problem implies to solve a linear system $\mathbf{A}\mathbf{E} = \mathbf{S}$. Assuming a &lt;strong&gt;total field formulation&lt;/strong&gt; of the forward  problem, then:&#xA;$$\mathbf{J} = \frac{\partial\mathbf{E}}{\partial\mathbf{m}} = \mathbf{A}^{-1}\left(-\frac{\partial\mathbf{A}}{\partial\mathbf{m}}\mathbf{E}\right)=\mathbf{A}^{-1}\left(-(i\omega\mu_0)\mathbf{E}\right).$$&#xA;Therefore:&#xA;$$\mathbf{J}^H\mathbf{J} = \omega^2\mu_0^2(\mathbf{A}^{-1}\mathbf{E})^H(\mathbf{A}^{-1}\mathbf{E}),$$&#xA;where we can define $\mathbf{u}$ to be the solution to the system $\mathbf{A}\mathbf{u} = \mathbf{E}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solution differs greatly from the originally-proposed identity: $\mathbf{J}^H\mathbf{J} = -(\omega\mu_0)^2(\mathbf{E}^H\mathbf{E})$. Specifically, in the later, there is no need to invert $\mathbf{A}$.&lt;/p&gt;&#xA;" OwnerUserId="24203" LastActivityDate="2017-05-22T10:04:34.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26942" PostTypeId="1" CreationDate="2017-05-22T14:29:51.453" Score="1" ViewCount="4362" Body="&lt;p&gt;I am working on a trajectory analysis project using python and its data science related libraries. I am planning to implement Frechet Distance algorithm for trajectory analysis, each trajectory has sequence of (x,y) coordinates along with timestamp, speed, dist between consecutive points, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would kindly request if anyone who has experience with in this field and trajectory analysis, can provide with some useful links for Python implementation of Frechet Distance. My ultimate goal is to cluster the similar trajectories (location and direction wise).&lt;/p&gt;&#xA;" OwnerUserId="24236" LastActivityDate="2018-01-06T09:24:52.723" Title="Python implementation for Frechet Distance" Tags="|python|data-analysis|hpc|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26943" PostTypeId="2" ParentId="26935" CreationDate="2017-05-22T14:43:57.787" Score="2" Body="&lt;p&gt;You can uniformly distribute points and apply transformation, &#xA;$$&#xA;N_s(\zeta)=1-\zeta\\&#xA;N_e(\zeta)=\zeta\\&#xA;\phi(\zeta)=x_sN_s(\zeta)+x_eN_e(\zeta)+ \sum_i \alpha_i L_i(2\zeta-1) N_s(\zeta)N_e(\zeta)&#xA;$$&#xA;where $L_i$ is Legendre polynomial. So if you set for example $\alpha_0=0$ and $\alpha_1=1$, you will get your distribution, when you apply function $\phi(\zeta)$ to points uniformly distributed between $(0,1)$.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-05-22T15:00:28.060" LastActivityDate="2017-05-22T15:00:28.060" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26944" PostTypeId="1" CreationDate="2017-05-22T15:34:36.780" Score="0" ViewCount="68" Body="&lt;p&gt;A boolean-valued monotonic function is defined in the set of positive integers, $\mathcal Z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(n) = \begin{cases} 0, &amp;amp;n_{min}\le n &amp;lt; n\ast\\1, &amp;amp;n\ast\le n\le n_{max} \end{cases} ; n \in \mathcal Z $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to search for $n\ast$. The bounds $n_{min}$ and $n_{max}$ are known apriori.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A salient property of $n\ast$ is that it has a higher probability of lying in a region around $\hat{n}$ (known apriori).&lt;/strong&gt; The exact probability distribution, although unknown, (probably doesn't matter) can be considered as continuous CDF having a higher weighting factor around $\hat{n}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already implemented a simple binary search, but I am looking for more efficient solutions that somehow account for the given probability information (i.e. $n\ast$ is concentrated around $\hat{n}$), I am looking for an efficient algorithm to find $n\ast$.  without losing accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that Binary search is worst-case $\mathcal{O}(log\ n)$, but $f(n)$ is so computationally expensive that I can't afford even this.&lt;/p&gt;&#xA;" OwnerUserId="16900" LastEditorUserId="16900" LastEditDate="2017-05-22T16:44:54.633" LastActivityDate="2017-05-22T23:13:41.710" Title="Efficient search strategy in a monotonic boolean function wherein the probability of solution location is known apriori" Tags="|optimization|algorithms|computational-geometry|search|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26946" PostTypeId="1" CreationDate="2017-05-22T20:04:29.863" Score="2" ViewCount="111" Body="&lt;p&gt;I have a system of equations that cannot be solved for in closed form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F_1(x_1,x_2,\beta)=0 ~\&amp;amp;~ F_2(x_1,x_2,\beta)=0 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve for functions $x_1=x_1(\beta) ~\&amp;amp;~ x_2=x_2(\beta)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach so far:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Fix $\beta=\beta_1$ &lt;/li&gt;&#xA;&lt;li&gt;Solve system numerically for $x^1_1,x^1_2$.&lt;/li&gt;&#xA;&lt;li&gt;Repeat for a different $\beta$ on some pre-defined finite support&lt;/li&gt;&#xA;&lt;li&gt;Generate a vector of $\mathbf{x_1}=(x^1_1...x^n_1)$ with corresponding  $\mathbf{b}=(\beta_1...\beta_n)$&lt;/li&gt;&#xA;&lt;li&gt;Approximate the relationship between $x_1$ and $\beta$ using Sieve to get $\hat{x}_1(\beta)$ [repeat for $x_2$]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The problem with this method is that it gets computationally out of hand once I have many equations and parameters due to step 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do know that the system has a solution for any $\beta$, which motivated this method. I also can easily get the Jacobian of $F$ analytically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a preferable method that is not as computationally burdensome?&lt;/p&gt;&#xA;" OwnerUserId="24330" LastActivityDate="2017-05-22T20:04:29.863" Title="Backing out a function of parameters from system of nonlinear equations" Tags="|nonlinear-equations|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="26948" PostTypeId="2" ParentId="26939" CreationDate="2017-05-22T21:46:38.507" Score="1" Body="&lt;p&gt;Creation of own visualisation code, it is hard to justify, unless you will have some strong arguments why existing codes are not good enough. It is a couple of great open codes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do visualisation on the fly when you do analysis, &lt;a href=&quot;http://www.paraview.org/in-situ/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.paraview.org/in-situ/&lt;/a&gt; great tool, or as result of post-processing, see Paraview, openDX, visit a large number of possibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paraviev catalyst is particularly interesting; you can link your app. to it and do all in parallel when your run the analysis.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-05-23T08:53:11.967" LastActivityDate="2017-05-23T08:53:11.967" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26949" PostTypeId="2" ParentId="26944" CreationDate="2017-05-22T23:13:41.710" Score="1" Body="&lt;p&gt;Since you have no derivatives for your function, you cannot beat bisection search in some asymptotic sense. But you can try to find good starting points for the bisection search. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, find $x_0$ so that $F(x_0)=\frac 12$ -- i.e., the most likely point for the switch point. Then, if $f(x_0)=0$, use $x_0$ as the left end point of your starting interval for the bisection search; otherwise, use it as the right starting point. In any case, $x_0$ is likely to be close to the final solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the other starting point of your interval, let's assume for a moment that you had $f(x_0)=1$ -- i.e., that we are looking for a left end point $a_0$ of the interval, with $b_0=x_0$ being the right one. Try, for example, $\alpha$ as the point that is one standard deviation away from $b_0$, i.e., choose $\alpha$ so that $F(\alpha)=(1-0.95)=0.05$. If indeed $f(\alpha)=0$, then set $a_0=\alpha$ and you know that the switch point is in the interval $[a_0,b_0]$. If on the other hand, $f(\alpha)=1$, then the switch point is in fact to the left of $\alpha$, and you would set $b_0=\alpha$ and try again with $\alpha$ so that $F(\alpha)=(1-0.95)^2=0.05^2$. Then repeat until you finally have a starting interval where $f(a_0)\neq f(b_0)$. With this, you would then start your bisection search.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-05-22T23:13:41.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26950" PostTypeId="2" ParentId="26942" CreationDate="2017-05-23T01:57:38.953" Score="1" Body="&lt;p&gt;There is a discussion in Tyler Reddy's very recent pycon2017 tutorial on youtube &lt;a href=&quot;https://youtu.be/ETJc3NfU9aA?t=9540&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/ETJc3NfU9aA?t=9540&lt;/a&gt;&#xA;(at 2:39.00) where he discusses implementing the Frechet Distance in scipy.spatial. It is not done yet but he runs an implementation that he has wrapped in Python.&lt;/p&gt;&#xA;" OwnerUserId="24332" LastActivityDate="2017-05-23T01:57:38.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26951" PostTypeId="2" ParentId="8913" CreationDate="2017-05-23T02:27:31.497" Score="1" Body="&lt;p&gt;I went through the same thing last year after my first fluid course. I liked the weirdness of fluid mechanics so asked a graduate student I knew what book they were using in their CFD course, which he said was Versteeg. Being overly confident, I thought I could do it you know actually understand it. The problem was that I lacked a bit of rigor or rather I did not have enough knowledge in either physics or the math to actually enjoy CFD to its fullest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I did the 12 steps CFD with Lorena Barba while studying Mechanics of fluid by Irving Shames. I tried reading intro to fluid dynamics by Batchelor after hearing about all the praise, but yet again I was discouraged by my lack of mathematical knowledge. So I stuck it out with Shames instead of Batchelor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now after being acquainted with tensor notations and following all the derivations from the texts mentioned above I started to read and do the problems out of Computational Fluid dynamics:The basics with Application by Anderson. It is great, it goes over the physical meanings of the equation very rigorously and leaves very few to your guessing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem I had with Versteeg was the fact that it lacked problem set for practice anyway, now after completing all the coding exercises in Anderson's book, I have returned to Batchelor's book. This time I actually have some idea on what is being said in that book. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I am about to start studying numerical computation of internal and external flows by Hirsch. What I like is the rigor and plenty of problems for practice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can look up diff.eq , algebra or any other math ideas you may encounter online as you proceed forward with your study. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for learning how to use numpy and scipy and python in general I just followed this thank me later ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://folk.ntnu.no/leifh/teaching/tkt4140/._main000.html#table_of_contents&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://folk.ntnu.no/leifh/teaching/tkt4140/._main000.html#table_of_contents&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="23212" LastActivityDate="2017-05-23T02:27:31.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26952" PostTypeId="1" CreationDate="2017-05-23T04:18:50.500" Score="1" ViewCount="73" Body="&lt;p&gt;I've read that in modeling structures problems, the finite element method (FEM) is typically used. I am unfamiliar with FEM, but I am wondering, in particular, if using FEM, as opposed to finite difference/volume, alleviates some of the numerical stability issues that may arise due to the large structural constants? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently trying to solve the transient linear elastic equations (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_elasticity#Direct_tensor_form&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Linear_elasticity#Direct_tensor_form&lt;/a&gt;) using FVM. For ease of demonstration, we can consider the 1-D form (just the classical second order hyperbolic wave equation) of this equation in the absence of shear and assuming constant density:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial^2 u}{\partial t^2} = \frac{2\mu + \lambda}{\rho} * \frac{\partial^2 u}{\partial x^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u$ is the x-displacement, $\rho$ is the mass density and $\mu$ and $\lambda$ are Lame constants (see &lt;a href=&quot;http://scienceworld.wolfram.com/physics/LameConstants.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://scienceworld.wolfram.com/physics/LameConstants.html&lt;/a&gt;). I solved this by turning the second derivative equation into two first order equations:&#xA;Let $$\frac{\partial^2 u}{\partial t^2} = \frac{dz}{dt}$$&#xA;where&#xA;$$z = \frac{du}{dt}$$&#xA;or in vector form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d\boldsymbol{q}}{t} = \boldsymbol{g}$$&#xA;where $\boldsymbol{q} = [u \ \ \ \ \  q]$ and $g = [z \ \  \ \ \ \frac{2\mu+\lambda}{\rho}* \frac{\partial^2u}{\partial x^2}=f $]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In preliminary modeling, I used Explicit Euler and set my constants = 1. The code &quot;appeared&quot; to be stable. Then I set the constants to be representative of steel and some other materials of interest, where the Lame constants fall between O(10^8)-O(10^11). This severely limited my the timestep size that I could use for my code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to this, I decided to try to implement an implicit scheme. To do this, I tried to formulate my problem using the Jacobian matrix as such &#xA;$$&#xA;\frac{d\boldsymbol{q}}{dt} = A\boldsymbol{q}&#xA;$$&#xA;where A is Jacobian matrix&#xA;$$&#xA;A = \frac{d\boldsymbol{g}}{d\boldsymbol{q}} = \begin{bmatrix}&#xA;0 &amp;amp; I \\&#xA;\frac{df}{du} &amp;amp; 0&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I found the eigenvalues of this Jacobian matrix and found eigenvalues with a POSITIVE REAL root, indicating that this problem can't be stable (regardless of the large order of magnitude of the structural constants) at least with most conventional time integration schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Earlier, when I stated that my explicit Euler scheme &quot;appeared&quot; to be stabled. I realized that it was mildly unstable. I had used very small structural constants such that the eigenvalues hugged the imaginary axis and the real roots magnitudes are smaller, but still positive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going back to my first question: Does FEM alleviate some of these mentioned issues? Second question: Does anyone have insight into solving such a problem where the jacobian matrix's eigenvalues have positive real roots? Third question: Any recommended schemes for tackling linear elasticity with FV that won't be severely limited by stability with EXPLICIT schemes? &lt;/p&gt;&#xA;" OwnerUserId="23915" LastEditorUserId="23915" LastEditDate="2017-05-23T04:23:57.833" LastActivityDate="2017-05-23T04:23:57.833" Title="FEM/FVM/FD for structural modeling and stability issues due to large structural constants?" Tags="|finite-element|finite-volume|time-integration|linear-system|jacobian|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26953" PostTypeId="1" AcceptedAnswerId="26954" CreationDate="2017-05-23T05:43:39.080" Score="0" ViewCount="266" Body="&lt;p&gt;I'm currently using ParMETIS and it is required to determine the halo region of the local elements in a parallel unstructured mesh. Assume that the mesh is large and cannot be stored on a single processor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My approach is to gather the global list of elements and their processor location in each processor. Then the neighbors of the local elements which do not belong to the current processor are identified and are then communicated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is redundant communication being performed as some of the non-local elements are clearly not neighbors to any of the local elements and are hence not going to be in the halo region, but their location is requested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most efficient procedure to communicate halo information in unstructured mesh calculations? (assuming communication paradigm is MPI and calls from ParMETIS are available for obtaining dual graphs, global adjacency and so on)&lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="15120" LastEditDate="2017-05-23T05:57:44.173" LastActivityDate="2018-08-11T13:07:02.380" Title="Halo Region Communication in Unstructured Mesh Problems" Tags="|finite-element|parallel-computing|finite-volume|mpi|unstructured-mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26954" PostTypeId="2" ParentId="26953" CreationDate="2017-05-23T07:10:09.677" Score="1" Body="&lt;p&gt;The best to see what others do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need so-called ghost nodes and ghost cells. The best is to look how ghost vectors are implemented in PETSc; you can find in the documentation how it works there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, parallel vector keeps a local copy of unknowns on each processor, also, also has extra ghost data at the end, which are used for communication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to start which nodes you own, which nodes are shared and which of them are ghosted. Typically which are in overlapped region nodes are owned by a lower processor,  but you can have more strategies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PETSc manages algebra (and mesh see DMPLEX) if you like to see how this is managed on mesh sied, see MOAB,&#xA;&lt;a href=&quot;http://sigma.mcs.anl.gov/moab-library/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://sigma.mcs.anl.gov/moab-library/&lt;/a&gt;&#xA;look at the documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programming this you would like to use a number of communication events if you do communication try to do all at once, MoAB has class CrystalRouter for this, see &#xA;&lt;a href=&quot;http://ftp.mcs.anl.gov/pub/fathom/moab-docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.mcs.anl.gov/pub/fathom/moab-docs/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ma not sure if this is what you are looking for.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastActivityDate="2017-05-23T07:10:09.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26955" PostTypeId="1" CreationDate="2017-05-23T07:17:40.020" Score="1" ViewCount="582" Body="&lt;p&gt;How can one simulate diffusion of a particle which has a natural axis and diffusion coefficient of the particle in the direction of the axis is D_1 and in perpendicular to the axis is D_2?&#xA;Could anyone provide a code?&lt;/p&gt;&#xA;" OwnerUserId="24280" LastActivityDate="2018-05-19T09:12:32.673" Title="2D simulation of a particle with different diffusion coefficient in different directions of the particle" Tags="|matlab|simulation|diffusion|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26956" PostTypeId="1" CreationDate="2017-05-23T07:40:00.873" Score="1" ViewCount="30" Body="&lt;p&gt;I am simulating the propagation of a light pulse using the equation&#xA;$$\frac{\partial}{\partial z}A=\frac{1}{2\cdot k_0}\nabla^2_rA$$&#xA;with&#xA;$$k_0=\frac{2\pi}{\lambda_0}$$&#xA;The propagation with a step size of $dz$ is done using the equation&#xA;$$\left(\vec{1}-i\cdot\frac{dz}{2}\nabla^2_r\right)A_{n+1}=\left(\vec{1}+i\cdot\frac{dz}{2}\nabla^2_r\right)A_n$$&#xA;with &#xA;$$A_0(\vec{r})=\exp\left(-\frac{r^2}{2}\right)$$&#xA;Now I add a lens with a focal length of $f$, which is done using the equation&#xA;$$A_{0,L}=A_0\exp\left(-i\frac{\left\vert\vec{r}\right\vert^2}{2f}\right)$$&#xA;My problem is now: I discretize $A$ along a vector $\vec{r}$ from $0$ to $r_{max}$ with $r_{num}$ elements (radial approach). That works for $F&amp;gt;F_{crit}$. If $F&amp;lt;F_{crit}$, the oscillations of the lens factor are not properly resolved anymore, resulting in artifacts in the simulation. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/3D2oB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3D2oB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;For my current input values I need at least more than $50000$ points, but even that is not enough due to a small value for $F$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using that amount of points increases the amount of computation time and (when displaying the moving pulse) the amount of storage. Is there any way to decrease the amount of points, while still being able to resolve the lens oscillations? I am already using sparse matrices for the calculation, but for storage I have to use a dense matrix.&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-05-23T07:40:00.873" Title="Simulation of a lens, insufficient points" Tags="|numerical-analysis|finite-difference|python|wave-propagation|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26957" PostTypeId="1" AcceptedAnswerId="26958" CreationDate="2017-05-23T08:54:54.677" Score="1" ViewCount="636" Body="&lt;p&gt;I am trying to implement a fem code on tet10 elements.  I follow the lecture notes for tet10 implementation given in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch10.d/AFEM.Ch10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch10.d/AFEM.Ch10.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have validated my stiffness matrix with the example provided. The  numerical values of the Jacobian at the Gauss points add up to the volume of the element when integrated against the unit function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when I take a different set of nodes with exactly the same ordering of the nodes,  I get a negative determinant.  I am now not sure of how to decide my code is correct or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am providing the matlab code snippet for the same both with the original data given in the paper as well as the new data. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc;&#xA;if 0&#xA;node = [19.39935000986642   0.1580775331976262  9.374695831233881&#xA;        19.85814434000072   0.6598506371289292  0.0841144105427571&#xA;        19.82724989196882   0.4316427461960032  9.89092611758984&#xA;        0.1049331859852762  0.5417518219571752  9.70657837113129&#xA;        19.62874717493357   0.4089640851632777  4.729405120888319&#xA;        19.84269711598477   0.5457466916624663  4.987520264066299&#xA;        19.61329995091762   0.2948601396968147  9.632810974411861&#xA;        9.752141597925849   0.3499146775774007  9.540637101182586&#xA;        9.981538762992999   0.6008012295430523  4.895346390837023&#xA;        9.966091538977048   0.4866972840765892  9.798752244360564&#xA;];% I get wrong results with same ordering uncomment to run the same&#xA;end;&#xA;node = [2 3 4&#xA;        6 3 2 &#xA;        2 5 1&#xA;        4 3 6&#xA;        4 3 3&#xA;        4 4 1.5&#xA;        2 4 2.5&#xA;        3 3 5&#xA;        5 3 4&#xA;        3 4 3.5]; % I get right results as in the reference&#xA;Y=480; %Youngs Modulus&#xA;nu=1/3; % Poisson ratio&#xA;alfa=(5.0+3.0*sqrt(5.0))/20; %Gauss Points&#xA;beta =(5.0-sqrt(5.0))/20;    %Gauss Points&#xA;weight=0.25; %weights for integration points&#xA;GaussPoints=[alfa beta beta beta;beta alfa beta beta;beta beta alfa  beta;beta beta beta alfa];&#xA;%Elasticity Matrix&#xA;E=[1-nu nu nu 0 0 0;nu 1-nu nu 0 0 0;nu nu 1-nu 0 0 0;0 0 0 0.5-nu 0 0;0 0 0 0 0.5-nu 0;0 0 0 0 0 0.5-nu];&#xA;E=Y/((1+nu)*(1-2*nu))*E;&#xA;%Stiffness Matrix&#xA;K=zeros(30,30);&#xA;&#xA;for j=1:4&#xA;xi=GaussPoints(:,j);&#xA;jx1=4.0*(node(1,1)*(xi(1)-0.25)+node(5,1)*xi(2)+node(7,1)*xi(3)+node(8,1)*xi(4));&#xA;jy1=4.0*(node(1,2)*(xi(1)-0.25)+node(5,2)*xi(2)+node(7,2)*xi(3)+node(8,2)*xi(4));&#xA;jz1=4.0*(node(1,3)*(xi(1)-0.25)+node(5,3)*xi(2)+node(7,3)*xi(3)+node(8,3)*xi(4));&#xA;&#xA;jx2=4.0*(node(5,1)*xi(1)+node(2,1)*(xi(2)-0.25)+node(6,1)*xi(3)+node(9,1)*xi(4));&#xA;jy2=4.0*(node(5,2)*xi(1)+node(2,2)*(xi(2)-0.25)+node(6,2)*xi(3)+node(9,2)*xi(4));&#xA;jz2=4.0*(node(5,3)*xi(1)+node(2,3)*(xi(2)-0.25)+node(6,3)*xi(3)+node(9,3)*xi(4));&#xA;&#xA;jx3=4.0*(node(7,1)*xi(1)+node(6,1)*xi(2)+node(3,1)*(xi(3)-0.25)+node(10,1)*xi(4));&#xA;jy3=4.0*(node(7,2)*xi(1)+node(6,2)*xi(2)+node(3,2)*(xi(3)-0.25)+node(10,2)*xi(4));&#xA;jz3=4.0*(node(7,3)*xi(1)+node(6,3)*xi(2)+node(3,3)*(xi(3)-0.25)+node(10,3)*xi(4));&#xA;&#xA;jx4=4.0*(node(8,1)*xi(1)+node(9,1)*xi(2)+node(10,1)*xi(3)+node(4,1)*(xi(4)-0.25));&#xA;jy4=4.0*(node(8,2)*xi(1)+node(9,2)*xi(2)+node(10,2)*xi(3)+node(4,2)*(xi(4)-0.25));&#xA;jz4=4.0*(node(8,3)*xi(1)+node(9,3)*xi(2)+node(10,3)*xi(3)+node(4,3)*(xi(4)-0.25));&#xA;&#xA;&#xA;J=[1 1 1 1;jx1 jx2 jx3 jx4;jy1 jy2 jy3 jy4;jz1 jz2 jz3 jz4];&#xA;&#xA;Jdet=det(J);&#xA;Jinv=inv(J);&#xA;Iaug=[0 0 0;1 0 0;0 1 0;0 0 1];&#xA;P=Jinv*Iaug;&#xA;&#xA;       a1=P(1,1);&#xA;       a2=P(2,1);&#xA;       a3=P(3,1);&#xA;       a4=P(4,1);&#xA;&#xA;       b1=P(1,2);&#xA;       b2=P(2,2);&#xA;       b3=P(3,2);&#xA;       b4=P(4,2);&#xA;&#xA;       c1=P(1,3);&#xA;       c2=P(2,3);&#xA;       c3=P(3,3);&#xA;       c4=P(4,3);&#xA;&#xA;Nfx=[(4.0*xi(1)-1)*a1 (4.0*xi(2)-1)*a2  (4.0*xi(3)-1)*a3 (4.0*xi(4)-1)*a4 ...&#xA;   4.0*(a1*xi(2)+a2*xi(1)) 4.0*(a2*xi(3)+a3*xi(2)) 4.0*(a1*xi(3)+a3*xi(1)) ...&#xA;   4.0*(a1*xi(4)+a4*xi(1)) 4.0*(a2*xi(4)+a4*xi(2)) 4.0*(a3*xi(4)+a4*xi(3))];&#xA;&#xA;Nfy=[(4.0*xi(1)-1)*b1 (4.0*xi(2)-1)*b2 (4.0*xi(3)-1)*b3 (4.0*xi(4)-1)*b4 ...&#xA;  4.0*(b1*xi(2)+b2*xi(1)) 4.0*(b2*xi(3)+b3*xi(2)) 4.0*(b1*xi(3)+b3*xi(1)) ...&#xA;  4.0*(b1*xi(4)+b4*xi(1)) 4.0*(b2*xi(4)+b4*xi(2)) 4.0*(b3*xi(4)+b4*xi(3))];&#xA;&#xA;Nfz=[(4.0*xi(1)-1)*c1 (4.0*xi(2)-1)*c2 (4.0*xi(3)-1)*c3 (4.0*xi(4)-1)*c4 ...&#xA;  4.0*(c1*xi(2)+c2*xi(1)) 4.0*(c2*xi(3)+c3*xi(2)) 4.0*(c1*xi(3)+c3*xi(1)) ...&#xA;  4.0*(c1*xi(4)+c4*xi(1)) 4.0*(c2*xi(4)+c4*xi(2)) 4.0*(c3*xi(4)+c4*xi(3))];&#xA;&#xA;B=[Nfx(1) 0 0 Nfx(2) 0 0 Nfx(3) 0 0 Nfx(4) 0 0 Nfx(5) 0 0 Nfx(6) 0 0 Nfx(7) 0 0 Nfx(8) 0 0 Nfx(9) 0 0 Nfx(10) 0 0;&#xA;  0 Nfy(1) 0 0 Nfy(2) 0 0 Nfy(3) 0 0 Nfy(4) 0 0 Nfy(5) 0 0 Nfy(6) 0 0 Nfy(7) 0 0 Nfy(8) 0 0 Nfy(9) 0 0 Nfy(10) 0;&#xA;  0 0 Nfz(1) 0 0 Nfz(2) 0 0 Nfz(3) 0 0 Nfz(4) 0 0 Nfz(5) 0 0 Nfz(6) 0 0 Nfz(7) 0 0 Nfz(8) 0 0 Nfz(9) 0 0 Nfz(10);&#xA;  Nfy(1) Nfx(1) 0 Nfy(2) Nfx(2) 0 Nfy(3) Nfx(3) 0 Nfy(4) Nfx(4) 0 Nfy(5) Nfx(5) 0 Nfy(6) Nfx(6) 0 Nfy(7) Nfx(7) 0 Nfy(8) Nfx(8) 0 Nfy(9) Nfx(9) 0 Nfy(10) Nfx(10) 0;&#xA;  0 Nfz(1) Nfy(1) 0 Nfz(2) Nfy(2) 0 Nfz(3) Nfy(3) 0 Nfz(4) Nfy(4) 0 Nfz(5) Nfy(5) 0 Nfz(6) Nfy(6) 0 Nfz(7) Nfy(7) 0 Nfz(8) Nfy(8) 0 Nfz(9) Nfy(9) 0 Nfz(10) Nfy(10);&#xA;  Nfz(1) 0 Nfx(1) Nfz(2) 0 Nfx(2) Nfz(3) 0 Nfx(3) Nfz(4) 0 Nfx(4) Nfz(5) 0 Nfx(5) Nfz(6) 0 Nfx(6) Nfz(7) 0 Nfx(7) Nfz(8) 0 Nfx(8) Nfz(9) 0 Nfx(9) Nfz(10) 0 Nfx(10)];  &#xA;&#xA;K=K+weight*(B'*E*B*Jdet/6.0);&#xA;Jdet&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The node ordering is as follows 1 2 3 4 (1+2) (2+3) (1+3) (1+4) (2+4) (3+4) where 1 2 3 4 denote the corner nodes and terms in the bracket (a+b) denote the node which forms the midpoint of the line node a and node b. The ordering of the nodes in the both the set of data is exactly the same &lt;/p&gt;&#xA;" OwnerUserId="23149" LastEditorUserId="23149" LastEditDate="2017-05-23T09:15:25.663" LastActivityDate="2017-05-23T13:37:38.260" Title="FEM on tet10 element: negetive determinant at the Gauss point" Tags="|finite-element|jacobian|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26958" PostTypeId="2" ParentId="26957" CreationDate="2017-05-23T09:35:42.433" Score="1" Body="&lt;p&gt;For 4 node tetrahedron if you take any face and calculate normal of it has to point in the direction of opposite to this face node then a determination of the Jacobian is positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of 10 node tetrahedron is more complicated, for example, if you move mid nodes on the edges, to quarter point distance, your Jacobian become singular (this is used in fracture mechanics at the crack tip to approximate singularity). If you can beyond quarter point and then  determinant of  Jacobian will be negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, not only ordering of the nodes is important but also the position of mid nodes. You have to check this.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-05-23T13:37:38.260" LastActivityDate="2017-05-23T13:37:38.260" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26960" PostTypeId="1" AcceptedAnswerId="26970" CreationDate="2017-05-23T10:53:02.620" Score="8" ViewCount="7039" Body="&lt;p&gt;I am interested in the specific differences of the following methods:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The &lt;strong&gt;conjugate gradient method (CGM)&lt;/strong&gt; is an algorithm for the numerical solution of particular systems of &lt;strong&gt;linear&lt;/strong&gt; equations.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;nonlinear conjugate gradient method (NLCGM)&lt;/strong&gt; generalizes the conjugate gradient method to nonlinear optimization.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;gradient descent/steepest descent algorithm (GDA)&lt;/strong&gt; is a first-order iterative optimization algorithm.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;stochastic gradient descent (SGD)&lt;/strong&gt; is a stochastic approximation of the gradient descent optimization method for minimizing an objective function that is written as a sum of differentiable functions.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Why do some publications introduce the CGM as a method for the solution of systems of equations, when it would be much more intuitive to introduce it as a linear optimization algorithm, for which one can define the residual of the system as the objective function, to then portrait the solution of a linear system, reformulated as a quadratic minimization problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the GDA can also be used to solve linear system of equations but it is rarely used, since the conjugate gradient method is one of the most popular alternatives. However, gradient descent can also be used to solve a system of nonlinear equations. But, besides conjugacy, convergence and benefits from preconditioning, are there any other differences worth mentioning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To add some context to this question: I am trying to better understand the different first order optimization algorithms, in order to make a more efficient use of them in inversion problems/deep learning problems.&lt;/p&gt;&#xA;" OwnerUserId="24203" LastEditorUserId="24203" LastEditDate="2017-05-24T09:21:53.453" LastActivityDate="2017-05-25T08:20:51.413" Title="What are the differences between the different gradient-based numerical optimization methods?" Tags="|optimization|conjugate-gradient|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26962" PostTypeId="2" ParentId="26955" CreationDate="2017-05-23T20:03:25.257" Score="0" Body="&lt;p&gt;The problem here amounts to apply the random force in the body reference frame of the particle. There, you can set the friction and/or diffusion coefficients separately. Once the forces are computed, you can rotate them back to the laboratory reference frame. A book such as &quot;Dynamics of colloids&quot; will provide you with all the necessary background. &lt;a href=&quot;https://www.amazon.fr/Introduction-Dynamics-Colloids-J-K-G-Dhont-ebook/dp/B00DTSEAY2/ref=sr_1_1?ie=UTF8&amp;amp;qid=1495569759&amp;amp;sr=8-1&amp;amp;keywords=dynamics+of+colloids&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.amazon.fr/Introduction-Dynamics-Colloids-J-K-G-Dhont-ebook/dp/B00DTSEAY2/ref=sr_1_1?ie=UTF8&amp;amp;qid=1495569759&amp;amp;sr=8-1&amp;amp;keywords=dynamics+of+colloids&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16340" LastActivityDate="2017-05-23T20:03:25.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26963" PostTypeId="1" CreationDate="2017-05-23T22:09:43.443" Score="1" ViewCount="404" Body="&lt;p&gt;I would like to implement P2 (or even P3...) finite elements in Matlab. I need to be able to control the construction of the rigidity and mass matrices ($ \int \nabla \varphi_i\cdot \nabla \varphi_j$ and $\int \varphi_i \varphi_j$) (since I want to tackle some precision computation problems afterwards, and maybe go beyond machine precision using Advanpix...)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If there are already some good Matlab codes which do this, please point me to them. If not, are there any good references from where I could learn to do this properly? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would like to avoid doing all the computations by hand if possible since I'm sure someone else has done them before :) ... (find the right basis functions, compute explicitly all integrals of products of neighbor basis functions, etc...)&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="20688" LastEditDate="2019-06-22T20:45:05.470" LastActivityDate="2019-06-22T20:45:05.470" Title="Assemble P2 finite elements - Matlab or references" Tags="|finite-element|matlab|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26964" PostTypeId="1" CreationDate="2017-05-24T02:26:08.247" Score="1" ViewCount="130" Body="&lt;p&gt;I am working on a trajectory analysis project using python and its data science related libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am planning to implement Frechet Distance algorithm for trajectory analysis, each trajectory has sequence of (x,y) coordinates along with timestamp, speed, dist between consecutive points, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is the example df with &quot;voygID&quot; and respective x, y coordinates. (I am not including timestamp, speed, etc columns here, as I assume it wont be required in implementing Frechet distance)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     voygID         x             y&#xA; 0   1             -7.935513     5.103579&#xA; 1   1             -7.935781     5.103300&#xA; 2   1             -7.936354     5.102726&#xA; 3   1             -7.935915     5.102802&#xA; 4   2             -7.935306     5.103424&#xA; 5   2             -7.945678     5.119876&#xA; 6   2             -7.954764     5.128738&#xA; 7   3             -7.954888     5.138898&#xA; 8   3             -7.955897     5.149808&#xA; 9   3             -7.965789     5.156789&#xA; 10  3             -7.983457     5.198610&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the above data frame (x,y) forms the coordinate data and all the points belonging to the same &quot;voygID&quot; forms one separate trajectory; Therefore in the above df rows(0-3) belonging to voygID = 1 is one trajectory, while points belonging to voygID = 2 is another trajectory and so on. There are thousands of trajectories each with 100s of points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to implement Frechet Distance to compare each trajectory with another and group the similar curves/ trajectories together and form different clusters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would kindly request if anyone who has experience in this field, can provide with some useful links for Python implementation of Frechet Distance to do the above task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am open to other similarity measure methods like &quot;Dynamic Time Warping&quot; or &quot;Longest common subsequence&quot;. Please suggest me if there are preprogrammed python implementations are available.&lt;/p&gt;&#xA;" OwnerUserId="24236" LastEditorUserId="24236" LastEditDate="2017-05-24T17:51:29.897" LastActivityDate="2017-05-24T17:51:29.897" Title="Implementing Frechet Distance on pandas data frame" Tags="|python|data-analysis|hpc|distance-measure|" AnswerCount="0" CommentCount="1" ClosedDate="2017-05-27T02:42:39.450" ContentLicense="CC BY-SA 3.0" />
  <row Id="26965" PostTypeId="1" CreationDate="2017-05-24T04:22:34.853" Score="0" ViewCount="271" Body="&lt;p&gt;I have two variations of an iterative algorithm. All the steps of both algorithms are equivalent except one. In this step: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithm 1 needs to compute the matrix $ABA^T$ for matrices $A \in \mathbb{R}^{p \times p}$ and $B \in \mathbb{R}^{p \times p}$; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithm 2 needs to invert the $p \times p$ matrix $C$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: assuming that these steps led to the exact same iterates of the algorithm (i.e., they do not change the convergence properties), would there be any benefit to using Algorithm 1? If so, what? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell, Algorithm 1 requires only matrix multiples which can be easily parallelized. However, it's not clear to me that Algorithm 1 would be more memory efficient since one must compute and store $AB$ to then multiply it by $A^T$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Assume that in general, these matrices are non-sparse. &lt;/p&gt;&#xA;" OwnerUserId="24342" LastEditorUserId="4405" LastEditDate="2017-05-24T12:37:03.797" LastActivityDate="2017-05-24T12:39:55.627" Title="Benefits of matrix multiply over inversion" Tags="|matrix|efficiency|linear-system|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26966" PostTypeId="2" ParentId="26965" CreationDate="2017-05-24T05:39:29.743" Score="2" Body="&lt;p&gt;It really depends on a lot of factors. Not knowing the problem, my concerns are these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The magnitude of $p$ will dictate a lot about how fast things run. I would think 2 multiplies would be faster than an inversion, although for sufficiently large matrices the inversion and multiply algorithms can be tuned to be about the same.&lt;/li&gt;&#xA;&lt;li&gt;If $C$ becomes near singular, inversion will be very imprecise or impossible.&lt;/li&gt;&#xA;&lt;li&gt;If $C$ is orthogonal, computing the inverse would be much simpler.&lt;/li&gt;&#xA;&lt;li&gt;Tradeoffs between memory efficiency and speed depend on your application.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All in all, I would choose multiplication.&lt;/p&gt;&#xA;" OwnerUserId="21131" LastActivityDate="2017-05-24T05:39:29.743" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="26967" PostTypeId="1" AcceptedAnswerId="26968" CreationDate="2017-05-24T08:33:14.233" Score="5" ViewCount="636" Body="&lt;p&gt;I have a 3-dimensional domain D, with 3 types of BC which I am trying to solve the Poisson equation on. $$-\nabla \cdot (\sigma(x,y,z)\nabla u)=0$$&lt;br&gt;&#xA;Insulator (Neumann BC)&lt;br&gt;&#xA;Electrode set at some potential (Dirichlet BC).&lt;br&gt;&#xA;Additionally I have some electrodes $E_l$ which are set to have a fixed current $I_l$, so on these the boundary condition is $\int_{E_l} \sigma(x,y,z)(du/dn) dA=I_l$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This boundary condition can given a mesh be translated into equations of the form $\sum a_iu_i=0$, where $u_i$ is the potential at node i and $a_i$ are known mesh dependend constants. But I dont know how to add this equation to the matrix to be solved, or what equations to replace it with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot assume that the current density is constant on each electrode, for instance some electrodes have net current 0 but current flowing in on one side and out the other, I know commercial software solves this problem just as efficiently as with regular BC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I go about incoprorating this third type of BC into FEM?&#xA;Any description or references is what im looking for.&lt;/p&gt;&#xA;" OwnerUserId="23558" LastActivityDate="2018-01-15T14:33:12.653" Title="Solving Poisson equation with current BC using FEM" Tags="|finite-element|linear-solver|boundary-conditions|poisson|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26968" PostTypeId="2" ParentId="26967" CreationDate="2017-05-24T09:45:18.433" Score="8" Body="&lt;p&gt;It is standard procedure for general case like that. You can enforce condition for electrode by Lagrange multipliers, &#xA;$$&#xA;L(u,\lambda) = \int_\Omega \sigma u_{,i} u_{,i} \, \textrm{d}V - \int_{\partial\Omega_\sigma} u n_i j_i \textrm{d}S + \lambda\left(\int_E \sigma u_{,n}\, \textrm{d}S - I \right)&#xA;$$&#xA;that for discretised problem is&#xA;$$&#xA;\left[&#xA;\begin{array}{cc}&#xA;A &amp;amp; C^T\\&#xA;C &amp;amp; 0 &#xA;\end{array}&#xA;\right]&#xA;\left\{&#xA;\begin{array}{c}&#xA;u \\ \lambda&#xA;\end{array}&#xA;\right\}&#xA;=\left[&#xA;\begin{array}{c}&#xA;f \\&#xA;I&#xA;\end{array}&#xA;\right]&#xA;$$&#xA;where &#xA;$$&#xA;C_i = &#xA;\int_E \sigma n_j \phi_{i,j} \textrm{d}S&#xA;$$&#xA;and $\phi_i$ is shape functions at the node. The technical difficulty here is to solve for the gradient of the shape functions on the surface, but that's solvable with good finite element library. You have to enforce Dirichlet bc a priori in standard way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note if you have more electrodes, that comes with more Lagrange multipliers. You can efficiently solve the problem using Schur complement and block solvers like &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCFIELDSPLIT.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCFIELDSPLIT.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way is to use projection matrices, avoiding explicitly Lagrange multipliers, once you have constrains matrix $Cu=g$, you can solve problem&#xA;$$&#xA;C^TC + Q^TKQ = C^Tg + Q^T(f-KRg)&#xA;$$&#xA;where &#xA;$$&#xA;Q = I-P^TKP&#xA;$$,&#xA;$$&#xA;P = C^T(CC^T)^{-1}C&#xA;$$&#xA;$$&#xA;R = (CC^T)^{-1}C&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find details of this in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mark Ainsworth. Essential boundary conditions and multi-point constraints in finite element analysis. Computer Methods in Applied Mechanics and Engineering, 190(48):6323–6339, 2001.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been using this couple of times, you can implement this such that you do not have explicitly $Q^TKQ$, you can see implementation &lt;a href=&quot;http://mofem.eng.gla.ac.uk/mofem/html/struct_constrain_matrix_ctx.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="19791" LastEditDate="2018-01-15T14:33:12.653" LastActivityDate="2018-01-15T14:33:12.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26970" PostTypeId="2" ParentId="26960" CreationDate="2017-05-24T10:09:08.913" Score="15" Body="&lt;p&gt;First, you should forget about solving linear equations for the moment -- that's a different context from optimization, and trying to consider both on equal footing will only lead to confusion. Why some people want to motivate an optimization algorithm by mentioning a linear solver is a question only they can answer, but I would just ignore that and focus on the method itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we'll assume that you're trying to minimize some function $F:\mathbb{R}^n\to \mathbb{R}$, which we'll assume to be differentiable but not necessarily convex. Most methods (including first-order methods) compute a sequence $\{x^k\}_{k\in\mathbb{N}}$ hopefully converging to a minimizer (but in general only to a stationary point) by choosing $x^0$ and then&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;picking a search direction $d^k\in\mathbb{R}^n$,&lt;/li&gt;&#xA;&lt;li&gt;picking a step size $\tau_k\in\mathbb{R}$,&lt;/li&gt;&#xA;&lt;li&gt;setting $x^{k+1} = x^k + \tau_k d^k$,&lt;/li&gt;&#xA;&lt;li&gt;and repeating until bored.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In first-order methods, only the function values $F(x^j)$ and gradients $\nabla F(x^j)$ for $j\leq k$ are used in the choice of $d^k$ (and $\tau_k$, which I'll not address here). This includes the methods you  list:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Steepest descent&lt;/strong&gt; consists in the choice $d^k = -\nabla F(x^k)$. It's slow -- the asymptotic complexity is $F(x^k)-\min_x F(x) = \mathcal{O}(1/k)$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;(Nonlinear) conjugate gradients&lt;/strong&gt; choose $d^k$ as suitable linear combination of $\nabla F(x^k)$ and $\nabla F(x^{k-1})$, where the linear combination is determined by the specific choice of $\beta^k$ -- there exist several different variants in the literature (Fletcher--Reeves, Hestenes--Stiefel, Polak--Ribière, Hager--Zhang,...) with slightly different properties; see &#xA;&lt;em&gt;Hager, William W.; Zhang, Hongchao&lt;/em&gt;, A survey of nonlinear conjugate gradient methods, Pac. J. Optim. 2, No. 1, 35-58 (2006). &lt;a href=&quot;https://zbmath.org/?q=an:1117.90048&quot; rel=&quot;noreferrer&quot;&gt;ZBL1117.90048&lt;/a&gt;. &#xA;(The $\alpha^k$ in NCG plays the role of the $\tau_k$.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the special case that $F$ is quadratic, i.e., $F(x) = \frac12x^TAx + b^Tx + c$, it is possible to derive an optimal choice of $\beta^k$, which leads to the &lt;strong&gt;(classical) conjugate gradient method&lt;/strong&gt;. Even more, one can show that this leads to the optimal method among all first-order methods (for quadratic functions), with complexity $F(x^k)-\min_x F(x) = \mathcal{O}(1/k^2)$ (see, e.g., Nesterov's book). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Of course, if $A$ is symmetric, minimizing $F$ is equivalent to solving $Ax=b$, so you can use CG as a linear solver as well. The same is not true for NCG, since, e.g., $F(x) = \|f(x)\|^2$ for $f$ nonlinear can have local minimizers that don't correspond to roots of $f$. Root finding and minimization are in general two completely different problems.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there's indeed little reason to choose steepest descent over NCG, since the additional work is negligible (a few inner products and AXPYs) and you potentially get much better performance.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Stochastic gradient descent&lt;/strong&gt; for $F(x) = \sum_{i=1}^n F_i(x_i)$ consists in the choice $d^k = -\nabla F_i(x_i^k)$ for some randomly chosen $i$. Since $\nabla F(x^k) = \sum_{i=1}^k \nabla F_i(x_i^k)$, under some conditions on the step size and the random choice, after enough iterations you can consider $n$ steps as a single &quot;averaged&quot; steepest descent iteration. So you'd expect about the same convergence speed asymptotically as steepest descent, but the single iterations are much cheaper, and you might get where you want faster (in wall clock time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also seen (nonlinear) conjugate gradient versions of it in the&#xA;literature, but I'm not sure how much you can prove about them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2017-05-25T08:20:51.413" LastActivityDate="2017-05-25T08:20:51.413" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="26971" PostTypeId="1" CreationDate="2017-05-24T10:12:09.943" Score="0" ViewCount="186" Body="&lt;p&gt;I don't know if anyone is using SGE anymore (what is the world of HPC using nowadays?), but I need to work on version 6.2u3, I believe, from sometime back in '11. I'm using it on a cluster of 64 nodes, 12 cores each.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thing is, I'm not sure what requesting X cores really means. Is it purely advisory? I've tried using both SMP and OpenMPI environments, and the impression I get is that spawning more threads within the application always increases performance, irrespective of how many cores you request. Even if you request a single core using &lt;code&gt;-pe smp 1&lt;/code&gt; or &lt;code&gt;-pe openmpi 1&lt;/code&gt;, you get an almost doubling in performance going from 1 to 2 to 4 threads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What then, is the purpose of scheduling cores?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question is related: I'm getting a doubling of performance in going from 1 to 2 to 4 threads. However, at about 8 or 10 threads (i.e. supposedly, 8 or 10 cores) the linear relationship breaks down, and in fact at 12 threads computation takes longer than that at 8 threads. Even using &lt;code&gt;-pe smp 12&lt;/code&gt;. I've tried this with a simple, embarrassingly parallel problem (generating an array of numbers), so there should be no contention. And I don't think it's thread overhead, because the workload is quite large, taking a few seconds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the nodes/cores scheduled, I'm told by the system that I have 12 cores on the same node (usually, but not always), yet still performance decreases at 12 threads. I suspect that SGE does not prevent third-party work from running on that node, even though I've requested enough cores to have 1 core per thread, and this is leading to thread interference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I on the right track or have I missed the point entirely?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="24349" LastEditorUserId="20688" LastEditDate="2018-03-23T18:24:11.950" LastActivityDate="2018-03-23T18:24:11.950" Title="Sun Grid Engine peculiarities" Tags="|parallel-computing|hpc|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26972" PostTypeId="2" ParentId="26965" CreationDate="2017-05-24T12:39:55.627" Score="2" Body="&lt;p&gt;There could be some benefits, but without knowing more details on the algorithm it is impossible to answer. For instance, the stability properties of the two algorithms could be completely different.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-05-24T12:39:55.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26973" PostTypeId="1" AcceptedAnswerId="26975" CreationDate="2017-05-24T23:05:37.697" Score="1" ViewCount="293" Body="&lt;p&gt;In order to calculate the volume of the solid defined by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{51}{100} (\cos x \cos y+\cos x \cos z+\cos y \cos z)+\cos x+\cos y+\cos z+1\le0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x,y,z\in[0,2\pi]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h67QT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h67QT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the following code (in MATLAB)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = @(x, y, z) double(1 + cos(x) + cos(y) + cos(z) + 0.51*(cos(x).*cos(y)  + cos(x).*cos(z) + cos(y).*cos(z))&amp;lt;=0)&#xA;q = 8*integral3(f, 0, pi, 0, pi, 0, pi)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but gives warning message and &lt;code&gt;NaN&lt;/code&gt; ：&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Warning: Reached the maximum number of function evaluations (10000). The result fails the global error test.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to obtain the desired result in as high as possible precision?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems &lt;a href=&quot;https://mathematica.stackexchange.com/questions/26401/determining-which-rule-nintegrate-selects-automatically&quot;&gt;Gauss-Kronrod Rule&lt;/a&gt; solves this problem correctly; a further variable substitution, $u_i=\cos x_i$ with Jacobian determinant involved may cause computation efficiency.&lt;/p&gt;&#xA;" OwnerUserId="6693" LastEditorUserId="6693" LastEditDate="2017-05-26T05:09:34.970" LastActivityDate="2017-05-26T05:09:34.970" Title="Solid volume calculation" Tags="|matlab|integration|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26974" PostTypeId="2" ParentId="4984" CreationDate="2017-05-25T03:12:01.787" Score="1" Body="&lt;p&gt;for your two variable case, $\beta_i$ should be same for both $\beta_i^x$ and $\beta_i^y$, it is computed as $\beta_i=\frac{\mathbf{g}_{i+1}^T\mathbf{g}_{i+1}}{\mathbf{g}_{i}^T\mathbf{g}_{i}}$ (where $\mathbf{g}_{i} = [\mathbf{g}_{i}^x; \mathbf{g}_{i}^y]$). The $\mathbf{g}$ is a 2x1 vector, thus $\mathbf{g}^T\mathbf{g}$ gives a single value. It may help to check Subrat Pathak's master thesis &quot;A COMPARATIVE STUDY OF NON LINEAR CONJUGATE GRADIENT METHODS&quot; where he numerically test the two variable nonlinear cg in several nonlinear functions. &lt;/p&gt;&#xA;" OwnerUserId="24335" LastActivityDate="2017-05-25T03:12:01.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26975" PostTypeId="2" ParentId="26973" CreationDate="2017-05-25T05:16:10.687" Score="4" Body="&lt;p&gt;I would do what @Tyler Olsen suggesting; you can mesh that. The right tool would be &lt;a href=&quot;http://doc.cgal.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CGAL&lt;/a&gt; &lt;a href=&quot;http://www.cgal.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cgal.org&lt;/a&gt;, linear elements is easiest to use, in fact, all is almost done, just follow example &lt;a href=&quot;http://doc.cgal.org/latest/Mesh_3/index.html#Chapter_3D_Mesh_Generation&quot; rel=&quot;nofollow noreferrer&quot;&gt;3D Mesh Generation&lt;/a&gt;, the section about  3D Domains Bounded by Implicit Isosurfaces. With mesh refinement you will converge to exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way is to generate surface mash only and apply Divergence theorem (Gauss's theorem - Ostrogradsky's theorem). &#xA;$$&#xA;V = \frac{1}{3}\int_S x\cdot n \textrm{d}S&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can as well generate surface patches, and implicitly calculate mapping for those surface and integrate them one by one using Gauss's theorem.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-05-25T20:22:48.787" LastActivityDate="2017-05-25T20:22:48.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26976" PostTypeId="1" CreationDate="2017-05-25T05:51:08.810" Score="1" ViewCount="366" Body="&lt;p&gt;I am writing a FEN solver on tet10 elements. I right now have a CFD grid gen tool (Pointwise) which generates tet4 meshes. From this I have the element connectivity list and the node list. How do I go about generating a tet10 mesh from this info.  Is there any particular tool/code to do this. &lt;/p&gt;&#xA;" OwnerUserId="23149" LastActivityDate="2017-05-25T05:51:08.810" Title="How to generate a tet10 mesh from a tet4 mesh" Tags="|finite-element|mesh-generation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26978" PostTypeId="1" AcceptedAnswerId="26982" CreationDate="2017-05-25T11:30:26.613" Score="0" ViewCount="73" Body="&lt;p&gt;In my code it often happens that I need to calculate values for a group.&#xA;For example, suppose there is the following data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;groups = [A, A, A, B, B, B]&#xA;values = [1, 2, 3, 0, 1, 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I want to calculate the cumsum by group:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cumsum = [1, 3, 6, 0, 1, 2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I always code this in the same way, as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from collections import defaultdict&#xA;import numpy as np&#xA;&#xA;N = 1000000&#xA;values = np.arange(N)&#xA;groups = np.random.choice(np.arange(100), N)&#xA;&#xA;def group_apply(values, groups, func):&#xA;    output = np.repeat(np.nan, len(values))&#xA;    ixs = get_group_name_and_rows(groups)&#xA;    for ix in ixs.itervalues():&#xA;        output[ix] = func(values[ix])&#xA;    return output&#xA;&#xA;def get_group_name_and_rows(groups):&#xA;    mapper = defaultdict(list)&#xA;    for i, group in enumerate(groups):&#xA;        mapper[group].append(i)&#xA;    return mapper&#xA;&#xA;group_apply(values, groups, np.cumsum)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is now a big bottleneck in my code, and I was wondering if you know of any way to speed this up (perhaps cythonise it?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="8121" LastEditorUserId="24110" LastEditDate="2017-05-26T20:03:12.180" LastActivityDate="2017-05-26T20:03:12.180" Title="Speeding up group apply in python" Tags="|python|performance|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26979" PostTypeId="2" ParentId="26963" CreationDate="2017-05-25T11:34:52.403" Score="1" Body="&lt;p&gt;Please check “The Finite Element Method: Theory, Implementation, and Applications” by Larson and Bengzon. They have Matlab implementation of P2 Lagrange elements (assembly of stiffness matrix) on page 217.&#xA;It should be easy to implement higher order elements based on their code as well.&lt;/p&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-05-25T11:34:52.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26980" PostTypeId="2" ParentId="17562" CreationDate="2017-05-25T15:56:23.197" Score="4" Body="&lt;p&gt;This question now has quite some age, but its subject crops up so repeatedly, &#xA;that perhaps the following answer and references may still be helpful and not out of place.&lt;/p&gt; &#xA;(a)  'Is there a reason to ... solve for ... the eccentric anomaly' (when &#xA;the true anomaly is more meaningful)?'&lt;/p&gt;&#xA;   Practically all of the numerical difficulty is in 'Kepler's &#xA;equation', i.e. the part of the calculation for an undisturbed elliptical orbit that starts with the mean anomaly and computes the eccentric anomaly. The onward calculation of true anomaly from eccentric, by &#xA;contrast, can be simple enough and numerically well-behaved when properly &#xA;arranged. (A better-behaved formula than the traditional 'tangent half-angle' formula quoted in the question is given by R Broucke &amp;amp; P Cefola (1973), &#xA;[&lt;a href=&quot;http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=1973CeMec...7..388B]&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=1973CeMec...7..388B]&lt;/a&gt;.) Direct series evaluations of true anomaly do exist, of course, but the coefficients are usually laborious to compute, except where that can be done once for all, e.g. when only a few predetermined eccentricity values will be used.  There are also a few iterative procedures that seem to go direct from mean to true anomaly, but they don't entirely avoid the eccentric anomaly, and it's arguable they only mix and mask difficulties without gaining simplicity or efficiency.) So it's usual to tackle separately the Kepler-equation part of the problem.&lt;/p&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(b)  There are many published good procedures for solving Kepler's equation, &#xA;thus arguably little point now in devising one afresh. The subject is &#xA;entirely 400 years old, but it received renewed interest in the 1960s on, when astrodynamics developed increased demand for procedures for orbit-tracking and control. These demand (as intermediates) unattended automatic computation of many thousands of instances of Kepler's equation. What emerged then were needs not only for computational efficiency, but also for the avoidance of added kinds of numerical misbehavior, such as misconvergence or failure to converge from a minority of pathological input values. Here is a review and a brief selection of solutions from that recent era.&lt;/p&gt;&#xA;**  There's a good explanatory survey of methods up to the mid-80s, and of the problems to be met, (plus some coded Fortran functions), in: &#xA;A W Odell, R H Gooding, 'Procedures for Solving Kepler's Equation' &#xA;[&lt;a href=&quot;http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=1986CeMec..38..307O]&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=1986CeMec..38..307O]&lt;/a&gt;.&lt;/p&gt; &#xA;** A very useful paper is that of A Nijenhuis (1991), 'Solving Kepler's &#xA;equation with high efficiency and accuracy'&#xA;[substitute the following bibcode in the URL given above: bibcode=1991CeMDA..51..319N], &#xA;it's accompanied by a complete and workable implementation (in Pascal, &#xA;from 1991, but easily portable e.g. to C). Methods of generating a starting- &#xA;approximation are selected according to the input values of M (radians) and &#xA;e (#), and a recently-devised cubic-approximation (Mikkola, 1987) is used &#xA;for starting values in the specially-problematic area near to the singularity &#xA;at (M,e) = (0, 1). &lt;/p&gt;** Another modern method, perhaps even potentially better than the foregoing, is described (without code example) by F L Markley (1995), 'Kepler Equation Solver' [substitute the following bibcode in the URL given above: bibcode=1995CeMDA..63..101M].&#xA;&lt;/p&gt;&#xA;** There are various methods that claim to avoid the calculation of trig functions. One class avoids them at run-time, but the cost is a precomputed lookup table (e.g. S A Feinstein, C A McLaughlin, 'Dynamic discretization method for solving Kepler's equation', [substitute the following bibcode in the URL given above: bibcode=2006CeMDA..96...49F].&#xA;Another method ingeniously does avoid computing trig functions; it achieves that by evaluating sin E (via sin E/3) rather than E itself:&lt;br&gt;&#xA;S Mikkola, 'A cubic approximation for Kepler's equation',&#xA;[substitute the following bibcode in the URL given above: bibcode=1987CeMec..40..329M].&#xA;&lt;/p&gt;&#xA;Hopefully there's something here to respond to most variants of this oft-recurring question.&lt;/p&gt;&#xA;" OwnerUserId="24369" LastActivityDate="2017-05-25T15:56:23.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="26981" PostTypeId="2" ParentId="26967" CreationDate="2017-05-25T17:21:23.167" Score="4" Body="&lt;p&gt;Like @likask points out in the other answer, using Lagrange multipliers is one way to do this. The other is to recognize that if you have constraints of the form $\sum_i a_i U_i=0$, then you can select one degree of freedom (typically the one with the largest $a_i$, to maintain numerical stability and rewrite the constraint as&#xA;$$&#xA;  U_\ast = -\sum_{i, i\neq \ast} \frac{a_i}{a_\ast} U_i,&#xA;$$&#xA;i.e., one degree of freedom is a linear combination of a bunch of others. This is then no different from what you do, for example, with hanging nodes, with tangential flow boundary conditions, and a bunch of other things in finite element codes. There is a well-developed machinery to deal with such constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example where this is done in practice is the &lt;a href=&quot;https://dealii.org/developer/doxygen/deal.II/step_11.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;step-11 tutorial program of deal.II&lt;/a&gt;. If you're unsure how constraints such as this are handled algorithmically, you may be interested in &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;my video lecture 16&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-05-25T17:21:23.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26982" PostTypeId="2" ParentId="26978" CreationDate="2017-05-26T06:24:40.653" Score="0" Body="&lt;p&gt;You don't have to cythonize anything, it would be sufficient to rethink the way you represent data (e.g. groups) and use numpy to do the hard work (i.e. iterating over arrays of length N) efficiently. Here is my take on the problem, definitely not optimal but enough to give you an idea. Your version was limited by the &lt;code&gt;defaultdict&lt;/code&gt; construction in the &lt;code&gt;get_group_name_and_rows&lt;/code&gt; function, mine is limited by the two &lt;code&gt;argsort&lt;/code&gt;s, so if you can have sorted data in the whole program (or at least sort only once for multiple invokations), then you can do much better.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from collections import defaultdict&#xA;import numpy as np&#xA;&#xA;N = 1000000&#xA;values = np.arange(N)&#xA;groups = np.random.choice(np.arange(100), N)&#xA;&#xA;def group_apply(values, groups, func):&#xA;    output = np.repeat(np.nan, len(values))&#xA;    ixs = get_group_name_and_rows(groups)&#xA;    for ix in ixs.values():&#xA;        output[ix] = func(values[ix])&#xA;    return output&#xA;&#xA;def get_group_name_and_rows(groups):&#xA;    mapper = defaultdict(list)&#xA;    for i, group in enumerate(groups):&#xA;        mapper[group].append(i)&#xA;    return mapper&#xA;&#xA;def my_group_apply(values, groups, func):&#xA;    output = np.repeat(np.nan, len(values))&#xA;&#xA;    # Sort values and groups. Note that a *stable* algorithm has to be used.&#xA;    perm = groups.argsort(kind=&quot;mergesort&quot;)&#xA;    sorted_values = values[perm]&#xA;    sorted_groups = groups[perm]&#xA;&#xA;    # Indices of the non-zero elements in the diffs array represent where each group ends.&#xA;    diffs = np.diff(sorted_groups)&#xA;    separators = np.append(np.flatnonzero(diffs), len(values))&#xA;&#xA;    # Iterate over the groups and compute the func on each group.&#xA;    left = 0&#xA;    for right in separators:&#xA;        output[left:right+1] = func(sorted_values[left:right+1])&#xA;        left = right + 1&#xA;&#xA;    # Permute the output array into the original order.&#xA;    iperm = perm.argsort()&#xA;    return output[iperm]&#xA;&#xA;output = group_apply(values, groups, np.cumsum)&#xA;my_output = my_group_apply(values, groups, np.cumsum)&#xA;&#xA;assert (output == my_output).all()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/rkern/line_profiler&quot; rel=&quot;nofollow noreferrer&quot;&gt;Profiler&lt;/a&gt; output with &lt;code&gt;N = 1000000&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;Timer unit: 1e-06 s&#xA;&#xA;Total time: 0.926312 s&#xA;File: test.py&#xA;Function: group_apply at line 8&#xA;&#xA;Line #      Hits         Time  Per Hit   % Time  Line Contents&#xA;==============================================================&#xA;     8                                           @profile&#xA;     9                                           def group_apply(values, groups, func):&#xA;    10         1         5339   5339.0      0.6      output = np.repeat(np.nan, len(values))&#xA;    11         1       684709 684709.0     73.9      ixs = get_group_name_and_rows(groups)&#xA;    12       101          399      4.0      0.0      for ix in ixs.values():&#xA;    13       100       235864   2358.6     25.5          output[ix] = func(values[ix])&#xA;    14         1            1      1.0      0.0      return output&#xA;&#xA;Total time: 0.207693 s&#xA;File: test.py&#xA;Function: my_group_apply at line 22&#xA;&#xA;Line #      Hits         Time  Per Hit   % Time  Line Contents&#xA;==============================================================&#xA;    22                                           @profile&#xA;    23                                           def my_group_apply(values, groups, func):&#xA;    24         1         4770   4770.0      2.3      output = np.repeat(np.nan, len(values))&#xA;    25                                           &#xA;    26                                               # Sort values and groups. Note that a *stable* algorithm has to be used.&#xA;    27         1        94151  94151.0     45.3      perm = groups.argsort(kind=&quot;mergesort&quot;)&#xA;    28         1        10533  10533.0      5.1      sorted_values = values[perm]&#xA;    29         1        10652  10652.0      5.1      sorted_groups = groups[perm]&#xA;    30                                           &#xA;    31                                               # Indices of the non-zero elements in the diffs array represent where each group ends.&#xA;    32         1         2856   2856.0      1.4      diffs = np.diff(sorted_groups)&#xA;    33         1         7574   7574.0      3.6      separators = np.append(np.flatnonzero(diffs), len(values))&#xA;    34                                           &#xA;    35                                               # Iterate over the groups and compute the func on each group.&#xA;    36         1            1      1.0      0.0      left = 0&#xA;    37       101          120      1.2      0.1      for right in separators:&#xA;    38       100         4967     49.7      2.4          output[left:right+1] = func(sorted_values[left:right+1])&#xA;    39       100          170      1.7      0.1          left = right + 1&#xA;    40                                           &#xA;    41                                               # Permute the output array into the original order.&#xA;    42         1        63462  63462.0     30.6      iperm = perm.argsort()&#xA;    43         1         8437   8437.0      4.1      return output[iperm]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And with &lt;code&gt;N = 10000000&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;Timer unit: 1e-06 s&#xA;&#xA;Total time: 12.6474 s&#xA;File: test.py&#xA;Function: group_apply at line 8&#xA;&#xA;Line #      Hits         Time  Per Hit   % Time  Line Contents&#xA;==============================================================&#xA;     8                                           @profile&#xA;     9                                           def group_apply(values, groups, func):&#xA;    10         1        63560  63560.0      0.5      output = np.repeat(np.nan, len(values))&#xA;    11         1      7986616 7986616.0     63.1      ixs = get_group_name_and_rows(groups)&#xA;    12       101          602      6.0      0.0      for ix in ixs.values():&#xA;    13       100      4596623  45966.2     36.3          output[ix] = func(values[ix])&#xA;    14         1            1      1.0      0.0      return output&#xA;&#xA;Total time: 2.63137 s&#xA;File: test.py&#xA;Function: my_group_apply at line 22&#xA;&#xA;Line #      Hits         Time  Per Hit   % Time  Line Contents&#xA;==============================================================&#xA;    22                                           @profile&#xA;    23                                           def my_group_apply(values, groups, func):&#xA;    24         1        64885  64885.0      2.5      output = np.repeat(np.nan, len(values))&#xA;    25                                           &#xA;    26                                               # Sort values and groups. Note that a *stable* algorithm has to be used.&#xA;    27         1      1354211 1354211.0     51.5      perm = groups.argsort(kind=&quot;mergesort&quot;)&#xA;    28         1       130488 130488.0      5.0      sorted_values = values[perm]&#xA;    29         1       144869 144869.0      5.5      sorted_groups = groups[perm]&#xA;    30                                           &#xA;    31                                               # Indices of the non-zero elements in the diffs array represent where each group ends.&#xA;    32         1        27008  27008.0      1.0      diffs = np.diff(sorted_groups)&#xA;    33         1        74815  74815.0      2.8      separators = np.append(np.flatnonzero(diffs), len(values))&#xA;    34                                           &#xA;    35                                               # Iterate over the groups and compute the func on each group.&#xA;    36         1            2      2.0      0.0      left = 0&#xA;    37       101          199      2.0      0.0      for right in separators:&#xA;    38       100        40465    404.6      1.5          output[left:right+1] = func(sorted_values[left:right+1])&#xA;    39       100          363      3.6      0.0          left = right + 1&#xA;    40                                           &#xA;    41                                               # Permute the output array into the original order.&#xA;    42         1       707766 707766.0     26.9      iperm = perm.argsort()&#xA;    43         1        86302  86302.0      3.3      return output[iperm]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-05-26T06:24:40.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26983" PostTypeId="1" AcceptedAnswerId="26985" CreationDate="2017-05-26T07:19:28.340" Score="3" ViewCount="1724" Body="&lt;p&gt;I know that an easy way to solve the matrix problem&#xA;$$A\cdot x=b$$&#xA;is the LU decomposition&#xA;$$\begin{split}&#xA;L,\,U&amp;amp;=\text{lu}(A)\\&#xA;y&amp;amp;=\text{solve}(L,\,b)\\&#xA;x&amp;amp;=\text{solve}(U,\,y)&#xA;\end{split}$$&#xA;Thus I implemented the following code in matlab:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r_num = 10000;&#xA;r_rounds = 10000;&#xA;&#xA;A = -30*sparse(diag(ones(r_num,1),0))+16*sparse(diag(ones(r_num-1,1),1))+16*sparse(diag(ones(r_num-1,1),-1))-sparse(diag(ones(r_num-2,1),-2))-sparse(diag(ones(r_num-2,1),2));&#xA;&#xA;b = rand(r_num,1);&#xA;&#xA;tic;&#xA;for i = 1:r_rounds&#xA;    c1 = A\b;&#xA;end&#xA;toc;&#xA;&#xA;tic;&#xA;[L, U] = lu(A);&#xA;for i = 1:r_rounds&#xA;    y = L\b;&#xA;&#xA;end&#xA;toc;&#xA;tic;&#xA;for i = 1:r_rounds&#xA;    c2 = U\y;&#xA;end&#xA;toc;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I already know that $A\setminus b$ is doing the LU-decomposition in the background, but I still was interested in the result. Now my script returned that the calculation of $L\setminus b$ was nearly as expensive as the direct calculation $A\setminus b$, while the calculation of $U\setminus y$ was extremely fast. But where does the difference come from? Or is my measurement method flawed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When timing the LU-decomposition separately, I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A\b&#xA;Elapsed time is 13.380155 seconds.&#xA;lu(A)&#xA;Elapsed time is 0.002662 seconds.&#xA;y=L\b&#xA;Elapsed time is 10.557989 seconds.&#xA;c2=U\y&#xA;Elapsed time is 1.229101 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The reason for timing the LU-decomposition separately (outside of the loop) is because for my later application I will have to deal with a constant $A$, but a large amount of changing $b$-values. Thus I try to keep the amount of elements in the loop as low as possible.&lt;/p&gt;&#xA;" OwnerUserId="20545" LastEditorUserId="24110" LastEditDate="2017-05-27T02:35:53.990" LastActivityDate="2017-05-27T02:35:53.990" Title="Why is the speed of the parts of the LU-decomposition so different?" Tags="|linear-solver|sparse-matrix|matrix|performance|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="26985" PostTypeId="2" ParentId="26983" CreationDate="2017-05-26T09:30:23.493" Score="6" Body="&lt;p&gt;First, don't forget to also time the LU decomposition in a loop! Otherwise it's not really a fair comparison. If I do that, I get the following timings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A\b:   Elapsed time is 10.294272 seconds.&#xA;lu(A): Elapsed time is 26.675160 seconds.&#xA;L\b:   Elapsed time is 9.934748 seconds.&#xA;U\y:   Elapsed time is 1.469325 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can see that both &lt;code&gt;A\b&lt;/code&gt; and &lt;code&gt;lu(A)&lt;/code&gt; are of the same order of magnitude, which is expected. Since the matrix in this example is in fact symmetric, you'd also expect that Matlab will &lt;em&gt;not&lt;/em&gt; do an LU decomposition. Replacing &lt;code&gt;lu&lt;/code&gt; by &lt;code&gt;chol&lt;/code&gt; gives a timing of &lt;code&gt;10.067633 seconds&lt;/code&gt; -- very close to Matlab's backslash. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In fact, you can ask Matlab what it's doing by setting &lt;code&gt;spparms('spumoni',2)&lt;/code&gt;, which will tell you on &lt;code&gt;A\b&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sp\: bandwidth = 2+1+2.&#xA;sp\: is A diagonal? no.&#xA;sp\: is band density (1) &amp;gt; bandden (0.5) to try banded solver? yes.&#xA;sp\: is LAPACK's banded solver successful? yes.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So it's in fact using a specialized solver.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why &lt;code&gt;L\b&lt;/code&gt; is so much slower than &lt;code&gt;U\y&lt;/code&gt;, it's instructive to look at the sparsity pattern of both via &lt;code&gt;spy&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yJ0Nl.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yJ0Nl.png&quot; alt=&quot;sparsity pattern of L&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kJnaR.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kJnaR.png&quot; alt=&quot;sparsity pattern of U&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that &lt;code&gt;L&lt;/code&gt; has a much more complicated sparsity pattern than &lt;code&gt;U&lt;/code&gt; (which is upper tridiagonal, corresponding to the sparsity pattern of &lt;code&gt;A&lt;/code&gt;), so the corresponding substitution step will be much slower. In fact, asking Matlab again what it's doing on &lt;code&gt;L\b&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sp\: bandwidth = 2+1+707.&#xA;sp\: is A diagonal? no.&#xA;sp\: is band density (0.0043793) &amp;gt; bandden (0.5) to try banded solver? no.&#xA;sp\: is A triangular? no.&#xA;sp\: is A morally triangular? yes.&#xA;sp\: permute and solve.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you see that extra work is being done here. &lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2017-05-26T10:29:33.637" LastActivityDate="2017-05-26T10:29:33.637" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26986" PostTypeId="2" ParentId="26939" CreationDate="2017-05-26T11:49:15.937" Score="2" Body="&lt;p&gt;A good way to obtain to verify the CFD portion of your code is to proceed using the method of manufactured solution (MMS). You can look at the book by &lt;a href=&quot;https://www.amazon.ca/Verification-Validation-Scientific-Computing-Oberkampf/dp/0521113601&quot; rel=&quot;nofollow noreferrer&quot;&gt;Oberkampf &amp;amp; Roy&lt;/a&gt; or even this article by &lt;a href=&quot;http://sciencedirect.com/science/article/pii/S0045793015000675&quot; rel=&quot;nofollow noreferrer&quot;&gt;me&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key principle of MMS is to decide on an analytical solution to your problem (in this case a velocity and/or pressure profile) and, using a symbolic calculator, to calculate the source term that corresponds to this flow profile inserted within the Navier-Stokes equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are numerous commercial (Matlab, Mathematica, Maple) or open source (Python + SymPy, Sage) ways to calculate this source term. I have found that SymPy is super easy to use and is very efficient. The source terms can be quite long, but if you take trigonometric functions in your flow profile, they generally simplify to simple expressions. You can take the flow profile from my article as an example.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2017-05-26T11:49:15.937" CommentCount="27" ContentLicense="CC BY-SA 3.0" />
  <row Id="26987" PostTypeId="1" CreationDate="2017-05-26T13:10:10.797" Score="1" ViewCount="465" Body="&lt;p&gt;I have a function of 3 variables like &lt;span class=&quot;math-container&quot;&gt;$V= f(r_1, p, r_2)$&lt;/span&gt;, I want to plot these 4 data in one graph to be able to capture the behavior of V with respect to all three variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched a little bit and read about &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/764-sliceomatic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sliceomatic&lt;/a&gt; but not sure how does it work and if it delivers my requirements. I would appreciate if someone could suggest a way for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example lets say I have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r1 = [1 2 3 4 5 6];&#xA;p = [2 4 5 8 9 7];&#xA;r2 = [10 45 1 0 7 9];&#xA;v = 5.*r1.^2 + 2.*p.^2 + r2.^2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24380" LastEditorUserId="9667" LastEditDate="2019-06-20T15:00:34.037" LastActivityDate="2019-06-20T15:00:34.037" Title="3D visualization in MATLAB" Tags="|matlab|visualization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="26988" PostTypeId="1" AcceptedAnswerId="27007" CreationDate="2017-05-26T16:47:55.163" Score="1" ViewCount="614" Body="&lt;p&gt;I have a very basic question, and I hope some of you might be able to help me: In Fluid Dynamics, a common equation turning up time and time again is the so-called continuity equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial}{\partial x}(T\frac{\partial h}{\partial x}) + R = S\frac{\partial h}{\partial t}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $T$ is the transmissivity of the soil matrix, defined by the product $Kh$, where $h$ is the hydraulic head and $K$ is the hydraulic conductivity (a constant), $R$ a recharge constant, and $S$ the storage coefficient, another constant. In FD solutions for the unconfined case (where the transmissivity varies with $h$) this equation is often linearized by assuming that $T$ is constant. If you ignore this linearization and continue, you get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial}{\partial x}(Kh\frac{\partial h}{\partial x}) + R = S\frac{\partial h}{\partial t}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I don't see a problem with solving this. Couldn't one just apply the chain rule of differentiation to get...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial Kh}{\partial x}\frac{\partial h}{\partial x} +Kh\frac{\partial^2 h}{\partial x^2}+ R = S\frac{\partial h}{\partial t}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... and then solve each of these terms by conventional centered differences and second order central differences? I can see that this kind of solution may be problematic for implicit schemes, where you will get a squared $h$ in the first term, but why is this not done in explicit schemes?&lt;/p&gt;&#xA;" OwnerUserId="24382" LastActivityDate="2017-05-30T03:14:33.560" Title="Linearization in Finite Difference Method: Why?" Tags="|finite-difference|nonlinear-equations|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26989" PostTypeId="2" ParentId="11658" CreationDate="2017-05-26T18:32:32.080" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;http://spatial.sourceforge.net/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spatial&lt;/a&gt; library allows to define a custom &lt;a href=&quot;http://spatial.sourceforge.net/metric_concept.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metric&lt;/a&gt;, which can ignore some specific dimensions, or more generally project the points onto the specified subspace before calculating the difference.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-05-26T18:32:32.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26991" PostTypeId="1" CreationDate="2017-05-26T23:02:10.080" Score="1" ViewCount="779" Body="&lt;p&gt;I am solving the following mixed boundary elasticity problem with the finite element method on the unit circle, using a triangle mesh with 3 nodes on each element. In my problem I know $u_1$  and $t_2$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LxUSU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LxUSU.png&quot; alt=&quot;problem given&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the following weak form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega \sum_{i,j}^2 \sigma_{ij}(u) \epsilon_{ij}(v)= \int_\tau v_2 t_2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already implemented it correctly for the pure Dirichlet problem. (so the global stiffness matrix is also good for this problem), but I do something wrong when implementing the Neumann boundary condition and I cannot figure out what. (I still get the right solution for $u_1$ where I have the Dirichlet condition). On the right side of the picture is the fem solution and on the left side is the analytical one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mV3A9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mV3A9.png&quot; alt=&quot;solution representation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I do when implementing the Neumann boundary condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I go over each triangular element:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I extract the coordinates of each node of the element:&#xA;$(x_1,y_1), (x_2,y_2), (x_3, y_3)$&#xA;(the elements are numbered in anticlockwise sense)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if $(x_1,y_1)$ and $(x_2,y_2)$ are on the boundary:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I calculate the outward unit normal:&#xA;$$ n2=\frac{x_1-x_2}{\sqrt{(y_2-y_1)^2+(x_1-x_2)^2}} $$&lt;/li&gt;&#xA;&lt;li&gt;I update the force vector F in the place corresponding to $C_e^12$ and $C_e^22$ ($v_2=\sum_{B=1}^3 N^BC_e^B2$) by adding :&#xA;$$ 0.5 \sigma_0 n_2 \sqrt{(x_2-x_1)^2+(y_2-y_1)^2}$$  &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if $(x_1,y_1)$ and $(x_3,y_3)$ are on the boundary:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$n2=\frac{x_3-x_1}{\sqrt{(y_1-y_3)^2 + (x_3-x_1)^2}}$&lt;/li&gt;&#xA;&lt;li&gt;update with&#xA;$ -0.5 \sigma_0 n_2 \sqrt{(x_3-x_1)^2 + (y_3-y_1)^2} $&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if $(x_2,y_2)$ and $(x_3,y_3)$ are on the boundary:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$ n2=\frac{x_2-x_3}{\sqrt{(y_3-y_2)^2+(x_2-x_3)^2}} $&lt;/li&gt;&#xA;&lt;li&gt;update with:&#xA;$ 0.5 \sigma_0 n_2 \sqrt{(x_3-x_2)^2 + (y_3-y_2)^2} $    &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="24385" LastEditorUserId="9667" LastEditDate="2017-05-27T14:59:53.017" LastActivityDate="2017-05-27T14:59:53.017" Title="Implementing Neumann boundary condition for elasticity problem using the finite element method" Tags="|finite-element|boundary-conditions|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="26992" PostTypeId="2" ParentId="26938" CreationDate="2017-05-27T06:01:19.263" Score="0" Body="&lt;p&gt;After some searching, I found the explanation in &lt;em&gt;&quot;Perspectives on Projective Geometry&quot;&lt;/em&gt; by Richter-Gebert.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is finding a line $g$ such that $g^TRg=0$, and $R=L^TQL$ where $Q$ describes the ellipse. The transformation with $L$ makes $R$ a dual conic consisting of two points of intersection and a double line that passes through them. $R$ is also a degenerate conic, so it has rank 1 or 2. For the rank 2 case, if R is symmetric (which I think it always is) it can be rewritten as $R = uv^T + vu^T$ (up to a scalar multiple), where $u, v$ are points where the line intersects the original conic. $uv^T$ alone can describe the degenerate conic since &#xA;$$0 = \langle g,u \rangle \cdot \langle g, v \rangle = (g^Tu)(v^Tg)= g^T(uv^T)g$$&#xA;with $\langle g, u \rangle=0$ and $\langle g, v \rangle=0$ if $u$ and $v$ are on $g$, respectively. The product $uv^T$ is rank-1, so we need to find a matrix $W$ such that $g^T(R+W)g=0$ and $(R+W)$ is rank-1. If $W$ is skew-symmetric, it will satisfy the first condition. The matrix sum then becomes:&#xA;$$R+W=\left ( \begin{matrix} a&amp;amp;b&amp;amp;d\\b&amp;amp;c&amp;amp;e\\d&amp;amp;e&amp;amp;f\end{matrix} \right )+\left ( \begin{matrix}0&amp;amp;\tau&amp;amp;-\mu\\-\tau&amp;amp;0&amp;amp;\lambda\\\mu&amp;amp;-\lambda&amp;amp;0 \end{matrix} \right )$$&#xA;Notice that $M_w\,q=w \times q$ for any vector $q$ and the vector $w=[\lambda,\mu,\tau]^T$. We now need to solve for $w$ so that $(R+W)$ is rank-1. Notice that if the line $l$ passes through $u$ and $v$, $u \times v=\alpha l$, where $\alpha$ is a scalar we can choose. If we consider the sum $uv^T-vu^T$, we can see through expanding and analyzing the elements that:&#xA;$$uv^T-vu^T = \alpha M_{u\times v}=M_{\alpha l}=\alpha M_l$$&#xA;Thus, $R+\alpha M_l = 2 uv^T$. Thus, we can then choose $\alpha$ to make $R+\alpha M_l$ rank-1. Since $L = M_l$, then we can say:&#xA;$$L^TQL+\alpha L = uv^T$$&#xA;Since $u$ and $v$ are in homogeneous coordinates, any scalar multiples don't affect the points when we project them back into $\mathbb{R}^2$.&lt;/p&gt;&#xA;" OwnerUserId="21131" LastActivityDate="2017-05-27T06:01:19.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26995" PostTypeId="2" ParentId="1391" CreationDate="2017-05-27T14:02:10.677" Score="1" Body="&lt;p&gt;Another class of problems that are hard to parallelize in practice are problems sensitive to rounding errors, where numerical stability is achieved by serialization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider for example the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process#Numerical_stability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gram–Schmidt process&lt;/a&gt; and its serial modification. The algorithm works with vectors, so you might use parallel vector operations, but that does not scale well. If the number of vectors is large and the vector size is small, using parallel classical Gram–Schmidt and reorthogonalization might be stable and faster than single modified Gram–Schmidt, although it involves doing several times more work.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-05-27T14:02:10.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="26996" PostTypeId="1" CreationDate="2017-05-27T20:33:40.070" Score="1" ViewCount="57" Body="&lt;p&gt;I am solving quadratic minimization problem&#xA;\begin{align}&#xA;\min_{x}\ \frac{1}{2} x^T A x -b^T x,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where matrix A results from discretization of Laplacian by FEM method, subjected to Dirichlet BC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having concerns about computing energy. &#xA;Should product $ \frac{1}{2} x^T A x -b^T x $ be computed by using matrix A with encoded modification for given BC (zero-ing off diagonals), or should it be done directly on unmodified (pure Neumann) matrix A ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any special treatment of boundary conditions needed ? &lt;/p&gt;&#xA;" OwnerUserId="17045" LastEditorUserId="17045" LastEditDate="2017-05-28T03:31:36.547" LastActivityDate="2017-05-28T03:31:36.547" Title="energy computation for BVP with Dirichlet boundary conditions" Tags="|optimization|finite-element|quadratic-programming|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="26998" PostTypeId="1" CreationDate="2017-05-28T08:23:08.663" Score="1" ViewCount="100" Body="&lt;p&gt;i have come across method of IMEX Runge-Kutta written by Kennedy and Carpenter in july 2001. The method consist of two kind that is &quot;implicit and explicit&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can i use the ERK@explicit part only to solve the intial value problem, as if i treat the method as another embedded Runge-Kutta?&#xA;is it against the algorithm/rule of the method?&lt;/p&gt;&#xA;" OwnerUserId="24394" LastActivityDate="2017-05-28T08:23:08.663" Title="How and when to use IMEX runge-kutta" Tags="|runge-kutta|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="26999" PostTypeId="2" ParentId="25167" CreationDate="2017-05-28T09:55:11.417" Score="1" Body="&lt;p&gt;It looks like there may be a possible using ReaxFF to study the properties of multiphase in the same simulation system. This group just has being developed the ReaxFF which might suit your need. &lt;a href=&quot;http://www.engr.psu.edu/adri/ResearchInterest.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dr. Adri Group Research Topics&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also here is the study of MD simulation of containing gaseous and aqueous mixing phases. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3658833/pdf/nihms-91029.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Molecular Modeling of Geometries, Charge Distributions, and Binding Energies of Small, Drug-Like Molecules Containing Nitrogen Heterocycles and Exocyclic Amino Groups in the Gas Phase and Aqueous Solution&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this guide you.&lt;/p&gt;&#xA;" OwnerUserId="22108" LastActivityDate="2017-05-28T09:55:11.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27000" PostTypeId="2" ParentId="23642" CreationDate="2017-05-28T10:07:40.133" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I think that the &lt;a href=&quot;http://pubs.acs.org/journal/chreay&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chemical Reviews (ACS Publications)&lt;/a&gt; might be came across to your question. You could make the criteria search by using some specific keyword. The results of your search depends on both of 'keyword that you used' and 'where you search'.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;However, I don't think that you could find the solution quickly in any Computational Chemistry textbooks. If you read the kind of books which have been released so far, of course the authors missed some point in conclusion of new era of DFT calculation that has been being developed very much, as well as the benchmarking in those book has already out date on the day you start reading them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Again, so you should have a look at the article or journal review that authors especially focuses on something, says &lt;em&gt;comparison of Meta GGA functional to study of Hydrogen bond in organometallic compounds&lt;/em&gt;, Comparison of diffuses pople basis set, etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22108" LastActivityDate="2017-05-28T10:07:40.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27002" PostTypeId="1" CreationDate="2017-05-28T13:13:55.933" Score="8" ViewCount="869" Body="&lt;p&gt;I am solving the Dirichlet problem&#xA;$$&#xA;  \begin{cases}&#xA;   \Delta u = 0, \\&#xA;   u|_{\partial D} = f,&#xA;  \end{cases}&#xA;$$&#xA;in a $2d$ domain $D$ using the finite element method. What I want to get is the normal derivative of the solution $u$ on the boundary: $\tfrac{\partial u}{\partial \nu}$. In other words, I'm interested in FEM implementation of the Dirichlet-to-Neumann (Poincare-Steklov) map $\Lambda$. In Wikipedia I've found the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When the partial differential equation is discretized, for example by&#xA;  finite elements or finite differences, the discretization of the&#xA;  Poincaré–Steklov operator is the Schur complement obtained by&#xA;  eliminating all degrees of freedom inside the domain.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Hovewer, I could not find an explicit answer to this problem in Google. I'm programming in Matlab and I would extremely appreciate an optimized solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; (30 May 2017). As it was pointed out by &lt;em&gt;Praveen Chandrashekar&lt;/em&gt; in comments, I tried to solve this problem by formulating it in a weak form: find $h \in L^2(\partial D)$ such that for any $w \in H^{\frac 1 2}(\partial D)$,&#xA;$$&#xA;   \int_{\partial D} hw \,dl = \int_D \nabla w \nabla u dS.&#xA;$$&#xA;As usually, I represent $h = \sum' h_j \psi_j$, $w = \sum w_i \psi_i$, $u = \sum u_l \psi_l$, where $\psi_i$ are the basis elements and $\sum'$ denotes summation over the boundary elements only, and I get&#xA;$$&#xA;  \sum_i' \sum_j' h_j w_i \int_{\partial D} \psi_i \psi_j dl =  \sum_k \sum_l w_k u_l \int_D \nabla \psi_k \nabla \psi_l dS.&#xA;$$ &#xA;Since $\{w_j\}$ are arbitrary, we get the system&#xA;$$&#xA;   \sum_j' \int_{\partial D} \psi_i \psi_j dl \cdot h_j = \sum_l\int_D \nabla \psi_k \nabla \psi_l dS \, \cdot u_l.&#xA;$$&#xA;Introducing the matrices&#xA;$$&#xA;   B_{ij} = \int_{\partial D} \psi_i \psi_j \, dl, \quad A_{ij} = \int_D \nabla \psi_i \nabla \psi_j dS, &#xA;$$&#xA;where in the first matrix $i$, $j$ correspond to boundary elements and in the second matrix $i$ corresponds to the boundary elements and $j$ corresponds to all elements, I get the matrix equation&#xA;$$&#xA;  B h = Au,&#xA;$$&#xA;for finding $h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I implement this approach, the results are far from being precise. For example, consider the following circular domain and boundary values $u_0(x,y) = -y$ (triangulation created using DistMesh):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DwKh9.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DwKh9.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the flux along the boundary must coincide with $u_0(x,y)$ but I get the following noisy picture:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jpIBv.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jpIBv.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="1180" LastEditorUserId="1180" LastEditDate="2017-05-30T15:41:32.363" LastActivityDate="2017-05-30T15:41:32.363" Title="Numerical implementation of the Dirichlet-to-Neumann map" Tags="|finite-element|matlab|boundary-conditions|elliptic-pde|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="27005" PostTypeId="1" CreationDate="2017-05-29T13:11:44.060" Score="1" ViewCount="1709" Body="&lt;p&gt;I am new into programming. Here, I am trying to simulate the motion of a cannonball until it hits the ground using python programming. I have used both analytical and numerical(Euler method) to plot its trajectory. But, I am not sure if my output is correct for both methods, especially for the analytical solution. For analytical solution, is there any way to show that for t=0, x=1m which is true. &#xA;Please look at the code below, and point out my mistakes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;v=0.0      #Initial 0 velocity&#xA;h=0.01     #dt&#xA;x=1        #Canonball is at a distance of 1m from the ground initially     &#xA;t=0.0&#xA;m=2.0&#xA;g=9.81 &#xA;&#xA;ta,xa,xb=[],[],[]   #xa for numerical and xb for analytical solution&#xA;&#xA;while x&amp;gt;=0.0:&#xA;    ta.append(t)&#xA;    xa.append(x)&#xA;    xb.append(-0.5*g*t**2.0)  #Analytical solution&#xA;    F=-m*g                       #Numerical solution&#xA;    v=v+(F/m)*h&#xA;    x=x+v*h&#xA;    t=t+h&#xA;&#xA;plt.figure()&#xA;plt.plot(ta,xa)&#xA;plt.plot(ta,xb,'ro-')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17278" LastActivityDate="2017-05-29T13:11:44.060" Title="Python program to simulate the trajectory of a cannonball" Tags="|python|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27007" PostTypeId="2" ParentId="26988" CreationDate="2017-05-30T03:14:33.560" Score="3" Body="&lt;p&gt;For explicit schemes, this indeed makes sense -- there is no reason to linearize the explicit terms. However, as others have already pointed out, you get into trouble with explicit schemes for the diffusion equation because the time step must be chosen proportional to something like&#xA;$$&#xA;  \Delta t \propto \min_x \frac{S(x) \delta x^2}{T(x)} \propto \min_x \frac{S(x) \delta x^2}{Kh},&#xA;$$&#xA;i.e., you need to choose the time step very small if either $S$ becomes small somewhere in the domain, or $h$ becomes large. In addition, of course, reducing the grid size by a factor of 2 requires you to reduce the time step size by an undesirable factor of 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reasons, explicit time stepping methods are not commonly used for the diffusion equation. Rather, one wants to use implicit methods to avoid the awkward CFL condition and take large time steps, in particular if one expects that after some time the solution finds some kind of equilibrium and we want to take large time steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you use an implicit method, you have to linearize somehow because otherwise you end up with a system of nonlinear systems that may be very large -- and we don't know how to solve these other than through a linearization procedure that reduce the large nonlinear problem to a sequence of large linear problems.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-05-30T03:14:33.560" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27008" PostTypeId="1" CreationDate="2017-05-30T03:59:38.087" Score="1" ViewCount="239" Body="&lt;p&gt;eg &lt;a href=&quot;https://m.youtube.com/watch?v=joyi-ig6_Kw&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; &lt;a href=&quot;https://youtu.be/X2wvP2gI-fI&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;. Is it just vorticity? What's actually happening? (Similar: steam engines, volcanoes, clouds).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;examples are grid-based, using &quot;vorticity confinement&quot; in &lt;a href=&quot;https://www.chaosgroup.com/phoenix-fd/maya&quot; rel=&quot;nofollow noreferrer&quot;&gt;Phoenix FD&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; Some techniques: &quot;divergence control&quot; is used to increase the smoke concentration; and fuel particles in the air may combust, further adding smoke at that point (i.e. not at the initial source of the smoke).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A reference: Feldman 2003, &lt;em&gt;Animating Suspended Particle Explosions&lt;/em&gt;, ACM Trans. Graph. (Proc. SIGGRAPH) 22 (2003), 708-715&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-08-11T13:12:29.293" LastActivityDate="2017-08-11T13:12:29.293" Title="Physics of explosions: just vorticity?" Tags="|fluid-dynamics|computational-physics|navier-stokes|vorticity|" AnswerCount="0" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="27009" PostTypeId="1" CreationDate="2017-05-30T08:44:00.100" Score="3" ViewCount="2888" Body="&lt;p&gt;I have always been interested in generating discrete approximations that tend to mimic the properties of their continuum counterparts. Usually, one approximation that is desired is that the discrete solution is divergence free. Take Maxwell equations in the time domain as an example:&#xA;$$&#xA;-\frac{\partial\mathbf{b}(\mathbf{r},t)}{\partial t} = \nabla\times\mathbf{e}(\mathbf{r},t) \\&#xA;\mathbf{j}(\mathbf{r},t) = \nabla\times\mathbf{h}(\mathbf{r},t) \\&#xA;\nabla\cdot\mathbf{b}(\mathbf{r},t) = 0 \\&#xA;\nabla\cdot\mathbf{j}(\mathbf{r},t) = 0&#xA;$$&#xA;The third equation implies that only two out of the three components of $\mathbf{b}$ are independent of each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would like a more intuitive explanation of what it means to be divergent free. Even if the context is other than EM.&lt;/p&gt;&#xA;" OwnerUserId="24203" LastEditorUserId="11716" LastEditDate="2017-05-30T11:17:56.990" LastActivityDate="2017-05-31T13:32:03.127" Title="Intuitively, what does it mean for a solution to be divergence free?" Tags="|divergence-free|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27012" PostTypeId="1" CreationDate="2017-05-30T19:00:47.660" Score="2" ViewCount="71" Body="&lt;p&gt;I have just finished a new version of a parallel code that I have been working on. I would like to do a strong scaling test, but I am always bothered by how much monotonous work scaling tests seem to require.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I make folders for each processor count (np1, np2, np4, ...), copy my code into each folder, assign the correct parameters to the PBS file for submission to the cluster, and then submit the jobs one-by-one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After job completion, I copy the data from each folder onto my machine and peice-by-piece collect the timing data from each folder. Then it goes into MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am curious if any of you have developed more efficient ways to run scaling tests. Do you write scripts to do most of this for you? If so, how do you accomplish that? Bash scripting? Perl, perhaps?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="22717" LastActivityDate="2017-05-30T19:00:47.660" Title="Best way to conduct parallel scaling tests?" Tags="|scaling|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27014" PostTypeId="2" ParentId="26987" CreationDate="2017-05-31T02:03:57.130" Score="2" Body="&lt;p&gt;As mentioned by @jlk, what you want is a 3D visualization, or visualization of Three-dimensional data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most approaches for 3D &lt;a href=&quot;https://en.wikipedia.org/wiki/Scientific_visualization#Volume_visualization&quot; rel=&quot;nofollow noreferrer&quot;&gt;visualization&lt;/a&gt; fall into one of the following categories (from reference 1):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Slicing techniques: Using a cut plane probe the data to extract a two-dimensional slice of the data, and then use a two-dimensional spatial data visualization methods.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Isosurface&quot; rel=&quot;nofollow noreferrer&quot;&gt;Isosurface techniques&lt;/a&gt;: Given a user-specified value, generate a (or several) surface description and visualize it using surface visualization techiniques.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Volume_rendering#Direct_volume_rendering&quot; rel=&quot;nofollow noreferrer&quot;&gt;Direct volume rendering&lt;/a&gt;: Where an opacity function is assigned to values in the data, and you can see through it when rendered.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Sliceomatic uses the slicing option, and it seems useful for exploratory visualization. In the case of MATLAB you can check their &lt;a href=&quot;https://www.mathworks.com/help/matlab/volume-visualization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; for options. Some of them are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/slice.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;slice&lt;/code&gt;&lt;/a&gt; for slicing; and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/isosurface.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;isosurface&lt;/code&gt;&lt;/a&gt; for isosurfaces.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And it seems that &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/19155-volume-render&quot; rel=&quot;nofollow noreferrer&quot;&gt;Volume Render&lt;/a&gt; from the File Exchange can be used for the third option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to go serious with these visualizations maybe you should go with a dedicated software for it. You can use &lt;a href=&quot;https://www.paraview.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ParaView&lt;/a&gt;, or &lt;a href=&quot;http://docs.enthought.com/mayavi/mayavi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MayaVi&lt;/a&gt; with Python, among others.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Matthew O. Ward, Georges Grinstein, Daniel Keim (2015). Interactive Data Visualization: Foundayions, Techniques, and Applications. CRC Press.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-05-31T13:56:32.253" LastActivityDate="2017-05-31T13:56:32.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27015" PostTypeId="1" CreationDate="2017-05-31T08:08:48.107" Score="1" ViewCount="72" Body="&lt;p&gt;I'm currently developing simulations for &lt;strong&gt;coupled, nonlinear, multi-region&lt;/strong&gt; systems. Basically, I use the Finite Element Method (FEM) to model each physical quantity in each region. The obtained matrices and the coupling matrices between the quantities are combined into one large system matrix. From this matrix plus the derivatives of the nonlinear functions, a Jacobi matrix is generated. The system is then finally solved by the Newton-Raphson method. This includes the inversion of the Jacobi matrix, which is either done iteratively or directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not have a specific question, but I observe that I'm &lt;strong&gt;running into problems over and over again&lt;/strong&gt;. Bad convergence, instabilities, oscillations, to name some of them. It's proably because I have almost &lt;strong&gt;no theoretical background&lt;/strong&gt; in the field. &lt;strong&gt;FEM itself is not the problem, but the solution of the equations in a coupled and nonlinear scenario.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for some good material to get some basic theoretical understanding. &lt;strong&gt;What's a good introductory book or lecture about the topic?&lt;/strong&gt; (I'm a graduated engineer, so at least the basic maths is - hopefully - not the problem.)&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-05-31T08:08:48.107" Title="How to get a theoretical background in nonlinear, coupled FEM systems" Tags="|finite-element|nonlinear-equations|coupling|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27016" PostTypeId="2" ParentId="27009" CreationDate="2017-05-31T09:24:08.600" Score="4" Body="&lt;p&gt;I find the following two notions helpful.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The continuity equation&#xA;$$&#xA;\frac{\partial\rho}{\partial t} + \nabla \cdot (\rho\boldsymbol{u}) = \frac{\mathrm{d}\rho}{\mathrm{d}t} + \rho (\nabla \cdot \boldsymbol{u}) = 0,&#xA;$$&#xA;implies that a flow with divergence-free velocity $\boldsymbol{u}$ is &lt;strong&gt;incompressible&lt;/strong&gt; ($\mathrm{d}\rho/\mathrm{d}t=0$: the density $\rho$ remains unchanged along the flow). The matter must neither diverge (when $\rho$ would drop) nor converge (when $\rho$ would grow), but the motions of adjacent fluid elements must conspire to keep the density constant along the flow.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Gauß's theorem&#xA;$$&#xA;\int_{\partial V}\boldsymbol{f}\cdot\mathrm{d}\boldsymbol{\Omega}=&#xA;\int_V\,(\boldsymbol{\nabla}\cdot\boldsymbol{f})\,\mathrm{d}V&#xA;$$&#xA;implies that the total flux of a divergence-free field $\boldsymbol{f}$ through any closed surface vanishes. (Applying this to $\boldsymbol{f}=\rho\boldsymbol{u}$ shows that the continuity equation expresses matter conservation.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5472" LastEditorUserId="11716" LastEditDate="2017-05-31T13:32:03.127" LastActivityDate="2017-05-31T13:32:03.127" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27018" PostTypeId="1" AcceptedAnswerId="27020" CreationDate="2017-05-31T10:26:52.127" Score="0" ViewCount="3016" Body="&lt;p&gt;I want the product of two functions $f(x),g(x)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = x + \sin(x)\\&#xA;g(x) = x - \sin(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x)g(x) = x^2 - \sin^{2}(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I compute the product of two functions?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried searching through Google which says to use inline or something like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; f = @(x) x - sin(x)&#xA;f =&#xA;&#xA;@(x) x - sin (x)&#xA;&#xA;&amp;gt;&amp;gt; g = @(x) x + sin(x) &#xA;g =&#xA;&#xA;@(x) x + sin (x)&#xA;&#xA;&amp;gt;&amp;gt; h = @(x) f(x)*g(x)&#xA;h =&#xA;&#xA;@(x) f (x) * g (x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But could not get the product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, will the syntax change if I give the functions as the arguments of another function say $fun$ and will require products of functions inside the function $fun$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using MATLAB 2013a.&lt;/p&gt;&#xA;" OwnerUserId="23297" LastEditorUserId="9667" LastEditDate="2017-05-31T15:21:45.883" LastActivityDate="2017-05-31T15:29:28.840" Title="Multiplying two functions in MATLAB" Tags="|matlab|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27019" PostTypeId="1" CreationDate="2017-05-31T14:53:06.777" Score="0" ViewCount="1116" Body="&lt;p&gt;I'm solving the Dirichlet problem for the Poisson equation in a 2d domain $D$:&#xA;$$&#xA; \begin{cases}&#xA;    \Delta u = 0 \quad \text{in $D$}, \\&#xA;    u|_{\partial D} = u_0.&#xA; \end{cases}&#xA;$$&#xA;I'm interested in finding the fluxes of the solution through the boundary and in order to get accurate fluxes (after a discussion in comments to &lt;a href=&quot;https://scicomp.stackexchange.com/q/27002/1180&quot;&gt;this post&lt;/a&gt;) I decided to pass to a mixed formulation (see &lt;a href=&quot;https://fenicsproject.org/olddocs/dolfin/1.5.0/python/demo/documented/mixed-poisson/python/documentation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this fenics documentation page&lt;/a&gt;):&#xA;$$&#xA; \begin{cases}&#xA;   \nabla u = \sigma, \\&#xA;   \nabla \cdot \sigma = 0, \\&#xA;   u|_{\partial D} = u_0.&#xA;  \end{cases}&#xA;$$&#xA;The weak form consists of two equations&#xA;$$&#xA;   \int_D (\sigma \cdot \tau + \nabla \cdot \tau \; u )dS = \int_{\partial D} \tau \cdot n \; u_0 \, ds \quad \forall \tau \in H^1(div), \\&#xA;   \int_D \nabla \cdot \sigma \; v \; dS = 0 \quad \forall v \in L^2.&#xA;$$&#xA;Next, I consider a triangular finite element basis $\psi_i$ and I put $\sigma = (\sigma^i_x \psi_i, \sigma^j_y \psi_j)$, $\tau = (\tau^i_x \psi_i,\tau^j_y\psi_j)$, $u = u^i \psi_i$, $v = v^j \psi_j$ (where the summation signs are ommited). Putting these expressions in the above equations and taking into account that $\tau$, $v$ are arbitrary, I get the equations&#xA;$$&#xA;   \int_D \psi_i \psi_j dS \; \sigma_x^j + \int_D \partial_x \psi_i \psi_j dS \; u^j = \int_{\partial D} \psi_i n_x u_0 \, ds, \\&#xA;   \int_D \psi_i \psi_j dS \; \sigma_y^j + \int_D \partial_y \psi_i \psi_j dS \; u^j = \int_{\partial D} \psi_i n_y u_0 \, ds, \\&#xA;  \int_D \psi_i \partial_x \psi_j dS \; \sigma_x^j + \int_D \psi_i \partial_y \psi_j dS \; \sigma_y^j = 0,&#xA;$$ &#xA;where there is a summation over all indices $j$. Now I introduce the matrices&#xA;$$&#xA;  B_{ij} = \int_D \psi_i \psi_j \, dS, \; B_{x,ij} = \int_D \partial_x \psi_i \psi_j dS, \; B_{y,ij} = \int_D \partial_y \psi_i \psi_j \, dS,&#xA;$$&#xA;and the vectors&#xA;$$&#xA;   f_{x,i} = \int_{\partial D} \psi_i n_x u_0 \, ds, \; f_{y,i} = \int_{\partial D} \psi_i n_y u_0 \, ds.&#xA;$$&#xA;The discrete system can be formulated as&#xA;$$&#xA;  \left(\begin{array}{ccc}&#xA;   0 &amp;amp; B_x^T &amp;amp; B_y^T \\&#xA;   B_x &amp;amp; B &amp;amp; 0 \\&#xA;   B_y &amp;amp; 0 &amp;amp; B &#xA;\end{array}\right)&#xA;\left(\begin{array}{c} u \\ \sigma_x \\ \sigma_y \end{array}\right)&#xA;= \left(\begin{array}{c} 0 \\ f_x \\ f_y \end{array}\right).&#xA;$$&#xA;However, if I implement these formulas, I get some rubbish. Could you please point out what is wrong?&lt;/p&gt;&#xA;" OwnerUserId="1180" LastActivityDate="2017-06-01T08:54:24.140" Title="Mixed formulation of the Poisson equation (FEM)" Tags="|finite-element|pde|poisson|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27020" PostTypeId="2" ParentId="27018" CreationDate="2017-05-31T15:09:23.573" Score="1" Body="&lt;p&gt;If you want a purely numerical result the MATLAB trick is to use a dot &lt;code&gt;.&lt;/code&gt; like:  &lt;code&gt;f(x).*g(x)&lt;/code&gt; It takes care of pointwise multiplication for matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want it symbolically you can use something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms x&#xA;f = x + sin(x);&#xA;g = x - sin(x);&#xA;equ = simplify(f*g); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23834" LastEditorUserId="9667" LastEditDate="2017-05-31T15:29:28.840" LastActivityDate="2017-05-31T15:29:28.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27024" PostTypeId="2" ParentId="27019" CreationDate="2017-05-31T20:36:35.500" Score="3" Body="&lt;p&gt;For this problem you cannot use arbitrary finite elements. The finite element spaces should satisfy the so-called discrete inf-sup condition. You did not mention what elements you used so I assume that you used something that does not work like $P_1$ for each variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately there are lots of literature on elements that work. Keywords are &quot;Raviart-Thomas elements&quot; and &quot;Brezzi-Douglas-Marini elements&quot;. The monograph by Boffi-Brezzi-Fortin lists various finite element spaces for this particular problem.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2017-05-31T20:36:35.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27026" PostTypeId="2" ParentId="27019" CreationDate="2017-06-01T08:54:24.140" Score="1" Body="&lt;p&gt;In addition to @knl’s answer: you may want to check chapter 4 “Mixed Finite Elements” by &lt;a href=&quot;https://folk.uio.no/kent-and/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mardal&lt;/a&gt; (he is one of the developers of Dolfin which you mentioned) et al. of &lt;a href=&quot;http://www.diffpack.com/?f=book/comp_pde_advanced&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They discuss mixed Poisson problem, inf–sup stable FEs, and how it is implemented in Diffpack.&lt;/p&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-06-01T08:54:24.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27027" PostTypeId="1" AcceptedAnswerId="27031" CreationDate="2017-06-01T11:58:10.313" Score="5" ViewCount="646" Body="&lt;p&gt;Actually I am interested in analyzing the soution to the ODE given as $\frac{dy}{dx} = A + By + C\sin(y) , y(l) = m$ and check how the solution gets affected like whether they exist or not depending on the initial conditions .If they exist how the solutions look like and how they vary when we vary the parameters $A,B,C$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to visualize it through DESMOS as here - &lt;a href=&quot;https://www.desmos.com/calculator/ernu8p9mnw&quot; rel=&quot;noreferrer&quot;&gt;https://www.desmos.com/calculator/ernu8p9mnw&lt;/a&gt;&#xA;But could not give the entry properly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any online source which can help me in visualizing this ?&#xA;Also can this be done in MATLAB? like varying the parameters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is great. &lt;/p&gt;&#xA;" OwnerUserId="23297" LastActivityDate="2017-06-03T09:35:01.850" Title="Visualizing the solutions of the Differential equations by varying different parameters" Tags="|software|visualization|differential-equations|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27029" PostTypeId="2" ParentId="27027" CreationDate="2017-06-01T12:52:45.497" Score="4" Body="&lt;p&gt;You can use &lt;a href=&quot;http://app.juliadiffeq.org/ode&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl Online&lt;/a&gt; to visualize solutions to differential equations without a hassle. It's built using the Julia suite &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt;, and the online interface is a subset of features which includes explicit parameters and visualization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://app.juliadiffeq.org/ode;settings=eyJkaWZmRXFUZXh0IjoiZHkgPSBBICsgQip5ICsgQypzaW4oeSkiLCJwYXJhbWV0ZXJzIjoiQT0xLjAsQj0tMS4yLEM9MC41IiwidGltZVNwYW4iOlswLDEwXSwiaW5pdGlhbENvbmRpdGlvbnMiOiIxLjAiLCJzb2x2ZXIiOiJUc2l0NSIsInZhcnMiOiJbOnldIiwidGl0bGUiOiIifQ%3D%3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's an example of your equation&lt;/a&gt;, assuming that &lt;code&gt;l&lt;/code&gt; was the initial time point and this was not a BVP. If it is a BVP, currently the BVP solvers do not have an online interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note: I am the developer&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-06-01T17:55:41.837" LastActivityDate="2017-06-01T17:55:41.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27030" PostTypeId="1" CreationDate="2017-06-01T12:31:28.297" Score="1" ViewCount="67" Body="&lt;p&gt;I like to simulate the free-space propagation of an electric field (beam radius ~ 1-10 cm) over extremely short distances (~ 5-10 cm). The reason is that the field is focused by a lens with a very small focal distance (~ 30 cm). For larger distances, the FFT based method works but it fails for short distances due to aliasing (and I tried extremely high spatial resolutions without success). So I tried to compute the Rayleigh-Sommerfeld diffraction integral directly, but the highly oscillating term in it demands an extremely high spatial resolution as well once the propagation distance becomes too short. The only other method I can think of at the moment is a finite-differences solver for the wave equation itself. But before I try to implement it, I thought about asking if anyone has experience with this kind of problem? And maybe even some advice what I can try to solve it?&lt;/p&gt;&#xA;" OwnerUserId="7040" OwnerDisplayName="DaPhil" LastEditorUserId="7040" LastEditDate="2017-06-08T15:00:15.060" LastActivityDate="2017-06-08T15:00:15.060" Title="Beam propagation over extremely short distances" Tags="|fourier-transform|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27031" PostTypeId="2" ParentId="27027" CreationDate="2017-06-01T13:00:53.977" Score="8" Body="&lt;p&gt;You can try Geogebra (it is free). With SolveODE command and sliders you can do what yo want.&#xA;For the usage of SolveODE command &lt;a href=&quot;https://wiki.geogebra.org/en/SolveODE_Command&quot; rel=&quot;nofollow noreferrer&quot;&gt;see&lt;/a&gt;. For example by using following command&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;SolveODE[ &amp;lt;f'(x, y)&amp;gt;, &amp;lt;Start x&amp;gt;, &amp;lt;Start y&amp;gt;, &amp;lt;End x&amp;gt;, &amp;lt;Step&amp;gt; ]&lt;/code&gt; with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;SolveODE[A + B y + C sin(y), l, m, 10, 0.1]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got the solution curve below. You can vary the values of parameters A,B,C,l,m with sliders. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TgkOf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TgkOf.png&quot; alt=&quot;[1]: https://wiki.geogebra.org/en&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="4282" LastEditDate="2017-06-03T09:35:01.850" LastActivityDate="2017-06-03T09:35:01.850" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27032" PostTypeId="1" CreationDate="2017-06-01T18:23:06.747" Score="1" ViewCount="28" Body="&lt;p&gt;I have vehicles gps information in my real time traffic application. Averaging that information I know the speed of every road at any time. The problem is that is too much data to send back to the devices, so I need a way to group that speed data into classes so I can to simplify the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now I have something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; road_id   time   speed&#xA;    1      09:00   45&#xA;    1      10:00   47&#xA;    1      11:00   38&#xA;    2      09:00   78&#xA;    2      10:00   77&#xA;    2      11:00   61&#xA;    3      09:00   51&#xA;    3      10:00   48&#xA;    3      11:00   41&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;when close to 12pm more cars are in the roads so the speed decrease. &lt;/li&gt;&#xA;&lt;li&gt;road 1 and road 3 has a similar behavior probably both are regular street.  &lt;/li&gt;&#xA;&lt;li&gt;road 2 is faster, maybe a highway.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So instead of save the speed for each road for each hour. I just want define general road behavior for the day and say that road is that type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I will have a table for &lt;code&gt;roads&lt;/code&gt; and another for &lt;code&gt;road_type&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;road_id      name        type_id&#xA;  1        A street         1&#xA;  2        B Highway        2&#xA;  3        C street         1&#xA;&#xA;type_id  spd_09-10  spd_10-11 spd_11-12 ....&#xA;  1         50         50        40&#xA;  2         80         80        60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I need an algorithm to create those types efficiently and reducing the error with the original data. I understand if I increase the number of types the error will decrease. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is there is something already related to this problem just don't know how is called, and I don't want invent the wheel again.&lt;/p&gt;&#xA;" OwnerUserId="24438" LastEditorUserId="24438" LastEditDate="2017-06-01T18:32:26.297" LastActivityDate="2017-06-01T18:32:26.297" Title="How group similar data in a single classes and reduce the error" Tags="|algorithms|error-estimation|partitioning|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27034" PostTypeId="1" AcceptedAnswerId="27052" CreationDate="2017-06-02T11:31:59.110" Score="1" ViewCount="135" Body="&lt;p&gt;Im trying to compute a triple cross product of vectors a,b, and c in real space and integrate over the entire space. The result is a term in the hamiltonian for an electronic system so there are indices for the band number too (you can think of a and b as gradient of wavefunctions in real space and of c as a vector field):&#xA;$$&#xA;h_{nm} = \int \boldsymbol{a}_n(x)\times\boldsymbol{c}(x)\times\boldsymbol{b}_m(x)\,d x&#xA;%![equation](https://i.stack.imgur.com/VeGe3.gif)&#xA;$$&#xA;where $n$ and $m$ are indices from i.e. 0 to 50. I can write the triple product as&#xA;$$&#xA;\boldsymbol{A}\times\boldsymbol{B}\times\boldsymbol{C} = &#xA;\boldsymbol{B}(\boldsymbol{A}\cdot\boldsymbol{C})-\boldsymbol{C}(\boldsymbol{A}\cdot\boldsymbol{B})&#xA;%![eq](https://i.stack.imgur.com/aiEle.gif)&#xA;$$&#xA;which should be faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for the first term in python I'm doing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;N = 50   #  band index&#xA;v = 3    # direction x,y,z&#xA;g = 30000 # real space grid points&#xA;&#xA;a = np.ones((N,v,g), dtype = 'complex')&#xA;b = np.ones((N,v,g), dtype = 'complex')&#xA;c = np.ones((v,g), dtype = 'complex')&#xA;H_vnn = np.zeros((N,v,N), dtype = 'complex')&#xA;&#xA;for i in range(g):&#xA;         H_vnn += np.array([c[v,i]*(np.dot(a[:,:,i],b[:,:,i].T)) for v in range(3)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which, when doesnt run into memory error, takes forever. Im pretty sure this is not the most efficient way of doing it, but the only alternative I can think of is looping over the g index which seems to be even more expansive. Thank to everybody and particular to those who can help me&lt;/p&gt;&#xA;" OwnerUserId="24433" LastEditorUserId="24433" LastEditDate="2017-06-02T12:45:06.667" LastActivityDate="2017-06-05T08:18:24.130" Title="triple cross prouct of tensor" Tags="|python|computational-physics|eigenvalues|tensor|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27035" PostTypeId="2" ParentId="27027" CreationDate="2017-06-02T11:41:11.837" Score="1" Body="&lt;p&gt;Using recent Wolfram Cloud functionality and the code below you can do it online in your browser. I've deployed the app in the cloud:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.wolframcloud.com/objects/e087e0f0-fe3e-4b82-a7c2-7d668ec205d3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.wolframcloud.com/objects/e087e0f0-fe3e-4b82-a7c2-7d668ec205d3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. You need to have Wolfram ID to log in before you can use it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Manipulate[&#xA; Plot[&#xA;  Evaluate[NDSolveValue[{&#xA;    u'[t]==a+b u[t]+c Sin[u[t]] &#xA;    ,u[l]==m},u,{t,0,10}][t]&#xA;  ]&#xA;,{t,0,10}&#xA;,PlotRange-&amp;gt;All]&#xA;,{a,-10,10}&#xA;,{b,-10,10}&#xA;,{c,-10,10}&#xA;,{l,0,10}&#xA;,{m,-10,10}&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8Nw5l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Nw5l.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="720" LastEditorUserId="720" LastEditDate="2017-06-02T16:33:29.723" LastActivityDate="2017-06-02T16:33:29.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27036" PostTypeId="1" AcceptedAnswerId="27038" CreationDate="2017-06-02T13:47:17.547" Score="2" ViewCount="78" Body="&lt;p&gt;I have two finite sets of events $\{x_1, ..., x_N\}$ and $\{y_1, ..., y_N\}$ that are sampled from the PDFs $f(x)$ and $g(x)$, respectively, where $x \in [-1,+1]$. I want to estimate the Legendre expansion of $r(x) = f(x)/g(x)$ using these events. What is the best method? (Note that $g(x)\neq0$ over $[-1,1]$)&lt;/p&gt;&#xA;" OwnerUserId="24450" LastEditorUserId="24450" LastEditDate="2017-06-02T14:56:53.550" LastActivityDate="2017-06-02T18:14:17.213" Title="Legendre expansion of $r(x) = f(x)/g(x)$ using a finite number of samples from $f(x)$ and $g(x)$" Tags="|numerical-analysis|statistics|polynomials|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27037" PostTypeId="1" AcceptedAnswerId="27039" CreationDate="2017-06-02T17:26:28.603" Score="0" ViewCount="230" Body="&lt;p&gt;The algorthim use for updating the voltage of each componant at every time step is &#xA;$$v(t+\Delta t)=\begin{cases}&#xA;v(t)e^{-\gamma \Delta t},&amp;amp; n=0\\ v(t)e^{-\gamma \Delta t}+h_{*}&amp;amp; n=1\end{cases}$$&#xA;where $n$ denotes  the number of impulses arriving at a componant in the interval $(t,t+\Delta t).$ Here $ h_{*}=0.03, \gamma=20, \Delta t=10^{-5}.$ I have no idea how to use this algorithm in monte carlo simulation. More precisely , I have no idea about  the use of monte carlo simulation. Plz help&lt;/p&gt;&#xA;" OwnerUserId="24022" LastActivityDate="2017-06-04T06:02:17.987" Title="Monte Carlo Simulation algorithm" Tags="|monte-carlo|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27038" PostTypeId="2" ParentId="27036" CreationDate="2017-06-02T18:14:17.213" Score="1" Body="&lt;p&gt;I believe you are asking how to compute&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{f(x)}{g(x)}\approx\sum_{n=0}^p a_n P_n(x).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could interpolate your samples of $f$ and $g$ to a uniform grid on $[-1,1]$, calculate the ratios at each of these points, and then use the trapezoid rule to calculate the integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{-1}^1 P_n(x) \frac{f(x)}{g(x)} dx \approx \sum_j w_j P_n(x_j)\frac{f(x_j)}{g(x_j)},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\{w_j,x_j\}$ represents the quadrature weights and abscissae, respectively. Given that I don't know what the functions $f,g$ behave like, I can't say much for the accuracy of this approach beyond that it should be OK if $f/g$ is smooth.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2017-06-02T18:14:17.213" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27039" PostTypeId="2" ParentId="27037" CreationDate="2017-06-02T19:53:51.507" Score="2" Body="&lt;p&gt;It seems to me you would just be randomly selecting $n$ at each time step, given some distribution you should be sampling $n$ from, and using your dynamics function to propagate the voltage from time $t$ to time $t + \Delta t$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would then run multiple Monte Carlo simulations from the initial time to the final time where you basically select a random value for $n$ at each time step. Using the final results, you could compute statistical information like the expected value for the voltage at some time $\hat{t}$, the variance of your voltage at time $\hat{t}$, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you write a code for this and use the values for the constants the way you set them, you might produce the following type of graphic:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/urbq4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/urbq4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2017-06-03T15:10:29.133" LastActivityDate="2017-06-03T15:10:29.133" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27040" PostTypeId="1" AcceptedAnswerId="27067" CreationDate="2017-06-02T20:42:33.017" Score="4" ViewCount="159" Body="&lt;p&gt;In May, there was a question on Stackoverflow, &lt;a href=&quot;https://stackoverflow.com/questions/44069854/3-dimension-prediction-limit-in-r&quot;&gt;3 dimension prediction limit in R&lt;/a&gt;, a special form of the &quot;smallest enclosing rectangle&quot; problem. Because the problem has not been solved there, I would like to formulate it here as a problem in discrete optimization and hoping for an efficient algorithm to solve it exactly.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given N points in, say, two dimensions, find an axes-parallel rectangle&lt;br&gt;&#xA;  of minimal area that encloses at least 95% of the points.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand there's not always a solution, for example when all the points lie on the boundary of a rectangle. So let us assume the points are somehow in &lt;em&gt;general position&lt;/em&gt;, even such that no two points have exactly the same x- or y-coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that a/the solution can be found by looping through all x- and y-coordinates, resp. those below the 0.05 or above the 0.95 quantiles. Of course, I hope for a smarter and more efficient algorithmic approach.&lt;/p&gt;&#xA;" OwnerUserId="1084" LastActivityDate="2017-06-06T12:28:12.763" Title="A 95% minimal rectangle problem" Tags="|discrete-optimization|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27042" PostTypeId="1" AcceptedAnswerId="27059" CreationDate="2017-06-02T23:21:46.717" Score="0" ViewCount="119" Body="&lt;p&gt;I am hoping to start (very slowly) on implementing some form of a computational photonics/electromagnetism package. I know things like Meep, S4, FDTD++, EMPy, and a host of other proprietary/free/closed source tools exist, but this is more about hopefully getting a long-term project for myself that I can use as a multi-language, multi-level learning experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given this, I have a few questions, in decreasing order of how much they have stumped me so far. Direct references to libraries, frameworks, and/or books/papers that can help me gain insight into these questions are greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How does one implement geometry? How do I state the location, orientation, of my polygons, or circles, spheres, et al?&lt;/li&gt;&#xA;&lt;li&gt;How does one go about meshing? And giving physical qualities to the geometric objects that have been meshed? For example, how do I go about adding my permittivity to my objects? It can't all just be spamming &lt;code&gt;if-then-else&lt;/code&gt; statements that return epsilon, can it? If it is, how is this even implemented? Are all the &lt;code&gt;if-then-else&lt;/code&gt; statements dealt with at the very beginning before we start simulating?&lt;/li&gt;&#xA;&lt;li&gt;For a scalar finite difference, say, 2nd derivative, I've seen it being applied using a sparse matrix, and also as a convolution kernel. Are there any benefits of one over the other?&lt;/li&gt;&#xA;&lt;li&gt;Is there any way to apply vector derivatives (the curl, in particular) using either sparse matrices, or convolution kernels, or are loops the only way to go about this?&lt;/li&gt;&#xA;&lt;li&gt;Any other resources that one may have encountered in this particular area are welcome.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17947" LastActivityDate="2017-06-05T19:38:41.540" Title="Questions about implementing an electromagnetism/photonics solver package" Tags="|pde|c++|simulation|electromagnetism|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27046" PostTypeId="1" AcceptedAnswerId="27055" CreationDate="2017-06-04T02:54:15.130" Score="0" ViewCount="761" Body="&lt;p&gt;So I have a 3D mesh made of elements with eight nodes, 12 sides per cell, and in the course of my simulations, I would have to interpolate data from those nodes onto a point inside the cell with a given x, y and z value. I know Trilinear Interpolation does the job, but would there be a faster way to do the same, given that the shapes of each cell can be completely different?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is there a simpler, faster way for me to pinpoint the cell in which the given point resides, than looping over all the cells and using inequalities to find the bounds?&lt;/p&gt;&#xA;" OwnerUserId="23565" LastEditorUserId="23565" LastEditDate="2017-06-05T16:58:58.337" LastActivityDate="2017-06-07T21:08:25.570" Title="What would be a good, fast, way to interpolate a point in 3D space" Tags="|interpolation|3d|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27047" PostTypeId="2" ParentId="27046" CreationDate="2017-06-04T03:42:25.983" Score="0" Body="&lt;p&gt;I don't have an answer for your first question, but I have some thoughts for the latter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To efficiently search and find cells that enclose some point, you can discretize your 3D domain into a set of boxes using something like a spatial hash or quad tree, where each box holds a list of cells that intersect with that box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could then efficiently find what box in your data structure enclosed a given point and loop through the small number of cells that intersect with that box to find which encloses your point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the above approach, you can greatly reduce the number of cells you check to find one that matches, no matter how unstructured your mesh is.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-06-04T03:42:25.983" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27048" PostTypeId="2" ParentId="27037" CreationDate="2017-06-04T06:02:17.987" Score="0" Body="&lt;p&gt;If $\Delta t$ is small enough then $n$ can be only 0 or 1. Otherwise you must either reduce $\Delta t$ or change your formula to $v(t+\Delta t) = v(t)e^{-\gamma \Delta t} + n \times h_{*}$ (which by the way also works for $n=0$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if $n$ can be only 0 or 1, there is no notion of distribution: only of probability. The probability of $n$ being 1 is $p_0 \times \Delta t$ (to first order, so $\Delta t$ must be small). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then it is just a matter of drawing a (pseudo)random number between 0 and 1 (equiprobably): if it is smaller than $p_0 \times \Delta t$ then $n:=1$, otherwise $n:=0$. The probability of a random number between 0 and 1 being less than $p_0 \times \Delta t$ is... $p_0 \times \Delta t$, which is exactly the probability you want to set $n$ to 1.&lt;/p&gt;&#xA;" OwnerUserId="22855" LastActivityDate="2017-06-04T06:02:17.987" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27050" PostTypeId="1" AcceptedAnswerId="27056" CreationDate="2017-06-05T01:54:37.657" Score="4" ViewCount="901" Body="&lt;p&gt;I have a Hermite Cubic Finite Element Space on a computer in the form of Matlab m-files. More specifically, I can evaluate four &quot;shape functions&quot; $N_1, N_2, N_3,$ and $N_4$, for which the following holds on the boundary of the &quot;reference element&quot; $K_R= [-1,1]$: &#xA;\begin{align*}&#xA;N_1(-1) = 1, N_1(1) = 0, N_1'(-1) = 0, N_1'(1) = 0,\\&#xA;N_2(-1) = 0, N_2(1) = 1, N_2'(-1) = 0, N_2'(1) = 0,\\&#xA;N_3(-1) = 0, N_3(1) = 0, N_3'(-1) = 1, N_3'(1) = 0,\\&#xA;N_4(-1) = 0, N_4(1) = 0, N_4'(-1) = 0, N_4'(1) = 1.&#xA;\end{align*}&#xA;Here is what they look like:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/u8BNv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u8BNv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can map an arbitrary element $K = [a,b]$ to $K_R$, evaluate the shape functions over $K_R$, then &quot;map back&quot;, so that the above conditions on my shape functions hold on the boundary of arbitrary elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use this Finite Element Space to approximate $u(x) = \sin(2\pi x)$ for $x \in \Omega = [2,3]$. To do so, I complete the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Make an evenly spaced Mesh of the domain.&lt;/li&gt;&#xA;&lt;li&gt;Make relevant data structures needed to access the shape functions.&lt;/li&gt;&#xA;&lt;li&gt;Create an array of values of $u(x)$ evaluated at the Mesh nodes.&lt;/li&gt;&#xA;&lt;li&gt;Loop over each element: Extract the values of $u(x)$ at the Mesh node for this element $K$ with $u_1 = u(a)$ and $u_2 = u(b)$, where $a$ and $b$ are the endpoints of the element. Then do $u(x) \approx u_1 N_1(x) + u_2 N_2(x) + u_1 N_3(x) + u_2 N_4(x)$ for x = linspace(a,b,resolution). Then plot the result.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My problem is the Hermite Cubic Finite Element Space doesn't approximate $\sin(2 \pi x)$ very well using this method; the approximation wiggles a lot. Below is my plot using a Mesh with 100 elements, and using a resolution of 21. &lt;a href=&quot;https://i.stack.imgur.com/DqMDi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DqMDi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I repeated the same procedure as above except using a Lagrange Cubic Finite Element Space, so that the shape functions over $K_R$ satisfy&#xA;\begin{align*}&#xA;N_1(-1) = 1, N_1(-1/3) = 0, N_1(1/3) = 0, N_1(1) = 0, \\&#xA;N_2(-1) = 0, N_2(-1/3) = 1, N_2(1/3) = 0, N_2(1) = 0, \\&#xA;N_3(-1) = 0, N_3(-1/3) = 0, N_3(1/3) = 1, N_3(1) = 0, \\&#xA;N_4(-1) = 0, N_4(-1/3) = 0, N_4(1/3) = 0, N_4(1) = 1.&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what the Lagrange shape functions look like:&lt;a href=&quot;https://i.stack.imgur.com/ALWeh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ALWeh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $u_1, u_2, u_3, u_4$ corresponding to $-1, -1/3, 1/3, 1$, the approximation I used on each element was $u(x) \approx u_1 N_1(x) + u_2 N_2(x) +  u_3 N_3(x) +  u_4 N_4(x)$. Doing this gets me much nicer results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/J8GD9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/J8GD9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the deal? Why does the Lagrange FE space work so nice, and the Hermite FE space fail? This might be more of a &quot;numerical analysis&quot; question rather than a &quot;scientific computing&quot; question, but I thought I would try here first.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the comments suggested to use the formula&#xA;$u(x) \approx u_1 N_1(x) + u_2 N_2(x) + u_1' N_3(x) + u_2' N_4(x)$ instead of $u(x) \approx u_1 N_1(x) + u_2 N_2(x) + u_1 N_3(x) + u_2 N_4 (x)$. I agree that using this formula is what I should have done in this first place. I implement this by computing $u_1' = 2\pi \cos(2\pi a)$ and $u_2' = 2\pi \cos(2\pi b)$ for each element $K = [a,b]$. Then my Hermite approximation to $\sin(2 \pi x)$ looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3wr6l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3wr6l.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is clearly very wrong. In case it helps, here is the MATLAB code for my problem. I use a few of my own finite element computer programs here.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Make the mesh:&#xA;domain = [2,3]; nElement = 100;&#xA;Mesh = mesh_generator_1d(domain, nElement);&#xA;&#xA;% Form a data structure to access the finite element basis:&#xA;iDegree = 3; Fem = femherm1d(Mesh, iDegree); hDegree = 1;&#xA;&#xA;% Create an array with the first 100 entries corresponding to the values&#xA;% of sin(2*pi*x) evaluated at the Mesh nodes:&#xA;uGlobal = sin(2*pi*Fem.point(1:nElement+1)); iDerivative = 0;&#xA;&#xA;% Extend the array so that the last 100 entries correspond to the values of&#xA;% 2*pi*cos(2*pi*x) evaluated at the Mesh nodes:&#xA;uGlobal(nElement+2:2*(nElement+1)) = 2*pi*cos(2*pi*Fem.point(nElement+2:2*(nElement+1)));&#xA;&#xA;% Set the resolution and prepare the figure:&#xA;reso = 20; figure(1); clf; hold on;&#xA;uInterpolate = zeros(1,(reso+1)*size(Mesh.element,2));&#xA;&#xA;for k = 1:size(Mesh.element,2) % Loop over each element in the mesh&#xA;    element = Mesh.node(Mesh.element(:,k)); % Extract element endpoints&#xA;    uLocal = uGlobal(Fem.T(:,k)); % uLocal(1) = u1, uLocal(2) = u2, uLocal(3) = u1', uLocal(4) = u2'&#xA;    x = linspace(element(1), element(2), reso+1); % x contains the points we are evaluating the shape functions at&#xA;    uInterpolate(((k-1)*(reso+1)+1):(k*(reso+1))) = evalfeherm1d(x, uLocal, ...&#xA;        element, hDegree, iDerivative); % uInterpolate = u1*N1 + u2*N2 + u1'*N3 + u2'*N4&#xA;    plot(x,uInterpolate(((k-1)*(reso+1)+1):(k*(reso+1)))); % Plot the result&#xA;end&#xA;hold off&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2017-06-05T15:56:58.347" LastActivityDate="2019-07-12T15:55:03.153" Title="Interpolating a mathematical function using a Hermite Cubic Finite Element Space" Tags="|finite-element|numerical-analysis|interpolation|polynomials|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27051" PostTypeId="2" ParentId="27040" CreationDate="2017-06-05T08:09:27.723" Score="-1" Body="&lt;p&gt;Here is a sketch of an algorithm, though I have no proof that it always produces the minimum.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;goal = 0.95*points.size()&#xA;rect = smallest_rectangle_containing_all(points)&#xA;while(rect.number_of_points() &amp;gt; goal):&#xA;    old_rect = rect&#xA;    rect.maximally_shrink_by_omitting_at_least_one_point()&#xA;if(rect.number_of_points() &amp;lt; goal)&#xA;    rect = old_rect;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;rectangle::maximally_shrink_by_omitting_at_least_one_point()&lt;/code&gt; is a bit tricky to get right, but otherwise is straightforward (I leave its implementation to you).&lt;/p&gt;&#xA;" OwnerUserId="5472" LastActivityDate="2017-06-05T08:09:27.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27052" PostTypeId="2" ParentId="27034" CreationDate="2017-06-05T08:18:24.130" Score="0" Body="&lt;p&gt;Try this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;N = 50   #  band index&#xA;v = 3    # direction x,y,z&#xA;g = 30000 # real space grid points&#xA;&#xA;a = np.ones((g,N,v), dtype = 'complex')&#xA;b = np.ones((g,N,v), dtype = 'complex')&#xA;c = np.ones((g,v), dtype = 'complex')&#xA;H_vnn = np.zeros((N,v,N), dtype = 'complex')&#xA;&#xA;for i in range(g):&#xA;    adotb  = np.dot(a[i,:,:],b[i,:,:].T)&#xA;    H_vnn += np.array([c[i,v]*adotb) for v in range(3)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5472" LastActivityDate="2017-06-05T08:18:24.130" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27053" PostTypeId="1" CreationDate="2017-06-05T12:49:14.807" Score="4" ViewCount="1146" Body="&lt;p&gt;I'm currently developing a finite element solver for academic/research purposes. Therefore I'm searching for a pre- and postprocessor in my toolchain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a previous project I have used gmsh as a mesher and paraview as a postprocessor, but I'm not satisfied. &lt;strong&gt;Gmsh&lt;/strong&gt; works for meshing but I had my fair share of problems with it. A complete preprocessor which I could use for model generation, meshing and defining the boundary conditions would make it so much easier for me. &lt;strong&gt;Paraview&lt;/strong&gt;, unfortunately, doesn't support gauss points without modifications. I would rather put work into improving my solver than modifing &lt;strong&gt;Paraview&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During my search for alternatives I found &lt;strong&gt;Salome&lt;/strong&gt;, which is interesting but the search of a complete documentation with the specification of input and output formats yielded nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know good pre- and postprocessors to use with a custom finite element solver? I prefer freeware/open source but I'm open to commercial alternatives, which support some 'open' formats which I can save my results with.&lt;/p&gt;&#xA;" OwnerUserId="21418" LastEditorUserId="9667" LastEditDate="2017-06-05T14:37:37.993" LastActivityDate="2019-08-12T21:38:15.343" Title="Pre/Post-processor for an academic finite element solver" Tags="|finite-element|software|visualization|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27054" PostTypeId="2" ParentId="27053" CreationDate="2017-06-05T16:38:18.983" Score="4" Body="&lt;p&gt;Cubit/Trelis for generating meshes (Exodus.II files) and Paraview/Visit usually work well for low order FE codes. The advantage of Exodus.II is that it is a standard and therefore edge/side/face numbering etc. are consistent. E.g., see slide 121 &lt;a href=&quot;http://www.geology.wisc.edu/~stali/defmod/pres.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2019-08-12T21:38:15.343" LastActivityDate="2019-08-12T21:38:15.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="27055" PostTypeId="2" ParentId="27046" CreationDate="2017-06-05T17:00:05.480" Score="0" Body="&lt;p&gt;To locate the cell that contains a given node, I'd suggest using an AABB-Tree (see introduction in [1]). There is an implementation in the Geogram library [2] that I'm developping. There is also one in CGAL [3] (if you love C++ templates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have located the cell that contains the point, trilinear interpolation (as you suggest) is quite easy to implement (and efficient, does not cost a lot in terms of computation). In general, locating the cell that contains the point dominates computation time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit (one more thought):&lt;/em&gt; if the border of your cell is decomposed into triangles (as it seems to be since you got 12 faces per cell), then, to have an interpolation that is coherent with your geometry, it may make more sense to (virtually) decompose the cells into tetrahedra, determine the tetrahedron that contains the point and do linear interpolation in it. It costs more than trilinear interpolation, but trilinear interpolation supposes that faces are bi-linear quads (instead of pairs of linear triangles).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.azurefromthetrenches.com/introductory-guide-to-aabb-tree-collision-detection/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.azurefromthetrenches.com/introductory-guide-to-aabb-tree-collision-detection/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/mesh__AABB_8h.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/mesh__AABB_8h.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://doc.cgal.org/4.9/AABB_tree/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doc.cgal.org/4.9/AABB_tree/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-06-07T21:08:25.570" LastActivityDate="2017-06-07T21:08:25.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27056" PostTypeId="2" ParentId="27050" CreationDate="2017-06-05T17:43:09.700" Score="9" Body="&lt;h2&gt;Short answer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You are missing the Jacobian of the transformation for the derivatives.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Long answer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The conditions that you propose for your interpolator translate into the following system of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}&#xA;1 &amp;amp;x_1 &amp;amp;x_1^2 &amp;amp;x_1^3\\&#xA;1 &amp;amp;x_2 &amp;amp;x_2^2 &amp;amp;x_2^3\\&#xA;0 &amp;amp;1 &amp;amp;2x_1 &amp;amp;3x_1^2\\&#xA;0 &amp;amp;1 &amp;amp;2x_2 &amp;amp;3x_2^2&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;a_{11} &amp;amp;a_{12} &amp;amp;a_{13} &amp;amp;a_{14}\\&#xA;a_{21} &amp;amp;a_{22} &amp;amp;a_{23} &amp;amp;a_{24}\\&#xA;a_{31} &amp;amp;a_{32} &amp;amp;a_{33} &amp;amp;a_{34}\\&#xA;a_{41} &amp;amp;a_{42} &amp;amp;a_{43} &amp;amp;a_{44}&#xA;\end{bmatrix} = &#xA;\begin{bmatrix}&#xA;1 &amp;amp;0 &amp;amp;0 &amp;amp;0\\&#xA;0 &amp;amp;1 &amp;amp;0 &amp;amp;0\\&#xA;0 &amp;amp;0 &amp;amp;1 &amp;amp;0\\&#xA;0 &amp;amp;0 &amp;amp;0 &amp;amp;1&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt; are the extremes of the interval, and &lt;span class=&quot;math-container&quot;&gt;$a_{ij}$&lt;/span&gt; are the coefficients. If you want to compute the interpolation in your reference element and then use it outside of it you need to take into account the chain rule. The formula would be then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(x) \approx N_1(x) u_1 + N_2(x) u_2 + |J|(N_3(x) u'_1 + N_4(x) u'_2)\quad \forall x\in [a, b]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$|J| = (b - a)/2$&lt;/span&gt; the Jacobian determinant of the transformation, and, for &lt;span class=&quot;math-container&quot;&gt;$a=-1, b=1$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;N_1 (x) &amp;amp;= \frac{1}{4} (x - 1)^2 (2 + x)\\&#xA;N_2 (x) &amp;amp;= \frac{1}{4} (x + 1)^2 (2 - x)\\&#xA;N_3 (x) &amp;amp;= \frac{1}{4} (x - 1)^2 (x + 1)\\&#xA;N_4 (x) &amp;amp;= \frac{1}{4} (x + 1)^2 (x - 1)\, .&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following code computes and plots the interpolation for your function using just 3 elements.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;def hermite_interp(fun, grad, x0=-1, x1=1, npts=101):&#xA;    jaco = (x1 - x0)/2&#xA;    x = np.linspace(-1, 1, npts)&#xA;    f1 = fun(x0)&#xA;    f2 = fun(x1)&#xA;    g1 = grad(x0)&#xA;    g2 = grad(x1)&#xA;    N1 = 1/4*(x - 1)**2 * (2 + x)&#xA;    N2 = 1/4*(x + 1)**2 * (2 - x)&#xA;    N3 = 1/4*(x - 1)**2 * (x + 1)&#xA;    N4 = 1/4*(x + 1)**2 * (x - 1)&#xA;    interp = N1*f1 + N2*f2 + jaco*(N3*g1 + N4*g2)&#xA;    return interp&#xA;&#xA;&#xA;def fun(x):&#xA;    return np.sin(2*np.pi*x)&#xA;&#xA;&#xA;def grad(x):&#xA;    return 2*np.pi*np.cos(2*np.pi*x)&#xA;&#xA;&#xA;a = 2&#xA;b = 3&#xA;nels = 5&#xA;npts = 100&#xA;x = np.linspace(a, b, 100)&#xA;y = fun(x)&#xA;plt.plot(x, y, color=&quot;black&quot;)&#xA;xi = np.linspace(2, 3, num=nels, endpoint=False)&#xA;dx = xi[1] - xi[0]&#xA;for x0 in xi:&#xA;    x1 = x0 + dx&#xA;    x = np.linspace(x0, x1, npts)&#xA;    y = hermite_interp(fun, grad, x0=x0, x1=x1, npts=npts)    &#xA;    plt.plot(x, y, linestyle=&quot;dashed&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the following result&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vHMpv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vHMpv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2019-07-12T15:55:03.153" LastActivityDate="2019-07-12T15:55:03.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27057" PostTypeId="1" AcceptedAnswerId="27062" CreationDate="2017-06-05T19:04:50.223" Score="4" ViewCount="1578" Body="&lt;p&gt;What methods are known for efficiently solving a large set of linear homogeneous ordinary differential equations of the following form?&#xA;\begin{equation}&#xA;\mathbf{B} \frac{d\mathbf{y}}{dt} = \mathbf{A} \mathbf{y}, &#xA;\end{equation}&#xA;where $\mathbf{B} \in \mathbb{C}^{N\times N}$ is Hermitian positive definite, &#xA;$\mathbf{A} \in \mathbb{C}^{N\times N}$ is Hermitian, $\mathbf{y}\in \mathbb{C}^N$, and $N$ is a large integer such that not all the elements of the matrix $\mathbf{A}$ or $\mathbf{B}$ can be stored on the main memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method that avoids computing $\mathbf{B}^{-1}$, solving a linear equation $\mathbf{B} \mathbf{x} = \mathbf{c}$ for any $\mathbf{c}$, and diagonalizing $\mathbf{B}$? There seems to be such an 'inverse free' method [1] for the related eigenvalue problem, $\mathbf{B} \lambda \mathbf{y} = \mathbf{A}\mathbf{y}$, and I would like to know if there is an extension of such a method to ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] e.g. Gene H. Golub, Qiang Ye, An Inverse Free Preconditioned Krylov Subspace Method for Symmetric Generalized Eigenvalue Problems, SIAM J. Sci. Comp. 24, 312 (2002). &lt;/p&gt;&#xA;" OwnerUserId="14516" LastEditorUserId="18981" LastEditDate="2017-06-06T02:08:46.973" LastActivityDate="2022-06-02T09:49:54.003" Title="Set of linear ordinary differential equations with a mass matrix" Tags="|linear-algebra|numerical-analysis|ode|eigenvalues|iterative-method|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27058" PostTypeId="2" ParentId="27053" CreationDate="2017-06-05T19:36:44.173" Score="1" Body="&lt;p&gt;I think trelis/Cubit is one of the best in business of mesh generation. Last I heard they were even about to offer fully confirmed unstructured hex meshes. For post processing visit/paraview are well built but they have their own limitations like the one you mentioned with access to gaussian point operations. If you are very much into customizing, vtk's C++ or python library will give you a lot of tools to work with which gives you access to low level variables which you can modify and extract data the way you want and visualize them in the same pipeline of operations. They have very nice example page with all kinds of problems &lt;a href=&quot;http://www.vtk.org/Wiki/VTK/Examples/Cxx&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24069" LastActivityDate="2017-06-05T19:36:44.173" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27059" PostTypeId="2" ParentId="27042" CreationDate="2017-06-05T19:38:41.540" Score="3" Body="&lt;p&gt;Your questions suggest that you are new to implementing solvers for PDEs, and that you are not familiar with the usual data structures and algorithms used in this field. It would probably be very useful for you to see how other people write codes like the one you are trying to write, because you will see how they arrange data, how they represent geometry and materials, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at the tutorial programs that come with all of the major finite element codes, for example, to get this kind of experience. All of FEniCS, libMesh, and deal.II have extensive collections of tutorial programs that will show you many of the relevant techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I'm one of the principal authors of deal.II.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-06-05T19:38:41.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27062" PostTypeId="2" ParentId="27057" CreationDate="2017-06-06T02:08:26.893" Score="6" Body="&lt;p&gt;&lt;code&gt;B&lt;/code&gt; is known as a mass matrix. There are many methods which can directly solve with mass matrices. The main ones are implicit methods: implicit Runge-Kutta methods and the multistep BDF methods. Also Rosenbrock methods can handle mass matrices directly, but are not implicit methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick overview for &quot;why it works&quot; is because at every step these methods have to solve linear equations involving the Jacobian of the RHS. The linear problem is normally of the form $(I - \gamma J)x=b$ where $J$ is the system Jacobian, $\gamma$ and $b$ are some constants, and $x$ is the evolution of the system that must be solved for. Well, it turns out that if you just use the mass matrix $B$ by changing the system to $(B - \gamma J)x=b$, these directly solve the linear problem with the mass matrix, implicitly inverting $B$ in calculations that it would already have been doing (this is because the $I$ comes from the LHS when taking the Jacobian, but with a mass matrix instead that value is just $B$!). Doing this implementation with the non-adaptive 2nd order version of the Rosenbrock method &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/S1064827594276424&quot; rel=&quot;noreferrer&quot;&gt;is outlined in Shampine's paper&lt;/a&gt; and is not difficult, though you lose a lot of efficiency without adaptive timestepping here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use Julia, DifferentialEquations.jl has access to these methods. Just change &lt;code&gt;mass_matrix&lt;/code&gt; in the construction of the &lt;code&gt;ODEProblem&lt;/code&gt; type. The &lt;code&gt;Rosenbrock23()&lt;/code&gt; method will handle mass matrices like this. In addition, mass matrices can be passed to the implicit RK methods &lt;code&gt;radau()&lt;/code&gt; and &lt;code&gt;radau5()&lt;/code&gt; which are higher order and tend to be faster if you are looking to solve stiff problems with high accuracy. Technically Sundials' &lt;code&gt;CVODE_BDF()&lt;/code&gt; could solve with a mass matrix, but there's an open issue for setting up the C FFI for doing so (it's high on my priority list. Full disclosure: I am a developer of DifferentialEquations.jl).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can handle the performance drop of going to MATLAB instead, you'll find the field is very similar. &lt;code&gt;ode23s&lt;/code&gt; is a 2nd-order Rosenbrock method, is very similar to &lt;code&gt;Rosenbrock23()&lt;/code&gt;, and can use constant mass matrices. &lt;code&gt;ode15s&lt;/code&gt; is very similar to &lt;code&gt;CVODE_BDF()&lt;/code&gt; and can take mass matrices. MATLAB does not have an implicit RK method, though it does have &lt;code&gt;ode23t&lt;/code&gt; which is good for problems where a symplectic integrator is needed and it can handle mass matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not see options for passing mass matrices to SciPy's ODE solvers, and have not used the SciPy wrappers. Those wrap some of the same codes as DifferentialEquations.jl so I know that in some cases the mass matrix can be solved by the underlying C/Fortran codes that they use, but it looks like that part of the API is not exposed. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-06-06T02:37:02.247" LastActivityDate="2017-06-06T02:37:02.247" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27063" PostTypeId="1" AcceptedAnswerId="27071" CreationDate="2017-06-06T05:21:42.210" Score="3" ViewCount="206" Body="&lt;p&gt;Let's pretend we have a spatially discretized PDE of the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial^2 \boldsymbol{u}^k}{\partial t^2} = D\boldsymbol{u}^k&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D$ can be any form for now and $k$ refers to this being the discretization at time $t_k$. Then let's suppose, for example, we use a central difference approximation for the time derivative. We would then arrive at the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial^2 \boldsymbol{u}^k}{\partial t^2} &amp;amp;= D\boldsymbol{u}^k \\&#xA;\frac{\boldsymbol{u}^{k+1} - 2\boldsymbol{u}^k + \boldsymbol{u}^{k-1}}{\Delta t^2} &amp;amp;= D\boldsymbol{u}^k \\&#xA;\boldsymbol{u}^{k+1} &amp;amp;= \left( 2I + \Delta t^2 D \right)\boldsymbol{u}^k - \boldsymbol{u}^{k-1}\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the formulation above, how could one approach stability? First thoughts go to using Von Neumann stability analysis, but I have never seen it used in a vector-wise fashion such that one could take into account an arbitrary $D$ matrix. Any thoughts or references would be very useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The references provided in the comments were useful, but I found that only a few simple connections were needed to approach stability for this scenario. The key link was to take the expression above and cast it into a different difference equation with a more suitable form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more suitable case is by defining $\boldsymbol{w}^k = \left[(\boldsymbol{u}^{k})^{T}, (\boldsymbol{u}^{k-1})^{T}\right]^T$. We can than recast our difference equation into the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\boldsymbol{w}^{k+1} = G \boldsymbol{w}^k&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;G = \begin{bmatrix}&#xA;(2I + \Delta t^2 D) &amp;amp; -I \\&#xA;I &amp;amp; 0&#xA;\end{bmatrix}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we know this system is stable if, given the set of eigenvalues $\lbrace \lambda_i \rbrace$ for $G$, the following holds:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\max_{i} \left| \lambda_i\right| \lt 1&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With these results, we can check whether some differential operator $D$ is going to be stable with a given time discretization, particularly if it uses older $\boldsymbol{u}$ states.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="123" LastEditDate="2017-06-07T03:35:12.477" LastActivityDate="2017-06-07T03:43:57.037" Title="Stability of PDE Discretizations with Multistep Time Discretizations" Tags="|pde|numerical-analysis|stability|differential-equations|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27064" PostTypeId="2" ParentId="27040" CreationDate="2017-06-06T07:33:45.787" Score="0" Body="&lt;p&gt;You can try non-deterministic algorithms, such as simulated annealing. They are not sure to get the mathematically exact answer, but they can get a good approximation fairly fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $d$ is the dimension, then there are 2$d$ values to be set (locations of two opposite corners). &#xA;1. Start with corners at 2.5% and 97.5% (good approximations if the distribution is fairly well behaved). This is for 1D, otherwise $(1-0.95^{1/d})/2$ instead of 2.5%.&#xA;2. Pick two of these 2$d$ values, increase (resp. decrease) one of the two and decrease (resp. increase) the other to get back to 95%. If the area is smaller, keep it. Otherwise, apply your policy (e.g. simulated annealing: some probability to keep it that goes down with time).&#xA;3. Repeat.&lt;/p&gt;&#xA;" OwnerUserId="22855" LastActivityDate="2017-06-06T07:33:45.787" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27066" PostTypeId="1" AcceptedAnswerId="27085" CreationDate="2017-06-06T09:26:38.333" Score="4" ViewCount="1469" Body="&lt;p&gt;Let $A$ be a $n\times n$ square symmetric matrix. In addition, $A\succeq0$ and $\mathrm{rank}(A)&amp;lt;n$. This means that all eigenvalues are non-negative, but also that there are some zero eigenvalues. I want to perform quadratic  optimization of the functional&#xA;$$&#xA;\frac{1}{2}x^HAx+b^Hx&#xA;$$&#xA;under convex conditions $x_i\geq0$ and $y^Hx=0$, where $b,y$ some fixed vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem arises when, due to numerical rounding errors, the matrix has some very small negative spurious eigenvalues. Then, then functional becomes unbounded. I have tried this with &lt;a href=&quot;http://sedumi.ie.lehigh.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeDuMi&lt;/a&gt; and with matrix &#xA;$$&#xA;A=\begin{bmatrix}10&amp;amp;17&amp;amp;25&amp;amp;-5&amp;amp;-9\\&#xA;17   &amp;amp; 29 &amp;amp; 43 &amp;amp; -9 &amp;amp;  -16\\&#xA;25 &amp;amp; 43 &amp;amp; 65 &amp;amp;-15 &amp;amp;-26\\&#xA;-5 &amp;amp; -9 &amp;amp; -15 &amp;amp;  5 &amp;amp;  8\\&#xA;-9 &amp;amp; -16 &amp;amp; -26 &amp;amp; 8 &amp;amp; 13&#xA;\end{bmatrix},\quad&#xA;b=-\begin{bmatrix}1\\1\\1\\1\\1&#xA;\end{bmatrix},\quad&#xA;y=\begin{bmatrix}&#xA;1\\1\\1\\-1\\-1&#xA;\end{bmatrix}&#xA;$$&#xA;If you calculate the rank with MATLAB you will get $\mathrm{rank}(A)=2$. However, the numerical calculation with the help of the &lt;code&gt;eig()&lt;/code&gt; function gives following results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; eig(A)&#xA;&#xA;ans =&#xA;&#xA; -1.6661e-014&#xA; -4.4496e-016&#xA;  4.2249e-015&#xA;       5.0536&#xA;       116.95&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;SeDuMi (via &lt;a href=&quot;https://yalmip.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yalmip&lt;/a&gt;, excuted in Matlab 2007b) gives following error message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Exiting: the solution is unbounded and at infinity;&#xA; the constraints are not restrictive enough.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do you have any idea, how to effectively address this problem numerically? I have tried diagonalizing $A$ and replacing the small negative eigenvalues with arbitrary small positive numbers, but this seems, well ... arbitrary, and I fear that it might produce numerical errors in the optimization. What do you think?&lt;/p&gt;&#xA;" OwnerUserId="24486" LastEditorUserId="20417" LastEditDate="2017-06-07T16:11:04.330" LastActivityDate="2017-07-05T13:14:31.353" Title="Quadratic programs with rank deficient positive semidefinite matrices" Tags="|linear-algebra|optimization|convex-optimization|quadratic-programming|rank|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="27067" PostTypeId="2" ParentId="27040" CreationDate="2017-06-06T12:28:12.763" Score="2" Body="&lt;p&gt;Your problem reads&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given $N$ points in, say, two dimensions, find an axes-parallel rectangle&#xA;   of minimal area that encloses at least 95% of the points.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is essentially equivalent to the following problem (the &lt;em&gt;smallest $k$-point enclosing rectangle&lt;/em&gt;), for which efficient deterministic algorithms are known:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given $N$ points in two dimensions and an integer $k \le N$, find an axes-parallel square or rectangle of minimal area that encloses $k$ of the points.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Some references:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0020019097002123&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Enclosing $k$ points in the smallest axis parallel rectangle&lt;/em&gt;&lt;/a&gt; - Segal and Kedem, &lt;em&gt;Information Processing Letters&lt;/em&gt;, 1998&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://pubman.mpdl.mpg.de/pubman/item/escidoc:1834660:2/component/escidoc:1857755/MPI-I-93-116.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Finding $k$ points with a smallest enclosing square&lt;/em&gt;&lt;/a&gt; - Smid, 1995.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0020019005000621&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Smallest $k$-point enclosing rectangle and square of arbitrary orientation&lt;/em&gt;&lt;/a&gt; - Das et al., &lt;em&gt;Information Processing Letters&lt;/em&gt;, 2005&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2017-06-06T12:28:12.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27070" PostTypeId="1" AcceptedAnswerId="27074" CreationDate="2017-06-07T02:21:44.850" Score="0" ViewCount="108" Body="&lt;p&gt;I am modelling the time-evolution of a soft material with a tetrahedral mesh. I use an FEM method to compute the forces on each node, and then numerically integrate the positions and velocity of the nodes over time. The basic equation is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$MU^{\prime\prime} + C^{\prime\prime} + F(U) = R$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $M$ and $C$ are the mass and damping matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now need to add constraints to prevent penetration with other objects and also friction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can’t see the connection between my current approach and the various “contact” methods I have seen in books, papers etc. I think I need to add Lagrangian terms to deal with the inequality constraint, but am getting lost in the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest an introductory book or paper that can help me get started?&lt;/p&gt;&#xA;" OwnerUserId="24493" LastEditorUserId="20688" LastEditDate="2017-06-07T13:48:28.157" LastActivityDate="2017-06-07T13:48:28.157" Title="Newbie help on FEM contact problem" Tags="|finite-element|reference-request|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27071" PostTypeId="2" ParentId="27063" CreationDate="2017-06-07T03:38:01.270" Score="3" Body="&lt;p&gt;What you're looking for is a discussion of absolute stability of multistep methods for ODEs. See for example chapter 7 of LeVeque's book on finite difference methods . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you know the region of absolute stability for your time discretization, it's just a matter of checking that the eigenvalues of D (scaled by the time step) are inside that region. This is typical method-of-lines stability analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach you've mentioned in your edit is equivalent, but in a sense less flexible since it doesn't separate the analysis of the time and space discretizations so clearly.   Also, the approach I'm recommending can be done entirely with pencil and paper. &lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2017-06-07T03:43:57.037" LastActivityDate="2017-06-07T03:43:57.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27072" PostTypeId="1" CreationDate="2017-06-07T09:53:17.347" Score="2" ViewCount="1069" Body="&lt;p&gt;Reading Nocedal's book on optimization, I came into the concept of local minimizer, which is a well-known concept in numerical optimization. However, I think I am having a rough time trying to come up with an intuitive understanding of the different types of local minimizer. Let us recall the following three definitions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Def&lt;/strong&gt;. A point $x^*$ is a &lt;strong&gt;weak local minimizer of $f$&lt;/strong&gt; if there is a neighborhood $\mathcal{N}$ of $x^*$ such that $f(x^*) \leq f(x)$ for all $x \in \mathcal{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Def&lt;/strong&gt;. A point $x^*$ is a &lt;strong&gt;strong local minimizer of $f$&lt;/strong&gt; if there is a neighborhood $\mathcal{N}$ of $x^*$ such that $f(x^*) &amp;lt; f(x)$ for all $x \in \mathcal{N}$ with $x \neq x^*$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Def&lt;/strong&gt;. A point $x^*$ is an &lt;strong&gt;isolated local minimizer of $f$&lt;/strong&gt; if there is a neighborhood $\mathcal{N}$ of $x^*$ such that $x^*$ is the only local minimizer in $\mathcal{N}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, a weak local minimizer is a point that achieves the smallest value of $f$ in its neighborhood. A strong local minimizer can be thought off as (to quote Nocedal et al.) &quot;the outright winner in its neighborhood&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am failing to see an intuition behind the isolated local minimizer. How is a strong local minimizer not an isolated one, if in the neighborhood, it is the only minimizer?&lt;/p&gt;&#xA;" OwnerUserId="24203" LastEditorUserId="24110" LastEditDate="2017-06-08T19:47:10.880" LastActivityDate="2017-06-08T19:47:10.880" Title="What is the most intuitive explanation for the concepts of weak local minimizer, strong local minimizer and isolated local minimizer?" Tags="|optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27074" PostTypeId="2" ParentId="27070" CreationDate="2017-06-07T10:34:48.293" Score="-1" Body="&lt;p&gt;Google for the “Forward Increment Lagrange Multiplier Method” of&#xA;Carpenter et al. [1991].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For implementation, see the code &lt;a href=&quot;https://bitbucket.org/stali/defmod/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (especially lines 463-497 in src/main.F90).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you also have friction then see the modified version &lt;a href=&quot;https://doi.org/10.1016/j.cageo.2016.11.014&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (implementation &lt;a href=&quot;https://bitbucket.org/chunfangmeng/defmod-dev&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (deprecated) and &lt;a href=&quot;https://github.com/chunfang/defmod-swpc&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="471" LastEditorUserId="471" LastEditDate="2017-06-07T10:40:46.350" LastActivityDate="2017-06-07T10:40:46.350" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27075" PostTypeId="1" CreationDate="2017-06-07T11:24:54.370" Score="0" ViewCount="871" Body="&lt;p&gt;I would like to perform post-optimal analysis using Matlab &lt;em&gt;linprog&lt;/em&gt;. But it does not provide any information about that. So required a way to get the info about optimal basis, basic and non-basic variables via the solution of &lt;em&gt;interior-point&lt;/em&gt; or &lt;em&gt;dual-simplex&lt;/em&gt; algorithm of &lt;em&gt;linprog&lt;/em&gt;. &lt;/p&gt;&#xA;" OwnerUserId="24215" LastActivityDate="2017-07-08T17:27:43.420" Title="How to perform linear programming sensitivity analysis in MATLAB?" Tags="|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27076" PostTypeId="2" ParentId="27066" CreationDate="2017-06-07T13:56:29.050" Score="0" Body="&lt;p&gt;The following &lt;a href=&quot;http://www.cvxpy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVXPY&lt;/a&gt; script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from cvxpy import *&#xA;import numpy as np&#xA;&#xA;# optimization variables&#xA;x = Variable(5)&#xA;&#xA;# matrix A and vectors b and c&#xA;A = np.array([[10, 17, 25, -5, -9],&#xA;              [17, 29, 43, -9,-16],&#xA;              [25, 43, 65,-15,-26],&#xA;              [-5, -9,-15,  5,  8],&#xA;              [-9,-16,-26,  8, 13]])&#xA;b = np.ones(5)&#xA;c = np.array([ 1, 1, 1,-1,-1])&#xA;&#xA;&#xA;# build optimization problem&#xA;objective = Minimize( 0.5 * quad_form(x, A) - b.T * x )&#xA;constraints = [ c.T * x == 0, x &amp;gt;= 0 ]&#xA;prob = Problem(objective, constraints)&#xA;&#xA;# solve optimization problem&#xA;prob.solve()&#xA;print &quot;x =&quot;, x.value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;produces the following vector&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = [[  4.44444306e-01]&#xA;     [  2.85606990e-11]&#xA;     [  1.71645999e-10]&#xA;     [  2.22221718e-01]&#xA;     [  2.22222588e-01]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which does satisfy the constraints. Perhaps the constraints are indeed &quot;restrictive enough&quot;.&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2017-07-05T13:14:31.353" LastActivityDate="2017-07-05T13:14:31.353" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27077" PostTypeId="1" AcceptedAnswerId="27078" CreationDate="2017-06-07T15:03:35.330" Score="1" ViewCount="828" Body="&lt;p&gt;I have written some Python code to determine the numerical roots of the following non-linear equation:&#xA;$$f_m=\tan\lambda_m - \frac{\lambda_m}{1+a}$$&#xA;where $\lambda_m\gt0$ and $a\geq0$. The code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.optimize import root&#xA;&#xA;func = lambda λm, a: np.tan(λm) - λm/(1+a)&#xA;λm0 = np.linspace(np.pi, m*np.pi, m) # initial guess&#xA;sol = root(func, λm0, args=(a), method='lm') # least-square optimization&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works fine, e.g. for input $m, a = 3, 1$ the result is:&#xA;$$\lambda_m=[4.27478227,   7.59654602,  10.81267333]$$&#xA;which are the roots to $f_m=0$ for $m=1,2,3$ as can be checked easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a next step, I would like to provide the Jacobian, which i think should be:&#xA;$$J_{m,n}=\frac{\partial f_{m}}{\partial\lambda_{n}}=\begin{cases}&#xA;\frac{1}{\cos^2\lambda_{m}}-\frac{1}{1+a} &amp;amp; m=n\\&#xA;0 &amp;amp; m\neq n&#xA;\end{cases}$$&#xA;i.e. in matrix-form with the above values for $\lambda_m$:&#xA;$$\boldsymbol{J}=\begin{bmatrix} 5.06844087 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 14.92687786 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 29.72847616&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the approximated Jacobian being output by the optimization routine is:&#xA;$$\boldsymbol{\tilde{J}}=\begin{bmatrix} -29.72850523 &amp;amp; 0 &amp;amp; 1\\&#xA;0 &amp;amp; -14.92688569 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 5.06844398&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly the magnitudes of approximated derivatives on the diagonal correspond to my analytical values for the Jacobian, but the order of the values is somehow different (running backwards from $n=3,2,1$) and some of the values are negative. Furthermore, an additional $\tilde{J}_{1,3}=1$ is present (more of these spurious ones seem to appear as $n$ grows) which shouldn't be present. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain the differences between the approximate and analytical Jacobian?&lt;/p&gt;&#xA;" OwnerUserId="18017" LastEditorUserId="18017" LastEditDate="2017-06-07T15:46:55.450" LastActivityDate="2017-06-07T18:56:14.550" Title="Confusion about determining the jacobian in a rootfinding algorithm" Tags="|python|scipy|roots|jacobian|rootfinding|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27078" PostTypeId="2" ParentId="27077" CreationDate="2017-06-07T18:28:56.507" Score="2" Body="&lt;p&gt;If you look at the full output of your script, &lt;code&gt;sol&lt;/code&gt; has fields &lt;code&gt;ipvt&lt;/code&gt; and &lt;code&gt;qtf&lt;/code&gt;. Both of these fields come from MINPACK (&lt;a href=&quot;http://www.netlib.org/minpack/lmdif.f&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.netlib.org/minpack/lmdif.f&lt;/a&gt;), which is the backend here (&lt;a href=&quot;https://github.com/scipy/scipy/blob/2526df72e5d4ca8bad6e2f4b3cbdfbc33e805865/scipy/optimize/_root.py#L246&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/scipy/scipy/blob/2526df72e5d4ca8bad6e2f4b3cbdfbc33e805865/scipy/optimize/_root.py#L246&lt;/a&gt;; &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.optimize.leastsq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.optimize.leastsq.html&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first, &lt;code&gt;ipvt&lt;/code&gt; explains what happened to variable ordering—there was a pivoting step when dealing with the Jacobian and when I ran your code, the pivots were $[3,2,1]$, matching the permutation of the matrix. The second, &lt;code&gt;qft&lt;/code&gt; is given in the documentation as &lt;code&gt;(q transpose)*fvec&lt;/code&gt;, which in my case is &lt;code&gt;[ -2.52376384e-07,  -1.46294147e-07,   5.03946448e-07]&lt;/code&gt;. This explains the sign changes for the first two entries, but the magnitudes are off by $10^7$—I don't know why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One other thing I noticed is that &lt;code&gt;lmdif&lt;/code&gt; doesn't produce the actual Jacobian—note that the field is called &lt;code&gt;fjac&lt;/code&gt; (as in lmdif), not &lt;code&gt;jac&lt;/code&gt; (&lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.optimize.OptimizeResult.html#scipy.optimize.OptimizeResult&quot; rel=&quot;nofollow noreferrer&quot;&gt;as in scipy&lt;/a&gt;). Quote:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c       fjac is an output m by n array. the upper n by n submatrix&#xA;c         of fjac contains an upper triangular matrix r with&#xA;c         diagonal elements of nonincreasing magnitude such that&#xA;c&#xA;c                t     t           t&#xA;c               p *(jac *jac)*p = r *r,&#xA;c&#xA;c         where p is a permutation matrix and jac is the final&#xA;c         calculated jacobian. column j of p is column ipvt(j)&#xA;c         (see below) of the identity matrix. the lower trapezoidal&#xA;c         part of fjac contains information generated during&#xA;c         the computation of r.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I haven't tested this, but there would probably be even more inconsistency if you tried a coupled system of equations.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-06-07T18:56:14.550" LastActivityDate="2017-06-07T18:56:14.550" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27079" PostTypeId="2" ParentId="26963" CreationDate="2017-06-08T04:37:51.737" Score="1" Body="&lt;p&gt;Is your question just regarding the final assembly of the system into a global one then you can find the answer here &#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/26915/fem-assembly-on-unstructured-meshes&quot;&gt;FEM assembly on unstructured meshes&lt;/a&gt;&#xA;although implementation is in c++ it is very easily translatable into Matlab. &lt;/p&gt;&#xA;" OwnerUserId="23149" LastActivityDate="2017-06-08T04:37:51.737" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27080" PostTypeId="1" CreationDate="2017-06-08T10:56:27.143" Score="2" ViewCount="219" Body="&lt;p&gt;I have a system of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A(u)f(u)=b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is basically a matrix originating from the Finite Element Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I try to solve it using the Newton method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R = A(u_{i}) f(u_{i}) - b $$&#xA;$$u_{i+1}=u_{i} - J^{-1}R$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $J$ is the Jacobian of the residuum $R$ with respect to $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At a first glance, it works fine. The residuum drops over several orders of magnitudes within just a few iterations. It seems that it converges to an error small enough for my application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But still - after a couple of iterations, it stagnates/oscillates at a small error. I wonder whether this is normal or whether I should suspect a bug in my code.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I found out so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The remaining error is around $10^{-7}$ with entries in $J$ around $10^{-8}$ and in $u$ around $10^{3}$. I suspect that this is not yet extreme enough to cause problems with limited floating point precision.&lt;/li&gt;&#xA;&lt;li&gt;The remaining error becomes smaller when the derivatives $dA/du$ and $df/du$ become larger.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm of course happy with an answer to my problem, but I'm also happy with some further reading on the topic.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-06-09T14:46:28.417" Title="Newton's method stagnates at small error" Tags="|finite-element|nonlinear-equations|newton-method|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27082" PostTypeId="2" ParentId="27080" CreationDate="2017-06-08T13:33:33.280" Score="1" Body="&lt;p&gt;I'd suspect a bug in the code unless you understand exactly why the seen behavior is expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for a particular references on how to write nonlinear solvers (globally convergent newton solvers), I was very happy working with:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Newton Methods for Nonlinear Problems,&lt;br&gt;&#xA;Affine Invariance and Adaptive Algorithms;&lt;br&gt;&#xA;Deufelhard, Peter; Springer, 2006 (Second Printing)&lt;br&gt;&#xA;ISBN 978-3-642-23898-7 (Softcover)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reference implementations for (1),&lt;br&gt;&#xA;&lt;a href=&quot;http://elib.zib.de/pub/elib/codelib/NewtonLib/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://elib.zib.de/pub/elib/codelib/NewtonLib/index.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A Family of Newton Codes for Systems of Highly Nonlinear Equations (1991);&#xA;Nowak, U., Weimann, L.;&#xA;?,&lt;br&gt;&#xA;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.43.3751&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.43.3751&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The NLEQ_ERR solver outlined in (1),(2) works very well for my applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another nice book with simple non-linear examples (including the (tangent) stiffness matrices of each newton step is):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Advanced Topics in Finite Element Analysis of Structures: With&#xA;Mathematica and MATLAB Computations; Bhatti M. Asghar; Wiley; ISBN:&#xA;978-0-471-64807-9&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="11893" LastEditDate="2017-06-09T14:46:28.417" LastActivityDate="2017-06-09T14:46:28.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27084" PostTypeId="1" CreationDate="2017-06-08T14:27:00.540" Score="4" ViewCount="136" Body="&lt;p&gt;Specifically, the diagonal elements (can possibly both positive and negative) of the matrix can be computed efficiently but the total number is large ($\mathcal O(10^{18})$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thought about this was running an MCMC (Markov Chain Monte-Carlo) with the weight set to be the absolute value of each element; however, this just exactly leads to the sign problem. But I'm still wondering if there is any smart way to tackle that. &lt;/p&gt;&#xA;" OwnerUserId="24512" LastEditorUserId="20688" LastEditDate="2018-04-01T00:37:33.703" LastActivityDate="2018-04-01T00:37:33.703" Title="How to stochastically estimate the trace of a matrix?" Tags="|linear-algebra|matrix|monte-carlo|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="27085" PostTypeId="2" ParentId="27066" CreationDate="2017-06-08T16:06:32.837" Score="5" Body="&lt;p&gt;To ensure this does not drown in the comments, I make it an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solver used is not SeDuMi, as claimed in the question. The solver used is quadprog, and that solver (or more specifically, a severely outdated version of it), apparently had numerical issues on this particular instance. A recent version of quadprog, or any reasonably robust solver, solves this problem without issues.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastEditorUserId="1662" LastEditDate="2017-06-08T16:38:13.430" LastActivityDate="2017-06-08T16:38:13.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27086" PostTypeId="2" ParentId="27075" CreationDate="2017-06-08T16:53:24.193" Score="2" Body="&lt;p&gt;The fifth output from linprog are the dual variables, which you can use for sensitivity analysis.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2017-06-08T16:53:24.193" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27087" PostTypeId="2" ParentId="27072" CreationDate="2017-06-08T17:48:06.290" Score="3" Body="&lt;p&gt;I'll quote Remark 1.1 from &lt;a href=&quot;https://www.math.purdue.edu/~wang838/notes/HW/cs520_review.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Isolated&quot; implies &quot;strict&quot;, but the inverse is not true. For example, let $f(x)$ be&#xA;  \begin{equation}&#xA;f(x) = \begin{cases} 0, &amp;amp; x = 0, \\&#xA;x^4 (\cos (1/x) + 2), &amp;amp; x \ne 0.&#xA;\end{cases}&#xA;\end{equation}&#xA;  Here, $x^\ast = 0$ is &quot;strict&quot; but not &quot;isolated&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &quot;strict minimizer&quot; used in the book corresponds to the strong minimizer according to your definition. The point $x^\ast$ is the global minimum, but there are other local minima in every of its neighborhood.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2017-06-08T17:48:06.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27088" PostTypeId="1" AcceptedAnswerId="27106" CreationDate="2017-06-08T18:11:59.680" Score="6" ViewCount="487" Body="&lt;p&gt;I have the biharmonic equation on a 2D rectangular domain $\Omega$ with the following boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta^2 u = f$ on $\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla u \bullet \mathbf{n}=0$ on $\partial \Omega$ (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{xy} = 0$ on $\partial \Omega$ (2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need the weak form of the equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiar with Nitsche's method and I can impose the first condition, if the second condition is a Dirichlet condition (e.g. $u=0$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_\Omega \Delta u \Delta v \mathrm{dx} -\int_{\partial \Omega } \Delta u \nabla v \ \bullet \mathbf{n} ds - \int_{\partial \Omega } \nabla u \bullet \mathbf{n} \Delta v ds +\eta \int_{\partial \Omega } \nabla u \bullet \mathbf{n} \nabla v \bullet \mathbf{n} = \int_{\Omega} fvdx$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{n}$ is the element normal and $\eta &amp;gt; 0$ is a penalty parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 1: Can I impose the $u_{xy} = 0$ somehow in a similar manner? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 2: What if I want to impose $u_x = 0$ instead of $\nabla u \bullet \mathbf{n}=0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In specific: I have a plate bending problem and I would like to compute only the quarter of the domain. Therefore I would like to impose symmetry boundary conditions on the blue dashed boundaries.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3zBad.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3zBad.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14907" LastEditorUserId="14907" LastEditDate="2017-06-12T21:32:50.770" LastActivityDate="2017-06-12T21:32:50.770" Title="How to impose boundary condition with mixed derivatives?" Tags="|boundary-conditions|elliptic-pde|nitsche-method|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27089" PostTypeId="2" ParentId="27053" CreationDate="2017-06-09T05:03:00.337" Score="1" Body="&lt;p&gt;In our research group we use &lt;a href=&quot;https://www.gidhome.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GiD &lt;/a&gt;. It has what you are looking for, a pre-processor with a nice geometry editor (2D and 3D) and a very good mesher. It generates preprocessing files using a flexible language, it can run your solver automatically and read the result. GiD also has a very functional post-processor with many options to display results, it can even be used to generate videos for dynamic problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything is well integrated and works in Windows, Linux and Mac OS X. The documentation is also good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can test it for free for several months, the single license cost is between 500-1600 euros (&lt;a href=&quot;https://www.gidhome.com/purchase/price-table&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gidhome.com/purchase/price-table&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="7020" LastEditorUserId="7020" LastEditDate="2017-06-09T05:08:16.493" LastActivityDate="2017-06-09T05:08:16.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27090" PostTypeId="1" AcceptedAnswerId="27095" CreationDate="2017-06-09T08:34:55.147" Score="10" ViewCount="1231" Body="&lt;p&gt;Say you have a triangular mesh on a flat plane. &#xA;This has been drawn to eventually solve some problem in mechanics, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A mesh of equilateral triangles is the best inasmuch as the distances between the vertices and between the centroids are the same all over. This makes interpolations and the calculation of gradients an easy and accurate task. However, because of constraints and circumstances, it is not always possible to work on a mesh of all equilateral triangles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the questions regard a mesh of triangular elements of arbitrary shape. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning &lt;strong&gt;individual mesh elements&lt;/strong&gt;. Which metrics are commonly used to quantify the dissimilarity of one generic triangle from some underlying ideal equilateral shape?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning &lt;strong&gt;the whole mesh&lt;/strong&gt;. Which metrics are in use to quantity the irregularity of a mesh of arbitrary triangles on the whole? These metrics should indicate how scrambled the mesh is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for thinking along.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;  All contributions from the finite-element community have been greatly appreciated.&#xA;For this question, please note that the interest is to quantify differences purely in the geometry (arbitrary vs equilateral triangles).&#xA;The subsequent effect on the interpolation and conditioning errors are outside scope. Granted these can be insightful and relevant, they complicate the mathematical handling.&lt;/p&gt;&#xA;" OwnerUserId="22948" LastEditorUserId="22948" LastEditDate="2017-06-10T13:47:00.880" LastActivityDate="2017-07-07T23:07:55.363" Title="Commonly-used metrics to quantify the irregularity of a triangular mesh" Tags="|mesh-generation|mesh|geometry|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27091" PostTypeId="2" ParentId="8025" CreationDate="2017-06-09T11:22:07.907" Score="5" Body="&lt;p&gt;The problem was solved recently: &lt;a href=&quot;https://arxiv.org/abs/1612.01482&quot; rel=&quot;noreferrer&quot;&gt;Lederer/Schöberl: Polynomial robust stability analysis for H(div)-conforming finite elements for the Stokes equations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2017-06-09T11:22:07.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27092" PostTypeId="1" CreationDate="2017-06-10T09:03:53.433" Score="1" ViewCount="111" Body="&lt;p&gt;Suppose I have a complicatedly shaped conductor with inhomogeneous conductivity. The conductor is modeled using the Finite Element Method. The conductor has electrical contacts on both ends. Those contacts are represented in the FEM mesh by a couple of nodes on either end of the conductor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'd like to simulate current and voltage on both contacts. Basically, I could just solve the whole FEM system of the conductor, for example by inverting the whole system matrix. But in fact, I'm only interested in the values on the nodes representing the contacts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when inverting the system, I do not need to find a matrix that contains the relationship from any node to any node, but just a matrix, that contains the relationship between all contact nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still could compute the inverse of the whole system matrix, and then cut-out only the relevant rows and columns. But that would be a waste of resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a way to directly compute the inverse of a matrix representing an equation system only for a subset of it's variables?&lt;/strong&gt; Same for iterative solution of only a subset of variables.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-06-11T11:22:48.613" Title="Invert a matrix only on a subset of variables / Compute the &quot;equivalent circuit&quot;" Tags="|linear-algebra|matrix|matrix-equations|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27093" PostTypeId="1" AcceptedAnswerId="27094" CreationDate="2017-06-10T15:19:19.980" Score="0" ViewCount="491" Body="&lt;p&gt;I am using OpenSees to model a simply supported beam with a point load in the middle. The model is in consistent units. The beam is made up of bilinear quad elements. I have used 30 elements along the length of the beam and 1 element along the height of the beam. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/crJHE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/crJHE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The material is isotropic with the following properties:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E = 80000$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nu = 0.0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The loading $P = -10$.&#xA;The model is 2 dimensions and 2 DOF's per node, although the software allows for input of element thickness in the 3rd dimension. The elements are $1$ unit by $1$ unit and the image does not reflect this (not to scale).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From basic structural mechanics the deflection at the base middle of the beam is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{max} = \frac {PL^3}{48EI}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which in this case equals $-0.84$ units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running a static analysis on the numerical model however yields a result of $-0.56$ units. This is very different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does the beam modelled by continuum elements not reflect the true displacement for such a simple problem? What could be going wrong here?&lt;/p&gt;&#xA;" OwnerUserId="22736" LastEditorUserId="9667" LastEditDate="2017-06-10T15:54:35.257" LastActivityDate="2017-06-10T17:08:00.103" Title="Euler-Bernoulli beam element versus continuum beam element" Tags="|finite-element|computational-physics|solid-mechanics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27094" PostTypeId="2" ParentId="27093" CreationDate="2017-06-10T16:19:36.723" Score="6" Body="&lt;p&gt;The standard, displacement formulation quadrilateral is notoriously bad at&#xA;representing bending behavior, especially with only one element through the&#xA;thickness of the beam. This is often referred to as &quot;shear locking&quot;, so named because when the element shape functions attempt to represent pure bending of the beam, they also produce large, non-physical in-plane shear stresses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most production structural analysis FE codes have special 4-node quadrilateral elements that&#xA;have better behavior in this case. Alternatively, the standard 8-node and 9-node quadrilateral elements and the 6-node triangle element have good performance &#xA;for bending problems like this one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in the reason for this problem and ways to develop better quadrilateral elements&#xA;for bending problems, take a look at these notes, &lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch17.d/IFEM.Ch17.pdf&quot; rel=&quot;noreferrer&quot;&gt;Felippa FEM Notes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are really interested in more details, a history and description of different element&#xA;designs to deal with shear locking is contained in this book by MacNeal, &lt;a href=&quot;http://rads.stackoverflow.com/amzn/click/0824791622&quot; rel=&quot;noreferrer&quot;&gt;Finite Elements: Their Design and Performance&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-06-10T17:08:00.103" LastActivityDate="2017-06-10T17:08:00.103" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27095" PostTypeId="2" ParentId="27090" CreationDate="2017-06-10T21:19:46.273" Score="5" Body="&lt;p&gt;As @Nicoguaro and @Paul have said in the comments to the question post, there are a great many ways to do this kind of thing, and I'm not sure if there is a single &quot;best&quot; approach. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;From a review study of &lt;a href=&quot;https://people.eecs.berkeley.edu/~jrs/jrspapers.html#quality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jonathan Richard Shewchuck&lt;/a&gt; at Berkley, an answer is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/opZTx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/opZTx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please refer to the &lt;a href=&quot;https://people.eecs.berkeley.edu/~jrs/papers/elemj.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original document&lt;/a&gt; (version 31/12/2002) for symbology, terminology, special features and possibly more (e.g. tetrahedra). &#xA;Chapter 6 is about quality measures. &#xA;The document linked to is the extended version, and in JRS's webpage there is also an abridged one.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Personally, I am a fan of the &quot;volume-length&quot; metric. It's a good robust scalar indicator of (isotropic) simplex quality and is cheap to compute. In two-dimensions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a = \frac{4\sqrt{3}}{3}\frac{A}{\|\mathbf{e}_{\mathrm{rms}}\|^{_2}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is the signed area of the triangle and $\|\mathbf{e}_{\mathrm{rms}}\|$ is the root-mean-square edge length. Ideal elements achieve $a=1$, which decreases toward zero with increased distortion. Inverted elements with reversed orientation have $a &amp;lt; 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To asses the quality of an unstructured triangulation it's typical to look at histograms of such element quality metrics. There are many implementations of such things out there, but one straight-forward &lt;code&gt;MATLAB&lt;/code&gt; code-base of mine is &lt;a href=&quot;https://github.com/dengwirda/mesh2d&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to volume-length scores, histograms of element angles and vertex degree are also computed by default.&lt;/p&gt;&#xA;" OwnerUserId="1139" LastEditorUserId="22948" LastEditDate="2017-07-07T23:07:55.363" LastActivityDate="2017-07-07T23:07:55.363" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27096" PostTypeId="1" AcceptedAnswerId="27112" CreationDate="2017-06-10T23:32:48.293" Score="1" ViewCount="344" Body="&lt;p&gt;I am trying to solve a 1D initial boundary value problem in MATLAB using Finite Elements with time stepping, for the purpose of learning scientific computing and to build up to more difficult problems. The initial boundary value problem is: &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Find $u(x,t)$ such that:&#xA;\begin{align*}&#xA;u_{xxxx}  + u_{tt} =  f(x,t), \\&#xA;u(0,t) = 0, \quad &amp;amp;u(1,t) = 0, \\&#xA;u_x(0,t) = \frac{\pi t^2}{100}, \quad &amp;amp;u_x(1,t) = -\frac{\pi t^2}{100}, \\&#xA;u(x,0) = 0, \quad &amp;amp; u(1,t) = 0, \\&#xA;f(x,t) = 0.01\sin(\pi x)(\pi^4  t^2 + 2),&#xA;\end{align*}&#xA;for $x \in \Omega = [0,1]$ and $t \in \Omega_t = [0,t_{end}]$. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The solution to this problem is $u(x,t) = 0.01 t^2 \sin(\pi x)$.&#xA;The weak/variational form is: &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Find $u \in S$ such that for all $v \in T$:&#xA;\begin{align*}&#xA;\int_0^1 u_{xx}v_{xx} dx + \int_0^1 u_{tt} v dx= \int_0^1 v f dx,&#xA;\end{align*}&#xA;where $$T = \{ v: \int |v|^2  &amp;lt; \infty, \int|v_x|^2 &amp;lt; \infty, \int |v_{xx}|^2 &amp;lt; \infty , v(0) =v(1)=v_x(0)=v_x(1) = 0 \},$$&#xA;and &#xA;$$ S = \{u:\int |u|^2  &amp;lt; \infty, \int|u_x|^2 &amp;lt; \infty, \int |u_{xx}|^2 &amp;lt; \infty , u(0,t) = u(1,t) = 0,  u_x(0,t) = -u_x(1,t) = \frac{\pi t^2}{100} \}.$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This weak form will lead to the final finite-element system:&#xA;$$&#xA;A_{22} u(t) + A_{00} u_{tt}(t) = f_0(t) + b_{A_{22}}(t) + b_{A_{00}(t)}.&#xA;$$&#xA;Here, the entries of the matrix $A_{22}$ can be computed &quot;locally&quot; from $\int_0^1 u_{xx} v_{xx} dx$ and the entries of the matrix $A_{00}$ can be computed locally from $\int_0^1 u_{tt} v dx$, then the locally computed entries can be assembled into the final &quot;global&quot; matrices. The entries of the time-dependent vector $f_0 (t)$ can be computed locally from $\int_0^1 vf dx$, and can once again be assembled into a final global &quot;load vector&quot;. Also the time-dependent vectors $b_{A_{22}}(t)$ and $b_{A_{00}}(t)$ come from enforcing the essential boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I have already solved a very similar steady-state problem by introducing a Mesh for the domain, making relevant data structures and functions to access $C^1$ finite element basis functions, integrating the products of these basis functions to compute the relevant matrices/vectors, and solving the system of equations. Doing this resulted in an array of coefficients of $u(x)$ and $u_x(x)$ at the Mesh points, so that the solution to the variational problem could be approximated using the $C^1$ finite element basis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem: &lt;strong&gt;Solve the time-dependent problem presented here using a fully discrete Finite Element scheme.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempt:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the fully discrete problem, we introduce a partition for the time interval $[0, t_{end}]$:&#xA;$$&#xA;0 = t_0 &amp;lt; t_1 &amp;lt; ... &amp;lt; t_m = t_{end},&#xA;$$&#xA;with time step sizes $\tau_{k} = (t_k - t_{k-1})$. Then our goal is to find an $m$ number of vectors $u^k \approx u(t_k)$, i.e., each vector contains the coefficients of $u(x,t_k)$ and $u_x(x,t_k)$ at the $k$-th time step (so that the solution at each time step can be interpolated using the basis functions). I specifically wanted to use the Crank-Nicolson method. First I rewrite my second-order system as a first-order system by introducing variables $y_1(t) = u(t)$ and $y_2(t) = u_t (t)$, then differentiating these variables gives us:&#xA;\begin{align*}&#xA;\dot{y}(t) = &#xA;\begin{bmatrix}&#xA;\dot{y}_1(t) \\&#xA;\dot{y}_2(t)&#xA;\end{bmatrix} = &#xA;\begin{bmatrix}&#xA;y_2(t) \\&#xA;A_{00}^{-1} (-A_{22} y_1(t) + f_0(t) + b_{A_{22}}(t) + b_{A_{00}}(t))&#xA;\end{bmatrix},&#xA;\end{align*}&#xA;which is a first order system. Letting $t_{k - 1/2} = (t_{k - 1} + t_k)/2$, we use the following approximations for $y$ and $y'$ at the $t_{k - 1/2}$-th time step:&#xA;\begin{align*}&#xA;y(t_{k - 1/2}) = &#xA;\begin{bmatrix}&#xA;y_1 (t_{k - 1/2}) \\&#xA;y_2 (t_{k - 1/2})&#xA;\end{bmatrix} \approx &#xA;\frac{y(t_{k-1}) + y(t_k)}{2}, \\&#xA;\dot{y}(t_{k - 1/2}) \approx \frac{y(t_k) - y(t_{k-1})}{\tau_k}.&#xA;\end{align*}&#xA;Substituting these into the first-order system and rearranging to obtain the $y_1 (t_k)$ and $y_2 (t_k)$ on the right hand side gives&#xA;\begin{align*}&#xA;\begin{bmatrix}&#xA;y_1 (t_k)/\tau_k - y_2 (t_k)/2 \\&#xA;A_{00} y_2(t_k)/\tau_k + A_{22} y_1 (t_k)/2&#xA;\end{bmatrix} = &#xA;\begin{bmatrix}&#xA;y_1(t_{k-1})/\tau_k + y_2(t_{k - 1})/2 \\&#xA;A_{00} y_2 (t_{k-1})/\tau_k - A_{22} y_1 (t_{k-1})/2 + f_0(t_{k - 1/2}) + b_{A_{22}}(t_{k - 1/2}) + b_{A_{00}}(t_{k - 1/2})&#xA;\end{bmatrix}.&#xA;\end{align*}&#xA;Is this the correct formula I should use, or is there anything I am missing? The initial conditions here are just $y_1(0) = 0$ and $y_2(0) = 0$, and I could just form $f_0 (t_k)$ at each time step, so it seems like things should run smoothly. For example, for the first time step $k = 1$ I would have&#xA;\begin{align*}&#xA;\begin{bmatrix}&#xA;y_1 (t_1)/\tau_1 - y_2 (t_1)/2 \\&#xA;A_{00} y_2(t_1)/\tau_1 + A_{22} y_1 (t_1)/2&#xA;\end{bmatrix} = &#xA;\begin{bmatrix}&#xA;0 \\&#xA;f_0(t_{1/2}) + b_{A_{22}}(t_{1/2}) + b_{A_{00}}(t_{1/2})&#xA;\end{bmatrix},&#xA;\end{align*} &#xA;and here the only unknowns are $y_1(t_1)$ and $y_2(t_2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT: It turns out that there was a small mistake in my original MATLAB code and everything seems to be working fine. Should I delete this question, or what?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2017-06-11T22:08:07.453" LastActivityDate="2017-06-12T14:04:49.807" Title="Fully discrete finite element method for 1D dynamic euler-bernoulli beam problem" Tags="|finite-element|finite-difference|time-integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27097" PostTypeId="1" CreationDate="2017-06-11T01:16:12.077" Score="1" ViewCount="190" Body="&lt;p&gt;Consider a nonlinear advection-diffusion equation of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} = \nabla \cdot (a(u) \nabla b(u) - \vec{c}(u)u) \tag{1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on a rectangular domain with Dirichlet boundary conditions. We have $a(u) &amp;gt; 0$ for all $u \in \mathbb{R}$ and $b(u)$ is a monotonically increasing function of $u$. I wish to solve (1) by finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can (1) always be discretized in such a way such that all matrix solves involve only symmetric positive definite matrices? If not, are there conditions on $a$, $b$, and $\vec{c}$ such that this is the case?&lt;/p&gt;&#xA;" OwnerUserId="16757" LastActivityDate="2017-07-11T17:39:23.787" Title="Can this nonlinear advection-diffusion equation be discretized as to only have to solve SPD systems?" Tags="|finite-difference|discretization|advection-diffusion|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27098" PostTypeId="1" CreationDate="2017-06-11T04:01:16.010" Score="1" ViewCount="742" Body="&lt;p&gt;Say I have a set of discrete acceleration data and want to integrate it to get a set of velocity data.  How accurate is the &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/cumtrapz.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;cumtrapz&lt;/code&gt; (Cumulative trapezoidal numerical integration)&lt;/a&gt; integrator in MATLAB?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it follow that the more acceleration data I have (for whatever fixed timescale), the better results I'll get from &lt;code&gt;cumptrapz&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="24532" LastEditorUserId="17294" LastEditDate="2017-06-11T10:59:45.653" LastActivityDate="2017-06-11T10:59:45.653" Title="How accurate is cumtrapz in MatLab?" Tags="|matlab|numerical-analysis|integration|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27100" PostTypeId="2" ParentId="27092" CreationDate="2017-06-11T11:22:48.613" Score="3" Body="&lt;p&gt;The procedure you need to get the &quot;reduced&quot; equations  is often referred to as &quot;static condensation&quot; in the FEM literature. You can&#xA;partition your FE equations as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\begin{array}{cc}&#xA;K_{cc} &amp;amp; K_{ci} \\&#xA;K_{ic} &amp;amp; K_{ii} \\&#xA;\end{array}\right]&#xA;\left\{\begin{array}{c}&#xA;V_c \\&#xA;V_i&#xA;\end{array}\right\} =&#xA;\left\{\begin{array}{c}&#xA;Q_c \\&#xA;Q_i&#xA;\end{array}\right\}&#xA;$$&#xA;where the subscript $c$ refers to the contact degrees of freedom and $i$ refers to the internal degrees of freedom in your model.&#xA;Form the lower set of equations, you can then obtain an expression for $V_i$ in terms of $V_c$&#xA;$$&#xA;V_i = K_{ii}^{-1} (Q_i - K_{ic}V_c)&#xA;$$&#xA;Substituting this expression into the upper  set of equations yields&#xA;$$&#xA;K_{cc}V_c + K_{ci}(K_{ii}^{-1} (Q_i - K_{ic}V_c)) = Q_c&#xA;$$&#xA;which can be rearranged to give&#xA;$$&#xA;(K_{cc} - K_{ci}K_{ii}^{-1}K_{ic})V_c = Q_c - K_{ci}K_{ii}^{-1} Q_i&#xA;$$&#xA;with only the $V_c$ set of unknowns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This process can be interpreted as is just a reordering of the original equations and then solving for only the $V_i$ degrees of freedom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Particularly in structural finite element analysis, the matrix, $K_{cc} - K_{ci}K_{ii}^{-1}K_{ic}$, is often described as the&#xA;stiffness matrix for the &quot;substructure&quot; or &quot;superelement&quot;. That is, you have created a new &quot;element&quot; that describes the&#xA;behavior in terms of only the contact degrees of freedom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally note that the inverse operation shown in these equations means that you should perform Gaussian elimination on the matrix&#xA;and then solve using the relevant right hand side.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-06-11T11:22:48.613" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27101" PostTypeId="2" ParentId="27097" CreationDate="2017-06-11T17:04:33.270" Score="2" Body="&lt;p&gt;One strategy may be to use a semi-implicit temporal discretion, treating the diffusive term implicitly and the advective term explicitly. Doing so alleviates the extreme timestep restriction typically imposed by explicit diffusion, and it keeps your linear system symmetric. This is a common trick with (eg) the Navier Stokes equations to avoid precisely the same issues.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-06-11T17:04:33.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27106" PostTypeId="2" ParentId="27088" CreationDate="2017-06-12T09:01:12.073" Score="2" Body="&lt;p&gt;That's basically two questions in one. The first is how to incorporate boundary conditions, the other how to treat the symmetry. I'll consider only the symmetry here, because the boundary value stuff has been already discussed on this page, see e.g. &lt;a href=&quot;https://scicomp.stackexchange.com/a/26660/14942&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt; Parity symmetry &lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In your case it's about parity symmetry, i.e. the symmetry that possibly occurs under a change of sign of the solution (--the Laplace operator is also rotationally invariant, but the rectangular boundary breaks this symmetry).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, let's assume your problem supports parity symmetry. One sufficient condition for this is that your inhomogeneity $f$ is an even function, $f(x,y)=f(-x,-y)$, and your $\Omega$ is also symmetric with respect to the origin.&#xA;If this holds, it follows that the solutions to your problem have definite parity,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u(-x,y) = (-1)^{m_x} \;u(x,y)\\&#xA;u(x,-y) = (-1)^{m_y} \;u(x,y)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even $m_x$ (resp. $m_y$) correspond to even parity and thus even solutions with respect to the origin, and odd ones to odd parity. (I used to call these &quot;quantum numbers&quot; in my application to quantum mechanics, but the concept comes from group theory).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions for your problem follow directly. For odd parity, one has $v(x) = -v(-x)$ and thus $v(0) = 0$. For even parity, it's $v(x) = v(-x)$ and therefore $v^\prime(0)=0$. This directly extends to two dimensions, and it allows you to specify the solution only in the upper left quadrant and extend it accordingly to the other ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As usual, there are two basic ways to incorporate this symmetry. (i) pick basis functions which already satisfy the symmetry and expand your solution in these, or (ii) explicitly enforce them in the solution of your problem (e.g. via Lagrange multipliers). Again, see the &lt;a href=&quot;https://scicomp.stackexchange.com/a/26660/14942&quot;&gt;linked thread&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2017-06-12T09:01:12.073" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27107" PostTypeId="1" CreationDate="2017-06-12T09:04:26.707" Score="1" ViewCount="170" Body="&lt;p&gt;I have the following two PDEs:&#xA;$$\partial_zU=\nabla_r^2U+\varrho U$$&#xA;$$\partial_t\varrho=a\vert U\vert^4$$&#xA;with $a$ a constant and &#xA;$$dt=dz\cdot\frac{n}{c}$$ with $n$ the refractive index of a material, and $c$ the speed of light.&lt;br&gt;&#xA;My current approach is that I apply a crank-nicholson-algorithm on the first equation by neglecting the second part on the right side, using a matrix solver:&#xA;$$\left(1-\frac{dz}{2.0}\partial_z\right)U_{n+1}=\left(1+\frac{dz}{2.0}\nabla_r^2\right)U_n$$&#xA;and then calculating $\varrho$ in between:&#xA;$$U_{n+\frac{1}{2}}=\frac{U_{n+1}+U_{n}}{2}$$&#xA;$$\varrho_{n+\frac{1}{2}} = a\left\vert U_{n+\frac{1}{2}}\right\vert\cdot \underbrace{dz\cdot\frac{n}{c}}_{\equiv dt}$$&#xA;But is there an easier/more accurate way to do that, maybe even include it in the matrix equation above?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-07-14T16:23:08.890" Title="Improving calculation algorithm for coupled PDEs" Tags="|pde|nonlinear-equations|coupling|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27109" PostTypeId="2" ParentId="27088" CreationDate="2017-06-12T12:19:01.787" Score="0" Body="&lt;p&gt;I understand your specific question. I don't understand the other two&#xA;questions or how they relate to the specific question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because your&#xA;weak form contains second derivatives of $u$, shape functions satisfying&#xA;$C_1$ continuity are required. This means that that both $u$ and&#xA;first derivatives of $u$ will be nodal unknowns. So you can satisfy the&#xA;symmetry conditions on your two symmetry edges simply by specifying &#xA;Dirichlet conditions on the derivative unknowns. That is how this&#xA;problem is routinely solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Developing a conforming plate bending finite element based on Kirchoff&#xA;thin plate theory (the biharmonic equation you show) is quite challenging.&#xA;If you are determined to proceed in this direction, I suggest taking a&#xA;look at these notes (&lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch22.d/AFEM.Ch22.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Felippa plate element notes&lt;/a&gt;)&#xA;to get an idea of the difficulties. Felippa has other sets of notes on this&#xA;site (his advanced FEM course) that provide more details about how to formulate&#xA;a workable element.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-06-12T12:19:01.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27110" PostTypeId="1" CreationDate="2017-06-12T12:34:19.813" Score="1" ViewCount="78" Body="&lt;p&gt;I have a fair background in pure mathematics and right now my project is a numerical implementation of a certain algorithm. I have some numerical background, but not all that much, so the question is simple:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my algorithm, I have to calculate the value of certain parameter&#xA;$$h_r = \lim_{t \to \infty} I(t,r),$$&#xA;which we have proven exists. Calculating the expression $I$ is computationally fairly expensive, as it involves solving a non-linear PDE. The parameter $r$ indexes a suitably dense set of directions (unit vectors) and all the calculations have to be done for all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I calculate the limit? I can guess at a few options, but I don't know what are the best practices, or if I am missing some elegant solution that is common knowledge.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Select a large enough $t$, heuristically or by experiment. This seems to be a bad idea except for a test to see if the algorithm as a whole works.&lt;/li&gt;&#xA;&lt;li&gt;Use an increasing sequence of $t$ with some stopping condition, such as $|I(t_j,r) - I(t_{j+1},r)|$ (or, better yet, relative change) being small.&lt;/li&gt;&#xA;&lt;li&gt;Calculate how large $t$ needs to be, in order for the error of $I$ to be sufficiently small. This might turn out to be very difficult, given the complicated nature of $I$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6706" LastActivityDate="2017-06-13T00:21:14.147" Title="Calculating a limit as parameter goes to infinity" Tags="|matlab|numerical-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27111" PostTypeId="2" ParentId="27088" CreationDate="2017-06-12T13:51:15.167" Score="0" Body="&lt;p&gt;Only essential boundary conditions are implemented by Nitsche's method in weak form. Since your problem is formulated in $H^2(\Omega)$, these can only involve function values and first derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For higher order derivatives, look at integration by parts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\int_\Omega \Delta^2 u v \,dx &amp;amp;= -\int_\Omega \nabla\Delta u\cdot\nabla v\,dx + \int_{\partial\Omega} \partial_n \Delta u v\,ds\\&#xA;&amp;amp;= \int_\Omega \Delta u \Delta v \,dx - \int_{\partial\Omega} \Delta u \partial_n v\,ds + \int_{\partial\Omega} \partial_n \Delta u v\,ds.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary terms allow you to set boundary values by replacing $\partial_n \Delta u$ and $\Delta u$ by their respective boundary values. Note though, that only normal derivatives are involved in this integration by parts. Therefore, most problems involving tangential derivatives are ill-posed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples for ill-posed boundary conditions involving tangential derivatives:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt; $u=f$ and $\partial_\tau 0= g$, since this can only work if $g=f'$ where the derivative is taken along the boundary.&#xA;&lt;li&gt; $\partial_n u=f$ and $\partial_{n}\partial_\tau u=g$ for the same reason.&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You could have oblique derivatives by subtracting the tangential derivative in the weak formulation. Be aware though that this requires sufficient regularity to write the boundary integrals. Otherwise, you will generate boundary layers.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2017-06-12T13:51:15.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27112" PostTypeId="2" ParentId="27096" CreationDate="2017-06-12T13:53:42.763" Score="0" Body="&lt;p&gt;Using the notation $a^k = a(t_k)$, we rearrange our time-stepping formula to obtain &#xA;\begin{align*}&#xA;y_1^k = \left(y_1^{k-1} + \tau_k \frac{y_2^{k-1}}{2} + \tau_k \frac{y_2^k}{2} \right), \\&#xA;y_2^k = \left(\frac{A_{00}}{\tau_k} + \frac{\tau_k A_{22}}{4}\right)^{-1} \left(\frac{-A_{22}}2 \right) \left(\frac{y_1^{k-1}}{\tau_k} + \frac{y_2^{k-1}}{2} + y_1^{k-1} \right) + f_0^{k - 1/2} + b_{A_{22}}^{k - 1/2} + b_{A_{00}}^{k - 1/2}.&#xA;\end{align*}&#xA;Then we can compute $y_2^k$ for each $k$ since the right hand side is known, and we can use $y_2^k$ to compute $y_1^k = u^k = u(t_k)$, the vector containing the coefficients that we desire.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A plot of the finite element coefficients (red circles) at $t = 4.5$ against the true solution $u(x,4.5) = 0.01*(4.5)^2*\sin(\pi x)$ is shown below. For this calculation, 100 elements were used, and the time step size was $\tau_k = 1/40$ for all $k$. The error measured by the $L^2$ norm is just:&#xA;\begin{align*}&#xA;\|u_{\text{true}}(x,4.5) - u_{\text{fe}}(x,4.5)\|_{L^2(0,1)} &amp;amp;= \sum_{\text{elements}} \int_{a}^b \left( u_{\text{true}}(x,4.5) - u_{\text{fe}}(x,4.5) \right)^2 dx \\&#xA;&amp;amp;\approx \text{3.782594050169693e-05}.&#xA;\end{align*}&lt;a href=&quot;https://i.stack.imgur.com/1AK93.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1AK93.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2017-06-12T14:04:49.807" LastActivityDate="2017-06-12T14:04:49.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27113" PostTypeId="2" ParentId="27107" CreationDate="2017-06-12T13:57:12.590" Score="0" Body="&lt;p&gt;What you are doing is a kind of leap-frog scheme, but any time-stepping scheme will do. But since your problem is stiff and nonlinear, you may want to use a B-stable method, in order to avoid having to use small time steps for stability reasons.&lt;/p&gt;&#xA;" OwnerUserId="4108" LastActivityDate="2017-06-12T13:57:12.590" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27114" PostTypeId="1" AcceptedAnswerId="27118" CreationDate="2017-06-12T16:24:05.177" Score="2" ViewCount="250" Body="&lt;p&gt;Suppose that I have a large (on the order of 10^6 unknowns) 3D scalar Poisson system which I would like to precondition. The boundary conditions have been treated so that the system is SPD. I.e.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla\cdot(\kappa(x)\nabla u(x)) = f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the matrices tend to have the following structure (this is a very small problem, the larger problems are indeed sparse):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/edeGe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/edeGe.png&quot; alt=&quot;sparsity pattern for small problem&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a consensus about how to precondition such systems (ideally in a way that can be implemented in parallel)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear system is SPD, so it seems natural to choose a PCG scheme, but I haven't been able to find a consensus about how to choose the preconditioner, since many papers deal with more complicated Stokes-like systems instead. &lt;/p&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="20148" LastEditDate="2017-06-12T19:48:16.960" LastActivityDate="2017-06-13T10:18:28.887" Title="Preconditioner for scalar laplacian system" Tags="|reference-request|preconditioning|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27117" PostTypeId="2" ParentId="27114" CreationDate="2017-06-12T20:28:51.577" Score="2" Body="&lt;p&gt;With SPD matrix, call it $A$, PCG is a good method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kind of preconditioner depend a lot by the matrix structure (read kind of matrix) that you have got and there is not a unique answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic criteria to choose the preconditioner, $P$, are more than one.&#xA;Considering the PCG the three most important are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$AP^{-1} \approx I$&lt;/li&gt;&#xA;&lt;li&gt;$P$ is cheap to built&lt;/li&gt;&#xA;&lt;li&gt;calculated $P^{-1}r_{k+1}$ is cheap&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In general for problem like laplacian you can try the two follow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Diagonal $D^{-1}$&lt;/strong&gt; The diagonal preconditioner is very easy to implement and cheap to apply, but is not very efficient. The matrix contains the verse of $A$ diagonal terms. Better if it is used with matrix $A$ &lt;a href=&quot;https://en.wikipedia.org/wiki/Diagonally_dominant_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;diagonally dominant&lt;/a&gt;, but when the extra diagonal terms are big you can find problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Incomplete Cholesky $\tilde{L}\tilde{L^T}$&lt;/strong&gt; Whit $A$ SPD is defined the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cholesky_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cholesky decomposition&lt;/a&gt;. The idea of this preconditioner is to use the decomposition but to conserve the sparsity pattern of $A$, i.e you use the term calculated by the decomposition only for the terms of $A$ different from zero. The existence of this incomplete decomposition depend by the structure of $A$, for example if $A$ is a $M$ matrix it is ok. This kind of preconditioner can improve if you reorder the nodes of $A$ so that the mustache of $A$ are shorter (here you can see an example that the structure of $A$ is important).&#xA;This preconditioned is apply with the method forward and back (as for the $LU$ decomposition)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you talk about parallel execution here is important to know the kind of hardware. Anyway you must parallelize the preconditioner application, for example with $D^{-1}$ you parallelize the $\frac{v_i}{d_i}$ operation for the whole vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you mentioned the Stokes's problem I add some rows about parallelition. If your goal is to use heavy parallelization as gpu is important to use preconditioner with an easy application for gpu, as the product matrix vector sparse (SpMV). In the article of Li and Saad [1] there is a comparison of various preconditioners for gpu.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of gpu &lt;strong&gt;polynomial preconditioner&lt;/strong&gt; is good. &#xA;I personally use this kind with good performance respect other traditional gpu preconditioner, see &lt;a href=&quot;https://groups.google.com/forum/#!searchin/cusp-users/mauro.vanzetto%7Csort:relevance/cusp-users/jqx_h0Cn7Jg/SxdqV3mDbkUJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt; to CUSP group where I give some detail, with reference, and also the code with some result test.&#xA;This is a least square preconditioner, it uses  Chebyshev polynomials so explicit quadrature formula are not involved, detail are in the article by J. Erhel, F. Guyomarc, and Y. Saad [2].&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[1] Ruipeng Li and Yousef Saad. Gpu-accelerated preconditioned iterative linear solvers. The Journal of Supercomputing, 63(2):443–466, 2013.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] J. Erhel, F. Guyomarc, and Y. Saad, Least-squares polynomial filters for ill-conditioned lin- ear systems, Tech. Report umsi-2001-32, Minnesota Supercomputer Institute, University of Minnesota, Minneapolis, MN, 2001.  &lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="22025" LastEditDate="2017-06-13T10:18:28.887" LastActivityDate="2017-06-13T10:18:28.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27118" PostTypeId="2" ParentId="27114" CreationDate="2017-06-12T20:59:10.313" Score="5" Body="&lt;p&gt;Preconditioning and iterative solvers are cool, but did you try to solve your problem with some kind of sparse direct solver? If not, try it first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;State–of–the–art preconditioning techniques for this kind of problems are &lt;strong&gt;multigrid techniques&lt;/strong&gt;. In many cases multigrid allows you to solve your (elliptic) problem in $O(n)$ time. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you don’t want to go deeper, a good start is probably algebraic multigrid. You may try it both as a stand–alone solver and a preconditioner for CG. It is implemented e.g. in &lt;a href=&quot;https://www.mcs.anl.gov/petsc/&quot; rel=&quot;noreferrer&quot;&gt;PETSc&lt;/a&gt;.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you do want to go deeper, a good start is &lt;a href=&quot;https://www.researchgate.net/publication/51994678_Software_Tools_for_Multigrid_Methods&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; paper. It has a nice collection of examples revealing multigrid’s behavior depending on problem’s geometry, parameters $f$ and $\kappa$, and so forth.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-06-12T20:59:10.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27119" PostTypeId="2" ParentId="27110" CreationDate="2017-06-13T00:21:14.147" Score="2" Body="&lt;p&gt;This is a bit primitive, but has a good chance to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plot $I$ versus $1/t$. Fit a smooth curve and find the intercept. This works a lot better if you have some theory predicting the behavior of $I$. For example, an error analysis might give $$I(t)=I_\infty+\frac{C}{t^2}+\cdots$$&#xA;If you have two different values of $t$, you can write two equations and solve them to find the unknowns $I_\infty$ and $C$. With more values, plot $I(t)$ against $t^{-2}$. Does it look like a straight line? Or only if $t$ is big enough. Then fit a straight line to the values that appear to lie on one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The human eye is very good at detecting patterns, such as spotting a linear trend. You can make various hypotheses about plots that might be linear and pick the ones that are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The more formal versions of this trick define Richardson's deferred approach to the limit, which you might Google. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-06-13T00:21:14.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27123" PostTypeId="1" CreationDate="2017-06-13T09:42:10.977" Score="3" ViewCount="83" Body="&lt;p&gt;The following least-square problem can be solved efficiently (e.g. using matlab's &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/lsqlin.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;lsqlin&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{x}^*=\arg\min_\vec{x} ||C\vec{x}-\vec{t}||^2\,\ s.t.\ Ax \le \vec{b}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the parameters of the problem are vectors $\vec{t},\vec{b}$, and matrices $A,C$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an efficient way to solve this in parallel for multiple $\vec{t}_i$, but using the same $A, C, \vec{b}$? A solution which further assumes $C=I$ (the identity matrix) would work for me.&lt;/p&gt;&#xA;" OwnerUserId="10450" LastEditorUserId="10450" LastEditDate="2017-06-13T11:49:40.443" LastActivityDate="2017-06-14T10:22:15.093" Title="Solving multiple least-square problems with the same constraints" Tags="|optimization|constrained-optimization|least-squares|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27125" PostTypeId="1" CreationDate="2017-06-13T11:40:20.387" Score="1" ViewCount="326" Body="&lt;p&gt;To study an approximation for the heat equation $$\frac{\partial^2 u}{\partial r^2}+\frac{1}{r}\frac{\partial u}{\partial r}+\frac{1}{r^2}\frac{\partial^2 u}{\partial\theta^2}=f(r,\theta)$$&#xA;on the disk $D=(0,1)\times(0,2\pi)$ with periodic boundary conditions, we used the following finite difference method&#xA;$$\frac{u_{i+1,j}-2u_{i,j}+u_{i-1,j}}{(\Delta r)^2}+\frac{1}{r_i}\frac{u_{i+1,j}-u_{i-1,j}}{2\Delta r}+\frac{1}{r_i^2}\frac{u_{i,j+1}-2u_{i,j}+u_{i,j-1}}{(\Delta\theta)^2}$$&#xA;where $$r_i=(i-\frac{1}{2})\Delta r, \space\space\space\space i=1,2...N+1$$ $$\theta_j=(j-1)\Delta \theta, \space\space\space\space j=1...M+1$$&#xA;$$\Delta r=\frac{2}{2N+1}, \space\space\space\space \Delta \theta=\frac{2\pi}{M}$$&#xA;$$M=2N$$&#xA;By Taylor expansion, we computed the LTE which should be $$\tau=(\Delta r)^2(\frac{1}{12}u^{IV}(r,\theta)+\frac{1}{6}u^{III}(r,\theta))+\frac{(\Delta \theta)^2}{12}u^{IV}(r,\theta)+O((\Delta r)^4,(\Delta \theta)^4)$$&#xA;Thus the method should show second order convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We implemented it in MATLAB with the following code (we tried different values of N and different source functions)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = Nvect(index);&#xA;&#xA;M=2*N;&#xA;&#xA;h = 2/(2*N+1);&#xA;&#xA;k=2*pi/M;&#xA;&#xA;for i=1:N&#xA;&#xA;gv(i) =(i-0.5)*h;&#xA;&#xA;end&#xA;&#xA;for j=1:M&#xA;&#xA;gv2(j)=(j-1)*k;&#xA;&#xA;end&#xA;&#xA;&#xA;[meshr,meshteta] = meshgrid(gv,gv2);&#xA;&#xA;&#xA;meshr = meshr'; &#xA;&#xA;&#xA;meshteta = meshteta';&#xA;&#xA;f = @(r,teta) 9.*r;&#xA;&#xA;&#xA;rhs = f(meshr,meshteta);&#xA;&#xA;&#xA;u = @(r,teta) r.^3;            &#xA;&#xA;&#xA;u_mesh = u(meshr,meshteta);&#xA;&#xA;boundary=u(gv(N),gv2);&#xA;&#xA;rhs=rhs*(h^2);&#xA;&#xA;&#xA;F = reshape(rhs,[M*N,1]);&#xA;&#xA;&#xA;for i=1:N&#xA;&#xA;  lambda(i,1)=1/(2*(i-0.5));&#xA;&#xA;end&#xA;&#xA;for j=1:M&#xA;&#xA;   F(j*N)=F(j*N)-(1+lambda(N))*boundary;&#xA;&#xA;end&#xA;&#xA;for j=1:N&#xA;&#xA;beta(j,1)=1/((j-0.5)^2*(k^2));&#xA;&#xA;end&#xA;&#xA;&#xA;&#xA;I = speye(M);&#xA;&#xA;I2=ones(N,1);&#xA;&#xA;%I2=speye(m);&#xA;&#xA;D=sparse(diag(beta));&#xA;&#xA;d=ones(M);&#xA;&#xA;T=spdiags([1-lambda, -2*I2, 1+lambda],[1,0,-1],N,N)';&#xA;&#xA;Ap= spdiags([d,d,d,d],[1,-1,M-1,-(M-1)],M,M);&#xA;&#xA;A=kron(I,T)-2*kron(I,D)+kron(Ap,D);&#xA;&#xA;&#xA;U = A\F;&#xA;&#xA;&#xA;U = reshape(U,N,M);&#xA;&#xA;u_int = u_mesh(1:N,1:M);&#xA;&#xA;&#xA;e = max(max(abs(U-u_int)));&#xA;&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We tried $N = [8,\space16,\space32,\space64,\space128,\space256,\space512,\space1024]$ and the error ratio $\frac{e_{N_{i+1}}}{e_{N_{i}}}$ stays always around 0.5 which shows a first order accuracy.&#xA;What could cause such inconsistency between theoretical and computational results? What did we get wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: As a test, we used a source function s.t. it's easy to compute true solution $U$ by hand so we just compute the difference between $U$ and its approximation and take $e_{N_{i}}=max(U_i-U^{appr}_i)$.&lt;/p&gt;&#xA;" OwnerUserId="24555" LastEditorUserId="24555" LastEditDate="2017-06-14T14:19:36.760" LastActivityDate="2017-06-14T14:19:36.760" Title="Accuracy of finite difference method for heat equation on a disk" Tags="|matlab|finite-difference|convergence|parabolic-pde|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27126" PostTypeId="1" CreationDate="2017-06-13T12:45:35.513" Score="1" ViewCount="693" Body="&lt;p&gt;I am trying to understand the nodal and modal basis formulation from the book of Hesthaven (&lt;a href=&quot;http://springer.com/gp/book/9780387720654&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nodal Discontinuous Galerkin Methods, Hesthaven, Jan S., Warburton, Tim&lt;/a&gt;). For $N=2$, I get the Vandermonde Matrix, which is computed by the Jacobi polynomial for $\alpha, \beta =0$ which is the Legendre polynomial:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;V = Vandermonde1D(2, x)&#xA;&#xA;V =&#xA;&#xA;0.7071   -1.2247    1.5811&#xA;0.7071         0   -0.7906&#xA;0.7071    1.2247    1.5811&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;code&gt;x = (-1,0,1)&lt;/code&gt; the LGL points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the book on page 48, it says that for $N=6$ the d) option on the table is for orthonormal basis with Legendre-Gauss-Lobatto points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if I run a code &lt;a href=&quot;https://de.mathworks.com/matlabcentral/fileexchange/4775-legende-gauss-lobatto-nodes-and-weights?focused=5055264&amp;amp;tab=function&quot; rel=&quot;nofollow noreferrer&quot;&gt;from here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[x,w,P]=lglnodes(2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x =&#xA;-1&#xA; 0&#xA; 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which are the LGL points and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w =&#xA;0.3333&#xA;1.3333&#xA;0.3333&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the weights.&#xA;But then I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P =&#xA;1.0000   -1.0000    1.0000&#xA;1.0000   -0.0000   -0.5000&#xA;1.0000    1.0000    1.0000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the comments for the program, it says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Computes the Legendre-Gauss-Lobatto nodes, weights, and the LGL Vandermonde matrix. The LGL nodes are the zeros of $(1-x^2)P'_N(x)$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But this is different than the Vandermonde I get from the original program of Hesthaven.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I kinda understand what this $P$ Matrix is.&#xA;If you have the &lt;a href=&quot;https://en.wikipedia.org/wiki/Legendre_polynomials&quot; rel=&quot;nofollow noreferrer&quot;&gt;Legendre Polynomials&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p_{0}(x) = 1,  p_{1}(x) = x, p_{2}(x) = \frac{1}{2}(3x^{2} -1)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and replace the LGL points &lt;code&gt;(-1,0,1)&lt;/code&gt; I get this $P$ matrix.&#xA;But how to get the Vandermonde matrix?&lt;/p&gt;&#xA;" OwnerUserId="23896" LastEditorUserId="20688" LastEditDate="2018-05-21T15:57:59.987" LastActivityDate="2018-06-05T15:36:11.203" Title="Vandermonde matrix DG Hestaven" Tags="|interpolation|quadrature|discontinuous-galerkin|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="27127" PostTypeId="1" AcceptedAnswerId="27149" CreationDate="2017-06-13T14:43:00.050" Score="1" ViewCount="264" Body="&lt;p&gt;For a 2D HDG code, I would like to be able to visualize the solution on the edge space between elements. Basically, this amounts to plotting the solution on the &quot;green&quot; nodes below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0sLLU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0sLLU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a standard way using VTK / Paraview to plot these values along all the green line segments (using a colorbar)? &lt;/p&gt;&#xA;" OwnerUserId="20148" LastActivityDate="2017-06-15T00:07:40.883" Title="Viewing HDG FEM edge variables in vtk / paraview" Tags="|discontinuous-galerkin|paraview|vtk|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27128" PostTypeId="1" CreationDate="2017-06-13T14:57:35.563" Score="1" ViewCount="128" Body="&lt;p&gt;I'm able to find many definitions and notions to this term including,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Implementing the algorithm (or part of it) in hardware :- Instructions like FMAD; FPGAs&lt;/li&gt;&#xA;&lt;li&gt;Parallelizing the algorithm at the hardware level :- SIMD, GPU etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, what exactly is Hardware Acceleration ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to understand the meaning of the word to not sound ambiguous in technical discussions.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2017-06-13T18:11:18.670" Title="What exactly is Hardware Acceleration?" Tags="|parallel-computing|vectorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27129" PostTypeId="2" ParentId="27090" CreationDate="2017-06-13T15:23:03.267" Score="5" Body="&lt;p&gt;I do not think that there exists an answer to this question &lt;strong&gt;in general&lt;/strong&gt;, because it all depends on the intended use for the mesh. For instance, if you are doing computational fluid dynamics, you may want to have a mesh that is extremely anisotropic near the boundary layer. Now if you are doing computational electromagnetics, the best mesh will be probably completely different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are in the literature many different definitions for a &quot;mesh quality&quot; criterion. Most of them will favor meshes with triangles that are as equilateral as possible. One can also mention the idea of maximizing the smallest angle (which is realized by Delaunay triangulation for a fixed set of points). It is justified by Jonathan Shewchuk's analysis mentioned in one of the comments, that relates this angle with the condition number of the stiffness matrix for the Laplace equation discretized with P1 elements, but again, depending on the intended use, somebody's good mesh can be somebody else's poor mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not think that it makes sense to &quot;quantify differences purely in the geometry (arbitrary vs equilateral triangles)&quot;: before measuring whether the triangles are equilateral and deciding which &quot;deviation w.r.t. equilaterality&quot; is the best one, it is necessary to figure out whether &quot;equilateral triangles&quot; is what we want, and it is not always the case ! It all comes from the &quot;interpolation and conditioning&quot; that you mention. Yes, as you said &quot;it complicates the mathematical handling&quot; but without it, it is not possible to make the difference between objective criteria for a given application and criteria that do not make sense at all.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-06-13T15:48:50.067" LastActivityDate="2017-06-13T15:48:50.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27131" PostTypeId="1" AcceptedAnswerId="27146" CreationDate="2017-06-13T17:58:15.347" Score="0" ViewCount="119" Body="&lt;p&gt;I have a problem for which I am trying to determine the most suitable software to use. Here is a brief description of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a mixture of inertial particles A and B with different properties. I want to inject this mixture into a channel filled with fluid, and observe the transport properties. I expect inter-particle collisions and two-way coupling between fluid and particles to be important to the physics we want to capture. I would also like to use non-Newtonian fluids. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for suggestions for commercial or open-source softwares that can tackle this problem. &lt;/p&gt;&#xA;" OwnerUserId="4837" LastActivityDate="2017-06-14T20:46:36.707" Title="Most suitable particle-tracking software" Tags="|software|multiphysics|particle|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27132" PostTypeId="2" ParentId="27128" CreationDate="2017-06-13T18:11:18.670" Score="3" Body="&lt;p&gt;Hardware acceleration means any code run on specialized hardware, as opposed to software run on general purpose CPUs such as standard x86 processors on your PC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose the term is inherently ambiguous (e.g. does &lt;a href=&quot;https://en.wikipedia.org/wiki/Streaming_SIMD_Extensions&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSE&lt;/a&gt; count as hardware acceleration?), and probably means different specific things in different industries. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Hardware_acceleration&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page&lt;/a&gt; is a good starting point. As a member of the scientific computing community, when I hear &quot;hardware acceleration&quot; I instantly think &lt;a href=&quot;https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPU acceleration&lt;/a&gt; of numerical algorithms, and my second thought is FPGA numerical implementations. But for example, &lt;a href=&quot;https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC_products_and_implementations#Prominent_hardware_implementations&quot; rel=&quot;nofollow noreferrer&quot;&gt;hardware implementations of video codecs&lt;/a&gt; can also be considered hardware acceleration. &lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2017-06-13T18:11:18.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27135" PostTypeId="1" CreationDate="2017-06-14T00:23:50.507" Score="0" ViewCount="146" Body="&lt;p&gt;I'm testing a finite volume scheme, Godunov type solver, using a problem with analytical solution. I'm not able to reproduce the solution, which includes source terms. I tried with different mesh sizes and I get the same result but it's not what I should get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I suggested to one of my professors that maybe the code is not actually capable to reproduce this problem, as the paper says, so he asked me to try with different mesh sizes and he said that if it's a numerical thing, the numerical solution should be improved with the mesh refinement, but it didn't. How should I interpret those results?   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the paper I'm working with:&#xA;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999112003464&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0021999112003464&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24560" LastEditorUserId="24560" LastEditDate="2017-07-05T21:34:57.117" LastActivityDate="2017-07-05T21:34:57.117" Title="If a numerical solution remains constant for different grid sizes, what does it mean?" Tags="|fluid-dynamics|finite-volume|numerical-modelling|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27136" PostTypeId="1" CreationDate="2017-06-14T00:49:03.437" Score="2" ViewCount="203" Body="&lt;p&gt;Eigen's &lt;a href=&quot;https://eigen.tuxfamily.org/dox/TopicUsingBlasLapack.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; says that any BLAS back-end can be used (e.g. &lt;a href=&quot;https://eigen.tuxfamily.org/dox/TopicUsingIntelMKL.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MKL&lt;/a&gt;) to perform the actual matrix calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is, however, very vague when discussing the criteria as to which the calculations are offloaded (with added emphasis):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When doing so, a number of Eigen's algorithms are silently substituted with calls to BLAS or LAPACK routines. These substitutions apply only for Dynamic or &lt;strong&gt;large enough objects&lt;/strong&gt; with one of the following four standard scalar types: float, double, complex, and complex. Operations on other scalar types or mixing reals and complexes will continue to use the built-in algorithms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, there is no further guidance as to what is considered &quot;&lt;a href=&quot;https://forum.kde.org/viewtopic.php?f=74&amp;amp;t=123117&quot; rel=&quot;nofollow noreferrer&quot;&gt;large&lt;/a&gt; &lt;a href=&quot;https://forum.kde.org/viewtopic.php?f=74&amp;amp;t=110509#p376921&quot; rel=&quot;nofollow noreferrer&quot;&gt;enough&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What is the criteria to be considered &quot;large enough&quot;&lt;/li&gt;&#xA;&lt;li&gt;Does it vary by platform?&lt;/li&gt;&#xA;&lt;li&gt;Is this dynamically determined?&lt;/li&gt;&#xA;&lt;li&gt;Can it be controlled by the user?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt; I have small (25-50 square) matrices, but are large-ish on the embedded platform (Intel Atom Silvermont) we are using. Since this consumes a significant amount of CPU, I am looking to experiment with BLAS back-ends without having to substantially re-write code.&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="761" LastEditDate="2017-06-14T01:06:37.277" LastActivityDate="2017-06-14T01:06:37.277" Title="What matrix criteria is &quot;large enough&quot; for Eigen to use a BLAS backend?" Tags="|blas|eigen|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27137" PostTypeId="2" ParentId="27135" CreationDate="2017-06-14T01:41:30.833" Score="2" Body="&lt;p&gt;Without more detail on what is &quot;wrong&quot; with your result I would say the follwing:  The numerical implementation is not providing a &quot;verification&quot; result.  In other words your implementation is not solving either the discretized equations correctly (e.g. a bug in your code) or the theoretical basis for your method is incorrect (e.g. the discretization is not consistant with the equations).  &lt;/p&gt;&#xA;" OwnerUserId="567" LastActivityDate="2017-06-14T01:41:30.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27139" PostTypeId="1" CreationDate="2017-06-14T06:35:03.317" Score="0" ViewCount="495" Body="&lt;p&gt;I am writing a 2D pseudo spectral code for turbulence in a box with 1024 grid points with 3/2 aliasing scheme in the vorticity/stream function formulation. the vortices tends to appear very slowly and energy is decaying faster than expected at a Reynolds number of 5*10^4. The faster decay in energy leads to very slow vortices (which is not the case) I am not sure where the problem is. Any help in this would be useful. I have tried different initial conditions like random velocity and gaussian white noise but nothing seems to work.&lt;/p&gt;&#xA;" OwnerUserId="24561" LastActivityDate="2018-03-14T20:10:10.983" Title="Turbulence Pseudospectral Code" Tags="|navier-stokes|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27140" PostTypeId="2" ParentId="27135" CreationDate="2017-06-14T09:13:53.660" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;numerical solution should be improved with the mesh refinement, but it didn't. How should I interpret those results? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, all one can say is it means something's wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reducing $\Delta x$ should decrease the error (compared to the analytic solution). This is because techniques for numerically estimating derivatives get more accurate with smaller $\Delta x$. If your code was perfectly correct, then the only error would be this numerical error (i.e. error due to the grid and timestep not being infinitesimally fine). But because your result isn't improving with finer grids in this way, it shows there's &quot;some other problem&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can analytically verify that the given analytical solution really is a solution to the governing differential equations (i.e. substitute the solution into the governing equation, and work out the derivatives, and then you can confirm that you end up with those given source terms). A symbolic calculator is probably needed, like &lt;code&gt;sympy&lt;/code&gt; (for python).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, there's something wrong with your program - a bug. It could be anything, from the cfd code (e.g. how you discretize), to typos in your program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from standard debugging, code review etc, one way to narrow it down is to simplify the governing equation, by removing some terms. Then, substitute the same given analytic solution from before into the governing equation - which should yield different source terms. (Because the solution is the same as before, the initial and boundary conditions stay the same).  Now, do as you did before, but with the new source terms: run the simulation, and compare the result with the analytic solution. Each time you refine the grid-size (i.e. reduce $\Delta x$), the error should decrease. By seeing which of the simplified governing equations behave coreectly or incorrectly, you can narrow down which terms are causing the problem, and thus which part of the code and/or program are buggy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW This is the Method of Manufactured Solutions, though there's usually more care with how much the error decreases with decreasing $\Delta x$ - it should be $O( (\Delta x)^p )$, where $p$ is the order of the estimation technique (for example finite difference &quot;forward Euler&quot; is 1st order, $p=1$; central difference is 2nd order, $p=2$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because programs usually have bugs, it's unlikely that this behaviour is not caused by a bug. But, if the code and your program are 100% bug-free, then it could be a deeper issue with this scheme and/or this solver, for this particular problem. But it's almost certainly a bug. So look into that first.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastActivityDate="2017-06-14T09:13:53.660" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27141" PostTypeId="2" ParentId="27123" CreationDate="2017-06-14T10:22:15.093" Score="2" Body="&lt;p&gt;Depends on dimensions, but for small problems you can compute an explicit piecewise affine representation of the solution, i.e., a function $x = f(t)$. This field is called multiparametric programming. There is a toolbox in MATLAB called MPT addressing this problem.&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2017-06-14T10:22:15.093" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27143" PostTypeId="1" CreationDate="2017-06-14T10:46:50.067" Score="1" ViewCount="55" Body="&lt;p&gt;I developed a parallel 2D overset mesh (chimera) assembler and I would like to test a large scale problem. I don't want to make up a test case myself but rather want something standard. Would you refer me to such test cases?&lt;/p&gt;&#xA;" OwnerUserId="8103" LastActivityDate="2017-06-14T10:46:50.067" Title="2D Test cases for overset meshes" Tags="|mesh|testing|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27144" PostTypeId="2" ParentId="20901" CreationDate="2017-06-14T14:41:41.913" Score="1" Body="&lt;p&gt;The solution I use is basically an implementation of the trapezium rule and makes use of the &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.misc.logsumexp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scipy.misc.logsumexp&lt;/code&gt;&lt;/a&gt; function to maintain precision. If you have some function &lt;code&gt;lny&lt;/code&gt; that returns the logarithm of &lt;code&gt;y&lt;/code&gt; then you can do, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;from scipy.misc import logsumexp&#xA;import numpy as np&#xA;&#xA;xmin = 1e-15&#xA;xmax = 1e-5&#xA;&#xA;# get values of x spaced logarithmically&#xA;xvs = np.logspace(np.log10(xmin), np.log10(xmax), 10000)&#xA;&#xA;# evaluate your function at xvs&#xA;lys = lny(xvs)&#xA;&#xA;# perform trapezium rule integration&#xA;deltas = np.log(np.diff(xvs))&#xA;logI = -np.log(2.) + logsumexp([logsumexp(lys[:-1]+deltas), logsumexp(lys[1:]+deltas)])&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The value &lt;code&gt;logI&lt;/code&gt; is the log of the integral that you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously this will not work if you need to set &lt;code&gt;xmin = 0&lt;/code&gt;. But, if you have some non-zero positive lower limit to the integral you can just play about with the number of points in &lt;code&gt;xvs&lt;/code&gt; to find a number where the integral converges.&lt;/p&gt;&#xA;" OwnerUserId="24565" LastActivityDate="2017-06-14T14:41:41.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27146" PostTypeId="2" ParentId="27131" CreationDate="2017-06-14T20:46:36.707" Score="3" Body="&lt;p&gt;Do you wish to model the two-way coupled solid-fluid flows or you wish to carry out PIV or tracking on particle images?&#xA;If you wish to model the two-way coupled solid-fluid flows, there are numerous type of models are that valid, most notably Euler-Euler approaches or Euler-Lagrange (CFD-DEM unresolved or resolved) approaches. Euler-Euler approaches are available in numerous open source (OpenFOAM) or commercial (Fluent, Star-CMM+) software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For CFD-DEM modeling you can look at the following open source project:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;CFDEM project&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://www.cfdem.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cfdem.com/&lt;/a&gt;), i was a developer of it during my thesis&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some commercial software are also incredibly powerful for these type of CFD-DEM problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fluent + EDEM&lt;/li&gt;&#xA;&lt;li&gt;Fluent + Rocky 4&lt;/li&gt;&#xA;&lt;li&gt;Star CCM+&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2017-06-14T20:46:36.707" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27149" PostTypeId="2" ParentId="27127" CreationDate="2017-06-15T00:07:40.883" Score="1" Body="&lt;p&gt;Yes -- in VTK you can output data as lines even if you are in 2d (and in 3d). You can see an example here, as part of the deal.II tutorial program step-51: &lt;a href=&quot;https://dealii.org/developer/doxygen/deal.II/step_51.html#Results&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dealii.org/developer/doxygen/deal.II/step_51.html#Results&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-06-15T00:07:40.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27150" PostTypeId="1" CreationDate="2017-06-15T10:36:54.657" Score="1" ViewCount="67" Body="&lt;p&gt;First, let just me stress that I'm not a an expert in computation chemistry, so now the problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have GCMC molecular simulation, in the Grand Canonical ensemble, using the standard metropolis algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would like to port the program to CUDA and run it in parallel.&#xA;There are several techniques to parallelize such simulation to system with local interaction. Unfortunately in our simulation we compute a &lt;strong&gt;total energy for the system taking into account all the particles in the system&lt;/strong&gt;.&#xA;We use a primitive model, and given the elementary charge $e$ and the absolute permittivity of vacuum $\epsilon_0$, the interaction between two charges species $i$ and $j$ separated by a distance $r_{ij}$ is calculated acconrding to:&#xA;$$\begin{cases}&#xA;u^{\text{el}}(r_{ij}) = \frac{q_iq_je^2}{4\pi\epsilon_0\epsilon_sr_{ij}}\\&#xA;u^{\text{hs}}(r_{ij}) = \infty,&amp;amp; \quad \text{if } r_{ij} \leq R_i + R_j\\&#xA;u^{\text{hs}}(r_{ij}) = 0, &amp;amp;\quad \text{if } r_{ij} &amp;gt; R_i + R_j\\&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hard radii sphere is set to $2\mathring{A}$. A one-body external field was use to used to correct for the long range electrostatic interaction and to account for the cell boundary contraints along the z-direction acting as hard walls.:&#xA;$$&#xA;\begin{cases}&#xA;v^{\text{ext}}(z_i) = \infty,&amp;amp; \quad \text{when } z_i \geq L_z||\;z_i \leq &amp;lt;0\\&#xA;v^{\text{ext}}(z_i) = q_ie\varphi^{\text{ext}}(z_i), &amp;amp;\quad \text{when } 0 &amp;lt; z_i &amp;lt; L_z\\&#xA;\end{cases}&#xA;$$&#xA;where $\varphi^{\text{ext}}(z_i)$ is the potential produced by the external charge distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full configurational energy of the system is then:&#xA;$$ U^\text{conf}= \sum _i^{N_0+N_s} v^\text{ext}(z_i) + \sum_{1&amp;lt;j}^{N_0+N_s} u^\text{el}(r_{ij}) + u^\text{hs}(r_{ij})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The heaviest part of the simulation the the calculation of $U^\text{conf}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main iteration is like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i&amp;lt;-ion_0 to last_ion do:&#xA;   actual_energy &amp;lt;- compute_u_conf(i)&#xA;   propose_a_move(i)&#xA;   new_energy &amp;lt;- compute_u_conf(i)&#xA;   accept_or_reject_move(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is where domain decomposition does not fit. Even if we can compute in parallel a move, then the computation of the energy would not be consistent because &lt;strong&gt;others&lt;/strong&gt; particles are moving. This would break the Markov chain and make the system not convergent because the acceptance of the proposed move is based also on the total energy calculated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to calculate the energy only locally and using some sort of mean for the rest of the space?&#xA;Is there a way to parallelize the calculation of the energy being consistent?&#xA;Have someone any idea on who can it be done? &#xA;Any help is welcome.&lt;/p&gt;&#xA;" OwnerUserId="24428" LastActivityDate="2017-06-15T10:36:54.657" Title="Parallelizing molecular simulation with full configuration energy" Tags="|parallel-computing|computational-physics|computational-chemistry|quantum-mechanics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27151" PostTypeId="2" ParentId="27139" CreationDate="2017-06-15T12:11:15.040" Score="0" Body="&lt;p&gt;I'm not sure I understand the specifics of your code. That aside, turbulence is not random and, assuming you're working on an incompressible flow, the velocity field that you produce using random generators is not likely to satisfy the continuity equation and will be rapidly killed by your numerical scheme. This could explain why you're seeing higher than expected turbulence decay rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try to generate a turbulent velocity field &quot;naturally&quot; by tripping a boundary layer flow in a periodic computational domain for example. Otherwise, I believe there are some standard benchmark fields for turbulent decay with a very specific initial conditions but I don't know where to find those.&lt;/p&gt;&#xA;" OwnerUserId="20916" LastActivityDate="2017-06-15T12:11:15.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27152" PostTypeId="1" CreationDate="2017-06-15T13:33:54.600" Score="12" ViewCount="579" Body="&lt;p&gt;In the paper &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0045782509003521&quot; rel=&quot;noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S0045782509003521&lt;/a&gt;, an HDG element-local equation is described on page 584 equation (4), with one of the equations taking the following form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-(u_h,\nabla q)_K = -\left\langle\hat{u}_h \cdot n,  q - \bar{q}\right\rangle_{\partial K}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the variational approximation to the continuous equation $\nabla \cdot u = 0$, with a scalar-valued test function $q$ in a space that makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper defines $$\bar{q} = \frac{1}{|\partial K|} \int_{\partial K} q $$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this interpreted, in a finite element sense? From my understanding, we multiply both sides by a test function $q$ and then attempt to find the solution which satisfies the equation for all possible choices of $q$. How is it possible to modify the test space in this manner?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper also states that this is necessary to enforce the identity &#xA;$$\left\langle\hat{u}_h\cdot n, q - \bar{q}\right\rangle_{\partial K} = 0$$ I agree with this statement, but how might a test function $q - \bar{q}$ be implemented in code? Should I take the basis functions on the element and subtract their mean when assembling the element local linear system?&lt;/p&gt;&#xA;" OwnerUserId="20148" LastActivityDate="2023-04-04T14:02:10.293" Title="DG local equation, how to interpret mean-averaged test function" Tags="|finite-element|discontinuous-galerkin|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27154" PostTypeId="1" CreationDate="2017-06-15T16:16:56.060" Score="4" ViewCount="401" Body="&lt;p&gt;As multigrid methods are known to have grid independent convergence rates with $O(N)$ computational cost, then why would one be interested in using Krylov subspace methods at all, for which convergence rates deteriorate with grid refinement? Are there any problems where Krylov solvers perform better than MG methods?&lt;/p&gt;&#xA;" OwnerUserId="18196" LastEditorUserId="18196" LastEditDate="2017-06-19T09:19:41.093" LastActivityDate="2017-06-19T19:28:18.723" Title="For which problems Krylov subspace methods are preferred over multigrid methods?" Tags="|iterative-method|multigrid|krylov-method|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27155" PostTypeId="1" CreationDate="2017-06-15T16:47:44.463" Score="3" ViewCount="259" Body="&lt;p&gt;I would like to get started solving Fluid-Structure interaction problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already have some experience with Finite Elements, including my own MATLAB and Julia software packages for developing Finite Element programs, i.e., I can confidently approximate the solution to some basic 1D and 2D initial-boundary-value-problems using my code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for resources (books or articles, or blog-posts, or any published material in general) that address issues related to the development of software for solving Fluid-Structure interaction problems. Can you make any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="23712" LastActivityDate="2018-06-11T02:59:44.270" Title="Resources for solving fluid-structure interaction problems" Tags="|finite-element|fluid-dynamics|computational-physics|reference-request|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27156" PostTypeId="1" AcceptedAnswerId="27168" CreationDate="2017-06-15T19:02:16.333" Score="0" ViewCount="78" Body="&lt;p&gt;We have a 3D simulation box (cubic, side $L$) filled with $N$ non-overlapping objects (say spheres). We are interested to study the evolution of the system under an applied pressure in the z-direction (so the pressure has only a non-zero component orthogonal to the top face of the cube). &#xA;In a simulation, let's for clarity assume it is a Monte Carlo type of simulation (so at each MC step objects are moved randomly such that no overlap is introduced), one often hears the following: &quot;pressure is modeled/mimicked by re-scaling simply the distances between the objects&quot;. I understand this intuitively, as it is just saying that: applying pressure reduces the volume, namely $\Delta V = \Delta z L_y L_x,$ this in turn means in the applied direction of the pressure the objects get closer and closer.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The question is more at the implementation level: How does one implement such a scheme, i.e., make the mapping of pressure to re-scaled distances? Are there existing techniques that are often employed for such purposes?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, does it matter for such scheme if the objects are anisotropic?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also any references (textbooks or papers) that you see fit for tackling such questions are very welcome here. &lt;/p&gt;&#xA;" OwnerUserId="24573" LastActivityDate="2017-06-16T14:50:36.187" Title="Applying pressure on simulation box" Tags="|simulation|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27158" PostTypeId="1" AcceptedAnswerId="27162" CreationDate="2017-06-15T23:21:24.650" Score="0" ViewCount="382" Body="&lt;p&gt;I am trying to apply a polynomial interpolation to 340 points in a 4D space, i.e.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x,y,z)=k\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to understand is this: if I use a 6th order polynomial I will end up with 343 degrees of freedom [(order+1)^(dimensions)] for my polynomial, which is a number greater than the actual available number of points. Is the problem underdetermined? Can I state that I cannot use such order for the interpolation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, if I use a 5th order polynomial, this has 216 degrees of freedom, that is a number smaller than my number of points. Is the problem overdetermined? Can I anyway use this order to interpolate my data without problems?&lt;/p&gt;&#xA;" OwnerUserId="24577" LastEditorUserId="9667" LastEditDate="2017-06-16T00:19:58.250" LastActivityDate="2017-06-16T05:09:57.780" Title="Underdetermined/overdetermined polynomial interpolation" Tags="|interpolation|polynomials|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27159" PostTypeId="2" ParentId="27154" CreationDate="2017-06-16T02:05:37.097" Score="6" Body="&lt;p&gt;The answer depends somewhat on the discretization; for example, some boundary integral discretizations result in very well-conditioned matrices, for which a Krylov solver works just fine without having to introduce the multi-level machinery of multigrid.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For ill-conditioned matrices arising from finite difference or finite element methods, Krylov solvers are almost always paired with a preconditioner, for which multigrid methods can work very well.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastActivityDate="2017-06-16T02:05:37.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27160" PostTypeId="2" ParentId="27154" CreationDate="2017-06-16T04:00:45.163" Score="1" Body="&lt;p&gt;There are multiple answers to your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For one, as others have said, multigrid (MG) and krylov based solvers can work really well &lt;em&gt;together&lt;/em&gt;, so this is a good reason to still use krylov solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second good reason to use krylov solvers is if the matrix to be inverted is ill conditioned ( for example due to discontinuous material properties ). In this case, MG does not guarantee convergence whereas Congugate Gradient method (e.g.) will.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2017-06-16T04:00:45.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27161" PostTypeId="1" AcceptedAnswerId="27163" CreationDate="2017-06-16T04:34:56.723" Score="4" ViewCount="237" Body="&lt;p&gt;Given two sets of (two-dimensional) points, say, &lt;span class=&quot;math-container&quot;&gt;$A=\{a_1,a_2,\ldots,a_{n_a}\}$&lt;/span&gt; and (you guessed it) &lt;span class=&quot;math-container&quot;&gt;$B=\{b_1,b_2,\ldots,b_{n_b}\}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$d^2_{i,j}=\mid a_i-b_j\mid^{\ 2}$&lt;/span&gt; the &quot;matrix&quot; (not necessarily square) of distances between them, I want to map &lt;span class=&quot;math-container&quot;&gt;$m:A\to B$&lt;/span&gt; (i.e., with each &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;-point &lt;span class=&quot;math-container&quot;&gt;$a\in A$&lt;/span&gt; associate a corresponding &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;-point &lt;span class=&quot;math-container&quot;&gt;$m(a)\in B$&lt;/span&gt;) such that the following...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The maximal point-to-point &lt;span class=&quot;math-container&quot;&gt;$d_{i,m(i)}$&lt;/span&gt; distance should be minimal. And once you've accomplished that first &lt;span class=&quot;math-container&quot;&gt;$m:i\to m(i)$&lt;/span&gt; correspondence, each set gets one point smaller, &lt;span class=&quot;math-container&quot;&gt;$A\to A\backslash\{a_i\}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B\to B\backslash\{b_{m(i)}\}$&lt;/span&gt;. And now, the second point-to-point correspondence made by &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; should similarly minimize the maximal distance on these one-point-smaller sets. And then iteratively. So, &lt;strong&gt;my question: what's an efficient algorithm for this?&lt;/strong&gt; (&lt;span class=&quot;math-container&quot;&gt;$n_a,n_b\sim10^4$&lt;/span&gt;, way too large for brute force)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it's not necessary that &lt;span class=&quot;math-container&quot;&gt;$n_a=n_b$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$n_a\lt n_b$&lt;/span&gt;, discard any of the excess &lt;span class=&quot;math-container&quot;&gt;$n_b-n_a$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;-points you like such that the set of all those maximal distances is minimized. That is, just do the iteration, and when you're done, just discard the &quot;unused&quot; &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;-points. On the other hand, if &lt;span class=&quot;math-container&quot;&gt;$n_b\lt n_a$&lt;/span&gt; just discard the &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;-points resulting from the &lt;em&gt;first&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$n_a-n_b$&lt;/span&gt; iteration steps (i.e., again minimize the set of maximal distances). At least, I think that's the right &quot;discard&quot; scheme, but please correct me if I'm wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;point&quot; (sorry:) of all this is, as per the subject, to &quot;morph&quot; one bitmapped image into another, whereby each &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;-point denotes a source image pixel, and each &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;-point denotes a target image pixel (and if &lt;span class=&quot;math-container&quot;&gt;$n_b\gt n_a$&lt;/span&gt;, I'll just inconspicuously/randomly &quot;pop&quot; a few extra &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;-pixels into place during each frame of the morph). Since this kind of morphing is all over the place, I'd have thought I could easily google algorithms/code/etc. But I strangely couldn't get google to cough it up. So if you're familiar with this kind of stuff and can just point me in the right direction, that would be great, too. Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; E d i t&lt;br&gt;-----------------&lt;/strong&gt;&lt;br&gt;&#xA;Thanks again, @LedHead and @BrunoLevy. It's been quite some time (better late&#xA;than never), but I finally got around to implementing this stuff, more or less&#xA;as originally intended. Here's an illustration, followed by some&#xA;explanatory remarks...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hS34L.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hS34L.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interpolation's nearest neighbor as suggested by you, one-to-many if&#xA;the source image has fewer pixels than the target (and vice-versa if&#xA;vice-versa). But the path from source--&gt;target pixel isn't a straight line.&#xA;Instead, it's a bezier curve whose first control point is the source row,col and whose last control point is the target's. In-between control points determine the shape of the path. They're basically chosen randomly, but constrained within the&#xA;row,col-bounds of the image. However, when specifying the gif, you can choose&#xA;multipliers for those constraints. For the illustrated gif, I chose constraints keeping all control points near the center. And that's why,&#xA;in this case, the morphing first contracts all the pixels inwards,&#xA;and then expands them outwards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, bezier also includes straight-line interpolation between&#xA;nearest neighbors, simply by choosing just two control points, which&#xA;are then necessarily source coordinates and target coordinates. The exact&#xA;same gif, changed only in that way, looks like...&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nk7ZW.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nk7ZW.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23314" LastEditorUserId="23314" LastEditDate="2020-05-25T08:35:45.620" LastActivityDate="2020-05-25T08:35:45.620" Title="Optimally &quot;morph&quot; one set of points into another" Tags="|computational-geometry|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="27162" PostTypeId="2" ParentId="27158" CreationDate="2017-06-16T05:09:57.780" Score="0" Body="&lt;p&gt;I'm assuming you're doing your interpolation something like the following way: Given a set of $(x,y,z)$ coordinates $(x_i,y_i,z_i)$ and corresponding value $f_i$ for $i=1,2,...,N$, find the coefficients $c_j$ of the multivariate polynomial &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x,y,z) = \sum_{j=1}^M c_j x^{\alpha_j} y^{\beta_j} z^{\gamma_j}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;such that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x_i, y_i, z_i) = f_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $\alpha_j$, $\beta_j$, and $\gamma_j$ define the orders of each monomial term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This yields a matrix equation of the following form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A c = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_{ij} = x_i^{\alpha_j} y_i^{\beta_j} z_i^{\gamma_j}$. This is a multivariate version of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vandermonde_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vandermonde matrix&lt;/a&gt;. If $A$ is square, then solving for $c$ gives your solution. Obviously, if there are multiple coordinates at the same location, $A$ is singular. I think there are other requirements to guarantee non-singular $A$. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you have more unknowns ($M$) than constraints ($N$) then your system is underdetermined, which means that there are an infinite number of solutions that satisfy your constraints. It's not that you &lt;em&gt;can't&lt;/em&gt; use this polynomial order, it's just that you can't find a unique solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have more constraints ($N$) than unknowns ($M$) then your system is overdetermined, so there is in general no solution which satisfies all of the constraints. You could solve for $c$ in the least-squares sense, by solving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c = (A^T A)^{-1} A^T f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to guarantee that $f(x_i,y_i,z_i)=f_i$ exactly, then this won't work for you. But if you just need it to be close, then this could be the ticket.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also start with the underdetermined case and pick some of the highest order terms to remove until $M=N$. Then (assuming $A$ isn't singular) your solution is unique and your constraints are exactly satisfied.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Some nitpicking:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say this is interpolation in &quot;4D space&quot;. I suppose you're considering $f$ to be the fourth dimension, but that's confusing. It's really interpolation of a field $f$ in 3D $(x,y,z)$ space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say that a 6th order multivariate polynomial in 3 dimensions has 343 degrees of freedom. Technically, a $p^{th}$ other polynomial contains no terms with order higher than $p$, or in other words $\alpha_j+\beta_j+\gamma_j \le p$ for all $j$. There are ${p+d \choose d}$ degrees-of-freedom, which for $d=3$ is $\frac{(p+3)(p+2)(p+1)}{6}$, and for $p=6,d=3$ is 84. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you're talking about is something akin to &lt;a href=&quot;https://en.wikipedia.org/wiki/Trilinear_interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;trilinear&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Tricubic_interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;tricubic&lt;/a&gt; functions, but for higher order. I don't know what you would actually call them, and I also don't really know why you would use them over the normal multivariate polynomial complete functions.  &lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2017-06-16T05:09:57.780" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27163" PostTypeId="2" ParentId="27161" CreationDate="2017-06-16T07:21:52.640" Score="5" Body="&lt;p&gt;You may consider numerical optimal transport.&#xA;It does not exactly fit your specification, but for your image morphing application it may be well suited.&#xA;In the discrete setting, given your two set of points $A$ and $B$, optimal transport computes an assignment matrix $m$ between $A$ and $B$ that has unit row sums and unit column sums (a so-called bi-stochastic matrix) and that minimizes $\sum_i \sum_j m_{ij} c_{ij} $ for some costs $c_{ij}$ (for instance, $c_{ij}$ = the Euclidean distance between point $i$ and point $j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Optimal Transport is a general theory, that applies in differente settings (comprising continuous functions and discrete objects like your point sets). There are good books on the general theory [1] and how to use it in practice [2]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your discrete case, the standard algorithm to be used is the so-called &quot;auction algorithm&quot; [3], but it is very slow. Thus, some accelerated counterparts were invented [4], based on the observation that adding a regularization term (entropy) makes it possible to use a much faster algorithm (Sinkhorn iteration). It is also possible to approximate transport using sums of Gaussians [5]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interesting alternative is to compute a continuous function that approximates one of the pointsets (while keeping the other one discrete). This (assymetric) setting is thus called &quot;semi-discrete&quot;, and can be solved numerically using efficient algorithms [6] (and my own one in [7]). It is implemented in my GEOGRAM software library [8].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Optimal Transport Old and New - Cedric Villani&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Optimal Transport for Applied Mathematicians - Fillipo Santambrogio&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://en.wikipedia.org/wiki/Auction_algorithm&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Auction_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;https://arxiv.org/abs/1306.0895&quot; rel=&quot;noreferrer&quot;&gt;https://arxiv.org/abs/1306.0895&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] Displacement Interpolation Using Lagrangian Mass Transport, Bonneel et.al, ACM Transactions on Graphics (SIGGRAPH ASIA), 2011&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] A multiscale approach to optimal transport, Quentin Merigot, Eurographics&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[7] A numerical algorithm for semi-discrete optimal transport in 3D, Bruno Lévy, M2AN&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[8] Geogram: &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;noreferrer&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2017-06-16T07:21:52.640" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27164" PostTypeId="1" AcceptedAnswerId="27170" CreationDate="2017-06-16T09:00:00.847" Score="0" ViewCount="220" Body="&lt;p&gt;I need a fast and accurate method to calculate 3d spherical volume integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have pre-calculated data of high precision that just needs a few trivial manipulations before each integration step - in other words function calls are mostly just array look ups and multiplication and so are relatively cheap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data is evenly spaced in spherical polar coordinates: 4 samples per degree for theta and phi, 500 radial samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need good precision, but I also need to perform several million such integrals, so speed is an important concern. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithm is most suitable for this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bonus points for a link to an implementation in C++!&lt;/p&gt;&#xA;" OwnerUserId="24581" LastEditorUserId="9667" LastEditDate="2017-06-16T14:39:35.070" LastActivityDate="2017-06-16T15:04:29.897" Title="Spherical volume integral from pre-calculated points - which algorithm is best?" Tags="|c++|quadrature|integration|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27167" PostTypeId="2" ParentId="27164" CreationDate="2017-06-16T13:45:28.433" Score="0" Body="&lt;p&gt;If the subvolumes are evenly distributed and you can assume the function is constant inside them you probably only need a lookup table for the volume size in the radius and do a weighted sum. If the assumptions are not true it will probably not be very accurate though :)&lt;/p&gt;&#xA;" OwnerUserId="19947" LastActivityDate="2017-06-16T13:45:28.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27168" PostTypeId="2" ParentId="27156" CreationDate="2017-06-16T14:50:36.187" Score="1" Body="&lt;p&gt;You are looking for numerical schemes to sample from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Isothermal%E2%80%93isobaric_ensemble&quot; rel=&quot;nofollow noreferrer&quot;&gt;isobaric-isothermal ensemble&lt;/a&gt; (also known as NPT ensemble). This amounts to sampling from a specific probability distribution (determined by the integrand in the partition function corresponding to the NPT ensemble).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find a discussion of the theory and practice of simulations in the NPT ensemble using Monte Carlo methods in Section 5.4 of Frenkel, D., &amp;amp; Smit, B. (2002). Understanding Molecular Simulation (Second). Elsevier. &lt;a href=&quot;http://doi.org/10.1016/B978-012267351-1/50000-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doi.org/10.1016/B978-012267351-1/50000-6&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another good reference would be Chapter 5 of Tuckerman, M. (2010). Statistical Mechanics: Theory and Molecular Simulation. Oxford University Press.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-06-16T14:50:36.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27169" PostTypeId="1" CreationDate="2017-06-16T14:53:53.640" Score="1" ViewCount="539" Body="&lt;p&gt;I am trying to fit a polynomial through 340 points in a 3D space, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x,y,z) = k$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I asked previously about the theory behind polynomial interpolation here -&gt;&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/27158/underdetermined-overdetermined-polynomial-interpolation/27162?noredirect=1#comment48379_27162&quot;&gt;Polynomial interpolation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Few interesting points came out from the discussion, but I would like to understand how to correctly implement them in Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am writing a script in Python to compute the Vandermonde matrix for a polynomial of a generic order $p$. Now the implementation problem arises.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;First method&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I have 340 points $(x,y,z)$. Supposing I have a polynomial of 6th order I have&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(order+1) ^ number_of_dimensions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;degrees of freedom for my polynomial. Therefore, it means that I have 343 degrees of freedom. Since I have 340 points I cannot use such order because otherwise the problem is underdetermined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use a 5th order polynomial I have 216 degrees of freedom, and in this case the problem is overdetermined and the polynomial is an approximating polynomial. In a preliminary phase I do not need the exact interpolation, since I can solve the problem with a complementary solution, but do you think it is feasible the solution given?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Second method&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As suggested by the user who helped me in the previous question, actually a polynomial of 6th order has 84 degrees of freedom. This is related to the constraint given by&#xA;$$a+b+c&amp;lt;p$$&#xA;with $a,b,c$ the exponents in each term $x^a y^b z^c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question which arises is if it more feasible to create a Vandermonde matrix which respects this constraint. Does it give an improved approximated solution compared to a full Vandermonde matrix?&lt;/p&gt;&#xA;" OwnerUserId="24577" LastEditorUserId="9667" LastEditDate="2017-06-16T17:05:28.617" LastActivityDate="2017-06-16T17:05:28.617" Title="Polynomial approximation - Vandermonde matrix creation - precision" Tags="|python|interpolation|polynomials|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27170" PostTypeId="2" ParentId="27164" CreationDate="2017-06-16T14:57:51.447" Score="2" Body="&lt;p&gt;So you want to evaluate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F=\int_0^r \int_0^{2\pi} \int_{0}^\pi f(r,\phi,\theta) \, r^2 \, \sin (\theta) \ dr \, d\phi \, d\theta&#xA;$$&#xA;and you have your function $f$ available on equidistant gridpoints $r_i$, $\phi_j$ and $\theta_k$. Call&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f_{ijk} =  f(r_i, \phi_j, \theta_k)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plain simple approach is then just to use the trapezoidal rule (I'll omit the one-halves at the endpoints):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F \approx \sum_{ijk}f_{ijk} \, r_i^2 \, \sin(\theta_i)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... or, if you're feeling for higher order, use another Newton-Cotes formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to note that once. For other approaches, see the comment section.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I've just noticed you were asking for the &quot;best&quot; method, so this will probably be quite boring for you.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2017-06-16T15:04:29.897" LastActivityDate="2017-06-16T15:04:29.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27171" PostTypeId="1" AcceptedAnswerId="27172" CreationDate="2017-06-16T15:47:31.687" Score="1" ViewCount="4174" Body="&lt;p&gt;The &lt;code&gt;scipy.special.sph_harm&lt;/code&gt; function evaluates a spherical harmonic function at a point. Does Scipy provide any functions to compute the spherical harmonic coefficients for a data set?&lt;/p&gt;&#xA;" OwnerUserId="9728" LastActivityDate="2017-06-16T17:12:35.483" Title="Computing spherical harmonic coefficients using Scipy" Tags="|python|spherical-harmonics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27172" PostTypeId="2" ParentId="27171" CreationDate="2017-06-16T17:08:01.213" Score="3" Body="&lt;p&gt;Given samples of a function $f(\theta,\phi)$, you will need to numerically evaluate the integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_0^\pi\int_0^{2\pi}f(\theta,\phi)\left[Y_n^{m}(\theta,\phi)\right]^*\sin\theta d\phi d\theta\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to obtain the $n,m$th coefficient of your expansion. Hopefully your samples are at some convenient locations on the unit sphere, either uniformly sampled or at Gauss-Legendre nodes.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastEditorUserId="9667" LastEditDate="2017-06-16T17:12:35.483" LastActivityDate="2017-06-16T17:12:35.483" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27174" PostTypeId="1" AcceptedAnswerId="27176" CreationDate="2017-06-16T19:00:46.783" Score="8" ViewCount="1012" Body="&lt;p&gt;I am trying to compute the eigenfunctions of the Laplace operator, i.e. finding $u$ in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ -\nabla^2 u = \lambda u .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now I am trying to do this in 1D, so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla^2 = \partial_{xx} .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the finite difference method. My boundary conditions are $u=0$ at $x = -1, 1$. Here is my code, and the results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.sparse.linalg import eigs&#xA;import matplotlib.pyplot as plt&#xA;&#xA;n = 200&#xA;h = 2/(n-1)     # domain for x and y is [-1, 1]&#xA;&#xA;L = np.diag(np.ones(n-1), k=-1) + np.diag(-2*np.ones(n)) + \&#xA;        np.diag(np.ones(n-1), k=1)&#xA;L *= -1/h**2&#xA;&#xA;eigvals, eigvecs = eigs(L)&#xA;&#xA;eig = np.real(eigvecs[:, 0])&#xA;&#xA;x = np.linspace(-1, 1, num=n)&#xA;&#xA;plt.plot(x, eig)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EmlS1.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EmlS1.png&quot; alt=&quot;plot of results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone see what I am doing wrong, or give suggestions as to how I might be able to fix my code?&lt;/p&gt;&#xA;" OwnerUserId="24583" LastEditorUserId="9667" LastEditDate="2017-06-16T19:43:27.067" LastActivityDate="2020-02-09T16:29:19.437" Title="Correct eigenfunctions of Laplace operator by Finite Differences" Tags="|finite-difference|python|eigenvalues|eigensystem|laplacian|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27176" PostTypeId="2" ParentId="27174" CreationDate="2017-06-16T19:36:12.987" Score="9" Body="&lt;p&gt;You should specify the eigenvalues you want with &lt;code&gt;which=&quot;SM&quot;&lt;/code&gt;, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check the following snippet. I also changed the solver, since your system is symmetric.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.sparse.linalg import eigsh&#xA;import matplotlib.pyplot as plt&#xA;&#xA;n = 200&#xA;h = 2/(n-1)     # domain for x and y is [-1, 1]&#xA;&#xA;L = np.diag(np.ones(n-1), k=-1) - np.diag(2*np.ones(n)) + \&#xA;        np.diag(np.ones(n-1), k=1)&#xA;L = -L/h**2&#xA;&#xA;eigvals, eigvecs = eigsh(L, which=&quot;SM&quot;)&#xA;&#xA;eig = eigvecs[:, 0]&#xA;&#xA;x = np.linspace(-1, 1, num=n)&#xA;&#xA;plt.plot(x, eig)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rd7cw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rd7cw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-02-27T12:29:14.803" LastActivityDate="2018-02-27T12:29:14.803" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27178" PostTypeId="1" AcceptedAnswerId="27179" CreationDate="2017-06-16T22:38:13.340" Score="22" ViewCount="3993" Body="&lt;p&gt;Are there any reasons for why one should choose high order implicit Runge Kutta (IMRK) over BDF time stepping?  BDF seems much easier to me as $q$ stage IMRK needs $q$ linear solves per time step.  Stability for BDF and IMRK looks to be a moot point.  I can't find any resources comparing/contrasting implicit time steppers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it helps, the end goal is for me to select a high order implicit time stepper for advection-diffusion PDE.&lt;/p&gt;&#xA;" OwnerUserId="11604" LastActivityDate="2020-07-19T17:46:57.940" Title="BDF vs implicit Runge Kutta time stepping" Tags="|time-integration|runge-kutta|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27179" PostTypeId="2" ParentId="27178" CreationDate="2017-06-16T23:52:40.737" Score="44" Body="&lt;p&gt;Yes, there aren't too many resources on this for some reason. For a very long time, the standard goto was &amp;quot;just use BDF methods&amp;quot;. This mantra was set in stone for few historical reasons: for one Gear's code was the first widely available stiff solver, and for another the MATLAB suite didn't/doesn't include an implicit RK method. However, this heuristic isn't always correct, and I would say from testing it's usually wrong. Let me explain in detail.&lt;/p&gt;&#xA;&lt;h3&gt;BDF methods have a high fixed cost&lt;/h3&gt;&#xA;&lt;p&gt;Adaptive timestepping and adaptive order in BDF methods has a really high cost. Coefficients have to be recalculated, or values need to be interpolated to different times. There has been a lot of work going into making the current BDF codes do better here (there are two well-known &amp;quot;forms&amp;quot; for implementation trying to handle this issue differently), but it's actually just a very difficult software engineering problem. Because of this, actually most of the BDF codes are all descendants from Gear's original code: Gear's, vode, lsoda, Sundial's CVODE, even the DAE solvers DASKR and DASSL are descendants from the same code.&lt;/p&gt;&#xA;&lt;p&gt;What this means is that, if you problem is &amp;quot;too small&amp;quot;, the high fixed cost really matters and Implicit RK methods will do better.&lt;/p&gt;&#xA;&lt;h3&gt;High order BDF methods are very unstable for complex eigenvalues&lt;/h3&gt;&#xA;&lt;p&gt;BDF methods allow you to control the maximum order and make it more conservative for a reason: the higher order BDF methods cannot handle even &amp;quot;medium sized&amp;quot; complex eigenvalues very well. So in these cases, in order to be stable, they have to drop their order. This is the reason why the 6th order BDF method, while technically stable, is often ignored because even the 5th order already has issues here (and the 6th order is even less stable). Only up to 2nd order is A-stable, so it can always fallback to there, but then stepping is error dominated.&lt;/p&gt;&#xA;&lt;p&gt;So when using BDF codes on non-trivial problems, you're not getting 5th order all of the time. Oscillations cause its order to drop.&lt;/p&gt;&#xA;&lt;h3&gt;BDF methods have a high starting cost&lt;/h3&gt;&#xA;&lt;p&gt;BDF methods which are autostarting have a high starting cost. They start with a small Euler step, then a small BDF-2 step, etc. For shorter time integrations, this has a very non-negligible effect. If you are stopping often, say for event handling, this will heavily impair the efficiency of the codes. This is one reason why you rarely if ever see multistep methods used in heavy event handling or delay equation situations (delay equations restart at each high enough order discontinuity, which is propagated at each &lt;span class=&quot;math-container&quot;&gt;$i\tau$&lt;/span&gt; for every delay-time &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt;)&lt;/p&gt;&#xA;&lt;h3&gt;BDF methods, being multistep methods, have the best scaling&lt;/h3&gt;&#xA;&lt;p&gt;But this is what BDF methods have going for them: they scale the best. This is because they have the least function calls. This erroneously has led to the idea that you should &amp;quot;always default to BDF methods&amp;quot; (which for the reason above is not always true for stability and efficiency reasons), but it does mean that, if the function call &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is expensive enough, then BDF methods will be best. Generally this means that for large enough and stiff enough PDE discretizations, BDF methods will be the most efficient. That said, this &amp;quot;large enough&amp;quot; is very problem dependent, and I have found PDE discretizations which do better with &lt;code&gt;radau&lt;/code&gt; methods of up to 1,000's of spatial variables, so YMMV. Only way to know is to test.&lt;/p&gt;&#xA;&lt;h3&gt;What benchmarks are available?&lt;/h3&gt;&#xA;&lt;p&gt;Hairer's book and DiffEqBenchmarks (explained below) are probably the best in terms of easily available work-precision diagrams. &lt;a href=&quot;http://www.springer.com/us/book/9783540604525&quot; rel=&quot;noreferrer&quot;&gt;Hairer's Solving Ordinary Differential Equations II&lt;/a&gt; has a bunch of work-precision diagrams on pages 154 and 155. The results on the problems he chose match what I stated above for the reasons I stated above: implicit RK will be more efficient if the problem is not &amp;quot;sufficiently large&amp;quot;. Another interesting thing to note is that high order Rosenbrock methods come out as the most efficient in many of his tests (like &lt;code&gt;Rodas&lt;/code&gt;) in the regime where error is higher, and the implicit RK &lt;code&gt;radau5&lt;/code&gt; is the most efficient at lower errors. But his tests problems are mostly not discretizations of large PDEs, so take the points above into account.&lt;/p&gt;&#xA;&lt;h3&gt;How do you test/benchmark?&lt;/h3&gt;&#xA;&lt;p&gt;I like to test this with &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl&quot; rel=&quot;noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt; in Julia (disclaimer: I am one of the developers). This is in Julia. Programming language should really get a note here. Remember that as the cost of the function call increases, BDF methods fair better. In R/MATLAB/Python, the user's function is the only R/MATLAB/Python code that the optimized solvers have to actually use: if you're using SciPy or Sundials wrappers, it's all C/Fortran code except for the function you pass. This means that, in dynamic languages (which aren't Julia), BDF methods do better than they normally would because the function calls are very unoptimized (this is probably the reason why Shampine included &lt;code&gt;ode15s&lt;/code&gt; in the MATLAB suite, but no implicit RK method).&lt;/p&gt;&#xA;&lt;p&gt;So Julia is a good testing ground for the more optimal case since if the function call is type-stable, your &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is as fast/efficient as any C/Fortran function. What's nice with DifferentialEquations.jl is you can test between &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html&quot; rel=&quot;noreferrer&quot;&gt;a ton of algorithms&lt;/a&gt; just by switching one line of code. For an &lt;code&gt;ODEProblem&lt;/code&gt; prob, you can switch between the two with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@time sol = solve(prob,CVODE_BDF())&#xA;@time sol = solve(prob,radau())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where the first uses Sundials' &lt;code&gt;CVODE&lt;/code&gt; (BDF method), and the second uses Hairer's &lt;code&gt;radau&lt;/code&gt; (implicit RK).&lt;/p&gt;&#xA;&lt;p&gt;For any &lt;code&gt;ODEProblem&lt;/code&gt;, you can use the benchmarking tools to see how the different algorithms scale for different adaptive tolerances. &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;noreferrer&quot;&gt;Some results are posted at DiffEqBenchmarks.jl&lt;/a&gt;. For example, &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffODE/ROBER.ipynb&quot; rel=&quot;noreferrer&quot;&gt;on the ROBER problem&lt;/a&gt; (system of 3 stiff ODEs) you can see that Sundials actually is unstable and diverges with a high enough tolerance (whereas the other methods converge just fine), showing the note above about stability issues. &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffODE/VanDerPol.ipynb&quot; rel=&quot;noreferrer&quot;&gt;On the Van Der Pol problem&lt;/a&gt;, you can see it's more of a wash. I have a lot more than I haven't posted, but probably won't get to it until I finish up some higher order Rosenbrock methods (Rodas is the Fortran version of those).&lt;/p&gt;&#xA;&lt;p&gt;(Note: those stiff benchmarks need updating. For one, the text needs updating since for some reason ODE.jl's methods diverge...)&lt;/p&gt;&#xA;&lt;h3&gt;Extrpolation methods and RKC&lt;/h3&gt;&#xA;&lt;p&gt;There are also extrapolation methods like &lt;code&gt;seulex&lt;/code&gt; which are made for stiff problems. These are &amp;quot;infinite adaptive order&amp;quot;, but that only means they are asymtopically good when you're looking for very low error (i.e. looking to solve stiff problems at lower than &lt;code&gt;1e-10&lt;/code&gt; or so, in which case you can probably use an explicit method though). However, in most cases they are not as efficient and should be avoided.&lt;/p&gt;&#xA;&lt;p&gt;Runge-Kutta Chebyschev is an explicit method which also works on stiff problems that you should consider. I don't have it wrapped in DifferentialEquations.jl yet so I don't have any hard evidence myself of how it fairs.&lt;/p&gt;&#xA;&lt;h3&gt;Other methods to consider: specialized methods for stiff PDEs&lt;/h3&gt;&#xA;&lt;p&gt;Should probably make a note that high order Rosenbrock methods do really well, many times even better, for small-medium sized stiff problems when you can easily compute the Jacobian. However, for some PDEs, I believe advection-diffusion problems fall into this category, Rosenbrock can lose some orders of accuracy. Also, they need very accurate Jacobians in order to keep their accuracy. In Julia this is easy because the solvers come with symbolic and autodifferentiation which can be correct to machine epsilon. However, things like MATLAB's &lt;code&gt;ode23s&lt;/code&gt; can have issues because these implementations use finite differencing. For BDF and implicit RK methods, errors in the Jacobian lead to slower convergence, while for Rosenbrock, since these are not implicit equations and are rather RK methods with Jacobian inversions in there, these just lose order of accuracy.&lt;/p&gt;&#xA;&lt;p&gt;Other methods to look at are exponential RK methods, exponential time-differencing (ETD), implicit integration factor (IIF), and exponential Rosenbrock methods. The first three make use of the fact that, in many PDE discretizations,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ u_t = Au + f(u) $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a linear operator, and this linearity allows you to exactly solve the &lt;span class=&quot;math-container&quot;&gt;$Au$&lt;/span&gt; operator part. The downside is, the resulting method has to use &lt;span class=&quot;math-container&quot;&gt;$e^A$&lt;/span&gt; which is dense even if &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; was sparse, and so there's a lot of work in Krylov methods. Still, these are A-stable methods which avoid implicit equations and so they can be very efficient for PDE discretizations.&lt;/p&gt;&#xA;&lt;p&gt;Exponential Rosenbrock methods don't need the &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, and instead split by &lt;span class=&quot;math-container&quot;&gt;$Ju + g(u)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is the Jacobian, and &lt;span class=&quot;math-container&quot;&gt;$f = Ju + g$&lt;/span&gt;, but it's the same idea.&lt;/p&gt;&#xA;&lt;h3&gt;Still other methods: the latest creations&lt;/h3&gt;&#xA;&lt;p&gt;Methods which are fully implicit obviously do well for stiff equations. If the PDE is not large enough, you cannot effectively &amp;quot;parallelize in space&amp;quot; enough to make use of HPCs. Instead, you can create parallel-in-time discretizations which are fully implicit and thus good for stiff equations, yet parallel to make full use of hardware. &lt;a href=&quot;https://computation.llnl.gov/projects/parallel-time-integration-multigrid&quot; rel=&quot;noreferrer&quot;&gt;XBraid&lt;/a&gt; is a solver that uses a technique like this, and the main methods are PFFAST and parareal methods. DifferentialEquations.jl is developing a neural net method which works similarly.&lt;/p&gt;&#xA;&lt;p&gt;Again, this is optimal when you don't have a large enough spatial discretization to efficiently parallelize, but have the resources for parallel computation available.&lt;/p&gt;&#xA;&lt;h3&gt;Conclusion: take asymptotic considerations with a grain of salt&lt;/h3&gt;&#xA;&lt;p&gt;For a very long time in numerical diffeq history, there was the idea that extrapolation methods were the best because they can technically achieve any order. This means that, for a small enough &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt;, they will be the most efficient (so, if you're looking for a very precise solution, yes!). However, almost no one actually works in the regime where these are efficient, which is why people generally don't use extrapolation methods anymore. This is a word of caution: take asymptotic considerations with a grain of salt&lt;/p&gt;&#xA;&lt;p&gt;BDF methods are asymptotically the best, but in most cases you probably aren't working in that regime. But if the spatial discretization has enough points, BDF methods can efficiently parallelize in space (by parallelizing the linear solving) and will have the least function calls, and thus will do the best. But if your PDE discretization is not large enough, take a good look at implicit RK, Rosenbrock, exponential RK, etc. methods. Using a software suite like DifferentialEquations.jl that makes it easy to swap between the different methods can be really helpful for you to understand what method does best for your problem domain, since in many cases it cannot be known in advance.&lt;/p&gt;&#xA;&lt;p&gt;[If you have any example problems you want added to the testing suite, please feel free to help get something in there. I want to keep a very comprehensive resource on this. I hope to have all of Hairer's benchmarks in runnable notebook forms &amp;quot;soon&amp;quot;, and would like other problems which are representative of scientific fields. Any help is appreciated!]&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T17:46:57.940" LastActivityDate="2020-07-19T17:46:57.940" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="27180" PostTypeId="1" CreationDate="2017-06-17T00:46:12.780" Score="1" ViewCount="76" Body="&lt;p&gt;Let $u_n$ be an array containing discrete values of the function $u(x,y)$. Performing a 2D FFT to this array we obtain $\hat{u_n}$ representing the values of $\hat{u}(k_x,k_y)$. I would like to perform something like the 2/3 rule often used in pseudospectral methods for fluid dynamics: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat{u}(k)=0\qquad if \quad k&amp;gt;(2/3)k_{max}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to do this in 1D, but I am having trouble implementing it in 2D. I have tried setting to zero all $\hat{u}(k_x,k_y)$ for which $k=\sqrt{k_x^2+k_y^2}&amp;gt;(2/3)k_{max}$ but this method is affecting the energy of my lower frequencies, which I would like to be preserved without numerical dissipation. I would like to obtain something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6vr02.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6vr02.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which I have only been able to obtain setting a filter in terms of the dependent magnitude (Normalized Energy), and not regarding the wavenumber $k=\sqrt{k_x^2+k_y^2}$ as I would want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if someone pointed me in the right direction to tackle my problem.&lt;/p&gt;&#xA;" OwnerUserId="23747" LastActivityDate="2017-06-17T00:46:12.780" Title="Perfect filtering of high frequencies in 2D FFT (Multidimensional 2/3 Rule)" Tags="|fluid-dynamics|fourier-analysis|spectral-method|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27181" PostTypeId="1" AcceptedAnswerId="27184" CreationDate="2017-06-17T12:31:58.170" Score="0" ViewCount="188" Body="&lt;p&gt;I am a beginner to Python. Currently I'm writing a code for developing a simple solver for non-linear ODE systems with initial value. The equations of the system are &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/r9hzp.jpg&quot; alt=&quot;as follow&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function of myu is evaluated first to get the value of myu, then used in dX/dt, dS/dt, and dDO/dt. At the next step, myu is evaluated again to get its new value based on new value of S and DO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using General Linear Method (GLM), proposed by J. C. Butcher, as my method. This method use a transition matrix, which value and size depends on numerical method that we use. In this case, I use Runge Kutta Cash-Karp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While you may find in the equation that D is also a function, here I set the value of D as a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In initialization, the value of h is set first, to get the number of step. I create a vector named 'initValue', with 8 columns and 4 rows, consist of values of k for each equations (row 1 to 6), initial value for fourth order of the Runge Kutta (row 7. I set it to 0 since it just act as a 'predictor'), and initial value for each equations (row 8).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transition matrix is created based on the GLM, which values inside it comes from the constants of stage equations (to find the value of k1 to k6) and step equations (to find the solutions) of Runge Kutta Cash-Karp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the looping, at the very first time, I simply add the initial values to an array named 'result'. At the first step, I simply multiple the transition matrix with vector 'initValue'. And at the next until final step, I initialize the 'initValue' based on result from previous step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm looking for is the solution which should look like &lt;a href=&quot;https://i.stack.imgur.com/END6b.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&#xA;My code works if h is less than 1. I compare my result with result from scipy.integrate.odeint. But when I set h bigger than 1, it show different result than the result it should be. For example, in the code, I set h = 100, which means that it will only display the initial value and final value (when time = 100). While X and S should going upward, and DO and Xr going downward, mine was the opposite of them. The result from odeint when h is set to bigger than 1 show the same result with the expected solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need help to fix my code so it can display the expected solution for any value of h.&lt;/p&gt;&#xA;" OwnerUserId="24586" LastEditorUserId="24586" LastEditDate="2017-06-20T15:42:10.687" LastActivityDate="2017-06-20T15:42:10.687" Title="ODE System doesn't work when step size (h) is bigger than 1" Tags="|python|ode|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27182" PostTypeId="1" AcceptedAnswerId="27211" CreationDate="2017-06-17T13:13:12.293" Score="3" ViewCount="1152" Body="&lt;p&gt;I am trying to solve an ill-posed linear system of equations. The particular system has 160 equations and 400 variables. Moreover, the condition number of the left hand side matrix is of order $10^{16}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across two methods to solve this problem: constrained optimization and regularization methods. My question is what are the pros and cons of each method; which one should I prefer? Any website or paper explaining the same will be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="24587" LastEditorUserId="24587" LastEditDate="2017-06-21T12:46:29.487" LastActivityDate="2017-06-23T07:41:25.673" Title="Regularization vs constrained optimization of an ill posed tomography problem" Tags="|linear-algebra|matrix|constrained-optimization|inverse-problem|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27183" PostTypeId="1" AcceptedAnswerId="27187" CreationDate="2017-06-17T15:32:22.557" Score="2" ViewCount="137" Body="&lt;p&gt;Suppose you have a classification problem, now what if I implement and train all classification models like logistic regression, KKN, naive Bayes, decision tree or random forest on the training data set (we can change our data set to big or small according to the model). And now what I do is keep these classifiers for various model as the node for the neural network and update their weights on the premise of who gives the correct answer. Now I am just a noob when it comes to machine learning, but can someone please settle my curiosity.&lt;/p&gt;&#xA;" OwnerUserId="24590" LastEditorUserId="9667" LastEditDate="2017-06-17T15:41:59.130" LastActivityDate="2017-06-20T15:26:15.363" Title="Why can't we just use machine learning to select which model to use for a given model?" Tags="|machine-learning|neural-network|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27184" PostTypeId="2" ParentId="27181" CreationDate="2017-06-17T16:38:16.147" Score="2" Body="&lt;p&gt;Increasing the timestep isn't always sensible either, because you are increasing the numerical error. Instead, you should be setting a requested value for the error, and letting the integrator make the stepsizes which are required to achieve that error estimate (adaptivity), or you need to balance between numerical error and runtime yourself. That said...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I need help to fix my code so it can display the expected solution for any value of h.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sorry, that's just not possible. This is due to numerical stability issues. Explicit integrators (like explicit RK) have a maximum stepsize that is &quot;stable&quot;, above which small perturbations are amplified causing the solver to diverge to infinity. This maximum stepsize is a property of the solver and the &quot;stiffness&quot; of the equation (related to the maximum and minimum eigenvalues of the equation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't want to be limited by &lt;code&gt;dt&lt;/code&gt; at all, you need to use an integrator designed to handle stiffness. &quot;Unlimited &lt;code&gt;dt&lt;/code&gt;&quot; only occurs in A-B-L-stable methods, which is an unreasonable goal. Things like the 2nd order Rosenbrock method, Midpoint method, or BDF2 display this though. You can't get higher than 2nd order with that property though, meaning error tends to be higher. So in order to do this well, you need to be adapting both the order and the timestep.&#xA;This is one reason why I would recommend sticking to using some dedicated ODE solver software: it has a ton of bells and whistles to make this work. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-06-17T16:38:16.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27185" PostTypeId="1" CreationDate="2017-06-17T17:18:15.030" Score="5" ViewCount="118" Body="&lt;p&gt;What are the techniques on inverse problems to discover the distribution of parameters from a discrete set of values? For instance, I know that my domain where the PDE is defined is made up of materials A, B and C, but I do not know how they are distributed inside the domain. Working with discrete optimization algorithms seems unfeasible. What are the techniques to solve such a problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I will describe the problem for just two materials, as it seems that the strategy is different than for multimaterials, according to Clason's comments (bang-bang vs multibang)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\underset{u \in U}{\text{min}} \frac{1}{2} \lVert F(u) - y^{\delta} \rVert_Y \\&#xA;$$&#xA;where&#xA;$$&#xA;U = \{ u \in X : u \in \{ u_1, u_2 \} ~\text{pointwise}\}\\&#xA;F: X \rightarrow Y&#xA;$$&#xA;$X$ and $Y$ are function spaces and $y^{\delta} \in Y$&lt;/p&gt;&#xA;" OwnerUserId="7023" LastEditorUserId="7023" LastEditDate="2017-07-05T17:12:10.457" LastActivityDate="2017-07-05T17:12:10.457" Title="Inverse problems with a discrete set of known parameters" Tags="|inverse-problem|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="27187" PostTypeId="2" ParentId="27183" CreationDate="2017-06-18T00:23:11.687" Score="2" Body="&lt;p&gt;You can. This is called an ensemble model. For example, a linear regression between the solutions of different predictive models is a way to take a weighted average of different models. Normally, the winners of all of the machine learning contests use ensemble models since they eek out a bit more accuracy. However, they are a lot more costly to train.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="393" LastEditDate="2017-06-20T15:26:15.363" LastActivityDate="2017-06-20T15:26:15.363" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27188" PostTypeId="1" CreationDate="2017-06-18T15:25:56.800" Score="2" ViewCount="1130" Body="&lt;p&gt;I am trying to compute the one-dimensional energy spectra for my channel-flow simulation. I have already written a post-processing script to achieve this; however, I need to validate my code before proceeding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do so, I am taking the two-point cross-correlation (per given plane) vector from a DNS database. Then, I am trying to apply my code and plot against the 1D energy spectra provided by the same database and in the same (homogeneous) direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am always getting the wrong output, but I cannot figure out what went wrong. I have looked into several reference books and forum posts here and there with no tangible result as of yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The steps I am following are: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a two-point cross-correlation vector in the streamwise (i.e. x-direction) and homogeneous direction, I am applying the following (e.g. in Matlab form):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% where Ruu is the correlation vector from the DNS database&#xA;N = length(Ruu);&#xA;Nk = 2^nextpow2(N);&#xA;&#xA;% Fourier transform data&#xA;Bx1 = zeros(Nk, 1); &#xA;&#xA;for k=1:Nk&#xA;    for n=1:N&#xA;        Bx1(k) = Bx1(k) + (1/N)*Ruu(n)*exp(-2i*pi*(k-1)*(n-1)/N);&#xA;    end&#xA;end&#xA;&#xA;% wavenumbers initialization&#xA;kx = zeros(Nk, 1);&#xA;% total distance between correlations used&#xA;Lx = (max(x) - min(x));&#xA;for n=1:Nk&#xA;    % streawise coordinates to wavenumber&#xA;    kx(n) = pi*(n-1)/Lx;&#xA;end&#xA;% calculate 1D streamwise energy spectra&#xA;Eu = Bx1.*conj(Bx1); &#xA;&#xA;% show only first half due to symmetry&#xA;loglog(kx(1:end/2), 2*Eu(1:end/2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The resulting figure, in case you were wondering, is depicted below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2UqAX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2UqAX.png&quot; alt=&quot;1D energy spectra&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output of such a figure is nowhere near what I am looking for. Can someone shed some insight on the issue? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Reference DNS Data&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Moser, Robert D.; Kim, John; Mansour, Nagi N.&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1063/1.869966&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Direct numerical simulation of turbulent channel flow up to &lt;span class=&quot;math-container&quot;&gt;$Re_{\tau} = 590$&lt;/span&gt;&lt;/strong&gt;&lt;/a&gt;, Phys. Fluids 11, No. 4, 943-945 (1999). &lt;a href=&quot;https://zbmath.org/?q=an:1147.76463&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL1147.76463&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18387" LastEditorUserId="20688" LastEditDate="2019-06-04T22:54:40.063" LastActivityDate="2023-06-21T16:05:15.753" Title="1-D turbulent energy spectra in homogeneous direction (non-isotropic)" Tags="|fluid-dynamics|computational-physics|numerics|fourier-analysis|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="27189" PostTypeId="1" AcceptedAnswerId="27190" CreationDate="2017-06-18T17:00:40.163" Score="1" ViewCount="902" Body="&lt;p&gt;Suppose we have the &lt;strong&gt;inhomogeneous&lt;/strong&gt; advection equation&#xA;$$\left(\frac{\partial}{\partial x}+\frac{1}{c}\frac{\partial}{\partial t}\right)u(t,x)=v(t,x)$$&#xA;for $u,v:\mathbb{R}\times\mathbb{R}\to\mathbb{R}$ (with boundary conditions not yet specified).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that we had no $v$, i.e. the homogeneous part of the equation, the Crank-Nicolson method would yield&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-c\frac{\mu}{4}u^{n+1}_{\ell-1}+u^{n+1}_{\ell}+c\frac{\mu}{4}u^{n+1}_{\ell+1}=c\frac{\mu}{4}u^n_{\ell-1}+u^n_{\ell}-c\frac{\mu}{4}u^n_{\ell+1},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mu=\frac{\Delta t}{\Delta x}$ and $u^n_\ell=u(n\Delta t,\ell\Delta x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to deal with the inhomogeneity in these schemes though.&lt;/p&gt;&#xA;" OwnerUserId="24427" LastActivityDate="2017-06-18T18:33:05.487" Title="Crank-Nicolson method for inhomogeneous advection equation" Tags="|pde|finite-difference|hyperbolic-pde|advection|crank-nicolson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27190" PostTypeId="2" ParentId="27189" CreationDate="2017-06-18T18:33:05.487" Score="5" Body="&lt;p&gt;Your equation can be written in the following fashion (any spatial derivative approximation is valid), once space is discretised:&#xA;$$\frac{1}{c}\frac{du_i}{dt}=-\left(\frac{\partial u}{\partial x}\right)_i(t) + v_i(t) \tag{*}$$&#xA;Keep in mind that $v_i(t) = v(x_i,t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the system of equations depends only on time $t$ you can apply Crack Nicholson method to solve the system of ODEs. Fo simplicity let the rght hand side of $(*)$ be named as:&#xA;$$F_i(t) = v_i(t)-\left(\frac{\partial u}{\partial x}\right)_i(t) $$&#xA;and therefore $(*)$ cab ne rewritten as follows:&#xA;$$\frac{1}{c}\frac{du_i}{dt}=F_i(t) \tag{**}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Cranck-Nicholson method can be deduced from a $\theta$ method setting $\theta=1/2$, therefore, the time discretised equation $(**)$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{c}\frac{u_i^{n+1}-u_i^{n}}{\delta t}=\theta\, F_i(t^{n+1}) + (1-\theta)F_i(t^n)=\frac{1}{2}\left[ F_i(t^{n+1})+F_i(t^n)\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do not forget that we had the expression for $F_i(t)$, in which the function $v_i(t)$ is also present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope you now know how to deal with that term.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2017-06-18T18:33:05.487" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="27192" PostTypeId="2" ParentId="14433" CreationDate="2017-06-19T03:21:37.710" Score="6" Body="&lt;p&gt;@DavidKetcheson's answer hits the big points: you can always construct methods of high enough order using extrapolation, that's a very pessimistic bound and you can always do a whole lot better, all the good ones are derived by hand (with the help of some computer algebra tools), no lower bound is known, and the highest order methods are due to Feagin. Given some of the comments, I wanted to round out the answer with a discussion of the current state-of-the-art tableaus in the field.&lt;/p&gt;&#xA;&lt;p&gt;If you want a compendium of RK tableaus, you can find one &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqDevTools.jl/blob/master/src/ode_tableaus.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this Julia code&lt;/a&gt;. Citations for the paper that they came from are in the docstrings for the tableau constructors. The developer documentation for DifferentialEquations.jl &lt;a href=&quot;http://devdocs.juliadiffeq.org/latest/internals/tableaus.html#Explicit-Runge-Kutta-Methods-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;lists all of these tableaus as available for use&lt;/a&gt;, and you &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqDevTools.jl/blob/master/test/ode_tableau_convergence_tests.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;can see here&lt;/a&gt; that these are all tested using Travis and AppVeyor continuous integration suites to make sure that not only the order conditions are satisfied, but they they actually achieve the requested convergence (verification testing). From these, you can see that there are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;5 order 9 methods&lt;/li&gt;&#xA;&lt;li&gt;6 order 10 methods&lt;/li&gt;&#xA;&lt;li&gt;2 order 12 methods&lt;/li&gt;&#xA;&lt;li&gt;1 order 14 method&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(that I could find that were published). Again, all derived by hand.&lt;/p&gt;&#xA;&lt;p&gt;The convergence tests show that some of the derivations were not carried out to high enough precision to work for more than 64-bit numbers (they are commented &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqDevTools.jl/blob/master/test/ode_tableau_convergence_tests.jl#L356&quot; rel=&quot;nofollow noreferrer&quot;&gt;like this&lt;/a&gt;). So that's an interesting quirk to be aware of: at these high orders you usually only get coefficients that &amp;quot;to an error &lt;code&gt;x&lt;/code&gt;&amp;quot; satisfy the order conditions, but when using arbitrary precision arithmetic you can actually detect these bounds. So the precision to which you carry out the coefficients do matter, and you should choose it to cover the precision you wish to test (/ use, of course).&lt;/p&gt;&#xA;&lt;p&gt;If you want a bunch of stability plots, you can just &lt;code&gt;plot(tableau)&lt;/code&gt; using the Plots.jl recipe. A good set of notes which has a lot of this written down &lt;a href=&quot;http://www.peterstone.name/Maplepgs/RKcoeff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be found on Peter Stone's website&lt;/a&gt; (go below and click on say the order 10 schemes and you'll get a bunch of PDFs). When developing DifferentialEquations.jl, I created that set of tableaus to systematically go through them on test problems / look at the analytical indicators to see which ones should be included in the main library. &lt;a href=&quot;http://devdocs.juliadiffeq.org/latest/internals/notes_on_algorithms.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;I made some quick notes here&lt;/a&gt;. As you can see &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html#OrdinaryDiffEq.jl-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;from the algorithms included in the main library&lt;/a&gt;, the ones that I found worthwhile were the Verner and Feagin methods. The Verner 9th order method is the highest order method with an interpolant matching the order too. That's something to recognize: the Feagin methods do not have a matching interpolant (though you can bootstrap Hermite, but that's really inefficient).&lt;/p&gt;&#xA;&lt;p&gt;Since they are all implemented with very efficient implementations, you can play around with them yourself and see how much the different features actually matter. &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqTutorials.jl/blob/master/ExtraODEFeatures/Feagin%27s%20Order%2010%2C%2012%2C%20and%2014%20methods.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a Jupyter notebook which shows the Feagin methods in use&lt;/a&gt;. Notice that the convergence plot really is going to &lt;code&gt;1e-48&lt;/code&gt; error. High order methods are only more efficient than lower order methods when you really need a very very low tolerance. You can find some benchmarks that use some of them &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;at DiffEqBenchmarks.jl&lt;/a&gt;, though when they are used it's usually the 9th order Verner method, and usually showing that the benchmark is not in the regime where this high of order is efficient.&lt;/p&gt;&#xA;&lt;p&gt;So if you want to play around and work with some high order methods, RK-Opt is what I found is a good tool for deriving some (as @DavidKetcheson mentioned), and DifferentialEquations.jl has all of the published methods (I think?) implemented so that you can easily test/benchmark against them. However, unless you find an assumption that can be dropped, from my tests I haven't been able to find something that beats the Verner (orders 6-9) and Feagin (orders 10+) methods. YMMV though, and I'd love to see more research in this.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:11:19.513" LastActivityDate="2020-07-19T18:11:19.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27193" PostTypeId="2" ParentId="2377" CreationDate="2017-06-19T03:44:44.323" Score="3" Body="&lt;p&gt;I found this topic and thought I should share &lt;a href=&quot;https://github.com/JuliaPlots/Plots.jl/issues/621&quot; rel=&quot;nofollow noreferrer&quot;&gt;the developer issue page&lt;/a&gt; for adding this to the Julia library Plots.jl. We tried a bunch of techniques to see what would give good results, starting from the notes on Mathematica's implementation. Adding some pruning, a small perturbation to not start exactly at the interval endpoints, a recursion limit, and a dual mesh error estimator were all necessary to &quot;get it just right&quot;. The thread also points you to the open source code for the implementation. So it did take a bit of tweaking, but adding those features made it pretty robust (according to the tests, as shown in the thread).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-06-19T03:44:44.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27194" PostTypeId="1" AcceptedAnswerId="27217" CreationDate="2017-06-19T04:21:50.737" Score="2" ViewCount="203" Body="&lt;p&gt;Are there error estimators and research on adaptive timestepping schemes for SSPRK methods? My Googling could not uncover papers which addressed this, so I was wondering if there was anything particularly difficult about deriving such error estimators. I think the difficult thing is that the SSP property only occurs when $$\Delta t &amp;lt; c \Delta t_{FE}$$ (&lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/07070485X&quot; rel=&quot;nofollow noreferrer&quot;&gt;in @DavidKetcheson's notation&lt;/a&gt;), and so part of the adaptivity would likely need to ensure it stays in this region. Are there any published attempts to do this? And any research on the effects of PI-controllers in this case?&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="123" LastEditDate="2018-07-04T21:58:27.877" LastActivityDate="2018-07-04T22:00:10.197" Title="Adaptive Timestepping for Stong Stability Preserving (SSP) Runge-Kutta Methods" Tags="|ode|adaptive-timestepping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27195" PostTypeId="2" ParentId="19253" CreationDate="2017-06-19T10:58:29.657" Score="4" Body="&lt;p&gt;Here is a method that can handle column additions:  &lt;a href=&quot;http://pcc.byu.edu/resources.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pcc.byu.edu/resources.html&lt;/a&gt;. I updated it to handle row additions: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [Up1,Sp,Vp1] = addblock_svd_update2( Uarg, Sarg, Varg, Aarg, force_orth )&#xA;&#xA;  U = Varg;&#xA;  V = Uarg;&#xA;  S = Sarg;&#xA;  A = Aarg';&#xA;&#xA;  current_rank = size( U, 2 );&#xA;  m = U' * A;&#xA;  p = A - U*m;&#xA;  P = orth( p );&#xA;  P = [ P zeros(size(P,1), size(p,2)-size(P,2)) ];&#xA;  Ra = P' * p;&#xA;  z = zeros( size(m) );&#xA;  K = [ S m ; z' Ra ];&#xA;  [tUp,tSp,tVp] = svds( K, current_rank );&#xA;  Sp = tSp;&#xA;  Up = [ U P ] * tUp;&#xA;  Vp = V * tVp( 1:current_rank, : );&#xA;  Vp = [ Vp ; tVp( current_rank+1:size(tVp,1), : ) ];&#xA;  if ( force_orth )&#xA;    [UQ,UR] = qr( Up, 0 );&#xA;    [VQ,VR] = qr( Vp, 0 );&#xA;    [tUp,tSp,tVp] = svds( UR * Sp * VR', current_rank );&#xA;    Up = UQ * tUp;&#xA;    Vp = VQ * tVp;&#xA;    Sp = tSp;&#xA;  end;&#xA;&#xA;  Up1 = Vp;&#xA;  Vp1 = Up;&#xA;&#xA;return;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Test it with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = [[ 2.180116   2.493767  -0.047867;&#xA;       -1.562426  2.292670   0.139761;&#xA;       0.919099  -0.887082  -1.197149;&#xA;       0.333190  -0.632542  -0.013330]];&#xA;&#xA;A = [1 1 1];&#xA;X2 = [X; A];&#xA;[U,S,V] = svds(X);&#xA;&#xA;[Up,Sp,Vp] = addblock_svd_update2(U, S, V, A, true);&#xA;&#xA;Up&#xA;Sp&#xA;Vp&#xA;&#xA;[U2,S2,V2] = svds(X2);&#xA;U2&#xA;S2&#xA;V2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will see U,S,V results on both sides are the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the Python version,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.linalg as lin&#xA;&#xA;def  addblock_svd_update( Uarg, Sarg, Varg, Aarg, force_orth = False):&#xA;  U = Varg&#xA;  V = Uarg&#xA;  S = np.eye(len(Sarg),len(Sarg))*Sarg&#xA;  A = Aarg.T&#xA;&#xA;  current_rank = U.shape[1]&#xA;  m = np.dot(U.T,A)&#xA;  p = A - np.dot(U,m)&#xA;  P = lin.orth(p)&#xA;  Ra = np.dot(P.T,p)&#xA;  z = np.zeros(m.shape)&#xA;  K = np.vstack(( np.hstack((S,m)), np.hstack((z.T,Ra)) ))&#xA;  tUp,tSp,tVp = lin.svd(K);&#xA;  tUp = tUp[:,:current_rank]&#xA;  tSp = np.diag(tSp[:current_rank])&#xA;  tVp = tVp[:,:current_rank]&#xA;  Sp = tSp&#xA;  Up = np.dot(np.hstack((U,P)),tUp)&#xA;  Vp = np.dot(V,tVp[:current_rank,:])&#xA;  Vp = np.vstack((Vp, tVp[current_rank:tVp.shape[0], :]))&#xA;&#xA;  if force_orth:&#xA;    UQ,UR = lin.qr(Up,mode='economic')&#xA;    VQ,VR = lin.qr(Vp,mode='economic')&#xA;    tUp,tSp,tVp = lin.svd( np.dot(np.dot(UR,Sp),VR.T));&#xA;    tSp = np.diag(tSp)&#xA;    Up = np.dot(UQ,tUp)&#xA;    Vp = np.dot(VQ,tVp)&#xA;    Sp = tSp;&#xA;&#xA;  Up1 = Vp;&#xA;  Vp1 = Up;&#xA;&#xA;  return Up1,Sp,Vp1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="621" LastEditorUserId="621" LastEditDate="2017-06-23T10:06:19.330" LastActivityDate="2017-06-23T10:06:19.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27196" PostTypeId="1" CreationDate="2017-06-19T12:24:01.283" Score="2" ViewCount="213" Body="&lt;p&gt;I'm searching for semi-empirical quantum chemistry solver which would be easy to integrate into my own software. I found a few implementations which can be in principle used e.g. &lt;a href=&quot;http://openmopac.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MOPAC&lt;/a&gt;, &lt;a href=&quot;https://orcaforum.cec.mpg.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ORCA&lt;/a&gt;, &lt;a href=&quot;http://alma.karlov.mff.cuni.cz/bio/99_Studenti/99_Later/07_VMD_Structure_Building/00_Software/Amber15_sqm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SQM&lt;/a&gt; some of which are integrated e.g. into &lt;a href=&quot;https://autodesk.github.io/molecular-design-toolkit/features.html#molecular-mechanics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Molecular Design Toolkit&lt;/a&gt;. But I don't want to depend on 1GB packages when I need only a few functions. Also I wan't something where I can dig-in and modify it more easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is something more convenient with respect to following aspects: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As much open as possible (GPL or better MIT licence)&lt;/li&gt;&#xA;&lt;li&gt;compact and simple to understand code-base (small number of lines, clear readability - no fortran77, minimum of external dependencies)&lt;/li&gt;&#xA;&lt;li&gt;C/C++ (optionally with python interface) ... I don't want to bother with frotran-C or fortran-python interface.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is in principle continuation of my previous question&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/26847/simple-open-source-quantum-chemistry-or-dft-code-in-c-c&quot;&gt;Simple open-source Quantum chemistry or DFT code in C/C++&lt;/a&gt;. While &lt;a href=&quot;https://github.com/evaleev/libint/wiki/using-modern-CPlusPlus--interface-v22&quot; rel=&quot;nofollow noreferrer&quot;&gt;Libint&lt;/a&gt; is very nice software, full Hartree-Fock is too slow for me. I really would like something cheaper, like semiempirical methods and DFTB.&lt;/p&gt;&#xA;" OwnerUserId="4696" LastActivityDate="2017-06-19T12:24:01.283" Title="lightweight implementation of semiempirical quantum chemistry (e.g. MNDO,AM-1,PM3)" Tags="|c++|c|quantum-mechanics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27197" PostTypeId="1" CreationDate="2017-06-19T15:26:53.863" Score="3" ViewCount="212" Body="&lt;p&gt;So I have been investigating a problem to get a glider with control of its elevator to fly as far as possible from any given initial state. To keep this simple, we will view this in 2D space with the following differential equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dot{\boldsymbol{q}} = \dot{\begin{bmatrix} &#xA;x \\&#xA;y \\&#xA;\theta \\&#xA;\phi \\&#xA;\dot{x}\\&#xA;\dot{y}\\&#xA;\dot{\theta}&#xA;\end{bmatrix}} &amp;amp;= \begin{bmatrix} &#xA;\dot{x} \\ &#xA;\dot{y} \\ &#xA;\dot{\theta} \\&#xA;u \\&#xA;-\left(f_w \sin\theta + f_e \sin\left(\theta + \phi\right)\right)m^{-1} \\&#xA;\left(f_w \cos\theta + f_e \cos\left(\theta + \phi\right)\right)m^{-1} - g\\&#xA;\left(f_e (l \cos\phi + l_e ) - f_w l_w\right) I^{-1}&#xA;\end{bmatrix} \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the following relationships:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;f_w &amp;amp;= \rho S_w |\dot{\boldsymbol{x}}_w|^2 \sin\alpha_w \\&#xA;f_e &amp;amp;= \rho S_e |\dot{\boldsymbol{x}}_e|^2 \sin\alpha_e \\&#xA;\alpha_w &amp;amp;= \theta - \tan^{-1}\frac{\dot{y}_w}{\dot{x}_w} \\&#xA;\alpha_e &amp;amp;= \theta + \phi - \tan^{-1}\frac{\dot{y}_e}{\dot{x}_e} \\&#xA;\dot{x}_w &amp;amp;= \dot{x} + l_w \dot{\theta} \sin\theta \\&#xA;\dot{y}_w &amp;amp;= \dot{y} - l_w \dot{\theta} \cos\theta \\&#xA;\dot{x}_e &amp;amp;= \dot{x} + l\dot{\theta}\sin\theta + l_e\left(\dot{\theta} + \dot{\phi}\right)\sin\left(\theta + \phi\right)\\&#xA;\dot{y}_e &amp;amp;= \dot{y} - l\dot{\theta}\cos\theta - l_e\left(\dot{\theta} + \dot{\phi}\right)\cos\left(\theta + \phi\right) \\&#xA;\dot{\boldsymbol{x}}_w &amp;amp;= \dot{x}_w \hat{e}_x + \dot{y}_w \hat{e}_y \\&#xA;\dot{\boldsymbol{x}}_e &amp;amp;= \dot{x}_e \hat{e}_x + \dot{y}_e \hat{e}_y \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is the control, essentially a choice for $\dot{\phi}$, $\phi$ is the relative elevator angle with respect to the pitch angle $\theta$, $x$ and $y$ are the horizontal and vertical positions, $\dot{x}$ and $\dot{y}$ are the horizontal and vertical speeds, $\dot{\theta}$ is the angular velocity of the glider, $f_w$ is the net aerodynamic force magnitude from the main wing, $f_e$ is the net aerodynamic force magnitude from the elevator wing, $g$ is gravitational acceleration constant, and the other constants are tied to glider physical traits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values being used for the various constants are the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;m &amp;amp;= 0.05 \\&#xA;g &amp;amp;= 9.81 \\&#xA;\rho &amp;amp;= 1.292 \\&#xA;S_w &amp;amp;= 0.1 \\&#xA;S_e &amp;amp;= 0.025 \\&#xA;I &amp;amp;= 6 \cdot 10^{-3} \\&#xA;l &amp;amp;= 0.35 \\&#xA;l_w &amp;amp;= -0.03 \\&#xA;l_e &amp;amp;= 0.05&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the initial condition I primarily use to test is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\boldsymbol{q}_0 = \begin{bmatrix} &#xA;x_0 \\&#xA;y_0 \\&#xA;\theta_0 \\&#xA;\phi_0 \\&#xA;\dot{x}_0\\&#xA;\dot{y}_0\\&#xA;\dot{\theta}_0&#xA;\end{bmatrix} &amp;amp;= \begin{bmatrix} &#xA;0 \\ &#xA;2 \\ &#xA;0 \\&#xA;0 \\&#xA;6 \\&#xA;0\\&#xA;0&#xA;\end{bmatrix} \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am experimenting with using Dynamic Programming to tackle this problem when $u$ is constrained such that $-1 \leq u \leq 1$. Since Dynamic Programming is memory intensive for large state spaces, I recognized that for an optimal distance controller, I don't actually need the first two states, $x$ and $y$. With this change, I defined $\boldsymbol{q} = \lbrack \theta, \phi, \dot{x}, \dot{y}, \dot{\theta} \rbrack^T$ along with the associated differential equations truncation. I also define the discrete dynamical system using the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\boldsymbol{q}_{k+1} &amp;amp;= \boldsymbol{q}_k + \Delta t \dot{\boldsymbol{q}}\left(\boldsymbol{q}_k, u_k\right)\\&#xA;&amp;amp;= f\left(\boldsymbol{q}_k,u_k\right)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To go along with this change, I made the overall optimization problem to maximize the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;V &amp;amp;= \sum_{i=1}^N \Delta t \dot{x}_i - \gamma \dot{y}^2_i\\&#xA;\text{subject to}&amp;amp; \begin{matrix} &#xA;-1 \leq u_k \leq 1 \\ &#xA;\boldsymbol{q}_{k+1} = f\left(\boldsymbol{q}_k,u_k\right)&#xA;\end{matrix}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because the cost function should approximate the value for $x$ at the end of a flight, which is obviously what I would have optimized if using the full system of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After doing some experiments, it seems the cost function chosen doesn't really work well for reasons I am unsure of. However, if I change the cost function to the following, it performs much better:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;V &amp;amp;= \sum_{i=1}^N \theta_i^2 + \gamma \dot{\theta}_i^2&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some $0 \leq \gamma \lt 1$. I chose this second cost function thinking one thing that might help a long flight is the glider remaining level instead of diving too soon and losing a lot of energy. It works decent, but I am still wondering why the first isn't doing too well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;With all this said, is there any problems with the first optimization formulation that stand out?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2017-06-26T18:11:47.260" LastActivityDate="2017-06-26T18:11:47.260" Title="Optimal Control using Dynamic Programming - Optimizing for Furthest Distance" Tags="|optimization|constrained-optimization|optimal-control|dynamic-programming|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="27198" PostTypeId="2" ParentId="27154" CreationDate="2017-06-19T16:30:15.427" Score="4" Body="&lt;p&gt;This question is pretty well discussed in literature. However, there are lots of questions concerning multigrid on SciComp, so I decided to compose more or less detailed answer.  &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;I. When multigrid DOES work fine as a stand–alone solver&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;MG works fine for rather simple elliptic problems. In fact, both MG and PCG (with MG cycles as inner iterations) usually give similar results for such problems in terms of iterations / working time and stay robust as you refine the mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solved diffusion–reaction 2D problem &#xA;$$&#xA;		-\epsilon \, \Delta u + u = f, \quad \mathbf x \in \Omega, \\&#xA;		u|_{\Gamma_D} = g_D, \\		&#xA;		\left( \epsilon \, \nabla u \cdot \hat{\mathbf n} + u \right)|_{\Gamma_R} = g_R&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;a href=&quot;https://github.com/CATSPDEs/CATSPDEs/blob/master/sln/FEM%20for%20Diffusion%20Reaction%20Problem/user.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;CATS’ PDEs&lt;/a&gt; on hierarchy of triangular meshes $\Omega_1 \subset \Omega_2 \subset \dots \subset \Omega_8$ of unit square with P1 FEs.&#xA;I obtained the following results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yByNN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yByNN.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\text{MG(V)}$ and $\text{MG(W)}$ are V– / W–cycles as stand–alone solvers; $\mathbf P_{\text{MG(V)}}\text{CG}$ is preconditioned conjugate gradient with one V–cycle as an inner iteration; $\mathbf P_{\text{ILU(0)}}\text{CG}$ is preconditioned conjugate with incomplete $(\mathbf L + \mathbf I) \, \mathbf D \, (\mathbf I + \mathbf L^T)$ preconditioner (zero fill–in).&lt;/li&gt;&#xA;&lt;li&gt;I used 2 pre– and post–smoothing SSOR iterations inside MG–cycles. &lt;/li&gt;&#xA;&lt;li&gt;Numb of DOFs on the coarsest grid: 17; numb of DOFs on the finest grid: 656 897.&lt;/li&gt;&#xA;&lt;li&gt;In the table: “numb of iterations, $m$” / “solving time (in seconds), $t$.” I always started with a random initial guess $\mathbf x_0$ in order to introduce components of different frequencies in the initial error vector $(\mathbf x - \mathbf x_0)$; I stopped solving when $||\mathbf r_m||/||\mathbf r_0|| &amp;lt; 10^{-12}$. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here’s comparison in terms of time $t$ as system size $n$ grows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Juxtv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Juxtv.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, you can see that $\text{MG}$ allows you to solve this problem in $O(n)$ time; so does $\mathbf P_{\text{MG}}\text{CG}$, but there is no difference between them in this case. I think this makes people ask questions like&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As multigrid methods are known to have grid independent convergence rates with $O(n)$ computational cost, then why would one be interested in using Krylov subspace methods at all […]?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h1&gt;II. When multigrid DOES NOT work fine as a stand–alone solver&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Consider the following cases.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;II.1 More complex elliptic problems&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;It is usual to have jumping coefficients. For one, consider the following magnetostatic Poisson problem&#xA;$$&#xA;	\nabla \cdot \left( \frac{1}{\hat{\mu}} \, \nabla A_z \right) = \mu_0 \, J_z.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vnX6S.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vnX6S.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is natural to have different values for magnetic permeability $\hat{\mu}$ for a magnet (gray box on the picture above) and air; current density $J_z$ is only nonzero within wires (red and blue squares on the picture above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MG as a stand–alone solver loses its robustness (or even diverges) with respect to jump sizes for such problems, although the configuration is very simple. &lt;strong&gt;However, it still works nice as a preconditioner for Krylov solvers&lt;/strong&gt;. If I have some free time, I will add a similar table for this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of simple elliptic problems for which MG does not work. It is useful to look through a paper I mentioned in &lt;a href=&quot;https://scicomp.stackexchange.com/a/27118/21916&quot;&gt;this answer&lt;/a&gt; in order to get a deeper understanding.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;II.2 “Real–life” problems&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Complex real–life problems involve different physics, several unknown fields and so forth. From the algebraic point of view it usually means block structure of the resulting (non-)linear systems. (Consider, for one, Stokes or Oseen problems, fluid–structure–interaction models typical for haemodynamics, thermoelasticity etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turns out that MG is in general a poor choice for such kind of problems. However, these problems usually “contain” some kind of “diffusion” (read: elliptic / Laplace blocks in the matrix), and one usually keeps this in mind while constructing preconditioners. Indeed, one often uses some kind of MG–cycles to approximate actions of inverses of elliptic blocks. One example I mentioned in &lt;a href=&quot;https://scicomp.stackexchange.com/a/26720/21916&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to get a better understanding, it is useful to check &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.676.38.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this lecture&lt;/a&gt; by &lt;a href=&quot;https://scicomp.stackexchange.com/users/393/wolfgang-bangerth&quot;&gt;@wolfgang-bangerth&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="21916" LastEditorUserId="21916" LastEditDate="2017-06-19T19:28:18.723" LastActivityDate="2017-06-19T19:28:18.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27199" PostTypeId="2" ParentId="24342" CreationDate="2017-06-19T20:07:23.613" Score="-1" Body="&lt;p&gt;Just thought I would post an answer to this question I had since I found solutions. First, the number of coefficients for a polynomial basis complete up to degree &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; dimensions, designated &lt;span class=&quot;math-container&quot;&gt;$N_p^n$&lt;/span&gt;, is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;N_p^n &amp;amp;= \begin{pmatrix}&#xA;n + p \\ p&#xA;\end{pmatrix}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Additionally, it's pretty straight forward to compute a basis efficiently using graph algorithms where you compute all sequences of powers that have a sum less than or equal to &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. Mathematically this is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Find all possible sequences for &lt;span class=&quot;math-container&quot;&gt;$(c_1, c_2, \cdots, c_n)$&lt;/span&gt; such that&#xA;&lt;span class=&quot;math-container&quot;&gt;$\sum_{k=1}^n c_k \leq p$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$0 \leq c_k \leq p$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This problem can be viewed as finding valid paths in a dense graph and it is pretty simple to solve in an efficient fashion.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2021-08-11T20:41:16.110" LastActivityDate="2021-08-11T20:41:16.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27200" PostTypeId="1" CreationDate="2017-06-19T21:41:23.577" Score="5" ViewCount="127" Body="&lt;p&gt;I have a question concerning computational geometry which arises in the simulation of fields with topological defects, and I'd like to know whether there's an efficient algorithm (or any algorithm) to solve it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is basically the following:  consider a grid cell $\mathcal{C}$ in a 3-D cubical grid.  On each of the eight vertices of this grid cell, we are given a triplet of numbers $(a_i, b_i, c_i)$ such that $a_i^2 + b_i^2 + c_i^2 = 1$ for each $i = 1 \dots 8$.  By triangulating the surface of the cube (which we denote $\partial \mathcal{C}$) and linearly interpolating between the vertices of each triangle in $\partial \mathcal{C}$, we can define a map $f : \partial \mathcal{C} \to \mathbb{R}^3$.  Let us assume that the image of this map does not contain 0, i.e., the map so defined is actually from $f: \partial \mathcal{C} \to \mathbb{R}^3 \setminus\{0\}.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have thus defined a map from a space homeomorphic to $S^2$ ($\partial \mathcal{C}$) to a space that has a non-trivial second homotopy group $\pi_2$.  This map may or may not be contractible, and in fact it should have some notion of a winding number associated with it.  My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there an algorithm that, given the vertex values $(a_i, b_i, c_i)$ for a triangulated cube, calculates a winding number for the map $f$ so defined?&lt;/li&gt;&#xA;&lt;li&gt;Is there an algorithm that, given the vertex values $(a_i, b_i, c_i)$ for a triangulated cube, calculates whether the map $f$ so defined is nulhomotopic?  (This is obviously a weaker question than the first one, but if this is all that can be done I wouldn't be too disappointed.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I would not be surprised if this question has been addressed in the literature somewhere, but it's quite hard to google for it:  every reference to &quot;winding number&quot; seems to be for maps of 1-D curves into some space rather than 2-D spheres.&lt;/p&gt;&#xA;" OwnerUserId="24606" LastActivityDate="2021-10-21T00:08:35.830" Title="Algorithms for computing winding numbers of 2-sphere maps" Tags="|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27201" PostTypeId="1" CreationDate="2017-06-19T22:25:42.027" Score="1" ViewCount="464" Body="&lt;p&gt;I have a function that I need to numerically integrate from $0$ to $+\infty$, given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$I = \int_0^{+\infty} \mathrm{d}x\,x\,T^2(x)f(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $T^2$ is an interpolated function that goes to $1$ at small $x$ and decays as $x^{-4}$ at large $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ is an oscillating function with known zeroes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x) = j^2_1(\alpha x)\times\cos(\beta\log x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $j_n$ is the spherical Bessel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I know the zeros $z_n$ of the integrand (thus having control on its oscillations), is it a good idea to split the integral in sub-integrals (from $z_n$ to $z_{n+1}$) and sum all the contributions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea was to change variables ($x = e^u$) and do the sub-integrals with SciPy's &lt;code&gt;scipy.integrate.quad&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24608" LastEditorUserId="9667" LastEditDate="2017-06-19T22:42:05.303" LastActivityDate="2017-06-19T22:42:05.303" Title="Numerical integral of oscillating function with known zeros" Tags="|interpolation|quadrature|scipy|integration|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27202" PostTypeId="1" CreationDate="2017-06-19T23:28:33.490" Score="1" ViewCount="119" Body="&lt;p&gt;I've implemented the Finite Element Method to model the heat transfer between two different materials where one material is surrounded by the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run the model I'm getting some strange results. Whenever I run the model, I get what appears to be a sinc function through space and a linear change through time. My gut has been telling me that a part of my implementation isn't right but I haven't been able to figure out what is going on. The positional axis is the position of each individual element, not the radial distance.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OCPjj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OCPjj.png&quot; alt=&quot;Surf plot of the generated model solution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model that I'm currently using is a one dimensional model over the radius of a cylinder with heat being generated within the submerged material and the solution is trying to solve how heat moves through the whole system. Due to symmetry in the system and ignoring how heat moves through the length of the cylinder, I'm modelling the elements as a series of concentric circles with a known thickness. Thinking maybe it was a problem with how I implemented the model, I implemented a two dimensional model looking at a small square to get a proof of concept and I found a similar result in the spatial dimensions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 1 dimensional problem I'm solving is &#xA;$$\rho c_p \frac{\partial T}{\partial t} = k \frac{\partial}{\partial r}\left(r \frac{\partial T}{\partial r}\right)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The domain can be seen in the picture below. The inner radius has the set of parameters defined as $k_1, \rho_1, c_{p1}$ and the outer radius has the set of parameters defined as $k_2, \rho_2, c_{p2}$. The inner material is much smaller than the outer material, such that the image is not to scale. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/32NYO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/32NYO.png&quot; alt=&quot;domain of the model, not to scale&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only boundary condition that I care about is the boundary between the two materials as I'm assuming the two materials are at the same temperature and the outer radius is insulated&#xA;$$k_1 \frac{\partial T}{\partial r}|_{r=R_1} = k_2 \frac{\partial T}{\partial r}|_{r=R_1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source term I'm using is a constant $Q$ and only being applied to the elements in the smaller radius once I get the model working properly I'll refine the source term some more. The large spike in the middle of the first image is the elements caused by the heat source - currently using two elements for now once I'll add more once things work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no formal training with the Finite Element Model and the only understanding I have is from reading numerous books and papers I've scrounged through trying to understand the model, so the problem I'm experiencing could be a simple oversight on my part or a pitfall that I've known nothing about. If anyone could point me in the right direction on how to solve this problem I would be grateful. &lt;/p&gt;&#xA;" OwnerUserId="24607" LastEditorUserId="24607" LastEditDate="2017-06-20T13:26:52.140" LastActivityDate="2017-06-20T13:26:52.140" Title="Strange solutions using Finite Element Analysis" Tags="|finite-element|computational-physics|numerical-modelling|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27203" PostTypeId="1" CreationDate="2017-06-20T07:20:10.623" Score="2" ViewCount="203" Body="&lt;p&gt;I am using a program that creates viscous/absorbing boundaries by implementing Kelvin-Voigt elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The theory behind 1-D Kelvin Voigt elements is given in &lt;a href=&quot;https://en.wikipedia.org/wiki/Kelvin%E2%80%93Voigt_material&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Wikipedia page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case I am running a dynamic analysis on a 2D continuum but applying viscous boundaries on the right and bottom boundaries as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PMBZSm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PMBZSm.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the parameters for the Kelvin Voigt elements are left up to the user in this program. Specifically, the user can input both the spring parameters ($E$, modulus of elasticity) and the dashpot parameters ($\eta$, viscosity).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how to analytically determine if the boundary is in fact absorbing the stress adequately? Let us assume that the input load at the top left corner is periodic (a sine function for example). I am able to obtain values for the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;displacement in both directions&lt;/li&gt;&#xA;&lt;li&gt;velocity in both directions&lt;/li&gt;&#xA;&lt;li&gt;acceleration in both directions&lt;/li&gt;&#xA;&lt;li&gt;any of the Cartesian stresses&lt;/li&gt;&#xA;&lt;li&gt;any of the Cartesian strains&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;for any points throughout the continuum and at the boundary. I can also manipulate the following parameters using &lt;code&gt;numpy&lt;/code&gt; or &lt;code&gt;matlab&lt;/code&gt;. Restating the question, how can I use the above parameters to determine whether my boundary is reflecting/absorbing stress? If it is doing both how do I determine what portion is reflected and what portion is absorbed?&lt;/p&gt;&#xA;" OwnerUserId="22736" LastEditorUserId="9667" LastEditDate="2017-06-20T16:58:32.753" LastActivityDate="2017-06-20T19:31:00.877" Title="How to determine if Kelvin-Voigt elements are dissipating stress correctly?" Tags="|finite-element|boundary-conditions|solid-mechanics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27204" PostTypeId="1" CreationDate="2017-06-20T10:01:30.143" Score="2" ViewCount="282" Body="&lt;p&gt;In a personal project of mine, I've derived a couple of equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum_{j=1}^Nu_{ij}^*(\theta_1,...,\theta_N,J) - k_{i} = 0 \qquad \forall 1\leq i \leq N$$&#xA;$$\sum_{i&amp;lt;j}^N\big(u_{ij}^*(\theta_1,...,\theta_N,J)\big)^2 - O^* = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_{ij}^*(\theta_1,...,\theta_N,J)$ is a solution to the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_{ij} =\frac{1}{2} + \frac{1}{2}\tanh{\left(  - \frac{\theta_i + \theta_j}{2} + 2\cdot J\cdot M \cdot u_{ij}\right)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $M,N,k_i^*, O^*$ are known parameters, and $\theta_1,...,\theta_N,J$ are $N+1$ unknown parameters. I thought this system should be solvable since the top two equations give us $N+1$ equations as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the difficulty in this problem is due to the presence of some unknown/hidden function $u_{ij}^*$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first try was to construct a quadratic cost function, and to minimize this quadratic cost function using gradient descent (involving the partial derivatives of $u_{ij}^*$ (which can be obtained implicitly), however this method did not work out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any idea on how to solve this or know of some tools which I can use to solve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or maybe you could point me to a different place?&lt;/p&gt;&#xA;" OwnerUserId="24613" LastEditorUserId="9667" LastEditDate="2017-06-20T13:48:57.180" LastActivityDate="2017-06-22T15:51:11.593" Title="Solving this system of equations numerically" Tags="|nonlinear-equations|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27205" PostTypeId="1" CreationDate="2017-06-20T11:52:47.327" Score="8" ViewCount="292" Body="&lt;p&gt;What are some &lt;strong&gt;applications&lt;/strong&gt; where you would absolutely go for either FEM, but not FVM, or vice versa? What are some applications where both methods are equally suited?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I worked with the FEM so far and have written various codes, but I sometimes wonder whether I should also use FVM in future.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-06-20T11:52:47.327" Title="What can be done with Finite Element Method and not with the Finite Volume Method, and vice versa?" Tags="|finite-element|finite-volume|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27206" PostTypeId="1" AcceptedAnswerId="27207" CreationDate="2017-06-20T15:23:47.143" Score="4" ViewCount="517" Body="&lt;p&gt;I am trying to implement the algorithm described in reference 1 using &lt;code&gt;cvxpy&lt;/code&gt;. However I am struggling to constrain the matrix $Z_j$ as described in equations (33-35). Citing the paper ($\bar{\mu_j}$ is just a vector):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Remaining constraint on $R_j$ is:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$Z_j= \left[ \begin{matrix} 1 &amp;amp;&amp;amp; (\bar{\mu_j})^{'} \\ \bar{\mu_j} &amp;amp;&amp;amp;&#xA;U \end{matrix} \right]$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where $U$ must be $\bar{\mu_j}(\bar{\mu_j})^{'}$. since the constraint&#xA;  is not linear, we relax it such that:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$U-\bar{\mu_j}(\bar{\mu_j})^{'}\succeq 0$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;by the property that:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$Z_j\succeq0 \Leftrightarrow U-\bar{\mu_j}(\bar{\mu_j})^{'}\succeq0$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where $Z_j \succeq 0$ denotes that $Z_j$ is positive semi-definite&#xA;  matrix.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried to force this structure by doing the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u = cvx.Variable(D)&#xA;U = u*u.T&#xA;Zj = cvx.bmat([[1, u.T], [u, U]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However &lt;code&gt;cvxpy&lt;/code&gt; complains that &lt;code&gt;u*u.T&lt;/code&gt; is incorrect: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;UserWarning: Forming a nonconvex expression (affine)*(affine).&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And solver fails:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cvxpy.error.DCPError: Problem does not follow DCP rules&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears that outer product of two vectors cannot be implemented as such. Any ideas on how to enforce this constraint?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Yun, Sungrack, and Chang D. Yoo. &lt;a href=&quot;https://groups.google.com/group/cvxpy/attach/190960cf8d135/05957267.pdf?part=0.1&amp;amp;authuser=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Loss-scaled large-margin Gaussian mixture models for speech emotion classification.&quot;&lt;/a&gt; IEEE Transactions on Audio, Speech, and Language Processing 20.2 (2012): 585-59&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="24615" LastEditorUserId="20417" LastEditDate="2018-01-26T03:26:52.507" LastActivityDate="2018-01-26T03:26:52.507" Title="Imposing special structure on Positive Semi-Definite matrix" Tags="|python|convex-optimization|semidefinite-programming|cvxpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27207" PostTypeId="2" ParentId="27206" CreationDate="2017-06-20T16:42:34.890" Score="7" Body="&lt;p&gt;The whole point is that you do NOT include the constraint $U = uu^T$ .  That constraint is non-convex.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead you include the constraint that $Z_j$ is (positive) semi-definite. This constraint is a convex relaxation of $U = uu^T$, obtained by Schur complement.  I.e., the relaxed constraint, which is convex, is used instead of the original non-convex constraint.  &lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="24615" LastEditDate="2017-06-21T11:55:57.907" LastActivityDate="2017-06-21T11:55:57.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27208" PostTypeId="1" CreationDate="2017-06-20T18:17:14.243" Score="1" ViewCount="88" Body="&lt;p&gt;I am interested in finding the eigenvalues/eigenfunctions of problems such as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \partial_{xx} u = \lambda \partial_{yy} u, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be solved as the generalised eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{A} \vec{u} = \lambda \mathbf{B} \vec{u}, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the finite difference method. My question is about how to incorporate Neumann conditions into the $\mathbf{A}$ and $\mathbf{B}$ matrices. Examples I have found involved changing the right hand side of the equation in equations such as $ \mathbf{A} \vec{u} = \vec{b} $, i.e. not eigenvalue problems, but this is not possible in this case. I am not necessarily specifically interested in solving the PDE above, I am just looking for a general method.&lt;/p&gt;&#xA;" OwnerUserId="24583" LastEditorDisplayName="user20857" LastEditDate="2017-06-20T18:49:26.113" LastActivityDate="2017-06-20T18:49:26.113" Title="Numerically solving generalized eigenproblem with Neumann conditions" Tags="|pde|finite-difference|boundary-conditions|eigenvalues|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27209" PostTypeId="2" ParentId="27203" CreationDate="2017-06-20T19:31:00.877" Score="3" Body="&lt;p&gt;The approach you are describing distributes discrete damping (not stiffness) elements along the&#xA;boundary of an elastic region to approximately absorb the stress waves that hit the boundary&#xA;instead of having them reflect back into the region. The classic reference for the approach&#xA;is &lt;a href=&quot;http://cedb.asce.org/CEDBsearch/record.jsp?dockey=0016353&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lysmer and Kuhlemeyer&lt;/a&gt;.&#xA;This approach is used in the ABAQUS FEA code and is described in their theoretical manual;&#xA;you might be able to find a copy of this document online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a brief summary. The speed of a longitudinal wave in an elastic material is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;c_L = \sqrt{\frac{\lambda + 2G}{\rho}}&#xA;$$&#xA;where $\lambda$ and  $G$ are the Lame constants, and $\rho$ is the density.&#xA;The speed of a shear wave is given by&#xA;$$&#xA;c_S=\sqrt{\frac{G}{\rho}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 2D model, distributed damping normal and tangential to the boundary will produce the following stresses&#xA;$$&#xA;\sigma_{xx} = -d_L \dot u_x&#xA;$$&#xA;and&#xA;$$&#xA;\tau_{xy} = -d_S \dot u_y&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $d_L=\rho c_L$ and $d_S=\rho c_S$ the boundaries will be approximately absorbing. The&#xA;details of how these values are determined can be found in the referenced paper.&#xA;The damping coefficients for the discrete dampers in the FE model can be set to match&#xA;these distributed values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach doesn't create a perfectly absorbing boundary but it does work well enough&#xA;for many cases. The effectiveness is somewhat problem-specific. Here is how I suggest&#xA;evaluating the effectiveness for your model.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Create two FE models, one with dampers on the boundary and a  larger one without dampers.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apply the load you show with a sinusoidal time dependency to both models.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Solve the time transient equations to a final time where the stress waves haven't&#xA;reached the boundaries of the larger model but have passed the boundaries of the smaller one.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compare the displacements and stresses between the two models at selected points. If the&#xA;waves are not being reflected at the boundaries with dampers, these quantities should&#xA;compare well.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-06-20T19:31:00.877" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27211" PostTypeId="2" ParentId="27182" CreationDate="2017-06-21T11:51:22.070" Score="2" Body="&lt;p&gt;We have the following linear system in &lt;span class=&quot;math-container&quot;&gt;$\mathrm x \in \mathbb R^n$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rm A x = b$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathrm A \in \mathbb R^{m \times n}$&lt;/span&gt; is &lt;strong&gt;fat&lt;/strong&gt; (i.e., &lt;span class=&quot;math-container&quot;&gt;$n &amp;gt; m$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$\mathrm b \in \mathbb R^m$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Least-norm&lt;/h3&gt;&#xA;&lt;p&gt;If the linear system is &lt;em&gt;consistent&lt;/em&gt;, we look for the &lt;strong&gt;least-norm&lt;/strong&gt; solution via the following (convex) quadratic program&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \| \mathrm x \|_2^2\\ \text{subject to} &amp;amp; \mathrm A \mathrm x = \mathrm b\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let the Lagrangian be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal L (\mathrm x, \lambda) := \frac 12 \mathrm x^{\top} \mathrm x + \lambda^{\top} (\mathrm A \mathrm x - \mathrm b)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Taking the partial derivatives of &lt;span class=&quot;math-container&quot;&gt;$\mathcal L$&lt;/span&gt; and finding where they vanish, we obtain the linear system&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix} \mathrm I_n &amp;amp; \mathrm A^\top\\ \mathrm A &amp;amp; \mathrm O_m\end{bmatrix} \begin{bmatrix} \mathrm x\\ \lambda \end{bmatrix} = \begin{bmatrix} 0_n\\ \mathrm b\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\mathrm A$&lt;/span&gt; has &lt;em&gt;full row rank&lt;/em&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\rm A A^\top$&lt;/span&gt; is invertible and we can conclude that the least-norm solution is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathrm x_{\text{LN}} := \color{blue}{\mathrm A^{\top} \left( \mathrm A \mathrm A^{\top} \right)^{-1} \mathrm b}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Least-squares&lt;/h3&gt;&#xA;&lt;p&gt;If the linear system is &lt;em&gt;inconsistent&lt;/em&gt;, we can look for the &lt;strong&gt;least-squares&lt;/strong&gt; solution via the following unconstrained (convex) quadratic program&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \| \mathrm A \mathrm x - \mathrm b \|_2^2\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Taking the gradient of the objective function and finding where it vanishes, we obtain the &lt;em&gt;normal equations&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathrm A^{\top} \mathrm A \,\mathrm x = \mathrm A^{\top} \mathrm b$&lt;/span&gt;. However, since &lt;span class=&quot;math-container&quot;&gt;$\rm A$&lt;/span&gt; is fat, its rank is at most &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and, thus,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mbox{rank} (\mathrm A^{\top} \mathrm A) \leq m &amp;lt; n$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hence, &lt;span class=&quot;math-container&quot;&gt;$\mathrm A^{\top} \mathrm A$&lt;/span&gt; is never invertible and, thus, the normal equations have infinitely many solutions. Thus, let us add a &lt;strong&gt;regularization&lt;/strong&gt; term to the objective function, i.e.,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \| \mathrm A \mathrm x - \mathrm b \|_2^2 + \gamma \| \mathrm x \|_2^2\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\gamma \geq 0$&lt;/span&gt;. The new normal equations are&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left( \mathrm A^{\top} \mathrm A + \gamma \mathrm I_n \right) \mathrm x = \mathrm A^{\top} \mathrm b$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\color{blue}{\gamma &amp;gt; 0}$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\mathrm A^{\top} \mathrm A + \gamma \mathrm I_n$&lt;/span&gt; is positive definite and, thus, invertible. Hence, the &lt;em&gt;regularized&lt;/em&gt; least-squares solution is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathrm x_{\text{LS}} := \color{blue}{\left( \mathrm A^{\top} \mathrm A + \gamma \mathrm I_n \right)^{-1} \mathrm A^{\top} \mathrm b}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using a &lt;a href=&quot;https://www.stats.ox.ac.uk/%7Elienart/blog_linalg_invlemmas.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;matrix inversion lemma&lt;/a&gt;, the regularized least-squares solution can be rewritten as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathrm x_{\text{LS}} := \color{blue}{\mathrm A^{\top} \left( \mathrm A \mathrm A^{\top} + \gamma \mathrm I_m \right)^{-1} \mathrm b}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which resembles the &lt;em&gt;least-norm&lt;/em&gt; solution. However, we now invert &lt;span class=&quot;math-container&quot;&gt;$\mathrm A \mathrm A^{\top} + \gamma \mathrm I_m$&lt;/span&gt;, which is positive definite whenever &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;gt; 0$&lt;/span&gt;, rather than &lt;span class=&quot;math-container&quot;&gt;$\mathrm A \mathrm A^{\top}$&lt;/span&gt; (which may be &lt;em&gt;ill-conditioned&lt;/em&gt; or even non-invertible).&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-06-23T07:41:25.673" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27212" PostTypeId="2" ParentId="159" CreationDate="2017-06-21T14:23:30.417" Score="-1" Body="&lt;p&gt;Libraries algorithms provide in contrast to own implementations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;They are generic and templated. You can later reparameterize your implementation without worrying changing your own code which is supposed to have a lot of constraints.&lt;/li&gt;&#xA;&lt;li&gt;There is savety versus degenerate cases of input data. A lot of computational geometry algorithms, e.g. convex hull ones, need to handle for example colinearity of three points. You might be able to ignore those cases if you never plan to distribute your code and also do not want to reuse your code in future often. &lt;/li&gt;&#xA;&lt;li&gt;They provide the minimal runtime complexity for either expected or worst case input configurations. Higher level algorithms have as building bricks often lower level algorithms,e.g. sorting algorithms, or special data types. Quick sort might be the most common choice for sorting data, but if your implementation of the algorithm has to garantee n(log(n)) you cannot use it. &lt;/li&gt;&#xA;&lt;li&gt;They are memory usage efficient &lt;/li&gt;&#xA;&lt;li&gt;They are further runtime optmized&lt;/li&gt;&#xA;&lt;li&gt;If supported, by far more closed to be &quot;bug&quot; free in general, especially if you work with the main branch. Nothing is more tested than a well distributed library. Not every bug crashes, not every bug produces unreasonable results. Your algorithm's implementation might still produce acceptable results, just not as good as it is designed for. The less visible a bug is, the less likely it is that you as a single person can even detect it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I still consider it good when entering a new field to implement one version of a well understandable algorithm on your own. I takes a lot of time in total. I bought and read books, the named Press et al. I always read much theory before and during those implementations. And after understanding the general concepts of a field and experiencing the traps in practice for me it is time to jump to the in all aspects better library implementations. I think you will become a better user of the library if you wrote an &quot;hello world&quot; algorithm in the libraries field on your own. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you work in a bigger team it might not be your own choice wether your team uses a specific library or not. The core team might do the decision. And there might be a person responsible for the library binding in your project with its own time plannings. Rewriting one algorithm you can do with your own time planning, not relying on other peoples decision. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are on your own and like to distribute there is another problem. I consider as well as many other source code as most useful resource. Many to all informaticians might agree here. In an applied field outside informatics it might be needed to provide a precompiled exectuable on windows. Under Linux you can set up things relatively easy on your own in case of open source usage libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rewriting an algorithm on your own gives lisence freedom. Your project might not support &lt;a href=&quot;https://www.gnu.org/licenses/gpl-3.0.en.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPL&lt;/a&gt; lisence of &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSL&lt;/a&gt; for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lisence might be the only constraint which is independent from the reseachers point of view. &lt;/p&gt;&#xA;" OwnerUserId="20162" LastEditorUserId="20162" LastEditDate="2017-07-02T06:52:12.030" LastActivityDate="2017-07-02T06:52:12.030" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27213" PostTypeId="2" ParentId="27204" CreationDate="2017-06-21T17:40:00.337" Score="2" Body="&lt;p&gt;First, let's rewrite your system in the form of a vector function for which you want to find a root.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{F}(\mathbf{x}) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;F_i(\mathbf{x}) &amp;amp;=  \sum_{j=1}^N u_{ij} - 1  \qquad \forall 1\leq i \leq N\\&#xA;F_i(\mathbf{x}) &amp;amp;= \sum_{i&amp;lt;j}^N u_{ij}^2 k_i^2 - R \qquad i  = N + 1\\&#xA;F_i(\mathbf{x}) &amp;amp;= u_{ij} -\frac{1}{2} - \frac{1}{2}\tanh{\left(  - \frac{\theta_i + \theta_j}{2} + 2 J  u_{ij}\right)} \qquad \forall i &amp;gt;N +1 \land i\leq N^2 + N + 1\, ,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{x} = (u_{1,1},u_{1,2},\cdots,u_{i,j},\cdots, u_{N-1,N},u_{N,N},θ_1,θ_2,\cdots,θ_N,J)\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameters $k_i$ are absorbed as part of the &lt;em&gt;new&lt;/em&gt; variables $u_{ij}$, and the same happened to $M$. That way, you have $N + 1$ less parameters to worry about (I also renamed $O^*$ as $R$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you can use a root finding algorithm. If you can compute the Jacobian, you can use something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton&amp;#39;s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Newton's method&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The option that you propose would be something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{\mathbf{x}} \Vert \mathbf{F}(\mathbf{x})\Vert^2\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what you can plug into a minimization algorithm&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below, I implemented both approaches in Python using &lt;code&gt;scipy.optimize&lt;/code&gt;. Notice that basin-hopping is a global minimization approach and it is not a fair comparison. Also, note that basin-hopping will run several initial points and it will take more time that just using a single initial point. If you have a good guess, then it might improve.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division, print_function&#xA;import numpy as np&#xA;from numpy.linalg import norm&#xA;from scipy.optimize import root, basinhopping&#xA;&#xA;&#xA;def fun(x, R, N, k):&#xA;    u = x[:N**2].copy()&#xA;    u.shape = (N, N)&#xA;    f = np.zeros_like(x)&#xA;    f[:N] = -np.ones((N))&#xA;    f[N] = -R&#xA;    for row in range(N):&#xA;        for col in range(N):&#xA;            f[row] += u[row, col]&#xA;            f[N + 1 + N*col + row] = u[row, col] - 0.5 -\&#xA;                0.5*np.tanh(-0.5*(x[N**2 + row] + x[N**2 + col]) +&#xA;                2*x[-1]*u[row, col])&#xA;            if row &amp;lt; col:&#xA;                f[N] += u[row, col]**2*k[row]**2&#xA;    return f&#xA;&#xA;&#xA;def opt_fun(x, R, N, k):&#xA;    return norm(fun(x, R, N, k))**2&#xA;&#xA;&#xA;N = 2&#xA;R = 1&#xA;size = N**2 + N + 1&#xA;x0 = np.random.normal(loc=1,size=size)&#xA;k = np.ones((N))&#xA;res_root = root(fun, x0, args=(R, N, k))&#xA;res_min = basinhopping(opt_fun, x0,&#xA;                       minimizer_kwargs={'args': (R, N, k)}, seed=1)&#xA;print(&quot;Root finding algorithm norm: {:.6g}&quot;.format(norm(res_root.fun)))&#xA;print(&quot;Minimization algorithm norm: {:.6g}&quot;.format(norm(res_min.fun)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For $N=2$ the root finding works better. For higher orders it seems that the optimization procedure works better. For example for $N=10$, this is the output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Root finding algorithm solution norm: 0.772523&#xA;Minimization algorithm solution norm: 1.67062e-13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-06-22T15:51:11.593" LastActivityDate="2017-06-22T15:51:11.593" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27217" PostTypeId="2" ParentId="27194" CreationDate="2017-06-22T04:43:15.730" Score="5" Body="&lt;p&gt;SSP methods are mainly used for integrating ODEs corresponding to nonlinear hyperbolic PDE semi-discretizations. In such ODEs, $\Delta t_{FE}$ depends on the solution and so it varies at each time step.  In all implementations I know of, SSPRK time stepping is done adaptively in order to ensure that the SSP time step constraint (stated in the question above) is satisfied.  This isn't difficult since usually $\Delta t_{FE}$ is just inversely proportional to the maximum characteristic speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the words in your question suggest that you are also interested in adapting the timestep to satisfy a local error tolerance.  This is not usually done, since for hyperbolic problems with shocks the spatial error typically dominates the temporal error.  Nevertheless, having some bound on the temporal error could certainly be useful, and there might be situations in which the error control restricts the time step more tightly than the SSP condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A recent preprint that contains several embedded SSP pairs for this purpose is available &lt;a href=&quot;https://arxiv.org/abs/1806.08693&quot; rel=&quot;nofollow noreferrer&quot;&gt;from arXiv&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll also mention that for SSP linear multistep methods, adapting the step size while maintaining the SSP condition is quite challenging; see &lt;a href=&quot;http://dx.doi.org/10.1137/15M101717X&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2018-07-04T22:00:10.197" LastActivityDate="2018-07-04T22:00:10.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27218" PostTypeId="1" CreationDate="2017-06-22T08:25:15.377" Score="1" ViewCount="212" Body="&lt;p&gt;I have two equations (coupled), with the variables $T_1$ and $T_2$ and the constant $T_0$, which are (when written unitless, i.e. without prefactors):&#xA;$$\partial_t T_1 = 1-T_1^3+T_1+\nabla\left(\frac{1}{T_2^{2.5}+T_2^{2.4}}\right)\nabla T_1+\frac{1}{T_2^{2.5}+T_2^{2.4}}\nabla^2T_1$$&#xA;and&#xA;$$ \partial_tT_2=\left(\frac{T_1}{1+T_1^2}\left(T_0-T_2\right)+\nabla\left(\frac{1}{1+T_2+T_2^2}\right)\nabla T_2+\frac{1}{1+T_2+T_2^2}\nabla^2T_2\right)$$&#xA;Now my usual approach is to divide such equations into a linear and a nonlinear term, as for the first equation:&#xA;$$\partial_t T_1 = \underbrace{1-T_1^3+T_1}_{\text{Nonlinear}}+\underbrace{\nabla\left(\frac{1}{T_2^{2.5}+T_2^{2.4}}\right)\nabla T_1+\frac{1}{T_2^{2.5}+T_2^{2.4}}\nabla^2T_1}_{\text{Linear}}$$&#xA;and then splitting the linear term up into $T_{x,0}$ and $T_{x,1}$ for the current and next step, as for a reduced equation&#xA;$$\begin{split}&#xA;\partial_tT_1&amp;amp;=\nabla T_1\\&#xA;&amp;amp;\Rightarrow\\&#xA;\frac{T_{1,1}-T_{1,0}}{dt}&amp;amp;=\frac{\nabla T_{1,1}+\nabla T_{1,0}}{2}\\&#xA;\left(\frac{1}{dt}-\nabla\right)T_{1,1}&amp;amp;=\left(\frac{1}{dt}+\nabla\right)T_{1,0}&#xA;\end{split}$$&#xA;which then can be solved using a matrix approach and -solver. In order to extend it to $T_2$, I can create a vector of both variables, such that &#xA;$$\vec{A}\begin{pmatrix}&#xA;T_{1,1}\\&#xA;T_{2,1}&#xA;\end{pmatrix}=\vec{B}\begin{pmatrix}&#xA;T_{1,0}\\&#xA;T_{2,0}&#xA;\end{pmatrix}+\begin{pmatrix}&#xA;\text{Nonlinear}_{T_1}\\&#xA;\text{Nonlinear}_{T_2}&#xA;\end{pmatrix}&#xA;$$&#xA;But now I am stuck how to do that for &#xA;$$\nabla\left(\frac{1}{T_2^{2.5}+T_2^{2.4}}\right)\nabla T_1$$&#xA;Of course I can split it up to&#xA;$$\begin{split}&#xA;\nabla\left(\frac{1}{T_2^{2.5}+T_2^{2.4}}\right)&amp;amp;=\frac{\nabla\left(\frac{1}{T_{2,1}^{2.5}+T_{2,1}^{2.4}}\right)+\nabla\left(\frac{1}{T_{2,0}^{2.5}+T_{2,0}^{2.4}}\right)}{2}&#xA;\end{split}$$&#xA;but then I do not know how to integrate it into the matrix equations, after I do not know how to create a matrix $\vec{C}$, such that&#xA;$$&#xA;\vec{C}\cdot T_2=\nabla\left(\frac{1}{T_2^{2.5}+T_2^{2.4}}\right)&#xA;$$&#xA;Is there such a matrix? Or is my whole approach skewed, and should be rethought?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-09-17T16:47:29.347" Title="Implicit method for two coupled PDEs" Tags="|pde|matrix|nonlinear-equations|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27219" PostTypeId="2" ParentId="159" CreationDate="2017-06-22T18:40:05.340" Score="0" Body="&lt;p&gt;Is little to add, we have to reuse code, it is about code sustainability and contribution to society, but that is all above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why we do not reuse code is that if you are beginning programmer is hard to understand others code. It is particularly difficult with advanced C++, and you can do some tricks in pure C as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Very often at the beginning, one understand the method, but not as it is implemented in the library, or simply how to use the library with its generic interface, error control and conventions, very often documented for experienced programmers if at all. This gives the illusion that is better to implement a standard method, like LU factorization by yourself.  Moreover, new programmers underestimate the value of code testing, validation and portability for different operating system. So at the end reason is laziness, writing own code looks like as a faster and easier solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reality is that we can learn more by using and reading code than programming by yourself from scratch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Laziness drives me for most of the time, I think also the majority of people. For the same reason, some write code from scratch and other use existing libraries. &lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-06-22T18:58:33.373" LastActivityDate="2017-06-22T18:58:33.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27220" PostTypeId="1" AcceptedAnswerId="27221" CreationDate="2017-06-22T19:06:48.477" Score="1" ViewCount="74" Body="&lt;p&gt;I am looking for examples of numerically finding eigenvalues/eigenfunctions of coupled DEs. If anyone is able to point me towards any examples, preferably with code included, it would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="24583" LastActivityDate="2017-06-23T16:37:47.750" Title="Examples of finding eigenfunctions of coupled DEs" Tags="|pde|ode|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27221" PostTypeId="2" ParentId="27220" CreationDate="2017-06-22T22:14:14.210" Score="1" Body="&lt;p&gt;A while back I wrote a few such examples over at &lt;a href=&quot;https://mathematica.stackexchange.com&quot;&gt;https://mathematica.stackexchange.com&lt;/a&gt;. You can find them &lt;a href=&quot;https://mathematica.stackexchange.com/a/57391/18437&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://mathematica.stackexchange.com/a/85675/18437&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://mathematica.stackexchange.com/a/56320/18437&quot;&gt;here&lt;/a&gt;. The last link is probably the best to understand - even though that is not for coupled PDEs; but there is not much difference. There also examples at the reference page of Mathematicas' function &lt;a href=&quot;https://reference.wolfram.com/language/ref/NDEigensystem.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NDEigensystem&lt;/a&gt; which does what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me reproduce a small example here, that is more suited to an audience that may not be as familiar with the language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To model the modes of a beam we define a plane stress operator (coupled PDE):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;planeStress = {&#xA; Div[{{0, -((Y*ν)/(1 -ν^2))}, {-(Y*(1 -ν))/(2*(1 - ν^2)), 0}}.Grad[v[t, x, y], {x, y}], {x, y}] + Div[{{-(Y/(1 - ν^2)),  0}, {0, -(Y*(1 - ν))/(2*(1 - ν^2))}}.Grad[u[t, x, y], {x, y}], {x, y}], &#xA; Div[{{0, -(Y*(1 - ν))/(2*(1 - ν^2))}, {-((Y*ν)/(1 - ν^2)), 0}}.Grad[u[t, x, y], {x, y}], {x, y}] + Div[{{-(Y*(1 -ν))/(2*(1 -ν^2)), 0}, {0, -(Y/(1 - ν^2))}}.Grad[v[t, x, y], {x, y}], {x, y}]&#xA;} /. {Y -&amp;gt; 210 10^9, ν -&amp;gt; 33/100};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We hold the left hand side of the beam fixed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(*held fixed at left*)&#xA;bcs = DirichletCondition[{u[t, x, y] == 0, v[t, x, y] == 0}, x == 0];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And call the numerical differential eigensolver:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;\[Rho] = 7850; length = 10; height = 0.7;&#xA;{vals, funs} = &#xA;  NDEigensystem[{Thread[\[Rho]*D[{u[t, x, y], v[t, x, y]}, {t, 1}] == &#xA;      planeStress], &#xA;    DirichletCondition[{u[t, x, y] == 0., v[t, x, y] == 0.}, &#xA;     x == 0]}, {u[t, x, y], v[t, x, y]}, &#xA;   t, {x, y} \[Element] Rectangle[{0, 0}, {length, height}], 9];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This returns a list of eigenvalues and eigenfunctions. To plot them:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Show[{Graphics3D[&#xA;     {Gray, &#xA;      GraphicsComplex[{{0, 0, 0}, {length, 0, 0}, {length, height, &#xA;         0}, {0, height, 0}}, Line[{{1, 2, 3, 4, 1}}]]}],&#xA;    Plot3D[&#xA;     Sqrt[10000*Total[#^2]], {x, y} \[Element] &#xA;      Rectangle[{0, 0}, {length, height}], &#xA;     ColorFunction -&amp;gt; &quot;TemperatureMap&quot;, Axes -&amp;gt; False, &#xA;     Mesh -&amp;gt; False]}, Boxed -&amp;gt; False] &amp;amp; /@ funs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/momXJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/momXJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more detailed information on how that works, please have a look at the posts I mentioned in the beginning.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="11893" LastEditDate="2017-06-23T16:37:47.750" LastActivityDate="2017-06-23T16:37:47.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27223" PostTypeId="1" AcceptedAnswerId="27231" CreationDate="2017-06-23T11:11:03.520" Score="2" ViewCount="1079" Body="&lt;p&gt;I have 3 functions which consist of 6 variables $p_1,p_2,p_3,p_4,p_5,p_6$. The value of each function is equal to $x$ (say):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;f_1 &amp;amp;= \operatorname{sign}(2-p_1) \sqrt{|2-p_1|} + \operatorname{sign}(2-p_2)\sqrt{|2-p_2|} + \operatorname{sign}(2-p_3)\sqrt{|2-p_3|}\cr&#xA;f_2 &amp;amp;= \operatorname{sign}(p_4-2)\sqrt{|p_4-2|} + \operatorname{sign}(p_5-2)\sqrt{|p_5-2|} + \operatorname{sign}(p_6-2)\sqrt{|p_6-2|}\cr&#xA;f_3 &amp;amp;= \operatorname{sign}(p_1-p_4)\sqrt{|p_1-p_4|} + \operatorname{sign}(p_2-p_5)\sqrt{|p_2-p_5|} + \operatorname{sign}(p_3-p_6)\sqrt{|p_3-p_6|}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find the combination of values of $p_1,p_2,p_3,p_4,p_5$ and $p_6$ for which $x$ is maximum. Constraints are:&#xA;$$0 &amp;lt;= p_1,p_2,p_3,p_4,p_5,p_6 &amp;lt;= 4$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply varying every variable from $0$ to $4$ taking small steps is not a good solution. Can someone tell me an efficient method to optimise the solution (preferably in python).&lt;/p&gt;&#xA;" OwnerUserId="24643" LastEditorUserId="9667" LastEditDate="2017-06-24T17:11:25.020" LastActivityDate="2017-06-24T17:11:25.020" Title="Optimization of multiple functions" Tags="|constrained-optimization|nonlinear-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27225" PostTypeId="1" CreationDate="2017-06-23T13:35:54.133" Score="1" ViewCount="63" Body="&lt;p&gt;This question is a follow-up of &lt;a href=&quot;https://stackoverflow.com/questions/44718160/solve-a-linear-system-for-fft-coefficients&quot;&gt;https://stackoverflow.com/questions/44718160/solve-a-linear-system-for-fft-coefficients&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;At some time (kt), the FT of the vorticity (omega) satisfies:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(1-nu*L*dt/2)fftomega(kt+1)=(1+nu*L*dt/2)fftomega(kt)-fft_J*dt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where fftomega is the 2d fft of omega, J(psi,omega) is the Jacobian (see below) and the streamfunction del^2(psi)=omega. L is an array of coefficients that when multiplied by say fftomega return the fft of the laplacian of omega. For a 6x6 L is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;0   -1   -4   -9   -4   -1&lt;/p&gt;&#xA;&lt;p&gt;-1   -2   -5  -10   -5   -2&lt;/p&gt;&#xA;&lt;p&gt;-4   -5   -8  -13   -8   -5&lt;/p&gt;&#xA;&lt;p&gt;-9  -10  -13  -18  -13  -10&lt;/p&gt;&#xA;&lt;p&gt;-4   -5   -8  -13   -8   -5&lt;/p&gt;&#xA;&lt;p&gt;-1   -2   -5  -10   -5   -2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The fft of the Jacobian is computed as follows (using matlab/octave notation for brevity):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Dx_psi=real(ifft2(Dx.*fftpsi));                    %v&#xA;Dy_psi=real(ifft2(Dy.*fftpsi));                    %-u &#xA;Dx_omega=real(ifft2(Dx.*fftomega));    &#xA;Dy_omega=real(ifft2(Dy.*fftomega));&#xA;Jacobian = - Dy_psi.*Dx_omega + Dx_psi.*Dy_omega;&#xA;&#xA;fft_J = fft2(Jacobian);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In practice, the Jacobian is integrated with a predictor corrector scheme due to Gadzag (the -fft_j*dt in the equation above is schematic). If arrays such as fftomega have the same dimension as omega, then the equation above can be solved as a system of linear equations, with LAPACK for instance.  However since the physical variables are real, procedures such as FFTW allow for output fft's to be non-square (r2c,c2r).  In this case it seems like the only way to integrate the equation is to expand these fft's back out to square, giving up the advantage afforded by the r2c,c2r scheme. Alternatively, one could make the physical variables complex with null imaginary part, and use the c2c scheme.&lt;/p&gt;&#xA;&lt;p&gt;Has anyone found an alternative that makes it possible to use r2c,c2c?&lt;/p&gt;&#xA;" OwnerUserId="24129" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-06-23T16:16:01.100" Title="Integrating the 2d vorticity equation on periodic boundaries" Tags="|time-integration|fftw|linear-system|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27226" PostTypeId="1" CreationDate="2017-06-23T14:17:44.820" Score="2" ViewCount="128" Body="&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/lapack/explore-html/da/dba/group__double_o_t_h_e_rcomputational_gae5d8ecd7fbd852fe3c3f71e08ec8332c.html#gae5d8ecd7fbd852fe3c3f71e08ec8332c&quot; rel=&quot;nofollow noreferrer&quot;&gt;dpstf&lt;/a&gt; and &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/da/dba/group__double_o_t_h_e_rcomputational_ga31cdc13a7f4ad687f4aefebff870e1cc.html#ga31cdc13a7f4ad687f4aefebff870e1cc&quot; rel=&quot;nofollow noreferrer&quot;&gt;dpstrf&lt;/a&gt; sometimes give different pivot results. Of course I can read the source code, but I don't get the idea from it. Since pivoting is for stability of the Cholesky decomposition, one of them is more stable. Which and why does the other exist?&lt;/p&gt;&#xA;" OwnerUserId="7894" LastEditorUserId="7894" LastEditDate="2017-06-23T15:20:33.887" LastActivityDate="2017-10-19T01:38:33.463" Title="What is the difference in the pivoting strategies between LAPACK's dpstrf and dpst2 and why?" Tags="|linear-algebra|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27227" PostTypeId="1" CreationDate="2017-06-23T16:14:28.980" Score="1" ViewCount="165" Body="&lt;p&gt;I have a very similar problem to the one described in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/23272/calculate-inverse-of-dense-matrix-with-entries-of-very-different-magnitude/23275&quot;&gt;Calculate inverse of dense matrix with entries of very different magnitude&lt;/a&gt;. The reason why I am opening a new question is because as far as I understood the answer given to the problem in that question was too specific for the problem considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My issue is that I have a matrix &lt;code&gt;C&lt;/code&gt; which is fairly large (say typically &lt;code&gt;500x500&lt;/code&gt; entries) and the entries are spanning some orders of magnitude, typically from $10^3$ to $10^{-16}$ and I need to INVERT this matrix. I really need to do this, as in fact in a following stage I need to use the inverse matrix in some multiplications with other matrices. This is also the reason why I am more interested in the PRECISION of the inversion rather than the performance in terms of speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore my obvious question would be: what is the best algorithm to use to get the inverse matrix in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried Singular Value Decomposition, but I noticed that it is producing unstable results.&lt;/p&gt;&#xA;" OwnerUserId="24645" LastActivityDate="2017-06-23T16:14:28.980" Title="Best algorithm for inversion of matrix spanning many orders of magnitude" Tags="|linear-algebra|matrix|svd|" AnswerCount="0" CommentCount="4" ClosedDate="2018-11-08T13:51:57.257" ContentLicense="CC BY-SA 3.0" />
  <row Id="27228" PostTypeId="1" AcceptedAnswerId="27281" CreationDate="2017-06-23T21:10:14.260" Score="3" ViewCount="204" Body="&lt;p&gt;The two-level additive Schwarz method (additive Schwarz with a coarse space correction) is often written like this:&#xA;$$&#xA;\mathbf{v} = \sum_{i=0}^N \mathbf{R}_i^T \mathbf{A}^{-1}_i\mathbf{R}_i\mathbf{w}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the term $i=0$ is said to be the coarse space correction. $\mathbf{A}_0$ is the solver for a global coarse grid problem, $\mathbf{R}_0$ and $\mathbf{R}_0^T$ the restriction and interpolation operators, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how to construct $\mathbf{A}_0$? Which problem does it correspond to? I get the idea of creating a coarse mesh, in fact, I am already using a coarse mesh to make an initial guess for what the solution of the global problem should be. But the two-level additive Schwarz method solves a coarse problem in each iteration and I don't understand what problem this is supposed to be. I assume we are not solving the same coarse grid problem over and over, something must change in between the iterations.&lt;/p&gt;&#xA;" OwnerUserId="23932" LastActivityDate="2017-06-29T22:40:30.260" Title="What is the global problem in the two-level additive Schwarz?" Tags="|finite-element|domain-decomposition|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27229" PostTypeId="1" AcceptedAnswerId="27285" CreationDate="2017-06-24T11:43:47.210" Score="2" ViewCount="869" Body="&lt;p&gt;I need to solve the Schrödinger equation with a time dependent Hamiltonian&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$i\hbar \frac{\partial}{\partial t} \Psi =  \left[-\frac{\hbar^2}{2m}\nabla^2 +\frac{1}{2} k(t)(x^2+y^2) + V(r)\right]\Psi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody recommend me an efficient numerical method or software package for solving the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;This is a suitable form of the equation for numerically solving(which we discussed in the one of the below answers with &lt;a href=&quot;https://scicomp.stackexchange.com/users/14942/davidhigh&quot;&gt;@davidhigh&lt;/a&gt;). &#xA;\begin{align*}&#xA;i\frac{\partial}{\partial t}u_{\ell}(r,t) = \Bigg(-\frac{1}{2} \frac{\partial ^2}{\partial r^2} + \frac{\ell(\ell+1)}{2r^2} +  V(r)\Bigg)u_{\ell}(r,t)\quad \quad \quad \quad \quad \quad \quad \nonumber\\  + \frac{2}{3}k(t)r^2 \sum_{\ell' = {max}(\ell-2,0)}^{{min}(\ell+2,L_{{max}})} \Bigg(\delta_{\ell,\ell'} - \sqrt{ \frac{4\pi}{5}} \alpha(\ell, \ell') \Bigg) u_{\ell'}(r,t)&#xA;\end{align*}&#xA;where the coefficient $\alpha(\ell, \ell')$ is given by&#xA;\begin{align*}&#xA;	\alpha(\ell, \ell') = \int Y^*_{\ell0}(\Omega)Y_{20}(\Omega)Y_{\ell'0}(\Omega) d\Omega&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which the integral can be solved by Wigner-3j coefficients (see  &lt;a href=&quot;http://mathworld.wolfram.com/SphericalHarmonic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest an efficient numerically method for solving the equation?&lt;/p&gt;&#xA;" OwnerUserId="24650" LastEditorUserId="24650" LastEditDate="2018-03-21T20:53:22.833" LastActivityDate="2018-03-21T20:53:22.833" Title="Schrödinger equation with time dependent Hamiltonian" Tags="|finite-element|time-integration|fenics|quantum-mechanics|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27230" PostTypeId="1" CreationDate="2017-06-24T12:16:58.353" Score="2" ViewCount="68" Body="&lt;p&gt;I have an eigenvalue problem emerging from the internal vibro-acoustic coupling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigenvalue problem is nonsymmetric but it was proven in literature that it results in  real eigenvalues and eigenvectors due to the relations between the off-diagonal coupling matrices. The eigenvalue problem reads as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A \phi = \lambda B \phi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A = \begin{bmatrix} K_1 &amp;amp;K_2\\ 0 &amp;amp;K_3\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ B = \begin{bmatrix} M_1 &amp;amp;0\\ -K_2^T &amp;amp;M_2\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the Arnoldi technique to solve this eigenvalue problem. However, I always have to apply a scaling in order to find the right eigenvalues and eigenvectors. What I am applying is the simple jacobi scaling such as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P1 = diag(1./sqrt(diag(K1)))&#xA;P2 = diag(1./sqrt(diag(K2)))&#xA;&#xA;P = diag(P1,P2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;eventually, what I am solving is the generalized eigenvalue problem with this scaling matrix, namely,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ P A P = \begin{bmatrix}&#xA;P_1 K_1 P_1 &amp;amp;P_1 K_2 P_2\\&#xA;0 &amp;amp;P_2 K_3 P_2\end{bmatrix}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ P B P = \begin{bmatrix}&#xA;P_1 M_1 P_1 &amp;amp;0\\&#xA;-P_2 K_2^T P_1 &amp;amp;P_2 M_2 P_2\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then I go back to the original space before scaling. It is fine up to this point but there is one more follow up step I would like to accomplish. Later on, I need to solve a linear system which involves $A$, such as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A x = b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I would like to use the factor I generated during the above eigenvalue solution in order to perform this solution. But if I perform the scaling, my original $A$ transforms to $PAP$ so I can not use that for a forward-backward solve and I need a new factorization for $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And as mentioned above, I can only find the right eigenvalues and eigenvectors with this scaling although my original matrices are not that ill-conditioned. I am looking for a way to solve my eigenvalue problem without scaling and use the available factorization at a later stage, but so far, I have no luck without scaling. I am using the built in solvers (or the Factorize package from Tim Davies) in my MATLAB implementations for the Arnoldi iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help/idea is greatly appreciated on what I can try to solve this problem without scaling.&lt;/p&gt;&#xA;" OwnerUserId="7184" LastEditorUserId="9667" LastEditDate="2017-06-24T17:03:15.403" LastActivityDate="2017-06-24T17:03:15.403" Title="Scaling for a nonsymmetric eigenvalue problem" Tags="|linear-solver|eigensystem|preconditioning|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27231" PostTypeId="2" ParentId="27223" CreationDate="2017-06-24T13:28:55.943" Score="1" Body="&lt;p&gt;Do you want f1 = f2 = f3 at the maximum?  If so, maximize x with respect to x,p1,p2,p3,p4,p5,p6, subject to f1 = x, f2 = x, f3 = x, 0 &amp;lt;= p1,p2,p3,p4,p5,p6 &amp;lt;= 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the solution using the BARON global solver with YALMIP under MATLAB. The first argument of optimize is the constraints. The second argument of optimize is the objective function, which is -x, because optimize always minimizes, so minimizing -x is the same as maximizing x.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; sdpvar x p1 p2 p3 p4 p5 p6&#xA;&amp;gt;&amp;gt; f1 = [sign(2-p1)*sqrtm(abs(2-p1))+sign(2-p2)*sqrtm(abs(2-p2))+sign(2-p3)*sqrtm(abs(2-p3))==x];&#xA;&amp;gt;&amp;gt; f2 = [sign(p4-2)*sqrtm(abs(p4-2))+sign(p5-2)*sqrtm(abs(p5-2))+sign(p6-2)*sqrtm(abs(p6-2))==x];&#xA;&amp;gt;&amp;gt; f3 = [sign(p1-p4)*sqrtm(abs(p1-p4))+sign(p2-p5)*sqrtm(abs(p2-p5))+sign(p3-p6)*sqrtm(abs(p3-p6))==x];&#xA;&amp;gt;&amp;gt; optimize([0&amp;lt;=[p1 p2 p3 p4 p5 p6]&amp;lt;=4,f1,f2,f3],-x,sdpsettings('solver','baron'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;% Optimal values&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; disp(value([x p1 p2 p3 p4 p5 p6]))&#xA;    0.6065    1.3333    4.0000    0.5501         0    2.6667    3.4499&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2017-06-24T13:28:55.943" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27236" PostTypeId="1" CreationDate="2017-06-25T20:57:31.173" Score="0" ViewCount="468" Body="&lt;p&gt;Define a sequence $(\mathbf{y})_{i=0}^N$ in $\mathbb{R}^n$ such that:&#xA;$$\mathbf{y}_{k+1} = \mathbf{y}_{k} + \lambda \nabla_\mathbf{y} E(\mathbf{y}_k,\mathbf{w}), \quad k=0,1,\ldots,N-1,$$&#xA;where $\lambda$ is a constant, $\mathbf{w}\in\mathbb{R}^m$, and $E:\mathbb{R}^{n+m}\to \mathbb{R}$ is some differentiable function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $Q:\mathbb{R}^{n}\to \mathbb{R}$ be a differentiable function and $L=Q(\mathbf{y} _N)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying the chain rule we have:&#xA;$$\frac{dL}{d\mathbf{w}} = \sum_{k=1}^N\frac{\partial \mathbf{y}_k}{\partial \mathbf{w}} \frac{dQ}{d\mathbf{y}_k}\qquad (1)$$&#xA;and&#xA;$$\frac{dQ}{d\mathbf{y}_k} = \frac{\partial \mathbf{y}_{k+1}}{\partial \mathbf{y}_{k}} \frac{dQ}{d\mathbf{y}_{k+1}}.\qquad (2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Source: &lt;a href=&quot;http://proceedings.mlr.press/v22/domke12/domke12.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, equation (12) and the one between (12) and (13).)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions:&lt;/strong&gt; how to obtain $(1)$ and $(2)$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can show that if $(2)$ holds then $(1)$ holds. But I cannot see why $(2)$ holds. Consider $k=N-1$ for example: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying &lt;a href=&quot;https://math.stackexchange.com/questions/1566376/derivation-of-the-multivariate-chain-rule/1567200#1567200&quot;&gt;the chain rule&lt;/a&gt; we have:&#xA;\begin{align}\frac{dQ}{d\mathbf{y}_{N-1}} &#xA;&amp;amp;= \frac{dQ(\mathbf{y}_{N}(\mathbf{w},\mathbf{y}_{N-1}))}{d\mathbf{y}_{N-1}} \\&#xA;&amp;amp;= \frac{d\mathbf{y}_{N}(\mathbf{w},\mathbf{y}_{N-1})}{d\mathbf{y}_{N-1}}\frac{dQ}{d\mathbf{y}_{N}} \\&#xA;&amp;amp;= \begin{bmatrix}\frac{d\mathbf{w}}{d\mathbf{y}_{N-1}} &amp;amp; \mathbf{I}\end{bmatrix}\begin{bmatrix}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}} \\ \frac{\partial \mathbf{y}_{N}}{\partial \mathbf{y}_{N-1}}\end{bmatrix}\frac{dQ}{d\mathbf{y}_{N}} \\&#xA;&amp;amp;= \frac{d\mathbf{w}}{d\mathbf{y}_{N-1}}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}}\frac{dQ}{d\mathbf{y}_{N}} + \frac{\partial \mathbf{y}_{N}}{\partial \mathbf{y}_{N-1}}\frac{dQ}{d\mathbf{y}_{N}},&#xA;\end{align}&#xA;which has an extra term $\frac{d\mathbf{w}}{d\mathbf{y}_{N-1}}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}}\frac{dQ}{d\mathbf{y}_{N}}$ compared to $(2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your help.&lt;/p&gt;&#xA;" OwnerUserId="7695" LastEditorUserId="713" LastEditDate="2017-06-25T23:17:17.013" LastActivityDate="2018-09-21T02:01:48.440" Title="Derivative chain rule" Tags="|optimization|calculus|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27239" PostTypeId="1" CreationDate="2017-06-26T11:59:45.520" Score="1" ViewCount="50" Body="&lt;p&gt;Assumed I have the following equations with the parameters $\vec{A}(z, t)$, $\vec{B}(z, t)$ and the constants $a,\,b,\,c$:&#xA;$$&#xA;\begin{split}&#xA;\partial_tA&amp;amp;=a\partial_zA\\&#xA;\partial_tB&amp;amp;=b\partial_zB+A\cdot B\cdot c&#xA;\end{split}&#xA;$$&#xA;Now the usual approach is to split the equations for an implicit scheme like &#xA;$$&#xA;\left(\frac{1}{dt}-\frac{\partial_z}{2}\right)A_1=\left(\frac{1}{dt}+a\frac{\partial_z}{2}\right)A_0&#xA;$$&#xA;which then can be solved. &#xA;The split is done using&#xA;$$\begin{split}&#xA;\frac{A_1-A_0}{dt}&amp;amp;=\partial_tA\\&#xA;\partial_zA&amp;amp;=\frac{\partial_zA_1+\partial_zA_0}{2}\\&#xA;\frac{A_1-A_0}{dt}&amp;amp;=a\frac{\partial_zA_1+\partial_zA_0}{2}\\&#xA;\left(\frac{A_1}{dt}-a\frac{\partial_zA_1}{2}\right)&amp;amp;=\left(\frac{A_0}{dt}+\frac{a\partial_zA_0}{2}\right)\\&#xA;\left(\frac{1}{dt}-a\frac{\partial_z}{2}\right)A_1&amp;amp;=\left(\frac{1}{dt}+\frac{a\partial_z}{2}\right)A_0&#xA;\end{split}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the equation system above I can then write (without the second term in the second equation):&#xA;$$\begin{split}&#xA;C&amp;amp;=\begin{pmatrix}&#xA;A\\B\end{pmatrix}\\&#xA;e&amp;amp;=\begin{pmatrix}&#xA;a\\b\end{pmatrix}\\&#xA;d&amp;amp;=\begin{pmatrix}&#xA;0\\0\end{pmatrix}\\&#xA;\left(\frac{1}{dt}-e\partial_z-\frac{d}{2}\right)C_1&amp;amp;=\left(\frac{1}{dt}+e\partial_z+\frac{d}{2}\right)C_0\end{split}$$&#xA;But when including the second term, I can choose between&#xA;$$d=\begin{pmatrix}B_0\cdot c\\0&#xA;\end{pmatrix}$$&#xA;or &#xA;$$d=\begin{pmatrix}0\\A_0\cdot c&#xA;\end{pmatrix}$$&#xA;depending on how I calculate&#xA;$$A\cdot B\cdot c$$&#xA;Is there a method to determine which $d$ I should use?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastEditorUserId="20545" LastEditDate="2017-06-26T14:02:14.110" LastActivityDate="2017-06-26T14:02:14.110" Title="How to determine which parameter I should linearize?" Tags="|pde|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27241" PostTypeId="2" ParentId="27236" CreationDate="2017-06-26T19:27:25.687" Score="1" Body="&lt;p&gt;The idea is that each $\vec{y}_k$ depends on $\vec{w}$. Therefore for the first equation, simply apply the chain rule for each $\vec{y}_k$ and sum them up. Componentwise one has:&#xA;$$\left(\frac{dL}{d\vec{w}}\right)_{i}=\sum_{k,\alpha}{\left(\frac{dQ}{d\vec{y}_k}\right)_{\alpha}\left(\frac{\partial \vec{y}_k}{\partial \vec{w}}\right)_{i\alpha}} \tag{*}$$&#xA;Maybe the order in which they appear multiplying each other has confused you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second is obtained supposing that the function $\vec{y}_{k}$ can be put as a function of $\vec{y}_{k+1}$ (your first equation suggets this). Therefore the derivative in $(*)$ can be expressed as:&#xA;$$\frac{dQ}{d\vec{y}_k}=\frac{\partial \vec{y}_{k+1}}{\partial \vec{y}_k}\frac{dQ}{d\vec{y}_{k+1}}$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-06-27T18:20:47.703" LastActivityDate="2017-06-27T18:20:47.703" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27242" PostTypeId="1" CreationDate="2017-06-26T20:52:59.190" Score="3" ViewCount="179" Body="&lt;p&gt;In oceanography, is there any particular reason why choosing large eddy simulations in stead of RANS (regardless of the type of flow)? In both cases, 2d simulations would be used (shallow water model).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros and contras, in general, of both approaches?&lt;/p&gt;&#xA;" OwnerUserId="24560" LastActivityDate="2017-11-24T14:18:00.923" Title="Reasons to choose LES in stead of RANS models? (turbulence)" Tags="|fluid-dynamics|computational-physics|numerical-modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27243" PostTypeId="1" CreationDate="2017-06-26T21:04:13.923" Score="-1" ViewCount="82" Body="&lt;p&gt;I found this one, but does not work: &lt;a href=&quot;http://www.karlin.mff.cuni.cz/~hron/warsaw_2014/pl2014_lecture5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.karlin.mff.cuni.cz/~hron/warsaw_2014/pl2014_lecture5.pdf&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="24679" LastActivityDate="2017-06-26T21:41:25.230" Title="Anyone knows where I can find a simple FEniCS code where I can understand basic implantation?" Tags="|finite-element|fenics|" AnswerCount="1" CommentCount="0" ClosedDate="2017-06-27T15:33:33.363" ContentLicense="CC BY-SA 3.0" />
  <row Id="27244" PostTypeId="2" ParentId="27243" CreationDate="2017-06-26T21:41:25.230" Score="1" Body="&lt;p&gt;You could try the &lt;a href=&quot;https://launchpadlibrarian.net/83776282/fenics-book-2011-10-27-final.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEniCS book&lt;/a&gt; or the &lt;a href=&quot;https://fenicsproject.org/pub/tutorial/pdf/fenics-tutorial-vol1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEniCS Tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2017-06-26T21:41:25.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27245" PostTypeId="2" ParentId="21846" CreationDate="2017-06-27T01:45:36.947" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the fastest way to do it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This very much depends on the form of the matrix.  You can exploit symmetries to reduce your problem: Is the matrix symmetric or Hermitian? Is it banded or tridiagonal?  Maybe it is even block-diagonal, then you can diagonalize the blocks independent of one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if you only need a few eigenvalues, then you probably want to use an iterative solver instead of doing a full diagonalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably want to take a look at this online resource:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is a huge table listing a lot of linear algebra codes, their level of parallelism, supported matrix and datatypes and much more.&lt;/p&gt;&#xA;" OwnerUserId="24680" LastActivityDate="2017-06-27T01:45:36.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27246" PostTypeId="2" ParentId="25283" CreationDate="2017-06-27T02:29:26.787" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If we consider vectorization, are we going to beat typical libraries like OpenBLAS?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I do not fully agree with the other answer and would like to say &lt;strong&gt;kind of&lt;/strong&gt;.  It is true that libraries like Intel (R) MKL have handwritten kernels which use a lot of optimizations.  On the other hand, you cannot compile Intel (R) MKL on your own machine, so it has to detect whether AVX instructions can be used at runtime (if this is possible at all).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is why I believe that you actually can beat these BLAS implementations using C++ template libraries which make heavy use of SIMD and OpenMP for multithreading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good example for that is the &lt;a href=&quot;https://bitbucket.org/blaze-lib/blaze&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blaze library&lt;/a&gt;.  The Bitbucket repository also has a benchmark page where they also compare &lt;a href=&quot;https://bitbucket.org/blaze-lib/blaze/wiki/Benchmarks#!blas-level-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAS level 3&lt;/a&gt; routines.  For very large matrices Blaze and Intel (R) MKL are almost the same in speed (probably memory limited) but for smaller matrices Blaze beats MKL.  It is even more obvious for the &lt;a href=&quot;https://bitbucket.org/blaze-lib/blaze/wiki/Benchmarks#!blas-level-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAS level 2&lt;/a&gt; routines.&lt;/p&gt;&#xA;" OwnerUserId="24680" LastActivityDate="2017-06-27T02:29:26.787" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27247" PostTypeId="2" ParentId="20277" CreationDate="2017-06-27T02:43:19.663" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://bitbucket.org/blaze-lib/blaze&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blaze library&lt;/a&gt; for linear algebra makes heavy use of C++14 in the form of deduced and trailing return types.  Other modern C++ features in use are &lt;code&gt;constexpr&lt;/code&gt;, alias templates, and a whole lot of template metaprogramming with expression SFINAE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use initializer lists for your vectors and matrices, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;blaze::DynamicVector&amp;lt;int&amp;gt; x{ 4, -1, 3 };&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For more details see their &lt;a href=&quot;https://bitbucket.org/blaze-lib/blaze/wiki/Getting_Started&quot; rel=&quot;nofollow noreferrer&quot;&gt;getting started page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24680" LastActivityDate="2017-06-27T02:43:19.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27248" PostTypeId="2" ParentId="20786" CreationDate="2017-06-27T02:55:26.297" Score="3" Body="&lt;p&gt;You can easily write a thin C++ wrapper around the GSL quadrature functions.  The following needs C++11.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;&#xA;#include &amp;lt;functional&amp;gt;&#xA;#include &amp;lt;memory&amp;gt;&#xA;#include &amp;lt;utility&amp;gt;&#xA;#include &amp;lt;gsl/gsl_errno.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_integration.h&amp;gt;&#xA;&#xA;template &amp;lt; typename F &amp;gt;&#xA;class gsl_quad&#xA;{&#xA;  F f;&#xA;  int limit;&#xA;  std::unique_ptr &amp;lt; gsl_integration_workspace,&#xA;                    std::function &amp;lt; void(gsl_integration_workspace*) &amp;gt;&#xA;                    &amp;gt; workspace;&#xA;&#xA;  static double gsl_wrapper(double x, void * p)&#xA;  {&#xA;    gsl_quad * t = reinterpret_cast&amp;lt;gsl_quad*&amp;gt;(p);&#xA;    return t-&amp;gt;f(x);&#xA;  }&#xA;&#xA;public:&#xA;  gsl_quad(F f, int limit)&#xA;    : f(f)&#xA;    , limit(limit)&#xA;    , workspace(gsl_integration_workspace_alloc(limit), gsl_integration_workspace_free)&#xA;  {}&#xA;&#xA;  double integrate(double min, double max, double epsabs, double epsrel)&#xA;  {&#xA;    gsl_function gsl_f;&#xA;    gsl_f.function = &amp;amp;gsl_wrapper;&#xA;    gsl_f.params = this;&#xA;&#xA;    double result, error;&#xA;    if ( !std::isinf(min) &amp;amp;&amp;amp; !std::isinf(max) )&#xA;    {&#xA;      gsl_integration_qags ( &amp;amp;gsl_f, min, max,&#xA;                             epsabs, epsrel, limit,&#xA;                             workspace.get(), &amp;amp;result, &amp;amp;error );&#xA;    }&#xA;    else if ( std::isinf(min) &amp;amp;&amp;amp; !std::isinf(max) )&#xA;    {&#xA;      gsl_integration_qagil( &amp;amp;gsl_f, max,&#xA;                             epsabs, epsrel, limit,&#xA;                             workspace.get(), &amp;amp;result, &amp;amp;error );&#xA;    }&#xA;    else if ( !std::isinf(min) &amp;amp;&amp;amp; std::isinf(max) )&#xA;    {&#xA;      gsl_integration_qagiu( &amp;amp;gsl_f, min,&#xA;                             epsabs, epsrel, limit,&#xA;                             workspace.get(), &amp;amp;result, &amp;amp;error );&#xA;    }&#xA;    else&#xA;    {&#xA;      gsl_integration_qagi ( &amp;amp;gsl_f,&#xA;                             epsabs, epsrel, limit,&#xA;                             workspace.get(), &amp;amp;result, &amp;amp;error );&#xA;    }&#xA;&#xA;    return result;&#xA;  }&#xA;};&#xA;&#xA;template &amp;lt; typename F &amp;gt;&#xA;double quad(F func,&#xA;            std::pair&amp;lt;double,double&amp;gt; const&amp;amp; range,&#xA;            double epsabs = 1.49e-8, double epsrel = 1.49e-8,&#xA;            int limit = 50)&#xA;{&#xA;  return gsl_quad&amp;lt;F&amp;gt;(func, limit).integrate(range.first, range.second, epsabs, epsrel);&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;  std::cout &amp;lt;&amp;lt; &quot;\\int_0^1 x^2 dx = &quot;&#xA;            &amp;lt;&amp;lt; quad([](double x) { return x*x; }, {0,1}) &amp;lt;&amp;lt; '\n'&#xA;            &amp;lt;&amp;lt; &quot;\\int_1^\\infty x^{-2} dx = &quot;&#xA;            &amp;lt;&amp;lt; quad([](double x) { return 1/(x*x); }, {1,INFINITY}) &amp;lt;&amp;lt; '\n'&#xA;            &amp;lt;&amp;lt; &quot;\\int_{-\\infty}^\\infty \\exp(-x^2) dx = &quot;&#xA;            &amp;lt;&amp;lt; quad([](double x) { return std::exp(-x*x); }, {-INFINITY,INFINITY}) &amp;lt;&amp;lt; '\n';&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;\int_0^1 x^2 dx = 0.333333&#xA;\int_1^\infty x^{-2} dx = 1&#xA;\int_{-\infty}^\infty \exp(-x^2) dx = 1.77245&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24680" LastActivityDate="2017-06-27T02:55:26.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27249" PostTypeId="2" ParentId="18760" CreationDate="2017-06-27T04:52:12.257" Score="1" Body="&lt;p&gt;I think it makes sense to present your data in logarithmic scale for both color and vector length. Here's a code snippet based on your example.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from matplotlib.colors import LogNorm&#xA;from pylab import *&#xA;&#xA;X=linspace(-2,2,40)&#xA;Y=linspace(-2,2,24)&#xA;X,Y=meshgrid(X, Y)&#xA;&#xA;&#xA;def E(x,y):&#xA;    r = sqrt(x**6 + y**6)&#xA;    return (x/r,y/r)&#xA;&#xA;def E_dir(x,y):&#xA;#direction field&#xA;Ex,Ey=E(x,y)&#xA;n=sqrt(Ex**2+Ey**2)&#xA;return [Ex/n, Ey/n]&#xA;&#xA;Ex,Ey = E(X,Y)&#xA;Exdir,Eydir = E_dir(X,Y)&#xA;EE=sqrt(Ex**2+Ey**2)&#xA;EElog=log10(EE-EE.min()+1.0)&#xA;Exlog=Exdir*EElog&#xA;Eylog=Eydir*EElog&#xA;&#xA;colormap='jet'&#xA;# your way&#xA;ff=figure(1)&#xA;ff.clf()&#xA;subplot(221)&#xA;Q  = quiver(X,Y,Exdir,Eydir,EE,cmap=colormap)&#xA;colorbar()&#xA;title('linear in color, constant vector length')&#xA;&#xA;# log colormap&#xA;subplot(222)&#xA;Q  = quiver(X,Y,Exdir,Eydir,EElog,cmap=colormap)&#xA;colorbar()&#xA;title('log in color (manually), constant vector length')&#xA;&#xA;# a fancier way of doing a log colormap&#xA;subplot(223)&#xA;Q  = quiver(X,Y,Exdir,Eydir,EE,norm=LogNorm(vmin=EE.min(),vmax=EE.max()),cmap=colormap)&#xA;colorbar()&#xA;title('log in color (using LogNorm), constant vector length')&#xA;&#xA;# log colormap and vector lengths&#xA;subplot(224)&#xA;Q  = quiver(X,Y,Exlog,Eylog,EE,norm=LogNorm(vmin=EE.min(),vmax=EE.max()),cmap=colormap,scale=50)&#xA;colorbar()&#xA;title('log in color and vector length')&#xA;&#xA;show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first plot is just a reproduction of yours, except I used a different color scale to accentuate the differences.&lt;/li&gt;&#xA;&lt;li&gt;The second and third plots both use logarithm mappings of the colors, one done manually, one done via &lt;code&gt;matplotlib.colors.LogNorm()&lt;/code&gt;. Note that they're not quite the same. I'm not sure the exact equation that &lt;code&gt;LogNorm()&lt;/code&gt; uses. Also, the colorbar knows about the log scaling for the automatic way, but not for the manual way.&lt;/li&gt;&#xA;&lt;li&gt;The last plot log scales both colors and vector magnitudes. It seems uncouth to apply a log scaling to the vector magnitudes, but it might be appropriate if it's clearly stated. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GjC9J.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GjC9J.jpg&quot; alt=&quot;Four quiver plots demonstrating logarithmic color and vector scalings.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2017-06-27T04:52:12.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27250" PostTypeId="2" ParentId="18760" CreationDate="2017-06-27T07:01:30.633" Score="8" Body="&lt;p&gt;&lt;sup&gt;[I took your sample program as a starting point and adapted &lt;a href=&quot;https://matplotlib.org/users/colormapnorms.html&quot; rel=&quot;noreferrer&quot;&gt;Colormap Normalization&lt;/a&gt; from the matplotlib wiki.]&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Almost everything of the picture just looks red.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Indeed.  They problem is that there is a very narrow divergence in your data and because the colormap is scaled linearly almost all of the plot will be mapped to the lower limit of the colorbar.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;Q = quiver(X,Y,Exdir,Eydir,EE,cmap='autumn')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OkAjt.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OkAjt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That means we need to rescale the colorbar.  For a diveregence a logscale sounds good.  First of all we need&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import matplotlib.colors as colors&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so we can mess with the colors.  Next we simply put&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;Q = quiver(X,Y,Exdir,Eydir,EE,cmap='autumn',&#xA;           norm=colors.LogNorm(vmin=EE.min(),vmax=EE.max()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will scale the colorbar logarithmically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JbBQp.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JbBQp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That looks a little better already.  Now let's use another colormap because yellow is so hard to see.  I think &lt;code&gt;coolwarm&lt;/code&gt; looks okay and suits the diverging data (I think Paraview uses this by default).&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;Q = quiver(X,Y,Exdir,Eydir,EE,cmap='coolwarm',&#xA;           norm=colors.LogNorm(vmin=EE.min(),vmax=EE.max()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Wx1hY.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wx1hY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a last step let's add a nice colorbar indicating the divergence.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;fig = figure()&#xA;Q = quiver(X,Y,Exdir,Eydir,EE,cmap='coolwarm',&#xA;           norm=colors.LogNorm(vmin=EE.min(),vmax=EE.max()))&#xA;fig.colorbar(Q,extend='max')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OVCLr.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OVCLr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course you can also put a colormap in the background as in the MATLAB answer.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;fig = figure()&#xA;I = imshow(EE,extent=[np.min(X),np.max(X),np.min(Y),np.max(Y)],cmap='autumn')&#xA;Q = quiver(X,Y,Exdir,Eydir)&#xA;fig.colorbar(I,extend='max')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gYnb8.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gYnb8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or log scaled&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;fig = figure()&#xA;I = imshow(EE,extent=[np.min(X),np.max(X),np.min(Y),np.max(Y)],cmap='autumn',&#xA;           norm=colors.LogNorm(vmin=EE.min(),vmax=EE.max()))&#xA;Q = quiver(X,Y,Exdir,Eydir)&#xA;fig.colorbar(I,extend='max')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/a9DJX.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a9DJX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another method is to clip the data to a maximum value.  The &lt;code&gt;imshow&lt;/code&gt; command can do this on its own using the &lt;code&gt;vmax&lt;/code&gt; key, for &lt;code&gt;quiver&lt;/code&gt; you have to do it manually using &lt;code&gt;numpy.clip&lt;/code&gt;.  This sets all values outside of a certain threshold to that threshold.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.clip.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;numpy.clip(a, a_min, a_max, out=None)&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But because your function is changing so rapidly you probably have to use a logarithmic colormap in this case as well.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;fig = figure()&#xA;Q = quiver(X,Y,Exdir,Eydir,clip(EE,None,1),cmap='autumn')&#xA;fig.colorbar(Q,extend='max')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5mnDK.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5mnDK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24680" LastEditorUserId="24680" LastEditDate="2017-06-27T07:18:30.210" LastActivityDate="2017-06-27T07:18:30.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27251" PostTypeId="1" AcceptedAnswerId="27254" CreationDate="2017-06-27T07:41:28.610" Score="2" ViewCount="1113" Body="&lt;p&gt;We have a variance-covariance matrix denoted with $X^TX$, where $X$ is the design matrix. In linear regression we can estimate beta coefficients with normal equations like $\hat{\beta} = (X^TX)^{-1}X^Ty$ and we can also compute the variance of betas with $Var(\hat{\beta}) = \hat{\sigma}^2(X^TX)^{-1}$, where $\hat{\sigma}^2$ is estimated from sample. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented linear regression in my library and I used QR factorization for solving betas and also I used QR factorization for computing $(X^TX)^{-1}$, and then took only square roots of the diagonal elements of it. The last one I computed by solving $(X^TX)A = I$, where $A$ is what I am searching for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a faster way to compute only the diagonal elements of the inverse of $X^TX$? I know about Cholesky, I did not consider it, since it is also $O(n^3)$ and is less stable numerically. Is there a shortcut only for those diagonal elements?&lt;/p&gt;&#xA;" OwnerUserId="4838" LastActivityDate="2017-06-27T15:53:16.800" Title="Fast way to compute the diagonal elements of the inverse of covariance matrix" Tags="|linear-algebra|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27252" PostTypeId="2" ParentId="27242" CreationDate="2017-06-27T11:54:30.013" Score="1" Body="&lt;p&gt;I am not an oceanographer, but meteorologist, but in the atmosphere using LES for 2D simulation is a non-sense. Even more so for a simplified model such as the shallow water equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LES to work properly must resolve a big part of the turbulence spectrum and therefore must be able to explicitly simulate the dynamics turbulent flow. In the limit of very large resolution and disabled subgrid model you need the code to approach DNS. You hardly can expect that from 2D shallow water model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is true that the Smagorinsky model originated in 2D simulations of atmosphere, but the true correct application proved to be elsewhere.&lt;/p&gt;&#xA;" OwnerUserId="1244" LastActivityDate="2017-06-27T11:54:30.013" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27253" PostTypeId="1" CreationDate="2017-06-27T15:04:45.513" Score="1" ViewCount="83" Body="&lt;p&gt;This is a question about hard-field back-projection as used in x-ray tomography, applied magnetic induction tomography. &lt;a href=&quot;http://iopscience.iop.org/article/10.1088/0031-9155/38/1/011&quot; rel=&quot;nofollow noreferrer&quot;&gt;Al-Zeibak and Saunders&lt;/a&gt; have shown that x-ray filtered backprojection can be applied to an exciter and receiver coil, with a straight line of magnetic flux between them along the central axes of the 2 coils, taking the place of a straight line beam of an x-rays, to scan/image and an object (plastic container of saline solution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aAYEA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aAYEA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In place of the summation of the x-ray attenuation coefficients $\mu(x,y)$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p_\theta(r) = ln \lgroup \frac{I}{I_0} \rgroup = - \int \mu(x,y) ds$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tomographic_reconstruction&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Tomographic_reconstruction&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;they appear to detect the summation of voltages at receiver coil:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p_\theta(r) = \int V(x,y) ds$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $V(x,y)$ is the voltage in the object at $(x,y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be possible to convert this (Al-Zeibak and Saunders method) to a square Helmholtz-coil assembly, providing a uniform AC magnetic field (i.e. straight lines of flux) and a square array of air-cored inductors as the receiving/sensor coils. So that a metallic object being scanned would generate a summation of voltages for each approximate straight line of flux of the Helmholtz coils?&lt;/p&gt;&#xA;" OwnerUserId="23181" LastEditorUserId="23181" LastEditDate="2017-07-18T11:33:39.703" LastActivityDate="2017-07-18T11:33:39.703" Title="Can x-ray back-projection be converted to hard-field magnetic induction tomography?" Tags="|image-processing|fourier-transform|inverse-problem|electromagnetism|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27254" PostTypeId="2" ParentId="27251" CreationDate="2017-06-27T15:53:16.800" Score="5" Body="&lt;p&gt;You can write the (i,i)'th diagonal entry of $(X^TX)^{-1}$ as the product $d=e_i^T\cdot(X^TX)^{-1}\cdot e_i$, where $e_i$ is the i'th euclidean vector (all zeroes, except for a single one at the i'th position).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you already possess the QR factorization $X=QR$, then $(X^TX)^{-1} = (R^TR)^{-1}$ and $d=e_i^T\cdot(R^TR)^{-1}\cdot e_i = (R^{-T}e_i)^T(R^{-T}e_i)$. If you introduce the vector $y = R^{-T}e_i$, then $d=y^Ty$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This procedure maps readily onto LAPACK/BLAS:&#xA;(a) Compute $X=QR$ using dgeqrf(), inplace. Afterward, the desired $R$ will be in triu($X$)&#xA;(b) For each diagonal $d$, form $e_i$ then backsolve it by $R^{-T}$ from the left using dtrsv(), yielding $y$. Then $d=y^Ty$. This is dnrm2().&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might be exactly what you are doing, but your OP suggested you were perhaps forming the inverse $(X^TX)^{-1}$ explicitly, which I don't think it necessary. The computational cost of this posts approach is asymptotically the same $O(n^3)$, though maybe it saves you a temporary.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2017-06-27T15:53:16.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27257" PostTypeId="1" CreationDate="2017-06-27T17:13:15.093" Score="2" ViewCount="90" Body="&lt;p&gt;How can I find positive integers $a$ and $p$ satisfying $a^{p-1}=1$ ($\mathrm{mod}\ p^2$)? Let's say $1\lt a \lt 100, p \lt 10^8$.&lt;/p&gt;&#xA;" OwnerUserId="24689" LastEditorUserId="651" LastEditDate="2017-06-30T18:15:21.223" LastActivityDate="2017-08-01T15:29:16.523" Title="How can I find positive integers $a$ and $p$ satisfying $a^{p-1}=1($mod $p^2)$?" Tags="|algorithms|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27258" PostTypeId="2" ParentId="27229" CreationDate="2017-06-27T17:50:02.530" Score="2" Body="&lt;p&gt;FEniCS users have solved this problem before, but keep in mind that FEniCS does not natively support complex numbers right now in its code. Therefore you have to make a workaround.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://fenicsproject.org/qa/9209/how-to-use-complex-numbers-iterative-solvers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fenicsproject.org/qa/9209/how-to-use-complex-numbers-iterative-solvers&lt;/a&gt;&#xA;&lt;a href=&quot;https://fenicsproject.org/qa/10671/mixed-function-spaces-for-complex-valued-problems-in-2016-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fenicsproject.org/qa/10671/mixed-function-spaces-for-complex-valued-problems-in-2016-0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24690" LastActivityDate="2017-06-27T17:50:02.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27259" PostTypeId="1" CreationDate="2017-06-27T18:08:02.803" Score="2" ViewCount="86" Body="&lt;p&gt;I am interested in numerically solving the following constrained minimization problem;  Find the value of $x\in \mathbb{R}^n$ that minimizes $f$ where&#xA; $f\colon \mathbb{R}^n\to \mathbb{R}$ is defined as&#xA;$$&#xA;f(x) = \frac{1}{2}(x,A x)-(x,b),&#xA;$$&#xA;for the matrix $A$ being large sparse, symmetric and positive definite, subject to &#xA;$$&#xA;   B^\mathrm{T} x = g.  &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the matrix $B\in \mathrm{R}^{n\times m}$ is large sparse, and $m$ is large but small compared to $n$.  So far so good, this is just the normal saddle point problem.  The wrinkle here is that $B$ may have linearly dependent columns, and that I know $g\in \mathrm{Range}(B^\mathrm{T})$, at least up to machine precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem comes from a finite element code where I generate the constraint matrices by against too many test functions.  I can explain more if anyone is interested. (I Should be testing against all elements in trace of some space,  S, but I don't have that space available to me easily so I test against all elements in S.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I efficiently solve this problem?  I am mainly interested in some sort of Krylov / Relaxation method.   Below I outline a method using the SVD of $B$, and a $QR$ method would work too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I would do for small matrices: (Spoiler Alert: this is very tedious and somewhat boring, I wrote it out in this much detail for my own reference, and to show that I did some work before asking here.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $(U\in \mathbb{R}^{n\times n},\Sigma\in \mathbb{R}^{n\times m}, V\in \mathbb{R}^{m\times m})$ constitute the singular value decomposition of $B$, so that &#xA;$$&#xA;U\Sigma V^\mathrm{T} = B.&#xA;$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $r$ is the rank of $B$, then the first $r$ columns of $U$  (call then $U_1$) span the range of $B$ and the last $n-r$ columns of $U$ (call them $U_2$) span the nullspace of $B^\mathrm{T}$.  Similarly, the first $r$ columns of $V$ (call then $V_1$) span the range of $B^\mathrm{T}$ and the last $m-r$ columns (call them $V_2$) span the null space of $B$.  We also define $\Sigma_1$ as $r\times r$ matrix that sits at tthe top-left hand corner of $\Sigma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We now decompose $x$ in $x = \hat{x} + x_\perp$ where $\hat{x} \in \mathrm{Null}(B^\mathrm{T})$ and $x_\perp\in (\mathrm{Null}(B^\mathrm{T}))^{\perp}= \mathrm{Range}(B)$.  This means that $x_\perp = U_1\alpha$ for some $\alpha\in \mathrm{R}^{n-r}$ and that $\hat{x} = U_2\beta$ for some $\beta\in \mathbb{R}^{n-t}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subbing this second identity into the constraint we get&#xA;$$&#xA;  V\Sigma^\mathrm{T} U^\mathrm{T} U_1\alpha = g,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can now multiply on the left by $V_1^\mathrm{T}$ which gives&#xA;$V_1^\mathrm{T}V\Sigma^\mathrm{T} U^\mathrm{T} U_1\alpha = V_1^\mathrm{T}g$, and noting that $V_1^\mathrm{T}V\Sigma^\mathrm{T} U^\mathrm{T} U_1 = \Sigma_1^\mathrm{T}=\Sigma_1$ we get $\alpha = \Sigma_1^{-1} V_1^{\mathrm{T}} g$ and $x_\perp$ is known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if we had multiplied on the left by $V_2^\mathrm{T}$ we have obtained $0 = 0$ since $g\in \mathrm{Range}(B^\mathrm{T})$ so all the &quot;information&quot; in the constraints was retained when we multiplied on the left by $V_1^\mathrm{T}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we define $F:\mathrm{R}^{n-r}\to \mathrm{R}$ as &#xA;$$&#xA;F(\beta) = \frac{1}{2}(x_\perp + U_2\beta,A(x_\perp+U_2\beta) ) &#xA;- (b, x_\perp + U_2\beta)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we find the $\beta$ that minimizes this functional (symmetric positive definite) and we can find $\hat{x}$.  &lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="16799" LastEditDate="2017-06-27T18:15:33.527" LastActivityDate="2017-06-27T18:15:33.527" Title="How can I numerically solve a saddle point problem with repeated constraints?" Tags="|linear-algebra|finite-element|iterative-method|saddlepoint|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27260" PostTypeId="1" AcceptedAnswerId="27262" CreationDate="2017-06-27T19:15:23.677" Score="0" ViewCount="105" Body="&lt;p&gt;Let us consider the conventions on names used in the theoretical derivation of Metropolis-Hastings Monte Carlo as outlined &lt;a href=&quot;https://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm#Formal_derivation&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, for the sake of common nomenclature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we are building is a step-by-step Markov Chain Monte Carlo (MCMC) algorithm to describe the evolution of a system in an initial state towards a final state distributed according to a desired probability distribution $P(x)$. &#xA;This final sentence is to be read in the sense that repeated iterations of the algorithm, on distinct initial states, lead to an ensemble of states distributed according to a desired distribution $P(x)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each of these iterations, during each step, given an initial state $x$ for the system and a final state $x'$, the probability of the system moving from $x$ to $x'$ is factorized into the proposal probability $g(x'|x)$ and the acceptance probability $A(x'|x)$ -- i.e. $P(x'|x) = g(x'|x) A(x'|x)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The meaning of the proposal probability is that of being the probability associated to proposing the next state to be $x'$ if we start from the state $x$. &#xA;That of the acceptance probability is the probability of accepting the state $x'$ if we start from the initial state $x$ and derives from the final desired distribution of the states $P(x)$ -- aside from physically justified fluctuations, the most probable states are accepted and the least ones rejected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that is stated above is valid for any MCMC method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a common scenario, the specific case of the Metropolis process -- which is a particular MCMC method -- we choose $g(x'|x)$ to be symmetrical. But beside it being so, we generally pose no further constraints on this choice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It left me with some open questions: how does the choice of the form of the proposal distribution influence the MCMC algorithm? Does it depend on the system in analysis?&#xA;Specifically, is there any physical meaning behind the choice of the proposal distribution $g(x'|x)$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[Cross posted from physics.stackexchange]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="24691" LastActivityDate="2017-06-27T20:23:40.260" Title="Physical meaning behind the choice of the proposal distribution in Markov Chain Monte Carlo (MCMC) methods" Tags="|algorithms|computational-physics|monte-carlo|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27261" PostTypeId="2" ParentId="27260" CreationDate="2017-06-27T19:58:48.273" Score="0" Body="&lt;p&gt;It's less a question of physical meaning, and more a question of computational efficiency. The closer the proposal density is matched to the desired probability density, the faster the chain will converge to the probability distribution. I do not know of the proof of this, but it is straightforward to examine the end points. If the proposal distribution is much much wider than $P$, then the fraction of proposals that are rejected will be very high, converging to $100\%$ as the proposal width becomes infinitely wider than the width of $P$, causing the chain to contain only repeats of the original point. If the proposal distribution is much narrower than than $P$, then it will take more and more steps to explore the support of $P$, approaching an infinite number of steps as the width of the proposal distribution divided by the width of $P$ goes to $0$ (i.e. a delta function proposal always proposes the same point). My understanding is that the ideal acceptance rate for converging to $P$ at optimal speed is $50\%$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take all of this with a grain of salt, though, because I do not know of a rigorous proof.&lt;/p&gt;&#xA;" OwnerUserId="8313" LastEditorUserId="8313" LastEditDate="2017-06-27T20:10:17.827" LastActivityDate="2017-06-27T20:10:17.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27262" PostTypeId="2" ParentId="27260" CreationDate="2017-06-27T20:23:40.260" Score="1" Body="&lt;p&gt;In an extreme case, if $g(x^\prime | x) = P(x^\prime | x)$ then $A(x^\prime | x) = 1$ for all $x$ and $x^\prime$. This means that the proposal coincides with the conditional probability of the desired distribution. It is tempting to think that by choosing $g(x^\prime | x)$ close (in some sense) to $P(x^\prime, x)$, we will have a good MCMC sampler: the physical interpretation of $g(x^\prime | x)$ is straight forward in that case and we can ensure a very high acceptance ratio because $A \approx 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in practice it is better to adopt a $g(x^\prime | x)$ that allows the system to transition from $x$ to a $x^\prime$ that is reasonably far away (in some suitable metric) from $x$ relatively often (i.e., with a relatively high expected acceptance ratio). Think, for example, of a system with many metastable states: if $g(x^\prime | x) = P(x^\prime | x)$ then the sampler will draw many points from the local metastable state but the MCMC chain will in general take a long number of iterations to explore the state space of the system (i.e., its &lt;em&gt;mixing time&lt;/em&gt; will be large and it will only converge to the stationary distribution after a huge number of steps). By contrast, if we know something about the system and we judiciously select a function $g$ that allows for jumps between far away points in the state space sufficiently often, then the chain will converge faster. An example of this approach can be found in:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minary, P., &amp;amp; Levitt, M. (2010). Conformational Optimization with Natural Degrees of Freedom: A Novel Stochastic Chain Closure Algorithm. Journal of Computational Biology, 17(8), 993–1010. &lt;a href=&quot;http://doi.org/10.1089/cmb.2010.0016&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doi.org/10.1089/cmb.2010.0016&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-06-27T20:23:40.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27267" PostTypeId="1" CreationDate="2017-06-28T09:55:52.800" Score="1" ViewCount="232" Body="&lt;p&gt;Computational Science has many more parameters than my previous programming, and I'm finding it difficult to keep track. What's the best way to track them?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;editing source, commenting-out&lt;/strong&gt;. Good for trying things quickly, but after some exploration, I've forgotten what the earlier ones were. Much harder when a setup requires changes to more than one part of the code (e.g. changing viscosity and initial conditions (IC)). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;in a class&lt;/strong&gt; For each experiment, write a class with a method for each part of the set-up, then call those methods from the appropriate parts of the main code (e.g. specific IC, &lt;strong&gt;and&lt;/strong&gt; specific boundary conditions (BC), &lt;strong&gt;and&lt;/strong&gt; a combination of values for $\nu$ and $\Delta x$, &lt;strong&gt;and&lt;/strong&gt; excluding/switching off some aspect, like advection, &lt;strong&gt;and&lt;/strong&gt; maybe also changing visualization parameters, like scaling - though that really should be separate). Having one class for all these separate changes keeps them together in one place. (To run an experiment, instantiate the class for it, and plug it into the main code.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;code&lt;/strong&gt;: A further benefit of a class is that actual code is easily included e.g. It's common for IC to be defined as a function or method, not data points, because the former is more intelligible and modifiable. Sometimes,  the experiment is to change mathematical expressions/discretizations (i.e. again, executable code).&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;file format&lt;/strong&gt; Store the parameters in a file, and read it in and configure the appropriate parts of the main code. This doesn't work so well for choosing different mathematical expressions, unless you're using a dynamic language (e.g. python). Could use a proper database (e.g. SQL) instead of files.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Though these don't include the results, such as data, plots, images.&#xA;The above are my programmer-centric thoughts, but it occurs to me that Computational Science may have a more &quot;science&quot; approach, i.e. methodically recording experimental inputs and results. It could be done by making notes on paper, or in some form easier for publication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps also, standard software packages include means for easily recording inputs and results?  e.g. automatically journaled, so you can easily go back, if and when you need to write-up or revisit a particular experiment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested both in what is considered the best way... but also, what  &lt;del&gt;people&lt;/del&gt; you actually do, which might be better than the so-called &quot;best&quot; (which might be too rigid, adding complexity overhead).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; We have limited attention, and code, paramaters, experiment and hypotheses can be complex enough, without unnecessary management overhead - especially if you're still learning (as I am). So, quick-and-dirty sometimes &lt;em&gt;can&lt;/em&gt; be better trade-off, in practice, taking into account the true context and needs, than a &quot;proper&quot; heavy-weight industrial/institutional/enterprise grade solution.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-07-10T09:12:25.000" LastActivityDate="2017-07-19T04:36:11.207" Title="Best way to track parameters and results, in your experience (i.e. how do you do it)?" Tags="|fluid-dynamics|data-management|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27268" PostTypeId="2" ParentId="11006" CreationDate="2017-06-28T13:09:45.957" Score="24" Body="&lt;p&gt;In 2014, I would've said Python. In 2017, I wholeheartedly believe that the language to teach undergraduates is Julia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Teaching is always about a tradeoff. On one hand, you want to choose something that is simple enough that it is easy to grasp. But secondly, you want to teach something that has staying power, i.e. something that can grow with you. The common dynamic languages (Python/MATLAB/R) all easily fall into category 1 due to their non-existent boilerplate code and the ease of opening up an interpreter and spitting out code, while C/C++/Fortran fall into the second category as the languages with which the core highly-performant software of today's world were written.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are issues with using a language which doesn't fully capture the other category. When using a language like Python, it nicely abstracts away things like types and integer overflow. This is nice for teaching the first semester computing, but as you want to dig deeper and deeper into how things are actually working, Python's language is too far abstracted away from the underlying metal to be a good teaching tool. But C/C++/Fortran (or Java... I learned Java first...) all have such a big startup cost that the hardest thing to learn is just how to get headers setup and &lt;code&gt;main&lt;/code&gt; compiled, which distracts from actually learning to program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enter Julia. When you first use Julia, you can abstract away the entire idea of types and use it just like MATLAB or Python. But as you want to learn more, there is a &quot;rabbit hole&quot; of depth to the language. Since it's really an abstraction layer based on a type system + multiple dispatch over LLVM, it is essentially &quot;an easy way to write statically compiled code&quot; (and type-stable functions can actually be statically compiled). What this means is that the details of C/C++ are also accessible. You can learn how to write simple loops and functions without boilerplate code, and then dig into the function pointers. Julia's metaprogramming features let you directly access the AST, and there are macros which show every part of the compilation chain. Also, as a Lisp, is amenable to functional programming styles. And it has a lot of parallel computing capabilities. Ideas like parametric typing and type-stability are fairly unique and deep in Julia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to study programming languages themselves, you can learn the steps of how compilation works by using &lt;code&gt;@code_lowered&lt;/code&gt; to see what lowering is, see the typed-AST with &lt;code&gt;@code_typed&lt;/code&gt;, see LLVM IR with &lt;code&gt;@code_llvm&lt;/code&gt;, and finally the native assembly code with &lt;code&gt;@code_native&lt;/code&gt;. This can be used to show what the cost of dynamic variables is and exactly how &quot;variable boxing&quot; works, &lt;a href=&quot;http://www.stochasticlifestyle.com/7-julia-gotchas-handle/&quot; rel=&quot;noreferrer&quot;&gt;and this blog post&lt;/a&gt; shows how these introspection tools can be used to teach how compiler optimizations can/cannot occur.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are not just computer science and software engineering ideas to explore, but also rich mathematical ideas. Since Julia's main libraries are written with generic typing in mind, it's trivial to create matrix-free operators and use IterativeSolvers.jl to perform GMRES using them. You can use introspection tools like &lt;code&gt;@which&lt;/code&gt; to show you exactly how anything was implemented. For example, how does &lt;code&gt;\&lt;/code&gt; work?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@which rand(10,10)\rand(10)&#xA;#\(A::AbstractArray{T,2} where T, B::Union{AbstractArray{T,1}, AbstractArray{T,2}} where T) in Base.LinAlg at linalg\generic.jl:805&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That points me straight to &lt;a href=&quot;https://github.com/JuliaLang/julia/blob/master/base/linalg/generic.jl#L804&quot; rel=&quot;noreferrer&quot;&gt;the definition of \&lt;/a&gt;. It is implemented in Julia, so someone who knows Julia can then learn the algorithm and how it works through identifying matrix subtypes and specializing when possible (falling back to Gaussian elimination). Since Julia's code is MIT licensed (and almost all packages are MIT licensed), students are then free to use these ideas in their own code (with attribution) (when code is GPL licensed, as is the case with most MATLAB and R packages, they need to be careful about licensing issues!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the language core is built with a very active open source community, there is also a rich resource on the history of the language development: &lt;a href=&quot;https://github.com/JuliaLang/julia/issues&quot; rel=&quot;noreferrer&quot;&gt;its Github issues&lt;/a&gt;. Understanding language questions like &lt;a href=&quot;https://github.com/JuliaLang/julia/issues/20978&quot; rel=&quot;noreferrer&quot;&gt;what really is a matrix transpose?&lt;/a&gt; can be very enlightening for understanding these mathematical objects in greater detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But lastly, in the end, you want to teach your students how to create. Sadly, learning Python or R doesn't necessarily mean you have what it takes to &quot;develop Python/R&quot; since most of the widely-used and well-optimized packages have a substantial amount of C/C++/Fortran code in them in order to get performance. Thus, for these students to be able to contribute to the scientific ecosystems for these languages, they will eventually have to learn another language at some point. While that's not entirely awful, it's sub-optimal now that Julia exists. Since type-stable Julia is able to achieve the speed of C/Fortran, most of the packages in the Julia ecosystem are pure Julia code. Learning Julia means one has learned to develop Julia. And since Base Julia is also mostly Julia code (just a few primitives and the parser isn't), they can also contribute there as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, there are some downsides to choice of Julia. For one, it is much newer than these other languages and so it's a bit more scarce on resources. You'll have to come up with a lot of teaching tools on your own, or pull from resources on the web &lt;a href=&quot;https://julialang.org/learning/&quot; rel=&quot;noreferrer&quot;&gt;which are listed on the Julia website&lt;/a&gt;. Also, the language details aren't quite settled, though 1.0 is coming out soon (by the end of 2017). And it's also quite likely that you, the potential teacher of a course in Julia, might not have that much experience with the language yourself. However, these are the kinds of problems which go away over time, whereas the benefits of Julia that I mentioned above are much more core to the languages themselves.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-06-28T13:09:45.957" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27269" PostTypeId="1" CreationDate="2017-06-28T16:25:54.763" Score="1" ViewCount="165" Body="&lt;p&gt;This question is the essence of this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/27259/how-can-i-numerically-solve-a-saddle-point-problem-with-repeated-constraints&quot;&gt;one&lt;/a&gt;.  After we remove all the cruft, we can recast it as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt; Given $b \in \mathbb{R}^n$, $C\in \mathbb{R}^{n\times m}$, and $g\in \mathrm{Range}(C^\mathrm{T})\subset \mathbb{R}^m$.  Find $x\in \mathbb{R}^n$ so that $\| x-b \|$ is minimized subject to the constraint $C^\mathrm{T} x = g$.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If $C$ had full column rank, this is straight forward problem, but I don't have that, and instead I know that $g\in \mathrm{Range}(C^\mathrm{T})$.  In my case $m$ is large but small compared to $n$. How can I do this?&lt;/p&gt;&#xA;" OwnerUserId="16799" LastEditorUserId="20417" LastEditDate="2017-07-01T14:00:51.937" LastActivityDate="2017-07-05T21:27:36.300" Title="How can I solve on a computer a large projection problem with redundant constraints?" Tags="|linear-algebra|optimization|convex-optimization|constrained-optimization|quadratic-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27270" PostTypeId="2" ParentId="18760" CreationDate="2017-06-28T19:54:23.233" Score="3" Body="&lt;p&gt;As already mentioned, the main problem is not with the visualization algorithm but with the data. Hence, you should use a different scaling (normalization) for the values. Furthermore, if you want to color glyphs  (arrows, in this case) they should be of a size that allow them to have aesthetic values (like color). Then, I have modified slightly the answers provided before to show the colormap/vector direction field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also added another option for the visualization of your vector field: streamplots. Besides that, my answer is just a summary of what people already mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See my code below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division, print_function&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;def elec_field(x, y, x_coords=[0], y_coords=[0], charges=[1]):&#xA;    Ex = np.zeros_like(x)&#xA;    Ey = np.zeros_like(x)&#xA;    for x0, y0, q in zip(x_coords, y_coords, charges):&#xA;        R = np.sqrt((x - x0)**2 + (y - y0)**2) + 1e-6&#xA;        Ex += q*(x - x0)/R**3&#xA;        Ey += q*(y - y0)/R**3&#xA;    return Ex, Ey&#xA;&#xA;&#xA;y, x = np.mgrid[-3:3:21j, -3:3:21j]&#xA;Ex, Ey = elec_field(x, y, x_coords=[-1, 1], y_coords=[0,0],&#xA;                    charges=[1, -1])&#xA;Emag = np.sqrt(Ex**2 + Ey**2)&#xA;dir_x = Ex/Emag&#xA;dir_y = Ey/Emag&#xA;&#xA;plt.figure(figsize=(8, 6))&#xA;plt.subplot(221)&#xA;plt.quiver(x[::2, ::2], y[::2, ::2], dir_x[::2, ::2], dir_y[::2, ::2],&#xA;           np.log10(Emag[::2, ::2]), cmap=&quot;autumn&quot;, scale_units=&quot;inches&quot;,&#xA;           scale=5, width=0.015, pivot=&quot;mid&quot;)&#xA;plt.colorbar()&#xA;plt.axis(&quot;image&quot;)&#xA;&#xA;plt.subplot(222)&#xA;plt.streamplot(x, y, Ex, Ey, color=np.log10(Emag),cmap=&quot;autumn&quot;)&#xA;plt.colorbar()&#xA;plt.axis(&quot;image&quot;)&#xA;&#xA;&#xA;plt.subplot(223)&#xA;plt.contourf(x, y, np.log10(Emag), cmap=&quot;autumn&quot;)&#xA;plt.colorbar()&#xA;plt.quiver(x[::2, ::2], y[::2, ::2], dir_x[::2, ::2], dir_y[::2, ::2],&#xA;           scale_units=&quot;inches&quot;, scale=5, width=0.010, pivot=&quot;mid&quot;)&#xA;plt.axis(&quot;image&quot;)&#xA;&#xA;&#xA;plt.subplot(224)&#xA;plt.contourf(x, y, np.log10(Emag), cmap=&quot;autumn&quot;)&#xA;plt.colorbar()&#xA;plt.streamplot(x, y, Ex/Emag, Ey/Emag, color=&quot;black&quot;)&#xA;plt.axis(&quot;image&quot;)&#xA;&#xA;plt.savefig(&quot;vector_field.png&quot;, dpi=300)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the result&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Qg3aA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Qg3aA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-06-28T20:03:50.140" LastActivityDate="2017-06-28T20:03:50.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27271" PostTypeId="1" CreationDate="2017-06-28T20:55:48.357" Score="0" ViewCount="181" Body="&lt;p&gt;I have a sparse symmetric matrix of dimension &lt;code&gt;1393x1393&lt;/code&gt; (8308 no zero elements), with bandwidth 1380. By Cuthill–McKee algorithm, I could achieve a new matrix with bandwidth 89. If we call this matrix &lt;code&gt;B&lt;/code&gt;, then I'm interested in calculate the inverse &lt;code&gt;I-rho*B&lt;/code&gt;, where &lt;code&gt;I&lt;/code&gt; is a diagonal matrix and &lt;code&gt;rho&lt;/code&gt; is a parameter updated in my MCMC algorithm. I need to calculate matrix inverse many times (during MCMC) and someone told me by reducing the matrix bandwidth, I can obtain faster matrix inverse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using R as my programming language. The &lt;code&gt;solve&lt;/code&gt; function is used to calculate the inverse of a matrix. I noticed no difference when calculating the inverse of a sparse matrix with bandwidth 1380 or bandwidth 89.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess I need some explicit commands/options in order to take advantage of the bandwidth reduction, but I don't know what keywords should I aim at. Can anyone give me some suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/44810180/inverse-a-sparse-matrix-with-reduced-bandwidth-faster-or-not&quot;&gt;A former post&lt;/a&gt; with more information.&lt;/p&gt;&#xA;" OwnerUserId="24710" LastActivityDate="2017-06-29T03:56:57.643" Title="Sparse matrix inverse with reduced bandwidth" Tags="|sparse-matrix|matrix|r|inverse|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27272" PostTypeId="1" CreationDate="2017-06-28T22:23:59.363" Score="0" ViewCount="159" Body="&lt;p&gt;I have this data plotted on a graph in which all points have the same value on the y-axis, e.g a constant integer &quot;c&quot;, while the x-axis is the time in seconds.&#xA;So, for a c = 25 on the y-axis, there are points on the graph corresponding to values on the x-axis equals to, let's say,  32, 59, 78, 142, 249, 286, 301, 310, 398, etc.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to find a sequence (arithmetic progression) of the type       &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a_n = a_{n-1} + T $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where, where $a_n$ is the time in the x-axis and $T$ the ratio of the AP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know the period $T$ nor the first element of the sequence (and therefore when the sequence begins) and there is white noise (probably most of the points that are in the data are not part of the sequence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in short, I would like to discover a sequence with period $T$ in the middle of a bunch of &quot;random&quot; numbers, given these numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking first about a brute force algorithm but that would have a running time of $O(n^2)$ but I believe there might be ways to improve or even algorithms that might suit this kind of job. Beforehand, I tried FFT (Fast Fourier Transform), but it did not work on this case because the data is not uniform. Any suggestions on algorithms or ideas to tackle this problem are more than welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the representation of the data on a graph: &lt;a href=&quot;https://i.stack.imgur.com/eZ0Qv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24709" LastActivityDate="2018-06-14T23:44:39.503" Title="Find hidden sequence $a_n = a_{n-1} + T $ , with period $T$, given some &quot;random&quot; numbers" Tags="|algorithms|software|interpolation|performance|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27274" PostTypeId="2" ParentId="27271" CreationDate="2017-06-29T03:56:57.643" Score="1" Body="&lt;p&gt;Three things immediately come to mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;R might not take advantage of sparsity when using the &lt;code&gt;solve&lt;/code&gt; command to compute the inverse of a matrix. Usually, the inverse of a sparse matrix is dense, so the built-in procedure might have been written to just convert it to a dense matrix, figuring that there's rarely anything to be gained from sparsity. To check this, see if you can print out any type info for the inverse matrix, or failing that check the number of non-zero entries in the inverse.&lt;/li&gt;&#xA;&lt;li&gt;Reducing the bandwidth of a sparse matrix does reduce the fill-in of sparse matrix factorization. However, this is an indirect way to get at what you really want: matrix factors that are as sparse as possible. You might instead try using an &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimum_degree_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;approximate minimum degree ordering&lt;/a&gt;, which could give you better results &lt;em&gt;if&lt;/em&gt; the factorization algorithm in R knows how to take advantage of sparsity.&lt;/li&gt;&#xA;&lt;li&gt;Finally, computing the inverse of a matrix is very poorly conditioned from a numerical perspective in a lot of cases. Do you really need to be computing the inverse matrix, or is it enough to be able to multiply several vectors by the inverse matrix? In that case, a sparse LU factorization is enough for your purposes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-06-29T03:56:57.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27276" PostTypeId="1" CreationDate="2017-06-29T14:35:27.940" Score="0" ViewCount="695" Body="&lt;p&gt;I want to solve the 2D poisson problem using the interior penalty discontinuous Galerkin methods: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;−∇a(x)(∇u)=0 in Ω.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variational formulation is such that :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a_{\epsilon}(u,v)=\sum_{K\in T_h}\int_K a\nabla u\cdot \nabla v-\sum_{e\in \Gamma_h\cup \Gamma_D}\int_e\{a\nabla u\cdot n_e\}[v]+ \sum_{e\in \Gamma_h\cup \Gamma_D}\int_e\epsilon\{a\nabla v\cdot n_e\}[u] + \sum_{e\in \Gamma_h\cup \Gamma_D}\frac{\sigma_0}{|e|}\int_e[u][v]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I managed to get the implementation right and to compute and assemble the flux terms for a quadrilateral element. Yet for the triangles, it gets a little bit hard and I dont know how to do it. Can someone help me please?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function Sigma = edgeT3_Lagrange(x_node_edge,y_node_edge,eps,alp,gam,bet,edge_data)&#xA;&#xA;    %% Calculate normal outward poining normalized vector&#xA;    delta_x = x_node_edge(:,end)-x_node_edge(:,1); &#xA;    delta_y = y_node_edge(:,end)-y_node_edge(:,1); &#xA;&#xA;    % Tangentiels &#xA;    t1 = delta_x ; &#xA;    t2 = delta_y ;  &#xA;&#xA;    % Normal outward poining normalized vector &#xA;    nv = [t2;-t1]/sqrt((t1^2)+(t2^2));&#xA;&#xA;    % Calculate the trace : Length of the edge &#xA;    trace_edge  = sqrt((delta_x)^2 + (delta_y)^2);&#xA;&#xA;    % the average local mesh size &#xA;    h_avg  = (h_E1 + h_E2)/2 ; /circumradius &#xA;    h_avg_beta  = h_avg.^bet ; &#xA;&#xA;    %% Space allocation&#xA;    % Matrices Initiation &#xA;    Sigma = [{zeros(DG_DATA.NN_elem)} {zeros(DG_DATA.NN_elem)} {zeros(DG_DATA.NN_elem)} {zeros(DG_DATA.NN_elem)}];&#xA;&#xA;    %% Gauss points and weights&#xA;    N_GAUSS_POINT = 3; %total number of gauss points = (N_GAUSS_POINT)^2&#xA;    GAUSS_POINT   = [-1 1 0 ]*sqrt(3/5);&#xA;    GAUSS_WEIGHT  = [5 5 8 ]/9;&#xA;&#xA;    %% Calculus Loop sur Sigma de s&#xA;    for ng=1:N_GAUSS_POINT&#xA;&#xA;        %-----------------------------------------------------------------------&#xA;        %-----------------------------------------------------------------------&#xA;            s = GAUSS_POINT(ng);&#xA;&#xA;            % Jacobian matrix for surface integral&#xA;            Jc     = trace_edge/2 ; &#xA;            detJc  = det(Jc); &#xA;&#xA;        %-----------------------------------------------------------------------&#xA;        %-----------------------------------------------------------------------&#xA;        % Assignment of the edge&#xA;        %-------------------------------------&#xA;        %-------------------------------------&#xA;        % For     :      the edge {i} of E1 &#xA;        % Thereby &amp;gt;&amp;gt;     the edge {j} of E2 &#xA;        % And the integration is done on s&#xA;        %-------------------------------------&#xA;        %-------------------------------------&#xA;&#xA;            S = (s+1)/2;    &#xA;&#xA;        %   t  = [xi  nu]&#xA;            t  = [S        0;&#xA;                  1-S      S;&#xA;                   0       S;];&#xA;&#xA;        %-----------------------------------------------------------------------&#xA;        %-----------------------------------------------------------------------&#xA;&#xA;            % Legendre polynomials p=1&#xA;            % corresponding to the element E1 &#xA;            xi1        = t(edge_number(1),1);  &#xA;            nu1        = t(edge_number(1),2);  &#xA;            P1         = elemT3_Poly(xi1,nu1); &#xA;            N_1        = P1{1,1}                ; % [N]&#xA;            N_dot_1    = [P1{1,2}; P1{1,3}]     ; % [N,xi ; N,nu]&#xA;&#xA;            % Jacobian matrix for the reference element&#xA;            J1         = (N_dot_1*[x_node_1' y_node_1'])' ;&#xA;            j1         = inv(J1'); % (J')^-1&#xA;            N_dotp_1   = ( j1 * N_dot_1); &#xA;&#xA;            % &#xA;            % Legendre polynomials p=1&#xA;            % corresponding to the element E2 &#xA;&#xA;            xi2        = t(edge_number(2),1);  &#xA;            nu2        = t(edge_number(2),2);&#xA;            P2         = elemT3_Poly(xi2,nu2); &#xA;            N_2        = P2{1,1}                ; % [N]&#xA;            N_dot_2    = [P2{1,2}; P2{1,3}]     ; % [N,xi ; N,nu]    &#xA;&#xA;            % Jacobian matrix for the reference element&#xA;            J2         = (N_dot_2*[x_node_2' y_node_2'])';&#xA;            j2         = inv(J2'); % (J')^-1&#xA;            N_dotp_2   = ( j2 * N_dot_2) ; &#xA;&#xA;        %-----------------------------------------------------------------------&#xA;        %-----------------------------------------------------------------------&#xA;&#xA;            % Flux expressions&#xA;            Sigma{1} = Sigma{1} + GAUSS_WEIGHT(ng)*( -0.5*N_1.'*(nv'*(N_dotp_1)) + (eps/2)*((N_dotp_1).'*nv)*N_1 + (alp/h_avg_beta)*(N_1.')*N_1 + (gam/h_avg_beta)*(N_dotp_1.')*N_dotp_1 )*detJc ; %B&#xA;            Sigma{2} = Sigma{2} + GAUSS_WEIGHT(ng)*( +0.5*N_2.'*(nv'*(N_dotp_2)) - (eps/2)*((N_dotp_2).'*nv)*N_2 + (alp/h_avg_beta)*(N_2.')*N_2 + (gam/h_avg_beta)*(N_dotp_2.')*N_dotp_2 )*detJc ; %C&#xA;            Sigma{3} = Sigma{3} + GAUSS_WEIGHT(ng)*( +0.5*N_2.'*(nv'*(N_dotp_1)) + (eps/2)*((N_dotp_2).'*nv)*N_1 - (alp/h_avg_beta)*(N_2.')*N_1 - (gam/h_avg_beta)*(N_dotp_2.')*N_dotp_1 )*detJc ; %D    &#xA;            Sigma{4} = Sigma{4} + GAUSS_WEIGHT(ng)*( -0.5*N_1.'*(nv'*(N_dotp_2)) - (eps/2)*((N_dotp_1).'*nv)*N_2 - (alp/h_avg_beta)*(N_1.')*N_2 - (gam/h_avg_beta)*(N_dotp_1.')*N_dotp_2 )*detJc ; %E&#xA;&#xA;        %-----------------------------------------------------------------------&#xA;        %-----------------------------------------------------------------------&#xA;&#xA;    end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23068" LastEditorUserId="9667" LastEditDate="2019-03-12T14:05:44.343" LastActivityDate="2019-03-12T14:05:44.343" Title="Interior penalty discontinuous Galerkin Matlab implementation" Tags="|matlab|discontinuous-galerkin|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="27277" PostTypeId="1" CreationDate="2017-06-29T16:07:34.547" Score="4" ViewCount="1113" Body="&lt;p&gt;The documentation is fairly confusing and I know that I need to keep track of the actual grid location of my data. Writing a python script to convert binary data into this vtk format. I want to leave the legacy VTK behind so that I can utilize the parallelism that XML will give. Any help here is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="24720" LastActivityDate="2018-06-25T18:11:01.100" Title="How do I migrate from Legacy VTK (Structured Points) to XML (Structured Grid)?" Tags="|visualization|vtk|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27279" PostTypeId="2" ParentId="27277" CreationDate="2017-06-29T17:04:13.100" Score="0" Body="&lt;p&gt;I don't think there is a good argument to write a converter -- whatever it produces has the same information content as the existing VTK files you already have, and the usual visualization programs can visualize them just fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there is a good case to be made for re-writing the code you have that produces VTK files. I agree that the VTU file is confusing and poorly documented, but take a look at some VTU files that you can find on the web. As an example, there are numerous VTU files in subdirectories here: &lt;a href=&quot;https://github.com/geodynamics/aspect/tree/master/tests&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/geodynamics/aspect/tree/master/tests&lt;/a&gt; (you may just want to check out the entire git repository to look for them). You can also look at code that writes VTU files -- e.g., the &lt;code&gt;write_vtu()&lt;/code&gt; function here: &lt;a href=&quot;https://github.com/dealii/dealii/blob/master/source/base/data_out_base.cc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dealii/dealii/blob/master/source/base/data_out_base.cc&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-06-29T17:04:13.100" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27281" PostTypeId="2" ParentId="27228" CreationDate="2017-06-29T22:29:19.467" Score="8" Body="&lt;p&gt;&lt;strong&gt;Continuous finite elements&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, if $A$ is your finite element discretization on the finest mesh, $A_i = R_i * A * R_i^T$.  So, for $i=0$, $A_0$ corresponds to the finite element discretization of your problem on the coarsest mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $R_0$ and $R_0^T$ are algebraically defined, you can use the &quot;Galerkin&quot; triple product to form $A_0=R_0 * A * R_0^T$.  This is what is done in algebraic multigrid.  If $R_0$ and $R_0^T$ have geometric meaning, $A_0$ can be constructed by simply assembling your finite element problem on the coarsest mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within the two-level additive Schwarz iteration, usually the matrices $A$ and $A_0$ remain fixed.  If you are trying to solve $Ax=b$, and $x_j$ is your previous guess to the problem, then the $j+1$ iteration of the two-level additive Schwarz method looks like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_{j+1} = x_j + \sum_{i=0}^N R_i^T A_i^{-1}R_i  (b-Ax_j).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, the matrices $A_0$ and $A_i$ do not depend on $j$.  &lt;a href=&quot;https://hal.archives-ouvertes.fr/cel-01100932/file/bookddm.pdf&quot; rel=&quot;noreferrer&quot;&gt;Chapter 4 in this free book&lt;/a&gt; explains why a coarse space is useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Nonlinear problems&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Nonlinear problems that are resolved with an outer Newton or Picard iteration, the matrices $A$ and $A_0$ are dependent on the outer iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Spectral elements&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of spectral elements, many authors select $A_0$ so that it corresponds to a finite element problem on the coarsest mesh with a smaller polynomial order (for example look at &lt;a href=&quot;http://www.mcs.anl.gov/~fischer/pubhtml/ima2000.pdf&quot; rel=&quot;noreferrer&quot;&gt;Fischer et al&lt;/a&gt; and &lt;a href=&quot;http://www.courant.nyu.edu/dd16/proceedings/pavarino_contributed.pdf&quot; rel=&quot;noreferrer&quot;&gt;Pasquetti et al&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discontinuous Galkerin methods&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has been shown that the selection of coarse spaces for discontinuous Galkerkin methods can be a little more sensitive.  Further details can be seen in the following references &lt;a href=&quot;https://www.cims.nyu.edu/dd16/proceedings/olson_contrib.pdf&quot; rel=&quot;noreferrer&quot;&gt;Olson et al&lt;/a&gt;, &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/nla.504/abstract&quot; rel=&quot;noreferrer&quot;&gt;Dobrev et al&lt;/a&gt;, &lt;a href=&quot;http://www1.mate.polimi.it/~antonietti/DOWNLOAD/Antonietti_PhDthesis.pdf&quot; rel=&quot;noreferrer&quot;&gt;Antonietti&lt;/a&gt;, and &lt;a href=&quot;http://trace.tennessee.edu/cgi/viewcontent.cgi?article=4887&amp;amp;context=utk_graddiss&quot; rel=&quot;noreferrer&quot;&gt;Collins&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11604" LastEditorUserId="11604" LastEditDate="2017-06-29T22:40:30.260" LastActivityDate="2017-06-29T22:40:30.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27282" PostTypeId="1" CreationDate="2017-06-30T08:31:36.170" Score="1" ViewCount="70" Body="&lt;p&gt;I want to solve a diffusion-convection problem numerically. I start from a PDE&#xA;of the form $$\partial_t f(x,t) = \partial_x(K(x)\;f(x,t))+\partial_{xx}\;f(x,t)^{m}\;.$$&#xA;It is possible to calculate a stationary solution $f_{st}(x)$, which looks something like this&#xA;$$&#xA;f_{st}(x) = (a \cosh(x-b))^c\;.&#xA;$$&#xA;Now I try to simplify the original PDE be assuming that the solution is similar to the stationary one, the only difference being that the coefficients $a$ and $b$ are now time dependent, i.e. $a(t)$ and $b(t)$.&#xA;This allows me to calculate the RHS of the PDE, reducing it formally to a first order ODE in time. My problem is that I'm not sure how to feed it to the computer, because due to the chain rule the spatial derivatives 'extract' additional instances of $a(t)$ and $b(t)$ out of $f$. In other words, I end up with a ODE like &#xA;$$&#xA;\partial_t(g(t)\;h(u(t)) = f(g(t),u(t))&#xA;$$&#xA;Intuitively I'd expect there to be a problem because I have two unknowns and just one equation but I'm trying to retrace a paper that seems to be doing just that. Anyways, thanks for reading, hope you can understand what I mean. &lt;/p&gt;&#xA;" OwnerUserId="24727" LastActivityDate="2017-06-30T08:31:36.170" Title="Solve ODE with two unknown functions" Tags="|ode|advection-diffusion|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27283" PostTypeId="1" CreationDate="2017-06-30T09:40:07.543" Score="1" ViewCount="243" Body="&lt;p&gt;I am looking for a R package that performs finite element analysis. Is there a fenics or equivalent FEM package for R?&lt;/p&gt;&#xA;" OwnerUserId="24728" LastActivityDate="2017-06-30T09:40:07.543" Title="Is there a FEM package for R" Tags="|r|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27285" PostTypeId="2" ParentId="27229" CreationDate="2017-06-30T15:29:24.753" Score="2" Body="&lt;p&gt;I'd suggest to try it on your own. Do an expansion of your wavefunction in terms of spherical harmonics,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\psi(\mathbf r) \ = \ \sum_{\ell} R_\ell(r,t) \, Y_{\ell 0} (\theta,\phi)\,.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I've set the index $m$ in $Y_{\ell m}$ to zero, in order to account for the symmetry of your Hamiltonian with respect to rotations around the $x-y$ plane. This makes your problem essentially two-dimensional. You can also write the spherical harmonic in terms of Legendre polynomials,&#xA;$$&#xA;Y_{\ell0}(\theta,\varphi) = \sqrt{\frac{2\ell+1}{4\pi}} P_{\ell}(\cos\theta).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Insertion into your Schrödinger equation and project on the spherical hamonics, and you'll end up with a coupled equation for the radial functions $R_l(r,t)$. Here, &quot;coupled&quot; means coupled in $\ell$, i.e. the function $R_\ell(r,t)$ depends on all the other quantum numbers $0\leq \ell \leq L_{\max}$. Solve that with standard finite differences, it's not that hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the insertion-and-projection step above, the only problem appears in evaluating the matrix elements with $x^2+y^2$. If you write it as $r^2 Y_{10}$, it boils down to an integral over three spherical harmonics, which is related to the Clebsch-Gordan or Wigner-3j coefficients. But It's an easy one, for which analytical formulae exist (--just google for the buzzwords in the previous sentence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you arrived at the working formula and need further assistance, let me know.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: summarizing our lengthy discussion in the comment section, here is the final equation which is about to be solved numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;i\frac{\partial}{\partial t} u_\ell(r,t) = \left(-\frac{1}{2} \frac{\partial^2}{\partial r^2} + \frac{\ell(\ell+1)}{2r^2} + V(r)\right) u_\ell(r,t) \\ \qquad \qquad\qquad\qquad\qquad\quad+  \frac{2}3 \,k(t)\, r^2 \; \sum_{\ell^\prime=\max(\ell-2,0)}^{\min(\ell+2,L_\max)} \left( \delta_{\ell,\ell^\prime} - \sqrt{\frac{4\pi}5} \alpha(\ell,\ell^\prime) \right)\,u_{\ell^\prime}(r,t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the coefficient $\alpha(\ell,\ell^\prime)$ which you introduced is given by&#xA;$$&#xA;\alpha(\ell,\ell^\prime) \ = \ \int Y^\ast_{\ell 0}(\Omega) Y_{20}(\Omega) Y_{\ell^\prime 0}(\Omega)\ d\Omega&#xA;$$&#xA;(you can also express that more in standard terms such as Wigner3j symbols, see e.g. &lt;a href=&quot;http://mathworld.wolfram.com/SphericalHarmonic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the restriction in the summation indices which come from the fact that $0\leq \ell^\prime \leq L_\max$ (where $L_\max$ is the maximum angular quantum number chosen in the numerical representation).&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2017-07-05T21:44:01.167" LastActivityDate="2017-07-05T21:44:01.167" CommentCount="22" ContentLicense="CC BY-SA 3.0" />
  <row Id="27288" PostTypeId="1" AcceptedAnswerId="27291" CreationDate="2017-06-30T20:38:23.800" Score="0" ViewCount="69" Body="&lt;p&gt;I have a random binary matrix $A$   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A=\left[\begin{array}{c c c c c}0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;1\\0&amp;amp;1&amp;amp;0&amp;amp;1&amp;amp;0\\1&amp;amp;1&amp;amp;1&amp;amp;1&amp;amp;0\\0&amp;amp;1&amp;amp;1&amp;amp;0&amp;amp;0\\1&amp;amp;0&amp;amp;1&amp;amp;1&amp;amp;1 \end{array}\right]&#xA;$$&#xA;and a permutation of the identity matrix (denoted later as $I_p$) that will filter it. I want to find the best permutation $I_p$ (using row/column swapping) so as to maximize $\text{sum}(AI_p)$ &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\text{sum}(AI_p)$ is computed using element-wise sum and multiplication   &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;ex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A=\left[\begin{array}{c c c}0&amp;amp;1&amp;amp;0\\1&amp;amp;0&amp;amp;1\\1&amp;amp;0&amp;amp;0 \end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the maximum $\text{sum}(AI)$ would be from &#xA;$$&#xA;I_p= \left[\begin{array}{c c c}0&amp;amp; 1&amp;amp; 0\\0&amp;amp; 0 &amp;amp;1\\1 &amp;amp;0&amp;amp; 0 \end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently checking every possible column permutation of $I$, which scales terribly. &lt;/p&gt;&#xA;" OwnerUserId="24735" LastEditorUserId="24735" LastEditDate="2017-06-30T22:00:24.083" LastActivityDate="2017-06-30T22:39:39.490" Title="Maximize sum(AI) for matrix A and any permutation of identity matrix I" Tags="|linear-algebra|optimization|algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27290" PostTypeId="2" ParentId="27257" CreationDate="2017-06-30T22:33:44.593" Score="1" Body="&lt;p&gt;When $p$ is an odd prime, the multiplicative group $\mathbb{Z}_{p^2}^*$ is cyclic of order $\varphi(p^2) = p(p-1)$ by the &lt;a href=&quot;http://people.math.gatech.edu/~mbaker/pdf/primroots.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primitive Element Theorem&lt;/a&gt;.  Thus any $a$ coprime to $p$ will have multiplicative order mod $p^2$ which divides $p(p-1)$, and there are $\varphi(p(p-1))$ elements of $\mathbb{Z}_{p^2}^*$ which attain order $p(p-1)$ exactly (here $\varphi$ is &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler%27s_totient_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler's totient function&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a possibility for confusion between the &lt;em&gt;primitive element&lt;/em&gt; of the cyclic ring $\mathbb{Z}_{p^2}$ under discussion and one for a &lt;em&gt;finite field&lt;/em&gt; of order $p^2$.  The latter is a quadratic field extension of $\mathbb{Z}_p$ and has multiplicative order $p^2-1$, while here we intend a residue mod $p^2$ with multiplicative order $p(p-1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the proof of the Primitive Element Theorem includes the observation that if $g$ is a primitive root in $\mathbb{Z}_p$, then either $g$ or $g+p$ will be primitive in $\mathbb{Z}_{p^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any one of these primitive elements $g$ &lt;em&gt;generates&lt;/em&gt; all of $\mathbb{Z}_{p^2}^*$ by taking successive powers, $g,g^2,g^3,\ldots,g^{p(p-1)}$.  But since such $g$ has the maximum order $p(p-1)$, it does &lt;em&gt;not&lt;/em&gt; satisfy $g^{p-1} \equiv 1 \bmod{p^2}$ (because that happens only if the order divides $p-1$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The elements $a\in \mathbb{Z}_{p^2}^*$ (residues mod $p^2$ coprime to $p$, and hence to $p^2$) whose order divides $p-1$ form a subgroup, as is easily checked by taking the product of two such elements.  We get the number of these whose orders divide $p-1$ by counting the powers of a generator $g$ which are multiples of $p$, i.e. $a=g^p,g^{2p},g^{3p},\ldots,g^{(p-1)p}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Omitting the last of these (which is $1\bmod{p^2}$), we have $p-2$ nontrivial solutions.  The difficulty (suggested by the Question's requirement that $1\lt a \lt 100$) lies in finding fairly small such solutions (because their density is about $1/p$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed finding primes where base $a=2$ works turns out to be a particularly difficult case (Wieferich primes, &lt;a href=&quot;https://oeis.org/A001220&quot; rel=&quot;nofollow noreferrer&quot;&gt;OEIS A001220&lt;/a&gt;), for which only two primes $1093$ and $3511$ are known to exist (possible primes have been checked out to $10^{18}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking into account the rather modest sizes for primes $p$ considered here (so that in particular $p-1$ is easily factored), and the desire for a small $a$, it would be natural to simply test the possible values of $1\lt a \lt 100$ to find any that satisfy $a^{p-1} \equiv 1 \bmod{p^2}$.  Possibly it will be useful to work out an addition chain for $p-1$ in connection with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Modular_exponentiation&quot; rel=&quot;nofollow noreferrer&quot;&gt;modular exponentiation scheme&lt;/a&gt; to check this.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2017-07-02T15:22:31.277" LastActivityDate="2017-07-02T15:22:31.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27291" PostTypeId="2" ParentId="27288" CreationDate="2017-06-30T22:39:39.490" Score="4" Body="&lt;p&gt;This is an &lt;a href=&quot;https://en.wikipedia.org/wiki/Assignment_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;assignment problem&lt;/a&gt;.  There are lots of fast algorithms for these problems.  How big are your instances?  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-06-30T22:39:39.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27292" PostTypeId="1" CreationDate="2017-07-01T06:12:56.647" Score="0" ViewCount="112" Body="&lt;p&gt;I have an image that looks like &lt;a href=&quot;https://drive.google.com/file/d/0B1X1jFSEgpjJNmMzZTZ6QkdSOTg/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; (it might appear low res in a browser because it is 16 MB). This image was taken with a scanning electron microscope, but because the equipment is outdated there is a lot of noise within the image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to do is develop a program (in Java) that would take this image as an input and try to remove the noise as much as possible to produce a high quality image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, being a beginner in the field of image/signal processing, I have no idea where to begin researching techniques. I was wondering if you guys had any good starting points for me to look into, or perhaps de-noising algorithms/techniques that I could research to potentially help me reach my goal.&lt;/p&gt;&#xA;" OwnerUserId="24736" LastActivityDate="2020-09-28T20:56:33.953" Title="Which technique to use for signal/image processing or noise removal?" Tags="|image-processing|signal-processing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27293" PostTypeId="2" ParentId="27292" CreationDate="2017-07-01T16:08:43.440" Score="2" Body="&lt;p&gt;You may start with median or Gaussian filters. There are many libraries that implement them and they are simple to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I think this approach may be not enough because from what I've seen this noise is not an image noise in the classical sense, i.e. it's not randomly distributed 'dots', but rather periodic 'waves' and their presence is connected to the measurement head performance. You could do a Fourier transform of this image, remove the highest spatial frequencies, which may get rid off most of these waves as they change much more quickly than the relevant information, and maybe smooth it with Gaussian filter afterwards.&lt;/p&gt;&#xA;" OwnerUserId="24737" LastActivityDate="2017-07-01T16:08:43.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27294" PostTypeId="1" CreationDate="2017-07-02T06:26:44.120" Score="1" ViewCount="109" Body="&lt;p&gt;&lt;strong&gt;My problem&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maximize&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{i} \{\ c_i \cdot \prod_{j \in A(i)} {x_{j}} \prod_{j \in B(i)} {y_{j}} \} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subject  to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\sum_{j \in C(k)} x_{j} = 1,\ \forall k \\&#xA;&amp;amp;l \leq x_{j} \leq u,\ \forall j\\&#xA;&amp;amp;\sum_{j \in D(k)} y_{j} = 1+p,\ \forall k\\&#xA;&amp;amp;y_{j} \in \{1,\ p\},\ \forall j&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c_i, l, u, p$ are positive constants less than 1, and sets $A, B, C, D$ are also given. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I have tried&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to use &lt;a href=&quot;http://ncpol2sdpa.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ncpol2sdpa&lt;/code&gt;&lt;/a&gt; to relax the polynomial programming into a semi-definite programming and call the &lt;a href=&quot;http://sdpa.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sdpa&lt;/code&gt;&lt;/a&gt; solver to solve it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is replaced with a new variable $ \ f$, and the following constraints are added&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ f \leq c_i \cdot \prod x_{j} \prod y_{j}, \forall i\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each discrete variable $y_i$ is replaced by $y_i = p \cdot (1-z_i) + z_i$, where $z_i = \{0,\ 1\}$ or equivalently, $z_i^2-z_i=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However even for a small problem (5 $x$'s and 8 $y$'s), it took hours for &lt;code&gt;ncpol2sdpa&lt;/code&gt; to relax and &lt;code&gt;sdpa&lt;/code&gt; to solve the relaxed problem. The level of relaxation is set to be 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to relax the constraint on $y$ to be continues, but still quite slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if I did anything wrong with both tools? Or is there a better method/solver for this problem?&lt;/p&gt;&#xA;" OwnerUserId="24742" LastEditorUserId="9667" LastEditDate="2017-07-02T19:19:38.630" LastActivityDate="2017-07-02T19:19:38.630" Title="Minimizing the products of variables" Tags="|optimization|nonlinear-programming|nonconvex|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27295" PostTypeId="2" ParentId="27257" CreationDate="2017-07-02T09:27:05.067" Score="0" Body="&lt;p&gt;The $p-1$ solutions of $a^{p-1}=1 \pmod{p^2}$ are the numbers of the form $n^p$, with $n=1,2,\dots,p-1$, I think. Compute them by binary powering modulo $p^2$, and see if any of them satisfies your additional requirement $1&amp;lt;a&amp;lt;100$.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-07-02T09:27:05.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27296" PostTypeId="2" ParentId="27269" CreationDate="2017-07-02T11:22:14.467" Score="1" Body="&lt;p&gt;We have the following linear system in &lt;span class=&quot;math-container&quot;&gt;$\mathrm x \in \mathbb R^n$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\rm A x = b \tag{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathrm A \in \mathbb R^{m \times n}$&lt;/span&gt; is &lt;strong&gt;fat&lt;/strong&gt; (i.e., &lt;span class=&quot;math-container&quot;&gt;$n &amp;gt; m$&lt;/span&gt;) and &lt;strong&gt;rank-deficient&lt;/strong&gt; (i.e., &lt;span class=&quot;math-container&quot;&gt;$r := \mbox{rank} (\mathrm A) &amp;lt; m$&lt;/span&gt;), and &lt;span class=&quot;math-container&quot;&gt;$\mathrm b \in \mathbb R^m$&lt;/span&gt; is in the column space of &lt;span class=&quot;math-container&quot;&gt;$\mathrm A$&lt;/span&gt;, i.e., the linear system is &lt;strong&gt;consistent&lt;/strong&gt;. We would like to find the solution of &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; that is closest to a given &lt;span class=&quot;math-container&quot;&gt;$\mathrm x_0 \in \mathbb R^n$&lt;/span&gt;. We thus have the following (convex) quadratic program&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \| \mathrm x - \mathrm x_0 \|_2^2\\ \text{subject to} &amp;amp; \mathrm A \mathrm x = \mathrm b\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's look for the solution of this QP using the singular value decomposition (SVD) of &lt;span class=&quot;math-container&quot;&gt;$\rm A$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Solution via unconstrained minimization&lt;/h3&gt;&#xA;&lt;p&gt;Let the SVD of &lt;span class=&quot;math-container&quot;&gt;$\rm A$&lt;/span&gt; be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathrm A = \mathrm U \Sigma \mathrm V^{\top} = \begin{bmatrix} \mathrm U_1 &amp;amp; \mathrm U_2\end{bmatrix} \begin{bmatrix} \Sigma_1 &amp;amp; \mathrm O\\ \mathrm O &amp;amp; \mathrm O\end{bmatrix} \begin{bmatrix} \mathrm V_1^{\top}\\ \mathrm V_2^{\top}\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hence, &lt;span class=&quot;math-container&quot;&gt;$\rm A x = b$&lt;/span&gt; can be written as &lt;span class=&quot;math-container&quot;&gt;$\rm \mathrm U \Sigma \mathrm V^{\top} x = b$&lt;/span&gt;, or, &lt;span class=&quot;math-container&quot;&gt;$\rm \Sigma \mathrm V^{\top} x = \mathrm U^{\top} b$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$\rm y := V^{\top} x$&lt;/span&gt;. We have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix} \Sigma_1 &amp;amp; \mathrm O\\ \mathrm O &amp;amp; \mathrm O\end{bmatrix} \begin{bmatrix} \mathrm y_1\\ \mathrm y_2\end{bmatrix} = \begin{bmatrix} \mathrm U_1^{\top} \mathrm b\\ \mathrm U_2^{\top} \mathrm b\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$\rm b$&lt;/span&gt; is in the column space of &lt;span class=&quot;math-container&quot;&gt;$\rm A$&lt;/span&gt;, it is orthogonal to the left null space of &lt;span class=&quot;math-container&quot;&gt;$\rm A$&lt;/span&gt;, i.e., &lt;span class=&quot;math-container&quot;&gt;$\mathrm U_2^{\top} \mathrm b = 0_{m-r}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix} \Sigma_1 &amp;amp; \mathrm O\\ \mathrm O &amp;amp; \mathrm O\end{bmatrix} \begin{bmatrix} \mathrm y_1\\ \mathrm y_2\end{bmatrix} = \begin{bmatrix} \mathrm U_1^{\top} \mathrm b\\ 0_{m-r}\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$\mathrm y_2$&lt;/span&gt; is free. The solution set is the &lt;span class=&quot;math-container&quot;&gt;$(m - r)$&lt;/span&gt;-dimensional affine space&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathrm y \in \left\{ \begin{bmatrix} \Sigma_1^{-1} \mathrm U_1^{\top} \mathrm b\\ \eta\end{bmatrix} : \eta \in \mathbb R^{m-r} \right\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$\rm x = V y$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathrm x \in \left\{ \color{blue}{\mathrm V_1 \Sigma_1^{-1} \mathrm U_1^{\top} \mathrm b + \mathrm V_2 \eta} : \eta \in \mathbb R^{m-r} \right\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathrm x_{\text{LN}} := \color{blue}{\mathrm V_1 \Sigma_1^{-1} \mathrm U_1^{\top} \mathrm b}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;is the &lt;strong&gt;least-norm&lt;/strong&gt; solution of &lt;span class=&quot;math-container&quot;&gt;$\rm A x = b$&lt;/span&gt;, as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\| \mathrm x \|_2^2 = \mathrm x^{\top} \mathrm x = \mathrm y^{\top} \underbrace{\mathrm V^{\top} \mathrm V}_{= \mathrm I_n} \, \mathrm y = \mathrm y^{\top} \mathrm y = \mathrm b^{\top} \mathrm U_1 \Sigma_1^{-2} \mathrm U_1^{\top} \mathrm b + \| \eta \|_2^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;is minimized when &lt;span class=&quot;math-container&quot;&gt;$\eta = 0_{m-r}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We now have an &lt;em&gt;unconstrained&lt;/em&gt; quadratic program in &lt;span class=&quot;math-container&quot;&gt;$\eta \in \mathbb R^{m-r}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \| \mathrm V_2 \eta - (\mathrm x_0 - \mathrm x_{\text{LN}}) \|_2^2\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$\rm V V^\top = V_1 V_1^\top + V_2 V_2^\top = I_n$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\rm V_1 V_1^\top$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rm V_2 V_2^\top$&lt;/span&gt; are the projection matrices that project onto the row space and the (right) null space of &lt;span class=&quot;math-container&quot;&gt;$\rm A$&lt;/span&gt;, respectively, the objective function can be written as follows&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{rl} \| \mathrm V_2 \eta - (\mathrm x_0 - \mathrm x_{\text{LN}}) \|_2^2 &amp;amp;= \| \mathrm V_2 \eta - \mathrm V_1 \mathrm V_1^\top (\mathrm x_0 - \mathrm x_{\text{LN}}) - \mathrm V_2 \mathrm V_2^\top (\mathrm x_0 - \mathrm x_{\text{LN}}) \|_2^2\\ &amp;amp;= \| \mathrm V_1 \mathrm V_1^\top ( \mathrm x_{\text{LN}} - \mathrm x_0 ) + \mathrm V_2 \left( \eta - \mathrm V_2^\top (\mathrm x_0 - \mathrm x_{\text{LN}}) \right) \|_2^2\\ &amp;amp;= \left\| \begin{bmatrix} \mathrm V_1 &amp;amp; \mathrm V_2\end{bmatrix} \begin{bmatrix} \mathrm V_1^\top ( \mathrm x_{\text{LN}} - \mathrm x_0 )\\ \eta - \mathrm V_2^\top (\mathrm x_0 - \mathrm x_{\text{LN}}) \end{bmatrix} \right\|_2^2\\ &amp;amp;= \left\| \begin{bmatrix} \mathrm V_1^\top ( \mathrm x_{\text{LN}} - \mathrm x_0 )\\ \eta - \mathrm V_2^\top (\mathrm x_0 - \mathrm x_{\text{LN}}) \end{bmatrix} \right\|_2^2\\ &amp;amp;= \| \mathrm V_1^\top ( \mathrm x_{\text{LN}} - \mathrm x_0 ) \|_2^2 + \| \eta - \mathrm V_2^\top (\mathrm x_0 - \mathrm x_{\text{LN}}) \|_2^2\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;which is minimized at&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\eta^* := \mathrm V_2^\top (\mathrm x_0 - \mathrm x_{\text{LN}}) = \mathrm V_2^\top \mathrm x_0 - \underbrace{\mathrm V_2^\top \mathrm V_1}_{= \mathrm O_{(n-r) \times r}} \Sigma_1^{-1} \mathrm U_1^{\top} \mathrm b = \mathrm V_2^\top \mathrm x_0$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and, thus, the solution of the quadratic program is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\boxed{\mathrm x^* := \mathrm x_{\text{LN}} + \mathrm V_2 \eta^* = \color{blue}{\mathrm V_1 \Sigma_1^{-1} \mathrm U_1^{\top} \mathrm b + \mathrm V_2 \mathrm V_2^\top \mathrm x_0}}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-07-05T21:27:36.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27298" PostTypeId="1" CreationDate="2017-07-02T21:52:54.153" Score="5" ViewCount="95" Body="&lt;p&gt;I plan on using a Quasi-Newton method (L-BFGS) to minimize a non-linear objective function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f: \mathbb{R}^n \rightarrow \mathbb{R}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient is kind of interesting: as the values of the first set of parameters approach infinity, the gradient of the objective function with respect to the remaining parameters will approach zero. In other words,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \lim_{x_i\to\infty}  \frac{\partial f}{\partial x_j} = 0$$&#xA;$$ i &amp;lt; j$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important concept: this relationship has no bearing on whether the objective function is being minimized or not. I fear the L-BFGS algorithm will follow a path where the gradient is minimized by this relationship instead of being minimized appropriately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I use this knowledge to aid in my optimization strategy?&lt;/p&gt;&#xA;" OwnerUserId="23647" LastActivityDate="2017-07-02T21:52:54.153" Title="Minimize interesting objective function with knowledge of gradient nonlinearity?" Tags="|optimization|quasi-newton|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27299" PostTypeId="1" CreationDate="2017-07-03T04:22:16.120" Score="2" ViewCount="625" Body="&lt;p&gt;The GMRES method is for solving the linear system $Ax=b$.&#xA;Given an initial guess $x_0$ and the corresponding residual $r_0:=b-Ax_0$, we have the Krylov subspace &#xA;$$\mathcal{K}_m:=\mathop{span}\{r_0,Ar_0,A^2r_0,\ldots,A^{m-1}r_0\}.$$&#xA;The GMRES method tries to find an approximate solution $x_m$ with the correction $d_m:=x_m-x_0$ from $\mathcal{K}_m$ such that $\|r_0-Ad_m\|_2$ is minimized. The GMRES method uses the Arnoldi process to get a decomposition $$AV_m=V_{m+1}H_m$$&#xA;with $V_{m+1}$ an orthonormal basis of $\mathcal{K}_{m+1}$ and $H_m$ an $(m+1)$-by-$m$ Hessenberg matrix. Let $d_m=V_my$ with $y\in\mathbb{R}^m$. Then, we have&#xA;$$\|r_0-Ad_m\|_2=\|V_{m+1}^Tr_0-H_my\|_2$$&#xA;which leads to a least square problem that is smaller and easier to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is that why not use directly the original basis $X_m:=[r_0,Ar_0,A^2r_0,\ldots,A^{m-1}r_0]$. Let $d_m=X_mz$ with $z\in\mathbb{R}^m$. Then, we do QR decomposition directly for $AX_m$ so that&#xA;$$AX_m=QR$$ with $R$ an upper triangular matrix.  Then, using this decomposition we can still minimize $\|r_0-Ad_m\|_2$ easily.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2017-07-03T04:54:15.367" LastActivityDate="2017-07-05T13:14:47.980" Title="Why do we need orthonormal basis of Krylov subspaces for GMRES?" Tags="|krylov-method|gmres|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27300" PostTypeId="1" CreationDate="2017-07-03T11:56:29.673" Score="1" ViewCount="82" Body="&lt;p&gt;How can i demonstrate that a LCG has full period? Is there a way to demonstrate it or do I just need to check the parameters(a, c, m)?&lt;/p&gt;&#xA;" OwnerUserId="24749" LastActivityDate="2017-07-03T11:56:29.673" Title="Random Number Generation-full period" Tags="|random-number-generation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27301" PostTypeId="1" CreationDate="2017-07-03T14:44:10.943" Score="0" ViewCount="240" Body="&lt;p&gt;I have to deal with FEM and the numerical solution of PDEs a lot. While I'm doing ok when just applying or implementing it, I observe a lack of understanding when authors begin to argue with &quot;Ritz&quot;, &quot;Galerkin&quot;, &quot;Weak Form&quot;, &quot;Integration by parts&quot; and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone recommend me a good book to get some basics in that field, specifically targeted to understand FEM and the numerical solution of PDEs.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-07-03T16:10:29.703" Title="Ritz, Galerkin, Weak Form, FEM: How to catch up the basics?" Tags="|finite-element|pde|weak-solution|" AnswerCount="1" CommentCount="0" ClosedDate="2017-07-03T17:05:06.100" ContentLicense="CC BY-SA 3.0" />
  <row Id="27302" PostTypeId="2" ParentId="27301" CreationDate="2017-07-03T16:10:29.703" Score="1" Body="&lt;p&gt;The book of Thomas J.R. Hughes : The finite Element Method, Linear static and dynamic finite element analysis, is probably the reference that you need.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could find in the first chapter the building of the discretized form of a simple one-dimensional boundary-value problem from its strong formulation. &lt;/p&gt;&#xA;" OwnerUserId="24755" LastActivityDate="2017-07-03T16:10:29.703" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27303" PostTypeId="1" AcceptedAnswerId="27332" CreationDate="2017-07-03T16:30:02.933" Score="0" ViewCount="549" Body="&lt;p&gt;My knowledge of finite difference is very basic so this could be very trivial. I've seen how multidimensional finite difference works for say fluid equations, but they are also dealing with a single variable. How would one solve a coupled multidimensional equation, for example of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a \nabla a = \nabla b,$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;in two dimensions? Meaning we have the system of equations &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a \, \partial_x a = \partial_x b,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a \partial_y a = \partial_y b.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a simplistic finite difference method I can solve each equation independently&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a(x+\Delta x, y) = \frac{b(x+\Delta x,y) - b(x,y)}{a(x,y)} + a(x,y),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a(x, y+\Delta y) = \frac{b(x, y+\Delta y) - b(x,y)}{a(x,y)} + a(x,y).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if I want to get to $a(x+\Delta x, y+\Delta y)$, there are two options either first moving along $x$ and then $y$ or vice versa. In this case I am not guaranteed to get the same result, meaning the answer is path dependent which is not desirable. What is the common practice to deal with this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I'll compute the values over a square grid with initial conditions $a(0,0) = a_0$ and $b(x,y)=x-y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My purpose is to solve this equation so that I may initialize the grid in $a$. This will be implemented in a program I am writing. The equation given is just a no-frills re-expression of what I'm attempting to do. My true equation is solving an isentropic hydrostatic atmosphere with a complicated potential, $\nabla \Phi$, for the density structure, $\rho$. The equation reduces to $$\gamma \, K \, \rho^{\gamma-2} \, \vec{\nabla} \rho = - \vec{\nabla} \Phi$$, where $K$ and $\gamma$ are some constants.&lt;/p&gt;&#xA;" OwnerUserId="18344" LastEditorUserId="18344" LastEditDate="2017-07-04T17:12:55.210" LastActivityDate="2017-07-06T07:42:16.463" Title="Finite difference methods for multidimensional coupled equations" Tags="|finite-difference|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27304" PostTypeId="1" CreationDate="2017-07-03T17:35:27.370" Score="1" ViewCount="40" Body="&lt;p&gt;Assumed I have the following coupled PDEs with the variables $A(z,t)$ and $B(z,t)$ and the constants $c_1,\,c_2$:&#xA;$$\begin{split}&#xA;\partial_tA&amp;amp;=\partial_zA+c_1\left(A+B\right)\\&#xA;\partial_tB&amp;amp;=\partial_zB+c_2\left(A+B\right)&#xA;\end{split}&#xA;$$&#xA;Now I have seen this algorithm for solving those equations:&lt;br&gt;&#xA;First, calculate the first equation using&#xA;$$\left(1-\frac{dt}{2}\partial_z\right)A_{n+1}=\left(1+\frac{dt}{2}\partial_z\right)A_n+dt\cdot c_1(A_n+B_n)$$&#xA;and then use the solution in the second equation as:&#xA;$$\left(1-\frac{dt}{2}\partial_z\right)B_{n+1}=\left(1+\frac{dt}{2}\partial_z\right)B_n+dt\cdot c_2(A_{n+1}+B_n)$$&#xA;I would have approached it differently by combining $A$ and $B$ into a vector $C$:&#xA;$$\begin{split}&#xA;C_{n+1}&amp;amp;=\begin{pmatrix}A_{n+1}\\B_{n+1}\end{pmatrix}\\&#xA;C_{n}&amp;amp;=\begin{pmatrix}A_{n}\\B_{n}\end{pmatrix}&#xA;\end{split}$$&#xA;and then expanding the corresponding matrices:&#xA;$$&#xA;\begin{pmatrix}&#xA;1-\frac{dt}{2}\partial_z-dt\frac{c_1}{2}&amp;amp;-dt\frac{c_1}{2}\\&#xA;-dt\frac{c_2}{2}&amp;amp;1-\frac{dt}{2}\partial_z-dt\frac{c_2}{2}&#xA;\end{pmatrix}C_{n+1}=\begin{pmatrix}&#xA;1+\frac{dt}{2}\partial_z+dt\frac{c_1}{2}&amp;amp;dt\frac{c_1}{2}\\&#xA;dt\frac{c_2}{2}&amp;amp;1+\frac{dt}{2}\partial_z+dt\frac{c_2}{2}&#xA;\end{pmatrix}C_n&#xA;$$&#xA;Is my approach &quot;more&quot; correct than the initial one? If yes, how can I estimate my error and compare it to the original approach?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-07-03T17:35:27.370" Title="Solving two coupled PDEs using two methods, how to calculate the error between them in theory?" Tags="|pde|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27305" PostTypeId="1" CreationDate="2017-07-03T19:18:05.763" Score="1" ViewCount="55" Body="&lt;p&gt;I'm using the Levenberg-Marquardt algorithm to fit my data with a Gaussian function:&#xA;$$&#xA;f(x)=a\cdot e^{-\frac{(x-c)^2}{2\sigma^2}}+f_0&#xA;$$&#xA;$a$, $c$, $\sigma$ and $f_0$ are the fitting parameters. The algorithm correctly computes $a$, $c$ and $\sigma$. Given that&#xA;$$&#xA;\frac{\partial f}{\partial f_0} = 1&#xA;$$&#xA;and a $\delta \vec{x}$ solution of the least-squared problem:&#xA;$$&#xA;\delta \vec{x} = (\vec{J}\vec{J}^T)^{-1} \cdot\vec{J}f&#xA;$$&#xA;I get that $\delta \vec{x}$ is not sensible to any change in $f_0$; so it says nothing on what to do to minimize the $\chi^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing? Do I have to switch to minimizing the $\chi^2$ function with the Brent algorithm (though it kinda spells trouble to me) to obtain the right known term or is there a more effective way to get there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or am I making it a lot harder than it is?&lt;/p&gt;&#xA;" OwnerUserId="24757" LastActivityDate="2017-07-03T19:18:05.763" Title="Fit constant term in LM algorithm" Tags="|numerical-modelling|curve-fitting|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27306" PostTypeId="2" ParentId="27303" CreationDate="2017-07-03T19:31:16.787" Score="1" Body="&lt;p&gt;Next time, provide please the complete problem, i.e. with boundary conditions. This simplify everything and allow us to provide you the best answer at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supposing your mesh has the same points in $x$ and $y$ axes, namely $n$, we have $2n^2$ unknowns for a discrete vector $\vec{w}$ in which, for the sake of simplicity, the unknowns are ordered by pairs $w_i = (a,b)_i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of the method is to find the solution of the following system:&#xA;$$M(\vec{w})\vec{w} = \vec{b}$$&#xA;Where $M$ is a matrix that depends in a nonlinear way on $\vec{w}$ (your PDE is nonlinear in $a$ and therefore $M=M(a)$), and $b$ is the vector in which the Boundary conditions would be applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First provide an easy ordering for your points through the mesh, i.e. begin to count from the lower left corner, from left to right and after the row is competed, keep going to the upper one, reading from left to right...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore the point indices: $$(1,1) = 1, (1,2) = 2,..., (n,1) = n, (2,1) = n+1, (2,2) = n+2, ...(n,n) = n^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To index $1$ corresponding to the point $(1,1)$ we would have a pair of values $(a_1,b_1)$ and so on...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notation for a variable $\phi$ would be considered here:&#xA;$$\phi(x,y) = \phi_{i,k}\qquad \phi(x+\Delta x,y) = \phi_{i+1,k} \qquad \phi(x,y-\Delta y) = \phi_{i,k-1} $$&#xA;Your discretised equations read in index notation:&#xA;$$a_{i,k}\left(a_{i+1,k}-a_{i,k}\right)-\left(b_{i+1,k}-b_{i,k}\right) = 0 \tag{1}$$&#xA;$$a_{i,k}\left(a_{i,k+1}-a_{i,k}\right)-\left(b_{i,k+1}-b_{i,k}\right) = 0 \tag{2}$$&#xA;You must arrange with care the matrix $M$ for the vector $\vec{w}$ given by:&#xA;$$\vec{w} = (a_1,b_1,a_2,b_2,...,a_n, b_n, ..., a_{n^2}, b_{n^2})^{T}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the $2n$th and $2n+1$th row of $M$ corresponding to $(1)$ and $(2)$ respectively you'll have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M(2n,2n) = -a_{n},\quad M(2n,2n+1) = 1, \quad M(2n,2n+2) = a_n,\quad M(2n,2n+3) = -1$$&#xA;$$M(2n+1,2n) = -a_{n},\quad M(2n+1,2n+1) = 1, \quad M(2n+1,4n+2) = a_n,\quad M(2n+1,4n+3) = -1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope I have not made any mistake in the indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, you will must provide an initial guess for $a(x,y)$ to evaluate the matrix $M$ at the first iteration, i.e., you will solve the following (simplest) iterative numerical scheme:&#xA;$$M(\vec{w}^{n})\vec{w}^{n+1}=\vec{b}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OF COURSE it is extremely important to set an appropiate $\vec{b}$ and modified $M$ entries in order to fulfill the Boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-07-03T19:38:32.390" LastActivityDate="2017-07-03T19:38:32.390" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27307" PostTypeId="2" ParentId="27152" CreationDate="2017-07-03T19:51:04.153" Score="1" Body="&lt;p&gt;I suppose that the space function in which $q$ is sought has a null mean, i.e. the new test functions $q^*$ have been defined such as:&#xA;$$\int_{\Omega}{q^*\,dx} = \int_{\Omega}{(q-\overline{q})\,dx} = 0$$&#xA;This is common in systems in which a compatibility condition arises must be fulfilled.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2017-07-03T19:51:04.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27309" PostTypeId="2" ParentId="23737" CreationDate="2017-07-03T22:25:39.547" Score="-1" Body="&lt;p&gt;An important special case of constrained optimization is constituted by optimization problems with simple bounds: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{x \in \mathbb R^n} \quad f(x) \quad \text{s.t.} \quad x \geq 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that $f$ is continuously differentiable. Show that the first order necessary conditions for this problem are equivalent to $\min \{x, \nabla f(x)\}=0$, where the minimum is component-wise?? Please answer it.&lt;/p&gt;&#xA;" OwnerUserId="24760" LastEditorUserId="20417" LastEditDate="2017-07-05T17:08:01.283" LastActivityDate="2017-07-05T17:08:01.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27311" PostTypeId="1" CreationDate="2017-07-04T18:45:55.230" Score="1" ViewCount="286" Body="&lt;p&gt;I have a data sample from an accelerometer from my phone (pretty bad accelerometer though). I'm trying to double integrate it in order to obtain the position as a function of time. I'm using a program in Python3 to integrate it, here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;nm = input(&quot;Filename: &quot;)&#xA;&#xA;fil1 = open(nm,&quot;r&quot;)&#xA;i = 0&#xA;s = 0.0&#xA;fil2 = open(&quot;int1.dat&quot;,&quot;w&quot;)&#xA;&#xA;for line in fil1:&#xA;    xi,yi = line.split()&#xA;    if i == 0:&#xA;        x = float(xi) ; y = float(yi)&#xA;    if i &amp;gt; 0:&#xA;        s = s  + (float(xi)-x)*(float(yi)+y)/2&#xA;        x = float(xi)&#xA;        y = float(yi)&#xA;    i = i + 1&#xA;    fil2.write(&quot;{} {} \n&quot;.format(xi,s))&#xA;fil2.close()&#xA;#I just integrated the file. Now, i will integrate it again:&#xA;fil3 = open(&quot;int1.dat&quot;,&quot;r&quot;)&#xA;s = 0.0&#xA;i = 0&#xA;fil4 = open(&quot;int2.dat&quot;,&quot;w&quot;)&#xA;for line in fil3:&#xA;    xi,yi = line.split()&#xA;    if i == 0:&#xA;        x = float(xi) ; y = float(yi)&#xA;    if i &amp;gt; 0:&#xA;        s = s  + (float(xi)-x)*(float(yi)+y)/2&#xA;        x = float(xi)&#xA;        y = float(yi)&#xA;    i = i + 1&#xA;    fil4.write(&quot;{} {} \n&quot;.format(xi,s))&#xA;fil3.close()&#xA;fil4.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see i'm approximating the area using trapezes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The acceleration data was obtained in a movement as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Started at a position that i defined as $x=0$.&lt;/li&gt;&#xA;&lt;li&gt;Moved in straight line towards the end of an table, and stopped.&lt;/li&gt;&#xA;&lt;li&gt;Waited a few seconds.&lt;/li&gt;&#xA;&lt;li&gt;Returned to the original position&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, the graph of position x time should me something similar to a &quot;hill&quot;: it goes up, than it goes down. It's value $x(t)$ at the peak would represent the total length of the table. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, things are not working quite well. I took out the noise by removing the mean value of all $|a(t)| &amp;lt; 0.15$. However, it still did not sum up well. The graphs get pretty bad. So, here is my first problem:&#xA;&lt;strong&gt;The noise stays too big when i integrate it.&lt;/strong&gt;&#xA;And so, my first question is: How can i reduce the noise?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anoter problem that is happening and that i'm finding odd is that my position doesn't &quot;go and come back&quot; to where it started. It starts in $0$, goes to $-7$ (which is approximately the length of the table) and then, instead of coming back to $0$, it goes all the way down to $-14$. &lt;strong&gt;Why is this?&lt;/strong&gt; I can't seem to find any particular reason for the integral not reckognizing the fact that i am coming &lt;em&gt;backwards&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to upload the graphics, but i couldn't right now. I will upload them as soon as i can. Much appreciate.&lt;/p&gt;&#xA;" OwnerUserId="24769" LastEditorUserId="24680" LastEditDate="2017-07-05T12:08:59.033" LastActivityDate="2017-07-05T12:08:59.033" Title="Double Integrating acceleration data to obtain position: 2 Problems" Tags="|numerical-analysis|computational-physics|time-integration|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27312" PostTypeId="1" CreationDate="2017-07-04T20:14:59.327" Score="1" ViewCount="314" Body="&lt;p&gt;Suppose we have the negative, inhomogeneous advection equation:&#xA;$$\left(\frac{\partial}{\partial x}-\frac{1}{c}\frac{\partial}{\partial t}\right)v(t,x)=u(t,x)\qquad(t\in\mathbb{R}_{+},x\in\mathbb{R})$$&#xA;I want to solve it numerically, &lt;strong&gt;backward in time and forward in space&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I have tried a few schemes without avail. For example, the Crank-Nicolson scheme going backwards in time:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, we may discretise as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{c}\frac{\mathrm{d}}{\mathrm{d}t}v_\ell(t)=\underbrace{\frac{v_{\ell+1}(t)-v_\ell(t)}{\Delta x}-u_\ell(t)}_{G_\ell(t)}+\mathcal{O}(\Delta x)\qquad\Delta x\to 0$$&#xA;where we have taken the forward difference of the spatial derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then applying the Crank-Nicolson scheme yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{c}\frac{v^{n+1}_\ell-v^n_\ell}{\Delta t}=\frac{1}{2}[G_\ell(t^{n+1})+G_\ell(t^n)]=\frac{1}{2}\left[\frac{v^{n+1}_{\ell+1}-v^{n+1}_\ell+v^n_{\ell+1}-v^n_\ell}{\Delta x}-u^{n+1}_\ell-u^n_\ell\right]$$&#xA;Hence&#xA;$$v^{n+1}_\ell-\frac{c\Delta t}{2\Delta x}(v^{n+1}_{\ell+1}-v^{n+1}_\ell)=v^n_\ell+\frac{c\Delta t}{2}\left[\frac{v^n_{\ell+1}-v^n_\ell}{\Delta x}-u^{n+1}_\ell+u^n_\ell\right]$$&#xA;i.e.&#xA;$$\left(\mathrm{I}-\frac{c\Delta t}{2\Delta x}\mathrm{A}\right)v^{n+1}_\ell=\left(\mathrm{I}+\frac{c\Delta t}{2\Delta x}\mathrm{A}\right)v^n_\ell-\frac{c\Delta t}{2\Delta x}(u^{n+1}_\ell+u^n_\ell)$$&#xA;where&#xA;$$\mathrm{A}=\begin{pmatrix}&#xA;-1 &amp;amp; 1 &amp;amp; &amp;amp; 0&#xA;\\&#xA;&amp;amp;\ddots &amp;amp; \ddots&#xA;\\&#xA;&amp;amp; &amp;amp; -1 &amp;amp; 1&#xA;\\&#xA;0 &amp;amp; &amp;amp; &amp;amp; -1&#xA;\end{pmatrix}$$&#xA;Now, since we want to go backwards in time, we replace $\Delta t$ with $-\Delta t$ and we get&#xA;$$v^{n-1}_\ell=\left(\mathrm{I}+\frac{c\Delta t}{2\Delta x}\mathrm{A}\right)^{-1}\left[\left(\mathrm{I}-\frac{c\Delta t}{2\Delta x}\mathrm{A}\right)v^n_\ell+\frac{c\Delta t}{2\Delta x}(u^{n-1}_\ell+u^n_\ell)\right]$$&#xA;Now, I try to implement this on Matlab via&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Parameters&#xA;L = 5; % size of domain &#xA;T = 5; % measurement time&#xA;dx = 1e-2; % spatial step&#xA;dt = 1e-3; % time step&#xA;x0 = 0; % point of measurement&#xA;c = 1; % speed of advection&#xA;%%&#xA;t = 0:dt:T; % time vector&#xA;x = [0:dx:L]'; % position vector&#xA;nt = length(t); % number of time steps&#xA;nx = length(x); % number of position steps&#xA;mu = dt/dx;&#xA;I = eye(nx,nx+1); % identity matrix&#xA;A = spdiags(ones(nx,1)*[-1 1],0:1,nx,nx);&#xA;%% Solve Backward Advection Equation&#xA;% preallocate the memory&#xA;v = zeros(nx,nt);&#xA;u = zeros(nx,nt);&#xA;% final condition, sinc function centered around x = .5&#xA;v(:,nt) = sinc((x-x0)/dx);&#xA;for k = nt-1:1&#xA;    v(:,k-1) = (I+(c/2)*mu*A)\((I-(c/2)*mu*A)*v(:,k)+(c/2)*mu*(u(:,k-1)+u(:,k)));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My solution simply &quot;blows up&quot; on the boundary of the domain and is zero everywhere else. I have similar results using backwards Euler instead of Crank-Nicolson. Where am I going wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I provide two plots:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0Va4u.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Va4u.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/a9VV8.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a9VV8.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24427" LastEditorUserId="24427" LastEditDate="2017-07-04T20:21:44.533" LastActivityDate="2017-07-04T20:21:44.533" Title="Why can I not solve the negative advection equation (backwards in time)?" Tags="|matlab|pde|hyperbolic-pde|advection|crank-nicolson|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27313" PostTypeId="2" ParentId="27267" CreationDate="2017-07-04T20:46:49.907" Score="2" Body="&lt;p&gt;I don't know what you can do to keep track of parameters and results, though @origimbo correctly noted that repeatability kinda makes results useless to store.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first point you make is obviously the versioning problem, so the solution is trivial: create a repository on Github or Bitbucket or Mercurial.&#xA;These services are very effective and they let you reverse a given commit if you messed up your code and you need to restore it. They are supported as plugin within most IDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plus: redundancy on cloud! Making backup copies will be a bit less urgent than it usually is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; implement a logger into your programs.&lt;/p&gt;&#xA;" OwnerUserId="24757" LastEditorUserId="24757" LastEditDate="2017-07-05T14:06:30.523" LastActivityDate="2017-07-05T14:06:30.523" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="27314" PostTypeId="1" AcceptedAnswerId="27318" CreationDate="2017-07-05T02:38:33.010" Score="4" ViewCount="4440" Body="&lt;p&gt;I'm interested in finding the roots of the following equation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\tan(x) = \frac{2x}{x^2-1}$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is easily seen that 0 is a root and the roots are symmetric w.r.t. 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if an analytical solution is known or how can I find, say, 1000 roots numerically with increasing magnitudes ? I've tried Matlab built-in functions but failed.&lt;/p&gt;&#xA;" OwnerUserId="16490" LastEditorUserId="16490" LastEditDate="2017-07-05T05:33:58.480" LastActivityDate="2017-07-05T09:41:41.640" Title="How to solve the transcendental equation: $\tan(x) = \frac{2x}{x^2-1}$" Tags="|software|numerics|rootfinding|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27315" PostTypeId="2" ParentId="27314" CreationDate="2017-07-05T03:58:27.477" Score="2" Body="&lt;p&gt;You can at least take a quick look at the zeros using &lt;code&gt;gnuplot&lt;/code&gt; by plotting contours at zero of the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$0 = \frac{2x}{x^2-1} - \tan(x)$$&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-gnuplot prettyprint-override&quot;&gt;&lt;code&gt;set terminal png&#xA;set output &quot;test.png&quot;&#xA;&#xA;set xlabel &quot;x&quot;&#xA;set ylabel &quot;y&quot;&#xA;&#xA;set contour&#xA;set cntrparam levels discrete 0&#xA;set view map&#xA;unset surface&#xA;set isosamples 1000,1000&#xA;splot 2*x/(x**2-1) - tan(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since the equation is totally independent of $y$ the solutions are lines parallel to the $y$-axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dTCz5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dTCz5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve the equation numerically you could use GNU R with the &lt;a href=&quot;https://cran.r-project.org/package=rootSolve&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;rootSolve&lt;/code&gt;&lt;/a&gt; package.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;require(&quot;rootSolve&quot;)&#xA;&#xA;f &amp;lt;- function(x) 2*x/(x^2-1) - tan(x)&#xA;r &amp;lt;- uniroot.all(f, c(-10,10))&#xA;print(r)&#xA;curve(f,-10,10,ylim=c(-5,5),n=1000)&#xA;points(r,rep(0,length(r)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt; [1]  0.0000000 -9.6316827 -7.8540438 -6.5846170 -4.7124512 -3.6731884&#xA; [7] -1.5708444 -1.3065367 -0.9999023  0.9999023  1.3065367  1.5708444&#xA;[13]  3.6731884  4.7124512  6.5846170  7.8540438  9.6316827&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y6d1J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y6d1J.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24680" LastEditorUserId="24680" LastEditDate="2017-07-05T04:20:29.773" LastActivityDate="2017-07-05T04:20:29.773" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27317" PostTypeId="1" AcceptedAnswerId="27322" CreationDate="2017-07-05T04:10:50.823" Score="6" ViewCount="1247" Body="&lt;p&gt;My question is quite simple, but the more I look at it, the less content I am.  My question is how to do a RK4 method for $y'=y$.  At first I would assume the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$k_1=y_n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$k_2=y_n+\frac{h}{2}k_1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and right here you can actually start to see part of the issue that I have, namely that it seems in the second equation we are adding apples and oranges since $h$ is not unitless.  Nonetheless, in continuing onward:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$k_3=y_n+\frac{h}{2}k_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$k_4=y_n+h\ k_3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and now put all together,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_{n+1}=y_n+\frac{h}{6}(k_1+2k_2+2k_3+k_4)$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for all of these it seems as though we are adding apples and oranges (my case is exactly this pretty much, finding position by taking the derivative of the velocity).  Furthermore, in this last equation to solve for the next step, $y'$ does not appear in it at all, which to me seems really weird (not right at all).  To clarify why I am confused on this point, is because normally the first step is an Euler Step, in which case the whole thing put together would look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y_{n+1}=y_n+h\ y'_{n}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of the reason I bring this up is because I am given $y'$ and need to now map it's derivative (just like the above Euler Method).  I have used RK4 many times on more complicated equations, but for some reason this one just doesn't seem quite right.  Thanks!&lt;/p&gt;&#xA;" OwnerUserId="24134" LastEditorUserId="24134" LastEditDate="2017-07-06T06:13:47.603" LastActivityDate="2017-07-06T06:13:47.603" Title="4th order Runge-Kutta for $y' = y$" Tags="|ode|runge-kutta|differential-equations|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27318" PostTypeId="2" ParentId="27314" CreationDate="2017-07-05T04:36:59.760" Score="5" Body="&lt;p&gt;Here is a simple (Matlab) Newton method as a first attempt to help get started.  It finds 1087 roots with error below $10^{-11}$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f  = @(x) ((2*x)./(x.^2-1)) - tan(x);&#xA;fp = @(x)-tan(x).^2+2.0./(x.^2-1.0)-x.^2.*1.0./(x.^2-1.0).^2.*4.0-1.0;&#xA;&#xA;x0 = 0;&#xA;&#xA;for jj = 1 : 1200 %number of iterations to find some roots&#xA;    x0 = x0 + (jj-1)*(jj/10^4);  %take previous guess and increment&#xA;&#xA;    for newton = 1 : 20 %number of newton steps&#xA;       x0 = x0 - f(x0)/fp(x0); &#xA;    end&#xA;    ROOTS(jj,1)  = x0; &#xA;    ERROR(jj,1)  = abs(f(x0));  %check that we really found a root&#xA;&#xA;end&#xA;&#xA;ROOTS = unique(ROOTS);  %roots may be duplicate&#xA;ERROR = unique(ERROR);  %&#xA;&#xA;length(ROOTS) %number of roots we actually found&#xA;&#xA;ROOTS(1:10)&#xA;ERROR(1:10)&#xA;&#xA;plot(abs(ERROR))&#xA;xlabel('root number')&#xA;ylabel('absolute error')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tcItb.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tcItb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some sample roots:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Root number    |    Root value &#xA;    300            2340.487381447216&#xA;    301            2356.195339018351&#xA;    302            2371.903296664941&#xA;    303            2387.611254385495&#xA;    304            2406.460803745755&#xA;    305            2425.310353208004&#xA;    306            2444.159902769882&#xA;    307            2463.009452429101&#xA;    308            2481.859002183444&#xA;    309            2500.708552030761&#xA;    310            2519.558101968962&#xA;    311            2538.407651996025&#xA;    312            2557.257202109985&#xA;    313            2576.106752308934&#xA;    314            2594.956302591019&#xA;    315            2613.805852954442&#xA;    316            2632.655403397457&#xA;    317            2651.504953918365&#xA;    318            2670.354504515517&#xA;    319            2689.204055187311&#xA;    320            2708.053605932186&#xA;    321            2726.903156748628&#xA;    322            2745.752707635163&#xA;    323            2764.602258590357&#xA;    324            2783.451809612815&#xA;    325            2802.301360701180&#xA;    326            2821.150911854131&#xA;    327            2840.000463070381&#xA;    328            2858.850014348680&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another option to try numerically in Matlab is the &lt;a href=&quot;http://www.chebfun.org/docs/guide/guide03.html&quot; rel=&quot;noreferrer&quot;&gt;Chebfun&lt;/a&gt; package.  It is based off the usage of orthogonal polynomials and the root finding is done by obtaining eigenvalues of the corresponding companion matrix.&lt;/p&gt;&#xA;" OwnerUserId="11604" LastActivityDate="2017-07-05T04:36:59.760" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27319" PostTypeId="2" ParentId="27317" CreationDate="2017-07-05T04:46:27.163" Score="2" Body="&lt;p&gt;You have to keep in mind that you are integrating something.  An integral usually looks like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(x) = \int f(x) \mathop{}\!dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $dx$ is also not unitless, so in principle you are multiplying an apple ($f$) by something ($dx$) to obtain an orange ($F$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue that $y'$ does not show up is unclear.  Perhaps it becomes clear when I restate the Runge-Kutta scheme here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given $y' = f(t,y)$ with initial value $y(t_0) = y_0$ you can calculate the discrete time series $y_n = y(t_n)$  with&#xA;$$ y_{n+1} = y_n + \frac{h}{6} (k_1 + 2 k_2 + 2 k_3 + k4) \;,\quad t_{n+1} = t_n + h$$&#xA;where&#xA;$$\begin{aligned}&#xA;  k_1 &amp;amp;= f(t,y_n) \\&#xA;  k_2 &amp;amp;= f(t+h/2 ,y_n + h k_1/2) \\&#xA;  k_3 &amp;amp;= f(t+h/2 ,y_n + h k_2/2) \\&#xA;  k_4 &amp;amp;= f(t+h ,y_n + h k_3) \\&#xA;\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here a quick RK4 implementation in Julia to show that there is nothing wrong.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-julia prettyprint-override&quot;&gt;&lt;code&gt;using Plots&#xA;&#xA;# Function definition y' = y&#xA;f(t,y) = y&#xA;&#xA;# Boundary condition y(0) = 1&#xA;y = 1&#xA;&#xA;# Time step&#xA;dt = 0.01&#xA;&#xA;# Arrays for time an solution&#xA;time = collect(0:dt:1)&#xA;sol = similar(time)&#xA;&#xA;# integration loop&#xA;for (n,t) in enumerate(time)&#xA;    sol[n] = y&#xA;    k1 = f(t     ,y        )&#xA;    k2 = f(t+dt/2,y+dt/2*k1)&#xA;    k3 = f(t+dt/2,y+dt/2*k2)&#xA;    k4 = f(t+dt  ,y+dt  *k3)&#xA;    y  = y + dt/6*(k1 + 2*k2 + 2*k3 + k4)&#xA;end&#xA;&#xA;plot(time,sol)&#xA;plot!(time,exp(time))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K25rK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K25rK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24680" LastEditorUserId="24680" LastEditDate="2017-07-05T04:52:58.930" LastActivityDate="2017-07-05T04:52:58.930" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27320" PostTypeId="2" ParentId="27317" CreationDate="2017-07-05T06:00:12.123" Score="2" Body="&lt;p&gt;This is a tricky question because mathematicians usually ignore units. I actually ran into this myself when making DifferentialEquations.jl compatible with units. &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqTutorials.jl/blob/master/ExtraODEFeatures/Unit%20Checked%20Arithmetic%20via%20Unitful.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this notebook&lt;/a&gt;, I use Unitful.jl's unitful-arithmetic. Essentially, you just use Julia's string macros to make &lt;code&gt;u&quot;s&quot;&lt;/code&gt; put units in seconds and implicit multiplication to make &lt;code&gt;2u&quot;s&quot;&lt;/code&gt; mean &quot;two seconds. Knowing this, notice that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using DifferentialEquations&#xA;f = (t,y) -&amp;gt; 0.5*y&#xA;u0 = 1.5u&quot;N&quot;&#xA;prob = ODEProblem(f,u0,(0.0u&quot;s&quot;,1.0u&quot;s&quot;))&#xA;sol = solve(prob,Tsit5())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;that throws an error, and is very close to your equation. Why? The error is very clear:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DimensionError: 1.5 N and 0.015 N s are not dimensionally compatible.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is exactly what you're seeing. It calculates $hk_1$ as having units of $Ns$ to generate the intermediate estimated value, but this value needs to have units of $N$. Is it wrong? No, I was wrong! Remember that $f(t,u) = y\prime = \frac{dy}{dt}$. It should be a rate, so it needs to be $y$ per second. When you do that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = (t,y) -&amp;gt; 0.5*y/3.0u&quot;s&quot;&#xA;prob = ODEProblem(f,u0,(0.0u&quot;s&quot;,1.0u&quot;s&quot;))&#xA;sol = solve(prob,Tsit5()) &#xA;&#xA;# the printout&#xA;&#xA;retcode: Success&#xA;Interpolation: specialized 4th order &quot;free&quot; interpolation&#xA;t: 3-element Array{Quantity{Float64, Dimensions:{}, Units:{s}},1}:&#xA;      0.0 s&#xA; 0.143116 s&#xA;      1.0 s&#xA;u: 3-element Array{Quantity{Float64, Dimensions:{  ^-2}, Units:{N}},1}:&#xA;     1.5 N&#xA; 1.53621 N&#xA; 1.77204 N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you see that it works. So yes, $k$ is a rate, so it needs to be in units per time, and then when you multiply by $h$, you now get a value in units again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's actually going on? $hk_1$ is just an estimate of $y(t+h)$ (notice the units line up). That step is just Euler. You then use $y(t)$ and the estimate of $y(t+h)$ to make an estimate of $y(t+h/2)$ (some kind of midpoint). You then use the current $y(t+h/2)$ to correct the estimate at $y(t+h/2)$ (a kind of Picard iteration). You then use the corrected $y(t+h/2)$ estimate to take a half step and get a $y(t+h)$ estimate. Then you average this $y(t+h)$ estimate with the first one (remember $hk_1$?), and double the changes to the midpoint to get two more estimates for the final step, and take a weighted average of all of these together. That is RK4. And if you check the units on this, they all line up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So $y\prime = y$ is bad shorthand if you care about units.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-07-05T06:11:08.040" LastActivityDate="2017-07-05T06:11:08.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27322" PostTypeId="2" ParentId="27317" CreationDate="2017-07-05T09:24:52.587" Score="9" Body="&lt;p&gt;The answer is quite simple. You are already comparing apples and oranges in the first equation. Garbage in, garbage out.&#xA;The equation $y'=y$ if written properly is $$dy/dx=y.$$ Do you see it now? To correct it, simply write: $dy/dx=ay,$ where $a$ is a constant and in our example, $a=1$ in units of $1/x$.&lt;/p&gt;&#xA;" OwnerUserId="24772" LastActivityDate="2017-07-05T09:24:52.587" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27323" PostTypeId="2" ParentId="27314" CreationDate="2017-07-05T09:27:45.967" Score="3" Body="&lt;p&gt;As this is a transcendent equation, finding all roots is not an option. You cannot (in general) find an expression that gives a closed-form for the roots of the equation. So you need to do some analysis first. Plotting the function, or plotting the left-hand-side and right-hand-side of the equation (and hence, the intersections of both curves are the roots) will give you some hints on how to tackle the problem. I define the LHS as $\tan(x)$ and the RHS as $\frac{2x}{x^{2}-1}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fvJRn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fvJRn.png&quot; alt=&quot;Plot of LHS and RHS&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you can observe is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Since both LHS and RHS are (anti-)symmetric around $x=0$, you only need to consider the positive $x$ domain. &lt;/li&gt;&#xA;&lt;li&gt;There is no root in the interval $[0,1[$ ($x=1$ being the location of the pole of the RHS function).&lt;/li&gt;&#xA;&lt;li&gt;There is a root in the interval $[1,\frac{\pi}{2}]$.&lt;/li&gt;&#xA;&lt;li&gt;Since the function of the RHS is monotonically decreasing for $x &amp;gt; 1$, there will be a single root in each interval $[(2k-1)\frac{\pi}{2},(2k+1)\frac{\pi}{2}]$ for $k=1,2,3, \ldots$.&lt;/li&gt;&#xA;&lt;li&gt;You can see that the RHS function goes to zero for $x\to\infty$. This implies that for larger $x$, the solutions will convergence to the zero crossings of the tangent function (i.e. $r_{l} \approx (l+1)\pi$, where $r_{l}$ is the $l$'-th root. You could use these estimates as initial guess for an iterative method.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So what will be the strategy to find (not all, but an arbitrary number) of roots? You use an iterative solving method in each interval. Now you have the choice between a plethora of methods but two methods seem very appropriate:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Since you have a very good estimate for the root (a good initial guess), you could use Newton-Raphson. But then you need to calculate the derivative (in this case, you could do it analytically). &lt;/li&gt;&#xA;&lt;li&gt;Since each root is known to be in an interval, you could use the derivative-free method of Dekker-Brent. Just make sure that you don't use the strict boundaries for each interval, because they are poles of the tangent function and Dekker-Brent will not know how to handle that. Add some margin to the initial interval. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using the Python code below, I can find the first $N$ roots using this approach. A plot for the interval [200,300] is given as an example. Since I'm lazy and didn't want to calculate the derivative, I chose the Dekker-Brent option (where I asked for a relative tolerance of $1e-14$. For what it's worth, the first 1000 roots took me 10 ms on an Intel i5 laptop. Of course, you can validate the results by calculating the residues (the function value in each calculated root). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5ysb4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5ysb4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.optimize import brentq&#xA;&#xA;plt.figure(figsize=(12,8))&#xA;plt.style.use('bmh')&#xA;x = np.linspace(0,20,1002)&#xA;plt.ylim(-5,5)&#xA;plt.plot(x,np.tan(x),label='$tan(x)$')&#xA;plt.plot(x,2*x/(x*x-1),label='$2x/(x^2-1)$')&#xA;plt.legend(loc=1)&#xA;plt.savefig('fun.png',dpi=300,bbox_inches = 'tight')&#xA;&#xA;def f(x):&#xA;    # The function&#xA;    return np.tan(x)-2*x/(x*x-1)&#xA;&#xA;&#xA;def roots(N):&#xA;    # Find N roots of the equation&#xA;&#xA;    #Allocate space&#xA;    roots = np.zeros(N)&#xA;&#xA;    #Margin to stay away from poles&#xA;    margin = 1e-8&#xA;&#xA;    #First root&#xA;    roots[0] = brentq(f, 1.0 + margin, np.pi/2 - margin, rtol=1e-14)&#xA;&#xA;    #Subsequent N-1 roots&#xA;    for i in range(1,N):&#xA;        left = (2*i - 1)*np.pi/2&#xA;        right = (2*i + 1)*np.pi/2&#xA;        roots[i] = brentq(f, left + margin, right - margin, rtol = 1e-14)&#xA;&#xA;    return roots &#xA;&#xA;result = roots(1000)&#xA;&#xA;left = 200&#xA;right = 300&#xA;&#xA;plt.figure(figsize=(24,8))&#xA;plt.style.use('bmh')&#xA;x = np.linspace(left,right,(right-left)*101)&#xA;plt.ylim(0,2e-2)&#xA;plt.plot(x,np.tan(x),label='$tan(x)$')&#xA;plt.plot(x,2*x/(x*x-1),label='$2x/(x^2-1)$')&#xA;rts = result[result&amp;gt;=left]&#xA;rts = rts[rts&amp;lt;=right]&#xA;plt.plot(rts,np.tan(rts),'o', markersize=14, fillstyle='none', label='roots')&#xA;plt.legend(loc=1)&#xA;plt.savefig('roots1.png',dpi=300,bbox_inches = 'tight')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2017-07-05T09:41:41.640" LastActivityDate="2017-07-05T09:41:41.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27324" PostTypeId="1" CreationDate="2017-07-05T10:13:38.863" Score="0" ViewCount="60" Body="&lt;p&gt;It seems that the 5th order is the corrector and the 4th order is the predictor. Can RK5 be considered as part of Predictor–corrector method?&lt;/p&gt;&#xA;" OwnerUserId="20760" LastActivityDate="2017-07-05T14:24:31.620" Title="Can RK5 be considered as part of Predictor–corrector method?" Tags="|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27325" PostTypeId="1" AcceptedAnswerId="27331" CreationDate="2017-07-05T11:41:23.440" Score="2" ViewCount="764" Body="&lt;p&gt;Assumed I have the following two coupled equations&#xA;$$\begin{split}&#xA;\partial_tA&amp;amp;=a_0AB\\&#xA;\partial_tB&amp;amp;=b_0AB&#xA;\end{split}&#xA;$$&#xA;but I am not sure how to calculate them. One approach is a crank-nicolson approach:&#xA;$$\begin{split}&#xA;\left(1-\frac{dt\cdot a_0B_0}{2}\right)A_1&amp;amp;=\left(1+\frac{dt\cdot a_0B_0}{2}\right)A_0\\&#xA;\left(1-\frac{dt\cdot b_0A_0}{2}\right)B_1&amp;amp;=\left(1+\frac{dt\cdot b_0A_0}{2}\right)B_0&#xA;\end{split}&#xA;$$ or a modified crank-nicolson approach, using the result already obtained:&#xA;$$\begin{split}&#xA;\left(1-\frac{dt\cdot a_0B_0}{2}\right)A_1&amp;amp;=\left(1+\frac{dt\cdot a_0B_0}{2}\right)A_0\\&#xA;\left(1-\frac{dt\cdot b_0A_1}{2}\right)B_1&amp;amp;=\left(1+\frac{dt\cdot b_0A_1}{2}\right)B_0&#xA;\end{split}&#xA;$$&#xA;Finally, I tested the usual forward euler method:&#xA;$$\begin{split}&#xA;A_1&amp;amp;=A_0+dt\cdot a_0AB\\&#xA;B_1&amp;amp;=B_0+dt\cdot b_0AB&#xA;\end{split}&#xA;$$&#xA;When choosing $a_0=1$, $b_0=2$ and $A_0=B_0=1$, all three methods result in different results, according to my matlab script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;&#xA;x_max = 1;&#xA;x_num = 1000;&#xA;a0_val = 1;&#xA;b0_val = 2;&#xA;dx = x_max/20000;&#xA;steps = 1/dx;&#xA;&#xA;%euler equations&#xA;A_equation = @(A_vec, B_vec) a0_val*A_vec.*B_vec;&#xA;B_equation = @(A_vec, B_vec) b0_val*A_vec.*B_vec;&#xA;&#xA;A_matrix_A = @(B_vec) speye(x_num)-speye(x_num).*a0_val*dx/2.*B_vec;&#xA;B_matrix_A = @(B_vec) speye(x_num)+speye(x_num).*a0_val*dx/2.*B_vec;&#xA;&#xA;A_matrix_B = @(A_vec) speye(x_num)-speye(x_num).*b0_val*dx/2.*A_vec;&#xA;B_matrix_B = @(A_vec) speye(x_num)+speye(x_num).*b0_val*dx/2.*A_vec;&#xA;&#xA;A_matrix = zeros(steps, x_num);&#xA;B_matrix = zeros(steps, x_num);&#xA;A_matrix_old = zeros(steps, x_num);&#xA;B_matrix_old = zeros(steps, x_num);&#xA;A_matrix_euler = zeros(steps, x_num);&#xA;B_matrix_euler = zeros(steps, x_num);&#xA;&#xA;B0_vec = ones(x_num, 1);&#xA;A0_vec = ones(x_num, 1);&#xA;A_matrix(1,:) = A0_vec;&#xA;B_matrix(1,:) = B0_vec;&#xA;A_matrix_old(1,:) = A0_vec;&#xA;B_matrix_old(1,:) = B0_vec;&#xA;A_matrix_euler(1,:) = A0_vec;&#xA;B_matrix_euler(1,:) = B0_vec;&#xA;&#xA;%Do steps&#xA;for i = 2:steps&#xA;    B_matrix(i,:) = A_matrix_B(A_matrix(i-1,:))\(B_matrix_A(A_matrix(i-1,:))*B_matrix(i-1,:).'+A_matrix(i-1,:).');&#xA;    A_matrix(i,:) = A_matrix_A(B_matrix(i,:))\(B_matrix_A(B_matrix(i,:))*A_matrix(i-1,:).'+B_matrix(i,:).');&#xA;end&#xA;&#xA;for i = 2:steps&#xA;    B_matrix_old(i,:) = A_matrix_B(A_matrix_old(i-1,:))\(B_matrix_A(A_matrix_old(i-1,:))*B_matrix_old(i-1,:).'+A_matrix_old(i-1,:).');&#xA;    A_matrix_old(i,:) = A_matrix_A(B_matrix_old(i-1,:))\(B_matrix_A(B_matrix_old(i-1,:))*A_matrix_old(i-1,:).'+B_matrix_old(i-1,:).');&#xA;end&#xA;&#xA;for i = 2:steps&#xA;    A_matrix_euler(i,:) = A_matrix_euler(i-1,:)+dx*A_equation(A_matrix_euler(i-1,:), B_matrix_euler(i-1,:));&#xA;    B_matrix_euler(i,:) = B_matrix_euler(i-1,:)+dx*B_equation(A_matrix_euler(i-1,:), B_matrix_euler(i-1,:));&#xA;end&#xA;&#xA;figure;&#xA;imagesc(B_matrix);&#xA;figure;&#xA;imagesc(B_matrix_old);&#xA;figure;&#xA;imagesc(B_matrix_euler);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thus I was wondering which method is correct in this situation?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-07-07T11:13:25.310" Title="Crank-Nicolson algorithm for coupled PDEs" Tags="|matlab|pde|crank-nicolson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27326" PostTypeId="2" ParentId="27299" CreationDate="2017-07-05T13:14:47.980" Score="2" Body="&lt;p&gt;Since nobody else has answered this question (which is a good question, by the way!) so far, I'm consolidating my comments here. Briefly, GMRES boils down to three key ideas:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Construct a sequence of approximations by solving a projected least squares problem on a nested sequence of subspaces $\mathcal{K}_m$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Choose the nested subspaces specifically as the Krylov spaces $\mathcal{K}_m=\mathrm{span} \{r^0,Ar^0,\dots,A^{m-1}r^0\}$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Solve the least squares problem efficiently using transformation to Hessenberg form and QR decomposition, exploiting the fact that the subspaces are nested and you only need to update a single column of the decomposition in each iteration. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Assuming you keep the general structure (in particular, the transformation to Hessenberg form and the column-wise update of $AX_m$ and its QR factorization), the only difference between GMRES and your approach is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In your approach, you&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;build the matrix $$AX_m = A[r^0, Ar^0,\dots,A^{m-1}r^0] = [Ar^0, A^2r^0,\dots,A^{m}r^0],$$&lt;/li&gt;&#xA;&lt;li&gt;transform it to Hessenberg form via Householder transformations, and&lt;/li&gt;&#xA;&lt;li&gt;compute (or update) its QR decomposition with Givens rotations.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that the QR decomposition implicitly constructs (as the columns of $Q$) an orthonormal basis of the space spanned by the columns of $AX_m$.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In comparison, GMRES&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;builds (through the Arnoldi process) an orthonormal basis of $X_m$ and a transformation (via the same basis!) of $AX_m$ to upper Hessenberg form, and&lt;/li&gt;&#xA;&lt;li&gt;computes (or updates) its QR decomposition with Givens rotations.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So the only difference is that GMRES avoids the formation of the matrix $AX_m$ and instead directly proceeds to its Hessenberg form. In effect, you are immediately orthogonalizing every new Krylov vector as soon as it is formed rather than waiting until the end. This is relevant because as in the power iteration, the vectors $A^mr^0$ converge to the dominant eigenvector of $A$, so that the matrix $AX_m$ becomes increasingly ill-conditioned. If you wish, you can say that GMRES is the modified Gram-Schmidt (which is more stable) to your standard Gram-Schmidt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;&#xA;(And if you now say, &quot;well, but I can do the same: orthogonalize $A^mr^0$ before adding it as a column to $AX_m$&quot;: congratulations, you've just reinvented GMRES.)&#xA;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2017-07-05T13:14:47.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27328" PostTypeId="2" ParentId="27324" CreationDate="2017-07-05T14:24:31.620" Score="1" Body="&lt;p&gt;No. The 4th order is not a predictor for the 5th order. Instead, what is done is that the $k_i$ are constructed in such a way that one linear combination $b_i k_i$ gives a 5th order method and $\hat{b}_i k_i $ gives a 4th order method. They are calculated independently, but constructed in a way such that the exact same intermediate steps ($k_i$) build both methods, saving the computation of actually computing two RK methods at the same time. This is called method embedding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Predictor-corrector is when one method is used as the input to the second (usually implicit method). That requires non-trivial calculation to compute the second.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-07-05T14:24:31.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27329" PostTypeId="1" CreationDate="2017-07-05T14:36:17.767" Score="1" ViewCount="43" Body="&lt;p&gt;Consider 2 particles in a 2D plane. There is a long-range interaction between two particles:&#xA;$$f (r_i-r_j)= \frac{\vec{r_i} - \vec{r_j}}{|r_i-r_j|^3}$$&#xA;$$\vec{r_i} =(x_i,y_i)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_i$ determines the position of particle number $i$ in the plane. Knowing position of particles in the 2D plane and also knowing that distances of particles is more than a threshold $r_0$ (so $f$ do not get infinite), how can one find $f_{tot}$ in the position of all particles in a periodic boundary condition simulation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_{tot} $ is sum of effect of images of $j$ particle in f:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{tot}(\vec{r_i})= \sum_{n=-\infty}^{\infty} \bigg ( \frac{(x_i - x_j + n L,0)}{|y_i - y_j + n L|^3} \bigg) +  \sum_{n=-\infty}^{\infty} \bigg ( \frac{(0,y_i - y_j + n L)}{|y_i - y_j + n L|^3} \bigg)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in above relation, r_i and r_j are known and n is an integer number which changes from -infinity to infinity &lt;/p&gt;&#xA;" OwnerUserId="23769" LastEditorUserId="9667" LastEditDate="2017-07-05T15:49:03.393" LastActivityDate="2017-07-05T15:49:03.393" Title="Simulating periodic boundary condition with long_range interaction" Tags="|boundary-conditions|simulation|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27330" PostTypeId="1" CreationDate="2017-07-05T15:11:51.803" Score="0" ViewCount="4810" Body="&lt;p&gt;I have attempted to create Poisson-distributed random numbers, seeing that it is not so easy as the simple multiplicative algorithm works accurately only if the mean is less than 500. Using logarithms is slightly slower, but it works at arbitrary mean values. However, it becomes extremely slow if the mean is very large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current algorithm is here in C-like language, assuming the functions &lt;code&gt;nextUniformRandomDouble&lt;/code&gt; (returning uniform value between 0.0, inclusive, and 1.0, exclusive) and &lt;code&gt;nextGaussianRandomDouble&lt;/code&gt; are defined:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int result = 0;&#xA;if (mean &amp;lt; 500.0)&#xA;{&#xA;    double L = exp(-mean);&#xA;    double p = 1;&#xA;    do&#xA;    {&#xA;        result++;&#xA;        p *= nextUniformRandomDouble();&#xA;    }&#xA;    while (p &amp;gt; L);&#xA;    result--;&#xA;}&#xA;else if (mean &amp;lt; 100000.0)&#xA;{&#xA;    double t = 0.0;&#xA;    while (t &amp;lt;= 1.0)&#xA;    {&#xA;        t += -log(1.0 - nextUniformRandomDouble())/mean;&#xA;        result++;&#xA;    }&#xA;    result--;&#xA;}&#xA;else&#xA;{&#xA;    result = (int)(nextGaussianRandomDouble() * sqrt(mean) + mean);&#xA;    if (result &amp;lt; 0)&#xA;    {&#xA;        // Probably never happens, but let's be safe&#xA;        result = 0;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I am dissatisfied of this solution because of the low performance at values slightly below 100000 and the inaccuracy of the Gaussian approximation at values slightly above 100000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way of doing the same? What I'm looking for is extreme accuracy on one hand and extreme performance on the other hand.&lt;/p&gt;&#xA;" OwnerUserId="24778" LastActivityDate="2017-07-06T14:18:07.320" Title="How to generate Poisson-distributed random numbers quickly and accurately?" Tags="|floating-point|random-number-generation|probability|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27331" PostTypeId="2" ParentId="27325" CreationDate="2017-07-05T16:03:56.360" Score="3" Body="&lt;p&gt;You should reformulate your problem.&#xA;Let's define the vector $u$ as $u=\left(\begin{array}{c}A\\ B\end{array}\right)$&#xA;Then you can write your coupled system as&#xA;$$\frac{\partial u}{\partial t}=\left(\begin{array}{c}a_0\\ b_0\end{array}\right)\left(u^T \left(\begin{array}{cc}0&amp;amp;0.5\\0.5&amp;amp;0\end{array}\right) u\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we can apply &lt;a href=&quot;https://en.wikipedia.org/wiki/Crank%E2%80%93Nicolson_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Crank Nicolson&lt;/a&gt; as defined in the wikipedia article:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{u^{n+1}-u^n}{\Delta t}=\frac{1}{2}\left(\left(\begin{array}{c}a_0\\ b_0\end{array}\right)\left({u^{n+1}}^T \left(\begin{array}{cc}0&amp;amp;0.5\\0.5&amp;amp;0\end{array}\right) u^{n+1}\right)+\left(\begin{array}{c}a_0\\ b_0\end{array}\right)\left({u^n}^T \left(\begin{array}{cc}0&amp;amp;0.5\\0.5&amp;amp;0\end{array}\right) u^n\right)\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one can see, this is a nonlinear function in $u$. So for solving this step of the Crank Nicolson scheme one should use an iterative scheme like Newton-Kantorovich with the Frechet derivative of the expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be $M=\left(\begin{array}{cc}0&amp;amp;0.5\\0.5&amp;amp;0\end{array}\right)$ and $c=\left(\begin{array}{c}a_0\\b_0\end{array}\right)$. To solve the nonlinear equation for each time step, we apply the Newton-Kantorovich iteration:&#xA;$$N_u\Delta=-N(u)$$, here $N_u\Delta$ is the Frechet derivative of &#xA;$$N(u)=\frac{u-u^n}{\Delta t}-\frac{1}{2}c\left(\left({u}^T M u\right)+\left({u^n}^T M u^n\right)\right)$$. I will use the greek index $\alpha$ to illustrate the fix point iteration. The Frechet derivative is defined as $$\lim_{\epsilon\rightarrow 0} \frac{N(u+\epsilon\Delta) -N(u)}{\epsilon}$$, hence $$N_u(u)\Delta=\left(\left(\begin{array}{cc}\frac{1}{\Delta t}&amp;amp;0\\0&amp;amp;\frac{1}{\Delta t}\end{array}\right)-c\left(u^T M\right)\right) \Delta$$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for each time step, you do the fix point iteration starting with $u^{0,n+1}=u^n$ by solving  $$N_u(u^{\alpha,n+1})\Delta=-N(u^{\alpha,n+1})$$ for $\Delta$, till $\Delta$ is small and updating $$u^{\alpha+1,n+1}=u^{\alpha,n+1}+\Delta$$ for each iteration step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the fix point iteration is converged (e.g.$|\Delta|&amp;lt;\epsilon$), you have found the next time step $u^{n+1}$ you can use in your Crank Nicolson scheme.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2017-07-07T11:13:25.310" LastActivityDate="2017-07-07T11:13:25.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27332" PostTypeId="2" ParentId="27303" CreationDate="2017-07-05T19:52:29.657" Score="1" Body="&lt;p&gt;Since your actual problem is totally different from your initial one I will explain what you should do in what follows, using three different approaches:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your equation to be solved is:&#xA;$$\gamma K\rho^{\gamma-2}\vec{\textrm{grad}}\rho=\vec{\textrm{grad}}\Phi \tag{*}$$&#xA;$\textbf{Fast solution}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can notice from $(*)$ that can be simply rewritten:&#xA;$$\vec{\textrm{grad}}\left(\frac{\gamma K }{\gamma-1}\rho^{\gamma-1}-\Phi\right)=0\tag{a}$$&#xA;Therefore your solution for $\rho$ from $(a)$ is:&#xA;$$\rho = \left(C+\frac{\gamma -1 }{\gamma K}\Phi\right)^{\frac{1}{\gamma-1}}$$&#xA;Where $C$ is a constant that can be adjusted using an arbitrary known value for $\rho$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{Equation as &quot;it is''}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compute explicitly the gradient of your potential $\Phi$ and denote by $\Phi_x$ and $\Phi_y$ its partial derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your discretised equations, following the simplified equation $C(\rho)\vec{grad}{\rho}=(\Phi_x,\Phi_y)^{T}$ would be:&#xA;$$C_{i,k}\left(\rho_{i+1,k}-\rho_{i,k}\right)=\Delta x\, \Phi_{x\,i,k} \quad i=1,...,N_x-1 \quad k=1,...,N_y\tag{b1}$$&#xA;$$C_{i,k}\left(\rho_{i+1,k}-\rho_{i,k}\right)=\Delta y\, \Phi_{x\,i,k}\quad i=1,...,N_x \quad k=1,...,N_y-1\tag{b2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we have the equation $(*)$ we know that $\rho$ is compatible with the right hand side, $i.e$ rho is a true potential and coincides with $\Phi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we have two equations: $(b1)$ and $(b2)$ (that are not independent) to solve one variable $\rho$, we have an overdetermined system of equations. But we know from the fact that $\rho$ is a true potential  that the integration along $x$ (equation $(b1)$) or along $y$ (equation $(b2)$) must result in the same function. The number of equations given by $(b1)$ is $(N_x-1)N_y$ and the given by $(b2)$ is $N_x(N_y-1)$. Therefore the total number of equations $2N_xN_y-N_x-N_y$ is bigger that the true number of unknowns $N_xN_y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Build the total system and drop off the equations that give nothing new. Do not forget to impose the given value that confers uniqueness to your problem replacing an additional equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{Well posed problem}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the kind of problem: find the potential $\phi$ that verifies $\vec{\textrm{grad}}\phi = \vec{f}$ one can set the Poisson problem that solves:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\triangle \phi = \textrm{div}\vec{f} \tag{**}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the boundary condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial\phi}{\partial n} = \vec{f}\cdot\vec{n}\tag{***}$$&#xA;Where $\vec{n}$ is the boundary unitary outer normal.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem $(**)$ and $(***)$ is numerically preferred compared with the second procedure, given by $(b1)$ and $(b2)$ and $\phi$ can be obtained with better precision choosing an appropiate numerical scheme, $e.g$ in your case central finite differences (compute analitically the derivatives of the RHS):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\phi_{i+1,k}-2\phi_{i,k}+\phi_{i-1,k}}{\Delta x^2}+\frac{\phi_{i,k+1}-2\phi_{i,k}+\phi_{i,k-1}}{\Delta y^2}=\left(\frac{\partial f_1}{\partial x}\right)_{i,k}+\left(\frac{\partial f_2}{\partial y}\right)_{i,k} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the discretised boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\phi_{N_x+1,k}-\phi_{N_x-1,k}}{2\Delta x} = \left(f_1\right)_{N_x,k}$$&#xA;$$\frac{\phi_{i,N_y+1}-\phi_{i,N_y-1}}{2\Delta y} = \left(f_1\right)_{i,N_y}$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-07-06T07:42:16.463" LastActivityDate="2017-07-06T07:42:16.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27333" PostTypeId="2" ParentId="27330" CreationDate="2017-07-05T21:01:05.487" Score="1" Body="&lt;p&gt;I found the answer from the book Numerical Recipes in C, but unfortunately the code in that book is copyrighted. So, I re-derived a similar code that is slightly different but does the same thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that we select a random number from distribution $g(x)$ that has the property that $f(x) &amp;lt; c g(x)$ always with a finite constant $c$, where $f(x)$ is our Poisson distribution consisting of boxes of width 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the selected $g(x)$ is $g(x) = \frac{1}{\pi \gamma}\frac{\gamma^2}{(x-x_0)^2+\gamma^2}$ which is the Cauchy distribution. The CDF is $y = F(x) = \frac{1}{\pi}\arctan(\frac{x-x_0}{\gamma}) + \frac{1}{2}$ and the inverse CDF is $x = x_0 + \gamma\tan(\pi(y-\frac{1}{2}))$. Thus, we generate $y$ randomly from uniform distribution between 0.0 and 1.0, and then calculate $x$. The parameters we select are $x_0 = \lambda$ and $\gamma = \sqrt{\lambda}$ where $\lambda$ is the parameter (mean value) of the Poisson distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This distribution has negative values as well, so every time a negative value is obtained, the $y$ and $x$ need to be recalculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this choice of distribution, it is true that $f(\lfloor x \rfloor)$ is always smaller than $2.4g(x)$ given any value of $\lambda$. So, we calculate the ratio $\frac{f(\lfloor x \rfloor)}{2.4g(x)}$ and reject if a uniformly distributed random variable $U$ between 0.0 and 1.0 is $U &amp;gt; \frac{f(\lfloor x \rfloor)}{2.4g(x)}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate $f(\lfloor x \rfloor) = f(m) = \exp(-\lambda)\frac{\lambda^m}{m!}$, we calculate $\exp(m\log\lambda - \lambda - \textrm{lgamma}(m+1))$. The function $\textrm{lgamma}$ has a series representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the C code follows. I have tested that the C code gives me integers that have very accurate mean and also very accurate standard deviation. Thus, I believe the code is correct. The break-even point 60 was empirically determined so that the two algorithms given this point are equally fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double lgamma(double xx)&#xA;{&#xA;  double pi = 3.14159265358979;&#xA;  double xx2 = xx*xx;&#xA;  double xx3 = xx2*xx;&#xA;  double xx5 = xx3*xx2;&#xA;  double xx7 = xx5*xx2;&#xA;  double xx9 = xx7*xx2;&#xA;  double xx11 = xx9*xx2;&#xA;  return xx*log(xx) - xx - 0.5*log(xx/(2*pi)) +&#xA;         1/(12*xx) - 1/(360*xx3) + 1/(1260*xx5) - 1/(1680*xx7) +&#xA;         1/(1188*xx9) - 691/(360360*xx11);&#xA;}&#xA;&#xA;int poissrnd_small(double mean)&#xA;{&#xA;  double L = exp(-mean);&#xA;  double p = 1;&#xA;  int result = 0;&#xA;  do {&#xA;    result++;&#xA;    p *= drand48();&#xA;  } while (p &amp;gt; L);&#xA;  result--;&#xA;  return result;&#xA;}&#xA;&#xA;int poissrnd_large(double mean)&#xA;{&#xA;  double r;&#xA;  double x, m;&#xA;  double pi = 3.14159265358979;&#xA;  double sqrt_mean = sqrt(mean);&#xA;  double log_mean = log(mean);&#xA;  double g_x;&#xA;  double f_m;&#xA;&#xA;  do {&#xA;    do {&#xA;      x = mean + sqrt_mean*tan(pi*(drand48()-1/2.0));&#xA;    } while (x &amp;lt; 0);&#xA;    g_x = sqrt_mean/(pi*((x-mean)*(x-mean) + mean));&#xA;    m = floor(x);&#xA;    f_m = exp(m*log_mean - mean - lgamma(m + 1));&#xA;    r = f_m / g_x / 2.4;&#xA;  } while (drand48() &amp;gt; r);&#xA;  return (int)m;&#xA;}&#xA;&#xA;int poissrnd(double mean)&#xA;{&#xA;  if (mean &amp;lt; 60)&#xA;  {&#xA;    return poissrnd_small(mean);&#xA;  }&#xA;  else&#xA;  {&#xA;    return poissrnd_large(mean);&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;b&gt;Edit:&lt;/b&gt; I hereby place the code under public domain. Thus, you are free to use it however you want to, no need to attribute anything. The code has been entirely written by me.&lt;/p&gt;&#xA;" OwnerUserId="24778" LastEditorUserId="24778" LastEditDate="2017-07-06T14:18:07.320" LastActivityDate="2017-07-06T14:18:07.320" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27334" PostTypeId="2" ParentId="27330" CreationDate="2017-07-05T22:17:43.060" Score="2" Body="&lt;p&gt;If you can use C++11 you can use the &lt;a href=&quot;http://en.cppreference.com/w/cpp/numeric/random&quot; rel=&quot;nofollow noreferrer&quot;&gt;built-in Poisson distribution&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;random&amp;gt;&#xA;&#xA;static std::random_device rd;&#xA;static std::mt19937 gen(rd());&#xA;&#xA;int main()&#xA;{&#xA;  std::poisson_distribution&amp;lt;int&amp;gt; pd(5);&#xA;&#xA;  for (int i = 0; i &amp;lt; 10000; ++i)&#xA;    std::cout &amp;lt;&amp;lt; pd(gen) &amp;lt;&amp;lt; '\n';&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I mean, come on, it doesn't get any easier than this.  And it's fast!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the output plotted with gnuplot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bcAal.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bcAal.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have to stick with plain C you could write a library which exposes the C++11 Mersenne Twister or use the &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Random-Number-Distributions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSL&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_rng.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_randist.h&amp;gt;&#xA;&#xA;int main()&#xA;{&#xA;  const gsl_rng_type * T;&#xA;  gsl_rng * r;&#xA;&#xA;  gsl_rng_env_setup();&#xA;  T = gsl_rng_default;&#xA;  r = gsl_rng_alloc (T);&#xA;&#xA;  for (int i = 0; i &amp;lt; 10000; ++i)&#xA;    printf(&quot;%d\n&quot;, gsl_ran_poisson(r, 5));&#xA;&#xA;  gsl_rng_free(r);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24680" LastEditorUserId="24680" LastEditDate="2017-07-05T22:25:36.780" LastActivityDate="2017-07-05T22:25:36.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27336" PostTypeId="1" CreationDate="2017-07-06T11:07:26.047" Score="0" ViewCount="30" Body="&lt;p&gt;For testing I would like to solve the equation&#xA;$$&#xA;\partial_tA=\nabla^2_xA,\,A_0=\sin(x)&#xA;$$&#xA;In order to solve it, I use an implicit solver&#xA;$$\left(1-\frac{\nabla^2_x}{2}dt\right)A_{n+1} = \left(1+\frac{\nabla^2_x}{2}dt\right)A_n&#xA;$$&#xA;or the Runge-Kutta-method in two different orders. For solving the whole equation I use the following matlab script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;&#xA;warning('off', 'all');&#xA;x_max = 1;&#xA;x_num = 1000;&#xA;a0_val = 1;&#xA;b0_val = 1;&#xA;dx = x_max/20000;&#xA;steps = 1/dx;&#xA;&#xA;x_vec = linspace(0, x_max, x_num).';&#xA;A0_vec = sin(x_vec);%linspace(1, x_num, x_num).';&#xA;ones_vec = ones(x_num, 1);&#xA;dh = (abs(x_vec(2)-x_vec(1)));&#xA;nabla_matrix = spdiags([-1*ones_vec 16*ones_vec -30*ones_vec 16*ones_vec -1*ones_vec], -2:2, x_num, x_num);&#xA;nabla_matrix(1, 1) = 35;&#xA;nabla_matrix(1, 2) = -104;&#xA;nabla_matrix(1, 3) = 114;&#xA;nabla_matrix(1, 4) = -56;&#xA;nabla_matrix(1, 5) = 11;&#xA;nabla_matrix(2, 1) = 11;&#xA;nabla_matrix(2, 2) = -20;&#xA;nabla_matrix(2, 3) = 6;&#xA;nabla_matrix(2, 4) = 4;&#xA;nabla_matrix(2, 5) = -1;&#xA;nabla_matrix(x_num, x_num) = 35;&#xA;nabla_matrix(x_num, x_num-1) = -104;&#xA;nabla_matrix(x_num, x_num-2) = 114;&#xA;nabla_matrix(x_num, x_num-3) = -56;&#xA;nabla_matrix(x_num, x_num-4) = 11;&#xA;nabla_matrix(x_num-1, x_num) = 11;&#xA;nabla_matrix(x_num-1, x_num-1) = -20;&#xA;nabla_matrix(x_num-1, x_num-2) = 6;&#xA;nabla_matrix(x_num-1, x_num-3) = 4;&#xA;nabla_matrix(x_num-1, x_num-4) = -1;&#xA;nabla_matrix = nabla_matrix / (12*(dh*dh));&#xA;&#xA;A_equation = @(A_vec) ((nabla_matrix*A_vec.'));&#xA;&#xA;A_matrix_single = speye(x_num)-dx/2.*nabla_matrix;&#xA;B_matrix_single = speye(x_num)+dx/2.*nabla_matrix;&#xA;&#xA;A_matrix = zeros(steps, x_num);&#xA;A_matrix_NL_rk4 = zeros(steps, x_num);&#xA;A_matrix_euler = zeros(steps, x_num);&#xA;A_matrix_rk4 = zeros(steps, x_num);&#xA;A_matrix_rk42 = zeros(steps, x_num);&#xA;&#xA;&#xA;A_matrix(1,:) = A0_vec;&#xA;A_matrix_NL_rk4(1,:) = A0_vec;&#xA;A_matrix_euler(1,:) = A0_vec;&#xA;A_matrix_rk4(1,:) = A0_vec;&#xA;A_matrix_rk42(1,:) = A0_vec;&#xA;&#xA;%Do steps&#xA;for i = 2:steps&#xA;    A_k1 = A_equation(A_matrix_rk4(i-1,:));&#xA;    A_k2 = A_equation(A_matrix_rk4(i-1,:)+A_k1.'*dx/2);&#xA;    A_k3 = A_equation(A_matrix_rk4(i-1,:)-A_k1.'*dx+2*dx*A_k2.');&#xA;    A_matrix_rk4(i,:) = A_matrix_rk4(i-1,:)+dx*(A_k1/6+2/3*A_k2+A_k3/6).';&#xA;end&#xA;&#xA;for i = 2:steps&#xA;    A_k1 = A_equation(A_matrix_rk42(i-1,:));&#xA;    A_k2 = A_equation(A_matrix_rk42(i-1,:)+A_k1.'*dx/2);&#xA;    A_k3 = A_equation(A_matrix_rk42(i-1,:)+0.5*dx*A_k2.');&#xA;    A_k4 = A_equation(A_matrix_rk42(i-1,:)+dx*A_k3.');&#xA;    A_matrix_rk42(i,:) = A_matrix_rk42(i-1,:)+dx*(A_k1/6+1/3*A_k2+A_k3/3+A_k4/6).';&#xA;end&#xA;&#xA;for i = 2:steps&#xA;    A_k1 = A_equation(A_matrix_NL_rk4(i-1,:));&#xA;    A_k2 = A_equation(A_matrix_NL_rk4(i-1,:)+A_k1.'*dx/2);&#xA;    A_k3 = A_equation(A_matrix_NL_rk4(i-1,:)+0.5*dx*A_k2.');&#xA;    A_k4 = A_equation(A_matrix_NL_rk4(i-1,:)+dx*A_k3.');&#xA;    A_matrix_NL_rk4(i,:) = speye(x_num)\(speye(x_num)*A_matrix_NL_rk4(i-1,:).'+speye(x_num)*(dx*(A_k1/6+1/3*A_k2+A_k3/3+A_k4/6)));&#xA;end&#xA;&#xA;for i = 2:steps&#xA;    A_matrix(i,:) = A_matrix_single\(B_matrix_single*A_matrix(i-1,:).');&#xA;end&#xA;&#xA;&#xA;&#xA;figure;&#xA;imagesc(A_matrix);&#xA;title('Forward step')&#xA;figure;&#xA;imagesc(A_matrix_rk4);&#xA;title('RK4 step')&#xA;figure;&#xA;imagesc(A_matrix_rk42);&#xA;title('RK42 step')&#xA;figure;&#xA;imagesc(A_matrix_NL_rk4);&#xA;title('RK4_NL step')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but the results are way of my expectations. I get a more or less correct result when using the implicit approach, but the latter three all explode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is that? Is my general approach flawed?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-07-06T11:07:26.047" Title="Comparison of different methods for solving $\partial_tA=\nabla^2_xA$" Tags="|matlab|pde|" AnswerCount="0" CommentCount="2" ClosedDate="2017-07-06T12:25:20.060" ContentLicense="CC BY-SA 3.0" />
  <row Id="27338" PostTypeId="1" CreationDate="2017-07-06T14:37:44.640" Score="1" ViewCount="195" Body="&lt;p&gt;This question was already asked &lt;a href=&quot;https://scicomp.stackexchange.com/questions/27229/schr%c3%b6dinger-equation-with-time-dependent-hamiltonian&quot;&gt;here&lt;/a&gt; and this is a suitable form of the equation for numerically solving. &#xA;\begin{align*}&#xA;i\frac{\partial}{\partial t}u_{\ell}(r,t) = \Bigg(-\frac{1}{2} \frac{\partial ^2}{\partial r^2} + \frac{\ell(\ell+1)}{2r^2} +  V(r)\Bigg)u_{\ell}(r,t)\quad \quad \quad \quad \quad \quad \quad \nonumber\\  + \frac{2}{3}k(t)r^2 \sum_{\ell' = {max}(\ell-2,0)}^{{min}(\ell+2,L_{{max}})} \Bigg(\delta_{\ell,\ell'} - \sqrt{ \frac{4\pi}{5}} \alpha(\ell, \ell') \Bigg) u_{\ell'}(r,t)&#xA;\end{align*}&#xA;where the coefficient $\alpha(\ell, \ell')$ is given by&#xA;\begin{align*}&#xA;	\alpha(\ell, \ell') = \int Y^*_{\ell0}(\Omega)Y_{20}(\Omega)Y_{\ell'0}(\Omega) d\Omega&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which the integral can be solved by Wigner-3j coefficients (see  &lt;a href=&quot;http://mathworld.wolfram.com/SphericalHarmonic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest an efficient numerically method for solving the equation?&lt;/p&gt;&#xA;" OwnerUserId="24650" LastEditorUserId="24650" LastEditDate="2017-07-08T08:11:58.377" LastActivityDate="2017-07-08T08:11:58.377" Title="Time-dependent Schrodinger equation(time-dependent Hamiltonian)" Tags="|numerical-analysis|finite-difference|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27339" PostTypeId="1" CreationDate="2017-07-06T18:22:31.543" Score="1" ViewCount="44" Body="&lt;p&gt;Let's say I have a random arbitrary geometry, like this one &lt;a href=&quot;https://ibb.co/dG8fUa&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ibb.co/dG8fUa&lt;/a&gt; (something I just made up). I make one of these in 3D in SolidWorks. I have molecules/particles coming in through the inlet and leaving out of the two outlets. In a standard geometry, like a pipe or square channel, a simple way to implement a wall BC is something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if(y_particle&amp;gt;0.3)&#xA;  v_particle=-v_particle&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;assuming a square channel height of 0.15 and the origin along the lengthwise axis. This obviously wont work for a complicated geometry with wildly varying X and Y. How might one implement codes which can work on any generated geometry? I am trying for MD simulations through such geometries.&lt;/p&gt;&#xA;" OwnerUserId="23565" LastActivityDate="2017-07-06T18:22:31.543" Title="How to implement boundary conditions in arbitrary geometries, in MD\Lagrangian FD simulations" Tags="|fluid-dynamics|molecular-dynamics|particle|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27340" PostTypeId="1" CreationDate="2017-07-06T19:29:17.347" Score="3" ViewCount="158" Body="&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: Thanks to comments, I realized that I have two problems, each which can be described more clearly on its own. This revised question covers the first.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I would like to solve &#xA;$$&#xA;-\Delta u - 1=0,\quad u(0,y)= 0,\quad u(1,y)=0&#xA;$$&#xA;over a rectangle using alternating Schwarz. The following is my mesh, where the small disks indicate the internal boundary nodes of the subdomains (the nodes that have Dirichlet conditions that I update on them):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/DEIR6.png&quot; width=&quot;500&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x = 0$ and $x = 1$ are the physical boundaries of the full mesh. The local solutions do not converge to the global solution. The problem is mostly in the overlapping part of the mesh, here is the difference between the actual solution and the solution given by the local solutions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/0629N.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I impose the Dirichlet conditions $u(x,0) = 0$ and $u(x,1/\phi) = 0$, where $1/\phi$ is the height of the mesh, then the local solutions converge to the global solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Should it not also work with only the boundary conditions that I mentioned about earlier?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&#xA;I solved it by selecting the artifical boundary in this manner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/PlZeQ.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="23932" LastEditorUserId="23932" LastEditDate="2017-07-21T22:57:24.357" LastActivityDate="2017-07-21T22:57:24.357" Title="Alternating Schwarz does not converge without Dirichlet conditions on physical boundaries" Tags="|finite-element|domain-decomposition|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27341" PostTypeId="1" CreationDate="2017-07-07T01:25:10.087" Score="-1" ViewCount="1038" Body="&lt;p&gt;Hailing from the scicomp community recently getting into computer-graphics, I have noticed that the scicomp communities talk about mesh-based methods like FDM, FVM, FEM, etc, vs meshfree or meshless methods like SPH (though I like to call them virtual particle methods), but I've never heard them talk about Eulerian approach vs Lagrangian approach, something I hear a lot in computer-graphics communities involved in computational-physics/mechanics, and they never talk about the terms meshfree or meshless, and rarely mention finite-elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be correct to say that Eulerian approach is another name for mesh-based simulation, while Lagrangian approach is a synonym for meshfree/meshless?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, can anyone clarify, or provide a good reading on this topic? (No need to invoke the boat analogy in which Lagrangian is like sitting in a boat in a river, and Eulerian is like watching the boat from the river bank. I've heard/read/watched that analogy about a dozen times and it has zero comparison to FEM, meshless, meshfree and is therefore useless in answering my question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="15198" LastActivityDate="2017-07-07T08:37:07.033" Title="Eulerian vs Lagrangian vs Mesh-based vs Meshfree/Meshless methods" Tags="|finite-element|meshless-methods|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27342" PostTypeId="2" ParentId="27341" CreationDate="2017-07-07T08:37:07.033" Score="0" Body="&lt;p&gt;Well, I do not claim to be an expert on the subject as I am still an early grad student, but I do have experience completing long-term projects in both areas, so I think I can probably frame an answer here in the context of fluid mechanics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Paul pointed out in the comment, Lagrangian/Eulerian refers to the frame of reference, the boat analogy :P. Strictly speaking, you cannot directly relate them to meshed/meshless methods. FEM, FVM are usually seen as meshed methods because they rely on the assumption of infinite, tiny control volumes making up a larger volume; normally people rely on the continuum assumption to hold and the fluid to have uniform properties. But   resolve the interior of the particles. In this case, it is still a meshfree method, but since we are tracking the particle trajectories, it is no longer a purely Eulerian problem. But if, the particle is significantly deformable, and FEM is used to figure out its internal stresses, we are in the realm of Eulerian tracking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As another example, molecular dynamics simulations-which I think helped form the part of the fundamental technology behind computer graphics simulations-are purely Lagrangian approaches, tracking particles through space. But use MD on couette/poiseuille flow to figure out the entry/exit velocities of the channel as a whole and voila! you are suddenly in the Eulerian regime(tracking averaged properties over a control volume)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My point is, that while these methods have developed differentiation based on the enormous applications in a wide range of study, in computer graphics I am principally concerned about making it look the way I want it to. So the discussion might usually boil droplet/particle tracking down to Lagrangian and water surface animations and stuff to Eulerian. But they are not interchangeable.&lt;/p&gt;&#xA;" OwnerUserId="23565" LastActivityDate="2017-07-07T08:37:07.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27344" PostTypeId="1" AcceptedAnswerId="27349" CreationDate="2017-07-07T11:01:52.267" Score="-1" ViewCount="5676" Body="&lt;p&gt;I have a set of three coupled autonomous equations:&lt;br&gt;&#xA;${y_{1}}\prime = y_{1}(\frac{\Omega_{m}}{y_{1}^3} + \frac{y_{3}^2}{6.0} + \frac{V(y_{2})}{2.H_{0}^2})$&lt;br&gt;&#xA;$y_{2}\prime = y_{3}$&lt;br&gt;&#xA;$y_{3}\prime = -3\frac{y_{1}\prime}{y_{1}}y_{3} - \frac{1}{H_{0}^2}\frac{\partial v(y2)}{\partial y_{2}}$&lt;br&gt;&#xA;And I have used the following code to solve it using scipy.odeint:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def f(y,t):&#xA;    Xi = y[0]&#xA;    Yi = y[1]&#xA;    Zi = y[2]&#xA;    f0 = Xi*(onr/(Xi**3) + (H**2)*(Zi**2)/(2.0*rho_c) + (v0*np.exp(-l*Yi*k))/(rho_c))**(1.0/2.0)&#xA;    f1 =  Zi&#xA;    f2 =  -3*Zi*(H**2)*(onr/(Xi**3) + (H**2)*(Zi**2)/(2.0*rho_c) + (v0*np.exp(-l*Yi*k))/(rho_c))**0.5 + (l*k*v0*np.exp(-l*Yi*k))/(H**2)&#xA;    return [f0,f1,f2]&#xA;&#xA;X0 = [1.0]&#xA;Y0 = [1.0]&#xA;Z0 = [c]&#xA;y0 = [X0,Y0,Z0]&#xA;t = np.linspace(start=1.0,stop=0.0,num=10001)&#xA;&#xA;soln = odeint(f,y0,t)&#xA;X = soln[:,0]&#xA;Y = soln[:,1]&#xA;Z = soln[:,2]  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But when I run it &lt;code&gt;ValueError: Initial condition y0 must be one-dimensional&lt;/code&gt; shows up. What should I do? I am new to python and this is the first time I am using this. Any help or explanation here would be great.&lt;/p&gt;&#xA;" OwnerUserId="24798" LastActivityDate="2017-07-07T18:11:13.327" Title="Using scipy.odeint to solve coupled equations" Tags="|numerical-analysis|python|ode|scipy|integration|" AnswerCount="1" CommentCount="1" ClosedDate="2017-07-08T03:37:26.510" ContentLicense="CC BY-SA 3.0" />
  <row Id="27345" PostTypeId="2" ParentId="27267" CreationDate="2017-07-07T11:55:42.613" Score="4" Body="&lt;p&gt;Since you've asked about experiences, I've seen all three of the methods you suggest used. However best practise is that inputs to the code, and the code framework itself are kept separate. This means that only parameters you don't expect to ever need to change should be hard coded into your source. For example, if you were modelling the flight of cannonballs, you might hard code the value for gravitational acceleration, but not the radius of the cannonball, which forms part of the metadata for the individual simulation. That way the code base in your version control system doesn't get polluted with input data, and remains sane for someone else (or you six months later) to understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of classes, versus configuration files for the simulation input data, that really comes down to the language being used and to whether you're developing an executable or a framework. Again, best practise is to use a self-documenting system like xml or JSON for configuration files (no need to roll your own implementation, there are prebuilt parsing libraries out there). Ideally the file should be human readable, either directly or via a helper utility. For the class based method, the attribute names should again be human understandable, although there are still people out there who are stuck in the FORTRAN 66 structure of six-letter variable names like &quot;GRNEW4&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Outputting should again be in a commonly used, self-describing file format, and ideally should include enough metadata to be able to reproduce the original input data which generated it if necessary. Since input files tend to be small compared to the output data, one option is to simply prepend the input file/source at the start of the output data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method of organisation for the final output is probably the point of most debate. While version control systems work pretty well for input files (in a separate repository from the source code remember), they don't always play that well with binary data. This leads to a lot of disk based organisation (use human understandable file names!), but a useful possibility is to take advantage of one of the various generic reference managers like JabRef, which are readily available and provide searchable tracking of the simulation metadata.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2017-07-07T11:55:42.613" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27346" PostTypeId="1" CreationDate="2017-07-07T14:08:23.113" Score="4" ViewCount="1339" Body="&lt;p&gt;As per wikipedia, scientists have not been successful to accurately predict the whether which is 2 weeks ahead.&#xA;Here is the excerpt: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The atmosphere is a chaotic system, as a result, small changes to one&#xA;  part of the system can grow to have large effects on the system as a&#xA;  whole.This makes it difficult to accurately predict weather more than&#xA;  a few days in advance, though weather forecasters are continually&#xA;  working to extend this limit through the scientific study of weather,&#xA;  meteorology. It is theoretically impossible to make useful day-to-day&#xA;  predictions more than about two weeks ahead, imposing an upper limit&#xA;  to potential for improved prediction skill&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="24799" LastActivityDate="2017-07-07T21:56:26.230" Title="Why is it not computationally possible to accurately predict the weather that would occur after 14 days?" Tags="|computational-physics|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="27348" PostTypeId="2" ParentId="27346" CreationDate="2017-07-07T15:15:14.733" Score="9" Body="&lt;p&gt;Errors grow exponentially in a chaotic system, and most people believe weather is chaotic. So even if you get a fairly exact numerical approximation, the fact that your input data (temperatures, etc.) and your model are slightly off from reality will cause results which exponentially diverge from reality as time goes on. Thus long time prediction of weather is extremely difficult!&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-07-07T16:46:10.837" LastActivityDate="2017-07-07T16:46:10.837" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27349" PostTypeId="2" ParentId="27344" CreationDate="2017-07-07T18:11:13.327" Score="3" Body="&lt;p&gt;You shouldn’t have defined &lt;code&gt;X0&lt;/code&gt;, &lt;code&gt;Y0&lt;/code&gt;, and &lt;code&gt;Z0&lt;/code&gt; as lists. As it stands, &lt;code&gt;y0&lt;/code&gt; is a list of lists – i.e., two-dimensional. You can solve this by replacing the respective lines with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X0 = 1.0&#xA;Y0 = 1.0&#xA;Z0 = c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2017-07-07T18:11:13.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27350" PostTypeId="2" ParentId="27346" CreationDate="2017-07-07T20:35:42.850" Score="13" Body="&lt;p&gt;Further to Chris' answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, weather (or the equations describing it) is extremely sensitive to the initial conditions. The fact that the weather system contains phenomena at pretty much all time and space scales (have a look &lt;a href=&quot;https://www.rsmas.miami.edu/users/bmapes/teaching/WhatIsMesoscale.Markowski.pdf&quot; rel=&quot;noreferrer&quot;&gt;at figures 1.1 and 1.2 of these notes&lt;/a&gt; for some examples) does not make predictions any easier.  Also, the &quot;weather&quot; (at various scales) is dependent on a whole bunch of other planetary systems (oceans, land-based processes, etc.) that come in with their own model and observational uncertainties:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/guZ6J.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/guZ6J.jpg&quot; alt=&quot;Figure 2 of https://www.nature.com/nature/journal/v525/n7567/full/nature14956.html&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Figure taken from &lt;a href=&quot;https://www.nature.com/nature/journal/v525/n7567/full/nature14956.html&quot; rel=&quot;noreferrer&quot;&gt;Bauer et al., Nature, 2015&lt;/a&gt; showing &quot;physical processes of importance to weather prediction&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, significant progress has been made in the past few decades, and people in the operational weather forecasting community (at least the folks over at &lt;a href=&quot;https://www.ecmwf.int&quot; rel=&quot;noreferrer&quot;&gt;ECMWF&lt;/a&gt;, which are really &lt;a href=&quot;https://www.ecmwf.int/en/forecasts/quality-our-forecasts&quot; rel=&quot;noreferrer&quot;&gt;at the top of their game&lt;/a&gt; when it comes to weather forecasting) refer to this graph that showcases these improvements:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NBvCz.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NBvCz.jpg&quot; alt=&quot;A measure of forecast skill at three-, five-, seven- and ten-day ranges, computed over the extra-tropical northern and southern hemispheres.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The figure above is from &lt;a href=&quot;https://www.nature.com/nature/journal/v525/n7567/full/nature14956.html&quot; rel=&quot;noreferrer&quot;&gt;Bauer et al., Nature, 2015&lt;/a&gt; and shows a measure of &lt;a href=&quot;https://en.wikipedia.org/wiki/Forecast_skill&quot; rel=&quot;noreferrer&quot;&gt;forecast skill&lt;/a&gt; at three-, five-, seven- and ten-day ranges, computed over the extra-tropical northern (NH) and southern hemispheres (their caption). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What the figure shows is that forecast skill has been steadily increasing for the past 3 decades, with some major &quot;bumps&quot; in performance brought in by &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the advent of satellite (space-borne) measurements, which significantly narrowed the gap between the southern and northern hemispheres (the ground-based/sea observational network in the SH is much sparser than that of its northern counterpart).&lt;/li&gt;&#xA;&lt;li&gt;the introduction of a probabilistic (ensemble-based) weather forecasting framework&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, the 10-day forecast quality is the same as that of a 7-day forecast made 30 years ago, so you can say we're up a day every 10 years :) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I, for one, think that there is no &lt;em&gt;hard physical barrier&lt;/em&gt; (boo, &lt;a href=&quot;https://history.aip.org/climate/chaos.htm&quot; rel=&quot;noreferrer&quot;&gt;Lorenz&lt;/a&gt;!) that does not/will not allow us to predict the weather, say, 14 or 20 or 25 days in advance. It's just that we're lacking (1) sufficient knowledge of the weather system (2) computing power to simulate the weather at high-enough resolution (3) high-quality observational data to constrain existing 3D/4D-var and ensemble forecasting systems.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-07-07T21:56:26.230" LastActivityDate="2017-07-07T21:56:26.230" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27351" PostTypeId="1" CreationDate="2017-07-07T22:27:29.567" Score="12" ViewCount="660" Body="&lt;p&gt;In a software project that I'm working on, certain computations are vastly easier for dense low-rank matrices.  Some problem instances involve dense low-rank matrices, but they're given to me in full, rather than as factors, so I'll have to check the rank and factor the matrix if I want to take advantage of the low-rank structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrices in question are typically fully or nearly fully dense, with n ranging from one hundred up to a few thousand.  If a matrix has low rank (say less than 5 to 10), then computing the SVD and using it form a low-rank factorization is worth the effort.  However, if the matrix is not of low rank, then the effort would be wasted.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I'd like to find a fast and reasonably reliable way of determining whether or not the rank is low before investing the effort to do a full SVD factorization.  If at any point it becomes clear that the rank is above the cutoff, the process can stop immediately.  If the procedure mistakenly declares the matrix to be of low rank when it isn't, this isn't a huge issue, since I'd still be doing a full SVD to confirm the low rank and find a low-rank factorization.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Options that I've considered include a rank revealing LU or QR factorization followed by a full SVD as the check.  Are there other approaches that I should consider?  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="20417" LastEditDate="2017-07-08T12:45:27.307" LastActivityDate="2021-07-30T08:00:28.093" Title="Rapidly determining whether or not a dense matrix is of low rank" Tags="|linear-algebra|matrix|lapack|rank|matrix-factorization|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27352" PostTypeId="1" AcceptedAnswerId="27353" CreationDate="2017-07-07T22:49:49.620" Score="1" ViewCount="71" Body="&lt;p&gt;In case of global temperatures we are interested in aggregate values rather than local distributions. Is this the reason we can accurately predict one but not the other?&lt;/p&gt;&#xA;" OwnerUserId="24803" LastActivityDate="2017-07-07T23:12:17.817" Title="Why can we predict global temperatures years ahead but we cannot reliably predict weather weeks ahead?" Tags="|simulation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27353" PostTypeId="2" ParentId="27352" CreationDate="2017-07-07T23:12:17.817" Score="3" Body="&lt;p&gt;Correct. Think about sitting by a little stream and looking at how it flows around a rock. There will be eddies that go this way and that way, and they are pretty chaotic. So you can't accurately predict the &lt;em&gt;exact&lt;/em&gt; velocity of the stream at one place for a time that is more than two or three times as large as it takes for an eddy to move past that time. That's the equivalent of the weather.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you &lt;em&gt;can&lt;/em&gt; accurately predict the &lt;em&gt;average&lt;/em&gt; velocity at the point long in the future -- it will simply be the current average velocity plus or minus a correction you can compute if you know whether the volume of water the stream transports per second increases or decreases between now and the point where you need your correction. That's the equivalent of climate.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-07-07T23:12:17.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27354" PostTypeId="2" ParentId="27351" CreationDate="2017-07-07T23:16:22.067" Score="8" Body="&lt;p&gt;The problem, of course, is that computing the true rank (e.g., via a QR decomposition) is not really any cheaper than computing a low-rank representation of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best you can probably do is to use a randomized algorithm to find low-rank approximations. These can, at least in theory, be significantly faster than working on the entire matrix because, in essence, they only compute decompositions for &lt;em&gt;projections&lt;/em&gt; of the matrix onto random subspaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether that's worth it for a matrix of size $100\times 100$ may be a good question, but if your problems really become large, I would suspect that it pays off.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-07-07T23:16:22.067" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27355" PostTypeId="2" ParentId="7008" CreationDate="2017-07-07T23:54:34.197" Score="4" Body="&lt;p&gt;The GSL also has &lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Complete-Fermi_002dDirac-Integrals.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;complete Fermi-Dirac integrals&lt;/a&gt;, $F_j(x)$  for integer  $j$ and $j=-\frac12, \frac12, \frac32$. These functions &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Complete_Fermi%E2%80%93Dirac_integral&quot; rel=&quot;nofollow noreferrer&quot;&gt;are equivalent to polylogs&lt;/a&gt; &#xA;$$&#xA;F_j(x) = -\text {Li}_{j+1}(-e^{-x})&#xA;$$&#xA;Though note the restriction to negative polylog arguments for real $x $.&lt;/p&gt;&#xA;" OwnerUserId="21653" LastEditorUserId="651" LastEditDate="2017-07-08T01:23:45.800" LastActivityDate="2017-07-08T01:23:45.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27358" PostTypeId="2" ParentId="8223" CreationDate="2017-07-08T11:29:58.270" Score="0" Body="&lt;p&gt;Here is a method to calculate the distance between an array of vectors (&lt;code&gt;X&lt;/code&gt;) and a single vector (&lt;code&gt;X(1, :)&lt;/code&gt; for example purposes):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;distances = sqrt(sum((X - X(1, :)) .^ 2, 2));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16337" LastActivityDate="2017-07-08T11:29:58.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27359" PostTypeId="1" CreationDate="2017-07-08T14:42:08.763" Score="1" ViewCount="833" Body="&lt;p&gt;Consider a potential/poisson equation on a very large, complicated geometry. Currently, an self-written FEM and linear solvers from NumPy are used. Performance is, of course, not good enough for larger problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to switch to PETSc and use algebraic multigrid preconditioners. Surprisingly, I could not find good documentation or examples on algebraic multigrid in PETSc. Are there any good examples? Or is it maybe to early to use algebraic multigrid, if I don't want to go into development too deep?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This question is mainly intended to find more documentation about algebraic multigrid in PETSc, but please let me know if my approach is generally a bad idea.)&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-07-08T16:34:39.680" Title="Algebraic multigrid in PETSc" Tags="|finite-element|linear-solver|petsc|multigrid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27360" PostTypeId="2" ParentId="25972" CreationDate="2017-07-08T15:24:52.313" Score="-1" Body="&lt;p&gt;One of the conventional methods to assess the spatial convergence of RANS simulations is to perform grid refinement studies based on &lt;a href=&quot;https://www.grc.nasa.gov/www/wind/valid/tutorial/spatconv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Richardson extrapolation&lt;/a&gt;. Its formulation and practice can be found in the link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, one should not select the &lt;em&gt;grid refinement ratio&lt;/em&gt; arbitrarily. As a rule of thumb:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The grid refinement ratio (i.e. r) should be a minimum of r &gt;= 1.1 to allow the discretization error to be differentiated from other error sources (iterative convergence errors, computer round-off, etc...).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The terms such as &lt;em&gt;accuracy&lt;/em&gt; of a CFD simulation should always be formalised for a serious study. &lt;a href=&quot;https://www.grc.nasa.gov/www/wind/valid/tutorial/errors.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This link&lt;/a&gt; might be a good start to those who may concern.&lt;/p&gt;&#xA;" OwnerUserId="21325" LastEditorUserId="21325" LastEditDate="2017-07-08T15:32:56.933" LastActivityDate="2017-07-08T15:32:56.933" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27361" PostTypeId="1" CreationDate="2017-07-08T15:38:01.027" Score="1" ViewCount="91" Body="&lt;p&gt;Consider a problem where space is filled with a liquid and a solid phase, with a large, complicated geometry. On each of the phases, there's an electrical potential/poisson equation. The equations are coupled by a nonlinear surface current through the phase boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, an FEM is constructed for both phases and assembled into one large Jacobian which is then solved as a black-box in a Newton method. This leads to bad matrix condition and bad performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To improve performance, I plan to use PETSc. For multi-physics problems, they suggest the usage of nested matrices and then to use a separate preconditioner for each of the equations. BUT, for all the examples I found so far, there's one big difference: The examples always consider different physical equations on one phase which are coupled in all points in space. In my case, I have just one equation per phase, but a coupling only on the boundaries of the phase, and nonlinear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you deal with that kind of problem in PETSc? (Emphasis on the combination of matrices, solvers and preconditioners.)&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-08-20T23:32:31.613" Title="How to deal with multi-region problems (in PETSc)" Tags="|finite-element|petsc|multiphysics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27362" PostTypeId="2" ParentId="27359" CreationDate="2017-07-08T16:34:39.680" Score="2" Body="&lt;p&gt;AMG can be used with all examples in PETSc.  There are three robust implementations that you might want to use&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-pc_type gamg&lt;/code&gt; is a native (smoothed) aggregation method.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-pc_type ml&lt;/code&gt; is a smoothed aggregation implementation from the ML project (part of Trilinos).  Configure PETSc with &lt;code&gt;--download-ml&lt;/code&gt; to make this solver available.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-pc_type hypre&lt;/code&gt; is a classical AMG implementation (BoomerAMG from Hypre).  Configure PETSc with &lt;code&gt;--download-hypre&lt;/code&gt; to make this solver available.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="119" LastActivityDate="2017-07-08T16:34:39.680" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27363" PostTypeId="1" AcceptedAnswerId="27364" CreationDate="2017-07-08T18:42:04.507" Score="0" ViewCount="59" Body="&lt;p&gt;I have an equation system, constructed as &#xA;$$\begin{split}&#xA;\partial_tA&amp;amp;=f_1(A)\cdot f_2(B)\\&#xA;\partial_tB&amp;amp;=f_3(B)\cdot\partial_tf_4(A)&#xA;\end{split}$$&#xA;with $f_1,\,f_2,\,f_4$ and $f_3$ nonlinear functions. One idea I had was to use the forward difference for $\partial_tf_4(A)$, i.e.&#xA;$$\partial_tf_{4,(n)}(A)=\frac{f_{4,(n-1)}-f_{4,(n-2)}}{dt}$$&#xA;For that approach I would have to store the last two results for $f_4(A)$. Are there other approaches which can be used to solve this situation?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-07-08T19:01:16.340" Title="Approach for coupled equations with included time derivative" Tags="|pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27364" PostTypeId="2" ParentId="27363" CreationDate="2017-07-08T19:01:16.340" Score="1" Body="&lt;p&gt;Its better to compute the time derivative as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d_tf_4(A) =d_Af_4(A)\,d_tA $$&#xA;and rearrange your system of ODEs to have:&#xA;$$ \left[\begin{matrix}1 &amp;amp; 0\\&#xA;-f_3(B)d_Af_4(A) &amp;amp; 1\end{matrix}\right]d_t\left[\begin{matrix}{A\\B}\end{matrix}\right]=\left[\begin{matrix}f_1(A)f_2(B)\\ 0\end{matrix}\right]$$&#xA;Which is shortly written with $\vec{W} = (A, B)^{T}$&#xA;$$M(\vec{W})d_t\vec{W}=\vec{F}(\vec{W})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest way to solve it is applying the $\theta$ methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\vec{W}^{n+1}-\vec{W}^{n}}{\Delta t}=\theta M^{-1}(\vec{W}^{n+1})\vec{F}(\vec{W}^{n+1})+(1-\theta)M^{-1}(\vec{W}^{n})\vec{F}(\vec{W}^{n})$$&#xA;For $\theta=0$ you obtain the explicit Euler scheme, and for $\theta=1$ the implicit one.&#xA;For $\theta=1/2$ you obtain a second order scheme in time (Crack-Nicholson method). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tha two last proposed values for $\theta$ lead you with a nonlinear scheme for $\vec{W}^{n+1}$ that requires some iterative method to be solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be careful with the singularity given by $\mathrm{det}{M}=0$ if any.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2017-07-08T19:01:16.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27365" PostTypeId="1" AcceptedAnswerId="27366" CreationDate="2017-07-08T21:33:26.197" Score="1" ViewCount="172" Body="&lt;p&gt;I have a rather long expression (&lt;a href=&quot;https://pastebin.com/jUsxdCCs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/jUsxdCCs&lt;/a&gt;) that is an analytical solution of a set of differential equations generated symbolically from Maple. I need to solve a set of equations like this in C, in a 32-bit system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The long expression has a variable &lt;code&gt;omega2&lt;/code&gt;, which when substituted leads to a number that I am interested in. When this operation is carried out in Maple, it results in a rather decent floating point number, but when I try to do it in C, (try with &lt;code&gt;omega2 = 100&lt;/code&gt;), I get &lt;code&gt;nan&lt;/code&gt;, the reason being that some of the sub-expressions in the long expression go out of range of &lt;code&gt;double&lt;/code&gt; i.e. 2.3E-308 to 1.7E+308. I have been able to resolve this using 'long double` datatype available on some systems, but this isn't platform independent and is definitely not available on &lt;a href=&quot;http://www.st.com/en/evaluation-tools/32l476gdiscovery.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;micro-controller that I plan to target&lt;/a&gt; which has a 32-bit ARM core. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution that I can imagine is to use the GMP library, that can handle numbers of infinite precision. Just want to know if there are any other hacks as GMP'ing the whole code can be fairly cumbersome and I don't have enough memory for GMP on the target machine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;The expression:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.137197706359762e-3 * (-0.708981540362208e2 * cosh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * (-0.727331795502673e6 * cosh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) - 0.209935630488671e8 * cosh(0.354490770181104e2 * sqrt(omega2)) * cos(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) - 0.727331795502673e6 * cos(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2))) * sqrt(omega2) / (0.348294222590750e3 * cosh(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2)) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sqrt(omega2) - 0.348294222590750e3 * cos(0.354490770181104e2 * sqrt(omega2)) * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * cos(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * cosh(0.354490770181104e2 * sqrt(omega2)) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sinh(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.120668159897616e2 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.120668159897616e2 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.502654824574368e4 * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2 + 0.502654824574368e4 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2) - 0.708981540362208e2 * cosh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) / (0.348294222590750e3 * cosh(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2)) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sqrt(omega2) - 0.348294222590750e3 * cos(0.354490770181104e2 * sqrt(omega2)) * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * cos(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * cosh(0.354490770181104e2 * sqrt(omega2)) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sinh(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.120668159897616e2 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.120668159897616e2 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.502654824574368e4 * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2 + 0.502654824574368e4 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2) * sqrt(omega2) * (-0.209935630488671e8 * sinh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) - 0.727331795502673e6 * cosh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) + 0.727331795502673e6 * cos(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2))) - 0.708981540362208e2 * cos(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2)) * (-0.727331795502673e6 * cosh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) - 0.209935630488671e8 * cosh(0.354490770181104e2 * sqrt(omega2)) * cos(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) - 0.727331795502673e6 * cos(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2))) * sqrt(omega2) / (0.348294222590750e3 * cosh(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2)) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sqrt(omega2) - 0.348294222590750e3 * cos(0.354490770181104e2 * sqrt(omega2)) * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * cos(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * cosh(0.354490770181104e2 * sqrt(omega2)) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sinh(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.120668159897616e2 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.120668159897616e2 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.502654824574368e4 * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2 + 0.502654824574368e4 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2) + 0.708981540362208e2 * cos(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2)) / (0.348294222590750e3 * cosh(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2)) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sqrt(omega2) - 0.348294222590750e3 * cos(0.354490770181104e2 * sqrt(omega2)) * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * cos(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * cosh(0.354490770181104e2 * sqrt(omega2)) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sinh(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.120668159897616e2 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.120668159897616e2 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.502654824574368e4 * pow(sinh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2 + 0.502654824574368e4 * pow(cosh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2) * sqrt(omega2) * (-0.209935630488671e8 * sinh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) - 0.727331795502673e6 * cosh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) + 0.727331795502673e6 * cos(0.354490770181104e2 * sqrt(omega2)) * sinh(0.354490770181104e2 * sqrt(omega2)))) * pow(omega2, -0.1e1 / 0.2e1) - 0.833338826540431e3;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Simplifying the expression in Maple, gives: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;.414741582884880e315/(.702517310158453e311*pow(.487478610966388,2.)+.702517310158453e311*pow(-.873134928776922,2.)+502666.891390358*pow(.449112744897583e154,2.)*pow(.487478610966388,2.)+502666.891390358*pow(-.873134928776922,2.)*pow(.449112744897583e154,2.))*pow(.1e3,-.500000000000000)-833.338826540431&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As can be seen, the sinh and cosh terms are blowing up inside, but cancel out ultimately. &lt;/p&gt;&#xA;" OwnerUserId="6981" LastEditorUserId="6981" LastEditDate="2017-07-11T22:49:11.090" LastActivityDate="2017-08-17T23:15:19.343" Title="How to deal with big numbers in intermediate calculations?" Tags="|numerical-analysis|c|precision|maple|numerical-limitations|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27366" PostTypeId="2" ParentId="27365" CreationDate="2017-07-08T23:45:18.047" Score="5" Body="&lt;p&gt;I think there's a simple way to do this. You have a rational function of identical cosh/sinh terms, where every expression is a homogeneous polynomial in cosh/sinh, and the only problem is that these exponential terms overflow. The function does not diverge as these terms approach infinity, so if you divide every numerator and denominator by the same power of cosh, you'll get rid of overflow and be left with harmless underflow only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the powers of cosh/sinh must match (the expression is bounded), we don't need to determine them by hand: we can (literally) replace every $\cosh(\cdots)$ by $1$, and every $\sinh(\cdots)$ by $\tanh(\cdots)$, getting the following expression:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.137197706359762e-3 * (-0.708981540362208e2 * 1 * sin(0.354490770181104e2 * sqrt(omega2)) * (-0.727331795502673e6 * 1 * sin(0.354490770181104e2 * sqrt(omega2)) - 0.209935630488671e8 * 1 * cos(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) - 0.727331795502673e6 * cos(0.354490770181104e2 * sqrt(omega2)) * tanh(0.354490770181104e2 * sqrt(omega2))) * sqrt(omega2) / (0.348294222590750e3 * 1 * tanh(0.354490770181104e2 * sqrt(omega2)) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sqrt(omega2) - 0.348294222590750e3 * cos(0.354490770181104e2 * sqrt(omega2)) * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * pow(1, 0.2e1) * cos(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * 1 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * tanh(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.120668159897616e2 * pow(1, 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.120668159897616e2 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.502654824574368e4 * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2 + 0.502654824574368e4 * pow(1, 0.2e1) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2) - 0.708981540362208e2 * 1 * sin(0.354490770181104e2 * sqrt(omega2)) / (0.348294222590750e3 * 1 * tanh(0.354490770181104e2 * sqrt(omega2)) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sqrt(omega2) - 0.348294222590750e3 * cos(0.354490770181104e2 * sqrt(omega2)) * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * pow(1, 0.2e1) * cos(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * 1 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * tanh(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.120668159897616e2 * pow(1, 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.120668159897616e2 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.502654824574368e4 * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2 + 0.502654824574368e4 * pow(1, 0.2e1) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2) * sqrt(omega2) * (-0.209935630488671e8 * tanh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) - 0.727331795502673e6 * 1 * sin(0.354490770181104e2 * sqrt(omega2)) + 0.727331795502673e6 * cos(0.354490770181104e2 * sqrt(omega2)) * tanh(0.354490770181104e2 * sqrt(omega2))) - 0.708981540362208e2 * cos(0.354490770181104e2 * sqrt(omega2)) * tanh(0.354490770181104e2 * sqrt(omega2)) * (-0.727331795502673e6 * 1 * sin(0.354490770181104e2 * sqrt(omega2)) - 0.209935630488671e8 * 1 * cos(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) - 0.727331795502673e6 * cos(0.354490770181104e2 * sqrt(omega2)) * tanh(0.354490770181104e2 * sqrt(omega2))) * sqrt(omega2) / (0.348294222590750e3 * 1 * tanh(0.354490770181104e2 * sqrt(omega2)) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sqrt(omega2) - 0.348294222590750e3 * cos(0.354490770181104e2 * sqrt(omega2)) * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * pow(1, 0.2e1) * cos(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * 1 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * tanh(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.120668159897616e2 * pow(1, 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.120668159897616e2 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.502654824574368e4 * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2 + 0.502654824574368e4 * pow(1, 0.2e1) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2) + 0.708981540362208e2 * cos(0.354490770181104e2 * sqrt(omega2)) * tanh(0.354490770181104e2 * sqrt(omega2)) / (0.348294222590750e3 * 1 * tanh(0.354490770181104e2 * sqrt(omega2)) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sqrt(omega2) - 0.348294222590750e3 * cos(0.354490770181104e2 * sqrt(omega2)) * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * pow(1, 0.2e1) * cos(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.348294222590750e3 * 1 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * tanh(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) + 0.120668159897616e2 * pow(1, 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.120668159897616e2 * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) + 0.502654824574368e4 * pow(tanh(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * pow(sin(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2 + 0.502654824574368e4 * pow(1, 0.2e1) * pow(cos(0.354490770181104e2 * sqrt(omega2)), 0.2e1) * omega2) * sqrt(omega2) * (-0.209935630488671e8 * tanh(0.354490770181104e2 * sqrt(omega2)) * sin(0.354490770181104e2 * sqrt(omega2)) * sqrt(omega2) - 0.727331795502673e6 * 1 * sin(0.354490770181104e2 * sqrt(omega2)) + 0.727331795502673e6 * cos(0.354490770181104e2 * sqrt(omega2)) * tanh(0.354490770181104e2 * sqrt(omega2)))) * pow(omega2, -0.1e1 / 0.2e1) - 0.833338826540431e3;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This only works when every expression is homogeneous in cosh/sinh and the degrees of numerators and denominators are the same, but it looks to me like they are, and numerically it checks out also. I think the same approach would work if the powers didn't match, but you'd probably have to cancel the powers manually or with a proper CAS.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-07-08T23:45:18.047" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27367" PostTypeId="2" ParentId="27365" CreationDate="2017-07-09T01:48:04.607" Score="2" Body="&lt;p&gt;If you do not want to (or just can't) juggle with the expression, just use the multiprecision library of your choice.  Below an example with Boost.Multiprecision in C++.  This has the invaluable advantage that you do not have to touch the original expression &lt;em&gt;at all&lt;/em&gt;, thanks to operator and function overloading.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;&#xA;#include &amp;lt;boost/multiprecision/cpp_dec_float.hpp&amp;gt;&#xA;&#xA;using boost::multiprecision::cpp_dec_float_50;&#xA;&#xA;// Forward declaration for brevity&#xA;cpp_dec_float_50 expr(cpp_dec_float_50 const &amp;amp;omega2);&#xA;&#xA;&#xA;int main ()&#xA;{&#xA;  std::cout &amp;lt;&amp;lt; expr (100) &amp;lt;&amp;lt; '\n';&#xA;}&#xA;&#xA;// Indentation by GNU indent&#xA;cpp_dec_float_50 expr(cpp_dec_float_50 const &amp;amp;omega2)&#xA;{&#xA;  return&#xA;    0.137197706359762e-3 * (-0.708981540362208e2 *&#xA;                cosh (0.354490770181104e2 * sqrt (omega2)) *&#xA;                sin (0.354490770181104e2 * sqrt (omega2)) *&#xA;                (-0.727331795502673e6 *&#xA;                 cosh (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 sin (0.354490770181104e2 * sqrt (omega2)) -&#xA;                 0.209935630488671e8 * cosh (0.354490770181104e2 *&#xA;                             sqrt (omega2)) *&#xA;                 cos (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 sqrt (omega2) -&#xA;                 0.727331795502673e6 * cos (0.354490770181104e2 *&#xA;                            sqrt (omega2)) *&#xA;                 sinh (0.354490770181104e2 * sqrt (omega2))) *&#xA;                sqrt (omega2) / (0.348294222590750e3 *&#xA;                         cosh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         sinh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         pow (sin&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) * sqrt (omega2) -&#xA;                         0.348294222590750e3 *&#xA;                         cos (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         pow (sinh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         sin (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sqrt (omega2) +&#xA;                         0.348294222590750e3 *&#xA;                         pow (cosh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         cos (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sin (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sqrt (omega2) +&#xA;                         0.348294222590750e3 *&#xA;                         cosh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         pow (cos&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         sinh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         sqrt (omega2) +&#xA;                         0.120668159897616e2 *&#xA;                         pow (cosh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         pow (sin&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) +&#xA;                         0.120668159897616e2 *&#xA;                         pow (cos&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         pow (sinh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) +&#xA;                         0.502654824574368e4 *&#xA;                         pow (sinh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         pow (sin&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) * omega2 +&#xA;                         0.502654824574368e4 *&#xA;                         pow (cosh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         pow (cos&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) * omega2) -&#xA;                0.708981540362208e2 * cosh (0.354490770181104e2 *&#xA;                            sqrt (omega2)) *&#xA;                sin (0.354490770181104e2 * sqrt (omega2)) /&#xA;                (0.348294222590750e3 *&#xA;                 cosh (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 sinh (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 pow (sin (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * sqrt (omega2) -&#xA;                 0.348294222590750e3 * cos (0.354490770181104e2 *&#xA;                            sqrt (omega2)) *&#xA;                 pow (sinh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * sin (0.354490770181104e2 *&#xA;                        sqrt (omega2)) *&#xA;                 sqrt (omega2) +&#xA;                 0.348294222590750e3 *&#xA;                 pow (cosh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * cos (0.354490770181104e2 *&#xA;                        sqrt (omega2)) *&#xA;                 sin (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 sqrt (omega2) +&#xA;                 0.348294222590750e3 * cosh (0.354490770181104e2 *&#xA;                             sqrt (omega2)) *&#xA;                 pow (cos (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * sinh (0.354490770181104e2 *&#xA;                         sqrt (omega2)) *&#xA;                 sqrt (omega2) +&#xA;                 0.120668159897616e2 *&#xA;                 pow (cosh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * pow (sin (0.354490770181104e2 *&#xA;                             sqrt (omega2)),&#xA;                        0.2e1) +&#xA;                 0.120668159897616e2 *&#xA;                 pow (cos (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * pow (sinh (0.354490770181104e2 *&#xA;                              sqrt (omega2)),&#xA;                        0.2e1) +&#xA;                 0.502654824574368e4 *&#xA;                 pow (sinh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * pow (sin (0.354490770181104e2 *&#xA;                             sqrt (omega2)),&#xA;                        0.2e1) * omega2 +&#xA;                 0.502654824574368e4 *&#xA;                 pow (cosh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * pow (cos (0.354490770181104e2 *&#xA;                             sqrt (omega2)),&#xA;                        0.2e1) * omega2) *&#xA;                sqrt (omega2) * (-0.209935630488671e8 *&#xA;                         sinh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         sin (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sqrt (omega2) -&#xA;                         0.727331795502673e6 *&#xA;                         cosh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         sin (0.354490770181104e2 *&#xA;                          sqrt (omega2)) +&#xA;                         0.727331795502673e6 *&#xA;                         cos (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sinh (0.354490770181104e2 *&#xA;                           sqrt (omega2))) -&#xA;                0.708981540362208e2 * cos (0.354490770181104e2 *&#xA;                               sqrt (omega2)) *&#xA;                sinh (0.354490770181104e2 * sqrt (omega2)) *&#xA;                (-0.727331795502673e6 *&#xA;                 cosh (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 sin (0.354490770181104e2 * sqrt (omega2)) -&#xA;                 0.209935630488671e8 * cosh (0.354490770181104e2 *&#xA;                             sqrt (omega2)) *&#xA;                 cos (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 sqrt (omega2) -&#xA;                 0.727331795502673e6 * cos (0.354490770181104e2 *&#xA;                            sqrt (omega2)) *&#xA;                 sinh (0.354490770181104e2 * sqrt (omega2))) *&#xA;                sqrt (omega2) / (0.348294222590750e3 *&#xA;                         cosh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         sinh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         pow (sin&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) * sqrt (omega2) -&#xA;                         0.348294222590750e3 *&#xA;                         cos (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         pow (sinh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         sin (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sqrt (omega2) +&#xA;                         0.348294222590750e3 *&#xA;                         pow (cosh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         cos (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sin (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sqrt (omega2) +&#xA;                         0.348294222590750e3 *&#xA;                         cosh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         pow (cos&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         sinh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         sqrt (omega2) +&#xA;                         0.120668159897616e2 *&#xA;                         pow (cosh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         pow (sin&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) +&#xA;                         0.120668159897616e2 *&#xA;                         pow (cos&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         pow (sinh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) +&#xA;                         0.502654824574368e4 *&#xA;                         pow (sinh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         pow (sin&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) * omega2 +&#xA;                         0.502654824574368e4 *&#xA;                         pow (cosh&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) *&#xA;                         pow (cos&#xA;                          (0.354490770181104e2 *&#xA;                           sqrt (omega2)),&#xA;                          0.2e1) * omega2) +&#xA;                0.708981540362208e2 * cos (0.354490770181104e2 *&#xA;                               sqrt (omega2)) *&#xA;                sinh (0.354490770181104e2 * sqrt (omega2)) /&#xA;                (0.348294222590750e3 *&#xA;                 cosh (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 sinh (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 pow (sin (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * sqrt (omega2) -&#xA;                 0.348294222590750e3 * cos (0.354490770181104e2 *&#xA;                            sqrt (omega2)) *&#xA;                 pow (sinh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * sin (0.354490770181104e2 *&#xA;                        sqrt (omega2)) *&#xA;                 sqrt (omega2) +&#xA;                 0.348294222590750e3 *&#xA;                 pow (cosh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * cos (0.354490770181104e2 *&#xA;                        sqrt (omega2)) *&#xA;                 sin (0.354490770181104e2 * sqrt (omega2)) *&#xA;                 sqrt (omega2) +&#xA;                 0.348294222590750e3 * cosh (0.354490770181104e2 *&#xA;                             sqrt (omega2)) *&#xA;                 pow (cos (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * sinh (0.354490770181104e2 *&#xA;                         sqrt (omega2)) *&#xA;                 sqrt (omega2) +&#xA;                 0.120668159897616e2 *&#xA;                 pow (cosh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * pow (sin (0.354490770181104e2 *&#xA;                             sqrt (omega2)),&#xA;                        0.2e1) +&#xA;                 0.120668159897616e2 *&#xA;                 pow (cos (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * pow (sinh (0.354490770181104e2 *&#xA;                              sqrt (omega2)),&#xA;                        0.2e1) +&#xA;                 0.502654824574368e4 *&#xA;                 pow (sinh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * pow (sin (0.354490770181104e2 *&#xA;                             sqrt (omega2)),&#xA;                        0.2e1) * omega2 +&#xA;                 0.502654824574368e4 *&#xA;                 pow (cosh (0.354490770181104e2 * sqrt (omega2)),&#xA;                  0.2e1) * pow (cos (0.354490770181104e2 *&#xA;                             sqrt (omega2)),&#xA;                        0.2e1) * omega2) *&#xA;                sqrt (omega2) * (-0.209935630488671e8 *&#xA;                         sinh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         sin (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sqrt (omega2) -&#xA;                         0.727331795502673e6 *&#xA;                         cosh (0.354490770181104e2 *&#xA;                           sqrt (omega2)) *&#xA;                         sin (0.354490770181104e2 *&#xA;                          sqrt (omega2)) +&#xA;                         0.727331795502673e6 *&#xA;                         cos (0.354490770181104e2 *&#xA;                          sqrt (omega2)) *&#xA;                         sinh (0.354490770181104e2 *&#xA;                           sqrt (omega2)))) *&#xA;    pow (omega2, -0.1e1 / 0.2e1) - 0.833338826540431e3;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24680" LastEditorUserId="24680" LastEditDate="2017-08-17T23:15:19.343" LastActivityDate="2017-08-17T23:15:19.343" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27368" PostTypeId="1" AcceptedAnswerId="27397" CreationDate="2017-07-09T09:05:20.813" Score="0" ViewCount="1252" Body="&lt;p&gt;I'm playing around with dynamic programming and need to calculate a multidimensional integral $E[V(W)]$ where we assume $W$ has a log normal distribution. I was looking at the following example in this &lt;a href=&quot;https://www.google.ch/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0ahUKEwi2ppD44fvUAhXlAMAKHQJaC7UQFggzMAA&amp;amp;url=http%3A%2F%2Fwww.zccfe.uzh.ch%2Fstatic%2Ffiles%2FCai_Judd_Dynamic_Programming.pdf&amp;amp;usg=AFQjCNE_N5vIj9Mu8o3mvknL8V892YNoQA&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf&lt;/a&gt;, section 9.4.3 on page 83.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give some background (I summarize from the section): The example is from economics and is about asset allocation. Assume $R$ is a return vector of dimension $n$ and is log-normally distrïbuted, i.e. $\log(R) = (\log(R_1),\dots, \log(R_n))$ has a multivariate normal distribution with given mean and covariance matrix, i.e. $\log(R)\sim\mathcal{N}((\mu-\frac{\sigma^2}{2})\Delta t,(\Lambda \Sigma\Lambda)\Delta t)$. The exact structure is not that important. $\Sigma$ is the correlation matrix and $\Lambda$ is simple a diagonal matrix with the standard deviation $\sigma_1,\dots,\sigma_n$ on its diagonal. Using Choleski decomposition we can write $\Sigma = LL^T$. Then one has&#xA;$$\log(R_1) = (\mu_1-\frac{\sigma^2_1}{2})\Delta t + (L_{11}z_1)\sigma_1\sqrt{\Delta t}$$&#xA;$$\log(R_2) = (\mu_2-\frac{\sigma^2_2}{2})\Delta t + (L_{21}z_1+ L_{22}z_2)\sigma_2\sqrt{\Delta t}$$&#xA;and so on, where $z_i$ are independent normal distribution. So that we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R_i=\exp{((\mu_i-\frac{\sigma^2_i}{2})\Delta t+\sigma_i\sqrt{\Delta t}\sum_{j=1}^iL_{ij}z_j)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let for simplicity no $\Delta t=1$ then we are interested in the quantity&#xA;$$ W_{t+1}= W_t(R_f(1-e^Tx_t) + \sum_{i=1}^n\exp{((\mu_i-\frac{\sigma_i^2}{2})+\sigma^2\sum_{j=1}^iL_{ij}z_j)}x_{ti}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e$ is a $n$ dimensional vector of $1$ and $x_t$ is some $n$ dimensional vector. For a given function $V$ the conditional expectatino of $V(W_{t+1})$ given $W_t, x_t$ can be calculated using Gauss Hermite quadrature &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{k_1,\dots,k_n=1}^m w_{k_1}\cdot\cdot\cdot w_{k_n} V\left(W_t(R_f(1-e^Tx_t) + \sum_{i=1}^n\exp{((\mu_i-\frac{\sigma_i^2}{2})+\sigma^2\sum_{j=1}^iL_{ij}q_{k_j})}x_{ti})\right)&#xA;$$ &#xA;where $w_{k_i}$ are the Gauss-Hermite weights and $q_i$ the corresponding nodes. My question is how can the above sum of sums be efficiently implemented in python? The exponential part can be precalculated using a cummulative sum if I'm not wrong.&lt;/p&gt;&#xA;" OwnerUserId="21291" LastEditorUserId="21291" LastEditDate="2017-07-13T12:28:04.220" LastActivityDate="2017-07-13T13:43:29.630" Title="How can this multidimensional integral be efficiently implemented in python using Gauss-Hermite quadrature" Tags="|python|numerics|integration|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27369" PostTypeId="2" ParentId="10466" CreationDate="2017-07-09T10:52:31.323" Score="2" Body="&lt;p&gt;As of today, there is no officially released functionality in OpenFOAM versions that converts VTK data to OpenFOAM grid data. Yet, a heuristically written python script was provided for such purposes: &lt;a href=&quot;https://github.com/edwardsp/VTKToFoam&quot; rel=&quot;nofollow noreferrer&quot;&gt;herein&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21325" LastActivityDate="2017-07-09T10:52:31.323" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27370" PostTypeId="1" CreationDate="2017-07-09T12:40:30.137" Score="1" ViewCount="1450" Body="&lt;p&gt;The question is in the context of iterative numerical solution of large PDE systems with Finite Differences or Finite Elements:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stating the Poisson equation with Neumann boundary conditions will lead to a singular system because it is invariant when adding a constant function. An often proposed option is to just &quot;pin&quot; the system to a fixed value in one point, which leads to a well-defined system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In my case, I'm rather interested in a solution where the constant part vanishes, so &quot;pinning&quot; a certain point does not help.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, I'd like to use an iterative solver, and those seem to be significantly unhappy with a singular system. In the PETSc documentation (I may or may not use PETSc then) I found the option to set a null space of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So how should I deal with that singluar system?&lt;/strong&gt; I'm interested in some theoretical views as well as practical solutions using linear solver libraries.&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-07-10T17:46:19.720" Title="Linear solvers: How to deal with a singular system? (Poisson equation with Neumann boundary conditions)" Tags="|pde|linear-solver|iterative-method|petsc|poisson|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27371" PostTypeId="2" ParentId="27351" CreationDate="2017-07-09T12:41:44.483" Score="7" Body="&lt;p&gt;There is a neat trick I have recently learned from &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/nla.2025/full&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt;. You start doing rank-revealing QR, and stop after the first $k$ Householder reflections, when you have a matrix of the form&#xA;$$&#xA;\begin{bmatrix}&#xA;R_1 &amp;amp; R_{12}\\&#xA;0 &amp;amp; R_{22}&#xA;\end{bmatrix},&#xA;$$&#xA;with $R_1$ triangular of size $k\times k$, and $R_{22}$ typically not triangular (since we stopped after the first $k$ iterations of our main loop). At this point, you check if $\|R_{22}\| \leq \varepsilon$: if it holds, then $A$ is at distance at most $\varepsilon$ from a matrix of rank $\leq k$; otherwise it shouldn't be (barring numerical errors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This procedure costs $O(n^2k)$ for a dense $n\times n$ matrix.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2017-07-09T12:48:59.113" LastActivityDate="2017-07-09T12:48:59.113" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27372" PostTypeId="2" ParentId="27370" CreationDate="2017-07-09T13:53:27.653" Score="3" Body="&lt;p&gt;Krylov solvers have no problem converging if the system is consistent, i.e., if the right-hand side is in the image of the system matrix $A$; see, e.g., &lt;em&gt;Iterative Krylov Methods for Large Linear Systems&lt;/em&gt; by van der Vorst. PETSc's CG solver should work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can even use a multigrid preconditioner if you make sure that the coarse solver preserves the null space. PETSc's default ILU solver is &lt;em&gt;not&lt;/em&gt; sufficient, better use Jacobi; see &lt;a href=&quot;http://lists.mcs.anl.gov/pipermail/petsc-users/2012-February/012139.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lists.mcs.anl.gov/pipermail/petsc-users/2012-February/012139.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2017-07-09T13:53:27.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27373" PostTypeId="2" ParentId="27370" CreationDate="2017-07-09T18:19:39.533" Score="2" Body="&lt;p&gt;The problem I imagine you are trying to solve is a diffusion equation with source term with homogeneous Neumann BC. To do so it must be well posed in order to obtain physical and good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resultant system leads to a singular differential operator matrix $A$, once the BC have been applied. This is so, as you already mentioned, because the following problem (for example I use a linear equation because for the nonlinear case the same would apply ):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\{\begin{array}{ll}%&#xA;-\partial_x^2u=f &amp;amp; x\in(0,1)\\&#xA;\partial_xu=0 &amp;amp; x=0,x=1&#xA;\end{array}\right. \tag{*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is invariant under the transformation $u\to u+constant$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And therefore some value for $u$ must be specify $\textbf{inside}$ the domain. Many people impose a Dirichlet BC instead one of the Neuman BC causing the solution to differ from the actual one, which now solves:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\{\begin{array}{ll}%&#xA;-\partial_x^2u=f &amp;amp; x\in(0,1)\\&#xA;u=0 &amp;amp; x=0\\&#xA;\partial_xu=0 &amp;amp; x=1\\&#xA;\end{array}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem $(*)$ is well posed if $\int_{0}^{1}{f\,dx}=0$, in fact the source function I propose:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f=cos(2\pi x)$$&#xA; fits well for the well-posedness of $(*)$, only some reference value for $u$ is required to manage the uniqueness, for example we require that $u(1/2)=1$  $\textbf{without any loss of generality}$. I put these words in bold due to your requirement that your constant must be set to $0$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 100; % #nodes&#xA;uref = 1; %Ref value for u for centre node&#xA;deltax = 1/(N-1); % step&#xA;x = (0:deltax:1)';&#xA;f = cos(x*2*pi); %distributed source&#xA;&#xA;&#xA;b = zeros(N,1); % Source vector&#xA;A=zeros(N,N); % Stiffness matrix&#xA;&#xA;for i = 2 : N-1&#xA;    A(i,i-1:i+1) = -1/deltax^2*[1, -2, 1];&#xA;    b(i) = f(i);&#xA;end&#xA;&#xA;%Neumann BC&#xA;A(1,1:2) = 2/deltax^2*[1,-1];&#xA;b(1) = f(1);&#xA;A(N,N-1:N) = 2/deltax^2*[-1, 1];&#xA;b(N) = f(N);&#xA;&#xA;%Uniqueness condition e.g. central node&#xA;idx = floor(N/2);&#xA;A(idx,:) = 0;&#xA;A(idx,idx) = 1;&#xA;b(idx) = uref;&#xA;&#xA;% System solution&#xA;u = A\b;&#xA;&#xA;% Verify the diff equation&#xA;ddu = zeros(N,1);&#xA;for i = 2 : N-1&#xA;     ddu(i) = -(u(i+1)-2*u(i)+u(i-1))/(deltax^2);&#xA;end&#xA;ddu(1) = f(1);&#xA;ddu(N) = f(N);&#xA;&#xA;%Solution residual&#xA;res = norm(ddu-f);&#xA;&#xA;%Plots&#xA;plot(x,u,'r','linewidth',2); %Solution&#xA;figure&#xA;plot(x,ddu-f,'k','linewidth',1) % Error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above code produces for $(*)$ a solution&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ew4GF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ew4GF.png&quot; alt=&quot;Solution of $(*)$&quot;&gt;&lt;/a&gt;&#xA;and the pointwise residual given by $e=\partial_x^2 u+f$ is given below for reference:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/q5Wvk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q5Wvk.png&quot; alt=&quot;Residual for $(*)$&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You now are free to choose the arbitrary constant (forget the fact that $u(1/2) = 1$) to your problem, just add it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formally the uniqueness condition that you mention is simply imposed by the restriction:&#xA;$$\int_{0}^{1}{u\,dx}=0$$&#xA;which constrains $u$ in a way that the transformation $u\to u+constant$ is not valid any more, and therefore the constant must be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This condition would be imposed in an analogous manner when solving the system:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%Uniqueness condition &#xA;idx = N-1;&#xA;A(idx,:) = 1;&#xA;b(idx) = 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or at the end, when $u_{calc}$ has been obtained, you do:&#xA;$$u=u_{calc}-\overline{u}_{calc}=u_{calc}-\int_{0}^{1}{u_{calc}\,dx}$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-07-10T17:46:19.720" LastActivityDate="2017-07-10T17:46:19.720" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27374" PostTypeId="2" ParentId="27267" CreationDate="2017-07-10T09:40:53.010" Score="0" Body="&lt;p&gt;Manage parameters with &lt;code&gt;switch&lt;/code&gt; statements: each &lt;code&gt;case&lt;/code&gt; is a self-contained set of parameters. This doesn't address different expressions/codes etc, but for Barba's 12 steps, it's enough so far. &lt;strong&gt;EDIT&lt;/strong&gt; and arbitrary boolean switches for different parts of the code (e.g. different boundary conditions; switching off a stage such as advection). Messy but flexible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Add parameters to output plots: so the input and interpretable results are documented together, in an image file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Automatically timestamping the plot images, and outputting to a tmp directory, so I can go back to them if need be, (and copy and rename any I want to keep). This is the journalling idea; and, I guess is a form of &quot;logging&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using &lt;code&gt;gnuplot&lt;/code&gt; for plotting, and send in the parameters and filename as &quot;call&quot; arguments (this line is auto-generated from my simulation code):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gnuplot -c myscript filename stringofparameternamesandvalues&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These argumemts are available as ARG1 and ARG2 in the script. In the script, I set the output file and title to them. One can also put them in a label, and position it (and use &lt;code&gt;\n&lt;/code&gt; for multiline), but I'm concerned it will sometimes overwrite part of the plot - using the title prevents that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thought: a parameter object. Using reflection, one can automatically serialize to JSON (or XML), or output to a logger (as @GoGoLander suggested). An advantage is if you add a new parameter to the object, you don't have to remember to add it the output code (the serializer automatically output all the object's fields). Though this doesn't capture expressions/code. &lt;strong&gt;EDIT&lt;/strong&gt; a parameter object is verbose because must add prefix (e.g. &lt;code&gt;params.dx&lt;/code&gt;). Instead, use fields on the code's own instance object as parameters, and can use them directly (e.g. &lt;code&gt;dx&lt;/code&gt;). Here's a way to auto-output them in java (assuming no other fields):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import java.lang.reflect.Field;&#xA;...&#xA;    public String getParams() {&#xA;      String a = &quot;&quot;;&#xA;      try {&#xA;        for (Field f : this.getClass().getDeclaredFields()) {&#xA;          f.setAccessible(true);  // so needn't make fields public&#xA;          a += f.getName() + &quot;=&quot;+ f.get(this) +&quot;, &quot;;&#xA;        }&#xA;      } catch (Exception e) { e.printStackTrace(); }&#xA;      return a;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think &quot;2. in a class&quot; is the right approach for compiled languages, because it can store  expressions and code, and keeps experimental config separate from the main code.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-07-19T04:36:11.207" LastActivityDate="2017-07-19T04:36:11.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27376" PostTypeId="1" CreationDate="2017-07-10T13:13:52.207" Score="1" ViewCount="377" Body="&lt;p&gt;I have following system of coupled Partial differential equations.&#xA;How can I solve the system by Maple?&#xA;\begin{align} 		m_1\frac{\partial^2 u_1}{\partial t^2}+A_1\frac{\partial ^4u_1(x,t)}{\partial x^4}+k(u_1-u_2)=F_1(t) \delta(x-x_1), 	\end{align}&#xA;\begin{align} 		 m_2\frac{\partial^2 u_2}{\partial t^2}-A_2\frac{\partial ^2u_2(x,t)}{\partial x^2}+k(u_2-u_1)=F_2(t)\delta(x-x_2). 	\end{align}&#xA;&lt;strong&gt;Code:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PDE1:=m1*diff(u1(x,t),t$2)+A1*diff(u1(x,t),x$4)+k*(u1(x,t)-u2(x,t))=F1(t)*delta(x-x1);&#xA;PDE2:=m2*diff(u2(x,t),t$2)-A2*diff(u2(x,t),x$2)+k*(u2(x,t)-u1(x,t))=F2(t)*delta(x-x2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In here,  $A_i, m_i, x_i$ and $k$ are constants where $i=1,2$ and $\delta$ is Dirac Delta function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Boundary conditions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_1(0,t)=\frac{\partial^2 u_1}{\partial x^2}(0,t)=u_1(l,t)=\frac{\partial^2 u_1}{\partial x^2}(l,t)=0$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_2(0,t)=u_2(l,t)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Initial conditions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_i(x,0)=w_{i0}(x),$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u_i}{\partial x}(x,0)=y_{i0}(x)$ for $i=1,2.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $F_1(t)$ and $F_2(t)$ are unspecified (ungiven) functions, solutions $u_1,u_2$ which we seek will be depended on  $F_1(t)$ and $F_2(t)$. &lt;/p&gt;&#xA;" OwnerUserId="23832" LastEditorUserId="23832" LastEditDate="2017-07-12T12:57:57.893" LastActivityDate="2017-07-12T21:41:02.263" Title="solution of system of coupled partial differential equations" Tags="|maple|differential-equations|" AnswerCount="1" CommentCount="5" ClosedDate="2017-07-18T20:16:41.113" ContentLicense="CC BY-SA 3.0" />
  <row Id="27377" PostTypeId="1" CreationDate="2017-07-10T15:16:54.853" Score="5" ViewCount="306" Body="&lt;p&gt;I need an efficient way to numerically find the first $n$ positive roots $\lambda_n$ of the transcendental equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \dfrac{J_0 (\lambda_n r) Y_1 (\lambda_n) - J_1 (\lambda_n) Y_0 (\lambda_n r)}{J_0 (\lambda_n r) Y_0 (\lambda_n) - J_0 (\lambda_n) Y_0 (\lambda_n r)} = \dfrac{\delta}{\lambda_n}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\delta$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/6924-newtzero&quot; rel=&quot;noreferrer&quot;&gt;newtzero&lt;/a&gt; and &lt;a href=&quot;http://www.chebfun.org/&quot; rel=&quot;noreferrer&quot;&gt;chebfun&lt;/a&gt; in MATLAB. They are both taking longer than I'd like. The roots are needed for the solution of a PDE (transient heat transfer in one spatial dimension) with time varying boundary conditions using eigenfunction expansions. If anyone knows of a technique to determine the roots even a little faster, it would be much appreciated. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="11493" LastEditorUserId="11493" LastEditDate="2017-07-10T15:23:47.390" LastActivityDate="2017-07-10T15:23:47.390" Title="Roots of transcendental equation involving bessel functions" Tags="|rootfinding|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="27378" PostTypeId="1" CreationDate="2017-07-10T20:45:59.790" Score="1" ViewCount="105" Body="&lt;p&gt;The issue I have is having to compute the derivative (in real time) of the solution produced by ode45 within the events function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some pseudo-code to explain what I'm mean is,&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;function dx = myfunc(t,x,xevent)&#xA;persistent xevent&#xA;% xevent is the solution at the event&#xA;dx(1) = x(2);&#xA;dx(2) = complicated function of x(1),x(2), and xevent;&#xA;end&#xA;&#xA;function [value,isterminal,direction] = myeventfcn(t,x)&#xA;position = function of x(1), x(2), and dx(2); &#xA;isterminal = 1;&#xA;direction = 0;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that if I didn't need to use the solution at the event within &lt;code&gt;myfunc&lt;/code&gt; I could just compute &lt;code&gt;dx=myfunc(t,x)&lt;/code&gt; within my event function and use &lt;code&gt;dx(2)&lt;/code&gt;, yet since &lt;code&gt;xevent&lt;/code&gt; is used within &lt;code&gt;myfunc&lt;/code&gt; I can't input &lt;code&gt;xevent&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there is a way of inputting constant parameters within the event function, but since it's the solution at the event location, which also changes, I'm not sure how to go about doing that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My work around to this is to approximate &lt;code&gt;dx(2)&lt;/code&gt; using the solution &lt;code&gt;x&lt;/code&gt;. What I wanted to know is if it will be satisfactory to just use a finite difference approximation here, using a small fixed step size relative to the step size od45 takes, which is a variable step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help!&lt;/p&gt;&#xA;" OwnerUserId="22543" LastEditorUserId="24680" LastEditDate="2017-07-14T06:52:24.893" LastActivityDate="2017-07-14T06:52:24.893" Title="Approximating derivative of numerical solution (using ode45) within event function" Tags="|matlab|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="27381" PostTypeId="1" AcceptedAnswerId="27387" CreationDate="2017-07-11T00:05:38.887" Score="0" ViewCount="87" Body="&lt;p&gt;I read the way of storing the five or seven point laplace matrix for some poisson problem but I don't understand how can i multiply, add and subtract this stored sparse  matrix by a vector or another matrix? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way says that we just store the diagonal entries which are the coefficients of the i,j,k unknown in adiag matrix and the coefficients of the i+,j,k neighbours will be stored in another array ax and for the j neighbours as well. &lt;/p&gt;&#xA;" OwnerUserId="23541" LastActivityDate="2017-07-11T23:40:24.760" Title="How to do matrix operations with this way of storing sparse matrices" Tags="|sparse-matrix|matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27382" PostTypeId="2" ParentId="27276" CreationDate="2017-07-11T01:55:33.100" Score="4" Body="&lt;p&gt;If you are adamant on using MATLAB, chapter 14 of the following book walks through a 2D IPDG Poisson problem using piecewise linear basis functions on triangles: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.springer.com/us/book/9783642332869&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Finite Element Method: Theory, Implementation, and Applications&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MATLAB code used for the most examples in the book is freely available on github:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Jumziey/FEM/tree/master/MatlabCodeFEMBook&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Jumziey/FEM/tree/master/MatlabCodeFEMBook&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The IPDG specific code is in the script &quot;dG1PoissonSolver2D.m&#xA;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in the comments, it might be worth while to look into a &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_finite_element_software_packages&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite element library&lt;/a&gt;, as you can probably have a much easier time extending your problem (3D, different elements, adaptivity, high order, and so on) than in your own MATLAB codes.&lt;/p&gt;&#xA;" OwnerUserId="11604" LastActivityDate="2017-07-11T01:55:33.100" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27386" PostTypeId="1" CreationDate="2017-07-11T17:13:53.633" Score="1" ViewCount="103" Body="&lt;p&gt;For an unconfined aquifer we have this PDE for the water table position( of course after somehow making the original Boussinesq equation linearized ):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial^2(h^2)}{\partial x^2} + \frac{\partial^2(h^2)}{\partial y^2} = \frac{2s}{k} \frac{\partial h}{\partial t} - \frac{2N}{k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $N$, $k$ and $s$ are constants . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you tell me what type mathematically it is? In respect to $h^2$ it seems elliptic, but physically it is not. Can we treat it and solve it like 2D heat equation? I need to solve it numerically via finite difference. Which FD scheme would you recommend?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried it with the simple explicit  and it works OK and I'm aware of instability restrictions. but with methods like , ADE  (alternating-direction explicit), because the second derivatives are in power 2  , it will lead to quadratic equation for each node in each time.&lt;/p&gt;&#xA;" OwnerUserId="24840" LastEditorUserId="24840" LastEditDate="2017-07-12T13:19:59.373" LastActivityDate="2017-07-12T13:19:59.373" Title="Finite difference scheme for unconfined aquifer equation" Tags="|pde|nonlinear-equations|computational-physics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27387" PostTypeId="2" ParentId="27381" CreationDate="2017-07-11T23:40:24.760" Score="2" Body="&lt;p&gt;Think about it this way: in your matrix, you know exactly where the entries are nonzero (and you store what these values are), and everything else is zero. If you just write down the usual matrix-vector product,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i=1...N)&#xA;  for (j=1...N)&#xA;    output[i] += matrix[i,j] * input[j];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then you realize that most of these additions just add zeros because the matrix entry is zero. You can avoid all of those unnecessary operations if you remember where your nonzero entries are, and then only do those products and additions that are necessary. This is in essence what the book is suggesting: have a way to know where the nonzero entries of the matrix are, store what their values are, and then shrink the loop above to only those operations that are necessary.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-07-11T23:40:24.760" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27389" PostTypeId="1" CreationDate="2017-07-12T04:12:27.013" Score="1" ViewCount="145" Body="&lt;p&gt;I have a small 2D system I'm trying to model using a non-linear extension of Darcy's law for fluid flow in porous media. I'm primarily interested in the local flow velocity, not necessarily the pressure, that's later used in the real model I'm working on:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original equation known as Darcy's Law used in ground water modeling is essentially Poisson's equation, where $\vec{q}$ is the fluid velocity, $\mu$ and $\kappa$ are constants that together describe the hydraulic conductivity of the porous medium, $p$ is local hydrostatic head and f is a source:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ -\frac{\mu}{\kappa} \nabla p = \vec{q} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla \cdot ( \vec{q} ) = f = -\frac{\mu}{\kappa} \nabla^2 p $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The non-linear equation I want to solve instead is the following, known as the Darcy–Forchheimer law. As you can see it's essentially a polynomial instead of a linear model like Darcy's law:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac {\partial p}{\partial x} = \frac {\mu }{\kappa} \vec{q} - \frac {\rho }{\kappa _{1}}\vec{q}^{2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've picked up on some authors [1] that go through the calculus and have figured out a way to represent $\vec{q}$ as a function of $p$ by calculating the inverse of the flow/pressure equation, and I see somewhat how they arrive at their methodology, but what I'm interested in is actually calculating the flux ($\vec{q}$) itself, which I use elsewhere in the model I'm developing - I don't have much of a use for $p$ besides in the initial problem setup with some simple Dirichlet boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Somewhat related, I've also read that for some numerical methods such as DG FEM, diffusion dominated problems can very unstable due to the non-directional flux. In this paper [2], the authors investigate some 'equation splitting' methods where they solve one equation for the state variable, and another for the flux, essentially the first two equations I showed above. This apparently relieves some of the instability issues caused by diffusion&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Looking at these two, it looks like I have two options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use the methodology of [1] to calculate the pressure throughout the system, and then calculate the gradient of the hydrostatic head to generate the local velocity, Or...&lt;/li&gt;&#xA;&lt;li&gt;Split my equation into two equations and solve them together. I end up having the solve two equations at once, but I need to have the velocity regardless so that doesn't seem so bad. Other than that I know nothing about this method.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I don't know how to implement the equation splitting method as I've never done it before, so I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this even a good idea? Is splitting the equations up into double the number of equations disadvantageous for any other reason than doubling the number of state equations to solve?&lt;/li&gt;&#xA;&lt;li&gt;Is there a source that goes into the actual implementation of equation splitting for (preferably) the finite element or finite difference methods. I am thoroughly intimidated by any DG theory I come across, and I'm hoping there is an easier way to solve this problem&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://arxiv.org/pdf/1508.00294.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1508.00294.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://www.cs.utah.edu/~kirby/Publications/Kirby-11.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.utah.edu/~kirby/Publications/Kirby-11.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19791" LastActivityDate="2017-07-12T04:12:27.013" Title="How to formulate Poisson's equation into flux eqution" Tags="|poisson|advection-diffusion|operator-splitting|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27390" PostTypeId="1" CreationDate="2017-07-12T13:48:51.333" Score="3" ViewCount="210" Body="&lt;p&gt;I have some Python code containing a couple of &lt;code&gt;for&lt;/code&gt; loops, which I would like to optimize by using low-level functions. My first approach is to use &lt;code&gt;numpy.einsum()&lt;/code&gt;, but the results don't match the &lt;code&gt;for&lt;/code&gt;-loop implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;## first term&#xA;T11 = 0.125 * numpy.einsum('abcd,efab,cdef,abcd,efcd',&#xA;    g[:Nelec,:Nelec,Nelec:,Nelec:], # &amp;lt;ab||rs&amp;gt; -&amp;gt; abcd&#xA;    g[:Nelec,:Nelec,:Nelec,:Nelec], # &amp;lt;cd||ab&amp;gt; -&amp;gt; efab&#xA;    g[Nelec:,Nelec:,:Nelec,:Nelec], # &amp;lt;rs||cd&amp;gt; -&amp;gt; cdef&#xA;    D,                              # e_a + e_b - e_r - e_s -&amp;gt; abcd&#xA;    D,                              # e_c + e_d - e_r - e_s -&amp;gt; efcd&#xA;)&#xA;&#xA;T12 = 0&#xA;for a in range(Nelec):&#xA;    for b in range(Nelec):&#xA;&#xA;        for c in range(Nelec):&#xA;            for d in range(Nelec):&#xA;&#xA;                for r in range(Nvirt):&#xA;                    for s in range(Nvirt):&#xA;&#xA;                        # nominator&#xA;                        nom  = g[a,b,Nelec+r,Nelec+s]&#xA;                        nom *= g[c,d,a,b]&#xA;                        nom *= g[Nelec+r,Nelec+s,c,d]&#xA;&#xA;                        # denominator&#xA;                        denom = D[a,b,r,s] * D[c,d,r,s]&#xA;&#xA;                        T12 += nom*denom&#xA;&#xA;T12 *= 0.125&#xA;&#xA;## second term&#xA;T21 = 0.125 * numpy.einsum('abcd,cdef,efab,abcd,abef',&#xA;    g[:Nelec,:Nelec,Nelec:,Nelec:], # &amp;lt;ab||rs&amp;gt; -&amp;gt; abcd&#xA;    g[Nelec:,Nelec:,Nelec:,Nelec:], # &amp;lt;rs||tu&amp;gt; -&amp;gt; cdef&#xA;    g[Nelec:,Nelec:,:Nelec,:Nelec], # &amp;lt;tu||ab&amp;gt; -&amp;gt; efab&#xA;    D,                              # e_a + e_b - e_r - e_s -&amp;gt; abcd&#xA;    D,                              # e_a + e_b - e_t - e_u -&amp;gt; abef&#xA;)&#xA;&#xA;T22 = 0&#xA;for a in range(Nelec):&#xA;    for b in range(Nelec):&#xA;&#xA;        for r in range(Nvirt):&#xA;            for s in range(Nvirt):&#xA;&#xA;                for t in range(Nvirt):&#xA;                    for u in range(Nvirt):&#xA;&#xA;                        # nominator&#xA;                        nom  = g[a,b,Nelec+r,Nelec+s]&#xA;                        nom *= g[Nelec+r,Nelec+s,Nelec+t,Nelec+u]&#xA;                        nom *= g[Nelec+t,Nelec+u,a,b]&#xA;&#xA;                        # denominator&#xA;                        denom = D[a,b,r,s] * D[a,b,t,u]&#xA;&#xA;                        T22 += nom*denom&#xA;&#xA;T22 *= 0.125&#xA;&#xA;## third term&#xA;T31 = numpy.einsum('abcd,edfb,cfae,abcd,aecf',&#xA;    g[:Nelec,:Nelec,Nelec:,Nelec:], # &amp;lt;ab||rs&amp;gt; -&amp;gt; abcd&#xA;    g[:Nelec,Nelec:,Nelec:,:Nelec], # &amp;lt;cs||tb&amp;gt; -&amp;gt; edfb&#xA;    g[Nelec:,Nelec:,:Nelec,:Nelec], # &amp;lt;rt||ac&amp;gt; -&amp;gt; cfae&#xA;    D,                              # e_a + e_b - e_r - e_s -&amp;gt; abcd&#xA;    D,                              # e_a + e_c - e_r - e_t -&amp;gt; aecf&#xA;)&#xA;&#xA;T32 = 0&#xA;for a in range(Nelec):&#xA;    for b in range(Nelec):&#xA;        for c in range(Nelec):&#xA;            for r in range(Nvirt):&#xA;                for s in range(Nvirt):&#xA;                    for t in range(Nvirt):&#xA;&#xA;                        # nominator&#xA;                        nom  = g[a,b,Nelec+r,Nelec+s]&#xA;                        nom *= g[c,Nelec+s,Nelec+t,b]&#xA;                        nom *= g[Nelec+r,Nelec+t,a,c]&#xA;&#xA;                        # denominator&#xA;                        denom = D[a,b,r,s] * D[a,c,r,t]&#xA;&#xA;                        T32 += nom*denom&#xA;&#xA;print(T11, T12)&#xA;print(T21, T22)&#xA;print(T31, T32)&#xA;print(T11+T21+T31, T12+T22+T32)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For my test case I get the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 0.016038311274  0.016038311274&#xA; 0.014256694984  0.014256694984&#xA;-0.044483006105 -0.044483006105&#xA;-0.014187999845 -0.014187999845&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first column contains the &lt;code&gt;einsum()&lt;/code&gt; results, the second column is for the correctly working &lt;code&gt;for&lt;/code&gt; loop implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;del&gt;So my question is: Why does &lt;code&gt;einsum()&lt;/code&gt; not give the same results (except for the second term $T_2$)? Is there some error in my &lt;code&gt;einsum()&lt;/code&gt; implementation or is there a conceptual problem here?&lt;/del&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; After fixing one error in &lt;code&gt;T11&lt;/code&gt; and another error in &lt;code&gt;T31&lt;/code&gt; both methods yield same results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not necessarily  important, but the equations are for the &lt;strong&gt;Møller-Plesset perturbation theory third-order energy correction&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA; E_0^{(3)} = T_1 + T_2 + T_3 \\&#xA; T_1 = \frac{1}{8} \sum_{abcdrs} \frac{\langle ab||rs\rangle\langle cd||ab\rangle\langle rs||cd\rangle}{(\epsilon_a+\epsilon_b-\epsilon_r-\epsilon_s)(\epsilon_c+\epsilon_d-\epsilon_r-\epsilon_s)} \\&#xA; T_2 = \frac{1}{8} \sum_{abrstu} \frac{\langle ab||rs\rangle\langle rs||tu\rangle\langle tu||ab\rangle}{(\epsilon_a+\epsilon_b-\epsilon_r-\epsilon_s)(\epsilon_c+\epsilon_d-\epsilon_t-\epsilon_u)} \\&#xA; T_3 = \sum_{abcrst} \frac{\langle ab||rs\rangle\langle cs||tb\rangle\langle rt||ac\rangle}{(\epsilon_a+\epsilon_b-\epsilon_r-\epsilon_s)(\epsilon_a+\epsilon_c-\epsilon_r-\epsilon_t)}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code I use:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;g[i,j,k,l]&lt;/code&gt; = $\langle ij||kl\rangle$ being the anti-symmetrized two electron repulsion integrals&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;D[a,b,r,s]&lt;/code&gt; = $\frac{1}{\epsilon_i+\epsilon_j-\epsilon_r-\epsilon_s}$ with $\epsilon_i$ being the energy of orbital $i$ &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Nelec&lt;/code&gt; for the number of electrons, which equals the number of occupied spin orbitals&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Norb&lt;/code&gt; for the total number of spin orbitals&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Nvirt = Norb - Nelec&lt;/code&gt; for the number of virtual (unoccupied) orbitals&lt;/li&gt;&#xA;&lt;li&gt;orbitals are ordered by energy, thus occupied first, then virtuals&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And orbital indices follow the convention:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;a,b,c,d&lt;/code&gt; running over occupied orbitals only&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;r,s,t,u&lt;/code&gt; running over virtual orbitals only&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;i,j,k,l&lt;/code&gt; running over all orbitals&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="24086" LastEditorUserId="24086" LastEditDate="2017-07-20T08:54:52.853" LastActivityDate="2017-07-20T08:54:52.853" Title="Optimizing for loops by using Einstein summations instead" Tags="|python|numpy|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27392" PostTypeId="1" AcceptedAnswerId="27399" CreationDate="2017-07-12T19:55:05.280" Score="0" ViewCount="107" Body="&lt;p&gt;$r$: cicle &lt;code&gt;C1&lt;/code&gt;'s radius&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w$,$h$: rectangle &lt;code&gt;R1&lt;/code&gt;'s edges: $x=w$, $y=h$, $x=0$, $y=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(w&amp;gt;2r, h&amp;gt;2r)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S(x,y)$: area of intersection of &lt;code&gt;C1&lt;/code&gt; and &lt;code&gt;R1&lt;/code&gt; when center of &lt;code&gt;C1&lt;/code&gt; is  at $(x, y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X$: follows uniform distributed in $[0,w]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Y$: follows uniform distributed in $[0,h]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X$ and $Y$ is independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is $E(S(X, Y))$, the expected value of area of intersection.&lt;/p&gt;&#xA;" OwnerUserId="24849" LastActivityDate="2017-07-13T14:18:12.357" Title="What is the expected value of area of intersection of a circle and a rectangle" Tags="|computational-geometry|probability|calculus|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27393" PostTypeId="2" ParentId="27376" CreationDate="2017-07-12T21:41:02.263" Score="0" Body="&lt;p&gt;If you replace &lt;code&gt;delta()&lt;/code&gt; with Maple synax for the Dirac delta: &lt;code&gt;Dirac()&lt;/code&gt; Maple's pdsolve given an error. But, not doing the replacement, it seems to make a little progress maybe:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pdsolve({PDE1, PDE2}, {u1(x, t), u2(x, t)});&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once I add the boundary conditions it stops making any progress though (and the same result if I replace your &lt;code&gt;delta&lt;/code&gt; with &lt;code&gt;Diract&lt;/code&gt;).  It just returns NULL.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pdsolve({PDE1, PDE2, u1(0, t) = 0, u1(1, t) = 0, u2(0, t) = 0, u2(1, t) = 0, &#xA;         (D[2](u1))(0, t) = 0, (D[2](u1))(1, t) = 0}, {u1(x, t), u2(x, t)})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suspect that you have too many symbolic constants and unknown functions for pdsolve to handle.  There may be other tricks you can play with options the pdsolve function, but its documentation is pretty impenetrable.&lt;/p&gt;&#xA;" OwnerUserId="24850" LastActivityDate="2017-07-12T21:41:02.263" CommentCount="0" CommunityOwnedDate="2017-07-12T21:41:02.263" ContentLicense="CC BY-SA 3.0" />
  <row Id="27395" PostTypeId="1" AcceptedAnswerId="27431" CreationDate="2017-07-13T06:04:14.510" Score="1" ViewCount="1461" Body="&lt;p&gt;Originally sieve of Eratosthenes requires a lot of memory. This algorithm is my attempt to limit the memory usage. In fact it requires ln(N) memory (for each found prime number we keep last crossed number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lDnVo.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lDnVo.gif&quot; alt=&quot;Sieve of Eratosthenes visualization&quot;&gt;&lt;/a&gt;&#xA;Amount of operation (sum and compare) is still limited by sequence N/2 + N/3 + N/5 + N/7 .... + N/Pk = O(N log log N).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithm is following: Suppose we have some found prime numbers (the described algorithm starts from 2 and 3) sorted. For each prime number we store highest number we crossed with the prime. We introduce a prime candidate number = last found prime + 1 (in fact the algorithm can skip even numbers so we can use last found prime + 2). Starting to check the candidate. For each found prime use the last crossed number and compare with the candidate. If the last crossed is less than the candidate increase the last crossed by the prime. If after next increasing candidate equals the last crossed then the candidate is not prime (can be divided by the current prime) and we should choose a new prime candidate (by increasing current candidate). And restart the algoritm. If we jumped over the candidate and last crossed is bigger we use next prime and do the same checks with the last crossed of next prime. If all the last crossed of each prime is bigger than candidate (jumped over the candidate) we found a new prime number. The found prime is added to the found primes sequence and last crossed is set to the prime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm's source on Java to express the logic.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import java.util.ArrayList;&#xA;import java.util.List;&#xA;&#xA;/**&#xA; * @author stanislav.lapitsky created 7/11/2017.&#xA; */&#xA;public class SieveEratosthenes {&#xA;    static class PrimePair {&#xA;        Integer prime;&#xA;        Integer lastCrossed;&#xA;&#xA;        PrimePair(Integer prime, Integer lastCrossed) {&#xA;            this.prime = prime;&#xA;            this.lastCrossed = lastCrossed;&#xA;        }&#xA;    }&#xA;&#xA;    private List&amp;lt;PrimePair&amp;gt; primes;&#xA;&#xA;    private SieveEratosthenes() {&#xA;        primes = new ArrayList&amp;lt;&amp;gt;();&#xA;        primes.add(new PrimePair(2, 2));&#xA;        primes.add(new PrimePair(3, 3));&#xA;    }&#xA;&#xA;    private void fillNPrimes(int n) {&#xA;        while (primes.size()&amp;lt;n) {&#xA;            addNextPrime();&#xA;        }&#xA;    }&#xA;&#xA;    private void addNextPrime() {&#xA;        int candidate = primes.get(primes.size()-1).prime + 2;&#xA;        for (int i = 1; i &amp;lt; primes.size(); i++) {&#xA;            PrimePair p = primes.get(i);&#xA;            while (p.lastCrossed &amp;lt; candidate) {&#xA;                p.lastCrossed += p.prime;&#xA;            }&#xA;            if (p.lastCrossed == candidate) {&#xA;                //restart&#xA;                candidate+=2;&#xA;                i=1;&#xA;            }&#xA;        }&#xA;        System.out.println(candidate);&#xA;        primes.add(new PrimePair(candidate, candidate));&#xA;    }&#xA;&#xA;    public static void main(String[] args) {&#xA;        SieveEratosthenes test = new SieveEratosthenes();&#xA;        test.fillNPrimes(1000);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The same on python&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;primes = [2, 3]&#xA;last_crossed = [2, 3]&#xA;&#xA;&#xA;def add_next_prime():&#xA;    candidate = primes[-1] + 2&#xA;    i = 0&#xA;    while i &amp;lt; len(primes):&#xA;        while last_crossed[i] &amp;lt; candidate:&#xA;            last_crossed[i] += primes[i]&#xA;        if last_crossed[i] == candidate:&#xA;            candidate += 2&#xA;            i = 0&#xA;        i += 1&#xA;&#xA;    primes.append(candidate)&#xA;    last_crossed.append(candidate)&#xA;&#xA;&#xA;def fill_primes(n):&#xA;    while len(primes) &amp;lt; n:&#xA;        add_next_prime()&#xA;&#xA;&#xA;fill_primes(1000)&#xA;print(primes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All the optimization based on wheel factorization is still can be applied when we choose next candidate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/StanislavLapitsky/prime&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is following - Is the algorithm above good enough or I reinvented wheel? Is my algorithm complexity calculation correct or not?&lt;/p&gt;&#xA;" OwnerUserId="24854" LastEditorUserId="24854" LastEditDate="2017-07-14T07:02:55.357" LastActivityDate="2017-07-17T19:08:21.697" Title="Sieve of Eratosthenes with minimized memory usage" Tags="|optimization|algorithms|" AnswerCount="1" CommentCount="2" ClosedDate="2017-07-18T20:17:06.120" ContentLicense="CC BY-SA 3.0" />
  <row Id="27397" PostTypeId="2" ParentId="27368" CreationDate="2017-07-13T13:43:29.630" Score="1" Body="&lt;p&gt;The summation over $q_{k_i}$ in $V$ is independent of the sums over the products $w_{k_1}w_{k_2}\cdots w_{k_n}$. So we can calculate $V$ first. To avoid double evaluations of same operations we can further decompose the equation:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Step 1: precalculate $\tilde L_{ij}$&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  \tilde L_{ij} = \sigma^2 L_{ij}q_{k_j}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Step 2: precalculate $E_i$&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  E_i = \exp\left(\left(\mu_i-\frac{\sigma_i^2}{2}\right) +\sum_{j=1}^i\tilde L_{ij} \right)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Step 3: calculate $X_n$&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;  X_n = W_t\left(R_f(1-e^Tx_t)+\sum_i^n x_{ti}E_i\right)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sums over the products $w_{k_1}w_{k_2}\cdots w_{k_n}$ we can use &lt;code&gt;numpy.linalg.multi_dot(*w)&lt;/code&gt;, where &lt;code&gt;w=[w1,w2,...]&lt;/code&gt; is the list of all vectors $w_{k_i}$. In a final step we multiply this result by $V(X_n)$.&lt;/p&gt;&#xA;" OwnerUserId="24086" LastActivityDate="2017-07-13T13:43:29.630" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27398" PostTypeId="2" ParentId="27107" CreationDate="2017-07-13T13:55:57.647" Score="1" Body="&lt;p&gt;Given the dependency between $z$ and $t$, we can rewrite the first equation with a time derivative: &#xA;$$\partial_tU=\frac{c}{n}\left(\nabla_r^2U+\rho U\right)$$.&#xA;Now we define $g=\left(\begin{array}{c}U\\\rho\end{array}\right)$. This allows us to rewrite the system of equations as: &#xA;$$\partial_t g=\left(\begin{array}{cc}\frac{c}{n}\nabla_r^2&amp;amp;0\\0&amp;amp;0\end{array}\right)g+\frac{c}{n}\left(\begin{array}{c}\frac{1}{2}\\0\end{array}\right)g^T\left(\begin{array}{cc}0&amp;amp;1\\1&amp;amp;0\\\end{array}\right)g+\left(\begin{array}{c}0\\1\end{array}\right)a\left(g^T\left(\begin{array}{cc}1&amp;amp;0\\0&amp;amp;0\end{array}\right)g\right)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can choose your favorite time stepping algorithm for the non linear system of equations:&#xA;$$\partial_tg=F(g,t,r)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. Crank Nicolson would be a possible choice. Now there is no need to neglect something. Beware that you need to solve the non linearity for the implicit part of Crank Nicolson each step. For details see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/27325/crank-nicolson-algorithm-for-coupled-pdes/27331#27331&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the additional term&#xA;$$\frac{c}{n} \beta|U|^2U $$.&#xA;you need to add $$\frac{c}{n} \beta \left(\begin{array}{cc}1&amp;amp;0\\0&amp;amp;0\end{array}\right)g \left(g^T\left(\begin{array}{cc}1&amp;amp;0\\0&amp;amp;0\end{array}\right)g\right)$$.&#xA;$U$ should not be in any expression, we want to solve for the field $g$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sure one can simplify the expressions further, but this should suffice as a starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have more terms in $\partial_tU$, simply guess for $g$ (it's not that hard), evaluate it and see if you arrive at the original equations.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2017-07-14T16:23:08.890" LastActivityDate="2017-07-14T16:23:08.890" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27399" PostTypeId="2" ParentId="27392" CreationDate="2017-07-13T14:18:12.357" Score="0" Body="&lt;p&gt;I solved it by divide the rectangle region into 13 sub-regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DQDPn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DQDPn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E(S(X,Y)) = \iint _{Rect} P(\sigma)S(x,y) d\sigma=\frac{1}{W \cdot H}\lbrack 4\iint_AS(x,y)d\sigma+2\iint_BS(x,y)+2\iint_CS(x,y)d\sigma+\iint_DS(x,y)d\sigma+4\iint_ES(x,y)d\sigma \rbrack$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each sub-regions can be worked out with some line integral or some triangular function. And can be solved with Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E(S(X,Y))=\frac{\frac{R^4\, \left(3\, \pi + 11\right)}{3} + 2\, \left(H - 2\, R\right)\, \left(\pi\, R^3 - \frac{2\, R^3}{3}\right) + 4\, R^4\, \left(\pi - \frac{4}{3}\right) - 2\, \left(2\, R - W\right)\, \left(\pi\, R^3 - \frac{2\, R^3}{3}\right) - \pi\, R^2\, \left(H - 2\, R\right)\, \left(2\, R - W\right)}{H\, W}$&lt;/p&gt;&#xA;" OwnerUserId="24849" LastActivityDate="2017-07-13T14:18:12.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27403" PostTypeId="1" CreationDate="2017-07-13T15:42:06.340" Score="1" ViewCount="70" Body="&lt;p&gt;Consider the unit circle $x^2 + y^2 = 1$, it also represents the solution to the ODE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x + y \frac{dy}{dx} = 0; y(0) = 1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we don't know how to solve the above equation analytically (there are more complex cases where analytical solution is impossible), a finite difference discretization gives:&#xA;$$&#xA;y^{n+1} = y^n - \frac{x^n}{y^n} \delta, x^0 = 0, y^0 = 1.&#xA;$$&#xA;$\delta$ is the step size. This method suffers from the singularity at $(x=1,y=0)$, where the slope of the circle diverges to infinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there are better methods that can recover the entire circle. A key aspect appears to be the integration direction, i.e., the sign of $\delta$, must be reversed at some point.&lt;/p&gt;&#xA;" OwnerUserId="8203" LastEditorUserId="9667" LastEditDate="2017-07-13T21:34:53.250" LastActivityDate="2017-07-13T21:34:53.250" Title="Calculation of integration curve" Tags="|ode|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27405" PostTypeId="2" ParentId="27403" CreationDate="2017-07-13T21:34:03.580" Score="2" Body="&lt;p&gt;I think that you can rewrite your equation as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x + \frac{1}{2}\frac{d u}{d x} = 0\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $u = y^2$. Then, the difference equation would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u^{n+1} = u^{n} - 2x \Delta x\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could recover the original variable by computing the square root of the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With these changes, I managed to solved it in the following snippet&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;#%% Solution&#xA;npts = 100&#xA;x = np.linspace(0, 1, npts)&#xA;u = np.ones_like(x)&#xA;dx = x[1] - x[0]&#xA;for cont in range(1, npts):&#xA;    u[cont] = u[cont - 1] - 2*x[cont - 1]*dx&#xA;&#xA;#%% Plotting&#xA;plt.figure(figsize=(2, 2))&#xA;plt.plot(x, np.sqrt(u))&#xA;plt.axis(&quot;image&quot;)&#xA;plt.xlabel(&quot;x&quot;)&#xA;plt.ylabel(&quot;y&quot;)&#xA;plt.xticks([0, 0.5, 1])&#xA;plt.yticks([0, 0.5, 1])&#xA;plt.savefig(&quot;ode_sol.png&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and, this is the plot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Yfw5A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yfw5A.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2017-07-13T21:34:03.580" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27406" PostTypeId="1" AcceptedAnswerId="27407" CreationDate="2017-07-13T22:24:39.590" Score="1" ViewCount="399" Body="&lt;p&gt;My background is mainly engineering and applied research and I have been a developer or some CFD software, but mostly at high level without worrying about linear solvers and the like. This has been changing lately, has we have been wishing to integrate PETSc without our own software instead of  using our own in-house sparse libraries (for reason that I believe are evident. There is no way our small lab can develop a better library of linear solver than PETSc or Trillinos).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, coming from an engineering background, I find I totally lack knowledge on iterative linear solvers (Krylov subspace, multigrid, etc.). Would there be a reference ( a book preferably, I am still oldschool) that would cover this type of material in a way that would be readable without undergraduate or graduate studies in math, but coming from a more engineering background?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a relevant note, I have seen that there are many tutorials regarding PETSc on their homepage (&lt;a href=&quot;http://www.mcs.anl.gov/petsc/documentation/tutorials/HandsOnExercise.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mcs.anl.gov/petsc/documentation/tutorials/HandsOnExercise.html&lt;/a&gt;), but I fear I am struggling to find a starting point to learn this type of library or even install it on my local linux distribution correctly. What would be the best starting document for a)installation (and making sure its ok with MPI) and b) learning steps by steps? &lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2019-03-08T05:20:25.613" Title="Practical reference on sparse linear solvers for PDEs (Navier-Stokes, Poisson) and on learning PETSc" Tags="|linear-solver|petsc|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27407" PostTypeId="2" ParentId="27406" CreationDate="2017-07-13T23:16:33.700" Score="3" Body="&lt;p&gt;If you're looking for a book and are happy with finite element discretizations, you can check out Elman, Silvester, and Wather, &quot;Finite Elements and Fast Iterative Solvers&quot;.  If you prefer finite volume or finite difference methods, Wesseling's CFD book has a section on multigrid that will at least give you the context for multigrid-specific material.  If you're specifically interested in multigrid, Trottenberg is a good choice.  Saad is a good iterative methods book, but that depth is not necessary to understand solvers in PETSc.&lt;/p&gt;&#xA;" OwnerUserId="119" LastEditorUserId="20526" LastEditDate="2017-07-14T06:52:35.433" LastActivityDate="2017-07-14T06:52:35.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27408" PostTypeId="2" ParentId="27406" CreationDate="2017-07-13T23:57:15.167" Score="4" Body="&lt;p&gt;Wow! A question that I can answer! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been using PETSc for the past year and a half to solve the Navier-Stokes equations (with some hard-coded MPI). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way to learn PETSc is to (1) read the manual so you generally know what's in PETSc, (2) decide roughly which part of PETSc you need, and then (3) begin experimenting with simple examples to build an understanding of the components that you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently learned how to use PETSc's Data Management for Distributed Arrays (DMDA) in conjunction with KSP. There is no straight-forward guide, as far as I know. The best way to learn is to choose a basic component of the system, for instance DMDACreate3D(), and try to build a working code that illustrates only that object. You build the working code by looking at PETSc's in depth examples of the function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After doing this with a number of different PETSc concepts, say DMDA, matrices and vectors, index sets, vector scattering, etc., then you can put together your own application code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately there is no shortcut... but you probably know that about HPC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best of luck.&lt;/p&gt;&#xA;" OwnerUserId="22717" LastEditorUserId="22717" LastEditDate="2019-03-08T05:20:25.613" LastActivityDate="2019-03-08T05:20:25.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27409" PostTypeId="1" AcceptedAnswerId="27410" CreationDate="2017-07-14T00:07:06.533" Score="1" ViewCount="934" Body="&lt;p&gt;I am curious if it is possible to request less than a node using SLURM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I use the commands,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --job-name=&quot;name&quot;&#xA;#SBATCH --output=&quot;name.%j.%N.out&quot;&#xA;#SBATCH --ntasks 1&#xA;#SBATCH --time=00:30:00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will SLURM automatically reserve an entire node for me, or will it try to fit me in on just one processor? Can this make it faster for my small jobs to run on the cluster?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if I use the commands&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --job-name=&quot;name&quot;&#xA;#SBATCH --output=&quot;name.%j.%N.out&quot;&#xA;#SBATCH --ntasks 13&#xA;#SBATCH --time=00:30:00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but there are only 12 cores per node, then what will SLURM do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="22717" LastActivityDate="2017-07-14T02:00:10.143" Title="Requesting less than a node with SLURM" Tags="|hpc|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27410" PostTypeId="2" ParentId="27409" CreationDate="2017-07-14T02:00:10.143" Score="1" Body="&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For &lt;em&gt;multiprocessing&lt;/em&gt; (MPI, message passing) use &lt;code&gt;ntasks&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;For &lt;em&gt;multithreading&lt;/em&gt; (OpenMP, pthreads) use &lt;code&gt;cpus-per-task&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;For hybrid codes you need both options and probably also want to tune &lt;code&gt;ntasks-per-node&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://slurm.schedmd.com/sbatch.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to the &lt;code&gt;sbatch&lt;/code&gt; manual&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is somewhat complicated.  It depends on whether your program needs tasks or cores.  For example an MPI based program will be launched several times and communicates via message passing, while an OpenMP based program will only be launched once and will then launch several threads which communicate via shared memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of message passing it doesn't matter on which node the tasks are launched as long as they can communicate (Infiniband, Ethernet, etc).  In the case of shared memory it is important that tasks run on the same node (in fact, it is required).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;ntasks&lt;/code&gt; option of SLURM specifies how many tasks your program will launch, which could be threads of independent instances of the MPI program.  However, SLURM assumes that when you say &lt;code&gt;ntasks&lt;/code&gt; you mean tasks which communicate by message passing and in case your machine has 12 cores but you requested 13 tasks, it will happily launch 12 tasks on one node and 1 on another node.  (I don't think this behaviour is guaranteed.  SLURM could also throw all 13 tasks on one node with 12 CPUs and let the CPU schedule the tasks.  You can get more fine-grained control using &lt;code&gt;ntasks-per-core&lt;/code&gt; and &lt;code&gt;ntasks-per-node&lt;/code&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a multithreaded program, then you want to use &lt;code&gt;cpus-per-task&lt;/code&gt; instead and set &lt;code&gt;ntasks&lt;/code&gt; to 1 (or leave it unspecified, as it defaults to 1).  This way, if you request 13 CPUs but the maximum available is 12, your job will just be rejected.&lt;/p&gt;&#xA;" OwnerUserId="24680" LastActivityDate="2017-07-14T02:00:10.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27411" PostTypeId="1" CreationDate="2017-07-14T07:15:48.093" Score="2" ViewCount="273" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatic_differentiation&quot; rel=&quot;nofollow noreferrer&quot;&gt;automatic differentiation&lt;/a&gt; on a function that contains a sparse nonsymmetric linear system to be solved. I was using &lt;a href=&quot;https://en.wikipedia.org/wiki/Biconjugate_gradient_stabilized_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;BiCGStab&lt;/a&gt; to solve this part of the function, but noticed the derivatives produced by the automatic differentiation that were carried through BiCGStab were diverging in some cases. Is there an iterative linear solver with proven convergence using automatic differentiation?&lt;/p&gt;&#xA;" OwnerUserId="23647" LastEditorUserId="23647" LastEditDate="2017-07-14T07:54:39.663" LastActivityDate="2017-07-14T07:54:39.663" Title="Iterative linear solvers compatible with automatic differentiation?" Tags="|linear-solver|automatic-differentiation|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27412" PostTypeId="2" ParentId="668" CreationDate="2017-07-14T13:30:48.287" Score="2" Body="&lt;p&gt;This question opens a can of worms, as is made evident by the variety of answers given. Many of these make useful points, but a really helpful answer would take account of considerations that have not been raised. Apart from making specific the exact nature of the geometry, it would be valuable to know a) What sort of accuracy do you need? b) Do you want to stay with regular square meshes? c) How much are you willing to invest in learning new technology?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regular square meshes make it hard to define the boundary precisely, so that many people change to conforming meshes. Conforming meshes with rectangular connectivity have difficulty fitting to very irregular shapes, so many people adopt unstructured meshes (triangles/tetraheda&#xA;or more general).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any data not having regular cartesian structure, it is difficult to evaluate derivatives, so many people reformulate their problems in integral form, which leads to finite-element/finite-volume methods (which CAN achieve high-order). There are mesh-free methods. There are boundary-element methods. There are immersed boundary methods. There are cut-cell methods. Often there is a method that is popular in some applications but not in others, for reasons that are mostly historical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish you good luck in navigating this maze, but you must realize that there  is not a one-size-fits-all solution to your question.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-07-14T13:30:48.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27413" PostTypeId="2" ParentId="24125" CreationDate="2017-07-14T20:05:16.773" Score="1" Body="&lt;p&gt;Take determinants and you have finished (apart from the difference with respect to the problem statement that you are considering the Schur complement of $A_{22}$ instead of that of $A_{11}$).&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-07-14T20:05:16.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27416" PostTypeId="1" AcceptedAnswerId="27436" CreationDate="2017-07-16T11:14:07.837" Score="0" ViewCount="109" Body="&lt;p&gt;I am reading &quot;&lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=0ahUKEwi1y4_czI3VAhUJSJoKHd31Ay4QFggiMAA&amp;amp;url=http%3A%2F%2Fwgropp.cs.illinois.edu%2Fusingmpiweb%2F&amp;amp;usg=AFQjCNEWKmnxowB4l7SvOtTtkQ0vkYi3fw&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using MPI (3rd edition)&lt;/a&gt;&quot; from William Gropp, where in chapter 4 application section 4.13, it introduces an MPI application Nek5000/NekCEM which is based on spectral element method (SEM) coupled with high-order timestepping to solve the governing IVM/BVM problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is one statement about SEM that I do not quite understand (Yes, it's not relevent to MPI):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&quot;A central idea in the SEM is never to form local elemental matrices, which would in general have $(N+1)^6$ nonzeros per element, but to instead use preconditioned iterative solvers that requrie only the action of operators applied to functions.&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some limited backgroud in FEM and SEM, and I know that SEM elemental matrices are usually much larger than FEM due to high polynomial orders, but I still can't get what the author is trying to conveying with this statement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please explain it with more concrete descriptions? Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="18224" LastActivityDate="2017-07-20T14:05:56.313" Title="Need clarification on a piece of book excerpt about spectral element method!" Tags="|finite-element|pde|parallel-computing|spectral-method|dense-matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27419" PostTypeId="1" AcceptedAnswerId="27429" CreationDate="2017-07-16T19:18:57.170" Score="0" ViewCount="689" Body="&lt;p&gt;This question is quite open, and the actual problem comes from something you would probably consider an everyday niche (something you'd probably take for granted without really thinking about it). However:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I may not disclose the actual application's nature, but I found that pizza bakeries and pizza consumers make for a reasonable replacement. We have been asked to create an optimal pizza delivery plan. I guess we will also need an appropriate means of visualizing the plan when we want to communicate our results to the audience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have been given&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a list of pizza bakeries (roughly 50):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;where each bakery is located&lt;/li&gt;&#xA;&lt;li&gt;how efficient each bakery is at making pizza&lt;/li&gt;&#xA;&lt;li&gt;each bakery's minimum and maximum load. A bakery may be shut down if the minimum load cannot be reached.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a list of pizza consumers (a few thousand):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;where each consumer is located&lt;/li&gt;&#xA;&lt;li&gt;how many pizzas each consumer needs - it's a constant flow of pizza&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a network of roads with&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;start, intermediate, and end node&lt;/li&gt;&#xA;&lt;li&gt;segment length (this might be different from the Euclidean distance between adjacent nodes)&lt;/li&gt;&#xA;&lt;li&gt;road capacity (maximum number of pizzas on this segment)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Bakeries and consumers are located at road start or end nodes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rules and metrics:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pizza from multiple bakeries may be combined to serve a customer. This is encouraged if it helps&lt;/li&gt;&#xA;&lt;li&gt;multiple paths may lead from a bakery to a customer, and pizzas may be sent on different routes in multiple packages. This is also encouraged if it helps.&lt;/li&gt;&#xA;&lt;li&gt;for simplicity, let's assume that all pizzas are of the same quality.&lt;/li&gt;&#xA;&lt;li&gt;no other traffic other than pizza delivery is taking place.&lt;/li&gt;&#xA;&lt;li&gt;the effort for delivering a pizza increases linearly with distance.&lt;/li&gt;&#xA;&lt;li&gt;the effort for delivering a pizza increases with the square of road load.&lt;/li&gt;&#xA;&lt;li&gt;bakeries may be closed down as an initial condition, and stay closed.&lt;/li&gt;&#xA;&lt;li&gt;There's no limit to the number of customers per bakery (apart from the bakery's pizza capacity) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Bakery efficiency and delivery effort are two different numbers: there's probably a large number of feasible solutions with different overall bakery efficiency. It's possible to state maximum allowed delivery effort and minimum required efficiency, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So in short: what is a suitable approach to this problem in terms of optimization algorithms?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's hard to visually compare delivery solutions just by showing the network with some numbers attached to it, so we're looking for a way of &quot;telling the story&quot;. Chord diagrams or circular diagrams might be a possibility, but someone more experienced might have a better suggestion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can we visualize results? We'd like to show how more efficient bakeries are utilized (if they are) but also how the optimization has an impact on the delivery effort.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3680" LastEditorUserId="20688" LastEditDate="2019-06-22T20:14:19.380" LastActivityDate="2019-06-22T20:14:19.380" Title="How to create an optimal pizza delivery plan and how to visualize it" Tags="|constrained-optimization|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27420" PostTypeId="1" AcceptedAnswerId="27426" CreationDate="2017-07-16T20:13:24.417" Score="1" ViewCount="1745" Body="&lt;p&gt;I am implementing a finite element solver (in 2D) to solve the generic differential equation :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\nabla(a(x) \nabla u) = f$$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Brief explanation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;By integrating and multipling by a test function, the weak form allows to integrate the stiffness matrix :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A_{ij} = \int a(x) \nabla \phi_i(x) \nabla \phi_j(x)\, dx\quad i=1,2,3\ .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More simply, we can write the local stiffness matrix (of a 3 node linear triangle) of the element $K$ (the assembly will be done in a next step):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{ij}^{K} = \int a(x) \nabla \phi_i(x) \nabla \phi_j(x)\, dx\quad i=1,2,3\ .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In first place, I use the central quadrature formula that allows to compute:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{ij}^{K} = a_\text{mean}(b_i b_j + c_i c_j) K_\text{area}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $a_\text{mean}$ = central value of $a(x)$ in the center of the triangle. The coefficient are the one form the shape function: $\phi_i = a_i + b_i x1 + c_i x_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For one single local stiffness matrix, it represents this code in Python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loc2glob = t[0:3,K] #local to global map&#xA;x = p[0,loc2glob] #get a list of the x-coordinates&#xA;y = p[1,loc2glob] #get a list of the y-coordinates&#xA;area,b,c = gradient(x,y) # compute the numerical gradient&#xA;xc = np.mean(x), yc = np.mean(y)&#xA;a_centroid = a(xc,yc) #simplification&#xA;b = np.atleast_2d(b)&#xA;c = np.atleast_2d(c)&#xA;AK = (np.dot(b.T, b) + np.dot(c.T, c))*a_centroid*area&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;And ...?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;After the verification of this method, I end up with implementing a more general integration method (based on quadrature). The coordinates $x,y$ are transformed in $r,s$ (local integration, to have a general method). The Jacobian is computed numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_{ij}^{K} = \int_{el_K} a(r, s) \nabla \phi_i (r, s) \nabla \phi_i(r,s) |J_K(r,s)|\ dr\ ds$$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;A little more explanation on $(r,s)$ coordinate&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In the domain of finite element method, computer implementation are used to switch the coordinate $(x,y)$ of a (triangle) element to an $(r,s)$ orthogonal base. The principle is very nice, it allows to do a variable change in the integral, and to have an easiest method to apply the quadrature :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\Omega_K} q(x,y) \phi_i(x,y)\phi_j(x,y)\ dx\ dy =&#xA;  \int_{\triangle} q(\xi, \eta) \psi_i(\xi, \eta)\psi_j(\xi, \eta) \left|\frac{\partial (x, y)}{\partial (\xi, \eta)}\right|\ d\xi\ d\eta$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Graphically, it corresponds to : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/XfYhZ.png&quot; alt=&quot;latex&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can use the coordinate $(r,s)$ of a generic triangle for the integral, providing the Jacobian of the coordinate transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, I use a 3 gauss point method. The basics are here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AK = np.zeros((3,3))&#xA;for q in range(len(qwgts)):&#xA;    r = rspts[q,0] # r coordinate of the q_th quadrature point&#xA;    s = rspts[q,1] # s coordinate of the q_th quadrature point&#xA;    a_gauss_point = a(r,s)&#xA;    S,dSdx,dSdy,detJ = isopmap(x,y,r,s,shapefcn)&#xA;    dSdx =np.matrix(dSdx),dSdy =np.matrix(dSdy)&#xA;    wxarea = a_gauss_point*qwgts[q]*detJ/2 #weight * area (corrected by 0.5)&#xA;    AK = AK + (dSdx.T.dot(dSdx) + dSdy.T.dot(dSdy))* wxarea&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that the method &quot;isopmap&quot; return a correct Jacobian, since I have tested with data found in the literature. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;My problem...&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;When I take $a(x,y) = 1$ , the two methods give the &lt;strong&gt;same result&lt;/strong&gt;.&#xA;But when I put $a(x,y) = x + y$ for example, I end with significant difference between the two matrix.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;And so?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Do you have any explanation? Of course the problem is about the value $a(r,s)$ in the &lt;em&gt;second code&lt;/em&gt;, but I don't see where the thing fails...&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;SOLUTION : EDIT (17.07.2017)&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In fact it comes that I didn't understand properly the logic of the mapping (x,y) domain to (r,s) domain. In fact the integral can be written like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://image.noelshack.com/fichiers/2017/29/1/1500288464-4f39b6566fe4e4c46b897679ffee90c9.png&quot; alt=&quot;Litterature&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I just have to change in my python code the evaluation of the function $a(x,y)$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_physical = np.dot(x,S)&#xA;y_physical = np.dot(y,S)&#xA;a_gauss_point = a(x_physical,y_physical)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;If you want to reproduce this behaviour (EDIT : correct code uploaded)&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;You can download the Python coded that contains 100 lines of code in order to compute very easily the $A_K$ matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division # avoid integer problem of division&#xA;&#xA;#Import zone&#xA;import numpy as np&#xA;&#xA;&#xA;#Define shape function&#xA;def P1shapes(r,s):&#xA;    S = np.array([1-r-s,r,s])&#xA;    dSdr = np.array([-1,1,0])&#xA;    dSds = np.array([-1,0,1])&#xA;    return S,dSdr,dSds&#xA;#Jacobian function&#xA;def isopmap(x,y,r,s,shapefcn):&#xA;    # x = vector of x coordinate of the element's point&#xA;    #shapefcn = P1shapes or P2shapes&#xA;    S,dSdr,dSds = shapefcn(r,s);&#xA;    j11=np.dot(dSdr,x)&#xA;    j12=np.dot(dSdr,y)&#xA;    j21=np.dot(dSds,x)&#xA;    j22=np.dot(dSds,y)&#xA;    detJ=j11*j22-j12*j21&#xA;    dSdx=( j22*dSdr-j12*dSds)/detJ&#xA;    dSdy=(-j21*dSdr+j11*dSds)/detJ&#xA;    return S,dSdx,dSdy,detJ&#xA;&#xA;#Gradient Function&#xA;def gradient(x,y):&#xA;    #for linear shape function (1st order)&#xA;    area = polyArea(x,y)&#xA;    b = np.array([y[1]-y[2],y[2]-y[0],y[0]-y[1]])/2/area&#xA;    c = np.array([x[2]-x[1],x[0]-x[2],x[1]-x[0]])/2/area&#xA;    return area,b,c&#xA;&#xA;def polyArea(x,y):&#xA;    #Shoelace formula =&amp;gt; Compute Area of Polygons, x and y set of pointd given (vertices coordinates)&#xA;    return 0.5*np.abs(np.dot(x,np.roll(y,1))-np.dot(y,np.roll(x,1)))&#xA;&#xA;#Define a(x,y)&#xA;def a(x,y):&#xA;    return x + y&#xA;&#xA;#Gauss point quadrature&#xA;qwgts=np.array([1/3,1/3,1/3])&#xA;rspts=np.array([[1/6,1/6],&#xA;[2/3,1/6],&#xA;[1/6,2/3]])&#xA;&#xA;#Point of the 3 nodes triangles (test case)&#xA;x = [-21.68467035,-21.17695462 ,-22.18700401]&#xA;y = [-9.94204652 ,-8.91258056 ,-9.12362242]&#xA;&#xA;#First method (simplifed one)&#xA;area,b,c = gradient(x,y) # compute the numerical gradient&#xA;xc = np.mean(x)&#xA;yc = np.mean(y)&#xA;a_centroid = a(xc,yc) #simplification&#xA;b = np.atleast_2d(b)&#xA;c = np.atleast_2d(c)&#xA;AK_simplified = (np.dot(b.T, b) + np.dot(c.T, c))*a_centroid*area&#xA;&#xA;#Second Method&#xA;AK_quadrature = np.zeros((3,3))&#xA;for q in range(len(qwgts)):&#xA;    r = rspts[q,0] # r coordinate of the q_th quadrature point&#xA;    s = rspts[q,1] # s coordinate of the q_th quadrature point&#xA;    S,dSdx,dSdy,detJ = isopmap(x,y,r,s,P1shapes)&#xA;    #Map the gauss point to the physical domain (r,s)_gauss to (x,y)_gauss &#xA;    x_physical = np.dot(x,S)&#xA;    y_physical = np.dot(y,S)&#xA;    a_gauss_point = a(x_physical,y_physical)&#xA;&#xA;    dSdx =np.matrix(dSdx)&#xA;    dSdy =np.matrix(dSdy)&#xA;    wxarea = a_gauss_point*qwgts[q]*detJ/2 #weight * area (corrected by 0.5)&#xA;    AK_quadrature = AK_quadrature + (dSdx.T.dot(dSdx) + dSdy.T.dot(dSdy))* wxarea&#xA;&#xA;#Show the matrix&#xA;print AK_simplified&#xA;print &quot; &quot;&#xA;print AK_quadrature&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24889" LastEditorUserId="24889" LastEditDate="2017-07-17T10:53:48.853" LastActivityDate="2017-07-17T16:01:11.503" Title="Computation of stiffness matrix with variable coefficient" Tags="|finite-element|quadrature|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27421" PostTypeId="2" ParentId="21402" CreationDate="2017-07-16T21:34:59.583" Score="0" Body="&lt;p&gt;I think you're right that double precision is not needed usually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a proxy, Apple's Metal framework doesn't support doubles at all, and it's being billed as a compute framework with Apple-provided functions for machine learning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Info about Metal data types: &lt;a href=&quot;https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24891" LastActivityDate="2017-07-16T21:34:59.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27422" PostTypeId="1" CreationDate="2017-07-17T08:34:02.733" Score="3" ViewCount="129" Body="&lt;p&gt;What would currently be the best way to share a matrix with the math community?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm aware of &lt;a href=&quot;http://math.nist.gov/MatrixMarket/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matrix Market&lt;/a&gt; but it seems the last update was in 2007...)&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2017-07-17T15:17:30.637" Title="Sharing a matrix with the math community" Tags="|matrix|reference-request|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27426" PostTypeId="2" ParentId="27420" CreationDate="2017-07-17T12:12:25.267" Score="2" Body="&lt;p&gt;It is not uncommon in a finite element model to have material properties or boundary conditions that vary as a function of the&#xA;spatial coordinates. Often these are implemented as a &quot;user-written&quot; function that takes the coordinates as input values and&#xA;returns the properties or boundary conditions at these coordinates. The input coordinates are typically defined in the global&#xA;Cartesian system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an isoparametric element formulation with numerical integration is used, the user-written property function will be called&#xA;at each integration point. Given the parametric coordinates of the integration point, say $r$ and $s$ in two dimensions, the&#xA;required global $x$ and $y$ coordinates are obtained from the basic isoparametric definition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x = \sum_{i=1}^{n_s} N_i(r,s) x_i&#xA;$$&#xA;$$&#xA;y = \sum_{i=1}^{n_s} N_i(r,s) y_i&#xA;$$&#xA;where $N_i$ are the shape functions, $x_i$ and $y_i$ are the global coordinates of the element nodes, and $n_s$ is the number of nodes in&#xA;the element.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2017-07-17T16:01:11.503" LastActivityDate="2017-07-17T16:01:11.503" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27427" PostTypeId="2" ParentId="27422" CreationDate="2017-07-17T14:33:24.813" Score="2" Body="&lt;p&gt;If it happens to be sparse, you might also consider the UF collection:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cise.ufl.edu/research/sparse/matrices/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cise.ufl.edu/research/sparse/matrices/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know that it's especially active in the last few years, a cursory search seemed to suggest the last additions were around 2013 or so.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2017-07-17T15:17:30.637" LastActivityDate="2017-07-17T15:17:30.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27428" PostTypeId="2" ParentId="27422" CreationDate="2017-07-17T14:58:50.007" Score="1" Body="&lt;p&gt;Most languages have the ability to read HDF5 files in some fashion. That would be a very cross-compatible way to save a sparse matrix to a file to share.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-07-17T14:58:50.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27429" PostTypeId="2" ParentId="27419" CreationDate="2017-07-17T17:37:08.817" Score="6" Body="&lt;h1&gt;Problem Formulation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I can't guarantee that this is a perfect (or smallest-possible) formulation of the problem, but maybe it will help guide a better one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The road network is a directed graph consisting of intersections (nodes) connected by roads (edges). As input information, assume that you have an adjacency matrix $\mathcal{A}$ enumerating the edges. $\mathcal{A}_{ij} = 1$ if road exists connecting traveling from $i$ to $j$, 0 otherwise. Also, for simplicity, assume that the producers/consumers live at intersections (roads can be split at actual locations to make this true, so no loss of generality here). It should be noted that $\mathcal{A}$ is likely to be extremely sparse (unless you work for Amazon and you're delivering things via drones...), so this should be exploited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let your decision variable, $X_{ijkl} \in \mathbb{R}_+$ represent the quantity of pizza being transported along road segment $ij$ (from $i$ to $j$) from producer $k$ to consumer $l$. Let $P_k \in\{0,1\}$ indicate whether producer $k$ is active. Let $Z_{kl}\in\{0,1\}$ represent whether producer $j$ is servicing consumer $k$. Let $Q_{kl}\in\mathbb{R}_+$ be the quantity of product sent from producer $k$ to consumer $l$. The other problem variables/parameters are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;c_{ij} \in \mathbb{R}_+&amp;amp;= \text{Quadratic traffic cost coefficient of driving on road segment $ij$} \\&#xA;L_{ij} \in \mathbb{R}_+&amp;amp;= \text{Cost due to length of road segment $ij$} \\&#xA;e_k \in \mathbb{R}_+&amp;amp;= \text{Efficiency of producer $k$} \implies \frac{1}{e_k} \text{is cost of producer $k$}\\&#xA;C^{max}_k \in \mathbb{R}_+&amp;amp;= \text{Capacity of producer $k$} \\&#xA;C^{min}_k \in \mathbb{R}_+&amp;amp;= \text{Minimum load of producer $k$}\\&#xA;D_l \in \mathbb{R}_+&amp;amp;= \text{Demand of consumer $l$} \\&#xA;C^{road} &amp;amp;= \text{Road capacity, same sparsity as $\mathcal{A}$}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Constraints&lt;/strong&gt;&#xA;\begin{align}&#xA;X_{ijkl} &amp;amp;\le \mathcal{A}_{ij}C^{max}_k\quad\forall \; i,j,k,l \\&#xA;X_{ijkl} &amp;amp;\le Z_{kl}C^{max}_k \quad\forall \;i,j,k,l \\&#xA;\sum_{k} Q_{kl} &amp;amp;\ge D_l \quad \forall \; l\\&#xA;\sum_{l} Q_{kl} &amp;amp;\le C^{max}_k P_k \quad \forall \; k\\&#xA;\sum_{l} Q_{kl} &amp;amp;\ge C^{min}_k P_k \quad \forall \; k\\&#xA;Z_{kl} &amp;amp;\le P_k \quad \forall \; k,l \\&#xA;Q_{kl} &amp;amp;\le Z_{kl}C^{max}_k \quad \forall \; k,l\\&#xA;\sum_{k,l} X_{ijkl} &amp;amp;\le C^{road}_{ij} \quad \forall \; i,j \\&#xA;\sum_j \left(X_{ijkl} - X_{jikl}  \right)&#xA;&amp;amp;\left\{&#xA;\begin{array}\\ &#xA;= Q_{kl} \text{ if $i$ is the location of producer k} \\&#xA;= -Q_{kl} \text{ if $i$ is the location of consumer k} \\&#xA;0 \text{ otherwise}&#xA;\end{array}&#xA;\right\} \forall \; i,k,l\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final constraint ensures that internal nodes along a path have the same number of incoming and outgoing routes, producers have more outgoing than incoming routes, and consumers have more incoming than outgoing routes. It is a variation on the constraint found on wikipedia that has been adjusted to allow for multiple routes between producer/consumer pairs: &lt;a href=&quot;https://en.wikipedia.org/wiki/Shortest_path_problem#Linear_programming_formulation&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Shortest_path_problem#Linear_programming_formulation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Objective Function&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is relatively simple compared to the constraints: simply minimize costs. For the traffic, I assume that the traffic costs are proportional to the quantity $X$ (or $X^2$), rather than the actual number of trucks that you might use to deliver them. If this well-approximates the actual application, then the following objective function should work reasonably well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;roadCost &amp;amp;= \sum_{i,j}\left(\sum_{k,l}X_{ijkl}\right)\left(L_{ij} + \left(\sum_{k,l}X_{ijkl}\right)c_{ij}\right)\\&#xA;\\&#xA;efficiencyCost &amp;amp;= \sum_{k,l} Q_{kl}/e_{k}&#xA;\\&#xA;\\&#xA;\min_{X,Z,Q,P} \;\; &amp;amp;roadCost + efficiencyCost&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Solving&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Due to the unfortunate quadratic term, this is a mixed integer-quadratic programming problem. Fortunately, there are good solvers out there for these (I am currently using Gurobi 7.0, which is expensive, but not for academics). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented it in Julia (0.5) using the JuMP package, which allows for very straightforward model specification, and it hooks into a wide variety of different commercial and open-source solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for some reason you're intent on writing your own solver, you'd need a QP solver as a primitive and then do the usual branch-and-bound algorithm on the integer variables. I &lt;em&gt;highly&lt;/em&gt; recommend that you not do this.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Visualization&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In Julia, there is the GraphPlot package which seems ideally suited to this. I haven't played with it much, but it seems well-suited to visualizing these sorts of flow problems.&#xA;&lt;a href=&quot;https://github.com/JuliaGraphs/GraphPlot.jl&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/JuliaGraphs/GraphPlot.jl&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't really say much more about what you should visualize except to point you to the GraphPlot tool above that will let you do pretty much anything you want. I put some examples using it for this problem in my implementation, which you can get to below.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;My Implementation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I have put a html version of the IJulia notebook for solving this problem up as a gist, which can be found here:&#xA;&lt;a href=&quot;https://gist.github.com/tjolsen/ea9c677f43b19df482a2ee344f5d467c&quot; rel=&quot;noreferrer&quot;&gt;https://gist.github.com/tjolsen/ea9c677f43b19df482a2ee344f5d467c&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To actually see it (rather than the source html), simply download the raw text as an html file and open it in your browser.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a large-scale version of this, you would want to explicitly exploit the sparsity of $\mathcal{A}$ to reduce the number of $X$ variables. I didn't do that here, since I set it up for a 2-producer, 1-consumer network.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastEditorUserId="7378" LastEditDate="2017-07-17T19:19:55.840" LastActivityDate="2017-07-17T19:19:55.840" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27430" PostTypeId="1" CreationDate="2017-07-17T18:43:35.980" Score="1" ViewCount="253" Body="&lt;p&gt;This question may be more suited for physics.stackexchange, but I saw &lt;a href=&quot;https://physics.stackexchange.com/questions/316665/second-quantization-using-matlab&quot;&gt;this post&lt;/a&gt; was recommended for StackOverflow or Computational Science, so I'm asking my question here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to write a program in Matlab to evaluate an expression with fermionic annihilation and creation operators. Define&#xA;$$\{f, g \} \equiv fg + gf$$&#xA;then the fermionic second quantization operators satisfy&#xA;$$\{a_i,a_j \} = \{ a_i^\dagger, a_j^\dagger\} = 0 \qquad \{a_i^\dagger, a_j\} = \delta_{ij}$$&#xA;$\forall i, j \in \{1, 2, ..., N \}$ for some given $N$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$W(t)$ is a $2N \times 2N$ matrix that I know that gives me the time evolution of the operators in the Heisenberg picture;&#xA;\begin{align*}&#xA;a_i(t) &amp;amp;= \sum_{j=1}^N \left(W_{2i-1,2j-1}(t)a_j + W_{2i-1,2j}(t)a_j^\dagger\right)\\&#xA;a_i^\dagger(t) &amp;amp;= \sum_{j=1}^N \left(W_{2i,2j-1}(t)a_j + W_{2i,2j}(t)a_j^\dagger\right)&#xA;\end{align*}&#xA;i.e.&#xA;$$\left(\begin{matrix}a_1(t)\\a_1^\dagger(t)\\\vdots\\a_N(t)\\a_N^\dagger(t) \end{matrix}\right) = W(t)\left(\begin{matrix}a_1\\a_1^\dagger\\\vdots\\a_N\\a_N^\dagger \end{matrix}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to write a function that takes in $W(t)$ (evaluated at a given $t$) and $j$ and returns&#xA;$$\langle \psi | \left(\prod_{i=1}^{j-1}\left(1-2a_i^\dagger(t)a_i(t) \right) \right)\left(a_j(t)^\dagger + a_j(t) \right) | \psi \rangle$$&#xA;where&#xA;$$| \psi \rangle = \left(\prod_{i=1}^N\frac{1+a_i^\dagger}{\sqrt{2}} \right)|0\rangle = 2^{-N/2}\left(|0...0\rangle + |0...1 \rangle + |0...1 \ 0 \rangle + |0 ...1 \ 1 \rangle + ... + |1...1 \rangle \right)$$&#xA;$|\psi\rangle$ is just a normalized, equally weighted linear combination of all states. Note that it does not change with time. The time evolution is encoded in the operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thought was to use Matlab's symbolic algebra, and try and code for what each operator does to a state. But the symbolic algebra does not always preserve order, which is crucial because the operators don't always commute. Also, trying to just naively go through is so computationally expensive and grows exponentially with $N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="24903" LastActivityDate="2017-07-17T18:43:35.980" Title="Second Quantization in Matlab" Tags="|matlab|computational-physics|quantum-mechanics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27431" PostTypeId="2" ParentId="27395" CreationDate="2017-07-17T19:08:21.697" Score="3" Body="&lt;p&gt;It is better to state the complexity based on the sum of the reciprocals of the primes, which is correctly pointed to grow at the indicated rate of $\ln \ln N$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Divergence_of_the_sum_of_the_reciprocals_of_the_primes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Divergence_of_the_sum_of_the_reciprocals_of_the_primes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the space complexity, you are looking at the 'prime counting function', which grows at $\frac{N}{\ln N}$ as opposed to $\ln N$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Prime-counting_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Prime-counting_function&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm implementation is just the conventional 'Sieve of Eratosthenes', but described in a complicated fashion.&lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2017-07-17T19:08:21.697" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27432" PostTypeId="1" CreationDate="2017-07-17T19:39:50.660" Score="0" ViewCount="102" Body="&lt;p&gt;I'm an undergrad who's studying electronics and fields like data sciences and mostly computing seem most interesting to me but have in-depth knowledge in neither. I learnt basic python last winter vacations and have built on it since then. I'm not too sure how further I should keep doing this and hence wanted to know how beneficial is python to an electronics student (keeping my interests in mind) and should I keep adding to my knowledge?&lt;/p&gt;&#xA;" OwnerUserId="24904" LastActivityDate="2017-07-17T21:26:37.930" Title="How much in depth should I cover Python?" Tags="|python|data-analysis|" AnswerCount="1" CommentCount="1" ClosedDate="2017-07-18T20:20:21.280" ContentLicense="CC BY-SA 3.0" />
  <row Id="27433" PostTypeId="2" ParentId="27432" CreationDate="2017-07-17T21:26:37.930" Score="2" Body="&lt;p&gt;If you understand the basics of Python, you should just try to get your hands dirty using some well known/documented Python libraries for the type of work you want to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you find you don't understand how to use certain features of the libraries due to inadequate knowledge in Python, just study up on those features and then get back to working with the library. Outside of this, it will be hard for any of us to comment on how much more you should learn Python since we don't really know where your skills are at. Good luck.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-07-17T21:26:37.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27434" PostTypeId="2" ParentId="26898" CreationDate="2017-07-18T08:42:59.110" Score="1" Body="&lt;p&gt;I am assuming you want heat flux continuity at the interface:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_i^1=T_i^2 \quad k_1\left.\frac{\partial T}{\partial r}\right|_i^1=k_2\left.\frac{\partial T}{\partial r}\right|_i^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $i$ indicates the interface and $1,2$ is the material respectively left, right of the interaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, you want a heat transfer model at the boundary:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-k_2\left.\frac{\partial T}{\partial r}\right|_b = h\left(T_b-T_a\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $T_b$ is the temperature at the boundary and $h$ is some heat transfer coefficient which depends on the fluid flow cooling the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, since you are using FVM you are using a staggered grid (i.e. domain boundaries are located in between grid nodes) and I will assume the use of ghost nodes to manipulate the values at the interface and boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Discretization of the temperature and its derivative at the boundary is done by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_i^\sigma = \frac{T_{i-}^\sigma+T_{i+}^\sigma}{2} \quad \left.\frac{\partial T}{\partial r}\right|_i^\sigma = \frac{T_{i+}^\sigma-T_{i-}^\sigma}{\Delta r}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $-/+$ indicates the (ghost) nodes just respectively left/right from the interface.&#xA;Applying it to the above conditions at the interface we arrive at the following system of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_{i+}^1 - T_{i-}^2 = -T_{i-}^1 + T_{i+}^2 $$&#xA;$$k_1T_{i+}^1 + k_2T_{i-}^2 = k_1T_{i-}^1 + k_2T_{i+}^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $T_{i+}^1$ and $T_{i-}^2$ are unknown temperatures at the ghost nodes in material 1 to the right of the interface and material 2 to the left of the interface respectively. Solving for these temperatures yields:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_{i+}^1 = \frac{k_1-k_2}{k_1+k_2}T_{i-}^1 + \frac{2k_2}{k_1+k_2}T_{i+}^2$$&#xA;$$T_{i-}^2 = \frac{2k_1}{k_1+k_2}T_{i-}^1 - \frac{k_1-k_2}{k_1+k_2}T_{i+}^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if $k_1=k_2$ (i.e. the two materials are the same) then $T_{i+}^1=T_{i+}^2$ and $T_{i-}^2=T_{i-}^1$ which indicates the material interface has disappeared.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same method can be applied to get a value at the ghost node near the boundary:&#xA;$$T_{b+}^2 = \frac{1-\mathrm{Nu}/2}{1+\mathrm{Nu/2}}T_{b-}^2 + \frac{\mathrm{Nu}}{1+\mathrm{Nu}/2}T_a$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathrm{Nu}=\frac{h\Delta r}{k_2}$ is the Nusselt number which is the relative strength of convection to conduction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to get to this equation will be left as an exercise for the reader.&lt;/p&gt;&#xA;" OwnerUserId="18017" LastActivityDate="2017-07-18T08:42:59.110" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27436" PostTypeId="2" ParentId="27416" CreationDate="2017-07-18T22:46:47.503" Score="0" Body="&lt;p&gt;In SEM there is usually an explicit formula for the action of the local matrices on local vectors. The local matrices can be thought of as numerically precomputed operators that evaluate: interpolation, differentiation, and integration. People precompute them in tabulated form as matrices because that is the only way to produce a sparse matrix, which is the format direct solvers like LU,Cholesky,... expect. For SEM though these local matrices require a lot of memory, and iterative solvers do &lt;strong&gt;not&lt;/strong&gt; require a sparse matrix input, so we can avoid precomputing these operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to avoid precomputing these is to compute interpolants, integrals, and derivatives by way of three-term-recurrences from underlying orthogonal polynomial basis. In SEM most basis have such a recurrence relation, or are lightly modified versions of a basis which does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many cases also you only need to store &quot;reference&quot; matrices by precomputing these operators on a reference element, and then compute the desired actions of these by geometric transformations alone. You can store geometric transformation factors for each element of the mesh, or even recompute that information at need to save even more memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This only does the local action though. It's still up to you to get inter-element details right. You will need to store some kind of connectivity graph(s) for your mesh and use that information to piece together all the local matrix-vector products into a full action of the operator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That full operator evaluation can be fed into a Krylov solver as a black-box function, and that is enough information for it to try and solve your resulting system.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2017-07-20T14:05:56.313" LastActivityDate="2017-07-20T14:05:56.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27437" PostTypeId="2" ParentId="7281" CreationDate="2017-07-19T00:11:43.107" Score="-2" Body="&lt;p&gt;There is a problem with a SUPG / PSPG implementation. The test does not converge. How do I solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="24916" LastActivityDate="2017-07-19T00:11:43.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27438" PostTypeId="2" ParentId="27218" CreationDate="2017-07-19T15:21:31.787" Score="1" Body="&lt;p&gt;Linearization of nonlinear PDEs is usually done by using the Frechet derivative of the system.&#xA;First, let us define $T=\left(\begin{array}{c}T_1\\T_2\end{array}\right)$.&#xA;Then, in your coupled system you can always replace $T_1=\left(\begin{array}{cc}1&amp;amp;0\end{array}\right)T$ and $T_2=\left(\begin{array}{cc}0&amp;amp;1\end{array}\right)T$ and write it as a coupled system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_tT= F(T)$$.&#xA;Now, you are looking for the frechet derivative of $F$. $$F_T\Delta:=\lim_{\epsilon\rightarrow0}\frac{F(T+\epsilon\Delta)-F(T)}{\epsilon}$$.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2017-07-19T15:21:31.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27439" PostTypeId="1" CreationDate="2017-07-19T18:42:15.803" Score="1" ViewCount="110" Body="&lt;p&gt;&lt;del&gt;Helmholtz&lt;/del&gt; Diffusion equation with reaction term:&#xA;$$&#xA;k\Delta u + u = f ~ \text{in} ~\Omega&#xA;$$&#xA;$$&#xA;\nabla u \cdot \mathbf{n} = 0 ~ \text{in} ~\partial \Omega&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sufficiently small $k$ (relative to the mesh), there can be oscillations (with Lagrange elements) that return a negative $u$ in some regions, specially when $f$ is a step function. I tried to apply a DG method with the Symmetric Interior Penalty formulation, and I am still seeing these negative values. Is this possible to happen? I was told that DG methods preserve positivity and cannot return negative values. Is there any way to preserve positivity in DG methods for elliptic equations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Preserve positivity: If $f(x)&amp;gt;0 ~\forall x\in\Omega$, then $u(x)&amp;gt;0 ~\forall x\in\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit again: What if the boundary conditions are of zero flux? Edited the problem.&lt;/p&gt;&#xA;" OwnerUserId="7023" LastEditorUserId="7023" LastEditDate="2017-07-22T17:19:27.513" LastActivityDate="2017-07-22T17:19:27.513" Title="Do DG methods for the Helmholtz equation always return positive quantities?" Tags="|finite-element|discontinuous-galerkin|elliptic-pde|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27440" PostTypeId="2" ParentId="27272" CreationDate="2017-07-19T18:50:28.897" Score="1" Body="&lt;p&gt;This is essentially a linear fit of the data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a_{n}=a_{n−1}+T(1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is obvious once you realize that an AP forms a straight line when plotted in 2-dimensions. Any linear regression tool (even Excel) should suffice for this exercise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T$ is given by the slope of the fit. &lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2017-07-19T18:50:28.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27441" PostTypeId="1" AcceptedAnswerId="27449" CreationDate="2017-07-19T20:07:53.470" Score="3" ViewCount="1952" Body="&lt;p&gt;I need to integrate the following function on the line segment from &#xA;$P_{1} = \begin{bmatrix}&#xA;-2\\-1&#xA;\end{bmatrix}$ to&#xA;$P_{2} = \begin{bmatrix}&#xA;1\\2&#xA;\end{bmatrix}$:&#xA;$$\int_{P_{1}}^{P_{2}} 4x + y \ ds$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question take part into the implementation of a 2D finite element solver. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;How I plan to do it&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Suppose that the following transformation is used to transform a general triangular element K to the standard triangular element $T_{st}$ :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x = P(\xi,\eta) $$&#xA;$$y = Q(\xi,\eta)$$&#xA;This corresponds to this physical mapping in fact:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ONjdc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ONjdc.png&quot; alt=&quot;Mapping&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we have:&#xA;$$dx = \frac{\delta x}{\delta \xi}d\xi + \frac{\delta x}{\delta \eta}d\eta = J_{11}d\xi + J_{21}d\eta $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$dy = \frac{\delta y}{\delta \xi}d\xi + \frac{\delta y}{\delta \eta}d\eta = J_{12}d\xi + J_{22}d\eta $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Along the side $P_{1} - P_{2}$, the coordinate $\xi$ is in fact always fixed. So we can write $d\eta = 0$ . Using this it comes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ dx = (\frac{\delta x}{\delta \xi})_{\eta = 0} \ d\xi  = J_{11}(\xi,0) d\xi $$&#xA;$$ dy = (\frac{\delta y}{\delta \xi})_{\eta = 0} \ d\xi  = J_{12}(\xi,0) d\xi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore:&#xA;$$ ds = \sqrt{J_{11}^{2}(\xi,0)+ J_{12}^{2}(\xi,0)} d\xi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can rewrite the integral with a variable change accordingly:&#xA;$$\int_{P_{1}}^{P_{2}} 4x + y \ ds = \int_{0}^{1} B(P(\xi,0),Q(\xi,0)) \ \sqrt{J_{11}^{2}(\xi,0)+ J_{12}^{2}(\xi,0)} d\xi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this isoparametrical formulation, we can finally compute the numerical integral thanks to a 4 node quadrature :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ I = \sum_{i=1}^{gauss \ point} w_{i} B(P(\xi_{i},0),Q(\xi_{i},0)) \ \sqrt{J_{11}^{2}(\xi_{i},0)+ J_{12}^{2}(\xi_{i},0)} d\xi $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done with this really simple code in Python (at least I was thinking):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit of the code : 20.07.2017 =&gt; &lt;strong&gt;removed an error (dividing by two without reason)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# -*- coding: utf-8 -*-&#xA;from __future__ import division # avoid integer problem of division&#xA;&#xA;#Import zone&#xA;import numpy as np&#xA;import math&#xA;&#xA;&#xA;#Define shape function&#xA;def P1shapes(r,s):&#xA;    S = np.array([1-r-s,r,s])&#xA;    dSdr = np.array([-1,1,0])&#xA;    dSds = np.array([-1,0,1])&#xA;    return S,dSdr,dSds&#xA;&#xA;&#xA;#Jacobian function&#xA;def isopmap(x,y,r,s,shapefcn):&#xA;    # x = vector of x coordinate of the element's point&#xA;    #shapefcn = P1shapes&#xA;    S,dSdr,dSds = shapefcn(r,s);&#xA;    j11=np.dot(dSdr,x)&#xA;    j12=np.dot(dSdr,y)&#xA;    j21=np.dot(dSds,x)&#xA;    j22=np.dot(dSds,y)&#xA;    detJ=j11*j22-j12*j21&#xA;    dSdx=( j22*dSdr-j12*dSds)/detJ&#xA;    dSdy=(-j21*dSdr+j11*dSds)/detJ&#xA;    return S,dSdx,dSdy,detJ,j11,j12,j21,j22&#xA;&#xA;#Gauss point quadrature&#xA;qwgts=np.array([-27/48,25/48,25/48,25/48])&#xA;rspts=np.array([[1/3,1/3],&#xA;[0.2,0.2],&#xA;[0.6,0.2],&#xA;[0.2,0.6]])&#xA;&#xA;&#xA;def B(x,y):&#xA;    z = 4 *x + y&#xA;    return z&#xA;&#xA;#Point of the 3 nodes triangles (test case)&#xA;x = [-2, 1 ,-1]&#xA;y = [-1 ,2 ,3]&#xA;&#xA;#P1-P2&#xA;int_total = 0&#xA;&#xA;#Begin integral on segment&#xA;for q in range(len(qwgts)):&#xA;    r = rspts[q,0] # r coordinate of the q_th quadrature point&#xA;    s = rspts[q,1] # s coordinate of the q_th quadrature point&#xA;    #Define ds and Map x_physical, y_physical&#xA;    S,dSdx,dSdy,detJ,j11,j12,j21,j22 = isopmap(x,y,r,0,P1shapes)&#xA;    ds = math.sqrt(math.pow(j11,2) + math.pow(j12,2))&#xA;&#xA;    x_physical = np.dot(x,S)&#xA;    y_physical = np.dot(y,S)&#xA;&#xA;    B_gausspoint = B(x_physical,y_physical)&#xA;    wxarea = B_gausspoint*qwgts[q]*ds&#xA;    int_total += wxarea&#xA;&#xA;print int_total&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result gives:&#xA;$$ I =  -16.97 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course if I change the third point of the triangle, it should change nothing. With this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = [-2, 1 ,-1]&#xA;y = [-1 ,2 ,7]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We again find for the path $P_{1} - P_{2}$:&#xA;$$ I =  -16.97 $$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What we should expect&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;From an analytical point of view we have for the parametrisation $x = -2 + 3t$ and $y = -1 + 3t$. This allow to have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ds = \sqrt{   (\frac{dx}{dt})^{2}  +  (\frac{dy}{dt})^{2}} \ dt = 3\sqrt{2} \ dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we have: &#xA;$$\int_{P_{1}}^{P_{2}} 4 x + y \ ds = \int_{0}^{1} 4(-2+3t) + (-1+3t)) \ 3\sqrt{2} \ dt = -6.36$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, we can see that the numerical method is not working...&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;My question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Did I make a mystake in the main concept? Is it an implementation error?&#xA;I think it could be an interesting question for all the people that try to implement a path integral on a FEM mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really did'nt find litterature example to check my implementation.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Extension of the question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Is there an other way/more elegant way to compute a path integral on the boundary of my FEM model? It seems that I have everything (non linear solver, quadrature integral for assembly of stiffness matrix, shape function, isoparametric formulation), but I'm really stuck at this point. Without this I never would be able to compute for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ R_{ij} = \int \kappa(x,y) \ \phi_{i} \ \phi_{j} \ dS \ for \ i \ = \ 1,2 $$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;EDIT 20.07.17 : Additional test&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Actually, if we take for the integral $B(x,y) = 1$, we are integrating the length of the path :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def B(x,y):&#xA;    z = 1&#xA;    return z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we run the code, we end up with :&#xA;$$ I = 4.24$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the correct value since the length of the path:&#xA;$$\sqrt{(P_{2}^{x} -P_{1}^{x})^{2} + (P_{2}^{y} -P_{1}^{y})^{2} } = 4.24$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Correct code (see explanation on accepted answer)&lt;/h1&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# -*- coding: utf-8 -*-&#xA;from __future__ import division # avoid integer problem of division&#xA;&#xA;#Import zone&#xA;import numpy as np&#xA;import math&#xA;&#xA;&#xA;#Define shape function&#xA;def P1shapes(r,s):&#xA;    S = np.array([1-r-s,r,s])&#xA;    dSdr = np.array([-1,1,0])&#xA;    dSds = np.array([-1,0,1])&#xA;    return S,dSdr,dSds&#xA;&#xA;&#xA;#Jacobian function&#xA;def isopmap(x,y,r,s,shapefcn):&#xA;    # x = vector of x coordinate of the element's point&#xA;    #shapefcn = P1shapes&#xA;    S,dSdr,dSds = shapefcn(r,s);&#xA;    j11=np.dot(dSdr,x)&#xA;    j12=np.dot(dSdr,y)&#xA;    j21=np.dot(dSds,x)&#xA;    j22=np.dot(dSds,y)&#xA;    detJ=j11*j22-j12*j21&#xA;    dSdx=( j22*dSdr-j12*dSds)/detJ&#xA;    dSdy=(-j21*dSdr+j11*dSds)/detJ&#xA;    return S,dSdx,dSdy,detJ,j11,j12,j21,j22&#xA;&#xA;#Gauss point quadrature    &#xA;def transfo1D(a,b,ti):&#xA;    #Inverse mapping to find r coordinate from 2D [r,0] space corresponding to 1D space [-1;1] defined by t&#xA;    epsylon_i = ((b-a)/2.0)*ti + ((b+a)/2.0)&#xA;    return epsylon_i&#xA;&#xA;def B(x,y):&#xA;    z = 4*math.pow(x,3)&#xA;    return z&#xA;&#xA;&#xA;coordinate=np.array([-0.774596669,0.000000000,0.774596669])&#xA;weight=np.array([0.555555556,0.888888889,0.555555556])&#xA;&#xA;&#xA;#Point of the 3 nodes triangles (test case)&#xA;x = [-2, 1 ,-1]&#xA;y = [-1 ,2 ,3]&#xA;&#xA;#P1-P2&#xA;int_total = 0&#xA;&#xA;#Begin integral on segment&#xA;for q in range(len(coordinate)):&#xA;    ti = coordinate[q] # r coordinate of the q_th quadrature point&#xA;    #Transform this to r space [r,0]&#xA;    a = 0&#xA;    b = 1&#xA;    r = transfo1D(a,b,ti)&#xA;    dettransform = (b-a)/2.0&#xA;&#xA;    #Define ds = fct(r) = fct(r(t))&#xA;    S,dSdx,dSdy,detJ,j11,j12,j21,j22 = isopmap(x,y,r,0,P1shapes)&#xA;    ds = math.sqrt(math.pow(j11,2) + math.pow(j12,2))&#xA;&#xA;    #Define B(P(r,0),Q(r,0)) = B(P(r(t),0),Q(r(t),0))&#xA;    x_physical = np.dot(x,S)&#xA;    y_physical = np.dot(y,S)&#xA;    B_gausspoint = B(x_physical,y_physical)&#xA;&#xA;    wxarea = weight[q] * B_gausspoint* ds * dettransform&#xA;    int_total += wxarea&#xA;&#xA;print int_total&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24889" LastEditorUserId="24889" LastEditDate="2017-07-20T11:20:17.537" LastActivityDate="2017-07-20T14:44:31.923" Title="Line integral along the edge of an isoparametrically mapped triangle" Tags="|finite-element|boundary-conditions|quadrature|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27446" PostTypeId="1" AcceptedAnswerId="29685" CreationDate="2017-07-20T04:47:11.913" Score="1" ViewCount="289" Body="&lt;p&gt;I am doing an exercise using MPI to count frequencies of words distributed in several different files following similar steps in &lt;a href=&quot;http://www.cs.uwec.edu/~stevende/cs252/programs/mpiWordCount.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this instruction&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I met a problem with step 2. In my implementation, I first sent out locally counted word-count pairs into corresponding processors according to the words' hashcode. At the same time, each processor might need to receive word-count pairs sented from other processors. As a result, one processor will have to both send and receive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In processor 1, there are word-count pairs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;cat: 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;dog: 4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fox: 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In processor 2, there are word-count pairs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;deer: 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;cat: 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fox: 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;red: 3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and so on. Suppose the word &quot;cat&quot; is computed to be sent to processor 2, and the words &quot;deer&quot;, &quot;fox&quot;, &quot;red&quot; are supposed to be sent to processor 1. So the pseudocode for each processor would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for each word-count pair:&#xA;    compute the word hash code&#xA;    MPI_Issend it to corresponding processor according to the hash code&#xA;&#xA;while (true):&#xA;    MPI_Recv word-count pair&#xA;    combine the counts of the same word in this processor&#xA;&#xA;MPI_Waitall&#xA;Do some other things&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the second loop will never stop, so I need to add a termination condition. My way of implementing this is to send a signal from the manager processor with rank 0 to each worker processor. So the second loop becomes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while (true):&#xA;    MPI_Recv word-count pair&#xA;    determine sender from the receive status&#xA;    if (sender != manager)&#xA;        combine the counts of the same word in this processor&#xA;    else&#xA;        break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this algorithm turned out to be not stable. Although it will get the intended result in most runs, it can enter into deadlock. My guess for the failure is that some workers may receive the termination signal from the manager too early  before receiving all the word-count pairs from other worker processors, so the corresponding MPI_Issend from those workers would have to wait indefinitely, thus a deadlock.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish someone could give some suggestions of improvement on my implementation. Or if someone have better algorithms for this step, that would also be appreciated! Thank you for your attention! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to prevent the manager from sending termination signals before any workers finish receiving word-count pairs, I summed up the total number of words from all processors involved in the shuffle stage, say $N$. Then, every time a processor receives a word-count pair, it should also send a signal to the manager. Accordingly, in the manager part, a loop is added to Recv the these signals, so that the manager will only send the termination signals when all the word-count pairs are received by workers. In this way, the manager signal will never interrupt with content sent from workers and thus the recv loop in the worker can terminate correctly. Below is the final modified pseudocode for this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Manager Part:&#xA;for (i = 1 : N)&#xA;    MPI_Recv signal with tag &quot;recvd&quot;&#xA;MPI_Send termination signal to all processors involved in the shuffle stage&#xA;&#xA;&#xA;Worker Part:&#xA;for each word-count pair:&#xA;    compute the word hash code&#xA;    MPI_Issend it to corresponding processor according to the hash code&#xA;&#xA;while (true):&#xA;    MPI_Recv word-count pair&#xA;    **MPI_Send signal with tag &quot;recvd&quot; to notify the manager that a wcp is received** &#xA;    if (sender != manager)&#xA;        combine the counts of the same word in this processor&#xA;    else&#xA;        break&#xA;&#xA;MPI_Waitall&#xA;Do some other things&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Although this algorithm worked successfully, I'm not sure whether I'm on the right track. Am I over complexing this problem?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="18224" LastEditDate="2017-07-24T03:18:40.020" LastActivityDate="2018-06-11T14:05:36.043" Title="MapReduce with MPI question" Tags="|mpi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27447" PostTypeId="1" CreationDate="2017-07-20T06:48:06.490" Score="1" ViewCount="189" Body="&lt;p&gt;I am using Tchebyshev discretization to solve a system of PDEs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, I map the Tchebyshev space($\xi$, from -1 to 1) to physical space ($x$, from 0 to L) using $$x = (\xi +1)*L/2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I also want my grids to be clustered around some point($x_c$) in the domain. Is there any standard mapping that can achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion/ reference to standard texts are greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="20670" LastActivityDate="2017-07-25T07:25:17.980" Title="Grid mapping from Tchebyshev" Tags="|grid|mapping-strategy|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27449" PostTypeId="2" ParentId="27441" CreationDate="2017-07-20T10:47:30.320" Score="4" Body="&lt;p&gt;Interestingly enough, you are using quadrature rule for a master &lt;strong&gt;triangle&lt;/strong&gt; in order to integrate over a &lt;strong&gt;segment&lt;/strong&gt;. You should use quadrature rule for a master segment (e.g. $[-1, 1]$) instead. You should also provide proper transformations $\mathbf r_i : [-1, 1] \rightarrow \partial K_i$, $i = 1, 2, 3$ for each edge of a physical triangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way of doing this is via introducing extra mappings $\mathbf T_i : [-1, 1] \rightarrow \partial \hat K_i$, $i = 1, 2, 3$ for each edge of the master triangle $\hat K$.&#xA;If $\hat K$ is spanned by vertices $\{(0,0), (1, 0), (0, 1)\}$, then &#xA;$$&#xA;\mathbf T_1(t) = \langle \, .5(1-t), .5(1+t) \, \rangle, \\&#xA;\mathbf T_2(t) = \langle \, 0, .5(1-t) \, \rangle, \\&#xA;\mathbf T_3(t) = \langle \, .5(1+t), 0 \, \rangle.&#xA;$$ &#xA;(I assume that the $i$th edge is the edge against the $i$th vertex.) Then you can set $\mathbf r_i := \mathbf T \circ \mathbf T_i$, where $\mathbf T : \hat K \rightarrow K$ is a usual mapping from the master triangle to the physical one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your element Robin matrix can be computed as&#xA;$$&#xA;\mathbf R^K_{ij} := \int_{\partial K_m} \kappa \, s^K_j \, s^K_i \, \text{d}s&#xA; = &#xA;\int_{-1}^{1} \left( \kappa \, s^K_j \, s^K_i \right) \circ \mathbf r_m \, || \mathbf r_m' || \, \text{d}t = \\&#xA;\int_{-1}^{1} \left( \kappa \circ \mathbf T \circ \mathbf T_m \right) \, \left( \hat s_j \circ \mathbf T_m \right) \, \left( \hat s_i \circ \mathbf T_m \right) \, || (\nabla \mathbf T \circ \mathbf T_m) \, \mathbf T_m' || \, \text{d}t.&#xA;$$ &#xA;Note that this choice of transformations $\mathbf r_i$ allows you to use master shape functions (so you can compute their images of quadrature nodes only once and then use them for every physical edge). Note also that the “length element” is constant unless you are using curvilinear mapping $\mathbf T$ (defined e.g. via $P2$ or $P3$ Lagrange shape functions).&#xA;Finally, note that you have to use quadrature rule for the master segment $[-1, 1]$. &lt;/p&gt;&#xA;" OwnerUserId="21916" LastEditorUserId="21916" LastEditDate="2017-07-20T14:44:31.923" LastActivityDate="2017-07-20T14:44:31.923" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27450" PostTypeId="1" AcceptedAnswerId="27451" CreationDate="2017-07-20T13:45:39.013" Score="2" ViewCount="2005" Body="&lt;p&gt;I'm attempting to use scipy's &lt;code&gt;spilu&lt;/code&gt; routine as a preconditioner and I'm finding bad performance for my application (solving a global linear system arising from a DG discretization of an time-dependent ADR PDE). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before I write tests to tune the options and compute the effect of the preconditioner on the eigenvalues of my linear system, I'd like to make sure I'm applying the preconditioning matrix in the &lt;code&gt;scipy&lt;/code&gt; intended manner, since the documentation for &lt;code&gt;spilu&lt;/code&gt; is pretty spartan...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;elif solver == 'spILUPCG':&#xA;    P = sp.sparse.linalg.spilu(A.tocsc())&#xA;    P = P.L * P.U&#xA;    x = sp.sparse.linalg.cg(A, b, M=P)&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="20688" LastEditDate="2019-11-07T02:28:17.190" LastActivityDate="2019-11-07T02:28:17.190" Title="Correct use of scipy's sparse.linalg.spilu" Tags="|sparse-matrix|scipy|preconditioning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27451" PostTypeId="2" ParentId="27450" CreationDate="2017-07-20T14:05:11.260" Score="6" Body="&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\mathbf L$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf U$&lt;/span&gt; give an approximate factorization of &lt;span class=&quot;math-container&quot;&gt;$\mathbf A$&lt;/span&gt;, you wouldn't want to use &lt;span class=&quot;math-container&quot;&gt;$\mathbf P = \mathbf L\cdot \mathbf U$&lt;/span&gt; as a preconditioner (that's approximately &lt;span class=&quot;math-container&quot;&gt;$\mathbf A$&lt;/span&gt;), you'd want to use &lt;span class=&quot;math-container&quot;&gt;$\mathbf P = \mathbf  U^{-1}\cdot \mathbf  L^{-1}$&lt;/span&gt; (that's approximately &lt;span class=&quot;math-container&quot;&gt;$\mathbf A^{-1}$&lt;/span&gt;). Even then, you would not want to multiply them out explicitly into &lt;span class=&quot;math-container&quot;&gt;$\mathbf P$&lt;/span&gt;, it's probably dense. Instead think of &lt;span class=&quot;math-container&quot;&gt;$\mathbf P$&lt;/span&gt; as an operation/operator that cascades two solve steps (first solve by &lt;span class=&quot;math-container&quot;&gt;$\mathbf L$&lt;/span&gt;, then solve by &lt;span class=&quot;math-container&quot;&gt;$\mathbf U$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect there to be some way to inject that operation as a closure/callback to precondition &lt;code&gt;cg()&lt;/code&gt;. Sadly I'm not a scipy expert, so I don't know the exact way to do it. But somewhere scipy should have an abstraction for applying the action of a linear operator, and I would expect both their matrix objects and their preconditioner objects to fulfill that abstraction. I suppose I'd try just passing your &lt;span class=&quot;math-container&quot;&gt;$\mathbf P$&lt;/span&gt; object itself.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2019-11-06T20:29:23.510" LastActivityDate="2019-11-06T20:29:23.510" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="27454" PostTypeId="1" AcceptedAnswerId="27457" CreationDate="2017-07-20T17:58:46.120" Score="4" ViewCount="646" Body="&lt;p&gt;The finite difference matrix for the first derivative is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{bmatrix} -1 &amp;amp; 1 &amp;amp; 0 \\ 0 &amp;amp; -1 &amp;amp; 1 \\ 0 &amp;amp; 0 &amp;amp; -1 \end{bmatrix}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite difference matrix for the second derivative is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{bmatrix} -2 &amp;amp; 1 &amp;amp; 0 \\ 1 &amp;amp; -2 &amp;amp; 1 \\ 0 &amp;amp; 1 &amp;amp; -2 \end{bmatrix}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{bmatrix} -1 &amp;amp; 1 &amp;amp; 0 \\ 0 &amp;amp; -1 &amp;amp; 1 \\ 0 &amp;amp; 0 &amp;amp; -1 \end{bmatrix}&#xA;\begin{bmatrix} -1 &amp;amp; 1 &amp;amp; 0 \\ 0 &amp;amp; -1 &amp;amp; 1 \\ 0 &amp;amp; 0 &amp;amp; -1 \end{bmatrix} = &#xA;\begin{bmatrix} 1 &amp;amp; -2 &amp;amp; 1 \\ 0 &amp;amp; 1 &amp;amp; -2 \\ 0 &amp;amp; 0 &amp;amp; 1 \end{bmatrix}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does multiplying together two first derivative matrices not give the second derivative matrix?&lt;/p&gt;&#xA;" OwnerUserId="24583" LastActivityDate="2017-07-20T19:27:57.043" Title="Why does multiplying two first derivative finite difference matrices not give the matrix for the second derivative?" Tags="|finite-difference|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27455" PostTypeId="1" CreationDate="2017-07-20T18:27:06.847" Score="1" ViewCount="67" Body="&lt;p&gt;I'm trying to solve the following equation&#xA;$$u_t = u_{xx} + u(1-u^2), u_x(\pm 1) = 0,$$&#xA;using the Fourier cosine transform. The nonlinear term gives a convolution which I would rather avoid, which is why I wanted to try a pseudo-spectral method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB I have the following script (simplified as much as possible)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function TestCosineSeries()&#xA;&#xA;Nx = 32;&#xA;x = linspace(-1,1,Nx)';&#xA;f = tanh(1000*x);&#xA;&#xA;dt = 1e-3;&#xA;Nt = 100;&#xA;&#xA;for k = 1 : Nt&#xA;&#xA;    t = k*dt&#xA;    [t,y] = ode45(@(t,y)RHS(t,y,L),[0 dt],f);&#xA;    f = y(end,:);&#xA;&#xA;end&#xA;&#xA;end&#xA;&#xA;function duout = RHS(t,u,L)&#xA;&#xA;N = length(u);&#xA;&#xA;uhat = fft(u);&#xA;kappa = fftshift((-N/2:N/2-1)');&#xA;DDuhat = -pi^2*kappa.^2.*uhat;&#xA;&#xA;DDu = ifft(DDuhat);&#xA;&#xA;duout = DDu + (1-u.^2).*u;&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I imagine I'm doing something wrong in using the cosine series (maybe I can't use FFT). Any help would be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="24931" LastActivityDate="2017-07-20T18:27:06.847" Title="Pseudo-Spectral cosine transform" Tags="|matlab|spectral-method|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27456" PostTypeId="2" ParentId="27454" CreationDate="2017-07-20T18:59:22.417" Score="0" Body="&lt;p&gt;Your matrices are not well defined. The second derivative operator for example, takes three consecutive data values and outputs one second derivative value. That is not expressed by writing a square matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a well-recognised Calculus of Finite Differences. You can find books with that phrase in the title, and you will find everything to be consistent and useful&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-07-20T18:59:22.417" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27457" PostTypeId="2" ParentId="27454" CreationDate="2017-07-20T19:27:57.043" Score="5" Body="&lt;p&gt;Look at the operators $$D_+^2 u = \frac{u_{n+2} - 2 u_{n+1} + u_n}{\Delta x^2}.$$&#xA;If you taylor expand this for small $\Delta x$ you arrive at&#xA;$$D_+^2 u = u_{xx} - \Delta x u_{xxx} + O(\Delta x^2)$$.&#xA;Thus $D_+^2 = D^2$ in the limit as $\Delta x \to 0$ (as it should), but note the error is first order in $\Delta x$ and so it is not a great approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider instead, the one you are thinking of&#xA;$$D_- D_+ u = \frac{u_{n+1} - 2 u_n + u_{n-1}}{\Delta x^2}.$$&#xA;Again taking a taylor expansion we have&#xA;$$D_- D_+ u = u_{xx} + \frac{\Delta x^2}{12} u_{xxxx} + O(\Delta x^4).$$&#xA;Thus this approximation is actually better since the error goes down quadratically rather than linearly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt; in the limit they are equivalent.&lt;/p&gt;&#xA;" OwnerUserId="24931" LastActivityDate="2017-07-20T19:27:57.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27458" PostTypeId="2" ParentId="27446" CreationDate="2017-07-20T20:45:37.843" Score="1" Body="&lt;p&gt;What you are trying to do is difficult -- namely, to receive messages without knowing how many messages and from which processor. The problem is that you can never be sure that you've received all of them, or whether there are more coming but some processor is just slow to send something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to this problem is that you first need to figure out who each processor is going to send something to. Think of this as a $P\times P$ matrix: in row $p$, you're going to put a one in column $q$ if processor $p$ wants to send something to processor $q$, and a zero otherwise. Processor $q$ would then know to expect a message from all processors $p$ for which there is a one in column $q$ of the matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, each processor only knows its own row. To know what's in a column implies transposing this matrix. Take a look, for example, at the &lt;code&gt;compute_point_to_point_communication_pattern()&lt;/code&gt; function that does that (without building a matrix, of course, but conceptually that's what it is doing): &lt;a href=&quot;https://github.com/dealii/dealii/blob/master/source/base/mpi.cc#L91&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dealii/dealii/blob/master/source/base/mpi.cc#L91&lt;/a&gt;&#xA;Its documentation is here: &lt;a href=&quot;https://www.dealii.org/8.5.0/doxygen/deal.II/namespaceUtilities_1_1MPI.html#a89b9a3309dffffe1447758157a33dbb6&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dealii.org/8.5.0/doxygen/deal.II/namespaceUtilities_1_1MPI.html#a89b9a3309dffffe1447758157a33dbb6&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have this information, you can loop over all senders and wait for the data they want to send to you. (A better pattern is to just query for &lt;em&gt;any&lt;/em&gt; incoming message and process it, until you know that every one of the known senders has sent you their message.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2017-07-20T21:14:13.917" LastActivityDate="2017-07-20T21:14:13.917" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27459" PostTypeId="1" AcceptedAnswerId="27651" CreationDate="2017-07-21T03:08:30.290" Score="2" ViewCount="329" Body="&lt;p&gt;I am trying to use a combination of JST and ROE scheme for the RANS solution of an aerodynamic problem. I notice that the ROE scheme performs poorly in terms of the convergence in the drag counts for higher angle of attacks compared to the JST scheme. I am now wondering if to run the ROE scheme to a certain degree of convergence and then start the JST scheme for final convergence. Will the spatial order of accuracy of the ROE scheme be preserved? Do i get a more accurate solution than just running the JST scheme from the beginning?&lt;/p&gt;&#xA;" OwnerUserId="23149" LastEditorUserId="23149" LastEditDate="2017-07-21T06:54:49.203" LastActivityDate="2017-08-17T12:30:15.903" Title="Order of accuracy in a combined scheme" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27460" PostTypeId="2" ParentId="27447" CreationDate="2017-07-21T12:08:24.390" Score="2" Body="&lt;p&gt;In the section about adaptive Methods Chapter 16. in &quot;Chebyshev and Fourier Spectral Methods&quot; from John P. Boyd several different coordinate transformations together with their application in different publications are presented. I have not used any of those transformation myself but they can serve as a starting point for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iv48u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iv48u.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&#xA;y is the physical (unmapped) coordinate and x is the computational coordinate.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2017-07-21T12:08:24.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27461" PostTypeId="1" CreationDate="2017-07-21T16:56:02.720" Score="1" ViewCount="659" Body="&lt;p&gt;I would like to do the following operation: I have a &quot;4D&quot; matrix A and a &quot;3D&quot; matrix B. Both A and B are actually 2D matrices, where for A, each element is a 2D matrix, and for B, each element is a 1D matrix or vector. I would like to, for each element in the 2D floor plan, multiply the matrix on A and the matrix in B. Graphically,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hvaua.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hvaua.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, my code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% size(A) = Nx, Ny, n, n&#xA;% size(B) = Nx, Ny, n&#xA;% size(C) = Nx, Ny, n&#xA;&#xA;for k1 = 1 : Nx&#xA;for k2 = 1 : Ny&#xA;    C(k1,k2,:) = squeeze(A(k1,k2,:,:)) * squeeze(B(k1,k2,:));&#xA;end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am looking for a way to &quot;vectorize&quot; the code and avoid loops completely. I am hoping such a vectorization will speed up my program.&lt;/p&gt;&#xA;" OwnerUserId="21221" LastActivityDate="2017-07-21T16:56:02.720" Title="Vectorizing Matrix Multiplication" Tags="|matlab|matrix|performance|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27462" PostTypeId="1" CreationDate="2017-07-21T17:03:48.057" Score="1" ViewCount="70" Body="&lt;p&gt;Let us consider a function $f(r,\theta,\phi)$ that can only be efficiently computed in spherical coordinates. We need its Cartesian gradient. We can implement its analytical spherical derivatives, compute the spherical gradient, $\left(\frac{\partial f}{\partial r},\frac{1}{r}\frac{\partial f}{\partial \theta},\frac{1}{r\sin\theta}\frac{\partial f}{\partial \phi}\right)$, and convert the gradient to Cartesian component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would you do with the values along the $z$-axis, where $\theta$ is undefined and usually set to 0, and in the origin, where $r$ is 0 as well? More specifically, what would you do with the terms in the denominators $r$ and $r\sin\theta$?&lt;/p&gt;&#xA;" OwnerUserId="4826" LastEditorUserId="4826" LastEditDate="2017-07-21T19:10:58.277" LastActivityDate="2017-07-21T19:10:58.277" Title="Spherical Gradient along $z$-axis" Tags="|spherical-harmonics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27463" PostTypeId="1" CreationDate="2017-07-21T21:40:06.733" Score="1" ViewCount="93" Body="&lt;p&gt;The context of my question is how to compute high order derivates on direct numerical simulation of turbulent channel flow. It is of particular interest for fluid dynamics and turbulence research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find the coefficients of a B-spline curve given the value of the resulting function on a set of collocatioin points. Is there any way to do it without resorting to solve a linear system each time I need to find the coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My specific case is to interpolate using a 7-th order B-spline scheme with knots given, but the values are set on 'Greville Abscisae'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The knots are given in this address: &#xA;    &lt;a href=&quot;http://turbulence.pha.jhu.edu/docs/channel/y-knots.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://turbulence.pha.jhu.edu/docs/channel/y-knots.txt&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the collocation ponits in this: &#xA;    &lt;a href=&quot;http://turbulence.pha.jhu.edu/docs/channel/y.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://turbulence.pha.jhu.edu/docs/channel/y.txt&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though not necessary, you can query any example of such coefficients on this website, just inputing the collocation points on this website:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://turbulence.pha.jhu.edu/webquery/query.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://turbulence.pha.jhu.edu/webquery/query.aspx&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quantity I am seeking to interpolate is the velocity field for the channel dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never worked with b-splines before, so I might be missing something, but I haven't found anywere a way to compute the coefficients from the collocation points, and this is bothering me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve a linear system for each line (in this case, a 512 x 506) would be bothersome, since I would need to do this for each vertical line on each snapshot of the simulation, and I would like to avoid this as much as possible, but I am not finding any other alternative. &lt;/p&gt;&#xA;" OwnerUserId="16394" LastActivityDate="2017-07-21T21:40:06.733" Title="Find B-spline coefficients from values on collocation points" Tags="|finite-difference|interpolation|spectral-method|b-spline|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27464" PostTypeId="2" ParentId="27361" CreationDate="2017-07-21T22:54:23.093" Score="2" Body="&lt;p&gt;The strategy would really be very similar -- partition your matrix and vectors into &quot;blocks&quot; that correspond to the variables living on the different phases. The difference is really just that the coupling isn't for all variables at the same location, but only for the variables located on different phases at the same points on interfaces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That means that the off-diagonal blocks are much sparser than they would be in a multiphysics problem because all of the variables in the interior of a phase never couple with any variable on a different phase and consequently there are no entries in the matrix for these variables outside the diagonal blocks of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is also that if you have a preconditioner for each phase (i.e., a block diagonal matrix), then it is likely a good preconditioner for the whole matrix because the off-diagonal blocks are mostly empty.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-07-21T22:54:23.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27465" PostTypeId="1" CreationDate="2017-07-22T03:07:02.227" Score="2" ViewCount="76" Body="&lt;p&gt;I have a number of evenly-spaced samples of functions that are polynomial in $1/x$ where $x$ is a continuous variable on $x\in\left[a,b\right]$, i.e. $$f(x_i)=\sum_{n=0}^{N} f_n x_i^{-n}.$$ I want to use some sort of local interpolation function like a hat function (linear interpolation), but it is pretty inaccurate (about 10% relative error in the $L_2$ norm) for values of $x$ near $a$. Near $b$, the interpolation is much better. I haven't yet implemented higher-order local Lagrange interpolation polynomials. I was wondering if anyone knows of a local interpolation method better suited to this problem. For reference, $N$ may be anywhere from about 5 up to 1000, and the number of samples reflects this. I'm not completely wedded to evenly-spaced samples, either, if a good interpolation method requires a different spacing. My thinking is that perhaps concentrating sampling points closer to $a$ and interpolating on a uniformly-spaced transform grid might work.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2017-07-22T03:07:02.227" Title="1-D local interpolation of sampled function that is polynomial in 1/x" Tags="|interpolation|local-coordinates|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27466" PostTypeId="1" AcceptedAnswerId="27470" CreationDate="2017-07-22T04:28:05.873" Score="2" ViewCount="151" Body="&lt;p&gt;I am solving the initial value problem&#xA;$$&#xA;\frac{d}{dt} (E C_g) = -\delta, \quad E(0) = E_0,&#xA;$$&#xA;for $E$, where $E$ and $C_g$ are functions of $t$, $C_g$ is completely known, and $\delta$ is a function of $E$. Using the following first order approximations for derivatives,&#xA;\begin{align*}&#xA;\frac{d}{dt} (E C_g) = E' C_g + E C_g', \\&#xA;E' \approx \frac{E_i - E_{i-1}}{\Delta t}, \quad C_g' \approx \frac{C_{gi} - C_{gi-1}}{\Delta t},&#xA;\end{align*}&#xA;we end up with the following finite difference scheme to step forward in time&#xA;\begin{align*}&#xA;E_i = \frac{-\delta_{i-1} \Delta t + 2E_{i-1} C_{gi-1} - E_{i-1}C_{gi}}{C_{gi-1}}.&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I implement this in MATLAB, I test my code by solving this IVP with a specific $E$, which leads to a specific right-hand side, and then can compute my error by comparing my computed solution to my true solution using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; error = norm(E_computed - E_true).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can then refine the grid spacing and save the spacing values in &quot;spacingArray&quot; and the error values in &quot;errorArray&quot;, then I can estimate the order of convergence using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; order = (log(errorArray(3)) - log(errorArray(6)))/(log(spacingArray(3)) - log(spacingArray(6))),&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but when I do this, I get order $\approx 0.5$, instead of order $\approx 1$ (this can also be seen by looking at a log-log plot of spacing against error). Does this mean that there is something wrong with the way that I have implemented the finite difference scheme in MATLAB, or could there be something else wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since my problem is a little more nuanced than I originally explained, here is the exact MATLAB code I run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Wave-energy model verification code for d/dx (E*C_g) = -delta&#xA;% To verify the model, we apriori choose H, h, and sigma;&#xA;% since d/dx (E*C_g) + delta = err, where err is some error.&#xA;% We obtain an analytical formula for err by determining&#xA;% d/dx (E*C_g) + delta (by using Mathematica or pencil/paper).&#xA;% Then, we solve the differential equation d/dx (E*C_g) = -delta + err,&#xA;% where the delta and err depend on our apriori chosen values of H, h,&#xA;% and sigma. &#xA;% 7/2/2017 9:38 AM&#xA;&#xA;%% Loop indices&#xA;start = 3; jump = 8;&#xA;&#xA;%% Initial arrays for testing&#xA;errorArray = zeros(jump, 1);&#xA;dxArray = zeros(jump, 1);&#xA;numPointArray = zeros(jump, 1);&#xA;&#xA;for kk = start:(start + jump)&#xA;&#xA;    %% Set up the &quot;mesh&quot;&#xA;    domain = [0,10];&#xA;    numPoint = 2^kk;&#xA;    numPointArray(kk - start + 1) = numPoint;&#xA;    x = linspace(domain(1), domain(2), numPoint);&#xA;    dx = (domain(2) - domain(1))/(numPoint - 1);&#xA;    dxArray(kk - start + 1) = dx;&#xA;&#xA;    %% Physical parameters&#xA;    rho = 1000; % water density&#xA;    g = 9.8;    % gravity &#xA;    gam = .78;  % ?&#xA;    rms = .707; % root-mean square constant&#xA;    B = 1;      % energy dissipation parameter&#xA;&#xA;&#xA;    %% Apriori choices for h, H, and sigma:&#xA;    h = @(x) 50 - x;&#xA;    h = h(x);&#xA;    h_prime = @(x) -1*ones(size(x));&#xA;    h_prime = h_prime(x);&#xA;    sigma = @(x) 2*ones(size(x));&#xA;    sigma = sigma(x); f = sigma/(2*pi);&#xA;    Ha = @(x) cos(x) + 5; % Ha = &quot;analytical height&quot;&#xA;    Ha = Ha(x);&#xA;    Ha_prime = @(x) -sin(x);&#xA;    Ha_prime = Ha_prime(x);&#xA;&#xA;&#xA;    %% Solving for wavenumber via the &quot;linear&quot; dispersion relation&#xA;    dispersionRelation = @(k) sigma.^2 - g.*k.*tanh(k.*h);&#xA;    k = ones(size(x)); % random initial guess for iterative solver&#xA;    k = fsolve(dispersionRelation, k); &#xA;&#xA;    %% Off-shore boundary conditions&#xA;    H = zeros(size(x)); E = zeros(size(x)); H_rms = zeros(size(x));&#xA;    H(1) = 6; % offshore wave height determines H_rms(1) and E(1)&#xA;    H_rms(1) = rms*H(1);        &#xA;    E(1) = (1/8)*rho*g*H_rms(1).^2;&#xA;    R = zeros(size(x)); delta = zeros(size(x));&#xA;    H_b = gam*h;&#xA;    R(1) = H_b(1)/H_rms(1);&#xA;&#xA;    %% Group Celerity&#xA;    c = sigma./k; &#xA;    C_g = (c/2).*(1 + (2*k.*h)/(sinh(2*k.*h)));&#xA;&#xA;    %% Energy flux, F = E*C_g&#xA;    F = zeros(size(x));&#xA;    F(1) = E(1)*C_g(1); % boundary condition&#xA;&#xA;    %% -------------------------------------------------------------------&#xA;    %% The analytical formulae (plural) for d/dx (E*C_g) from Mathematica&#xA;    % using shallow water approximation for wavenumber:&#xA;    theta1 = 2*sigma.*sqrt(h)/sqrt(g);&#xA;    shallowFluxDerivative = (1/16)*rho*g^(3/2)*(rms)^2*(Ha.^2.*(2.*sigma.*csch(theta1).* ...&#xA;        h_prime/sqrt(g) + h_prime./(2*sqrt(h)) - 2*sigma.^2.*coth(theta1).* ...&#xA;        csch(theta1).*sqrt(h).*h_prime/g) + 2*(sqrt(h) + 2*sigma.*csch(theta1).*h/sqrt(g)).* ...&#xA;        Ha.*Ha_prime);&#xA;    % using the deep water approximation for wavenumber:&#xA;    theta2 = 2*h.*sigma.^2/g;&#xA;    deepFluxDerivative = (1/16)*rho*g^2./sigma.*(rms)^2.*(Ha.^2.* (2*sigma.^2.*csch(theta2).*h_prime/g - ...&#xA;        4*sigma.^4.*coth(theta2).*csch(theta2).*h.*h_prime/(g^2) ) + 2*(1 + 2*sigma.^2.*csch(theta2).*h/g).* ...&#xA;        Ha.*Ha_prime);&#xA;&#xA;    %% Analytical formula for delta&#xA;    Ha_rms = rms*Ha;&#xA;    Ra = H_b./Ha_rms;&#xA;    deltaAnalytic = (1./(4.*h)).*B.*rho.*g.*(sigma./(2.*pi)).*Ha_rms.^3.*( (Ra.^3 + 1.5.*Ra).*exp(-Ra.^2) + 0.75.*sqrt(pi).*(1 - erf(Ra)));&#xA;&#xA;    %% Analytical &quot;err&quot; formula, d/dx(flux) + delta = err&#xA;    shallowError = shallowFluxDerivative + deltaAnalytic;&#xA;    deepError = deepFluxDerivative + deltaAnalytic;&#xA;    %% -------------------------------------------------------------------&#xA;&#xA;    %% Solve the ODE via finite differences&#xA;    for ii = 2:length(x)&#xA;        F(ii) = -delta(ii - 1)*dx + F(ii - 1); % finite difference formula&#xA;                                               % to compute new flux&#xA;&#xA;        % determine which regime we are in, then add the approriate error&#xA;        if h &amp;lt; 0.05*2*pi./k(ii) % shallow&#xA;            F(ii) = F(ii) + dx*shallowError(ii);&#xA;        elseif h &amp;gt; 0.5*2*pi./k(ii) % deep&#xA;            F(ii) = F(ii) + dx*deepError(ii);&#xA;        else % default to deep?...&#xA;            F(ii) = F(ii) + dx*deepError(ii);&#xA;        end&#xA;&#xA;        E(ii) = F(ii)./C_g(ii); % compute new energy from flux&#xA;        H(ii) = sqrt(8./(rho*g)*E(ii))/rms; % compute height from energy&#xA;        H_rms(ii) = rms*H(ii); &#xA;        delta(ii) = 1/(4*h(ii))*B*rho*g*f(ii)*H_rms(ii)^3*(R(ii)^3 + (3/2)*R(ii))*exp(-R(ii)^2) + ...&#xA;            (3/4)*sqrt(pi)*(1 - erf(R(ii))); % compute new delta&#xA;    end&#xA;&#xA;    %% post processing&#xA;    error = norm(H - Ha); % difference between computed and true solution&#xA;    errorArray(kk - start + 1) = error;&#xA;&#xA;end&#xA;&#xA;%% create a log-log plot&#xA;figure(), loglog(dxArray, errorArray), title('log-log plot of error against spacing')&#xA;axis equal, xlabel('log(dx)'), ylabel('log(norm(abs(H - H_a)))')&#xA;&#xA;%% determine the slope of the log-log graph &#xA;% this should match with the theoretical order of convergence&#xA;order = (log(errorArray(jump)) - log(errorArray(1)))/(log(dxArray(jump)) - log(dxArray(1)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For some reason, I can't save and upload the plot, so here is the outputted order of convergence:    0.512087039390502.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing that is interesting is that it looks like the order of convergence is a little bit better for the first refinement, and in fact I can compute &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; order = (log(errorArray(2)) - log(errorArray(1)))/(log(dxArray(2)) - log(dxArray(1)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which produces $\text{order} \approx 0.629469265412451$, which is at least a little closer...&lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2017-07-22T15:23:53.340" LastActivityDate="2017-07-23T22:16:32.800" Title="First-order ODE scheme implementation giving less than first-order convergence?" Tags="|finite-difference|convergence|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27467" PostTypeId="2" ParentId="27466" CreationDate="2017-07-22T05:33:23.427" Score="4" Body="&lt;p&gt;I cannot tell you whether that is the cause for your diminished order, but it is asking for trouble to add other approximations than just the one you want to do on purpose. In the current context, since you know $C_g$, there is no need to replace $C_g'$ by a finite difference quotient -- just use the explicit derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even better, introduce a new variable $\epsilon = EC_g$, express $\delta=\delta(E)=\delta(\epsilon/C_g)$ in terms of $\epsilon$, and then solve the ODE&#xA;$$&#xA;  \epsilon' = -\delta(\epsilon/C_g)&#xA;$$&#xA;instead -- there is no ambiguity how it should be discretized.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-07-22T05:33:23.427" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27469" PostTypeId="1" AcceptedAnswerId="27475" CreationDate="2017-07-22T16:59:18.680" Score="5" ViewCount="941" Body="&lt;p&gt;I have two matrices $A, B$ coming from a finite element discretization of a system of partial differential equations. $A$ represents the system matrix and is symmetric and indefinite. $B$ is symmetric positive definite and is used as a preconditioner. Direct solver is used for computation of action of $B^{-1}$. The preconditioner is quite good, $\sigma(B^{-1}A) \subset (-10, -0.1) \cup (0.1, 10)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I solve this system using GMRES preconditioned by $B$ I get the approximate solution after a reasonable number of iterations. However the same preconditioner fails when used in MINRES. The number of iterations for the preconditioned system is even higher than for unpreconditioned system. (I tried this computation using PETSc, Matlab and scipy with the same qualitative result.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please navigate me to some theoretical result or a keyword that would explain such behaviour?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I found a mistake in my code, the question is no longer valid, $B$ was (slightely) not symmetric which makes MINRES useless. My deepest apologies.&lt;/p&gt;&#xA;" OwnerUserId="24942" LastEditorUserId="24942" LastEditDate="2017-07-24T13:00:52.350" LastActivityDate="2017-07-24T14:53:52.883" Title="Difference in performance of preconditioned GMRES and MINRES" Tags="|linear-algebra|numerical-analysis|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27470" PostTypeId="2" ParentId="27466" CreationDate="2017-07-22T19:00:13.387" Score="5" Body="&lt;p&gt;Let me modify slighly your ODE to see the essence of the problem. Let&#xA;$$\left\{\begin{array}{l}y'=-y \\ y(0) = 1 \end{array} \right. \qquad t\in[0,T]\tag{*}$$&#xA;the ODE to be solved. Everyone knows that the exact solution of $(*)$ is:&#xA;$$y=\exp{(-t)} \tag{*a}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we discretise $(*)$ one would obtain:&#xA;$$\frac{y^{n+1}-y^n}{\Delta t}=-y^{n+1} \qquad y^0=1\tag{*b}$$&#xA;in which have followed the unconditionally stable scheme being the RHS of $(*)$ implicitly treated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact equation $(*b)$ solves with second order of accuracy the following problem:&#xA;$$\left\{\begin{array}{l}y'=-y-\Delta t\,y'' \\ y(0) = 1 \end{array} \right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exact numerical solution of $(*b)$ is:&#xA;$$y^n=(1+\Delta t)^{-n}$$&#xA;At time $t_i$, with $i\in[0,N]$ where $N$ is large wrt. $T$, the error would be &#xA;$$\epsilon_i=|y(t_i)-y^{i}|=|\exp{(-t_i)}-(1+\Delta t)^{-i}|=|\exp{(-iT/N)}-(1+T/N)^{-i}|\approx \frac{iT}{2N}\frac{T}{N}=\frac{t_i}{2}\Delta t=\mathcal{O}(\Delta t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, we can conclude from the latter equation that the error can be bounded by a constant that depends only on the domain in which $(*)$ is integrated, $i.e.$&#xA;$$\epsilon\leq C(T)\Delta t=T\,\Delta t \tag{*c}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This analysis can be performed with any ODE leading to the same order in $\Delta t$ according to the chosen scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conclusion $(*c)$ can be determined numerically if we notice that the error in the $i$th time is bounded by the same power $p$ of $\Delta t$ with the same proportionality constant, $i.e$ for two different meshes $\Delta t$ and $\Delta t'$ we will obtain errors $\epsilon$ and $\epsilon'$ respectively such that&#xA;$$\frac{\epsilon'_i}{\epsilon_i}=\left(\frac{\Delta t'}{\Delta t}\right)^{p} \tag{*d}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use $(*d)$ choose, for example the last node (which always independent on the used mesh and it coincides with $t_i=T$) and calculate $p$ with &#xA;$$p=\frac{\log{\left[\epsilon'(T)/\epsilon(T)\right]}}{\log{\left[\Delta t'/\Delta t\right]}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This must be always around $p=1$ and is always dependent on the discretisation method and the regularity properties of the $RHS$ (derivatives) of $(*)$ that can negatively interfere in the order of the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is more I leave you a code that summarises what I previously said:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T = 1;&#xA;&#xA;deltat1 = 1e-3; %Multiple of 10&#xA;deltat2 = 1e-4; %Multiple of 10&#xA;&#xA;%Exact solution evaluated at $T$&#xA;yTexact = exp(-T);&#xA;&#xA;%Numerical solution for the 1st mesh&#xA;N = T/deltat1;&#xA;y1 = (1+deltat1).^(-[0:N]);&#xA;y1T = y1(end); %Numerical solution at t=T&#xA;epsilon1 = abs(yTexact-y1T); %error for the last node&#xA;&#xA;%Numerical solution for the 2nd mesh&#xA;N = T/deltat2;&#xA;y2 = (1+deltat2).^(-[0:N]);&#xA;y2T = y2(end); %Numerical solution at t=T&#xA;epsilon2 = abs(yTexact-y2T); %error for the last node&#xA;&#xA;%Compute the method order:&#xA;p = log(epsilon2/epsilon1)/log(deltat2/deltat1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result is $p=0.998$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;If you compute the global error, $i.e.$ the norm (for $N$ large):&#xA;$$\sqrt{\sum{\epsilon_i^2}}\approx \frac{T}{2N}\frac{T}{N}\sqrt{\sum_{i=1}^{N}{i^2}}\approx \frac{T}{2N}\frac{T}{N}\sqrt{2}N^{3/2}=\mathcal{O}(\sqrt{\Delta t})$$&#xA;Which is the exponent you are obtaining. &#xA;$$p=0.5$$&#xA;Keep in mind that the pointwise error, which must be of the order of the time step, is not the same as what you have computed, which is another error.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-07-23T22:16:32.800" LastActivityDate="2017-07-23T22:16:32.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27471" PostTypeId="1" CreationDate="2017-07-22T20:54:04.327" Score="1" ViewCount="162" Body="&lt;p&gt;I have the following PDE in 2D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U_{x} + U_{xx} + U_y + U_{yy} + U_{xy} = f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I'm trying to create a matrix $A$ to solve the PDE through finite differences: $AU = f$. I will invert $A$ such that $U = A^{-1}f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have mixed boundary conditions in this form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0 = a_0U + a_1U_{x} + a_2U_{xx} + a_3U_y + a_4U_{yy} + a_5U_{xy}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm just not sure how I could specify or code this BC equation into my matrix. I did a lot of searching but most of the tutorials are for 1d equations only. I know about the ghost point method but I have only done it for $0 = a_0U + a_1U_{x}$. However, I'm not sure what I should do when it comes to the point at $f(x_1, y_1)$, since it would involve multiple ghost points ($f(x_0,y_1), f(x_1,y_0)$, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Separately, should $a_{0, ..., 5}$ be a function of $x$ or $y$?&lt;/p&gt;&#xA;" OwnerUserId="24946" LastEditorUserId="24946" LastEditDate="2017-07-23T21:23:51.463" LastActivityDate="2017-07-23T21:23:51.463" Title="How would you specify mixed boundary conditions for a 2D PDE in the matrix used for finite differences" Tags="|pde|finite-difference|boundary-conditions|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27472" PostTypeId="1" AcceptedAnswerId="27473" CreationDate="2017-07-23T08:41:06.617" Score="0" ViewCount="312" Body="&lt;p&gt;I'm looking for a good Matlab package/library for B-Splines approximation. Ideally, it would take knots $t_1, \ldots , t_n$, and data points $g(t_1),\ldots , g(t_n)$, and Produce $$Vg :\,= \sum\limits_{j=1}^n B_i ^k (t) g_i \, , $$ &#xA;where the $B_i ^k$'s are the B-spline basis of order $k$ with respect to the given knots-set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Remarks:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Really, I just need the cubic B-Splines, and &quot;simple&quot; knots, i.e., without repeating a note twice.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I follow the definitions in pp. 160 of C. De Boor, A Practical Guide to Splines, 1978.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15188" LastActivityDate="2017-07-23T09:55:06.493" Title="B-Splines Matlab Package" Tags="|matlab|approximation|b-spline|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27473" PostTypeId="2" ParentId="27472" CreationDate="2017-07-23T09:55:06.493" Score="2" Body="&lt;p&gt;You may use Curve fitting toolbox which is provided by MATLAB.&#xA;The function you need is &lt;code&gt;spcol&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16621" LastActivityDate="2017-07-23T09:55:06.493" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27475" PostTypeId="2" ParentId="27469" CreationDate="2017-07-23T17:24:28.893" Score="3" Body="&lt;p&gt;Are you applying MINRES to the operator $B^{-1}A$? Unless $B$ and $A$ commute, which is unlikely, the product $B^{-1}A$ is not symmetric. I'm guessing that this is what happened in your case because GMRES, which works even for nonsymmetric matrices, did converge, while MINRES did not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned using a direct factorization of $B$. Provided you used the Cholesky decomposition $B = LL^*$, then the matrix $L^{-1}AL^{-*}$ is symmetric. If you apply MINRES to this system you might have better luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a little weird that the preconditioned system requires more iterations than the original system. But, all the nonsymmetric iterative methods have weird failure modes and can be made to converge slowly on certain input matrices. See &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0613049&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for examples. In any case, you mentioned using PETSc, so you can always try other preconditioners and see what works best.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-07-23T17:24:28.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27476" PostTypeId="1" AcceptedAnswerId="27507" CreationDate="2017-07-24T04:23:49.920" Score="1" ViewCount="124" Body="&lt;p&gt;Let $F: \mathbb{R} \rightarrow \mathbb{R}$ be a linear map. I want to evaluate an expression of the type $$F((ax+b)^k)$$ in terms of $F(x)$ for some fixed value of $x$ (I already know $F(x)^r$ for $r=1,...,k$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x$ is typically small ($0&amp;lt;x&amp;lt;1$), and $k$ is typically about $15$. $a$ is always positive (about $30$) and $b$ is always negative (about $-30$) so that $ax+b$ is always between $0$ and $1$. Further it is also known that the range of the map $F$ is $[0,1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the binomial theorem to expand $(ax+b)^k$ involves very large numbers, which cause errors due to overflow (in Matlab).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is already known that the argument $(ax+b)^k$ as well as its image $F((ax+b)^k)$ are always small, I am interested to know if there are methods to evaluate $F((ax+b)^k)$ in terms of $F(x)$ stably or without involving large numbers. Any help will be much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; The actual problem I want to solve is not quite the same. I am describing it below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two functions $f,g:\mathbb{R} \to \mathbb{R}$. $g$ is positive and unit normalized, i.e. $\int_{-\infty}^{\infty} g(x)dx=1$ The function $f$ is not known, but it is known that $f(x) \in [0,1]$ for all $x$. Also the quantity $\int_{-\infty}^{\infty} g(x) f(x)^r dx$ is known for $r=1,...,k$ (which is, of course, in $[0,1]$). I want to calculate the quantity $$I = \int_{-\infty}^{\infty} g(x) (a f(x) + b)^k dx$$ where $a,b \in \mathbb{R}$. It is known that $a&amp;gt;0,b&amp;lt;0$ and $a,b$ are chosen such that $af(x)+b \in [0,1]$. Hence $I \in [0,1]$. $a$ is typically about $30$, and $k$ is about $15$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since only $\int_{-\infty}^{\infty} g(x) f(x)^r dx$ is known I have no option but to expand $(a f(x) + b)^k = \sum_{r=0}^k \binom{k}{r} a^r f(x)^r b^{k-r}$ using the binomial theorem. However this involves the product of large numbers like $\binom{k}{r}$ and powers of $a$ and $b$. Since $I \in [0,1]$, compuing $I$ by adding and subtracting such large nos. does not seem like a good idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the original question I intended the linear map $F$ to correspond to the map $f \mapsto \int_{-\infty}^{\infty} g(x) f(x) dx$, but that was clearly not a good example.&lt;/p&gt;&#xA;" OwnerUserId="24933" LastEditorUserId="24933" LastEditDate="2017-07-26T04:53:33.357" LastActivityDate="2017-07-26T21:14:49.193" Title="Numerically stable computation of $F((ax+b)^k)$" Tags="|linear-algebra|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27477" PostTypeId="1" CreationDate="2017-07-24T07:22:41.827" Score="0" ViewCount="486" Body="&lt;p&gt;(I'm not sure which of math.se / stackoverflow / scicomp.se is the right place to ask this question)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a C++ code which generates a complex matrix and then calculates its eigenvalues and eigenvectors using LAPACK zgeev. As an example this is one of those matrices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8QmX0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8QmX0.jpg&quot; alt=&quot;Matrix&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to test my code so I entered the same matrix in Matlab and Mathematica but I got two other different results! They all give the same eigenvalues but different eigenvectors. Here are my results from three programs. (Note that Matlab gives eigenvectors in column but the other two in rows and Matlab only shows 5 first digits so imaginary parts are shown 0 but I have entered them in the script)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KetOk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KetOk.jpg&quot; alt=&quot;Mathematica&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2jQAj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2jQAj.jpg&quot; alt=&quot;Matlab&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/83nzS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/83nzS.jpg&quot; alt=&quot;LAPACK&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is going on?!&lt;/p&gt;&#xA;" OwnerUserId="23599" LastEditorUserId="23599" LastEditDate="2017-07-24T08:15:40.093" LastActivityDate="2017-07-24T11:23:21.853" Title="Matlab, Mathematica &amp; LAPACK returning 3 different eigenvectors" Tags="|matlab|c++|eigenvalues|lapack|mathematica|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27479" PostTypeId="2" ParentId="27477" CreationDate="2017-07-24T09:39:12.150" Score="7" Body="&lt;p&gt;Seems that you have a duplicate eigenvalue. Thus, you have two eigenpairs $(\lambda_1, x_1)$ and $(\lambda_2, x_2)$ where $\lambda_1 = \lambda_2$. Denote $\lambda = \lambda_1 = \lambda_2$. Let $\alpha$ and $\beta$ be arbitrary complex numbers. Then&#xA;$$A (\alpha x_1 + \beta x_2) = \alpha A x_1 + \beta A x_2 = \alpha \lambda x_1 + \beta \lambda x_2 = \lambda (\alpha x_1 + \beta x_2).$$&#xA;It hence seems that $(\lambda, \alpha x_1 + \beta x_2)$ is also an eigenpair for arbitrary numbers $\alpha$ and $\beta$!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to pick which eigenpair to display? The program that you use must choose it somehow. The strategy seems to be different for the different programs you use.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2017-07-24T09:39:12.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27480" PostTypeId="2" ParentId="27477" CreationDate="2017-07-24T11:23:21.853" Score="3" Body="&lt;p&gt;There are two possible issues going on here. Firstly as noted in the answer by knl any linear combination of eigenvectors corresponding to degenerate eigenvalues is also an eigenvector, as shown in that answer. However even for the non-degenerate case eigenvectors may vary between different invocations of a diagonaliser even if the input matrix is the same. This is because there is an arbitrary random phase which the routine is free to &quot;pick&quot;, and may vary from call to call. To see this consider the eigenvalue equation&#xA;$${\bf A} {\bf q}=\lambda {\bf q}$$&#xA;Here $(\lambda,{\bf q})$ is clearly the eigenpair. Let's multiply this from the left with a diagonal matrix ${\bf D}$ where all the diagonal elements are the same and of the form $D_{ii}=e^{i\phi}$. Such matrices commute with any other matrix so it follows that&#xA;$${\bf DAq}={\bf A(Dq)}=\lambda ({\bf Dq})$$&#xA;Then let ${\bf q^{'}={\bf Dq}}$ we can see that&#xA;$${\bf q^{'}}^{\dagger} {\bf q^{'}}={\bf q^{\dagger}D^{\dagger}Dq}={\bf q^{\dagger}q}$$&#xA;because of the form of ${\bf D}$. Thus ${\bf q^{'}}$ is also an eignevector of ${\bf A}$, and further has the same normalisation, and thus is a perfectly valid result for (in your case) zheev to return whatever the value of $\phi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of this about the only good way to check an eigenvector is to make sure ${\bf q^{\dagger}Aq}=\lambda$ to whatever tolerance is acceptable; visual inspection og the elements doesn't work. Even in the real case there is still a factor of -1 that can be applied. &lt;/p&gt;&#xA;" OwnerUserId="17475" LastActivityDate="2017-07-24T11:23:21.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27481" PostTypeId="1" CreationDate="2017-07-24T12:44:31.893" Score="1" ViewCount="5877" Body="&lt;p&gt;I'm sure this would be a common question, however yet to find an answer for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have two, 2D vectors in Python&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RaceA =[[ Runner ID, Heart Rate, Kilometers Done,],&#xA;[1234, 67, 50],&#xA;[1256, 83,64],&#xA;[1356,92,82],&#xA;[845,80,18]]&#xA;&#xA;RaceB =[[ Runner ID, Heart Rate, Kilometers Done,],&#xA;[845, 79, 74],&#xA;[5363, 86,34],&#xA;[1256,84,53],&#xA;[12233,93,74],&#xA;[4233,40,34]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, both races had different racers. I'd like to compare the heart rates of racers who ran both of the races.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence my answer would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[Racer ID, Heart Rate A, Heart Rate B],&#xA;[1256,83,84],&#xA;[845,80,79]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've used the '&amp;amp;' operator and that returns me the racers IDs for example. But I feel like I'm missing out on a function that will return me heart rate values for both racers in a swift fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This obviously isn't my problem (I have vectors 6000 and 50000 rows long) but the premise is there.&lt;/p&gt;&#xA;" OwnerUserId="24952" LastEditorUserId="9667" LastEditDate="2017-07-24T15:01:42.223" LastActivityDate="2017-07-25T21:40:46.160" Title="Python: Comparing 2D arrays and Returning independent values from those arrays" Tags="|python|numpy|" AnswerCount="1" CommentCount="2" ClosedDate="2017-07-26T02:16:50.947" ContentLicense="CC BY-SA 3.0" />
  <row Id="27482" PostTypeId="2" ParentId="27469" CreationDate="2017-07-24T14:53:52.883" Score="3" Body="&lt;p&gt;Using a preconditioner $B$ for solving the system $Ax=b$ does &lt;em&gt;not&lt;/em&gt; correspond to solving the system $B^{-1}Ax = B^{-1}b$, since, as Daniel pointed out, $B^{-1}A$ is generally not symmetric. Rather, it corresponds to solving&#xA;$$&#xA;B^{-1/2} A B^{-1/2} \tilde{x} = B^{-1/2}b.&#xA;$$&#xA;You could also formally consider the system&#xA;$$&#xA;B^{-1}Ax = B^{-1}b&#xA;$$&#xA;and use the inner product $\left\langle\cdot,\cdot\right\rangle_B$, or the system&#xA;$$&#xA;AB^{-1}\tilde{x} = B^{-1}b&#xA;$$&#xA;and use the inner product $\left\langle\cdot,\cdot\right\rangle_{B^{-1}}$. All three viewpoints will lead to the exact same algorithm, the preconditioned MINRES. (The same with CG, btw.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the spectrum is equal for all three systems,&#xA;$$&#xA;\sigma(B^{-1/2} A B^{-1/2}) = \sigma(B^{-1}A) = \sigma(AB^{-1}),&#xA;$$&#xA;so looking at $\sigma(B^{-1}A)$ is okay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to why MINRES and GMRES differ in convergence behavior is hard to say. Perhaps you made the mistake of applying MINRES to $B^{-1}A$. If not, another explanation could be loss of orthogonality in MINRES. Note that for symmetric matrices in exact arithmetic MINRES and GMRES to &lt;em&gt;exactly&lt;/em&gt; the same thing. The difference is that GMRES keeps a basis of Krylov space to orthogonalize against, while MINRES relies on certain properties to only orthogonalize against the last few vectors. This could be error-prone. Since the two algorithms do exactly the same thing, some people say that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;GMRES is the most robust MINRES implementation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you're not short on memory, GMRES is certainly safe to use.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2017-07-24T14:53:52.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27485" PostTypeId="2" ParentId="27447" CreationDate="2017-07-25T05:25:56.427" Score="0" Body="&lt;p&gt;I have found a simple algebraic mapping (Eq 18.12 from &lt;a href=&quot;https://books.google.co.in/books?hl=en&amp;amp;lr=&amp;amp;id=Hm4ECAAAQBAJ&amp;amp;oi=fnd&amp;amp;pg=PA1&amp;amp;dq=%20Peyret,%20R.%20(2013).%20Spectral%20methods%20for%20incompressible%20viscous%20flow%20(Vol.%20148).%20Springer%20Science%20%26%20Business%20Media&amp;amp;ots=abSEpKmgd-&amp;amp;sig=H83SDXjauyaFAJczmK22HBtdQWQ#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)  and modified it to suit my need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mapping is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x = ((1-w)\xi^3 - x_p\xi^2 + w\xi + x_p + 1 )*(L/2) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x_p = \frac{2x_c}{L}-1$; $x_c$ is approximately the point around which you want to cluster the grids;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w\ (0 &amp;lt; w \ll 1)$ controls the width of the cluster.&lt;/p&gt;&#xA;" OwnerUserId="20670" LastEditorUserId="20670" LastEditDate="2017-07-25T07:25:17.980" LastActivityDate="2017-07-25T07:25:17.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27486" PostTypeId="1" CreationDate="2017-07-25T05:45:33.347" Score="1" ViewCount="625" Body="&lt;p&gt;This is a follow up to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/27447/grid-mapping-from-tchebyshev/27485#27485&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have mapped from Chebyshev space to the physical space using the mapping&#xA;$x = ((1-w)\xi^3 - x_p\xi^2 + w\xi + x_p + 1 )*(L/2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When in Chebyshev space I have easily defined the spectral differentiation matrix as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(D_N)_{00} = \frac{2N^2 +1}{6}$$&#xA;$$(D_N)_{NN} = -\frac{2N^2 +1}{6}$$&#xA;$$(D_N)_{jj} = -\frac{\xi_j}{2(1-\xi_j^2)}, \ \forall \ 1 \leq j \leq N-1$$ &#xA;$$(D_N)_{ij} = \frac{c_i}{c_j} \frac{(-1)^{i+j}}{\xi_i - \xi_j}, \ \forall \ i \neq j$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$    c_i= &#xA;\begin{cases}&#xA;    2,&amp;amp; \text{if } i = 0 \text{ or } N\\&#xA;    1,              &amp;amp; \text{otherwise}&#xA;\end{cases}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$    c_j= &#xA;\begin{cases}&#xA;    2,&amp;amp; \text{if } j = 0 \text{ or } N\\&#xA;    1,              &amp;amp; \text{otherwise}&#xA;\end{cases}$&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I find the analogous differentiation matrix for the mapping&lt;br&gt;&#xA;mentioned above? &lt;/li&gt;&#xA;&lt;li&gt;Will this mapping affect the stability and convergence of collocation method?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20670" LastActivityDate="2017-07-27T13:07:15.883" Title="Chebyshev spectral differentiation matrix for mapped domain" Tags="|spectral-method|chebyshev|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27487" PostTypeId="1" CreationDate="2017-07-25T07:26:52.177" Score="1" ViewCount="126" Body="&lt;p&gt;I am trying to implement in C or C++ a solution for a fft and Ifft when the signal values are not obtained at a constant rate, making it having a desviation between the values and the periodic ones. I do not know how I can change the algorithm of calculating the fft with synchronized signals as in &lt;a href=&quot;https://wikimedia.org/api/rest_v1/media/math/render/svg/02a7699d274d226871209491c8a05976d6e46cf3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wikimedia.org/api/rest_v1/media/math/render/svg/02a7699d274d226871209491c8a05976d6e46cf3&lt;/a&gt; into a uneven spacing values of signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of my ideas is using the values to make the completed signal and then catchiing the correct points, but it is a very time consuming solution, and i think that as we have a deviation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overall objective is to have the fft as described and the ifft values in that points&lt;/p&gt;&#xA;" OwnerUserId="24964" LastActivityDate="2017-08-05T19:04:39.613" Title="fft with non uneven spacing between the value of the signal" Tags="|fourier-analysis|fourier-transform|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27488" PostTypeId="2" ParentId="11249" CreationDate="2017-07-25T09:30:54.930" Score="0" Body="&lt;p&gt;To increase the precision of numerical differentiation do the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Chose your favorite high-precision &quot;standard&quot; method based on some step size &lt;em&gt;h&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Compute the value of the derivative with the method chosen in 1) many times with different but reasonable step sizes &lt;em&gt;h&lt;/em&gt;. Each time you may pick &lt;em&gt;h&lt;/em&gt; as a random number from the interval (0.5*H/10, 1.5*H/10) where H is an appropriate step size for the method you use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Average the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your result may gain 2-3 orders of magnitude in the absolute error wrt. the non-averaged result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1706.10219&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1706.10219&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24965" LastEditorUserId="24965" LastEditDate="2017-07-31T10:41:32.693" LastActivityDate="2017-07-31T10:41:32.693" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27489" PostTypeId="2" ParentId="21021" CreationDate="2017-07-25T10:02:06.373" Score="4" Body="&lt;p&gt;You have noticed that any eigenvector of $A$ is also an eigenvector of your composed matrix. When you compute eigenvectors of your matrix, you can use them in a deflation-type preconditioner, as described in [1, 2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Frank, J. &amp;amp; Vuik, C. &lt;em&gt;On the construction of deflation-based preconditioners&lt;/em&gt; SIAM Journal on Scientific Computing, 2001, 23, 442-462&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Saad, Y.; Yeung, M.; Erhel, J. &amp;amp; Guyomarc'h, F. &lt;em&gt;A deflated version of the conjugate gradient algorithm&lt;/em&gt; SIAM Journal on Scientific Computing, 2000, 21, 1909-1926&lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2017-07-25T10:02:06.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27490" PostTypeId="1" AcceptedAnswerId="30992" CreationDate="2017-07-25T14:21:03.023" Score="4" ViewCount="525" Body="&lt;p&gt;This will be a very general question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a 3D finite element code in Python which I would like to extend to handle &quot;large&quot; problems (~10^8 unknowns in the global system). Right now I am using the &lt;code&gt;scipy.sparse&lt;/code&gt; library, which gives decent performance for iterative solvers, but I'm finding the following problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I'm quickly running into memory limitations for problems larger than 10^6 unknowns&lt;/li&gt;&#xA;&lt;li&gt;The linear systems which I am solving are symmetric positive definite, but while &lt;code&gt;scipy.sparse&lt;/code&gt; doesn't seem to have a storage format which knows about symmetry, so I think I am storing many more entries than necessary. &lt;/li&gt;&#xA;&lt;li&gt;Even before solving the global system, computing the element-local contributions (held in a multidimensional &lt;code&gt;numpy&lt;/code&gt; array) are abutting the memory requirements.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Therefore, it seems clear to me that I need to either write my own matrix-free iterative solver, or to use an external library with that functionality. My questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I understand that &lt;code&gt;scipy.sparse&lt;/code&gt; wraps lower level routines such as lapack. What I need to write basically overloads sparse matrix vector multiplication. If I write this in Python (or maybe in C and wrap it with Python), do I have any hope of attaining decent performance, or is it simply necessary to be able to call these lower level libraries? I have no intuition for this, and don't want to spend 3 weeks writing my own matrix-free A*x routine which is too slow.&lt;/li&gt;&#xA;&lt;li&gt;Is it a good idea to write this routine in Python? Or do I need to write it in C and wrap it?&lt;/li&gt;&#xA;&lt;li&gt;Do there exist libraries which support this functionality? It seems unlikely, since a code-specific knowledge of how to compute A*x without assembling A would be necessary.&lt;/li&gt;&#xA;&lt;li&gt;Rather than write a matrix-free routine, could I write an out-of-core solver? Do such approaches scale to problems like mine?&lt;/li&gt;&#xA;&lt;li&gt;I have access to a cluster with many multi-core compute nodes. Eventually I would like to parallelize this implementation. Are there good tools or references for implementation guidelines?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or is there a good way to handle the above while using only &lt;code&gt;scipy&lt;/code&gt;? For example, is it possible to provide &lt;code&gt;scipy.sparse.linalg.cg&lt;/code&gt; a pointer to a function which computes A * x (obviously the cg routine is limited by the A*x speed, but it would be nice to avoid coding CG, GMRES, etc. myself)?&lt;/p&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="20148" LastEditDate="2017-07-26T13:51:21.803" LastActivityDate="2019-02-03T18:48:33.617" Title="Leveraging scipy for matrix free finite elements" Tags="|sparse-matrix|reference-request|scipy|hpc|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27492" PostTypeId="2" ParentId="27490" CreationDate="2017-07-25T16:23:52.700" Score="5" Body="&lt;p&gt;I would say that the implementation + verification + unit testing would take you more than just 3 weeks. Although, if you are planning to invest that time, you might add that capabilities to &lt;code&gt;scipy.sparse&lt;/code&gt; or &lt;a href=&quot;https://github.com/scikit-sparse/scikit-sparse&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;scikits-sparse&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding symmetric sparse matrices, you can check &lt;a href=&quot;http://pysparse.sourceforge.net/index.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;Pysparse&lt;/code&gt;&lt;/a&gt;. It has &lt;a href=&quot;http://pysparse.sourceforge.net/formats.html#sparse-skyline-format&quot; rel=&quot;noreferrer&quot;&gt;Sparse Skyline format (SSS)&lt;/a&gt; that is used for symmetric matrices. You can also directly use the &lt;a href=&quot;http://pysparse.sourceforge.net/spmatrix.html&quot; rel=&quot;noreferrer&quot;&gt;Low-Level Sparse Matrices&lt;/a&gt; that they provide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This package is used by &lt;a href=&quot;http://sfepy.org/doc-devel/installation.html#requirements&quot; rel=&quot;noreferrer&quot;&gt;SfePy&lt;/a&gt; that is a FEM package for Python.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2017-07-25T16:23:52.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27493" PostTypeId="1" CreationDate="2017-07-25T18:15:46.380" Score="1" ViewCount="733" Body="&lt;p&gt;I have large arrays of data organized so that it can be processed efficiently using array processing libraries. However, there are times when I only need to process slices of the arrays where a slice might be an arbitrary, non-contiguous subset defined by a select set of index values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result = someArrayProcess(objects)&#xA;result = result(chosenSet)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;might be replaced by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result = someArrayProcess(objects(chosenSet),t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a way I can get some confidence whether the second example is more or less efficient without actually running tests? What would I be looking for in the array packages that might be available to ascertain whether they handle this sort of slicing efficiently (without running baseline tests)? Up to this point, I have used array slicing as a convenient way to develop readable and manageable code, but now I want to take it to production, where efficiency will be an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I write this, I'm threading my way through some of the similar questions. I see that striding is a way to efficiently access non-contiguous array elements. I assume that some array packages would leverage methods like this and some wouldn't. What packages maximize the use of methods like this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would probably be coded in C++. I am interested in how array packages in this and other languages (including Python) treat slicing with arbitrary subsets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Motivation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking into making some architectural revisions that leverage array processing (and eventually moving it into GPU processing). The objective is to provide architectural simplification (for easier design management) using array expressions rather than element-level processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this would be a radical transition from the current approach, I will have to address concerns about efficiency trade-offs. The current solutions sacrifice exactness in performance to obtain efficiency, while being overwhelmingly difficult to manage due to their convoluted elements. They are now outdated, considering the advances that have been made in processing hardware since they were originally developed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I prefer design simplification and performance accuracy over efficiency, but real-time requirements require that I consider efficiency, too. &lt;/p&gt;&#xA;" OwnerUserId="24968" LastActivityDate="2018-03-23T05:07:06.200" Title="Efficiency of Array Slicing" Tags="|matrix|performance|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27494" PostTypeId="2" ParentId="10398" CreationDate="2017-07-25T18:47:17.077" Score="2" Body="&lt;p&gt;Given that the accepted answer has typos and does not give full code, I decided to post mine.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;Eigen/Dense&amp;gt;&#xA;using namespace std;&#xA;using namespace Eigen;&#xA;int main()&#xA;{&#xA;    MatrixXd A;&#xA;    A.resize(3,3);&#xA;    VectorXd b;&#xA;    b.resize(3);&#xA;&#xA;    A &amp;lt;&amp;lt;&#xA;      13, 5, 7 ,&#xA;      5 , 9, 3 ,&#xA;      7 , 3, 11;&#xA;    b &amp;lt;&amp;lt; 3, 3, 4;&#xA;&#xA;    cout &amp;lt;&amp;lt; &quot;Here is the matrix A:\n&quot; &amp;lt;&amp;lt; A &amp;lt;&amp;lt; endl;&#xA;    cout &amp;lt;&amp;lt; &quot;Here is the vector b:\n&quot; &amp;lt;&amp;lt; b &amp;lt;&amp;lt; endl;&#xA;    LDLT&amp;lt;MatrixXd&amp;gt; L = A.ldlt();&#xA;    VectorXd x = L.solve(b);&#xA;    cout &amp;lt;&amp;lt; &quot;The solution is:\n&quot; &amp;lt;&amp;lt; x &amp;lt;&amp;lt; endl;&#xA;    cout &amp;lt;&amp;lt; &quot;reconstructed b is:\n&quot;  &amp;lt;&amp;lt; A*x &amp;lt;&amp;lt; endl; &#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11717" LastActivityDate="2017-07-25T18:47:17.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27495" PostTypeId="2" ParentId="27481" CreationDate="2017-07-25T21:40:46.160" Score="0" Body="&lt;p&gt;One liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ x[0], x[1], y[1] ] for x in RaceA for y in RaceB if x[0] == y[0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However for such big lists (not vectors) this might be a bit slow. If you aim for performance, there are better ways to do it probably.&lt;/p&gt;&#xA;" OwnerUserId="23566" LastActivityDate="2017-07-25T21:40:46.160" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27496" PostTypeId="1" CreationDate="2017-07-25T20:25:54.020" Score="8" ViewCount="11934" Body="&lt;p&gt;A simulation I'm doing requires me to calculate the partial trace of a large density matrix. I am trying to calculate it using tools from numpy, but my code seems to be having some problems. For background, let me explain the arrays I am interested in a little more, and the way I'm defining the partial trace. Then, I will give the code I have and the errors I am getting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, the partial trace. If I have a tensor product of vector spaces &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ V = \prod_i V_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a linear operator $T: V \to V$, then given a basis I can store all the information about $T$ as a multidimensional array $T_{out_1,..,out_n,in_1,..,in_n}$ where $T_{out_1,..,out_n,in_1,..,in_n}$ is the &#xA;$v_{out_1} \otimes v_{out_2}\otimes \dots \otimes v_{out_n}$ component of $T(v_{in_1} \otimes \dots \otimes v_{in_n} )$. (Compare this to a matrix in a basis where $M_{ij}$ is the $i^{th}$ component of $M(v_j)$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The partial trace over some of these indices is a new operator given as a multidimensional array defined by $\widetilde{T}_{kept_{out},kept_{in}} = \sum_{traced} T_{keep_{out},traced, keep_{in},traced}$, where $kept_{in}$, $kept_{out}$, and $traced$ are all multi indices referring to a subset of basis states. The sum is taken over all combinations of indices in the traced set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code for computing this in numpy is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def trace_index(array,i):&#xA;&quot;&quot;&quot;&#xA;Given an array and an index i, traces out the index.&#xA;&quot;&quot;&quot;&#xA;    n = len(array.shape)&#xA;    tot = list(range(n))&#xA;    tot[n//2+i-1] = i-1&#xA;    return np.einsum(array,list(tot))&#xA;&#xA;def partial_trace(array,indices):&#xA;&quot;&quot;&quot;&#xA;Given an array and a list of indices to trace out, traces out the indices&#xA;&quot;&quot;&quot;&#xA;&#xA;    in_sorted = sorted(indices,reverse=True) &#xA;    cur_trace = array&#xA;    for i in in_sorted:&#xA;        cur_trace = trace_index(cur_trace,i)&#xA;    return cur_trace&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I trace them in the descending order eg. 5,4,... because then I can apply trace_index to the indices one at a time. If I trace index 5 and then index 4, index 4 is still index 4 after tracing index 5. If I do it the other way, after I traced index 4, there is no index 5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code seems to work well for small cases, but for larger ones I&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: invalid subscript '}' in einstein sum subscripts string, subscripts must be letters&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, my question is this: Is there a better way to do what I am trying to do than what I am doing?&lt;/p&gt;&#xA;" OwnerUserId="45514" OwnerDisplayName="msm" LastEditorUserId="20417" LastEditDate="2018-03-29T06:17:31.553" LastActivityDate="2020-05-09T20:15:52.083" Title="Calculating partial trace of array in NumPy" Tags="|python|computational-physics|numpy|quantum-mechanics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27498" PostTypeId="2" ParentId="21915" CreationDate="2017-07-26T06:30:25.230" Score="5" Body="&lt;h1&gt;Continuous&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;It looks like you only need 2d curl, so let's start with a simpler continuous definition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \omega = \nabla \times \mathbf{u} = \frac{\delta v}{\delta x} - \frac{\delta u}{\delta y}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where 2d vector field $\mathbf{u}=(u,v)$ (same as your $\mathbf F = (F_1,F_2,F_3)$, dropping $F_3$). Note that curl is a vector and in the 2d version, and is purely in the z direction, and is customarily written as a scalar.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Discretization&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;We'll assume $(u,v)$ is sampled on a grid (i.e. at &quot;vertices&quot;, where lines cross, and indices have integer values), with spacing $\Delta x$ and $\Delta y$ (equal for a square grid, but helps &quot;typecheck&quot; the discretization).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Forward differences&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The simplest Finite Difference discretization method is &lt;em&gt;forward differences&lt;/em&gt;:&#xA;$$ &#xA;\omega_{i,j} = \frac{ v_{i+1,j} - v_{i,j} }{ \Delta x } - \frac{ u_{i,j+1}-u_{i,j} }{ \Delta y }&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Central differences&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;But &lt;em&gt;central differences&lt;/em&gt; is more accurate (note the 2 in the denominator, because we're discretizing over 2 grid cells):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\omega_{i,j} = \frac{ v_{i+1,j} - v_{i-1,j} }{ 2\Delta x } - \frac{ u_{i,j+1}-u_{i,j-1} }{ 2\Delta y }&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Finer central differences&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you look again at the first &lt;em&gt;forward differences&lt;/em&gt; method, you'll see it's the same as &lt;em&gt;central differences&lt;/em&gt;, just for  $\omega_{i+\frac{1}{2},j+\frac{1}{2}}$ instead of $\omega_{ i,j}$. &lt;strong&gt;EDIT&lt;/strong&gt; Actually not, because the central differences are for different locations: the first for  $\omega_{i+\frac{1}{2},j}$ and the second for  $\omega_{i,j+\frac{1}{2}}$. Instead, for each term, we can average the other co-ordinate values to interpolate the center. That is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;  \omega_{i+\frac{1}{2},j+\frac{1}{2}} = \frac{&#xA;  \frac{ v_{i+1,j} - v_{i,j} }{ \Delta x } + \frac{ v_{i+1,j+1} - v_{i,j+1} }{ \Delta x }&#xA;}{2}&#xA;- \frac{&#xA;  \frac{ u_{i,j+1}-u_{i,j} }{ \Delta y } + \frac{ u_{i+1,j+1}-u_{i+1,j} }{ \Delta y }&#xA;}{2}&#xA;$$&#xA;At this point, the advantages of the Calculus of Finite Differences (in Philip Roe's answer) become stark. The above is:&#xA;$$&#xA;  \omega_{i+\frac{1}{2},j+\frac{1}{2}} = \mu_y \delta_x v_{i,j} - \mu_x \delta_y u_{i,j}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me unpack that first term: the $\mu_y$ averages over values at $j$ and $j+1$ in y-direction (i.e. the two expressions divided by $2$), the $\delta_x$ is the difference between values at $i$ and $i+1$ in x-direction (i.e. each of the aforementioned expressions, which are divided by $\Delta x$). I'm not sure what happens to the $\Delta x$; I &lt;em&gt;think&lt;/em&gt; the idea is to let $\Delta x=1$, so it can be ignored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or (if you're better than me at remembering how the offset indices go) just:&#xA;$$&#xA;\omega = \mu_y \delta_x v - \mu_x \delta_y u&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These seem what you're going for with your $\nabla_1$ and $\nabla_2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; Now I think you're going for finite differences along diagonal lines for your $\nabla_2$ (the corners of a square). IDK, but I think that would work, but you'd need to calculate $(u,v)$ perpendicular to each diagonal, and also adjust the lengths that you're dividing by (though I guess the wikipedia integral version may take care of this, since it uses area, not lengths). Of course, like the above, it would be for the center of a cell, $\omega_{ i+\frac{1}{2},j+\frac{1}{2} }$, rather than at the data points.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Accuracy&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I won't go into it here, but the accuracy of the above Finite Difference methods is usually analysed with Taylor Series. &lt;em&gt;Forward differences&lt;/em&gt; is first order $O(\Delta x)$, &lt;em&gt;central differences&lt;/em&gt; is second order $O( \Delta x^2 )$.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-08-08T07:49:49.133" LastActivityDate="2017-08-08T07:49:49.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27499" PostTypeId="1" CreationDate="2017-07-26T08:09:12.537" Score="1" ViewCount="73" Body="&lt;p&gt;I'd like to build a system in Python, consisting of (broadly speaking) objects which are internally described with (not necessarily just linear) equations, that I can connect with each other - similar to what can be done in OpenModelica. Is there anything like this already available or do I have to use OpenModelica for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, a steady-state solution is just fine. That's also why I think that OpenModelica would be overkill.&lt;/p&gt;&#xA;" OwnerUserId="3680" LastActivityDate="2017-07-26T08:09:12.537" Title="Object-oriented non-linear solving in python" Tags="|python|nonlinear-equations|nonlinear-programming|openmodelica|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27500" PostTypeId="1" CreationDate="2017-07-26T09:26:21.577" Score="1" ViewCount="46" Body="&lt;p&gt;I am trying to understand the mathematical side of an algorithm, which approximates the solution of coulomb's law. &lt;a href=&quot;https://arxiv.org/abs/1111.5035&quot; rel=&quot;nofollow noreferrer&quot;&gt;Their approach&lt;/a&gt; is to balance the two elements with the biggest positive/negative error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to understand the stability conditions of the algorithm, I am searching if there is an iterative solver with an similiar approach, as the problem can be stated as a linear system (but has a lot more computational complexity when solving the system with usual solvers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm does not use any derivatives, but reduces the error induced from two two elements with the biggest residuum on each other to zero and recalculates the residuum of all other elements after the balancing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The potential they are balancing is defined by $U_i = \sum_j I_{ij} q_j$ at points $x_i$ at the center of triangles $\Delta S_j$ with charge $q_j$, where &#xA;$$I_{ij} = \frac{1}{\Delta S_j} \int_{\Delta S_j}k|\vec{x}_i-\vec{x}'|^{-1}dS'.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In each step the points m, n with the biggest positive/negative deviation to an equipotential are balanced by transferting a charge $$q=\frac{U_m - U_n}{I_{mm}+I_{nn}-I_{mn}-I_{nm}}$$ giving the new potentials at the points $x_i, x_j$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;U'_m = U_m - I_{mm}q + I_{mn}q \\&#xA;U'_n = U_n + I_{mm}q - I_{mn}q&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this transfer $U'_m = U'_n$ holds and all other potentials $U_i$ are updated: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U'_i = U_i + q_m(U'_m - U_m) + q_m(U'_n - U_n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After each step, the potentials $U'_n, U'_m$ are equal and in the setting of electrostatics all other potentials come closer to an equipotential on the surface as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect, that this way of optimization is already described somewhere else with more numerical analysis about errors, convergence criteria and stability conditions, but was not able to find it in literature, yet.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="14700" LastEditDate="2017-08-07T05:54:10.463" LastActivityDate="2017-08-07T05:54:10.463" Title="Iterative solver, which balances the equations with the biggest errors" Tags="|linear-solver|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27501" PostTypeId="1" CreationDate="2017-07-26T10:36:24.327" Score="7" ViewCount="213" Body="&lt;p&gt;I have two sets of large positive numbers $a_1,\ldots,a_n$ and $b_1,\ldots,b_n$. By 'large' I mean of the order of $10^{10}$. I want to calculate the ratio $$R = \frac{a_1 - a_2 + \cdots +(-1)^{n+1}a_n}{b_1 - b_2 + \cdots +(-1)^{n+1}b_n}.$$ It is already known that the exact value of $R$ is between $0$ and $1$. Each of the numerator and denominator is expected to be of the order of $10^{-1}$ to $10^1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there is an algorithm to compute ratios like $R$ stably (exactly or approximately), so that the numbers involved in each step are not too large to cause overflow errors using finite-precision arithmetic. I am using the usual 64-bit &lt;code&gt;double&lt;/code&gt; data type in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Kahan summation algorithm&lt;/a&gt;, but using it to compute the numerator and denominator separately may not guarantee that their ratio is also as accurate.&lt;/p&gt;&#xA;" OwnerUserId="24933" LastEditorUserId="20688" LastEditDate="2018-03-21T06:08:02.347" LastActivityDate="2018-06-19T07:33:38.433" Title="Stable computation of ratio of sums of large numbers" Tags="|algorithms|stability|floating-point|precision|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27502" PostTypeId="2" ParentId="19209" CreationDate="2017-07-26T11:25:58.610" Score="3" Body="&lt;p&gt;You say that the problem with complex arithmetic is that there are different ways to define the scalar product for complex vectors, compared to just one way in the real case. I think the real problem with the complex scalar product is another one, which is, however, closely related to your observation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In complex arithmetic the order of the arguments of the scalar product do matter, while in real arithmetic they do not. Many algorithms are essentially the same in complex and real arithmetic, meaning you just have to write them once and then use the same code for complex and real arithmetic. (For example, in C++ you can use templates for this purpose.) When you are done writing your code, you usually test it. To uncover mistakes in the ordering of the arguments in some scalar product, you have to test your code with a complex-valued test case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You often get the real-valued code for an algorithm for free when you have a working code for complex valued problems. When you have tested your code with a complex-valued test case, the code is often also correct for real numbers. Turning, a real-valued code into a complex one, however, requires additional work. Therefore, there are just more codes that just work (and are thoroughly tested) for real-valued than for complex valued problems.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My question is: does this complication cause true danger for use of complex arithmetic in scientific computing?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would say &quot;Yes&quot;, in the following way. When the code is not well tested for complex-valued problems, there is a higher probability of bugs in the code, but this depends on the concrete code you are looking at. When the code is well tested, there is no problem.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My final question is: shall we just always avoid using complex numbers?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As already pointed out, there are problems that cannot be solved using real numbers. For example, the computation of eigenvalues of unsymmetric matrices. Hence, we need complex arithmetic.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2017-07-26T11:25:58.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27503" PostTypeId="1" AcceptedAnswerId="27561" CreationDate="2017-07-26T15:05:58.303" Score="4" ViewCount="2475" Body="&lt;p&gt;I need to have a finite difference stencil  for the mixed derivative&#xA;$$f_{xy}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on nonuniform grids such as this one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AtbwC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AtbwC.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I could not find a stencil in the literature, I tried to derive it by my self. As usual I started with a Taylor series expansion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(I)\qquad f(x+b,y+d)\approx f + b f_x+d f_y +\frac{b^2}{2}f_{xx} +bdf_{xy} +\frac{d^2}{2}f_{yy}$&#xA;$(II)\qquad f(x+b,y-c)\approx f + b f_x-c f_y +\frac{b^2}{2}f_{xx} -bcf_{xy} +\frac{c^2}{2}f_{yy}$&#xA;$(III)\qquad f(x-a,y-c)\approx f - a f_x-c f_y +\frac{a^2}{2}f_{xx} +acf_{xy} +\frac{c^2}{2}f_{yy}$&#xA;$(IV)\qquad f(x-a,y+d)\approx f - a f_x+d f_y +\frac{a^2}{2}f_{xx} -adf_{xy} +\frac{d^2}{2}f_{yy}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to find a combination of the equations above ($I - IV$) where all derivative terms on the right hand side ($f_x, f_y, f_{xx}, f_ {yy}$) except the mixed derivative vanish. This leads to the following linear system of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\begin{bmatrix} bf_x &amp;amp;&amp;amp; bf_x &amp;amp;&amp;amp; -af_x &amp;amp;&amp;amp; -af_x \\ &#xA;df_y &amp;amp;&amp;amp; -cf_y &amp;amp;&amp;amp; -cf_y &amp;amp;&amp;amp; df_y \\&#xA;\frac{b^2}{2}f_{xx} &amp;amp;&amp;amp; \frac{a^2}{2}f_{xx} &amp;amp;&amp;amp; \frac{a^2}{2}f_{xx} &amp;amp;&amp;amp; \frac{a^2}{2}f_{xx}\\&#xA;\frac{d^2}{2}f_{yy} &amp;amp;&amp;amp; \frac{c^2}{2}f_{yy} &amp;amp;&amp;amp; \frac{c^2}{2}f_{yy} &amp;amp;&amp;amp;&#xA;\frac{d^2}{2}f_{yy}&#xA;\end{bmatrix}$ &#xA;$\begin{bmatrix} P \\ Q \\ R \\ S \end{bmatrix}$ = &#xA;$\begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \end{bmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A solution to this problem is $P=1, Q=-1, R=1, S=-1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiplying the equations $I,II,III,IV$ with the respective factors $P,Q,R,S$ and adding them together:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x+b,y+d) - f(x+b,y-c) + f(x-a,y-c) - f(x-a,y+d) = (bd + bc + ac +ad)f_{xy}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And thus:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_{xy} = \frac{f(x+b,y+d) - f(x+b,y-c) + f(x-a,y-c) - f(x-a,y+d)}{bd + bc + ac +ad}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody assure me that this is a correct stencil for the mixed derivative for nonuniform grids? What worries me a little bit is that the individual terms ($f(x+b,y+d)$ etc.) do have a uniform weight. So the value at $x+b,y+d$ has equal weight as the point $x-a,y-c$ even though the latter is much closer to the center point $(x,y)$.&lt;/p&gt;&#xA;" OwnerUserId="23586" LastEditorUserId="9667" LastEditDate="2017-07-26T15:53:43.683" LastActivityDate="2021-06-22T08:12:06.333" Title="Finite difference for mixed derivatives on nonuniform grid" Tags="|finite-difference|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27504" PostTypeId="2" ParentId="27476" CreationDate="2017-07-26T16:19:03.333" Score="0" Body="&lt;p&gt;This is probably not an answer, more like random thoughts. We know that&#xA;$$\int g(x) f(x)^kdx=F_k$$.&#xA;Let's assume you can invert $y=f(x)$, we get&#xA;$$\int g(f^{-1}(y)){f^{-1}}^\prime(y) y^k dy =F_k$$&#xA;Hence, we can interpret $F_k$ as the moments of the function:&#xA;$$g(f^{-1}(y)){f^{-1}}^\prime(y) = \sum_k \frac{y^k}{k!}F_k$$&#xA;which is $$\frac{g(x)}{f^\prime(x)} = \sum_k \frac{f(x)^k}{k!}F_k$$&#xA;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's do the same exercise with your shifted form:&#xA;$$\int g(x) (a f(x)+b)^kdx$$ with $y(x)=a f(x)+b$ and $x(y)=f^{-1}\left(\frac{y-b}{a}\right)$. Hence, $\frac{dx}{dy}={f^{-1}}^\prime\left(\frac{y-b}{a}\right)\frac{1}{a}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int g\left(f^{-1}\left(\frac{y-b}{a}\right)\right){f^{-1}}^\prime\left(\frac{y-b}{a}\right)\frac{1}{a} y^k dy =\hat{F}_k$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the same interpretation:&#xA;$$g\left(f^{-1}\left(\frac{y-b}{a}\right)\right){f^{-1}}^\prime\left(\frac{y-b}{a}\right)\frac{1}{a}=\sum_k \frac{y^k}{k!}\hat{F}_k$$&#xA;$$\frac{g(x)}{a f^\prime(x)} = \sum_k \frac{\left(a f(x)+b\right)^k}{k!}\hat{F}_k$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since both series should be equal in powers of $f(x)^k$:&#xA;$$a \sum_k \frac{\left(a f(x)+b\right)^k}{k!}\hat{F}_k=\sum_k \frac{f(x)^k}{k!}F_k$$ we have a relation between $\hat{F}_k$ and $F_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this easier to compute? Is this even correct? I don't know, but maybe it's helpful.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2017-07-26T16:19:03.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27506" PostTypeId="2" ParentId="27503" CreationDate="2017-07-26T20:32:48.443" Score="1" Body="&lt;p&gt;In your sketch, you are having 9 grid points so you could have fit all 9 values to a 2-D cubic polynomial. Then the Taylor series expansion becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(I)\qquad f(x+b,y+d)\approx f + b f_x+d f_y +\frac{b^2}{2}f_{xx} +bdf_{xy} +\frac{d^2}{2}f_{yy} + \frac{b^3}{6}f_{xxx} + \frac{d^3}{6}f_{yyy} + \frac{1}{2} b^2df_{xxy}+ \frac{1}{2} d^2bf_{xyy}$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(II)\qquad f(x+b,y-c)\approx f + b f_x-c f_y +\frac{b^2}{2}f_{xx} -bcf_{xy} +\frac{c^2}{2}f_{yy}+ \frac{b^3}{6}f_{xxx} - \frac{c^3}{6}f_{yyy} - \frac{1}{2} b^2cf_{xxy}+ \frac{1}{2} c^2bf_{xyy}$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(III)\qquad f(x-a,y-c)\approx f - a f_x-c f_y +\frac{a^2}{2}f_{xx} +acf_{xy} +\frac{c^2}{2}f_{yy}- \frac{a^3}{6}f_{xxx} - \frac{c^3}{6}f_{yyy} - \frac{1}{2} a^2cf_{xxy}- \frac{1}{2} a^2cf_{xyy}$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$(IV)\qquad f(x-a,y+d)\approx f - a f_x+d f_y +\frac{a^2}{2}f_{xx} -adf_{xy} +\frac{d^2}{2}f_{yy}- \frac{a^3}{6}f_{xxx} + \frac{d^3}{6}f_{yyy} + \frac{1}{2} a^2df_{xxy}- \frac{1}{2} a^2df_{xyy}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(V)\qquad f(x,y+d)\approx f +d f_y   +\frac{d^2}{2}f_{yy} + \frac{d^3}{6}f_{yyy} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(VI)\qquad f(x,y-c)\approx f -c f_y   +\frac{c^2}{2}f_{yy} - \frac{c^3}{6}f_{yyy} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(VII)\qquad f(x+b,y)\approx f + b f_x +\frac{b^2}{2}f_{xx}  + \frac{b^3}{6}f_{xxx}  $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(VIII)\qquad f(x-c,y)\approx f - c f_x +\frac{c^2}{2}f_{xx}  - \frac{c^3}{6}f_{xxx}  $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(IX)\qquad f(x,y)= f  $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$a_1$&lt;/span&gt; be the weight of equation (I), &lt;span class=&quot;math-container&quot;&gt;$a_2$&lt;/span&gt;be the weight of equation (II) &#xA;by equating &lt;span class=&quot;math-container&quot;&gt;$\sum a_i\cdot$&lt;/span&gt; (equation number) =&lt;span class=&quot;math-container&quot;&gt;$0\cdot f+0\cdot f_x+\ldots+ 1\cdot f_{xy}+\ldots+0\cdot f_{xyy}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can obtain a system of 9 linear equations with 9 unknowns, and by solving them you can obtain &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt;, Then you can obtain &lt;span class=&quot;math-container&quot;&gt;$f_{xy}$&lt;/span&gt;. You may use a symbolic tool to solve that in Matlab, Mathematica, Maple, python etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this system ends up in the singular case for &lt;span class=&quot;math-container&quot;&gt;$f_{xy}$&lt;/span&gt; calculation using 9 points. In addition to that, we have 10 equations with 9 variables so satisfying all terms of Taylor's series up to the third order term is impossible for this system. The easiest way to make it deterministic is freeing some variable and compromising higher-order terms so we can get a lot of solutions to this problem.&#xA;Matlab script used is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all; clc    &#xA;syms a1 a2 a3 a4 a5 a6 a7 a8 a7 a8 a9 a b c d % weight to each points&#xA;    eq1=a1 + a2 + a3 + a4 + a5 + a6 + a7 + a8+a9; % first order equation&#xA;&#xA;    eq2= a1*b - a*a4 - a*a3 + a2*b + a7*b - a8*c;&#xA;&#xA;    eq3=a1*d - a3*c - a6*c - a2*c + a4*d + a5*d;&#xA;&#xA;    eq4=a^2*a3 + a^2*a4 + a1*b^2 + a2*b^2 + a7*b^2 + a8*c^2;&#xA;&#xA;    eq5=a*a3*c - a2*b*c - a*a4*d + a1*b*d;&#xA;&#xA;    eq6=a2*c^2 + a3*c^2 + a6*c^2 + a1*d^2 + a4*d^2 + a5*d^2;&#xA;&#xA;    eq7=a1*b^3 - a^3*a4 - a^3*a3 + a2*b^3 + a7*b^3 - a8*c^3;&#xA;&#xA;    eq8=a1*d^3 - a3*c^3 - a6*c^3 - a2*c^3 + a4*d^3 + a5*d^3;&#xA;&#xA;    eq9=a^2*a4*d - a2*b^2*c - a^2*a3*c + a1*b^2*d;&#xA;&#xA;    eq10=- a3*a^2*c - a4*a^2*d + a2*b*c^2 + a1*b*d^2;&#xA;&#xA;    sol1=solve(eq1,eq2,eq3,eq4,eq5,eq6,eq7,eq9,'a1','a2','a3','a4','a5','a6','a7','a8') % one of the possible solutions&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For different equation combinations, you may get different solutions, make sure you are not trading lower-order terms to make it deterministic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then find the errors in your formulas by applying them to the Taylor expansion of an arbitrary function. They will have the form&lt;span class=&quot;math-container&quot;&gt;$$G(a,b,c,d)\frac{\partial^{h+k}f}{\partial x^h\partial y^k}$$&lt;/span&gt;&#xA;The terms of the lowest order are called the &quot;leading error terms&quot; and are usually the most important. They are called the truncation error because they are caused by taking only a finite number of terms in the Taylor series. Of course, to use more terms you would need more data points.&#xA;The most accurate formulas are the ones with the smallest values of &lt;span class=&quot;math-container&quot;&gt;$G(a,b,c,d)$&lt;/span&gt;. This is what you might hope to achieve by employing more of the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something to be aware of when handling irregular grids is that the errors you will obtain this way depend on the point around which you expand the solution. In other words, the accuracy of a finite-difference expression is tied to a particular location. When you analyse the accuracy of a finite-difference &lt;em&gt;method&lt;/em&gt; that seeks an equilibrium among many terms, then every term must be evaluated at the same place.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastEditorUserId="20688" LastEditDate="2019-01-27T03:16:16.717" LastActivityDate="2019-01-27T03:16:16.717" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="27507" PostTypeId="2" ParentId="27476" CreationDate="2017-07-26T21:14:49.193" Score="0" Body="&lt;p&gt;I think your problem is ill-conditioned and that this is not a numerical issue: the condition number with respect to $F_j$ is proportional to $\binom{k}{j}a^j b^{k-j}$, which with yours numbers can be huge (e.g., $30^{15}$ for $j=k$). Think for example about how the answer would change if $F_k$ was changed by $10^{-16}$%—this has nothing to do with the exact way in which you compute that number, only how you define it. So since the number you are trying to compute is so sensitive to minute changes in the input data, this is not a well-posed problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any correct algorithm, however numerically accurate, will be just as unstable because it is a property of the problem as posed. In principle, this could change if you picked a different but similar problem with a different parametrization, but not as it stands.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-07-26T21:14:49.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27508" PostTypeId="1" AcceptedAnswerId="27515" CreationDate="2017-07-26T23:56:55.190" Score="2" ViewCount="286" Body="&lt;p&gt;Can we use the von Neumann stability analysis to investigate the stability of the discrete form of the following problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t+\frac{\partial(x^2u)}{\partial x}=S(u,x)\ .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please give some hint how to use this analysis in the above problem.&lt;/p&gt;&#xA;" OwnerUserId="24980" LastEditorUserId="9667" LastEditDate="2017-07-27T01:08:34.120" LastActivityDate="2017-07-29T15:43:07.800" Title="von Neumann stability analysis for spatial variable flux" Tags="|stability|fourier-analysis|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27514" PostTypeId="1" AcceptedAnswerId="27676" CreationDate="2017-07-27T11:26:29.117" Score="9" ViewCount="434" Body="&lt;p&gt;A naïve approach to solving stochastic differential equations (SDEs) would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;take a regular multi-step Runge–Kutta method,&lt;/li&gt;&#xA;&lt;li&gt;use a sufficiently fine discretisation of the underlying Wiener process,&lt;/li&gt;&#xA;&lt;li&gt;make each step of the Runge–Kutta method analogous to an Euler–Maruyama.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, this fails on multiple levels and I understand why.&#xA;However, now I am tasked to convince people of this fact who have little knowledge of Runge–Kutta methods and stochastic differential equations to begin with.&#xA;All the arguments I am aware of are nothing I can communicate well in the given context.&#xA;Hence, I am looking for an &lt;strong&gt;easily understandable&lt;/strong&gt; argument that the above approach is doomed.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2017-08-21T21:33:02.293" Title="Easily understandable argument that normal Runge–Kutta methods cannot be generalised to SDEs?" Tags="|runge-kutta|education|stochastic-ode|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27515" PostTypeId="2" ParentId="27508" CreationDate="2017-07-27T11:57:50.497" Score="1" Body="&lt;p&gt;Im not sure there is a rigorous justification, but consider that stability is defined (should be defined) in the following way. That the numerical solution remains bounded as the mesh sizes in time and space tend to zero. So stability is about what happens on fine meshes. On fine meshes, nonlinear problems such as the one you offer &quot;look more linear&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other methods of defining stability in the nonlinear case such as energy stability. As far as I know, the outcome is always consistent with von Neumann stability.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-07-27T11:57:50.497" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27516" PostTypeId="2" ParentId="27508" CreationDate="2017-07-27T12:33:13.253" Score="1" Body="&lt;p&gt;Use the chain rule to re-write your original PDE as a inhomogeneous advection equation with variable advection speed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t + x^2 u_x = S(u,x) - 2 x u.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then use the method of &lt;em&gt;frozen coefficients&lt;/em&gt; to investigate stability of the corresponding discrete PDE problem (taking into account, of course, any boundary and initial conditions you may have). Check out &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6863/method-of-frozen-coefficients-and-its-relation-to-von-neumann-stability-analysis&quot;&gt;this older scicomp post&lt;/a&gt;. Here's a couple more references to get you started:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.physics.arizona.edu/~restrepo/475B/Notes/sourcehtml/node37.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.physics.arizona.edu/~restrepo/475B/Notes/sourcehtml/node37.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cs.ubc.ca/~ascher/520/chapt05.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cs.ubc.ca/~ascher/520/chapt05.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-07-29T15:43:07.800" LastActivityDate="2017-07-29T15:43:07.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27517" PostTypeId="2" ParentId="27486" CreationDate="2017-07-27T13:07:15.883" Score="1" Body="&lt;p&gt;Let's say you have derivatives of different order $u_x$ in your problem on your physical domain and a transformation $x=f(\xi)$.&lt;br&gt;&#xA;Instead for solving on the original ode on the old domain, you solve the transformed ode on the new domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following example is taken from Boyd's book.&#xA;Be the original ode:&#xA;$$a_2(x)u_{xx}+a_1(x)u_{x}+a_0(x)u=g(x)$$&#xA;you solve now:&#xA;$$a_2(f(\xi))\frac{f^\prime(\xi)u_{\xi\xi}-f^{\prime\prime}(\xi)u_\xi}{f^\prime(\xi)^3}+a_1(f(\xi))\frac{u_\xi}{f^\prime(\xi)}+a_0(f(\xi))u=g(f(\xi))$$&#xA;Now you can use the defined derivative operators for $\xi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is essentially a variable transformation in your derivatives: $$\frac{du}{dx}=\frac{du}{d\xi}\frac{d\xi}{dx}=\frac{du}{d\xi}\frac{1}{\frac{dx}{d\xi}}=\frac{du}{d\xi}\frac{1}{f^\prime(\xi)}$$&#xA;This can of course be expanded to the multidimensional case, although it gets a bit more complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beware that you need to transform all boundary conditions as well.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2017-07-27T13:07:15.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27518" PostTypeId="1" CreationDate="2017-07-27T13:09:18.307" Score="0" ViewCount="327" Body="&lt;p&gt;I'm trying to reproduce the work in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21360/computing-turbulent-energy-spectrum-from-isotropic-turbulence-flow-field-in-a-bo/21361#21361?newreg=00cd5389bebc475ca76c578ceea1c8be&quot;&gt;computing turbulent energy spectrum from isotropic turbulence flow field in a box&lt;/a&gt; but I have a hard time understanding James' answer about the binning part. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you be so kind as to try to explain it to me ? Many thanks !&lt;/p&gt;&#xA;" OwnerUserId="24988" LastActivityDate="2017-10-26T17:42:21.913" Title="Turbulent energy spectrum from isotropic turbulence field in a box (bis)" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27521" PostTypeId="1" CreationDate="2017-07-27T17:06:25.423" Score="0" ViewCount="3242" Body="&lt;p&gt;I'm trying to solve the linear advection equation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u_{t} = cu_{x}, \\&#xA;x \in [x_{0}, x_{e}], \quad t \in (0, T], \quad c \in \mathbb{R} \\&#xA;u(x,0) = f(x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that for &lt;span class=&quot;math-container&quot;&gt;$c &amp;gt; 0$&lt;/span&gt;, the solution is a left travelling wave so requires a boundary condition on the left, while &lt;span class=&quot;math-container&quot;&gt;$c &amp;lt; 0$&lt;/span&gt; is a right travelling wave so requires a boundary condition on the right. Now, implementing Dirichlet (&lt;span class=&quot;math-container&quot;&gt;$u(\cdot, t) = 0$&lt;/span&gt;) and periodic (&lt;span class=&quot;math-container&quot;&gt;$u(x_{0},t) = u(x_{e},t)$&lt;/span&gt;) boundary conditions has been quite straight forward. However, I've been trying to implement Neumann conditions (&lt;span class=&quot;math-container&quot;&gt;$u_{x}(\cdot,t) = \alpha$&lt;/span&gt;) and I'm not exactly sure how to or if it is possible?&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$c &amp;gt; 0$&lt;/span&gt;, then the discretisation in matrix form is given as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{pmatrix}&#xA;u_{1}^{k+1} \\&#xA;u_{2}^{k+1} \\&#xA;u_{3}^{k+1} \\&#xA;\vdots \\&#xA;u_{N}^{k+1}&#xA;\end{pmatrix} = \begin{pmatrix}&#xA;1+\gamma \\&#xA;-\gamma &amp;amp; 1+\gamma \\&#xA; &amp;amp; -\gamma &amp;amp; 1+\gamma \\&#xA; &amp;amp; &amp;amp; \ddots &amp;amp; \ddots \\&#xA; &amp;amp; &amp;amp; &amp;amp; -\gamma &amp;amp; 1+\gamma&#xA;\end{pmatrix} \begin{pmatrix}&#xA;u_{1}^{k} \\&#xA;u_{2}^{k} \\&#xA;u_{3}^{k} \\&#xA;\vdots \\&#xA;u_{N}^{k}&#xA;\end{pmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If we are to use the ghost point approach, then applying a central difference we get&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;u_{x}(x_{N},t) &amp;amp;= \frac{u_{N+1} - u_{N-1}}{2 \Delta x} \\&#xA;&amp;amp;= \alpha \\&#xA;\implies u_{N+1} &amp;amp;= u_{N-1} + 2 \alpha \Delta x, \quad \forall k&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem is, nowhere in our spatial discretisation up to &lt;span class=&quot;math-container&quot;&gt;$n = N$&lt;/span&gt; do we have our function indexed at &lt;span class=&quot;math-container&quot;&gt;$N+1$&lt;/span&gt;, so we can't use that approach. If we use a backward difference instead, we get&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;u_{x}(x_{N},t) &amp;amp;= \frac{u_{N} - u_{N-1}}{\Delta x} \\&#xA;&amp;amp;= \alpha \\&#xA;\implies u_{N} &amp;amp;= u_{N-1} + \alpha \Delta x, \quad \forall k&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and hence our final ODE becomes&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u_{N}^{k+1} = u_{N}^{k} + \alpha \Delta x$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;but that didn't seem to work either. So my question is, how do we implement Neumann boundary conditions using the upwind method?&lt;/p&gt;&#xA;&lt;p&gt;Here is my code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def Upwind(N,x0,xe,t0,te,m,c,alpha,f,boundary='Dirichlet'):&#xA;&#xA;# ----------------------&#xA;### Timestep&#xA;&#xA;t = np.linspace(t0, te, N)&#xA;dt = t[1]-t[0]&#xA;&#xA;# ----------------------&#xA;### Direction conditions&#xA;&#xA;if c == 0:&#xA;    &#xA;    print('u(x,t) = f(x)')&#xA;&#xA;else:&#xA;    &#xA;    if c &amp;gt; 0:&#xA;&#xA;        x = np.linspace(xe, x0, N)&#xA;        &#xA;    else:&#xA;&#xA;        x = np.linspace(x0, xe, N)&#xA;        &#xA;        &#xA;    dx = x[1]-x[0]&#xA;    r = c*np.ones(N)*dt/dx&#xA;&#xA;    # ----------------------&#xA;    ### Stability conditions&#xA;&#xA;    if abs(r[0]) &amp;gt; 1+1e-15:&#xA;&#xA;        print('r = {}'.format(r[0]))&#xA;        print('This solution is unstable, abs(r) must be less \&#xA;              than or equal to one for stability.')&#xA;           &#xA;    # ----------------------&#xA;    ### Derivative matrix&#xA;    &#xA;    DL = spdiags([np.ones(N)], (0), N, N).tocsr() # NOT NECESSARY, MORE FOR &#xA;                                                  # CONTINUITY WITH OTHER SCHEMES &#xA;                                                  # THAT HAVE A LHS&#xA;    DR = spdiags([-r, 1+r], (-1, 0), N, N).tocsr()&#xA;&#xA;    # ----------------------&#xA;    ### Boundary conditions&#xA;&#xA;    if boundary == 'Dirichlet': # AGAIN, NOT NECESSARY, MORE FOR &#xA;                                # CONTINUITY. HERE u(.,t) = 0.&#xA;&#xA;        BL = spdiags([0], (0), N, N)&#xA;        BR = spdiags([0], (0), N, N)&#xA;&#xA;    elif boundary == 'Neumann': # NEEDS TO BE FIXED&#xA;&#xA;        BL = spdiags([0], (0), N, N)&#xA;        BR = spdiags([0], (0), N, N)&#xA;&#xA;    else:&#xA;        &#xA;        BL = spdiags([0], (0), N, N)&#xA;        BR = spdiags([-r], (N-1), N, N)&#xA;&#xA;    # ----------------------    &#xA;    ### Initial condition&#xA;&#xA;    u = f(x)&#xA;&#xA;    # ----------------------&#xA;    ### Loop&#xA;&#xA;    for k in range(m+1):&#xA;&#xA;        u = spsolve(DL+BL, (DR+BR)*u)&#xA;        &#xA;        if k % 25 == 0:&#xA;        &#xA;            plt.plot(x,u)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and to test&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Upwind(200,-40,40,0,40,100,-2,1,lambda x: np.exp(-x**2),boundary='Periodic')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16852" LastEditorUserId="16852" LastEditDate="2022-04-04T02:43:54.360" LastActivityDate="2022-04-04T02:43:54.360" Title="Neumann boundary conditions for the upwind scheme applied to the advection equation (Python)" Tags="|python|boundary-conditions|advection|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="27523" PostTypeId="2" ParentId="21915" CreationDate="2017-07-28T04:08:57.437" Score="2" Body="&lt;p&gt;This is made a lot easier by introducing the Calculus of Finite Differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $u_{i,j}$ is grid function defined for integer $i,j$ then the y-differencing operator&#xA;$$\delta_y u=u_{j+1}-u_j$$ is defined at $i,j+1/2$, so no need to write indices. Similarly, $$\mu_xu=(u_{i,j}+u_{i+1,j})/2 $$ is the x-averaging operator and is defined at $i+1/2,j$. We  complete the list of operators with $\delta_x$ and $\mu_y$ These operators are binary and the output is defined halfway between the inputs. In the examples so far, they are on the edges of the cells that have primary quantities at vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The operators can be combined according to most laws of algebra, added, multiplied, or raised to powers. For example $\mu_x\delta_x$ is a central difference on the original grid, and $\mu_y\delta_x$ approximates an x-derivative at a cell center. An important rule however, is that two operators can only be equated or combined if their outputs are at the same location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice discrete representation of curl in the 2D case is&#xA;$$\mu_y\delta_xv-\mu_x\delta_yu$$&#xA;at the cell centers if the data were at vertices or vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also use&#xA;$$\mu_x\delta_xv-\mu_y\delta_yu$$&#xA;A drawback of this this formula is that if you imagine the array of data to be colored like a chessboard, it only uses data of one color. In some circumstances this leads to trouble.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastEditorUserId="23873" LastEditDate="2017-08-01T23:01:52.777" LastActivityDate="2017-08-01T23:01:52.777" CommentCount="20" ContentLicense="CC BY-SA 3.0" />
  <row Id="27524" PostTypeId="1" CreationDate="2017-07-28T06:05:06.170" Score="1" ViewCount="177" Body="&lt;p&gt;I want to develop a distributed algorithm with MPI which have load balancing capability. But there is a situation that I'm not very clear about, which is, when the number of tasks is less than the number of processors available, I need to detect that and terminate those processors whose ranks (actually rank + 1) are greater than the total number of tasks. In my implementation, I just add a condition to determine that, if the condition is true, do something, otherwise, call MPI_Finalize() to shut down these processors. For your reference, here is the psuedocode:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MPI_Init();&#xA;MPI_Comm_size(MPI_COMM_WORLD, &amp;amp;size);&#xA;MPI_Comm_rank(MPI_COMM_WORLD, &amp;amp;proc_rank);&#xA;if (proc_rank + 1 &amp;lt; task_counts):&#xA;    do something (including some collective communications);&#xA;MPI_Finalize();&#xA;return 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This algorithm seems to be quiter obvious. However, it doesn't work. I added some fprintf code after MPI_Finalize(), but this code never got there for those processors whose ranks are out of range. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's the problem here?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="18224" LastEditDate="2017-07-28T08:12:17.600" LastActivityDate="2017-07-28T08:12:17.600" Title="MPI_Finalize() only some processors" Tags="|c++|mpi|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27525" PostTypeId="1" AcceptedAnswerId="27527" CreationDate="2017-07-28T07:06:53.223" Score="0" ViewCount="548" Body="&lt;p&gt;I am trying to plot a &lt;a href=&quot;http://farside.ph.utexas.edu/teaching/315/Waves/node29.html#f6.4&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference solution&lt;/a&gt; for the 1D wave equation using python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above link states the following: For a rod fixed at the right end and free at the left end and subjected to a hammer blow at the left end at $t = 0$ s where the hammer blow induces a constant velocity $V_0$ to occur from $x&amp;lt;a&amp;lt;L$, the displacement is given as follows: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_n = \frac{2V_0 a \sin((n-\frac 1 2)\pi a /L)}{v \pi (n-\frac 1 2)^2 \pi a/L}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(x,t) = \sum_{n=1,\infty}\cos((n-\frac 1 2)\pi x/L)\sin((n-\frac 1 2)\pi t/\tau) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\tau = L/c$,  $\psi$ is the displacement, $v = \sqrt{E/\rho}$ and $a$ is the distance where the initial velocity $V_0$ occurs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It appears the author has used $c$ in $\tau = L/c$ instead of using $v$ for the wave speed? This is confusing to me but my intuition says that $c = v = \sqrt{E/\rho}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For normalized rod displacement $\hat \psi(x,t) = (c / V_0 a) \psi(x,t)$ calculated from the preceding equations using the first 100 normal modes (i.e $n = 1,100$) and choosing $a/L = 0.1$, this is what we get for $t/\tau = 1.28$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YeSvd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YeSvd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now coding in &lt;code&gt;python&lt;/code&gt;: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;from numpy import *&#xA;plt.ion()&#xA;&#xA;n = 100&#xA;L = 25&#xA;a = 2.5&#xA;E = 100&#xA;rho = 1&#xA;V0 = 3&#xA;c = (E/rho)**0.5&#xA;x = linspace(0, L, 100)&#xA;u = []&#xA;ti = 1.28*L/c&#xA;for xi in x:&#xA;    SUM = 0&#xA;    for i in range(1,n+1,1):&#xA;        Ai = V0*a*2*sin((i-0.5)*a*pi/L)/(c*pi*(i-0.5)**2*pi*a/L)&#xA;        SUM = SUM+Ai*cos((i-0.5)*pi*xi/L)*sin((n-0.5)*pi*ti/(L/c))&#xA;    u.append(SUM)&#xA;&#xA;&#xA;u = [x * (c/V0*a) for x in u]&#xA;plt.plot(x/L,u)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Yields the following plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XggdP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XggdP.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which really has nothing to do with the correct plot above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have checked and rechecked the equations. I am not sure where I am going wrong and hope it is not an error from what the author has provided? The most important thing is the final result so I hope I am implementing it correctly? Some help please.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the answer below, here is the working code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;from numpy import *&#xA;plt.ion()&#xA;&#xA;n = 100&#xA;L = 25&#xA;a = 2.5&#xA;E = 100&#xA;rho = 1&#xA;V0 = 3&#xA;c = (E/rho)**0.5&#xA;x = linspace(0, L, 100)&#xA;u = []&#xA;ti = 1.28*L/c&#xA;for xi in x:&#xA;    SUM = 0&#xA;    for i in range(1,n+1,1):&#xA;        Ai = V0*a*2*sin((i-0.5)*a*pi/L)/(c*pi*(i-0.5)**2*pi*a/L)&#xA;        SUM = SUM+Ai*cos((i-0.5)*pi*xi/L)*sin((i-0.5)*pi*ti/(L/c))&#xA;    u.append(SUM)&#xA;&#xA;&#xA;&#xA;u = [x * (c/V0/a) for x in u]&#xA;plt.plot(x/L,u)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22736" LastEditorUserId="22736" LastEditDate="2017-07-31T05:54:08.627" LastActivityDate="2017-07-31T05:54:08.627" Title="Trying to plot 1D wave equation for benchmarking" Tags="|numerical-analysis|wave-propagation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27526" PostTypeId="1" AcceptedAnswerId="27531" CreationDate="2017-07-28T08:28:02.290" Score="0" ViewCount="78" Body="&lt;p&gt;I like to represent the following numbers in a data structure, like in trees or list but with one parent node and possibly many child nodes&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;1.3.1.1.0&lt;/p&gt;&#xA;&lt;p&gt;1.3.1.1.1&lt;/p&gt;&#xA;&lt;p&gt;1.3.1.1.2&lt;/p&gt;&#xA;&lt;p&gt;1.3.1.2.0&lt;/p&gt;&#xA;&lt;p&gt;1.3.1.2.1&lt;/p&gt;&#xA;&lt;p&gt;1.3.1.3.0&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Which data structure allow me to store and search the given values?&lt;/p&gt;&#xA;" OwnerUserId="24997" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-07-28T17:00:08.490" Title="Which data structure contains of one parent node and many child nodes" Tags="|data-structures|" AnswerCount="1" CommentCount="0" ClosedDate="2017-07-28T17:21:14.097" ContentLicense="CC BY-SA 3.0" />
  <row Id="27527" PostTypeId="2" ParentId="27525" CreationDate="2017-07-28T10:09:46.663" Score="2" Body="&lt;p&gt;There are two problems with your code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) You are mixing $i$ and $n$ in the following line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SUM = SUM+Ai*cos((i-0.5)*pi*xi/L)*sin((n-0.5)*pi*ti/(L/c))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SUM = SUM+Ai*cos((i-0.5)*pi*xi/L)*sin((i-0.5)*pi*ti/(L/c))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) You have to normalize the rod displacement. The coefficients are then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Ai = 2*sin((i-0.5)*a*pi/L)/(pi*(i-0.5)**2*pi*a/L)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result of these changes is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/US5Gq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/US5Gq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16561" LastActivityDate="2017-07-28T10:09:46.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27528" PostTypeId="2" ParentId="27518" CreationDate="2017-07-28T10:55:35.593" Score="1" Body="&lt;p&gt;After a shallow read, I guess that the main point is that you have a limit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you expand your velocity ${\bf{u}}(x,y)$ (2D for simplicity) in a Fourier series you will have modes like:&#xA;$$\exp{(ik_{xn}x)} \qquad\exp{(ik_{ym}y)}$$&#xA;Therefore you can write in discrete case with $N+1$ points:&#xA;$${\bf u}(x,y) = \sum_{nm}{\hat{\bf{u}}_{nm}(k_{nx},k_{my},t)\exp{(ik_{xn}x)} \exp{(ik_{ym}y)}}\qquad n,m \in[-N/2,N/2]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\hat{\bf{u}}_{nm}(k_{nx},k_{my},t)$ are the Fourier coefficients that describe $u$ in Fourier space given by wavenumbers $k_{xn}$ and $k_{ym}$ is:&#xA;$$k_{xn} =\frac{2\pi n}{L}=k_0n \qquad k_{ym} = k_0m$$ &#xA;The total energy $E_T(t)$ can be computed as:&#xA;$$E_T(t) = \frac{k_0^2}{2}\sum_{n,m}{\hat{\bf{u}}_{nm}(k_{nx},k_{my},t)\cdot \hat{\bf{u}}^*_{nm}(k_{nx},k_{my},t)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This energy can also be computed with the energy spectrum $E(k,t)$ with $k^2=k_{xn}^2+k_{ym}^2$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E_{T}(t) = \sum_{k}E(k,t)\Delta k$$&#xA;where $\Delta k$ must be defined with care and  $E(k,t)$ is&#xA;$$E(k,t) = \frac{1}{2}\sum_{k_{xn}^2+k_{ym}^2=k^2}{\hat{\bf{u}}_{nm}(k_{nx},k_{my},t)\cdot \hat{\bf{u}}^*_{nm}(k_{nx},k_{my},t)2\pi k }$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last sum is related with the discrete version of a line integral. It can be obtained from what he refers with &quot;binning&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear that values that do not meet with the restriction $k_{xn}^2+k_{ym}^2=k^2$ are discarded ( red zone in James' answer) and therefore you cannot reach values $k_{xN/2}$ or $k_{yN/2}$ and other points that are ourside the circle $k_{xn}^2+k_{ym}^2=k^2$.  It can be also shown in the $3D$ case with the sphere: $k_{xn}^2+k_{ym}^2+k_{zl}^2=k^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not an expert on this area...  I would be grateful if someone sees necessary to correct me in aspects that are not correct.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-07-28T16:07:01.403" LastActivityDate="2017-07-28T16:07:01.403" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27529" PostTypeId="2" ParentId="26440" CreationDate="2017-07-28T12:12:00.497" Score="0" Body="&lt;p&gt;You need to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{(p_1-1)}{2}  \frac{(p_2-1)}{2} \cdots \frac{(p_k-1)}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;congruence system of the form $x=q(\mod\ p_i)$ where $q$ is $1^2, 2^2, 3^2, \cdots, \left(\frac{(p_i-1)}{2}\right)^2$ using Chinese remainder theorem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give you all the distinct  $(p_1-1)/2\ (p_2-1)/2 \cdots (p_k-1)/2$ square solutions.&lt;/p&gt;&#xA;" OwnerUserId="24998" LastEditorUserId="9667" LastEditDate="2017-07-28T17:16:51.187" LastActivityDate="2017-07-28T17:16:51.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27531" PostTypeId="2" ParentId="27526" CreationDate="2017-07-28T17:00:08.490" Score="0" Body="&lt;p&gt;Sounds like a trie? (&lt;a href=&quot;https://en.wikipedia.org/wiki/Trie&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Trie&lt;/a&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Semantically it's like a BST-based/hashtable-based set (can check for presence/absence of a string, can enumerate all contained strings), but has different lookup costs (based on the length of the sought item, instead of the number contained items). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are especially handy when you need to group/retrieve items by a common prefix (phone/SMS autocomplete could use a data structure like this).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2017-07-28T17:00:08.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27532" PostTypeId="2" ParentId="18656" CreationDate="2017-07-28T22:42:36.377" Score="-3" Body="&lt;p&gt;On the other hand, BDF2 is stiffly accurate (and more effective for stiff equations) whereas TR-BDF2 along with all other DIRK schemes reduce to just first order in the stiff limit, and BDF2 tends to be more efficient in terms of solves required for a given level of accuracy.&lt;/p&gt;&#xA;" OwnerUserId="25008" LastActivityDate="2017-07-28T22:42:36.377" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27535" PostTypeId="1" CreationDate="2017-07-29T17:19:53.887" Score="1" ViewCount="338" Body="&lt;p&gt;Consider the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial}{\partial t}v(t,x)=\frac{\partial}{\partial x}v(t,x)$$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for $t,x\in\mathbb{R}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to solve this equation forward in space and backward in time, updating in space given the initial condition in space: $v(t,0)=f(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can discretise in time and take the backward difference of the time derivative, which yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\mathrm{d}}{\mathrm{d}x}v^n(x)=\frac{v^n(x)-v^{n-1}(x)}{\Delta t}\qquad(1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the Crank-Nicolson scheme gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{v^n_{i+1}-v^n_i}{\Delta x}=\frac{1}{2}\left[G^n(x_{i+1})+G^n(x_{i})\right],\qquad(2)$$&#xA;where $G^n(x)$ is the RHS of equation $(1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then $(2)$ becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{v^n_{i+1}-v^n_i}{\Delta x}=\frac{1}{2}\frac{v^n_{i+1}-v^{n-1}_{i+1}}{\Delta t}+\frac{1}{2}\frac{v^n_i-v^{n-1}_i}{\Delta t}$$&#xA;i.e.&#xA;$$\left(\frac{1}{\Delta x}-\frac{1}{2\Delta t}\right)v^n_{i+1}+\frac{1}{2\Delta t}v^{n-1}_{i+1}=\left(\frac{1}{\Delta x}-\frac{1}{2\Delta t}\right)v^n_{i}+\frac{1}{2\Delta t}v^{n-1}_{i}$$&#xA;so that&#xA;$$\mathrm{A}\vec{v}_{i+1}+\frac{1}{2\Delta t}\mathrm{B}\vec{v}_{i+1}=\mathrm{A}\vec{v}_{i}+\frac{1}{2\Delta t}\mathrm{B}\vec{v}_{i}$$&#xA;for $\mathrm{A},\mathrm{B}$ matrices yet to be specified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this implies that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{v}_{i+1}=\vec{v}_i,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which would appear to be nonsense. Have I done something wrong?&lt;/p&gt;&#xA;" OwnerUserId="24427" LastActivityDate="2017-07-29T17:19:53.887" Title="Crank-Nicolson scheme in space for advection equation" Tags="|pde|finite-difference|hyperbolic-pde|advection|crank-nicolson|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27536" PostTypeId="1" CreationDate="2017-07-29T18:59:44.383" Score="1" ViewCount="335" Body="&lt;p&gt;Consider one dimensional hyperbolic pde &#xA;$$u_t+f'(u)u_x=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the above problem ,CFL condition is $\Delta t\leq \dfrac{\Delta x}{|f'(u)|}.$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if we include  the source term $,S(u,t),$ which depend upon $u$  and $t$ in above equation i.e.&#xA;$$u_t+f'(u)u_x=S(u,t)$$&#xA; What is the influence of $S(u, t)$ in the CFL condition ?.&lt;/p&gt;&#xA;" OwnerUserId="24980" LastActivityDate="2017-07-29T18:59:44.383" Title="CFL condition of source term" Tags="|finite-difference|cfl|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27537" PostTypeId="1" CreationDate="2017-07-30T12:09:35.410" Score="4" ViewCount="5284" Body="&lt;p&gt;Let A be a n×n positive-definite Hermitian matrix. I already have the QR decomposition of A. Is there an efficient way to utilize this knowledge to speed up the Cholesky decomposition of A?&lt;/p&gt;&#xA;" OwnerUserId="24885" LastEditorUserId="19136" LastEditDate="2017-07-30T22:15:49.320" LastActivityDate="2017-07-30T22:15:49.320" Title="Computing the Cholesky decomposition based of the QR decomposition" Tags="|linear-algebra|matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27538" PostTypeId="1" CreationDate="2017-07-30T14:32:46.290" Score="0" ViewCount="279" Body="&lt;p&gt;I work in Molecular dynamics of biomolecules. I am familiar with the software &lt;a href=&quot;http://www.ks.uiuc.edu/Research/namd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NAMD&lt;/a&gt;. We use NAMD in our CPU cluster with the aid of &lt;a href=&quot;https://www.ibm.com/support/knowledgecenter/en/SSZTET_10.1.0/smpi02/smpi02_using_mpirun.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;mpirun&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without GPUs can I implement large scale deep learning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is there any scalable software out there like NAMD, but for Machine learning and deep learning? Is there any Machine Learning software that can run in parallel and that you can specify the number of CPUs to use?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;UPDATE : I know that NAMD is not for machine learning. But I am asking for the possibility of a similar software to use with CPU cluster, where GPUs are not available.&lt;/p&gt;&#xA;" OwnerUserId="5003" LastEditorUserId="5003" LastEditDate="2017-07-31T11:17:21.523" LastActivityDate="2018-07-14T07:20:49.723" Title="Is there any scalable machine learning tool like molecular dynamics software NAMD?" Tags="|software|machine-learning|molecular-dynamics|hpc|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27539" PostTypeId="2" ParentId="27537" CreationDate="2017-07-30T15:16:14.257" Score="11" Body="&lt;p&gt;No. In general, the QR decomposition has no relation to the Cholesky decomposition of a symmetric positive definite matrix. Moreover, the QR decomposition is substantially more expensive to compute than Cholesky 's decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the QR decomposition of tall matrix $A$ of full rank is closely related to the problem of computing a Cholesky factorization of the nonsingular matrix $A^T A$. Specifically, if $A = QR$, then $A^TA = R^T Q^T Q R = L L^T$ where the lower triangular matrix $L = R^T$ can now be identified as the Cholesky factor of $A^TA$.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2017-07-30T15:16:14.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27540" PostTypeId="2" ParentId="27538" CreationDate="2017-07-30T16:26:06.920" Score="0" Body="&lt;p&gt;I think that it does not make sense to call NAMD program as neither machine nor deep learning tools. As far as I concerned, the very brief definition of these two vocab is &lt;em&gt;you have to teach the program or machine until they can learn themselves&lt;/em&gt;. But, for NAMD, which is an instant program, it is just one of computational chemistry program package which is being used as a tool to straightforwardly provide you the energies of molecule, lipid, protein, etc. This means that you just put data in, and get data out. No need to teach NAMD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is many other open-source/freeware programs in research field of chemistry/biology like NAMD such as GROMACS, AMBER, and LAMMPS. Also these can be used with GPU architecture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Actually I should have made a comment for your question instead of answer, but my reputation is still lower than 50. LOL.&lt;/p&gt;&#xA;" OwnerUserId="22108" LastEditorUserId="22108" LastEditDate="2017-07-30T16:41:23.493" LastActivityDate="2017-07-30T16:41:23.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27542" PostTypeId="1" CreationDate="2017-07-28T21:50:10.217" Score="0" ViewCount="273" Body="&lt;p&gt;There is a problem I want to solve. The function is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\underset{a,b,\textbf{vec}}{\text{minimize} \text{ }\text{ }\text{ }\text{ } } f=\sum_{i=1}^{b}(\textbf{vec}_i)^{a}\\&#xA;&amp;amp;\text{subject to    } \text{ }\text{ } f \geq F, \text{ } \sum_{i=1}^b \textbf{vec}_i = D, \text{ } a\leq N, b \leq D, \{\textbf{vec}_1,\cdots, \textbf{vec}_b\}\leq D&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, all the variables are non-negative values. I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Because the size of $\textbf{vec}$ is dependent on another parameters $b$, then the number of variables in the objective function are not fixed beforehand. Does it still a MINLP problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How to solve it? is there some open-source software can deal with it easily? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="WYC" LastEditorUserId="9667" LastEditDate="2017-08-01T14:13:44.720" LastActivityDate="2017-08-01T14:13:44.720" Title="Mixed Integer Nonlinear Programming Problem" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27543" PostTypeId="2" ParentId="27542" CreationDate="2017-07-29T06:02:42.170" Score="1" Body="&lt;p&gt;I'm no expert on &lt;a href=&quot;http://reference.wolfram.com/language/ref/NMinimize&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;NMinimize&lt;/code&gt;&lt;/a&gt;, but the following might be a useful starting point:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;len = 200;&#xA;NMinimize[&#xA;    {&#xA;    Sum[x[i]^3, {i, len}],&#xA;    Sum[x[i]^3, {i, len}] &amp;gt;= 2000 &amp;amp;&amp;amp;&#xA;    Sum[x[i], {i, len}] == len &amp;amp;&amp;amp;&#xA;    And @@ Table[x[i] &amp;gt;= 0, {i, len}]&#xA;    },&#xA;    Table[x[i],{i,len}],&#xA;    Integers&#xA;] //AbsoluteTiming&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;NMinimize::cvmit: Failed to converge to the requested accuracy or precision within 100 iterations.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;{30.3566, {2000., {x[1] -&gt; 11, x[2] -&gt; 3, x[3] -&gt; 0, x[4] -&gt; 1, x[5] -&gt; 1, &#xA;     x[6] -&gt; 3, x[7] -&gt; 0, x[8] -&gt; 1, x[9] -&gt; 3, x[10] -&gt; 0, x[11] -&gt; 0, &#xA;     x[12] -&gt; 0, x[13] -&gt; 2, x[14] -&gt; 0, x[15] -&gt; 2, x[16] -&gt; 1, x[17] -&gt; 1, &#xA;     x[18] -&gt; 1, x[19] -&gt; 1, x[20] -&gt; 0, x[21] -&gt; 1, x[22] -&gt; 1, x[23] -&gt; 2, &#xA;     x[24] -&gt; 1, x[25] -&gt; 1, x[26] -&gt; 2, x[27] -&gt; 0, x[28] -&gt; 1, x[29] -&gt; 0, &#xA;     x[30] -&gt; 0, x[31] -&gt; 0, x[32] -&gt; 1, x[33] -&gt; 2, x[34] -&gt; 1, x[35] -&gt; 0, &#xA;     x[36] -&gt; 2, x[37] -&gt; 0, x[38] -&gt; 0, x[39] -&gt; 0, x[40] -&gt; 2, x[41] -&gt; 0, &#xA;     x[42] -&gt; 1, x[43] -&gt; 0, x[44] -&gt; 1, x[45] -&gt; 2, x[46] -&gt; 0, x[47] -&gt; 1, &#xA;     x[48] -&gt; 1, x[49] -&gt; 1, x[50] -&gt; 1, x[51] -&gt; 2, x[52] -&gt; 1, x[53] -&gt; 1, &#xA;     x[54] -&gt; 2, x[55] -&gt; 0, x[56] -&gt; 0, x[57] -&gt; 1, x[58] -&gt; 2, x[59] -&gt; 1, &#xA;     x[60] -&gt; 1, x[61] -&gt; 1, x[62] -&gt; 0, x[63] -&gt; 2, x[64] -&gt; 1, x[65] -&gt; 0, &#xA;     x[66] -&gt; 1, x[67] -&gt; 0, x[68] -&gt; 3, x[69] -&gt; 0, x[70] -&gt; 1, x[71] -&gt; 1, &#xA;     x[72] -&gt; 2, x[73] -&gt; 1, x[74] -&gt; 1, x[75] -&gt; 2, x[76] -&gt; 1, x[77] -&gt; 1, &#xA;     x[78] -&gt; 2, x[79] -&gt; 0, x[80] -&gt; 1, x[81] -&gt; 2, x[82] -&gt; 0, x[83] -&gt; 0, &#xA;     x[84] -&gt; 0, x[85] -&gt; 0, x[86] -&gt; 1, x[87] -&gt; 2, x[88] -&gt; 0, x[89] -&gt; 2, &#xA;     x[90] -&gt; 0, x[91] -&gt; 0, x[92] -&gt; 0, x[93] -&gt; 1, x[94] -&gt; 0, x[95] -&gt; 0, &#xA;     x[96] -&gt; 2, x[97] -&gt; 0, x[98] -&gt; 1, x[99] -&gt; 0, x[100] -&gt; 1, x[101] -&gt; 2, &#xA;     x[102] -&gt; 3, x[103] -&gt; 0, x[104] -&gt; 1, x[105] -&gt; 2, x[106] -&gt; 0, &#xA;     x[107] -&gt; 2, x[108] -&gt; 1, x[109] -&gt; 1, x[110] -&gt; 1, x[111] -&gt; 0, &#xA;     x[112] -&gt; 1, x[113] -&gt; 2, x[114] -&gt; 2, x[115] -&gt; 0, x[116] -&gt; 1, &#xA;     x[117] -&gt; 0, x[118] -&gt; 3, x[119] -&gt; 1, x[120] -&gt; 2, x[121] -&gt; 1, &#xA;     x[122] -&gt; 0, x[123] -&gt; 1, x[124] -&gt; 1, x[125] -&gt; 0, x[126] -&gt; 0, &#xA;     x[127] -&gt; 1, x[128] -&gt; 1, x[129] -&gt; 1, x[130] -&gt; 1, x[131] -&gt; 1, &#xA;     x[132] -&gt; 1, x[133] -&gt; 0, x[134] -&gt; 2, x[135] -&gt; 2, x[136] -&gt; 0, &#xA;     x[137] -&gt; 0, x[138] -&gt; 1, x[139] -&gt; 2, x[140] -&gt; 3, x[141] -&gt; 1, &#xA;     x[142] -&gt; 2, x[143] -&gt; 1, x[144] -&gt; 0, x[145] -&gt; 2, x[146] -&gt; 2, &#xA;     x[147] -&gt; 0, x[148] -&gt; 1, x[149] -&gt; 2, x[150] -&gt; 0, x[151] -&gt; 0, &#xA;     x[152] -&gt; 1, x[153] -&gt; 1, x[154] -&gt; 0, x[155] -&gt; 1, x[156] -&gt; 2, &#xA;     x[157] -&gt; 0, x[158] -&gt; 1, x[159] -&gt; 1, x[160] -&gt; 0, x[161] -&gt; 2, &#xA;     x[162] -&gt; 2, x[163] -&gt; 0, x[164] -&gt; 0, x[165] -&gt; 2, x[166] -&gt; 1, &#xA;     x[167] -&gt; 1, x[168] -&gt; 1, x[169] -&gt; 1, x[170] -&gt; 0, x[171] -&gt; 1, &#xA;     x[172] -&gt; 2, x[173] -&gt; 0, x[174] -&gt; 0, x[175] -&gt; 2, x[176] -&gt; 1, &#xA;     x[177] -&gt; 1, x[178] -&gt; 0, x[179] -&gt; 1, x[180] -&gt; 0, x[181] -&gt; 1, &#xA;     x[182] -&gt; 1, x[183] -&gt; 1, x[184] -&gt; 0, x[185] -&gt; 1, x[186] -&gt; 1, &#xA;     x[187] -&gt; 1, x[188] -&gt; 1, x[189] -&gt; 2, x[190] -&gt; 4, x[191] -&gt; 0, &#xA;     x[192] -&gt; 0, x[193] -&gt; 2, x[194] -&gt; 0, x[195] -&gt; 0, x[196] -&gt; 2, &#xA;     x[197] -&gt; 0, x[198] -&gt; 2, x[199] -&gt; 1, x[200] -&gt; 0}}}&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerDisplayName="Carl Woll" LastActivityDate="2017-07-29T06:02:42.170" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27545" PostTypeId="1" AcceptedAnswerId="27548" CreationDate="2017-07-31T12:00:14.710" Score="1" ViewCount="910" Body="&lt;p&gt;I'm using LAPACK zgeev routine to get eigenvalues and eigenvectors of a symmetric matrix in C++. Problem is zgeev is being called in a loop but it sorts eigenvalues (and eigenvectors) differently sometimes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this is the eigenvalues from the first round of loop:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(-1.29007e-5 - 5.207e-6*i)&#xA;(1.28782e-5 + 7.40505e-6*i)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is it's result from the second time:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1.28782e-5 + 7.40505e-6*i)&#xA;(-1.29007e-5 - 5.207e-6*i)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to plot the evolution of these eigenvalues and vectors as a function of the loop's variable but they keep getting swapped each time and gives me a combination of the plots I need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to fix this?&lt;/p&gt;&#xA;" OwnerUserId="23599" LastEditorUserId="23599" LastEditDate="2017-07-31T14:40:40.003" LastActivityDate="2017-08-01T10:53:10.383" Title="LAPACK sorting eigenvalues differently each time" Tags="|c++|eigenvalues|matrix|eigensystem|lapack|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27547" PostTypeId="1" AcceptedAnswerId="27577" CreationDate="2017-07-31T23:58:04.520" Score="1" ViewCount="49" Body="&lt;p&gt;I'm working on a project that requires unknown image elements to be filled in, and am using a simple inpainting algorithm, which I would like to understand better. It works by repeatedly convolving a 3x3 filter (in 2D) with the image. The center point of the filter has coefficient zero (of course!), while the corners have $b = 0.073235$, and the vertical and horizontal coefficients are $a = 0.176765$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a link to the paper I am using: &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=6&amp;amp;ved=0ahUKEwiFt8KI3LTVAhXGx4MKHdcPAZcQFghLMAU&amp;amp;url=http%3A%2F%2Fwww.inf.ufrgs.br%2F~oliveira%2Fpubs_files%2Finpainting.pdf&amp;amp;usg=AFQjCNFgj0Kq_xfHI9Zpx-GRb7qgfj_pYQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inpainting paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to understand how these were derived, but can't find the method in the literature. Note that $b/a \approx 2.41367 \approx \sqrt{2}+1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, for normalization, $4 a + 4 b = 1$. I thought they might come from setting an approximation of the Laplacian = 0, but have gotten nowhere with this.&lt;/p&gt;&#xA;" OwnerUserId="4220" LastEditorUserId="9667" LastEditDate="2017-08-01T12:35:27.773" LastActivityDate="2017-08-08T06:06:45.257" Title="Filter coefficients for convolutional inpainting" Tags="|algorithms|image-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27548" PostTypeId="2" ParentId="27545" CreationDate="2017-08-01T10:53:10.383" Score="4" Body="&lt;p&gt;You write, that you are computing the eigenvalues of a symmetric matrix. Does the matrix have real entries? In this case all eigenvalues are real, and you can use a symmetric eigenvalue solver, which returns only real entries. Hence, sorting them should not be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When your matrix has complex entries, you have to track the eigenvalues. I am assuming that your matrices change only slightly from one iteration of your loop to the next, meaning that the eigenvalues also change only slightly. Hence, you can find the eigenvalue of the next iteration that corresponds to the eigenvalue of the current iteration, by looking for the eigenvalue of the next iteration that is closest to the eigenvalue of the current iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, sorting complex eigenvalues does not solve your problem. Consider the matrix&#xA;\begin{equation}&#xA;A(t) =&#xA;\begin{bmatrix}&#xA;e^{\mathrm{i} t} &amp;amp; 0 \\&#xA;0 &amp;amp; e^{\mathrm{i} (t + \pi)}&#xA;\end{bmatrix}&#xA;\end{equation}&#xA;for $t = [0, 2\pi)$. The matrix has two eigenvalues, both lie on the circle of radius one. The two eigenvalues lie on opposit sides of the circle and with increasing $t$ they rotate around zero. When $t$ is large enough the first eigenvalue reaches the point where the second eigenvalue has been, and vice versa. Hence, any sorting technique will (at latest) at that point, switch the roles of the two eigenvalues, even though the first eigenvalue moved slowly to the position of the second, meaning it has not changed its role.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your eigenvalues vary only a little, you might get away with sorting the eigenvalues first by real part and then by imaginary part or by their absolute value. In general, however, this does not work.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2017-08-01T10:53:10.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27550" PostTypeId="2" ParentId="27503" CreationDate="2017-08-02T01:34:04.277" Score="4" Body="&lt;p&gt;My old answer has been edited in a way that I cannot agree with, but AG took such care to type everything out that Ill start again. This is a very old difficulty and the best textbook is Strang and Fix, An analysis of the Finite Element Method, Prentice Hall 1973 (I said it was old) The question was about finite differences, but the issue is the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You cannot match a Taylor series consistently to any order because with nine data points you have three points too many for a second-order expansion and one too few for a third-order one. The Finite Element approach, which is more rigorous and systematic than finite differences, assumes that the unknown function is drawn from some specific class of functions (called basis functions, usually polynomials or Fourier modes)  The assumption that I make (incomplete quartic, or biquadratic) is standard and makes the most symmetric choice possible. This helps with things like rotational invariance. It may help to notice that along any straight line the function is quadratic in that coordinate, which explains the name.&#xA;You cannot include a term like $x^3$ because you have no information relating to it. So assume that the unknown function is&#xA;$$f(x,y)=a_{0,0}+a_{1,0}x+a_{0,1}uy+a_{2,0}x^2+a_{1,1}xy+a_{0,2}y^2+a_{2,1}x^2y+a_{1,2}xy^2+a_{22}x^2y^2$$ where I choose a notation that avoids suggesting a Taylor expansion, and merely defines a chosen interpolating function. Fitting this function to the data gives 9 equations in nine unknowns. Thanks to MAPLE these can be solved for all of the coefficients, and one of these is&#xA;$$a_{1,1}={\frac {a-b}{ \left( c+d \right) ab} \left( {\frac {d{\it u_7}}{c}}-{&#xA;\frac {c{\it u_3}}{d}} \right) }+{\frac {c-d}{ \left( a+b \right) cd}&#xA; \left( {\frac {b{\it u_5}}{a}}-{\frac {a{\it u_1}}{b}} \right) }-{&#xA;\frac {1}{ \left( c+d \right)  \left( a+b \right) } \left( {\frac {ad{&#xA;\it u_8}}{bc}}+{\frac {bc{\it u_4}}{ad}}-{\frac {bd{\it u_6}}{ac}}-{&#xA;\frac {ac{\it u_2}}{bd}} \right) }+{\frac { \left( c-d \right)  \left( &#xA;a-b \right) {\it u_0}}{dcab}}&#xA;  $$&#xA;My notation is that $u_0$ is the value at the origin, $u_1$ is the value at $x=b,y=0$, and all other data is numbered anticlockwise around the square.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because this formula would be exact if the data were quadratic it is second-order accurate. It cannot be third-order because we cannot satisfy all the possible cubic terms. If this formula is used to predict the mixed derivative at any location except the origin of coordinates, it is only first order accurate. At the center, the correct second-order expression is&#xA;$$u_{xy}(NOT\:\: a_{1,1}!)={\frac {{\it u_2}}{ \left( c+d \right)  \left( a+b \right) }}-{\frac {{&#xA;\it u_4}}{ \left( c+d \right)  \left( a+b \right) }}+{\frac {{\it u_6}}{&#xA; \left( c+d \right)  \left( a+b \right) }}-{\frac {{\it u_8}}{ \left( c&#xA;+d \right)  \left( a+b \right) }}&#xA; $$&#xA;I do hope that I have managed to make things clearer this time.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-08-02T01:34:04.277" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27551" PostTypeId="1" CreationDate="2017-08-02T07:24:50.047" Score="1" ViewCount="291" Body="&lt;p&gt;I'm using LAPACK &lt;code&gt;zgeev&lt;/code&gt; to calculate eigenvectors of a symmetric complex matrix of high dimensions ($n \approx 2000$). I need these eigenvectors to satisfy &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{m=1}^{n} \frac{\lvert m \rangle \langle \overline m \lvert}{\langle \overline m\lvert m \rangle} = I_n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\lvert m\rangle$ is an eigenvector and $\langle \overline m\lvert$ is the same eigenvector transposed (&lt;strong&gt;Attention: Not conjugated&lt;/strong&gt;) and $I_n$ is the identity matrix of size $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know if all symmetric complex matrices satisfy this but this matrix has specific structure and it always should:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n=6$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vgAFb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vgAFb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Algorithm to generate these matrices are way too complicated to share, But I think one can get their pattern from these two examples)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$n=9$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MKhV4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MKhV4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the eigenvectors I get from &lt;code&gt;zgeev&lt;/code&gt; do not follow the equation I mentioned and they are not orthogonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested these matrices in Matlab and eigenvectors generated by Matlab satisfy this condition &lt;strong&gt;AND&lt;/strong&gt; are orthogonal. (i.e $\langle m_i\lvert \overline m_j \rangle=\delta_{ij}$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get the same eigenvectors from LAPACK &lt;code&gt;zgeev&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="23599" LastEditorUserId="23599" LastEditDate="2017-08-02T07:40:42.850" LastActivityDate="2017-08-02T07:40:42.850" Title="How to make LAPACK eigenvectors orthogonal like Matlab?" Tags="|linear-algebra|eigenvalues|eigensystem|lapack|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27552" PostTypeId="1" CreationDate="2017-08-02T12:45:07.130" Score="1" ViewCount="203" Body="&lt;p&gt;I want to write a simple simulation for heat conduction in a unstructured triangular mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already made it work for a structured rectangular grid with the ADI method, but now I need more complex geometries.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $d_x=\frac{\kappa \Delta t}{(\Delta x)^2}$ and $d_y=\frac{\kappa \Delta t}{(\Delta y)^2}$ you can say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d_x T_{i+1,j}^{n+1}+d_x T_{i-1,j}^{n+1}-(2d_x+2d_y+1)T_{i,j}^{n+1}+d_y T_{i,j-1}^{n+1}+d_y T_{i,j+1}^{n+1}=-T_{i,j}^n\ ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and you basically have a penta-diagonal equations system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the triangular grid, the only thing I came up with till now is to calculate the heat-flux-density over every edge, sum them up for every cell and add them to the temperature of the cell. For me this seems to be a explicit method, which (if it follows the same behaviour than the FTCS approximation) should be pretty instable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thats why I want to come up with an implicit method, but till now, I'm not able to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anybody some advice or some estimation on the stability of my explained method?&lt;/p&gt;&#xA;" OwnerUserId="25041" LastEditorUserId="9667" LastEditDate="2017-08-02T19:32:55.370" LastActivityDate="2017-08-02T19:32:55.370" Title="BTCS-like method for heat conduction in unstructured triangular grid" Tags="|pde|unstructured-mesh|heat-transfer|" AnswerCount="0" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27553" PostTypeId="1" CreationDate="2017-08-02T14:31:13.710" Score="5" ViewCount="1131" Body="&lt;p&gt;To improve the time efficiency of my code, I'd like to test a lower precision for real number, using e.g. half precision (2 bytes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not sure if I can do that in Fortran.&#xA;After playing with the intrinsic &lt;code&gt;SELECTED_REAL_KIND&lt;/code&gt; procedure, it seems that gfortran can only deal with 4, 8, 10 or 16 bytes precisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Actually, I could have directly &lt;a href=&quot;https://gcc.gnu.org/onlinedocs/gfortran/KIND-Type-Parameters.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;read the manual&lt;/a&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it then possible to use half precision floats in Fortran (maybe with another compiler)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, would there be an easy alternative to prototype half precision code?&lt;/p&gt;&#xA;" OwnerUserId="19808" LastEditorUserId="19808" LastEditDate="2017-08-05T12:55:28.123" LastActivityDate="2017-08-05T12:55:28.123" Title="Half precision in Fortran" Tags="|fortran|precision|efficiency|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27554" PostTypeId="2" ParentId="27553" CreationDate="2017-08-02T15:30:55.843" Score="3" Body="&lt;p&gt;The Fortran standard doesn't specify what precisions and ranges of floating point numbers need be supported by a given compiler. All it says is that at least two different kinds of real numbers need be supported and that larger of them takes exactly twice the memory to represent a number as the other (smaller) kind. What is actually supported depends upon what the compiler writer chose, though of course today to not find something that looks like IEEE floating point numbers would be rare.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However note I say &quot;at least two&quot;, the compiler is free to support as many as the compiler writer wants to implement. Thus in theory there may be a compiler that supports what you want. Unfortunately I don't know of one. &lt;/p&gt;&#xA;" OwnerUserId="17475" LastActivityDate="2017-08-02T15:30:55.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27558" PostTypeId="1" CreationDate="2017-08-03T16:33:18.380" Score="1" ViewCount="506" Body="&lt;p&gt;I am getting this strange error with paraview 5.4. &#xA;For example, if I load one of the example visualisations:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zohPU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zohPU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and if I export it as eps or pdf, then the color legend is place in the bottom left corner:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Olg9E.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Olg9E.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does not happen for jpg or png files.&#xA;Anybody know how to fix this?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="25062" LastActivityDate="2017-08-03T16:33:18.380" Title="Color legend location in eps files, exported from paraview v5.4" Tags="|paraview|" AnswerCount="0" CommentCount="2" ClosedDate="2017-08-05T07:14:23.223" ContentLicense="CC BY-SA 3.0" />
  <row Id="27559" PostTypeId="1" CreationDate="2017-08-03T18:14:50.550" Score="3" ViewCount="288" Body="&lt;p&gt;I'm solving 2D Euler's equations in Cartesian coordinates:&#xA;$$&#xA;\frac{\partial U}{\partial t} + \frac{\partial F}{\partial x} + \frac{\partial G}{\partial y} = 0, \qquad &#xA;U = \left( \begin{array}{c} \rho \\ \rho u \\ \rho v \\ e \end{array} \right), \quad&#xA;F = \left( \begin{array}{c} \rho u \\ p+\rho u^2 \\ \rho uv \\ (e+p)u \end{array} \right), \quad&#xA;G = \left( \begin{array}{c} \rho v \\ \rho uv \\ p+\rho v^2 \\ (e+p)v \end{array} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using finite difference MacCormack method with Davis artificial viscosity which dampens non-physical oscillations. The MacCormack method is pretty simple and not shown here. Davis viscosity works good in my simulations, but is quite difficult; I'm trying to fully understand it in order to apply it cylindrical coordinates later. I hope that some of you may help me with unclear parts of it (shown in &lt;strong&gt;bold&lt;/strong&gt; font below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Viscous terms are added after 2nd MacCormack step as follows (formulated in 1D for simplicity):&#xA;$$&#xA;U_j^{n+1}=U_j^{(m)} + \left( D_{j+\frac{1}{2}}-D_{j-\frac{1}{2}} \right),&#xA;$$&#xA;where $U_j^{(m)}$ denote field values (at $j$-th coordinate point and $(n+1)$-th time layer) calculated by MacCormack scheme as usual, and $D_{j+\frac{1}{2}}, D_{j-\frac{1}{2}}$, are forward and backward differences of U with non-linear coefficients:&#xA;$$&#xA;D_{j+\frac{1}{2}} := \frac{1}{2} C(\nu_j) \left( 2 - \phi(r_j^+) - \phi(r_{j+1}^-) \right) \Delta U_{j+\frac{1}{2}}, \quad \Delta U_{j+\frac{1}{2}} = U_{j+1} - U_j \\&#xA;D_{j-\frac{1}{2}} := \frac{1}{2} C(\nu_{j-1}) \left( 2 - \phi(r_{j-1}^+) - \phi(r_{j}^-) \right) \Delta U_{j-\frac{1}{2}}, \quad \Delta U_{j-\frac{1}{2}} = U_{j} - U_{j-1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $C(\nu)$ is a limited coefficient based on Courant number:&#xA;$$&#xA;C(\nu) := \begin{cases} \nu(1-\nu) &amp;amp; \;\; \text{if} \;\; \nu \le 0.5 \\ 0.25 &amp;amp; \;\; \text{otherwise} \end{cases}, \qquad \nu = \rho(A(U_j)) \frac{\Delta t}{\Delta x},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A(U)$ is the Jacobian matrix of vector $F$, $\rho(A(U))$ is it's spectral radius &lt;strong&gt;(for Euler's equations it equals $\text{max}(|u|+a, |u|-a)$, where $a$ is the speed of sound)&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi(r)$ is simple limiting function: $\begin{cases} 0, &amp;amp; r &amp;lt; 0, \\ 2r, &amp;amp; 0 \le r \le 0.5, \\ 1, &amp;amp; r &amp;gt;0.5\\ \end{cases}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally $r_j^+$,$r_j^-$ are slope ratios:&#xA;$$&#xA;r_j^+ := \frac{\left( \Delta U_{j-\frac{1}{2}}, \Delta U_{j+\frac{1}{2}}\right)}{\left( \Delta U_{j+\frac{1}{2}}, \Delta U_{j+\frac{1}{2}}\right)}, \qquad&#xA;r_j^- := \frac{\left( \Delta U_{j-\frac{1}{2}}, \Delta U_{j+\frac{1}{2}}\right)}{\left( \Delta U_{j-\frac{1}{2}}, \Delta U_{j-\frac{1}{2}}\right)}&#xA;$$&#xA;Where $\left( \cdot , \cdot \right)$ is scalar product. These ratios are positive in monotone areas and negative in non-monotone areas, so that in general coefficients like $\left( 2 - \phi(r_j^+) - \phi(r_{j+1}^-) \right)$ are equal 2 in non-monotone areas; about 1 in monotone, but &quot;curvy&quot; areas; and equal 0 in monotone (and rectilinear) areas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If both these ($2-\phi-\phi$) coefficients are equal 2, and both $C(\nu)$ are equal 0.25 we have an effective &lt;strong&gt;second derivative of the field value&lt;/strong&gt;:&#xA;$$&#xA;D_{j+\frac{1}{2}}-D_{j-\frac{1}{2}} = 0.25 \left( U_{j+1} - 2U_j + U_{j-1} \right).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But where is the division by $\Delta x^2$?&lt;/strong&gt; If the viscosity would be simply the 2nd derivative, the scheme would look like&#xA;$$&#xA;U_j^{n+1}=U_j^{(m)} + \frac{\Delta t}{\Delta x^2} \left( U_{j+1} - 2U_j + U_{j-1} \right).&#xA;$$&#xA;Is $C(\nu)$ effectively equals $\frac{\Delta t}{\Delta x^2}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;And, what if $D_{j+\frac{1}{2}}$ is non-zero and $D_{j-\frac{1}{2}}$ is zero?&lt;/strong&gt; Does it still yield an effective 2nd derivative?&lt;/p&gt;&#xA;" OwnerUserId="25060" LastEditorUserId="25060" LastEditDate="2017-08-06T08:23:10.830" LastActivityDate="2017-08-06T08:23:10.830" Title="Understanding Davis artificial viscosity" Tags="|finite-difference|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27560" PostTypeId="1" CreationDate="2017-08-04T05:42:49.940" Score="2" ViewCount="551" Body="&lt;p&gt;I am solving a 1d Shock tube problem of compressible fluid obeying Euler equations(Hyperbolic pde). I am trying to simulate it using Finite Volume Method using Roe's Scheme. Half of the tube contains highly compressed fluid and other half is not compressed with density ratio of ~1000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Euler Equation: $$ ∂Q/∂t + ∂F/∂x = 0; A = ∂F/∂Q$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Roe's Scheme, the parameters of velocity 'u' and Enthalpy 'H', at the interface are need for determining the 'A' a 3x3 matrix(for 1D Euler equation) . They are obtained by satisfying the following condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A (Q_\text{Left} - Q_\text{Right}) = \text{F}_\text{Left} - \text{F}_\text{Right}\ .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the density, $\rho$ remains undetermined. While reading through literature, it was mentioned that it is 'Natural' to consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho = \sqrt{\rho_\text{Left}.\rho_\text{Right}}\ .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is why is it so and what happens if we consider density to be a arithmetic mean(or some other mean) of the two density values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to obtain Matrix 'A' for a different Equation of State(EoS) other than ideal gas EoS. If I choose a particular value of the density(calculated from density on either side of the face), the Roe's average values are greatly simplified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EoS  considered: $$ Pressure, P(\rho, E) = (\gamma-1).\rho E+g(\rho)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was obtaining the density from following condition for simplification of equations for 'u' and 'H'.&#xA;$$ dg(\rho)/d\rho = (g(\rho_\text{Left})-g(\rho_\text{Right}))/(\rho_\text{Left}-\rho_\text{Right}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for my question is enquire about the mathematical implications in choosing a density, other than 'Natural' square root value.&lt;/p&gt;&#xA;" OwnerUserId="24939" LastEditorUserId="24939" LastEditDate="2017-08-09T18:50:07.567" LastActivityDate="2019-03-14T15:14:07.533" Title="Computing Roe's average density for General Equation of State" Tags="|finite-volume|hyperbolic-pde|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27561" PostTypeId="2" ParentId="27503" CreationDate="2017-08-04T06:21:20.820" Score="3" Body="&lt;p&gt;Here we shall follow two procedure to calculate $f_{xy}$ one using Taylors series and another using polynomial fitting. Though both the procedure end up in the same formulation there is some advantages and disadvantage for each one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Using Taylors series:&lt;/strong&gt;&#xA;The advantage of this procedure is: having control over truncation error is easy because we can easily decide what order we want by choosing the order equations that we desired. Programming this is slightly tedious than polynomial series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's expand all the points using Taylors series will end up in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(I)\qquad f(x+b,y+d)\approx f + b f_x+d f_y +\frac{b^2}{2}f_{xx} +bdf_{xy} +\frac{d^2}{2}f_{yy} + \frac{b^3}{6}f_{xxx} + \frac{d^3}{6}f_{yyy} + \frac{1}{2} b^2df_{xxy}+ \frac{1}{2} d^2bf_{xyy} +\frac{1}{4} d^2b^2f_{xxyy}$&#xA;$(II)\qquad f(x+b,y-c)\approx f + b f_x-c f_y +\frac{b^2}{2}f_{xx} -bcf_{xy} +\frac{c^2}{2}f_{yy}+ \frac{b^3}{6}f_{xxx} - \frac{c^3}{6}f_{yyy} - \frac{1}{2} b^2cf_{xxy}+ \frac{1}{2} c^2bf_{xyy}+\frac{1}{4} c^2b^2f_{xxyy}$&#xA;$(III)\qquad f(x-a,y-c)\approx f - a f_x-c f_y +\frac{a^2}{2}f_{xx} +acf_{xy} +\frac{c^2}{2}f_{yy}- \frac{a^3}{6}f_{xxx} - \frac{c^3}{6}f_{yyy} - \frac{1}{2} a^2cf_{xxy}- \frac{1}{2} a^2cf_{xyy}+\frac{1}{4} a^2c^2f_{xxyy}$&#xA;$(IV)\qquad f(x-a,y+d)\approx f - a f_x+d f_y +\frac{a^2}{2}f_{xx} -adf_{xy} +\frac{d^2}{2}f_{yy}- \frac{a^3}{6}f_{xxx} + \frac{d^3}{6}f_{yyy} + \frac{1}{2} a^2df_{xxy}- \frac{1}{2} a^2df_{xyy}+\frac{1}{4} d^2a^2f_{xxyy}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(V)\qquad f(x,y+d)\approx f +d f_y   +\frac{d^2}{2}f_{yy} + \frac{d^3}{6}f_{yyy} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(VI)\qquad f(x,y-c)\approx f -c f_y   +\frac{c^2}{2}f_{yy} - \frac{c^3}{6}f_{yyy} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(VII)\qquad f(x+b,y)\approx f + b f_x +\frac{b^2}{2}f_{xx}  + \frac{b^3}{6}f_{xxx}  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(VIII)\qquad f(x-c,y)\approx f - c f_x +\frac{c^2}{2}f_{xx}  - \frac{c^3}{6}f_{xxx}  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(IX)\qquad f(x,y)= f  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $a_1$ is the weight of equation (I), $a_2$ is the weight of equation (II) &#xA;by equating $\sum a_i$*(equation number) =$0*f+0*f_x+...+ 1*f_{xy}+...+0*f_{xyy}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we are having nine points we need nine equations to solve this but we are having more equations than unknown but only some combinations can give results. Choosing those combinations could be based on what order of accuracy required. Let fix our highest derivative term shouldn't be more than 3. If we tried to solve we can't obtain an exact solution for 9*9, the easiest way to get a solution is freeing some variables.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    clc&#xA;    clear all&#xA;&#xA;    syms a1 a2 a3 a4 a5 a6 a7 a8 a7 a8 a9 a b c d % weight to each points&#xA;        eq1=a1 + a2 + a3 + a4 + a5 + a6 + a7 + a8+a9; % first order equation&#xA;&#xA;        eq2= a1*b - a*a4 - a*a3 + a2*b + a7*b - a8*c;&#xA;&#xA;        eq3=a1*d - a3*c - a6*c - a2*c + a4*d + a5*d;&#xA;&#xA;        eq4=a^2*a3 + a^2*a4 + a1*b^2 + a2*b^2 + a7*b^2 + a8*c^2;&#xA;&#xA;        eq5=a*a3*c - a2*b*c - a*a4*d + a1*b*d-1;&#xA;&#xA;        eq6=a2*c^2 + a3*c^2 + a6*c^2 + a1*d^2 + a4*d^2 + a5*d^2;&#xA;&#xA;        eq7=a1*b^3 - a^3*a4 - a^3*a3 + a2*b^3 + a7*b^3 - a8*c^3;&#xA;&#xA;        eq8=a1*d^3 - a3*c^3 - a6*c^3 - a2*c^3 + a4*d^3 + a5*d^3;&#xA;&#xA;        eq9=a^2*a4*d - a2*b^2*c - a^2*a3*c + a1*b^2*d;&#xA;&#xA;        eq10=- a3*a^2*c - a4*a^2*d + a2*b*c^2 + a1*b*d^2;&#xA;&#xA;        eq11 =d^2*b^2*a1+c^2*b^2*a2+a^2*c^2*a3+d^2*a^2*a4;&#xA;&#xA;         sol1=solve(eq1,eq2,eq3,eq4,eq5,eq6,eq9,eq10,'a1','a2','a3','a4','a5','a6','a7','a8') % one of the possible solutions&#xA;    sol1=solve(eq1,eq2,eq3,eq4,eq5,eq8,eq9,eq10,'a1','a2','a3','a4','a5','a6','a7','a8') % another possible solution&#xA;    sol1=solve(eq1,eq2,eq3,eq4,eq5,eq7,eq9,eq10,'a1','a2','a3','a4','a5','a6','a7','a8') % another possible solution&#xA;&#xA;    a9=1; % free variable&#xA;    w1=sol1.a1;&#xA;    w2=sol1.a2;&#xA;    w3=sol1.a3;&#xA;    w4=sol1.a4;&#xA;    w5=sol1.a5;&#xA;    w6=sol1.a6;&#xA;    w7=sol1.a7;&#xA;    w8=sol1.a8;&#xA;    w9=a9;&#xA;&#xA;%     Test problem&#xA;    syms x y&#xA;    f=@(x,y) 1 +1*x +1*y +2*x*y +1*x^2 +1*y^2 +1*x^2*y+1*x*y^2+1*x^3+1*y^3+x^4*y;&#xA;&#xA;f_x=diff(f,x);&#xA;f_xy=diff(f_x,y);&#xA;&#xA;f_xy_00=subs(f_xy,[x y], [0 0]);&#xA;&#xA;f_00=subs(f_xy,[x y],[0 0]); &#xA;disp('exact solution is')&#xA;disp(f_00)&#xA;a=0.1;&#xA;b=0.1;&#xA;c=0.1;&#xA;d=0.1;&#xA;P00=f(0,0);&#xA;P10=f(b,0);&#xA;P11=f(b,d);&#xA;P01=f(0,d);&#xA;P_11=f(-a,d);&#xA;P_10=f(-a,0);&#xA;P_1_1=f(-a,-c);&#xA;P0_1=f(0,-c);&#xA;P1_1=f(b,-c);&#xA;&#xA;&#xA;f_xyn=w1*P11+w2*P1_1+w3*P_1_1+w4*P_11+w5*P01+w6*P0_1+w7*P10+w8*P_10+w9*P00; % numerical solution&#xA;&#xA;disp('Numerical solution is')&#xA;disp(f_xyn) %sry I'm unable to use subs here so pls copy paste it agin in command window&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Though all the answers are more or less same, they differ slighly based on what order we have considered. Similar to another answer, this sytem can be closed by omitting $f_{xxx}$, $f_{yyy}$ because they needs 4 points in $x$ or $y$ direction but we have only 3 points but we can calculate $f_{xxyy}$, it needs only 3 points in both the directions. If we include then the code become&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc&#xA;clear all&#xA;&#xA;syms a1 a2 a3 a4 a5 a6 a7 a8 a7 a8 a9 a b c d % weight to each point&#xA;    eq1=a1 + a2 + a3 + a4 + a5 + a6 + a7 + a8+a9; % first order equation&#xA;&#xA;    eq2= a1*b - a*a4 - a*a3 + a2*b + a7*b - a8*c;&#xA;&#xA;    eq3=a1*d - a3*c - a6*c - a2*c + a4*d + a5*d;&#xA;&#xA;    eq4=a^2*a3 + a^2*a4 + a1*b^2 + a2*b^2 + a7*b^2 + a8*c^2;&#xA;&#xA;    eq5=a*a3*c - a2*b*c - a*a4*d + a1*b*d-1;&#xA;&#xA;    eq6=a2*c^2 + a3*c^2 + a6*c^2 + a1*d^2 + a4*d^2 + a5*d^2;&#xA;&#xA;    eq7=a1*b^3 - a^3*a4 - a^3*a3 + a2*b^3 + a7*b^3 - a8*c^3;&#xA;&#xA;    eq8=a1*d^3 - a3*c^3 - a6*c^3 - a2*c^3 + a4*d^3 + a5*d^3;&#xA;&#xA;    eq9=a^2*a4*d - a2*b^2*c - a^2*a3*c + a1*b^2*d;&#xA;&#xA;    eq10=- a3*a^2*c - a4*a^2*d + a2*b*c^2 + a1*b*d^2;&#xA;&#xA;    eq11 =d^2*b^2*a1+c^2*b^2*a2+a^2*c^2*a3+d^2*a^2*a4;&#xA;&#xA;    sol1=solve(eq1,eq2,eq3,eq4,eq5,eq6,eq9,eq10,eq11,'a1','a2','a3','a4','a5','a6','a7','a8','a9');&#xA;    w1=sol1.a1;&#xA;    w2=sol1.a2;&#xA;    w3=sol1.a3;&#xA;    w4=sol1.a4;&#xA;    w5=sol1.a5;&#xA;    w6=sol1.a6;&#xA;    w7=sol1.a7;&#xA;    w8=sol1.a8;&#xA; w9=sol1.a8;&#xA;&#xA;%     Test problem&#xA;    syms x y&#xA;    f=@(x,y) 1 +1*x +1*y +2*x*y +1*x^2 +1*y^2 +1*x^2*y+1*x*y^2+1*x^3+1*y^3+x^4*y;&#xA;&#xA;f_x=diff(f,x);&#xA;f_xy=diff(f_x,y);&#xA;&#xA;f_xy_00=subs(f_xy,[x y], [0 0]);&#xA;&#xA;f_00=subs(f_xy,[x y],[0 0]); &#xA;disp('exact solution is')&#xA;disp(f_00)&#xA;a=0.1;&#xA;b=0.1;&#xA;c=0.1;&#xA;d=0.1;&#xA;P00=f(0,0);&#xA;P10=f(b,0);&#xA;P11=f(b,d);&#xA;P01=f(0,d);&#xA;P_11=f(-a,d);&#xA;P_10=f(-a,0);&#xA;P_1_1=f(-a,-c);&#xA;P0_1=f(0,-c);&#xA;P1_1=f(b,-c);&#xA;f_xyn=w1*P11+w2*P1_1+w3*P_1_1+w4*P_11+w5*P01+w6*P0_1+w7*P10+w8*P_10+w9*P00; % numerical solution&#xA;&#xA;disp('Numerical solution is')&#xA;disp(f_xyn)  %sry I'm unable to use subs here so pls copy paste it again in the command window&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Using polynomial fit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can fit a 2-D polynomial over the domain and you can find the derivative value. &#xA;First, we shall consider 2-D cubic polynomial then we shall find $f_{xy}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and matlab code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc&#xA;clear all&#xA;close all&#xA;syms a0 a10 a01 a11 a20 a02  a12 a21 a30 a22 x y&#xA;syms a b c d&#xA;syms P00 P10 P11 P01 P_11 P_10 P_1_1 P0_1 P1_1&#xA;f=@(x,y) a0 +a10*x +a01*y +a11*x*y +a20*x^2 +a02*y^2 +a21*x^2*y+a12*x*y^2+a30*x^3;&#xA;&#xA;f_x=diff(f,x);&#xA;f_xy=diff(f_x,y);&#xA;&#xA;f_00=subs(f_xy,[x y],[0 0]);&#xA;&#xA;p00=f(0,0);&#xA;p10=f(b,0);&#xA;p11=f(b,d);&#xA;p01=f(0,d);&#xA;p_11=f(-a,d);&#xA;p_10=f(-a,0);&#xA;p_1_1=f(-a,-c);&#xA;p0_1=f(0,-c);&#xA;p1_1=f(b,-c);&#xA;&#xA;sol1=solve(p00-P00,p10-P10,p11-P11,p01-P01,p_11-P_11,p_10-P_10,p_1_1-P_1_1,p0_1-P0_1,'a0','a10','a01','a11','a20','a02','a12','a21');&#xA;&#xA;F_xy=sol1.a11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$f_{xy} =\frac{(P_{10} - P_{11} - P_{-10} + P_{-11})}{(d*(a + b))}$ + $\frac{(P_{00} - P_{01} - P_{10} + P_{11})}{(b*d)} $- $\frac{(P_{00}*c^2-P_{01}*c^2-P_{-10}*c^2+P_{-11}*c^2-P_{00}*d^2+P_{0-1}*d^2+P_{-10}*d^2-P_{-1-1}*d^2)}{(a*c*d*(c + d))}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The notation used here. $P_{xy}$ is the point  located in the quadrent where $(x,y)$ =($0+x$, $0+y$) lies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar to previous arguments, for $f_{xxyy}$ we can fit a quartic polynomial without $x^3$ and $y^3$ and the Matlab code is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc&#xA;clear all&#xA;close all&#xA;syms a0 a10 a01 a11 a20 a02  a12 a21 a22 x y&#xA;syms a b c d&#xA;syms P00 P10 P11 P01 P_11 P_10 P_1_1 P0_1 P1_1&#xA;f=@(x,y) a0 +a10*x +a01*y +a11*x*y +a20*x^2 +a02*y^2 +a21*x^2*y+a12*x*y^2+a22*x^2*y^2;&#xA;&#xA;f_x=diff(f,x);&#xA;f_xy=diff(f_x,y);&#xA;&#xA;f_00=subs(f_xy,[x y],[0 0]);&#xA;&#xA;p00=f(0,0);&#xA;p10=f(b,0);&#xA;p11=f(b,d);&#xA;p01=f(0,d);&#xA;p_11=f(-a,d);&#xA;p_10=f(-a,0);&#xA;p_1_1=f(-a,-c);&#xA;p0_1=f(0,-c);&#xA;p1_1=f(b,-c);&#xA;&#xA;sol1=solve(p00-P00,p10-P10,p11-P11,p01-P01,p_11-P_11,p_10-P_10,p_1_1-P_1_1,p0_1-P0_1,p1_1-P1_1,'a0','a10','a01','a11','a20','a02','a12','a21','a22');&#xA;&#xA;F_xy=sol1.a11 %f_xy result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$f_{xy} =\frac{(P_{00}*a^2*c^2 - P_{01}*a^2*c^2 - P_{10}*a^2*c^2 + P_{11}*a^2*c^2 - P_{00}*a^2*d^2 - P_{00}*b^2*c^2 + P_{01}*b^2*c^2 + P_{10}*a^2*d^2 + P_{0-1}*a^2*d^2 - P_{1-1}*a^2*d^2 + P_{-10}*b^2*c^2 - P_{-11}*b^2*c^2 + P_{00}*b^2*d^2 - P_{0-1}*b^2*d^2 - P_{-10}*b^2*d^2 + P_{-1-1}*b^2*d^2)}{(a*b*c*d*(a + b)*(c + d))}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Test case:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc&#xA;clear all&#xA;close all&#xA;&#xA;syms x y&#xA;f=@(x,y) 1 +1*x +1*y +2*x*y +1*x^2 +1*y^2 +1*x^2*y+1*x*y^2+1*x^3+1*y^3+x^4*y;&#xA;&#xA;f_x=diff(f,x);&#xA;f_xy=diff(f_x,y);&#xA;&#xA;f_xy_00=subs(f_xy,[x y], [0 0]);&#xA;&#xA;f_00=subs(f_xy,[x y],[0 0]);&#xA;a=0.1;&#xA;b=0.1;&#xA;c=0.1;&#xA;d=0.1;&#xA;P00=f(0,0);&#xA;P10=f(b,0);&#xA;P11=f(b,d);&#xA;P01=f(0,d);&#xA;P_11=f(-a,d);&#xA;P_10=f(-a,0);&#xA;P_1_1=f(-a,-c);&#xA;P0_1=f(0,-c);&#xA;P1_1=f(b,-c);&#xA;&#xA;disp('exact solution is')&#xA;disp(f_00)&#xA;f_22=(P00*a^2*c^2 - P01*a^2*c^2 - P10*a^2*c^2 + P11*a^2*c^2 - P00*a^2*d^2 - P00*b^2*c^2 + P01*b^2*c^2 + P10*a^2*d^2 + P0_1*a^2*d^2 - P1_1*a^2*d^2 + P_10*b^2*c^2 - P_11*b^2*c^2 + P00*b^2*d^2 - P0_1*b^2*d^2 - P_10*b^2*d^2 + P_1_1*b^2*d^2)/(a*b*d*(a + b)*(c^2 + d*c));&#xA;&#xA;&#xA;disp('Numerical solution is')&#xA;disp(f_22)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Though there are a lot of ways to derive it, I personally prefer FD derived using polynomial because of its simplicity and accuracy of last $f_{xy}$ is higher than the previous one because it satisfies more terms in Taylors series.&lt;/p&gt;&#xA;" OwnerUserId="16930" LastActivityDate="2017-08-04T06:21:20.820" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27562" PostTypeId="1" AcceptedAnswerId="27612" CreationDate="2017-08-04T13:08:07.163" Score="0" ViewCount="185" Body="&lt;p&gt;I am solving a non linear second order implicit initial value problem using finite difference method, but my results do not converge. Please guide me with an example, how we can apply finite difference method with big accuracy.&#xA;The problem is $$(A+By(t)-C(-y'(t))^-1/3)y''(t)=-(By'(t)^2+C),\qquad y(t=0)=0,$$ using fdm, time can be from zero to 2 mints. A,B and C are constants.&lt;/p&gt;&#xA;" OwnerUserId="25067" LastEditorUserId="1804" LastEditDate="2017-08-04T14:48:31.783" LastActivityDate="2017-08-11T22:59:50.937" Title="implicit odes solution using fdm" Tags="|ode|nonlinear-equations|time-integration|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27564" PostTypeId="2" ParentId="26871" CreationDate="2017-08-04T14:32:53.597" Score="2" Body="&lt;p&gt;Additionally to what @richard-zang said, instead of a &quot;naive brute-force&quot; search, you can often use some refinement, e.g. a &lt;a href=&quot;https://en.wikipedia.org/wiki/Locality-sensitive_hashing&quot; rel=&quot;nofollow noreferrer&quot;&gt;locality-based hashing&lt;/a&gt; or if you have fixed neighbor distance radius, a common approach is to pre-grid/sort the search space to limit the lookup to neighboring cells (commonly used in molecular simulation and referred to as &lt;a href=&quot;https://en.wikipedia.org/wiki/Cell_lists&quot; rel=&quot;nofollow noreferrer&quot;&gt;linked cell list&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5359" LastActivityDate="2017-08-04T14:32:53.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27565" PostTypeId="2" ParentId="27562" CreationDate="2017-08-04T15:40:03.160" Score="1" Body="&lt;p&gt;Since you have a second order ODE you must provide 2 initial conditions, say:&#xA;$$y'(0) = y'_0 \qquad y(0) = y_0\tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you can do is transform the original second order equation:&#xA;$$\left(A+By-Cy'^{-1/3}\right)y''=-\left(By'^2+C\right) \tag{2}$$&#xA;into a system of first order ODEs&#xA;$$\left\{\begin{array}{ll}%&#xA;v' =&amp;amp; -\frac{Bv^2+C}{A+By-Cv^{-1/3}}\\&#xA;y'=&amp;amp;v&#xA;\end{array}\right.  \tag{3}$$&#xA;with initial conditions:&#xA;$$ v(0) = y'_0 \qquad  y(0) = y_0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;System $(2)$ can be put into one general vector equation:&#xA;being $\vec{x} = [x_1,x_2]^{T}$ for $x_1=v$ and $x_2=y$ and $\vec{f} = \left[-\frac{Bv^2+C}{A+By-Cv^{-1/3}},\,v\right]^T$. Therefore:&#xA;$$\frac{d \vec{x}}{dt} = \vec{f}(\vec{x}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yu can solve it easily with matlab using any ODE solver that supports stiff equations (implicit numerical method).&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-08-04T15:54:13.723" LastActivityDate="2017-08-04T15:54:13.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27566" PostTypeId="2" ParentId="27292" CreationDate="2017-08-05T12:11:08.057" Score="0" Body="&lt;p&gt;I would use the Bilateral Filter or the Non Local Means Filter.&lt;/p&gt;&#xA;&lt;p&gt;Both are straight forward to implement and their results are surprisingly good.&lt;br /&gt;&#xA;They retain much more details than the Spatially Invariant Linear Filter such as the Gaussian Filter.&lt;/p&gt;&#xA;" OwnerUserId="7951" LastEditorUserId="7951" LastEditDate="2020-09-28T20:56:33.953" LastActivityDate="2020-09-28T20:56:33.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27567" PostTypeId="1" CreationDate="2017-08-05T14:55:40.773" Score="1" ViewCount="71" Body="&lt;p&gt;I've a read a paper about investing where they used a dynamic programming approach to solve a finite horizon problem, i.e. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{x_t} E[u(W_T)] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is a utility function and $W_T$ denotes the terminal wealth. The Bellman equation can then be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ V_t(W_t) = \max_{x_t}E[V_{t+1}(W_{t+1})|W_t]$$&#xA;with $V_T=u$. I have a continuous and concave utilitiy function which penalizes not hitting a certain target amount&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$  u(w) = &#xA;  \begin{cases} &#xA;   K_1 &amp;amp; \text{if } w \geq K_1 \\&#xA;   w - c\max{(0,(K_2-w))}^2       &amp;amp; \text{if } w &amp;lt; K_1&#xA;  \end{cases}$$&#xA;where $K_1&amp;gt;K_2$ are constant and $c&amp;gt;0$ a scaling factor. Above $K_1$, $u$ is constant, below $K_2$ you add a penalty. $c$ is a control parameter how much we should penalize $u$. For $K_1 = 12$, $K_2=10$ and $c= 100$ we get the following picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dWuQv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dWuQv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and to see the linearity of $u$ between $K_2$ and $K_1$ another picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gFFmS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gFFmS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I'm using a backward induction algorithm and discretize the space I've naively tried a uniform space grid $\{x_i\}$ in a fairly large interval $[w_1,w_2]$. Afterwards, I've solved for all the grid points $\{x_i\}$ the problem leading to solutions $y_i$. The pair $(x_i, y_i)$ was then used to approximate $V_t$. For this I've used a shape preserving method. What I've seen from experimenting with the problem that the approximation is very good far out on the left and right side, away from high curvature region. Moreover, the solution becomes quite stationary in these &quot;wing regions&quot;, too. What I then thought is to generate a non uniform grid which is denser where curvature is high. The problem is I don't know a priori the shape of $V_t$ (expect for $t=T$) so that it's hard to come up with a rule. Is there a generic way how to construct such non uniform grids? If someone has a concrete example or could point me into the right direction to do my own research, that would be really appreciated. I will potentially add another dimension to the state space so that the same question would arise in a higher dimension. That's why I put the mesh-generation tag as well&lt;/p&gt;&#xA;" OwnerUserId="21291" LastEditorUserId="24110" LastEditDate="2017-08-07T06:55:37.787" LastActivityDate="2017-08-07T06:55:37.787" Title="adaptive / smart grid choice for dynamic programming" Tags="|mesh-generation|grid|dynamic-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27568" PostTypeId="2" ParentId="27487" CreationDate="2017-08-05T19:04:39.613" Score="1" Body="&lt;p&gt;Like @Steve already said in a comment, the correct answer is to look for &quot;nonuniform fast Fourier transform&quot; algorithms. There is both a significant literature, as well as open source software packages on this topic.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-08-05T19:04:39.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27569" PostTypeId="1" AcceptedAnswerId="27570" CreationDate="2017-08-05T20:50:22.990" Score="2" ViewCount="303" Body="&lt;p&gt;I'm trying to solve a very simple advection PDE &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial u}{\partial t}+c\frac{\partial u}{\partial x}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c&amp;lt;0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been able to implement &lt;a href=&quot;https://stackoverflow.com/questions/45496863/openmodelica-solving-a-pde-initialization-error/45526159#45526159&quot;&gt;a simple Modelica code&lt;/a&gt; to solve the equation and it works just fine for $c&amp;gt;0$ with the exact same boundary conditions, though. So my question is why my code is not able to finish the simulation:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it a mathematical issue? Like such PDE never has a solution for $c&amp;lt;0$?&lt;/li&gt;&#xA;&lt;li&gt;is it a finite difference limitation and maybe I should use other numerical methods?&lt;/li&gt;&#xA;&lt;li&gt;or there is some issue with &lt;a href=&quot;https://stackoverflow.com/questions/45496863/openmodelica-solving-a-pde-initialization-error/45526159#45526159&quot;&gt;the code&lt;/a&gt; or Modelica compiler?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if you could help me figure this issue out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. the goal for me is to learn how to solve PDEs in Modelica and this is just one example.&lt;/p&gt;&#xA;" OwnerUserId="22193" LastEditorUserId="549" LastEditDate="2017-08-07T18:22:32.963" LastActivityDate="2017-08-07T18:22:32.963" Title="Finite difference method not working for advection PDE with negative coefficient" Tags="|pde|advection|openmodelica|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27570" PostTypeId="2" ParentId="27569" CreationDate="2017-08-05T21:54:22.213" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;and it works just fine for c&gt;0 with the exact same boundary conditions, though.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's your issue. Look up the method of characteristics:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://web.stanford.edu/class/math220a/handouts/firstorder.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/class/math220a/handouts/firstorder.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The characteristics for a 1D advection problem flow in one direction. So there are two things involved. One is that the BCs on one side of the interval matter since the characteristics are only flowing in one direction. Secondly,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Upwind_scheme&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Upwind_scheme&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the upwind scheme has to change directions depending on the direction of the characteristics. So you need to change your implementation to match the direction as given by &lt;code&gt;sign(c)&lt;/code&gt;. When &lt;code&gt;c&amp;lt;0&lt;/code&gt;, you need to use &lt;code&gt;u[i+1]&lt;/code&gt; instead of &lt;code&gt;u[i-1]&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-08-05T21:54:22.213" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27571" PostTypeId="1" CreationDate="2017-08-07T00:19:37.720" Score="1" ViewCount="1497" Body="&lt;p&gt;I am trying to use backwards finite difference method to numerically solve a pair of partial differential equations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$\frac{\partial \left(pv\right)}{\partial x}+\frac{\partial p}{\partial t}=0$&lt;/li&gt;&#xA;&lt;li&gt;$\frac{\partial \left(pv^2\right)}{\partial x}+\frac{\partial \left(pv\right)}{\partial t}+c\frac{\partial p}{\partial x}=0$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;with initial and boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$v\left(x,0 \right)=0$&lt;/li&gt;&#xA;&lt;li&gt;$p\left(x,0 \right)=p_a$&lt;/li&gt;&#xA;&lt;li&gt;$p\left(L,t \right)=p_a$&lt;/li&gt;&#xA;&lt;li&gt;$\left\{\begin{matrix}&#xA;  if \, t&amp;lt;t_s \, then &amp;amp;  p\left(0,t \right)=p_a \, and \, v\left(0,t \right)=0 \\&#xA;  else &amp;amp;  p\left(0,t \right)=p_r&#xA; \end{matrix}\right.$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These are the Navier-Stokes equations for a one dimensional compressible inviscid isothermal fluid in a tube with one open end and step pressure signal at $t=t_s$ in the other end. gravity, radiation and conduction are negligible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a backwards finite difference method we have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;${p_t}_{i,j}=\frac{\left(p_{i,j}-p_{i,j-1}\right)}{\delta t}$&lt;/li&gt;&#xA;&lt;li&gt;${p_x}_{i,j}=\frac{\left(p_{i,j}-p_{i-1,j}\right)}{\delta x}$&lt;/li&gt;&#xA;&lt;li&gt;${v_t}_{i,j}=\frac{\left(v_{i,j}-v_{i,j-1}\right)}{\delta t}$&lt;/li&gt;&#xA;&lt;li&gt;${v_x}_{i,j}=\frac{\left(v_{i,j}-v_{i-1,j}\right)}{\delta x}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;putting these in the PDEs:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;$v_{i,j}\frac{\left(p_{i,j}-p_{i-1,j}\right)}{\delta x}+&#xA;p_{i,j}\frac{\left(v_{i,j}-v_{i-1,j}\right)}{\delta x}+&#xA;\frac{\left(p_{i,j}-p_{i,j-1}\right)}{\delta t}=0$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$v_{i,j}^2\frac{\left(p_{i,j}-p_{i-1,j}\right)}{\delta x}+&#xA;v_{i,j}\frac{\left(p_{i,j}-p_{i,j-1}\right)}{\delta t}+&#xA;2v_{i,j}p_{i,j}\frac{\left(v_{i,j}-v_{i-1,j}\right)}{\delta x}+&#xA;p_{i,j}\frac{\left(v_{i,j}-v_{i,j-1}\right)}{\delta t}+&#xA;c\frac{\left(p_{i,j}-p_{i-1,j}\right)}{\delta x}=0$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have applied these in a very primitive python code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;import math&#xA;from scipy.optimize import fsolve&#xA;&#xA;l=1.0 #Length of the tube in meters&#xA;t=10.0 # length of the simulation in seconds&#xA;M=10 # spatial discretisation resolution&#xA;N=4 # temporal discretisation resolution&#xA;pr=200 #reservoir pressure pascal&#xA;pa=100 #ambient pressure pascal&#xA;Ta=300 #ambient temperature in kelvin&#xA;R=286.9 #Gas Constant for air&#xA;&#xA;#step signal&#xA;ts=1 #in seconds,  after ts p start will be pr&#xA;&#xA;dx=l/(M-1) #meters&#xA;dt=t/(N-1) #meters&#xA;p=numpy.ones((M,N))*pa&#xA;v=numpy.zeros((M,N))&#xA;p[0,(math.ceil(ts/dt)):N]=numpy.ones(N-(math.ceil(ts/dt)))*pr #assign the start boundry condition as step signal&#xA;&#xA;def eqns(ivars, *data):&#xA;    pij, vij=ivars&#xA;    pij_1,pi_1j,vij_1,vi_1j=data&#xA;    return (vij*(pij-pi_1j)/dx+pij*(vij-vi_1j)/dx+(pij-pij_1)/dt, 2*pij*vij*(vij-vi_1j)/dx-(vij**2)*(pij-pi_1j)/dx+vij*(pij-pij_1)/dt+pij*(vij-vij_1)/dt+R*Ta*(pij-pi_1j)/dx)&#xA;&#xA;for i in range(1,M):&#xA;    for j in range(1,N):&#xA;        data=(p[i][j-1],p[i-1][j],v[i][j-1],v[i-1][j])&#xA;        p[i][j], v[i][j]=fsolve(eqns,(p[i][j-1],v[i][j-1]), args=data)&#xA;&#xA;print(v)&#xA;print(p)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have three issues:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the code above runs without any errors, but the result is nothing but the initial condition!&lt;/li&gt;&#xA;&lt;li&gt;I'm obviously not able to incorporate the boundary condition 3 at the end&lt;/li&gt;&#xA;&lt;li&gt;I'm not able to calculate $v_{0,j}$s&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;and my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;are my equations correct?&lt;/li&gt;&#xA;&lt;li&gt;are my boundary conditions realistic and enough?&lt;/li&gt;&#xA;&lt;li&gt;am I implementing backwards finite difference method correctly?&lt;/li&gt;&#xA;&lt;li&gt;how to change my python code to solve the three issues above?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22193" LastEditorUserId="22193" LastEditDate="2017-08-07T22:42:20.940" LastActivityDate="2017-08-18T18:21:33.487" Title="How to implement finite difference method for one dimensional Navier-Stokes PDEs" Tags="|pde|finite-difference|python|navier-stokes|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="27572" PostTypeId="1" AcceptedAnswerId="27573" CreationDate="2017-08-07T07:56:16.380" Score="1" ViewCount="324" Body="&lt;p&gt;I'm trying to solve the heat equation in FiPy and right now it works, but I have one problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The material I have to simulate has different diffusion coefficients in the x and y direction (due to its fibre layerd structure). I know, that you can make your diffusion coefficient dependent to a variable in FiPy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My grid is rectengular, but in the future is has to be able to be unstructured. This would mean, the diffusion coefficient is different for every face. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anybody a clue on how to implement this type of dependency?&lt;/p&gt;&#xA;" OwnerUserId="25041" LastActivityDate="2017-08-07T09:18:22.683" Title="FiPy: Make diffusion coefficient dependent to orientation" Tags="|finite-element|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27573" PostTypeId="2" ParentId="27572" CreationDate="2017-08-07T09:18:22.683" Score="0" Body="&lt;p&gt;The term you're looking for is &lt;em&gt;anisotropic diffusion&lt;/em&gt;, which handles differing magnitudes of diffusion based on spatial orientation - usually these are defined using tensors (2x2 for 2D problems and 3x3 for 3D problems). I searched for '&lt;em&gt;diffusion tensor FiPy&lt;/em&gt;' and found a link to their FAQ here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ctcms.nist.gov/fipy/documentation/FAQ.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ctcms.nist.gov/fipy/documentation/FAQ.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="19791" LastActivityDate="2017-08-07T09:18:22.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27574" PostTypeId="1" AcceptedAnswerId="27575" CreationDate="2017-08-07T17:52:29.267" Score="1" ViewCount="172" Body="&lt;p&gt;I have a 9 systems of nonlinear ODEs to solve. I want to determine the endemic equilibrium points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I go about it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried manual calculation but it becomes cumbersome. Can software be used for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="25085" LastEditorUserId="9667" LastEditDate="2017-08-07T20:17:32.403" LastActivityDate="2019-07-16T13:32:02.820" Title="Solving systems of nonlinear ODEs in epidemiology" Tags="|ode|nonlinear-equations|symbolic-computation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27575" PostTypeId="2" ParentId="27574" CreationDate="2017-08-07T20:59:45.937" Score="3" Body="&lt;p&gt;If you want to find the equilibrium points you need to write your system as a first-order one&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d\mathbf{x}}{dt} = \mathbf{F}(\mathbf{x})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and solve the non-linear system of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{F}(\mathbf{x})=\mathbf{0}\ ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that might not be solvable analytically. After that, you need to find the eigenvalues of the Jacobian matrix around each equilibrium point &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}^*$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{J}(\mathbf{x}^*)$&lt;/span&gt;. You can use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_algebra_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer Algebra System (CAS)&lt;/a&gt; for the job. SymPy, Maxima, Mathematica and Maple are some examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of this, using SymPy, for the Lotka-Volterra model is the following&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from sympy import symbols, Matrix, solve, diff&#xA;x, y = symbols(&quot;x y&quot;)&#xA;alpha, beta, gamma, delta = symbols(&quot;alpha beta gamma delta&quot;)&#xA;dXdt = Matrix([alpha*x - beta*x*y, delta*x*y -gamma*y])&#xA;eqpts = solve(dXdt, [x, y])&#xA;J = Matrix(2, 2, lambda i, j: diff(dXdt[i], [x, y][j]))&#xA;vals1 = J.subs({x: eqpts[0][0], y: eqpts[0][1]}).eigenvals()&#xA;vals2 = J.subs({x: eqpts[1][0], y: eqpts[1][1]}).eigenvals()&#xA;print(vals1)&#xA;print(vals2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{-gamma: 1, alpha: 1}&#xA;{sqrt(-alpha*gamma): 1, -sqrt(-alpha*gamma): 1}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here you should take into account two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The system of non-linear equations might not have an analytical solution, and you might need to solve it numerically (with Newton method, for example).&lt;/li&gt;&#xA;&lt;li&gt;Even if you have the analytical solution(s) for your system, the eigensystem might not be solvable analytically.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2019-07-16T13:32:02.820" LastActivityDate="2019-07-16T13:32:02.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27576" PostTypeId="1" CreationDate="2017-08-08T05:16:13.710" Score="1" ViewCount="69" Body="&lt;p&gt;I'm measuring the integral of a quantity which, mathematically, requires the computation of a dilogarithm function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\operatorname{Li}_2(be^{ax})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $b$ and $a$ (are real and) can be positive or negative. Dilogarithm has very nice properties that enable us to compute its value efficiently, For $|z|&amp;lt;1$, the following series converges absolutely:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\operatorname{Li}_2(z)=\sum_{n=1}^{\infty}\frac{z^n}{n^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering is there any explicit way to compute or numerically approximate dilogarithm -- especially in COMSOL?&lt;/p&gt;&#xA;" OwnerUserId="4858" LastActivityDate="2017-08-08T05:16:13.710" Title="Computing dilogarithm" Tags="|finite-element|integration|approximation|comsol|integral-equations|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27577" PostTypeId="2" ParentId="27547" CreationDate="2017-08-08T06:06:45.257" Score="1" Body="&lt;p&gt;This is basically a computation of stencil coefficients for a finite difference method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expand each cell value in terms of the center using Taylor series. Note that $\Delta_{x} = \Delta_{y} = \Delta$ in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_{i+1,j+1} = u_{i,j} + \Delta \left( u_{x} + u_{y} \right) + \frac{\Delta^{2}}{2} \left( u_{xx} + 2 u_{xy} + u_{yy} \right) + \cdots $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with all symbolic derivatives shown being evaluated at the center.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 8 variables(each cell) and 5 unknowns(each symbolic derivative), which form an underdetermined system. The aim is to approximate a Laplacian smoothing, thus $u_{i,j}$, $u_{xx}$ and $u_{yy}$ need to have coefficients of 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Gaussian kernel is sometimes used as it satisfies the Laplace equation. The elements in the diagonal are further away from the center than the edge-centers. In fact, one can compute the Gaussian used here based on the weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F = \mathcal{N}(0,\sigma^{2})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$a = F(\sqrt{2})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$b = F(1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On substituting the values provided, $\sigma \approx 0.75$&lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2017-08-08T06:06:45.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27579" PostTypeId="2" ParentId="26435" CreationDate="2017-08-08T11:52:14.600" Score="1" Body="&lt;p&gt;I presume that your fluid phase is liquid, for example, water. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is a case, you have liquid and gas, and porous material, thus you have capillary pressure in the liquid. At the microscopic scale, this is described by Young-Laplace equation. At the macro scale, your flow is the unsaturated flow problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The capillary pressure depends on water content density. It is a semi-empirical relation between liquid content and capillary pressure, called resorption curve. Such problems have been studied in soil mechanics and civil engineering in general, including modelling of concrete. The hydraulic conductivity is strongly nonlinear, which drops quickly to very small values (to zero) when with dropping water content. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find several models developed for soils, most popular is Van Genuchten model, with in applied with Richards equation give strongly nonlinear parabolic equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know exact conditions under which you solve your problem,  but it could be premature, that phenomenon which you try to solve is driven by the convection-diffusion equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested, I have free/open code for unsaturated flow using mix formulation (using RT elements) which can be, relatively easily, generalised to the case with the flow of gas phase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About your question, what boundary conditions you like to apply depending on what you know about fluid phase, if you now pressure on both sides, then apply it. If you measure influx, apply it.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastActivityDate="2017-08-08T11:52:14.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27580" PostTypeId="1" AcceptedAnswerId="27581" CreationDate="2017-08-08T13:17:11.210" Score="1" ViewCount="178" Body="&lt;p&gt;I have been looking into simple ways of calculating numerically the total fluid stress tensor $ \pi_{ij}$ if the flow field is known in every $ x,y,z $ point. Applying the fluid stress equation (see below) isn't really straightforward since it requires an analytical velocity field. I understand this question might be &quot;naive&quot; but could you refer me to proper literature?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \pi_{ij}=-p\textbf{I}+\tau_{ij} =-p\textbf{I}+\mu(\nabla{\mathbf{u}}+\nabla{\mathbf{u}^T}) $$ &lt;/p&gt;&#xA;" OwnerUserId="25093" LastEditorUserId="549" LastEditDate="2017-08-08T16:39:25.837" LastActivityDate="2017-10-09T07:05:42.717" Title="Numerical Calculation of the Fluid Stress Tensor" Tags="|fluid-dynamics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27581" PostTypeId="2" ParentId="27580" CreationDate="2017-08-08T13:29:13.300" Score="0" Body="&lt;p&gt;Where come the $p$ distribution and $\vec{v}$ field from? If you have any mesh, you can always calculate the polynomial that interpolate the variables and you can differentiate it analytically, for an arbitrary set of points. To illustrate this, $e.g$ the velocity can be approximated as &#xA;$$\vec{u} = \sum{\vec{u}_i\,\phi_i(x,y,z)} $$ &#xA;and therefore its gradient is&#xA; $$\vec{\textrm{grad}}\,\vec{v} = \vec{\textrm{grad}}\sum{\vec{u}_i\,\phi_i(x,y,z)}=\sum{\vec{u}_i\otimes\vec{\textrm{grad}}\phi_i(x,y,z)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can create the polynomial basis such that $\phi_i(x,y,z)\in C^1(\Omega)$, $i.e$ must be continuous and derivable once (at least). Hermite polynomials are perfect for this task.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2017-08-08T13:29:13.300" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27584" PostTypeId="1" AcceptedAnswerId="29014" CreationDate="2017-08-08T19:36:40.287" Score="1" ViewCount="288" Body="&lt;p&gt;I am computing the inverse of a complex matrix. I execute ZGETRF but U(2,2) = 0. When I compute ZGETRI, the inverse is determined. Can I trust this inverse?&lt;/p&gt;&#xA;" OwnerUserId="25099" LastActivityDate="2018-03-12T03:47:09.310" Title="LAPACK: ZGETRF with INFO greater than zero but ZGETRI does not fail." Tags="|fortran|lapack|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27586" PostTypeId="1" CreationDate="2017-08-09T10:32:43.407" Score="2" ViewCount="1631" Body="&lt;p&gt;I have the same problem as in &lt;a href=&quot;https://scicomp.stackexchange.com/a/10218/25101&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But can someone elaborate on the answer? The poster says that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Solving this system of 4 ODE's with rk4 will solve for all your state variables simultaneously. position and velocity do not need to be handled seperately (and shouldn't be).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How is this actually implemented? There is still no expression that I can see for $v$, so I do not see how we can obtain the RK coefficients, and I cannot see how to handle position and velocity ‘not seperately’.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a system of 2 coupled oscillators we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$m_1\ddot{x}_1=-k_1x_1+k_2(-x_1+x_2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$m_2\ddot{x}_2=-k_3x_2+k_2(x_1-x_2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x_1$ and $x_2$ are the displacements of masses 1 and 2 from their equilibrium positions. If we then substitute $v=\dot{x}$, we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot{v}_1m_1=-k_1x_1+k_2(-x_1+x_2)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and similar for $\dot{v}_2$. The answer to the question I linked suggested we combine the system to give (they have used a slightly different functional form but the underlying mechanics is the same):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left(\begin{array}{c}&#xA;	\dot{x}_1 \\&#xA;	\dot{x}_2 \\&#xA;	\dot{v}_1 \\&#xA;	\dot{v}_2&#xA;\end{array}\right)&#xA;= \left(\begin{alignat}{1}&#xA;	&amp;amp; v_1 \\&#xA;	&amp;amp; v_2 \\&#xA;	-\omega_1^2(x_1-R_1) &amp;amp;+ \omega_2^2(x_2-x_1-w-R_2)\\&#xA;	&amp;amp;-\omega_2^2(x_2-x_1-w-R_2)&#xA;   \end{alignat}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I'm unsure how to actually proceed from here. RK4 algorithm needs to evaluate the RK coefficients at different timesteps, and my issue is $\dot{v}=f(x_1,x_2)$ instead of $\dot{v}=f(v)$ in normal (uncoupled) RK4. I can't find $v$ at a new timestep as I need $x$ to do that, and I can't find $x$ without $v$. Is it a matter of writing some operator matrix equation and solving that?&lt;/p&gt;&#xA;" OwnerUserId="25101" LastEditorUserId="25101" LastEditDate="2017-08-09T12:48:44.493" LastActivityDate="2017-08-09T13:33:58.150" Title="Runge Kutta (RK4) to solve coupled harmonic oscillators" Tags="|numerical-analysis|python|" AnswerCount="2" CommentCount="0" ClosedDate="2017-08-24T15:36:59.390" ContentLicense="CC BY-SA 3.0" />
  <row Id="27587" PostTypeId="2" ParentId="27586" CreationDate="2017-08-09T13:16:16.960" Score="1" Body="&lt;p&gt;The Runge-Kutta approximates the solution of the initial value problem&#xA;$$&#xA;\dot{y}(t) = F(t, y(t)), \quad y(0) = y_0.&#xA;$$&#xA;The method expects $y_0$ and the function $F$ as input. If we set&#xA;$$&#xA;y(t) &#xA;= \begin{pmatrix} y_1(t) \\ y_2(t) \\ y_3(t) \\ y_4(t) \end{pmatrix}&#xA;= \begin{pmatrix} x_1(t) \\ x_2(t) \\ v_1(t) \\ v_2(t) \end{pmatrix}&#xA;$$&#xA;and define $F: [0, \infty) \times \mathbb{R}^4 \to \mathbb{R}^4$ by&#xA;$$&#xA;F(t, y) = &#xA;\begin{pmatrix}&#xA;y_3 \\&#xA;y_4 \\&#xA;-\omega_1^2(y_1-R_1) + \omega_2^2(y_2-y_1-w-R_2) \\&#xA;-\omega_2^2(y_2-y_1-w-R_2)&#xA;\end{pmatrix}&#xA;\,,&#xA;$$&#xA;then $\dot{y} = F(t, y(t))$ is your ODE. To use the Runge-Kutta method you just have to implement the function $F$ and hand it, together with the initial conditions, to the method. (Note that the right hand side of the function $F$ does not explicitly depend on $t$.)&lt;/p&gt;&#xA;" OwnerUserId="24953" LastEditorUserId="24953" LastEditDate="2017-08-09T13:21:57.007" LastActivityDate="2017-08-09T13:21:57.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27588" PostTypeId="2" ParentId="27586" CreationDate="2017-08-09T13:33:58.150" Score="0" Body="&lt;p&gt;Differential equations as solved by Runge–Kutta methods generally have the form¹:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot{y} = \frac{\mathrm{d}y(t)}{\mathrm{d}t} = f(y(t))&#xA;\quad \text{with} \quad y:ℝ→ℝ^n,~ f:ℝ^n→ℝ^n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For solving these equations, it doesn’t matter:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what the physical interpretation of the components is, e.g., whether they represent velocity or position,&lt;/li&gt;&#xA;&lt;li&gt;whether any components form a subsystem, here, form one of the original oscillators,&lt;/li&gt;&#xA;&lt;li&gt;which components of the input of $f$ influence which components of the output, i.e., how the components interact with each other.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For a single oscillator ($\dot{x} = v$, $\dot{v} = -kx$), you have $n=2$ and have to substitute $x$ and $v$ with components of $y$ in the above, e.g., $y_1=x$ and $y_2=v$.&#xA;With this, the function $f$ for your Runge–Kutta method is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(y) = \pmatrix{y_2\\-ky_1}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For two oscillators, you can do the same, just that you now have $n=4$.&#xA;If you do substitute $y_1=x_1$, $y_2=x_2$, $y_3=v_1$, and $y_4=v_2$ in the equations you give, you get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(y) = &#xA;\left(\begin{alignat}{1}&#xA;	&amp;amp; y_3 \\&#xA;	&amp;amp; y_4 \\&#xA;	 -\omega_1^2(y_1-R_1) &amp;amp;+ \omega_2^2(y_2-y_1-w-R_2)\\&#xA;						  &amp;amp;- \omega_2^2(y_2-y_1-w-R_2)&#xA;\end{alignat}\right).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;&#xA;¹ To keep things simple, I here ignore the possibility that $f$ depends on $t$ explicitly, as it is not relevant to this problem.&#xA;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2017-08-09T13:33:58.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27590" PostTypeId="1" CreationDate="2017-08-09T15:00:49.060" Score="4" ViewCount="1222" Body="&lt;p&gt;I have written Poisson solvers using two different methods:&lt;/p&gt;&#xA;&lt;p&gt;A classic Jacobi scheme and one using the multigrid solver &lt;a href=&quot;https://computation.llnl.gov/projects/hypre-scalable-linear-solvers-multigrid-methods/software&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hypre&lt;/a&gt;. I made up a couple of test cases ensuring the validity of those solvers.&lt;/p&gt;&#xA;&lt;p&gt;For both cases the domain is defined as &lt;span class=&quot;math-container&quot;&gt;$(x,y) \in [-1,1]^2$&lt;/span&gt; with periodic boundary conditions. Also, the grids first and last point are the same:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(0,y) = p(N_x-1,y)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(x,0) = p(x,N_y-1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Test case 1&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$f_{rhs} = -8 \pi^2 \sin(2\pi x) \sin(2\pi y)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{exact} = \sin(2\pi x) \sin(2\pi y)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For both solvers, the solution is 2&lt;span class=&quot;math-container&quot;&gt;$^{\mathrm{nd}}$&lt;/span&gt; order accurate is space. No problems here so far.&lt;/p&gt;&#xA;&lt;h2&gt;Test Case 2&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{rhs} = e^{-10 (x^2 + y^2)}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{exact}:$&lt;/span&gt; No analytical solution, and therefore the numerical solution is differentiated using a high-order compact scheme and compared to &lt;span class=&quot;math-container&quot;&gt;$p_{rhs}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note that in this case, &lt;span class=&quot;math-container&quot;&gt;$\int_V p_{rhs}dV  \neq 0$&lt;/span&gt; and therefore the problem is ill defined. Therefore, the &lt;span class=&quot;math-container&quot;&gt;$rhs$&lt;/span&gt; must be modified:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{rhs} = e^{-10 (x^2 + y^2)} - \dfrac{\int_D e^{-10 (x^2 + y^2)} dx dy}{V}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is the domain volume. The integral is computed using the Trapezoidal rule.&lt;/p&gt;&#xA;&lt;p&gt;This is where things get tricky. No matter how fine my grid is, &lt;span class=&quot;math-container&quot;&gt;$\left(p_{num}\right)_{xx} + \left(p_{num}\right)_{yy}$&lt;/span&gt; never converge to &lt;span class=&quot;math-container&quot;&gt;$p_{rhs}$&lt;/span&gt;. When the grid is fine enough, the solvers converge, but the solution is off by ~20% while the overall profiles are relatively correct. When the grid is coarse, the Hypre solver simply diverges.&lt;/p&gt;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&lt;p&gt;Have I missed anything? Is my approach inconsistent/wrong?&lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2022-12-16T23:02:00.280" Title="Convergence problem for Poisson equation with periodic BC" Tags="|convergence|poisson|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27592" PostTypeId="1" CreationDate="2017-08-09T18:37:28.027" Score="1" ViewCount="92" Body="&lt;p&gt;I am trying to estimate the FD stencil for boundary as mentioned in this &lt;a href=&quot;https://www.researchgate.net/profile/Sanjiva_Lele/publication/222488464_Lele_SK_Compact_Finite_Difference_Schemes_with_Spectral-like_Resolution_Journal_of_Computational_Physics_1031_16-42/links/559ea0ee08aea946c06a3842.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; (section 4.1)  using MATLAB. The stencil order (6th) is higher than the one mentioned in paper (4th).&#xA;$$ f_1' +\alpha f_2' = \frac{1}{h} (c_1f_1 + c_2f_2 + c_3f_3 +c_4f_4+ c_5f_5 + c_6 f_6 + c_7 f_7 )$$&#xA;Each term of both sides is expanded using Taylor series expansion and then derivatives are matched to estimate the coefficients [$c1,c2,c3,c4,c5,c6,c7$ and $\alpha$].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solved the above system as following:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LlQ9i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LlQ9i.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Above system of equation can be written as M*x=y   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) The alpha was taken to the left to form a part of unknown vector x&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) the factorial on both side were simplified and adjusted to right side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote following code for solving the above system of equation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LM = zeros(8);                &#xA;LM(1,1:7)=1;                LM(1,8)=0;  &#xA;LM(2,2:7)=1:6;              LM(2,8)=-1;&#xA;LM(3,2:7)=(1:6).^2;         LM(3,8)=-2;&#xA;LM(4,2:7)=(1:6).^3;         LM(4,8)=-3;&#xA;LM(5,2:7)=(1:6).^4;         LM(5,8)=-4;&#xA;LM(6,2:7)=(1:6).^5;         LM(6,8)=-5;&#xA;LM(7,2:7)=(1:6).^6;         LM(7,8)=-6;&#xA;LM(8,2:7)=(1:6).^7;         LM(8,8)=-7;&#xA;y=[0;1;0;0;0;0;0;0];&#xA;x=LM\y;&#xA;rats(x) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives completely different answer than reported one. Both are given below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution i.e. $x =[c1,c2,c3,c4,c5,c6,c7, \alpha$] by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB estimated solution:     [-69/20, -17/10, 15/2, -10/3, 5/4, -3/10, 1/30, 6]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution reported in literature: [-197/60, -5/12, 5, -5/3, 5/12, -1/20, 0, 5]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20015" LastActivityDate="2017-08-09T18:37:28.027" Title="Discrepancy in estimating boundary stencil for finite difference method" Tags="|matlab|finite-difference|boundary-conditions|numerics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27593" PostTypeId="1" CreationDate="2017-08-09T18:38:23.237" Score="5" ViewCount="1736" Body="&lt;p&gt;In order to integrate a two dimensional function of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{1}^\infty \int_{-\sqrt{x^2-1}}^{\sqrt{x^2-1}} e^{-x} \rm{d}y \rm{d}x,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been attempting to use the following code (written in C++) taken mostly from the Numerical Recipes book which calls a gaussian quadrature routine for the integration: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; static float xsav;&#xA; static float (*nrfunc)(float,float);&#xA;&#xA; float quad2d(float (*func)(float, float), float x1, float x2)&#xA; {&#xA;     float qgaus(float (*func)(float), float a, float b); &#xA;     float f1(float x);&#xA;&#xA;     nrfunc=func;&#xA;     return qgaus(f1,x1,x2); &#xA; }&#xA;&#xA; float f1(float x) &#xA; {&#xA;     float qgaus(float (*func)(float), float a, float b); &#xA;     float f2(float y);&#xA;     float yy1(float),yy2(float);&#xA;&#xA;     xsav=x;&#xA;     return qgaus(f2,yy1(x),yy2(x)); &#xA; }&#xA;&#xA; float f2(float y)  &#xA; {&#xA;     return (*nrfunc)(xsav,y);&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code works fine for two dimensional integrals with finite limits, but fails as the outer limit is taken to infinity. To account for this, I have attempted to use a change of variables: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#define FUNC(x) ((*funk)(-log(x))/(x))&#xA;&#xA;float qgaus(float (*funk)(float), float aa, float bb)&#xA;{&#xA;    int j;&#xA;    float xr,xm,dx,s,a,b;&#xA;&#xA;    b=exp(-aa); &#xA;    a=0.0;&#xA;&#xA;    static float x[]={0.0,0.1488743389,0.4333953941,&#xA;        0.6794095682,0.8650633666,0.9739065285};&#xA;    static float w[]={0.0,0.2955242247,0.2692667193, &#xA;        0.2190863625,0.1494513491,0.0666713443};&#xA;&#xA;    xm=0.5*(b+a); &#xA;    xr=0.5*(b-a);&#xA;    s=0;&#xA;    for (j=1;j&amp;lt;=5;j++)&#xA;    {&#xA;        dx=xr*x[j];&#xA;        s += w[j]*(FUNC(xm+dx)+FUNC(xm-dx)); &#xA;    }&#xA;    return s *= xr; &#xA;}&#xA;&#xA;&#xA;float f(float x, float y)&#xA;{&#xA;    float a = exp(-x);&#xA;    return a;&#xA;}&#xA;&#xA;float yy1(float x)&#xA;{&#xA;    float y = -sqrt(x*x-1);&#xA;    return y;   &#xA;}&#xA;&#xA;float yy2(float x)&#xA;{&#xA;    float y = sqrt(x*x-1);&#xA;    return y;&#xA;}&#xA;&#xA;&#xA;&#xA;static float xsav;&#xA;static float (*nrfunc)(float, float);&#xA;&#xA;float quad2d(float (*func)(float, float), float x1, float x2)&#xA;{&#xA;    float qgaus(float (*func)(float), float aa, float bb);&#xA;    float f1(float x);&#xA;&#xA;    nrfunc=func;&#xA;    float t = qgaus(f1,x1,x2);&#xA;    return t;&#xA;}&#xA;&#xA;float f1(float x)&#xA;{&#xA;    float qgaus(float (*func)(float), float aa, float bb);&#xA;    float f2(float y);&#xA;    float yy1(float);&#xA;    float yy2(float);&#xA;&#xA;    xsav=x;&#xA;    float r = qgaus(f2,yy1(x),yy2(x));&#xA;    return r;&#xA;}&#xA;&#xA;float f2(float y) &#xA;{&#xA;    float k = (*nrfunc)(xsav,y);&#xA;    return k;&#xA;}&#xA;&#xA;int main ()&#xA;{&#xA;    float z;&#xA;    z = quad2d(f, 1.0, 20.0);&#xA;    cout &amp;lt;&amp;lt; z &amp;lt;&amp;lt; endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this still doesn't give the right answer. It should be $2 \times \rm{BesselK}[1,1] \approx 1.20381$, but instead gives 2.15501. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how I could modify this code to account for the infinite limit would be greatly appreciated! &lt;/p&gt;&#xA;" OwnerUserId="25108" LastEditorUserId="25108" LastEditDate="2017-08-10T16:24:56.870" LastActivityDate="2017-09-25T09:16:11.547" Title="2D numerical integration with infinite limit (C++)" Tags="|c++|quadrature|integration|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27595" PostTypeId="1" CreationDate="2017-08-10T07:55:48.437" Score="1" ViewCount="77" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The water height $h$ and 2d velocity field $(u,w)$ are &lt;em&gt;&quot;extrapolated&#xA;  to non-fluid-cells, i.e., setting $h$ equal to the value in the&#xA;  nearest fluid cell.&quot;&lt;/em&gt; [Bridson]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm using finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the &lt;a href=&quot;http://www.cs.ubc.ca/~rbridson/fluidsimulation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;course notes&lt;/a&gt;, this extrapolation seems involved: tracking the &lt;em&gt;zero  level set&lt;/em&gt; surface $\phi$ (using &lt;em&gt;signed distance&lt;/em&gt; from the surface), then taking its gradient $\nabla \phi$.&#xA;It seems to be done for &lt;em&gt;all&lt;/em&gt; non-fluid cells, not just those adjacent to fluid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand why this done (BTW for the 2d SWE case, if the non-fluid cell is lower e.g. the lip of a dam wall, then extrapolating the water height will make large quantities of water appear from nowhere (because the height is so far above the ground).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simple explanation, and a simpler way to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; Two different quantities are being extrapolated, &lt;em&gt;velocity&lt;/em&gt; and &lt;em&gt;height&lt;/em&gt;. The &lt;em&gt;velocity&lt;/em&gt; extrapolation seems to be needed for semi-Lagranian advection. This is where you advect a quantity $q$ by projecting velocities &lt;em&gt;backwards&lt;/em&gt; over a timestep, to find what value of $q$ was at that location (interpolating between the grid locations where $q$ is stored). The issue is that this back-projection &lt;em&gt;can&lt;/em&gt; land in a region where there is no $q$... so, you extrapolate the value from the nearest cell that does have $q$ (BTW which is along the normal to the surface).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; The other extrapolation, of height $h$, seems to only be used for pressure acceleration. (Its effect seems to be a Neumann BC, zero derivative normal to surface, by making the other side the same as this one...?)  Height $h$ is derived from depth $d$ and bottom $b$, $h=b+d$, and $h$ isn't stored, and doesn't seem to be used again. Although, logically, when $d$ is advected semi-Lagrangianly, it should need extrapolation, the same for any other $q$...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any good references or articles with a nice precis/lit review, or survey papers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Bridson] I'm implementing the SWE from chapter 12 of Bridson's book &lt;a href=&quot;https://books.google.com.au/books?id=Z9eYCgAAQBAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Fluid Simulation for Computer Animation&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-08-11T12:24:08.540" LastActivityDate="2017-08-11T12:24:08.540" Title="Extrapolating to non-fluid cells (for Shallow Water Equations), for a shore/beach?" Tags="|finite-difference|fluid-dynamics|boundary-conditions|advection|extrapolation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27596" PostTypeId="1" CreationDate="2017-08-10T09:39:10.033" Score="1" ViewCount="451" Body="&lt;p&gt;I am trying to solve Stokes problem using Finite element method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how to impose that total pressure over the surface is zero to remove the constant pressure mode?&lt;/p&gt;&#xA;" OwnerUserId="25119" LastEditorUserId="9667" LastEditDate="2017-08-11T00:30:37.280" LastActivityDate="2021-04-22T15:40:38.627" Title="Imposing total pressure over surface in FEM" Tags="|finite-element|navier-stokes|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27597" PostTypeId="1" CreationDate="2017-08-10T10:11:29.217" Score="1" ViewCount="263" Body="&lt;p&gt;I would like to perform a decane/water(tip4p) interface simulation using GROMACS to analyse the fluctuation of the interface over a long period of time (50ps - 1ns). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though I have a rough idea on how to perform it, I still am unsure about how to create a initial configuration which looks like the figure give below (A.R. Buuren, Ph.D. Thesis, University of Groningen, 1995). &lt;a href=&quot;https://i.stack.imgur.com/fkZUz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fkZUz.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also once I start my production run I would like to know, how to obtain the height fluctuations of the interface as a function of time. &lt;/p&gt;&#xA;" OwnerUserId="25121" LastActivityDate="2017-08-10T11:50:49.227" Title="Oil/Water interface simulation using GROMACS" Tags="|computational-physics|simulation|molecular-dynamics|computational-chemistry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27598" PostTypeId="2" ParentId="27596" CreationDate="2017-08-10T10:45:10.023" Score="2" Body="&lt;p&gt;You must add an additional equation to your system. It is:&#xA;$$ \int_{\Gamma_p}{p\,d\sigma}=0\implies \sum_{p_i\in \Gamma_p}p_i\int_{\Gamma_{pi}}{\phi(t)\,dt}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\phi(t)$ is the basis function of $p$ along the parameterised boundary described with the parameter $t$ in the element $\Gamma_{pi}$. Remember that $\Gamma_p=\cup{ \Gamma_{pi}}$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2017-08-10T10:45:10.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27599" PostTypeId="2" ParentId="27560" CreationDate="2017-08-10T10:56:52.703" Score="1" Body="&lt;p&gt;A detailed derivation for roe averages is provided in one of the earlier articles of Roe and Pike. Which I am skipping as the link to article is provided below.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a more vague note, roe average of density is not assumed but rather derived from the Jacobian matrix. Literature you have read are probably second or third account which states it is natural to assume such a geometric mean. By placing a geometric mean it is easy to show the derivation of other means, but the right way to derive the averages is rather complicated and is provided in this article. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take away: You will violate flux conservation and rule of hyperbolicity by playing around with averages that are not physical. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Roe, P.L.; Pike, J.&lt;/em&gt;, Efficient construction and utilisation of approximate Riemann solutions, Computing methods in applied sciences and engineering VI, Proc. 6th Int. Symp., Versailles 1983, 499-518 (1984). &lt;a href=&quot;http://www.jackpike.co.uk/38.Efficient%20Construction%20and%20Utilisation%20of%20Approximate%20Riemann%20Solutions.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL0558.76001&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25121" LastActivityDate="2017-08-10T10:56:52.703" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27600" PostTypeId="2" ParentId="27597" CreationDate="2017-08-10T11:50:49.227" Score="2" Body="&lt;p&gt;A simple solution to begin with would be to create two systems: one with decane in a box of width 1.295nm and another one with water in a box of width 2nm (depth and height should be as in the picture you included with your question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have the two systems, run them independently for a sufficiently long time so that they equilibrate. Then take two snapshots (sets of positions and velocities) of the decane box that are uncorrelated and one snapshot of the water box. Translate the positions of the water and one of the decane boxes so that they form the configuration you want. You can do this, for example, with &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/current/ug/node122.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VMD&lt;/a&gt; or with &lt;a href=&quot;http://mdtraj.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDTraj&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, create the target system having all the molecules but with the conformations (and velocities) taken from the translated boxes that you obtained in the previous step. Equilibrate again, and run.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-08-10T11:50:49.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27601" PostTypeId="1" CreationDate="2017-08-10T15:03:35.040" Score="0" ViewCount="222" Body="&lt;p&gt;Assume I have two functions $f$ and $g$, with derivatives of $g$ at point $x$ and derivatives of $f$ at point $g(x)$ available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the fastest way of computing derivatives of $f \circ g (x)$ ? &lt;/p&gt;&#xA;" OwnerUserId="16732" LastEditorUserId="25082" LastEditDate="2017-08-11T12:35:24.157" LastActivityDate="2017-08-14T20:28:35.157" Title="Fast chain rule algorithm" Tags="|algorithms|" AnswerCount="1" CommentCount="3" ClosedDate="2017-08-15T22:01:56.520" ContentLicense="CC BY-SA 3.0" />
  <row Id="27602" PostTypeId="2" ParentId="27596" CreationDate="2017-08-11T01:36:59.217" Score="1" Body="&lt;p&gt;For Finite Element formulation for fluid flow, it is sufficient to impose zero-pressure on one of the nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, this node is chosen on the outflow boundary. If the outflow boundary does not exist, then pick a node far away from the region(s) of steep pressure gradients.&lt;/p&gt;&#xA;" OwnerUserId="24915" LastActivityDate="2017-08-11T01:36:59.217" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27603" PostTypeId="2" ParentId="27590" CreationDate="2017-08-11T02:01:25.123" Score="0" Body="&lt;p&gt;In your Test Case 1, the solution is periodic. Hence, you had no problems when you imposed periodic boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Test Case 2, the solution may not be periodic (I suspect that it is not). By enforcing the periodic conditions, you get some solution which does not actually corresponding to the source term. That is why your solution is off by 20%. This should be due to incompatibility between the actual solution and the obtained solution at the boundaries. If you check carefully, you may find that the maximum error occurs at the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can verify your code with another benchmark example from FEniCS.&#xA;&lt;a href=&quot;https://fenicsproject.org/olddocs/dolfin/1.3.0/python/demo/documented/periodic/python/documentation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fenicsproject.org/olddocs/dolfin/1.3.0/python/demo/documented/periodic/python/documentation.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24915" LastActivityDate="2017-08-11T02:01:25.123" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27605" PostTypeId="1" CreationDate="2017-08-11T04:37:38.280" Score="1" ViewCount="136" Body="&lt;p&gt;I have a PDE (coming from a Bellman equation with $z$ under brownian motion).  Let $z \in [0,\infty)$ and $t \in [0,T]$.   To sketch the equation:&#xA;$$&#xA;(r - g(t)) v(t,z) = \pi(t,z) + (\gamma - g(t))v_z(t,z) + \frac{\sigma^2}{2}v_{zz}(t,z) + v_t(t,z)&#xA;$$&#xA;with a terminal condition $v(T,z)$ and ugly nonlinear boundary conditions involving integrals for $v(t,0)$ and $v_z(t,0)$.  There is a very good chance that the PDE will be stiff and $v(t,z)$ fairly close to $v(T,z)$ for all $T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am considering two alternatives:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use a 2-dimensional Chebyshev basis&#xA;$$v(t,z)\approx \sum_{i=0}^{N_z}\sum_{j=0}^{N_t}a_{ij}T_i(z)T_j(t)$$&#xA;for some $N_z$ and $N_t$ and $T(\cdot)$ are the chebyshev polynomials adapted to the domain.  Then solve the BVP using spectral collocation methods for the coefficients (i.e., the tensor product of the polynomial rather than any sparse grid method)&lt;/li&gt;&#xA;&lt;li&gt;Use finite-differences in time, and a 1-dimensional chebyshev in space.  Hence, for each $t_j$ in the time grid,&#xA;$$v(t_j,z) \approx \sum_{i=0}^N a^{j}_i T_i(z) $$&#xA;and solve for the set of $a^j_i$ for each $j$ in the finite difference scheme, and the set of $N$ nodes at each period.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My question is:&#xA;If I don't care all that much for extreme precision, but care a great deal about convergence and stability, is either case reasonable or is there a good reason not to use a 2-dimensional Chebyshev basis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without going into details, the $\pi(t,z)$ and $g(t)$ come from an outside set of equilibrium conditions that need to be solved, and it would be very convenient to be able to approximate them as chebyshev polynomials themselves (and solve for all of the approximating coefficients for the $g(t)$ and the above $a_{ij}$ at the same time using auto-differentiation and a good nonlinear solver).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I found this, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10772/spectral-methods-in-time&quot;&gt;Spectral Methods in time&lt;/a&gt; but the exact set of problems didn't seem clear from the answer.  The typical advantages of a finite-difference time stepping are less clear in my case).&lt;/p&gt;&#xA;" OwnerUserId="9692" LastActivityDate="2017-08-11T04:37:38.280" Title="Solve a PDE BVP with Spectral methods in time and space?" Tags="|pde|spectral-method|chebyshev|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27606" PostTypeId="2" ParentId="27521" CreationDate="2017-08-11T07:33:11.583" Score="-1" Body="&lt;p&gt;You seem to be using explicit Euler in time, so your scheme would be&#xA;\begin{equation}&#xA;u^{k+1}_j = u^k_j + \frac{c \Delta t}{\Delta x} (u^k_j-u^k_{j-1})&#xA;\end{equation}&#xA;(if so, I don't really understand why you would make a sparse matrix). With the second approach that would give&#xA;\begin{equation}&#xA;u^{k+1}_N = u^k_N + \alpha c \Delta t&#xA;\end{equation}&lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2017-08-11T07:33:11.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27607" PostTypeId="1" CreationDate="2017-08-11T11:44:31.947" Score="1" ViewCount="193" Body="&lt;p&gt;Are there any high performance integer BLAS libraries that implement matrix multiplication i.e. i32gemm and i64gemm ? I need to use them for a cryptographic application and can tolerate overflows, i.e. I don't require the kernel to do any wraparound checks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between a Igemm and an Fgemm is that the latter retains the higher order bits and the previous retains the lower-order ones. My application would be performing computations on ciphers and not on the actual data. Since the data is selected in such a way that none of the elements of the product matrix (which are all signed) need more than 32 bits, the application can tolerate overflows on the ciphers.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2017-08-12T15:45:34.970" LastActivityDate="2018-04-09T19:55:13.457" Title="GPU libraries for integer matmul | overflow tolerated" Tags="|matrix|blas|dense-matrix|fixed-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27608" PostTypeId="2" ParentId="27521" CreationDate="2017-08-11T12:48:32.163" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;The problem is, nowhere in our spatial discretisation up to n=N do we&#xA;  have our function indexed at N+1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You just add an extra element (i.e. at N+1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, every loop set it to the value of the last element times $\alpha\Delta x$. When the &lt;em&gt;forward difference&lt;/em&gt; is calculated, it uses a value at $i+1$ to work out $i$. So although there's an extra element, it isn't part of the domain and doesn't appear in the result: it is purely a condition on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, in your code, you can handle the forward difference for the last element as a special case: instead of looking up the next element (i.e. past the last element), use the last element times $\alpha\Delta x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW: I think Barba introduces boundary conditions and other issues more simply in her &lt;a href=&quot;http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;12 Steps to Navier Stokes&lt;/em&gt;&lt;/a&gt; (which don't use a matrix solver). Might be worth a look; though it is more work.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-08-11T12:57:06.833" LastActivityDate="2017-08-11T12:57:06.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27610" PostTypeId="1" CreationDate="2017-08-11T18:45:30.513" Score="1" ViewCount="711" Body="&lt;p&gt;I'm prototyping a system that finds the 3D pose of a object in a video sequence. For this I minimize a error function involving the rotation and translation of the object as parameters and two sets of points (polluted with gaussian noise but no outliers) as data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to parameterize the rotation matrix both using 3 euler angles, and a rotation vector. Then I test my system with a synthetic data sequence and I see I obtain substantially less error with the 3 euler angles parameterization that with the rotation vector. I was expecting the opposite (notice the sequence I use doesn't result in any gimbal locks for the euler angles).  This makes me wonder if I'm using the rotation vector parameterization correctly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The steps I follow are, first I convert the rotation matrix to rotation vector (using the Rodrigues formula available in OpenCV) the resulting 3-vector is part of the parameters of my error function. Inside the error function I convert the rotation vector to rotation matrix and compute the residuals. I'm using python and scipy.optimize.leastsq (which internally uses MINPACK.lmdif) to minimize this error function. leastsq  will compute the jacobian needed for the minimization numerically from the error function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading a bit of theory about the lie group SO(3) and the lie algebra so(3), I understand that the so(3) (for me the rotation vector) corresponds to a tangent plane at the identity of SO(3). So my rotation vector is a local approximation to the corresponding rotation matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I thought that maybe I'm missing in my minimization is to convert the rotation vector back to a rotation matrix after each minimization step, and then back to a rotation vector (to take a new local approximation to the rotation matrix) for the next step. I can't test this possibility as I'm using a standard minimizer, which doesn't allow to do this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I think that the Jacobian calculated at each minimization step should be enough to direct the rotation vector parameters in the correct direction without the conversions I mention in the above paragraph. However, I'm not fully sure and as I get worse performance with the rotation vector parameterization wrt. the euler angle parameterization, I assume I'm doing something wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would someone clarify/confirm the correct way of using the rotation vector in a minimization problem? &lt;/p&gt;&#xA;" OwnerUserId="3785" LastEditorUserId="3785" LastEditDate="2017-08-14T11:56:54.287" LastActivityDate="2017-08-14T11:56:54.287" Title="Optimizing an error function involving rotation vectors in Python" Tags="|optimization|python|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27611" PostTypeId="2" ParentId="27521" CreationDate="2017-08-11T18:45:30.583" Score="1" Body="&lt;p&gt;The time-derivative scheme you are using is referred to as Forward Euler as is one of the simplest schemes out there, which I will assume for the rest of my answer. Even in the case that you would like to extend this, it is quite straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming to your implementation, we basically need to use &#xA;\begin{equation}&#xA;u_{j}^{N+1} = u_{j}^{N} + c \Delta t u_{x}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where you replace $u_{x}$ with the numerical derivative at the interior points and with the boundary condition for the end points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that you've stuck to a matrix notation, which complicates the implementation, the only change you'll need to make is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RHS = (DR + BR)*u&#xA;if boundary == 'Neumann':&#xA;    RHS[0] += c*dt*(alpha-u[0]/dx)&#xA;u = spsolve(DL+BL, RHS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This replaces the left boundary derivative with the boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Wikipedia article for the first-order upwind scheme is clear and would recommend going through it (&lt;a href=&quot;https://en.wikipedia.org/wiki/Upwind_scheme&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Upwind_scheme&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="15120" LastEditorUserId="15120" LastEditDate="2017-08-11T19:28:02.333" LastActivityDate="2017-08-11T19:28:02.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27612" PostTypeId="2" ParentId="27562" CreationDate="2017-08-11T22:59:50.937" Score="0" Body="&lt;p&gt;You also have to specify which scheme you want to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say that you want to use the simple Backward-Euler (BE) scheme. For BE scheme, your governing equation becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left(A+By_{n+1}-C(y'_{n+1})^{-1/3}\right)y''_{n+1}=-\left(B(y'_{n+1})^2+C\right) \tag{1}$&#xA;with&#xA;$y'_{n+1}=\frac{y_{n+1}-y_n}{\Delta t} \tag{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y''_{n+1}=\frac{y'_{n+1}-y'_n}{\Delta t} \tag{3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using (2) in (3),&#xA;$y''_{n+1}=\frac{1}{\Delta t^2}[y_{n+1}-y_n] - \frac{y'_n}{\Delta t}  \tag{4}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, using (2) and (4) in (1), you will get a nonlinear equation of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(y_n,y'_n, y_{n+1})=0 \tag{5}$&#xA;which can be solved for $y_{n+1}$ using an iterative scheme, for example, Newton-Raphson scheme. Once you get $y_{n+1}$ you can compute velocity and acceleration from (2) and (4), respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can follow the same procedure for other schemes. Equations (1), (2), (3) and (4) will be different for different schemes.&lt;/p&gt;&#xA;" OwnerUserId="24915" LastActivityDate="2017-08-11T22:59:50.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27613" PostTypeId="2" ParentId="27607" CreationDate="2017-08-12T02:17:50.013" Score="-1" Body="&lt;p&gt;I think &lt;a href=&quot;https://developer.nvidia.com/cublas&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuBLAS&lt;/a&gt; and &lt;a href=&quot;https://www.khronos.org/opencl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCL&lt;/a&gt; are the libraries you are looking for.&lt;/p&gt;&#xA;" OwnerUserId="24915" LastActivityDate="2017-08-12T02:17:50.013" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27614" PostTypeId="1" AcceptedAnswerId="31407" CreationDate="2017-08-12T07:15:53.990" Score="4" ViewCount="802" Body="&lt;p&gt;Suppose I have a multi-dimensional vector space $X$, and a collection of $n$ vectors $\{x_i\}_{i=1}^n \subset X$, which are not evenly &quot;spaced-out&quot; in $X$. I am searching for $m&amp;lt;&amp;lt;n$ of these $x_i$ that are good representatives of $X$, or in other words, they are roughly equi-distant to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I find these representatives? What algorithm (hopefully with an available implementation in Python) could I use? Would K-Medoids be a good choice?&lt;/p&gt;&#xA;" OwnerUserId="25131" LastEditorUserId="9667" LastEditDate="2017-08-12T16:31:54.783" LastActivityDate="2021-04-28T11:17:46.243" Title="Find representatives of vector-space in set of vectors?" Tags="|optimization|python|vector|" AnswerCount="4" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27615" PostTypeId="2" ParentId="8166" CreationDate="2017-08-12T10:14:31.893" Score="1" Body="&lt;p&gt;In the meantime we did some more research into this issue and documented our findings in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;M. Huber, U. Rüde, C. Waluga, B. Wohlmuth. Highly sparse surface&#xA;couplings for subdomain-wise isoviscous Stokes finite element&#xA;discretizations, Journal of Scientific Computing, 2017 (&lt;a href=&quot;https://link.springer.com/article/10.1007%2Fs10915-017-0470-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;,&#xA;&lt;a href=&quot;http://arxiv.org/abs/1604.07994&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In summary, there are a couple of points that can be made:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Yes, it is possible to combine the Laplacian formulation with some boundary modification to tackle Stokes interface problems and/or traction-type boundary conditions with reduced cross-couplings between the velocity components.&lt;/li&gt;&#xA;&lt;li&gt;To make this work we either need finite element spaces (V,Q) for velocity and pressure, respectively, where div(V)=Q, or we need to add some suitable pressure stabilization (we discuss and analyze one choice). The problem with (higher order) spaces that do not require such a stabilization is mostly inf-sup stability and this is a problem of the spaces themselves, not of our modification. For instance in 2D one would have to go to special meshes or use a P4-P3disc-pairing to avoid stability-related issues. In 3D the question of stability of such spaces is much harder and to my best knowledge a long-standing open question (see references in the paper).&lt;/li&gt;&#xA;&lt;li&gt;The modification adds boundary integral terms at the interface/boundary that introduce only couplings in tangential direction for H1 conforming velocity spaces.&lt;/li&gt;&#xA;&lt;li&gt;The modified Laplacian approach can also be exploited in matrix-free codes in such a way that only stencils at the interfaces need to be replaced and no integration over the boundary facets is required. Combined with suitable matrix-free solvers this can speed-up computations on large-scale architectures with only minor changes to the code.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4260" LastActivityDate="2017-08-12T10:14:31.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27616" PostTypeId="1" CreationDate="2017-08-12T17:15:39.123" Score="1" ViewCount="1471" Body="&lt;p&gt;Considering that COMSOL forum is dead and &lt;a href=&quot;http://www.cfd-online.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;CFD-online&lt;/a&gt; is not responsive, I'm gonna try asking my question here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done a 2D simulation in COMSOL multiphysics. I have two independent variables x and t and two dependent variable u1 and u2. Comsol gives me 2D projections of the 3D graphs u1(x,t) and u2(x,t) as below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NStWi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NStWi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I need is a 3D plot like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yuhjV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yuhjV.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried exporting the data. There are two options in COMSOL. Text which generates a file with several columns like this &#xA;&lt;a href=&quot;https://i.stack.imgur.com/u6PKQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u6PKQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other options is Unstructured VTKFile (*.vtk). I tried importing both into ParaView but I could not manage to get any plots. &lt;a href=&quot;https://ufile.io/7qp3y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; and &lt;a href=&quot;https://ufile.io/t5ew7&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; I uploaded the exported data and &lt;a href=&quot;https://ufile.io/ajuar&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is my COMSOL file&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if you could help me know how to do this in COMSOL, ParaView or any other possible software.&lt;/p&gt;&#xA;" OwnerUserId="22193" LastActivityDate="2023-01-17T06:32:10.403" Title="3D visualization of the results of a 2D simulation in COMSOL" Tags="|visualization|comsol|paraview|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27617" PostTypeId="1" CreationDate="2017-08-12T21:27:24.153" Score="0" ViewCount="725" Body="&lt;p&gt;I have $n$ points that form a grid with empty space and I need to find an algorithm that would calculate the total distance of those points with time complexity lower than $O(n^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a grid with $n=5$ could be represented as a matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;        \begin{matrix}&#xA;        0 &amp;amp; 0 &amp;amp; 1 \\&#xA;        1 &amp;amp; 0 &amp;amp; 1 \\&#xA;        0 &amp;amp; 1 &amp;amp; 1 \\&#xA;        \end{matrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the distance for point $(1,3)$ is:&#xA;$1+\sqrt{5}+2+\sqrt{5}=3+2\sqrt{5}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the distance for point $(2,1)$ is:&#xA;$2+\sqrt{2}+\sqrt{5}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the distance for point $(2,3)$ is:&#xA;$\sqrt{2}+1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the distance for point $(3,2)$ is:&#xA;$1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the distance for point $(3,3)$ is:&#xA;$0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the total distance is: $7+2\sqrt{2}+3\sqrt{5}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach simply calculates separately the distance and then adds it up which doesn't take into account the fact that they are positions in layers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't look like a super unique issue - is there any existing algorithm for this or does anyone have an idea how to speed this up?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;By Total Distance I mean a situation like this:&#xA;I pick a point and then calculate the euclidean distance between the picked point and the rest $n-1$ points. Then I choose the next point and I calculate it's distance betwenn it and the rest $n-2$ points, and so on. I then sum up all the distances to get the total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explanation for distance calculation:&#xA;The distance for point $(1,3)$ is the sum of euclidean distances between point $(1,3)$ and $(2,1)$, $(2,3)$, $(3, 2)$, $(3,3)$.&#xA;The distance for point $(2,1)$ is the sum of euclidean distances between point $(2,1)$ and  $(2,3)$, $(3, 2)$, $(3,3)$.&#xA;and so on...&lt;/p&gt;&#xA;" OwnerUserId="25133" LastEditorUserId="25133" LastEditDate="2017-08-14T11:07:11.600" LastActivityDate="2017-08-15T10:50:02.900" Title="Total distance between points on a grid" Tags="|distance-measure|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27618" PostTypeId="2" ParentId="27614" CreationDate="2017-08-12T23:02:03.623" Score="1" Body="&lt;p&gt;Sounds like you want to &lt;strong&gt;thin&lt;/strong&gt; your data where it is dense, and learn the support of your data summarized by data points. If you don't have too many points, you can generate a distance matrix, and prune the points with the closest neighbors. (I don't think this method has a name.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, if you are trying to find the extremal points, then &lt;a href=&quot;https://en.wikipedia.org/wiki/Archetypal_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;archetypal analysis&lt;/a&gt; might be a better direction.&lt;/p&gt;&#xA;" OwnerUserId="1128" LastActivityDate="2017-08-12T23:02:03.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27619" PostTypeId="1" CreationDate="2017-08-13T01:04:14.940" Score="1" ViewCount="265" Body="&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to solve a nonlinear integro-differential equations on a 2D square domain $\Omega$ subject to Neumann boundary conditions using finite differences:&#xA;$$\frac{\partial u}{\partial t} = f(u) + \int_\Omega G(x'-x)\left[u(x')-u(x)\right] dx'.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variable $u(x,t)$ depends on space and time, $f$ is a nonlinear function and $G$ is the integral kernel. Neumann (also called no-flux or reflective) boundary conditions are given as&#xA;$$ \frac {\partial u}{\partial x}\Bigr|_{\partial\Omega} = 0$$&#xA;at the boundary $\partial\Omega$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution Attempt&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually to implement Neumann boundary conditions, it is either possible to use ghost cells or modify the stencil. Let's try the latter. Starting from the heat (or diffusion) equation in 1d, we know the $2^{nd}$ derivative can be discretized as follows:&#xA;$$ \Delta u \rightarrow \frac{u_{i+1}-2u_i+u_{i-1}}{h^2}=\frac{1}{h^2}\left((u_{i+1}-u_i)+(u_{i-1}-u_i)\right),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u_i$ is shorthand notation for $u(x_i)$ and $h$ is the uniform distance between neighboring points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $G=1/h^2$, this formula can be rephrased as a discretized version of the integral equation from the top:&#xA;$$ \sum_{j=i-1}^{i+1} G(x_j-x_i)\left[u_j - u_i\right].$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Differences that involve points $x_j$ outside the domain are zero, due to the Neumann boundary condition:&#xA;$$ \frac {\partial u}{\partial x}\Bigr|_{\partial\Omega} \rightarrow \frac{u_j-u_i}{h} \overset{!}{=} 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pseudo C code might look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(j=max(0,i-n); j&amp;lt;min(n-1,i+n); j++){&#xA;   uIntegro[i] += G[j-i]*u[j];&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coming back to the original problem, given that the integral kernel ranges beyond the first neighbor, let's say it takes into account the first 5 neighbors, is it correct to just set all differences in the integral involving outside points to zero in order to realize Neumann boundary conditions (in 2d as well)? Bonus: If this is not Neumann, what would it be called instead?&lt;/p&gt;&#xA;" OwnerUserId="25135" LastEditorUserId="9667" LastEditDate="2017-08-13T20:59:50.460" LastActivityDate="2017-08-13T20:59:50.460" Title="Implementing Neumann boundary condition in nonlinear integro-differential equation" Tags="|finite-difference|differential-equations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27621" PostTypeId="1" CreationDate="2017-08-13T14:04:26.350" Score="5" ViewCount="89" Body="&lt;p&gt;I need to diagonalize such kind of matrix during research: it's n-by-n, with it's upper-left (n-1)-by-(n-1) corner be diagonal while the nth row &amp;amp; column are dense. It's observed that the self adjoint eigensolver is not quite efficient as a solution when n is greater than roughly 2000 and there is no existing eigensolver to take advantage of the sparseness. A possible way I just came up with is to do the LU via the corresponding sparse package first and then attack the two triangular matrices. However I'd love to hear more suggestions &amp;amp; insight. &lt;/p&gt;&#xA;" OwnerUserId="24512" LastActivityDate="2017-08-13T22:22:49.153" Title="How to construct a eigensolver targeting a specific type of matrix" Tags="|sparse-matrix|eigensystem|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27624" PostTypeId="2" ParentId="27590" CreationDate="2017-08-14T07:55:16.350" Score="1" Body="&lt;p&gt;So after much testing, I have found a solution to this problem. However, I am not quite sure of the mathematical justification for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, when using purely periodic BC, iterative solvers would not converge and the solution will keep growing. By fixing Dirichlet BC on all sides of the domain to a fixed constant $C \in \mathbb{R}$, on top of using periodicity, the RHS term can be obtained by differentiating the solution with an order of accuracy of 2 (note that high-order scheme are using for numerical differentiation to ensure the leading error term comes from the Poisson solver).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, this got me thinking that there was no need for periodic BC to begin with since only the gradient of the solution is of interest to me:$\; \dfrac{\partial p}{\partial x_i}$. By just enforcing dirichlet BC on all sides, this gradient can be retrieved with the desire order of accuracy. I will be happy to hear more of why that is though.&lt;/p&gt;&#xA;" OwnerUserId="16329" LastActivityDate="2017-08-14T07:55:16.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27625" PostTypeId="1" CreationDate="2017-08-14T08:47:21.363" Score="1" ViewCount="218" Body="&lt;p&gt;These are the discretized drift diffusion equations as taken from the book &quot;Analysis and Simulation of Semiconductor Devices&quot;. The electron continuity equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$((\Delta_{j-1}^y+\Delta_j^y)/2\Delta_i^x).D_{i+1/2,j}^n.B((\varphi_{i+1,j}-\varphi_{i,j})/V_T).n_{i+1,j}  + ((\Delta_{j-1}^y+\Delta_j^y)/2\Delta_{i-1}^x).D_{i-1/2,j}^n.B((\varphi_{i-1,j}-\varphi_{i,j})/V_T).n_{i-1,j}  - [(((\Delta_{j-1}^y+\Delta_j^y)/2\Delta_i^x).D_{i+1/2,j}^n.B((\varphi_{i,j}-\varphi_{i+1,j})/V_T)) + (((\Delta_{j-1}^y+\Delta_j^y)/2\Delta_{i-1}).D_{i-1/2,j}^n.B((\varphi_{i,j}-\varphi_{i-1,j})/V_T))  + (((\Delta_{i-1}^x+\Delta_i^x)/2\Delta_j^y).D_{i,j+1/2}^n.B((\varphi_{i,j}-\varphi_{i,j+1})/V_T)) +  (((\Delta_{i-1}^x+\Delta_i^x)/2\Delta_{j-1}^y).D_{i,j-1/2}^n.B((\varphi_{i,j}-\varphi_{i,j-1})/V_T))].n_{i,j}  + ((\Delta_{i-1}^x+\Delta_i^x)/2\Delta_j^y).D_{i,j+1/2}^n.B((\varphi_{i,j+1}-\varphi_{i,j})/V_T).n_{i,j+1}  + ((\Delta_{i-1}^x+\Delta_i^x)/2\Delta_{j-1}^y).D_{i,j-1/2}^n.B((\varphi_{i,j-1}-\varphi_{i,j})/V_T).n_{i,j-1}  = R_{i,j}^{EFFECTIVE}.((\Delta_{i-1}^x+\Delta_i^x)/2).((\Delta_{j-1}^y+\Delta_j^y)/2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hole continuity equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$((\Delta_{j-1}^y+\Delta_j^y)/2\Delta_i^x).D_{i+1/2,j}^p.B((\varphi_{i,j}-\varphi_{i+1,j})/V_T).p_{i+1,j}  + ((\Delta_{j-1}^y+\Delta_j^y)/2\Delta_{i-1}^x).D_{i-1/2,j}^p.B((\varphi_{i,j}-\varphi_{i-1,j})/V_T).p_{i-1,j}  - [(((\Delta_{j-1}^y+\Delta_j^y)/2\Delta_i^x).D_{i+1/2,j}^p.B((\varphi_{i+1,j}-\varphi_{i,j})/V_T)) + (((\Delta_{j-1}^y+\Delta_j^y)/2\Delta_{i-1}^x).D_{i-1/2,j}^p.B((\varphi_{i-1,j}-\varphi_{i,j})/V_T))  + (((\Delta_{i-1}^x+\Delta_i^x)/2\Delta_j^y).D_{i,j+1/2}^p.B((\varphi_{i,j+1}-\varphi_{i,j})/V_T)) +  (((\Delta_{i-1}^x+\Delta_i^x)/2\Delta_{j-1}^y).D_{i,j-1/2}^p.B((\varphi_{i,j-1}-\varphi_{i,j})/V_T))].p_{i,j}  + \\ ((\Delta_{i-1}^x+\Delta_i^x)/2\Delta_j^y).D_{i,j+1/2}^p.B((\varphi_{i,j}-\varphi_{i,j+1})/V_T).p_{i,j+1}  + ((\Delta_{i-1}^x+\Delta_i^x)/2\Delta_{j-1}^y).D_{i,j-1/2}^p.B((\varphi_{i,j}-\varphi_{i,j-1})/V_T).p_{i,j-1}  = R_{i,j}^{EFFECTIVE}.((\Delta_{i-1}^x+\Delta_i^x)/2).((\Delta_{j-1}^y+\Delta_j^y)/2\ ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Delta_i$ and $\Delta_j$ are the mesh spacing for finite difference grid along X and Y directions. $D_n$,$D_p$ are electron and hole diffusion coefficients.$B(x) = x/(e^x-1)$ is Bernoulli's function and $V_T$ is the thermal voltage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, $$R^{EFFECTIVE} = R^{SRH} + R^{AU} + R^{OPT} + R^{II}$$ where&#xA;$R^{EFFECTIVE}$ is the effective generation/recombination term. $R^{SRH}$, $R^{AU}$, $R^{OPT}$ and $R^{II}$ are SRH, Auger, Radiative and Impact Ionization Recombination terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I am applying the coupled Newton's iterative scheme to solve the above equations where I need to differentiate the equations. When differentiating with respect to potential '$\varphi$', I noticed that $\varphi$ appears in three places namely,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;High field Electron/Hole Mobility, $\mu_{n,p}^{high}=\mu_{n,p}^{low}⁄ [1 + (\mu_{n,p}^{LICN}.E_{n,p}/v_{n,p}^{sat})^{\beta_{n,p}}]^{1/\beta_{n,p}}$ and $D_{n,p} = \mu_{n,p}.V_T$;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Bernoulli's function, $B(x)$ where 'x' here means difference between potentials at neighboring grid points and&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Impact Ionization $R^{II} = -\alpha_n.(|J_{i,j}^n|/q) - \alpha_p.(|J_{i,j}^p|/q)$, $\alpha_n = a_n.e^{-b_n/|E_{i,j}|}$, $\alpha_p = a_p.e^{-b_p/|E_{i,j}|}$ where $J^{n,p}$ and $E$ represent electron/hole currents and electric field respectively.&#xA;Current terms are made of potential terms as in,&#xA;$J_{i+1/2,j}^{nx}=D_{i+1/2,j}^n.[B(\frac{\varphi_{i,j}-\varphi_{i+1,j}}{V_T}).n_{i,j} - B(\frac{\varphi_{i+1,j}-\varphi_{i,j}}{V_T}).n_{i+1,j}]/\Delta_i^x  \rightarrow  (31)$, $J_{i,j+1/2}^{ny}=D_{i,j+1/2}^n.[B(\frac{\varphi_{i,j}-\varphi_{i,j+1}}{V_T}).n_{i,j} - B(\frac{\varphi_{i,j+1}-\varphi_{i,j}}{V_T}).n_{i,j+1}]/\Delta_j^y  \rightarrow  (32)$, $J_{i+1/2,j}^{px}=D_{i+1/2,j}^p.[B(\frac{\varphi_{i,j}-\varphi_{i+1,j}}{V_T}).p_{i+1,j} - B(\frac{\varphi_{i+1,j}-\varphi_{i,j}}{V_T}).p_{i,j}]/\Delta_i^x  \rightarrow  (33)$, $J_{i,j+1/2}^{py}=D_{i,j+1/2}^p.[B(\frac{\varphi_{i,j}-\varphi_{i+1,j}}{V_T}).p_{i,j+1} - B(\frac{\varphi_{i+1,j}-\varphi_{i,j}}{V_T}).p_{i,j}]/\Delta_j^y  \rightarrow  (34)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, basically I need to differentiate all of these to find the Jacobian with respect to $\varphi$. But, the problem arises when the difference between potentials become zero. This means that the entire electron/hole equation becomes a constant when the potential difference is zero. Hence, the Jacobian is also zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would mean that the final Jacobian matrix would have entire columns and rows made out of zeros thus making the matrix singular and it is not possible to solve a singular matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the correct approach to solving using the Newton's method. What should I do when the potential difference is zero?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Selberherr, Siegfried. Analysis and simulation of semiconductor devices. Springer Science &amp;amp; Business Media, 2012.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25146" LastEditorUserId="9667" LastEditDate="2017-08-14T17:45:28.943" LastActivityDate="2017-08-14T17:45:28.943" Title="Jacobian of the electron and hole drift diffusion equations with respect to potential in semiconductors" Tags="|newton-method|advection-diffusion|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27627" PostTypeId="2" ParentId="27617" CreationDate="2017-08-14T12:55:04.473" Score="3" Body="&lt;p&gt;If I understand the questions correctly, you have the following situation.&#xA;Be $S=\{x_1,\ldots,x_n\}$ the set of points. You need to calculate the total distance $d_t$of $S$, defined as$$d_t=\sum_{j=1}^{n-1}\sum_{i_j=j+1}^n |x_{i_j}-x_j|$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition is equivalent to the sum of the upper or lower triangle matrix of the distance matrix of the set S.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D=\left(\begin{array}{cccc}&#xA;0&amp;amp;x_{12}&amp;amp;x_{13}&amp;amp;\ldots &amp;amp;x_{1n}\\&#xA;x_{21}&amp;amp;0&amp;amp;x_{23}&amp;amp;\ldots &amp;amp;x_{2n}\\&#xA;\vdots &amp;amp;&amp;amp; \ddots &amp;amp;&amp;amp; \vdots \\&#xA;x_{n1}&amp;amp;x_{n2} &amp;amp;\ldots &amp;amp;&amp;amp;0&#xA;\\&#xA;\end{array}\right)$$&#xA;Here $x_{ij}=|x_i-x_j|$.To calculate the distance matrix alone you need $n(n-1)/2=O(n^2)$ operations. So no, I am not aware of an algorithm which is better than $O(n^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an approximation to $d_t$ is in order, than you can use nearest neighbor search algorithms which do better than $O(n^2)$ and sum only those distance pairs which dominate the sum.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2017-08-14T12:55:04.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27628" PostTypeId="1" CreationDate="2017-08-14T13:41:53.093" Score="1" ViewCount="1949" Body="&lt;p&gt;I'm trying to implement Dinur-Nissim algorithm and am stuck at how to solve the set of linear inequalities with multiple unknowns and a large number of equations along with constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/frwMC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/frwMC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;0.2 \leq c_4 \leq 0.66\\&#xA;0.66 \leq c_3 \leq 1.56\\&#xA;0.96 \leq c_3 + c4 \leq 2.26\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$0 \leq c_i \leq 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and many other equations with the number of unknowns going up to $c_n$ where $n$ is the size of the database, so I need a solution which works for a large number of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to look for some libraries but most of them solve Maximization or Minimization problems so am not sure if its possible to convert these equations to one of those problems.&lt;/p&gt;&#xA;" OwnerUserId="25149" LastEditorUserId="9667" LastEditDate="2017-08-14T17:52:39.303" LastActivityDate="2017-09-13T18:16:20.913" Title="Solve a set of multivariate linear inequalities with constraints in Python" Tags="|optimization|python|linear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27629" PostTypeId="2" ParentId="27628" CreationDate="2017-08-14T13:47:49.200" Score="1" Body="&lt;p&gt;If you have a set of linear inequalities and linear equalities, you can use linear programming software to find a solution, or determine there is no solution.  You can make up any (linear) objective function to be minimized or maximized, for instance, the function which equals 0.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2017-08-14T13:47:49.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27630" PostTypeId="2" ParentId="27601" CreationDate="2017-08-14T13:55:20.153" Score="1" Body="&lt;p&gt;For real valued function g(x), you can do that as follows,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\textrm{d} f}{\textrm{d} x}=&#xA;\frac{\textrm{d} f}{\textrm{d} g}\frac{\textrm{d} g}{\textrm{d} x} \approx&#xA;\frac{\textrm{d} f}{\textrm{d} g}\frac{\textrm{Im}\left\{g(x+i\epsilon)\right\}}{\epsilon}&#xA;\end{equation}&#xA;where $i$ is imaginary number. $\textrm{Im}\{z\}$ gives the imaginary part of the complex number z.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can implement that in any language which can do operations on complex numbers. You can have very small $\epsilon$, to have small truncation error $O(\epsilon^2)$  and avoid subtractive cancellation error. I don't know how fast it is but is very fast to implement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details see,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Squire, William; Trapp, George&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1137/S003614459631241X&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Using complex variables to estimate derivatives of real functions&lt;/strong&gt;&lt;/a&gt;, SIAM Rev. 40, No.1, 110-112 (1998). &lt;a href=&quot;https://zbmath.org/?q=an:0913.65014&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL0913.65014&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-08-14T20:28:35.157" LastActivityDate="2017-08-14T20:28:35.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27631" PostTypeId="1" CreationDate="2017-08-14T14:06:45.290" Score="1" ViewCount="57" Body="&lt;p&gt;I have &lt;a href=&quot;https://pastebin.com/A3s9dUKp&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; dataset, and I am using &lt;code&gt;y = (a * x^n) / (b + x^n)&lt;/code&gt; Hill function as the model, where &lt;code&gt;a&lt;/code&gt; is the limit of the Hill curve, &lt;code&gt;b&lt;/code&gt; is the point at which &lt;code&gt;a/2&lt;/code&gt; is reached (for &lt;code&gt;n&lt;/code&gt; = 1) and &lt;code&gt;n&lt;/code&gt; is the cooperativity or steepness of the curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am storing all &lt;code&gt;X,y&lt;/code&gt; values, computing the parameters from &lt;code&gt;scipy.optimize.curve_fit&lt;/code&gt;, and plotting the curve. If new data points come along, I re-calculate the parameters with the old+new data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to update the parameters of the model without storing &lt;strong&gt;all&lt;/strong&gt; of the previous old data points, once the initial parameters are obtained from the previous data points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example, I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My existing code is as follows (not super elegant).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;from scipy.optimize import curve_fit&#xA;&#xA;def file_stream(file_name):&#xA;    with open(file_name, 'r') as in_file:&#xA;        for line in in_file:&#xA;            yield map(float, line.strip().split('\t'))&#xA;&#xA;def hill_model(X, a, b, n):&#xA;    return [float((a * x**n)) / (b + x**n) for x in X]&#xA;&#xA;def get_params(X_all, y_all, prev_par=None, fn=hill_model):&#xA;    if prev_par is None:&#xA;        a_init, b_init, n_init = y_all[-1], y_all[0], 1.0&#xA;    else:&#xA;        a_init, b_init, n_init = prev_par&#xA;    opt_par, opt_cov = curve_fit(fn, X_all, y_all, p0=[a_init, b_init, n_init])&#xA;    a_final, b_final, n_final = opt_par&#xA;    return a_final, b_final, n_final&#xA;&#xA;def main():&#xA;    file_name     = 'data.tsv'&#xA;    file_streamer = file_stream(file_name)&#xA;    X_all, y_all  = [], []&#xA;&#xA;    # Get some intial data from stream&#xA;    for _ in xrange(1000):&#xA;        X, y = file_streamer.next()&#xA;        X_all.append(X)&#xA;        y_all.append(y)&#xA;    plt.scatter(X_all, y_all)&#xA;&#xA;    # Initialize params of model&#xA;    a, b, n = get_params(X_all, y_all)&#xA;    y_model = hill_model(X_all, a, b, n)&#xA;    plt.plot(X_all, y_model, 'r-')&#xA;    plt.show()&#xA;&#xA;    # Rolling update&#xA;    seen_all = False # Helps stop when all data is fit&#xA;    while True:&#xA;        for _ in xrange(1000):&#xA;            try:&#xA;                X, y = file_streamer.next()&#xA;                X_all.append(X)&#xA;                y_all.append(y)&#xA;            except:&#xA;                seen_all = True&#xA;                break&#xA;        a, b, n = get_params(X_all, y_all, prev_par=[a, b, n], fn=hill_model)&#xA;        y_model = hill_model(X_all, a, b, n)&#xA;        plt.scatter(X_all, y_all)&#xA;        plt.plot(X_all, y_model, 'r-')&#xA;        plt.show()&#xA;&#xA;        # Nothing more to update, return&#xA;        if seen_all:&#xA;            return&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code currently reads in some &lt;code&gt;X,y&lt;/code&gt; values, calculates the &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;b&lt;/code&gt;, &lt;code&gt;n&lt;/code&gt; parameters and when more &lt;code&gt;X,y&lt;/code&gt; values are added, the code updates &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;b&lt;/code&gt;, and &lt;code&gt;n&lt;/code&gt; params. As you can see, I need to store previous &lt;code&gt;X,y&lt;/code&gt; values, which I do not &lt;strong&gt;want&lt;/strong&gt;. I want to update the parameters as new &lt;code&gt;X,y&lt;/code&gt; values are seen and from the previous &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;b&lt;/code&gt;, and &lt;code&gt;n&lt;/code&gt; values only.&lt;/p&gt;&#xA;" OwnerUserId="25145" LastEditorUserId="25145" LastEditDate="2017-08-16T01:43:05.650" LastActivityDate="2017-08-16T01:43:05.650" Title="Update model parameter with new data, discarding old data" Tags="|python|scipy|machine-learning|numpy|regression|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27632" PostTypeId="2" ParentId="27614" CreationDate="2017-08-14T16:27:44.007" Score="0" Body="&lt;p&gt;Another option, in the direction of clustering, could be k-Maxoids clustering:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/8606/8216d21cb43403f8fb1fbfae4ca49d06f8dc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pdfs.semanticscholar.org/8606/8216d21cb43403f8fb1fbfae4ca49d06f8dc.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://ceur-ws.org/Vol-1670/paper-65.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ceur-ws.org/Vol-1670/paper-65.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Another answer could be &lt;em&gt;(Growing) Neural Gas&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25131" LastEditorUserId="25131" LastEditDate="2021-04-28T11:17:46.243" LastActivityDate="2021-04-28T11:17:46.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27633" PostTypeId="1" AcceptedAnswerId="27673" CreationDate="2017-08-14T19:09:39.390" Score="1" ViewCount="7317" Body="&lt;p&gt;I want to simulate a simple 3D diffusion (e.g., an ink released from one side of a vessel) using &lt;code&gt;SciPy&lt;/code&gt;. There are some tutorials for one-dimensional diffusion. Although the ink goes in one direction, it will not be straight. I appreciate if you can help me to consider the spread of the ink along with other dimensions too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the Fick's second law in one-dimension gives a straightforward equation to draw the concentration profiles against &lt;em&gt;x&lt;/em&gt; at different times. How to compute the profile of &lt;em&gt;c(x,y,z,t)&lt;/em&gt; at different times?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If introducing the code in any other programming language, I can transform it into python. I just want to understand the algorithm structure.&lt;/p&gt;&#xA;" OwnerUserId="25151" LastEditorUserId="25151" LastEditDate="2017-08-14T20:09:21.473" LastActivityDate="2017-08-21T12:17:39.570" Title="How to simulate 3D diffusion in python?" Tags="|python|scipy|advection-diffusion|diffusion|differential-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27634" PostTypeId="1" CreationDate="2017-08-14T19:36:59.897" Score="5" ViewCount="119" Body="&lt;p&gt;Suppose I am given a vector $v^0\in\mathbb{R}^n$ and integers $k,\ell\in\mathbb{Z}$.  Assuming $k$ is close to zero (e.g. $0\leq k\leq5$), is there an algorithm for solving the following integer optimization problem?&#xA;$$&#xA;\boxed{&#xA;\begin{array}{rl}&#xA;\min_v &amp;amp; \|v-v^0\|_2\\&#xA;\mathrm{s.t.} &amp;amp; v_i\in\{-k,\ldots,0,\ldots,k\}\ \forall i\\&#xA;&amp;amp; \sum_i v_i = \ell.&#xA;\end{array}&#xA;}&#xA;$$&#xA;In other words, I wish to approximate $v^0$ with a vector $v$ that sums to $\ell$ and has integer entries between $-k$ and $k$.  The $L_2$ norm here is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This looks &lt;em&gt;somewhat&lt;/em&gt; like an instance of the knapsack problem, but the $L_2$ norm and potentially negative entries in $v$ have me confused how to formalize this, e.g. using some sort of dynamic programming technique.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2017-08-16T02:07:57.107" Title="$L_2$ projection with integer constraints and prescribed sum" Tags="|optimization|nonlinear-programming|mixed-integer-programming|projection|dynamic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27635" PostTypeId="2" ParentId="27139" CreationDate="2017-08-15T07:11:49.213" Score="0" Body="&lt;p&gt;Standard initialization of homogeneous, isotropic turbulence is through the Passot-Pouquet spectrum$^{[1]}$ or the Von Karman-Pao spectrum$^{[2]}$. Both initializations are done in wavenumber space and hence easier to implement with a spectral/pseudo-spectral code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Passot, Thierry, and Annick Pouquet. &quot;Numerical simulation of compressible homogeneous flows in the turbulent regime.&quot; Journal of Fluid Mechanics 181 (1987): 441-466.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://turbulence.utah.edu/media/karman.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://turbulence.utah.edu/media/karman.html&lt;/a&gt;. Also &#xA;Pao, Yih‐Ho. &quot;Structure of turbulent velocity and scalar fields at large wavenumbers.&quot; The Physics of Fluids 8.6 (1965): 1063-1075. &#xA;for the derivation of the form.&lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2017-08-15T07:11:49.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27637" PostTypeId="1" CreationDate="2017-08-15T20:37:09.620" Score="0" ViewCount="44" Body="&lt;p&gt;I'm interested in modeling the electromagnetic fields radiated from an accelerated charge, but do not want to reinvent stuff if possible. I suspect there are software packages already out there which allows a person to provide the incoming accelerating field or trajectory of the charge, and is returned an array of values of the field in some region of space etc.&lt;/p&gt;&#xA;" OwnerUserId="25155" LastActivityDate="2017-08-16T00:05:33.760" Title="What software packages are designed towards modelling the radiation from accelerated charges?" Tags="|software|electromagnetism|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27638" PostTypeId="2" ParentId="27637" CreationDate="2017-08-16T00:05:33.760" Score="1" Body="&lt;p&gt;An answer to this could go in a lot of directions. I think the closest class of software matching the question as asked would be a &quot;particle in cell (PIC) method&quot;, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Particle-in-cell&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Particle-in-cell&lt;/a&gt; for an overview. It's basically an FDTD-like method, with &quot;particle pushing&quot; (Lorentz forces) bolted on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &quot;modeling electromagnetic fields radiated from an accelerated charge&quot; is pretty abstract, and most electromagnetic modeling software is aimed/tuned at a more specific class of problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A PIC method, for instance, would be used to model a high power microwave source (magnetrons etc, a particle beam shaped/controlled by quasistatic magnetic field). But a different engineering problem might be more easily modeled using a different method (antenna radiation is easy to model using finite elements, radar scattering is easy to model using surface integral equations, etc). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though all of these scenarios can be abstractly lumped as &quot;radiation due to accelerated charge&quot;, each would probably lead you to pursue a different technique.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2017-08-16T00:05:33.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27639" PostTypeId="2" ParentId="27634" CreationDate="2017-08-16T01:45:16.900" Score="1" Body="&lt;p&gt;Presuming you have a suitable solver (CPLEX, GUROBI, MOSEK, SCIP, many others), you can solve this as a Mixed-Integer Quadratic Program (by squaring the objective) or as a Mixed-Integer Second Order Cone Problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is code to solve it in YALMIP under MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v = intvar(size(v_o))&#xA;optimize([-k &amp;lt;= v &amp;lt;= k, sum(v) == l],norm(v-v_0))&#xA;disp(value(v)) $ displays the optimal value of v&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2017-08-16T02:07:57.107" LastActivityDate="2017-08-16T02:07:57.107" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27640" PostTypeId="1" CreationDate="2017-08-16T02:49:59.617" Score="1" ViewCount="81" Body="&lt;p&gt;I want to compute the difference $\Delta f(x_1,x_2) = f(x_1)-f(x_2)$ of a smooth function $f(x)$ at two points $x_1$ and $x_2$ which are close to each other. The magnitude of the expected result, $|\Delta f(x_1,x_2)|$, is often small compared to $|f(x_1)|$ and $|f(x_2)|$ and several significant digits are lost if I evaluate the expression $f(x_1)-f(x_2)$ straightforwardly. What is a recommended way to compute $\Delta f(x_1,x_2)$ accurately?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an analytical expression of $f(x)$. If a detail matters, it has a form like&#xA;$f(x) = \exp(-a x^2) \sin(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps one method would be to use the Taylor expansion of $f(x)$, but I am afraid that the convergence might not be very fast. Also, writing down the higher order derivatives is tedious. I'd like to know something easier and more efficient.&lt;/p&gt;&#xA;" OwnerUserId="14516" LastActivityDate="2017-08-19T03:39:08.740" Title="Computing the change of function at two close points without cancellation" Tags="|numerical-analysis|finite-difference|algorithms|precision|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27641" PostTypeId="1" AcceptedAnswerId="27642" CreationDate="2017-08-16T14:28:22.490" Score="4" ViewCount="988" Body="&lt;p&gt;I am trying to calculate the singular value decomposition of &lt;a href=&quot;https://pastebin.com/takPtiVD&quot; rel=&quot;nofollow noreferrer&quot;&gt;this matrix&lt;/a&gt; using &lt;code&gt;numpy.linalg.svd&lt;/code&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, reconstructing the matrix from the SVD gives a poor reconstruction - in particular, the first row and second column are far smaller in the reconstruction than in the original matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u, w, vt = np.linalg.svd(M)&#xA;np.allclose(M, np.dot(u, np.dot(np.diag(w), vt)))&#xA;&#xA;&amp;gt;&amp;gt; False&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The matrix is singular ( &lt;code&gt;w[-1]&lt;/code&gt; is zero), and the largest singular value is much bigger than the other non-zero values (&lt;code&gt;3e+24&lt;/code&gt; vs &lt;code&gt;7e-2&lt;/code&gt; for the next largest).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I expect that the reconstruction from the SVD be poor for a matrix this badly conditioned? Are their other more stable ways I could calculate the SVD otherwise?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I am computing the SVD is a diagnosis for the pseudoinverse of &lt;code&gt;M&lt;/code&gt; I calculate using &lt;code&gt;numpy.linalg.pinv&lt;/code&gt;, which should return the Moore-Penrose pseudoinverse. As I understand it, this particular pseudoinverse should be symmetric since &lt;code&gt;M&lt;/code&gt; is symmetric, but it is not. My assumption is that this is because of problems with the SVD, which &lt;code&gt;numpy.linalg.pinv&lt;/code&gt; calculates.&lt;/p&gt;&#xA;" OwnerUserId="21311" LastActivityDate="2017-08-16T16:04:34.780" Title="Poor SVD reconstruction of singular matrix" Tags="|linear-algebra|python|numpy|svd|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27642" PostTypeId="2" ParentId="27641" CreationDate="2017-08-16T16:04:34.780" Score="9" Body="&lt;p&gt;Algorithms for the SVD, as more or less every classical linear algebra algorithm based on orthogonal transformations, are &lt;em&gt;normwise&lt;/em&gt; backward stable, i.e., it should be guaranteed that $\frac{\|USV^* - A \|}{\|A\|} = O(u)$, where the norms are Euclidean norms, $u$ is the machine precision, and &quot;$O(u)$&quot; means that the first-order term in $u$ is bounded by a polynomial in the matrix dimensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, this bound in norm does &lt;em&gt;not&lt;/em&gt; guarantee that all the elements are close: for instance, compare&#xA;$$&#xA;a=\begin{bmatrix}&#xA;10^{15}\\1&#xA;\end{bmatrix}&#xA;\quad&#xA;\text{vs.}&#xA;\quad&#xA;b=\begin{bmatrix}&#xA;10^{15}\\-1&#xA;\end{bmatrix}.&#xA;$$&#xA;These two vectors are normwise very close to each other, so $\frac{\|a-b\|}{\|b\|}$ is very small, but they are not so elementwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.scipy.org/doc/numpy/reference/generated/numpy.allclose.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;numpy.allclose&lt;/code&gt;&lt;/a&gt; compares elementwise, not normwise.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-08-16T16:04:34.780" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27643" PostTypeId="1" AcceptedAnswerId="27644" CreationDate="2017-08-16T16:21:37.987" Score="2" ViewCount="2265" Body="&lt;p&gt;I would like to volumetrically render 3D scalar data in Paraview, and I'm not sure if my inability to do so is incorrect usage of VTK or Paraview.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have built a *.vtu VTK unstructured grid file containing 2 cells consisting of 10 PolyVertex objects with scalar data associated with each point. These are meant to represent points where the value of the numerical solution is known in space (quadrature points for a finite element, for example). I can load the file into Paraview and view the output as points:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sgpXm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sgpXm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No problem. However, when I choose the &quot;Volume&quot; representation, rather than points, the points simply disappear, without any volume rendering. I am looking for paraview to linearly interpolate the solution between the points for each cell, the way it would if I provide an Image (Uniform Rectilinear Grid) from a data file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0rqx6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0rqx6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I seem to be unable to find documentation for how to do this. I imagine the finite element community must commonly render unstructured volume data, so this is surprising. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the source code used to write out the VTK file in Python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class VtkPolyVertCloud(object):&#xA;    &quot;&quot;&quot; save each finite element as a set of polyvertices, but lose cell information &quot;&quot;&quot;&#xA;&#xA;    def __init__(self):&#xA;&#xA;        # geometry&#xA;        self.points= vtk.vtkPoints()&#xA;        self.grid = vtk.vtkUnstructuredGrid()&#xA;&#xA;        # data&#xA;        self.values = vtk.vtkDoubleArray()&#xA;        self.values.SetName('point_values_array')&#xA;&#xA;        self.grid.SetPoints(self.points)&#xA;        self.grid.GetPointData().SetScalars(self.values)&#xA;&#xA;    def add_polyVertex_cell(self, points, data):&#xA;        &quot;&quot;&quot;&#xA;        adds points according to user-supplied numpy arrays, for convenience and to eliminate loops&#xA;        in calling code&#xA;&#xA;        @param points: numpy array of 3d point coords -- points.shape = (npoints, 3)&#xA;        @param data: scalar-valued data belonging to each point -- data.shape = (npoints,)&#xA;        &quot;&quot;&quot;&#xA;        npts = points.shape[0]&#xA;        assert(points.shape[1] == 3)             # make sure 3d points passed in&#xA;        assert(data.shape[0] == npts) # make sure same number of data, points&#xA;&#xA;        pv = vtk.vtkPolyVertex()&#xA;        pv.GetPointIds().SetNumberOfIds(npts)&#xA;        for idx, point in enumerate(points):&#xA;            pointID = self.points.InsertNextPoint(point)&#xA;            pv.GetPointIds().SetId(idx, pointID)&#xA;            self.values.InsertNextValue(data[idx])&#xA;&#xA;        self.grid.InsertNextCell(pv.GetCellType(), pv.GetPointIds())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the calling code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def test_vtkPolyVertexCloud_writeToFile():&#xA;    &quot;&quot;&quot; adds a set of polyvertices meant to represent a finite element &quot;&quot;&quot;&#xA;    pc = vtku.VtkPolyVertCloud()&#xA;    points, data = get_random_points_and_data(10)&#xA;    pc.add_polyVertex_cell(points, data)&#xA;    pc.add_polyVertex_cell(points + 1, data)&#xA;&#xA;    # write&#xA;    fn = 'test_PolyVertexCloud.vtu'&#xA;    writer = vtk.vtkXMLUnstructuredGridWriter()&#xA;    writer.SetFileName(fn)&#xA;    writer.SetInputData(pc.grid)&#xA;    writer.Write()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Update: I took heed of the accepted answer below and did the following:&#xA;1. Performed a spatial Delaunay triangulation on each of my finite elements (the numerical solution is known at the nodes of each finite element). The triangulation is fast since there aren't that many points even for a high-order finite element.&#xA;2. Constructed a VTK file where each cell is a tetrahedron from the spatial Delaunay triangulation on each element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paraview is able to volumetrically plot this.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8tTXv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8tTXv.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="20148" LastEditDate="2017-08-16T22:35:37.100" LastActivityDate="2017-08-16T22:35:37.100" Title="Volume rendering 3D VTK *.vtu UnstructuredGrid file in Paraview" Tags="|paraview|vtk|" AnswerCount="1" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="27644" PostTypeId="2" ParentId="27643" CreationDate="2017-08-16T20:02:06.280" Score="3" Body="&lt;p&gt;It is not surprising that VTK can't generate a volume rendering for the poly_vertex cell type since there is &#xA;no topology associated with that type of cell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way finite element integration point results is often dealt with is described below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is common in finite element methods to calculate results (e.g. stresses) at the element integration points.&#xA;But, typically, for plotting purposes, the result quantities are needed at the nodes. One way to achieve this&#xA;is, for each element, to fit a parametric function to the integration point data. Then this function can be&#xA;evaluated at each of the element nodes. The exact form of this function depends on the particular element topology and the number of integration points but it is often similar to the shape functions used for the element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, since this procedure is applied element-by-element, the result function will be discontinuous when&#xA;more than one element is connected to a node. Most visualization libraries (e.g. VTK) don't directly support&#xA;discontinuous nodal results so one of the following approaches is typically used.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can simply average the element-nodal values for all the elements attached to a particular node.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can create &quot;extra&quot; nodes for each element attached to a particular node. These nodes are at the same&#xA;location as the actual node. But having duplicate nodes allows each element to have its own, unique set nodes&#xA;to which results can be attached. This is the approach used by Deal II, for example.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-08-16T20:02:06.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27645" PostTypeId="1" CreationDate="2017-08-16T20:52:39.707" Score="2" ViewCount="1667" Body="&lt;p&gt;Thank you all&lt;/p&gt;&#xA;&lt;p&gt;1.) I am trying to solve the lid-driven cavity problem for an incompressible Stokes and Navier Stokes equations using the general &amp;quot;Mixed&amp;quot; finite element method. Dirichlet boundary conditions are specified everywhere on the boundary.&lt;/p&gt;&#xA;&lt;p&gt;When I try to solve these systems of equations, my matrix showing is that it is singular. I doubt whether it is necessary to specify pressure value at some point. Otherwise how to impose the constraint &amp;quot;average pressure over the domain is zero&amp;quot;? in FEM.&lt;/p&gt;&#xA;&lt;p&gt;2.) I am trying to simulate incompressible flow over a cylinder using the mixed finite element method. Dirichlet boundary conditions are specified at the inlet, top, and bottom on a circle and do nothing on the outlet.&lt;/p&gt;&#xA;&lt;p&gt;Is it necessary to give pressure value at some point?&lt;/p&gt;&#xA;" OwnerUserId="25119" LastEditorUserId="20688" LastEditDate="2023-08-05T13:15:06.120" LastActivityDate="2023-08-05T13:15:06.120" Title="Pressure boundary condition in lid driven cavity using finite element method" Tags="|finite-element|fluid-dynamics|numerics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27646" PostTypeId="2" ParentId="27645" CreationDate="2017-08-16T21:18:09.943" Score="3" Body="&lt;p&gt;It makes sense that your matrix is singular, because the pressure is only known up to a constant. That is, if $u_h, p_h$ is a solution to your system of equations, then $u_h, p_h + C$ is also a solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see that this implies singularity, note that if the saddle point linear system is written as $A(u,p) = b$, then $u_h, p_h = (0,0)$ is a solution to the system with $b=0$. But then $u_h, p_h = (0, 1)$ also solves the system with $b=0$, hence a non trivial nullspace and a singular linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing the specifics of your scheme, it's hard to say exactly what you should do. But some general approaches you can take:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Add a row to your linear system enforcing that either the average of the mean element pressures is 0 over the domain, or that the pressure at some point in the domain is 0. Then solve the system in a least squares sense (using some iterative linear solver). This is probably the easiest approach.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rather than solving the saddle point system, use an augmented lagrangian approach where pressure and velocity are decoupled and iterate between pressure and velocity guesses until convergence. If you start with a zero-mean guess for the pressure, the mean-zero property will be preserved. See &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999110005887&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for a more detailed discussion. Decoupling pressure and velocity can have computational advantages as well, since it reduces the size of the linear systems you have to solve. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rather than solve the saddle point system, use a projection method to decouple pressure and velocity (see Chorin, Guermond papers). In this case, you will have a singular pressure correction equation poisson equation, where you can apply a penalization method to remove the singularity. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="20148" LastEditDate="2017-08-16T21:23:33.057" LastActivityDate="2017-08-16T21:23:33.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27647" PostTypeId="2" ParentId="27645" CreationDate="2017-08-16T21:27:20.617" Score="2" Body="&lt;p&gt;Please check &lt;a href=&quot;https://scicomp.stackexchange.com/a/26627/21916&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Zero mean pressure space is used for convenience when one is interested in FEA theory (basically, we cannot enforce $p(x_0) = p_0$ for $p \in \mathbb L^2$ since it does not make sense); from the computational point of view, it is easier to fix one of the pressure DOFs (although you can subtract mean value at the post–processing step if you want to). When you are working w/ polynomial spaces—and this is exactly what you do in FEM—it is perfectly fine to enforce $p(x_0) = p_0$. &lt;strong&gt;Handle this constraint like you usually handle Dirichlet BCs&lt;/strong&gt; (e.g., via modifying your matrix).&lt;br&gt;&#xA;It is also fine to ignore this constraint in some cases (e.g., Krylov solvers can do fine with this).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In this case your matrix will be well-posed. Neumann condition fixes the issue with pressure defined up to a constant.    &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-08-16T21:27:20.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27648" PostTypeId="2" ParentId="27645" CreationDate="2017-08-17T00:42:15.993" Score="1" Body="&lt;p&gt;When you say &lt;strong&gt;using general &quot;Mixed&quot; finite element method&lt;/strong&gt;, I assume that you are using the coupled velocity-pressure formulation. Correct me if I am wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have not specified whether you are using an LBB-stable element or a stabilised formulation. Nevertheless, irrespective of whether you use a stabilised formulation or LBB-stable element, when you apply Dirichlet BCs using elimination approach, it would help to fix the pressure at one point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you use the stabilised formulation, you may not face any convergence issues if you do not fix pressure at one node for some examples. But, when you use LBB-stable elements, you &lt;strong&gt;have to&lt;/strong&gt; fix pressure at one node; you will have convergence issues if you don't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using the elimination approach to apply Dirichlet BCs, then you can simply eliminate the corresponding row and column of the fixed pressure node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the example problems you are interested in, I suggest to refer to this excellent &lt;a href=&quot;http://ukacm.org/wp-content/uploads/phdThesis/2004_WDettmer.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PhD thesis&lt;/a&gt; on stabilised finite element formulations for incompressible fluid flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The text book by &lt;a href=&quot;http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471496669.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Donea and Huerta&lt;/a&gt; is also an excellent reference on mixed formulations for incompressible fluid flow.&lt;/p&gt;&#xA;" OwnerUserId="24915" LastActivityDate="2017-08-17T00:42:15.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27649" PostTypeId="1" AcceptedAnswerId="27748" CreationDate="2017-08-17T07:59:18.080" Score="5" ViewCount="3488" Body="&lt;p&gt;I am looking for different approaches to extract 3D polylines out of Point Clouds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are able to create these point clouds out of different data sources of real world surveys (LIDAR, RADAR, etc...) and to classify points (as metallic in common soil for example). What I am looking for is academic work/papers on algorithms with the purpose of extracting polylines out of linear features in classified Point Clouds. I am asking for academic work especially, as this is a first shot to possibly kick off my own thesis if the topic shows further potential. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="25169" LastEditorUserId="6910" LastEditDate="2017-08-30T09:04:09.583" LastActivityDate="2019-04-02T15:19:48.910" Title="Algorithms to extract trajectory lines out of 3D point clouds" Tags="|computational-geometry|computer-vision|3d|spatial-data|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27651" PostTypeId="2" ParentId="27459" CreationDate="2017-08-17T12:30:15.903" Score="1" Body="&lt;p&gt;&lt;strong&gt;The Jameson-Schmidt-Turkel (JST)&lt;/strong&gt; is an impressive high-resolution scheme for steady state simulations involving shock waves. It tries to retain a second-order spatial accuracy in the smooth flow regions and employs some kind of limiter in the shock region. There are various versions of this scheme, some of which can be reformulated as the TVD scheme &lt;a href=&quot;http://aero-comlab.stanford.edu/Papers/jst_2015_updated_07_03_2015.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Roe scheme on the other hand is a very popular scheme based on local linearization of fluxes. However, the spatial order of accuracy of the original Roe scheme is $O(1)$. If you are using a higher-order reconstruction method along with the Roe fluxes, then the solution will be higher-order accurate in space. Otherwise, Roe scheme is just an upwind scheme. The numerical dissipation can vanish at sonic as well as stagnation points for the Roe scheme &lt;a href=&quot;http://math.tifrbng.res.in/~praveen/notes/acfd2013/jst.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, coming to the questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Will the spatial order of accuracy of the ROE scheme be preserved?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes. Since JST scheme is a TVD-like scheme, it will preserve atleast first-order accuracy on the domain, unless you are using a higher-order reconstruction for the Roe simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.  Do i get a more accurate solution than just running the JST scheme from the beginning?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No. You will get a more accurate solution if you are using JST from the beginning. In your strategy, you are using a lower-order method followed by a high-resolution method. So a lot of diffusion is already being added. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt; (you may have to search online if links are broken):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A. Jameson, &lt;em&gt;The origins and Further Development of the Jameson-Schmidt-Turkel (JST) Scheme&lt;/em&gt; (&lt;a href=&quot;http://aero-comlab.stanford.edu/Papers/jst_2015_updated_07_03_2015.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;C. Praveen, &lt;em&gt;JST, SLIP and CUSP schemes&lt;/em&gt;, notes on Computational Fluid Dynamics (&lt;a href=&quot;http://math.tifrbng.res.in/~praveen/notes/acfd2013/jst.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;).  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2017-08-17T12:30:15.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27652" PostTypeId="1" AcceptedAnswerId="27657" CreationDate="2017-08-17T14:22:43.070" Score="1" ViewCount="347" Body="&lt;p&gt;I'm trying to implement finite-difference WENO method to progressively complex equations and systems.&#xA;At the moment I'm successful with singular scalar equations (constant advection and Burgers), and trying to advance to 1D system of Euler equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read that there are two approaches to apply WENO to systems of conservation laws:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Component-wise (by applying it to primitive values (rho, rho*u, e)). They say that this method is OK for simple test cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Characteristic-wise (by proper decomposition via eigen vectors of flux Jacobian etc). This method is said to be necessary for more complex flows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before trying complex characteristic decomposition, I try to implement simpler component-wise method. So far it work acceptably well on test case with contact discontinuities (i.e. only density varies while pressure and speed are constant) - it advects them with some smearing. But when I introduce even a weak shock, it generates growing instabilities near its front and computing collapses after some time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it normal behavior of the method, or should it be able to compute shocks as well (maybe with some oscillations or much smearing etc)? Maybe the problem is that I use non-splitting flux function of Lax-Friedrich?&#xA;Here is this flux function:&#xA;$$&#xA;h(a,b) = \frac{1}{2} \left( f(a) + f (b) - \alpha (b-a) \right)&#xA;$$&#xA;where $\alpha = \text{max}|\lambda|$ is maximum eigenvalue.&lt;/p&gt;&#xA;" OwnerUserId="25060" LastActivityDate="2017-08-18T06:52:50.187" Title="Does 1D component-wise Euler WENO work with shocks at all?" Tags="|finite-difference|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27653" PostTypeId="1" CreationDate="2017-08-17T14:51:45.683" Score="1" ViewCount="466" Body="&lt;p&gt;What data structure (or C++ library implementing it) is most suitable for efficient high dimensional histogramming?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an application where I need to compute something similar to a histogram in a high dimensional space ($d=10..20$). Even with a large linear bin size, there would be too many bins in total. However, I know in advance that only a tiny fraction of the bins will hold a non-zero value after finishing the computation (though it is very hard to estimate which).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, I need a &quot;sparse array&quot; which supports very fast indexing and fast updates.  There are many possible representations for sparse arrays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To save me a lot of benchmarking, I was wondering which one (or better: which particular C++ implementation) will perform &quot;the best&quot; for this task.&lt;/p&gt;&#xA;" OwnerUserId="25172" LastActivityDate="2017-08-17T16:05:51.710" Title="Data structure for efficient high dimensional histogramming" Tags="|sparse-matrix|high-dimensional|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27654" PostTypeId="2" ParentId="26920" CreationDate="2017-08-17T15:22:32.620" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How is the KDE used for calculating the new residual value here?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assume that you have already processed $n$ samples $x_1,x_2,\ldots,x_n \in \mathbb{R}^3$. Then the residual images $r_j : \mathbb{R}^2 \rightarrow \mathbb{R}$, $j=1,2,\ldots$, take the form&#xA;$$r_j(x) = m_j(x) - \sum_{i=1}^n e_i K(x-proj_j(x_i))$$&#xA;where $m_j$ is the $j$-th projection image,&#xA;where $e_i \in \{-e_d, +e_d\}$ is the emissivity for the $i$-the sample $x_i$,&#xA;where $K$ is the kernel you have chosen,&#xA;and where $proj_j$ is the coordinate of the projection of $x_i$ into the $j$-th projection image.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To incorporate a new sample $x_{n+1}$, define new residual images&#xA;$$\widetilde{r}_j^{\pm}(x) := r_j(x) - (\pm e_d) K(x-proj_j(x_{n+1}))$$&#xA;for positive and negative emissivity.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are the input values for the KDE calculation actually all sample emissitvities which were already incorporated into the residual image at the pixel position the sample gets projected to?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the end, the KDE calculation depends on all sample emissivities, yes.&lt;/p&gt;&#xA;" OwnerUserId="25163" LastEditorUserId="25163" LastEditDate="2017-08-17T21:22:23.350" LastActivityDate="2017-08-17T21:22:23.350" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27655" PostTypeId="2" ParentId="27653" CreationDate="2017-08-17T16:05:51.710" Score="1" Body="&lt;p&gt;If the index for a bin is some $d$-tuple of integers $\{k_1, \ldots, k_d\}$, one approach that might work is to come up with a hash function $h$ for bin indices and use an &lt;a href=&quot;http://www.cplusplus.com/reference/unordered_map/unordered_map/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;unordered_map&lt;/code&gt;&lt;/a&gt; from the C++ standard template library. It's hard to beat a well-tuned hash table implementation for serial performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will of course need a good hash function. The easy way to represent the bin index is to use a &lt;code&gt;std::array&amp;lt;size_t, d&amp;gt;&lt;/code&gt;; unfortunately there's no built-in hash function for arrays, so you'll have to specialize &lt;code&gt;std::hash&lt;/code&gt; for your use case. You can combine the hashes of two numbers &lt;code&gt;m&lt;/code&gt; and &lt;code&gt;n&lt;/code&gt; by computing &lt;code&gt;p * hash(m) + hash(n)&lt;/code&gt; where &lt;code&gt;p&lt;/code&gt; is a large prime number. Applying this iteratively to the whole array should give you a good hash function. You could also use &lt;code&gt;boost::hash_combine&lt;/code&gt; or &lt;code&gt;boost::hash_range&lt;/code&gt;, but boost is a big dependency. In any case, you might want to do some amount of benchmarking on realistic input to make sure that there aren't lots of hash collisions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A hash table will work fine just for storage purposes, but it'll be inadequate if you need more sophisticated spatial queries. For example, given a non-empty bin, you might want to find the $m$ nearest non-empty bins. Conventional data structures like k-d trees or octrees do very badly on high-dimensional data. I haven't used them so I can't comment on their efficiency, but X-trees and PK-trees that are supposed to handle high-dimensional input gracefully.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-08-17T16:05:51.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27656" PostTypeId="2" ParentId="27593" CreationDate="2017-08-17T22:54:35.307" Score="4" Body="&lt;p&gt;There is a standard way to deal with infinite limits in integrals.  This is explained nicely in the &lt;a href=&quot;https://github.com/stevengj/cubature#infinite-intervals&quot; rel=&quot;nofollow noreferrer&quot;&gt;README of the &lt;code&gt;cubature&lt;/code&gt; package&lt;/a&gt; (which you should use for low-but-multi-dimensional integration).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h1&gt;Infinite intervals&lt;/h1&gt;&#xA;  &#xA;  &lt;p&gt;Integrals over infinite or semi-infinite intervals is possible by a change of variables. This is best illustrated in one dimension.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;To compute an integral over a semi-infinite interval, you can perform the change of variables $x=a+t/(1-t)$:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$\int_a^\infty f(x) dx = \int_0^1 f\biggl(a+\frac{t}{1-t}\biggr) \frac{1}{(1-t)^2} dt.$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For an infinite interval, you can perform the change of variables $x=t/(1 -&#xA; t^2)$:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$\int_{-\infty}^\infty f(x) dx = \int_{-1}^1 f\biggl(\frac{t}{1-t^2}\biggr) \frac{1+t^2}{(1-t^2)^2} dt.$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Note the Jacobian factors multiplying $f(\dots)$ in both integrals, and also that the limits of the $t$ integrals are different in the two cases.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In multiple dimensions, one simply performs this change of variables on each dimension separately, as desired, multiplying the integrand by the corresponding Jacobian factor for each dimension being transformed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In your specific case where you can easily work out the $y$ integration by hand, I'd use the GSL package to compute the integral with the existing quadrature package, which can deal with infinite intervals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using&#xA;$$\int_{-\sqrt{x^2-1}}^{\sqrt{x^2-1}} \exp(-x) dy = 2 \exp(-x) \sqrt{x^2-1}$$&#xA;one then has&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_integration.h&amp;gt;&#xA;&#xA;double f(double x, void * params)&#xA;{&#xA;  return 2*exp(-x)*sqrt(x*x-1);&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;  gsl_integration_workspace * w = gsl_integration_workspace_alloc(100);&#xA;  double result, error;&#xA;&#xA;  gsl_function F;&#xA;  F.function = &amp;amp;f;&#xA;&#xA;  gsl_integration_qagiu(&amp;amp;F, 1, 1e-7, 1e-7, 100, w, &amp;amp;result, &amp;amp;error);&#xA;&#xA;  printf (&quot;result          = %.18f\n&quot;, result);&#xA;  printf (&quot;estimated error = %.18f\n&quot;, error);&#xA;  printf (&quot;intervals       = %zu\n&quot;, w-&amp;gt;size);&#xA;&#xA;  gsl_integration_workspace_free (w);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24680" LastEditorUserId="24680" LastEditDate="2017-09-25T09:16:11.547" LastActivityDate="2017-09-25T09:16:11.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27657" PostTypeId="2" ParentId="27652" CreationDate="2017-08-18T06:52:50.187" Score="1" Body="&lt;p&gt;Although the FD formulation has been used for nonlinear conservation laws [1], in principle, FV formulation is the way to go in case of flows with discontinuities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That been said, growing instability can be a result of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Incorrect time-step size.&lt;/strong&gt; Try reducing the Courant number and check if instability still persists. &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Incorrect upwinding.&lt;/strong&gt; The characteristic fields in case of a supersonic region always advect in the same direction (i.e. eigenvalues $u$, $u+a$ and $u-a$ have the same sign since $u &amp;gt; a$). However, the characteristic fields for a subsonic flow have to be correctly advected. It is recommended to go for characteristic decoupling for nonlinear equations (especially with discontinuities in the solution) [2]. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The checks you can perform:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Turn off the WENO mechanism and perform a simple first-order reconstruction keeping everything else the same. If you still get the oscillations, then you need to revisit the flux function you are employing. Try Roe or splitting solvers. You have plenty of choices of Riemann solvers if you implement FV formulation. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try reducing time-step size by reducing the Courant number. Try with the Courant number $&amp;lt; 0.3$ for a system of equations with order of accuracy greater than $3$.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Try implementing Finite Volume form of the WENO method. The reconstruction polynomials for the FV formulation are derived based on the cell averaged values unlike the cell-center values in FD [2]. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If none of this works, you may have to perform the characteristic decoupling. Refer &lt;a href=&quot;https://scicomp.stackexchange.com/questions/17477/eno-weno-component-wise-vs-characteristic-wise&quot;&gt;this answer&lt;/a&gt; on stack exchange for more discussion and details. Note that, in case of the FV formulation, you have to linearize the conservative variables vector for computing the Jacobian at a face (based on the left and the right cell average values). After that, find out the eigenvector matrix for decoupling the cons. vects. in rest of the stencil. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Answering your questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could not find a reference where FD formulation with component-wise reconstruction has been used for shock computing. Thus, I don't precisely know answer the answer to the first question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lax Friedrich flux function is more dissipative than Roe for example. So it is unlikely that you are getting oscillations because of this. However, this can be easily ruled out by implementing some other method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the best! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reference:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Jiang, G.-S., &amp;amp; Wu, C. (1999). A High-Order WENO Finite Difference Scheme for the Equations of Ideal Magnetohydrodynamics. Journal of Computational Physics, 150(2), 561–594.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Shu, C. W. (1997). Essentially Non-Oscillatory and Weighted Essentially Non-Oscillatory Schemes for Hyperbolic Conservation Laws. ICASE Report, (97 - 65)&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2017-08-18T06:52:50.187" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27659" PostTypeId="2" ParentId="27496" CreationDate="2017-08-18T12:37:49.783" Score="7" Body="&lt;p&gt;I didn't follow exactly your notation so I can't say for sure what this would look like for your example, but two suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if you really want your life made simple, check out &lt;code&gt;qutip&lt;/code&gt;, the Quantum Toolbox in Python. It has classes specifically for quantum operators and state vectors, supports tensor product of operators in different spaces and keeps track of the original shape; as a result, it can quite easily carry out partial traces as well.&lt;/li&gt;&#xA;&lt;li&gt;if you want to go on with &lt;code&gt;numpy&lt;/code&gt; (and you may have reason to: in at least one occasion in the past I have found out that &lt;code&gt;qutip&lt;/code&gt; mysteriously killed performance in a computation when changing basis to an operator), then look into &lt;code&gt;.reshape&lt;/code&gt;. Specifically, if your tensor space is of size $n\otimes m$, you'll want to use &lt;code&gt;A.reshape((n, m, n, m))&lt;/code&gt; for an operator &lt;code&gt;A&lt;/code&gt;. This should preserve order and split the different parts along different axes, which then you can sum or trace along as you need.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25184" LastActivityDate="2017-08-18T12:37:49.783" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27661" PostTypeId="2" ParentId="27571" CreationDate="2017-08-18T18:21:33.487" Score="1" Body="&lt;p&gt;First assume that the flow is adiabatic and not isothermal, since the change of pressure at the left is a very fast process and thus the flow does not have time to exchange heat enough with the surroundings to be at its same temperature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the problem is governed by the Euler equations, which can be written in conservative form as&#xA;\begin{equation*}&#xA;\dfrac{\partial}{\partial t}\begin{bmatrix} \rho \\ \rho u \\ \frac{p}{\gamma-1} + \frac{1}{2}\rho u^2 \end{bmatrix} + &#xA;\dfrac{\partial}{\partial x}\begin{bmatrix} \rho u \\ p + \rho u^2 \\ \frac{\gamma}{\gamma-1} pu + \frac{1}{2}\rho u^3 \end{bmatrix} = 0.&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we know the values of $p$, $\rho$ and $u$ at each point $i=1,\ldots,N$ of the grid at an instant $t^n$. With any of the (explicit) schemes shown in &lt;a href=&quot;http://www.astro.uu.se/~hoefner/astro/teach/ch10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.astro.uu.se/~hoefner/astro/teach/ch10.pdf&lt;/a&gt;, it is possible to compute $p$, $\rho$ and $u$ in the center points $i=2,\ldots,N-1$ at the following instant $t^{n+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question now is how to impose the boundary conditions. It can be shown that, at each point, there are three magnitudes ---the so-called Riemann invariants--- that keep their value constant along some characteristics. These invariants are&#xA;\begin{equation*}&#xA;\zeta_1 = \dfrac{p}{\rho^\gamma}, \quad \zeta_2 = u + \dfrac{2c}{\gamma-1}, \quad \zeta_3 = u - \dfrac{2c}{\gamma-1},&#xA;\end{equation*}&#xA;which propagate at speeds $u$, $u+c$ and $u-c$, respectively. Thus, in the case of a subsonic inlet (as ours), in which $0&amp;lt;u&amp;lt;c$, there are two characteristics coming from the outside of the tube ---which correspond to $\zeta_1$ and $\zeta_2$--- and one coming from the inside ---$\zeta_3$---. Since the characteristics can be seen as waves that propagate information, it is deduced that we must impose two boundary conditions at $x=0$ (pressure input and conservation of the stagnation enthalpy), whereas the other condition that determines the three unknowns ($p_1^{n+1},\rho_1^{n+1},u_1^{n+1}$) is obtained from the invariant that arrives from the inside of the tube ($\zeta_3$).&#xA;According to &lt;a href=&quot;http://inis.jinr.ru/sl/Simulation/Hirsch,_Numerical_Computation_of_Internal&amp;amp;External_Flows,1994/Hirsch,_Numerical_Computation_of_Internal&amp;amp;External_Flows,v2,1994/ch19.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://inis.jinr.ru/sl/Simulation/Hirsch,_Numerical_Computation_of_Internal&amp;amp;External_Flows,1994/Hirsch,_Numerical_Computation_of_Internal&amp;amp;External_Flows,v2,1994/ch19.pdf&lt;/a&gt;, this condition is implemented by&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Imposing a &lt;em&gt;compatibility equation&lt;/em&gt;. In the book, the compatibility equations have been linearized, but in the end is the same as imposing (in this case) that&#xA;\begin{equation*}&#xA;	\dfrac{\partial \zeta_3}{\partial t} + (u-c) \dfrac{\partial \zeta_3}{\partial x} = 0.&#xA;	\end{equation*}&#xA;Thus, the sought values of $p_1^{n+1},\rho_1^{n+1},u_1^{n+1}$ are the solutions of the following system of equations&#xA;\begin{align*}&#xA;	p_1^{n+1}-p_r(t^{n+1}) &amp;amp;= 0, \\&#xA;	c_p T_1^{n+1} + \dfrac{u^2}{2} - c_p T_r &amp;amp;= 0, \\&#xA;	\dfrac{(\zeta_3)_1^{n+1} - (\zeta_3)_1^{n}}{\Delta t} + (u_1^n-c_1^n) \dfrac{(\zeta_3)_2^{n} - (\zeta_3)_1^{n}}{\Delta x} &amp;amp;= 0.&#xA;	\end{align*}&#xA;Note that we use forward differencing in the last formula since the information comes precisely from the interior of the tube.&lt;/li&gt;&#xA;&lt;li&gt;Using an &lt;em&gt;extrapolation technique&lt;/em&gt;. In this case, $p_1^{n+1},\rho_1^{n+1},u_1^{n+1}$ are the solutions of:&#xA;\begin{align*}&#xA;	p_1^{n+1}-p_r(t^{n+1}) &amp;amp;= 0, \\&#xA;	c_p T_1^{n+1} + \dfrac{u^2}{2} - c_p T_r &amp;amp;= 0, \\&#xA;	(\zeta_3)_1^{n+1}-2(\zeta_3)_2^{n+1}+(\zeta_3)_3^{n+1} &amp;amp;= 0.&#xA;	\end{align*}&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the outlet, we have to consider two cases. If the flow is subsonic at the exit, we will have two characteristics coming from the inside of the tube (those corresponding to $\zeta_1$ and $\zeta_2$) and one coming from the outside. Thus, in this case we must impose only one boundary condition ($p_N^{n+1}=p_a$). The other two equations needed for determining $\rho_N^{n+1}$ and $u_N^{n+1}$ are obtained from ''conservation of the invariants'', that is, imposing&#xA;\begin{align*}&#xA;\dfrac{(\zeta_1)_N^{n+1} - (\zeta_1)_N^{n}}{\Delta t} + u_N^n \dfrac{(\zeta_1)_N^{n} - (\zeta_1)_{N-1}^{n}}{\Delta x} &amp;amp;= 0, \\&#xA;\dfrac{(\zeta_2)_N^{n+1} - (\zeta_2)_N^{n}}{\Delta t} + (u_N^n+c_N^n) \dfrac{(\zeta_2)_N^{n} - (\zeta_2)_{N-1}^{n}}{\Delta x} &amp;amp;= 0.&#xA;\end{align*}&#xA;(note that backward differences are used now) or&#xA;\begin{align*}&#xA;(\zeta_1)_{N-2}^{n+1}-2(\zeta_1)_{N-1}^{n+1}+(\zeta_1)_N^{n+1} &amp;amp;= 0, \\&#xA;(\zeta_2)_{N-2}^{n+1}-2(\zeta_2)_{N-1}^{n+1}+(\zeta_2)_N^{n+1} &amp;amp;= 0.&#xA;\end{align*}&#xA;However, if the flow is supersonic at the exit, we have three characteristics leaving the tube, so no information comes from the outside (choked tube) and therefore we cannot impose any boundary conditions. In that case, we should proceed in a similar way but imposing the values of the three Riemann invariants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some methods may present some spurious oscillations at the beginning. I'd suggest (i) not to impose a step $p_r(t)$ but a arctangent one (or similar) and (ii) if $v_1^{n+1}&amp;lt;0$ (or $v_N^{n+1}&amp;lt;0$) at some of the beginning instants, set it to zero or take $v_1^{n+1}=v_2^{n+1}$, otherwise the boundary conditions should change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take into account that these methods require $\Delta t/\Delta x$ to verify the CFL condition to be stable. You can also try implicit methods that are more stable but have to solve a big system of nonlinear equations at every instant. In those cases, it is important (from the performance point of view) to provide the solver an expression of the jacobian ---which is sparse, and this is also very important--- or, at least, its sparsity pattern. I do not know anything about python, but it seems for me that you were trying an implicit scheme without providing the solver any information about the jacobian.&lt;/p&gt;&#xA;" OwnerUserId="23633" LastActivityDate="2017-08-18T18:21:33.487" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27662" PostTypeId="1" CreationDate="2017-08-18T21:51:43.070" Score="2" ViewCount="295" Body="&lt;p&gt;Let's say that I have a FEM code which yields the following problem:&#xA;$$&#xA;\mathbf{A}\mathbf{x} = \mathbf{b}.&#xA;$$&#xA;In order to solve this more efficiently with an iterative method, I would like to precondition it,&#xA;$$&#xA;\mathbf{BAx} = \mathbf{Bb},&#xA;$$&#xA;using domain decomposition. This is a common application for domain decomposition methods. My question is: How do I retrieve $\mathbf{B}$ from a Schwarz domain decomposition method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the alternating Schwarz algorithm for the Poisson problem first solves this problem:&#xA;$$&#xA;-\nabla^2u=f\quad\rm{in}\ \Omega_1,\\&#xA;u_1^n=g\quad \rm{on}\ \partial\Omega_1\backslash\Gamma_1,\\&#xA;u_1^n=u_2^{n-1}|_{\Gamma_1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then solves this problem:&#xA;$$&#xA;-\nabla^2u=f\quad\rm{in}\ \Omega_2,\\&#xA;u_2^n=g\quad \rm{on}\ \partial\Omega_2\backslash\Gamma_2,\\&#xA;u_2^n=u_1^{n-1}|_{\Gamma_2}&#xA;$$&#xA;where $\Omega = \Omega_1\cup \Omega_2$ is the global domain, and $\Gamma_i$ denotes the part of the boundary which overlaps the other subdomain. By solving these equations repeatedly, I get better and better estimates for the global solutions $u_1$ and $u_2$. Let's now say I want to use this to precondition $\mathbf{Ax} = \mathbf{b}$, how do I do that? How do I get the $\mathbf{B}$ matrix mentioend earlier from these solutions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, I am also interested in additive Schwarz and multiplicative Schwarz.&lt;/p&gt;&#xA;" OwnerUserId="23932" LastActivityDate="2017-08-20T22:55:39.683" Title="How to precondition FEM problems using domain decomposition?" Tags="|finite-element|domain-decomposition|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27666" PostTypeId="1" AcceptedAnswerId="27671" CreationDate="2017-08-19T14:49:33.410" Score="6" ViewCount="2226" Body="&lt;p&gt;I tried to used f2py, Cython and Numba to make a simple loop structure be faster in python. Python implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def Python(A):&#xA;    B = 0.0&#xA;    for i in range(100):&#xA;        for j in range(100):&#xA;            for k in range(100):&#xA;                for l in range(100):&#xA;                    B = B + A[i,j,k,l]&#xA;return B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Numa implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@jit(double(double [:,:,:,:]), nopython=True)&#xA;def Numba(A):&#xA;    B = 0.0&#xA;    for i in range(100):&#xA;        for j in range(100):&#xA;            for k in range(100):&#xA;                for l in range(100):&#xA;                    B = B + A[i,j,k,l]&#xA;return B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Cython implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cdef double Cython(double [:,:,:,:] A):&#xA;    cdef double B&#xA;    cdef int i, j, k, l&#xA;    B = 0.0&#xA;    for i in range(0,100):&#xA;        for j in range(0,100):&#xA;            for k in range(0,100):&#xA;                for l in range(0,100):&#xA;                    B = B + A[i,j,k,l]&#xA;return B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;f2py implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SUBROUTINE F2PY_run(A,B)&#xA;    INTEGER :: i, j, k, l&#xA;    REAL(8), DIMENSION(100,100,100,100), INTENT(in) :: A&#xA;    REAL(8), INTENT(out) :: B&#xA;&#xA;    B = 0.0d0&#xA;    DO i=1, 100&#xA;        DO j=1, 100&#xA;            DO k=1, 100&#xA;                DO l=1,100&#xA;                   B = B + A(l,k,j,i)&#xA;                END DO&#xA;            END DO&#xA;        END DO&#xA;    END DO&#xA;END SUBROUTINE F2PY_run&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Cython compilation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;setup(ext_modules=cythonize(Extension('runtest',['runtest.pyx'])), include_dirs=[numpy.get_include()])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;f2py compilation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f2py -c runFortran.F90 --f90flags=-O3 -m fortranrun&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The functions was run by passing a np.random.rnd(100,100,100,100) into the functions. The timings were as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;22.998 sec; Python &#xA; 0.114 sec; Cython &#xA; 2.678 sec; f2py   &#xA; 0.106 sec; numba  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that for this specific problem I have set up, I can use np.einsum(). np.einsum() had a timing of 0.061 sec. But for my &quot;real&quot; code the problem cannot be solved with einsum. It can be seen that Cython and Numba executes at about the same speed, whereas f2py is much slower. The f2py function is written in Fortran90, so I would have thought it would have been atleast as fast. My question is thus, am I using f2py wrongly? Or does f2py have speed problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 1&lt;/strong&gt;&#xA;By suggestion from Stefano M, I made a F2PY perfermance report.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                  /-----------------------\ &#xA;                 &amp;lt; F2PY performance report &amp;gt;  &#xA;                  \-----------------------/ &#xA;Overall time spent in ...&#xA;(a) wrapped (Fortran/C) functions           :      105 msec &#xA;(b) f2py interface,                1 calls  :     2925 msec&#xA;(c) call-back (Python) functions            :        0 msec&#xA;(d) f2py call-back interface,      0 calls  :        0 msec&#xA;(e) wrapped (Fortran/C) functions (acctual) :      105 msec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As it can be seen almost all the time is used in the interface! Here is how to function is called:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import time&#xA;import fortranrun&#xA;&#xA;A = np.random.rand(100,100,100,100)&#xA;&#xA;start = time.time()&#xA;B = fortranrun.f2py_run(A)&#xA;print(time.time()-start, B, 'f2py')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25082" LastEditorUserId="25082" LastEditDate="2017-08-19T21:54:44.860" LastActivityDate="2017-08-19T23:05:56.173" Title="Loop optimization with f2py, Cython and Numba" Tags="|python|fortran|performance|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="27667" PostTypeId="2" ParentId="27662" CreationDate="2017-08-19T17:57:27.877" Score="2" Body="&lt;p&gt;Let us consider the abstract linear problem&#xA;$$ \mathcal{A} x = b \,, $$&#xA;where $\mathcal{A}$ is a linear operator and $x$ and $b$ some functions on a&#xA;certain domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question let me first discuss, what a preconditioner is&#xA;supposed to do. A preconditioner is choosen such that&#xA;$\mathcal{B} \approx \mathcal{A}^{-1}$, implying that&#xA;$\mathcal{B} \mathcal{A} \approx I$, which is a system that is easy to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to note that&#xA;a preconditioner should compute approximations to&#xA;$\mathcal{A}^{-1} b$ for&#xA;&lt;em&gt;any&lt;/em&gt; possible right hand side $b$.&#xA;The question now is, what corresponds to $b$ in your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a general situation $b$ is the input of the problem you are solving,&#xA;i.e. $b$ consists of $f$ and the boundary function $g$. Let us for simplicity,&#xA;however, assume that $g=0$, and therefore, the input of the preconditioner is&#xA;only the function $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this situation $\mathcal{B} b$&#xA;is the result of performing a few steps of&#xA;the alternating Schwarz procedure with right hand side $f=b$,&#xA;starting with some (prefarably zero) initial iterate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, if you want to obtain the matrix $B$, you have to write down the matrix&#xA;that represents this procedure on your space of piecewise polynomial&#xA;functions. The right hand side $b$ ($ = f$), will be some some piecewise&#xA;polynomial function.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2017-08-19T17:57:27.877" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27668" PostTypeId="1" AcceptedAnswerId="27681" CreationDate="2017-08-19T20:33:23.697" Score="1" ViewCount="238" Body="&lt;p&gt;I'm trying to apply finite-difference characteristic-wise WENO method to 1D Euler equations:&#xA;$$&#xA;U_t + F(U)_x = 0, \; \;\text{where} \;\; U = (\rho, \rho u, e) \;\; \text{and} \;\;F(U) = (\rho u, \rho u^2+p, (e+p)u)&#xA;$$&#xA;Physical values are set at grid points $x_j$, so we need to approximate fluxes at mid-points $x_{j\pm \frac{1}{2}}$:&#xA;$$&#xA;U_t \approx \frac{1}{h} \left[ \widehat{f}_{j+\frac{1}{2}} - \widehat{f}_{j-\frac{1}{2}} \right]&#xA;$$&#xA;where $\widehat{f}_{j\pm\frac{1}{2}}$ are sought-for approximation of physical fluxes $f(U(x_{j\pm\frac{1}{2}}))$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My algorithm is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) For each $x_{j+\frac{1}{2}}$, we calculate simple average state $U_{j+\frac{1}{2}} = \frac{1}{2} (U_j + U_{j+1})$ and, using it, local eigenvalues $(u-a, u, u+a)_{j+\frac{1}{2}}$, right eigenvector matrix $R_{j+\frac{1}{2}}$ and it's left counterpart $R^{-1}_{j+\frac{1}{2}}$. (Subscript ${j+\frac{1}{2}}$ will be omitted below)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Now we transform $U$, its differences $\Delta U$ and flux differences $\Delta F(U)$ to local characteristic field:&#xA;$$&#xA;W=R^{-1}U, \;\; \Delta W=R^{-1}\Delta U, \;\; \Delta F(W)=R^{-1}\Delta F(U)&#xA;$$&#xA;It is done only for relevant grid points which in my case ($r=2$) are $x_{j-1}, x_j, x_{j+1}, x_{j+2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Then we reconstruct characteristic variable values by WENO method using $W$, $\Delta W$ and $\Delta F(W)$. We get two &quot;candidates&quot;:&#xA;$$&#xA;\widehat{W}^- = R_j(x_{j+\frac{1}{2}}) \; \text{and} \; \widehat{W}^+ = R_{j+1}(x_{j+\frac{1}{2}})&#xA;$$&#xA;where $R_j(x), R_{j+1}(x)$ are a convex combinations of corresponding polynomial functions obtained by WENO method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step would be to compute 2-point flux function $h(\widehat{W}^-,\widehat{W}^+)$ using these reconstructed values. I'm using Lax-Friedrich's function:&#xA;$$&#xA;h(a,b) = \frac{1}{2} \left[ F(a) + F(b) + \alpha(b-a) \right]&#xA;$$&#xA;where $\alpha$ is maximum value of eigenvalue (different for each of equations, since they are decoupled now).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how to compute these $F(a)$ and $F(b)$ using $\widehat{W}^\pm$? In component-wise approach it is simple: we reconstruct base values $-$ $(\widehat{\rho}^\pm, \widehat{u}^\pm, \widehat{p}^\pm)$, for example $-$ and the use explicit formulas to compute $F(a) = (\widehat{\rho}^- \widehat{u}^-,\; \widehat{\rho}^-{\widehat{u}^-}^2 + \widehat{p}^-,\; ...)$ and $F(b) = (\widehat{\rho}^+ \widehat{u}^+,\; \widehat{\rho}^+{\widehat{u}^+}^2 + \widehat{p}^+,\; ...)$, but we don't have such formulas for $F(\widehat{W})$ because $\widehat{W}$ is expressed in local characteristic field...&lt;/p&gt;&#xA;" OwnerUserId="25060" LastEditorUserId="25060" LastEditDate="2017-08-20T06:54:02.897" LastActivityDate="2017-08-23T06:16:13.377" Title="How to compute WENO-reconstructed flux in local characteristic field?" Tags="|finite-difference|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27669" PostTypeId="1" AcceptedAnswerId="27670" CreationDate="2017-08-19T21:04:37.130" Score="0" ViewCount="543" Body="&lt;p&gt;I was using &lt;code&gt;scipy.integrate.odeint&lt;/code&gt; function , the ode is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{y\ dx - x\ dy}{(x+y)^2} + dy = dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y^2 - x^2 - y = c (x + y)\ .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving it via &lt;code&gt;odeint&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.integrate import odeint&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;def dy_dx(y, x):&#xA;  return ((x+y)**2 - y)/((x+y)**2 - x)&#xA;&#xA;xs = np.linspace(1,100,100)&#xA;y0 = 1&#xA;ys = odeint(dy_dx, y0, xs ,atol = 2)&#xA;&#xA;ys = np.array(ys).flatten()&#xA;&#xA;plt.plot(xs, ys)&#xA;plt.xlabel(&quot;x&quot;)&#xA;plt.ylabel(&quot;y&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CyZ49.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CyZ49.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which certainly doesn't match the answer. What could be going wrong?&lt;/p&gt;&#xA;" OwnerUserId="23022" LastEditorUserId="9667" LastEditDate="2017-08-20T00:28:55.490" LastActivityDate="2017-08-20T00:28:55.490" Title="``scipy.odeint`` giving different answer than analytical" Tags="|python|ode|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27670" PostTypeId="2" ParentId="27669" CreationDate="2017-08-19T22:45:37.157" Score="4" Body="&lt;p&gt;Nothing is going wrong.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Your &lt;strong&gt;differential equation&lt;/strong&gt; is:&#xA;$$\frac{\mathrm{d}y}{\mathrm{d}x} = \frac{(x+y)^2 - y}{(x+y)^2 - x}$$&#xA;Obviously if $x=y$, you have $\frac{\mathrm{d}y}{\mathrm{d}x} = 1$, i.e., $x$ grows exactly as “fast” as $y$.&#xA;Thus, if $x=y$ for your initial conditions, it will stay that way.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your initial conditions are $x=y=1$ and your &lt;strong&gt;numerical result&lt;/strong&gt; looks like $x=y$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For $c=-\tfrac{1}{2}$ and $c=\tfrac{1}{2}$, respectively, $x=y$ solves your &lt;strong&gt;analytical solutions&lt;/strong&gt; (“solved answers”).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2017-08-19T22:45:37.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27671" PostTypeId="2" ParentId="27666" CreationDate="2017-08-19T23:05:56.173" Score="9" Body="&lt;p&gt;I think that the problem is linked to the way in which f2py generates the fortran interface: the argument to &lt;code&gt;fortranrun.f2py&lt;/code&gt; should be stored as a F_CONTIGUOUS array, otherwise the interface will create an internal copy with the correct storage order.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Python 3.6.2 (default, Jul 22 2017, 21:19:22) &#xA;[GCC 7.1.1 20170516] on linux&#xA;Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.&#xA;&amp;gt;&amp;gt;&amp;gt; import numpy as np&#xA;&amp;gt;&amp;gt;&amp;gt; import fortranrun&#xA;&amp;gt;&amp;gt;&amp;gt; A = np.random.rand(100,100,100,100)&#xA;&amp;gt;&amp;gt;&amp;gt; A.flags&#xA;  C_CONTIGUOUS : True&#xA;  F_CONTIGUOUS : False&#xA;  OWNDATA : True&#xA;  WRITEABLE : True&#xA;  ALIGNED : True&#xA;  UPDATEIFCOPY : False&#xA;&amp;gt;&amp;gt;&amp;gt; A.T.flags&#xA;  C_CONTIGUOUS : False&#xA;  F_CONTIGUOUS : True&#xA;  OWNDATA : False&#xA;  WRITEABLE : True&#xA;  ALIGNED : True&#xA;  UPDATEIFCOPY : False&#xA;&amp;gt;&amp;gt;&amp;gt; b = fortranrun.f2py(A.T)&#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;                      /-----------------------\&#xA;                     &amp;lt; F2PY performance report &amp;gt;&#xA;                      \-----------------------/&#xA;Overall time spent in ...&#xA;(a) wrapped (Fortran/C) functions           :      761 msec&#xA;(b) f2py interface,                1 calls  :        0 msec&#xA;(c) call-back (Python) functions            :        0 msec&#xA;(d) f2py call-back interface,      0 calls  :        0 msec&#xA;(e) wrapped (Fortran/C) functions (acctual) :      761 msec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here you see that &lt;code&gt;A&lt;/code&gt; is C_CONTIGUOUS, while the transpose &lt;code&gt;A.T&lt;/code&gt; is F_CONTIGUOUS, and calling &lt;code&gt;b = fortranrun.f2py(A.T)&lt;/code&gt; has no f2py interface overhead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the contrary, when you call &lt;code&gt;b = fortranrun.f2py(A)&lt;/code&gt;, since &lt;code&gt;A&lt;/code&gt; is not F_CONTIGUOUS, the f2py interface will allocate scratch memory and copy the original array &lt;code&gt;A&lt;/code&gt; in the correct storage order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Memory-order is a common topic for C-FORTRAN interfaces: you should either construct &lt;code&gt;A&lt;/code&gt; as a F_CONTIGUOUS array, or pass &lt;code&gt;A.T&lt;/code&gt; to your fortran routine, but remember that you are operating on the transpose array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Warning about array argument copies can be generated with compile option &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;-DF2PY_REPORT_ON_ARRAY_COPY=1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://docs.scipy.org/doc/numpy/reference/internals.html#internal-organization-of-numpy-arrays&quot; rel=&quot;noreferrer&quot;&gt;Internal organization of numpy arrays&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1744" LastActivityDate="2017-08-19T23:05:56.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27672" PostTypeId="2" ParentId="27662" CreationDate="2017-08-20T22:55:39.683" Score="2" Body="&lt;p&gt;Don't use domain decomposition methods. They're from the 1990s, but we have much better ways of preconditioning problems today. All of them work on the &lt;em&gt;global&lt;/em&gt; problem, rather than ones on subdomains. One example are algebraic multigrid methods.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-08-20T22:55:39.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27673" PostTypeId="2" ParentId="27633" CreationDate="2017-08-21T06:59:40.573" Score="3" Body="&lt;p&gt;You just add the diffusion along the other dimensions. This superposition from orthogonal directions makes some sense, as they are independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, going by wikipedia for &lt;a href=&quot;https://en.wikipedia.org/wiki/Fick&amp;#39;s_laws_of_diffusion#Fick.27s_second_law&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fick's second law of diffusion in 1D&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial \psi}{\partial t}&#xA;=&#xA;D \frac{\partial^2 \psi}{\partial x^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We extend it to 2d as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial \psi}{\partial t}&#xA;=&#xA;D \frac{\partial^2 \psi}{\partial x^2} + D \frac{\partial^2 \psi}{\partial y^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second derivative is called the &quot;Laplacian operator&quot;, and for vector calculus (more than 1D) you may see it notated as $\nabla^2$. When applied to a scalar value, as here, it represents the sum of the partial differentials with respect to each dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to &quot;discretize&quot; this. (I might add details here later.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Math, discretization and Python code for &lt;a href=&quot;http://nbviewer.jupyter.org/github/barbagroup/CFDPython/blob/master/lessons/04_Step_3.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;1D diffusion (step 3)&lt;/a&gt;&#xA;and for &lt;a href=&quot;http://nbviewer.jupyter.org/github/barbagroup/CFDPython/blob/master/lessons/09_Step_7.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;2D diffusion (step 7)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think once you've seen the 2D case, extending it to 3D will be easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;HOWEVER&lt;/strong&gt; This diffusion won't be very interesting, just a circle (or sphere in 3d) with higher concentration (&quot;density&quot;) in the center spreading out over time - like heat diffusing through uniform metal. Although I've addressed your specific question about multi-dimensional diffusion here, from &quot;ink released from one side of a vessel&quot; and the #advection-diffusion tag it may be that you're wanting to simulate diffusion through a &lt;em&gt;fluid&lt;/em&gt;, which requires a fluid simulation, which is of course more complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case, I'd suggest looking at the full 12 steps of the above, as a start.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-08-21T12:17:39.570" LastActivityDate="2017-08-21T12:17:39.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27674" PostTypeId="1" CreationDate="2017-08-21T08:01:30.613" Score="3" ViewCount="487" Body="&lt;p&gt;I would like to solve a system of second-order differential equations to describe the dynamics of a system of particles.&#xA;Two Newton-like forces are responsible for the motion of each particle $i$: A force acting on each particle due to other particles $f_{i,j}$ and a stochastic term of noise $f_{\mathrm{noise}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The force acting on each particle due to other particles $f_{i,j}$ depends on the current position $s_i$ and velocity $v_i$ of particle $i$ and the position $s_j$ and velocity $v_j$ of the other particles $j$ of the system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F_i= f_{i,j}(v_i,v_j,s_i,s_j) + f_{\mathrm{noise}}$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two components of motion in 2D are included for every term before mentioned.&#xA;Under an Euler scheme, the velocity and position of each particle would be updated as follows:&#xA;$$&#xA;\begin{alignat}{1}&#xA;v_i &amp;amp;← v_{i} + \frac{F_i}{m} \Delta t \\&#xA;s_i &amp;amp;← s_i + v_i \Delta t&#xA;\end{alignat}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $m$ is the mass of particle $i$ and $\Delta t$ is the integration step.&#xA;However, I would like to use the Milstein’s algorithm for the velocity update (since we have a term of noise) and the fourth-order Runge-Kutta method to update the position $s_i$. I am confused due to having $f_{i,j}$ dependent on $s_i$, $s_j$, $v_i$ and $v_j$. How should I operate?&lt;/p&gt;&#xA;" OwnerUserId="25195" LastEditorUserId="7574" LastEditDate="2017-08-21T09:22:19.513" LastActivityDate="2020-07-19T18:07:10.073" Title="Runge Kutta and Milstein – system of second-order coupled differential equations with noise" Tags="|runge-kutta|stochastic|differential-equations|particle|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27675" PostTypeId="2" ParentId="27674" CreationDate="2017-08-21T10:08:57.190" Score="2" Body="&lt;p&gt;First of all, it is impossible to intertwine a multi-step Runge–Kutta method and the Milstein–Itō methods for a multitude of reasons that go beyond the scope of this question¹. So the best you can possibly do in is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Make a deterministic Runge–Kutta step, ignoring the noise term.&lt;/li&gt;&#xA;&lt;li&gt;Apply noise (in a Milstein–Itō fashion).&lt;/li&gt;&#xA;&lt;li&gt;Go to 1.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This has the problem that the Runge–Kutta step totally ignores that the noise affects the system during the step.&#xA;If your noise acts on much longer time scales than your Runge–Kutta steps, this may be acceptable, but then you should consider the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If your Runge–Kutta time steps are this small to accurately capture the deterministic forces, i.e., the deterministic forces are acting on much shorter time scales than your stochastic forces:&#xA;Do you really need noise in your model?&#xA;(Note that there are certainly applications where this makes sense; just think about whether yours is one of them.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Otherwise, i.e., if your stochastic forces are acting on the same or even smaller time scales than your deterministic forces:&#xA;Do you really gain anything from making the steps this small or use a fourth-order Runge–Kutta method in the first place?&#xA;Remember that one of the main advantages of multi-step Runge–Kutta methods is to reduce the systematic errors that are the main problem of the Euler method.&#xA;If you have a huge stochastic term anyway, those errors may be irrelevant to you.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;That being said, there now are multi-step integrators for stochastic differential equations which combine the advantages of both sides (and are generally faster), e.g., &lt;a href=&quot;http://dx.doi.org/10.1137/09076636X&quot; rel=&quot;nofollow noreferrer&quot;&gt;those proposed by Rößler&lt;/a&gt;.&#xA;Recently, &lt;a href=&quot;http://dx.doi.org/10.3934/dcdsb.2017133&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rackauckas and Nie proposed&lt;/a&gt; an &lt;a href=&quot;https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods#Adaptive_Runge.E2.80.93Kutta_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;embedded method&lt;/a&gt; based on this.&#xA;You usually do not want to implement those yourself, but there are implementations available &lt;a href=&quot;http://diffeq.sciml.ai/latest/tutorials/sde_example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;in Julia&lt;/a&gt; and &lt;a href=&quot;https://github.com/neurophysik/jitcsde&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python&lt;/a&gt; (by yours truly).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;sup&gt;¹ if you have a concise and digestible argument for this, please answer &lt;a href=&quot;https://scicomp.stackexchange.com/q/27514/7574&quot;&gt;this question of mine&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="18981" LastEditDate="2020-07-19T18:07:10.073" LastActivityDate="2020-07-19T18:07:10.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27676" PostTypeId="2" ParentId="27514" CreationDate="2017-08-21T15:20:19.670" Score="14" Body="&lt;p&gt;Let's take a stochastic differential equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ X_t = f(t,X_t)dt + g(t,X_t)dW_t $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a few different arguments which lead to intuitive understandings of why the mathematics behind the higher order methods is necessary. I will be discussing in terms of strong order, which is the same as saying &quot;for a given Brownian motion $W(t)$, how well does the numerical integral solve that trajectory?&quot;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Regularity of the Equation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;First of all, your proposed method fails to take into account the fact that $X_t$ is not continuously differentiable. Actually, you can use Rossler's results to show that extending normal RK methods like you suggested will result in convergent methods, but they will only have strong order 0.5. The reason is because they were derived using calculus with $X_t$ being differentiable and having a Taylor series. Brownian motion is not differentiable, and instead has Holder continuity of $\alpha &amp;lt; 0.5$ a.s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, like in perturbation theory, processes which are not regular enough are not expandable in terms of a Taylor series, but with Holder regularity $\alpha$ they can be expanded in terms of a Puiseux series with terms of $\alpha$, i.e. for Brownian motion there is an extension to the Taylor series notion which is expanded in terms of something like $\frac{1}{2}$ derivatives. Like in regular calculus, the first term is the &quot;linear term&quot;, i.e. change $dt$ to $\Delta t$ and $dW_t$ to $N(0,dt)$ and you get something about right. This is why the methods, including things like Euler-Maruyama, converge with strong order 0.5: they get the first term in the Taylor series correct. However, the higher order terms need to have the corrections for the fact that $X_t$ is not continuously differentiable, which is why normal methods fail to do so.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Instantaneous Correlations and Iterated Integrals&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;That's a quick heuristic explanation, but there's a bit more to it. Let's look at a few other details. A Taylor series is not just the expansion in terms of derivatives, but it can also be thought of as the number higher order terms to integrate. $X_t = X_0 + \Delta t f(t,X_t)$ is integrating once. But if you add the $dt^2$ term, to get the units right you need to do double integrals. $dt^2$ is easy to integrate twice, but what is $dW_t^i dW_t^j$? These are the instantaneous correlations between the Brownian motions. You need to know this to compute the double integral. If you're only looking at averages, you can lop this off. But in any trajectory there are correlations between the different Brownian motions of a system of differential equations. Assuming there are no correlations between the Brownian motions is another way of characterizing the Maruyama extension of deterministic methods, but to get the next term in the series (the 1.0 term) you have to get this right. The Milstein correction is precisely adding these correlation terms. When noise is diagonal, this is equivalent to understanding that there is no correlation except with itself, but correlation with ones self is just the variance which is $dt$, and so there must be a correction of $dW_t^2$ vs $dt$, i.e. $dW^2 - dt$. When there is non-diagonal noise, these double integrals have to be approximated so that the instantaneous correlations of the Brownian motions are taken into account, and the common approximation here is the Wiktorsson approximation which is what then makes non-diagonal noise simulations so complicated (since there is no analytical solution even to the double integrals).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Average Effect of Diffusion&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;But this leads us to another way of thinking about the problem. Thinking of expanding in terms of moments,  in some heuristic sense the first order term, the strong order 1.0 or $\mathcal{O}(\Delta t)$ term, must get the average movements correct, right? Here's a question: what's the derivative of $g$ in time? The easiest answer would be to define the derivative the normal way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but this isn't actually correct when putting $g$ into the context of the SDE. If we think about the derivative of $g$ in terms of how much it changes $X_t$, it's not always on average pointing in the same direction since it's always multiplied by this random factor $dW_t$. The question is: what is the average size of this $dW_t$? Diffusion has changes on average on the scale of $\sqrt{\Delta t}$, so in reality the affect that $g(t,X_t)$ has is more like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{g(t+\Delta t,X_{t+\Delta t}) - g(t,X_t)}{\sqrt{\Delta t}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can show more rigorously that the numerical derivative should be this with $X_{t + \Delta t} = X_t + g(t,X_t)\sqrt{\Delta t}$ as the &quot;predictor forward in time&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But intuitively, this is just understanding the average effect that $g$ has on the trajectory of $X_t$:  about $g(t,X_t)\sqrt{\Delta t}$. In a Runge-Kutta method, an internal step at time $c_i$ is supposed to be an approximation of the value of $X_{t + c_i\Delta t}$, but even from this quick physical heuristic argument about diffusion we see that the easy extension of a Runge-Kutta method is already wrong on average: it's wrong by about $g(t,X_t)\sqrt{c_i \Delta t}$ which is another way to explain why it's at most strong order 0.5 (it's surprising that methods still work! But you can attribute this to the fact that the sum of the stages in an RK method must be 1, and so this error is somewhat canceled out). Interestingly, this heuristic argument goes pretty deep, since higher order stochastic Runge-Kutta methods like those due to Rossler have corrections which are precisely related to $g(t,X_t)\sqrt{\Delta t}$.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Those are 3 different heuristic ways to understand why higher orders must involve stochastic calculus. Higher orders must take into account the fact that Holder regularity is 1/2 and thus there are extra terms in the Taylor series, they must take into account the instantaneous correlations, and they must at least take into account the average effects of the diffusion term. Otherwise they are doomed to not be correct to $\mathcal{O}(\Delta t)$, and instead only satisfy the &quot;linear approximation&quot; of the first term and receive $\mathcal{O}(\sqrt{\Delta t})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in some circumstances there are ways to find appropriate generalizations that do give higher order methods, but I'll leave this as a dangling thread because that's one point of a paper I'll be submitting soon. Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-08-21T21:33:02.293" LastActivityDate="2017-08-21T21:33:02.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27677" PostTypeId="2" ParentId="27674" CreationDate="2017-08-21T21:28:59.273" Score="2" Body="&lt;p&gt;You cannot take deterministic methods and slap a kind of Milstein correction on them to get higher order. Instead, you should be looking at methods which are specifically designed for integrating SDEs at high order if you want good accuracy. If pure efficiency is what matters and the noise is additive/diagonal, then the adaptive Rossler methods mentioned by @Wrzlprmft is the way to go. For non-diagonal, the other Rossler schemes do well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in many cases when talking about &quot;accurate&quot; handling of second order ODEs people usually discuss symplectic integrators and energy conservation. Those methods will not conserve energy. However, there is a method due to Burrage which does conserve energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/050646032&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://epubs.siam.org/doi/pdf/10.1137/050646032&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you want better than strong order 0.5, then as stated in the paper, &quot;A challenge is to extend the idea in a stochastic setting to construct explicit partitioned methods that approximate the stationary correlation matrix with high-order accuracy&quot;, i.e. you're on your own. I think a promising method to get there would be to extend Rossler's work to comparable Stochastic Runge-Kutta Nystrom methods and prove symplecticity conditions, but I don't think anyone has done that yet.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-08-21T21:28:59.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27678" PostTypeId="2" ParentId="26955" CreationDate="2017-08-22T01:45:46.190" Score="1" Body="&lt;p&gt;If I am reading you correctly, recall that the diffusion operator is just&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D_1 u_{yy} + D_2u_{xx} = D_1 \frac{u_{i+1,j} - 2u_{i,j} + u_{i-1,j}}{\Delta y^2} + D_2 \frac{u_{i,j+1} - 2u_{i,j} + u_{i,j-1}}{\Delta x^2} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to second order. That should give you an idea for how to write the loop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there's another way to do this. If $A$ is the diffusion operator which is a Strang matrix (-2 on the diagonal and 1 on the off diagonals), then notice that if you make $u$ a matrix where $u_{i,j}(t) = u(x_j,y_i,t)$, i.e. the each column is a separate $x$ and each row is a separate $y$ (notice the indexing swap there), then this is very easy to code up in a vectorized form. Notice that $Au$ uses the columns of $u$, so this is diffusing along $y$. Meanwhile, $uA$ is diffusing along the $x$. Thus, multiplying by $\frac{D_z}{\Delta z^2}$ for $z=x,y$ respectively,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A_yu + uA_x $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is an easy vectorized way to perform the diffusion. What's nice about this is that BLAS is multithreaded, so this will auto-parallelize in many languages like Julia, MATLAB, and Python. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-08-22T04:17:33.050" LastActivityDate="2017-08-22T04:17:33.050" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27680" PostTypeId="1" AcceptedAnswerId="27685" CreationDate="2017-08-22T07:13:10.917" Score="2" ViewCount="2941" Body="&lt;p&gt;I am looking for libraries for solving large scale  linear system (10e5) of equations using parallelization and shared memory. &#xA;1. Sparse, complex symmetric, SPD. &#xA;2. Suitable for higher order FEM ,DGFEM using Domain decomposition.&#xA;3. Suitable for iterative solvers with built in preconditioners.&#xA;4. Suitable  for C++ and/or MATLAB. &#xA;OS preferred- Ubuntu. Intel compiler. ( Paradiso does not support intel Mkl)&lt;/p&gt;&#xA;" OwnerUserId="25201" LastEditorUserId="25201" LastEditDate="2017-08-26T15:51:29.380" LastActivityDate="2017-08-28T22:45:54.043" Title="c++ Libraries for large linear system of equations" Tags="|linear-algebra|parallel-computing|mpi|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27681" PostTypeId="2" ParentId="27668" CreationDate="2017-08-22T09:01:05.133" Score="1" Body="&lt;p&gt;It seems that I was misunderstanding how the flux is reconstructed: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea was to reconstruct base values $-$ $U$ (or $W$ if we are performing characteristic decomposition) $-$ and then substitute them to formulas of $F(U)$ (or $F(W)$) to obtain reconstructed fluxes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like the correct way is to apply reconstruction procedure to $F(U)$ (or $F(W)$ which is obtained by transformation $F(W) = R^{-1}F(U)$) itself. Some quick tests I performed with this approach show very plausible results.&lt;/p&gt;&#xA;" OwnerUserId="25060" LastActivityDate="2017-08-22T09:01:05.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27683" PostTypeId="2" ParentId="27680" CreationDate="2017-08-22T17:16:49.653" Score="1" Body="&lt;p&gt;For dense linear algebra, the LAPACK and BLAS libraries are nearly always the way to go.  There are a number of C++ packages that interface to LAPACK and BLAS.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-08-22T17:16:49.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27684" PostTypeId="2" ParentId="26518" CreationDate="2017-08-22T18:11:43.847" Score="1" Body="&lt;p&gt;For LASSO Case you probably can estimate $ L $ pretty good and the best would be using FISTA for acceleration of the Proximal Gradient Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general you can use the Line Search as above (&lt;a href=&quot;https://pdfs.semanticscholar.org/c924/20f001e023c693db762758f9590571256e35.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neal Parikh and Stephen Boyd - Proximal Algorithms&lt;/a&gt; Page 20):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E1peJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E1peJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or (&lt;a href=&quot;http://www.seas.ucla.edu/~vandenbe/236C/lectures/proxgrad.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;L. Vandenberghe - Proximal Gradient Method&lt;/a&gt; Page 6-20):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uH6lu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uH6lu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect them to be equivalent, Though it seems Boyd's is easier computationally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Better and more robust choice would be &lt;a href=&quot;https://en.wikipedia.org/wiki/Augmented_Lagrangian_method#Alternating_direction_method_of_multipliers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alternating Direction Method of Multipliers&lt;/a&gt; (ADMM).&lt;/p&gt;&#xA;" OwnerUserId="7951" LastActivityDate="2017-08-22T18:11:43.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27685" PostTypeId="2" ParentId="27680" CreationDate="2017-08-22T22:19:58.577" Score="0" Body="&lt;p&gt;Eigen does not use parallelism for solving linear systems (see comment in &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=FAQ#How_does_Eigen_compare_to_BLAS.2FLAPACK.3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eigen.tuxfamily.org/index.php?title=FAQ#How_does_Eigen_compare_to_BLAS.2FLAPACK.3F&lt;/a&gt; ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://trilinos.org/capability-areas/linear-and-eigen-solvers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trilinos&lt;/a&gt; provides different linear solvers (direct, preconditioners, iterative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://pardiso-project.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pardiso&lt;/a&gt; and &lt;a href=&quot;http://mumps.enseeiht.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUMPS&lt;/a&gt; are other direct solvers that can run in parallel.&lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2017-08-22T22:19:58.577" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27686" PostTypeId="2" ParentId="20178" CreationDate="2017-08-22T22:32:28.580" Score="2" Body="&lt;p&gt;For the second-order derivative (and the Laplacian on simple domains), expressions for the discrete eigenpairs (i.e. after discretization) are available.&#xA;For example, for finite-difference, the eigenpairs are listed &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors_of_the_second_derivative#The_discrete_case&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Expression for the eigenpairs with a finite-element discretization can be found similarly (for P1 and P2 discretization).&lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2017-08-22T22:32:28.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27687" PostTypeId="1" CreationDate="2017-08-23T02:22:46.067" Score="3" ViewCount="179" Body="&lt;p&gt;Given a set of ODE's $ \dot{z} = f(z) $ (or discrete time $ z_{t+1} = f(z_t) $), is there a way to numerically find constants of motion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $ f(z_t) \approx M z_t $, diagonalizing the matrix $ M $ and finding eigenvalues equal to 1 works, but this only finds linear combinations of $ z $ that are constant. I would like to find things like radius, energy, and momentum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am specifically interested in constants of motion as I integrate Lie groups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to clarify, a constant of motion, at least for the sake of this questions, is defined as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;f: \mathbb{R}^n \rightarrow \mathbb{R} \\&#xA;\text{s.t.}~~\frac{d}{dt} f(z(t)) = 0&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for a given trajectory $ z(t) $, we would expect perhaps multiple constants of motion $ f_i(z(t)) $, but I would like to find constants of motion $ f_i $ such that they are constant for all possible trajectories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{d}{dt} f_i(z(t; z_0^{(k)})) = 0 ~~ \forall z_0^{(k)} \in \mathbb{R}^n&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerDisplayName="user23933" LastEditorDisplayName="user23933" LastEditDate="2017-08-23T16:27:09.883" LastActivityDate="2020-07-19T18:10:37.517" Title="Numerically finding constants of motion" Tags="|computational-physics|differential-equations|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27688" PostTypeId="2" ParentId="27668" CreationDate="2017-08-23T06:16:13.377" Score="2" Body="&lt;p&gt;Although @omican has found the answer, if one HAS to proceed with the characteristic decoupling of the conservative variable vector only (and not the flux vector) as asked in the original question, here is how it can be done:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of the algebraic averages as shown in point $1$ of the question, find the 'Roe linearized state' $U_{j+\frac{1}{2}} = Roe(U_j, U_{j+1})$, where $Roe()$ is the function returning Roe averaged quantities. Any standard book on CFD can be referred for this. Then proceed to finding eigensystem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Get $\widehat{W}^- = \left[ \widehat{W_1}^- \widehat{W_2}^- \widehat{W_3}^- \right] ^ T$, and $\widehat{W}^+ = \left[ \widehat{W_1}^+ \widehat{W_2}^+ \widehat{W_3}^+ \right] ^ T$. This remains the same as described in points $2$ and $3$ in the question.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now, based on the eigenvalues of the linearized (averaged) Jacobian (remember to perform Roe linearization and not take the algebraic average), construct a vector of characteristic variables at the face $\left(j+\frac{1}{2}\right)$. i.e. Consider the system to be subsonic, so that one of the eigenvalues is negative (let it be the first eigenvalue) and the remaining two are positive. So, the assembled vector of characteristic variables reads as, $\widehat{W}^* =  \left[ \widehat{W_1}^+ \widehat{W_2}^- \widehat{W_3}^- \right] ^ T$. Here, owing to the wave propagation direction, the first characteristic field is advected from right to left (-ve eigenvalue) and remaining to waves from left to right. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now find the vector of conserved variables from $\widehat{W}^*$, i.e. $U^* = R (\widehat{W}^*)$ where $R$ is the matrix of right-eigenvectors. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The numerical flux function $\widehat{f}_{j+\frac{1}{2}}$ is simply $f(U^*)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is another way of finding the numerical flux function . But this requires entropy fix since the numerical dissipation at the contact points reduces to zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the answer posted by @omican is correct if one wants to go with the Local Lax Friedrich's flux function. &lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2017-08-23T06:16:13.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27689" PostTypeId="1" AcceptedAnswerId="27694" CreationDate="2017-08-23T10:19:53.897" Score="21" ViewCount="5294" Body="&lt;p&gt;When applying the classical formula for the angle between two vectors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha = \arccos \frac{\mathbf{v_1} \cdot \mathbf{v_2}}{\|\mathbf{v_1}\| \|\mathbf{v_2}\|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one finds that, for very small/acute angles, there is a loss of precision and the result is not accurate. As explained in &lt;a href=&quot;https://stackoverflow.com/a/16996138&quot;&gt;this Stack Overflow answer&lt;/a&gt;, one solution is to use the arctangent instead:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha = \arctan2 \left(\|\mathbf{v_1} \times \mathbf{v_2}\|, \mathbf{v_1} \cdot \mathbf{v_2} \right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this indeed gives better results. However, I wonder if this would give bad results for angles very close to $\pi / 2$. Is it the case? If so, is there any formula to accurately compute angles without checking for a tolerance inside an &lt;code&gt;if&lt;/code&gt; branch?&lt;/p&gt;&#xA;" OwnerUserId="782" LastEditorUserId="127" LastEditDate="2017-09-01T15:42:04.203" LastActivityDate="2017-09-01T15:42:04.203" Title="Numerically stable way of computing angles between vectors" Tags="|algorithms|precision|numerical-limitations|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27690" PostTypeId="1" CreationDate="2017-08-23T10:25:50.187" Score="0" ViewCount="274" Body="&lt;p&gt;How to determine if a PDE is structure-preserving (energy，mass  conserved)? Are there some standards in judging the preserving-structure?  Or rather, how to derive the formulation of energy-preserved form of a PDE?  I am new to this field, I am long for receiving some references (some books,  representative papers or some experts' homepage) about these topics.   &lt;/p&gt;&#xA;" OwnerUserId="23763" LastEditorUserId="25082" LastEditDate="2017-08-23T12:59:12.937" LastActivityDate="2017-09-26T18:31:27.780" Title="How to determine a PDE which is structure-preserving (energy, mass conserved)?" Tags="|numerical-analysis|numerical-modelling|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27691" PostTypeId="1" CreationDate="2017-08-23T14:17:17.793" Score="2" ViewCount="152" Body="&lt;p&gt;I tried to simulate soliton propagation by solving the nonlinear Schrödinger equation using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Split-step_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;split-step Fourier method&lt;/a&gt;. The following is an example of the Matlab code copied from a textbook.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alpha = 0;&#xA;beta_2 = 1;&#xA;gamma = 1;&#xA;&#xA;A = N*sech(T);   % initial pulse shape&#xA;L = max(size(A));&#xA;delta_omega = 1/L/delta_T*2*pi;&#xA;omega = (-L/2:1:L/2-1)*delta_omega;&#xA;&#xA;solution = A;   % analytical solution (same as original it is periodic)&#xA;&#xA;step_num_hist = round(10.^linspace(1,3.5,20));   % total number of steps&#xA;avg_err = zeros(3,max(size(step_num_hist)));&#xA;&#xA;end_pt = 0.5*pi;   % the end of simulation, one soliton period&#xA;&#xA;for step_num = step_num_hist&#xA;    h = end_pt/step_num;   % step size&#xA;    A_t = A;&#xA;    for n=1:step_num&#xA;        A_f = fftshift(fft(A_t));     &#xA;        A_f = A_f.*exp(-alpha*(h/2)-1i*beta_2/2*omega.^2*(h/2));&#xA;        A_t = ifft(ifftshift(A_f));&#xA;&#xA;        A_t = A_t.*exp(1i*gamma*(abs(A_t).^2*h));&#xA;&#xA;        A_f = fftshift(fft(A_t));&#xA;        A_f = A_f.*exp(-alpha*(h/2)-1i*beta_2/2*omega.^2*(h/2));&#xA;        A_t = ifft(ifftshift(A_f));&#xA;    end&#xA;    comp_result = abs(A_t);&#xA;    avg_err(1,m) = sum(abs(comp_result.^2 - abs(solution).^2))/max(size(solution))/max(abs(solution).^2);   % avg relative intensity error, found in a paper&#xA;    m = m+1&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code seems to produce something that makes sense, so I don't think it is wrong. The problem is that when the step number increases (step size &lt;code&gt;h&lt;/code&gt; decreases) beyond a certain number (in my case around 1e-3), the error between the computed solution and the analytical solution doesn't decrease any further. This happens in both Matlab and Python. I also tried implementing a few higher-order methods that I found in the literature. They do converge faster, but the 'average relative intensity error' does not go below somewhere about 1e-3, while a paper I found which compares various algorithms can go down to like 1e-7 or even 1e-10 (total number of steps also of the order of 10e4). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is why is that? Why does the error decrease with step size initially but then stops decreasing after the step size gets to a certain point? What can possibly go wrong?&lt;/p&gt;&#xA;" OwnerUserId="21426" LastEditorUserId="238" LastEditDate="2017-08-24T05:24:11.063" LastActivityDate="2017-08-24T05:24:11.063" Title="Precision not improving by decreasing step-size in nonlinear Schrödinger" Tags="|matlab|python|nonlinear-equations|error-estimation|fourier-analysis|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27692" PostTypeId="2" ParentId="27687" CreationDate="2017-08-23T14:58:57.997" Score="2" Body="&lt;p&gt;If you can write down the problem using generic constants then this becomes a parameter estimation problem. The &lt;a href=&quot;http://diffeq.sciml.ai/latest/analysis/parameter_estimation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl docs&lt;/a&gt; explains a bunch of methods in more detail, but I'll focus on a simple and standard method &lt;a href=&quot;http://diffeq.sciml.ai/latest/analysis/parameter_estimation.html#Local-Optimization-Examples-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;exemplified here&lt;/a&gt;. That example is about finding biological constants (birth and death rates in Lotka-Volterra) but the idea is the same as finding constants of motion: convert the problem into an ODE with parameters and optimize the parameter values.&lt;/p&gt;&#xA;&lt;p&gt;The inner loop is essentially:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve the problem with parameters &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Compute a cost function.&lt;/li&gt;&#xA;&lt;li&gt;Have the optimization routine update the parameters &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; to hopefully better parameters.&lt;/li&gt;&#xA;&lt;li&gt;Go back to 1 if your cost is too high.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If you have some data for what the path of the differential equation should be, then you can define a cost function that is the &lt;code&gt;l2&lt;/code&gt; difference between the data and the numerical solution at each point. Minimizing this cost is the same as maximizing the Normal log-likelihood, and results in finding the parameters of the ODE which best fit the data.&lt;/p&gt;&#xA;&lt;p&gt;But this technique doesn't require data. You can define any loss function on the solution which makes sense, and have an optimizer fit it. For example, you might only know that you want the maximum value of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; at times &lt;span class=&quot;math-container&quot;&gt;$t = 1,2,3,4$&lt;/span&gt; to be &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. So you can make the cost function be&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C(u) = \sum_{t=1,2,3,4} \Vert u(t) - 1 \Vert $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and have the optimizer search for parameters which best satisfy this constraint. Of course, you get better results the better you define your cost function to match what you really want, though more complicated cost functions will put more of a strain on the optimizer.&lt;/p&gt;&#xA;&lt;p&gt;Basic optimization methods here include Levenberg-Marquardt and you'll see some tutorials using that, but you really shouldn't be using that because it's very sensitive to finding local minima and these parameter estimation problems are very tough for local optimizers. Instead, you should be using a good global optimization library like NLopt to get decent results (it'll still be difficult!)&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:10:37.517" LastActivityDate="2020-07-19T18:10:37.517" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="27694" PostTypeId="2" ParentId="27689" CreationDate="2017-08-23T15:45:21.643" Score="25" Body="&lt;p&gt;(&lt;em&gt;I have tested this approach before, and I remember it worked correctly, but I haven't tested it specifically for this question.&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell, &lt;em&gt;both&lt;/em&gt; $\|\mathbf{v}_1\times \mathbf{v}_2\|$ and $\mathbf{v}_1\cdot \mathbf{v}_2$ can suffer from catastrophic cancellation if they are almost parallel/perpendicular—atan2 can't give you good accuracy if either input is off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Start by reformulating the problem as finding the angle of a triangle with side lengths $a=|\mathbf{v}_1|$, $b=|\mathbf{v}_2|$ and $c=|\mathbf{v}_1-\mathbf{v}_2|$ (these are all accurately computed in floating point arithmetic). There is a well-known variant of &lt;a href=&quot;https://en.wikipedia.org/wiki/Heron%27s_formula&quot; rel=&quot;noreferrer&quot;&gt;Heron's formula&lt;/a&gt; due to Kahan (&lt;a href=&quot;https://people.eecs.berkeley.edu/~wkahan/Triangle.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Miscalculating Area and Angles of a Needle-like Triangle&lt;/em&gt;&lt;/a&gt;), which allows you to compute the area and angle (between $a$ and $b$) of a triangle specified by its side lengths, and do so numerically stably. Because the reduction to this subproblem is accurate as well, this approach should work for arbitrary inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quoting from that paper (see p.3), assuming $a\geq b$,&#xA;$$ \mu = \begin{cases}&#xA;c-(a-b),&amp;amp;\text{if }b\geq c\geq 0,\\&#xA;b-(a-c),&amp;amp;\text{if }c&amp;gt;b\geq 0,\\&#xA;\text{invalid triangle},&amp;amp;\text{otherwise}&#xA;\end{cases}$$&#xA;$$&#xA;\mathrm{angle} = 2\arctan\left(&#xA;\sqrt{\frac{((a-b)+c)\mu}{(a+(b+c))((a-c)+b)}}&#xA;\right)$$&#xA;All the parentheses here are placed carefully, and they matter; if you find yourself taking the square root of a negative number, the input side lengths are not the side lengths of a triangle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an explanation of how this works, including examples of values for which other formulas fail, in Kahan's paper. Your first formula for $\alpha$ is $C''$ on page 4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main reason I suggest Kahan's Heron's formula is because it makes a very nice primitive—lots of potentially tricky planar geometry questions can be reduced to finding the area/angle of an arbitrary triangle, so if you can reduce your problem to that, there is a nice stable formula for it, and there's no need to come up with something on your own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit&lt;/em&gt; Following Stefano's comment, I made a plot of relative error for $v_1=(1,0)$, $v_2=(\cos\theta, \sin\theta)$ (&lt;a href=&quot;https://gist.github.com/anonymous/e2a6427fdf4a0ba38208a72f31f603fc&quot; rel=&quot;noreferrer&quot;&gt;code&lt;/a&gt;). The two lines are the relative errors for $\theta=\epsilon$ and $\theta=\pi/2-\epsilon$, $\epsilon$ going along the horizontal axis. It seems that it works.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zqAr8.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zqAr8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-08-26T16:03:52.493" LastActivityDate="2017-08-26T16:03:52.493" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="27695" PostTypeId="2" ParentId="27687" CreationDate="2017-08-23T16:38:25.060" Score="0" Body="&lt;p&gt;Still hoping for a better answer, but here is something I have thought of (inspired by other what Christ Rackauckas said), but don't particularly like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define a parameterized &quot;constant of motion&quot; $ f_i(z(t); \theta) $ and then optimize on $ \theta $ like,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mathcal{L}(\theta_i, \vec{\lambda}) &amp;amp;= \frac{1}{2}\big(\|\theta\|^2 - 1\big) &#xA;+ \sum_{t, k} \frac{1}{2} \big( f_i(z^{(k)}(t); \theta_i) - \lambda_{ki} \big)^2&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We leave $ \lambda_{ki} $ &quot;free&quot; so that each constant for each trajectory, can still scale, but is subject to be a function of the state at each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Training for the next constant of motion could be done by adding a loss term to account for the last constant of motion.&#xA;\begin{align}&#xA;\sum_{t, k} \sum_{j=1}^{i-1} \big( f_i(z^{(k)}(t); \theta_i) - f_j(z^{(k)}(t); \theta_j) \big)^2&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't like this for the obvious reason that it's a horrible optimization problem to solve.&lt;/p&gt;&#xA;" OwnerDisplayName="user23933" LastActivityDate="2017-08-23T16:38:25.060" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27696" PostTypeId="2" ParentId="27687" CreationDate="2017-08-23T16:59:23.340" Score="2" Body="&lt;p&gt;This seems like a difficult problem to solve in general, but here's one idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you look for all constants of motion of the form&#xA;$$ \sum_{i=1}^m \lambda_i \varphi_i(t), $$&#xA;where $\lambda_i$ are some unknown coefficients, and $\{\varphi_i\}_i$ is some set of basis functions that can be computed by numerical integration of the ODE. For example, $\varphi_i$'s could be all monomials up to a certain degree, composed of $z,\dot z,\ddot z,\ldots$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the condition that $\sum_i \lambda_i \varphi_i(t)=\mathrm{const}$ could be approximated as the matrix equation&#xA;$$ \Phi \lambda = 0, \qquad \text{where }\Phi_{ij} = \varphi_j(t_i) - \varphi_j(0), $$&#xA;where $\lambda$ is a vector unknown, $\Phi$ is an $n\times m$ matrix, and $t_1,\ldots,t_n$ are some arbitrary convenient points in time ($n&amp;gt;m$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way you could find all possible coefficients $\lambda$ that generate constants of motion by asking for the numerical kernel of $\Phi$. If the constants of motions are polynomials in $z$ and its derivatives, this would work, but the matrix could be very large due to how many different monomials you can have. This does require you to guess in advance what the constants of motion might look like, I don't know if that's realistic for you.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-08-23T16:59:23.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27697" PostTypeId="2" ParentId="27690" CreationDate="2017-08-24T05:21:51.140" Score="0" Body="&lt;p&gt;A standard reference for basic topics in partial differential equations is the textbook by Lawrence C. Evans &quot;Partial Differential Equations&quot;. His chapter 8.6 discusses Noether's theorem in a very condensed manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is very broad, so it is difficult to pinpoint a specific reference. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Noether&amp;#39;s_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt; article on Noether's theorem is probably a good start, including the references given there.&lt;/p&gt;&#xA;" OwnerUserId="238" LastActivityDate="2017-08-24T05:21:51.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27698" PostTypeId="1" CreationDate="2017-08-24T07:32:56.963" Score="0" ViewCount="1032" Body="&lt;p&gt;Is it possible to solve an equation with only a single derivative such as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial U(x,t)}{\partial t} = A - BU(x,t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with finite difference methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask as I am trying to solve the below equation using a finite difference method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \vec{m}(x,t)}{\partial t} = -\frac{\partial \vec{j}_{m}}{\partial x} - \frac{\vec{m}(x,t) \times \hat{M}(x)}{\lambda_{J}^2} - \frac{\hat{M}(x) \times \left( \vec{m}(x,t) \times \hat{M}(x) \right)}{\lambda_{\phi}} - \frac{\vec{m}(x,t) - m_\infty}{\lambda_{sf}^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have implemented this with a forward difference scheme and it is unstable (even when $dt \ll \frac{dx}{2}$). I have attached an outline of the implementation if anyone wants to check this out! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/file/d/0B_PYf5uDfxwNZXloeHlmSUZvU1E/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Click Me For Difference Scheme Implementation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $\vec{j}_m$ term is called the &lt;em&gt;spin current&lt;/em&gt; and is defined in the document. It is calculated from the gradient of the &lt;em&gt;spin accumulation&lt;/em&gt; $\frac{ \partial \vec{m}(x,t)}{\partial x}$ as in the document (bottom of the page).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used Dirichlet boundary conditions for now, although will probably use Neumann BCs at a later stage. The set up would involve using a magnetic material ($\hat{M} &amp;gt; 0$) surrounded by enough non-magnetic material either side to allow the solution to decay to $0$.&lt;/p&gt;&#xA;" OwnerUserId="22832" LastEditorUserId="20688" LastEditDate="2018-08-20T15:28:25.550" LastActivityDate="2019-08-15T19:01:15.133" Title="Finite Difference Method Limitations/Stability Criteria" Tags="|finite-difference|stability|explicit-methods|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="27700" PostTypeId="1" AcceptedAnswerId="29580" CreationDate="2017-08-24T09:13:37.673" Score="2" ViewCount="143" Body="&lt;p&gt;I want to trace electrostatic field lines emerging from 2D surfaces in 3D space. Eventually I want to find their intersection with an (uncharged) mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The charge distribution $\sigma(x), x \in \Gamma \subset \mathbb{R}^3$ is defined by requiring a constant potential $\phi(x) = \phi_\Gamma\ \forall x\in\Gamma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the potential field $\phi$ and the electrostatic field $E$ we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\phi(x) = \int_\Gamma \sigma(y) \frac{1}{\|x-y\|} dy,\quad x\in\mathbb{R}^3 \\&#xA;E(x) = \nabla \phi(x) = \int_\Gamma \sigma(y) \frac{x-y}{\|x-y\|^3} dy,\quad x\in\mathbb{R}^3&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The potential function $\phi$ is harmonic ($\Delta\phi=0$), so the points on the charged surfaces are the maxima of $\phi$ and the electric field is $0$ there. To trace the field lines I place two particles with a small offset in/against normal direction. The particles have in general a non-zero $E$ field, which can be used to trace the field lines with a time stepping scheme&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x_{i+1} = x_i + E(x_i)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a surface like a plane you get two field lines in opposite directions and the field vanishes at infinity. When the surface belongs to a solid object, e.g a sphere, the potential field is constant and the electric field is $0$ on the inside, because $\phi$ is harmonic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the field line, I use an approximation $\tilde{E}$ by replacing the integral over $\Gamma$ with a finite sum over triangles $T_i$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;E(x) \approx \sum_j q_j \int_{T_j} \frac{1}{\|x-y\|} dy \approx \sum_j q_j \sum_k w_k \frac{1}{\|x-y_k\|} = \tilde{E}(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $q_j = \int_{T_j} \sigma(y) dy$ and the last sum is a gauss quadrature of the integral over the triangle $T_j$. The potential $\tilde{\phi}$ can be approximated in the same way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the problem is that as the functions are approximations, we have on the inside $\Omega \subset \mathbb{R}^3$ of the solid object &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\|\tilde{E}(x)\| &amp;gt; 0 \\&#xA;\tilde{\phi}(x) \not\equiv \text{const}, x \in \Omega&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the field $\tilde{E}$ often points outside the object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While $\|\tilde{E}\|$ inside the object is typically small, the field lines still eventually cross the surface at a step $x_i \to x_{i+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The field has a discontinuity on the surface, so $\|E\|$ is zero on the inside and large on the outside near the surface. When tracing a particle in the approximated field, you have near the surface $\tilde{E}(x_i) \ll \tilde{E}(x_{i+1}), x_i \in \Omega, \ x_{i+1} \not\in \Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the error between the position $x_k$ on an analytic field line and the position $\tilde{x}_k$ is small for $k \le i$ and huge for $k &amp;gt; i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem does not only affect solids, but creates local extrema in the space between non-closed charged surfaces as well, which leads to wrong field lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought of different ways to fix this numeric problems, but all ways seem to affect valid field lines as well:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rounding small values to $0$ can fix the field inside solids. But this will make local minima even worse.&lt;/li&gt;&#xA;&lt;li&gt;Disallowing field lines to cross surfaces. This fixes the solid problem but not the local extrema.&lt;/li&gt;&#xA;&lt;li&gt;Adding some inertia term could fix field lines getting stuck in local minima, but will worsen the problem of the non-zero field inside solids.&lt;/li&gt;&#xA;&lt;li&gt;Normalizing $E$ to $E/\|E\|$. This makes the problem much worse for errors inside solid objects and requires choosing a smaller timestep size for all steps.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The approximations, which influence the accurracy of the solution:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The the resolution of the triangulation. Can be refined if needed, but makes the calculation much more costly as calculating the field at a single point is in $\mathcal{O}(n)$. So calculating one timestep of the field lines is in $\mathcal{O}(n^2)$.&lt;/li&gt;&#xA;&lt;li&gt;The approximation of the charges. I have a piecewise constant charge on each triangle and currently no way to approximate it with higher order.&lt;/li&gt;&#xA;&lt;li&gt;The approximation of the integrals in $\int_{T_j} q_j \frac{x-y}{\|x-y\|^3}dy$. I currently use 6th order gauss quadrature.&lt;/li&gt;&#xA;&lt;li&gt;The calculation of the charges $q_j$ by requiring $\phi(x)=\phi_\Gamma \ \forall x\in\Gamma$ is approximated by requiring $\phi(x_i)=\phi_\Gamma$, where $x_i \in T_i$ are the centers of the triangles.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="14700" LastEditDate="2017-08-30T09:20:00.187" LastActivityDate="2018-06-19T13:46:15.957" Title="How to deal with numerical errors in electrostatic field calculations" Tags="|numerical-analysis|vector|electromagnetism|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="27702" PostTypeId="1" CreationDate="2017-08-24T15:58:39.690" Score="3" ViewCount="104" Body="&lt;p&gt;I have a 2d distribution of moving points with known trajectories represented in a 640x480 image. Here is the initial state:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rkmMO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rkmMO.png&quot; alt=&quot;2d points distribution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to find the minimum number of rectangles with fixed dimensions (35x42 pixels) that are able to cover all my input data, minimizing also the overlapping areas for each step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I tried to solve the problem using the hierarchical clustering algorithm cutting the dendogram when observations in each flat cluster have no greater a cophenetic distance than $t$. This approach works pretty well when points are quite scattered but, little by little they get closer and the solution gets worse because the number of overlapping rectangles increases:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gHS5c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gHS5c.png&quot; alt=&quot;image with closer points&quot;&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking to use an optimization approach but I don't know how to model the system. Do you have some idea how to do this? Or, do you know other approaches to face this problem? Any help is greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="25227" LastEditorUserId="17294" LastEditDate="2017-08-26T18:35:32.253" LastActivityDate="2017-08-26T18:35:32.253" Title="Minimize number of rectangles that cover all the points" Tags="|optimization|constrained-optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27706" PostTypeId="2" ParentId="23854" CreationDate="2017-08-24T23:11:45.610" Score="1" Body="&lt;p&gt;Polynomial smoothers as described in M. Adams, M. Brezina, J. Hu, and R. Tuminaro, &quot;Parallel multigrid smoothing: polynomial versus Gauss-Seidel&quot;, J. Comp. Physics, vol. 188, pp. 593-610 (2003).&lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2017-08-24T23:11:45.610" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27709" PostTypeId="2" ParentId="27698" CreationDate="2017-08-25T07:46:01.040" Score="1" Body="&lt;p&gt;In essence it does not matter what $\vec{j}$ means, but what it is. All worth to know is that your conservation equation is something like:&#xA;$$\frac{\partial m}{\partial t} +\textrm{div}\,\vec{j}(m) = S(m) \tag{*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The flux $\vec{j}(m)$ is a vector quantity that usually (and in this case it does) depends on the spatial difference in the variable $m$, $i.e.$ its gradient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For parabolic equations, $i.e$ where a magnitude $m$ diffusses, the flux $\vec{j}$ is measured in the inverse direction of its gradient:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{j}=-\kappa\, \vec{\textrm{grad}}\,m$$&#xA;Where the proportionality matrix $\kappa$ is definite positive, $i.e.$ $x^T\kappa x\geq 0$. If $\kappa = c i$ (being $c$ a scalar and $i$ the unit matrix) it is clear that $k&amp;gt;0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\kappa$ is negative definite, then the equation is ill-posed or simply may not have a solution. On the other hand if $\kappa$ is none of them the equation is hyperpolic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution of $(*)$ (for the sake of clarity will be treated as the equation $\partial_t m- \partial_x^2m=0$) can be discretised as follows:&#xA;$$\frac{m^{n+1}-m^n}{\Delta t}-\left(\frac{\partial^2m}{\partial x^2}\right)^n=0$$&#xA;which is solving a boundary value problem in each time step:&#xA;$$ m(x)^{n+1}=m(x)^n + \left(\frac{\partial^2m}{\partial x^2}\right)^n(x)\quad s.t.\quad  m(0) = m_0 \quad m(L) = m_L$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-08-25T08:16:24.987" LastActivityDate="2017-08-25T08:16:24.987" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27711" PostTypeId="1" AcceptedAnswerId="27712" CreationDate="2017-08-25T11:28:58.490" Score="2" ViewCount="1077" Body="&lt;p&gt;It has been discussed that Intel MKL can exhibit irreproducible behavior under certain conditions. In fact, this is a known thing and described by Intel as &lt;a href=&quot;https://software.intel.com/en-us/mkl-linux-developer-guide-getting-started-with-conditional-numerical-reproducibility&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conditional Numerical Reproducibility&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A colleague has recently encountered this issue while trying to find the eigenvalues of a matrix and he realized that the function &lt;code&gt;eigen&lt;/code&gt; in R is not deterministic on the server he was using. In particular, eigenvalues can be slightly different (&amp;lt;1e-12) and the signs of the eigenvectors are non-deterministic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tested his hypothesis by considering this matrix&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; corr &amp;lt;- structure(c(1, 0.250050163743823, 0.23435347961023, 0.482039807584937, &#xA;                0.260244618444847, 0.245304368749426, 0.486808023309575, 0.479348415857738, &#xA;                0.250050163743823, 1, 0.24648658148082, 0.246713558484002, 0.249547953702824, &#xA;                0.249424786521267, 0.247921981042718, 0.493767069248003, 0.23435347961023, &#xA;                0.24648658148082, 1, 0.0140300592972217, 0.229687680789224, 0.251843633087952, &#xA;                0.0102624595100619, 0.0160021143836412, 0.482039807584937, 0.246713558484002, &#xA;                0.0140300592972217, 1, 0.47661703967983, 0.00364166259778339, &#xA;                0.257242621615393, 0.507588614471663, 0.260244618444847, 0.249547953702824, &#xA;                0.229687680789224, 0.47661703967983, 1, 0.243407621930236, 0.482342591096063, &#xA;                0.473425275160343, 0.245304368749426, 0.249424786521267, 0.251843633087952, &#xA;                0.00364166259778339, 0.243407621930236, 1, 0.00266373582273809, &#xA;                0.00415353194451873, 0.486808023309575, 0.247921981042718, 0.0102624595100619, &#xA;                0.257242621615393, 0.482342591096063, 0.00266373582273809, 1, &#xA;                0.505031653312059, 0.479348415857738, 0.493767069248003, 0.0160021143836412, &#xA;                0.507588614471663, 0.473425275160343, 0.00415353194451873, 0.505031653312059, &#xA;                1), .Dim = structure(c(8L, 8L), .Names = c(&quot;rt&quot;, &quot;rt&quot;)), .Dimnames = structure(list(&#xA;                    rt = c(&quot;A&quot;, &quot;B&quot;, &quot;C&quot;, &quot;D&quot;, &quot;E&quot;, &quot;F&quot;, &quot;G&quot;, &quot;H&quot;&#xA;                    ), rt = c(&quot;A&quot;, &quot;B&quot;, &quot;C&quot;, &quot;D&quot;, &quot;E&quot;, &quot;F&quot;, &quot;G&quot;, &#xA;                              &quot;H&quot;)), .Names = c(&quot;rt&quot;, &quot;rt&quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;after computing its eigenvectors and eigenvalues I saw that they are indeed not the same:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; percDet &amp;lt;- 100 * mean(sapply(1:1000, function(...) {&#xA;+     identical(eigen(corr, symmetric = TRUE),&#xA;+               eigen(corr, symmetric = TRUE))}))&#xA;&amp;gt; message(&quot;% eigen determinism: &quot;, percDet, &quot;%&quot;)&#xA;% eigen determinism: 75.7%&#xA;&#xA;&amp;gt; percDet &amp;lt;- 100 * mean(sapply(1:1000, function(...) {&#xA;+     identical(eigen(corr, symmetric = TRUE)$values,&#xA;+               eigen(corr, symmetric = TRUE)$values)}))&#xA;&amp;gt; message(&quot;% eigenvalues determinism: &quot;, percDet, &quot;%&quot;)&#xA;% eigenvalues determinism: 76.3%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After some investigation I've found out that the issue was due to the Intel MKL configuration we used for R. Specifically, in our Ansible role we were configuring R as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./configure --enable-R-shlib --with-blas=\&quot;-Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -L{{ mkl_install_dir }}/compiler/lib/intel64 -lmkl_gf_lp64 -lmkl_core -lmkl_intel_thread -liomp5 -lpthread -lm\&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Changing &quot;-lmkl_gf_lp64&quot; part to &quot;-lmkl_intel_lp64&quot; as below &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./configure --enable-R-shlib --with-blas=&quot;-Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -L{{ mkl_install_dir }}/compiler/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread -liomp5 -lpthread\&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;solved the issue:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; percDet &amp;lt;- 100 * mean(sapply(1:1000, function(...) {&#xA;+ identical(eigen(corr, symmetric = TRUE),&#xA;+ eigen(corr, symmetric = TRUE))}))&#xA;&amp;gt; message(&quot;% eigen determinism: &quot;, percDet, &quot;%&quot;)&#xA;% eigen determinism: 100%&#xA;&#xA;&amp;gt; percDet &amp;lt;- 100 * mean(sapply(1:1000, function(...) {&#xA;+ identical(eigen(corr, symmetric = TRUE)$values,&#xA;+ eigen(corr, symmetric = TRUE)$values)}))&#xA;&amp;gt; message(&quot;% eigenvalues determinism: &quot;, percDet, &quot;%&quot;)&#xA;% eigenvalues determinism: 100%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;While I have found a related question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5380/intel-mkl-difference-between-mkl-intel-lp64-and-mkl-gf-lp64&quot;&gt;&quot;Intel MKL - Difference between mkl_intel_lp64 and mkl_gf_lp64&quot;&lt;/a&gt; helpful, it doesn't explain why this leads to a reproducibility issue when multiple threads are used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I'm wondering whether there is a good explanation to this behavior and why changing &quot;mkl_gf_lp64&quot; flag to &quot;mkl_intel_lp64&quot; resolves the issue.&lt;/p&gt;&#xA;" OwnerUserId="6580" LastEditorUserId="6580" LastEditDate="2017-09-04T07:44:01.127" LastActivityDate="2017-09-04T07:44:01.127" Title="The difference between mkl_intel_lp64 vs mkl_gf_lp64 in a numerical reproducibility issue with Intel MKL" Tags="|blas|r|compiling|intel-mkl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27712" PostTypeId="2" ParentId="27711" CreationDate="2017-08-25T11:54:31.253" Score="6" Body="&lt;p&gt;The various Fortran standards allow a lot of compiler dependent behaviour in terms of function binary interfaces when being called with &quot;complicated&quot; data types such as Fortran90 style arrays and complex numbers. This means calling code compiled with one compiler from another one is &lt;strong&gt;not&lt;/strong&gt; guaranteed to do what you expect, and can lead to grabbing the wrong bit of memory for a variable, thus giving spurious results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're using the intel compiler suite to build R, so you need to to link to the intel build of the fortran library for your BLAS to behave correctly. This is what the &lt;code&gt;-lmkl_intel_lp64&lt;/code&gt;, embedded inside an instruction to the linker, does (as opposed to your previous version, which was linking against a gfortran version, `libmkl_gf_lp64&quot;). Since threading makes it more likely that the wrong memory you grabbed was different between different runs, this showed up as non-deterministic behaviour, but there's no prior reason to thing that a serial run linking against the wrong library was generating the right result either.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2017-08-25T11:54:31.253" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27713" PostTypeId="1" CreationDate="2017-08-25T13:33:45.637" Score="1" ViewCount="77" Body="&lt;p&gt;For a Bounday Element Method problem I require the solution of a system of linear equations with multiple right-hand sides. Though this is a dense system, I still want to do it via Petsc &lt;strong&gt;in parallel&lt;/strong&gt;. Would the best way to do this be through something such as &lt;code&gt;-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore and importantly, I refine my grid in a regular way by adding $N-1$ points between the already present $N$ points. This leads to a new system of equations of size $N\left(N-1\right) \times N\left(N-1\right)$ in $N\left(N-1\right)$ unknowns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The upper-left matrix $A_{11}$ in this new system is identical to the unrefined matrix (multiplied by 0.5). The other three blocks $A_{12}$, $A_{21}$ and $A_{22}$ are new:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix}A_{11}&amp;amp;A_{12}\\A_{21}&amp;amp;A_{22}\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is now whether knowledge about the unrefined matrix $A_{11}$ can be used to speed up calculation of the refined system? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking, for example, about using the LU decomposition of $A_{11}$ to calculate the LU decomposition of the entire matrix $A$ using the well-known formula's&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\{\begin{aligned}&#xA;L_{21} U_{11} &amp;amp;= A_{21} \\&#xA;L_{11} U_{12} &amp;amp;= A_{12} \\&#xA;L_{22} U_{22} &amp;amp;= A_{22} - L_{21} U_{12}&#xA;\end{aligned}\right.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L_{21}$ and $U_{12}^T$ are $N-1\times N$ full matrices, $L_{11}$, $L_{22}$ are lower triangular $N\times N$ and $N-1 \times N-1$ matrices and $U_{11}$, $U_{22}$ are upper triangular with the same sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to do this in Petsc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or is there a better thing I can do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, due to the fact that $A_{21}$, $A_{12}$ and $A_{22}$ result from mesh refinement of the original system, they all look a lot like $A_{11}$: Definining&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ V = \left.\underbrace{\left[\begin{array}{cccc}&#xA;0.5 &amp;amp; 0.5 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 0.5 &amp;amp; 0.5 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0.5 &amp;amp; 0.5&#xA;\end{array}\right]}_{N}\right\} N-1&#xA;$$&#xA;an $N-1 \times N$ matrix, it can be seen that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$A_{21} \approx V A_{11} $, i.e. an average of the rows of $A_{11}$.&lt;/li&gt;&#xA;&lt;li&gt;$A_{21} \approx A_{11} V^T $, i.e. an average of the columns of $A_{11}$.&lt;/li&gt;&#xA;&lt;li&gt;$A_{22} \approx V A_{11} V^T $, i.e. an average of the rows and columns of $A_{11}$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This would therefore mean that a good approximate for $A$ would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A \approx \left(\begin{array}{c} 1_N \\ V \end{array}\right) A_{11} \left(\begin{array}{c} 1_N \\ V \end{array}\right)^T , $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from which it might be possible to construct some preconditioner easily using the LU decomposition from $A_{11}$: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left\{\begin{aligned}&#xA;L_{21} &amp;amp;\approx V L_{11} \\&#xA;U_{12} &amp;amp;\approx U_{11} V^T &#xA;\end{aligned}\right. ,&#xA;$$&#xA;from the equations above. However, this implies that $L_{22} \approx 0$ and $U_{22}\approx 0$, which I do not really understand well as it leads to non-invertible approximations for $U$ and $L$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ L \approx \left(\begin{array}{cc} L_{11} &amp;amp; 0 \\ V L_{11} &amp;amp; 0 \end{array}\right) \ \text{and} \ U \approx \left(\begin{array}{cc} U_{11} &amp;amp; U_{11} V^T \\ 0 &amp;amp; 0 \end{array}\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would this be used as a preconditioner?&lt;/p&gt;&#xA;" OwnerUserId="21549" LastEditorUserId="21549" LastEditDate="2017-08-25T14:12:57.147" LastActivityDate="2017-08-25T14:12:57.147" Title="Parallel dense solve with submatrices from mesh refinement with Petsc" Tags="|linear-solver|parallel-computing|petsc|dense-matrix|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27715" PostTypeId="2" ParentId="27680" CreationDate="2017-08-25T15:00:08.917" Score="1" Body="&lt;p&gt;Some other packages in this space that might be of use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://faculty.cse.tamu.edu/davis/suitesparse.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UMFPACK&lt;/a&gt;: unsymmetric multifrontal method, sequential but robust&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://crd-legacy.lbl.gov/~xiaoye/SuperLU/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SuperLU&lt;/a&gt;: supernodal sparse LU, has parallel variants (both sharedmem and distmem)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.tau.ac.il/~stoledo/taucs/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;TAUCS&quot;&gt;TAUCS&lt;/a&gt;: sharedmem parallel multifrontal, can exploit complex-symmetry (I think)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.myracore.com/myramath/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MyraMath&lt;/a&gt;: sharedmem parallel multifrontal, can exploit complex-symmetry, has some interesting algorithms to support substructuring/DDM (disclaimer: I authored this one)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, PARDISO is generally the fastest.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2017-08-25T15:00:08.917" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27716" PostTypeId="2" ParentId="27680" CreationDate="2017-08-25T16:21:34.113" Score="0" Body="&lt;p&gt;For sparse systems, you can use my own library OpenNL: &#xA;&lt;a href=&quot;https://gforge.inria.fr/frs/?group_id=1252&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gforge.inria.fr/frs/?group_id=1252&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has no dependency, simple API for matrix assembly, it is parallel, works also on the GPU. It has built-in iterative algorithms (CG, preCG, BiCG, GMRES) and supports sparse direct solvers (SuperLU, CHOLMOD) with a plugin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Documentation of OpenNL:&#xA;&lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/nl_8h.html#details&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/nl_8h.html#details&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2017-08-25T16:21:34.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27717" PostTypeId="2" ParentId="20641" CreationDate="2017-08-25T20:35:46.943" Score="1" Body="&lt;p&gt;I think that the problem encountered in this question is not specificly for DFT. But more in general for QM methods. You say that $E(r\rightarrow \infty) \neq0$. Now let us consider the molecular Hamiltonian:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat{H}=\hat{T}_N+\hat{T}_e+\hat{U}_{NN}+\hat{U}_{Ne}+\hat{U}_{ee}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kinetic energy of the nucleis:&#xA;$$\hat{T}_N=-\sum_i\frac{\hbar^2}{2M_i}\nabla^2_{R_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kinetic energy of the electrons:&#xA;$$\hat{T}_e=-\sum_i\frac{\hbar^2}{2m_e}\nabla^2_{r_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The potential energy between the nucleis:&#xA;$$\hat{U}_{NN}=\sum_i\sum_{i&amp;gt;j}\frac{Z_iZ_je^2}{4\pi\epsilon_0|R_i-R_j|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The potential energy between the electrons:&#xA;$$\hat{U}_{ee}=\sum_i\sum_{i&amp;gt;j}\frac{e^2}{4\pi\epsilon_0|r_i-r_j|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The potential energy between the nucleis and electrons:&#xA;$$\hat{U}_{Ne}=-\sum_i\sum_{j}\frac{Z_ie^2}{4\pi\epsilon_0|R_i-r_j|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would usually make a calculation under the clamped nuclei approximation, thus we will let the term for the kinetic energy of the nuclei fall out, $\hat{T}_N=0$. Now if we consider the Hamiltonian being an operator depending on the distance between two nucleis. We can write it as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat{H}(|R_1-R_2|)=-\sum_i\frac{\hbar^2}{2m_e}\nabla^2_{r_i}+\frac{Z_1Z_2e^2}{4\pi\epsilon_0|R_1-R_2|}+\sum_i\sum_{i&amp;gt;j}\frac{e^2}{4\pi\epsilon_0|r_i-r_j|}+\left[ -\sum_{j}\frac{Z_1e^2}{4\pi\epsilon_0|R_1-r_j|} -\sum_{j}\frac{Z_2e^2}{4\pi\epsilon_0|R_2-r_j|} \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we can consider the limit of $|R_1-R_2|\rightarrow \infty$. It can be seen in the above equation that only one term depends directly on $|R_1-R_2|$, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\lim_{|R_1-R_2|\rightarrow \infty}\frac{Z_1Z_2e^2}{4\pi\epsilon_0|R_1-R_2|}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therfore:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat{H}(|R_1-R_2|\rightarrow \infty)=-\sum_i\frac{\hbar^2}{2m_e}\nabla^2_{r_i}+\sum_i\sum_{i&amp;gt;j}\frac{e^2}{4\pi\epsilon_0|r_i-r_j|}+\left[ -\sum_{j}\frac{Z_1e^2}{4\pi\epsilon_0|R_1-r_j|} -\sum_{j}\frac{Z_2e^2}{4\pi\epsilon_0|R_2-r_j|} \right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be seen that $\hat{H}(|R_1-R_2|\rightarrow \infty) \neq 0$. It have to be noted in the above I have made no assumptions regarding the electrons placements to the nucleis. The terms that depends on the electrons will not be constant when the distance between the nucleis are increased, but will not go to zero. I did not look more explicit at these terms, to conserve simplicity of the argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expectation value will thus not go to zero for infinitly seprated atoms. As a fruit for thought, what is the energy of a single atom? What should then the energy of two atoms at infinite distance be?&lt;/p&gt;&#xA;" OwnerUserId="25082" LastActivityDate="2017-08-25T20:35:46.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27718" PostTypeId="1" CreationDate="2017-08-26T00:03:11.217" Score="0" ViewCount="619" Body="&lt;p&gt;I am trying to solve Lid driven square cavity flow problem of Stokes equation using finite element method. I have boundary conditions for velocity as zeros on every boundary but u=1 on top boundary. can you please help me with the boundary conditions of stream function and how to obtain ? especially on top boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have gone through many literature but why most of them considered as zero on every boundary.&lt;/p&gt;&#xA;" OwnerUserId="25119" LastActivityDate="2019-12-15T13:02:40.410" Title="Boundary conditions for streamlines in enclosed flow" Tags="|finite-element|numerical-analysis|fluid-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27719" PostTypeId="2" ParentId="27718" CreationDate="2017-08-26T13:22:46.147" Score="0" Body="&lt;p&gt;For the streamline formulation the divergence equation for the velocity $\vec{v}=(u,v)^{T}$ &#xA;$$\mathrm{div}\,\vec{v}=0$$&#xA;holds identically if &#xA;$$ \vec{v} =(u,v)^{T}= \left( \frac{\partial \Psi}{\partial y},-\frac{\partial \Psi}{\partial x}\right)^{T}\tag{*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For boundaries in which the velocity is $\vec{v}=\vec{0}$, it is easy to see from $(*)$ that $\Psi=constant$ and $\vec{n}\cdot\vec{\mathrm{grad}}\,\Psi=0$. Without any loss of generality you can set this constant to $0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, on the top of the cavity in which $\vec{v}=(u_0,0)^{T}$ the streamline function $\Psi$ must fulfill:&#xA;$$ \frac{\partial\Psi}{\partial y}=v_0\qquad \frac{\partial\Psi}{\partial x}=0 $$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-08-26T13:53:47.710" LastActivityDate="2017-08-26T13:53:47.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27721" PostTypeId="1" CreationDate="2017-08-26T14:59:24.857" Score="1" ViewCount="429" Body="&lt;p&gt;After successful implementation of characteristic-wise finite-difference WENO method to 1D Euler equations, I'm moving to 2D equations on cartesian grid:&#xA;$$&#xA;\frac{\partial U}{\partial t} + \frac{\partial F(U)}{\partial x} + \frac{\partial G(U)}{\partial y} = 0&#xA;$$&#xA;where&#xA;$$&#xA;U = \left( \begin{array}{c} \rho \\ \rho u \\ \rho v \\ e\end{array} \right), \quad &#xA;F(U) = \left( \begin{array}{c} \rho u \\ \rho u^2 + p \\ \rho uv \\ (e+p)u\end{array} \right), \quad&#xA;G(U) = \left( \begin{array}{c} \rho v \\ \rho uv \\ \rho v^2 + p \\ (e+p)v\end{array} \right), \quad&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Shu's guidebook, numerical method for 2D case is quite simply derived from 1D case:&#xA;$$&#xA;\frac{dU_{ij}}{dt} = - \frac{1}{\Delta x} \left( \hat{f}_{i+\frac{1}{2}, j} - \hat{f}_{i-\frac{1}{2}, j} \right) - \frac{1}{\Delta y} \left( \hat{g}_{i, j+\frac{1}{2}} - \hat{g}_{i, j-\frac{1}{2}} \right)&#xA;$$&#xA;where $\hat{f},\hat{g}$ are found by 1D WENO method using corresponding eigenmatrices for $F(U)$ and $G(U)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrices I found in literature are (notation used: $q = \frac{u^2+v^2}{2},\; a=\sqrt{\frac{\gamma p}{\rho}}, \; h=\frac{a^2}{\gamma-1} + q$):&#xA;$$&#xA;J_f = \frac{\partial F(U)}{\partial U} = \left( \begin{array}{cccc}&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;(\gamma-1)q-u^2 &amp;amp; (3-\gamma)u &amp;amp; (1-\gamma)v &amp;amp; \gamma-1\\&#xA;-uv &amp;amp; v &amp;amp; u &amp;amp; 0\\&#xA;\Big((\gamma-1)q-h\Big)u &amp;amp; h-(\gamma-1)u^2 &amp;amp; (1-\gamma)uv &amp;amp; \gamma u&#xA;\end{array}\right)&#xA;$$&#xA;$$&#xA;R_f = \left( \begin{array}{cccc}&#xA;1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0\\&#xA;u-a &amp;amp; u &amp;amp; u+a &amp;amp; 0\\&#xA;v &amp;amp; v &amp;amp; v &amp;amp; -1\\&#xA;h-au &amp;amp; q &amp;amp; h+au &amp;amp; -v&#xA;\end{array}\right), \quad \lambda_f=(u-a,\;u,\;u+a,\;u)&#xA;$$&#xA;$$&#xA;L_f = R^{-1}_f = \left( \begin{array}{cccc}&#xA;\frac{(\gamma-1)q+au}{2a^2} &amp;amp; \frac{(1-\gamma)u-a}{2a^2} &amp;amp; \frac{(1-\gamma)v}{2a^2} &amp;amp; \frac{(\gamma-1)}{2a^2}\\&#xA;\frac{a^2-(\gamma-1)q}{a^2} &amp;amp; \frac{(\gamma-1)u}{a^2} &amp;amp; \frac{(\gamma-1)v}{a^2} &amp;amp; \frac{(1-\gamma)}{a^2}\\&#xA;\frac{(\gamma-1)q-au}{2a^2} &amp;amp; \frac{(1-\gamma)u+a}{2a^2} &amp;amp; \frac{(1-\gamma)v}{2a^2} &amp;amp; \frac{(\gamma-1)}{2a^2}\\&#xA;v &amp;amp; 0 &amp;amp; -1 &amp;amp; 0&#xA;\end{array}\right)&#xA;$$&#xA;These are used to calculate fluxes over x-direction ($\hat{f}$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also:&#xA;$$&#xA;J_g = \frac{\partial G(U)}{\partial U} = \left( \begin{array}{cccc}&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\&#xA;-uv &amp;amp; v &amp;amp; u &amp;amp; 0\\&#xA;(\gamma-1)q-v^2 &amp;amp; (1-\gamma)u &amp;amp; (3-\gamma)v &amp;amp; \gamma-1\\&#xA;\Big((\gamma-1)q-h\Big)v &amp;amp; (1-\gamma)uv &amp;amp; h-(\gamma-1)v^2 &amp;amp; \gamma v&#xA;\end{array}\right)&#xA;$$&#xA;$$&#xA;R_g = \left( \begin{array}{cccc}&#xA;1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0\\&#xA;u &amp;amp; u &amp;amp; u &amp;amp; 1\\&#xA;v-a &amp;amp; v &amp;amp; v+a &amp;amp; 0\\&#xA;h-av &amp;amp; q &amp;amp; h+av &amp;amp; u&#xA;\end{array}\right), \quad \lambda_g=(v-a,\;v,\;v+a,\;v)&#xA;$$&#xA;$$&#xA;L_g = R^{-1}_g = \left( \begin{array}{cccc}&#xA;\frac{(\gamma-1)q+av}{2a^2} &amp;amp; \frac{(1-\gamma)u}{2a^2} &amp;amp; \frac{(1-\gamma)v-a}{2a^2} &amp;amp; \frac{(\gamma-1)}{2a^2}\\&#xA;\frac{a^2-(\gamma-1)q}{a^2} &amp;amp; \frac{(\gamma-1)u}{a^2} &amp;amp; \frac{(\gamma-1)v}{a^2} &amp;amp; \frac{(1-\gamma)}{a^2}\\&#xA;\frac{(\gamma-1)q-av}{2a^2} &amp;amp; \frac{(1-\gamma)u}{2a^2} &amp;amp; \frac{(1-\gamma)v+a}{2a^2} &amp;amp; \frac{(\gamma-1)}{2a^2}\\&#xA;-u &amp;amp; 1 &amp;amp; 0 &amp;amp; 0&#xA;\end{array}\right)&#xA;$$&#xA;These are used to calculate fluxes over y-direction ($\hat{g}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correctness of the matrices have been tested numerically: they satisfy $&#xA;LR=E$, $LJR = \Lambda = \text{diag}(\lambda_i)$ for both $F$ and $G$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the code for $x$-wise and $y$-wise WENO reconstruction is very symmetrical: just substitute correct matrices and use array differences along [j] index instead of [i].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my problem: when I test the program on simple stationary shock normal to $x$-axis, everything is OK. But for similar test for $y$-axis, some instability grows in the uniform-state area ahead of the shock. I still can't find any asymmetry in my code; is it possible that I'm using matrices incorrectly? Maybe the order of eigenvalues and eigenvectors should be different $(\lambda_y=(v-a,\; v,\; v,\; v+a)$, for example)?&lt;/p&gt;&#xA;" OwnerUserId="25060" LastEditorUserId="25060" LastEditDate="2017-08-26T18:59:33.353" LastActivityDate="2018-10-29T15:01:05.910" Title="(FD WENO) Correct characteristic decomposition of 2D Euler equations" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ClosedDate="2018-10-30T15:41:19.763" ContentLicense="CC BY-SA 3.0" />
  <row Id="27722" PostTypeId="1" AcceptedAnswerId="27725" CreationDate="2017-08-26T16:30:29.523" Score="1" ViewCount="278" Body="&lt;p&gt;I can't seem to find a good algorithm for the one-to-exactly-two assignment problem. Good algorithms are known for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Assignment_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;classical assignment&lt;/a&gt; problem, where N tasks need to be assigned to to M agents in a one-to-one correspondence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case of the one-to-exactly-two assignment problem, I have N tasks and M agents. However, each tasks can only be solved if two agents are assigned to it. Similar to the classical assignment problem, the goal is to minimize the cost, given by a cost matrix $C_{ij}$. Here assigning task $i$ to agent $j$ costs an amount $C_{ij}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas how this can be solved efficiently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already considered the review by Pentico, D. &lt;a href=&quot;https://www.ida.liu.se/~TDDD10/papers_to_read/survey.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Assignment Problems: A Golden Anniversary Survey'&lt;/a&gt;, but could not find my problem there.&lt;/p&gt;&#xA;" OwnerUserId="8329" LastEditorUserId="8329" LastEditDate="2017-08-26T19:35:18.507" LastActivityDate="2017-11-01T18:11:19.833" Title="Algorithms for the one-to-two assignment problem" Tags="|algorithms|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27723" PostTypeId="2" ParentId="27721" CreationDate="2017-08-26T19:17:12.897" Score="0" Body="&lt;p&gt;After 10th or 20th code sweep I finally found the bug in it.&#xA;Quick tests show that everything is OK now, method works nice.&#xA;Let this question hang in here for as a help for future googlers of 2D Euler decomposition.&lt;/p&gt;&#xA;" OwnerUserId="25060" LastActivityDate="2017-08-26T19:17:12.897" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27725" PostTypeId="2" ParentId="27722" CreationDate="2017-08-26T21:01:33.450" Score="0" Body="&lt;p&gt;Not sure if that's a good solution but will post anyways:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume that we can form a bipartite graph and can solve the one-to-one assignment problem. We will start by assigning the first task in the classical fashion: every node will be assigned to exactly one node in the target. We can then remove the assigned links (edges) from the graph, because these already exist in the previous solution (e.g. we can set the assignment costs to infinity). We could then re-run the assignment algorithm, i.e. Hungarian, and generate the next assignments, which are guaranteed to be different than the first. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this algorithm will probably result in a sub-optimal solution as it looks greedy. Yet, the first solution is always the optimal of the classic assignment problem and it might be a good simple start because this approach allows you to benefit from the existing solvers. Complexity wise it's just $2O(\text{Hungarian})$, which is very acceptable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a similar manner, one could use auction algorithms, which might reduce the runtime to $2O(N^2)$.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2017-11-01T18:11:19.833" LastActivityDate="2017-11-01T18:11:19.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27734" PostTypeId="1" AcceptedAnswerId="27736" CreationDate="2017-08-28T21:57:40.550" Score="1" ViewCount="621" Body="&lt;p&gt;I have data for a random variable and I wish to test whether it conforms to the Tracy-Widom distribution. However, the T-W distribution is hard to compute. Is there a readly available table of its values I can just download?&lt;/p&gt;&#xA;" OwnerUserId="25266" LastActivityDate="2017-08-29T22:53:05.177" Title="Data for Tracy-Widom distribution" Tags="|statistics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27735" PostTypeId="2" ParentId="27680" CreationDate="2017-08-28T22:45:54.043" Score="1" Body="&lt;p&gt;This page offers comparisons between various linear solvers&#xA;&lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.netlib.org/utk/people/JackDongarra/la-sw.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a distributed, C++ based solver which can handle SPD specifically for sparse matrices along with built-in preconditioning, I would strongly recommend &lt;a href=&quot;http://libelemental.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elemental&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2017-08-28T22:45:54.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27736" PostTypeId="2" ParentId="27734" CreationDate="2017-08-29T03:26:49.893" Score="5" Body="&lt;p&gt;It's actually not all that complicated to calculate the Tracy-Widom CDF just from its definition: see &lt;a href=&quot;https://arxiv.org/abs/0804.2543&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;On The Numerical Evaluation Of Fredholm Determinants&lt;/em&gt; by Folkmar Bornemann&lt;/a&gt;. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Tracy%E2%80%93Widom_distribution&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia page&lt;/a&gt; gives the definition as&#xA;$$ F_2(s) = \det(I - A_s), $$&#xA;where $A_s$ is the integral operator on $[s,\infty)$ with the kernel&#xA;$$ &#xA;\DeclareMathOperator{\Ai}{Ai}&#xA;K(x,y) = \frac{\Ai(x)\Ai'(y)-\Ai'(x)\Ai(y)}{x-y}, $$&#xA;$$ A_s[f](x) = \int_s^\infty K(x,y)f(y)\,\mathrm{d}y. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just going from this, you can take the weights $w_i$ and nodes $x_i$ of any good quadrature rule on the interval $[s,\infty)$, such as the excellent &lt;a href=&quot;https://en.wikipedia.org/wiki/Tanh-sinh_quadrature&quot; rel=&quot;noreferrer&quot;&gt;tanh-sinh quadrature&lt;/a&gt;, and compute the determinant of the matrix&#xA;$$ \det\big( \delta_{ij} - w_i^{1/2} K(x_i, x_j) w_j^{1/2}\big)_{i,j}. $$&#xA;This works because the kernel is smooth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make sure this actually works, I wrote the code below (using Julia 0.6): there is a little bit of inaccuracy at $F_2(-8)$, but it's fine otherwise. Most of the time is spent calculating Airy functions, so it should be okay for generating tables of values for future use.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;module TracyWidom&#xA;&#xA;using SpecialFunctions&#xA;using Base.Test&#xA;&#xA;function K(x, y)&#xA;  try&#xA;    if x == y&#xA;      airyaiprime(x)^2 - x * airyai(x)^2&#xA;    else&#xA;      (airyai(x)*airyaiprime(y) - airyaiprime(x)*airyai(y)) / (x - y)&#xA;    end&#xA;  catch ex # underflow is not an error here&#xA;    if isa(ex, SpecialFunctions.AmosException) &amp;amp;&amp;amp; ex.id == 4&#xA;      return 0.0&#xA;    else&#xA;      rethrow(ex)&#xA;    end&#xA;  end&#xA;end&#xA;&#xA;function tanhSinhNodes(s, n=64)&#xA;  tf = 3.158587875883174 # fzero(t -&amp;gt; log1p(-tanh(pi/2*sinh(t))) - log(eps(typeof(s))), 3.1)&#xA;  t = linspace(-tf, tf, 2n+1)&#xA;  u = tanh.(pi/2*sinh.(t))&#xA;  w = pi/2*cosh.(t)./cosh.(pi/2*sinh.(t)).^2&#xA;  w * step(t) .* 2./(u-1).^2, s + (u+1) ./ (1-u)&#xA;end&#xA;&#xA;function tanhSinh(f, s, n=64)&#xA;  w, x = tanhSinhNodes(s, n)&#xA;  dot(w, f.(x))&#xA;end&#xA;&#xA;function F2(s, n=128)&#xA;  w, x = tanhSinhNodes(s, n)&#xA;  m = length(w)&#xA;  det([Int(i==j) - sqrt(w[i])*sqrt(w[j])*K(x[i], x[j]) for i=1:m, j=1:m])&#xA;end&#xA;&#xA;function test()&#xA;  @testset &quot;TracyWidom&quot; begin&#xA;    @testset &quot;tanh-sinh&quot; begin&#xA;      @test tanhSinh(x -&amp;gt; 1/(1+x^2), 0., 32) ≈ π/2&#xA;      @test tanhSinh(x -&amp;gt; 1/(1+x^2), 1., 32) ≈ π/2 - atan(1.)&#xA;      @test tanhSinh(x -&amp;gt; (1+x^2)^-1.25, 2.0, 32) ≈ 0.20885622523501382&#xA;    end&#xA;&#xA;    @testset &quot;F2&quot; begin&#xA;      @test F2(0.1) ≈ 0.9754704606594619&#xA;      @test F2(-0.1) ≈ 0.9620142937272265&#xA;      @test F2(-1.0) ≈ 0.8072142419992853&#xA;      @test F2(-2.0) ≈ 0.41322414250512257&#xA;      @test F2(-3.0) ≈ 0.08031955293933454&#xA;      @test F2(-4.0) ≈ 0.0035445535955092003&#xA;      @test F2(-5.0) ≈ 2.135996984741116e-5&#xA;      @test F2(-6.0) ≈ 1.062254674124451e-8&#xA;      @test F2(-7.0) ≈ 2.639614767246062e-13&#xA;      @test F2(-8.0) ≈ 1.9859004257636574e-19&#xA;&#xA;      @test F2(1.0) ≈ 0.9975054381493893&#xA;      @test F2(2.0) ≈ 0.9998875536983092&#xA;      @test F2(3.0) ≈ 0.9999970059566077&#xA;      @test F2(4.0) ≈ 0.9999999504208784&#xA;      @test F2(5.0) ≈ 0.9999999994682207&#xA;      @test F2(6.0) ≈ 0.9999999999961827&#xA;      @test F2(7.0) ≈ 0.9999999999999811&#xA;      @test F2(8.0) ≈ 0.9999999999999999&#xA;    end&#xA;  end&#xA;  nothing&#xA;end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-08-29T04:26:11.630" LastActivityDate="2017-08-29T04:26:11.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27737" PostTypeId="1" CreationDate="2017-08-29T05:04:14.000" Score="5" ViewCount="2943" Body="&lt;p&gt;I am a newbie in finite difference methods, so I apologize in advance if the question is trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve the advenction equation, i.e.&#xA;$\frac{\partial \phi(x,t)}{\partial t} + v \frac{\partial \phi(x,t)}{\partial x}=0$, where $v$ is a velocity, with the finite difference method. The system is of course also solvable analytically, and it turns out that every function of the kind $f(x,t) = f(x-vt)$ is a solution. Therefore, given an arbitrary initial condition $\phi(x,0)$, the &quot;shape&quot; of $\phi(x,0)$ will move towards positive x (for $v&amp;gt;0$) or negative x (for $v&amp;lt;0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first formulated the problem with the backward formula for the spatial derivative, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi(x_i,t_{j+1}) = \phi(x_i,t_j) - v\cdot\Delta T\cdot(\phi(x_i,t_j)-\phi(x_{i-1},t_j))/\Delta x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial condition is given by a Gaussian packet centered at the origin,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi(x_i,t_1) = exp(-(x_i/2\sigma)^2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $v&amp;gt;0$ the gaussian packet does indeed move towards right with increasing times, and maintaining always the same shape. However, if I set $v&amp;lt;0$, I obtain large instabilities, with an exponentially increasing amplitude. If I reformulate the problem in terms of the forward formula for the spatial derivative &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi(x_i,t_{j+1}) = \phi(x_i,t_j) - v\cdot\Delta T\cdot(\phi(x_{i+1},t_j)-\phi(x_{i},t_j))/\Delta x$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I obtain the opposite behavior: with a negative $v$, the packet propagates nicely towards left, but a positive $v$ leads to a large divergence of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I tried to use the centered formula for the spatial derivative, &#xA;$\phi(x_i,t_{j+1}) = \phi(x_i,t_j) - v\cdot\Delta T\cdot(\phi(x_{i+1},t_j)-\phi(x_{i-1},t_j))/(2\Delta x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and in this case I obtain divergences for both signs of $v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a particular reason for this kind of behaviour? What should be the &quot;most correct&quot; way to implement this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attach here a short Matlab script that I made, and that reproduces what I described.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            L = 1; %Length&#xA;            v = 1; %Speed&#xA;            i=sqrt(-1);&#xA;            DeltaT = ((L/abs(v))/1000)&#xA;            DeltaX = 1*abs(v)*DeltaT&#xA;            x=-L:DeltaX:L;&#xA;            t = 0:DeltaT:(L/abs(v));&#xA;            sigma = L/50;&#xA;            Tbreak = 1e-5; %Parameter for animation&#xA;&#xA;            %% v = 1, backward formula&#xA;            v=1;&#xA;            phi = zeros(length(x),length(t));phi(:,1) = exp(-((x)./(2*sigma)).^2);&#xA;            for n=1:(length(t)-1)&#xA;                for k=2:(length(x)-1)&#xA;                    phi(k,n+1)=phi(k,n) - v*DeltaT*(phi(k,n)-phi(k-1,n))/(DeltaX);&#xA;                end&#xA;            end&#xA;            figure(1);&#xA;            for n=1:(length(t))&#xA;            plot(x,phi(:,n))&#xA;            pause(Tbreak)&#xA;            end&#xA;&#xA;            %% v = -1, backward formula&#xA;            v=-1;&#xA;            phi = zeros(length(x),length(t));phi(:,1) = exp(-((x)./(2*sigma)).^2);&#xA;            for n=1:(length(t)-1)&#xA;                for k=2:(length(x)-1)&#xA;                    phi(k,n+1)=phi(k,n) - v*DeltaT*(phi(k,n)-phi(k-1,n))/(DeltaX);&#xA;                end&#xA;            end&#xA;            figure(1);&#xA;            for n=1:(length(t))&#xA;            plot(x,phi(:,n))&#xA;            pause(Tbreak)&#xA;            end&#xA;&#xA;            %% v = 1, forward formula&#xA;            v=1;&#xA;            phi = zeros(length(x),length(t));phi(:,1) = exp(-((x)./(2*sigma)).^2);&#xA;            for n=1:(length(t)-1)&#xA;                for k=2:(length(x)-1)&#xA;                    phi(k,n+1)=phi(k,n) - v*DeltaT*(phi(k+1,n)-phi(k,n))/(DeltaX);&#xA;                end&#xA;            end&#xA;            figure(1);&#xA;            for n=1:(length(t))&#xA;            plot(x,phi(:,n))&#xA;            pause(Tbreak)&#xA;            end&#xA;&#xA;            %% v = -1, forward formula&#xA;            v=-1'&#xA;            phi = zeros(length(x),length(t));phi(:,1) = exp(-((x)./(2*sigma)).^2);&#xA;            for n=1:(length(t)-1)&#xA;                for k=2:(length(x)-1)&#xA;                    phi(k,n+1)=phi(k,n) - v*DeltaT*(phi(k+1,n)-phi(k,n))/(DeltaX);&#xA;                end&#xA;            end&#xA;            figure(1);&#xA;            for n=1:(length(t))&#xA;            plot(x,phi(:,n))&#xA;            pause(Tbreak)&#xA;            end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25268" LastActivityDate="2017-08-31T03:30:47.547" Title="Advection equation with finite difference: importance of forward, backward or centered difference formula for the first derivative" Tags="|finite-difference|advection|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27738" PostTypeId="2" ParentId="27737" CreationDate="2017-08-29T05:53:12.857" Score="1" Body="&lt;p&gt;This is well known. I'll give you the part I understand, and await a better answer&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's to do with the &lt;em&gt;physicality&lt;/em&gt; of the system: where information is coming from. In your 1d scenario, when fluid is advecting to the &lt;em&gt;right&lt;/em&gt;, what happens at a point can only be affected by new fluid coming to that point from the &lt;em&gt;left&lt;/em&gt;. Therefore, the &lt;em&gt;backward difference&lt;/em&gt; discretization, using $x_i$ and $x_{i-1}$, is more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this is called an &lt;em&gt;upwind&lt;/em&gt; scheme (that's wind like a breeze, not wind like a wristwatch); aka &lt;em&gt;upstream&lt;/em&gt;, because upstream/upwind is where the fluid is coming from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, if one extrapolates a slope beyond the data points defining it, it is not only inaccurate, but increasingly inaccurate. Furthermore, with each timestep, any oscillating effect will be amplified in each direction. This compunding effect over timesteps is a crucial ingredient of instability. However, I don't fully understand this process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might expect the greater accuracy of &lt;em&gt;central differences&lt;/em&gt; to also be more stable; but note that because it uses data points on either side of the point of interest, with one upstream and one downstream, half its information is irrelevant (the downstream data point is about fluid moving away).  It is &lt;em&gt;unconditionally unstable&lt;/em&gt; - there's no way to prevent it &quot;blowing up&quot;. Again, I don't fully understand this.&lt;/p&gt;&#xA;" OwnerUserId="24091" LastActivityDate="2017-08-29T05:53:12.857" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27739" PostTypeId="2" ParentId="27737" CreationDate="2017-08-29T05:56:29.207" Score="7" Body="&lt;p&gt;Okay. Let's begin with the first situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your equation is:&#xA;$$ \frac{\partial f}{\partial t}+v\frac{\partial f}{\partial x}=0\tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{Previous comments}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are plenty of webs that can explain you what happens with the error when you does not choose properly the values of your mesh to approximate the derivatives in an equation (&lt;a href=&quot;https://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&quot; rel=&quot;noreferrer&quot; title=&quot;Von Neumann stability analysis&quot;&gt;Von Neumann stability analysis&lt;/a&gt;), but I prefer writing the following for your better understanding based on physical grounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned that the solutions of that equation are of the form $f(x-vt)$, therefore as you can prove that the function $f$ along the characteristic lines ($x=x_0+vt$) is constant a of the motion:&#xA;$$\frac{df}{dt} = \frac{\partial f}{\partial t}+\frac{\partial f}{\partial x}\frac{dx}{dt}=0$$&#xA;because $dx/dt=v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{Case 1}:v&amp;gt;0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, physical information propagates through time and space according these lines: &#xA;$$x=x_0+vt \tag{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your in the $n$th timestep you have the following available information for the node $i$ from $(2)$&#xA;$$x_{i} = x_{i-1}+v\,n\Delta t$$&#xA;i.e. the information at each node $i$ depends on the previous value and moves from left o right. To see this, the function $f$ evaluated at this point is &#xA;$$f(x_i-v\,n\Delta t) = f(x_{i-1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this reason, the backward derivative is stable, because it uses the values $x_i$ and $x_{i-1}$, values in which the function $f$ has information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{Case 2}:v&amp;lt;0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand if $v$ is negative, information is transmitted in other way. Now the point $x_i$ depends on which was the point $x_{i+1}$ from $(2)$:&#xA;$$x_i=x_{i+1}-v\,\Delta t$$&#xA;Analogously the the previous case we have:&#xA;$$f(x_i+v\,n\Delta t) = f(x_{i+1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore a stable scheme for this sign of $v$ is the one which uses the values at $x_{i}$ and $x_{i+1}$, because information is transmitted from right to left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{Last comments}:$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the centred scheme, you use prohibited points $x_{i-1}$ and $x_{i+1}$ for the $v&amp;lt;0$ and $v&amp;gt;0$ cases respectively, where you have no available information. That is the reason of the instabilities for this centred scheme which always uses a prohibited value whatever the sign of $v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{Solution}$:Two in one&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, you can try this scheme:&#xA;$$v\frac{\partial f}{\partial x}\approx \frac{1}{2}(v+|v|)\frac{f_i-f_{i-1}}{\Delta x}+ \frac{1}{2}(v-|v|)\frac{f_{i+1}-f_{i}}{\Delta x}$$&#xA;This will give you the right flux whatever the sign of $v$. Can you see it?&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-08-29T06:14:00.227" LastActivityDate="2017-08-29T06:14:00.227" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27740" PostTypeId="1" CreationDate="2017-08-29T08:44:48.400" Score="1" ViewCount="249" Body="&lt;p&gt;I want to solve the following optimization problem:&#xA;$$\min_{A,B,X} \|Y-AX\|_F^2 + \lambda_1 \|Z-BX\|_F^2+ \lambda_2 \|B\|_F^2$$&#xA;$$s.t ~~x_{ij}~ \geq  0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in which, $Y$ and $Z$ are data matrices and are $d \times n$ and $l \times n$ (the corresponding columns of $Y$ and $Z$ are data samples), and $A,B,X$ are the parameter matrices and are $d \times k$, $l \times k$ and $k \times n$ respectively. $n$ is around 100 and $d,k$ are around 20.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is called a multivariate optimization problem. I tried to solve it using a general solver like Matlab's &quot;fmincon&quot; and it found a good minimum point. I think it uses the sequential programming algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how can i solve this optimization problem in a systematic way to find a good optimum point comparable to what the Matlab solver found?&lt;/p&gt;&#xA;" OwnerUserId="25272" LastActivityDate="2018-01-04T21:18:34.473" Title="How can i solve this non-convex multi-variable optimization problem?" Tags="|optimization|constrained-optimization|nonlinear-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27742" PostTypeId="2" ParentId="27277" CreationDate="2017-08-29T10:34:20.107" Score="2" Body="&lt;p&gt;I recently came across a Python tool that claims to convert between all kinds of different mesh formats. I haven't gotten around to trying it yet, but maybe it can help you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nschloe/meshio&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nschloe/meshio&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14598" LastActivityDate="2017-08-29T10:34:20.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27743" PostTypeId="2" ParentId="8839" CreationDate="2017-08-29T11:07:21.000" Score="3" Body="&lt;p&gt;Particles in SPH simulations mimic the motion of material points by simply updating their positions due to their velocities. In situations like heat conduction you don't need to move them, in fact, there is no advection at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SPH is a collocation scheme like FDM (finite difference method) but without the rigid interpretation of nodes over a grid. The SPH meshless interpolant allows the evaluation of quantities regardless of the particle distribution (however, the interpolation error depends on the distribution). That is why SPH is powerful in fluid mechanics, where the distribution of the particles is changing in every instant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question: &#xA;You are right, you don't need to move the particles, but in such a case FDM is probably a better choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information about the solution of the heat conduction equation with SPH read this paper:&#xA;&lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0010465503001553&quot; rel=&quot;nofollow noreferrer&quot;&gt;J.H. Jeong, M.S. Jhon, J.S. Halow, J. van Osdol, Smoothed particle hydrodynamics: Applications to heat conduction, In Computer Physics Communications, Volume 153, Issue 1, 2003, Pages 71-84.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20475" LastEditorUserId="20475" LastEditDate="2017-11-08T10:57:16.893" LastActivityDate="2017-11-08T10:57:16.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27744" PostTypeId="2" ParentId="25120" CreationDate="2017-08-29T11:34:08.363" Score="1" Body="&lt;p&gt;Firstly, XSPH does not behave as viscosity, it simply calculates the positions based on averaged velocity considering the close vicinity of a particle. But it is important, that the velocity itself is not affected, but only used to update positions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dr_i}{dt}=v_i+0.5\sum_j{(v_j-v_i)\frac{m_j}{\rho_j}W_{ij}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, it might be counter intuitive, but errors in SPH (at least in fluid simulations) are larger when the fluid motion is slow. That is why it is more efficient in the simulation of dam breaks and other problems where inertial forces are dominant. The results are relevant only as long as the system is under a heavily transient motion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem you experienced is still not acceptable and probably depends on the pressure evaluation scheme (equation of state) and the time step size. Check the sound speed $c$ and the smoothing distance $h$ and estimate the order of the required time step size by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\Delta t=\frac{h}{c}$.&lt;/p&gt;&#xA;" OwnerUserId="20475" LastActivityDate="2017-08-29T11:34:08.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27745" PostTypeId="2" ParentId="16363" CreationDate="2017-08-29T11:51:34.170" Score="0" Body="&lt;p&gt;The most important is that the kernels have to be normalised within the influence radius $\Delta$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int_{-\Delta}^\Delta W(r-r',h)dr'=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial interparticle distance in the grid should be calculated as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$dx=\sqrt{\frac{\Delta^2\pi}{n}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in 2D or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$dx=\sqrt[3]{\frac{4\Delta^3\pi}{3n}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in 3D. $n$ is the estimated number of neighbours: $n\approx20$ in 2D and $n\approx 50$ in 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If these are satisfied and we assume that the parameters are physically correct, both 2D and 3D simulations should be stable using the same parameters.&lt;/p&gt;&#xA;" OwnerUserId="20475" LastActivityDate="2017-08-29T11:51:34.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27746" PostTypeId="1" CreationDate="2017-08-29T14:13:46.457" Score="0" ViewCount="200" Body="&lt;p&gt;I have two arrays $f(z)$ and $z$ both indexed by k and I want to solve $\frac{df}{dz}=\mu(1-f)$ to find $\mu(z)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best numerical method to solve this equation?&lt;/p&gt;&#xA;" OwnerUserId="23709" LastEditorUserId="17294" LastEditDate="2017-08-30T19:25:22.977" LastActivityDate="2017-08-31T07:13:19.603" Title="Parameter identification for first order ODE" Tags="|ode|fortran|" AnswerCount="1" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="27747" PostTypeId="1" CreationDate="2017-08-29T16:00:59.010" Score="1" ViewCount="461" Body="&lt;p&gt;I want to use FEM to solve a 2D eigenvalue problem for the biharmonic equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;K \psi= \lambda M \psi\\&#xA;&amp;amp;\psi=\nabla \psi\cdot \hat{n}=0\quad \text{on boundaries}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K$ is the stiffness matrix corresponding to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$K_{ij} = \int_{A} \nabla^2 \psi_i  \nabla^2 \psi_j \mathrm{d}A$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $M$ is the mass matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M_{ij} = \int_{A} \psi_i \psi_j \mathrm{d}A$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the equation is fourth order I am using cubic trial functions, both height and slope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to get the stiffness and mass matrices in 1D, but I am stumped by the 2d problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of a suitable reference?&lt;/p&gt;&#xA;" OwnerUserId="25277" LastEditorUserId="9667" LastEditDate="2017-08-29T18:02:24.747" LastActivityDate="2017-08-29T18:02:24.747" Title="FEM Stiffness and Mass matrices for 2d cubic trial functions" Tags="|finite-element|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27748" PostTypeId="2" ParentId="27649" CreationDate="2017-08-29T19:19:11.237" Score="5" Body="&lt;p&gt;I will summarize a couple of possibilities:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;As a baseline, I would begin with a Hough transform kind of approach:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://www.ipol.im/pub/art/2017/208/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iterative Hough Transform for Line Detection in 3D Point Clouds&lt;/a&gt;&#xA;  &lt;em&gt;Christoph Dalitz, Tilman Schramke, Manuel Jeltsch&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is also an online demo as well as source code. Here is another paper of the same Hough-approach:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://lionel.kr.hs-niederrhein.de/~dalitz/data/publications/hough3d-visapp16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hough Parameter Space Regularisation for Line Detection in 3D&lt;/a&gt;&#xA;  &lt;em&gt;Manuel Jeltsch, Christoph Dalitz and Regina Pohle-Frohlich&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;A topological approach to detect structure lines:&#xA;&lt;a href=&quot;https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XXXIX-B3/143/2012/isprsarchives-XXXIX-B3-143-2012.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XXXIX-B3/143/2012/isprsarchives-XXXIX-B3-143-2012.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Structure Line Detection from LIDAR Point Clouds using Toplogical&#xA;  Elevation Analysis. &lt;em&gt;C. Y. Loa, L. C. Chen&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;It is possible to detect the contours in the point clouds, using e.g. &lt;a href=&quot;http://Contour%20detection%20in%20unstructured%203D%20point%20clouds&quot; rel=&quot;nofollow noreferrer&quot;&gt;this algorithm&lt;/a&gt;. Next step would be to cluster the points which are co-linear. This can be achieved for instance, using RANSAC.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some algorithms try to find some linear structures using geometric analysis:&#xA;&lt;a href=&quot;https://pdfs.semanticscholar.org/3e4b/f8123cea0195a440ce1296ddde2f026965e9.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Edge Detection and Feature Line Tracing in 3D-Point Clouds by Analyzing Geometric Properties of Neighborhoods&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have a multi-view setting, then &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-642-19309-5_31&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt; would also be of interest.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I hope these resources help get you in the right direction.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;UPDATE: Recently I have come across this work that seems to generate a visually pleasing outcome in a reasonable amount of time:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Lu, Xiaohu and Liu, Yahui and Li, Kai. &quot;Fast 3D Line Segment Detection&#xA;  From Unorganized Point Cloud.&quot; arXiv preprint arXiv:1901.02532 (2019).&#xA;  &lt;a href=&quot;https://arxiv.org/pdf/1901.02532.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1901.02532.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2019-03-31T11:36:37.267" LastActivityDate="2019-03-31T11:36:37.267" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="27749" PostTypeId="1" CreationDate="2017-08-29T20:35:10.423" Score="3" ViewCount="91" Body="&lt;p&gt;Is there an algorithm that can determine the interaction laws between particles (or point-like objects) based on their motion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple case: an algorithm that determines the gravitational interaction from nothing but the motion of the planets in the solar system. Without a &lt;em&gt;lookup table of particle interactions&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20475" LastActivityDate="2017-08-31T01:29:00.627" Title="Reverse engineering of particle motion" Tags="|particle|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27750" PostTypeId="2" ParentId="27749" CreationDate="2017-08-29T20:58:21.463" Score="3" Body="&lt;p&gt;The recent &lt;a href=&quot;https://sinews.siam.org/Details-Page/data-driven-discovery-of-governing-physical-laws&quot; rel=&quot;nofollow noreferrer&quot;&gt;survey published in SIAM News&lt;/a&gt; is a good starting point to study the work done in this area.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-08-29T20:58:21.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27751" PostTypeId="2" ParentId="27734" CreationDate="2017-08-29T22:53:05.177" Score="3" Body="&lt;p&gt;Mathematica has the TW distributions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://reference.wolfram.com/language/ref/TracyWidomDistribution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://reference.wolfram.com/language/ref/TracyWidomDistribution.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25278" LastActivityDate="2017-08-29T22:53:05.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27752" PostTypeId="1" AcceptedAnswerId="27756" CreationDate="2017-08-30T13:08:16.500" Score="2" ViewCount="118" Body="&lt;p&gt;I have small scale dense least squares problem with box constraints&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mbox{argmin}||Ax - b||^2 \quad $$&#xA;$$\mbox{subject to} \quad l_i \leq x_i \leq u_i,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Number of variables is about 10-50, several hundreds in worst case. Number of constraints is equal to number of variables. So i can factorize $A$ and/or $A^TA$. Also in my particular problem very often solution will &quot;touch&quot; only several &quot;sides of box&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that many modern numerical packages have functions to deal with such QP problem.&#xA;What i seek is detailed description or paper of efficient algorithm (first of all in terms of speed and then accuracy) because i want to understand how it works and implement it by myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently i only found &lt;a href=&quot;http://digitalassets.lib.berkeley.edu/sdtr/ucb/text/394.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; based on active set. But it looks like (if i am not mistaken) it adds only one variable into &quot;free&quot; variables set per iteration which looks like not very effective for me.&lt;/p&gt;&#xA;" OwnerUserId="15233" LastActivityDate="2017-08-31T01:31:55.117" Title="Description of algorithm for small scale linear least squares with box constraints" Tags="|algorithms|constrained-optimization|least-squares|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27753" PostTypeId="2" ParentId="27746" CreationDate="2017-08-30T19:26:41.073" Score="1" Body="&lt;p&gt;What you have is a linear ODE-constrained parameter identification problem: given the two data arrays $(z_k,f_k)$ (indexed by $k$) and the ODE model (the constraint), find the parameter $\mu$ that &lt;em&gt;best&lt;/em&gt; fits the data, where the notion of &lt;em&gt;best&lt;/em&gt; is defined in a statistically precise way. A very simple example would be a least-squares type regularized functional&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \arg \min_\mu \sum_k \left( f_k - f(z_k) \right)^2 + \| \mu \|^2 \; {\rm subject \; to\;} f_z = \mu (1-f).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One approach to this would be to write it as an ODE-constrained optimization problem with a properly regularized cost function, then solve its discrete equivalent using a numerical method and off-the shelf optimizer, as exemplified in &lt;a href=&quot;https://www.zib.de/susanna.roeblitz/Roeblitz_paramest.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these slides (1)&lt;/a&gt;. Note that the optimization algorithm used there is a &lt;em&gt;local&lt;/em&gt; Newton-type procedure. This means, for most practical problems, that you will need a reasonably good initial guess for the Newton procedure to converge to a &quot;good-quality&quot; minimum point, i.e., to a $\bar\mu$ not &quot;too far&quot; (in some norm) from the unknown &quot;true&quot; parameter $\mu^{\rm true}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If local minima are a major concern, and you do not have a good &quot;guess&quot; at $\mu^{\rm true}$ to start your Newton routine from, then &lt;a href=&quot;http://ab-initio.mit.edu/wiki/index.php/NLopt_Introduction#Global_versus_local_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;global optimization&lt;/a&gt; is an alternative (caveat emptor: these procedures are more computationally expensive than a local routine). Or you can bring out the best of the two approaches into a combined global/local optimization strategy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have the time to implement all this, then there are other options to consider:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The MATLAB system identification toolbox can help (Google &lt;code&gt;idgrey&lt;/code&gt;, &lt;code&gt;greyest&lt;/code&gt;). The workflow for your problem is described in detail &lt;a href=&quot;https://mathworks.com/help/ident/ug/estimating-coefficients-of-odes-to-fit-given-solution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;There are some FORTRAN libraries that do this. For example &lt;a href=&quot;http://www.ai7.uni-bayreuth.de/pdefit.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDEFIT&lt;/a&gt; and/or &lt;a href=&quot;http://www.ai7.uni-bayreuth.de/easy_fit.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;EASY-FIT&lt;/a&gt;. I'd suggest you to read the manual to get acquainted with the features (and limitations) of each.&lt;/li&gt;&#xA;&lt;li&gt;Other programming languages appear to have at least some support for parameter identification: &lt;a href=&quot;https://dsp.stackexchange.com/questions/915/system-identification-packages&quot;&gt;R&lt;/a&gt;, &lt;a href=&quot;https://stackoverflow.com/questions/11278836/fitting-data-to-system-of-odes-using-python-via-scipy-numpy&quot;&gt;python with numpy/scipy&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-08-31T07:13:19.603" LastActivityDate="2017-08-31T07:13:19.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27754" PostTypeId="2" ParentId="27737" CreationDate="2017-08-30T22:52:51.353" Score="5" Body="&lt;p&gt;You stumbled across something very fundamental that is important to understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For any PDE we can define for each point its &quot;domain of dependence&quot; This is the region of space/time that is able to have an effect on the solution at that point. For your advection problem, it is just the characteristic line through that point. If the point of interest is $x_0,t_0$ the domain of dependence is the line $(x-x_0)=v(t-t_0)$ with $t&amp;lt;t_0$. For more general hyperbolic problems it is usually a region bounded by more than one characteristic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same idea applies to the discrete model. The discrete domain of dependence for&#xA;a grid point P is the set of grid points where a change of data would affect the solution at P. When you use backward differences, the discrete domain of dependence is a pyramid with its vertex at P. One side extends&#xA;vertically downward and consists of points Q such that $x_Q=x_P$. The other side slopes down to the left, and consists of points Q such that $(x_Q-x_P)/\Delta x=(t_Q-t_P)/\Delta t$ (in both cases $t_Q&amp;lt;t_P$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A necessary, but not sufficient, condition for stability is that &lt;em&gt;The discrete domain of dependence contains the true domain of dependence&lt;/em&gt;. This is not a mathematical result (although often presented as such) but a logical result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider refining the grid in a way that preserves the ratioof $\Delta t/\Delta x$, so the shape of the discrete DoD does not change. We do this with a discrete DoD that does not contain the true DoD &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                      / O&#xA;                                   /  O O&#xA;                                /.  O O O&#xA;                              /   O O O O&#xA;                           /    O O O O O&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(I hope this conveys the idea. for some reason I could not import an image, x goes to the right, t goes up, the slashes represent the characteristic line) Hoping to make the answer better by refining the grid is hopeless&#xA;because it will never contain the single relevant piece of information that we need. This much worse that just the error associated with extrapolation. Remember we want to solve for arbitrary (nonsmooth) initial data, and the data inside and outside the discrete DoD may be quite independent. It follows that if $v&amp;gt;0$, we must use (out of the three choices) backward differencing, and that $\Delta t$ must be less than $v\Delta x$. This the Courant (Courant-Friedrichs-Levy, CFL) condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case with $v&amp;lt;0$ follows by symmetry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The central difference is allowable by this test, but is NOT stable. I do not think that there is a simple intuitive explanation. However, there IS a stable (but very bad) method using this stencil. It is called the Lax-Friedrichs method&#xA;$$\phi_j^{n+1}=\frac{1}{2}(\phi_{j-1}^n+\phi_{j+1}^n)-\frac{v\Delta t}{2\Delta x}(\phi_{j+1}^n-\phi_{j-1}^n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be very hard to investigate the stability of more general cases, but the necessary Courant condition is universal because it is a logical requirement.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-08-30T22:52:51.353" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27755" PostTypeId="2" ParentId="27749" CreationDate="2017-08-31T01:29:00.627" Score="1" Body="&lt;p&gt;If you make the assumption that the force the two objects exert on each other is along the connecting line (this is necessary for the conservation of angular momentum, but assumes an infinite speed with which forces penetrate space), then you have that the differential equation for particle 2 is&#xA;$$&#xA;  \ddot x_2(t) = f(\|x_2(t)-x_1(t)\|) (x_2(t)-x_1(t))&#xA;$$&#xA;and similarly for $x_1(t)$. If you know the trajectories of both particles, then you can compute&#xA;$$&#xA;  f(\|x_2(t)-x_1(t)\|) = \frac{\|\ddot x_2(t)\|}{\|x_2(t)-x_1(t)\|}&#xA;$$&#xA;and you can tabulate this force $f(r)$ for all values of $r$ that appear as distances $r=\|x_2(t)-x_1(t)\|$ over the course of the observation period. At least for this range of $r$ values, you then know what the interaction force is.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-08-31T01:29:00.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27756" PostTypeId="2" ParentId="27752" CreationDate="2017-08-31T01:31:55.117" Score="3" Body="&lt;p&gt;You should take a look at the excellent book by Nocedal and Wright on &quot;Numerical Optimization&quot;. It has a great deal of material on these kinds of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm you refer to, where one only adds or subtracts &lt;em&gt;one&lt;/em&gt; variable to the active set at a time is described in detail in the book. The reason for only adding or subtracting one variable at a time is that if you do more than one, one can come up with examples where the algorithm cycles without converging. There are, however, other methods (e.g., &quot;primal-dual active set methods&quot;) that are more liberal in this regard.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-08-31T01:31:55.117" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27757" PostTypeId="2" ParentId="27737" CreationDate="2017-08-31T03:30:47.547" Score="1" Body="&lt;p&gt;Central difference with forward Euler in time will be unstable. But if you put in RK4 as mentioned in another comment, it will be L2 stable. This is enough for smooth solutions as in your case with a Gaussian initial condition. But for discontinuous solutions, it will still generate oscillatory solutions. But it should not blow up since it is L2 stable. Of course the solution may be useless since it is has too much error. I have a python code here &lt;a href=&quot;https://github.com/cpraveen/numpde/blob/master/linhyp1d/rk4cs2.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/cpraveen/numpde/blob/master/linhyp1d/rk4cs2.py&lt;/a&gt; that uses RK4. You can set the initial condition in ic.py file or define the function uinit with your own initial condition.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2017-08-31T03:30:47.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27758" PostTypeId="1" AcceptedAnswerId="27766" CreationDate="2017-08-31T04:11:05.733" Score="6" ViewCount="470" Body="&lt;p&gt;It is &lt;a href=&quot;https://stackoverflow.com/questions/32435796/when-to-use-stdhypotx-y-over-stdsqrtxx-yy&quot;&gt;conventional wisdom&lt;/a&gt; that programmers should use &lt;a href=&quot;http://en.cppreference.com/w/cpp/numeric/math/hypot&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;std::hypot&lt;/code&gt;&lt;/a&gt; whenever one implements an expression of the form $r = \sqrt{x^2 + y^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my example, my expression is $r = \sqrt{1 + c^2}$, where $0 \le c \le 1, c \in \Re$.  My first instinct is to reach for &lt;code&gt;hypot()&lt;/code&gt; to implement - I am interested in both double and float variants.  In my problem, for most instances $c \ll 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a quick read of &lt;a href=&quot;https://en.wikipedia.org/wiki/Hypot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; suggests I might be wasting my (CPU) time. Since the suggested implementation where $x &amp;gt; y$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r = \sqrt { x^2 + y^2 } \\&#xA;  = \sqrt { x^2 ( 1 + (y/x)^2) } \\&#xA;  = |x| \sqrt {1 + (y/x)^2 }&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in my instance $x = 1$ and therefore it is algebraically the same as the naive implementation $r = \sqrt{1 + y^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the impact of not using &lt;code&gt;hypot()&lt;/code&gt; in this problem, and are there any other numerical pitfalls to look for?&lt;/p&gt;&#xA;" OwnerUserId="761" LastEditorUserId="127" LastEditDate="2017-09-01T06:31:37.693" LastActivityDate="2017-09-01T06:31:37.693" Title="Is there any point to using hypot() for $\sqrt{1+c^2}$, $0 \le c \le 1$ for real numbers" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27759" PostTypeId="2" ParentId="16338" CreationDate="2017-08-31T09:55:15.473" Score="2" Body="&lt;p&gt;Given the fundamental idea of the SPH interpolant with an arbitrary function &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (which could be the velocity &lt;span class=&quot;math-container&quot;&gt;$\textbf{v}$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A(r)=\int_{-\infty}^{\infty} A(r')\delta(r-r')dr'$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the convolution of an arbitrary &lt;span class=&quot;math-container&quot;&gt;$A(r)$&lt;/span&gt; with the Dirac delta &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt;. Obviously, is not applicable in numerical simulations due to two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; has infinitesimally small radius, and infinite height&lt;/li&gt;&#xA;&lt;li&gt;The integral can be evaluated only analytically.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To solve both issues, first we change &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; to a mollifier function (kernel) to achieve a finite (or sometimes infinitely large) influence radius:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A(r)\approx\int_{\Omega} A(r')W(r-r',h)dr'$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is analogous to the standard deviation in case of Gaussian function and &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; is the domain of the kernel. &lt;span class=&quot;math-container&quot;&gt;$W(r-r',h)$&lt;/span&gt; is usually a compact bell shaped function of unit volume. Secondly, change the integral to the summation form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\langle A(r_i)\rangle = \sum_{j}{A(r_j)\frac{m_j}{\rho_j}W(r_i-r_j,h)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now &lt;span class=&quot;math-container&quot;&gt;$r_i$&lt;/span&gt; is corresponding to the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; discrete point often referred to as &lt;em&gt;particle&lt;/em&gt;. This is can be considered as a discrete convolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now apply the continuum convolution to the derivative of A:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_{r} A(r)=\int_{\Omega} \nabla_{r'} A(r')W(r-r',h)dr'$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\nabla_{r}$&lt;/span&gt; denotes the derivative at &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the rules of differentiation, it also can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_{r} A(r)\approx\int_{\Omega} \nabla_{r'} \big[A(r')W(r-r',h)\big]dr'-\int_{\Omega}A(r')\nabla_{r'} W(r-r',h)dr'$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transform the first term in the RHS from volume integral to surface integral using the Gauss-Ostrogradsky theorem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla A(r)\approx\int_{\partial\Omega}A(r')W(r-r',h)dS-\int_{\Omega}A(r')\nabla_{r'} W(r-r',h)dr'$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$W(r-r',h)$&lt;/span&gt; is compact and has a zero value at its boundary, the first term on RHS is now zero, so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla A(r)\approx-\int_{\Omega}A(r')\nabla_{r'} W(r-r',h)dr'$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we can use the identity&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla_{r'}W(r-r',h)=-\nabla_{r}W(r-r',h)$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to obtain the discrete form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\langle \nabla A_i\rangle = \sum_{j}{A_j\frac{m_j}{\rho_j}\nabla_i W(r_i-r_j,h)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this gives a poor estimation on the derivative, because it is extremely sensitive to the particle distribution. To overcome this problem, you can apply the discrete convolution to the formula you've shown:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\langle\nabla A_i\rangle=\frac{\langle \nabla(\rho_i A_i)\rangle-A_i\langle\nabla \rho_i\rangle}{\rho_i} = \frac{1}{\rho_i}\sum_{j}{(\rho_jA_j)\frac{m_j}{\rho_j}\nabla_i W(r_i-r_j,h)}-\frac{1}{\rho_i}\sum_{j}{(\rho_jA_i)\frac{m_j}{\rho_j}\nabla_i W(r_i-r_j,h)}$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is finally:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\langle\nabla A_i\rangle=\frac{1}{\rho_i}\sum_{j}{(A_j-A_i)m_j \nabla_i W(r_i-r_j,h)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important, that this is only one of the several possible forms of differential operators in SPH. They have different consistency and conservation properties forming a trade-off. Which to use? It depends on the equation you want to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, you are free to change the product to use the obtained operator for curl:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\langle\nabla\times \textbf{v}_i\rangle=\frac{1}{\rho_i}\sum_{j}{(\textbf{v}_j-\textbf{v}_i)\times\nabla_i W(r_i-r_j,h)m_j}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20475" LastEditorUserId="20475" LastEditDate="2020-02-27T09:21:46.773" LastActivityDate="2020-02-27T09:21:46.773" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="27761" PostTypeId="1" AcceptedAnswerId="27790" CreationDate="2017-08-31T12:31:54.613" Score="4" ViewCount="256" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bBTVQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bBTVQ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Given a 2D irregular spaced data like shown in the figure, I would like to know how to find derivatives at '*' by interpolating the values at 'o'. Does lagrange 2D interpolation work at irregular spaced data&lt;/p&gt;&#xA;" OwnerUserId="25292" LastEditorUserId="25292" LastEditDate="2017-09-01T13:05:52.070" LastActivityDate="2017-09-03T17:49:09.227" Title="interpolation 2D irregular nodes" Tags="|numerical-analysis|finite-difference|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27762" PostTypeId="1" AcceptedAnswerId="27764" CreationDate="2017-08-31T15:12:10.877" Score="9" ViewCount="5207" Body="&lt;p&gt;I'm a novice to finite element and I'm finding quite hard to find the actual difference between Test function(s) and Basis function(s).&#xA;I would be glad if somone could explain me that and point out &lt;strong&gt;how can they differ from one another and in which cases its useful that they differ (or not).&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="25239" LastEditorUserId="25239" LastEditDate="2017-08-31T15:17:21.513" LastActivityDate="2017-09-01T06:53:19.083" Title="Galerkin method: Test functions vs. Basis functions" Tags="|finite-element|pde|numerical-analysis|fluid-dynamics|computational-physics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27763" PostTypeId="2" ParentId="27762" CreationDate="2017-08-31T15:42:32.993" Score="0" Body="&lt;p&gt;Formally a test function $\psi$ is a function that belongs to a function space $\mathcal{V}$. If you select among all test functions of such function space one type, e.g. polynomials of second order $\varphi\in\mathcal{P}_2(x)$, you can depompose any function $f(x)$ in terms of this basis.&#xA;Take into account, for example that the basis vectors (i.e. basis functions) are orthogonal to each other, i.e.:&#xA;$$ \int_{-L}^{L}{\varphi_i(x)\varphi_k(x)\,dx}=\delta_{ik}\tag{*}$$&#xA;Because $f(x)$ is defined as the linear combination of basis functions (do you remember that this is analogous to vector space?):&#xA;$$f(x) = \sum_n{a_i\varphi_i(x)} \tag{**}$$&#xA;To find $a_n$ multiply scalarly the last equation $(**)$ by $\varphi_k$ and apply $(*)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{-L}^{L}{f(x)\varphi_k(x)\,dx}=a_i\int_{-L}^{L}{\varphi_i(x)\varphi_k(x)\,dx}=a_{k}$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-08-31T18:17:15.037" LastActivityDate="2017-08-31T18:17:15.037" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27764" PostTypeId="2" ParentId="27762" CreationDate="2017-08-31T16:53:32.113" Score="14" Body="&lt;p&gt;Suppose that the solution $u$ of the PDE lives in some function space $X$. We'll write the PDE as a bilinear form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A(u, v) = f(v)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all $v$ in $X$, where $f$ is some element of the dual space $X^*$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To approximate the solution $u$ of this PDE, we can instead look for some field $u_N$ that lives in a finite-dimensional subspace $V_N$ of $X$. Typically, we pick this finite-dimensional subspace by choosing a set of functions $\{\phi_1, \ldots, \phi_N\}$ and setting $V_N$ to be the span of these functions, in which case we can write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_N = \sum_{n = 1}^NU_n\phi_n$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\{U_n\}$ are the coefficients of $u_N$ in this basis. &lt;strong&gt;The functions $\{\phi_1,\ldots,\phi_N\}$ that we use to describe the solution are the basis functions.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, to reduce our infinite-dimensional problem to a finite-dimensional one, we pick another finite-dimensional subspace $W_N$ of $X$, and choose the coefficients of $u_N$ so that it is the solution of the (finite-dimensional) linear system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A(u_N, \psi) = f(\psi)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all $\psi$ in $W_N$. Much like with how we selected the basis functions, we can instead write $W_N$ as the span of a set of functions $\{\psi_1,\ldots, \psi_N\}$. &lt;strong&gt;The functions $\{\psi_1,\ldots,\psi_N\}$ that we integrate against are the test functions.&lt;/strong&gt; In that case, we can form the matrix $\mathscr{A}$ with entries&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathscr{A}_{ij} = A(\phi_j, \psi_i)$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in which case the linear system can be written in terms of the vector $U$ of coefficients as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathscr{A}U = F$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the basis functions and the trial functions do not have to be the same thing. What we need in order for the finite-dimensional approximations $u_N$ to converge to $u$ as we take larger and larger approximation spaces is that (1) the finite-dimensional problem is solvable for every $N$ and (2) a stability property, guaranteeing that $\mathscr{A}^{-1}$ is bounded as $N \to \infty$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could be forgiven for finding this confusing because, &lt;strong&gt;for self-adjoint linear elliptic PDE, the basis and test spaces are almost always the same,&lt;/strong&gt; so people usually talk about the two spaces as if they're one and the same thing. For nice elliptic PDE, taking $W_N = V_N$ guarantees that the linear system is symmetric, and you can also prove that the approximation is quasi-optimal in a certain sense, so there's little reason to do otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for other kinds of PDE, it can be advantageous to use different basis and test spaces. For example, consider an advective transport problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$w\cdot \nabla q = f$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $w$ is a vector field and we want to solve for $q$. Using the same basis and test functions can lead to lots of oscillatory artifacts. A popular approach to combat this is the streamline upwind Petrov-Galerkin discretization. Suppose that the advection is along a vector field $w$, and let $\phi_n$ be a basis function. The test function $\psi_n$ is then defined in terms of the basis function and the advection field as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\psi_n = \phi_n + \alpha w\cdot\nabla\phi_n$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha$ is an appropriately-selected parameter. The resulting equation is similar to upwind finite differencing, with similar improvements to numerical stability. In this case, the original problem isn't symmetric or necessarily even positive-definite. There's no symmetry to preserve by taking the basis and test spaces to be identical.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="393" LastEditDate="2017-09-01T03:00:26.630" LastActivityDate="2017-09-01T03:00:26.630" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27765" PostTypeId="2" ParentId="26369" CreationDate="2017-08-31T17:14:32.323" Score="0" Body="&lt;p&gt;You need to update the step size at each iteration instead of using a fixed step size. At each iteration, you need to update the search direction $p$ and residual $r$. So the step size is updated based on the new search direction and residual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, maybe we are reading different materials. Mine does not have the negative signed for the step size. &lt;/p&gt;&#xA;" OwnerUserId="22994" LastActivityDate="2017-08-31T17:14:32.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27766" PostTypeId="2" ParentId="27758" CreationDate="2017-08-31T20:54:04.197" Score="10" Body="&lt;p&gt;&lt;em&gt;Summary&lt;/em&gt;: it is cheapest &lt;em&gt;and&lt;/em&gt; most accurate to use &lt;code&gt;sqrt(fma(c, c, 1))&lt;/code&gt; if you have FMA, and &lt;code&gt;sqrt(1+c*c)&lt;/code&gt; otherwise. In my testing, though, the difference is extremely marginal: of the 1065353216 32-bit floating point numbers $0\leq c\leq 1$, the first formula is better 532509 times (0.05%), and worse 382159 times (0.035%), and neither formula has error worse than 1 ulp, so the obvious &lt;code&gt;sqrt(1+c*c)&lt;/code&gt; is good enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a little bit of a fine point, and Wikipedia can sometimes be a little unreliable on this. One good way to settle these types of questions is to go to a mature library that already implements hypot, such as openlibm (&lt;a href=&quot;https://github.com/JuliaLang/openlibm&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/JuliaLang/openlibm&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quoting the source code comment that explains it (&lt;a href=&quot;https://github.com/JuliaLang/openlibm/blob/master/src/e_hypot.c&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/JuliaLang/openlibm/blob/master/src/e_hypot.c&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/* __ieee754_hypot(x,y)&#xA; *&#xA; * Method :                  &#xA; *  If (assume round-to-nearest) z=x*x+y*y &#xA; *  has error less than sqrt(2)/2 ulp, than &#xA; *  sqrt(z) has error less than 1 ulp (exercise).&#xA; *&#xA; *  So, compute sqrt(x*x+y*y) with some care as &#xA; *  follows to get the error below 1 ulp:&#xA; *&#xA; *  Assume x&amp;gt;y&amp;gt;0;&#xA; *  (if possible, set rounding to round-to-nearest)&#xA; *  1. if x &amp;gt; 2y  use&#xA; *      x1*x1+(y*y+(x2*(x+x1))) for x*x+y*y&#xA; *  where x1 = x with lower 32 bits cleared, x2 = x-x1; else&#xA; *  2. if x &amp;lt;= 2y use&#xA; *      t1*y1+((x-y)*(x-y)+(t1*y2+t2*y))&#xA; *  where t1 = 2x with lower 32 bits cleared, t2 = 2x-t1, &#xA; *  y1= y with lower 32 bits chopped, y2 = y-y1.&#xA; *      &#xA; *  NOTE: scaling may be necessary if some argument is too &#xA; *        large or too tiny&#xA; *&#xA; * Special cases:&#xA; *  hypot(x,y) is INF if x or y is +INF or -INF; else&#xA; *  hypot(x,y) is NAN if x or y is NAN.&#xA; *&#xA; * Accuracy:&#xA; *  hypot(x,y) returns sqrt(x^2+y^2) with error less &#xA; *  than 1 ulps (units in the last place) &#xA; */&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So if you can compute $1+c^2$ sufficiently accurately (and in your case with $|c|\leq 1$ you will not have over-/underflow), the final result will be accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a modern CPU with a fused multiply-add (FMA) instruction, this is trivial using &lt;code&gt;fma&lt;/code&gt;, which most languages' standard math libraries have, so you can use &lt;code&gt;sqrt(fma(c, c, 1))&lt;/code&gt; (cost: just 1 flop plus the cost of a square root), this is even marginally cheaper than what hypot does. The error in &lt;code&gt;fma(c, c, 1)&lt;/code&gt; is at most $\frac12$ an ulp with round-to-nearest, so you'll get the same accuracy as hypot, with error $&amp;lt;1$ ulp, so this is the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the formulas that hypot actually uses, I don't really understand what they're doing. It chooses between $1+c^2$ and $2c+(c-1)^2$ depending on whether $c\leq\frac12$. It almost looks like a special case of double-double arithmetic, where you compute a product accurately by writing it as a sum of two numbers, but I'm not sure. I imagine if they could use fma, the formulas would be a lot simpler. In my testing they're at most as accurate as the plain $1+c^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would happen if you tried $1+c^2$ directly? The relative error of evaluating $\hat w = \mathrm{fl}(1+\mathrm{fl}(c^2))$ is at most $\epsilon_1\leq \frac34\mathrm{ulp}$, which gives the error in $\mathrm{fl}(\sqrt{\hat w})$ as $\sqrt{1+\epsilon_1}-1+\epsilon_2 \leq \tfrac78\mathrm{ulp}$, which is at most a single ulp, so it's accurate too, and as good as hypot.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-08-31T21:18:12.707" LastActivityDate="2017-08-31T21:18:12.707" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="27768" PostTypeId="1" CreationDate="2017-09-01T03:27:58.257" Score="1" ViewCount="173" Body="&lt;p&gt;In the course of trying to implement algorithms for Voronoi and &lt;a href=&quot;https://en.wikipedia.org/wiki/Power_diagram&quot; rel=&quot;nofollow noreferrer&quot;&gt;Laguerre diagrams&lt;/a&gt;, I realized I needed to verify if my implementation is working correctly using a point (or circle) configuration with a known Voronoi (or power) diagram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not been successful in looking for datasets with known diagrams. Does anyone know a good source for these diagrams?&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2017-09-01T05:10:33.790" Title="Sources for verified Voronoi and power diagrams" Tags="|computational-geometry|data-sets|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27769" PostTypeId="2" ParentId="27689" CreationDate="2017-09-01T03:57:27.497" Score="14" Body="&lt;p&gt;The efficient answer to this question is, not too surprisingly, in &lt;a href=&quot;http://www.cs.berkeley.edu/~wkahan/MathH110/Cross.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;another&lt;/em&gt; note by Velvel Kahan&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha=2\arctan\left(\left\|\frac{\mathbf v_1}{\|\mathbf v_1\|}+\frac{\mathbf v_2}{\|\mathbf v_2\|}\right\|,\left\|\frac{\mathbf v_1}{\|\mathbf v_1\|}-\frac{\mathbf v_2}{\|\mathbf v_2\|}\right\|\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I use $\arctan(x,y)$ as the angle made by $(x,y)$ with the horizontal axis. (You might have to flip the order of the arguments in some languages.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I gave a &lt;em&gt;Mathematica&lt;/em&gt; demonstration of Kahan's formula &lt;a href=&quot;https://mathematica.stackexchange.com/a/97854&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="127" LastActivityDate="2017-09-01T03:57:27.497" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27770" PostTypeId="2" ParentId="24477" CreationDate="2017-09-01T04:07:48.003" Score="1" Body="&lt;p&gt;I believe what you are looking at can be reposed as the &lt;a href=&quot;http://statweb.stanford.edu/~tibs/sta305files/Rudyregularization.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LASSO problem.&lt;/a&gt; . Different ways of doing this. This is the problem you are looking at it a &lt;a href=&quot;https://www.hindawi.com/journals/aai/2017/1736389/&quot; rel=&quot;nofollow noreferrer&quot;&gt;weighted least squares problem&lt;/a&gt;. It is actually convex. There are ways to do this with PCA.Which is the SVD of the covariance matrix. &lt;/p&gt;&#xA;" OwnerDisplayName="user25299" LastActivityDate="2017-09-01T04:07:48.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27771" PostTypeId="2" ParentId="27768" CreationDate="2017-09-01T04:53:59.040" Score="3" Body="&lt;p&gt;The most trivial pointsets are regular grid of points. They look innocent, but are the most difficult to handle in Voronoi/Laguerre codes because the Delaunay triangulation is non-unique, so they can be used to detect bugs in your code. They can be handled by adapted geometric predicates (see my publication [6] on geometric predicates and the references herein).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To detect bugs in my Voronoi/Laguerre diagram code in 2D and 3D [1], I used the following approach:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the Voronoi/Laguerre diagram for a collection of pointsets&#xA;(random) and test that the output of your code satisfies the empty&#xA;ball property. For Voronoi, this means that the circumscribed circle(2d) or sphere(3d)&#xA;of each Delaunay triangle does not contain any point. For Laguerre, there is a similar condition;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the Voronoi/Laguerre diagram of random pointsets with a code&#xA;that you trust [1] (if you trust me), [2],[3] (I trust them) and compare with the output of your code;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Add to the list of pointsets difficult ones. The difficult datasets&#xA;are those that have cocyclic (2d) / cospheric (3d) points such as regular grids that I mentioned before. Note that when you got cocyclic / cospheric points, the Delaunay triangulation is non-unique (then the output of your code may differ as compared to other codes);&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When all the weights are set to zero, the Laguerre diagram is a Voronoi diagram, then you can compare the output of your algorithms (if they mismatch, then both are probably wrong, like it happened to me).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I &lt;strong&gt;strongly&lt;/strong&gt; recommend to use a continuous integration platform [4] to do automatic non-regression testing. Voronoi/Laguerre code is not very complicated (approx. 300 lines of code), but there are some gotchas/subtleties, in particular in the predicates for handling cocyclic points. Automatic daily tests (doing 1., 2., 3., 4. + Valgrind [5] memory test) allowed to detect several bugs in my own code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Geogram: &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] CGAL: &lt;a href=&quot;http://www.cgal.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cgal.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Tetgen:&lt;a href=&quot;http://wias-berlin.de/software/tetgen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://wias-berlin.de/software/tetgen/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;https://jenkins.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://jenkins.io/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] &lt;a href=&quot;http://valgrind.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://valgrind.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6] &lt;a href=&quot;https://hal.inria.fr/hal-01225202&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.inria.fr/hal-01225202&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-09-01T05:10:33.790" LastActivityDate="2017-09-01T05:10:33.790" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27773" PostTypeId="2" ParentId="27762" CreationDate="2017-09-01T06:53:19.083" Score="5" Body="&lt;p&gt;I'll try to give a short answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Basis functions are those that you want in your solution. Test functions are those that you don't want in your residual.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You want them to be same in Bubnov-Galerkin method. The (Bubnov-Galerkin collab) want you, not to be able to build the residual with the same functions you used to build the solution. How can you do that? Well, e.g. by testing the orthogonality of each of the functions you used to build the solution, and the residual.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2017-09-01T06:53:19.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27775" PostTypeId="1" CreationDate="2017-09-01T11:12:26.567" Score="0" ViewCount="468" Body="&lt;p&gt;I need to do a matching on a hypergraph. I read that in the case of a hypergraph there is no adjacency matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I represent edges then? &lt;/p&gt;&#xA;" OwnerUserId="25162" LastActivityDate="2018-01-07T20:54:00.537" Title="Hypergraph matching -&gt; adjacency matrix?" Tags="|algorithms|graph-theory|discrete-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27777" PostTypeId="1" CreationDate="2017-09-01T14:51:59.470" Score="6" ViewCount="182" Body="&lt;p&gt;Let $p$ be a polynomial with floating-point coefficients and let $a$ be a floating-point value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method for accurately evaluating the sign of $p(a)$ in floating-point arithmetic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't care about the actual value $f(a)$, only about its sign. But I want the exact sign.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it helps, we can assume that $ 0 \le a \le 1$.&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2017-09-01T21:47:10.290" Title="Accurate evaluation of the sign of a polynomial" Tags="|floating-point|polynomials|accuracy|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27778" PostTypeId="2" ParentId="27777" CreationDate="2017-09-01T15:19:36.140" Score="2" Body="&lt;p&gt;You could use &lt;a href=&quot;https://en.wikipedia.org/wiki/Interval_arithmetic&quot; rel=&quot;nofollow noreferrer&quot;&gt;interval arithmetic&lt;/a&gt; for this purpose.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-09-01T15:19:36.140" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27779" PostTypeId="1" CreationDate="2017-09-01T19:14:57.207" Score="6" ViewCount="1016" Body="&lt;p&gt;I wanted to find and plot the eigenvalues of large (around &lt;span class=&quot;math-container&quot;&gt;$1000\times1000$&lt;/span&gt;) matrices. But discovered when using the &lt;code&gt;eig&lt;/code&gt; function in matlab, it gives complex eigenvalues when it shouldn't. For example, in the code below I have a &lt;a href=&quot;https://en.wikipedia.org/wiki/Tridiagonal_matrix#Eigenvalues&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tridiagonal&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Toeplitz_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Toeplitz matrix&lt;/a&gt;, which should have all real eigenvalues.&lt;/p&gt;&#xA;&lt;p&gt;But it seems &lt;code&gt;eig&lt;/code&gt; is unstable for &lt;code&gt;n=90&lt;/code&gt; and returns a small complex error in a few of the eigenvalues. Is there a way I can get the eigenvalues more accurately?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;clear parameters&#xA;close all&#xA;clc&#xA;n=90;&#xA;dd=-2.*ones(n,1);&#xA;ud=1.8*ones(n,1);&#xA;ld=.1*ones(n,1);&#xA;A = spdiags([ld dd ud],-1:1,n,n);&#xA;C=full(A);&#xA;g=eig(C);&#xA;g=sort(g);&#xA;cond(C)&#xA;plot(g,'.')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25309" LastEditorUserId="23791" LastEditDate="2022-01-11T15:36:59.473" LastActivityDate="2022-01-11T15:36:59.473" Title="Complex Eigenvalues using eig (Matlab)" Tags="|matlab|eigenvalues|complex-arithmetic|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="27780" PostTypeId="2" ParentId="27779" CreationDate="2017-09-01T19:36:32.893" Score="11" Body="&lt;p&gt;You have an ill-conditioned eigenvalue problem. Consider a perturbation &lt;span class=&quot;math-container&quot;&gt;$\delta A$&lt;/span&gt; in your matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;—with double-precision floats this is &lt;span class=&quot;math-container&quot;&gt;$O(10^{-16})$&lt;/span&gt;. It turns the eigendecomposition from&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ X^{-1}A X = \Lambda $$&lt;/span&gt;&#xA;into (approximately keeping &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; fixed)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ X^{-1}(A+\delta A)X = \Lambda + \delta \Lambda. $$&lt;/span&gt;&#xA;This means that the change in eigenvalues is bounded by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \|\delta \Lambda\| \leq \|X^{-1}\| \|X\| \|\delta A\| = \kappa(X)\|\delta A\|, $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; is the condition number.&lt;/p&gt;&#xA;&lt;p&gt;Assuming that the perturbation in &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is on the order of machine epsilon, which is about &lt;span class=&quot;math-container&quot;&gt;$10^{-16}$&lt;/span&gt;, there will be an error on the order of &lt;span class=&quot;math-container&quot;&gt;$\kappa(X) \times 10^{-16}$&lt;/span&gt; in the eigenvalues. This is not instability in the numerical method of &lt;code&gt;eig&lt;/code&gt;, but rather a property of your mathematical problem. For example, you wouldn't have this problem if &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; was symmetric, in which case &lt;span class=&quot;math-container&quot;&gt;$\kappa(X)=1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In your case, I calculated the condition number &lt;span class=&quot;math-container&quot;&gt;$\kappa_2(X)$&lt;/span&gt; with extra precision for &lt;span class=&quot;math-container&quot;&gt;$n=64$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$n=90$&lt;/span&gt; failed to converge), and already it is &lt;span class=&quot;math-container&quot;&gt;$4.09\times 10^{39}$&lt;/span&gt;, which is much too large for ordinary double-precision floating-point arithmetic.&lt;/p&gt;&#xA;&lt;p&gt;If you really need to solve such ill-conditioned eigenvalue problems, the easiest way is probably to use enough extra precision to keep &lt;span class=&quot;math-container&quot;&gt;$\kappa(X)\epsilon_{\mathrm{mach}}$&lt;/span&gt; small, so you'd need about 60 digits for &lt;span class=&quot;math-container&quot;&gt;$n=64$&lt;/span&gt; and more for larger &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="27325" LastEditDate="2021-04-14T00:44:19.657" LastActivityDate="2021-04-14T00:44:19.657" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="27781" PostTypeId="1" CreationDate="2017-09-01T20:04:42.987" Score="2" ViewCount="92" Body="&lt;p&gt;I'm simulating a nano particle and wish to calculate the activation energy of the reaction where an atom is displaced from one point on the lattice to a neighboring void( might be at higher energy than initial). &#xA;Calculating this activation energy using energy difference will require me to calculate the complete trajectory to determine the point with highest energy, which is computationally very expensive. &#xA;Is there any other tested method to find this activation energy without having to perform so many computations each time.&#xA;Note: the new site is a neighbor of the old site and I expect( not assume) the atom to roll over two of its adjacent atoms to reach this new site. &#xA;Also all the atoms are identical.&lt;/p&gt;&#xA;" OwnerUserId="25310" LastActivityDate="2017-09-04T18:52:20.773" Title="Calculate activation energy of inter molecular displacement on a nano particle" Tags="|computational-chemistry|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27782" PostTypeId="2" ParentId="27777" CreationDate="2017-09-01T20:20:54.000" Score="3" Body="&lt;p&gt;Compensated Horner method (&lt;a href=&quot;http://www-pequan.lip6.fr/~jmc/polycopies/Compensation-horner.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www-pequan.lip6.fr/~jmc/polycopies/Compensation-horner.pdf&lt;/a&gt;) has an error bound of the form&#xA;$$ |\mathrm{comphorner}(p, x) - p(x)| \leq u|p(x)| + \gamma_{2n}^2\tilde p(x), \qquad \tilde p(x) = \sum_i |a_i||x|^i, \quad \gamma_n = \frac{n u}{1-n u}, $$&#xA;where $u$ is the unit roundoff, so it will give the correct sign so long as $|p(x)| \geq \gamma_{2n}^2\tilde p(x)$. This will hold everywhere except very near the roots. Since you can scale your polynomial so that $|a_i|\leq 1$ also holds in addition to $|x|\leq 1$, the region of failure around a root is about the size $u^{2/m}$, where $m$ is the multiplicity of the root. For single roots this means failure is only possible if the given argument is the root exactly, and for double roots it must be with a few machine epsilons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think there's a way to do this that works in general without relying on extra precision, it's a very ill-conditioned problem near the roots.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-09-01T20:20:54.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27783" PostTypeId="2" ParentId="27777" CreationDate="2017-09-01T21:47:10.290" Score="1" Body="&lt;p&gt;You can use expansion arithmetics [1] combined with arithmetic filters [2] to improve performance. The idea is to first do the computation with standard floating point arithmetics together with bounds indicating whether the sign is accurate, and only use the (more expensive) expansion arithmetic when the sign could not be evaluated. See also my survey article [3] together with the companion implementation in my programming library geogram [4]. It has a standalone implementation as a .h/.cpp pair of files (use multiprecision_PSM from the download section).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://people.eecs.berkeley.edu/~jrs/papers/robustr.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://people.eecs.berkeley.edu/~jrs/papers/robustr.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://hal.inria.fr/inria-00344297/fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.inria.fr/inria-00344297/fr/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;https://hal.inria.fr/hal-01225202&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.inria.fr/hal-01225202&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2017-09-01T21:47:10.290" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27784" PostTypeId="2" ParentId="26369" CreationDate="2017-09-02T06:07:40.723" Score="3" Body="&lt;p&gt;(&lt;strong&gt;Disclaimer:&lt;/strong&gt; I have taken this material from Ref.[1])&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For nonlinear CG, three changes are required for the linear algorithm&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Recursive formula for the stepsize can not be used&lt;/li&gt;&#xA;&lt;li&gt;Time step size $\alpha$ computation is more complicated &lt;/li&gt;&#xA;&lt;li&gt;Several different choices for $\beta$ &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For nonlinear CG, the residuals are set to negation of gradients of the smooth, differentiable function. i.e. $r_{i} = - f'(x_i)$. As with the linear CG, the value of $\alpha$ is computed such that it minimizes $f(x_i + \alpha_i d_i)$ by ensuring that the gradient is orthogonal to the search direction. So now we need a formula to get zeros of expression $[f'(x_i + \alpha_i d_i)]^T d_i$. Either Newton-Raphson (N-R) method (involves computation of the Hessian matrix $f''(x)$) or Secant (S) method can be used. For both methods, the function needs to be twice continuously differentiable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A) According to N-R method, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\alpha = -\frac{f'^T d}{d^T f'' d}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B) According to Secant method,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \alpha = -\sigma \frac{[f'(x)]^T d}{[f'(x+\sigma d)]^T d - [f'(x)]^T d}$$&#xA;where, $\sigma$ is an arbitrary small non-zero number. The Secant method performs exact line search without computing $f''(x)$. Both NR and S methods are terminated when $x$ is reasonably close to the exact solution. For a non quadratic function, repeated steps must be taken along the line until $f'^T$ is zero, thus one CG iteration may involve many N-R iterations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nonlinear CG algorithm is summarized as &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Set $d_0 = r_0 = -f'(x_0)$&lt;/li&gt;&#xA;&lt;li&gt;Find $\alpha$ as discussed earlier&lt;/li&gt;&#xA;&lt;li&gt;Find $x_{i+1} = x_i + \alpha_i d_i$&lt;/li&gt;&#xA;&lt;li&gt;set $r_{i+1} = -f'(x_{i+1})$&lt;/li&gt;&#xA;&lt;li&gt;Find $\beta_{i+1}$ by an appropriate method (such as Fletcher-Reeves or Polak-Ribière formula)&lt;/li&gt;&#xA;&lt;li&gt;then, $d_{i+1} = r_{i+1} + \beta_{i+1} d_i$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have taken most of the material of this answer from an amazing document &#xA;'&lt;strong&gt;Conjugate Gradients Without Agonizing Pain&lt;/strong&gt;' edition $1\frac{1}{4}$&lt;a href=&quot;http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt; [1]&lt;/a&gt;. Highly recommended! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Jonathan R. Shewchuk, '&lt;em&gt;&lt;a href=&quot;http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An introduction to the conjugate-gradient method without the agonizing pain&lt;/a&gt;&lt;/em&gt;', pp.42-43.&lt;/p&gt;&#xA;" OwnerUserId="1567" LastActivityDate="2017-09-02T06:07:40.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27785" PostTypeId="2" ParentId="27779" CreationDate="2017-09-02T08:07:16.933" Score="6" Body="&lt;p&gt;If the superdiagonal and the subdiagonal have the same sign, like in your example, you can scale your matrix to be symmetric. Replace your matrix $C$ with $DCD^{-1}$, with $D=\operatorname{diag}(1,d,d^2,\dots,d^{n-1})$, choosing $d$ so that it becomes symmetric. (I believe it's going to transform $\operatorname{tridiag}(a,b,c)$ into $\operatorname{tridiag}(\sqrt{c/a},b,\sqrt{c/a})$, but I am too lazy to check.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then Matlab will use algorithms for the symmetric eigenproblem, which are faster, more accurate, and always return real eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-09-02T08:07:16.933" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27786" PostTypeId="2" ParentId="14450" CreationDate="2017-09-02T12:32:01.203" Score="1" Body="&lt;p&gt;A few constraints that usually work in scientific SPH (weakly compressible) computations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;particle radius or influence radius, $\Delta$&lt;/strong&gt;: define it arbitrarily to set the resolution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;smoothing radius, $h$&lt;/strong&gt;: it depends on the kernel function, in your case $h=\Delta/4$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;rest density $\rho_0$&lt;/strong&gt;: 1000 for water.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;particle mass, $m$&lt;/strong&gt;: The number of neighbours in 3D should be about $n=50$. The mass can be calculated as: $m=\frac{4\Delta^3\pi}{3n}\rho_0$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Stiffness&lt;/strong&gt;: $c&amp;lt;10u_{max}$, where $u_{max}$ is the maximum velocity expected in the simulation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Viscosity $\mu$&lt;/strong&gt;: The link is dead, so I don't see the equation but is assume $\mu\approx0.01..0.1$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;time step&lt;/strong&gt;: $dt_{max}\approx0.2\min\bigg(\sqrt{\frac{h}{a_{max}}},\frac{h}{c}\bigg)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;initial interparticle distance&lt;/strong&gt;: $dx=\sqrt[3]{\frac{4\Delta^3\pi}{3n}}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20475" LastActivityDate="2017-09-02T12:32:01.203" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27787" PostTypeId="1" CreationDate="2017-09-02T14:45:14.480" Score="4" ViewCount="198" Body="&lt;p&gt;I am interested in reading about the analysis (existence, uniqueness, error estimates) of elliptic problems solved with a Mixed method that uses the Raviart Thomas elements (so far so good, easy to find) but with the presence of hanging nodes,  (I haven't found anything).  I have a  paper from 2002 that claims this that this was a then-current open problem.  Even just a proof of the inf-sup condition would be appreciated.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this still open?  I would really appreciate a link to a paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The normal Raviart Thomas projection doesn't work (as far as I can tell) in the proof of the inf-sup condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="25314" LastActivityDate="2017-09-03T00:13:01.803" Title="Reference Request: Raviart Thomas with hanging nodes" Tags="|finite-element|numerical-analysis|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27788" PostTypeId="2" ParentId="27787" CreationDate="2017-09-03T00:13:01.803" Score="1" Body="&lt;p&gt;I think it is an interesting question. I do not have a specific paper to cite for you. But I am familar with the usage of hanging nodes with Nedelec elements for electromagnetism, and there is some care required regarding the continuity constraints for elements in the vicinity of a hanging node. For mundane H-grad functions, you have to enforce continuity of the solution. For H-curl (Nedelec) it is similar, but you should enforce continuity only for the tangential component (the normal components can/must &quot;float&quot; freely across the hang). When both spaces are in play (for instance, using H-grad for scalar potential and H-curl for electric field) you must enforce both these continuities, which will guarantee that your E-field functions can span the gradients of your potentials (this is essentially the motivation behind the Nedelec functions to begin with, to model the nullspace of the curl operator correctly, the &quot;exact sequence&quot; concept).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect similar &quot;compatibility conditions&quot; to be your guide when using Raviart-Thomas / H-div functions. I'd think that upon the elements near the hanging node you should enforce normal continuity and let the tangential component &quot;float&quot;. I have successfully used this approach to design stable subgridding schemes for FDTD (E-field uses H-curl functions with tangential continuity enforced at the hang, and B-field uses H-div functions with normal continuity enforced at the hang, setting up a similar compatibility between curl of E and span of B). I can point you to my thesis for some details .. but all this CEM might be wandering a bit afield from the PDE that motivated your original question (which sounds, hmm, Stokesy? not really my area).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Title: &#xA;H-, P- and T-Refinement Strategies for the Finite-Difference-Time-Domain (FDTD) Method Developed via Finite-Element (FE) Principles&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linky:&#xA;&lt;a href=&quot;https://etd.ohiolink.edu/pg_10?0::NO:10:P10_ACCESSION_NUM:osu1219064270&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://etd.ohiolink.edu/pg_10?0::NO:10:P10_ACCESSION_NUM:osu1219064270&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2017-09-03T00:13:01.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27789" PostTypeId="1" CreationDate="2017-09-03T11:28:20.713" Score="1" ViewCount="99" Body="&lt;p&gt;I'm working with python vtkDelaunay3D package for a special purpose. I have set some points but I do not know how can I measure euclidean distance between points with this package. Is there anyone who knows measuring euclidean distance between points with this package in python?&lt;/p&gt;&#xA;" OwnerUserId="25321" LastActivityDate="2017-09-03T11:28:20.713" Title="How to measure euclidean distance between points with vtkDelaunay3D package?" Tags="|python|delaunay-triangulation|vtk|" AnswerCount="0" CommentCount="1" ClosedDate="2017-09-04T04:19:07.600" ContentLicense="CC BY-SA 3.0" />
  <row Id="27790" PostTypeId="2" ParentId="27761" CreationDate="2017-09-03T17:43:44.210" Score="4" Body="&lt;p&gt;It seems that your mesh is a Voronoi diagram. In that case, &quot;Natural Neighbors Interpolation&quot; [1] may be a &quot;natural&quot; choice (depending on how you construct the mesh and what you want to do with the derivatives). See also the Wikipedia entry on Natural Interpolation [2]. The interpolant is smooth (it is $C^1$ everywhere except on the data points where it is $C^0$, see [3] and references herein). Derivatives can be evaluated, but computations are a bit involved. However, if the points were you want to evaluate the derivatives are the centers of the Delaunay edges (as it seems to be in your figure), then there are good chances that the expression of the derivatives can be simplified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Sibson, R. (1981). &quot;A brief description of natural neighbor interpolation (Chapter 2)&quot;. In V. Barnett. Interpreting Multivariate Data. Chichester: John Wiley. pp. 21–36&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_neighbor_interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Natural_neighbor_interpolation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;a href=&quot;http://www-umlauf.informatik.uni-kl.de/~bobach/work/publications/dagstuhl06.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www-umlauf.informatik.uni-kl.de/~bobach/work/publications/dagstuhl06.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-09-03T17:49:09.227" LastActivityDate="2017-09-03T17:49:09.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27794" PostTypeId="1" AcceptedAnswerId="27795" CreationDate="2017-09-05T02:51:22.133" Score="4" ViewCount="380" Body="&lt;p&gt;I am curious of whether optimisation algorithms (whatever simplex, active-set quadratic programming, interior point sequential etc.) can fail due to numerical errors and how to avoid them. But I cannot find related literature online. It seems people mostly study the numerical stability of various matrix factorisation methods and solving partial differential equations. So my questions are, why there are not so many such research concerning iterative optimisation techniques? And would you please recommend some?&lt;/p&gt;&#xA;" OwnerUserId="20865" LastActivityDate="2017-09-12T04:23:05.177" Title="Literatures on numerical stability of optimisation algorithms" Tags="|optimization|numerical-analysis|numerics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27795" PostTypeId="2" ParentId="27794" CreationDate="2017-09-05T04:26:10.517" Score="4" Body="&lt;p&gt;Most optimization algorithms are asked to converge to a tolerance that is far from machine precision, and a lot of them have a rate of convergence is slow enough that it is prohibitively expensive to ask for an answer correct to machine precision. So numerical roundoff just wouldn't affect them very much, compared to all the other sources of error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the difference with your other examples: numerical stability of various matrix factorizations is important because you want the answer correct to machine precision and you want the mathematical properties of the answer to be satisfied to machine precision too (e.g., after factorizing $A=LU$ you really would like to expect that $LU\approx A$). And so roundoff can seriously affect their usefulness when they are used as building blocks for other applications, such as numerical optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stability of methods for solving PDEs is not quite same sense of the word, it is more like the mathematical sense of not diverging exponentially due to perturbations, which may include roundoff errors, but includes just perturbations in general. (PDE methods also usually have rather coarse tolerances, not close to machine precision at all.) This isn't the same as for matrix factorizations: a numerically unstable method there could produce an inaccurate result due to accumulation of roundoff errors, even though there is a single answer that is not particularly sensitive to perturbations in the input data—such a method wouldn't even be backward stable.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-09-05T04:26:10.517" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27797" PostTypeId="1" CreationDate="2017-09-05T11:46:30.703" Score="1" ViewCount="134" Body="&lt;p&gt;I have a quadratic binary optimization problem of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\max x^TQx \cr&#xA;&amp;amp;\text{subject to }x\in\mathcal{X}\subseteq\{0,1\}^n,&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{X}$&lt;/span&gt; is the feasible area defined by linear constraints and binary requirement on the variables. I can assume that all entries in the matrix &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; are non-negative. My question is as follows: is it possible to write the function &lt;span class=&quot;math-container&quot;&gt;$x^TQx$&lt;/span&gt; as a &quot;combination&quot; of two non-negative linear functions in the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x^TQx = aF(x)^2 + bG(x)^2+cF(x)G(x) + dF(x) + eG(x) \tag{1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$F(x)=\sum_{i=1}^nf_ix_i + f_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G(x)=\sum_{i=1}^ng_ix_i + g_0$&lt;/span&gt; are linear funtions, and &lt;span class=&quot;math-container&quot;&gt;$a,b,c,d,e\geq 0$&lt;/span&gt; are real/rational constants. Or stated in another way; can I find values for &lt;span class=&quot;math-container&quot;&gt;$f_i,g_i,\ i=0,\dots,n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a,b,c,d,e$&lt;/span&gt; such that (1) is true?&lt;/p&gt;&#xA;" OwnerUserId="25340" LastEditorUserId="20688" LastEditDate="2019-06-22T19:49:21.363" LastActivityDate="2019-06-22T19:49:21.363" Title="Factoring a quadratic function" Tags="|optimization|quadratic-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27799" PostTypeId="2" ParentId="27797" CreationDate="2017-09-05T22:01:13.613" Score="3" Body="&lt;p&gt;Your conjecture can not be correct, for two purely formal reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In $x^T Q x$, if you multiplied this out as a sum $\sum_i \sum_j Q_{ij}x_ix_j$, &lt;em&gt;every&lt;/em&gt; term in the sum is the product of two entries of the vector $x$. On the other hand, in your conjecture the formula (1) contains terms that are &lt;em&gt;linear&lt;/em&gt; in the components of $x$. This can only be correct if you set $d=e=0$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your functions $F,G$ depend on coefficients $f_i,g_i$, of which there are only $2n$. (I'm neglecting $f_0,g_0$ for the same reason as above: they must be zero, because otherwise the resulting function would have terms that are linear in some of the $x_i$.) But $Q$ contains $n^2$ entries. So, for a fixed $x$, the term $x^T Q x$ is an $n^2$ dimensional object depending on the coefficients of $Q$, whereas $aF(x)^2 + bG(x)^2 + cF(x)G(x)$ is only a $2n+3$ dimensional object. These cannot, in general, be the same. In other words, you try to represent $Q$ in a space that is too small.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-09-05T22:01:13.613" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27800" PostTypeId="1" AcceptedAnswerId="27801" CreationDate="2017-09-06T09:27:47.237" Score="4" ViewCount="176" Body="&lt;p&gt;I am learning C/C++ for Scientific Computing and I have a question regarding the usage of scientific libraries for basic operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have to write a small program in C for a bioinformatics project: for some operations I need to use matrices and very basic computations on them (LU for instance). I went through some notes online and it seems like there are different ways to declare matrices in C/C++ like &lt;code&gt;[][]&lt;/code&gt; or by using double pointers. It is very confusing as a beginner and many say that some of these methods are not efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I went through some examples using GSL: it seems really easy and elegant. I will probably just use the functions to declare and set matrices and to use LU decomposition. My question is then: does it make sense to use this library for such a small project? Even simpler: does it make sense to use it just to avoid the complications related to using multidimensional array in C/C++?&lt;/p&gt;&#xA;" OwnerUserId="22185" LastActivityDate="2017-09-06T12:21:36.350" Title="Using GSL for basic operations" Tags="|c++|matrix|c|libraries|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27801" PostTypeId="2" ParentId="27800" CreationDate="2017-09-06T12:21:36.350" Score="6" Body="&lt;p&gt;Of course it makes sense to use the GSL (or another library for that matter) for several reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Don't reinvent the wheel. The work has been done, you can spend your time on more useful things. &lt;/li&gt;&#xA;&lt;li&gt;If you do decide to implement these basic things yourself, the risk  that your code will probably contain some bugs and will be slower, less memory efficient etc is quite high. Not because I think you are a lousy programmer but simply because these &quot;established&quot; libraries have multiple developers and large user bases in different applications (many eyes see more than two, many brains are , in general, smarter than a single one, ...)&lt;/li&gt;&#xA;&lt;li&gt;If you're interested in what's going on behind the curtains  by all means read the documentation (user guide, developer's guide, design documents, ...) and even the source code. You will learn a lot ...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I don't take a position on GSL itself, these are observations valid for any &quot;established &quot; ( open source or commercial) library.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2017-09-06T12:21:36.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27802" PostTypeId="1" AcceptedAnswerId="27806" CreationDate="2017-09-06T19:49:31.787" Score="1" ViewCount="196" Body="&lt;p&gt;I'm pretty new to translating simulation to reality so please forgive the perhaps naive approach I'm taking here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have a (quasi-2D) experimental video of a certain concentration changing with time, and we wish to determine the diffusion coefficient by comparing the video with a simulation, one can use the discretized diffusion equation (for a 2d square grid $\Delta x$ spacing) to map the behavior: &#xA;$$\frac{\partial}{\partial t} \rho = D \nabla^2 \rho \Rightarrow \rho(i,j,t+\Delta t) = \rho(i,j,t) + \frac{D \Delta t}{\Delta x ^2}[\rho(i+\Delta x, j, t) + \rho(i-\Delta x, j, t) + \rho(i,j+\Delta x, t) + \rho(i, j-\Delta x, t) - 4\rho(i, j, t)]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the parameter of interest is $\frac{D \Delta t}{\Delta x^2}$.&#xA;I remember learning about the Courant number in Computational Physics, ($\frac{v \Delta t}{\Delta x}$) and it made sense the value couldn't be greater than 1, (or the real world was too fast for our simulation), but could be less than 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been given that the magical value for $\frac{D \Delta t}{\Delta x^2}$ is $\frac{1}{2}$ in order to maintain numerical stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now what confuses me is that the only value that the simulation cares about is $\frac{D \Delta t}{\Delta x^2}$, but the value has 3 degrees of freedom (sorry for butchering that term).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So then, I'm allowed to choose $\Delta x$ and $\Delta t$, as long as $\frac{D \Delta t}{\Delta x^2} &amp;lt; .5$, correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what confuses me is that if I take a given value of D and fix $\Delta x$, but change the value of $\Delta t$ by dividing $\frac{D \Delta t}{\Delta x^2}$ by 2 and increasing the number of loops through the diffusion process by 2, I do not receive the same outputs after n &quot;seconds&quot; have gone by (corresponding to x amount of relaxation steps for the simulation with $\frac{D \Delta t}{\Delta x^2}$ and 2x relaxation steps for the simulation with $\frac{1}{2} \frac{D \Delta t}{\Delta x^2}$. This seems to imply that if I were to attempt to find the real $D$ by fitting the experimental video to the simulation output, the value I determine for $D$ is dependent on $\Delta t$, which is certainly not good! And surely, $\Delta x$ would play a role as well then?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone is kind enough to help me with any of these questions that would be really lovely! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) Is it down to an error in the code? (Or maybe the interpretation of it...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) Is this issue a matter of convergence, and if so, how do I know how small to make $\Delta x, \Delta t$? Is there some proportionality with the test value for D?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.) What's up with the $\frac{1}{2}$ value? (I've tried running values of $\frac{D \Delta t}{\Delta x^2}$ closer to $\frac{1}{2}$ and received errors...)&lt;/p&gt;&#xA;" OwnerUserId="25349" LastActivityDate="2017-09-09T00:36:09.400" Title="Computational Physics: Finding the Diffusion Coefficient from the Discretized Diffusion Equation" Tags="|computational-physics|diffusion|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27803" PostTypeId="1" AcceptedAnswerId="27804" CreationDate="2017-09-07T06:18:25.283" Score="1" ViewCount="3239" Body="&lt;p&gt;I am using an FEM code written in Fortran which I did not design. For a particular problem, the program complains that the determinant of the Jacobian matrix is inferior to zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I vaguely understand that the Jacobian matrix is related to the shape functions and that it must be inverted to map shape function coordinates to global coordinates. To invert the matrix it must compute the determinant. Therefore I suspect this is a mesh problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone provide added insight into this? What conditions in an FE problem would typically lead to a Jacobian determinant that is less than zero? Why is this a problem? How can it be avoided?&lt;/p&gt;&#xA;" OwnerUserId="22736" LastEditorUserId="9667" LastEditDate="2017-09-07T14:00:07.000" LastActivityDate="2017-09-07T14:00:07.000" Title="Determinant of jacobian matrix" Tags="|finite-element|mesh-generation|mesh|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27804" PostTypeId="2" ParentId="27803" CreationDate="2017-09-07T06:51:23.093" Score="4" Body="&lt;p&gt;The determinant of the Jacobian, as a determinant changes its sign when odd permutations of columns (or rows) are applied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine, for simplicity a two dimensional case in which the reference element is a triangle with vertices $V_1$, $V_2$ and $V_3$ (chosen counter-clockwise), and basis functions $1-\lambda-\mu$, $\lambda$ and $\mu$ respectively. The pair of reference coordinates will be defined such as: $F:(\lambda,\mu)\to(x, y)$ where $F=[x,y]^T=F(\lambda,\mu)$ and it is given by:&#xA;$$F(\lambda,\mu) = V_1(1-\lambda-\mu)+V_2\lambda + V_3\mu$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear that the Jacobian is:&#xA;$$J=\vec{\textrm{grad}}\,F=[V_2-V_1,V_3-V_1]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that the determinant of this Jacobian coincides with two times the area of the triangle: $det(J) = (y_3-y_1)(x_2-x_1)-(x_3-x_1)(y_2-y_1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the contrary, if vertices are chosen in a clockwise fashion, the Jacobian turns to:&#xA;$$\vec{\textrm{grad}}\,F=[V_3-V_1,V_2-V_1]$$&#xA;interchanging its columns resulting in a negative determinant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other causes may result from the fact  that the vertices are not ordered consecutively, leading to edge crosses which cause a negative area (or volume in 3D).&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-09-07T08:40:12.843" LastActivityDate="2017-09-07T08:40:12.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27805" PostTypeId="1" CreationDate="2017-09-07T09:55:27.543" Score="2" ViewCount="576" Body="&lt;p&gt;My goal is to compute the k smallest eigenvalues of large symmetric sparse matrices. For this purpose I use python scipy's eigsh method in shift-invert mode which uses ARPACK. The matrices usually have around 900,000-1,000,000 rows. Now the computation time for these matrices on my system is normally around 30 minutes. The larger the matrices, the longer it takes - makes sense to me. Then there's one matrix that has only around 780,000 rows for which the computation time explodes. It takes around 10 hours for it to finish. Nothing else of importance running on the machine that would occupy noteworthy resources. I ran it three times, it always takes this long, and I doublechecked that I computed the matrix correctly. The only thing I noticed is that this matrix is even more sparse than the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, what could be happening here? What else can influence the computation time in this way except for the size of the matrix? Unfortunately I'm not that into linear algebra that I really understand what exactly ARPACK does (meaning, how Lanczos algorithm works).&#xA;Any experts here?&lt;/p&gt;&#xA;" OwnerUserId="25352" LastActivityDate="2017-09-07T17:55:07.997" Title="Computation time of eigenvalues with ARPACK depends on what?" Tags="|scipy|arpack|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27806" PostTypeId="2" ParentId="27802" CreationDate="2017-09-07T12:08:51.863" Score="0" Body="&lt;p&gt;Your question contains many components, I will try to answer each individually&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) Is it down to an error in the code? (Or maybe the interpretation of it...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two main concepts in explicit finite difference solution of a linear parabolic equation (such as the transient diffusion equation). The first is stability. In order for your scheme to be stable, you need to have &#xA;$$\frac{\Delta t D}{(\Delta x)^2} &amp;lt; 0.5 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will you to a scheme that is stable in time and will not generate non-physical oscillations.&#xA;The second one is accuracy. You are using first order in time explicit Euler scheme and second order in space centered discretization scheme. Therefore, the leading term of the norm (such as the $L^2$ norm) of your error will be :&#xA;$$ e(\rho) \approx A_1O(\Delta x^2) + A_2 O (\Delta t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, even if your scheme is stable, you are committing an error compared to the exact solution that is proportional to $\Delta x^2$ or $\Delta t$. Therefore, it is normal that your solution changes as you decrease $\Delta x$ and $\Delta t$ even if your stability condition is reached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Gven a $\Delta x$, you can choose a $\Delta t$ for which your solution will not change significantly even if you decrease the time step by a factor of two. This means that your spatial discretization is now dominating the error term. If you do not find that you are converging to the same by solution by choosing smaller and smaller $\Delta t$ it means there is an error in your code. You can try to validate your code first using an analytical solution or the method of manufactured solution ( see this answer : &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26939/in-practice-what-are-the-most-useful-ways-to-visualize-2d-fluid-flow-to-tell-w/26986#26986&quot;&gt;In practice, what are the most useful ways to visualize 2d fluid flow, to tell what is happening in the simulation? Esp for verification and debugging&lt;/a&gt; )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) The $\frac{1}{2}$ value arises from stability analysis. A crude explanation (not rigorous) can be given taking your equation in 1D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho = D \nabla^2 \rho \Rightarrow \rho(i,t+\Delta t) = \rho(i,t) + \frac{D \Delta t}{\Delta x ^2}[\rho(i+\Delta x, t) + \rho(i-\Delta x, t) - 2\rho(i, t)]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we take a simple example where $\rho(i,t)=0 \forall i  \;\;except\;\;  i=i_0$. Then the scheme for $i_0$ is :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \rho(i_0,j,t+\Delta t) = \rho(i_0,t) + \frac{D \Delta t}{\Delta x ^2}[\rho(i_0+\Delta x, t) + \rho(i_0-\Delta x, t) - 2\rho(i_0, t)]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But given the initial conditions on $\rho$ we get :&#xA;$$ \rho(i_0,j,t+\Delta t) = \rho(i_0,t) + \frac{D \Delta t}{\Delta x ^2}[ - 2\rho(i_0, t)]$$&#xA;This means that if $\frac{D \Delta t}{\Delta x ^2}&amp;gt;\frac{1}{2}$ then the concentration will become negative after a single iteration, something that is absolutely non-physical. Thus, for your scheme to remain stable you need $\frac{D \Delta t}{\Delta x ^2}&amp;lt;\frac{1}{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is really an hand waving explanation, but many books on the finite difference demonstrate much more rigourous ways to obtain this stability criterion (using the spectral radius of the finite difference scheme)&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2017-09-09T00:36:09.400" LastActivityDate="2017-09-09T00:36:09.400" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27808" PostTypeId="1" CreationDate="2017-09-07T14:53:37.367" Score="1" ViewCount="974" Body="&lt;p&gt;I have been trying to solve Time Independent Schrodinger's equation in one dimension using Numerov Method as discussed in this excellent lecture notes I found on net.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Numerov method can solve an equation of the following kind: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{{d^2}y}{dx^2}=-g(x) y(x) +s(x) $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can compare this with out Time Independent Schrodinger Equation : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{{d^2}\psi}{dx^2}=-\frac{2m}{\bar h^2} \big(E-V) \psi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $ g(x)= -\frac{2m}{\bar h^2}(E-V) $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can discretize the equation using Taylor expansion and write the iterative formula as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{i+1}= \frac{\psi_i\big[2-\frac{5}{6}\cdot h^2 g_i]-\psi_{i-1}\big[1+\frac{h^2}{12}\cdot g_{i-1}]      }{(1+\frac{h^2}{12}\cdot g_i)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the known Energy of Harmonic Oscillator I solved the equation and my solutions were working well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also used the shooting method to determine Energy value for a particular Eigen State and got a great result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wlKEy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wlKEy.png&quot; alt=&quot;Solution for First Energy state for a Harmonic Oscillator (plotted using gnuplot)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was succesfully able to solve the schrodinger equation for Most of Even potential including Morse Potential and The Double Well Potential. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Problem arrises if I try to take a non symmetric solution. Because I have taken the symmetry of my wave function into account and just solved for $\psi$ from $x=0 $ to $ x=xmax$, and then took output in a file by multiplying a parity to the $\psi$ based on weather my function is odd or not.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But When I solved the same problem from $x=-xmax $ to $x=xmax$ using same algorithm so that I could devise a general algorithm for Numerov Method. The output started getting bizarre. While the ground state solution is same , the odd state solutions are getting inverted somehow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hQldZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hQldZ.png&quot; alt=&quot;The solution for same problem when computed across whole mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is a part of my program for the first (working) Numerov Algorithm. (ignore the signchange counter I have used it to determine number of nodes for shooting method) &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cout&amp;lt;&amp;lt;&quot;Enter mesh number&quot;&amp;lt;&amp;lt;endl;&#xA; cin&amp;gt;&amp;gt;n;&#xA;&#xA; cout&amp;lt;&amp;lt;&quot;Enter maximum value of x&quot;&amp;lt;&amp;lt;endl; &#xA; cin&amp;gt;&amp;gt;xmax;&#xA; dx=xmax/n;&#xA;&#xA; if (nodes % 2==0) //Even Nodes, function is even function , i.e. y(-1) =      y(1) and f(-1) = f (1); Thus using algorithm!&#xA;        {&#xA;      y[0]=1.0;&#xA;      y[1]=( y[0]*( 12.0 - 10.0 *f[0] ) )/( 2*f[1] );&#xA;    }&#xA;  else //nodes are odd, i.e there is a node at x=0;&#xA;    {&#xA;      y[0]=0;&#xA;      y[1]=dx;           //arbitrary small value&#xA;    }&#xA;&#xA;    signchange=0;&#xA;&#xA;    //outward integration using algorithm and counting number of sign changes//&#xA;&#xA; for(int i=1; i &amp;lt;= n; i++) &#xA;&#xA;    {&#xA;      y[ i+1 ]= ( ( 12.0 - 10.0*f[i] )*y[i] -( y[i-1]*f[i-1] ) ) / f[i+1]; &#xA;&#xA;     if (y[i] != copysign(y[i],y[i+1]))  &#xA;       ++signchange;&#xA;    }&#xA;//For Output :&#xA;&#xA;if (nodes % 2 == 0) &#xA;    p=1;&#xA;    else&#xA;    p=-1;&#xA;    for (int i = n; i &amp;gt; 0; --i)&#xA;    file&amp;lt;&amp;lt;setw(1)&amp;lt;&amp;lt;-1*x[i]&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;p*y[i]&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;y[i]*y[i]&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;V[i]&amp;lt;&amp;lt;endl;&#xA;&#xA;    //-------x&amp;lt;0-------//&#xA;    for (int i = 0; i &amp;lt;= n; ++i)&#xA;    file&amp;lt;&amp;lt;setw(1)&amp;lt;&amp;lt;x[i]&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;y[i]&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;y[i]*y[i]&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;V[i]&amp;lt;&amp;lt;endl;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;And this is part of the second program which is giving me inverted output. Where I am iterating across the whole mesh &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cout&amp;lt;&amp;lt;&quot;Enter maximum value of x&quot;&amp;lt;&amp;lt;endl;&#xA;cin&amp;gt;&amp;gt;xmax;&#xA;&#xA;xmin=-1*xmax;&#xA;&#xA;h=(2*xmax)/n;&#xA;&#xA;y[0]=0;&#xA;    y[1]=h;&#xA;&#xA;    /*calculating the wave function psi or y at all points for Energy e*/&#xA;        for(int i=1; i &amp;lt;= n; i++) &#xA;&#xA;    {&#xA;      y[ i+1 ]= ( ( 12.0 - 10.0*f[i] )*y[i] -( y[i-1]*f[i-1] ) ) / f[i+1]; &#xA;&#xA;     if (y[i] != copysign(y[i],y[i+1]))  &#xA;       ++signcount;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would love to know why this is happening. It does not make sense to me. But i am also a beginner in Computational Physics. Also I would like to know what should be the best way to solve the equation for Non-Symmetric potentials, for example a Coulumb potential (inverse R). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Since I am trying to solve Harmonic Oscillator problem the wave function should be bound on both sides. So i am taking $\psi_0$ as 0. And $\psi_1$ as some random small value which will be taken in to account when the wave function will be normalized later. This is for the case when I am solving the equatioin across the whole mesh, from $-xmax$ to $xmax$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But When I am solving the equation for symetric potential(i.e. from 0 to xmax), I am using a property of solution that I have already known, that if i am solving for odd Energy state, the wave funcion $\psi_{x=0}$ will always be zero. I have provided my declaration of initial value in code too. I hope this helps, in clarifying my problem a little. &lt;/p&gt;&#xA;" OwnerUserId="25252" LastEditorUserId="25252" LastEditDate="2017-09-07T23:19:07.467" LastActivityDate="2017-09-07T23:19:07.467" Title="One Dimensional Schrodinger's Equation solution using Numerov Method" Tags="|ode|computational-physics|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27809" PostTypeId="2" ParentId="27805" CreationDate="2017-09-07T17:55:07.997" Score="4" Body="&lt;p&gt;The eigen-distribution is governing the convergence of the Lanczos algorithm. When the eigenvalues are &quot;well-separated&quot;, the Lanczos algorithm should be fairly fast. When some eigenvalues are clustered or multiples, computing the clustered (or multiple) eigenvalues slows down the convergence. The spread of the eigenvalues also matters (i.e. the ratio between the smallest to the largest). For further details, you can look at the ARPACK user's guide (and/or eigenvalue estimates in Parlett, &quot;The symmetric eigenvalue problem&quot;). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you ask for k-10, k-5, k-2, k, k+2, k+5, k+10 eigenvalues, does the time change dramatically? If so, it indicates the presence of &quot;clustered/multiple&quot; eigenvalues.&lt;/li&gt;&#xA;&lt;li&gt;You could increase the size of the Krylov subspace (i.e. the parameter ncv). &lt;/li&gt;&#xA;&lt;li&gt;The default tolerance on eigsh is machine precision. You could increase the tolerance.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2017-09-07T17:55:07.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27811" PostTypeId="2" ParentId="27802" CreationDate="2017-09-08T18:16:41.350" Score="1" Body="&lt;p&gt;Just wanted to point out that if your grid spacing is $\Delta x$ and $\Delta y$ in $x$ and $y$ directions respectively, the stability criterion for the diffusion equation in 2-D is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ &#xA;D \Delta t \left[\frac{1}{\Delta x^2} + \frac{1}{\Delta y^2} \right] \le \frac12.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, when $\Delta x = \Delta y$, the above reduces to&#xA;$$&#xA;\frac{D \Delta t}{\Delta x^2} \le \frac14.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="1156" LastActivityDate="2017-09-08T18:16:41.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27812" PostTypeId="2" ParentId="27797" CreationDate="2017-09-08T18:54:18.573" Score="1" Body="&lt;p&gt;The answer is &quot;yes&quot; if the matrix $Q$ is at most rank-2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we note that for the equality to hold for all $x$, we must&#xA;have $d=e=f_{0}=g_{0}=0$. The reason is that function $\Phi(x)=x^{T}Qx$&#xA;is quadratic homogenous, meaning it satisfies $\Phi(0)=0$ and $\Phi(\alpha x)=\alpha^{2}\Phi(x)$,&#xA;whereas (1) would not be quadratic homogenous if these values weren't&#xA;set to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, we show that the RHS is the quadratic form of a rank-2 matrix.&#xA;First, note that with $d=e=0,$ we have&#xA;$$&#xA;\text{(1) =}\begin{bmatrix}F(x)\\&#xA;G(x)&#xA;\end{bmatrix}^{T}\begin{bmatrix}a &amp;amp; \frac{1}{2}c\\&#xA;\frac{1}{2}c &amp;amp; b&#xA;\end{bmatrix}\begin{bmatrix}F(x)\\&#xA;G(x)&#xA;\end{bmatrix}&#xA;$$&#xA;and that with $f_{0}=g_{0}=0$, we have&#xA;$$&#xA;\begin{bmatrix}F(x)\\&#xA;G(x)&#xA;\end{bmatrix}=\begin{bmatrix}f_{1} &amp;amp; \cdots &amp;amp; f_{n}\\&#xA;g_{1} &amp;amp; \cdots &amp;amp; g_{n}&#xA;\end{bmatrix}\begin{bmatrix}x_{1}\\&#xA;\vdots\\&#xA;x_{n}&#xA;\end{bmatrix}=\begin{bmatrix}f &amp;amp; g\end{bmatrix}^{T}x,&#xA;$$&#xA;by defining $f=[f_{1},\ldots,f_{n}]^{T}$ and similarly for $g=[f_{1},\ldots,f_{n}]^{T}$.&#xA;Combining the two and applying Gaussian elimination yields&#xA;$$\begin{align*}&#xA;(1) &amp;amp; =x^{T}\begin{bmatrix}f &amp;amp; g\end{bmatrix}\begin{bmatrix}a &amp;amp; \frac{1}{2}c\\&#xA;\frac{1}{2}c &amp;amp; b&#xA;\end{bmatrix}\begin{bmatrix}f &amp;amp; g\end{bmatrix}^{T}x.\\&#xA; &amp;amp; =x^{T}\begin{bmatrix}f &amp;amp; g\end{bmatrix}\begin{bmatrix}1 &amp;amp; 0\\&#xA;\frac{1}{2}c &amp;amp; 1&#xA;\end{bmatrix}\begin{bmatrix}a &amp;amp; 0\\&#xA;0 &amp;amp; b-\frac{1}{4}c^{2}/a&#xA;\end{bmatrix}\begin{bmatrix}1 &amp;amp; 0\\&#xA;\frac{1}{2}c &amp;amp; 1&#xA;\end{bmatrix}^{T}\begin{bmatrix}f &amp;amp; g\end{bmatrix}^{T}x\\&#xA; &amp;amp; =x^{T}\left(\begin{bmatrix}f &amp;amp; g\end{bmatrix}\begin{bmatrix}1 &amp;amp; 0\\&#xA;\frac{1}{2}c &amp;amp; 1&#xA;\end{bmatrix}\right)\begin{bmatrix}a &amp;amp; 0\\&#xA;0 &amp;amp; b-\frac{1}{4}c^{2}/a&#xA;\end{bmatrix}\left(\begin{bmatrix}f &amp;amp; g\end{bmatrix}\begin{bmatrix}1 &amp;amp; 0\\&#xA;\frac{1}{2}c &amp;amp; 1&#xA;\end{bmatrix}\right)^{T}x\\&#xA; &amp;amp; =x^{T}\left(\mu_{1}v_{1}v_{1}^{T}+\mu_{2}v_{2}v_{2}^{T}\right)x&#xA;\end{align*}$$&#xA;where&#xA;$$&#xA;\mu_{1}=a,\quad\mu_{2}=b-\frac{c^{2}}{4a},\quad v_{1}=f+\frac{c}{2}g,\quad v_{2}=g.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we use the above result to write a closed-form solution.&#xA;Given a rank-2 decomposition for the left-hand side $Q=\mu_{1}v_{1}v_{1}^{T}+\mu_{2}v_{2}v_{2}^{T}$,&#xA;we pick any choice of $c$ and write&#xA;$$&#xA;a=\mu_{1},\quad b=\mu_{2}+\frac{c^{2}}{4a},\quad G(x)=v_{2}^{T}x,\quad F(x)=(v_{1}-\frac{c}{2}v_{2})^{T}x&#xA;$$&#xA;to yield the desired decomposition. The decomposition is clearly nonunique,&#xA;because it remains valid for any choice of $c$ that conforms to the&#xA;equation above.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastActivityDate="2017-09-08T18:54:18.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27813" PostTypeId="1" CreationDate="2017-09-09T02:04:53.197" Score="1" ViewCount="229" Body="&lt;p&gt;I am trying to make a code for 1D shallow water equation (nonlinear without source terms) using the MacCormack method for sinusoidal wave propagation. My issue is that the wave fluctuates and does not produce a smooth result. I think there might be an issue with my boundary conditions as I am giving boundary condition in linear terms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;h=a\sin\left(\frac{2\pi n}{T}\right)\\&#xA;u=\left(\frac{g}{h_0}\right)^{1/2}h&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $n$ is temporal, $a$ is $0.1$, and $h_0$ is $1$ meter. $\Delta t$ is $0.01$ and $\Delta x$ is $0.1$. If I understood my problem correctly, please tell me how I can give boundary conditions for the nonlinear terms of shallow water equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ikW61.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ikW61.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25362" LastEditorUserId="9804" LastEditDate="2017-09-09T07:11:46.453" LastActivityDate="2017-09-09T07:11:46.453" Title="shallow water equation maccormack method" Tags="|boundary-conditions|nonlinear-equations|wave-propagation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27814" PostTypeId="1" AcceptedAnswerId="27819" CreationDate="2017-09-09T04:43:40.530" Score="3" ViewCount="122" Body="&lt;p&gt;Cross-posted from Math Stackexchange: &lt;a href=&quot;https://math.stackexchange.com/questions/2421964/approximating-solutions-to-quadratic-recurrence&quot;&gt;https://math.stackexchange.com/questions/2421964/approximating-solutions-to-quadratic-recurrence&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a branching process problem that has been reduced to solving a system of recurrences defined for $0 \leq i \leq T$, for some large positive integer $T$ and fixed constants $d, \mu, \beta$ by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_{i} = d + (1-d)((1-\mu-\beta)p_{i}^{2} + \mu p_{i}p_{i+1} + \beta p_{i}p_{i-1}), p_{-1} = 0, p_{T} = 1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, $0 &amp;lt; d &amp;lt; 1$ and $0 &amp;lt; \mu + \beta &amp;lt; 1$. &#xA;Since this is a non-homogeneous quadratic recurrence, I highly doubt that there exist closed form solutions for the $p_i$. Does anyone know of a way to instead approximate the $p_i$, either with an analytical approach or with a computer algorithm? I see that there is a post here regarding a more general form of this problem: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21419/algorithm-for-solving-system-of-quadratic-equations-and-linear-equations&quot;&gt;Algorithm for solving system of quadratic equations and linear equations&lt;/a&gt;. However, given the extreme sparsity of the corresponding matrix here, I feel like there must be an easier solution. Any help is appreciated. Thanks. &lt;/p&gt;&#xA;" OwnerUserId="25363" LastEditorUserId="713" LastEditDate="2017-09-09T18:45:41.213" LastActivityDate="2017-09-09T23:21:14.117" Title="Approximating solutions to quadratic recurrence boundary value problem" Tags="|nonlinear-equations|quadratic-programming|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27815" PostTypeId="2" ParentId="23200" CreationDate="2017-09-09T14:30:51.183" Score="1" Body="&lt;p&gt;Read &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S1877050915010492&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/article/pii/S1877050915010492&lt;/a&gt; &quot;Nonsymmetric Preconditioning for Conjugate Gradient and Steepest Descent Methods&quot;&lt;/p&gt;&#xA;" OwnerUserId="25364" LastActivityDate="2017-09-09T14:30:51.183" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27816" PostTypeId="1" CreationDate="2017-09-09T16:19:39.560" Score="3" ViewCount="608" Body="&lt;p&gt;I have a numerical optimization problem that I am trying to scale appropriately, in order to allow for the solver to achieve faster and more accurate results. I found a paper &lt;a href=&quot;https://dspace.mit.edu/handle/1721.1/8608&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; that had a short section on nondimensionalization on page 44, that simply stated the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In order to nondimensionalize a quantity, just multiply by the appropriate constant.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The nondimensionalization constants are given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$n_{length}=\frac{1}{R_{P}}$$&#xA;$$n_{velocity}=\sqrt{\frac{R_{P}}{\mu}}$$&#xA;$$n_{mass}=\frac{1}{m_{0}}$$&#xA;$$n_{time}=\frac{n_{length}}{n_{velocity}}$$&#xA;$$n_{force}=\frac{n_{mass}n_{length}}{n_{time}^{2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $R_{P}$ is the planet's radius, $\mu=GM$ is the planet's gravitational parameter and $m_{0}$ is the spacecraft's initial mass. I have very similar equations to those used in the thesis, where the following system of first-order ODEs describes the spacecraft's dynamics:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot{r_{x}} = v_{x}$$&#xA;$$\dot{r_{y}} = v_{y}$$&#xA;$$\dot{v_{x}} = -\frac{GMr_{x}}{||\mathbf{r}||^{3}}+\frac{T_{x}}{m}$$&#xA;$$\dot{v_{y}} = -\frac{GMr_{y}}{||\mathbf{r}||^{3}}+\frac{T_{y}}{m}$$&#xA;$$\dot{m} = -\frac{||\mathbf{T}||}{g_{0}I_{sp}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a little confused how I should go about actually implementing the nondimensionalization in order to discretize the equations in their scaled/nondimensionalized form. From my simple understanding of the topic, my nondimensionalized equations would look as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: In response to Kirill's answer, would the transformed system of equations instead look as follows?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot{\tilde{r_{x}}} = \frac{n_{length}}{n_{time}}v_{x}$$&#xA;$$\dot{\tilde{r_{y}}} = \frac{n_{length}}{n_{time}}v_{y}$$&#xA;$$\dot{\tilde{v}_{x}} = \frac{n_{force}}{n_{mass}}\left(-\frac{GMr_{x}}{||\mathbf{r}||^{3}}+\frac{T_{x}}{m}\right)$$&#xA;$$\dot{\tilde{v}_{y}} = \frac{n_{force}}{n_{mass}}\left(-\frac{GMr_{y}}{||\mathbf{r}||^{3}}+\frac{T_{y}}{m}\right)$$&#xA;$$\dot{\tilde{m}} = \frac{n_{mass}}{n_{time}}\left(-\frac{||\mathbf{T}||}{g_{0}I_{sp}}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT 2: Since I'm not really understanding the answers given (apologies, I'm a bit slow), and can't seem to find any well explained examples online, I thought it may help if I show some of the code from the optimization problem I'm working on. Given the original system of ODEs, they are discretized using $N$ nodes, where the final time is given by $t_{f}$ and the time-step between nodes is given by $dt=t_{f}/N$. Thus, using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Trapezoidal_rule_(differential_equations)&quot; rel=&quot;nofollow noreferrer&quot;&gt;trapezoidal integration scheme&lt;/a&gt;, the system of ODEs is discretized as follows in MATLAB,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for t = 2:N&#xA;   y(:,t) = (I-0.5*dt*A(:,:,t))\((I+0.5*dt*A(:,:,t-1))*y(:,t-1)+0.5*dt*B(u(:,t)+u(:,t-1)));&#xA;   .&#xA;   .&#xA;   .&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the state vector is given by $y = [r_{x},r_{y},v_{x},v_{y},m]^{T}$ and the control vector is given by $u = [T_{x},T_{y},||\mathbf{T}||]^{T}$, with $A$ given by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = [0 0 1 0 0;&#xA;     0 0 0 1 0;&#xA;     a 0 0 0 0;&#xA;     0 a 0 0 0;&#xA;     0 0 0 0 0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where&lt;code&gt;a = -(G*M)/(norm(r(:,t)).^3)&lt;/code&gt;, and&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;B = [0 0 0;&#xA;     0 0 0;&#xA;     1/m 0 0;&#xA;     0 1/m 0;&#xA;     0 0 -1/(g0*Isp)];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How could the above system be scaled? I thought about simply applying the following diagonal preconditioner to both the left and right-hand sides of the discretized equations&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M_scale = [&#xA;    n_length/n_time 0 0 0 0;&#xA;    0 n_length/n_time 0 0 0;&#xA;    0 0 n_force/n_mass 0 0;&#xA;    0 0 0 n_force/n_mass 0;&#xA;    0 0 0 0 n_mass/n_time];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where I then have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for t = 2:N&#xA;   M_scale*y(:,t) = M_scale*(I-0.5*dt*A(:,:,t))\((I+0.5*dt*A(:,:,t-1))*y(:,t-1)+0.5*dt*B(u(:,t)+u(:,t-1)));&#xA;   .&#xA;   .&#xA;   .&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but unfortunately this doesn't seem to work.&lt;/p&gt;&#xA;" OwnerUserId="21148" LastEditorUserId="21148" LastEditDate="2017-09-20T21:19:17.397" LastActivityDate="2017-09-20T21:19:17.397" Title="Scaling/nondimensionalization for numerical optimization" Tags="|optimization|constrained-optimization|convergence|numerical-modelling|scaling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27817" PostTypeId="2" ParentId="27816" CreationDate="2017-09-09T18:06:23.787" Score="6" Body="&lt;p&gt;One thing that makes your non-dimensionalized ODE confusing is that you use the same symbols for dimensionalized and nondimensionalized variables, even though they are different variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider writing it in a different way: let there be a new set of variables, $\tilde t, \tilde x,\ldots$, all unitless, defined by&#xA;$$ \tilde t = n_{\mathrm{time}}t, \qquad \tilde x = n_{\mathrm{length}} x, \qquad \ldots $$&#xA;Then it's a straightforward change of variables, from $(x(t),\ldots)$ to $(\tilde x(\tilde t), \ldots)$, applied to the system of ODEs. You get&#xA;$$ \frac{\mathrm{d}\tilde x}{\mathrm{d}\tilde t} = \frac{n_{\mathrm{length}}}{n_{\mathrm{time}}} \dot x = \frac{n_{\mathrm{length}}}{n_{\mathrm{time}}} v_x = \frac{n_{\mathrm{length}}}{n_{\mathrm{time}}} \frac{\tilde v_x}{n_{\mathrm{length}}/n_{\mathrm{time}}} = \tilde v_x. $$&#xA;The slight danger in not doing this explicitly is that you might accidentally produce an inequivalent system of ODEs. The transformation between the two sets of variables must be an equivalence transformation, and so it should not change the actual system dynamics. Simply multiplying the r.h.s. by the factors is not valid for this reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason it's called nondimensionalization is because you change the units in which you measure all the quantities that enter the ODE, but the ODE itself remains the same. This is not the case for the ODE you gave, because if you look at $\dot x = \frac{n_l}{n_t}v_x$, this contradicts even the very definition of velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When your quote says &quot;In order to nondimensionalize a quantity, just multiply by the appropriate constant&quot;, it means precisely that: you get a new variable by multiplying an original variable by a constant, but this is not quite the same as multiplying the r.h.s. of the ODE by that constant.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-09-09T18:40:54.687" LastActivityDate="2017-09-09T18:40:54.687" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27818" PostTypeId="2" ParentId="27775" CreationDate="2017-09-09T18:48:17.237" Score="2" Body="&lt;p&gt;Edges are represented as sets of vertices. With classical graphs, an edge can be represented by the set containing its 2 endpoints. With hypergraphs, they are represented by a set containing more than 2 nodes e.g. $e_i = \lbrace v_1, v_2, ... , v_n \rbrace$.&lt;br&gt;&#xA;Incidence matrices are straightforward, just look at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hypergraph&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia page&lt;/a&gt;. About incidence matrices you may be interested by &lt;a href=&quot;https://books.google.fr/books?id=DWZc9Ti_vBwC&amp;amp;pg=PA312&amp;amp;lpg=PA312&amp;amp;dq=hypergraph+adjacency+matrix&amp;amp;source=bl&amp;amp;ots=2U-r4yz3W5&amp;amp;sig=8nJ5RZ2V8elXWFcPxby0X0msxk8&amp;amp;hl=fr&amp;amp;sa=X&amp;amp;ved=0ahUKEwjvidfC3pjWAhWSaVAKHYUKCrIQ6AEIOzAC#v=onepage&amp;amp;q=hypergraph%20adjacency%20matrix&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24585" LastActivityDate="2017-09-09T18:48:17.237" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27819" PostTypeId="2" ParentId="27814" CreationDate="2017-09-09T20:35:56.863" Score="3" Body="&lt;p&gt;Newton-type methods are standard methods for solving system of nonlinear equations numerically. To make sure they can handle this, I tried solving your problem with &lt;a href=&quot;https://github.com/JuliaNLSolvers/NLsolve.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLsolve.jl&lt;/a&gt; in julia, and it seems to work just fine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using NLsolve&#xA;using LineSearches&#xA;using ForwardDiff&#xA;&#xA;function solve(d::Real, μ::Real, β::Real, T::Integer)&#xA;    @assert 0 &amp;lt; d &amp;lt; 1&#xA;    @assert 0 &amp;lt; μ + β &amp;lt; 1&#xA;    n = T + 1&#xA;    rhs(p₋, p, p₊) = -p + d + (1-d)*p*((1-μ-β)*p+μ*p₊+β*p₋)&#xA;    function f!(p, val)&#xA;        val[1] = p[1]&#xA;        for k = 2:T&#xA;            val[k] = rhs(p[k-1], p[k], p[k+1])&#xA;        end&#xA;        val[n] = p[n] - 1&#xA;    end&#xA;    function g!(p, J)&#xA;        J[1,1] = J[n,n] = 1&#xA;        for k = 2:T&#xA;            J[k,k-1:k+1] = ForwardDiff.gradient(x -&amp;gt; rhs(x...), p[k-1:k+1])&#xA;        end&#xA;    end&#xA;    Jₚ = spzeros(n, n)&#xA;    Jₚ[1,1] = Jₚ[n,n] = 1&#xA;    for k = 2:T; Jₚ[k,k-1:k+1] = 1; end&#xA;    p₀ = ones(n) * min(1, d/(1-d))&#xA;    df = DifferentiableGivenSparseMultivariateFunction(f!, g!, Jₚ)&#xA;    @time sol = nlsolve(df, p₀, method=:newton, linesearch! = LineSearches.strongwolfe!, show_trace = true)&#xA;    g!(sol.zero, Jₚ)&#xA;    @printf &quot;Jacobian condition number: %.2e\n&quot; cond(Array(Jₚ), Inf)&#xA;&#xA;    sol&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I noticed that the case $\mu+\beta\approx 1$ seems to be the hard case here, and the solution depends on a good initial guess, because the equation is stiff and there are boundary layers at $0$ and $T$. So this isn't a complete answer, it just works in the non-extreme cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking, one can't expect such a problem to have a closed form solution. Also, Groebner bases are feasible on relatively small systems, not like this one. It is true that the Jacobian here is sparse, but that's not really enough. Simply having a &lt;em&gt;sparse&lt;/em&gt; system of nonlinear equations doesn't necessarily mean there will be a simple efficient way of solving them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, as mentioned in &lt;a href=&quot;https://mathoverflow.net/questions/153436&quot;&gt;this question on MO&lt;/a&gt;, multivariate systems of quadratic equations are a very difficult type of problem in general, at least NP-hard and possibly harder.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-09-09T20:35:56.863" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27820" PostTypeId="2" ParentId="27814" CreationDate="2017-09-09T23:21:14.117" Score="2" Body="&lt;p&gt;In addition to Kirill's excellent answer, one thing I think work mentioning is Smale's $\alpha$-theory. One of the hard parts about applying Newton's method is that, if you don't make a really good initial guess, the iteration can diverge. To remedy this problem, it's common to use a damped Newton iteration, but to select the damping parameter all you've got are heuristics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Smale's theory tells you sufficient conditions for when an initial guess to Newton's method is in the quadratic convergence region of some root of a nonlinear equation when the function is analytic. The sufficiency condition requires that you have uniform bounds on the Taylor coefficients of the function in question. For polynomials, the conditions are especially easy to evaluate since the Taylor coefficients are 0 after the degree of the polynomial, in your case 2. Additionally, the sparsity of the system you're solving makes evaluating the conditions to apply Smale's theory even simpler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My old officemate has &lt;a href=&quot;http://www.cswiercz.info/assets/files/narc-smale.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;some slides&lt;/a&gt; that I think are a really good reference for the subject, and he also has some &lt;a href=&quot;https://github.com/cswiercz/smale&quot; rel=&quot;nofollow noreferrer&quot;&gt;example code&lt;/a&gt; in python.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-09-09T23:21:14.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27821" PostTypeId="1" CreationDate="2017-09-09T23:38:54.613" Score="1" ViewCount="206" Body="&lt;p&gt;I'm in the context of this publication: &lt;a href=&quot;http://www.gilbertbernstein.com/resources/booleans2009.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.gilbertbernstein.com/resources/booleans2009.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I applied quantization to my point coordinates: All coordinates are integer lying in [0, 2 power 20]. The C++ type representing a coordinate is int64_t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A plane p is represented in integer homogeneous coordinates (a, b, c, d):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p = { (x, y, z) | ax + by + cz + d = 0 }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A point X can be represented as 3 planes (p, q, r) (provided a validity condition, cf the publication).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a valid point X and a plane s. I want to know if X lies on the positive side, negative side or on s. According to the publication, I need to compute the sign of these determinants:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;| p_a, p_b, p_c, p_d |&#xA;| q_a, q_b, p_c, q_d |&#xA;| r_a, r_b, p_c, r_d |&#xA;| s_a, s_b, p_c, s_d |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;| p_a, p_b, p_c|&#xA;| q_a, q_b, p_c|&#xA;| r_a, r_b, p_c|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The quantization bound implies the following bounds:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|p_a|, |p_b| and |p_c| &amp;lt; 2 pow 40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|p_d| &amp;lt; 2 pow 60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to compute exactly and quickly the sign of that determinant? Using BigInt or anything based on the Chinese Remainder theorem is too slow (I haven't tried the GMP library though)... What libraries that could solve this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Clue&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Simulation of simplicity (&lt;a href=&quot;http://www.cs.sandia.gov/~samitch/unm_math_579/course_papers/p66_edelsbrunner_simulation_of_simplicity.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cs.sandia.gov/~samitch/unm_math_579/course_papers/p66_edelsbrunner_simulation_of_simplicity.pdf&lt;/a&gt;): I convert everything to double and compute the determinant in double. If the absolute value of the result is less than 0.5, I use another method (like something based on the Chinese Remainder theorem). Is there an even faster and yet exact method?&lt;/li&gt;&#xA;&lt;li&gt;How to transform my matrices and/or subdivide the input mesh facets such that &#xA;the exact predicates from Shewchuck: &lt;a href=&quot;http://www.cs.cmu.edu/~quake/robust.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cs.cmu.edu/~quake/robust.html&lt;/a&gt; are applicable?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17299" LastActivityDate="2017-09-10T13:39:09.950" Title="Sign of integer determinant 4 by 4" Tags="|c++|computational-geometry|performance|mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27822" PostTypeId="1" AcceptedAnswerId="27831" CreationDate="2017-09-10T00:22:49.527" Score="0" ViewCount="478" Body="&lt;p&gt;I want to solve&#xA;\begin{align}&#xA;\nabla^4\psi+\alpha\nabla^2\psi+\beta\psi=F(x,y);\quad&#xA;\nabla \psi\cdot \hat{n}=\nabla^3\psi\cdot \hat{n}=0\quad \text{on boundaries}&#xA;\end{align}&#xA;with a 2d FEM scheme. With the biharmonic operator, linear test functions are ruled out.  I can integrate the first term by parts twice, so that the weak version of the PDE includes a term&#xA;\begin{align}&#xA;\int_A\nabla^2\phi_i \nabla^2\phi_j\,da&#xA;\end{align}&#xA;Quadratic test functions will allow me to evaluate the integral.  I understand that quadratic test functions have nodes at the vertices and at the mid-point of each side of each triangle .  My question relates to how to deal with this at assembly.  If the coordinates of the vertices of each triangle are in p, and the connectivity matrix is t, then what is the procedure to extend these for the additional nodes located on the sides of each triangle?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This wonderful reference &lt;a href=&quot;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch23.d/AFEM.Ch23.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.colorado.edu/engineering/CAS/courses.d/AFEM.d/AFEM.Ch23.d/AFEM.Ch23.pdf&lt;/a&gt; '23.2.7. Six Node Quadratic Interpolation' describes the quadratic elements perfectly, but I can't figure out how to integrate the midway points into my node list?&lt;/p&gt;&#xA;" OwnerUserId="25277" LastEditorUserId="25277" LastEditDate="2017-09-10T00:58:08.193" LastActivityDate="2017-09-11T19:17:39.473" Title="quadratic trial functions for a 2d FEM calculation" Tags="|finite-element|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27823" PostTypeId="2" ParentId="27821" CreationDate="2017-09-10T12:55:52.057" Score="2" Body="&lt;p&gt;There are several options for deriving predicates with arbitrary precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If you use integers&lt;/strong&gt;, then you can use Haramard inequality (see standard algebra textbooks, for instance Greub 1975, Linear Algebra, Springer Verlag). This inequality says that the absolute value of a determinant is not greater than the product of the lengths of the column vectors. You can use it to bound the number of bits of the determinant in function of the number of bits of the coefficients. For the 4x4 determinant of the &lt;em&gt;in_circle&lt;/em&gt; predicate of a 2D Voronoi diagram, if coordinates are between $-K$ and $K$, the absolute value of the determinant is smaller than $32K^4$.If you use long integers (64 bits), for a 2D Voronoi diagram, $K=23170$ ensures that you got sufficient precision. If you are in 3D, then 64 bits may not suffice, and you will need to use a bigint package (GNU GMP as you suggested). It is probably possible to first do the computation with standard 64 bit integers and detect dynamically whether an overflow occured, then relaunch the computation with the bigint package only when needed.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If you use floating point numbers&lt;/strong&gt;, see my answer to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/27777/accurate-evaluation-of-the-sign-of-a-polynomial/27783#27783&quot;&gt;this question&lt;/a&gt;, with reference to my survey article and software package. The idea is to compute the determinant with floating point numbers as well as a bound indicating whether the sign is accurate, then relaunch the computation with (more expensive) arbitrary precision if this was not the case. My software package &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt; can be used to derive predicates involved in intersections, as explained in my article &lt;a href=&quot;https://hal.inria.fr/hal-01225202&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;. This uses Shewchuk's expansions (mentioned in the question) combined with an arithmetic filter &lt;a href=&quot;https://www.researchgate.net/publication/29616057_FPG_A_code_generator_for_fast_and_certified_geometric_predicates&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;. This strategy is easier to implement than the adaptive evaluation used in Shewchuk's original package, and it is reasonably easy to derive the new predicates that you will need for your intersections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;more thoughts / references:&lt;/strong&gt; I recommend the following article by a friend of mine (&lt;a href=&quot;http://kinovis.inrialpes.fr/static/QuickCSG/&quot; rel=&quot;nofollow noreferrer&quot;&gt;QuickCSG&lt;/a&gt;) on the very same topic (robust evaluation of boolean operations on meshes). It recasts the intersection problem as an algorithm that only depends on a very small number of predicates, for which valid simulation of simplicity can be reasonably easily defined.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-09-10T13:05:54.723" LastActivityDate="2017-09-10T13:05:54.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27824" PostTypeId="2" ParentId="27821" CreationDate="2017-09-10T13:39:09.950" Score="1" Body="&lt;p&gt;Essentially, we are trying to compute the determinant of integer matrices. See if these helps:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Exact Determinant of Integer Matrices, &lt;em&gt;Takeshi Ogita&lt;/em&gt;&#xA;  &lt;a href=&quot;http://www.eng.nus.edu.sg/civil/REC2010/documents/papers/038.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.eng.nus.edu.sg/civil/REC2010/documents/papers/038.pdf&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Computing the sign or the value of the determinant of an integer&#xA;  matrix: A complexity survey, &lt;em&gt;Erich Kaltofena, Gilles Villard&lt;/em&gt;&#xA;  &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.21.9676&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.21.9676&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also, if we think in more geometric terms, we could always assemble a numerically more stable, easy to compute way to determine this. Let $\mathbf{n}=(a,b,c)$ denote the normal of the plane and $\mathbf{p} \in \mathbb{R}^3$ the point. Let's take a random point on this plane $\mathbf{p}_0=[0,0,-c/d]^T$ and construct a direction vector $\mathbf{d}=\mathbf{p}-\mathbf{p}_0$. The point lies on the positive side of the plane iff:&#xA;$$&#xA;\mathbf{n}\cdot\mathbf{d} &amp;gt; 0 \rightarrow \text{positive side}\\&#xA;\mathbf{n}\cdot\mathbf{d} &amp;lt; 0 \rightarrow \text{negative side}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;doing these operations might be fairly easy also for integer valued/quantized input, where BigInt can actually be used.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2017-09-10T13:39:09.950" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27825" PostTypeId="1" CreationDate="2017-09-10T21:36:18.873" Score="1" ViewCount="217" Body="&lt;p&gt;Let's say I have a matrix $\mathbf{A}\in\mathbb{R}^{n\times n}$, and its associated inverse, $\mathbf{A}^{-1}$. The elements of $\mathbf{A}$ are given in IEEE single precision, i.e. 23-bit mantissa, and I use the lapack routines $\texttt{sgetrf}$ and $\texttt{sgetri}$ to compute $\mathbf{A}^{-1}$ through a single-precision LU-decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How am I to ascertain the error associated with the matrix product $\mathbf{A}\mathbf{A}^{-1}$. More concretely, I'd quite like to be able to approximate, &lt;em&gt;a priori&lt;/em&gt;, what the maximum error, $\varepsilon = \max_i \Big|\big( \mathbf{AA}^{-1}\big)_{ii} - 1.0\Big|$, will be. Furthermore, is there a way to approximate the errors for other matrix inversion routines, such as Gauss-Jordan or a QR decomposition? I'm guessing I should use their computational scaling, e.g. the LU-decomposition scales as $\frac{4}{3}n^3$, but I'm unsure how to proceed beyond that.&lt;/p&gt;&#xA;" OwnerUserId="25375" LastActivityDate="2017-09-10T21:36:18.873" Title="Numerical Precision in Matrix Inversion Routines" Tags="|linear-algebra|matrix|precision|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27826" PostTypeId="2" ParentId="27822" CreationDate="2017-09-11T01:25:38.923" Score="-1" Body="&lt;p&gt;Following @TylerOlson 's suggestion here is an Octave/Matlab script that generates a mesh with six points in each triangle: the three vertices and the three mid-points.  If anyone knows a way to simplify this inelegant code, I would appreciate it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear&#xA;%% Persson's routine to make simple triangles in a rectangular grid&#xA;nx=4;ny=3;&#xA;[x,y]=ndgrid(linspace(-1,1,nx),linspace(-1,1,ny)); % forms x and y lists&#xA;p=[x(:),y(:)]; &#xA;t=[1,nx+2,nx+1;1,2,nx+2]; &#xA;t=kron(t,ones(nx-1,1))+kron(ones(size(t)),(0:nx-2)');&#xA;t=kron(t,ones(ny-1,1))+kron(ones(size(t)),(0:ny-2)'*nx);&#xA;np=length(p(:,1));nt=length(t(:,1));&#xA;%%make a list of all sides from t;  There will be redundant entries&#xA;s=zeros(3*nt,6);&#xA;for kt=1:nt&#xA;  for ks=1:3&#xA;    if ks==1&#xA;      s(3*(kt-1)+ks,1)=min(t(kt,1:2));&#xA;      s(3*(kt-1)+ks,2)=max(t(kt,1:2));&#xA;      s(3*(kt-1)+ks,3)=kt;&#xA;    elseif ks==2&#xA;      s(3*(kt-1)+ks,1)=min(t(kt,2:3));&#xA;      s(3*(kt-1)+ks,2)=max(t(kt,2:3));&#xA;      s(3*(kt-1)+ks,3)=kt;&#xA;    else&#xA;      s(3*(kt-1)+ks,1)=min(t(kt,[1,3]));&#xA;      s(3*(kt-1)+ks,2)=max(t(kt,[1,3]));&#xA;      s(3*(kt-1)+ks,3)=kt;&#xA;    endif&#xA;  endfor&#xA;endfor&#xA;%% once this has been done for all triangles, work on s to eliminate dupes&#xA;for kk=1:max(s(:,1))&#xA;  order1=find(s(:,1)==kk);%s(order1,:)&#xA;  [junk,order2]=sort(s(order1,2));&#xA;  dupe=find(junk(1:end-1)==junk(2:end));&#xA;  n=s(order1(order2),:);&#xA;  for kl=1:length(dupe)&#xA;    n(dupe(end+1-kl),4)=n(dupe(end+1-kl)+1,3);% add on triangle no of dupe&#xA;  endfor&#xA;  [junk,order3,mojunk]=unique(n(:,2),&quot;first&quot;);&#xA;  n=n(order3,:);&#xA;  if kk==1&#xA;    snodupe=n;&#xA;  else&#xA;    snodupe=[snodupe;n];&#xA;  endif&#xA;endfor&#xA;%% add to position table the position of new nodes&#xA;for kn=1:length(snodupe)&#xA;    p(np+kn,1)=0.5*(p(snodupe(kn,1),1)+p(snodupe(kn,2),1));&#xA;    p(np+kn,2)=0.5*(p(snodupe(kn,1),2)+p(snodupe(kn,2),2));&#xA;endfor&#xA;%% build a new connectivity table, t6&#xA;t6=zeros(nt,6);&#xA;for kt=1:nt&#xA;  %t(kt,:)&#xA;  for ks=1:3&#xA;    if ks==1&#xA;      ss=t(kt,1);&#xA;      se=t(kt,2);&#xA;    elseif ks==2&#xA;      ss=t(kt,2);&#xA;      se=t(kt,3);&#xA;    else&#xA;      ss=t(kt,3);&#xA;      se=t(kt,1);&#xA;    endif&#xA;    %% where is this side in snodupe table?&#xA;    %% to lookup in snodupe need ends to be in order&#xA;    s1=min([ss se]);s2=max([ss se]);&#xA;    ind1=find(snodupe(:,1)==s1);&#xA;    ind2=find(snodupe(ind1,2)==s2);&#xA;    sideno=ind1(1)-1+ind2;&#xA;    t6(kt,2*ks-1)=ss;&#xA;    t6(kt,2*ks)=np+sideno;&#xA;  endfor&#xA;endfor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25277" LastEditorUserId="25277" LastEditDate="2017-09-11T03:08:12.860" LastActivityDate="2017-09-11T03:08:12.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27828" PostTypeId="2" ParentId="1169" CreationDate="2017-09-11T07:35:21.590" Score="0" Body="&lt;p&gt;Then consider trying to factor your matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A famous example is the DFT (Discrete Fourier Transform) matrices which are dense, but if you factor them you can get a matrix representation for the FFT (Fast Fourier Transform) where each factor ideally has only 2 nonzero elements each row and there are only $\log(N)$ such matrices where $N$ is the data size.&lt;/p&gt;&#xA;" OwnerUserId="18579" LastActivityDate="2017-09-11T07:35:21.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27829" PostTypeId="1" CreationDate="2017-09-11T07:36:01.400" Score="1" ViewCount="32" Body="&lt;p&gt;Let's say I am running a FDTD simulation of a wave-equation to determine a transfer function of an LTI system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;H(f) = \dfrac{Y(f)}{X(f)}\&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $Y(f)$ and $X(f)$ are Fourier transforms of signals measured at some point in the FDTD grid for a grid with an &quot;obstacle&quot; inserted and an empty grid, respectively, due to an excitation $s(t)$. Assume that it's an acoustic wave simulation and that the &quot;obstacle&quot; is modelling a fully reflective object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: &lt;strong&gt;does the type of an excitation signal&lt;/strong&gt; $s(t)$ matter in that kind of problem? For example, how broadband the signal is? And if does matter, then - how? My intuition tells me since in the end we're computing a relation of two signals, any propagation issues of the input signal will be the same in both simulations and therefore cancel each other in the above equation.&lt;/p&gt;&#xA;" OwnerUserId="25376" LastActivityDate="2017-09-11T07:36:01.400" Title="Does excitation type matter in a time-domain simulation/computation of a transfer function of an LTI system?" Tags="|numerical-analysis|finite-difference|simulation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27831" PostTypeId="2" ParentId="27822" CreationDate="2017-09-11T14:29:52.917" Score="2" Body="&lt;p&gt;Here's the version that I had in mind. It's a bit terse, but there is a commented version up on github (use that one, it actually defines the inputs/outputs and explains the logic for the various intermediate calculations). Based on some initial benchmarking, it beats out your loop/branch-heavy code by about a factor of 300 (Matlab 2017a on a large-ish mesh), since it spends all of its time in vectorized calls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a link to the real code:&#xA;&lt;a href=&quot;https://github.com/tjolsen/Mesh_Utilities/blob/master/linear_to_quad_tris/linear_to_quadratic.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tjolsen/Mesh_Utilities/blob/master/linear_to_quad_tris/linear_to_quadratic.m&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [QuadTris,Points] = linear_to_quadratic(t,p)&#xA;&#xA;nt = size(t,1);&#xA;t_idx = (1:nt)';&#xA;&#xA;all_edges = [t(:,1) t(:,2) t_idx t_idx 3*ones(nt,1);&#xA;t(:,2) t(:,3) t_idx t_idx 1*ones(nt,1);&#xA;t(:,3) t(:,1) t_idx t_idx 2*ones(nt,1)];&#xA;&#xA;forward_edges = all_edges(:,2) &amp;gt; all_edges(:,1);&#xA;all_edges(forward_edges,4) = 0;&#xA;all_edges(~forward_edges,3) = 0;&#xA;all_edges(~forward_edges,[1,2]) = all_edges(~forward_edges,[2,1]);&#xA;&#xA;[unique_edges,~,ic] = unique(all_edges(:,[1,2]), 'rows');&#xA;Edges = [unique_edges, &#xA;         accumarray(ic, all_edges(:,3),[],@sum),&#xA;         accumarray(ic, all_edges(:,4),[],@sum)];&#xA;t2f = accumarray([all_edges(:,3)+all_edges(:,4),all_edges(:,5)], &#xA;                 sign(all_edges(:,3) - all_edges(:,4)).*ic, [], @sum);&#xA;&#xA;newPoints = [(p(Edges(:,1),1)+p(Edges(:,2),1))/2, &#xA;             (p(Edges(:,1),2)+p(Edges(:,2),2))/2];&#xA;Points = [p; newPoints];&#xA;&#xA;np = size(p,1);&#xA;QuadTris = [t, np+(abs(t2f))];&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-09-11T14:29:52.917" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27832" PostTypeId="2" ParentId="27822" CreationDate="2017-09-11T19:17:39.473" Score="4" Body="&lt;p&gt;I suspect that you will be disappointed to find out that, once you are able to implement the quadratic shape functions, the solution you will get is not correct. That's because for the bilaplacian operator ($\nabla^4$) it is not enough to have shape functions that are quadratic polynomials on each cell, but you also need to have special kind of (Hermite) shape functions that have no &quot;kinks&quot; across cell interfaces but are continuously differentiable at faces. These are substantially more difficult to implement than the quadratic shape functions you are looking for, but necessary to produce the correct solution of the equation you are trying to solve.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-09-11T19:17:39.473" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27834" PostTypeId="1" AcceptedAnswerId="27843" CreationDate="2017-09-11T22:28:06.797" Score="8" ViewCount="7122" Body="&lt;p&gt;I am looking for an alternative and robust alternative to Gram-Schmidt orthogonalization, but with one constraint:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a unit vector $\mathbf{v}_1 \in \mathbb{R}^d \,\text{s.t.} \,\|\mathbf{v}_1^T\mathbf{v}_1\|=1$ and I want to find a set of $d-1$ vectors $\{\mathbf{v}_2\cdots \mathbf{v}_{d}\} \perp \mathbf{v}$, orthogonal to $\mathbf{v}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, my goal is not to generate a set of arbitrary vectors, but to find $d-1$ other vectors which form a $d \times d$ square matrix $\mathbf{V}$ with orthonormal columns:&#xA;$$&#xA;\mathbf{V}=&#xA;\begin{bmatrix}&#xA;\mathbf{v}_1 &amp;amp; \mathbf{v}_2  &amp;amp; \cdots &amp;amp; \mathbf{v}_d&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gram-Schmidt and modified Gram-Schmidt algorithms can accomplish that, but it is known that even modified Gram-Schmidt suffers from numerical inaccuracy. There are methods for doing this in 3D such as &lt;a href=&quot;https://pdfs.semanticscholar.org/9db0/6b69e24af51120aa04b44c83df7c0e662dd7.pdf&quot; rel=&quot;noreferrer&quot;&gt;this one&lt;/a&gt;. However, as cross-product does not generalize to n-dimensions we cannot extend the simple 3D methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also aware of the similar posts such as &lt;a href=&quot;https://math.stackexchange.com/questions/924127/alternative-to-the-gram-schmidt-procedure-for-orthogonalization&quot;&gt;this one&lt;/a&gt; but my question is different as the first vector is specified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I am looking for a computationally efficient and accurate method to perform this task. Simplicity is important because I will also like to compute the gradient of this operation. &lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2017-09-13T16:58:20.630" Title="Generate a set of orthogonal vectors to a given vector" Tags="|linear-algebra|computational-geometry|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27835" PostTypeId="2" ParentId="27834" CreationDate="2017-09-12T00:38:44.860" Score="14" Body="&lt;p&gt;In order to find such a set of vectors, you can use the &lt;a href=&quot;https://en.wikipedia.org/wiki/QR_decomposition#Using_Householder_reflections&quot; rel=&quot;noreferrer&quot;&gt;Householder QR factorization&lt;/a&gt;. Let your unit vector $v$ be given. Define a nonsingular matrix $A$ by&#xA;$$&#xA;A = \left(\begin{array}{cccc} v &amp;amp; a_2 &amp;amp; \cdots &amp;amp; a_d \end{array}\right),&#xA;$$&#xA;where it is not so important how you obtain the columns $a_2, \ldots, a_d$ of $A$, as long as $\{v, a_2, \ldots, a_d\}$ forms a linearly independent set of vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the QR factorization of $A$ results in an orthogonal matrix $Q$ such that&#xA;$$&#xA;A = QR,&#xA;$$&#xA;where $R$ is an upper triangular matrix. We can write&#xA;$$&#xA;R = \left( \begin{array}{cccc}&#xA;r_{11} &amp;amp; \times &amp;amp; \cdots &amp;amp; \times \\&#xA;0 &amp;amp; \times &amp;amp; \cdots &amp;amp; \times \\&#xA;%0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \times \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \times&#xA;\end{array}\right).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $Q$ is an orthogonal matrix, its columns, which we will denote $q_j$, form an orthonormal set of vectors. The relationship&#xA;$$ Q^T A = R $$&#xA;tells us that $q_1^T v = r_{11}$, and $q_j^T v = 0$ for $j &amp;gt; 1$. Therefore, we &#xA;have found an orthonormal set of vectors&#xA;$$ \{ v, q_2, \ldots, q_d \}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, since both $q_1$ and $v$ are orthogonal to the remaining $(d-1)$ vectors $q_j$, we have that $q_1$ and $v$ belong to the same one-dimensional subspace and hence are collinear. Since they are both unit vectors, we know $q_1 = \pm v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the numerical stability of this algorithm, if the QR factorization is performed using Householder reflectors, then $Q$ is generated as a product of orthogonal matrices $Q_d Q_{d-1} \cdots Q_2 Q_1$ of the form $I - 2 w_j w_j^T$. The product of orthogonal matrices is numerically stable, and hence we can expect that the rows of $Q$ will indeed be numerically orthogonal (up to machine precision).&lt;/p&gt;&#xA;" OwnerUserId="25388" LastActivityDate="2017-09-12T00:38:44.860" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27836" PostTypeId="2" ParentId="27740" CreationDate="2017-09-12T03:39:26.847" Score="1" Body="&lt;p&gt;Note that this is a nonconvex problem. So expecting to solve the problem to an optimal solution might not be reasonable. But with a good method and good initialization, you might be able to converge to a good solution, although it might be difficult to prove that it is the optimal solution. &#xA;One method that you can use to solve such a problem is to use block coordinate descent or alternative optimization. There are many of papers on this, e.g. &lt;a href=&quot;http://www.math.ucla.edu/~wotaoyin/papers/bcu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.ucla.edu/~wotaoyin/papers/bcu/&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the blocks would be $X$ and $(A,B)$. So in each iteration of the algorithm, you fix one block and update the other block by directly minimizing the objective with respect to the other block. If this is computationally intractable you can always linearize the objective. This is equivalent to just taking the gradient of the objective with respect to the non-fixed block and project it back to the feasible set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of your problem, in the $X$ iterations, you fix $(A,B)$ and then update $X$ by a gradient step and then project it back to non-negative orthant to keep it feasible. Then, alternatively, you fix $X$ and update $(A,B)$ by a gradient step (or optimizing with $X$ fixed). You perform this update until convergence. Note that you need to be careful with the step-size in each of those steps, if you are using the gradient, in order to make sure you always reduce the objective by each update.&lt;/p&gt;&#xA;" OwnerUserId="25390" LastActivityDate="2017-09-12T03:39:26.847" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27837" PostTypeId="2" ParentId="27794" CreationDate="2017-09-12T04:23:05.177" Score="0" Body="&lt;p&gt;Let's take a look at the simplest case of optimizing a simple unconstrained convex quadratic function $\frac{1}{2} x^T H x - b^T x$. In this case, the optimization method is just a fancy word for solving the linear system of equations $H x = b$. It is well-known that solving such linear system of equation is not very stable if the condition number of the matrix $H$ blows up. And there is a lot of research on the stability of solving such linear system of equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea generalizes to the rest of the optimization methods that are fancier. The reason is that no matter how fancy the methods are, they are comprised of linear algebraic steps such as solving linear systems of equation. And therefore, they are as sensitive as those linear algebraic methods if not more. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want more rigorous results regarding the stability of the methods with respect to the errors in computing the iterates, you can look for the inexact version of those methods. For example, I just googled inexact interior point method and found this paper:&lt;a href=&quot;https://link.springer.com/article/10.1023/A:1022663100715&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/article/10.1023/A:1022663100715&lt;/a&gt;. You can do the same with other optimization methods of your choice. Note that the inexact method analysis usually assumes that you can find the iterates with a specific accuracy. But in practice, due to the linear algebraic problems, such as bad conditioning, that we talked about earlier, finding an iterate with a pre-specified accuracy might be challenging. &lt;/p&gt;&#xA;" OwnerUserId="25390" LastActivityDate="2017-09-12T04:23:05.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27838" PostTypeId="1" CreationDate="2017-09-12T12:06:34.570" Score="1" ViewCount="882" Body="&lt;p&gt;A solution to the 1D consolidation problem is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial}{\partial t} p = c_{v} \frac{\partial^{2}}{\partial y^{2}}  p$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $p$ is the pore water pressure, $c_v$ is the consolidation coefficient, $t$ is the elapsed time and $y$ is the height of the column. Further background is provided by &lt;a href=&quot;http://nptel.ac.in/courses/105104132/Module5/lecture27.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is reproduced below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(y,t) = \sum_{n=1}^{n = \infty}\frac{2p_0}{n\pi}(1-\cos(n\pi))\sin(\frac{n \pi y}{2h}\exp(\frac{-n^2\pi^2c_v t}{4h^2}))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;strong&gt;initial conditions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(y,0) = p_0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;strong&gt;boundary conditions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(0,t) = 0$$&#xA;$$p(2h,t) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge that initial condition is a straight line equivalent to the initial loading applied on the soil layer. At $t = 0$ the graph of $y$ versus $p(y,t)$ should look like this for $p_0 = 5000000$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BW6gv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BW6gv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my &lt;code&gt;python&lt;/code&gt; implementation of the above solution. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import cos, exp,pi, linspace, array,sin&#xA;import matplotlib.pyplot as plt&#xA;&#xA;#PRESSURE VERSUS HEIGHT&#xA;&#xA;G = 5000000 # Pa = N/m2&#xA;E = 10000000# Pa = N/m2&#xA;K = E*G/3/(3*G-E)#Bulk modulus of soil skeleton Pa = N/m2&#xA;Kl = 2000000000 #Bulk modulus of liquid Pa = N/m2&#xA;n = 0.33 #porosity&#xA;gammal = 9810 #unit weight of liquid N/m3&#xA;k = 1.157E-17 #intrinsic permeability liquid m2&#xA;h = 0.5 #m&#xA;&#xA;alpha = 1/(K+4*G/3) #m*s2/kg&#xA;beta = 1/Kl #m*s2/kg&#xA;&#xA;cv = k/(gammal*(alpha+n*beta))#m^2/s&#xA;&#xA;n = 200#number of terms to include in series&#xA;timevalues = array([0])&#xA;yvalues = linspace(0,1,100)&#xA;h = 0.5&#xA;p0 = 5000000&#xA;&#xA;&#xA;for t in timevalues:&#xA;    pvalues = []&#xA;    for y in yvalues:&#xA;        SUM = 0&#xA;        SUM2 = 0&#xA;        for j in range(1,n,1):&#xA;            SUM = SUM+(2*p0/j/pi)*(1-cos(j*pi))*sin(j*pi*y/2/h)*exp(-j**2*pi**2**(cv*t/h**2)/4)&#xA;        pvalues.append(SUM)&#xA;    plt.plot(pvalues, yvalues)&#xA;plt.grid()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using arbitrary values for all the parameters and a special relation between liquid bulk modulus, soil bulk modulus and porosity to determine the consolidation coefficient. These things are not relevant however. What is relevant it the initial loading &lt;code&gt;p0 = 5000000&lt;/code&gt; and the &lt;code&gt;timevalues&lt;/code&gt; array assignment of &lt;code&gt;0&lt;/code&gt;. I am creating this array and looping over it because later I wish to use other time values. For now however, using the initial time of $t = 0 s$ I am getting the nonsensical result as shown below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o7kgK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o7kgK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am plotting the values for $t = 0$. Why am I getting this weird sine-like function rather than a constant function? I am also quite sure the analytical result is correct because I have tested it with &lt;code&gt;sympy&lt;/code&gt;. Please help thanks.&lt;/p&gt;&#xA;" OwnerUserId="22736" LastEditorUserId="9667" LastEditDate="2017-09-12T17:08:38.983" LastActivityDate="2017-09-12T17:08:38.983" Title="Solution to 1D consolidation problem python implementation" Tags="|pde|python|fluid-dynamics|solid-mechanics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27839" PostTypeId="2" ParentId="27834" CreationDate="2017-09-12T13:06:42.393" Score="1" Body="&lt;p&gt;You can achieve something like what you are looking for by taking the equivalent of the cross product to keep producing vectors that are perpendicular to the ones you already have. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're only interested in the 2d or 3d cases, here is a function that does this (including explanation of how it works):&#xA;&lt;a href=&quot;https://github.com/geodynamics/aspect/blob/master/source/utilities.cc#L511&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/geodynamics/aspect/blob/master/source/utilities.cc#L511&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation of the function is here:&#xA;&lt;a href=&quot;https://github.com/geodynamics/aspect/blob/master/include/aspect/utilities.h#L213&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/geodynamics/aspect/blob/master/include/aspect/utilities.h#L213&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-09-12T13:06:42.393" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27840" PostTypeId="1" AcceptedAnswerId="27932" CreationDate="2017-09-12T16:35:32.383" Score="0" ViewCount="97" Body="&lt;p&gt;I'm looking for an image filter that does the following operation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my first image, I have two spheres. After applying the filter, I'd like to have them &quot;glued together&quot; which something that kind of resembles surface tension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remarks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The image will be 2D/3D, with the signed distance to the surface of the spheres encoded as grayscale.&lt;/li&gt;&#xA;&lt;li&gt;It would be great if it is something that's already in Numpy / Scipy.&lt;/li&gt;&#xA;&lt;li&gt;I know that people do similiar stuff on meshes, but I'd like to have it as an image filter.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example images (sorry for the different formats):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/epsRq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/epsRq.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HF3Co.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HF3Co.jpg&quot; alt=&quot;Image 2 - after filtering&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2017-09-27T13:34:31.477" Title="Which image filter acts like &quot;surface tension&quot;?" Tags="|image-processing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27841" PostTypeId="2" ParentId="27840" CreationDate="2017-09-12T16:59:53.473" Score="2" Body="&lt;p&gt;I think that you could use a Gaussian Filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following Python code does something similar to what you show in your images.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;import numpy as np&#xA;from scipy.ndimage.filters import gaussian_filter, laplace&#xA;import matplotlib.pyplot as plt&#xA;&#xA;y, x = np.mgrid[-2:2:101j, -2:2:101j]&#xA;z = np.maximum(0, 0.2 - (x + 0.5)**2 - y**2) +\&#xA;    np.maximum(0, 0.2 - (x - 0.5)**2 - y**2)&#xA;z2 = gaussian_filter(z, sigma=7)&#xA;&#xA;plt.figure(figsize=(8, 4))&#xA;plt.subplot(1, 2, 1)&#xA;plt.contourf(x, y, z)&#xA;plt.subplot(1, 2, 2)&#xA;plt.contourf(x, y, z2)&#xA;plt.savefig(&quot;gaussian_filter.png&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results is the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cGy86.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cGy86.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The caveat is the selection of &lt;code&gt;sigma&lt;/code&gt;. That parameter would determine how spread is the Gaussian filter.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2017-09-12T16:59:53.473" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27842" PostTypeId="2" ParentId="27834" CreationDate="2017-09-12T20:14:55.650" Score="4" Body="&lt;p&gt;How about doing an SVD to get the null space of the $1 \times d$ matrix $A = [\bar{v}]$ with $\bar{v} := v^T/\|v\|$? The &lt;a href=&quot;https://mathworks.com/help/matlab/ref/null.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;null&lt;/code&gt;&lt;/a&gt; function in MATLAB does exactly this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d = 4; v = rand(d,1); vbar = v.'/norm(v); Q = null(vbar).'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then your orthonormal vectors are the rows of &#xA;  $ \begin{bmatrix}  &#xA;   \bar{v} \\&#xA;    Q &#xA;  \end{bmatrix}.$&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2017-09-12T20:14:55.650" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27843" PostTypeId="2" ParentId="27834" CreationDate="2017-09-12T22:53:18.427" Score="13" Body="&lt;p&gt;There is a known mathematical question here: you are given a unit&#xA;vector, lying on the $(n-1)$-sphere in $\mathbb{R}^n$, $v\in S^{n-1}$, and you would&#xA;like to associate with each such vector a frame in the tangent bundle&#xA;of $S^{n-1}$ at $v$. This is a map $S^{n-1}\to \mathrm{F}S^{n-1}$ from&#xA;the sphere to its frame bundle, also known as a global section of the&#xA;frame bundle of $S^{n-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the frame bundle (&lt;a href=&quot;https://en.wikipedia.org/wiki/Frame_bundle&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Frame_bundle&lt;/a&gt;) is&#xA;the set of all pairs $(v, (u_1, \ldots, u_{n-1}))$, of a unit vector&#xA;$v$ and $n-1$ orthonormal vectors all orthogonal to $v$, and a global section&#xA;means a map $v \mapsto (u_1, \ldots, u_{n-1})$ that is defined for all&#xA;$v$&#xA;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Section_(fiber_bundle)#Extending_to_global_sections&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Section_(fiber_bundle)#Extending_to_global_sections&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Manifolds that can have such a map are known as parallelizable&#xA;(&lt;a href=&quot;https://en.wikipedia.org/wiki/Parallelizable_manifold&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Parallelizable_manifold&lt;/a&gt;), and as&#xA;Wikipedia says, among the unit spheres only $S^0$, $S^1$, $S^3$ and $S^7$ are&#xA;parallelizable. So if you are looking for a way to get the orthogonal&#xA;complement $v^\perp$ of $v$ that is valid and smooth in $v$ for all&#xA;$v$, you can only do that for $n=1,2,4,8$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $n=3$, in fact, the proof of impossibility is simple: if the two orthogonal vectors such an algorithm would compute are $f(v)$ and $g(v)$, and both depend continuously on $v$, then $f(v)$ defines a tangent vector at $v$ to the unit sphere at the point $v$, and such a tangent vector field would contradict the hairy ball theorem (&lt;a href=&quot;https://en.wikipedia.org/wiki/Hairy_ball_theorem&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Hairy_ball_theorem&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $n=2$, you have the map $(a,b) \mapsto (-b, a)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $n=4$, you have the map&#xA;$$ (a, b, c, d) \mapsto&#xA;\begin{pmatrix}&#xA;-b &amp;amp; -c &amp;amp; -d\\&#xA;a  &amp;amp; d  &amp;amp; -c\\&#xA;-d &amp;amp; a  &amp;amp; b\\&#xA;c  &amp;amp; -b &amp;amp; a&#xA;\end{pmatrix}&#xA;.&#xA;$$&#xA;This corresponds to writing down the quaternion $w =&#xA;a+\mathrm{i}b+\mathrm{j}c+\mathrm{k}d$, where the dot product is computed as&#xA;$v_1^\top v_2 =\Re(\bar w_1 w_2)$), and the three&#xA;orthogonal vectors are $\mathrm{i}w$, $\mathrm{j}w$, $\mathrm{k}w$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $n=8$, you can read the answer off of the Cayley table for octonions (&lt;a href=&quot;https://en.wikipedia.org/wiki/Octonion&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Octonion&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $n=3$ and other $n$, I think the best you can do is to pick a &quot;pole&quot;, such as&#xA;the first basis vector $e_1$, choose the rotation matrix that rotates&#xA;in the $(e_1, v)$-plane only and maps $Re_1 = v$, and use the&#xA;$(n-1)$-frame $(Re_2, \ldots, Re_n)$. The matrix $R$ can be computed&#xA;as a sequence of Givens rotations, which means its directional&#xA;derivatives should be straightforward to compute. This map has a&#xA;singularity at $v=-e_1$. This is equivalent to parallel-transporting a tangent frame at $e_1$ from $e_1$ to $v$ along the shortest path (a great circle on $S^{n-1}$) from $e_1$ to $v$, so there is a geometric interpretation to doing this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More explicitly, write $v = \cos\theta e_1 + \sin\theta e_v$, where $e_v$ is a unit vector, $e_1^\top e_v = 0$. Then the rotation matrix in the $(e_1,e_v)$-basis is &#xA;$$R_0 = \begin{pmatrix}\cos\theta&amp;amp;-\sin\theta\\\sin\theta&amp;amp;\cos\theta\end{pmatrix},$$&#xA; and the change of basis from $(e_1,e_2,\ldots)$ to $(e_1,e_v,\ldots)$ is given by any orthonormal matrix $X$ such that $Xe_1 = e_1$, $Xe_2 = v$. This can be computed by QR decomposition of the $n\times(n+1)$ matrix with columns $(e_1,v,e_2,\ldots,e_n)$, and the answer is the matrix $R_0$ with the change-of-basis formula applied to it,&#xA;$$ R = X \begin{pmatrix} R_0&amp;amp;0\\0&amp;amp;I \end{pmatrix} X^{-1}. $$&#xA;This is independent of which basis $X$ is chosen, because however the QR decomposition completes the basis $R$ is just the identity on it, and it would also only fail on the two special cases, $v=\pm e_1$. &lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-09-13T16:58:20.630" LastActivityDate="2017-09-13T16:58:20.630" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27844" PostTypeId="1" AcceptedAnswerId="27845" CreationDate="2017-09-13T13:32:13.510" Score="1" ViewCount="1566" Body="&lt;p&gt;I am given the data about nodes and elements of a tetrahedral mesh. What algorithm do I use to find the boundary faces of the mesh?&lt;/p&gt;&#xA;" OwnerUserId="25312" LastActivityDate="2017-09-13T13:50:11.383" Title="Algorithm to find boundary faces of mesh" Tags="|mesh-generation|mesh|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27845" PostTypeId="2" ParentId="27844" CreationDate="2017-09-13T13:50:11.383" Score="9" Body="&lt;p&gt;First, create a list of faces in the mesh. From there you should be able to create a map from faces to tets, as each face must belong to either one or two tets. The faces that belong to only one tet are your boundary faces.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2017-09-13T13:50:11.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27846" PostTypeId="1" CreationDate="2017-09-13T19:17:02.607" Score="-1" ViewCount="86" Body="&lt;p&gt;Has anybody ever managed to simulate even the simplest form of life by merely simulating the interactions of the molecules of the organisms body?&#xA;What would be the theoretical minimum number of molecules to build such a numerical simulation?&lt;/p&gt;&#xA;" OwnerUserId="20475" LastActivityDate="2017-09-13T20:58:09.193" Title="Is it possible to simulate life?" Tags="|computational-biology|" AnswerCount="1" CommentCount="3" ClosedDate="2017-09-13T21:39:20.177" ContentLicense="CC BY-SA 3.0" />
  <row Id="27847" PostTypeId="2" ParentId="27846" CreationDate="2017-09-13T20:58:09.193" Score="3" Body="&lt;p&gt;The answer as of 2017, to the best of my knowledge, is &quot;not yet.&quot; Your question is treated in detail &lt;a href=&quot;http://www.ks.uiuc.edu/Research/dbps/dbp9.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here and in the references therein&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-09-13T20:58:09.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27848" PostTypeId="1" CreationDate="2017-09-13T22:26:01.347" Score="1" ViewCount="3166" Body="&lt;p&gt;Using NumPy and attempting to multiply matrices together sometimes doesn't work. For example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;x = np.matrix('1, 2; 3, 8; 2, 9')&#xA;y = np.matrix('5, 4; 8, 2')&#xA;&#xA;print(np.multiply(x, y))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;can return &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &quot;vector-practice.py&quot;, line 6, in &amp;lt;module&amp;gt;&#xA;    print(np.multiply(x, y))&#xA;ValueError: operands could not be broadcast together with shapes (3,2) (2,2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I understand that I can't multiply these shapes, but why not? I can multiply these two matrices on paper, so why not in NumPy? Am I missing something obvious here?&lt;/p&gt;&#xA;" OwnerUserId="25403" LastActivityDate="2018-09-16T06:00:06.790" Title="Why doesn't multiplying two matrices with NumPy work sometimes?" Tags="|numpy|" AnswerCount="1" CommentCount="1" ClosedDate="2017-09-14T09:26:02.763" ContentLicense="CC BY-SA 3.0" />
  <row Id="27849" PostTypeId="2" ParentId="27848" CreationDate="2017-09-14T00:14:04.787" Score="1" Body="&lt;p&gt;Use &lt;code&gt;np.dot()&lt;/code&gt; not &lt;code&gt;np.multiply()&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;np.multiply()&lt;/code&gt; is for element-wise multiplication.&lt;/p&gt;&#xA;" OwnerUserId="14868" LastEditorUserId="20688" LastEditDate="2018-09-16T06:00:06.790" LastActivityDate="2018-09-16T06:00:06.790" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="27850" PostTypeId="1" AcceptedAnswerId="27853" CreationDate="2017-09-14T07:49:11.923" Score="6" ViewCount="343" Body="&lt;p&gt;I work in a field (elasticity reconstruction) that frequently uses standard Finite Element methods to solve the first order PDE, where we are given u and solve for mu and lambda:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;(\mu(u_{i,j} + u_{j,i}))_{,j} + (\lambda~div~\mathbf{u})_{,i} = -\rho \omega^2\mathbf{u}_{i}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;known as the Navier-Lame equation. Here we try to solve the &quot;inverse&quot; case (given u solve for mu and lambda) which is a first-order PDE (while the equation is more commonly solved for u which is a second order PDE).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results overall have been a bit underwelming. Over the past year I was able to speak briefly about this problem to four quite prominent applied mathematicians. Their consensus was that standard FEM, while appropriate for elliptic PDEs of second order, is an inappropriate and unstable mathematical basis for a first order PDE, while better solutions might be Finite Volume or discontinuous Galerkin. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All I remember of the discussions is that first order PDEs have characteristic curves and can handle jumps, while finite elements, like second order PDEs, are smooth in C1 throughout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot find any supporting information for this viewpoint in a text. Can anyone explain why standard FEM is a poor choice for a first order PDE? Or is this not right?&lt;/p&gt;&#xA;" OwnerUserId="3657" LastEditorUserId="123" LastEditDate="2017-09-15T05:46:56.753" LastActivityDate="2017-09-15T05:46:56.753" Title="Why is the continuous Galerkin Finite Element Method a poor choice for the inverse problem for the Navier-Lame equation?" Tags="|finite-element|pde|finite-volume|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27851" PostTypeId="1" CreationDate="2017-09-14T13:48:31.457" Score="1" ViewCount="227" Body="&lt;p&gt;I was able to draw to the outer airtight boundary of hole. Now I would like to implement interior flood/fill algorithm. To get an internal seed, only one method comes to my mind which is testing neighbors of red cells for ray casting. However, I would like to know is there a faster method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X0CYT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X0CYT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8103" LastActivityDate="2017-09-14T13:48:31.457" Title="How to choose seed in flood/fill algorithm" Tags="|algorithms|mesh|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27852" PostTypeId="2" ParentId="25850" CreationDate="2017-09-14T15:29:17.107" Score="1" Body="&lt;p&gt;This is specified in the &lt;code&gt;blockMeshDict&lt;/code&gt;. So it is a property of the patch itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;code&gt;pitzDailyMapped&lt;/code&gt; we have the following in the &lt;code&gt;blockMeshDict&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inlet&#xA;{&#xA;    type mappedPatch;&#xA;    offset          ( 0.0495 0 0 );&#xA;    ...&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the sampling plane is located 0.0495 downstream of the inlet in the x direction.&lt;/p&gt;&#xA;" OwnerUserId="1088" LastActivityDate="2017-09-14T15:29:17.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27853" PostTypeId="2" ParentId="27850" CreationDate="2017-09-14T20:08:36.600" Score="6" Body="&lt;p&gt;The issue with finite elements in the current context is not that you have a first order differential equation, but &lt;em&gt;with the kind of first order equation you have&lt;/em&gt;. In general, finite element methods can be made to perform quite well for first order problems such as advection equations. But that's beside the point here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To illustrate the issue, think of the simpler problem of identifying the coefficient $a(x)$ from the equation&#xA;$$&#xA;  -\nabla \cdot (a(x) \nabla u(x)) = f(x)&#xA;$$&#xA;assuming that you know $f(x)$ and $u(x)$. It is true that you can write this as a first order equation for the unknown $a(x)$:&#xA;$$&#xA;  \beta \cdot \nabla a + \gamma a = f(x)&#xA;$$&#xA;where $\beta(x) = -\nabla u(x)$ looks a lot like an advection direction, and $\gamma(x)=-\Delta u(x)$ like a reaction coefficient. So this looks like an advection-reaction equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the solution of this problem is not well posed unless (i) $|\beta|\ge \beta_\text{min}&amp;gt;0$, i.e., the &quot;wind speed&quot; is always bounded away from zero, (ii) unless each point $x$ in the domain is connected by a &quot;streamline&quot; of the vector field $\beta$ to the boundary, and (iii) you have boundary values for $a(x)$ at all inflow boundary parts, i.e., where $\beta\cdot n&amp;lt;0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, in general cases, none of these three conditions are satisfied. You can easily see this if you consider a circular domain and you have $f=1$ in the outer half of the circle and $f=-1$ in the inner half (and let's assume $a=1$ everywhere). Then your solution $u(x)$ will be rotationally symmetric, and from the boundary first rise up and the decrease again -- i.e., you'll have a circular ring. In this case, each point outside the rim of the hill is connected to the boundary by a streamline, but not on the inside of the hill. Furthermore, $\nabla u=0$ along the rim of the circular hill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In situations like this, the first-order PDE is simply not a well posed problem for $a(x)$. &lt;em&gt;No&lt;/em&gt; method will be able to yield a satisfactory solution in such cases, and that includes the finite element method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-09-14T20:08:36.600" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27855" PostTypeId="1" AcceptedAnswerId="27856" CreationDate="2017-09-14T20:55:46.720" Score="0" ViewCount="141" Body="&lt;p&gt;Suppose i have a bunch of 10 data points and i have to conclude whether the increase is $n^2,n^3,\cdots,2^n,3^n, e^n,\cdots$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example i have the image:-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d0956m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d0956m.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the increase is either polynomial or exponential. First question is how to decide that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 1:&lt;/strong&gt; increasing polynomially, i.e., $n^c$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then how to find constant $c$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Case 2:&lt;/strong&gt; increasing exponentially, i.e., $c^n$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then how to find constant $c$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a comprehensive pseudocode that solves this?&lt;/p&gt;&#xA;" OwnerUserId="24783" LastActivityDate="2017-09-14T22:18:00.577" Title="finding the growth rate from numerical data" Tags="|matlab|numerical-analysis|complexity|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27856" PostTypeId="2" ParentId="27855" CreationDate="2017-09-14T22:18:00.577" Score="4" Body="&lt;p&gt;Plot your data in two different ways:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$\log(y)$ vs $x$&lt;/li&gt;&#xA;&lt;li&gt;$\log(y)$ vs $\log(x)$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If your data appears to be linear in the first case, then your data takes the form $y(x) = A\cdot c^x$, and the line will have slope $\log(c)$. This can be&#xA;seen from the relation $\log(y) = \log(A\cdot c^x) = \log(c) x + \log(A)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data appears to be linear in the second case, then your data may come from a function of the form $y(x) = A\cdot x^c$, and the line will have slope $c$. This can be seen from $\log(y) = \log(A\cdot x^c) = c\log(x) + \log(A)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will mention that if your polynomial-generated data has a non-zero intercept, this can sometimes make it appear exponential if your range of $x$ values is not large. It is best if you have some idea going in to determine whether your data &lt;em&gt;should&lt;/em&gt; have exponential or polynomial behavior. This can help you when you are cleaning your data before applying this technique. This effect is obviously exacerbated if your data is noisy, because it is much more difficult to remove the intercept without driving any y-values below 0 (your regression will complain about complex numbers...).&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-09-14T22:18:00.577" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27857" PostTypeId="1" AcceptedAnswerId="27871" CreationDate="2017-09-14T23:03:24.903" Score="0" ViewCount="64" Body="&lt;p&gt;I'm wondering if this is a valid methodology to find the best regression equation for a given data set.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;User provides a rang of estimated value  for some set of variables.  Th algorithm uses the given range to randomly generate a large number of regression equations like the one bellow.&lt;br&gt;&#xA;&lt;code&gt;Price = actualValue1* estimatedValue1 + actualValue2* estimatedValue2;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We solve them all for price and pick the best one as regression equation.  Will this methodology give us a valid regression equation or will it just find some random set of estimated values That happens to give a price that is close to the actual price?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS.  I've taken stats but I'm not a statistician So I'm in a little over my head here. If I need to clarify something please let me know. &lt;/p&gt;&#xA;" OwnerUserId="25415" LastActivityDate="2017-09-17T22:51:44.300" Title="will this methodology end up giving me a nonsense regression equation." Tags="|statistics|regression|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27858" PostTypeId="2" ParentId="27857" CreationDate="2017-09-14T23:15:07.393" Score="0" Body="&lt;p&gt;No, this is not a valid way to proceed. There is no guarantee of finding the best regression coefficients. Use multiple linear regression: &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Linear_regression&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14481" LastActivityDate="2017-09-14T23:15:07.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27859" PostTypeId="2" ParentId="6758" CreationDate="2017-09-15T07:32:58.273" Score="1" Body="&lt;p&gt;Here is a function that does it (without printing any message):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function stoprun()&#xA;  ms.message='';&#xA;  ms.stack = dbstack('-completenames');&#xA;  ms.stack(1:end) = [];&#xA;  ds = dbstatus();&#xA;  stoponerror = any(strcmp('error', {ds.cond}));&#xA;  setappdata(0, 'dberrorkeep', stoponerror);&#xA;  dbclear error&#xA;  error(ms);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25416" LastActivityDate="2017-09-15T07:32:58.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27860" PostTypeId="1" AcceptedAnswerId="27883" CreationDate="2017-09-15T07:41:25.610" Score="1" ViewCount="1787" Body="&lt;p&gt;I was wondering whether there is a smart and efficient way in Matlab to compute the kronecker product of several 1D arrays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean is something like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = [a1, a2];&#xA;B = [b1, b2];&#xA;C = [c1, c2];&#xA;K = f(A,B,C) = [a1*b1*c1, a1*b1*c2, a1*b2*c1, ... ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One possible way it is to use the kron(X,Y) function, but it has to be put inside a loop to obtain the aforementioned result since the kron function accepts only 2 arguments per time.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K = 1;&#xA;tot_arrays = [A, B, C];&#xA;for i=1:num_arrays&#xA;    K = kron(K,tot_arrays(:,i));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Isn't there a smarter and more efficient way to obtain the same result for the kron multiplication of more than 2 arrays per time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14817" LastActivityDate="2017-09-19T18:25:17.957" Title="Kronecker product among multiple arrays" Tags="|matlab|algorithms|matrix|performance|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27861" PostTypeId="1" AcceptedAnswerId="27862" CreationDate="2017-09-15T09:37:52.123" Score="0" ViewCount="77" Body="&lt;p&gt;I'm facing a non-linear programming problem which currently I'm solving with fmincon function of MATLAB. However, I'm not very happy with computation times and solution convergence since it only converges when setting the algorithm option to 'active-set' and computation times depends heavily on the initial condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if there exist free good general non-linear programming solvers with MATLAB interface since I would like to check if they can solve it more efficiently than fmincon. &lt;/p&gt;&#xA;" OwnerUserId="25418" LastActivityDate="2017-09-15T11:52:53.117" Title="Non linear programming solvers with API for MATLAB?" Tags="|optimization|matlab|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27862" PostTypeId="2" ParentId="27861" CreationDate="2017-09-15T11:52:53.117" Score="2" Body="&lt;p&gt;OPTI Toolbox (&lt;a href=&quot;https://www.inverseproblem.co.nz/OPTI/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.inverseproblem.co.nz/OPTI/&lt;/a&gt;) gives you access to a number of solvers, including IPOPT which often performs well. MATLAB binaries are only shipped for WIN64 though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are you supplying derivatives to fmincon?&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2017-09-15T11:52:53.117" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27865" PostTypeId="1" AcceptedAnswerId="27866" CreationDate="2017-09-16T23:37:33.187" Score="8" ViewCount="1166" Body="&lt;p&gt;The naive summation of a Chebyshev series&#xA;\begin{align*}&#xA;f(x) = \frac{c_0}{2} + \sum_{k=1}^{n-1} c_{k}T_{k}(x)&#xA;\end{align*}&#xA;which employs the three-term recurrence for evaluation of the Chebyshev polynomials is known to be numerically unstable near $x=\pm 1$. For this reason, Clenshaw presented an algorithm which avoids this instability; see &lt;a href=&quot;http://www.ams.org/journals/mcom/1955-09-051/S0025-5718-1955-0071856-0/S0025-5718-1955-0071856-0.pdf&quot; rel=&quot;noreferrer&quot;&gt;Clenshaw's original work&lt;/a&gt; and an exposition in Algorithm 3.1 of &lt;a href=&quot;https://www.siam.org/books/ot99/OT99SampleChapter.pdf&quot; rel=&quot;noreferrer&quot;&gt;Numerical Methods for Special Functions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there literature which presents a numerically stable method for evaluating the derivative of a Chebyshev series, perhaps using an analogy to the Clenshaw recurrence presented above? Or is the naive method stable?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2017-09-17T20:06:14.020" LastActivityDate="2017-09-17T20:06:14.020" Title="Clenshaw-type recurrence for derivative of Chebyshev series" Tags="|numerical-analysis|stability|chebyshev|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27866" PostTypeId="2" ParentId="27865" CreationDate="2017-09-17T01:15:32.653" Score="13" Body="&lt;p&gt;You can just take Clenshaw's recurrence&#xA;$$ u_k(x) = 2xu_{k+1}(x)-u_{k+2}(x)+\color{red}{a_k},\\&#xA;f(x) = x u_1(x)-u_2(x)+\color{red}{a_0} $$&#xA;and differentiate it directly:&#xA;$$ u_k'(x) = 2xu_{k+1}'(x)-u_{k+2}'(x) + \color{blue}{2u_{k+1}(x)},\\&#xA;f'(x) = x u_1'(x)-u_2'(x) + \color{blue}{u_1(x)}. $$&#xA;Note that now the derivatives of partial sums, $u_k'(x)$ satisfy &lt;em&gt;exactly&lt;/em&gt; the same recurrence, except that the coefficients $a_k$ are replaced by $2u_{k+1}(x)$. This means that the series for the derivative can be written as the Chebyshev series&#xA;$$ f'(x) = \tfrac12 u_1(x) + \sum_{k\geq1} 2u_{k+1}(x)T_k(x). $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since Clenshaw's recurrence evaluates Chebyshev series accurately, it will evaluate the Chebyshev series with coefficients $2u_{k+1}(x)$ accurately too, and it doesn't need to be analyzed separately. A very similar thing happens when differentiating polynomials using Horner's rule.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-09-17T01:15:32.653" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27867" PostTypeId="1" CreationDate="2017-09-17T08:27:36.000" Score="2" ViewCount="441" Body="&lt;p&gt;I have a mesh with irregular hexaedra and I need a fast way to interpolate values at points inside these cells. I know that trilinear interpolation does not work well for large skews. Barycentric coordinates work for tetrahedrons. Is there any equivalent method for hexaedra, which is really efficient, possibly parallelization-friendly?&lt;/p&gt;&#xA;" OwnerUserId="23565" LastEditorUserId="9667" LastEditDate="2017-09-19T00:27:16.207" LastActivityDate="2017-09-19T21:30:05.973" Title="Barycentric interpolation equivalent for irregular hexahedra" Tags="|finite-element|interpolation|mesh|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27868" PostTypeId="1" AcceptedAnswerId="27869" CreationDate="2017-09-17T16:04:18.997" Score="3" ViewCount="672" Body="&lt;p&gt;Some reference in adaptive techniques say that when  the solution $u$ is smooth enough we can use p-refinement instead of h-refinement. And when we have for example singularity,  we should use h-refinement. I can not understand it. why?&#xA;When we use a higher order polynomial for approximation we can get smoother solution, so it seems there is no problem for using p-refinement.&lt;/p&gt;&#xA;" OwnerUserId="10403" LastActivityDate="2017-09-17T22:45:52.233" Title="p-refinement in adaptive methods" Tags="|finite-element|adaptive-mesh-refinement|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27869" PostTypeId="2" ParentId="27868" CreationDate="2017-09-17T18:43:14.887" Score="8" Body="&lt;p&gt;The decision whether to do p- or h- adaptivity is to achieve faster, potentially exponential convergence rate. In other words to get a solution with given error with minimal computational effort. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Limiting considerations to elliptic problems. The approximate solution can converge to exact solution if you do h- or p- adaptivity. If the exact solution is smooth, convergence is faster if you do p- adaptivity. To see this look at a priory error estimator,&#xA;\begin{equation}&#xA;\| u - u^h \| \leq C h^p \left\| \frac{\partial^{p+1} u}{\partial x^{p+1}} \right\|&#xA;\end{equation}&#xA;where $p$ is polynomial order of base functions. If the solution is smooth, higher order derivatives fast approach zero, so that a priory error estimator. So if you increasing order of base functions, you very quickly converge to an exact solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, that is not working if the solution is not regular, for example if you have a singularity. Higher order derivatives are not bounded, and p- adaptivity give a solution with larger computational cost comparing to h- adaptivity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example see L-shape body with transport problem,&#xA;&lt;a href=&quot;https://i.stack.imgur.com/KS9yK.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KS9yK.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &#xA;&lt;a href=&quot;https://i.stack.imgur.com/pWgIl.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pWgIl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;For details see here &lt;a href=&quot;http://mofem.eng.gla.ac.uk/mofem/html/mix_transport.html&quot; rel=&quot;noreferrer&quot;&gt;http://mofem.eng.gla.ac.uk/mofem/html/mix_transport.html&lt;/a&gt;. Mix formulation is applied there, but the same goes for classical finite elements. You can see that both h- and p- adaptivity show convergence, however the fastest convergence you get if you locally at singularity do h- refinement.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-09-17T22:45:52.233" LastActivityDate="2017-09-17T22:45:52.233" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27871" PostTypeId="2" ParentId="27857" CreationDate="2017-09-17T22:51:44.300" Score="1" Body="&lt;p&gt;That depends on (i) how many random samples you draw, and (ii) how good a regression you want to have. The point of finding the best fit regression is that you want &lt;em&gt;the best&lt;/em&gt; model out of &lt;em&gt;all&lt;/em&gt; possible models; you only have finitely many, and so in general you will not get the &lt;em&gt;best possible&lt;/em&gt; model, just the best out of the random samples you've generated. Obviously, if you draw more samples (assuming they are truly random), then the best among them will be at least as good as before, and maybe better, but in general not &quot;best&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-09-17T22:51:44.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27872" PostTypeId="2" ParentId="26445" CreationDate="2017-09-18T00:46:42.927" Score="0" Body="&lt;p&gt;For finite element method in fluids &quot; Finite Element Methods for Flow Problems&quot;  &lt;a href=&quot;https://g.co/kgs/H6ZVpQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://g.co/kgs/H6ZVpQ&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="25408" LastActivityDate="2017-09-18T00:46:42.927" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27873" PostTypeId="1" AcceptedAnswerId="27875" CreationDate="2017-09-18T01:32:33.713" Score="0" ViewCount="1899" Body="&lt;p&gt;I am assigned to compute eigenvalues and eigenvectors in MATLAB of a 2x2 matrix:&#xA;$$&#xA;A = \left( \begin{matrix}&#xA;3 &amp;amp;0\\&#xA;4 &amp;amp;5\\&#xA;\end{matrix} \right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the textbook's solution states that eigenvalue 3 corresponds to an eigenvector $(1 \; -2)$, and eig 5 corresponds to $(0 \; 1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I do. Why am I not getting the correct eigenvectors?&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;% Define the matrix&#xA;A = [3 0;4 5];&#xA;&#xA;% Find Eigenvalues&#xA;E1 = eig(A);&#xA;&#xA;% Display Eigenvalues&#xA;disp('Eigenvalues of the matrix A:')&#xA;E1&#xA;&#xA;% Determine Eigenvectors&#xA;[V,D] = eig(A);&#xA;&#xA;% V1 corresponds to eigenvalue 1 and V2 corresponds to eigenvalue 2&#xA;V1 = V(:,1)&#xA;V2 = V(:,2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25437" LastEditorUserId="9667" LastEditDate="2017-09-18T14:14:18.273" LastActivityDate="2017-09-18T14:14:18.273" Title="Computing Eigenvectors in MATLAB" Tags="|linear-algebra|matlab|eigenvalues|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27874" PostTypeId="1" CreationDate="2017-09-18T01:50:30.657" Score="1" ViewCount="56" Body="&lt;p&gt;When calculating a type-II DCT with FFTW, you create an &lt;code&gt;fftw_plan&lt;/code&gt;, via&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;   fftw_plan plan = fftw_plan_r2r_1d(size, in_ptr, out_ptr, FFTW_REDFT10, FFTW_ESTIMATE);&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, the plan precomputes the twiddle factors so that the plan can be efficiently reused for multiple calls to &lt;code&gt;fftw_execute_r2r(plan, in_ptr, out_ptr)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I'm trying to calculate&#xA;\begin{align*}&#xA;c_{k} = \frac{1}{N} \sum_{j=0}^{N-1} f\left(\cos\left(\frac{(j+1/2)\pi}{N}\right)\right) \cos\left(\frac{(j+1/2)k\pi}{N}\right)&#xA;\end{align*}&#xA;I figure that the &lt;code&gt;fftw_plan&lt;/code&gt; already has computed the argument to $f$, so I don't want to recompute it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you extract the twiddle factors from the &lt;code&gt;fftw_plan&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2017-09-18T01:50:30.657" Title="Extracting Real Part of Twiddle Factors from `fftw_plan`" Tags="|fftw|chebyshev|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27875" PostTypeId="2" ParentId="27873" CreationDate="2017-09-18T02:09:56.743" Score="2" Body="&lt;p&gt;Eigenvectors are only unique to within a scale factor (can be + or - scale factor).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $x$ satisfies $Ax=\lambda x$, and hence is an eigenvector of $A$ corresponding to eigenvalue $\lambda$, then any multiple of $x$ also satisfies the equation, and hence is also an eigenvector of $A$ corresponding to eigenvalue $\lambda$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB normalizes eigenvectors to have 2-norm equal to 1, but even that leaves a choice of sign.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2017-09-18T02:09:56.743" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27876" PostTypeId="2" ParentId="27873" CreationDate="2017-09-18T02:10:24.430" Score="0" Body="&lt;p&gt;I doubt you're doing anything wrong. Matlab (and LAPACK, the guts underneath Matlab) will normalize eigenvectors to unit length, so you won't get (1,-2) for an eigenvector, you'll get (1,-2)/sqrt(5) instead.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2017-09-18T02:10:24.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27881" PostTypeId="2" ParentId="27867" CreationDate="2017-09-19T12:26:32.663" Score="5" Body="&lt;p&gt;A hexahedron with straight edges is the image of the unit cube under a trilinear mapping. So, if you have values on the eight vertices of a hexahedron, and you are asking to interpolate between them to a point $x$ somewhere inside the hexahedron, then the correct approach is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Invert the trilinear mapping to find the corresponding point $\hat x = \Phi^{-1}(x)$ on the reference cell.&lt;/li&gt;&#xA;&lt;li&gt;Use the trilinear shape functions on the reference cell to evaluate at $\hat x$ to obtain the interpolated value of your function based on the values at the vertices.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The second step is easy. The first requires inverting the mapping $\Phi$, which in general requires a nonlinear Newton iteration because it is a nonlinear, polynomial you are inverting.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2017-09-19T21:30:05.973" LastActivityDate="2017-09-19T21:30:05.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27882" PostTypeId="1" AcceptedAnswerId="27885" CreationDate="2017-09-19T13:14:18.970" Score="1" ViewCount="1416" Body="&lt;p&gt;I see images of steel connections, concrete dams, and other complicated 3D objects in papers which finite element analysis has been performed on them. My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How these objects are created in MATLAB? Should we give all the points ourselves?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can we create mesh for them to perform finite element analysis? This one might be a general question with many answers but if it is possible please give some main points or references about it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are these objects drawn in a software then their coordinates exported to MATLAB? If so, then what are these software?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22763" LastEditorUserId="9667" LastEditDate="2017-09-19T20:14:22.260" LastActivityDate="2019-12-11T08:50:30.230" Title="Create mesh for complicated 3D object for finite element analysis" Tags="|finite-element|matlab|mesh-generation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27883" PostTypeId="2" ParentId="27860" CreationDate="2017-09-19T18:25:17.957" Score="3" Body="&lt;p&gt;Faster to write: I don't think there is. Faster to run with larger array lengths: the first thing I would try is the following.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; a = [1 2];&#xA;&amp;gt;&amp;gt; b = [3 4];&#xA;&amp;gt;&amp;gt; c = [5 6];&#xA;&amp;gt;&amp;gt; a = reshape(a, [length(a) 1 1]);&#xA;&amp;gt;&amp;gt; b = reshape(b, [1 length(b) 1]);&#xA;&amp;gt;&amp;gt; c = reshape(c, [1 1 length(c)]);&#xA;&amp;gt;&amp;gt; P = a.*b.*c;&#xA;&amp;gt;&amp;gt; P = reshape(P, [1 length(P)])&#xA;P =&#xA;    15    30    20    40    18    36    24    48&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-09-19T18:25:17.957" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27884" PostTypeId="1" CreationDate="2017-09-19T18:58:34.790" Score="6" ViewCount="160" Body="&lt;p&gt;I am attempting to use an iterative solver to solve &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;Jp = -r&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$m\times m$&lt;/span&gt; matrix (&lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is in the order of &lt;span class=&quot;math-container&quot;&gt;$10^5$&lt;/span&gt; and never explicitly stored). &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; is a dense matrix representing a Jacobian for a certain function, and, in general, is not symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this is an ill-posed problem and possibly infinite versions of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; exist. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot compute the least-squares version&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J^TJp=J^Tr&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because I only can access the product of &lt;span class=&quot;math-container&quot;&gt;$Jp$&lt;/span&gt; and not &lt;span class=&quot;math-container&quot;&gt;$J^Tp$&lt;/span&gt; (a limitation of an automatic differentiation approach I am taking). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a way I can compute &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$Jp = -r$&lt;/span&gt; according to some criterion (e.g. least-squares solution) without transposing&#xA;&lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;li&gt;If I use an iterative solver (e.g. GMRES, BiCGStab, etc...) with an&#xA;initial guess of &lt;span class=&quot;math-container&quot;&gt;$p=0$&lt;/span&gt;, what solution will the solver converge to?&#xA;The minimum-norm?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="23647" LastEditorUserId="20688" LastEditDate="2020-03-29T07:55:19.430" LastActivityDate="2020-03-29T07:55:19.430" Title="Solve ill-posed linear system without transposing matrices?" Tags="|least-squares|linear-system|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="27885" PostTypeId="2" ParentId="27882" CreationDate="2017-09-20T08:38:10.587" Score="2" Body="&lt;p&gt;Depending on how complicated your meshing needs are, you might find the following Matlab program useful: &lt;a href=&quot;http://persson.berkeley.edu/distmesh/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Distmesh&lt;/a&gt;. This program makes it possible to generate simple shapes and geometries in Matlab itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to model actual parts, and you don't have a FEM code to use yourself, then Matlab offers a PDE toolbox that can be used to solve various FEM problems. From looking at their documentation, the easiest way to import models is by using .stl files &lt;a href=&quot;https://nl.mathworks.com/help/pde/ug/plot-3-d-solutions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;. There are a number of CAD programs that can be used to build models and export them as stl files, as mentioned by @knl in the comments&lt;/p&gt;&#xA;" OwnerUserId="19791" LastActivityDate="2017-09-20T08:38:10.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27886" PostTypeId="2" ParentId="25141" CreationDate="2017-09-20T11:27:44.487" Score="0" Body="&lt;p&gt;Just see this question.&#xA;The way I did it in c++ is that instead of implementing the Dirichlet Condition afterwards, I do not put the element into the row/column where Dirichlet Condition apply.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is just a condition loop inside the main loop when you do the triplets of each element, based on either the index belongs to those row/columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And you can use a hash-map (unsorted_set/unsorted_map) to store the fixed nodes.&#xA;Total cost is still O(1).&lt;/p&gt;&#xA;" OwnerUserId="25455" LastActivityDate="2017-09-20T11:27:44.487" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27887" PostTypeId="1" CreationDate="2017-09-20T14:57:36.047" Score="1" ViewCount="104" Body="&lt;p&gt;I'm trying to approximate an unknown, non-linear function $f: A \to \mathbb{R}$ (in my case: $A = B \times B$ where $B$ is an infinite but compact subset of $\mathbb{R}$). That is, evaluating $f$ is costly and I'm looking for an easy-to-evaluate function $g$ such that $f$ is close to $g$ on $A$. Given an error bound, how can I compute $g$ with as few evaluations of $f$ as possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm flexible regarding the precise definition of the error. For example, both the expected average error and the expected maximum error would work. Similarly, I don't really care about the type of representation used for $g$ (splines, polynomials, etc.) as long it can be efficiently evaluated. The number of evaluations of $f$ must not necessarily be provably minimal, methods that work well in practice are sufficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, it sounds like &lt;a href=&quot;https://en.wikipedia.org/wiki/Design_of_experiments&quot; rel=&quot;nofollow noreferrer&quot;&gt;design of experiments&lt;/a&gt; might offer suitable tools, however, I'm not sure how to match my setting to that statistical context. &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian optimization&lt;/a&gt; also seems to have a very similar setting, but I'm not sure how one would adapt it to optimize a general approximation of $f$ instead of its optimization.&lt;/p&gt;&#xA;" OwnerUserId="4230" LastActivityDate="2017-09-21T11:50:46.017" Title="Approximate function with minimum number of evaluations" Tags="|interpolation|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27888" PostTypeId="1" AcceptedAnswerId="27889" CreationDate="2017-09-20T21:26:30.060" Score="1" ViewCount="297" Body="&lt;p&gt;Is there something like a rule of thumb for an adequate time-step size when solving Maxwell's equation for the interaction of light with matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess a single wave oscillation has to be resolved within at least ~10 steps, which would give a time-step below 1 fs for visible light?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really hope it's not that bad. What's your experience?&lt;/p&gt;&#xA;" OwnerUserId="21797" LastActivityDate="2017-09-22T21:01:00.703" Title="Rule of thumb for time-step for solving Maxwell's Equation using 3D-FDTD?" Tags="|numerical-modelling|electromagnetism|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27889" PostTypeId="2" ParentId="27888" CreationDate="2017-09-21T00:22:03.677" Score="1" Body="&lt;p&gt;For visible light, a timestep on the order fs is correct. But you have to balance that against the fact that you would typically only need to use FDTD (or any fullwave technique) when your scatterers/structures are on the scale of the wavelength anyway. And since they're so small, the total interaction time is probably not especially long. It's probably in nanoseconds unless they are extremely high Q resonators (and if they are high-Q, use a frequency domain technique directly, like finite elements or method of moments, instead of just waiting ages for a transient solver to reach oscillatory steady state). I don't mean to imply that optical FDTD models are inexpensive, just pointing out that you rarely need to model long time durations with tiny timesteps (and you may have ways to escape, anyway).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When structures are vastly larger than a wavelength (ie too big for full-wave/FDTD) then asymptotic/raytracing-like techniques grow increasingly accurate. They are not bound by the nyquist rate in space nor the courant criteria in time, so their runtime is not so strongly sensitive to frequency (just their accuracy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do admit and agree that there's still a large &quot;unconquered middle&quot; of important structures that are electrically/optically large, yet still packed with enough fine detail to demand fullwave accuracy (antenna arrays, photonic crystals, computer chips, many more). Even here, there might be tricks (periodicity, for instance, in the case of an array) that allow for a tailor-made full wave solution.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2017-09-21T00:22:03.677" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27891" PostTypeId="2" ParentId="27887" CreationDate="2017-09-21T10:10:29.790" Score="1" Body="&lt;p&gt;You could probably use a surrogate model $g$ of your function $f$ using kriging see for example &lt;a href=&quot;https://books.google.fr/books?hl=fr&amp;amp;lr=&amp;amp;id=ulMHmeMnRCcC&amp;amp;oi=fnd&amp;amp;pg=PR5&amp;amp;dq=engineering%20design%20via%20surrogate%20modelling:a%20practical%20guide&amp;amp;ots=geDP_nEa5E&amp;amp;sig=OvIxzxozVPrA8bMytQWXnrHUwEc#v=onepage&amp;amp;q=engineering%20design%20via%20surrogate%20modelling%3Aa%20practical%20guide&amp;amp;f=false%22engineering%20design%20via%20surrogate%20modelling%22&quot; rel=&quot;nofollow noreferrer&quot;&gt;Engineering design via surrogate model&lt;/a&gt;, and GaussianProcess from sklearn for python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really briefly; using it you can build a prediction $g(x)$ and its variance $s^2(x)$ for all $x$ from a limited number of samples points $X$ such as : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}\label{eq:pred}&#xA;    g(x) ={p}_x^T\tilde{\beta} + {r_x}^T{R}^{-1}{f_x}-{P}{\tilde{\beta}})&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}\label{eq:var}&#xA;    s^2({x}) = \tilde{\sigma}^2[1+{r_x}^T{R}^{-1}{r_x}]&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $p_x$ is the response of $x$ to a chosen polynomial regression and $P$ the response of the learning points to this regression. $r_x$ is the correlation vector of $x$ and $R$ the correlation matrix of the learning points $X$. Finally, $\tilde{\beta}$ and $\tilde{\sigma^2}$ are known coefficients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, to save the number of calling to your expensive function $f$ : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- Build a first surrogate $g$ with a limited and reasonable number of learning point (DOE)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2-  Enrich your surrogate with a new learning point chosen by a criterion based on $s$. Add a learning point where the surrogate is the worse, where the error is the biggest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3- Repeat 2 until obtain a satisfactory $g$, which is easy to evaluate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the number of evaluation is the number of samples points $X$ plus the number of enrichment.  &lt;/p&gt;&#xA;" OwnerUserId="24755" LastEditorUserId="24755" LastEditDate="2017-09-21T11:50:46.017" LastActivityDate="2017-09-21T11:50:46.017" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27896" PostTypeId="1" CreationDate="2017-09-22T09:55:40.227" Score="3" ViewCount="120" Body="&lt;p&gt;I have the following optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given is a set of $n$ points $x_i, \ldots, x_n$ with $x_i \in \mathbb{C}, \|x_i\| &amp;lt; \infty$. Note that $n$ is usually not large, i.e. $n &amp;lt; 1000$. We want to find a point $c \in \mathbb{C}$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max_i \| x_i + c \|$ is minimized&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, this problem is solved by means of an iterative descent algorithm: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialize $c^{(0)} = 0$&lt;/li&gt;&#xA;&lt;li&gt;For $j = 0, \ldots, j_\mathrm&#xA;{iter}$&#xA;&#xA;&lt;ol start=&quot;21&quot;&gt;&#xA;&lt;li&gt;Find $\max_i \| x_i + c^{(j)} \|$  &lt;/li&gt;&#xA;&lt;li&gt;Set $c^{(j+1)} = c^{(j)} - \alpha \frac{x_i + c^{(j)}}{\| x_i + c^{(j)} \|}$&#xA;,where $\alpha$ is some stepwidth. &lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This algorithm works fine, but: I am looking for another (faster) solution. I wonder if it is possible to use any non-iterative solver. So far, I couldn't come up with any. So do you have any ideas how to tackle this?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="25475" LastEditorUserId="25475" LastEditDate="2017-09-22T10:16:25.570" LastActivityDate="2017-09-22T10:16:25.570" Title="Optimization Problem: Minimize the absolute value of a set of points" Tags="|optimization|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27898" PostTypeId="1" AcceptedAnswerId="27901" CreationDate="2017-09-22T19:50:51.530" Score="1" ViewCount="61" Body="&lt;p&gt;I am trying to solve a system of non-linear index-1 DAEs in which the derivatives of the state variables, &lt;span class=&quot;math-container&quot;&gt;$x(t)$&lt;/span&gt; are corrupted by additive noise, &lt;span class=&quot;math-container&quot;&gt;$w(t)$&lt;/span&gt; (whose covariance matrix is known).&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\dot x(t) = f(x(t),t,u(t)) + w(t)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there a known (open source) solver/library/package/toolbox in MATLAB/Python/Julia (with or without adaptive time-stepping capability) that can solve the system?&lt;/p&gt;&#xA;" OwnerUserId="16900" LastEditorUserId="20688" LastEditDate="2022-01-03T06:49:48.380" LastActivityDate="2022-01-03T06:49:48.380" Title="Open source solver for continuous-time stochastic non-linear DAEs (SDAEs)" Tags="|time-integration|stochastic|adaptive-timestepping|dae|stochastic-ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27899" PostTypeId="2" ParentId="27888" CreationDate="2017-09-22T21:01:00.703" Score="1" Body="&lt;p&gt;There is a very simple rule of thumb for the time step in FDTD: set it as large as possible while still satisfying the CFL condition. Basically your space discretization dicatates you your time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;rchilton1980 wrote:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It's probably in nanoseconds unless they are extremely high Q resonators (and if they are high-Q, use a frequency domain technique directly, like finite elements or method of moments, instead of just waiting ages for a transient solver to reach oscillatory steady state).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This remark seems to assume that you excite the domain by a delta pulse in the time domain. However, it often makes more sense to use a pulse of finite width (Hann window aka raised cosine modulated by the desired frequency, size should be a multiple of modulation), because it will significantly reduce the transient time in case of high-Q structures. My advice: use (=increase) the transient time to cover for wave propagation delay, and use (=increase) the pulse width to deal with high-Q effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This remark remains true even if you use ADI instead of FDTD. If you use ADI, another rule of thumb for the time step will be required. I would use around 20 per wavecycle for a start. Then you can find out whether it is too slow or too inaccurate, and change accordingly.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2017-09-22T21:01:00.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27901" PostTypeId="2" ParentId="27898" CreationDate="2017-09-23T05:10:39.217" Score="3" Body="&lt;p&gt;DifferentialEquations.jl in Julia can do it if you can write it in mass-matrix form. You won't find it mentioned &lt;a href=&quot;http://diffeq.sciml.ai/latest/tutorials/sde_example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the tutorial&lt;/a&gt;, but you can &lt;a href=&quot;http://diffeq.sciml.ai/latest/types/sde_types.html#Fields-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;provide a mass matrix&lt;/a&gt; as part of the &lt;code&gt;SDEProblem&lt;/code&gt;. &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/sde_solve.html#Stiff-Methods-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some of the stiff solvers can handle the problem&lt;/a&gt; (I see it's not well-documented yet which ones, but &lt;a href=&quot;https://github.com/JuliaDiffEq/StochasticDiffEq.jl/blob/master/src/alg_utils.jl#L58&quot; rel=&quot;nofollow noreferrer&quot;&gt;it's the symplectic and implicit Euler forms&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I will caution that this is pretty new so I would test it on a non-DAEs, i.e. using the mass-matrix functionality and then adding &lt;span class=&quot;math-container&quot;&gt;$M^{-1}$&lt;/span&gt; to your SDE equations and checking to make sure that they calculate the same thing. If you notice any discrepancies, please file an issue. The architecture for the implicit handling here is based on the ODE solvers which are very well-tested, but since I see we don't explicitly has a mass matrix SDE test in the continuous integration suite yet I would just do this test before trusting it.&lt;/p&gt;&#xA;&lt;p&gt;There are no adaptive timestepping algorithms for SDAEs quite yet (only for SDEs)... well... that are released. There is an adaptive timestepping stiff solver for additive noise SDAEs in mass matrix form which is in production but won't be released until after publication. Hopefully in a few months I can edit this post to be a more solid yes, but for now what we have should at least solve the problem.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:09:53.440" LastActivityDate="2020-07-19T18:09:53.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27902" PostTypeId="1" CreationDate="2017-09-23T11:06:45.450" Score="1" ViewCount="161" Body="&lt;p&gt;I have written an 2d N-body simulation in Python which allows collisions between the bodies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A body is modeled as a circle whose area is proportional to its mass. Each time the sim advances by one time-step, I identify which bodies have overlapping radii and I replace them by one particle (respecting conservation of momentum and treating the collision as inelastic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This worked well for N&amp;lt;100 but for higher N I encountered (not systematically, but depending on the initial conditions of the sim) a weird error: the number of particles N actually increased (and quickly exceeded the maximum integer value Python can handle).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe what is occuring is, after the new positions of the bodies have been calculated, it happens that (at least) 4 bodies mutually overlap. Each of the 6 pairs creates a new body and but we also delete the 4 bodies,so N increased by os we have 6-4=2. But the 6 new bodies are spawned in roughly the same positions as the original 4 bodies so now we have even more overlap (N can now increase by potentially (6 choose 2)-6=9 bodies). Hence why N can grow very fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I avoid this error?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only thing I can think of is to reduce the radii of the bodies to reduce the change of 4 particles mutually overlapping. But this does not definitively fix the problem.&lt;/p&gt;&#xA;" OwnerUserId="24246" LastActivityDate="2017-09-23T11:06:45.450" Title="Dealing with simultaneous collisions in N-body sim" Tags="|computational-physics|simulation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27903" PostTypeId="1" AcceptedAnswerId="27904" CreationDate="2017-09-23T11:14:17.693" Score="0" ViewCount="94" Body="&lt;p&gt;In FEM, to discrete the weak form, elements are needed to build the FE approximation basis space. FE method involves integration of the weak form. Due to &#xA;discretization, the solution is an approximation close to the exact solution. I have a doubt that if FEM discretization error is mainly the error when us integrating with elements the bi-linear term in the weak form? If not the case, what is the other error source for the discretization error? &lt;/p&gt;&#xA;" OwnerUserId="24268" LastActivityDate="2017-09-23T13:31:57.290" Title="Is FEM discretization error equivalent to quadrature error of bi-linear term in the weak form?" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27904" PostTypeId="2" ParentId="27903" CreationDate="2017-09-23T13:31:57.290" Score="4" Body="&lt;p&gt;&lt;strong&gt;No&lt;/strong&gt;, it is not equivalent. The main discretization error comes from restricting the space in which the solution is sought from a space of (weakly) differentiable functions to a subspace of piecewise polynomials -- this is &lt;em&gt;Céa's lemma&lt;/em&gt;, which says that the discretization error is bounded by the best approximation error. Typically, these can then be integrated (and differentiated) exactly, so that there is &lt;em&gt;no&lt;/em&gt; additional integration error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do integrate the weak form inexactly (e.g., if you have spatially varying, non-polynomial, coefficients), the (first) &lt;em&gt;Strang lemma&lt;/em&gt; can be used to estimate the additional error (which is usually of the same order as the approximation error, if the quadrature rule is exact for constant coefficients).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible other source of discretization error arises in the case that the  space for the discrete solution is not a subspace of the space for the continuous solution (so-called &lt;em&gt;non-conforming&lt;/em&gt; approaches such as, e.g., discontinuous Galerkin methods). The error from this &quot;gap&quot; can be estimated by the second &lt;em&gt;Strang lemma&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2017-09-23T13:31:57.290" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27905" PostTypeId="1" CreationDate="2017-09-24T00:25:08.497" Score="1" ViewCount="425" Body="&lt;p&gt;As described in &lt;a href=&quot;https://en.wikipedia.org/wiki/Kronecker_sum_of_discrete_Laplacians&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Wikipedia article&lt;/a&gt;, a discrete laplacian matrix can be made for a 3D regular grid using Kronecker products. I'd like to use the same methodology for $(n-1)\times n$ matrices of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{bmatrix}&#xA;    -1 &amp;amp; 1 \\&#xA;    &amp;amp; \ddots &amp;amp; \ddots \\&#xA;    &amp;amp; &amp;amp; -1 &amp;amp; 1 \\&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given 3 of these matrices with sizes appropriate for each dimension, I'd like use Kronecker products to build a matrix for 3D problems. For example, given 3D dimensions of $m\times n \times k$, I'd expect a finite-difference matrix of dimension $(m-1)(n-1)(k-1) \times mnk$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I build this desired matrix using Kronecker products?&lt;/p&gt;&#xA;" OwnerUserId="23647" LastActivityDate="2017-09-24T00:25:08.497" Title="3D Finite-Difference using Kronecker Products" Tags="|finite-difference|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27906" PostTypeId="1" CreationDate="2017-09-24T01:05:17.173" Score="1" ViewCount="38" Body="&lt;p&gt;I am modeling the structure of a crystal and need to find what the &quot;lowest energy structure&quot; is. There are three free parameters (let's say, $a$, $b$, and $c$) to change, and the search space, $E(a,b,c)$ is convex. What is the best way to set this up so I vary $a$, $b$, and $c$ to find the combination that yields the minimum of $E(a,b,c)$ without running extraneous calculations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am currently doing is picking a relatively wide range of $a$, $b$, and $c$ values and computing $E$ for each combination. Then I fit a paraboloid to series of 3D plots of $E$ vs. $a$ vs. $b$ at fixed $c$ value and find where the global minimum is located. This was fine when my calculations were quick, but I need to be more efficient now with how to improve how I am searching the parameter space. One can think of something simple like making a range of $a$, $b$, and $c$, and keep expanding this range until $E$ increases when changing in all 3 dimensions from a given point, for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions of an adaptive search algorithm for a problem like this?&lt;/p&gt;&#xA;" OwnerUserId="22245" LastActivityDate="2017-09-24T01:05:17.173" Title="3-parameter optimization of a convex function" Tags="|optimization|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27907" PostTypeId="1" AcceptedAnswerId="27914" CreationDate="2017-09-24T05:23:32.260" Score="8" ViewCount="734" Body="&lt;p&gt;I'm writing a small numerical library for 2x2, 3x3, and 4x4 matrices (real, unsymmetric).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of numerical analysis texts highly recommend against computing the roots of the characteristic polynomial and recommend using the double-shifted QR algorithm. However, the size of the matrices makes me question whether or not simply computing the characteristic polynomial and finding the roots can be sufficient for this. I found &lt;a href=&quot;https://math.stackexchange.com/questions/1583883/how-do-i-compute-the-eigenvalues-of-a-small-matrix&quot;&gt;this answer on StackExchange&lt;/a&gt; that supports this, but I know that errors in the polynomial coefficients can produce inaccurate zeros of the polynomial (and thus different eigenvalues). On the other hand, the equation is only quartic at best and we have analytical formulas for the polynomial roots so we shouldn't get too far off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros and cons to using the characteristic polynomial for getting eigenvalues this specifically for this case?&lt;/p&gt;&#xA;" OwnerUserId="21131" LastActivityDate="2017-10-02T01:03:55.860" Title="Eigenvalues of Small Matrices" Tags="|linear-algebra|eigenvalues|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27908" PostTypeId="1" AcceptedAnswerId="27909" CreationDate="2017-09-24T09:20:59.137" Score="0" ViewCount="237" Body="&lt;p&gt;I'm trying to understand the passage through this equation can be written for easily solved with the fortran alghorithm in particular i don't understood the meaning of L_x and L_xx ... what (-1,0,1) stands for ?&#xA;could somebody explain me ? thanks a lot !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the equation is reported here (explicit scheme is clear .. obviously) &#xA;&lt;a href=&quot;http://hmf.enseeiht.fr/travaux/CD0001/travaux/optmfn/hi/01pa/hyb41/reportbid.htm#explicit&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25479" LastActivityDate="2017-09-24T11:36:20.450" Title="implicit method (crank-Nicolson) I not understand the procedure" Tags="|pde|fortran|hyperbolic-pde|crank-nicolson|" AnswerCount="1" CommentCount="0" ClosedDate="2017-09-28T13:47:27.117" ContentLicense="CC BY-SA 3.0" />
  <row Id="27909" PostTypeId="2" ParentId="27908" CreationDate="2017-09-24T10:18:22.283" Score="1" Body="&lt;p&gt;$L_x$ and $L_{xx}$ are shorthands (operators) to denote the more extended notation: $$ L_x u_i=(u_{i+1}-u_{i-1}) $$ and $$ L_{xx}u_i = (u_{i-1}-2u_i+u_{i+1})$$. Therefore $L_x$ can be written in the node $i$ as the vector $L_x=(-1,0,1)$ (see the coefficients of $u_{i-1}$ which is $-1$, $u_i$ zero and $u_{i+1}$ one). The same results for $L_{xx}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a better comprehension one could formally write, for a given stencil: $\tilde{u}_i=(u_{i-1}, u_i,u_{i+1})^{T}$ the scalar product:&#xA;$$L_x \tilde{u}_i=(-1,0,1)(u_{i-1}, u_i,u_{i+1})^{T}=u_{i+1}-u_{i-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. For a description of the procedure of the CN scheme you can see: &lt;a href=&quot;https://scicomp.stackexchange.com/a/27190/22590&quot;&gt;https://scicomp.stackexchange.com/a/27190/22590&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-09-24T10:28:28.427" LastActivityDate="2017-09-24T10:28:28.427" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27912" PostTypeId="1" AcceptedAnswerId="27913" CreationDate="2017-09-24T15:41:38.797" Score="4" ViewCount="304" Body="&lt;p&gt;I have the following set of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x(t) = x_0 \psi, \qquad y(t) = \kappa \ln \psi - x_0 \psi +1,\qquad z(t) =-\kappa\ln \psi,$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;t- t_0 = \int ^\psi_{\psi_0} \frac{d\eta}{\eta(1+\kappa\ln \eta + x_0 \eta)}.&#xA;$$&#xA;Here $t$ represents time. I would like to plot these equations in Python for the time range $t\in[0,1000]$. Of course, plotting $x,y$ or $z$ should no problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I am struggling to generate this plot. I am not looking of anyone to actually do this for me, but can I get a pointer on how to approach this? In particular, how should I generate the required $\psi$ for this problem given $\psi_0, x_0$ and $\kappa$?&lt;/p&gt;&#xA;" OwnerUserId="17499" LastEditorUserId="7574" LastEditDate="2017-09-24T16:25:48.827" LastActivityDate="2017-09-24T16:25:48.827" Title="How to solve this set of equations involving an integral?" Tags="|integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27913" PostTypeId="2" ParentId="27912" CreationDate="2017-09-24T16:04:56.177" Score="6" Body="&lt;p&gt;It is always easier to solve a differential equation rather than an integral equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can easily differentiate your last equation w.r.t the time variable $t$, and set the initial condition $\psi(0) = \psi_0$ for the following differential equation:&#xA;$$\frac{d\psi}{dt}=\psi(1+\kappa \ln{(\psi)}+x_0\psi).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have the solution for the above, substitute it in your trajectories of $x, y$ and $z$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="7574" LastEditDate="2017-09-24T16:24:51.060" LastActivityDate="2017-09-24T16:24:51.060" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27914" PostTypeId="2" ParentId="27907" CreationDate="2017-09-24T17:34:43.130" Score="12" Body="&lt;p&gt;The first thing to note is that the correspondence between finding&#xA;roots of a polynomial (&lt;em&gt;any&lt;/em&gt; polynomial) and finding the eigenvalues&#xA;of an arbitrary matrix is really direct, and it's a rich subject, see &lt;a href=&quot;https://doi.org/10.1007/s002110050069&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Pseudozeros of&#xA;polynomials and pseudospectra of companion matrices&lt;/em&gt; by Toh and&#xA;Trefethen&lt;/a&gt; and the references&#xA;there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, the 2×2 case is trivial and the standard formula,&#xA;$$ x_1 = \frac{-b-\mathrm{sign}(b)\sqrt{Δ}}{2a}, \qquad x_2 = c/(ax_1), \qquad Δ = \det \begin{pmatrix} b &amp;amp; 2a\\ 2c &amp;amp; b \end{pmatrix} $$&#xA;is numerically stable and accurate, so long as the determinant&#xA;$Δ$ is evaluated accurately—the direct formula will be inaccurate when $b^2≈4ac$, but there is an accurate formula due to Kahan that uses FMAs (&lt;a href=&quot;https://hal.inria.fr/ensl-00649347v1/document&quot; rel=&quot;noreferrer&quot;&gt;https://hal.inria.fr/ensl-00649347v1/document&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no such direct equivalent even for cubic polynomials. (&lt;em&gt;Edit: see the link to Kahan's method below in CADJunkie's comment. This might well be wrong.&lt;/em&gt;) The&#xA;direct formula is not always numerically stable (contrary to your assumption, I think), and it can't be made&#xA;numerically stable the same way as the quadratic formula by inserting&#xA;the right signs somewhere. You could try evaluating it with extra precision, e.g. with double-native arithmetic. But the approaches that do work directly on&#xA;the polynomial are pretty complicated. For example&#xA;(&lt;a href=&quot;https://doi.org/10.1145/2699468&quot; rel=&quot;noreferrer&quot;&gt;https://doi.org/10.1145/2699468&lt;/a&gt;, which also works on quartic polynomials) you could use Newton's method with a&#xA;good precomputed first guess, but it gets really rather complicated, and the&#xA;speedup is not even all that large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explicit formulas for degree-4 polynomials are likewise not always&#xA;numerically stable. The polynomials that are the hardest tend to have&#xA;uncommon roots (small, or close to each other, differing in magnitude&#xA;by a lot), but even testing your code on a few billion purely random polynomials can usually reveal numerical errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One curious thing related to this is that Jenkins-Traub, which is a good common way of finding polynomial roots, is actually an eigenvalue algorithm (inverse iteration) in disguise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would say the explicitness of the formulas is, in a way, misleading:&#xA;it tricks you into thinking that because the formula has a closed form&#xA;that means it's somehow cheaper/easier. I would really recommend that you actually test/benchmark this on some test data.&#xA;It doesn't have to be true:&#xA;determining the roots of degree-$≥3$ polynomials is within a small&#xA;integer factor of difficulty of the full problem of determining the&#xA;eigenvalues of a small matrix, and the standard library routines for&#xA;eigenvalues are much more robust and well-tested. So by reducing the small&#xA;eigenvalue problem to a low-degree polynomial roots problem, you&#xA;aren't necessarily simplifying it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are the pros and cons to using the characteristic polynomial for getting eigenvalues this specifically for this case?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think the main con is that this assumption that you make:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;On the other hand, the equation is only quartic at best and we have analytical formulas for the polynomial roots so we shouldn't get too far off.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;that because a formula is in closed form, and analytical, that means it's easy/cheap/accurate, is not necessarily true. It could be true on specific data you might have, but as far as I know it's not true in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. The whole distinction between in-closed-form and not-in-closed-form gets really finicky with computer arithmetic: you might think that $\cos (\cdots)$ in a cubic formula is a closed form, but as far as computer arithmetic is concerned, that's just another approximate rational function, it is maybe faster, but not fundamentally different, from the approximate rational function that defines the result of an eigenvalue algorithm.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-09-25T18:32:15.793" LastActivityDate="2017-09-25T18:32:15.793" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27917" PostTypeId="1" AcceptedAnswerId="27919" CreationDate="2017-09-25T04:15:35.770" Score="1" ViewCount="364" Body="&lt;p&gt;Consistent variation occurs in papers multiple times when reading . So what does it really mean from a maths view? I searched online and found an answer for it. Someone explained it as ‘Solution is consistent if when you plug it to the strong form, the solution satisfies it. For example, a penalty method is not consistent; however, a Nitsch method is’. Can anyone shed some light on this please?&lt;/p&gt;&#xA;" OwnerUserId="24268" LastActivityDate="2017-09-25T12:33:10.927" Title="What does Consistent Variation mean?" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27919" PostTypeId="2" ParentId="27917" CreationDate="2017-09-25T11:13:42.227" Score="0" Body="&lt;p&gt;The variational method is consistent if it converges to exact solution for series of mesh refinements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For penalty method, $\gamma$ is a model parameter, whereas for Nitsche method $\gamma$ is stabilisation parameter, which controls convergence rate, and for any stable $\gamma$ numerical solution converges to an exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that the exact solution of a continuous problem with constraints&#xA;($\mathcal{C}\mathbf{u} - \mathbf{g} = \mathbf{0}$), is exactly interpolated&#xA;on the finite element approximation space. We show that Nitsche's method is&#xA;consistent with such an exact solution. Taking the weak equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{split}&#xA;a(\mathbf{u},\mathbf{v})&#xA;-&#xA;\sum_E&#xA;\int_\Gamma&#xA;\mathbf{t}^{\textrm{T}}(\mathbf{u})&#xA;\phi\gamma\mathcal{P}\mathbf{t}(\mathbf{v})&#xA;\textrm{d}\Gamma&#xA;+&#xA;\sum_E&#xA;\int_\Gamma&#xA;\frac{1}{\gamma_0}&#xA;\left\{&#xA;  \mathcal{R}(\mathcal{C}\mathbf{u}-\mathbf{g}-\gamma\mathcal{C}\mathbf{t}(\mathbf{u}))&#xA;\right\}^\textrm{T}&#xA;\mathcal{R}(\mathcal{C}\mathbf{v}-\phi\gamma\mathcal{C}\mathbf{t}(\mathbf{v}))&#xA;\textrm{d}\Gamma&#xA;= 0&#xA;\end{split}&#xA;\end{equation}&#xA;where $\mathcal{R} = \mathcal{C}^\textrm{T}(\mathcal{C}\mathcal{C}^\textrm{T})^{-1}$, $\mathcal{P}=\mathcal{R}\mathcal{C}$ and $\phi$ takes values $-1,0,1$, respectively for skew-symmetric, non-symmetric, symmetric variant of Nitche method, respectively. Substituting the tractions approximation&#xA;\begin{equation}&#xA;\mathbf{t}(\mathbf{u}) =&#xA;-\frac{1}{\gamma}&#xA;\mathcal{R}(\mathcal{C}\mathbf{u}-\mathbf{g}-\gamma\mathcal{C}\mathbf{t}(\mathbf{u})),&#xA;\end{equation}&#xA;as a result of simple algebraic manipulations, the standard weak from is recovered&#xA;\begin{equation}&#xA;a(\mathbf{u},\mathbf{v})&#xA;-&#xA;\int_\Gamma&#xA;{\boldsymbol\sigma}^\textrm{T}(\mathbf{u})\mathbf{N}^\textrm{T}\mathbf{v} \textrm{d}&#xA;\Gamma = 0.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This shows the solution of Nitsche's method is a solution of original problem&#xA;with constraints. In other words, as long as the&#xA;problem is well-posed, solution of Nitsche's method converges to the original&#xA;problem with the given constraints. Note that penalty method solution is not consistent,&#xA;in a sense that the solution of the original problem with constraints is not&#xA;a solution when penalty terms enforce constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In using the classical penalty method, to verify correctness of the results one needs&#xA;to show that results converge for both increasing mesh density (approximation&#xA;order) and for $\gamma_0 \to 0$. Since Nitsche's method is consistent, it is only&#xA;necessary to demonstrate solution convergence when we refining mesh or increasing approximation&#xA;order. Here $\gamma_0$ is understood as the stabilisation&#xA;parameter which controls the convergence rate. However, since we work with finite&#xA;precision arithmetics, $\gamma_0$ cannot be too small to have well a&#xA;conditioned matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information about notation above and more detail explanation you will find here &lt;a href=&quot;http://mofem.eng.gla.ac.uk/mofem/html/nitsche_periodic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mofem.eng.gla.ac.uk/mofem/html/nitsche_periodic.html&lt;/a&gt;&#xA;where Nitche method is used to enforce periodic constraints.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-09-25T12:33:10.927" LastActivityDate="2017-09-25T12:33:10.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27920" PostTypeId="1" CreationDate="2017-09-25T15:07:43.353" Score="3" ViewCount="664" Body="&lt;p&gt;I have to repeatedly calculate a function which contains a sum of a large number (~100) of exponential terms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x) =  \sum_{r=1}^{100} C_r e^{b_r \cdot x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no relation between the $C_r$ and $b_r$ (i.e. something like $C_{r+1}$ = 2 $C_r$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know a fast way to determine a numerical approximation for such a sum of exponentials?&lt;/p&gt;&#xA;" OwnerUserId="25502" LastActivityDate="2018-02-24T22:05:22.963" Title="Numerical calculation sum of exponential functions" Tags="|numerics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27922" PostTypeId="2" ParentId="27690" CreationDate="2017-09-25T16:52:03.893" Score="2" Body="&lt;p&gt;If a PDE is derived from a principle of least action and if it has invariance, then some quantities are conserved. For instance, in mechanics, invariance with respect to time implies conservation of a certain scalar quantity (called &quot;energy&quot;), invariance with respect to the origin of the frame implies conservation of a certain vector quantity (called &quot;momentum&quot;) and invariance with respect to the orientation of the frame implies conservation of another vector quantity (&quot;angular momentum&quot;). There is a general theorem due to Emmy Noether that studies the structure of these symmetries and conserved quantities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the example of point mechanics, Landau's course [1] is crystal clear and very complete. About Emmy Noether's general theory, [2] is very enlighting and reasonably easy to read without requiring too much background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Landau and Lifshitz, Course on Theoretical Physics, Volume 1 - Mechanics&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Dwight Neuenschwander, Emmy Noether's Wonderful Theorem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that mass convervation does not belong to the same category of preserved quantities. For instance in fluid dynamics it is explicitly enforced by the continuity equation.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-09-26T07:08:08.217" LastActivityDate="2017-09-26T07:08:08.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27923" PostTypeId="1" CreationDate="2017-09-25T18:38:28.253" Score="0" ViewCount="1174" Body="&lt;p&gt;I have taken a course on undergrad scientific computing which discussed nonlinear algebraic equations about half-way in, and PDEs at the very end, but never discussed nonlinear PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in my research work, we have to use a solver that deals with a nonlinear coupled system of time-independent PDEs and I'm trying to have a very high level conceptual understanding of how Newton's method might have been applied in that solver internally, as well as a practical question of dealing with an unstable (stiff would be a better word?) scheme (Note: I'm familiar with FD/FV/FE and our solver uses FV for discretization, but I would use FD stencils for my discussion below). Kindly confirm or correct my line of thinking as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conceptual question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Newton's method applied to two nonlinear algebraic equations in two variables, the Jacobian matrix would be 2 by 2. However, if we have a coupled system of 2 nonlinear PDEs, let's say in 1-D (think a Poisson equation, and a continuity equation derived from Maxwell's equations, both nonlinearly coupled to each other in space, but otherwise time-independent),&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;First we would decide on the discretization and pick N points lying along the 1-D simulation domain. This will result in 2 times N algebraic equations (N algebraic equations for each of the 2 differential equations).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Each algebraic equation would have M independent variables, the value depending on the method of discretization, e.g., using 3-point FD stencil for 1-D, we have each algebraic equation in 3 variables? (except at the boundaries where the variables will be less than 3)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What would be the size of Jacobian? I guess the number of rows would be 2 times N, but the number of columns would depend on discretization?&#xA;Again for 3-point FD, we'll have a 2-times-N by 3 Jacobian matrix? Or would it be 2-times-N by 2-times-N sparse Jacobian matrix? or something completely different? (&lt;em&gt;edit 0&lt;/em&gt;: on further thought, I think it would be 2-times-N by N since we have 2-times-N algebraic equations but only N variables?)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If dimensionality of the system goes from 1-D to 2-D, and let's say we now use 5-point FD stencil, with N^2 discretization points. Would the Jacobian matrix be 2-times-N^2 by 5? or sparse 2-times-N^2 by 2-times-N^2?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Practical question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, (somewhat separate from above), from a practical point of view, such a Newton scheme would be very unstable, especially if the PDEs are highly nonlinear in character, and therefore one way to mitigate this is to use, what our solver is using, a ramp-up Newton's method? where &quot;ramp-up&quot; is described as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In a first step we ignore the given external Dirichlet BC, and use zero-everywhere BC (think Poisson equation). (As an aside, we could further split this into two steps by first doing, not only zero BC, but zero RHS, which turns it into a Laplace equation, then, in the second step, we iteratively ramp-up the RHS 1% at every step, similar to what I describe below).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And we solve one of the two equations first, e.g., only the Poisson equation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then we use the solution in a coupled solve, still zero-everywhere DBC.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now we crank up the BC to 1% of target value, and do a coupled solve.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And we keep going, 1% at every step (call them steps of an outer loop), until we hit target DBC.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there a better name for this kind of ramp-up Newton scheme, or &quot;guided Newton scheme&quot;, which feels like a forward Euler even though there is no time involved (it's not an initial value problem)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, a third question, if I try to solve the two PDEs in a &quot;de-coupled&quot; way, i.e., solve Poisson equation, use the solution as input to the continuity equation and solve it, and use that solution back into the next Poisson solve, that would make it a self-consistent scheme, which is different from Newton scheme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also a recommended reading on this topic that gives such a rundown would be highly appreciated. Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1&lt;/strong&gt;: Came across &lt;a href=&quot;https://scicomp.stackexchange.com/questions/21123/solving-nonlinear-differential-equations-with-newtons-method#comment33743_21123&quot;&gt;a very useful pdf shared here&lt;/a&gt;. I realized a better name for &quot;guided Newton's method&quot; is Newton's method with continuation (or numerical continuation). Plus there is other very useful information in the pdf. However I would still highly appreciate further comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2&lt;/strong&gt;: I guess another question related to the second one above is, which of the two is a better answer for the reasoning behind using Newton's method with continuation instead of direct Newton's method:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Because high nonlinearity makes Newton's method unstable/stiff? (the typical explanation being, Newton's method works best if it starts very close to the solution, which means the initial guess for NM probably needs to be a solution of some non-NM solver that does not have that problem.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Because high nonlinearity means solutions are not unique, and in a space of solutions, 99% are non-physical, and in order to reach the 1 physically meaningful solution, we have to &quot;guide&quot; the method we're using, through continuation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(in other words, is it an issue with the method, unstable/stiff, or is it an issue with the system of PDEs, many non-unique solutions most of which are physically meaningless?)&lt;/p&gt;&#xA;" OwnerUserId="15198" LastEditorUserId="15198" LastEditDate="2017-09-25T20:13:32.117" LastActivityDate="2017-09-26T06:51:27.707" Title="Newton method for a nonlinear system of time-independent PDEs" Tags="|pde|nonlinear-equations|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27924" PostTypeId="2" ParentId="27923" CreationDate="2017-09-26T00:09:51.583" Score="4" Body="&lt;p&gt;Conceptually, forget about the fact that the problem is time dependent for a moment. That's because if you apply a time stepping scheme to your time dependent PDE, you're going to get a (in your case nonlinear) PDE or a system of PDEs that solves for the solution $u^n(x)$ of the PDE at time step $n$ and where $u^{n-1}$, the solution of the previous time step, appears as known data. So whether or not the PDE is time dependent, ultimately you'll have to solve a nonlinear equation that's not time dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how to approach this stationary problem is a different question. I could try to write a lot of words on this, but let me instead just link to a bunch of lectures I've already recorded -- see lectures 31.5 and following here: &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-09-26T00:09:51.583" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27925" PostTypeId="2" ParentId="27690" CreationDate="2017-09-26T01:00:53.880" Score="2" Body="&lt;p&gt;As Bruno Levy pointed out, certain classes of physical systems have well-known structural properties. For example, consider a Hamiltonian system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot z = J\nabla H(z)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $J$ is a symplectic matrix and $H$ is the Hamiltonian. The Hamiltonian $H$ itself is preserved, as is any other function $F$ such that the Poisson bracket $\{H, F\} = \nabla H^\top J\nabla F = 0$ for all $z$. Symmetries of these kinds of systems give rise to conservation laws, and there are &lt;a href=&quot;http://epubs.siam.org/doi/10.1137/1030094&quot; rel=&quot;nofollow noreferrer&quot;&gt;software packages&lt;/a&gt; using symbolic algebra that can automatically discover some symmetries of arbitrary ODE systems (see also &lt;a href=&quot;http://www.swmath.org/?term=Lie%20symmetry&quot; rel=&quot;nofollow noreferrer&quot;&gt;this list&lt;/a&gt;). Hamiltonian systems have other useful properties not directly related to symmetries or conservation laws. For example, every equilibrium point is either a saddle point or a center, i.e. there are no asymptotically stable equilibria; the volume of a region of phase space is always preserved under Hamiltonian flow; and many others. However, it is arguably &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverse_problem_for_Lagrangian_mechanics&quot; rel=&quot;nofollow noreferrer&quot;&gt;not feasible&lt;/a&gt; to algorithmically determine whether some totally arbitrary system is Hamiltonian if you weren't already sure to begin with, especially when you take into account non-canonical Poisson structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other kinds of systems have different structural properties. For example, consider a gradient flow&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dot z = -\nabla F(z)$;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then $dF/dt \le 0$ along the trajectories of this system. The heat equation is the prototypical example of a gradient flow. Gradient flows have equilibria at extrema of $F$ and the stability of an equilibrium point depends on the eigenvalues of the second derivative of $F$. Unlike Hamiltonian systems, a gradient flow can have stable equilibria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both Hamiltonian systems and gradient flows have interesting and useful structure, but in general there is no algorithm to take an arbitrary differential equation and find what class of system it belongs to. You can of course rule out some possiblities. For example, if the system clearly has an asymptotically stable equilibrium, it can't be Hamiltonian. Likewise, if the linearization of the system is not a symmetric matrix, it can't be a gradient flow by &lt;a href=&quot;https://en.wikipedia.org/wiki/Symmetry_of_second_derivatives&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clairaut's theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, there are always weird examples from real applications that defy any classification scheme. For example, linear PDE are often classified as being either elliptic, hyperbolic, or parabolic, but there are numerous examples that don't fit neatly into these categories: the &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler%E2%80%93Tricomi_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler-Tricomi equation&lt;/a&gt; changes type within the spatial domain; the PDEs that describe thin-film flows are a mixed elliptic-hyperbolic system; etc.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2017-09-26T18:31:27.780" LastActivityDate="2017-09-26T18:31:27.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27926" PostTypeId="2" ParentId="27923" CreationDate="2017-09-26T06:38:10.057" Score="0" Body="&lt;p&gt;Let me propose you a PDE example:&#xA;$$-L(x,u)u = f(x)  \qquad + \quad BC\tag{1}$$&#xA;Where $L$ is an operator depending on the spatial coordinates $x$ and the variable $u$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly the problem $(1)$ is nonlinear. This nonlinearity may be high or low, it depends on the form of the operator $L$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Newton's method, linearises equation $(1)$ with $u^{n+1} = u^{n}+ \delta u$, leading to the following scheme:&#xA;$$-L(x,u^n+\delta u)(u^n+\delta u) = f(x)$$&#xA;Expanding the latter in powers of $\delta u$ one obtains the equation for $\delta u$, provided $u^n$&#xA;$$ -L_u(x,u^n)\delta u =L(x,u^n)u^n+ f(x) \tag{2}$$&#xA;Where $L_u$ is the derivative of the operator w.r.t. $u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scheme presented in $(2)$, if highly nonlinear, strongly depends (as you say) on the seed $u^0$, which must be as close as possible to the true solution of $(1)$. One smart option is solving the problem with the hypothesis $u\ll 1$:&#xA;$$\lim_{u\to 0}\left[-L(x,u)\right]u^0 = f(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other methods consist on solving a time-dependent version of $(1)$, asserting (strong hypothesis) that &#xA;$$\lim_{t\to \infty}\frac{\partial u}{\partial t} = 0\tag{3}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hypothesis $(3)$ is not always true. For example in the Navier-Stokes equation for the cylinder problem, from one specific Reynolds number, $(3)$ does not hold any more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, if you are sure of that your problem will tend to  stationary solution you can safely add the time derivative term to $(1)$:&#xA;$$\frac{\partial u}{\partial t}-L(x,u)u = f(x)  \qquad + \quad BC\quad+ \quad IC\tag{4}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The set-up in $(4)$ converges to the true solution iff you provide an initial condition close enough to the actual stationary solution (as for the seed of Newton's method, but now instead of seed it is identified with initial condition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest way to obtain a solution of $(4)$ is discretising the time derivative with a first order accuracy scheme. The rest of the terms belonging to the operator $L$ deserve a deeper discussion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The high order derivatives (more that second order) must be implicitly treated (otherwise the timestep would be prohibitively small in order to solve the system as a set of quasi-isentropic states), I mean, you must solve them for the next time step, for example $\triangle u^{n+1}$, while the first order derivatives  may be explicitly treated (stability issues arise when diffusive effects do not dominate or Courant number is small enough) as for example $u^n\textrm{grad}u^n$, or implicitly treated (stability issues do not srise but you pay the cost of the nonlinearity of the system you have to solve by means of e.g. Newton Raphson because it will converge extremely fast within 2 iterations) for example $u^{n+1}\textrm{grad}u^{n+1}$. Other options involve a &quot;linearisation&quot; including, instead, the term $u^n\textrm{grad}u^{n+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The safest method (a large time-step is required because, recall that we only want to reach a stationary solution) is the implicit scheme, given by the discretised version of $(4)$&#xA;$$\frac{ u^{n+1}-u^n}{\delta t}-L(x,u^{n+1})u^{n+1} = f(x)  \tag{5}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scheme $(5)$ converges more easily to a physical solution than $(2)$, once provided that $(3)$ holds for your problem. This is so because the previous $u^n$ used for the obtainment of $u^{n+1}$ has a physical meaning, on the contrary of what occurred in $(2)$.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-09-26T06:51:27.707" LastActivityDate="2017-09-26T06:51:27.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27928" PostTypeId="1" CreationDate="2017-09-26T10:18:22.420" Score="2" ViewCount="48" Body="&lt;p&gt;I am implementing a Kalman filter (for a linear ODE system for now). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My model represents a physical device that has 6 &quot;parameters&quot;, i.e. those values of the device do not evolve over time (within a time-domain simulation environment for now).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 100 such &quot;slightly differing devices&quot;, i.e. whose parameter values differ from the nominal, but the distribution of all 6 of the model parameters are known apriori (Gaussian, with known mean (nominal values) and standard deviations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming that there is no other source of process noise (i.e. no unmodeled states or other dynamics are present)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am implementing a Kalman Filter with a single tuning parameter across all these 100 devices. I think that these deviations in model parameters (from the nominal) can be treated as process noise.&#xA;Is there a deterministic way, i.e. a recipe, to compute the &quot;Process noise covariance matrix&quot;, typically denoted as $Q$ in Kalman Filter literature?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking that one can somehow exploit the affine properties of Guassian distributions to arrive at the error covariance matrix, although I am a bit rusty with the math.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Partially related, but different question asked by someone else:&#xA;&lt;a href=&quot;https://dsp.stackexchange.com/questions/21796/question-about-q-matrix-noise-process-covariance-in-kalman-filter&quot;&gt;https://dsp.stackexchange.com/questions/21796/question-about-q-matrix-noise-process-covariance-in-kalman-filter&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16900" LastEditorUserId="16900" LastEditDate="2017-09-26T22:31:40.700" LastActivityDate="2017-09-26T22:31:40.700" Title="Deterministic method to compute &quot;Process noise covariance matrix, Q&quot; for a Kalman filter when parameter variations of the model is known apriori" Tags="|linear-algebra|matrix|error-estimation|probability|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27931" PostTypeId="1" CreationDate="2017-09-26T20:00:38.793" Score="3" ViewCount="125" Body="&lt;p&gt;I've got a data set of points $M_O .. M_N$ for time points $t_0 .. t_N$, where $N$ is approximately 10-20, and the spacing of time is not uniform (i.e., $t_{i+1}-t_i$ is not constant for all i). It is believed that this data should be able to be modeled by a differential equation of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{dM}{dt} = -k (M - M_{e})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to solve for the constant $k$. If $M_e$ were a constant, then the solution of the function $M$ would of course be something like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M = M_{e} + Ce^{-kt}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I could then fit this function to the data and determine $k$ with any regular minimization routine (say, using the RMSE).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, $M_{e}$ is not a constant, but instead varies with $t$. I already have the values of $M_e$ for all $N$ points in my data set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what is the best way to numerically optimize for the $k$ which fits the model best? Naively, I could approximate $\frac{dM}{dt}\approx \frac{\Delta M}{\Delta t} $, compute the differences for each segment, and choose something like the mean value of $k$ from each $\Delta t$. I'm curious as to whether this is the best way to do this, or if there's another more appropriate method. One possible weakness of what I'm describing is that averaging $k$ for all segments ignores the fact that some $\Delta t$ intervals are larger than others, and perhaps smaller $\Delta t$ should be systematically favored over larger time intervals. Finally, is there any good method to get a somewhat closed form solution for $M(t)$, or does this hinge on whether there's a simple closed-form solution to $M_e(t)$ (which is doubtful in my case because of its specific physical origin). &lt;/p&gt;&#xA;" OwnerUserId="5196" LastEditorUserId="5196" LastEditDate="2017-09-26T20:05:45.043" LastActivityDate="2017-09-26T20:05:45.043" Title="How to optimize for decay constant in exponential-like function?" Tags="|optimization|numerical-analysis|finite-difference|python|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="27932" PostTypeId="2" ParentId="27840" CreationDate="2017-09-27T13:34:31.477" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Closing_(morphology)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Morphological Closing&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A morphological closing is a combination of dilation followed by erosion; typical image processing operations available in &lt;em&gt;most&lt;/em&gt; image processing libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SciPy has a &quot;simple&quot; &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.16.0/reference/generated/scipy.ndimage.morphology.binary_closing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;binary closing method&lt;/a&gt; which does this on binary images. By experimenting with different &lt;em&gt;structuring elements&lt;/em&gt; you can close smaller or bigger gaps.&lt;/p&gt;&#xA;" OwnerUserId="1789" LastActivityDate="2017-09-27T13:34:31.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27934" PostTypeId="1" AcceptedAnswerId="27935" CreationDate="2017-09-27T14:09:46.890" Score="1" ViewCount="228" Body="&lt;p&gt;I posted a similar question yesterday but I deleted it since I think that I had to reformulate it after some insights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to calculate &#xA;$$&#xA;\exp(-i\Delta t\,\mathcal{H}) = V\,\mathrm{diag}(\{\exp(-i\Delta t\,\epsilon_k\})\,V^{-1} ~~~~~~~~~~~~~(*)&#xA;$$&#xA;where $\mathcal{H}$ is a hermitian matrix (the hamilton matrix) and $V$, $\epsilon_k$ are its eigenvectors and -values.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Since I'm implementing this in a given DFT package, I make use of the existing eigensolver routine. All numerics are done by ScaLAPACK. It first computes the Cholesky factorization of the overlap matrix $S_{ij}=\langle\phi_i|\phi_j\rangle$:&#xA;$$&#xA;S = U^TU&#xA;$$&#xA;where $U$ is the upper triangular matrix. In the next step, $U$ is inverted for further usage. It then solves the eigenvalue problem for the transformed hamilton matrix&#xA;$$&#xA;\mathcal{H}' = U^{-\dagger}\mathcal{H}\,U^{-1}&#xA;$$&#xA;which yields the eigenvectors $V'$. This is done since we have a generalized eigenvalue problem because a non-orthogonal basis set is used. The eigenvectors are afterwards backtransformed via &#xA;$$&#xA;V = U^{-1} V'.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to calculate $(*)$ by inverting $V$ which needs a LU decomposition since $V$ is a non-hermitian matrix. This turned out to be unstable after some iterations although it initially produced correct results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a different way to circumvent inverting $V$ which uses the not backtransformed eigenvectors $V'$. Here, the exponential is explicitly calculated via&#xA;$$&#xA;(U^{-1}V')\,\mathrm{diag}(\{\exp(-i\Delta t\,\epsilon_k\})(U^{-1}V')^{-1}=U^{-1}V'\,\mathrm{diag}(\{\exp(-i\Delta t\,\epsilon_k\})V'^\dagger U&#xA;$$&#xA;where $V^{-1}=V'^{\dagger}$ can be applied since $V'$ is unitary. Unfortunately, this yields oscillating solutions around the right solution (e.g. total energy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure why this isn't working and I'm quite new to the field. I tried $S^{-1}$ and $S$ in above formula instead of $U^{-1}$ and $U$ but it didn't help, also I tried a lot more modifications but I just can't figure out where the problem could be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would really appreciate some help. Thanks and best regards!&lt;/p&gt;&#xA;" OwnerUserId="25510" LastActivityDate="2017-09-28T11:40:57.520" Title="Matrix exponential by eigenvectors - implementation issues" Tags="|linear-algebra|computational-physics|eigensystem|quantum-mechanics|density-functional-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27935" PostTypeId="2" ParentId="27934" CreationDate="2017-09-27T17:03:01.207" Score="0" Body="&lt;p&gt;(Heavily edited because I made a big mistake in the original post)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We write the Time Dependent Kohn-Sham equations as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Hc = i\hbar S\frac{\delta c}{\delta t} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c$ are the coefficients of your wavefunction. This is also the same as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S^{-1}Hc = i\hbar\frac{\delta c}{\delta t} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and so we can update the wavefunctions as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c(t+\Delta t) = exp(-i\hbar S^{-1}H\Delta t)c(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now we want to compute $exp(-i\hbar S^{-1}H\Delta t)$, through the eigendecomposition as you said.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S^{-1}H = X^{-1}DX$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's write that in terms of a familiar problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S^{-1}HX = DX$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$HX = DSX$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the cholesky decomposition, as you suggested, we can turn this into the standard eigenvalue problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S = LL^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(L^{-1}HL^{-T})(L^{T}X) = D(L^{T}X)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L^T X = X'$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L^{-1}HL^{-T} = H'$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then just solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H'X' = DX'$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's work our way back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X = L^{-T}X'$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X^{-1} = X'^{-1}L^{T} = L^{T}X'^{T}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(since $X'$ is an orthogonal matrix).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now we know D, X, and $X^{-1}$, and thus can compute:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$exp(-i\hbar S^{-1}H\Delta t) = exp(-i\hbar X^{-1}DX\Delta t) = X^{-1}exp(-i\hbar D\Delta t)X$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's make sure I'm correct with some source code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.linalg&#xA;import numpy.random&#xA;import numpy&#xA;&#xA;# Setup&#xA;N = 10&#xA;H = 0.01*numpy.random.rand(N,N)&#xA;H = H+H.T # symmetric&#xA;S = numpy.random.rand(N,N)&#xA;S = S+S.T # symmetric&#xA;S = S.dot(S) # positive definite&#xA;Sinv = scipy.linalg.inv(S)&#xA;I = numpy.identity(N)&#xA;&#xA;# Reference answer&#xA;ref = scipy.linalg.expm(Sinv.dot(H))&#xA;&#xA;# Our solution&#xA;## first our cholesky factors&#xA;L = scipy.linalg.cholesky(S,lower=True)&#xA;Linv = scipy.linalg.solve_triangular(L, I, lower=True)&#xA;&#xA;## now solve the normal eigenvalue problem&#xA;Hprime = Linv.dot(H).dot(Linv.T)&#xA;D, Xprime = scipy.linalg.eigh(Hprime)&#xA;D = numpy.diag(D)&#xA;&#xA;## back subsite for X and X^{-1}&#xA;X = Linv.T.dot(Xprime)&#xA;Xinv = Xprime.T.dot(L.T)&#xA;&#xA;our_answer = X.dot(scipy.linalg.expm(D)).dot(Xinv)&#xA;&#xA;print(scipy.linalg.norm(ref - our_answer))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23968" LastEditorUserId="23968" LastEditDate="2017-09-28T11:40:57.520" LastActivityDate="2017-09-28T11:40:57.520" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27936" PostTypeId="1" CreationDate="2017-09-27T17:18:36.793" Score="1" ViewCount="50" Body="&lt;p&gt;I have a set $N$ points in $d$-dimensional space, $\mathbf{x}_i \in \mathbb{R}^d ~~ \forall i=\{1,2,...,N\}$. How to choose $n$ points from the set such that it maximises&#xA;$$&#xA;f = |\mathbf{S}|^{-n/2} \exp\left[-\frac{1}{2} \mathrm{tr}\left(\mathbf{S}\boldsymbol{\Sigma}^{-1}\right) - \frac{n}{2} (\boldsymbol{\mu} - \mathbf{\bar{x}})^T \boldsymbol{\Sigma}^{-1} (\boldsymbol{\mu} - \mathbf{\bar{x}})\right],&#xA;$$&#xA;for given $\boldsymbol{\Sigma}$ and $\boldsymbol{\mu}$, where $\mathbf{S} = \sum_j (\mathbf{x}_j - \mathbf{\bar{x}})  (\mathbf{x}_j - \mathbf{\bar{x}})^T$ and $\mathbf{\bar{x}} = \sum_j\mathbf{x}_j/n$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution I am thinking of is to try combinations of the points, but that will take exponential time to try all combinations. I wonder if there is a faster way, utilising the knowledge of function $f$? I don't need an exact set of point that has the global maximum of $f$. Random algorithms that achieve reasonable result in faster time are much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="25518" LastEditorUserId="25518" LastEditDate="2017-09-28T09:38:29.720" LastActivityDate="2017-09-28T09:38:29.720" Title="Selecting n points to have a given mean and covariance" Tags="|linear-algebra|optimization|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27937" PostTypeId="2" ParentId="27920" CreationDate="2017-09-27T18:11:55.783" Score="3" Body="&lt;p&gt;Sometimes, the best way to do this kind of things is rather simple. If the problem is computing this function a lot of times, then... don't compute it!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, all you have to do is to write a table for a finite set of values $x_j$. If you are going to compute $f(x)$ with $x\in[a,b]$, then you compute $f(x_j)$, where $x_j=a+j\Delta x$, $\Delta x=(b-a)/N$. The larger $N$, the better representation of the function you get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You compute this table once, at the beginning of the program. You could additionally write it to a file and simply recover it at the beginning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you need $f(x)$, with $x_j&amp;lt;x&amp;lt;x_{j+1}$, you return $f(x)\simeq(f(x_j)+f(x_{j+1}))/2$. Since your function is continuous, if $\Delta x$ is small enough, you are going to have a very good approximation for $f(x)$. And you avoid computing the function, since all the $f(x_j)$ are stored in memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &quot;trick&quot; really saves a lot of computations (at expense of having the results stored in memory) and in my opinion it is not used as much as it should.&lt;/p&gt;&#xA;" OwnerUserId="17425" LastActivityDate="2017-09-27T18:11:55.783" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27938" PostTypeId="1" CreationDate="2017-09-27T18:47:09.633" Score="0" ViewCount="58" Body="&lt;p&gt;In my problem, I have a lattice with a stochastic cellular automaton. In order to simplify a bit, let's say it is 1D. In my system, each node can be type A, B or C. A way to represent the system and operate fast over it is using an integer 64-bit variable, working bitwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me give an example of implementation for a system with 4 nodes. Let's say I have that nodes 0 and 3 are in state A, node 1 is in B, and 2 is in C, then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A -&amp;gt; 1001 -&amp;gt; A = 9&#xA;B -&amp;gt; 0100 -&amp;gt; B = 4&#xA;C -&amp;gt; 0010 -&amp;gt; C = 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You see that 3 integers can represent the whole state of the system. I can update the system with a single bit-wise operation. This is used a lot in cellular automatons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, now, suppose I have reactions such as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A+B\overset{\mu}{\rightarrow}2B$$&#xA;$$B\overset{\nu}{\rightarrow}C$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to simulate this reactions in a synchronous way using a discrete time $\Delta t$. I can obtain the number of B individuals in the next timestep with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;B = A &amp;amp;&amp;amp; r&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $r$ is a random number such that the bit at position $j$ is 1 with probability $p_j=1-\exp(-\mu\Delta t\cdot b_j)$, where $b_j$ is the number of B neighbours of individual $j$. This is, it is 1 with the probability of executing reaction 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: &lt;strong&gt;is there an efficient method to obtain $r$?&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="17425" LastEditorUserId="17425" LastEditDate="2017-09-28T07:55:05.310" LastActivityDate="2017-09-30T10:14:40.690" Title="Implementation of stochastic cellular automata" Tags="|discretization|stochastic|random-number-generation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27939" PostTypeId="1" AcceptedAnswerId="27940" CreationDate="2017-09-27T22:04:52.053" Score="2" ViewCount="54" Body="&lt;p&gt;I am trying to simulate an ODE, like $ \dot{x} = \xi(x) $ that should have a stationary distribution (a la Stat Mech). Assuming that my ODE algorithm generates time samples of my system state $ x $ like $ x^{(0)}, ..., x^{(t)}, x^{(t+1)}, ..., x^{(n)} $, I would like to verify that the samples generated from the ODE numerically approximately match the theoretical distribution (or at least in metrics like entropy/KL Divergence they match).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about running Monte Carlo and the ODE and trying to compare the generated results, but I don't know how to compare the samples generated as they are generated in a high dimensional phase space. Normally (low dimensions), I would use binning and compute KL divergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My motivation is to further convince myself that the results from &lt;a href=&quot;http://iopscience.iop.org/article/10.1209/epl/i1999-00139-0/meta&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;On the classical statistical mechanics of non-Hamiltonian systems&quot; by Tuckerman&lt;/a&gt; are real.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user23933" LastActivityDate="2017-09-27T22:53:14.647" Title="Verifying that ODE integration generates Theoretical Stationary distribution" Tags="|ode|statistics|probability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27940" PostTypeId="2" ParentId="27939" CreationDate="2017-09-27T22:53:14.647" Score="2" Body="&lt;p&gt;In general, your type of question would be called a &quot;multivariate goodness of fit test&quot;. If $F(x_1,\ldots,x_n)$ is the $n$-dimensional CDF for the theoretical distribution, and the random variables $(X_1,\ldots,X_n)$ are a sample from your ODE, then&#xA;$$ Z_1 = F(X_1), \quad Z_2 = F(X_2\mid X_1), \quad\cdots\quad Z_n = F(X_n\mid X_1,\ldots,X_{n-1}) $$&#xA;are random variables that are uniformly independently distributed in the unit cube $[0,1]^n$. This is  &lt;a href=&quot;https://doi.org/10.1214/aoms/1177729394&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosenblatt's transform&lt;/a&gt;, similar to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Probability_integral_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability integral transform&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if your system is ergodic and enough time passes between the ODE samples for them to be independent, you can just test that the vectors $z^{(0)},\ldots,z^{(n)}$ are u.i.d. on the unit cube. This in turn can be done, for example, by splitting the cube into bins and using the chi-square test. &lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-09-27T22:53:14.647" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27942" PostTypeId="1" CreationDate="2017-09-28T17:25:48.233" Score="1" ViewCount="65" Body="&lt;p&gt;I'm working through the book Approximate Dynamic Porgramming by Warren B. Powell. Currently I'm solving exercise 3.3 and 3.5. Let me state the exercise (I'm summarizing the text from the boook)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We have the following Markov Decision Problem. We have a two-stage Markov chain with one-step transition matrix&#xA;  $$ &#xA;P=&#xA;\begin{pmatrix}&#xA;0.7 &amp;amp; 0.3  \\&#xA;0.05 &amp;amp; 0.95 \\&#xA;\end{pmatrix}&#xA; $$&#xA;  The contribution from each transition from state $i\in\{1,2\}$ to state $j\in\{1,2\} $is given by &#xA;  $$&#xA;C = \begin{pmatrix}&#xA;10 &amp;amp; 30  \\&#xA;20 &amp;amp; 10 \\&#xA;\end{pmatrix}&#xA; $$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;that means going from state $1$ to state $2$ has a contribution (reward) of $30$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the exercise are asking to solve the problem using value iteration (infinite horizon) and policy iteration let me state the two algorithms from the book (again I'm quoting)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Value Iteration&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Set the initial value $v^0 = 0 $ for all state $s\in S$, fix a tolerance parameter &#xA;  $\epsilon$ and set $n=1$.&lt;/li&gt;&#xA;  &lt;li&gt;For each state $s$  compute&#xA;  $$ v^n(s) = \max_a (C(s,A) + \gamma\sum_{s'}P(s'|s,a) v^{n-1}(s'))$$&lt;/li&gt;&#xA;  &lt;li&gt;If $\|v^n-v^{n-1}\| &amp;lt;\epsilon(1-\gamma)/2\gamma $ let $ \pi^\epsilon$ be the resulting policy and $v^n$ the resulting value. Else set $n=n+1$ and go to step 2&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and the policy iteration (again quoting the book)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Policy Iteration&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Select a policy $\pi^0$ and set $n=1$. &#xA;  $\epsilon$ and set $n=1$.&lt;/li&gt;&#xA;  &lt;li&gt;Given the policy $\pi^{n-1}$ compute the transition matrix $P^n$ and the contribution vector $c^n$ where the element for state $s$ is given by $c^{n-1} = C(s, \pi^{n-1})$&lt;/li&gt;&#xA;  &lt;li&gt;let $v^{n-1}$ be the solution to ($I$ the identity matrix) $$(I - \gamma P^n) v = c^{n-1}$$&lt;/li&gt;&#xA;  &lt;li&gt;find a policy $\pi^n$ defined by $$ a^n(s) = \arg\max_a(C(a)+\gamma P^nv^n)$$ this requires to compute the action $a$ for each state $s$.&lt;/li&gt;&#xA;  &lt;li&gt;if $a^n(s)=a^{n-1}(s)$ for all $s$ then stop. Otherwise set $n=n+1$ and go to step $2$.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now let me state the two exercises. In exercise $3.3$ we need to achieve the following two tasks:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Exercise 3.3&lt;/strong&gt;&#xA;  Plot the value of being in state 1 as a function of the number of iterations if your initial estimate of value being in each state is $0$. Show the graph for $100$ iteration. Repeat the exercise with initial estimates of $100$ each and $100$ for state $1$ and $0$ for state. The author says (note that you are not choosing a decision so there is no maximization step)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I've used R to implement the value iteration:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P &amp;lt;- matrix(c(0.7, 0.3, 0.05, 0.95), nrow=2, byrow=T)&#xA;C &amp;lt;- matrix(c(10, 30, 20, 5),nrow=2, byrow = T)&#xA;discount &amp;lt;- 0.8&#xA;v &amp;lt;- matrix(rep(0,200), nrow=100)&#xA;for(i in 2:100){&#xA;  v[i,] &amp;lt;- (C[1,] + discount*sum(P[1,]*v[i-1,1]))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where I've used $P[1, ]$ since we are only interested in the value of being in state $1$. Here are the graphs for the different initial states&lt;/p&gt;&#xA;&#xA;&lt;p&gt;both $0$ initial value&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Qv65M.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Qv65M.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;both $100$ initial value&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZQrbn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZQrbn.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;and $100$ and $0$ initial value&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lh4v4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lh4v4.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;FIRST QUESTION&lt;/strong&gt; is the solution correct? I'm a bit unsure since the last initial value pair did not change the plot compared to the two above ones.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the second exercise is the following&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Exercise 3.5&lt;/strong&gt;&#xA;  Apply policy iteration to the above problem. Plot the average value function after each iteration alongside the average value function found using value iteration after each iteration (initial value $0$). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With this I'm struggling. It's not really clear to me how to use policy iteration if there is no policy to choose from as the author outlined above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the discussion with Kirill in the comments you can see that I've first tried to introduce a policy to solve this exercise which seems to be wrong. However, for completeness I leave the results in the old outdated part below. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OLD OUTDATED PART&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly there are policies of the form: being in state $i$ go either to state $j$ or stay at state $i$. We can model this via tuples $(i,j)$ meaning if I'm in state $1$ I go to state $i$ and if I'm in state $2$ I go to state $j$. I've also implemented the algorithm in R. Note that several steps of the above policy iteration are not needed to be recalculated since they are constant. What confuses me is that the author say above there are no decision but now we should solve it via policy iteration. Moreover, my resulting value is different which means something is wrong. Can someone help me to understand what's going wrong.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P &amp;lt;- matrix(c(0.7, 0.3, 0.05, 0.95), nrow=2, byrow=T)&#xA;C &amp;lt;- matrix(c(10, 30, 20, 5),nrow=2, byrow = T)&#xA;discount &amp;lt;- 0.8&#xA;v &amp;lt;- c(0,0)&#xA;current_policy = c(1,2)&#xA;id &amp;lt;- matrix(c(1,0,0,1), nrow=2, byrow=T)&#xA;inverse_factor = solve(id-discount*P)&#xA;new_policy = c(1,1)&#xA;while(any(new_policy != current_policy)){&#xA;  current_policy = new_policy&#xA;  c_pi = c(C[1,current_policy[1]],C[2,current_policy[2]])&#xA;  current_value = inverse_factor%*%c_pi&#xA;  a_1 = which.max(c(C[1,1]+discount*(P%*%current_value)[1],C[1,2]+discount*(P%*%current_value)[2]))&#xA;  a_2 = which.max(c(C[2,1]+discount*(P%*%current_value)[1],C[2,2]+discount*(P%*%current_value)[2]))&#xA;  new_policy = c(a_1,a_2)&#xA;  v &amp;lt;- rbind(v,t(current_value))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21291" LastEditorUserId="21291" LastEditDate="2017-09-29T06:38:47.000" LastActivityDate="2017-09-29T06:38:47.000" Title="Why do the policy iteration and value function iteration not match in this exercise" Tags="|dynamic-programming|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="27943" PostTypeId="1" AcceptedAnswerId="27944" CreationDate="2017-09-28T18:16:30.927" Score="3" ViewCount="314" Body="&lt;p&gt;Consider the BVP: find $u = u(x)$, for $x \in (0,1)$ that satisfies&#xA;\begin{align}&#xA;u'' + u u' = f, \\&#xA;u'(0) = g_n, u(1) = g_d.&#xA;\end{align}&#xA;To derive the weak form for this BVP, we multiply the first equation by a suitably smooth test function $\Phi = \Phi(x)$ and integrate both sides. This leads to &#xA;\begin{align}&#xA; - \int_\Omega \Phi ' u'  + \int_\Omega \Phi u u' = \int_\Omega \Phi f + \Phi(0) g_n -&#xA; \Phi(1) u'(1).&#xA;\end{align}&#xA;From this equation, we choose our test function space $\mathscr{T}$ to be a subspace of the Sobolev space $H^1(\Omega)$ that eliminate that $\Phi(1) u'(1)$ term, i.e.,&#xA;\begin{align}&#xA;\mathscr{T} = \{w: w \in H^1(\Omega), w(1) = 0 \}.&#xA;\end{align}&#xA;Furthermore, we choose our set of trial functions $\mathscr{S}$ to be a subset of $H^1(\Omega)$ that satisfy&#xA; the essential boundary condition, i.e., &#xA;\begin{align}&#xA;\mathscr{S} = \{v:v \in H^1(\Omega), v(1) = g_d \}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then consider the WP associated with the original BVP: find $u \in \mathscr{S}$ such that &#xA;\begin{align}&#xA;-\int_\Omega \Phi'u'+\int_\Omega \Phi u u' = \int_\Omega \Phi f + \Phi(0)g_n, \qquad \forall \Phi \in \mathscr{T}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Galerkin methods, we fix a positive integer $n$ and determine an n-dimensional subspace of $\mathscr{T}$ by specifying a basis, i.e., &#xA;\begin{align}&#xA;\mathscr{T}_n = \text{span}\{\phi_1, ..., \phi_n\} \subset\mathscr{T}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Bubnov-Galerkin, we determine an n-dimensional subset of $\mathscr{S}$ by taking $\mathscr{S}_n = \mathscr{T}_n \cup \{g(x)\}$, where $g(x) = g_d$ handles the essential boundary. Our ``finite element solution' can then be written $u_h(x) = \sum_{j = 1}^n u_j \phi_j(x) + g(x)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting $u_h$ into the weak equation should lead to a nonlinear system of equations ($\forall \Phi \in \mathscr{T}$ becomes $\forall \phi_i \in \mathscr{T}_n$): &#xA;\begin{align}&#xA;-\int \phi_i \left( \sum_{j = 1}^n u_j \phi_j + g\right)' + \int \phi_i \left( \sum_{j = 1}^n u_j \phi_j + g \right) \left( \sum_{j = 1}^n u_j \phi_j + g \right)' = \int \phi_i f + \phi_i (0).&#xA;\end{align}&#xA;The first term on the left-hand-side can be split up into &#xA;\begin{align}&#xA;- \sum_{j = 1}^n \int u_j \phi_i \phi_j' - \int \phi_i g',&#xA;\end{align}&#xA;which is just a bilinear form (matrix) and linear form (vector). The entire right hand side consists of just linear forms (vectors). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What do I do with the term in the middle?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I distribute or ``foil'' the middle term, I get &#xA;\begin{align}&#xA;\int \phi_i \left(\sum_{j = 1}^n u_j \phi_j \right)\left(\sum_{j = 1}^n u_j \phi_j' \right) + \int \phi_i \left(\sum_{j = 1}^n u_j \phi_j g' \right) + \int \phi_i \left(g \sum_{j = 1}^n u_j \phi_j' \right) + \int \phi_i g g'.&#xA;\end{align}&#xA;Here the second and third terms look like bilinear forms (matrices), the fourth term looks like a linear form (vector), but what about the first term, the &quot;trilinear form&quot;? It seems to me that this first term would end up like $u^T A u$ since the unknown coefficients $u_j$ appear twice, but I am not too sure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, my motivation for this problem is to understand similar looking nonlinearities are handled practically, because they seem to appear a lot (e.g., in Navier-Stokes).&lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2017-09-30T00:00:16.140" LastActivityDate="2017-09-30T00:00:16.140" Title="Bubnov-Galerkin method in 1D: how to handle convective-type nonlinearity?" Tags="|finite-element|pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27944" PostTypeId="2" ParentId="27943" CreationDate="2017-09-28T20:44:27.603" Score="3" Body="&lt;p&gt;You're on the right track -- the term &#xA;$$&#xA;  \int \phi_i u u'&#xA;$$&#xA;can be expanded to&#xA;$$&#xA;  \int \phi_i \left(\sum_j u_j \phi_j\right) \left(\sum_k u_k \phi_k'\right).&#xA;$$&#xA;(Note how I use a different &quot;silent&quot; index $j$ and $k$ in the two expansions.) This leads to a trilinear form&#xA;$$&#xA; B_{ijk} = \int \phi_i \phi_j \phi_k'&#xA;$$&#xA;and the term in the variational formulation would be $(BU)U$ with summation over the last index of the tensor and the only index of the vector of coefficients $U$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, all of this does not help you in practice. We do not know how to solve general quadratic problems in multiple variables other than by resorting to linearization and solving a sequence of linear problems. In other words, we cannot directly solve the problem you have, but only a sequence of problems that are linear in a solution variable. These linear problems must all have bilinear forms of the traditional form.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-09-28T20:44:27.603" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27945" PostTypeId="1" AcceptedAnswerId="27948" CreationDate="2017-09-28T22:31:28.020" Score="0" ViewCount="1116" Body="&lt;p&gt;I am trying the implement the following odeint solver example but my differential equations are different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/headmyshoulder/odeint-v2/blob/master/examples/stiff_system.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/headmyshoulder/odeint-v2/blob/master/examples/stiff_system.cpp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system I have is stiff so the rosenbrock method is a good fit for that. However, the solution process is really slow, often taking 2 million steps to reach t = 2.5s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to tweak the adaptive stepping settings and see what steps are being taken by the solver? I haven't been able to find a good documentation for this, the only one I found is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://headmyshoulder.github.io/odeint-v2/doc/boost_numeric_odeint/tutorial/stiff_systems.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://headmyshoulder.github.io/odeint-v2/doc/boost_numeric_odeint/tutorial/stiff_systems.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is a way to speed it up without tweaking the stepping, that would work as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="25530" LastActivityDate="2017-09-29T14:16:25.420" Title="Speeding up ODEINT stiff example" Tags="|ode|c++|" AnswerCount="1" CommentCount="0" ClosedDate="2017-09-29T15:58:22.883" ContentLicense="CC BY-SA 3.0" />
  <row Id="27948" PostTypeId="2" ParentId="27945" CreationDate="2017-09-29T10:09:36.110" Score="3" Body="&lt;p&gt;From &lt;a href=&quot;http://www.boost.org/doc/libs/1_65_1/libs/numeric/odeint/doc/html/boost_numeric_odeint/odeint_in_detail/steppers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation&lt;/a&gt; and &lt;a href=&quot;https://github.com/headmyshoulder/odeint-v2/blob/master/include/boost/numeric/odeint/stepper/rosenbrock4_controller.hpp#L148&quot; rel=&quot;nofollow noreferrer&quot;&gt;this spot in the code&lt;/a&gt; show that it is doing Gustafsson acceleration. That's usually a good technique and is advocated by Hairer. I have found in my own tests that actually PI-controllers work surprisingly well with high order Rosenbrock methods, but YMMV. It's probably problem and tableau dependent. But anyways, that's the spot in the code you'd modify. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I do want to address one other thing.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The system I have is stiff so the rosenbrock method is a good fit for that.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not necessarily, I wouldn't not blindly say this at all. The Rosenbrock methods tend to be good for stiff equations when you are trying to solve for error less than $\approx10^{-6}$ and your system of equations is not sufficiently large (I'll point to &lt;a href=&quot;https://www.google.com/search?q=hairer+solving+ordinary+differential+equations+ii&amp;amp;oq=hairer+solving+ord&amp;amp;aqs=chrome.1.69i57j0l3.4835j0j7&amp;amp;sourceid=chrome&amp;amp;ie=UTF-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer II&lt;/a&gt;, the &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffEqBenchmarks&lt;/a&gt;, and lots of other currently unpublished benchmarks which all pretty much say the same thing). But then again, multistep methods can still &quot;sneak a win&quot; if the system is smooth enough every one in awhile, so it's not a hard-fast rule. If you really want to be sure what's good for your problem, I would recommend solving a small part of your real equation using some suite that has easy access to Radau, both high and low order Rosenbrock, a BDF, and optionally high and low order (E)SDIRK methods since these tend to do the best in various regimes. You should also generate a reference solution since comparing timings without comparing error is never a good idea (many times the faster one can just have a higher error), which is why work-precision diagrams are a good tool if you want to do this comparison easily. &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffODE/ROBER.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;This example in the DiffEqBenchmarks&lt;/a&gt; shows an easy way to run such a test, though you can also setup something similar with MATLAB or with wrappers to Hairer's solver suite for example to see similar results (but with some missing methods in each case).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="20526" LastEditDate="2017-09-29T14:16:25.420" LastActivityDate="2017-09-29T14:16:25.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27950" PostTypeId="1" AcceptedAnswerId="27951" CreationDate="2017-09-29T12:59:17.637" Score="13" ViewCount="1608" Body="&lt;p&gt;According to the &lt;a href=&quot;https://scicomp.stackexchange.com/a/19290/18289&quot;&gt;answer here&lt;/a&gt;, large condition number (for linear system solving) decreases the guaranteed number of correct digits in the floating point solution. Higher order differentiation matrices in pseudospectral methods are typically very ill-conditioned. Why is it then that they are still very accurate methods? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the low precision coming from the ill-conditioned matrices is only a &lt;em&gt;guaranteed&lt;/em&gt; value, but still it makes me wonder why ill-conditioned matrices are accurately solved by direct methods in practice - e.g., the &lt;code&gt;LCOL&lt;/code&gt; columns of Table 3.1 on page 11 of &lt;a href=&quot;https://arxiv.org/pdf/1305.2041.pdf&quot; rel=&quot;noreferrer&quot;&gt;Wang et al., &lt;em&gt;A WELL-CONDITIONED COLLOCATION METHOD USING A PSEUDOSPECTRAL INTEGRATION MATRIX&lt;/em&gt;, SIAM J. Sci. Comput., 36(3)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18289" LastEditorUserId="17294" LastEditDate="2017-09-29T20:03:45.747" LastActivityDate="2021-04-13T21:40:58.850" Title="Why can ill-conditioned linear systems be solved precisely?" Tags="|spectral-method|precision|condition-number|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27951" PostTypeId="2" ParentId="27950" CreationDate="2017-09-29T19:28:26.260" Score="8" Body="&lt;p&gt;Added after my initial answer:&lt;/p&gt;&#xA;&lt;p&gt;It appears to me now that the author of the referenced paper is giving condition numbers (apparently 2-norm condition numbers but possibly infinity-norm condition numbers) in the table while giving maximum absolute errors rather than norm relative errors or maximum elementwise relative errors (these are all different measures.)  Note that the maximum elementwise relative error is not the same thing as the infinity-norm relative error.  Furthermore, the errors in the table are relative to the exact solution to the original differential equation boundary value problem rather than the discretized linear system of equations. Thus the information provided in the paper really isn't appropriate for use with the error bound based on the condition number.&lt;/p&gt;&#xA;&lt;p&gt;However, In my replication of the computations, I do see situations where the relative infinity norm error (or two-norm relative error) is substantially smaller than the bound set by the infinity-norm condition number (respectively 2-norm condition number.)  Sometimes you just get lucky.&lt;/p&gt;&#xA;&lt;p&gt;I used the DMSUITE MATLAB package and solve the example problem from this paper using the pseudospectral method with Chebyshev polynomials. My condition numbers and maximum absolute errors were similar to those reported in the paper.&lt;/p&gt;&#xA;&lt;p&gt;I also saw norm relative errors that were somewhat better than one might expect based on the condition number.  For example, on the example problem with &lt;span class=&quot;math-container&quot;&gt;$\epsilon=0.01$&lt;/span&gt;, using &lt;span class=&quot;math-container&quot;&gt;$N=1024$&lt;/span&gt;, I get&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cond(A,2)=7.9e+8&#xA;&#xA;cond(A,inf)=7.8e+8&#xA;&#xA;norm(u-uexact,2)/norm(uexact,2)=3.1e-12&#xA;&#xA;norm(u-uexact,inf)/norm(uexact,inf)=2.7e-12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It would appear that the solution is good to about 11-12 digits, while the condition number is on the order of 1e8.&lt;/p&gt;&#xA;&lt;p&gt;However, The situation with element-wise errors is more interesting.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;max(abs(u-uexact))=2.7e-12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That still looks good.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;max(abs((u-uexact)./uexact)=6.1e+9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Wow- there's a very large relative error in at least one component of the solution.&lt;/p&gt;&#xA;&lt;p&gt;What happened?  The exact solution of this equation has components that are tiny (e.g. 1.9e-22) while the approximate solution bottoms out at a much larger value of 9e-14.  This is hidden by the norm relative error measurement (whether it be the 2-norm or infinity-norm) and only becomes visible when you look at the elementwise relative errors and take the maximum.&lt;/p&gt;&#xA;&lt;p&gt;My original answer below explains why you can get a norm relative error in the solution that is less than the bound given by the condition number.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;As you've noted in the question, the condition number, &lt;span class=&quot;math-container&quot;&gt;$\kappa(A)$&lt;/span&gt;, of a non-singular matrix gives a &lt;em&gt;worst-case&lt;/em&gt; relative error bound for the solution to a perturbed system of equations.  That&#xA;is, if we solve &lt;span class=&quot;math-container&quot;&gt;$A(x+\Delta x)=b+\Delta b$&lt;/span&gt; exactly and solve &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; exactly, then&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\| \Delta x \|}{\| x \|} \leq \kappa(A) \frac{\| \Delta b \|}{\| b \|}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Condition numbers can be computed with respect to a variety of norms, but the two-norm condition number is often used, and that's the condition number used in the paper that you refer to.&lt;/p&gt;&#xA;&lt;p&gt;The worst case error occurs when &lt;span class=&quot;math-container&quot;&gt;$\Delta b$&lt;/span&gt; is a left singular vector of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; corresponding to the smallest singular value of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.  The best case occurs when &lt;span class=&quot;math-container&quot;&gt;$\Delta b$&lt;/span&gt; is a left singular vector of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; corresponding to the largest singular value of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.  When &lt;span class=&quot;math-container&quot;&gt;$\Delta b$&lt;/span&gt; is random, then you have to look at the projections of &lt;span class=&quot;math-container&quot;&gt;$\Delta b$&lt;/span&gt; onto all of the left singular vectors of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and the corresponding singular values.  Depending on the spectrum of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, things may go very badly or very well.&lt;/p&gt;&#xA;&lt;p&gt;Consider two matrices &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, both with 2-norm condition number &lt;span class=&quot;math-container&quot;&gt;$1.0 \times 10^{10}$&lt;/span&gt;.  The first matrix has the singular values &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1 \times 10^{-10}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\ldots$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1 \times 10^{-10}$&lt;/span&gt;.  The second matrix has singular values &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\ldots$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$1 \times 10^{-10}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the first case, a random perturbation is unlikely to be in the direction of the first left singular vector, and more likely to be close to one of the singular vectors with singular value &lt;span class=&quot;math-container&quot;&gt;$1 \times 10^{-10}$&lt;/span&gt;.  Thus the relative change in the solution is likely to be very large.  In the second case, almost any perturbation will be close in direction to a singular vector with singular value &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, and the relative change in the solution will be small.&lt;/p&gt;&#xA;&lt;p&gt;P.S. (added later after I got back from yoga class...)&lt;/p&gt;&#xA;&lt;p&gt;The formula for the solution to &lt;span class=&quot;math-container&quot;&gt;$A\Delta x = \Delta b$&lt;/span&gt; is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Delta x = V \Sigma^{-1} U^{T} \Delta b=\sum_{i=1}^{n} \frac{U_{i}^{T}\Delta b}{\sigma_{i}} V_{i}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;By the Pythagorean theorem,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\| \Delta x \|_{2}^{2}= \sum_{i=1}^{n} \left( \frac{U_{i}^{T}\Delta b}{\sigma_{i}} \right)^{2} $&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If we keep &lt;span class=&quot;math-container&quot;&gt;$\| \Delta b \|_{2}=1$&lt;/span&gt;, then this sum is maximized when &lt;span class=&quot;math-container&quot;&gt;$\Delta b=U_{n}$&lt;/span&gt; and minimized when &lt;span class=&quot;math-container&quot;&gt;$\Delta b=U_{1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the situation considered here, &lt;span class=&quot;math-container&quot;&gt;$\Delta b$&lt;/span&gt; is the result of random round-off errors, so the &lt;span class=&quot;math-container&quot;&gt;$U_{i}^{T}\Delta b$&lt;/span&gt; values should all be of roughly the same magnitude.  The terms with smaller values of &lt;span class=&quot;math-container&quot;&gt;$\sigma_{i}$&lt;/span&gt; will contribute a lot to the error, while terms with larger values of &lt;span class=&quot;math-container&quot;&gt;$\sigma_{i}$&lt;/span&gt; won't contribute much. Depending on the spectrum, this could easily be much smaller than the worst case bound.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="27325" LastEditDate="2021-04-13T21:40:58.850" LastActivityDate="2021-04-13T21:40:58.850" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="27952" PostTypeId="2" ParentId="27950" CreationDate="2017-09-29T19:31:35.483" Score="1" Body="&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; They reported &lt;em&gt;a&lt;/em&gt; condition number, not necessarily the &lt;em&gt;right&lt;/em&gt; condition number for the matrix, because there is a difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is specific to the matrix and the right hand side vector. If you look at &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-getrs&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation for &lt;code&gt;*getrs&lt;/code&gt;&lt;/a&gt;, it says the forward error bound is&#xA;$$ \frac{\|x-x_0\|_\infty}{\|x\|_\infty} \lesssim \mathrm{cond}(A,x)u \leq \mathrm{cond}(A)u. $$&#xA;Here $\mathrm{cond}(A,x)$ is not quite the usual condition number $\kappa_\infty(A)$, but rather &#xA;$$ \mathrm{cond}(A,x) = \frac{\||A^{-1}||A||x|\|_\infty}{\|x\|_\infty},\\&#xA;\mathrm{cond}(A) = \||A^{-1}||A|\|. $$&#xA;(Here inside the norm these are component-wise absolute values.) See, for example, &lt;a href=&quot;https://doi.org/10.1093/imanum/17.4.495&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Iterative refinement for linear systems and LAPACK&lt;/em&gt; by Higham&lt;/a&gt;, or Higham's &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; (7.2). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your example, I took a pseudospectral differential operator for a similar problem with $n=128$, and there is in fact a big difference between $\||A^{-1}||A|\|$ and $\kappa_\infty(A)$, I computed $7\times 10^3$ and $2.6\times 10^7$, which is enough to explain the observation that this happens for all right hand sides, because the orders of magnitudes roughly match what is seen in Table 3.1 (3-4 orders better errors). This doesn't work when I try the same for just a random ill-conditioned matrix, so it has to be a property of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An explicit example for which the two condition numbers don't match, which I took from Higham (7.17, p.124), due to Kahan is&#xA;$$ \begin{pmatrix}2&amp;amp;-1&amp;amp;1\\-1&amp;amp;\epsilon&amp;amp;\epsilon\\1&amp;amp;\epsilon&amp;amp;\epsilon\end{pmatrix}, \qquad \begin{pmatrix}2+2\epsilon\\-\epsilon\\\epsilon\end{pmatrix}. $$&#xA;Another example I found is just the plain Vandermonde matrix on &lt;code&gt;[1:10]&lt;/code&gt; with random $b$. I went through &lt;code&gt;MatrixDepot.jl&lt;/code&gt; and some other ill-conditioned matrices also produce this type of result, like &lt;code&gt;triw&lt;/code&gt; and &lt;code&gt;moler&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, what's going on is that when you analyze the stability of solving linear systems with respect to perturbations, you first have to specify &lt;em&gt;which&lt;/em&gt; perturbations you are considering. When solving linear systems with LAPACK, this error bound considers component-wise perturbations in $A$, but no perturbation in $b$. So this is different from the usual $\kappa(A) = \|A^{-1}\|\|A\|$, which considers normwise perturbations in both $A$ and $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider (as a counterexample) also what would happen if you &lt;em&gt;don't&lt;/em&gt; make the distinction. We know that using iterative refinement with double precision (see link above) we can get the best possible forward relative error of $O(u)$ for those matrices with $\kappa(A)\ll 1/u$. So if we consider the idea that linear systems can't be solved to accuracy better than $\kappa(A)u$, how would refining solutions possibly work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;P.S.&lt;/em&gt; It matters that &lt;code&gt;?getrs&lt;/code&gt; says the computed solution is the true solution of &lt;code&gt;(A + E)x = b&lt;/code&gt; with a perturbation $E$ in $A$, but no perturbation in $b$. Things would be different if perturbations were allowed in $b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit&lt;/em&gt; To show this working more directly, in code, that this is not a fluke or a matter of luck, but rather the (unusual) consequence of two condition numbers being very different for some specific matrices, i.e.,&#xA;$$ \mathrm{cond}(A,x) \approx \mathrm{cond}(A) \ll \kappa(A). $$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function main2(m=128)&#xA;    A = matrixdepot(&quot;chebspec&quot;, m)^2&#xA;    A[1,:] = A[end,:] = 0&#xA;    A[1,1] = A[end,end] = 1&#xA;    best, worst = Inf, -Inf&#xA;    for k=1:2^5&#xA;        b = randn(m)&#xA;        x = A \ b&#xA;        x_exact = Float64.(big.(A) \ big.(b))&#xA;        err = norm(x - x_exact, Inf) / norm(x_exact, Inf)&#xA;        best, worst = min(best, err), max(worst, err)&#xA;    end&#xA;    @printf &quot;Best relative error:       %.3e\n&quot; best&#xA;    @printf &quot;Worst relative error:      %.3e\n&quot; worst&#xA;    @printf &quot;Predicted error κ(A)*ε:    %.3e\n&quot; cond(A, Inf)*eps()&#xA;    @printf &quot;Predicted error cond(A)*ε: %.3e\n&quot; norm(abs.(inv(A))*abs.(A), Inf)*eps()&#xA;end&#xA;&#xA;julia&amp;gt; main2()&#xA;Best relative error:       2.156e-14&#xA;Worst relative error:      2.414e-12&#xA;Predicted error κ(A)*ε:    8.780e-09&#xA;Predicted error cond(A)*ε: 2.482e-12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit 2&lt;/em&gt; Here is another example of the same phenomenon where the different conditions numbers unexpectedly differ by a lot. This time,&#xA;$$ \mathrm{cond}(A, x) \ll \mathrm{cond}(A) \approx \kappa(A). $$&#xA;Here $A$ is the 10×10 Vandermonde matrix on $1:10$, and when $x$ is chosen randomly, $\mathrm{cond}(A,x)$ is noticably smaller than $\kappa(A)$, and the worst case $x$ is given by $x_i = i^a$ for some $a$.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function main4(m=10)&#xA;    A = matrixdepot(&quot;vand&quot;, m)&#xA;    lu = lufact(A)&#xA;    lu_big = lufact(big.(A))&#xA;    AA = abs.(inv(A))*abs.(A)&#xA;    for k=1:12&#xA;        # b = randn(m) # good case&#xA;        b = (1:m).^(k-1) # worst case&#xA;        x, x_exact = lu \ b, lu_big \ big.(b)&#xA;        err = norm(x - x_exact, Inf) / norm(x_exact, Inf)&#xA;        predicted = norm(AA*abs.(x), Inf)/norm(x, Inf)*eps()&#xA;        @printf &quot;relative error[%2d]    = %.3e (predicted cond(A,x)*ε = %.3e)\n&quot; k err predicted&#xA;    end&#xA;    @printf &quot;predicted κ(A)*ε      = %.3e\n&quot; cond(A)*eps()&#xA;    @printf &quot;predicted cond(A)*ε   = %.3e\n&quot; norm(AA, Inf)*eps()&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Average case (almost 9 orders of magnitude better error):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; T.main4()&#xA;relative error[1]     = 6.690e-11 (predicted cond(A,x)*ε = 2.213e-10)&#xA;relative error[2]     = 6.202e-11 (predicted cond(A,x)*ε = 2.081e-10)&#xA;relative error[3]     = 2.975e-11 (predicted cond(A,x)*ε = 1.113e-10)&#xA;relative error[4]     = 1.245e-11 (predicted cond(A,x)*ε = 6.126e-11)&#xA;relative error[5]     = 4.820e-12 (predicted cond(A,x)*ε = 3.489e-11)&#xA;relative error[6]     = 1.537e-12 (predicted cond(A,x)*ε = 1.729e-11)&#xA;relative error[7]     = 4.885e-13 (predicted cond(A,x)*ε = 8.696e-12)&#xA;relative error[8]     = 1.565e-13 (predicted cond(A,x)*ε = 4.446e-12)&#xA;predicted κ(A)*ε      = 4.677e-04&#xA;predicted cond(A)*ε   = 1.483e-05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Worst case ($a=1,\ldots,12$):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; T.main4()&#xA;relative error[ 1]    = 0.000e+00 (predicted cond(A,x)*ε = 6.608e-13)&#xA;relative error[ 2]    = 1.265e-13 (predicted cond(A,x)*ε = 3.382e-12)&#xA;relative error[ 3]    = 5.647e-13 (predicted cond(A,x)*ε = 1.887e-11)&#xA;relative error[ 4]    = 8.895e-74 (predicted cond(A,x)*ε = 1.127e-10)&#xA;relative error[ 5]    = 4.199e-10 (predicted cond(A,x)*ε = 7.111e-10)&#xA;relative error[ 6]    = 7.815e-10 (predicted cond(A,x)*ε = 4.703e-09)&#xA;relative error[ 7]    = 8.358e-09 (predicted cond(A,x)*ε = 3.239e-08)&#xA;relative error[ 8]    = 1.174e-07 (predicted cond(A,x)*ε = 2.310e-07)&#xA;relative error[ 9]    = 3.083e-06 (predicted cond(A,x)*ε = 1.700e-06)&#xA;relative error[10]    = 1.287e-05 (predicted cond(A,x)*ε = 1.286e-05)&#xA;relative error[11]    = 3.760e-10 (predicted cond(A,x)*ε = 1.580e-09)&#xA;relative error[12]    = 3.903e-10 (predicted cond(A,x)*ε = 1.406e-09)&#xA;predicted κ(A)*ε      = 4.677e-04&#xA;predicted cond(A)*ε   = 1.483e-05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit 3&lt;/em&gt; Another example is the Forsythe matrix, which is a perturbed Jordan block of any size of the form&#xA;$$A = \begin{pmatrix} 0&amp;amp;1&amp;amp;0&amp;amp;0\\ 0&amp;amp;0&amp;amp;1&amp;amp;0\\ 0&amp;amp;0&amp;amp;0&amp;amp;1 \\ \epsilon&amp;amp;0&amp;amp;0&amp;amp;0 \end{pmatrix}. $$&#xA;This has $\|A\|=1$, $\|A^{-1}\|=\epsilon^{-1}$, so $\kappa_\infty(A) = \epsilon^{-1}$, but $|A^{-1}| = A^{-1} = |A|^{-1}$, so $\mathrm{cond}(A) = 1$. And as can be verified by hand, solving systems of linear equations like $Ax=b$ with pivoting is extremely accurate, despite the potentially unbounded $\kappa_\infty(A)$. So this matrix too will yield unexpectedly precise solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit 4&lt;/em&gt; &lt;a href=&quot;http://math.nist.gov/MatrixMarket/data/MMDELI/kahan/kahan.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kahan matrices&lt;/a&gt; are also like this, with $\mathrm{cond}(A)\ll \kappa(A)$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = matrixdepot(&quot;kahan&quot;, 48)&#xA;κ, c = cond(A, Inf), norm(abs.(inv(A))*abs.(A), Inf)&#xA;@printf &quot;κ=%.3e c=%.3e ratio=%g\n&quot; κ c (c/κ)&#xA;&#xA;κ=8.504e+08 c=4.099e+06 ratio=0.00482027&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-10-03T00:00:23.523" LastActivityDate="2017-10-03T00:00:23.523" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27953" PostTypeId="1" AcceptedAnswerId="27960" CreationDate="2017-09-29T23:04:12.503" Score="1" ViewCount="224" Body="&lt;p&gt;I'm trying to solve/implement a system of linear equations of the following form/structure:&#xA;$$ Ax=b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A =&#xA;\begin{bmatrix}&#xA;* &amp;amp; * &amp;amp; 0 &amp;amp; * &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \cr&#xA;* &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 \cr&#xA;0 &amp;amp; * &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 0 \cr&#xA;* &amp;amp; 0 &amp;amp; * &amp;amp; * &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 \cr&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \cr&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \cr&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \cr&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0&#xA;\end{bmatrix} ,\quad b =&#xA;\begin{bmatrix}&#xA;0\cr 0\cr 0\cr 0\cr 1\cr 1\cr 1\cr 1&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where *'s indicate non-negative numbers (some or all of these could be zeros as well, but very unlikely).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any suggestions on what algorithms I should look into to solve this specific problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing to point out is that I'm doing this to solve for KKT (Karush–Kuhn–Tucker) conditions with 4 inequality constraints, so I have to re-solve the above equation multiple times in order to get an optimal solution by removing one or more columns and rows 5 through 8.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently solving this in MATLAB using &lt;code&gt;mldivide&lt;/code&gt;, but I'm not sure what specific algorithm they are using to solve for this.&lt;/p&gt;&#xA;" OwnerUserId="25540" LastEditorUserId="9667" LastEditDate="2017-09-29T23:18:23.010" LastActivityDate="2017-09-30T13:18:49.860" Title="Methods for solving $Ax=b$, small and sparse A" Tags="|linear-algebra|algorithms|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27955" PostTypeId="2" ParentId="2369" CreationDate="2017-09-30T04:17:36.357" Score="1" Body="&lt;p&gt;Walter Landry developed a code for three-dimensional elastostatic deformation using adaptive multigrid. You can find the code at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bitbucket.org/wlandry/gamra&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bitbucket.org/wlandry/gamra&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can include the effect of the thermal forcing with isotropic eigenstrain and equivalent body forces. Once these are in place, the solver would work just fine. &lt;/p&gt;&#xA;" OwnerUserId="25542" LastActivityDate="2017-09-30T04:17:36.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27956" PostTypeId="1" CreationDate="2017-09-30T04:43:19.027" Score="2" ViewCount="346" Body="&lt;p&gt;I am interested in understanding how to perform stability analysis for coupled (to keep things do-able, lets say linear) PDE's&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a single PDE, i understand the logic behind the VN analysis: mode expansion and look at the amplification factor of any given mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However for coupled PDE's it seems like you won't be able to cancel things as nicely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, basing off the notes here&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Von_Neumann_stability_analysis&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;specifically starting at Eqn (2): If this represents the error for one function it will depend on the other via the coupling. It seems that the $e^{at}$ term will not cancel nicely to get Eqn (6). Presumably i can still restrict the analysis to one mode at a time assuming both PDEs are linear?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, rather than the $e^{at}$ terms canceling as they do in that simple example, in the coupled case i would be left terms containing the relative size i.e. $e^{(a-b)t}$ where $a$ and $b$ are the exponents of the mode expansion amplitudes of the errors for the two PDE functions respectively (whoa, a mouthful...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like the solution is to cast as a linear matrix problem but I get a bit stuck here too. Certainly one can write the time discretization in terms of an amplification factor/matrix $A$ acting $n$ times on some initial vector to give the new state, but how does one then look at the spatial discretization (contained inside the amp matrix $A$ in this context? Or do we just look for eigenvalues of that amplification matrix and then do finite-difference afterwards?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any and all help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="25543" LastActivityDate="2017-09-30T04:43:19.027" Title="Von Neumann analysis for coupled PDE's" Tags="|pde|stability|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27959" PostTypeId="2" ParentId="27938" CreationDate="2017-09-30T10:14:40.690" Score="1" Body="&lt;p&gt;Not sure if I understood correctly, but I think that if your parameters &lt;code&gt;µ&lt;/code&gt; and $\Delta t$ are constant during all the simulations and for all nodes, you can generate a list of possible $p_j$ values in the beginning of the simulations. Because you have a limited possiblity of $b_j$ values (you have maximum 8 neighbors in a 2D grid if you look at first-neighborhood, so you can do an array of 9 possibles $p_j$ values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you generate a random number with uniform law each time and compare if it's smaller than its corresponding $p_j$ value from the list. &lt;/p&gt;&#xA;" OwnerUserId="25545" LastActivityDate="2017-09-30T10:14:40.690" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27960" PostTypeId="2" ParentId="27953" CreationDate="2017-09-30T13:18:49.860" Score="8" Body="&lt;p&gt;By direct substitution, trivially.&#xA;$$&#xA;\begin{bmatrix}&#xA;0\\f&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;M &amp;amp; -I\\&#xA;I &amp;amp; 0&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;y\\z&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;My-z\\&#xA;y&#xA;\end{bmatrix}&#xA;$$&#xA;gives $y=f$, $z=My$.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-09-30T13:18:49.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27962" PostTypeId="2" ParentId="27907" CreationDate="2017-10-02T01:03:55.860" Score="1" Body="&lt;p&gt;Using a QR algorithm is the better way. I think it is best to use an algorithm best-suited for the task at hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, even if you were trying to compute the roots of a polynomial, without intending to use them as the eigenvalues of a matrix, it is often recommended to create the Companion Matrix for that polynomial, and solve for the eigenvalues of that matrix. (i.e., do the opposite process to what you are considering.) The process is extremely robust, but not very computationally efficient. An algorithm specific to the task of finding polynomial roots (e.g., Jenkins-Traub, Method of Laguerre, etc.) would be more efficient. And even if you use one of these methods, there may still be some cases for which forming the Companion Matrix and computing its eigenvalues &lt;em&gt;still&lt;/em&gt; gives better results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, as Kirill indicated, there is no way to take advantage of the closed form solutions for a 3rd or 4th degree polynomial. I looked into this some years ago before writing a translation of the Jenkins-Traub algorithm. For numerical results, it is still best to write an algorithm from the ground up as a discrete solver and ignore the closed form solutions. &lt;/p&gt;&#xA;" OwnerUserId="13202" LastActivityDate="2017-10-02T01:03:55.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27963" PostTypeId="2" ParentId="26665" CreationDate="2017-10-02T06:30:54.717" Score="1" Body="&lt;p&gt;The comments detail a perfectly valid way to solve this by defining the integral as another part of the ODE and adding a boundary condition on that. I'll discuss another way to approach this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bvp4c&lt;/code&gt; is a two-point boundary-value problem solver. Instead of a two-point boundary value problem solver, you need the ability to specify a condition that uses multiple points or, even better, uses a continuous extension of the current solution. You can do something like that using &lt;a href=&quot;http://diffeq.sciml.ai/latest/tutorials/bvp_example.html#Boundary-Condition-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl because it lets you define residuals with continuous extensions of the solution&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;function bc3(residual, sol)&#xA;    residual[1] = quadgk((t)-&amp;gt;sol(t;idxs=1)^2,0,pi)&#xA;    residual[2:8] .= 0 #there's no other boundary conditions mentioned?&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works because &lt;code&gt;sol(t)&lt;/code&gt; is the solution at time &lt;code&gt;t&lt;/code&gt; using an order-matching interpolation.&lt;/p&gt;&#xA;&lt;p&gt;Okay, so it's possible to handle this directly, but what's the difference? This is the part that's more interesting. Two different cases. When using a shooting-type boundary value problem solver, continuous extensions come naturally and one is doing rootfinding of the initial condition, treating the BVP as an IVP with an unknown initial condition. Thus there is no downside to using this kind of continuous extension condition (provided the BVP solver allows it). But shooting methods of course have a downside that they can't be used for problems which are sensitive to the initial condition, or start with a singularity.&lt;/p&gt;&#xA;&lt;p&gt;That is not true for collocation or MIRK (Implicit RK) type methods. In these cases, one grids space &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and relaxes the solution at every point in space &lt;span class=&quot;math-container&quot;&gt;$y(x_i)$&lt;/span&gt; via a rootfinder. In this case, the &amp;quot;Jacobian&amp;quot; is the dependency of each point on every other point. Since for Runge-Kutta or collocation methods you only use values from just before to compute the new, this typically has a banded structure. However, that's assuming that the boundaries only depend on &lt;span class=&quot;math-container&quot;&gt;$y(x_1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y(x_{end})$&lt;/span&gt;. If you impose a boundary condition which depends on all points in space, like this quadrature form for the integral, then the Jacobian of the full system has a full column which destroys the banded structure.&lt;/p&gt;&#xA;&lt;p&gt;Since the Jacobian is of size &lt;code&gt;M*N*S&lt;/code&gt; where &lt;code&gt;M&lt;/code&gt; is the number of spatial points, &lt;code&gt;N&lt;/code&gt; is the number of ODEs in the system, and &lt;code&gt;S&lt;/code&gt; is the number of stages in the ODE, this can be very large even for simple problems. But as noted before, the two-point BVP problem makes the dependencies such that this is an &lt;code&gt;S&lt;/code&gt;-banded matrix, making it easy to solve. &lt;code&gt;bvp4c&lt;/code&gt; is one of many such solvers which thus assumes the two-point BVP form because it allows solving this system to be greatly simplified.&lt;/p&gt;&#xA;&lt;p&gt;If it's not a two-point BVP, this  Jacobian is a sparse matrix. Either it's stored actually as a sparse matrix or if the problem is small enough one uses a dense matrix for it, but the inversion algorithm cannot use a banded form and thus resorts to a much slower &lt;code&gt;\&lt;/code&gt;. Therefore a solver which generalizes like this should only be used when necessary.&lt;/p&gt;&#xA;&lt;p&gt;Let me conclude by summarizing this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you can do a transformation to not have a more complicated boundary condition, that's probably your best bet if it's not difficult to do. Integrals can be added as extra parts in the system of ODEs since an ODE solver is actually just a quadrature method.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the problem is more difficult and such a transformation is not possible, then a shooting-type method is a good bet and doesn't have an overhead for having these kinds of conditions (you just need to find a BVP solver which lets you do it).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Shooting methods require that the problem isn't sensitive to the initial condition or have an initial singularity. If that's not the case, you need to use collocation or MIRK-type methods. However, most of these methods, like &lt;code&gt;bvp4c&lt;/code&gt;, do not allow this form because they can greatly optimize by assuming the problem has a two-point condition. The few solvers which can handle this case should only be used as a last resort.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Hopefully that explanations helps clear your mind that the workaround from the comments isn't just a hack to get it done, but probably the better way to do it.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:06:04.517" LastActivityDate="2020-07-19T18:06:04.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27964" PostTypeId="2" ParentId="26182" CreationDate="2017-10-02T06:52:22.520" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Can we expect radau5 to cope with discontinuities, or should we integrate the two trajectories separately?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Differential equation methods cannot easily handle discontinuities like this. If you step over a discontinuity, you cannot prove that you will not have order loss. In fact, you normally will have loss of accuracy. Because of this, you want to make sure you hit each discontinuity &quot;exactly&quot;, i.e. step right on it (or integrate the two pieces separately). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A case study for this is delay differential equation solvers where an initial discontinuity gets propagated, and the goal of a delay equation solver is to use an ODE method with an interpolation in a specific way that you track and hit the discontinuities. From this it's very well quantified how error-costly it is to not track discontinuities.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="713" LastEditDate="2017-10-03T00:03:45.880" LastActivityDate="2017-10-03T00:03:45.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27965" PostTypeId="1" CreationDate="2017-09-30T15:01:27.127" Score="5" ViewCount="2376" Body="&lt;p&gt;We start off with a unit vector $\mathbf{v}$ randomly oriented in 3D space and we want to generate another unit vector $\mathbf{w}$ so that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{w}\cdot \mathbf{v} = \cos \beta $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\beta$ is a given fixed angle. The new vector $\mathbf{w}$ must be randomly oriented, so that on average the cross product&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \langle \mathbf{w}\times \mathbf{v} \rangle = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normalization is $|\mathbf{v}|^2 = |\mathbf{w}|^2 = 1 $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you generate a particular vector $\mathbf{w}$ which satisfies the above conditions, using only 1 random number, and requiring as little computations as possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a &quot;dirty&quot; solution using polar coordinates, where I use an orthonormal set of vectors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{v} = \begin{pmatrix}&#xA;\sin \theta \cos \phi\\&#xA;\sin \theta \sin \phi\\&#xA;\cos \theta&#xA;\end{pmatrix} \quad &#xA;\mathbf{v}^* = \begin{pmatrix}&#xA;\cos \theta \cos \phi\\&#xA;\cos \theta \sin \phi\\&#xA;-\sin \theta&#xA;\end{pmatrix} \quad &#xA;\mathbf{v}^{\dagger} = \begin{pmatrix}&#xA;-\sin \phi\\&#xA;\cos \phi\\&#xA;0&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or, in Cartesian components&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{v} = \begin{pmatrix}&#xA;x\\&#xA;y\\&#xA;z&#xA;\end{pmatrix} \quad &#xA;\mathbf{v}^* = \frac{1}{\sqrt{1-z^2}}\begin{pmatrix}&#xA;xz\\&#xA;yz\\&#xA;z^2-1&#xA;\end{pmatrix} \quad &#xA;\mathbf{v}^{\dagger} = \frac{1}{\sqrt{1-z^2}} \begin{pmatrix}&#xA;-y\\&#xA;x\\&#xA;0&#xA;\end{pmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The random orientation is then selected by a linear superposition&#xA;\begin{equation}&#xA;\mathbf{a} = \mathbf{v}^* \cos(2\pi \alpha) + \mathbf{v}^{\dagger} \sin(2\pi \alpha)&#xA;\end{equation}&#xA;where $\alpha \in (0,1)$ is a random number drawn from a uniform distribution. The rotation is given simply by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{w} = \mathbf{v}\cos \beta + \mathbf{a}\sin \beta $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically this seems perfect, except in the situation where $z=1$ or close to it. Also, if I just keep rotating my vector, the normalization drifts away from 1 due to roundoff errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have some ideas on a more general formula which would be numerically robust and maintain normalization more rigorously, preferably with not too many computations?&lt;/p&gt;&#xA;" OwnerUserId="25554" OwnerDisplayName="Airidas Korolkovas" LastEditorUserId="25554" LastEditDate="2017-10-03T13:30:07.030" LastActivityDate="2017-10-03T13:30:07.030" Title="Rotate a vector by a randomly oriented angle" Tags="|geometry|random-sampling|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27966" PostTypeId="2" ParentId="27965" CreationDate="2017-09-30T15:56:44.947" Score="0" Body="&lt;p&gt;You have $$\mathbf{v}\cdot\mathbf{w} = v_1w_1 + v_2w_2 + v_3w_3 = \cos\beta$$ and $$ v_1^2 + v_2^2 + v_3^2 = 1\,.$$ Randomly generate one component of $\mathbf{v}$ and use the two equations above to solve for the other components. Does this work?&lt;/p&gt;&#xA;" OwnerDisplayName="occamsrazor" LastActivityDate="2017-09-30T15:56:44.947" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="27967" PostTypeId="2" ParentId="27965" CreationDate="2017-09-30T17:07:28.150" Score="1" Body="&lt;p&gt;Find a vector $\vec a$ such that $\vec v \times \vec a \ne 0$. Define $\hat x = (\vec v\,\times \vec a)/||\vec v \times \vec a||$. By construction, $\hat x$ is a unit vector and is orthogonal to $\vec v$. Define $\hat y = \vec v \times \vec x$. The triple $(\hat v, \hat x, \hat y)$ form a right handed coordinate system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Form a second coordinate system $(\hat v, \hat x', \hat y')$ by rotating about $\hat v$ by an angle $\alpha$, where $\alpha$ is drawn from $U(0,2\pi)$. Finally, form a third coordinate system $(\hat w, \hat x', \hat y'')$ by rotating about $\hat x'$ by the desired angle $\beta$. By construction, $\hat w$ is a unit vector and $\hat w \cdot \hat v = \cos \beta$. Thanks to the original random rotation about $\hat v$, $\langle \hat w \times \hat v \rangle = 0$.&lt;/p&gt;&#xA;" OwnerUserId="7865" OwnerDisplayName="David Hammen" LastActivityDate="2017-09-30T17:07:28.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27968" PostTypeId="2" ParentId="27965" CreationDate="2017-09-30T17:33:48.823" Score="2" Body="&lt;p&gt;If you want to use a single random number, you first need to find &quot;a&quot; solution (one vector w that meets the criterion), then rotate that vector about $\mathbf{v}$ by a randomly generated angle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have some flexibility when finding that first vector $w_0$ - we just need to find &quot;a&quot; vector at right angles to $v$ and rotate about that axis by $\beta$. We can choose any vector that is not $v$ and take the cross product with $v$ to get a vector that is at right angles. The general equation for rotating about a vector by an angle is given by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rodrigues%27_rotation_formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rodriguez formula&lt;/a&gt; - if we are rotating about one of the unit axes, that simplifies considerably and allows you to get the initial vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rotating that initial vector about $\mathbf{v}$ is going to take a bit of work. It might be convenient to use the formulation given &lt;a href=&quot;https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation#Quaternion-derived_rotation_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; which uses quaternions to simplify the calculation of the rotation matrix. It means that you have to first express the rotation about the axis as a quaternion (where $\phi$ is a randomly generated angle between 0 and $2\pi$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q = \begin{bmatrix}{&#xA;q_r\\&#xA;q_i\\&#xA;q_j\\&#xA;q_k}&#xA;\end{bmatrix}= \begin{bmatrix}{&#xA;\cos\frac{\phi}{2}\\&#xA;v_1 \sin{\frac{\phi}{2}}\\&#xA;v_2 \sin{\frac{\phi}{2}}\\&#xA;v_3 \sin{\frac{\phi}{2}}}&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After which you define the rotation as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{R} = \begin{bmatrix}&#xA;1-2s(q_j^2+q_k^2) &amp;amp; 2s(q_i q_j - q_k q_r) &amp;amp; 2s(q_i q_k + q_j q_r)\\&#xA;2s(q_i q_j + q_k q_r) &amp;amp; 1 - 2s(q_i^2 + q_k^2) &amp;amp; 2s(q_j q_k - q_i q_r)\\&#xA;2s(q_i q_k - q_j q_r) &amp;amp; 2s(q_j q_k + q_i q_r) &amp;amp; 1 - 2s(q_i^2 + q_j^2)&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $s=1$ if you constructed your quaternion to have unit length (more generally, $s = ||\mathbf{q}||^{-2}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computationally, a lot of terms appear in pairs so you only need to do the multiplication once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible there is a clever shortcut - but I don't know of one. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I decided to see if I could implement the above in a piece of C code, and the result is here. No warranties that this is bug free, but some simple testing suggested it is &quot;mostly correct&quot;. Might be a good starting point (note - in latest edit I made some changes to the calculation of w0 as the original was wrong...)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;&#xA;#define TWOPI (4*acos(0))&#xA;#define NUM 1000000&#xA;&#xA;float normalize(float *a) {&#xA;// normalize the vector given&#xA;  float temp = sqrt(a[0]*a[0]+a[1]*a[1]+a[2]*a[2]);&#xA;  for(int ii=0; ii&amp;lt;3; ii++) a[ii]/=temp;&#xA;  return temp;&#xA;}&#xA;&#xA;void cross(float *a, float *b, float *c) {&#xA;// return cross product of a and b in c&#xA;  c[0] = a[1]*b[2] - b[1]*a[2];&#xA;  c[1] = a[2]*b[0] - a[0]*b[2];&#xA;  c[2] = a[0]*b[1] - a[1]*b[0];&#xA;  return;&#xA;}&#xA;&#xA;float dot(float *a, float *b) {&#xA;  float sum=0;&#xA;  for(int ii=0; ii&amp;lt;3; ii++) sum+=a[ii]*b[ii];&#xA;  return sum;&#xA;}&#xA;&#xA;void printmat(float r[][3]) {&#xA;  for(int ii=0; ii&amp;lt;3; ii++) {&#xA;    for(int jj=0; jj&amp;lt;3; jj++) {&#xA;      printf(&quot;%.4f\t&quot;, r[ii][jj]);&#xA;    }&#xA;    printf(&quot;\n&quot;);&#xA;  }&#xA;  printf(&quot;\n&quot;);&#xA;  return;&#xA;}&#xA;&#xA;void printquat(float q[]) {&#xA;  for(int ii=0; ii&amp;lt;4; ii++) printf(&quot;%.4f\n&quot;, q[ii]);&#xA;  printf(&quot;\n&quot;);&#xA;  return;&#xA;}&#xA;&#xA;void rotvec(float *axis, float *vec, float angle, float* result) {&#xA;// rotate vec about axis by angle&#xA;// return result&#xA;  float q[4]; // quaternion representation of vector&#xA;  float R[3][3]; // rotation matrix&#xA;  float s = sin(angle/2), c = cos(angle/2);&#xA;  // construct quaternion:&#xA;  q[0] = c;&#xA;  for(int ii=0; ii&amp;lt;3; ii++) q[ii+1]=axis[ii]*s;&#xA;&#xA;  #ifdef VERBOSE&#xA;    printf(&quot;quaternion:\n&quot;);&#xA;    printquat(q);&#xA;  #endif&#xA;  // construct rotation matrix:&#xA;  float qii, qij, qik, qir, qjj, qjk, qjr, qkk, qkr;&#xA;  qii=q[1]*q[1]; qij=q[1]*q[2]; qik=q[1]*q[3]; qir=q[1]*q[0];&#xA;  qjj=q[2]*q[2]; qjk=q[2]*q[3]; qjr=q[2]*q[0];&#xA;  qkk=q[3]*q[3]; qkr=q[3]*q[0];&#xA;  R[0][0]=1-2*(qjj+qkk);&#xA;  R[0][1]=2*(qij-qkr);&#xA;  R[0][2]=2*(qik+qjr);&#xA;  R[1][0]=2*(qij+qkr);&#xA;  R[1][1]=1-2*(qii+qkk);&#xA;  R[1][2]=2*(qjk-qir);&#xA;  R[2][0]=2*(qij-qjr);&#xA;  R[2][1]=2*(qjk+qir);&#xA;  R[2][2]=1-2*(qii+qjj);&#xA;&#xA;  #ifdef VERBOSE&#xA;    printf(&quot;rotation matrix: \n&quot;);&#xA;    printmat(R);&#xA;  #endif&#xA;&#xA;  for(int ii=0; ii&amp;lt;3; ii++) {&#xA;    result[ii]=0;&#xA;    for(int jj=0; jj&amp;lt;3; jj++) {&#xA;      result[ii]+=R[ii][jj]*vec[jj];&#xA;    }&#xA;  }&#xA;  return;&#xA;}&#xA;&#xA;void printvec(float *v) {&#xA;  for(int ii=0; ii&amp;lt;3; ii++) printf(&quot;%f\n&quot;, v[ii]);&#xA;  printf(&quot;\n&quot;);&#xA;  return;&#xA;}&#xA;&#xA;&#xA;void findw0(float *v, float b, float *w0) {&#xA;  // find any vector that has an angle b with vector v&#xA;  int mi=0;&#xA;  float minax=fabs(v[0]);&#xA;  for(int ii=1; ii&amp;lt;3; ii++) {&#xA;    if(fabs(v[ii])&amp;lt;minax) {&#xA;      mi=ii;&#xA;      minax = fabs(v[ii]);&#xA;    }&#xA;  }&#xA;  float X[]={0,0,0};&#xA;  X[mi]=1;&#xA;  float A[3]; // the axis we want to rotate about at right angles to v&#xA;  cross(v, X, A); // this finds &quot;a&quot; vector at right angles&#xA;  normalize(A); // make it unit length&#xA;  rotvec(A, v, b, w0); // rotate v about A by angle b&#xA;  return;&#xA;}&#xA;&#xA;int main(void) {&#xA;// given an input vector v&#xA;// and an angle b&#xA;// find a random vector w that gives dot(v,w)=cos(b)&#xA;&#xA;  float v[]={0,0,1};&#xA;  float b = acos(0)/2; // pi/4&#xA;  normalize(v);&#xA;  // find &quot;a vector&quot; w0 that has the right relationship:&#xA;  float w0[3];&#xA;  findw0(v, b, w0);&#xA;  printf(&quot;vector w0 that is pi/4 from Z axis:\n&quot;);&#xA;  printvec(w0);&#xA;  float w[3];&#xA;  float crossSum[]={0,0,0};&#xA;  float crossTemp[3];&#xA;  float bsum = 0;&#xA;  for(int ii=0; ii&amp;lt;NUM; ii++) {&#xA;    // generate random angle between -pi and pi:&#xA;    b = TWOPI*(rand()*1.0/RAND_MAX-0.5);&#xA;    bsum+=b;&#xA;    rotvec(v, w0, b, w);&#xA;    cross(v, w, crossTemp);&#xA;    for(int jj=0; jj&amp;lt;3; jj++) crossSum[jj]+=crossTemp[jj]/NUM;&#xA;  }&#xA;  printf(&quot;after %d iterations, mean cross product is\n&quot;, NUM);&#xA;  printvec(crossSum);&#xA;  printf(&quot;mean angle of rotation is %.3f\n&quot;, bsum/NUM);&#xA;  return 1;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16935" OwnerDisplayName="Floris" LastActivityDate="2017-09-30T19:58:50.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27969" PostTypeId="2" ParentId="27965" CreationDate="2017-09-30T22:04:40.887" Score="3" Body="&lt;p&gt;First, set up an orthonormal coordinate system $\{ \hat{\bf x}, \hat{\bf y}, {\bf v} \}$ with ${\bf v}$ as one axis. Pick any random vector ${\bf a}$ which is not collinear with ${\bf v}$. Then either choice of unit vector&#xA;$$\hat{\bf x} := \frac{{\bf a} \times {\bf v}}{|{\bf a} \times {\bf v}|} \quad \text{or} \quad \hat{\bf x} := \frac{{\bf a} - ({\bf a} \cdot {\bf v}) {\bf v}}{|{\bf a} - ({\bf a} \cdot {\bf v}) {\bf v}|}$$&#xA;will be orthogonal to ${\bf v}$. $\hat{\bf y} := {\bf v} \times \hat{\bf x}$ completes the orthonormal coordinate frame. Now you can simply use spherical coordinates with ${\bf v}$ as the axis that is usually denoted as the $z$-axis: just let&#xA;$${\bf w} = \sin \beta \cos \phi\, \hat{\bf x} + \sin \beta \sin \phi\, \hat{\bf y} + \cos \beta\, {\bf v},$$&#xA;where $\alpha$ is drawn uniformly from $[0,2 \pi)$.&lt;/p&gt;&#xA;" OwnerDisplayName="tparker" LastActivityDate="2017-09-30T22:04:40.887" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="27970" PostTypeId="1" AcceptedAnswerId="28133" CreationDate="2017-10-02T13:33:07.440" Score="2" ViewCount="2105" Body="&lt;p&gt;Does the CFL condition play any role in a pure Stokes flow, i.e. convective term is neglibile, or vanishing? If not, what is the &quot;equivalent&quot; condition for stability? I have read something about the diffusional time scale but it's quite vague. Anyone with in-depth insight?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And a quotation from a paper [1]:&lt;br&gt;&#xA;&lt;em&gt;&quot;The time step is $3.10^-3\dot\gamma^-1$, corresponding to a CFL number based on the diffusional time scale $CFL_d=v\Delta t/\Delta^2$ of 50.&quot;&lt;/em&gt;   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Gallier, S., Lemaire, E., Lobry, L., &amp;amp; Peters, F. (2014). A fictitious domain approach for the simulation of dense suspensions. Journal of Computational Physics, 256, 367-387.&lt;/p&gt;&#xA;" OwnerUserId="25093" LastEditorUserId="25093" LastEditDate="2017-10-05T12:00:37.307" LastActivityDate="2017-10-25T03:50:07.420" Title="CFL condition in Stokes equation" Tags="|navier-stokes|advection-diffusion|cfl|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="27973" PostTypeId="2" ParentId="26778" CreationDate="2017-10-02T20:51:29.110" Score="1" Body="&lt;p&gt;You can also have a look at detecting crossing with the local Hessian matrix : &lt;a href=&quot;https://dsp.stackexchange.com/questions/10579/how-hessian-feature-detector-works&quot;&gt;https://dsp.stackexchange.com/questions/10579/how-hessian-feature-detector-works&lt;/a&gt;&#xA;It can detect &quot;corners&quot; (intersection of 2 lines), but maybe the limitation would be to determined between a sharp curve (like if it goes a 90° angle) and an actual intersection of 2 independant curves.&lt;/p&gt;&#xA;" OwnerUserId="25545" LastActivityDate="2017-10-02T20:51:29.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27974" PostTypeId="2" ParentId="20875" CreationDate="2017-10-03T02:43:14.563" Score="3" Body="&lt;p&gt;I'm going through some of my old StackExchange posts and came across this one.  As it turns out, the answer led to a section in &lt;a href=&quot;http://people.csail.mit.edu/jsolomon/assets/gw.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a published paper&lt;/a&gt;!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As detailed in that paper (and in its notation), if you wish to minimize $\mathrm{KL}(B|AA^\top)$, the following iterations will decrease the objective---and in practice work well---&#xA;$$&#xA;\begin{array}{rl}&#xA;U_k&amp;amp;\gets A_k\otimes[(B\oslash AA^\top)A]\\&#xA;A_{k+1}&amp;amp;\gets U\cdot\mathrm{diag}[1\oslash&#xA;\sqrt{U^\top\mathbf{1}}]&#xA;\end{array}&#xA;$$&#xA;Here, $\mathbf{1}$ denotes the vector of all ones, $\otimes$ denotes elementwise multiplication, and $\oslash$ denotes elementwise division.   $A$ should be initialized using random entries.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2017-10-03T02:43:14.563" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27975" PostTypeId="1" AcceptedAnswerId="27976" CreationDate="2017-10-03T22:41:43.783" Score="3" ViewCount="141" Body="&lt;p&gt;Cross-posting from &lt;a href=&quot;https://math.stackexchange.com/q/2455697/487123&quot;&gt;math.stackexchange&lt;/a&gt;, since there might be people here familiar with this topic.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Assume working in floating point arithmetic with finite precision, bounded exponent and rounding to nearest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $x,y$ be positive. It is not hard to find examples such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(x+y)-x &amp;gt; y$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to construct a test for a software I wanted to find an example or prove that such example doesn't exist of following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $s(x)$ be the successor of $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to have both $$\begin{align}(x+y)-x&amp;amp;&amp;gt;y\\(x+y)-s(x)&amp;amp;&amp;gt;y\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is easy to write a program that searches for such an example, but also it is unfeasible to test all possibilities and show that the example doesn't exist in this manner. So far my code hasn't got any example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: In case seeing an example of $(x+y)-x&amp;gt;y$ helps somehow, take &#xA;$$&#xA;\begin{align}&#xA;x&amp;amp;=1.1234567891234568\\&#xA;y&amp;amp;=1e-5\text{ ( denoting }10^{-5})&#xA;\end{align}&#xA;$$&#xA;Then $(x+y)-x=1.0000000000065512e-05 &amp;gt; y$. Examples of the first inequality there are many.&lt;/p&gt;&#xA;" OwnerUserId="25572" LastEditorUserId="25572" LastEditDate="2017-10-03T23:08:46.887" LastActivityDate="2017-10-04T00:06:28.520" Title="Associativity in floating point arithmetic failing by two values" Tags="|floating-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27976" PostTypeId="2" ParentId="27975" CreationDate="2017-10-03T23:08:50.897" Score="6" Body="&lt;p&gt;These kinds of floating point counterexamples are notoriously tricky to discover by hand. Usually you &lt;em&gt;do&lt;/em&gt; expect random brute-force search to work, especially on 32-bit floats, but sometimes it fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These days, you have SMT solvers, like Z3 available to you, so you can try formulating your formula in Z3 to see if it can find a counterexample. This is the &quot;high-tech&quot; method of solving your problem. This approach is nice because it doesn't require you to know very much about floating point arithmetic to be able to use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried this, using Haskell, with the SBV package using Z3 as the backend. It looks only at the range $x\in(1,2)$, where $s(x) = x + \epsilon$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import Data.SBV&#xA;&#xA;ε = 1.1920928955078125e-07&#xA;&#xA;f :: SFloat -&amp;gt; SFloat -&amp;gt; SBool&#xA;f x y = 1 .&amp;lt; x &amp;amp;&amp;amp;&amp;amp; x .&amp;lt; 2 &amp;amp;&amp;amp;&amp;amp; fpIsNormal y &amp;amp;&amp;amp;&amp;amp; (x + y) - (x + ε) .&amp;gt; y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it immediately found this counterexample:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;λ&amp;gt; sat f&#xA;Satisfiable. Model:&#xA;  s0 =       1.5 :: Float&#xA;  s1 = 3.9999998 :: Float&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This satisfies $(x+y)-s(x) &amp;gt; y$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; x, y = 1.5f0, 3.9999998f0&#xA;julia&amp;gt; ((x + y) - nextfloat(x)) - y&#xA;2.3841858f-7&#xA;julia&amp;gt; (x + y) - nextfloat(x) &amp;gt; y&#xA;true&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; Looking at your example, it's unclear to me if you have restrictions on the magnitudes on $x$ and $y$. For example, if you restrict it to $x\in(1,2)$, $y\in(0,1)$, then there is no way to satisfy $(x+y)-s(x)&amp;gt;y$. But restricting only $x\in(1,2)$, but not $y$, it seems you get very large $n$ with $(x+y)-(x+n\epsilon) &amp;gt; y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at some small examples, it seems the worst case can be reached by taking (in binary, single precision)&#xA;$$ x = 1.5 = (1.1)_2, \qquad y = 2^k\times (0.111111111111111111111111)_2 = 2^k\times (1-2^{-24}), $$&#xA;which gives $n=2^{k-1}$, so $(x+y)-(x+2^{k-1}\epsilon)=y$. In double precision it works to take $x=1.5$ and $y=(1-2^{-53})\times 2^k$ (which is just all ones in binary), and the relationship only starts to break down from $k\geq51$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-10-04T00:06:28.520" LastActivityDate="2017-10-04T00:06:28.520" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="27977" PostTypeId="1" CreationDate="2017-10-04T02:37:13.933" Score="3" ViewCount="2394" Body="&lt;p&gt;I've written a C++ function that multiplies a sparse matrix (stored in CSR format) by a dense vector.  Here's the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VectorXd csrMult(VectorXd x, vector&amp;lt;double&amp;gt; Adata, vector&amp;lt;int&amp;gt; Aindices, vector&amp;lt;int&amp;gt; Aindptr, int numRowsA) &#xA;{&#xA;    VectorXd Ax = VectorXd::Zero(numRowsA); &#xA;    for (int i = 0; i &amp;lt; numRowsA; i++)&#xA;    {&#xA;        for (int dataIdx = Aindptr[i]; dataIdx &amp;lt; Aindptr[i + 1]; dataIdx++)&#xA;        {&#xA;            Ax[i] += Adata[dataIdx] * x[Aindices[dataIdx]];&#xA;        }&#xA;    }&#xA;&#xA;    return Ax;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here VectorXd is a data type provided by the Eigen3 linear algebra library.  The inputs Adata, Aindices, and Aindptr describe a matrix A in CSR format.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be precise: Adata is a list of nonzero entries of A (stored in row major order), Aindices[i] tells us which column of A the nonzero entry Adata[i] belongs to, and Aindptr[i+1] - Aindptr[i] is the number of nonzero entries in the ith row of A. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm observing that Eigen3's sparse matrix-vector multiplication operation is about 5 times faster than my csrMult function, even when openMP is disabled.  However, when I look at the source code &lt;a href=&quot;https://github.com/libigl/eigen/blob/master/Eigen/src/SparseCore/SparseDenseProduct.h&quot; rel=&quot;nofollow noreferrer&quot;&gt;SparseDenseProduct.h&lt;/a&gt; that I believe Eigen3 is using to compute this matrix-vector product, it's not clear to me how Eigen3 is faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Do you have any suggestions to improve the speed of my code?  Can you explain why Eigen3 is faster?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 3:&lt;/strong&gt; I noticed an &lt;strong&gt;important clue&lt;/strong&gt; about what's going on.  When I make the problem smaller by setting all but the top N entries of A equal to 0 (so that A becomes more sparse), my implementation compares better with Eigen3.  In fact, if I set all but the top 10,000 nonzero entries of A equal to 0, my implementation beats Eigen3 by a factor of about 3.  However, for the very large problem size I am interested in, Eigen3 beats my implementation by a factor of 2.  I would still really like to tie with Eigen3 for large problem sizes.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2:&lt;/strong&gt; I changed the inputs to pass by reference, as @TylerOlsen suggested, and now Eigen3 is only about twice as fast as my code (with openMP disabled).  So that was a significant improvement. Here's the latest version of my code.  I still need to figure out how to make my code twice as fast in order to tie with Eigen3.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void csrMult_v3(VectorXd&amp;amp; Ax, VectorXd&amp;amp; x, vector&amp;lt;double&amp;gt;&amp;amp; Adata, vector&amp;lt;int&amp;gt;&amp;amp; Aindices, vector&amp;lt;int&amp;gt;&amp;amp; Aindptr)&#xA;{&#xA;    // This code assumes that the size of Ax is numRowsA.&#xA;    for (int i = 0; i &amp;lt; Ax.size(); i++)&#xA;    {&#xA;        double Ax_i = 0.0;&#xA;        for (int dataIdx = Aindptr[i]; dataIdx &amp;lt; Aindptr[i + 1]; dataIdx++)&#xA;        {&#xA;            Ax_i += Adata[dataIdx] * x[Aindices[dataIdx]];&#xA;        }&#xA;&#xA;        Ax[i] = Ax_i;&#xA;    }       &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1:&lt;/strong&gt; I also tried the following code, where the value Ax[i] is accumulated in a temporary variable, but the effect on runtime was negligible.  I'm still observing Eigen3 is about 5 times faster (without openMP enabled).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VectorXd csrMult_v2(VectorXd x, vector&amp;lt;double&amp;gt; Adata, vector&amp;lt;int&amp;gt; Aindices, vector&amp;lt;int&amp;gt; Aindptr, int numRowsA)&#xA;{&#xA;    VectorXd Ax = VectorXd::Zero(numRowsA);&#xA;    for (int i = 0; i &amp;lt; numRowsA; i++)&#xA;    {&#xA;        double Ax_i = 0.0;&#xA;        for (int dataIdx = Aindptr[i]; dataIdx &amp;lt; Aindptr[i + 1]; dataIdx++)&#xA;        {&#xA;            Ax_i += Adata[dataIdx] * x[Aindices[dataIdx]];&#xA;        }&#xA;&#xA;        Ax[i] = Ax_i;&#xA;    }&#xA;&#xA;    return Ax;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3274" LastEditorUserId="3274" LastEditDate="2017-10-05T07:09:13.110" LastActivityDate="2017-10-05T07:09:13.110" Title="How can I speed up this code for sparse matrix-vector multiplication?" Tags="|sparse-matrix|matrix|eigen|" AnswerCount="1" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="27978" PostTypeId="2" ParentId="27977" CreationDate="2017-10-04T14:31:53.110" Score="1" Body="&lt;p&gt;After your second edit, you are already here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (int i = 0; i &amp;lt; Ax.size(); i++)&#xA;{&#xA;  double Ax_i = 0.0;&#xA;  for (int dataIdx = Aindptr[i]; dataIdx &amp;lt; Aindptr[i + 1]; dataIdx++)&#xA;  {&#xA;     Ax_i += Adata[dataIdx] * x[Aindices[dataIdx]];&#xA;  }&#xA;&#xA;  Ax[i] = Ax_i;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But notice how you in row &lt;code&gt;i+1&lt;/code&gt; you are initializing &lt;code&gt;dataIdx&lt;/code&gt; to the value &lt;code&gt;Aindptr[i+1]&lt;/code&gt; it &lt;em&gt;already&lt;/em&gt; has because you stopped the loop in row &lt;code&gt;i&lt;/code&gt; when it had that value. So you can transform this to the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int dataIdx = Aindptr[0];&#xA;for (int i = 0; i &amp;lt; Ax.size(); i++)&#xA;{&#xA;  double Ax_i = 0.0;&#xA;  for (; dataIdx &amp;lt; Aindptr[i + 1]; dataIdx++)&#xA;  {&#xA;     Ax_i += Adata[dataIdx] * x[Aindices[dataIdx]];&#xA;  }&#xA;&#xA;  Ax[i] = Ax_i;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The next piece of information you need to know is that you are walking the &lt;code&gt;Aindices&lt;/code&gt; array &lt;em&gt;linearly, from left to right&lt;/em&gt; because your indices &lt;code&gt;dataIdx&lt;/code&gt; are incremented one by one. So you can save yourself the subscripting and do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int dataIdx = Aindptr[0];&#xA;int *Aindex = &amp;amp;Aindices[dataIdx];&#xA;for (int i = 0; i &amp;lt; Ax.size(); i++)&#xA;{&#xA;  double Ax_i = 0.0;&#xA;  for (; dataIdx &amp;lt; Aindptr[i + 1]; ++dataIdx,++Aindex)&#xA;  {&#xA;     Ax_i += Adata[dataIdx] * x[*Aindex];&#xA;  }&#xA;&#xA;  Ax[i] = Ax_i;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Give this a try.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-04T14:31:53.110" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="27979" PostTypeId="2" ParentId="16002" CreationDate="2017-10-04T17:57:22.483" Score="1" Body="&lt;p&gt;Fitting together B-splines with a given continuity is a hard problem. This was actually the motivation behind developing a &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.87.4374&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;generalization of NURBS called T-splines&lt;/a&gt;. There are many articles on T-splines and T-NURCCs that could be helpful to you if you're interested in going that route. &lt;/p&gt;&#xA;" OwnerUserId="21131" LastActivityDate="2017-10-04T17:57:22.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27980" PostTypeId="2" ParentId="14412" CreationDate="2017-10-05T08:59:06.627" Score="3" Body="&lt;p&gt;I would like to add that while what Geoff Oxberry suggests for long-term integration (using symplectic integrators) is true, in some cases it won't work.&#xA;More specifically, if you have dissipative forces, your system does not preserve the energy anymore, and therefore you cannot resort to symplectic integrators in that case. The person asking the question was talking about low Earth orbits, and such orbits exhibit a big amount of atmospheric drag, that is a dissipative force that precludes using such symplectic integrators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that specific case (and for cases where you cannot use /do not have access to/do not want to use symplectic integrators), I would recommend the use of Bulirsch-Stoer integrator if you need precision and efficiency over long timescales. It works well by experience, and is also recommended by the Numerical Recipes (Press et al., 2007).&lt;/p&gt;&#xA;" OwnerUserId="25589" LastEditorUserId="25589" LastEditDate="2017-12-07T14:17:19.193" LastActivityDate="2017-12-07T14:17:19.193" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="27981" PostTypeId="1" AcceptedAnswerId="27983" CreationDate="2017-10-05T10:28:16.643" Score="3" ViewCount="134" Body="&lt;p&gt;I have been following Toro's book on Riemann solvers to implement a finite-volume scheme for computational fluid dynamics. The Riemann solvers presented in the book seem to be fairly tightly coupled to the equations in question. There are papers on how to construct the schemes for various sets of equations such as the Euler equations, relativistic hydrodynamics, or magneto-hydrodynamics (for example).&lt;/p&gt;&#xA;&lt;p&gt;I am wondering: is there prior work on a more general formulation of a scheme (like HLLC or Roe) that solves the Riemann problem for a larger class of equations instead of constructing it case-by-case?&lt;/p&gt;&#xA;" OwnerUserId="25590" LastEditorUserId="20688" LastEditDate="2022-01-01T22:06:19.917" LastActivityDate="2022-01-01T22:06:19.917" Title="Are there general formulations of Riemann solvers?" Tags="|fluid-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27983" PostTypeId="2" ParentId="27981" CreationDate="2017-10-05T13:17:11.350" Score="4" Body="&lt;p&gt;I cant think of any truly generic ones except the Lax-Friedrichs flux, which is very dissipative. The next simplest would be the Rusanov flux which needs knowledge of minimum and maximum speeds arising in the Riemann problem, which of course depends on the particular form of the flux function. While this is also called a central flux, it can be interpreted as a Riemann solver. Most other more sophisticated solvers need knowledge of other intermediate speeds. The HLL type solvers require solving jump conditions to find intermediate states which depends on the flux. Still more sophisticated ones like Roe need knowledge of eigenvectors also.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2017-10-05T13:17:11.350" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27984" PostTypeId="2" ParentId="14412" CreationDate="2017-10-05T15:17:44.817" Score="6" Body="&lt;p&gt;summarizing some points:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If it's a long-term integration of a non-dissapative model, a symplectic integrator is what you're looking for.&lt;/li&gt;&#xA;&lt;li&gt;Otherwise, since it's an equation of motion, Runge-Kutta Nystrom methods will be more efficient than a transformation to a first order system. There are high order RKN methods due to DP. There are some implementations, like &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/dynamical_solve.html#Runge-Kutta-Nystr%C3%B6m-Integrators-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here in Julia they are documented&lt;/a&gt; and &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/25291-rkn1210-a-12th-10th-order-runge-kutta-nystr%C3%B6m-integrator&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's a MATLAB one&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;High order Runge-Kutta methods are only needed if you want a high accuracy solution. If it's lower tolerances then a 5th order RK will likely be faster (for the same error). The best thing to do if you need to solve this often is to test a bunch of different methods. &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/NonStiffODE/ThreeBody%20Work-Precision%20Diagrams.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this set of benchmarks on 3-body problems&lt;/a&gt; we see that (for the same error) high order RK methods are only a marginal improvement in speed, though as error -&amp;gt; 0 you can see that the improvement already goes to &amp;gt;5x against Dormand-Prince 45 (&lt;code&gt;DP5&lt;/code&gt;) when you're looking at 4 digits of accuracy (tolerances are wayyyy lower for this though. Tolerances are only a ballpark in any problem). As you pull the tolerances even lower the improvement from a high order RK method grows, but you may need to start using higher precision numbers.&lt;/li&gt;&#xA;&lt;li&gt;Dormand-Prince order 7/8 algorithm has a different 8th order tableau than the DP853 method of Hairer's &lt;code&gt;dop853&lt;/code&gt; and DifferentialEquations.jl's &lt;code&gt;DP8&lt;/code&gt; (which are the same). The latter 853 method cannot be implemented in standard tableau version of an Runge-Kutta method since it's error estimator is non-standard. But this method is much more efficient and I wouldn't recommend even using the older Fehlberg 7/8 or DP 7/8 methods.&lt;/li&gt;&#xA;&lt;li&gt;For high order RK methods, the &lt;a href=&quot;http://people.math.sfu.ca/%7Ejverner/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Verner &amp;quot;Efficient&amp;quot; methods&lt;/a&gt; are the gold standard. That shows up in the benchmarks I linked. You can code those into Boost yourself, or use one of the 2 packages which implement them if you want those easier (Mathematica or DifferentialEquations.jl).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:14:28.650" LastActivityDate="2020-07-19T18:14:28.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="27986" PostTypeId="1" AcceptedAnswerId="27988" CreationDate="2017-10-05T19:33:27.170" Score="4" ViewCount="174" Body="&lt;p&gt;Assume that we have a 2D electromagnetic field (in the sense that: $E=(E_x,E_y,0)$, $B=(0,0,B_z)$, and all derivatives with respect to $z$ are $0$), and that we are considering a system made up of two walls at $x=-b$ and $x=b$, held at fixed potential $\phi(y)$, with a third wall at $y=0$ also held at a fixed potential $\phi(x)=1$. The current in our system is $\vec{j} \propto B \times E+\sigma E$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Yee scheme to solve the Maxwell equations for our system how do I force the fixed potential boundary equations at the walls? Is there another scheme more appropriate for these types of problems?&lt;/p&gt;&#xA;" OwnerUserId="25597" LastActivityDate="2017-10-06T02:51:15.913" Title="How to force potential boundary conditions in the Yee scheme for solving Maxwell's equations?" Tags="|finite-difference|computational-physics|electromagnetism|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27987" PostTypeId="1" CreationDate="2017-10-05T20:04:50.207" Score="4" ViewCount="517" Body="&lt;p&gt;Why do active set methods or the simplex method pivot only one variable at a time?  Ostensibly, we could add multiple columns to the basis during pivoting, but the standard presentation of the methods don't do this.  Likely, something breaks, but I don't understand what.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If something more concrete would help, on page 472 of Numerical Optimization, we have the active set method for a convex QP in Algorithm 16.3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BNics.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BNics.png&quot; alt=&quot;Active Set Method for Convex QP&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pivoting occurs in the choice of $j$ and and that single index $j$ entering the working set $W$.  Why choose a single $j$?  Couldn't we choose multiple indices that correspond to multiple negative multipliers $\lambda$?&lt;/p&gt;&#xA;" OwnerUserId="10094" LastActivityDate="2017-11-08T05:48:50.050" Title="Why do active set methods or the simplex method pivot only one variable at a time?" Tags="|optimization|convex-optimization|constrained-optimization|nonlinear-programming|linear-programming|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27988" PostTypeId="2" ParentId="27986" CreationDate="2017-10-05T20:36:30.000" Score="4" Body="&lt;p&gt;Interesting question. I would expect the Yee scheme to be indifferent to static (bias) fields induced by constant potentials. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the electrostatic case, if you have a constant electric potential $\phi$, it induces a field $\vec E = \nabla \phi$. The update equation is (modulo constants) $\frac{d}{dt}\vec B = \nabla \times \vec E$. But since $\vec E$ is a gradient, it lies in the nullspace of curl, so $\frac{d}{dt}\vec B=\nabla \times \nabla \phi= \vec 0$ (that is, $\vec B$ doesn't vary in time). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The elegance of the Yee scheme (and other schemes drawn from Nedelec/Whitney-type elements) is that all of these identities are upheld faithfully after (spatial) discretization. In the discrete setting, your potential $\mathbf v$ will give rise to a static field $\mathbf e = \mathbf G \mathbf v$, where $\mathbf G$ is the sparse stencil induced by the signed vertex-to-edge adjacency. The (semi)discrete update equation is  $\frac{d}{dt}\mathbf b = \mathbf C \mathbf e$, where C is the edge-to-facet adjacency. By construction, $\mathbf C \mathbf G = \mathbf 0$, so $\mathbf b$ doesn't vary in time, either. Note, incorporating the gauss law $\nabla \cdot \vec B=0$ basically forces trivial/null $\vec B$ as an initial condition, so when you read &quot;$\vec B$ doesn't vary in time&quot;, what you should be really thinking is &quot;$\vec B$ starts as zero and stays that way forever&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My point being, an electrostatic (gradient) field is basically ignored by the scheme, as the curl operator / $\mathbf C$-stencil doesn't &quot;sense&quot; or &quot;measure&quot; it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that said, some care is required when you (i) set initial conditions and (ii) incorporate sources, to make sure you do not &quot;source&quot; $\mathbf e$ and $\mathbf b$ in any way that's inconsistent with Maxwell's equations. You should consult literature, people have studied this sort of thing. In particular, that $\vec J$ source term is murky. Most codes I have worked upon use engineering-motivated sources like planewaves and lumped ports. I am not sure how to interpret the role of your volume source .. looks like poynting vector and ohmic conduction? The latter is normally accounted for by directly modifying the $\mathbf e$ updates in a &quot;semi-implicit&quot; fashion, not an explicit source. I don't know what the former term is trying to represent.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2017-10-06T02:51:15.913" LastActivityDate="2017-10-06T02:51:15.913" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27989" PostTypeId="1" AcceptedAnswerId="27990" CreationDate="2017-10-05T23:55:33.653" Score="3" ViewCount="1174" Body="&lt;p&gt;I hope someone knows an efficient computational approach to the following 2D problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given two vectors $\mathbf{A}$ and $\mathbf{B}$, find all grid points that lie within the parallelogram spanned by these vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GoWbX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GoWbX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This feels like it should be a &quot;known&quot; problem, but I lack the vocabulary to search for the right terms. I know how to determine that a given point is inside; I would like not to have to test lots of points... are there any tricks I should know?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I look at the picture, I am thinking &quot;starting at the origin, you need to move to the left to find points; at a certain X coordinate you can go up a step without crossing A&quot;. But as A and B can be swapped, and pointing in any direction, the approach I need has to be a little more robust. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately I need to know not only the coordinates, but actually the values of $a$ and $b$ for each valid (green, in the diagram) grid point $\mathbf{P}$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf{P} = a\mathbf{A} + b\mathbf{B}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all integer-valued P(i,j) where $a,b \in [0,1\rangle$. If that is actually easier... that's the problem I ultimately need to solve (so I need coordinates (i,j) and their transform (a,b); obviously when I have one, I can find the other).&lt;/p&gt;&#xA;" OwnerUserId="16935" LastEditorUserId="16910" LastEditDate="2017-10-06T10:46:58.137" LastActivityDate="2017-11-02T04:43:51.903" Title="find grid points inside the parallelogram defined by an origin and two vectors" Tags="|computational-geometry|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27990" PostTypeId="2" ParentId="27989" CreationDate="2017-10-06T00:25:37.140" Score="3" Body="&lt;p&gt;I suggest that you break the query into two parts, points_inside(parallelogram) := points_inside(triangle1) union points_inside(triangle2), where the triangles are formed by T(origin,origin+A,origin+A+B) andT (origin,origin+A+B,origin+B).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the find-all-points-in-a-triangle query, this is basically a scanline-conversion problem. You can find solutions in computer-graphics like sources (for drawing triangles onto grids of pixels, that sort of thing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the (many) hits for &quot;scanline conversion of a triangle&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://vis.uky.edu/~ryang/Teaching/CS335-spr05/Lectures/g_05_fill.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://vis.uky.edu/~ryang/Teaching/CS335-spr05/Lectures/g_05_fill.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can probably find better ones. You could probably work with the polygon directly, it'll just be a little more complicated and might be a bit harder to find good tutorials. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try drawing the edges into a small image, then do some floodfill-like search until you hit them. Bresenhams is the prototypical algorithm for scanline conversion of a line.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2017-10-06T00:32:34.500" LastActivityDate="2017-10-06T00:32:34.500" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27991" PostTypeId="2" ParentId="27989" CreationDate="2017-10-06T04:54:43.893" Score="1" Body="&lt;p&gt;I'm also unaware of the standard methods of doing this, but I think there's a straightforward way to do it by exploiting linear algebra, assuming both $\mathbf{A}$ and $\mathbf{B}$ can be taken to have their tail at the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\mathbf{A} = (A_x, A_y)$, then define $\mathbf{A}_\perp := (A_y, -A_x)$, which is a vector perpendicular to $\mathbf{A}$. The projection of a position vector $\mathbf{P}$ on to $\mathbf{A}_\perp$ gives the &lt;a href=&quot;https://en.wikipedia.org/wiki/Distance_from_a_point_to_a_line&quot; rel=&quot;nofollow noreferrer&quot;&gt;distance between $\mathbf{P}$ and the line on which $\mathbf{A}$ lies&lt;/a&gt;, up to a sign which indicates whether $\mathbf{P}$ is above or below that line.&#xA;$$d_{\mathbf{A}}(\mathbf{P}) := \frac{\mathbf{P}\cdot\mathbf{A}_\perp}{\lVert\mathbf{A}_\perp\rVert}$$&#xA;The points in your parallelogram are precisely those for which $d_{\mathbf{A}}(\mathbf{P})$ lies between zero and the equivalent value for $\mathbf{B}$, taken as a position vector, &lt;em&gt;and&lt;/em&gt; the same is true with the roles of $\mathbf{A}$ and $\mathbf{B}$ reversed:&#xA;$$\bigl\{\mathbf{P}\ :\ 0 \leq d_{\mathbf{A}}(\mathbf{P}) \leq d_{\mathbf{A}}(\mathbf{B})\ \land\ 0 \leq d_{\mathbf{B}}(\mathbf{P}) \leq d_{\mathbf{B}}(\mathbf{A})\bigr\}$$&#xA;(you may have to reverse the signs of one or both inequalities depending on whether $d_{\mathbf{A}}(\mathbf{B})$ and $d_{\mathbf{B}}(\mathbf{A})$ are positive or negative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your number of points is not too large, you can devise some enclosing shape - for example a circle with radius $\lVert\mathbf{A}\rVert + \lVert\mathbf{B}\rVert$ (although it shouldn't be hard to do better) - and test all integer grid points within that shape to see which ones satisfy the inequalities. With access to a vectorized math library it should be only a brief piece of code. The computation you wind up doing boils down to computing two dot products and comparing each to the endpoints of a range, for example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(0 &amp;lt;= c1 * px + c2 * py &amp;lt;= c3) and (0 &amp;lt;= c4 * px + c5 * py &amp;lt;= c6)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the &lt;code&gt;cn&lt;/code&gt;'s are constants.&lt;/p&gt;&#xA;" OwnerUserId="103" LastActivityDate="2017-10-06T04:54:43.893" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="27992" PostTypeId="2" ParentId="27989" CreationDate="2017-10-06T10:35:09.170" Score="3" Body="&lt;p&gt;Depending on the performances that you want, there are an easy and a more complicated algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Algo 1: interior test in bbox (simple)&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;For each point p with integer coordinates in the bounding box of the parallelogram (q1,q2,q3,q4)&#xA;  If sign(det(p-q1, p-q2)) = sign(det(p-q2, p-q3)) = sign(det(p-q3, p-q4)) = sign(det(p-q4, p-q1))&#xA;     mark p&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where det( (x1,y1), (x2, y2)) = x1*y2 - x2*y1 denotes the determinant and sign() its sign (positive, zero or negative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Algo 2: scanline rasterization (more delicate)&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Compute the interval [Ymin Ymax] that bounds the Y's of the parallelogram&#xA;Determine for each value of Y in [Ymin Ymax] the leftmost XL[Y] and rightmost XR[Y] intersection between the horizontal line and the border of the parallelogram&#xA;    For Y in Ymin ... Ymax  &#xA;       For X in XL[Y] ... XR[Y]&#xA;           Mark X,Y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Step 2 (computing the scalines) can be done using Bresenham algorithm. The algorithm is described in full details in [1]. The second algorithm was used in early 3D game engines (before graphic boards were available) and is very efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More notes (if performance is really an issue)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting from Algo 1 and refining it, it is possible to obtain an even faster algorithm: the determinant computations can be split into what depends on X and what depends on Y, and one can compute at the beginning of the loop how the determinant vary. Then in the loop there are only additions and sign comparisons to do. The method is fully detailed in [2] (excellent article by Nick Capens, who implemented a full software emulation of DirectX 10 compatible graphic boards ages ago).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://www.idav.ucdavis.edu/education/GraphicsNotes/Rasterizing-Polygons/Rasterizing-Polygons.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.idav.ucdavis.edu/education/GraphicsNotes/Rasterizing-Polygons/Rasterizing-Polygons.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://forum.devmaster.net/t/advanced-rasterization/6145&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://forum.devmaster.net/t/advanced-rasterization/6145&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2017-10-06T10:43:52.207" LastActivityDate="2017-10-06T10:43:52.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27994" PostTypeId="2" ParentId="11399" CreationDate="2017-10-06T19:12:41.967" Score="2" Body="&lt;p&gt;&quot;Restart&quot; in this context does not mean that you reuse the Krylov space computed on a matrix $A$ for a nearby matrix $B$. It means that you start the method again, on the &lt;em&gt;same&lt;/em&gt; matrix, with a different starting vector $b$ for the Krylov subspace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not an expert in this area, but I would suggest you to search for &quot;recycling Krylov subspaces&quot; and &quot;inexact Arnoldi methods&quot; instead.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2017-10-06T19:12:41.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27995" PostTypeId="2" ParentId="19331" CreationDate="2017-10-07T00:44:24.917" Score="0" Body="&lt;p&gt;the answer maybe this &lt;a href=&quot;https://scicomp.stackexchange.com/a/13048/25606&quot;&gt;https://scicomp.stackexchange.com/a/13048/25606&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It sounds like you want the ** multi-precision plot function from SymPy**, which is capable of plotting arbitrary black-box functions over a given range.&lt;/p&gt;&#xA;" OwnerUserId="25606" LastActivityDate="2017-10-07T00:44:24.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27996" PostTypeId="2" ParentId="19331" CreationDate="2017-10-07T01:16:14.367" Score="1" Body="&lt;p&gt;Looks like you probably would like to steal some ideas &lt;a href=&quot;https://github.com/JuliaPlots/Plots.jl/issues/621&quot; rel=&quot;nofollow noreferrer&quot;&gt;from this discussion about implementing something similar in Julia&lt;/a&gt;. It's hard to cleanly summarize since there's a lot of different details, but essentially it's an algorithm which does a simple grid and then adaptively adds points recursively in the regions where the linear approximation is worse. It's like a course grid adaptivity in finite element methods where you take more grid points and see how far your interpolation was off at the grid points, and if it was a lot, you accept the new grid points and recursively re-apply the coarsening operation. Then details ensue, but it's all open source with a related discussion so you can check it out all the gory details. Taking that from 1D to 2D likely involves new code but the same principles.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-10-07T01:16:14.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="27998" PostTypeId="1" AcceptedAnswerId="28023" CreationDate="2017-10-07T07:52:34.537" Score="2" ViewCount="643" Body="&lt;p&gt;Is an error indicator an index for size measurement of something? And how is the relationship between the error estimator and error indicator? Is the error indicator just used for the (e.g. derivative) recovery based error estimation? If the solution of strong form has low regularity, such as including a discontinuous interface in the domain, then how one may design both stuff? Anyone can give me some intuitive pictures?&lt;/p&gt;&#xA;" OwnerUserId="24268" LastActivityDate="2021-05-27T08:13:48.020" Title="error estimator VS error indicator in the context of FEM error estimation" Tags="|finite-element|error-estimation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="27999" PostTypeId="1" AcceptedAnswerId="28002" CreationDate="2017-10-07T10:56:04.280" Score="0" ViewCount="66" Body="&lt;p&gt;I am trying to plot a figure similar to the one attached here,I tried searching how to do a similar kinds of  plot and i found about 'colormap','colorbar' in MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually I want to plot this situation -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are $1000$ cars indexed by $i=1,2,3,...,1000$ and the position changes as a function of time,now i plot index versus time and the pair(time,index) plots a value(here a colour) that is position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2rbQt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2rbQt.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am unable to find similar kind of plotting technique, any example on how to do this kind of plots in MATLAB or reference would be very much helpful.&lt;/p&gt;&#xA;" OwnerUserId="23297" LastEditorUserId="23297" LastEditDate="2017-10-07T13:52:42.707" LastActivityDate="2017-10-07T13:59:19.300" Title="Plotting using 'colorbar' in MATLAB" Tags="|matlab|plotting|" AnswerCount="1" CommentCount="1" ClosedDate="2017-10-08T19:43:35.183" ContentLicense="CC BY-SA 3.0" />
  <row Id="28000" PostTypeId="1" AcceptedAnswerId="28014" CreationDate="2017-10-07T11:39:29.327" Score="6" ViewCount="2309" Body="&lt;p&gt;I searched online about the way optimal is defined mathematically,but without any information acquired? How to prove whether a numerical method, e.g. FEM, has an &lt;strong&gt;&lt;em&gt;optimal&lt;/em&gt;&lt;/strong&gt; convergence rate or not? Anyone sheds a light on this?&lt;/p&gt;&#xA;" OwnerUserId="24268" LastActivityDate="2022-01-03T08:42:35.577" Title="What does optimal convergence rate mean? How to prove whether a numerical method, e.g. FEM, has an optimal convergence rate or not?" Tags="|finite-element|error-estimation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28001" PostTypeId="2" ParentId="28000" CreationDate="2017-10-07T13:45:54.697" Score="2" Body="&lt;p&gt;Assume you have a metric for some problem, e.g., the asymptotic convergence rate. If you can prove that there can be no algorithm that has a better value in the specific metric, then the algorithm is &lt;em&gt;optimal&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider multigrid methods. Solving a linear system with $n$ unknowns cannot be done with less than $\mathcal{O}(n)$ operations, because every variable has to be set at least once. For some problems it can be shown that a multigrid method can solve a linear system with $\mathcal{O}(n)$ operations up to the discretization accuracy. Hence, if we consider the metric asymptotic-number-of-operations, multigrid methods are optimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether an algorithm is optimal, therefore, depends on the metric. And the metric will depend on the context and the assumptions made. In the example above we restricted ourselves to the accuracy of the discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all, what &lt;em&gt;optimal&lt;/em&gt; means needs to be explained in the specific context.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2017-10-07T13:45:54.697" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28002" PostTypeId="2" ParentId="27999" CreationDate="2017-10-07T13:59:19.300" Score="1" Body="&lt;p&gt;The plot you have shown was probably created using &lt;a href=&quot;https://de.mathworks.com/help/matlab/ref/imagesc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;imagesc&lt;/a&gt;. The colors were choosen with the &lt;a href=&quot;https://de.mathworks.com/help/matlab/ref/colormap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;colormap&lt;/a&gt; command and the color bar is shown with &lt;a href=&quot;https://de.mathworks.com/help/matlab/ref/colorbar.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;colorbar&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2017-10-07T13:59:19.300" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28003" PostTypeId="2" ParentId="10936" CreationDate="2017-10-07T14:56:25.207" Score="0" Body="&lt;p&gt;I created a demo for sorting Distance matrix with the Fiedler Vector using PyTorch. You can take a look at the source code here:&#xA;&lt;a href=&quot;https://github.com/dimkastan/PyTorch-Spectral-clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dimkastan/PyTorch-Spectral-clustering&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="25612" LastActivityDate="2017-10-07T14:56:25.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28004" PostTypeId="1" CreationDate="2017-10-07T18:41:13.513" Score="5" ViewCount="160" Body="&lt;p&gt;Suppose we are given a set of symmetric, positive definite matrices $A_1,A_2,\ldots,A_k\in\mathbb{R}^{n\times n}$.  &lt;strong&gt;Is there any numerical method or reduction to a known problem (e.g. eigenvalue computation) that can solve the following problem?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{rl}&#xA;\min_{v_1,\ldots,v_k\in\mathbb{R}^n} &amp;amp; \sum_{i=1}^kv_i^\top A_iv_i\\&#xA;\textrm{subject to }&amp;amp;v_i^\top v_j=\delta_{[i=j]}.&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; The problem has some resemblance to the &quot;joint diagonalization&quot; problem, e.g. in &lt;a href=&quot;http://www.jmlr.org/papers/volume5/ziehe04a/ziehe04a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.  The difference seems to be that in joint diagonalization you use the same basis to diagonalize multiple matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note 2:&lt;/em&gt; If the $A_i$'s do not have the same eigenvectors, then the $v_i$'s will not be exactly eigenvectors of the $A_i$'s.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note 3:&lt;/em&gt;  My intuition is that this will give a set of &lt;strong&gt;mutually orthonormal&lt;/strong&gt; vectors $v_1,\ldots,v_k\in\mathbb{R}^n$, with the property that vector $v_i$ is similar to an eigenvector of $A_i$. But this is &lt;strong&gt;not&lt;/strong&gt; an eigenvalue problem.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastEditorUserId="5129" LastEditDate="2017-10-09T00:47:57.973" LastActivityDate="2017-10-10T14:20:44.880" Title="Multi-matrix orthogonal basis problem" Tags="|linear-algebra|eigenvalues|eigensystem|quadratic-programming|basis-set|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28006" PostTypeId="2" ParentId="10936" CreationDate="2017-10-08T06:38:57.817" Score="1" Body="&lt;p&gt;We gave with a colleague (Richard Hao Zhang) a course at SIGGRAPH 2010 on spectral mesh processing. We give different intuitions of spectral decomposition, as a constrained optimization problem (Rayleigh quotient), as vibrating modes (Chladni plates), as an algebraic problem, as a geometric problem and connections with the heat kernel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference on ACM Digital Library:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dl.acm.org/citation.cfm?id=1837109&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dl.acm.org/citation.cfm?id=1837109&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The slides are here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://alice.loria.fr/WIKI/index.php/Graphite/SpectralMeshProcessing&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/WIKI/index.php/Graphite/SpectralMeshProcessing&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My (older) articles on the subject:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://members.loria.fr/Bruno.Levy/papers/Laplacian_SMI_2006.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://members.loria.fr/Bruno.Levy/papers/Laplacian_SMI_2006.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://hal.inria.fr/inria-00186931/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.inria.fr/inria-00186931/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2017-10-08T06:38:57.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28008" PostTypeId="2" ParentId="27998" CreationDate="2017-10-08T10:34:38.900" Score="2" Body="&lt;p&gt;I would recommend J.T. Oden book on the mathematical foundations of FEM, and the paper of  M. Ainsworth, at least this is what I read to educate myself as a civil engineer. It takes time to follow math, but I strongly recommend this.&lt;/p&gt;&#xA;&lt;p&gt;In essence, if you know the error, you know the solution. Using mathematics, you can estimate error, by calculating upper or lower bound, or both. You can do that before or after you get an approximate solution, i.e. a priory or a posteriorly. Error estimator is better if the lower and upper bound are closer together. Error estimator has to be efficient, in such a way, that for refining solution (f.e. making mesh denser), it converges to exact error. It has to be cheap to calculate, at least faster than uniform mesh refinement and solving the refined problem.&lt;/p&gt;&#xA;&lt;p&gt;It easy to implement finite element, the real difficulty is to derive and implement a good error estimator, it takes a longer time significantly. That is why many develop mix-finite elements since an error evaluator is built into implementation, or apply error indicators.&lt;/p&gt;&#xA;&lt;p&gt;The error indicator is always a posteriori and does not have bounds, it has no proof that will converge to exact error once a sequence of subsequent mesh refinements is analysed. It just tells that something is potentially wrong with the solution. Applying it to drive mesh adaptivity will not guarantee that you would converge.&lt;/p&gt;&#xA;&lt;p&gt;Zienkiewicz-Zhu is error indicator. However, as an example of an error indicator, I like to use material/configurational/Elsheby force. Material force in the homogeneous body and elastic problem is zero. However, if a solution is not exact, the material force calculated on mesh nodes is not zero. So if one calculates material forces on mesh nodes, and those are not zero, he or she knows that is an error in the solution. However, if we try to displace nodes of the mesh, to minimalise that material force we quickly find out that not necessary improve the solution. Displacing mesh nodes in the direction of material forces initially can improve the solution, but ultimately will create a distorted mesh with poor approximation qualities.&lt;/p&gt;&#xA;&lt;p&gt;Recently we recorder video about mix-element and error estimation,&#xA;&lt;a href=&quot;https://youtu.be/T40n76UwKo0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/T40n76UwKo0&lt;/a&gt;. That should answer some questions.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2021-05-27T08:13:48.020" LastActivityDate="2021-05-27T08:13:48.020" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="28009" PostTypeId="1" CreationDate="2017-10-08T14:46:32.587" Score="1" ViewCount="80" Body="&lt;p&gt;We have a 2D electromagnetic field (in the sense that: $E=(E_x,E_y,0)$, $B=(0,0,B_z)$, and all derivatives with respect to $z$ are $0$), and we are considering a system made up of two walls at $x=-b$ and $x=b$ held at fixed potential $\phi(y)=e^{-y}$, with a third conducting wall at $y=0$ held at $\phi(x)=1$. The current in our system is $\vec{j} \propto \nabla \cdot E\left(B \times E+\sigma E \right)$. The initial conditions are that $E=-\nabla \phi$, where $\phi$ is the solution to the Laplace equation $\nabla^2 \phi = 0$, and $B=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yee's scheme is resulting in diverging $E_y$, so what numerical scheme would you suggest be used to solve Maxwell's equations in this case? &lt;/p&gt;&#xA;" OwnerUserId="25597" LastEditorUserId="25597" LastEditDate="2017-10-08T15:21:31.837" LastActivityDate="2017-10-08T15:21:31.837" Title="Numerical scheme to solve Maxwell equations with fixed potential boundaries?" Tags="|finite-difference|computational-physics|electromagnetism|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28011" PostTypeId="2" ParentId="27580" CreationDate="2017-10-08T19:08:56.153" Score="1" Body="&lt;p&gt;Typically your sampled fluid velocity data would 'live' on a specific element of the mesh. For example in 3D, the velocity field 'lives' on the faces of your volume mesh. They are scalar values on faces representing the flux of fluid through that face. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a good explanation of how you calculate differential quantities on the discrete mesh.&#xA;&lt;a href=&quot;http://www.geometry.caltech.edu/pubs/ETKSD07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.geometry.caltech.edu/pubs/ETKSD07.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The flux on a mesh is a 2-form and the gradient of that would be a 3-form, meaning it lives on cells of the mesh. In data it would be 3 numbers stored at the cells representing difference in flux in the x direction, y dir, and z dir. The gradient operator on the mesh is literally a difference flux1-flux2. So your numbers at a cell become [flux(x+)-flux(x-), flux(y+)-flux(y-), flux(z+)-flux(z-)].&#xA;I assume you're on a regular cube mesh... If not, then everything is a little different...&#xA;This is an implicit representation of gradient of u. Which is a diagonal matrix in the basis representing the orientation of the cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presumably your pressure data also lives on cells of the mesh so you can simply multiply by the identity and add them to your grad(u)+grad(u)T. Your result in data would be a 3x3 matrix at each cell of the mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an implicit representation of your stress tensor. I believe the resulting diagonal matrix will represent stresses in the orientation of the cell. The choice of discretizing velocity as fluxes forces your stress matrix to be diagonal. You can rotate the basis of the stress tensor to something else, if needed, then it won't necessarily be diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for some reason the data you recorded for velocity doesn't 'live' on faces, that's still ok. You can do an additional step to convert from wherever your data lives to the faces. For example, if you chose to represent velocity data as (x,y,z) vectors at the cells of the mesh, you can integrate the implied velocity field across the faces, resulting in the flux representation of your velocity field. You would need an interpolation scheme to do the integration, like HBR said. But a super simple method could just be to take the difference between the component of your (x,y,z) velocity in the direction of the cell face, with the component of the (x,y,z) velocity on the other side of the cell face. This is clearly not a smooth interpolation though. There's lots of interpolation schemes.&lt;/p&gt;&#xA;" OwnerUserId="25618" LastActivityDate="2017-10-08T19:08:56.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28013" PostTypeId="2" ParentId="27987" CreationDate="2017-10-09T05:20:44.587" Score="1" Body="&lt;p&gt;What breaks is typically that you can't prove any more that the method actually &lt;em&gt;converges&lt;/em&gt;, as opposed to cycle between active sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that Nocedal and Wright discuss this at least in passing, but I know that examples have been found in which the standard methods indeed cycle when one adds/subtracts more than one constraint at the time. I don't have a reference for that, but you should be able to find them in the literature.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-09T05:20:44.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28014" PostTypeId="2" ParentId="28000" CreationDate="2017-10-09T05:34:13.927" Score="13" Body="&lt;p&gt;The term is commonly used in the following way in the finite element context:&lt;/p&gt;&#xA;&lt;p&gt;Let's assume that &lt;span class=&quot;math-container&quot;&gt;$u\in V$&lt;/span&gt; is the exact solution of the PDE, and &lt;span class=&quot;math-container&quot;&gt;$u_h\in V_h$&lt;/span&gt; the finite element approximation. Then we ask for the convergence order of the norm of the error &lt;span class=&quot;math-container&quot;&gt;$\|u-u_h\|_X$&lt;/span&gt; with regard to some norm &lt;span class=&quot;math-container&quot;&gt;$\|\cdot\|_X$&lt;/span&gt;. We then say that the approximate solution &amp;quot;converges with optimal order&amp;quot; if&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \|u-u_h\|_X \le C \inf_{v_h \in V_h} \|u-v_h\|_X,&#xA;$$&lt;/span&gt;&#xA;i.e., if the error can be bounded by a multiple of the &lt;em&gt;best possible error&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In many cases, e.g. if &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; are the usual piecewise polynomial finite element spaces and &lt;span class=&quot;math-container&quot;&gt;$\|\cdot\|_X$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt; norm, then one can show that for all &lt;span class=&quot;math-container&quot;&gt;$v\in V$&lt;/span&gt; there holds that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;    \inf_{v_h \in V_h} \|v-v_h\|_X \le \|v-I_h v\|_X \le C \inf_{v_h \in V_h} \|v-v_h\|_X,&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$I_h v$&lt;/span&gt; is the interpolant of &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; onto &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt;, and in those cases we say that the numerical solution &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt; of the PDE converges &amp;quot;with optimal order&amp;quot; if&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \|u-u_h\|_X \le C \|u-I_h u\|_X,&#xA;$$&lt;/span&gt;&#xA;i.e., if the numerical solution converges at the same rate as the interpolant.&lt;/p&gt;&#xA;&lt;p&gt;This is, for example, the case for the usual discretizations of the Laplace equation. But it is not true, for example, for many/most discretizations of the advection equation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2022-01-03T08:42:35.577" LastActivityDate="2022-01-03T08:42:35.577" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="28015" PostTypeId="2" ParentId="27580" CreationDate="2017-10-09T07:05:42.717" Score="0" Body="&lt;p&gt;This is done by explicit calculation. You might first reconstruct gradients using least square method, e.g. one described in paper S. Muzaferija D. Gosman - Finite-Volume CFD Procedure and Adaptive Error Control Strategy for Grids of Arbitrary Topology, JCP, 138(2), pp. 766-787, 1997. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999197958538&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;link&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From there, the calculation is straight forward.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastActivityDate="2017-10-09T07:05:42.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28016" PostTypeId="1" CreationDate="2017-10-09T07:26:20.750" Score="0" ViewCount="89" Body="&lt;p&gt;The polyMesh format used in OpenFOAM is an intelligent format for which 'cells' file, i.e. the file giving list of nodes that make a cell, is not needed. The way to reconstruct geometry data such as cell centroids and cell volume is by using divergence theorem an therefore only face data is needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the OpenFOAM is used, the post-processing tool Paraview knows how to build a geometry for visualization given the OpenFOAM case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I want, with my own code that can read and process polyMesh format, to write a Paraview &quot;.vtu&quot; format (a format for unstructured mesh) I need to give cell data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where I found a problem, and still can't find proper resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A side note is that I use CSR format for sparse matrices, which I create using 'owner' and 'neighbour' polyMesh data. From CSR format I can easily find cell neighbour indices in the form of ja(ia(icell)),..,ja(ia(icell+1)-1), except the diagonal element, which is the particular cell. The 'ja', 'ia' are typical CSR sparse matrix format arrays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;To make it precise:&lt;/strong&gt; &#xA;I'm looking for a list of nodes for each cell (or element), given the list of nodes for each cell face (element face), two lists of cells (elements) that share the face, a list for each side of the face, and of course the list of nodal coordinates. The list of nodes for each cell should be in correct order, consistent with faces list.&lt;/p&gt;&#xA;" OwnerUserId="1050" LastEditorUserId="1050" LastEditDate="2017-10-09T09:02:42.840" LastActivityDate="2017-10-09T09:02:42.840" Title="Reconstruction of cells information when given mesh in polyMesh format" Tags="|fluid-dynamics|finite-volume|unstructured-mesh|openfoam|paraview|" AnswerCount="0" CommentCount="3" ClosedDate="2019-06-06T21:40:25.003" ContentLicense="CC BY-SA 3.0" />
  <row Id="28018" PostTypeId="1" AcceptedAnswerId="28031" CreationDate="2017-10-09T17:10:24.827" Score="4" ViewCount="1108" Body="&lt;p&gt;I want to look at time evolution of the density matrices of some, very simple, spin systems, but I am having trouble with my approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use a simple &lt;code&gt;for&lt;/code&gt;-loop time-stepping method, not as ODE solver like &lt;code&gt;ode45&lt;/code&gt; as this is supposed to be a demonstration code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work in Matlab (numerical calculations), so I will include code snippets where relevant.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What I do&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I construct the spin-only hamiltonian for my system in two parts. First part describes the Zeeman interactions of the spin system with an external magnetic field. $S_A$ and $S_B$ are two electrons. $S_C$ is a nucleus:&#xA;$$&#xA;H_{magnetic} = \omega_0^e(S_{Az} + S_{Bz}) + \omega_{0}^nS_{Cz}&#xA;$$&#xA;where $\omega_0^e = B_0*\gamma_e$ is the electron Larmor frequency, and $\omega_0^n$ the Nuclear Larmor frequency. (Arguably the nuclear Zeeman interaction could be ignored, but I've included it for the completeness' sake).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part of the Hamiltonian describes the hyperfine interaction of one of the electrons with the nucleus via the hyperfine interaction:&#xA;$$&#xA;H_{hyperfine} = a\mathbf{S_A}\mathbf{S_C} = a*(S_{Ax}S_{Cx} + S_{Ay}S_{Cy} + S_{Az}A_{Cz})&#xA;$$&#xA;where $a$ is the hyperfine coupling constant. My spin-only hamiltonian is then:&#xA;$$&#xA;H = H_{magnetic} + H_{hyperfine}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my Matlab code this reads as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Pauli spin matrices&#xA;Ix = 0.5 * [0  1;  1   0];&#xA;Iy = 0.5 * [0 -1i; 1i  0];&#xA;Iz = 0.5 * [1  0;  0  -1];&#xA;Ie = eye(2);&#xA;&#xA;% Successive kronecker product of three elements&#xA;tkron = @(x,y,z) kron(kron(x,y),z);&#xA;&#xA;% Spin matrices&#xA;SAx = tkron(Ix,Ie,Ie); SAy = tkron(Iy,Ie,Ie); SAz = tkron(Iz,Ie,Ie);&#xA;SBx = tkron(Ie,Ix,Ie); SBy = tkron(Ie,Iy,Ie); SBz = tkron(Ie,Iz,Ie);&#xA;SCx = tkron(Ie,Ie,Ix); SCy = tkron(Ie,Ie,Iy); SCz = tkron(Ie,Ie,Iz);&#xA;&#xA;% Input constants&#xA;gamma_n = 42.577e3; % Hz/mT&#xA;gamma_e = 1.76e8; % Hz/mT&#xA;hfc = 1.0; % Hyperfine coupling in mT&#xA;B0 = 0.01; % Magnetic field in mT&#xA;&#xA;% Derived constants&#xA;a = gamma_e * hfc;&#xA;omega_0 = gamma_e*B0; % Electron Larmor frequency&#xA;omega0_n = -gamma_n*B0; % Nuclear Larmor frequency&#xA;&#xA;% Hamiltonian&#xA;H_mag = omega_0.*(SAz + SBz) + omega0_n.*SCz;&#xA;H_hyperfine = a*(SAx*SCx + SAy*SCy + SAz*SCz);&#xA;H = H_mag + H_hyperfine;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, because there is no election-electron spin interaction I then define a similarity transform from xyz basis to electronic Singlet-Tiplet (ST) basis such that:&#xA;$$&#xA;H_{ST} = M^{-1}HM&#xA;$$&#xA;which in Matlab is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s2 = 1/sqrt(2);&#xA;%       S  T0  T+  T-&#xA;Me = [ 0  0   1   0&#xA;        s2 s2  0   0&#xA;       -s2 s2  0   0&#xA;        0  0   0   1];&#xA;Mn = eye(2); % Nucleus stays in alpha/beta basis&#xA;M = kron(Me,Mn);&#xA;&#xA;% Change to ST basis&#xA;Hst = M'*H*M; % M is unitary therefore use transpose&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I then start my spin density in a pure singlet state and use the integrated form one the Liouville von Neumann equation&#xA;$$&#xA;\hat\rho(t) = e^{-I\hat{H}t}\hat\rho(0)e^{I\hat{H}t}&#xA;$$&#xA;to iterate through time points as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rh0 = zeros(8);&#xA;rh0(1,1) = 0.5;&#xA;rh0(2,2) = 0.5;&#xA;T = linspace(0,1e-6,1000); % 1 us of spin evolution&#xA;for i = 1:length(T);&#xA;    t = T(i);&#xA;    e = 1i*H*t;&#xA;    em = expm(-e);&#xA;    ep = expm(e)';&#xA;    rh_t = em*rh0*ep; % Propagate rh0 to time t&#xA;    Sd(i) = rh_t(1,1) + rh_t(2,2);&#xA;end&#xA;plot(T,Sd);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Well. It doesn't work. More specifically; I'd expect the singlet density &lt;code&gt;Sd&lt;/code&gt; to oscillate between 1 and 0.25 due to S$\leftrightarrow$T interconversion. The results I get are indeed oscillatory, but between +1 and -1. Furthermore the trace of the density matrix is not always 1.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Questions&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can you address any of the problems mentioned above directly?&lt;/li&gt;&#xA;&lt;li&gt;Can you reference me to a piece of literature (or, even better, code) which does this step-by-step propagation of the density matrix with the LvN equation?&lt;/li&gt;&#xA;&lt;li&gt;If this is not a possible way of doing density matrix propagation, can you explain to me why? (Please remember that I realize this is not the optimal way of doing spin density simulations. I &lt;em&gt;specifically&lt;/em&gt; want to do a simulation this very way)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25624" OwnerDisplayName="MarcinKonowalczyk" LastEditorUserId="25624" LastEditDate="2017-10-09T20:14:23.753" LastActivityDate="2017-10-14T19:17:15.927" Title="Numerical propagation of a density matrix using Liouville von Neumann equation" Tags="|matlab|computational-physics|software|quantum-mechanics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28019" PostTypeId="1" AcceptedAnswerId="28078" CreationDate="2017-10-09T20:52:15.487" Score="1" ViewCount="206" Body="&lt;p&gt;I am trying to locate suitable computational algorithms for a optimization problem that requires repeated solution of transient 2D incompressible Euler equation on a 2D domain (say rectangular).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is whether an off-the-shelf PDE solver can solve incompressible Euler equations on 2D domains in a robust manner, or if there are any customized solvers for 2D incompressible Euler.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="25631" LastEditorUserId="25631" LastEditDate="2017-10-12T14:00:21.457" LastActivityDate="2017-10-16T16:12:19.057" Title="Robust/Tested Solver for incompressible 2D Euler (Fluid dynamics) Equation" Tags="|optimization|fluid-dynamics|optimal-control|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28022" PostTypeId="1" CreationDate="2017-10-10T10:37:08.717" Score="2" ViewCount="195" Body="&lt;p&gt;Background: The solution space of original problem (which requires a fine enough mesh to resolve the microstructure) can be split into a macroscale solution space and microscale solution space. This can be based on variational Multiscale Method (VMM) proposed by Hughes in 1998. By doing this, we can derive a multiscale formulation for the original heterogeneous problem. The final goal of most (multiscale) homogenisation based methods is to gain a smooth solution at the macroscale, but it also contains some information from the sub-scale. We can consider such a original problem a simple elasticity or Poisson's equation, which is bilinear form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I was reading a mechanics paper on two scale adaptive FE analysis for application in multicsale setting, I came across a sentence that the continuous space (i.e. the macroscale space) is the space of all fucnctions inside each RVE, that can reproduce the &lt;strong&gt;&lt;em&gt;trace of functions&lt;/em&gt;&lt;/strong&gt; in original solution space on the boundary of RVE's. &#xA;So my question is what &lt;strong&gt;&lt;em&gt;trace of functions&lt;/em&gt;&lt;/strong&gt; means in PDEs. To try solving this myself, I did some online Google search and came out a Wiki link explaining trace operator  &lt;a href=&quot;https://en.wikipedia.org/wiki/Trace_operator&quot; rel=&quot;nofollow noreferrer&quot;&gt;trace operator&lt;/a&gt;, where an application into Poisson's equation is discussed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to incomplete knowledge on functional analysis and measure theory, I cannot quite understand what the Wiki page is trying to describe? I attached a snapshot with my doubts highlighted in red. How to understand them? Also, you can elaborate exposition on trace of functions on boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1eHK6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1eHK6.jpg&quot; alt=&quot;https://en.wikipedia.org/wiki/Trace_operator&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24268" LastActivityDate="2017-10-10T14:23:53.730" Title="Some questions on Trace (operators) on the boundary in the context of PDEs" Tags="|finite-element|differential-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28023" PostTypeId="2" ParentId="27998" CreationDate="2017-10-10T14:16:50.310" Score="4" Body="&lt;p&gt;In general, for a finite element approximation $u_h$ of the exact (but unknown) solution $u$ of a partial differential equation, it would of course be nice if we could compute something like $\|u-u_h\|_K$ for each cell $K$ of the finite element mesh -- in other words, a measure for &lt;em&gt;how wrong&lt;/em&gt; the approximate solution is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But we can't do this because we don't know the exact solution $u$. So, over the years, people have come up with &lt;em&gt;estimates&lt;/em&gt; of the error, i.e., for each cell we have a quantity $\eta_K$ so that $\|u-u_h\|_K \le \eta_K$. Such quantities $\eta_K$ -- called &lt;em&gt;error estimators&lt;/em&gt; -- can be derived for a number of equations including the Laplace equation, elasticity, Stokes, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in order to do this, one typically needs some structure in the equations and not all equations have that. So, for example for the Navier-Stokes equations or other equations with nonlinearity, it's often very difficult to derive these estimators $\eta_K$ and in fact, for most equations, we may have an idea how such an $\eta_K$ would look like, but we can't prove that it's an upper bound for the actual error. This may be because our idea is wrong and the form we have assumed is not in fact an upper bound. Or -- and that's probably the case for most equations -- our idea is correct, but we lack the mathematical techniques to &lt;em&gt;prove&lt;/em&gt; that it's an upper bound; in other words, it is true that for our assumed $\eta_K$ there holds $\|u-u_h\|_K \le \eta_K$, but cannot &lt;em&gt;prove&lt;/em&gt; that that inequality holds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In such cases, we call $\eta_K$ an &lt;em&gt;error indicator&lt;/em&gt;: it is not strictly speaking an &lt;em&gt;estimator&lt;/em&gt;, but it is still indicative of the size of the error, as verified in numerical experiments. We consequently cannot use it to attach a &lt;em&gt;guarantee&lt;/em&gt; to a computation that states that the error is less than, say, 5%, but we can still use it to refine the mesh, and maybe to do things such as recovery methods to get an even better approximation $\tilde u_h$. In practice, refining the mesh is probably the most common use: you want to reduce the size of cells in areas where the error estimate suggests that the error is large, and keep the mesh coarse where the error estimate suggests that the error is already small.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-10T14:16:50.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28025" PostTypeId="2" ParentId="28022" CreationDate="2017-10-10T14:23:53.730" Score="6" Body="&lt;p&gt;I think the quote from Wikipedia is misleading (and I should put that on my list of things to fix :-) ). You can think of the trace of a function $u: \Omega \rightarrow {\mathbb R}$ that lives on a domain $\Omega$ as that function $g: \partial\Omega \rightarrow {\mathbb R}$ that lives on the &lt;em&gt;boundary&lt;/em&gt; of $\Omega$ and so that $u(\mathbf x)=g(\mathbf x)$ for all points $\mathbf x$ on the boundary $\partial \Omega$. In other words, it is the &lt;em&gt;restriction&lt;/em&gt; of $u$ to the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things are a bit more complicated than that because $u$ may be discontinuous at individual locations, and then it's not quite obvious how one would define $g$, but fundamentally the description above is what you should think of when you think of the trace.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-10T14:23:53.730" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28029" PostTypeId="1" CreationDate="2017-10-11T12:33:17.127" Score="1" ViewCount="205" Body="&lt;p&gt;Calculus of variations problems are generally cast in in the following simple form: find $u(t)$ that satisfies some boundary conditions and minimises&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F[u] = \int_{t=0}^{t=t_f} f(u(t),u'(t),t) dt.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have run into a far more general problem. I have a functional $F[u]$ that is substantially more complicated. Given $u$ (or in this case the set of functions $u^i(\mathbf{x})$ where $\mathbf{x}\in\mathbb{R}^n$) I solve a pde which depends on these functions, and the solution is analysed to give the value of the functional $F$. Given a set of functions $u^i(\mathbf{x})$, I can numerically evaluate $F[u^i]$ very efficiently. However, I want to explore the function space to find the set of $u^i$ that maximises $F$. To complicate matters, I have further constraints on the $u^i$ I can choose: for instance, I require a particular path integral of $u^i$ to equal a certain value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the methods I have found in the calculus of variations seem to assume, in one way or another, that the functional $F$ is of the simple form outlined above. Do more general numerical methods exist? I'm looking for a method or algorithm that requires nothing more than the ability to numerically evaluate the functional for any given function, and from that will search the function space and converge on an optimal solution. The ability to place (ideally arbitrary) constrains on $u^i$ would be an added bonus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I ask for too much. I'd appreciate any help or advice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6799" LastActivityDate="2017-10-11T12:33:17.127" Title="General-purpose Numerical Calculus of Variations Problem" Tags="|variational-calculus|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28030" PostTypeId="1" CreationDate="2017-10-11T05:55:08.387" Score="1" ViewCount="211" Body="&lt;p&gt;I haven't been doing math in years and I'm facing the following problem. I'm trying to implement in Java a linear regression under a set of inequality constraints. Sorry in advance for all the approximations while trying to formulate this problem...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, I have a linear function $f_\Theta:R^n-&amp;gt;R$ defined as such: $f_\Theta(X) = \theta_0 + \theta_1x_1+ \theta_0x_2+... \theta_nx_n$, with  $\Theta$ a (n+1) vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My aim is to $\min_\Theta \sum_{i}||f_\Theta(X_i)-Y_i||^2$ under the constraint that $f_\Theta(X_i)-Y_i &amp;lt; 0$ for a maximum of $i$ (I wish I could avoid to take into account &quot;obvious&quot; outliers in my observations)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, bottom line, I'm looking for the linear regression that will act as the &quot;best fit&quot; providing a lower bound for most/all the observations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do that ? I've been looking for a Java Library that would implement such a thing. I known the &lt;a href=&quot;http://commons.apache.org/proper/commons-math/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apache Common Math&lt;/a&gt; package and I also found this &lt;a href=&quot;https://www.ee.ucl.ac.uk/~mflanaga/java/Regression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flanagan&lt;/a&gt; library, but it does not seem to address this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help greatly appreciated. Thanks&lt;/p&gt;&#xA;" OwnerUserId="25653" OwnerDisplayName="BlackLabrador" LastActivityDate="2017-10-11T12:59:18.720" Title="Linear regression with inequality constraint in Java" Tags="|algorithms|optimization|linear-algebra|java|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28031" PostTypeId="2" ParentId="28018" CreationDate="2017-10-11T17:17:03.227" Score="2" Body="&lt;p&gt;Ok, I figured it out. There were a couple of problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;My similarity transform was bad (this was the major issue)&lt;/li&gt;&#xA;&lt;li&gt;I should not have transposed the &lt;code&gt;ep&lt;/code&gt; propagator&lt;/li&gt;&#xA;&lt;li&gt;My nuclear gyromagnetic ratio was wrong (although that ended up not really mattering due to the nuclear Zeeman interaction being small in comparison to other parts of the Hamiltonian)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have fixed the problem by doing the (correct) similarity transform first, and then working in ST basis throughout. What helped was to define the singlet and triplet projection operators (question 3d from &lt;a href=&quot;http://www.physics.brocku.ca/Courses/4P52/Assignments/a2_07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; assignment) because even if I end up transforming to some cookie basis, as long as I'm consistent throughout (i.e. also transforming the projection operator), I end up with the correct result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway; &lt;a href=&quot;https://uk.mathworks.com/matlabcentral/fileexchange/64703-liouville-von-neumann-simulation-by-density-matrix-propagation&quot; rel=&quot;nofollow noreferrer&quot;&gt;HERE&lt;/a&gt; is my code, and &lt;a href=&quot;http://www.tandfonline.com/doi/abs/10.1080/00268979809483134&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is why I wanted to understand that subject better. I hope this helps someone at some point.&lt;/p&gt;&#xA;" OwnerUserId="25624" LastEditorUserId="25624" LastEditDate="2017-10-14T19:17:15.927" LastActivityDate="2017-10-14T19:17:15.927" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28032" PostTypeId="1" CreationDate="2017-10-12T02:41:56.120" Score="1" ViewCount="58" Body="&lt;p&gt;Suppose I have a nonlinear second order Cauchy PDE $\dfrac{\partial p(x,t)}{\partial t}=N(p(x,t))$, where $N:L^2(\mathbb{R})\rightarrow L^2(\mathbb{R})$, and a known fixed point $u(x)$. Mathematically, the only boundary condition known is that any solution $p$ decays to 0 as ${x\rightarrow\pm \infty}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From physical reasoning however, it is known that $u$ decays rapidly away from $x=0$, and hence, I am able to solve for $u$ numerically on a finite domain by prescribing zero Dirichlet boundary conditions on a finite domain around 0 (i.e, on $[-L,L]$, where $L$ is chosen judiciously).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to linearize $N$ around $u$, and find eigenvalues of the resulting linear operator $L_{u}$. I have a dilemma as to what boundary conditions should I impose on the eigenfunctions to be able to solve this problem numerically on a finite domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any standard ways to do this ? Is there a way to justify same boundary conditions as $u$.&lt;/p&gt;&#xA;" OwnerUserId="25631" LastEditorUserId="25631" LastEditDate="2017-10-12T03:21:57.050" LastActivityDate="2017-10-12T03:21:57.050" Title="Appropriately handling boundary conditions in a PDE eigenvalue problem" Tags="|pde|boundary-conditions|eigenvalues|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28033" PostTypeId="1" CreationDate="2017-10-12T05:49:44.180" Score="6" ViewCount="648" Body="&lt;p&gt;Let me illustrate the issue with an simplified example. Suppose we want to solve the following problem with finite difference method (FDM):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u(t,x)}{\partial t}=\frac{\partial^2 u(t,x)}{\partial x^2}$$&#xA;$$u(0,x)=x(1-x),\ u(t,0)=0$$&#xA;$$t\in[0,1],\ x\in[0,1] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One boundary condition (b.c.) is missing, but let's ignore it for a while and discretize the equation, initial condition (i.c.) and b.c. with difference formula. Here we use 2nd order difference formula. Clearly if we only use&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f' (x_i)\simeq \frac{f (x_{i}+h)-f (x_{i}-h)}{2 h}$$&#xA;$$f'' (x_i)\simeq\frac{f (x_{i}-h)-2 f (x_i)+f (x_{i}+h)}{h^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we cannot generate equation for $t=1,\ x=1$, so we need one-sided formula (If you're not familiar with one-sided formula, start from page 6 of &lt;a href=&quot;https://disciplinas.stoa.usp.br/pluginfile.php/41896/mod_resource/content/1/LeVeque%20Finite%20Diff.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book&lt;/a&gt; ):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f' (x_n)\simeq \frac{f (x_{n}-2h)-4 f (x_{n}-h)+3 f (x_n)}{2 h}$$&#xA;$$f'' (x_n)\simeq \frac{-f (x_{n}-3h)+4 f (x_{n}-2h)-5f (x_{n}-h)+2 f (x_{n})}{h^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK, now here comes the problem: we surprisingly find that, even if b.c. is not enough, we still obtain a closed algebraic equation system! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One may suspect the system will be kind of ill-posed e.g. &quot;This system won't be solvable!&quot; or &quot;This system will have infinite many solutions!&quot; etc., but once again, surprisingly, further check shows the system determines a unique solution and seems to be stable i.e. with smaller grid size or higher order difference formula, the solution doesn't have significant change. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to explain this solution? If a hidden b.c. has been imposed, what's it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the very least, if the solution is really nothing more than numeric error, I'd like to know the &lt;strong&gt;exact&lt;/strong&gt; cause of the error.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here's the source code for solving the problem above in &lt;em&gt;Mathematica&lt;/em&gt;. I've used &lt;a href=&quot;http://mathematica.stackexchange.com/a/127997/1871&quot;&gt;&lt;code&gt;pdetoae&lt;/code&gt;&lt;/a&gt; for the generation of difference formula:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eq = D[u[t, x], t] == D[u[t, x], x, x];&#xA;ic = u[0, x] == x (1 - x);&#xA;bc = u[t, 0] == 0;&#xA;points = 25;&#xA;grid = tgrid = Array[# &amp;amp;, points, {0, 1}];&#xA;ptoafunc = pdetoae[u[t, x], {tgrid, grid}, 2];&#xA;ae = Rest /@ (ptoafunc[eq] // Rest);&#xA;aeic = ptoafunc[ic] // Rest;&#xA;aebc = ptoafunc[bc];&#xA;var = Outer[u, tgrid, grid] // Flatten;&#xA;{b, m} = CoefficientArrays[{ae, aeic, aebc} // Flatten, var];&#xA;solarray = Partition[LinearSolve[N@m, -b], points];&#xA;solfunc = ListInterpolation[solarray, {tgrid, grid}];&#xA;&#xA;style = Style[#, 16] &amp;amp;;&#xA;Plot3D[solfunc[t, x], {t, 0, 1}, {x, 0, 1}, AxesLabel -&amp;gt; style /@ {&quot;t&quot;, &quot;x&quot;}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/2v0bP.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Plot[solfunc[t, #] &amp;amp; /@ grid // Evaluate, {t, 0, 1}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/LDMhs.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is the norm of solution $\sqrt{\int_0^1 u(t,x)^2 \, dx}$:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;norm[t_?NumericQ, order_: 2, func_: solfunc] := &#xA; Power[NIntegrate[func[t, x]^order, {x, 0, 1}, &#xA;   Method -&amp;gt; {Automatic, &quot;SymbolicProcessing&quot; -&amp;gt; 0}], 1/order]&#xA;&#xA;Plot[norm[t], {t, 0, 1}, AxesLabel -&amp;gt; {t, norm}, PlotRange -&amp;gt; All]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GRLLW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GRLLW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Remark&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;Notice that the value of i.c. is used at the corner $t=0,\ x=1$.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;code&gt;LinearSolve&lt;/code&gt; is not a iterative solver but a symbolic solver for linear algebraic equation system.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also, it's actually enough to reproduce the issue by discretizing in $x$ direction only, and the resulting ordinary differential equation (ODE) system can be solved &lt;strong&gt;symbolically&lt;/strong&gt;. The following is the corresponding &lt;em&gt;Mathematica&lt;/em&gt; code. I've used &lt;a href=&quot;http://mathematica.stackexchange.com/a/127997/1871&quot;&gt;&lt;code&gt;pdetoode&lt;/code&gt;&lt;/a&gt; for the generation of ODE system:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eq = D[u[t, x], t] == D[u[t, x], x, x];&#xA;ic = u[0, x] == x (1 - x);&#xA;bc = u[t, 0] == 0;&#xA;points = 4;&#xA;grid = Array[# &amp;amp;, points, {0, 1}];&#xA;ptoofunc = pdetoode[u[t, x], t, grid, 2];&#xA;ode = ptoofunc[eq] // Rest;&#xA;odeic = ptoofunc[ic] // Rest;&#xA;odebc = ptoofunc[bc];&#xA;var = u /@ grid;&#xA;&#xA;asol = DSolve[{ode, odeic, odebc}, var[t] // Through, t] // Simplify // First&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{array}{l}&#xA; u(0)(t)=0 \\&#xA; u\left(\frac{1}{3}\right)(t)=\frac{1}{18} \left(-18 t+e^{-18 t}+3\right) \\&#xA; u\left(\frac{2}{3}\right)(t)=\frac{2}{9}-2 t \\&#xA; u(1)(t)=-3 t-\frac{e^{-18 t}}{18}+\frac{1}{18} \\&#xA;\end{array}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;style = Style[#, 16] &amp;amp;;&#xA;ParametricPlot3D[&#xA; Flatten@{t, #} &amp;amp; /@ ({grid, asol[[All, -1]]}\[Transpose]) // Evaluate, {t, 0, 1}, &#xA; BoxRatios -&amp;gt; {1, 1, 0.4}, AxesLabel -&amp;gt; style /@ {&quot;t&quot;, &quot;x&quot;, &quot;u&quot;}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/hKtPs.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only used 4 grid points, or else the symbolic calculation will be too slow, but as you can see the result agrees well with the FDM solution.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For completeness, here are implementations in some other programming languages. Notice I've only discretized $x$ in the following.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Maple&lt;/h1&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;points := 24: y[0](t):=0: h := 1/points:  &#xA;eq := seq((D(y[i]))(t) = (y[i-1](t)-2*y[i](t)+y[i+1](t))/h^2, i = 1 .. points-1), &#xA;          (D(y[points]))(t) = &#xA;            (-y[points-3](t)+4*y[points-2](t)-5*y[points-1](t)+2*y[points](t))/h^2:  &#xA;ic := seq(y[points*x](0) = -x^2+x, x = h .. 1, h):  &#xA;var := [seq([t, y[i](t)], i = 1 .. points)]:  &#xA;p := dsolve([eq, ic], numeric, range = 0 .. 1):  &#xA;with(plots):  &#xA;odeplot(p, var, size = [default, .618]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/18MFH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/18MFH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice &lt;em&gt;Maple&lt;/em&gt; can also calculate the symbolic solution of the sytem with the following line (remember to choose a small &lt;code&gt;points&lt;/code&gt; first):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dsolve([eq, ic]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The solution is consistent with that of &lt;em&gt;Mathematica&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Octave&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The code isn't tested in, but should be compatible with MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice $u(t,0)=0$ isn't plotted in the resulting graph.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;points=24;&#xA;dx=1/points;&#xA;one=ones(points,1);&#xA;mat1=spdiags([one -2*one one],[-1 0 1],points-1,points);&#xA;mat2=zeros(1,points);&#xA;mat2(points-3:points)=[-1 4 -5 2];&#xA;mat=[mat1;mat2]/dx^2;&#xA;span=dx:dx:1;&#xA;[T,Y]=ode45(@(t,y) mat*y,[0,1],span.*(1-span));&#xA;plot(T,Y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hnj95.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hnj95.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Python&lt;/h1&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import zeros,linspace&#xA;from scipy.integrate import odeint&#xA;&#xA;points=24;h=1/points;&#xA;&#xA;def func(y,t0):&#xA;    dydt=zeros(points+1)&#xA;    dydt[0]=0&#xA;    for i in range(1,points):&#xA;        dydt[i]=(y[i-1]-2*y[i]+y[i+1])/h**2&#xA;    dydt[points]=(-y[points-3]+4*y[points-2]-5*y[points-1]+2*y[points])/h**2&#xA;    return dydt&#xA;&#xA;y0=[x*h-x*x*h*h for x in range(0,points+1)]&#xA;t=linspace(0,1,200)&#xA;sol=odeint(func,y0,t)&#xA;&#xA;from matplotlib.pyplot import plot,show&#xA;&#xA;plot(t,sol)&#xA;show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Cm1PF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Cm1PF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, all these implementations lead to the same solution.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Background Information&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;OK, let me explain a bit about why I insist on finding the meaning of this strange difference scheme. Actually what I've reproduced in this question is the behavior of &lt;em&gt;Mathematica&lt;/em&gt; function &lt;code&gt;NDSolve&lt;/code&gt; when insufficient b.c. is added to it for solving time dependent PDE. For more information, check &lt;a href=&quot;https://mathematica.stackexchange.com/q/73961/1871&quot;&gt;this post&lt;/a&gt;. As I wrote there:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I know the best countermeasure is to add the missing boundary&#xA;  condition, I'm just curious. Anyway, if the output with insufficient&#xA;  boundary condition is completely meaningless, why doesn't &lt;code&gt;NDSolve&lt;/code&gt;&#xA;  simply stop calculating and return the input?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, perhaps we'll have to admit the solution is really meaningless in the end, but I think this should be the last thing to do.&lt;/p&gt;&#xA;" OwnerUserId="5331" LastEditorUserId="5331" LastEditDate="2017-10-21T12:32:33.477" LastActivityDate="2017-10-21T21:12:05.873" Title="Why do I still obtain a unique solution with one-sided formula when b.c. isn't enough?" Tags="|pde|finite-difference|" AnswerCount="4" CommentCount="37" ContentLicense="CC BY-SA 3.0" />
  <row Id="28034" PostTypeId="1" CreationDate="2017-10-11T08:55:43.983" Score="2" ViewCount="34" Body="&lt;p&gt;I am working on developing a Dissipative Particle Dynamics (DPD) model of a colloid in a bulk fluid. I essentially want to ensure that the perturbations of my meshed colloid are completely uncoupled from the duplicated copies of itself that are present as a result of using periodic boundary condition. This includes any hierarchical interactions. For example, if fluid particles are disturbed from interacting with  a duplicated version of the colloid, the propagation of these interactions should dissipate out enough such that by the time this propagation reaches the colloid of interest, that it essentially has a negligible effect on the behavior of the colloid. Unfortunately I myself am not certain about what 'negligible' means, but perhaps someone with more experience in the field could offer some advice on that point in particular. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a starting point, I would assume the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L-D &amp;gt; r_c$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L$ is the box length, $D$ is the diameter of the colloid and $r_c$ is the largest cutoff for the bead-bead interactions forces. This ensures that the colloid will not interact with itself as a direct result of colloid-colloid interactions, but does not ensure the prevention of the hierarchical interactions I described earlier.  &lt;/p&gt;&#xA;" OwnerUserId="25660" OwnerDisplayName="cwm5412" LastEditorUserId="25660" LastEditDate="2017-10-12T14:58:22.960" LastActivityDate="2017-10-12T14:58:22.960" Title="Minimum box size in order for an object in the center to not 'see' duplications of itself under periodic boundary conditions?" Tags="|boundary-conditions|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28035" PostTypeId="1" AcceptedAnswerId="30477" CreationDate="2017-10-12T13:56:35.387" Score="2" ViewCount="1679" Body="&lt;p&gt;Can the following strictly convex optimization problem be reformulated into a standard form that is also a strictly convex problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;&amp;amp;\text{Minimize }\frac{1}{2} x^T Q x + a^T x + c^T|x| \\&#xA;&amp;amp;\text{subject to } Gx \leq b&#xA;\end{align}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is positive definite matrix, &lt;span class=&quot;math-container&quot;&gt;$c^T \gt 0$&lt;/span&gt; and the rest are vectors (assume standard QP notation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Standard form is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;&amp;amp;\text{Minimize }\frac{1}{2} x^T A x + b^Tx \\&#xA;&amp;amp;\text{subject to } Hx \leq d&#xA;\end{align}$$&lt;/span&gt; &#xA;where I am hoping &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is positive definite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Background&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm expecting the standard form to also have a positive definite matrix in the quadratic term (and maybe this is an incorrect assumption, which would explain why I'm struggling!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many reference that throw out suggestions like &quot;let &lt;span class=&quot;math-container&quot;&gt;$x = y^+ - y^-$&lt;/span&gt;&quot; or &quot;replace &lt;span class=&quot;math-container&quot;&gt;$|x|$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and solve over &lt;span class=&quot;math-container&quot;&gt;$x,y$&lt;/span&gt;&quot;. I haven't come across any reference that explicitly states the standard-form matrices - but it is fairly easy using such hints to formulate a problem in the standard form that gives the correct solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working under the assumption that if the original problem had &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; variables, then the auxiliary variables in the standard formulation mean we need a &lt;span class=&quot;math-container&quot;&gt;$2N \times 2N$&lt;/span&gt; matrix in the quadratic term that yields an equivalent objective. However- no matter how I approach it I can't find an equivalent quadratic term that is also positive definite. I'm hoping to find such a formulation so I can use Pythons &lt;code&gt;quadprog&lt;/code&gt; optimizer (which is the Goldfarb/Idnani dual algorithm) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Goldfarb, D.; Idnani, A.&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1007/BF02591962&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;A numerically stable dual method for solving strictly convex quadratic programs&lt;/strong&gt;&lt;/a&gt;, Math. Program. 27, 1-33 (1983). &lt;a href=&quot;https://zbmath.org/?q=an:0537.90081&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL0537.90081&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="25658" LastEditDate="2018-11-05T02:26:38.510" LastActivityDate="2022-02-23T15:42:52.280" Title="Reformulate a strictly convex QP problem containing absolute value term" Tags="|convex-optimization|quadratic-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="28036" PostTypeId="2" ParentId="28033" CreationDate="2017-10-12T16:00:07.853" Score="1" Body="&lt;p&gt;Following the same procedure that for the first problem below, one would have the operator you proposed:&#xA;$$ \tilde{D}=\frac{1}{\Delta x^2}\begin{bmatrix}-2&amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\ 1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \ddots&amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\ \vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\ 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0\\ 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1\\ 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 4 &amp;amp; -5 &amp;amp; 2\end{bmatrix} \tag{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that your matrix $\tilde{D}$ is now applied to the vector $\vec{u} = (u_1,...,u_N)^T$. Note that $\tilde{D}$ has the BC imposed on the first but on the last row there is not any Bc as it should be. Note also that the last row: $r_N$ can be obtained with the following operation:&#xA;$$r_N = 2r_{N-1} -r_{N-2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore $\tilde{D}$ is non invertible and your problem does not have any definite solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This was for the first problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us begin defining the following problem (without any BC as you do):&#xA; $$ D\,u(x) = f(x) \qquad x\in[0,L] \tag{P}$$&#xA;where $D$ is the operator derivative:&#xA;$$D = \frac{\partial }{\partial x} \tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that acts on some continuous function $u(x)$ in order to provide the derivative: $u_x = D\,u(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discrete version of $D$, namely $\tilde{D}$ is the matrix:&#xA;$$ \tilde{D}=\frac{1}{\Delta x}\begin{bmatrix}-1&amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots\\ 0 &amp;amp; -1 &amp;amp; 1 &amp;amp; 0 &amp;amp; \ddots\\ \vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots \end{bmatrix} \tag{2}$$&#xA;that acts on the discrete version of the continuous function: $\vec{u}=(u_0,u_1,...,u_N)^{T}$, to provide its discrete derivative: $\vec{u}_x=\tilde{D}\,\vec{u}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that $\tilde{D}$ is not a square matrix, and the discrete version of $(P)$, i.e. the equation $\tilde{D}\,\vec{u}=\vec{f}$ does not make any sense. This also occurs to $(1)$ where the equation $D\,u=f$ suffers from the same.&#xA;The problem is that due to the fact that we have more variables than equations, the operator $\tilde{D}$ cannot be inverted and therefore the solution exists up to a constant. It can be shown that if $\vec{u}_0$ a the solution of the problem $(P)$ in its discrete form, any vector $\vec{v}$ such as $\vec{v} =\lambda (1,...,1)^T$ is also a solution. This means that the solution of this problem would be:&#xA;$$\vec{u} = \vec{u}_0 + \lambda (1,...,1)^T$$&#xA;where $\lambda$ is an arbitrary constant. What is more, in the continuous case it is the same!! I mean, this results in continuum reads:&#xA;$$u(x) = u_0(x) + \lambda$$&#xA;where $\lambda$ is again an arbitrary constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now guess where do we set the value of $\lambda$ from!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To set $\lambda$ only ONE boundary condition is needed to be defined, i.e. $u(0) = 0$ or $u(L) = 0$ for $(1)$ or $u_0=0$ or $u_N=0$ for $(2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, is theory. The operator $\tilde{D}$, in practice, is defined WITH the BC. For example, if the BC considered is $u_0 = 0$ the operator $\tilde{D}$ reads:&#xA;$$ \tilde{D}=\frac{1}{\Delta x}\begin{bmatrix} 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots\\ -1 &amp;amp; 1 &amp;amp; 0 &amp;amp; \dots\\ \vdots &amp;amp; \vdots &amp;amp; \vdots  &amp;amp; \ddots \end{bmatrix} $$&#xA;And it would be applied to the vector of unknowns: $\vec{u} = (u_1,...,u_n)^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This reasoning, can be extended to any order and derivative. Try the operator that approximates the second derivative w.r.t. $x$. You will be convinced of the need for two BC's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result, what you propose: applying the forward and backward  discretisation on the first and end nodes respectively a proper BC on one of them: must lead to a non-solvable system.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2017-10-13T07:07:43.993" LastActivityDate="2017-10-13T07:07:43.993" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28037" PostTypeId="1" AcceptedAnswerId="28039" CreationDate="2017-10-13T00:17:19.723" Score="3" ViewCount="67" Body="&lt;p&gt;I am investigating various methods for adaptive-step integration of stochastic differential equations and trying to implement them. All of the papers that I've seen (e.g. &lt;a href=&quot;http://doi.org/10.1016/j.cam.2003.05.001&quot; rel=&quot;nofollow noreferrer&quot;&gt;H. Lamba, J. Comp. App. Math. 161(2), 417–430 (2003)&lt;/a&gt;) mention in some form that one has to always preserve the sampled values of the Wiener process. In other words, if some steps were rejected, one still has to go through all the sampled values that are located later in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose that you are at time $t_0$ and tried to step to $t_1$, sampling the Wiener increment $dW_1$ for that. The step turned out to have an unacceptable error and was rejected. Now you try to make a step to $t_2 &amp;lt; t_1$. The proper procedure, it appears, is to sample an intermediate increment $dW_2$ for the step from $t_0$ to $t_2$ based on $dW_1$ (the exact formula is not important here), and take the increment for the step from $t_2$ to $t_1$ to be $dW_1 - dW_2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose that this step was rejected as well. Repeating the procedure above, you now have three increments, $dW_3$, $dW_2 - dW_3$ and $dW_1 - dW_2$ for the steps from $t_0$ to $t_3$, then to $t_2$ and then to $t_1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This part I have not seen explained, although it seems that all the authors imply that if your step to $t_3$ succeeds, your next step must not go past $t_2$ and should use the sampled increment $dW_2$ for that part. If you follow that logic, in the implementation of the integrator you will have to maintain a dynamic list of sampled increments, which increases the memory footprint (in an unpredictable way, too). This supposedly avoids bias in the sampled Wiener process. Let's call it &lt;strong&gt;approach A&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to replace it with a simpler logic, where only the furthest sample in time ($dW_1$) is preserved. This way, in the situation above, after you sample $dW_3$, you forget $dW_2$, and for your next step the sampled increment only depends on $dW_1$. Let's call it &lt;strong&gt;approach B&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for the long explanation, now to my question. Supposedly my simplified logic should introduce bias in the solutions. The problem is that I haven't been able to detect it in any of the tests that I tried. Do you have any ideas of how to design a test that will be able to distinguish approach A from approach B?&lt;/p&gt;&#xA;" OwnerUserId="7070" LastActivityDate="2017-10-13T05:06:24.580" Title="Testing Wiener process splitting in adaptive-step SDE integrators" Tags="|testing|random-sampling|stochastic-ode|adaptive-timestepping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28039" PostTypeId="2" ParentId="28037" CreationDate="2017-10-13T04:25:54.790" Score="2" Body="&lt;p&gt;I discuss the method you describe in more detail in &lt;a href=&quot;http://chrisrackauckas.com/assets/Papers/ChrisRackauckas-AdaptiveSRK.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper (Rackauckas and Nie 2017)&lt;/a&gt; as RSwM2. In that paper I am ever so slightly able to detect that it's sometimes doing something wrong, but since it only has issues with re-rejections it isn't that big of a deal. Those 3 methods (RSwM1, RSwM2, RSwM3) are now the basis of &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqNoiseProcess.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffEqNoiseProcess.jl&lt;/a&gt; and subsequently are the methods which build the adaptive timestepping in &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Needless to say, I have a lot better tests now than I did back when the paper was published. The first thing to note is that this is a lot less of an issue if you switch to using PI-controlled adaptive timestepping (see Hairer II for details) as this reduces the chance of re-rejection tremendously. Also tweaking the adaptivity parameters helps as well. But, a very good test is to use adaptive timestepping to recapitulate the properties of the Brownian Bridge. Using the notation from the paper, if you start by putting a value into the future information stack $S_1$ at the final timepoint then you have to hit that value. So this means that starting with a value in $S_1$ should make the noise process be a Brownian Bridge. Since you know properties like the mean and variance of the Brownian Bridge at every point in time, it's easy to run a large number of simulations Monte Carlo and test for these properties in a very sensitive way. I have this test &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqNoiseProcess.jl/blob/master/test/bridge_test.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;in my continuous integration testing suite&lt;/a&gt; and have found that it is super sensitive to getting the algorithm correct. Just to check, I ran it with the RSwM2 implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;W = BrownianBridge(0.0,1.0,0.0,1.0,0.0,0.0;rswm = RSWM(adaptivealg=:RSwM2))&#xA;prob = NoiseProblem(W,(0.0,1.0))&#xA;monte_prob = MonteCarloProblem(prob)&#xA;@time sol = solve(monte_prob,dt=0.1,num_monte=10000)&#xA;&#xA;# Spot check the mean and the variance&#xA;qs = 0:0.1:1&#xA;for i in 2:10&#xA;  q = qs[i]&#xA;  @test ≈(timestep_mean(sol,i),q,atol=1e-2)&#xA;  @test ≈(timestep_meanvar(sol,i)[2],(1-q)*q,atol=1e-2)&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This test is still stochastic but tends to pass more for 1 and 3 than 2. Its tolerances are set so that way it can go fast enough on Travis and Appveyor, but it should be able to distinguish them more cleanly if you up the number of trajectories. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And of course, you can do this same kind of thing in your own implementation to see. The test could probably be made better if you can get an analytical solution which has a sudden stiffness to induce re-rejections reliably, but at least the Brownian Bridge variation is the most exacting test I have so far.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-10-13T05:06:24.580" LastActivityDate="2017-10-13T05:06:24.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28040" PostTypeId="1" CreationDate="2017-10-13T06:03:21.223" Score="3" ViewCount="455" Body="&lt;p&gt;While implementing in Matlab the Frobenius norm of a matrix &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\| A\|_{\text F} := \sqrt{ \sum_{i=1}^m \sum_{j=1}^n a_{ij}^2 },$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a problem arises when numbers are too big or too small:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If a number in the matrix is &lt;strong&gt;too big&lt;/strong&gt; (but within bounds of Matlab), taking its square will result in “infinity”. Adding anything else to it and taking the root will also result in infinity. However, the norm may actually be much smaller, since the root is taken at the end.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Similarly, if all numbers in the matrix are sufficiently &lt;strong&gt;small&lt;/strong&gt;, their squares and, thus, the norm will return $0$, while in reality it may be big enough to be represented by Matlab, since the root is taken.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How would we go about dealing with cases like this in the implementation of the function? I am aware that &lt;code&gt;norm(A,'fro')&lt;/code&gt; exists in Matlab, but we have been asked to deal with this problem ourselves, and I’m baffled.&lt;/p&gt;&#xA;" OwnerUserId="25669" LastEditorUserId="7574" LastEditDate="2017-10-16T09:37:44.630" LastActivityDate="2017-10-16T09:37:44.630" Title="How to compute the Frobenius norm of matrices whose entries are either too large or too small?" Tags="|matlab|numerical-analysis|algorithms|matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28041" PostTypeId="1" CreationDate="2017-10-13T07:17:52.563" Score="1" ViewCount="274" Body="&lt;p&gt;In an Eulerian frame of reference, the CFL number is defined as $$\sigma=\frac{u \Delta t}{\Delta x}$$&#xA;with $u$ the magnitude of the fluid velocity. A restriction such as $\sigma&amp;lt;1$ for time integration then roughly means that the initial information travels at most over one cell-width during one time-step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an Arbitrary Lagrangian-Eulerian (ALE) framework the grid also moves and the convective velocity becomes $u-u_\mathrm{grid}$. Should the definition of the CFL number now be based on $u-u_\mathrm{grid}$ rather than $u$? &lt;/p&gt;&#xA;" OwnerUserId="1973" LastActivityDate="2017-10-13T07:17:52.563" Title="Definition of CFL number in Arbitrary Lagrangian-Eulerian framework" Tags="|numerical-analysis|fluid-dynamics|time-integration|cfl|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28042" PostTypeId="2" ParentId="28040" CreationDate="2017-10-13T07:51:37.247" Score="3" Body="&lt;p&gt;You can scale your matrix by, say, the inverse of the largest/smallest(nonzero) entry before you compute the norm and use that $\|aM\|=|a|\|M\|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And have a look into Higham's book &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt; to understand how these scalings have an effect on the accuracy. &lt;/p&gt;&#xA;" OwnerUserId="4244" LastEditorUserId="4244" LastEditDate="2017-10-13T08:34:11.830" LastActivityDate="2017-10-13T08:34:11.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28043" PostTypeId="1" CreationDate="2017-10-13T09:34:46.820" Score="1" ViewCount="39" Body="&lt;p&gt;&lt;strong&gt;I would like to understand how division of round-off errors can lead to large errors???&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a method to measure the frequency dependent dissipation in simulations performed by Discontinuous Galerkin(Finite element based) Time Domain method on wave propagation problems. To this end, I excite my domain with a Gaussian pulse and measure the propagated signal $y_1$ and $y_2$ at 2 monitoring points $z_1$ and $z_2$ of the domain. The discrete Fourier transform(DFT) $F_1(f)$ and $F_2(f)$ of both signals are computed and the dissipation $\alpha_{diss}$ is evaluated through : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\alpha_{diss}(f) = \frac{log(|F_2(f)|/|F_1(f)|)}{z_2-z_1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is no dissipation, the amplitude spectrums $|F_1|$ and $|F_2|$ should theoretically be the exact same Gaussian $F_{th}$. However, due to finite length signal, the DFT makes round-off errors(different for $y_1$ and $y_2$). As a result, there is an increasing overall error in the measure of $\alpha_{diss}(f)$ which prevents the method to be used far away from the mid-band frequency $f_0$ of the Gaussian spectrum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The figure below shows this error increase of $|\alpha_{diss}(f)-0|$(in log scale) in red. The blue and cyan curves show the round-off errors $F_2-F_{th}$ and $F_1-F_{th}$ respectively. The mid-band frequency is $f_0=2$ and Gaussian line-width $\Delta f_0=0.5$. We can clearly see that beyond $3\Delta f_0$ the evaluation of $\alpha_{diss}(f)$ is not reliable anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I really wonder why is this happening?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eQcIK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eQcIK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25671" LastActivityDate="2017-10-13T09:34:46.820" Title="Round-off errors in ratio of Gaussian functions" Tags="|error-estimation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28044" PostTypeId="1" AcceptedAnswerId="28050" CreationDate="2017-10-13T09:59:11.497" Score="1" ViewCount="279" Body="&lt;p&gt;I am responsible for assisting with certain error measurements for an FE program. The idea is to set up some benchmark comparisons which we can use for future development of the program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To simplify the task I chose to look at a simple finite difference scheme for the equation of a parabola. Consider discrete values of $x_i$ where $\mathbb{x}$ is a vector:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbb{x} = \left&amp;lt; \begin{matrix} 0&amp;amp;1&amp;amp;2&amp;amp;3&amp;amp;4&amp;amp;5&amp;amp;6&amp;amp;7\end{matrix}\right&amp;gt;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider a mapping of $\mathbb{x}\rightarrow\mathbb{f}$ where $f_i = x_i^2$. It sounds fancy when I say it like that but its really just making a new vector with each value in it equalling the value $x_i^2$.&#xA;$$\mathbb{f} = \left&amp;lt; \begin{matrix} 0&amp;amp;1&amp;amp;4&amp;amp;9&amp;amp;16&amp;amp;25&amp;amp;36&amp;amp;49\end{matrix}\right&amp;gt;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotting $\mathbb{f} $ versus $\mathbb{x}$ yields some sort of discretized parabola. The numerical derivative of this would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbb{df}_{num} = \left&amp;lt; \begin{matrix} 1&amp;amp;3&amp;amp;5&amp;amp;7&amp;amp;9&amp;amp;11&amp;amp;13&amp;amp;15\end{matrix}\right&amp;gt;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a very simple finite difference scheme of:&#xA;$$df_i = \frac{f_{i+1}-f_i}{x_{i+1}-x_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will also be noted that $\mathbb{df}_{num}$ contains one less term than $\mathbb{f}$. This, I believe, is inherent to finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analytical counterpart for $\mathbb{df}$ which follows from $\frac {d }{d x}x^2 = 2x$ would be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbb{df}_{a} = \left&amp;lt; \begin{matrix} 0&amp;amp;2&amp;amp;4&amp;amp;6&amp;amp;8&amp;amp;10&amp;amp;12&amp;amp;14\end{matrix}\right&amp;gt;$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now subtracting the 2 vectors we get $\mathbb{f}_a-\mathbb{f}_{num} = \left&amp;lt; \begin{matrix} -1&amp;amp;-1&amp;amp;-1&amp;amp;-1&amp;amp;-1&amp;amp;-1&amp;amp;-1&amp;amp;-1\end{matrix}\right&amp;gt;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is a sort of crude error measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be a better error measure? Something that would be simple to implement for things like this but precise enough that it can spot deviations efficiently in the context of a script? The idea is to have an error vector like this and compare each value in it with some sort of threshhold value, though we have no idea what this threshhold value would be...&lt;/p&gt;&#xA;" OwnerUserId="22736" LastActivityDate="2017-10-13T14:30:50.313" Title="Error measure for a simple finite difference scheme" Tags="|numerical-analysis|error-estimation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28045" PostTypeId="1" AcceptedAnswerId="28051" CreationDate="2017-10-13T12:07:44.907" Score="0" ViewCount="340" Body="&lt;p&gt;Computing an axis aligned bounding box of a point set is trivial. Computing a bounding sphere of a point set is also trivial &lt;em&gt;when the center is known&lt;/em&gt;. Computing the center of the bounding sphere is not-so-trivial, as far as I see. So my plan is to approximate the center of the bounding sphere by the center of the bounding box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there any situations where this approach will lead to silly results?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2019-05-28T17:39:10.823" Title="Is it valid to assume the center of a bounding sphere to be also the center of the bounding box?" Tags="|computational-geometry|geometry|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28047" PostTypeId="1" CreationDate="2017-10-13T14:20:24.513" Score="1" ViewCount="156" Body="&lt;p&gt;I need to add to an existing FEM solver some &lt;strong&gt;embedded reinforcement element&lt;/strong&gt;. This would give me the possibility to model/solve concrete structure (reinforced with steel rebar) taking into account &lt;em&gt;the bond slip&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What is allready implemented (no bond-slip)&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://image.noelshack.com/fichiers/2017/41/5/1507899404-steel-truss-element.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://image.noelshack.com/fichiers/2017/41/5/1507899404-steel-truss-element.png&quot; alt=&quot;Mapping&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the current model, the steel rebar are taken into account with the assumption of a perfect bond with the concrete element. The local stiffness matrix of this element is the following:&#xA;$$K_{local} = \frac{EA}{L} \begin{bmatrix}&#xA; 1&amp;amp;  0&amp;amp; -1 &amp;amp; 0\\ &#xA; 0&amp;amp;  0&amp;amp;  0&amp;amp;0 \\ &#xA; -1&amp;amp;0 &amp;amp; 1 &amp;amp; 0\\ &#xA; 0&amp;amp;  0&amp;amp; 0 &amp;amp; 0&#xA;\end{bmatrix}$$&#xA;It's possible to have the global stiffness matrix with a little rotating matrix:&#xA;$$K_{global} = T^{^{T}}K_{local} T$$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What I plan to do&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://image.noelshack.com/fichiers/2017/41/5/1507899423-link-element.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://image.noelshack.com/fichiers/2017/41/5/1507899423-link-element.png&quot; alt=&quot;Link_element&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will use the &lt;strong&gt;link element&lt;/strong&gt; develloped by Ngo and Scordelis (1967).The link element may be conceptualized as two orthogonal springs, linking the reinforced&#xA;concrete element and the discrete reinforcement element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we can see in the upper figure, the steel point &lt;em&gt;j&lt;/em&gt; is connected to the concrete point &lt;em&gt;i&lt;/em&gt;. We can define the following vector (corresponding dofs):&#xA;$$\bar{r} = \begin{bmatrix}&#xA;r_{i,x}\\ &#xA;r_{i,y}\\ &#xA;r_{j,x}\\ &#xA;r_{j,y}\\ &#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tangeantial/radial displacement vector can be computed by:&#xA;$$\begin{bmatrix}&#xA;\Delta _{t}\\ &#xA;\Delta _{r}\\ &#xA;\end{bmatrix} = \begin{bmatrix}&#xA;-cos(\phi) &amp;amp; -sin(\phi) &amp;amp; cos(\phi) &amp;amp;sin(\phi) \\ &#xA; sin(\phi)&amp;amp;-cos(\phi)  &amp;amp;  -sin(\phi)&amp;amp; cos(\phi)&#xA;\end{bmatrix} \bar{r}$$&#xA;The corresponding Force vector (in the two springs) can be computed by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{bmatrix}&#xA;F_{t}\\ &#xA;F_{r}&#xA;\end{bmatrix}=A\begin{bmatrix}&#xA;k_{t} &amp;amp; 0 \\ &#xA;0 &amp;amp; k_{r}&#xA;\end{bmatrix}\begin{bmatrix}&#xA;\Delta _{t}\\ &#xA;\Delta _{r}\\ &#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With &lt;em&gt;Kt&lt;/em&gt; corresponding to the stiffness of the interface, &lt;em&gt;Kr&lt;/em&gt; set to a big value (typically 100 &lt;em&gt;Kt&lt;/em&gt;) to neglect dowel action, and &lt;em&gt;A&lt;/em&gt; being the circunferential area of the rebar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we can rotate back the forces in the global (x-y) coordinate system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar{F} = \begin{bmatrix}&#xA;F_{i,x}\\ &#xA;F_{i,y}\\ &#xA;F_{j,x}\\ &#xA;F_{j,y}\\&#xA;\end{bmatrix} = T^{T}\begin{bmatrix}&#xA;F_{t}\\ &#xA;F_{r}&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to remark that the final (assembled) equation is under this form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar{F} = [K] \bar{r}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or more generally:&#xA;$$\begin{bmatrix}&#xA;\bar{F_{conc}}\\ &#xA;\bar{F_{steel}}&#xA;\end{bmatrix} = \begin{bmatrix}&#xA;[K_{cc}] &amp;amp; [K_{cs}]\\ &#xA;[K_{cs}] &amp;amp; [K_{ss}] &#xA;\end{bmatrix} \begin{bmatrix}&#xA;\bar{d_{conc}}\\ &#xA;\bar{d_{steel}}&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible to apply a static condensation of the steel &lt;em&gt;dofs&lt;/em&gt;:&#xA;$$\bar{F_{c}^{*}} = \bar{F_{conc}} - [K_{cs}][K_{ss}]^{-1}\bar{F_{steel}}$$&#xA;$$[K_{c}^{*}]= [K_{cc}] - [K_{cs}][K_{ss}]^{-1}[K_{cs}]$$&#xA;With this &lt;em&gt;equivalent&lt;/em&gt; formulation, the system can be seen under this form:&#xA;$$\bar{F_{c}^{*}} = [K_{c}^{*}]\bar{d_{conc}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, only the degrees of freedom of the concrete are involved.&#xA;The big advantage is that we can now simply &lt;strong&gt;add&lt;/strong&gt; the inside Force vector coming from the steel rebar, the concrete element and the bonding link. We can also sum the tangeant stiffness matrix of the concrete, steel rebar, and bonding link (and apply after a standart Newton/Rapson step).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;So where is my problem?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I will take a very simple example to express my problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://image.noelshack.com/fichiers/2017/41/5/1507902544-test-case.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://image.noelshack.com/fichiers/2017/41/5/1507902544-test-case.jpg&quot; alt=&quot;Link_element&quot;&gt;&lt;/a&gt;&#xA;As you can see:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Horizontal bar $\phi = 0 $&lt;/li&gt;&#xA;&lt;li&gt;Circunferential area of the rebar : $0.015 m^{2}$&lt;/li&gt;&#xA;&lt;li&gt;$k_{t} = 10000 [Mpa]$ for the tangeantial imposed slip (0.5mm)&lt;/li&gt;&#xA;&lt;li&gt;$k_{r} = 300000 [Mpa]$ to avoid radial displacement&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We get quickly:&#xA;$$\begin{bmatrix}&#xA;\Delta _{t}\\ &#xA;\Delta _{r}\\ &#xA;\end{bmatrix} = \begin{bmatrix}&#xA;0.0005\\ &#xA;0\\ &#xA;\end{bmatrix} = \begin{bmatrix}&#xA;-1 &amp;amp; 0 &amp;amp;1 &amp;amp;0 \\ &#xA; 0&amp;amp;-1  &amp;amp;  0&amp;amp; 1&#xA;\end{bmatrix} \begin{bmatrix}&#xA;0\\ &#xA;0\\ &#xA;0.0005\\ &#xA;0&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching the force we get:&#xA;$$&#xA;\begin{bmatrix}&#xA;F_{t}\\ &#xA;F_{r}&#xA;\end{bmatrix}=\begin{bmatrix}&#xA;0.075\\ &#xA;0&#xA;\end{bmatrix}=0.015\begin{bmatrix}&#xA;10000 &amp;amp; 0 \\ &#xA;0 &amp;amp; 300000&#xA;\end{bmatrix}\begin{bmatrix}&#xA;0.0005\\ &#xA;0\\ &#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying the rotating matrix:&#xA;$$\bar{F} = \begin{bmatrix}&#xA;F_{i,x}\\ &#xA;F_{i,y}\\ &#xA;F_{j,x}\\ &#xA;F_{j,y}\\&#xA;\end{bmatrix} = \begin{bmatrix}&#xA;-0.075\\ &#xA;0\\ &#xA;0.075\\ &#xA;0\\&#xA;\end{bmatrix} =T^{T}\begin{bmatrix}&#xA;0.075\\ &#xA;0&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which seems to be physically correct (opposite forces in concrete/steel elements, correct value...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding stiffness matrix is the following:&#xA;$$[K] = \begin{bmatrix}&#xA;150 &amp;amp; 0 &amp;amp; -150 &amp;amp; 0 \\ &#xA; 0&amp;amp; 4500 &amp;amp; 0 &amp;amp; -4500 \\ &#xA; -150 &amp;amp; 0  &amp;amp; 150 &amp;amp; 0\\ &#xA; 0&amp;amp; -4500 &amp;amp; 0 &amp;amp; 4500&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying the previous condensed equation, we can build:&#xA;$$\bar{F_{c}^{*}} = \begin{bmatrix}&#xA;0\\ &#xA;0&#xA;\end{bmatrix}$$&#xA;$$[K_{c}^{*}]= \begin{bmatrix}&#xA;0 &amp;amp; 0\\ &#xA;0 &amp;amp; 0&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;And...?&lt;/h1&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The &lt;em&gt;equivalent&lt;/em&gt; bonding load vector is &lt;strong&gt;full of zeros&lt;/strong&gt;! What is the meaning of this? Even with a consequent bond-slip, no force are applied in the condensed concrete model?&lt;/li&gt;&#xA;&lt;li&gt;The &lt;em&gt;equivalent&lt;/em&gt; tangeant stiffness (bonding link) is &lt;strong&gt;full&lt;/strong&gt; of zeros! This is a big problem, since it will raise a &lt;em&gt;singular matrix&lt;/em&gt; error in the next step of the Newton/Rapson iteration!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there any physical explanation for this? For sure, I am misunderstanding a concept, but this is a real problem in my implementation purpose. Maybe it's a question of boundary condition... I am really not sure.&lt;/p&gt;&#xA;" OwnerUserId="24889" LastEditorUserId="24889" LastEditDate="2017-10-13T16:52:21.417" LastActivityDate="2019-04-18T14:51:17.800" Title="Correct way to model an embedded reinforcement (non linear FEM)?" Tags="|finite-element|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28048" PostTypeId="2" ParentId="28045" CreationDate="2017-10-13T14:20:42.213" Score="1" Body="&lt;p&gt;Off-hand, I think the more skewed your point distribution, the worse this approximation would be. Example, say in 2D you have a point at $[x,y]=(0,0)$, and points along the vertical line $(1,-1) \rightarrow (1,1)$, the bounding box would be $(0,-1)\times(1,1)$ with a centroid at $(0.5,0)$, but the centroid of the bounding circle would be at $(1,0)$.&lt;/p&gt;&#xA;" OwnerUserId="4736" LastActivityDate="2017-10-13T14:20:42.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28049" PostTypeId="2" ParentId="28045" CreationDate="2017-10-13T14:22:58.523" Score="2" Body="&lt;p&gt;I am almost certain that a literature review will find an algorithm that is exact. But I would also suggest to start with the center of mass of the convex hull -- which is easy to compute.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-13T14:22:58.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28050" PostTypeId="2" ParentId="28044" CreationDate="2017-10-13T14:30:50.313" Score="4" Body="&lt;p&gt;The error measure you have is not appropriate. That's because if you compute the quantities $df_i = \frac{f_{i+1}-f_i}{x_{i+1}-x_i}$, then they essentially approximate the error at location $\frac{x_{i+1}+x_i}{2}$, i.e., the midpoint between $x_{i+1}$ and $x_i$, and not at $x_i$ as you suggest. Consequently, your difference $f_a-f_{num}$ will only ever converge with ${\cal O}(\Delta x)$ even if your scheme is actually of higher order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for a better suggestion: Think of the solution of your PDE as a function, $u(x)$, and of the discrete approximation as a function $u_h(x)$. In the case of finite differences you could, for example, interpolate linearly between adjacent nodes. Then an appropriate measure would be $\int |u(x)-u_h(x)|^2 dx$ or the &quot;energy error&quot;  $\int |u'(x)-u'_h(x)|^2 dx$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will end by noting that defining error measures is a field with a long history. There is much literature on the topic. The fact that you suggest a measure you made up from scratch would indicate to me that you have some reading left to do :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-13T14:30:50.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28051" PostTypeId="2" ParentId="28045" CreationDate="2017-10-13T14:53:48.280" Score="2" Body="&lt;p&gt;The book &quot;Real-Time Collision Detection&quot; suggests that the AABB centroid is a decent approximation to the bounding sphere. In fact, the AABB centroid is much more accurate than the center of mass of the points, in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick survey of the techniques mentioned in the book (Chapter 4):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Centroid of AABB: quick and dirty, not terrible&lt;/li&gt;&#xA;&lt;li&gt;Center of data in direction of maximum variance (eigenvalue/eigenvector of covariance matrix) --&gt; This method seems like it could generalize to find a good bounding ellipsoid of your data, potentially better for high aspect-ratio point sets&lt;/li&gt;&#xA;&lt;li&gt;Randomized O(N) algorithm (Welzl's algorithm) which maintains a set of points which must lie on the minimum sphere boundary. An implementation is available in CGAL.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The book is written toward video game programmers, so there is an emphasis on being &quot;good enough&quot; quickly rather than necessarily correct. A typical use-case for these bounding volumes in this context is for collision detection. Since the goal is to minimize the overall time to do the collision detection, having exact bounding volumes that are potentially expensive to compute is counter-productive, hence the emphasis on approximate methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an official link to the book, but there are other &quot;less official&quot; links floating around the internet as well...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sciencedirect.com/science/book/9781558607323&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sciencedirect.com/science/book/9781558607323&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-10-13T14:53:48.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28054" PostTypeId="1" CreationDate="2017-10-13T17:36:58.303" Score="1" ViewCount="199" Body="&lt;p&gt;Is there any modern online resources which compares the most popular finite element method libraries/packages/languages? &lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2017-10-17T13:35:53.517" LastActivityDate="2017-10-17T13:35:53.517" Title="Comparison between FEM libraries &amp; languages" Tags="|finite-element|reference-request|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28055" PostTypeId="2" ParentId="28054" CreationDate="2017-10-13T17:41:58.340" Score="3" Body="&lt;p&gt;This is the best resource that I know: &lt;a href=&quot;https://github.com/kostyfisik/FEA-compare&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/kostyfisik/FEA-compare&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2017-10-13T17:41:58.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28057" PostTypeId="1" CreationDate="2017-10-14T01:41:43.773" Score="2" ViewCount="839" Body="&lt;p&gt;I have solved a simple problem on Openfoam, which is a 3D rectangular parallelopiped, filled with rectangular hexahedrons as mesh elements. The structured, mapped mesh was made in ICEM and solved in OF. But the mesh and data, are in unstructured format i.e. they are represented with connectivity. I now want to convert the unstructured data to a structured data w/o connectivity i.e. the mesh should be represented with loop like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(k=0;k&amp;lt;znocells;k++)&#xA;for(j=0;j&amp;lt;ynocells;j++)&#xA;for(i=0;i&amp;lt;xnocells;i++)&#xA; { &#xA;  //access elements...&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried to write a code for the same, but have failed. A simple workaround involved involved sorting the array of elements, but that won't work for geometries like bifurcation and stuff. Is there a code available, or a software to import the OF mesh and spit out a structured mesh?&lt;/p&gt;&#xA;" OwnerUserId="23565" LastActivityDate="2018-03-15T03:10:09.663" Title="Convert Unstructured Mesh to Structured" Tags="|mesh|unstructured-mesh|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28059" PostTypeId="1" CreationDate="2017-10-14T05:43:09.930" Score="4" ViewCount="775" Body="&lt;p&gt;I am implementing an arbitrary bits random number generator (in [$0,2^n$)) and also want to ensure that the generator always generate unique numbers until all possible numbers generated, so I implement it using LCG (I also considered the linear feedback shift register method, but it cannot be scaled to arbitrary number of bits because the dependence on primitive polynomial).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm as follows:   &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;create an array length $n$&lt;/li&gt;&#xA;&lt;li&gt;using random bits to fill the array. This array is $X_0$&lt;/li&gt;&#xA;&lt;li&gt;generate the second length-$n$ array ($a$) and fill in three lowest bits with $1,0,1$ (to ensure $a-1$ is divisible by $4$) and use random bits to fill other cell.&lt;/li&gt;&#xA;&lt;li&gt;generate the third length-$n$ array ($c$) and set the lowest bit to $1$ (other cell random), so this array is odd and relatively prime to $m=2^n$.&lt;/li&gt;&#xA;&lt;li&gt;Generate random numbers by $X_n=(aX_{n-1}+c) \mod m$ ( the mod is by keeping only $n$ bits in add and multiply operations).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As the result, this LCG's repeating period is always maximum, however, I find that the $i$-th lowest bit always have the repeating period of $2^{i}$. For example, the lowest bit is in sequence of &lt;code&gt;1,0,1,0,...&lt;/code&gt;, the second lowest bit is like&lt;code&gt;1,0,0,1,1,0,0,1,...&lt;/code&gt;, and so on. This makes me wonder if somewhere wrong in my implementation or it is actually the weakness of LCG. I tested the &lt;code&gt;rand&lt;/code&gt; in C and did not find this pattern, so I guess this problem is from my implementation. Can anyone gives me some advice about this? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="25682" LastEditorUserId="25682" LastEditDate="2017-10-14T15:09:51.667" LastActivityDate="2018-01-17T06:45:40.747" Title="Why my linear congruential generator generate low quality random numbers?" Tags="|random-number-generation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28060" PostTypeId="2" ParentId="28047" CreationDate="2017-10-14T07:48:56.760" Score="1" Body="&lt;p&gt;It looks like you statically condense dofs node by node. To make it work this should be done at least element by element. Your matrix &lt;span class=&quot;math-container&quot;&gt;$K_{cc}$&lt;/span&gt; is 8 by 8, whereas matrix &lt;span class=&quot;math-container&quot;&gt;$K_{ss}$&lt;/span&gt; is 4 by 4. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the assumption that you fibres are short or micro-cracking make the response of long fibre limited to short length. Static condensation, which you presenting has an advantage, that you can implement that on element level and in well-implemented cold be numerically efficient and be easily implemented in standard FE code, for example using UMAT matrix in ABAQUS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you should consider applying condensation on the system level, to get realistic redistribution of stresses for long reinforced fibres. In that case, static condensation is applied for each fibre independently. Instead of that, you can consider using multiplicative/additive block solver or block solver with Schur complement. In fact, you don't have to implement this by yourself and use PETSc field split preconditioner, see &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCFIELDSPLIT.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCFIELDSPLIT.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note bar does not have to be restricted to edge of solid/concrete element, you have to express displacements at the end of the bar by linear combination of dofs of concrete element. In principle method will be the same.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="22284" LastEditDate="2019-04-18T14:51:17.800" LastActivityDate="2019-04-18T14:51:17.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28061" PostTypeId="1" AcceptedAnswerId="30755" CreationDate="2017-10-14T12:16:55.270" Score="6" ViewCount="233" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to analyse  fourier characteristics of a derivative. For example if I have a first order derivative approximated as following:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial \Psi(x)}{\partial x} = \frac{\Psi_{i+1}-\Psi_{i-1}}{\Delta x}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to get its fourier characteristics, substitute &lt;span class=&quot;math-container&quot;&gt;$\Psi(x)= e^{j\, kx}$&lt;/span&gt; in LHS, &#xA;and its discrete counter part&#xA; &lt;span class=&quot;math-container&quot;&gt;$\Psi(x=i\Delta x) = \Psi_{i}= e^{j \,ki\Delta x}$&lt;/span&gt; RHS.&#xA;Here &lt;span class=&quot;math-container&quot;&gt;$j=\sqrt{-1}$&lt;/span&gt; in  and k represents the wavenumber. It gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$F \left[\frac{\partial \Psi(x)}{\partial x}\right] = \frac{e^{jk(i+1)\Delta x}-e^{jk(i-1)\Delta x}}{\Delta x}$$&lt;/span&gt;&#xA;Comparing both sides gives&#xA;&lt;span class=&quot;math-container&quot;&gt;$$k= \sin(j \, k\Delta x)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem statement&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I like to carry out similar analysis for &lt;strong&gt;implicit derivatives&lt;/strong&gt; of &lt;a href=&quot;https://www.researchgate.net/publication/222488464_Lele_SK_Compact_Finite_Difference_Schemes_with_Spectral-like_Resolution_Journal_of_Computational_Physics_1031_16-42&quot; rel=&quot;nofollow noreferrer&quot;&gt;lele (1992)&lt;/a&gt;, given as following&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f_i' + \alpha (f_{i+1}' + f_{i-1}') +  \beta (f_{i+2}' + f_{i-2}')  \nonumber \\&#xA;				=a\frac{f_{i+1}- f_{i-1}}{(2h)^2} + &#xA;				b\frac{f_{i+2}- f_{i-2}}{(4h)^2} +&#xA;				c\frac{f_{i+3}- f_{i-3}}{(6h)^2} $$&lt;/span&gt;&#xA;The coefficients are determined after Taylor series expansion of both sides and then comparing coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem contains a derivative term of type &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial x} \left[a(x) \frac{\partial}{\partial x} \right]\psi $&lt;/span&gt;. &#xA;It can be simplified if I assume constant media for Fourier analysis:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial}{\partial x} \left[a(x) \frac{\partial}{\partial x} \right]\psi =&#xA;  a(x) \frac{\partial}{\partial x}  \frac{\partial\psi}{\partial x} \psi= a(x) \frac{\partial^2\psi}{\partial x^2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In simulation I am using first derivative two times for computing the second derivatives as following: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;step computes : &lt;span class=&quot;math-container&quot;&gt;$\zeta = a(x) \frac{\partial}{\partial x} $&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;step compute : &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial \zeta}{\partial x}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For &lt;strong&gt;explicit derivatives&lt;/strong&gt; (example given in background part) the equality holds i.e. &lt;span class=&quot;math-container&quot;&gt;$\partial_x.\partial_x ==\partial_x^2$&lt;/span&gt; is valid in numerical form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;whether &lt;span class=&quot;math-container&quot;&gt;$\partial_x.\partial_x$&lt;/span&gt; is equal to &lt;span class=&quot;math-container&quot;&gt;$ \partial_x^2$&lt;/span&gt; for implicit derivative.  &lt;/li&gt;&#xA;&lt;li&gt;If it doesn't then how can I carry out its analysis.&lt;/li&gt;&#xA;&lt;li&gt;How to handle the cross derivative terms e.g. &lt;span class=&quot;math-container&quot;&gt;$\partial_x.\partial_y$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="20015" LastEditorUserId="20015" LastEditDate="2019-01-21T13:28:59.097" LastActivityDate="2019-01-24T18:42:17.780" Title="Fourier characteristics of repeated numerical derivative" Tags="|finite-difference|fourier-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28062" PostTypeId="1" CreationDate="2017-10-14T15:46:54.527" Score="1" ViewCount="193" Body="&lt;p&gt;I am a Computational Biology master's student and I would like to study differential equations (both ODEs and PDEs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to have a look at the available textbooks but what I found is either too theory-oriented or too applications-oriented. I do not want to waste time proving theorems but I do not want to simply solve basic equations using finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only text I found is &lt;em&gt;Ordinary and Partial Differential Equations&lt;/em&gt; by Henner. Unfortunately, I was not able to find any review of it and it's quite huge. I also toke into consideration doing the ODE part by watching Gilbert Strang's short videos online and the PDE part watching some lectures on YouTube (&lt;em&gt;commutant&lt;/em&gt;'s channel), but I was wondering whether this is too basic (MOOC style). Any suggestion?&lt;/p&gt;&#xA;" OwnerUserId="22185" LastActivityDate="2017-10-15T09:30:52.037" Title="Learning differential equations: a textbook" Tags="|pde|ode|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28064" PostTypeId="2" ParentId="28062" CreationDate="2017-10-15T03:43:49.317" Score="5" Body="&lt;p&gt;Check out &lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/ExplODE/&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; new book about ODEs by Trefethen et al.&#xA;It focuses on interpretation of solutions' behavior and on nonlinear problems.   &lt;/p&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2017-10-15T03:43:49.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28065" PostTypeId="2" ParentId="28062" CreationDate="2017-10-15T09:30:52.037" Score="3" Body="&lt;p&gt;For ODEs I'd learn them as dynamical systems through &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0738204536&quot; rel=&quot;nofollow noreferrer&quot;&gt;Strogatz's book&lt;/a&gt;. Getting analytical solutions is far less important than analysis of nonlinear ODEs in most contexts since having an analytical solution is a special case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For PDEs I wouldn't treat it as a subject to learn unless you are a mathematician and really want to dive into function spaces and regularity theory. I would treat the types of PDEs your interested in as the subject. Pick a PDE from your subject you'll find books which cover that subject in detail. I would say you have to learn the numerical methods to really explore the pde as well.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-10-15T09:30:52.037" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28066" PostTypeId="1" CreationDate="2017-10-15T18:54:59.687" Score="1" ViewCount="175" Body="&lt;p&gt;Here, $n$ could be a few hundreds, or even thousands, so it is not possible to pregenerate a list of $$\{0, 1, \dots, 2^n-1\}$$ and shuffle it. Because it is traversal, no number should be visited twice. I want the process of generating the next number as fast as possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was trying to use LCG, but it was too slow ($O(n^2)$) to generate the next number. Linear feedback shift register (LFSR) might be a choice in terms of efficiency ($O(n)$), but I cannot ensure the LFSR always in maximum repeating period for arbitrary $n$. Is there any way to achieve this job effectively?&lt;/p&gt;&#xA;" OwnerUserId="25682" LastEditorUserId="20417" LastEditDate="2017-10-16T09:55:32.807" LastActivityDate="2017-10-18T08:34:44.380" Title="How to traverse $\{0, 1, \dots, 2^n-1\}$ in random order?" Tags="|random-number-generation|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28067" PostTypeId="2" ParentId="28066" CreationDate="2017-10-15T23:45:26.447" Score="6" Body="&lt;p&gt;A random number generator will give you random numbers that you can tweak to be between zero and &lt;code&gt;2^n&lt;/code&gt; and consequently it will allow you to sample random locations in your interval. These numbers may, in principle, repeat themselves -- so they are not a traversal, but that may not matter for the following reason:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If $n$ is large (in the thousands), you will not be able to visit all numbers in the interval of interest anyway. For example, for $n=1000$, there are $2^{1000}\approx 10^{100}$ integers in this interval, but on a single processor you will at the very most be able to visit about $10^9$ per second -- so $10^{100}$ is out of the question.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you really used a &lt;em&gt;random&lt;/em&gt; order, there is no way you could even store which numbers you have already visited.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Consequently, there is little actual difference between a random order, or using the first few billion or trillion elements of an actual (non-repeating) permutation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-15T23:45:26.447" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28068" PostTypeId="2" ParentId="28057" CreationDate="2017-10-16T01:29:50.817" Score="1" Body="&lt;p&gt;I think you can do it in a fairly straightforward way, as I do in PETSc. You find all the faces between cells, which gives you the mesh topology. To find the faces, you loop over cells and each of the 6 faces uses the hash for the 4 vertices (i,j,k,l) in sorted order. Then you just read out the hash table at the end. My code is here &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bitbucket.org/petsc/petsc/src/c99b676fdc4d0978b5621d138ceee2e1b7869aa6/src/dm/impls/plex/plexinterpolate.c?at=master&amp;amp;fileviewer=file-view-default#plexinterpolate.c-137&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bitbucket.org/petsc/petsc/src/c99b676fdc4d0978b5621d138ceee2e1b7869aa6/src/dm/impls/plex/plexinterpolate.c?at=master&amp;amp;fileviewer=file-view-default#plexinterpolate.c-137&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its a little longer because I first count everything so I can statically allocate the structure to hold it.&lt;/p&gt;&#xA;" OwnerUserId="21" LastActivityDate="2017-10-16T01:29:50.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28070" PostTypeId="1" CreationDate="2017-10-16T07:42:00.803" Score="1" ViewCount="93" Body="&lt;p&gt;I have found the &lt;a href=&quot;https://www.rocscience.com/help/RS3/webhelp2/pdf_files/verification/Verification_018_(Drained_Triaxial_Compressive_Test_of_Modified_Cam_Clay_Material).pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;following example&lt;/a&gt; which is produced as documentation for RS3 geotechnical FE software. This is a validation document that goes through an analytical verification of triaxial testing for modified cam clay material model. It relies heavily upon the p-q stress representation of the model to make its argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find it to be quite pedagogical in the way it presents its content. It has enabled me to somewhat understand the following diagram:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QxvEsm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QxvEsm.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this example triaxial test the isotropic stresses are initialized at point $(p,q) =(p_i, q_i)=(200,0)$. The deviatoric stress $q$ is then applied until the sample 'fails'. This is the intersection between the green line and the red line (CSL).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that for a given axisymmetric FE analysis with following boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vP4Ubm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vP4Ubm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is only possible to generate &lt;strong&gt;ONE&lt;/strong&gt; p-q green dashed line. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there is a way to plot the &lt;strong&gt;ENTIRE&lt;/strong&gt; surface. This would perhaps imply that I would have to force the sample to take several different stress paths and have them each intersect with the ellipse? Computational efficiency is not really the issue here, it is more the method. How do I force the sample to take different stress paths such that I can plot the entire yield surface? Is this even possible? Am I even asking the right question?&lt;/p&gt;&#xA;" OwnerUserId="22736" LastActivityDate="2017-10-19T08:30:23.483" Title="Plotting yield surface for geotechnical constitutive model" Tags="|finite-element|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28073" PostTypeId="2" ParentId="28033" CreationDate="2017-10-16T11:36:16.500" Score="2" Body="&lt;p&gt;This is not the ideal answer; better rigorous analysis can be done here. However, I hope that this has some value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We all agree that partial differential equation applies to all points in the domain without boundary.  Here we apply PDE as well on some points on boundary, and it has consequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, using one side formula on the grid 4x3 (space x time) gives close analytical solution, starting numeration of nodes from zero, numerating by rows:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/PvKOk.png&quot; alt=&quot;Mathematica graphics&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;u_0:=0\\&#xA;u_1:=h(h-1)\\&#xA;u_2:=(2h)(2h-1)\\&#xA;u_3:=(3h)(3h-1)\\&#xA;u_4:=0\\&#xA;u_8:=0\\&#xA;u_5={\frac {{h}^{2} \left( {h}^{2}+4\,h+6 \right) }{{h}^{2}+3\,h+4}}\\&#xA;u_6=4\,{h}^{2}\\&#xA;u_7={\frac {{h}^{2} \left( 9\,{h}^{2}+26\,h+34 \right) }{{h}^{2}+3\,h+4}}\\&#xA;u_9={\frac { \left( {h}^{3}+6\,{h}^{2}+9\,h+4 \right) h}{{h}^{2}+3\,h+4}}\\&#xA;u_{10}=2\, \left( 2\,h+1 \right) h\\&#xA;u_{11}={\frac {h \left( 9\,{h}^{3}+28\,{h}^{2}+43\,h+12 \right) }{{h}^{2}+3\,&#xA;h+4}}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we apply one side formula to enforce PDE on right side, for example at node 7, we can equivalently write,&#xA;\begin{equation}&#xA;(u_6-2u_7+u^*_6)/h^2 = (u_3-u_{11})/(2h),&#xA;\end{equation}&#xA;and solve for $u^*_6$ to satisfy PDE equation using this central finite diffrence scheme,&#xA;\begin{equation}&#xA;u^*_6=4\,{\frac {{h}^{2} \left( 3\,{h}^{2}+8\,h+10 \right) }{{h}^{2}+3\,h+4}&#xA;}&#xA;\end{equation}&#xA;with that at hand slope can be evaluated,&#xA;\begin{equation}&#xA;u'_7 \approx -2\,{\frac {h \left( 2\,{h}^{2}+5\,h+6 \right) }{{h}^{2}+3\,h+4}}&#xA;\end{equation}&#xA;And that what you implicitly apply at that node for this particular case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, applying on side rule on top-right boundary you assume that PDE governs values at that nodes, so this PDE has to be satisfied as well when central finite difference scheme is applied.  That enables you to calculate derivatives implicitly at that nodes. Values of those derivatives are hidden boundary conditions enforced at that nodes. Here PDE itself become boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An interesting question to ask, is a wave is reflected from such boundary? &lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-10-17T09:01:33.840" LastActivityDate="2017-10-17T09:01:33.840" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28074" PostTypeId="1" AcceptedAnswerId="28080" CreationDate="2017-10-16T13:33:02.090" Score="0" ViewCount="201" Body="&lt;p&gt;I have a linear elliptic PDE (unfortunately not allowed to be shown here) with a constant parameter $\epsilon$ giving the stable solutions qualitatively shown by the functions below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pj4ri.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pj4ri.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As we smoothly increase $\epsilon$, the solution changes &quot;qualitatively&quot; from the viewpoint of the number of extrema relating to the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I refer to the solution at $\epsilon_{cr}$ as a &lt;strong&gt;bifurcation point&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="20475" LastEditorUserId="713" LastEditDate="2017-10-16T18:59:09.450" LastActivityDate="2018-03-16T05:12:01.937" Title="Bifurcation of linear PDE" Tags="|elliptic-pde|linear-system|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28076" PostTypeId="2" ParentId="28033" CreationDate="2017-10-16T14:10:22.490" Score="1" Body="&lt;p&gt;You are doing explicit time stepping, so solvability is not much of a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can write for the last grid point&#xA;$$&#xA;du_n/dt = (u_{n-1}-2u_n+u_{n+1})/h^2&#xA;$$&#xA;with the ghost value $u_{n+1}$ being&#xA;$$&#xA;u_{n+1} = -u_{n-3} + 4 u_{n-2} - 6 u_{n-1} + 4 u_n&#xA;$$&#xA;A Taylor expansion shows&#xA;$$&#xA;u_{n+1} = u_n + h u'_n + (h^2/2) u''_n + (h^3/6) u'''_n + O(h^4)&#xA;$$&#xA;So the ghost value is just obtained by a fourth order extrapolation of the solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the Taylor expansion were of the form&#xA;$$&#xA;u_{n+1} = h u'_n + (h^2/2) u''_n + (h^3/6) u'''_n + O(h^4)&#xA;$$&#xA;then implicitly you would have applied zero Dirichlet bc. If it was of the form&#xA;$$&#xA;u_{n+1} = u_n + (h^2/2) u''_n + (h^3/6) u'''_n + O(h^4)&#xA;$$&#xA;then implicitly you would have applied zero Neumann bc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But we see all terms upto $O(h^3)$, so no boundary condition is being applied, explicitly or implicitly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does not correspond to anything physical or mathematical. The solutions in fact seem to be growing unboundedly with time when I ran the Python code. So I would not attach any significance to the solution you are getting out of this.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2017-10-17T05:39:12.290" LastActivityDate="2017-10-17T05:39:12.290" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28078" PostTypeId="2" ParentId="28019" CreationDate="2017-10-16T16:12:19.057" Score="1" Body="&lt;p&gt;Since you are solving inviscid equations, you will need some form of stabilization like SUPG or a DG scheme, to get a robust scheme. You wont need the pressure I presume. I would recommend solving the equations in vorticity-velocity form where the pressure is eliminated. See this paper for a DG scheme&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Jian-Guo Liu and Chi-Wang Shu, A High-Order Discontinuous Galerkin Method for 2D Incompressible Flows, Journal of Computational Physics 160(2):577-596, August 1999, DOI: 10.1006/jcph.2000.6475&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2017-10-16T16:12:19.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28080" PostTypeId="2" ParentId="28074" CreationDate="2017-10-17T03:18:30.453" Score="1" Body="&lt;p&gt;No -- as long as you keep saying &quot;the solution&quot;. &lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2017-10-17T03:18:30.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28082" PostTypeId="1" CreationDate="2017-10-17T12:01:28.163" Score="1" ViewCount="124" Body="&lt;p&gt;&lt;strong&gt;Cross posted &lt;a href=&quot;https://mathematica.stackexchange.com/q/157983/1871&quot;&gt;in mathematica.SE&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; : I want to test the solution which is given below is right by Matlab/Maple/Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please look &lt;a href=&quot;https://math.stackexchange.com/questions/2473201/how-to-solve-heat-equation-on-the-semi-infinite-rod-by-fourier-transform/2476391#2476391&quot;&gt;the post&lt;/a&gt; in mathstackexhange &lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please look below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; Solve the following heat equation on the semi-infinite rod &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_t=ku_{xx}$$ where $x,t&amp;gt;0$ and&#xA;    $u_x(0,t) =0$ and $u(x,0)=\begin{cases} &#xA;      1, &amp;amp; 0 &amp;lt; x &amp;lt;2 \\&#xA;      0, &amp;amp;  2\leq x  &#xA;   \end{cases}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We found the following answer &#xA;$$&#xA;u(x,t) = \frac{2}{\pi}\int_{0}^{\infty}e^{-s^2 t}\frac{1-\cos(2s)}{s}\cos(sx)ds.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am not sure the solution is right. I am not capable of to write a code in order to test it. Could you help me?&lt;/p&gt;&#xA;" OwnerUserId="23832" LastEditorUserId="5331" LastEditDate="2017-10-18T11:16:58.757" LastActivityDate="2017-10-18T11:16:58.757" Title="Code for solving the heat equation on the semi-infinite rod" Tags="|pde|fourier-analysis|fourier-transform|differential-equations|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28089" PostTypeId="2" ParentId="28066" CreationDate="2017-10-18T08:34:44.380" Score="1" Body="&lt;p&gt;Use a trivial $n$-bit counter, and encrypt it using a block cipher in any configuration &lt;em&gt;other than&lt;/em&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_Codebook_.28ECB.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;ECB&lt;/a&gt; or CTR, starting with the least-significant bits as your first block so the small change propagates to all the other bits of the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $n$ is not divisible by the size of your block cipher then you'll need to treat the remainder using &lt;a href=&quot;https://en.wikipedia.org/wiki/Format-preserving_encryption&quot; rel=&quot;nofollow noreferrer&quot;&gt;format-preserving encryption&lt;/a&gt;, because the output must be the same size as the input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives you your new index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have the means to decrypt the counter then you can trivially prove that this sequence visits every value exactly once: if any two input states mapped to the same output then you would not be able to decrypt the counter and the encryption itself would be demonstrably broken.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One caveat is that if you jump ahead in the sequence by, eg., $2^{256}$ (presumed to be a multiple of your block cipher size), then the first $2^{256}$ bits of the encrypted count would not change.  If you're not happy with this then reverse the order of the bits and encrypt it a second time.  Still O(n), but a little more complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously cryptographic primitives aren't a great way to speed things up, but it shows how to make things linear.  If you take that construction but replace the block cipher with something more economical (eg., a 64-bit perfect hash) then you can recover a lot of that lost performance.&lt;/p&gt;&#xA;" OwnerUserId="21143" LastActivityDate="2017-10-18T08:34:44.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28090" PostTypeId="1" CreationDate="2017-10-18T15:39:20.563" Score="1" ViewCount="81" Body="&lt;p&gt;I have a question concerning the branch of successive iteration methods (Newton, Runge-Kutta). I definitely know (or can read in Wikipedia) the implementation of these methods. But I was wondering about the precision in special case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a function&#xA;$$&#xA;D(\omega)=det(L_{m,n}(\omega)), \omega = \omega'-i\omega''&#xA;$$&#xA;and need to solve the equation&#xA;$$&#xA;D(\omega)=0&#xA;$$&#xA;For that task I want to use simple fixed-point iteration method&#xA;$$&#xA;\omega_{0}, \omega_{1} \mathrm{- given}\\&#xA;\omega_{n+1}=\alpha \omega_{n}+(1-\alpha)[\omega_{n}-\frac{D(\omega_{n})}{dD/d\omega|_{\omega_{n}}}]\\&#xA;dD/d\omega|_{\omega_{n}}=\frac{D(\omega_{n})-D(\omega_{n-1})}{\omega_{n}-\omega_{n-1}}&#xA;$$&#xA;&lt;strong&gt;So, could someone tell me if this method is well converging or not.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am asking because previously I've used this method only for real-valued variables and now I have complex-valued. This may lead to component entanglement while calculating the derivative.&lt;/p&gt;&#xA;" OwnerUserId="25713" LastActivityDate="2017-10-18T15:39:20.563" Title="Successive iteration method for solving eigenvalue ploblem" Tags="|linear-algebra|convergence|runge-kutta|newton-method|fixed-point|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28092" PostTypeId="2" ParentId="27226" CreationDate="2017-10-19T01:38:33.463" Score="2" Body="&lt;p&gt;The difference is which BLAS level they are utilizing. TRF is BLAS3 and TF2 is BLAS2. BLAS level 3 routines, roughly speaking, optimized for computers with cache memory such that they can perform block calls. BLAS 2 is optimized for vector computers with possibly shared memory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see in &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/da/dba/group__double_o_t_h_e_rcomputational_ga31cdc13a7f4ad687f4aefebff870e1cc.html#ga31cdc13a7f4ad687f4aefebff870e1cc&quot; rel=&quot;nofollow noreferrer&quot;&gt;the code (line 209)&lt;/a&gt;, if the optimum block size turns out to be either larger than the matrix size(which implies the impossibility of blocked algorithm) or less than or equal to 1 then DPSTRF falls back to the unblocked version DPSTF2.&lt;/p&gt;&#xA;" OwnerUserId="21617" LastActivityDate="2017-10-19T01:38:33.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28094" PostTypeId="2" ParentId="28059" CreationDate="2017-10-19T05:51:57.603" Score="1" Body="&lt;p&gt;This is simply how LCG works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can reduce the problem for your application by simply exclusive-or-ing some of the high bits over some of the low bits in the output (but &lt;em&gt;do not&lt;/em&gt; use this modified output as the next input to the generator loop).  This will retain the full period of the counter, but it improves the period of the low bits to be the same period as the high bits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not a very good tweak and still has substantial flaws as a PRNG, but it gets a basic job done for cheap.  You can make it a lot better by multiplying by another odd constant and doing another shifted exclusive-or, but it depends on what you really need and how much time you want to spend on it.&lt;/p&gt;&#xA;" OwnerUserId="21143" LastActivityDate="2017-10-19T05:51:57.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28102" PostTypeId="1" CreationDate="2017-10-20T14:00:39.713" Score="4" ViewCount="231" Body="&lt;p&gt;I implemented as an exercise a program to sample the statistics of the escape time of a Brownian particle in a potential well. I used the Euler-Maruyama method to numerically integrate the trajectories of the solution of the Ornstein-Uhlenbeck equation. For each trajectory I stored the escape time from the well.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;random&amp;gt;&#xA;#include &amp;lt;array&amp;gt;&#xA;#include &amp;lt;iterator&amp;gt;&#xA;#include &amp;lt;memory&amp;gt;&#xA;#include &amp;lt;omp.h&amp;gt;&#xA;&#xA;typedef std::mt19937_64 rng_t;&#xA;typedef double real_t;&#xA;&#xA;namespace params {&#xA;  constexpr real_t t0 = 0.0;&#xA;  constexpr real_t x0 = 0.0;&#xA;  constexpr real_t v0 = 0.0;&#xA;  constexpr real_t tau = 0.005;&#xA;&#xA;  constexpr real_t beta = 1.0;&#xA;  constexpr real_t alpha = 1.0;&#xA;  constexpr int iters = 10000;&#xA;&#xA;  constexpr real_t well_width = 1.0;&#xA;  constexpr real_t well_depth = 1.0;&#xA;  constexpr real_t ramp_width = 0.1;&#xA;};&#xA;&#xA;using namespace std;&#xA;using namespace params;&#xA;&#xA;constexpr auto half_width = well_width / 2;&#xA;constexpr auto g = well_depth / ramp_width;&#xA;&#xA;template&amp;lt;typename T&amp;gt; T gamma(T x) {&#xA;  if (x &amp;gt; -half_width &amp;amp;&amp;amp; x &amp;lt; -half_width + ramp_width)    return g;&#xA;  else if (x &amp;gt; half_width - ramp_width &amp;amp;&amp;amp; x &amp;lt; half_width) return -g;&#xA;  else                                                    return 0.0;&#xA;}&#xA;&#xA;template&amp;lt;typename T&amp;gt; class OrnsteinUhlenbeck {&#xA;  public:&#xA;    T EscapeTime() { while (IsInBounds()) EMTimestep(); return t; };&#xA;    OrnsteinUhlenbeck(shared_ptr&amp;lt;rng_t&amp;gt; r) { rng = r; };&#xA;  private:&#xA;    void EMTimestep(){&#xA;      normal_distribution&amp;lt;T&amp;gt; dW(0.0, sqrt(dt));&#xA;      t += dt;&#xA;      x += v * dt;&#xA;      v += (gamma&amp;lt;T&amp;gt;(x) - alpha * v) * dt + beta * dW(*rng);&#xA;    };&#xA;    bool IsInBounds() { return fabs(x) &amp;lt; half_width; };&#xA;    T t = t0;&#xA;    T dt = tau; &#xA;    T x = x0;&#xA;    T v = v0;&#xA;    shared_ptr&amp;lt;rng_t&amp;gt; rng;&#xA;};&#xA;&#xA;int main() {&#xA;  rng_t master_rng;&#xA;  uniform_int_distribution&amp;lt;unsigned&amp;gt; random_seed(314); &#xA;&#xA;  auto threads = omp_get_max_threads();&#xA;  vector&amp;lt;shared_ptr&amp;lt;rng_t&amp;gt;&amp;gt; rngs(threads);&#xA;  for (auto &amp;amp;r : rngs) r = make_shared&amp;lt;rng_t&amp;gt;(random_seed(master_rng));&#xA;&#xA;  cout &amp;lt;&amp;lt; &quot;Using &quot; &amp;lt;&amp;lt; threads &amp;lt;&amp;lt; &quot; threads.&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;  cout &amp;lt;&amp;lt; &quot;Simulation running...&quot; &amp;lt;&amp;lt; endl;&#xA;  array&amp;lt;real_t, iters&amp;gt; data;&#xA;  #pragma omp parallel for&#xA;  for (int n = 0; n &amp;lt; iters; n++) {&#xA;    auto thread = omp_get_thread_num();&#xA;    OrnsteinUhlenbeck&amp;lt;real_t&amp;gt; particle(rngs[thread]);&#xA;    data[n] = particle.EscapeTime();&#xA;  }&#xA;  cout &amp;lt;&amp;lt; &quot;Done.&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;  ofstream dataFile(&quot;data.dat&quot;);&#xA;  copy(data.begin(), data.end(), ostream_iterator&amp;lt;real_t&amp;gt;(dataFile, &quot; &quot;));&#xA;  dataFile.close();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Afterward I plotted the result with the following Python script&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.stats import expon&#xA;import matplotlib.pyplot as plt&#xA;&#xA;data = np.loadtxt(&quot;data.dat&quot;)&#xA;&#xA;plt.title(&quot;Escape Time Statistics&quot;)&#xA;plt.xlabel(&quot;escape time&quot;)&#xA;plt.ylabel(&quot;frequency&quot;)&#xA;&#xA;loc, scale = expon.fit(data)&#xA;_, bins, _ = plt.hist(data, bins=30, normed=True)&#xA;x = np.linspace((bins[0] + bins[1])/2, (bins[-1]+bins[-2])/2, 100)&#xA;y = expon.pdf(x, loc, scale)&#xA;plt.plot(x,y)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the plot of the statistics sometimes look like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XRAVO.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XRAVO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sometimes like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3HKaf.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3HKaf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;depending on the value of the seed of the &lt;code&gt;random_seed&lt;/code&gt; generator. How come?&lt;/p&gt;&#xA;" OwnerUserId="25733" LastActivityDate="2018-04-01T06:42:32.653" Title="Simple Monte Carlo in C++, result dependent from seed" Tags="|c++|monte-carlo|random-number-generation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28104" PostTypeId="1" CreationDate="2017-10-20T20:29:30.323" Score="4" ViewCount="315" Body="&lt;p&gt;For using any programming language, a mathematical formula should be written in the corresponding code. I wonder if there is any service (for any programming language, Matlab, Mathematica, Python, etc) to directly translating a mathematical formula (e.g., written in TeX) into the programming code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input: a mathematical formula in TeX&#xA;Output: the corresponding code to be executed in Matlab, Mathematica, Python, etc&lt;/p&gt;&#xA;" OwnerUserId="25735" LastActivityDate="2020-07-19T18:08:43.980" Title="Writing a programming code directly from the mathematical formula?" Tags="|matlab|python|mathematica|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28105" PostTypeId="2" ParentId="28104" CreationDate="2017-10-20T21:24:03.393" Score="5" Body="&lt;p&gt;Julia's macros and string macro are made for doing this kind of thing. You can define a domain-specific language (DSL) within a macro scope and use that as a way to define problems. One example of this is the &lt;a href=&quot;http://diffeq.sciml.ai/latest/analysis/parameterized_functions.html#Function-Definition-Macros-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl &lt;code&gt;@ode_def&lt;/code&gt; macro&lt;/a&gt;. You may have to use a string macro (due to parsability into ASTs) but this sort of tool is what allows you to auto-translate syntax. Some interesting examples of string macros are the &lt;a href=&quot;https://github.com/JuliaInterop/MATLAB.jl#the-mat-custom-string-literal&quot; rel=&quot;nofollow noreferrer&quot;&gt;MATLAB.jl&lt;/a&gt; &lt;code&gt;mat&amp;quot;&amp;quot;&lt;/code&gt; strings which allows you to write MATLAB code and have it execute directly in MATLAB:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mat&amp;quot;&amp;quot;&amp;quot;&#xA;    &lt;span class=&quot;math-container&quot;&gt;$u = $&lt;/span&gt;x + &lt;span class=&quot;math-container&quot;&gt;$y&#xA;    $&lt;/span&gt;v = &lt;span class=&quot;math-container&quot;&gt;$x - $&lt;/span&gt;y&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Similarly the Rcall.jl &lt;code&gt;r&amp;quot;&amp;quot;&lt;/code&gt; string which runs R code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R&amp;quot;rnorm(10)&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Those call to other languages, whereas a string macro like &lt;a href=&quot;https://github.com/swadey/LispSyntax.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;LispSyntax.jl&lt;/a&gt; just translates Clojure syntax to Julia:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lisp&amp;quot;(fib 30)&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So writing one that converts from LaTeX to standard Julia is definitely possible, but it looks no one has hit that exact niche yet.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:08:43.980" LastActivityDate="2020-07-19T18:08:43.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28106" PostTypeId="1" AcceptedAnswerId="28107" CreationDate="2017-10-20T22:30:42.147" Score="2" ViewCount="253" Body="&lt;p&gt;&lt;strong&gt;The equation and its meaning:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider two sets $(A)_{l=0,...,m_a},$,$(B)_{l=0,...,m_b}$ of hermitian matrices and a set of positive semidefinite matrices $(C)_{l=0,...,m_c}$. Each matrix has the dimension $n \times n$. These form the ODE:&#xA;$$&#xA;y^\prime(t) = \sum_{i=0}^{m_a} (A_iy+yA_i^\dagger) + \sum_{i=0}^{m_b}(B_iy-yB_i^\dagger) + \sum_{i=0}^{m_c}C_iyC^\dagger_i. \qquad(*)&#xA;$$&#xA;Note that y(0) is a hermitian $n\times n$ matrix too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eqs. like $(*)$ describe the norm-preserving evolution of a linear operator with respect to the anti-commutator (1st), commutator (2nd) and &quot;basis-transform&quot; (3rd). Eqs. like this occur often in Quantum mechanics, see &quot;master equations&quot; for more information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem with vectorization of the matrix matrix form:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common ODE solver operates on vectors. So say x(t) is a vector that contains all columns of y(t) then the ODE becomes with the Kronecker product $\otimes$:&#xA;$$&#xA;x^\prime(t) = \left(\sum_{i=0}^{m_a} (1 \otimes A_i + A_i^*\otimes 1) + \sum_{i=0}^{m_b}(1 \otimes B_i - B_i^*\otimes 1) + \sum_{i=0}^{m_c}C_i^*\otimes C_i\right)x \equiv Xx(t).&#xA;$$ &#xA;But X is a full $n^2 \times n^2$ matrix! Note that the Jacobi-Matrix coincides with X. So each call to $Xx(t)$ is asymptotically worse than the matrix-matrix multiplication above. Thus, I'm looking for an algorithm to solve the ODE in the matrix form (*) for $y(t_{i=0,...,m_t})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A few additional properties&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The eigenvalues of the matrices may vary over several magnitudes,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The dimension of $n$ is never bigger than $10^3$, usually it is between $10^1 -10^2$,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The matrices $A_i,B_i,C_i$ may be sparse as single ones, but the sum, i.e. X, is not sparse,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The eigenvalues can be negative,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stiffness and time-dependent scaling of the matrices, i.e. $A_j \to \alpha(t)_j A_j$, can occur.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Precision should be at least $10^{-5}$, but never more than  $10^{-8}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I've tried&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know ODEPACK, but they rely on the Jacobian, which is unnecessary big here. RK4 fails, except for very, very small step sizes. So, are they solutions which are easy to adapt for this problem, preferable in Fortran?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/3216/algorithms-for-linear-system-of-odes&quot;&gt;Algorithms for linear system of ODEs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/3262/how-to-choose-a-method-for-solving-linear-equations?rq=1&quot;&gt;How to choose a method for solving linear equations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25736" LastEditorUserId="25736" LastEditDate="2017-10-21T07:57:05.130" LastActivityDate="2020-07-19T18:07:48.387" Title="Method to solve linear, first order ODE of generalized matrix matrix form" Tags="|linear-algebra|ode|linear-solver|stability|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28107" PostTypeId="2" ParentId="28106" CreationDate="2017-10-21T08:13:40.103" Score="7" Body="&lt;p&gt;There are a few ways you can do this. Even though RK4 fails unless stepsizes are really small, one thing that can happen a lot is that the model can start out more stiff than it really is in the full timespan. Thus I would still try something adaptive like MATLAB's &lt;code&gt;ode45&lt;/code&gt; or Julia's &lt;code&gt;Tsit5()&lt;/code&gt; before ruling out non-stiff solvers. If that fails, then I would try something made for large equations like Sundials. You can directly call Sundials from something like DifferentialEquations.jl and &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html#Sundials.jl-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;use the options for linear solver choices&lt;/a&gt; and see if anything does well enough. If nothing does well enough, then you need to get fancy.&lt;/p&gt;&#xA;&lt;p&gt;Then there are three options. The problem I see here is you want to avoid vectorization / building the matrix and instead want to do something that is matrix-free. The most standard option is then option 1: use a matrix-free representation of your operator in a Krylov subspace method. So take any stiff solver like BDF, Rosenbrock, SDIRK, etc. What you want to do is re-define the linear solver that it uses to not use a factorization method, but instead use an iterative solver like GMRES and do so without constructing the matrices. In Julia, you can do this by defining a &lt;code&gt;A = LinearMap{T}(f)&lt;/code&gt; from &lt;a href=&quot;https://github.com/Jutho/LinearMaps.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;LinearMaps.jl&lt;/a&gt; where &lt;code&gt;f&lt;/code&gt; defines the matrix-vector product (your equation *).&lt;/p&gt;&#xA;&lt;p&gt;Now you can use &lt;code&gt;A&lt;/code&gt; in &lt;a href=&quot;https://github.com/JuliaMath/IterativeSolvers.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;IterativeSolvers.jl&lt;/a&gt; (or any library which allows abstract types) to define how to solve &lt;code&gt;Ax=b&lt;/code&gt; using your * without having to do the slow path of forming &lt;code&gt;X&lt;/code&gt;. Great! We're almost there. Now we have to get it working like this in the ODE solver. The trick is that the &lt;code&gt;LinearMap&lt;/code&gt; type knows how to compose linear maps, so if you do &lt;code&gt;I - gamma*A&lt;/code&gt; (&lt;code&gt;I&lt;/code&gt; is a type for the identity matrix in Julia, and &lt;code&gt;gamma&lt;/code&gt; here is a scalar) it will automatically generate another &lt;code&gt;LinearMap&lt;/code&gt; which can also be used in an iterative solver. This is the fact we want to exploit.&lt;/p&gt;&#xA;&lt;p&gt;I am sad to say that right now you don't have full control over the internal Jacobian typing so there is a quick modification to the algorithms that would be required. But let's say you want to use the 2nd order adaptive Rosenbrock method. Then you'd &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/integrators/rosenbrock_integrators.jl#L49-L64&quot; rel=&quot;nofollow noreferrer&quot;&gt;go to the code which defines the implicit system&lt;/a&gt; and make it so that way the Rosenbrock W is simply&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;W = I - γ*A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(you'll need to do scoping to define A in there, etc. etc. but that's standard stuff). Just see &lt;a href=&quot;https://www.mathworks.com/help/pdf_doc/otherdocs/ode_suite.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shampine's MATLAB ODE Suite paper which describes this algorithm&lt;/a&gt; and it's pretty clear exactly why this is the needed change. Then use the &lt;a href=&quot;http://diffeq.sciml.ai/latest/features/linear_nonlinear.html#Linear-Solvers:-linsolve-Specification-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;linear solver specification&lt;/a&gt; to tell it to use that matrix-free GMRES to solve the implicit equations. And there you go, that's an adaptive stiff solver you can use your equation * to solve all of the implicit equations.&lt;/p&gt;&#xA;&lt;p&gt;In a few months I plan on releasing an update to DifferentialEquations.jl's function definitions which will allow you to explicitly define &lt;code&gt;W&lt;/code&gt; so that way Option 1 can be done without modifying the code. But for now that's what you got.&lt;/p&gt;&#xA;&lt;p&gt;Option 2 would be to implement the full method yourself and make all of the linear solves matrix-free. The second order Rosenbrock method really isn't difficult and Shampine explains it well so an implementation where you replace the linear solver would be pretty quick to make.&lt;/p&gt;&#xA;&lt;p&gt;Option 3 would be to use some form of an exponential integrator. There are specialized &amp;quot;Strang splitting&amp;quot; methods which can handle this type of equation well, but I don't know of any implementations. We're building one into DifferentialEquations.jl right now, so you can &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/issues/116&quot; rel=&quot;nofollow noreferrer&quot;&gt;see some of our discussions&lt;/a&gt; and use those plus Hairer's geometric integration methods book to piece together how this is done and implement one yourself before us (it'll be a lot easier since you can make it specifically for your set of operators instead of making it generic. But the hard part is you'll want to code a fast-path for computing &lt;code&gt;expmv&lt;/code&gt;, which is &lt;code&gt;w = exp(A*dt)*v&lt;/code&gt; using a matrix-free &lt;code&gt;A&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:07:48.387" LastActivityDate="2020-07-19T18:07:48.387" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="28108" PostTypeId="1" CreationDate="2017-10-21T10:42:46.120" Score="1" ViewCount="45" Body="&lt;p&gt;I have a CFD solver at hand which can deal with 2 phases and so far is able to exchange momentum and energy across the interface, where both phases meet.&#xA;How this is done is well documented in the following paper &quot;A conservative interface method for compressible flows&quot; by Hu, Khoo and Adams, which publicly available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper, equation (10) is giving the general idea of how to update the conservative variables in so called &quot;cut-cells&quot;, cell's which are cut by the interface (tracked by a level-set method).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this update routine $ \hat{X} $ holds the average energy and momentum exchange. Now I want to extend the code such that it can deal with reactive fronts (like combustion or cavitation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The arising Riemann problem is now solved by a reactive Riemann solver, which provides next to pressure and velocity across the contact discontinuity, also the mass flow through the reactive front.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far all good, but now I am having troubles to adjust $\hat{X}$ such that it transports the mass (density) across the front.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be very grateful for any thoughts or hints.&lt;/p&gt;&#xA;" OwnerUserId="25742" LastEditorUserId="9667" LastEditDate="2017-10-21T14:44:21.923" LastActivityDate="2017-10-21T14:44:21.923" Title="CFD Mass exchange across a reactive front" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28109" PostTypeId="1" CreationDate="2017-10-21T14:18:46.317" Score="1" ViewCount="138" Body="&lt;p&gt;I am trying to start using &lt;code&gt;FEniCS&lt;/code&gt;, but have a problem with the simple hello world examples given in the books. Could you please give me the simplest hello world example for solving the Fick's second law.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial c}{\partial t} = D \frac{\partial^2 c}{\partial x^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we apply the boundary condition to the simplest example of &lt;code&gt;FEniCS&lt;/code&gt; for solving this PDE (e.g., drawing the concentration profiles at given times)?&lt;/p&gt;&#xA;" OwnerUserId="25745" LastActivityDate="2017-10-21T14:18:46.317" Title="Hello word in FEniCS?" Tags="|pde|python|fenics|diffusion|" AnswerCount="0" CommentCount="1" ClosedDate="2017-10-21T17:09:32.003" ContentLicense="CC BY-SA 3.0" />
  <row Id="28110" PostTypeId="2" ParentId="28033" CreationDate="2017-10-21T21:12:05.873" Score="3" Body="&lt;p&gt;So the main issue with this is that you are trying to take uniqueness in your approximation, which is based on solving a difference equation, to infer something about your ill-posed PDE problem, like perhaps some unknown physics (i.e. unknown boundary condition). Unfortunately, the approximation cannot reveal such information because it does not imply anything is hidden in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given you discretize your PDE into the form $u_{k+1} = A u_k$ where $u_k$ is the solution at time $t_k$, you can start to treat it as a difference equation and see what you can understand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know the solution to this difference equation is $u_k = A^k u_0$ and it is trivial to use this result to show uniqueness in the solution for this difference equation. This uniqueness holds basically no matter what form $A$ takes, so you could have some matrix $A$ that represents an ill-posed physics problem and it would still produce a unique solution. The fact it produces a solution does not mean the problem is not still ill-posed or that there's hidden information you can extract, it just means you lost some of that information as you tried to approximate the problem and that loss in information led the approximation to be unique.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-10-21T21:12:05.873" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28111" PostTypeId="1" AcceptedAnswerId="30528" CreationDate="2017-10-22T14:09:09.070" Score="6" ViewCount="694" Body="&lt;p&gt;Given a matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; and a vector &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, is there an efficient method to find the normalized eigenvector of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; that is closest to &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, in that it has maximal overlap. More explicitly, a vector &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; can be decomposed like so&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;v = \sum_j \alpha_j u_j,&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$u_j$&lt;/span&gt; are the eigenvectors of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;. I wish to find the subset for which &lt;span class=&quot;math-container&quot;&gt;$\alpha_j = v \cdot u_j$&lt;/span&gt; are the largest, without solving the full eigenproblem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular I'm interested in Hermitian matrices, often symmetric and often sparse. By efficient I mean that I should be able to find &lt;span class=&quot;math-container&quot;&gt;$R&amp;lt;N$&lt;/span&gt; eigenvectors of an &lt;span class=&quot;math-container&quot;&gt;$N \times N$&lt;/span&gt; matrix with the largest overlap with better scaling than diagonalising the full matrix then sorting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this is an interesting question because all algorithms (at least that I know) work by utilising the properties of the dominant eigenvalue - that multiple applications of the matrix projects onto the eigenvector with that eigenvalue. By shifting/inverting it is possible to pick which should be the dominant eigenvalues. It therefore seems that these algorithms fundamentally can't pick out specific eigenvectors only eigenvalues. I am wondering if it is even possible to select a particular eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this question has been asked over three years ago on the maths stackexchange and over 2 years ago on this stackexchange, but with no answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/692468/find-the-eigenvector-with-maximum-overlap&quot;&gt;https://math.stackexchange.com/questions/692468/find-the-eigenvector-with-maximum-overlap&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/17465/calculating-eigenvector-components-of-a-given-vector&quot;&gt;calculating eigenvector components of a given vector&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution (added 31 Jan 2019) -------------------------------------------------------------&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you to demaregee for pointing me to the paper &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://journals.aps.org/prb/abstract/10.1103/PhysRevB.66.245104&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.aps.org/prb/abstract/10.1103/PhysRevB.66.245104&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which provides a nice solution to the problem using a variation of the Jacobi-Davidson alogirthm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is to consider instead the projected eigenproblem&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;V^\dagger M V c = \theta c, &#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$N\times K$&lt;/span&gt; matrix where the columns are orthonormal vectors which span the &quot;test space&quot;, and &lt;span class=&quot;math-container&quot;&gt;$u = Vc$&lt;/span&gt; is the approximation of the vector. The problem is then how to choose and update the test space until u converges to the correct eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; be our current best solution and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; our test space. We then solve the correction equation&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;(M - \theta I) z = -r, \qquad r = (1 - VV^\dagger)(M - \theta I)u,&#xA;\end{equation}&lt;/span&gt;&#xA;for the correction vector &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is our current approximation of the eigenvalue and &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; the current approximation of the target vector. We then orthogonalize &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, normalize, and then extend the test space &lt;span class=&quot;math-container&quot;&gt;$V \rightarrow [Vz]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then solve the projected eigenproblem and take the solution &lt;span class=&quot;math-container&quot;&gt;$u_j = Vc_j$&lt;/span&gt;  that has the largest overlap with &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; as the next approximation to &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and update the corresponding eigenvalue &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. We start with the test space being the single vector &lt;span class=&quot;math-container&quot;&gt;$V = v$&lt;/span&gt;, the vector we want to target and the whole process is repeated until &lt;span class=&quot;math-container&quot;&gt;$\|(M - \theta I)u \| &amp;lt; \epsilon$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is our chosen accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm can be made more complicated by adding restarts that limit the number of vectors in the test space. We can also target multiple eigenvectors by projecting out all converged vectors from the problem.&lt;/p&gt;&#xA;" OwnerUserId="25756" LastEditorUserId="25756" LastEditDate="2019-02-08T15:11:32.477" LastActivityDate="2019-02-08T15:11:32.477" Title="Eigenvector with maximum overlap" Tags="|linear-algebra|algorithms|matrix|eigenvalues|eigensystem|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="28112" PostTypeId="1" CreationDate="2017-10-22T16:23:01.447" Score="8" ViewCount="438" Body="&lt;p&gt;Iterative eigensolvers such as ARPACK, give the option to find a subset of the eigenvalues which have the largest imaginary part. My question is how do these algorithms work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, generally such algorithms use Krylov spaces and shift-invert methods which are very good at finding eigenvalues either at the extremes of the spectrum or around a given value. This is essentially because repeated applications of a matrix projects onto the eigenvector with largest magnitude eigenvalue. However, since the imaginary part of an eigenvalue need not have any correlation with its real part, I can't see how it's able to `ignore' the real part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has any idea how these algorithms work, or alternatively know a way to search for eigenvalues which is ignorant of the real part of the eigenvalue I'd be very grateful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I previously asked this question on the maths stackexchange but didn't receive any answers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/2437587/eigenvalue-with-largest-imaginary-part&quot;&gt;https://math.stackexchange.com/questions/2437587/eigenvalue-with-largest-imaginary-part&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25756" LastActivityDate="2018-06-06T22:04:14.537" Title="Eigenvalue with largest imaginary part" Tags="|linear-algebra|matrix|eigenvalues|eigensystem|arpack|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28113" PostTypeId="1" CreationDate="2017-10-21T23:24:19.293" Score="4" ViewCount="164" Body="&lt;p&gt;I'm working on trying to extract the eigenvectors from a series of observations from a random variable, by using the PAST algorithm, see e.g. 6.2.3 in this book:&#xA; &lt;a href=&quot;https://www.cs.helsinki.fi/u/ahyvarin/papers/bookfinal_ICA.pdf&quot; rel=&quot;noreferrer&quot;&gt;Large pdf&lt;/a&gt;.&#xA;I don't understand the cost function as described&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wo792.png&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;The citation refers to &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;B.Yang.Projection Approximation Subspace Tracking. IEEE&#xA;  Transactions on Signal Processing, 43:95–107, Jan 1995&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Why should any other matrix than the identity matrix minimize the cost function, regardless of the value of the input x? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;I've been told to use this method to find all the eigenvectors, not for compression. Later in the section, the algorithm is presented step by step, and it says to initialise the weight matrix as the n x n unit matrix. The term seems ambiguous; if I supply the identity matrix, it never gets updated. If I on the other hand supply a matrix with only ones, the resulting matrix is always of the type  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{matrix}&#xA;  a &amp;amp; b &amp;amp; c \\&#xA;  a &amp;amp; b &amp;amp; c \\&#xA;  a &amp;amp; b &amp;amp; c&#xA; \end{matrix}&#xA;I.e. the columns are clearly not independent.&lt;/p&gt;&#xA;" OwnerUserId="25759" OwnerDisplayName="knajp" LastActivityDate="2018-04-20T20:16:52.513" Title="Using the PAST algorithm to find eigenvectors" Tags="|statistics|machine-learning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28114" PostTypeId="2" ParentId="28113" CreationDate="2017-10-22T06:37:21.353" Score="1" Body="&lt;p&gt;If the identity matrix is a feasible solution, then clearly it is an optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, most of the time, the identity matrix is not a feasible solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$W\in \mathbb{R}^{m \times d}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where usually, $m$ is chosen to be less than $d$ to perform compression/ dimension reduction.&lt;/p&gt;&#xA;" OwnerUserId="22260" OwnerDisplayName="Siong Thye Goh" LastActivityDate="2017-10-22T06:37:21.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28115" PostTypeId="1" AcceptedAnswerId="28117" CreationDate="2017-10-22T20:45:28.347" Score="3" ViewCount="90" Body="&lt;p&gt;So i have been working (as an undergrad, by working i mean &quot;Redoing a few things my professor does&quot;) in a SIRS model for epidemies. SIRS stands here for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Susceptible -&gt; Infected -&gt; Recovered -&gt; Susceptible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the system of 1st order differential equations that rule the model is:&#xA;$$ S'(t) = - \beta I(t) S(t) + \mu R(t)$$&#xA;$$ I'(t) = \beta I(t) S(t) - \gamma I(t)$$&#xA;$$ R'(t) = \gamma I(t) - \mu R(t)$$&#xA;Where:&#xA;$R'(t)$ stands for derivative of $R$ with respect to time $t$ (the same holds for $I$ and $S$). $\gamma, \beta$ and $\mu$ are constants, that depend on the properties of the diesase (if its very contagious, if it has a high chance of killing you, and so on). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i intended to solve these equations using Euler's Method in &lt;strong&gt;Python3&lt;/strong&gt;, here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while t&amp;lt;140:&#xA;  Sold = S&#xA;  Iold = I&#xA;  S = S + dt*(u*R - B*I*S)&#xA;  I = I + dt*(B*I*S - G*I)&#xA;  R = R + dt*(G*I - u*R)&#xA;  t = t + dt&#xA;  Arq.write(&quot;{} {} {} {} {} \n&quot;.format(t,S,I,R,R+I+S))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice that there is a definition of &lt;em&gt;old&lt;/em&gt; variables. I was using them to save the previous value of the function before using it in the other equation. (I mean, instead of updating $R$ using the new value of $I$, use the old value of $I$, before being updated in the line above). I stopped using the old variables and nothing changed &lt;em&gt;graphically&lt;/em&gt; speaking. If you want the code to run in using old variables, here it is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while t&amp;lt;140:&#xA;  Sold = S&#xA;  Iold = I&#xA;  S = S + dt*(u*R - B*I*S)&#xA;  I = I + dt*(B*I*Sold - G*I)&#xA;  R = R + dt*(G*Iold - u*R)&#xA;  t = t + dt&#xA;  Arq.write(&quot;{} {} {} {} {} \n&quot;.format(t,S,I,R,R+I+S))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Okay. So let's now talk about what is happening. I'm using a 'normalized' population, i.e, S + I + R = 1 everytime. So, i choosed arbitrary values between 0 and 1 for the constants $\mu, \gamma$ and $\beta$, and set the following initial values for $I,R$ and $S$: $I(0) = 0.1$, $R(0) = 0.05$ and $S(0) = 0.85$. I got the following result, in a graphic population $\times$ time:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Vf45c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vf45c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm sorry, i don't know how to reduce the image size here). So now, i'll change the initial values to very different values: $S(0) = 0.20$, $R(0) = 0.55$, and $I(0) = 0.25$. Here is the result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NdkO6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NdkO6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;And what we can see here is that, despite the initial values being &lt;em&gt;very different&lt;/em&gt;, the two cases seem to converge to the same values of the three populations. I won't put other images here otherwise this question would be a kilometer long, but all the initial conditions i've put didn't changed the result. (or didn't seem to). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is this?&lt;/strong&gt; Is my model correct?If not, what is the problem with it? If it is correct, can this be explained mathematically? This is a model that tries to simulate diseases. Are there any example of a non-killing disease that had (or have) this property?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&#xA;(I would like to add that an edit to add colors to the code and reduce the image's size would be very appreciated (especially if the one who did it explained how! :))&lt;/p&gt;&#xA;" OwnerUserId="24769" LastEditorUserId="24769" LastEditDate="2017-10-22T23:39:29.890" LastActivityDate="2017-10-23T15:55:07.577" Title="SIRS Model doesn't depend on initial conditions?" Tags="|numerical-analysis|differential-equations|computational-biology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28117" PostTypeId="2" ParentId="28115" CreationDate="2017-10-23T15:50:05.640" Score="3" Body="&lt;p&gt;Your model seems to have a single stable critical point that attracts&#xA;all orbits. You're plotting $(S(t), I(t), R(t))$ as functions of time&#xA;for specific initial conditions, but actually the easier way to see&#xA;this directly is to plot the vector field $(\dot S, \dot I)$ in the&#xA;$S$-$I$ plane like so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TH6VU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TH6VU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Immediately you can see that for this particular choice of parameters&#xA;(I don't remember which values I used), every solution of the ODE will&#xA;converge to the same critical point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To analyze this behaviour mathematically, you first identify the&#xA;location of the critical point by solving $\mathrm{rhs}(S_0,I_0,R_0) =&#xA;0$ ($R=1-S-I$), and then linearize the ODE around that critical point&#xA;and look at the eigenvalues of the Jacobian of the r.h.s. This doesn't&#xA;quite show that &lt;em&gt;all&lt;/em&gt; orbits converge to the critical point, only&#xA;those that start close enough to it. To show that &lt;em&gt;all&lt;/em&gt; orbits&#xA;converge to the same critical point you could look for a &lt;a href=&quot;https://en.wikipedia.org/wiki/Lyapunov_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lyapunov&#xA;function&lt;/a&gt; for your system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are all very standard topics discussed in any ODE or dynamical&#xA;systems textbook. One recent book that I liked was &lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/ExplODE/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Exploring ODEs&lt;/em&gt;&#xA;by Trefethen, Birkisson,&#xA;Driscoll&lt;/a&gt; (free&#xA;online), look at Chapters 15–16.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I stopped using the old variables and nothing changed graphically speaking.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Imagine you're solving $\dot y = f(y)$ and you make a small error and&#xA;you accidentally end up solving $\dot y = f(y) + \delta(t)$. In the&#xA;case of $\dot I$, $\delta$ would be $\beta I (S(t_{n+1}) -&#xA;S(t_n))\,\delta t$, although this expression isn't very well&#xA;defined mathematically. This $\delta$ is merely a perturbation of size $|\dot S|\delta&#xA;t^2$. If you used a higher-order method that forward Euler you would&#xA;identify this by noticing that your method doesn't converge with the&#xA;right order. Instead of accurately solving the right ODE, you&#xA;(accurately) solve a slightly wrong ODE, and the difference for your&#xA;model cannot be seen because it just converges to the same critical&#xA;point. Other ODEs with higher-order methods wouldn't be this forgiving.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2017-10-23T15:55:07.577" LastActivityDate="2017-10-23T15:55:07.577" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28119" PostTypeId="1" AcceptedAnswerId="28122" CreationDate="2017-10-23T18:46:46.660" Score="1" ViewCount="177" Body="&lt;p&gt;This &lt;a href=&quot;https://stackoverflow.com/questions/46892953/how-to-solve-a-polynomial-with-multiple-solutions-including-complex-solutions&quot;&gt;question&lt;/a&gt; on StackOverflow has led me to ask myself what would be involved in solving an equation like this one using tools available to the Python programmer.&#xA;$$ \frac{0.125567841}{d^{2.25}} = \frac{2.513274d+0.10053+2.11\pi}{d+0.04}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the fundamental numerical tools that one would use to get the one real and four complex roots of this equation (&lt;a href=&quot;https://www.wolframalpha.com/input/?i=solve+0.12556841%2Fd%5E2.25+%3D+(2.513274+d+%2B+0.10053%2B2.11+pi)%2F(d+%2B+0.04)+for+d&quot; rel=&quot;nofollow noreferrer&quot;&gt;WolframAlpha&lt;/a&gt;)? I have looked (hard) at the Mathematica documentation but cannot discern what they might be. It's probably mainly ignorance on my part.&lt;/p&gt;&#xA;" OwnerUserId="22133" LastEditorUserId="6775" LastEditDate="2017-10-24T18:27:10.273" LastActivityDate="2017-10-24T18:27:10.273" Title="How does Mathematica compute real and complex solutions to single, non-polynomial equations?" Tags="|numerical-analysis|algorithms|nonlinear-equations|rootfinding|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28121" PostTypeId="2" ParentId="7369" CreationDate="2017-10-24T01:29:08.787" Score="3" Body="&lt;p&gt;I would like to comment on Daniel Shapero's answer but I don't have enough SE reputation.&lt;/p&gt;&#xA;&lt;p&gt;The accepted answer confuses me a lot. I think shift-invert mode can be readily used to compute interior eigenvalues. See:&#xA;&lt;a href=&quot;https://docs.scipy.org/doc/scipy/tutorial/arpack.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/scipy/tutorial/arpack.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To answer the original question: it's rarely the case that you want all eigenvalues of a large sparse matrix. Usually, you want extrems or some cluster of interior values. In that case, for a Hermitian matrix &lt;code&gt;eigsh&lt;/code&gt; is faster. For non-Hermitian, you will have to go with &lt;code&gt;eigs&lt;/code&gt;.&#xA;And they are much faster than numpy &lt;code&gt;eig&lt;/code&gt; or &lt;code&gt;eigh&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25779" LastEditorUserId="3481" LastEditDate="2023-09-22T18:33:32.390" LastActivityDate="2023-09-22T18:33:32.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28122" PostTypeId="2" ParentId="28119" CreationDate="2017-10-24T04:43:55.893" Score="4" Body="&lt;p&gt;You can rewrite with integer exponents through a change of variable. This looks like $d=x^r$ where $r$ is the LCM of the denominator of the rationalized exponents. In your case we have $r=4$ and:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$6.72427 - 0.125568 x^4 + 2.51327 x^{13}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which can be solved in the usual ways. In &lt;em&gt;Mathematica&lt;/em&gt; there is a clear distinction between symbolic and numerical solutions. You have floating point numbers bouncing around, so the solution that pops out is a result of numerical approximation. These are described in their extended internal documentation: &lt;a href=&quot;http://reference.wolfram.com/language/tutorial/SomeNotesOnInternalImplementation.html#2118&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://reference.wolfram.com/language/tutorial/SomeNotesOnInternalImplementation.html#2118&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 13 roots lie along a circle in the complex plane:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AKLIK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AKLIK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2017-10-24T04:43:55.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28123" PostTypeId="1" CreationDate="2017-10-24T06:44:44.777" Score="0" ViewCount="284" Body="&lt;p&gt;My variables are stored at the center of the cells. How can I transfer these values to grid points? If I calculate the algebraic average value there may be a shock.&lt;/p&gt;&#xA;" OwnerUserId="25695" LastEditorUserId="20688" LastEditDate="2018-08-11T05:00:47.170" LastActivityDate="2018-10-10T06:00:04.157" Title="How to represent CFD result when I use grid-centered FVM?" Tags="|fluid-dynamics|interpolation|visualization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="28125" PostTypeId="1" AcceptedAnswerId="28128" CreationDate="2017-10-24T08:40:43.813" Score="0" ViewCount="87" Body="&lt;p&gt;Consider a non-convex distributed optimization problem.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We have $X$ = a set of $n$ decision variables: $x_i$ where $i=1..n$ and $x_i \in R$, the set of Reals.&lt;/li&gt;&#xA;&lt;li&gt;We have $F$ = a set of $m$ constraint cost functions: $f_j($&lt;strong&gt;x&lt;/strong&gt;$)$ where $j=1..m$ and each $f_j$ is defined for a subset of $X$ and each $f_j$ is known only to the respective $x_i$'s $\in X$. We assume each variable to be an agent in a distributed multiagent system.  &lt;/li&gt;&#xA;&lt;li&gt;We have $D = \{d_1, d_2, ..d_n\}$, a set of domains for each $x_i$ where $d_i$ is of the form: $l_i&amp;lt;=x_i&amp;lt;=u_i$ i.e. $x_i$ can take values between $l_i$, the lower bound and $u_i$, the upper bound. Hence each $x_i$ is bounded in the space of real numbers.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We have to $minimize \sum_{j=1}^{m} f_{j}($&lt;strong&gt;x&lt;/strong&gt;$)$ which is the global objective. We also have to satisfy certain hard constraints (e.g.: $x_1 = x_2 + x_3$, $x_2+x_5&amp;lt;=5.5$, etc.) - these are again known to the respective variables/agents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of the $f_j$'s could be non-convex Eg:$f_2(x_2, x_3) = x_2^3 + x_3^3 + x_2x_3$, etc. which may lead to a hills and valleys terrain for the overall objective function (something like in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/23948/why-should-non-convexity-be-a-problem-in-optimization&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question in the exam was about what approach one would use to solve such a problem.&#xA;Begin familiar with Distributed Constraint Optimization Problems (&lt;a href=&quot;https://en.wikipedia.org/wiki/Distributed_constraint_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;DCOPs&lt;/a&gt;), I thought that one could &lt;strong&gt;&lt;em&gt;discretize&lt;/strong&gt; the continuous space to integers&lt;/em&gt; (or into even more granularity if required, depending on the domain - this can be thought of as sampling) and use a DCOP algorithm to solve this problem (since DCOP algorithms don't make convexity assumptions and are general purpose) Once we get the integer solutions, we could do a distributed gradient descent (assuming that we've ended up in a convex space) to get a solution between integers (do this if needed).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that we will (may) not get an exact real valued solution because of the granularity of the discretization and that this isn't mathematically elegant by any means, but isn't this one of the best possible approaches &lt;strong&gt;in practice&lt;/strong&gt;, given that we have the conditions of distributed optimization (assuming that the problem is inherently distributed and cannot be solved centrally) and that we have optimal, complete DCOP algorithms? Is this a feasible approach or am I missing something? If I am missing something, what is the state of the art? If this is a feasible approach, how may I prove to the professor that this approach works in practice?&lt;/p&gt;&#xA;" OwnerUserId="25781" LastActivityDate="2017-10-24T15:14:15.353" Title="Why can't we discretize continuous domains in distributed non-convex constraint optimization problems?" Tags="|optimization|nonlinear-programming|nonconvex|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28126" PostTypeId="2" ParentId="28123" CreationDate="2017-10-24T12:01:23.203" Score="1" Body="&lt;p&gt;As mentioned in a previous comment, this depends on what your goal is. Modern visualization software such as Paraview allow a representation of data at cell centers or at cell vertices. Furthermore, you can switch on the fly and interpolate the data from the cell center to the vertices and vice-versa. In paraview this is achieved using the &quot;Cell Data to Point Data&quot; Filter (to go from cell center to vertices) and the Point Data to Cell Data (to go from cell vertices to cell center). I suggest you look into that, this is an easy way to switch from one visualisation option to another. However, I suggest that except for the &quot;making nice movie part&quot;, you focus on visualizing the data at the cell center because this can give you a lot more information. If you have occurence of say, checkboard effect, those might end-up being smudged out when using the point data representation because of the interpolation.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2017-10-24T12:22:59.460" LastActivityDate="2017-10-24T12:22:59.460" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28127" PostTypeId="1" CreationDate="2017-10-24T12:29:58.457" Score="1" ViewCount="61" Body="&lt;p&gt;I want to filter on 5000 stocks, each of which has 4 month or more worth of data (&gt;= 500 data points each). my filtering criteria will be based on 8 calculated values from the data points. for example, moving average, standard deviation and values alike.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my tools of choice are python pandas, numpy and CUDA C++; please take this as the default setting which can be expanded on, especially on the C++ side, I believe I can expand on Linear Algebra packs but I am not very familiar with them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;my current design is to: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;read the data into pandas dataframe.&lt;/li&gt;&#xA;  &lt;li&gt;pass one stock at a time to C++ to compute all filtering criteria values.    here I plan to use CUDA to speed up the calculations for&#xA;  each criteria value.    e.g. running moving average calculation&#xA;  parallel on CUDA cores.&lt;/li&gt;&#xA;  &lt;li&gt;pass a list containing filter criteria values of each stock back to python. &lt;/li&gt;&#xA;  &lt;li&gt;sorting the stocks in terms of the filter criteria values; each criterion will weigh differently when I do the sorting. &lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am still exploring, I know there will be better design/strategy with different tool set, but with these tools as given, is there a better strategy? Can I use Linear Algebra to speed this even further (potentially less codes)?&lt;/p&gt;&#xA;" OwnerUserId="25784" LastEditorUserId="25784" LastEditDate="2017-10-24T13:42:48.467" LastActivityDate="2017-10-24T13:42:48.467" Title="Algorithm design to filter on 5,000 stocks each of which has 4 months worth of data points" Tags="|linear-algebra|optimization|algorithms|parallel-computing|cuda|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28128" PostTypeId="2" ParentId="28125" CreationDate="2017-10-24T15:14:15.353" Score="3" Body="&lt;p&gt;You are missing that, in general, &lt;em&gt;integer optimization problems are much more expensive to solve&lt;/em&gt; than real-valued optimization problems. That's because for real-values optimization problems, you have the tools of gradients and Hessians that can guide you to a minimum, whereas the integer equivalents are at best crutches that can help you, but are not nearly as efficient. In other words, unless you discretize each variable into only a few discrete values, and unless you have only a small number of variables, discretization will make your problem vastly more expensive to solve.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-24T15:14:15.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28129" PostTypeId="2" ParentId="28119" CreationDate="2017-10-24T16:22:00.963" Score="5" Body="&lt;p&gt;As the other Answer already touches on the possibility of a symbolic root-solver being applied to this particular equation (by transforming into a polynomial form, albeit of degree $\ge 5$), I'll make some remarks about numerical root-finding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we want to find &lt;em&gt;real roots&lt;/em&gt; of a &lt;em&gt;continuous function&lt;/em&gt;, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Intermediate_value_theorem&quot; rel=&quot;noreferrer&quot;&gt;Intermediate Value Theorem&lt;/a&gt; is a basic tool.  On a closed bounded interval where the real function is continuous and &lt;em&gt;changes sign&lt;/em&gt; between endpoints, there will be a real root somewhere in the interior of that interval.  The &lt;a href=&quot;https://en.wikipedia.org/wiki/Bisection_method&quot; rel=&quot;noreferrer&quot;&gt;Bisection Method&lt;/a&gt; allows us to approximate at least one such root in that interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here we can transform the equation into finding roots of a real continuous function by cross-multiplying and collecting terms on one side:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{0.125567841}{d^{2.25}} = \frac{2.513274d+0.10053+2.11\pi}{d+0.04} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ 0.125567841(d+0.04) = (2.513274d+0.10053+2.11\pi)d^{2.25} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(d) \equiv 2.513274d^{3.25} + (0.10053+2.11\pi)d^{2.25} - 0.125567841d - 0.00502271364 = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This form of the problem is easier to analyze, but we should check whether the cross-multiplication introduced &lt;a href=&quot;https://math.stackexchange.com/a/594086/3111&quot;&gt;artifact roots&lt;/a&gt;, i.e. roots for the revised equation that do not satisfy the original equation.  When we multiply both sides by the denominators $d^{2.25}$ and $d+0.04$, it is possible that this introduces one or more new roots where such factors are zero, i.e. where $d=0$ or $d=-0.04$.  Fortunately $F(0)\neq 0$ and $F(-0.04)\neq 0$, so we have &lt;em&gt;not&lt;/em&gt; introduced artifact roots in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One should be careful to define what (if anything) $d^{2.25}$ means for &lt;em&gt;negative&lt;/em&gt; real values of $d$.  Inevitably one must abandon any &lt;em&gt;real value&lt;/em&gt; for this expression because it requires taking square roots of a negative number.  This is a particular weakness of relying on tools such as Wolfram Alpha, because it attempts to be helpful by aggressively assuming that &lt;em&gt;some sense&lt;/em&gt; is to be made of the expression even for negative arguments $d$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately in this case what roots Wolfram Alpha finds are &lt;em&gt;not&lt;/em&gt; on the negative half of the real axis for $d$.  There is a positive real root $d\approx 0.06087$, and the other roots found are in complex conjugate pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real root can be found by standard methods such as bisection once we realize that $F(d)$ is continuous and changes sign on the interval $[0,1]$.  That is, $F(0)=-0.00502271364$ and $F(1)\approx 9.11197$, so there must be an intermediate value of $d_0$ where the function $F(d_0)=0$.  Once the location of the root $d_0$ is suitably narrowed by bisection, etc., more rapidly converging methods (like Newton's method) can be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a function like $F(d)$ with real coefficients on rational powers has complex roots, these will occur naturally in complex conjugate pairs.  One can then replace the &lt;em&gt;one dimensional&lt;/em&gt; search for a real root with a &lt;em&gt;two dimensional&lt;/em&gt; search for a complex conjugate root pair $d = x \pm iy$.  A classic approach to this is &lt;a href=&quot;https://en.wikipedia.org/wiki/Bairstow%27s_method&quot; rel=&quot;noreferrer&quot;&gt;Bairstow's Method&lt;/a&gt;, which uses Newton-Raphson iterations to locate a complex conjugate root pair using only real arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as Python users interested in symbolic root-finding should be familiar with &lt;code&gt;sympy&lt;/code&gt;, those interested in numerical root-finding will be attracted to the &lt;code&gt;numpy&lt;/code&gt; library.  In many problems a synthesis of the two approaches is advantageous, e.g. using &lt;code&gt;sympy&lt;/code&gt; to get derivatives of a function which can be evaluated by routines in &lt;code&gt;numpy&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2017-10-24T16:22:00.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28130" PostTypeId="1" AcceptedAnswerId="28132" CreationDate="2017-10-25T01:50:21.560" Score="6" ViewCount="303" Body="&lt;p&gt;I am modeling some aerodynamics equations and am using meters / centimeters, kilograms, and seconds.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've heard that, &quot;matlab doesn't know units&quot;.  So, how can I make sure that it does?  Just simply by being consistent in all of my code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g., if I run an ode solver in matlab for a time scale of [0 30], can I think of it as matlab giving me ode solutions for &quot;30 seconds&quot;?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="25675" LastActivityDate="2020-04-15T23:30:39.767" Title="In Matlab, how can I be consistent with units?" Tags="|matlab|fluid-dynamics|computational-physics|numerical-modelling|modeling|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28131" PostTypeId="2" ParentId="28130" CreationDate="2017-10-25T03:00:38.710" Score="8" Body="&lt;p&gt;&lt;strong&gt;Just simply by being consistent in all of my code?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes this is the only way. Matlab or any other programming language does not know about units. They only know about numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example consider incompressible flow. If you set your velocity in m/sec, length in meters (how you generate the grid), pressure in Newton/m^2, kinematic viscosity in m^2/sec, then the time would be in seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can build code which will respect units. E.g., openfoam does this. You must define your own data types and operations on them. Each data type should have an associated unit. The program should warn you if you try to add/subtract two variables of different units. &lt;strong&gt;Even then, it is your responsibility to set these variables with correct numerical values.&lt;/strong&gt; Suppose v should be in m/sec and you have to set it to 1 m/sec. If you say 1 m/sec = 100 cm/sec and you set v = 100, then that would be wrong, and no program can detect such that mistake.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2017-10-25T03:00:38.710" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28132" PostTypeId="2" ParentId="28130" CreationDate="2017-10-25T03:26:48.127" Score="8" Body="&lt;p&gt;I would say that you have, mainly, two methods:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Being consistent in all your code, as already suggested in another answer. For that purpose, I always keep a table like this one with me, since it might prove really useful.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yO8vW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yO8vW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Use nondimensional equations. That way, all my parameters and variables are consistent already. For that purpose, I suggest reference 1.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Langtangen, Hans Petter, and Geir K. Pedersen. &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/978-3-319-32726-6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scaling of Differential Equations&lt;/a&gt;. Springer, 2016.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2020-04-15T23:30:39.767" LastActivityDate="2020-04-15T23:30:39.767" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="28133" PostTypeId="2" ParentId="27970" CreationDate="2017-10-25T03:50:07.420" Score="3" Body="&lt;p&gt;Essentially, the time dependent Stokes equation looks like the heat equation:&#xA;$$&#xA;  \frac{\partial u}{\partial t} - \nu\Delta u = f-\nabla p,&#xA;$$&#xA;plus the incompressibility condition $\nabla \cdot u=0$ that for the current discussion is immaterial. Thus, the same considerations for time step choice apply as for the heat equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consequently, using an &lt;em&gt;implicit&lt;/em&gt; method such as the backward Euler method generally yields an unconditionally stable method for which you can choose the time step as large as you want. (Although you may want to choose it not too large for accuracy reasons -- stability does not imply accuracy.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, for an &lt;em&gt;explicit&lt;/em&gt; method, you need to choose the time step $\Delta t$ subject to some CFL-like condition that says that &#xA;$$&#xA;  \Delta t \le C \frac{1}{\nu} \Delta x^2&#xA;$$&#xA;where $\Delta x$ is the mesh size. This is not practical: it requires you to choose the time step &lt;em&gt;four times smaller for each mesh refinement&lt;/em&gt;. These time steps would be so small that the time discretization error is vastly smaller than the spatial discretization error, and that's not useful. As a consequence, practical implementations do not choose explicit methods for the Stokes equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[For an explanation of the &lt;em&gt;why&lt;/em&gt; this condition arises, see lecture 27 at &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt; .]&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-25T03:50:07.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28136" PostTypeId="1" CreationDate="2017-10-25T18:00:49.717" Score="0" ViewCount="82" Body="&lt;p&gt;Devil-may-care that I am, suppose that I wanted to 'simplify' the following expression, utterly ignoring the fact that it's very naughty to raise various kinds of numbers to arbitrary powers or to divide by zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{0.125567841}{d^{2.25}} = \frac{2.513274d+0.10053+2.11\pi}{d+0.04}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is to say, I would like to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;substitute $$x^4$$ for  $$d$$, &lt;/li&gt;&#xA;&lt;li&gt;'cross-multiply' the two sides of the equation, and&lt;/li&gt;&#xA;&lt;li&gt;replace the floating point, 2.25, with 9/4.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want to pretend I'm back in school. Is there software that will do this for me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT: Where I wrote the 4th root of &lt;code&gt;x&lt;/code&gt; I should have had the 4th power of &lt;code&gt;x&lt;/code&gt;. My apologies for so much inconvenience!&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="22133" LastEditorUserId="22133" LastEditDate="2017-10-25T19:37:43.883" LastActivityDate="2017-10-25T23:02:43.740" Title="Software that does naïve or formal simplification of mathematical expressions" Tags="|software|libraries|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28137" PostTypeId="1" AcceptedAnswerId="28140" CreationDate="2017-10-25T18:28:26.723" Score="3" ViewCount="364" Body="&lt;p&gt;I'm trying to use Riks' method and I'm not sure how to set the initial values for the loading coefficient, nor the tangent vector (i.e. the derivative of the displacements and loading coefficient with respect to the path length).  At the bottom of page 538, the author assumes that these are known at the outset.  The best I can come up with is to set an initial value for the loading coefficient, small enough so that there is no risk of encountering any critical point, and then taking the solution as both the state vector and its normalized vector as the tangent vector.  Is there a better way to do this?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Riks, E. &quot;An incremental approach to the solution of snapping and buckling problems.&quot; International Journal of Solids and Structures 15.7 (1979): 529-551.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25235" LastEditorUserId="9667" LastEditDate="2017-10-25T19:25:44.683" LastActivityDate="2017-10-26T07:31:28.443" Title="Initialize arc length control in Riks' method" Tags="|nonlinear-equations|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28138" PostTypeId="2" ParentId="28136" CreationDate="2017-10-25T19:17:24.577" Score="3" Body="&lt;p&gt;What you are looking for is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Computer_algebra_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computer Algebra System&lt;/a&gt;. You should be able to do that in Mathematica, Maple, Maxima or SymPy. Particularly, I show an example in SymPy below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sympy as sym&#xA;&#xA;d, x = sym.symbols(&quot;d x&quot;, real=True)&#xA;eq = sym.Eq(0.12556841/d**2.25, (2.513274*d + 0.10053 + 2.11*sym.pi)/(d + 0.04))&#xA;eq_new = sym.nsimplify(eq).subs(d, x**4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the result would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{12556841 \left|{x}\right|}{100000000 x^{10}} = \frac{1}{x^{4} + \frac{1}{25}} \left(\frac{1256637 x^{4}}{500000} + \frac{10053}{100000} + \frac{211 \pi}{100}\right)$$&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2017-10-25T20:01:47.840" LastActivityDate="2017-10-25T20:01:47.840" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28139" PostTypeId="2" ParentId="28136" CreationDate="2017-10-25T23:02:43.740" Score="0" Body="&lt;p&gt;Thanks for the answer and the comments. They definitely helped. This code gets me at least part of the way and may serve to explain what I was seeking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining both of the symbols to be real and positive made it possible for sympy to allow the operations on exponents involved, at least for this situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is indeed a polynomial.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from sympy import *&#xA;&amp;gt;&amp;gt;&amp;gt; x = Symbol('x', real=True, positive=True)&#xA;&amp;gt;&amp;gt;&amp;gt; d = Symbol('d', real=True, positive=True)&#xA;&amp;gt;&amp;gt;&amp;gt; (0.125567841*(d+0.04)-d**Rational(9,4)*(2.513274*d+0.10053+2.11*pi)).subs(d, x**4)&#xA;-x**9*(2.513274*x**4 + 0.10053 + 2.11*pi) + 0.125567841*x**4 + 0.00502271364&#xA;&amp;gt;&amp;gt;&amp;gt; (-x**9*(2.513274*x**4 + 0.10053 + 2.11*pi) + 0.125567841*x**4 + 0.00502271364).expand()&#xA;-2.513274*x**13 - 2.11*pi*x**9 - 0.10053*x**9 + 0.125567841*x**4 + 0.00502271364&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22133" LastActivityDate="2017-10-25T23:02:43.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28140" PostTypeId="2" ParentId="28137" CreationDate="2017-10-25T23:19:21.833" Score="3" Body="&lt;p&gt;It depends on the control which you have, if your control function is linear, for example crack mouth opening control, it will work with zero initial step. For  spherical control you need kick-start analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equilibrium equation is arguments with control equation as follows,&#xA;\begin{equation}&#xA;\left\{&#xA;\begin{array}[l]&#xA;\mathbf{r}(\mathbf{x},\lambda) = \mathbf{F}^{int}(\mathbf{x})-\lambda \mathbf{F}_{\lambda}\\ &#xA;r_\lambda(\mathbf{x},\lambda) = f_\lambda(\mathbf{x},\lambda) - \Delta s^2&#xA;\end{array}&#xA;\right.&#xA;\end{equation}&#xA;where $\mathbf{F}^{int}(\mathbf{x})$ is internal force, $\lambda \mathbf{F}_{\lambda}$ is external force, $\lambda$ is additional unknown controlled by second equation and $\mathbf{x}$ is vector of unknowns. For spherical control $f_\lambda(\mathbf{x},\lambda)$ is  constrain function as follows&#xA;\begin{equation}&#xA;f_\lambda(\mathbf{x},\lambda) = \|\Delta\mathbf{x}\|^2 +&#xA;\beta^2 \Delta\lambda^2 \| \mathbf{F}_{\lambda} \|^2&#xA;\end{equation}&#xA;where the increment vector is&#xA;\begin{equation}&#xA;\Delta \mathbf{x}_1 = \mathbf{x}_1-\mathbf{x}_0.&#xA;\end{equation}&#xA;where $1$ is next step, and $0$ is initial step. The sub increment is &#xA;\begin{equation}&#xA;\Delta \mathbf{x}_1^{i+1} = \Delta \mathbf{x}_1^{i}+\delta \mathbf{x}^{i+1}   &#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linearising both equations we get &#xA;\begin{equation}&#xA;\left[&#xA;\begin{array}{cc}&#xA;\frac{\partial \mathbf{F}^{int}}{\partial \mathbf{x}} &amp;amp; -\mathbf{F}_{\lambda} \\&#xA;2\|\Delta\mathbf{x}\| &amp;amp; 2\Delta\lambda \beta^2 \| \mathbf{F}_{\lambda} \|^2  \\&#xA;\end{array}&#xA;\right]&#xA;\left\{&#xA;\begin{array}{c}&#xA;\delta \mathbf{x}\\&#xA;\delta \lambda&#xA;\end{array}&#xA;\right\}=&#xA;\left[&#xA;\begin{array}{c}&#xA;-\mathbf{r}\\&#xA;-r_\lambda&#xA;\end{array}&#xA;\right]&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Immediately you can see that initial state, you have to have non-zero $\lambda$ or $\Delta\mathbf{x}$. You can get both by solving linearised equation at beginning of analysis for some initial $\lambda$. $\lambda$ has to be small such that structure response in linear range, i.e. need one or two iterations to converge. Then you choose $\beta$, it can be zero, and then calculate $\Delta s$ from second equation. And this is kick-start for analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to avoid problems, too small or too big steps, as an advice arc-length control works the best when you have step adaptivity, i.e.&#xA;\begin{equation}&#xA;\Delta s^{n+1} = \Delta s^{n} \left (\frac{I_d}{I} \right)^\gamma&#xA;\end{equation}&#xA;where $I_d$ is desired number of iterations, $I$ number of iterations at last step $n$, and $\gamma=0.5$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that above is when you linearise both equations. Second equations is quadratic polynomial, so you can solve it analytically. It can have some advantage near bifurcation points, where you can choose right direction. However I prefer to introduce imperfections, and use linearised system of equations.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-10-26T07:31:28.443" LastActivityDate="2017-10-26T07:31:28.443" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="28141" PostTypeId="2" ParentId="27718" CreationDate="2017-10-26T02:51:29.240" Score="1" Body="&lt;p&gt;The velocity and stream function are related by&#xA;$$&#xA;u = \psi_y, \qquad v = - \psi_x&#xA;$$&#xA;This can be related to the vorticity&#xA;$$&#xA;-\Delta \psi = \omega = v_y - u_x&#xA;$$&#xA;On left and right, we have $(u,v)=(0,0)$&#xA;$$&#xA;\psi = const, \quad \psi_x = 0&#xA;$$&#xA;On bottom $(u,v) = (0,0)$&#xA;$$&#xA;\psi_y = 0, \qquad \psi = const&#xA;$$&#xA;On top $(u,v)=(u_0,0)$&#xA;$$&#xA;\psi_y = u_0, \qquad \psi = const&#xA;$$&#xA;You cannot enforce two boundary conditions. Let us use Dirichlet bc&#xA;$$&#xA;\psi = const&#xA;$$&#xA;on all boundaries. Since $\psi$ must be continuous (assuming $\omega$ in $L^2$ atleast), the constant must be same on all sides which you can set to zero.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2017-10-26T02:51:29.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28142" PostTypeId="2" ParentId="26942" CreationDate="2017-10-26T11:47:42.183" Score="8" Body="&lt;p&gt;I realize this question was asked a while ago, but I recently needed the Freschet distance as well.&#xA;I couldn't find any implementations for Python, so I wrote my own based on the paper: &quot;Computing Discrete Frechet Distance&quot; by &quot;Thomas Eiter and Heikki Mannila&quot;, and thought I would share it for future reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's written in Cython (save as frechet.pyx)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Julius Bier Kirkegaard 2017&#xA;# Based on &quot;Computing Discrete Frechet Distance&quot; by &quot;Thomas Eiter and Heikki Mannila&quot;.&#xA;#cython: boundscheck=False&#xA;#cython: wraparound=False&#xA;import numpy as np&#xA;cimport numpy as np&#xA;from libc.math cimport sqrt, cos, sin&#xA;&#xA;cdef double d(double p, double q):&#xA;    return (p - q)*(p - q)&#xA;&#xA;&#xA;cdef double c(int i, int j, double* P, double* Q, double* ca, int N):&#xA;    cdef int index = i*N+j&#xA;    cdef double d_t = d(P[i], Q[j])&#xA;    cdef double m1, m2&#xA;&#xA;    if ca[index] &amp;gt; -1:&#xA;        return ca[index]&#xA;&#xA;    elif i == 0 and j == 0:&#xA;        ca[index] = d(P[i], Q[j])&#xA;&#xA;    elif i &amp;gt; 0 and j == 0:&#xA;        if d_t &amp;gt; ca[(i-1)*N+j]:&#xA;            ca[index] = d_t&#xA;        else:&#xA;            ca[index] = ca[(i-1)*N+j]&#xA;&#xA;    elif i == 0 and j &amp;gt; 0:&#xA;        if d_t &amp;gt; ca[i*N+(j-1)]:&#xA;            ca[index] = d_t&#xA;        else:&#xA;            ca[index] = ca[i*N+(j-1)]&#xA;&#xA;    elif i &amp;gt; 0 and j &amp;gt; 0:&#xA;        m1 = c(i - 1, j, P, Q, ca, N)&#xA;        m2 = c(i, j - 1, P, Q, ca, N)&#xA;        if m2 &amp;lt; m1:&#xA;            m1 = m2&#xA;        m2 = c(i - 1, j - 1, P, Q, ca, N)&#xA;        if m2 &amp;lt; m1:&#xA;            m1 = m2&#xA;        if d_t &amp;gt; m1:&#xA;            ca[index] = d_t&#xA;        else:&#xA;            ca[index] = m1&#xA;    else:&#xA;        ca[index] = 1e50&#xA;&#xA;    return ca[index]&#xA;&#xA;def frechet(np.ndarray[np.float64_t,ndim=1,&#xA;                    negative_indices=False,&#xA;                    mode='c'] P,&#xA;            np.ndarray[np.float64_t,ndim=1,&#xA;                    negative_indices=False,&#xA;                    mode='c'] Q):&#xA;&#xA;    cdef np.ndarray[np.float64_t,ndim=2,&#xA;                    negative_indices=False,&#xA;                    mode='c'] ca = np.zeros((len(P), len(Q))) - 1&#xA;&#xA;    return c(len(P)-1, len(Q)-1, &amp;amp;P[0], &amp;amp;Q[0], &amp;amp;ca[0,0], ca.shape[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pyximport; pyximport.install()&#xA;from frechet import frechet&#xA;import numpy as np &#xA;&#xA;P = np.arange(5, dtype=np.float64)&#xA;Q = 2*np.arange(7, dtype=np.float64)&#xA;&#xA;print P, Q&#xA;&#xA;ca = frechet(P, Q)&#xA;print ca&#xA;&#xA;&#xA;print (P[-1] - Q[-1])**2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14394" LastEditorUserId="14394" LastEditDate="2018-01-06T09:24:52.723" LastActivityDate="2018-01-06T09:24:52.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28143" PostTypeId="1" CreationDate="2017-10-26T15:32:28.573" Score="6" ViewCount="280" Body="&lt;p&gt;I believe this might be a recurring topic, but i have not found a post that directly related to this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I come from a finite volume background and my experience is more with predictor-corrector schemes (such as SIMPLE or PISO), so my knowledge of stabilized finite element (Galerkin Least Square (GLS) or Variable Multi-Scale (VMS) for instance) is limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current understanding is this: P1-P1 (velocity-pressure) elements (lets say on tetrahedron) are known to be invalid for the incompressible Navier-Stokes equation because they do not respect the famous LBB condition. We can surpass this limitation by stabilizing the P1-P1 element using a GLS or VMS approach. This alters the saddle-point problem, notably adding term on the pressure diagonal of the pressure block matrix, and one can then solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used such an approach and the results are coherent. Concretly, manufactured solutions for the velocity converge at the right order. My problem arises when I am trying to solve problem in a closed domain (such as a lid driven cavity, where all the normal velocities are equal to 0). In this case, I do not understand how the pressure level should be defined and it seems my global system is ill-posed (missing an equation). If I fix the pressure to constant a single node like I would in FVM, then I seem to instantly lose mass conservation. What is the right approach to &quot;fix&quot; the pressure level in closed simulations for stabilized method? Should one impose the average pressure to be zero? Would that not lead to a quasi-full line for this equation thus wrecking the band of the matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sounds like such a trivial question, but I am at loss here.&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2017-10-26T19:05:15.830" LastActivityDate="2018-06-24T06:09:56.913" Title="Pressure definition/convergence issues for the Incompressible Navier-Stokes when using a stabilized P1-P1 finite element formulation" Tags="|finite-element|fluid-dynamics|computational-physics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28144" PostTypeId="1" CreationDate="2017-10-26T17:13:53.190" Score="1" ViewCount="464" Body="&lt;p&gt;I am attempting to solve a large system, $\bf{Ax} = \bf{b}$ with the help of PETSc. Due to the size of the problem, I'm using a matrix-free approach, where $\bf{A}$ is just a shell. I'm also providing my own preconditioner (which is not a shell), and I'm using ilu(2) factorization on the preconditioner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The issue&lt;/strong&gt;: the setup stage of the solver (see relevant block of code below) is taking an extremely long time. I suspect that this would mostly be the ilu of the preconditioner that's taking a while. I know that it is expected that the ilu will take time, but I'm concerned for the following reason: when I try to solve the same problem using a direct solver (using MKL Lapacke to find the LU decomposition and then inversion of $\bf{A}$, outside of PETSc), the LU is 10x faster. I expect that PETSc's ilu should take a comparable time to the full LU factorization, but it is 10x slower, which seems very strange. (By the way, you may ask, why even use an iterative solver if I can do it with LU, but this example is not as large as the ones I actually want to run, at which point I won't be able to use a direct solver).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code snippet relevant to the issue:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MatCreateShell(comm, Nu, Nu, Nu, Nu, ctx, &amp;amp;A_shell);&#xA;MatShellSetOperation(A_shell, MATOP_MULT, (void(*)(void))usermult);&#xA;&#xA;KSPCreate(comm, &amp;amp;solver);&#xA;KSPSetOperators(solver, A_shell, PreconditionerMatrix);&#xA;KSPSetInitialGuessNonzero(solver, PETSC_TRUE);&#xA;KSPSetNormType(solver, KSP_NORM_UNPRECONDITIONED);&#xA;&#xA;KSPSetFromOptions(solver);&#xA;KSPSetUp(solver);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Things I know / have tried&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The condition number of the matrix can be as large as $10^7$, but I don't think that my issue has anything to do with that, because again, the time sink is in the setup. If this was the problem, it would also manifest itself when I do the full LU decomposition, but it doesn't.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I know that I have to provide a large enough fill factor guess for the post-ilu matrix, and I've set the option &lt;code&gt;-pc_factor_fill&lt;/code&gt; to 3. After running the code with &lt;code&gt;-info&lt;/code&gt;, I confirmed that this is sufficient to prevent any reallocation of memory. &lt;strong&gt;Interesting side note&lt;/strong&gt;: when I do run it with &lt;code&gt;-info&lt;/code&gt;, it reports the required fill factor fairly quickly. Does this mean that it actually does the ilu expectedly fast, but then gets stuck somewhere else? Am I barking up the wrong tree? Here's what it reports:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  [0] PetscCommDuplicate(): Using internal PETSc communicator 7412512 20851120&#xA;  [0] PetscCommDuplicate(): Using internal PETSc communicator 7412512 20851120&#xA;  [0] PCSetUp(): Setting up PC for first time&#xA;  [0] PetscCommDuplicate(): Using internal PETSc communicator 7412512 20851120&#xA;  [0] PetscCommDuplicate(): Using internal PETSc communicator 7412512 20851120&#xA;  [0] PetscCommDuplicate(): Using internal PETSc communicator 7412512 20851120&#xA;  [0] PetscCommDuplicate(): Using internal PETSc communicator 7412512 20851120&#xA;  [0] MatILUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 3. needed 1.7385&#xA;  [0] MatILUFactorSymbolic_SeqAIJ(): Run with -[sub_]pc_factor_fill 1.7385 or use&#xA;  [0] MatILUFactorSymbolic_SeqAIJ(): PCFactorSetFill([sub]pc,1.7385);&#xA;  [0] MatILUFactorSymbolic_SeqAIJ(): for best performance.&#xA;  [0] MatSeqAIJCheckInode_FactorLU(): Found 2030 nodes of 6096. Limit used: 5. Using Inode routines&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then it gets stuck for a really long time...so maybe I set the fill factor too large? I tried the same thing again with a fill factor of 2 instead, but it made no difference.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have already made sure I'm not using the debugging installation of PETSc; when timing the code, I'm definitely using &lt;code&gt;--with-debugging=0&lt;/code&gt; in the PETSc configuration step.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;I am not using any parallelization. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt; Here is the output generated with &lt;code&gt;-log_view&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Using Petsc Release Version 3.7.6, Apr, 24, 2017&#xA;&#xA;                         Max       Max/Min        Avg      Total&#xA;Time (sec):           4.057e+02      1.00000   4.057e+02&#xA;Objects:              7.050e+02      1.00000   7.050e+02&#xA;Flops:                2.161e+11      1.00000   2.161e+11  2.161e+11&#xA;Flops/sec:            5.327e+08      1.00000   5.327e+08  5.327e+08&#xA;MPI Messages:         0.000e+00      0.00000   0.000e+00  0.000e+00&#xA;MPI Message Lengths:  0.000e+00      0.00000   0.000e+00  0.000e+00&#xA;MPI Reductions:       0.000e+00      0.00000&#xA;&#xA;Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)&#xA;                            e.g., VecAXPY() for real vectors of length N --&amp;gt; 2N flops&#xA;                            and VecAXPY() for complex vectors of length N --&amp;gt; 8N flops&#xA;&#xA;Summary of Stages:   ----- Time ------  ----- Flops -----  --- Messages ---  -- Message Lengths --  -- Reductions --&#xA;                        Avg     %Total     Avg     %Total   counts   %Total     Avg         %Total   counts   %Total&#xA; 0:      Main Stage: 4.0567e+02 100.0%  2.1612e+11 100.0%  0.000e+00   0.0%  0.000e+00        0.0%  0.000e+00   0.0%&#xA;&#xA;------------------------------------------------------------------------------------------------------------------------&#xA;See the 'Profiling' chapter of the users' manual for details on interpreting output.&#xA;Phase summary info:&#xA;   Count: number of times phase was executed&#xA;   Time and Flops: Max - maximum over all processors&#xA;                   Ratio - ratio of maximum to minimum over all processors&#xA;   Mess: number of messages sent&#xA;   Avg. len: average message length (bytes)&#xA;   Reduct: number of global reductions&#xA;   Global: entire computation&#xA;   Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().&#xA;      %T - percent time in this phase         %F - percent flops in this phase&#xA;      %M - percent messages in this phase     %L - percent message lengths in this phase&#xA;      %R - percent reductions in this phase&#xA;   Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)&#xA;------------------------------------------------------------------------------------------------------------------------&#xA;Event                Count      Time (sec)     Flops                             --- Global ---  --- Stage ---   Total&#xA;                   Max Ratio  Max     Ratio   Max  Ratio  Mess   Avg len Reduct  %T %F %M %L %R  %T %F %M %L %R Mflop/s&#xA;------------------------------------------------------------------------------------------------------------------------&#xA;&#xA;--- Event Stage 0: Main Stage&#xA;&#xA;MatMult              624 1.0 3.2336e+01 1.0 4.23e+10 1.0 0.0e+00 0.0e+00 0.0e+00  8 20  0  0  0   8 20  0  0  0  1307&#xA;MatMultAdd          2480 1.0 5.4122e-02 1.0 5.00e+07 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0   923&#xA;MatMultTranspose    3100 1.0 2.8153e+00 1.0 9.05e+09 1.0 0.0e+00 0.0e+00 0.0e+00  1  4  0  0  0   1  4  0  0  0  3215&#xA;MatSolve             608 1.0 4.6020e+01 1.0 7.41e+10 1.0 0.0e+00 0.0e+00 0.0e+00 11 34  0  0  0  11 34  0  0  0  1611&#xA;MatLUFactorNum         1 1.0 4.4004e+01 1.0 9.31e+10 1.0 0.0e+00 0.0e+00 0.0e+00 11 43  0  0  0  11 43  0  0  0  2115&#xA;MatILUFactorSym        1 1.0 3.4659e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  9  0  0  0  0   9  0  0  0  0     0&#xA;MatAssemblyBegin      27 1.0 1.5497e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;MatAssemblyEnd        27 1.0 6.8570e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;MatGetRow        10650098 1.0 6.6575e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  2  0  0  0  0   2  0  0  0  0     0&#xA;MatGetRowIJ            1 1.0 9.5367e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;MatGetOrdering         1 1.0 1.3018e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;MatZeroEntries        39 1.0 1.5883e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;MatMatMult             2 1.0 3.5977e+00 1.0 5.97e+09 1.0 0.0e+00 0.0e+00 0.0e+00  1  3  0  0  0   1  3  0  0  0  1660&#xA;MatMatMultSym          2 1.0 7.2353e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;MatMatMultNum          2 1.0 2.8741e+00 1.0 5.97e+09 1.0 0.0e+00 0.0e+00 0.0e+00  1  3  0  0  0   1  3  0  0  0  2078&#xA;VecMDot              302 1.0 4.2068e-02 1.0 2.06e+08 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  4907&#xA;VecNorm              621 1.0 1.4746e-02 1.0 3.03e+07 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  2054&#xA;VecScale             314 1.0 2.0843e-03 1.0 7.66e+06 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  3674&#xA;VecCopy              636 1.0 1.0492e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;VecSet              4096 1.0 4.1316e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;VecAXPY             5278 1.0 9.9347e-02 1.0 1.96e+08 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  1977&#xA;VecAYPX              306 1.0 4.3933e-03 1.0 7.46e+06 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  1698&#xA;VecMAXPY             608 1.0 5.9476e-02 1.0 4.16e+08 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  6993&#xA;VecAssemblyBegin    2493 1.0 1.5733e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;VecAssemblyEnd      2493 1.0 1.5340e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;VecNormalize         314 1.0 1.0375e-02 1.0 2.30e+07 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  2214&#xA;KSPGMRESOrthog       302 1.0 7.3104e-02 1.0 4.13e+08 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  5647&#xA;KSPSetUp               1 1.0 1.2398e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0&#xA;KSPSolve               4 1.0 7.8504e+01 1.0 1.17e+11 1.0 0.0e+00 0.0e+00 0.0e+00 19 54  0  0  0  19 54  0  0  0  1491&#xA;PCSetUp                1 1.0 7.8663e+01 1.0 9.31e+10 1.0 0.0e+00 0.0e+00 0.0e+00 19 43  0  0  0  19 43  0  0  0  1183&#xA;PCApply              608 1.0 4.6022e+01 1.0 7.41e+10 1.0 0.0e+00 0.0e+00 0.0e+00 11 34  0  0  0  11 34  0  0  0  1611&#xA;------------------------------------------------------------------------------------------------------------------------&#xA;&#xA;Memory usage is given in bytes:&#xA;&#xA;Object Type          Creations   Destructions     Memory  Descendants' Mem.&#xA;Reports information only for process 0.&#xA;&#xA;--- Event Stage 0: Main Stage&#xA;&#xA;              Matrix    33             30   1306648980     0.&#xA;              Vector   663            663     65247472     0.&#xA;       Krylov Solver     1              1        35264     0.&#xA;      Preconditioner     1              1         1008     0.&#xA;              Viewer     2              0            0     0.&#xA;           Index Set     5              5        87624     0.&#xA;========================================================================================================================&#xA;Average time to get PetscTime(): 5.96046e-07&#xA;#PETSc Option Table entries:&#xA;-ksp_atol 1e-8&#xA;-ksp_converged_reason&#xA;-ksp_monitor&#xA;-ksp_monitor_true_residual&#xA;-ksp_rtol 1e-8&#xA;-log_view&#xA;-pc_factor_fill 3&#xA;-pc_factor_levels 2&#xA;-pc_type ilu&#xA;#End of PETSc Option Table entries&#xA;Compiled with FORTRAN kernels&#xA;Compiled with full precision matrices (default)&#xA;sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 16 sizeof(PetscInt) 4&#xA;Configure options: PETSC_ARCH=arch-linux2-cxx-nodebug --with-scalar-type=complex --with-fortran-kernels=1 --with-clanguage=c++ --with-debugging=0 --with-cxx=g++ CXXOPTFLAGS=-O3 COPTFLAGS=O3 FOPTFLAGS=-O3 --download-openmpi --with-blaslapack-dir=/opt/intel/mkl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I know that there may not be an easy / obvious solution to this, but at least I'd like insight on why this happens, and what PETSc is doing internally that takes this long&lt;/li&gt;&#xA;&lt;li&gt;If there is no clear solution, what are some steps I can take to try and mitigate this or investigate this further?&lt;/li&gt;&#xA;&lt;li&gt;Is this expected / normal, and should I stop worrying about it and just suck it up?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Sorry about the length of this question. Thanks for your time!&lt;/p&gt;&#xA;" OwnerUserId="25809" LastEditorUserId="25809" LastEditDate="2017-10-26T19:32:15.797" LastActivityDate="2017-10-26T19:32:15.797" Title="Why does PETSc take unexpectedly long to set up its KSP solver with a custom preconditioner?" Tags="|iterative-method|petsc|preconditioning|linear-system|" AnswerCount="0" CommentCount="4" ClosedDate="2017-10-29T16:18:53.407" ContentLicense="CC BY-SA 3.0" />
  <row Id="28145" PostTypeId="1" AcceptedAnswerId="28147" CreationDate="2017-10-26T23:17:10.517" Score="0" ViewCount="106" Body="&lt;p&gt;I have a m*n matrix (m&gt;1000000, n&gt;50000). What I want to do now is to pairwise every two rows of the matrix and do some further computation (e.g computing common elements of two rows, or reveal subsumption relationship between two rows). I wrote applications with naive code (using two for loops) which can work for small matrix, but not for big matrix like the one listed at the beginning. I need some suggestions on how to solve this computation problem in this case.&lt;/p&gt;&#xA;" OwnerUserId="25813" LastActivityDate="2017-10-27T00:23:57.590" Title="How to improve the efficiency when I need to exhaustively pairwise millions of list in Python?" Tags="|parallel-computing|efficiency|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28146" PostTypeId="2" ParentId="28143" CreationDate="2017-10-26T23:29:14.843" Score="2" Body="&lt;p&gt;Using VMS doesn't affect how you deal with the pressure for continuous galerkin methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of enclosed flow like you have with all dirichlet conditions on velocity, it is common to set the mean value of the pressure to 0. This can usually be done as a post processing step after each linear solve in your nonlinear solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is to use a pressure penalization term like $ \varepsilon (p, q) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case where you have outflow conditions (I.e. Channel flow), using the no normal stress condition is sufficient on the outflow. &lt;/p&gt;&#xA;" OwnerUserId="18009" LastActivityDate="2017-10-26T23:29:14.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28147" PostTypeId="2" ParentId="28145" CreationDate="2017-10-27T00:23:57.590" Score="3" Body="&lt;p&gt;You can't -- if you want to consider all pairs of rows, then your algorithm is inherently ${\cal O}(m^2)$ in the number $m$ of rows. There is fundamentally nothing you can do about this unless you find a way to avoid looking at &lt;em&gt;all&lt;/em&gt; pairs of rows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem should be easy to parallelize!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-27T00:23:57.590" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28148" PostTypeId="1" AcceptedAnswerId="28150" CreationDate="2017-10-27T03:55:52.077" Score="4" ViewCount="1032" Body="&lt;p&gt;Suppose $ q $ and $ A $ are given and that $ q, p \in R^{N} $ and $ A \in R^{NxN} $, then how can I find the vector $ p $ such that &#xA;$$ (q - p)^{T}A(q - p) $$ is minimized constraint to $ \sum_{i=1}^{N} |p_{i}| \leq 1 $ (thus the $ ||p||_{1} \leq 1 $ ) using projected gradient descent? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it also possible to apply projected gradient descent with the constraint of $ ||p||_{1} = 1 $ as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can find the gradient of the cost function itself, but given the &quot;updated&quot; $p$ value, I need to project it via $argmin_{p^{*}} (||p - p^{*}||)$ where the $p^{*}$, which is what I want, is the updated p value projected into the feasible space, but I'm not sure how I can go about doing that part.&lt;/p&gt;&#xA;" OwnerUserId="25815" LastActivityDate="2017-11-20T18:50:28.010" Title="How can I use Projected Gradient Descent for this optimization problem with constraint?" Tags="|optimization|convex-optimization|constrained-optimization|newton-method|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28149" PostTypeId="2" ParentId="28148" CreationDate="2017-10-27T07:06:35.577" Score="0" Body="&lt;p&gt;You can use the algorithm in the following paper to project the gradient update to l1 norm ball:&#xA;&lt;a href=&quot;https://stanford.edu/~jduchi/projects/DuchiShSiCh08.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://stanford.edu/~jduchi/projects/DuchiShSiCh08.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25390" LastActivityDate="2017-10-27T07:06:35.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28150" PostTypeId="2" ParentId="28148" CreationDate="2017-10-27T12:21:56.550" Score="1" Body="&lt;p&gt;So you want to solve&#xA;$$&#xA;\min_{\|p\|_1 \leq 1} f(q).&#xA;$$&#xA;with $f(p)=(p-q)^TA(p-q)$. If $A$ is symmetic positive semidefinite this is a smooth convex objective with a convex constraint and hence, this can be done by projected gradient by iterating&#xA;$$&#xA;p^{n+1} = P(p^n - t_n 2A(p^n-q))&#xA;$$&#xA;where $P$ is the orthogonal projection onto the set $\{\|p\|_1\leq 1\}$. This projection can be computed pretty quick and algorithms for this are rediscovered basically every year. Maziar Sanjabi gave a link to one good source already. The $t_n$ are stepsizes and you can use $t_n = 1/(2\|A\|)$ (if I am not mistaken). If this is slow, a cheap acceleration is accelerated projected gradient descent (aka FISTA or Nesterov's accelerated gradient method).&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="302" LastEditDate="2017-11-20T18:50:28.010" LastActivityDate="2017-11-20T18:50:28.010" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28151" PostTypeId="2" ParentId="28148" CreationDate="2017-10-27T12:26:40.247" Score="3" Body="&lt;p&gt;Unfortunately, there is no closed form expression for the projection to the $\ell^1$ ball; it is also not a componentwise operation. In principle, the projection onto the scaled unit ball $\alpha B_{\ell^1}$ for $\alpha&amp;gt;0$ is given by&#xA;$$&#xA;\Pi(x) = \begin{cases} x &amp;amp; \|x\|_1 \leq \alpha \\ S_{\lambda(x)}(x) &amp;amp; \|x\|_1&amp;gt;\alpha\end{cases}&#xA;$$&#xA;where $S_\lambda$ is the &lt;em&gt;soft thresholding&lt;/em&gt; (or &lt;em&gt;soft shrinkage&lt;/em&gt;) operator, which for $\lambda&amp;gt;0$ is given component-wise by&#xA;$$&#xA;[S_\lambda(x)]_i = \begin{cases} x_i-\lambda &amp;amp; x_i\geq \lambda\\ 0 &amp;amp; |x_i| \leq \lambda \\ x_i+\lambda &amp;amp; x_i\leq -\lambda \end{cases}&#xA;$$&#xA;and $\lambda(x)&amp;gt;0$ has to be chosen such that $\|S_{\lambda(x)}x\|_1=\alpha$ or, equivalently, that&#xA;$$ &#xA;\sum_i \max\{|x_i|-\lambda(x),0\} = \alpha.&#xA;$$&#xA;This is a scalar (but nondifferentiable) root-finding problem, which can be solved using bisection, the algorithm given in the link in Maziar Sanjabi's answer (which is an efficient implementation based on partitioning the components of the vector $x$) or a semismooth Newton method (or any combinations thereof).&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2017-10-27T13:35:38.663" LastActivityDate="2017-10-27T13:35:38.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28152" PostTypeId="2" ParentId="28130" CreationDate="2017-10-27T14:32:35.910" Score="0" Body="&lt;p&gt;In research and teaching, mostly use dimensionless equations. A neat way in aerodynamics is to put the free stream values as $p=1, U=M, \rho=\gamma$ (so that $a=1$) But if you are designing something to be made, use units and take care to get them right$&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2017-10-27T14:32:35.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28156" PostTypeId="1" AcceptedAnswerId="28157" CreationDate="2017-10-28T07:55:48.920" Score="3" ViewCount="183" Body="&lt;p&gt;I am thinking to use discontinuous galerkin FEM (DGFEM) method to estimate discontinuous displacement field $u: \Omega \rightarrow \mathbb{R}^2$ at the crack surface of a material. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The domain is discretized with triangles. I can think two ways to define control points where we evaluate displacement fields&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Control points are defined on vertices/nodes of a triangle (probably knows as nodal DGFEM in the literature)&lt;/li&gt;&#xA;&lt;li&gt;Control points are defined on mid-points of the edges. (like in Crouzeix–Raviart elements)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I could not find a reference where second method has been employed to model DGFEM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will there be something wrong mathematically if we define control points on mid-points of edges to model DGEM?&lt;/p&gt;&#xA;" OwnerUserId="13142" LastActivityDate="2017-10-28T13:17:19.010" Title="Discontinuous Galerkin FEM : Control points are mid-points of edges instead of nodes" Tags="|finite-element|pde|discretization|mesh|discontinuous-galerkin|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28157" PostTypeId="2" ParentId="28156" CreationDate="2017-10-28T13:17:19.010" Score="3" Body="&lt;p&gt;For the discontinuous Galerkin method, the choice of control points is entirely unimportant because they conceptually lie in the interior of the cell (even if they are physically on the boundary of the cell). We think of a control point that is conceptually located at a vertex to be one that all cells adjacent to that vertex share; and of one on an edge that all cells adjacent to that edge share.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in the DG methods, no control points are shared -- if you happen to physically locate a control point on an edge, then it will have to exist twice: one for each of the adjacent cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, put your control points where it is convenient for you to define the function space on each cell.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-10-28T13:17:19.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28158" PostTypeId="1" AcceptedAnswerId="28174" CreationDate="2017-10-29T08:01:46.043" Score="0" ViewCount="40" Body="&lt;p&gt;I have two sets of experimental data: $\phi(t)$ and $I(t)$. In theory they are related to each other as: $\phi(t) = nI(t) $. By fitting these curves together I can find the value of $n$ (which is a constant). I can calculate the error of this fit using residue and covariance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for each point $t$ of my data sets I also have error of $\phi(t)$ and $I(t)$. How to propagate this error through the fitting procedure, so the final error of the $n$ parameter is also influenced by them?&lt;/p&gt;&#xA;" OwnerUserId="25831" LastActivityDate="2017-10-30T14:46:22.963" Title="Propagation of error in fitting two sets of data to each other" Tags="|error-estimation|regression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28161" PostTypeId="1" CreationDate="2017-10-29T12:02:22.420" Score="3" ViewCount="135" Body="&lt;p&gt;I am interested to study crack propagation in a hyperelastic material in a variational setting. The crack surface exhibits a jump discontinuity. The function space for displacement field should consist of discontinuous functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discontinuous galerkin finite element (DGFEM) approximation space consist of piecewise discontinuous functions which seems to a valid choice to solve crack problem. However, the required degree of freedom is extremely large for a high resolution discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recently proposed, enriched galerkin finite element method (EGFEM) enriched the continuous galerkin approximation space with a piecewise constant function. This allows discontinuities along the element face. For example, a one dimensional function $f(x)$ can be approximated in a unit interval as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ f(x) = f_1 \phi_1(x) + f_2 \phi_2(x) + f_3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\phi_1(x)$ and $\phi_2(x)$ are basis functions. EGFEM seems to be a better choice to solve crack problem as required degree of freedom is quite less than for DGFEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EGFEM does introduce the discontinuity along the edges/faces, but in the deformed configuration the disjoint edges/faces would always be parallel. This is one of the limitation of EGFEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before going to implement these methods, I would like to understand advantage and disadvantage of DGFEM and EGFEM approach to solve the hyperelastic energy minimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What all points I should consider to evaluate these methods ?&lt;/p&gt;&#xA;" OwnerUserId="13142" LastEditorUserId="13142" LastEditDate="2017-10-31T07:28:47.313" LastActivityDate="2017-10-31T07:28:47.313" Title="Piecewise Constant Enrichment of the continuous galerkin method" Tags="|finite-element|pde|discontinuous-galerkin|solid-mechanics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28162" PostTypeId="1" CreationDate="2017-10-29T15:56:34.557" Score="2" ViewCount="43" Body="&lt;p&gt;I am trying to solve the following equation: &#xA;$$\dfrac{\partial }{\partial \epsilon}\left[B\left(\epsilon\right)f_e\left(1-f_e\right)+D\left(\epsilon\right)\dfrac{\partial f_e}{\partial \epsilon}\right] = R\left(f_e, f_h, E\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Till now, there is nothing special at all, and I can solve this (with a simple $R$). &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;But, the most important part of this equation is a form of recombination $R$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R \approx \dfrac{1}{A-B\cdot f_e}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A,B$ - are known constants and $(A-B\cdot f_e)$ should always be positive!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But numerical scheme which I use does not know anything about it, and it simply jumps through this boundary, which means no special recombination is included. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typical value of $(A-B\cdot fe) \in [ 10^{-14},10^{-7}]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any methods to solve it? &#xA;The main problem is resonance characteristic of recombination $R$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I trying to change $A$ on each iteration and wait for the convergence, after change boundary parameters (which is a parameter to change exactly) and solve again. I perform this procedure until $|A - A_\text{const}| = 1e-4$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This method works bad because of the relaxation coefficient within Newton iteration ~ 1e-6.)&lt;/p&gt;&#xA;" OwnerUserId="22777" LastEditorUserId="9667" LastEditDate="2018-03-17T16:44:08.933" LastActivityDate="2018-03-17T16:44:08.933" Title="One dimensional differential equation with resonance recombination (RHS)" Tags="|pde|numerical-modelling|" AnswerCount="0" CommentCount="2" ClosedDate="2019-06-05T22:10:19.480" ContentLicense="CC BY-SA 3.0" />
  <row Id="28163" PostTypeId="1" CreationDate="2017-10-29T17:03:15.533" Score="0" ViewCount="2070" Body="&lt;p&gt;I am trying to solve equations of motion for an harmonic oscillator using 4th order runge kutta method, but as a result I get almost constant velocity and position; I feel that the problem is that I did not fully understood the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is th code I am using (in c++)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;iomanip&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;ofstream output;&#xA;&#xA;void rk(double,double*,double*, double*,double*, int);&#xA;void print_file(double*, double*,double*,int n);&#xA;&#xA;int main(int argc, char* argv[]){&#xA;    double *t,*x,*v,*ic;&#xA;&#xA;    double h,in=0,fin=2*M_PI;&#xA;    int n=10000;&#xA;&#xA;    char* file=argv[1];&#xA;    output.open(file);&#xA;&#xA;    h=(fin-in)/n;&#xA;&#xA;    t=new double[n];&#xA;    x=new double[n];&#xA;    v=new double[n];&#xA;    ic=new double[2];&#xA;&#xA;    ic[0]=1;//in. cond. pos.&#xA;    ic[1]=0;//in. cond. vel.&#xA;&#xA;    rk(fun,h,ic,t,v,x,n);&#xA;    for(int i=0;i&amp;lt;n;i+=500){cout &amp;lt;&amp;lt; x[i] &amp;lt;&amp;lt; &quot;  &quot; &amp;lt;&amp;lt; v[i] &amp;lt;&amp;lt; endl;}&#xA;&#xA;    print_file(t,x,v,n);&#xA;&#xA;    delete[] t;&#xA;    delete[] x;&#xA;    delete[] v;&#xA;    delete[] ic;&#xA;}&#xA;&#xA;void rk(double step,double *ic,double* t, double* v, double* x, int n){&#xA;    double k1,k2,k3,k4;&#xA;    for(int i=1;i&amp;lt;n;i++){&#xA;        t[i]=i*step;&#xA;&#xA;        x[0]=ic[0];&#xA;        v[0]=ic[1];&#xA;&#xA;        k1=step*(-x[i-1]);&#xA;        k2=step*(-x[i-1]+step/2);&#xA;        k3=step*(-x[i-1]+step/2);&#xA;        k4=step*(-x[i-1]+step);&#xA;        v[i]=v[i-1]+(k1+2*k2+2*k3+k4)*step/6;&#xA;&#xA;        k1=step*(v[i-1]);&#xA;        k2=step*(v[i-1]+step/2);&#xA;        k3=step*(v[i-1]+step/2);&#xA;        k4=step*(v[i-1]+step);&#xA;        x[i]=x[i-1]+(k1+2*k2+2*k3+k4)*step/6;&#xA;}}&#xA;&#xA;void print_file(double* t, double* x, double* v, int n){&#xA;    int i;&#xA;    for(i=0; i&amp;lt;=n; i++)&#xA;    output&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;t[i]&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;x[i]&amp;lt;&amp;lt;setw(15)&amp;lt;&amp;lt;v[i]&amp;lt;&amp;lt;endl;&#xA;    return;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where are my errors?&lt;/p&gt;&#xA;" OwnerUserId="25758" LastActivityDate="2017-10-29T19:43:35.153" Title="4th order runge-kutta and harmonic oscillator" Tags="|ode|runge-kutta|" AnswerCount="1" CommentCount="0" ClosedDate="2017-10-30T03:32:16.780" ContentLicense="CC BY-SA 3.0" />
  <row Id="28164" PostTypeId="1" CreationDate="2017-10-29T18:31:24.727" Score="1" ViewCount="48" Body="&lt;p&gt;I'm working on the Nonlinear Schrodinger equation (NLSE) in 1d:&#xA;$$i\psi _t (t,x)+ \psi _{xx} + |\psi|^2\psi = 0 \, ,$$ for $t\geq 0$ and $x\in \mathbb{R}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation is integrable, and so similarly to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverse_scattering_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;KdV equation&lt;/a&gt;, there is a way to solve it using the Direct and Inverse Scattering Transform. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I'm looking for&lt;/strong&gt; is an algorithm for computing the Direct Scattering Transform (DST), i.e., given $\psi(t,x)$, compute the discrete and continuous spectrum. The help could be in the form of a tutorial, a paper, pseudo-code or even an implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank!&lt;/p&gt;&#xA;" OwnerUserId="15188" LastActivityDate="2017-10-29T18:31:24.727" Title="Computing Direct Scattering Transform" Tags="|nonlinear-equations|reference-request|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28165" PostTypeId="2" ParentId="28163" CreationDate="2017-10-29T19:37:50.470" Score="2" Body="&lt;pre&gt;&lt;code&gt;    k1=step*(-x[i-1]);&#xA;    k2=step*(-x[i-1]+step/2);&#xA;    k3=step*(-x[i-1]+step/2);&#xA;    k4=step*(-x[i-1]+step);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's not the form of RK4. Each &lt;code&gt;k&lt;/code&gt; has a derivative calculation that uses the previous derivative estimate. The first should be &lt;code&gt;step*f(x[i-1])&lt;/code&gt;, then the next should be &lt;code&gt;step*f(x[i-1] + step/2 * k1)&lt;/code&gt;, ... (written non-homogeneous). Then I don't see your derivative &lt;code&gt;f&lt;/code&gt; defined anywhere, and then you also need to take that an make it work for a vector equation (probably make it inplace, etc.) since you have to update the &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;v&lt;/code&gt; simultaneously. I would highly recommend just using software which is already made to do this since in the end, RK4 won't even be the most efficient way to handle this, even after you put in the work to somewhat test it.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2017-10-29T19:43:35.153" LastActivityDate="2017-10-29T19:43:35.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28166" PostTypeId="2" ParentId="27406" CreationDate="2017-10-29T21:09:48.993" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What would be the best starting document for ... b) learning steps by steps?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See my project to write an intro book on using PETSc to solve PDEs:&#xA;  &lt;a href=&quot;https://github.com/bueler/p4pdes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bueler/p4pdes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An early draft PDF is at the &quot;Releases&quot; tab.&lt;/p&gt;&#xA;" OwnerUserId="25836" LastActivityDate="2017-10-29T21:09:48.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28168" PostTypeId="2" ParentId="25199" CreationDate="2017-10-29T23:33:08.760" Score="-3" Body="&lt;p&gt;The simplest way is to use fourth-order Runge-Kutta integration.&lt;/p&gt;&#xA;" OwnerUserId="25838" LastActivityDate="2017-10-29T23:33:08.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28169" PostTypeId="2" ParentId="25199" CreationDate="2017-10-30T00:44:00.663" Score="3" Body="&lt;h2&gt;tl;dr&lt;/h2&gt;&#xA;&lt;p&gt;The Nose-Hoover equations are normally defined by their Hamiltonian. The two partial derivatives of the Hamiltonian define a partitioned set of ODEs. From there the partitioned ODEs are either solved by an integrator for 1st order ODEs or some symplectic method.&lt;/p&gt;&#xA;&lt;h2&gt;Details shown using DifferentialEquations.jl&lt;/h2&gt;&#xA;&lt;p&gt;As a rather thorough example, let's look at the architecture for how DifferentialEquations.jl handles it. &lt;a href=&quot;http://diffeq.sciml.ai/latest/types/dynamical_types.html#Hamiltonian-Problems-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;It allows you to define a dynamical ODE problem&lt;/a&gt; directly using the Hamiltonian. What this is really &lt;a href=&quot;http://diffeq.sciml.ai/latest/types/dynamical_types.html#Mathematical-Specification-of-a-Dynamical-ODE-Problem-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;doing is defining the partitioned ODE version&lt;/a&gt; using the partial derivatives of the Hamiltonian, i.e. the equations of motion for the position and momentum given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\dot{p} = -\frac{\partial H}{\partial q}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\dot{q} = \frac{\partial H}{\partial p}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;from Hamilton's relations. For systems of equations this is the gradient, so you can see &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqPhysics.jl/blob/master/src/hamiltonian.jl#L21&quot; rel=&quot;nofollow noreferrer&quot;&gt;from the source code&lt;/a&gt; how this is done using autodifferentiation and define it similarly in any language.&lt;/p&gt;&#xA;&lt;p&gt;Once you have the partitioned set of ODEs, then there are many things you can do. If you group them together, you can call this one giant system of first order ODEs. In that case, you can use any standard ODE solver on this system (which for DifferentialEquations.jl &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;it's this list&lt;/a&gt;, or in MATLAB that would be something like &lt;code&gt;ode45&lt;/code&gt;, or in SciPy you could use &lt;code&gt;LSODA&lt;/code&gt;, etc.).&lt;/p&gt;&#xA;&lt;p&gt;But you &amp;quot;know more structure&amp;quot; about the equation, and so theoretically methods could be made to use this structure and be more efficient, right? Yes, that is the case. There's a big list of them &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/dynamical_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here on the DifferentialEquations.jl page&lt;/a&gt;, but we can split them into two classes: Runge-Kutta Nystrom methods and Symplectic integrators. Runge-Kutta Nystrom methods are extensions of Runge-Kutta methods specifically developed for second order ODEs, and this structure you have here (a position equation and a momentum equation) is this second order ODE structure. These are able to be more efficient than the first order methods on this specific class of problems by cutting out some function evaluations.&lt;/p&gt;&#xA;&lt;p&gt;Symplectic integrators are more interesting. &lt;a href=&quot;https://en.wikipedia.org/wiki/Symplectic_integrator#Introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;These integrators satisfy the property that their integration conserves the symplectic two-form&lt;/a&gt;. This commonly makes people believe that they conserve energy, that is not true. However, for long term integration they will stay on the symplectic manifold and thus not drift away, leading to better estimates of things like energy and angular momentum in the long run. However, these methods have a few extra costs associated with them (of course, because they have to satisfy more properties!). One big thing is that they are fixed timestep (there are adaptive timestep variants, but the recent research in the topic doesn't show that they are that efficient, mostly because there's a lot of extra stuff that needs to go on for adaptive timestep to stay symplectic).&lt;/p&gt;&#xA;&lt;h2&gt;Recap&lt;/h2&gt;&#xA;&lt;p&gt;So let's recap a little bit. You either analytically, symbolically, numerically, or via autodifferentiation calculate the partitioned ODE for &lt;span class=&quot;math-container&quot;&gt;$(\dot{p},\dot{q})$&lt;/span&gt;: which methods do you choose? If your problem isn't too hard, just throw that into your standard ODE solver and you're good. If you're doing long time integration and need to preserve some extra properties like energy and angular momentum &amp;quot;better&amp;quot;, use some symplectic integrator (&lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/dynamical_solve.html#Symplectic-Integrators-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's a whole list&lt;/a&gt;, a popular one for this is Velocity Verlets because it's easy to implement and reduces force calculations, which isn't actually super efficient but I'll get to that some other time...). If need it efficient and aren't doing long time integration, one of the &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/dynamical_solve.html#Runge-Kutta-Nystr%C3%B6m-Integrators-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Runge-Kutta Nystrom methods&lt;/a&gt; is what you're looking for.&lt;/p&gt;&#xA;&lt;h2&gt;Available software&lt;/h2&gt;&#xA;&lt;p&gt;So DifferentialEquations.jl has it all of course as shown above. But if you're not using Julia, the papers for each of these methods &lt;a href=&quot;http://juliadiffeq.org/citing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be found on the citations page&lt;/a&gt;. If you need a symplectic integrator and need to roll it yourself, &lt;a href=&quot;https://en.wikipedia.org/wiki/Verlet_integration#Velocity_Verlet&quot; rel=&quot;nofollow noreferrer&quot;&gt;Velocity Verlet&lt;/a&gt; is a good thing to try first. Other than that, &lt;a href=&quot;http://www.unige.ch/%7Ehairer/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;you can find a few of these methods in Fortran&lt;/a&gt;, the &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/25291-rkn1210-a-12th-10th-order-runge-kutta-nystr%C3%B6m-integrator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dormand-Prince Nystrom method in MATLAB&lt;/a&gt;, &lt;a href=&quot;https://pypi.python.org/pypi/rebound/1.2.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;a few symplectic integrators in Python which wrap REBOUND&lt;/a&gt;, and &lt;a href=&quot;http://reference.wolfram.com/language/tutorial/NDSolveSPRK.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a symplectic integrator in Mathematica&lt;/a&gt;, but I don't know of another large set of methods which has it all together to benchmark with so Julia is the choice if you want/need to do that.&lt;/p&gt;&#xA;&lt;h2&gt;Concluding benchmarks and examples&lt;/h2&gt;&#xA;&lt;p&gt;I'll end by leading to a few benchmarks and examples which could help you choose the right method. These are taken from &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqTutorials.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffEqTutorials.jl&lt;/a&gt; and &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffEqBenchmarks.jl&lt;/a&gt;, but let me point you to ones which are relevant:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqTutorials.jl/blob/master/PhysicalModels/ClassicalPhysics.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is a quick demonstration of the difference in energy error between a symplectic integrator and a standard 1st order integrator on a few classic problems&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqTutorials.jl/blob/master/PhysicalModels/KeplerProblem.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;This problem shows the &amp;quot;drift&amp;quot; that occurs in non-symplectic integrators over long time integrations in the Kepler problem&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/DynamicalODE/Henon-Heiles_energy_conservation_benchmark.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;This notebook benchmarks high-precision solutions using various methods when high energy accuracy is needed&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/DynamicalODE/Quadrupole_boson_Hamiltonian_energy_conservation_benchmark.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is another high accuracy benchmark&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;While none of those are the Nose-Hoover equation itself, those are all benchmarks of other problems defined directly from the Hamiltonian and display the differences between the types of methods, so that should help you to extrapolate from there to see what you need.&lt;/p&gt;&#xA;&lt;h2&gt;Ending quick recommendation&lt;/h2&gt;&#xA;&lt;p&gt;If I had to say one recommendation without knowing exactly what you're doing, and assuming you have to implement it yourself (otherwise... try them all and benchmark!), I'd say go with the Velocity Verlet (on the partitioned set of ODEs defined from the Hamiltonian). It's pretty standard here. Given the benchmarks I have done, symplectic is usually unnecessary and overused (people overplay what symplectic actually means, and in this class Verlets aren't even the most efficient), but they are easy to implement, standard, and give you safety for long-time integrations.&lt;/p&gt;&#xA;&lt;h2&gt;Edit&lt;/h2&gt;&#xA;&lt;p&gt;Oops, I mixed it up with the Nose system which is a Hamiltonian system. The Nose-Hoover system is non-Hamiltonian, so the conversion to 1st order ODEs for standard integrators works &lt;a href=&quot;https://www2.ph.ed.ac.uk/%7Edmarendu/MVP/MVP03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;and so does a form of the Verlet method&lt;/a&gt;, but standard symplectic integrators do not apply. We'll get some implementations to benchmark this field &amp;quot;soon enough&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:06:50.330" LastActivityDate="2020-07-19T18:06:50.330" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="28170" PostTypeId="1" CreationDate="2017-10-30T10:04:32.277" Score="1" ViewCount="138" Body="&lt;p&gt;Consider the following logistic function in 1D for the domain $0$ to $1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) = \frac{1}{1 + \exp(-50(x-0.5))} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I resolve this on two meshes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;first with 11 equispaced points (0.0, 0.1, 0.2, ... 1.0)&lt;/li&gt;&#xA;&lt;li&gt;second with 101 equispaced points (0.0, 0.01, 0.02, ... 1.0)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The plots for the two meshes look like this (blue is first, green is second):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IovhO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IovhO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I compute the error norm for the two meshes? I don't know if L2 norm is the best choice but let's say it is. In that case, the discrete version of the L2 norm is usually stated as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ ||\mathrm{error}||_{2}&#xA;^{2} = h \sum_{i} |f_{i,\mathrm{ref}} - f_{i,h}|^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $f_{i,h}$ is function value for a given mesh at mesh point $i$, and $f_{i,\mathrm{ref}}$ is the value of the analytical function used as a reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However this will produce zero error norm for both cases, since the function evaluated at the mesh points is exact, i.e., it comes directly from the analytical expression. But we can clearly see in the plot that it's a wrong answer for the mesh with 11 points, i.e., error should be non-zero around the middle of the domain, resulting in a non-zero error norm. The zero-error answer is wrong for 101 points too but in the plot the error is not very visible (but it's still there, due to discretization/sampling). In other words, the second mesh has reduced the error compared to the first mesh. (what I described is beginner stuff; just putting things in context).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For correct error computation, a possible alternative is to compute the norm through quadrature. But before we even think about quadrature, we need to compute the error function itself. But how do we do that? Do we use a high resolution mesh (like 1001, or 10001 points) where we resolve $f_{i,\mathrm{ref}}$ as well as interpolate $f_{i,h}$ (say we use linear interpolation)? but this feels very sloppy/adhoc. Is there a better, more systematic way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm just trying to find out what are the &quot;standard&quot; ways of doing such error analysis for mesh resolution, such that I can show that the error in the second case goes down compared to that in the first case. Any tips for doing similar analysis in 2D on unstructured meshes would also be highly appreciated).&lt;/p&gt;&#xA;" OwnerUserId="15198" LastEditorUserId="15198" LastEditDate="2017-10-30T12:55:31.520" LastActivityDate="2017-10-30T12:55:31.520" Title="Mesh error quantification using error norm for a sharp logistic function" Tags="|error-estimation|mesh|" AnswerCount="0" CommentCount="11" ClosedDate="2017-11-27T00:55:57.810" ContentLicense="CC BY-SA 3.0" />
  <row Id="28172" PostTypeId="1" CreationDate="2017-10-30T13:50:35.457" Score="2" ViewCount="807" Body="&lt;p&gt;I'm working on optimizing the parameters of a mathematical model to fit experimental data, using an existing formula for the likelihood of observing the data given a set of parameter values. At the moment I use an algorithm (bound optimization by quadratic approximation, BOBYQA) to find the values that give the maximum likelihood, but I'd like to try using MCMC methods to integrate over the likelihood function, but haven't used them before. Can anyone recommend a good approach and/or existing library for implementing MCMC methods in C/C++?&lt;/p&gt;&#xA;" OwnerUserId="21537" LastActivityDate="2017-10-30T13:50:35.457" Title="Using C/C++ for Markov chain Monte Carlo (MCMC) methods" Tags="|optimization|c++|c|monte-carlo|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28174" PostTypeId="2" ParentId="28158" CreationDate="2017-10-30T14:46:22.963" Score="1" Body="&lt;p&gt;This is an example of what is called an &lt;em&gt;errors in variables&lt;/em&gt; regression model.  Rather than having an independent variable that is known exactly and a dependent variable that is measured with noise, but the independent and dependent variables are measured with noise.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most commonly used technique for fitting linear errors in variables model is the method of &lt;em&gt;total least squares&lt;/em&gt;.  See for example the &lt;a href=&quot;https://en.wikipedia.org/wiki/Total_least_squares&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia article on TLS&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-10-30T14:46:22.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28176" PostTypeId="1" CreationDate="2017-10-30T22:15:40.050" Score="1" ViewCount="711" Body="&lt;p&gt;I have the linear underdetermined system $$Ax=b$$ and I need to find $x$ constrained by the maximization of a score function $g(x)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could find the minimum of a function like $$|Ax-b|^2+1/g(x)$$ but I wanted to know if there exists a canonical way to solve this problem, exploiting the linear part to have a more regular convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using python with scikit-learn, but I can also try to convert MATLAB code to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT. The $g(x)$ function is not convex and has many maximums not necessarily where $ Ax=b$.&lt;/p&gt;&#xA;" OwnerUserId="18255" LastEditorUserId="18255" LastEditDate="2017-10-31T03:18:42.913" LastActivityDate="2017-11-03T13:37:32.977" Title="Minimization of least square function together with a nonlinear function" Tags="|constrained-optimization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28177" PostTypeId="1" AcceptedAnswerId="30556" CreationDate="2017-10-31T03:45:48.930" Score="1" ViewCount="354" Body="&lt;p&gt;Counting FLOP may not be representative of the actual algorithm real world performance but still all the GPU manufacturers mention FLOPS as a metric of Performance on GPU. Is there any way that this metric can be misleading in terms of GPU performance?  &lt;/p&gt;&#xA;" OwnerUserId="25846" LastActivityDate="2018-11-19T03:05:33.120" Title="Why is FLOP(Floating Point Operations Per Second) mentioned as a specification on every GPU?" Tags="|parallel-computing|machine-learning|gpu|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28178" PostTypeId="1" CreationDate="2017-10-31T09:53:01.700" Score="1" ViewCount="314" Body="&lt;p&gt;I need to take the Fourier transform of a 3D function $h(r)=h(|r|)$ so that I can invert a convolution problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to do this with Python?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know the the FT is equivalent to a sine transform (see below) but I can't find a Python library that appears to let me do this integral. One issue I have is that although my sample points for $h(r)$ are regularly spaced they do not start from $r=0$ due to how they've been generated, does this cause any issues?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\tilde{h}(k)=\frac{2}{k}\int_0^{\infty}rh(r)\sin(2\pi kr)dr$$&lt;/p&gt;&#xA;" OwnerUserId="23709" LastEditorUserId="9667" LastEditDate="2017-10-31T14:29:07.673" LastActivityDate="2017-10-31T14:29:07.673" Title="Fourier transform spherical system" Tags="|python|fourier-analysis|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28179" PostTypeId="1" CreationDate="2017-10-31T10:06:22.870" Score="7" ViewCount="354" Body="&lt;p&gt;I've been learning about the Discontinous Galkerin Method by reading the book by Hesthaven and Warburton and have ran into a problem with the advection equation with forcing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u_t + u_x = g(x,t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following the book I've been able to implement a method that works when $g(x,t) = 0$ in Mathematica but am having trouble to get a stable method when the forcing function is active. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally I've got the domain as $[0,1]$ with 4 elements and have tried both situations below with differing order of elements and boundary and initial conditions are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(0,t) = sin(t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u(x,0) = -sin(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume I'm missing something very obvious but I've tried implementing it as a 'direct method' &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{u}_t =-\mathcal{M}^{-1}\mathcal{S} \textbf{u} + \mathcal{M}^{-1}[\textbf{f}^*] + \mathcal{M}^{-1} \textbf{g}_h $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$[\textbf{f}^*]$ are the DG fluxes and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{g}_h = \int g(x,t) l(x) dx $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $l(x)$ is the Lagrange polynomial, which appears to work so long as the forcing function goes to zero at the origin but fails when it doesn't and is particularly bad when g(x,t) = 1 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;as well as an 'indirect' method where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{u}_t =-\mathcal{M}^{-1}\mathcal{S} \textbf{u} + \mathcal{M}^{-1}[\textbf{f}^*] + g(x,t) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. Following the book the mass matrix you get from the discretisation of g (which I'm not 100% clear on but that's a different question) is cancelled out by the inverse mass matrix from $u_t$ term which doens't work at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The questions are, am I correct in assuming that in general $g(x,t)$ can be any function, including a non-zero constant, without additional corrections e.g. artificial viscosity? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And am I missing anything in the formulation, e.g. do the inter-element fluxes need to include additional terms from $g(x,t)$?&lt;/p&gt;&#xA;" OwnerUserId="20410" LastActivityDate="2017-11-12T20:38:32.190" Title="Solving the Advection Equation with Forcing using the Discontinuous Galerkin Method" Tags="|discontinuous-galerkin|advection|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28180" PostTypeId="1" AcceptedAnswerId="28182" CreationDate="2017-11-01T01:33:50.470" Score="2" ViewCount="1132" Body="&lt;p&gt;Is the exponential function problematic and very expensive to compute in Matlab?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I write a new term for my model of ODEs that has an exponential term in it, the program almost never finishes debugging, and I hear the fan on my computer go crazy.  So I quickly stop debugging and remove that term in my model.  And then all is well again.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will using the exp() function even physically harm my computer, say, for instance, burn out my hard drive?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="25675" LastActivityDate="2017-11-01T03:31:34.507" Title="Is the exponential function, e^x, very expensive to compute in Matlab and harmful to my computer?" Tags="|matlab|numerical-modelling|numerical-limitations|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28181" PostTypeId="2" ParentId="1763" CreationDate="2017-11-01T01:43:14.423" Score="1" Body="&lt;p&gt;I am in the same (Fortran) boat. Once I have my 1D, 2D, 3D and 4D (I do projective geometry) elements I create the same operators for each type and then write my logic with high level equations that make it clear what is going on. It is not as slow as you might think to have separate loops of each operation and lots of memory copy. I let the compiler/processor do the optimizations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;interface operator (.x.)&#xA;    module procedure cross_product_1x2&#xA;    module procedure cross_product_2x1&#xA;    module procedure cross_product_2x2&#xA;    module procedure cross_product_3x3&#xA;end interface &#xA;&#xA;subroutine cross_product_1x2(a,b,c)&#xA;    real(dp), intent(in) :: a(1), b(2)&#xA;    real(dp), intent(out) :: c(2)&#xA;&#xA;    c = [ -a(1)*b(2), a(1)*b(1) ]&#xA;end subroutine&#xA;&#xA;subroutine cross_product_2x1(a,b,c)&#xA;    real(dp), intent(in) :: a(2), b(1)&#xA;    real(dp), intent(out) :: c(2)&#xA;&#xA;    c = [ a(2)*b(1), -a(1)*b(1) ]&#xA;end subroutine&#xA;&#xA;subroutine cross_product_2x2(a,b,c)&#xA;    real(dp), intent(in) :: a(2), b(2)&#xA;    real(dp), intent(out) :: c(1)&#xA;&#xA;    c = [ a(1)*b(2)-a(2)*b(1) ]&#xA;end subroutine&#xA;&#xA;subroutine cross_product_3x3(a,b,c)&#xA;    real(dp), intent(in) :: a(3), b(3)&#xA;    real(dp), intent(out) :: c(3)&#xA;&#xA;    c = [a(2)*b(3)-a(3)*b(2), a(3)*b(1)-a(1)*b(3), a(1)*b(2)-a(2)*b(1)]&#xA;end subroutine&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To be used in equations like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m = e .x. (r .x. g)  ! m = e×(r×g)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;e&lt;/code&gt; and &lt;code&gt;r&lt;/code&gt; and &lt;code&gt;g&lt;/code&gt; can have any dimensionality that makes mathematical sense.&lt;/p&gt;&#xA;" OwnerUserId="1030" LastEditorUserId="1030" LastEditDate="2017-11-01T01:53:07.970" LastActivityDate="2017-11-01T01:53:07.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28182" PostTypeId="2" ParentId="28180" CreationDate="2017-11-01T03:31:34.507" Score="16" Body="&lt;p&gt;Computing the term $e^x$ is definitely significantly more expensive than computing a lower-order polynomial -- say $x^4$. But it may be ten to 100 times more expensive at most, not &quot;crazy&quot; expensive. So I suspect that if Matlab takes forever to compute something, then that's because the character of your ODE changes significantly. For example, the presence of that term may make your ODE &quot;stiff&quot; and require a significantly smaller time step than otherwise. That would mean that it's not the evaluation of the right hand side that's expensive, but simply that you need to do &lt;em&gt;many&lt;/em&gt; more time steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for physical harm: No, computers are designed for this kind of work. There is no need to worry about the computer; you may have to worry about getting your knees too warm, though :-)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-11-01T03:31:34.507" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28183" PostTypeId="2" ParentId="1763" CreationDate="2017-11-01T06:02:11.377" Score="5" Body="&lt;p&gt;The clear answers if you want to keep Fortran speed are to use a language which has proper code generation like Julia or C++. C++ templates have already been mentioned, so I'll mention Julia's tools here. Julia's &lt;a href=&quot;https://docs.julialang.org/en/stable/manual/metaprogramming/#Generated-functions-1&quot; rel=&quot;noreferrer&quot;&gt;generated functions&lt;/a&gt; let you use its metaprogramming to build functions on demand via type information. So essentially what you can do here is do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@generated function f(x)&#xA;   N = ndims(x)&#xA;   quote&#xA;     # build the code for the function&#xA;   end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then you use the &lt;code&gt;N&lt;/code&gt; to programmatically build the code you'd want to execute given that it's &lt;code&gt;N&lt;/code&gt; dimensional. Then Julia's &lt;a href=&quot;https://docs.julialang.org/en/latest/devdocs/cartesian/&quot; rel=&quot;noreferrer&quot;&gt;Cartesian library&lt;/a&gt; or packages like &lt;a href=&quot;https://github.com/ahwillia/Einsum.jl&quot; rel=&quot;noreferrer&quot;&gt;Einsum.jl&lt;/a&gt; expressions can be easily built for the &lt;code&gt;N&lt;/code&gt; dimensional function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's nice about Julia here is that this function is statically compiled and optimized for each new dimensional array you use, so it won't compile more than you need yet it'll get you the C/Fortran speed. In the end this is similar to using C++ templates, but it's a higher level language with a lot of tools to make it easier (easy enough that this would be a nice homework problem for an undergrad).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another language which is good for this is a Lisp like Common Lisp. It is easy to use since like Julia it gives you the compiled AST with a lot of built in introspection tools, but unlike Julia it won't automatically compile it (in most distributions).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-11-01T06:02:11.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28184" PostTypeId="1" CreationDate="2017-11-01T12:04:51.540" Score="3" ViewCount="804" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HHnuS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HHnuS.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;I am trying to solve a &lt;em&gt;Vertex Centered&lt;/em&gt;, &lt;em&gt;Finite Difference&lt;/em&gt;, &lt;em&gt;Poisson&lt;/em&gt; equation $-\nabla^{2}u=f$ with &lt;em&gt;Dirichlet&lt;/em&gt; boundaries on the left and bottom and &lt;em&gt;Neumann&lt;/em&gt; boundaries on the top and right using &lt;em&gt;Geometric Multigrid&lt;/em&gt;. An illustration is shown in 2-D (apologies for the hand drawn figure). &lt;strong&gt;Crosses&lt;/strong&gt; indicate unknowns, &lt;strong&gt;red squares&lt;/strong&gt; indicate ghost points for Neumann boundary, &lt;strong&gt;D&lt;/strong&gt; indicates Dirichlet, &lt;strong&gt;9-pt&lt;/strong&gt; stencil is shown in green lines.  It is given that $\frac{\partial u}{\partial n}=0$ at Neumann boundaries. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If I use a &lt;strong&gt;9-pt&lt;/strong&gt; stencil in 2-D to approximate variable $u$ at various mesh points, what will be the value of the point at the &lt;code&gt;Green questions mark&lt;/code&gt;? &lt;/li&gt;&#xA;&lt;li&gt;Should I be using a &lt;strong&gt;5-pt&lt;/strong&gt; stencil at the Neumann-Neumann boundary intersection? &lt;/li&gt;&#xA;&lt;li&gt;Will using a 5-pt stencil at the corner point not deteriorate the &lt;code&gt;convergence rate&lt;/code&gt; in Multigrid?&lt;/li&gt;&#xA;&lt;li&gt;If I use the 9-pt stencil at the top-left boundary, are we not taking a Dirichlet point which is outside the physical domain ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14303" LastEditorUserId="14303" LastEditDate="2017-11-02T13:06:54.723" LastActivityDate="2017-11-02T13:06:54.723" Title="Neumann-Neumann boundary intersection" Tags="|finite-difference|boundary-conditions|multigrid|elliptic-pde|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28187" PostTypeId="2" ParentId="27989" CreationDate="2017-11-02T04:43:51.903" Score="0" Body="&lt;p&gt;I think you're just looking for a straight-forward way of mapping from your original space, call it $(x,y)$ space to your $(a,b)$ space, where $(x,y)=(i\Delta x, j\Delta y)$. Once you're in $(a,b)$ space, you just need to check that $0 \le a \le 1$ and $0 \le b \le 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reverse is easier to see. Given vectors $\mathbf{A}=(A_x,A_y)$ and $\mathbf{B}=(B_x,B_y)$, some origin $\mathbf{O}=(O_x,O_y)$ where $\mathbf{A}$ and $\mathbf{B}$ start (just to be complete), and scalars $a$ and $b$, a point $\mathbf{P}=(x,y)$ can be written&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{P} = a \mathbf{A} + b \mathbf{B} + \mathbf{O}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or in matrix notation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \begin{matrix} x \\ y \end{matrix} \right) &#xA; = \left( \begin{matrix} A_x &amp;amp; B_x\\ A_y &amp;amp; B_y\end{matrix} \right)&#xA;   \left( \begin{matrix} a \\ b\end{matrix} \right)&#xA;+&#xA;  \left( \begin{matrix} O_x \\ O_y \end{matrix} \right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you know $(x,y)$ and are looking for $(a,b)$, simply invert this&#xA;operation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \begin{matrix} a \\ b\end{matrix} \right)&#xA;=&#xA;\left( \begin{matrix} A_x &amp;amp; B_x\\ A_y &amp;amp; B_y\end{matrix} \right)^{-1}&#xA; \left( \begin{matrix} x-O_x \\ y-O_y \end{matrix} \right) &#xA;   $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not that this 2x2 matrix inversion is expensive, but it can be performed once ahead of time to save a few electrons. Obviously, this matrix is singular if $\mathbf{A}$ and $\mathbf{B}$ are parallel. &lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2017-11-02T04:43:51.903" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28188" PostTypeId="1" CreationDate="2017-11-02T11:27:48.773" Score="1" ViewCount="401" Body="&lt;p&gt;I have implemented finite-difference WENO scheme for Euler equations (with some variants - WENO-JS, WENO-Z, WENO-M, different flux splitting). It works well, but have problem with so-called carbuncles (shock instability in the areas where its front is almost aligned with the grid). This problem seems to be well-studied for finite-volume methods (quick search shows that there are some fixes for Riemann solvers), but I can't find almost any information regarding carbuncles in finite-difference methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any known fixes for finite-difference methods?&lt;/p&gt;&#xA;" OwnerUserId="25060" LastEditorUserId="25060" LastEditDate="2017-11-02T17:13:09.907" LastActivityDate="2024-02-06T08:08:51.053" Title="Is there a simple way to avoid carbuncles for FD WENO methods?" Tags="|finite-difference|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28189" PostTypeId="2" ParentId="28176" CreationDate="2017-11-02T11:49:33.823" Score="2" Body="&lt;p&gt;Right name for your problem is maximization of g(x) subject to linear constraint Ax = b. I.e. you have requirement that Ax=b exactly, and you want to maximize g(x) subject to this requirement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference from your approach is that such formulation explicitly states that we tolerate no deviations from Ax=b, whilst your formulation allows complicated trade-offs between accuracy of Ax=b and maximization of g(x).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There exist numerous libraries for constrained nonlinear optimization. As for Python, I'd recommend scipy.optimize.minimize(), SLSQP solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: if you are unsatisfied with SciPy, another option is to try &lt;a href=&quot;http://www.alglib.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ALGLIB&lt;/a&gt;. This library is mature and free and has Python wrapper (actually, it is C++ and C# library). It has linearly constrained optimizer called MinBLEIC, which allows to specify entire constraining matrix A as one array. The only drawback is that library has no support for numpy's ndarray, so if you prefer numpy for arrays, you have to perform conversion of ndarray to list-of-lists (see &lt;a href=&quot;https://stackoverflow.com/questions/9721884/convert-2d-numpy-array-into-list-of-lists&quot;&gt;this question&lt;/a&gt;). Disclosure: I am one of the developers :)&lt;/p&gt;&#xA;" OwnerUserId="25872" LastEditorUserId="25872" LastEditDate="2017-11-03T13:37:32.977" LastActivityDate="2017-11-03T13:37:32.977" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28190" PostTypeId="1" AcceptedAnswerId="28191" CreationDate="2017-11-02T14:59:16.510" Score="2" ViewCount="369" Body="&lt;p&gt;Suppose I have a nonlinear system of PDEs. I am actually interested in Navier-Stokes, but, for the sake of simplicity and example, suppose I had&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial f}{\partial t} - f \frac{\partial g}{\partial x} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial g}{\partial t} - g \frac{\partial f}{\partial x} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that the domain $\Omega$ is rectangular, say, and that some kind of appropriate boundary conditions have been specified on the boundary $\partial \Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I go about using a Galerkin method to solve for &lt;em&gt;both&lt;/em&gt; $f$ and $g$? I understand the derivation of the method for one function fairly well, but I do not see how to generalize it to multiple unknown functions.&lt;/p&gt;&#xA;" OwnerUserId="25718" LastActivityDate="2017-11-02T17:09:14.383" Title="Galerkin method for a system of nonlinear PDEs" Tags="|finite-element|pde|navier-stokes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28191" PostTypeId="2" ParentId="28190" CreationDate="2017-11-02T17:09:14.383" Score="4" Body="&lt;p&gt;So first, represent your system as the following in your case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial q_k}{\partial t} - q_k \nabla \cdot F_k(\boldsymbol{q}) &amp;amp;= 0 &amp;amp; \forall k \in \lbrace 1, 2 \rbrace&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\boldsymbol{q} = [q_1, q_2]^T = [f, g]^T$, $F_1(\boldsymbol{q}) = q_2 \hat{e}_1$, and $F_2(\boldsymbol{q}) = q_1 \hat{e}_1$. Then you just perform the normal Galerkin for each equation using a set of weight functions for that equation. Let us assume we have some space of functions defined for each function/equation, call them $\mathcal{V}_k$. Then we can proceed with the Galerkin projection like so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\int_{\Omega} w_j \left(\frac{\partial q_k}{\partial t} - q_k \nabla \cdot F_k(\boldsymbol{q})\right) d\Omega &amp;amp;= 0 &amp;amp;\forall w_j \in \mathcal{V}_k \\&#xA;%&#xA;\int_{\Omega} w_j \frac{\partial q_k}{\partial t} d\Omega &amp;amp;= \int_{\Omega} w_j q_k \nabla \cdot F_k(\boldsymbol{q}) d\Omega &amp;amp;\forall w_j \in \mathcal{V}_k \\&#xA;%&#xA;&amp;amp;= \int_{\Omega} \nabla \cdot \left( w_j q_k F_k(\boldsymbol{q}) \right) - F_k(\boldsymbol{q}) \cdot \nabla \left( w_j q_k \right)d\Omega &amp;amp;\forall w_j \in \mathcal{V}_k \\&#xA;%&#xA;&amp;amp;= \int_{\Gamma} w_j q_k F_k(\boldsymbol{q}) \cdot \hat{n}\; d\Gamma - \int_{\Omega} F_k(\boldsymbol{q}) \cdot \nabla \left( w_j q_k \right)d\Omega &amp;amp;\forall w_j \in \mathcal{V}_k \\&#xA;&amp;amp;= G_k(\boldsymbol{q}, w_j) &amp;amp;\forall w_j \in \mathcal{V}_k&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here you can discretize things further with your approximate representation for the functions $q_k(x)$ using your space of functions $\mathcal{V}_k$ and come up with the equations you need in terms of any coefficients you are solving for.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2017-11-02T17:09:14.383" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28192" PostTypeId="2" ParentId="2591" CreationDate="2017-11-02T22:18:57.873" Score="1" Body="&lt;p&gt;The use of a database is great for helping you organize/find simulation data (Search by protein, search by simulation parameters).  The database should then tell you where to find the relevant information on disk, where I imagine it is likely best stored on a per-simulation run basis in whatever file type is most convenient to load for analysis (whether custom or from whichever simulation suite you're using).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will let you swiftly find the simulations you wish, and give you the freedom/performance of using sparse matrixes or whatever tools you need to do efficient analysis.&lt;/p&gt;&#xA;" OwnerUserId="25876" LastActivityDate="2017-11-02T22:18:57.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28195" PostTypeId="1" CreationDate="2017-11-03T21:36:16.513" Score="3" ViewCount="16718" Body="&lt;p&gt;I am trying to calculate the energy, magnetization and specific heat of a two dimensional lattice using the metropolis monte carlo algorithm.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import random&#xA;&#xA;#creating the initial array&#xA;def init_spin_array(rows, cols):&#xA;    return np.ones((rows, cols))&#xA;&#xA;#calcuating the nearest neighbours&#xA;def find_neighbors(spin_array, lattice, x, y):&#xA;    left   = (x, y - 1)&#xA;    right  = (x, (y + 1) % lattice)&#xA;    top    = (x - 1, y)&#xA;    bottom = ((x + 1) % lattice, y)&#xA;&#xA;    return [spin_array[left[0], left[1]],&#xA;            spin_array[right[0], right[1]],&#xA;            spin_array[top[0], top[1]],&#xA;            spin_array[bottom[0], bottom[1]]]&#xA;&#xA;#calculating the energy of the configuration&#xA;def energy(spin_array, lattice, x ,y):&#xA;    return 2 * spin_array[x, y] * sum(find_neighbors(spin_array, lattice, x, y))&#xA;&#xA;&#xA;#main code&#xA;def main10():&#xA;    #defining the number of initial sweeps, the lattice size, and number of monte carlo sweeps&#xA;    RELAX_SWEEPS = 50&#xA;    lattice = 10&#xA;    sweeps = 1000&#xA;    e1= e0 = 0&#xA;    for temperature in np.arange(0.1, 4.0, 0.2):&#xA;        #setting up initial variables&#xA;        spin_array = init_spin_array(lattice, lattice)&#xA;        mag = np.zeros(sweeps + RELAX_SWEEPS)&#xA;        spec = np.zeros(sweeps + RELAX_SWEEPS)&#xA;        Energy = np.zeros(sweeps + RELAX_SWEEPS)&#xA;        # the Monte Carlo&#xA;        for sweep in range(sweeps + RELAX_SWEEPS):&#xA;            for i in range(lattice):&#xA;                for j in range(lattice):&#xA;                    e = energy(spin_array, lattice, i, j)&#xA;                    if e &amp;lt;= 0:&#xA;                        spin_array[i, j] *= -1&#xA;                    elif np.exp((-1.0 * e)/temperature) &amp;gt; random.random():&#xA;                        spin_array[i, j] *= -1&#xA;&#xA;            #Thermodynamic Variables &#xA;&#xA;            #Magnetization&#xA;            mag[sweep] = abs(sum(sum(spin_array))) / (lattice ** 2)&#xA;&#xA;            #Energy&#xA;            Energy[sweep] = energy(spin_array,lattice,i,j)/ (lattice ** 2)&#xA;&#xA;            #Specific Heat&#xA;            e0 = e0 + energy(spin_array,lattice,i,j)               &#xA;            e1 = e1 + energy(spin_array,lattice,i,j) *energy(spin_array,lattice,i,j)&#xA;            spec[sweep]=((e1/(sweeps*lattice) - e0*e0/(sweeps*sweeps*lattice*lattice)) / (temperature * temperature))&#xA;&#xA;        #Printing the thermodynamic variables    &#xA;&#xA;        print(temperature,sum(Energy[RELAX_SWEEPS:]) / sweeps, sum(mag[RELAX_SWEEPS:]) / sweeps,sum(spec[RELAX_SWEEPS:]) / sweeps)&#xA;&#xA;&#xA;&#xA;main10()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I appear to be calculating the Magnetization correctly, which makes me convinced that my monte carlo algorithm is correct. But I appear to have some errors when calculating the Total Energy and the Specific Heat. I am trying to think of possible errors for them but I am genuinely struggling! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The graphs of them are shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7GWr9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7GWr9.jpg&quot; alt=&quot;Energy&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/q0J6z.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q0J6z.jpg&quot; alt=&quot;Magnetization&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/637Qh.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/637Qh.jpg&quot; alt=&quot;Specific Heat&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated! :)&lt;/p&gt;&#xA;" OwnerUserId="25887" LastActivityDate="2017-11-09T07:22:43.590" Title="2D Ising Model in Python" Tags="|python|computational-physics|monte-carlo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28196" PostTypeId="1" CreationDate="2017-11-04T00:51:35.833" Score="4" ViewCount="394" Body="&lt;p&gt;This is a follow-up of a &lt;a href=&quot;https://scicomp.stackexchange.com/questions/27777/accurate-evaluation-of-the-sign-of-a-polynomial&quot;&gt;previous question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $p$ be a polynomial with floating-point coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method for finding intervals where evaluating $p$ in floating-point arithmetic always gives the correct sign?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find (ideally, maximal) disjoint interval $I_1, \dots, I_m$ such that for all floating-point numbers $a \in I_k$, the sign of $f(a)$ is always correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is clearly related to isolating the zeros of $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A standard example, where the sign of $f$ is wrong around a zero is $f(x)=x^6 - 6 x^5 + 15 x^4 - 20 x^3 + 15 x^2 - 6 x + 1$ near the multiple zero $x=1$. Note the $f(x)$ is the expansion of $(x-1)^6$.&lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="640" LastEditDate="2017-12-22T20:26:17.817" LastActivityDate="2017-12-22T20:26:17.817" Title="Intervals where the sign of a polynomial can be computed reliably" Tags="|floating-point|polynomials|accuracy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28197" PostTypeId="2" ParentId="28196" CreationDate="2017-11-04T01:38:09.520" Score="4" Body="&lt;p&gt;Yes. You can compute a running error bound, i.e, a number $\mu$ such that the difference between the exact value of $y = p(x)$ and the computed value satisfies $\hat{y}$ satisfies $$|y - \hat{y}| \leq \mu u.$$&#xA;Here $u$ is the unit roundoff. You can trust the sign computed sign of $y$, when $|\hat{y}| &amp;gt; \mu u$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $p(x) = \sum_{j=0}^n a_j x^j$, then Horner's method computes $$p_0 = a_n, \quad p_i = x p_{i-1} + a_{n-i}, \quad i = 1,2,\dotsc,n.$$ If $a_i$ and $x$ are machine numbers, then a running error bound can be computed as follows &#xA;$$\mu_0 = 0, \quad z_j = p_{j-1} x, \quad p_j = z_j + a_{n-j}, \quad \mu_j = \mu_{j-1} |x| + |z_j| + |p_j|, \quad j=1,2,\dotsc, n.$$&#xA;The algorithm returns $y = p_n$ and $\mu = \mu_n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to reduce the cost of this algorithm from 5n flops to 4n flops, see Higham's book &quot;Accuracy and Stability of Numerical Algorithms&quot; for details.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2017-11-04T01:38:09.520" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28199" PostTypeId="1" AcceptedAnswerId="28220" CreationDate="2017-11-04T08:11:51.390" Score="0" ViewCount="146" Body="&lt;p&gt;I am using &lt;a href=&quot;https://people.sc.fsu.edu/~jburkardt/cpp_src/triangle_dunavant_rule/triangle_dunavant_rule.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dunavant quadrature&lt;/a&gt; in my software, specifically &lt;a href=&quot;https://people.sc.fsu.edu/~jburkardt/cpp_src/triangle_dunavant_rule/triangle_dunavant_rule.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;this file&lt;/a&gt; by John Burkardt. Recently, I wanted to convert the code into a &lt;code&gt;constexpr&lt;/code&gt; code in C++. But doing so requires me understanding what the code does. I have read the code and the &lt;a href=&quot;http://maxwell.sze.hu/~marcsa/docs/277540.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and I am confused about the suborders. I have specifically three questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the suborders, how are they related to the list of points and weights? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why are the suborders apprarently calculated in three dimensions and then only two of them are used?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why does the author use a &lt;code&gt;mod&lt;/code&gt; function to calculate some suborders like in this code?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (k = 0; k &amp;lt; 3; k++) {&#xA;   xy[0 + o * 2] = suborder_xyz[i4_wrap(k, 0, 2) + s * 3];&#xA;   xy[1 + o * 2] = suborder_xyz[i4_wrap(k + 1, 0, 2) + s * 3];&#xA;   w[o] = suborder_w[s];&#xA;   o = o + 1;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user1800" LastEditorDisplayName="user1800" LastEditDate="2017-11-04T08:17:25.407" LastActivityDate="2017-11-08T00:43:14.470" Title="Reference for Dunavant Quadrature Implementations" Tags="|quadrature|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28201" PostTypeId="2" ParentId="28196" CreationDate="2017-11-04T17:22:10.790" Score="2" Body="&lt;p&gt;I want to add that in addition to Carl Christian's suggestion of using a running error bound, you can also take the general relative error bound&#xA;$$ \frac{|\hat p(x)-p(x)|}{|p(x)|} \leq \gamma_{2n}\,\mathrm{cond}(p,x),\qquad \gamma_{2n} \approx 2nu,\\&#xA;\mathrm{cond}(p,x) = \frac{\sum |a_i||x|^i}{|\sum a_i x^i|} = \frac{\tilde p(x)}{|p(x)|},$$&#xA;(see &lt;a href=&quot;http://www-pequan.lip6.fr/~jmc/polycopies/Compensation-horner.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www-pequan.lip6.fr/~jmc/polycopies/Compensation-horner.pdf&lt;/a&gt;), and then the condition that the sign is right can be imposed be imposing &#xA;$$ \gamma_{2n}\mathrm{cond}(p,x) \leq 1, \quad\text{or}\quad&#xA;|p(x)|\gtrsim 2nu\tilde p(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially all this is doing is identifying those $x$ for which evaluating the polynomial at $x$ is well-conditioned, so it's just a bound on the condition number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $(x-1)^6$, this would result in something like $|x-1| &amp;gt; c u^{1/6}$ where $c$ is a small constant $c$ and $u$ is the unit roundoff. For a simple root $\alpha$, it would be $|x-\alpha| &amp;gt; c u$.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2017-11-04T17:22:10.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28204" PostTypeId="1" CreationDate="2017-11-05T15:00:06.333" Score="1" ViewCount="50" Body="&lt;p&gt;I have a very high dimensional signal, say $15$ dimensions. Across each dimension the width is $N$ points. So total number of points is $N^{15}$. I already have the FFT given to me. Only low frequency coeffs are non zero. The non zero coeffs are the ones inside of the hypercube of width 5. So only $5^{15}$ are non zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to take an IFFT to compute samples in signal domain.But I dont want to compute all samples of signalbut only a $k$ of them. $k$ is much smaller compared to $N^{15}$. The samplesin signal domain are by no means sparse, but its just that I want to compute only a small number $k$ of them. These $k$ samples are distributed arbitrarily and are not confined to any region.&#xA;How can I compute them efficiently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across Sparse FFT, but here my signal is not sparse but just that i am interested only in a few samples in signal domain. So I am not sure I can use SFFT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS : I don't want to take a full IFFT due to memory constraints.&lt;/p&gt;&#xA;" OwnerUserId="13214" LastActivityDate="2017-11-05T15:00:06.333" Title="Computing IFFT for only $k$ samples in a high dimensional signal" Tags="|fftw|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28207" PostTypeId="1" AcceptedAnswerId="28208" CreationDate="2017-11-06T17:26:42.387" Score="1" ViewCount="451" Body="&lt;p&gt;Hi all and thank you in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on a time-dependent transport-chemistry model to study the composition of planetary atmospheres.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations are the following&#xA;$$\frac{\partial n(z,t)}{\partial t} = -\frac{\partial \Phi (z,t)}{\partial z} + p(z,t) -n(z,t)l(z,t)$$&#xA;for each chemical component, characterized by a density profile $n(z,t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's focus in the chemistry part, because the problem I am going to explain does not take place for transport alone (modeled as a thermodynamic diffusion flux $\Phi$).&#xA;$$\frac{\partial n(z,t)}{\partial t} = p(z,t) -n(z,t)l(z,t).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose we have three molecules interacting following the following reactions&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;H2O + O --&amp;gt; 2OH&#xA;&#xA;2OH --&amp;gt; H2O + O&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The set of equations to be modeled are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial n_{H2O}/\partial{t} = \partial n_{O}/\partial{t} = -An_{H2O}n_{O} +Bn_{OH}^{2}\\&#xA;\partial n_{OH}/\partial{t} = 2An_{H2O}n_{O} -2Bn_{OH}^{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm I am testing is a semi-implicit Bulirsch-Stoer method (see Numerical recipes in Fortran, page 735, for a detailed explanation). I basically adapted the code is provided within the book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This algorithm is suitable for my project because it makes an error estimation so that time step is constantly adapted. I need to integrate up to million of years in about 1 hour, i.e. my simulation up to a certain point has to integrate with $h &amp;gt; 10^5 s$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next I am displaying a screenshot from this &quot;toy model&quot; typical realization (This would be a first step before including hundreds of reactions of the same kind)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hy0E2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hy0E2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system clearly achieves stationary state, but the time step does not increase beyond than $h \approx 10^{-3}$. The idea is that in a Stiff Systems time step increases as reactions contributions as these become stationary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I think the core of my problem is the piece of code devoted to determine the error associated for each integrated step&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;errmax=SMALL&#xA;do i=1,nv&#xA;  errmax=max(errmax,abs(yerr(i)/yscal(i)))&#xA;end do&#xA;errmax=errmax/eps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where what I less understand is the &quot;yscal&quot; factor. In the previous reference (Numerical Recipes in Fortran, page 735) it is mentioned &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We now mention an important point: &quot;&lt;em&gt;It is absolutely crucial to scale your variables properly when integrating stiff problems with automatic stepsize adjustment.&lt;/em&gt; As in our nonstiff routines, you will be asked to supply a vector $y_{scal}$ with which the error is to be scaled. For example, to get constant fractional errors, simply set $y_{scal} = |y|$. You can get constant absolute errors relative to some maximum values by setting $y_{scal}$ equal to those maximum values. In stiff problems, there are often strongly decreasing pieces of the solution which you are not particularly interested in following once they are small. You can control the relative error above some threshold $C$ and the absolute error below the threshold by setting&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$y_{scal} = max(C,|y|)$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If you are using appropriate non-dimensional units, then each component of $C$ should be of order unity. If you are not sure what values to take for $C$, simply try setting each component equal to unity. &lt;em&gt;We strongly advocate the previous expression for stiff problems&lt;/em&gt; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&quot;Playing&quot; with such $y_{scal}$ definitely the behavior changes so that I believe including the proper expression for it I can get the desired result. When I include &quot;transport&quot; alone, I obtain the desired evolution in $h$ the time step. Something happens with chemistry. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am open to any kind of suggestions. Even if you can provide me other ideas to implement. I experimented a bit with Rosenbrock methods with the same exact result. I am really stuck with such model.&lt;/p&gt;&#xA;" OwnerUserId="23659" LastEditorUserId="23659" LastEditDate="2017-11-07T09:29:31.667" LastActivityDate="2017-11-07T09:29:31.667" Title="Bulirsch-Stoer algorithm to solve simple chemistry. Spanning long time intervals after stationarity" Tags="|algorithms|ode|fortran|numerical-modelling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28208" PostTypeId="2" ParentId="28207" CreationDate="2017-11-07T00:27:36.847" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.uwyo.edu/buerkle/misc/wnotnr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Recipes is well-known as a bad resource for implementations of methods for DiffEqs&lt;/a&gt;, and you'll note that the mention that extrapolation methods are generally worse can be found in there (and it's pretty easy to verify on most test equations). So this whole thing somewhat starts off in the wrong direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyways, if you take a look at &lt;a href=&quot;http://www.springer.com/us/book/9783540604525&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer II&lt;/a&gt; you'll notice there are a lot things that could be going on. First and foremost, if you're using the standard proportional error control that NR seems to recommend then you'll run into something called &quot;the hump behavior&quot; where the error difference can take a bit to decrease after going past a stiff increase. This is why it's generally recommended that you use a Gustafsson accelerated type of controller for stiff ODE solvers. So maybe that has something to do with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing that can be going on is just floating point error. Your numbers are far too large and spread out in order of magnitude to perform operations well without getting a lot of floating point error. You might want to rescale the units so that things are closer to one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that Hairer's &lt;code&gt;dopri5&lt;/code&gt; and &lt;code&gt;dop853&lt;/code&gt; actually display this same behavior that if the error is too low then their timesteps end up becoming constant. Usually this isn't a big deal but there are some test equations which can really show this, and I haven't found the time to investigate why it does this (maybe large exponents do something weird in Fortran?). So maybe you could be seeing something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or maybe that error scaling is the issue. Honestly, I have never seen that in a production ODE solver or in a book/publication about ODE solver methods from a professional in the field. If you want to do the standard algorithm, it's something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;qtmp = EEst^(1/(order(alg)+1))/gamma&#xA;q = max(inv(qmax),min(inv(qmin),qtmp))&#xA;dtnew = dt/q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;q&lt;/code&gt; is the scaling factor for &lt;code&gt;dt&lt;/code&gt;. &lt;code&gt;gamma = 0.9&lt;/code&gt; is the safety factor. &lt;code&gt;qmax&lt;/code&gt; is the maximum amplification factor usually taken to be between 2-10, &lt;code&gt;qmin&lt;/code&gt; is the minimum de-amplification usually &lt;code&gt;0.2&lt;/code&gt;. It's done in the inversion to make it more numerically stable. &lt;code&gt;EEst&lt;/code&gt; is the error estimate, which is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EEst = norm(err./(abstol .+ max.(abs.(uprev),abs.(u)).*reltol))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;err&lt;/code&gt; is the the difference between embedded methods (or whatever estimate you use) the &lt;code&gt;norm&lt;/code&gt; is the semi-norm Hairer writes about:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;norm(u) = sqrt(sum(u.^2)/length(u)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, &lt;code&gt;EEst &amp;lt; 1&lt;/code&gt; only if the tolerances are satisfied, in which case you reject steps where it's &lt;code&gt;&amp;gt; 1&lt;/code&gt;. I can guarantee you that basic solvers are using this, and more advanced solvers are using a play on this with a PI-controller or acceleration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But anyways, this is probably all besides the point because as stated in the comments, there are well-built ODE solvers so unless you are specifically doing something with the intent on improving them... don't recreate them. In fact, here's a well known stiff extrapolation code in Fortran already:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.unige.ch/~hairer/prog/stiff/seulex.f&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.unige.ch/~hairer/prog/stiff/seulex.f&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-11-07T00:27:36.847" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28211" PostTypeId="1" CreationDate="2017-11-07T12:41:54.537" Score="0" ViewCount="48" Body="&lt;p&gt;Hard to come up with a good title. I'm looking for an algorithm to adjust a range. Say I have 5.34 as min and 23.96 as max value. This gives it a range/difference of 18.62. I now want to divide this number by n (in my case always 5.5) and result of that division should be a &quot;nice&quot; or meaningful number to make a series from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case we could enlarge the difference to 22 and get a series of 4,8,12,16,20,24 and set the limits to 4 and 26. The first problem obviously is the definition of a &quot;nice number&quot;. A whole number fits that but also 0.25 or 0.5 would be good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having trouble coming up with an algorithm. Of course mod is involved but i above case 5-25 would be better fit as humans will look at this. How would I encode this &quot;order of importance&quot;?  Scale can be arbitrary so maybe 100 is good or 100000. So a fixed list won't work either?&lt;/p&gt;&#xA;" OwnerUserId="25914" LastActivityDate="2017-11-07T14:46:45.463" Title="Algorithm to adjust range / scale so it is &quot;nicley&quot; dividable by n" Tags="|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28212" PostTypeId="2" ParentId="28211" CreationDate="2017-11-07T14:46:45.463" Score="1" Body="&lt;p&gt;Take a look how plotting programs do this -- for example &lt;code&gt;gnuplot&lt;/code&gt;. They also get odd values and need to find ways to display that with a meaningful scale.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-11-07T14:46:45.463" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28213" PostTypeId="1" CreationDate="2017-11-07T15:18:09.803" Score="1" ViewCount="153" Body="&lt;p&gt;I have implemented a split-step-method for an equation of the shape &#xA;$$\partial_z E = i\partial_x^2E+ic|E|^2E$$&#xA;resulting in a split into the linear part&#xA;$$L=\partial_x^2$$&#xA;and the nonlinear part&#xA;$$N=c|E|^2E$$&#xA;giving me&#xA;$$E(x, z+\Delta z)=\exp(i\Delta z(N+L))E(x, t)$$&#xA;But now my $c$-variable is non-constant, but rather &#xA;$$\partial_tc=|E|^4$$&#xA;How do I couple that to my equation above in the split-step approach (either using an FFT or a DHT)? I can not include it as nonlinear term, after it does not depend on $E$, only on the absolute value.&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2017-11-07T15:18:09.803" Title="Split-step-method for coupled equations" Tags="|time-integration|operator-splitting|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28214" PostTypeId="1" CreationDate="2017-11-07T16:17:50.950" Score="1" ViewCount="305" Body="&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/3188/dealing-with-the-inverse-of-a-positive-definite-symmetric-covariance-matrix&quot;&gt;In this post&lt;/a&gt; I found a very similar probem to the one I have, but not a satisfactory answer for my purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a set of matrices $C_\ell$. They are exactly symmetric by construction. Unfortunately, they are also singular. Their structure is similar to a block matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here a &quot;picture&quot; of how they look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0000000000&#xA;0000000000&#xA;00------00 &#xA;00------00&#xA;00------00 &#xA;00------00&#xA;0000000000&#xA;0000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(and they are square, although ti doesn't seem like that from the picture). The dashed lines represent values different from 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I need to do is to calculate the Fisher matrix from them. The formula for the Fisher matrix looks like this&#xA;$$F_{\alpha \beta} = \sum_\ell \frac{1}{2} Tr [ C^{-1}_\ell C_{\ell,\alpha} C_\ell^{-1} C_{\ell,\beta}]$$&#xA;where $C_{\ell,\alpha}$ denotes derivative of the $C_\ell$ w.r.t the parameter $\alpha$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have been using a Singuar Value Decomposition to operate the inversion of $C_\ell$'s. But recently I discovered that, as the dimension of my $C_\ell$'s increases (now it has reached &lt;code&gt;900x900&lt;/code&gt;) this method gives very imprecise results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am implementing everything in &lt;code&gt;C++&lt;/code&gt;. Any idea on how to perform better these inversions? Speed is not an issue (as long as the operation remains feasible in human times), I really care about precision&lt;/p&gt;&#xA;" OwnerUserId="24645" LastEditorUserId="24645" LastEditDate="2017-11-07T19:05:30.313" LastActivityDate="2017-12-10T11:57:58.003" Title="Inverting big symmetric and singular matrices" Tags="|matrix|c++|svd|inverse|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28215" PostTypeId="2" ParentId="28214" CreationDate="2017-11-07T16:57:21.653" Score="3" Body="&lt;p&gt;You might be better served by either the LDL' decomposition or the Cholesky decomposition (in the event that C's are positive definite in addition to symmetric, they probably are). Though all of the algorithms have cubic complexity, the constants are much better for these &quot;linear&quot; solvers than the SVD (which is more of a &quot;spectral&quot; solver). They are also easier to parallelize (Cholesky is particularly easy). These decompositions are readily available in LAPACK or MKL. The respective factorization routines are DSYTRF (for LDL') and DPOTRF (for Cholesky). You probably don't want to form the inverse explicity, you can instead use the decomposition to apply the &quot;action&quot; of the inverses to your forcing data.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2017-11-07T17:05:27.837" LastActivityDate="2017-11-07T17:05:27.837" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28216" PostTypeId="2" ParentId="28195" CreationDate="2017-11-07T21:31:52.570" Score="3" Body="&lt;p&gt;Your specific heat is indeed not correct. You should get a peak centered on the critical temperature $T_c\simeq 2.27$. The specific heat is&#xA;    $$C=\big[\langle E^2\rangle-\langle E\rangle^2\big]/k_BT^2$$&#xA;where $\langle\ldots\rangle$ denotes the average over thermal fluctuations. In a Monte Carlo simulation, this average becomes&#xA;   $$\langle E^n\rangle\simeq {1\over\rm sweeps}\sum_{{\rm sweep}=1}^{\rm sweeps}[E({\rm sweep})]^n$$&#xA;to keep your notation. The specific heat can only be computed after all the sweeps have been performed. In your Python code, you compute the specific heat for each iteration. The calculation should be outside the loop over 'sweep'. You should accumulate $E$ and $E^2$ at each iteration (what it is done correctly with $e_0$ and $e_1$) and then, at the end of the loop, normalize them by dividing them by sweeps ($e_1=e_1/{\rm sweeps}$) and finally compute $\langle E^2\rangle-\langle E\rangle^2$ as $e_1-e_0^2$.&lt;/p&gt;&#xA;" OwnerUserId="360" LastEditorUserId="360" LastEditDate="2017-11-09T07:22:43.590" LastActivityDate="2017-11-09T07:22:43.590" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28218" PostTypeId="1" CreationDate="2017-11-07T20:11:07.910" Score="1" ViewCount="278" Body="&lt;p&gt;I have the following coupled differential equations also known as Guiding Center Approximation. It is used to explain the position- and velocity change of particles (electrons and protons, N = 1000) moving in plasma. &#xA;I am trying to either solve this using the Runge-Kutta 4th order method or the built-in ODE45 in Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations are as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;  {d u_{||} \over {dt}} &amp;amp;= {q \over m}  E_{||} - {\mu \over {\gamma m B}} (\mathbf{B} \cdot \nabla) B \\&#xA;  {{d \mathbf{R}} \over dt} &amp;amp;= {u_{||} \over {\gamma B}} \mathbf{B} + {{\mathbf{E} \times \mathbf{B}} \over B^2}&#xA;  + { \mu \over {\gamma q B^2}} \mathbf{B} \times \nabla B + {{ m u^2_{||}} \over {\gamma q B^4}} \mathbf{B} \times &#xA;  (\mathbf{B} \cdot \nabla) \mathbf{B}\\&#xA;  \mu &amp;amp; = {{m u^2_{\perp}} \over {2B}} = \mathrm{constant}&#xA;  \end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where, &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;|| is the velocity component parallel to the magnetic field&lt;/li&gt;&#xA;&lt;li&gt;$u$ is the particle velocity&lt;/li&gt;&#xA;&lt;li&gt;$q$ is the charge&lt;/li&gt;&#xA;&lt;li&gt;$m$ is the mass of the particle&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{E}$ is the electric field vector&lt;/li&gt;&#xA;&lt;li&gt;$\mu $ is the constant determined in the last equation above&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{B}$ is the magnetic field vector&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{R}$ is the position of the particle&lt;/li&gt;&#xA;&lt;li&gt;$\gamma $ is the Lorentz factor given by $\gamma = (1-\frac{v^2}{c^2})^{-\frac{1}{2}}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am given the initial position and velocity of each particle. Furthermore I'm given $\mathbf{B}$ &amp;amp; $\mathbf{E}$. I'm also given the initial dt.&#xA;When $B$ is used without being bold it's either the norm of $\mathbf{B}$ or the component going in the direction of $\mathbf{B}$ and same goes for the electric field. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mathbf{B} &amp;amp; = {B_0 \over L} (x, y, -2 z)\\&#xA;\mathbf{E} &amp;amp; = E_0 L \left( { -y^2 \over {(x^2+y^2)^{1.5}}}, {xy \over &#xA;     {(x^2+y^2)^{1.5}}}, 0 \right)&#xA;\end{align} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve these coupled equations I tried using the RK4 method. What I'm unsure of is how to write my equations in a way of $f(t_n,y_n)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$k_1 = f(t_n,y_n) $&lt;/li&gt;&#xA;&lt;li&gt;$k_2 = f(t_n + \frac{h}{2}, y_n + \frac{h}{2} \cdot k_1)$ &lt;/li&gt;&#xA;&lt;li&gt;$k_3 = f(t_n + \frac{h}{2}, y_n + \frac{h}{2} \cdot k_2)$&lt;/li&gt;&#xA;&lt;li&gt;$k_4 = f(t_n + h, y_n + h \cdot k_3)$&lt;/li&gt;&#xA;&lt;li&gt;$y_{n+1} = y_n + \frac{h}{6} \cdot (k_1 + 2\cdot k_2 +2\cdot k_3 + k_4 )$&lt;/li&gt;&#xA;&lt;li&gt;$t_{n+1} = t_n + h$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And furthermore, how do I using Matlab, calculate these two equations at the same time (as I assume I must). Lastly, would it be easier or more accurate to use the builtin ODE45?&lt;/p&gt;&#xA;" OwnerDisplayName="J.Doe" LastActivityDate="2017-11-07T21:42:28.317" Title="Solving complicated coupled ODE using RK4/ODE45 in Matlab" Tags="|computational-physics|software|differential-equations|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28219" PostTypeId="2" ParentId="28214" CreationDate="2017-11-08T00:41:57.810" Score="6" Body="&lt;p&gt;Think of your matrix as block-diagonal with blocks&#xA;$$&#xA; C_\ell = \begin{pmatrix} 0 &amp;amp; &amp;amp; \\ &amp;amp; D_\ell &amp;amp; \\ &amp;amp;  &amp;amp; 0\end{pmatrix}.&#xA;$$&#xA;Then it is clear that&#xA;$$&#xA; C_\ell^{-1} = \begin{pmatrix} 0^{-1} &amp;amp; &amp;amp; \\ &amp;amp; D_\ell^{-1} &amp;amp; \\ &amp;amp;  &amp;amp; 0^{-1}\end{pmatrix}&#xA;$$&#xA;where of course $0^{-1}$ is not well defined. But that doesn't matter because when you compute a term such as &#xA;$$&#xA; C_\ell^{-1}C_{\ell,\alpha}&#xA;$$&#xA;you will get&#xA;$$&#xA; C_\ell^{-1}C_{\ell,\alpha} = \begin{pmatrix} 0^{-1}0 &amp;amp; &amp;amp; \\ &amp;amp; D_\ell^{-1}D_{\ell,\alpha} &amp;amp; \\ &amp;amp;  &amp;amp; 0^{-1}0\end{pmatrix}&#xA;$$&#xA;which shows that you only ever multiply the ill-defined terms with another zero matrix. So&#xA;$$&#xA; C_\ell^{-1}C_{\ell,\alpha} = \begin{pmatrix} 0 &amp;amp; &amp;amp; \\ &amp;amp; D_\ell^{-1}D_{\ell,\alpha} &amp;amp; \\ &amp;amp;  &amp;amp; 0\end{pmatrix}&#xA;$$&#xA;and similarly for the second product:&#xA;$$&#xA; C_\ell^{-1}C_{\ell,\alpha}C_\ell^{-1}C_{\ell,\beta} = \begin{pmatrix} 0 &amp;amp; &amp;amp; \\ &amp;amp; D_\ell^{-1}D_{\ell,\alpha}D_\ell^{-1}D_{\ell,\beta} &amp;amp; \\ &amp;amp;  &amp;amp; 0\end{pmatrix}&#xA;$$&#xA;and consequently&#xA;$$&#xA; \text{trace}\left(C_\ell^{-1}C_{\ell,\alpha}C_\ell^{-1}C_{\ell,\beta}\right) = \text{trace}\left(D_\ell^{-1}D_{\ell,\alpha}D_\ell^{-1}D_{\ell,\beta}\right).&#xA;$$&#xA;This is perfectly well defined as long as $D_\ell$ is invertible.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-11-08T00:41:57.810" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28220" PostTypeId="2" ParentId="28199" CreationDate="2017-11-08T00:43:14.470" Score="1" Body="&lt;p&gt;The integration formulas for triangles are defined in terms of the&#xA;three area coordinates of the triangle and a weight. Since the sum of&#xA;the three area coordinates must equal one, only two of them are&#xA;independent (possibly this is what you mean when you say&#xA;&quot;calculated in three dimensions and then only two of them are used&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important that an integration formula return the same value &#xA;irrespective of how the vertices of the triangle are ordered, i.e.&#xA;the coordinates of the integration points should not be biased toward&#xA;a particular vertex. The centroid of the triangle is one such point&#xA;where all three area coordinates are equal. This would be a &quot;suborder&quot;&#xA;of size one. If two of the area coordinates at a particular integration&#xA;point are the same, then there should be two additional integration points&#xA;that are permutations of these coordinates; this would be a suborder of&#xA;size three. Finally, if all three area coordinates at an integration point&#xA;are different, there would be six permutations of these; i.e. a suborder&#xA;of size six.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mod function is a clever way of calculating these different permutations.&#xA;But, in this case, since there are only three or six of them, simply&#xA;listing them explicitly is somewhat more obvious and not an unreasonable&#xA;way to implement the routine. &lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-11-08T00:43:14.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28222" PostTypeId="2" ParentId="10549" CreationDate="2017-11-08T11:49:35.890" Score="-1" Body="&lt;p&gt;I am trying to do something similar to what you are writing here. &#xA;In short what I understand is that, in the first approach, you do not explicitly compute the $\partial u /\partial n$ operator while in the second you do this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question here is about the signs of $gL$ and $gR$ in your first formulation. How do you decide on this ?&#xA;If I am using a CG-FEM discretization is this the same as FETI-H approach ?&#xA;Do you have some paper or some work where you explain this in greater depth ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current project is precisely working on TC for DDT problems in frequency domain for Helmholtz problems.&lt;/p&gt;&#xA;" OwnerUserId="25201" LastActivityDate="2017-11-08T11:49:35.890" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28224" PostTypeId="1" CreationDate="2017-11-09T07:27:11.127" Score="1" ViewCount="55" Body="&lt;p&gt;If I have a partial differentail equation such as $\frac{\partial^2 u}{\partial t^2} + c^2 \frac{\partial^2 u}{\partial x^2}$, with boundary conditions $u(0,t) = 0$ and $u(1,t)=0$, I can solve this system using separation of variables: $u=X(x)T(t)$. This means I have an ODE: $X''(x) + (c^2-\lambda)X(x)=0$, with $\lambda$ as separation constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To look for solutions I set up the Wronskian and search for $\lambda$ values for which the Wronskian determinant is zero, meaning I found an eigenvalue. For 'easy'boundary conditions this can also be done manually, however I want to do it numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I look over a range of $\lambda$, look for minima, and optimize around those minima.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing how the Wronskian is set up, is there an optimization method or numerical trick I can perform to speed up the process?&lt;/p&gt;&#xA;" OwnerUserId="10193" LastActivityDate="2017-11-09T07:27:11.127" Title="Finding minimum of Wronskian determinant" Tags="|optimization|pde|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28225" PostTypeId="1" AcceptedAnswerId="28234" CreationDate="2017-11-09T19:15:44.203" Score="11" ViewCount="1730" Body="&lt;p&gt;In research on inverse problems, it's common to construct a synthetic data set from a known set of parameters and then test whether the inversion technique can reconstruct those parameters.  In doing so, it's important to add appropriate levels of random noise to the synthetic data.  Furthermore, if the method used to compute the synthetic data is based on a finite difference or finite element grid, it's also important to not use that same grid in the inversion process.  Otherwise, the inversion process is really inverting the approximate numerical forward model.  The phrase &quot;inverse crime&quot; has been used to describe this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This phrase was in common use when I first became interested in these problems.  I'm aware that it appears in the book &lt;em&gt;Inverse Acoustic and Electromagnetic Scattering Theory&lt;/em&gt; by Colton and Kress, published in 1992.  I'd be interested in any earlier uses of the phrase.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="1804" LastEditDate="2017-11-11T14:09:17.603" LastActivityDate="2017-11-11T14:09:17.603" Title="First appearance of the phrase &quot;inverse crime&quot;" Tags="|reference-request|inverse-problem|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28226" PostTypeId="2" ParentId="46" CreationDate="2017-11-10T02:58:44.463" Score="2" Body="&lt;p&gt;If you're looking for large graphs or network data to test on. The &lt;a href=&quot;http://snap.stanford.edu/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stanford Network Analysis Project (SNAP)&lt;/a&gt; has many large graph datasets typically in the form of an anonymized adjacency list. Some of their options include:&lt;/p&gt;&#xA;&lt;h3&gt;Data&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://snap.stanford.edu/data/index.html#socnets&quot; rel=&quot;nofollow noreferrer&quot;&gt;Social networks&lt;/a&gt;: Facebook, Gplus, Twitter, bitcoin, etc.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://snap.stanford.edu/data/index.html#citnets&quot; rel=&quot;nofollow noreferrer&quot;&gt;Citation networks&lt;/a&gt;: nodes represent papers, edges are citations.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://snap.stanford.edu/data/index.html#web&quot; rel=&quot;nofollow noreferrer&quot;&gt;Web graphs&lt;/a&gt;: nodes represent wedbpages, edges are hyperlinks.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://snap.stanford.edu/data/index.html#road&quot; rel=&quot;nofollow noreferrer&quot;&gt;Road networks&lt;/a&gt;: nodes represent intersections, edges are roads connecting intersections.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://snap.stanford.edu/data/index.html#signnets&quot; rel=&quot;nofollow noreferrer&quot;&gt;Signed networks&lt;/a&gt;: positive and negative edge weights.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://snap.stanford.edu/data/index.html#temporal&quot; rel=&quot;nofollow noreferrer&quot;&gt;Temporal networks&lt;/a&gt;: edges have timestamps.&lt;/li&gt;&#xA;&lt;li&gt;Many, many, &lt;a href=&quot;http://snap.stanford.edu/data/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;others&lt;/a&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Properties of Data&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of edges: anywhere from ~10 to ~400 million&lt;/li&gt;&#xA;&lt;li&gt;Number of nodes: anywhere from ~10 to ~100 million&lt;/li&gt;&#xA;&lt;li&gt;Edge types: directed, undirected, weighted, unweighted, signed, and unsigned.&lt;/li&gt;&#xA;&lt;li&gt;Network types: directed, undirected, bipartite, multigraph, temporal, labeled.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Ground truth stats available on the datasets:&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of Nodes&lt;/li&gt;&#xA;&lt;li&gt;Number of Edges&lt;/li&gt;&#xA;&lt;li&gt;Number of Nodes in largest &lt;a href=&quot;http://mathworld.wolfram.com/WeaklyConnectedComponent.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;weakly connected component&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Number of Edges in largest &lt;a href=&quot;http://mathworld.wolfram.com/WeaklyConnectedComponent.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;weakly connected component&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Number of Nodes in largest &lt;a href=&quot;http://mathworld.wolfram.com/StronglyConnectedGraph.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;strongly connected component&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Number of Edges in largest &lt;a href=&quot;http://mathworld.wolfram.com/StronglyConnectedGraph.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;strongly connected component&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://en.wikipedia.org/wiki/Clustering_coefficient&quot; rel=&quot;nofollow noreferrer&quot;&gt;Average clustering coefficient&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Number of triangles&lt;/li&gt;&#xA;&lt;li&gt;Fraction of closed triangles&lt;/li&gt;&#xA;&lt;li&gt;Diameter (longest shortest path)&lt;/li&gt;&#xA;&lt;li&gt;90-percentile effective diameter&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Tools&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://snap.stanford.edu/snap/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNAP Tool for C++&lt;/a&gt;: A general purpose tool with high performance for analysis and manipulation of large networks.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://snap.stanford.edu/snappy/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNAP Tool for Python&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25940" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-11-10T02:58:44.463" CommentCount="1" CommunityOwnedDate="2017-11-10T02:58:44.463" ContentLicense="CC BY-SA 3.0" />
  <row Id="28227" PostTypeId="1" AcceptedAnswerId="28236" CreationDate="2017-11-10T10:48:39.897" Score="0" ViewCount="411" Body="&lt;p&gt;I'm trying to simulate a 1D model of two bodies sliding on each other with a Coulomb friction in between. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9flZ2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9flZ2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I have explained &lt;a href=&quot;https://physics.stackexchange.com/questions/348240/modeling-static-coulomb-friction-in-a-multibody-system/348254#348254&quot;&gt;here&lt;/a&gt; I modeled the friction as :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The force required to keep two bodies adhered $F_{a12}=\frac{m_2F_1-m_1F_2}{m_1+m_2}$&lt;/li&gt;&#xA;&lt;li&gt;kinetic friction force $F_{f_k12}=Sign\left(v_1-v_2\right)\mu_k F_{n12}$&lt;/li&gt;&#xA;&lt;li&gt;maximum static friction $F_{f_s12}=\mu_s F_{n12} $&lt;/li&gt;&#xA;&lt;li&gt;$F_{f12}=\left\{\begin{matrix}&#xA;  if \, v_1=v_2 \, and \, \left| F_{a12} \right|&amp;lt;F_{f_s12} \,\, then &amp;amp;  F_{a12} \\&#xA;  else &amp;amp;  F_{f_k12}&#xA; \end{matrix}\right.$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Considering the values as $\mu_s=0.3$, $\mu_k=0.2$, $F_{n12}=3$, $m_1=1$, $m_2=2$, $F_1=2\sin(5t)$, $F_2=2\sin(7t)$ I implemented the model in modelica language as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model friction&#xA;  //constants&#xA;  parameter Real muk = 0.2;&#xA;  parameter Real mus = 0.3;&#xA;  parameter Real m1 = 1.0;&#xA;  parameter Real m2 = 2.0;&#xA;  parameter Real Fn12 = 3.0;&#xA;  parameter Real absTol = 0.1;&#xA;  //variables&#xA;  Real X1, X2, V1, V2, A1, A2, F1, F2, Ff12, Fs12;&#xA;&#xA;initial equation&#xA;  X1 = 0;&#xA;  X2 = 0;&#xA;&#xA;  V1 = 0;&#xA;  V2 = 0;&#xA;&#xA;equation&#xA;  F1 = 2*sin(5 * time);&#xA;  F2 = 2*sin(7 * time);&#xA;&#xA;  V1 = der(X1);&#xA;  V2 = der(X2);&#xA;&#xA;  A1 = der(V1);&#xA;  A2 = der(V2);&#xA;&#xA;  m1 * A1 = F1 - Ff12;&#xA;  m2 * A2 = F2 + Ff12;&#xA;&#xA;&#xA;&#xA;  Fs12=(m2*F1-m1*F2)/(m1+m2);&#xA;&#xA;&#xA;  if abs(V1 - V2) &amp;lt; absTol and abs(Fs12) &amp;lt; mus * Fn12 then&#xA;    Ff12=Fs12;&#xA;  else&#xA;    Ff12 = muk * Fn12 * sign(V1 - V2);&#xA;  end if;&#xA;&#xA;&#xA;end friction;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this is the result for $x_1$ and $x_2$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9luC4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9luC4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I modeled the exact same thing in SIMULINK (&lt;a href=&quot;https://ufile.io/1gu10&quot; rel=&quot;nofollow noreferrer&quot;&gt;download the .slx model here&lt;/a&gt;) and there is a completely different result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dojKI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dojKI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why am I getting two completely different results?! Am I doing a mistake in implementation or the difference is due to the different solvers these two programs use? Which one should I trust in that case?!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I just realized that there is something wrong with Wolfram SystemModeler solver. When I do the simulation in OponModelica the result is more similar. &lt;/p&gt;&#xA;" OwnerUserId="22193" LastEditorUserId="1804" LastEditDate="2017-11-20T09:26:36.623" LastActivityDate="2017-11-20T09:26:36.623" Title="Modelica and SIMULINK yield completely different results" Tags="|matlab|openmodelica|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28228" PostTypeId="1" AcceptedAnswerId="28232" CreationDate="2017-11-10T15:23:17.057" Score="1" ViewCount="87" Body="&lt;p&gt;I saw in a code for discretization of something like $\frac{d^2T(x)}{d^2x}$ , ( $x = sin(\theta)$ ) tries&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   !============== here are some defined variables&#xA;   deltaTheta = pi/150&#xA;   deltax(i) = cos(theta(i)) * deltaTheta  &#xA;   dx = 0.5*(deltax(i) + deltax(i+1))&#xA;   dx1 = 0.5*(deltax(i-1)+ deltax(i))&#xA;   !============== here is the discretization that I am talking about&#xA;   ... = ((T(i+1) - T(i))/dx - (T(i) - T(i-1))/dx1)/(0.5 *(dx + dx1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this discretization was for non-boundary cites; what does this discretization called? &lt;/p&gt;&#xA;" OwnerUserId="25942" LastActivityDate="2017-11-10T23:12:12.450" Title="finite difference for a second order ode" Tags="|numerical-analysis|finite-difference|differential-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28229" PostTypeId="1" CreationDate="2017-11-10T16:48:27.913" Score="1" ViewCount="92" Body="&lt;p&gt;I would like to know if there exists a package or how one can fast calculate the quantiles of a function within python, where the inverse of the function for calculating the quantile depends on the first and second derivative. More concretely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we are given a $C^2([a, b])$ function $f$ and define the curvature $\kappa$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \kappa(x) = \frac{|f''(x)|}{|1+(f'(x))^2|^{\frac{3}{2}}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a positive function. I then normalize $\kappa$ s.t. $\int_a^b \kappa = 1$. For simplicity let's denote the normalized function with $g(x):=\frac{\kappa(x)}{\int_a^b \kappa}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I define the cumulative distribution function (CDF) $F$ as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(x) = \int_a^x g(t) dt $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we know that $F(a) = 0$ and $F(b) = 1$. The problem I want to solve is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For given value $y$ of $F$ find $x$ s.t. $F(x) = y$, i.e. the inverse problem of $F$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm interested in a solution within python (or any other similar language like Matlab, R). In my problem I only have data points of $f$ and need to approximate it. The following code snippet is an example for the $sin^2(x)$ curve. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import timeit&#xA;import numpy as np&#xA;from scipy.interpolate import UnivariateSpline&#xA;import scipy.integrate as integrate&#xA;from scipy.optimize import brentq&#xA;&#xA;&#xA;x = np.linspace(0, 6, 100)&#xA;interp = UnivariateSpline(x, np.sin(x)**2, k=4, s=0)&#xA;&#xA;# calculate the normalization constant&#xA;&#xA;&#xA;def kappa(x):&#xA;    return abs(interp.derivative(2)(x))/abs((1+interp.derivative(1)(x)**2)**(1.5))&#xA;&#xA;&#xA;k = integrate.quad(kappa, 0, 6)[0]&#xA;&#xA;&#xA;def g(x):&#xA;    return kappa(x)/k&#xA;&#xA;&#xA;x = np.linspace(0, 6, 100)&#xA;interp2 = UnivariateSpline(x, g(x), k=4, s=0)&#xA;&#xA;&#xA;def F(x):&#xA;    return integrate.quad(interp2, 0, x)[0]&#xA;&#xA;# get 20 test points&#xA;# not 0 and 1 are clear so we don't need&#xA;# to estimate these points.&#xA;&#xA;&#xA;N = 20&#xA;x2 = [(i - 1)/(N-1) for i in range(2, N)]&#xA;&#xA;# first approach would be&#xA;start_time = timeit.default_timer()&#xA;y = [brentq(lambda u: F(u) - i, 0, 6) for i in x2]&#xA;elapsed = timeit.default_timer() - start_time&#xA;print(elapsed)&#xA;&#xA;# just add the two default points&#xA;&#xA;# y = [0] + y&#xA;# y = y.insert(len(y), 6)&#xA;&#xA;&#xA;def F2(x):&#xA;    return integrate.quad(g, 0, x)[0]&#xA;&#xA;&#xA;start_time = timeit.default_timer()&#xA;y = [brentq(lambda u: F2(u) - i, 0, 6) for i in x2]&#xA;elapsed = timeit.default_timer() - start_time&#xA;print(elapsed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note, I've calculated the inverse with two different methods. First I use another interpolation to approximate $\kappa$. In the second version I run it via directly using $\kappa$ where it has to evaluate the first and second derivative. If I'm running this on my pc I get the elapsed times:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.449614048004&#xA;7.88159608841&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What a huge different! Why is the second one so much faster? Now given what I want to do what is the best way to speed this up (even more than in case 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; I've actually had a bug in the code. Since I've used python 2.7 I needed to import the division module from future. With this I get the following. Time is still a huge difference and values are very close for both system: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Time and values for first method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2.38353395462&#xA;[0.1437681151139289, 0.35456065909107926, 1.1539903423627926, 1.3971602056486263, 1.5460856607363669, 1.686478225698852, 1.8751150788777717, 2.6398043923321652, 2.9355086219617466, 3.091753998559681, 3.230312179133905, 3.4029515812391478, 4.058035698689046, 4.470426379024948, 4.636764697040471, 4.774960068200856, 4.93583175345252, 5.273899782448736]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Time and values for second method&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;57.0844209194&#xA;[0.1437734552194811, 0.354572374885639, 1.1530574841452361, 1.3967524608790778, 1.545758378815468, 1.6861161414005308, 1.8745107127405394, 2.637806584725155, 2.9348568464428215, 3.0912665934557833, 3.229804581185758, 3.4021836211797964, 4.0530805425833485, 4.469683619221007, 4.636255181639525, 4.774458238739285, 4.935134505525859, 5.271168509499432]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21291" LastEditorUserId="21291" LastEditDate="2017-11-10T19:50:03.760" LastActivityDate="2017-11-10T19:50:03.760" Title="How to fast estimate derivates for calculating quantiles" Tags="|python|inverse|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28230" PostTypeId="1" CreationDate="2017-11-10T17:20:46.670" Score="1" ViewCount="175" Body="&lt;p&gt;I'm trying to understand when and why one would use a staggered vs. a colocated grid in problems that have velocities and scalars that they transport (e.e. density).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If scalars are defined cell-centered, then in the physics world you often hear that velocities should be defined between cells, because apart from being intuitive, the resulting numerical schemes are stable.&lt;br&gt;&#xA;I'm certain to having had heard this statement in the context of finite difference codes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But now I'm learning finite volume methods with the Riemann Solvers by E. Toro. There, in the math world, vectors like velocities are just 3-scalars in equation systems that are all solved on one and the same colocated grid. There is never any mention of staggering grids at all (at least not as far as I got in the book, around Chapter 16, 3rd Ed.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is the choice of defining velocities on staggered vs. colocated grids related to the numerical methods one is using, or rather if one is solving ODEs vs. PDEs, or is the answer something completely different?&lt;/p&gt;&#xA;" OwnerUserId="20043" LastActivityDate="2017-11-10T17:20:46.670" Title="Choice of velocity grid - staggered or not?" Tags="|finite-volume|grid|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28232" PostTypeId="2" ParentId="28228" CreationDate="2017-11-10T23:12:12.450" Score="2" Body="&lt;p&gt;It is just the computation of the second derivative from three non-equidistant points. It is computed by first computing (an approximation of) the derivatives on each of the two intervals adjacent to point $i$ using&#xA;$$&#xA;  \frac{T(i+1)-T(i)}{\Delta(i+1)}&#xA;$$&#xA;and&#xA;$$&#xA;  \frac{T(i)-T(i-1)}{\Delta(i)}&#xA;$$&#xA;and then computing (an approximation of) the second derivative by taking a finite difference stencil with regard to the midpoints of the two intervals:&#xA;$$&#xA;  \frac{&#xA;  \frac{T(i+1)-T(i)}{\Delta(i+1)}&#xA;  -&#xA;  \frac{T(i)-T(i-1)}{\Delta(i)}&#xA;  }{\frac{\Delta(i+1)+\Delta(i)}{2}}&#xA;$$&#xA;Here, $\Delta(i+1)=x(i+1)-x(i)$ and $\Delta(i)=x(i)-x(i-1)$ are the &lt;code&gt;dx1&lt;/code&gt; and &lt;code&gt;dx&lt;/code&gt; terms in the formula of the code you're looking at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no idea how this relates to the &lt;code&gt;dx&lt;/code&gt; and &lt;code&gt;dx1&lt;/code&gt; variables -- these look wrong to me since they are not &lt;em&gt;increments&lt;/em&gt; but averages of the &lt;code&gt;deltax(i)&lt;/code&gt; which themselves are not deltas (differences) but point locations as far as I can see. I can't seem to see that this choice of variable names or expressions makes sense.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-11-10T23:12:12.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28234" PostTypeId="2" ParentId="28225" CreationDate="2017-11-11T14:08:59.127" Score="9" Body="&lt;p&gt;The term &lt;em&gt;inverse crime&lt;/em&gt; for a numerical test of a parameter identification method that uses data contained in the range of the discrete(!) forward operator used for the inversion (thus essentially reducing the problem to a well-posed finite-dimensional one that behaves fundamentally different from the original infinite-dimensional one -- it is important to stress that being in the range is the problem here, not the finite-dimensionality) is indeed commonly attributed to &lt;a href=&quot;http://num.math.uni-goettingen.de/kress/&quot; rel=&quot;noreferrer&quot;&gt;Rainer Kress&lt;/a&gt;. From what I have heard (this was before my time), he coined this term in one of his talks; the first time it is found in print seems to be indeed in his book [1]&#xA;(on page 154 in the &lt;a href=&quot;https://link.springer.com/book/10.1007/978-1-4614-4942-3&quot; rel=&quot;noreferrer&quot;&gt;current, third, edition&lt;/a&gt;). This is in fact the usual reference given when people feel they need to give one for this concept. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also sometimes seen citations to [2], where the term is used frequently albeit in a slightly different context (but with the same general meaning); the authors also attribute it to Rainer Kress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;em&gt;Colton, David; Kress, Rainer&lt;/em&gt;, Inverse acoustic and electromagnetic scattering theory, Applied Mathematical Sciences. 93. Berlin: Springer-Verlag. x, 305 p. (1992). &lt;a href=&quot;https://zbmath.org/?q=an:0760.35053&quot; rel=&quot;noreferrer&quot;&gt;ZBL0760.35053&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;em&gt;Kaipio, Jari; Somersalo, Erkki&lt;/em&gt;, Statistical and computational inverse problems., Applied Mathematical Sciences 160. New York, NY: Springer (ISBN 0-387-22073-9/hbk). xvi, 339&amp;nbsp;p. (2005). &lt;a href=&quot;https://zbmath.org/?q=an:1068.65022&quot; rel=&quot;noreferrer&quot;&gt;ZBL1068.65022&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2017-11-11T14:08:59.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28236" PostTypeId="2" ParentId="28227" CreationDate="2017-11-11T21:19:24.800" Score="3" Body="&lt;p&gt;Switch solver to &lt;code&gt;CVODES&lt;/code&gt; or tighten the tolerance to 1e^-8.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Kwm80.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kwm80.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performing one of those changes results in this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QY8TX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QY8TX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24745" LastEditorUserId="414" LastEditDate="2017-11-20T07:30:03.960" LastActivityDate="2017-11-20T07:30:03.960" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28238" PostTypeId="2" ParentId="28130" CreationDate="2017-11-12T19:37:50.483" Score="2" Body="&lt;p&gt;You can work with &lt;strong&gt;classes&lt;/strong&gt;. There's a book called &quot;What every engineer should know about MATLAB and Simulink&quot; by Biran, and the problem you're describing is the example that is given in the OOP chapter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, he defines a class of &quot;physicalProperty&quot; whose objects have properties which are the power of the basic units for &lt;em&gt;length&lt;/em&gt;, &lt;em&gt;mass&lt;/em&gt; and &lt;em&gt;time&lt;/em&gt;. Whenever a value is created, you must specify the power of each basic unit (e.g. if you want to specify a length, the power of meters will be 1 and the powers of second and kg will be 0). Then, mathematical operations are defined such that multiplication/division is always allowed, resulting in the summation/subtraction of the different powers belonging to both operands, and addition/subtraction is only allowed between objects having the same powers for all units (i.e. if you try to subtract 5m from 1s you'll get an error).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation should not be difficult if the idea is clear. Good luck! l&lt;/p&gt;&#xA;" OwnerUserId="22380" LastActivityDate="2017-11-12T19:37:50.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28239" PostTypeId="2" ParentId="28179" CreationDate="2017-11-12T20:38:32.190" Score="1" Body="&lt;p&gt;Your numerical integration technique also need to be stable to $u_{t}=g(x,t)$ which may have different time step restrictions than the hyperbolic terms.&lt;/p&gt;&#xA;" OwnerUserId="25959" LastActivityDate="2017-11-12T20:38:32.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28240" PostTypeId="1" AcceptedAnswerId="28241" CreationDate="2017-11-12T23:06:01.720" Score="3" ViewCount="8510" Body="&lt;p&gt;I have a simple question today. I am new to GMSH and I am currently learning how to use the software program. I am using GMSH 3.0.4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am looking to do is to set a different mesh size for different faces in my 2D geometry. As a test, I am creating 2 squares next to each other. The square on the left will have a mesh size set by the element size factor in the options dialog.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been doing some research on this matter and came across this for posting:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://onelab.info/pipermail/gmsh/2009/004647.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://onelab.info/pipermail/gmsh/2009/004647.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read section 6.3.1 in the user manual. In this section, it is stated that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A MathEval field specifies the size of the mesh using an explicit mathematical function. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I went to Mesh-&gt;Define-&gt;Size Fields and then New-&gt;MathEval and then typed in 0.001 in the box. I also set a restrict field by going to New-&gt;restrict. I typed my edge list and the face list that I want to restrict the field. Since this I want to restrict my first Mesh field, I typed in 1 in the IFields box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also ensured that the 2D mesh algorithm is set to Delauney.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am still having difficulty in setting up a custom mesh size. I am attempting to bound a face to specific mesh size. I am posting my .geo file here so that others may take a look. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this: What is the best method that I can do on the GUI in order to bound different faces to different mesh sizes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that currently the Element size factor is set to 1. Also note that I am generating a mesh of only quads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My .geo file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//+&#xA;SetFactory(&quot;Built-in&quot;);&#xA;//+&#xA;Point(1) = {0, 0, 0, 1.0};&#xA;//+&#xA;Point(2) = {1, 0, 0, 1.0};&#xA;//+&#xA;Point(3) = {1, 1, 0, 1.0};&#xA;//+&#xA;Point(4) = {0, 1, 0, 1.0};&#xA;//+&#xA;Line(1) = {1, 4};&#xA;//+&#xA;Line(2) = {4, 3};&#xA;//+&#xA;Line(3) = {3, 2};&#xA;//+&#xA;Line(4) = {2, 1};&#xA;//+&#xA;Line Loop(1) = {2, 3, 4, 1};&#xA;//+&#xA;Plane Surface(1) = {1};&#xA;&#xA;Recombine Surface{1};&#xA;//+&#xA;Point(5) = {2, 1.2, 0, 1.0};&#xA;//+&#xA;Point(6) = {2, 0, 0, 1.0};&#xA;//+&#xA;Line(5) = {3, 5};&#xA;//+&#xA;Line(6) = {5, 6};&#xA;//+&#xA;Line(7) = {6, 2};&#xA;//+&#xA;Line Loop(2) = {5, 6, 7, -3};&#xA;//+&#xA;Plane Surface(2) = {2};&#xA;//+&#xA;Field[2] = Restrict;&#xA;//+&#xA;Field[2].EdgesList = {5, 6, 7, 3};&#xA;//+&#xA;Field[2].FacesList = {2};&#xA;//+&#xA;Field[1] = MathEval;&#xA;//+&#xA;Field[1].F = &quot;1.0&quot;;&#xA;&#xA;&#xA;//+&#xA;//Background Field = 1;&#xA;&#xA;//+&#xA;&#xA;//+&#xA;Field[1].F = &quot;0.01&quot;;&#xA;//+&#xA;Background Field = 1;&#xA;//+&#xA;Background Field = -1;&#xA;//+&#xA;Field[1].F = &quot;0.001&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2017-11-13T18:27:52.620" Title="Custom Mesh Sizes in GMSH" Tags="|mesh-generation|gmsh|" AnswerCount="1" CommentCount="0" ClosedDate="2017-11-13T20:37:29.877" ContentLicense="CC BY-SA 3.0" />
  <row Id="28241" PostTypeId="2" ParentId="28240" CreationDate="2017-11-13T08:42:07.687" Score="2" Body="&lt;p&gt;There are a couple of different ways to specify element mesh sizes in GMSH, I will explain the simplest one using the points here but for the others I suggest you check the manual:&#xA;&lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#Specifying-mesh-element-sizes&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://gmsh.info/doc/texinfo/gmsh.html#Specifying-mesh-element-sizes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you have in your example you can define a GMSH point as:&#xA;    Point(point #) = {x, y, z, lc};&#xA;where x, y, z are the coordinate values and lc is the element mesh size. So if you want an element mesh size of 0.1 you write&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Point(1) = {0,0,0,0.1};&#xA;Point(2) = {0,1,0,0.1};&#xA;Line(1)={1,2};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the line will be split into elements of 0.1 length. If you specify different lc's for different points GMSH will interpolate between them.&lt;/p&gt;&#xA;" OwnerUserId="16737" LastEditorUserId="16737" LastEditDate="2017-11-13T18:27:52.620" LastActivityDate="2017-11-13T18:27:52.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28242" PostTypeId="1" CreationDate="2017-11-13T13:28:35.783" Score="0" ViewCount="127" Body="&lt;p&gt;I am working with finite elements for Maxwell's Equations (i.e. with Nedelec's edge elements) and for computation I'm using the &lt;a href=&quot;https://fenicsproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEniCS-project&lt;/a&gt;. While implementing the Augmented Lagragian Method, I need to solve a PDE with a non-linearity of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;f(v) := \begin{cases} &#xA;v, \quad &amp;amp;\text{on } A=\lbrace x:|v(x)| \leq g(x)\rbrace\\&#xA;\frac{v}{|v|}g, &amp;amp;\text{on } \Omega\backslash A &#xA;\end{cases}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is this non-linearity computable. More specific: How can I check that the finite element-approximation of $v$ ist smaller than $g$ almost everywhere?&lt;/p&gt;&#xA;" OwnerUserId="25965" LastEditorUserId="25965" LastEditDate="2017-11-13T15:23:16.210" LastActivityDate="2017-11-13T16:47:16.403" Title="Computable alternative to &quot;almost everywhere&quot;" Tags="|pde|nonlinear-equations|fenics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28244" PostTypeId="2" ParentId="28242" CreationDate="2017-11-13T15:02:51.840" Score="0" Body="&lt;p&gt;Computability is not your only problem. How do you know that your finite element approximation preserves the property? Actually, it is not hard to show that there is no linear approximation scheme that does it for all $f$ and $g$, and once you make a single error somewhere, the whole PDE approximation may get totally worthless (that depends on the PDE, of course). In all honesty, your definition of $f(v)$ looks extremely fishy to me in the context of any PDE. Are you sure that what you really need is not&#xA;$$&#xA;f(v)=\begin{cases}v, |v|\le g\\\frac v{|v|}g, |v|&amp;gt;g\end{cases}&#xA;$$ &#xA;(i.e. the standard pointwise truncation by some fixed majorant)?&lt;/p&gt;&#xA;" OwnerUserId="25967" LastActivityDate="2017-11-13T15:02:51.840" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28245" PostTypeId="2" ParentId="28242" CreationDate="2017-11-13T16:30:51.500" Score="1" Body="&lt;p&gt;Regarding the question of how to check whether $|v|&amp;lt;g$ (which I would recommend to split into $v(x)&amp;lt;g(x)$ and $v(x)&amp;gt;-g(x)$, since $|v|$ will in general not be a piecewise polynomial), this is rather easy for piecewise constant functions and piecewise linear functions (for which the maximum and minimum are always attained in the nodes) since there it suffices to compare the coefficients of the corresponding vectors. For higher-order approximations, this is no longer true. (I'm assuming that $g$ is also a finite element function of the same order, otherwise you'll have the same problem as for higher-order approximations.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's not the &quot;almost everywhere&quot; that's the problem (since finite element functions are always continuous on each element), but the fact that you need to separate integrals over each element $T$ into contributions from  $T\cap A$ and $T\setminus A$ to assemble the term arising from the nonlinearity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest approach is to work with piecewise constant approximations to $v$, since then either $T\cap A= T$ or $T\cap A=\emptyset$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For higher-order approximations, you can approximate the integrals by either &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;applying the nonlinearity componentwise to the vector of expansion coefficients with respect to the (nodal) basis and multiply this by the (lumped) mass or stiffness matrix (depending on where exactly this term arises, which you haven't told us) or&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;using a quadrature scheme to evaluate the nonlinearity.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How to do that in FEniCS is off-topic here (but for 2., look at the &lt;code&gt;quadrature&lt;/code&gt; element together with UFL &lt;code&gt;conditional&lt;/code&gt;s); the error arising from this approximation can be estimated using Strang's Lemma.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;I see you've already posted your &lt;a href=&quot;https://www.allanswered.com/post/eqxnv/how-to-compute-fg-amost-everywhere-for-trialfunctions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;question&lt;/a&gt; there, but &lt;a href=&quot;https://fenicsproject.org/qa/5945/ufl-conditional-with-coefficient?show=5945#q5945&quot; rel=&quot;nofollow noreferrer&quot;&gt;this older question&lt;/a&gt; might be more what you are looking for.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2017-11-13T16:47:16.403" LastActivityDate="2017-11-13T16:47:16.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28248" PostTypeId="1" AcceptedAnswerId="28258" CreationDate="2017-11-14T01:09:54.047" Score="0" ViewCount="2398" Body="&lt;p&gt;I am writing a finite element code for heat transfer (scalar field problem) and starting from simple 4 node quadrilateral element. I tried computing conductance (stiffness) matrix in the physical coordinate systems and comparing the answer with isoparametric system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Element looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; |Y&#xA; |&#xA;4 _____________3&#xA; |             |&#xA; |             |&#xA; |             |&#xA; |             |&#xA; |             |&#xA; |_____________|  ----&amp;gt;X&#xA; 1             2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Side &lt;code&gt;1-2&lt;/code&gt; is not straight.)&#xA;Coordinates of nodes &lt;code&gt;1,2,3,4&lt;/code&gt; are :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1: (0,0)    &#xA;2: (2,0.5)    &#xA;3: (2,1)    &#xA;4: (0,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I make isoparametric transformation to bi-unit &lt;code&gt;(e,n)&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       |n&#xA;       |&#xA;4 _____________3&#xA; |     |       |&#xA; |     |       |&#xA; |     |------ | -----&amp;gt;e&#xA; |             |&#xA; |             |&#xA; |_____________|  &#xA; 1             2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, Coordinates of nodes &lt;code&gt;1,2,3,4&lt;/code&gt; become :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1: (-1,-1)    &#xA;2: (1,-1)    &#xA;3: (1,1)    &#xA;4: (-1,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I created a python simpy script to compute stiffness matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sympy as sp&#xA;import numpy as np&#xA;from sympy import init_printing&#xA;&#xA;n,e = sp.symbols('n e')&#xA;# shape functions in isoparametric&#xA;N1_iso = (1/4)*(e-1)*(n-1)&#xA;N2_iso = (1/4)*(e+1)*(-n+1)&#xA;N3_iso = (1/4)*(e+1)*(n+1)&#xA;N4_iso = (1/4)*(-e+1)*(n+1)&#xA;&#xA;#gradient of shape function matrix in isoparametric&#xA;GN_mat = sp.Matrix([ [sp.diff(N1_iso,e),sp.diff(N2_iso,e), sp.diff(N3_iso,e), sp.diff(N4_iso,e)],&#xA;         [sp.diff(N1_iso,n),sp.diff(N2_iso,n), sp.diff(N3_iso,n), sp.diff(N4_iso,n)]])&#xA;&#xA;#point matrix&#xA;P_mat = sp.Matrix([[0.0,0.0],[2.0,0.5],[2.0,1.0],[0.0,1.0]])&#xA;#Jacobian matrix&#xA;J_mat = (1.0/4.0)*(GN_mat*P_mat)&#xA;&#xA;#Jacobian matrix determinant and inverse &#xA;J_det=J_mat.det()&#xA;J_inv=J_mat.inv()&#xA;&#xA;#&quot;B&quot; matrix, assuming unit conductivity matrix &#xA;B_mat=J_inv*GN_mat&#xA;&#xA;#conductance (stiffness) matrix&#xA;expr1 = B_mat.T*B_mat*J_det&#xA;&#xA;#Double integrating conductance (stiffness) matrix&#xA;expr2 = sp.integrate(expr1,(n,-1,1))&#xA;#Stiffness (conductance) matrix using isoparametric transformation&#xA;K_mat=sp.integrate( (expr2),(e,-1,1))&#xA;print('K_mat = \n {}'.format(K_mat ))&#xA;&#xA;#Now in physical coordinate system&#xA;x,y = sp.symbols('x y')&#xA;#Area of the element = 2&#xA;A=1.5&#xA;&#xA;#shape function&#xA;N1 = (1/A)* (x-2)*(y-1)&#xA;N2 = (1/A)* (x-0)*(1-y)&#xA;N3 = (1/A)* (x-0)*(y-0.5)&#xA;N4 = (1/A)* (2-x)*(y-0)&#xA;&#xA;B_mat2 = sp.Matrix([[sp.diff(N1,x), sp.diff(N2,x), sp.diff(N3,x), sp.diff(N4,x)],&#xA;                    [sp.diff(N1,y), sp.diff(N2,y), sp.diff(N3,y), sp.diff(N4,y)]])&#xA;&#xA;expr10 = sp.integrate(B_mat2.T*B_mat2,(y,0,1))&#xA;#Stiffness (conductance) matrix using physical coordinates&#xA;K_mat2 = sp.integrate(expr10,(x,0,2))&#xA;print('     K_mat2 = \n {}'.format(K_mat2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I make point 2: (2,0.0), domain becomes rectangle and I get same answers. But, if it is not the case, I get different answers for &lt;code&gt;K_mat&lt;/code&gt; and &lt;code&gt;K_mat2&lt;/code&gt;. Any suggestions what is going wrong?&lt;/p&gt;&#xA;" OwnerUserId="25978" LastEditorUserId="25978" LastEditDate="2017-11-14T09:43:54.303" LastActivityDate="2017-11-15T19:19:50.663" Title="Stiffness matrix computation for 4 node quadrilateral element" Tags="|finite-element|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28249" PostTypeId="1" CreationDate="2017-11-14T08:03:52.953" Score="1" ViewCount="323" Body="&lt;p&gt;The augmented global stiffness matrix is not positive semi-definite when using Lagrange Multipliers method to enforce boundary constraints on a simple square domain of integral form:&lt;a href=&quot;https://i.stack.imgur.com/GofAL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GofAL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;I am considering linear elastic material constitutive.&#xA;I checked the condition number which is order 10e3 indicating the augmented matrix is not much ill-conditioned. However, there are some negative eigenvalues for the matrix. What are possible reasons for the lack of semi-definite positiveness? It should be always positive semi-definite for the (linearized) global stiffness matrix whatever Lagrange multipliers method is employed. Is my Lagrange multipliers discretization/approximation space is not well set up or I lost something very important? Anyone can share some experience? &lt;/p&gt;&#xA;" OwnerUserId="24268" LastActivityDate="2017-11-14T08:03:52.953" Title="the augmented global stiffness matrix is not positive semi-definite using Lagrange Multipliers method within FEM" Tags="|finite-element|numerical-analysis|boundary-conditions|constraints|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28250" PostTypeId="1" CreationDate="2017-11-14T12:37:57.380" Score="2" ViewCount="412" Body="&lt;p&gt;I am trying to solve a system of PDEs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H_{t} = \frac{0.3}{0.7} - \frac{0.005 B f(h(H))}{\theta} - \frac{0.3 f(h(H))}{0.7} + \frac{500}{0.7} (HH_x)_x + (HH_y)_y$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$N_t = \frac{N_{in} - 0.002 [N] B f(h(H)) + 0.1 \frac{0.002}{g} B - 0.1 N}{0.7 H} + 10 ([N]_{xx} + [N]_{yy}) + \frac{500}{0.7} ([N] H_x)_x + ([N] H_y)_y$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B_t = g [N] B f(h(H)) - 0.3 B + 2 (B_{xx} + B_{yy})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the method of lines. What finite difference method(s) would be suitable for attempting a solution? Most of the examples I can find are either in 1 spatial dimension only or for a single variable. Since this is a convection-diffusion system &lt;/p&gt;&#xA;" OwnerUserId="25954" LastEditorUserId="25954" LastEditDate="2017-11-14T15:26:05.517" LastActivityDate="2021-11-24T05:01:35.153" Title="Suitable finite difference method for a convection-diffusion system?" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28251" PostTypeId="2" ParentId="28250" CreationDate="2017-11-14T13:17:19.763" Score="3" Body="&lt;p&gt;The $B$ equation is parabolic and you can use standard central finite difference. The $H$ equation also looks like parabolic so you can do a central difference like&#xA;$$&#xA;(H H_x)_x = \frac{H_{i+1/2}(H_{i+1}-H_i) - H_{i-1/2}(H_i - H_{i-1})}{h^2}, \qquad H_{i+1/2}=\frac{H_i + H_{i+1}}{2}&#xA;$$&#xA;Equivalently we could approximate as&#xA;$$&#xA;(HH_x)_x = \frac{(H^2)_{xx}}{2} = \frac{H_{i-1}^2 - 2H_i^2 + H_{i+1}^2}{2h^2}&#xA;$$&#xA; If $H$ becomes zero or negative, that could be problematic since it corresponds to vanishing viscosity or negative viscosity. Does this happen in your model? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there no time derivative in the middle equation? For $N_{xx}$ etc., you can do central difference and terms like $(NH_x)_x$ can be again central differenced as in the first formula above. Note that these suggestions are just based on consistency and you have to look at the stability of these schemes.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="20688" LastEditDate="2018-09-10T21:27:04.177" LastActivityDate="2018-09-10T21:27:04.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28252" PostTypeId="1" CreationDate="2017-11-14T15:01:58.970" Score="1" ViewCount="23" Body="&lt;p&gt;Posted this at math stack exchange as well, but alas no replies! So, I have been trying to find ways of distributing particles of spherical or other shape in 3D space, e.g. rectangular space. Random() and checking for overlaps will only get you up to a certain volume fraction percentage, then it is nearly impossible to find a non-overlapping position. I know there is a way to &quot;drive&quot; particles in empty spaces by some sort of Monte Carlo distribution or using some kind of potential difference? A dirty example in pseudo code or Matlab would be awesome!&lt;/p&gt;&#xA;" OwnerUserId="25093" LastActivityDate="2017-11-14T15:01:58.970" Title="Uniform distribution in 3D space" Tags="|monte-carlo|numerical-modelling|statistics|" AnswerCount="0" CommentCount="1" ClosedDate="2017-11-14T21:37:30.230" ContentLicense="CC BY-SA 3.0" />
  <row Id="28253" PostTypeId="1" CreationDate="2017-11-14T16:16:59.077" Score="0" ViewCount="469" Body="&lt;p&gt;I'm trying to do a high precision discrete fourier transform on a signal. To examine the precision, I use a gaussian function as the signal, because the fourier transform is also a gaussian function. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/AfWDx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AfWDx.png&quot; alt=&quot;fourier transform of gaussian function using Mathematica&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;tgmath.h&amp;gt;&#xA;&#xA;typedef long double complex cplx;&#xA;long double PI;&#xA;void _fft(cplx buf[], cplx out[], int n, int step)&#xA;{&#xA;if (step &amp;lt; n) {&#xA;    _fft(out, buf, n, step * 2);&#xA;    _fft(out + step, buf + step, n, step * 2);&#xA;&#xA;    for (int i = 0; i &amp;lt; n; i += 2 * step) {&#xA;        cplx t = cexpl(- I * PI * i / n) * out[i + step];&#xA;        buf[i / 2]     = out[i] + t;&#xA;        buf[(i + n)/2] = out[i] - t;&#xA;    }&#xA;}&#xA;}&#xA;&#xA;void fft(cplx buf[], int n)&#xA;{&#xA;cplx out[n];&#xA;for (int i = 0; i &amp;lt; n; i++) out[i] = buf[i];&#xA;&#xA;_fft(buf, out, n, 1);&#xA;}&#xA;&#xA;&#xA;int main()&#xA;{&#xA;const int nPoints = pow(2, 12);&#xA;PI = atan2(1, 1) * 4;&#xA;cplx dt = 1e-3;&#xA;cplx dOmega = 1 / (dt * nPoints);&#xA;long double T[nPoints];&#xA;long double DOmega[nPoints];&#xA;&#xA;cplx At[nPoints];&#xA;cplx tau = 28.4e-3;&#xA;for (int i = 0; i &amp;lt; nPoints; ++i)&#xA;{&#xA;    T[i] = dt*(i-nPoints/2);&#xA;    DOmega[i] = dOmega * (i - nPoints / 2);&#xA;    At[i] = cexpl(-T[i]*T[i]/2/(tau*tau));&#xA;}&#xA;&#xA;fft(At, nPoints);&#xA;FILE* fw;&#xA;fw = fopen(&quot;fft_01.txt&quot;, &quot;w+&quot;);&#xA;&#xA;for (int i = 0; i &amp;lt; nPoints; ++i)&#xA;{&#xA;    fprintf(fw, &quot;%.15Le, %.15Le\n&quot;, DOmega[i], fabs(At[i]) );&#xA;}&#xA;&#xA;return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I change the defined type from float complex to double complex, the result did show a improvement in precision. However, When I change the data type from double complex to long double complex, there's no improvement while the numbers are far away from the minimal limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RbveH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RbveH.png&quot; alt=&quot;result with float data&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ETXrL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ETXrL.png&quot; alt=&quot;result using double data &quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hDZ1S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hDZ1S.png&quot; alt=&quot;result using long double data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that the two wings of the peak should go down like the results by Mathematica, but here it seems stuck around 10^(-35) and can not further get down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I further apply a iDFT to the result of DFT, it's more apparent that the long double precision has been lost. The blue one is the original signal, with high precision. The green one is the result after &lt;strong&gt;iDFT(DFT(x))&lt;/strong&gt;, which should be coincide with the blue one, but the minimal numbers are much larger.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5kBuI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5kBuI.png&quot; alt=&quot;result by iDFT(DFT(x)) &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone can tell me where is the problem, thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I am on Mac OS 10.13.1, using &lt;code&gt;gcc&lt;/code&gt; as the compiler.&#xA;I have checked the numerical limits on my platform&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;storage size for float: 4&#xA;minimum float positive value: 1.175494e-38&#xA;Maximum float positive value: 3.402823e+38&#xA;precision value: 6&#xA;&#xA;storage size for double: 8&#xA;minimum double positive value: 2.225074e-308&#xA;Maximum double positive value: 1.797693e+308&#xA;precision value: 15&#xA;&#xA;storage size for long double: 16&#xA;minimum long double positive value: 3.362103e-4932&#xA;Maximum long double positive value: 1.189731e+4932&#xA;precision value: 18&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25985" LastEditorUserId="25985" LastEditDate="2017-11-15T08:29:22.643" LastActivityDate="2018-07-17T03:40:12.143" Title="High precision Discrete Fourier Transform in c" Tags="|c|precision|fourier-transform|" AnswerCount="1" CommentCount="19" ContentLicense="CC BY-SA 3.0" />
  <row Id="28254" PostTypeId="2" ParentId="27139" CreationDate="2017-11-14T18:22:38.777" Score="2" Body="&lt;p&gt;2D turbulence has an inverse energy cascade which feeds the formation of large-scale vortices.  Starting with a random field will produce some structures that are rapidly dissipated, and then you settle into normal evolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it's not clear from your question what your initial conditions are, nore really what you expect to see.&lt;/p&gt;&#xA;" OwnerUserId="3182" LastActivityDate="2017-11-14T18:22:38.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28256" PostTypeId="2" ParentId="28253" CreationDate="2017-11-15T04:34:31.523" Score="1" Body="&lt;p&gt;In addition to the various comments that point out that most built-in functions such as &lt;code&gt;atan&lt;/code&gt; or &lt;code&gt;exp&lt;/code&gt; operate on &lt;code&gt;double precision&lt;/code&gt; variables, there is also the issue that &lt;code&gt;long double&lt;/code&gt; on typical x86-type hardware is not actually much more accurate (if at all) than regular &lt;code&gt;double&lt;/code&gt;. In particular, &lt;code&gt;long double&lt;/code&gt; is not an &lt;em&gt;arbitrary precision&lt;/em&gt; data type such as the ones that Mathematica uses -- Mathematica lets you choose the accuracy of variables however you want it, and it will compute things to this accuracy, but this is not the case for C's &lt;code&gt;long double&lt;/code&gt; data type that has a fixed accuracy just like &lt;code&gt;double&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-11-15T04:34:31.523" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28257" PostTypeId="1" AcceptedAnswerId="28266" CreationDate="2017-11-15T11:21:40.050" Score="8" ViewCount="397" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;Background:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lagrange multiplier method has been employed in numerous fields, such as contact problems, material interfaces, phase transformation, stiff constraints or sliding along interfaces.&lt;br&gt;&#xA;It is well known that a bad choice or design of Lagrange multiplier space will produce oscillatory results (unstable problem) on Lagrange multipliers. A huge amount of literature has illustrated this observation and some modifications or improvements have been made to remove oscillations which are typically incurred by deviation of inf-sup condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Question:&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When reading literature on XFEM, I came across the below argument highlighted in red, which is quite mathematical. How to interpret or understand the space is locally too rich and then as a result the inf-sup condition violates? Thanks for any contribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pR0gJ.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pR0gJ.jpg&quot; alt=&quot;Lagrange multipliers&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24268" LastActivityDate="2017-11-16T11:38:06.380" Title="Lagrange multipliers space is too rich in a mathematical view" Tags="|finite-element|numerical-analysis|stability|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28258" PostTypeId="2" ParentId="28248" CreationDate="2017-11-15T11:35:06.613" Score="1" Body="&lt;p&gt;As I'm sure you're already aware, the Galerkin finite element method expands both test and trial function spaces in terms of a finite basis, $$T=\sum_i T_i N_i(\mathbf{x}).$$&#xA;Hopefully you're also aware that different choices of the $N$s will give different discretizations, and thus different matrix problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The  isoparametric bilinear element for quadrilaterals, Q4, is &lt;strong&gt;defined&lt;/strong&gt; by the shape functions you give for the reference quadrilateral (your $N^\mbox{iso}$), along with a spatial transformation from the reference to physical space, $$ x(e,n) = x_1 N_1(e,n) + x_2 N_2(e,n) +x_3 N_3(e,n) + x_4 N_4(e,n),$$&#xA;$$ y(e,n) = y_1 N_1(e,n) + y_2 N_2(e,n) +y_3 N_3(e,n) + y_4 N_4(e,n).$$&#xA;This means that for a given element we can trivially write&#xA;$$ N_i(x(e,n),y(e,n)) = N_i(e,n), $$&#xA;but to give an explicit definition in terms of $x$ and $y$ we have to invert the relation above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point it should be obvious that you've chosen a different physical basis, since (1) your physical $N_1$ shape function isn't equal to unity at vertex 1 and (2) your shape functions don't all vanish on the edges not defined by &quot;their&quot; vertex. The last point is important, since it's what guarantees that the Q4 representation is continuous across elements. The reason that your code works when you change the vertex position is that you then have a physical quadrilateral which is a linear translation of the reference quadrilateral. In that case, the (trivial) invertion does give back what you've written.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="19263" LastEditDate="2017-11-15T19:19:50.663" LastActivityDate="2017-11-15T19:19:50.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28259" PostTypeId="2" ParentId="28257" CreationDate="2017-11-15T15:55:16.917" Score="5" Body="&lt;p&gt;Each Lagrange multiplier corresponds to a constraint. So if the space of Lagrange multipliers is too large, then you have too many constraints that can no longer be all satisfied at the same time without significantly restricting the number of unknowns you have available to satisfy the physics of the problem. This is what happens in locking: each constraint reduces the number of unknowns by one, and you end up with too few unknowns to be physically accurate.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2017-11-16T02:23:19.717" LastActivityDate="2017-11-16T02:23:19.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28261" PostTypeId="2" ParentId="28104" CreationDate="2017-11-15T20:39:01.673" Score="1" Body="&lt;p&gt;One problem with converting Tex syntax to equations is that it can be ambiguous - the main goal with TeX is typesetting the look, not the actual meaning.    If you have many formulas already in TeX, then looking to parse them into a symbolic form is okay.  But if starting from scratch, a symbolic mathematics package would be a better choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue is that converting symbolic equations to code is very domain-specific.  In order for mathematical notation to be concise, there are many assumptions and implicit context surrounding the written formulas.  Any system for interpreting formulas needs to know about these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most symbolic mathematics packages have support for code generation.  For some examples in Sympy, see the &lt;a href=&quot;http://www.sympy.org/scipy-2017-codegen-tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sympy code generation tutorial&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20181" LastActivityDate="2017-11-15T20:39:01.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28263" PostTypeId="1" CreationDate="2017-11-16T02:40:51.990" Score="6" ViewCount="160" Body="&lt;p&gt;I have implemented a pretty straightforward finite element solver for the following Poisson equation.  For the purposes of this question we can assume the source term and the Dirichlet data both vanish.  I assume that $g\in H^{-1/2}(\partial \Omega_N) \cap L^2(\partial \Omega_N)$ (instead of just $g\in H^{-1/2}(\partial\Omega_N)$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{aligned}&#xA;\mathbf{q} + \mathbf{\nabla}u &amp;amp;= 0 &amp;amp; \mathrm{in\,} \Omega,\\&#xA;\mathbf{\nabla} \cdot \mathbf{q} &amp;amp;= 0&amp;amp; \mathrm{in\ } \Omega,\\&#xA;u&amp;amp; = 0 &amp;amp;\mathrm{on\ } \partial\Omega_D,\\&#xA;\mathbf{q}\cdot \mathbf{\eta} &amp;amp;= g &amp;amp; \mathrm{on\ } \partial\Omega_N.&#xA;\end{aligned}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I accounted for Neumann boundary conditions on all or some of the boundary by using Lagrange multipliers as follows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the The domain $\Omega$ have a triangulation $T$ made up of simplexes $K$.&#xA;  We denote $$\mathbf{W}_h= \{\mathbf{p}\in \mathbf{H}^{\mathrm{div}}(\Omega)\colon \forall K\in T \quad \mathbf{p}\big|_K \in \mathrm{RT}^k(K))$$ &#xA;$$V_h = \{v\in L^2(\Omega)\colon \forall K \quad v\big|_K \in P^k(K)\}$$&#xA;$$X_h = \{\mu \in L^2(\partial \Omega_N)\colon \forall K\in T\,\, \forall \mathrm{edge}\subset\partial K\cap \partial\Omega_N\quad \mu \big|_{\mathrm{edge}} \in P^k(\mathrm{edge})\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathrm{RT}^k(K)$ is the standard Raviart Thomas space and $P^k(K)$ is the space of polynomials of total degree no more than $k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is to find the triple $(\mathbf{q}_h,u_h,\lambda_h) \in \mathbf{W}_h \times V_h \times X_h$ so that for all test functions $(\mathbf{p}, v, s) \in (W_h \times V_h \times X_h)$ we have&#xA;\begin{aligned}&#xA;0 &amp;amp;= \int_\Omega \mathbf{p}\cdot \mathbf{q}_h\, \mathrm{d} x - \int_\Omega (\mathbf{\nabla} \cdot \mathbf{p}) u_h\, \mathrm{d} x + \int_{\partial\Omega_N} \mathbf{p} \lambda_h \cdot \mathbf{\eta}\, \mathrm{d} x \\&#xA;0 &amp;amp;= \int_\Omega v (\mathbf{\nabla} \cdot \mathbf{q}_h)\, \mathrm{d} x\\&#xA;\int_{\partial\Omega_N}s g \, \mathrm{d}x&amp;amp;= \int_{\partial\Omega_N} s\mathbf{q}_h \cdot \mathbf{\eta}\, \mathrm{d} x&#xA;\end{aligned}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scheme works (well defined, converges to true solution, etc.) and is (I think) close to standard practice. However, in all the references I have seen in the literature (&lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1002/num.10040/abstract&quot; rel=&quot;noreferrer&quot;&gt;On the Mixed Finite Element Method with Lagrange Multipliers&lt;/a&gt; Babuska and Gatica, 2003 Numerical Methods for Partial Differential Equation and section 4.4 of &lt;a href=&quot;http://www.springer.com/us/book/9783319036946&quot; rel=&quot;noreferrer&quot;&gt;A simple Introduction to the Mixed Finite Element Method&lt;/a&gt;  by Gatica) require that the trial and test space for the Lagrange multiplier, $X_h$, have elements that are continuous across elements of the &quot;boundary mesh&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this scheme close to standard practice?&lt;/li&gt;&#xA;&lt;li&gt;Are there any references I could read that discuss this scheme?&lt;/li&gt;&#xA;&lt;li&gt;What problems am I allowing by choosing a piecewise discontinuous space for the Lagrange multiplier?&lt;/li&gt;&#xA;&lt;li&gt;A disadvantage of the scheme is that it requires extra regularity on $g$.  Are there more disadvantages?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16799" LastActivityDate="2017-11-16T02:40:51.990" Title="Do practice and theory differ substantially when implementing Neumann Boundary Conditions using a Mixed Method?" Tags="|finite-element|boundary-conditions|reference-request|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28266" PostTypeId="2" ParentId="28257" CreationDate="2017-11-16T11:32:49.083" Score="5" Body="&lt;p&gt;The saddle point matrix you must solve takes the following form:&#xA;$$\begin{bmatrix}&#xA;A &amp;amp; B^T \\ B&#xA;\end{bmatrix},$$&#xA;where $A$ is the unconstrained matrix and $B$ is the matrix that observes the values of the solution at nodes along the immersed interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The conditioning of the saddle point matrix is, in part, controlled by the minimum singular value of $B$. The smaller the minimum singular value of $B$, the worse the conditioning. For details I highly recommend the following paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Krendl, Wolfgang, Valeria Simoncini, and Walter Zulehner. &quot;Stability&#xA;  estimates and structural spectral properties of saddle point&#xA;  problems.&quot; Numerische Mathematik 124.1 (2013): 183-213. &lt;a href=&quot;https://arxiv.org/pdf/1202.3330.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://arxiv.org/pdf/1202.3330.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now let us look in detail at the matrix $B$ for an interface immersed in a mesh like in the following picture (black is the mesh, red is the immersed interface):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Aa2oI.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Aa2oI.png&quot; alt=&quot;boundary locking&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $B$ takes the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B = \begin{bmatrix}&#xA;1/2 &amp;amp; 1/2 \\&#xA;&amp;amp; 1/2 &amp;amp; 1/2 \\&#xA;&amp;amp; &amp;amp; 1/2 &amp;amp; 1/2 \\&#xA;&amp;amp;&amp;amp;&amp;amp; \ddots&amp;amp; \ddots&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you right-multiply $B$ by a vector that oscillates highly, the result is small, since the positive and negative contributions tend to cancel out. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\underbrace{\begin{bmatrix}1 &amp;amp; -1 &amp;amp; 1 &amp;amp; -1 &amp;amp;\dots\end{bmatrix}}_{\text{large norm}} ~B = \underbrace{\begin{bmatrix}1/2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \dots\end{bmatrix}}_{\text{small norm}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, $B$ has small singular values, corresponding to oscillatory interface vectors. Hence the saddle point matrix is ill-conditioned, and it gets worse the finer the mesh is.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2017-11-16T11:38:06.380" LastActivityDate="2017-11-16T11:38:06.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28267" PostTypeId="1" CreationDate="2017-11-16T19:51:17.377" Score="1" ViewCount="387" Body="&lt;p&gt;This is perhaps a more focused version of this &lt;a href=&quot;https://scicomp.stackexchange.com/questions/28263/do-practice-and-theory-differ-substantially-when-implementing-neumann-boundary-c&quot;&gt;question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using standard notation, I have a code that works and solves the following PDE using a Raviart Thomas mixed method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;0 &amp;amp;= \mathbf{q} + \mathbf{\nabla}u &amp;amp;&amp;amp; x\in \Omega\\&#xA;0 &amp;amp;= \mathbf{\nabla} \cdot \mathbf{q} &amp;amp;&amp;amp; x\in \Omega\\&#xA;g &amp;amp;= \mathbf{q}\cdot \mathbf{\eta} &amp;amp; &amp;amp;x\in \partial\Omega_N\\&#xA;0 &amp;amp;= u&amp;amp;&amp;amp; x\in \partial\Omega_D&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that the measure of $\partial \Omega_D$ is strictly positive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this.  Where can I find a reference that discusses how to solve this problem using Raviart Thomas Elements?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the on page 404 and continuing onto the next page of &quot;Mixed Finite Elements and Applications&quot;, by Brezzi, Boffin and Fortin more or less assumes that we have a way of generating an exact solution to this problem easily.  Indeed, my $\mathbf{q}$ is their $\widetilde{\mathbf{u}}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in reading and citing a source that discusses this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you. &lt;/p&gt;&#xA;" OwnerUserId="16799" LastActivityDate="2017-11-17T15:14:33.867" Title="Raviart Thomas Mixed Finite Element with Mixed boundary conditions reference request" Tags="|finite-element|boundary-conditions|elliptic-pde|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28268" PostTypeId="2" ParentId="28267" CreationDate="2017-11-17T09:30:53.733" Score="2" Body="&lt;p&gt;In this case,  the essential boundary condition is Neumann and natural is Dirichlet. Since Neumann is essential and is nonhomogeneous, you can use the concept of lift, referred by a couple of authors, for example Demkowicz book about HP finite elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Follow explanation from Demokowicz book.The $\tilde {\mathbf{q}}(x)$ is a lift on Neumann data $g$. Lift $\tilde {\mathbf{q}}(x)$ coincide with the data on boundary if $x \in \partial \Omega _N$. In other words, lift is an extension of $g$ to the whole domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With lift at hand, we can subtract it from the solution, and the difference will vanish on the Neumann boundary. The collection of such sums is identified as the algebraic sum of lift $\tilde {\mathbf{q}}(x)$ and space $H^\textrm{div}_0(\Omega)=\{\mathbf{q} \in H^\textrm{div} \,|\, \eta \mathbf{q}(x)=0\;x\in\partial\Omega_N   \}$, and called an affine space:&#xA;\begin{equation}&#xA;\tilde {\mathbf{q}} + H^\textrm{div}_0(\Omega) = \left\{&#xA;\tilde {\mathbf{q}} + \mathbf{q}: \mathbf{q} \in  H^\textrm{div}_0(\Omega)&#xA;\right\}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final variational formulation can be expressed in variational formulation&#xA;\begin{equation}&#xA;\left\{&#xA;\begin{array}{l}&#xA;\mathbf{q} \in \tilde {\mathbf{q}} + H^\textrm{div}_0(\Omega) \\&#xA;u \in L^2(\Omega) \\&#xA;(\mathbf{q},\tau) + (u,\nabla \cdot \boldsymbol\tau) = 0\quad\forall \boldsymbol\tau \in H^\textrm{div}_0(\Omega)  \\&#xA;(\nabla \cdot \mathbf{q}, v) = 0\quad\forall v \in L^2(\Omega) &#xA;\end{array}&#xA;\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That can be formulated in another way, more convenient for implementation. Once we have found particular function $\tilde{\mathbf{q}}(x)$ that satisfies the nonhomogeneous Neumann data, we can simply make substitution $\mathbf{q} = \tilde{\mathbf{q}} + \mathbf{w}$ where $\mathbf{w} \in H^\textrm{div}_0(\Omega) $ satisfies homogenous boundary data, that corresponds to formulation&#xA;\begin{equation}&#xA;\left\{&#xA;\begin{array}{l}&#xA;\mathbf{w} \in H^\textrm{div}_0(\Omega) \\&#xA;u \in L^2(\Omega) \\&#xA;(\mathbf{w},\boldsymbol\tau) + (u,\nabla \cdot \boldsymbol\tau) = -(\tilde{\mathbf{q}},\boldsymbol\tau) \quad\forall \boldsymbol\tau \in H^\textrm{div}_0(\Omega)  \\&#xA;(\nabla \cdot \mathbf{w}, v) = -(\nabla \cdot \tilde{\mathbf{q}}, v) \quad\forall v \in L^2(\Omega) &#xA;\end{array}&#xA;\right.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At that point, one can see that there is no difference what we do in classical elements and mix formulation, only we need to identify what is essential boundary condition, which has to be satisfied a priori. Practically we eliminate DOFs, and calculating residuals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, is small technical difference, in classical finite element formulation DOFs have physical meaning, and we used to express $\tilde{\mathbf{q}}$ in the space of finite elements by setting nodal values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case like here, for convenience one like to approximate $\tilde{\mathbf{q}}$ on finite element base functions. That simplify implementation. It can be noticed that for RT elements, attention can be focussed on faces on Neumann boundary. For each face where Neumann boundary is applied, small face-by-face problem can be solved to find $\tilde{\mathbf{q}}^h$ as a linear combination of finite element base functions&#xA;\begin{equation}&#xA;&amp;lt;\eta\cdot\gamma(\boldsymbol\tau^{h,f}),\eta\cdot\gamma (\tilde{\mathbf{q}}^{h,f})-g&amp;gt;_{\partial\Omega^{h,f}_N} = 0 \quad\forall \gamma(\boldsymbol\tau)^{h,f}&#xA;\end{equation}&#xA;where $f$ is face index where boundary condition is applied and $\gamma$ is a trace operator, and trace of base function on face is piecewise polynomial for RT element. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not found implementation or paper for a general transient nonlinear problem with nonhomogenous Neumann bc like this, but we manage to implement this with hierarchical spaces in MoFEM (I am the one developers of that code), which can handle such problems easily. So you can look how is done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practical implementation for finding lift is here,&#xA;&lt;a href=&quot;http://mofem.eng.gla.ac.uk/mofem/html/struct_mix_transport_1_1_mix_transport_element_1_1_op_evaluate_bc_on_fluxes.html#ad175a300e635a0f472a947373a2e7b43&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mofem.eng.gla.ac.uk/mofem/html/struct_mix_transport_1_1_mix_transport_element_1_1_op_evaluate_bc_on_fluxes.html#ad175a300e635a0f472a947373a2e7b43&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is applied to time dependent nonlinear problem of fluid flow in unsaturated soil&#xA;&lt;a href=&quot;http://mofem.eng.gla.ac.uk/mofem/html/mix_us_flow.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mofem.eng.gla.ac.uk/mofem/html/mix_us_flow.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-11-17T15:14:33.867" LastActivityDate="2017-11-17T15:14:33.867" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28269" PostTypeId="1" AcceptedAnswerId="28382" CreationDate="2017-11-17T16:57:51.217" Score="6" ViewCount="218" Body="&lt;p&gt;The &lt;a href=&quot;https://www.csee.umbc.edu/%7Etsimo1/CMSC455/IEEE-754-2008.pdf&quot; rel=&quot;noreferrer&quot;&gt;IEEE Standard for Floating-Point &#xA;Arithmetic&lt;/a&gt; section &lt;em&gt;&quot;9.2 Recommended correctly rounded functions&quot;&lt;/em&gt; lists functions that are recommended (but not required) by a language standard to provide, among them an &lt;code&gt;exp2&lt;/code&gt; function as a particular example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a language provides &lt;code&gt;exp2&lt;/code&gt;, does the standard &lt;em&gt;require&lt;/em&gt; it to be correctly rounded?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a particular example, I took the &lt;code&gt;exp2&lt;/code&gt; function from the &lt;a href=&quot;https://github.com/JuliaLang/openlibm&quot; rel=&quot;noreferrer&quot;&gt;openlibm&lt;/a&gt; library, and found it to be correctly rounded in 99.96% of cases. However, in the rest of the cases (e.g., 37 cases from 100,000), it produced an answer that was off by 1 ulp. It's obviously very accurate, but it nevertheless was not correctly rounded in those 37 cases. Does this mean that the &lt;code&gt;exp2&lt;/code&gt; function in &lt;code&gt;openlibm&lt;/code&gt; is not IEEE-754 compliant?&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2017-12-04T22:26:32.920" Title="Does the IEEE-754 standard mandate that exp2 is rounded correctly?" Tags="|floating-point|libraries|accuracy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28270" PostTypeId="1" CreationDate="2017-11-17T22:32:39.613" Score="2" ViewCount="218" Body="&lt;p&gt;I'm trying to understand what a mixed FE method is, and I was recommended this pdf: &lt;a href=&quot;http://www.ima.umn.edu/~arnold/papers/mixed.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ima.umn.edu/~arnold/papers/mixed.pdf&lt;/a&gt; (Arnold, Mixed Finite Element Methods for Elliptic Problems).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On first page the author states the motivation as 'treating stress and displacement fields as primary variables'. However the equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbb{A}\mathcal{S} = \mathcal{E}(\mathbf{u})&#xA;\\&#xA;\mathbf{div}\,[\mathcal{S}] = \mathbf{f}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author says $\mathbb{A}$ is a SPD operator so first equation can be rewritten as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathcal{S} = \mathbb{C}\,\mathcal{E}(\mathbf{u})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However we can substitute this into the 2nd of the first two equations and get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{div}\,[\mathbb{C}\mathcal{E}(\mathbf{u})] = \mathbf{f}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and so $\mathcal{S}$ is eliminated, and we don't need to consider two primary variables (all we need to do is to solve for $\mathbf{u})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what's a better motivation for using mixed-method or what is wrong with the above reasoning?&lt;/p&gt;&#xA;" OwnerUserId="15198" LastActivityDate="2017-11-26T22:30:41.403" Title="Question about motivation of mixed FE method" Tags="|finite-element|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28271" PostTypeId="1" CreationDate="2017-11-17T23:16:07.533" Score="4" ViewCount="712" Body="&lt;p&gt;I have heard this many times that &quot;FD is a special case of FE&quot; and separately, that &quot;FV is a special case of FE&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I have never come across a brief document that substantiates that claim, through derivation, examples, proofs, whatever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the basics of FE and the 3 manifestations: Collocation, Least-squares, Galerkin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also understand that these three can be considered as various kinds of &quot;weighted residual&quot; schemes, and the unifying principle is MWR (method of weighted residuals) which is a variational method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there two easy-to-read resources, one for FD and one for FV, that can make the connection to either FE or MWR or something like that?&lt;/p&gt;&#xA;" OwnerUserId="15198" LastActivityDate="2024-01-12T15:51:29.920" Title="Finite Difference and Finite Volume as special cases of Finite Element" Tags="|finite-element|finite-difference|finite-volume|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28272" PostTypeId="1" CreationDate="2017-11-18T04:09:14.367" Score="1" ViewCount="40" Body="&lt;p&gt;I wonder why does this algorithm below work well? Many appreciates!&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Task:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Suppose all elements in an array with length $n = 2^m$ are in descending order: $a_1 &amp;gt; a_2 &amp;gt; ... &amp;gt; a_n$, we need to turn them into ascending order.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Algorithm:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;For each step from $1$ to $m$: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Swap each adjacent (odd index, even index) pair into ascending order, so that:&#xA;$a_1 \le a_2, a_3 \le a_4, ... , a_{n-1} \le a_n$&lt;/li&gt;&#xA;&lt;li&gt;Move all elements with odd index to the front of those&#xA;with even index, so $a_1, a_2, a_3, a_4, ..., a_{n - 1}, a_n$ turns to : $a_1, a_3, ..., a_{n - 1}, a_2, a_4, ..., a_n$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h1&gt;Example:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Sort result after each step with $n = 8$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[0] 8 7 6 5 4 3 2 1&lt;br/&gt;&#xA;[1] 7 5 3 1 8 6 4 2&lt;br/&gt;&#xA;[2] 5 1 6 2 7 3 8 4&lt;br/&gt;&#xA;[3] 1 2 3 4 5 6 7 8&lt;br/&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Code:&lt;/h1&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdlib.h&amp;gt;&#xA;&#xA;void swap(int *arr, int i, int j)&#xA;{&#xA;    int temp = arr[i];&#xA;    arr[i] = arr[j];&#xA;    arr[j] = temp;&#xA;}&#xA;&#xA;void magic_sort(int *arr, int *result, int n)&#xA;{&#xA;    int half = n &amp;gt;&amp;gt; 1;&#xA;    int i;&#xA;&#xA;    /* Swap adjacent (odd index, even index) pair into ascending order */&#xA;    for (i = 1; i &amp;lt;= n; i += 2)&#xA;    {&#xA;        if (arr[i] &amp;gt; arr[i + 1])swap(arr, i, i + 1);&#xA;    }&#xA;&#xA;    /* Move all elements with odd index to front half */&#xA;    for (i = 1; i &amp;lt;= n; i += 2)&#xA;    {&#xA;        result[(i + 1) / 2] = arr[i];&#xA;    }&#xA;&#xA;    /* Move all elements with even index to tail half */&#xA;    for (i = 2; i &amp;lt;= n; i += 2)&#xA;    {&#xA;        result[half + i / 2] = arr[i];&#xA;    }&#xA;&#xA;    for (i = 1; i &amp;lt;= n; i++)&#xA;    {&#xA;        arr[i] = result[i];&#xA;    }&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    int m = 3;&#xA;    int n = 1 &amp;lt;&amp;lt; m;&#xA;    int *arr = (int *)malloc(sizeof(*arr) * (n + 1));&#xA;    int *result = (int *)malloc(sizeof(*arr) * (n + 1));&#xA;    int step, i;&#xA;&#xA;    printf(&quot;[%d] &quot;, 0);&#xA;&#xA;    for (i = 1; i &amp;lt;= n; i++)&#xA;    {&#xA;        arr[i] = n + 1 - i;&#xA;&#xA;        printf(&quot;%d &quot;, arr[i]);&#xA;    }&#xA;&#xA;    printf(&quot;\n&quot;);&#xA;&#xA;    for (step = 0; step &amp;lt; m; step++)&#xA;    {&#xA;        magic_sort(arr, result, n);&#xA;&#xA;        printf(&quot;[%d] &quot;, step + 1);&#xA;&#xA;        for (i = 1; i &amp;lt;= n; i++)&#xA;        {&#xA;            printf(&quot;%d &quot;, arr[i]);&#xA;        }&#xA;&#xA;        printf(&quot;\n&quot;);&#xA;    }&#xA;&#xA;    free(arr);&#xA;    free(result);&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26008" LastActivityDate="2017-11-18T04:09:14.367" Title="An algorithm to turn descending order into ascending order, but why correct?" Tags="|sorting|" AnswerCount="0" CommentCount="0" ClosedDate="2017-11-21T14:48:15.340" ContentLicense="CC BY-SA 3.0" />
  <row Id="28273" PostTypeId="2" ParentId="28270" CreationDate="2017-11-18T10:04:29.260" Score="3" Body="&lt;p&gt;Mix formulation is conceptually more difficult, and one has to choose approximation spaces appropriately to make discrete system stable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrix emerging from mix formulation is very sparse but has size bigger compared to equivalent single filed formulation. However, very often mix and classical formation has the comparable number of non-zero entries. Properties of matrix emerging from mix formulation are not as good as fro classical formulation. Very often one need solver tailored to the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For elastic problems, locking is an argument for mix formulation, however, range of methods/remedies are available which removing locking with small cost, for example, efficient and straightforward B-bar method in plasticity,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So overall you have to have good argument why you like to apply mix elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me, mix-formulation has four attractive adventages;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) You have error evaluator build in the formulation of mix element. To implement classical element, you need half an hour, but to implement good error estimator you need a month. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Stresses or in general fluxes has a higher order of convergence.  Those are usually primary variable of interest for an engineer, evaluating the strength of the structure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) For nonlinear problems, you can decouple nonlinear (e.g. constitutive/physical equation) and linear (e.g. momentum conservation) part of the equation. That enables you to build efficient, stable nonlinear solvers, for example LaTIn method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) You can capture irregular solutions, abrupt changes, shock easier, for example in acoustics or unsaturated flows when material properties changes sharply.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-11-18T17:47:28.833" LastActivityDate="2017-11-18T17:47:28.833" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28277" PostTypeId="1" CreationDate="2017-11-19T14:05:13.687" Score="3" ViewCount="193" Body="&lt;p&gt;In standard Monte Carlo simulations, say for hard sphere systems, how should one compute the mean-squared displacement of the spheres in order to extract dynamical properties such as the diffusion coefficient of the spheres? &#xA;First problem that comes to mind is, how do we map our Monte Carlo steps to a real physical time, such that the resulting MSD and the extracted diffusion coefficient turn out to be physically meaningful and correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm just trying to learn how this can work out in standard Monte Carlo, though I've heard that to study such properties, Brownian dynamics or Kinetic Monte Carlo may be more suited, but that's another discussion. &lt;/p&gt;&#xA;" OwnerUserId="24573" LastActivityDate="2017-11-20T00:51:44.293" Title="Diffusion properties of hard spheres in Monte Carlo simulation" Tags="|simulation|monte-carlo|diffusion|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28278" PostTypeId="2" ParentId="28277" CreationDate="2017-11-20T00:51:44.293" Score="1" Body="&lt;p&gt;To my knowledge, conventional Monte Carlo simulations cannot tell you anything about the time-dependent properties of a system. The progression from state to state in, say, Metropolis Monte Carlo has nothing to do with the system's evolution in time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's contrast with equilibrium molecular dynamics simulation. In MD, you actually evolve particles in time according to their microscopic equations of motion. In the course of evolving one time step, you have to ask &lt;em&gt;dynamical&lt;/em&gt; questions like, &quot;What is each particles' current velocity?&quot; and &quot;What is the net force acting on each particle?&quot; In contrast, with each Monte Carlo step, you're singling out one particle, teleporting it and asking, &quot;Did this move bring the system closer to or further from equilibrium?&quot; Note my word choice here: &quot;teleporting&quot; in Monte Carlo versus &quot;evolving&quot; in MD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are two routes to the same goal: getting accurate estimates for equilibrium ensemble-averaged quantities like the pressure, pair distribution function, or free energy. In Monte Carlo, the averaging happens over several static configurations, each assumed statistically independent from one another. In MD, the averaging happens over a long time interval. If the system under study is ergodic, these two methods should give the same answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for time-dependent quantities, you have to care about how the system gets from point A to point B. In MD, the reason for making each move is unambiguous: we're just obeying Newton's second law (within some numerical accuracy). In Monte Carlo, you choose small moves based on convenience. No physical law prevents you from trying to totally reshuffle the particles every step; it's just really inefficient because there's a high probability of having to reject that move later on. Experience (and statistical mechanics) shows small moves are better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In each case, we end up with the situation that sequential steps are correlated with one another. In the MD case, those correlations are meaningful, and we can use them to compute things like diffusion coefficients and viscosities and so on. In the Monte Carlo case, the correlations are artificial; sequential steps are correlated because we designed them to be so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see this, let's say you try to extract the mean-square displacement from a Monte Carlo sequence. The answer you get will depend on what you set to be the maximum trial displacement!&lt;/p&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2017-11-20T00:51:44.293" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28279" PostTypeId="2" ParentId="26955" CreationDate="2017-11-20T06:47:10.083" Score="1" Body="&lt;p&gt;If you want to simulate the (stochastic) Brownian motion of an object with a diffusion coefficient that depends on the axis, this is discussed in detail in the paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Brownian Motion of a Particle of General Shape in Newtonian Fluid&#xA;Makino and Doi, J. Phys. Soc. Jpn. 73, pp. 2739-2745 (2004)&#xA;&lt;a href=&quot;https://doi.org/10.1143/JPSJ.73.2739&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1143/JPSJ.73.2739&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The critical issue is that rotational diffusion will alter what the axes are, leading to very different behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should also be possible to write and solve a Fokker-Planck equation for the probability distribution of the orientation variables and object position. This is (I think) discussed in Doi + Edwards's book The Theory of Polymer Dynamics.&lt;/p&gt;&#xA;" OwnerUserId="4494" LastActivityDate="2017-11-20T06:47:10.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28281" PostTypeId="1" AcceptedAnswerId="28304" CreationDate="2017-11-20T10:40:08.350" Score="-2" ViewCount="396" Body="&lt;p&gt;I'm running out of time for this code so any help would be greatly appreciated. I am currently coding the 2D heat/diffusion equation in matlab but i'm having trouble adding in the source term. my equation looks like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial u}{\partial t} = \alpha_x \frac{\partial^2 u}{\partial x^2} + \alpha_y \frac{\partial^2 u}{\partial y^2} + Au \left( 1 - \frac{u}{K} \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Neumann bc's with the flux=0 everywhere. u is the population number, A, the Intrinsic growth rate and K the carrying capacity. The code I'm using is a code I found here that I've edited to use the implicit/BTCS method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thank you so much.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;%%&#xA;%Specifying parameters&#xA;nx=40;                           %Number of steps in space(x)&#xA;ny=50;                           %Number of steps in space(y)       &#xA;nt=50;                           %Number of time steps &#xA;dt=0.01;                         %Width of each time step&#xA;dx=2/(nx-1);                     %Width of space step(x)&#xA;dy=2/(ny-1);                     %Width of space step(y)&#xA;x=0:dx:2;                        %Range of x(0,2) and specifying the grid points&#xA;y=0:dy:2;                        %Range of y(0,2) and specifying the grid points&#xA;u=zeros(nx,ny);                  %Preallocating u&#xA;un=zeros(nx,ny);                 %Preallocating un&#xA;visx = 2;                        %Diffusion coefficient/viscocity&#xA;visy = 2;                        %Diffusion coefficient/viscocity&#xA;% RepRateA = 0.3;                %Intrinsic Growth Rate of Population&#xA;% CarCapK = 1.7;                 %Carrying Capacity of Environment &#xA;&#xA;UnW=0;                           %x=0 Neumann B.C (du/dn=UnW)&#xA;UnE=0;                           %x=L Neumann B.C (du/dn=UnE)&#xA;UnS=0;                           %y=0 Neumann B.C (du/dn=UnS)&#xA;UnN=0;                           %y=L Neumann B.C (du/dn=UnN)&#xA;&#xA;%%&#xA;%Initial Conditions&#xA;for i=1:nx&#xA;    for j=1:ny&#xA;        if ((0.5&amp;lt;=y(j))&amp;amp;&amp;amp;(y(j)&amp;lt;=1.5)&amp;amp;&amp;amp;(0.5&amp;lt;=x(i))&amp;amp;&amp;amp;(x(i)&amp;lt;=1.5))&#xA;            u(i,j)=2;&#xA;        else&#xA;            u(i,j)=0;&#xA;        end&#xA;    end&#xA;end&#xA;&#xA;%%&#xA;%B.C vector&#xA;bc=zeros(nx-2,ny-2);&#xA;&#xA;bc(1,:)=-UnW/dx;       %Neumann B.Cs&#xA;bc(nx-2,:)=UnE/dx;     %Neumann B.Cs&#xA;bc(:,1)=-UnS/dy;       %Neumann B.Cs&#xA;bc(:,nx-2)=UnN/dy;     %Neumann B.Cs&#xA;&#xA;%B.Cs at the corners:&#xA;bc=visx*dt*bc+visy*dt*bc;&#xA;&#xA;%Calculating the coefficient matrix for the implicit scheme&#xA;Ex=sparse(2:nx-2,1:nx-3,1,nx-2,nx-2);&#xA;% Ax=Ex+Ex'-2*speye(nx-2);&#xA;Ax(1,1)=-1; &#xA;Ax(nx-2,nx-2)=-1;                 %Neumann B.Cs&#xA;Ey=sparse(2:ny-2,1:ny-3,1,ny-2,ny-2);&#xA;% Ay=Ey+Ey'-2*speye(ny-2);&#xA;Ay(1,1)=-1; &#xA;Ay(ny-2,ny-2)=-1;                 %Neumann B.Cs&#xA;A=kron(Ay/dy^2,speye(nx-2))+kron(speye(ny-2),Ax/dx^2);&#xA;D=speye((nx-2)*(ny-2))-visx*dt*A-visy*dt*A;&#xA;%%&#xA;%Calculating the field variable for each time step&#xA;i=2:nx-1;&#xA;j=2:ny-1;&#xA;&#xA;for it=0:nt&#xA;    un=u;&#xA;    h=surf(x,y,u','EdgeColor','none');       %plotting the field variable&#xA;    shading interp&#xA;    axis ([0 2 0 2 0 2])&#xA;    title({['2-D Diffusion with alphax = ',num2str(visx)];['2-D Diffusion with alphay = ',num2str(visy)];['time (\itt) = ',num2str(it*dt)]})&#xA;    xlabel('Spatial co-ordinate (x) \rightarrow')&#xA;    ylabel('{\leftarrow} Spatial co-ordinate (y)')&#xA;    zlabel('Transport property profile (u) \rightarrow')&#xA;    drawnow; &#xA;    refreshdata(h)&#xA;&#xA;    %Implicit method:&#xA;    U=un;&#xA;    U(1,:)=[];&#xA;    U(end,:)=[];&#xA;    U(:,1)=[];&#xA;    U(:,end)=[];&#xA;    U=reshape(U+bc,[],1);&#xA;    U=D\U;&#xA;    U=reshape(U,nx-2,ny-2);&#xA;    u(2:nx-1,2:ny-1)=U;&#xA;%Neumann:&#xA;    u(1,:)=RepRateA*u(2,:)*(1-u(2,:).'/CarCapK)-UnW*dx;&#xA;    u(nx,:)=RepRateA*u(nx-1,:)*(1-u(nx-1,:).'/CarCapK)+UnE*dx;&#xA;    u(:,1)=RepRateA*u(:,2).'*(1-u(:,2)/CarCapK)-UnS*dy;&#xA;    u(:,ny)=RepRateA*u(:,ny-1).'*(1-u(:,ny-1)/CarCapK)+UnN*dy;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26026" LastEditorUserId="26026" LastEditDate="2017-11-30T10:35:13.040" LastActivityDate="2017-11-30T10:35:13.040" Title="Adding Non-Linear source term to 2d Implicit MATLAB code" Tags="|matlab|finite-difference|diffusion|implicit-methods|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28282" PostTypeId="1" CreationDate="2017-11-20T10:44:17.187" Score="2" ViewCount="580" Body="&lt;p&gt;Which Python package is suited for solving problems of the following type?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the rigid body depicted in violet in the following sketch&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2uKra.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2uKra.jpg&quot; alt=&quot;Example: Three coordinate systems, force and moments&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to do the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Define cartesian coordinate systems $\text{cs0}(x_0, y_0, z_0)$, $\text{cs1}(x_1, y_1, z_1)$ and $\text{cs2}(x_2, y_2, z_2)$.&lt;/li&gt;&#xA;&lt;li&gt;Specify forces and moments acting on the center of coordinate systems.&lt;/li&gt;&#xA;&lt;li&gt;Calculate forces and moments acting on fixed clamp at origin of $\text{cs0}$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In order to preserve flexibility and intuitive usage, object orientated syntax would be preferred.&lt;br&gt;&#xA;Pseudocode:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;cs0 = initCartesianCoordinatesystem()&#xA;cs1 = cartesianCoordinatesystem( cs0, tranlationX = 0, translationY = 20,\&#xA;    translationZ = 0, rotationX = 0, rotationY = 0, rotationZ = 0)&#xA;cs2 = cartesianCoordinatesystem( cs1, translationX = 10, translationY = 0,\&#xA;    translationZ = 0, rotationX = 0, rotationY = 135, rotationZ = 0)&#xA;&#xA;body = rigidBody()&#xA;body.addForce( cs1, (1, 0, 0) )&#xA;body.addMoment( cs1, (0, 0, 1) )&#xA;body.addMoment( cs2, (0, 2, 0) )&#xA;force, moment = body.staticEquilibrium( cs0 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Leading to &#xA;$$ force = \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}_{\text{cs0}} $$&#xA;$$ moment = \begin{pmatrix} 0 \\ 2+20 \\ 1 \end{pmatrix}_{\text{cs0}} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following features would be helpful:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Visualization&lt;/li&gt;&#xA;&lt;li&gt;Units&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26027" LastEditorUserId="9667" LastEditDate="2017-11-20T19:30:42.340" LastActivityDate="2017-11-20T19:30:42.340" Title="Python package to calculate static force and moments of rigid body" Tags="|python|software-recommendation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28283" PostTypeId="1" CreationDate="2017-11-20T13:55:08.353" Score="5" ViewCount="116" Body="&lt;p&gt;It seems to me that a useful concept to define when studying Krylov subspace methods is the idea of a vector &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; that belongs to a Krylov subspace &lt;span class=&quot;math-container&quot;&gt;$\mathcal{K}_{n+1}(A,b)$&lt;/span&gt; but not to the preceding one &lt;span class=&quot;math-container&quot;&gt;$\mathcal{K}_{n}(A,b)$&lt;/span&gt;, i.e., &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; can be written as a polynomial &lt;span class=&quot;math-container&quot;&gt;$v = p(A)b$&lt;/span&gt; of degree &lt;em&gt;exactly&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, with a nonzero leading coefficient.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there a standard name or notation for vectors that belong to &lt;span class=&quot;math-container&quot;&gt;$\mathcal{K}_{n+1}(A,b) \setminus \mathcal{K}_{n}(A,b)$&lt;/span&gt;?&lt;/strong&gt; Or, how can I describe or denote them? Something like &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is a proper vector in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{K}_{n+1}(A,b)$&lt;/span&gt;&amp;quot; sounds good, but I don't want to invent a new term when there already exists a word for it.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Continuation vector&amp;quot; is something close, but it is too vague and specific to Arnoldi, when the concept appears in many other contexts.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2023-12-12T08:19:57.030" LastActivityDate="2023-12-12T08:19:57.030" Title="Name for vectors in a Krylov space but not the preceding one" Tags="|linear-algebra|krylov-method|terminology|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="28284" PostTypeId="2" ParentId="28271" CreationDate="2017-11-20T14:07:52.387" Score="3" Body="&lt;p&gt;I can't speak much for FV, never having worked much with the technique. But many FD methods can be rewritten as an FE method that judiciously chooses basis sets and quadrature rules in such a way that the implicit &amp;quot;LHS&amp;quot; matrix is simplified (typically to diagonal form). Some of these techniques have been around a long time under the name &amp;quot;mass lumping&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I wrote about this relationship a bit in my thesis, there's a link below (see especially section 2.4 starting on page 11). It's all couched in terms of solving the Maxwell system using FDTD. In 1D this reduces to just scalar Helmholtz. You can follow the same process for Laplace. For other PDE's, you might have to ask an expert what the appropriate basis sets are.&lt;/p&gt;&#xA;&lt;p&gt;Title: H-, P- and T-Refinement Strategies for the Finite-Difference-Time-Domain (FDTD) Method Developed via Finite-Element (FE) Principles&lt;/p&gt;&#xA;&lt;p&gt;Linky: &lt;a href=&quot;https://etd.ohiolink.edu/acprod/odb_etd/ws/send_file/send?accession=osu1219064270&amp;amp;disposition=inline&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://etd.ohiolink.edu/acprod/odb_etd/ws/send_file/send?accession=osu1219064270&amp;amp;disposition=inline&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2024-01-12T15:51:29.920" LastActivityDate="2024-01-12T15:51:29.920" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="28288" PostTypeId="1" CreationDate="2017-11-20T18:36:35.470" Score="1" ViewCount="69" Body="&lt;p&gt;I wish to solve an eigenvalue problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^{2}f=Ef $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I assume spherical symmetry $f(r,\theta,\phi)=f(r)$, I can reduce the problem to 1D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\frac{2}{r}\frac{d}{dr}+\frac{d^{2}}{dr^{2}})f=Ef$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My boundary conditions are $f(r=0)=0$ and $f(r=L)=0$, where $L$ is the size of the discrete grid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am attempting to solve this problem using a orthogonal basis set expansion, but the issue I am experiencing is the divergence of $r$ at $r=0$. Is there a better method for solving this type of problem? How can I accommodate the singularity?&lt;/p&gt;&#xA;" OwnerUserId="21221" LastEditorUserId="9667" LastEditDate="2017-11-20T21:15:56.963" LastActivityDate="2017-11-20T21:15:56.963" Title="Eigenvalue ODE in Spherical Coordinates--Numerical" Tags="|eigensystem|differential-equations|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28289" PostTypeId="1" CreationDate="2017-11-20T19:49:10.223" Score="2" ViewCount="281" Body="&lt;p&gt;I'm trying to solve a problem of finding the displacements of an elastic material subjected to external forces.  Those external forces are themselves a nonlinear function of the material displacements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step is to define a functional for the energy in terms of the displacements.  The solution is then the displacement which minimizes that functional over sufficiently smooth functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this functional, Euler-Lagrange equations can be derived, which must be satisfied by the solution.  Those equations are a system of elliptic PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this problem in practice, it is necessary to discretize.  This can be done directly for the functional, so that the integral of the Lagrangian over the spatial domain becomes a sum over a spatial grid, with the appropriate translation of spatial derivatives to finite differences.  The problem is to minimize the sum, which is generally non-convex.  The alternative is to discretize the Euler-Lagrange equations, which results in a large nonlinear system of equations.  The functional contains terms which are fourth order polynomials of the displacements, while the Euler-Lagrange equations have terms which are third order polynomials of the displacements.  There are no discontinuities in the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about the relative advantages and disadvantages of the two approaches.  I have been trying quite hard to solve the second problem, i.e. solve the large nonlinear system, and I just can't seem to make progress.  I have an analytic expression for the Jacobian, and I can ensure that I have a good initial guess in the sense that it is &quot;close&quot; to the solution as measured by the norm of the difference, but I have no guarantee that, for example, Newton's method will converge from this initial guess.  I've attempted some simple globalization strategies, such as a parametrized continuation, pseudo-transient continuation, and some ad hoc regularization.  The problem with Newton's method without a continuation is that it seems to always find a point at which the Jacobian becomes singular.  The parametrized continuation seems to avoid this, but the Newton iterations stagnate.  Quasi-Newton methods like L-BFGS work very well up to a point, and then fail catastrophically, with massive increase in the residual norm in a single iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not attempted recasting the problem as a minimization for two reasons - first, I don't understand what advantages that would have over solving the nonlinear PDEs, and second, I would have to do a lot of calculations arising from discretizing at an early step.  I realize this question may be a bit vague but I'd really appreciate any guidance - thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I'll try to include some more information.  One advantage I can see of discretizing the energy functional and approaching the problem as a minimization is that the Hessian would be symmetric.  This contrasts with the discretization of the Euler-Lagrange equations, for which the Jacobian is not symmetric.  In both approaches the Hessian/Jacobian is not positive definite.  It should be possible to calculate a value which can be added to the diagonal to make a positive definite matrix, as discussed in Nocedal and Wright, but I'm unsure how to actually implement that.  I suppose the idea is to put in a positive definite approximation to the Jacobian without altering the function itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: Another thought about why I seem to be having so much trouble is that because my external data comes on a Cartesian grid, I am using finite difference instead of finite element.  I realize that the vast majority of work in elasticity uses finite element, but I have yet to see an explicit discussion of the advantages in terms of convergence.  I use centered difference operators on a 27 point stencil.  The boundary conditions are homogeneous Dirichlet.&lt;/p&gt;&#xA;" OwnerUserId="25235" LastEditorUserId="25235" LastEditDate="2017-11-22T16:19:40.943" LastActivityDate="2017-11-22T16:19:40.943" Title="Pros and cons of optimization vs. variational calculus, re: nonlinear elasticity" Tags="|nonlinear-equations|" AnswerCount="0" CommentCount="0" ClosedDate="2017-11-29T13:34:53.560" ContentLicense="CC BY-SA 3.0" />
  <row Id="28290" PostTypeId="1" CreationDate="2017-11-20T21:45:48.730" Score="2" ViewCount="880" Body="&lt;p&gt;I want to solve the diffusion equation using the method of lines with Neumann boundary conditions&#xA;$$&#xA;\frac{\partial p}{\partial t}=\frac{\partial^2p}{\partial x^2}\\&#xA;\frac{\partial p}{\partial x}(x=0)=\frac{\partial p}{\partial x}(x=1)=0\\&#xA;p(x,0)=\begin{cases}&#xA;       1/dx \quad x=0.5,\\&#xA;        0 \quad  elsewhere&#xA;\end{cases}&#xA;$$&#xA;For sake of simplicity I use the forward Euler so I choose $dx=0.1, \; dt=0.001$ to ensure the stability.&#xA;Using a second order scheme I solve the boundary conditions imposing&#xA;$$&#xA;\frac{p_1-p_{-1}}{2 dx}=0,\\&#xA;\frac{p_{N+1}-p_{N-1}}{2 dx}=0.\\&#xA;$$&#xA;So the tridiagonal matrix looks like &#xA;$$&#xA;\frac{1}{dx^2}&#xA;\begin{pmatrix}&#xA;-2&amp;amp;2&amp;amp;0&amp;amp;0\\&#xA;1&amp;amp;-2&amp;amp;1&amp;amp;0\\&#xA;0&amp;amp;1&amp;amp;-2&amp;amp;1\\&#xA;0&amp;amp;0&amp;amp;2&amp;amp;-2&#xA;\end{pmatrix}.&#xA;$$&#xA;At this point the solution of my equation is given by&#xA;$$&#xA;\vec{p}^{k+1}=dt(T\vec{p}^k)+\vec{p}^k.&#xA;$$&#xA;This boundary conditions impose the conservation of probability but it seems that is not the case since for different $N$ (number of grid points on the $x$ axis) I obtain different probabilities. For example, for $N=10,20,100,1000$ I obtained that the total probability is $1.11111,1.05263,1.00012,1.0000$.&#xA;It seems that I need a really fine grid to obtain the correct physical result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did I make a conceptual mistake or I just made an error in my code?&#xA;Thank for the answers&lt;/p&gt;&#xA;" OwnerUserId="26029" LastActivityDate="2018-12-18T05:15:58.387" Title="Neumann boundary conditions diffusion equations methods of lines" Tags="|boundary-conditions|diffusion|method-of-lines|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28293" PostTypeId="2" ParentId="28290" CreationDate="2017-11-21T03:01:15.947" Score="0" Body="&lt;p&gt;Consider a uniform grid&#xA;$$&#xA;x_i = ih, \quad i=0,1,\ldots,N&#xA;$$&#xA;Your equations are&#xA;$$&#xA;dp_0/dt = 2(p_1 - p_0)/h^2&#xA;$$&#xA;$$&#xA;dp_i/dt = (p_{i-1} - 2 p_i + p_{i+1})/h^2, \quad 1 \le i \le N-1&#xA;$$&#xA;$$&#xA;dp_N/dt = 2(p_{N-1} - p_N)/h^2&#xA;$$&#xA;This can also be written as&#xA;$$&#xA;dp/dt = Tp, \qquad p = [p_0, p_1, \ldots, p_N]^\top&#xA;$$&#xA;where $T$ is tridiagonal matrix. Compute integral of $p$ by Trapezoidal rule&#xA;$$&#xA;I(t) = \left(p_0/2 + \sum_{i=1}^{N-1} p_i + p_N/2\right) h&#xA;$$&#xA;Using the scheme&#xA;$$&#xA;dI/dt = \frac{1}{h} \left(p_1 - p_0 + \sum_{i=1}^{N-1} (p_{i-1} - 2 p_i + p_{i+1}) + p_{N-1} - p_N \right)&#xA;$$&#xA;Rearrange the sum to show that&#xA;$$&#xA;dI/dt = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not clear from the question how the total probability was calculated. As shown above, using Trapezoidal rule, the numerical scheme will conserve the total probability.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2017-11-22T02:32:13.637" LastActivityDate="2017-11-22T02:32:13.637" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28296" PostTypeId="1" CreationDate="2017-11-21T15:41:42.033" Score="1" ViewCount="4473" Body="&lt;p&gt;I've just finished learning the physics behind the problem and would like to write a program in C++ than can solve the problem. I'm actually stuck at the start. I've quite a bit of research, the problem is there is not too many examples of code used to solve the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to solve the problem using finite-difference form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dfrac{d^2\psi}{dx^2}\approx \dfrac{\psi_{n+1}+\psi_{n-1}-2\psi_{n}}{(\Delta x)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which allows us to rearrange in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{n+1}=2\psi_{n} - \psi_{n-1}-2(\Delta x)^2(E-V_n)\psi_n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the even-parity solution, we have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(0)=1 \quad \quad \psi'(0)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;at $n=0 \implies x=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Letting $m=1$ and $\hbar = 1$ I know that at the ground state $E_g =\frac{\pi^2}{8}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm struggling to write code that can help me find $\psi_n$ for values of $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I assume that $V_0 = 0$ since we want the wave function to be inside the well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not asking for someone to write the code for me, I'd just like tips on what I need to define and which method I should use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: This is what I have so far&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;unistd.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;stdarg.h&amp;gt;&#xA;#include &amp;lt;assert.h&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;const double PI = 3.14159265358979323846264338327950;&#xA;int main() {&#xA;&#xA;    cout &amp;lt; &quot;1D Particle-in-a-Box&quot;\n;&#xA;&#xA;    double psi0, dpsi0, N, dx, x_end, E;&#xA;    int number_steps, nr;&#xA;&#xA;    double value_x [number_steps];&#xA;    double psi [number_steps];&#xA;    double V [number_steps];&#xA;&#xA;    //read parameters (even-parity)&#xA;    cout &amp;lt; &quot;psi0 = &quot;;&#xA;    cin &amp;gt; psi0;&#xA;    cout &amp;lt; &quot;dpsi0 = &quot;;&#xA;    cin &amp;gt; dpsi0; &#xA;&#xA;    dx = x_end / number_steps;&#xA;&#xA;    value_x [0] = 0;&#xA;    psi [0] = psi0;&#xA;    V [0] = 0.0&#xA;    //finite-difference form&#xA;&#xA;    while (nr &amp;lt; number_steps)&#xA;    {&#xA;        value_x [nr+1] = dx*(nr+1); &#xA;        psi[nr+1] = 2*psi[nr] - psi[nr-1] - (2*(dt*dt))*(E-V[nr])*(psi[nr]);&#xA;        nr = nr + 1;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25885" LastEditorUserId="25885" LastEditDate="2017-11-21T16:02:03.033" LastActivityDate="2019-04-02T11:44:38.840" Title="Solving the 1D Particle-in-a-Box using C++" Tags="|c++|computational-physics|quantum-mechanics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28298" PostTypeId="1" AcceptedAnswerId="28299" CreationDate="2017-11-21T18:06:18.260" Score="7" ViewCount="242" Body="&lt;p&gt;I'm trying to simulate the growth of a brain tumor using a 3d reaction-diffusion model $ \partial_{t}u = \nabla.{(D\nabla{u}) } + ku.(1-u)$ , knowing the initial distribution of tumor $u^0$, the non-homogeneous diffusion coefficient $D$ and the non-homogeneous net-proliferation map $k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I first tried to proceed to simulation simply by dividing it into small time steps and directly simulate the process by simply replacing the derivatives what is appearing to me now as forward Euler finite difference. I got serious instability problems which, if I understand well, are due to the difference scheme I'm using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm trying to learn how to correctly code finite difference algorithms.&#xA;At this point there's something I don't understand and I hope someone will be able to explain me where I'm wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let forget the proliferation term and do simply a 1D diffusion ( $ \partial_{t}u = \partial_{x}(D\partial_{x}u) $ ) using a Crank-Nicolson scheme for the time difference and a central difference for the spatial derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if $D$ depends on $x$, we have&#xA;$ u^{n+1}_{i} = u^{n}_{i} + \frac{\Delta{t}}{2} \{ \partial_{x}(D\partial_{x}u^{n})  + \partial_{x}(D\partial_{x}u^{n+1}) \} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which develop to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$  \Rightarrow ( 1 - \frac{\Delta t}{2}\partial_{x}(D\partial_{x}\cdot) ) u^{n+1}_{i} = ( 1 + \frac{\Delta t}{2}\partial_{x}(D\partial_{x}\cdot) ) u^{n}_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and using $ \partial_{x}(D\partial_{x}V) = \partial_{x}D.\partial_{x}V + D.\partial_{xx}V $,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_{x}V = \frac{V_{i+1}-V_{i-1}}{2\Delta x}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\partial_{xx}V = \frac{V_{i+1}+V_{i-1}-2V_{i}}{{\Delta x}^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \partial_{x}(D\partial_{x}u_{i}) = \frac{(D_{i+1}-D_{i-1}).(u_{i+1}-u_{i-1})}{4{\Delta x}^2}  + \frac{D_{i}.(u_{i+1}+u_{i-1}-2u_{i})}{{\Delta x}^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ordering it according to the spatial indice of $u$ , one get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \partial_{x}(D\partial_{x}u_{i}) =  \frac{1}{4{\Delta x}^2}.\{ (D_{i+1}-D_{i-1}+4D_{i})u_{i+1} + 8D_{i}u_{i} - (D_{i+1}-D_{i-1}-4D_{i})u_{i-1} \} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In matrix form, this can be written&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \partial_{x}(D\partial_{x}u) = \tilde{D}u  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $ \tilde{D} \equiv \frac{1}{4{\Delta x}^2}\begin{bmatrix} -8D_1 &amp;amp; (D_2+4D_1) &amp;amp;  0 &amp;amp; \cdots  \\&#xA;                         -(D_3 - D_1 -4D_2) &amp;amp; -8D_2 &amp;amp; (D_3-D_1+4D_2) &amp;amp; \cdots \\&#xA;                         \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots \\&#xA;                         0 &amp;amp; \cdots &amp;amp; -(- D_N -4D_{N-1}) &amp;amp; -8D_N&#xA;                  \end{bmatrix} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, one have to solve, at each time step, the equation&#xA;$ (1 - \frac{\Delta t}{2}\tilde{D})u^{n+1} = (1 + \frac{\Delta t}{2}\tilde{D})u^{n} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my problem is that on every course/paper I read on this subject, the authors are explaining that one have to solve the equation at each time step which is the time consuming step.&#xA;However, as it is written here at least, the matrix $\tilde{D}$ is completely determined from the (fixed in time) $D$ distribution and it looks to me that one can just simply generate it, then calculate the total matrix &#xA;$ M = (1 - \frac{\Delta t}{2}\tilde{D})^{-1} \cdot (1 + \frac{\Delta t}{2}\tilde{D})$ and then simply calculate $u^n = M^nu^0 $ which is very fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to code it in MATLAB, and indeed, doing it step by step, or just at once gives me the same results ( and which are coherent with a simple Gaussian blurring with $\sigma = \sqrt{2D.T} $ in the case of an homogeneous D, just to check the validity of the simulation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i must have missed something but what ? Maybe finite difference is not the tool to use in case of diffusion-only and are actually used because of the reaction term, but still, if I understand well, one can separate the simulation into smaller sub-time steps to perform only diffusion or only proliferation (solved analytically for the later) with Douglas-Gunn or equivalent approaches.&#xA;In such a case, the diffusion step can still be solved using a simple matrix multiplication at each step which do not require a long processing ? So I would be very happy if someone can tell me if I'm going on the right track or if (as I suspect) I missed any fundamental issue...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;D. Guez&lt;/p&gt;&#xA;" OwnerUserId="25963" LastEditorUserId="25963" LastEditDate="2018-05-16T09:53:06.647" LastActivityDate="2018-05-16T09:53:06.647" Title="finite difference : why should we solve linear equation at each step" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="28299" PostTypeId="2" ParentId="28298" CreationDate="2017-11-22T00:19:05.567" Score="4" Body="&lt;p&gt;You're absolutely right that the matrix doesn't change during each time step, so there's no need to invert it more than once. In practice, however, you probably shouldn't be inverting the matrix at all!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that the coefficient matrix is constant is exactly why matrix factorization (e.g. LU) can provide significant speedup. Factorize once at the beginning of the simulation, and you just have to solve triagonal matrix problems, which is $O(N^2)$ at each time step, rather than the $O(N^3)$ for a direct solution (though computing the the factorization is $O(N^3)$, you only do it once). &lt;a href=&quot;https://math.stackexchange.com/a/266360/374791&quot;&gt;This answer&lt;/a&gt; over at Math explains it better the I do, I think. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also worth noting that tridiagonal systems like your simplified example are especially easy, you can solve them in $O(N)$ using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Thomas Algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really only care about the result at time step $n$, and don't care at all about the intervening values, your $u^n = M^nu^0 $ approach is valid, though I'm not sure how much that will help. In fact, it will probably slow you down since matrix multiplication is between $O(N^{2.3})$ and $O(N^3)$ (i.e. not &quot;very fast&quot;, and &lt;a href=&quot;https://stackoverflow.com/a/17717767/4555567&quot;&gt;this&lt;/a&gt; suggests Matlab uses an $O(N^3)$ implementation), and the initial matrix inverse itself is costly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's nothing wrong with finite difference for diffusion only problems, though gotta be careful with Euler forward to keep your $\Delta t$ small enough for stability issues ($\Delta t &amp;lt; \frac{ \Delta x^2}{2D}$, iirc). Even Crank-Nicholson can be oscillatory (though stable) if too long a step is taken. Also, since your proliferation term is non-linear, if you do an implicit solution, you're going to have to solve a non-linear system of equation at each time step. &lt;/p&gt;&#xA;" OwnerUserId="20509" LastActivityDate="2017-11-22T00:19:05.567" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28300" PostTypeId="1" CreationDate="2017-11-22T09:11:10.560" Score="1" ViewCount="100" Body="&lt;p&gt;In Machine Learning field, is there any work done, or research, or info in general &lt;strong&gt;about finding optimal permutations&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For eg. to find a number of optimal arrangements of sliced shapes on 2D sheet of material - so that there is low number of leftovers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume it can be done by brute-forcing permutations, but what about neural networks or Machine Learning in general? &lt;/p&gt;&#xA;" OwnerUserId="24909" LastEditorUserId="24909" LastEditDate="2017-11-22T09:24:28.557" LastActivityDate="2017-11-22T09:24:28.557" Title="Is there any ML solutions for permutations?" Tags="|machine-learning|neural-network|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28302" PostTypeId="1" CreationDate="2017-11-22T11:45:54.187" Score="12" ViewCount="762" Body="&lt;p&gt;I have a linear &lt;a href=&quot;https://en.wikipedia.org/wiki/Ordinary_differential_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;ordinary differential equation&lt;/a&gt; (ODE) with a system matrix with constant coefficients: $$\dot{y}(t) = \mathcal{A}\; y(t), \quad y(0) = y_0$$ with $y(t) \in \mathbb{R}^{n \times 1}$ and $\mathcal{A} \in \mathbb{R}^{n \times n}$ where $\mathcal{A}$ has constant coefficients. I have a (large) series of $r$ (with $r \gg n^2$) datapoints $y_{i}$ for time points $t_i$ (including $y_0$ for $t=t_0$). The elements of the matrix $\mathcal{A}$ are unknown and I'm interested in estimating these. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, this is considered as an inverse problem. Is there a more specific terminology for this example (constant coefficients, linear ODE, system matrix identifcation)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the current techniques to solve this type of problem? How sensitive is the problem to noisy data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My &quot;brute force&quot; approach would be&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write the elements of matrix $\mathcal{A}$ in a vector $a = [a_{11}, a_{12}, \ldots, a_{1n}, a_{21}, \ldots, a_{2n}, \ldots, a_{nn}]$&lt;/li&gt;&#xA;&lt;li&gt;Write down the equations for each $t_i$ time point using some approximation for $\dot{y}(t) \approx \frac{y_{i} - y_{i-1}}{\Delta t}$&lt;/li&gt;&#xA;&lt;li&gt;Build a linear least squares problem using this systems matrix and the vector of unknowns. &lt;/li&gt;&#xA;&lt;li&gt;I can perform an SVD decomposition $\mathcal{A} = U\Sigma V^{t}$&lt;/li&gt;&#xA;&lt;li&gt;Since the problem is homogeneous, the LS optimum would be the last column of $V$. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But the problem is of course, that if the number of data points gets larger, the computation of the full SVD becomes memory and time consuming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the simplicity of the problem formulation (leading to a very structured matrix $\mathcal{A}$), I would be surprised if there is no specific, more optimal, method developed for this type of problem. Would an approach with the formal solution using the matrix exponential $y(t) = e^{\mathcal{A}t}y_{0}$ lead to a more optimal algorithm? &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="9667" LastEditDate="2018-01-02T14:10:47.320" LastActivityDate="2020-08-26T18:14:06.303" Title="Inverse problem in linear ODE" Tags="|linear-algebra|ode|inverse-problem|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28303" PostTypeId="1" CreationDate="2017-11-22T12:55:25.280" Score="1" ViewCount="120" Body="&lt;p&gt;I'm trying to find the largest eigenvalues of very large $N \times N$ matrices  ($N = 10^{10}$ and larger). The matrices are not sparse but the multiplication operation is fast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, I'm using methods without explicit matrix storage, from the simple power method to Krylov-Schur (using petsc/slepc in C). But all these methods require to store at least a vector of size $N$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a diagonalisation algorithm which does not need as much memory?&lt;/p&gt;&#xA;" OwnerUserId="26047" LastEditorUserId="9667" LastEditDate="2017-11-22T16:16:16.843" LastActivityDate="2017-11-22T16:16:16.843" Title="Low memory algorithm for matrix diagonalisation" Tags="|linear-algebra|algorithms|eigenvalues|matrix|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28304" PostTypeId="2" ParentId="28281" CreationDate="2017-11-22T16:24:28.347" Score="1" Body="&lt;p&gt;You can solve your problem in two ways: &lt;em&gt;explicitly&lt;/em&gt; or &lt;em&gt;implicitly&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to solve your system &lt;em&gt;explicitly&lt;/em&gt;, then the solution is quite simple: the only term you would be solving for is the $u_{i+1}$ term in the time derivative. All the other instances of $u$ are known quantities ($u_{i}$), which means they can be placed on the RHS - because they are knowns! To ensure numerical stability, the size of the time step should be small enough that the difference between $u_i$ and $u_{i+1}$ is small. The definition of &lt;em&gt;small&lt;/em&gt; is not trivial in some cases. Most textbooks on PDEs will give you more information on what stable means w.r.t explicit solvers of PDEs. If you really want to get fancy you could set up your PDE into a system of ODEs in time and use one of the Matlab supplied ODE solvers (e.g. &lt;em&gt;ode45&lt;/em&gt;, &lt;em&gt;ode15&lt;/em&gt;, etc.). This has the added benefit of using &lt;em&gt;adaptive timesteps&lt;/em&gt; because those Matlab routines are able to calculate the size of a timestep based on error measurements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to solve your equation &lt;em&gt;implicitly&lt;/em&gt;, then you move into the world of &lt;em&gt;non-linear&lt;/em&gt; PDEs, which require a non-linear solver within every time step. This is more expensive than explicit solvers &lt;em&gt;when using the same sized timestep&lt;/em&gt;; however, implicit solvers are commonly more stable than explicit ones and can sometimes warrant the use of larger timesteps to speed up the solver. I noticed you wrote &lt;em&gt;implicit&lt;/em&gt; in your code above: if that's what you decide to do then you would need to write a non-linear solver for your problem.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A rather simple non-linear implicit solver to implement is called the &lt;em&gt;Newton-Raphson method&lt;/em&gt;, and may be useful for your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't want to implement a non-linear implicit solver, you may instead decide to implement a hybrid approach where you treat all &lt;em&gt;linear&lt;/em&gt; terms &lt;em&gt;implicitly&lt;/em&gt; and &lt;em&gt;non-linear&lt;/em&gt; terms &lt;em&gt;explicitly&lt;/em&gt;. What is best for solving your problem can only be determined by looking more closely at literature on the subject or experimentation. Since you're already working with the BTCS method, I would recommend extending it by including your source term explicitly into the RHS. This would result in a 'lag' of your source term with respect to time, but may be reasonable for your problem.&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="19791" LastEditDate="2017-11-22T18:49:02.173" LastActivityDate="2017-11-22T18:49:02.173" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="28306" PostTypeId="1" AcceptedAnswerId="28308" CreationDate="2017-11-22T19:00:19.193" Score="1" ViewCount="58" Body="&lt;p&gt;How can I write an algorithm to calculate $R_n$s for any desired $n$ from the following equation with given $f,R_0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the output of the code would be $R_1,R_2,...$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R_n^2-2fR_n=R_0^2-2fR_0+\sum_{i=0}^n\sum_{j=0}^nR_iR_j$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have derived this equation for my specific problem myself and I can calculate every $R_n$ manually but since $R_n$ is not separable I don't know how to write a code for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="23599" LastEditorUserId="23599" LastEditDate="2017-11-22T19:30:01.763" LastActivityDate="2017-11-23T09:23:18.680" Title="An algorithm to calculate $R_n$s from this recursive equation" Tags="|algorithms|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28308" PostTypeId="2" ParentId="28306" CreationDate="2017-11-23T09:23:18.680" Score="0" Body="&lt;p&gt;So I did a little math and I noticed $R_n^2$ cancels from both sides as @Kirill noticed in the comments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I post this for clarification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$R_n^2-2fR_n=R_0^2-2fR_0+\sum_{i=0}^n\sum_{j=0}^nR_iR_j$$&#xA;$$R_n^2-2fR_n=R_0^2-2fR_0+\sum_{i=0}^{n-1}\sum_{j=0}^{n-1}R_iR_j+R_n\sum_{j=0}^nR_j+R_n\sum_{i=0}^{n-1}R_i$$&#xA;$$R_n^2-2fR_n=R_0^2-2fR_0+\sum_{i=0}^{n-1}\sum_{j=0}^{n-1}R_iR_j+R_n\sum_{j=0}^{n-1}R_j+R_n^2+R_n\sum_{i=0}^{n-1}R_i$$&#xA;$$R_n=\frac{R_0^2-2fR_0+\sum_{i=0}^{n-1}\sum_{j=0}^{n-1}R_iR_j}{-2(f+\sum_{i=0}^{n-1}R_i)}$$&lt;/p&gt;&#xA;" OwnerUserId="23599" LastActivityDate="2017-11-23T09:23:18.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28309" PostTypeId="1" AcceptedAnswerId="28310" CreationDate="2017-11-23T15:16:33.957" Score="0" ViewCount="3531" Body="&lt;p&gt;I am using Chebychev collocation nodes for approximation, and my problem requires me to calculate derivatives of the polynomial. I have been reading from a few sources, but I am not sure I understand how to do it. Could anyone please help me with how I go about this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anup&lt;/p&gt;&#xA;" OwnerUserId="26057" LastActivityDate="2017-11-24T11:26:00.883" Title="Derivatives of a Chebychev polynomial" Tags="|matlab|approximation|polynomials|automatic-differentiation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28310" PostTypeId="2" ParentId="28309" CreationDate="2017-11-23T19:14:19.927" Score="0" Body="&lt;p&gt;A specific polynomial and any number $k$ of its derivatives can be evaluated using Horner's methods. The case of $k = 2$ is discussed &lt;a href=&quot;https://math.stackexchange.com/a/2457970/307944&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the event that you need all Chebyshev polynomials of degree less than $n$ and their first derivatives, I suggest that you proceed directly from the defining recurrence relation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of Chebyshev polynomials of the first kind, we have&#xA;$$T_0(x) = 1, \quad T_1(x) = x, \quad T_{k+1}(x) = 2xT_{k}(x) - T_{k-1}(x).$$&#xA;Let $Q_k$ denote the derivative of $T_k$ with respect to $x$. Then by the product rule of differentiation&#xA;$$Q_0(x) = 0, \quad Q_1(x) = 1, \quad Q_{k+1}(x) = 2 T_k(x) + 2x Q_k(x) - Q_{k-1}(x)$$&#xA;It follows, that any code which computes $T_k$ for $k=0,1,2,\dotsc,n$ can be easily augmented to also produce the derivatives $Q_k$ for $k=0,1,2,\dotsc,n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If running error bounds are desired, then the principles applied &lt;a href=&quot;https://scicomp.stackexchange.com/a/28197/19136&quot;&gt;here&lt;/a&gt; can be extended to cover you situation as well.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2017-11-24T11:26:00.883" LastActivityDate="2017-11-24T11:26:00.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28311" PostTypeId="1" AcceptedAnswerId="28321" CreationDate="2017-11-23T23:16:17.033" Score="2" ViewCount="92" Body="&lt;p&gt;I am trying to solve for the matrix $C$ in $Q = YCZ$ in matlab. I have preliminary results but they don't seem realistic. Here, $Q$ is $n \times m-1$, $Y$ is $n \times p$, $C$ is $p \times m$ and $Z$ is $m \times m-1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of $n$ is very large, say $n = 10000$. On the other hand, $m$ and $p$ are of the same order of magnitude. In the case I considered, $m=201, p =175$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among all matrices above, $Z$ is the only one with a seemingly unique structure. By eliminating the first row of $Z$, it becomes upper triangular and invertible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In matlab, I tried to solve for $C$ using: $C = Y\backslash (Q/Z)$. However, I ran into the following issues:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) The last column of $C$ is all zero which is unrealistic in my problem. It turns out that the last column of $Q/Z$ is also a zero vector which is probably the cause of this. Are there suggestions how to compute $Q/Z$ without having the last column being 0?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If I want to find $C$ such that every row of $C$ has minimum norm, how can I modify my problem in matlab? Based on the motivation below, every row of $C$ corresponds to values of a function on the unit interval. It is highly preferable for these functions to not be highly oscillatory in contrast to what I'm seeing now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any suggestions on how to proceed.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The motivation of the problem is as follows: let $Q(x)$ be a stochastic process defined as $Q(x) = \int_0^x P(s)\,ds$ where $P(s)$ is another stochastic process. Suppose that $P(s) = q_1(s)Y_1 + \dots + q_n(s) Y_n$ where $Y_1,\dots,Y_n$ are random variables and $q_i(s)$ are deterministic functions of $s$. It will be assumed that the domain of $P(s)$ and $Q(x)$ is the interval $[0,1]$. Suppose that I have observations of $Q(x)$ at discrete time points and that this information is stored in $Q$. Each row of this matrix is an observation while every column corresponds to a point in the spatial discretization. In addition, suppose that I have observations of $Y_1,\dots,Y_n$ stored in a matrix $Y$ where every row is an observation and every column corresponds to $Y_i$. My interest is to estimate the deterministic functions $q_1(s),\dots,q_n(s)$. To do this, I use a Finite element basis, i.e. &lt;a href=&quot;http://hplgit.github.io/num-methods-for-PDEs/doc/pub/approx/sphinx/._main_approx004.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;, such that $q_i(s) = \sum_{k=1}^m c_i^k \phi_k (s)$ where $\phi_k$ are the basis functions and $c_i^k$ are the nodal values. The matrix $C$ is then obtained such that $C_{ik} = c_i^k$ using notation above and is the quantity of interest. The definition of $Z$ follows from the relationship $Q(x) = \int_0^x P(s) \,ds$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="26058" LastActivityDate="2017-11-26T19:13:31.753" Title="Solving for $C$ in $Q = YCZ$ using least squares in Matlab" Tags="|linear-algebra|optimization|matlab|linear-solver|least-squares|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28312" PostTypeId="1" AcceptedAnswerId="28315" CreationDate="2017-11-23T23:34:32.343" Score="0" ViewCount="125" Body="&lt;p&gt;I was looking for some help with derivatives of Chebychev polynomials at zero points. The recursive expression,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;T_{(j+1)}(x) = 2xT_j(x) - T_{(j-1)}(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;has the derivative &#xA;$$&#xA;T'_{j+1}(x) = 2T_j(x) + 2xT'_j(x) - T'_{j-1}(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $T'_0=0$ and $T'_1=1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I was looking at Trefethen's book (Chapter 6, Page 54) where he uses the following code to construct a derivative matrix for Chebychev polynomials at extreme points:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% CHEB compute D = differentiation matrix, x = Chebyshev grid &#xA;function [D,x] = cheb(N) &#xA;if N==0, D=0; x=1; return, end &#xA;x = cos(pi*(0:N)/N)'; &#xA;c = [2; ones(N-1,1); 2].*(-1).^(0:N)'; &#xA;X = repmat(x,1,N+1); &#xA;dX = X-X'; &#xA;D = (c*(1./c)')./(dX+(eye(N+ 1))) ; % off-diagonal entries &#xA;D = D - diag(sum(D')); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How are these two derivative calculations related? Or is it that the derivatives at extreme points and zero points are calculated differently, as above? Any references I can read to better understand this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="26057" LastActivityDate="2017-11-24T05:25:33.040" Title="Chebychev Polynomial derivatives at zero points and extreme points" Tags="|matlab|finite-difference|approximation|polynomials|chebyshev|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28313" PostTypeId="1" CreationDate="2017-11-24T01:33:36.487" Score="0" ViewCount="241" Body="&lt;p&gt;I am developing a parallel 2D CFD code in C++ using PETSc and would like to use it as an opportunity to learn about VTK/Paraview. Right now, I have each processor output an ASCII file for the portion of the domain that it owns. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where is a good place to start to learn how to format the data to make a VTK file and view it in Paraview? I have the VTK guide (&lt;a href=&quot;https://www.kitware.com/products/books/VTKUsersGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kitware.com/products/books/VTKUsersGuide.pdf&lt;/a&gt;) but was curious if anyone else has experience with this and could offer some advice on where to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is based on the finite volume method with a uniform Cartesian mesh. For each cell center, I have information about the pressure, velocity, density, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="22717" LastActivityDate="2017-11-24T04:51:49.757" Title="Code to output VTK point data from decomposed domain for Paraview" Tags="|visualization|paraview|" AnswerCount="1" CommentCount="0" ClosedDate="2017-11-29T13:35:39.110" ContentLicense="CC BY-SA 3.0" />
  <row Id="28314" PostTypeId="2" ParentId="28313" CreationDate="2017-11-24T04:51:49.757" Score="1" Body="&lt;p&gt;I use VisIt to see vtk files but this may work with Paraview also. Each partition saves its own file which is a standard format and you name them with partition number, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sol01.vtk&#xA;sol02.vtk&#xA;sol03.vtk&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then VisIt can open them together. Make sure the cells in the partitions overlap by adding some ghost cells, otherwise you may have gaps in the visualization.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2017-11-24T04:51:49.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28315" PostTypeId="2" ParentId="28312" CreationDate="2017-11-24T05:25:33.040" Score="1" Body="&lt;p&gt;On p. 53 of the book of Trefethen, you can read &#xA;&quot;D_{N} contains the derivative of the degree N polynomial interpolant p_j(x) to the delta function supported at x_j [= cos(j*pi/N)], sampled at the grid {x_i}&quot;&#xA;So the entries of D are the derivatives of the Lagrange interpolatory polynomials and not the Chebyshev polynomials.&lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2017-11-24T05:25:33.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28316" PostTypeId="1" CreationDate="2017-11-24T10:31:56.697" Score="3" ViewCount="182" Body="&lt;p&gt;I'm working on a Real-Time TDDFT implementation and I am currently comparing different propagation schemes for the propagation of the Kohn-Sham wave function,&#xA;$$&#xA;\phi(t+\Delta t) = \hat{\mathcal{U}}\phi(t).&#xA;$$&#xA;An external field is incorporated by a vector potential and the kinetic energy matrix element is modified such that the hamilton matrix gets a complex part:&#xA;$$&#xA;\mathcal{H}_{ij} = \mathcal{H}_{ij}^{(R)} + i \mathcal{Z}_{ij}&#xA;$$&#xA;where $\mathcal{H}^{(R)}$ denotes all the real parts in the hamilton matrix and $\mathcal{Z}$ is a matrix that incorporates the imaginary part of the terms arising due to the vector potential. I think details are not that important here.&#xA;Among other things, I use the instantaneous total energy $E_\text{tot}(t)$ to compare different propagators$^*$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most implemented integration schemes are exponential-based, i.e.&#xA;$$&#xA;\mathcal{U}=\exp\left(f(\mathcal{H})\right),&#xA;$$&#xA;e.g. Exponential Midpoint with $f(\mathcal{H})=-i\mathcal{S}^{-1}\mathcal{H}(t+\Delta t/2)$ or Enforced Time-Reversal Symmetry with $f(\mathcal{H})=-i\Delta t/2(\mathcal{S}^{-1}\mathcal{H}(t+\Delta t)+\mathcal{S}^{-1}\mathcal{H}(t))$ ($\mathcal{S}$ is the overlap matrix). These methods mostly compute the same results but when I use the Crank-Nicolson propagator,&#xA;$$&#xA;\mathcal{U}=\frac{\mathcal{S}-i\mathcal{H}(t)\Delta t/2}{\mathcal{S}+i\mathcal{H}(t)\Delta t/2},&#xA;$$&#xA;it only reproduces the results when $\mathcal{Z}=0$, else I get (a while after the pulse ends, constant) amplitude oscillations that even persist after a pulse is switched off. Reducing the step size does not change anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IXgK5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IXgK5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is what the reason for this could be. I don't know too much about details of applying these propagation schemes to the given kind of problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$^*$ $E_\text{tot}(t) = \sum_i f_i \epsilon_i(t) - F[\rho(t)] + E_{nn}$ where $\epsilon_i$ are the eigenvalues of the time-dependent hamiltonian, $F$ denotes some functionals incorporating XC and electrostatic corrections and $E_{nn}$ is the internuclear repulsion energy.&lt;/p&gt;&#xA;" OwnerUserId="25510" LastEditorUserId="25510" LastEditDate="2017-12-06T09:07:12.947" LastActivityDate="2017-12-06T09:07:12.947" Title="Crank-Nicolson integrator: oscillations with complex matrix" Tags="|computational-physics|stability|time-integration|density-functional-theory|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28317" PostTypeId="1" CreationDate="2017-11-24T19:25:46.507" Score="0" ViewCount="400" Body="&lt;p&gt;I am curious if anyone had literature references or knowledge on how to apply the Crank-Nicolson (with approximate factorization) to the &#xA;$$&#xA;\nabla \cdot (\nu (\nabla \mathbf{u} + \left(\nabla \mathbf{u}\right)^{T}))&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am unsure about is how to handle the mixed derivative.  Currently I am using Crank-Nicolson for&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nu_{Laminar}\nabla^{2} \mathbf{u}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is straight forward as I can solve it dimension-by-dimension.  &lt;/p&gt;&#xA;" OwnerUserId="21337" LastEditorUserId="713" LastEditDate="2017-11-24T22:09:43.617" LastActivityDate="2018-12-21T05:01:40.510" Title="Crank-Nicolson method and mixed derivatives" Tags="|numerical-analysis|finite-difference|finite-volume|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28318" PostTypeId="2" ParentId="28296" CreationDate="2017-11-24T20:26:27.727" Score="1" Body="&lt;p&gt;I'm not too familiar with QM, but I would use FDM.  You could treat it as a steady-diffusion equation with a source term.  You would have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-\frac{h^{2}}{2 m} \frac{\psi_{i-1} - 2 \psi_{i} + \psi_{i+1}}{\Delta x^{2}} + V \psi_{i} = E \psi_{i}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can rearrange this to be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-\frac{h^{2}}{2 m} \frac{\psi_{i-1} - 2 \psi_{i} + \psi_{i+1}}{\Delta x^{2}} = (E-V)\psi_{i}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use TDMA (Tridiagonal Matrix Algorithm) to solve this ODE as the left-hand side is a tridiagonal matrix and the right-hand side is a forcing function.  Therefore for the tridiagonal matix you have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;a_{i} = -\frac{h^{2}}{2m \Delta x^{2}}\\&#xA;b_{i} = \frac{h^{2}}{m \Delta x^{2}}\\&#xA;c_{i} = -\frac{h^{2}}{2m \Delta x^{2}}\\&#xA;d_{i} = (E - V) \psi_{i}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the boundary condition $\psi(0) = 0$ you make the adjust to the matrix (assuming 1 is your starting index)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;a_{1} = 0\\&#xA;b_{1} = 1\\&#xA;d_{1} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Neumann BC $\psi'(0) = 0$ you will need to evaluate $\psi''$ at $i=1$ and then use a first order approximate $\psi'_{\partial D}$ and then plug in your first order approximate to $\psi''$ evaluate at the boundary.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:  I noticed you said you wanted to use RK4, but I'm confused as you would do that for a Time Dependent Schrodinger Equation TDSE as opposed to TISE.  If you are solving TDSE then you could use RK4 for $\frac{\partial \psi}{\partial t},$ however since type of PDE is parabolic I would suggest using implicit time stepping, such as Crank-Nicholson.  In the transient case you would have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;-\frac{h^{2}}{2 m} \frac{\psi_{i-1} - 2 \psi_{i} + \psi_{i+1}}{\Delta x^{2}} +&#xA; V_{i} \psi_{i} = i h \frac{\psi^{n+1}_{i} - \psi^{n}}{\Delta t} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would give you a $O(\Delta x^{2}, \Delta t)$ approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In explicit form this would give you &lt;/p&gt;&#xA;" OwnerUserId="21337" LastEditorUserId="21337" LastEditDate="2017-11-27T19:06:59.460" LastActivityDate="2017-11-27T19:06:59.460" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28319" PostTypeId="2" ParentId="28317" CreationDate="2017-11-25T21:20:42.130" Score="1" Body="&lt;p&gt;I suggest you just use CG method, and write a routine that computes Ax, where Ax is your momentum diffusion term. Then just use a matrix free implementation of CG. The only requirement is that A is symmetric positive definite, which I think it is for this case.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2017-11-25T21:20:42.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28321" PostTypeId="2" ParentId="28311" CreationDate="2017-11-26T19:02:46.877" Score="3" Body="&lt;p&gt;You can reformulate this problem as a conventional linear least squares problem as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, write $YCZ$ as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$YCZ=\sum_{i=1}^{p} \sum_{j=1}^{m} C_{i,j} (Y_{:,i}Z_{j,:})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, define the $\mbox{vec}$ operator, which stacks column $1$ of a matrix over columns $2$, $3$, $\ldots$ to form a vector.  We'll adopt the convention that lowercase letters are used to denote the vector version of the corresponding matrix which is denoted by an uppercase letter.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can write &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{vec}(YCZ)=Ac$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $c=\mbox{vec}(C)$ and $A$ is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=\left[&#xA;\mbox{vec}(Y_{:,1}Z_{1,:}) \; \mbox{vec}(Y_{:,1}Z_{2,:}) \; \ldots \; \mbox{vec}(Y_{:,p} Z_{m,:})&#xA;\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your least squares problem is now &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| Ac-q \|_{2}^{2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this problem appears to be overdetermined because $A$ has $n(m-1)$ rows and only $pm$ columns, it's actually column rank deficient because of the special structure of $A$.  This means that there will be infinitely many least squares solutions.    MATLAB's \ operator gives a least squares solution with c=A\q, but provides no control over which of the infinitely many least squares solutions you'll get.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather, you'll want to include some kind of regularization term.  For example, if you want to minimize the Frobenius norm of $C$, you could minimize &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| Ac-q \|_{2}^{2} +\lambda^{2} \| c \|_{2}^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\lambda$ is a small regularization parameter.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| \bar{A}c - \bar{q} \|_{2}^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bar{A}=\left[ \begin{array}{c} A \\ \lambda I \end{array} \right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\bar{q}=\left[ \begin{array}{c} q \\ 0 \end{array} \right]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other kinds of quadratic regularization are relatively easy to obtain by replacing $\lambda I$ with an appropriate matrix.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In actually solving this linear least squares problem, you'll want to avoid forming the very large (and presumably dense) matrix $A$.  You can use an iterative method such as LSQR (included in MATLAB) for the least squares problem and take advantage of the special structure of $A$ to provide your own routines for multiplying $A$ times a vector and $A^{T}$ times a vector.  For example, to multiply $Au$, simply take $\mbox{vec}(YUZ)$ where $u=\mbox{vec}(U)$. &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2017-11-26T19:13:31.753" LastActivityDate="2017-11-26T19:13:31.753" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28324" PostTypeId="2" ParentId="28270" CreationDate="2017-11-26T22:30:41.403" Score="3" Body="&lt;p&gt;The primary motivation for mixed methods is that you &lt;em&gt;want&lt;/em&gt; both the displacement and the stress to be primary variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you substitute the one equation into the other, then you only have the displacement as the primary variable. You can then, at a later time, compute the stress from the displacement, but this post processing step usually leads to lower accuracy in the computed stress than in the computed displacement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you keep both of these variables as primary variables, you can choose the approximation order of the two independently (within limits), and you can guarantee that the computed stress is at least as accurate as the computed displacement, and in general the stress will be more accurate this way than if you computed it as a postprocessing step from the displacement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If you now ask why anyone would &lt;em&gt;need&lt;/em&gt; the computed stress to be accurate: Think about cases where you need to know whether a material will break -- i.e., whether the stress exceeds a certain limit -- or where the stress determines the behavior of the material -- for example, in plasticity.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-11-26T22:30:41.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28326" PostTypeId="1" AcceptedAnswerId="28327" CreationDate="2017-11-27T04:38:31.087" Score="1" ViewCount="843" Body="&lt;p&gt;This is to help me relate continuous and discrete, predict what my scheme should be doing, and move toward using the method of manufactured solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a solution for constant velocity $c$ advecting a quantity $u(x,t)$ over a 1D domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} =&#xA;- c\frac{\partial u}{\partial x}&#xA;\\&#xA;  u_t=-cu_x&#xA;$$&#xA;of $\require{enclose} u(x,t) = \enclose{horizontalstrike}{f(x+ct) +} g(x-ct)$, where $g$ is a profile moving with velocity $c$. [edited]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's more complicated when the quantity being advected by velocity is the velocity itself:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  u_t=-uu_x&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I worked out one solution $u(x,t)=x/t$: a simple linear profile, starting at $t_0 = 1$. It generalizes to $u(x,t) = (x+a)/(t+b)$, where $a$ changes the x-intercept, and $b$ changes the slope of the initial profile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A positive slope is divergent (positive 1D $div$), and the line flattens over time as the quantity moves away. However, a negative slope (achieved with $b&amp;lt;1$) is compressive (negative 1D $div$), and leads to a singularity as $t+b \to 0$, and the quantity accelerates towards the x-intercept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But is there a general solution? Although there's apparently no known solultion to the full navier-stokes equations, it seems to me there should be one for advection of velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I think $u(x, t_0) = cos x$ would diverge on the positive slopes (approaching zero), and compress on the negative slopes (becoming a step function) - but I can't work out a solution (maybe using Taylors series?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a general analytical solution to the 1D advection of velocity, $u_t=-uu_x$?&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2017-11-27T06:04:13.557" LastActivityDate="2017-11-27T06:48:38.913" Title="Is there a general analytic solution to 1D advection of velocity, $u_t=-uu_x$?" Tags="|pde|advection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28327" PostTypeId="2" ParentId="28326" CreationDate="2017-11-27T04:57:31.237" Score="2" Body="&lt;p&gt;Your solution for linear case is not correct. For the equation&#xA;$$&#xA;u_t + c u_x = 0&#xA;$$&#xA;the solution is&#xA;$$&#xA;u(x,t) = u(x-ct,0) = u_0(x-ct)&#xA;$$&#xA;For Burger's eqn, as long as the solution is smooth it can be written as&#xA;$$&#xA;u(x,t) = u(x - u(x,t)t,0) = u_0(x-u(x,t)t)&#xA;$$&#xA;This is implicit equation and in some simple cases, you may be able to find analytical solutions. If there are discontinuities, then the solution in smooth parts is still given as above, but you have to patch them up with the jump condition at places where it is discontinuous. Again this can be done analytically in some simple cases, say if your initial condition is not too nonlinear. You have to find where the shocks will form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to trace the characteristics backward from any point $(x,t)$ to the initial point $(x_0,0)$. These are straight lines so&#xA;$$&#xA;dx/dt = u(x,t), \qquad x = x_0 + u(x,t) t = x_0 + u_0(x_0) t&#xA;$$&#xA;Solve for $x_0 = x_0(x,t)$ which can be done analytically if $u_0$ is simple function. Then&#xA;$$&#xA;u(x,t) = u_0(x_0(x,t))&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are shocks, then the solution is given by Lax-Oleinik formula, see [1]. But this does not give an explicit analytical solution since the formula involves some minimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It depends on what you mean by analytic solution. The Lax-Oleinik formula is a general analytical solution, but it may not give you an explicit formula for the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution $u(x,t) = x/t$ is a special solution for a rarefaction wave. If you have initial condition&#xA;$$&#xA;u(x,0) = \begin{cases}&#xA;u_L &amp;amp; x &amp;lt; 0 \\&#xA;u_R &amp;amp; x &amp;gt; 0&#xA;\end{cases}&#xA;$$&#xA;with $u_L &amp;lt; u_R$, then the solution for Burgers equation is&#xA;$$&#xA;u(x,t) = \begin{cases}&#xA;u_L &amp;amp; x &amp;lt; u_L t \\&#xA;x/t &amp;amp; u_L t \le x \le u_R t\\&#xA;u_R &amp;amp; x &amp;gt; u_R t&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Evans, Partial Differential Equations, Chap. 3, Thm 1, page 145&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2017-11-27T06:48:38.913" LastActivityDate="2017-11-27T06:48:38.913" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28330" PostTypeId="1" CreationDate="2017-11-27T11:02:37.827" Score="1" ViewCount="312" Body="&lt;p&gt;I have a coupled nonlinear PDE system in 1 spatial dimension, in which I want to solve using FiPy. The dependent variables are $n$ and $T$:&#xA;\begin{align}&#xA;\frac{\partial n}{\partial t} \,&amp;amp;=\, D\,\frac{\partial^2 n}{\partial x^2} \\&#xA;\frac{\partial T}{\partial t} \,&amp;amp;=\, \frac{D}{\zeta}\,\frac{\partial^2 T}{\partial x^2} \,+\, \left(\frac{\zeta + 1}{\zeta}\right) \frac{D}{n} \, \frac{\partial n}{\partial x} \, \frac{\partial T}{\partial x}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of now, both $D$ and $\zeta$ are simple numerical parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having serious trouble representing the second term in the $\frac{\partial T}{\partial t}$ equation, as there seems to be no nice way of writing it. Should it be written as a Convection term, or as a Source term?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code so far, omitting some things:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;L = 5.0&#xA;nx = 100&#xA;mesh = Grid1D(nx=nx, dx=L/nx)&#xA;x = mesh.cellCenters[0]&#xA;&#xA;density = CellVariable(name=r&quot;$n$&quot;, mesh=mesh)     # The n dependent variable&#xA;temperature = CellVariable(name=r&quot;$T$&quot;, mesh=mesh) # The T dependent variable&#xA;&#xA;D = 5.0&#xA;zeta = 0.5&#xA;&#xA;density_equation = TransientTerm(var=density) == \&#xA;                    DiffusionTerm(coeff=D, var=density)&#xA;&#xA;temp_equation = TransientTerm(var=temperature) == \&#xA;                 DiffusionTerm(coeff= D / zeta, var=temperature) \&#xA;                          + S_T    # &amp;lt;- What should be filled in for this?&#xA;&#xA;semifull_eq = density_equation &amp;amp; temp_equation&#xA;&#xA;if __name__ == '__main__':&#xA;    viewer = Viewer((density, temperature))&#xA;&#xA;timeStepDuration = dx**2 / (2*diffusivity)&#xA;for steps in range(100):&#xA;    semifull_eq.solve(dt=timeStepDuration)&#xA;    if __name__ == '__main__':&#xA;        viewer.plot()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have left out the boundary and initial conditions for brevity.&lt;/p&gt;&#xA;" OwnerUserId="26082" LastEditorUserId="26082" LastEditDate="2017-11-27T11:17:49.810" LastActivityDate="2017-11-27T11:17:49.810" Title="FiPy with derivative source terms" Tags="|pde|python|finite-volume|elliptic-pde|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28333" PostTypeId="1" AcceptedAnswerId="28334" CreationDate="2017-11-27T19:52:05.960" Score="1" ViewCount="1080" Body="&lt;p&gt;Having both programmed my share of c++ and studied some beginners group theory some year ago, I got curious about this...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any particularly popular template based (object oriented) design pattern for systematically programming any kind of stuff related to group theory?&lt;/p&gt;&#xA;" OwnerUserId="18579" LastEditorUserId="18579" LastEditDate="2017-11-27T20:14:55.700" LastActivityDate="2017-11-27T23:10:43.450" Title="C++ template design pattern for groups (algebra)" Tags="|c++|reference-request|nonlinear-equations|design-patterns|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28334" PostTypeId="2" ParentId="28333" CreationDate="2017-11-27T21:14:27.740" Score="4" Body="&lt;p&gt;Chapters 6 and 7 of &lt;a href=&quot;http://www.fm2gp.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stepanov and Rose's book From Mathematics to generic programming&lt;/a&gt; deal precisely with what you asked. The first author is the designer of the C++ Standard Template Library.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2017-11-27T21:14:27.740" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28335" PostTypeId="2" ParentId="28333" CreationDate="2017-11-27T23:10:43.450" Score="2" Body="&lt;p&gt;An earlier alternative to Stepanov's treatment of abstract algebra (including groups) is the one given in Barton and Nackman's &quot;Scientific and Engineering C++&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In more modern c++, the &lt;a href=&quot;http://www.boost.org/doc/libs/1_59_0/libs/concept_check/concept_check.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;boost concept checking&lt;/a&gt; library would seem to be a quite natural means of modularising the hierarchy of algebraic types (semigroups, monoids, groups etc).&lt;/p&gt;&#xA;" OwnerUserId="17504" LastActivityDate="2017-11-27T23:10:43.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28336" PostTypeId="1" CreationDate="2017-11-28T11:34:03.727" Score="1" ViewCount="1085" Body="&lt;p&gt;I am computing the maximum of a function (with two-variables) using Newton-Raphson method. The function is : $e^{-(x \ - x_0)^2 - (y \ - y_0)^2}$, whose maxima exists at $(x_0,y_0)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobian will be a function of the second derivatives (including mixed) of this function. It can be written as : $J = e^{-(x \ - x_o)^2 - (y \ - y_o)^2} \ &#xA;\begin{Bmatrix}&#xA;-2 \ + \ 4 \times (x \ - \ x_o)^2 &amp;amp;&amp;amp; 4 \times (x \ - \ x_o) \ (y \ - \ y_o) \\&#xA;4 \times (x \ - \ x_o) \ (y \ - \ y_o) &amp;amp;&amp;amp; -2 \ + \ 4 \times (y \ - \ y_o)^2&#xA;\end{Bmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above expression analytically calculates the second derivatives to evaluate the Jacobian. The Newton-Raphson can now be formulated as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\vec{x}^{\, n+1} \ = \ \vec{x}^{\, n} \ - \ J^{-1} \ f(\vec{x}^{\,n})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\vec{x}$ includes $x$ and $y$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative way to evaluate the Jacobian (second derivatives) includes using Finite Differences like : $J_{i,j} (\vec{x}) \ = \ \frac{f_i(\vec{x} \ + \ h_j \vec{e_j}) \ - \ f_i(\vec{x})}{h_j}$, where  $h_j$ is approximately taken to be $x_j \ \times \sqrt{\epsilon}$, with $\epsilon$ being the computer precision (generally taken as $10^{-16}$). This can be roughly called as Secant Method (though not exactly, as it means). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically, analytical way of calculating the Jacobian should yield faster convergence (in a few iterations) in comparison to the aternative way described above. However, I seem to get the converse result (for the guess values ($ x_0$ and $y_0$ respectively) taken to be 3.6 and 4.6 in the code below !). In other words, I seem to get the converged solution (for the case of Analytical Jacobi) after 26 iterations, whereas 15 for the other case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code, where I have implemented this. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include&amp;lt;iostream&amp;gt;&#xA;#include&amp;lt;cmath&amp;gt;&#xA;#include&amp;lt;fstream&amp;gt;&#xA;#include&amp;lt;iomanip&amp;gt;&#xA;&#xA;using namespace std;&#xA;&#xA;double eps = 1e-16;&#xA;&#xA;void user_input(double x0, double y0)&#xA;{&#xA;        cout &amp;lt;&amp;lt; &quot;Enter the value of x0 - to find the maximum of the exponential function\n&quot; &amp;lt;&amp;lt; endl;&#xA;        cin &amp;gt;&amp;gt; x0;&#xA;        cout &amp;lt;&amp;lt; &quot;Enter the value of y0 - to find the maximum of the exponential function\n&quot; &amp;lt;&amp;lt; endl;&#xA;        cin &amp;gt;&amp;gt; y0;&#xA;&#xA;}&#xA;&#xA;//evaluates the exponential at x0 and y0&#xA;double exp_fn(double x, double y, double x0, double y0)&#xA;{&#xA;    double val;&#xA;    val = exp(-(x - x0)*(x - x0) - (y - y0)*(y - y0));&#xA;    //cout &amp;lt;&amp;lt; val &amp;lt;&amp;lt; endl;&#xA;&#xA;    return val;&#xA;}&#xA;&#xA;//x- gradient of the given function&#xA;double f_x(double x, double y, double x0, double y0)&#xA;{&#xA;&#xA;  double value;&#xA;  value = -2*exp_fn(x,y,x0,y0)*(x - x0);&#xA;&#xA;  return (value);&#xA;}&#xA;&#xA;//y- gradient of the given function&#xA;double f_y(double x, double y, double x0, double y0)&#xA;{&#xA;&#xA;  double value;&#xA;  value = -2*exp_fn(x,y,x0,y0)*(y - y0);&#xA;&#xA;  return (value);&#xA;}&#xA;&#xA;//first component in Jacobian&#xA;double f_xx(double x, double y, double x0, double y0)&#xA;{&#xA;        double value;&#xA;        value = exp_fn(x,y,x0,y0)*( -2 + 4*(x - x0)*(x - x0));&#xA;&#xA;        return value;&#xA;}&#xA;&#xA;//second component in Jacobian&#xA;double f_xy(double x, double y, double x0, double y0) //returns the same value as f_yx&#xA;{&#xA;        double value;&#xA;        value = exp_fn(x,y,x0,y0)*(x - x0)*(y - y0)*4;&#xA;&#xA;        return value;&#xA;}&#xA;&#xA;//last component in Jacobian&#xA;double f_yy(double x, double y, double x0, double y0)&#xA;{&#xA;        double value;&#xA;        value = exp_fn(x,y,x0,y0)*( -2 + 4*(y - y0)*(y - y0));&#xA;&#xA;        return value;&#xA;}&#xA;&#xA;//equivalent to f_xx, but using finite difference, not analytical&#xA;double f_xx_sec(double x, double y, double x0, double y0)&#xA;{&#xA;  double value, h_x;&#xA;  h_x = x*sqrt(eps);&#xA;&#xA;  value = (f_x(x + h_x, y, x0, y0) - f_x(x, y, x0, y0))/h_x;&#xA;&#xA;  return value;&#xA;}&#xA;&#xA;//equivalent to f_xy, but using finite difference, not analytical&#xA;double f_xy_sec(double x, double y, double x0, double y0)&#xA;{&#xA;  double value, h_y;&#xA;  h_y = y*sqrt(eps);&#xA;&#xA;  value = (f_x(x, y + h_y, x0, y0) - f_x(x, y, x0, y0))/h_y;&#xA;&#xA;  return value;&#xA;}&#xA;&#xA;//equivalent to f_yx, but using finite difference, not analytical&#xA;double f_yx_sec(double x, double y, double x0, double y0)&#xA;{&#xA;  double value, h_x;&#xA;  h_x = x*sqrt(eps);&#xA;&#xA;  value = (f_y(x + h_x, y, x0, y0) - f_y(x, y, x0, y0))/h_x;&#xA;&#xA;  return value;&#xA;}&#xA;&#xA;//equivalent to f_yy, but using finite difference, not analytical&#xA;double f_yy_sec(double x, double y, double x0, double y0)&#xA;{&#xA;  double value, h_y;&#xA;  h_y = y*sqrt(eps);&#xA;&#xA;  value = (f_y(x, y + h_y, x0, y0) - f_y(x, y, x0, y0))/h_y;&#xA;&#xA;  return value;&#xA;}&#xA;&#xA;//determinant of the matrix &#xA;double det(double a, double b, double c, double d)&#xA;{&#xA;    double det_val;&#xA;    det_val = (a*d - b*c);&#xA;    return det_val;&#xA;&#xA;}&#xA;&#xA;&#xA;int main(int argv, char** argc)&#xA;{&#xA;&#xA;    int user = 0, iter = 1; //user = 0 determines the default x0 and y0 values to be used. &#xA;    //Else, for user to input these values, equate user = 1&#xA;&#xA;    int secant_met = 1; //1 causes secant method of Jacobian computation. 0 uses the conventional way&#xA;&#xA;    double x0 = 4, y0 = 7, x_old =3.6, y_old =6.7, error = 1., magnitude; //x0, y0 - guess  values&#xA;&#xA;    double x_new, y_new, mult_x, mult_y, lambda = 1.0; // lambda refers to the relaxation factor&#xA;&#xA;    ofstream fout;&#xA;    fout.open(&quot;convergence_data.dat&quot;);&#xA;    fout &amp;lt;&amp;lt; &quot;Error&quot; &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; &quot;X_updated&quot; &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; &quot;Y_updated&quot; &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; &quot;Iteration&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;&#xA;    if (user == 1)&#xA;      user_input(x0,y0);&#xA;&#xA;    while(error &amp;gt; 1e-8)&#xA;    {&#xA;&#xA;      if (secant_met ==1) //using the secant way of evaluating Jacobian - Finite differences&#xA;        {&#xA;          mult_x = f_x(x_old, y_old, x0, y0)*f_yy_sec(x_old, y_old, x0, y0) - f_y(x_old, y_old, x0, y0)*f_xy_sec(x_old, y_old, x0, y0);&#xA;          mult_y = f_y(x_old, y_old, x0, y0)*f_xx_sec(x_old, y_old, x0, y0) - f_x(x_old, y_old, x0, y0)*f_yx_sec(x_old, y_old, x0, y0);&#xA;&#xA;          x_new = x_old - lambda*mult_x/det(f_xx_sec(x_old, y_old, x0, y0), f_xy_sec(x_old, y_old, x0, y0), f_yx_sec(x_old, y_old, x0, y0), f_yy_sec(x_old, y_old, x0, y0)); //Generalized newton's method&#xA;          y_new = y_old - lambda*mult_y/det(f_xx_sec(x_old, y_old, x0, y0), f_xy_sec(x_old, y_old, x0, y0), f_yx_sec(x_old, y_old, x0, y0), f_yy_sec(x_old, y_old, x0, y0));&#xA;        }&#xA;&#xA;      else // implementing conventional way of computing Jacobian&#xA;        {&#xA;          mult_x = f_x(x_old, y_old, x0, y0)*f_yy(x_old, y_old, x0, y0) - f_y(x_old, y_old, x0, y0)*f_xy(x_old, y_old, x0, y0);&#xA;          mult_y = f_y(x_old, y_old, x0, y0)*f_xx(x_old, y_old, x0, y0) - f_x(x_old, y_old, x0, y0)*f_xy(x_old, y_old, x0, y0);&#xA;&#xA;          cout &amp;lt;&amp;lt; setprecision(7) &amp;lt;&amp;lt; f_xx(x_old, y_old, x0, y0) &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; setprecision(7) &amp;lt;&amp;lt; f_xy(x_old, y_old, x0, y0) &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; setprecision(7)&amp;lt;&amp;lt; f_yy(x_old, y_old, x0, y0) &amp;lt;&amp;lt; endl;&#xA;&#xA;            x_new = x_old - lambda*mult_x/det(f_xx(x_old, y_old, x0, y0), f_xy(x_old, y_old, x0, y0), f_xy(x_old, y_old, x0, y0), f_yy(x_old, y_old, x0, y0)); //Generalized newton's method&#xA;            y_new = y_old - lambda*mult_y/det(f_xx(x_old, y_old, x0, y0), f_xy(x_old, y_old, x0, y0), f_xy(x_old, y_old, x0, y0), f_yy(x_old, y_old, x0, y0));&#xA;        }&#xA;&#xA;        magnitude = fabs( (x_new - x_old)*(x_new - x_old) + (y_new - y_old)*(y_new - y_old)); &#xA;        error = sqrt(magnitude); //calculating the error for convergence &#xA;&#xA;        x_old = x_new;&#xA;        y_old = y_new;&#xA;&#xA;        cout &amp;lt;&amp;lt; fixed;&#xA;        cout &amp;lt;&amp;lt; &quot;Error = &quot; &amp;lt;&amp;lt;  error &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; &quot;Iteration&quot; &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt;  iter &amp;lt;&amp;lt; endl &amp;lt;&amp;lt; &quot;X updated&quot; &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; x_new &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; &quot;Y_updated&quot; &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; y_new &amp;lt;&amp;lt; endl;&#xA;        fout &amp;lt;&amp;lt; fixed;&#xA;        fout &amp;lt;&amp;lt; error &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; x_new &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; y_new &amp;lt;&amp;lt; &quot;\t&quot; &amp;lt;&amp;lt; iter &amp;lt;&amp;lt; endl;&#xA;&#xA;        iter++;&#xA;    }&#xA;&#xA;&#xA;    fout.close();&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone give me an insight as to where I am going wrong in the code, wherein I have implemented both these methods ? Any kind of help would be deeply appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the plot of convergence error vs iterations which I seem to get when using both the methods, as described above. &lt;a href=&quot;https://i.stack.imgur.com/lmO44.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lmO44.jpg&quot; alt=&quot;Plot of Convergence error vs Iterations when using both the methods&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25983" LastEditorUserId="25983" LastEditDate="2017-11-29T16:05:24.950" LastActivityDate="2017-11-29T16:05:24.950" Title="Newton - Raphson method : maxima of function in 2 variables" Tags="|numerical-analysis|c++|newton-method|jacobian|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28337" PostTypeId="2" ParentId="533" CreationDate="2017-11-28T15:08:44.240" Score="0" Body="&lt;p&gt;The exact exchange functional is known, this is the HF exchange energy. The problem here is that it depends explicitly on the orbitals and not in the electronic density as the usual functionals. By means of the Optimized Effective Potential Method is possible to evaluate exactly the exchange energy in the frame of DFT. This has been successfully done for atoms and molecules and more recently for periodic systems. So I think the problem is not anymore in the exchange, it is in the correlation functional that is still unknown and we have to use approximations. But most of the time correlations effects are really small and the aproximations are pretty good.&lt;/p&gt;&#xA;" OwnerUserId="20281" LastActivityDate="2017-11-28T15:08:44.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28340" PostTypeId="1" AcceptedAnswerId="28349" CreationDate="2017-11-29T10:38:58.067" Score="1" ViewCount="85" Body="&lt;p&gt;What is the best method to find for a linear system of inequalities $Ax\ge b$ with dense $A$ of moderate dimension the affine subspace spanned by the feasible points (i.e., the implied equalities $(Ax)i=b_i$) and a relatively interior feasible point in this subspace?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It shouldn't need the solution of many linear programs. I recall vaguely from long ago that appropriate postprocessing of the solution of a single appropriate LP should be sufficient, but don't remember details.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2017-12-02T16:45:11.267" LastActivityDate="2017-12-02T20:24:03.737" Title="implied equalities and relative interior" Tags="|linear-programming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28341" PostTypeId="1" CreationDate="2017-11-29T12:54:44.687" Score="0" ViewCount="289" Body="&lt;p&gt;I'm trying to implement &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.298.4533&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linear Least-Squares Optimization for Point-to-Plane ICP Surface Registration&lt;/a&gt;&#xA;Paper describes linear approximation for point to plane distance for rigid ICP. This approach is pretty straightforward however during implementation I faced with some strange degenerative case where such approximation lead to solving singular linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me explain. We want to solve following optimization problem (ordinary sum of point-to-plane distances)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;argmin_{\alpha, \beta, \gamma, tx, ty, tz}\sum_i((M(\alpha, \beta, \gamma, tx, ty, tz)s_i - d_i)^Tn_i)^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $M$ is 4x4 transformation matrix computed from euler angles and translations, $\{s_i\}$ and $\{d_i\}$ are ordered sets of 3d points sampled from surfaces $S$ and $D$, $\{n_i\}$ is ordered set of normals from $D$ which correspond to points $\{d_i\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously this problem is non-linear least squares due to trigonometric functions inside $M$. The author proposes linear version of $M$ by replacing $cos(\alpha)$ by 1 and $sin(\alpha)$ by $\alpha$. After this approximation problem above becomes simple linear least squares problem and can be expressed in form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;||Ax - b||_2^2&#xA;$$&#xA;where $x$ is vector of optimization variables. Matrix $A$ and vector $b$ can be easily expressed from $s_i, d_i, n_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From paper $i$ row of $A$ is equal (by eq 10 from paper)&#xA;$$&#xA;a_i = [n_{iz}s_{iy} - n_{iy}s_{iz}, n_{ix}s_{iz} - n_{iz}s_{ix}, n_{iy}s_{ix} - n_{ix}s_{iy}, n_{ix}, n_{iy}, n_{iz}]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let consider simple geometric example. Let $\{s_i\} = \{[1, 0, 0], [0, 1, 0], [0, 0, 1], [-1, 0, 0], [0, -1, 0], [0, 0, -1]\}$ and ${d_i} = {n_i} = {s_i}$. Now vertices $s_i$ are parallel with normals $n_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously first 3 columns of $A$ are equal zero which leads us to ill-posed least squares problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&#xA;What is geometrical interpretation for such case? Why collinearity of normals and points leads to bad optimization problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sory for my English/Latex skills&lt;/p&gt;&#xA;" OwnerUserId="15233" LastActivityDate="2018-01-03T18:24:48.627" Title="Linear Least-Squares Point-to-Plane ICP degenerative case" Tags="|linear-algebra|computational-geometry|approximation|least-squares|3d|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28343" PostTypeId="1" CreationDate="2017-11-29T16:43:21.847" Score="0" ViewCount="96" Body="&lt;p&gt;Say I have a matrix like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{bmatrix}&#xA;    0  &amp;amp; 0 &amp;amp; 0 &amp;amp;  1 \\&#xA;    0  &amp;amp; 0 &amp;amp; 1 &amp;amp;  1 \\&#xA;    0 &amp;amp; 1 &amp;amp; 1 &amp;amp;  1 \\&#xA;\end{bmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{bmatrix}&#xA;    0  &amp;amp; 0 &amp;amp; 0 &amp;amp;  1 \\&#xA;    0  &amp;amp; 0 &amp;amp; 1 &amp;amp;  1 \\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp;  1 \\&#xA;\end{bmatrix}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between this two matrices is 2 elements in 12, or 16,67%.&#xA;So, I can say that they have 83,33% similarity.Which are some good algorithms or heuristics(I don't need more precision than +- 5%) to get this number(the 83,33% on this case) other than the naive algorithm that walk both matrices and compare each element using a nested loop structure?&lt;/p&gt;&#xA;" OwnerUserId="26118" LastActivityDate="2018-03-17T04:25:02.543" Title="Which are some good algorithms and heuristics to calculate the similarity between two matrices?" Tags="|algorithms|matrix|approximation|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28344" PostTypeId="1" CreationDate="2017-11-29T19:57:19.033" Score="1" ViewCount="87" Body="&lt;p&gt;I need to solve the following partial differential equation in seven variables with four boundary conditions. I don't think Mathematica has the capacity to solve this differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EHdWH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EHdWH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know of scientific packages which can be used to solve this equation and plot graphs of the dependent variable against each of the independent variables?&lt;/p&gt;&#xA;" OwnerUserId="17326" LastEditorUserId="17326" LastEditDate="2017-11-30T04:37:27.767" LastActivityDate="2017-11-30T04:37:27.767" Title="Second-order PDE with seven variables" Tags="|differential-equations|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28345" PostTypeId="2" ParentId="26803" CreationDate="2017-11-29T20:02:17.160" Score="1" Body="&lt;p&gt;&lt;em&gt;Disclaimer&lt;/em&gt;: I was searching for 'higher order FE interpolation' when I came across this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Univariate Interpolation (i.e., 1-D)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Higher order interpolation could be interpreted in multiple ways. But one of the most common ways of doing higher order interpolation is &lt;em&gt;piecewise polynomial interpolation&lt;/em&gt; where the polynomials are of degree 2 or higher (degree is same as order in this case). Degree 3 is most popular, and the method is known as &lt;em&gt;cubic spline interpolation&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;: As far as I can tell, piecewise polynomial interpolation might as well be called &lt;em&gt;FE interpolation&lt;/em&gt; (in fact, a specific version of FE, due to choice of basis/trial and weight/test functions) and I fail to understand why I haven't come across a textbook that points that out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;: 'FE interpolation' is different from 'FE for differential equation solving'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Multivariate Interpolation (i.e., 2-D and higher)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 2-D, a degree 3 interpolation on structured rectilinear mesh would be called bicubic, for 3-D tricubic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A structured rectilinear mesh is one in which the grid points are ordered in X and Y dimension, and in case of 3-D, Z dimension as well. Structured or regular. Mesh or grid. Basis or trial. Weight or test. (If you think proliferation of terminology is a pathological problem of this discipline, welcome to the club.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MATLAB (as well as the python scientific stack) provide ways of doing such interpolation, e.g., in &lt;code&gt;scipy&lt;/code&gt; the API is &lt;code&gt;RectBivariateSpline&lt;/code&gt; for 2-D. However finite element codes deal with simplices, e.g., triangles in 2-D, and tetrahedrons in 3-D, and this poses a problem (the mesh is unstructured). You can use the API &lt;code&gt;griddata&lt;/code&gt; to &quot;griddify&quot; your solution, but my guess is that it would be less accurate than directly interpolating $u_h$, plus my personal experience is that &lt;code&gt;griddata&lt;/code&gt; based interpolation is very finicky in handling boundaries, and generally fails when a bit of extrapolation is involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to interpolate directly from a solution on unstructured mesh? I don't have a very good answer. Based on what I'm learning these days, FE interpolation (or can it also be called unstructured multivariate spline interpolation?) should work but the book I have (Heath, Scientific Computing) doesn't have any mention of FE in the chapter on interpolation (but then it doesn't discuss multivariate interpolation at all). Langtangen's FE texts do start with FE interpolation (that's where I learned about it) before moving on to FE DE solving, but even his discussion is for pedagogical purposes (to help your understand later chapters), not as an application. Plus Langtangen makes no mention of whether or not hat/bubble FE interpolation is just spline interpolation in disguise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Remark&lt;/em&gt;: In computer graphics, (e.g., in Foley, Computer Graphics) there is a similar concept of 'parametric cubic curves' (1-D), and 'parametric bicubic surfaces' (2-D). How does that compare or contrast with interpolation? I haven't come across a text that answers that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Should we use FE approximation with higher order elements?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell, the authors you mentioned do not mean using higher-order FE for differential equation solving. And as I said, in my non-expert opinion, FE can be used for interpolation (regardless of what you used for your differential equation solving), but again, I can't find textbook level literature on FE interpolation, and I fail to understand why it's not routinely used for interpolation purposes (for multivariate solutions on unstructured meshes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;P.S.&lt;/em&gt;: The book you mentioned doesn't have the word &quot;Solving&quot; in its title.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1&lt;/strong&gt;: &lt;a href=&quot;https://stackoverflow.com/a/32763635/3973303&quot;&gt;This stackoverflow answer&lt;/a&gt; mentions &lt;em&gt;radial basis functions&lt;/em&gt; as an option for unstructured multivariate interpolation. Firstly, I would disagree with the statement &quot;splines are radial basis functions&quot;. Splines are based on polynomials, while RBF could be any function of $r$ that you want. Linear and cubic spline are definitely not radial in 1-D, since both $x$ and $x^3$ are odd. Secondly, last I checked, the RBF implementation in the python scientific stack was even slower than &lt;code&gt;griddata&lt;/code&gt; followed by &lt;code&gt;RectBivariateSpline&lt;/code&gt; (but I'm using an older version of numpy/scipy so it's possible they've sped it up in the recent versions, see &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/interpolate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; for more info).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, &lt;a href=&quot;https://en.wikipedia.org/wiki/Multivariate_interpolation#Irregular_grid_.28scattered_data.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Irregular grid section in wikipedia page on multivariate interpolation&lt;/a&gt; is probably a good point of reference. 'Triangular irregular network' looks similar to what I'm talking about (but in 2-D only). However it only mentions two options, natural neighbor, and linear interpolation, which are 'probably', and 'definitely', of degree 1, respectively (nearest neighbor would be degree 0). Hence they are not higher order. For higher order, you would need to look for a triangular irregular network of degree 2 or higher. And in 3-D it would probably be called tetrahedral irregular network.&lt;/p&gt;&#xA;" OwnerUserId="15198" LastEditorUserId="15198" LastEditDate="2017-11-30T21:10:45.823" LastActivityDate="2017-11-30T21:10:45.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28348" PostTypeId="1" CreationDate="2017-11-29T23:24:12.553" Score="2" ViewCount="94" Body="&lt;p&gt;Suppose a matrix $M\in\mathbb{R}^{n\times n}$ is defined as the solution to a Sylvester equation $$AM+MB=C,$$ for some fixed (known) matrices $A,B,C$.  In the regime where $n$ is large, we may with not to compute $M$ explicitly.  For instance, suppose $A,B,C$ are sparse; it may be the case that $M$ is a dense matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a vector $x\in\mathbb{R}^n$.  &lt;strong&gt;Is there a way to compute the product $Mx$ faster than solving for $M$ explicitly and then multiplying?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My basic inspiration here comes from algorithms like conjugate gradients, which can compute the vector $A^{-1}x$ for positive definite $A$ without finding the matrix $A^{-1}$ explicitly. Generalizations to $M$ defined for Riccati/Lyapunov systems of equations are appreciated as well!&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2017-11-30T19:15:20.487" Title="Implicit solution to Sylvester equation" Tags="|linear-algebra|linear-solver|iterative-method|implicit-methods|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28349" PostTypeId="2" ParentId="28340" CreationDate="2017-11-30T05:15:52.203" Score="3" Body="&lt;p&gt;Probably not the most efficient way, but you could do:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Formulate the system of inequalities as &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$Ax - s = b$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s \geq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then maximize $s_{i}$ with respect to the inequalities for $i=1, 2, \ldots, m$.  If the maximum value of some $s_{i}$ is 0, then that inequality is one of the implied equalities. You'll want to remove any redundant equations that appear. &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Once you've found your system of equations, use LP to find a particular point that satisfies the original inequalities, and then express the affine hull in terms of that particular solution and a basis for the null space of the matrix of implied equality constraints.  You can now rewrite the original system of inequalities in terms of the coefficients with respect to this basis and reduce the dimension.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In terms of this new basis, find the Chebyshev center of the feasible set within the implied affine set by solving another LP.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;P.S.  A little research on the internet found a solution to this problem that requires the solution of only one LP.  However, the solution must be strictly complementary (something that you don't get from the simplex method but can get using a primal-dual interior point method.)  See Theorem 8 in this 1996 paper by Harvey Greenberg and the earlier sources that he cites:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.128.8466&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.128.8466&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2017-12-02T20:24:03.737" LastActivityDate="2017-12-02T20:24:03.737" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28351" PostTypeId="2" ParentId="28340" CreationDate="2017-11-30T06:51:55.680" Score="0" Body="&lt;p&gt;You may want to take a look at a book on linear programming. Finding a starting point for the vertex algorithm of linear programming entails exactly the question you are asking on here. The algorithm used is usually called &quot;phase 1&quot; in linear programming.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-11-30T06:51:55.680" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28352" PostTypeId="2" ParentId="28188" CreationDate="2017-11-30T10:17:29.387" Score="0" Body="&lt;p&gt;After quick testing, H-correction seems to solve the problem!&#xA;For future readers: see the first comment for the relevant paper link.&lt;/p&gt;&#xA;" OwnerUserId="25060" LastActivityDate="2017-11-30T10:17:29.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28353" PostTypeId="1" AcceptedAnswerId="28354" CreationDate="2017-11-30T11:42:00.027" Score="0" ViewCount="109" Body="&lt;p&gt;In the second answer &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26774/computing-accurate-fluxes-with-fem/26779#26779&quot;&gt;here&lt;/a&gt; one considers a function $w\in H^{\frac{1}{2}}(\Gamma_g)$. What is this space and what is the finite element functions that one should use that belongs to this space? I have tetrahedral elements.&lt;/p&gt;&#xA;" OwnerUserId="24446" LastActivityDate="2017-11-30T13:27:50.573" Title="What is $H^{\frac{1}{2}}(\Gamma_g)$" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28354" PostTypeId="2" ParentId="28353" CreationDate="2017-11-30T13:27:50.573" Score="2" Body="&lt;p&gt;Suppose that $\Omega$ is your domain and $\Gamma_g \subset \partial \Omega$. You can think of $H^{1/2}(\Gamma_g)$ as the space of $H^1(\Omega)$ functions restricted to $\Gamma_g$. So any function belonging to $u \in H^1(\Omega)$ has its trace, $u|_{\Gamma_g}$, in $H^{1/2}(\Gamma_g)$. Therefore you can use, for example, piecewise linear elements defined on the boundary (triangular $P_1$ element in your case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Piecewise constant elements do not work since such a function would belong to $H^{1/2-\varepsilon}(\Gamma_g)$ , $\varepsilon &amp;gt; 0$.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2017-11-30T13:27:50.573" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28358" PostTypeId="1" CreationDate="2017-11-30T15:25:13.140" Score="1" ViewCount="92" Body="&lt;p&gt;I apologise beforehand if the question isn't well defined or is too broad, I'm just having some difficulty finding the information I need. I also apologise if this should have been asked on stack overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have computed some numerical solutions using spectral methods to some different hydrodynamics problems that I'm interested in. The solutions are given as time series arrays. For example, at each time $t = 1, \dots, n,$ the solution for the velocity field $v$ is an $m \times m$ array.&#xA;I would like to analyse my system to determine as much information about it as possible. For example, in my 2D vortex simulations, I would like to numerically determine the dominant behaviour of the system, or numerically find/predict any splitting or merging of vortices, or perhaps find relationships between the different variables of the system etc. The problem is, I don't actually know of many computational methods that I can use to determine these kinds of behaviour. Some things I do know we can use (and have implemented) are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Autocorrelation functions (in both time and space).&lt;/li&gt;&#xA;&lt;li&gt;Singular value decomposition (to determine dominant behaviour).&lt;/li&gt;&#xA;&lt;li&gt;Analysing the spectra of the solution (radially averaged bins, FFT).&lt;/li&gt;&#xA;&lt;li&gt;Basic statistical analysis, analysing fluctuation data.&lt;/li&gt;&#xA;&lt;li&gt;Coupling tracer particles/dyes into the DE system to observe phenomena.&lt;/li&gt;&#xA;&lt;li&gt;Visualisation of the numerical solutions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Obviously, comparing the numerical solution with the analytical solution is also a useful technique, though most of the time this isn't possible in hydrodynamics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping that there were more numerical techniques available that I could learn and use to study my system. So I was wondering if someone could let me know of any papers or books that detail approaches to analysing numerical solutions, or could list down some methods that they know of. Any help would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 02/12&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was given a review paper today by &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.94.159&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jiang et al.&lt;/a&gt; which outlines nine numerical methods for detecing vortices, and also a paper by &lt;a href=&quot;https://www.cambridge.org/core/journals/journal-of-fluid-mechanics/article/on-the-identification-of-a-vortex/D26006DDB95FB28DA80E28A581182DF1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jeong et al.&lt;/a&gt;. It is mostly these types of algorithms/computational methods that I am interested in using.&lt;/p&gt;&#xA;" OwnerUserId="16852" LastEditorUserId="16852" LastEditDate="2017-12-03T02:23:55.017" LastActivityDate="2017-12-03T02:23:55.017" Title="Computational approaches to analyse numerical solutions" Tags="|fluid-dynamics|verification|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28359" PostTypeId="2" ParentId="28348" CreationDate="2017-11-30T18:07:58.247" Score="1" Body="&lt;p&gt;Yes, there is active research on that, especially in the Lyapunov case: it turns out under some conditions $M$ is well approximated by a low-rank or banded matrix. So you can find an implicit representation of the solution $M$ and then use it to compute matrix-vector products, in your case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good starting point is Valeria Simoncini's research; for instance, check &lt;a href=&quot;https://doi.org/10.1137/130912839&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Computational methods for linear matrix equations&quot; on SIAM review&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="713" LastEditDate="2017-11-30T19:15:20.487" LastActivityDate="2017-11-30T19:15:20.487" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28360" PostTypeId="2" ParentId="25399" CreationDate="2017-11-30T21:23:03.087" Score="1" Body="&lt;p&gt;Here are two more precise formulations of the problem which will allow for an approximation scheme as asked for by the OP. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let $A \in \mathbb{K}^{N \times N}$ be a sparse matrix and well-conditioned matrix and $b \in \mathbb{K}^N$ a sparse vector. Then we can associate $A$ with a graph $G = (V,E)$ where &#xA;$$&#xA;V := \{1, \ldots, N\}, \qquad&#xA;E := \{(i,j) \in V^2 \mid A(i,j) \neq 0\},&#xA;$$ &#xA;which in turns allows us to define a distance $d(i,j)$ between indices given by the minimal number of edges in $G$ we need to take in order to get from $j$ to $i$ (i.e., $d(i,j)$ is the graph distance in $G$). Then one can show&#xA;$$&#xA;|A^{-1}(i,j)| \leq C \, \exp\big(-\gamma \, d(i,j)\big)&#xA;\qquad \forall i,j \in V&#xA;$$&#xA;for some $\gamma &amp;gt; 0$, see e.g. &quot;Decay Rates for Inverses of Band Matrices&quot; by Demko, Moss and Smith (1984). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the above, it is clear that $x = A^{-1}b$ decays exponentially if we arrange its coefficients according to the graph distance from the non-zero entries of $b$, which is the ordering we assume for the following construction of an approximation $x_n = A_n^{-1} b$ with only $n \ll N$ degrees of freedom. Let $A_n$  be the matrix in which all interactions between entries $\leq n$ and $&amp;gt; n$ are dropped, i.e.&#xA;$$&#xA;A_n(i,j) := \begin{cases}&#xA;A(i,j) &amp;amp; \text{if } i,j \leq n \text{ or } i,j &amp;gt; n, \\&#xA;0 &amp;amp; \text{otherwise}.&#xA;\end{cases}&#xA;$$&#xA;It is clear that then $x_n(i) = 0$ for $i &amp;gt; n$, and hence it only remains to show that $x_n(i) \approx x(i)$ for $i \leq n$. This can be seen by expressing the error as &#xA;$$&#xA;x - x_n = (A^{-1} - A_n^{-1})\, b = A^{-1} \, (A_n - A) \, A_n^{-1} \, b. &#xA;$$&#xA;We now assume that $A_n^{-1}$ satisfies the same decay estimate as $A^{-1}$, which boils down to assuming that the conditioning of $A_n$ is not worse than that of $A$. This is trivial if $A$ is positive definite but may be hard to verify otherwise, yet it is an assumption you would probably want to make anyways if you want to numerically solve $A_n x_n = b$. Under this assumption, we estimate the error as &#xA;$$&#xA;|x(i) - x_n(i)|&#xA;\leq &#xA;C \, \sum_{k &amp;gt; n} \sum_{\ell \leq n} \sum_{j \in \mathrm{supp}(b)} A(k,\ell) \, \exp\big( - \gamma (d(i,k) + d(\ell,j)) \big),&#xA;$$&#xA;which is at most $C \, Nn|\mathrm{supp}(b)| \, \exp(-\gamma \, d(n+1,\mathrm{supp}(b))\big)$ and can be as small as $C \, Nn|\mathrm{supp}(b)| \, \exp(-2\, \gamma \, d(n+1,\mathrm{supp}(b))\big)$ for $i$ close to $1$. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if $A$ is not sparse but $x$ is known to be decaying for other reasons, we can still order the degrees of freedom such that $|x(i)|$ is decreasing for increasing $i$. We then define the projector onto the first $n$ degrees of freedom, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;P_n : \mathbb{K}^N \to \mathbb{K}^N, &#xA;\qquad&#xA;a \mapsto \begin{cases}&#xA;a(i) &amp;amp; \text{if } i \leq n,\\&#xA;0 &amp;amp; \text{otherwise},&#xA;\end{cases}&#xA;$$&#xA;and set $A_n := P_n A P_n$ and $x_n := A_n^\dagger b$, where $A_n^\dagger$ is the pseudo-inverse of $A_n$ satisfying $A_n^\dagger A_n = A_n A_n^\dagger = P_n$. Note that once again $x_n$ is formally an element of $\mathbb{K}^N$ but has only $n$ non-zeros. We can then write the error as $x_n - x = (x_n - P_n x) + (P_n x - x)$. The term $P_n x - x$ is easily estimated using the decay of $x$, and for the other term we get&#xA;\begin{aligned}&#xA;x_n - P_n x&#xA;&amp;amp;=&#xA;P_n \, (A_n^\dagger - A^{-1}) \, b&#xA;\\&amp;amp;=&#xA;A_n^\dagger \, (A - A_n) \, A^{-1} \, b&#xA;\\&amp;amp;= &#xA;A_n^\dagger \, (A - A_n) \, x&#xA;\\&amp;amp;= &#xA;A_n^\dagger \, A \, (x - P_n x)&#xA;\end{aligned}&#xA;and can hence be estimated as well by the projection error $x - P_n x$ if $A_n^\dagger$ and $A$ are bounded in suitable norms. &lt;/p&gt;&#xA;" OwnerUserId="21342" LastEditorUserId="21342" LastEditDate="2018-03-13T22:52:31.400" LastActivityDate="2018-03-13T22:52:31.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28361" PostTypeId="1" CreationDate="2017-11-30T23:40:52.107" Score="1" ViewCount="1237" Body="&lt;p&gt;We want to compute the relative condition number of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_1+x_2+x_3+\cdots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We assume all values are positive, and we will do a limit of a large $x_1=10^{8}$, and smaller values for all the other numbers $x_2=x_3=\cdots=10^{-3}$, as a worse-case scenario:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1 term&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_1) = x_1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the relative condition number is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa={\|J\|\|x\|\over \|f(x)\|}={1\cdot x \over x} = 1$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;2 terms&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_1, x_2) = x_1+x_2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we get in the infinity norm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa={\|J\|_\infty \|x\|_\infty\over \|f(x)\|}={2\cdot \max(x_1, x_2) \over x_1 + x_2} \rightarrow 2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which for $x_1=10^{8}$ and $x_2=10^{-3}$ approaches 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exact value is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [1]: x1 = 1e8; x2 = 1e-3; 2*max(x1, x2)/(x1+x2)&#xA;Out[1]: 1.99999999998&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;3 terms&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_1, x_2, x_3) = x_1+x_2+x_3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we get in the infinity norm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa={\|J\|_\infty \|x\|_\infty\over \|f(x)\|}={3\cdot \max(x_1, x_2, x_3) \over x_1 + x_2 + x_3} \rightarrow 3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which for $x_1=10^{8}$ and $x_2=x_3=10^{-3}$ approaches 3. The exact value is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [2]: x1 = 1e8; x2 = 1e-3; x3 = 1e-3; 3*max(x1, x2, x3)/(x1+x2+x3)&#xA;Out[2]: 2.99999999994&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;n terms&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The relative condition number for summing n-terms of the type:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$10^8 + 10^{-3} + 10^{-3} + \cdots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa=n$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that correct? So for large $n$, the sum of such values becomes ill-conditioned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the only way it becomes well-conditioned is if the magnitude of the numbers is similar, since then the condition number seems to stay low even for large $n$, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [3]: x1 = 1e-3; x2 = 2e-3; x3 = 3e-3; x4 = 4e-3; 4*max(x1, x2, x3, x4)/(x1+x2+x3+x4)&#xA;Out[3]: 1.6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="236" LastActivityDate="2020-05-12T08:34:21.527" Title="What is a relative condition number of a sum of positive values?" Tags="|numerical-analysis|condition-number|conditioning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28362" PostTypeId="2" ParentId="28361" CreationDate="2017-11-30T23:46:55.627" Score="1" Body="&lt;p&gt;The mistake I made is that the Jacobian in the infinity norm is equal to 1, not $n$. As such, the relative condition number of the sum is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\kappa=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and as such is always well-conditioned, no matter what magnitudes the values are.&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2017-11-30T23:46:55.627" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28363" PostTypeId="1" CreationDate="2017-12-01T06:08:28.297" Score="5" ViewCount="309" Body="&lt;p&gt;Let's say I'm given a system of three first-order differential equations in three variables, where all of the equations are known, and we additionally know the trajectory of two of the variables at a set of timepoints. Is there a fast algorithm to determine the starting value of the third variable, whose trajectory is unknown. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best I've been able to come up with so far randomly initializing the starting point of the third variable, and then using a numerical solver to determine the hypothetical trajectory that the two variables with known trajectory follow, and then optimizing on the sum of the difference of squares between the hypothetical trajectory and the actual trajectories of  these two variables. However, this can take a while to converge, which is why I'm looking for something faster. &lt;/p&gt;&#xA;" OwnerUserId="25363" LastEditorUserId="1804" LastEditDate="2017-12-01T09:53:23.227" LastActivityDate="2017-12-31T22:50:41.240" Title="Algorithm for finding initial conditions of differential equations given trajectory" Tags="|ode|inverse-problem|differential-equations|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28364" PostTypeId="2" ParentId="28363" CreationDate="2017-12-01T09:29:49.180" Score="0" Body="&lt;p&gt;Suppose your differential equations are:&#xA;$$&#xA;\dot{x} = f_x(x,y,z)\\&#xA;\dot{y} = f_y(x,y,z)\\&#xA;\dot{z} = f_z(x,y,z),&#xA;$$&#xA;and you know $x$ and $y$ at (ordered) times $t_1, t_2, …$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $t_{i+1}-t_i$ is sufficiently small for some $i$ (preferably all of them), you have two ways to estimate the derivative of the first variable at $:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{alignat*}{1}&#xA;\dot{x}\left( \frac{t_i+t_{i+1}}{2}\right )&#xA;&amp;amp;\approx \frac{x(t_{i+1})-x(t_i)}{t_{i+1}-t_i} \\&#xA;\dot{x}\left( \frac{t_i+t_{i+1}}{2}\right )&#xA;&amp;amp;\approx f_x \left(&#xA;  \frac{x(t_i)+x(t_{i+1})}{2},&#xA;  \frac{y(t_i)+y(t_{i+1})}{2},&#xA;  z\left( \frac{t_i+t_{i+1}}{2}\right )&#xA;\right).&#xA;\end{alignat*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you equate the right-hand sides, the only unknown in this equation is $z\left( \frac{t_{i+1}+t_i}{2}\right )$, so all you have to do is to solve it for this term. You can then apply the same approach to $y$ to get a second estimate and refine your result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(While I used the middle of the interval, you can apply the same approach to the beginning or end as well. This has less accuracy but may be more comfortable and even faster, as it saves one step in the following.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have this estimate for $z\left( \frac{t_{i+1}+t_i}{2}\right )$, you can integrate backwards and forwards to the known time points and refine your result. The best strategy here depends on the accuracy of your data and how your times $t_i$ are distributed. If $t_1$ is not your initial time, first obtain an accurate estimate of $z(t_1)$ and then integrate backwards to your initial time. This may not be much different from what you have been doing, but you should have a better initial guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be aware of the possibility that your backwards differential equation may be unstable (e.g., for strongly dissipative systems), in which case you may want to avoid backwards integrations as far as possible in the above (or only perform them for very short time steps), in which you are essentially back at your method. Also, if $t_1$ is much later than your initial time (with respect to instability time scales), you have the more fundamental problem that the solution to your problem is overly sensitive on measurement and numerical errors – every solution to your problem would also pose a method to backwards-integrate unstable differential equations.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2017-12-01T10:33:23.647" LastActivityDate="2017-12-01T10:33:23.647" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28368" PostTypeId="1" CreationDate="2017-12-02T10:17:55.863" Score="1" ViewCount="923" Body="&lt;p&gt;I have an array of size (254, 80) which I am trying to use Scipy's fsolve on. I have found that the speed of using fsolve on a vector is quicker than it is in a for loop but only for vectors upto about 100 values long. After this, the speed quickly drops off and becomes very slow, sometimes completely stopping. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently looping through one dimension of the array and using a vectorised fsolve on the smaller dimension but it's still taking longer than I would expect/like. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have a good work around for this or a know of a similar function which will be happy handling a vector of a larger size? I have tried Scipy's Newton Krylov function which, upon testing, is really fast for simple functions but it's slower than fsolve on my function. - Will this be faster if I use the analytical jacobian? If so, how do I implement it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wondered if it is a problem with my initial solutions but the difference is in the range of ~0.001 - 0.8 which doesn't seem too bad. Although perhaps it is this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the current code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(array.shape[0]):&#xA;    f = lambda y: a[i] - m[i]*y - md[i]*(( y**4 + 2*(y**2)*np.cos(Thetas[i,:]) )**0.25)&#xA;    ystar[i,:] = fsolve(f, y0[i]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Variables a, m and md are all the same size i.e. (254,80))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Digging in to this further, I have found that a function such as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = lambda y: y*np.tanh(y) - a0/(m**2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is faster to solve than &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = lambda y: (m**2)y*np.tanh(y) - a0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where m and a0 are large 2D np arrays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain why this is?&lt;/p&gt;&#xA;" OwnerUserId="26162" LastEditorUserId="9667" LastEditDate="2017-12-02T14:58:46.620" LastActivityDate="2017-12-02T14:58:46.620" Title="Vectorised root finding in Python" Tags="|numerical-analysis|python|newton-method|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28370" PostTypeId="1" AcceptedAnswerId="28371" CreationDate="2017-12-02T21:53:51.080" Score="4" ViewCount="2542" Body="&lt;p&gt;I am writing a code for steady state heat transfer on a rectangular domain. I am specifying temperature on the edges - nonzero Dirichlet boundary condition. The equations can be written in form of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$KT=Q$$&#xA;$K$ is conductivity matrix, $T$ is unknown nodal temperature vector, and $Q$ is thermal load vector consisting of internal heat generation. For example, the system can look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin {bmatrix} &#xA;K_{11} &amp;amp; K_{12} &amp;amp; K_{13} &amp;amp; \dots &amp;amp; K_{1N}\\&#xA;K_{21} &amp;amp; K_{22} &amp;amp; K_{23} &amp;amp; \dots &amp;amp; K_{2N}\\&#xA;\vdots\\&#xA;K_{N1} &amp;amp; K_{N2} &amp;amp; K_{N3} &amp;amp; \dots &amp;amp; K_{NN}&#xA;\end{bmatrix}&#xA;\begin {bmatrix}&#xA;100\\&#xA;200\\&#xA;\vdots\\&#xA;T_N&#xA;\end{bmatrix}&#xA;=&#xA;\begin {bmatrix}&#xA;Q_1\\&#xA;Q_2\\&#xA;\vdots\\&#xA;Q_N&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Non zero temperatures are applied on the edges of the rectangular domain (non-homogeneous Dirichlet boundary condition). How can I handle computationally non-zero Dirichlet boundary condition to solve for unknown temperature vector? &lt;/p&gt;&#xA;" OwnerUserId="25978" LastActivityDate="2017-12-02T22:28:36.960" Title="How to apply non zero Dirichlet boundary condition in finite elements?" Tags="|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28371" PostTypeId="2" ParentId="28370" CreationDate="2017-12-02T22:28:36.960" Score="6" Body="&lt;p&gt;Consider the fact that your linear system can be broken up into two parts: the equations that correspond to the known Dirichlet dofs, and the equations that correspond to the unknown dofs. For the sake of convenience, let's say that your unknown vector is broken up so the first $k$ values of $T$ are known, and the remainder are unknown. Let a &quot;$u$&quot; subscript denote the unknown indices of $T$, and let &quot;$d$&quot; denote the known (Dirichlet) indices in $T$. Then, the linear system can be written in the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\begin{matrix}&#xA;K_{dd}  &amp;amp;  K_{du} \\&#xA;K_{ud} &amp;amp; K_{uu}&#xA;\end{matrix}\right]&#xA;\left[&#xA;\begin{matrix} T_d \\ T_u \end{matrix}\right] = &#xA;\left[\begin{matrix} Q_d \\ Q_u \end{matrix}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the values of $T_d$ are known, we can immediately discard the equations in those rows, leading to this system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\begin{matrix}&#xA;K_{ud} &amp;amp; K_{uu}&#xA;\end{matrix}\right]&#xA;\left[&#xA;\begin{matrix} T_d \\ T_u \end{matrix}\right] = &#xA;\left[\begin{matrix} Q_u \end{matrix}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then move your known values over to the right-hand side of the linear system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;K_{uu}T_u = Q_u - K_{ud}T_d&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve this reduced set of linear equations, and you've got your unknowns.&#xA;Another technique to avoid mucking around with the original $K$ matrix too much is to zero out $K_{du}$ and $K_{ud}$, set $K_{dd}$ to the Identity matrix, and set $Q_d = T_d$. You still need to modify the entries of $Q_u$ in the same way as described above, but this removes the need to actually re-create the sparse matrix to only include the $K_{uu}$ block while still preserving the symmetry of the linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth mentioning that these techniques are still appropriate, even when the index sets &quot;$d$&quot; and &quot;$u$&quot; are non-contiguous. There's a slightly higher bookkeeping burden for you, but it's conceptually identical.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2017-12-02T22:28:36.960" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28372" PostTypeId="1" AcceptedAnswerId="28378" CreationDate="2017-12-03T12:17:19.423" Score="0" ViewCount="252" Body="&lt;p&gt;I was wandering - what kind of normalization does Matlab use in &lt;code&gt;hermiteH&lt;/code&gt;, its implementation of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hermite_polynomials&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hermite polynomials&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is certainly &lt;strong&gt;not&lt;/strong&gt; the case that they use either $\| H_n \|_{L^2 (e^{-x^2})}=1$, and they don't use $H_n(1)=1$ either.&lt;/p&gt;&#xA;" OwnerUserId="15188" LastEditorUserId="9667" LastEditDate="2017-12-03T15:54:31.337" LastActivityDate="2017-12-04T16:52:21.640" Title="Normalization of MATLAB HermiteH" Tags="|matlab|numerical-analysis|special-functions|" AnswerCount="1" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28376" PostTypeId="1" AcceptedAnswerId="28381" CreationDate="2017-12-03T21:26:31.520" Score="1" ViewCount="216" Body="&lt;p&gt;I am solving Ax=b. A has a very large condition number (&gt; O(10^10))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the conjugate gradients method with point jacobi pre-conditioning. I obtained a solution 'x' that &quot;looks&quot; reasonable. What is the best way to verify that my solution is correct using this method with preconditioning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I cannot obtain an exact solution to the original equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My matrix is symmetric, and I believe it is negative definite/semidefinite as at least one of the its eigenvalues is 0.&lt;/p&gt;&#xA;" OwnerUserId="23915" LastEditorUserId="23915" LastEditDate="2017-12-04T05:18:56.227" LastActivityDate="2021-04-08T16:40:06.147" Title="How to verify solution to pre-conditioned linear systems solver?" Tags="|linear-solver|preconditioning|conjugate-gradient|linear-system|jacobian|" AnswerCount="2" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="28377" PostTypeId="1" CreationDate="2017-12-04T02:07:22.457" Score="1" ViewCount="82" Body="&lt;p&gt;I'm interested in (locally) minimizing a smooth nonconvex objective function:&#xA;$$&#xA;f(\textbf{x}_1, \textbf{y}_1,\cdots, \textbf{x}_n, \textbf{y}_n)=\sum_{i=1}^ng(\textbf{x}_i, \textbf{y}_i)&#xA;$$&#xA;Subject to $\textbf{y}_{i+1}=\textbf{h}(\textbf{x}_i, \textbf{y}_i)$ for each $i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the pros and cons of using the penalty method vs Augemented Lagrangian for this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Penalty method. Until convergence:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Minimize wrt $\textbf{x}_i, \textbf{y}_i$ the objective $f+\sum_i\nu_i\|\textbf{y}_{i+1}-\textbf{h}(\textbf{x}_i, \textbf{y}_i)\|$ with gradient descent.&lt;/li&gt;&#xA;&lt;li&gt;Update  $\nu_i\gets\nu_i+\alpha\|\textbf{y}_{i+1}-\textbf{h}(\textbf{x}_i, \textbf{y}_i)\|^2$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Augmented Lagrangian method. Until convergence:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Minimize wrt $\textbf{x}_i, \textbf{y}_i$ the objective $f+\sum_i\boldsymbol\lambda_i\cdot(\textbf{y}_{i+1}-\textbf{h}(\textbf{x}_i, \textbf{y}_i))+\sum_i\nu_i\|\textbf{y}_{i+1}-\textbf{h}(\textbf{x}_i, \textbf{y}_i)\|^2$ with gradient descent.&lt;/li&gt;&#xA;&lt;li&gt;Update $\boldsymbol \lambda_i\gets \boldsymbol\lambda_i+\alpha(\textbf{y}_{i+1}-\textbf{h}(\textbf{x}_i, \textbf{y}_i))$ and $\nu_i\gets\nu_i+\alpha\|\textbf{y}_{i+1}-\textbf{h}(\textbf{x}_i, \textbf{y}_i)\|^2$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="23164" LastActivityDate="2017-12-04T02:07:22.457" Title="What are the numerical properties to consider between Augmented Lagrangian and the Penalty Method?" Tags="|constrained-optimization|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28378" PostTypeId="2" ParentId="28372" CreationDate="2017-12-04T16:52:21.640" Score="3" Body="&lt;p&gt;Several packages (MATLAB, Mathematica, SciPy, mpmath) have implemented the physicist Hermite polynomials, they are defined by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H_n(x)=(-1)^n e^{x^2}\frac{\mathrm d^n}{\mathrm dx^n}e^{-x^2}=\left (2x-\frac{\mathrm d}{\mathrm dx} \right )^n \cdot 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are not normalized, the norm should be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$||H_n(x)||_{L^2(e^{-x^2})} = 2^n \sqrt{\pi}  n!\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These polynomials are orthogonal with respect to the weight $e^{-x^2}$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{-\infty}^\infty H_m(x) H_n(x)\, e^{-x^2}\, \mathrm{d}x = 2^n \sqrt{ \pi} n! \delta_{nm}\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to check this orthogonality/normalization numerically you need to be careful. On one hand, you have an infinite interval. As you say, you can make the integration region &quot;big-enough&quot;, but you have a caveat: the maximum value of the polynomials grow really fast, and you will need to increase the size of the region, really fast, as well. See, for example, the result of using the trapezoidal rule for the orthogonality between $H_0$ and $\{H_2, H_4, H_6, H_8\}$ using the interval [-3, 3] as you propose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Cbdyo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Cbdyo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can increase the integration interval to, say, [-10, 10] to obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dlsiu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dlsiu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we see that all the considered polynomials are &quot;orthogonal&quot;. Nevertheless, you have another complication when you increase the order of the polynomials: the number of changes of sign increases. For that you need to consider a special type of quadrature or use something like &lt;a href=&quot;http://www.chebfun.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chebfun&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2017-12-04T16:52:21.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28379" PostTypeId="1" CreationDate="2017-12-04T18:48:39.740" Score="0" ViewCount="364" Body="&lt;p&gt;I have a matrix of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VODWE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VODWE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I would like to invert it. Currently I am using the lapack routines zgetrf and zgetri. I.e. I am performing LU factorisation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: Are there any packaged routines (lapack or otherwise) that invert banded matrices? I have found routines for tridiagonal and block diagonal matrices but little else.&lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2017-12-04T20:56:50.587" Title="Are there any packaged routines (in lapack or elsewhere) for inverting a banded matrix?" Tags="|lapack|banded-matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28380" PostTypeId="2" ParentId="28379" CreationDate="2017-12-04T20:56:50.587" Score="4" Body="&lt;p&gt;Well, other than the usual &quot;don't invert your matrices unless you need the inverse itself&quot; you can still use the banded routines &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d2/d2d/group__double_g_bcomputational_ga7fc91ba3f250ad3844eba25d59f5d7be.html#ga7fc91ba3f250ad3844eba25d59f5d7be&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;?gbtrf&lt;/code&gt;&lt;/a&gt; and then use &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d2/d2d/group__double_g_bcomputational_ga37ca5bf29c33e026922c06b3bf188513.html#ga37ca5bf29c33e026922c06b3bf188513&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;?gbtrs&lt;/code&gt;&lt;/a&gt; with the right hand side being an identity matrix.&lt;/p&gt;&#xA;" OwnerUserId="21617" LastActivityDate="2017-12-04T20:56:50.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28381" PostTypeId="2" ParentId="28376" CreationDate="2017-12-04T21:44:53.630" Score="5" Body="&lt;p&gt;You've started with a singular linear system of equations $Ax=b$.  As a practical matter, it's unlikely that $b$ lies exactly in the range of $A$, so at best you can find a least squares solution that minimizes &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| Ax - b \|_{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the system is singular, the null space of $A$ is non-empty, and there will be an infinite number of solutions to this least squares problem.  In particular, if $x_{LS}$ is some particular least squares solution and $v$ is any vector in the null space of $A$, then any $x$ of the form $x=x_{LS}+\alpha v$ will be a least squares solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which of these infinitely many least squares solutions is the &quot;correct&quot; one?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps you don't care and will accept any least squares solution.  In that case, any solution that satisfies the normal equations $A^{T}Ax=A^{T}b$ will be correct.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps you want to simultaneously minimize the norm of $x$ to get a minimum norm least squares solution?   You can regularize the least squares problem by minimizing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \| Ax-b \|_{2}^{2} + \lambda^{2} \| x \|_{2}^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\lambda$ is a small parameter.  This least squares problem is strictly convex, so it has a unique optimal solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens in floating point arithmetic with limited precision?  In that case, it's likely that $A$ will have a very large but still finite condition number.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can throw a preconditioner $M$ at $A$, in hopes that $M^{-1}A$ will be better conditioned than $A$.  This might work in the sense at $M^{-1}A$ is numerically better conditioned than $A$, but since we know that $A$ is theoretically singular, this is a mirage! $M^{-1}A$ should still be singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is the solution to the numerically well-conditioned system $M^{-1}Ax=M^{-1}b$ doing?  This is similar to regularization in the sense that $M^{-1}A$ is better conditioned than $A$, but you don't know much about the direction in which the preconditioner has pushed the solution.  If $M^{-1}Ax-M^{-1}b$ is small, and $M^{-1}$ is well behaved, then perhaps $Ax-b$ is small and you've got a least squares solution.  On the other hand, if $M^{-1}$ is very badly scaled, it may be that the residual in the original problem $Ax-b$ is quite large.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been some research on preconditioners that are designed to achieve a particular regularization.  See for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calvetti, Daniela. 2007. “Preconditioned Iterative Methods for Linear Discrete Ill-Posed Problems from a Bayesian Inversion Perspective.” Journal of Computational and Applied Mathematics, Special Issue: Applied Computational Inverse Problems, 198 (2):378–95. &lt;a href=&quot;https://doi.org/10.1016/j.cam.2005.10.038&quot; rel=&quot;noreferrer&quot;&gt;https://doi.org/10.1016/j.cam.2005.10.038&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-12-04T21:44:53.630" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28382" PostTypeId="2" ParentId="28269" CreationDate="2017-12-04T22:26:32.920" Score="5" Body="&lt;p&gt;According to [1]: &quot;However, the IEEE-754 standard specifies nothing for elementary functions&quot; and &quot;Indeed, the mathematical libraries (libm) provided by operating systems do not guarantee correct rounding.&quot;. As such, according to [1], the answer is that the IEEE-754 standard does &lt;em&gt;not&lt;/em&gt; require the &lt;code&gt;exp2&lt;/code&gt; function to be correctly rounded, it only recommends it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] de Dinechin, F., Ershov, A. V., &amp;amp; Gast, N. (2005). Towards the Post-Ultimate libm. In 17th IEEE Symposium on Computer Arithmetic (ARITH’05) (pp. 288–295). IEEE. &lt;a href=&quot;http://doi.org/10.1109/ARITH.2005.46&quot; rel=&quot;noreferrer&quot;&gt;http://doi.org/10.1109/ARITH.2005.46&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="236" LastActivityDate="2017-12-04T22:26:32.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28383" PostTypeId="1" CreationDate="2017-12-04T23:28:57.717" Score="-3" ViewCount="71" Body="&lt;p&gt;Modulus of a (discretized) function, $|f_h(x)|$, where $h$ refers to the mesh spacing, would, in general, have zeros, and those zeros would not necessarily lie exactly at the mesh points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A naive &lt;code&gt;abs&lt;/code&gt; call on such a function, &lt;code&gt;abs(f_h)&lt;/code&gt;, is a very bad representation of such a modulus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is especially relevant when a quadrature of such a result is attempted. The most sensible remedy I can think of is something like a &lt;em&gt;modulus-aware quadrature&lt;/em&gt;, i.e., you can no longer treat the 'integral of the modulus of a function' as a two step process, a modulus, followed by an integration. A modulus-aware quadrature would involve detection of discrete intervals where zeros are present (discrete root-finding), and computation of the quadrature over those intervals differently, compared to intervals where there are no sign changes. (And very likely &lt;em&gt;modulus-aware computation&lt;/em&gt; is relevant for non-quadrature purposes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the importance of such an operation, in computational error analysis at the minimum, there are two serious issues in relation to this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The notation $\int_{\Omega} |f_h(x)| dx$, as well as $h \sum |f_h|$, is not only wrong, it's misleading to students. This notation is telling a student that you can treat the integral of a modulus on a discretized function (the discretization denoted by the presence of '$h$' in the subscript) as a two step process. You cannot blame a student when he/she looks at that notation and goes ahead and tries to do a &lt;code&gt;quadrature(abs(f_h))&lt;/code&gt; or &lt;code&gt;sum(abs(f_h))&lt;/code&gt;. This shoehorning of mathematical notation, meant for continuous analysis, into discrete analyses, should be nothing short of a crisis in computational science, which, unfortunately, is treated as business-as-usual as far as I can tell. (and since it has direct relevance to $L_2$ and similar norm calculations, the addition of a squaring operation not making things any better, I wonder about the percentage of computational scientists out there going about their daily lives doing an incorrect error and norm analysis without realizing it).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The discussion of &lt;em&gt;modulus-aware quadrature&lt;/em&gt; in texts (e.g., the one that I have, Heath, &lt;em&gt;Scientific Computing&lt;/em&gt;), as well as computational libraries (e.g., MATLAB, python scientific stack, etc) is either completely absent, or seriously lacking.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Not to single out, but as a case in point, Nico Schlomer has developed a great quadrature library for python &lt;a href=&quot;https://github.com/nschloe/quadpy&quot; rel=&quot;nofollow noreferrer&quot;&gt;quadpy&lt;/a&gt;. If you take a look at the front-page (readme.md) of that github project, you would be truly impressed with the level of depth and breadth of the available schemes, to say the least. Except that you wouldn't find a single mention of the issue I described above. All the greatness of such a quadrature library would be fairly useless to someone trying to compute a simple modulus-aware quadrature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, why is there so much lack of serious treatment of modulus-aware quadrature in numerical analysis as well as computational science circles?&lt;/p&gt;&#xA;" OwnerUserId="15198" LastEditorUserId="15198" LastEditDate="2017-12-05T02:07:32.977" LastActivityDate="2017-12-05T02:07:32.977" Title="Modulus (absolute) of a function, its quadrature, and relevance of zeros" Tags="|quadrature|error-estimation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28384" PostTypeId="2" ParentId="28383" CreationDate="2017-12-05T02:01:08.773" Score="2" Body="&lt;p&gt;As I've already pointed out in a comment, the drama and calling out by name of people (who are in fact quite respected in the community) is entirely uncalled for in this forum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think that, in the face of this, I want to go into great detail of why the approach everyone uses is completely appropriate. Suffice it to say that while $|f_h|$ is indeed a poorly behaved quantity, the error is in fact computed through integrals of the form&#xA;$$&#xA;  e = \left(\int |u-u_h|^2\right)^{1/2}.&#xA;$$&#xA;In other words, one takes the square of the absolute value. This is a quantity that does not have a kink and is in fact well-behaved. Nothing terrible will happen if you integrate it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-12-05T02:01:08.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28386" PostTypeId="2" ParentId="18782" CreationDate="2017-12-05T10:13:09.743" Score="0" Body="&lt;p&gt;Actually, the density never becomes zero because of self-contribution (i=j). However, you need neighboring particles, unless the sampling of mass density is poor. The optimal number of neighbors is around 20 in 2D and 50 in 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also check the discussion about the calculation of the desired smoothing length &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16363/how-to-choose-h-in-sph/27745#27745&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20475" LastEditorUserId="20475" LastEditDate="2017-12-05T10:33:56.027" LastActivityDate="2017-12-05T10:33:56.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28387" PostTypeId="2" ParentId="28271" CreationDate="2017-12-05T14:17:23.067" Score="7" Body="&lt;p&gt;A good example is if you use bilinear finite elements for the Laplace equation on a uniform mesh, and then approximate the integrals using the trapezoidal rule, then you get the usual 3-point stencil (in 1d) or 5-point stencil (in 2d) that is well known from finite difference methods. The only difference is that both the left and right hand side of the linear system are multiplied by the square of the mesh size, but other than that, the linear system is exactly the same as the one you'd get from a finite difference method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-12-05T14:17:23.067" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28388" PostTypeId="1" AcceptedAnswerId="28389" CreationDate="2017-12-05T15:56:30.107" Score="10" ViewCount="584" Body="&lt;p&gt;I know that this is not really apart of the rules as this is a recommendation question and these don't really have an answer per say. But, like this forum posting: &lt;a href=&quot;https://stackoverflow.com/questions/388242/the-definitive-c-book-guide-and-list&quot;&gt;https://stackoverflow.com/questions/388242/the-definitive-c-book-guide-and-list&lt;/a&gt;. To be honest, I am not really sure where else to ask this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to start a book guide list on Finite Element and meshing. The thing is, I am not doing this as university research nor as job research. I am interested in this topic and have been for some years. I would like to learn this on my own time. I have been working on building my own simulator for non-linear electromagnetic simulations in 2-D and  I am currently using gmsh as the mesher. I am currently working on integrating gmsh into my source. Progress has been positive. I am able to create a mesh with gmsh where the source code is directly integrated into my project. I would like to modify the current workflow for meshing which means that I need to code to work around GMSH limitations. THis also means, I need to have an understanding of creating a numerical grid for simulations so that I can navigate the source of GMSH and have a better understanding of what is going on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am finding that I have a lack on this aspect (numerical grid generation) since the source is referring to alot of terms that I am not familiar with. I apologize if this is breaking any forum rules. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, I was wondering if anyone can point me to any resources about numerical grid generation? What would be some good references to use for introductory, beginner, intermediate, and advanced? Right now, I feel like I am at beginner. I currently have the resource &quot;Numerical Grid Generation: Foundations and Applications by Joe F. Thompson&quot;. Would this resource be good as a starting point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I will definitely be going through the deal.II tutorials/manual as the documentation is very detailed. I like that it doesn't quite overload the reader with technical details but explains it with simplicity but to the point)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side note, I think that there should be a section for videos. Sometime, those can be helpful.&lt;/p&gt;&#xA;" OwnerUserId="18230" LastEditorUserId="18230" LastEditDate="2017-12-06T20:46:11.417" LastActivityDate="2020-11-26T00:28:05.373" Title="Resources on mesh generation for finite element methods" Tags="|finite-element|reference-request|mesh-generation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28389" PostTypeId="2" ParentId="28388" CreationDate="2017-12-05T17:21:18.547" Score="5" Body="&lt;p&gt;Starting a community answer, in line with your model question&lt;/p&gt;&#xA;&lt;h1&gt;Mesh Generation&lt;/h1&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://onlinelibrary.wiley.com/book/10.1002/9780470611166&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mesh Generation: Application to Finite Elements&lt;/a&gt; (P.-L. George and P. Frey) Hermes, Lyon, 2000. A clear primer on the core technology and terminology of mesh generation. Written in a fairly mathematical style, which might not appeal to those of a more practical outlook. Will fit well with Gmsh, due to the cross-pollination of authors and project.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.routledge.com/Finite-Element-Mesh-Generation/Lo/p/book/9781138749245&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite Element Mesh Generation&lt;/a&gt; (Daniel S.H. Lo), CRC, 2015. This book also covers several topics related to mesh generation for finite elements. The style is less mathematical-oriented in comparison with the previous one. The book has code for several algorithms in C++ or Fortran, but they could be better formatted.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://people.eecs.berkeley.edu/%7Ejrs/meshbook.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Delaunay Mesh Generation&lt;/a&gt; (Cheng, Dey, Shewchuk), CRC, 2012. Covers some of the really challenging implementation details for Delaunay meshing in 2D and 3D, like exactly how all the edge and face flips work, sliver exudation, etc.&lt;/p&gt;&#xA;&lt;h1&gt;Geometry and Topology&lt;/h1&gt;&#xA;&lt;p&gt;Edelsbrunner, Herbert. &lt;a href=&quot;https://books.google.com.co/books?id=v6BybEYVGqYC&amp;amp;lpg=PR11&amp;amp;ots=nurp-0Dfnh&amp;amp;dq=Geometry%20and%20Topology%20for%20Mesh%20Generation&amp;amp;lr&amp;amp;pg=PR11#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geometry and topology for mesh generation&lt;/a&gt;. Vol. 7. Cambridge University Press, 2001. This book concentrates on creation of meshes of triangles and tetrahedra, and the mathematics behind them.&lt;/p&gt;&#xA;&lt;p&gt;Joseph O'Rourke. &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0521649765&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Geometry in C&lt;/a&gt;. A book that gives overview of different geometry algorithms such as point in polygon.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="3481" LastEditDate="2020-11-26T00:28:05.373" LastActivityDate="2020-11-26T00:28:05.373" CommentCount="1" CommunityOwnedDate="2017-12-05T17:21:18.547" ContentLicense="CC BY-SA 4.0" />
  <row Id="28391" PostTypeId="1" AcceptedAnswerId="28395" CreationDate="2017-12-05T22:50:31.450" Score="0" ViewCount="66" Body="&lt;p&gt;I suspect that a function $f(x,y)$ is of the form $f(x,y)=a(bx+c)^{dy+e}$. I have access to several values of $f(x,y)$. How do I proceed numerically to find the parameters $\{a,b,c,d,e\}$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By plotting $\log f$ versus $\log y$ for a fixed $x$, I would get a linear curve and this would give me hope to find $\{d,e\}$. But getting $\{b,c\}$ is beyond me right now.&lt;/p&gt;&#xA;" OwnerUserId="26242" LastEditorUserId="9667" LastEditDate="2017-12-06T00:04:29.433" LastActivityDate="2017-12-06T05:31:58.683" Title="Finding parameters numerically" Tags="|regression|least-squares|curve-fitting|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28392" PostTypeId="1" CreationDate="2017-12-06T01:08:19.160" Score="5" ViewCount="264" Body="&lt;p&gt;The abscissas for Gauss quadrature are given by the zeros of the Legendre polynomials. The Legendre polynomials form an orthogonal set over &lt;span class=&quot;math-container&quot;&gt;$[-1, 1]$&lt;/span&gt;, and it is shown in (for instance) &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1461268338&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kress&lt;/a&gt; that the zeros of a set of orthogonal polynomials give the abscissas for Gauss-type quadrature schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this result hold in higher dimensions? In particular, I am interested in where the zeros of the Zernike polynomials can be used to generate a 2D Gaussian-type quadrature scheme over the unit disk. If so, references are appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Since the answers have been posted, a new paper presents results in this direction, see &lt;a href=&quot;https://arxiv.org/pdf/1811.02720.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2019-01-30T20:52:11.210" LastActivityDate="2019-01-30T20:52:11.210" Title="Building Gaussian-type quadrature schemes with Zernike polynomials" Tags="|quadrature|polynomials|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28394" PostTypeId="2" ParentId="28392" CreationDate="2017-12-06T01:52:56.927" Score="5" Body="&lt;p&gt;Unfortunately the 1D reasoning will not directly carry into 2D and beyond. Depending on the domain for your integral, you will have considerable latitude in the node choice - but of course the quadrature weights can be generated from polynomials as usual. The closest you could achieve to a classic Gaussian-type rule is to coordinate-transform your domain to a product of 1D intervals and construct the rule by tensor product and coordinate transform back to get a rule for the domain of interest. This will generally not be an exact for polynomials, but should still retain good convergence rates provided the Jacobian of the coordinate transformation is well behaved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put briefly: in 1D you have a nice alignment of many goals for a quadrature rule in that they can be simultaneously achieved with a single rule. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;semi-analytic expression for the rule (e.g. zeros of orthogonal polynomials)&lt;/li&gt;&#xA;&lt;li&gt;Stability of rule (positive weights)&lt;/li&gt;&#xA;&lt;li&gt;Symmetry properties&lt;/li&gt;&#xA;&lt;li&gt;Fast convergence (e.g. exact on polynomials below a certain order)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It turns out that in 1D, all four of these conditions may be satisfied simultaneously with Gaussian quadrature rules. In 2D and beyond one must generally trade between the above features. For example often the fastest converging rules (those satisfying criterion (4)) can be incredibly expensive to compute, and often fail in (2) and (3). Rules that satisfy (1) (such as my example using coordinate transformations) often fail (2),(3) and (4). &lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2017-12-09T02:45:13.823" LastActivityDate="2017-12-09T02:45:13.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28395" PostTypeId="2" ParentId="28391" CreationDate="2017-12-06T04:52:13.910" Score="2" Body="&lt;p&gt;If it is it OK to recommend a solution that uses Python and SciPy, you can use the &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;curve-fit&lt;/code&gt;&lt;/a&gt; function from scipy to do the job.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.optimize as op&#xA;import numpy as np&#xA;&#xA;def func(xydata,a,b,c,d,e):&#xA;    x = xydata[:,0]&#xA;    y = xydata[:,1]&#xA;    return a*(b*x + c)**(d*y + e)&#xA;&#xA;# Assuming you have your f(x,y) data in a file with&#xA;# three columns separated by comma e.g. x1,y1,f1&#xA;data = np.loadtxt('data.txt',delimiter=',')&#xA;xydata = data[:,0:2] # the first two columns&#xA;fdata = data[:,2] # the last column&#xA;&#xA;# Use scipy to calculate the optimum parameter values&#xA;p_opt, p_conv = op.curve_fit( func, xydata, fdata )&#xA;print(popt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user26248" LastActivityDate="2017-12-06T04:52:13.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28396" PostTypeId="2" ParentId="28391" CreationDate="2017-12-06T05:31:58.683" Score="5" Body="&lt;p&gt;In general, you can formulate this as a nonlinear least squares problem.  If your values are known at points $(x_{i},y_{i})$, and the known values are $f_{i}$, then you can minimize&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min_{a,b,c,d,e} \sum_{i=1}^{n} \left( f_{i} - a(bx_{i}+c)^{dy_{i}+e} \right)^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Levenberg-Marquardt method is commonly used to solve nonlinear least squares problems like this.  You can find implementations in many software packages.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2017-12-06T05:31:58.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28397" PostTypeId="1" AcceptedAnswerId="28404" CreationDate="2017-12-06T08:43:04.540" Score="1" ViewCount="171" Body="&lt;p&gt;I'm trying to solve a problem where I have a initial and final distribution of tumor, and my goal is to find the best map of parameters (diffusion and reaction terms) for a reaction-diffusion equation, which explain at best the final distribution, knowing the initial one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By doing some literature search, I figured out that the concept I'm dealing with is the so called &quot;pde-constrained optimization problem&quot; and I saw some works using things like &quot;adjoint variables&quot;, Lagrangian, etc... that I'm not familiar with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I can't find any good tutorial, online course, or article explaining these concepts and how to use them in practice. Maybe some of you can give me insights about such lesson available ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a fairly background level in mathematics and programmation, so I'm not looking for only understanding &quot;roughly&quot; the field, but indeed being able to apply solutions to my real-life work...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thank for any help you can give me...&lt;/p&gt;&#xA;" OwnerUserId="25963" LastEditorUserId="25963" LastEditDate="2017-12-25T08:09:01.873" LastActivityDate="2017-12-25T08:09:01.873" Title="pde-constrained optimization" Tags="|optimization|pde|reference-request|constrained-optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28398" PostTypeId="2" ParentId="28392" CreationDate="2017-12-06T09:35:50.343" Score="5" Body="&lt;p&gt;Yes, one can extend the Gaussian quadrature (in one dimension) to multiple dimensions. Often this is called cubature. Much work on this has been done by Ronald Cools and collaborators (see &lt;a href=&quot;http://nines.cs.kuleuven.be/ecf/&quot; rel=&quot;noreferrer&quot;&gt;his Encyclopedia of Cubature Formulae&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A nice paper for your application (integration on the unit disk) is &lt;a href=&quot;https://link.springer.com/article/10.1007/BF03012263&quot; rel=&quot;noreferrer&quot;&gt;A survey of known and new cubature formulas for the unit disk&lt;/a&gt;. It explains how to generate cubature formulas and provides links to sets of nodes/weights for different degrees. Especially the reference &lt;a href=&quot;https://link.springer.com/article/10.1007/BF03011388&quot; rel=&quot;noreferrer&quot;&gt;Symmetric quadrature formulas over a unit disk&lt;/a&gt; mentioned therein is of use for you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the online article by &lt;a href=&quot;http://www.holoborodko.com/pavel/numerical-methods/numerical-integration/cubature-formulas-for-the-unit-disk/&quot; rel=&quot;noreferrer&quot;&gt;Pavel Holoborodko&lt;/a&gt; is good introductory read. He explains the two approaches: direct derivation and product rule (as mentioned by Reid Atcheson). &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2017-12-06T09:35:50.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28400" PostTypeId="1" AcceptedAnswerId="28402" CreationDate="2017-12-06T13:18:06.167" Score="3" ViewCount="304" Body="&lt;p&gt;I am solving the Dirichlet problem&#xA;$$&#xA;  \begin{cases}&#xA;   \Delta u = 0, \\&#xA;   u|_{\partial D} = f,&#xA;  \end{cases}&#xA;$$&#xA;in a $2d$ domain $D$ using the finite element method. What I want to get is the normal derivative of the solution $u$ on the boundary: $\tfrac{\partial u}{\partial \nu}$. In other words, I'm interested in FEM implementation of the Dirichlet-to-Neumann (Poincare-Steklov) map $\Lambda$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hovewer, I could not find an explicit answer to this problem in Google. I'm programming in Matlab and I would extremely appreciate an optimized solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to solve this problem by formulating it in a weak form: find $h \in L^2(\partial D)$ such that for any $w \in H^{\frac 1 2}(\partial D)$,&#xA;$$&#xA;   \int_{\partial D} hw \,dl = \int_D \nabla w \nabla u dS.&#xA;$$&#xA;As usually, I represent $h = \sum' h_j \psi_j$, $w = \sum w_i \psi_i$, $u = \sum u_l \psi_l$, where $\psi_i$ are the basis elements and $\sum'$ denotes summation over the boundary elements only, and I get&#xA;$$&#xA;  \sum_i' \sum_j' h_j w_i \int_{\partial D} \psi_i \psi_j dl =  \sum_k \sum_l w_k u_l \int_D \nabla \psi_k \nabla \psi_l dS.&#xA;$$ &#xA;Since $\{w_j\}$ are arbitrary, we get the system&#xA;$$&#xA;   \sum_j' \int_{\partial D} \psi_i \psi_j dl \cdot h_j = \sum_l\int_D \nabla \psi_k \nabla \psi_l dS \, \cdot u_l.&#xA;$$&#xA;Introducing the matrices&#xA;$$&#xA;   B_{ij} = \int_{\partial D} \psi_i \psi_j \, dl, \quad A_{ij} = \int_D \nabla \psi_i \nabla \psi_j dS, &#xA;$$&#xA;where in the first matrix $i$, $j$ correspond to boundary elements and in the second matrix $i$ corresponds to the boundary elements and $j$ corresponds to all elements, I get the matrix equation&#xA;$$&#xA;  B h = Au,&#xA;$$&#xA;for finding $h$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I implement this approach, I get some nonsense. For example, consider the following circular domain and boundary values $u_0(x,y) = -y$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DwKh9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DwKh9.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the flux along the boundary must coincide with $u_0(x,y)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to compute the flux of with the equations used above. However I was wondering what functions to use for $h$? If I use piecewise constant on elements, I get that the $B$ matrix has rank 1 too small (because its a circle), but how can one know this beforehand? (I tried to adjoin this with that the total flux must be $0$ but it gives wrong results). If I try to use a continous at vertices and linear on elements function, I get a flux which misses a factor of $1.4$. If I try to use discontinous but linear elements the $B$ matrix has more columns then rows.  For $w$ I used continous piecewise linear elements which are $1$ only at nodes on the boundary, is this right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the precise elements to use for all the functions, and can someone write a pseudo code for this problem?&lt;/p&gt;&#xA;" OwnerUserId="24446" LastEditorUserId="24446" LastEditDate="2017-12-06T14:16:06.397" LastActivityDate="2017-12-07T01:41:48.863" Title="FEM current toy problem" Tags="|finite-element|matlab|boundary-conditions|elliptic-pde|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28401" PostTypeId="1" CreationDate="2017-12-06T16:29:08.620" Score="1" ViewCount="622" Body="&lt;p&gt;I am trying to solve the following equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \partial_t g(x,y,t) = - v\left(\partial_x + \partial_y\right) g(x,y,t)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;using finite differences (here $v&amp;gt;0$). The equation is also solvable analytically, and every function of the form $g(x,y,t) = g(x-vt,y-vt)$ is a solution. So, an initial &quot;shape&quot; $g(x,y,0)$ will simply translate along the $(1,1)$ direction in the x-y plane for increasing times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the following scheme, with backward differences for space and forward differences for time,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$g(i,j,n+1) = g(i,j,n) - v\;dt\frac{ \displaystyle g(i,j,n) - g(i-1,j,n) + g(i,j,n) - g(i,j-1,n)}{\displaystyle dx} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $dx = 2\;v\;dt$. I typically use a 2D gaussian pulse as initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scheme works fairly well, but the amplitude of the pulse decreases as the pulse moves. Typically I lose about 10% of amplitude when the pulse moves over a distance of about $L=10^4 v\;dt$. This improves a bit (i.e., I have less decrease of the pulse amplitude) when I increase the time and spatial resolutions, but the increase in accuracy seems quite &quot;slow&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know if I can optimize my scheme in any way (other than increasing resolution)? Would it help using a central difference scheme for the time?&lt;/p&gt;&#xA;" OwnerUserId="25268" LastEditorUserId="21418" LastEditDate="2018-01-08T19:11:20.240" LastActivityDate="2018-01-08T19:11:20.240" Title="Advection equation in 2D using finite differences - the scheme works, but the pulse loses &quot;energy&quot;" Tags="|finite-difference|advection|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28402" PostTypeId="2" ParentId="28400" CreationDate="2017-12-07T01:41:48.863" Score="6" Body="&lt;p&gt;The formulation of this problem is tricky. Here is what you have in your original post:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find $h \in L^2(\partial D)$ such that for any $w \in H^{\frac 1 2}(\partial D)$,&#xA;$$&#xA;   \int_{\partial D} hw \,dl = \int_D \nabla w \nabla u dS.&#xA;$$&#xA;This already can't be quite right because you have $w$ on the right in a domain integral, so it cannot be a function $w \in H^{\frac 1 2}(\partial D)$ that only lives on the boundary. Rather, you need to consider all test functions $w\in H^1(D)$ for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, this means that on the left you have that the functions $h,w$ are from different spaces: you have &quot;many more&quot; test functions $w$ than solution functions $h$. If you discretized this, you'd get a rectangular matrix with more rows than columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, in practice the &quot;vast majority&quot; of $H^1$ functions $w$ are in fact zero on the boundary. Indeed, the set of test functions $w\in H^1$ whose traces $w|_{\partial D}$ are linear independent (i.e.,for which the corresponding rows of the matrix on the left will be linearly independent) can all be written as the &lt;em&gt;extension&lt;/em&gt; of $H^{1/2}(\partial D)$ functions into the interior. How exactly one extends into the interior does not actually matter; let us just say that $Ew \in H^1(D)$ is an extension of $w\in H^{1/2}(\partial D)$ so that $Ew|_{\partial D}=w$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the &lt;em&gt;now correct&lt;/em&gt; problem formulation is as follows:&#xA;Find $h \in H^{-1/2}(\partial D)$ such that for any $w \in H^{\frac 1 2}(\partial D)$,&#xA;$$&#xA;   \int_{\partial D} hw \,dl = \int_D \nabla (Ew) \cdot \nabla u dS.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is now the point where you can discretize. To this end, let $V_h \subset H^{1/2}(\partial D)$ be a finite dimensional subspace, for example consisting of piecewise constant functions on the boundary relative to the faces of finite element mesh of $D$. Then, if $\{\psi_j\}_{j=1}^N$ is a basis of $V_h$, you would get the following linear system for the finite element approximation $h_h = \sum_j H_j \psi_j$:&#xA;$$&#xA;   \sum_j \left\{ \int_{\partial D} \psi_i \psi_j dl \right\} H_j = \int_D \nabla (E\psi_i) \nabla u_h dS.&#xA;$$&#xA;But, importantly, the finite element approximation $u_h$ is defined with respect to a different set of shape functions, say $\varphi_k \in W_h \subset H^1(D)$ so that $u_h = \sum_k U_k \varphi_k$. It is important to realize that the $\varphi_k$ are not just &lt;em&gt;extensions&lt;/em&gt; of the $\psi_k$ -- their number is entirely different, and they form an unrelated space. With this, you can then write the problem as you wanted to:&#xA;$$&#xA;   \sum_j \left\{ \int_{\partial D} \psi_i \psi_j dl \right\} H_j = \sum_k \left\{\int_D \nabla (E\psi_k) \nabla \varphi_k dS\right\}  U_k.&#xA;$$&#xA;If you want to write this as a linear system with matrices, you'd get this:&#xA;$$&#xA;   B_{ij} = \int_{\partial D} \psi_i \psi_j \, dl, \quad A_{ij} = \int_D \nabla (E\psi_i) \nabla \varphi_j dS, &#xA;$$&#xA;and the linear system to find the coefficients $H_j$ of $h_h$ is given by&#xA;$$&#xA;  B H = A U.&#xA;$$&#xA;Here, $B$ is simply a mass matrix on the boundary, and consequently symmetric and positive definite. $A$, on the other hand, is a rectangular matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we had not chosen an extension operator. For practical reasons, it would of course be nice if $Ew_i$ is a finite element function, for example one in $W_h$ (or at least a polynomial on each cell) because then you can just apply quadrature as always. This is easily achieved by making sure that $Ew_i$ is zero at all interior nodes. If you do this, then $A_{ij}$ is zero for all $j$ corresponding to nodes that are neither at the boundary or on cells that are adjacent to the boundary. (This would suggest more efficient ways to store the matrix $A$, but that's besides the point for the current post.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-12-07T01:41:48.863" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28403" PostTypeId="1" CreationDate="2017-12-07T02:46:50.517" Score="1" ViewCount="824" Body="&lt;p&gt;I want to achieve the following in 2D (and without obstacles):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given start position A and end position B, generate the path between the two points that optimizes a cost function that depends on total length and overall smoothness. The one constraint is a tangential constraint (ie. velocity constraint) at point A.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the cost function could be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U = \int\limits_{t_\mathrm{A}}^{t_\mathrm{B}} a^2(t) dt + L$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $a$ the acceleration and $L$ the path length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know of a simple way I could solve this optimization problem, preferably using Python?&lt;/p&gt;&#xA;" OwnerUserId="26260" LastEditorUserId="7574" LastEditDate="2017-12-07T08:44:56.017" LastActivityDate="2017-12-13T23:46:07.857" Title="Trajectory optimization for smoothness" Tags="|optimization|python|constraints|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28404" PostTypeId="2" ParentId="28397" CreationDate="2017-12-07T18:25:17.770" Score="2" Body="&lt;p&gt;I found the following books pretty helpful:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Vogel - &lt;a href=&quot;https://books.google.com/books?isbn=0898717574&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Methods for Inverse Problems&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Parker - &lt;a href=&quot;https://books.google.com/books?isbn=0691036349&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geophysical Inverse Theory&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Aster, Borchers, Thurber - &lt;a href=&quot;https://books.google.com/books?isbn=0123850495&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parameter Estimation and Inverse Problems&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While it's not about PDE inverse problems per se, you will at some point have to either use or implement numerical optimization procedures, for which &lt;a href=&quot;https://books.google.com/books?isbn=0387400656&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nocedal and Wright&lt;/a&gt; is an invaluable reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I haven't found any one perfect book on the topic. If you post a detailed explanation of your problem in another question here, someone will probably be able to offer you more explanation of the underlying math or, at the very least, offer some more specific references.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-12-07T18:25:17.770" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28406" PostTypeId="1" CreationDate="2017-12-07T20:40:23.323" Score="0" ViewCount="1388" Body="&lt;p&gt;I am reading some notes from a course in numerical analysis for physical sciences and it is my impression that there are still people that prefer Fortran 77 over new version due to the implicit declaration of memory and control over the use of the memory. But I am not really familiar with Fortran my question is then why it looks that even young scientists prefer Fortran 77 when coding numerical linear algebra methods my question is why? is there a true advantage in using Fortran 77 over the latest versions?  &lt;/p&gt;&#xA;" OwnerUserId="26263" LastActivityDate="2017-12-08T14:40:29.077" Title="Why there are people that still prefer fortran 77 over new versions?" Tags="|numerical-analysis|computational-physics|fortran|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28407" PostTypeId="2" ParentId="28406" CreationDate="2017-12-07T20:59:24.197" Score="8" Body="&lt;p&gt;I think it's generally true that there are no &lt;em&gt;advantages&lt;/em&gt; of Fortran 77 over either newer versions of Fortran or in fact any number of other programming languages that are widely used in scientific computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason it's still used is because there are millions of lines of code around that are written in Fortran 77. Now, recall that it takes a good programmer a year to write 20,000 lines of code. So converting existing codes to other programming languages is just not an option. Even just to refactor Fortran 77 to newer Fortran styles is not feasible. And so people deal with it.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-12-07T20:59:24.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28408" PostTypeId="2" ParentId="28403" CreationDate="2017-12-07T23:48:10.527" Score="2" Body="&lt;p&gt;You will have to &lt;strong&gt;&lt;em&gt;transcribe&lt;/em&gt;&lt;/strong&gt; the control problem in some manner so that you can feed it to a non-linear programming solver.  You may want to look at  &lt;a href=&quot;http://www.matthewpeterkelly.com/research/MattKelly__Transcription_Methods_for_Trajectory_Optimization.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matthew Kelly's&lt;/a&gt; paper for a good introduction to transcription methods for trajectory optimization problems.  It basically summarizes transcription methods as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various classifications of transcription methods. The most intuitive is to discretize the problem first, then find an optimal solution to the discrete problem.  This is referred to as a Direct Method.  One could do the reverse, which is to optimize before discretizing.  This is referred to as the Indirect Method.  According to the references in the paper, the Direct Method is easier to pose and solve than the Indirect Method, but also leads to a less accurate solution than the indirect method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I'm not particularly familiar with indirect methods, I can certainly speak about direct methods.  In particular, there are two main classes of discretization for a direct method:  Shooting and Collocation.  Shooting works essentially by trial and error, adjusting control parameters until the trajectory becomes optimal.  Shooting can even be broken up over multiple discrete intervals in the time domain (also called Multiple Shooting).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, collocation works by replacing the ordinary differential equation with a system of algebraic ones (say, using finite difference approximation) and by representing the continuous control in terms of interpolants whose values at fixed timesteps (also known as knots) are resolved by the non-linear programming solver.  It is common to use spline interpolants for this purpose so that .  In the end, one obtains a non-linear objective function with algebraic constraints over a finite number of knots.  One systematically obtains higher accuracy by either increasing the number of timesteps (h-refinement) or by increasing the order of the interpolants (p-refinement).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I highly recommend reading the aforementioned paper, along with watching a lecture the author gave about this very same topic:  &lt;a href=&quot;https://youtu.be/wlkRYMVUZTs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Trajectory Optimization &lt;/a&gt;.  The video includes a worked out example involving control of an inverted pendulum.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The control function between the knots will be smooth by virtue of the interpolant function.  I'm not sure how to impose smoothness at the knots themselves, and I'm not sure if techniques to impose this constraint exist for this kind of problem.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2017-12-13T23:46:07.857" LastActivityDate="2017-12-13T23:46:07.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28409" PostTypeId="1" AcceptedAnswerId="28430" CreationDate="2017-12-08T00:03:23.640" Score="5" ViewCount="401" Body="&lt;p&gt;I'm trying to code a parameter estimation for a reaction-diffusion problem.  Namely, knowing the distribution of tumor density $u$ at time $0$ and $T_f$ ($u^0$ and $u^f$), what are the best distributions of the $k$ and $D$ parameters explaining the observed tumor evolution, assuming &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial{u}/\partial{t} = D(x)u + k(x)u(1-u).  \;\; \mbox{(1)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already coded the forward simulation code, giving the final distribution, knowing $D$, $k$ and $u^0$. I first tried to just set evenly distributed control points and simply used the matlab &lt;code&gt;fmincon&lt;/code&gt; function to try to solve the problem by interpolating the $D$ and $k$ maps between the control points (thus limiting the number of free parameters to ~200, and implicitly enforcing smoothness of the $D$ and $k$ fields) but unsuccessfully (I still don't understand why but that's another question)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm slowly understanding that using the adjoint method would be a (the?) correct way to solve my problem, but while I'm reading some tutorial papers about this method I'm stucked on how to apply it to my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let say that my goal is to find the set of distribution $D$ and $k$ which minimize the function &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ G(u,D,k)= \sum_i{(\tilde{u}_i-u^f_i)^2} + \sum_i{D_i}^2 + \sum_i{k_i}^2 \;\; \mbox{(2)},$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to an evolution function $$F(u^f,D,k) = 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal is to find the set of $D$ and $k$ minimizing $G$, which requires to evaluate $\partial{G}/\partial{D}=\partial_D{G}$ (and $\partial_k{G}$) multiple times per iteration if simply using iterative minimization algorithm on the function defined in (2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand well the concept of the adjoint method (as explained &lt;a href=&quot;https://www.google.fr/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=0ahUKEwjYicDri_nXAhUDKlAKHUdXCK0QFggmMAA&amp;amp;url=http%3A%2F%2Fmath.mit.edu%2F~stevenj%2F18.336%2Fadjoint.pdf&amp;amp;usg=AOvVaw2VcBAFgZaJru4PQ23P2fR4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), the idea is to replace the evaluation of $\partial_D{G}$ by observing that the optimal parameters are those for which $dG/dD$ and $dG/dk$ are zeros. First, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$dG/dD = \partial_D{G} + \partial_{u^f}{G}.\partial_D{u^f}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since one has $F(u^f,D,k)=0$ everywhere, one also has the relation $$dF/dD = \partial_D{F} + \partial_{u^f}{F}.\partial_D{u^f} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Rightarrow \partial_D{u^f} = - (\partial_{u^f}{F})^{-1} . \partial_D{F}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Rightarrow dG/dD = \partial_D{G} - \partial_{u^f}{G}. (\partial_{u^f}{F})^{-1} . \partial_D{F}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So by defining $\lambda$ by $$^T(\partial_{u^f}{F})\lambda = \partial_{u^f}{G}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one gets $$dG/dD = \partial_D{G} - \lambda . \partial_D{F} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the trick being that $\lambda$ does not depend on $D$ and has to be evaluated only once, and that $\partial_D{G}$ and $\partial_D{F} $ usually have an analytical expression, and do not require costly minimization. This should allow for fast evaluation of the derivative of $G$ and consequently faster minimization of $G$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm trying to apply it to my specific problem of reaction diffusion, and the only expression I can find to fit with the above definition of $F$ would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F(u^f,D,k) = u^0 + \int_0^{Tf}{((D\nabla u + k.u(1-u)) dt)}-u^f = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but then I don't understand how to determine $\partial_D{F}$ and $\partial_k{F}$ (I mean, not numerically but analytically).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, I missed some points... but I'm lost at this stage... If someone can give me a hand on this topic, I'd be very grateful!&lt;/p&gt;&#xA;" OwnerUserId="25963" LastEditorUserId="17294" LastEditDate="2017-12-08T09:50:32.483" LastActivityDate="2017-12-12T09:30:41.583" Title="adjoint method for reaction-diffusion problem" Tags="|pde|constrained-optimization|inverse-problem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28410" PostTypeId="2" ParentId="28406" CreationDate="2017-12-08T00:03:36.867" Score="4" Body="&lt;p&gt;In general the features that are being removed from the language are either pointless or dangerous, so I'd definitely recommend you may sure you write new code to compile under a later standard, even if you are using syntax supported in the old standard. However one key area where there is still some incentive to write code that is &quot;FORTRAN 77-like&quot; is when interoperability with C is a consideration, as it often will be for library codes for numerical methods. Although the ISO C bindings are good (and I would highly recommend anyone writing interoperable Fortran code to learn and use them in preference to tricks that work with one compiler), it's still often necessary to pass the size and shape of arrays explicitly from one side of the fence to the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The net result of this is large, confusing, interfaces to functions etc, rather than the clean, simple to understand (and get right) approach favoured by the later, more object oriented language standards. I'm hopeful this will improve in the future, with the latest work on passing array descriptors C being implemented, but it doesn't seem to be there yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, it's somewhat reassuring to note that FORTRAN 77 was a subset of valid Fortran 90, so that the code is only a quarter of a century behind the new fashion, rather than a third.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2017-12-08T00:03:36.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28411" PostTypeId="1" AcceptedAnswerId="28412" CreationDate="2017-12-08T01:55:50.080" Score="1" ViewCount="1371" Body="&lt;p&gt;A sparse matrix $\mathbf{L}$ is formed by $\mathbf{L} = \mathbf{DKG}$.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\mathbf{D}$ is a sparse matrix of size $m \times n$.&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{G}$ is a sparse matrix of size $n \times m$.&lt;/li&gt;&#xA;&lt;li&gt;$\mathbf{K}$ is a diagonal matrix of size $n \times n$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$\mathbf{L}$ needs to be formed many times based on the updated diagonal entries of $\mathbf{K}$. Both $\mathbf{D}$ and $\mathbf{G}$ never change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a faster way to form $\mathbf{L}$ without preforming two full sparse matrix multiplications?&lt;/p&gt;&#xA;" OwnerUserId="23647" LastActivityDate="2017-12-08T05:26:22.913" Title="More efficient matrix multiplication with diagonal matrix" Tags="|linear-algebra|sparse-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28412" PostTypeId="2" ParentId="28411" CreationDate="2017-12-08T03:48:05.313" Score="3" Body="&lt;p&gt;Each entry $K_{i,i}$ on the diagonal of $K$ is multiplied by the outer product of column $i$ of $D$ and row $i$ of $G$.  You can write the product as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$DKG=\sum_{i=1}^{n} K_{i,i}(D_{:,i}G_{i,:})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you precompute and store the sparse outer products $D_{:,i}G_{i,:}$, then you can quickly multiply these by $K_{i,i}$ and add those terms into your product.  Since the product matrix sparsity pattern doesn't change, you can preallocate a data structure with the sparsity pattern of the product and reuse it each time you update $K$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In doing this, you can save time by precomputing where in the sparse product matrix each entry of $D_{:,i}G_{i,:}$ belongs.  e.g. if there is an entry in $D_{:,1}G_{1,:}$ at position (3,4), and you know that position (3,4) is stored in the 33rd entry of the DKG product matrix, then record the fact that this is the 33rd entry in the product matrix rather than  recording (3,4).       &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2017-12-08T05:26:22.913" LastActivityDate="2017-12-08T05:26:22.913" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28416" PostTypeId="2" ParentId="28406" CreationDate="2017-12-08T14:40:29.077" Score="2" Body="&lt;p&gt;1) For a long time writing explicitly loops was more efficient than using array syntax because array syntax used a lot of intermediate copies of arrays. Today, this is not true any more, but many Fortran practitioners (like me!) have taken the habit to avoid array syntax because of performance issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Using pointers makes the code more difficult to optimize by the compiler, and pointers are inexistent in F77. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) When all the array dimensions are known at compile time, the compiler can do many more optimizations than when the arrays are allocatable. So there is also a performance gain using statically-sized array. For example, the address calculation in multi-dimensional arrays can be done at compile-time avoiding a lot of integer multiplications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Generally the F77 style is such that the compiler has much more hints on how many cycles loops will do, as many dimensioning variables are given in parameter. When everything is dynamic, this information is missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) For some time, &lt;code&gt;dot_product&lt;/code&gt; and &lt;code&gt;matmul&lt;/code&gt; were known to produce inefficient binary code with usual compilers. Fortunately this is not true any more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize, I would say that F77 is so old and simple that compilers can generate super-fast binaries. But for modern Fortran, the language is more complex, so it is more difficult to optimize and it takes some time for the compilers to catch up.&lt;/p&gt;&#xA;" OwnerUserId="17681" LastActivityDate="2017-12-08T14:40:29.077" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28417" PostTypeId="1" AcceptedAnswerId="28419" CreationDate="2017-12-08T17:25:40.503" Score="0" ViewCount="136" Body="&lt;p&gt;I'm working on a problem in nonlinear elasticity, for which the external forces (loadings) depend on the displacements.  Following Klaus-Jürgen Bathe's book &quot;Finite Element Procedures&quot;, the virtual work is&#xA;\begin{equation}&#xA;\int_{{}^{t+\Delta t}V}{}^{t+\Delta t}\tau_{ij}\delta{}_{t+\Delta t}e_{ij}d{}^{t+\Delta t}V={}^{t+\Delta t}\mathscr{R}&#xA;\end{equation}&#xA;where the prescript notation denotes the time at which the quantities are considered, $V$ is the volume, $\boldsymbol{\tau}$ is the Cauchy stress, $\delta\boldsymbol{e}$ is the strain tensor for the virtual displacement $\delta\boldsymbol{u}$, and the external work term on the right-hand side is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;{}^{t+\Delta t}\mathscr{R}=\int_{{}^{t+\Delta t}V}{}^{t+\Delta t}f_i^B\delta u_id{}^{t+\Delta t}V+\int_{{}^{t+\Delta t}S_f}{}^{t+\Delta t}f_i^S\delta u_i^Sd{}^{t+\Delta t}S&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\boldsymbol{f}^B$ is the externally applied body force, $\boldsymbol{f}^S$ is the externally applied surface traction, $S_f$ is the surface on which the tractions are applied, and $\delta\boldsymbol{u}^S$ is the virtual displacement evaluated on the surface.  My problem has no surface tractions so the second term on the right-hand side can be omitted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later in the chapter, Bathe considers deformation-dependent loading, e.g. equations (6.83) and (6.84), where he argues that the external work can be approximated by integrating over the most recently calculated volume and surface area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question relates to the derivation of the equation for the external work.  Specifically, if the body force $\boldsymbol{f}^B$ depends on the displacement $\boldsymbol{u}$, why is there no term in the virtual work that looks like $\boldsymbol{u}\delta{\boldsymbol{f}^B}$?&lt;/p&gt;&#xA;" OwnerUserId="25235" LastActivityDate="2017-12-09T16:31:22.877" Title="Principle of virtual work - extra term needed for deformation dependent loading?" Tags="|finite-element|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28418" PostTypeId="1" CreationDate="2017-12-09T14:44:12.890" Score="2" ViewCount="233" Body="&lt;p&gt;In &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0521675952&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Writing Scientific Software: A Guide to Good Style&lt;/em&gt;&lt;/a&gt;, several disasters are mentioned including a missle defence system which led to &lt;a href=&quot;http://www.nytimes.com/1991/02/26/world/war-in-the-gulf-scud-attack-scud-missile-hits-a-us-barracks-killing-27.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;deaths at a US base in Saudi Arabia in 1991&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error was caused due to catastrophic cancellation in tracking the incoming Scud missile's velocity, in order to fire a missile before it hit the base:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{x_{i+1}-x_i}{t_{i+1}-t_i}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear that for large times, and a small difference, this can lead to poor estimates because of catastrophic cancellation. When this was noticed in their C program, it was apparently fixed but how could this be done? Excluding solutions with third party libraries, e.g. GMP. &lt;/p&gt;&#xA;" OwnerUserId="26271" LastActivityDate="2017-12-10T11:05:01.163" Title="Fixing catastrophic cancellation in velocity formula" Tags="|finite-difference|floating-point|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28419" PostTypeId="2" ParentId="28417" CreationDate="2017-12-09T16:31:22.877" Score="1" Body="&lt;p&gt;If I understand correctly, in the book, the author says that the term $\mathbf{u}\,\delta\mathbf{f}^{B}$ can be moved over to the left hand side and included in the stiffness matrix along with other coefficients of $\mathbf{u}$.&lt;/p&gt;&#xA;" OwnerDisplayName="user26248" LastActivityDate="2017-12-09T16:31:22.877" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28420" PostTypeId="1" AcceptedAnswerId="28423" CreationDate="2017-12-09T17:36:08.213" Score="3" ViewCount="697" Body="&lt;p&gt;As stated in the title, it's said in the book that Gauss-Seidel iterative method is equivalent to successively setting each component of residual vector to zero. After rearranging G-S scheme, I got the following relation regarding the update of residual:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r^{(k+1)} = U(D-L)^{-1}r^{(k)},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I still can't see why this conclusion holds. Any help?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastActivityDate="2017-12-10T02:20:29.423" Title="How to show that Gauss-Seidel iterative method is equivalent to successively setting each component of residual vector to zero?" Tags="|linear-algebra|numerical-analysis|iterative-method|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28421" PostTypeId="2" ParentId="28418" CreationDate="2017-12-09T19:27:33.350" Score="4" Body="&lt;p&gt;The problem with the Patriot missile system was that time was stored in tenths of seconds as an integer. Before calculations (time difference between a radar pulse sent and detected back again), this integer was converted in a 24-bit floating point number. This proved too low in precision: after 100 hours of operation, the actual time was 360,000 seconds, the value in the floating point variable was 359,999.6567 seconds. This seems small, but with a velocity of such a SCUD missile (approx. 1676 m/s), this time off-set corresponded to a distance off-set of approximately 687 meters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using more bits in the floating point representation (classical double precision being 64 bits) would have solved the problem. No need to go to more advanced fixed precision libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness, this resulted in 28 casualties. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2017-12-09T19:27:33.350" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28422" PostTypeId="1" CreationDate="2017-12-09T20:29:24.493" Score="1" ViewCount="175" Body="&lt;p&gt;I am working on a problem (solid state physics, I am stripping all the details for brevity but if more details can help I'll elaborate) where I need to numerically calculate an integral of the form:&#xA;$$\vec{A}(\vec{r}) = \int_{\mathbb{R}^{3}\setminus E}\vec{B}(\lambda, \vec{r},\vec{w}) d^{3}w,&#xA;\quad&#xA;\vec{r} \in E $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where E is volume of rotational ellipsoid centered in the origin with z-axis as its axis of symmetry.  My questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) If the function $\vec{B}$ is well behaved (finite, no sharp peaks or wild oscillations, decays exponentially far from the origin so convergence should not be a problem) should one try to integrate this with Monte Carlo or classical integration methods? I am considering the &lt;a href=&quot;http://www.feynarts.de/cuba/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuba4.2&lt;/a&gt; library for the implementation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) Would you expect any problems when integrating over domain without some interior, line-connected volume? The implementation would be the obvious one - integrand function would simply return 0.0 whenever $\vec{w}$ is within $E$. Cuba allows for some preexisting knowledge to be supplied (like location of peaks) but not flexible boundaries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;c) The real problem here is amount of work. Basically, $\lambda$ describes parameter space which I need to parse fairly densely - $10^{4}$-$10^{5}$ points and for each $\lambda$ I need to calculate integral for $10^{5}$ different $\vec{r}$, so considerable amount of work - preliminary runs give estimate around 45 days straight on my PC. Cuba itself is written for parallel execution, but if anyone has used cuba library - are there room for any speedups, such as using &quot;state&quot; files to provide an preliminary mash that can be further refined or something like that? Idea would be, since I got to do $10^5$ integrals for each $\lambda$, a reasonable starting mash could help achieve the precision goal faster. Any other advice?&lt;/p&gt;&#xA;" OwnerUserId="26272" LastEditorUserId="713" LastEditDate="2017-12-29T21:04:30.467" LastActivityDate="2017-12-29T21:04:30.467" Title="Integrating over $\mathbb{R}^{3}$ without a convex subset" Tags="|libraries|numerics|integration|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28423" PostTypeId="2" ParentId="28420" CreationDate="2017-12-10T01:59:12.737" Score="5" Body="&lt;p&gt;Given approximated solution $\mathbf x^{(k)}$, we define the residual vector $\mathbf r^{(k)} := \mathbf b - \mathbf A\,\mathbf x^{(k)} $. Now we want to update $x^{(k)}_i \rightarrow x^{(k+1)}_i$ in such a way so that the $i$th component of the new residual vanishes:&#xA;$$&#xA;r^{(k+1)}_i = b_i - \sum_{j=1}^n a_{ij}\,x^{(k+1)}_j = 0, \\&#xA;b_i - \left(\sum_{j=1}^{i-1} a_{ij}\,x^{(k+1)}_j + a_{ii}\,x^{(k+1)}_i + \sum_{j=i+1}^n a_{ij}\,x^{(k+1)}_j\right) = 0, \\&#xA;x^{(k+1)}_i = \frac{1}{a_{ii}} \left(b_i - \sum_{j=1}^{i-1} a_{ij}\,x^{(k+1)}_j - \sum_{j=i+1}^n a_{ij}\,x^{(k+1)}_j\right).&#xA;$$ &#xA;Assuming that all the other components stay the same, we may rewrite this formula as&#xA;$$&#xA;x^{(k+1)}_i = \frac{1}{a_{ii}} \left(b_i - \sum_{j=1}^{i-1} a_{ij}\,x^{(k)}_j - \sum_{j=i+1}^n a_{ij}\,x^{(k)}_j\right).&#xA;$$&#xA;Now, if we introduce matrix notation $\mathbf A = \mathbf L + \mathbf D + \mathbf U$ with strictly lower, diagonal, and strictly upper parts of the original matrix, we may rewrite the last equation as&#xA;$$&#xA;\mathbf L\,\mathbf x^{(k)} + \mathbf D\,\mathbf x^{(k+1)} + \mathbf U\,\mathbf x^{(k)} = \mathbf b, \\&#xA;\mathbf x^{(k+1)} = -\mathbf D^{-1}\left(\mathbf L + \mathbf U\right)\mathbf x^{(k)} + \mathbf D^{-1}\,\mathbf b;&#xA;$$&#xA;this is well-known Jacobi method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that at the $i$th step we have already updated components $1, 2, \dots, i-1$, so let’s use these updated values:&#xA;$$&#xA;x^{(k+1)}_i = \frac{1}{a_{ii}} \left(b_i - \sum_{j=1}^{i-1} a_{ij}\,x^{(k+1)}_j - \sum_{j=i+1}^n a_{ij}\,x^{(k)}_j\right),&#xA;$$&#xA;or&#xA;$$&#xA;\mathbf L\,\mathbf x^{(k+1)} + \mathbf D\,\mathbf x^{(k+1)} + \mathbf U\,\mathbf x^{(k)} = \mathbf b, \\&#xA;\mathbf x^{(k+1)} = -(\mathbf D + \mathbf L)^{-1}\,\mathbf U\,\mathbf x^{(k)} + (\mathbf D + \mathbf L)^{-1}\,\mathbf b.&#xA;$$&#xA;This is Gauss–Seidel method.&lt;/p&gt;&#xA;" OwnerUserId="21916" LastEditorUserId="21916" LastEditDate="2017-12-10T02:20:29.423" LastActivityDate="2017-12-10T02:20:29.423" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28424" PostTypeId="2" ParentId="28418" CreationDate="2017-12-10T10:59:58.113" Score="4" Body="&lt;p&gt;The answer given by @GertVdE contains some of the information given in the official report from US General Accouting Office &lt;a href=&quot;http://archive.gao.gov/t2pbat6/145960.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GAO IMTEC-92-26&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, that report is incomplete!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When tracking an incoming missile the &lt;em&gt;absolute&lt;/em&gt; time is irrelevant, rather it is the elapsed time between two radar pulses which is essential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Patriot system's internal clock tick was 0.1 seconds. It counted time using integers. A function call converted the integer into a real time through multiplication with a binary approximation of 0.1 seconds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original function used a 24 bit approximation of 0.1 seconds. A upgrade, designed to cope with faster missiles, used a 48 bit approximation. Unfortunately, at least one call to the old function was not replaced with a call to the new function!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, time increments where computed by subtracting a highly accurate absolute time from a less accurate absolute time. Had the software patch not been applied, then the errors would have cancelled and the missile would have fired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not been able to find an official report confirming this. The best I can do is cite &lt;a href=&quot;http://www-users.math.umn.edu/~arnold/disasters/Patriot-dharan-skeel-siam.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIAM News July 1992, Volume 25, Number 4, page 11&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&#xA;Since the author, Robert Skeel, later served on my thesis committee at Purdue, I am inclined to accept it :)&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2017-12-10T11:05:01.163" LastActivityDate="2017-12-10T11:05:01.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28430" PostTypeId="2" ParentId="28409" CreationDate="2017-12-11T15:00:26.253" Score="5" Body="&lt;p&gt;Some remarks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notation: $[t_1,t_2]$: the time interval; $\Omega$: the spatial domain; $\bar{u}^i(x)$: the known tumour profile at $t_i$; $\left\| \cdot \right\|_\Omega$: a suitably chosen norm on $\Omega$. I assume that the design variables $D$ and $k$ do not depend on $t$.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The cost functional should be a scalar (real-valued), something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${\cal G}(u; D, k) := \frac{1}{2} \sum_{i=1,2} \left\| u^i(x) - \bar{u}^i(x) \right\|^2_\Omega + \frac{1}{2} \left\| D(x) \right\|^2_{\Omega} \ + \frac{1}{2} \left\| k(x) \right\|^2_{\Omega}.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your reaction-diffusion PDE problem (the &quot;forward model&quot;, including appropriate &lt;em&gt;boundary and initial conditions&lt;/em&gt;) acts as a constraint for the optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ {\cal F}(u; D,k) = 0.$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Lagrangian functional for the constrained problem reads something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${\cal L} (u; \lambda; D,k) := {\cal G}(u;D,k) + \left&amp;lt; \lambda , {\cal F}(u; D,k) \right&amp;gt;_{[t_1,t_2] \times \Omega}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $\lambda$ is the adjoint variable defined on $[t_1,t_2] \times \Omega$ and $&amp;lt;\cdot&amp;gt;$ an inner product. It's up to you to define the latter.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The first-order necessary conditions for a local optimum (the so-called first-order KKT conditions, see chapter 12 of &lt;a href=&quot;http://www.springer.com/gp/book/9780387303031&quot; rel=&quot;nofollow noreferrer&quot;&gt;the book by Nocedal and Wright&lt;/a&gt;) give you the optimality system: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the adjoint equation: ${\cal L}_u (u; \lambda; D,k)(\delta u) = 0 \;\; \forall \; \delta u$;&lt;/li&gt;&#xA;&lt;li&gt;the primal equation (the forward model): ${\cal L}_\lambda (u; \lambda; D,k)(\delta \lambda) = 0 \;\; \forall \; \delta \lambda$;&lt;/li&gt;&#xA;&lt;li&gt;the &lt;em&gt;gradient&lt;/em&gt; (or optimal) equations: ${\cal L}_D (u; \lambda; D,k)(\delta D) = 0 \;\; \forall \; \delta D$ and ${\cal L}_k (u; \lambda; D,k)(\delta k) = 0 \;\; \forall \; \delta k$.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The choice of cost functional and the primal (forward) PDE have to be &lt;em&gt;compatible&lt;/em&gt; such that the dual (adjoint) problem is well posed. Compatibility is discussed further &lt;a href=&quot;http://www.numerik.uni-hd.de/~hartmann/publications/2007/Har07a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://people.maths.ox.ac.uk/gilesm/files/AIAA-97-1850.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999114002290&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The proper way to derive the analytic form of these equations (using integration by parts) is shown in chapters 2 and 3 of &lt;a href=&quot;http://epubs.siam.org/doi/book/10.1137/1.9780898718720&quot; rel=&quot;nofollow noreferrer&quot;&gt;Max Gunzburger's optimal control book.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-12-12T09:30:41.583" LastActivityDate="2017-12-12T09:30:41.583" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28433" PostTypeId="1" CreationDate="2017-12-11T19:25:56.647" Score="1" ViewCount="132" Body="&lt;p&gt;I'm interested in finding a &lt;strong&gt;free&lt;/strong&gt; online textbook that is an introductory textbook on numerical PDE for engineers. It should cover the basics of finite differences, finite elements, and Fourier series methods for very simple PDE cases, mainly the 1D heat and 1D wave equations, and should &lt;em&gt;not&lt;/em&gt; be concerned with rigor, proofs of convergence, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of this request is to find a decent online resource for students to use as an alternative to an official course textbook. The students are undergraduate engineers who have seen multivariate calculus and linear algebra but no other mathematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody have suggestions?&lt;/p&gt;&#xA;" OwnerUserId="3477" LastEditorUserId="3477" LastEditDate="2017-12-11T20:20:02.707" LastActivityDate="2017-12-11T20:20:02.707" Title="Reference request: free/open source books on intro numerical PDE" Tags="|pde|reference-request|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28434" PostTypeId="1" AcceptedAnswerId="28435" CreationDate="2017-12-12T16:33:38.277" Score="2" ViewCount="77" Body="&lt;p&gt;We have a cylinder of length $l$ (in units of its radius $d,$ as basic unit of length set to $d=1.$) in a box, and we consider an orthonormal Cartesian coordinate system with its origin placed at the centre of the box. We know the position of the cylinder in terms of the coordinates of its centre of mass, so $\vec{R}=(r_x,r_y,r_z)$ and in terms of its orientation vector (vector along its main/long axis) $\vec{O}=(o_x,o_y,o_z).$ Assuming at a later time the position and orientation of the cylinder have been randomized (still in the box), we'd like to estimate the displacement vector in terms of: how much the cylinder has moved parallel to its long axis ($\Delta \vec{R}_{||}$), how much it's moved in the plane perpendicular to its long axis ($\Delta \vec{R}_{\perp}$), how much it's been rotated in terms of $\Delta \theta$ and $\Delta \phi$ (usual angular definitions in spherical coordinates). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing $\vec{R}$ and $\vec{O}$ both before and after displacement, I'm wondering how to estimate these correctly. For instance I guess for the displacement along the long axis, it's probably easy and given by: $\Delta \vec{R}_{||} = (\Delta \vec{R}\cdot \vec{O}_0)\vec{O}_0/l, $ right? How about the other ones? &lt;/p&gt;&#xA;" OwnerUserId="24573" LastActivityDate="2017-12-13T14:01:23.977" Title="Vector characterization of cylinder displacements in a box" Tags="|geometry|vector|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28435" PostTypeId="2" ParentId="28434" CreationDate="2017-12-12T18:41:14.260" Score="2" Body="&lt;p&gt;You're correct, if the orientational vector is unitary. Otherwise you must calculate the unitary vector in the direction of $\vec{O}_1$ and then perform the projection (just divide by the norm of the orientational vector). The perpendicular component is just the rejection vector of $\Delta \vec{R}$, given by the definition: $\Delta\vec{R}_{\perp} = \Delta\vec{R} - \Delta\vec{R}_{\parallel}  $. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you draw the vectors you will be able to visualize that this is correct and then convince yourself. Take a look:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XLv4a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XLv4a.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can determine straight forward the rotation by the internal product between the two orientational vectors $\vec{O}_1$ and $\vec{O}_2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\theta = \arccos{\left( \frac{\vec{O}_1 \cdot \vec{O}_2}{|\vec{O}_1||\vec{O}_2|}\right)} $&lt;/p&gt;&#xA;" OwnerUserId="26289" LastEditorUserId="26289" LastEditDate="2017-12-13T14:01:23.977" LastActivityDate="2017-12-13T14:01:23.977" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28436" PostTypeId="1" CreationDate="2017-12-12T21:14:49.850" Score="6" ViewCount="278" Body="&lt;p&gt;I'm writing a differential geometry library, and one minor convenience I'd like to offer is to generate a set of points on a surface given by a constraint. For example, for a sphere,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^2+y^2+z^2-r^2=0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if it were an equation in two variables, I could step through one of them and then solve for the other, or choose the first based on some random distribution, and solve for the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here though, I'm not sure what the ideal approach is, and what can be done to make sure I've sort of 'evenly' sampled points on the surface, for triangulation later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithms would be most appropriate for this task?&lt;/p&gt;&#xA;" OwnerUserId="26301" LastActivityDate="2017-12-14T18:08:31.007" Title="Generating set of points on a surface defined by constraint" Tags="|c++|computational-geometry|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28438" PostTypeId="2" ParentId="27155" CreationDate="2017-12-12T23:41:41.607" Score="1" Body="&lt;p&gt;You may like this &lt;a href=&quot;https://github.com/precice/elastictube1d&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial for simulating the flow in a deformable artery&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this 1D example you can actually see how the fluid and the solid solver work and how &lt;a href=&quot;http://www.precice.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;preCICE&lt;/a&gt; couples them. Both a serial and a parallel version of the solvers are available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As soon as you get some understanding of the basics, you can start building more complex simulations, using existing solvers. See this tutorial for example: &lt;a href=&quot;https://github.com/precice/precice/wiki/FSI-tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;FSI with SU2 and CalculiX&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have your own solvers, you can actually &lt;a href=&quot;https://github.com/precice/precice/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;create your own adapter&lt;/a&gt;. As preCICE is free software, you can even modify preCICE itself to fit your needs.&lt;/p&gt;&#xA;" OwnerUserId="17650" LastEditorUserId="17650" LastEditDate="2017-12-12T23:58:46.813" LastActivityDate="2017-12-12T23:58:46.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28439" PostTypeId="1" CreationDate="2017-12-13T06:36:29.347" Score="1" ViewCount="156" Body="&lt;p&gt;I am trying to solve a 1 Dimensional eigenvalue of poisson problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla \phi ^2 +\nabla \phi = k\phi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the boundary condition: $\phi (0)=0 , \nabla \phi(1) = 0 $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could solve this directly, but I am trying to solve it for parallel calculation framework by dividing the problems into 3 or 4 meshes. Then, I connect the middle boundary with the robin boundary condition and the most left mesh with the zero flux and robin boundary, and the most right mesh with robin boundary and right mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I solve this problem, I find an inconsistency problem when I vary the number of mesh divisions. Could you guys give me some insight?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more detail, actually i just try to do the simple implementation first to test whether my algorithm will work or not. So what i did is: let say i have 3 parallel meshes. Previously i only have 1 FEM main equation, now i will have 3 FEM Equation (each mesh will have their own governing equation), Then i connect my parallel mesh by using the robin boundary condition for the same interface. Is this okay?&lt;/p&gt;&#xA;" OwnerUserId="26303" LastEditorUserId="26303" LastEditDate="2017-12-14T03:40:24.533" LastActivityDate="2017-12-14T03:40:24.533" Title="Parallel calculation in finite elements" Tags="|finite-element|boundary-conditions|parallel-computing|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28441" PostTypeId="2" ParentId="1745" CreationDate="2017-12-13T16:37:53.443" Score="0" Body="&lt;p&gt;Templates are very good at removing type/domain checks at run-time. These can be taken care of at compilation time. This can in theory increase performance over the same type of implementation in C or Fortran where type checking can only be done at run-time - checks are implemented in the source code. However, you can achieve the same results in C using precompiler options but these have to be done by hand unlike templates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, templates can produce significant overhead too. They can often create code bloat which can impact upon the use of the instruction cache. Furthermore generic approaches can often shackle the compiler during optimisation - it isn't always straight forward for code analysis when use generic approaches. It is always a problem with automation - including compiler optimisation - very often the output code is not cache friendly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The benefits of type/domain checking, although certainly safer, is the only real benefit I can see in terms of performance and these are typically imperceptible. But as I say the overall effect can be negative depending what you're doing. That's why it is often better to hand optimise your code where you have significant bottlenecks. &lt;/p&gt;&#xA;" OwnerUserId="26310" LastEditorUserId="26310" LastEditDate="2017-12-14T00:14:27.097" LastActivityDate="2017-12-14T00:14:27.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28442" PostTypeId="1" CreationDate="2017-12-13T17:35:56.617" Score="0" ViewCount="117" Body="&lt;p&gt;The common case of for instance a Monte Carlo simulation is, if we want to run our simulation for $N$ steps, we define a delta $\Delta,$ such that $N/\Delta = n$ tells us the frequency with which we measure/evaluate quantities of interest during the MC run. &#xA;For instance, say $N=1000,$ and we want to measure the average of an observable $f,$ after each $n=100$ steps (so here $\Delta=10$). The average is obtained by dividing $f$ by the number of times $f$ has been measured. &#xA;This is &lt;em&gt;linearly&lt;/em&gt; sampling the time or MC steps and the simulation looks schematically like (writing in python style):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;interval = N/delta&#xA;f = 0&#xA;countmeasurement = 0&#xA;for mcstep in range(1,N):&#xA;  interval -= 1&#xA;  .&#xA;  .&#xA;  .&#xA;  if interval == 0:&#xA;    f += computef()&#xA;    .&#xA;    .&#xA;    interval = N/delta&#xA;    countmeasurement += 1&#xA;  #save or print&#xA;  print &quot;steps &quot;, mcstep, &quot;average f &quot;, f/countmeasurement &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this scheme, if later we plot $f$ as a function of MC-steps, on a log-log scaled plot, we will not have sampled the same amount of datapoints for each time scale as we sampled the total steps only linearly. How do we alter our sampling scheme, thus redefining &lt;code&gt;interval&lt;/code&gt;, such that we gather the same number of datapoints for $f$ for each part (time-scale) of the log-log plot? Is it common to sample the MC-steps logarithmically in Monte Carlo simulations? &lt;/p&gt;&#xA;" OwnerUserId="24573" LastActivityDate="2017-12-19T12:55:26.770" Title="Sampling simulation steps logarithmically" Tags="|simulation|monte-carlo|random-sampling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28443" PostTypeId="2" ParentId="28439" CreationDate="2017-12-13T17:51:38.390" Score="2" Body="&lt;p&gt;The solution must be the same independent of the number of processors you are using to integrate the equation. Your inconsistence is probably due to a bug in your algorithm of mesh division and/or virtual topology generation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please provide more details about your implementation and the algorithm you are using to divide the mesh and assign the pieces among the processors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; &lt;em&gt;I know that this kind of comment is better suitable to be made as a comment to the question and not as an answer. But I'm new in this stack and I don't have enough reputation to comment in a question. So please, don't downvote my answer. I'm genuinely looking forward to help and to collaborate in this stack, but receiving downvotes in this kind of comments will likely demotivate myself to continue participating actively here.&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="26289" LastEditorUserId="26289" LastEditDate="2017-12-13T23:38:37.510" LastActivityDate="2017-12-13T23:38:37.510" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28444" PostTypeId="2" ParentId="28442" CreationDate="2017-12-13T21:30:03.790" Score="1" Body="&lt;p&gt;If you want your sampling over the (a,b) interval to appear uniform when viewed on a log-log scale, linearly interpolate the logarithms: &lt;code&gt;samples = 10.^linspace(log10(a),log10(b))&lt;/code&gt;. This is basically what &lt;code&gt;samples=logspace(a,b)&lt;/code&gt; does anyway.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2017-12-13T21:38:22.333" LastActivityDate="2017-12-13T21:38:22.333" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28446" PostTypeId="2" ParentId="28436" CreationDate="2017-12-13T23:45:36.590" Score="3" Body="&lt;p&gt;If you're interested in surface meshing, the book &lt;a href=&quot;https://books.google.com/books?isbn=1584887311&quot; rel=&quot;nofollow noreferrer&quot;&gt;Delaunay Mesh Generation&lt;/a&gt; by Cheng, Dey, and Shewchuk is very good. Surface meshing is a pretty involved topic and I couldn't attempt to summarize it here. The constrained Delaunay triangulation and Voronoi diagram are especially important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also mentioned being able to find seed points that are on the surface. Part of this will involve being able to solve nonlinear equations, for which you'll want to use Newton's method. If you're dealing with algebraic surfaces, Smale's $\alpha$-theory can give you convergence guarantees that aren't available in general. My old officemate gave a talk (&lt;a href=&quot;http://www.cswiercz.info/assets/files/narc-smale.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slides&lt;/a&gt;) that I think makes a great introduction. You can also take the sum of the squares of all the equations defining the surface and think of it as an optimization problem, but in that case Smale's theory applies too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you do have some point on the surface, another important operation is to move that point following the flow of some vector field. You might also want to be able to compute geodesics. This amounts to solving a differential-algebraic system. &lt;a href=&quot;https://books.google.com/books?isbn=3662050188&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geometric Numerical Integration&lt;/a&gt; by Hairer, Lubich and Wanner has a lot of information about ODEs on manifolds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a real application, you can check out the manifold class in the &lt;a href=&quot;http://dealii.org/developer/doxygen/deal.II/classManifold.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;deal.II library&lt;/a&gt;. deal.II uses manifolds to solve the problem of refining initially coarse meshes on surfaces. If you try to refine a mesh of, say, an annulus, a naive implementation would just put more points on the boundary segments without adjusting the curvature. With deal.II, you can say that a certain part of the boundary of a mesh is actually some manifold, and the refined mesh will adapt to the real surface much better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, you might find a lot of the engineering literature on boundary representation or constructive solid geometry modelling to be inspirational. Sounds like a cool project and good luck! Do post a link on here if you make any of it open source.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-12-13T23:45:36.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28447" PostTypeId="1" CreationDate="2017-12-14T05:03:33.193" Score="10" ViewCount="236" Body="&lt;p&gt;The central difference scheme:&#xA;$$\frac{d^2u}{dx^2}=\frac{u_{n+1}-2u_i + u_{n-1}}{\Delta x^2}$$&#xA;yields a tridiagonal coefficient matrix [1 -2 1]; As the number of points gets larger, this matrix becomes ill-conditioned. Yet this is a popular discretization. Why is this scheme so commonly used when it is prone to ill-conditioning, and what is the typical workaround to the illconditoning? &lt;/p&gt;&#xA;" OwnerUserId="26313" LastEditorUserId="713" LastEditDate="2017-12-14T17:50:04.087" LastActivityDate="2020-09-16T16:36:59.800" Title="Central differencing scheme for second derivative leads to ill-conditioning" Tags="|conditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28449" PostTypeId="1" CreationDate="2017-12-14T12:06:39.250" Score="1" ViewCount="617" Body="&lt;p&gt;As the title says, I am trying to code in FEM a plate structure that either has a hole in one of the layers or one of the layers is made of patches of plates, rather than one whole plate. However, while I have a slight idea of how to implement this, I'm really not very sure and I would like some questions answered before I start trudging through my code haphazardly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand of FEM, I understand that in order to represent a hole or a patch (which is basically a plate with a hole AROUND it) is to change the connectivity matrix for a given layer. However, I don't quite get how I would go about doing that. For example, if I had a 6x6 element plate structure and one of the layers of that plate was to have a 2x2 element hole in the middle, do I simply remove those elements from my connectivity vector? I just have a feeling that would cause some of my matrix dimensions to not match and throw the whole thing into chaos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, what if the location/dimensions of my hole or plate didn't align perfectly with the coordinates of my elements? how do I implement that? Would I have to write a completely new coordinate system for my nodes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sorry if my questions seem either too obvious or too vague. I just wanted to know if I'm heading in the right direction before investing in any major change to my code. If any of you could point me in the right direction or provide some good practice/sample pieces of code that would be great. I'm learning FEM basically through trial and error.&lt;/p&gt;&#xA;" OwnerUserId="22445" LastEditorUserId="9667" LastEditDate="2017-12-14T16:39:52.283" LastActivityDate="2020-04-05T21:01:55.260" Title="Finite Element Analysis for Laminated Plates with Holes or Patches" Tags="|finite-element|matlab|solid-mechanics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28450" PostTypeId="2" ParentId="28447" CreationDate="2017-12-14T14:47:09.253" Score="19" Body="&lt;p&gt;TL;DR: The continuous operator exhibits this behavior, any faithful discretization will just inherit it.&lt;/p&gt;&#xA;&lt;p&gt;Deeper cut: If you look at the spectrum (eigenpairs) of the continuous operator &lt;span class=&quot;math-container&quot;&gt;$\frac{d^2}{dx^2}$&lt;/span&gt;, the eigenvectors are trigonometric functions of the form &lt;span class=&quot;math-container&quot;&gt;$\cos(kx)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sin(kx)$&lt;/span&gt;, with eigenvalue &lt;span class=&quot;math-container&quot;&gt;$k^2$&lt;/span&gt;. Low frequency functions (ie constant or very smooth functions) can be well represented on any grid. As you refine and sample with more points, you will more closely approximate the high frequency solutions too. Therefore, basically any convergent discretization of this operator will have a nearly constant &lt;span class=&quot;math-container&quot;&gt;$\lambda_{min}$&lt;/span&gt; eigenvalue and a &lt;span class=&quot;math-container&quot;&gt;$\lambda_{max}$&lt;/span&gt; eigenvalue that grows quadratically with number of samples (thus, a condition number &lt;span class=&quot;math-container&quot;&gt;$\kappa = \lambda_{max}/\lambda_{min}$&lt;/span&gt; that grows quadratically with N). So you can't really escape this fate just by fiddling around with FD schemes versus FE schemes, etc.&lt;/p&gt;&#xA;&lt;p&gt;This is problematic for (eg) Krylov solvers or other techniques whose convergence hinges upon condition number (and therefore will take more-than-linear time complexity to solve this problem). But you can get around the issue using multiresolution analysis (especially multigrid, which can solve this problem, and many others like it, in &lt;span class=&quot;math-container&quot;&gt;$O(N)$&lt;/span&gt; time). The convergence properties of multigrid are all based on this sort of spectral/fourier analysis.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2020-09-16T16:36:59.800" LastActivityDate="2020-09-16T16:36:59.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28451" PostTypeId="2" ParentId="28436" CreationDate="2017-12-14T18:08:31.007" Score="1" Body="&lt;p&gt;I think what you're looking for is a &lt;code&gt;surface-meshing&lt;/code&gt; algorithm. There are a multitude of different approaches and algorithms that can be used for these type of problems - I'll try to (briefly) outline a few options here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Parametric space approaches&lt;/strong&gt;: One approach is to build a mesh in a two-dimensional parametric space associated with the surface. While the meshing problem here becomes a 2D task (an obvious upside!), the construction of robust parametric mappings can be difficult. Such methods typically also require that the parametric space meshes be anisotropic, so that they are 'nicely shaped' when mapped back onto the 3D surface.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Delaunay-refinement methods&lt;/strong&gt;: It's also possible to build meshes in 3D space directly; dodging around the parametric mapping problem entirely. One way of doing this is to incrementally build a &lt;em&gt;restricted&lt;/em&gt; Delaunay triangulation of the surface - a subset of the triangular faces embedded in a 3D Delaunay tetrahedralisaton that form a good approximation to the surface. Typically, vertices are inserted one-by-one according to a given &lt;em&gt;refinement-strategy&lt;/em&gt; until the surface mesh is 'good enough'. One thing that (I think) is nice about these methods is that they come with various &lt;em&gt;provable guarantees&lt;/em&gt; on mesh quality, surface approximation error, and etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Variational techniques&lt;/strong&gt;: Meshing can also be cast as an optimisation problem: &lt;em&gt;find the set of vertex positions and mesh topology that maximise a given mesh-quality functional&lt;/em&gt;. Given some initial distribution of vertices, such methods essentially 'slide' vertices over the surface to improve the shape of the mesh; re-computing mesh topology (usually via Delaunay-type methods) as they go. Popular variational methods include Centroidal Voronoi Tessellation (CVT) and Optimal Delaunay Triangulation (ODT).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are a number of high quality meshing packages that can handle surface problems. To list a few: &lt;a href=&quot;https://doc.cgal.org/latest/Surface_mesher/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CGAL&lt;/a&gt;, &lt;a href=&quot;http://gmsh.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMSH&lt;/a&gt;, &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEOGRAM&lt;/a&gt;, &lt;a href=&quot;https://sourceforge.net/projects/netgen-mesher/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NETGEN&lt;/a&gt;, and (my own library) &lt;a href=&quot;https://github.com/dengwirda/jigsaw-matlab&quot; rel=&quot;nofollow noreferrer&quot;&gt;JIGSAW&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1139" LastActivityDate="2017-12-14T18:08:31.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28452" PostTypeId="2" ParentId="19176" CreationDate="2017-12-14T23:06:27.043" Score="-1" Body="&lt;p&gt;Can it be proved that the very existance of these convergence / divergence boundaries is stable in the face of numerical rounding artifacts? They seem to bear an uncanny resemblance to quantization error and sinc functions.&lt;/p&gt;&#xA;" OwnerUserId="26318" LastActivityDate="2017-12-14T23:06:27.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28454" PostTypeId="1" CreationDate="2017-12-15T08:25:39.347" Score="0" ViewCount="1403" Body="&lt;p&gt;I have understood the stiffness matrix for 3D truss, and programmed Ku=f from scratch (in Java) to find the displacements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I moved to 3D solid but lost in too many concepts and equations, such as shape function and so on. Books at hand: &lt;strong&gt;D. Hutton&lt;/strong&gt;'s Fundamentals of finite element analysis, &lt;strong&gt;O.C. Zienkiewicz&lt;/strong&gt;'s The finite element method, and &lt;strong&gt;T. Hughes&lt;/strong&gt;' The finite element method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing with truss, the solid should include shear forces. As in Chapter 7.3 of Hutton's book, $\gamma_{xy},\gamma_{yz}$ denote shear forces. Then the stiffness matrix should differs from that of truss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a paper for a specific chapter give a formula Ku=f for 3D regular grid (the simplest mesh)?&lt;/p&gt;&#xA;" OwnerUserId="26321" LastEditorUserId="9667" LastEditDate="2017-12-18T16:58:02.027" LastActivityDate="2019-07-09T20:19:03.243" Title="stiffness matrix for 3D regular grid in FEM" Tags="|finite-element|solid-mechanics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28455" PostTypeId="1" CreationDate="2017-12-15T15:37:03.123" Score="15" ViewCount="1184" Body="&lt;p&gt;I'm looking for resources on &lt;em&gt;design patterns and principles&lt;/em&gt; for numerical software, potentially but not necessarily with a focus on object-oriented approaches to numerical codes.&lt;/p&gt;&#xA;&lt;p&gt;I am aware of the text, &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0521675952&quot; rel=&quot;noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Writing Scientific Software: A Guide for Good Style&lt;/em&gt;&lt;/a&gt;, but this still deals a lot with things like floating point issues, how and when to optimise, etc.&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for information on the &lt;em&gt;bigger picture&lt;/em&gt; of designing numerical codes, not on anything that deals with algorithms and the like.&lt;/p&gt;&#xA;&lt;p&gt;I'd hope there are resources for this, and one isn't left to just going through library after library, especially since there's no guide on which do things correctly design wise, apart from my own knowledge of good idioms and practices in C++.&lt;/p&gt;&#xA;&lt;p&gt;(This is not language specific, but if there is any language specific content, I'd prefer C++.)&lt;/p&gt;&#xA;" OwnerUserId="26308" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-12-17T23:37:16.707" Title="Design patterns in writing numerical software in C++" Tags="|design-patterns|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28456" PostTypeId="1" CreationDate="2017-12-15T18:23:04.940" Score="6" ViewCount="209" Body="&lt;p&gt;I want to compute a fast matrix-vector product using a matrix $T$ which has a peculiar quasi-Hankel structure. For example,&#xA;\begin{equation}&#xA;T_2=&#xA;\left(&#xA;\begin{array}{c|ccc|cccccc}&#xA;a &amp;amp; b &amp;amp; c &amp;amp; d &amp;amp; e &amp;amp; f &amp;amp; g &amp;amp; h &amp;amp; i &amp;amp; j\\\hline&#xA;b &amp;amp; e &amp;amp; f &amp;amp; h &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;c &amp;amp; f &amp;amp; g &amp;amp; i &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;d &amp;amp; h &amp;amp; i &amp;amp; j &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\\hline&#xA;e &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;f &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;g &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;h &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;i &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;j &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0&#xA;\end{array}&#xA;\right).&#xA;\end{equation}&#xA;\begin{equation*}&#xA;T_3=\left(&#xA;\begin{array}{c|ccc|cccccc|cccccccccc}&#xA;a &amp;amp; b &amp;amp; c &amp;amp; d &amp;amp; e &amp;amp; f &amp;amp; g &amp;amp; h &amp;amp; i &amp;amp; j &amp;amp; k &amp;amp; l &amp;amp; m &amp;amp; n &amp;amp; o &amp;amp; p &amp;amp; q &amp;amp; r &amp;amp; s &amp;amp; t\\\hline&#xA;b &amp;amp; e &amp;amp; f &amp;amp; h &amp;amp; k &amp;amp; l &amp;amp; m &amp;amp; o &amp;amp; p &amp;amp; r\\&#xA;c &amp;amp; f &amp;amp; g &amp;amp; i &amp;amp; l &amp;amp; m &amp;amp; n &amp;amp; p &amp;amp; q &amp;amp; s\\&#xA;d &amp;amp; h &amp;amp; i &amp;amp; j &amp;amp; o &amp;amp; p &amp;amp; q &amp;amp; r &amp;amp; s &amp;amp; t\\\hline&#xA;e &amp;amp; k &amp;amp; l &amp;amp; o\\&#xA;f &amp;amp; l &amp;amp; m &amp;amp; p\\&#xA;g &amp;amp; m &amp;amp; n &amp;amp; q \\&#xA;h &amp;amp; o &amp;amp; p &amp;amp; r\\&#xA;i &amp;amp; p &amp;amp; q &amp;amp; s\\&#xA;j &amp;amp; r &amp;amp; s &amp;amp; t\\\hline&#xA;k &amp;amp; \\&#xA;l &amp;amp; \\&#xA;m &amp;amp; \\&#xA;n &amp;amp; \\&#xA;o &amp;amp; \\&#xA;p &amp;amp; \\&#xA;q &amp;amp; \\&#xA;r &amp;amp; \\&#xA;s &amp;amp; \\&#xA;t&#xA;\end{array}&#xA;\right)&#xA;\end{equation*}&#xA;Larger matrices $(T_4,T_5,...)$ have the same block structure; in fact, each successive $T_i$ is contained as a partial block of all $T_j,j&amp;gt;i$ (see above examples). All of the unique elements are contained in the first column or row, but the matrix does not possess classical Hankel/Toeplitz/etc. structure typical of fast structured matrix vector multiplication. This matrix arises from a convolution of a certain sort, so I am convinced that the matrix-vector product can be computed in $\mathcal{O}(N\log N)$ time or something close rather than $\mathcal{O}(N^2)$. I'd appreciate the input of others, or potentially helpful references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The block structure is controlled by the parameter $P$ so that the matrix $T_P$ has a $(P+1)\times(P+1)$ upper left triangular block structure. The $p$th row or column block is of dimension $(p+1)(p+2)/2$.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastEditorUserId="20688" LastEditDate="2018-05-17T00:10:59.473" LastActivityDate="2018-05-17T09:48:21.207" Title="Fast multiplication of highly structured matrix" Tags="|matrix|acceleration|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28457" PostTypeId="1" CreationDate="2017-12-16T00:54:32.990" Score="-1" ViewCount="456" Body="&lt;p&gt;I am currently programming a code to find the equilibrium function that satisfies the poisson equation in 2D. In order to do this I use finite difference methods and the discrete equation I want to satisfy is:&#xA;$$\frac{T_{j, i+1} + T_{j, i-1} + T_{j+1, i} + T_{j-1, i} - 4T_{j, i}}{h^2} = q_{j,i},$$&#xA;where $T_{j,i}$ is a temperature on a rectangular grid. In order to find the resulting array $\vec{T}$ that satisfies the equation I write it as a matrix equation:&#xA;$$\begin{pmatrix} -4 &amp;amp; 1 &amp;amp;  &amp;amp;  &amp;amp; \\ 1&amp;amp;-4&amp;amp;1&amp;amp;\\&amp;amp;\ddots&amp;amp;\ddots&amp;amp;\ddots&amp;amp; \\  &amp;amp;  &amp;amp;\ddots  &amp;amp;\ddots&amp;amp; 1\\  &amp;amp;  &amp;amp;  &amp;amp; 1 &amp;amp; -4 \end{pmatrix}\begin{pmatrix}T_1\\T_2\\\vdots  \\T_{N-1}  \\T_{N} \\ \end{pmatrix} = \begin{pmatrix}q_1\\q_2\\\vdots  \\q_{N-1}  \\q_{N} \\ \end{pmatrix}, $$&#xA;where the array $\vec{q}$ is a constant array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I iteratively solve this equation using the Gauss-Seidel method, however if I just run it forever and look at the average of $T$ after every iteration, it never converges to zero, i.e. if I run the iteration forever the average $T \rightarrow \infty$. Step-size by which the average temperature increases between iterations becomes constant eventually, but never goes the zero, how do I establish convergence of this method then, if not by a threshold on the change of the average temperature?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this imply my matrix is not convergent? I am fairly sure the matrix I use should be convergent, as it is widely claimed by many credible sources.&lt;/p&gt;&#xA;" OwnerUserId="26328" LastActivityDate="2018-04-15T04:33:48.703" Title="Gauss-Seidel method convergence" Tags="|iterative-method|matrix-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28458" PostTypeId="2" ParentId="28457" CreationDate="2017-12-16T02:20:43.763" Score="0" Body="&lt;p&gt;If you're continuously injecting heat into the system through the q function I can see the temperature continuously growing depending on your boundary conditions, but that's a formulation problem that you need to consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Halting iterative methods of solving equations is done using the residual:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have the system as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ A.x = b  $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We define the nth residual as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ r_n = (b - A.x_n) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you just have to determine a stopping criteria, usually a norm (L1, L2, Linf) of the absolute or relative residual, where the relative residual is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_{r,n} = r_n/r_i$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i is typically taken as zero, i.e. the first residual, but depending on a variety of factors you might instead choose to use the fifth or tenth residual after the system has settled a little.&lt;/p&gt;&#xA;" OwnerUserId="20410" LastActivityDate="2017-12-16T02:20:43.763" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28459" PostTypeId="1" CreationDate="2017-12-16T06:12:46.693" Score="1" ViewCount="340" Body="&lt;p&gt;I am currently solving PDEs using the finite volume method. The surface integrals of the equations that I am solving involves computing face gradients. The current algorithm that we use to compute face gradients is&#xA;$$\nabla\phi_{face}=\frac{1}{2}(\nabla\phi_L + \nabla\phi_R) - \boldsymbol{\hat{d}}(\frac{1}{2}(\nabla\phi_L + \nabla\phi_R)\cdot\boldsymbol{\hat{d}})+(\phi_L-\phi_R)\frac{\boldsymbol{\hat{d}}}{|\boldsymbol{d}|}$$&#xA;The LHS is the face gradient of some quantity $\phi$. $\nabla\phi_L$ and $\nabla\phi_R$ are the cell gradients at the left and right neighboring cells of the face, respectively. $\boldsymbol{\hat{d}}$ and $\boldsymbol{d}$ are the unit distance and distance vector between the left and right cell centers, respectively. The RHS consists of 3 terms. The latter 2 terms are the &quot;odd-even decoupling&quot; correction terms that are applied to avoid numerical oscillations that would otherwise arise if the computation of the face gradient was a simple averaging (i.e., just the first term on the RHS). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some questions regarding odd-even decoupling:&#xA;(1) Is odd-even decoupling also necessary for computing non-gradient values at the face? For example, if instead of computing the temperature gradient at the face, I instead wanted the temperature at that face. (2) &#xA;Is there a variant of this equation for non-gradient terms? For example, what if I just want to compute a face value? This equation can almost be duplicated for a plain $\phi$, non-gradient, term if it weren't for the 3rd term on the RHS. &lt;/p&gt;&#xA;" OwnerUserId="23915" LastEditorUserId="713" LastEditDate="2017-12-16T17:38:03.000" LastActivityDate="2017-12-16T17:38:03.000" Title="Odd-even decoupling at faces of cells" Tags="|finite-volume|oscillations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28460" PostTypeId="1" AcceptedAnswerId="28464" CreationDate="2017-12-16T10:22:33.157" Score="0" ViewCount="314" Body="&lt;p&gt;I would like to use Matlab's &lt;code&gt;pdepe&lt;/code&gt; to solve this system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ s_t =(sr)_x + s_{ xx } \\&#xA; r_t =(\frac{ A }{ B }r^2+s)_x + \frac{ A }{ -K } r_{ xx } $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$, $B$ and $K$ are constants. I need to solve this using MATLAB &lt;code&gt;pdepe&lt;/code&gt;. The problem is the nonlinear term in my equation: I don't know how to treat the nonlinear part. Boundary and initial conditions are not a problem in that part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I've tried to write my PDE in the form that &lt;code&gt;pdepe&lt;/code&gt; expects:&#xA;$$\left(  \begin{array}{c}    1 \\    1 \\ \end{array}\right) \times \frac{ \partial  }{ \partial t }\left(  \begin{array}{c}    s \\    r \\ \end{array}\right)=\left(  \begin{array}{c}    s\frac{ \partial r  }{ \partial x } + r\frac{ \partial s }{ \partial x } \\    2\frac{A}{B}r\frac{ \partial r }{ \partial x } +\frac{ \partial s }{ \partial x }\\ \end{array}\right)+\frac{ \partial  }{ \partial x }\left(  \begin{array}{c}    \frac{ \partial s  }{ \partial x } \\    \frac{A}{-K}\frac{ \partial r }{ \partial x } \\ \end{array}\right)+\left(  \begin{array}{c}    0 \\    0 \\ \end{array}\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [c,f,s] = eqtn(x,t,u,DuDx)&#xA;&#xA;c = 1;&#xA;&#xA;f = ????&#xA;&#xA;s = 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to code the function $f$?&lt;/p&gt;&#xA;" OwnerUserId="25912" LastEditorUserId="713" LastEditDate="2017-12-16T17:35:47.727" LastActivityDate="2017-12-16T17:35:47.727" Title="PDEPE nonlinear" Tags="|matlab|nonlinear-equations|numerical-modelling|parabolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28461" PostTypeId="1" AcceptedAnswerId="28462" CreationDate="2017-12-16T10:22:53.697" Score="2" ViewCount="170" Body="&lt;p&gt;With double precision, I get $\log(\exp(-3)+1)=0.048587351573741958$, which already has $4$ incorrect digits, and $\log(\exp(-30)+1)=9.348... \times10^{-14}$, which only has two correct digits. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the most elegant way to circumvent this? I tried a Taylor series of $\log(y)$ around $y=1$, but even with 14 terms I still didn't get all digits correctly for $x=-3$.&lt;/p&gt;&#xA;" OwnerUserId="7789" LastActivityDate="2017-12-16T18:05:58.270" Title="Evaluating $\log(\exp(x)+1)$ for negative $x$" Tags="|floating-point|precision|computer-arithmetic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28462" PostTypeId="2" ParentId="28461" CreationDate="2017-12-16T10:55:12.917" Score="12" Body="&lt;p&gt;Use the (IEEE standard) library function &lt;code&gt;log1p&lt;/code&gt;, which should be present in all programming languages. The function &lt;code&gt;log1p(x)&lt;/code&gt; returns $\log(1+x)$, and is implemented with particular attention to accuracy when $x$ is small. It is designed to solve exactly this kind of problem.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2017-12-16T17:56:39.820" LastActivityDate="2017-12-16T17:56:39.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28463" PostTypeId="2" ParentId="28455" CreationDate="2017-12-16T11:09:01.117" Score="7" Body="&lt;p&gt;I recommend this &#xA;Matthew G Knepley. Programming languages for scientific computing. arXiv preprint arXiv:1209.1711, 2012 (&lt;a href=&quot;https://arxiv.org/pdf/1209.1711.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1209.1711.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also advise to look and read other codes, for example, triangle, PETSc,  MoAB, deal.II, fenics and many others.  Not all solutions will work for you, but at the end, coping and learning from others (experienced developers and successful codes) is the best lesson how to design code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, I think is better to join to an existing project and become contributor or developer. Sometimes you like to kick start something new, but  you have to have solid arguments to do that.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2017-12-16T20:08:42.727" LastActivityDate="2017-12-16T20:08:42.727" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28464" PostTypeId="2" ParentId="28460" CreationDate="2017-12-16T13:03:33.967" Score="1" Body="&lt;p&gt;The most straightforward way to write the &lt;code&gt;eqtn&lt;/code&gt; function is&#xA;to define the nonlinear terms as part of the returned &lt;code&gt;s&lt;/code&gt; vector as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = [DuDx(1); -A/K*DuDx(2)];&#xA;s = [u(1)*DuDx(2)+u(2)*DuDx(1); 2*A/B*u(2)*DuDx(2)+DuDx(1)];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The question that immediately comes to mind is which terms are &#xA;appropriate to include in the &lt;code&gt;f&lt;/code&gt; vector compared to &lt;code&gt;s&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PDE system for many physical problems is derived based on the&#xA;notion of some quantity being conserved over time. This directly&#xA;leads to the idea of a &quot;flux&quot; which is what the &lt;code&gt;f&lt;/code&gt; vector represents.&#xA;That is, &lt;code&gt;f&lt;/code&gt; has a well-defined meaning in the physical problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The definition of &lt;code&gt;f&lt;/code&gt; is particularly important in defining boundary&#xA;conditions. Boundary conditions in &lt;code&gt;pdepe&lt;/code&gt; take the form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p + q*f=0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;at the right and left ends of the domain. If the boundary condition&#xA;has a non-zero &lt;code&gt;q&lt;/code&gt;, its value depends directly on how &lt;code&gt;f&lt;/code&gt; is defined.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2017-12-16T13:03:33.967" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28465" PostTypeId="2" ParentId="28461" CreationDate="2017-12-16T18:05:58.270" Score="4" Body="&lt;p&gt;If you want to use your own implementation to evaluate the function, take a look on &lt;a href=&quot;https://www.johndcook.com/blog/2012/07/25/trick-for-computing-log1x/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;. Otherwise, just go with log1p as suggested by @Frederico Poloni in his answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information about the log1p function read &lt;a href=&quot;https://www.johndcook.com/blog/2010/06/07/math-library-functions-that-seem-unnecessary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this other article&lt;/a&gt; from the same author.&lt;/p&gt;&#xA;" OwnerUserId="26289" LastActivityDate="2017-12-16T18:05:58.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28467" PostTypeId="2" ParentId="28449" CreationDate="2017-12-17T12:24:14.347" Score="1" Body="&lt;p&gt;The strength of the FE method is its ability to represent complex geometries. That way is it preferred method by engineers. Thus representation of holes or other geometrical features is for it natural.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, to solve your problem, you have to have a global enumeration of nodes. Then you iterate over finite elements, calculate local stiffness matrix and assemble it global stiffness matrix, by copying block of DOFs adjacent to nodes, one by one,  into the global matrix using global nodal indexes to find the right place. In your case, you have 6-DOFs at the node, i.e. three displacements and three rotations. You can find many books about that, for example, Zienkewicz and Taylor book or look at internet resources &lt;a href=&quot;https://www.colorado.edu/engineering/cas/courses.d/AFEM.d/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.colorado.edu/engineering/cas/courses.d/AFEM.d/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you raise two issues which are problematic, i.e. laminate plate (1) and non-conforming meshes (2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) For Kirchoff-Love or Reissner-Mindlin you can consider laminate plate by modifying the stiffness equation and appropriate integration through the thickness. I refer you to a vast number of papers in this area.  However, you can not stack Kirchoff-Love or Reissner-Mindlin FE element on top of each other; you will not capture adequately bending stiffness. If you like that, you need to modify formulation. I strongly believe the best solution is with using solid-shell elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) You consider the case when nodes on subsequent layers do not much, in that case, you have non-conforming mesh. In principle, you can not do that, unless you have a special methodology to resolve those type of problems.  You need to look at the mortar method or Nitsche method which both which provide a solution for non-conforming meshes.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="20688" LastEditDate="2018-07-15T15:39:20.077" LastActivityDate="2018-07-15T15:39:20.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28468" PostTypeId="1" AcceptedAnswerId="28503" CreationDate="2017-12-17T14:31:25.387" Score="1" ViewCount="217" Body="&lt;p&gt;Following my &lt;a href=&quot;https://scicomp.stackexchange.com/q/28409/25963&quot;&gt;previous question&lt;/a&gt; and its answer, after some reading of the advised books, I'm still confused about how to get the derivative of the functional to find the best parameter of my reaction diffusion problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, I need to find the best (time independent, space dependent) parameters $D$ and $k$ which minimize the functional&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathcal{G}(u,D,k) = \int_{\Omega} (u(t=T_f)-u_f)^2 d\Omega + \int_{\Omega} (u(t=0)-u_i)^2 d\Omega + \frac{1}{2}\int_{\Omega} D^2 d\Omega + \frac{1}{2}\int_{\Omega} k^2 d\Omega $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to $$ \partial_t{u} - \nabla \cdot (D\nabla{u}) - k u (1-u) = 0  $$&#xA;$$ u(t=0,x)=u_i \space \forall x$$&#xA;$$ u(t,x)=0 \space \forall x \in \Gamma $$&#xA;where $\Gamma$ is the border of $\Omega$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the adjoint method,  I define the Lagrangian $\cal L$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathcal{L} = \mathcal{G} - \int_t\int_\Omega{\lambda( \partial_t{u} - \nabla \cdot (D\nabla{u}) - k u (1-u)) d\Omega.dt }$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for exemple, in order to find the &lt;em&gt;optimal equations&lt;/em&gt; according to the parameter $D$, my understanding is that I need to specify that&#xA;$$ \frac{\mathcal{L}(u,D+\epsilon\tilde D,k) - \mathcal{L}(u,D,k)}{\epsilon}=0 \space\space \forall{\tilde D} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which gives me (considering $u$, $D$, $k$ and $\lambda$ as independent variables)&#xA;$$  \int_\Omega{D \tilde D d\Omega} +  \int_t\int_\Omega{\lambda \nabla \cdot (\tilde D \nabla u) d\Omega dt} = 0 \space \forall {\tilde D}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using integration by parts, I get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega{D \tilde D} d\Omega + \int_t\int_\Omega{\nabla (\lambda\tilde D\nabla u) d\Omega dt} - \int_t\int_\Omega{\tilde D\nabla u \nabla\lambda d\Omega dt }= 0 \space \forall {\tilde D}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and using the divergence theorem,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega{D \tilde D} d\Omega + \int_t\oint_\Gamma{(\lambda\tilde D\nabla u) {\vec n} d\Gamma dt} - \int_t\int_\Omega{\tilde D \nabla u \nabla\lambda  d\Omega dt }= 0 \space \forall {\tilde D}$$&#xA;the goal of these transformation being to factorize the expression with $\tilde D$ in order to get a relation between $D$, $k$, $\lambda$, $u$ and their derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, with such an expression, while the $\tilde D$ is indeed factorized, the domains of each part of the equation are different (the first is time independent, the second is a surface integral and the third is volumetric), so I can't merge them together to get a unique equation like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D + \lambda\nabla u - \nabla u\nabla\lambda =0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can eventually require each part to be 0 at the same time, but then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_\Omega{D \tilde D} d\Omega= 0 \space \forall {\tilde D}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;leads just to $D=0$ which is obviously wrong...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be very welcome...&lt;/p&gt;&#xA;" OwnerUserId="25963" LastEditorUserId="25963" LastEditDate="2017-12-19T20:20:06.767" LastActivityDate="2017-12-27T09:18:08.707" Title="proper derivation of a functional for a time dependent parameter estimation problem" Tags="|pde|optimal-control|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28469" PostTypeId="1" CreationDate="2017-12-17T15:28:57.437" Score="2" ViewCount="272" Body="&lt;p&gt;This was a question on a past final that we can't figure out. Take the least squares system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_x ||Ax-b||_2\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A\in\mathbb{R}^{mxn}$, $m&amp;lt;n$, and A is full rank. A has $\mathcal{O}(n)$ non-zero entries. Assume you only have $\mathcal{O}(n)$ computer memory.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Suggest an algorithm for approximating the solution to the least squares problem&lt;/li&gt;&#xA;&lt;li&gt;What is the expected number of iterations and overall computational cost?&lt;/li&gt;&#xA;&lt;li&gt;What is the expected error in the solution x? (Provide an appropriate definition of error for this problem)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We've only learned a little bit about iterative methods, just like Rayleigh Quotient, Inverse Iteration, Power Iteration, and Conjugate Gradient. None of these seem to help. Any ideas would be much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I'm thinking now, what if we take a random mm columns so that a square matrix can be formed, and then use CG on that problem? And then do this with various random columns to find average solutions?&lt;/p&gt;&#xA;" OwnerUserId="26334" LastEditorUserId="9667" LastEditDate="2017-12-17T17:55:30.017" LastActivityDate="2018-01-03T18:19:16.270" Title="Solving sparse least squares system with limited memory" Tags="|linear-algebra|iterative-method|least-squares|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28471" PostTypeId="1" CreationDate="2017-12-17T20:27:11.570" Score="1" ViewCount="58" Body="&lt;p&gt;It is known that there exist a number of radioisotopes of elements commonly used in computer systems. Is the decay of these materials known to affect device performance over time, or is its impact so small as to be immeasurable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question occurred to me last night as a friend &quot;cleaned the N64 cartridge&quot; by blowing on the contacts. I've also heard that cosmic ray collisions regularly affect satellite computation. There are a number of &quot;material parameters&quot; to computational science we, as developers and computer scientists, rarely consider.&lt;/p&gt;&#xA;" OwnerDisplayName="user26339" LastEditorUserId="17294" LastEditDate="2017-12-17T20:40:39.873" LastActivityDate="2017-12-17T20:40:39.873" Title="Does radioisotope decay affect computer systems?" Tags="|computational-physics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28472" PostTypeId="2" ParentId="28455" CreationDate="2017-12-17T21:33:02.073" Score="9" Body="&lt;p&gt;Having developed numerical software for 20 years now, I find that we've probably used the majority of the design patterns described in the Book by the Gang of Four (&lt;a href=&quot;https://stackoverflow.com/questions/14831606/what-is-gang-of-four-design-pattern&quot;&gt;see here&lt;/a&gt;). At the end of the day, numerical software is just large-scale, complex software as well, so it is maybe not surprising that the same design considerations apply as in other software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are, of course, a number of cases that really are specific to numerical algorithms (see, for example, &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/publications/2013-pattern.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;) but knowing the &quot;traditional&quot; set of design patterns is really a good start already.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-12-17T21:33:02.073" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28473" PostTypeId="1" CreationDate="2017-12-18T12:51:56.880" Score="1" ViewCount="123" Body="&lt;p&gt;Single-component Euler equations are solved with finite-difference WENO methods very well. Now I'm trying to apply them to gas mixtures (with aim to reacting mixtures).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While searching for extension to multi-component equations, I find very little on finite-difference WENO methods for them, mostly there are finite-volume formulations. &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0021999114004082&quot; rel=&quot;nofollow noreferrer&quot;&gt;One paper&lt;/a&gt; even containts the phrase: &quot;... Care must&#xA;then be taken when solving the resulting governing equations. &lt;strong&gt;They must be cast in a finite-volume framework&lt;/strong&gt; and discretized with a non-oscillatory spatial and temporal method, with the primitive state variables, rather than the conservative ones, spatially reconstructed&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it impossible to achieve all needed equilibriums (mass, momentum, energy) across interfaces in finite-difference formulation? &#xA;If it is possible, are there any widely accepted FD WENO methods for gas mixtures? &lt;/p&gt;&#xA;" OwnerUserId="25060" LastActivityDate="2017-12-29T18:05:36.423" Title="Is it possible to solve multi-component Euler equations with finite-difference WENO methods?" Tags="|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28475" PostTypeId="2" ParentId="28454" CreationDate="2017-12-18T17:26:55.557" Score="2" Body="&lt;p&gt;To understand what you are doing you need to dance along with all those &lt;em&gt;many concepts and equations&lt;/em&gt;. There is no shortcut there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are asking about an explicit expression for the local stiffness matrix for a trilinear element for elasticity, I would say that it might be unpractical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stiffness matrix is computed by the following integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ K = \int\limits_{[-1,1]^3} B^T C B\, |J| \mathrm{d}r\, \mathrm{d}s\, \mathrm{d}t$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; the stiffness tensor in &lt;a href=&quot;https://en.wikipedia.org/wiki/Voigt_notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Voigt notation&lt;/a&gt;, and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; the displacement-to-strain matrix. The 3 columns related to node &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$B = \begin{bmatrix}&#xA;\frac{\mathrm{d} N_i}{\mathrm{d}x} &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp;\frac{\mathrm{d} N_i}{\mathrm{d}y} &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp;\frac{\mathrm{d} N_i}{\mathrm{d}z}\\&#xA;0 &amp;amp;\frac{\mathrm{d} N_i}{\mathrm{d}z} &amp;amp; \frac{\mathrm{d} N_i}{\mathrm{d}y}\\&#xA;\frac{\mathrm{d} N_i}{\mathrm{d}z} &amp;amp;0 &amp;amp; \frac{\mathrm{d} N_i}{\mathrm{d}x}\\&#xA;\frac{\mathrm{d} N_i}{dy} &amp;amp;\frac{\mathrm{d} N_i}{\mathrm{d}x} &amp;amp; 0&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the remaining columns are similar, just change the index &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. The indices correspond to the interpolation functions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$N = \begin{bmatrix}&#xA;\frac{1}{8} \left(- r + 1\right) \left(- s + 1\right) \left(t + 1\right)\\&#xA;\frac{1}{8} \left(r + 1\right) \left(- s + 1\right) \left(t + 1\right)\\&#xA;\frac{1}{8} \left(r + 1\right) \left(s + 1\right) \left(t + 1\right)\\&#xA;\frac{1}{8} \left(- r + 1\right) \left(s + 1\right) \left(t + 1\right)\\&#xA;\frac{1}{8} \left(- r + 1\right) \left(- s + 1\right) \left(- t + 1\right)\\&#xA;\frac{1}{8} \left(r + 1\right) \left(- s + 1\right) \left(- t + 1\right)\\&#xA;\frac{1}{8} \left(r + 1\right) \left(s + 1\right) \left(- t + 1\right)\\&#xA;\frac{1}{8} \left(- r + 1\right) \left(s + 1\right) \left(- t + 1\right)&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the derivatives in &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are in &lt;span class=&quot;math-container&quot;&gt;$(x, y, z)$&lt;/span&gt; and the interpolation functions are in &lt;span class=&quot;math-container&quot;&gt;$(r, s, t)$&lt;/span&gt;. And you would need to compute the chain derivates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we express the stiffness tensor as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C = \lambda C_1 + \mu C_2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C_1 = \begin{bmatrix}&#xA;1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;1 &amp;amp; 1 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C_2 = \begin{bmatrix}&#xA;2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And you split your stiffness matrix as &lt;span class=&quot;math-container&quot;&gt;$K = K_1 + K_2$&lt;/span&gt;, with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{288 K_1}{L}= \begin{bmatrix}&#xA;8 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; 6 &amp;amp; -6 &amp;amp; -4 &amp;amp; -6 &amp;amp; -3 &amp;amp; 4 &amp;amp; -6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 3 &amp;amp; 6 &amp;amp; -4 &amp;amp; 3 &amp;amp; 6 &amp;amp; -2 &amp;amp; -3 &amp;amp; 3 &amp;amp; 2 &amp;amp; -3 &amp;amp; 3\\&#xA;6 &amp;amp; 8 &amp;amp; -6 &amp;amp; -6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; -4 &amp;amp; -3 &amp;amp; 6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -3 &amp;amp; 2 &amp;amp; 3 &amp;amp; -3 &amp;amp; -2 &amp;amp; 3 &amp;amp; 3 &amp;amp; -4 &amp;amp; 6\\&#xA;-6 &amp;amp; -6 &amp;amp; 8 &amp;amp; 6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 3 &amp;amp; 3 &amp;amp; 2 &amp;amp; -3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -6 &amp;amp; -6 &amp;amp; -8 &amp;amp; 6 &amp;amp; -3 &amp;amp; -4 &amp;amp; 3 &amp;amp; 3 &amp;amp; -2 &amp;amp; -3 &amp;amp; 6 &amp;amp; -4\\&#xA;-8 &amp;amp; -6 &amp;amp; 6 &amp;amp; 8 &amp;amp; -6 &amp;amp; 6 &amp;amp; 4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 2 &amp;amp; 3 &amp;amp; -3 &amp;amp; -2 &amp;amp; 3 &amp;amp; -3\\&#xA;6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 8 &amp;amp; -6 &amp;amp; -6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6 &amp;amp; -4 &amp;amp; -3 &amp;amp; 3 &amp;amp; 2 &amp;amp; 3 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -3 &amp;amp; -4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -2 &amp;amp; 3\\&#xA;-6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -6 &amp;amp; 8 &amp;amp; 3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; 3 &amp;amp; 2 &amp;amp; -6 &amp;amp; -3 &amp;amp; -4 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; 3 &amp;amp; 6 &amp;amp; -4 &amp;amp; -3 &amp;amp; 3 &amp;amp; -2\\&#xA;-4 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 3 &amp;amp; 8 &amp;amp; 6 &amp;amp; 6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; -2 &amp;amp; -3 &amp;amp; -3 &amp;amp; 2 &amp;amp; -3 &amp;amp; -3 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; -4 &amp;amp; 3 &amp;amp; -6\\&#xA;-6 &amp;amp; -4 &amp;amp; 3 &amp;amp; 6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; 8 &amp;amp; 6 &amp;amp; -6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -3 &amp;amp; -2 &amp;amp; -3 &amp;amp; 3 &amp;amp; -4 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; -3 &amp;amp; 2 &amp;amp; -3\\&#xA;-3 &amp;amp; -3 &amp;amp; 2 &amp;amp; 3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 6 &amp;amp; 6 &amp;amp; 8 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -3 &amp;amp; -3 &amp;amp; -2 &amp;amp; 3 &amp;amp; -6 &amp;amp; -4 &amp;amp; 6 &amp;amp; 6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 3 &amp;amp; -4\\&#xA;4 &amp;amp; 6 &amp;amp; -3 &amp;amp; -4 &amp;amp; 6 &amp;amp; -3 &amp;amp; -8 &amp;amp; -6 &amp;amp; -6 &amp;amp; 8 &amp;amp; -6 &amp;amp; -6 &amp;amp; 2 &amp;amp; 3 &amp;amp; 3 &amp;amp; -2 &amp;amp; 3 &amp;amp; 3 &amp;amp; -4 &amp;amp; -3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; 6\\&#xA;-6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; -4 &amp;amp; 3 &amp;amp; 6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; 8 &amp;amp; 6 &amp;amp; -3 &amp;amp; -4 &amp;amp; -6 &amp;amp; 3 &amp;amp; -2 &amp;amp; -3 &amp;amp; 3 &amp;amp; 2 &amp;amp; -3 &amp;amp; -3 &amp;amp; 4 &amp;amp; -6\\&#xA;-3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -3 &amp;amp; 2 &amp;amp; 6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 6 &amp;amp; 8 &amp;amp; -3 &amp;amp; -6 &amp;amp; -4 &amp;amp; 3 &amp;amp; -3 &amp;amp; -2 &amp;amp; 6 &amp;amp; 3 &amp;amp; -4 &amp;amp; -6 &amp;amp; 6 &amp;amp; -8\\&#xA;4 &amp;amp; 3 &amp;amp; -6 &amp;amp; -4 &amp;amp; 3 &amp;amp; -6 &amp;amp; -2 &amp;amp; -3 &amp;amp; -3 &amp;amp; 2 &amp;amp; -3 &amp;amp; -3 &amp;amp; 8 &amp;amp; 6 &amp;amp; 6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; -4 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 3\\&#xA;3 &amp;amp; 4 &amp;amp; -6 &amp;amp; -3 &amp;amp; 2 &amp;amp; -3 &amp;amp; -3 &amp;amp; -2 &amp;amp; -3 &amp;amp; 3 &amp;amp; -4 &amp;amp; -6 &amp;amp; 6 &amp;amp; 8 &amp;amp; 6 &amp;amp; -6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; -4 &amp;amp; 3 &amp;amp; 6 &amp;amp; -8 &amp;amp; 6\\&#xA;6 &amp;amp; 6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -3 &amp;amp; -2 &amp;amp; 3 &amp;amp; -6 &amp;amp; -4 &amp;amp; 6 &amp;amp; 6 &amp;amp; 8 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -3 &amp;amp; -3 &amp;amp; 2 &amp;amp; 3 &amp;amp; -6 &amp;amp; 4\\&#xA;-4 &amp;amp; -3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; 6 &amp;amp; 2 &amp;amp; 3 &amp;amp; 3 &amp;amp; -2 &amp;amp; 3 &amp;amp; 3 &amp;amp; -8 &amp;amp; -6 &amp;amp; -6 &amp;amp; 8 &amp;amp; -6 &amp;amp; -6 &amp;amp; 4 &amp;amp; 6 &amp;amp; -3 &amp;amp; -4 &amp;amp; 6 &amp;amp; -3\\&#xA;3 &amp;amp; 2 &amp;amp; -3 &amp;amp; -3 &amp;amp; 4 &amp;amp; -6 &amp;amp; -3 &amp;amp; -4 &amp;amp; -6 &amp;amp; 3 &amp;amp; -2 &amp;amp; -3 &amp;amp; 6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; 8 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; -4 &amp;amp; 3\\&#xA;6 &amp;amp; 3 &amp;amp; -4 &amp;amp; -6 &amp;amp; 6 &amp;amp; -8 &amp;amp; -3 &amp;amp; -6 &amp;amp; -4 &amp;amp; 3 &amp;amp; -3 &amp;amp; -2 &amp;amp; 6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 6 &amp;amp; 8 &amp;amp; -3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -3 &amp;amp; 2\\&#xA;-2 &amp;amp; -3 &amp;amp; 3 &amp;amp; 2 &amp;amp; -3 &amp;amp; 3 &amp;amp; 4 &amp;amp; 3 &amp;amp; 6 &amp;amp; -4 &amp;amp; 3 &amp;amp; 6 &amp;amp; -4 &amp;amp; -6 &amp;amp; -3 &amp;amp; 4 &amp;amp; -6 &amp;amp; -3 &amp;amp; 8 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; 6 &amp;amp; -6\\&#xA;-3 &amp;amp; -2 &amp;amp; 3 &amp;amp; 3 &amp;amp; -4 &amp;amp; 6 &amp;amp; 3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -3 &amp;amp; 2 &amp;amp; 3 &amp;amp; -6 &amp;amp; -4 &amp;amp; -3 &amp;amp; 6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6 &amp;amp; 8 &amp;amp; -6 &amp;amp; -6 &amp;amp; 4 &amp;amp; -3\\&#xA;3 &amp;amp; 3 &amp;amp; -2 &amp;amp; -3 &amp;amp; 6 &amp;amp; -4 &amp;amp; -6 &amp;amp; -6 &amp;amp; -8 &amp;amp; 6 &amp;amp; -3 &amp;amp; -4 &amp;amp; 3 &amp;amp; 3 &amp;amp; 2 &amp;amp; -3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -6 &amp;amp; -6 &amp;amp; 8 &amp;amp; 6 &amp;amp; -3 &amp;amp; 4\\&#xA;2 &amp;amp; 3 &amp;amp; -3 &amp;amp; -2 &amp;amp; 3 &amp;amp; -3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6 &amp;amp; 8 &amp;amp; -6 &amp;amp; 6\\&#xA;-3 &amp;amp; -4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -2 &amp;amp; 3 &amp;amp; 3 &amp;amp; 2 &amp;amp; 3 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6 &amp;amp; -4 &amp;amp; -3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 8 &amp;amp; -6\\&#xA;3 &amp;amp; 6 &amp;amp; -4 &amp;amp; -3 &amp;amp; 3 &amp;amp; -2 &amp;amp; -6 &amp;amp; -3 &amp;amp; -4 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; 3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; 3 &amp;amp; 2 &amp;amp; -6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -6 &amp;amp; 8&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{288 K_2}{L} = \begin{bmatrix}&#xA;32 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6 &amp;amp; -10 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; -10 &amp;amp; -3 &amp;amp; 6 &amp;amp; -8 &amp;amp; -3 &amp;amp; 3 &amp;amp; -4 &amp;amp; 3 &amp;amp; -3\\&#xA;6 &amp;amp; 32 &amp;amp; -6 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; -10 &amp;amp; 3 &amp;amp; -6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -3 &amp;amp; -8 &amp;amp; 3 &amp;amp; -3 &amp;amp; -10 &amp;amp; 6\\&#xA;-6 &amp;amp; -6 &amp;amp; 32 &amp;amp; -6 &amp;amp; -3 &amp;amp; 4 &amp;amp; -3 &amp;amp; -3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 6 &amp;amp; 6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 3 &amp;amp; -10 &amp;amp; 3 &amp;amp; 3 &amp;amp; -8 &amp;amp; 3 &amp;amp; 6 &amp;amp; -10\\&#xA;-8 &amp;amp; 6 &amp;amp; -6 &amp;amp; 32 &amp;amp; -6 &amp;amp; 6 &amp;amp; 4 &amp;amp; -6 &amp;amp; 3 &amp;amp; -10 &amp;amp; 6 &amp;amp; -3 &amp;amp; -10 &amp;amp; 3 &amp;amp; -6 &amp;amp; 4 &amp;amp; -3 &amp;amp; 6 &amp;amp; -4 &amp;amp; -3 &amp;amp; 3 &amp;amp; -8 &amp;amp; 3 &amp;amp; -3\\&#xA;-6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 32 &amp;amp; -6 &amp;amp; 6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; -10 &amp;amp; 3 &amp;amp; -3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 3 &amp;amp; -10 &amp;amp; 6 &amp;amp; 3 &amp;amp; -8 &amp;amp; 3\\&#xA;6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -6 &amp;amp; 32 &amp;amp; 3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -3 &amp;amp; -4 &amp;amp; -6 &amp;amp; 3 &amp;amp; -10 &amp;amp; -6 &amp;amp; 6 &amp;amp; -8 &amp;amp; -3 &amp;amp; 6 &amp;amp; -10 &amp;amp; -3 &amp;amp; 3 &amp;amp; -8\\&#xA;-10 &amp;amp; -6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; 3 &amp;amp; 32 &amp;amp; 6 &amp;amp; 6 &amp;amp; -8 &amp;amp; -6 &amp;amp; -6 &amp;amp; -8 &amp;amp; -3 &amp;amp; -3 &amp;amp; -4 &amp;amp; 3 &amp;amp; 3 &amp;amp; 4 &amp;amp; 3 &amp;amp; 6 &amp;amp; -10 &amp;amp; -3 &amp;amp; -6\\&#xA;-6 &amp;amp; -10 &amp;amp; -3 &amp;amp; -6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6 &amp;amp; 32 &amp;amp; 6 &amp;amp; 6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -3 &amp;amp; -8 &amp;amp; -3 &amp;amp; -3 &amp;amp; -10 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -4 &amp;amp; 3\\&#xA;3 &amp;amp; 3 &amp;amp; -4 &amp;amp; 3 &amp;amp; 6 &amp;amp; 4 &amp;amp; 6 &amp;amp; 6 &amp;amp; 32 &amp;amp; 6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -3 &amp;amp; -3 &amp;amp; -8 &amp;amp; -3 &amp;amp; -6 &amp;amp; -10 &amp;amp; -6 &amp;amp; -6 &amp;amp; -8 &amp;amp; -6 &amp;amp; -3 &amp;amp; -10\\&#xA;4 &amp;amp; -6 &amp;amp; -3 &amp;amp; -10 &amp;amp; 6 &amp;amp; 3 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; 32 &amp;amp; -6 &amp;amp; -6 &amp;amp; -4 &amp;amp; -3 &amp;amp; -3 &amp;amp; -8 &amp;amp; 3 &amp;amp; 3 &amp;amp; -10 &amp;amp; 3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6\\&#xA;6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6 &amp;amp; -10 &amp;amp; -3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; 32 &amp;amp; 6 &amp;amp; 3 &amp;amp; -10 &amp;amp; -6 &amp;amp; 3 &amp;amp; -8 &amp;amp; -3 &amp;amp; -3 &amp;amp; -4 &amp;amp; 3 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6\\&#xA;-3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; 3 &amp;amp; -4 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 6 &amp;amp; 32 &amp;amp; 3 &amp;amp; -6 &amp;amp; -10 &amp;amp; 3 &amp;amp; -3 &amp;amp; -8 &amp;amp; 6 &amp;amp; -3 &amp;amp; -10 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8\\&#xA;4 &amp;amp; 3 &amp;amp; 6 &amp;amp; -10 &amp;amp; -3 &amp;amp; -6 &amp;amp; -8 &amp;amp; -3 &amp;amp; -3 &amp;amp; -4 &amp;amp; 3 &amp;amp; 3 &amp;amp; 32 &amp;amp; 6 &amp;amp; 6 &amp;amp; -8 &amp;amp; -6 &amp;amp; -6 &amp;amp; -10 &amp;amp; -6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; 3\\&#xA;3 &amp;amp; 4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -4 &amp;amp; 3 &amp;amp; -3 &amp;amp; -8 &amp;amp; -3 &amp;amp; -3 &amp;amp; -10 &amp;amp; -6 &amp;amp; 6 &amp;amp; 32 &amp;amp; 6 &amp;amp; 6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; -10 &amp;amp; -3 &amp;amp; -6 &amp;amp; -8 &amp;amp; -6\\&#xA;-6 &amp;amp; -6 &amp;amp; -8 &amp;amp; -6 &amp;amp; -3 &amp;amp; -10 &amp;amp; -3 &amp;amp; -3 &amp;amp; -8 &amp;amp; -3 &amp;amp; -6 &amp;amp; -10 &amp;amp; 6 &amp;amp; 6 &amp;amp; 32 &amp;amp; 6 &amp;amp; 3 &amp;amp; 4 &amp;amp; 3 &amp;amp; 3 &amp;amp; -4 &amp;amp; 3 &amp;amp; 6 &amp;amp; 4\\&#xA;-10 &amp;amp; 3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; -4 &amp;amp; -3 &amp;amp; -3 &amp;amp; -8 &amp;amp; 3 &amp;amp; 3 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; 32 &amp;amp; -6 &amp;amp; -6 &amp;amp; 4 &amp;amp; -6 &amp;amp; -3 &amp;amp; -10 &amp;amp; 6 &amp;amp; 3\\&#xA;-3 &amp;amp; -4 &amp;amp; 3 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; 3 &amp;amp; -10 &amp;amp; -6 &amp;amp; 3 &amp;amp; -8 &amp;amp; -3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; 32 &amp;amp; 6 &amp;amp; 6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6 &amp;amp; -10 &amp;amp; -3\\&#xA;6 &amp;amp; -3 &amp;amp; -10 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; 3 &amp;amp; -6 &amp;amp; -10 &amp;amp; 3 &amp;amp; -3 &amp;amp; -8 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 6 &amp;amp; 32 &amp;amp; -3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3 &amp;amp; 3 &amp;amp; -4\\&#xA;-8 &amp;amp; -3 &amp;amp; 3 &amp;amp; -4 &amp;amp; 3 &amp;amp; -3 &amp;amp; 4 &amp;amp; 3 &amp;amp; -6 &amp;amp; -10 &amp;amp; -3 &amp;amp; 6 &amp;amp; -10 &amp;amp; -6 &amp;amp; 3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -3 &amp;amp; 32 &amp;amp; 6 &amp;amp; -6 &amp;amp; -8 &amp;amp; -6 &amp;amp; 6\\&#xA;-3 &amp;amp; -8 &amp;amp; 3 &amp;amp; -3 &amp;amp; -10 &amp;amp; 6 &amp;amp; 3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -6 &amp;amp; -10 &amp;amp; 3 &amp;amp; -6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; 32 &amp;amp; -6 &amp;amp; 6 &amp;amp; 4 &amp;amp; -3\\&#xA;3 &amp;amp; 3 &amp;amp; -8 &amp;amp; 3 &amp;amp; 6 &amp;amp; -10 &amp;amp; 6 &amp;amp; 6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 3 &amp;amp; -10 &amp;amp; -3 &amp;amp; -3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 4 &amp;amp; -6 &amp;amp; -6 &amp;amp; 32 &amp;amp; -6 &amp;amp; -3 &amp;amp; 4\\&#xA;-4 &amp;amp; -3 &amp;amp; 3 &amp;amp; -8 &amp;amp; 3 &amp;amp; -3 &amp;amp; -10 &amp;amp; 3 &amp;amp; -6 &amp;amp; 4 &amp;amp; -3 &amp;amp; 6 &amp;amp; 4 &amp;amp; -6 &amp;amp; 3 &amp;amp; -10 &amp;amp; 6 &amp;amp; -3 &amp;amp; -8 &amp;amp; 6 &amp;amp; -6 &amp;amp; 32 &amp;amp; -6 &amp;amp; 6\\&#xA;3 &amp;amp; -10 &amp;amp; 6 &amp;amp; 3 &amp;amp; -8 &amp;amp; 3 &amp;amp; -3 &amp;amp; -4 &amp;amp; -3 &amp;amp; -3 &amp;amp; 4 &amp;amp; -6 &amp;amp; 6 &amp;amp; -8 &amp;amp; 6 &amp;amp; 6 &amp;amp; -10 &amp;amp; 3 &amp;amp; -6 &amp;amp; 4 &amp;amp; -3 &amp;amp; -6 &amp;amp; 32 &amp;amp; -6\\&#xA;-3 &amp;amp; 6 &amp;amp; -10 &amp;amp; -3 &amp;amp; 3 &amp;amp; -8 &amp;amp; -6 &amp;amp; 3 &amp;amp; -10 &amp;amp; -6 &amp;amp; 6 &amp;amp; -8 &amp;amp; 3 &amp;amp; -6 &amp;amp; 4 &amp;amp; 3 &amp;amp; -3 &amp;amp; -4 &amp;amp; 6 &amp;amp; -3 &amp;amp; 4 &amp;amp; 6 &amp;amp; -6 &amp;amp; 32&#xA;\end{bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases, the matrix is lacking the factor to change coordinates &lt;span class=&quot;math-container&quot;&gt;$L/8$&lt;/span&gt; and are multiplied by 36 to make numbers prettier. Still, after all this cosmetic manipulation of the computation, it is not practical to use these &lt;span class=&quot;math-container&quot;&gt;$24\times 24$&lt;/span&gt; matrices explicitly.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2019-07-09T20:19:03.243" LastActivityDate="2019-07-09T20:19:03.243" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="28476" PostTypeId="1" AcceptedAnswerId="28480" CreationDate="2017-12-19T02:29:39.647" Score="0" ViewCount="151" Body="&lt;p&gt;You can find the description of the inertial partition algorithm either in &lt;a href=&quot;http://yifanhu.net/PROJECT/pdcp_siam/node6.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt; or &lt;a href=&quot;http://people.eecs.berkeley.edu/~demmel/cs267/lecture18/lecture18.html#link_3.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think, I understood the main idea behind this algorithm: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We have the spatial coordinates of points that we want to bisect. We find a line (or, hyperplane, in general) $L$ through these points, such that it is a total least squares fit of the nodes. We project the points onto this line. We find the &quot;median projection&quot;, which then is used to split the points into two partitions: that is, the perpendicular hyperplane to $L$ that goes through the &quot;median projection&quot; divides the points into two partitions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, I have a doubt regarding the theory behind the inertial partitioning algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, it turns out that the idea above can be framed as an eigenvalue-eigenvector problem (as for &lt;em&gt;spectral partitioning&lt;/em&gt; algorithm). More specifically, it turns out, after a derivation (see &lt;a href=&quot;http://people.eecs.berkeley.edu/~demmel/cs267/lecture18/lecture18.html#link_3.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;), we want to minimise the quantity $$\vec{u}^T M  \vec{u} = \lambda,$$ i.e. find $\vec{u}$ such that $\lambda$ is minimised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;http://people.eecs.berkeley.edu/~demmel/cs267/lecture18/lecture18.html#link_3.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;, $u=[a, b]$ is the unit eigenvector of $M$ corresponding to the smallest eigenvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that $\vec{u}$ is a &lt;strong&gt;unit&lt;/strong&gt; eigenvector, I suppose, is related to the fact that the author of &lt;a href=&quot;http://people.eecs.berkeley.edu/~demmel/cs267/lecture18/lecture18.html#link_3.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt; assumes (w.l.o.g.) that $a^2 + b^2 = 1$. Why does the author assume $a^2 + b^2 = 1$? Why would that be convenient? Why is the assumption w.l.o.g.? Note $-\frac{a}{b}$ is the slow of the line $L$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the assumption $a^2 + b^2 = 1$, I understand that $\vec{u} = [a, b]$ is a unit eigenvector, because $\|\vec{u}\| = \sqrt{a^2 + b^2} = 1 \iff a^2 + b^2 = 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is $\vec{u}$ the eigenvector corresponding to the smallest eigenvalue of $M$? Why do we care about the smallest eigenvalue of $M$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once we have found $\vec{u}$, we have the slow of the line. We can also retrieve the center of mass, that is $(\bar{x},\bar{y})$, which is used to formulate the equation of the line of $L$: it is basically an average of the points.&lt;/p&gt;&#xA;" OwnerDisplayName="user9925" LastActivityDate="2017-12-19T17:02:17.297" Title="Linear algebra of the inertial partition algorithm" Tags="|algorithms|partitioning|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28477" PostTypeId="1" AcceptedAnswerId="28478" CreationDate="2017-12-19T06:58:16.633" Score="2" ViewCount="567" Body="&lt;p&gt;I have a problem as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We have a set of circles (we know the radius &lt;code&gt;r&lt;/code&gt; and the center point &lt;code&gt;c&lt;/code&gt; in R&lt;sup&gt;d&lt;/sup&gt; of each circle)&lt;/li&gt;&#xA;&lt;li&gt;We need to find lists. Each list contains circles that overlap and connect each other. &lt;/li&gt;&#xA;&lt;li&gt;For example, we have 6 circles in 2D as bellows. We need to find two lists. The first list contains red, yellow, orange, and magenta circles. The second list contains blue and green circles.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An obvious solution is to check overlap of each pair, then find connected components. However, the complexity is O(N&lt;sup&gt;2&lt;/sup&gt;). My question is whether there is any better solution. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/syKq1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/syKq1.png&quot; alt=&quot;example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26347" LastEditorUserId="26347" LastEditDate="2017-12-21T02:47:04.863" LastActivityDate="2017-12-23T22:17:05.170" Title="Find connected circles" Tags="|algorithms|computational-geometry|graph-theory|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28478" PostTypeId="2" ParentId="28477" CreationDate="2017-12-19T08:20:10.997" Score="5" Body="&lt;p&gt;You don't need to check each pair of circles, so you can apply one of the neighour search algorithms. They restrict the distance calculations to the circles in the vicinity of each other by generating a list of potential neighbours based on a certain division of space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest to use the &lt;a href=&quot;http://pointclouds.org/documentation/tutorials/kdtree_search.php&quot; rel=&quot;noreferrer&quot;&gt;kd-tree&lt;/a&gt; method, which is efficient for circles with variable radii and has linear complexity. However, if you have only a few circles, brute-force method is more efficient.&lt;/p&gt;&#xA;" OwnerUserId="20475" LastEditorUserId="20475" LastEditDate="2017-12-19T08:32:38.083" LastActivityDate="2017-12-19T08:32:38.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28479" PostTypeId="2" ParentId="11290" CreationDate="2017-12-19T10:20:37.120" Score="-1" Body="&lt;p&gt;Sparselizard (www.sparselizard.org) is new but robust, rather general and can solve easily your elasticity problems with millions of unknown in 2D and hundreds of thousands in 3D on your laptop in two or three minutes (it is openmp parallelised). You can also use high order interolations for accurate mechanical computations at a low computational cost. The documentation is also up to date and convenient. From my (of course subjective) point of view it is the most concise and user friendly library I have found.&lt;/p&gt;&#xA;" OwnerUserId="26348" LastActivityDate="2017-12-19T10:20:37.120" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28480" PostTypeId="2" ParentId="28476" CreationDate="2017-12-19T14:13:19.790" Score="0" Body="&lt;p&gt;We have the assumption $a^2 + b^2 =\vec{u}\vec{u}^T= 1$. By construction/definition (see &lt;a href=&quot;http://people.eecs.berkeley.edu/~demmel/cs267/lecture18/lecture18.html#link_3.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;), $M$ is a symmetric matrix. Hence, we know that the Rayleigh quotient of $M$ and $\vec{u}$ is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ R(M,\vec{u})={\vec{u}^{T}M\vec{u} \over \vec{u}^{T}\vec{u}} = \vec{u}^{T}M\vec{u} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turns out that $$\min_{\vec{x}} R(M, \vec{x}) = \lambda_1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\lambda_1$ is the smallest eigenvalue of $M$. See section $3$ of paper &lt;a href=&quot;http://people.math.gatech.edu/~ecroot/notes_linear.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://people.math.gatech.edu/~ecroot/notes_linear.pdf&lt;/a&gt; for a proof of this fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our original goal was to minimize $ R(M,\vec{u}) = \vec{u}^{T}M\vec{u}$. We now know that that quantity is the smallest eigenvalue of $M$, i.e. $\lambda_1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what is the vector $\vec{x}$ which corresponds to $\lambda_1$? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp; \vec{x}^{T}M\vec{x} = \lambda_1\\&#xA;\iff &amp;amp; \vec{x} \vec{x}^{T}M\vec{x} = \vec{x} \lambda_1 \\&#xA;\iff &amp;amp; \vec{x} \vec{x}^{T}M\vec{x} = \lambda_1 \vec{x} \\&#xA;\iff &amp;amp; M\vec{x} = \lambda_1 \vec{x} \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that $\lambda_1$ is the smallest eigenvalue of $M$, it must follow that $\vec{x}$ is the corresponding eigenvector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I suppose, according to section $2$ of &lt;a href=&quot;http://people.math.gatech.edu/~ecroot/notes_linear.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://people.math.gatech.edu/~ecroot/notes_linear.pdf&lt;/a&gt;, the assumption $a^2 + b^2 =\vec{u}\vec{u}^T= 1$ seems to come from the fact that minimizing the Rayleigh quotient $R(M,\vec{u}) = \frac{\vec{u}^{T}M\vec{u}}{\vec{u}^{T}\vec{u}}$ is equivalent to minimizing $R(M,\vec{u}) = \vec{u}^{T}M\vec{u}$, that is we can let $\|\vec{u}\| = 1$. In the same article, the author calls this property &quot;scaling&quot;.&lt;/p&gt;&#xA;" OwnerDisplayName="user9925" LastEditorDisplayName="user9925" LastEditDate="2017-12-19T17:02:17.297" LastActivityDate="2017-12-19T17:02:17.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28482" PostTypeId="1" CreationDate="2017-12-20T12:09:34.540" Score="1" ViewCount="22" Body="&lt;p&gt;What are the differences in parameters in the force field for MD simulation study of &lt;strong&gt;RIGID&lt;/strong&gt; and &lt;strong&gt;FLEXIBLE&lt;/strong&gt; ZIF-8 framework?Is the electrostatic interaction term present in the rigid one?&lt;/p&gt;&#xA;" OwnerUserId="26340" LastActivityDate="2017-12-20T12:09:34.540" Title="Rigid and flexible ZIF-8 framework" Tags="|molecular-dynamics|" AnswerCount="0" CommentCount="1" ClosedDate="2017-12-22T02:08:05.837" ContentLicense="CC BY-SA 3.0" />
  <row Id="28483" PostTypeId="1" CreationDate="2017-12-20T13:33:21.900" Score="1" ViewCount="94" Body="&lt;p&gt;I would like first to state that it is beyond my capability to identify whether this is a BFGS issue or a R package problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been doing some mixed logit regression using the R package &quot;mlogit&quot;. In total I have 32 random parameters and one fixed parameter to be estimated. Correlations are allowed among random parameters. &#xA;With BFGS method, the code stopped after 13 iteration. Judging from the Chi-square value, it is far from converge. With BHHH method, however, it usually converge (although very slowly). Is this a package problem? or is it BHHH method more appropriate than BFGS in my context?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I doubt if it is related to the step size, but it's only my doubt.&#xA;The code and data I've used are &lt;a href=&quot;https://drive.google.com/open?id=1JIExSz5pq6iGgTWteW49quZ15Ptw_gCv&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26360" LastEditorUserId="9667" LastEditDate="2017-12-20T17:58:17.570" LastActivityDate="2017-12-20T17:58:17.570" Title="BFGS convergence problem" Tags="|optimization|convergence|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28484" PostTypeId="1" AcceptedAnswerId="28520" CreationDate="2017-12-20T14:22:08.063" Score="2" ViewCount="831" Body="&lt;p&gt;Suppose we have a system of $N$ particles that interact via the Lennard-Jones potential $$V(r)=V_0\left[\left(\frac{r_0}{r}\right)^{12}-2\ \left(\frac{r_0}{r}\right)^{6}\right].$$ No other forces exist. The objective is to perform molecular dynamics simulations with this system. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm stuck on an important part. I do not know exactly how to calculate the forces on each atom. I know that to calculate the force on &lt;em&gt;one&lt;/em&gt; atom, I must consider the interaction of it with all other atoms. However, I'm stuck as to how I put together these forces. Particularly, I don't know how to get the &lt;em&gt;direction&lt;/em&gt; (sign) of the forces right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, my approach is to consider the $x$, $y$, and $z$ components of a given atom separately in calculating the forces/accelerations. I visit the interaction of one atom with all other atoms and compute the force in $x$, $y$, $z$. For each component, I add them all up, assuming this approach leaves me with a &lt;em&gt;net&lt;/em&gt; acceleration in each direction at the end. I think I can use the final values of &lt;code&gt;atm.ax&lt;/code&gt;, &lt;code&gt;atm.ay&lt;/code&gt;, and &lt;code&gt;atm.az&lt;/code&gt; to update the position and velocity of the atom.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void computeForce(Atom atm){&#xA;   for(int i=0;i&amp;lt;N_particles;i++){&#xA;   if(i==atm.index)&#xA;      continue;&#xA;   else{&#xA;      dx = PBC1D(atm.x - allAtm[i].x);&#xA;      dy = PBC1D(atm.y - allAtm[i].y);&#xA;      dz = PBC1D(atm.z - allAtm[i].z);&#xA;&#xA;      atm.ax += force(dx) / m;&#xA;      atm.ay += force(dy) / m;&#xA;      atm.az += force(dz) / m;&#xA;   }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(&lt;code&gt;Atom&lt;/code&gt; is a user-defined class; &lt;code&gt;PBC1D&lt;/code&gt; allows for periodic boundary conditions---not important to my question here.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although my current approach makes sense to me, I have a feeling that it is incomplete and perhaps very inefficient. Can anyone help me understand if what I'm doing is correct or not? And, perhaps teach me better ways?&lt;/p&gt;&#xA;" OwnerUserId="26366" LastActivityDate="2017-12-28T23:11:55.070" Title="How to compute forces in multi-particle MD" Tags="|computational-physics|molecular-dynamics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28485" PostTypeId="1" AcceptedAnswerId="28490" CreationDate="2017-12-20T14:48:07.593" Score="3" ViewCount="178" Body="&lt;p&gt;I want to compare a few solvers of partial differential equations. I need to include the computational time and the solution accuracy (compared to an analytical solution or something similar).&lt;/p&gt;&#xA;&lt;p&gt;What kind of measures I could use?&lt;/p&gt;&#xA;" OwnerUserId="4334" LastEditorUserId="20688" LastEditDate="2021-12-30T04:16:38.690" LastActivityDate="2021-12-30T04:16:38.690" Title="How to measure efficiency of the differential equations solver" Tags="|pde|benchmarking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28486" PostTypeId="2" ParentId="28485" CreationDate="2017-12-20T15:55:10.320" Score="2" Body="&lt;p&gt;Well the two main measures you should compare obviously is the computational time and the accuracy - do not overthink. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can compare also the convergency/accuracy as a function of the time step. If you will test adaptive methods you should analyse the time step as function of time too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the kind of problems you are interested, you could compare the error propagation on some system's properties. In addition, it is interesting to verify if the methods are conservatives or not, for some physical systems this is very relevant.&lt;/p&gt;&#xA;" OwnerUserId="26289" LastActivityDate="2017-12-20T15:55:10.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28487" PostTypeId="2" ParentId="28484" CreationDate="2017-12-20T17:02:28.183" Score="3" Body="&lt;p&gt;I have the following remarks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Force calculation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation of motion of the $i^\text{th}$ atom with mass $m_i$ reads as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textbf{a}_i=\frac{d\textbf{v}_i}{dt}=\frac{1}{m_i}\cdot f(\vert\textbf{r}_j-\textbf{r}_i\vert)\textbf{n}_{ji}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the interparticle force $f(\vert\textbf{r}_j-\textbf{r}_i\vert)$ depends only on the distance between particles $i$ and $j$. In your implementation you calculate the force using the distances in each direction, which is incorrect. Instead, you need to compute the force for each pair of particles only once (using the scalar distance between them) and multiply it with the normalised direction vector $\textbf{n}_{ji}=\frac{\textbf{r}_j-\textbf{r}_i}{\vert\textbf{r}_j-\textbf{r}_i\vert}$ to obtain the components of the force-vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Neighours&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not so efficient in case of large number of atoms because of the $\mathcal{O}(n^2)$ computational complexity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the interaction between the atoms becomes smaller and smaller with increasing distance, it is beneficial to apply a certain threshold or cutoff distance to consider only the neighbouring ones of each atom. Based on the threshold you can divide your domain into threshold-sized cells and perform a neighbour search. It is a technique to find potential neighbours without the actual calculation of any of the distances between any pair of atoms. Based on the list of potential neighbours to be considered you can calculate the distances and check if they are really inside the influence radius or not. There are several methods to find potential neighbours, but in general $\mathcal{O}(n)$ complexity can be achieved. Read more about the neighbour search algorithms &lt;a href=&quot;http://research.ijcaonline.org/volume95/number25/pxc3897073.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, ignoring the particles outside of the influence radius may cause errors in your simulation if your interaction law has infinite radius. Therefore, application of a threshold is a trade-off between accuracy and requirement of computational performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Container of particles&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your current implementation stores the particles as an AoS (array of structures). It is a convenient approach from programming point of view, but less efficient than the SoA (structure of arrays), especially for parallel computations. In the latter you would create a large structure with arrays inside storing all the particle properties. It ends up in a well-aligned data in the memory that is preferable for this type of calculations.&lt;/p&gt;&#xA;" OwnerUserId="20475" LastEditorUserId="20475" LastEditDate="2017-12-26T12:56:57.367" LastActivityDate="2017-12-26T12:56:57.367" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28489" PostTypeId="1" CreationDate="2017-12-20T18:46:20.620" Score="7" ViewCount="379" Body="&lt;p&gt;Let's say I have two numbers $x$ and $y$ which I'd like to compare to see whether they are equal up to their first $N$ digits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance $ 1.002 \approx 1.001$ and $1002 \approx 1001$ up to 3 digits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, this means that the difference of $x$ and $y$ should be $N$ orders of magnitude smaller than (say) $x$. In symbols, the digits of accuracy, $D$ is&#xA;$$ D = \log_{10}(|x|) -\log_{10} (|x-y|) = - \log_{10} \left| \frac{y}{x} - 1\right| $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make the definition symmetric, it seems to make sense to replace the $\frac{y}{x} - 1$ by $\ln(\frac{y}{x})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a final definition would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ D = \begin{cases}&#xA;-\log_{10} \left| \ln |x| - \ln |y| \right| &amp;amp; \text{ if } &#xA;xy &amp;gt; 0 \\&#xA;-\infty &amp;amp; \text{ otherwise }&#xA;\end{cases}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a reasonable definition? If so, does it have a name? Where is it referenced?&lt;/p&gt;&#xA;" OwnerUserId="21499" LastEditorUserId="21499" LastEditDate="2017-12-21T00:26:17.950" LastActivityDate="2018-11-03T23:44:15.597" Title="What is a good definition of &quot;accuracy to N digits&quot;?" Tags="|accuracy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28490" PostTypeId="2" ParentId="28485" CreationDate="2017-12-20T19:57:49.493" Score="7" Body="&lt;p&gt;There are many different ways to do this. One of the standard is a work-precision plot where you plot the amount of time or function calls that it takes in order to achieve a certain level of accuracy. You can find tons of examples at &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;noreferrer&quot;&gt;DiffEqBenchmarks.jl&lt;/a&gt;. Generally you slide a timestep or adaptivity tolerances along a window and plot all of the (time,error) coordinates. Then the question &quot;to get an error of X digits, what method performs best on this problem?&quot; is nicely answered by looking at the plot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to choose a measure for your error. &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffODE/VanDerPol.ipynb&quot; rel=&quot;noreferrer&quot;&gt;This is a nice example which runs through quite a few on a small problem&lt;/a&gt;. One common way to do this is to compute a reference solution to compare against (here I use common solvers like &lt;code&gt;radau&lt;/code&gt; at very low tolerances, lower tolerances than what's used in the tests. You can use multiple references to make sure it's sufficiently converged for the tests). Then you use some measurement of error against the reference. l2 error at the final timepoint is easy to do. l2 error along the whole timeseries is a nice measurement of average error (maybe average it by the number of steps). L2 error using dense output is a nice test to do as well if you have dense output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While work-precision plots are quite standard, there can be reasons to do something more specific to your application. For example, you may want to &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/DynamicalODE/Quadrupole_boson_Hamiltonian_energy_conservation_benchmark.ipynb&quot; rel=&quot;noreferrer&quot;&gt;time and estimate the divergence of some essential properties of your system like conservation of energy&lt;/a&gt;. In some cases, building a full work-precision plot is simply out of reach (getting the reference solution may take a cluster, especially if you want to keep the whole timeseries for a large PDE!). Thus you may want to just do &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffSDE/StochasticHeat.ipynb&quot; rel=&quot;noreferrer&quot;&gt;a more targeted analysis&lt;/a&gt;. The good thing is, if you're only benchmarking deterministic (non stochastic) equations, then it's quite easy to put the time in and get a single reference solution (stochastic is much much harder to benchmark!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully that gives some ideas.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2017-12-20T19:57:49.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28491" PostTypeId="1" CreationDate="2017-12-20T21:16:04.770" Score="0" ViewCount="182" Body="&lt;p&gt;It looks to me like a very common problem, yet I haven't been able to find any practical guide on the subject despite many hours searching. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a clearer statement of my question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a system of (linear) PDEs in the variables $x$ and $t$. I can use the method of lines to discretise the space coordinate $x$ which yields a system of ODEs in time:&#xA;$$&#xA;\frac{d\vec{u}}{dt}=A\vec{u}+\vec{b}&#xA;$$&#xA;$A$ is a &lt;strong&gt;square matrix&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to me, the straightforward way to solve the above on a computer is to use an explicit time-stepping method of some sort. So I write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\vec{u}_{n+1}=\vec{u}_{n}+\Delta t(A\vec{u}_{n}+\vec{b}_n)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now, how do I enforce boundary conditions in all this ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In their most general form, these are&#xA;$$&#xA;B\vec{u}=0&#xA;$$&#xA;$B$ is a &lt;strong&gt;rectangular matrix&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't really see how this can fit into the above. Any help would be super appreciated.&lt;/p&gt;&#xA;" OwnerUserId="19584" LastActivityDate="2017-12-21T02:26:10.697" Title="How do you apply boundary conditions in a time-stepping problem?" Tags="|boundary-conditions|time-integration|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28492" PostTypeId="2" ParentId="28491" CreationDate="2017-12-21T00:23:50.787" Score="2" Body="&lt;p&gt;A simple way which should work well for an implicit time-integrator that can handle stiff equations is to write your BC's as relaxation ODEs. For example, if we want to enforce $u_j=u_{j0}$  where $u_j$ is the value of the unknown at a boundary grid point $j$, then use $\frac{d}{dt} u_j =  - \lambda_{rlx} (u_j - u_{j0})$, where $\lambda_{rlx}$ is a &quot;relaxation rate&quot; chosen to be fast compared to time scales in the operator A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of this method is it is very easy to implement. This way your BC is maintained not exactly (depending on the chosen $\lambda_{rlx}$), but the error can be made small, and sometimes it is actually desired to have some freedom in the boundary values, e.g., if your main goal is achieving a steady state. &lt;/p&gt;&#xA;" OwnerUserId="4325" LastActivityDate="2017-12-21T00:23:50.787" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28493" PostTypeId="1" CreationDate="2017-12-21T01:55:42.700" Score="2" ViewCount="138" Body="&lt;p&gt;Suppose there are multiple intervals on $\mathbb{R}$, e.g., $[0,0.5]$, $[0.4,1]$, $[1.5,2]$, the union of them should be $[0,1]$ and $[1.5,2]$. Is there a specific datastructure (or algorithm) for computing such union?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A variation of this problem is to union intervals on cyclic domain, for example, angles from $-\pi$ to $\pi$. Now,  $[-\pi/2, \pi/2]$ and  $[\pi/2, -\pi/2]$ present different intervals of angles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have coded a brute force  algorithm, however, I am wondering there already exist studies on this. &lt;/p&gt;&#xA;" OwnerUserId="26321" LastEditorUserId="26321" LastEditDate="2017-12-21T02:03:29.593" LastActivityDate="2017-12-21T02:25:02.450" Title="Union of closed intervals on $\mathbb{R}$" Tags="|algorithms|data-structures|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28494" PostTypeId="2" ParentId="28493" CreationDate="2017-12-21T02:25:02.450" Score="3" Body="&lt;p&gt;If you order the intervals by their starting point (possible with complexity ${\cal O}(N \log N)$), then you can test whether adjacent intervals overlap and merge them as necessary in ${\cal O}(N)$ operations. What you end up with is list of disjoint intervals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do this in a periodic domain is only marginally more difficult -- do the same as above, and at the end of the operations see if you can merge the first and the last of the merged intervals.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-12-21T02:25:02.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28495" PostTypeId="2" ParentId="28491" CreationDate="2017-12-21T02:26:10.697" Score="4" Body="&lt;p&gt;As HBR mentioned, the boundary conditions can often be immediately incorporated into $A$ and $b$. For example, suppose we wish to solve the 1D heat equation with Dirchilet boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = \sigma u_{xx} + h(x,t), \quad u(a,t) = f(t), \quad u(b,t) = g(t).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then discretize $u_j^n \approx u(x_j, t_n)$ where $x_j = a+j \Delta x$ and $t_n = n\Delta t$. Here $j = 0,1,2,\ldots,N+1$ and $\Delta x = (b-a)/(N+1)$. Note that by our boundary conditions $u_0^n = f(t_n)$ and $u_{N+1}^n = g(t_n)$. As these values are completely determined for all time, we don't even include them in our linear system. For interior gridpoints ($2 \le j \le N-1$), we take a second-order centered difference&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_{xx}(x_j,t_n) \approx \frac{u_{j-1}^n - 2u_j^n + u_{j+1}^n}{\Delta x^2}. \tag{1}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the boundary-adjacent gridpoints, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;u_{xx}(x_1,t_n) &amp;amp;\approx \frac{u_{0}^n - 2u_1^n + u_{2}^n}{\Delta x^2}=\frac{f(t_n) - 2u_1^n + u_{2}^n}{\Delta x^2}, \tag{2} \\&#xA;u_{xx}(x_N,t_n) &amp;amp;\approx \frac{u_{N-1}^n - 2u_{N}^n + u_{N+1}^n}{\Delta x^2}=\frac{u_{N-1}^n - 2u_{N}^n + g(t_n)}{\Delta x^2}. \tag{3}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Collecting our discretization, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d}{dt} \begin{pmatrix} u_1 \\ u_2 \\ u_3 \\ \vdots \\ u_{n-1} \\ u_n \end{pmatrix} = \frac{1}{\Delta x^2}\begin{pmatrix}&#xA;-2 &amp;amp; 1  &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;1  &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp; \cdots  &amp;amp; 0  &amp;amp; 0&amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; -2 &amp;amp; 1 &amp;amp; \cdots  &amp;amp; 0&amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; -2 &amp;amp; \cdots  &amp;amp; 0 &amp;amp; 0&amp;amp; 0 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 1 &amp;amp; -2 &amp;amp; 1 \\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; 1 &amp;amp; -2&#xA;\end{pmatrix}\begin{pmatrix} u_1 \\ u_2 \\ u_3 \\ u_4 \\ \vdots \\ u_{n-1} \\ u_n \end{pmatrix} + \begin{pmatrix} f(t)/\Delta x^2 + h(x_1,t) \\&#xA;h(x_2,t) \\&#xA;h(x_3,t) \\&#xA;h(x_4,t) \\&#xA;\vdots \\&#xA;h(x_{n-1},t) \\&#xA;g(t)/\Delta x^2 + h(x_n, t) \end{pmatrix}. \tag{4}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, we have baked our boundary conditions into the matrix $A$ and the vector $b$. (Make sure you can see how we got from the (1)-(3) to (4).) Similar approaches work for other PDEs and other discretization. Handling boundary conditions for hyperbolic problems is more tricky though, and you should always try to be very careful when treating the boundary in your discretization.&lt;/p&gt;&#xA;" OwnerUserId="16757" LastActivityDate="2017-12-21T02:26:10.697" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28497" PostTypeId="1" CreationDate="2017-12-21T09:33:24.353" Score="1" ViewCount="120" Body="&lt;p&gt;I have a nonlinear Schrodinger equation which reads:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{1}{2} \frac{d^2u}{dx^2}+ |u|^2u + V(x)u = -i \frac{du}{dz},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $V(x)=\cos(wx)+ i a \sin(wx)$ and $w$, $a$ are numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to solve its band structure, meaning its dispersion relation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plane wave expansion method is used for a real potential. Can it be applied for this complex potential also? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are other methods, if any, in Matlab or Mathematica or other software?&lt;/p&gt;&#xA;" OwnerUserId="26373" LastEditorUserId="9667" LastEditDate="2017-12-22T21:59:53.137" LastActivityDate="2017-12-22T21:59:53.137" Title="Band structure of nonlinear Schrodinger equation with one dimensional potential" Tags="|matlab|computational-physics|mathematica|electromagnetism|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28499" PostTypeId="1" CreationDate="2017-12-21T15:23:32.277" Score="0" ViewCount="152" Body="&lt;p&gt;I want to solve non-classic diffusion equation in Fourier space. The equation is $$∂c/∂t=-∇.J$$ Where $J$ is $$J= -M.∇μ$$ Where M is mobility. It depends on c and  $\mu$ is $$ μ= g(c) - \nabla^2c $$ Now i am trying to solve above equation  $$∂c/∂t= -∇.(-M.∇μ)$$ $$  =∇.[M.(∇(g(c)-∇^2 c))]$$  Now how to do Fourier transform and what is real and imaginary part of this.&lt;/p&gt;&#xA;" OwnerUserId="26375" LastEditorUserId="26375" LastEditDate="2017-12-21T20:01:20.093" LastActivityDate="2018-05-21T02:13:34.433" Title="How to solve diffusion equation in Fourier space when mobility is not constant" Tags="|fourier-transform|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28500" PostTypeId="2" ParentId="28499" CreationDate="2017-12-21T20:56:54.767" Score="0" Body="&lt;p&gt;This is a standard problem; in general, you can't solve it purely in Fourier Space (i.e. the representation of the problem is not &quot;diagonal&quot; in the Fourier space basis). However, if you want to get numerical solutions, you can use a pseudospectral method. One very relevant paper, which should give you enough ideas to implement something like this, is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Coarsening kinetics from a variable-mobility Cahn-Hilliard equation: Application of a semi-implicit Fourier spectral method&quot; Zhu et al. Phys. Rev. E 1999&#xA;&lt;a href=&quot;https://doi.org/10.1103/PhysRevE.60.3564&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1103/PhysRevE.60.3564&lt;/a&gt;&#xA;(there are also PDF copies floating around the net).&lt;/p&gt;&#xA;" OwnerUserId="4494" LastActivityDate="2017-12-21T20:56:54.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28501" PostTypeId="1" CreationDate="2017-12-21T21:13:43.910" Score="1" ViewCount="102" Body="&lt;p&gt;I've asked this question on &lt;a href=&quot;https://stackoverflow.com/questions/47706028/nonlinear-global-optimization-algorithm-that-can-use-dynamic-programming&quot;&gt;stackoverflow&lt;/a&gt; 2 weeks ago, but, judging by zero response, that probably was the wrong forum. Therefore copying it here:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;code&gt;F0,...,Fn&lt;/code&gt; be functions with parameters &lt;code&gt;p0,...,pn&lt;/code&gt; and &lt;code&gt;G&lt;/code&gt; be a composite function. Say, for &lt;code&gt;n=3&lt;/code&gt; &lt;code&gt;G(x)=F3(p3,F2(p2,F1(p1,F0(p0,x))))&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to globally optimize the parameters &lt;code&gt;p0,...,pn&lt;/code&gt; with making as few function calls as possible. There are several ways to optimize:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Ignore that &lt;code&gt;G&lt;/code&gt; is a composition of &lt;code&gt;Fi&lt;/code&gt;s, treat it as a blackbox with parameters &lt;code&gt;p0,...,pn&lt;/code&gt;, and use any generic global optimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Optimize hierarchically; for clarity let's assume &lt;code&gt;n=3&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Loop A: Freeze parameters &lt;code&gt;p0,p1,p2&lt;/code&gt; and optimize &lt;code&gt;p3&lt;/code&gt; in isolation. because &lt;code&gt;G&lt;/code&gt; depends only on &lt;code&gt;p3&lt;/code&gt; and the output of prior functions that have frozen params I only have to evaluate &lt;code&gt;F3&lt;/code&gt;; the outputs of other &lt;code&gt;F&lt;/code&gt;s don't change between iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Loop B: Freeze parameters &lt;code&gt;p0,p1&lt;/code&gt;, change &lt;code&gt;p2&lt;/code&gt;, and repeat the above loop A. Repeat for optimizing &lt;code&gt;p2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Loop C: Freeze &lt;code&gt;p0&lt;/code&gt;, change &lt;code&gt;p1&lt;/code&gt;, repeat loop B, thus optimizing &lt;code&gt;p1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Loop D: Obvious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &quot;dynamic programming&quot; approach would lead to optimizing &lt;code&gt;G&lt;/code&gt; by, hopefully, making fewer iterations. I have a couple of questions though:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I'm not sure that such hierarchical optimization approach would lead to solution of the same quality as the generic &quot;blackbox&quot; approach. Has any research have been done regarding the quality of results produced by hierarchical versus blackbox optimization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, are there any optimization packages available out there supporting hierarchical optimization like that? I'm aware of a few available generic blackbox optimizers (NLOpt, etc), but not generic hierarchical optimizers.&lt;/p&gt;&#xA;" OwnerUserId="5281" LastActivityDate="2017-12-21T21:13:43.910" Title="Nonlinear global optimization algorithm that can use dynamic programming" Tags="|optimization|nonlinear-programming|dynamic-programming|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28502" PostTypeId="2" ParentId="19862" CreationDate="2017-12-21T21:38:16.453" Score="1" Body="&lt;p&gt;The monolithic approach is not necessarily better, as stated in another answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you only want to solve one specific problem in one way, then the monolithic approach could maybe be more robust. But why not get a more general solution you can?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;partitioned approach&lt;/strong&gt; has some very important advantages:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;You can use existing single-physics solvers for each domain.&lt;/strong&gt; This is not only easier, as you don't re-invent the wheel, but you may also already have and know some very good, special, or simply closed-source solvers that you want to reuse. You can use completely different meshes, numerical solvers etc. in each subdomain. They don't need to restrict each other.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;You can achieve better performance&lt;/strong&gt; in a cluster, since you are free to assign e.g. less computing resources to the (usually faster) solid simulation and more to the fluid. This way, you can also minimize the communication time, since you only need to exchange information on the coupling times.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Read more in the section 1.2 (page 4 / pdf-page 9) of [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;There are tools that can couple single-physics solvers to a multi-physics simulation.&lt;/strong&gt; More precisely, since you are looking for open-source tools, you can try &lt;a href=&quot;http://www.precice.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;preCICE&lt;/a&gt; (on &lt;a href=&quot;https://github.com/precice/precice&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;). There are quite a few &lt;a href=&quot;http://www.precice.org/codes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;codes&lt;/a&gt; already coupled (see also [2]).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, see the &lt;a href=&quot;https://github.com/precice/precice/wiki/1D-Example&quot; rel=&quot;nofollow noreferrer&quot;&gt;1D FSI example: flow through a deformable artery&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like some people are already using preCICE for the &lt;a href=&quot;http://www.precice.org/testimonials/&quot; rel=&quot;nofollow noreferrer&quot;&gt;evaluation of heart valve biomechanics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Uekermann, B., &lt;em&gt;&lt;a href=&quot;https://mediatum.ub.tum.de/doc/1320661/document.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Partitioned Fluid-Structure Interaction on Massively Parallel Systems&lt;/a&gt;&lt;/em&gt;, PhD Thesis, Institut für Informatik, Technische Universität München, 2015.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Uekermann, B., Bungartz, H.-J., Cheung Yau, L., Chourdakis, G., Rusch, A., &lt;em&gt;&lt;a href=&quot;https://www.gacm2017.uni-stuttgart.de/registration/Upload/ExtendedAbstracts/ExtendedAbstract_0138.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Official preCICE Adapters for Standard Open-Source Solvers&lt;/a&gt;&lt;/em&gt;, in Conference Proceedings at the 7th GACM Colloquium on Computational Mechanics for Young Scientists from Academia, 2017.&lt;/p&gt;&#xA;" OwnerUserId="17650" LastActivityDate="2017-12-21T21:38:16.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28503" PostTypeId="2" ParentId="28468" CreationDate="2017-12-22T10:29:06.597" Score="2" Body="&lt;p&gt;You want to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \arg \min_{D(x),k(x)} \mathcal{G}(u;D,k) := \frac{1}{2} \int_{\Omega} \left(u(t=T) - {\bar{u}}(T) \right)^2 d\Omega + \frac{1}{2}\int_{\Omega} D^2 d\Omega + \frac{1}{2}\int_{\Omega} k^2 d\Omega $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{\partial u}{\partial t} - \nabla \cdot (D \nabla{u}) - k u (1-u) &amp;amp;= 0  &amp;amp; \space \mathrm{on} \space \Omega \\&#xA;u(0,x) &amp;amp;= u_0 &amp;amp; \space \forall x \in \Omega \\&#xA;u(t,x) &amp;amp;= 0 &amp;amp; \space \forall x \in \Gamma := \partial \Omega. &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's define an inner product on $[0,T] \times \Omega$:&#xA;$$ \left&amp;lt; v , w \right&amp;gt;_{[0,T] \times \Omega} := \int_{0}^{T} \int_\Omega v \space w \mathrm{d}\Omega \mathrm{d}t.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this, the Lagrangian for the constrained problem reads&#xA;$$&#xA;\mathcal{L}(u;\lambda;D,k) = \mathcal{G}(u;D,k) - \left&amp;lt; \lambda , \frac{\partial u}{\partial t} - \nabla \cdot (D \nabla{u}) - k u (1-u) \right&amp;gt;_{[0,T] \times \Omega}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're now ready to write out the first order necessary conditions for a local minimum (the KKT system):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The primal equation ${\cal L}_\lambda (u; \lambda; D,k)(\delta \lambda) = 0 \space \forall \space \delta \lambda$. This is nothing other than the &quot;forward&quot; PDE constraint.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The adjoint (or dual) equation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$${\cal L}_u (u; \lambda; D,k)(\delta u) = 0 \space \forall \space \delta u. $$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The gradient equations (the derivatives with respect to the control parameters $D$ and $k$):&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;   {\cal L}_D (u; \lambda; D,k)(\delta D) &amp;amp;= 0 \space \forall \space \delta D \\ &#xA;   {\cal L}_k (u; \lambda; D,k)(\delta k) &amp;amp;= 0 \space \forall \space \delta k.&#xA; \end{align} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take the left-hand side of the adjoint equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;   {\cal L}_u (u; \lambda; D,k)(\delta u) &#xA;   &amp;amp;= &#xA;   \left( u(T,x) - {\bar{u}}(T,x) \right) \delta u(T,x) \\&#xA;   &amp;amp; \underbrace{ &#xA;       - \int_{0}^{T} \int_\Omega \lambda \frac{\partial \delta u}{\partial t}  \mathrm{d}\Omega \mathrm{d}t&#xA;	  }_{\mathbf{A}}&#xA;  \space  \underbrace{ &#xA;     + \int_{0}^{T} \int_\Omega \lambda \nabla \cdot (D \nabla u) \mathrm{d}\Omega \mathrm{d}t &#xA;      }_{\mathbf{B}}&#xA;  \space \underbrace{ &#xA;     + \int_{0}^{T} \int_\Omega \lambda k u (1-u) \mathrm{d}\Omega \mathrm{d}t &#xA;	 }_{\mathbf{C}}.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and look at each of the three integral terms in turn.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integrating ($\mathbf{A}$) by parts, we get:&#xA;$$&#xA;\mathbf{A} = - \int_\Omega \left. \left( \lambda \delta u \right) \right|_0^T \mathrm{d}\Omega + \int_{0}^{T} \int_\Omega \delta u  \frac{\partial \lambda}{\partial t} \mathrm{d}\Omega \mathrm{d}t.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similary for the second term, &lt;a href=&quot;https://en.wikipedia.org/wiki/Green%27s_identities#Green&amp;#39;s_second_identity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Green's second identity&lt;/a&gt; gives us&#xA;$$&#xA;\require{cancel}&#xA;\mathbf{B} = &#xA;\int_{0}^{T} \int_\Gamma \lambda D (\nabla \delta u \cdot \vec{n}) \mathrm{d}\Gamma \mathrm{d}t &#xA;-&#xA;\cancelto{}{&#xA;\int_{0}^{T} \int_\Gamma \delta u D (\nabla \lambda \cdot \vec{n}) \mathrm{d}\Gamma \mathrm{d}t &#xA;}&#xA;+&#xA;\int_{0}^{T} \int_\Omega  \delta u \nabla \cdot (D \nabla \lambda)&#xA;\mathrm{d}\Omega \mathrm{d}t,&#xA;$$&#xA;where the second term vanishes because $\delta u(t,x) = 0$ on $\Gamma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting all the terms together, and requiring that all terms vanish &lt;em&gt;for all&lt;/em&gt; $\delta u$, we arrive at the following adjoint problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA; \frac{\partial \lambda}{\partial t} + \nabla \cdot (D \nabla \lambda) + k(1-2u) &amp;amp; = 0 &amp;amp; \mathrm{on} \space \Omega \\&#xA; \lambda(T,x) &amp;amp;= u(T,x) - \bar{u}(T,x) &amp;amp; \forall x \in \Omega \\&#xA; \lambda(t,x) &amp;amp;= 0 &amp;amp; \forall x \in \Gamma, \space \forall t.&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, on to the gradient equations. For $D$, this reads:&#xA;$$&#xA;\int_\Omega D \delta D \mathrm{d}\Omega - \int_{0}^{T} \int_\Omega \lambda \nabla \cdot (\delta D \nabla u) \mathrm{d}\Omega \mathrm{d}t &#xA;= 0,&#xA;\space \forall \delta D.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, the integration by parts formula comes in handy here. After some algebra, this is what we're left with:&#xA;$$&#xA;\int_\Omega \delta D D \mathrm{d}\Omega &#xA;+&#xA;\int_0^T \int_\Omega \delta D &#xA;\left( &#xA; \nabla u \cdot \nabla \lambda&#xA;\right) &#xA;= 0 &#xA;\space \forall \space \delta D,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from which the equation for $D$ immediately follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;D(x) = \int_0^T \nabla u(x,t) \cdot \nabla \lambda(x,t)\mathrm{d}t.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;correct&quot; way to interpret this equation depends on the particular choice of function spaces for $u$, $\lambda$ and $D$. See &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26557/zero-terminal-value-of-the-adjoint-based-optimal-control&quot;&gt;this post&lt;/a&gt; for an example of why this is really important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll let you derive a similar equation for $k$.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2017-12-27T09:18:08.707" LastActivityDate="2017-12-27T09:18:08.707" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28504" PostTypeId="1" CreationDate="2017-12-22T14:35:44.097" Score="4" ViewCount="1261" Body="&lt;p&gt;The usual problem of graph partitioning is to split a graph (or a mesh) into two partitions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to its Github repository's README file, &lt;a href=&quot;https://github.com/SebastianSchlag/kahypar&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;KaHyPar&lt;/em&gt;&lt;/a&gt; supports both &lt;em&gt;recursive bisection&lt;/em&gt; and &lt;em&gt;direct k-way partitioning&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose that recursive bisection is just a recursive application on the partitions of a graph produced by a partition algorithm which produces two partitions, whereas &lt;em&gt;direct k-way partitioning&lt;/em&gt; may not use recursion, but it's simply an algorithm which tries to divide the graph directly into the desired number of partitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is correct, are there any other differences between the two approaches? If not, what is the difference between the two approaches? By the way, is the terminology consistent throughout the literature?&lt;/p&gt;&#xA;" OwnerDisplayName="user9925" LastActivityDate="2017-12-22T17:35:43.973" Title="What is the difference between recursive bisection and direct k-way partition?" Tags="|partitioning|terminology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28505" PostTypeId="2" ParentId="28504" CreationDate="2017-12-22T17:35:43.973" Score="2" Body="&lt;p&gt;Your intuition is correct -- a bisection method cuts the (hyper)graph in two, and recursive bisection repeatedly applies this strategy until the desired number of cuts have been made. Direct partitioning on the other hand tries to immediately divide up the graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of the divide between the two is historical. Some of the earliest successful heuristics for graph partitioning were based on bisection rather than direct partitioning. These bisection methods were based on using the second eigenfunction of the graph Laplacian (the Fielder vector). This technique is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_partition#Spectral_partitioning_and_spectral_bisection&quot; rel=&quot;nofollow noreferrer&quot;&gt;spectral bisection&lt;/a&gt; if you want the term to search for. There are also some really nice notes about spectral bisection &lt;a href=&quot;http://yifanhu.net/PROJECT/pdcp_siam/node10.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. In any case, if the number of partitions is a power of two, you can apply spectral bisection recursively and get a usable (if not necessarily optimal) partition, but for an arbitrary number of partitions it's not immediately obvious how to proceed. Good heuristics for directly partitioning a graph only came later. Many of them built on the ideas behind spectral graph partitioning, i.e. using not just the second but many eigenfunctions of the graph Laplacian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nowadays the same core ideas are still used, albeit by borrowing ideas from multigrid methods for linear systems; this is how the underlying algorithm in &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/metis/metis/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;METIS&lt;/a&gt; works.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-12-22T17:35:43.973" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28506" PostTypeId="2" ParentId="8899" CreationDate="2017-12-22T20:19:51.060" Score="9" Body="&lt;p&gt;I needed an algorithm that has&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;little branching (hopefully CMOVs)&lt;/li&gt;&#xA;&lt;li&gt;no trigonometric function calls&lt;/li&gt;&#xA;&lt;li&gt;high numerical accuracy even with 32 bit floats&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We want to calculate $c_1, s_1, c_2, s_2, \sigma_1$ and $\sigma_2$ as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A = USV$, which can be expanded like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\begin{bmatrix}&#xA;a &amp;amp; b \\ c &amp;amp; d&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;c_1 &amp;amp; s_1 \\ -s_1 &amp;amp; c_1&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\sigma_1 &amp;amp; 0 \\ 0 &amp;amp; \sigma_2&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;c_2 &amp;amp; -s_2 \\ s_2 &amp;amp; c_2&#xA;\end{bmatrix}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main idea is to find a rotation matrix $V$ that diagonalizes $A^TA$, that is $VA^TAV^T=D$ is diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$USV = A$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$US = AV^{-1} = AV^T$ (since $V$ is orthogonal)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$VA^TAV^T = (AV^T)^TAV^T = (US)^TUS = S^TU^TUS = D$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiplying both sides by $S^{-1}$ we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(S^{-T}S^T)U^TU(SS^{-1}) = U^TU = S^{-T}DS^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $D$ is diagonal, setting $S$ to $\sqrt{D}$ will give us $U^TU=Identity$, meaning $U$ is a rotation matrix, $S$ is a diagonal matrix, $V$ is a rotation matrix and $USV = A$, just what we are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculating the diagonalizing rotation can be done by solving the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t_2^2 - \frac{\beta-\alpha}{\gamma}t_2-1 = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;A^TA&#xA;=&#xA;\begin{bmatrix}&#xA;a &amp;amp; c \\ b &amp;amp; d&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;a &amp;amp; b \\ c &amp;amp; d&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;a^2+c^2 &amp;amp; ab+cd \\ ab+cd &amp;amp; b^2+d^2&#xA;\end{bmatrix}&#xA;= &#xA;\begin{bmatrix}&#xA;\alpha &amp;amp; \gamma \\ \gamma &amp;amp; \beta&#xA;\end{bmatrix}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $t_2$ is the tangent of angle of $V$. This can be derived by expanding $VA^TAV^T$ and making its off-diagonal elements equal to zero (they are equal to each other).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with this method is that it loses significant floating point precision when calculating $\beta-\alpha$ and $\gamma$ for certain matrices, because of the subtractions in the calculations. The solution for this is to do an RQ decomposition ($A=RQ$, $R$ upper triangular and $Q$ orthogonal) first, then use the algorithm to factorize $USV' = R$. This gives $USV=USV'Q=RQ=A$. Notice how setting $d$ to 0 (as in $R$) eliminates some of the additions/subtractions. (The RQ decomposition is fairly trivial from the expansion of the matrix product).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm naively implemented this way has some numerical and logical anomalies (e.g. is $S$ $+\sqrt{D}$ or $-\sqrt{D}$), which I fixed in the code below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I threw about 2000 million randomized matrices at the code, and the largest numerical error produced was around $6\cdot10^{-7}$ (with 32 bit floats, $error = ||USV-M||/||M||$). The algorithm runs in about 340 clock cycles (MSVC 19, Ivy Bridge).&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;template &amp;lt;class T&amp;gt;&#xA;void Rq2x2Helper(const Matrix&amp;lt;T, 2, 2&amp;gt;&amp;amp; A, T&amp;amp; x, T&amp;amp; y, T&amp;amp; z, T&amp;amp; c2, T&amp;amp; s2) {&#xA;    T a = A(0, 0);&#xA;    T b = A(0, 1);&#xA;    T c = A(1, 0);&#xA;    T d = A(1, 1);&#xA;&#xA;    if (c == 0) {&#xA;        x = a;&#xA;        y = b;&#xA;        z = d;&#xA;        c2 = 1;&#xA;        s2 = 0;&#xA;        return;&#xA;    }&#xA;    T maxden = std::max(abs(c), abs(d));&#xA;&#xA;    T rcmaxden = 1/maxden;&#xA;    c *= rcmaxden;&#xA;    d *= rcmaxden;&#xA;&#xA;    T den = 1/sqrt(c*c + d*d);&#xA;&#xA;    T numx = (-b*c + a*d);&#xA;    T numy = (a*c + b*d);&#xA;    x = numx * den;&#xA;    y = numy * den;&#xA;    z = maxden/den;&#xA;&#xA;    s2 = -c * den;&#xA;    c2 = d * den;&#xA;}&#xA;&#xA;&#xA;template &amp;lt;class T&amp;gt;&#xA;void Svd2x2Helper(const Matrix&amp;lt;T, 2, 2&amp;gt;&amp;amp; A, T&amp;amp; c1, T&amp;amp; s1, T&amp;amp; c2, T&amp;amp; s2, T&amp;amp; d1, T&amp;amp; d2) {&#xA;    // Calculate RQ decomposition of A&#xA;    T x, y, z;&#xA;    Rq2x2Helper(A, x, y, z, c2, s2);&#xA;&#xA;    // Calculate tangent of rotation on R[x,y;0,z] to diagonalize R^T*R&#xA;    T scaler = T(1)/std::max(abs(x), abs(y));&#xA;    T x_ = x*scaler, y_ = y*scaler, z_ = z*scaler;&#xA;    T numer = ((z_-x_)*(z_+x_)) + y_*y_;&#xA;    T gamma = x_*y_;&#xA;    gamma = numer == 0 ? 1 : gamma;&#xA;    T zeta = numer/gamma;&#xA;&#xA;    T t = 2*impl::sign_nonzero(zeta)/(abs(zeta) + sqrt(zeta*zeta+4));&#xA;&#xA;    // Calculate sines and cosines&#xA;    c1 = T(1) / sqrt(T(1) + t*t);&#xA;    s1 = c1*t;&#xA;&#xA;    // Calculate U*S = R*R(c1,s1)&#xA;    T usa = c1*x - s1*y; &#xA;    T usb = s1*x + c1*y;&#xA;    T usc = -s1*z;&#xA;    T usd = c1*z;&#xA;&#xA;    // Update V = R(c1,s1)^T*Q&#xA;    t = c1*c2 + s1*s2;&#xA;    s2 = c2*s1 - c1*s2;&#xA;    c2 = t;&#xA;&#xA;    // Separate U and S&#xA;    d1 = std::hypot(usa, usc);&#xA;    d2 = std::hypot(usb, usd);&#xA;    T dmax = std::max(d1, d2);&#xA;    T usmax1 = d2 &amp;gt; d1 ? usd : usa;&#xA;    T usmax2 = d2 &amp;gt; d1 ? usb : -usc;&#xA;&#xA;    T signd1 = impl::sign_nonzero(x*z);&#xA;    dmax *= d2 &amp;gt; d1 ? signd1 : 1;&#xA;    d2 *= signd1;&#xA;    T rcpdmax = 1/dmax;&#xA;&#xA;    c1 = dmax != T(0) ? usmax1 * rcpdmax : T(1);&#xA;    s1 = dmax != T(0) ? usmax2 * rcpdmax : T(0);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Ideas from:&lt;br&gt;&#xA;&lt;a href=&quot;http://www.cs.utexas.edu/users/inderjit/public_papers/HLA_SVD.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://www.cs.utexas.edu/users/inderjit/public_papers/HLA_SVD.pdf&lt;/a&gt; &lt;br&gt;&#xA;&lt;a href=&quot;http://www.math.pitt.edu/~sussmanm/2071Spring08/lab09/index.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.math.pitt.edu/~sussmanm/2071Spring08/lab09/index.html&lt;/a&gt; &lt;br&gt;&#xA;&lt;a href=&quot;http://www.lucidarme.me/singular-value-decomposition-of-a-2x2-matrix/&quot; rel=&quot;noreferrer&quot;&gt;http://www.lucidarme.me/singular-value-decomposition-of-a-2x2-matrix/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26386" LastEditorUserId="26386" LastEditDate="2018-03-14T11:37:29.530" LastActivityDate="2018-03-14T11:37:29.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28509" PostTypeId="1" AcceptedAnswerId="30575" CreationDate="2017-12-23T21:15:43.157" Score="0" ViewCount="60" Body="&lt;p&gt;In an nbody gravity simulation, instead of doing exact(all-pair brute force) solution, I added masses of each body into cells of a 3D grid(each cell is just a float value having a mass value). Then smoothed the grid using a stencil. Now I'm trying to get an approximation of point masses for point masses, from grid. I have already tried directly doing body-cell interaction(similar to body-body but using smoothed mass of cell and center position of cell) but this selects originally non-mass containing cells as if they are attracting and leads to high errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the most precise(or logical) way to do one of below options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a unit mass at an approximated position&lt;/li&gt;&#xA;&lt;li&gt;an approximated mass at a unit distance and angle&lt;/li&gt;&#xA;&lt;li&gt;approximation of both mass and distance&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;so that long-range component of total force properly acts same as all-pair brute-force version(long-range part of it)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, what could be a proper way other than upper options? Can I use derivate of mass density(by neighboring cells) for these?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now I'm calculating only an RMS error between all pair(but only long range part) version and this center-of-cell with smoothed mass (only with closest neighbor cells after long range distance(which equals 2*cell size)) to check how close the simulation is to all-pair version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing this grid smoothing because I want to learn multigrid acceleration for nbody. To learn it step by step (and to do a performance comparison against later), only using a single grid with simple smoothing(16x16x16 cubes(as grid size) and 16 smoothing steps with 26 closest neighbor cells(using successive over relaxation ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially both versions start in a cube volume.(Total particles are just 3072)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://image.ibb.co/b5BMem/multigrid0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://image.ibb.co/b5BMem/multigrid0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then at each timestep, rms of error of all body positions between two versions is calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://image.ibb.co/iacYs6/multigrid1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://image.ibb.co/iacYs6/multigrid1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Addition of point masses to cells are done very simply as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;atomic_add(closest_cell_to_current_particle,all_of_current_particle_mass);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2014" LastEditorUserId="2014" LastEditDate="2017-12-23T22:41:08.590" LastActivityDate="2018-11-22T06:31:44.790" Title="Converting mass density to point mass approximation on a grid" Tags="|multigrid|grid|particle|acceleration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28510" PostTypeId="2" ParentId="28477" CreationDate="2017-12-23T22:06:02.517" Score="0" Body="&lt;p&gt;Allow me to introduce a linear time, $C\cdot O(N) + O(W\cdot H)$ algorithm where $C$ denotes the number of circles, plus some small coefficient, and $O(W\cdot H)$ is always constant. In this one, you will not need KD-trees, which make it $O(N\cdot log(N))$ anyways. This of course, comes at the expense of some additional space consumption - though KD-trees also do, and a very acceptable approximation in certain cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What one could do is to render the circles on a fixed spatial domain, such as an image, and find the connected components on that image. It is then possible to directly find the equivalences, from the components in the image. Here is a pseudo-code of the idea:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;$\mathbf{I}$ $\leftarrow$ $\text{empty image}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;$\text{for all} (circle \in \mathbf{C}$) $\{$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;$\mathbf{I} \gets \text{render_circle}(\mathbf{I}$, $circle$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;$\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;$\mathbf{CC} \gets \text{conn_comp}(\mathbf{I})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;$\text{look-up the label of each } circle \in \mathbf{C} \text{ from }\mathbf{CC}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one likes to consider the nested circles as connected, then the rendering (e.g. using Bresenham or midpoint algorithm) should be made filled, otherwise, boundary drawing is sufficient and makes the algorithm far more efficient (image is never fully traversed). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take the following configuration:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PJBo1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PJBo1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The connected components, $\mathbf{CC}$, looks like:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZS1V6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZS1V6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the mapping array contains values such as: $[2 ,3, 1, 2, 1, 4]$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below, I provide a very dirty, but hopefully working solution in MATLAB. It can be implemented much more efficiently, but this one in general, demonstrates the concept.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;close all;&#xA;&#xA;% some experimental parameters &#xA;n = 6; % # circles&#xA;width = 256;&#xA;height = 256;&#xA;maxRadius = 60;&#xA;&#xA;% generate some random data &#xA;cx = rand(n, 1)*width;&#xA;cy = rand(n, 1)*height;&#xA;r = 4+rand(n, 1)*maxRadius;&#xA;&#xA;circIds = cell(width*height, 1); % image, where each pixel is a an array&#xA;&#xA;% now render the circles to integer coordinates&#xA;figure, hold on;&#xA;for i=1:n&#xA;    h = viscircles([cx(i), cy(i)], r(i), 'Color', colors(i,:));&#xA;&#xA;    list = double(midPointCircle(r(i), cx(i), cy(i)));&#xA;    li = double(list(:,2));&#xA;    lj = double(list(:,1));&#xA;    invalid = (li&amp;lt;=0 | li&amp;gt;height | lj&amp;lt;=0 | lj&amp;gt;width);&#xA;    li(invalid)=[];&#xA;    lj(invalid)=[];&#xA;    ind = sub2ind(([height, width]), li, lj);&#xA;    for j=1:length(circIds(ind))&#xA;        circIds(ind(j)) = {[cell2mat(circIds(ind(j))) i]};&#xA;    end&#xA;end&#xA;drawnow; hold off;&#xA;&#xA;% compute the label image &#xA;I = zeros(height, width);&#xA;Ilogical = (false(height, width));&#xA;for i=1:length(circIds)&#xA;    C = circIds{i};&#xA;    for j=1:length(C)&#xA;        [row, col] = ind2sub(([height, width]), i);&#xA;        I(row, col) = C(j);&#xA;        Ilogical(row, col) = 1;&#xA;    end&#xA;end&#xA;figure, imagesc(I);&#xA;&#xA;% find the connected components on the image&#xA;Ccomp = bwlabel(Ilogical,8); % O(WH)&#xA;figure, imagesc(Ccomp);&#xA;&#xA;mapping = zeros(n, 1);&#xA;% now collect the equivalences&#xA;for i=1:length(circIds)&#xA;    C = circIds{i};&#xA;    for j=1:length(C)&#xA;        [row, col] = ind2sub(([height, width]), i);&#xA;        mapping(C(j)) = Ccomp(row, col);&#xA;        if (all(mapping))&#xA;            break;&#xA;        end&#xA;    end&#xA;    if (all(mapping))&#xA;        break;&#xA;    end&#xA;end&#xA;&#xA;% the final connected components&#xA;mapping &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2017-12-23T22:17:05.170" LastActivityDate="2017-12-23T22:17:05.170" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28511" PostTypeId="1" CreationDate="2017-12-23T22:22:36.877" Score="1" ViewCount="232" Body="&lt;p&gt;&lt;strong&gt;UPDATED&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've coded a multiparticle MD simulation in 3D. It is based on Langevin Dynamics, with random impulse and dissipation. I think the program works correctly now? I have attached the plots of kinetic, potential, and total energy. I believe that in the case of &lt;strong&gt;no&lt;/strong&gt; Langevin Dynamics, total energy should be conserved as dissipation is turned off. But when dissipation is turned on, one shouldn't expect total energy conservation anymore. At this point, I would like to know if my results seem reasonable. I have included some of the more important code snippets (please ask for more if needed) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Physical parameters:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//V0 is the potential depth, r0 is the effective radius for the potential&#xA;double const m = 1., V0 = 1., r0 = 1., boxLength = 15., kT = 0.5;&#xA;int const n_atoms = 12;&#xA;double gamma = 0.5;   //damping coeff.&#xA;double var = 2.*gamma*m*kT*dt;  //variance of the gaussian distributed I&#xA;double c = (2.-gamma*dt)/(2.+gamma*dt);    //due to damping&#xA;double dt = 0.005;   //time step&#xA;int N = (int)(1000 / dt);   //simulation time&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Initializing the system&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void init(struct Atom system[n_atoms]){&#xA;    double rc = 0.9;&#xA;    double xx = 0., yy = boxLength, zz = boxLength;&#xA;    int atoms_in_row = (int)(boxLength / rc);&#xA;    int n = 0;&#xA;&#xA;    for(int k = 0; k &amp;lt; (int)boxLength; k++) {&#xA;        for(int i = 0; i &amp;lt;= atoms_in_row; i++) {&#xA;            for(int j = 0; j &amp;lt;= atoms_in_row; j++) {&#xA;                if(n &amp;gt;= n_atoms) break;&#xA;                system[n].x = xx;&#xA;                system[n].y = yy;&#xA;                system[n].z = zz;&#xA;                system[n].vx = system[n].vy = system[n].vz = 0.;&#xA;                system[n].ax = system[n].ay = system[n].az = 0.;&#xA;                xx += rc;&#xA;                n++;&#xA;            }&#xA;            yy -= rc;&#xA;            xx = 0.;&#xA;        }&#xA;        yy = 0.;&#xA;        zz -= rc;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are the energy plots with dissipation/random impulse turned off and on, respectively. The first one shows the plot of energy for two atoms only, in a non LD case. The next two are for 12 atoms. In particular, I think the result for non LD cases is fine as total energy is more or less conserved. And maybe the results are good for LD case as well? I'm not quite sure on this one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d5usk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d5usk.png&quot; alt=&quot;2 atoms no LD&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1Ug0z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1Ug0z.png&quot; alt=&quot;No LD&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wPeSX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wPeSX.png&quot; alt=&quot;LD&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26366" LastEditorUserId="26366" LastEditDate="2018-01-05T00:39:48.123" LastActivityDate="2018-01-05T00:39:48.123" Title="Discussing the energy spectrum of Langevin Dynamics simulation of many atoms" Tags="|computational-physics|simulation|molecular-dynamics|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28512" PostTypeId="2" ParentId="21320" CreationDate="2017-12-23T23:01:46.697" Score="1" Body="&lt;p&gt;If approximations suffice, maybe we could begin by developing the $\exp$ operator as follows:&#xA;$$&#xA;\exp(Z) = \sum\limits_{k=0}^{\infty} \frac{Z^k}{k!} = 1+Z+\frac{Z^2}{2} + \frac{Z^3}{6} + \frac{Z^4}{24} + ...&#xA;$$&#xA;Note that the power terms follow your element-wise notation and refer to the Hadamard powers, slightly abusing the standard conventions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gives possibilities for re-writing the expression:&#xA;$$&#xA;\begin{align}&#xA;\exp{(-XY^T)}G &amp;amp;= \Big(I-XY^T+\frac{(XY^T)^2}{2} - \frac{(XY^T)^3}{6} + \frac{(XY^T)^4}{24} - ...\Big)G \\&#xA;&amp;amp;=G-XY^TG+\frac{(XY^T)^2G}{2} - \frac{(XY^T)^3G}{6} + \frac{(XY^T)^4G}{24} - ...&#xA;\end{align}&#xA;$$&#xA;where $I$ is an appropriately sized identity matrix. If one could live with a first order approximations, then:&#xA;$$&#xA;\begin{align}&#xA;\exp{(-XY^T)}G &amp;amp;= G-XY^TG+ O(max(|XY^T|)^2)\\&#xA;&amp;amp;\approx G-XY^TG = G-X(Y^TG) &#xA;\end{align}&#xA;$$&#xA;resulting in an easier operation. For higher order approximations, you would like to work out the solution for the Hadamard power, which is probably a bit harder or I haven't seen an immediate solution yet.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2017-12-25T07:44:46.613" LastActivityDate="2017-12-25T07:44:46.613" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28513" PostTypeId="1" CreationDate="2017-12-24T09:25:08.103" Score="0" ViewCount="142" Body="&lt;p&gt;Can we linearize $T^2$ When solving $T''_x+T''_y+aT^2=0$ by finite difference? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solved $T''_x+T''_y=0$ in Matlab using a finite difference explicit scheme. But when there is a source term, I come up with a system of nonlinear algebraic equations and I can't solve it anymore. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better method for solving nonlinear equations without linearizing them? &lt;/p&gt;&#xA;" OwnerUserId="4950" LastEditorUserId="4950" LastEditDate="2017-12-24T18:49:35.130" LastActivityDate="2017-12-24T22:44:20.963" Title="How to treat non-linear term in finite difference solution of $T''_x+T''_y+aT^2=0$?" Tags="|pde|numerical-analysis|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28514" PostTypeId="2" ParentId="28513" CreationDate="2017-12-24T22:44:20.963" Score="1" Body="&lt;p&gt;The only systems of equations we know how to solve are linear. Everything else is solved using iterations of linear equations that we hope converge to the solution oft the linear systems. There are a number of ways to do this for nonlinear PDEs, and I've attempted to discuss them in lectures 31.5 to 31.7 here: &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2017-12-24T22:44:20.963" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28519" PostTypeId="1" CreationDate="2017-12-27T14:27:46.597" Score="6" ViewCount="1712" Body="&lt;p&gt;Simulated annealing is a useful technique for finding near-optimal solutions to combinatorial problems. I have found a lot of tutorials on implementing the basic algorithm, but miss a general guide as to how constraints are incorporated into the optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if anyone knows any sources for advice or algorithms for incorporating constraints into simulated annealing applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: I have constructed some algorithms for generating feasible permutations of the solutions, but would like to see more general results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is greatly appreciated&lt;/p&gt;&#xA;" OwnerUserId="23540" LastActivityDate="2018-04-19T10:08:57.827" Title="Constrained simulated annealing" Tags="|constrained-optimization|constraints|discrete-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28520" PostTypeId="2" ParentId="28484" CreationDate="2017-12-27T14:55:38.290" Score="3" Body="&lt;p&gt;I will only be developing the computation of forces on a given atom (part 1 of BalazsToth answer), for this point is actually a bit tricky and I think it deserves to be a bit more explained. You can find useful remarks in any book of molecular simulation my personal reference being &quot;Computational Simulation of Liquids&quot; from Allen and Tildesley and the &quot;Molecular Dynamics&quot; Lecture Notes on Physics from W. Hoover.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The force of a pair of particles interacting is directed along the vector defined by the difference of the coordinates of the particles. Let's call this vector $\mathbf{r}_{ij}$ and the unitary vector with the same direction:&#xA;$$ \mathbf{r}_{ij} = \mathbf{r}_i - \mathbf{r}_j\ ;\ \mathbf{n}_{ij}=\frac{\mathbf{r}_{ij}}{|\mathbf{r}_{ij}|}=\frac{\mathbf{r}_{ij}}{r}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $|\mathbf{r}_{ij}|=\sqrt{r_{ij,x}^2+r_{ij,y}^2+r_{ij,z}^2}=r$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you compute the force $f$ with regard to $r$ :&#xA;$$f_{ij} = -\frac{dV}{dr}$$&#xA;You will note that :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This value is positive when $r&amp;lt;r_0$ and negative when $r&amp;gt;r_0$.&lt;/li&gt;&#xA;&lt;li&gt;The force is pushing the atoms away from one another in the first case and getting them closer in the second.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Therefore the force exerced on atom $i$ is opposed in direction to vector $\mathbf{n}_{ij}$ as it is defined here. Some authors define it with opposite value, or for example, BalazsToth used $\mathbf{n}_{ji}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then add the force to the total force on the atom $i$ :&#xA;$$ \mathbf{F}_{i} = \sum_{j=1\ ;\ j \ne i}^N -f_{ij}\mathbf{n}_{ij}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One remark though concerning your algorithm. Given that $f_{ij}=-f_{ji}$ due to Newton's third law, you can make your loop outside of your function computing the forces. You can actually make your piece of code more like :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void ComputeForces{&#xA;for(int i=0;i&amp;lt;N_particles;i++){&#xA;  for(int j=i+1;j&amp;lt;N_particles,j++){&#xA;    Computeforce(atm[i], atm[j]);&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the function ComputeForce, taking two arguments allow you to shorten the loop as you go on and on as you can update the forces on atom $j$ as well.&lt;/p&gt;&#xA;" OwnerUserId="19790" LastEditorUserId="19790" LastEditDate="2017-12-28T23:11:55.070" LastActivityDate="2017-12-28T23:11:55.070" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28522" PostTypeId="1" CreationDate="2017-12-27T22:51:18.490" Score="2" ViewCount="177" Body="&lt;p&gt;What is this doing? Looks like the original eigenvalue problem is converted into generalized eigenvalue problems with different dimensions of collocation points. Can someone explain more about this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;eigs.m&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Linear combination coefficients for convergence test. The convergence of the&#xA;% combination is the same as the worst constituent function. The nontrivial&#xA;% coefficents are to make accidental cancellations extremely unlikely.&#xA;coeff = 1./(2*(1:k)');&#xA;for dim = [dimVals NaN]&#xA;    [V, D, P] = getEigenvalues(discA, discB, k, sigma);&#xA;&#xA;    % Combine the eigenfunctions into a composite.&#xA;    v = V*coeff(1:size(V,2));&#xA;&#xA;    % Convert the different components into cells&#xA;    u = partition(discA, P*v);&#xA;&#xA;    % Test the happiness of the function pieces:&#xA;    vscale = zeros(1, sum(isFun));   % intrinsic scaling only&#xA;    [isDone, cutoff] = testConvergence(discA, u(isFun), vscale, prefs);&#xA;&#xA;    if ( all(isDone) )&#xA;        break&#xA;    elseif ( ~isnan(dim) )&#xA;        % Update the discretiztion dimension on unhappy pieces:&#xA;        discA.dimension(~isDone) = dim;&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;eigs.m/getEigenvalues&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [V, D, P] = getEigenvalues(discA, discB, k, sigma)&#xA;% Formulate the discrete problem and solve for the eigenvalues&#xA;&#xA;    % Discretize the LHS operator (incl. constraints/continuity):&#xA;    [PA, P, C, ignored, PS] = matrix(discA);&#xA;&#xA;    % Discretize the RHS operator, or use identity.&#xA;    if ( ~isempty(discB) )&#xA;        % TODO: This is untidy. Can we make a method to do this? NH Apr 2014.&#xA;        discB.dimension = discA.dimension;&#xA;        PB = matrix(discB);&#xA;        % Project RHS matrix and prepend rows for the LHS constraints.&#xA;        PB = [ zeros(size(C)) ; PB ];&#xA;    else&#xA;        PB = [ zeros(size(C)) ; PS ];&#xA;    end&#xA;&#xA;    % Compute eigenvalues.&#xA;    if ( length(PA) &amp;lt;= 2000 )&#xA;        [V, D] = eig(full(PA), full(PB));&#xA;        lam = diag(D);&#xA;&#xA;        % Find the ones we're looking for.&#xA;        lam = deflate(lam, size(C,1));&#xA;        idx = nearest(lam, sigma);&#xA;        idx = filter(idx, P*V, k, discA);&#xA;&#xA;        % Extract them:&#xA;        V = V(:,idx);&#xA;        D = D(idx,idx);&#xA;&#xA;    else&#xA;        % TODO: Experimental.&#xA;        [V, D] = eigs(PA, PB, k, sigma);&#xA;    end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the full code see &lt;a href=&quot;https://github.com/chebfun/chebfun/blob/development/%40linop/eigs.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/chebfun/chebfun/blob/development/%40linop/eigs.m&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26416" LastActivityDate="2017-12-27T22:51:18.490" Title="What is Chebfun `eigs` doing" Tags="|matlab|pde|numerical-analysis|eigenvalues|chebyshev|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28525" PostTypeId="1" CreationDate="2017-12-28T14:17:25.937" Score="6" ViewCount="948" Body="&lt;p&gt;I am trying to understand Dirichlet and Neumann boundary conditions in FEM and I wanted to know if my inference is correct. To articulate my understanding, lets consider a simple case of TE and TM propagation in rectangular waveguides.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sqlon.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sqlon.png&quot; alt=&quot;TE and TM modes&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the case of TE (&lt;span class=&quot;math-container&quot;&gt;$E_z$&lt;/span&gt;=0 and &lt;span class=&quot;math-container&quot;&gt;$H_z$&lt;/span&gt;=&lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;) and in the case of TM (&lt;span class=&quot;math-container&quot;&gt;$H_z$&lt;/span&gt;=0 and &lt;span class=&quot;math-container&quot;&gt;$E_z$&lt;/span&gt;=&lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;). Consequently, in TE we solve for &lt;span class=&quot;math-container&quot;&gt;$H_z$&lt;/span&gt; using Helmholtz equation with Neumann boundary condition on all sides and in the case of TM we solve for &lt;span class=&quot;math-container&quot;&gt;$E_z$&lt;/span&gt; with Dirichlet boundary condition on all sides.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZuqsG.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZuqsG.png&quot; alt=&quot;FEM in Sadiku's book&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/pYy3v.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pYy3v.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The particular method of FEM and assembly procedure I am using is given by Clayton R Paul and Sadiku wherein the nodal value vector is bifurcated into two sub vectors namely the free node vector(&lt;span class=&quot;math-container&quot;&gt;$ \phi_{f} $&lt;/span&gt;) and the prescribed node vector(&lt;span class=&quot;math-container&quot;&gt;$ \phi_{p}$&lt;/span&gt;). The prescribed node vector is filled by applying Dirichlet boundary conditions to exterior nodes or by assigning excitation to nodes. The free node vector is unknown and has to be computed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ON9oV.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ON9oV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is particularly about Neumann boundary condition. My understanding is that since Neumann is implicit and a natural boundary condition in FEM, the node vector(&lt;span class=&quot;math-container&quot;&gt;$ \phi$&lt;/span&gt;) is entirely comprised of free node vector implying that every node in the domain is free and unknown. Is my understanding correct?&lt;/p&gt;&#xA;" OwnerUserId="26420" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2017-12-28T14:17:25.937" Title="Understanding Boundary Condition in FEM" Tags="|finite-element|boundary-conditions|electromagnetism|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28527" PostTypeId="2" ParentId="28422" CreationDate="2017-12-29T17:11:43.297" Score="1" Body="&lt;p&gt;Ok, just a small followup - maybe someone would find it useful. So I took a test volume to integrate which is, in prolate spheroidal coordinate system, per @Kirill suggestion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{z^{2}}{\mu ^{2}}+\frac{\rho^{2}}{\mu ^{2}-1} = L^{2} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{z^{2}}{\nu ^{2}}+\frac{\rho^{2}}{1-\nu ^{2}} = L^{2} $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;bounded with two curves, namely &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$2 \leqslant \mu \leqslant 3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gFDOQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gFDOQ.png&quot; alt=&quot;Curves bounding calculation volume&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the volume integral is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V = \int_{0}^{2\pi}\int_{-1}^{1}\int_{2}^{3} (\mu ^{2}-\nu ^{2})d\mu d\nu d\phi$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with exact value of $V = 24\pi \approx  75.398223686 $. When translated and rescaled into unit hypercube as required by cube4.2, the integral to be calculated is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V = 4\pi \int_{0}^{1}\int_{0}^{1}\int_{0}^{1} (p-2q+3)(p+2q+1)dp dq dw$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Results of all algorithms except Suave (cant get it to run for some reason) of cuba4.2 is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-------------------- Vegas test -------------------&#xA;VEGAS RESULT:   neval 4500      fail 0&#xA;VEGAS RESULT:   75.40052478 +- 0.06988593       p = 0.000&#xA;&#xA;-------------------- Divonne test -------------------&#xA;DIVONNE RESULT: neval 2134      fail 0  nregions 14&#xA;DIVONNE RESULT: 75.40013735 +- 0.03957392       p = 0.000&#xA;&#xA;-------------------- Cuhre test -------------------&#xA;CUHRE RESULT:   neval 381       fail 0  nregions 2&#xA;CUHRE RESULT:   75.39816000 +- 0.00000000       p = 0.099&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My original proposal was trying to integrate the characteristic function $\chi $ of the subset for which @Kirill suggested would break classical integration methods. Here, we'd have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V = 6^{3} \int_{0}^{1}\int_{0}^{1}\int_{0}^{1} \chi dx dy dz$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\chi = \left\{\begin{matrix}&#xA; &amp;amp; (2z-1)^{2}+1.125((2x-1)^{2}+(2y-1)^{2}) \leqslant 1\\ &#xA;1 &amp;amp;  \wedge\\ &#xA; &amp;amp; 2.25(2z-1)^{2}+3((2x-1)^{2}+(2y-1)^{2}) &amp;gt; 1\\ &#xA;0 &amp;amp; &#xA;\end{matrix}\right.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical integration results are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-------------------- Vegas test -------------------&#xA;VEGAS RESULT:   neval 52000     fail 1&#xA;VEGAS RESULT:   75.47990417 +- 0.36806625       p = 0.005&#xA;&#xA;-------------------- Divonne test -------------------&#xA;DIVONNE RESULT: neval 31419     fail 0  nregions 24&#xA;DIVONNE RESULT: 75.24224767 +- 0.07119902       p = 0.915&#xA;&#xA;-------------------- Cuhre test -------------------&#xA;CUHRE RESULT:   neval 50165     fail 1  nregions 198&#xA;CUHRE RESULT:   75.54317090 +- 19.77503339      p = 0.000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Ok, so in conclusion I can say two things - expressing integral function in prolate system makes considerable difference in precision and speed of calculation - entire order of magnitude. Trying to integrate characteristic function of a set doesn't break classical function in a sense that they can't deliver, it breaks them in as they need to do 100x evaluation more to get a much more imprecise result. Divonne has best performance in characteristic function integration, as it seems its importance mash adjust to borders way better than VEGAS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are speed tests made in python 3.6 with pycuba 4.2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Characteristic function integration:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;VEGAS: 237 ms ± 5.28 ms per loop&lt;/li&gt;&#xA;&lt;li&gt;DIVONNE: 145 ms ± 933 µs per loop &lt;/li&gt;&#xA;&lt;li&gt;CUHRE: 214 ms ± 2.87 ms per loop&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Prolate coordianate system:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;VEGAS: 37.6 ms ± 217 µs per loop&lt;/li&gt;&#xA;&lt;li&gt;DIVONNE: 29.6 ms ± 266 µs per loop&lt;/li&gt;&#xA;&lt;li&gt;CUHRE: 24.5 ms ± 260 µs per loop&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14941" LastActivityDate="2017-12-29T17:11:43.297" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28528" PostTypeId="2" ParentId="28473" CreationDate="2017-12-29T18:05:36.423" Score="1" Body="&lt;p&gt;Check this out: Wang, W., Shu, C.W., Yee, H.C., Kotov, D.V. and Sjögreen, B., 2015. High order finite difference methods with subcell resolution for stiff multispecies discontinuity capturing. Communications in Computational Physics, 17(2), pp.317-336.&lt;/p&gt;&#xA;" OwnerUserId="26429" LastActivityDate="2017-12-29T18:05:36.423" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28530" PostTypeId="1" CreationDate="2017-12-29T23:49:34.543" Score="4" ViewCount="932" Body="&lt;p&gt;I am using projection method and P2/P1 finite element method to solve the incompressible Navier-Stokes equations while the mesh is constantly adapted as the body moves (edge swapping, splitting and collapsing is performed locally based on a certain condition). The velocity on the new vertex is interpolated locally. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The test case considered is the flow past a cylinder at Reynolds number of 40. The drag coefficient is used to validate the results; and it compares well and does converge on the theoretical result of 1.50. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it can be observed from the figure that the results is not smooth and there is sudden change in drag coefficient every time when re-mesh is performed (see &quot;drag.e4&quot;). If we do not perform re-mesh, a smooth results are generated (see 'drag.117610' and 'drag.117740'). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think this happens because the interpolated velocity does not satisfy the divergence-free condition. I am wondering if there is any way to correct this interpolated velocity and make it divergence-free. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lRIK5.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lRIK5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26225" LastActivityDate="2017-12-31T07:00:39.153" Title="How can I make sure the flow is divergence-free when I use moving mesh?" Tags="|finite-element|fluid-dynamics|adaptive-mesh-refinement|projection|incompressible|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28531" PostTypeId="2" ParentId="28530" CreationDate="2017-12-30T03:00:07.433" Score="1" Body="&lt;p&gt;What sort of field is it? You've said &quot;divergence free&quot;, but do you mean harmonic (zero divergence and curl), solenoidal (non-curl), or a mix of both? The distinction is important because it will affect the complexity of the algorithm. The goal is to do a &lt;A HREF=&quot;https://en.wikipedia.org/wiki/Helmholtz_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Helmholtz decomposition&lt;/A&gt; numerically, interpolate the potentials, and then recalculate the velocities from the potentials. From the application, I'm going to guess that you're dealing with a velocity field that is a mix of both solenoidal and harmonic. So, you'll want to find a way to compute these integrals numerically,&#xA;\begin{align}&#xA;    \Phi(\mathbf{r}) &amp;amp; = \hphantom{\frac{1}{4\pi} \int_V \frac{\nabla'\times \mathbf{v}(\mathbf{r}')}{\left|\mathbf{r}-\mathbf{r}'\right|} \operatorname{d}r'^3 }-\frac{1}{4\pi} \oint_{\partial V} \hat{n}' \cdot \frac{\mathbf{v}(\mathbf{r}')}{\left|\mathbf{r} - \mathbf{r}'\right|} \operatorname{d}^2r' \\&#xA;    \mathbf{A}(\mathbf{r}) &amp;amp; = \frac{1}{4\pi} \int_V \frac{\nabla'\times \mathbf{v}(\mathbf{r}')}{\left|\mathbf{r}-\mathbf{r}'\right|} \operatorname{d}r'^3 - \frac{1}{4\pi} \oint_{\partial V} \hat{n}'\times \frac{\mathbf{v}(\mathbf{r}')}{\left|\mathbf{r}-\mathbf{r}'\right|}\operatorname{d}r'^2&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that's the case, you're going to need to decide how local you want the computation to be (how large the volume $V$ is). In principle, you could make $V$ the entire space, or so small that it contains no points but has them on the surface. From there, you'd need to numerically calculate the integrals at enough points that you can numerically calculate&#xA;$$&#xA;  \mathbf{v}(\mathbf{r}) = -\nabla \Phi(\mathbf{r}) + \nabla \times \mathbf{A}(\mathbf{r})&#xA;$$&#xA;at the newly desired mesh points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you could use these potentials as the basis for the computation, eliminating the need to step back and forth. Then you'd only need to keep the function $\Phi$ harmonic ($\nabla^2\Phi=0$) to maintain the divergence free condition. Of course, you'd also be introducing an unphysical degree of freedom in $\mathbf{A}$, and I have no idea what that would do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your mesh is fine enough (small compared to the size of any vortices in the calculation), you could make the local approximation that the velocity is the gradient of a harmonic scalar function. When I say local, I mean that the scalar function would only be temporarily defined for the interpolation step on a suitably small region without any expectation of global consistency (e.g. the &lt;A HREF=&quot;https://en.wikipedia.org/wiki/Magnetic_potential#Magnetic_scalar_potential&quot; rel=&quot;nofollow noreferrer&quot;&gt;magnetic scalar potential&lt;/A&gt;). Calculating new velocities based on that harmonic function will guarantee that the results remain divergence free. Deriving explicit formulae to do this is left to others, though looking information on &lt;A HREF=&quot;https://en.wikipedia.org/wiki/Polyharmonic_spline&quot; rel=&quot;nofollow noreferrer&quot;&gt;biharmonic splines&lt;/A&gt; should prove useful.&lt;/p&gt;&#xA;" OwnerUserId="8313" LastEditorUserId="8313" LastEditDate="2017-12-30T18:31:07.543" LastActivityDate="2017-12-30T18:31:07.543" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28532" PostTypeId="2" ParentId="28530" CreationDate="2017-12-30T03:07:59.677" Score="4" Body="&lt;p&gt;It's important to realize that the original velocity field $\mathbf v_0$ is &lt;em&gt;also&lt;/em&gt; not divergence free in a pointwise sense. Rather, it is only divergence free when tested with the pressure test functions, i.e., $(\nabla\cdot\mathbf v_0,q)=0$ for all $q\in P_1(T_0)$ where $T_0$ is the original mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you want to achieve is to get a velocity field $\mathbf v_1$ that lives on your new mesh, somehow resembles the $\mathbf v_0$ defined on the old mesh, and that satisfies $(\nabla\cdot\mathbf v_1,q)=0$ for all $q\in P_1(T_1)$ where $T_1$ is the new mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't do this through a local interpolation, but you can do this via a projection. However, this will require you to solve a global Laplace problem on the entire mesh -- in other words, it is no longer a local operation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As a postscript: To find this projection, you want $\mathbf v_1$ to be closest to $\mathbf v_0$ but subject to the constraint that it is (discretely) divergence free. In other words, you want to solve the following problem:&#xA;$$&#xA;  \min{\mathbf v_1 \in P_2(T_1)} \frac 12 \| \mathbf v_1 - \mathbf v_0 \|^2&#xA;  \\&#xA;  \text{subject to} \ (\nabla \cdot \mathbf v_1, q_i) = 0 \qquad  \forall q_i \in P_1(T_1).&#xA;$$&#xA;This is a constrained optimization problem that is easiest solved using a Lagrange multiplier approach. To this end, let $L: \in P_2(T_1) \times \in P_1(T_1) \rightarrow {\mathbb R}$ be the Lagrangian:&#xA;$$&#xA;L(\mathbf v_1,\lambda) =&#xA;\frac 12 \| \mathbf v_1 - \mathbf v_0 \|^2&#xA;  +&#xA; (\nabla \cdot \mathbf v_1, \lambda).&#xA;$$&#xA;Then you need to find that $\mathbf v_1, \lambda$ so that&#xA;$$&#xA;\frac{\partial L(\mathbf v_1,\lambda)}{\partial\mathbf v_1}(\mathbf w) =&#xA;(\mathbf w, \mathbf v_1 - \mathbf v_0)&#xA;+&#xA; (\nabla \cdot \mathbf w, \lambda) = 0,&#xA;\\&#xA;\frac{\partial L(\mathbf v_1,\lambda)}{\partial\lambda}(q) =&#xA; (\nabla \cdot \mathbf v_1, q) =0 .&#xA;$$&#xA;for all $w\in P_2(T_1), q \in P_1(T_1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is equivalent to finding the finite element solution of the system of equations&#xA;$$&#xA;  \mathbf v_1 - \nabla\lambda = \mathbf v_0,&#xA;\\&#xA;  \nabla \cdot \mathbf v_1 = 0.&#xA;$$&#xA;BY substituting the second into the (divergence of the) first equation, you will see that this corresponds to solving $-\nabla \cdot \nabla \lambda = \nabla\cdot v_0$, i.e., a Laplace equation. You could then compute $\mathbf v_1 = \mathbf v_0 +\nabla\lambda$. However, all of these equations have to be understood in the (finite dimensional) weak form shown above for the optimality condition, and you will want to solve the mixed problem instead.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2017-12-30T21:12:17.330" LastActivityDate="2017-12-30T21:12:17.330" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28533" PostTypeId="2" ParentId="28530" CreationDate="2017-12-30T19:39:48.270" Score="0" Body="&lt;p&gt;How about the Killing Vector Field? It is divergence-free (volume preserving),  generates isometric motion but does not regularize angular motion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, a vector field $X$ is a Killing field if the Lie derivative with respect to $X$ of the metric $g$ vanishes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;{\mathcal  {L}}_{{X}}g=0&#xA;$$&#xA;It is invariant of the coordinate frame and each Killing vector corresponds to a quantity which is conserved along geodesics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read more about it at &lt;a href=&quot;https://en.wikipedia.org/wiki/Killing_vector_field&quot; rel=&quot;nofollow noreferrer&quot;&gt;this wiki post&lt;/a&gt; and &lt;a href=&quot;http://www.physics.usu.edu/Wheeler/GenRel2013/Notes/GRKilling.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lately, Killing fields are used in deformable fusion of signed distance fields termed &lt;a href=&quot;http://campar.in.tum.de/pub/slavcheva2017cvpr/slavcheva2017cvpr.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;KillingFusion&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2017-12-31T07:00:39.153" LastActivityDate="2017-12-31T07:00:39.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28534" PostTypeId="1" CreationDate="2017-12-31T10:49:14.150" Score="1" ViewCount="92" Body="&lt;p&gt;I read several introductory texts about &lt;a href=&quot;https://en.wikipedia.org/wiki/Potential_flow&quot; rel=&quot;nofollow noreferrer&quot;&gt;potential flow&lt;/a&gt; and &lt;a href=&quot;http://s6.aeromech.usyd.edu.au/aerodynamics/index.php/sample-page/subsonic-aerofoil-and-wing-theory/3d-vortex-lattice-method/&quot; rel=&quot;nofollow noreferrer&quot;&gt;vortex lattice method&lt;/a&gt;. Basically, it is fitting of some velocity field described by conditions on velocity at some control points using basis functions placed in other points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;horse-shoe vortex:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to fullfill Helmholdz theorem that vortex filaments $\Gamma$ are closed loops (does not end in the air) &lt;a href=&quot;https://en.wikipedia.org/wiki/Horseshoe_vortex&quot; rel=&quot;nofollow noreferrer&quot;&gt;horse-shoe vortex&lt;/a&gt; is used as basis function. But it is not very nice because:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It is quite computationally costly to evaluate resulting velocity field using &lt;a href=&quot;https://en.wikipedia.org/wiki/Biot%E2%80%93Savart_law&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biot–Savart law&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;It imposes that vortex fillament is stright line behind the wing while in reality &lt;a href=&quot;https://en.wikipedia.org/wiki/File:Aircraft_wing_lift_distribution_showing_trailing_vortices_(3).svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;vortex fillaments curve and merge&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Delocalized trailing vortex:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we don't know how exactly filament looks like far behind the wing it would be better not to impose particular shape, but rather assume its position is uncertain (=delocalized) =&gt; vorticity $\Gamma$ smoothly blurred behind the wing. &lt;strong&gt;Do we really need strictly &lt;a href=&quot;https://en.wikipedia.org/wiki/Vortex#Irrotational_vortices&quot; rel=&quot;nofollow noreferrer&quot;&gt;irotational vortex&lt;/a&gt;&lt;/strong&gt; which concentrate vorticity in singular line?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Brainstorming:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm thinking if it would be possible to use somethink like &lt;strong&gt;&lt;em&gt;rotational dipole&lt;/em&gt;&lt;/strong&gt; which is velocity field generated by one infinitisimal element of vortex filament $d\Gamma$. I'm not exaclty sure about formula but something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vec \phi(\vec r)=(v_x,v_y,v_z) = (0, z/|\vec r|^3, y/|\vec r|^3)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This function is both faster to evaluate than horse-shoe and smoothly decays behind the wing. But I'm not sure how well it fulfills condition for proper potential flow basis function. Maybe with little modifications the errors (residual divergence and rotation) can be minimized ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe residual of such simple basisfunctions can be fitted to capture viscosity (like &lt;a href=&quot;https://en.wikipedia.org/wiki/Lamb%E2%80%93Oseen_vortex&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lamb–Oseen vortex&lt;/a&gt; )? Possibly somebody already have done this?&lt;/p&gt;&#xA;" OwnerUserId="4696" LastActivityDate="2017-12-31T10:49:14.150" Title="Vortex Lattice Method: better basis than horse-shoe vortex?" Tags="|fluid-dynamics|vorticity|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28535" PostTypeId="2" ParentId="28363" CreationDate="2017-12-31T22:50:41.240" Score="1" Body="&lt;p&gt;As Christian Clason alluded to in his comment, you have reinvented the ensemble Kalman filter! The &lt;a href=&quot;https://en.wikipedia.org/wiki/Kalman_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kalman filter&lt;/a&gt; is an algorithm for estimating, from imperfect measurements, the true state of a system governed by a set of first-order ODEs or difference equations. For example, you might only be able to make noisy measurements of the state of the system, or you might only be able to measure some of the state variables but not all. The measurements $y$ are assumed to be a linear function of the state variables $x$, plus some noise $\eta$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$y = Hx + \eta$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $H$ is the measurement matrix. In your case, the measurement matrix is not of full rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the great parts about the Kalman filter is that it gives you not just an estimate of the state of the system but a probability distribution, assuming that the errors are normally distributed, &lt;em&gt;even when the measurement matrix is not of full rank&lt;/em&gt;. While the backward ODE may be unstable, by also tracking covariance of the Kalman filter estimates of the system state, you can also put some bound on the uncertainty of your guess for the initial condition. This doesn't remedy your lack of information; it only tells you how this lack of information, i.e. only being able to observe the first two variables, results in uncertainty in the third variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://books.google.com/books?isbn=0387271325&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaipio and Somersalo&lt;/a&gt; have a chapter on Kalman filtering. I also found &lt;a href=&quot;https://sites.google.com/site/saravkin/publications/KalmanChapterSpringer.pdf?attredirects=0&amp;amp;d=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;these notes&lt;/a&gt; to be really helpful. If your ODE system is nonlinear, it might be beneficial to try implementing the algorithm for linear systems of ODE first. It's especially valuable to see how the results change when the governing system is stable, unstable, or oscillatory.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2017-12-31T22:50:41.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28536" PostTypeId="2" ParentId="28511" CreationDate="2018-01-01T12:39:26.647" Score="0" Body="&lt;p&gt;Based on a cursory look at your code, I think you might want to change the following lines in &lt;code&gt;calculateForce&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        system[i].ax += f * dx / d;&#xA;        system[i].ay += f * dy / d;&#xA;        system[i].az += f * dz / d;&#xA;        system[j].ax += -system[i].ax;&#xA;        system[j].ay += -system[i].ay;&#xA;        system[j].az += -system[i].az;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and replace them with something to the effect of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        system[i].ax += f * dx / d;&#xA;        system[i].ay += f * dy / d;&#xA;        system[i].az += f * dz / d;&#xA;        system[j].ax -= f * dx / d;&#xA;        system[j].ay -= f * dy / d;&#xA;        system[j].az -= f * dz / d;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Irrespective of your current problem, a good debugging strategy is to turn off the fluctuations and the dissipation introduced by Langevin dynamics (i.e., simply use Verlet) and see if you conserve energy. If the answer is negative, it means there is a bug that you need to fix. If you don't do this, it is possible that the stochastic dynamics will make it harder to spot a problem in your code.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2018-01-01T12:39:26.647" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28537" PostTypeId="1" AcceptedAnswerId="28561" CreationDate="2018-01-01T15:50:11.233" Score="0" ViewCount="380" Body="&lt;p&gt;This particular FEM question concerns waveguides and FEM 3D simulation. To excite a waveguide with waveport (TE10 and so on), we typically have to solve for eigenvalues ($k$) of helmholtz equation using FEM 2D.$$ \bigtriangledown^{2}\phi_{2D}+k^{2}\phi_{2D}=0 ,\quad\phi_{2D}=E_{z}(TM)\quad or \quad H_{z}(TE)$$ Upon solving, $k$ gives the permissible modes for the particular waveport dimensions and $\phi_{2D}$ gives the field patterns. I have got everything working and understood till this point but I am not sure of how to include these  results into FEM 3D simulation. &lt;a href=&quot;https://www.google.co.in/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=4&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=0ahUKEwjV0YjhirfYAhUBpI8KHaI-DDEQFgg9MAM&amp;amp;url=https%3A%2F%2Fwww.researchgate.net%2Fpost%2FHow_do_you_identify_whether_to_use_lumped_port_or_wave_port_in_a_microstrip_patch_antenna&amp;amp;usg=AOvVaw3vI5IYx8L4u5FkXF_WXqJ2&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; document from researchgate (pdf downloads on clicking) says that the mode patterns $\phi_{2D}$ found using FEM 2D are used as the excitation for FEM 3D. However, I dont quite understand how to include this source into FEM 3D.&#xA;What I think I must do is incorporate $\phi_{2D}$ (FEM2D) into the in-homogeneous helmholtz equation for FEM 3D as the source $g$ as given below.$$\bigtriangledown^{2}\phi+k^{2}\phi=g$$ But, I am not quite sure if this is correct. I haven't found good sources for waveports either.&lt;/p&gt;&#xA;" OwnerUserId="26420" LastActivityDate="2018-01-04T17:48:19.257" Title="Introducing EigenModes from 2D FEM into 3D FEM" Tags="|finite-element|wave-propagation|electromagnetism|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28539" PostTypeId="1" CreationDate="2018-01-01T23:15:11.733" Score="1" ViewCount="556" Body="&lt;p&gt;In structural mechanics, are the boundary conditions &quot;free surface,&quot; &quot;Traction free&quot;, &quot;stress free&quot; all equivalent Neumann boundary conditions? &lt;/p&gt;&#xA;" OwnerUserId="23915" LastActivityDate="2018-01-01T23:15:11.733" Title="Structural mechanics traction boundary condition question" Tags="|boundary-conditions|solid-mechanics|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28541" PostTypeId="1" CreationDate="2018-01-02T09:31:09.867" Score="1" ViewCount="62" Body="&lt;p&gt;Suppose we have this simple optimization problem&#xA;\begin{align*}&#xA;    \underset{x\in V}{\text{min}} &amp;amp;~ f(x) \\&#xA;    \text{s.t.}&amp;amp; ~x \leq \beta&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using slack variables&#xA;\begin{align*}&#xA;    \underset{x\in V}{\text{min}} &amp;amp;~ f(x) \\&#xA;    \text{s.t.}&amp;amp;~ x - \beta + s \leq 0 \\&#xA;               &amp;amp; s \geq 0&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lagrangian&#xA;\begin{equation*}&#xA;    \mathcal{L} \left(x, \lambda, s, \mu \right) = f(x) + \langle \lambda, x -\beta + s \rangle_{L_2} +\langle \mu, s \rangle_{L_2}&#xA;\end{equation*}&#xA;KKT conditions $\forall \delta \lambda, \delta x, \delta \mu, \delta s$&#xA;\begin{align*}&#xA;    \mathcal{L}_x (\delta x) &amp;amp;= \langle f'(x), \delta x \rangle_{L_2} + \langle \lambda, \delta x \rangle_{L_2} = 0\\&#xA;    \mathcal{L}_{\lambda} (\delta \lambda) &amp;amp;= \langle x - \beta + s, \delta \lambda \rangle_{L_2} = 0 \\&#xA;    \mathcal{L}_{s} (\delta s) &amp;amp;= \langle \lambda , \delta s \rangle_{L_2} + \langle \mu, \delta s \rangle_{L_2}= 0 \\&#xA;    s \geq 0 ~ \mu \leq 0 &amp;amp; ~\mu s= 0 ~\text{a.e.}&#xA;\end{align*}&#xA;Now assume we use an optimization algorithm that replaces the function $f(x)$ with an approximation easier to solve. It also replaces the box constraint $\beta$ with a value closer to the current optimization iteration.  One such method is &lt;a href=&quot;https://people.kth.se/~krille/mmagcmma.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MMA&lt;/a&gt;. Because of this substitution, the KKT conditions are different throughout the optimization. After all, the box constraint is different and their lagrange multiplier will be different as well. Is there any way to recover the lagrange multipliers of the original problem during the optimization?&lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2018-01-02T09:31:09.867" Title="Obtaining the lagrangian multipliers in an optimization problem" Tags="|optimization|constrained-optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28542" PostTypeId="1" CreationDate="2018-01-02T12:08:33.867" Score="1" ViewCount="39" Body="&lt;p&gt;I wanna ask a question that confuses me quite a long time. I saw many guys, in the context of computational mechanics,  they seemed to choose the virtual functions or kinematics in a way that some kinematical quantities are kept/fixed zero, but  they ask the remaining  of  them   to act as roles during the derivation of  a weak form equation. In the end, we can generally get a system of weak equations. I am all the time wondering whether we have a chance to change the nature of the original physical problem of a strong form.  Who can shed lights on my doubts in a physcial or mathematical view?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: &#xA;Say, u=u1+u2+u3 for whatever reason. Its variation is delta u1+ delta u2 +delta u3. The weak form may be integral sigma(u): sym gradient(delta u1) =0, integral sigma(u): sym gradient(delta u2) =0, integral sigma(u): sym gradient(delta u3) =0.&lt;/p&gt;&#xA;" OwnerUserId="24268" LastEditorUserId="24268" LastEditDate="2018-01-02T21:48:58.430" LastActivityDate="2018-01-02T21:48:58.430" Title="How do 'virtual kinematics/functions' play a role during deriving weak form formulations for physical problems?" Tags="|finite-element|pde|weak-solution|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="28544" PostTypeId="2" ParentId="28511" CreationDate="2018-01-02T18:29:30.810" Score="1" Body="&lt;p&gt;From your comments I think you should spend more time debugging your code. You have said that sometimes the program get trapped into an infinite loop: this is the first thing you need to fix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you to read three books that are very clear and well written in which concerns Molecular Dynamics method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://espressomd.org/wordpress/wp-content/uploads/2016/10/01-lennard_jones.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; &lt;em&gt;D. Frenkel &amp;amp; B. Smit&lt;/em&gt;, &lt;strong&gt;Understanding Molecular Simulation: From Algorithms to Applications&lt;/strong&gt;, 2nd Edition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;em&gt;Allen &amp;amp; Tildesley&lt;/em&gt;, &lt;strong&gt;Computer Simulation of Liquids&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] D.C. Rapaport, &lt;strong&gt;The Art of Molecular Dynamics Simulation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They bring routines implemented where you can compare to your code and verify if you are translating well the method to the programming. Follow the first one as the main reference for Molecular Dynamics and take a look into the others if you may have some doubt not clarified in the Frenkel.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Lennard-Jones Molecular Dynamics parametrisation &lt;a href=&quot;http://espressomd.org/wordpress/wp-content/uploads/2016/10/01-lennard_jones.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U(r) = 4 \epsilon \left[ \left(\frac{\sigma}{r}\right)^{12} - \left(\frac{\sigma}{r}\right)^6 \right]$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tstep = 0.01; // time step&#xA;temperature = 0.728; &#xA;epsilon = 1.0; // potential depth&#xA;sigma = 1.0; // effective radius&#xA;cut_off = 2.5*sigma; // LJ cut-off&#xA;min_distance = 0.87; // minimum initial distance between atoms&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26289" LastEditorUserId="26289" LastEditDate="2018-01-03T22:00:49.460" LastActivityDate="2018-01-03T22:00:49.460" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28545" PostTypeId="1" AcceptedAnswerId="28546" CreationDate="2018-01-02T21:35:32.103" Score="1" ViewCount="3351" Body="&lt;p&gt;This is a follow up question to my question yesterday &lt;a href=&quot;https://scicomp.stackexchange.com/questions/28539/structural-mechanics-traction-boundary-condition-question&quot;&gt;Structural mechanics traction boundary condition question&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does a traction free boundary condition also mean that the displacement gradient is 0 where the boundary condition is applied? My thinking is that it does not because it seems you can have traction free (zero stress) surface with a non-zero displacement gradient.&lt;/p&gt;&#xA;" OwnerUserId="23915" LastActivityDate="2018-01-03T00:31:46.567" Title="Structural mechanics - traction free = Zero displacement gradient?" Tags="|boundary-conditions|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28546" PostTypeId="2" ParentId="28545" CreationDate="2018-01-02T22:10:18.283" Score="4" Body="&lt;p&gt;The traction is defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{t} = \mathbf{n} \cdot \boldsymbol{\sigma}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of components, the zero-traction condition is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  t_j = \sum_i n_i \sigma_{ij} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the above you can see that the stress components don't necessarily have to be zero for the traction to be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a &lt;strong&gt;linear elastic&lt;/strong&gt; material,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \boldsymbol{\sigma} = \mathsf{C}:\nabla{\mathbf{u}}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In component form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  \sigma_{ij} = \sum_k \sum_l C_{ijkl} \frac{\partial u_k}{\partial x_l}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once again, since there is no requirement that all the $C_{ijkl}$ values have to be positive, you don't need all the displacement gradients to be zero for the stresses to be zero (which is not strictly necessary for zero-tractions anyway).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, zero displacement gradients will lead to zero tractions.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastEditorUserId="7626" LastEditDate="2018-01-03T00:31:46.567" LastActivityDate="2018-01-03T00:31:46.567" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28547" PostTypeId="1" AcceptedAnswerId="28548" CreationDate="2018-01-03T01:12:35.557" Score="2" ViewCount="289" Body="&lt;p&gt;According to &lt;a href=&quot;https://www.researchgate.net/publication/285908034_Numerical_Methods_for_Fluid_Dynamics&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper the following finite difference approximation is third-order accurate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d\rho_j}{dx}\approx\frac{2-\eta}{3}\frac{\rho_{j+1/2}-\rho_{j-1/2}}{\Delta x}+\frac{1+\eta}{3}\frac{\rho_{j-1/2}-\rho_{j-3/2}}{\Delta x}\text{, with }\eta=\frac{u\Delta t}{\Delta x}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't see how this is third-order accurate. If you Taylor expand $\rho_j$ you get:&#xA;$$\frac{\rho_{j+1/2}-\rho_{j-1/2}}{\Delta x}=\frac{d\rho_i}{dx}+\frac{\Delta x^2}{24}\frac{d^3\rho_j}{dx^3}+O(\Delta x^3),$$&#xA;$$\frac{\rho_{j-1/2}-\rho_{j-3/2}}{\Delta x}=\frac{d\rho_i}{dx}-\Delta x\frac{d^2\rho_j}{dx^2}+\frac{13\Delta x^2}{24}\frac{d^3\rho_j}{dx^3}+O(\Delta x^3).$$&#xA;Therefore, at best this scheme is first-order accurate? For reference see page 6 of the paper that is linked. Or this image here:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OKA7r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OKA7r.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26454" LastEditorUserId="26454" LastEditDate="2018-01-03T01:21:28.077" LastActivityDate="2018-01-03T05:32:02.430" Title="Computational Fluid Dynamics: Question on a third-order accurate finite difference approximation" Tags="|finite-difference|fluid-dynamics|accuracy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28548" PostTypeId="2" ParentId="28547" CreationDate="2018-01-03T05:32:02.430" Score="0" Body="&lt;p&gt;I think you are meant to examine the order of the whole scheme for the equation, rather than just this particular approximation of $\rho_x$. Substitute the whole expression back into the definition of the scheme (top of page 6, with $u$ missing):&#xA;$$ \frac{\rho^{n+1}_{j+\frac12}-\rho^n_{j+\frac12}}{\delta t} + u \frac{\bar\rho_{j+1}-\bar\rho_j}{\delta x} = 0 $$&#xA;and evaluate the residual for a true solution $\rho$ of the equation $\rho_t + u \rho_x = 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the 1st order scheme I get 1st order (&lt;a href=&quot;https://gist.github.com/anonymous/498da5e79bafe2c7a6928d0c3aa6fe3e&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;):&#xA;$$ \tfrac12 h u(\eta-1)\rho_{xx}, $$&#xA;second order:&#xA;$$ -\tfrac16 h^2u(\eta^2-1)\rho_{xxx}, $$&#xA;third order:&#xA;$$ \tfrac1{24}h^3u(\eta^3-2\eta^2-\eta+2)\rho_{xxxx}, $$&#xA;so the orders come out as the paper says.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-01-03T05:32:02.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28553" PostTypeId="2" ParentId="28469" CreationDate="2018-01-03T18:19:16.270" Score="1" Body="&lt;p&gt;Your call for $O(N)$ immediately triggers the &lt;a href=&quot;https://en.wikipedia.org/wiki/Multigrid_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;multigrid methods&lt;/a&gt; in my mind. &#xA;Multigrid techniques give algorithms that solve sparse linear systems $Ax = b$ &#xA;of $N$ unknowns with $O(N)$ computational and storage cost for large classes of problems. Thus it is also very suitable for your problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The steps of the general method follow:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Compute residuals &lt;/li&gt;&#xA;&lt;li&gt;Restrict the residual to coarse grid&lt;/li&gt;&#xA;&lt;li&gt;Solve the coarse system&lt;/li&gt;&#xA;&lt;li&gt;Prolongate the error to a finer grid&lt;/li&gt;&#xA;&lt;li&gt;Update the approximation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of these schemes are quite well studied and many approaches exist. Multigrids are also an active research area. &lt;a href=&quot;http://www.mgnet.org/mgnet/books/Wesseling/bookr-orig.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Check this out&lt;/a&gt; for an intro.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2018-01-03T18:19:16.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28554" PostTypeId="2" ParentId="28341" CreationDate="2018-01-03T18:24:48.627" Score="3" Body="&lt;p&gt;This approach is the result of a Taylor approximation, which assumes that we are not very far from the solution (solution is the case when $R\approx I$ - no more updates can be made). Under large rotations this linearization will fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, point to plane distance is prone to sliding errors when normals and points have particular configurations, such as the point lying on the plane. In these cases it is not the approximation, but the metric that suffers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, there is a &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/47152-icp-registration-using-efficient-variants-and-multi-resolution-scheme?focused=5751728&amp;amp;tab=function&quot; rel=&quot;nofollow noreferrer&quot;&gt;MATLAB code&lt;/a&gt;, as well as an &lt;a href=&quot;https://github.com/opencv/opencv_contrib/blob/master/modules/surface_matching/src/icp.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCV module&lt;/a&gt; implementing this paper. They also suffer from such drawback when the rotation is far apart.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2018-01-03T18:24:48.627" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28555" PostTypeId="2" ParentId="3128" CreationDate="2018-01-03T18:52:07.637" Score="4" Body="&lt;p&gt;Advantages of traditional Monte-Carlo integration over quasi-Monte Carlo integration are discussed in Kocis and Whiten's paper &lt;a href=&quot;https://dl.acm.org/citation.cfm?doid=264029.264064&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. They list the following reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The error bound of qmc methods $\mathcal{O}(\log(N)^{d}/N)$ is &quot;theoretically&quot; better than the $\mathcal{O}(N^{-1/2})$ bound given by naive Monte-Carlo. However, for values of $N$ that are achievable in reasonable time on current hardware, $\mathcal{O}(N^{-1/2})$ is the better option for $d \approx 40$. (Kocis and Whiten were writing in 1997, so presumably $d$ has increased a little since then.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The error of a QMC integrations is bound by the Koksma-Hlawka inequality&#xA;$$&#xA;    \mathrm{error} \le V[f]D_{N}^{*}&#xA;   $$&#xA;where $V[f]$ is the variation of $f$ and $D_{N}^{*}$ is the star of the discrepancy. But quoting from Kocis' paper,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unfortunately, the theoretical discrepancy bound&#xA;  of the existing sequences is not useable for moderate and large values of&#xA;  s. The other option, a numerical evaluation of the star discrepancy of a&#xA;  sequence for large s, requires an excessive computational effort, and even reasonable numeric estimates of such discrepancies are very difficult to&#xA;  obtain.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With traditional Monte-Carlo integration, we can specify an error goal and wait because the error bound is easily computable. With QMC, we have to specify a number of function evaluations and &lt;em&gt;hope&lt;/em&gt; the error is within our goal. (Note that there are techniques to overcome this, like randomized quasi-Monte Carlo, where multiple quasi-Monte Carlo estimates are used to estimate the error.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since the &quot;constant terms&quot; of the error bounds for many low-discrepancy sequences we actually possess grow exponentially with dimension, Kocis and Whiten use another metric to estimate the error: The maximum distance between sample points. This gives an error estimate of $\mathcal{O}(1/N^{1/2+2/d})$, and they claim it fits the observed behavior of many integrands.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For quasi-Monte Carlo to beat traditional Monte-Carlo, the integrand must have &quot;low effective dimension&quot;. See Art Owen's paper on this subject &lt;a href=&quot;http://statweb.stanford.edu/~owen/reports/necessity.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2018-01-03T20:36:51.373" LastActivityDate="2018-01-03T20:36:51.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28556" PostTypeId="1" CreationDate="2018-01-04T00:17:11.327" Score="5" ViewCount="82" Body="&lt;p&gt;I need to solve an integral equation in the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A(z)+\int\limits^{z_2}_{z_1}B(z') \frac{z^N}{z^N-z'^N} \frac{e^{i\beta}}{|z|}\mathrm{d}z'=0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A(z)$ distribution is known and we are solving for $B(z')$ distribution and $z,z'$ are complex variables. I also know the value of $B(z_2)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me how I can solve this equation using MATLAB (preferably numerically)?&lt;/p&gt;&#xA;" OwnerUserId="21174" LastEditorUserId="9667" LastEditDate="2018-01-04T00:48:46.927" LastActivityDate="2018-01-04T00:48:46.927" Title="Complex Integral Equation Solution in MATLAB" Tags="|matlab|integral-equations|complex-analysis|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28559" PostTypeId="1" AcceptedAnswerId="28560" CreationDate="2018-01-04T13:10:25.607" Score="3" ViewCount="758" Body="&lt;p&gt;I feel like publishing some previous works from my PhD thesis. I was using Mathematica to build a system of 2N partial differential equations for 2N functions by symbolic spatial Taylor expansion, then numerically integrate them with &lt;code&gt;NDSolve&lt;/code&gt; with respect to time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematica had a strange behaviour for some N orders, stopping to some apparent numerical singularities I was not able to manage at that time. Furthermore, I have an access to a supercomputer, which doesn't offer Mathematica. I would therefore like to rebuild my model in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the symbolic part, I guess SymPy will handle easily the Taylor expansion part (notice that after order 2 or 3, it took far more than a full page to Mathematica to write the PDE system, should I ask it to). But how do you numerically (Runge-Kutta or whatever) solve the objects coming out of SymPy? &lt;/p&gt;&#xA;" OwnerUserId="26471" LastEditorUserId="9667" LastEditDate="2018-01-04T14:48:49.087" LastActivityDate="2018-01-04T17:03:32.030" Title="Developping PDE with Python symbolically and numericaly" Tags="|pde|time-integration|runge-kutta|symbolic-computation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28560" PostTypeId="2" ParentId="28559" CreationDate="2018-01-04T15:14:39.240" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;But how do you numerically (Runge-Kutta or whatever) solve the objects coming out of SymPy? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That depends a lot on your specific problem and what level of optimization you need.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Most ODE solvers (such as those included in SciPy) require that you provide them a Python function representing the right-hand side of the ODE.&#xA;As you do not want things to be horribly slow, you have to take care of vectorisation somehow:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Depending on how your PDE is structured, this may work using SymPy’s &lt;code&gt;lambdify&lt;/code&gt; with a NumPy backend.&lt;/li&gt;&#xA;&lt;li&gt;An alternative is &lt;a href=&quot;http://docs.sympy.org/latest/modules/utilities/autowrap.html&quot; rel=&quot;noreferrer&quot;&gt;SymPy’s &lt;code&gt;ufuncify&lt;/code&gt; function&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Finally, I wrote &lt;a href=&quot;https://github.com/neurophysik/jitcode&quot; rel=&quot;noreferrer&quot;&gt;a module&lt;/a&gt; that takes SymPy input, compiles it, and feeds it into a SciPy integrator. This does not require vectorisable structures, but it can also not exploit them very well (as it is made with non-vectorisable problems in mind).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Either way, his means that you have to take care of spacial discretisations on the SymPy level.&#xA;You also will not be able to make use of PDE-specific optimisation techniques, in particular those involving GPUs; you may get multi-kernel support though (if that’s how you want to make use of the supercomputer)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You convert SymPy’s results into whatever input is needed by PDE-specific Python tools.&#xA;This probably involves meta-programming (code that writes code) of some sort, but could be rather harmless.&#xA;Note that SymPy features several code printers and similar that may assist you with this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2018-01-04T15:14:39.240" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28561" PostTypeId="2" ParentId="28537" CreationDate="2018-01-04T15:27:18.730" Score="3" Body="&lt;p&gt;You're looking for waveguide port boundary conditions. I think the most accessible treatment is within Jin &amp;amp; Riley's Finite Element Analysis of Antennas and Arrays, Chapter 5. It's available on Amazon, see &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0470401281&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.amazon.com/dp/0470401281/&lt;/a&gt;. A lot of these formulations were first introduced by Jin-Fa Lee, you can find his works in IEEE Microwave theory and techniques, see &lt;a href=&quot;http://ieeexplore.ieee.org/abstract/document/85399/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ieeexplore.ieee.org/abstract/document/85399/&lt;/a&gt; for example. This is a surprisingly complicated feature, you need to choose your discretization approach in the 2D solver rather carefully (mixed elements) if you need to deal with inhomogeneous cross sections that support hybrid modes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tend to think about this formulation almost like a domain decomposition approach, where in the main computational domain you expand ${\vec E}$ in subsectional FEM basis (Nedelec) and in the adjacent &quot;port&quot; or &quot;infinite waveguide&quot; domain, you expand ${\vec E}$ in a basis of waveguide modes. The modal solution ${\vec E_{2D}}$ plays two roles, it gives you the &quot;shape&quot; of the excitation currents ${\vec J}$/${\vec M}$, but it also provides the change-of-basis coefficients to establish tangential field continuity between the two domains. You don't need to use PML to match, the &quot;absorption&quot; behavior is built into the modal expansion (in fact, I would advise against using PML here, it's less accurate for modes near cutoff and doesn't absorb evanescent waves, but terminating with a modal expansion can handle both of these).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are free to either compute these modes numerically by solving the 2D eigenproblem, or represent them analytically (with tensor products of sines and cosines for rectangular waveguides, or Bessel functions for cylindrical waveguides, and so on). While the numerical solution is the most general, there can be advantages to the analytic expansions as well, mainly when it comes to robustly separating/orienting degenerate modes (modes that differ in polarization state or TE vs TM structure, but still propagate with the same k, so the numerical solver can't help but mix them all together). This is particularly acute for circular waveguides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a surprisingly long tail of code to implement this correctly/robustly for the general case: Code to detect canonical shapes and match them with analytic expansions, code to detect the presence of multiple conductors and invoke a 2D poisson solver to find TEM modes (which are also possibly degenerate, think common mode versus differential mode), and finally the general 2D eigensolver for extracting TE/TM modes on non-canonical shapes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that said, even though the 2D eigensolve is portrayed as the &quot;last line of defense&quot;, you can get pretty far using it alone, at least on simple structures (TE10 waveguides, coaxial cable). So I do think it's a good place to start.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-01-04T17:48:19.257" LastActivityDate="2018-01-04T17:48:19.257" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28562" PostTypeId="1" CreationDate="2018-01-04T16:24:14.777" Score="4" ViewCount="277" Body="&lt;p&gt;I'm running some simulation using FEM. In my model I need to apply a constraint condition to the governing equation.  My governing equation similar to the diffusion equation as below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial c}{\partial t}=\nabla(D\nabla c)\quad\text{on}\quad\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the flux is defined as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-D\nabla c\cdot n=J$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the constraint condition is an integral. For short, I need to constrain the total flux alone the specific surface equals to a constant, which means:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{\partial\Omega_{1}}(-D\nabla c\cdot n)d\Gamma=\text{Constant}$$&#xA;where $$\partial\Omega_{1}$$ is part of my domain's surface, not the total surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can write out the weak form for the governing equation(based on N-R method) as follow:&#xA;$$R_{c}^{I}=\int_{\Omega}\dot{c}N^{I}d\Omega+\int_{\Omega}D\nabla c\nabla N^{I}d\Omega-\int_{\partial\Omega}D\nabla c\cdot n N^{I}d\Gamma$$&#xA;and the related stiffness matrix:&#xA;$$K_{c\dot{c}}^{IJ}=-\frac{\partial R_{c}^{I}}{\partial\dot{c}^{J}}=-\int_{\Omega}N^{J}N^{I}d\Omega$$&#xA;$$K_{cc}^{IJ}=-\frac{\partial R_{c}^{I}}{\partial c^{J}}=-\int_{\Omega}D\nabla N^{J}\nabla N^{I}d\Omega+\int_{\partial\Omega_{1}}D\nabla N\cdot n N^{I}d\Omega$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I apply $$\int_{\partial\Omega_{1}}(-D\nabla c\cdot n)d\Gamma=\text{Constant}$$ to my FEM system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone told me that Lagrange multiplier can solve this problem, but I still don't know how to do it. I'm a newbie of FEM, so any details are helpful for me.&lt;/p&gt;&#xA;" OwnerUserId="26391" LastEditorUserId="26391" LastEditDate="2018-01-05T12:48:24.697" LastActivityDate="2018-01-05T12:48:24.697" Title="How to apply an integrated constrain condition in FEM?" Tags="|finite-element|pde|numerical-analysis|boundary-conditions|constrained-optimization|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28563" PostTypeId="2" ParentId="28559" CreationDate="2018-01-04T17:03:32.030" Score="3" Body="&lt;p&gt;I want to point out that &lt;a href=&quot;http://bjodah.github.io/pyodesys/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyODESys&lt;/a&gt; is a Python library that does exactly what you're looking for: send SymPy expressions to ODE solvers. It's nice because it links to more solvers, but @Wrzlprmft's JITCODE is probably better because it compiles the resulting expression and the runtime of the user's function is very important for solving an ODE efficiently.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-01-04T17:03:32.030" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28566" PostTypeId="2" ParentId="27740" CreationDate="2018-01-04T21:18:34.473" Score="0" Body="&lt;p&gt;If you care about finding global optimality, your best bet is to use a more advanced MINLP solver such as BARON. Since you're using MATLAB, try checking out the &lt;a href=&quot;https://www.inverseproblem.co.nz/OPTI/index.php/Solvers/BARON&quot; rel=&quot;nofollow noreferrer&quot;&gt;OPTI Toolbox&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="26477" LastActivityDate="2018-01-04T21:18:34.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28567" PostTypeId="1" CreationDate="2018-01-04T22:25:49.120" Score="1" ViewCount="131" Body="&lt;p&gt;I was asked to compute the Stokes flow (i.e. a low Reynolds fluid) near and obstacle. This is the first time I face a fluid and I am lost. What reference/general ideas/big theorems can you recommend me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More details:&lt;/strong&gt; given and obstacle (lets say, a bounded simply connected region $D\subset\mathbb{R}^2$ with $C^\infty$ boundary and all necessary stuff) and a steady flow $W\hat{x}$ from $-\infty\hat{x}$, I want to calculate the resultanting Stokes flow near $D$. For $D=\text{circle}$, the solution is well known (see, for example, page 4: &lt;a href=&quot;http://web.mit.edu/2.21/www/Lec-notes/chap2_slow/2-5Stokes.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://web.mit.edu/2.21/www/Lec-notes/chap2_slow/2-5Stokes.pdf&lt;/a&gt;). In this reference, the formulas are derived from the continuity equation $\nabla\cdot q =0$ with boundary conditions $\text{at infinity}$. My idea was to use this approach, but I don't even know if it is possible to solve numerically a PDE with this kind of boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="26478" LastEditorUserId="26478" LastEditDate="2018-01-04T22:33:36.917" LastActivityDate="2018-01-24T07:46:54.653" Title="Simulating Stokes flow with an obstacle" Tags="|fluid-dynamics|navier-stokes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28571" PostTypeId="1" AcceptedAnswerId="28576" CreationDate="2018-01-05T17:21:48.223" Score="0" ViewCount="1690" Body="&lt;p&gt;Numba offers JIT for Python. In its documentation it says &quot;One objective of Numba is having a seamless integration with NumPy.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why including some of the simplest features from numpy isn't possible:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from numba import *&#xA;&#xA;@jit(nopython=True)&#xA;def testfun(x):&#xA;     y = np.size(x)&#xA;     return y&#xA;&#xA;x=np.array([1 ,2, 3],dtype=float)&#xA;testfun(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run this code, I get the error &quot;Unknown attribute 'size' of type Module.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numba understands calls to NumPy ufuncs. I assume simple numpy functions such  as size, shape, sum, reshape, etc are ufuncs. &lt;/p&gt;&#xA;" OwnerUserId="26485" LastEditorUserId="26485" LastEditDate="2018-01-05T17:29:49.470" LastActivityDate="2018-01-06T07:07:34.560" Title="Numpy attributes not recognized in Numba" Tags="|python|numpy|" AnswerCount="1" CommentCount="0" ClosedDate="2018-01-06T08:50:24.343" ContentLicense="CC BY-SA 3.0" />
  <row Id="28572" PostTypeId="1" AcceptedAnswerId="28574" CreationDate="2018-01-05T18:41:33.007" Score="3" ViewCount="139" Body="&lt;p&gt;As in title, which method is the most optimal for numerical calculating value of:&#xA;$\arg(z_1z_2\cdots z_n)$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Method 1:&lt;/em&gt; one can first calculate $Z=z_1z_2\cdots z_n$ and then calculate $\arg(Z)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Method 2:&lt;/em&gt; using property $\arg(z_1z_2\cdots z_n) = \arg(z_1) + \dots +\arg(z_n)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or maybe something else?&lt;/p&gt;&#xA;" OwnerUserId="26487" LastEditorUserId="26487" LastEditDate="2018-01-07T08:50:50.077" LastActivityDate="2018-01-07T08:50:50.077" Title="How to calculate $\arg(z_1z_2\cdots z_n)$ to minimize results error?" Tags="|numerics|complex-analysis|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28573" PostTypeId="2" ParentId="28572" CreationDate="2018-01-05T22:40:29.200" Score="2" Body="&lt;p&gt;Because $\mbox{arg}$ wraps around at $2\pi$, the problem is ill-conditioned. You can get a forward error of $2\pi$ (when the sum of the arguments is close to a multiple of $2\pi$) with a backward error that approaches 0.  This is true of both of the methods you've proposed. &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-01-05T22:40:29.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28574" PostTypeId="2" ParentId="28572" CreationDate="2018-01-05T23:22:44.027" Score="6" Body="&lt;p&gt;As Brian Borchers pointed out, the question only makes sense up to factors of $2\pi$. I'll assume that you want an answer in $[0, 2\pi)$ then.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are really big contrasts in the magnitudes $|z_1|, \ldots, |z_n|$ of the numbers you're looking at, you could get some bogus results because of cancellation errors. In that case, it might arguably be better to compute all the arguments first and then add the result, since the magnitude of the product isn't even what you care about. On the other hand, you can concoct a similar edge case for the second approach by taking $z_1 = i$, $z_2 = \ldots = z_n = e^{i\epsilon}$ for really small $\epsilon$, and all of those numbers have the same magnitude. So it might be a tossup depending on the nature of the inputs. You'll probably want to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Pairwise_summation&quot; rel=&quot;noreferrer&quot;&gt;pairwise summation&lt;/a&gt; or better yet &lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Kahan summation&lt;/a&gt;, both of which can dramatically improve the accuracy of big sums.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No matter how you choose to go about the problem, measuring the results yourself will always be better than the advice of strangers on the internet. The program &lt;a href=&quot;https://github.com/uwplse/herbgrind&quot; rel=&quot;noreferrer&quot;&gt;herbgrind&lt;/a&gt; is a tool for finding where floating-point expressions go awry. You can also try using a library for arbitrary-precision floating point arithmetic like &lt;a href=&quot;https://gmplib.org/&quot; rel=&quot;noreferrer&quot;&gt;GMP&lt;/a&gt; to check how well one approach works over the other.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-01-05T23:22:44.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28575" PostTypeId="2" ParentId="28572" CreationDate="2018-01-05T23:36:13.053" Score="1" Body="&lt;p&gt;To sum the argument works faster and the calculus error is smaller. I think that that is the best way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we perform the product $ Z = z_1 \times z_2 \times ... \times z_n $ the error will be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ dZ = z_2 z_3 ... z_n dz_1 + z_1 z_3 .... z_n dz_2 + ... + z_1 z_2 ... z_{n-1} dz_n $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We conclude that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ dZ \approx Z \, (1/z_1 + 1/z_2 + ... 1/z_n) \, dz $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $ dz $ is the approximate error of each factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error in the argument of $Z$ will be proportional to this error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we sum the arguments, the error will be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ d \arg(Z) = \sum_{k=1}^n dz_k \approx n \, dz $$&lt;/p&gt;&#xA;" OwnerUserId="25160" LastEditorUserId="25160" LastEditDate="2018-01-06T14:00:55.270" LastActivityDate="2018-01-06T14:00:55.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28576" PostTypeId="2" ParentId="28571" CreationDate="2018-01-06T07:07:34.560" Score="1" Body="&lt;p&gt;Not sure why it is not supported, but this works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from numba import *&#xA;&#xA;@jit(nopython=True)&#xA;def testfun(x):&#xA;     y = x.size&#xA;     return y&#xA;&#xA;x=np.array([1 ,2, 3],dtype=float)&#xA;testfun(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You might want to have a look at the Supported NumPy features on their web-site.&lt;/p&gt;&#xA;" OwnerUserId="22185" LastActivityDate="2018-01-06T07:07:34.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28577" PostTypeId="1" CreationDate="2018-01-07T12:58:37.843" Score="1" ViewCount="114" Body="&lt;p&gt;Hi and thank you all again. I am solving the reaction-diffusion-advection equation as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \partial_{t} n\left(t,z\right) = -\partial_{z}\left(\Phi\left(t,z\right)\right)  + p\left(t,z\right) -n\left(t,z\right)l\left(t,z\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms $p\left(t,z\right)$ and $l\left(t,z\right)$ are computed according to big sets of chemical, and photo-chemical reactions. These reactions are characterized by rates which span huge different scales. Hence the system is stiff. I clarify this point because otherwise a simple Euler-Implicit scheme is perfectly valid for any time step if only transport $\Phi\left(t,z\right)$ is present. Nevertheless the solution absolutely fails involving chemistry if I cannot handle the truncation error. Since I need to integrate for great amounts of times (up to millions of years) to achieve the stationary state, hence I need an adaptive time step scheme. I solve the stationary state by pure time evolution because of further research requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the best solution was using ODEPACK. Taking the spatial discretization we end up with a system of ordinary differential equations in which each dependent variable is the density of a certain chemical species at a certain height.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://computation.llnl.gov/casc/odepack/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://computation.llnl.gov/casc/odepack/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0032063310002266&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; I attach a publication which basically solves the same kind of system as mine (planetary atmosphere), and uses ODEPACK. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to use ODEPACK within my fortran90 code I work with a modern fortran &lt;a href=&quot;https://sites.google.com/site/pjpapasot/fortran/libraries/dlsodar_f95&quot; rel=&quot;nofollow noreferrer&quot;&gt;ODEPACK interface&lt;/a&gt; to call the old fashion fortran77 ODEPACK routines. In particular the one known as DLSODAR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I faced the problem that sometimes DLSODAR yields negative densities which does not make sense in this physical context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My solution was simple. The integration step is made through calling another routine within DLSODAR code, named DSTODA. I introduce some code to just detect if any dependent variable becomes negative &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C-----------------------------------------------------------------------&#xA;C   CALL DSTODA(NEQ,Y,YH,NYH,YH,EWT,SAVF,ACOR,WM,IWM,F,JAC,DPRJA,DSOLSY)&#xA;C-----------------------------------------------------------------------&#xA;  DO 295 I = 1,N&#xA;  295    YOLD(I) = Y(I)&#xA;  TOLD = TN&#xA;  HOLD = H&#xA;  SIGNVAL = .TRUE.&#xA;  CALL DSTODA (NEQ, Y, RWORK(LYH), NYH, RWORK(LYH), RWORK(LEWT),&#xA; 1   RWORK(LSAVF), RWORK(LACOR), RWORK(LWM), IWORK(LIWM),&#xA; 2   F, JAC, DPRJA, DSOLSY)&#xA;&#xA;  CALL POSITIVENESS (SIGNVAL,NEQ,Y) ! (30 - 11 - 2017)&#xA;  IF(SIGNVAL.EQV..FALSE.) KFLAG = - 3&#xA;&#xA;  KGO = 1 - KFLAG&#xA;  GO TO (300, 530, 540, 400), KGO ! IF KFLAG = -3, solution is negative and DLSODAR returns&#xA;                                  ! from the last non negative solution to repeat if desired&#xA;                                  ! with a smaller time step H. (14 - 12 - 2017)&#xA;&#xA;C-----------------------------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;CALL POSITIVENESS(SIGNVAL,NEQ,Y)&lt;/code&gt; statement basically looks for each &lt;code&gt;Y(I)&lt;/code&gt; possible and when it is less than zero, the routine returns the last &lt;code&gt;Y&lt;/code&gt; with all values greater than zero, altogether with the attempted time step. Both stored before &lt;code&gt;DSTODA&lt;/code&gt; call in &lt;code&gt;YOLD&lt;/code&gt; &lt;code&gt;HOLD&lt;/code&gt; respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that returned unfinished solution, I reduce the last attempted time step by a factor $1/10$ and proceed from the last step. Furthermore, when from &lt;code&gt;DSTODA&lt;/code&gt; routine the time derivative and jacobians are called, if some density is negative, those auxiliar routines do not change the previous computed time derivative and jacobian. I found this important because sometimes DSTODA may yield a positive integrated solution but not correct. Hence not changing the time derivative in any case will change the solution's sign. I think this is due to the fact that sometimes the time derivative or jacobian are computed with intermediate &lt;code&gt;Y&lt;/code&gt; solutions that not being positive may get a positive one.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;--------------------------------------------------------------------------&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;It works, except for a certain kind of reactions. The physical origin of the equation does not care because as other reactions is characterized by a forward and reverse reaction rates. Anyway the forward mechanism has $k_{f} \approx {10^{-24},10^{-12}} cm^{3} s^{-1}$ and reverse rate $k_{r} \approx {10^{-38},10^{-27}} s^{-1}$. I will give the particular example as an independent simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reaction is as follows, considering we have D = 500 height layers (index j) and nElems = 3 (index i) chemical species&lt;/p&gt;&#xA;&#xA;&lt;p&gt;y(1+(j-1)*nElems) + y(2+(j-1)*nElems) &amp;lt;---&gt; y(3+(j-1)*nElems)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;use dlsodar_f95&#xA;&#xA;implicit none&#xA;integer i,j,jj,u, D, nElems&#xA;integer lrw, liw, MU, ML&#xA;double precision p(500),kf(500),kr(500)&#xA;double precision n(3,500),sol_f(3*500),dummy1(7),dummy2(2)&#xA;double precision sol(3*500)&#xA;double precision t, xend, dt, dti, tf&#xA;double precision Rtol, Atol(3*500)&#xA;character(len=1024) forward, reverse&#xA;&#xA;&#xA;D = 500&#xA;nElems = 3&#xA;&#xA;! Importing reaction constants&#xA;open(unit=10,file='reaction.dat')&#xA;  read(10,*) p(:)&#xA;  read(10,*) forward, kf(:)&#xA;  read(10,*) reverse, kr(:)&#xA;close(10)&#xA;&#xA;&#xA;! Importing intial water density profile&#xA;n(:,:) = 0.0d0&#xA;open(unit=10,file='inProf.dat')&#xA;  read(10,*)&#xA;  do i = 1,D&#xA;        read(10,*) dummy1(:), n(3,i), n(1,i), dummy2(:), n(2,i)&#xA;  end do&#xA;close(10)&#xA;&#xA;! Solving using DLSODAR&#xA;t = 0.0d0&#xA;dt = 0.0d0&#xA;Rtol = 1e-1&#xA;Atol(:) = 1e-2&#xA;MU = 3&#xA;ML = 3&#xA;lrw = 22+(10 + (2*(MU)+ ML))*nElems*D&#xA;liw = 20+nElems*D&#xA;tf = 1e1&#xA;dti = 1d-15 ! dti is the trial initial time step &#xA;u = 0&#xA;do i = 1,D&#xA;  do j=1,nElems&#xA;    u = u + 1&#xA;    sol(u) = n(j,i)&#xA;  end do&#xA;end do&#xA;do while (t.lt.tf)&#xA;      xend = tf&#xA;  call ODE_DLSODAR(t,xend,sol,derivs,jac,&#xA; *                     DLSODAR_dummy_Constraint,0,&#xA; *                     xend,sol_f,dti,dt,nElems,nElems,liw,lrw,&#xA; *                     relative_tolerance=RTOL,&#xA; *                     absolute_tolerances=ATOL,&#xA; *                     max_steps=int(1e9), ! If max_steps is too big, maybe there is an allocation error&#xA; *                     store_steps=.false.,verbose_level=1)&#xA;  ! &#xA;  ! If solution is negative this loop will repeat from last non negative&#xA;  ! iteration but with a smaller time step&#xA;  ! &#xA;  t = xend&#xA;  if (t.lt.tf) then&#xA;&#xA;    dti =  dt/10.&#xA;    u = 0&#xA;    do i = 1,D&#xA;      do j=1,nElems&#xA;        u = u + 1&#xA;        sol(u) = sol_f(u)&#xA;      end do&#xA;    end do&#xA;  end if&#xA;end do&#xA;&#xA;u = 0&#xA;do i = 1,D&#xA;  do j=1,nElems&#xA;    u = u + 1&#xA;    n(j,i) = sol_f(u)&#xA;  end do&#xA;end do&#xA;open(unit=10,file='finProf.dat')&#xA;  do i = 1,D&#xA;        write(10,*) p(i), n(1,i), n(2,i), n(3,i),&#xA; *                  kf(i),kr(i)&#xA;  end do&#xA;close(10)&#xA;&#xA;contains&#xA;c --------------------------------------------------------------&#xA;&#xA;subroutine jac(NEQ, X, Y, ML, MU, PD, NROWPD)  &#xA;  implicit none&#xA;  ! Arguments&#xA;  integer, intent(in) :: NEQ,ML,MU,NROWPD&#xA;  double precision, intent(in) :: X&#xA;  double precision, dimension(NEQ), intent(in) :: Y&#xA;  double precision, dimension(NROWPD,NEQ), intent(out) :: PD&#xA;  ! Local variables&#xA;  integer i, j,u    &#xA;  double precision kf(500),kr(500)&#xA;  character(len=1024) forward, reverse&#xA;&#xA;  do u = 1, NEQ&#xA;    if (Y(u).NE.Y(u)) then&#xA;      print*, &quot;yields NAN&quot;&#xA;      stop&#xA;    end if&#xA;    if (Y(u).lt.0.0d0) then&#xA;      return&#xA;    end if&#xA;  end do&#xA;&#xA;      PD(:,:) = 0.0d0&#xA;&#xA;! Importing reaction constants&#xA;open(unit=10,file='reaction.dat')&#xA;  read(10,*) &#xA;  read(10,*) forward, kf(:)&#xA;  read(10,*) reverse, kr(:)&#xA;close(10)&#xA;&#xA;  print*, &quot;dentro jac:&quot;,x&#xA;&#xA;  do i=1,500&#xA;      PD(2,3+(i-1)*3) =  kr(i)&#xA;      PD(3,2+(i-1)*3) = -kf(i)*Y((i-1)*3+1)&#xA;      PD(3,3+(i-1)*3) =  kr(i)&#xA;      PD(4,1+(i-1)*3) = -kf(i)*Y((i-1)*3+2)&#xA;      PD(4,2+(i-1)*3) = -kf(i)*Y((i-1)*3+1)&#xA;      PD(4,3+(i-1)*3) = -kr(i)&#xA;      PD(5,1+(i-1)*3) = -kf(i)*Y((i-1)*3+2)&#xA;      PD(5,2+(i-1)*3) =  kf(i)*Y((i-1)*3+1)&#xA;      PD(6,1+(i-1)*3) =  kf(i)*Y((i-1)*3+2)&#xA;  end do &#xA;&#xA;  return&#xA;end subroutine jac&#xA;c --------------------------------------------------------------&#xA;&#xA;subroutine derivs(NEQ,X,Y,DYDX) &#xA;  implicit none&#xA;  integer, intent(in) :: NEQ! number of variables&#xA;  double precision, intent(in) :: X! independent variable, time&#xA;  double precision, dimension(NEQ), intent(in) :: Y ! dependent variables. Unknowns algorithm is solving&#xA;  double precision, dimension(NEQ), intent(out) :: DYDX ! time derivative&#xA;  integer u ! running through nElems*D positions&#xA;  integer i, j, counter&#xA;  double precision dummy(2) ! dummy variable    &#xA;  double precision kf(500),kr(500)&#xA;  character(len=1024) forward, reverse&#xA;&#xA;&#xA;  do u = 1, NEQ&#xA;    if (Y(u).NE.Y(u)) then&#xA;      print*, &quot;yields NAN&quot;&#xA;      stop&#xA;    end if&#xA;    if (Y(u).lt.0.0d0) then&#xA;      return&#xA;    end if&#xA;  end do&#xA;&#xA;      DYDX(:) = 0.0D0&#xA;&#xA;! Importing reaction constants&#xA;open(unit=10,file='reaction.dat')&#xA;  read(10,*) &#xA;  read(10,*) forward, kf(:)&#xA;  read(10,*) reverse, kr(:)&#xA;close(10)&#xA;&#xA;&#xA;  print*, &quot;dentro dydx:&quot;,x&#xA;&#xA;  do i=1,500&#xA;      DYDX((i-1)*3+1) = -kf(i)*Y((i-1)*3+1)*Y((i-1)*3+2)&#xA; *                           + kr(i)*Y((i-1)*3+3)&#xA;      DYDX((i-1)*3+2) = -kf(i)*Y((i-1)*3+1)*Y((i-1)*3+2)&#xA; *                           + kr(i)*Y((i-1)*3+3)&#xA;      DYDX((i-1)*3+3) =  kf(i)*Y((i-1)*3+1)*Y((i-1)*3+2)&#xA; *                           - kr(i)*Y((i-1)*3+3)&#xA;  end do&#xA;&#xA;&#xA;  return&#xA;end subroutine derivs&#xA;c --------------------------------------------------------------&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And part of the output is as follows (I removed from the source code some prints* for the sake of being clearer)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; dentro dydx:   4.1576048498046223     &#xA; dentro dydx:   4.1577169032470946     &#xA; dentro dydx:   4.1577169032470946     &#xA; dentro dydx:   4.1578289566895670     &#xA; dentro dydx:   4.1578289566895670     &#xA; dentro dydx:   4.2698823991657884     &#xA; dentro dydx:   4.2698823991657884     &#xA; dentro dydx:   4.1858423173086221     &#xA; dentro dydx:   4.1858423173086221     &#xA; Y(I) :  -1.4252481539273311E-013&#xA; Where I :          25&#xA; Ha salido negativo&#xA; el xend alcanzado es :   4.1578289566895670     &#xA; pasos temporales, dti :   1.1205344247195033E-004 , dt :  0.11205344247622159     &#xA; dentro dydx:   4.1578289566895670     &#xA; dentro dydx:   4.1690343009371889     &#xA; dentro dydx:   4.1690343009371889     &#xA; Y(I) :  -8.0043582938772017E-007&#xA; Where I :           4&#xA; Ha salido negativo&#xA; el xend alcanzado es :   4.1578289566895670     &#xA; pasos temporales, dti :   1.1205344247622160E-002 , dt :   1.1205344247622160E-002&#xA; dentro dydx:   4.1578289566895670     &#xA; dentro dydx:   4.1589494911143294     &#xA; dentro dydx:   4.1589494911143294     &#xA; Y(I) :  -1.0391305382066468E-010&#xA; Where I :           4&#xA; Ha salido negativo&#xA; el xend alcanzado es :   4.1578289566895670     &#xA; pasos temporales, dti :   1.1205344247622159E-003 , dt :   1.1205344247622159E-003&#xA; dentro dydx:   4.1578289566895670     &#xA; dentro dydx:   4.1579410101320429     &#xA; dentro dydx:   4.1579410101320429     &#xA; dentro dydx:   4.1580530635745188     &#xA; dentro dydx:   4.1580530635745188     &#xA; dentro dydx:   4.2701065061024490     &#xA; dentro dydx:   4.2701065061024490     &#xA; dentro dydx:   4.1860664242065013     &#xA; dentro dydx:   4.1860664242065013     &#xA; Y(I) :  -1.0170965354084183E-013&#xA; Where I :          25&#xA; Ha salido negativo&#xA; el xend alcanzado es :   4.1580530635745188     &#xA; pasos temporales, dti :   1.1205344247622159E-004 , dt :  0.11205344252793006  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically the time step is stuck between 1e-3 s and 1e-1 s. It is not worth to wait more than a day just to see if ODEPACK finally decides to increase dt. I tried changing tolerances too. But the biggest issue is that the solution is wrong too. As an example a t = 100 s I display the simulation output&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Vu01F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vu01F.png&quot; alt=&quot;simulation output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the red solution (corresponding to Y(3)) those peaks are wrong. The initial condition is taken from a well-mixed hydrostatic atmosphere with a given composition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an extension, if we let DLSODAR to run on its own, without any modifications at al, we have the following output, for t = 1e12 s &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UW3UC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UW3UC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the white discontinuity is how gnuplot deals when we have negative values on a logarithmic scale. I should not accept negative solutions at any time value. This output only takes a few seconds to be computed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final (hopefully correct) stationary state solution if anyway we let DLSODAR to keep computing no matter negative numbers looks like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XLM4j.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XLM4j.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Any suggestions when ODEPACK yields negative non physical solutions?. Is this a known problem common to occur?&lt;/strong&gt;. I feel the problem should be so easy to solve, but my mind cannot see it clearer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, thank you so much.&lt;/p&gt;&#xA;" OwnerUserId="23659" LastActivityDate="2018-01-07T12:58:37.843" Title="Dealing with non-physical negative ODE solutions using ODEPACK" Tags="|pde|fortran|libraries|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28579" PostTypeId="2" ParentId="28401" CreationDate="2018-01-08T00:14:55.293" Score="2" Body="&lt;p&gt;To answer your question, I will for the sake of simplicity let $v=1$ and consider the one-dimensional case&#xA;\begin{equation}&#xA;g_t + g_x = 0&#xA;\tag{1}&#xA;\end{equation}&#xA;where subscripts denote partial derivatives. The extension to 2D and $v \neq 1$ is trivial but algebraically a little more tedious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you say, the solution to (1) is a 'shape' propagating through the domain, i.e. it is a wave. We can decompose this shape into a series of plane waves (a Fourier series). To see what (1) does to each component of the shape, we simply assign a plane wave solution to our problem,&#xA;\begin{equation}&#xA;g(x,t) = \exp{i(\kappa x - \omega t)},&#xA;\tag{2}&#xA;\end{equation}&#xA;for some wavenumber $\kappa$ and frequency $\omega$. We see indeed that (2) is a solution to (1) if&#xA;$$&#xA;\omega = \kappa.&#xA;\tag{3}&#xA;$$&#xA;Relation (3) is called the &lt;em&gt;analytic dispersion relation&lt;/em&gt; and tells us the frequency of a wave with given (real) wavenumber.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's discretise in space with backward differences as you have done:&#xA;$$&#xA;(g_i)_t + \frac{g_i - g_{i-1}}{dx} = 0.&#xA;\tag{4}&#xA;$$&#xA;Plugging in a plane wave solution&#xA;$$&#xA;g_i = \exp{i(\kappa x_i - \bar{\omega} t)}&#xA;\tag{5}&#xA;$$&#xA;now yields a &lt;em&gt;numeric dispersion relation&lt;/em&gt;&#xA;$$&#xA;\bar{\omega} = -\frac{i}{dx}(1 - \exp{(-i \kappa dx)}),&#xA;\tag{6}&#xA;$$&#xA;providing an expression for the numeric frequency $\bar{\omega}$ of a wave with a given (real) wavenumber $\kappa$. (6) is an approximation of (3), and indeed, as $dx \rightarrow 0$ they become identical. However, note that the numerical frequency has an imaginary and non-positive component for any choice of $\kappa$. Inserting (6) into (5) thus gives, not just a propagating wave, but an exponentially decaying one. This is what you have observed in your simulation. The decay is suppressed by choosing a small $dx$ but it is nonetheless exponential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ways to remedy this situation. A higher order backward stencil will give a numerical dispersion relation that approximates (3) better, and will have a smaller decay rate. However, if you want to rid yourself of the decay all together, choose a central stencil in space, e.g.&#xA;$$&#xA;g_t + \frac{g_{i+1} - g_{i-1}}{2dx} = 0.&#xA;\tag{7}&#xA;$$&#xA;Inserting the plane wave (5) into (7) gives the numeric dispersion relation&#xA;$$&#xA;\bar{\omega} = \frac{1}{dx} \sin{(\kappa dx)},&#xA;$$&#xA;i.e. the numeric frequency is real and no decay will be seen. Observe however that the central stencil is unstable with the forward difference in time, so use it with e.g. the 4th order Runge-Kutta method instead.&lt;/p&gt;&#xA;" OwnerUserId="19512" LastActivityDate="2018-01-08T00:14:55.293" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28582" PostTypeId="1" CreationDate="2018-01-10T05:08:40.200" Score="1" ViewCount="640" Body="&lt;p&gt;The stiffness matrix $K$ in a finite element analysis is a symmetric positive definite matrix. When we introduce essential boundary conditions, we remove rows and columns associated with prescribed displacements, for example. And we obtain a linear system of equations $\bar{K}\bar{X}=\bar{F}$ in which $\bar{K}$ is the reduced stiffness matrix and is obtained by removing rows and columns of stiffness matrix $K$ associated with essential boundary conditions. My project is about damage in cyclic loading (low-cycle, $10^3$). This is a non-linear analysis because of plasticity and damage involved in. So, the run-time of the code that I am writing will be too large. I want to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conjugate Gradient method&lt;/a&gt; for solving the systems of linear equations during the analysis. But the method requires that the reduced stiffness matrix $\bar{K}$ is symmetric and positive definite. The symmetry of $\bar{K}$ is trivial, but I do not know whether it is always positive definite or not.&lt;/p&gt;&#xA;" OwnerUserId="26520" LastActivityDate="2018-01-10T05:08:40.200" Title="Is reduced stiffness matrix positive definite too?" Tags="|linear-algebra|finite-element|krylov-method|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28583" PostTypeId="1" CreationDate="2018-01-10T10:23:43.273" Score="0" ViewCount="92" Body="&lt;p&gt;If I have a set A constituded by $n$ 2-dimensional points, how can I find a $\sqrt{2}$-approximation diameter of A, that is the maximum Euclidean distance between any two points in A, in linear time?&#xA;Can you help me with this? &lt;/p&gt;&#xA;" OwnerUserId="26522" LastEditorUserId="26522" LastEditDate="2018-01-13T14:40:59.187" LastActivityDate="2018-01-13T16:59:06.970" Title="Approximation diameter 2-dimensional space" Tags="|algorithms|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28584" PostTypeId="1" AcceptedAnswerId="28591" CreationDate="2018-01-10T10:57:40.757" Score="7" ViewCount="722" Body="&lt;p&gt;I trying to wrap my head of derivation of the analytic FEM Jacobian&#xA;for the Newton method. Say we have a nonlinear Poisson problem of the&#xA;(weak) form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int a(u)\nabla\ u\cdot \nabla v = \int f v $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a(u)$ is a coefficient. From the derivation of the Newton&#xA;method the Jacobian matrix (in the Newton update $\Delta u =&#xA;-J(u_k)R(u_k) $ will be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ J(u) = \int a(u)\nabla\ \phi_i\cdot \nabla \phi_j + \int \frac{\partial a(u)}{\partial u}\phi_i\nabla\ u\cdot \nabla \phi_j $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which essentially is the original stiffness matrix plus a contribution&#xA;due to the derivative of $a(u)$. Now if for example $a(u)=u^2$ then&#xA;the 2nd Jacobian term will be $J_2=\int 2u\ \phi_i\nabla\ u\cdot \nabla&#xA;\phi_j$ which is ok. But if now $a(u)=(\frac{\partial u}{\partial&#xA;x})^2$ or even $a(u)=\frac{\partial u}{\partial x}\frac{\partial&#xA;u}{\partial y}$, how does this work as the problem is still nonlinear&#xA;but $\frac{\partial a(u)}{\partial u}$ is essentially zero?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Could someone verify that these Jacobian forms are&#xA;indeed correct so that I have understood it correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ a(u) = u\frac{\partial u}{\partial x} \Rightarrow J_2 = \int&#xA;(u\frac{\partial \phi_i}{\partial x} + \frac{\partial u}{\partial&#xA;  x}\phi_i)(\nabla\ u\cdot \nabla \phi_j) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ a(u) = (\frac{\partial u}{\partial x})^2 \Rightarrow J_2 = \int&#xA;2\frac{\partial u}{\partial x}\frac{\partial \phi_i}{\partial&#xA;  x}(\nabla\ u\cdot \nabla \phi_j) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ a(u) = \frac{\partial u}{\partial x}\frac{\partial u}{\partial y} \Rightarrow J_2 = \int&#xA;(\frac{\partial u}{\partial x}\frac{\partial \phi_i}{\partial y} + \frac{\partial u}{\partial&#xA;  y}\frac{\partial \phi_i}{\partial x})(\nabla\ u\cdot \nabla \phi_j) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ and $\nabla u$ are evaluated explicitly with the current&#xA;solution in the assembly.&lt;/p&gt;&#xA;" OwnerUserId="26523" LastEditorUserId="20688" LastEditDate="2019-06-22T20:37:13.933" LastActivityDate="2019-06-22T20:37:13.933" Title="How to calculate/derive analytic FEM Newton Jacobian" Tags="|finite-element|newton-method|jacobian|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28585" PostTypeId="1" CreationDate="2018-01-10T11:11:10.333" Score="1" ViewCount="338" Body="&lt;p&gt;I've been trying to design an algorithm for computing determinant of Big square Matrices (N &amp;lt;= 1000). I am allowed to use multithreading, but still, I can't design the algorithm. The assignment states, that I should use Gaussian elimination method. Yet, there are no input data, so the testing data is on me. For big matrices (N ~1000) it's a binary matrix, otherwise - matrix of integers. I wonder, if there exists a relatively time efficient ( less than 10 seconds ) multithread algorithm for this task. Would be glad to hear opinions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:: I came up with idea of using partial pivoting to find a row with maximum at current iterating position, swapping it with current row (such heuristic keeps precision the best). And then, the parallel part, is to do row adding in multithread ( I used 8 threads, since it gave the best results on quad-core) if the matrix is big enough. &#xA;Such method gave more precise results, and efficiency raised with matrix size relatively to one thread method.&#xA;It actually turned out to be quite easy to implement.&lt;/p&gt;&#xA;" OwnerUserId="26518" LastEditorUserId="26518" LastEditDate="2018-01-30T11:23:46.720" LastActivityDate="2018-01-30T11:23:46.720" Title="Multithreading determinant computation of a Big Matrix" Tags="|linear-algebra|matrix|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="28587" PostTypeId="1" CreationDate="2018-01-10T12:41:07.277" Score="1" ViewCount="78" Body="&lt;p&gt;I want to find a candidate vector &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min|(W - A_i * B_i)|$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ a_i &amp;gt; 0,\ A_i=\{a_0,...,a_i\},\ B_i=\{-1,0,1\}^i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$W = 0.6,\quad A_4 = [0.1, 0.2, 0.4, 0.7] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one of answer will be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$B_4 = [1, -1, 0, 1] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only get that by searching an entire tree (DFS, BFS).&#xA;I tried with a ternary tree, but 0 value in vector &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; shuffles the ascendant order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dynamic-programming doesn't seem to work, divide-conquered also look at the entire tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be an efficient approach to the problem?&lt;/p&gt;&#xA;" OwnerUserId="26524" LastEditorUserId="20688" LastEditDate="2019-05-19T06:39:28.450" LastActivityDate="2019-05-19T06:39:28.450" Title="Find a vector B that minimizes |W-A*B|" Tags="|optimization|algorithms|constrained-optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="28588" PostTypeId="1" AcceptedAnswerId="28589" CreationDate="2018-01-10T14:37:19.073" Score="3" ViewCount="1408" Body="&lt;p&gt;I am looking for reference on how to implement nonlinear boundary conditions. Specifically, I am interested in implementing a radiation boundary condition for heat transfer with the FEM:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-k \frac {\partial T} {\partial n}  =F (T^{4} - T_{\infty}^{4})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $T$ temparature, $F$ is some factor, $k$ the thermal conductivity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An ideal reference has the theory and a simple (1D/2D) example and possibly some code to try/verify. I have looked at Finite Element Analysis for Heat Transfer (Hou-Cheng Huang) but for the price the review was not favorable. Maybe some knows of some lecture nodes, a deal ii or fenics implementation I could look at?&lt;/p&gt;&#xA;" OwnerUserId="11893" LastActivityDate="2018-01-13T04:47:02.987" Title="Radiation boundary condition (heat transfer)" Tags="|finite-element|boundary-conditions|nonlinear-equations|reference-request|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28589" PostTypeId="2" ParentId="28588" CreationDate="2018-01-10T14:55:27.980" Score="4" Body="&lt;p&gt;This is not particularly difficult once you realize that the nonlinear boundary condition simply yields a nonlinear term in the weak formulation. Let's assume that you want to solve the steady state problem, i.e.,&#xA;$$&#xA;  -k \Delta T = f \qquad \text{in} \ \Omega\\&#xA;  -k \frac{\partial T}{\partial n} = F(T^4 - T_\infty^4)  \qquad \text{on} \ \partial\Omega.&#xA;$$&#xA;You convert this into the weak form by multiplying the equation by a test function and integrating by parts, which yields&#xA;$$&#xA;  (\nabla \varphi,k\nabla T)_\Omega &#xA;  -(\varphi,k\partial_n T)_{\partial\Omega} = (\varphi,f)_\Omega&#xA;$$&#xA;Now realize that you can substitute the boundary condition in the second term on the left to obtain&#xA;$$&#xA;  (\nabla \varphi,k\nabla T)_\Omega &#xA;  +(\varphi,F T^4)_{\partial\Omega} = (\varphi,f)_\Omega + (\varphi,F T_\infty^4)_{\partial\Omega}&#xA;$$&#xA;You then only have to implement a nonlinear solver for this problem -- but that works the same way as for all other nonlinear problems and you can find many references for this. (Among them are &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;my video lectures on nonlinear problems&lt;/a&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="348" LastEditDate="2018-01-13T04:47:02.987" LastActivityDate="2018-01-13T04:47:02.987" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28590" PostTypeId="2" ParentId="28584" CreationDate="2018-01-10T15:27:36.507" Score="3" Body="&lt;p&gt;You need to understand how to actually compute derivatives when you want to take the derivative with respect to a &lt;em&gt;function&lt;/em&gt;. I've recorded a lengthy example in lecture 31.55 here: &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, you need to consider the derivative as the limit of a variation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-01-10T15:27:36.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28591" PostTypeId="2" ParentId="28584" CreationDate="2018-01-10T17:09:42.227" Score="11" Body="&lt;p&gt;Your example is a pretty good indication that the two derivatives (with respect to $x$ and with respect to $u$) do not commute :) (In fact, they're very different beasts -- one is a Fréchet derivative, the other a weak derivative.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather, you should consider $a(u) = (\partial_x u)^2$ as the composition $a = f\circ g$, of two functions &#xA;$$ f:  v(x)\mapsto v(x)^2$$&#xA;and&#xA;$$ g:  v(x) \mapsto (\partial_x v)(x)$$&#xA;and apply the chain rule&#xA;$$ \partial_u a(u) = \partial_u f(g(u)) \circ \partial_u g(u).$$&#xA;Here, it is important to note that derivatives of such a mapping $a$ between functions are actually linear operators, not functions (and the function spaces are important, but I'll ignore that here) -- the derivative $\partial_u a(u)$ acts on a given function $h$ to show how the output $a(u)$  changes if the input $u$ is perturbed in the direction of $h$. When computing such derivatives, it is therefore very helpful to keep these directions around until the end!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $f$ (which is called a &lt;em&gt;superposition&lt;/em&gt; or &lt;em&gt;Nemytskii&lt;/em&gt; operator), one can under some conditions (again, ignored here) show that for any function $h$&#xA;$$\partial_u f(v) h = 2 v * h,$$&#xA;where $*$ denotes the pointwise multiplication. (Basically, the rule is if you have a mapping $v(x)\mapsto f(v(x))$ for some differentiable $f:\mathbb{R}\to\mathbb{R}$, then its derivative in $u$ -- &lt;em&gt;if&lt;/em&gt; it exists -- is given by the pointwise multiplication with $f'(u(x))$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $g$, you can use that the (weak) derivative is linear and so the derivative is the linear operator itself, i.e.,&#xA;$$\partial_u g(v)h = \partial_x h.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Together, you obtain&#xA;$$\partial_u a(u)h = 2\partial_x u * \partial_x h,$$&#xA;where $*$ denotes the pointwise multiplication. In your equation for $J(u)$, $\phi_i$ is taking the place of $h$, so the corresponding Jacobian term would be&#xA;$$ \int_\Omega 2\partial_x u(x) \partial_x \phi_i(x) (\nabla u(x) \cdot \nabla \phi_j(x))\,dx.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your second example, you proceed similar using the product rule instead of the chain rule to get&#xA;$$\partial a(u)h = \partial_x u * \partial_y h + \partial_y u * \partial_x h.$$&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2018-01-11T08:42:15.867" LastActivityDate="2018-01-11T08:42:15.867" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28593" PostTypeId="1" CreationDate="2018-01-11T12:40:17.993" Score="6" ViewCount="302" Body="&lt;p&gt;We know finite difference method (FDM) can replace $y''(x)$ as $\frac{1}{h^2}[y(x+h)+y(x-h)-2y(x)]$ or so. One naive way to write down the matrix of the differential operator is like the following, which somehow simply cuts off at the two edges. Similarly, we can write the matrix for $y'=\frac{1}{2h}[y(x+h)-y(x-h)]$ as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix}&#xA;2t_0 &amp;amp;-t_0 &amp;amp; &amp;amp; &amp;amp; \\&#xA;-t_0 &amp;amp;2t_0 &amp;amp;-t_0 &amp;amp; &amp;amp; \\&#xA;&amp;amp;\ddots  &amp;amp;\ddots &amp;amp;\ddots &amp;amp; \\&#xA;&amp;amp; &amp;amp; -t_0 &amp;amp;2t_0 &amp;amp;-t_0\\&#xA; &amp;amp; &amp;amp; &amp;amp; -t_0 &amp;amp;2t_0&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But what boundary condition (BC) correspond to this?&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first thought this means zero-value Dirichlet BC (like $y(a)=y(b)=0$). However, it turns out not to be the case. &lt;strong&gt;It &lt;em&gt;automatically&lt;/em&gt; handles either zero or nonzero boundary values&lt;/strong&gt; as shown in the solution below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on such naive FDM matrics, I solve the eigenvalue ($\lambda$) problem of this linear ODE $y''+\frac{2}{x}y'+[2\lambda-x^2-\frac{l(l+1)}{x^2}]y=0$ with nonnegative integer $l$.&lt;br&gt;&#xA;It can be analytically solved and we know at the boundary, $y(0)\neq0$ for $l=0$, $y(0)=0$ for $l&amp;gt;0$, and $y(\infty)=0$. Surely we use some large enough interval to mimic the infinity. The following code solves the two cases well. &lt;strong&gt;But again, why?&lt;/strong&gt; &lt;em&gt;Is this chop scheme generally correct for fixed (arbitrary) valued Dirichlet boundaries or anything else?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Following is my simple Mathematica code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;l = 0; a = 1; n = 1001; h = 16/(n - 1);&#xA;M1 = -2.0 IdentityMatrix[n] + DiagonalMatrix[Table[1, {n - 1}], 1] + &#xA;  DiagonalMatrix[Table[1, {n - 1}], -1];(*//MatrixForm*)&#xA;M2 = DiagonalMatrix[Table[-1/(i + 1), {i, 1, n - 1}], -1] + &#xA;  DiagonalMatrix[Table[1/i, {i, 1, n - 1}], 1];&#xA;M3 = DiagonalMatrix[Table[(i)^2, {i, 1, n}]];(*//MatrixForm*)&#xA;M4 = l (l + 1) DiagonalMatrix[Table[(i)^-2, {i, 1, n}]];&#xA;M = 1/h^2 M1 + h^-2 M2 - h^2 M3 - h^-2 M4;&#xA;qq = Eigenvectors[M];&#xA;ListPlot[qq[[n - a + 1]], PlotRange -&amp;gt; All]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;$a$ means the $a$th smallest eigenvalue and certainly one can change the value of $l$ and $a$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The exact solution looks like &#xA;$$&#xA;y_{0,l}(x)\propto x^l e^{-\frac{1}{2}x^2}\\&#xA;y_{1,l}(x)\propto x^l e^{-\frac{1}{2}x^2}(\frac{2l+3}{2}-x^2)\\&#xA;y_{2,l}(x)\propto x^l e^{-\frac{1}{2}x^2}(\cdots)\\&#xA;\cdots&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="9408" LastEditorUserId="9408" LastEditDate="2018-01-16T08:15:23.640" LastActivityDate="2018-01-28T19:54:22.030" Title="Why naively chopped finite difference matrix works for different ODE boundary conditions" Tags="|pde|finite-difference|ode|boundary-conditions|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28594" PostTypeId="2" ParentId="28593" CreationDate="2018-01-11T13:30:07.017" Score="4" Body="&lt;p&gt;If you convert the first line of your matrix operator back into a difference equation you get $$2y(x)-y(x+h)=h^2b,$$&#xA;where $b$ is whatever is on the right hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is consistent with a non-centred finite difference approximation to &#xA;$$ y - h y' = h^2 b.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an example of &lt;a href=&quot;https://en.wikipedia.org/wiki/Robin_boundary_condition&quot; rel=&quot;nofollow noreferrer&quot;&gt;a Robin, or third type boundary condition&lt;/a&gt;, although a slightly unusual one, since it contains your mesh length inside it. Note that in the limit of $h\rightarrow 0$, this does tend towards a zero Dirichlet condition, so you would expect your scheme to tend towards that solution as your mesh converges, providing the gradient of your solution at the boundary stays well behaved.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2018-01-11T13:30:07.017" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28595" PostTypeId="1" CreationDate="2018-01-11T19:10:45.837" Score="1" ViewCount="1088" Body="&lt;p&gt;I am computing the stationary, incompressible, inviscid and irrotational flow around a circular cylinder using a discretization in general coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I derived a PDE and proper boundary conditions in the physical domain (left image) given by $G := \{(x_1,x_2 : R_1^2 \leq x_1^2 + x_2^2 \leq R_2^2, x_2 \geq 0\}$. This PDE is $\Delta \phi = 0$, where $\phi$ is the perturbation potential.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vCaEY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vCaEY.png&quot; alt=&quot;Physical and logical domain&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the assignment asks to simplify the Laplacian in general coordinates. For this is calculate the covariant/contravariant basis vectors and the Jacobian. Next I define the mapping $\mathbf{x}(\xi)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am stuck on deriving proper boundary conditions in the logical domain (right image). These have to be defined in terms of the covariant/contravariant basis vectors and the metric tensor. As a hint, it is given: &lt;em&gt;Do not use the specific mapping at this stage. Use the fact that the boundaries of the domain are of the form $\xi^\alpha(\mathbf{x}) = $ constant, and therefore the unit normal vector on the boundary is given by $\pm \mathbf{a}^{(\alpha)}/|\mathbf{a}^{(\alpha)}|$&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't see how to use the covariant/contravariant basis vectors. We have a new basis for the physical domain G for a polar coordinate system, so what do they have to do with the computational domain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary conditions in invariant form:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Along the cylinder(flow is inviscid) $0 = \mathbf{u}\cdot\mathbf{n} = -U_\infty n_1$, where $\mathbf{u}$ is the velocity field given by $u_\alpha = \frac{\partial \Phi}{\partial x_\alpha}$ and $\mathbf{\Phi}$ the total velocity potential function given by $\Phi = U_\infty x_1 + \phi$. Now $U_\infty$ is the free stream velocity in the $x_1$ direction and $\phi$ is the perturbation potential.&lt;/li&gt;&#xA;&lt;li&gt;On the horizontal boundary of the physical domain we have $\frac{\partial u}{\partial x_2} = 0$, so $\frac{\partial^2 \phi}{\partial x_1x_2} = 0$ and $\frac{\partial^2 \phi}{\partial x_2^2} = 0$.&lt;/li&gt;&#xA;&lt;li&gt;Along the top of the domain we have a homogeneous Dirichlet boundary condition for the perturbation potential. $\phi(\mathbf{x}) = 0$ for $\mathbf{x}$ on this boundary.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I left out a lot of specifics, because I'm not sure if these are needed. &lt;/p&gt;&#xA;" OwnerUserId="26537" LastEditorUserId="26537" LastEditDate="2018-01-12T18:19:18.803" LastActivityDate="2019-05-08T02:02:50.687" Title="Proper boundary conditions for potential flow around cylinder" Tags="|pde|numerical-analysis|fluid-dynamics|boundary-conditions|computational-physics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28596" PostTypeId="2" ParentId="27614" CreationDate="2018-01-11T20:01:36.397" Score="1" Body="&lt;p&gt;Initialize the cluster centers as your subset $V\in X$, where $V=\{x_i\}$. Then run a couple of K-mediods iterations. After that you will see that the certain vectors will come closer, essentially trying to represent similar peaks. It is then possible to merge them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way to do this is &lt;a href=&quot;https://en.wikipedia.org/wiki/Mean_shift&quot; rel=&quot;nofollow noreferrer&quot;&gt;mean-shift algorithm&lt;/a&gt;, where the modes are being updated (I think you want to find the modes). You start by over-clustering and at a dynamic number of clusters (discovered) that best represents the modes. For discrete data such as yours &lt;a href=&quot;https://www.cs.cmu.edu/~yaser/ModeSeekingByMedoidShifts_SheikhKhanKanade2007.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the medioids variation&lt;/a&gt; does exist.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2018-01-11T20:01:36.397" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28597" PostTypeId="2" ParentId="5314" CreationDate="2018-01-11T21:22:55.133" Score="0" Body="&lt;p&gt;If $N$ is even (which we typically use for the fft to be fast), why set $N/2$ mode  to zero? Well if the signal is real  $x \in \mathbb R$,   taking the inverse fourier transform, the mode  $N/2$ may produce an imaginary part since there is no conjugate  $-N/2$ component to cancel    the   $N/2$ mode contribution, so we set it to zero.&lt;/p&gt;&#xA;" OwnerUserId="22299" LastActivityDate="2018-01-11T21:22:55.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28598" PostTypeId="2" ParentId="28583" CreationDate="2018-01-11T21:46:28.367" Score="1" Body="&lt;p&gt;Due to admitting the $\sqrt{2}$-approximate, you could probably live with an $L^2_1$ (L1-metric in 2 dimensions) approximation of the problem, which upper bounds the Euclidean distance. Then, I would point you to an approach based on metric embedding following &lt;a href=&quot;http://www.cs.toronto.edu/~avner/teaching/S6-2414/LN2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bourgain’s theorem&lt;/a&gt;. The idea is based on transforming the problem onto a different domain using an isometric embedding, where it is much simpler to solve the problem. It is remarkable how easy one could formulate solutions to certain problems this way, such as maximum Manhattan distance between two points ($L_1$-diameter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us begin by embedding the space into a space of dimension $2^d$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider $x\in L^d_1$, where $d=2$ for 2D case and $\|x\|_1=\sum_i |x_i|=\sum_i sgn(x_i)\cdot x_i \geq \sum_i y_i \cdot x_i$ for all $y\in \{-1,+1\}^d$. Therefore:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\|x\|_1 = max\{ \langle x,y \rangle \}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is obvious that the suggested isometric mapping is $f_y:X \rightarrow \mathbb{R}$ for each vector $y \in \{-1,1\}^2$ with $f_y(z)=\{ \langle z,y \rangle \}$. In the final map, all coordinates are concatenated together resulting in $2^d$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then solve the transformed problem in which every coordinate can be iteratively max-ed. See &lt;a href=&quot;http://ttic.uchicago.edu/~harry/teaching/pdf/lecture1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;section 2.1.1 in here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;\max_{u,v,\in S} (u-v) &amp;amp;= \max_{u,v,\in S} \max_{i=1}^{2^d} |u_i-v_i|\\&#xA;&amp;amp;= \max_{i=1}^{2^d} \text{furthest point along $\text{i}^{\text{th}}$ coordinate}&#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result, the complexity is $O(2^d N)$ with constant $d$, which is linear in $N$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a simple pseudocode (thanks to &lt;a href=&quot;http://perso.telecom-paristech.fr/~simsekli/&quot; rel=&quot;nofollow noreferrer&quot;&gt;umut's implementation&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d = 2;&#xA;n = 1000;&#xA;X = 10*rand(d,n);&#xA;L = zeros(d,2^d);&#xA;for i = 1:size(L,2)&#xA;    L(:,i) = bitget((i-1),d:-1:1);    &#xA;end&#xA;L(L ==0) = -1;&#xA;Y = X'*L;&#xA;diam = max(max(Y)-min(Y))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that you could precompute &lt;code&gt;L&lt;/code&gt; and store for future use. For $n=1000$, this just takes 3.5ms in MATLAB whereas a naive computation would easily exceed half a second. &lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2018-01-13T16:59:06.970" LastActivityDate="2018-01-13T16:59:06.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28599" PostTypeId="2" ParentId="28595" CreationDate="2018-01-12T07:31:22.937" Score="1" Body="&lt;p&gt;Be careful with the Neumann BC. $\partial u/\partial x=0$ does no imply that $\partial^2 u/\partial x^2=0$, e.g. $u=x^2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have the BC defined in the physical domain... and you have an expression that relates physical coordinates $x$ with computational coordinates $ξ$ in the computational domain $x=x(ξ)$, or vice-versa.&#xA;Substitute these in your BC and apply the chain rule. If the map is well defined, you will not have any problem computing its inverse.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-01-12T08:05:19.600" LastActivityDate="2018-01-12T08:05:19.600" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28601" PostTypeId="1" AcceptedAnswerId="28604" CreationDate="2018-01-12T20:27:32.747" Score="0" ViewCount="104" Body="&lt;p&gt;For the linear system $\mathbf A \mathbf x = \mathbf b$ generated from 2D Poisson equation using the standard central finite difference method,&lt;br&gt;&#xA;$$&#xA;\mathbf A =&#xA;\begin{bmatrix}&#xA;\mathbf K &amp;amp; -\mathbf I \\&#xA;-\mathbf I &amp;amp; \mathbf K &amp;amp; -\mathbf I \\&#xA;&amp;amp; -\mathbf I &amp;amp; \mathbf K &amp;amp; -\mathbf I \\&#xA;&amp;amp; &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots&#xA;\end{bmatrix}&#xA;$$&#xA;where $\mathbf I$ is the identity matrix and $\mathbf K$ is the tridiagonal matrix with stencil $[-1 \  4 \ -1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Matlab backslash, does anyone know what reordering algorithm matlab will use to solve this sparse system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in general, how matlab decide wihch reordering algorithm to use?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="18224" LastEditDate="2018-01-12T20:33:07.320" LastActivityDate="2018-01-13T02:39:00.670" Title="Matlab backslash reordering algorithm" Tags="|linear-algebra|matlab|linear-solver|sparse-matrix|banded-matrix|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28602" PostTypeId="1" AcceptedAnswerId="28611" CreationDate="2018-01-12T22:32:51.293" Score="1" ViewCount="143" Body="&lt;p&gt;The general solution to the 2nd order wave equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2u}{dt^2} = c^2\frac{d^2u}{dx^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is known as d'alembert's formula-&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/D%27Alembert%27s_formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/D%27Alembert%27s_formula&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does an analytical particular solution exist for a problem with a Neumann BC on one end, and dirichlet on the other? I am unable to find a solution. I have only been able to find solutions for pure dirichlet BCs.&lt;/p&gt;&#xA;" OwnerUserId="23915" LastActivityDate="2018-01-14T13:02:49.383" Title="Is the 1D wave equation analytically solvable for a Neumann BC?" Tags="|pde|" AnswerCount="1" CommentCount="8" ClosedDate="2018-01-14T17:25:01.550" ContentLicense="CC BY-SA 3.0" />
  <row Id="28604" PostTypeId="2" ParentId="28601" CreationDate="2018-01-13T02:39:00.670" Score="2" Body="&lt;p&gt;@gohokies has already given the correct answer in a comment, but just for more context: Matlab backslash calls the UMFPACK (now SuiteSparse) solver for sparse linear systems. The default ordering used by UMFPACK is indeed the Approximate Minimum Degree (AMD) method, or a variation thereof.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-01-13T02:39:00.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28605" PostTypeId="1" AcceptedAnswerId="28657" CreationDate="2018-01-13T14:41:38.993" Score="0" ViewCount="65" Body="&lt;p&gt;Time is continuous time. I have a 3D state space, and transition rates across all of these. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the transition rates, I can compute a &lt;em&gt;generator&lt;/em&gt; matrix &lt;code&gt;A&lt;/code&gt; associated with the markov process. My &lt;code&gt;A&lt;/code&gt; satisfies all requirements on a  generator matrix:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;A_ii &amp;lt; 0&lt;/code&gt; and  &lt;code&gt;A_ij &amp;gt;=0&lt;/code&gt; for all i and j&lt;/li&gt;&#xA;&lt;li&gt;Also, rows sum up to zero. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I want to compute the implied stationary distribution &lt;code&gt;g&lt;/code&gt; given by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dot(A.T,  g) = z&#xA;sum(g) = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;z&lt;/code&gt; is a zero vector. To ensure the second condition, I set &lt;code&gt;z_i = 1&lt;/code&gt; for some &lt;code&gt;i&lt;/code&gt;, &lt;code&gt;A_ii = 1&lt;/code&gt;, &lt;code&gt;A_ij = 0&lt;/code&gt; for all &lt;code&gt;j != i&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my problem is that I don't always get to invert &lt;code&gt;A&lt;/code&gt;, despite it satisfying all the conditions for a generator matrix. Python just gives me&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scipy.sparse.linalg.dsolve.linsolve.MatrixRankWarning: Matrix is exactly singular&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I have a suspect: My grid size. Let's call one of the three dimensions of the state space &lt;code&gt;u&lt;/code&gt;. For a fixed &lt;code&gt;u.min()&lt;/code&gt;, &lt;code&gt;u.max()&lt;/code&gt;, if I increase the grid size enough, I can usually invert &lt;code&gt;A&lt;/code&gt;. For the order of magnitudes: If the transition rate &lt;em&gt;across grid points&lt;/em&gt; is around 100, inverting &lt;code&gt;A&lt;/code&gt; fails. When it's only around &lt;code&gt;10&lt;/code&gt;, it works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I can't always throw more grid points at the problem. Also, I'm not sure of the fundamental problem behind why the grid size matters where. Could anyone shed some light on what's happening here, and whether there's an alternative way forward?&lt;/p&gt;&#xA;" OwnerUserId="9694" LastActivityDate="2018-01-20T16:19:10.847" Title="Inverting a transition matrix with small grid size" Tags="|linear-algebra|python|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28606" PostTypeId="1" CreationDate="2018-01-13T16:19:04.063" Score="1" ViewCount="66" Body="&lt;p&gt;$$&#xA;\text{Let}\\&#xA;A, B \in  \mathbb{C}^{n \times n} \text{ and } \hat{\alpha}, \hat{\beta} \in \mathbb{C}^{n}, \hat{f} \in \mathbb{C}^{2n}&#xA;\\&#xA;\text{Find }\\&#xA;\underline{\mathbf{x}} \in \mathbb{C}^{2n} \text{ such that}\\&#xA; \begin{bmatrix}&#xA;A &amp;amp; -B\\&#xA;\hat{A} &amp;amp; \hat{B}&#xA;\end{bmatrix}&#xA;\cdot&#xA;\underline{\mathbf{x}}&#xA;= \hat{f}\\&#xA;\hat A = \text{diag}(\hat\alpha) ,\quad \hat B = \text{diag}(\hat\beta)&#xA;$$&#xA;For all rows, only one of $\hat{\alpha}$ and $\hat{\beta}$ in this row can be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically we are dealing with a block matrix, where the upper row consists of two fully populated, dense, complex square matrices and the lower row consists of two diagonal complex matrices.&#xA;I need to solve this kind of equation system, where $n$ can be anywhere between $\approx3000$ and $\approx 20000$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thought about using an augmented GMRES (adapted multiplication for the lower half of the block matrix), but I think I would need a solid preconditioner, because the system might be poorly conditioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any direct algorithms that would profit from the fact that the lower half of the block matrix consists of two diagonal matrices?&lt;/p&gt;&#xA;" OwnerUserId="26557" LastEditorUserId="26557" LastEditDate="2018-01-14T16:53:17.583" LastActivityDate="2018-01-15T15:43:11.407" Title="Optimal algorithm choice for mixed diagonal/dense problem" Tags="|matrix|linear-solver|iterative-method|dense-matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28608" PostTypeId="1" CreationDate="2018-01-13T23:44:28.730" Score="1" ViewCount="56" Body="&lt;p&gt;The coupled mode space NEGF method for computing transistor characteristics involves expanding the electronic wavefunction in a mode space basis&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Psi(x,y,z) = \sum_n\phi_n(x)\xi_n(y,z;x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\xi_n(y,z;x)$ is the nth mode at position x (the x dimension being the direction of transport).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method involves computing coupling between modes along the transport direction. E.g. coupling coefficients like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$b_{mn}(x) = \langle\xi_m|\frac{\partial}{\partial x}|\xi_n\rangle$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numerical algorithm will divide the transistor into $X$ slices $\Delta x$ apart, and compute the modes on each slice. The issue is my numerical solver will sometimes compute $\xi_n$ as the nth eigenvalue, and sometimes compute $-\xi_n$, so phases might not be consistent across each slice. This changing phase will result in a large $\frac{\partial}{\partial x}|\xi_n\rangle$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Do the phases need to be consistent across each slice? If they do, how do NEGF algorithms ensure they are consistent? &lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2018-01-13T23:44:28.730" Title="When numerically computing eigenstates during a coupled-mode-space NEGF calculation, do phases matter?" Tags="|eigensystem|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28609" PostTypeId="1" CreationDate="2018-01-14T01:32:37.100" Score="1" ViewCount="63" Body="&lt;p&gt;I use arpack to solve the 2D Schrodinger, and eigenvalue problem of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Hx = \epsilon x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on a uniform grid. All eigenvectors are real in my case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Arpack doesn't normalise the eigenvectors, and phases between the eigenvectors will differ (some are multiplied by -1, some aren't).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normalising the eigenvectors after the arpack run is easy, but I am not sure how to ensure all eigenvectors have the same phase. Is there some way to compare the phase between two orthogonal eigenvectors (say, by checking the sign near a boundary)?&lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2018-01-14T01:32:37.100" Title="Solving 2D Schrodinger Equation with ARPACK: Can I ensure all eigenvectors have the same phase?" Tags="|eigensystem|arpack|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28610" PostTypeId="1" CreationDate="2018-01-14T12:19:07.803" Score="1" ViewCount="117" Body="&lt;p&gt;I'm trying to build a numerical solution for a parameter estimation problem of reaction-diffusion equation, using the adjoint method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize it, I'm trying to minimize the function &#xA;$$ G=\frac{1}{2}\int_\Omega{(u(T,x)-u_f)^2d\Omega} + \frac{1}{2}\int_\Omega{D^2d\Omega}+ \frac{1}{2}\int_\Omega{k^2d\Omega} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;F = \partial_t u - \nabla(D\nabla u) - ku(1-u) = 0 \\&#xA;u(0,x) = u_0 \\&#xA;u(t,x) = 0 ~\forall x\in\Gamma :=\partial\Omega&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is the tumor density function, $\Omega$ is the space domain of the problem and $D$, $k$ are the unknown diffusion and proliferation terms (to optimize in order to find the minimal $G$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank to &lt;a href=&quot;https://scicomp.stackexchange.com/q/28468/25963&quot;&gt;GoHookie's help&lt;/a&gt;, I have now a proper derivation of the Lagrangian&#xA;$$ L = G - \lambda F $$ giving me an expression of the total derivative of $G$ w.r.t the unknown parameters $D$ and $k$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial L}{\partial D}(\delta D) = \int_\Omega\int_t^T{(\frac{D}{T}-\nabla \lambda\nabla u)\delta Dd\Omega dt} \\&#xA;\frac{\partial L}{\partial k}(\delta k) = \int_\Omega\int_t^T{(\frac{k}{T}-\lambda u(1-u))\delta k d\Omega dt}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the problem is non-linear, I have to use an iterative optimization methods, which requires me also to provide an expression of the Hessian of $G$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using again an adjoint method, leads to calculate the various second derivatives of $L$ w.r.t $u, D$ and $k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I can manage it except for the case of&#xA;$$ \frac{\partial^2 L}{\partial u\partial D}(\delta u,\delta D) = -\int_\Omega\int_t^T{(\nabla \lambda\nabla \delta u)\delta Dd\Omega dt} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;since then, I think I need to factorize this expression both for $\delta u$ and $\delta D$ (i.e. transform the expression in something like $\int_\Omega\int_0^T{\left( J(u,\lambda,\nabla u,\nabla \lambda)\delta u\delta \lambda\right) d\Omega dt}$ ) but none of the divergence theorem, Green theorem or integration by part methods seems to help me doing this factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thx&lt;/p&gt;&#xA;" OwnerUserId="25963" LastEditorUserId="25963" LastEditDate="2018-01-14T19:14:31.183" LastActivityDate="2018-01-14T19:14:31.183" Title="functional second derivative" Tags="|constrained-optimization|variational-calculus|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="28611" PostTypeId="2" ParentId="28602" CreationDate="2018-01-14T13:02:49.383" Score="3" Body="&lt;p&gt;The answer is yes. There are lots of ways you can analytically solve this equation.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, you can solve this problem by &lt;a href=&quot;https://en.wikipedia.org/wiki/Discrete_Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Discrete Fourier Transform.&lt;/a&gt; (DFT)&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For Dirichlet BC on both sides: we use {$sin\frac{n\pi x}{L}$} as eigenfunction&lt;br&gt;&#xA;  For Neumann BC: we use {$cos\frac{n\pi x}{L}$}  as eigenfunction&lt;br&gt;&#xA;  And when we have for one side Dirichlet and for the other Neumann we may use either {$sin\frac{(n+\frac{1}{2})\pi x}{L}$} or {$cos\frac{(n+\frac{1}{2})\pi x}{L}$} depending on the side of each BC's type, as eigenfunction.&lt;br&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For the problem you asked, it can be solved with the third type eigenfunction.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may derive these eigenfunctions from Sturm–Liouville theory.&lt;br&gt;&#xA;Please let me know if this was the thing you were looking at.&lt;/p&gt;&#xA;" OwnerUserId="26565" LastActivityDate="2018-01-14T13:02:49.383" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28612" PostTypeId="1" AcceptedAnswerId="28614" CreationDate="2018-01-14T16:19:08.697" Score="3" ViewCount="105" Body="&lt;p&gt;The following problem is given:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ - \bigtriangleup u = f, \quad \partial_n u = g \quad \text{on } \Gamma_N, \quad u = 0 \quad \text{on } \Gamma_D $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $\Gamma_N$ or $\Gamma_D$ denoting the Neumann/Dirichlet boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently having a look at a residual error estimator using the Clément-Interpolant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was a theorem stating that the error &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \| \nabla (u -u_N) \|_{L^2(\Omega)} \leq C \, \text{EST} = C \, \sqrt{\sum_{K \in T} \eta_K^2},$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\text{EST}$ is an expression that can be calculated with known functions (only dependent on $u_N$, $f$ and $g$). The $\eta_K$ are therefore elementwise indicators for the refinement stategy for each element $K$ in the triangulation $T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next theorem tries to give estimates of $\text{EST}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \text{EST}^2 \leq C \left(\| \nabla (u -u_N) \|^2_{L^2(\Omega)} + \sum_{K \in T} h_K^2 \| f - f_T \|^2_{L^2(K)} + \sum_{e \in E_N }h_e \| g - g_T \|^2_{L^2(e)}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $f_T$ the averaging over the triangle $K$, namely $f_T = \frac{1}{|K|} \, \int_{|K|} f$, and $g_T$ the averaging over the edge $e$. $E_N$ are the Neumann edges. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Afterwards, there is a small remark saying that we can hope that the errors of the $\text{EST}^2$-estimate behave approximately like the first term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The justification of that should be the following: Given a uniform grid, and $f$, $g$ piecewise smooth (like: $f|_K \in H^1(K)$ for all $K$ and &#xA;$\sum_K \| f \|_{H^1(K)}$ and same for $g$ on the Neumann edge). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Then the terms with $\| f - f_T \|^2_{L^2(K)}$ and $\| g - g_T \|^2_{L^2(e)}$ in the estimate should&#xA;  be of the size $\mathcal{O}({h^3})$ but for the exact solution $u \in H^2$, we expect $\mathcal{O}({h^2})$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can you help me to understand, why we are expecting these orders stated in the block?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $\mathcal{O}({h^2})$ could be due to Poincaré-inequality and changes to the reference element. But I cannot really see how it works.&lt;/p&gt;&#xA;" OwnerUserId="22586" LastEditorUserId="22586" LastEditDate="2018-01-14T17:34:02.393" LastActivityDate="2018-01-15T00:51:42.610" Title="FEM problem: how to get a feeling for size of problem" Tags="|finite-element|pde|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28614" PostTypeId="2" ParentId="28612" CreationDate="2018-01-15T00:51:42.610" Score="3" Body="&lt;p&gt;If $f_T$ is the piecewise constant cellwise mean, then indeed $\|f-f_T\|_{L_2} \propto {\cal O}(h)$. Consequently, the term&#xA;$$&#xA;  \sum_K h_K^2 \|f-f_T\|^2_{L_2(K)} \le h^2 \|f-f_T\|_{L_2}^2&#xA;  \le C h^4,&#xA;$$&#xA;which is of higher order than the error term&#xA;$$&#xA;  \|\nabla (u-u_N)\|_{L_2}^2 \le C h^2.&#xA;$$&#xA;In other words, the &quot;data oscillation term&quot; can be neglected asymptotically -- though it may be large on a given mesh if the mesh is coarse enough.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-01-15T00:51:42.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28616" PostTypeId="1" CreationDate="2018-01-15T11:20:23.747" Score="4" ViewCount="71" Body="&lt;p&gt;I am working on developing my own solver(for implementation on hardware), based on IPM for following problem:&#xA;\begin{equation}&#xA;\begin{split}&#xA;\min_{X} \;  \frac{1}{2}&amp;amp;\|X\|_F^2 + trace(CX)\\&#xA;\text{s.t. } &amp;amp; trace(AX)\leq b\\&#xA;&amp;amp; X\succeq 0\\&#xA;\end{split}&#xA;\end{equation} &#xA;As a reference I already know how to obtain KKT equations for &#xA;\begin{equation}&#xA;\begin{split}&#xA;\min_{X} \;  \frac{1}{2}&amp;amp;\|X\|_F^2 + trace(CX)\\&#xA;\text{s.t. } &amp;amp; trace(AX) = b\\&#xA;&amp;amp; X\succeq 0\\&#xA;\end{split}&#xA;\end{equation} &#xA;Dual of this problem will be&#xA;\begin{equation}&#xA;\begin{split}&#xA;\max_{X,y,S}\;&amp;amp; -\frac{1}{2}\|X\|_F^2+by+\beta\log\det(S)\\&#xA;\text{s.t. }\;&amp;amp; A^Ty-X+S=C,\; S\succeq 0&#xA;\end{split}&#xA;\end{equation}&#xA;KKT system will be &#xA;\begin{equation}&#xA;\begin{array}{ccccccc}&#xA;-X &amp;amp; + &amp;amp; A^Ty &amp;amp; + &amp;amp; S &amp;amp; = C, &amp;amp; S\succeq 0\\&#xA;trace(AX) &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp; = b, &amp;amp; X\succeq 0\\&#xA;&amp;amp;  &amp;amp; XS  &amp;amp;  &amp;amp;  &amp;amp; = 0, &amp;amp;\\&#xA;\end{array}&#xA;\end{equation}&#xA;Later I sove it with Newton-Rhapson method, have no problems there.&#xA;My question, therefore, be, how to obtain KKT system, when $trace(AX) \leq b$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All responses or references are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20930" LastActivityDate="2018-01-15T11:20:23.747" Title="Obtainting KKT for QSDP for the trace inequality constraint" Tags="|convex-optimization|constrained-optimization|semidefinite-programming|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28617" PostTypeId="1" AcceptedAnswerId="28619" CreationDate="2018-01-15T11:30:09.463" Score="5" ViewCount="3380" Body="&lt;p&gt;I am currently using the following three methods to solve differential equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4th order Runge Kutta Method&#xA;Euler Method&#xA;Internal scipy methods:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scipy.integrate.odepack&#xA;#odeint&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am aware that eulers method is first order, with a total error after N steps being O(h) (I think).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And that RK4 is 4th order, hence its name i.e. has error of: O(h^4), however, I am unsure of this, as I have also been told that when using this method there is no way to estimate errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, when considering the internal 'integrator' I was unaware of the method that this used, I have found the following site, which explains the algorithms behind them, but it doesn't explain the errors, hence why I am asking on here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://computation.llnl.gov/casc/odepack/&quot; rel=&quot;noreferrer&quot;&gt;https://computation.llnl.gov/casc/odepack/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If someone could inform me of the error of ODEint (in big O notation), and verify that I am correct about the errors on the other two methods, that'd be great.&lt;/p&gt;&#xA;" OwnerUserId="26571" LastEditorUserId="9667" LastEditDate="2018-01-15T14:35:18.083" LastActivityDate="2020-07-19T17:52:01.640" Title="Comparing Algorithmic complexity, ODE Solvers (Big O)" Tags="|python|ode|complexity|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28618" PostTypeId="2" ParentId="28606" CreationDate="2018-01-15T13:47:00.270" Score="1" Body="&lt;p&gt;This is what I wrote in the comments, formulated as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\hat{\beta}_j=0$, then $\hat{\alpha}_j\neq 0$ otherwise the matrix wouldn't be invertible. So you can swap column $j$ with column $n+j$ to make sure $\hat{\beta}_j \neq 0$ (or better, as @wim suggests in the comments, $|\hat{\beta}_j| \geq |\hat{\alpha}_j|$) (and you need to swap $x_j$ and $x_{n+j}$ conformably to get an equivalent system). You can do this for all $j=1,2,\dots,n$, hence ensuring that $\hat{B}$ is nonsingular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can solve the system using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Schur_complement#Application_to_solving_linear_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schur complement&lt;/a&gt; of $\hat{B}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cost of this direct algorithm is essentially that of forming the Schur complement ($O(n^2)$ flops, since two matrices are diagonal, as correctly noted by @wim) plus that of solving the linear system with it ($\frac23 n^3$ flops).&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2018-01-15T15:43:11.407" LastActivityDate="2018-01-15T15:43:11.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28619" PostTypeId="2" ParentId="28617" CreationDate="2018-01-15T15:26:17.720" Score="13" Body="&lt;p&gt;&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;odeint&lt;/code&gt; from the SciPy library&lt;/a&gt; defaults to the &lt;code&gt;lsoda&lt;/code&gt; integrator &lt;a href=&quot;http://www.oecd-nea.org/tools/abstract/detail/uscd1227&quot; rel=&quot;nofollow noreferrer&quot;&gt;described here&lt;/a&gt;. However, any simple description of asymptotic computation time is impossible. The reason is many fold.&lt;/p&gt;&#xA;&lt;p&gt;First, let me describe the algorithm. A common multistep algorithm for non-stiff equations are the &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_multistep_method#Adams%E2%80%93Moulton_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adams-Moulton methods&lt;/a&gt;. While these are implicit, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Linear_multistep_method#Adams%E2%80%93Bashforth_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adams-Bashforth methods&lt;/a&gt; are generally too unstable to be useful. To solve the implicit equation, a simple Picard iteration (or if you want to get fancy and make it a little better, &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/10078356X&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anderson acceleration&lt;/a&gt;, but LSODA uses Picard) since it's cheap. Using Picard iteration here decreases the stability region (as discussed in &lt;a href=&quot;http://www.springer.com/us/book/9783540604525&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer II&lt;/a&gt;), but this is considered okay since the algorithm is for non-stiff problems. This is the Adams integrator. &lt;a href=&quot;http://www.stochasticlifestyle.com/comparison-differential-equation-solver-suites-matlab-r-julia-python-c-fortran/&quot; rel=&quot;nofollow noreferrer&quot;&gt;As I describe in a blog post comparing the different ODE suites developed over the years&lt;/a&gt;, pretty much all of the multistep methods (except a few from Shampine) are based on the same implementation starting from GEAR, so &lt;code&gt;lsode&lt;/code&gt;, &lt;code&gt;lsoda&lt;/code&gt;, &lt;code&gt;vode&lt;/code&gt;, &lt;code&gt;CVode&lt;/code&gt; (SUNDIALS) all have pretty much the same Adams implementation for non-stiff problems.&lt;/p&gt;&#xA;&lt;p&gt;For stiff problems, the go-to multistep method is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Backward_differentiation_formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;Backwards Differentiation Formula (BDF) method&lt;/a&gt;. The Wiki page is quite good and shows how the stability gets bad &amp;gt;5, so generally no one uses the 6th order. Every step these have to solve an implicit equation and these use quasi-Newton methods to do so because this does not increase the theoretical stability as much as fixed-point iteration. However, to keep the costs down, it's quasi-Newton instead of Newton because in many cases the Jacobians can be similar between two steps, so re-using the same Jacobian will allow it to still converge to the correct implicit solution without having to factorize &lt;span class=&quot;math-container&quot;&gt;$(I - \Delta t J)$&lt;/span&gt; again.&lt;/p&gt;&#xA;&lt;p&gt;Multistep methods require previous time points in order to calculate their solution (look at the formulas). Because of this, they need some other solutions in order to get started. Thus a very simple way to do this is to create an adaptive order method. These start with a 1st order method (Euler for Adams, Backwards Euler for BDF), and then after calculating one value can move up to second order (based on some criteria), and so forth. As the order increases, the error per step decreases, and thus the stepsize can change. Thus the standard non-stiff multistep solver is an adaptive order 1-12 Adams method with adaptive stepsize, and the standard stiff multistep solver is an adaptive 1-5 BDF method with adaptive stepsize. I won't even describe how the choice of next order or time step occurs here because that gets quite complicated quite fast.&lt;/p&gt;&#xA;&lt;p&gt;Now in the 90's a few people like Shampine and Petzold had the idea that these kinds of methods could be combined. Using ideas from Shampine, Petzold created LSODA, which is like LSODE except it has a way to adaptively change from the Adams method to the BDF method and vice versa. Basically, it does eigenvalue estimates to compute how stiff the equation is, and if it's too stiff / too non-stiff it switches behaviors.&lt;/p&gt;&#xA;&lt;p&gt;So LSODA is both an Adams method and a BDF method, which changes its stepsize and its order according to internal error estimates, and in many cases it has to factorize Jacobians for BDF, but it doesn't factorize Jacobians every step if it realizes that the implicit equations will solve fast enough. It is completely and utterly impossible to ascribe any asymptotic algorithmic complexity to this!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;One tangent though:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;as I have also been told that when using this method there is no way to estimate errors&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, ODE solvers do have internal ways to estimate the (local) error, and this is actually what's used in adaptive time stepping methods like LSODA. Most of these use an embedded method, i.e. a separate lower order method that is a different linear combination of the same stages. By being the same stages, it's almost free to compute. However, no embedded method with RK4 is really used. There is one really good error estimator for RK4, but it's very very rare in practice. In fact, it's only used in state-dependent delay solvers and &lt;a href=&quot;http://faculty.smu.edu/shampine/residuals.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shampine describes it in his paper for &lt;code&gt;ddesd&lt;/code&gt;&lt;/a&gt;. The DifferentialEquations.jl implementation has this error estimator implemented (mostly for its use in delay differential equations), and so if you're curious &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/perform_step/fixed_timestep_perform_step.jl#L247-L260&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's some open-source code you can take a look at&lt;/a&gt; for how it's done (&lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/perform_step/fixed_timestep_perform_step.jl#L294-L315&quot; rel=&quot;nofollow noreferrer&quot;&gt;and a more efficient version&lt;/a&gt; for mutable arrays). That's some niche stuff though.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So okay, how do you in practice find something like &amp;quot;algorithmic complexity&amp;quot; of ODE solvers then? The best and most common method is work-precision diagrams. What you do is you take some test equations, solve them at really high accuracy to get a reference solution, then solve it with a bunch of different adaptivity tolerances and stepsize to find the relation between time and error. That's crucial: time doesn't mean anything without its relation to error.&lt;/p&gt;&#xA;&lt;p&gt;There are three big benchmark sets. The first is &lt;a href=&quot;http://www.springer.com/us/book/9783540566700&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer I&lt;/a&gt; and &lt;a href=&quot;http://www.springer.com/us/book/9783540604525&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer II&lt;/a&gt; which are both very good references for the solver algorithms. The second is &lt;a href=&quot;https://archimede.dm.uniba.it/%7Etestset/testsetivpsolvers/?page_id=26&quot; rel=&quot;nofollow noreferrer&quot;&gt;this IVP test suite&lt;/a&gt;. These are pretty outdated though and only have very old algorithms. They don't tend to use methods which solve the linear systems in a multithreaded fashion either, which is something that is quite common these days and pretty necessary for stiff solver efficiency. They also don't have LSODA in them.&lt;/p&gt;&#xA;&lt;p&gt;The third benchmark set is &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffEqBenchmarks.jl&lt;/a&gt;. It utilizes the &lt;a href=&quot;http://diffeq.sciml.ai/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt; software which directly links &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;to a ton of methods&lt;/a&gt; in the Julia programming language and then has a bunch of methods specifically for benchmarking solvers. These benchmarks contain &lt;code&gt;lsoda&lt;/code&gt;, among a whole wealth of other options. On non-stiff problems, &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/NonStiffODE/Pleiades%20Work-Precision%20Diagrams.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;lsoda tends to do okay but not as well as high order Runge-Kutta methods&lt;/a&gt; (I'll just be linking one of many test equations that highlight the overarching summary, feel free to view more). For stiff problems, it tends to do &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffODE/Hires.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;quite well for high tolerance but is pretty ehh when high accuracy is necessary&lt;/a&gt; (note: the color of &lt;code&gt;Rodas5&lt;/code&gt; and &lt;code&gt;lsoda&lt;/code&gt; is close but you can see which one is &lt;code&gt;Rodas5&lt;/code&gt; by comparing to the next set... that needs to get fixed). In some cases, at high accuracy its time &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffODE/Pollution.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;can blow up like seen here&lt;/a&gt;, likely because the adaptivity algorithm starts erroneously switching to the explicit mode for small enough time steps only to need to switch back.&lt;/p&gt;&#xA;&lt;p&gt;A point of comparison is in order. This &lt;code&gt;lsoda&lt;/code&gt; is a wrapper over a &lt;a href=&quot;https://github.com/sdwfrost/liblsoda&quot; rel=&quot;nofollow noreferrer&quot;&gt;C-translation of the Fortran code that SciPy uses&lt;/a&gt;, so it will have similar characteristics. The usage in SciPy will be a quite slower though since Python link a compiled &lt;code&gt;f&lt;/code&gt; derivative function like Julia does, but other than that there's not going to be anything different about the algorithm's behavior.&lt;/p&gt;&#xA;&lt;h2&gt;Summary&lt;/h2&gt;&#xA;&lt;p&gt;So let me end by summing up a few characteristics about the algorithm. It's very multi-paradigm. It does fine in almost all of the standard benchmarked cases, though never is really the best. But it can switch modes to handle non-stiff problems okay, and stiff problems well (it has issues with stiff problems when the tolerance gets lower though). But if all you ever wanted was one algorithm to use in any case, this would be a great pick (which is most likely what it's the SciPy default).&lt;/p&gt;&#xA;&lt;p&gt;Notice though that I said &amp;quot;standard benchmarked cases&amp;quot;. There are many cases where you would not want to use this algorithm. For example, &lt;a href=&quot;http://www.stochasticlifestyle.com/algorithm-efficiency-comes-problem-information/&quot; rel=&quot;nofollow noreferrer&quot;&gt;I describe in a blog post&lt;/a&gt; that the cost of using multistep methods increases tremendously if you have lots of events (because they cannot use past time points anymore and need to &amp;quot;go back to Euler&amp;quot;). Also, BDF methods are only L-stable up to second order, so they have problems with very stiff ODEs (which is why for example MATLAB recommends you use something other than &lt;code&gt;ode15s&lt;/code&gt; if the problem is too stiff). Multistep methods in general are more advantageous for large systems then they are for small systems due to their single step per time step, though other methods like Runge-Kutta methods can take much larger steps and get the same error due to optimization of constants in the expansion (&lt;a href=&quot;https://scicomp.stackexchange.com/questions/27178/bdf-vs-implicit-runge-kutta-time-stepping/27179#27179&quot;&gt;described here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;That's probably the simplest way to comprehensively describe the algorithmic complexity (or efficiency in solving ODEs) of LSODA, the default choice for &lt;code&gt;odeint&lt;/code&gt; in SciPy 1.0.&lt;/p&gt;&#xA;&lt;h2&gt;Edit&lt;/h2&gt;&#xA;&lt;p&gt;For future reference, &lt;a href=&quot;http://epubs.siam.org/doi/abs/10.1137/0904010&quot; rel=&quot;nofollow noreferrer&quot;&gt;this is the original paper on &lt;code&gt;lsoda&lt;/code&gt;&lt;/a&gt;. &lt;code&gt;lsoda&lt;/code&gt; is a modification of the &lt;code&gt;lsode&lt;/code&gt; Adams and BDF methods, so it's not referred to by name here and is instead referred to as &amp;quot;a modified version of the LSODE package&amp;quot; (which makes it extremely hard to find to search for haha).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T17:52:01.640" LastActivityDate="2020-07-19T17:52:01.640" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="28621" PostTypeId="1" AcceptedAnswerId="28643" CreationDate="2018-01-15T21:34:23.903" Score="2" ViewCount="1123" Body="&lt;p&gt;I am trying to compute the Ising model for NiO. As O carries no magnetic moment, I only need to consider the case of Ni which requires a second nearest neighbour Ising model. As can be seen in the figure below, the Ni atoms interact with the their nearest neighbours with a coupling constant J1 = 2.3 meV and their second nearest neighbours with a coupling constant J2 = -21 meV. &lt;a href=&quot;https://i.stack.imgur.com/UWM0d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UWM0d.png&quot; alt=&quot;Structure of NiO&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created some code that generates a matrix that alternates 1 and -1 (spin up/spin down) in every second entry and 0 for every other entry (representing oxygen). I have also defined functions that will flip the spin for every nearest neighbour and second nearest neighbour. As the dominating coupling constant J2 &amp;lt; 0, the system should be antiferromagnetic so the spins should align diagonally repeating the pattern (1,0,-1,0) e.g:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   [ 0 -1  0  1  0 -1  0  1  0 -1]&#xA;   [ 1  0 -1  0  1  0 -1  0  1  0]&#xA;   [ 0  1  0 -1  0  1  0 -1  0  1]&#xA;   [-1  0  1  0 -1  0  1  0 -1  0]&#xA;   [ 0 -1  0  1  0 -1  0  1  0 -1]&#xA;   [ 1  0 -1  0  1  0 -1  0  1  0]&#xA;   [ 0  1  0 -1  0  1  0 -1  0  1]&#xA;   [-1  0  1  0 -1  0  1  0 -1  0]&#xA;   [ 0 -1  0  1  0 -1  0  1  0 -1]&#xA;   [ 1  0 -1  0  1  0 -1  0  1  0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However when I run the code, I am not able to achieve that. I can reach a certain amount of order at low temperatures (T~2) but not total ferromagnetism as can be seen below. Going lower (e.g. T~0.01) will yield disorder:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cX7gV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cX7gV.png&quot; alt=&quot;Incorrect Simulated Ising Model for NiO&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #!/usr/bin/env python&#xA;    import numpy as np&#xA;    import scipy as sp&#xA;    import matplotlib.pyplot as plt&#xA;    import random&#xA;&#xA;&#xA;     #constants&#xA;     N = 10 #dimensions of matrix&#xA;     J1 = 2.3 #coupling constant&#xA;     J2 = -21&#xA;     h = 0 #magnetic field, must be set to 0 to compute observables &#xA;     counts = 100&#xA;     T = 2 #temperature&#xA;     k=1 #boltzmann constant&#xA;&#xA;&#xA;    class initial_lattice: &#xA;        def __init__(self,N):   #create initial matrix of size NxN&#xA;            self.N=N&#xA;            self.matrix_lattice()&#xA;&#xA;        def matrix_lattice(self):&#xA;            self.lattice = np.random.choice([-1, 1], (N, N))&#xA;            self.lattice[::2, ::2] = 0&#xA;            self.lattice[1::2, 1::2] = 0&#xA;&#xA;    lattice1=initial_lattice(N) &#xA;&#xA;    #function that sums up all neighbouring sites of the inital position. %N imposes a boundary condition so the function knows when to stop &#xA;    def diagonal_neighbours(matrix,x,y,N): &#xA;        d1 = matrix[(x+1)%N, (y+1) %N]&#xA;        d2 = matrix[(x+1)%N, (y-1) %N]&#xA;        d3 = matrix[(x-1)%N, (y+1)%N]&#xA;        d4 = matrix[(x-1) %N, (y-1)%N]&#xA;        return d1 + d2 + d3 + d4&#xA;&#xA;    def lateral_neighbours(matrix,x,y,N): &#xA;        l1 = matrix[x, (y+2) %N]&#xA;        l2 = matrix[x, (y-2) %N]&#xA;        l3 = matrix[(x+2) %N, y]&#xA;        l4 = matrix[(x-2) %N, y]&#xA;        return l1 + l2 + l3 + l4&#xA;&#xA;&#xA;&#xA;    #function for change in energy&#xA;    def deltaE(matrix, x, y, N, J1, J2, h): &#xA;        return -(2*J1*matrix[x,y]*(diagonal_neighbours(matrix,x,y,N)))-(2*J2*matrix[x,y]*(lateral_neighbours(matrix,x,y,N)))+2*h*matrix[x,y] &#xA;&#xA;    #metropolis algorithim &#xA;    def metropolis(matrix, counts,N, T, J1,J2, h, k):&#xA;        for n in range (counts):&#xA;            for y in range(0, N):&#xA;                for x in range(0,N):&#xA;                    if deltaE(matrix, x, y, N, J1, J2, h)&amp;gt;=0: &#xA;                        matrix[x,y] *= -1 #if energy change is greater than/equal to 0, flips spin&#xA;                    else:&#xA;                        r = random.random() #generates random number&#xA;                        if r&amp;lt;np.exp(deltaE(matrix, x, y, N, J1, J2, h)/(k*T)):      &#xA;                                matrix[x,y] *= -1 #if random number generated between 0 and 1 is less than exp^dE/k*T flips spin&#xA;        return matrix&#xA;&#xA;&#xA;   print metropolis(lattice1.lattice, counts,N, T, J1, J2, h, k)&#xA;&#xA;   plt.imshow(metropolis(lattice1.lattice, counts,N, T, J1, J2, h, k),cmap='bwr',interpolation=&quot;none&quot;) &#xA;   plt.show() #plots Ising model in equilibrium &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="26577" LastActivityDate="2020-06-21T10:21:17.730" Title="Computing the Ising Model for NiO" Tags="|python|monte-carlo|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28622" PostTypeId="1" CreationDate="2018-01-15T21:50:40.053" Score="2" ViewCount="160" Body="&lt;p&gt;Assume some basic knowledge of numerical methods for PDEs, acquired through A. Quarteroni's &lt;em&gt;Numerical models for differential problems&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a reference to &lt;em&gt;get started&lt;/em&gt; on the analysis of PDEs and modeling for industrial problems and related computational approaches (with sample Matlab code available to illustrate them).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are there monographs that deal with concrete complicated (possibly messy)  industrial problems (elasticity, fluid mechanics, continuum mechanics, chemical reactions, etc.) involving PDEs (i.e. &lt;em&gt;PDE models that arise in real-life &quot;industrial&quot; settings&lt;/em&gt;)? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm looking for rigorous analysis and numerical analysis. Also, I'd prefer if such books included fairly complete source codes of implementations of the proposed numerical algorithms in MATLAB.&lt;/p&gt;&#xA;" OwnerDisplayName="user26579" LastEditorDisplayName="user26579" LastEditDate="2018-01-16T23:36:37.043" LastActivityDate="2018-01-18T09:47:29.197" Title="Analysis and numerical methods for PDEs arising in industrial problems" Tags="|pde|reference-request|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="28625" PostTypeId="1" AcceptedAnswerId="28627" CreationDate="2018-01-16T16:12:37.653" Score="0" ViewCount="173" Body="&lt;p&gt;Given two Hilbert spaces $X$ and $Y$ and $a \colon X \times X \to \mathbb{R}$ as well as $b \colon X \times M \to \mathbb{R}$, both being continuous bilinear forms with $a(\cdot, \cdot)$ being symmetric and coercive for all $v \in X$. $f \in X'$ and $g \in M'$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am worried about the proper connection between the minimum formulation and the saddle point formulation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Find the minimum $u \in X(g) = \{ u \in X \mid b(u,q) = g(q) \, \forall q \in M \}$ of the quadratic functional $J(u) := \frac12 \, a(u,u) - l(u)$. With other words:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$\min_{u \in X(g)} \, J(u).$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now the Lagrangian Functional $L \colon X \times M \to \mathbb{R}$ is introduced, given as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ L(u, \lambda) := J(u) + (b(u,\lambda) - g(\lambda)).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for all $\lambda$ we do the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ X \times M \to X \times {\lambda} \cong X  \to \mathbb{R}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with finding $u_\lambda \in X$ for all $\lambda$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then: We are trying to find out, which $\lambda \in M$ do fulfill the condition that $u_\lambda \in X$. This must be a solution of the minimising problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there is a conversion to a system of equations of the approach given above, namely:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Find $(u, \lambda) \in X \times M$ such that $$ a(u,v) + b(v, \lambda)&#xA; = l(v), \quad \forall v \in X \\ b(u,\mu) = g(\mu), \quad \forall \mu \in M$$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I cannot see how to transfer the given procedure to the equations given above.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thoughts so far: As we are dealing with a Lagrangian, the best seems to be to take the derivatives, both of $u$ and $\lambda$. What puzzles me most is what happens to the factor $\frac12$ in the defintion of $a(\cdot, \cdot)$. Is this gone due to the symmetry? How do the $v$ and $\mu$ come into place when we take a look at the derivative of the Lagrangian? Especially: how do we know which argument of $a$ can be replaced by $v$?&lt;/p&gt;&#xA;" OwnerUserId="22586" LastEditorUserId="17294" LastEditDate="2018-01-16T17:23:15.610" LastActivityDate="2018-01-16T19:43:46.167" Title="Saddle point formulation and minima" Tags="|saddlepoint|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28626" PostTypeId="1" CreationDate="2018-01-16T17:20:45.140" Score="1" ViewCount="65" Body="&lt;p&gt;I want to solve the following system via Matlab &#xA;$\Omega=(0,1)^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Delta y=\frac{1}{\alpha} p$$&#xA;$$ -\Delta p= y -1 $$&#xA;$$p|_{\partial \Omega}=0,~y|_{\partial \Omega}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    n=50;&#xA;    alpha=1;&#xA;    A=gallery('poisson',n);&#xA;&#xA;    K=[A -1/alpha*speye(n^2);&#xA;       speye(n^2) A];&#xA;    b=[zeros(n^2,1) -ones(n^2,1)];&#xA;    solution=K\b;&#xA;    %embedding with boundaries...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To my surprise this leads to (I think) a correct answer, however I usually have this scaling factor $\frac{1}{h^2}=(n+1)^2$ in it. Taking $-\Delta y =1$ with zero boundaries as an example, this leads to $$\frac{1}{h^2}Ay=1_{n\times1}$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my code incorrect, or does this factor vanish somehow?&lt;/p&gt;&#xA;" OwnerUserId="26590" LastEditorUserId="17294" LastEditDate="2018-01-16T18:39:25.173" LastActivityDate="2018-01-16T18:39:25.173" Title="scaling in discretized PDE system" Tags="|matlab|elliptic-pde|optimal-control|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="28627" PostTypeId="2" ParentId="28625" CreationDate="2018-01-16T19:38:39.237" Score="3" Body="&lt;p&gt;To expand on @GoHokies comment: The answer to your all your questions  is basically &quot;because $a$ is bilinear and symmetric&quot; (necessarily so; without symmetry this approach wouldn't work). Specifically, don't think of gradients but of directional derivatives: You have a saddle point if the derivative in every direction vanishes, i.e., if the Fréchet derivative (you should look this up; understanding how derivatives of functionals are defined is important) of $L$ at $u\in X$ applied to &lt;em&gt;every&lt;/em&gt; direction $\nu\in X$ gives zero:&#xA;$$L'(u,\lambda)\nu = 0 \qquad\text{for all } \nu\in X.\label{cc:1}\tag{1}$$&#xA;Now $l$ and $b$ are linear in $u$, hence $l'(u)\nu = l(\nu)$ and similarly for $b$. The bilinear form is quadratic in $u$ and so by the product rule,&#xA;$$\frac{d}{du} a(u,u)\nu = \frac{d}{du} a_1(u,u)\nu + \frac{d}{du} a_2(u,u)\nu = a(\nu,u) + a(u,\nu) = 2 a(u,\nu),$$&#xA;where $a_1$ is $a$ with the first argument frozen (i.e., you only take the derivative with respect to the first argument) and similarly for $a_2$, and I have used that $a$ is bilinear in each argument separately. The last step is of course due to the symmetry of $a$ (which also means that &lt;em&gt;it doesn't matter&lt;/em&gt; which argument of $a$ is replaced, answering your third question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, \eqref{cc:1} reduces to&#xA;$$a(u,\nu) + b(\nu,\lambda) = l(\nu)\qquad\text{for all }\nu \in X.$$&#xA;The second equation works in the same way by looking at the derivative with respect to $\lambda$ in all directions $\mu$. &lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2018-01-16T19:43:46.167" LastActivityDate="2018-01-16T19:43:46.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28628" PostTypeId="1" AcceptedAnswerId="28631" CreationDate="2018-01-16T19:56:04.490" Score="0" ViewCount="203" Body="&lt;p&gt;I have an ODE system of the general form&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  y' = k(y)(x) + q(z)(x)&#xA;  x' = a(z)(x) + b(x)(x) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where k,q,a and b are also dependent on the states x and y. The solver(ODEINT) is throwing the error: &quot;Max number of iterations exceeded. A new step size was not found&quot;. I am using the Rosenbrock method on a stiff system. Is the step size for Rosenbrock method iteratively calculated? If so, what parameters should I look to adjust to prevent this isssue?&lt;/p&gt;&#xA;" OwnerUserId="25530" LastActivityDate="2018-01-17T05:58:54.147" Title="Is the time step size of a Rosenbrock method for stiff systems iteratively calculated?" Tags="|ode|c++|stiffness|adaptive-timestepping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28630" PostTypeId="1" CreationDate="2018-01-17T00:04:09.363" Score="1" ViewCount="70" Body="&lt;p&gt;I a solving the 2d steady state convection-diffusion problem on the famous flow around a cylinder in rectangular domain benchmark. My numerical results show that with Q1 finite elements, the solver is converging for Pe=5000, whereas with Q2  discretization, the solver fails to converge. What can be the reason for poor performance in case of Q2 FEM discretization?  &lt;/p&gt;&#xA;" OwnerUserId="18196" LastActivityDate="2018-01-17T00:04:09.363" Title="Why do higher order finite elements (Q2) do not perform well for large Peclet number flows, as compared to Q1 finite elments?" Tags="|finite-element|linear-solver|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28631" PostTypeId="2" ParentId="28628" CreationDate="2018-01-17T05:58:54.147" Score="2" Body="&lt;p&gt;Rosenbrock methods utilize embedded lower order methods in order to calculate errors for adaptive time stepping. In addition, Rosenbrock methods do not have to solve an implicit system (just a linear system). There is no form of iteration then that takes place in them (unless you're using a Krylov linear solver).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maximum number of steps for stiff solver can be due to many reasons. The first one is simply that the solver diverged. If it diverges, then the errors grow and the time steps shrink. In many cases, the number of steps could just run out before the solver notices it diverged, and throws this error accidentally. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Divergence of the solver can happen for many reasons. One reason this could happen is because the tolerances are too high. While in theory these methods are L-stable, adaptive time stepping and errors in the Jacobian change this in practice. But it will always be stable below some tolerance, so give that a try. If that doesn't work, then look at your derivative function. Most likely it's user-error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's not divergence (which you'd be able to see from a plot), then you might just be taking too many steps if you're asking for a really low tolerance. In that case, make sure you really need that much accuracy and if you do, there's no choice except increase the number of max steps.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-01-17T05:58:54.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28633" PostTypeId="1" CreationDate="2018-01-17T19:53:27.267" Score="1" ViewCount="126" Body="&lt;p&gt;I am trying to implement an incremental projection method for the 2D incompressible Navier-Stokes.  The type of projection method I am trying is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{u^{*} - u^{n}}{dt} = - \nabla p^{n} - u \cdot \nabla u + \nu \nabla^{2} u\\&#xA;u^{*}|_{\partial D} = u^{n+1}|_{\partial D}\\&#xA;\nabla^{2} \phi^{n+1} = \frac{\rho}{dt} \nabla \cdot u^{*}\\&#xA;\nabla \phi^{n+1}|_{\partial D} = \frac{1}{dt} \left(u^{*} - u^{n+1}\right)\\&#xA;\frac{u^{n+1} - u^{*}}{dt} = - \nabla \phi^{n+1}\\&#xA;p^{n+1} = p^{n} + \phi^{n+1} (5)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of the velocity-pressure arrangement I am using a collocated formulation using the Rhie-Chow interpolation to have stable coupling.  I however have run into a problem in terms of the practical implementation of the pressure update.  The way I update pressure is as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Pnew(1:nx,1:ny) = P(1:nx,1:ny) + Phi(1:nx,1:ny)&#xA;P(1:nx,1:ny) = Pnew(1:nx,1:ny)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am testing my solver on the lid driven cavity.  My velocity profile is correct, however my pressure profile is completely wrong as it contains wiggles.  Here is a picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xVLNb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xVLNb.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested my solver on a pressure-free projection method(where there is no $\nabla p^{n}$ in the intermediate veloctiy step and it works fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: (1/17/18)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I forgot to mention but I am using 2nd-order central discretization for both convection and diffusion.  My integration technique is currently forward Euler for debugging purposes.  &lt;/p&gt;&#xA;" OwnerUserId="21337" LastEditorUserId="21337" LastEditDate="2018-01-17T22:06:25.420" LastActivityDate="2018-01-17T22:06:25.420" Title="(Approximate) Incremental Projection Method for Navier-Stokes equations" Tags="|numerical-analysis|finite-difference|fluid-dynamics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28634" PostTypeId="1" AcceptedAnswerId="28638" CreationDate="2018-01-17T21:20:46.393" Score="2" ViewCount="1033" Body="&lt;p&gt;I have been learning about the impact of cache size on code performance. I wrote a small code to see how using a column major loop in MATLAB would be better than using a row major loop, since MATLAB stores matrices in column major like FORTRAN. I also compared against MATLAB's internal multiplication routine. Here is the code and the results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% row major access&#xA;tic&#xA;for i=1:n&#xA;    for j=1:n&#xA;        b1(i)=b1(i)+A(i,j)*x(j);&#xA;    end&#xA;end&#xA;t1(count)=toc;&#xA;&#xA;% column major access&#xA;tic&#xA;for j=1:n&#xA;    for i=1:n&#xA;        b2(i)=b2(i)+A(i,j)*x(j);&#xA;    end&#xA;end&#xA;t2(count)=toc;&#xA;&#xA;% column major vector ops&#xA;tic&#xA;for i=1:n&#xA;    b3(i)=b3(i)+A(i,:)*x(:);&#xA;end&#xA;t3(count)=toc;&#xA;&#xA;% row major vector ops&#xA;for j=1:n&#xA;    b4(:)=b4(:)+A(:,j)*x(j);&#xA;end&#xA;t4(count)=toc;&#xA;&#xA;% MATLAB built in&#xA;tic&#xA;b5=A*x;&#xA;t5(count)=toc;&#xA;&#xA;% double vop&#xA;tic&#xA;b6=A(:,:)*x(:);&#xA;t6(count)=toc;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dphVV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dphVV.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The column major loop is faster than the row major loop, but how is MATLAB &lt;strong&gt;so much&lt;/strong&gt; faster?&lt;/p&gt;&#xA;" OwnerUserId="22717" LastEditorUserId="22717" LastEditDate="2018-01-18T18:51:02.680" LastActivityDate="2018-01-19T03:37:49.587" Title="Matrix vector multiplication performance" Tags="|matrix|numerics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28635" PostTypeId="2" ParentId="28634" CreationDate="2018-01-17T21:29:32.050" Score="4" Body="&lt;p&gt;Matlab is probably handing off to a vendor BLAS (MKL, etc). Those libraries are totally different animals than a Matlab double-for loop (thread-parallel, SIMD intrinsics, algorithmically reblocked, the works).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for loops are generally pokey in Matlab anyway. You'll probably see speedups (in both cases) just by rewriting the calculation in some form of vector notation (handle whole columns/rows by writing &lt;code&gt;A(:,j)&lt;/code&gt;/&lt;code&gt;A(i,:)&lt;/code&gt;, that kind of thing).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-01-18T13:21:57.727" LastActivityDate="2018-01-18T13:21:57.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28636" PostTypeId="1" CreationDate="2018-01-17T23:01:50.810" Score="2" ViewCount="366" Body="&lt;p&gt;I am using Python but I wouldn't mind changing language. All I have gotten from my research are tools to &lt;em&gt;count&lt;/em&gt; the number of (lattice) points inside a region given the equations for the planes that enclose it. Other tools are made to optimize a given function inside the polytope (linear programming).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How about finding the lattice points alone? For example, a function of the kind&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;latticePoints( 'x &amp;lt; 5 &amp;amp; x &amp;gt; 0' ) = [ 1, 2, 3, 4]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Plus I am looking for something to work in the multivariate scenario (constrains on x, y, z, ...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently trying to solve this using &lt;a href=&quot;https://pypi.python.org/pypi/pplpy/0.7&quot; rel=&quot;nofollow noreferrer&quot;&gt;ppl&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26600" LastEditorUserId="20417" LastEditDate="2018-01-18T17:27:41.123" LastActivityDate="2018-01-18T17:27:41.123" Title="Finding integer/lattice points (coordinates) inside a polytope/polyhedra?" Tags="|linear-algebra|algorithms|python|linear-programming|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28638" PostTypeId="2" ParentId="28634" CreationDate="2018-01-18T00:16:30.983" Score="5" Body="&lt;p&gt;MATLAB loops are slow. This is known. If you use something like C++ or Julia you'll get much closer (like 5x-10x or so IIRC). So that's why it's super crazy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, the algorithms that MATLAB is calling are actually &lt;a href=&quot;http://www.netlib.org/blas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAS&lt;/a&gt; libraries. Specifically, MATLAB is calling &lt;a href=&quot;https://software.intel.com/en-us/mkl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel MKL&lt;/a&gt; under the hood here, so it's a blazing fast multithreaded code. In fact, it does all sorts of things. It first loads all inputs into contiguous stack-allocated buffers  Then it does some bit twiddling on pointers to make things move faster, SIMD's a few things, and does this all with parallelism. It's wild stuff to try and understand. If you're curious &lt;a href=&quot;http://www.openblas.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenBLAS&lt;/a&gt; is open-source (instead of MKL which is not), so you can take a look and try to understand what's going on, &lt;a href=&quot;http://web.cs.ucdavis.edu/~bai/ECS231/optmatmul.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;but it's crazy&lt;/a&gt;!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Edit:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I originally said &quot;and then does operations via stencils so that way it's actually an order $\mathcal{O}(n^{log_2 7})$ instead of the naive $\mathcal{O}(n^{3})$ that using the definition of matrix multiplication gives&quot;. I guess it doesn't actually use Strassen in practice. Also, that's for matrix multiplication, not specifically matrix-vector multiplication like the question asked. Sorry.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2018-01-19T03:37:49.587" LastActivityDate="2018-01-19T03:37:49.587" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28640" PostTypeId="1" CreationDate="2018-01-18T03:47:05.623" Score="4" ViewCount="690" Body="&lt;p&gt;What are the differences between &lt;a href=&quot;http://www.hsl.rl.ac.uk/catalogue/ma57.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MA57&lt;/a&gt; and &lt;a href=&quot;http://www.hsl.rl.ac.uk/catalogue/hsl_ma57.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;HSL_MA57&lt;/a&gt; solvers? I'm in an optimization class that will make use of symmetric indefinite factorizations, and I'm trying to learn about the distinction between these two. I know that &lt;a href=&quot;http://www.mathworks.com/help/matlab/ref/ldl.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MATLAB&lt;/a&gt; uses the MA57 solver but am unsure of the differences beyond this. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="22292" LastActivityDate="2021-06-24T12:46:39.900" Title="MA57 vs HSL_MA57: symmetric indefinite solvers" Tags="|linear-algebra|linear-solver|sparse-matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28642" PostTypeId="2" ParentId="28634" CreationDate="2018-01-18T08:33:19.300" Score="3" Body="&lt;p&gt;An addition to the other two (excellent) answers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Understanding the source code of something like OpenBLAS can be a daunting task. As an alternative starting point, you could read through&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://users.ece.cmu.edu/~franzf/papers/gttse07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Write Fast Numerical Code: A Small Introduction&lt;/a&gt; by Chellappa et al.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Section 5 deals with the MMM (matrix-matrix multiplication) operation. You start from a &quot;naïve&quot; triple-loop MMM kernel and examine its memory data access patterns. Then the authors show you how to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;optimize cache memory usage via blocking and buffering (section 5.1)&lt;/li&gt;&#xA;&lt;li&gt;optimize MMM at the CPU and register level using blocking and loop unrolling (section 5.2)&lt;/li&gt;&#xA;&lt;li&gt;&quot;tune&quot; the performance by selecting the optimal parameters (cache/register block size, etc.) for your CPU architecture (section 5.3). You do this with a tool like &lt;a href=&quot;http://math-atlas.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ATLAS&lt;/a&gt; (section 5.4).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This won't get you anywhere near the performance of a highly-optimized library like Intel MKL, but you should nevertheless see a significant improvement over the naïve kernel. And hey, it's a fun exercise :)&lt;/p&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2018-01-18T08:33:19.300" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28643" PostTypeId="2" ParentId="28621" CreationDate="2018-01-18T12:51:14.410" Score="1" Body="&lt;p&gt;Your Metropolis algorithm's loop is wrong as this is not a Metropolis procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're looping through all the spins, flipping them is the flip reduces the energy when you actually have to pick $x,y \in [0,N]$ randomly at each step and check if flipping is reducing the energy. If it does, you keep the flip, if it doesn't you keep it with the probability distribution you're using (i.e. keep if $\chi &amp;gt; 1/k_BT$ with $\chi$ uniformly distributed on $[0,1]$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would write something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;[...]&#xA;for n in range(counts):&#xA;    x = random.randint(0,N)&#xA;    y = random.randint(0,N)&#xA;    if deltaE(matrix, x, y, N, J1, J2, h)&amp;gt;=0:&#xA;    [...]&#xA;return matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this loop only return one matrix. If you want to compute any statistically interesting value, you have to compute regularly values like energy or mean orientation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit :&lt;/strong&gt;&#xA;I tried to make your code work and beside this algorithm problem, a few other comments came to my mind. Here is my code with a few comments:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import numpy as np&#xA;#import scipy as sp&#xA;import matplotlib.pyplot as plt&#xA;import random&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Simple note, I didn't need scipy.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#constants&#xA;N    = 10 #dimensions of matrix&#xA;J1   = 2.3*10**(-3) #coupling constant in Ev&#xA;J2   = -21*10**(-3)&#xA;h    = 0 #magnetic field, must be set to 0 to compute observables &#xA;counts = 1000&#xA;T    = 1 #temperature in K&#xA;k    = 8.6173303*10**(-5) #boltzmann constant in Ev/K&#xA;beta = 1/(k*T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Global variable are rarely a good idea (in short code like that it's no big deal). Use conditions to use/define them like &quot;if (main):&quot;. More importantly, when using physical units (here Ev and K), convert everything in the same set of units. $k_B$ has a dimension of energy per kelvin, use it. If you want to use dimensionless units, you have to compute $J1/k_BT$ and $J2/k_BT$ first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've made a function to compute the energy of a single configuration:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Compute actual energy don't forget periodic boundary conditions&#xA;def Energy(matrix, J1, J2, h):&#xA;    E = 0&#xA;    for x in range(N):&#xA;        for y in range(N):&#xA;            N1 = (x+1)%N&#xA;            N2 = (y+1)%N&#xA;            N3 = (x-1)%N&#xA;            N4 = (y-1)%N&#xA;            for i in [N1,N3]:&#xA;                for j in [N2,N4]:&#xA;                    E += matrix[x,y]*J1*matrix[i,j]/2.&#xA;            N1 = (x+2)%N&#xA;            N2 = (y+2)%N&#xA;            N3 = (x-2)%N&#xA;            N4 = (y-2)%N&#xA;            for i in [N1,N3]:&#xA;                for j in [N2,N4]:&#xA;                    E += matrix[x,y]*J2*matrix[i,j]/2.&#xA;            E -= h*matrix[x,y]&#xA;    return E&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is important to note the evolution of your instantaneous observables for 2 reasons:&#xA; 1. If your outputs are wrong, you need to check how the different variables evolve during the simulation&#xA; 2. If you want to compute macroscopic observables, you need to make some statistics on instantaneous observables.&#xA;Therefore your initial value of the energy is important to check the convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my version of your Metropolis algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#metropolis algorithm &#xA;def metropolis(matrix, counts, N, beta, J1, J2, h):&#xA;# This array E will stock energy values, E_conf will be the instantaneous value.&#xA;    E = np.zeros(counts)&#xA;    E_conf = Energy(matrix, J1, J2, h)&#xA;    for n in range (counts):&#xA;# Coordinates of the flip have to be reset for each move and randomly chosen!&#xA;        x = 0&#xA;        y = 0&#xA;# I know that (0,0) gives an Oxygen coordinates, I randomly chose values until I get a Ni atom&#xA;        while (matrix[x,y] == 0):&#xA;            x = random.randint(0,N-1)&#xA;            y = random.randint(0,N-1)&#xA;        dE = deltaE(matrix, x, y, N, J1, J2, h)&#xA;        if (dE &amp;lt;= 0):&#xA;            matrix[x, y] *= -1&#xA;            E_conf += dE&#xA;        else:&#xA;            r = random.uniform(0,1)&#xA;            tau = np.exp(-dE*beta)&#xA;            if (r &amp;lt; tau) :&#xA;                matrix[x, y] *= -1&#xA;                E_conf += dE&#xA;        E[n] = E_conf&#xA;    plt.plot(E)&#xA;    plt.show()&#xA;    return matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I modified your algorithm in several ways. Your version was looping in the atoms in a deterministic way as I commented first. This one pick the atoms in a pseudo-random way which is the heart of the Metropolis algorithm. Also, you can use intermediate variables for condition in order to make the code easier to read. Writing lines like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (function(x,y,z,a,b,c) &amp;gt; 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Quickly make things confusing. Python is not really designed for direct functional evaluation like this. (Also note that you got the signs wrong in your initial version.)&#xA;You also used random.random() library which doesn't explicitly give you uniform distribution. random.uniform() is here for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Initialising&#xA;lattice1=initial_lattice(N)&#xA;mat = lattice1.lattice&#xA;&#xA;print (&quot;Starting from initial configuration:&quot;)&#xA;print (mat)&#xA;&#xA;plt.imshow(mat,cmap='bwr',interpolation=&quot;none&quot;) &#xA;plt.show() #plots initial configuration&#xA;&#xA;mat = metropolis(mat, counts, N, beta, J1, J2, h)&#xA;&#xA;print (mat)&#xA;&#xA;plt.imshow(mat,cmap='bwr',interpolation=&quot;none&quot;) &#xA;plt.show() #plots last configuration&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Don't hesitate to stock intermediate variable in this kind of situation. If you pass a function like metropolis(...) as a variable of another function (say print) it will cause a new evaluation of the function. Again, python is not well suited for this kind of functional programming style (unless you're using generators with lazy evaluation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A final comment on your expected behavior: MC-metropolis algorithm is random. You expected to observe the configuration at equilibrium but after 100 random flips (not all of them accepted), it is not said you'll converge to equilibrium. the shortest convergence I observed with my algorithm was around 250 flips but don't hesitate to set them to 10000. Such a simple code must be quite fast. Be aware though that the convergence will require more and more cycles as the system grows.&lt;/p&gt;&#xA;" OwnerUserId="19790" LastEditorUserId="19790" LastEditDate="2018-01-25T08:55:48.980" LastActivityDate="2018-01-25T08:55:48.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28644" PostTypeId="1" CreationDate="2018-01-18T21:21:47.253" Score="1" ViewCount="53" Body="&lt;p&gt;I am dealing with a integral equation&#xA;$$ f'(t) = -\int_0^t K(s) f(t-s)\quad t\in [0,t_\max] \tag{1}$$&#xA;in which $f(t)$ and $f'(t)$ are known, well-behaved functions of $t$ and $K(t)$ is the unknown. In addition, the physics of the problem dictate that $f(t)$ and $K(t)$ are even about $t=0$. I have discretized this equation on a uniform grid with $n$ intervals of width $\Delta t$. That is, I map&#xA;$$&#xA;\begin{align}&#xA;t &amp;amp;\mapsto i \Delta t \\&#xA;t_\max &amp;amp;\mapsto n\Delta t \\&#xA;\int_0^t ds &amp;amp;\mapsto \Delta t\sum_{j=0}^i w^{(i+1)}_j&#xA;\end{align}&#xA;$$&#xA;where $w^{(i+1)}$ are the weights for an $(i+1)$-point quadrature rule.&#xA;With this, I arrive at a linear system for $K_1,\dots,K_n$:&#xA;$$&#xA;f_i' + \Delta t w_0^{(i+1)}f_i K_0 = -\Delta t \sum_{j=1}^i w_j^{(i+1)} f_{i-j} K_j\quad i=1,\dots,n \tag{2}&#xA;$$&#xA;with $K_0 = K(0)$ assumed known. In order to accommodate quadrature rules that require an odd number of points (e.g., Simpson's), I use the evenness of $f$ and $K$ to produce the equivalent discretization&#xA;$$&#xA;f_i' + \frac{\Delta t}{2} w_i^{(2i+1)}f_i K_0 = -\Delta t \sum_{j=1}^i w_{i-j}^{(2i+1)} f_{i-j} K_j\quad i=1,\dots,n \tag{3}&#xA;$$&#xA;based on extending the integral to the range $[-t,t]$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am testing my numerical solutions by comparing with analytic solutions to Eq. (1). Namely, &#xA;$$&#xA;f(t) = \cos{(\omega t)} &#xA;\implies K(t) = \omega^2&#xA;$$&#xA; and&#xA;$$&#xA;f(t) = e^{-t/2} \left[\cos{(\alpha t/2)} + \frac{\sin{(\alpha t/2)}}{\alpha}\right]&#xA;\implies K(t) = \frac{1+\alpha^2}{4}e^{-|t|}&#xA;$$&#xA;In both cases, I find that the trapezoidal rule produces a numerical solution $K(t)$ that is in excellent agreement with the analytic one. On the other hand Simpson's rule appears to be unstable, producing numerical solutions that oscillate and rapidly grow in amplitude at $t$ increases. A sample for each test case is shown below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Gkc8C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gkc8C.png&quot; alt=&quot;test cases&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Evidently, there is &lt;em&gt;something&lt;/em&gt; about Simpson's rule that makes the linear system in Eq. (3) ill-conditioned. Writing Eq. (3) in standard form, I have&#xA;$$&#xA;b_i = \sum_{j}A_{ij}x_j&#xA;$$&#xA;with &#xA;$$&#xA;b_i=f'_i+\frac{\Delta t}{2}w^{(2i+1)}_if_iK_0\\&#xA;A_{ij}=-\Delta t w_{i-j}^{(2i+1)} f_{i-j}\\&#xA;x_j=K_j&#xA;$$&#xA;Since the coefficient matrix $A_{ij}$ is lower-triangular, its eigenvalues are its diagonal entries:&#xA;$$\lambda_i = A_{ii} = -\Delta t w_0^{(2i+1)} f_0\quad i=1,\dots,n$$&#xA;Whatever the quadrature rule, $w_0^{(2i+1)}$ is the same for all $i\ge 1$. For the trapezoidal rule $w_0=\frac{1}{2}$, and for Simpson's rule $w_0=\frac{1}{3}$. Then in both cases, there is only one unique eigenvalue of the matrix $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can I use this information about the eigenvalues to ascertain why Simpson's rule is unstable, while trapezoidal is stable?&lt;/li&gt;&#xA;&lt;li&gt;Is there some analytic feature of Eq. (1) that could have led me to know &lt;em&gt;a priori&lt;/em&gt; that Simpson's rule is unstable?&lt;/li&gt;&#xA;&lt;li&gt;If so, is there a more general result (or even rules of thumb) for 1st-kind Volterra equations that says certain types quadrature rules are to be preferred?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2018-01-18T21:21:47.253" Title="Stability of different quadrature rules in 1st-kind Volterra integral equation" Tags="|stability|quadrature|integral-equations|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28645" PostTypeId="1" AcceptedAnswerId="34477" CreationDate="2018-01-18T22:47:38.503" Score="2" ViewCount="1129" Body="&lt;p&gt;I'm developing a finite volume solver for the simple twodimensional advection equation with constant velocities $u, v$ and constant mesh spaces $\Delta x$:&#xA;$$ \frac{\partial \rho}{\partial t} + u \frac{\partial \rho}{\partial x} + v \frac{\partial \rho}{\partial y} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u,v &amp;gt; 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;http://www.mpia.de/homes/dullemon/lectures/hydrodynamicsII/chap_4.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this lecture&lt;/a&gt;, the piecewise linear update scheme in 1D is&#xA;$$ \rho_i^{n+1} = \rho_i^n - \frac{u \Delta t}{\Delta x} ( \rho_i ^n - \rho_{i-1}^n) - \frac{u \Delta t}{\Delta x} \frac{1}{2} (s_i^n - s_{i-1}^n)(\Delta x - u\Delta t) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $s_i^n$ is  a slope, that may be chosen as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Centered slope: $s_i^n = \frac{\rho_{i+1} - \rho_{i-1}}{2\Delta x}$ (Fromm's method)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upwind slope: $s_i^n = \frac{\rho_{i} - \rho_{i-1}}{\Delta x}$ (Beam-Warming method)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Downwind slope: $s_i^n = \frac{\rho_{i+1} - \rho_{i}}{\Delta x}$ (Lax-Wendroff method)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and any of these choices results in second-order accurate methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented these methods in 1D, and they work as expected. In the plot below is the 1D advection for $u=1$ with periodic boundaries for a step function at various time steps with 1000 cells. An integer time step means that the curve has gone that many times through the whole domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tkiLb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tkiLb.png&quot; alt=&quot;1d advection results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the 2D method, I evaluate the new value simultaneously in both directions, according to the 2d advection equation as above, while approximating the divergence terms by the piecewise linear scheme as stated above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I only let the density advect in one direction, i.e. $u = 0, v=1$ or $u=1, v=0$, I get the expected results, corresponding to the 1D solution. If I choose the timestep to be $\Delta t = [u/\Delta x + v/\Delta y]^{-1}$ as &lt;a href=&quot;https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia suggests&lt;/a&gt; (as opposed to $t&amp;lt; [u/\Delta x + v/\Delta y]^{-1}$), I even get perfect advection without diffusion, as it is predicted by the 1d case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I give both $u$ and $v$ nonzero values, e.g. $u = v = \sqrt{2}/2$, at some point, apparently at some point in time, the solution becomes unstable, and I do not know why. This happens for any choice of the slope. Or does it only look unstable to the inexperienced me, but is in fact just the interference of the oscillations produced by the piecewise linear scheme without any slope limiters? (The situation gets a bit better with a minmod or Van Leer slope limiter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below you can see the results of a 2d simulation at various time steps for $nx = ny = 200$ for a 2D step function, using the downwind slope. (The timesteps are 0, 0.2, 0.4, 0.6, 0.8, 1.0, 2.0, 5.0, 10.0)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ClEOV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ClEOV.png&quot; alt=&quot;2d advection results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me how/why this apparent instability occurs, and what to do against it?&lt;/p&gt;&#xA;" OwnerUserId="26461" LastActivityDate="2020-02-21T11:46:34.277" Title="Finite volume piecewise linear 2D advection develops instability" Tags="|fluid-dynamics|advection|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28647" PostTypeId="1" AcceptedAnswerId="29031" CreationDate="2018-01-19T13:53:35.100" Score="1" ViewCount="184" Body="&lt;p&gt;Is it possible, in general, to combine hydrodynamical motion and expansion of material with, say, a finite difference time domain method to simulate light-matter interaction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, how is this done in practice? I guess you would need some kind of operator splitting scheme. If you know more about it, can you suggest me some papers, textbooks, etc.?&lt;/p&gt;&#xA;" OwnerUserId="21797" LastEditorUserId="20688" LastEditDate="2018-03-13T18:52:01.370" LastActivityDate="2018-03-13T18:52:01.370" Title="Combine Hydrodynamics and Electromagnetics" Tags="|finite-difference|simulation|electromagnetism|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28649" PostTypeId="2" ParentId="28645" CreationDate="2018-01-19T16:24:36.897" Score="0" Body="&lt;p&gt;Each numerical scheme can have a different stability condition, so you can not use the single one as suggested by the page in Wikipedia. Simply try a smaller time step and if your code is correct you should obtain a stable solution for enough small time step. I am on phone so i can not check it and write it in a mathematical mode, but in your case, I think, your time step must be smaller than the minimal time step of two related one dimensional stability restriction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way if you are showing us a typical example you want to solve, you should check the so called Corner Transport Upwind scheme, e.g. in the book of LeVeque on hyperbolic problems, that has the stability condition you are quoting.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastActivityDate="2018-01-19T16:24:36.897" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28650" PostTypeId="1" AcceptedAnswerId="28662" CreationDate="2018-01-19T16:54:01.130" Score="3" ViewCount="445" Body="&lt;p&gt;I am looking at an optimization problem that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\text{minimize}\;\; \mathbf{x}^TQ\mathbf x \;\;, \; \mathbf x \in \mathbb R^n,  x_i \in \lbrace 0, 1 \rbrace\\&#xA;\text{subject to}\;\; ||\mathbf x||_1 = m &amp;lt; n&#xA;$$&#xA;In words, the goal is to select a fixed number $m$ of elements from $\mathbf x$ that minimizes the quadratic term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q$ is a correlation matrix, i.e. a $n \times n$ symmetric matrix with $Q_{ji} \in [-1, 1]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for an efficient algorithm for this problem, by approximation if necessary, and was wondering if this problem belongs to any known class of optimization problems. It is certainly similar to the Quadratic Knapsack Problem, but in this case the constraint here is not an upper limit on a weighted sum, but simply the number of non-zero elements. &lt;/p&gt;&#xA;" OwnerUserId="17170" LastEditorUserId="6910" LastEditDate="2018-02-05T16:54:20.137" LastActivityDate="2018-02-05T16:54:20.137" Title="Knapsack problem with fixed number of elements?" Tags="|optimization|sparse-matrix|constrained-optimization|discrete-optimization|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28651" PostTypeId="1" AcceptedAnswerId="28691" CreationDate="2018-01-18T01:59:19.940" Score="5" ViewCount="171" Body="&lt;p&gt;Consider a $\mathcal{C}^1$ function $V:\Omega\rightarrow\mathbb{R}$ where $\Omega\subset\mathbb{R}^n$. If a random vector $X$ has a parametric density $p_\theta(\textbf{x})$ that's smooth in its parameters and uniformly supported on $\Omega$ we can use the score estimator for a stochastic derivative wrt $\theta\in\mathbb{R}^m$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla \mathbb{E}\left[V(X)\right] = \mathbb{E}\left[V(X)\nabla \log p_\theta(X)\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider now $p_\theta$ as the density for $X\sim N(f(\theta), \epsilon I)$. Then $\nabla \log p_\theta (X)= \epsilon^{-1}J_f(\theta)^\top(f(\theta)- X)$. This then makes it look like we can get a deterministic approximate derivative for $V$ without access to a gradient oracle for $V$, for small $\epsilon$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla V(f(\theta))\approx\epsilon^{-1} J_f(\theta)^\top\int d\textbf{x}\;p_\theta(\textbf{x})V(\textbf{x})(f(\theta)-\textbf{x})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By continuity of $\nabla V$ equality should hold in the limit. Since the integral above looks like a Gaussian convolution, perhaps this is some kind of generalization of the standard $m$-dimensional finite difference procedure for generating an approximation to $\nabla V\circ f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this have a name? If $V$ is, say, $L$-Lipschitz at $f(\theta)$, then would using a quadrature rule for the integral yield a sensible gradient approximation (sensible being as good or better than the $O(m\epsilon^2)$ 2-norm error of symmetric finite differences)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;. Perhaps the integral approximation is distracting. Let's assume we can compute the exact value of the integral and ask the same question about its accuracy for a fixed $\epsilon$ (indeed, for a Monte Carlo approximation, we can make the the integral accurate with high probability by the weak LLN). Though any thoughts on the internal integral would be much-appreciated.&lt;/p&gt;&#xA;" OwnerUserId="23164" OwnerDisplayName="VF1" LastEditorUserId="23164" LastEditDate="2018-01-23T01:44:07.593" LastActivityDate="2018-01-29T21:34:29.687" Title="Kernel-based differentiation" Tags="|probability|quadrature|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28652" PostTypeId="1" CreationDate="2018-01-20T09:14:46.660" Score="1" ViewCount="3025" Body="&lt;p&gt;I'm using python 2.7 (on jupyter notebook, win10 64 bit) to perform my analysis. I need to perform continuum removal (CR) on a reflectance spectrum data. I need it to be as described [here][1].&lt;/p&gt;&#xA;&lt;p&gt;EDITED:&lt;/p&gt;&#xA;&lt;p&gt;I used the following code to perform CR on a reflectance spectrum:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.spatial import ConvexHull&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;i = Xdata.iloc[15,:] #get the data from a pandas df&#xA;ii = i.tolist()&#xA;mat = np.column_stack((wavelenght_list,ii))&#xA;hull = ConvexHull(mat)&#xA;plt.plot(mat[:,0], mat[:,1], 'o')&#xA;for simplex in hull.simplices:&#xA;    plt.plot(mat[simplex, 0], mat[simplex, 1], 'k-')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result I get is:&lt;/p&gt;&#xA;&lt;p&gt;[![enter image description here][2]][2]&lt;/p&gt;&#xA;&lt;p&gt;I have 3 questions about it:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why it did not capture the local maxima at 550?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I remove the baseline of the convelHull (the black curve)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I divide my spectrum by the &amp;quot;upper&amp;quot; conveHull in order to get CR spectrum?&#xA;[1]: &lt;a href=&quot;http://www.harrisgeospatial.com/docs/ContinuumRemoval.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.harrisgeospatial.com/docs/ContinuumRemoval.html&lt;/a&gt;&#xA;[2]: &lt;a href=&quot;https://i.stack.imgur.com/C5qah.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/C5qah.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&#xA;Thanks you all for your help-  I found a more suitable module for my needs to perform the continuum removal algorithm: &lt;code&gt;spectro&lt;/code&gt; in &lt;code&gt;pysptools&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="26607" LastEditorUserId="26607" LastEditDate="2023-01-18T12:50:57.277" LastActivityDate="2023-06-17T13:10:01.213" Title="Continuum removal algorithm in python" Tags="|python|spectral-method|special-functions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28653" PostTypeId="1" AcceptedAnswerId="28656" CreationDate="2018-01-20T14:11:50.163" Score="1" ViewCount="1293" Body="&lt;p&gt;Given $N$ data points, does &lt;code&gt;polyfit&lt;/code&gt; of degree $N-1$ produces the unique interpolating polynomial?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For concreteness, here is a code example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x=[1:10]&#xA;y= x.^3;&#xA;pp = polyfit(x,y,9)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the following warning is issued:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Warning: Polynomial is badly conditioned. Add points with&#xA;distinct X values, reduce the degree of the polynomial, or try&#xA;centering and scaling as described in HELP POLYFIT. &#xA;&amp;gt; In polyfit (line 79) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;pp is indeed $x^3$, as it should be, but why is this ill-conditioned? Interpolation is only ill-conditioned with respect to some bases (say, the monomials $1,x,x^2,\ldots$), but not with respect to others?&lt;/p&gt;&#xA;" OwnerUserId="15188" LastActivityDate="2018-01-20T16:00:13.833" Title="Polynomial Interpolation with Matlab polyfit" Tags="|matlab|interpolation|polynomials|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28655" PostTypeId="2" ParentId="28653" CreationDate="2018-01-20T15:00:50.340" Score="2" Body="&lt;p&gt;In the end, this is a linear system. Its matrix is ill-conditioned: &lt;code&gt;cond(vander(1:10))&lt;/code&gt; returns &lt;code&gt;2.1063e+12&lt;/code&gt;. So if you change &lt;code&gt;y&lt;/code&gt; by a certain amount, the change is amplified (in the worst case, using relative errors) by a factor &lt;code&gt;2.1063e+12&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The picture should be clear even without formulating statements such as &quot;Interpolation is only ill-conditioned with respect to some bases&quot; (which is not completely correct: if your nodes are the roots of unity, for instance, interpolation in the monomial basis is perfectly well conditioned).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, here the interpolation basis is fixed: the function computes the coefficients of the interpolating polynomial with respect to the monomial basis, so that's what we have to return in the end.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2018-01-20T15:00:50.340" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28656" PostTypeId="2" ParentId="28653" CreationDate="2018-01-20T16:00:13.833" Score="2" Body="&lt;p&gt;According with the &lt;a href=&quot;https://it.mathworks.com/help/matlab/ref/polyfit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; polyfit is a function that it can make an approximation in the sense of least square. For this reason you can choose the last input parameter, i.e. the degree. &#xA;Oblivious if you put the degree equals to $N-1$ you are in the interpolation environment (sorry for the pedantic part :-) ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, as Federico's answer explain, the function build the &lt;a href=&quot;https://en.wikipedia.org/wiki/Vandermonde_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vandermonde matrix&lt;/a&gt; and solve the relative linear system (see &lt;a href=&quot;https://it.mathworks.com/help/matlab/ref/polyfit.html#bufchyu-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). This matrix is ill-conditioned and when matlab try to run&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p = V\y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then here come the problem. Note that matlab try to use a $QR$ decomposition to solve the linear system and when the matrix $R$ is ill-conditioned with 1-norm matlab/octave, $||R||_1 ||R^{-1}||_1 &amp;gt; 10^{10}$ , there is the error message. &lt;em&gt;I find this in my course notes in Italian, at the moment I can't get to you a better reference than &lt;a href=&quot;http://www.math.uconn.edu/~leykekhman/courses/MATH3795/Lectures/Lecture_8_Linear_least_squares_orthogonal_matrices.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; (see slide 10).&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid this issue you can use &lt;em&gt;Barycentric Lagrange Interpolation&lt;/em&gt;, see &lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/barycentric.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this pdf&lt;/a&gt; by Berrut and Trefethen. The abstract is a good resume:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Barycentric interpolation is a variant of Lagrange polynomial interpolation that is fast and&#xA;  stable. It deserves to be known as the standard method of polynomial interpolation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can find the respective code, by Greg von Winckel, &lt;a href=&quot;https://it.mathworks.com/matlabcentral/fileexchange/4478-barycentric-lagrange-interpolating-polynomials-and-lebesgue-constant&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2018-01-20T16:00:13.833" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28657" PostTypeId="2" ParentId="28605" CreationDate="2018-01-20T16:19:10.847" Score="1" Body="&lt;p&gt;It turns out that there was some floating point error. The key insight to get around this was that the stationary distribution is invariant to the speed of convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So then I just rescaled &lt;code&gt;A2 = A/x&lt;/code&gt;, for &lt;code&gt;x &amp;gt; 0&lt;/code&gt;. Depending which way the floating point errors went, &lt;code&gt;x&lt;/code&gt; was either above or below &lt;code&gt;1&lt;/code&gt;. Inverting &lt;code&gt;A2&lt;/code&gt; was then possible without problems.&lt;/p&gt;&#xA;" OwnerUserId="9694" LastActivityDate="2018-01-20T16:19:10.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28659" PostTypeId="2" ParentId="28499" CreationDate="2018-01-20T22:35:02.283" Score="1" Body="&lt;p&gt;It's quite normal to have nonlinearities in equations and still want to use a transformation based method. The way that this is commonly done is to backtransform to calculate the nonlinear parts, i.e. if the equation is solved in phase space, evaluate $g(c)$ as $\mathcal{F}[g(\mathcal{F}^{-1}[c])]$. &lt;a href=&quot;http://epubs.siam.org/doi/pdf/10.1137/S1064827502410633&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; displays this in section 4 and shows code like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g.*fft(real(ifft(a)).^2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also use something like &lt;a href=&quot;https://github.com/JuliaApproximation/ApproxFun.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;ApproxFun.jl&lt;/a&gt; to calculate approximations of your nonlinear operators in your function space.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-01-20T22:35:02.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28661" PostTypeId="2" ParentId="28650" CreationDate="2018-01-21T17:26:26.690" Score="0" Body="&lt;p&gt;The number of subsets of size $m$ of a set of size $n$ is $\binom{n}{m}$, which is bounded by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left(\frac{n}{m}\right)^m \le \binom{n}{m} \le \left(\frac{en}{m}\right)^m$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in the limit of $m \ll n$, you can come up with an $O(n^m)$ algorithm by trying all subsets of size $m$, and in the limit of $m \approx n$, you can try complements of all subsets of size $n - m$. For $m$ on the order of $n/2$, however, the brute force approach has a runtime on the order of $2^n$ and I don't think there's any getting around this, i.e. the problem is NP-hard in general.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several heuristics on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Quadratic_knapsack_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki article&lt;/a&gt; about the quadratic knapsack problem; many of them were either designed for the 0-1 problem or can be adapted from the weighted. The article also has a comprehensive list of references, many of which discuss both the weighted and 0-1 cases. You could also try searching github for example codes (&lt;a href=&quot;https://github.com/madcat1991/knapsack&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; one came up right away but it's for the linear knapsack problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple heuristic is to pick an initial candidate set $x$ of size $m$, and improve it by swapping elements. Let $e_i$ be the standard basis vector in the $i$-th direction. To implement this heuristic, pick an index $i$ such that $x_i = 1$, and evaluate the objective function at $x - e_i + e_j$ for all $j$ such that $x_j = 0$. If there are any $j$ such that the exchange $i \leftrightarrow j$ reduces the objective function, pick the one that gives the smallest value of the objective, and then repeat for some other value of $i$. If run all the way to optimality, this could take $O(2^n)$ swaps, but it might give you an acceptable solution in less time. Additionally, the inner loop over $j$ can be parallelized.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2018-01-21T20:07:53.177" LastActivityDate="2018-01-21T20:07:53.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28662" PostTypeId="2" ParentId="28650" CreationDate="2018-01-21T22:21:22.550" Score="1" Body="&lt;p&gt;I don't know if this exactly works for you, but will give the relaxed version a shot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Preliminaries&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Correlation matrix can be seen as the covariance matrix of the standardized random variables $X_{i}/\sigma (X_{i})$. And, any correlation matrix can be converted to a covariance matrix as:&#xA;$$Q={\text{corr}}({X} )=\left({\text{diag}}(\Sigma )\right)^{-{\frac {1}{2}}}\,\Sigma \,\left({\text{diag}}(\Sigma )\right)^{-{\frac {1}{2}}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If standard deviations $\{\sigma_i\}$ are known, then using $D = \sqrt{diag(\Sigma)}$: &#xA;$$&#xA;\Sigma = DQD.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Formulation&lt;/strong&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the definition above we can equivalently write $Q = D^{-1}\Sigma D^{-1}$. If we use this definition of $Q$ and set $B=D^{-1}$, then:&#xA;$$&#xA;\begin{align}&#xA;x^T Q x &amp;amp;= x^T B^T \Sigma B x \\&#xA;&amp;amp;= x^T B^T V^T \Lambda V B x \\&#xA;&amp;amp;= x^T B^T V^T \sqrt{\Lambda}^T\sqrt{\Lambda} V B x \\&#xA;\end{align}&#xA;$$&#xA;$\Sigma=V^T \Lambda V$ follows from the eigen-decomposition and nicely exists for the covariance matrix. Then, substituting $M = B^T V^T \sqrt{\Lambda}^T$, one can show that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x^T Q x = x^T M^T M x = \lVert M x \rVert^2.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem then turns to :&#xA;$$&#xA;\arg\min_x \lVert M x \rVert^2 + \lambda (\|x\|_0-m)&#xA;$$&#xA;which is generally tackled via the $L_1$ variant of the form:&#xA;$$&#xA;\arg\min_x \lVert M x \rVert^2 + \lambda \|x\|_1.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now instead of thinking about $x$ as a binary vector, we could relax this constraint and seek for the sparsest solution under real $x$. This is a non-quadratic $L_1$-regularized Least Squares problem and one can solve it by standard techniques, such as (I guess) &lt;a href=&quot;https://en.wikipedia.org/wiki/Lasso_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lasso&lt;/a&gt;. Maybe it is then possible to look at the solution and determine $m$. This gives local optimality but should work well in practice. For hard constraints Lagrange multipliers can be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I might have to double check the formulation. As I don't have much time now, I'll do it later, but the method above should give you the idea.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2018-02-04T19:29:44.150" LastActivityDate="2018-02-04T19:29:44.150" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28663" PostTypeId="2" ParentId="28652" CreationDate="2018-01-21T23:00:19.240" Score="1" Body="&lt;p&gt;You will need to fit a convex hull to the data (e.g., reflectance as a function of the wavelength).  The SciPy packages has the tool you need, see &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.ConvexHull.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.ConvexHull.html&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that you will need to subtract the convex hull value from the actual value after interpolation along the convex hull, see &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/interpolate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/scipy/reference/interpolate.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2018-01-21T23:00:19.240" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28664" PostTypeId="1" CreationDate="2018-01-22T02:16:20.043" Score="5" ViewCount="186" Body="&lt;p&gt;I’m working with a heat equation of the form. &#xA;$$\frac{d(\rho(T)c_p(T)T)}{dt}-\nabla\cdot(k(T)\nabla T)=f$$ with temperature dependent density $\rho(T)$, specific heat $c_p(T)$, and thermal conductivity $k(T)$.  Ordinarily this would lead to a system of nonlinear equations that would need to be solved.  I was searching for any other way to solve this problem without using a nonlinear solve at every timestep.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found an approach in Reddy &amp;amp; Garling’s &lt;em&gt;The Finite Element Method for Heat Transfer and Fluid Dynamics&lt;/em&gt;.  In it, they take a quasi-linearization approach which attempts to predict the temperature T* at the current timestep ${n+1}$ using values from previous time steps $T_n$, $T_{n-1}$ using the expression: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T^*=\frac{3}{2}T_n-\frac{1}{2}T_{n-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So then, for any timestepping scheme used to discretize the equation, one can evaluate the coefficients of the equation using $T^*$; thus linearizing the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors don’t give a rigorous explanation of when this technique is applicable, only stating that it is appropriate for “mildly nonlinear problems”.  I’m not sure how nonlinearity is quantified in general, much less for heat transfer problems.  I tried searching for literature on this technique, but it is hard to find since the authors don’t really give it a name.  It appears to be some sort of finite difference extrapolation using a 3 point interpolation.  If so, i would think that it would be applicable for coefficients that are roughly linear or quadratic in temperature.  Is there a way i can justify this rigorously?  If not, how can i quantify “mildly nonlinear” in the context of this problem?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2018-01-22T03:04:19.350" LastActivityDate="2018-01-22T03:49:09.793" Title="Quantifying the degree of nonlinearity in a heat transfer problem" Tags="|pde|nonlinear-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28665" PostTypeId="2" ParentId="28664" CreationDate="2018-01-22T03:49:09.793" Score="4" Body="&lt;p&gt;A quick and dirty approach to quantify the nonlinearity could be to evaluate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{1}{k}\frac{dk}{dT}\frac{\partial T}{\partial t}\delta t.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's dimensionless and quantifies how much relative change you can expect in $k$ over a single timestep $\delta t$. The rough idea being that, if the thermal conductivity exhibits a large relative sensitivity to temperature and the temperature is changing a lot in a single timestep, then the error you incur by freezing the coefficients at $T_n$ in order to evaluate $T_{n + 1}$ could be large. By using the extrapolation method, you might be able to effectively increase the order of accuracy -- it's like assuming that the temperature is piecewise linear in time within each timestep rather than piecewise constant. You can always make the problem mildly nonlinear by reducing the timestep enough, but with the extrapolation you might be able to use larger timesteps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know the word for this trick either, which is bad because I've used it. You might be able to Taylor expand like crazy and show that the extrapolation scheme + Crank-Nicholson is formally 2nd-order accurate, even without doing a fully implicit discretization in time. I'm guessing this has already been done to study stability of methods that use a discontinuous Galerkin discretization in time.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-01-22T03:49:09.793" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28666" PostTypeId="1" AcceptedAnswerId="28670" CreationDate="2018-01-22T09:14:06.133" Score="1" ViewCount="547" Body="&lt;p&gt;I have a 3D solver for the incompressible Navier-Stokes equations which uses a FFT library for the Poisson equation with a uniform grid on all directions. In 2D the Poisson equation is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_{xx} + p_{yy} = f_{rhs}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When using a non-uniform grid, we usually map the domain to a computational space where the grid is uniform. Let $x = x(\xi), y = y(\eta)$. The Poisson equation on the computational grid is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_{\xi \xi} \, \xi_{x}^2 + p_{\eta \eta} \, \eta_{y}^2 + p_{\xi}\, \xi_{xx} + p_{\eta} \, \eta_{yy} = f_{rhs}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I do have a background in numerical methods, my knowledge of FFT methods is limited. &lt;strong&gt;Is the Poisson equation on a non-uniform grid solvable with FFT methods?&lt;/strong&gt; If not, what are the alternatives? Note that for reasons that take some time to explain, multigrid and classic iterative methods (e.g. jacobi...) are not an option. Or rather they were shown unstable for certain applications. &lt;/p&gt;&#xA;" OwnerUserId="26633" LastEditorUserId="26633" LastEditDate="2018-01-22T09:34:16.553" LastActivityDate="2018-01-22T17:35:21.373" Title="FFT Poisson Solver for non-uniform grid" Tags="|numerical-analysis|fluid-dynamics|poisson|fftw|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28667" PostTypeId="1" CreationDate="2018-01-22T09:31:42.620" Score="6" ViewCount="4421" Body="&lt;p&gt;If I take the Fourier transform of data &lt;span class=&quot;math-container&quot;&gt;$x \pm \sigma$&lt;/span&gt;, is there a standard approach to what the error in the outputs will be? Would the best way be a direct evaluation of the upper and lower bounds?&lt;/p&gt;&#xA;" OwnerUserId="23709" LastEditorUserId="20688" LastEditDate="2019-06-09T19:09:01.080" LastActivityDate="2023-07-14T19:51:09.020" Title="Error propagation through an FFT" Tags="|error-estimation|fourier-analysis|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="28670" PostTypeId="2" ParentId="28666" CreationDate="2018-01-22T17:35:21.373" Score="2" Body="&lt;p&gt;First, there are non-uniform mesh FFT variations that you could use without having to do the coordinate transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the FFT is not easily applicable to the transformed problem. The reason why the FFT is easily applicable to the Laplace equation without transformation is that (showing this for the 1d case here)&#xA;$$&#xA;  {\cal F}[p_{xx}](k) = -k^2 {\cal F}[p](k),&#xA;$$&#xA;i.e., the derivative simply yields a factor under transformation. Consequently, the equation $-p_{xx}=f$ gives you&#xA;$$&#xA;  k^2 {\cal F}[p](k) = {\cal F}[f](k),&#xA;$$&#xA;and thus&#xA;$$&#xA;  {\cal F}[p](k) = \frac{1}{k^2}{\cal F}[f](k),&#xA;$$&#xA;which is easily invertible to&#xA;$$&#xA;  p(x) = {\cal F}^{-1}\left[\frac{1}{k^2}{\cal F}[f](k)\right](x).&#xA;$$&#xA;But if, on the left hand side, you start with $g(\xi)p_{\xi\xi}(\xi)$, then things are not that simple because&#xA;$$&#xA;  {\cal F}[g(\xi)p_{\xi\xi}(\xi)] \neq k^2 {\cal F}[g](k){\cal F}[p](k)&#xA;$$&#xA;or anything similarly simple that would allow you to isolate ${\cal F}[p](k)$ to one side.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-01-22T17:35:21.373" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28673" PostTypeId="1" AcceptedAnswerId="28674" CreationDate="2018-01-23T01:22:11.737" Score="0" ViewCount="80" Body="&lt;p&gt;I'm trying to build HSL's &lt;a href=&quot;http://www.hsl.rl.ac.uk/catalogue/ma57.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MA57&lt;/a&gt; library, and to do so, it requires &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/fsroot/sw/metis/OLD&quot; rel=&quot;nofollow noreferrer&quot;&gt;METIS&lt;/a&gt;. I've downloaded the .tar files to my &lt;code&gt;Downloads&lt;/code&gt; folder, but where do I actually &quot;build&quot; the libraries (and subsequently store the compiled files) on a Mac? Is there a best practices location, like &lt;code&gt;usr/local&lt;/code&gt;? &lt;/p&gt;&#xA;" OwnerUserId="22292" LastActivityDate="2018-01-23T03:21:33.097" Title="standard directory for compiling scientific libraries" Tags="|compiling|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28674" PostTypeId="2" ParentId="28673" CreationDate="2018-01-23T03:09:14.390" Score="1" Body="&lt;p&gt;I don't think there's any one right answer here. I tend to put non-system libraries in /opt, then add link directives for them manually to the Makefiles etc of whatever apps are looking for them (eg, add &lt;code&gt;-L/opt/path/to/metis&lt;/code&gt; and &lt;code&gt;-lmetis&lt;/code&gt; to the Makefile of MA57). It's a compromise between putting it somewhere easy to find, but trying to be hygienic and keep your installs out of the system packages. This is probably a little old school, so ymmv.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also, &lt;a href=&quot;https://unix.stackexchange.com/questions/11544/what-is-the-difference-between-opt-and-usr-local&quot;&gt;https://unix.stackexchange.com/questions/11544/what-is-the-difference-between-opt-and-usr-local&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-01-23T03:21:33.097" LastActivityDate="2018-01-23T03:21:33.097" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28675" PostTypeId="2" ParentId="28640" CreationDate="2018-01-23T05:16:07.363" Score="3" Body="&lt;p&gt;Nice summary of HSL_MA77 vs MA57 taken from &lt;a href=&quot;http://www.maths.ed.ac.uk/sites/default/files/atoms/files/hogg.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (p.45) but note that it's different from the original Q:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;HSL MA77 [Reid and Scott 2008; 2009b] is also a multifrontal solver that is designed to solve&#xA;  positive definite and indefinite sparse symmetric systems. A different inner kernel is used in each&#xA;  case to achieve the best performance. The fundamental difference between MA57 and HSL MA77&#xA;  is that the latter is an out-of-core solver capable of holding all the data out of core, enabling&#xA;  the solution of much larger problems. Further, care has been taken to allow the addressing of&#xA;  fronts with 64-bit integers. This is essential as some problems require the factorization of dense&#xA;  matrices containing this many elements. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;It exploits the virtual memory system of Reid and Scott [2009a] to minimize overheads due&#xA;  to the out-of-core approach while remaining robust. The ability to work in core is also available,&#xA;  though some overhead from the out-of-core design remains. Despite this, on problems that MA57&#xA;  is able to solve, the performance of HSL MA77 is favourable in the factorization phase, though&#xA;  the solve phase can be comparatively slow.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;While full control over the factorization is still available, the large range of support routines&#xA;  that come with MA57 are not present. The user is expected to supply their own ordering and&#xA;  scaling. This reflects the limited availability of subroutines capable of performing these routines&#xA;  in an out-of-core fashion for large problems.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22292" LastEditorUserId="22292" LastEditDate="2018-02-05T23:24:53.213" LastActivityDate="2018-02-05T23:24:53.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28676" PostTypeId="1" AcceptedAnswerId="29254" CreationDate="2018-01-23T16:32:39.467" Score="0" ViewCount="2879" Body="&lt;p&gt;I am trying to use the numpy polyfit method to add regularization to my solution.&#xA;My non-regularized solution is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coefficients = np.polynomial.polynomial.polyfit(x,y,5)&#xA;ypred = np.polynomial.polynomial.polyval(x,coefficients)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How would I modify this to add L2-regularization?&lt;/p&gt;&#xA;" OwnerUserId="26645" LastActivityDate="2018-04-07T19:31:59.353" Title="Numpy.polyfit with regularization" Tags="|python|interpolation|numpy|regression|curve-fitting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28678" PostTypeId="1" CreationDate="2018-01-23T23:31:15.613" Score="7" ViewCount="181" Body="&lt;p&gt;Define the element-wise thresholding operator $T_\tau(\cdot)$ with threshold $\tau$ as &#xA;$$ [T_\tau(X)]_{i,j} = \begin{cases} X_{i,j} &amp;amp;\mbox{if } |X_{i,j}| \ge \tau, \\ &#xA;0 &amp;amp; \mbox{if } |X_{i,j}| &amp;lt; \tau. \end{cases} $$&#xA;Clearly, $T_\tau(X)$ can be computed in quadratic $O(n^2)$ time for some $n\times n$ matrix $X$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Suppose that the thresholded matrix $T_\tau(X)$ contains only $O(n)$ nonzero elements. Is it possible to compute $T_\tau(X)=T_\tau(UV^T)$ in linear $O(n)$ time (on a serial computer) from a low-rank factorization $U,V\in\mathbb{R}^{n\times r}$ of $X=UV^T$? (Take the rank $r$ to be an absolute constant, i.e. $r=O(1)$)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some insights:&lt;/strong&gt; Let the rows of $U$ and $V$ be written as $u_{1},\ldots,u_{n}$&#xA;and $v_{1},\ldots,v_{n}$, as in&#xA;$$&#xA;U=\begin{bmatrix}u_{1}^{T}\\&#xA;u_{2}^{T}\\&#xA;\vdots\\&#xA;u_{n}^{T}&#xA;\end{bmatrix},\qquad V=\begin{bmatrix}v_{1}^{T}\\&#xA;v_{2}^{T}\\&#xA;\vdots\\&#xA;v_{n}^{T}&#xA;\end{bmatrix}.&#xA;$$&#xA;Then we have $X_{i,j}=u_{i}^{T}v_{j}$. By Hölder's inequality, we&#xA;have&#xA;$$&#xA;|u_{i}^{T}v_{j}|\le\|u_{i}\|_{p}\|v_{j}\|_{q}\qquad\frac{1}{p}+\frac{1}{q}=1.&#xA;$$&#xA;So it seems like we can just compute the $p$-norm of all rows of&#xA;$U$ and $V$ in $O(n)$ time, and threshold the rows directly. However,&#xA;this approach seems to be extremely conservative.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2018-01-24T17:59:58.720" LastActivityDate="2019-05-12T07:35:17.007" Title="Element-wise thresholding a low-rank matrix in O(n) time?" Tags="|numerical-analysis|algorithms|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28679" PostTypeId="2" ParentId="28567" CreationDate="2018-01-24T07:46:54.653" Score="1" Body="&lt;p&gt;As previously mentioned, you can't &lt;em&gt;technically&lt;/em&gt; solve a problem on an infinite domain. However, you &lt;strong&gt;can&lt;/strong&gt; solve a problem over a domain that is sufficiently large enough to approximate the conditions you would find at infinity- that is the spacial gradient at your boundary is numerically zero (Dirichlet boundary condition). This is known as the &quot;farfield&quot; boundary. Since you are just starting out, just add more space away from your point of interest and see how your solution changes. &lt;/p&gt;&#xA;" OwnerUserId="26650" LastActivityDate="2018-01-24T07:46:54.653" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28681" PostTypeId="1" CreationDate="2018-01-24T15:41:47.590" Score="5" ViewCount="835" Body="&lt;p&gt;I’m trying to find a Runge–Kutta integrator that only requires the absolute minimum of storage, i.e. it fulfills one of the following three, presumably equivalent criteria:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Each evaluation of the derivative should only depend on the previous stage.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It stores only store three vectors, namely the beginning of the step, an intermediate temporary copy, and the end of the step, which is incrementally updated each stage (the temporary copy can then be overwritten in each stage).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It has a “diagonal Butcher tableau”.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The classical fourth-order Runge–Kutta method (RK4) is an example of this, but I am struggling to find such methods for higher orders. Do they exist?&lt;/p&gt;&#xA;" OwnerUserId="26654" LastEditorUserId="123" LastEditDate="2018-01-25T07:40:23.307" LastActivityDate="2018-01-26T16:36:13.680" Title="Do there exist low-storage Runge–Kutta methods with an order larger than four?" Tags="|ode|time-integration|runge-kutta|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28682" PostTypeId="1" CreationDate="2018-01-24T18:30:45.033" Score="3" ViewCount="777" Body="&lt;h1&gt;1d diffusion equation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Integrating the diffusion equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} = D \frac{\partial^2 u}{\partial x^2},&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with a constant diffusion coefficient D using forward Euler for time and the finite difference approximation for space,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_i^{t+1} = u_i^t + D\frac{\Delta t}{\Delta x^2} ( u_{i+1}^{t} + u_{i-1}^{t} - 2u_i^{t} ),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;leads to the conservation of $\bar{u}=\sum_i \Delta x\, u_i$ over time (see animation 1 and figure 1), because reflective Neumann boundary conditions,$\partial_x u=0$, are employed at the borders (forward differences: $u_i = u_{i \pm 1}$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_i^{t+1} = u_i^t + D\frac{\Delta t}{\Delta x^2} \cdot ( u_{i\pm1}^{t} - u_{i}^{t} )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wUA8v.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wUA8v.gif&quot; alt=&quot;animation_1&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1yueY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1yueY.png&quot; alt=&quot;figure_2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Space and time are discretized with $\Delta x=0.01$ and $\Delta t=10^{-7}$. The diffusion coefficient is $D=10$.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;radial diffusion equation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In 2d polar coordinates $(r,\phi)$, the Laplacian is given by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^2 u = \frac{\partial^2 u}{\partial r^2} + \frac{1}{r}\frac{\partial u}{\partial r} + \frac{1}{r^2} \frac{\partial u^2}{\partial \phi^2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of an axisymmetric distribution $u(r,\phi)=u(r)$, the Laplacian reduces to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^2 u = \frac{\partial^2 u}{\partial r^2} + \frac{1}{r}\frac{\partial u}{\partial r}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forward Euler and finite difference approximation with central differences for the advective term leads to (&lt;a href=&quot;http://homepages.see.leeds.ac.uk/~amt6xw/Distance%20Learning/CFD5030/node10.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;derivation&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_i^{t+1} = u_i^t + D\frac{\Delta t}{\Delta r^2} \big( u_{i+1}^{t} +  u_{i-1}^{t} - 2u_i^{t} \big) + D\frac{\Delta t}{2 \Delta r} \big( u_{i+1}^{t} - u_{i-1}^{t} \big) \\&#xA;= u_i^t + D\frac{\Delta t}{\Delta r^2} \big( (1+0.5/i)u_{i+1}^{t} + (1-0.5/i)u_{i-1}^{t} -2u_i^{t} \big).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same approximation is obtained with the Finite Volume Method (&lt;a href=&quot;https://www.particleincell.com/2011/finite-volume/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;link&quot;&gt;derivation&lt;/a&gt;). The boundary condition at the origin exploits the rotational symmetry and thus is $\partial_r u = 0$, leading to (central differences: $u_{i-1}=u_{i+1}$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_i^{t+1} = u_i^t + D\frac{\Delta t}{\Delta r^2} \cdot 4( u_{i+1}^{t} - u_i^{t} )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the other boundary, Neumann conditions $\partial_r u = 0$ are employed and realized as (central differences: $u_{i-1}=u_{i+1}$): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_i^{t+1} = u_i^t + D\frac{\Delta t}{\Delta r^2} \cdot 2( u_{i-1}^{t} - u_{i}^{t} )&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is a problem: Conservation of u is violated (see animation 2 and figure 2). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eQg4F.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eQg4F.gif&quot; alt=&quot;animation_2&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CIKmt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CIKmt.png&quot; alt=&quot;figure_2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over time the total amount of u grows as calculated via &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;2\pi \int_\Omega u(r) r dr = 2\pi \sum_{i=0}^{n} u_i\; (i \,\Delta r) \; \cdot \; \Delta r,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $(i \Delta r)$ is the discretized volume element from 2d polar coordinates. The dashed line gives the analytical result and the regular line the numerical result. Numerical parameters $\Delta r=\Delta x$, $\Delta t$ and $D$ are as before.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Why is conservation of u violated when going from the 1d system to the 1d radial (reduced from 2d polar coordinates)? What can I do to regain the conservation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite volume methods lead to the same discretization scheme as shown above. Robin boundaries (as used for conservation in advection equations) are not applicable, since the flux $j$ at the boundaries is given by $j=r\partial_r u$, which leads to the already employed Neumann boundary conditions $\partial_r u=0$ at $r=0$ and $r=r_{end}$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Edit:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;While testing a few things, I came up with an MWE in c++ for others interested to try. Compilation instructions are given at the top. It reproduces the conservation violation problem :(&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// test program to check conservation in radial diffusion&#xA;&#xA;// compilation: g++ -Wall radial_diffusion.cpp -std=c++11 -fopenmp -O3 -o radial_diffusion.exe&#xA;&#xA;&#xA;#include &amp;lt;fstream&amp;gt;                              // std::ofstream&#xA;#include &amp;lt;string&amp;gt;                               // std::string&#xA;#include &amp;lt;cmath&amp;gt;                                // exp&#xA;&#xA;&#xA;&#xA;int main(){&#xA;&#xA;    // initialize variables&#xA;    std::string name = &quot;/tmp/u_sum.dat&quot;;&#xA;    int nx = 1000;&#xA;    size_t nsteps=100000;&#xA;    double d = 10;&#xA;    double dr = 0.01;&#xA;    double dt = 1e-7;&#xA;    double diffcoeff = d*dt/(dr*dr);&#xA;    double *u = new double[nx];&#xA;    double *unew = new double[nx];&#xA;    double *usum = new double[nsteps]();&#xA;    double usum0;&#xA;&#xA;    // initial condition&#xA;    float mu_x = 0*nx;  // mean x&#xA;    float xsigma = 0.01*nx; // variance x&#xA;    for(int x=0; x&amp;lt;nx; x++) u[x] = exp(-0.25*((x-mu_x)*(x-mu_x)/(xsigma*xsigma)));&#xA;&#xA;    // time evolution&#xA;    for(size_t step=0; step&amp;lt;nsteps; step++){&#xA;&#xA;        #pragma omp parallel for&#xA;        for(int x=0; x&amp;lt;nx; x++){&#xA;&#xA;            int left=x-1;&#xA;            int right=x+1;&#xA;&#xA;            // calculation, central difference with lHospital at r-&amp;gt;0 with du/dr -&amp;gt; 0&#xA;            if(x==0){&#xA;                unew[x] = u[x] + diffcoeff*( 4*(u[right] - u[x]) );&#xA;            }else if(x==nx-1){&#xA;                unew[x] = u[x] + diffcoeff*( 2*(u[left] - u[x]) );&#xA;            }else{&#xA;                unew[x] = u[x] + diffcoeff*( u[right]*(1+0.5/x) + u[left]*(1-0.5/x) - 2*u[x] );&#xA;            }&#xA;        }&#xA;&#xA;        // sum up to check conservation&#xA;        usum0 = 0;&#xA;        for(int x=0; x&amp;lt;nx; x++) usum0 += 2*M_PI*dr*dr*x*unew[x];&#xA;        if(!(step%10000)) printf(&quot;%12.12f\n&quot;,usum0);&#xA;&#xA;        for(int x=0; x&amp;lt;nx; x++) usum[step] += 2*M_PI*dr*dr*x*unew[x];&#xA;&#xA;        // update u,unew&#xA;        std::swap(u,unew);&#xA;    }&#xA;&#xA;    // save stuff&#xA;    FILE *fp;&#xA;    if((fp=fopen(name.c_str(), &quot;w&quot;))==NULL){ printf(&quot;Cannot open file.\n&quot;); }&#xA;    for(size_t step=0; step&amp;lt;nsteps; step++) fprintf(fp,&quot;%12.12f\n&quot;,usum[step]);&#xA;    fclose(fp);&#xA;&#xA;&#xA;    // free memory&#xA;    delete[] u;&#xA;    delete[] unew;&#xA;    delete[] usum;&#xA;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In the paper &quot;High-order schemes for cylindrical/spherical&#xA;geometries with cylindrical/spherical symmetry&quot; by Wang et al. (2013),&#xA;the authors propose the following scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{d u_i}{dt} = \frac{1}{\Delta V} (r_{i+\frac{1}{2}}j_{i+\frac{1}{2}} - r_{i-\frac{1}{2}}j_{i-\frac{1}{2}}), \\&#xA;\Delta V = \frac{1}{2}(r^2_{i+\frac{1}{2}} - r^2_{i-\frac{1}{2}}), \\&#xA;j_i = \nabla u_i = \frac{1}{2\Delta r}(u_{i+1} - u_{i-1}).&#xA;$$&#xA;The half index, $i+\frac{1}{2}$, implies an arithmetic mean. Is this scheme applicable here? Is the gradient discretized correctly?&lt;/p&gt;&#xA;" OwnerUserId="25135" LastEditorUserId="25135" LastEditDate="2018-01-29T22:04:27.487" LastActivityDate="2018-01-30T04:12:39.913" Title="Conservation violation in axisymmetric Diffusion Equation" Tags="|finite-difference|diffusion|conservation|symmetry|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28683" PostTypeId="2" ParentId="28681" CreationDate="2018-01-24T19:30:22.427" Score="10" Body="&lt;p&gt;This area has been fairly well researched, you may check e.g. Ketcheson's review of such methods: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1016/j.jcp.2009.11.006&quot; rel=&quot;noreferrer&quot;&gt;https://doi.org/10.1016/j.jcp.2009.11.006&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which does contain some low-storage Runge-Kutta methods for fifth and sixth orders.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2018-01-24T19:30:22.427" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28684" PostTypeId="1" CreationDate="2018-01-24T22:01:55.450" Score="3" ViewCount="53" Body="&lt;p&gt;Mallat's pyramidal algorithm for the discrete wavelet transform operates on power-of-2 vector lengths. Can it be extended to work on inputs of any size without resorting to zero padding?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2018-10-31T16:31:12.467" LastActivityDate="2018-10-31T16:31:32.700" Title="Can Mallat's pyramidal algorithm be extended to non-power of 2 input sizes?" Tags="|signal-processing|wavelet|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28686" PostTypeId="1" CreationDate="2018-01-25T02:04:34.837" Score="1" ViewCount="62" Body="&lt;p&gt;I understand the idea of constructing Low-pass and High-pass filters as a projection on the Numerical Range and Numerical Kernel of dyadic powers of a diffusion operator  in the work &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S1063520306000558&quot; rel=&quot;nofollow noreferrer&quot;&gt;Diffusion Wavelet Packets&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be represented as a &lt;a href=&quot;http://www.math.jhu.edu/~mauro/Images/DiffusionWavelets/diffusionwavelets_html_c200bca.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;tree&lt;/a&gt; and is implemented here:&#xA; &lt;a href=&quot;https://github.com/aweinstein/dw/blob/master/Wavelets/DWPTree.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aweinstein/dw/blob/master/Wavelets/DWPTree.m&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand how to split each $W_{j}$. To my understanding I could make another QR decomposition, as is done for the $V_{j}$ subspace at level $J$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors say (in the end of section 4 of the above mentioned article) to choose each $W_{j}$'s children as having the same dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is: they don't explain why and how to do this splitting, so I started to read the code on github.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In line 404, using a handle to function DefaultSplitFcn, I found a method nowhere explained in articles or lectures about diffusion wavelets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the researches' words (line 503, DefaultSplitFcn function):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Averages the endpoints of the approximate frequency range of the node&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions are: why can I do this?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16782" LastActivityDate="2018-01-25T02:04:34.837" Title="How to construct Diffusion Wavelet Packets?" Tags="|wavelet|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28687" PostTypeId="1" CreationDate="2018-01-25T02:35:29.560" Score="2" ViewCount="156" Body="&lt;p&gt;I'm trying to numerically integrate a multi-dimensional expression. The integrand is complicated; for example this is the integrand for $N=4$:&#xA;$$\begin{aligned}&amp;amp;x_1^6x_2^5x_3^3x_4^2(x_1-x_1x_2)(x_1-x_1x_2x_3)(x_1-x_1x_2x_3x_4)\times{}\\&amp;amp;{}\times(x_1x_2-x_1x_2x_3)(x_1x_2-x_1x_2x_3x_4)(x_1x_2x_3-x_1x_2x_3x_4)\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I'm most interested in expressions that have even larger $N$. The integration limits in my trial case are $x_1$ from 0 to 10, and everything else from 0 to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To numerically integrate this I'm using Mathematica while double checking with the VEGAS integrator in C++ (&lt;a href=&quot;https://www.gnu.org/software/gsl/manual/html_node/Monte-Carlo-Examples.html#Monte-Carlo-Examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;via the GNU scientific library&lt;/a&gt;). The problem is the two integrators match for lower dimensions but not for higher ones. The agreement is great for $N =2, 3$, but for $N=4, 5$ there's a considerable discrepancy of about a factor of 5 that gets worse with higher $N$. Agreement doesn't fail completely; the results are still of the same order of magnitude, but a factor of 5 is too large for comfort especially with both integrators reporting error bars much lower than this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unfamiliar with the internals of both integrators and how numerical integration in higher dimensions works. &lt;strong&gt;Is there a (non-bug) reason for why the two integrators give the same results at low dimensions but not at higher dimensions?&lt;/strong&gt; If the only possible explanation is &quot;a bug in your code&quot; that's fine, but if there's some other possibility I'd like to check that out too.&lt;/p&gt;&#xA;" OwnerUserId="26660" LastEditorUserId="26660" LastEditDate="2018-01-25T06:47:02.977" LastActivityDate="2018-01-25T06:47:02.977" Title="What could be causing multi-dimensional numerical integration inconsistency?" Tags="|numerical-analysis|c++|integration|mathematica|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="28688" PostTypeId="2" ParentId="28681" CreationDate="2018-01-25T07:45:45.853" Score="7" Body="&lt;p&gt;Another good starting point is the excellent paper of Kennedy, Carpenter, &amp;amp; Lewis (KCL2000):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1016/S0168-9274(99)00141-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1016/S0168-9274(99)00141-5&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own paper focuses more on the mechanics of low-storage implementation and what it implies for method construction, while KCL2000 is heavily focused on optimization and testing of methods. They give an extensive investigation of methods up to fifth order.  Very little has been done regarding low-storage explicit RK methods of higher than fifth order (my own paper, linked in the other answer, is an exception).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will add that there is a better way to implement RK4 in low storage than the obvious one, and every 4th-order RK4 method has a low-storage implementation (3 registers).  These things are explained in detail in references provided in the introduction of my paper already linked by @reid.atcheson.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2018-01-26T16:36:13.680" LastActivityDate="2018-01-26T16:36:13.680" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28689" PostTypeId="2" ParentId="28682" CreationDate="2018-01-25T08:52:41.770" Score="0" Body="&lt;p&gt;Your diffusive equation leads always to the conservation of energy in your spatial domain if Neumann  BC are imposed. This is natural because there is no heat flux through walls (analogy to heat equation). Therefore integrating our equation whatever coordinate system is used leads to this conservation. For example in cylindrical coordinates:&#xA;$$\frac{d}{d t} \int_{\Omega}{ur\,dr}= r\frac{\partial u}{\partial r}\hat{n}\bigg|_{\partial \Omega}=0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finite element/volume schemes in 1D leads to finite differences schemes. Make sure it coincides with:&#xA;$$\frac{\partial u_i}{\partial t}=\frac{D}{\Delta r^2 }[u_{i-1}-2u_{i}+u_{i+1}+\frac{\Delta r}{2r_i}(u_{i+1}-u_{i-1})]  \qquad i=2,...,N-1$$&#xA;And for the first and last nodes:&#xA;$$\frac{\partial u_i}{\partial t}=\frac{2D}{\Delta r^2}(u_{i+1}-u_i) \qquad i=1$$&#xA;$$\frac{\partial u_i}{\partial t}=-\frac{2D}{\Delta r^2}(u_{i}-u_{i-1})\qquad i=N$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-01-25T09:08:15.377" LastActivityDate="2018-01-25T09:08:15.377" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28690" PostTypeId="1" AcceptedAnswerId="28699" CreationDate="2018-01-25T14:47:22.850" Score="4" ViewCount="162" Body="&lt;p&gt;As far as I can tell, nodal interpolation estimates proven with Bramble / Hilbert require higher regularity of the functions being interpolated. E.g. with linear elements in 2 dimensions, one needs $v \in W^{m,p}$ with $m p &amp;gt; 2, p \geq 1$ in order to have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ |v - I(v)|_{W^{r,p}} \leq C h^{m-r} |v|_{W^{m,p}}, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $I$ is the nodal interpolant, $| \cdot |$ are seminorms in the indicated spaces and $0 \leq r \leq m$. In particular note that the bound is using the $m$-th derivative of $v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my application I only have $m = 1$ and $p \leq 2$. More specifically this is what I want:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $V_h$ be a continuous $P_1$ space over a polygonal domain $U \subset \mathbb{R}^2$ with a quasi-uniform mesh, and $(v_h) \subset V_h$ a sequence bounded in $W^{1,2}(U)$. For each $h$ let $I_h:C^0 \rightarrow V_h$ be &lt;em&gt;some&lt;/em&gt; interpolant (I'd use the nodal interpolant but I'm not working with $W^{2,2}$) and $q$ some quadratic function. To simplify maybe take $q(x)=x^2$. I want to show&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \|q(v_h) - I_h(q(v_h))\|_{L^2(U)} \leq C h^\alpha \|D q(v_h) \|_{L^2(U)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for any $\alpha &amp;gt; 0$ which makes this work. Reading some papers, I see people change $V_h$ to be discontinuous and claim to have bounds using just the first derivative of $q \circ v_h$, but I don't know where to find precise statements and proofs of the properties of the interpolant in this case. I'm happy to make $V_h$ discontinuous, I just want the damn thing to converge :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help, anyone?&lt;/p&gt;&#xA;" OwnerUserId="3872" LastEditorUserId="3872" LastEditDate="2018-01-26T14:11:14.340" LastActivityDate="2018-01-26T14:51:49.803" Title="Interpolation estimates for $H^1$ into $P_1$" Tags="|finite-element|interpolation|error-estimation|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28691" PostTypeId="2" ParentId="28651" CreationDate="2018-01-25T18:48:05.357" Score="2" Body="&lt;p&gt;Posting a partial answer in hopes of inspiring a full one. For simplicity take $f(x)=x$. Reparameterize to $\xi\sim N(0,I)$. We are then interested in the quality of this approximation:&#xA;$$&#xA;\nabla V(\theta)\approx \nabla \mathbb{E}V(\theta+\epsilon \xi)&#xA;$$&#xA;Now, add an extra assumption that $V$ is $\beta$-smooth. Then we have enough regularity to interchange differentiation with integration:&#xA;$$&#xA;\nabla \mathbb{E}V(\theta)+\epsilon\xi)= \mathbb{E}\nabla V(\theta+\epsilon \xi)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that we assume that we can estimate the above term without access to a gradient oracle for $V$ with the score estimator and Monte Carlo integation. Now, note that by smoothness:&#xA;$$&#xA;\|\nabla V(\theta+\epsilon \xi)-\nabla V\|\le \epsilon_1\beta\|\epsilon_2\xi\|&#xA;$$&#xA;We're going to need the split $\epsilon_1\epsilon_2=\epsilon$. Then for any $r&amp;gt;0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbb{E}\nabla V(\theta+\epsilon \xi)=\nabla V(\theta)(1+\varphi_1)(1-\mathbb{P}\{\|\xi\|&amp;gt; r/\epsilon_2\})+\boldsymbol \varphi_2&#xA;$$&#xA;Where we can characterize all our error terms. $\mathbb{P}\{\|\xi\|&amp;gt; r/\epsilon_2\}\le O(e^{-r/\epsilon_2})$ by a $\chi_n$ &lt;a href=&quot;https://stats.stackexchange.com/questions/4816/what-are-the-sharpest-known-tail-bounds-for-chi-k2-distributed-variables&quot;&gt;tail bound&lt;/a&gt; for any $r$ as $\epsilon_2\rightarrow 0$. Next $\left|\varphi_1\right|=O(r\epsilon_1\beta)$. Finally with $\zeta=\|\xi\|\sim\chi_n$:&#xA;$$&#xA;\|\boldsymbol \varphi_2\|=\int_r^\infty d\zeta\, p(\zeta)\sup_{\|{\xi}\|=\zeta}\nabla V(\theta+\epsilon \xi)\le \|\nabla V(\theta)\|\int_r^\infty d\zeta\, p(\zeta)(1+\beta \epsilon \zeta)&#xA;$$&#xA;Re-using the same $\chi_n$ tail bound we get $\|{\boldsymbol\varphi_2}\|=O(e^{-r})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some tension because increasing $r$ reduces $\boldsymbol \varphi_2$ error at the cost of $\varphi_1$ error, but the epsilon-splitting trick with should save us. E.g., to get $\ell_2$ gradient error below $O(\alpha)$ choose $r = \Theta(-\log\alpha)$ and $\epsilon_1=\Theta(\alpha/(-\log\alpha))$ and finally $\epsilon_2=\Theta(1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is probably very loose: the analysis was lazy and I dropped a lot of lower order terms. Moreover, I didn't optimize the bounds. But at the very least it seems that we can get $O(\epsilon (-\log \epsilon))$ (relative!) accuracy. This is worse than symmetric finite differences, but we didn't use second order information here. Integrating a first-order Taylor series of $\nabla V$ about $\theta$ would probably make the error absolute rather than relative and explicitly depend on $\beta$.&lt;/p&gt;&#xA;" OwnerUserId="23164" LastEditorUserId="23164" LastEditDate="2018-01-29T21:34:29.687" LastActivityDate="2018-01-29T21:34:29.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28692" PostTypeId="1" CreationDate="2018-01-25T20:22:01.717" Score="3" ViewCount="128" Body="&lt;p&gt;I just need to calculate the largest real part of eigenvalues of a Jacobian which is highly non-normal and singular. Most of the eigenvalues are negative, and some of them are positive but near to zero relatively. I want to find the largest one except the zeroes. And I do not need the eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are several zero eigenvalues. So I want to know if it is worthy to do some transformation to cancel the known zero eigenmode (similar transform to the null space of the modes). Or I can choose to calculate the first few eigenvalues directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the sample matlab code to test how much is the cost to calculate first K eigenvalue with largest real part of a random matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N=3000; % N hundreds to thousand&#xA;M=rand(N,N)+eye(N); % shift to avoid zero&#xA;K=8; % K is less than 10&#xA;t = zeros(K,1);&#xA;for k=1:K&#xA;    f=@()(eigs(M,k,&quot;largestreal&quot;));&#xA;    t(k) = timeit(f);&#xA;end&#xA;figure&#xA;plot(t)&#xA;ylabel(&quot;time(s)&quot;)&#xA;xlabel(&quot;k&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the plot output is wavy and non-monotonic. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any analysis on this problem? I read the ARPACK's document, there is no detail of how the it is implemented.&lt;/p&gt;&#xA;" OwnerUserId="24589" LastActivityDate="2018-01-25T20:22:01.717" Title="What is the complexity of calculating K-th largest real part eigenvalue of non-normal sparse matrix" Tags="|eigenvalues|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28693" PostTypeId="5" CreationDate="2018-01-25T22:43:51.970" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-01-25T22:43:51.970" LastActivityDate="2018-01-25T22:43:51.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28694" PostTypeId="4" CreationDate="2018-01-25T22:43:51.970" Score="0" Body="CVXPY is a Python-embedded modeling language for convex optimization problems." OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2018-01-26T03:26:38.123" LastActivityDate="2018-01-26T03:26:38.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28696" PostTypeId="2" ParentId="28682" CreationDate="2018-01-26T03:11:58.493" Score="3" Body="&lt;p&gt;You may want a method that works right down to the coordinate singularity at $r=0$. I will do the spherical case, but the cylindrical case is similar. We want to avoid ever dividing by $r$. To think of a Finite Volume method, consider the control volumes to be concentric shells.  The inner and outer radii of shell i are $(r_i-\Delta r/2, r_i+\Delta r/2)$, containing an amount of heat $$4\pi\left((r_i+\Delta r/2)^3-(r_i-\Delta r/2)^3\right)u_i=4\pi\Delta r(3r_i^2+\Delta r^2/4)&#xA;\;u_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now take the first &quot;shell&quot; to have $r_0=\Delta r/2$ so that it is actually a little sphere of radius $\Delta r$, whose volume is given correctly by $4\pi\Delta r(3\Delta r^2/4+\Delta r^2/4)=4\pi\Delta r^3$.&#xA;The control volumes are separated by interfaces that have surface area&#xA;$$S_{i+1/2}=4\pi/3(r_i+\Delta r/2)^2$$ carrying heat fluxes $D(u_{i+1}-u_i)/\Delta r$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should find that this all works out. The secret of conservation in non-Cartesian coordinates is to to define carefully the non-overlapping control volumes and then to do the geometry without any approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADDITION&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that this, and some related topics, are not well treated in the literature,so I give my version. You always have conservation in some approximate sense, if you solve the equations consistently. But as a guard against accumulation of secular errors, we usually require conservation in some exact sense. That is, if you specify the quadrature rule that you will use to check for conservation, it is guaranteed to hold exactly (It might not hold for a different quadrature) This property makes no reference to taking any infinitesimal limit. It is a property of the discrete method alone. It is a mistake to derive it by going to the limit (writing a pde) and then moving away from the limit. The limit is not involved, so dont go there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FURTHER ADDITION&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(Volume)x(New \;average\; conserved \;value)=(Volume)x(Old\; average \;conserved \;variable) = \Sigma(Rate\; of \;flux \;through \;faces)$$&#xA;This works for arbitrary control volumes in any number of dimensions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to see the importance of exact geometry is to consider a uniform flow, that should not change. The flux is a constant vector, and the RHS is&#xA;$$\Sigma(Rate\;of\;flow\;through\;surfaces)=\Sigma(Flux \;vector)x(Normal\;to\; Surface) = (Flux Vector)x\Sigma(Normal\;to\;Surface)$$&#xA;If the geometry is calculated exactly then $\Sigma(Normal\; to\; Surface)$ vanishes for any closed body and there will be no changes. But if the body does not close there will be changes. This is a rather drastic loss of conservation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have control volumes that move and distort, but you need to think of this in four dimensions.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastEditorUserId="23873" LastEditDate="2018-01-30T04:12:39.913" LastActivityDate="2018-01-30T04:12:39.913" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28698" PostTypeId="1" CreationDate="2018-01-26T11:26:55.583" Score="4" ViewCount="175" Body="&lt;p&gt;In the problem I am dealing with, I require to repeatedly solve $Ax=b$ where $A$ is a weighted Laplacian matrix of a sparse graph. The right-hand side remains constant. However each time I solve the system, only &lt;strong&gt;one&lt;/strong&gt; weight of the graph changes, effectively changing 4 coefficients in the Laplacian matrix (2 diagonal and 2 off-diagonal entries). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently using the GMRES solver in Petsc, and using the previous solution as the initial guess. However the coefficient changes by 1 or 2 orders of magnitude, and it is not as fast as if the coefficient were changing slightly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is anyway to solve this problem any faster than what I have been currently doing. Perhaps something which takes advantage of the linear nature of the problem and involves a direct method.&lt;/p&gt;&#xA;" OwnerUserId="16379" LastEditorUserId="16379" LastEditDate="2018-01-26T12:55:51.330" LastActivityDate="2018-01-27T09:05:29.123" Title="Solve $Ax=b$ repeatedly where $A$ is a sparse weighted Laplacian matrix with changing weights" Tags="|sparse-matrix|iterative-method|linear-system|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28699" PostTypeId="2" ParentId="28690" CreationDate="2018-01-26T11:32:39.393" Score="1" Body="&lt;p&gt;What I was looking for is Clément interpolation, which is defined by averaging a target function $v \in L^1$ over cells sharing a node and assigning this value to the interpolant at that node (or equivalently by local $L^2$ projections). It is explained in detail in §6.3 of &lt;a href=&quot;https://www.wias-berlin.de/people/john/LEHRE/NUM_PDE_FUB/num_pde_fub_6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these lecture notes&lt;/a&gt; of &lt;a href=&quot;https://www.wias-berlin.de/people/john/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Volker John at the WIAS&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, the estimate obtained for $v \in W^{1,q}$ is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \| D^k(v - I_h^{\operatorname{cle}}(v)) \|_{L^q} \leq C h^{l-k} \| D^l v\|_{L^q}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for all $0 \leq k \leq l \leq 2 $ and $1 \leq q \leq \infty$, assuming a nice enough mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, I guess I could just use &lt;em&gt;global&lt;/em&gt; $L^2$ projection instead of interpolation, which guarantees the convergence I want, but I need to use the same operator in other places and I don't want my code to be performing costly projections.&lt;/p&gt;&#xA;" OwnerUserId="3872" LastEditorUserId="3872" LastEditDate="2018-01-26T14:51:49.803" LastActivityDate="2018-01-26T14:51:49.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28700" PostTypeId="1" CreationDate="2018-01-26T12:21:39.250" Score="1" ViewCount="1669" Body="&lt;p&gt;Does anyone have any suggestions for good open source plotting software which has good graphical spherical coordinate plotting (from a data file). I have a three column data file where column 1 is $\theta$, column 2 is $\phi$ and column three is $f(\theta, \phi)$. I would like to plot this in spherical coordinates $(\theta, \phi, \rho = f(\theta, \phi))$, where points of equal $\rho$ are represented by equivalent colours. Does anyone have any recommendations regarding this type of plotting?   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is it possible to easily represent the plot with different colours as a function of the spherical coordinate $\rho$ (hence distance from the origin)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way I used Python to produce these data files.&lt;/p&gt;&#xA;" OwnerUserId="26678" LastEditorUserId="26678" LastEditDate="2018-01-27T09:43:36.530" LastActivityDate="2018-01-27T09:43:36.530" Title="Software for easy spherical coordinate plotting from data file" Tags="|python|plotting|software-recommendation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28702" PostTypeId="2" ParentId="28698" CreationDate="2018-01-26T16:24:51.843" Score="4" Body="&lt;p&gt;It appears to me that you are dealing with a sequence of linear systems $A_j x_j = b$, where $A_{j+1}$ is a low rank modification of $A_{j}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, I would investigate if the Krylov subspace $K = \text{range $V$}$ which you have built to solve one linear system, say, $A x  = b$ is relevant for the solution of the next problem, i.e., $$(A + \Delta A) (x + \Delta x) =b.$$ Specifically, I would first solve the small dense problem $$V^T(A + \Delta A) V z = V^T b$$ and use $x_0 = Vz$ as my initial guess for $x + \Delta x$ when applying GMRES to the next problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am gambling that this initial guess will serve you better than the solution of the original problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the small dense problem is a low rank modification of the Hessenberg system $$ H w = V^T b = e_1, \quad H = V^T A V$$ which is solved internally by GMRES. Here you may be well served with an application of the Sherman-Morrison-Woodbury formula mentioned in the comments.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2018-01-26T16:24:51.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28703" PostTypeId="2" ParentId="22071" CreationDate="2018-01-27T01:08:00.453" Score="1" Body="&lt;p&gt;&lt;strong&gt;Answering for posterity&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The physics you are trying to model exhibits parabolic and hyperbolic properties (some form of the Advection-Diffusion equation for calculating stream function or velocity) as well as elliptic properties in calculating vorticity. Let's assume a uniform grid ($\Delta x$=$\Delta y$=$h$). Because the physics is both hyperbolic and parabolic, the time step you should use is determined by the minimum time step calculated from the CFL for stability criteria for a hyperbolic equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$c^{2} = \frac{(\sqrt{u^{2}+v^{2}})_\text{max}^{2}\Delta t^{2}}{h^{2}} \leq 2d$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and from a parabolic equation in 2D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d = \frac{\alpha \Delta t}{h^{2}} \leq \frac{1}{4} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha$ is related to your Reynolds number, $c$ is your CFL value, and $\sqrt{u^{2}+v^{2}})_\text{max}^{2}$ is the maximum velocity over your domain, calculated at each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's important to note that these equations give you the &lt;em&gt;maximum&lt;/em&gt; value you can use for the timestep. A smaller timestep won't hurt, especially if you're interested in the transient solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To address the problem you first explained, it's possible that your solution isn't set to a low enough error tolerance for your velocity or stream function calculation. The result of this would be more numerical dissipation and &quot;smoothing&quot; of your solution so that you wouldn't be able to see the smaller structures being formed at the lower Reynolds numbers. &lt;/p&gt;&#xA;" OwnerUserId="26650" LastEditorUserId="20688" LastEditDate="2018-09-11T03:13:45.500" LastActivityDate="2018-09-11T03:13:45.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28704" PostTypeId="2" ParentId="28698" CreationDate="2018-01-27T09:05:29.123" Score="4" Body="&lt;p&gt;Michael Saunders wrote a sparse LU package that can do rank-1 updates, &lt;a href=&quot;https://web.stanford.edu/group/SOL/software/lusol/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LUSOL&lt;/a&gt;. You could try to use that, since you write that direct solvers are viable for your problem.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2018-01-27T09:05:29.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28705" PostTypeId="2" ParentId="20901" CreationDate="2018-01-27T09:11:37.050" Score="3" Body="&lt;p&gt;If the function looks smooth in a log-log plot, you can interpolate using a power law on each interval (power laws are, of course, linear in log-log). Thus, between points $(x_i, y_i)$ and $(x_{i+1}, y_{i+1})$ under the assumption that $y = C_i x^{n_i}$ within interval $i$, you obtain $n_i = \ln(y_i/y_{i+1})/\ln(x_i/x_{i+1})$ and $C_i = \ln(y_i) - n_i\ln(x_i)$. The contribution to the integral from interval $i$ is then&#xA;$$\Delta F_i = \int_{x_i}^{x_{i+1}} C_i x^{n_i}dx = \left\{ \begin{array}{l@{\quad}l} \frac{C_i}{n_i + 1} (x_{i+1}^{n_i+1} - x_i^{n_i+1}), &amp;amp; n_i \neq -1 \\ C_i(\ln x_{i+1} - \ln x_i), &amp;amp; n_i=-1, \end{array} \right. $$&#xA;where you obviously need some tolerance for identifying the special case $n_i=-1$ in your implementation.&lt;/p&gt;&#xA;" OwnerUserId="26686" LastActivityDate="2018-01-27T09:11:37.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28706" PostTypeId="1" AcceptedAnswerId="28737" CreationDate="2018-01-27T10:38:06.850" Score="3" ViewCount="696" Body="&lt;p&gt;I want to solve a small deformation solid structure problem applying periodic boundary conditions in FEM. The geometry is a square and the equations are:&#xA;$$&#xA;\text{div} \, \sigma = 0 \\&#xA;\sigma = f(\epsilon)\\&#xA;\epsilon_{ij} = \frac{1}{2}(u_{i,j} + u_{j,i}) \\&#xA;u^+ - u^- = c \\&#xA;\sigma^+ \cdot \hat{n} + \sigma^- \cdot \hat{n} = 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The variables with the $.^+$ or $.^-$ correspond to opposite sides in the square.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A weak form the equilibrium equation can be written as:&#xA;$$&#xA;\int_{V} \text{div} \, \sigma: \epsilon(\delta v) \,dV = 0 \\&#xA;$$&#xA;where $\delta v$ is a test function. Discretizing by FE and applying Newton-Raphson iterative scheme I wrote a residual of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;r = \left[&#xA;\begin{array}{c}&#xA;  \int_{V} \text{div} \, \sigma: \epsilon(\delta v_j) \,dV \quad \text{ for } j = 1...N_\text{int} - N_\text{ext}/2\\&#xA;  u_j^+ - u_j^- - c \quad \text{ for } j = 1...N_\text{ext}/2\\&#xA;\end{array}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; The weak formulation above does not have the surface contribution so the correct form is:&#xA;$$&#xA;\int_{V} \text{div} \, \sigma: \epsilon(\delta v) \,dV + \int_{\partial V}  \sigma \cdot \hat{n} \,  \,dS = 0 \\&#xA;$$&#xA;If you take the condition $\sigma^+ \cdot \hat{n} + \sigma^- \cdot \hat{n} = 0$, how does the condition $\sigma^+ \cdot \hat{n} + \sigma^- \cdot \hat{n} = 0$ affects that term in order to by enforce ? (It seems that that term should vanish but that did not work for me)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not the correct way how do I enforce the conditions $\sigma^+ \cdot \hat{n} + \sigma^- \cdot \hat{n} = 0$ ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: &lt;a href=&quot;https://github.com/GG1991/fe2_tests/tree/master/homog_bcs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; I have the &lt;em&gt;octave&lt;/em&gt; code &lt;em&gt;run_periodic.m&lt;/em&gt; for you to see. The &lt;em&gt;ass_periodic.m&lt;/em&gt; function do the job of assembly the residual and jacobian and tries to enforce the boundary conditions.&lt;/p&gt;&#xA;" OwnerUserId="24445" LastEditorUserId="24445" LastEditDate="2018-01-28T09:39:53.723" LastActivityDate="2018-01-31T12:48:59.003" Title="Periodic boundary condition in solid" Tags="|finite-element|boundary-conditions|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28707" PostTypeId="1" CreationDate="2018-01-27T15:11:02.017" Score="1" ViewCount="871" Body="&lt;p&gt;I would like to model heat diffusion at the gold / water interface after excitation of the metal surface by an ultrafast laser pulse (ca. 80 fs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An appropriate model to start with would be the &quot;two temperature model&quot; (TTM) or &quot;parabolic two step&quot; (PTS). This model considers the electronic and lattice systems as separate with electronic and lattice temperature $T_e$ and $T_l$, respectively. The electronic and lattice systems are coupled through an electron-phonon coupling constant $G$ and the electronic system is excited by a source term $S$ representing the incoming laser pulse. The model can be written as two coupled PDEs [from Smith (2001) Appl. Phys. Lett. 78:1240]:&#xA;$$&#xA;C_e(T_e) \frac{\partial T_e}{\partial t} = \frac{\partial}{\partial x} \left( k_e(T_e, T_l) \frac{\partial T_e}{\partial x} \right) - G \left[ T_e - T_l \right] + S \\&#xA;C_l \frac{\partial T_l}{\partial t} = G \left[ T_e - T_l \right],&#xA;$$&#xA;where $C_e$, $C_l$ are electron and lattice heat capacities, respectively, and $k_e$ is the thermal conductivity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am specifically interested in determining the temperatures of the system for the first 100 ps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question I would like to ask is: What are the different methods and tools available to solve the problem, with an emphasis on entry-level methodologies?&lt;/p&gt;&#xA;" OwnerUserId="26687" LastActivityDate="2018-01-27T15:11:02.017" Title="Methods and tools to solve the two-temperature model (TTM)" Tags="|pde|heat-transfer|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="28709" PostTypeId="2" ParentId="26742" CreationDate="2018-01-28T12:58:38.703" Score="1" Body="&lt;p&gt;Your code is mostly correct. Just change &lt;code&gt;u&lt;/code&gt; to &lt;code&gt;val&lt;/code&gt; when you call &lt;code&gt;integrate.quad&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy import integrate&#xA;import matplotlib.pyplot as plt&#xA;&#xA;a = 0.1&#xA;pi = np.pi&#xA;u = np.linspace(-100,100,200)&#xA;&#xA;&#xA;def integrand(a,u,y):&#xA;    top = np.exp(-y**2)&#xA;    bottom = a**2 + (u - y)**2&#xA;    return top/bottom&#xA;&#xA;def func(u):&#xA;    res = np.zeros_like(u)&#xA;    for i,val in enumerate(u):&#xA;        y,err = integrate.quad(integrand, -np.inf, np.inf, args=(a,val))&#xA;        res[i] = y&#xA;    return (a/pi)*res&#xA;&#xA;plt.plot(u,func(u))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As mentioned by @Stelios, passing &lt;code&gt;u&lt;/code&gt; (an array) to &lt;code&gt;integrand&lt;/code&gt; means it will return an array, while &lt;code&gt;integrate.quad&lt;/code&gt; expects a scalar. Here, we just use the fact that you already run over every &lt;code&gt;val&lt;/code&gt; of the array with  &lt;code&gt;enumerate&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26687" LastActivityDate="2018-01-28T12:58:38.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28713" PostTypeId="2" ParentId="28593" CreationDate="2018-01-28T19:54:22.030" Score="2" Body="&lt;p&gt;When first discretizing equations, it's useful to draw a diagram showing where the data lives relative to physical boundaries. Here are some schematics for visual aid: &lt;a href=&quot;https://i.stack.imgur.com/uSbQf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uSbQf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying finite difference to a differential equation takes a different form for cell-centered (CC) and node (N) data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;--------------------------------------------------------------------------------------------------------------------------------&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Node data&lt;/strong&gt;&#xA;Consider N data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the Laplace stencil operating on data $u$ with $f$ on the right-hand side (RHS),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;   \left(\frac{\partial^2 u}{\partial x^2}\right)_{boundary} =&#xA;   \frac{u_g - 2 u_b + u_i}{\Delta x^2} = f_b&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $u_g,u_b,u_i$ are the ghost, boundary and first interior node points. This Laplacian operator may be written in matrix form as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;A = \frac{1}{\Delta x^2} \left[\begin{array}{ccccccccc}&#xA;0  &amp;amp; 0     &amp;amp; 0         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;  0 \\&#xA;1  &amp;amp; -2    &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;0  &amp;amp; 1     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; 1       &amp;amp;  0 \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  1        &amp;amp; -2      &amp;amp;  1 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  0        &amp;amp; 0       &amp;amp;  0 \\&#xA;\end{array} \right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this stencil reaches to the ghost points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Dirichlet BCs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider Dirichlet BCs, $u_b$ &lt;em&gt;is known, and the solve is only from&lt;/em&gt; $u_i$ &lt;em&gt;onward&lt;/em&gt;. Therefore, the only equation we must consider is at location $i$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;   \left(\frac{\partial^2 u}{\partial x^2}\right)_{i} = \frac{u_b - 2 u_i + u_{i+1}}{\Delta x^2} = f_i&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we were to write an equation for the boundary point, we could write&#xA;\begin{equation}&#xA;   \left(\frac{\partial^2 u}{\partial x^2}\right)_{b} = \frac{u_g - 2 u_b + u_{i}}{\Delta x^2} = f_b \rightarrow u_g  = 2 u_b - u_{i} - \Delta x^2 f_b&#xA;\end{equation}&#xA;If we insist $f_b=0$ then we have a simplified version:&#xA;\begin{equation}&#xA;   u_g  = 2 u_b - u_{i}&#xA;\end{equation}&#xA;So the equation changes to&#xA;\begin{equation}&#xA;   \left(\frac{\partial^2 u}{\partial x^2}\right)_{b} = 0 = 0&#xA;\end{equation}&#xA;And we may remove it from our system. Looking back the first interior point, let's move boundary value to the RHS:&#xA;\begin{equation}&#xA;   \left(\frac{\partial^2 u}{\partial x^2}\right)_{i} = \frac{u_b - 2 u_i + u_{i+1}}{\Delta x^2} = f_i \rightarrow \frac{- 2 u_i + u_{i+1}}{\Delta x^2} = f_i - \frac{u_b}{\Delta x^2}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that the ghost point &lt;em&gt;should not enter the computations at all&lt;/em&gt;. They are a means to an end to apply the desired BCs. Correspondingly, the matrix $A$ is:&#xA;$&#xA;A = \frac{1}{\Delta x^2} \left[\begin{array}{ccccccccc}&#xA;0  &amp;amp; 0     &amp;amp; 0         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;  0 \\&#xA;1  &amp;amp; -2    &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;0  &amp;amp; 1     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; 1       &amp;amp;  0 \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  1        &amp;amp; -2      &amp;amp;  1 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  0        &amp;amp; 0       &amp;amp;  0 \\&#xA;\end{array} \right]&#xA;\\ \rightarrow&#xA;A = \frac{1}{\Delta x^2} \left[\begin{array}{ccccccccc}&#xA;0  &amp;amp; 0     &amp;amp; 0         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;  0 \\&#xA;0  &amp;amp; 0     &amp;amp; 0         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;0  &amp;amp; \textbf{0}     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;0  &amp;amp; 0     &amp;amp; 1         &amp;amp; -2        &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; -2        &amp;amp; 1         &amp;amp; 0       &amp;amp;  0 \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; \textbf{0}       &amp;amp;  0 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  0        &amp;amp; 0       &amp;amp;  0 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  0        &amp;amp; 0       &amp;amp;  0 \\&#xA;\end{array} \right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is nice because the equation has effectively become smaller. NOTE: the first two 0's in the above equation refer to the ghost and boundary equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, this &quot;truncation&quot; of the first and last column in $A$ will effectively apply Dirichlet BCs to our system, so long $u_b$ is in fact zero, so this only works for a special case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;--------------------------------------------------------------------------------------------------------------------------------&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cell-Centered data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the Laplace stencil operating on data $u$ with $f$ on the right-hand side (RHS),&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;   \left(\frac{\partial^2 u}{\partial x^2}\right)_{1} = \frac{u_g - 2 u_1 + u_{2}}{\Delta x^2} = f_1&#xA;\end{equation}&#xA;Where $u_g,u_1,u_2$ are the ghost, first interior and second interior cells. This Laplacian operator may be written in matrix form as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;A = \frac{1}{\Delta x^2} \left[\begin{array}{ccccccccc}&#xA;1  &amp;amp; 1     &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;  0 \\&#xA;1  &amp;amp; -2    &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp; 1     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; 1       &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  1        &amp;amp; -2      &amp;amp;  1 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1       &amp;amp;  1 \\&#xA;\end{array} \right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's substitute $u_g$ and adjust the stencil accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Dirichlet&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider Dirichlet BCs, $u_g$ may be computed observing that the boundary value is the average of the neighboring two cell center values.&#xA;\begin{equation}&#xA;    u_b = \frac{u_g + u_1}{2} \rightarrow u_g = 2u_b - u_1&#xA;\end{equation}&#xA;So our Laplacian stencil becomes:&#xA;\begin{equation}&#xA;   \left(\frac{\partial^2 u}{\partial x^2}\right)_{1} = \frac{(2 u_b - u_1) - 2 u_1 + u_{2}}{\Delta x^2} = f_1&#xA;\end{equation}&#xA;This boundary point must be moved to the RHS to maintain a consistent matrix-vector multiplication. Therefore our equation changes:&#xA;\begin{equation}&#xA;   \frac{u_g - 2 u_1 + u_{2}}{\Delta x^2} = f_1 \rightarrow \frac{- 3 u_1 + u_{2}}{\Delta x^2} = f_1 - \frac{2 u_b}{\Delta x^2}&#xA;\end{equation}&#xA;Correspondingly, the matrix $A$ changes:&#xA;$&#xA;A = \frac{1}{\Delta x^2} \left[\begin{array}{ccccccccc}&#xA;1  &amp;amp; 1     &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;  0 \\&#xA;1  &amp;amp; -2    &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp; 1     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; 1       &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  1        &amp;amp; -2      &amp;amp;  1 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1       &amp;amp;  1 \\&#xA;\end{array} \right]&#xA;\\ \rightarrow&#xA;A = \frac{1}{\Delta x^2} \left[\begin{array}{ccccccccc}&#xA;0  &amp;amp; 0     &amp;amp; 0         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;  0 \\&#xA;0  &amp;amp; -3    &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;0  &amp;amp; 1     &amp;amp; -2        &amp;amp; 1         &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp; \ddots    &amp;amp;           &amp;amp;         &amp;amp;    \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp; 1         &amp;amp; -2        &amp;amp; 1       &amp;amp;  0 \\&#xA;   &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  1        &amp;amp; -3      &amp;amp;  0 \\&#xA;0  &amp;amp;       &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;           &amp;amp;  0        &amp;amp; 0       &amp;amp;  0 \\&#xA;\end{array} \right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the first and last equations are identities ($0=0$) which are reserved for the ghost points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I stand corrected with my comment (on the OP's question) about assuming cell centered data. This &quot;truncation&quot; only works in a special case for Node data.&lt;/p&gt;&#xA;" OwnerUserId="8174" LastActivityDate="2018-01-28T19:54:22.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28714" PostTypeId="1" CreationDate="2018-01-29T07:35:13.463" Score="0" ViewCount="80" Body="&lt;p&gt;How to solve second order non-linear ODE &#xA;$$y'^2+y y''+\frac{2}{x} y y' -0.1 y^2=0$$ subject to $y'(1)=0$ and $y(1)=1$ over the interval $0 &amp;lt; x \le 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I turned the equation to a PDE $y'^2+y y''+\frac{2}{x} y y' -0.1 y^2=y'_t$. I was trying to find the steady state solution when $t \to \infty$, which is the solution to the ODE. I used an explicit finite difference scheme in MATLAB. But it doesn't seem to give the right solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have problem implementing boundary conditions in the MATLAB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be grateful if you help me solve this. Also mentioning any other numerical method will be great. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4950" LastActivityDate="2018-01-29T10:19:45.030" Title="Finite difference for 2nd order ode $y'^2+y y''+\frac{2}{x} y y' -0.1 y^2=0$ with $y'(1)=0$ and $y(1)=1$" Tags="|matlab|finite-difference|ode|boundary-conditions|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28715" PostTypeId="2" ParentId="28714" CreationDate="2018-01-29T08:28:01.603" Score="1" Body="&lt;p&gt;Sometimes, pdes result in solutions that are not steady (flow around a cylinder) and therefore you face a problem in which infinitely many solutions arise. This kind of problems are not suitable for temporal discretisations if one particular solution must be reached. I mean the limit&#xA;$$\lim_{t\to \infty}y(t)$$&#xA;does not make any sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead you can linearise the problem (Newton's method) by setting:&#xA;$$y^{n+1}=y^n+\delta y^n$$&#xA;And solve for $\delta y^n$ from linear problems carrying a substitution into your nonlinear model. Apply homogeneous dirichlet BC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that nonlinear terms in $\delta y^n$ can be neglected because it is supposed small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the funcion $y$ is smooth enough near the possible solutions Newton's method will do the job with quadratic convergence.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-01-29T09:16:54.070" LastActivityDate="2018-01-29T09:16:54.070" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28716" PostTypeId="1" CreationDate="2018-01-29T09:54:55.363" Score="5" ViewCount="233" Body="&lt;p&gt;I am working on a library that requires subdivision of polygons into triangles. The polygons are divided into triangles by (more or less) random points that are inside them. In general, the approach is to perform the Delaunay triangulation on a set of those points. This works very well, however sometimes I get intersecting triangles in my outcome. This case happens when I have more than three co-circular points. For the Delaunay triangulation I am using simple Watson's algorithm. My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;what to do in case of four or more co-circular points. None of the CG books I've read does not mention about handling it. Just that it is some special case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there other triangulations (similar to Delaunay - I cannot insert any additional points) I could potentially use.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4334" LastEditorUserId="9667" LastEditDate="2018-01-31T20:31:06.383" LastActivityDate="2018-02-02T14:47:54.227" Title="Delaunay triangulation for datasets with four or more co-circular points" Tags="|computational-geometry|delaunay-triangulation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28717" PostTypeId="2" ParentId="28714" CreationDate="2018-01-29T10:19:45.030" Score="2" Body="&lt;p&gt;You have two conditions at x=1, I hope this is correct in which case you have an initial value problem. If yes, then you can write this as a first order system by introducing $z=y'$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;z' = y/10 - z^2/y - 2z/x, \qquad y' = z&#xA;$$&#xA;with initial condition&#xA;$$&#xA;y(1) = 1, \qquad z(1) = 0&#xA;$$&#xA;Solve this using an ode solver from $x=1$ to $x=0$. As you approach $x=0$, you will face the problem of division by zero, unless $z \to 0$. There is also possibility of $y$ becoming zero. Do you have a reason to expect your equation should have non-negative solutions ? &lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-01-29T10:19:45.030" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28719" PostTypeId="1" CreationDate="2018-01-29T10:34:50.930" Score="2" ViewCount="292" Body="&lt;p&gt;Let's say I have n variables. Each variable has lower and upper bounds. I want to find all suitable combinations of these variables to sum up to a required value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example with two variables:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x_1 + x_2 = 100\\&#xA;0 &amp;lt;= x_1 &amp;lt;= 70  \\&#xA;0 &amp;lt;= x_2 &amp;lt;= 60&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the result would look like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(x_1, x_2) \in \{(70, 30), (69.8, 30.2), \cdots\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These variables could take integer and decimal values as given above. The perturbation values for each of these variables can be chosen according to the requirements. The perturbation values used here are Pert[X1,X2]=[0.2,0.2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem of solving for two variables is comparatively simple than having n number of variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm available in Python to solve such a problem ? It would be even more helpful, if a Python code using the algorithm to solve a similar problem could be provided.  &lt;/p&gt;&#xA;" OwnerUserId="25459" LastEditorUserId="9667" LastEditDate="2018-01-29T14:04:52.883" LastActivityDate="2018-03-01T03:31:49.903" Title="Find combinations of variables with variable bounds to get summed up to a required value (using Python)" Tags="|optimization|python|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28720" PostTypeId="2" ParentId="28719" CreationDate="2018-01-29T11:57:22.970" Score="1" Body="&lt;p&gt;This problem could be written as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Knapsack_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;restricted knapsack&quot;&lt;/a&gt; problem: given a set of items each with a weight $w_i$, fill up your knapsack with at most a certain number $c_i$ of each item respecting the total weight of the knapsack. Mathematically, this is written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\textrm{max} \sum_{i=1}^{N} w_{i} x_{i} \ \mathrm{s.t.}\ \sum_{i=1}^{N} w_{i} x_{i} &amp;lt; W \ \mathrm{and}\ x_{i} \leq c_{i}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, the weight of each variable should correspond to the perturbation. Then you solve the above optimization problem where your values are the $w_i$'s and your upper limits the $c_{i}$'s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case you could set: $w_1 = 0.2,\, w_2 = 0.2,\, c_1 = 350,\, c_2 = 300$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to be a Python snippet &lt;a href=&quot;https://codereview.stackexchange.com/questions/20569/dynamic-programming-solution-to-knapsack-problem&quot;&gt;at code review s.e.&lt;/a&gt;. No guarantees on the correctness or effiency of this snippet. You could also use the &lt;a href=&quot;https://www.gnu.org/software/glpk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU Linear Programming Kit&lt;/a&gt; (GLPK) through the &lt;a href=&quot;http://cvxopt.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVXOPT&lt;/a&gt; package. I don't know if these tools can provide all feasible solutions or if they would stop if an optimal (i.e. the equality constraint is satisfied; i.e. the knapsack is completely full) combination is found.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2018-01-29T11:57:22.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28722" PostTypeId="1" CreationDate="2018-01-29T18:11:30.130" Score="3" ViewCount="54" Body="&lt;p&gt;In Floater's &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.115.9578&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; on barycentric rational interpolation, he shows that a stable interpolant using irregularly space points can be evaluated in $O(N)$ operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For equally spaces samples, cubic b-splines can be used to generate an interpolator that requires $O(1)$ evaluation, and using the Catmull-Rom curve, $O(\log(N))$ operations can interpolate the curve, but the parameterization cannot be supplied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm which creates a $C^1$ interpolant from irregular samples (with user-supplied parameterization) which can be evaluated $O(1)$ or $O(\log(N))$ time?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2018-03-18T02:10:15.543" LastActivityDate="2018-03-18T02:10:15.543" Title="1D irregular interpolation with $O(1)$ or $O(\log(N))$ evaluation" Tags="|interpolation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28723" PostTypeId="1" CreationDate="2018-01-29T20:53:37.413" Score="0" ViewCount="236" Body="&lt;p&gt;I am working on a 3-D problem using the finite volume method (FVM). I came across a problem dealing with the computation of gradients at a face. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The geometry that I'm working with is discretized into cells. Every face is aligned with one of the three cartesian axes. The specific FVM that I am using computes cell-averaged values. At each face, I am interested in computing the face gradients in all 3 directions, i.e., $\frac{du}{dx}$, $\frac{du}{dy}$, $\frac{du}{dz}$. I use a simple central finite difference scheme to compute the gradients, i.e.,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{du}{dy} = \frac{u_{i+1}-u_{i-1}}{2\Delta y}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this formula appears to be problematic for this problem. For example, if we consider a face parallel to the y-axis and z-axis, it's normal would be in the x-axis. The 2 bounding cells share this face and have centroids at, say, (0,0,0) and (1,0,0). The face centroid is at (0.5,0,0). Using the above formula and these 2 cells, this would give $\Delta y=\Delta z = 0$, making the above formula undefined for the gradients in the y and z directions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this method simply inappropriate for this geometry? What could be used as an alternative?&lt;/p&gt;&#xA;" OwnerUserId="23915" LastActivityDate="2018-10-30T23:34:18.053" Title="Finite difference for computing gradients at face in finite volume code" Tags="|finite-difference|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28724" PostTypeId="1" CreationDate="2018-01-29T18:37:18.843" Score="0" ViewCount="152" Body="&lt;p&gt;I have acceleration data from a sensor. X Y &amp;amp; Z.&#xA;I move the senor in the Y axis. Mostly in a straight line. So I ignore x &amp;amp; z.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the sensor documentation&#xA;5.2.1 Acceleration output： &lt;/p&gt;&#xA;&#xA;&lt;p&gt;ax=((AxH&amp;lt;&amp;lt;8)|AxL)/32768*16g(g is Gravity acceleration，9.8m/s2) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;ay=((AyH&amp;lt;&amp;lt;8)|AyL)/32768*16g(g is Gravity acceleration，9.8m/s2) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;az=((AzH&amp;lt;&amp;lt;8)|AzL)/32768*16g(g is Gravity acceleration，9.8m/s2) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data is in (m/s2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need a simple calculation that java or C# can take easily. I want to write something in code that calculates the acceleration over time to maximum velocity and average velocity. I need a &quot;speed&quot; value that I can display. &#xA;For Ex. Max speed 12MPH and Average Speed 8MPH.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this data the device was moved from the zero point to about 6 inches away less than 1 second.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Time(s) Acc X   Acc Y   Acc Z&#xA;48.547  0.4756  0.0864  1.2207&#xA;48.563  0.2051  0.2651  1.3350&#xA;48.563  0.0044  0.6621  1.3140&#xA;48.578  -0.2876 1.0117  1.4292&#xA;48.578  -0.0732 1.5586  1.4653&#xA;48.594  -0.0659 1.8984  1.3447&#xA;48.594  -0.2344 2.4453  1.4043&#xA;48.641  -0.2690 3.2148  1.3677&#xA;48.656  -0.4072 3.0083  1.4995&#xA;48.656  -0.2573 3.2700  1.3545&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26707" OwnerDisplayName="Steve Coleman" LastActivityDate="2018-02-28T22:31:45.097" Title="Converting acceleration over time to velocity or speed in code" Tags="|acceleration|computational-physics|software|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28725" PostTypeId="2" ParentId="28724" CreationDate="2018-01-29T18:53:05.280" Score="1" Body="&lt;p&gt;The equations at the start look like they're converting sensor data (pairs of 8 bit integer values) into floating point numbers.  They're irrelevant to your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need to do is :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$v_k = v_{k-1}+a_k(t_k-t_{k-1})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each time $t_k$ and with $t_0$ and $v_0$ the initial time and velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the &lt;em&gt;average&lt;/em&gt; velocity over the whole time you can do :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u=\frac {\sum v_k(t_k-t_{k-1})}{\sum(t_k-t_{k-1})}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sum on the bottom simplifies to $t_N-t_0$.&lt;/p&gt;&#xA;" OwnerUserId="26709" OwnerDisplayName="StephenG" LastActivityDate="2018-01-29T18:53:05.280" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28726" PostTypeId="2" ParentId="28719" CreationDate="2018-01-30T03:29:46.677" Score="0" Body="&lt;p&gt;With the constraints you have listed, you may declare $x_i$ as integer, then solve with an IP solver (see the open-source coin-or solvers, and cplex or gurobi if you have access to them) to enumerate all feasible integer solutions. From here you can easily calculate the perturbed values, and find the target set. &lt;/p&gt;&#xA;" OwnerUserId="21728" LastActivityDate="2018-01-30T03:29:46.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28727" PostTypeId="2" ParentId="28723" CreationDate="2018-01-30T08:38:55.417" Score="2" Body="&lt;p&gt;I am not sure what are referring to when you say face gradients. But I will tell you how I would compute a gradient at any point in space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have a set of values for some quantity $\{u_i\}$ corresponding to some set of points in space $\{x_i\}$. You can build a polynomial that interpolates locally $N_p$ points of the set, say &#xA;$$u(x)=\sum_{k=1}^{N_p}{u_k \phi_k(x)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\{\phi_k(x)\}$ is the interpolant basis, and $\phi_k(x_i)=\delta_{ik}$ and therefore $u(x_i)=u_i$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore you can approximate the gradient as &#xA;$$\vec{grad}\,u(x)=\sum_{k=1}^{N_p}{u_k \vec{grad}\,\phi_k(x)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to compute the face gradient:&#xA; $$\vec{Grad}\,u(x)= [I-\vec{n} \otimes\vec{n}]   \vec{grad}\,u(x)$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-01-30T08:54:04.953" LastActivityDate="2018-01-30T08:54:04.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28728" PostTypeId="1" CreationDate="2018-01-30T09:30:52.797" Score="1" ViewCount="295" Body="&lt;p&gt;Can anybody help me to find books or MATLAB code examples for solving electric field of the electron gun(fig.1)with finite difference method? Python code examples are also perfect.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/d7se3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d7se3.png&quot; alt=&quot;the schematic of the electron gun&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The electron gun includes three electrodes,1,2,3.The voltages of three electrodes are 0V,100V,1000V,respectively.&lt;/p&gt;&#xA;" OwnerUserId="26711" LastActivityDate="2018-01-31T03:31:59.287" Title="Finite difference method for the electric field of the electron gun" Tags="|finite-element|matlab|finite-difference|python|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28730" PostTypeId="1" CreationDate="2018-01-30T11:22:02.110" Score="0" ViewCount="222" Body="&lt;p&gt;If you want to develop numerical algorithms for variational inequalities, you often choose a &lt;em&gt;Semismooth Newton Method&lt;/em&gt;. In many cases, this approach involves derivatives of $\max$ or $\min$ functions in respective $L^p$-spaces, which result in &lt;em&gt;pointwise evaluation&lt;/em&gt; of functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this isn't too much of a problem, if you're discretizing with, for example, Lagrange elements, since their degrees of freedom represent pointwise evaluation in the nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your algorithm and the Newton-steps obviously can't converge, if the DOFs are integrals of the tangential components over the edges! This is the case for Nédélec Elements for $H_0(curl, \Omega)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wasn't able to find anything concerning this topic, so my question: Is there literature, that works around those problems or even an easy solution?&lt;/p&gt;&#xA;" OwnerUserId="26585" LastActivityDate="2018-01-30T23:29:51.523" Title="Nédélec Elements and Newton-Methods" Tags="|finite-element|pde|numerical-analysis|newton-method|electromagnetism|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28731" PostTypeId="2" ParentId="28730" CreationDate="2018-01-30T13:07:24.107" Score="1" Body="&lt;p&gt;The constraints need to be based on reasonable physics. So if you are using a Nedelec element, then that's presumably because your physics are based on curls and tangential components, and any constraint you have is reasonably also only going to contain these. So the problem is not bigger than in other cases. If your problem requires pointwise continuity in the constraints, then you are either asking for something that's not physically reasonable, or you have chosen the wrong element.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-01-30T13:07:24.107" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28732" PostTypeId="2" ParentId="28730" CreationDate="2018-01-30T23:21:03.657" Score="0" Body="&lt;p&gt;Forgive if I am wrong but in your question you confusing regularity of the solution as result of non-smooth coefficients in PDF, and approximation space which you choose appropriately to used operators in your week form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As rule of thumb, if on your approximate field acts gradient operator you need H1 space, if divergence H-div, and H-curl curl operator. If to given filed no differential is applied, you can use L2 space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is the third component of the puzzle approximation base, which in principle can be independently selected from the space. Particular realisation of H-div and H-curl space with polynomial base are RT-elements and Nédélec elements. You can have approximation base with the singularity, where the solution is not regular at the point or on a curve. In plasticity or in the case mentioned by you (max function in coefficients), because of regularity of the solution you need low order elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your coefficients in PDE have max term which depends on solution, then you can regularise problem, for example like this&#xA;\begin{equation}&#xA;max(u^h,a) =    (u^h + a  + \frac{1}{r}| u^h-a |^r)/2&#xA;\end{equation}&#xA;where $r\geq 1$, for example $r=1.01$. With that trick, your Newton method will work. You can look what is done in contact mechanics, where substantial work has been done to solve this type of problems, for example, Popp method of active set strategy.&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2018-01-30T23:29:51.523" LastActivityDate="2018-01-30T23:29:51.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28733" PostTypeId="2" ParentId="28728" CreationDate="2018-01-31T03:31:59.287" Score="2" Body="&lt;p&gt;I think the canonical way to solve this problem (at least in a classroom setting) is to approximate the Laplacian (the PDE that governs the scalar potential $V$) using finite differences in Cartesian coordinates. When you apply Gauss-Jacobi algorithm to such a linear system, you end up with a particularly simple scheme: at each successive iteration, every potential sample $V_{i,j}$ should be overwritten with the average of its four neighbors (up/down/left/right). Your dirichlet conditions ($V = 1000$, etc) just become &quot;dead&quot; sample points whose values never change, but they do still contribute to the averages of any adjacent &quot;live&quot; points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So initially, you'll have an all-zero grid of $V_{i,j}$'s, except for the dirichlet samples that are initialized with the prescribed values. Each iteration, updated values of $V$ will &quot;creep&quot; across the model one point at a time. In the limit (infinite number of iterations) the $V_{i,j}$'s will converge to the solution to the Laplacian (albeit rather slowly). You can embellish the scheme slightly into Gauss-Seidel or successive over-relaxation if you wish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you are satisfied with your solution for $V_{i,j}$, you can use it to find the $\vec E$ field, using the usual relationship $\vec E=-\nabla V$. This postprocessing step can also be accomplished by finite differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't necessarily a great method. But it's easy to derive and requires minimal programming skill, so it's commonly taught. See, for example, &lt;a href=&quot;http://faculty.otterbein.edu/DRobertson/compsci/em-stud.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://faculty.otterbein.edu/DRobertson/compsci/em-stud.pdf&lt;/a&gt;, you could probably find other similar materials with some searching.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2018-01-31T03:31:59.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28737" PostTypeId="2" ParentId="28706" CreationDate="2018-01-31T08:48:43.570" Score="1" Body="&lt;p&gt;I could arrive to the solution at the end applying the *Lagrange Multiplier Method. The surface term at the weak form:&#xA;$$&#xA;\int_{V} \text{div} \, \sigma: \epsilon(\delta v) \,dV + \int_{\partial V}  \sigma \cdot \hat{n} \,  \,dS = 0 \\&#xA;$$&#xA;should vanish, i.e.:&#xA;$$&#xA;\int_{V} \text{div} \, \sigma: \epsilon(\delta v) \,dV = 0 \\&#xA;$$&#xA;because in these problem we are not applying Neumann or Robin Boundary conditions, we are a applying a Dirichlet type:&#xA;$$&#xA;u^+ - u^- = c&#xA;$$&#xA; that we don't know at first the value of $u^+$ and $u^-$. The other thing we know is that the external force applied should be the same :&#xA;$$\sigma^+ \cdot \hat{n} + \sigma^- \cdot \hat{n} = 0$$&#xA;This mean that is logical to add a Lagrange multiplier in the force term (in the correct degrees of freedom and with the right sign) as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{V} \text{div} \, \sigma: \epsilon(\delta v_j) \,dV - \lambda_j = 0 \text{ for } j = 1...P&#xA;$$&#xA;By this way we have added $P$ more unknown $\lambda_j$ to the system. To solve the problem we add the displacement periodic boundary conditions:&#xA;$$u_j^+ - u_j^- = c_j \text{ for } j = 1...P.$$&#xA;There are other techniques that can be used such as the &lt;em&gt;Master Slave Method&lt;/em&gt;, &lt;em&gt;Penalty Method&lt;/em&gt;, &lt;em&gt;Augmented Lagrangian&lt;/em&gt;, between others. I didn't go deeper with them yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I give a picture of how a body is deformed under a certain value of $c$ using the Lagrange Multiplier Method :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/C8e7L.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C8e7L.jpg&quot; alt=&quot;Body deformed using periodic boundary conditions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code to obtained that solution can be seen &lt;a href=&quot;https://github.com/GG1991/fe2_tests/tree/master/homog_bcs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; and can be executed as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;octave run_periodic_lm.m&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="24445" LastEditorUserId="24445" LastEditDate="2018-01-31T12:48:59.003" LastActivityDate="2018-01-31T12:48:59.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28738" PostTypeId="1" CreationDate="2018-01-31T09:21:44.830" Score="0" ViewCount="1107" Body="&lt;p&gt;There are many software packages for the finite element method, of which the most popular are listed e.g. on &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_finite_element_software_packages&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;. When it comes to the finite volume method, I'm not aware of any similarly comprehensive list. In fact, the only software package I'm actually aware of is OpenFOAM. Do you know any alternatives?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm mostly interested in software capable of describing (a system of) general advection-diffusion-reaction PDEs, not just a specific model such as Navier-Stokes. But if there are good extensible Navier-Stokes solvers, they can be listed as well.)&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2018-01-31T14:06:19.340" Title="Finite volume software packages" Tags="|fluid-dynamics|software|finite-volume|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28739" PostTypeId="2" ParentId="28738" CreationDate="2018-01-31T13:11:47.343" Score="4" Body="&lt;p&gt;Have a look at &lt;a href=&quot;http://www.clawpack.org/pyclaw/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pyclaw&lt;/a&gt;. This library has been around for quite a while and is fairly robust. It offers:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Implementations of several &lt;a href=&quot;http://www.clawpack.org/wp_algorithms.html#wp-algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;Godunov-type methods&lt;/a&gt; and &lt;a href=&quot;http://www.clawpack.org/riemann.html#riemann&quot; rel=&quot;nofollow noreferrer&quot;&gt;Riemann solvers&lt;/a&gt; in 1/2/3D.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Adaptive mesh refinement (via &lt;a href=&quot;http://www.clawpack.org/amrclaw.html#amrclaw&quot; rel=&quot;nofollow noreferrer&quot;&gt;AMRclaw&lt;/a&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Support for &lt;a href=&quot;http://www.clawpack.org/pyclaw/parallel.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;parallel computing&lt;/a&gt; - among other things, it ties-in nicely with &lt;a href=&quot;https://www.mcs.anl.gov/petsc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc&lt;/a&gt;, as shown &lt;a href=&quot;http://people.cs.uchicago.edu/~knepley/presentations/SC14Keynote.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Support for some geophysical flow problems (e.g., shallow water) via &lt;a href=&quot;http://www.clawpack.org/geoclaw.html#geoclaw&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geoclaw&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Comprehensive documentation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some of the authors / contributors to the library are active on this forum.&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2018-01-31T14:06:19.340" LastActivityDate="2018-01-31T14:06:19.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28740" PostTypeId="1" CreationDate="2018-01-31T14:45:00.050" Score="2" ViewCount="514" Body="&lt;p&gt;I have a Matlab code computing the solution to an eigenvalue PDE. It consists of two parts: assembly of the stiffness/rigidity matrices and solving a generalized eigenvalue problem. I mention that the assembly is done using vectorized code in Matlab. It is likely that I would need to run the program on a quite large database of test-cases (a few million cases). In this case, performance in terms of computational time becomes noticeable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering what could be the gain if I tried to code the algorithm in C/C++ ? (not sure I am capable of that... :) but I want to weigh the potential benefits) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess assembly could go faster, but the eigenvalue solver is pretty efficient in Matlab. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is your experience with situations like this? Can coding in a faster language give important computation time reduction compared to Matlab?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;More details on the problem as requested:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;eigenvalue computation for the Laplace-Beltrami operator on a portion of the sphere&lt;/li&gt;&#xA;&lt;li&gt;I use Lagrange P1 finite elements on a triangular mesh of the surface domain&lt;/li&gt;&#xA;&lt;li&gt;example of Matlab computation time for roughly 780000 points (degrees of freedom): assembly 3 seconds, eigenvalue computation 26 seconds&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Apparently, the significant part of the computation is finding the eigenvalue. Maybe using SlepC could help with this... (In some old manual I saw a SlepC interface for Matlab, but not in the new ones...)&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="3548" LastEditDate="2018-02-01T09:45:22.033" LastActivityDate="2019-05-19T06:15:55.830" Title="Potential gain - Matlab vs C/C++ - assembly and eigenvalues" Tags="|matlab|performance|c|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28741" PostTypeId="2" ParentId="28716" CreationDate="2018-01-31T18:17:20.413" Score="3" Body="&lt;p&gt;You can use exact predicates to detect co-circular points, and use symbolic perturbation to consistently decide which triangles to generate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Regarding exact predicates:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If your point coordinates are integers, then it is easier, and you can use Hadamar's inequality to determine the range of valid coordinates for which the values computed by the predicates will not overflow.&lt;/li&gt;&#xA;&lt;li&gt;If you use floating-point coordinates, then it is more tricky to implement. You can use either an arbitrary precision library (such as MPFR) or arithmetic expansions (as suggested by Shewchuk [1], see also his 'triangle' software)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Regarding symbolic perturbation:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever the predicates answers 'right on the circle', you need to consistently disambiguate. One way to do that is explained in the 'SOS' article [2]. Basically, in a certain sense, the idea is to consider the Voronoi diagram as the limit of a power diagram with different weights for the points that tend to 0 with different speeds, and consider the first non-zero coefficient in a Taylor development of the so-parameterized predicate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note on efficiency:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To detect the cases where the answer can be easily/quickly determined, you can use arithmetic filters, as suggested in [3] (this has dramatic impact on the performances).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;More references (my own stuff)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All these algorithms are implemented in my geogram library [4], see my associated publication [5].&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;[1] Jonathan Richard Shewchuk, Robust Adaptive Precision Floating Point Geometric Predicates, SoCG, 1996&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;[2] Edelsbrunner and Mucke, Simulation of Simplicity, ACM Trans. Graph., 1990&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;[3] Meyer and Pion, FPG: a code generator for fast and certified geometric predicates, Real Number and Computers, 2008 &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;[4] &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;[5] The Predicates Construction Kit: &lt;a href=&quot;https://hal.inria.fr/hal-01225202&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.inria.fr/hal-01225202&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="26479" LastEditDate="2018-02-02T14:47:54.227" LastActivityDate="2018-02-02T14:47:54.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28742" PostTypeId="1" CreationDate="2018-02-01T02:39:02.737" Score="2" ViewCount="295" Body="&lt;p&gt;I have a matrix $J$, and I know there are 3 existing zero eigenvalues and their eigenvectors. I want to detect if there is one extra eigenvalue to go cross zero (if it is zero, its eigenvalue will rotate to the range of previous zero eigenvectors, and the matrix will be defective!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can do QR decomposition to the zero eigenvectors and construct a orthonormal basis $N$ for their Null space, then I can project the matrix $J$ to the null subspace as $N^TJN$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\det(N^TJN)$ can tell me if there is another eigenvalue being zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the $J$ matrix is sparse and its size is large, the cost is too high. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use ARPACK's subroutines with &lt;code&gt;which = 'LR'&lt;/code&gt; to find the largest real part of eigenvalue. However, it does not work well with defective matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The background is chemical kinetics.&lt;/p&gt;&#xA;" OwnerUserId="24589" LastActivityDate="2018-06-01T09:10:44.637" Title="how to calculate the determinant of a projection of matrix to a subspace" Tags="|algorithms|eigenvalues|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28743" PostTypeId="2" ParentId="28742" CreationDate="2018-02-01T06:58:25.953" Score="1" Body="&lt;p&gt;Instead of using determinants one could try to calculate the rank of the matrix. If the rank is less than the size of matrix there is an eigenvalue equal to 0.&lt;/p&gt;&#xA;" OwnerUserId="26733" LastActivityDate="2018-02-01T06:58:25.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28744" PostTypeId="1" CreationDate="2018-02-01T07:51:43.183" Score="3" ViewCount="321" Body="&lt;p&gt;We know the classical solution of transport equation is determined by one initial (boundary?) condition, for example, the solution of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u(t,x)}{\partial t}+\frac{\partial u(t,x)}{\partial x}=0$$&#xA;$$u(0,x)=e^{-x^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is &#xA;$$u(t,x)= e^{-(x-t)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how to find this solution numerically with finite difference method (FDM)? One initial condition (i.c.) is apparently insufficient. Do we need a special difference scheme, or a special boundary condition (b.c.) that amounts to &quot;nothing&quot; in $x$ direction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;But why do you need that? This problem can be easily solved analytically!&quot; Yeah, I thought so, too, until I came across this boundary value problem of Navier-Stokes equation recently:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.featflow.de/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark2_re100.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.featflow.de/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark2_re100.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one can see, only one b.c. is given for $p$, so there seems to be a family of partial differential equation that &quot;suffers&quot; the same issue, and that's the reason I decided to look into it.&lt;/p&gt;&#xA;" OwnerUserId="5331" LastActivityDate="2018-03-10T13:40:54.347" Title="Find classical solution of transport equation with FDM" Tags="|pde|finite-difference|fluid-dynamics|boundary-conditions|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28745" PostTypeId="2" ParentId="28744" CreationDate="2018-02-01T10:18:26.343" Score="1" Body="&lt;p&gt;This equation is a pure transport equation. It defines the conservation of the quantity $u$ all over its path, i.e. &#xA;$$\frac{du}{dt}=0 \qquad \forall t,x$$&#xA;and thus whatever the initial condition is, it is transported with constant velocity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the equation is solved in a bounded domain, the concept of boundary condition arise. It is necessary to provide what happens with the incomings at the boundary. Mathematically the operator $\partial/\partial x$ always comes with a boundary condition when is bounded. In an unbounded domain this BC is implicitly defined in the initial condition, but in a bounded one this BC must be provided. This BC is imposed in the inflow boundary, that in your case is always on the left side of your domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wrt. the NS equations... the pressure does not need any BC but to be defined in one point if only Dirichlet BC are provided for the velocity. If one Do nothing BC (free stress, usually in outflow boundary) is imposed, the pressure is completely defined implicitly in the normal stress:&#xA;$$\vec{n}^T\sigma\vec{n}|_\Gamma=-p+2\vec{n}^T\frac{\partial \vec{v}}{\partial n}=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us rewrite your problem to be able to compare analytical vs numerical results. Let us define a bounded region in which the equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t}+v\frac{\partial u}{\partial x}=0$$ &#xA;is valid, namely, for $t_0&amp;gt;0$ and $x\in[x_1,x_2]$&#xA;with an initial condition of :$u(x,t_0)=u_0(x)$&#xA;Therefore, due to the fact of the boundedness of the domain an inflow BC is required. Because the transport velocity $v$ is constant and positive the BC will be for the extremum in the coordinate $x_1$. Let this BC be $u(x_1,t)=g(t)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of simplicity let us take $v=1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the analytical solution is not what you said, but $u_0(x)$ moving right and a new curve drawn by the function $g(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With ANY discretisation, say FDM, FEM, FVM... you will have a discrete system as follows:&#xA;$$\frac{du}{dt}+Du+BC=0$$&#xA;Where $u=[u_2,...,u_N]^T$, and $BC=[g(t)/\Delta x,0,...,0]$ and $D(i,i)=1/\Delta x,D(i,i-1)=-1/\Delta x$ for $i\geq 2$ and $D(1,1)=1$.&#xA; It is clear that $u_1=g(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case of NSE, one has the following system:&#xA;$$div(\vec{v})=0$$&#xA;$$\frac{\partial \vec{v}}{\partial t}+(\vec{v}^T\vec{grad})\vec{v}=div(\widetilde{\sigma})$$&#xA;with $\widetilde{\sigma}=-p\mathbb{I} +\tfrac{1}{2}[\vec{grad}\,\vec{v}+(\vec{grad}\,\vec{v})^T] $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be spatially discretised, following any method, as follows:&#xA;$$D\vec{v}=0$$&#xA;$$ \frac{d \vec{v}}{dt}+\vec{H}(\vec{v})=L\vec{V}-Gp+\vec{BC}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to have a stable discretisation the rank of the divergence matrix $D$ must be less than the number of velocity unknowns. Otherwise $\vec{v}$ would be uniquely determined as the kernel of $D$. This induces the usage of different meshes for pressure and velocity, i.e. you cannot use whatever mesh you want to solve the problem due to stability issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The operators $G$ and $L$ are the gradient and Laplacian respectively. The vector $\vec{H}$ corresponds to the nonlinear terms in NSE.&#xA;The system, once discretised in time may be arranged as follows:&#xA;$$\left[\matrix{A &amp;amp;G\\ D&amp;amp;0}\right]\left[\matrix{\vec{v}^{n+1}\\ p^{n+1}}\right]=\left[\matrix{\vec{R}(\vec{v}^n,\vec{v}^{n-1})\\ 0}\right]+\vec{BC}$$&#xA;One example of matrix $A$ coincides with:&#xA;$$A=\left[\mathbb{I}-c\frac{\Delta t}{Re}L\right]/\Delta t$$&#xA;Where c is a constant and depends on the selected temporal scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term $\vec{BC}$ only has BC for the velocity. This can be easily seen applying FEM.&#xA;For FDM pressure is usually calculated at interior nodes. Keep in mind that the pressure is a Lagrange multiplier that forces the incompressibility condition, and plays a mathematical rather a physical role. Thats the reason of its lack of BC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Particularisation for finite difference scheme&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I previously commented, the mesh must be different for pressure and for velocity (Inf-sup condition). You can use an square element in which there is only a pressure node in the middle, 2 nodes for vertical component of velocity in the upper and lower part, and finally 2 more for the horizontal component of the velocity in the right and left part of the element. This mesh is known as a staggered grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has a clear advatage: the face fluxes are calculated exactly in order to assure the incompressibility condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, due to the position of the pressure nodes, it is not required any boundary condition. &lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-02-03T21:58:53.880" LastActivityDate="2018-02-03T21:58:53.880" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28747" PostTypeId="1" CreationDate="2018-02-01T16:35:20.453" Score="0" ViewCount="137" Body="&lt;p&gt;I'm using 4th order RK to solve the schroedinger equation in atomic units. Say I want to simulate 400fs in intervals of h=10fs, then in atomic units this is h=413a.u and 400fs=16500a.u. 4RK involves repeated multiplication by the timestep, so with h=413 everything blows up. Could someone explain what I'm missing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: to be clear, I'm solving &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align} \frac{d}{dt} \psi = -i H \psi \end{align}&#xA;Where $\psi$ is a vector and $H$ a hermitian matrix.  &lt;/p&gt;&#xA;" OwnerUserId="26740" LastEditorUserId="26740" LastEditDate="2018-02-01T16:42:53.243" LastActivityDate="2020-07-19T18:07:23.657" Title="Runge-Kutta timestep in atomic units" Tags="|ode|runge-kutta|differential-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28748" PostTypeId="2" ParentId="28747" CreationDate="2018-02-01T17:35:11.677" Score="1" Body="&lt;p&gt;This phenomena of an explicit Runge-Kutta method is known as stability. For a given equation, the eigenvalues of your system (here, the Hamiltonian) determine a maximum time step for which the numerical integration scheme will be stable (i.e. not blow up). &lt;a href=&quot;http://www.staff.science.uu.nl/%7Efrank011/Classes/numwisk/ch10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a good resource which mentions the stability region for RK4&lt;/a&gt;. As these resources show, there are special methods such as implicit methods which can be used in order to take larger time steps.&lt;/p&gt;&#xA;&lt;p&gt;But even explicit Runge-Kutta methods can be enhanced. For example, using adaptive Runge-Kutta methods with step-stabilization (PI-adaptive step controllers) will allow the method to only take smaller time steps when they are needed for stability, and this then allows the steps to be (usually) larger on average. Standard ODE solver software like those in &lt;a href=&quot;https://www.mathworks.com/help/matlab/math/choose-an-ode-solver.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MATLAB&lt;/a&gt;, &lt;a href=&quot;https://www.unige.ch/%7Ehairer/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fortran&lt;/a&gt;, or &lt;a href=&quot;http://diffeq.sciml.ai/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Julia&lt;/a&gt; make use of this. So I would recommend using whatever ODE solvers you have available in your language, since not only will they have (potentially PI-controlled) adaptive methods, but also integrators for stiff systems. These will be much more efficient than RK4 which is really a method just for teaching and not for production use (other explicit RK methods have larger stability regions and/or less error for the same amount of work).&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:07:23.657" LastActivityDate="2020-07-19T18:07:23.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28749" PostTypeId="1" AcceptedAnswerId="28935" CreationDate="2018-02-01T20:31:55.613" Score="1" ViewCount="1988" Body="&lt;p&gt;I am currently learning the Smoothed Particle Hydrodynamics method that I need to use later in my thesis. I have studied the mathematics behind the method and I want to code a simple example to show how the classical SPH does not reproduce a constant field exactly. So, if we consider for example &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x,y)=\frac{1}{2}\left(\frac{\partial x}{\partial x}+\frac{\partial y}{\partial y}\right)=1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know how to get the following approximation of $f$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ATnYo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ATnYo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiar with Fortran, so I would prefer to use it for this example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help with this would be appreciated and thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="26742" LastEditorUserId="9667" LastEditDate="2018-02-01T21:17:30.137" LastActivityDate="2020-10-30T15:30:43.433" Title="implementation of a simple SPH example" Tags="|fortran|sph|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="28751" PostTypeId="1" AcceptedAnswerId="28757" CreationDate="2018-02-02T05:23:45.180" Score="1" ViewCount="1013" Body="&lt;p&gt;I am trying to implement the algorithm from the research paper &lt;a href=&quot;https://www.ri.cmu.edu/pub_files/2009/10/Calib.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Accurate Camera Calibration using Iterative Refinement of Control Points&quot;&lt;/a&gt; from Datta et al.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running many iterations does not show a convergence of the camera parameters.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The algorithm is defined as:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Detect calibration pattern control (also called image points) points (checkerboard corners, circle or ring centers) from a set of N images.&lt;/li&gt;&#xA;&lt;li&gt;Estimation of the camera parameters (intrinsic and extrinsic) using those control points.&lt;/li&gt;&#xA;&lt;li&gt;Loop until convergence&lt;br&gt;&#xA; a. Un-distort the input images using the estimated camera parameters and then re-project the un-distorted images to a canonical pattern.&lt;br&gt;&#xA; b. Localize the control points from the canonical pattern.&lt;br&gt;&#xA; c. Re-project the new control points using the previously estimated camera parameters.&lt;br&gt;&#xA; d. Use the new projected points to estimate one more time the camera parameters.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am using OpenCV and Python for this implementation. I use a standard camera calibration procedure for the two first steps. As a result, I get an initial estimation of the camera parameters.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;K: Intrinsic matrix [3x3]&lt;/li&gt;&#xA;&lt;li&gt;D: Lens distortion coefficient [1x5]&lt;/li&gt;&#xA;&lt;li&gt;rvec: rotation vector [Nx3]&lt;/li&gt;&#xA;&lt;li&gt;T: translation vector [Nx3]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem I see is that the focal lengths and principal points coordinates are slowly diverging and the RMS error for the camera calibration does not decrease as expected.&#xA;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Initialization Step&lt;/strong&gt;&lt;br&gt;&#xA;RMS=  0.2353278818397754&lt;br&gt;&#xA;Fx=1436.241, Fy=1435.402, Cx=597.9362, Cy=342.601, Dk1=0.065, Dk2=0.172&lt;br&gt;&#xA;&lt;strong&gt;Iterative Step 1&lt;/strong&gt;&lt;br&gt;&#xA;RMS=  0.24574215365963115&lt;br&gt;&#xA;Fx=1449.949, Fy=1446.909, Cx=597.509, Cy=341.320, Dk1=0.050, Dk2=0.556&lt;br&gt;&#xA;&lt;strong&gt;Iterative Step 2&lt;/strong&gt;&lt;br&gt;&#xA;RMS=  0.2451931123439023&lt;br&gt;&#xA;Fx=1474.403, Fy=1468.206, Cx=595.064, Cy=331.250, Dk1=0.043, Dk2=0.731&lt;br&gt;&#xA;&lt;strong&gt;Iterative Step 3&lt;/strong&gt;&lt;br&gt;&#xA;RMS=  0.24178717443873096&lt;br&gt;&#xA;Fx=1498.545, Fy=1489.129, Cx=595.771, Cy=320.921, Dk1=0.044, Dk2=0.773&lt;br&gt;&#xA;&lt;strong&gt;Iterative Step 4&lt;/strong&gt;&lt;br&gt;&#xA;RMS=  0.24339333469368066&lt;br&gt;&#xA;Fx=1526.091, Fy=1513.149, Cx=595.340, Cy=308.466, Dk1=0.040, Dk2=0.919&lt;br&gt;&#xA;&lt;strong&gt;Iterative Step 5&lt;/strong&gt;&lt;br&gt;&#xA;RMS=  0.24257897963694885&lt;br&gt;&#xA;Fx=1559.688, Fy=1543.050, Cx=595.479, Cy=293.856, Dk1=0.045, Dk2=0.968&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have checked each steps but I don't know where I am mistaking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please find my code below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import cv2 as cv&#xA;import numpy as np&#xA;&#xA;&#xA;# Find corners to remap the image to canonical pattern&#xA;def canonique(fname, corner_qty, criteria): &#xA;    img = cv.imread(fname)&#xA;    grayscale = cv.cvtColor(img, cv.COLOR_BGR2GRAY)&#xA;    image_size = grayscale.shape[::-1]&#xA;    ret, old_corners = cv.findChessboardCorners(grayscale, corner_qty, None)&#xA;&#xA;    if ret is True:&#xA;        cv.cornerSubPix(grayscale, old_corners, (11, 11), (-1, -1), criteria)&#xA;        old_corners = np.asarray(old_corners).reshape(-1, 2)&#xA;        temp = np.zeros((old_corners.shape[0], 1))&#xA;        old_corners = np.hstack((old_corners, temp))&#xA;        old_corners = np.asarray(old_corners).astype('float32')&#xA;&#xA;        af_trans = []&#xA;        width_step = float(image_size[0]) / float(corner_qty[0] + 1)&#xA;        height_step = float(image_size[1]) / float(corner_qty[1] + 1)&#xA;        af_trans.append([width_step, height_step])&#xA;        af_trans.append([image_size[0] - width_step, height_step])&#xA;        af_trans.append([width_step, image_size[1] - height_step])&#xA;        af_trans.append([image_size[0] - width_step, image_size[1] - height_step])&#xA;        af_trans = np.asarray(af_trans).astype('float32')&#xA;&#xA;        bf_trans = [old_corners[0, 0:2],                                       # top-left point&#xA;                    old_corners[corner_qty[0] - 1, 0:2],                       # top-right point&#xA;                    old_corners[(corner_qty[1] - 1) * corner_qty[0], 0:2],     # bottom-left point&#xA;                    old_corners[corner_qty[1] * corner_qty[0] - 1, 0:2]]       # bottom-right point&#xA;        bf_trans = np.asarray(bf_trans).astype('float32')&#xA;&#xA;        trans = cv.getPerspectiveTransform(bf_trans, af_trans)&#xA;        result = cv.warpPerspective(grayscale, trans, image_size)&#xA;        cv.imwrite(fname, result)&#xA;&#xA;&#xA;# Find corners in the list of images&#xA;def find_corners(filename, criteria, ctrl, corner_size, export_result=False, base_folder=''):&#xA;    image_size_read = (0, 0)&#xA;    control_pts, projected_pts = [], []&#xA;&#xA;    for index, fname in enumerate(filename):&#xA;        img = cv.imread(fname)&#xA;        gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)&#xA;        image_size_read = gray.shape[::-1]&#xA;        ret_find, corners = cv.findChessboardCorners(gray, corner_size, None)&#xA;&#xA;        if ret_find is True:&#xA;            control_pts.append(ctrl)&#xA;            cv.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)&#xA;            projected_pts.append(corners)&#xA;&#xA;            if export_result is True:&#xA;                cv.drawChessboardCorners(img, corner_size, corners, ret_find)&#xA;                cv.imwrite(base_folder + '/ID{}FindCorner.png'.format(index + 1), img)&#xA;&#xA;    return image_size_read, np.asarray(control_pts), projected_pts&#xA;&#xA;&#xA;def get_homography(k_matrix, r_vector, t_vector):&#xA;    r_matrix, _ = cv.Rodrigues(r_vector)&#xA;    hr = np.dot(k_matrix, np.dot(np.transpose(r_matrix), np.linalg.inv(k_matrix)))&#xA;&#xA;    c = -np.dot(np.transpose(r_matrix), t_vector)&#xA;    u0 = -np.dot(k_matrix, c) / c[2]&#xA;    u = np.zeros((3, 1))&#xA;&#xA;    u[0] = u0[0]&#xA;    u[1] = u0[1]&#xA;    u[2] = -u0[2]&#xA;&#xA;    ht = np.array([[1, 0], [0, 1], [0, 0]])&#xA;    ht = np.hstack((ht, u))&#xA;    h = np.dot(ht, hr)&#xA;&#xA;    return h&#xA;&#xA;&#xA;def normalize_control_points(ctrl_pts, image_res, corner_qty, norm_length):&#xA;    # Process the list of control points in each image&#xA;    for item, pt in enumerate(ctrl_pts):&#xA;        # the points are organized in the list by n_columns of n_rows points&#xA;        # We have&#xA;        #   corner_size[0] = n_rows&#xA;        #   corner_size[1] = n_columns&#xA;        pt[:, 0] *= float(corner_qty[0] + 1) * norm_length / image_res[0]&#xA;        pt[:, 1] *= float(corner_qty[1] + 1) * norm_length / image_res[1]&#xA;        ctrl_pts[item] = pt&#xA;&#xA;    return ctrl_pts&#xA;&#xA;&#xA;corner_size = (9, 6)  # Quantity of corners on the checker board&#xA;corner_length = 1.0&#xA;iteration_quantity = 20&#xA;image_quantity = 9&#xA;res_folder = './Results'&#xA;&#xA;# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0), ...&#xA;control = np.zeros((corner_size[0] * corner_size[1], 3), np.float32)&#xA;control[:, :2] = np.mgrid[1: corner_size[0] + 1,&#xA;                          1: corner_size[1] + 1].T.reshape(-1, 2) * corner_length&#xA;&#xA;TERMINATION_CRITERIA = (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 30, 0.001)&#xA;file = []&#xA;&#xA;# There should be image_quantity files for the camera&#xA;for i in range(image_quantity):&#xA;    file.append('./CheckerBoard-Images/checker_board_{:04d}_right.png'.format(i + 1))&#xA;&#xA;# Process all of them to retrieve the checker boards corners.&#xA;# This is the initialization step of the iterative algorithm&#xA;image_size, control_points, projected_points = find_corners(file,&#xA;                                                            TERMINATION_CRITERIA,&#xA;                                                            control,&#xA;                                                            corner_size,&#xA;                                                            True,&#xA;                                                            res_folder)&#xA;&#xA;control_points = np.asarray(control_points).astype('float32')&#xA;projected_points = np.asarray(projected_points).astype('float32')&#xA;&#xA;rms, K, D, rvec, T = cv.calibrateCamera(control_points,&#xA;                                        projected_points,&#xA;                                        image_size,&#xA;                                        None, None)&#xA;rvec = np.asarray(rvec)&#xA;T = np.asarray(T)&#xA;&#xA;# Compute the re-projection error to serve as baseline for further computation&#xA;print('Initialization Step')&#xA;print('Calibration RMS= ', rms)&#xA;print('Fx={}, Fy={}, Cx={}, Cy={}, Dk1={}, Dk2={}'.format(K[0, 0],&#xA;                                                          K[1, 1],&#xA;                                                          K[0, 2],&#xA;                                                          K[1, 2],&#xA;                                                          D[0, 0],&#xA;                                                          D[0, 1]))&#xA;&#xA;# From the research paper &quot;Accurate Camera Calibration using Iterative&#xA;# Refinement of Control Points&quot;, the following steps are&#xA;# LOOP START&#xA;# 1/ Un-distort and un-project: Use M/D/R/T to project the input images&#xA;#                               to a canonical pattern after removing distortions&#xA;# 2/ Localize control points: Localize calibration pattern control points&#xA;#                             in the canonical pattern&#xA;#&#xA;# 3/ Re-project: Project the control points using the estimated camera parameters&#xA;# 4/ Parameter Fitting: Use the projected control points to refine the camera&#xA;#                       parameters using Levenberg-Marquardt&#xA;# Note: THE CONVERGENCE CRITERIA IS NOT YET DEFINED&#xA;&#xA;for iteration in range(iteration_quantity):&#xA;    iter_filename = []&#xA;&#xA;    ################################################################################################&#xA;    # 1/ Un-distort and un-project&#xA;    for i in range(image_quantity):&#xA;        my_image = cv.imread(file[i])&#xA;        undist = cv.undistort(my_image, K, D)&#xA;        name = res_folder + '/ID{}UndistortIT{}.png'.format(i + 1, iteration)&#xA;        cv.imwrite(name, undist)&#xA;&#xA;        H = get_homography(K, rvec[i], T[i])&#xA;        result = cv.warpPerspective(undist, H, (1280, 720), flags=cv.INTER_LINEAR)&#xA;&#xA;        # For each image, we take the four &quot;out&quot; corners&#xA;        # Then we re-project them on a perfect mapping&#xA;&#xA;        name = res_folder + '/ID{}UnprojectIT{}.png'.format(i + 1, iteration)&#xA;        cv.imwrite(name, result)&#xA;        iter_filename.append(name)&#xA;        canonique(name, corner_size, TERMINATION_CRITERIA)&#xA;&#xA;    ################################################################################################&#xA;    # 2a/ Find the location of the new control points&#xA;    iter_ctrl_points = []&#xA;    iter_proj_points = []&#xA;&#xA;    for index, fname in enumerate(iter_filename):&#xA;        img = cv.imread(fname)&#xA;        grayscale = cv.cvtColor(img, cv.COLOR_BGR2GRAY)&#xA;        image_size_read = grayscale.shape[::-1]&#xA;        ret, corners = cv.findChessboardCorners(grayscale, corner_size, None)&#xA;&#xA;        if ret is True:&#xA;            cv.cornerSubPix(grayscale, corners, (11, 11), (-1, -1), TERMINATION_CRITERIA)&#xA;            cv.drawChessboardCorners(img, corner_size, corners, ret)&#xA;            cv.imwrite(res_folder + '/ID{}NewpointsIT{}.png'.format(index + 1, iteration), img)&#xA;&#xA;            corners = np.asarray(corners).reshape(-1, 2)&#xA;            temp = np.zeros((corners.shape[0], 1))&#xA;            corners = np.hstack((corners, temp))&#xA;        else:&#xA;            print('Error (It={}) on image {}'.format(iteration + 1, index + 1))&#xA;&#xA;        iter_ctrl_points.append(corners)&#xA;&#xA;    iter_ctrl_points = np.asarray(iter_ctrl_points).astype('float32')&#xA;&#xA;    # 2b/ Normalization of the control points&#xA;    iter_ctrl_points = normalize_control_points(iter_ctrl_points,&#xA;                                                image_size,&#xA;                                                corner_size,&#xA;                                                corner_length)&#xA;&#xA;    ################################################################################################&#xA;    # 3/ Project the new control points&#xA;    for i in range(len(iter_ctrl_points)):&#xA;        proj_points, _ = cv.projectPoints(iter_ctrl_points[i], rvec[i], T[i], K, D)&#xA;        iter_proj_points.append(proj_points)&#xA;&#xA;    ################################################################################################&#xA;    # 4/ Parameter fitting (new camera calibration)&#xA;    rms, K, D, rvec, T = cv.calibrateCamera(control_points,&#xA;                                            iter_proj_points,&#xA;                                            image_size, K, D)&#xA;&#xA;    rvec = np.asarray(rvec)&#xA;    T = np.asarray(T)&#xA;    print('Iterative Step {}'.format(iteration + 1))&#xA;    print('Calibration RMS= ', rms)&#xA;    print('Fx={}, Fy={}, Cx={}, Cy={}, Dk1={}, Dk2={}'.format(K[0, 0],&#xA;                                                              K[1, 1],&#xA;                                                              K[0, 2],&#xA;                                                              K[1, 2],&#xA;                                                              D[0, 0],&#xA;                                                              D[0, 1]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is an example of the input picture I am using.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gxSBJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gxSBJ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3754" LastEditorUserId="6910" LastEditDate="2018-02-03T23:10:52.237" LastActivityDate="2018-02-04T11:27:41.790" Title="Iterative camera calibration - No convergence" Tags="|algorithms|python|image-processing|computer-vision|3d|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28752" PostTypeId="1" CreationDate="2018-02-02T08:56:02.790" Score="0" ViewCount="107" Body="&lt;p&gt;Apologies if this is not the right forum to ask, but there has been &lt;a href=&quot;https://scicomp.stackexchange.com/questions/15884/apply-for-a-cluster-for-scientific-computing-from-a-developing-country&quot;&gt;a somewhat related question&lt;/a&gt; here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on a piece of software (nonlinear constrained optimisation, coded in C++11 and Python3) that I would like to try out on a Xeon Phi. Fujitsu Europe apparently &lt;a href=&quot;http://www.fujitsu.fr/page/remote-access-program&quot; rel=&quot;nofollow noreferrer&quot;&gt;offers free remote access&lt;/a&gt; to a Knight's Landing-based small cluster which, in their words on the webpage,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[...] is freely available to users who register and agree with the&#xA;  conditions of usage.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The cluster environment enables users to participate in activities&#xA;  such as benchmarking, code development, as well as code optimization&#xA;  and validation either using the pre-installed applications or those&#xA;  brought by the user.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This sounds too good to be true. And probably it is not true :-( , because I have tried to sign up 3 (three) times in the past 2 weeks, but received no reply whatsoever. I also wrote them an e-mail, inquiring about what happened to my sign-up requests: again, total silence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Has anyone succeeded getting an account on this Fujitsu cluster?&lt;/li&gt;&#xA;&lt;li&gt;If not, then can you suggest a similar freely available resource with Xeon Phi coprocessors for code development, testing and tuning?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If the tests show that my code can be accelerated by a Xeon Phi then I plan to convince my boss to purchase such a system. But without benchmarking numbers he won't open the purse strings (and rightly so, I must say).&lt;/p&gt;&#xA;" OwnerUserId="7024" LastActivityDate="2018-02-24T11:58:35.617" Title="Free access to Xeon Phi clusters" Tags="|optimization|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28753" PostTypeId="1" AcceptedAnswerId="29046" CreationDate="2018-02-02T11:42:51.090" Score="6" ViewCount="1605" Body="&lt;p&gt;Trying to study the error of FDM for a second order derivative versus step size I calculated the coefficients and validated them, but the output has errors for small step sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function in question is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f(x) = e^{\sin(x)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a derivative of&#xA;$$ f''(x) = \left(\cos^2(x) - \sin(x)\right)e^{\sin(x)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The calculated FDM&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial^{(2)}f}{\partial x^{(2)}}\approx&#xA;-\frac{5}{2h^2}f(x) - \frac{1}{12h^2}f(x-2h) + \frac{4}{3h^2}f(x-h) + \frac{4}{3h^2}f(x+h) - \frac{1}{12h^2}f(x+2h)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my Octave implementation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;stps = 1e-7:1e-6:3e-5;&#xA;outs = [];&#xA;x = pi/2;&#xA;for h = stps&#xA;  coefficients = [-5/2 -1/12 4/3 4/3 -1/12]./h^2;&#xA;  steps = [0 -2 -1 1 2].*h;&#xA;&#xA;  outs(end+1) = sum(coefficients .* exp(sin(x + steps))); &#xA;end&#xA;&#xA;plot(stps,outs, &quot;linewidth&quot;,1.5 , stps, ones(size(stps)).* (cos(x)^2 - sin(x))*exp(sin(x)) , &quot;linewidth&quot;,1.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which produces the following plot:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/R10Ay.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R10Ay.png&quot; alt=&quot;FDM output plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are those rounding errors caused by the nature of floating point numbers and operations or something else?&lt;/p&gt;&#xA;" OwnerUserId="26748" LastEditorUserId="20688" LastEditDate="2018-03-15T16:49:41.967" LastActivityDate="2018-03-15T16:49:41.967" Title="Finite difference method basic implementation on Octave" Tags="|numerical-analysis|finite-difference|floating-point|octave|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28754" PostTypeId="1" CreationDate="2018-02-02T22:43:06.203" Score="3" ViewCount="145" Body="&lt;p&gt;I have to find the orthogonal projection of a vector $b$ onto the matrix $A$ of size $m \times n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my application, I don't have the luxury of calculating the QR factorization. All I have are routines that give me the least squares solutions with minimum norm, i.e., ${A^\dagger b}$. I can, of course, use matrix multiplication. There are two ways to get $\mbox{proj}_{A}(b)$, $A (A^{\dagger}b)$ or $(A^T)^{\dagger}(A^Tb)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the better strategy for following cases in terms of backward stability?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$m&amp;gt;n$ and A is full rank.&lt;/li&gt;&#xA;&lt;li&gt;$m&amp;gt;n$ and A is not full rank.&lt;/li&gt;&#xA;&lt;li&gt;$n&amp;gt;m$ and A is not full rank.&lt;/li&gt;&#xA;&lt;li&gt;$n=1$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the last part, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5010/backward-stable-projection-and-normalization-of-a-vector&quot;&gt;this&lt;/a&gt; suggests that there is no backward stable algorithm on the account of the solution having an outer product. I am not convinced, though, because we never explicitly calculate the outer product.&lt;/p&gt;&#xA;" OwnerUserId="23066" LastEditorUserId="23066" LastEditDate="2018-02-03T18:42:18.403" LastActivityDate="2018-02-03T18:42:18.403" Title="Backward stable algorithm to get orthogonal projection onto the column space of a matrix" Tags="|linear-algebra|numerical-analysis|matrix|least-squares|projection|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28755" PostTypeId="2" ParentId="4928" CreationDate="2018-02-03T06:51:39.740" Score="2" Body="&lt;p&gt;This problem can be seen as a difference of convex functions (DC) programming problem. There is an extensive literature on DC programming, you can search related studies there. One of the most well known method is the DCA method, see for example:&#xA;&lt;a href=&quot;http://lma.univ-pau.fr/meet/mamern09/en/Lethi-MAMERN09.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lma.univ-pau.fr/meet/mamern09/en/Lethi-MAMERN09.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another recent paper that surveys the DC literature to some extent and might be handy is:&#xA;&lt;a href=&quot;https://arxiv.org/pdf/1511.01796.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1511.01796.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use some more general method for nonsmooth problems, for example the prox-based method given in:&#xA;&lt;a href=&quot;http://num.math.uni-goettingen.de/~ssabach/BST2013.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://num.math.uni-goettingen.de/~ssabach/BST2013.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26753" LastActivityDate="2018-02-03T06:51:39.740" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28757" PostTypeId="2" ParentId="28751" CreationDate="2018-02-03T23:09:36.420" Score="5" Body="&lt;p&gt;I have to tell you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented this algorithm (even double checked with other experienced people), however no luck. I guess, this worked for the authors, but with our data there was really no improvement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, once upon a time, the authors provided a code. I also tried it on my own dataset. Again, didn't work. Therefore, I am now sure that this method is tuned for particular datasets, and won't generalize. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, here is another work, which is maybe more promising:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;More accurate pinhole camera calibration with imperfect planar target&#xA;  &lt;em&gt;Klaus H. Strobl,   Gerd Hirzinger&lt;/em&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;http://elib.dlr.de/71888/1/strobl_2011iccv.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://elib.dlr.de/71888/1/strobl_2011iccv.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The algorithm might be implemented in &lt;a href=&quot;http://www.dlr.de/rm/en/desktopdefault.aspx/tabid-3925/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DLR calibration toolbox&lt;/a&gt;. I would suggest starting from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed better calibration is a serious topic for people to get vision working and no clear procedure is established on obtaining the best calibration. Maybe out of scope, but I would suggest to check works of photogrammetry / remote sensing scholars (e.g. ISPRS), which only at seldom assume linearized camera models.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2018-02-04T11:27:41.790" LastActivityDate="2018-02-04T11:27:41.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28759" PostTypeId="1" CreationDate="2018-02-04T02:01:36.807" Score="1" ViewCount="159" Body="&lt;p&gt;I've encountered an odd issue with solving for the eigenvalues of the following matrix, in Matlab format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ 0 1 0 1 0 0 0 0 0 0 0 0 1 1;&#xA;  1 0 0 0 0 0 0 0 0 0 0 0 0 0;&#xA;  0 1 0 1 0 0 0 0 0 0 0 0 0 0;&#xA;  0 0 1 0 0 0 0 0 0 0 0 0 0 0;&#xA;  0 0 0 0 0 1 0 0 0 0 0 0 0 0;&#xA;  0 0 0 0 1 0 0 0 0 0 0 0 0 0;&#xA;  0 0 0 0 0 0 0 1 0 0 0 0 0 0;&#xA;  0 0 0 0 0 0 1 0 0 0 0 0 0 0;&#xA;  0 0 0 0 0 0 0 0 0 1 0 0 0 0;&#xA;  0 0 0 0 0 0 0 0 1 0 0 1 0 0;&#xA;  0 0 0 0 0 0 0 0 1 0 0 1 0 0;&#xA;  0 0 0 0 0 0 0 0 0 0 1 0 0 0;&#xA;  0 1 0 0 0 0 0 0 0 0 0 0 1 1;&#xA;  0 1 0 0 0 0 0 0 0 0 0 0 1 1 ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When stored as a matrix of doubles, calling dgeev on this matrix correctly finds the eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when stored as a matrix of complex doubles (with all the imaginary components set to $0$), calling zgeev on it returns an error value of $13$, indicating that the QR algorithm failed to converge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strangely, multiplying this matrix by $i$ (so that the real components are $0$ instead) results in the correct eigenvalues being found again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is causing the QR algorithm not to converge in the first complex case, and is there a way to get around it?&lt;/p&gt;&#xA;" OwnerUserId="26761" LastActivityDate="2018-02-04T18:09:26.617" Title="LAPACK non-convergent eigenvalue algorithm for complex but not double matrix" Tags="|linear-algebra|eigenvalues|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28760" PostTypeId="2" ParentId="11276" CreationDate="2018-02-04T03:14:37.493" Score="6" Body="&lt;p&gt;I think this new &lt;a href=&quot;https://github.com/tensor-compiler/taco&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;taco&lt;/code&gt;&lt;/a&gt; lib is really good too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Tensor Algebra Compiler (taco) is a C++ library that computes tensor algebra expressions on sparse and dense tensors. It uses novel compiler techniques to get performance competitive with hand-optimized kernels in widely used libraries for both sparse tensor algebra and sparse linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use taco as a C++ library that lets you load tensors, read tensors from files, and compute tensor expressions. You can also use taco as a code generator that generates C functions that compute tensor expressions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Talk: &lt;a href=&quot;https://www.youtube.com/watch?v=Kffbzf9etLE&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=Kffbzf9etLE&lt;/a&gt;&#xA;Paper: &lt;a href=&quot;http://tensor-compiler.org/kjolstad-oopsla17-tensor-compiler.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://tensor-compiler.org/kjolstad-oopsla17-tensor-compiler.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26762" LastEditorUserId="26762" LastEditDate="2018-02-05T06:03:47.557" LastActivityDate="2018-02-05T06:03:47.557" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28761" PostTypeId="2" ParentId="28759" CreationDate="2018-02-04T03:19:36.847" Score="4" Body="&lt;p&gt;Your matrix is not diagonalizable, in the Jordan decomposition of it there is a block for the eigenvalue $0$ of the form&#xA;$$\begin{pmatrix}0&amp;amp;0&amp;amp;0\\0&amp;amp;0&amp;amp;1\\0&amp;amp;0&amp;amp;0\end{pmatrix},$$&#xA;meaning a triple zero eigenvalue with only two eigenvectors.&#xA;If you compute the matrix of eigenvectors numerically it has condition number $\kappa(X)\sim 10^{16}$ (the actual value is $\kappa(X)=\infty$), which immediately tells you something is wrong because $\kappa(X)$ determines the sensitivity of eigenvalues to perturbations in the original matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Lapack algorithms probably assume the matrix is diagonalizable, in which case it is sensible for them to give an error. Whether it gives an error or not probably depends on effects of roundoff errors and doesn't have much to do with multiplying by $i$, I wouldn't read much into that.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2018-02-04T18:09:26.617" LastActivityDate="2018-02-04T18:09:26.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28764" PostTypeId="1" AcceptedAnswerId="30361" CreationDate="2018-02-04T18:00:04.160" Score="5" ViewCount="294" Body="&lt;p&gt;We have a fairly large (maybe 1000 equations) differential-algebraic equation model written in ACSLX, an obsolete modelling environment similar to Modelica. The model represents the evolution of a biological system through time, with several forcing functions such as atmospheric temperature which are usually provided on a daily or sub-daily timestep. ACSLX translates the model, including a DAE solver, to C, which is then compiled into a DLL. We run this DLL inside a somewhat complex GUI, often in batch mode consisting of ~100 simulation years. We are in Windows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We need to convert this model into a new language/environment for future-proofing reasons, and are debating the best option.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it should be free (to avoid the expense and licensing pain associated with things like Matlab).&lt;/li&gt;&#xA;&lt;li&gt;it should be well supported and relatively popular&lt;/li&gt;&#xA;&lt;li&gt;the model code should be able to be easily read by biological scientists&lt;/li&gt;&#xA;&lt;li&gt;it would be advantageous if it were able to be run within a development environment such as RStudio or Spyder&lt;/li&gt;&#xA;&lt;li&gt;it should be able to be called from our GUI (e.g. as a DLL)&lt;/li&gt;&#xA;&lt;li&gt;it should be fast &lt;/li&gt;&#xA;&lt;li&gt;it should facilitate model calibration, sensitivity analysis, etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We currently use C++, Fortran, Javascript, R, SmallTalk for various other projects. Other options seem to be Modelica, Python, C, Java, C#. It's not clear whether we need a strong DE solver like ODEPACK or whether we could write our own, simple time stepper, since the forcing function tends to negate the benefit of efficient ODE solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a similar model which is written in Fortran 90 and compiles to a DLL which we run/calibrate from RStudio. This works quite well, although Fortran expertise is hard to come by and it's a little hard to debug. Spyder/Python would allow both sides to be done in a single language, including debugging, although the performance hit might be a problem. Although Cython or Dask might help there. Modelica seems like another option, although support seems patchy, and it might be slow to run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We would really appreciate any suggestions/thoughts/experience from the SO community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help. &lt;/p&gt;&#xA;" OwnerUserId="26265" LastEditorUserId="26265" LastEditDate="2018-10-17T19:09:22.000" LastActivityDate="2020-07-19T17:53:48.640" Title="Recommended language/environment for large scale semi-continuous biological models" Tags="|python|c++|fortran|r|openmodelica|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28765" PostTypeId="2" ParentId="28764" CreationDate="2018-02-04T18:53:06.073" Score="9" Body="&lt;p&gt;You should consider giving Julia a try. Let me explain what's going on in the design space right now that would be of interest to you. Full disclosure I am the lead developer of JuliaDiffEq.&lt;/p&gt;&#xA;&lt;p&gt;JuliaDiffEq and &lt;a href=&quot;http://diffeq.sciml.ai/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt; has a large feature set dedicated to efficiently integrating computationally-difficult differential equations. &lt;a href=&quot;http://diffeq.sciml.ai/latest/tutorials/ode_example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;It has a simple high level interface&lt;/a&gt; in a high-level language of growing popularity, so it has a low barrier to entry. It can be used via development tools like &lt;a href=&quot;http://junolab.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Juno&lt;/a&gt; or &lt;a href=&quot;https://github.com/JuliaEditorSupport/julia-vscode&quot; rel=&quot;nofollow noreferrer&quot;&gt;VSCode&lt;/a&gt;. The Julia programming language has all of the mutation features you'd expect in a low level language &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqTutorials.jl/blob/master/Introduction/OptimizingDiffEqCode.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;to optimize your derivative functions to be non-allocating and fast&lt;/a&gt; in ways you can't do with higher level &amp;quot;vectorized code&amp;quot;. It compiles all of your code (along with doing things like interprocedural optimization and inlining) to get you the speed you'd expect from standard (type-stable) Julia functions.&lt;/p&gt;&#xA;&lt;p&gt;And the key is that the stack goes very deep. You mentioned you want access to ODEPACK, well this will give you access to &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a lot of ODE solvers&lt;/a&gt; and &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/dae_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a lot of DAE solvers&lt;/a&gt; which includes Netlib and ODEPACK items like &lt;a href=&quot;https://github.com/JuliaDiffEq/DASKR.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DASKR&lt;/a&gt;, &lt;a href=&quot;https://github.com/rveltz/LSODA.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;LSODA&lt;/a&gt;, and &lt;a href=&quot;https://github.com/JuliaDiffEq/Sundials.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sundials&lt;/a&gt;. For example, the components like &lt;a href=&quot;https://github.com/JuliaDiffEq/Sundials.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sundials.jl&lt;/a&gt; are direct auto-generated interfaces directly to the underlying C++ libraries, so you can &lt;a href=&quot;https://github.com/JuliaDiffEq/Sundials.jl/blob/master/test/ida_Roberts_dns.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;directly translate any IDA C++ code to Julia code and use that&lt;/a&gt;. You can use that to directly call the sensitivity analysis and preconditioner features of Sundials, or use them through the higher level interface. Our ecosystem includes high level features like &lt;a href=&quot;http://diffeq.sciml.ai/latest/analysis/sensitivity.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sensitivity analysis&lt;/a&gt; and &lt;a href=&quot;http://diffeq.sciml.ai/latest/analysis/parameter_estimation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;parameter estimation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;That's if you want to write the differential equations directly. The step above that is modeling features. There are modeling libraries being built on top of the JuliaDiffEq components. We have a &lt;a href=&quot;http://diffeq.sciml.ai/latest/models/biological.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;small chemical reaction network modeling setup being created&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But it's likely not what you're looking for. What you want is something like Modelica, and that is being created as &lt;a href=&quot;https://github.com/ModiaSim/Modia.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Modia.jl&lt;/a&gt;. An overview &lt;a href=&quot;https://www.youtube.com/watch?v=hVg1eL1Qkws&amp;amp;feature=youtu.be&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be found in this video&lt;/a&gt;. That is being built on top of Sundials's IDA and does things like &lt;a href=&quot;https://www.modelica.org/events/modelica2017/proceedings/html/submissions/ecp17132565_OtterElmqvist.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;index reduction&lt;/a&gt; automatically. In fact, it's being developed by the developers of Modelica as the next way forward. It's not released quite yet since there's a few things we are cleaning up (like &lt;a href=&quot;https://github.com/JuliaDiffEq/SundialsBuilder/issues/3&quot; rel=&quot;nofollow noreferrer&quot;&gt;compiling the KLU sparse linear solvers into our auto-generated Sundials binaries since these are the most efficient choice for this application&lt;/a&gt;), but IMO Hilding's &amp;quot;new Modelica&amp;quot; already looks great so I hope it can be released soon.&lt;/p&gt;&#xA;&lt;p&gt;Also, Julia is a language that can be compiled. This is at a very early stage, but ahead-of-time compilation (AOT) tools like &lt;a href=&quot;https://github.com/JuliaComputing/static-julia&quot; rel=&quot;nofollow noreferrer&quot;&gt;static-julia&lt;/a&gt; and &lt;a href=&quot;https://github.com/SimonDanisch/PackageCompiler.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;PackageCompiler.jl&lt;/a&gt; show this in action.&lt;/p&gt;&#xA;&lt;p&gt;Lastly, it's all free, has &lt;a href=&quot;https://discourse.julialang.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an active community&lt;/a&gt; along with &lt;a href=&quot;https://slackinvite.julialang.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;chatrooms&lt;/a&gt; (and one specifically for &lt;a href=&quot;https://gitter.im/JuliaDiffEq/Lobby&quot; rel=&quot;nofollow noreferrer&quot;&gt;differential equations&lt;/a&gt;). The team is quite large: we have thirty people on the team, and had &lt;a href=&quot;https://summerofcode.withgoogle.com/archive/2017/organizations/6682672759832576/&quot; rel=&quot;nofollow noreferrer&quot;&gt;10 mentors for 6 Google Summer of Code projects&lt;/a&gt; specifically related to differential equations in Julia. So while young, that demonstrates a lot more developer support and activity than many (most?) other open source differential equation ecosystems. And anything that's missing can be written in Julia (and be fast etc.). I think that hits your points, but if not you can always leave &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl/issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;a feature request&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T17:53:48.640" LastActivityDate="2020-07-19T17:53:48.640" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="28766" PostTypeId="1" AcceptedAnswerId="28768" CreationDate="2018-02-05T10:55:02.367" Score="0" ViewCount="1317" Body="&lt;p&gt;I want to solve the following differential equation from a paper with the boundary condition:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VWKFr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VWKFr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper used the Crank–Nicolson method for solving it. I think I understand the method after googling it, but most websites discussing it use the heat equation as an example. Here we can replace the usual &lt;code&gt;t&lt;/code&gt; variable with &lt;code&gt;Xi&lt;/code&gt;, and the usual &lt;code&gt;x&lt;/code&gt; as &lt;code&gt;Rho&lt;/code&gt;. Now unlike the heat equation, this equation has an additional first derivative of &lt;code&gt;Xi&lt;/code&gt; term (the first term gives both first and second derivative of &lt;code&gt;Xi&lt;/code&gt;). Does it matter what finite difference scheme I use to approximate this term?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing is that the equation has nonlinear terms involving &lt;code&gt;|A|^2&lt;/code&gt; and &lt;code&gt;|A|^4&lt;/code&gt;. How can I incorporate this to the matrix equation I am going to solve? (PS: &lt;code&gt;A&lt;/code&gt; is complex)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am pretty new to numerical methods so forgive me if I have neglected anything significant.&lt;/p&gt;&#xA;" OwnerUserId="21426" LastActivityDate="2018-02-09T11:12:07.270" Title="Crank–Nicolson method for nonlinear differential equation" Tags="|pde|numerical-analysis|nonlinear-equations|differential-equations|crank-nicolson|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28767" PostTypeId="1" CreationDate="2018-02-05T12:34:30.080" Score="2" ViewCount="75" Body="&lt;p&gt;I would like to use the PDD algorithm developed by &lt;a href=&quot;http://www.dtic.mil/dtic/tr/fulltext/u2/a263151.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sun&lt;/a&gt; to solve tridiagonal matrices in parallel for the following compact finite difference scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\begin{align}&#xA;\dfrac{1}{4}f^{'}_{i-1} + f^{'}_i + \dfrac{1}{4}f^{'}_{i+1} &amp;amp;= \dfrac{3}{4} \dfrac{-f_{i-1} + f_{i+1}}{dx} \; \; \; \; \;\;\;\;\;\;\;\;\;\; \text{(Interior Points)}&#xA;\\&#xA;f^{'}_0 + 3f^{'}_1 &amp;amp;= \dfrac{-17 f_0 + 9 f_1 + 9 f_2 - f_3}{6 \Delta x} \; \; \; \text{(Boundary points)}&#xA;\end{align}&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PDD algorithm consists in splitting the diagonal matrix into a block diagonal, $\tilde{A}$, and offset matrix, $\Delta A$. The following example is for a 8 grid points domain divided into 2 sub-domains (processors):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\begin{align}&#xA;A &amp;amp;= \begin{bmatrix}&#xA;    1       &amp;amp; 3 &amp;amp; &amp;amp; &amp;amp;  \\&#xA;    \alpha       &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;\alpha       &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;\alpha       &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;&amp;amp;\alpha       &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;\alpha       &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp; \alpha &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp; 3 &amp;amp; 1 &#xA;     \end{bmatrix} \\&#xA;&amp;amp;= \begin{bmatrix}&#xA;    1       &amp;amp; 3 &amp;amp; &amp;amp; &amp;amp;  \\&#xA;    \alpha       &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;\alpha       &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;\alpha       &amp;amp; 1 &amp;amp;  \\&#xA;    &amp;amp;&amp;amp;&amp;amp;        &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;\alpha       &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp; \alpha &amp;amp; 1 &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp; 3 &amp;amp; 1 &#xA;     \end{bmatrix} + &#xA;    \begin{bmatrix}&#xA;    0&amp;amp; &amp;amp; &amp;amp; &amp;amp;  \\&#xA;    &amp;amp;  0    &amp;amp;  &amp;amp;  \\&#xA;    &amp;amp;&amp;amp;   0    &amp;amp;  &amp;amp;  \\&#xA;    &amp;amp;&amp;amp;&amp;amp;  0       &amp;amp; \alpha \\&#xA;    &amp;amp;&amp;amp;&amp;amp;  \alpha      &amp;amp; 0 &amp;amp;  \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;   0   &amp;amp;   \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp; 0 &amp;amp;    \\&#xA;    &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;  0 &#xA;    \end{bmatrix} \\&#xA;\end{align} \\&#xA;\Delta A = [\alpha \, e_4, \alpha \, e_3][e^T_3, e^T_4] = VE^T&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $e_i$ represents a column vector with its ith elements being 1 and zero otherwise for $ 0 &amp;lt; i &amp;lt; n-1$. With this splitting, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;x^{-1} = A^{-1} d = (\tilde{A} + VE^T)^{-1} d = \tilde{A}^{-1} d - \tilde{A}^{-1}(1+E^T\tilde{A}^{-1}V)^{-1}E^T\tilde{A}^{-1}d&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This generally leads to have a pentadiagonal reduced matrix solved in parallel. However, the author shows that for a sufficiently large number of grid points per processors, the block-diagonal terms are below machine precision and the matrix reduces into 2x2 independent block problems. This is even more true for diagonally dominant matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm consists of the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve independently on each $i$ processor: $A_i[\tilde{x}^i, v^i, w^i] = [d^i, \alpha e_0, \alpha e_{m-1}]$&lt;/li&gt;&#xA;&lt;li&gt;Send $\tilde{x}^i_0, v_0^i$ from the $i^{\text{th}}$ node to the $(i-1)^{\text{th}})$ node.&lt;/li&gt;&#xA;&lt;li&gt;Solve:&#xA;$&#xA;\begin{bmatrix}&#xA;1 &amp;amp; w_{m-1}^i \\ v_0^{i+1} &amp;amp; 1&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;y_{2i} \\ y_{2i+1}&#xA;\end{bmatrix} = &#xA;\begin{bmatrix} \tilde{x}^i_{m-1} \\ \tilde{x}_0^{i+1}\end{bmatrix}&#xA;$&lt;/li&gt;&#xA;&lt;li&gt;Send $y_{2i+1}$ from the $i^{\text{th}}$ to the $(i+1)^{\text{th}}$ node.&lt;/li&gt;&#xA;&lt;li&gt;$x^i = \tilde{x}^i - \begin{bmatrix}v^i, w^i\end{bmatrix} \begin{bmatrix} y_{2i-1} \\ y_{2i} \end{bmatrix}$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I wrote a MPI fortran code which follows the algorithm. I tested the code with explicit finite difference and also in serial to make sure the parallelism and compact scheme were implemented correctly. In parallel, I am getting large errors right at the nodes boundary. I have verified that my implementation follows the algorithm perfectly. Anyone know where the problem may lie?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the standalone PDD algorithm for the compact scheme:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine tridiag_parallel(nPoints, dx, phi, dphi, src, des)&#xA;&#xA;    integer,  intent(in) :: nPoints, src, des                                                     &#xA;    real(dp), intent(in) :: dx                                                                    &#xA;    real(dp), dimension(-buf:), intent(in)  :: phi                                                &#xA;    real(dp), dimension(   0:), intent(out) :: dphi                                               &#xA;    integer  :: ii, min_x, max_x                                                                  &#xA;    integer  :: recv_rqst, send_rqst                                                              &#xA;    real(dp) :: a_im, c_ip1m, x_recv, v_recv                                                      &#xA;    real(dp) :: y_2im1, y_2i, y_2ip1                                                              &#xA;    real(dp), dimension(0:nPoints-1) :: Ai_low, Ai_mid, Ai_upp, e_m                               &#xA;    real(dp), dimension(0:nPoints-1) :: x_tilde, v, w, rhs, rhs_v, rhs_w                          &#xA;&#xA;    Ai_low = alpha; Ai_mid = 1._dp; Ai_upp = alpha                                                &#xA;    y_2i = 0._dp; y_2ip1 = 0._dp; y_2im1 = 0._dp; x_recv = 0._dp; v_recv = 0._dp                  &#xA;&#xA;    ! Build RHS                                                                                   &#xA;    min_x = 0; max_x = nPoints-1                                                                  &#xA;    if(src == mpi_proc_null) then                                                                 &#xA;        min_x = 1; Ai_upp(0) = 3._dp&#xA;        rhs(0) = 1._dp/(6._dp*dx) * (-17._dp*phi(0) + 9._dp*(phi(1)+phi(2)) - phi(3))             &#xA;    end if                                                                                        &#xA;&#xA;    if(des == mpi_proc_null) then                                                                 &#xA;        max_x = nPoints-2; Ai_low(nPoints-1) = 3._dp                                              &#xA;&#xA;        rhs(nPoints-1) = 1._dp/(6._dp*dx) * (phi(nPoints-4) - 9._dp*(phi(nPoints-3)+phi(nPoints-2)) &amp;amp;&#xA;                                        + 17._dp*phi(nPoints-1))                                     &#xA;    end if                                                                                           &#xA;&#xA;    do ii=min_x,max_x                                                                                &#xA;        rhs(ii) = 3._dp/(4._dp*dx) * (-phi(ii-1) + phi(ii+1))                                        &#xA;    end do                                                                                        &#xA;&#xA;    ! Build a_im, c_im, e_m, rhs_v, rhs_w                                                            &#xA;    a_im = alpha; c_ip1m = alpha; e_m = 0._dp                                                        &#xA;    if(src == mpi_proc_null) a_im   = 0._dp                                                          &#xA;    if(des == mpi_proc_null) c_ip1m = 0._dp&#xA;&#xA;    rhs_v = 0._dp; rhs_v(0)         = a_im                                                           &#xA;    rhs_w = 0._dp; rhs_w(nPoints-1) = c_ip1m                                                      &#xA;&#xA;    ! Solve for x_tilde, v and w                                                                  &#xA;    call tridiag(nPoints, Ai_low, Ai_mid, Ai_upp, rhs,   x_tilde)                                 &#xA;    call tridiag(nPoints, Ai_low, Ai_mid, Ai_upp, rhs_v, v)                                       &#xA;    call tridiag(nPoints, Ai_low, Ai_mid, Ai_upp, rhs_w, w)                                       &#xA;&#xA;    ! First communication: send tilde{x_0}, v_0 to previous processor                                                                         &#xA;    call mpi_isend(x_tilde(0), 1, mpi_dp, src, 200, mpi_comm_world, send_rqst, ierror)            &#xA;    call mpi_irecv(x_recv,     1, mpi_dp, des, 200, mpi_comm_world, recv_rqst, ierror)            &#xA;    call mpi_wait(send_rqst, stat, ierror); call mpi_wait(recv_rqst, stat, ierror)                &#xA;&#xA;    call mpi_isend(v(0),   1, mpi_dp, src, 300, mpi_comm_world, send_rqst, ierror)                &#xA;    call mpi_irecv(v_recv, 1, mpi_dp, des, 300, mpi_comm_world, recv_rqst, ierror)                &#xA;    call mpi_wait(send_rqst, stat, ierror); call mpi_wait(recv_rqst, stat, ierror)                &#xA;&#xA;    ! Solve for y_2i, y_2ip1                                                                      &#xA;    y_2i   = (x_tilde(nPoints-1) - w(nPoints-1)*x_recv) / (1._dp - w(nPoints-1)*v_recv)&#xA;    y_2ip1 = x_recv - v_recv * y_2i&#xA;&#xA;    ! Second communication: send y_2i to next processor                                           &#xA;    call mpi_isend(y_2i  , 1, mpi_dp, des, 400, mpi_comm_world, send_rqst, ierror)                &#xA;    call mpi_irecv(y_2im1, 1, mpi_dp, src, 400, mpi_comm_world, recv_rqst, ierror)                &#xA;    call mpi_wait(send_rqst, stat, ierror); call mpi_wait(recv_rqst, stat, ierror)                                                        &#xA;&#xA;    ! Derivative                                                                                  &#xA;    do ii=0,nPoints-1                                                                             &#xA;        dphi(ii) = x_tilde(ii) - (v(ii)*y_2im1 + w(ii)*y_2i)                                      &#xA;    end do                                                                                        &#xA;end subroutine tridiag_parallel&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Below is the actual program (not sure if necessary but here it is anyway):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program main                                                                                          &#xA;    use, intrinsic :: iso_fortran_env, dp =&amp;gt; real64                                                   &#xA;    use mpi, mpi_dp =&amp;gt; mpi_double_precision, stat =&amp;gt; mpi_status_ignore                                &#xA;    implicit none                                                                                     &#xA;&#xA;    ! Grid                                                                                            &#xA;    integer  :: nx, grid_nx                                                                           &#xA;    integer, parameter :: buf = 2                                                                     &#xA;    real(dp) :: dx, proc_xmin                                                                         &#xA;    real(dp), parameter :: grid_xmin = -1._dp, grid_xmax = 1._dp                                      &#xA;    real(dp), allocatable, dimension(:) :: x, x_grid                                                  &#xA;&#xA;    ! MPI                                                                                             &#xA;    integer :: nProcs, myrank, ierror, Xsrc, Xdes                                                     &#xA;    logical :: master_proc = .false.                                                                  &#xA;&#xA;    ! Compact                                                                                         &#xA;    real(dp), parameter :: alpha = 1._dp/4._dp                                                        &#xA;&#xA;    ! Subscripts =&amp;gt; s: serial, p: parallel, e: exact                                                  &#xA;    real(dp), allocatable, dimension(:) :: u_s, u_p, dudx_s, dudx_p, dudx_e                           &#xA;    integer :: ii, nn, send_rqst, recv_rqst                                                           &#xA;&#xA;    ! Init mpi                                                                                        &#xA;    call mpi_init(ierror)                                                                             &#xA;    call mpi_comm_size(mpi_comm_world, nProcs, ierror)                                                &#xA;    call mpi_comm_rank(mpi_comm_world, myrank, ierror)                                                &#xA;    if(myrank == 0) master_proc = .true.                                                              &#xA;&#xA;    call init_grid                                                                                    &#xA;&#xA;    ! Serial                                                                                          &#xA;    if(master_proc) then                                                                              &#xA;        Xsrc = mpi_proc_null; Xdes = mpi_proc_null                                                    &#xA;        call setup_case(grid_nx, x_grid, u_s, dudx_s)                                                 &#xA;        call tridiag_parallel(grid_nx, dx, u_s, dudx_s, Xsrc, Xdes)                                   &#xA;        !call FD2_x(grid_nx, dx, u_s, dudx_s, Xsrc, Xdes)                                             &#xA;    end if                                                                                            &#xA;&#xA;    ! Parallel                                                                                        &#xA;    call init_blocks                                                                                  &#xA;    call setup_case(nx, x, u_p, dudx_p)                                                               &#xA;    call tridiag_parallel(nx, dx, u_p, dudx_p, Xsrc, Xdes)                                            &#xA;    !call FD2_x(nx, dx, u_p, dudx_p, Xsrc, Xdes)                                                      &#xA;    call compute_exact                                                                                &#xA;&#xA;    ! Compare                                                                                         &#xA;    call compare(dudx_s, dudx_p)                                                                      &#xA;    call compute_errors(dudx_e, dudx_p)                                                               &#xA;&#xA;    ! Finalize                                                                                        &#xA;    call mpi_finalize(ierror)&#xA;contains&#xA;subroutine init_grid                                                                              &#xA;    ! Read in grid size                                                                           &#xA;    if(master_proc) then                                                                          &#xA;        print'(A)', &quot;Enter the total number of grid points&quot;                                       &#xA;        read(*,*) grid_nx                                                                         &#xA;        do nn=1,nProcs-1                                                                          &#xA;            call mpi_isend(grid_nx, 1, mpi_int, nn, 100, mpi_comm_world, send_rqst, ierror)       &#xA;            call mpi_wait(send_rqst, stat, ierror)                                                &#xA;        end do                                                                                    &#xA;    else                                                                                          &#xA;        call mpi_irecv(grid_nx, 1, mpi_int, 0, 100, mpi_comm_world, recv_rqst, ierror)            &#xA;        call mpi_wait(recv_rqst, stat, ierror)                                                    &#xA;    end if                                                                                        &#xA;&#xA;    if(mod(grid_nx, nProcs) /= 0) then                                                            &#xA;        print'(A)', &quot;The number of points is not divisible by the number of processors.&quot;          &#xA;        print'(A)', &quot;Stopping the simulation&quot;                                                     &#xA;        call mpi_abort(mpi_comm_world, 10, ierror)                                                &#xA;    end if                                                                                        &#xA;&#xA;    dx = (grid_xmax - grid_xmin) / real(grid_nx - 1, dp)                                          &#xA;    nx = grid_nx / nProcs                                                                         &#xA;    proc_xmin = grid_xmin + mod(myrank, nProcs) * dx * nx                                         &#xA;&#xA;    allocate(x(0:nx-1), x_grid(0:grid_nx-1))                                                      &#xA;    x = [ (proc_xmin + ii*dx, ii=0,nx-1) ]                                                        &#xA;    x_grid = [ (grid_xmin + ii*dx, ii=0,grid_nx-1) ]                                              &#xA;end subroutine init_grid                                                                          &#xA;&#xA;subroutine init_blocks                                                                            &#xA;    Xsrc = myrank-1; Xdes = myrank+1                                                              &#xA;    if(mod(myrank,   nProcs) == 0) Xsrc = mpi_proc_null                                           &#xA;    if(mod(myrank+1, nProcs) == 0) Xdes = mpi_proc_null                                           &#xA;end subroutine                                                                                    &#xA;&#xA;subroutine apply_bc(u)                                                                            &#xA;    real(dp), dimension(-2:), intent(inout) :: u                                                  &#xA;    call mpi_sendrecv(u(0),  2, mpi_dp, Xsrc, 100,  &amp;amp;                                             &#xA;                      u(nx), 2, mpi_dp, Xdes, 100, mpi_comm_world, stat, ierror)                  &#xA;    call mpi_sendrecv(u(nx-2), 2, mpi_dp, Xdes, 200, &amp;amp;                                            &#xA;                      u(-2),   2, mpi_dp, Xsrc, 200, mpi_comm_world, stat, ierror)                &#xA;end subroutine apply_bc                                                                           &#xA;&#xA;subroutine setup_case(nx, x, u, dudx)                                                             &#xA;    integer,  intent(in) :: nx                                                                    &#xA;    real(dp), dimension(0:), intent(in) :: x                                                      &#xA;    real(dp), allocatable, dimension(:), intent(out) :: u, dudx                                   &#xA;&#xA;    allocate(u(-2:nx+1), dudx(0:nx-1)); u = 0._dp                                                 &#xA;    u(0:nx-1) = [(exp(-10._dp*x(ii)**2), ii=0,nx-1)]                                              &#xA;    call apply_bc(u)                                                                              &#xA;end subroutine setup_case                                                                         &#xA;&#xA;subroutine compute_exact                                                                          &#xA;    allocate(dudx_e(0:nx-1))                                                                      &#xA;&#xA;    dudx_e = [(-20._dp*x(ii)*exp(-10._dp*x(ii)**2), ii=0,nx-1)]                                   &#xA;end subroutine compute_exact&#xA;&#xA;! 1st order derivative, 2nd order explicit central difference                                     &#xA;subroutine FD2_x(nx, dx, phi, dphi, Xsrc, Xdes)                                                   &#xA;    integer,  intent(in) :: nx, Xsrc, Xdes                                                        &#xA;    real(dp), intent(in) :: dx                                                                    &#xA;    real(dp), dimension(-2:), intent(in)    :: phi                                                &#xA;    real(dp), dimension( 0:), intent(inout) :: dphi                                               &#xA;    integer :: nn, min_x, max_x                                                                   &#xA;&#xA;    min_x = 0; max_x = nx-1                                                                       &#xA;&#xA;    ! BC                                                                                          &#xA;    if(Xsrc == mpi_proc_null) then                                                                &#xA;        min_x = 1                                                                                 &#xA;        dphi(0) = (phi(1)-phi(0)) / dx                                                            &#xA;    end if                                                                                        &#xA;    if(Xdes == mpi_proc_null) then                                                                &#xA;        max_x = nx-2                                                                              &#xA;        dphi(nx-1) = (-phi(nx-1)+phi(nx-2)) / dx                                                  &#xA;    end if                                                                                        &#xA;&#xA;    ! Interior points                                                                             &#xA;    do nn=min_x,max_x                                                                             &#xA;        dphi(nn) = (phi(nn+1)-phi(nn-1)) / (2._dp*dx)                                             &#xA;    end do                                                                                        &#xA;end subroutine FD2_x&#xA;&#xA;subroutine tridiag(n, a, b, c, r, u)                                                              &#xA;    integer, intent(in) :: n                                                                      &#xA;    real(dp), dimension(0:), intent(in)  :: a, b, c, r                                            &#xA;    real(dp), dimension(0:), intent(out) :: u                                                     &#xA;    real(dp), dimension(0:n-1) :: gam                                                             &#xA;    integer  :: j                                                                                 &#xA;    real(dp) :: bet                                                                               &#xA;&#xA;    bet  = b(0)                                                                                   &#xA;    u(0) = r(0)/bet                                                                               &#xA;&#xA;    do j = 1,n-1                                                                                  &#xA;        gam(j) = c(j-1) / bet                                                                     &#xA;        bet    = b(j) - a(j) * gam(j)                                                             &#xA;&#xA;        u(j) = (r(j) - a(j) * u(j-1))/bet                                                         &#xA;    end do                                                                                        &#xA;&#xA;    do j =n-2,0,-1                                                                                &#xA;        u(j) = u(j) - gam(j+1) * u(j+1)                                                           &#xA;    end do                                                                                        &#xA;end subroutine tridiag                                                                            &#xA;&#xA;subroutine compare(serial, parallel)                                                              &#xA;    real(dp), dimension(0:), intent(in) :: serial, parallel                                       &#xA;    integer  :: proc_id                                                                           &#xA;    real(dp) :: Ltwo, Linf                                                                        &#xA;    real(dp), dimension(0:nx-1,0:nProcs-1) :: U_all                                               &#xA;    real(dp), dimension(0:grid_nx-1) :: grid_parallel                                             &#xA;&#xA;    if(master_proc) then                                                                          &#xA;        U_all(:,0) = parallel(:)                                                                  &#xA;        do proc_id=1,nProcs-1                                                                     &#xA;            call mpi_recv(U_all(:,proc_id), nx, mpi_dp, proc_id, proc_id+100, mpi_comm_world,   &amp;amp; &#xA;                            stat, ierror)                                                         &#xA;        end do                                                                                    &#xA;    else                                                                                          &#xA;        call mpi_send(parallel(0:nx-1), nx, mpi_dp, 0, myrank+100, mpi_comm_world, ierror)        &#xA;    end if                                                                                        &#xA;    call mpi_barrier(mpi_comm_world, ierror)                                                      &#xA;&#xA;    if(master_proc) then                                                                          &#xA;        grid_parallel = [(U_all(mod(ii,nx), int(ii/nx)), ii=0,grid_nx-1)]                         &#xA;&#xA;        Ltwo = sqrt(sum((serial-grid_parallel)**2)/real(grid_nx,dp))                              &#xA;        Linf = maxval(abs(serial-grid_parallel))                                                  &#xA;&#xA;        print'(A)',&quot;Serial vs Parallel:&quot;                                                          &#xA;        print'(2(A,ES15.6))', &quot;Ltwo =&quot;, Ltwo, &quot;; Linf =&quot;, Linf                                    &#xA;&#xA;        open(unit=100,file='postProcessing/compare.dat')                                          &#xA;        do ii=0,grid_nx-1                                                                         &#xA;            write(100,*) x_grid(ii), serial(ii), grid_parallel(ii), abs(grid_parallel(ii)-serial(ii))&#xA;        end do&#xA;        close(100)&#xA;    end if                            &#xA;end subroutine compare&#xA;subroutine compute_errors(exact, finite)                                                          &#xA;    real(dp), dimension(0:), intent(in) :: exact, finite                                          &#xA;    real(dp) :: m_sum, m_max, Ltwo, Linf                                                          &#xA;&#xA;    m_sum = sum((exact-finite)**2)                                                                &#xA;    call mpi_reduce(m_sum, Ltwo, 1, mpi_dp, mpi_sum, 0, mpi_comm_world, ierror)                   &#xA;    Ltwo = sqrt(Ltwo/real(nx,dp))                                                                 &#xA;&#xA;    m_max = maxval(abs(exact-finite))&#xA;    call mpi_reduce(m_max, Linf, 1, mpi_dp, mpi_max, 0, mpi_comm_world, ierror)                   &#xA;&#xA;    if(master_proc) then                                                                          &#xA;&#xA;        print'(A)', &quot;Finite difference vs Exact:&quot;&#xA;        print'(2(A,ES15.6))', &quot;Ltwo =&quot;, Ltwo, &quot;; Linf =&quot;, Linf     &#xA;    end if                                                                                        &#xA;end subroutine compute_errors&#xA;end program&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26633" LastEditorUserId="26633" LastEditDate="2018-02-05T16:07:43.693" LastActivityDate="2018-02-05T16:07:43.693" Title="Parallel compact schemes using the Parallal Diagonal Dominant (PDD) algorithm" Tags="|linear-algebra|algorithms|parallel-computing|fortran|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28768" PostTypeId="2" ParentId="28766" CreationDate="2018-02-05T13:50:58.657" Score="1" Body="&lt;p&gt;You need at least one initial condition for $\xi$ and two BC for $\rho$. Where are they? Your equation looks like the heat equation in cylindrical coordinates assuming angular and plane symmetry, with nonlinear heat sources: &#xA;$$ 2ik\partial_tA-\triangle A=N(A)$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would discretise the spatial derivative with finite differences (which is essentialy the same as finite elements in this case) and discretise the dynamic system with what you've proposed. If you discretise in space you will obtain:&#xA;$$ \partial_t\vec{A}-L(\rho) \vec{A}=\vec{N}(\vec{A})$$&#xA;Where $L(\rho)$ is the matrix operator for the Laplacian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This operator looks like&#xA;$$L(\rho)=L+RG$$&#xA;Where $L$ is the famous Laplacian matrix operator in cartesian coordinates and $G$ the gradient operator.&#xA;Without any BC these operators look like:&#xA;$$L(i,i)=-2/\Delta \rho^2, L(i-1,i)=L(i,i+1)=1/\Delta \rho^2 $$&#xA;$$ G(i,i+1)=-G(i,i-1)=1/\Delta\rho$$&#xA;$$  R(i,i)=1/\rho_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting scheme will be sth. like this:&#xA;$$ [2ki\mathbb{I}-\Delta \xi/2L(\rho)]\vec{A}^{n+1}+\Delta\xi/2\vec{N}(\vec{A}^{n+1})= 2ki\vec{A}^n+\Delta\xi/2 L(\rho)\vec{A}^{n} +\Delta\xi/2\vec{N}(\vec{A}^{n})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has been obtained from the general Crank-Nicolson scheme:&#xA;$$\frac{W^{n+1}-W^n}{\Delta t}=\frac{1}{2}[F(W^{n+1})+F(W^n)]+\mathcal{O}(\Delta t^2)$$&#xA;for the differential equation:&#xA;$$ \frac{dW}{dt}=F(t,W)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each $n$ you will solve a nonlinear system of equations...&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-02-09T11:12:07.270" LastActivityDate="2018-02-09T11:12:07.270" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="28770" PostTypeId="1" CreationDate="2018-02-05T23:43:53.217" Score="2" ViewCount="531" Body="&lt;p&gt;What are the differences between supernodal and multifrontal matrix factorizations? Can you provide a few references or high-level points about the approaches?&lt;/p&gt;&#xA;" OwnerUserId="22292" LastEditorUserId="22292" LastEditDate="2018-02-06T03:10:39.037" LastActivityDate="2018-02-07T04:30:14.703" Title="Supernodal vs multifrontal factorizations" Tags="|matrix-factorization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28774" PostTypeId="2" ParentId="28770" CreationDate="2018-02-06T20:44:56.813" Score="7" Body="&lt;p&gt;The scopes of these terms are (in my opinion) somewhat different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd use &quot;supernodal&quot; as a broad term, to describe any sparse direct solver that applies sufficient intelligence during the symbolic analysis phase to recognize consecutive columns that share the same nonzero structure, and reorder/aggregate them into one &quot;supervariable&quot; that will be eliminated, backsolved and schur-downdated simultaneously. As indicated in the other question, these operations can use BLAS3 arithmetic and deliver higher performance than &quot;scalar&quot; solvers. Some solvers are very aggressively supernodal, and will introduce new stored-zeroes into the pattern, if it leads to better coarsening of elimination tree (more BLAS3 than BLAS2). This is not unlike the landscape of dense algorithms, which might happily perform extra BLAS3 work yet still end up being faster than a conventional BLAS2/BLAS1 algorithm that expends fewer flops (but is overall less coherent).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, &quot;multifrontal&quot; is a narrower term, which implies a particular storage and scheduling strategy for a &quot;right-looking&quot; factorization. Their key feature is the introduction of additional (small, numerous, memory-contiguous) workspaces that hold the schur-downdates due to each supernode. This strategy relaxes/forestalls a &quot;write race&quot; in the right-looking algorithm and exposes parallelism across multiple branches of the elimination tree. If you look at the actual data structures involved in multifrontal methods, they're not array-like CSR/CSC layouts, they're all tracking small dense fragments of numeric storage that dangle off some underlying symbolic DAG's/trees. Multifrontal methods are generally supernodal as well (but the reverse is not true, as there are many supernodal solvers that are not at all based on multifrontal techniques).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, multifrontal methods are harder to write (initially) but easier to parallelize (later). They tend to use more memory (the workspaces), but can be refactored into out-of-core solvers with modest effort.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-02-07T04:30:14.703" LastActivityDate="2018-02-07T04:30:14.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28775" PostTypeId="1" CreationDate="2018-02-07T11:52:35.203" Score="0" ViewCount="158" Body="&lt;p&gt;I'm solving 2D axisymmetrical Euler's equations in conservative form: &#xA;$$&#xA;\frac{\partial U}{\partial t} + \frac{\partial F(U)}{\partial x} + \frac{\partial G(U)}{\partial r} = H(U)&#xA;$$&#xA;where&#xA;$$&#xA;U = \left( \begin{array}{c} r\rho \\ r\rho u \\ r\rho v \\ re\end{array} \right), \; &#xA;F(U) = \left( \begin{array}{c} r\rho u \\ r(\rho u^2 + p) \\ r\rho uv \\ r(e+p)u\end{array} \right), \;&#xA;G(U) = \left( \begin{array}{c} r\rho v \\ r\rho uv \\ r(\rho v^2 + p) \\ r(e+p)v\end{array} \right), \;&#xA;H(U) = \left( \begin{array}{c} 0 \\ 0 \\ p \\ 0 \end{array} \right),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using finite-difference WENO5 method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to correctly impose discrete axisymmetric boundary condition at $r=0$? One &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0021999199963829&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; regarding boundary condition (for full cylindrical coordinates) just mentions that symmetry condition should be used for axisymmetric flows, but without any details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I was using MacCormack's method (2nd order, for each grid point one more neighboring point in each direction is needed), the procedure was pretty simple (C-syntax):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//internal points&#xA;for (k = 1; k &amp;lt;= k_max - 1; k++)&#xA;{&#xA;    for (l = 1; l &amp;lt;= l_max - 1; l++)&#xA;    {&#xA;        R[k][l] = ... (calculated by MacCormack's method); &#xA;        U[k][l] = ...;&#xA;        V[k][l] = ...;&#xA;        P[k][l] = ...;&#xA;    }&#xA;}&#xA;//boundary at r = 0&#xA;for (k = 0; k &amp;lt;= k_max; k++)&#xA;{&#xA;    R[k][0] = R[k][1]; &#xA;    U[k][0] = U[k][1];&#xA;    V[k][0] = 0;&#xA;    P[k][0] = P[k][1];&#xA;}&#xA;//other boundaries...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where R,U,V,P are arrays for $\rho, u, v, p$, first array index &lt;code&gt;k&lt;/code&gt; is for $x$, second &lt;code&gt;l&lt;/code&gt; is for $r$ (uniform square grid) and &lt;code&gt;l=0&lt;/code&gt; correspond to $r=0$. This boundary condition seems to work well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With WENO5, the problem is that for each grid point, three more points in each direction are used in the stencil, so specifying one point at &lt;code&gt;l=0&lt;/code&gt; isn't enough. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current ideas are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; Explicitly set three near-axis points:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//internal points&#xA;for (k = 3; k &amp;lt;= k_max - 3; k++)&#xA;{&#xA;    for (l = 3; l &amp;lt;= l_max - 3; l++)&#xA;    {&#xA;        R[k][l] = ... (calculated by WENO5 method); &#xA;        U[k][l] = ...;&#xA;        V[k][l] = ...;&#xA;        P[k][l] = ...;&#xA;    }&#xA;}&#xA;//boundary at r = 0&#xA;for (k = 0; k &amp;lt;= k_max; k++)&#xA;{&#xA;    for (l = 0; l &amp;lt;= 2; l++)&#xA;    {&#xA;        R[k][l] = R[k][l]; &#xA;        U[k][l] = U[k][l];&#xA;        V[k][l] = 0;&#xA;        P[k][l] = P[k][l];&#xA;    }&#xA;}&#xA;//other boundaries...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; Add three ghost points for $r&amp;lt;0$ (so that &lt;code&gt;l=3&lt;/code&gt; at $r=0$) and set them with mirrored values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//internal points&#xA;for (k = 3; k &amp;lt;= k_max - 3; k++)&#xA;{&#xA;    for (l = 3; l &amp;lt;= l_max - 3; l++)&#xA;    {&#xA;        R[k][l] = ... (calculated by WENO5 method); &#xA;        U[k][l] = ...;&#xA;        V[k][l] = ...;&#xA;        P[k][l] = ...;&#xA;    }&#xA;}&#xA;//boundary at r = 0&#xA;for (k = 0; k &amp;lt;= k_max; k++)&#xA;{&#xA;    /values at r &amp;lt; 0&#xA;    for (l = 0; l &amp;lt;= 2; l++)&#xA;    {&#xA;        R[k][l] = R[k][6 - l]; &#xA;        U[k][l] = U[k][6 - l];&#xA;        V[k][l] = -V[k][6 - l];&#xA;        P[k][l] = P[k][6 - l];&#xA;    }&#xA;    //values at r = 0&#xA;    R[k][3] = R[k][4]; &#xA;    U[k][3] = U[k][4];&#xA;    V[k][3] = 0;&#xA;    P[k][3] = P[k][4];&#xA;}&#xA;//other boundaries...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or mirror with reversed sign (since values at ghost points are multiplied by negative $r$ in equations):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    ...&#xA;    for (l = 0; l &amp;lt;= 2; l++)&#xA;    {&#xA;        R[k][l] = -R[k][6 - l]; &#xA;        U[k][l] = -U[k][6 - l];&#xA;        V[k][l] = V[k][6 - l];&#xA;        P[k][l] = -P[k][6 - l];&#xA;    }&#xA;    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Methods &lt;strong&gt;1&lt;/strong&gt; and &lt;strong&gt;2&lt;/strong&gt; generate noticable artifacts near the symmetry axis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; Shift the grid by half-step so that no point is placed exactly at $r=0$ and use ghost-points. This method leads to computation instability, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the correct way? &lt;/p&gt;&#xA;" OwnerUserId="25060" LastEditorUserId="25060" LastEditDate="2018-02-08T08:07:57.257" LastActivityDate="2018-04-13T01:25:01.977" Title="Multi-point axisymmetric boundary condition for Euler equations" Tags="|finite-difference|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28776" PostTypeId="1" CreationDate="2018-02-07T14:39:30.527" Score="1" ViewCount="67" Body="&lt;p&gt;Quantum Monte Carlo (QMC) calculations historically have been used to parameterise the exchange-correlation functionals for Density Functional Theory (DFT). For example, this article explains a way to do that: &lt;a href=&quot;https://spiral.imperial.ac.uk/bitstream/10044/1/864/1/Quantum%20Monte%20Carlo%20investigations%20of%20density.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantum Monte Carlo investigations of density functional theory of the strongly inhomogeneous electron gas&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I cannot seem to find any software package that allows one to do this for arbitrary input potentials. Is there some QMC program package that gives this possibility, or are they all in-house software? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that the &lt;a href=&quot;http://qmcpack.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;QMCPACK&lt;/a&gt; has a &lt;code&gt;vhxc&lt;/code&gt; potential option, which might be what I am looking for, but it is undocumented. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="25184" LastEditorUserId="22108" LastEditDate="2018-04-24T11:41:52.347" LastActivityDate="2018-04-24T11:41:52.347" Title="Is there a software package for Quantum Monte Carlo estimation of the exchange correlation functional?" Tags="|monte-carlo|quantum-mechanics|density-functional-theory|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28778" PostTypeId="2" ParentId="6968" CreationDate="2018-02-07T22:39:05.507" Score="0" Body="&lt;p&gt;Need to be careful specifically in generating method of manufactured solutions for Poisson's equation. Since definition of the source term has to satisfy the strong form of the PDE as well as the weak form of the PDE.&#xA;The derivation given above is basically using the weak form of the PDE.&#xA;In other words, there is a relation between the gradient of the solution at the boundary and the source term integral on the domain.&lt;/p&gt;&#xA;" OwnerUserId="26771" LastActivityDate="2018-02-07T22:39:05.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28780" PostTypeId="1" CreationDate="2018-02-08T02:45:04.297" Score="3" ViewCount="332" Body="&lt;p&gt;Is there a known error estimate for Gaussian quadrature when applied to a discontinuous function?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simple one-dimensional experiments, the error appears to be bounded by $C h$, where $C$ is some constant and $h = \max |x_{i+1}-x_i|$ is the maximum spacing between two nodes, but I haven't found a clean way to show this.&lt;/p&gt;&#xA;" OwnerUserId="4912" LastEditorUserId="4912" LastEditDate="2018-02-08T03:22:37.867" LastActivityDate="2018-02-08T03:22:37.867" Title="Convergence of Gauss quadrature for a discontinuous function" Tags="|numerical-analysis|quadrature|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28781" PostTypeId="1" AcceptedAnswerId="28809" CreationDate="2018-02-08T04:06:32.667" Score="4" ViewCount="191" Body="&lt;p&gt;I am using MATLAB to prototype a few matrix multiply techniques and compare efficiency. Eventually, I will move the prototype codes to C. It is for a homework assignment where we need to write an efficient matrix multiply routine (by being aware of cache size, locality, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am curious about the efficiency differences between these two very similar loops:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Matrix Multiply Loop 1&lt;/strong&gt;&#xA;- sum over columns of A times elements of B -&gt; column of C&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [C] = dgemm_naivepe( A,B,C,n )&#xA;&#xA;for j=1:n&#xA;    tempcol=zeros(n,1);&#xA;    for k=1:n&#xA;        for i=1:n&#xA;            tempcol(i)=tempcol(i)+A(i+(k-1)*n)*B(k+(j-1)*n);&#xA;        end&#xA;    end&#xA;    for k=1:n&#xA;        C(k+(j-1)*n)=tempcol(k);&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;end&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Matrix Multiply Loop 2&lt;/strong&gt;&#xA;- sum over columns of A times elements of B -&gt; column of C&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [C] = dgemm_naivepe( A,B,C,n )&#xA;&#xA;for j=1:n&#xA;    for k=1:n&#xA;        for i=1:n&#xA;            C(i+(j-1)*n)=C(i+(j-1)*n)+A(i+(k-1)*n)*B(k+(j-1)*n);&#xA;        end&#xA;    end&#xA;end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After several test runs of various matrix sizes, I found that &lt;strong&gt;Loop 1&lt;/strong&gt; is faster than &lt;strong&gt;Loop 2&lt;/strong&gt;. Could someone help me understand why this is?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I posted this on a general coding stack exchange but didn't get much of a response, so I figured I could post it here as well.&lt;/p&gt;&#xA;" OwnerUserId="22717" LastActivityDate="2018-02-12T19:06:18.223" Title="MATLAB Matrix Multiply Efficiency" Tags="|linear-algebra|matlab|matrix|efficiency|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28782" PostTypeId="1" CreationDate="2018-02-08T09:44:40.780" Score="0" ViewCount="121" Body="&lt;p&gt;I want to change my main platform from Matlab to Python due to my work, and I mainly used Matlab for the neural network, so I want to do the same thing in the Python either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used to make my own network architecture like the picture below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/u8lc9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u8lc9.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to replicate this network in Python either, but I can't find the module which can adjust the input connection and layer connection so far.&#xA;As far as I know, 'neurolab' and 'sklearn' doesn't provide the function what I said. Every code I found connect input data to the first layer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, anybody who knows the module or code which can make network architecture like the attached picture, please help me.&lt;/p&gt;&#xA;" OwnerUserId="26796" LastActivityDate="2018-02-09T14:18:29.313" Title="Anyone who knows fine neural network code or module for python?" Tags="|matlab|python|neural-network|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28783" PostTypeId="2" ParentId="14337" CreationDate="2018-02-08T14:01:24.863" Score="0" Body="&lt;h1&gt;Short answer&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;No, there is not a better way to do this.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Long answer&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;WORKSHARE&lt;/code&gt; is much harder to implement than explicit loop parallelism.  Compilers have been known to do correct but useless (aka &quot;no-op&quot;) implementations of &lt;code&gt;WORKSHARE&lt;/code&gt; that map it to &lt;code&gt;SINGLE&lt;/code&gt;.  This is correct but obviously provides no performance benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When compilers support &lt;code&gt;WORKSHARE&lt;/code&gt;, they may do so selectively, e.g. &lt;a href=&quot;https://software.intel.com/en-us/articles/openmp-workshare-constructs-now-parallelize-with-intel-fortran-compiler-150&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel Fortran 15&lt;/a&gt;, which means that depending on it may lead to mixed results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even when compilers do a good job, in parallel computing in general, you will find that explicit parallelization wins, because the user is required to make good design choices up front and compilers, which are much less intelligent than humans, don't have to make important decisions.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2018-02-08T14:01:24.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28789" PostTypeId="1" CreationDate="2018-02-08T17:49:06.003" Score="8" ViewCount="584" Body="&lt;p&gt;I am doing work which requires as fast matrix multiplication as possible and just want to double-check with this community that the Winograd variant of Strassen's MM algorithm is the fastest practical (so no Coppersmith-Winograd) algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing a lot of processing with the data before, between, and after each MM to the point where using Mathematica or Matlab would be a hindrance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am curious if anyone has a good idea for the error in the regular Strassen vs Winograd variant?  In &quot;Exploiting Parallelism in Matrix Computation Kernels for SMP systems&quot; D'Alberto et al. briefly mention Strassen as being more accurate but this seems counter-intuitive since Winograd has less operations overall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: We are using matrices up to size 2^16 x 2^16 ~ 4 billion doubles, so a sub-cubic algorithm is definitely faster than naive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: On the accuracy of Strassen vs Winograd, if anyone is interested.  In &quot;Accuracy and Stability of Numerical Algorithms&quot; Higham has an in-depth analysis of the error of the two algorithms and shows Strassen has slower error growth w/respect to size of matrices.  Also of note, Strassen more error-prone (against itself) for matrices with all positive entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="26785" LastEditorUserId="26785" LastEditDate="2018-02-10T21:07:01.963" LastActivityDate="2018-02-10T21:07:01.963" Title="Speed and accuracy of Strassen vs Winograd matrix multiplication algorithms" Tags="|matrix|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28790" PostTypeId="2" ParentId="28781" CreationDate="2018-02-08T18:08:35.683" Score="1" Body="&lt;p&gt;Maybe the size of C vs the size of tempcol is what is making the difference?&lt;br&gt;&#xA;If it is pulling the C matrix into cache it has to make more room than for tempcol, possibly removing data from A and/or B which wouldn't have been removed for the smaller tempcol.&lt;/p&gt;&#xA;" OwnerUserId="26785" LastActivityDate="2018-02-08T18:08:35.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28791" PostTypeId="1" CreationDate="2018-02-08T20:40:28.020" Score="1" ViewCount="160" Body="&lt;p&gt;A standard Quadratically Constrained Quadratic Program (QCQP) is of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \underset{x}{minimize} \frac{1}{2}x^TP_{0}x +  q_{0}^{T}x&#xA;$$&#xA;$$&#xA;   subject \; to \quad \frac{1}{2}x^TP_{i}x +  q_{i}^{T}x + r_{i}\leq 0\quad i \in \{1,2,\ldots,m\}&#xA;$$&#xA;$$&#xA;\quad Ax = b&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that this type of program is specifically well suited for a Sequential Quadratic Programming (SQP) solution, especially when the $P_{i}$ are positive semi-definite. However, I have only seen SDP relaxations in literature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any commerical software packages that solve these types of problems with SQP? What other methods are actually used in practice?&lt;/p&gt;&#xA;" OwnerUserId="26805" LastActivityDate="2018-02-08T20:40:28.020" Title="Sequential Quadratic Programming for Quadratically Constrained Quadratic Programs" Tags="|optimization|algorithms|convex-optimization|constrained-optimization|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28793" PostTypeId="2" ParentId="28782" CreationDate="2018-02-09T14:18:29.313" Score="1" Body="&lt;p&gt;A very flexible Python library you could use to build your network and perform the various Machine Learning operations is Tensorflow. The premise of this library is you construct graphs that you can use to represent models, perform computation, and many other things. Using this framework, you can build arbitrary neural network models, amongst other things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this framework does have a little bit of a learning curve, though. &lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2018-02-09T14:18:29.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28794" PostTypeId="1" AcceptedAnswerId="28797" CreationDate="2018-02-09T14:23:55.047" Score="3" ViewCount="196" Body="&lt;p&gt;I'm trying to apply the finite-volume method (FVM), with which I'm not so familiar, so a simple 1D PDE equation. I already asked this question in the math stackexchange, but was told that it could be a better fit here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation I want to solve is, to simplify, &#xA;$$\frac{\partial U}{\partial t} + A\left(x\right) \frac{\partial}{\partial &#xA;x}\left(\frac{F(U)}{A(x)}\right) + \frac{\partial G(U)}{\partial x} = 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U$ is my unknown, $F$ and $G$ some flux functions computed from $U$ and $A$ a function of $x$, that is strictly positive and very smooth. Most of the time, I can actually assume $A$=constant, and therefore the equation becomes &#xA;$$\frac{\partial U}{\partial t} + \frac{\partial(G(U)+F(U))}{\partial x} = 0,$$ which is straightforward to solve using FVM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not sure how to handle the case where $A$ is not constant. My idea was, when applying the integration step of the finite volume method, to write &#xA;$$\int_{\text{cell}} A\left(x\right) \frac{\partial}{\partial x}\left(\frac{F(U)}{A(x)}\right) \mathrm{d}\Omega \approx \bar{A} \int_{\text{cell}}  \frac{\partial}{\partial x}\left(\frac{F(U)}{A(x)}\right) \mathrm{d}\Omega$$ where $\bar{A}$ is the mean value of $A$ over the considered cell. Once fully discretized, this term would therefore end up looking like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar{A}_{i} \left( \frac{1}{\Delta x} \left( \frac{F_{i+1/2}}{A_{i+1/2}}  - \frac{F_{i-1/2}}{A_{i-1/2}} \right) \right).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it make sense, or is it completely wrong? Is there a better way to proceed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried changing the coordinate, but I'm not familiar enough with that, and of course I'd like to keep the $\dfrac{\partial G(U)}{\partial x}$ term as it is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot for any inputs and advices on how to deal with this kind of problem.&lt;/p&gt;&#xA;" OwnerUserId="26815" LastActivityDate="2018-02-09T17:58:22.970" Title="Finite-volume method applied to a particular advection equation" Tags="|simulation|finite-volume|numerical-modelling|elliptic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28795" PostTypeId="1" CreationDate="2018-02-09T15:17:59.013" Score="2" ViewCount="1202" Body="&lt;p&gt;I was looking into the Book of Riviere &quot; Discontinuous Galerkin Methods for solving Elliptic and Parabolic Equations&quot;. In the comparaison of section 2.12 (copied below), the example of rectangular mesh indicated that the DG is more economic (has les DOFs) then the CG when using a certain space of elements. Can any one explain in more details please ? What is the difference between the two spaces Qk and Pk ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot; Size of problem:For DG, the total number of degrees of freedom is proportional to the number of elements in the mesh. The constant of proportionality is a function of the polynomial degree. For CG, the degrees of freedom depend on the number of vertices and possibly the number of vertices and elements in the mesh. For instance, consider a structured mesh of 5 × 5 rectangular elements. The degrees of freedom for a DG approximation of degree 1, 2, 3, 4 are 75, 150, 250, 375, respectively, whereas the degrees of freedom for a CG approximation of degree 1, 2, 3, 4 are 36, 121, 256, 441, respectively. Thus, on such small mesh, if k ≥ 3, the CG method is more costly than DG. The reason is that we have to use the space Qk on rectangular elements for CG, but we can still use the space Pk on rectangular elements for DG.&quot;&lt;/p&gt;&#xA;" OwnerUserId="23068" LastEditorUserId="393" LastEditDate="2018-02-09T21:42:32.990" LastActivityDate="2018-02-09T21:49:28.717" Title="Discontinuous Galerkin method VS Continuous Galerkin method Degrees of freedom" Tags="|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28797" PostTypeId="2" ParentId="28794" CreationDate="2018-02-09T17:58:22.970" Score="3" Body="&lt;p&gt;Your proposed discretisation appears to be consistent, but wouldn't normally be interpreted as a finite volume discretisation. Indeed, it looks a lot more like a finite difference method with a slightly unusual stencil. Further, this looks like a nonlinear hyperbolic, rather than elliptic PDE, which fits with your desire to use FVM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generically, you can build yourself a finite volume model by integrating the equation over a fixed cell. In this case we can use integration by parts to write that in $n$ dimensions, and for a cell $C$ bounded by a contour, $\delta C$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial}{\partial t} \int_C U d\mathbf{x} = -\int_{\delta C} \mathbf{F}(U)+\mathbf{G}(U)\cdot\mathbf{n} dS +\int_C \frac{\mathbf{F}(U)}{A}\cdot\nabla A d\mathbf{x}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Comparing this to the constant $A$ case, you have an additional source/sink for your tracer, which in 1D looks like&#xA;$$F(U)\frac{1}{A}\frac{\partial A}{\partial x},$$ which you can discrete using normal cell centred finite differences for the derivative. This actually impacts more on your time stepping scheme, since for sinks ($\frac{\partial F}{\partial U}\frac{\partial A}{\partial x}$ negative) you may want to consider an implicit method in the case that $U$ needs to stay bounded.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2018-02-09T17:58:22.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28798" PostTypeId="1" CreationDate="2018-02-09T20:30:54.900" Score="0" ViewCount="249" Body="&lt;p&gt;I would like to do non negative mf and I wanna ask a question about my main matrix. The question is should I include rows and columns that have no non-zero entry in them. I think if there is not a single relation information between any row and column, it doesn't make sense. Is that correct to prune those rows and columns that don't have a single non-empty entry from the main matrix?&lt;/p&gt;&#xA;" OwnerUserId="25063" LastActivityDate="2018-02-09T21:41:03.083" Title="Matrix factorization empty rows and columns" Tags="|sparse-matrix|matrix|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28799" PostTypeId="2" ParentId="28798" CreationDate="2018-02-09T21:41:03.083" Score="3" Body="&lt;p&gt;No. If you have a zero row but a nonzero corresponding right hand side, then the linear system has no solution. If you have a zero column, then the solution has infinitely many solutions because the entry in the corresponding position of the solution vector does not matter -- any value there is allowed, and so the solution is not unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, empty rows or columns of a matrix have &lt;em&gt;meaning&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-02-09T21:41:03.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28800" PostTypeId="2" ParentId="28795" CreationDate="2018-02-09T21:49:28.717" Score="8" Body="&lt;p&gt;The argument is misleading. The different spaces that are used are $P_1=\text{span}\{1,x,y\}$ and $Q_1=\text{span}\{1,x,y,xy\}$. For higher orders, they are&#xA;$P_2=\text{span}\{1,x,y,x^2,y^2\}$ and $Q_2=\text{span}\{1,x,y,x^2,y^2,xy,x^2y,xy^2,x^2y^2\}$. If you continue this, it is true that the number of basis functions for the $Q_k$ spaces grows faster than the number of basis functions for $P_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this is not the important quantity. What matters is not how many basis functions you have on a given mesh for a given polynomial degree $k$, but this: *How many $Q_k$ shape functions do you need to reach a certain level of accuracy vs. how many do you need for $P_{k'}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this metric, $Q_k$ spaces typically win because the additional shape functions help you cancel terms in the Taylor expansion of the solution, and consequently the error you get on one quadrilateral cell with a $Q_k$ element is typically substantially smaller than the error you get by splitting this one cell into two triangles and using a $P_k$ element on it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, while it is true that for the same polynomial degree, $Q_k$ continuous elements have more unknowns than $P_k$ discontinuous elements if $k$ is sufficiently large, this does not matter: You &lt;em&gt;get&lt;/em&gt; something for these additional shape functions in terms of error reduction.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-02-09T21:49:28.717" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28801" PostTypeId="1" CreationDate="2018-02-10T19:53:57.040" Score="1" ViewCount="50" Body="&lt;p&gt;I have a set of real-valed vectors, for example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S = \{v_1, v_2, ..., v_k\}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_i = \begin{pmatrix} age_i \\ height_i \\ weight_i \\ ... \end{pmatrix}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or whatever. Each vector has on the order of 30 components, and $k\approx 10,000$. The vectors are not normalized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two objectives:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;count how many clusters are naturally formed by the data set,&lt;/li&gt;&#xA;&lt;li&gt;given arbitrary $N$, choose $N$ representatives from the set (or construct representative vectors) which span the clusters.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are lots of clustering algorithms to choose from. I can be flexible on what the definition of clusters entails, but I do want &quot;reasonable&quot; behaviour if $N$ is different from the number of clusters which naturally exist:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if N is too small, the biggest, densest clusters should be represented first,&lt;/li&gt;&#xA;&lt;li&gt;if N is too big the extra representatives should sit near the big clusters rather than anywhere else, and the extra representatives should not all congregate in the same place.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Simplicity and performance are the priority. The definition of a cluster is negotiable. It would be nice if the algorithm could be tuned to specify that certain components are &quot;important&quot; in defining a cluster, but that's just a wish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be a fast, simple clustering method for this kind of data?&lt;/p&gt;&#xA;" OwnerUserId="26822" LastActivityDate="2018-02-10T19:53:57.040" Title="What is a performant clustering algorithm for approx 10,000 vectors of approx 30 dimension?" Tags="|linear-algebra|machine-learning|statistics|clustering|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28802" PostTypeId="1" CreationDate="2018-02-10T20:47:38.767" Score="1" ViewCount="127" Body="&lt;p&gt;I am working on a project in numerical analysis which I have to program in C (using Lapack and Blas). Matrix is given which is tridiagonal and &quot;almost&quot; symmetric (one element is to be changed to make it symmetric). In order to calculate certain eigenvalues and eigenvectors I am planning to use &lt;code&gt;dsyevx()&lt;/code&gt; routine which works only for symmetric matrices. Is there any routine for orthogonal similarity transformation to make it symmetric, or should I opt for different routine for calculating eigenvalues of general $N\times N$ matrix?&lt;/p&gt;&#xA;" OwnerUserId="26823" LastEditorUserId="9667" LastEditDate="2018-02-11T12:03:39.500" LastActivityDate="2018-02-11T12:03:39.500" Title="Eigenvalue problem (LAPACK)" Tags="|numerical-analysis|lapack|blas|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="28803" PostTypeId="1" CreationDate="2018-02-11T08:25:38.800" Score="4" ViewCount="306" Body="&lt;p&gt;I have a problem with the stability of finite difference method for the wave equation in cylindrical coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the equation is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial^2 \omega_n}{\partial r^2}+\frac{1}{r}\frac{\partial \omega_n}{\partial r}+\frac{\partial^2\omega_n}{\partial z^2}-\frac{\omega_n}{r^2}=\frac{1}{c^2}\frac{\partial^2\omega_n}{\partial t^2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and when I run it in Matlab, I get the following result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gxjHS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gxjHS.png&quot; alt=&quot;figure 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know where the problem is?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I put the code here so that if anyone knows where the problem is, help me out.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%domain&#xA;clear;&#xA;Lx=10;&#xA;Ly=10;&#xA;dx=0.1;&#xA;dy=dx;&#xA;nx=fix(Lx/dx);&#xA;ny=fix(Ly/dy);&#xA;x=linspace(0,Lx,nx);&#xA;y=linspace(0,Ly,ny);&#xA;%time&#xA;T=10;&#xA;%varables&#xA;wn=zeros(nx,ny);&#xA;wnm1=wn; %w at time n-1&#xA;wnp1=wn; %w at tome n+1&#xA;%parameters&#xA;CFL=0.5; %CFL=c.dt/dx&#xA;c=1;&#xA;dt=(CFL*dx/c);&#xA;&#xA;%dt=1e-6;&#xA;&#xA;%%initial conditions&#xA;%%time stepping loop&#xA;t=0;&#xA;while(t&amp;lt;T)&#xA;    %reflecting boundary conditions&#xA;    wn(:,[1 end])=0;&#xA;    wn([1 end],:)=0;&#xA;&#xA;&#xA;    %solution&#xA;    t=t+dt;&#xA;    wnm1=wn; wn=wnp1; %save current and previous arrays&#xA;&#xA;    %sourse&#xA;    wn(50,50)=dt^2*20*sin(30*pi*t/20);&#xA;    for i=2:nx-1 for j=2:ny-1&#xA;&#xA;           wnp1(i,j)=2*wn(i,j)-wnm1(i,j)...&#xA;               +CFL^2*(((wn(i+1,j)-2*wn(i,j)+wn(i-1,j))/dx^2+(1/(2*i*dx^2))*(wn(i+1,j)-wn(i-1,j))+(wn(i,j+1)-2*wn(i,j)+wn(i,j-1))/dy^2-wn(i,j)/(i*dx)^2));&#xA;&#xA;        end,end&#xA;    %chek convergence&#xA;&#xA;    %visualize at selected steps&#xA;    clf;&#xA;    subplot(2,1,1);&#xA;    imagesc(x,y,wn'); colorbar;caxis([-0.02 0.02])&#xA;    title(sprintf('t=%.2f',t));&#xA;    subplot(2,1,2);&#xA;    mesh(x,y,wn'); colorbar;caxis([-0.02 0.02])&#xA;    axis([0 Lx 0 Ly -0.05 0.05]);&#xA;    shg; pause(0.01);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26799" LastEditorUserId="20688" LastEditDate="2018-11-25T06:11:23.207" LastActivityDate="2018-11-25T06:11:23.207" Title="How can I solve the wave equation for a circular rod in cylindrical coordinates using finite differences?" Tags="|finite-difference|electromagnetism|wave-propagation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="28804" PostTypeId="1" CreationDate="2018-02-11T11:03:41.153" Score="4" ViewCount="82" Body="&lt;p&gt;I'm currently trying to minimize the following function:&#xA;$$\min_{\mathbf  x,\mathbf  y,s} \|F\mathbf x-\mathbf a\|_2^2 + \|SF\mathbf y-\mathbf b\|_2^2 + \lambda \|\max(\mathbf x, \mathbf y)\|_1,$$&#xA;$$\mathbf x&amp;gt;\mathbf 0, \mathbf y&amp;gt;\mathbf 0.$$&#xA;Here $F$ and $S$ are both linear operators. $F$ is a discrete Fourier transform, $S$ is &quot;shift by $s$ in Fourier domain&quot; operator: $(Sf)_k = f_k \cdot e^{-2\pi i u_k s}$, and $\max(\mathbf x, \mathbf y)$ is elementwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the problem is basically regularized least squares, and should hopefully lead to sparse solutions $\mathbf x, \mathbf y$ having nonzero elements in about the same locations (up to explicit shift $s$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that it is almost linear but for the discontinuity in regularizer and nonlinear dependency on $s$. For fixed $s$ I get good results ($\mathbf  x,\mathbf  y$ close to true values) using LBFGS method. When optimizing separately for each $s$ from a range, I see a clear minimum around the true value. But when I include the $s$ variable into optimization, its value basically doesn't change from the initial one, no matter if it is close or far from the truth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess the problem is that the function (of $s$) has many local minima all over the range. How to approach the problem correctly, or where to read about similar ones? I had no success with other common optimization algorithms from scipy I tried.&lt;/p&gt;&#xA;" OwnerUserId="26827" LastEditorUserId="26827" LastEditDate="2018-02-11T14:26:54.020" LastActivityDate="2018-02-11T15:25:16.140" Title="Large-scale almost-linear optimization" Tags="|optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28806" PostTypeId="2" ParentId="28804" CreationDate="2018-02-11T15:25:16.140" Score="3" Body="&lt;p&gt;Here is one possibility to try out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you could decompose this problem into a convex and non-convex one by alternation. One could maybe first optimize for $\mathbf x$ and $\mathbf y$:&#xA;$$\min_{\mathbf  x,\mathbf  y} \|F\mathbf x-\mathbf a\|_2^2 + \lambda \|\max(\mathbf x, \mathbf y)\|_1,$$&#xA;keeping $\mathbf S$ constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the next stage, we could solve for $\mathbf S$:&#xA;$$\min_{s} \|SF\mathbf y-\mathbf b\|_2^2$$&#xA;this time keeping all the rest fixed. Maybe it is easier to solve this second equation via direct methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting from certain initialization, we could alternate both solutions until we arrive at the desired one. Other splits are also possible and might work better, such as first solving for $\mathbf x$, etc.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2018-02-11T15:25:16.140" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28807" PostTypeId="2" ParentId="25955" CreationDate="2018-02-11T15:37:34.317" Score="1" Body="&lt;p&gt;In case you do not want to take a simple convex hull (results in overestimations and not composed only of elements), here is an algorithm for, maybe, solving a specialized version of your problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find all the simply connected components (using e.g. union-find algorithm).&lt;/li&gt;&#xA;&lt;li&gt;For each component, find the &lt;a href=&quot;https://en.wikipedia.org/wiki/Alpha_shape&quot; rel=&quot;nofollow noreferrer&quot;&gt;alpha shape&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Naturally extends to 3D. Haven't proved if it's the smallest though.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2018-02-11T15:37:34.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28808" PostTypeId="1" CreationDate="2018-02-11T19:39:48.947" Score="1" ViewCount="48" Body="&lt;p&gt;I am reading a paper that summarizes a set of simulations. Basically, the authors are trying to minimize some function using different optimization algorithms. They conclude: &quot;Our findings point to instances of convergence at points where the first- and second-order optimality conditions fail.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does this mean in plain English -- that parameter updates have essentially stopped even though the algorithm has not identified an optimal point? My understanding is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First-order conditions: Ensure that a critical point has been found -- maximum, minimum, or inflection.&lt;/li&gt;&#xA;&lt;li&gt;Second order conditions: Determine that this point is of the nature we want (e.g. that it is indeed a minimum).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is this correct? My knowledge of optimization is a bit rusty, so I just wanted to confirm that I understand.&lt;/p&gt;&#xA;" OwnerUserId="26834" LastActivityDate="2018-02-11T19:39:48.947" Title="Meaning of &quot;points where the first- and second-order optimality conditions fail&quot;" Tags="|optimization|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28809" PostTypeId="2" ParentId="28781" CreationDate="2018-02-11T21:50:16.477" Score="4" Body="&lt;p&gt;Your loop 2 does not &lt;a href=&quot;https://it.mathworks.com/help/matlab/matlab_prog/preallocating-arrays.html?requestedDomain=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;preallocate&lt;/a&gt; $C$. Hence it gets resized $n$ times, which requires many reallocations. Try adding &lt;code&gt;C = zeros(n, n)&lt;/code&gt; before the for loops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT: so does the first one, so probably this isn't the true reason.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(That said, I second &lt;a href=&quot;https://it.mathworks.com/help/matlab/matlab_prog/preallocating-arrays.html?requestedDomain=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChrisRackauckas's comment&lt;/a&gt; --- there is little point in testing timings under Matlab at this level.)&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2018-02-12T19:06:18.223" LastActivityDate="2018-02-12T19:06:18.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28810" PostTypeId="1" CreationDate="2018-02-11T22:34:51.210" Score="1" ViewCount="78" Body="&lt;p&gt;Are there any open source tools available which can take a surface triangulation of mesh volumes and convert them to a boundary representation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in refining multiple region tetrahedral meshes with boundary conditions applied to some of the surfaces.  I have attempted to read a mesh into gmsh, but it is unclear what steps would be necessary to convert it to a boundary representation.&lt;/p&gt;&#xA;" OwnerUserId="7681" LastActivityDate="2018-02-11T22:34:51.210" Title="surface triangulation to boundary representation" Tags="|mesh-generation|gmsh|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28811" PostTypeId="1" CreationDate="2018-02-12T00:31:38.500" Score="0" ViewCount="63" Body="&lt;p&gt;I need to be able to set the properties of the materials (density, size etc) and see what happens according to laws in Physics when landmasses collide each others, with or without bodies of water, such as in the plate tectonics theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you recommend a software for this?&lt;/p&gt;&#xA;" OwnerUserId="26838" LastActivityDate="2018-02-13T01:42:32.447" Title="On which software can I simulate landmass collisions?" Tags="|simulation|modeling|3d|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28815" PostTypeId="1" AcceptedAnswerId="28823" CreationDate="2018-02-11T23:39:12.723" Score="0" ViewCount="125" Body="&lt;p&gt;I'm attempting to write a FORTRAN program that calculates the magnetic field, B, at any point outside of a bar magnet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm going to use a first order euler scheme, where each side of the bar magnet is split into small cells, each with centres at (xi,yi,zi). I know I can ignore all of the sides that have any z values, and just focus on the top and bottom sides that are orientated in the x-y plane. So the method says this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\int f(x,y,z)dS = \Delta S \cdot  \sum f(x_{i},y_{i}, z_{i})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the integral is over the surface S, and the summation is over i.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Delta S, each area, is given by $\hat{n}\cdot d\vec{S}$ , so if the cell is oriented in the x-y plane it's just $\Delta x\cdot \Delta y$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Gd3Gf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a screenshot of the specific method instructions with a figure that demonstrates it&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function for the magnetic field is this: $\vec{B}(\vec{r}) = \frac{\mu _{0}}{4\pi }\cdot \int\frac{(r-{r}')\cdot M(r)\cdot \hat{n}}{|(r-{r}')|^{3}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the integral is over the surface S&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm struggling to understand this method. I've tried to construct a flow chart, but can't get very far so I figured the problem is with the mathematics. Any help to understand it would be appreciated, and also any help with the flow chart would be fantastic. &lt;a href=&quot;https://i.stack.imgur.com/hKaOH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is my flowchart&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26837" OwnerDisplayName="webberwizard" LastActivityDate="2018-02-13T10:13:49.517" Title="Help understanding this numerical surface integration technique?" Tags="|electromagnetism|computational-physics|software|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28816" PostTypeId="2" ParentId="25196" CreationDate="2018-02-12T07:25:19.820" Score="5" Body="&lt;p&gt;I think we can compute it using the chain rule, similar to how we compute the Jacobian. For example, if we want to compute the second derivative of $\Phi$ w.r.t physical coordinates in 2D, i.e $\dfrac{\partial^2 \Phi}{\partial x^2}$, $\dfrac{\partial^2 \Phi}{\partial y^2}$ and $\dfrac{\partial^2 \Phi}{\partial x \partial y}$, we can first write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\dfrac{\partial^2 \Phi}{\partial \xi^2} = \dfrac{\partial}{\partial \xi} \left( \dfrac{\partial \Phi}{\partial \xi} \right) &amp;amp;= \dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial x} \dfrac{\partial x}{\partial \xi} + \dfrac{\partial \Phi}{\partial y} \dfrac{\partial y}{\partial \xi} \right) \nonumber \\&#xA;&amp;amp;=\left[ \dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial x} \right) \dfrac{\partial x}{\partial \xi} + \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi^2} \right] + \left[ \dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial y} \right) \dfrac{\partial y}{\partial \xi} + \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi^2} \right] \tag{1}\label{eq:d2_phi_d_xi_2-1}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms $\dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial x} \right)$ and $\dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial y} \right)$ can be rewritten in physical coordinates, which yields&#xA;\begin{align}&#xA;\dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial x} \right) = \dfrac{\partial }{\partial x} \left( \dfrac{\partial \Phi}{\partial x} \right) \dfrac{\partial x}{\partial \xi} + \dfrac{\partial }{\partial y} \left( \dfrac{\partial \Phi}{\partial x} \right) \dfrac{\partial y}{\partial \xi} = \dfrac{\partial^2 \Phi}{\partial x^2} \dfrac{\partial x}{\partial \xi} + \dfrac{\partial^2 \Phi}{\partial x \partial y} \dfrac{\partial y}{\partial \xi} \tag{2a}\label{eq:d2_phi_d_x_d_xi}\\&#xA;\dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial y} \right) = \dfrac{\partial }{\partial x} \left( \dfrac{\partial \Phi}{\partial y} \right) \dfrac{\partial x}{\partial \xi} + \dfrac{\partial }{\partial y} \left( \dfrac{\partial \Phi}{\partial y} \right) \dfrac{\partial y}{\partial \xi} = \dfrac{\partial^2 \Phi}{\partial x \partial y} \dfrac{\partial x}{\partial \xi} + \dfrac{\partial^2 \Phi}{\partial y^2} \dfrac{\partial y}{\partial \xi} \tag{2b}\label{eq:d2_phi_d_y_d_xi}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replace \eqref{eq:d2_phi_d_x_d_xi} and \eqref{eq:d2_phi_d_y_d_xi} to \eqref{eq:d2_phi_d_xi_2-1} yields&#xA;\begin{equation}&#xA;\dfrac{\partial^2 \Phi}{\partial \xi^2} = \left[ \dfrac{\partial^2 \Phi}{\partial x^2} \left( \dfrac{\partial x}{\partial \xi} \right)^2 + \dfrac{\partial^2 \Phi}{\partial x \partial y} \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \xi} + \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi^2} \right] + \left[ \dfrac{\partial^2 \Phi}{\partial x \partial y} \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \xi} + \dfrac{\partial^2 \Phi}{\partial y^2} \left( \dfrac{\partial y}{\partial \xi} \right)^2 + \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi^2} \right] \tag{3}\label{eq:d2_phi_d_xi_2-2}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rearrange \eqref{eq:d2_phi_d_xi_2-2} to localize the second derivatives terms w.r.t physical coordinates&#xA;\begin{equation}&#xA;\dfrac{\partial^2 \Phi}{\partial \xi^2} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi^2} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi^2} = \dfrac{\partial^2 \Phi}{\partial x^2} \left( \dfrac{\partial x}{\partial \xi} \right)^2 + \dfrac{\partial^2 \Phi}{\partial y^2} \left( \dfrac{\partial y}{\partial \xi} \right)^2 + 2\dfrac{\partial^2 \Phi}{\partial x \partial y} \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \xi} \tag{4}\label{eq:local_2d_2nd_derivative-1}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the same manner, we can also derive for $\dfrac{\partial^2 \Phi}{\partial \eta^2}$ and yield&#xA;\begin{equation}&#xA;\dfrac{\partial^2 \Phi}{\partial \eta^2} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \eta^2} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \eta^2} = \dfrac{\partial^2 \Phi}{\partial x^2} \left( \dfrac{\partial x}{\partial \eta} \right)^2 + \dfrac{\partial^2 \Phi}{\partial y^2} \left( \dfrac{\partial y}{\partial \eta} \right)^2 + 2\dfrac{\partial^2 \Phi}{\partial x \partial y} \dfrac{\partial x}{\partial \eta} \dfrac{\partial y}{\partial \eta}\tag{5} \label{eq:local_2d_2nd_derivative-2}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Derivation of $\dfrac{\partial^2 \Phi}{\partial \xi \partial \eta}$ yield&#xA;\begin{align}&#xA;\dfrac{\partial^2 \Phi}{\partial \xi \partial \eta} &amp;amp;= \dfrac{\partial}{\partial \xi} \left( \dfrac{\partial \Phi}{\partial \eta} \right) = \dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial x} \dfrac{\partial x}{\partial \eta} + \dfrac{\partial \Phi}{\partial y} \dfrac{\partial y}{\partial \eta} \right) \nonumber \\&#xA;&amp;amp;=\left[ \dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial x} \right) \dfrac{\partial x}{\partial \eta} + \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi \partial \eta} \right] + \left[ \dfrac{\partial }{\partial \xi} \left( \dfrac{\partial \Phi}{\partial y} \right) \dfrac{\partial y}{\partial \eta} + \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi \partial \eta} \right] \nonumber \\&#xA;&amp;amp;= \left[ \dfrac{\partial^2 \Phi}{\partial x^2} \dfrac{\partial x}{\partial \xi} \dfrac{\partial x}{\partial \eta} + \dfrac{\partial^2 \Phi}{\partial x \partial y} \dfrac{\partial y}{\partial \xi} \dfrac{\partial x}{\partial \eta} + \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi \partial \eta} \right] + \left[ \dfrac{\partial^2 \Phi}{\partial x \partial y} \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \eta} + \dfrac{\partial^2 \Phi}{\partial y^2} \dfrac{\partial y}{\partial \xi} \dfrac{\partial y}{\partial \eta} + \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi \partial \eta} \right] \nonumber&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or equivalently&#xA;\begin{equation}&#xA;\dfrac{\partial^2 \Phi}{\partial \xi \partial \eta} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi \partial \eta} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi \partial \eta} = \dfrac{\partial^2 \Phi}{\partial x^2} \dfrac{\partial x}{\partial \xi} \dfrac{\partial x}{\partial \eta} + \dfrac{\partial^2 \Phi}{\partial y^2} \dfrac{\partial y}{\partial \xi} \dfrac{\partial y}{\partial \eta} + \dfrac{\partial^2 \Phi}{\partial x \partial y} \left( \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \eta} + \dfrac{\partial x}{\partial \eta} \dfrac{\partial y}{\partial \xi} \right) \tag{6}\label{eq:local_2d_2nd_derivative-3}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Grouping \eqref{eq:local_2d_2nd_derivative-1}, \eqref{eq:local_2d_2nd_derivative-2} and \eqref{eq:local_2d_2nd_derivative-3} into the matrix form, we have&#xA;\begin{equation}&#xA;\begin{bmatrix}&#xA;\left( \dfrac{\partial x}{\partial \xi} \right)^2 &amp;amp; \left( \dfrac{\partial y}{\partial \xi} \right)^2 &amp;amp; 2 \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \xi} \\[1em]&#xA;\left( \dfrac{\partial x}{\partial \eta} \right)^2 &amp;amp; \left( \dfrac{\partial y}{\partial \eta} \right)^2 &amp;amp; 2 \dfrac{\partial x}{\partial \eta} \dfrac{\partial y}{\partial \eta} \\[1em]&#xA;\dfrac{\partial x}{\partial \xi} \dfrac{\partial x}{\partial \eta} &amp;amp; \dfrac{\partial y}{\partial \xi} \dfrac{\partial y}{\partial \eta} &amp;amp; \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \eta} + \dfrac{\partial x}{\partial \eta} \dfrac{\partial y}{\partial \xi}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\dfrac{\partial^2 \Phi}{\partial x^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial y^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial x \partial y}&#xA;\end{bmatrix}&#xA;=\begin{bmatrix}&#xA;\dfrac{\partial^2 \Phi}{\partial \xi^2} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi^2} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial \eta^2} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \eta^2} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \eta^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial \xi \partial \eta} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi \partial \eta} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi \partial \eta}&#xA;\end{bmatrix}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve the above system will give us the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear system in 3D has the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{bmatrix}&#xA;\left( \dfrac{\partial x}{\partial \xi} \right)^2 &amp;amp; \left( \dfrac{\partial y}{\partial \xi} \right)^2 &amp;amp; \left( \dfrac{\partial z}{\partial \xi} \right)^2 &amp;amp; 2 \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \xi} &amp;amp; 2 \dfrac{\partial y}{\partial \xi} \dfrac{\partial z}{\partial \xi} &amp;amp; 2 \dfrac{\partial x}{\partial \xi} \dfrac{\partial z}{\partial \xi} \\[1em]&#xA;\left( \dfrac{\partial x}{\partial \eta} \right)^2 &amp;amp; \left( \dfrac{\partial y}{\partial \eta} \right)^2 &amp;amp; \left( \dfrac{\partial z}{\partial \eta} \right)^2 &amp;amp; 2 \dfrac{\partial x}{\partial \eta} \dfrac{\partial y}{\partial \eta} &amp;amp; 2 \dfrac{\partial y}{\partial \eta} \dfrac{\partial z}{\partial \eta} &amp;amp; 2 \dfrac{\partial x}{\partial \eta} \dfrac{\partial z}{\partial \eta} \\[1em]&#xA;\left( \dfrac{\partial x}{\partial \zeta} \right)^2 &amp;amp; \left( \dfrac{\partial y}{\partial \zeta} \right)^2 &amp;amp; \left( \dfrac{\partial z}{\partial \zeta} \right)^2 &amp;amp; 2 \dfrac{\partial x}{\partial \zeta} \dfrac{\partial y}{\partial \zeta} &amp;amp; 2 \dfrac{\partial y}{\partial \zeta} \dfrac{\partial z}{\partial \zeta} &amp;amp; 2 \dfrac{\partial x}{\partial \zeta} \dfrac{\partial z}{\partial \zeta} \\[1em]&#xA;\dfrac{\partial x}{\partial \xi} \dfrac{\partial x}{\partial \eta} &amp;amp; \dfrac{\partial y}{\partial \xi} \dfrac{\partial y}{\partial \eta} &amp;amp; \dfrac{\partial z}{\partial \xi} \dfrac{\partial z}{\partial \eta} &amp;amp; \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \eta} + \dfrac{\partial x}{\partial \eta} \dfrac{\partial y}{\partial \xi} &amp;amp; \dfrac{\partial y}{\partial \xi} \dfrac{\partial z}{\partial \eta} + \dfrac{\partial y}{\partial \eta} \dfrac{\partial z}{\partial \xi} &amp;amp; \dfrac{\partial x}{\partial \xi} \dfrac{\partial z}{\partial \eta} + \dfrac{\partial x}{\partial \eta} \dfrac{\partial z}{\partial \xi} \\[1em]&#xA;\dfrac{\partial x}{\partial \eta} \dfrac{\partial x}{\partial \zeta} &amp;amp; \dfrac{\partial y}{\partial \eta} \dfrac{\partial y}{\partial \zeta} &amp;amp; \dfrac{\partial z}{\partial \eta} \dfrac{\partial z}{\partial \zeta} &amp;amp; \dfrac{\partial x}{\partial \eta} \dfrac{\partial y}{\partial \zeta} + \dfrac{\partial x}{\partial \zeta} \dfrac{\partial y}{\partial \eta} &amp;amp; \dfrac{\partial y}{\partial \eta} \dfrac{\partial z}{\partial \zeta} + \dfrac{\partial y}{\partial \zeta} \dfrac{\partial z}{\partial \eta} &amp;amp; \dfrac{\partial x}{\partial \eta} \dfrac{\partial z}{\partial \zeta} + \dfrac{\partial x}{\partial \zeta} \dfrac{\partial z}{\partial \eta} \\[1em]&#xA;\dfrac{\partial x}{\partial \xi} \dfrac{\partial x}{\partial \zeta} &amp;amp; \dfrac{\partial y}{\partial \xi} \dfrac{\partial y}{\partial \zeta} &amp;amp; \dfrac{\partial z}{\partial \xi} \dfrac{\partial z}{\partial \zeta} &amp;amp; \dfrac{\partial x}{\partial \xi} \dfrac{\partial y}{\partial \zeta} + \dfrac{\partial x}{\partial \zeta} \dfrac{\partial y}{\partial \xi} &amp;amp; \dfrac{\partial y}{\partial \xi} \dfrac{\partial z}{\partial \zeta} + \dfrac{\partial y}{\partial \zeta} \dfrac{\partial z}{\partial \xi} &amp;amp; \dfrac{\partial x}{\partial \xi} \dfrac{\partial z}{\partial \zeta} + \dfrac{\partial x}{\partial \zeta} \dfrac{\partial z}{\partial \xi}&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;\dfrac{\partial^2 \Phi}{\partial x^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial y^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial z^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial x \partial y} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial y \partial z} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial x \partial z}&#xA;\end{bmatrix}&#xA;=\begin{bmatrix}&#xA;\dfrac{\partial^2 \Phi}{\partial \xi^2} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi^2} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi^2} - \dfrac{\partial \Phi}{\partial z} \dfrac{\partial^2 z}{\partial \xi^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial \eta^2} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \eta^2} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \eta^2} - \dfrac{\partial \Phi}{\partial z} \dfrac{\partial^2 z}{\partial \eta^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial \zeta^2} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \zeta^2} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \zeta^2} - \dfrac{\partial \Phi}{\partial z} \dfrac{\partial^2 z}{\partial \zeta^2} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial \xi \partial \eta} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi \partial \eta} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi \partial \eta} - \dfrac{\partial \Phi}{\partial z} \dfrac{\partial^2 z}{\partial \xi \partial \eta} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial \eta \partial \zeta} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \eta \partial \zeta} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \eta \partial \zeta} - \dfrac{\partial \Phi}{\partial z} \dfrac{\partial^2 z}{\partial \eta \partial \zeta} \\[1em]&#xA;\dfrac{\partial^2 \Phi}{\partial \xi \partial \zeta} - \dfrac{\partial \Phi}{\partial x} \dfrac{\partial^2 x}{\partial \xi \partial \zeta} - \dfrac{\partial \Phi}{\partial y} \dfrac{\partial^2 y}{\partial \xi \partial \zeta} - \dfrac{\partial \Phi}{\partial z} \dfrac{\partial^2 z}{\partial \xi \partial \zeta}&#xA;\end{bmatrix}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Credit: Thanks C. Clason to edit the label for the equation on Mathjax.)&lt;/p&gt;&#xA;" OwnerUserId="20037" LastEditorUserId="20037" LastEditDate="2018-02-12T13:50:35.930" LastActivityDate="2018-02-12T13:50:35.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28821" PostTypeId="2" ParentId="28811" CreationDate="2018-02-13T01:42:32.447" Score="2" Body="&lt;p&gt;There are a number of software packages out there that you can use -- most are curated by the Computational Infrastructure in Geodynamics initiative (see &lt;a href=&quot;http://www.geodynamics.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.geodynamics.org&lt;/a&gt;). The fundamental question you will need to answer is what time scale and material description you want to investigate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you care about short term (at the intraseismic time scale) deformation, then PyLith is probably the code you want to look at. On the other hand, if you care about much longer time scales, you could take a look at ASPECT, which has been used for these sorts of simulations. There are many other codes in this area, and they all have their reason and realm of applicability, depending on what exactly you want to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Full disclosure: I'm one of the authors and maintainers of ASPECT.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-02-13T01:42:32.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28822" PostTypeId="1" CreationDate="2018-02-13T09:55:31.547" Score="1" ViewCount="287" Body="&lt;p&gt;Let's say that we have an objective function $f(\mathbf x,\mathbf y)$ which has the parameters $\mathbf x=[x_1\ldots x_n]$ and $\mathbf y=[y_1\ldots y_n]$. Here, $\mathbf y$ is a blackbox variable which is calculated from a simulation of a network $\mathcal N$ by taking  $\mathbf x$ as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$ is an objective function to be minimized for a given problem. Let's say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(\mathbf x,\mathbf y) = \sum\limits_{i=1}^{n}\alpha_ix_i+\beta_ix_iy_i&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf y= \mathcal N(\mathbf x)$ is a blackbox function whose analytical form is unknown which takes $\mathbf x$ as input&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal N$ refers to a network being simulated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\alpha_i$ and $\beta_i$ are constants, $i=1,\ldots,n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem has the following constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sum\limits_{i=1}^{n}x_i=C\\&#xA;x_i^\min \leq x_i \leq x_i^\max\\&#xA;0 \leq y_i \leq y_i^\max\\&#xA;0 \leq x_iy_i \leq (x_iy_i)^\max&#xA;$$&#xA;$C,x_i^\min, x_i^\max,y_i^\max,(x_iy_i)^\max$ being some fixed constants, $i=1,\ldots,n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A two-variable ($n=2$) example is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Objective:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\min 2x_1+4x_1y_1+3x_2+5x_2y_2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;[y_1,y_2]=\mathcal N([x_1,x_2])&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and $\alpha_1=2,\alpha_2=3,\beta_1=4,\beta_2=5$ according to the previous definitions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Constraints:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;x_1+x_2 = 10\\&#xA;0\leq x_1\leq 10\\&#xA;5\leq x_2 \leq 10\\&#xA;0\leq y_1\leq 5\\&#xA;0\leq y_2\leq 10\\&#xA;0\leq x_1y_1\leq 50\\&#xA;0\leq x_2y_2\leq 50\\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have gone through some of the stochastic algorithms such as &lt;em&gt;simulated annealing&lt;/em&gt;, &lt;em&gt;hill climbing&lt;/em&gt;,  &lt;em&gt;evolutionary algorithms&lt;/em&gt; like genetic algorithms, and so on. And I'm also aware that my problem containing a blackbox function within itself and subjected to a number of constraints has a high chance of falling into the ugly world of &quot;No Free Lunch Theorem&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any appropriate algorithms or suggestions that can solve such a problem?&lt;/p&gt;&#xA;" OwnerUserId="25459" LastEditorUserId="20688" LastEditDate="2018-03-29T20:09:46.050" LastActivityDate="2018-04-24T16:30:14.407" Title="Optimization of a blackbox function" Tags="|optimization|constrained-optimization|machine-learning|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28823" PostTypeId="2" ParentId="28815" CreationDate="2018-02-13T10:06:05.933" Score="1" Body="&lt;p&gt;Let us simplify the problem to understand better what is going on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us begin with the following 1D integral:&#xA;$$\int_{x_0}^{x_N}{f(x)\,dx}$$&#xA;We know that since the integral is a sum, the sum of the cut integral is equal to the previous one, i.e.&#xA;$$\int_{x_0}^{x_N}{f(x)\,dx}=\int_{x_1}^{x'}{f(x)\,dx}+\int_{x'}^{x_N}{f(x)\,dx}$$&#xA;Where $x'$ is a point such as $x'\in[x_0,x_N]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine, we cut the integral in $N$ intervals... if this number is big enough we are &quot;seeing&quot;  that the function $f(x)$ along a cut is almost a straight line (e.g the Earth is a sphere but from our point of view is a plane). If we do further assumptions, we can say even that the function $f$ in this tiny interval has an almost constant value. Let us name the corresponding value of $f(x)$ in the $i$-th cut $f_i$, thus the cut integral is simplified to:&#xA;$$\int_{x_0}^{x_N}{f(x)\,dx}=\sum_{i=0}^{N-1}\int_{x_i}^{x_{i+1}}{f(x)\,dx}\approx\sum_{i=0}^{N-1}\int_{x_i}^{x_{i+1}}{f_i\,dx}=\sum_{i=0}^{N-1}f_i\int_{x_i}^{x_{i+1}}{\,dx}=\sum_{i=0}^{N-1}{f_i\Delta x_i}$$&#xA;The last step is due the fact that $f_i$ is a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some type of functions that decay really quickly. Imagine that $f(x)=\frac{1}{x^2}$ (like in your problem), the integrand will be then:&#xA;$$\int_{x_0}^{x_N}{\frac{dx}{x^2}}$$&#xA;The contributions of the integral are negligible once $x$ is large enough, i.e. the error in the integration is proportional to $1/x$. Therefore you can stop once $x$ is large enough, i.e. it is not necessary to sum the $N$ intervals you have.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-02-13T10:13:49.517" LastActivityDate="2018-02-13T10:13:49.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28824" PostTypeId="1" AcceptedAnswerId="28899" CreationDate="2018-02-13T10:24:09.983" Score="0" ViewCount="123" Body="&lt;p&gt;I'm trying to analyse the behaviour of a wind turbine blade rigidly connected to a wall during fatigue testing, being excited in it's first mode in two orthogaonal directions simultaneously (the first 'up and down' and the first 'side to side' mode). I currently do this in two ways, both of which use a beam element model of the blade:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Modal superposition (build the stiffness and mass matrices of the system, perform modal analysis, project applied forcing onto eigenmodes, create time history by superposition of sinusoidal time histories)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Nonlinear transient analysis (update the stiffness and mass matrix at each time step, apply forcing using beam elements which displace like the hydraulic cylinders actually used to apply the force). Very computationally expensive so no use in an optimisation loop!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both methods produce comparable natural frequencies and mode shapes, but the modal superposition fails to capture behaviour which is seen in real life. The plot below shows this. The red line is generated by modal superposition, the blue line is real test data (but is similar to what is obtained with the nonlinear transient analysis). If I do a linear transient analysis then the behaviour is as the red line - it seems like when the loading in the $M_X$ direction is in phase with the loading in the $M_Y$ direction, the modes create some extra loading (and the opposite thing happens) when the two modes are opposed to each other, which keeps the overall amount of energy in the system constant over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know of any less computationally expensive methods than nonlinear transient which may be able to capture this behaviour? I don't want to create a fudge ideally - I'd like a physics based result! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IEqBd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IEqBd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be very much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="7637" LastActivityDate="2018-07-24T17:00:04.433" Title="Non-linearities in modal analysis of flexible beam" Tags="|finite-element|nonlinear-equations|eigensystem|numerical-modelling|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28825" PostTypeId="1" CreationDate="2018-02-13T13:26:41.573" Score="1" ViewCount="35" Body="&lt;p&gt;let´s say I estimated a Markov chain by splitting my data into traning and test data, and estimating the transition matrix for the traning data yields &#xA;$$ P=&#xA;    \begin{bmatrix}&#xA;    p11 &amp;amp; p12 &amp;amp; p13 \\&#xA;    p21 &amp;amp; p22 &amp;amp; p23 \\&#xA;    p31 &amp;amp; p32 &amp;amp; p33 \\&#xA;    \end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e, the transitions between states 1, 2 and 3. It is, of course, possible to obtain the n-step ahead matrix from this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 1: How do I evaluate the predictive capacity of this model on the test data? I know I should evaluate the MSE of this matrix on the transitions that actually occurred on the test, but how to do it? The caret package in R has some code, but only for other types of predictors (Knn, naive Bayes, etc, in any case, nothing whose output is a matrix). An easy approach would be to obtain the observed transitions matrix for the test data, divide by rows, and then get the element-wise difference with P, but thats probably not the best approach...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question2 (Related to question 1): I am unable to deploy a model in my entire dataset, due to data size. Is it possible to evaluate several Markov models on samples and then take some kind of aggregation of these to better reflect the overall data? Kind of what bagging is about, but once again, no examples exist for Markov chains, only decision trees, etc. &lt;/p&gt;&#xA;" OwnerUserId="26817" LastActivityDate="2018-02-13T13:26:41.573" Title="Validating a Markov chain and Bagging to find overall model" Tags="|data-analysis|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28826" PostTypeId="1" CreationDate="2018-02-13T15:57:18.637" Score="2" ViewCount="59" Body="&lt;p&gt;Maybe you can point me to some results already developed for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to solve for $X$ the following &quot;Sylvester-like&quot; equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ AX - XB = F(X)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A\in\mathbb{R}^{a\times n}$, $B\in\mathbb{R}^{p\times b}$, $X\in\mathbb{R}^{n\times p}$ and $F(X)$ is a nonlinear function of the elements of X.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I can solve this using generic numerical algorithms to solve for the roots of $ AX - XB - F(X) = 0$. Nevertheless, I would like to know if there is any algorithm or procedure that actually exploits the &quot;Sylvester-like&quot; structure of the equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="26859" LastActivityDate="2018-02-13T15:57:18.637" Title="Nonlinear Sylvester-Like Equation" Tags="|nonlinear-programming|nonlinear-equations|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28827" PostTypeId="1" AcceptedAnswerId="28829" CreationDate="2018-02-13T16:51:31.517" Score="3" ViewCount="174" Body="&lt;p&gt;I need to numerically evolve a &lt;a href=&quot;https://en.wikipedia.org/wiki/Density_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;density matrix&lt;/a&gt; using this formula(Actually I have more terms but right nows I am starting with this and facing problems):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot\rho(t) = -i[H(t), \rho(t)]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H(t)$ is the Hamiltonian which is of a form $H_o + \epsilon(t)H_1$ where both $H_o$ and $H_1$ are constant $20\times20$ matrices and $\epsilon(t)$ is a piece-wise constant function.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my numerical scheme is basically&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho(t + dt) = \rho(t) -i[H(t), \rho(t)]dt$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and substituting $\rho(t + dt)$ to the right hand side to get $\rho(t + 2dt)$ and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I am facing is that the entries of the density matrix quickly blow up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried checking whether the density matrix remains a valid density matrix at each iteration: namely if it is Hermitian, if it's trace is one and if it's eigenvalues sum to one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the first two conditions are always met but it's eignevalues are becoming larger and larger. In some sense it seems reasonable too as I could have changed the time-step $dt$ to say 2 times it's value and the eigenvalues of $\rho(t+dt)$ would change. So is it necessary to employ some sort of normalization procedure to the density matrix after each iteration?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I have tried to use very small values of $dt$ and there is no improvement so the problem is not in the value of the time-step.&lt;/p&gt;&#xA;" OwnerUserId="26855" LastEditorUserId="26855" LastEditDate="2018-02-28T05:52:43.980" LastActivityDate="2018-02-28T05:52:43.980" Title="Normalizing a density matrix at each iteration" Tags="|matrix|computational-physics|quantum-mechanics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28829" PostTypeId="2" ParentId="28827" CreationDate="2018-02-13T18:42:12.940" Score="5" Body="&lt;p&gt;As always with numerical problems, it's a good idea to simplify things as much as possible to isolate what's really going wrong. You can simplify the problem by considering the evolution of a pure state $\psi$ instead of an entire density matrix, so solving&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$i\dot\psi = H\psi$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the vector $\psi$ instead of the matrix $\rho$. Additionally, you can forget the perturbation $\epsilon$ and just assume that $H = H_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numerical method you're using is called the explicit Euler scheme. The explicit Euler scheme is guaranteed to give a sequence of solutions that, for any instant in time, will converge to the true solution in the limit as $\delta t$ goes to 0. However, it is &lt;em&gt;not&lt;/em&gt; guaranteed to preserve every property that might be useful to you, for example the fact that the probability amplitude should be conserved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see this in more detail, let's compare the exact solution $\Psi$ with the explicit Euler solution $\psi$. At time $t + \delta t$, you can write down an expression for $\Psi$ using the matrix exponential:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Psi(t + \delta t) = e^{-i\delta t\cdot H}\Psi(t).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the Hamiltonian $H$ is Hermitian, the operator $e^{-i\delta t\cdot H}$ is unitary. One consequence is that the norm of $\Psi(t + \delta t)$ is the same as the norm of $\Psi(t)$, so the probability amplitude is conserved for the exact system. With the explicit Euler approximation, however, we can write the value $\psi_{n + 1}$ of the wave function at time $(n + 1)\delta t$ in terms of the previous value $\psi_n$ as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi_{n + 1} = \psi_n - i\delta t H \psi_n = (I - i\delta t\cdot H)\psi_n.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The operator $I - i\delta t\cdot H$ is not unitary, so you can only expect conservation of probability in the limit as $\delta t \to 0$ and $n \to \infty$ such that $n\delta t$ stays fixed. This is really unfortunate because our approximation scheme doesn't preserve one of the most important properties of the physical system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Luckily, there are many more timestepping schemes besides explicit Euler. Another approach might be to approximate $e^{-i\delta tH}$ by the 1, 1-&lt;a href=&quot;https://en.wikipedia.org/wiki/Pad%C3%A9_approximant&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pade approximant&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$e^{-i\delta tH} \approx \left(1 + \frac{i\delta tH}{2}\right)^{-1}\left(1 - \frac{i\delta tH}{2}\right).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can show with a little linear algebra that this approximation is a unitary operator, in which case all of your discrete approximations to $\Psi$ will conserve probability up to floating-point error. This approximation is equivalent to using the Crank-Nicholson discretization of the problem instead of explicit Euler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://books.google.com/books?isbn=0898716292&quot; rel=&quot;nofollow noreferrer&quot;&gt;Leveque's book&lt;/a&gt; is a good introductory reference on finite difference methods. This will give you a lot of the necessary background on constructing stable discretizations of ODE and PDE. It's possible to design numerical methods explicitly for preserving things like the total probability amplitude; these are called &lt;em&gt;symplectic&lt;/em&gt; methods. The best reference I know is &lt;a href=&quot;https://books.google.com/books?isbn=3662050188&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer, Lubich and Wanner&lt;/a&gt;, but unfortunately this book is quite advanced. You could start by looking at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Verlet_integration&quot; rel=&quot;nofollow noreferrer&quot;&gt;Verlet method&lt;/a&gt;, which is the simplest symplectic method for Hamiltonian systems.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2018-02-15T22:30:41.373" LastActivityDate="2018-02-15T22:30:41.373" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28833" PostTypeId="1" CreationDate="2018-02-14T15:59:44.393" Score="1" ViewCount="64" Body="&lt;p&gt;I'm simulating a magnetic dipole that is subjected to an evolving magnetic field.  In ISO convention ($\theta$ is the polar angle and $\phi$ is the azimuthal angle), my equation of motion that is giving me issues is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot\phi = \frac{\textrm{garbage}}{\sin{\theta}}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this equation makes perfect sense, as $\theta$ goes to one of the two poles, $0$ or $\pi$, then the azimuthal angle has to quickly change (if $\theta$ went directly through the pole then $\phi$ would change almost instantly to be $\phi_0 \pm \pi$)  My question is not understanding what is going on; rather, how is the best way to approach this problem since $\ddot\phi$ blows up and the simulation stops at the poles?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have tried moving my coordinate system so the poles were in a place where possibly the trajectory wouldn't pass through it then convert my coordinates back (this worked but occasionally I do get a trajectory close and it blows up).  I've tried decreasing step size, which makes it better, but still if you get too close it blows up.  I've tried looking on-line since I'm sure I'm not the first to come across this problem, but was unable to find anything.  Any help would be great!  &lt;/p&gt;&#xA;" OwnerUserId="24134" LastActivityDate="2018-02-14T15:59:44.393" Title="Rotational kinematics problem @ $\theta = 0$" Tags="|nonlinear-equations|numerical-modelling|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28835" PostTypeId="1" AcceptedAnswerId="28838" CreationDate="2018-02-15T06:37:14.243" Score="0" ViewCount="98" Body="&lt;p&gt;I have the equilibrium equation in elasticity for a static case.i.e &#xA;&lt;strong&gt;Div T=0.&lt;/strong&gt;&#xA;For certain implementation, I have to get the x and y component equations and then derive the weak form separately. How is it done in that case?&#xA;I am aware of getting the weak form using the vector equation as such, but what I need are two separate weak form for the following equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WvMI1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WvMI1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26882" LastActivityDate="2018-02-15T20:58:52.630" Title="Deriving weak form of a set of scalar equations" Tags="|finite-element|variational-calculus|weak-solution|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28836" PostTypeId="1" CreationDate="2018-02-15T09:01:15.347" Score="1" ViewCount="380" Body="&lt;p&gt;I would like to apply a rank filter on an nXm numpy array. Let's say I have this array:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = np.random.randint(100,size=(5,5))&#xA;&#xA;array([[ 9, 71, 55, 72, 43],&#xA;       [46, 56, 44, 17, 64],&#xA;       [53, 51, 17, 71, 36],&#xA;       [12, 28, 97, 46, 35],&#xA;       [29, 55, 27, 48,  5]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to run a rank filter (kernel window of 3X3) that will give each central pixel in the kernel a value\rank (from 1-9 because it is a 3X3 window) according its position in the histogram. The largest number will get the value of 9 and the lowest number will get the value of 1. For example, the value 56 in position 1,1 will get the value 8 because according to its 8 neighbors, it is the second largest number. The value 44 in position 1,2 will get the value of 3 because it is the 3rd lowest number, there are only two values below it both are 17. (let's ignore for now the edges issues and the tied values issues).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.16.0/reference/generated/scipy.stats.rankdata.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.stats.rankdata&lt;/a&gt; can achieve my desired results but the problem is that it is not a convolved filter, meaning, that it works on the entire array and not according to a kernel method, also it gives an output of a 1Xm array, meaning that it does not keep the original nXm array shape.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think I need somehow to use this scipy.stats.rankdata filter as a moving kernel of a size of my choise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that scipy has convolve methods such as &lt;code&gt;ndimage.convolve&lt;/code&gt;, which I used in the past, and this method works as a running kernel window. But I can't figure out how to combine them in order to get a &quot;proper&quot; kernel ranking filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also saw that scipy has &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.16.1/reference/generated/scipy.ndimage.filters.rank_filter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.ndimage.filters.rank_filter&lt;/a&gt; but I don't understand how the ranking is actually work there.&lt;/p&gt;&#xA;" OwnerUserId="26607" LastActivityDate="2018-02-15T09:01:15.347" Title="Rank filter on an nXm array using python" Tags="|python|scipy|numpy|rank|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28838" PostTypeId="2" ParentId="28835" CreationDate="2018-02-15T11:24:10.320" Score="2" Body="&lt;p&gt;For your first equation you obtain the weak form integrating by parts:&#xA;$$\int_\Omega\left[ \frac{\partial T_{11}}{\partial x}+\frac{\partial T_{21}}{\partial y}\right]\phi(x,y)\,dxdy=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega\left[ \frac{\partial\phi}{\partial x}T_{11}+\frac{\partial\phi}{\partial y}T_{21}\right]\,dxdy=\int_\Omega\left[ \frac{\partial T_{11}\phi}{\partial x}+\frac{\partial T_{21}\phi}{\partial y}\right]\,dxdy$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Green's theorem, the right hand side of the previous equation equals to:&#xA;$$\int_{\partial\Omega} T_{11}\phi\,dy-\int_{\partial\Omega}T_{21}\phi\,dx$$&#xA;Since $n_x=\frac{dy}{dl}$ and $n_y=-\frac{dx}{dl}$ you can write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_\Omega\left[ \frac{\partial\phi}{\partial x}T_{11}+\frac{\partial\phi}{\partial y}T_{21}\right]\,dxdy=\int_{\partial\Omega} \left[T_{11}\phi n_x+T_{21}\phi n_y\right]\,dl$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can follow the same procedure with the other equation.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-02-15T20:58:52.630" LastActivityDate="2018-02-15T20:58:52.630" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28841" PostTypeId="1" CreationDate="2018-02-15T22:27:21.997" Score="3" ViewCount="108" Body="&lt;p&gt;I have been thinking about the &lt;a href=&quot;https://en.wikipedia.org/wiki/No_free_lunch_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;No Free Lunch (NFL) theorems&lt;/a&gt; lately, and I have a question which probably every one who has ever thought of the NFL theorems has also had. I am asking this question here, because I have not found a good discussion of it anywhere else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NFL theorems are very interesting theoretical results which do not hold in most practical circumstances, because a key assumption of the NFL theorems is rather strong. This assumption is, roughly speaking, that the performance of an algorithm is averaged over all problem instances drawn from a uniform probability distribution. In realistic applications the problems an algorithm typically encounters are NOT drawn from a uniform distribution, and are instead drawn from what is likely a very interesting and complicated distribution specific to the general problem setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, while the NFL theorems are quite interesting results, do they have any practical implications? Or are they merely theoretical results?&lt;/p&gt;&#xA;" OwnerUserId="26891" LastActivityDate="2018-02-15T22:27:21.997" Title="Are the No Free Lunch Theorems Useful for Anything?" Tags="|optimization|machine-learning|search|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28846" PostTypeId="1" CreationDate="2018-02-16T20:46:15.210" Score="7" ViewCount="2473" Body="&lt;p&gt;I'm having an issue solving an LP of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min z = c^Tx$$&#xA;$$\text{s.t.}$$&#xA;$$Ax \geq b$$&#xA;$$x\geq p$$&#xA;$1 \leq a_{ij} \ll b_i$, $p \leq 0$, and $c \geq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specific problems I'm running into with the ortools &lt;a href=&quot;https://developers.google.com/optimization/lp/glop&quot; rel=&quot;noreferrer&quot;&gt;GLOP&lt;/a&gt; solver are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The LP takes an insanely long time to solve on a distributed system where each worker has $\geq 30$ cores and roughly $240$ GB of memory. To me seems a little too long. I've solved instances of TSP in less time on smaller machines that did more computations (with CPLEX / Gurobi via AMPL). Furthermore, lasso regression is reducible to simplex and can run on similarly sized data in a fraction of a fraction of the time. Latency is extremely important and I'd love to get this cooking in less than $45$–$60$ minutes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I've checked and double checked the formulation generated, and the output (after $\sim 6$ hours) is that the problem is infeasible by returning a solution of $x = 0$. Obviously, this can't be the case since if we take $x = b$, we have a basic feasible solution, which implies the existence of an optimal solution.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;About the data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A$ has $\sim 3 \cdot 10^6$ rows and $\sim 1 \cdot 10^6$ columns. However, $A$ is also &lt;em&gt;very&lt;/em&gt; sparse with &lt;em&gt;no more&lt;/em&gt; than $2$ $nonzero$ entries per row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I'm new to the Google's ortools and am having difficulty finding documentation on the specific objects and methods in the $python$ API.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guesses as to where issues can be arising and possible solutions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If the solver is using two phase simplex or &quot;Big M&quot; to find an initial BFS. I want to know if I can speed up the process by providing an initial BFS to the GLOP solver (e.g. $x = b$) to speed things up? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If there's not an issue in $(1)$, then the problem must be too hard (doubtful). But, a better formulation may help. Sparsity of $A$ suggests I may be able to look at this as a Network problem. Anyone have any experience with this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I go a step further by using something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Dantzig%E2%80%93Wolfe_decomposition&quot; rel=&quot;noreferrer&quot;&gt;Dantzig Wolfe&lt;/a&gt; decomposition to separate the &quot;easy&quot; constraints ($x \geq p$) from the hard ones? Related, but I'm wondering if I can also lift the easy constraints into the objective. Any experience with this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(Related to 3) Not sure if &lt;a href=&quot;https://en.wikipedia.org/wiki/Column_generation&quot; rel=&quot;noreferrer&quot;&gt;column generation&lt;/a&gt; works here, but can it? Does anyone have any experience with it for problems of this nature? My intuition is that I can look at x as a set of patterns since x's are tightly coupled and belong to independent variable sets.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most importantly, if you've worked with Google's ortools before, can you point me to traditional API docs (not examples) for $python$? And, if you can, can you share what your experience has been as well as common gotchas and helpful performance improvement tips?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last, I'd love to use an open source solver like ortools rather than pay for something more expensive for this application. If you know of another open source solution that exists, please share in the comments. Anything that has a decent Java, Scala, or python API would be awesome. Extra points for distributed solutions. Extra Extra points for something that leverages Apache Spark's distributed environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="26899" LastEditorUserId="26899" LastEditDate="2018-02-17T16:42:12.050" LastActivityDate="2018-08-10T19:49:56.947" Title="Ways to speed up solving an LP with Google's ortools" Tags="|optimization|convex-optimization|constrained-optimization|linear-programming|mixed-integer-programming|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28851" PostTypeId="1" CreationDate="2018-02-18T10:38:19.913" Score="1" ViewCount="50" Body="&lt;p&gt;I'm faced with an interesting problem and need some help finding existing best techniques to solve it. The setup is that we're analyzing a large system trying to find what I call &quot;hidden subassemblies.&quot; Using round numbers, there are 1,000 different basic parts that are used in 10,000 different combinations or &quot;assemblies&quot;. (I call a unique full combination an &quot;assembly.&quot;) One assembly might have 150 parts, another 700. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we're trying to do is efficiently detect &quot;hidden subassemblies.&quot; Namely, groups of parts that frequently show up together. I'm sure there's an existing body of research and practice, algorithms and statistical methods for exactly this problem...but I don't know what it is, can't invent it myself (sad but true), and don't know the terminology to look it up. Can anyone point me in the right direction? This seems like a problem that would be found in manufacturing, anything related to groups of people, and genetics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case I haven't been clear enough, what I'm trying to find are groups of parts that are commonly (or always) used together. Say that there's a #3 bolt and it's always used along with a #3 nut. That's what I mean by a &quot;hidden subassembly.&quot; In practice, we're actually likely to find &lt;em&gt;much&lt;/em&gt; larger hidden subassemblies, but they may not be 100% the same across all assemblies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd be incredibly grateful for suggestions. While I'm no good at inventing mathematical solutions, I can usually bang them out (eventually) if pointed in the right direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks very much!&lt;/p&gt;&#xA;" OwnerUserId="26917" LastActivityDate="2018-02-18T10:38:19.913" Title="Finding &quot;hidden&quot; subassemblies: Suggestions for algorithms" Tags="|algorithms|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28852" PostTypeId="1" AcceptedAnswerId="28854" CreationDate="2018-02-18T15:39:08.807" Score="1" ViewCount="1712" Body="&lt;p&gt;&lt;strong&gt;Preface&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I seem to lack a fundamental understanding of best practise recommendations given by Intels MKL user guides for using MKL in threaded applications. So let's clarify it together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Wording and the question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are especially two different ways to optimize numerical code. Either OpenMP or the Message Passing Interface (MPI) - and combinations of both. It seems to me MKL settles with OpenMP internally out of the box. Intel recommends to not combine OpenMP parallelization with a manual parallelization (e.g., a domain decomposition at higher level) with the aid of MPI (&lt;a href=&quot;https://software.intel.com/en-us/mkl-macos-developer-guide-linking-with-threading-libraries&quot; rel=&quot;nofollow noreferrer&quot;&gt;see here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused now about what to do in multi-processor environments (read clusters) with many physical computation units. My first approach towards the topic was:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use MPI, do a domain decomposition of the problem, distribute the small chunks to all CPUs in the cluster and use sequential MKL (=&lt;code&gt;MKL without OpenMP&lt;/code&gt;) there.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I especially ask myself how it could be done differently when relying only on multi-threaded MKL (=&lt;code&gt;MKL with OpenMP&lt;/code&gt;) in the cluster: What if one matrix in a matrix vector product is to large to fit into memory of one compute node? Will it be automatically spread over all compute nodes by MKL? &lt;/p&gt;&#xA;" OwnerUserId="26920" LastActivityDate="2018-02-18T18:56:45.723" Title="General understanding of Intel MKL, threads and MPI" Tags="|parallel-computing|performance|mpi|intel-mkl|multicore|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28854" PostTypeId="2" ParentId="28852" CreationDate="2018-02-18T17:55:34.710" Score="3" Body="&lt;p&gt;Nothing stops you from decomposing the problem up yourself and feeding the relevant partitioned data into MKL sequentially, or even in parallel. It will work as long as you avoid data races, but you may experience performance penalties unless you are very careful about how you do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason it's discouraged to combine OpenMP code with MKL is that OpenMP is generally not equipped to handle the parallel scheduling challenge of nested parallelism (this is slowly starting to change). Thus the overhead of this limitation results in decreased performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no issue in doing this with MPI however, but you will have to be very careful with how you set process affinities in the MPI ranks and thread affinities in OpenMP, or else you will have again very degraded performance. You should look at the MPI documentation as well as the OpenMP documentation for how to coordinate the two correctly. It's not possible to give general advice here because there is no standardized way yet to handle this - you have to read the documentation of your OpenMP and MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, Intel also releases a version of MKL that is threaded with the Threading Building Blocks (TBB) library. This library can very efficiently handle nested parallelism, but then you will have to use TBB to achieve this (while it is possible to mix OpenMP and TBB, it is not recommended). This means you can call MKL within TBB parallelized code and experience less performance penalty than if you had done the same within OpenMP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I should add that MKL is strictly focused on &lt;em&gt;single node performance,&lt;/em&gt; thus any extra parallelism you add to it in the form of OpenMP or MPI must be managed yourself. It will not distribute matrices for you nor perform block linear algebra. You will have to block the matrices yourself and perform the correct sequence of BLAS or LAPACK calls to achieve the result you want.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2018-02-18T18:56:45.723" LastActivityDate="2018-02-18T18:56:45.723" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28857" PostTypeId="1" CreationDate="2018-02-19T11:54:53.220" Score="2" ViewCount="338" Body="&lt;p&gt;I am using the FFTW MPI in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a simple question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quoting from  fftw.org &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The multi-dimensional transforms of FFTW, in general, compute simply the separable product of the given 1d transform along each dimension of the array. Since each of these transforms is unnormalized, computing the forward followed by the backward/inverse multi-dimensional transform will result in the original array scaled by the product of the normalization factors for each dimension (e.g. the product of the dimension sizes, for a multi-dimensional DFT).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Imagine we have $\tilde{A}$ in the Fourier space with the coresponding $A$ in the physical space. I want to compute $\tilde{B}$ that is $\tilde{A}^2$. The normalization factor is $F_{norm}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is $A=IFFT(\tilde{A})$ (inverse fourier transform, A now is real)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B=A^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=A*F_{norm}$, $B=B*F_{norm}$ (we have to normalize  before to transform back to Fourier space.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{A}=FFT(A)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{B}=FFT(B)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Do I normalize A before or after the computation of B ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Does anything change if B is used directly in physical space ?&lt;/p&gt;&#xA;" OwnerUserId="26384" LastActivityDate="2018-02-19T11:54:53.220" Title="Use of multidimensional FFTW and normalisation factor" Tags="|c|fftw|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28858" PostTypeId="1" CreationDate="2018-02-19T15:39:42.133" Score="1" ViewCount="149" Body="&lt;p&gt;I have a task to plot average temperature profile at each cross-section in the axial direction of a combustion chamber. I have $x$, $y$, $z$ coordinates data in excel as well as the corresponding temperature according to the rows, example ($x=1$,$y=1$,$z=1$, temperature = 500°C). I have around 25000 rows of it. I have also run a simulation in COMSOL using eigenfrequency study. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) From Matlab, I'm trying to create multiple 2-D meshgrids or slices in the axial direction, then from each slice calculate the average temperature according to the grids and area then plot up the average temperature values according to the z coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) or I could create a volume from the set of points using Delaunay or voxelization then do multiple volumetric slices and from the slices calculate the average temperature values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or is there a better way to compute? like in COMSOL?&lt;/p&gt;&#xA;" OwnerUserId="26896" LastEditorUserId="20688" LastEditDate="2018-04-08T14:20:43.010" LastActivityDate="2021-04-23T00:06:25.563" Title="temperature profile on axial direction of a chamber" Tags="|matlab|visualization|comsol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28859" PostTypeId="1" AcceptedAnswerId="28861" CreationDate="2018-02-19T19:37:19.790" Score="2" ViewCount="233" Body="&lt;p&gt;This question is a continuation of &#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/21866/fourth-order-imex-runge-kutta-method&quot;&gt;Fourth order IMEX Runge-Kutta method&lt;/a&gt;, concerning the implementation. Is seems to me that the first implicit stage value involves a direct evaluation, rather than solving some equation. That is, if &#xA;$\frac{d U(t)}{dt} = F_{\textit{STIFF}}(t)+F_{\textit{NONSTIFF}}(t)$, and the stages are denoted by $k^{i}_{\textit{STIFF}}$ and $k^{i}_{\textit{NONSTIFF}}$, then $k^{1}_{\textit{STIFF}} = F_{\textit{STIFF}}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems to be in common for most schemes, as the Butcher tableau for the implicit part has only nonzero elements in the first column. However, this does not seem like an implicit treatment. What have I missed?&lt;/p&gt;&#xA;" OwnerUserId="19807" LastActivityDate="2020-07-19T18:09:05.617" Title="Implementation details for high order IMEX methods by Kennedy and Carpenter" Tags="|ode|runge-kutta|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28861" PostTypeId="2" ParentId="28859" CreationDate="2018-02-19T22:52:33.490" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;This seems to be in common for most schemes, as the Butcher tableau for the implicit part has only nonzero elements in the first column. However, this does not seem like an implicit treatment. What have I missed?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is called an ESDIRK method, or an Explicit first step Singly-Diagonal Implicit Runge-Kutta method. The explicit first step is required to achieve the high order conditions for DAEs (called &amp;quot;stiffly-accurate&amp;quot;). Another order condition is that FSAL is required, so you'll notice that all of the schemes do that too, and since it's ESDIRK you can just flip that over to be the first stage and so actually there's no explicit evaluation required (depending on the quantities you are saving).&lt;/p&gt;&#xA;&lt;p&gt;You need to be careful there though since the iterations will only be numerically stable for highly stiff equations if you relax the &lt;code&gt;dt*f&lt;/code&gt; term instead of the input itself. This is something discussed in &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/0168927495001158&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shampine's TRBDF2&lt;/a&gt; paper. Thus &lt;a href=&quot;http://runge.math.smu.edu/arkode_dev/doc/guide/build/html/Mathematics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Sundials implementation is a good resource&lt;/a&gt;, but if you've ever opened up Sundials you'll notice that something weird happens in the test equations (and the test equations require dropping the convergence coefficient quite a bit). My theory is that this is due to &lt;a href=&quot;http://runge.math.smu.edu/arkode_dev/doc/guide/build/html/Mathematics.html#nonlinear-solver-methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;their iteration choice&lt;/a&gt; since the &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/split_ode_solve.html#Implicit-Explicit-(IMEX)-ODE-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl versions&lt;/a&gt; don't have this issue. This means that, when you do find an equation where they work, &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffODE/Hires.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;they have higher runtimes than they should&lt;/a&gt;... For reference, here's the open source &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/v3.2.0/src/perform_step/kencarp_kvaerno_perform_step.jl#L425-L641&quot; rel=&quot;nofollow noreferrer&quot;&gt;Julia implementation of the 3rd order KenCap in out-of-place form if you want to copy some things&lt;/a&gt;. In-place forms along with the other orders are also in that file.&lt;/p&gt;&#xA;&lt;h2&gt;Edit&lt;/h2&gt;&#xA;&lt;p&gt;As mentioned &lt;a href=&quot;https://scicomp.stackexchange.com/a/29042/18981&quot;&gt;in another post&lt;/a&gt;, the FSAL property is only on the implicit part of the tableau. Thus when you use this as an ESDIRK method it is FSAL, and that's one of the stiffly-accurate order conditions. However, the update value is not directly from the implicit tableau when solving with IMEX, so the first step of the implicit part does need a function evaluation.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:09:05.617" LastActivityDate="2020-07-19T18:09:05.617" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="28862" PostTypeId="2" ParentId="28822" CreationDate="2018-02-19T22:55:44.050" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://nlopt.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLopt&lt;/a&gt; is a nonlinear optimization library that can handle these kinds of problems. Most libraries cannot handle all of those constraints, but NLopt has some algorithms that can. From &lt;a href=&quot;https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the algorithms page&lt;/a&gt;, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Of these algorithms, only ISRES and ORIG_DIRECT support nonlinear inequality constraints, and only ISRES supports nonlinear equality constraints.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ISRES can be a little computationally intensive but it's the only global optimization algorithm I know of that allows nonlinear equality constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NLopt library has really nice bindings to Julia at &lt;a href=&quot;https://github.com/JuliaOpt/NLopt.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLopt.jl&lt;/a&gt; if you'd like to use it in a higher level language. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-02-19T22:55:44.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28863" PostTypeId="1" CreationDate="2018-02-20T06:45:58.447" Score="1" ViewCount="307" Body="&lt;p&gt;Has anyone successfully used the linear elasticity equations to model a cantilever problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation I am solving is simply:&#xA;  $$\nabla \cdot \boldsymbol{\sigma} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to boundary conditions at the fixed end, free surfaces, as well as where the load is applied. I apply the load as a shear stress/traction boundary condition on the end face.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having no luck getting my results to match the analytical solution. I plotted the lateral displacement along the length of the beam and the shape of my curve is approximately linear, whereas the analytical solution is a cubic. When comparing max displacement in my numerical result vs the analytical result, I am off by orders of magnitude. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've studied my code and numerics carefully and I do not see any mistakes, but I could be wrong. Is the cantilever problem able to be modeled with the equation above? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Editing this post based on the comments: In addition to the above equation, I am also using Hooke's law and the stress strain displacement relationship:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\boldsymbol{\sigma} = 2\mu \boldsymbol{\epsilon} + \lambda tr(\boldsymbol{\epsilon}) I$$&#xA;$$\boldsymbol{\epsilon} =0.5[\boldsymbol{\nabla u} + \boldsymbol{(\nabla u)}^T]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Editing the post to include the analytical solution and dimensions.&lt;/em&gt; &#xA;The analytical solution that I am using is (&lt;a href=&quot;http://ruina.mae.cornell.edu/Courses/ME4735-2012/Rand4770Vibrations/BeamFormulas.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ruina.mae.cornell.edu/Courses/ME4735-2012/Rand4770Vibrations/BeamFormulas.pdf&lt;/a&gt;):&#xA;$$v = \frac{Fx^2}{6EI}(3L-x)$$&#xA;Where v is the lateral displacement or the y-displacement in the link. The beam I am currently modeling has L = 10m&#xA;with a square cross section of 1m x 1m. So the moment of inertia, $I=\frac{1}{12}$ for this problem. &#xA;I apply a force of $10^6$N at the free end, which is equivalent to a $10^6$Pa shear stress at the free end. I prescribe this as a traction boundary condition so that the boundary condition at the free surface is:&#xA;$$\sigma = \begin{bmatrix}&#xA;0 &amp;amp; -10^6 &amp;amp; 0 \\&#xA;-10^6 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 0&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or in terms of tractions:&#xA;$$&#xA;T = \sigma n = \begin{bmatrix}&#xA;0 \\&#xA;-10^6 \\&#xA;0&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attached is the result I am getting for lateral displacement for $E=2*10^{11} Pa$. &lt;a href=&quot;https://i.stack.imgur.com/2wWqU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2wWqU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23915" LastEditorUserId="23915" LastEditDate="2018-02-20T21:43:06.077" LastActivityDate="2018-02-20T21:43:06.077" Title="Solving linear elasticity equations for a cantilever problem" Tags="|solid-mechanics|" AnswerCount="0" CommentCount="20" ContentLicense="CC BY-SA 3.0" />
  <row Id="28864" PostTypeId="1" AcceptedAnswerId="28906" CreationDate="2018-02-20T10:10:51.873" Score="1" ViewCount="700" Body="&lt;p&gt;I am solving the 1D diffusion equation by discretization using the method of lines. My problem is that I don't manage to ensure mass conservation. I have read many similar questions about the topic but it seems that the problems are related to advection terms (which I don't have here). I actually believe that my problem is related to the boundary condition implementation as changing the partition coefficient has an impact on how mass is conserved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation I am trying to solve is represents the release/uptake of a component from a membrane or film into a liquid  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;{\partial c\over\partial t}=D{\partial^2 c\over\partial x^2} \text{    for t&amp;gt;0 and 0&amp;lt;x&amp;lt;H}  \quad[1]&#xA;$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;c(x,0)=1 \quad[2], \text{ initial condition}&#xA;$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;{dc\over dx} \mid_{(0,t)} =0\ \text{  for t&amp;gt;0,} \:[3], \text{  no flux on one side as it is insulated}&#xA;$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;D{dc\over dx} \mid_{(H,t)} =k_L(C^*_L(t)-C\mid_{(H,t)}) \quad[4],\text{ known flux on the other side}&#xA;$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;{dc_L\over dt} =-k_L a(C^*_L(t)-C\mid_{(H,t)}) \quad[5], \text{ mass conservation in the liquid}&#xA;$$   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\text{with } a=\frac{A}{V_L}=\frac{\text{exchange area}}{\text{liquid volume}} \quad[6] &#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA; \text{  and }  C^*_L = C_L ·K \quad[7], \text{ being K the partition coefficient defined as} \quad[7]&#xA;$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ &#xA;K=\frac{ C^{\infty}_F }{C^{\infty}_L}=\frac{\text{concentration in the film after a long time}}{\text{concentration in the liquid after a long time}} \quad[8]&#xA;$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have discretized the previous equations as follows. Let the index $i$ the space coordinate $x_i$ so that  $h=H/N$ and $i=0,1,\ldots,N$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dc_0}{dt}=D \frac{-2c_0+2c_1}{h^2} \quad [9]&#xA;$$&#xA;$$&#xA;{dc_i\over dt}  =D{ c_{i+1}-2c_i+c_{i-1}\over h^2} \text{ for i = 1...N-1} \quad[10]&#xA;$$&#xA;and&#xA;$$&#xA;\frac{dc_N}{dt}=D \frac{-2c_N+2c_{N-1}}{h^2}+\frac{2k_L}{h}(C^*_L-c_N) \quad [11]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have evaluated the conservation of mass as the sum of the mass in the film and in the liquid. I have realised that several things did &lt;strong&gt;not&lt;/strong&gt; affect the mass conservation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Increasing or decreasing the number of nodes&lt;/li&gt;&#xA;&lt;li&gt;Increasing or decreasing the diffusivity&lt;/li&gt;&#xA;&lt;li&gt;Increasing or decreasing the mass transfer coefficient ($k_L$)&lt;/li&gt;&#xA;&lt;li&gt;Increasing or decreasing the specific exchange area ($a$)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But the following did affect how mass was conserved:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Changing the partition coefficient ($K$)&lt;/li&gt;&#xA;&lt;li&gt;Changing the film thickness ($H$) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18706" LastEditorUserId="18706" LastEditDate="2018-02-26T09:44:33.173" LastActivityDate="2018-02-26T09:44:33.173" Title="Mass conservation in 1d diffusion by method of lines" Tags="|pde|finite-difference|method-of-lines|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28865" PostTypeId="2" ParentId="26763" CreationDate="2018-02-20T11:11:20.433" Score="1" Body="&lt;p&gt;Although the question has a great answer, here's &#xA;a rule of thumb for small singular values, with a plot.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If a singular value is nonzero but very small,&#xA;  you should define its reciprocal to be zero,&#xA;  since its apparent value is probably an artifact of roundoff error, not a meaningful number.&#xA;  A plausible answer to the question &quot;how small is small ?&quot;&#xA;  is to edit in this fashion all singular values whose ratio to the largest&#xA;  is less than $N$ times the machine precision $\epsilon$ .&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;$\qquad$ &amp;mdash; &lt;a href=&quot;http://apps.nrbook.com/empanel/index.html?pg=795&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Recipes p. 795&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added: the following couple of lines calculate this rule-of-thumb.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python2&#xA;&#xA;from __future__ import division&#xA;import numpy as np&#xA;from scipy.sparse.linalg import svds  # sparse, dense or LinOp&#xA;&#xA;#...............................................................................&#xA;def howsmall( A, singmax=None ):&#xA;    &quot;&quot;&quot; singular values &amp;lt; N float_eps sing_max  may be iffy, questionable&#xA;        &quot;How small is small ?&quot;&#xA;        [Numerical Recipes p. 795](http://apps.nrbook.com/empanel/index.html?pg=795)&#xA;    &quot;&quot;&quot;&#xA;        # print &quot;%d singular values are small, iffy&quot; % (sing &amp;lt; howsmall(A)).sum()&#xA;        # small |eigenvalues| too ?&#xA;    if singmax is None:&#xA;        singmax = svds( A, 1, return_singular_vectors=False )[0]  # v0=random&#xA;&#xA;    return max( A.shape ) * np.finfo( A.dtype ).eps * singmax&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&#xA;The Hilbert matrix seems to be widely used as a test case for roundoff error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/megsx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/megsx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here low-order bits in the mantissas of the Hilbert matrix are zeroed,&#xA;&lt;code&gt;A.astype(np.float__).astype(np.float64)&lt;/code&gt;,&#xA;then &lt;code&gt;np.linalg.svd&lt;/code&gt; is run in &lt;code&gt;float64&lt;/code&gt;.&#xA;(Results with &lt;code&gt;svd&lt;/code&gt; all &lt;code&gt;float32&lt;/code&gt; are about the same.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply truncating to &lt;code&gt;float32&lt;/code&gt; might even be useful for denoising high-dimensional data,&#xA;e.g. for train / test classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Real test cases would be welcome.&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2018-02-27T17:23:33.287" LastActivityDate="2018-02-27T17:23:33.287" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28866" PostTypeId="1" AcceptedAnswerId="28875" CreationDate="2018-02-20T14:18:59.543" Score="0" ViewCount="156" Body="&lt;p&gt;To solve stationary, incompressible, inviscid and irrotational flow around a circular cylinder, I am using general coordinates. Since the flow is symmetrical, we only consider the upper half of the plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I derived a PDE and proper boundary conditions in the physical domain (left image) given by $G := \{(x_1,x_2 : R_1^2 \leq x_1^2 + x_2^2 \leq R_2^2, x_2 \geq 0\}$. This PDE is $\Delta \phi = 0$, where $\phi$ is the perturbation potential.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vCaEY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vCaEY.png&quot; alt=&quot;Physical and logical domain&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general coordinates, the expression for $\Delta \varphi$ can be expressed as (using the Einstein summation convention)&#xA;\begin{align*}&#xA;	\Delta \phi = \frac{1}{\sqrt{g}}\frac{\partial}{\partial \xi^\alpha}\left(\sqrt{g}g^{\alpha\beta}\frac{\partial\varphi}{\partial\xi^\beta}\right),&#xA;\end{align*}&#xA;where $\xi(\mathbf{x})$ is the (inverse) coordinate mapping and $g$ is the (contravariant) metric tensor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the cylinder itself, the BC is given by &#xA;$$&#xA;    \nabla\phi\cdot\mathbf{n} = -U_\infty n_1,&#xA;$$&#xA;where $U_\infty$ is is the free stream velocity in the $x_1$ direction and $\mathbf{n}$ is the normal on the surface of the cylinder. In general coordinates, this BC becomes&#xA;$$&#xA;    \frac{\partial \varphi}{\partial \xi^2} = -U_\infty (\mathbf{a}^{(2)})_1,&#xA;$$&#xA;where $\mathbf{a^{(\alpha)}}$ is the contravariant basis vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other boundary condition (horizontal left and right of the cylinder) are given by (see derivation below)&#xA;$$&#xA;    \frac{\partial^2 \varphi}{\partial x_2^2} = 0 \ \text{ and } \ \frac{\partial^2 \varphi}{\partial x_1\partial x_2} = 0&#xA;$$&#xA;and I would like to also write this is in coordinate invariant form (in terms of the covariant/contravariant basis vectors and the metric tensor), but I have no idea where to start. Any suggestions would be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; The derivation of the symmetric boundary condition is as follows. It is given that the velocity field $\mathbf{u}$ can be expressed as $u_i = \partial_i{\Phi}$, where $\Phi$ is the velocity potential function given by&#xA;$$&#xA;\Phi = U_\infty x_1 + \phi.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought that on the horizontal boundary&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;   \frac{\partial\mathbf{u}}{\partial x_2} = \mathbf{0}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Leading to the two boundary conditions given above.&lt;/p&gt;&#xA;" OwnerUserId="26537" LastEditorUserId="26537" LastEditDate="2018-02-21T08:17:19.493" LastActivityDate="2018-02-22T06:36:56.277" Title="Second derivative in coordinate invariant form" Tags="|pde|fluid-dynamics|boundary-conditions|tensor|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28868" PostTypeId="1" CreationDate="2018-02-20T03:03:10.383" Score="1" ViewCount="52" Body="&lt;p&gt;I'm trying to make a numerical simulation of pulsar magnetosphere using FDTD on a log-spherical Yee lattice for fields and PIC for plasma particles. Field part is working like charm but issue arises when adding the plasma.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To simulate the dynamics of charged plasma particles I use Newton-Lorenz equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{d\gamma\vec{v}}{dt} = \frac{e}{m}\left( \vec{E} + \frac{\vec{v}}{c} \times \vec{B} \right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adopting CGS system of units, characteristic values for $\vec{E}$ and $\vec{B}$ are of order of magnitule $10^{12}$, charge to mass ratio for electron is of order of magnitude $10^{17}$, so the value for acceleration is ~$10^{29}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the code itself I normalized all variables in the following manner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$&#xA;\vec{u} = \frac{\gamma\vec{v}}{c}\\&#xA;\vec{E}^{*} = \vec{E}/B_0\\&#xA;\vec{B}^{*} = \vec{B}/B_0\\&#xA;\tau = \frac{ct}{R}\\&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $B_0=10^{12}$, $R=10^6$ (radius of the pulsar) and $c$ is the speed of light. As a result, my equation looks as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{d\vec{u}}{dt} = \frac{eRB_0}{mc^2}\left(\vec{E}^{*}+\frac{\vec{u}\times\vec{B}^{*}}{\gamma}\right)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normalization factor $\frac{eRB_0}{mc^2}\sim 10^{15}$, acceleration is still huge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I may choose different normalization for time as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$t = t_0\tau\\&#xA;\frac{eB_0}{mc}t_0 = 1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But with $\tau$ chosen in that way it takes literally forever for fields to get to stationary configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can I do?&lt;/p&gt;&#xA;" OwnerUserId="26990" OwnerDisplayName="Tajimura" LastActivityDate="2018-02-22T02:13:37.440" Title="Huge accelerations in plasma simulation" Tags="|computational-physics|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28869" PostTypeId="1" CreationDate="2018-02-20T20:49:02.443" Score="0" ViewCount="278" Body="&lt;p&gt;I have a very simple problem, but can't seem to understand what I need to do.  In simulating a pendulum from it's jerk equation, I'm having a hard time setting initial conditions to get it to work out.  So for example the equation of motion for a pendulum is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\ddot{\theta} = -A \sin{\theta}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I'm interested in modeling from it's jerk,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dddot{\theta}=-A\, \dot{\theta} \cos{\theta}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a little change of coordinates $\theta = x, \dot{\theta}=y$, and $\ddot{\theta} = z$, can yield three equations of motion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot{x}=y$$&#xA;$$\dot{y}=z$$&#xA;$$\dot{z}=-A \, y \cos{x}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem are the initial conditions.  Since the problem is nonlinear and cannot be solved, how do I pick initial conditions that will yield the correct results (oscillating symmetrically around $x=0$ (from the view only looking at the $x$ axis)).  Specifically I would like to know $(x_0,y_0,z_0)=(a,0,?)$ meaning I pull it back at angle $a$ and let it go.  The general process is what I'm after, meaning I could also solve,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot{x}=y$$&#xA;$$\dot{y}=z$$&#xA;$$\dot{z}=-B\, z-A \, y \cos{x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or anything else that would be a symmetric perturbation (around $x=0$) if you will.  I'm able so solve this, but currently I'm cheating.  I set $(x_0,y_0,z_0)=(0,v,0)$ (which is always good for any $v$ since the time derivative at the bottom of the swing will be zero for the velocity).  I then look at $x_{max}$ and the corresponding $z$, but I would like it to be a lot more clean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="24134" LastEditorUserId="24134" LastEditDate="2018-02-20T21:21:33.163" LastActivityDate="2018-02-25T09:43:27.387" Title="Initial conditions for pendulum Jerk equation" Tags="|nonlinear-equations|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28870" PostTypeId="1" CreationDate="2018-02-20T21:07:50.547" Score="0" ViewCount="451" Body="&lt;p&gt;I would like to do a Generalized Schur (or QZ) decomposition for a pair of complex matrices $A$ and $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the following class:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Eigen::GeneralizedEigenSolver&amp;lt;_MatrixType&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but it seems that it only works for real matrices. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="26941" LastEditorUserId="20688" LastEditDate="2018-02-24T08:55:35.473" LastActivityDate="2018-02-24T08:55:35.473" Title="How to do a Generalized Complex Schur (or QZ) Decomposition with Eigen C++?" Tags="|linear-algebra|c++|eigensystem|lapack|eigen|" AnswerCount="1" CommentCount="5" ClosedDate="2018-03-02T15:29:52.403" ContentLicense="CC BY-SA 3.0" />
  <row Id="28872" PostTypeId="2" ParentId="28869" CreationDate="2018-02-21T03:53:02.813" Score="1" Body="&lt;p&gt;You can't independently choose initial conditions for $x,y,z$. That's because you have that $z=\ddot\theta$ and consequently $z(0)=\ddot\theta(0)=-A\sin(\theta(0))=-A\sin(x(0))$. So the initial conditions for $x$ and $z$ are not independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you go with the original equation of second order, you will see that you can independently choose $\theta(0)$ (the initial angle) and $\dot\theta(0)$ (the initial angular velocity of the pendulum). You can then solve the differential equation to obtain $\theta(t)$, and the jerk is simply the third derivative of this solution.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-02-21T03:53:02.813" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28874" PostTypeId="1" AcceptedAnswerId="28877" CreationDate="2018-02-21T07:31:00.660" Score="0" ViewCount="7697" Body="&lt;p&gt;I am trying to solve a cubic equation in Python. However I am getting only one root of the equation. Please find the code snippet below. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;from scipy import optimize as op&#xA;&#xA;def my_func(p):&#xA;&#xA;    k = 0.17&#xA;    d = 3e-6 &#xA;    A = 1.6e-9 &#xA;    epsilon = 8.85e-12 &#xA;    Vspi = np.sqrt((8*k*np.power(d,3))/(27*epsilon*A)) &#xA;&#xA;    Vdc = 0.2 * Vspi&#xA;&#xA;    xeq = p&#xA;    F = (k*np.power(xeq,3))-(2*k*d*np.power(xeq,2))+(2*k*np.power(d,2)*xeq)-(epsilon*A*np.power(Vdc,2))&#xA;    return F&#xA;&#xA;zguess = 0.5e-6&#xA;&#xA;z = op.fsolve(my_func,zguess)&#xA;&#xA;print(z)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the mistake ? I am supposed to obtain three roots. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards,&#xA;Raghu&lt;/p&gt;&#xA;" OwnerUserId="26893" LastEditorUserId="414" LastEditDate="2018-02-21T10:26:33.397" LastActivityDate="2018-02-21T10:26:33.397" Title="Solving a cubic equation in Python using fsolve - only one root is obtained" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28875" PostTypeId="2" ParentId="28866" CreationDate="2018-02-21T08:33:45.433" Score="0" Body="&lt;p&gt;&lt;strong&gt;IMPORTANT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To keep things clear... the Laplacian operator $\triangle$ must come with ONLY one BC in each point of the boundary, this is, you can not specify 2 BC as you propose on the boundary. To se this.. I ask you sth. How many BC you need in the boundary to solve this unidimensional BVP?&#xA;$$\frac{d^2 u}{d x^2}=0,\quad x\in[x_1,x_2]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the BC you are looking for this boundary is the following:&#xA;$$ \vec{n}^T\sqrt{g^{\alpha\beta}}\left[\vec{grad}(\phi)\right]=0$$&#xA;Where $g^{\alpha\beta}$ is the contravariant metric tensor, $\vec{n}$ is the exterior normal vector to upper and lower boundaries and the gradient operator is expressed in the new coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;Since in a symmetry plane no flow crosses this boundary we have that the BC is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{n}^T\vec{u}=0$$&#xA;Therefore, since $\vec{u}=\vec{grad}\,\phi$ you have the required zero normal gradient BC&#xA;$$\vec{n}^T\vec{grad}\,\phi=0$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-02-22T06:36:56.277" LastActivityDate="2018-02-22T06:36:56.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28876" PostTypeId="1" AcceptedAnswerId="28894" CreationDate="2018-02-21T09:50:53.743" Score="4" ViewCount="203" Body="&lt;p&gt;In some of my computations I calculate a scalar value $\lambda_h$ (in my case an eigenvalue) depending on a finite element discretization of the domain. Usually we can manage to find estimates of the form &#xA;$$ |\lambda - \lambda_h| \leq C h^k $$&#xA;where $k$ depends on the problem, the degree of the approximation, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I was wondering under which circumstances (if any) it is possible to say that the error of a scalar function depending on the FE approximation has a Taylor-like expansion:&#xA;    $$ \lambda_h = \lambda_{exact} + C_1h^{\alpha_1}+C_2h^{\alpha_2}+...$$&#xA;  where $\alpha_i$ is an increasing sequence. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;If there are any relevant references, I am interested. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Context:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is in connection with &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26590/extrapolation-after-successive-finite-element-refinement&quot;&gt;one eariler question of mine&lt;/a&gt;, regarding the good behavior of some extrapolation techniques applied to my problem. It turns out that the extrapolation I use (Wynn's epsilon algorithm) gives the limit of the sum of $n$ geometric sequences starting from $2n+1$ values. The existence of a Taylor-like expansion for the eigenvalue would justify this behavior since the first few Taylor coefficients will be cancelled when doing the extrapolation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example presented below the initial computations using only finite elements give a convergence of order 2, while the extrapolation gives something like order 6, quickly running towards machine precision. See the picture:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JgxTK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JgxTK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="3548" LastEditDate="2018-02-23T12:05:09.017" LastActivityDate="2018-02-23T12:05:09.017" Title="Taylor expansion of error - Finite elements approximation" Tags="|finite-element|eigenvalues|error-estimation|approximation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28877" PostTypeId="2" ParentId="28874" CreationDate="2018-02-21T10:11:44.283" Score="5" Body="&lt;p&gt;Since you use a numerical solver starting from an initial guess, the algorithm will give you the first solution found. You should run the algorithm three times starting from good guesses for the root in order to find all three roots. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, since you are looking for roots of polynomials, why not use directly numpy.roots? This should give you all roots. See &lt;a href=&quot;https://docs.scipy.org/doc/numpy/reference/generated/numpy.roots.html&quot; rel=&quot;noreferrer&quot;&gt;this link&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2018-02-21T10:11:44.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28878" PostTypeId="1" CreationDate="2018-02-21T13:30:14.183" Score="1" ViewCount="82" Body="&lt;p&gt;I want to minimize some multivariable function $\Delta(\alpha, \beta)$. I &lt;strong&gt;know&lt;/strong&gt; that this function has a zero point, $\Delta(5, 5) = 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting from some $(\alpha, \beta)$ close to $(5,5)$ (e.g. (4.8, 5.2)), I want to use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient descent&lt;/a&gt; method to recover the 'correct' values $(5,5)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can plot the surface $\alpha, \beta, \Delta$ and it clearly has a minimum point, but my algorithm fails to converge&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DWt5y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DWt5y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;My general approach is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start with some initial $\alpha, \beta$. Compute $\Delta$ &lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate $\frac{\partial \Delta}{\partial \alpha}$ and $\frac{\partial \Delta}{\partial \beta}$ by evaluating e.g. $$ \frac{\partial \Delta}{\partial \alpha} = \frac{\Delta(\alpha+\epsilon, \beta)-\Delta(\alpha-\epsilon, \beta)}{2 \epsilon} $$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Update $\alpha, \beta$ as $$\alpha = \alpha - \gamma \frac{\partial \Delta}{\partial \alpha} $$ and the equivalent for $\beta$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Iterate&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there any reason why this general approach should not work?&lt;/p&gt;&#xA;" OwnerUserId="26949" LastActivityDate="2018-02-21T19:40:00.567" Title="Adaptive gradient descent" Tags="|optimization|algorithms|conjugate-gradient|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28879" PostTypeId="2" ParentId="28866" CreationDate="2018-02-21T14:03:19.270" Score="1" Body="&lt;p&gt;The symmetry condition enforces a Neumann Boundary condition on the velocity field and reads $$n\cdot\nabla u=0$$. These are actually two equations.&#xA;In any coordinate system this is done by the contraction of the covariant derivative of the velocity field with the normal vector of the boundary: $$n^i \nabla_i u^k=n^i \left(\frac{\partial u^k}{\partial \xi^i}+\Gamma^k_{il} u^l\right)=0$$.  In cartesian coordinates where the coordinate lines align with the boundary, they are exactly the stated equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the velocity field $u^\alpha=g^{\alpha\beta}\frac{\partial\Phi}{\partial \xi^\beta}$ you have $$n^\gamma \nabla_\gamma g^{\alpha\beta}\frac{\partial\Phi}{\partial \xi^\beta}=n^\gamma g^{\alpha\beta}\left(\frac{\partial^2 \Phi}{\partial \xi^\beta\partial\xi^\gamma}-\Gamma^l_{\gamma \beta} \frac{\partial\Phi}{\partial \xi^l}\right)=0$$. These are two equations one for each part of the curved coordinate system with respect to the normal condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, since $\Phi=U_\infty x_1(\xi_1,\xi_2)+\phi$ you get for the derivatives of $\Phi$:$$\frac{\partial\Phi}{\partial\xi^\alpha}=U_\infty \frac{\partial x_1}{\partial \xi^\alpha}+\frac{\partial \phi}{\partial \xi^\alpha}$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So finally for $\phi$ this reads:&#xA;$$n^\gamma g^{\alpha\beta}\left(\frac{\partial }{\partial \xi^\gamma}\left(U_\infty \frac{\partial x_1}{\partial \xi^\beta}+\frac{\partial \phi}{\partial \xi^\beta}\right)-\Gamma^l_{\gamma \beta} \left(U_\infty \frac{\partial x_1}{\partial \xi^l}+\frac{\partial \phi}{\partial \xi^l}\right)\right)=0$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check this with cartesian coordinates, $x_1$ is independent of $x_2$ so these derivatives vanish. Christoffel symbols are all 0. Metric is diagonal 1, $n^1=0$, $n^2=1$:&#xA;$$n^\gamma g^{\alpha\beta}\left(\frac{\partial }{\partial \xi^\gamma}\left(\frac{\partial \phi}{\partial \xi^\beta}\right)\right)=0$$ so for each component:$$&#xA;n^2 g^{1\beta}\left(\frac{\partial }{\partial x^2}\left(\frac{\partial \phi}{\partial x^\beta}\right)\right)=\left(\frac{\partial }{\partial x^2}\left(\frac{\partial \phi}{\partial x^1}\right)\right)=0\\&#xA;n^2 g^{2\beta}\left(\frac{\partial }{\partial x^2}\left(\frac{\partial \phi}{\partial x^\beta}\right)\right)=\left(\frac{\partial }{\partial x^2}\left(\frac{\partial \phi}{\partial x^2}\right)\right)=0$$.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2018-02-21T16:54:11.307" LastActivityDate="2018-02-21T16:54:11.307" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28880" PostTypeId="2" ParentId="28878" CreationDate="2018-02-21T19:40:00.567" Score="2" Body="&lt;p&gt;There are a few issues that can cause the problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;first, you use gradient computation using finite difference approximations. Is this necessary? Can you compute the partial derivatives of $\Delta$ analytically?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;secondly, the finite difference approximation is only valid for small $\varepsilon$. However, using too small value can cause instabilities if the function $\Delta$ is not very smooth (yours seems smooth enough). When functions are well behaved I use something like $\varepsilon = 10^{-6}$ to test against the analytic gradient. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;let's say that you manage to compute the gradient correctly. Then the choice of the step $\gamma$ is also important. There are different ways of choosing the descent step size, but if you want to keep it simple, then do something like the following: &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; Choose a starting value for $\gamma$ which is not very large, like $\gamma=0.001$ or $\gamma=0.01$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; At each iteration, if you manage to decrease the value of the function, increase $\gamma$ using a rule like $\gamma \gets \min (\gamma_{max},1.1\gamma)$ (where $\gamma_{max}$ is an upper limit for the step size, like $\gamma_{max} = 1$ or smaller). If the value of the function does not decrease, then change $\gamma$ by doing something like $\gamma \gets 0.9\gamma$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; When $\gamma$ is too small or the decrease in the value of the function is too small, you stop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a convex and smooth function, these three should be enough to get you close to the minimum. For more complex examples, using an existing optimized approach like quasi-newton or lbfgs can give very good results. &lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2018-02-21T19:40:00.567" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28881" PostTypeId="1" CreationDate="2018-02-22T01:21:06.873" Score="3" ViewCount="89" Body="&lt;p&gt;In Knuth, the following method for computing an average is presented:&#xA;\begin{align*}&#xA;M_{n} = M_{n-1} + (x_{n} - M_{n-1})/n&#xA;\end{align*}&#xA;(See &lt;a href=&quot;https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, if you don't have TAOCP.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the samples all lie a bounded distance away from the average, then as $n \to \infty$, $(M_{n-1}-x_n)/n$ becomes very small, and eventually $M_{n-1} \oplus (x_{n}\ominus M_{n-1}) \oslash n$ becomes bitwise equal to $M_{n-1}$. The average computed by this formula then stops evolving in response to new data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the known techniques to avoid this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Update&lt;/em&gt;: I believe that this scheme has error $|\hat{M}_n -M_{n}| \le n\mu_{M} \underline{M}_{n}$, where&#xA;\begin{align*}&#xA;\underline{M}_{n} := \frac{1}{n} \sum_{i=0}^{n-1} |x_{i}|,&#xA;\end{align*}&#xA;and $\mu_{M}$ is the unit roundoff, but I couldn't prove it. This won't directly answer the question, but might put me on the track to get improvements. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2018-02-22T23:23:15.217" LastActivityDate="2018-02-22T23:23:15.217" Title="Stabilizing online average calculation" Tags="|numerical-analysis|error-estimation|floating-point|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28884" PostTypeId="2" ParentId="28869" CreationDate="2018-02-22T08:08:07.197" Score="0" Body="&lt;p&gt;The most general case that can be set is the following:&#xA;$$f(\dot{x},x,t)=0, \qquad x(0)=x^0\tag{*}$$&#xA;If you are interested in the obtainment of the variable $\dot{x}$ you can differentiate the previous equation wrt. $t$.&#xA;This gives you:&#xA;$$f_\dot{x}\ddot{x}+f_x\dot{x}+f_t=0\tag{**}$$&#xA;Where the subscripts indicate partial derivatives. It is clear that the equation $(**)$ must come with an initial condition for $\dot{x}(0)=\dot{x}^0$. This is done by means of the equation $(*)$. If it is nonlinear in the variable of interest... one may use iterative methods to solve &#xA;$$f(\dot{x}^0,x^0,t^0)=0$$&#xA;for $\dot{x}^0$.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-02-22T08:08:07.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28885" PostTypeId="1" CreationDate="2018-02-22T10:11:56.960" Score="5" ViewCount="901" Body="&lt;p&gt;I believe this would be an interesting problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a blackbox function which can take 2-60 input variables &lt;strong&gt;$(X_1,X_2,...X_n)$&lt;/strong&gt; which are to be optimized. I'm calling this objective function as a blackbox function because it's  parameters consists of the input variables &lt;strong&gt;$(X_1,X_2,...X_n)$&lt;/strong&gt; and variables from a simulation output &lt;strong&gt;$(Y_1,Y_2,...Y_n)$&lt;/strong&gt;.These simulation output variables &lt;strong&gt;$(Y_1,Y_2,...Y_n)$&lt;/strong&gt; takes the input variables &lt;strong&gt;$(X_1,X_2,...X_n)$&lt;/strong&gt; as inputs for their simulation. Each variable in &lt;strong&gt;$(Y_1,Y_2,...Y_n)$&lt;/strong&gt; can be represented as a function of all &lt;strong&gt;$X_i$&lt;/strong&gt;'s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. &lt;strong&gt;$Y_i = N(X_1,X_2,...X_n)$&lt;/strong&gt;. &#xA;Where &lt;strong&gt;N&lt;/strong&gt; represents a network being simulated.&lt;br&gt;&#xA;      Both &lt;strong&gt;$Xi$&lt;/strong&gt;'s and &lt;strong&gt;$Yi$&lt;/strong&gt;'s have variable bounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My optimization problem can be formulated as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Objective:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimize $\sum_{i=0}^n (A_iX_i+B_iX_iY_i)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Constraints:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;$\sum_{i=0}^n X_i =Const$&lt;/strong&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;$X_{imin}&amp;lt;=X_i&amp;lt;=X_{imax}$&lt;/strong&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;$0&amp;lt;=Y_i&amp;lt;=Y_{imax}$&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;$0&amp;lt;=X_iY_i&amp;lt;=(X_iY_i)_{max}$&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;$A_i$&lt;/strong&gt;'s and &lt;strong&gt;$B_i$&lt;/strong&gt;'s are constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Further information:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Each simulation might takes 3-20 sec depending of the complexity of the network and number of input variables &lt;strong&gt;$(X_i)$&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The objective function is nonlinear for sure but other&#xA;characteristics are unknown (Uni-modal or multi-modal, smooth or&#xA;non-smooth, convex or non convex, noisy or non-noisy)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;One good news is that I have an initial feasible point &lt;strong&gt;$X0=(X_1,X_2...X_n)$&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;We can also treat the &lt;strong&gt;$X_i$&lt;/strong&gt;'s as integers. This also means that the constant on the equality constraint &lt;strong&gt;$(X_1+X_2+...X_n=Const)$&lt;/strong&gt; can also be treated as an integer by rounding the decimal to the nearest integer.(I don't know if it helps). Whereas &lt;strong&gt;$A_i$&lt;/strong&gt;'s and &lt;strong&gt;$B_i$&lt;/strong&gt;'s are positive real numbers.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My need:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.Choice of optimization algorithm? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.A suitable open source solver in python that has strong computational power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So far what I've come across:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;With my search for a solution to my needs, I have most often come across SURROGATE MODELLING, BAYESIAN APPROACH; Derivative FREE METHODS, EVOLUTION ALGORITHMS&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Solvers for Surrogate Modelling &lt;a href=&quot;http://pysot.readthedocs.io/en/latest/index.html&quot; rel=&quot;noreferrer&quot;&gt;PYSOT&lt;/a&gt;; &lt;a href=&quot;https://projects.coin-or.org/RBFOpt&quot; rel=&quot;noreferrer&quot;&gt;RBFOpt&lt;/a&gt; from COIN-OR ; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Solvers for Bayesian approach &lt;a href=&quot;https://github.com/rmcantin/bayesopt&quot; rel=&quot;noreferrer&quot;&gt;BayesOpt&lt;/a&gt;,&lt;a href=&quot;http://hyperopt.github.io/hyperopt-sklearn/#documentation&quot; rel=&quot;noreferrer&quot;&gt;hyperopt&lt;/a&gt;;&lt;a href=&quot;http://nbviewer.jupyter.org/github/SheffieldML/GPyOpt/blob/devel/manual/index.ipynb&quot; rel=&quot;noreferrer&quot;&gt;GPyOpt&lt;/a&gt;; &lt;a href=&quot;https://github.com/fmfn/BayesianOptimization&quot; rel=&quot;noreferrer&quot;&gt;fmfn&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Solvers for Derivative free blackbox optimization methods &lt;a href=&quot;https://github.com/pablormier/yabox&quot; rel=&quot;noreferrer&quot;&gt;yabox&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Most of the above methods don't accept equality constraints which is my main concern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is basically to satisfy my needs.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a suitable algorithm to solve my problem? If yes, suggest&#xA;some free python solvers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Also welcome:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any new ideas or suggestions to solve such a problem are also welcome. Various suggestions on how I could tailor my problem to place it in a suitable class of problems and then applying a suitable solver are also welcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really appreciate your time. Thanks :-)&lt;/p&gt;&#xA;" OwnerUserId="25459" LastEditorUserId="25459" LastEditDate="2018-02-22T15:23:13.350" LastActivityDate="2019-05-22T18:44:05.250" Title="Optimization of a blackbox function with an equality constraint?" Tags="|optimization|python|constrained-optimization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28886" PostTypeId="1" CreationDate="2018-02-22T11:41:06.000" Score="2" ViewCount="3922" Body="&lt;p&gt;I'd like to solve numerically a system of quadratic equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{11}x_1+A_{12}x_2+A_{13}x_3+B_{12}x_1x_2+B_{13}x_1x_3=C_1$&#xA;$A_{21}x_1+A_{22}x_2+A_{23}x_3+B_{21}x_2x_1+B_{23}x_2x_3=C_2$ &#xA;$A_{31}x_1+A_{32}x_2+A_{33}x_3+B_{31}x_3x_1+B_{32}x_3x_2=C_3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_{ij}$, $B_{ij}$, $C_i$ are real numbers and $x_i$ the variables (here only three but I have in mind to solve systems up to 50 variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried Newton-Krylov method (implemented in scipy) but it does not seem to be stable. I'm now trying to use routines implemented in sympy but I face a problem illustrated by this example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sympy import *&#xA;from sympy.abc import x,y,z&#xA;Equations = [x**2 + y + z - 1,&#xA;           x + y**2 + z - 1,&#xA;           x + y + z**2 - 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we use &lt;code&gt;solve_poly system&lt;/code&gt; we obtain&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solve_poly_system(Equations, x, y, z)&#xA;[(0, 0, 1), (0, 1, 0), (1, 0, 0), (-1 + sqrt(2), -1 + sqrt(2), -1 + sqrt(2)), (-sqrt(2) - 1, -sqrt(2) - 1, -sqrt(2) - 1)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and, if we use &lt;code&gt;solve_triangulated&lt;/code&gt; we obtain&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;solve_triangulated(Equations, x, y, z)&#xA;[(0, 0, 1), (0, 1, 0), (1, 0, 0)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Some solutions are not found by &lt;code&gt;solve_triangulated&lt;/code&gt;.&#xA;What is the reason? What kind of solutions can we find with  &lt;code&gt;solve_triangulated&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="26962" LastEditorUserId="9667" LastEditDate="2018-09-10T15:41:43.460" LastActivityDate="2018-09-10T15:41:43.460" Title="Solving a system of quadratic equations in Python" Tags="|python|nonlinear-equations|scipy|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="28887" PostTypeId="1" CreationDate="2018-02-22T15:02:28.593" Score="2" ViewCount="99" Body="&lt;p&gt;Given a set of points in a plane, and series of measurements of the distances between those points, how would I go about generating a best-fit model of the position of the points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given 3 points (A, B, C) that define a 30-40-50 right triangle, the data set might look something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(0, 29.1, 39.7) - at point A, measuring to B, C &lt;br /&gt;&#xA;(NULL, 0, 49.4) - at point B, missing measurement to A &lt;br /&gt;&#xA;(40.4, 50.5, 0) - at point C, measuring to A, B &lt;br /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to do is fit this data to a model that will make it possible to determine the distance between any two arbitrary data points, as well as determine the error (given a known error in each measurement). I'm not even sure what type of problem this is, so any pointers to methods or algorithms would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="26965" LastActivityDate="2018-02-24T17:18:31.030" Title="Fitting 2D mapping data from multiple measurements" Tags="|regression|distance-measure|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28889" PostTypeId="1" CreationDate="2018-02-22T16:36:22.310" Score="3" ViewCount="475" Body="&lt;p&gt;I want to simulate the shape of the free surface in a small fuel tank in microgravity, which is very slowly being emptied. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The tank is not symmetric, the geometry is given by CAD (e.g. step file). &lt;/li&gt;&#xA;&lt;li&gt;Modelling of surface tension and contact angle is a must. The contact angle will also vary between different wall regions of the tank.&lt;/li&gt;&#xA;&lt;li&gt;I don't necessarily need dynamic behaviour. I don't (right now) need a flow field solution&lt;/li&gt;&#xA;&lt;li&gt;The possibility to later add a thermal simulation would be a plus.&lt;/li&gt;&#xA;&lt;li&gt;An interface possibility with Python would be ideal, but driven by CLI or interface file is sufficient (for use in later optimization studies)&lt;/li&gt;&#xA;&lt;li&gt;An open-source (but maintained) or free code would be ideal&lt;/li&gt;&#xA;&lt;li&gt;I am not a numerical algorithms person, so it would be great if I don't have to implement a solver myself :D&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The relevant equations are the &lt;a href=&quot;https://en.wikipedia.org/wiki/Young%E2%80%93Laplace_equation#Capillary_action_in_general&quot; rel=&quot;nofollow noreferrer&quot;&gt;Young-Laplace equation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${\displaystyle \Delta p=\rho gh-\gamma \left({\frac {1}{R_{1}}}+{\frac {1}{R_{2}}}\right)}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Delta p$ is the pressure jump across the interface, $\rho g h$ is the hydrostatic pressure (which can be neglected in zero gravity), $\gamma$ is the surface tension, and $R_1$, $R_2$ are the principal radii of curvature of the interface at a given point; and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Wetting#Simplification_to_planar_geometry.2C_Young.27s_relation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Young equation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;${\displaystyle \gamma _{SG}\ =\gamma _{SL}+\gamma _{LG}\cos {\theta }}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which relates, at the contact line, the surface tensions between the three phases: solid, liquid and gas with the equilibrium contact angle $\theta$ between the solid surface and the liquid-gas interface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have investigated so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://facstaff.susqu.edu/brakke/evolver/evolver.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Surface Evolver&lt;/a&gt;/SE-Fit: This looked at first glance ideal, but later on we realized that there is a reason why most of the examples featured primitive bodies/domains - working with an imported geometry is, according to a colleague, extremely cumbersome. Also, he encountered quite some problems around convex edges, instability, crashes, and generally it seems to be a pain to work with for non-primitive geometries.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://basilisk.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Basilisk&lt;/a&gt;: Seems very nice, has a self-adaptive mesher, solves the flow field (which I don't necessary need), but while it accounts for surface tension, nicely preserves phase volume, and apparently does great interface resolution apparently does not have an implementation of the contact angle boundary condition! Also, it's unclear if it can work nicely with impoerte geometry, most of the examples are boxes or otherwise primitive geometries.&lt;/li&gt;&#xA;&lt;li&gt;It's predecessor, &lt;a href=&quot;http://gfs.sourceforge.net/wiki/index.php/Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gerris&lt;/a&gt;, could be a candidate, as it can at least prescribe a contact angle in an axisymmetric case. Unfortunately, it does not seem to be developed any more.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://fenicsproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fenics&lt;/a&gt; looks very nice, and in Python too, but the documentation and variational formulation approach seem daunting. Also, I could not find a worked example for a surface tension interface shape problem, like sessile drop etc, and nobody seems to work in that area&lt;/li&gt;&#xA;&lt;li&gt;I have also found this obscure thing called &lt;a href=&quot;https://sites.google.com/site/hydroenglish/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;HyDro&lt;/a&gt;, which seems really simple and nice, but is only for flat substrates.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Am I missing any other programs or good contenders to do what I need?&lt;/p&gt;&#xA;" OwnerUserId="26969" LastEditorUserId="26969" LastEditDate="2018-03-19T07:52:31.753" LastActivityDate="2022-01-23T03:02:35.323" Title="Simulation of surface tension-dominated interfaces" Tags="|fluid-dynamics|simulation|microfluidics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28890" PostTypeId="1" CreationDate="2018-02-22T19:04:50.847" Score="3" ViewCount="110" Body="&lt;p&gt;I have tried to set up a model for the diffusion of a gas into a liquid. The two media are next to each other and the geometry is spherical because the system should simulate the diffusion out of a gas bubble. &#xA;I have made a diffusion model for the gas phase (the left compartment) and a corresponding diffusion model for the liquid phase (the right compartment). &#xA;The two diffusion models are as follows: &#xA;\begin{align}&#xA;  \frac{\partial c_\mathrm{left}}{\partial t} &#xA;  &amp;amp;= D_\mathrm{left} \cdot \frac{1}{r^2} \frac{\partial}{\partial r} &#xA;     \left(r^2 \frac{\partial c_{left}}{\partial r} \right)\\&#xA;  \frac{\partial c_\mathrm{right}}{\partial t} &#xA;  &amp;amp;= D_\mathrm{right} \cdot \frac{1}{r^2} \frac{\partial}{\partial r}&#xA;  \left(r^2 \frac{\partial c_{right}}{\partial r} \right)\\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I have discretized the volume. So the PDEs are turned into ODEs of the form: &#xA;\begin{align}&#xA;  \frac{\mathrm{d}c_{i,\mathrm{left}}}{\mathrm{d}t} &#xA;  &amp;amp;= D \left(\frac{c_{i+1,\mathrm{left}} &#xA;     - 2 \cdot c_i + c_{i-1,\mathrm{left}}}{\Delta r^2} &#xA;     + \frac{2}{r_{i,\mathrm{left}}} &#xA;     \cdot \frac{c_{i+1,\mathrm{left}} &#xA;     - c_{i-1,left}}{2 \cdot \Delta r} \right)\\&#xA;  \frac{\mathrm{d}c_{i,\mathrm{right}}}{\mathrm{d}t}&#xA;  &amp;amp;= D \left(\frac{c_{i+1,\mathrm{right}}&#xA;     - 2 \cdot c_i + c_{i-1,\mathrm{right}}}{\Delta r^2} &#xA;     + \frac{2}{r_{i,\mathrm{right}}} &#xA;     \cdot \frac{c_{i+1,\mathrm{right}} &#xA;     - c_{i-1,\mathrm{right}}}{2 \cdot \Delta r} \right)&#xA;\end{align} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When discretizing I got the following expressions: &#xA;\begin{align}&#xA;  \frac{\mathrm{d}^2c_i}{\mathrm{d}r^2} &#xA;  &amp;amp;= \frac{c_{i+1} - 2 \cdot c_i + c_{i-1}}{\Delta r^2}\\&#xA;  \frac{\mathrm{d}c_i}{\mathrm{d}r} &#xA;  &amp;amp;= \frac{c_{i+1} - c_{i-1}}{2 \cdot \Delta r}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how does one incorporate the following two boundary conditions at the gas-liquid-interface, where $H$ is Henry's constant? &#xA;\begin{align} &#xA;  c_\mathrm{left,interface} &amp;amp;= H \cdot c_\mathrm{right,interface}\tag1\\ &#xA;  J_\mathrm{left,interface} &amp;amp;= J_\mathrm{right,interface}\tag2&#xA;\end{align} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meaning that there is equal flux across the interface between the two compartments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concentration in the last compartment of the gas phase (before the interface) is approximated by linear extrapolation. &#xA;This means that the concentration just before the interface is given by: &#xA;$$c_{3} = \frac{c2-c1}{r2-r1} \cdot r_3 + c_1 - r_1 \frac{c_2-c_1}{r_2 - r_1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example it is assumed that only three compartments are present on each side of the interface. &lt;/p&gt;&#xA;" OwnerUserId="26970" LastEditorUserId="26970" LastEditDate="2018-02-24T09:41:30.797" LastActivityDate="2018-02-25T10:41:28.450" Title="How to account for the interface between two different phases in a discretized diffusion model?" Tags="|pde|finite-difference|ode|discretization|diffusion|" AnswerCount="0" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28891" PostTypeId="2" ParentId="21985" CreationDate="2018-02-22T21:44:52.337" Score="1" Body="&lt;p&gt;I'm not really sure about the sloping saturation at the ends. I think a bit of slope at either end is normal. In theory, the saturation is flat, but in practice, I would say the saturation is usually a little angled. A simple trick you could use is to fit a logistic model to your data, and use the saturated value of the model as the saturated magnetic field strength.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The remnant magnetization is the magnetization that remains when the applied magnetic field goes to zero. You could, in fact, use the logistic model to find the remnant magnetization by just subtracting the value of your logistic function at the mean $x$ value from the value of your data in the neighborhood of that value. In MATLAB, you could do this by finding the indices of your $x$ vector (in your case Field) such that $x$ is within some tolerance; dividing those into two groups that contain $y$ (Kerr signal) above and below its mean; averaging the $y$'s that are in each of those subgroups; and subtracting the value of the logistic function from each of those means (both values should be about the same but of opposite sign).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coercive field is the field that must be applied in order to get the magnetization back to zero (or twice that, depending on how you define it). You can simply look at the $x$ value at the mean value of your fitted $y$. To do this in MATLAB, you have to do something very similar to the above but replace the $y$'s with $x$'s and vice versa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the saturation field, the best way to do it is probably to differentiate your logistic model and find where the derivative with respect to $x$ equals zero. The derivative of the logistic function can be found &lt;a href=&quot;https://en.wikipedia.org/wiki/Logistic_function#Derivative&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;; just substitute $\mathbf{x}^T\boldsymbol{\beta}$ for $\mathbf{x}$ (using a vector for $\mathbf{x} = [1 \:x]^T$ since the data may have a nonzero intercept) and don't forget to use the chain rule. Once you find $x:f'(\mathbf{x}^T\boldsymbol{\beta}) = 0$, just evaluate that $x$ in your model or take the mean of the $y$'s around that value of $x$.&lt;/p&gt;&#xA;" OwnerUserId="15273" LastEditorUserId="15273" LastEditDate="2018-02-23T19:08:38.660" LastActivityDate="2018-02-23T19:08:38.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28892" PostTypeId="2" ParentId="28887" CreationDate="2018-02-22T22:03:53.653" Score="2" Body="&lt;p&gt;If you know that you have $n$ points $X_1,...,X_n$ and the desired pairwise distances $d_{ij}$ between $X_i$ and $X_j$ you could try and optimize the functional&#xA;$$ \sum_{i&amp;lt;j} (d_{ij}-|X_i-X_j|)^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion this looks good as an optimization problem. You can compute the gradient explicitly (even the Hessian, if you are motivated) and then use some Newton or quasi-Newton solver to find an optimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, you could use a $p$-norm in stead of the $2$-norm above. For $p$ large it will try to minimize the largest error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It remains to be seen if it works well. This may depend on the choice of the initial condition. Multiple starts from random initializations may help avoid local minima...&lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2018-02-22T22:03:53.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28893" PostTypeId="1" CreationDate="2018-02-22T22:57:10.563" Score="1" ViewCount="70" Body="&lt;p&gt;This is a follow-up of my previous scicomp question (&lt;a href=&quot;https://scicomp.stackexchange.com/posts/28863/edit&quot;&gt;https://scicomp.stackexchange.com/posts/28863/edit&lt;/a&gt;). I figured I'd start a new thread on this as the question is a bit different from my previous question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving the linear elasticity equations, using Hooke's Law and the strain-displacement relationship. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My discretization is as follows using a finite volume formulation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla \cdot \boldsymbol{\sigma} = \nabla \cdot (\mu \boldsymbol{\nabla u} + \mu(\boldsymbol{\nabla u})^T + \lambda\boldsymbol{I}tr(\boldsymbol{\nabla u})) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In integral form, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\iint_A [\mu \boldsymbol{\nabla u} + \mu(\boldsymbol{\nabla u})^T + \lambda\boldsymbol{I}tr(\boldsymbol{\nabla u})] \cdot dA = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I evaluate this surface integral using:&#xA;$$\iint_A [\mu \boldsymbol{\nabla u} + \mu(\boldsymbol{\nabla u})^T + \lambda\boldsymbol{I}tr(\boldsymbol{\nabla u})] \cdot dA = \sum\limits_{k=1}^N |A_k|n_k\cdot[\mu (\boldsymbol{\nabla u})_k + \mu(\boldsymbol{\nabla u})_k^T + \lambda\boldsymbol{I}tr(\boldsymbol{\nabla u})_k] $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I set the above equation to $f(\boldsymbol{u})$ for simplicity. I can expand this term using a Taylor series expansion of the form:&#xA;$$f(\boldsymbol{u}) = f(0) + \frac{df}{d\boldsymbol{u}}\boldsymbol{u} = f(0) + \boldsymbol{Ju}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I compute the jacobian, $\boldsymbol{J}$, using a forward differencing scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J = \frac{d\boldsymbol{f}}{d\boldsymbol{u}} = \frac{f(\boldsymbol{u}+h)-f(\boldsymbol{u})}{h}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this gives us a linear system of the form:&#xA;$$\boldsymbol{Ju} = \boldsymbol{b} = -f(0)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this is a finite volume formulation, my mesh consists of cells. I compute the cell gradients using the discrete green gauss method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\boldsymbol{\nabla} u = -\frac{1}{V}\sum\limits_{j=1}^N \hat{u}_jA_j\boldsymbol{n_j}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\hat{u}=\frac{u_L+u_R}{2}$, i.e., the face displacement is obtained by averaging the neighboring cells' displacement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I compute face gradients using a simple average and applying a correction term:&#xA;$$\nabla u_{face} = \frac{1}{2}(\nabla u_L + \nabla  u_R) - \underbrace{\hat{\boldsymbol{d_{lr}}} [\frac{1}{2}(\nabla u_L + \nabla u_R)\cdot\hat{\boldsymbol{d_{lr}}} ] + (u_L - u_R)\frac{\hat{\boldsymbol{d_{lr}}}}{|\boldsymbol{d_{lr}}|}}_{\text{Correction term}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So this is the gist of my formulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to model a cantilever problem, and it ended in a disaster (see previous question). I have tested out this formulation on the classical 1-D compression problem and was able to match analytical results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone see anything wrong with this formulation? &lt;/p&gt;&#xA;" OwnerUserId="23915" LastActivityDate="2018-02-22T22:57:10.563" Title="Discretization used for steady linear elasticity - followup to previous question" Tags="|finite-volume|solid-mechanics|" AnswerCount="0" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="28894" PostTypeId="2" ParentId="28876" CreationDate="2018-02-22T23:45:27.210" Score="5" Body="&lt;p&gt;The question you ask is simpler than you may think. For any mesh size $h$, you can compute $\lambda(h)$. If this dependence is continuous, then you can do a Taylor expansion of $\lambda(h)$ around $h=0$ to obtain exactly the form you want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only question in your case is whether $\lambda(h)$ is a continuous function. This is made slightly more complicated than one may think because in fact not all values of $h$ are of course possible, but at least for sufficiently small $h$, you can evaluate $\lambda$ for essentially any $h$. In that case, let us assume that $h$ can be chosen arbitrarily, and then the question can be reduced to the following: If $\lambda$ is a function of the solution $u(h)$, it is continuous if $u(h)$ is continuous as a function of $h$. For most problems, $u(h)$ is indeed continuous, and in that case $\lambda(h)$ is a continuous function if $\lambda(u)$ is continuous. Whether that is true is typically a functional analysis question: if $\lambda(u)$ is, for example, the value of $u$ at a particular point, then for $H^1$ functions $u$, the function $\lambda(u)$ is not continuous. But if $\lambda(u)$ is the &lt;em&gt;average&lt;/em&gt; value of $u$, then the dependence is continuous.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-02-22T23:45:27.210" CommentCount="12" ContentLicense="CC BY-SA 3.0" />
  <row Id="28895" PostTypeId="2" ParentId="28886" CreationDate="2018-02-23T12:31:57.980" Score="3" Body="&lt;p&gt;Why don't you use regular Newton? Your system is simple enough that you can find its closed-form Jacobian and write your own Newton solver. If you just need one solution which is close to a given starting point (like you wrote on &lt;a href=&quot;https://mathoverflow.net/questions/293605/solving-a-system-of-quadratic-equations-with-sympy?noredirect=1#comment728798_293605&quot;&gt;MO&lt;/a&gt;), then it rates to work pretty well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(As I wrote on MO, I guess that there can be up to $2^{\text{number of variables}}$ real solutions, so finding all of them is going to be problematic for $n\approx 50$.)&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2018-02-23T12:31:57.980" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28896" PostTypeId="1" CreationDate="2018-02-23T12:54:22.560" Score="4" ViewCount="212" Body="&lt;p&gt;I have some questions about the implementation of non-reflecting OUTFLOW boundary condition for Navier Stokes equations. &#xA;Following &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Poinsot, Lele &quot;Boundary Conditions for Direct Simulations of Compressible Viscous Flows&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pirozzoli, Colonius &quot;Generalized characteristic relaxation boundary conditions for unsteady compressible flow simulations&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;both authors suggest an equation like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{d\mathbf{u}_b}{dt} + \mathbf{d(u_b)} + \mathcal{\mathbf{T(u_b)}} = \mathbf{S(u_b)}\quad\quad (1)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{u}_b = (\rho, \rho u, \rho v, \rho w, \rho e)$ are the conservative variable at the bound b; $\mathbf{d(u_b)}$ is a certain characteristic treatment of x-flux; $\mathcal{\mathbf{T(u_b)}}$ are the transverse therms; $\mathbf{S(u_b)}$ are the source therm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider a finite difference method and suppose that the bound is located at $n+1/2$ (as usual in this problems); considering furthermore $n+1, n+2,\dots, n+GN$ ghost points. &#xA;Equation (1) can be applied just in $n+1/2$, so my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can I discretise the derivatives in $n+1/2$ with some FD scheme in order to solve (1) at this location. &lt;/li&gt;&#xA;&lt;li&gt;How can I treat the ghost nodes?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="26978" LastEditorUserId="9667" LastEditDate="2018-02-23T14:56:45.090" LastActivityDate="2018-02-23T14:56:45.090" Title="Non-reflecting boundary conditions for compressible Navier-Stokes equations" Tags="|fluid-dynamics|boundary-conditions|computational-physics|navier-stokes|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="28897" PostTypeId="2" ParentId="28870" CreationDate="2018-02-23T23:32:51.053" Score="1" Body="&lt;p&gt;Numerical computation of Generalized Complex Schur decomposition can be performed by calling &lt;em&gt;zgges&lt;/em&gt;() LAPACK function.&#xA;For example, see &lt;a href=&quot;http://www.netlib.org/lapack/lapack-3.1.1/html/zgges.f.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NETLIB zgees&lt;/a&gt; documentation, or a documentation for any other BLAS/LAPACK library implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eigen is technically nothing else, but a very convenient templated library of wrappers and algorithms, also including some custom implementation. However, for maximum Eigen performance, it is recommended to link it to a LAPACK\BLAS library, for example, Intel MKL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, regarding Schur decomposition in Eigen. I have not used the functionality from &lt;a href=&quot;https://eigen.tuxfamily.org/dox/classEigen_1_1GeneralizedEigenSolver.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneralizedEigenSolver&lt;/a&gt;, but according to the documentation, only real matrices are supported. &#xA;The following code compiles correctly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigen::GeneralizedEigenSolver&amp;lt;Eigen::MatrixXf&amp;gt; solver;&#xA;Eigen::MatrixXf A(4,4),B(4,4);&#xA;solver.compute(A,B);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the complex version results in a compilation error that pretty much describes itself: no support for complex matrices yet:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Eigen::GeneralizedEigenSolver&amp;lt;Eigen::MatrixXcd&amp;gt; solver;&#xA;Eigen::MatrixXcd A(4,4),B(4,4);&#xA;solver.compute(A,B);&#xA;&#xA;Eigen/src/Eigenvalues/GeneralizedEigenSolver.h:367:20: error: no viable conversion from 'CoeffReturnType' (aka 'const std::__1::complex&amp;lt;double&amp;gt;') to 'RealScalar' (aka 'double')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The example is tested on Eigen 3.3.4.&#xA;So, I would suggest you to use LAPACK function directly (you have to have some BLAS\LAPACK library connected to your code) without using Eigen wrappers for now.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-02-23T23:32:51.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28898" PostTypeId="2" ParentId="28752" CreationDate="2018-02-24T11:58:35.617" Score="3" Body="&lt;p&gt;Well, I can answer at least my first question: yes, it is indeed possible to get an account on Fujitsu Europe's KNL cluster. It just takes a veeeery long time.... but finally they managed it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First impressions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Access is granted only for 30 days, but apparently a prolongation is possible. You have to ask their support for an extension.&lt;/li&gt;&#xA;&lt;li&gt;Access is only possible via a (nice) web interface which offers a virtual &quot;remote terminal&quot; and a graphical file manager, among other things. Quite attractive. Direct command-line SSH access is blocked, however.&lt;/li&gt;&#xA;&lt;li&gt;Some of the software installed on the cluster is outdated. For instance &lt;code&gt;cmake&lt;/code&gt;: they offer Version 2.8.12 (!). Also there is no Python3, only Python2 (which IMO should go the way of Python1, but I digress).&lt;/li&gt;&#xA;&lt;li&gt;I had some initial questions about how to use the system, the help desk was very responsive and gave me useful answers.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;TL;DR: worth trying out, but be prepared for a long initial &quot;lag phase&quot;.&lt;/p&gt;&#xA;" OwnerUserId="7024" LastActivityDate="2018-02-24T11:58:35.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28899" PostTypeId="2" ParentId="28824" CreationDate="2018-02-24T13:26:29.487" Score="0" Body="&lt;p&gt;Your numerical results for the transient response are as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Results obtained with the modal superposition method match very well with the linear transient analysis because you use the same stiffness matrix in both the analyses. However, when the deformations are significant i.e, in the finite strain regime, the results obtained with the nonlinear transient analysis would be significantly different from those obtained with the linear methods, be it linear transient analysis or modal superposition. You can not capture this nonlinear behaviour no matter how many modes you use in the modal superposition method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For stiffening materials, which are used in Engineering in general, the stiffness increases as it is loaded; hence, the frequencies will be little higher for nonlinear models. Therefore, when the material is deformed significantly, its natural frequencies and mode shapes would be different. You can verify this behaviour by computing the natural frequencies with the stiffness matrix computed after the end of non-linear static analysis. Even a 10% difference in the natural frequencies would result in a significant differences, as you observed, in the time response. This observation also reflects in your statement &quot;Both methods produce comparable natural frequencies and mode shapes, but the modal superposition fails to capture behaviour which is seen in real life.&quot; &quot;Comparable natural frequencies&quot; is not the same as &quot;exactly the same natural&quot; frequencies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Options for you&lt;/strong&gt;&#xA;(I assume that you are using implicit scheme for non-linear transient response as you mentioned that you update stiffness and mass matrices.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) Instead of an implicit time integration scheme, you can use an explicit scheme, for example, the well-known Central Difference scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) As you are interested in the design optimisation, your best option would be to use &lt;strong&gt;nonlinear normal modes&lt;/strong&gt; (&lt;strong&gt;NNM&lt;/strong&gt;), which is an extension of linear modal analysis to non-linear regime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that in &lt;strong&gt;NNM&lt;/strong&gt;, instead of using the natural frequencies and mode shapes from the linear stiffness matrix, you use those corresponding to the nonlinear stiffness matrix. As non-linear static analysis, and modal analysis thereafter, is much less expensive than the full nonlinear transient analysis, &lt;strong&gt;NNM&lt;/strong&gt; would be the best method for your task. I am not quite familiar with this &lt;strong&gt;NNM&lt;/strong&gt; technique. I suggest you to refer to the relevant literature for comprehensive details on &lt;strong&gt;NNM&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24915" LastActivityDate="2018-02-24T13:26:29.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28902" PostTypeId="2" ParentId="28887" CreationDate="2018-02-24T17:18:31.030" Score="0" Body="&lt;p&gt;This is an application of Euclidean Distance Matrices. Here's a good overview of the problem, including some algorithms for solving it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Abstract: &lt;a href=&quot;https://arxiv.org/abs/1502.07541&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1502.07541&lt;/a&gt;&#xA;Full article: &lt;a href=&quot;https://arxiv.org/pdf/1502.07541&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1502.07541&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26965" LastActivityDate="2018-02-24T17:18:31.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28903" PostTypeId="1" CreationDate="2018-02-24T17:55:32.923" Score="1" ViewCount="87" Body="&lt;p&gt;Projection of a vector $v$ onto the column space of a matrix $A$ is given by $AA^\dagger v$. From the &lt;a href=&quot;https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse#Definition&quot; rel=&quot;nofollow noreferrer&quot;&gt;definition of Moore-Penrose Inverse&lt;/a&gt; we know that $AA^\dagger v = (A^T)^\dagger A^T v $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the code for implementing the projection of a random vector onto the space of a random matrix. This is related to my other question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/28754/backward-stable-algorithm-to-get-orthogonal-projection-onto-the-column-space-of&quot;&gt;Backward stable algorithm to get orthogonal projection onto the column space of a matrix&lt;/a&gt;. I didn't get an answer to that question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know why is there a huge difference between the two methods of calculating the projection.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  % testingprojfrostackexchange&#xA;  clear;    &#xA;  M = 1400;&#xA;  N = 1300;&#xA;  r = 1;&#xA;  A = rand(M,N);&#xA;  u = rand(M,r);&#xA;&#xA;  projLN = pinv(A')*(A'*u);%This is projection through Least Norm&#xA;  projLS = A*(pinv(A)*u);%This is projection through Least Square&#xA;&#xA;  [Q R] = qr(A);&#xA;  Q = Q(:,1:N);&#xA;  z1 = Q*(Q'*u);%This is the actual projection&#xA;&#xA;  display('(projection through QQT) - projLS');&#xA;  norm(z1-projLN)/norm(projLN)&#xA;&#xA;  display('(projection through QQT) - projLN');&#xA;  norm(z1-projLS)/norm(projLS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; stackexchange&#xA;   (projection through QQT) - projLS&#xA;&#xA;   ans =&#xA;&#xA;   2.1569e-13&#xA;&#xA;   (projection through QQT) - projLN&#xA;&#xA;   ans =&#xA;&#xA;   8.3546e-15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The actual projection which is given by $Q(:,1:N)Q(:,1:N)^T$, where $Q$ is from the Housholder decomposition of $A$. We get the projections from $AA^\dagger$ and $(A^T)^\dagger A^T$ and find that $AA^\dagger$ is much better when $A$ is a tall matrix.&lt;/p&gt;&#xA;" OwnerUserId="23066" LastEditorUserId="23066" LastEditDate="2018-02-24T18:11:43.467" LastActivityDate="2018-02-24T18:11:43.467" Title="Stable Method of orthogonal projection onto a subspace with the help of Moore-Penrose inverse," Tags="|numerical-analysis|least-squares|projection|matrix-factorization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28904" PostTypeId="2" ParentId="28864" CreationDate="2018-02-25T08:04:18.407" Score="-1" Body="&lt;p&gt;The method of lines is not applicable to this equation (which is called advection-diffusion equation) in the same sense as it is applicable to the advection equation. There are no paths x(t) along which c is constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would use a finite-difference approach throughout. You have already written down a centered finite difference in [10]; just use a forward-time difference, and then write down finite difference expressions for the spatial derivatives at the two boundaries. I could fill in details, but you seem to be well-familiar with how this works. For a forward time-difference, there will be a stability condition on the time step. As long as D and r are constant, this should automatically conserve mass.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could also use a spectral solver, but implementing the boundary conditions in Fourier space would be some work.&lt;/p&gt;&#xA;" OwnerUserId="18092" LastActivityDate="2018-02-25T08:04:18.407" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28905" PostTypeId="2" ParentId="28869" CreationDate="2018-02-25T09:43:27.387" Score="0" Body="&lt;p&gt;The common sense in physics simulations is to first simplify the equations as much as you can, and only set-up your numerical method after that. Everything that can be done analytically has to be done analytically, lest you multiply numerical artifacts. So, you do not need to solve third-order equation (it can even be counter-productive). Solve the second order equation, and then just calculate the third order derivative of $\theta$ as someone already said in his/her answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the equation (2nd order one) being &quot;unsolvable&quot;: it is solvable and its solutions are Jacobi ellyptic functions. Howevei, you don't need to solve it analytically. Just get a numerical solution and then find it's 3rd order derivative using finite differences.&lt;/p&gt;&#xA;" OwnerUserId="26990" LastActivityDate="2018-02-25T09:43:27.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28906" PostTypeId="2" ParentId="28864" CreationDate="2018-02-25T12:50:22.577" Score="2" Body="&lt;p&gt;What do you mean by &quot;I have realised that several things did not affect the mass conservation&quot;. How did you measure conservation ? Are you asking if your method conserves total mass ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My answer below is based on a conservation law I notice in your model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You wrote an $r$ in your PDE but did not use it later, so I am assuming it is zero. Your model has this quantity conserved&#xA;$$&#xA;m(t) = a \int_0^H c(x,t)dx + c_L(t)&#xA;$$&#xA;wrt time, i.e.,&#xA;$$&#xA;m(t) = \textrm{constant}&#xA;$$&#xA;To approximate the integral, we will use trapezoidal rule&#xA;$$&#xA;m = a\left(c_0/2 + \sum_{i=1}^{N-1} c_i + c_N/2\right) h + c_L&#xA;$$&#xA;Then this changes as&#xA;$$&#xA;dm/dt = \frac{aD}{h} \left(c_1 - c_0 + \sum_{i=1}^{N-1} (c_{i-1} - 2 c_i + c_{i+1}) + c_{N-1} - c_N\right) + a k_L(C_L^* - c_N) - a k_L (C_L^* - c_N)&#xA;$$&#xA;By summing up the terms inside the big bracket, you see it is zero, so that&#xA;$$&#xA;dm/dt = 0&#xA;$$&#xA;Hence the total &quot;mass&quot; $m$ is conserved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have answered a similar question here &lt;a href=&quot;https://scicomp.stackexchange.com/questions/28290/neumann-boundary-conditions-diffusion-equations-methods-of-lines/28293#28293&quot;&gt;Neumann boundary conditions diffusion equations methods of lines&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-02-25T12:50:22.577" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28907" PostTypeId="1" AcceptedAnswerId="28919" CreationDate="2018-02-26T14:35:04.137" Score="3" ViewCount="321" Body="&lt;p&gt;I would like to obtain a feasible solution for an &lt;strong&gt;under-determined&lt;/strong&gt; system of linear equations,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax=b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where,  $A \in \mathbb{R}^{7\times9}, \, x \in \mathbb{R}^{9\times1}\text{and } b\in\mathbb{R}^{7\times1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, there is no unique solution, but I would like to obtain a solution that satisfies some &lt;strong&gt;non-linear inequality&lt;/strong&gt; conditions for a few components in $x$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;f_1(x_1,x_2,x_3) &amp;amp;\le c_1 \\&#xA;f_2(x_7,x_8,x_9) &amp;amp;\le c_2 \\&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be specific, the inequalities are $|x_1 + x_2 + x_3 - k| \le c_1$ and $|x_7 + x_8 + x_9 - k| \le c_2$ , i.e. the only non-linearity here is the &lt;code&gt;abs()&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am happy with any solution $x^*$ that satisfies the underdetermined system exactly whilst also satisfying the two constraints above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently using MATLAB as my programming environment, but pseudo-code/links to useful resources etc shall be more than enough to get started. An example MATLAB code for a dummy problem shall also be very helpful.&lt;/p&gt;&#xA;" OwnerUserId="16900" LastEditorUserId="16900" LastEditDate="2018-02-27T18:57:05.213" LastActivityDate="2018-02-27T22:30:04.470" Title="Obtaining a feasible solution for underdetermined system of linear equations satisfying inequality constraints" Tags="|linear-algebra|optimization|constrained-optimization|nonlinear-programming|least-squares|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="28908" PostTypeId="1" AcceptedAnswerId="28910" CreationDate="2018-02-26T20:07:45.090" Score="0" ViewCount="307" Body="&lt;p&gt;I have written some multithreaded code using PyOpenCL, which works fine under the following conditions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GPU 32-bit floating point values (Intel Iris GPU can only handle single precision values)&lt;/li&gt;&#xA;&lt;li&gt;CPU 32-bit floating point values.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All data buffers are defined, and the program executed and output data enqueued, all inside a function which is called within a loop. When using 64-bit double-precision values, the loop runs twice before throwing the error 'Segmentation Fault: 11'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I don't believe it is due to the string-manipulations involved in preparing the OpenCL kernel; more likely seems to be something to do with allocated memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, the CPU max_work_group is 1024, and GPU max_work_group is 512.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What possible causes could there be for the success using single-precision values, but failure when using double-precision values?&lt;/p&gt;&#xA;" OwnerUserId="25837" LastActivityDate="2018-02-26T20:52:20.150" Title="Why does multithreaded code Segfault in 64-bit but not 32-bit" Tags="|python|parallel-computing|opencl|memory-management|" AnswerCount="1" CommentCount="2" ClosedDate="2018-03-02T15:28:33.630" ContentLicense="CC BY-SA 3.0" />
  <row Id="28910" PostTypeId="2" ParentId="28908" CreationDate="2018-02-26T20:52:20.150" Score="2" Body="&lt;p&gt;It's impossible to tell without knowing the code you are using. But fortunately, segmentation faults are easy to debug: basically, a segfault means that you are accessing memory you should not access, and the operating system stops your program at the point where this is happening. This means, that if you run your program under a debugger, then you will see exactly which line is causing the problem -- you can inspect the line where it happened, the local variables and their values at this point, and the backtrace (i.e., which function called the one where the problem happened, which function called that function, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, run your program in a debugger and see what concretely happened. Everything else is essentially just speculation that may or may not be correct.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-02-26T20:52:20.150" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28914" PostTypeId="2" ParentId="25777" CreationDate="2018-02-27T09:14:31.900" Score="0" Body="&lt;p&gt;some times ago I have tried to find an algorithm on a similar problem: find the optimal partitioning of a time series by performing linear regression on each segments, without prior knowledge of the optimal number of segments and their location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had searched for papers with keywords as 'Multiple Structural Change', or 'Time serie segmentation'...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the main idea is to consider this as a minimal distance problem on a directed network. Each point in time is a node, which is connected to all its successor nodes. We look for the shortest path from the first to the last time. The measure for the &quot;distance&quot; is, I think, the average error to the fit (constant in your case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difficulty is maybe to not over-fit the data: i.e. to end up with as many segments as data points. The cost of creating a new segment had to be taken into account. I am not sure about how to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the time, I didn't look further so I am curious about the right way to do this,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;" OwnerUserId="15117" LastActivityDate="2018-02-27T09:14:31.900" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28915" PostTypeId="1" AcceptedAnswerId="28916" CreationDate="2018-02-27T15:34:13.103" Score="0" ViewCount="69" Body="&lt;p&gt;I'm using LAPACK's zggev_ routine to solve some generalized eigenvalue problem.&#xA;While it produces the correct results, I want the eigenvalues and according eigenvectors sorted by absolute value.&#xA;For this I implemented a naive sorting algorithm which provides me with the permutation vector needed by the LAPACK routine zlapmt_ which should sort the eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When running the code it freezes before even entering the main() function. If the zlapmt_ call is commented out everything works fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone experienced this issue as well? Or do I do something wrong in the code? I attached a MWE below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;complex.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;&#xA;extern double dznrm2_( int *n, double complex *x, int *incx );&#xA;extern void zdrscl_( int *n, double *alpha, double complex *x, int *incx );&#xA;extern void zlapmt_( int *fwd, int *m, int *n, double complex *X, int *ldx, int *k );&#xA;extern void zggev_( char *jobvl, char *jobvr, int *n, double complex *A, int *lda, double complex *B, int *ldb, &#xA;                double complex *alpha, double complex *beta, double complex *vl, int *ldvl, &#xA;                double complex *vr, int *ldvr, double complex *work, int *lwork, double *rwork, int *info );&#xA;&#xA;int main(void) {&#xA;  int m = 3, lwork = 2*m*m, pi[m], info;&#xA;  double rwork[8*m];&#xA;  char jobvl = 'N', jobvr = 'V';&#xA;  double complex theta[m], alpha[m], beta[m], ev[m*m], work[lwork];&#xA;  double complex A[9] = { 12, -14, 3, 4, -4, 13, 2,  0, -4 }; // matrices stored columnwise&#xA;  double complex B[9] = { 4, -9, 21, -2, 1, -6, 15, 17, 3};&#xA;&#xA;  zggev_( &amp;amp;jobvl, &amp;amp;jobvr, &amp;amp;m, A, &amp;amp;m, B, &amp;amp;m, alpha, beta, NULL, &amp;amp;m, ev, &amp;amp;m, work, &amp;amp;lwork, rwork, &amp;amp;info );&#xA;  if( info ) printf(&quot;LAPACK error! Errorcode: %d\n&quot;, info);&#xA;&#xA;  // obtain eigenvalues&#xA;   for ( int i=0; i&amp;lt;3; i++ ) theta[i] = alpha[i]/beta[i];&#xA;&#xA;  // scale eigenvectors&#xA;  info = 1;&#xA;  for ( int i=0; i&amp;lt;m; i++ ) {&#xA;    rwork[0] = dznrm2_( &amp;amp;m, ev+i*m, &amp;amp;info );&#xA;    zdrscl_( &amp;amp;m, rwork, ev+i*m, &amp;amp;info );&#xA;  }&#xA;&#xA;  // sort eigenvalues ascending in absolute value (implicit reordering)&#xA;  for( int i=0; i&amp;lt;m; i++ )&#xA;    pi[i] = i;&#xA;  for( int i=m; i&amp;gt;1; i-- ) {&#xA;    for (int j=0; j&amp;lt;i-1; j++ ) {&#xA;      if ( fabs(ev[pi[j]]) &amp;gt; fabs(ev[pi[j+1]]) ) {&#xA;        info = pi[j];&#xA;        pi[j] = pi[j+1];&#xA;        pi[j+1] = info;&#xA;      }&#xA;    }&#xA;  }&#xA;  info = 1;&#xA;  zlapmt_( &amp;amp;info, &amp;amp;m, &amp;amp;m, ev, &amp;amp;m, pi );&#xA;&#xA;  // output&#xA;  for ( int i=0; i&amp;lt;3; i++ ) {&#xA;    printf(&quot;theta[%d] = %lf\nev[%d] = &quot;,i, creal(theta[i]), i);&#xA;    for ( int j=0; j&amp;lt;3; j++ ) &#xA;      printf(&quot;%lf &quot;, creal(ev[i*m+j]));&#xA;    printf(&quot;\n\n&quot;);&#xA;  }&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27012" LastActivityDate="2018-02-27T17:36:48.790" Title="LAPACK zlapmt_ freezes code" Tags="|eigenvalues|lapack|sorting|" AnswerCount="1" CommentCount="0" ClosedDate="2018-02-27T20:16:49.443" ContentLicense="CC BY-SA 3.0" />
  <row Id="28916" PostTypeId="2" ParentId="28915" CreationDate="2018-02-27T17:36:48.790" Score="1" Body="&lt;p&gt;A couple of notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By introducing &lt;em&gt;printf&lt;/em&gt; statements throughout the code, you would be able to see that even when your &lt;em&gt;zlapmt_&lt;/em&gt; call is present, the code certainly enters the main function (it would be obscure if it does not).&lt;/li&gt;&#xA;&lt;li&gt;Right now, your sorting algorithm is certainly non-optimal. I would strongly suggest you use quicksort or something else. Preferably, not a custom implementation.&lt;/li&gt;&#xA;&lt;li&gt;In your sorting algorithm, you are using &lt;em&gt;fabs&lt;/em&gt; to get an absolute value of the argument. In your case, it is &lt;em&gt;double complex&lt;/em&gt;. According to the specification, you should use &lt;em&gt;cabs&lt;/em&gt; in this case. I even got a compile error with &lt;em&gt;fabs&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding permutations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, you are using indexing from $0$ to $M-1$ in your permutation array &lt;em&gt;pi&lt;/em&gt;. According to &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-lapmt&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;lapmt_&lt;/em&gt; documentation&lt;/a&gt;, you indexing starts from $1$.&#xA;So, changing to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for( int i=0; i&amp;lt;m; i++ )&#xA;    pi[i] = i+1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Should solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I think there is a problem with your output code in the end. You never sort your &lt;em&gt;theta&lt;/em&gt;'s; however, you output the sorted array &lt;em&gt;ev&lt;/em&gt;. Make sure, you are doing what you actually intend to be doing there.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-02-27T17:36:48.790" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28917" PostTypeId="1" AcceptedAnswerId="28924" CreationDate="2018-02-27T20:59:02.500" Score="3" ViewCount="140" Body="&lt;p&gt;I am looking for assistance with calculating the fabry-perot standing modes in a resonator which has a non-static refractive index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a resonator with perfectly reflective mirrors only the standing modes experience sustained population. For a resonator with no internal medium the wavelengths at which a standing mode occurs can be written easily:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\lambda_m = \frac{mL}{2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a material with a refractive index the length of the cavity can be modified to have an effective length given by the product of the refractive index with the real length. The inclusion of a frequency-dependent refractive index causes an issue though. The frequencies that the standing modes appear can no longer be simply found because I believe the result is a non-analytic transcendental equation;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E_m(w) = \frac{2 hc}{m\ n(w)\ L}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the left is the energy of the $m$-th standing mode, and on the right h is Planck's constant, $c$ is the speed of light, m is the mode index, $L$ is the length of the cavity, and $n(w)$ is the frequency dependent refractive index. Written entirely in terms of angular frequency (instead of energy) the equation would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$w_m = \frac{4 \pi c}{m\ L\ n(w)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anyone be able to suggest a computational technique to try to solve this problem? Generally $n(w)$ is some complicated function of frequency, $w$, but it could always be fit to some polynomial if necessary. Any and all insight would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="27015" LastEditorUserId="9667" LastEditDate="2018-02-28T14:52:46.340" LastActivityDate="2018-02-28T14:52:46.340" Title="Fabry-Perot Interferometer with Frequency-Dependent Refractive Index" Tags="|numerical-analysis|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28918" PostTypeId="2" ParentId="16033" CreationDate="2018-02-27T22:13:51.343" Score="3" Body="&lt;p&gt;This can be formulated as a linear programming problem. Since you want to choose the right positions on each line and column, you may formulate this as finding a matrix $X$ with entries zero and $1$ such that the sum on each line and column is equal to $1$ and the dot product between $A$ and $X$ is maximized. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we want to maximize a linear functional with linear constraints and we expect the variables to take values in $\{0,1\}$ we may relax this to allow variables to belong to $[0,1]$. This is because linear functions always attain their extremal values at the vertices of the computational domain. That's why the simplex algorithm works in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complexity of the simplex algorithm is not polynomial, apparently, but for small examples it works really well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a Matlab implementation using linprog and the simplex algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function sci_comp1_mat&#xA;&#xA;A = [16 16 18 18 18; &#xA;     20 18 16 12 10;&#xA;     20 18 18 16 16;&#xA;     18 18 16 16 8;&#xA;     10 12 14 14 14];&#xA;&#xA;n = size(A,1);&#xA;f = double(A(:)); % coeffs matrix&#xA;% constraints&#xA;% sum on lines&#xA;&#xA;C = zeros(2*n,n^2);&#xA;for i=1:n&#xA;   C(i,linspace((i-1)*n+1,i*n,n))=1;&#xA;   C(n+i,linspace(i,n^2-n+i,n))=1;&#xA;end&#xA;b = ones(2*n,1);&#xA;&#xA;lb = zeros(size(f));&#xA;ub = ones(size(f));&#xA;&#xA;options.Algorithm = 'simplex';&#xA;x = linprog(-f,C,b,[],[],lb,ub,[],options);&#xA;dot(x,f)&#xA;x = reshape(x,n,n);&#xA;round(x)&#xA;&#xA;&#xA;Res = A.*x&#xA;&#xA;Res(Res==0) = '';&#xA;Res(:)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="3548" LastEditDate="2018-02-28T08:22:09.773" LastActivityDate="2018-02-28T08:22:09.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28919" PostTypeId="2" ParentId="28907" CreationDate="2018-02-27T22:30:04.470" Score="2" Body="&lt;p&gt;The problem was already solved in the comments section using linprog. Also CVX... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted just to point out that &lt;code&gt;fmincon&lt;/code&gt; can also be used: just set the linear equality constraints &lt;code&gt;Ax=b&lt;/code&gt; and construct a nonlinear constraint function after the model &lt;code&gt;[c,ceq,G,Geq] = nonlcon(x)&lt;/code&gt;, where the constraints will be of the form &lt;code&gt;c &amp;lt;= 0&lt;/code&gt;, &lt;code&gt;ceq =0&lt;/code&gt; and &lt;code&gt;G&lt;/code&gt; and &lt;code&gt;Geq&lt;/code&gt; are the respective gradients. This works for more general non-linear functions $f_1,f_2$ than the ones mentioned in the question. For the objective function just put a fictious one, like &lt;code&gt;@(x) 0&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2018-02-27T22:30:04.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28920" PostTypeId="1" CreationDate="2018-02-27T22:38:16.160" Score="1" ViewCount="106" Body="&lt;p&gt;I am solving 1D laplace problem discretized with finite differences (3-point stencil).&#xA;I would like to use additive Schwarz method in classical form: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$U_{k+1}=U_{k}+M^{−1} r_k,$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $r_k=F−A U_k$ and $  M^{−1} =  \sum_{i=1}^N R_i^T(R_i A R_i^T)^{−1} R_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Restriction operators $R_i$ are defined as binary matrices, with ones marking existing parts of the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Method converges nicely for non-overlapping case (block Jacobi). However, I am not able to obtain convergence in case of any overlap. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use restricted additive Schwarz, method is convergent again. &#xA;What is reason for such a behaviour? &lt;/p&gt;&#xA;" OwnerUserId="17045" LastActivityDate="2018-02-27T22:38:16.160" Title="overlapping additive schwarz" Tags="|optimization|domain-decomposition|" AnswerCount="0" CommentCount="6" ClosedDate="2018-03-01T16:44:12.040" ContentLicense="CC BY-SA 3.0" />
  <row Id="28921" PostTypeId="2" ParentId="23222" CreationDate="2018-02-27T22:43:27.757" Score="0" Body="&lt;p&gt;The answer to this depends on the hypotheses you consider for the points. If one or more points are collinear or if more than two segments meet at a point then things may become more complicated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'll assume the simple configuration where no three points are collinear and no three segments meet. Then the number of points a segment intersects is $L \times R$ where $L$ denotes the number of points on the &quot;left&quot; and $R$ on the &quot;right&quot; of the given segment. Since $R+L+2$ should give the total number of points, which is fixed, the maximal value of $L\times R$ is attained when $R$ and $L$ are as close as possible. A double loop on the vertices should give you the right answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm sure other more optimal algorithms could exist) &lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2018-02-27T22:43:27.757" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28922" PostTypeId="1" AcceptedAnswerId="28923" CreationDate="2018-02-26T04:49:30.670" Score="1" ViewCount="171" Body="&lt;p&gt;I've may have reached a point in my neural network research that I cannot continue without significant financial investment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using neuroevolution to evolve a network on the EMNIST data set. It is embarrassingly parallel (100 individuals * 116,323 records). The program is &lt;em&gt;impossible&lt;/em&gt; to accelerate with a GPU. I have tried. This leaves OpenMP, which to my surprise is granting a speedup near-equal to the number of cores. However, even a 72-core Virtual Machine is projected to take 34 days (200 iterations, minimum), and set me back &gt;$2,400.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MPI is on my list of changes to make to the program, but (assuming seamless parallelization) the cores and time required will not change. Several combinations of virtual machines and core-counts (Amazon, Azure, Google) have revealed a cost estimate of $1,500-$2,500. For 1 experiment. Assuming that ALL settings are correct, and I do not need to run further experiments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a sophomore-level undergraduate. My university has NO computing clusters, nor access to any. The best they have are 8-core workstations (that would take a year to run the job). I don't have $1,500 to spare. Are there any resources (sponsors, grants, laboratories, etc) that would help me get the cores I need (cloud, or &quot;home supercomputer&quot;)?&lt;/p&gt;&#xA;" OwnerDisplayName="user27075" LastEditorUserId="20688" LastEditDate="2018-05-17T07:53:48.657" LastActivityDate="2018-05-17T07:53:48.657" Title="Where can an undergraduate go to find cores on a budget?" Tags="|hpc|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="28923" PostTypeId="2" ParentId="28922" CreationDate="2018-02-28T00:09:08.653" Score="5" Body="&lt;p&gt;You need funding, you need a better algorithm, a better implementation, or you need to change your problem. To get funding, you need to work with your adviser. This could be about using grant money or getting a grant for an &lt;a href=&quot;https://www.xsede.org/&quot; rel=&quot;noreferrer&quot;&gt;XSEDE allocation&lt;/a&gt;. If your adviser has neither of those, it may not be a good choice to work with them even if you like their research: science requires funding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to make it cheaper, then think about a different algorithm. Even if it's doing a lot of branching and stuff like that, there must be some way to utilize linear algebra operations in some parts to make use of GPU (or Xeon Phi) acceleration. Maybe you need to precompute some values, or need a wholly different approach. Only you (and your adviser) would know. In fact, this algorithm development is many times the entire purpose of the research. Your adviser may have given you this project because no simple computer can solve this problem. Your goal isn't to find a bigger computer, it's to find out how to make your computer solve this problem (or at least, find out how to solve it better). No online tutorial or anything can help you with this: it's real research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also need to take a look at your implementation. If you're looping and branching in a high level language like Python, this could be the problem. You either need to re-write the code to be vectorized to do a bit better, or to get &quot;production-quality&quot; speeds you need to re-write it in a non-allocating format in a performance language (one which generate strictly typed and interprocedurally optimized compiled code) like C++ or Julia. It's not hard to find non-vectorized Python codes which can be made 1000x faster just by doing a better implementation and being careful with memory use (even vectorization is bad with memory use so that still leaves a lot to optimize).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And lastly, if this is a highly researched area where it's well-known it cannot do any better and you cannot get the funds/allocations to run the algorithms, pick a different project. Sometimes problems currently aren't possible. I highly doubt that your adviser would give you a problem that is in this domain though given that you're an undergraduate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, given you're a younger undergraduate, I would double/triple/quadruple check that you are actually correct that this kind of computing power is even required before going to blame anyone else. Embarrassingly parallel and (100 individuals * 116,323 records) is quite small, even to train on a desktop computer without a GPU. You should really make sure everything is as fast as it can be (rewrite it a few times in a few different languages, benchmark and profile each part, get rid of all memory allocations in the inner loops, etc.), make sure there's no better algorithm choices, make sure there is no cluster allocation you can get on, and only then should you consider moving to a new project that is more suitable to your current access to computing power.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-02-28T00:09:08.653" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28924" PostTypeId="2" ParentId="28917" CreationDate="2018-02-28T02:18:58.817" Score="3" Body="&lt;p&gt;I agree with your general assessment .. Fabry-Perot modes of a constant material are easy to compute/emumerate, but introducing dispersion (frequency dependent index) leads to a nonlinear solve. If you are able to differentiate your dispersion relation (write down the equation for dn/dw analytically) I would suggest you try Newton's method. If the dispersion relation is reasonably well behaved (some average/background index, plus a small perturbation that varies slowly/smoothly with w), you can probably use that to warm-start the iteration with a good initial guess (based on the constant index case, just use the average).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also want to draw inspiration from the procedures used to find cutoff frequencies for e.g. dielectric slab waveguide/optical fiber. Their dispersion relationships are non-linear, so you end up doing similar things .. graphical solution, Newton methods, and the like.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2018-02-28T02:18:58.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28932" PostTypeId="1" AcceptedAnswerId="28937" CreationDate="2018-03-01T11:22:44.130" Score="3" ViewCount="120" Body="&lt;p&gt;This question is rather simple but I have some difficulties to find code. Let us suppose that I wrote a routine, in a given language, that computes the evolution of a particle doing Brownian motion in one dimension. Given all the steps, how should I write an algorithm to derive the heat kernel from them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some good references are fine as an answer.&lt;/p&gt;&#xA;" OwnerUserId="3208" LastActivityDate="2018-03-02T04:10:05.740" Title="Computation of the heat kernel from Brownian motion" Tags="|computational-physics|reference-request|stochastic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28934" PostTypeId="1" CreationDate="2018-03-01T21:51:56.107" Score="1" ViewCount="280" Body="&lt;p&gt;I have implemented a schur complement domain decomposition method for solving large scale problems in Matlab. It works well but I think there should be some alternative approaches to implement some parts of the algorithm more efficiently to circumvent high memory usage. Specifically, I need to evaluate $M=BQ^{-1}B^T$ where $Q$ is a $p\times p$ real-valued, symmetric, sparse and positive-definite matrix and $B$ is a $q\times p$ extremely sparse matrix only containing $\{-1,+1\}$ nonzero entries ($q$ is usually much greater than $p$). My approach is $M=B*(Q\backslash B^T)$; however its evaluation becomes very memory- and time-consuming for not very large values of $p$. Any suggestion to get around this problem is warmly welcome.&lt;/p&gt;&#xA;" OwnerUserId="27035" LastEditorUserId="27035" LastEditDate="2018-03-01T21:58:58.140" LastActivityDate="2018-09-17T16:44:28.543" Title="Efficient evaluation of $BQ^{-1}B^T$ (Domain Decomposition Implementation)" Tags="|domain-decomposition|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28935" PostTypeId="2" ParentId="28749" CreationDate="2018-03-01T22:19:03.540" Score="5" Body="&lt;p&gt;The SPH implementation of the function in your question reads as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x,y)=\frac{1}{2}\big(\frac{\partial x}{\partial x}+\frac{\partial y}{\partial y}\big)=\frac{1}{2}\nabla\cdot\textbf{r}\approx\frac{1}{2}\sum_j^n(r_j-r_i)\frac{m_j}{\rho_j}\nabla W_{ij}=\langle f_i\rangle.\qquad(*)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very useful simulation tool could be the open-source &lt;strong&gt;Aboria&lt;/strong&gt;, constructed to facilitate the application of particle methods like SPH. Although it requires C++ programming skills, this is a good choice for your problem. You can find Aboria on GitHub &lt;a href=&quot;https://github.com/martinjrobins/Aboria&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and a related SPH example &lt;a href=&quot;https://martinjrobins.github.io/Aboria/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As long as programming is not a requirement, probably the easiest way to investigate this and other properties of SPH is using &lt;strong&gt;Nauticle&lt;/strong&gt;, which is also an open-source tool with objectives similar to that of Aboria. The Nauticle code is available &lt;a href=&quot;https://bitbucket.org/nauticleproject/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; with examples and user-guide. However, this tool is not an option if you prefer programming SPH manually. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running Nauticle with the following simple configuration file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;simulation:&#xA;  case:&#xA;    workspace:&#xA;      constants:&#xA;        - L: 1&#xA;        - rho0: 1000&#xA;        - csize: L/2&#xA;        - dx: csize/2.5&#xA;        - mass: dx^2*rho0&#xA;      variables:&#xA;        - dt: 1&#xA;      particle_system:&#xA;        domain:&#xA;          cell_size: csize|csize&#xA;          minimum: 0|0&#xA;          maximum: L/csize|L/csize&#xA;          boundary: 2|2&#xA;        grid:&#xA;          gpos: 0|0&#xA;          gsize: L|L&#xA;          goffset: 0|0&#xA;          gip_dist: dx|dx&#xA;      fields:&#xA;        - f: 0&#xA;    equations:&#xA;      - eq: f=sph_D00(r,mass,rho0,Wp52220,csize)/2&#xA;  parameter_space:&#xA;    simulated_time: dt&#xA;    print_interval: dt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives the result you requested. The equation in the fourth line from below implements the SPH differential operator $(*)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Visualisation of the result in Paraview:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KOqOo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KOqOo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20475" LastEditorUserId="20475" LastEditDate="2018-03-02T21:13:22.430" LastActivityDate="2018-03-02T21:13:22.430" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28936" PostTypeId="2" ParentId="28934" CreationDate="2018-03-02T03:23:05.983" Score="2" Body="&lt;p&gt;This is always going to be a somewhat costly calculation because the inverse of any &quot;interesting&quot; sparse matrix is generally dense and therefore so is $\mathbf M$. That said, there are smarter techniques than the obvious one you mention, that are asymptotically no slower than the factorization of $\mathbf Q$ itself, as long as $\mathbf B$ has no more rows than the root separator of $\mathbf Q$. This is likely the case, because in most DDM's the matrix $\mathbf Q$ comes from a volume discretization of the underlying PDE over a single domain (volume) and the operator $\mathbf M$ represents a Dirichlet-to-Neumann map or Robin-to-Robin map upon the boundary of a domain, such that the $\mathbf M$ has &quot;surface cardinality&quot; just like the root separator of $\mathbf Q$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a GPL-licensed sparse direct package called Myramath to perform exactly this calculation (see &lt;a href=&quot;http://www.myracore.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.myracore.com&lt;/a&gt;, it's also in my profile). In addition to the usual &lt;code&gt;factor()/solve()&lt;/code&gt; API, the solver also possesses a method called &lt;code&gt;.schur()&lt;/code&gt; that can form a matrix like $\mathbf M$. I'd also recommend another algorithm called &lt;code&gt;.partialsolve()&lt;/code&gt;, which also has interesting applications for substructuring/DDM. See the tutorial material for details, it starts &lt;a href=&quot;http://www.myracore.com/myramath/doc/html/a05647.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The &lt;code&gt;schur()&lt;/code&gt; and &lt;code&gt;partialsolve()&lt;/code&gt; methods are addressed &lt;a href=&quot;http://www.myracore.com/myramath/doc/html/a05666.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="20688" LastEditDate="2018-09-17T16:44:28.543" LastActivityDate="2018-09-17T16:44:28.543" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="28937" PostTypeId="2" ParentId="28932" CreationDate="2018-03-02T04:10:05.740" Score="3" Body="&lt;p&gt;You're interested in the solution $u(x,t)$ of the heat equation subject to some boundary conditions and an initial condition $u(x,0)=\delta(x-x_{0})$.  You can approximate this at times $t_{1}$, $t_{2}$, $\ldots$, $t_{n}$ by simulating the diffusion of a bunch of particles from $x_{0}$ and constructing histograms of the particle distributions at times $t_{1}$, $t_{2}$, $\ldots$, $t_{n}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a very inefficient and inexact way of getting at the heat kernel.  Why do you want to use this approach? What are you actually trying to do? &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-03-02T04:10:05.740" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28938" PostTypeId="1" CreationDate="2018-03-02T11:02:09.167" Score="1" ViewCount="285" Body="&lt;p&gt;I have to use Kronecker product of a matrix with a unit matrix is there any routine in ScaLapack or Lapack which can do so efficiently.&lt;/p&gt;&#xA;" OwnerUserId="27042" LastActivityDate="2018-03-02T11:02:09.167" Title="Kronecker product of matrices" Tags="|lapack|blas|scalapack|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28942" PostTypeId="2" ParentId="28296" CreationDate="2018-03-02T16:48:07.667" Score="1" Body="&lt;p&gt;Easiest solution: set up a grid &lt;span class=&quot;math-container&quot;&gt;$x_0, \ldots, x_N$&lt;/span&gt; (with spacing &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt;) large enough that your wavefunction (approximately) vanishes at the boundary. Then set up the (symmetric tridiagonal) Hamilton matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H_{ij}= \begin{cases} V(x_i) + \frac{1}{\Delta x^2} &amp;amp;&amp;amp; i=j\,,\\ -\frac{1}{2\Delta x^2} &amp;amp;&amp;amp; |i-j| = 1\,, \\ 0 &amp;amp;&amp;amp; \text{otherwise,} \end{cases}  $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$V(x)$&lt;/span&gt; is the potential and &lt;span class=&quot;math-container&quot;&gt;$\left\{-\frac{1}{2\Delta x^2}, \frac{1}{\Delta x^2}, -\frac{1}{2\Delta x^2}\right\}$&lt;/span&gt; is the usual three-point finite-difference approximation to the kinetic energy operator &lt;span class=&quot;math-container&quot;&gt;$-\frac12 \frac {\partial^2}{\partial x^2}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Diagonalize this matrix (e.g. using Eigen) to obtain &lt;span class=&quot;math-container&quot;&gt;$M = U E U^T$&lt;/span&gt;. The &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th column of &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; contains the eigenstate of you system, and the eigenvalue &lt;span class=&quot;math-container&quot;&gt;$E_i$&lt;/span&gt; is the corresponding eigenenergy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's as simple as it can get but at the same time very accurate for most one-dimensional problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your question it seems you want to use the shooting method or similar things. Don't do that. There is no reason in applying the shooting method to a linear one-dimensional problem: it's inefficient and not very stable.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is some code to try yourself:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;Eigen/Dense&amp;gt;&#xA;&#xA;int main()&#xA;{&#xA;    //parameters&#xA;    int N = 200;&#xA;    double x0 = -10.0;&#xA;    double xN = 10.0;     &#xA;    auto potential = [] (double x) {return 0.5*x*x;};&#xA;&#xA;    //set up the Hamiltonian&#xA;    Eigen::MatrixXd H = Eigen::MatrixXd::Zero(N,N);&#xA;    double dx = (xN - x0) / (N-1);&#xA;&#xA;    for(int i=0; i&amp;lt;N; ++i)&#xA;    {&#xA;        double x= x0 + i*dx;&#xA;&#xA;        if(i&amp;gt;0) { H(i-1,i) = -0.5/(dx*dx);  }&#xA;        H(i,i) = potential(x) + 1.0/(dx*dx);&#xA;        if(i&amp;lt;(N-1)) { H(i+1,i) = -0.5/(dx*dx);  }&#xA;    }&#xA;&#xA;    //... and diagonalize it&#xA;    Eigen::SelfAdjointEigenSolver&amp;lt;Eigen::MatrixXd&amp;gt; eigensolver(H);&#xA;    std::cout&amp;lt;&amp;lt; eigensolver.eigenvectors().transpose() &amp;lt;&amp;lt; std::endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="19794" LastEditDate="2019-04-02T11:44:38.840" LastActivityDate="2019-04-02T11:44:38.840" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="28944" PostTypeId="1" AcceptedAnswerId="28948" CreationDate="2018-03-02T19:25:40.800" Score="0" ViewCount="61" Body="&lt;p&gt;I'm trying to solve an equation whose solutions I know are plane waves but there are a few nuances. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, the equation is of the form&#xA;$$ \partial^2_t \psi + A(r)\partial^2_r \psi +B(r) \partial_r \psi + C(r)\psi =0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, my boundary conditions are $ \psi(a)=\psi(b)=0 $ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, my &quot;initial&quot; conditions are: $$ \psi(t,r)=f(r) $$ and&#xA;$$\frac{\partial \psi}{\partial t}(t,r) =g(r)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally, I'm integrating the equation backwards in time. I have a solution at a certain time $t$ and I need the solution at a much earlier time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given these conditions, can I just convert the partial derivatives into finite differences and proceed with the usual method applied to wave equations? Will this yield the correct results? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I apologize if this is a trivial question but I just want to confirm that I'm on solid footing before I begin developing the tedious algorithm.&lt;/p&gt;&#xA;" OwnerUserId="27049" LastEditorUserId="27049" LastEditDate="2018-03-03T08:12:15.503" LastActivityDate="2018-03-03T08:12:15.503" Title="Trying to solve a wave-like equation" Tags="|pde|finite-difference|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28945" PostTypeId="1" AcceptedAnswerId="28949" CreationDate="2018-03-02T22:29:06.983" Score="1" ViewCount="74" Body="&lt;p&gt;I am solving an equation of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(*)$ $0 = a(f) (\partial_rf)^2 + b(f) (\partial_rf) + c(f),$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$ is a real function of $r\in \mathbb{R}$, and $a,b,c$ are real functions of $f$. The boundary condition $f=f_0$ at $r=0$. Right now, I am solving $(*)$ by finite differencing $(*)$ and then applying a relaxation method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve $(*)$ using an explicit method, such as a Runge-Kutta type method. The only idea I have thought of so far is treating $(*)$ as an algebraic equation for $\partial_rf$, and then solving the resulting linear ODE using a Runge-Kutta solver. I find that procedure to be very inelegant though, and I'd like to know if there there is a body of algorithms that can solve (at least) specific kinds of nonlinear ODE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recognize there (may be?) issues with uniqueness of the solution(s) of $(*)$; I would appreciate any insights on how to numerically &lt;em&gt;select&lt;/em&gt; a specific solution as well.&lt;/p&gt;&#xA;" OwnerUserId="23388" LastActivityDate="2020-07-19T18:05:48.973" Title="Integrating a nonlinear ordinary differential equation" Tags="|nonlinear-equations|differential-equations|explicit-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28948" PostTypeId="2" ParentId="28944" CreationDate="2018-03-02T23:49:18.807" Score="2" Body="&lt;p&gt;Yes. You can. The wave equation is time reversal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your PDE, namely &#xA;$$\partial_{tt}u-\Delta u=0$$&#xA;is invariant under the transformation $t\to -t$.&#xA;This means that if $u(x,t)$ is a solution, $u(x,-t)$ is also a solution.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-03-02T23:49:18.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28949" PostTypeId="2" ParentId="28945" CreationDate="2018-03-03T06:09:57.840" Score="4" Body="&lt;p&gt;You don't have just a first-order ODE so you cannot use an explicit Runge-Kutta method. Because of the square term, you cannot bring this into mass matrix form even. Instead, what you have is an implicit ODE. This falls into the class of problems known as &lt;a href=&quot;https://en.wikipedia.org/wiki/Differential-algebraic_system_of_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Differential-Algebraic Equations&lt;/a&gt;, and that still works even though you don't have any pure algebraic variables. A common method to solve DAEs is to use a fully implicit BDF method. Software for this is available in &lt;a href=&quot;https://computation.llnl.gov/projects/sundials&quot; rel=&quot;nofollow noreferrer&quot;&gt;C++&lt;/a&gt;, &lt;a href=&quot;http://diffeq.sciml.ai/latest/tutorials/dae_example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Julia&lt;/a&gt;, &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/ode15i.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MATLAB&lt;/a&gt;, etc.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:05:48.973" LastActivityDate="2020-07-19T18:05:48.973" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="28950" PostTypeId="1" AcceptedAnswerId="28951" CreationDate="2018-03-03T12:25:21.463" Score="1" ViewCount="90" Body="&lt;p&gt;I want to implement an algorithm to solve a heat equation, i.e.&#xA;\begin{align*}&#xA;\partial_t u - \Delta u = f \text{ in } \Omega\times(0,T)\\&#xA;\partial_nu = 0 \text{ in } \partial\Omega \times (0,T)\\&#xA;u(0) = u_0&#xA;\end{align*}&#xA;I'm looking for some reasonable functions $f$ and $u_0$ to model a very hot area (e.g. a box in the middle of $\Omega = [0,1]^3$), which cools down slowly since the rest of the room has room temperature. Can you give me a proper example for this problem and explain me the physical meaning of $f$ and $u_0$?&lt;/p&gt;&#xA;" OwnerUserId="27054" LastActivityDate="2018-03-04T11:21:20.897" Title="Physical data for heat equation" Tags="|pde|parabolic-pde|data-sets|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28951" PostTypeId="2" ParentId="28950" CreationDate="2018-03-03T12:47:04.603" Score="1" Body="&lt;p&gt;The problem you proposes must be treated with care.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $f$ must have zero mean in the domain $\Omega$,i.e.&#xA;$$\int_\Omega{f\, dV}=0$$&#xA;Otherwise the problem is not well posed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Physically the terms of your equation mean:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\partial_tu$: the energy change of your system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-\Delta u$:the diffusion of the temperature through the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f$: Heat sources/sinks in the domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the example you proposes:&#xA;the function $f=0$, since there are not sources/sinks, and the initial condition, given by $u_0$ can be any distribution you want. It can be discontinuous. The only restriction is that $u_0\in L^2(\Omega)$ i.e. it must be square integrable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally the BC you want is not what you have written but:&#xA;$$\partial_n u=-h(u-u_\infty)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $h$ may be a function of $u$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This BC takes into account the fact that in equilibrium there is no heat loss through the boundaries to the outside (the room). Otherwise, if the flux is zero, the temperature reaches a value (homogeneous) being its energy a constant through time and it will never be at room temperature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The BC has also a physical meaning: the heat loss is proportional to the temperature differente between the hot object and its surroundings, i.e. the hotter the object is the greater the amount of heat that is loss. For example ice is melt quicker when is outside the fridge than outside because the $u-u_\infty$ in the first case is greater than the second given $u$ as the ice temperature.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-03-04T11:21:20.897" LastActivityDate="2018-03-04T11:21:20.897" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28952" PostTypeId="1" AcceptedAnswerId="28966" CreationDate="2018-03-03T16:06:25.040" Score="4" ViewCount="212" Body="&lt;p&gt;I have a table of the orbital coordinates and velocities of an object with time steps of &lt;strong&gt;1 minute&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I would like to interpolate this to a finer time increments with time steps of the order of &lt;strong&gt;1 second&lt;/strong&gt;.&#xA;How to make use of the velocities in coordinate interpolation and what might be the most accurate way to do this?&lt;/p&gt;&#xA;" OwnerUserId="24004" LastEditorUserId="24004" LastEditDate="2018-03-03T17:36:08.280" LastActivityDate="2018-03-06T00:42:39.163" Title="Interpolate the orbital coordinates of an object using coordinates and velocities vector" Tags="|interpolation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28953" PostTypeId="1" CreationDate="2018-03-03T17:26:30.673" Score="1" ViewCount="135" Body="&lt;p&gt;I am reading Chapter 5 of &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Saad's iterative methods book&lt;/a&gt;, and I don't understand section 5.2.1 about the two propositions of optimality results.&#xA;In the statements of the propositions, what does it mean that: $\tilde{x}$ is the result of a projection method onto $\mathcal K$ with the starting vector $x_0$? Because $\tilde{x}\in x_0 + \mathcal K$, does this mean $\tilde{x} = x_0 + Px_*$ or $\tilde{x} = Px_*$ for the true solution $x_* \in R^n$? I also can't see how section 1.12.4 can easily leads to the proof of the two conclusions,because they have a starting vector $x_0$. Also the A-norm in the first proposition is also different from the 2-norm in section 1.12.4. Could anyone give some explanations?&lt;/p&gt;&#xA;" OwnerUserId="18224" LastEditorUserId="18224" LastEditDate="2018-03-03T17:35:15.623" LastActivityDate="2018-03-05T04:04:31.743" Title="Questions about iterative projection methods in Saad book" Tags="|numerical-analysis|linear-solver|iterative-method|krylov-method|projection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28954" PostTypeId="2" ParentId="28952" CreationDate="2018-03-03T20:37:17.090" Score="8" Body="&lt;p&gt;That depends on how well you know the coordinates and velocities. If you have exact values, you can get a reasonable answer using &lt;a href=&quot;https://en.wikipedia.org/wiki/Cubic_Hermite_spline&quot; rel=&quot;noreferrer&quot;&gt;Hermite interpolation&lt;/a&gt;. This will give you a degree-3 polynomial in each window that matches the coordinates and velocities at all the endpoints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you do not know the coordinates and velocities exactly, but you do know something about the probability distribution of the measurement errors, you'll want to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Kalman_filter&quot; rel=&quot;noreferrer&quot;&gt;Kalman smoothing&lt;/a&gt; or a similar algorithm. Kalman-type methods are especially nice because you can also incorporate the physics into how you interpolate the data.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-03-03T20:37:17.090" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="28955" PostTypeId="1" AcceptedAnswerId="28956" CreationDate="2018-03-04T14:36:58.450" Score="4" ViewCount="440" Body="&lt;p&gt;Is measuring mean-squared-displacement in Monte Carlo simulations uncommon? I'm very interested to find out if this has actually ever been tried.&#xA;For instance, in the context of spheres, or cylindrical particles in a periodic simulation box, where we have our usual Monte Carlo moves of translation and rotation.&#xA;So a particle randomly selected and moved (either translation or rotation), this is repeated $N$ times in one Monte Carlo cycle, $N$ being the number of particles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can after each cycle, or even each displacement move, also calculate the MSD with respect to the initial configuration of the system. &#xA;For example having the centre of masses $\vec{R}$ of each cylinder, and their respective orientation vectors $\vec{O},$ after each move we can update the MSD along the direction of the cylinder by:&#xA;$$&#xA;\begin{align}&#xA;\Delta \vec{R}(t) &amp;amp;= \vec{R}(t)-\vec{R}(0) \\&#xA;\Delta \vec{R_{||}} (t) &amp;amp;= (\Delta\vec{R}(t)\cdot \vec{O}(0))\vec{O}(0)/l^2 &#xA;\end{align}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the total msd along the parallel direction gets updated (i.e., incremented) by $\Delta \vec{R_{||}}\cdot \Delta \vec{R_{||}}/N.$ Where $\vec{O}(0)$ is the orientation vector of the chosen cylinder at time 0 ( so initial condition). From this, I guess we can write the perpendicular to long axis contribution to msd as, $\Delta \vec{R}_{\perp} = \Delta \vec{R} - \Delta \vec{R}_{||}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, from the MSD curves calculated during the MC simulation, we'd extract diffusion properties among other things. &#xA;I wonder, is this at all meaningful to do in Monte Carlo simulation? Has the validity of this been discussed somewhere? I so far haven't found anything work on this, curious to find out if there are in fact any.&lt;/p&gt;&#xA;" OwnerUserId="24573" LastEditorUserId="24573" LastEditDate="2018-03-05T00:09:06.103" LastActivityDate="2018-03-25T14:12:36.333" Title="Mean-squared displacement in Monte Carlo studies" Tags="|simulation|monte-carlo|diffusion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28956" PostTypeId="2" ParentId="28955" CreationDate="2018-03-04T15:29:19.113" Score="5" Body="&lt;p&gt;This is possible (see [1]) but uncommon, as it requires Monte Carlo moves that alter the current conformations by a very small perturbation. In that setting of &quot;small&quot; Metropolis MC moves, it is usually easier (both in theory and in practice) to just use Molecular Dynamics instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Kikuchi, K., Yoshida, M., Maekawa, T., &amp;amp; Watanabe, H. (1991). Metropolis Monte Carlo method as a numerical technique to solve the Fokker—Planck equation. Chemical Physics Letters, 185(3–4), 335–338. &lt;a href=&quot;https://doi.org/10.1016/S0009-2614(91)85070-D&quot; rel=&quot;noreferrer&quot;&gt;https://doi.org/10.1016/S0009-2614(91)85070-D&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2018-03-04T15:29:19.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28957" PostTypeId="2" ParentId="28955" CreationDate="2018-03-04T22:25:48.450" Score="5" Body="&lt;p&gt;It is in some cases possible to map the dynamics obtained in MC simulations to other (more realistic) dynamics, especially for the case of dense colloidal suspensions. The following two papers talk about the problems and caveats of performing such a mapping:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://aip.scitation.org/doi/10.1063/1.3414827&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://aip.scitation.org/doi/10.1063/1.3414827&lt;/a&gt; (spherical particles)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://aip.scitation.org/doi/10.1063/1.3629452&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://aip.scitation.org/doi/10.1063/1.3629452&lt;/a&gt; (spherical particles with orientation degrees of freedom)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Even though both papers are behind paywalls, it should be possible to find reprints online.&lt;/p&gt;&#xA;" OwnerUserId="27040" LastEditorUserId="27040" LastEditDate="2018-03-25T14:12:36.333" LastActivityDate="2018-03-25T14:12:36.333" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28958" PostTypeId="2" ParentId="28953" CreationDate="2018-03-05T04:04:31.743" Score="3" Body="&lt;p&gt;In order to understand these results, you need to know how minimization and projection problems are connected. Namely,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let $\mathbb V$ be a subspace of $\mathbb C^n$ and take $y \in \mathbb C^n$; then $\hat y = \text{argmin}_{x \in \mathbb V} || y - x ||$ iff $(\hat y, x) = (y, x)$ for all $x \in \mathbb V$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here $(.,.)$ is a scalar product (not necessarily Euclidean) and $||.|| := (.,.)^{1/2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that $A\,x^* = b$ and consider a projection problem: find $x \in \mathbb L$ s.t. $(A\,x,k) = (b, k)$ for all $k \in \mathbb K$.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Idea #1: $\mathbb K := \mathbb L$&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Take $\mathbb K := \mathbb L$ and assume $A = A^* &amp;gt; 0$ (so that $A$ induces a scalar product $(u, v)_A := (A\,u, v)$ and corresponding norm). Note that in the projection problem $(A\,x,k) =: (x, k)_A$ and $(b, k) = (A\,x^*, k) =: (x^*, k)_A$, so it may be rewritten as follows: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find $x \in \mathbb L$ s.t. $(x, k)_A = (x^*, k)_A$ for all $k \in \mathbb K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From our theorem it follows that &#xA;$$&#xA;x = \text{argmin}_{l \in \mathbb L} || x^* - l ||_A,&#xA;$$&#xA;i.e. our $x$ is the best approximation of $x^*$ in $\mathbb L$ (in the sense of the energy norm).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Idea #2: $\mathbb K := A\,\mathbb L$&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Now choose $\mathbb K := \{A\,l : l \in \mathbb L \} =: A\,\mathbb L$. Applying similar arguments, you will get that &#xA;$$&#xA;x = \text{argmin}_{l \in \mathbb L} || b - A\,l ||.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Idea #1 gives rise to CG method and idea #2 gives rise to GMRES method when one chooses to use Krylov subspaces for $\mathbb L$ (but this is another story).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In the statements of the propositions, what does it mean that: $x$ is the result of a projection method onto $\mathbb K$ with the starting vector $x_0$?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assume that you have some guess $x_0$ for $x^*$. Than you want to search for &lt;strong&gt;correction&lt;/strong&gt;: find $c$ s.t. $A\,(x_0 + c) = b$, or $A\,c = r_0 := b - A\,x_0$. Then you can set $x = x_0 + c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally you want $c = e_0 := x^* - x_0$, but you do not know the error since you do not know the exact solution $x^*$. So it is natural to apply projection method to search for an approximation of $e_0$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find $c \in \mathbb L$ s.t. $(A\,c, k) = (r_0, k)$ for all $k \in \mathbb K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying e.g. idea #1 we have that&#xA;$$&#xA;c = \text{argmin}_{l \in \mathbb L} || e_0 - l ||_A = \\&#xA;\text{argmin}_{l \in \mathbb L} || (x^* - x_0) - l ||_A = \\&#xA;\text{argmin}_{l \in \mathbb L} || x^* - (x_0 + l) ||_A = \\&#xA;[ y := x_0 + l] = \\&#xA;\text{argmin}_{y \in x_0 + \mathbb L} || x^* - y ||_A - x_0,&#xA;$$&#xA;or &#xA;$$&#xA;x = \text{argmin}_{y \in x_0 + \mathbb L} || x^* - y ||_A.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $x_0 + \mathbb L := \{x_0 + l : l \in \mathbb L \}$ is an &lt;strong&gt;affine set&lt;/strong&gt; (shifted subspace). That is, when you have some guess for your solution, you are minimizing some kind of functional (energy norm of the error in this case) over an affine set instead of a subspace.&lt;/p&gt;&#xA;" OwnerUserId="21916" LastActivityDate="2018-03-05T04:04:31.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28959" PostTypeId="1" CreationDate="2018-03-05T06:55:19.660" Score="1" ViewCount="77" Body="&lt;p&gt;Given a flow inside a square duct with constant temperature at the walls $(T_{w1} = T_{w2} = T_w)$ the physical property in terms of temperature that repeats itself in a periodic fashion is the $\textbf{Dimensionless Temperature} ~ \mathbf{\theta}$ defined as shown in the picture below :- &lt;a href=&quot;https://i.stack.imgur.com/iwHr1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iwHr1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in defining $\theta$ that is valid for inhomogenous Dirichlet boundary conditions i.e. when $T_{w1} \neq T_{w2}$ which shall show a periodic behavior in fully developed flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. In an actual 3D flow through square duct there would be four faces(or walls) and all can be at different fixed temperature. For brevity purpose I have shown a case here which shows two walls in a 2D system (can be considered to be a flow between parallel plates with adibatic/free slip boundary condition for temperature/flow respectively).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am looking to extend the implementation of streamwise periodic boundary condition for non-equal wall temperatures. In case of equal wall temperatures the $\theta$ can be defined as depicted in the picture which then can be transcended to the periodic boundary condition requirement for Dirichlet conditions to be &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\theta_{0,y,z}$ = $\theta_{L,y,z}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $L$ in the fully developed periodic length. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting for $\theta$ and doing simplification one gets &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_{0,y,z} = c_1 T_{L,y,z} + c_2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$c_1 = \frac{T_b(0) - T_{wall}}{T_b(L) - T_{wall}} $ and&#xA;$c_2 = T_{wall} (1 - c_1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now for a constant and equal wall temperature using this formulation gives the right nusselt number values as well as temperature values (fixing $T_b(0)$ to be some reference tempearture).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of two constant but, different wall temperatures we have two possibilities &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T_{wall} = T_{wall1}$ or $T_{wall} = T_{wall2}$ or $T_{wall} = avg(T_{wall1}, T_{wall2}) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choosing one among the first two makes the temperature bounded by either of the wall temperatures but not both. I observe then a few cells near the wall2 or wall2 where the temperatures go below the cold wall temperature or hotter than the hot wall temperature (depending on which wall is taken as T_{wall} in the expression the problem occurs at the opposite wall). Choosing the third definition, if we see the expression for $c_1$ it is really possible to have a bulk temperature at the outlet to be that of the average wall temperatures which is then a condition of singularity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question remains how should we define $\theta$ so as to have a consistent representation of periodicity and also having a bounded solution as well. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/publication/44086753_On_the_Implementation_of_Stream-Wise_Periodic_Boundary_Conditions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Streamwise periodic boundary conditions for Constant $T_{wall}$&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0020722506000528&quot; rel=&quot;nofollow noreferrer&quot;&gt;Asymmteric Graez Nusslet problem&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17615" LastEditorUserId="17615" LastEditDate="2018-08-21T06:26:19.677" LastActivityDate="2018-08-21T06:26:19.677" Title="Defining dimensionless tempearture for Periodic flow systems" Tags="|fluid-dynamics|computational-physics|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="28960" PostTypeId="2" ParentId="28959" CreationDate="2018-03-05T08:58:53.640" Score="1" Body="&lt;p&gt;You could define $T_w$ as &#xA;$$T_w=\frac{1}{mes(\partial\Omega)} \int_{\partial \Omega}T_W\,dS$$&#xA;Where $T_W$ is the wall temperature and $T_w$ is the average wall temperature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 2D problem with constant wall temperatures, this integral reduces to&#xA;$$T_w=\frac{1}{2}(T_{W1}+T_{W2})$$&#xA;This is so, because in this context $\partial\Omega$ is the upper and lower boundary line of the channel with total length ($mes(\partial\Omega)$) equal to $2L$ and if its temperature is constant the integral over the whole boundary is equal to: $T_{W1}L+T_{W2}L$. Divide them and you will obtain the derired result.&#xA; For the 3D case it will be straightforward. Now the measure ($mes(\partial\Omega)$) will be the area of a plane, and you will be integrating the wall temperature over this plane...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you can simply put &#xA;$$\theta=T/T_b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if I say you that the nondimensionalisation with a difference is the same as with a single temperature?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your definition of $\theta$ change $T\to T+c$, with $c$ arbitrary. You see that this change also results in $T_b\to T_b+c$. If you put $c=T_w$ you will have:&#xA;$$\theta=T/T_b$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-03-05T12:45:25.670" LastActivityDate="2018-03-05T12:45:25.670" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="28961" PostTypeId="1" CreationDate="2018-03-05T13:14:03.820" Score="2" ViewCount="116" Body="&lt;p&gt;A non-linear molecule has $3N-6$ degrees of freedom ($N$ is the number of atoms; ignoring translation and rotation). Therefore, a set of $3N-6$ distances and/or angles is enough, to describe the whole system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am searching for an algorithm, which takes $3N-6$ ($N$ is the number of points) distances as an input and calculates all ${{N(N-1)/2}}$ distances between all points.&lt;/p&gt;&#xA;" OwnerUserId="27066" LastEditorUserId="27066" LastEditDate="2018-03-06T14:25:02.703" LastActivityDate="2018-07-06T10:42:03.170" Title="Algorithm to construct all distances of a system described by $3N-6$ distances" Tags="|algorithms|computational-geometry|distance-measure|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28962" PostTypeId="1" AcceptedAnswerId="28965" CreationDate="2018-03-05T14:11:10.540" Score="2" ViewCount="334" Body="&lt;p&gt;I am currently trying to parallelize the incompressible flow solver code. &#xA;However, when I run the code I realise that the parallel code takes much longer time than sequential code to finish one iteration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mesh is partitioned by ParMetis so the work load is balanced. I also timed my code, the main time-consuming part is the linear solver part. The method I used is the conjugate gradient and jacobi as the preconditioner from Petsc Krylov methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am wondering if there is any appropriate method and preconditioner for solving linear algebra in parallel?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="26225" LastActivityDate="2018-03-06T00:11:27.680" Title="Which iterative method and preconditioner from petsc should be used when solving linear algebra in parallel?" Tags="|linear-algebra|linear-solver|parallel-computing|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28963" PostTypeId="1" CreationDate="2018-03-05T16:08:43.060" Score="2" ViewCount="3220" Body="&lt;p&gt;I have a fluid dynamic solver written in python which I want to accelerate by moving the most expensive computations to the GPU. Ideally all arrays and sparse matrices used in my code should remain on the gpu, and matrices in COO format should be built directly from arrays on the gpu.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my search it seems that there are two python packages which may help; pyviennacl and petsc4py. Is there a reason I should choose one over the other?&lt;/p&gt;&#xA;" OwnerUserId="16379" LastActivityDate="2018-03-17T11:50:35.350" Title="Which python library for GPU sparse linear system solver library" Tags="|fluid-dynamics|sparse-matrix|petsc|cuda|viennacl|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28964" PostTypeId="1" AcceptedAnswerId="29011" CreationDate="2018-03-05T17:16:58.383" Score="5" ViewCount="157" Body="&lt;p&gt;Suppose I have $n$ generally nonlinear equations for $n$ variables, like e.g. for $n=2$ the system $F(x,y)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;x^2+2y-4&amp;amp;=0\\&#xA;\sqrt{8}x+y^2-5&amp;amp;=0&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By introducing variables for intermediate results, I can transform this into an (almost, at least, up to some additional singularities, which I don't care about for my question) equivalent set of equations with more variables, like e.g. the set of 4 equations for 4 variables $G(x,y,z,w)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;x\cdot x-z&amp;amp;=0\\&#xA;y\cdot y-w&amp;amp;=0\\&#xA;z+2y-4&amp;amp;=0\\&#xA;\sqrt{8}x+w-5&amp;amp;=0&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to solve a system like this numerically, say e.g., with a Newton-Solver like &lt;a href=&quot;https://computation.llnl.gov/projects/sundials/kinsol&quot; rel=&quot;noreferrer&quot;&gt;KINSOL&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there any expectable or even provable advantage regarding convergence in using the system $G$ with redundant variables ($z$ and $w$) over using the reduced system $F$? Or is it normally rather worse?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;If this question can be answered at all, is the answer sensitive to choosing a different solver?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With hand-waving arguments I can equally well come to opposite speculations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The more variables, the more freedom the algorithm has for finding the roots and doesn't get stuck so easily in difficult regions (like locally bad condition number or high curvature)&lt;/li&gt;&#xA;&lt;li&gt;The more variables, the higher dimensional the search space, and so the more likely it is that the algorithm can go astray&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I can't imagine that this problem has never been dealt with before. But obviously, I don't know the right keywords.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Addendum (20180310):&lt;/strong&gt;&#xA;the main idea behind this question is that the above principle can be driven to the limit by introducing auxlliary variables for &lt;strong&gt;every&lt;/strong&gt; intermediate result in a complicated set of equations. What remains is a set of kind of 'atomic' equations for every binary operator (referencing three variables), $b(x_i,x_j)-x_k=0$, or unary operator/elementary function (referencing two variables), $u(x_l)-x_m = 0$, instead of having more complicated compound equations $f(x_1,...,x_n)=0$ the constraint violations of which can be computed directly (which is good), but that are referencing many variables (which is bad). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a dense nonlinear system is reduced to a sparse nonlinear system, and similarly during solution this results in a sparse linear system. The 'atomic' equations are probably (hopefully) easier for a root-finding algorithm to be dealt with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the question could be reformulated as: &lt;strong&gt;is it better (w.r.t. nonlinear convergence speed) to solve a dense (or very nonlocally coupled) system with less variables, or is it better to solve an equivalent sparse (or only quite locally coupled) system with many variables (most of which just represent intermediate results of function computations).&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="27072" LastEditorUserId="27072" LastEditDate="2018-03-10T13:12:45.823" LastActivityDate="2018-03-10T16:03:25.150" Title="Can redundant variables be beneficial for root-finding convergence" Tags="|nonlinear-equations|sparse-matrix|newton-method|rootfinding|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28965" PostTypeId="2" ParentId="28962" CreationDate="2018-03-06T00:11:27.680" Score="3" Body="&lt;p&gt;There are many methods for solving linear systems in parallel, but fundamentally what works and what doesn't depends crucially on the properties of the linear system you are trying to solve. Among those, the Jacobi preconditioner is about the worst you can choose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've summarized what I know about this topic in lectures 34-38 and 41.75 here: &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-03-06T00:11:27.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28966" PostTypeId="2" ParentId="28952" CreationDate="2018-03-06T00:42:39.163" Score="4" Body="&lt;p&gt;You can interpolate values between $[a, b]$ using Hermite interpolation. You first map the interval $[a, b]$ to $[-1,1]$, and the values are computed as:&#xA;$$f(x) \approx N_1(x) f(a) + N_2(x) f(b) + \frac{b - a}{2}[N_3(x) f'(a) + N_4(x) f'(b)]\quad \forall x\in [-1, 1]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&#xA;\begin{align}&#xA;N_1 (x) &amp;amp;= \frac{1}{4} (x - 1)^2 (2 + x)\\&#xA;N_2 (x) &amp;amp;= \frac{1}{4} (x + 1)^2 (2 - x)\\&#xA;N_3 (x) &amp;amp;= \frac{1}{4} (x - 1)^2 (x + 1)\\&#xA;N_4 (x) &amp;amp;= \frac{1}{4} (x + 1)^2 (x - 1)\, .&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you just need to loop over each pair of points.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-03-06T00:42:39.163" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28967" PostTypeId="1" CreationDate="2018-03-06T08:38:20.323" Score="2" ViewCount="133" Body="&lt;p&gt;I am trying to solve the LWH traffic flow equation, which is a nonlinear hyperbolic equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \rho}{\partial t}+\frac{\partial (v\rho)}{\partial x}=0,$$&#xA;where &#xA;$$v=v_0(1-\frac{\rho}{c}).$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$v_0$ is a given constant that may or may not vary with position, which represents the optimal velocity for the vehicles, $\rho$ is the density of the cars, and $c$ is a given constant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a discontinuous initial condition, simply using Lax-Wendroff will give overshoots. So I am thinking of adding a flux-limiter, say Superbee. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But most of the resources I have read only talk about applying flux-limiters to linear equations. So how should I apply a flux-limiter to Lax-Wendroff with this nonlinear problem?&lt;/p&gt;&#xA;" OwnerUserId="27078" LastEditorUserId="17294" LastEditDate="2018-03-06T11:08:10.370" LastActivityDate="2022-06-14T14:08:10.850" Title="Apply flux-limiter to nonlinear hyperbolic equation" Tags="|numerical-analysis|hyperbolic-pde|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="28969" PostTypeId="1" AcceptedAnswerId="28975" CreationDate="2018-03-06T14:01:23.240" Score="2" ViewCount="3418" Body="&lt;p&gt;I am quite new in snappyhexmesh and OpenFoam world and I am trying to mesh a multi stl geometry. My geometry is a geometry which is split in 3 different stl files. Globally it is closed, but each single part is not closed. As far as each stl is not closed the feature extract step crashes. How can I solve this problem? Have I to close each stl and automatically snappy will connect them?&lt;/p&gt;&#xA;" OwnerUserId="27080" LastActivityDate="2018-03-07T09:44:03.953" Title="SnappyHexMesh and multi stl geometry" Tags="|openfoam|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28970" PostTypeId="1" AcceptedAnswerId="28971" CreationDate="2018-03-06T14:04:51.723" Score="4" ViewCount="752" Body="&lt;p&gt;Suppose I have a symmetric matrix $K$.  Subdivide $K$ into pieces as&#xA;$$K=\begin{pmatrix} K_{11} &amp;amp; K_{12} \\ K_{21} &amp;amp; K_{22}\end{pmatrix},$$&#xA;where $K_{21}=K_{12}^\top$.  Then, the Nystrom approximation of $K$ replaces $K_{22}$ with the approximation $$K_{22}\approx K_{12}^\top K_{11}^{-1}K_{12}.$$  See e.g. &lt;a href=&quot;https://papers.nips.cc/paper/1866-using-the-nystrom-method-to-speed-up-kernel-machines.pdf&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt; for an introduction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, suppose $K$ is &lt;em&gt;asymmetric&lt;/em&gt;.  Here's my question:  &lt;strong&gt;If we subdivide $K$ as above, does there exist a &quot;Nystrom-style&quot; approximation of $K_{22}$?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm OK with assuming $K_{11}$ is chosen to be square and invertible if that helps.  It seems naively that I could use exactly the same formula when $K$ is not square, but of course the proof of the Nystrom approximation no longer holds; I assume an eigenvalue argument has to be replaced with an SVD.&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2018-03-06T17:18:50.033" Title="Nystrom approximation of SVD for asymmetric matrices" Tags="|linear-algebra|eigenvalues|eigensystem|svd|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="28971" PostTypeId="2" ParentId="28970" CreationDate="2018-03-06T17:18:50.033" Score="4" Body="&lt;p&gt;Nemtsov, Averbuchm, and Schclar's &quot;&lt;a href=&quot;https://content.iospress.com/articles/intelligent-data-analysis/ida854&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matrix compression using the Nyström method&lt;/a&gt;&quot; (2016) seems relevant:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Nyström method is routinely used for out-of-sample extension of kernel matrices. We describe how this method can be applied to find the singular value decomposition (SVD) of general matrices and the eigenvalue decomposition (EVD) of square matrices. [...] Our algorithm is applicable to general matrices whereas previous methods&#xA;  focused on kernel matrices.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In particular, they show that a matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M=&#xA;\begin{bmatrix}&#xA;A_M &amp;amp; B_M \\ F_M &amp;amp; C_M&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can be approximated as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\hat M = &#xA;\begin{bmatrix}&#xA;A_M &amp;amp; B_M \\&#xA;F_M &amp;amp; F_M A_M^+ B_M&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A_M^+$ denotes the pseudo-inverse of $A_M$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kumar and Schneider's seemingly thorough &quot;&lt;a href=&quot;https://www.tandfonline.com/doi/abs/10.1080/03081087.2016.1267104&quot; rel=&quot;nofollow noreferrer&quot;&gt;Literature survey on low rank approximation of matrices&lt;/a&gt;&quot; (2016) provides a handy bibliography of Nyström method error estimates, ensemble methods, and adaptive sampling techniques. The foregoing is the only paper it cites as applying Nyström to general matrices.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2018-03-06T17:18:50.033" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28973" PostTypeId="1" AcceptedAnswerId="28988" CreationDate="2018-03-06T18:10:34.877" Score="0" ViewCount="222" Body="&lt;p&gt;I am currently working on a scheme that uses finite differences WENO5-Z with 3rd Order Runge-Kutta time integration for solving the Euler equations. The code projects the conserved variables and fluxes onto the eigenspace using Roe-averages at i+1/2 and uses a local Lax-Friedrichs flux splitting before applying the WENO reconstruction to the interface fluxes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I perform several numerical test (Sod Problem, Modified Sod, and Lax Problem) I get a spurious behavior in the left-traveling rarefaction wave (looks a lot like an expansion shock) in two of them, specifically in the Sod and Lax Problem. The lax problem also displays an oscillatory behavior in across the contact discontinuity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attached below the plots for the density, pressure, and velocity. I was wondering if any of you have encountered this issue before. I am surprised to see that kind of behavior since I am implementing Lax-Friedrichs flux splitting. Thank you very much in advance for your help!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hiO6P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hiO6P.png&quot; alt=&quot;WENO5-Z Numerical Tests&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26429" LastActivityDate="2018-03-08T02:12:20.893" Title="Unphysical Behaviour Characteristic-Wise WENO5-Z" Tags="|finite-difference|fluid-dynamics|oscillations|weak-solution|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28974" PostTypeId="2" ParentId="22037" CreationDate="2018-03-06T18:43:57.297" Score="1" Body="&lt;p&gt;What you're describing (each node needs every other node's data) is exactly implemented by &lt;a href=&quot;https://www.mpich.org/static/docs/v3.2/www3/MPI_Allgather.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MPI_Allgather&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1T5os.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1T5os.png&quot; alt=&quot;Diagram of the &amp;lt;code&amp;gt;MPI_Allgather&amp;lt;/code&amp;gt; operation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might expect &lt;code&gt;MPI_Allgather&lt;/code&gt; to be more efficient than numerous &lt;code&gt;MPI_ISEND&lt;/code&gt;/&lt;code&gt;MPI_IRECV&lt;/code&gt; pairings since making MPI aware of your intended communication pattern could allow it to optimize information flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you have wide variation in per-node calculation time, the &lt;code&gt;MPI_ISEND&lt;/code&gt;/&lt;code&gt;MPI_IRECV&lt;/code&gt; pairings would allow you to begin transferring information immediately as the nodes complete their operations, which might reduce latency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of these trade-offs are discussed in the answers to &lt;a href=&quot;https://scicomp.stackexchange.com/q/1772/17088&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best thing to do is to profile your communication using, say, Vampir, and address the problems you find. Otherwise, try both and compare their performance.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2018-03-06T18:43:57.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28975" PostTypeId="2" ParentId="28969" CreationDate="2018-03-06T22:36:16.313" Score="1" Body="&lt;p&gt;Did you try to merge the three STLs to create one, closed STL. OpenFOAM brings with it a tool&lt;code&gt;surfaceAdd&lt;/code&gt; to join STLs.&#xA;Apart from the problems you encountered in the feature extraction step, how do you think, the cell removal stage of snappy would cope with a non-closed surface?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Answer to first comment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Join the STLs as separate solids. So, that you have one resulting STL, with each original STL, being an individual STL solid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In snappy, you can now specify the refinement level based on these solids, see the regions sub-dictionary.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;castellatedMeshControls&#xA;{&#xA;  refinementSurfaces&#xA;  {&#xA;    mySTL&#xA;    {&#xA;      level (2 2);&#xA;&#xA;      regions&#xA;      {&#xA;        STL_region01&#xA;        {&#xA;          level (3 3);&#xA;        }&#xA;&#xA;        STL_region02&#xA;        {&#xA;          level (4 4);&#xA;        }&#xA;      }&#xA;    }&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the resulting mesh, the STL solids will be individual patches, following the naming scheme: &lt;strong&gt;stlName&lt;/strong&gt;_&lt;strong&gt;stlSolid&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3510" LastEditorUserId="3510" LastEditDate="2018-03-07T09:44:03.953" LastActivityDate="2018-03-07T09:44:03.953" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28978" PostTypeId="2" ParentId="10809" CreationDate="2018-03-06T22:45:29.497" Score="2" Body="&lt;p&gt;As it was discussed in the comments, the fluid density is most likely the culprit. If you are using an incompressible solver, such as &lt;code&gt;icoFoam&lt;/code&gt;, &lt;code&gt;simpleFoam&lt;/code&gt; or &lt;code&gt;pimpleFoam&lt;/code&gt;, then no density is used.&#xA;If you take a look at the source code of the said solvers, you will find no reference to any density. Also, in the case set-up, the pressure field will have the dimension of pressure divided by density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OpenFOAM operates on a need-to-know basis, it reads the information it needs from the case files, all other content of the case files in ignored. So, even if you specify a density in the transportProperties, an incompressible solver will always solve the equations of fluid motion, which are divided by density.&lt;/p&gt;&#xA;" OwnerUserId="3510" LastActivityDate="2018-03-06T22:45:29.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28980" PostTypeId="1" CreationDate="2018-03-07T08:26:47.643" Score="0" ViewCount="100" Body="&lt;p&gt;Let's say I have an algorithm that can be tuned by a parameter $h&amp;gt;0$ and is expected to converge as $h\to 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to study the computational complexity of this algorithm, i.e., how the required runtime increases with increasing accuracy. For this purpose, I run the algorithm for different values of $h$ (say, $h_1&amp;gt;h_2&amp;gt;\dots&amp;gt;h_n&amp;gt;\dots &amp;gt;h_N$) and summarize the accuracy (computed in comparison to a different, established method, with high accuracy) and runtime of these runs in a plot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plot shows runtime on the horizontal axis and accuracy on the vertical axis. However, sometimes, the runtimes are non-monotonic as I increase $n$. Should I reorder the runs so as to have a plot where the accuracy is a function of the work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the plots may look like this if I just connect run $n$ to run $n+1$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kcKhn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kcKhn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that run $n=2$ took more time than run $n=3$ but also ended up being more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reordered plot would look as below. Note that the data points have not changed, they are just connected in a different order. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fatv5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fatv5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another example would be that run $n=2$ takes longer but is still less accurate than $n=3$. The unshuffled and shuffled plots then would look like this:&#xA; &lt;a href=&quot;https://i.stack.imgur.com/pv2I2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pv2I2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7789" LastEditorUserId="7789" LastEditDate="2018-03-07T12:24:08.507" LastActivityDate="2018-03-07T15:03:10.223" Title="How to report non-monotonic runtimes in convergence plots" Tags="|convergence|visualization|plotting|publications|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="28982" PostTypeId="2" ParentId="19253" CreationDate="2018-03-07T10:29:45.000" Score="2" Body="&lt;p&gt;An alternative to the Incremental SVD is the Hierarchical Approximate Proper Orthogonal Decomposition &lt;a href=&quot;http://arxiv.org/abs/1607.05210&quot; rel=&quot;nofollow noreferrer&quot;&gt;HAPOD&lt;/a&gt;, of which an implementation can be found on github: &lt;a href=&quot;http://git.io/hapod&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://git.io/hapod&lt;/a&gt; . The HAPOD has rigorous error bounds and a special case is an incremental variant.&lt;/p&gt;&#xA;" OwnerUserId="27091" LastActivityDate="2018-03-07T10:29:45.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28983" PostTypeId="2" ParentId="28980" CreationDate="2018-03-07T15:03:10.223" Score="2" Body="&lt;p&gt;It sounds like you actually have three pieces of information here -- h, error, and runtime.  Plotting accuracy vs runtime isn't that useful if there is no correspondence with h (and so the user of your algorithm can't know how to pick h to get the desired accuracy/runtime).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd do scatter plots with color of the glyph to indicate h.  In this way your first example would make it clear, even with only glyphs, that something weird is going on here.  Reordering the runs hides the fact that some sort of strange cancelation/unexpected behavior CAN happen, so expecting monotonic (convergence, runtime increase) is not to be expected.&lt;/p&gt;&#xA;" OwnerUserId="508" LastActivityDate="2018-03-07T15:03:10.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28987" PostTypeId="2" ParentId="28961" CreationDate="2018-03-07T22:54:12.363" Score="1" Body="&lt;p&gt;Assuming you can live with an approximate solution: you can re-formulate the problem as a graph embedding (or metric embedding) problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your $N$ atoms $a_1,\dots,a_N$ are the vertices (nodes) of a graph $G$. &#xA;For every pair of atoms $(a_i, a_j)$ for which you know a distance, you create an edge (link) in $G$. Now assign a length to each edge, using the known distance $d(a_i,a_j)$. Note that the edges in this graph will have nothing to do with the atomic bonds in your molecule, they are just there to capture the known distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your task is to find an embedding $f: G \rightarrow R^3$ of the graph into the 3-dimensional space $R^3$, such that $dist(f(a_i), f(a_j))=d(a_i, a_j)$, where $dist$ is the actual euclidean distance in $R^3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to do that is to model the edges as springs, with lengths $d(a_i, a_j)$ and simulate the resulting mechanical system. Start with arbitrary positions of the atoms. At each time step compute the resulting force from all the springs that are attached to an atom. Move a little into that direction. Repeat. The system will come to rest at or near a solution of your problem (ignoring oscillations and such). Stopping criteria for the simulation might be: it (nearly) stops moving, total energy has reached a minimum, distances are close enough to the desired $d(a_i, a_j)$ etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out &lt;a href=&quot;https://en.wikipedia.org/wiki/Force-directed_graph_drawing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Force directed graph drawing&lt;/a&gt;, or just use some physics engine to simulate the whole thing. There is a bunch of literature about these kinds of simulations, how to optimize them, how to avoid getting stuck in local minima etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding exact solutions, I'd suspect that you cannot get a closed-form solution, i.e. some formula that takes the $d(a_i, a_j)$ as input and produces valid positions of the atoms as outputs. Same for getting all the atom distances as outputs.&lt;/p&gt;&#xA;" OwnerUserId="2148" LastActivityDate="2018-03-07T22:54:12.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28988" PostTypeId="2" ParentId="28973" CreationDate="2018-03-08T02:12:20.893" Score="1" Body="&lt;p&gt;I finally found the problem and corrected in the code. The issue was that the scheme was not able to handle left-traveling waves due to an incorrect implementation of the eigendecomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The projection on the characteristic space was being done only on the point-values of the vector of conserved variables, that is:&#xA;$$ V_i = L(u_{i+1/2})U_i$$&#xA;$$ G_i = L(u_{i+1/2})F(U_i)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U_i$ is the vector of conserved variables, $F(U_i)$ is the vector of fluxes, and $L(u_{i+1/2})$ represents the left eigenvectors evaluated at $x_{i+1/2}$ using Roe-averages. Note that this is incorrect since the eigenprojection must be done for ALL the numerical stencil associate with $x_{i+1/2}$. The right expressions would be:&#xA;$$ V_k = L(u_{i+1/2})U_k$$&#xA;$$ G_k = L(u_{i+1/2})F(U_k)$$&#xA;where for WENO5 $k= i-2,i-1,i,i+1,i+2,i+3$. I did not find this obvious when reading most of the references so I am attaching a quick diagram below for completeness. It is not intended to be rigorous but simply informative:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XdB2I.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XdB2I.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Cheers!&lt;/p&gt;&#xA;" OwnerUserId="26429" LastActivityDate="2018-03-08T02:12:20.893" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28989" PostTypeId="2" ParentId="28961" CreationDate="2018-03-08T08:17:09.980" Score="1" Body="&lt;p&gt;Similar to Berth's suggestion, you could fix one atom and try to minimize the function $$g(f)=\sum_{(i,j)\in G}\left[d(f_i,f_j)^2 -d_{ij}^2\right]^2$$ where $f$ is a matrix of 3D coords. This function is non-convex and should be optimized using a global optimization technique.&lt;/p&gt;&#xA;" OwnerUserId="23072" LastActivityDate="2018-03-08T08:17:09.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28991" PostTypeId="1" CreationDate="2018-03-08T11:24:12.047" Score="1" ViewCount="81" Body="&lt;p&gt;I am trying to solve the radially symmetric polar form of the PDE with homogeneous Neumann BC in deal.II on a unit circle:&#xA;$$ u_t = \Delta u - \nabla \cdot (u \nabla h) $$&#xA;$$ h_t = \Delta h $$&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I am not sure how to handle the origin &amp;amp; how the bilinear form change in this case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, how to use the grid and triangulation in deal.II.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Could someone please explain or suggest similar literature?&lt;/p&gt;&#xA;" OwnerDisplayName="user27099" LastEditorUserId="9667" LastEditDate="2018-03-08T13:54:19.917" LastActivityDate="2018-03-08T13:54:19.917" Title="Radially symmetric system of PDEs in deal.II" Tags="|finite-element|boundary-conditions|deal.ii|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28992" PostTypeId="1" AcceptedAnswerId="28993" CreationDate="2018-03-08T16:01:40.937" Score="2" ViewCount="215" Body="&lt;p&gt;A jump-diffusion process is a stochastic process where both continuous noise (in my case complex Wiener noise $dZ,dZ^*$ such that $dZ^2=dZ^{*2}=0,|dZ|^2=dt$) and discrete Jumps (in my case Poissonian $dN,dM$ such that $dN^2=dN$, $dM^2=dM$, $\langle dN\rangle=A dt$, $\langle dM \rangle=B dt$ ). My equations are in the Itô sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using an update rule with a deterministic evolution term $dt$ as well as the noises, I can evolve a complex vector (or ket-if you like) $v$ through time. Coeficients $A$ and $B$ depend on $v$, but not so much that a $dN&amp;gt;1$ would cause problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently solving this system with the plain forward Euler method in MATLAB, generating Wiener nose with &lt;code&gt;randn&lt;/code&gt; and Poisson-noise with &lt;code&gt;poissrnd&lt;/code&gt;. In addition, after each timestep I do an additional renormalization $v\rightarrow v/||v||$ (I know the norm is a constant of motion).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in order to make everything converge properly, I have to set the timestep $dt$ so small that the solving of the equations takes a long time. Are there good practical solvers (e.g. Runge-Kutta) with more performance that Euler?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try to look around what is commonly used for jump-diffusion processes, e.g in  DifferentialEquations.jl or QuantumOptics.jl, the expected time between two jumps is typically much larger than the timestep( $dN,dM\approx 0$) so that one can distinguish deterministic evolution from jumps taking place at random times. This assumption is not fulfilled in my case, the jumprate is so high (it dominates the dynamics) that jumps may coincide the same timestep. On the other hand, it is not so macroscopically large that we can just use the central-limit-theorem and approximate the poisson-terms by gaussian noise.&lt;/p&gt;&#xA;" OwnerUserId="27102" LastActivityDate="2018-03-08T18:16:52.127" Title="Jump-Diffusion process: practical solver beyond Euler method?" Tags="|quantum-mechanics|stochastic|differential-equations|stochastic-ode|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="28993" PostTypeId="2" ParentId="28992" CreationDate="2018-03-08T16:48:17.143" Score="2" Body="&lt;p&gt;The setup that is used in DifferentialEquations.jl and QuantumOptics.jl is what's known as time-adaptive jumping. It's nice because it allows for jump events to do things like change the number of DEs, and the jumps are computed exactly. However, it does have the limitation that jumps are computed exactly, so if you have a high jump rate then this slows down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So then the other option is to use regular jumps instead of time-adapted jumps. The easiest of course is the Euler method, also known as tau-leaping in systems biology circles. The best resource on this topic for jump diffusions is probably &lt;a href=&quot;https://link.springer.com/article/10.1007/s10614-006-9066-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Solution of Stochastic Differential Equations with Jumps in Finance&lt;/a&gt;. This book includes many high order schemes for regular jumping, but they do require lots of derivatives except for a few Runge-Kutta type schemes. There are many schemes for pure-jump problems (i.e. where the drift and diffusion terms are zero) that are not mentioned in that book but only in systems biology literature (such as binomial leaping), but of course if you always have drift and diffusion terms that doesn't matter. This is still a very active area of research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for implementations that are out there, I do not know of any that are not for pure-jump problems (and for pure-jump problems the ones I know of are for biological problems like &lt;a href=&quot;http://www.stochss.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;StochSS&lt;/a&gt;). FWIW DifferentialEquations.jl is &lt;a href=&quot;https://github.com/JuliaDiffEq/DifferentialEquations.jl/issues/250&quot; rel=&quot;nofollow noreferrer&quot;&gt;restructuring to allow for implementing the regular jump integrators&lt;/a&gt; (i.e. going through and implementing most of Platen's, along with using &lt;a href=&quot;https://github.com/JuliaDiffEq/SciCompDSL.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciCompDSL.jl&lt;/a&gt; to automatically build the Jacobians for the user), but I wouldn't expect that for a few months.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2018-03-08T18:16:52.127" LastActivityDate="2018-03-08T18:16:52.127" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="28994" PostTypeId="1" CreationDate="2018-03-08T17:00:41.913" Score="1" ViewCount="35" Body="&lt;p&gt;Based on your empirical knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/article/10.1007/s11075-012-9625-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; suggests a nonlinearly accelerated Fourier series approach, such as the one proposed &lt;a href=&quot;https://epubs.siam.org/doi/abs/10.1137/0903022&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but I have one constraint: we should be able to express the method as a linear combination of the function $F$ at different frequencies $s$, that is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(t) \approx \sum\limits_{i=1}^N w_i^t \, F(s_i^t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this equation I am looking for the most accurate algorithm that requires the lowest $N$.&lt;/p&gt;&#xA;" OwnerUserId="4826" LastEditorUserId="4826" LastEditDate="2018-03-08T17:21:34.080" LastActivityDate="2018-03-08T17:21:34.080" Title="What is the inverse Laplace transform algorithm that is most accurate given the fewest frequencies considered?" Tags="|fourier-transform|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="28999" PostTypeId="2" ParentId="28858" CreationDate="2018-03-09T02:56:19.727" Score="1" Body="&lt;p&gt;You can use an interpolation function (see &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/interp3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;interp3&lt;/a&gt;) to estimate the temperature at each position in a cross-section. Set the extrapolation value to NaN so that points outside your chamber do not contribute. &lt;/p&gt;&#xA;" OwnerUserId="26904" LastActivityDate="2018-03-09T02:56:19.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29001" PostTypeId="1" CreationDate="2018-03-09T19:02:58.693" Score="1" ViewCount="17" Body="&lt;p&gt;I have a system of 1st order odes given by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\dot{x_1}(t) = \alpha_1 f_1(x_1,t) + \beta_1 u(t) \\&#xA;\dot{x_2}(t) = \alpha_2 f_2(x_2,t) + \beta_2 u(t)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are constrained by an algebraic equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_1(t) + x_2(t) =  k $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\left( \alpha_1,\alpha_2, \beta_1,\beta_2 , k \right) \in \mathbb{R}$  are known constants (i.e. parameters). $f_1(t)$ and $f_2(t)$ are both unknown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting from a rich set of input-output &lt;strong&gt;noise-free&lt;/strong&gt; data available from simulating a complex proxy system, what would be the best procedure to identify (even a subset of repeatable/characteristic properties) the unknown &lt;strong&gt;&lt;em&gt;possibly time-varying&lt;/em&gt;&lt;/strong&gt; functions $f_1(x_1,t)$ and $f_2(x_2,t)?$ I am almost certain that $f_1(x_1,t)$ and $f_2(x_2,t)$ are both linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a suitable approach that shall work well to arbitrary excitations in all future simulations. (NOT merely a curve-fitting procedure to match a specific input-output dataset)&lt;/p&gt;&#xA;" OwnerUserId="16900" LastActivityDate="2018-03-09T19:02:58.693" Title="Procedure to identify characteristic properties of unknown functions in a DAE model" Tags="|optimization|error-estimation|differential-equations|optimal-control|dae|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29002" PostTypeId="1" AcceptedAnswerId="29003" CreationDate="2018-03-09T23:36:30.230" Score="0" ViewCount="71" Body="&lt;p&gt;I have a triangulated surface that I want to work with. In order to optimize certain quantities related to this surface I want to find all points which are accessible from a given direction. To be more precise, let's denote by $S_h$ the surface and by $B$ the body enclosed by $S_h$. Given a direction $\vec d \in \Bbb{R}^3$ I want to find all points $x \in S_h$ such that the semi-line starting at $x$ going in the direction of $\vec d$ does not intersect $S_h$ a second time. An analogue way of saying this is given a direction $\vec d$ I want to find all the points which can be &quot;seen&quot; looking at $S_h$ from that direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the feeling that this could be something which is known in the field of computational geometry, but maybe I don't know the right terminology in order to find relevant references.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there an algorithm which is not too costly to implement in order to find all &quot;visible points&quot; from a certain direction? Do you know references dealing with efficient algorithmical implementation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2018-03-10T05:36:04.573" Title="Detect all &quot;visible&quot; points on a triangulated surface" Tags="|computational-geometry|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29003" PostTypeId="2" ParentId="29002" CreationDate="2018-03-10T00:30:25.420" Score="3" Body="&lt;p&gt;As far as I understand, you are looking at some variation of ray tracing algorithm. Such problems will be very common for geometrical optics to determine the direct line of sight. Example:visible surface ray tracing (ray casting)&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-03-10T05:36:04.573" LastActivityDate="2018-03-10T05:36:04.573" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29004" PostTypeId="1" CreationDate="2018-03-10T02:07:47.893" Score="0" ViewCount="60" Body="&lt;p&gt;Since the permittivity of any material is usually complex function of temperature, frequency, density, etc. I was wondering if it is possible to use a dynamic permittivity which changes as a function of time using the fdtd method. This would also enable to simulate moving objects. Any hints, keywords, etc.?&lt;/p&gt;&#xA;" OwnerUserId="21797" LastEditorUserId="20688" LastEditDate="2018-03-12T14:57:17.260" LastActivityDate="2018-03-12T14:57:17.260" Title="Finite difference time domain and dynamic permittivity" Tags="|finite-difference|electromagnetism|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29005" PostTypeId="1" AcceptedAnswerId="29010" CreationDate="2018-03-10T04:20:12.143" Score="1" ViewCount="1328" Body="&lt;p&gt;In the context of the solution of advection-diffusion equations by finite volume method, many numerical schemes, papers and book chapters are dedicated to address the numerical diffusion and/or numerical dispersion that comes from the discretization of the advection term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understand it correctly, the discretization of the diffusion term also creates numerical diffusion and/or dispersion. However, given the lack of literature about it, it seems not to be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is that so?&lt;/p&gt;&#xA;" OwnerUserId="18932" LastActivityDate="2018-03-14T12:49:08.913" Title="Don't we care about the numerical diffusion in the diffusion term?" Tags="|finite-volume|discretization|advection-diffusion|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29006" PostTypeId="2" ParentId="29005" CreationDate="2018-03-10T11:38:15.180" Score="0" Body="&lt;p&gt;What I know is that when discretizing ad-/convective-diffusive equations, this problem (numerical diffusion) faces both of the ad-/convection term and the diffusion term due to the numerical approximations when computing the gradient. Therefore, the numerical diffusion is a numerical/truncation error and is a function of a few parameters, mainly the grid size and the alignment of the streamlines with the grids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tend to think that &quot;solving&quot; this problem (numerical diffusion) for the ad-/convection term and the diffusion term is the same. Note that higher order approximations have less numerical diffusion. However, they have numerical oscillations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am sure someone else would answer your question better than I did. I am still a graduate student in CFD.&lt;/p&gt;&#xA;" OwnerUserId="22411" LastEditorUserId="22411" LastEditDate="2018-03-10T13:45:19.143" LastActivityDate="2018-03-10T13:45:19.143" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29007" PostTypeId="2" ParentId="29005" CreationDate="2018-03-10T11:51:56.170" Score="0" Body="&lt;p&gt;&quot;Caring&quot; is a modeling question. Whether you care depends on the granularity of your model, the required precision, and the question you're trying to answer. Of course methods can always be more precise, but that's not the purpose of a numerical approximation. A numerical approximation is a fast calculation that's a good enough solution of your equation to analyze your results, so there is no general answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to ask yourself whether the amount of numerical diffusion due to your chosen $dx$, $dt$, spatial discretization methods and order, etc. is an appropriate tradeoff in terms of runtime and your development time, and whether the amount of error that is numerically introduced does not interfere with your modeling conclusions.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-03-10T11:51:56.170" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29008" PostTypeId="2" ParentId="29005" CreationDate="2018-03-10T13:35:09.020" Score="1" Body="&lt;p&gt;Numerical diffusion is not a (big) problem in diffusive equation solvers because it is there in your model. If you had no diffusion in your model, say Euler's equations or nondiffusive Shallow Water equation or any other hyperbolic equation, then numerical diffusion becomes an issue because it denatures the numerical solution by giving (or removing from) it properties that the exact solution hasn't (or has). For example, an exact solution that forms shocks will be approximated by a smeared out numerical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a mathematical point of view, the minimum we require from a numerical method is that a sequence of numerical solutions corresponding to a sequence of ever finer grids (or meshes) converges (in some sense) to the exact solution. If you have diffusion in the exact model, the exact solution is smooth and there are numerous convergence theorems that guarantee the convergence of the numerical method.  However if there is diffusion in the exact model, its solutions are more singular (they can have shocks for example, or lack even a first derivative and must be interpreted in some generalized sense). The few convergence results are restricted to special situations and the convergence rates are those dictated by the smoothness of the exact solution (which is low). And if there is too much diffusion then this convergence is further slowed down (and in some cases completely obliterated).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the reason we don't need to worry (too much) about numerical diffusion  for convection-diffusion equations is that the solution begin smooth, it is easier to approximate with smooth numerical solutions (where smoothness of a discrete function can be measured by using discrete smooth norms).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerical dispersion is a different story, which I'm not really entitled to comment about, but because your equation is not dispersive it might be important to think about it.  However, generally speaking numerical dispersion is much less of a problem when compared to numerical diffusion.&lt;/p&gt;&#xA;" OwnerUserId="19120" LastEditorUserId="19120" LastEditDate="2018-03-10T13:41:33.213" LastActivityDate="2018-03-10T13:41:33.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29009" PostTypeId="2" ParentId="28744" CreationDate="2018-03-10T13:40:54.347" Score="1" Body="&lt;p&gt;Let me try to answer your question concerning the transport&#xA;equation from a different perspective.&#xA;Consider the following initial-value problem.&#xA;We search for a function $v(x,t)$ such that&#xA;\begin{align*}&#xA;  v_t + v_x &amp;amp;= 0 \\&#xA;  v(x, 0)     &amp;amp;= f(x)&#xA;\end{align*}&#xA;for $t \ge 0$ and $x \in \mathbb{R}$.&#xA;The solution of this PDE can be approximated using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Upwind_scheme&quot; rel=&quot;nofollow noreferrer&quot;&gt;upwind scheme&lt;/a&gt;,&#xA;which can be derived as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\Delta t &amp;gt; 0$ and $\Delta x &amp;gt; 0$ be two small real numbers.&#xA;We can use that&#xA;$$&#xA;  v_t(x,t) \approx \frac{v(x, t+\Delta t) - v(x, t)}{\Delta t}&#xA;  \quad \text{and} \quad&#xA;  v_x(x,t) \approx \frac{v(x, t) - v(x - \Delta x, t)}{\Delta x}&#xA;$$&#xA;to find an approximation to the solution.&#xA;Replacing the partial derivatives in the original equation, we find that&#xA;$$&#xA;  \frac{v(x, t+\Delta t) - v(x, t)}{\Delta t} +&#xA;  \frac{v(x, t) - v(x - \Delta x, t)}{\Delta x}&#xA;  = 0&#xA;$$&#xA;approximates the PDE. Solving this equation for $v(x, t+\Delta t)$&#xA;gives&#xA;$$&#xA;  v(x, t+\Delta t)&#xA;  = v(x, t) + \frac{\Delta t}{\Delta x}(v(x, t) - v(x - \Delta x, t))&#xA;  \,.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to turn this approximation into an algorithm,&#xA;consider the grid&#xA;$$&#xA;  \mathcal{G}&#xA;  := \{ (k\, \Delta x, n\, \Delta t) : k \in \mathbb{Z}, n \in \mathbb{N}_0 \}&#xA;  \,,&#xA;$$&#xA;and denote $u_k^n$ as the approximation to the value of&#xA;$v(k\, \Delta x, n\, \Delta t)$. Then,&#xA;\begin{align*}&#xA;  u_{k}^0 &amp;amp;= f(k\,\Delta x) \\&#xA;  u_{k}^{n+1} &amp;amp;= u_k^n + \frac{\Delta t}{\Delta x}(u_k^n - u_{k-1}^n)&#xA;\end{align*}&#xA;is the upwind scheme for solving the transport equation. Which is a convergent&#xA;scheme in case $\Delta t$ is not too large in comparison to $\Delta x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, that the grid $\mathcal{G}$ has infinitely&#xA;many points and thus is not suitable for a computer implementation. However,&#xA;if we take a closer look, we will realize that we actually only need a finite&#xA;subset of $\mathcal{G}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we would like to compute the value $u_k^n$ using the upwind scheme. We&#xA;see that to compute $u_k^n$,&#xA;we need to compute the values of $u_{k-1}^{n-1}$ and $u_k^{n-1}$, first.&#xA;Using the same reasoning, we see that to compute these two values, we need the&#xA;values of $u_{k-2}^{n-2}$, $u_{k-1}^{n-2}$, and $u_{k}^{n-2}$. We can repeat&#xA;this steps until we finally reach $n = 0$, where we can use the values given by the&#xA;function $f$.&#xA;Hence, to compute the value of $u_k^n$, we need the values given in the&#xA;following diagramm:&#xA;$$&#xA;  \newcommand{\iddots}{\kern3mu\raise1mu{.}\kern3mu\raise6mu{.}\kern3mu\raise12mu{.}}&#xA;  \begin{matrix}&#xA;  &amp;amp;&amp;amp;&amp;amp;&amp;amp; u_k^n \\&#xA;  &amp;amp;&amp;amp;&amp;amp; u_{k-1}^{n-1} &amp;amp; u_{k}^{n-1} \\&#xA;  &amp;amp;&amp;amp; u_{k-2}^{n-2} &amp;amp; u_{k-1}^{n-2} &amp;amp; u_{k}^{n-2} \\&#xA;  &amp;amp; \iddots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots \\&#xA;  u_{k-n}^0 &amp;amp; \dots &amp;amp; u_{k-2}^{0} &amp;amp; u_{k-1}^{0} &amp;amp; u_{k}^{0}&#xA;  \end{matrix}&#xA;$$&#xA;The triangle shown above is the so called &lt;em&gt;numerical domain of dependence&lt;/em&gt;&#xA;of the point $u_k^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you can see is that to compute the point $u_k^n$, you just need to&#xA;compute the values in this triangular subset of the grid. Hence, no additional&#xA;boundary conditions are needed. In case you want to compute more than one&#xA;point, your domain of dependence will be trapezoidal, as can be seen easily.&#xA;Note, that for other PDEs these shapes will usually be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for this behavior is that the transport equation is a hyperbolic&#xA;PDE, and thus the information of the initial values are only transported with&#xA;finite speed, in contrast to, e.g., the heat equation where the speed of&#xA;transport of information is infinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that for the scheme to actually converge to the true solution, the&#xA;domain of dependence of the numerical scheme has to include the domain of&#xA;dependence of the PDE, otherwise the scheme will diverge.&#xA;This is the so called&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&quot; rel=&quot;nofollow noreferrer&quot;&gt;Courant–Friedrichs–Lewy (CFL) condition&lt;/a&gt;, which&#xA;explains the restriction on the step size $\Delta t$,&#xA;mentioned earlier.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2018-03-10T13:40:54.347" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29010" PostTypeId="2" ParentId="29005" CreationDate="2018-03-10T15:25:43.590" Score="7" Body="&lt;p&gt;We have the following problem:&#xA;$$\frac{\partial u}{\partial t}+v\color{red}{\frac{\partial u}{\partial x}}-\nu\color{blue}{\frac{\partial^2u}{\partial x^2}}=0 \tag{*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function $u$ may represent for example the concentration that propagates at velocity $v&amp;gt;0$ and disperses in a medium with viscosity $\nu&amp;gt;0$. Since only we are discussing how terms are discretised initial and boundary conditions are unnecesary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine we discretise the equation $(*)$ according to the following scheme:&#xA;$$\color{red}{\frac{\partial u}{\partial x} = \frac{u_i-u_{i-1}}{\Delta x}}$$&#xA;$$\color{blue}{\frac{\partial^2 u}{\partial x^2}\approx \frac{u_{i-1}-2u_i+u_{i+1}}{\Delta x^2}}$$&#xA;Giving the following numerical approximation:&#xA;$$\frac{\partial u}{\partial t}+v\color{red}{\frac{u_i-u_{i-1}}{\Delta x}}-\nu\color{blue}{\frac{u_{i-1}-2u_i+u_{i+1}}{\Delta x^2}}=0 \tag{**}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we reverse the transformation, to find what equation $(**)$ solves by means of taylor expansion:&#xA;$$u_{i\pm 1}=u_i\pm\frac{\partial u}{\partial x}\Delta x+\frac{1}{2}\frac{\partial^2u}{\partial x^2}\Delta x^2\pm\frac{1}{6}\frac{\partial ^3 u}{\partial x^3}\Delta x^3+\mathcal{O}(\Delta x^4)$$&#xA;we find:&#xA;$$\frac{\partial u}{\partial t}+v\left[\color{red}{\frac{\partial u}{\partial x}-\frac{1}{2}\frac{\partial^2 u}{\partial x^2}\Delta x+\mathcal{O}(\Delta x^2)}\right]-\nu\left[\color{blue}{\frac{\partial^2u}{\partial x^2}+\mathcal{O}(\Delta x^2)}\right]=0  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can say that when using the scheme $(**)$ we approximate the following equation exactly (up to an error $\mathcal{O}(\Delta x^2)$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t}+v\color{red}{\frac{\partial u}{\partial x}}-\left(\nu+\color{red}{\frac{v\Delta x}{2}}\right)\color{blue}{\frac{\partial^2u}{\partial x^2}}=\color{blue}{\mathcal{O}(\Delta x^2)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The added viscosity is then $v\Delta x/2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would have used a centred scheme for convective term , you would have solved equation $(*)$ exactly up to second order (check it!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the main reason of the fact that books does not stop to describe the added viscosity from the discretisation of the diffusive term (the leading one is the added by convective terms for stable schemes, e.g. upwind).&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="18932" LastEditDate="2018-03-12T14:26:45.857" LastActivityDate="2018-03-12T14:26:45.857" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29011" PostTypeId="2" ParentId="28964" CreationDate="2018-03-10T16:03:25.150" Score="4" Body="&lt;p&gt;In terms of computational effort, it is useless. I mean, you are still having a nonlinear problem with a bigger Jacobian (which is the worst part to be computed quickly). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For next thoughts: dense problems are always worse that sparse ones. Try to solve a sparse system and compare it with the time needed to solve a dense one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convergence does depend on the function that is going to be solved and the used scheme for that purspose. For example, the idea you proposed may be useful if fixed point iteration is used, I mean, you need that the Jacobian largest eigenvalue is less than unity for the scheme to be convergent. This can be achieved with smart arrangements of the function to be solved.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-03-10T16:03:25.150" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29012" PostTypeId="1" AcceptedAnswerId="29074" CreationDate="2018-03-10T20:38:42.020" Score="1" ViewCount="596" Body="&lt;p&gt;I would like to solve&#xA;$$\alpha u_{tt} = -\nabla^2u$$&#xA;with $\frac{\partial u}{\partial n} = 1$. On using a Galerkin approximation I obtain&#xA;$$M\ddot{c}=\frac{1}{\alpha}(Dc+b)$$&#xA;where $M$ is the mass matrix and and $D$ is the stiffness matrix. I note that I can write this in the form&#xA;$$\dot{U} = ZU+B$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $U = [c, \dot{c}]^T$ and $Z$ has off diagonal blocks top right: $I$ and bottom left $M^{-1}D$ and $B$ has only non-trivial entries in the &quot;second half&quot; of the vector given by $M^{-1}b$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can go ahead and use in implicit finite difference scheme on this problem then.(?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know my explanation is sparse but I am hoping that someone who might have solved this before can confirm whether this is a viable method. My solution is very &quot;unstable&quot; looking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I am being really naive using this method. Are there simple methods to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="27115" LastEditorUserId="27115" LastEditDate="2018-03-10T21:49:20.497" LastActivityDate="2018-03-18T19:58:07.060" Title="How would one solve the wave equation using the finite element method?" Tags="|finite-element|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29013" PostTypeId="2" ParentId="7574" CreationDate="2018-03-11T08:34:17.413" Score="1" Body="&lt;p&gt;Since the question was asked a long time ago, I would give a detailed overview answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first question is how the assembled matrix $A$ will be used later, as the construction of the matrix itself is rarely a goal. In the question, it is clearly defined that one needs to have a dense matrix in the end of the day. However, for some purposes, having a dense matrix instead of its factorization into several matrices might be detrimental and should be avoided. The simplest example would be a solution of the system of linear equations using an iterative solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if the construction of the dense matrix is unavoidable and\or required by the further algorithms you have several options.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As clipper pointed out, the entries of the dense matrix $A$ can be manually computed column-by-column by applying matrix-vector products to the columns of the identity matrix: $D(B(D^H(CI_n)))$, where $I_n$ is the $n$th column of the identity matrix. In such a way, you will be able to avoid forming dense intermediate results and take advantage of the sparsity of $D$, $B$, and $C$.&lt;/li&gt;&#xA;&lt;li&gt;Depending on the tools you are using, matrix library can already support sparse matrix-matrix products. For example, a commonly used template-based library Eigen will naturally support such products.&lt;/li&gt;&#xA;&lt;li&gt;Explicitly use some SparseBLAS implementation. For example, Intel MKL now supports subroutines &lt;code&gt;mkl_?csrmultcsr&lt;/code&gt; and &lt;code&gt;mkl_?csrmultd&lt;/code&gt; that compute the product of two sparse matrices and store the results in a sparse CSR or dense format, respectively.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using some SparseBLAS implementation has an advantage of a well-optimized and debugged code for matrix operations, already built-in potential for parallelization. However, benchmarking it against an MVP-based solution would be required for the sparsity patterns and problem sizes you are interested in. Using Eigen-like library is the easiest options, though you will lack a lot of control and can lose some performance, especially if for those particular functions Eigen does not use external BLAS/LAPACK engines and relies only on its own implementations.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-11T08:34:17.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29014" PostTypeId="2" ParentId="27584" CreationDate="2018-03-11T20:53:16.050" Score="0" Body="&lt;p&gt;I have answered a &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6770/zgetrf-and-zgetrs-from-mkl-zgetrf-fails-and-still-zgetrs-works?rq=1&quot;&gt;similar question&lt;/a&gt; when a failed &lt;code&gt;zgetrf&lt;/code&gt; will be followed by a successful &lt;code&gt;zgetrs&lt;/code&gt;. In this question, the situation is similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To call &lt;code&gt;zgetri&lt;/code&gt;, you have to call &lt;code&gt;zgetrf&lt;/code&gt; first. Therefore, if &lt;code&gt;zgetrf&lt;/code&gt; failed (in a sense that $U_{2,2}=0$) in the first place, there is no point in calling &lt;code&gt;zgetri&lt;/code&gt; and hope for a reliable result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why &lt;code&gt;zgetri&lt;/code&gt; does not result in an explicit error itself is another question, and probably will depend on the particular implementation of LAPACK/BLAS library. But, to sum it up, since you got an error in the first step of your two-step process of obtaining the inverse of the matrix, you certainly cannot trust the obtained inverse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The strategy how to understand what is actually happening with the matrix is discussed in the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/6770/zgetrf-and-zgetrs-from-mkl-zgetrf-fails-and-still-zgetrs-works?rq=1&quot;&gt;aforementioned question&lt;/a&gt; but boils down to the analysis of the spectrum of your matrix, its singular values, and condition numbers, followed by the investigation of the physics where you matrix is coming from originally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are interested in the inverse of the matrix (as you noted in the comments) and your matrices are &quot;problematic&quot;, I would suggest using an SVD-route process to obtain the inverse as opposed to LU-route. It is slower; however, you will be able to see how the singular values behave in the first place and identify problems earlier and see them explicitly.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-03-12T03:47:09.310" LastActivityDate="2018-03-12T03:47:09.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29016" PostTypeId="2" ParentId="20360" CreationDate="2018-03-12T02:08:05.070" Score="2" Body="&lt;p&gt;I don't know about the existence of Matlab interface for original Fortran version of MINPACK-2. However, &lt;a href=&quot;https://www.cranfield.ac.uk/people/dr-shaun-forth-806215&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dr. Shaun Forth&lt;/a&gt; at some point was working on Matlab implementation of Minpack-2 test problem suite, including the library itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a couple of his relevant conference papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Forth S, &quot;A Matlab Implementation of the Minpack-2 Test Problem Collection,&quot; in &lt;em&gt;18th European W. Automatic Differentiation&lt;/em&gt;, Paderborn, Nov. 2015.&lt;/li&gt;&#xA;&lt;li&gt;Forth S, &lt;a href=&quot;http://www.autodiff.org/ad16/Oral/Forth_Matlab.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;A Matlab implementation of the Minpack-2 Test Problem Collection,&quot;&lt;/a&gt; in &lt;em&gt;7th Int. Conf. Algorithmic Differentiation&lt;/em&gt;, Oxford, Sept. 2016.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-12T02:08:05.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29017" PostTypeId="1" CreationDate="2018-03-12T05:00:20.263" Score="0" ViewCount="69" Body="&lt;p&gt;When solving a reduced KKT system of a nonlinear (and nonconvex) constrained program after eliminating slack and dual variables, how do we actually take the next step in a primal-dual method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, following notation from &lt;a href=&quot;http://www.springer.com/us/book/9780387303031&quot; rel=&quot;nofollow noreferrer&quot;&gt;NW&lt;/a&gt;, if the original nonlinear system is like (19.12)&#xA;\begin{align}&#xA;\begin{bmatrix}&#xA;\nabla_{xx}^2 L &amp;amp; 0 &amp;amp; A_E(x)^T &amp;amp; A_I(x)^T \\&#xA;0 &amp;amp; \Sigma &amp;amp; 0 &amp;amp; -I \\&#xA;A_E(x) &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;A_I(x) &amp;amp; -I &amp;amp; 0 &amp;amp; 0 \\&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;p_x \\ p_s \\ -p_y \\ -p_z&#xA;\end{bmatrix}&#xA;= &#xA;-\begin{bmatrix}&#xA;\nabla f(x) - A_E(x)^Ty - A_I(x)^T z \\ z - \mu S^{-1}e \\ c_E(x) \\ c_I(x) - s&#xA;\end{bmatrix},&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then I see how a solution gives us a way to update $x,s,y,z$. However, if we solve a reduced system of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\begin{bmatrix}&#xA;\nabla_{xx}^2 L + A_I(x)^T\Sigma A_I(x) &amp;amp; A_E(x)^T \\&#xA;A_E(x) &amp;amp; 0&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;p_x \\ -p_y&#xA;\end{bmatrix}&#xA;=&#xA;\text{?}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then (1) what is the RHS; and (2) how do we update $s,z$?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; Can I have some help on the details for how we eliminate variables in moving from the larger system to the smaller system?&lt;/p&gt;&#xA;" OwnerUserId="22292" LastEditorUserId="22292" LastEditDate="2018-03-13T03:17:17.237" LastActivityDate="2018-03-13T03:17:17.237" Title="reduced system: primal-dual interior point method for nonconvex constrained problem" Tags="|optimization|constrained-optimization|nonlinear-programming|nonconvex|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29019" PostTypeId="2" ParentId="29004" CreationDate="2018-03-12T07:06:23.683" Score="1" Body="&lt;p&gt;FDTD can be applied for modeling of objects with time-varying coefficients. Depending on the type of variation, different techniques can be applied.&#xA;Commonly, in computational EM the following situations are usually considered:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;linear dispersion: dielectric permittivity ($\epsilon$) and magnetic permeability ($\mu$) of the material vary with frequency $\omega$: $\epsilon(\omega)$, $\mu(\omega)$&lt;/li&gt;&#xA;&lt;li&gt;nonlinearity dispersion: $\epsilon(\omega,E,H)$, $\mu(\omega,E,H)$, dependence on the field intensity&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In addition, your question implies some multiphysics component (temperature), which, depending on the model can be considered as a simple contribution to material properties nonlinearity, as well as a whole additional physics to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My go-to reference for FTDT in computational EM is &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Taflove, &quot;&lt;a href=&quot;https://isbnsearch.org/isbn/9781580538329&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Electromagnetics&lt;/a&gt;&quot;, Artech House, 2005. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Chapter 9 &quot;Dispersive and Nonlinear Materials&quot; is talking about the relevant techniques that should be of interest to you.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-12T07:06:23.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29020" PostTypeId="2" ParentId="28963" CreationDate="2018-03-12T07:29:45.600" Score="2" Body="&lt;p&gt;I have found &lt;a href=&quot;https://documen.tician.de/pycuda/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pycuda&lt;/a&gt; particularly useful as wrapper for cuda in python.&#xA;Especially the section on metaprogramming is useful if you are interested in building more sophisticated frameworks.&#xA;It's a mature package, there is an active mailing list and the developer gives very useful advice.&lt;/p&gt;&#xA;" OwnerUserId="18017" LastActivityDate="2018-03-12T07:29:45.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29021" PostTypeId="1" AcceptedAnswerId="29022" CreationDate="2018-03-12T09:17:17.380" Score="1" ViewCount="227" Body="&lt;p&gt;Consider a linear system, $Ax=b$ where $A$ is the coefficient matrix, $x$ is the unknown vector of variables, and $b$ is a vector of constants in which all entries are same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to find out relative values of $x$ without actually solving the system of equations? In other words, can we say by just looking at $A$ that $x_1$ will be greater than $x_2$ because the sum of entries of first row is greater than the sum of entries of the 2nd row of $A$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have intentionally made the assumption that all entries of $b$ are same, so that they have no effect on our approximate solution.&lt;/p&gt;&#xA;" OwnerDisplayName="user19180" LastEditorUserId="17294" LastEditDate="2018-03-12T09:58:11.490" LastActivityDate="2018-03-12T10:32:20.563" Title="Approximation to Solution of a Linear System of Equations" Tags="|linear-algebra|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29022" PostTypeId="2" ParentId="29021" CreationDate="2018-03-12T10:32:20.563" Score="1" Body="&lt;p&gt;A counterexample of what you propose:&#xA;$$A = \left[\begin{matrix} 1 &amp;amp; 1&amp;amp;0\\0 &amp;amp; 1&amp;amp; 0\\ 0 &amp;amp; 0 &amp;amp;1\end{matrix}\right], \quad b=\left[\begin{matrix}1\\1\\1\end{matrix}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sum of he first row of $A$ is greater than the rest and $x_1=0&amp;lt;x_2=x_3=1$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-03-12T10:32:20.563" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29023" PostTypeId="2" ParentId="29017" CreationDate="2018-03-12T11:00:25.853" Score="1" Body="&lt;p&gt;Once you have $p_x,p_y$, you can use the second and fourth of the block system you show in your question to compute the updates $p_s,p_z$. They satisfy the equations&#xA;\begin{align}&#xA;\begin{bmatrix}&#xA;\Sigma &amp;amp; -I \\&#xA;-I &amp;amp; 0 \\&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;p_s \\ -p_z&#xA;\end{bmatrix}&#xA;= &#xA;\begin{bmatrix}&#xA;z - \mu S^{-1}e \\ c_I(x) - s - A_I(x)p_x&#xA;\end{bmatrix}.&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2018-03-13T02:58:20.533" LastActivityDate="2018-03-13T02:58:20.533" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29024" PostTypeId="1" CreationDate="2018-03-12T18:27:46.670" Score="1" ViewCount="487" Body="&lt;p&gt;This question might be so obvious and trivial that I'm having a hard time googling it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a multivariate root finding problem that I'm trying to solve in C# and the library that I'm trying to use is alglib.  That package doesn't have any dedicated root solvers in it, but has Levenberg-Marquardt.  To do root finding, do I just take the squared vector result and feed it to the LM solver?  Then simply check the final result against the zero vector to ensure that it found the global minimum I'm looking for?&lt;/p&gt;&#xA;" OwnerUserId="27129" LastActivityDate="2018-03-12T18:38:35.830" Title="Levenberg-Marquardt for root-finding: just square the function?" Tags="|optimization|rootfinding|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29027" PostTypeId="1" CreationDate="2018-03-13T05:11:47.050" Score="1" ViewCount="860" Body="&lt;p&gt;As far as I have seen, solving problems of nonlinear elasticity using the finite element method proceeds by linearizing, either around the initial configuration (total Lagrangian approach) or around the most recently solved configuration (updated Lagrangian approach).  This produces a system of linear equations, and represents an example of the Newton-Raphson method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about possible generalizations to other methods of solving systems of nonlinear equations.  I realize that for engineering problems, the linearization may not cause any problems, because the goal is often to study deformations starting from a known, unstressed configuration, which can then be gradually loaded.  In other words, there is a good small parameter, so linearization is fine.  However, I'm interested in the case that external forces (loadings) depend on the deformation, and the dependence can exhibit strong nonlinearities.  In this case, the global solution may not be anywhere near the undeformed configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that attempts to address this problem are known as &quot;globalization strategies&quot;, and I'm interested in the application of these to nonlinear elasticity.  I know about methods like arc-length control, but I'm interested in other methods such as full multigrid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know enough about the lingo or the literature to even search the literature properly - so far my naive attempts have not yielded anything.  I also don't know good examples of problems with strong nonlinearities that might be solved with finite element - maybe Navier-Stokes?  I don't even see how to properly formulate the problem as a system of nonlinear equations - do I just numerically integrate the principal of virtual work?&lt;/p&gt;&#xA;" OwnerUserId="25235" LastActivityDate="2018-03-13T05:11:47.050" Title="Alternatives to Newton-Raphson for nonlinear elasticity via finite element" Tags="|finite-element|nonlinear-equations|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29028" PostTypeId="1" CreationDate="2018-03-13T09:14:26.900" Score="4" ViewCount="163" Body="&lt;p&gt;Suppose that we have a linear system of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax=b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is a $3 \times 3$ matrix and $x$ and $b$ are $3$-vectors. Let $y$ denote the solution of this system of equations. I want to change  matrix $A$ such that the new solution is vector $z$ in which &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$z_1 &amp;gt; y_1, \qquad z_2 = y_2, \qquad z_3 &amp;lt; y_3$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a systematic way to achieve this? In other words, I want a systematic way of finding out what changes I should introduce in matrix $A$ such that &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;some entries of the new solution $z$ are greater than the corresponding entries of the old solution $y$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;other entries of the new solution $z$ are equal to the corresponding entries of the old solution $y$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;some other entries of the new solution $z$ are less than the corresponding entries of the old solution $y$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there a method or technique to achieve this? What is it called? Thank you.&lt;/p&gt;&#xA;" OwnerDisplayName="user19180" LastEditorUserId="20417" LastEditDate="2018-07-11T15:25:56.153" LastActivityDate="2018-11-09T00:02:45.547" Title="Modifying solution of system of linear equations" Tags="|linear-algebra|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29030" PostTypeId="2" ParentId="29028" CreationDate="2018-03-13T10:16:33.753" Score="0" Body="&lt;p&gt;If you want to, given a solution, obtain the matrix $A$, for the system $Ax=b$. You can do:&#xA;$$Ax=[a_1,a_2,a_3]x=x_1a_1+x_2a_2+x_3a_3=b$$&#xA;Being $a_i$ the columns of the matrix A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Choose linearly independent vectors $a_1$ and $a_2$ and compute $a_3$ as follows:&#xA;$$a_3=\frac{1}{x_3}b-\frac{x_1}{x_3}a_1-\frac{x_2}{x_3}a_2$$&#xA;Make sure that the resulting matrix $A$ is nonsingular if you want a unique solution, i.e. choose for example $a_1$ and $a_2$ perpendicular to $b$.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-03-13T10:16:33.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29031" PostTypeId="2" ParentId="28647" CreationDate="2018-03-13T12:10:38.997" Score="3" Body="&lt;p&gt;Regarding simulation:&#xA;There are several commercial solvers that might be helpful.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A very popular FDTD Maxwell equation solvers for nanophotonics is &lt;a href=&quot;https://www.lumerical.com/tcad-products/fdtd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lumerical&lt;/a&gt;. It features opto-thermal and &lt;em&gt;liquid crystal&lt;/em&gt; (which I would say is another keyword you should consider) simulations that should be very relevant.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another go-to multiphysics solver would be COMSOL. Searching for &lt;em&gt;electrohydrodynamics&lt;/em&gt; and &lt;em&gt;magnetohydrodynamics&lt;/em&gt; will results in a lot of results. Several relevant papers for COMSOL-based modeling:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;M. G. Sailer, B. J. Kirby, &quot;&lt;a href=&quot;https://www.comsol.com/paper/computational-simulation-of-electrohydrodynamic-systems-pertaining-to-micro-and--5400&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational simulation of electrohydrodynamic systems pertaining to micro and nano scale fluid flow phenomenon&lt;/a&gt;,&quot; in &lt;em&gt;Proc. COMSOL Conf.&lt;/em&gt;, Boston, MA, 2008.&lt;/li&gt;&#xA;&lt;li&gt;A. Mulye, S. Potnis, &quot;&lt;a href=&quot;https://www.comsol.com/paper/electrohydrodynamic-micropump-modeling-for-performance-optimization-15883&quot; rel=&quot;nofollow noreferrer&quot;&gt;Electrohydrodynamic micropump modeling for&#xA;performance optimization&lt;/a&gt;,&quot; in &lt;em&gt;Proc. COMSOL Conf.&lt;/em&gt;, Bangalore, India, 2013.&lt;/li&gt;&#xA;&lt;li&gt;N. Kleinknecht, &quot;&lt;a href=&quot;https://www.comsol.com/paper/high-frequency-magnetohydrodynamic-calculations-in-comsol-10670&quot; rel=&quot;nofollow noreferrer&quot;&gt;High frequency magnetohydrodynamic calculations in COMSOL&lt;/a&gt;,&quot; in &lt;em&gt;Proc. COMSOL Conf.&lt;/em&gt;, Stuttgart, Germany, 2011.&lt;/li&gt;&#xA;&lt;li&gt;Y. Yan, S. Smolentsev, M. Abdou, &quot;&lt;a href=&quot;https://www.comsol.com/paper/validation-of-comsol-multiphysics-for-magnetohydrodynamics-mhd-flows-in-fusion-a-50552&quot; rel=&quot;nofollow noreferrer&quot;&gt;Validation of COMSOL multiphysics for magnetohydrodynamics (MHD) flows in fusion applications&lt;/a&gt;,&quot; in &lt;em&gt;Proc. COMSOL Conf.&lt;/em&gt;, Boston, MA, 2017.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding numerical techniques themselves:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I recently stumbled upon the following paper (that was my main motivation even to write an answer to your question): E. Veerapaneni, &quot;&lt;a href=&quot;https://doi.org/10.1016/j.jcp.2016.08.052&quot; rel=&quot;nofollow noreferrer&quot;&gt;Integral equation methods for vesicle electrohydrodynamics in three dimensions&lt;/a&gt;,&quot; &lt;em&gt;J. Comp. Phys.&lt;/em&gt;, vol. 326, pp. 278-289, Dec. 2016. From its abstract: &lt;em&gt;The method uses a semi-implicit time-stepping scheme to overcome the numerical stiffness associated with the governing equations.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;And another paper that crossed my eyes several years ago that features numerical study in magnetohydrodynamics: M. Kinet, B. Knaepen, and S. Molokov, &quot;&lt;a href=&quot;https://link.aps.org/doi/10.1103/PhysRevLett.103.154501&quot; rel=&quot;nofollow noreferrer&quot;&gt;Instabilities and transition in magnetohydrodynamic flows in ducts with electrically conducting walls&lt;/a&gt;,&quot; &lt;em&gt;Phys. Rev. Lett.&lt;/em&gt;, vol. 103, 154501, Oct. 2009.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-13T12:10:38.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29032" PostTypeId="2" ParentId="8398" CreationDate="2018-03-13T13:02:46.767" Score="3" Body="&lt;p&gt;The following paper uses MapReduce for distributed computation of non-negative matrix factorization:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;J. Yin, L. Gao, Zh. Zhang, &quot;&lt;a href=&quot;https://doi.org/10.1007/978-3-662-44845-8_22&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scalable Nonnegative Matrix Factorization&#xA;with Block-wise Updates&lt;/a&gt;,&quot; in &lt;em&gt;Proc. Eu. Conf. Machine Learning Knowledge Discovery&lt;/em&gt;, Nancy, France, Sep. 2014, pp.337-352.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This paper describes an implementation and parallelization of the non-negative factorization and claims the results for matrices exceeding your specifications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding software libraries:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;python library &lt;a href=&quot;http://mloss.org/software/view/397/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIMFA&lt;/a&gt; supports both dense and sparse input (&lt;a href=&quot;http://www.jmlr.org/papers/volume13/zitnik12a/zitnik12a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;associated paper&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/renozao/NMF&quot; rel=&quot;nofollow noreferrer&quot;&gt;R-package NMF&lt;/a&gt; (no info on sparse that I was able to find quickly)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sites.google.com/site/nmftool/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NMF Matrix toolbox&lt;/a&gt; (v 1.4 added support for sparse input)&lt;/li&gt;&#xA;&lt;li&gt;high-performance library &lt;a href=&quot;https://smallk.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SmallK&lt;/a&gt;, supports both dense and sparse&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-13T13:02:46.767" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29033" PostTypeId="1" CreationDate="2018-03-13T14:16:14.147" Score="2" ViewCount="44" Body="&lt;p&gt;Imagine I want to compute the eigenvalues of an operator $\hat O$ defined on $ L^2(\mathbb{R})$, however using a properly scaled N-dimensional polynomial basis of $ L^2([-a,a]) $ which fulfills Dirichlet boundary conditions: $P_n(\pm a)=0$ (e.g. scaled Legendre polynomials), i.e. $ O_{i,j}=\langle P_i|\hat O|P_j\rangle$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I expect convergence to the exact result for $N, a\to\infty$?&lt;/p&gt;&#xA;" OwnerUserId="27139" LastEditorUserId="27139" LastEditDate="2018-03-13T15:58:49.580" LastActivityDate="2018-03-13T15:58:49.580" Title="Can I expect reasonable results when expanding in scaled Legendre polynomials?" Tags="|boundary-conditions|polynomials|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29034" PostTypeId="1" CreationDate="2018-03-13T14:41:20.647" Score="4" ViewCount="81" Body="&lt;p&gt;Typical finite volume methods are conservative, because fluxes (of e.g., mass or energy) are always between neighboring cells. Is the same generally true for finite element codes? Do I correctly assume that implicit time stepping would then be 'globally' conservative, except for flows through the domain boundaries?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Background information: The reason I'm asking is that colleagues who use the &lt;a href=&quot;https://www.comsol.com/plasma-module&quot; rel=&quot;nofollow noreferrer&quot;&gt;plasma module&lt;/a&gt; of Comsol do not observe charge conservation in their simulations. The equations are of the following form:&#xA;$$\partial_t n_1 + \nabla \cdot \vec{f} = S \, n_1\\&#xA;\partial_t n_2 = S \, n_1,$$&#xA;where $\vec{f}$ and $S$ non-linearly depend on other variables. The quantity $\int n_2 - n_1 dV$ is not conserved, but the conservation error shrinks for smaller time steps. I believe that this module internally works with $log(n)$ instead of $n$, where $n$ is the density, to avoid certain numerical problems.&lt;/p&gt;&#xA;" OwnerUserId="18662" LastEditorUserId="18662" LastEditDate="2018-03-13T15:41:40.637" LastActivityDate="2018-03-13T15:41:40.637" Title="Conservation in finite element codes" Tags="|finite-element|conservation|" AnswerCount="0" CommentCount="2" ClosedDate="2018-03-19T14:20:56.127" ContentLicense="CC BY-SA 3.0" />
  <row Id="29035" PostTypeId="2" ParentId="24187" CreationDate="2018-03-13T22:00:09.623" Score="4" Body="&lt;p&gt;A key to solving such problems is in the understanding of the definition of a matrix-matrix product. Without loss of generality, for square matrices $A,B\in\mathbb R^{n\times n}$, $C\in \mathbb R^{n\times n}, C=AB$ is defined as follows:&#xA;$$&#xA;C=\left[ &#xA;\begin{array}{cccc}&#xA;\sum\limits_{k=1}^{n}a_{1k}b_{k1} &amp;amp; \sum\limits_{k=1}^{n}a_{1k}b_{k2} &amp;amp; \ldots &amp;amp; \sum\limits_{k=1}^{n}a_{1k}b_{kn}\\&#xA;\sum\limits_{k=1}^{n}a_{2k}b_{k1} &amp;amp; \sum\limits_{k=1}^{n}a_{2k}b_{k2} &amp;amp; \ldots &amp;amp; \sum\limits_{k=1}^{n}a_{2k}b_{kn}\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots\\&#xA;\sum\limits_{k=1}^{n}a_{nk}b_{k1} &amp;amp; \sum\limits_{k=1}^{n}a_{nk}b_{k2} &amp;amp; \ldots &amp;amp; \sum\limits_{k=1}^{n}a_{nk}b_{kn}\\&#xA;\end{array}&#xA;\right],  \quad \text{or}\quad c_{vw}=\sum\limits_{k=1}^{n}a_{vk}b_{kw}&#xA;$$&#xA;where $a_{vw}$ denotes the $(v,w)$th element of matrix $A$.&#xA;Now, depending on the patterns\type\properties of $A$ and $B$, one can say a lot about the resultant product $C$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a particular case (ii) (sorry, some changes to your notation): $C=L^iL^j$, where $1\le i\le j&amp;lt;n$, ($L^i,L^j$ being lower triangular column-form matrices according to your definition):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;c_{vw}=\sum\limits_{k=1}^{n}\lambda^i_{vk}\lambda^j_{kw}=&#xA;\begin{cases}&#xA;1,\quad v=w \quad(\text{product of lower-}\Delta\text{matrices})\\&#xA;0,\quad w&amp;gt;v \quad(\text{product of lower-}\Delta\text{matrices})\\&#xA;\sum\limits_{k=1}^{n}\lambda^i_{vk}\lambda^j_{kw}=\lambda^i_{vw}\cdot 1+1\cdot\lambda^j_{vw}, \quad w&amp;lt;v&#xA;\end{cases}&#xA;$$&#xA;So, the resultant matrix $C$ is a lower triangular, where each ($v,w$)th entry under the main diagonal is a sum of $L^i$ and $L^j$ ($v,w$)th entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: &#xA;$$&#xA;C=L^1 L^2=\left[&#xA;\begin{array}{cccccc}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp;0 &amp;amp; \ldots &amp;amp; 0\\&#xA;\lambda^1_{21} &amp;amp; 1 &amp;amp; 0 &amp;amp; 0&amp;amp; \ldots &amp;amp; 0\\&#xA;\lambda^1_{31} &amp;amp; \lambda^2_{32} &amp;amp; 1 &amp;amp; 0 &amp;amp; \ldots &amp;amp; 0\\&#xA;\lambda^1_{41} &amp;amp; \lambda^2_{42} &amp;amp; 0 &amp;amp; 1  &amp;amp; \ldots &amp;amp; 0\\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; 0 \\&#xA;\lambda^1_{n1} &amp;amp; \lambda^2_{n2} &amp;amp; 0 &amp;amp; 0  &amp;amp; \ldots &amp;amp; 1\\&#xA;\end{array}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for case (i), when $i\neq j$, the resultant matrix $C$ can technically be named $L^{i,j}$, lower triangular &lt;em&gt;two-column-form matrix&lt;/em&gt;, where all non-zero sub-diagonal elements are contained in columns $i$ and $j$ only. Computational complexity &amp;amp; memory (if only nnz are stored): $\mathcal O(n)$ if the result has to be written into a separate matrix storage (or overwritten in one of $A$, $B$) or nothing if an explicit copy is not required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For case (ii), when $i\le j$, the computational complexity, and memory stays the same if a separate storage for $C$ is required. However, since $i$ can be equal to $j$, $i$th column of $C$ requires $\mathcal O(n)$ summations (1 per each subdiagonal element) anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can also derive exact flop count without Big-O notation by using $i$ and $j$.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-13T22:00:09.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29036" PostTypeId="2" ParentId="28775" CreationDate="2018-03-14T01:18:22.240" Score="1" Body="&lt;p&gt;As you mentioned above, the symmetry axis is treated in the same way that a general symmetry plane setting the ghost point values to their “mirror image” counterparts (negating the velocity). However, you have to be careful because you are using the geometric flux form of the Euler equations. You are &quot;redefining&quot; your governing equations multiplying them by the radial coordinate $r$. Your variables and fluxes at the ghost nodes where $r&amp;lt;0$ are being affected by this change in sign of your radial coordinate and so you might be imposing unphysical negative densities in those nodes. You can find more details in this paper: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S004579301500119X&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S004579301500119X&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I want to point out to you that should have $\alpha r^{\alpha-1}p$ instead of only $p$ in your pressure nozzling term on the right-hand side of your governing equations. Here is another paper that shows the derivation of the form of the Euler equations that you are currently using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/article/10.1007/s00193-017-0784-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/article/10.1007/s00193-017-0784-y&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26429" LastActivityDate="2018-03-14T01:18:22.240" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29037" PostTypeId="2" ParentId="21162" CreationDate="2018-03-14T04:03:43.677" Score="1" Body="&lt;p&gt;Though this question is about a particular functionality of an individual library, I stumbled upon the same problem some time ago.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, there is no ready matrix function in PETSc (3.8.3) that will do it for you right out-of-the-box. I was surprised not to find a solution back a couple of years ago and now, considering the existence of such functions as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecAbs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;VecAbs&lt;/code&gt;&lt;/a&gt; that will do exactly what you want, but for vectors only&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatImaginaryPart.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatImaginaryPart&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatRealPart.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatRealPart&lt;/code&gt;&lt;/a&gt;, and &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatConjugate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatConjugate&lt;/code&gt;&lt;/a&gt; which will perform $A_{ij}=\Re(A_{ij})$, $A_{ij}=\Im(A_{ij})$, and $A_{ij}=A^*_{ij}$, respectively.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Considering that element-wise Re, Im, and conjugation operations on matrices are supported, I would assume that all the necessary building blocks for the &lt;em&gt;MatAbs&lt;/em&gt; are there, but not formally coded up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/mat/interface/matrix.c.html#MatRealPart&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatRealPart&lt;/code&gt; subroutine source code&lt;/a&gt;, the following excerpt is doing the job&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(*mat-&amp;gt;ops-&amp;gt;realpart)(mat);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The list of supported and propagated operations on matrices is stored in &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/include/petscmat.h.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;include/petscmat.h&lt;/em&gt;&lt;/a&gt; in &lt;code&gt;MatOperation&lt;/code&gt; typedef. Technically, you can code up your own operation and propagate it to the datatypes you require (maybe, even by using custom user matrix &lt;code&gt;MatOperation&lt;/code&gt; and &lt;code&gt;MatShellSetOperation&lt;/code&gt;). However, the question is if such a development is worth the required time and pain.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-14T04:03:43.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29038" PostTypeId="1" CreationDate="2018-03-14T10:51:10.430" Score="1" ViewCount="431" Body="&lt;p&gt;Is there a direct way to use SLEPc from Matlab? I remember that in some old manuals there was some Matlab interface. However, in the last one, I cannot find any reference to this. For me, it would be really useful in the computation of eigenvalues for some large, sparse matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Matlab's &lt;code&gt;eigs&lt;/code&gt; does a pretty good job for matrices up to &lt;span class=&quot;math-container&quot;&gt;$12e6\times 12e6$&lt;/span&gt; (that I tested), but with a huge cost in RAM. I'm not sure if parallelization will consume as much or more...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any tutorials, packages which can make the connection between the two: Matlab and SLEPc? &lt;/p&gt;&#xA;" OwnerUserId="3548" LastEditorUserId="20688" LastEditDate="2019-05-19T06:18:21.377" LastActivityDate="2019-05-19T06:18:21.377" Title="Use SLEPc from Matlab" Tags="|matlab|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29039" PostTypeId="2" ParentId="29005" CreationDate="2018-03-14T12:49:08.913" Score="0" Body="&lt;p&gt;This answer is in complement of &lt;a href=&quot;https://scicomp.stackexchange.com/a/29010/18932&quot;&gt;HBR's answer&lt;/a&gt;, following the discussion we had in the comments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took from him the structure of it and colours so to keep consistency and make it easier for those who read both. Any mistakes are mine.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;We have the following problem:&#xA;$$\frac{\partial u}{\partial t}+v\color{red}{\frac{\partial u}{\partial x}}-\nu\color{blue}{\frac{\partial^2u}{\partial x^2}}=0 \tag{*}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;were $u$ is a quantity that diffuses with diffusion coefficient $\nu&amp;gt;0$ and is advected by velocity $v&amp;gt;0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine we discretise the equation $(*)$ according to the following scheme (upwind for the advection term, central difference for the diffusion term) on a 1D equispaced mesh:&#xA;$$\color{red}{\frac{\partial u}{\partial x} = \frac{u_i-u_{i-1}}{\Delta x}}$$&#xA;$$\color{blue}{\frac{\partial^2 u}{\partial x^2}\approx \frac{u_{i-1}-2u_i+u_{i+1}}{\Delta x^2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Giving the following numerical approximation:&#xA;$$\frac{\partial u}{\partial t}+v\color{red}{\frac{u_i-u_{i-1}}{\Delta x}}-\nu\color{blue}{\frac{u_{i-1}-2u_i+u_{i+1}}{\Delta x^2}}=0 \tag{**}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taylor expansion for $u_{i\pm 1}$ around $u_i$ is&#xA;$$u_{i\pm 1}  = &#xA;u_i&#xA;\pm \frac{\partial u}{\partial x}\Delta x&#xA;+\frac{1}{2}\frac{\partial^2u}{\partial x^2}\Delta x^2&#xA;\pm\frac{1}{6}\frac{\partial ^3 u}{\partial x^3}\Delta x^3&#xA;+ \frac{1}{24}\frac{\partial^4u}{\partial x^4}\Delta x^4&#xA;\pm\frac{1}{120}\frac{\partial^5 u}{\partial x^5}\Delta x^5&#xA;+\mathcal{O}(\Delta x^6)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we use this Taylor expansion in $(**)$ find out what equation $(**)$ solves, we find:&#xA;$$\frac{\partial u}{\partial t}+v&#xA;\left[\color{red}{&#xA;\frac{\partial u}{\partial x}&#xA;-\frac{1}{2}\frac{\partial^2 u}{\partial x^2}\Delta x&#xA;+\mathcal{O}(\Delta x^2)&#xA;}\right]&#xA;-\nu\left[\color{blue}{&#xA;\frac{\partial^2u}{\partial x^2}&#xA;+\frac{1}{12}\frac{\partial^4u}{\partial x^4}\Delta x^2&#xA;+\mathcal{O}(\Delta x^4)&#xA;}\right]=0  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can say that when using the scheme $(**)$ we approximate the following equation exactly (up to an error $\mathcal{O}(\Delta x^2)$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial u}{\partial t}+v\color{red}{\frac{\partial u}{\partial x}}-\left(\nu+\color{red}{\frac{v\Delta x}{2}} -&#xA;\color{blue}&#xA;{\frac{\nu\Delta x^2}{12}\frac{\partial^2}{\partial x^2}}&#xA;     \right)\color{blue}{\frac{\partial^2u}{\partial x^2}}=\color{red}{\mathcal{O}(\Delta x^2)}$$&#xA;where $\frac{\partial^2}{\partial x^2}$ is not a quantity, but an operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lowest order term introduced by the discretization of the advection term is the numerical diffusion $\frac{v\Delta x}{2}\frac{\partial^2u}{\partial x^2}$, which would be unimportant just if&#xA;$$&#xA;\color{red}&#xA;{\frac{v \Delta x}{2 \nu} \ll 1} \tag{***}&#xA;$$&#xA;The quantity $\frac{v \Delta x}{\nu}$ is known as the grid Péclet number. $(** $$*)$ is rarely true, so the lowest order term introduced by the discretization of the advection term is rarely unimportant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The lowest order term introduced by the discretization of the diffusion term is $\frac{\nu\Delta x^2}{12}\frac{\partial^4u}{\partial x^4}$, which is unimportant if $\frac{v\Delta x}{2}\frac{\partial^2u}{\partial x^2}$&#xA;$$\color{blue}{\frac{\nu\Delta x^2}{12}\frac{\partial^4u}{\partial x^4}} \ll \color{red}{\frac{v\Delta x}{2}\frac{\partial^2u}{\partial x^2}}$$&#xA;$$\frac{\nu\Delta x}{6x_0^2} \ll v \tag{****}$$&#xA;where $x_0$ is some characteristic length. $(**$$**)$ is usually true, so the lowest order term introduced by the discretization of the diffusion term is usually unimportant.&lt;/p&gt;&#xA;" OwnerUserId="18932" LastActivityDate="2018-03-14T12:49:08.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29040" PostTypeId="1" CreationDate="2018-03-14T14:28:22.217" Score="2" ViewCount="146" Body="&lt;p&gt;When it comes to discretization schemes for finite volume method, the following terms can be found in literature:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;monotone schemes&lt;/li&gt;&#xA;&lt;li&gt;monotonicity preserving schemes&lt;/li&gt;&#xA;&lt;li&gt;local extremum diminishing schemes&lt;/li&gt;&#xA;&lt;li&gt;TVD schemes&lt;/li&gt;&#xA;&lt;li&gt;NVD schemes&lt;/li&gt;&#xA;&lt;li&gt;bounded schemes&lt;/li&gt;&#xA;&lt;li&gt;stable schemes&lt;/li&gt;&#xA;&lt;li&gt;stability preserving schemes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A single work doesn't use all of them, rendering it more difficult to understand how these terms compare.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please explain the relationship between the terms? For example: are some of them synonyms, subclasses of others, equivalent provided that some other condition is met?&lt;/p&gt;&#xA;" OwnerUserId="18932" LastEditorUserId="18932" LastEditDate="2018-03-14T16:21:43.150" LastActivityDate="2018-03-14T16:21:43.150" Title="Monotone, monotonicity preserving, LED, TVD, NVD, bounded, stable and stability preserving discretization schemes" Tags="|pde|finite-volume|discretization|" AnswerCount="0" CommentCount="6" ClosedDate="2018-03-16T19:07:45.110" ContentLicense="CC BY-SA 3.0" />
  <row Id="29041" PostTypeId="1" AcceptedAnswerId="29042" CreationDate="2018-03-14T14:52:01.357" Score="0" ViewCount="80" Body="&lt;p&gt;This question is a continuation of &#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/21866/fourth-order-imex-runge-kutta-method&quot;&gt;Fourth order IMEX Runge-Kutta method&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/28859/implementation-details-for-high-order-imex-methods-by-kennedy-and-carpenter&quot;&gt;Implementation details for high order IMEX methods by Kennedy and Carpenter&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need confirmation that ARK3(2)4L[2]SA by Kennedy and Carpenter has the &quot;First Same As Last&quot;-property. I have verified it numerically for ARK4(3)6L[2]SA and ARK5(4)8L[2]SA, but do not get it for ARK3(2)4L[2]SA. I have attached my MatLab-code, and if you run in you will see that differenceStep in function ARK3 is not zero.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [] = testARK()&#xA;    a = -1;&#xA;    b = 1;&#xA;    c = 2;&#xA;    f_s = @(t,y) a*c*y;&#xA;    f_ns = @(t,y) 2*b*t;&#xA;    f = @(t,y) f_s(t,y) + f_ns(t,y);&#xA;    solution = @(t) c*exp(a*c*t) + (2*b*(-1 + exp(a*c*t) - a*c*t))/(a^2*c^2);&#xA;&#xA;    % Initial data&#xA;    t0 = 0;&#xA;    T = 1; % Final time&#xA;    y0 = solution(t0);&#xA;    dt_v = T*2.^(-1:-1:-9);&#xA;&#xA;&#xA;    err_V = zeros(length(dt_v),1);&#xA;    err_est_V = zeros(length(dt_v),1);&#xA;    for i = 1:length(dt_v)&#xA;        yn = y0;&#xA;        t = 0;&#xA;        dt = dt_v(i);&#xA;        ks_4 = f_s(t0,y0);&#xA;        while t&amp;lt;T&#xA;            [yn,err_est, t,dt,ks_4] = ARK3(yn,t,dt,f_s, f_ns,ks_4);&#xA;        end&#xA;        err_V(i) = abs(yn-solution(t));&#xA;        err_est_V(i) = err_est;&#xA;    end&#xA;    figure(1)&#xA;    loglog(dt_v,err_V,'-')&#xA;    hold on&#xA;    loglog(dt_v,err_est_V,':')&#xA;    loglog(dt_v,dt_v.^(3),'o-')&#xA;    hold off&#xA;    leg = legend('err','err estimate','$\mathcal{O}(\Delta t^{3})$');&#xA;    set(leg,'interpreter','latex');&#xA;    title('ARK3')&#xA;    Table = table(err_V(1:end-1)./err_V(2:end));&#xA;    Table.Properties.VariableNames = {'RateOfDecay'};&#xA;    disp(Table)&#xA;end&#xA;&#xA;&#xA;&#xA;function [yn,err_est,t,dt,ks_4] = ARK3(yn,t,dt,f_s, f_ns,ks_4)&#xA;    %% ARK3(2)4L[2]SA-ERK&#xA;    A_E = [0,0,0,0;...&#xA;    1767732205903/2027836641118, 0 ,0, 0;...&#xA;    5535828885825/10492691773637,788022342437./10882634858940, 0, 0;...&#xA;    6485989280629/16251701735622,-4246266847089/9704473918619,...&#xA;    10755448449292/10357097424841,0];&#xA;&#xA;    b = [1471266399579/7840856788654,...&#xA;    -4482444167858/7529755066697,  ...&#xA;    11266239266428/11593286722821,...&#xA;    1767732205903/4055673282236];&#xA;&#xA;    b_hat = [2756255671327/12835298489170,...&#xA;    -10771552573575/22201958757719,...&#xA;    9247589265047/10645013368117,...&#xA;    2193209047091/5459859503100];&#xA;&#xA;    c = [0;1767732205903/2027836641118;3/5;1];&#xA;&#xA;    %% ARK4(3)6L[2]SA-ESDIRK&#xA;&#xA;    A_I = [0,0,0,0;...&#xA;    1767732205903/4055673282236,1767732205903/4055673282236,0,0;...&#xA;    2746238789719/10658868560708,-640167445237/6845629431997,...&#xA;    1767732205903/4055673282236,0;...&#xA;    b];&#xA;&#xA;    %%&#xA;    s = 4; % Number of stages&#xA;    k_v = zeros(s,2); % First column: stiff. Second column: non-stiff&#xA;    k_v(1,1) = f_s(t,yn);&#xA;    differenceStep = abs(f_s(t,yn)-ks_4)&#xA;    k_v(1,2) = f_ns(t,yn);&#xA;    for i = 2:s&#xA;        k_v(i,1) = f_s(t + dt * c(i), yn +...&#xA;        dt * A_E(i,1:(i-1))*k_v(1:(i-1),2) +...&#xA;        dt * A_I(i,1:(i-1))*k_v(1:(i-1),1))...&#xA;        /(1-f_s(t + dt * c(i), dt*A_I(i,i)));&#xA;        k_v(i,2) = f_ns(t + dt * c(i), yn +...&#xA;        dt * A_E(i,1:(i-1))*k_v(1:(i-1),2) +...&#xA;        dt * A_I(i,1:i)*k_v(1:i,1));&#xA;    end&#xA;    ks_4 = k_v(s,1);&#xA;    yn_hat = yn + dt*b_hat*sum(k_v,2);&#xA;    yn = yn + dt*b*sum(k_v,2);&#xA;    err_est = abs(yn-yn_hat);&#xA;    t = t + dt;&#xA;    end    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="19807" LastActivityDate="2018-03-14T16:17:44.163" Title="Confirmation of FSAL property for IMEX methods by Kennedy and Carpenter" Tags="|time-integration|runge-kutta|adaptive-timestepping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29042" PostTypeId="2" ParentId="29041" CreationDate="2018-03-14T16:17:44.163" Score="0" Body="&lt;p&gt;I don't think I was clear so let me clarify my previous post. The FSAL property is on the implicit tableau of the ESDIRK method. When used as an ESDIRK method it is FSAL. When the IMEX portion is in there, it's not ESDIRK because the updated dependent variable adds in the explicit portion. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-03-14T16:17:44.163" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29043" PostTypeId="1" AcceptedAnswerId="29050" CreationDate="2018-03-14T17:03:52.597" Score="5" ViewCount="948" Body="&lt;p&gt;My project involves large simulation and estimation. For each simulation I need to solve 600,000 systems of nonlinear equations. Currently I am using Newton's method to find the solutions. That involves inverting 600,000 million Jacobian matrices at each iteration. Currently I iterate 100 times for convergence in MATLAB with a mex C file, it takes 250 seconds for one simulation on one core. This speed renders it infeasible for estimation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you guys tell me which should be the fastest way to invert large number of small matrices. All the matrices are between 2 by 2 to 6 by 6. My gut feeling is that CUDA might be the only method feasible for estimation. I am currently translating the code to MKL fortran and I have no experience with CUDA at all. So please give me some advice - I need to decide which platform to implement, and soon. &lt;/p&gt;&#xA;" OwnerUserId="27151" LastEditorUserId="17294" LastEditDate="2018-03-14T18:10:35.723" LastActivityDate="2018-03-15T15:07:55.020" Title="What is the fastest method to invert millions of matrices?" Tags="|matrix|inverse|jacobian|" AnswerCount="1" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="29046" PostTypeId="2" ParentId="28753" CreationDate="2018-03-15T03:55:54.823" Score="7" Body="&lt;p&gt;Summarizing and formulating the answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(part of it was already given in the comments by &lt;em&gt;user14717&lt;/em&gt;, &lt;em&gt;Christian Clason&lt;/em&gt;, and &lt;em&gt;Kirill&lt;/em&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While performing numerical differentiation using finite differences, one would observe two sources of error: &lt;strong&gt;truncation&lt;/strong&gt; and &lt;strong&gt;rounding&lt;/strong&gt;. Truncation error (for simplicity, what order terms are truncated from the Taylor series expansion) will be determined by a discretization scheme that is being used and will decrease with $h$. Unfortunately, that will be eventually stopped by an increase in the rounding error. Therefore, there usually exists an optimal discretization step $h$ when the total error (the combined effect of Taylor series truncation and rounding in finite-precision arithmetic) is minimal. Certainly, this value depends on the discretization scheme, function under-approximation, chosen point, etc. So, one would expect to see a plot that is similar to the following one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BXSxP.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BXSxP.png&quot; alt=&quot;relative error vs step size&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, I plot the relative error in the approximation of the second derivative using your scheme vs analytical derivative as a function of the step size $h$.&#xA;Since you are using a standard five-point stencil for a second-order derivative, the error is expected to drop as $\mathcal O(h^4)$, until you hit $h\approx5\cdot 10^{-3}$. At this point, the rounding error starts dominating and we see the oscillations in the relative error together with its gradual increase. However, using this scheme you are able to get 10 significant digits from your derivative pretty safely using double precision, which I would say is a very reasonable result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If interested, you might invest your time adjusting your discretization scheme, increased precision (&lt;code&gt;vpa&lt;/code&gt;), etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple of minor notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;loglog&lt;/code&gt; plots are usually very useful when studying refinements, errors, timings, etc.&lt;/li&gt;&#xA;&lt;li&gt;a line on the plot usually implies continuity of the data; in this particular case, where you are using finite differences at discrete points, markers are preferred, in my opinion.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Slightly modified Octave code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = pi/2;&#xA;stps = logspace(-8,0,80);&#xA;numpoints = size(stps,2);&#xA;outs = zeros(numpoints,1);&#xA;analytic = ones(numpoints,1)*(cos(x)^2 - sin(x))*exp(sin(x));&#xA;for i=1:numpoints&#xA;  h=stps(i);&#xA;  coefficients = [-5/2 -1/12 4/3 4/3 -1/12]./h^2;&#xA;  steps = [0 -2 -1 1 2].*h;&#xA;  outs(i) = sum(coefficients .* exp(sin(x + steps))); &#xA;end&#xA;&#xA;figure(1)&#xA;loglog(stps',abs(outs-analytic)./abs(analytic),&quot;kx&quot;);hold on;&#xA;xlim([1E-8,1E-0]);   ylim([1E-12,1E+1]);&#xA;xlabel(&quot;h&quot;);         ylabel(&quot;Rel. error: |num-an|/|an|&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-15T03:55:54.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29047" PostTypeId="2" ParentId="20332" CreationDate="2018-03-15T05:54:48.710" Score="1" Body="&lt;p&gt;I haven't used PETSc via Python, but I suppose you can use all the functions provided by PETSc. Then you can use &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/Mat/MatGetSubMatrix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatGetSubmatrices()&lt;/code&gt;&lt;/a&gt; on your processor 0 to obtain all rows and columns, thus obtaining a local copy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://lists.mcs.anl.gov/pipermail/petsc-users/2016-June/029686.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;following discussion on PETSc-maillist&lt;/a&gt; has a very similar question answered and some sample code provided, though, without python involved.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-15T05:54:48.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29048" PostTypeId="2" ParentId="18639" CreationDate="2018-03-15T06:54:41.427" Score="1" Body="&lt;p&gt;The following paper describes how to modify Gauss-Jordan elimination to form Drazin inverse:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;K. M. Anstreicher, U. G. Rothblum, &quot;&lt;a href=&quot;https://doi.org/10.1016/0024-3795(87)90219-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using Gauss-Jordan elimination to compute the index, generalized nullspaces, and Drazin inverse&lt;/a&gt;,&quot; &lt;em&gt;Linear Algebra Appl.&lt;/em&gt;, vol. 85, pp. 221-239, Jan. 1987.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am not aware of a Matlab or C library that implemented that approach; however, the algorithm inside this particular paper does not seem to be too hard to be implemented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, this paper has several papers discussing efficient algorithms to compute Drazin inverse inside the references.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was also able to find &lt;a href=&quot;https://stackoverflow.com/questions/28908731/drazin-inverse-of-a-matrix&quot;&gt;a similar question on SO&lt;/a&gt;, where one of the answers cites &lt;a href=&quot;https://doi.org/10.1016/S0096-3003(02)00814-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;another paper&lt;/a&gt; &lt;strong&gt;that has a Matlab code&lt;/strong&gt; for computing Drazin inverses of two-variable polynomial matrices, which is probably an overkill. As I was reading this paper, it is based on the Greville algorithm for finding Drazin inverse (the algorithm listed there is also pretty straightforward):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;T. N. E. Greville, &quot;&lt;a href=&quot;https://doi.org/10.1016/0024-3795(73)90021-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Souriau-Frame algorithm and the Drazin pseudoinverse&lt;/a&gt;,&quot;  &lt;em&gt;Linear Algebra Appl.&lt;/em&gt;, vol. 6, pp. 205-208, Feb. 1973&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While doing a literature review, I was also able to find the other paper &lt;strong&gt;that has Mathematica code&lt;/strong&gt; for finding Drazin inverse for polynomial matrices inside. Again, it is probably an overkill for you; though, I guess, the modifications and optimizations that you can do for your constant matrix case should not be too drastic:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;M. D. Petkovich, P. S. Stanimirovich, &quot;&lt;a href=&quot;https://doi.org/10.1016/j.laa.2006.11.011&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpolation algorithm for computing Drazin inverse of polynomial matrices&lt;/a&gt;,&quot; &lt;em&gt;Linear Algebra Appl.&lt;/em&gt;, vol. 422, no. 2-3, pp. 526-539, Apr. 2007&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The thing I am the least confident about is a Matlab package for finding Drazin inverse using gradient neural network &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/59444-gnn-drazin&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNN_Drazin&lt;/a&gt;. I personally do not have enough knowledge to either judge how feasible is this numerical approach using neural networks, nor the quality of the package itself.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-15T06:54:41.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29049" PostTypeId="1" AcceptedAnswerId="29054" CreationDate="2018-03-14T18:00:40.580" Score="8" ViewCount="163" Body="&lt;p&gt;I have the following situation: I have a sequence of vectors $x_1, x_2,.. $ and for each I want to compute the product $Ax_i$ where $A$ is fixed at the outset. Although there is no information about the structure of $x_i$, $A$ typically has a particular pattern where many values are repeated and I would like to compute these products as fast as possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example of $A$ looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oBFof.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oBFof.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the white regions are 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is some way of storing information about $A$ or modify it somehow that would allow me to reduce the number of operations for each product. For rows that are all 0 this is trivial – one can just store the row indicies that indicate such rows. It's also possible to store information about which rows are duplicated so as to reuse row computations. I have also considered ordering the rows of the matrix such as to minimize the mean difference between each row and only compute the difference at each row. This seems to run into problems for the more complicated patterns, however. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is any known methods for these kinds of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: another idea I had is that since the no. of unique values in the matrix is fairly low, one could decompose the product as $Ax = A_1x + A_2x + \dots A_nx$ where $A_i$ contains only one unique value, but I'm still not sure if this can provide any advantage for this problem.&lt;/p&gt;&#xA;" OwnerUserId="17170" OwnerDisplayName="Slug Pue" LastActivityDate="2018-03-15T18:25:37.477" Title="Exploiting patterns in matrix for efficient matrix-vector multiplication" Tags="|matrix|linear-algebra|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="29050" PostTypeId="2" ParentId="29043" CreationDate="2018-03-15T14:17:20.983" Score="1" Body="&lt;p&gt;&lt;em&gt;I known that this is not a definitive answer, but it can give an idea how to move with CUDA. At this stage is difficult to give advice because for this is necessary a detailed know about the actually code&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already write in comments, &lt;em&gt;in general&lt;/em&gt;, is better not invert the matrix, but solve the linear system (for detail about why see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26423/practical-example-of-why-it-is-not-good-to-invert-a-matrix&quot;&gt;this question&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is the possibility that CUDA can help you , but before you should consider some aspects.&#xA;In general the real bottleneck in a CUDA application are the memory transfer, sometimes is better to recalculate things respect than move them, you must consider the memory available and in other case the memory transfer use more time respect the use of cpu.&#xA;Another point to focus is the presence of &lt;code&gt;if&lt;/code&gt; condition along the flow you like to parallelize, in this case the flows is stopped and the two different branches are execute in sequence with a big degradation of the performance. &#xA;What precision do I need? This is important for the choose of the hardware, and can depend also by the algorithm that you use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this in mind you can start study what parts of your algorithm have got the best advantage from a parallelization. For example is better assign to each CUDA Kernel a linear system or is better parallelize only some task?&#xA;Maybe an idea, not really radical respect your code, is to parallelize the solution of the linear system (= invert the matrix). Another, if you use feval, is try to use the &lt;a href=&quot;https://it.mathworks.com/help/distcomp/feval.html?s_tid=gn_loc_drop&quot; rel=&quot;nofollow noreferrer&quot;&gt;parallel feval&lt;/a&gt; in matlab.&#xA;Other possibility can be to consider variants of Newton's method to obtain speed up, this is not only related with the use of GPU.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Some link:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://developer.nvidia.com/cusolver&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUDA library&lt;/a&gt; for solver in cuda.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://imsc.uni-graz.at/haasegu/Lectures/GPU_CUDA/WS15/HPC_I_rohrhofer.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf&lt;/a&gt; with an example in CUDA for a particular case of Netwon's method&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/4a43/b507b97affa0d363fa17def18c042f410695.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Article pdf&lt;/a&gt; with a variant of Newton's method with CUDA. &lt;strong&gt;Parallel interval newton method on CUDA&lt;/strong&gt; by &lt;em&gt;Philip-Daniel Beck,   Marco Nehmeier&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2018-03-15T14:17:20.983" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29053" PostTypeId="2" ParentId="29049" CreationDate="2018-03-15T16:25:45.437" Score="0" Body="&lt;p&gt;Disclaimer: I have no idea if this will actually speed up your calculation, as it adds quite a bit of computational overhead. Since it looks like your matrix isn't very sparse, it's hard to imagine beating a &lt;a href=&quot;http://www.netlib.org/blas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAS&lt;/a&gt; implementation like &lt;a href=&quot;https://software.intel.com/en-us/mkl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel MKL&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;That said, here's an idea: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Store an array of sparse matrices, one for each unique value in your matrix, where each sparse matrix is stored in &lt;a href=&quot;https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_(CSR,_CRS_or_Yale_format)&quot; rel=&quot;nofollow noreferrer&quot;&gt;compressed row format&lt;/a&gt;. The clever thing here is that you don't need to actually store the sparse matrix entries ($A$ from the Wikipedia article), only the sparse patterns $IA$ and $JA$ since the matrix entries are all the same. When performing a matrix-vector product with one of these sparse matrices, you only need to sum the entries from your vector $x$ since the multiplication can take place at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have some values in the matrix that don't have duplicates, you can throw them all into one conventional sparse matrix, and do the MVP the &quot;normal&quot; sparse matrix way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the implementation for this approach is not trivial. Even if it is faster than a straight BLAS implementation, you would probably need to have a lot of $x_i$ vectors to compensate for the computational overhead in the sparse matrix setup. I do think that this approach would provide quite a bit of storage savings, for what that's worth. &lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2018-03-15T16:25:45.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29054" PostTypeId="2" ParentId="29049" CreationDate="2018-03-15T18:25:37.477" Score="3" Body="&lt;p&gt;I suggest a different point of view. Maybe you can obtain a improvement of the performance with some clever matrix multiplication, but there are more than one possibility that you obtain small results respect the effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This kind of matrix is small, we are talking about $138 \times 78$, the moder cpus have got a lot of power and no problem to work over this size. The bottleneck is move the data to the cpu. This kind of problems have been addressed by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blas&lt;/a&gt; libraries. The Blas library take care not only of the multiplication, but how to optimize the data moving inside the hardware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is very difficult, to be clear near impossible for us, try to obtain best performance respect the Blas function. The classical examples are the nested loops. For example the &lt;a href=&quot;https://en.wikipedia.org/wiki/Automatically_Tuned_Linear_Algebra_Software&quot; rel=&quot;nofollow noreferrer&quot;&gt;Atlas&lt;/a&gt; a particular implementation of Blas when is installed make an auto tuning over the hardware (see &lt;a href=&quot;https://fossies.org/linux/atlas/doc/atlas_install.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this pdf&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reason the fist suggestion I tell to you is try to use a Blas library. See the before wiki page for a list, there are commercial or free,&#xA;here depend of you (maybe you can start with OpenBlas).&#xA;Note that also there are library that use Blas under them and they are more comfortable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not enough try with other way, but remember use Blas for the multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The case is different if the number of zero elements are more and more, no this the case, to give an idea about 90%. Here you have &lt;a href=&quot;https://en.wikipedia.org/wiki/Sparse_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;sparse matrix&lt;/a&gt; and you can use a different storage methods to obtain advantage. Note that also in this case you can find &lt;a href=&quot;https://math.nist.gov/spblas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sparse Blas&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2018-03-15T18:25:37.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29055" PostTypeId="2" ParentId="8144" CreationDate="2018-03-15T21:40:52.090" Score="0" Body="&lt;p&gt;An alternative to FFTW could be using Intel MKL (&lt;code&gt;mkl_trig_transforms.h&lt;/code&gt;, in particular). Various &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-trigonometric-transform-routines&quot; rel=&quot;nofollow noreferrer&quot;&gt;trigonometric transforms are implemented there&lt;/a&gt;, and, in addition, using the wrapper interface to FFTW3 is also possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also join Bill Barth in the opinion that calling a high-performance optimized library is far better than coding it yourself. Unless you are specifically interested in understanding how a certain transform is done.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-15T21:40:52.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29056" PostTypeId="2" ParentId="7770" CreationDate="2018-03-15T22:05:01.973" Score="1" Body="&lt;p&gt;It is hard to come with 100% explanation without seeing the source code, but I predict that one of the following things had happened:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;most likely, your matrix &lt;code&gt;A&lt;/code&gt; was &lt;strong&gt;not initially created with a support for &lt;code&gt;long int&lt;/code&gt; indexes&lt;/strong&gt; and increased size limitations. And simple manual changing &lt;code&gt;A-&amp;gt;itype&lt;/code&gt; will not solve this problem because effectively the matrix has to be reconstructed to support &lt;code&gt;long int&lt;/code&gt; indexes. At least, I am not aware of CHOLMOD routine that converts one to another.&lt;/li&gt;&#xA;&lt;li&gt;You are continuing to use &lt;code&gt;int&lt;/code&gt; indexes instead of &lt;code&gt;long int&lt;/code&gt; ones somewhere during the process. CHOLMOD has an internal type &lt;code&gt;SuiteSparse_long&lt;/code&gt; and that is the one that you should be using.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;CHOLMOD actually has two very good examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;demo/cholmod_demo.c&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;demo/cholmod_l_demo.c&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;that perform exactly the same set of actions on &quot;standard&quot; and &quot;large&quot; matrices.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-15T22:05:01.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29057" PostTypeId="2" ParentId="16326" CreationDate="2018-03-15T23:21:38.857" Score="1" Body="&lt;p&gt;Debugging MPI programs is not easy. I used NetBeans for a while until I had to use distributed parallel programming a lot as the convenience of having this particular IDE decreased significantly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For debugging parallel codes, usually, a special tool like &lt;a href=&quot;https://www.roguewave.com/products-services/totalview&quot; rel=&quot;nofollow noreferrer&quot;&gt;TotalView&lt;/a&gt; or &lt;a href=&quot;https://www.arm.com/products/development-tools/hpc-tools/cross-platform/forge/ddt&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDT&lt;/a&gt; is used. However, that detaches from IDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some people have success with &lt;a href=&quot;http://www.eclipse.org/ptp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eclipse Parallel Tools Platform (PTP)&lt;/a&gt;. That brings a parallel debugger closer to the IDE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, that does not answer your specific question. In Netbeans, you would certainly be able to compile and run the code by changing the path to the compiler and run command accordingly. Unfortunately, the debugger is not attached. Technically, one can attach a serial debugger (say, &lt;code&gt;gdb&lt;/code&gt;) to an MPI code. But NetBeans (in March 2018) does not have a plugin, like Eclipse PTP, to support an interface to it, even if user magically does the attachment of the debugger to the running code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, the following &lt;a href=&quot;https://www.open-mpi.org/faq/?category=debugging&quot; rel=&quot;nofollow noreferrer&quot;&gt;FAQ on open-mpi.org&lt;/a&gt; provides you with a lot of answers on MPI debugging, including ways to attach &lt;code&gt;gdb&lt;/code&gt; to your MPI code (Q6).&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-15T23:21:38.857" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29058" PostTypeId="1" CreationDate="2018-03-16T12:54:50.967" Score="1" ViewCount="95" Body="&lt;p&gt;Let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x,y) = \exp \left(- \frac{1}{2}a x^2 - \frac{1}{2}c y^2 + bxy \right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $a,b,c\ge 0$. I want to integrate numerically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{x_0}^{x_1}\mathrm{d}x \int_{y_0}^{y_1}\mathrm{d}y \, f(x,y) x^ny^m$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $-\infty &amp;lt; x_0 &amp;lt; x_1 &amp;lt; \infty$, $-\infty &amp;lt; y_0 &amp;lt; y_1 &amp;lt; \infty$, and $n,m\in\{0,1,2\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A naive method has problems when the peak of $f(x,y)$ is far from the rectangle of integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method that I can use?&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="988" LastEditDate="2018-05-27T00:49:05.063" LastActivityDate="2018-05-27T00:49:05.063" Title="Numerical integration of a quadratic form exponential in two variables over a rectangle" Tags="|integration|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="29059" PostTypeId="1" CreationDate="2018-03-16T13:07:47.523" Score="2" ViewCount="113" Body="&lt;p&gt;The 2014 paper &quot;Iteration-Free Computation of Gauss--Legendre Quadrature Nodes and Weights&quot; by I. Bogaert (&lt;a href=&quot;https://doi.org/10.1137/140954969&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1137/140954969&lt;/a&gt;) contains the following expression in Appendix A:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fsbnw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fsbnw.png&quot; alt=&quot;Expression from the paper.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the meaning of the &quot;LL&quot; here? It is not defined in the body of the paper and it's such a commonly occurring character sequence that Googling for it has proven futile.&lt;/p&gt;&#xA;" OwnerUserId="27171" LastActivityDate="2018-03-16T13:07:47.523" Title="Meaning of a symbol in a research paper" Tags="|numerical-analysis|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29061" PostTypeId="2" ParentId="29058" CreationDate="2018-03-16T16:50:43.207" Score="2" Body="&lt;p&gt;I just tried the integration algorithm that comes with &lt;code&gt;scipy.integrate&lt;/code&gt; for double integrals (&lt;code&gt;dblquad&lt;/code&gt;), and it seems to work just fine for your problem&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division, print_function &#xA;import numpy as np&#xA;from scipy.integrate import dblquad&#xA;&#xA;a = 2&#xA;b = 2&#xA;c = 1&#xA;fun = lambda x, y: np.exp(-a/2*x**2 - b/2*y**2 + c*x*y)*x**3&#xA;inte2 = dblquad(fun, 1, 4, lambda y: -5, lambda y: 8)&#xA;print(inte2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(0.5814009878561697, 8.012117719083942e-09)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first value is the integral and the second is an estimate of the error on the integral.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-03-16T16:50:43.207" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29063" PostTypeId="2" ParentId="12979" CreationDate="2018-03-17T02:48:02.660" Score="4" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Kerr, T. H., “&lt;a href=&quot;https://doi.org/10.2514/3.20246&quot; rel=&quot;nofollow noreferrer&quot;&gt;Testing Matrices for Definiteness and Application Examples that Spawn the Need&lt;/a&gt;,” &lt;em&gt;AIAA Journal of Guidance&lt;/em&gt;, Control, and Dynamics, Vol. 10, No. 5, pp. 503-506, Sept.-Oct., 1987.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Kerr, T. H., “&lt;a href=&quot;https://doi.org/10.2514/3.25371&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Misstatements of the Test for Positive Semidefinite Matrices&lt;/a&gt;,” &lt;em&gt;AIAA Journal of Guidance, Control, and Dynamics&lt;/em&gt;, Vol. 13, No. 3, pp. 571-572, May-Jun. 1990. (as occurred in Navigation &amp;amp; Target Tracking software in the 1970’s &amp;amp; 1980’s using counterexamples)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Kerr, T. H., “&lt;a href=&quot;https://doi.org/10.1109/7.53451&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fallacies in Computational Testing of Matrix Positive Definiteness/Semidefiniteness&lt;/a&gt;,” &lt;em&gt;IEEE Transactions on Aerospace and Electronic Systems&lt;/em&gt;, Vol. 26, No. 2, pp. 415-421, Mar. 1990. [Lists fallacious algorithms that the author found to routinely exist in U.S. Navy submarine navigation and sonobuoy software in the late 1970’s and early 1980’s using counterexamples to point out the problems.]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27180" LastEditorUserId="20688" LastEditDate="2018-03-17T11:42:29.673" LastActivityDate="2018-03-17T11:42:29.673" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29064" PostTypeId="2" ParentId="28343" CreationDate="2018-03-17T04:25:02.543" Score="2" Body="&lt;p&gt;The short answer is: &quot;it depends&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the first question that is not entirely clear from the question is a &lt;em&gt;definition of similarity&lt;/em&gt; between two matrices. Technically there is a term &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_similarity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matrix similarity&lt;/a&gt;, but there is certainly nothing common between it and what you are asking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To justify the inquiry about the &lt;em&gt;similarity definition&lt;/em&gt;, consider three matrices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A_1=\left[\begin{array}{ccc}&#xA;1 &amp;amp;0 &amp;amp;0 \\&#xA;1 &amp;amp;1 &amp;amp; 0 \\&#xA;1 &amp;amp; 1 &amp;amp; 1&#xA;\end{array}\right], \quad&#xA;A_2=\left[\begin{array}{ccc}&#xA;1 &amp;amp;1 &amp;amp;1 \\&#xA;0 &amp;amp;1 &amp;amp; 1 \\&#xA;0 &amp;amp; 0 &amp;amp; 1&#xA;\end{array}\right],\quad&#xA;A_3=\left[\begin{array}{ccc}&#xA;1 &amp;amp;0 &amp;amp;0 \\&#xA;0 &amp;amp;1 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 1&#xA;\end{array}\right]&#xA;$$&#xA;Here, $A_1 = A_2^T$. However, according to element-wise similarity definition you proposed, the difference between them will be huge: 2/3. Moreover, the identity matrix $A_3=I_3$ has a smaller difference of just 1/3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other examples demonstrating more complicated similarities/differences between two matrices can be shown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, let's assume that we are just interested performing strict element-to-element in two matrices. A natural metric would be the one that &lt;em&gt;nicoguaro&lt;/em&gt; suggested:&#xA;$&#xA;\left|\left| A_1-A_2 \right|\right|_{1,2,F,\ldots}&#xA;$&#xA;Here, $1,2,F,\ldots$ in the subscript correspond to different matrix norms (1-norm, 2-norm, Frobenius norm, etc.) Unfortunately, the calculation of the norm requires at least browsing through all the elements of the matrix $A_1-A_2$. And it will be not easy to come up with the speed-up of that calculation without knowing where your matrix comes from. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various algorithms that will be able to estimate your norm, provided some information:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www4.ncsu.edu/~ipsen/ps/slides_ima.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Matrix-Free, Transpose-Free Norm Estimator, First IMA International Conference on Numerical Linear Algebra and Optimisation, Birmingham, UK, 13-15 September 2007&lt;/a&gt; - requires a matrix-vector product.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1109/TAP.2014.2386306&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Heldring, E. Ubeda, J. M. Rius, &quot;Stochastic Estimation of the Frobenius Norm in the ACA Convergence Criterion,&quot; &lt;em&gt;IEEE Antennas Propagat.&lt;/em&gt;, vol. 63, no. 3, pp. 1155-1158, Mar. 2015.&lt;/a&gt; - using adaptive cross approximation, which is efficient if your matrix is low-rank to begin with.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But I think all that is an overkill. At this moment, there is no justification why a naïve algorithm for calculating a norm of a matrix $A_1-A_2$ is not sufficient. Since, even if the size of the matrix is large, you are storing them in the memory anyway; thus, you certainly can afford to compute at least the Frobenius norm.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-17T04:25:02.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29065" PostTypeId="1" AcceptedAnswerId="29066" CreationDate="2018-03-17T05:21:40.520" Score="1" ViewCount="61" Body="&lt;p&gt;I'm reading this paper and trying to follow everything that the author has done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://article.sapub.org/10.5923.j.ajcam.20110102.13.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Comparative Study of Finite Volume Method and Finite Difference Method for Convection-Diffusion Problem&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there is a problem. I cannot derive the following equation (third line in equation (6) of [&lt;a href=&quot;http://article.sapub.org/10.5923.j.ajcam.20110102.13.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]) for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WNIOa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WNIOa.png&quot; alt=&quot;fig1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I cannot match the expression marked by &quot;1&quot; in red.&lt;/li&gt;&#xA;&lt;li&gt;The signs are opposite for the part marked by &quot;2&quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Does anybody know how the author has derived this equation?&lt;/p&gt;&#xA;" OwnerUserId="23673" LastEditorUserId="20688" LastEditDate="2018-03-17T06:47:43.793" LastActivityDate="2018-03-17T06:47:43.793" Title="Problems with deriving an equation for a finite-difference scheme given in the journal paper" Tags="|numerical-analysis|finite-difference|finite-volume|discretization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29066" PostTypeId="2" ParentId="29065" CreationDate="2018-03-17T06:17:42.710" Score="3" Body="&lt;p&gt;Initially, the equation (6) is derived from equation (4):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F_e\Phi_e-F_w\Phi_w=D_e(\Phi_E-\Phi_P)-D_w(\Phi_P-\Phi_W)&#xA;$$&#xA;under the central differencing approximation $\Phi_e=(\Phi_P+\Phi_E)/2$, $\Phi_w=(\Phi_W+\Phi_P)/2$. Naturally, (4) becomes the first line of (6):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{F_e}{2}(\Phi_P+\Phi_E)-\frac{\color{red}{F_w}}{2}(\Phi_W+\Phi_P) = D_e(\Phi_E-\Phi_P)-D_w(\Phi_P-\Phi_W)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice, there is probably a typo (highlighted in red) in the first line of the equation (6) as in the second term $F_w$ is mistyped with $F_e$. Otherwise, those terms just simplify and the second line of (6), where $F_w$ magically appears again does not make any sense. Nevermind, the proposed by authors continuity equation $F_e-F_w=0$ equate them anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, naturally, we will move all the terms with $\Phi_P$ to the left-hand side, and everything else to the right-hand side and group the coefficients in front of $\Phi_P$, $\Phi_W$, and $\Phi_E$ in a convenient form. So,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{F_e}{2}\Phi_P-\frac{F_w}{2}\Phi_P+D_e\Phi_P+D_w\Phi_P =-\frac{F_e}{2}\Phi_E+\frac{F_w}{2}\Phi_W+ D_e\Phi_E+D_w\Phi_W&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;finally resulting in the second line of (6), where the sign seems to be wrong, as far as I understand, (highlighted in red):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\underbrace{\left(D_w-\frac{F_w}{2}\right)+\left(D_e\color{red}{+}\frac{F_e}{2} \right)}_{*}\right]\Phi_P =\left(D_e-\frac{F_e}{2}\right)\Phi_E+\left(D_w+\frac{F_w}{2}\right)\Phi_W&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No, by adding $(F_w-F_e)$ to (*) (I can do it since by continuity equation the added expression is 0):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[\left(D_w-\frac{F_w}{2}+F_w\right)+\left(D_e+\frac{F_e}{2} -F_e\right)\right]\Phi_P =\left(D_e-\frac{F_e}{2}\right)\Phi_E+\left(D_w+\frac{F_w}{2}\right)\Phi_W&#xA;$$&#xA;I can arrive to:&#xA;$$&#xA;\left[\left(D_w+\frac{F_w}{2}\right)+\left(D_e-\frac{F_e}{2}\right)\right]\Phi_P =\left(D_e-\frac{F_e}{2}\right)\Phi_E+\left(D_w+\frac{F_w}{2}\right)\Phi_W&#xA;$$&#xA;and now by adding $F_e-F_w$ I arrive to the final third line of (6):&#xA;$$&#xA;\left[\left(D_w+\frac{F_w}{2}\right)+\left(D_e-\frac{F_e}{2}\right)+(F_e-F_w)\right]\Phi_P =\left(D_e-\frac{F_e}{2}\right)\Phi_E+\left(D_w+\frac{F_w}{2}\right)\Phi_W&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like the final expression of (6) in the paper you cited is correct (provided (4) is correct, continuity equation is physical and central differencing is taken care of - which it probably is). However, the derivation that they have shown in the first two lines of equation (6) contains $\approx 1.5$ confusing bugs. 1.5 - because $F_e$ is technically equal to $F_w$.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-03-17T06:24:24.233" LastActivityDate="2018-03-17T06:24:24.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29068" PostTypeId="2" ParentId="25464" CreationDate="2018-03-17T08:11:20.503" Score="0" Body="&lt;p&gt;Even though the biggest part of the question has been answered in the comments, I want to point out a detail that is very important in my opinion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several questions about the complexity you can ask:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what is the &quot;minimum/tight&quot; asymptotic complexity to find a certain quantity (in your case, it is the Gini index)?&lt;/li&gt;&#xA;&lt;li&gt;what is the asymptotic complexity of a certain algorithm (or even its implementation if we want to be very fine-grained)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding a difference between tight ($\Theta(N)$) and upper bound ($\mathcal O(N))$ complexities, the &lt;a href=&quot;https://stackoverflow.com/questions/471199/what-is-the-difference-between-%CE%98n-and-on&quot;&gt;following question on SO&lt;/a&gt; can be useful. I will be using $\Theta$ notation for this answer; though, using $\mathcal O$ is sufficient as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for the Gini index, let's consider two &quot;algorithms&quot; to find the desired quantity: variant $A$: that will be described exactly by the formula in your question and variant $B$ described by the modified formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\text{Gini}_A(\mathbf{x}) = 1-2\sum_{k=1}^{N}\frac{\mathbf{x}(k)}{\color{red}{||\mathbf{x}||_1}}\frac{N-k+0.5}{N}\\&#xA;\text{Gini}_B(\mathbf{x}) = 1-\frac{2}{\color{red}{||\mathbf{x}||_1}}\sum_{k=1}^{N}\mathbf{x}(k)\frac{N-k+0.5}{N}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice, that in algorithm $B$ I explicitly moved the calculation of the vector norm $||\mathbf{x}||_1$ outside of the summation to signal that &lt;strong&gt;its computation happens only once&lt;/strong&gt;. Now, if we assume that algorithm $A$ actually computes the norm $||\mathbf{x}||_1$ $N$ times (for no reason), then:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the complexity of algorithm $A$ is $\Theta(N^2).$ The calculation of the norm happens $N$ times, and one norm calculation is $\Theta(N)$ complexity.&lt;/li&gt;&#xA;&lt;li&gt;the complexity of algorithm $B$ is $\Theta(N)$. First the norm is calculated in $\Theta(N)$, followed by computation of the sum that touches every element of $\mathbf{x}$ once - another $\Theta(N)$ and $\Theta(1)$ multiplication and subtraction.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-17T08:11:20.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29069" PostTypeId="2" ParentId="28963" CreationDate="2018-03-17T11:50:35.350" Score="1" Body="&lt;p&gt;Pycuda is one of the more pythonic way to handle cuda in python as @nluigi suggested. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are open to call C/C++ code inside python  there is also &lt;a href=&quot;https://cusplibrary.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUSP&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Cusp is a library for sparse linear algebra and graph computations based on Thrust. Cusp provides a flexible, high-level interface for manipulating sparse matrices and solving sparse linear systems. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a template library that I found user friendly and with this you can do with it some heavy task. I use both pycuda and CUSP some time ago for a GPU solver for Navier-Stokes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also other library I cite two:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/lebedov/scikit-cuda&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit-cuda&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;scikit-cuda provides Python interfaces to many of the functions in the CUDA device/runtime, CUBLAS, CUFFT, and CUSOLVER libraries distributed as part of NVIDIA's CUDA Programming Toolkit, as well as interfaces to select functions in the CULA Dense Toolkit. Both low-level wrapper functions similar to their C counterparts and high-level functions comparable to those in NumPy and Scipy are provided.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pythonhosted.org/PyCULA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyCULA&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;PyCULA provides an efficient and simple CUDA GPU environment for python. PyCULA accomplishes this feat by combining the power of driver based PyCUDA with nVidia’s runtime libraries and, most importantly, CULA GPU-LAPACK functionality in a single environment. We aim to hide complications without limiting the enduser.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2018-03-17T11:50:35.350" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29070" PostTypeId="1" CreationDate="2018-03-17T20:41:38.047" Score="1" ViewCount="49" Body="&lt;p&gt;I'm writing a genetic algorithm to minimize a function. I have two questions, one in regards to selection and the other with regards to crossover and what to do when it doesn't happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an outline of what I'm doing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while (number of new population &amp;lt; current population)&#xA;    # Evaluate all fitnesses and give them a rank. Choose individual based on rank (wheel roulette) to get first parent.&#xA;    # Do it again to get second parent, ensuring parent1 =/= parent2&#xA;&#xA;    # Elitism (do only once): choose the fittest individual and immediately copy to new generation&#xA;&#xA;    Multi-point crossover: 50% chance&#xA;    if (crossover happened)&#xA;        do single point mutation on child (0.75%)&#xA;    else&#xA;        pick random individual to be copied into new population.&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And all of this is under another &lt;code&gt;while&lt;/code&gt; loop which tracks fitness progression and number of iterations, which I didn't include. So, my questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;As you can see, two parents are chosen randomly in each&#xA;iteration until the new population is filled up. So, the two same&#xA;parents may mate more than once and surely several fit parents will&#xA;mate many more times than once. Is this in any way bad?&lt;/li&gt;&#xA;&lt;li&gt;In the &lt;a href=&quot;http://obitko.com/tutorials/genetic-algorithms/ga-basic-description.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;obitko tutorial&lt;/a&gt;, it says if crossover doesn't&#xA;happen, then child is exact copy of parents. I don't even understand&#xA;what that means, so, as you can see, I just picked a random parent&#xA;(uniformly; no fitness considered) and copied to new population.&#xA;This seems weird to me. Whether I actually do this or not, my results really don't change that much. What's the proper way to handle the case&#xA;when crossover doesn't happen?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27189" LastEditorUserId="6775" LastEditDate="2018-03-17T21:29:41.677" LastActivityDate="2018-03-17T21:29:41.677" Title="Genetic Algorithm: Need some clarification on selection and what to do when crossover doesn't happen" Tags="|optimization|genetic-algorithms|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29071" PostTypeId="2" ParentId="2578" CreationDate="2018-03-18T01:02:16.630" Score="1" Body="&lt;p&gt;In general, the eigenvalues of $B\circ A$ are unrelated to those of $A$ and $B$ individually. So, in a general case, not a lot can be gained from the knowledge of the eigenvalues of $A$ and $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following info might be useful:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://mathoverflow.net/questions/207836/hadamard-product-and-eigendecomposition&quot;&gt;Hadamard product and eigendecomposition – on MathOverflow&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;M. Fang, &quot;&lt;a href=&quot;https://doi.org/10.1016/j.laa.2007.03.024&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bounds on eigenvalues of the Hadamard product and the Fan product of matrices&lt;/a&gt;,&quot; &lt;em&gt;Linear Algebra Appl.&lt;/em&gt;, vol. 425, no. 1, pp. 7-15, Aug. 2007&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and confirms that not a lot can be gained for obtaining the eigenvalues of $B\circ A$, and even estimating the spectrum is limited to a very particular cases now.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-18T01:02:16.630" CommentCount="0" CommunityOwnedDate="2018-03-18T01:02:16.630" ContentLicense="CC BY-SA 3.0" />
  <row Id="29072" PostTypeId="2" ParentId="5682" CreationDate="2018-03-18T02:20:29.327" Score="1" Body="&lt;p&gt;It is not the equally spaced &lt;em&gt;points&lt;/em&gt; that are the problem. It's the global support of the basis functions along with equally spaced points that is the problem. A perfectly well conditioned interpolant using equally spaced points is described in Kress's Numerical Analysis, using cubic-b spline basis functions of compact support.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2018-03-18T02:20:29.327" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29073" PostTypeId="2" ParentId="5682" CreationDate="2018-03-18T18:05:47.960" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What exactly happens in the case of equi-spaced points?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Why does increase in polynomial order cause the error to rise after a certain point? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is similar to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Runge%27s_phenomenon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Runge's phenomenon&lt;/a&gt; where, with equi-spaced nodes, the interpolation error goes to infinity with the increase of the polynomial degree, i.e. the number of points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the roots of this problem can be found in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lebesgue_constant_(interpolation)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lebesgue's constant&lt;/a&gt; as noted by @Subodh's comment to @Pedro answer. This constant relates the interpolation with the best approximation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Some notations&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a function $f \in C([a,b])$ to interpolate over the nodes $x_k$. In the Lagrange interpolation are defined the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lagrange_polynomial&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lagrange polynomials&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;L_k(x) = \prod_{i=0, i\neq j}^{n}\frac{x-x_i}{x_k-x_i}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with this is defined the interpolation polynomial $p_n \in P_n$ over the couples $(x_k, f(x_k))$ for light notation $(x_k, f_k)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_n(x) = \sum_{k=0}^n f_kL_k(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now consider a perturbation over the data, this can be for example for rounding, so we have got $\tilde{f}_k$. With this the new polynomial $\tilde{p}_n$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde{p}_n(x) = \sum_{k=0}^n \tilde{f}_k L_k(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error estimates are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p_n(x) - \tilde{p}_n(x) = \sum_{k=0}^n (f_k - \tilde{f}_k) L_k(x)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;| p_n(x) - \tilde{p}_n(x) | \leq \sum_{k=0}^n |f_k - \tilde{f}_k| |L_k(x)|&#xA;\leq \left ( \max_k |f_k - \tilde{f}_k| \right) \sum_{k=0}^n |L_k(x)|&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now it is possible define the &lt;strong&gt;Lebesgue's constant&lt;/strong&gt; $\Lambda_n$ as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\Lambda_n = \max_{x \in [a,b]}  \sum_{k=0}^n |L_k(x)|&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this the final estimates is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;|| p_n - \tilde{p}_n ||_{\infty} \leq \left ( \max_k |f_k - \tilde{f}_k| \right) \Lambda_n &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(marginal note, we look only $\infty$ norm also because we are over a space of finite measure so $L^{\infty} \subseteq \dots \subseteq L^1 $)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the above calculation we have got that $\Lambda_n$ is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;independent from the date:&lt;/li&gt;&#xA;&lt;li&gt;depends only from the nodes distribution;&lt;/li&gt;&#xA;&lt;li&gt;an indicator of stability (the smaller it is, the better it is).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It is also the norm of the interpolation operator respect the&#xA;$|| \cdot||_\infty$ norm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Withe the follow theorem we con have got an estimate of the interpolation error with the Lebesgue's constant:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let $f$ and $p_n$ as above we have&#xA;  $$ || f - p_n ||_{\infty} \leq (1 + \Lambda_n) d_n(f) $$&#xA;  where&#xA;  $$ d_n(f) = \inf_{q_n \in P_n} || f - q_n ||_{\infty} $$ &#xA;  is the error by the best uniform approximation polynomial&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I.e. if $\Lambda_n$ is small the error of the interpolation is not far from the error of the best uniform approximation and the theorem compares the interpolation error with the smallest possible error which is the error of best uniform approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this the behavior of the interpolation depends by the nodes distribution.&#xA;There is a lower bounds about $\Lambda_n$ that given a node distribution exist a constant $c$ such that:&#xA;$$ \Lambda_n \geq \frac{2}{\pi} \log(n) - c $$&#xA;so the constant grows, but how it grow is importan.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;strong&gt;equi-spaced nodes&lt;/strong&gt;&#xA;$$\Lambda_n \approx \frac{2^{n+1}}{en \log(n)} $$&#xA;I omitted some details, but we see that the grow is exponential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For  &lt;strong&gt;Chebyshev nodes&lt;/strong&gt;&#xA;$$\Lambda_n \leq \frac{2}{\pi} \log(n) + 4 $$&#xA;also here I omitted some details, there are more accurate and complicate estimate. See [1] for more details.&#xA;Note that nodes of Chebyshev family have got logarithmic grow and from the previous estimates is near the best you can obtain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other nodes distributions see for example table 1 of &lt;a href=&quot;https://arxiv.org/pdf/1305.6104.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There are a lot of reference on book about interpolation.&#xA;On-line these &lt;a href=&quot;http://www.lorene.obspm.fr/school/polynom.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;slides&lt;/a&gt; are nice as resume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also &lt;a href=&quot;https://link.springer.com/article/10.1186/s13660-016-1030-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;this open article&lt;/a&gt; ([1])&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ac.els-cdn.com/S0898122199002047/1-s2.0-S0898122199002047-main.pdf?_tid=aa1076bd-ab9f-4311-8878-b15f995380d8&amp;amp;acdnat=1521395872_d486741448589112579b35553af71e52&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Numerical Seven Grids Interpolation Comparison of for polynomial on the Interval&lt;/a&gt;  for various comparisons.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2018-03-18T18:05:47.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29074" PostTypeId="2" ParentId="29012" CreationDate="2018-03-18T19:58:07.060" Score="1" Body="&lt;p&gt;The solution was to use &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dealii.org/8.5.0/doxygen/deal.II/step_23.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dealii.org/8.5.0/doxygen/deal.II/step_23.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and to be vigilant of rogue minus signs. &lt;/p&gt;&#xA;" OwnerUserId="27115" LastActivityDate="2018-03-18T19:58:07.060" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29075" PostTypeId="1" CreationDate="2018-03-19T07:32:44.633" Score="0" ViewCount="226" Body="&lt;p&gt;I'm trying to model a projectile motion, considering every detail such as projectile aerodynamics, effects of wind, air temperature etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a good software to model it in with good graphics? I like how COMSOL multiphysics works and it's parameter sweep. It could be used to find the optimum angle for longest range and many other things but I don't know if I can model a projectile motion in it.&lt;/p&gt;&#xA;" OwnerUserId="23599" LastEditorUserId="23599" LastEditDate="2018-03-19T08:39:56.540" LastActivityDate="2018-03-19T09:14:47.913" Title="Simulation of a complicated projectile motion" Tags="|comsol|solid-mechanics|modeling|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29076" PostTypeId="2" ParentId="29075" CreationDate="2018-03-19T09:14:47.913" Score="4" Body="&lt;p&gt;You need to know which equations you need to solve inc initial conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMO, a disadvantage of click&amp;amp;result software is that it's not transparent to what you are actually solving. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why don't you try solving the equations using a ODE solver in Python (using SciPy) and visualize using Matplotlib?&#xA;At least you will have exact control over what you are solving.&#xA;It's trivial to do a parameter sweep using a couple of loops in Python.&lt;/p&gt;&#xA;" OwnerUserId="18017" LastActivityDate="2018-03-19T09:14:47.913" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29079" PostTypeId="1" CreationDate="2018-03-19T14:22:16.037" Score="0" ViewCount="310" Body="&lt;p&gt;This question is mainly about hardware, but also about software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my current work I have approximately 68 millions of combinations that I am iterating through, in parallel. For each of those combinations - there is processing involved on the same data set. The idea is very similar to how neural networks train, I guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current problem is, that it takes 10 hours to process all combinations. I am using the &lt;code&gt;C#&lt;/code&gt; and the code can be simplified to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;long combinationCount = 68,211,200; // (68m)&#xA;List&amp;lt;Data&amp;gt; data = new List&amp;lt;Data&amp;gt;();&#xA;long dataCount = data.Count(); // 65,800 (65k)&#xA;// itterate every combination&#xA;Parallel.For(0, combinationCount, new ParallelOptions { MaxDegreeOfParallelism = 4}, (index, loopState) =&amp;gt; {&#xA;    for(int i = 0; i &amp;lt; dataCount; i++) {&#xA;        // proccess each data record&#xA;    }&#xA;});&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My current hardware is an HP laptop with processor &lt;code&gt;i7-6500U&lt;/code&gt; (2 cores, 4 threads). Quite old perhaps not the best choice for these kinds of tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is - how do I optimize and improve the performance of my applications? To be more precise - I would like to hear out ideas on following topics:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;CPU&lt;/code&gt;. Since I am planning on purchasing a stationary PC - would it be best to make a choice in flavor of:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;AMD Ryzen 1700/1700x/1800x&lt;/code&gt; - 8 cores, 16 threads&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Intel's i7 8700/8700k&lt;/code&gt; - 6 core, 12 threads, higher frequency per core&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Please note!&lt;/em&gt; The question is not about &lt;code&gt;AMD&lt;/code&gt; or &lt;code&gt;Intel&lt;/code&gt;, but more about whether it is better to have 8 physical cores at a lower clock speed than, a CPU with 6 cores, but a higher clock speed. &lt;em&gt;Specifically for programming&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;code&gt;GPU&lt;/code&gt;. It has come to my attention, that you can leverage calculations to &lt;code&gt;GPU&lt;/code&gt; cores instead of &lt;code&gt;CPU&lt;/code&gt;. This in theory will increase the performance X-fold. The issue is that you need to use a lower-level programming language such as &lt;code&gt;C++&lt;/code&gt; to make advantage of that. Since &lt;code&gt;C#&lt;/code&gt; is the only toolset in my possession - I was wondering whether it is worth investing into a better &lt;code&gt;Video card&lt;/code&gt; instead on an &lt;code&gt;CPU&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have googled a bit and found a project &lt;code&gt;altimesh Hybridizer&lt;/code&gt; which seems to allow to code on &lt;code&gt;C#&lt;/code&gt; and compile it in such a way that a &lt;code&gt;GPU&lt;/code&gt; will run it. The idea is neat. If the above is true - then would that make More sense to purchase a higher-end GPU instead? Overall, how much of a performance increase can you expect from calculations being executed on GPU (CUDA cores?) as opposed to 6/8 core CPU? If it matters - I am looking at a &lt;code&gt;GTX 1060 3GB/6GB&lt;/code&gt; price range video card.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And again. I am sorry if this is not the best place to ask such question. My main concern is building a system that would sustain nowadays requirements for computations, be it AI/neural networks/ or huge data processing, as in my case &lt;/p&gt;&#xA;" OwnerUserId="27209" OwnerDisplayName="Alex" LastEditorUserId="27209" LastEditDate="2018-03-19T18:20:49.047" LastActivityDate="2018-03-22T21:02:15.067" Title="CPU and GPU influence on task parallel execution performance" Tags="|optimization|algorithms|performance|cuda|opencl|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29080" PostTypeId="1" AcceptedAnswerId="29081" CreationDate="2018-03-19T18:11:30.240" Score="0" ViewCount="2723" Body="&lt;p&gt;I am trying to use &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/optimize.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.optimize.minimize&lt;/a&gt; to minimise a quadratic objective function: $f(x) =x^\top Q x$. As a start, I have successfully implemented this using the built-in Nelder-Mead Simplex algorithm, by defining a function: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def objective(x):&#xA;    Q = np.asmatrix(DF.cov())       # Covariance matrix&#xA;    x = np.asmatrix(x)&#xA;    return x.transpose() * Q * x &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is passed to SciPy's minimise function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minimize(objective, x0, method='Nelder-Mead',options={'xtol': 1e-6, 'disp': True})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, following the documentation at the above link, there are a number of other optimisation routines available, that I would like to try. Each of these require the calculation of the function derivative, $\nabla f(x)$, which must be written inside a python function similar to the above, and some require the Hessian $\nabla^2f(x)$. My initial attempt is below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def der_objective(x):&#xA;    Q = np.asmatrix(DF.cov())       # Covariance matrix&#xA;    x = np.asmatrix(x).reshape(len(Q),1)&#xA;    return 2 * Q * x &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this obviously returns the wrong dimensions. I'm not entirely sure how SciPy expects the result, and couldn't work it out from the Rosenbrock example in the tutorial &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please explain how the rosen_der example in the SciPy tutorial works (e.g. Why does it take the derivative with respect to a given $x_j$? Why does it return a vector of the same shape as $x$?) and how I should design my der_objective(x) function to achieve the same thing?&lt;/p&gt;&#xA;" OwnerUserId="25837" LastEditorUserId="20417" LastEditDate="2018-03-20T13:05:44.070" LastActivityDate="2018-03-20T13:05:44.070" Title="How to define the derivative for Scipy.Optimize.Minimize" Tags="|optimization|python|convex-optimization|scipy|quadratic-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29081" PostTypeId="2" ParentId="29080" CreationDate="2018-03-19T20:26:33.513" Score="1" Body="&lt;p&gt;If you ignore the mathematical formulae in the tutorial you link to, and just look at the call itself,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;res = minimize(rosen, x0, method='BFGS', jac=rosen_der,&#xA;  ...                options={'disp': True})&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are two python functions defined. One is the &lt;em&gt;scalar&lt;/em&gt; functional, $f(\vec{x})$,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;def rosen(x):&#xA;  ...     &quot;&quot;&quot;The Rosenbrock function&quot;&quot;&quot;&#xA;  ...     return sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and the other is the &lt;em&gt;vector&lt;/em&gt; gradient function, $$\nabla f(\vec{x}) = \left(\left.\frac{\partial f}{\partial x_0}\right|_{\vec{x}},\ldots,\left.\frac{\partial f}{\partial x_{n-1}}\right|_{\vec{x}}\right)^T,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is automatically the same size as $\vec{x}$.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;def rosen_der(x):&#xA;  ...     xm = x[1:-1]&#xA;  ...     xm_m1 = x[:-2]&#xA;  ...     xm_p1 = x[2:]&#xA;  ...     der = np.zeros_like(x)&#xA;  ...     der[1:-1] = 200*(xm-xm_m1**2) - 400*(xm_p1 - xm**2)*xm - 2*(1-xm)&#xA;  ...     der[0] = -400*x[0]*(x[1]-x[0]**2) - 2*(1-x[0])&#xA;  ...     der[-1] = 200*(x[-1]-x[-2]**2)&#xA;  ...     return der&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In terms of the code, this is deliberately written in the same form as $\vec{x}$ so that the iterative minimisation routine can form objects like $$\vec{x}^{(n+1}=\vec{x}^n-\alpha \nabla f(\vec{x}^n) $$ so that the variable can move &quot;downhill&quot; towards a local minimum through successive iterations of the vector $\vec{x}$ (note just using the last answer is the method &lt;a href=&quot;https://en.wikipedia.org/wiki/Gradient_descent&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;steepest descent&quot;&lt;/a&gt;, which nobody would ever use choose to use outside of a classroom). Hopefully this isn't too much of a surprise to you. If it is, you might want to read up a bit more on the theory of gradient based optimisation before getting back to your coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of your actual code, it should be enough to use basic arrays with numpy.dot to perform matrix multiplication,rather than writing everything as matrices (which numpy makes 2d), or if you really want to keep this style in your functions, use &lt;code&gt;numpy.asarray&lt;/code&gt; and &lt;code&gt;numpy.ravel&lt;/code&gt; when returning the result. At a worst case, you need to double check that your matrix $Q$ is $N\times N$ and that you make $x$ a column vector (which as I said before &lt;code&gt;asmatrix&lt;/code&gt; doesn't do).&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2018-03-19T20:26:33.513" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29082" PostTypeId="1" CreationDate="2018-03-19T21:24:44.503" Score="3" ViewCount="379" Body="&lt;p&gt;I'm trying to find both the dominant $k$ left &lt;i&gt;and&lt;/i&gt; right eigenvectors, that is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$V_L\mathcal{A} = \Lambda V_L\\&#xA;\mathcal{A}V_R = V_R\Lambda\\&#xA;V_LV_R = I_{k\times k}$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;  &lt;li&gt; $V_L$ being the $k\times N$ matrix of left eigenvectors,&#xA;  &lt;li&gt; $V_R$ being the $N\times k$ matrix of right eigenvectors, and&#xA;  &lt;li&gt; $\Lambda$ being the diagonal $k\times k$ matrix of eigenvalues.&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm using ARPACK (via &lt;code&gt;scipy.sparse.linalg&lt;/code&gt;). &lt;code&gt;scipy.sparse.eigs&lt;/code&gt; gives me  the right eigensystem $V_R$ and $\Lambda$; I've tried two methods to get $V_L$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA; &lt;li&gt;&#xA;  &lt;ol&gt;&#xA;   &lt;li&gt;Apply &lt;code&gt;scipy.sparse.linalg.eigs&lt;/code&gt; to $\mathcal{A}^T$ to get left eigenvectors and eigenvalues (call them $V_{L,0}$ and $\Lambda_L$)&lt;/li&gt;&#xA;   &lt;li&gt;If $V_{L,0}V_R$ is singular or $\Lambda_L$ is too different from $\Lambda$, reduce $k$; if that doesn't work, restart&lt;/li&gt;&#xA;   &lt;li&gt;Set $V_L \leftarrow (V_{L,0}V_R)^{-1}V_{L,0}$; or&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA; &lt;i&gt;In principle&lt;/i&gt; this is reasonably fast (2x the running time of just getting the right eigenvectors), but in practice, in the cases where it needs to be restarted, it may need to re-run many times.&lt;/li&gt;&#xA; &lt;li&gt; For each of the $k$ eigenvalues $\lambda_i$, apply &lt;code&gt;scipy.sparse.linalg.lsmr&lt;/code&gt; to the overdetermined system&#xA;$$\lambda_i^{-1}v_L^{(i)}\mathcal{A} = v_L,\\&#xA;v_L^{(i)}v_R^{(j)} = \delta_{i,j}$$&#xA; This is much more &lt;i&gt;reliable&lt;/i&gt; than the other method, but it has a much worse best-case running time and it also has a quite limited accuracy.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I imagine it should in principle be possible to get left and right eigenvectors via IRAM; is this available in some way on ARPACK? (As mentioned I've been using &lt;code&gt;scipy.sparse.linalg&lt;/code&gt;, but I'm amenable to accessing ARPACK more directly as a last resort). Is there some way to repackage this approximate eigendecomposition as something that ARPACK &lt;em&gt;can&lt;/em&gt; handle (sort of like the $AA^\dagger$ trick for SVD)? Or is there some other way to get $V_L$, after having gotten $V_R$ and $\Lambda$, that's reasonably efficient, reliable, and accurate?&lt;/p&gt;&#xA;&#xA;&lt;hr/&gt;&#xA;&#xA;&lt;p&gt;On the matrices I'm trying to diagonalize:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;  &lt;li&gt; Complex, non-Hermitian (also not symmetric, unitary, normal, etc.)&#xA;  &lt;li&gt; Represented via &lt;code&gt;scipy.sparse.linalg.LinearOperator&lt;/code&gt;, not technically a sparse matrix&#xA;  &lt;li&gt; Equally efficient to perform left- and right- vector multiplication&#xA;  &lt;li&gt; Magnitude of eigenvalues should theoretically decay fast (between polynomially and exponentially); initial eigenvalues tend to decay even faster&#xA;  &lt;li&gt; For an example see &lt;a href=&quot;https://arxiv.org/abs/1510.00689&quot; rel=&quot;nofollow noreferrer&quot;&gt;arXiv:1510.00689&lt;/a&gt;; operator is defined in Fig. 2 and eigenvalue problem is described in Fig. A.4.b.&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27210" LastActivityDate="2018-03-19T21:24:44.503" Title="Left eigenvectors using ARPACK" Tags="|linear-algebra|sparse-matrix|eigensystem|scipy|arpack|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29083" PostTypeId="1" AcceptedAnswerId="29210" CreationDate="2018-03-19T23:44:49.580" Score="1" ViewCount="460" Body="&lt;p&gt;I usually work with Python, but my basic knowledge of c++ allows me to switch when I need to increase the speed of my code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I have a python script that (among other things) computes the mean and standard deviation of all principal minors (of a specified order) of a square matrix. Minors are determinants of submatrices (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Minor_(linear_algebra)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; for definition). Even a matrix with 30x30 entries has more than a billion different submatrices and thus more than a billion different minors. So even tiny time differences in the computation of minors matter a lot to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python/NumPy uses LAPACK routines for creating submatrices and computing determinants. So I assumed that it would be best to use the same routines in c++. However, it seems that more experienced programmers tend to dissuade beginners from using the LAPACK library in c++ (see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26395/how-to-start-using-lapack-in-c?newreg=a3f9d7f0242e40ef94fc5b4660988952&quot;&gt;other question on stackexchange&lt;/a&gt;) and instead point to other packages, such as Armadillo or Eigen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, Armadillo and Eigen are libraries that again rely on LAPACK, but I don't know if these libraries generate some kind of overhead that might slow things down. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: Should I expect that I pay for Armadillo's or Eigen's &quot;beginner friendliness&quot; with an increase in computation time (if compared to LAPACK)? If so, how much? &lt;/p&gt;&#xA;" OwnerUserId="27212" LastActivityDate="2018-04-01T18:12:59.153" Title="C++: Efficient library for dense linear algebra operations (determinant &amp; principal minors)" Tags="|c++|lapack|eigen|armadillo|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29084" PostTypeId="1" AcceptedAnswerId="29088" CreationDate="2018-03-20T08:40:39.250" Score="3" ViewCount="287" Body="&lt;p&gt;I was suggested to move that &lt;a href=&quot;https://stackoverflow.com/questions/49366122/lax-wendroff-order-of-accuracy-on-burgers-equation&quot;&gt;question&lt;/a&gt; here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question to be as follows.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Statement of the problem&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Is it possible to achieve the second order of convergence (OOC) of &lt;a href=&quot;https://en.wikipedia.org/wiki/Lax%E2%80%93Wendroff_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lax-Wendroff &lt;/a&gt; (LxW) scheme applied to solve &lt;a href=&quot;https://en.wikipedia.org/wiki/Burgers%27_equation#Inviscid_Burgers&amp;#39;_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;inviscid Burgers&lt;/a&gt; equations with discontinuous initial data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If no, then how to achieve (and prove numerically) OOC of 2nd order (of LxW scheme) in quasilinear problem?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Wolfram Mathematica code&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(*Initial data*)&#xA; u0[x_] := 1 - UnitStep[x - 0.1];&#xA;Flux[u_] := 0.5 u^2;&#xA;u = u0[xTbl];&#xA;dt = Abs[\[Sigma]] /Abs[Max[u]] dx;&#xA;un = 0*u;&#xA;F = Flux[u];&#xA;&#xA;(*LW*)&#xA;t = 0;&#xA;While[t &amp;lt; tFin,&#xA;(*Main loop*)&#xA;un[[2 ;; nx - 1]] = u[[2 ;; nx - 1]] -&#xA;  0.50 \[Sigma] (F[[3 ;; nx]] - F[[1 ;; nx - 2]]) +&#xA;  0.25 \[Sigma]^2 ((u[[3 ;; nx]] + u[[2 ;; nx - 1]]) (F[[3 ;; nx]] - &#xA;     F[[2 ;; nx - 1]]) - (u[[2 ;; nx - 1]] + &#xA;     u[[1 ;; nx - 2]]) (F[[2 ;; nx - 1]] - F[[1 ;; nx - 2]]));&#xA;(*BC*)&#xA;un[[1]] = u[[1]];&#xA;un[[nx]] = u[[nx]];&#xA;(*Update*)&#xA;u = un;&#xA;F = Flux[u];&#xA;nstep = nstep + 1;&#xA;t = t + dt&#xA;];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Output&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nx = {50, 100, 200, 400, 800};&#xA;L1err = {0.0217352, 0.0107321, 0.00533915, 0.00207726, 0.00132978};&#xA;p = {1.0181, 1.00725, 1.36192, 0.643502}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Average OOC equals&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.00769&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21795" LastActivityDate="2018-03-20T12:26:23.513" Title="Numerical Lax-Wendroff scheme order of convergence on Burgers equation" Tags="|finite-difference|finite-volume|convergence|numerics|conservation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29085" PostTypeId="2" ParentId="28519" CreationDate="2018-03-20T09:22:45.023" Score="4" Body="&lt;p&gt;Simulated annealing comes from computations in statistical mechanics. When I think of simulated annealing, I very much think in terms of physics: I want to minimize some potential energy function that depends on the configuration, and whose global minimum corresponds to an optimal solution of your problem. At high temperature, there is enough kinetic energy available to go against the potential gradient, i.e. you are more likely to move toward a configuration of higher potential energy than when the temperature is low. If you think about a mountainous landscape with some marbles in it, and you shake it like crazy, the marbles may move in all directions, often uphill. If you only slightly shake it, marbles in local minima will slightly move around the minimum, only when the minimum is very shallow it may move out and roll down to further lower the potential energy. If you start at high temperature and slowly go down, the marbles will initially be shaken out of (and into) even very deep local minima, but later on they will not return to minima for which a high potential barrier has to be overcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In simulated annealing this time factor is absent, but otherwise the idea is the same. Instead of the next configuration in terms of the system's time evolution, a general nearby configuration (in any sense) is considered and accepted or rejected with the appropriate, temperature dependent, probability. When the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ergodic_hypothesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;ergodic hypothesis&lt;/a&gt; is assumed for the system, in the long run you will, roughly speaking, find the same configurations, but for practical applications you don't need to bother with such technicalities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In statistical mechanics, the transition probability would be based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Boltzmann_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boltzmann distribution&lt;/a&gt; of configurations, but again, if you just want to optimize a wide range of transition probabilities will do the trick for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After all these digressions, let's assume that your transition probability as a function of the energies of the two configurations and the temperature is given. Then there are two steps left where you can deal with your particular constraints: the generation of neighboring states, and the definition of the energy function. If you have a hard constraint on state, i.e. some states are strictly forbidden, you could directly exclude their generation as candidate neighbors. You may even find a way to directly generate random neighboring states following a distribution that takes their energy into account.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather than the sampling of neighbors, the obvious step in which to take care of your constraints is in the definition of the energy function, as André said in his comment. Here you can exclude configurations in a straightforward way, by assigning an infinite energy to them; if you assign a very high energy, they will be essentially inaccessible except at very high temperature, allowing you to essentially &quot;tunnel&quot; through them (if you need to allow that). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a reference for algorithms in statistical mechanics, which explains the context of simulated annealing in an excellent way, I recommend &lt;a href=&quot;https://books.google.co.uk/books?id=g30SDAAAQBAJ&amp;amp;lpg=PP1&amp;amp;pg=PP1#v=onepage&amp;amp;q&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;this book by Werner Krauth&lt;/a&gt; and the &lt;a href=&quot;https://www.coursera.org/learn/statistical-mechanics&quot; rel=&quot;nofollow noreferrer&quot;&gt;companion online course&lt;/a&gt;. If you have a physical analogue for your optimization problem, then this can directly guide you in assigning energies to configurations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if this addresses your question, but I hope it is helpful anyway.&lt;/p&gt;&#xA;" OwnerUserId="5521" LastActivityDate="2018-03-20T09:22:45.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29086" PostTypeId="1" CreationDate="2018-03-20T10:23:56.347" Score="1" ViewCount="77" Body="&lt;p&gt;I have two sets of magnets. One set consists of two electromagnets (Shown below: Blue) and the other set consists of two NdFeB N40 permanent magnets. Both sets of magnets lie on the same plane. I want to plot the force of attraction/ repulsion between the two sets of magnets as the distance between them varies from 10cm to 100cm (Y-axis below is in mm - and shows the initial condition of 10cm (100mm) separation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3ogfl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3ogfl.png&quot; alt=&quot;Two electromagnets in Blue and two permanent NdFeB Magnets - all lie on the same plane&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have modeled this setup using Radia &lt;a href=&quot;http://www.esrf.eu/Accelerators/Groups/InsertionDevices/Software/Radia&quot; rel=&quot;nofollow noreferrer&quot;&gt;Free software found here&lt;/a&gt;. You must have Mathematica (I am currently using version 11.2) to use Radia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem: When Radia solves the model and produces an output of Force vs. Distance I expect ALL the force measurements to be EITHER positive OR negative - meaning either attraction or repulsion between the two sets of magnets. What actually happens though is after about 50cm separation Radia produces a mixture of positive and negative force results. The First plot looks accurate (yellow and blue lines) but on closer inspection some of the values are negative and this becomes glaringly obvious when the Force axis (Y) is made logarithmic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l9wXS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l9wXS.png&quot; alt=&quot;Force (N) vs. Distance (cm)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the logarithmic version:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0rqQV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0rqQV.png&quot; alt=&quot;Force (N - logarithmic) vs. Distance (cm)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Included in the graph (orange) is the solution I obtained (Force vs. Distance) from a 2D magnetostatics software called FEMM. This is purely a reference (which I believe is what the yellow and blue lines should look like especially in the logarithmic version) showing a Force&gt;0 for all distances from 10cm to 100cm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know where I am going wrong and would really appreciate some help. I have read on the Radia documentation that correct subdivision is crucial - especially to iron components. I have tried to subdivide my iron core using &lt;code&gt;radObjDivMag[]&lt;/code&gt; &lt;a href=&quot;http://www.esrf.eu/Accelerators/Groups/InsertionDevices/Software/Radia/Documentation/ReferenceGuide/Index&quot; rel=&quot;nofollow noreferrer&quot;&gt;Radia Functions&lt;/a&gt; but I can't get it to work (keep in mind I used the same function on a bar magnet (cuboid) and the function worked perfectly, I have also tried to use the function argument {&quot;cyl&quot;,....} but it will still not subdivide my component, if you know where I am going wrong please comment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to attach my Mathematica notebook (heavily commented however if you have any questions please ask me) which I used to create and solve the model and the Excel spreadsheet used to plot my data but don't seem to be able to - Please message me and I can send it to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone used Radia before? Should I be using different software?&lt;/p&gt;&#xA;" OwnerUserId="27220" LastEditorUserId="20688" LastEditDate="2018-03-20T13:10:41.030" LastActivityDate="2018-03-20T13:10:41.030" Title="Magnetostatic modelling Radia: Increasing distance between magnets a produces positive force until a certain point, beyond which force goes haywire" Tags="|simulation|software|electromagnetism|software-recommendation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29087" PostTypeId="1" CreationDate="2018-03-20T12:19:36.423" Score="3" ViewCount="1123" Body="&lt;p&gt;I know L-BFGS-B only supports simple box constrains of the form: $l_i \leq x_i \leq u_i$, where $l_i$ and $u_i$ are constants. For my specific optimization problem, I need to specify some simple linear constraints of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{i-1} + k \leq x_i \leq x_{i+1}-k $ where $k&amp;gt;0$ is a fixed constant. Essentially, I need each variable $x_i$ to stay in-between (and within a certain distance away from) the two variables at index $i-1$ and $i+1$. Is it possible to achieve this with L-BFGS-B?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can the $l$ and $u$ constraint vectors be modified after each iteration? One silly idea I have tried (and seems to work, although I have not extensively tested it) is to modify $l$ and $u$ after each iteration with this rule:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$l_i = x_{i-1} + k$$&#xA;$$u_i = x_{i+1} - k$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; I forgot to mention that the first variable ($x_0$) is also subject to a constant lower bound and the last variable ($x_{N-1}$) is subject to an constant upper bound:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_0 \geq k$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_{N-1} \leq k_{max}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes recasting the problem into a basis where the unknowns are the differences of two $x$ variables not feasible since these two extra bounds (which in the current system are actual box constraints) would then become linear constraints.&lt;/p&gt;&#xA;" OwnerUserId="1480" LastEditorUserId="1480" LastEditDate="2018-03-21T03:48:11.110" LastActivityDate="2018-03-21T03:48:11.110" Title="Linear constraints for L-BFGS-B" Tags="|optimization|constrained-optimization|conjugate-gradient|constraints|quasi-newton|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29088" PostTypeId="2" ParentId="29084" CreationDate="2018-03-20T12:19:40.857" Score="2" Body="&lt;p&gt;Probably not. If you have inviscid Burgers equation then your discontinuous initial condition should (somewhere) stay discontinuous because there is no viscosity. If there is discontinuity in the solution then there is Godunov's order barrier theorem which limits your convergence order to 1...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be careful to use fine enough grid to prove any order of convergence, i.e. when you plot nx-Lerror graph in log-log scale you should get straight line (except for rough discretization). I'm not sure but if there are some problems with L1 norm try L2 or Linfinity norm... &lt;/p&gt;&#xA;" OwnerUserId="27223" LastEditorUserId="27223" LastEditDate="2018-03-20T12:26:23.513" LastActivityDate="2018-03-20T12:26:23.513" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29090" PostTypeId="1" CreationDate="2018-03-20T13:39:41.620" Score="8" ViewCount="4477" Body="&lt;p&gt;We can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Random_optimization&quot; rel=&quot;noreferrer&quot;&gt;random optimization/ derivative free/ direct search&lt;/a&gt; to find the minimum of some black box function $f$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have some 2D black box function, $f(x,y)$ - which I know to be convex -  what is the best derivative-free method to use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. from a speed perspective is it best to use e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method&quot; rel=&quot;noreferrer&quot;&gt;Nelder-Mead&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Pattern_search_(optimization)&quot; rel=&quot;noreferrer&quot;&gt;Compass/Pattern search&lt;/a&gt;, &lt;a href=&quot;https://www.gerad.ca/~charlesa/PUB/mads2004jopt.pdf&quot; rel=&quot;noreferrer&quot;&gt;MADS&lt;/a&gt; etc?&lt;/p&gt;&#xA;" OwnerUserId="26961" LastEditorUserId="20688" LastEditDate="2019-06-22T22:44:21.590" LastActivityDate="2021-02-18T22:36:44.770" Title="What is the most appropriate derivative free optimization algorithm" Tags="|optimization|convex-optimization|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29091" PostTypeId="2" ParentId="29087" CreationDate="2018-03-20T14:33:37.847" Score="2" Body="&lt;p&gt;One approach to this problem is to reparameterize your problem in terms of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{1}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$z_{i}=x_{i}-x_{i-1}$, $i=2, \ldots, n-1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then rewrite your objective function in terms of the the $z_{i}$ variables by substituting &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$x_{i}=x_{1}+z_{1}+\ldots + z_{i}$, $i=2, 3, \ldots, n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your separation constraints become &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$k \leq z_{i}$&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-03-20T14:33:37.847" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29092" PostTypeId="5" CreationDate="2018-03-20T16:01:30.030" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-20T16:01:30.030" LastActivityDate="2018-03-20T16:01:30.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29093" PostTypeId="4" CreationDate="2018-03-20T16:01:30.030" Score="0" Body="Questions regarding the numerical solution and analysis of equations that feature an integral transform on the unknown function. Problems with integral-equation formulations, their discretization, calculation and usage of Green's functions, eigenvalue analysis of the integral operators, and software recommendations should be marked with this tag." OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-03-20T17:03:15.443" LastActivityDate="2018-03-20T17:03:15.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29094" PostTypeId="2" ParentId="7199" CreationDate="2018-03-20T17:11:24.850" Score="2" Body="&lt;p&gt;This is an old question and it does not contain enough information to answer it with certainty. But since I already stumbled on it once, I want to propose a list of actions which should be able to help resolve it with all the uncertainties.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The very first action to do is to check if the original matrix itself contains valid entries. &lt;code&gt;hasNaN()&lt;/code&gt; and &lt;code&gt;allFinite()&lt;/code&gt; methods will help.&lt;/li&gt;&#xA;&lt;li&gt;Incomplete LU preconditioner provides the &lt;code&gt;info()&lt;/code&gt; function, that will answer the question if the computation of ILU was successful.&lt;/li&gt;&#xA;&lt;li&gt;One can try playing with the ILU parameters, such as tolerance and fill-factor via &lt;code&gt;setDropTol()&lt;/code&gt; and &lt;code&gt;setFillfactor()&lt;/code&gt; and see if it results in a different &lt;code&gt;info()&lt;/code&gt; outcome or helps the iterative solution process.&lt;/li&gt;&#xA;&lt;li&gt;It is also unclear if Eigen is linked against some BLAS/LAPACK library (say, Intel MKL) and/or sparse solver (say, SuperLU). Turning those options on/off can certainly help to identify the source of the problem. The cleanest run, where Eigen is linked against the reliable BLAS/LAPACK without any additional libraries should be expected to be successful, provided 1-3 are alright.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Next, it is unclear, if the system being solved is &lt;em&gt;sparse&lt;/em&gt; or &lt;em&gt;dense&lt;/em&gt;.&#xA;If the original matrix is dense, there might be a memory problem. For $N=200$, dense matrix together with the preconditioner will take only a few megabytes; however, in case of a dense matrix, $N=20000$, large fill-in and tolerance in the preconditioner, one can certainly exceed the memory, say, while trying to &quot;invert&quot; the ILU.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;Check the memory usage and requirements for your problems.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If something very peculiar happens, it is very interesting to understand what kind of beast we are trying to solve. Then:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Plot the singular values for your matrix&lt;/li&gt;&#xA;&lt;li&gt;Calculate the condition number&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The same debugging algorithm generally should apply to other similar problems with slight variations on the preconditioner parameters.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-20T17:11:24.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29097" PostTypeId="1" CreationDate="2018-03-20T19:20:56.433" Score="1" ViewCount="37" Body="&lt;p&gt;I have two models which I would like to investigate for my data.  One form is:&#xA;\begin{equation}&#xA;\label{one}&#xA;    f(r) = A e^{-B r}&#xA;\end{equation}&#xA;and the second is:&#xA;\begin{equation}&#xA;\label{two}&#xA;    g(r) = \frac{A}{r^{C}} e^{-B r}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any advice on how to tell the difference between these two?  I typically have about ~ seven points that I can fit in all my situations.  The problem is these seven points don't span a particularly large range of $r$ and so it's hard to see the difference between $g$ and $f$.  The least-squares fits in the case of $g$ are more unstable, but I assume it's because it's very similar to $f$ and the parameters have more freedom...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advise or wisdom would be welcome, and I can clarify if there are any questions.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="9667" LastEditDate="2018-03-20T19:21:54.480" LastActivityDate="2018-03-20T19:21:54.480" Title="fitting exponential versus exponential w/ power" Tags="|regression|least-squares|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="29098" PostTypeId="1" CreationDate="2018-03-20T20:20:54.040" Score="1" ViewCount="149" Body="&lt;p&gt;I have the discrete likelihood estimation problem $\max \sum m_i\log p_i $ where $m$ is a given vector of length $n$. The constraints are $0 \preceq p \preceq 1$, $\sum_{i=1}^n p_i = 1, $ and one final constraint which is not convex: $$ \log p_i \geq \frac{1}{2}(\log p_{i-1} + \log p_{i+1}); \quad i = 2, \dots, n-1. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This constraint ensures that the probability distribution is log concave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to make this constraint something I can put into a convex program. I've tried some basic substitutions and combining terms, but that hasn't proven fruitful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I write $x_i = \frac{p_i^2}{p_{i-1}p_{i+1}}$, then I don't know how I'd rewrite the objective in terms of the $x_i$'s.&lt;/p&gt;&#xA;" OwnerUserId="27228" LastEditorUserId="27228" LastEditDate="2018-03-20T20:50:33.340" LastActivityDate="2018-03-21T13:35:15.733" Title="Making difference of log constraints convex" Tags="|convex-optimization|constrained-optimization|constraints|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29099" PostTypeId="2" ParentId="29098" CreationDate="2018-03-20T21:09:38.137" Score="2" Body="&lt;p&gt;The constraints are not convex. Consider the example below in which x1 and x2 are each vectors of 3 elements which satisfy the inequality in question, as shown.  0.5(x1 + x2) does not satisfy the inequality, thereby proving it is not convex.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; x1 = [0.868417827606570   0.121582145814843 0.010000025679806];&#xA;&amp;gt;&amp;gt; x2 = [0.017508300926335 0.264007818119070 0.718483881445100];&#xA;&amp;gt;&amp;gt; x3 = 0.5*(x1 + x2)&#xA;x3 =&#xA;   0.442963064266453   0.192794981966956   0.364241953562453&#xA;&amp;gt;&amp;gt; log(x1(2)) - 0.5*(log(x1(1)) + log(x1(3)))&#xA;ans =&#xA;   0.265959817560572&#xA;&amp;gt;&amp;gt; log(x2(2)) - 0.5*(log(x2(1)) + log(x2(3)))&#xA;ans =&#xA;   0.856069527413664&#xA;&amp;gt;&amp;gt; log(x3(2)) - 0.5*(log(x3(1)) + log(x3(3)))&#xA;ans =&#xA;  -0.734025017605155&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In order to use a convex programming method, you could do something like Difference of Convex Functions Programming.  That would entail doing a first order (linear) expansion of the right-hand-side terms in the non-convex constraint about their current (or an initial) value.  You can only use first order expansion because the quadratic term would go in the wrong direction relative to having an overall convex constraint.  You will need to start the iteration with initial values (expansion points).  Then update the expansion point on each subsequent iteration with the optimal value of the convex program just solved. There is no guarantee of convergence to anything, let alone to a local or global optimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could throw it in a global optimizer, such as BARON, perhaps using a slightly positive lower bound of p so as to avoid difficulties at 0.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2018-03-20T21:09:38.137" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29101" PostTypeId="2" ParentId="27501" CreationDate="2018-03-21T06:06:26.937" Score="2" Body="&lt;p&gt;Using Kahan summation algorithm should suffice your purpose. Having:&#xA;$$&#xA;R=\frac{\sum\limits_{i=1}^{N}a_i}{\sum\limits_{i=1}^{N}b_i}=\frac{A}{B}&#xA;$$&#xA;if $A$ and $B$ are computed accurate enough, the final ratio $R$ should not be a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $A$ and $B$ are computed using Kahan summation, you can expect the relative error to be $\mathcal{O}(\epsilon_\text{mach}+N\epsilon_\text{mach}^2)$. For double-precision and $N\approx 10^{10}$ you are safe ($N\ll 1/\epsilon_\text{mach}$), and your error is practically independent of $N$: $\mathcal{O}(\epsilon_\text{mach})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth to mention, that the constant in front of the error estimate (condition # of summation $\kappa_{\sum}$) can still be relatively large.&#xA;If summation condition number:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\kappa_{\sum_A}=\frac{\sum\limits_{i=1}^{N}|a_i|}{\left|\sum\limits_{i=1}^{N}a_i\right|}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is large, your best bets would probably be in &lt;a href=&quot;http://www.cs.berkeley.edu/~jrs/papers/robustr.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shewchuk's algorithm&lt;/a&gt;. With that, you will be able to calculate your numerator and denominator to full double precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $\tilde{A}$ and $\tilde{B}$ are calculated in full double precision (say using Shewchuk's algorithm):&#xA;$$&#xA;\tilde{A}=A(1+\delta_A),\quad \tilde{B}=B(1+\delta_B),\quad |\delta_A|,|\delta_B|\le\epsilon_\text{mach}&#xA;$$&#xA;, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;\tilde{R}&amp;amp;=\tilde{A}\oslash\tilde{B}&amp;amp;=\frac{A(1+\delta_A)}{B(1+\delta_B)}(1+\delta_{\oslash})=\frac{A}{B}(1+\theta)\\&#xA;&amp;amp;&amp;amp;|\theta|\leq\frac{3\epsilon_\text{mach}}{1-3\epsilon_\text{mach}}&#xA;\end{aligned}&#xA;$$&#xA;Here, $\tilde{A},\tilde{B},\tilde{R}$ are floating point (IEEE-754) representation of $A$, $B$, and $R$, respectively, $\oslash$ - floating point analog of division.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, even if $A$ and $B$ are not calculated to the full precision (Kahan), it will just increase $\delta_A$ and $\delta_B$ (not allowing approximation by $\theta$ with such a tight bound). However, I still do not see how the division can be the problem here. Especially, since the true expected values of $A$ and $B$ are in $[10^{-1}, 10^{1}]$.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-03-21T06:13:18.060" LastActivityDate="2018-03-21T06:13:18.060" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29102" PostTypeId="1" AcceptedAnswerId="29106" CreationDate="2018-03-21T07:01:53.200" Score="7" ViewCount="2873" Body="&lt;p&gt;I know that it is generally a bad idea to compute the inverse matrix directly. However, if it is necessary to compute the inverse of an ill-conditioned invertible dense matrix, then what can I try?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I know that scaling a matrix may decrease the condition number of it. This may be an example of preconditiong, but I don't know much about the theory of preconditioners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also know that the computation technique can be varied depending on properties of the matrix. Questions posted on this site (e.g., &lt;a href=&quot;https://scicomp.stackexchange.com/q/16264/27229&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://scicomp.stackexchange.com/q/3188/27229&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://scicomp.stackexchange.com/q/23272/27229&quot;&gt;3&lt;/a&gt;) have their specific answer, not a general one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I can't find the general principle to compute the inverse because I am new to this field. What properties of the matrix should I check to reduce the condition number? Is there any strategy, or, any books/papers which explain more accurate methods to compute the inverse directly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I don't know if there is a short answer of this. Feel free to close this question as too broad.)&lt;/p&gt;&#xA;" OwnerUserId="27229" LastEditorUserId="27229" LastEditDate="2018-05-31T11:34:17.103" LastActivityDate="2022-04-26T15:31:36.550" Title="How to directly compute the inverse of an ill-conditioned dense matrix" Tags="|reference-request|preconditioning|condition-number|inverse|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29103" PostTypeId="1" CreationDate="2018-03-21T08:10:08.633" Score="1" ViewCount="1376" Body="&lt;p&gt;I was wondering if we could model rigid body motion of bodies using finite element models. Particularly I'm interested to know if we can model motion of objects with no constraints or with some degrees of freedom (such as only rotation). Though I've used word 'rigid body' to convey the situations closest to what I'm considering, In reality there will be stress, considering non-uniform application of force on surface/body or because of rotation they undergo, and these are not small enough to ignore.&lt;/p&gt;&#xA;" OwnerUserId="27122" LastEditorUserId="27122" LastEditDate="2018-03-21T11:11:44.513" LastActivityDate="2018-03-25T17:12:44.240" Title="Can we simulate rigid body motion using finite element analysis?" Tags="|finite-element|computational-physics|solid-mechanics|multiphysics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29104" PostTypeId="1" CreationDate="2018-03-21T09:03:00.387" Score="1" ViewCount="70" Body="&lt;p&gt;Mesh Adaptive Direct Search (&lt;a href=&quot;http://www.optimization-online.org/DB_FILE/2009/07/2352.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MASH&lt;/a&gt;) is an algorithm for &lt;a href=&quot;https://en.wikipedia.org/wiki/Random_optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;black box optimization&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to understand an implement this method to solve some 2D multivariate blackbox function $f(x,y)$, but am having trouble understanding the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I understand so far&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start with some initial trial value $x_0, y_0$&lt;/li&gt;&#xA;&lt;li&gt;Create the 'Mesh'&lt;/li&gt;&#xA;&lt;li&gt;Search a finite number of points on the mesh&lt;/li&gt;&#xA;&lt;li&gt;If the search fails, perform a 'Poll' step $P_k = x_k + \Delta_k d$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Where I am confused&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;How do we 'create' the Mesh? How does the mesh change with each iteration?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do we perform a search step? How do we choose which points on the mesh to search?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What does it mean to perform a poll step? How should $\Delta_k$ change? What is meant by $d$?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for an help&lt;/p&gt;&#xA;" OwnerUserId="26961" LastActivityDate="2018-03-21T09:03:00.387" Title="How does the MADS algorithm work in practice" Tags="|optimization|algorithms|convex-optimization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29106" PostTypeId="2" ParentId="29102" CreationDate="2018-03-21T12:37:32.830" Score="13" Body="&lt;p&gt;Though it is a relatively rare situation when you actually have to calculate an inverse of the matrix, not all techniques were created equally.&lt;/p&gt;&#xA;&lt;p&gt;I would use the term &lt;em&gt;badly-conditioned&lt;/em&gt; instead of &lt;em&gt;ill-conditioned&lt;/em&gt;. For badly conditioned matrices, you might opt in the SVD-route to calculate the inverse:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A=U\Sigma V^H \implies A^{-1}=V\Sigma^{-1}U^H.&#xA;$$&lt;/span&gt;&#xA;If your &lt;span class=&quot;math-container&quot;&gt;$N\times N$&lt;/span&gt; matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is actually &lt;em&gt;badly conditioned&lt;/em&gt;, you still should be able to perform &lt;span class=&quot;math-container&quot;&gt;$\Sigma^{-1}$&lt;/span&gt;. However, you see a problem right away: if the condition number is large (ill-conditioning), the relation of &lt;span class=&quot;math-container&quot;&gt;$\sigma_1/\sigma_N$&lt;/span&gt; is huge (&lt;span class=&quot;math-container&quot;&gt;$\sigma_1,\ldots,\sigma_N$&lt;/span&gt; being the singular values of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, the entries constituting a diagonal matrix &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\sigma_1\ge\sigma_2\ge\ldots\ge \sigma_N$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;This approach also enables (and is close by virtue of &lt;span class=&quot;math-container&quot;&gt;$^{-1}\to ^{+}$&lt;/span&gt;) to use a notion of a pseudo-inverse &lt;span class=&quot;math-container&quot;&gt;$A^{+}$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A^+=V\Sigma^{+}U^H&#xA;$$&lt;/span&gt;&#xA;Now, only non-zero elements of &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt; are &amp;quot;reciprocated&amp;quot;, and you can filter out &lt;em&gt;too small&lt;/em&gt; singular values and corresponding singular vectors based on some tolerance (truncated SVD). Again, that does not help the ill-conditioning of the original problem in any way, just offers a way to compute something that should be avoided in a more accurate way.&lt;/p&gt;&#xA;&lt;p&gt;Again, a more solid answer would be on the topic of avoiding the calculation of the inverse for an ill-conditioned problem.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2022-04-26T15:31:36.550" LastActivityDate="2022-04-26T15:31:36.550" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29107" PostTypeId="2" ParentId="29098" CreationDate="2018-03-21T13:35:15.733" Score="2" Body="&lt;p&gt;Let $z_i = \log p_i$ (i.e. $p_i = e^{z_i}$) and everything changes to linear constraints except the constraint $\sum e^{z_i} = 1$ which you can relax to $\sum e^{z_i} \leq 1$ since it will be tight at optimality (assuming $m_i$ positive).&lt;/p&gt;&#xA;" OwnerUserId="1662" LastActivityDate="2018-03-21T13:35:15.733" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29108" PostTypeId="2" ParentId="29079" CreationDate="2018-03-21T20:25:48.057" Score="1" Body="&lt;p&gt;Both CPUs you list are much more powerful than your existing laptop and assuming you're getting even reasonable parallelism out of the loop should contribute significant speedup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing your specific application I [personal opinion] would likely choose the processor with fewer faster cores since on a wide variety of tasks single core performance is still important (is everything you do parallelized?), and it's easier to scale parallel applications to effectively use smaller numbers of cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At 68,000,000 elements you are well into the size of items that may be amenable to GPU programming.  That said, there are many caveats without knowing the exact problem domain, and many problems fit poorly into GPU architectures.  GPUs are somewhat harder to program effectively -- memory access patterns very important and conditionals/branching statements may be catastrophic to performance.  If the problem turns out to be 68e6 independent mathematical formulas over 32 bit floating point numbers there is potential for &lt;a href=&quot;https://developer.nvidia.com/thrust&quot; rel=&quot;nofollow noreferrer&quot;&gt;many-fold&lt;/a&gt; performance over CPUs.  Be mindful that gaming GPUs are &lt;a href=&quot;http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#arithmetic-instructions&quot; rel=&quot;nofollow noreferrer&quot;&gt;awful at &lt;s&gt;both integer&lt;/s&gt; and double precision&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:  Striking out comment about integer performance - see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/29079/cpu-and-gpu-influence-on-task-parallel-execution-performance/29108?noredirect=1#comment53058_29108&quot;&gt;comments below from njuffa&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="25876" LastEditorUserId="25876" LastEditDate="2018-03-22T21:02:15.067" LastActivityDate="2018-03-22T21:02:15.067" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29109" PostTypeId="1" AcceptedAnswerId="29124" CreationDate="2018-03-21T23:33:22.037" Score="1" ViewCount="161" Body="&lt;p&gt;I used pretty much all my expertise with finite differences to solve an advection-diffusion equation with space-dependent coefficient with a grid in the $x$,$z$ domain with regular spacings. Something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l3TWu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l3TWu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is nice because it is easy to initialize with some experimental measurements that I have. However, near the sloping boundary, I have to carefully choose the grid spacings or sometimes my code behaves funny. This also limits the vertical resolution that I can have very close to the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this problem I was thinking to change reference system by rotating it by an angle $\theta$. Something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qxcSz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qxcSz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so that I could easily increase the resolution near the boundary. What I do not know is if this imposes some limitations on the physics. Is it sufficient to multiply my equations by a rotation matrix in order to be consistent with the new reference system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is it possible (and fairly easy to do) to add triangular cells near the slope and still use finite difference approximations?&lt;/p&gt;&#xA;" OwnerUserId="27064" LastEditorUserId="27064" LastEditDate="2018-03-22T00:35:36.843" LastActivityDate="2018-03-23T17:26:58.270" Title="finite differences on a slanted grid --- advection diffusion equation" Tags="|finite-element|finite-difference|advection-diffusion|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="29110" PostTypeId="1" CreationDate="2018-03-22T06:42:00.137" Score="2" ViewCount="11346" Body="&lt;p&gt;OK, I know a bit of C++ (very basic syntax), and I want to do physics simulation in C++, like stuff like (also the things mentioned &lt;a href=&quot;http://www.falstad.com/mathphysics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ripples and waves over a 2-d surface&lt;/li&gt;&#xA;&lt;li&gt;Vibrating string/membrane&lt;/li&gt;&#xA;&lt;li&gt;Various electric/magnetic fields (as per my wish) in 2d/3d&lt;/li&gt;&#xA;&lt;li&gt;Gas molecules simulation&lt;/li&gt;&#xA;&lt;li&gt;Newtonian mechanics stuff (rotating objects, Newtonian gravity etc)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can anybody suggest some sources/libraries (or what that's called, something like vpython for python) so where I can learn so that I can simulate such stuff on my PC?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I'm NOT asking for books for doing it (though recommendations are welcome)&lt;/p&gt;&#xA;" OwnerUserId="27238" LastEditorUserId="20688" LastEditDate="2018-03-23T04:15:54.263" LastActivityDate="2018-03-23T04:15:54.263" Title="Physics Simulation in C++" Tags="|c++|computational-physics|simulation|reference-request|numerical-modelling|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="29111" PostTypeId="1" CreationDate="2018-03-22T11:58:02.467" Score="2" ViewCount="213" Body="&lt;p&gt;I need help computing the following integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{}\frac{(1+jk|\vec{r}-\vec{r}^\prime|)e^{-jk|\vec{r}-\vec{r}^\prime|}}{|\vec{r}-\vec{r}^\prime|}d\vec{r}^\prime&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in this integral $\vec{r}$ and $\vec{r}^\prime$ are the position vectors that correspond to the observation point and source point, respectively. &#xA;This integral comes from the Electric-Field Integral Equation (EFIE) kernel.&lt;/p&gt;&#xA;" OwnerUserId="27243" LastEditorUserId="20688" LastEditDate="2018-03-23T03:38:19.693" LastActivityDate="2018-03-23T03:38:19.693" Title="Calculation of the EFIE integral" Tags="|finite-element|numerical-analysis|numerical-modelling|electromagnetism|integral-equations|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29112" PostTypeId="2" ParentId="29110" CreationDate="2018-03-22T14:50:50.113" Score="7" Body="&lt;p&gt;I think you are missing a very important and crucial step that lies exactly between the physics and simulation: the mathematical model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to model any physics, one has to formulate the mathematical description of the physical phenomenon. Depending on the goals of the simulation, different approximations and assumptions can be made resulting in various complexity of the systems and requiring more or less complicated numerical techniques to solve them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned several very different areas of physics that you are interesting simulating, so, I guess, your interest lies more in the numerical methods/visualization rather than in any particular field. I will take electromagnetics (EM) as an example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the full glory of EM is covered by Maxwell's Equations&lt;/li&gt;&#xA;&lt;li&gt;various forms of which (and approximations) can be solved in 1D, 2D, and 3D.&lt;/li&gt;&#xA;&lt;li&gt;sometimes it is appropriate to use surface discretization (PEC - perfect electric conductor), sometimes you have to use volume discretization&lt;/li&gt;&#xA;&lt;li&gt;different properties of materials (nonlinearity, anisotropy, etc) might need to be taken into account.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What I am trying to say, is that to simulate any physics you first have to understand (to some extent) the mathematical model behind it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, usually, you need to solve this model using some numerical technique. Depending on the model, different techniques can be applicable and preferable.&#xA;Again, I will use the ones that are commonly used for EM, but other fields will be similar:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;finite-difference (finite-difference time-domain)&lt;/li&gt;&#xA;&lt;li&gt;finite-element method (FEM)&lt;/li&gt;&#xA;&lt;li&gt;integral-equation methods (boundary element)&lt;/li&gt;&#xA;&lt;li&gt;physical/geometrical optics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;...etc. Different libraries (including C++ ones) can be used to solve those problems: deal.ii, FEniCS...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my advice would be to start with something simpler and use a more appropriate language/platform to begin with numerical simulation of the physical phenomenon. Say, starting solving wave-equation in 1-D using Matlab (Octave) and visualizing simple propagation. Also using numerous already available toolboxes for PDE, ODE, and particular areas of physics should be easier there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also suggest a book/course that has nothing to do with C++, but was my &quot;bible&quot; in computational science:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ocw.mit.edu/courses/mathematics/18-085-computational-science-and-engineering-i-fall-2008/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIT course on computational science and engineering (Gilbert Strang)&lt;/a&gt;, syllabus, video lectures, notes are available&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://books.google.ca/books/about/Computational_Science_and_Engineering.html?id=GQ9pQgAACAAJ&amp;amp;source=kp_cover&amp;amp;redir_esc=y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gilbert Strang, &quot;Computational Science and Engineering&quot;&lt;/a&gt;, very accessible book.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-03-23T01:54:45.440" LastActivityDate="2018-03-23T01:54:45.440" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29114" PostTypeId="2" ParentId="29110" CreationDate="2018-03-22T20:51:44.557" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://fenicsproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fenics &lt;/a&gt; may be one of the simplest free software to use to simulate wave equations and other differential equations using finite element solutions.  It runs with a python interface with C++ underneath for performance, which makes it a bit user-friendlier than a pure C++ solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gas molecule simulations are commonly studied using molecular dynamics (crudely, integrating Newton's equations of motion).  Depending on how advanced of realistic of particle interactions there are quite a few choices of simulators to choose from, such as &lt;a href=&quot;https://hoomd-blue.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HOOMD&lt;/a&gt;, &lt;a href=&quot;http://lammps.sandia.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAAMPS&lt;/a&gt;, &lt;a href=&quot;http://www.gromacs.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GROMACS&lt;/a&gt;, CHARMM, AMBER, NAMD...  Each project probably comes with a recommended visualization.  I recall at least LAAMPS is pretty friendly with VMD (GUI), and HOOMD and LAAMPS I believe both work with &lt;a href=&quot;https://ovito.org/index.php/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ovito&lt;/a&gt;.  Many of these also are python script interfaces over a C++ library of some kind. The technical barrier to entry here may also be lower than for the finite-element simulations as well.&lt;/p&gt;&#xA;" OwnerUserId="25876" LastActivityDate="2018-03-22T20:51:44.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29115" PostTypeId="1" AcceptedAnswerId="29118" CreationDate="2018-03-23T02:16:03.443" Score="0" ViewCount="35" Body="&lt;p&gt;I have &lt;strong&gt;n&lt;/strong&gt; spots with &lt;strong&gt;2&lt;/strong&gt; possible positions each. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to explore all of this possibilities in C++ using loops (or something else if there is a better option).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of looping from 0 to power(2,n) and convert it to a binary number, for position j, take j-th digit of the binary number use 0 for first possibility and 1 for second possibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way? Because this way of doing introduces parsing operations and is limited to a binary positions problem, I mean if the problem had more possibilities for each position Ill be in trouble.&lt;/p&gt;&#xA;" OwnerUserId="27249" LastActivityDate="2018-03-23T03:57:37.287" Title="combinatory exploration c++" Tags="|c++|combinatorics|" AnswerCount="1" CommentCount="1" ClosedDate="2018-03-23T16:01:54.980" ContentLicense="CC BY-SA 3.0" />
  <row Id="29116" PostTypeId="2" ParentId="29111" CreationDate="2018-03-23T03:31:28.863" Score="5" Body="&lt;p&gt;A classic paper for evaluation of the integrals commonly present in computational electromagnetics (EM) is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;D. R. Wilton, S. M. Rao, A. W. Glisson, D. H. Schaubert, O. M. Al-Bundak, and C. M. Butler, &quot;&lt;a href=&quot;https://doi.org/10.1109/TAP.1984.1143304&quot; rel=&quot;noreferrer&quot;&gt;Potential integrals for uniform and linear source distributions of polygonal and polyhedral domains,&lt;/a&gt;&quot; &lt;em&gt;IEEE Trans. Antennas Propag.&lt;/em&gt;, vol. AP-32, no. 3, pp. 276-281, Mar. 1984.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The challenge to evaluate the EFIE integrals is the required singularity treatment. If both $\mathbf{r}$ and $\mathbf{r}^\prime$ belong to the same patch in the discretized geometry, one cannot use just quadrature-based integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to subtract a singularity and integrate it analytically and the rest (already without a singularity), numerically. So, say you need to calculate the integral ($R=|\mathbf{r}-\mathbf{r'}|$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int\limits_S\frac{e^{-jkR}}{R}ds'=\underbrace{\int\limits_S\frac{e^{-jkR}-1}{R}ds'}_{I_1}+\underbrace{\int\limits_S\frac{1}{R}ds'}_{I_2}&#xA;$$&#xA;Here, $S$ is the source patch and $\mathbf{r}$ is fixed.&#xA;Now, integral $I_1$ does not have a singularity and can be evaluated using, say, 2-D Gaussian quadrature (use barycentric coordinates and apply Gaussian quadrature over $\xi$, $\eta$ with a proper Jacobian)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integral $I_2$ can be found analytically. The formula for a planar polygon $S$ is given in the referenced paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You still might want to use this technique of singularity subtraction even when observation and source patches are different, but relatively close together. Such singularity treatment will decrease the error coming from numerical integration and reduce the number of quadrature points required to achieve the accuracy for nearby interactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume, that right now you've discretized your EFIE using pulse-basis functions and you are testing your EFIE using pulses as well. That might be good enough for certain scenarios, but I will point out that the use of proper basis functions and testing procedure might be important later in your development process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: &#xA;there are other techniques to tackle the singularity in this Green's function, including special quadrature rules, but I personally prefer this approach.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-23T03:31:28.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29118" PostTypeId="2" ParentId="29115" CreationDate="2018-03-23T03:57:37.287" Score="0" Body="&lt;p&gt;I would generalize this problem the following way (if I understood your problem right):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a function &lt;code&gt;bool nextVariant(const &amp;amp;prev_x, &amp;amp;new_x)&lt;/code&gt; that is going to generate a new candidate &lt;code&gt;new_x&lt;/code&gt; based on the previously explored one &lt;code&gt;prev_x&lt;/code&gt;. Then you can organize your code as follows (I made a sketch of a pseudocode using C-style syntaxis)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int main()&#xA;{&#xA;    CandidateType x(0); //initialize the starting candidate (possibility)&#xA;    do&#xA;    {&#xA;         performAnalysis(x); //do whatever you want with the candidate&#xA;         CandidateType prev_x = x; //maybe you need to save it, maybe not....&#xA;         bool isThereNext = nextVariant(prev_x,x);&#xA;    }&#xA;    while (isThereNext);&#xA;}&#xA;&#xA;bool nextVariant(const CandidateType&amp;amp; prev_x, CandidateType&amp;amp; x)&#xA;{&#xA;    if (notReachTheEnd)&#xA;    {&#xA;       //your logic to create x from prev_x. (assign x)&#xA;       return true;&#xA;    }&#xA;    else return false;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;code&gt;CandidateType&lt;/code&gt; is the &quot;type&quot; of your &quot;possibility&quot;. You will define it based on your problem, whether it is an N-spot with 2-positions, or something more complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that, you moved the complication towards generating a new possibility from a previous one, which is usually an easy thing to do. Since I don't know exactly what could be the possible intent of generating those &quot;candidates&quot;, I might have introduced certain inefficiencies that could be avoided.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-23T03:57:37.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29119" PostTypeId="2" ParentId="27493" CreationDate="2018-03-23T05:07:06.200" Score="2" Body="&lt;p&gt;First of all, it will be hard to proceed with certainty without &quot;actually running baseline tests&quot;. Because your dilemma can be summarized, as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;readability and convenience vs. performance and efficiency&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;While readability, convenience, and maintainability of the code are very important, the heavy computations still have to be done efficiently and use as much of the computational power as possible. Arrays are very special in that regard because they are stored contiguously in memory (column- or row-wise); thus, if one &lt;strong&gt;accesses them in the order they are stored&lt;/strong&gt;, it minimizes the amount of cache misses, improving the usage of the CPU. Moreover, the vectorization instructions can be applied easier (depends on the compiler and its settings).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, with the introduction of slicing the array, excessive using strides, one will definitely sacrifice some efficiency. How much? Depends on the heaviness of the array computations, surrounding logic, basic quality of the other code, compiler, CPU, RAM etc. So, actual benchmarking is important to make a decision on how to organize the data-flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The picture below (IT Hare) has much more information, but you might look into the difference between the time required for L1-, L2-, L3-, RAM-read. By introducing non-contiguous access to your array, you will increase the amount of time your code will have to do L2-read instead of L1 and L3 instead of L2. Or if your arrays are very large and access is completely random, most reads will be performed from RAM every time, which is horrendous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DwS54.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DwS54.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-23T05:07:06.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29120" PostTypeId="1" CreationDate="2018-03-23T07:53:03.363" Score="4" ViewCount="80" Body="&lt;p&gt;Let assume that one would like to translate an input-output model, into a state-space one. It is well-known that the process of realization addresses this aspect. Let assume to have the following input-output ARX model&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;y(k) = \sum_{i=1}^{N_a}a_iy(k-i) + \sum_{i=1}^{N_b}b_iu(k-i)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in which, at the discrete time step $k$, $y$ represents the output of the model, and $u$ represents the input, while $a_i$ and $b_i$ are the model's parameters. $N_a$ and $N_b$ can be chosen independently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This model can be translated into a suitable quartet of matrices $A,B,C,D$ such to obtain the state-space representation&#xA;$$&#xA;\boldsymbol{x}(k+1) = A \boldsymbol{x}(k) + B\boldsymbol{u}(k)&#xA;\\&#xA;\boldsymbol{y}(k) = C \boldsymbol{x}(k) + D \boldsymbol{u}(k)&#xA;$$&#xA;that will have the same transfer-function (and the same dynamics) of the aforementione ARX model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it is known, the realization process in not unique, i.e. there are infinite possible realizations in state-space form of the same input-output model. Among the different possibilities, canonical forms are quite adopted (controllability and observability) for different reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one of the main drawback with these canonical forms, is that one will loose the relationship between the original input and output quantities (i.e., $y(k-i)$ and $u(k-i)$) and the states resulting from the canonical realization process of the ARX model. This usually leads to a states array $\boldsymbol{x}$ that does not necessarily reflect physical quantities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could always implement his/her own version of a realization scheme in order to ensure the maintaining of physical meaning of the resulting states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question here is: Is there a way to recover the states meaning and map them onto the original physical quantities of inputs and outputs of the ARX model when canonical forms are used for state-space realizations?&lt;/p&gt;&#xA;" OwnerUserId="14817" LastActivityDate="2018-09-20T01:01:40.083" Title="How to preserve or recover states meanings in canonical state-space realizations" Tags="|matlab|linear-system|modeling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29121" PostTypeId="2" ParentId="13035" CreationDate="2018-03-23T12:59:50.617" Score="2" Body="&lt;p&gt;The comments above give some very good reasons for using octrees (i.e., recursively &lt;em&gt;halving&lt;/em&gt; the computational cube in each dimension as opposed to a more general orthogonal bisection). Symmetry and simplicity of calculating interaction lists is a big plus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would argue that perhaps the most important feature that octrees bring to the table is that the addition theorem underwriting the FMM is &lt;em&gt;systematically&lt;/em&gt; satisfied for far-zone interactions independent of the geometry with the extremely simple well-separatedness criterion of one or more &quot;buffer&quot; boxes. In other words, the FMM sum representation of the potential field is guaranteed to converge with increasing order under non-pathological circumstances.&lt;/p&gt;&#xA;" OwnerUserId="27171" LastEditorUserId="27171" LastEditDate="2018-07-23T01:41:55.390" LastActivityDate="2018-07-23T01:41:55.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29122" PostTypeId="1" CreationDate="2018-03-23T14:32:18.047" Score="4" ViewCount="474" Body="&lt;p&gt;I would like to use &lt;a href=&quot;https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/mcg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiscale Combinatorial Grouping&lt;/a&gt; for my PhD research. However, I am restricted to use open-source implementations and this one runs on Matlab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of an equivalent &lt;strong&gt;contour detection method&lt;/strong&gt; that works on large images that is implemented &lt;strong&gt;open-source&lt;/strong&gt;, preferably in Python? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here some more background info. Equivalent methods would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/resources.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contour detection and hierarchical image segmentation&lt;/a&gt; -&gt; cannot handle images of more than 1000x1000 pixels and runs on Matlab&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ivrl.epfl.ch/research/superpixels&quot; rel=&quot;nofollow noreferrer&quot;&gt;SLIC Superpixels&lt;/a&gt; -&gt; creates too many redundant segments&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am interested in detecting contours of visible objects in drone imagery with a resolution of 2-5cm per pixel and extents of up to 1kmx1km for the purpose of automating cadastral mapping. Here an example of what the contour detection output should look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IexPB.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IexPB.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27255" LastActivityDate="2018-03-23T14:32:18.047" Title="Open source implementation of Multiscale Combinatorial Grouping" Tags="|algorithms|python|image-processing|computer-vision|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29124" PostTypeId="2" ParentId="29109" CreationDate="2018-03-23T16:44:45.907" Score="3" Body="&lt;p&gt;In case you can easily implement the roated grid, this is possibly the easist&#xA;fix, so let me answer your question about the invariance of the convection&#xA;diffusion equation under rotation of the coordinate system.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;TL;DR&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Your physical problem does not depend on the coordinate system that you choose.&#xA;Only the definition of the Laplace operator and the gradient require that you&#xA;choose a cartesian coordinate system. Thus, rotating the coordinate system&#xA;will change nothing, as long as you rotate the velocities apropriately.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Derivation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Consider the convection diffusion equation&#xA;$$&#xA;\underbrace{\sum_i \frac{\partial^2 u}{\partial x_i^2}}_{\text{diffusion}}&#xA;+ \underbrace{\sum_i \frac{\partial (v_i u)}{ \partial x_i}}_{\text{convection}}&#xA;= f&#xA;\,.&#xA;$$&#xA;A rotation is a linear transformation.&#xA;Hence, assume the new coordinates $x_i'$ are given by&#xA;$$ x_i = \sum_j a_{ij} x_j', $$&#xA;which is the multiplication by a matrix $A$.&#xA;Furthermore, assume that we can obtain the new coordinates given the old ones&#xA;by&#xA;$$x'_i = \sum_j b_{ij} x_j .$$&#xA;The numbers $b_{ij}$ are the entries of the matrix $B := A^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We denote the function $u$ in the new coordinates also by $u$:&#xA;$$ u(x') = u(x(x'))\,. $$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Computation of the Diffusion Term&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;To compute the diffusion term, we need to use the chain rule. We start with&#xA;the first partial derivative.&#xA;We have that&#xA;$$&#xA;\frac{\partial u}{\partial x_i}&#xA;= \sum_k \frac{\partial u}{\partial x_k'} \frac{\partial x_k'}{\partial x_i}&#xA;= \sum_k \frac{\partial u}{\partial x_k'} b_{ki}&#xA;\,.&#xA;$$&#xA;We use this equation to compute the second partial derivative.&#xA;We obtain that&#xA;\begin{align*}&#xA;\frac{\partial}{\partial x_i}&#xA;\left(\frac{\partial u}{\partial x_i} \right)&#xA;&amp;amp;=&#xA;  \frac{\partial}{\partial x_i}&#xA;  \left(&#xA;  \sum_k \frac{\partial u}{\partial x_k'} b_{ki}&#xA;  \right) \\&#xA;&amp;amp;=&#xA;  \sum_k&#xA;  \frac{\partial}{\partial x_i}&#xA;  \left(\frac{\partial u}{\partial x_k'} b_{ki} \right) \\&#xA;&amp;amp;=&#xA;  \sum_k \sum_\ell&#xA;  \frac{\partial}{\partial x_\ell'}&#xA;  \left( \frac{\partial u}{\partial x_k'} b_{ki} \right)&#xA;  \frac{\partial x_\ell'}{\partial x_i} \\&#xA;&amp;amp;=&#xA;  \sum_k \sum_\ell&#xA;  \frac{\partial^2 u}{\partial x_k' \partial x_\ell'} b_{ki} b_{li}&#xA;  \,.&#xA;\end{align*}&#xA;The Laplace operator is the sum over all second partial derivatives.&#xA;Hence, we compute&#xA;\begin{align*}&#xA;  \sum_i \sum_k \sum_\ell&#xA;  \frac{\partial^2 u}{\partial x_k' \partial x_\ell'} b_{ki} b_{\ell i}&#xA;  &amp;amp;= \sum_k \sum_\ell&#xA;     \frac{\partial^2 u}{\partial x_k' \partial x_\ell'}&#xA;     \sum_i b_{ki} b_{\ell i} \\&#xA;  &amp;amp;= \sum_k \sum_\ell&#xA;   \frac{\partial^2 u}{\partial x_k' \partial x_\ell'} c_{k\ell}&#xA;   \,,&#xA;\end{align*}&#xA;where&#xA;$$&#xA;  c_{kl} := \sum_i b_{ki} b_{\ell i}&#xA;  \,.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Computation of the Convection Term&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;To compute the convection term, we have to use the chain rule, again.&#xA;Using this rule,&#xA;\begin{align*}&#xA;  \frac{\partial (v_i u)}{\partial x_i}&#xA;  &amp;amp;= \frac{\partial (v_i u)}{\partial x_i} \\&#xA;  &amp;amp;= \sum_k \frac{\partial (v_i u)}{\partial x_k'}&#xA;            \frac{\partial x_k'}{\partial x_i} \\&#xA;  &amp;amp;= \sum_k \frac{\partial (v_i u)}{\partial x_k'} b_{ki} \\&#xA;  \,.&#xA;\end{align*}&#xA;We obtain the gradient by summing over all partial derivatives. Thus,&#xA;$$&#xA; \sum_i \frac{\partial (v_i u)}{\partial x_i}&#xA; = \sum_i \sum_k \frac{\partial (v_i u)}{\partial x_k'} b_{ki}&#xA; = \sum_k \frac{\partial (v_k' u)}{\partial x_k'}&#xA; \,,&#xA;$$&#xA;where&#xA;$$&#xA;  v_k' = \sum_i b_{ki} v_i&#xA;  \,.&#xA;$$&#xA;The numbers $v_k'$ are just the velocities in the new coordinate system.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Rewriting the PDE&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;We can now give the PDE in the new coordinate system. The PDE is&#xA;$$&#xA;\sum_k \sum_\ell&#xA;\frac{\partial^2 u}{\partial x_k' \partial x_\ell'} c_{k\ell}&#xA;+&#xA;\sum_k \frac{\partial (v_k' u)}{\partial x_k'} b_{ki}&#xA;= f&#xA;\,.&#xA;$$&#xA;However, since we have assumed that $A$ is a rotation matrix, we can simplify&#xA;this equation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Simplification&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A rotation is a special linear transformation; the matrix $A$ is orthogonal,&#xA;i.e., the inverse $B$ of $A$ is the matrix $A$ transposed&#xA;($b_{ij} = a_{ji}$).&#xA;Thus,&#xA;$$&#xA;  \delta_{ij} = \sum_{\ell} b_{i\ell} a_{\ell j}&#xA;  = \sum_{\ell} b_{i \ell} b_{j \ell}&#xA;  = c_{ij}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, the diffusion term simplifies to&#xA;$$&#xA;  \sum_k \sum_\ell&#xA;  \frac{\partial^2 u}{\partial x_k' \partial x_\ell'} c_{k\ell}&#xA;  =&#xA;    \sum_k&#xA;    \frac{\partial^2 u}{\partial x_k' \partial x_k'}&#xA;  \,.&#xA;$$&#xA;And the PDE becomes&#xA;$$&#xA;  \sum_k&#xA;  \frac{\partial^2 u}{\partial {x_k'}^2}&#xA;  + \sum_k \frac{\partial (v_k' u)}{\partial x_k'}&#xA;  = f&#xA;  \,,&#xA;$$&#xA;which is just the convection diffusion equation with rotated velocities. &lt;/p&gt;&#xA;" OwnerUserId="24953" LastEditorUserId="24953" LastEditDate="2018-03-23T17:26:58.270" LastActivityDate="2018-03-23T17:26:58.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29126" PostTypeId="2" ParentId="25630" CreationDate="2018-03-23T17:31:29.357" Score="1" Body="&lt;p&gt;If the main idea is to &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;highlight the advantages of open-source over commercial software in terms of parallel processing&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;one has to first answer the question of what one wants to achieve from the simulation. Commercial software packages offer more than just solution of PDE/ODE, meshing, etc. They offer support, documentation, convenient graphical interface (not to be underestimated!), material/sample libraries, tutorials on basic simulation needs. So, even if the open-source software is able to model the same physics, its usability depends a lot on the &lt;em&gt;maturity of the field&lt;/em&gt; and &lt;em&gt;market expectations&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, coming back to parallelization. An average user usually has enough troubles and complications with the simulation itself, and if one adds the requirements of setting up the parallel compiler, libraries with distributed memory support, and tuning parallelization settings – that might be too much. Moreover, if a commercial solver offers parallelization, they usually invested some time into optimizing their code and providing certain ready-out-of-the-box functionality. They also have a luxury of hiring software engineers who are focused on the parallelization, while open-source simulators are usually parallelized by computational scientists who might be good in parallel programming, but it is rarely their major field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding parallel capabilities of the commercial software. Nowadays, most of the solvers will come with the support of the shared-memory parallelization, basically being able to take advantage of the multicore architecture of modern CPUs. The distributed memory parallelization (say, MPI) requires more drastic changes to the code and software architecture; moreover, special decomposition methods become a must. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following page from &lt;a href=&quot;https://www.ansys.com/products/electronics/option-electronics-hpc&quot; rel=&quot;nofollow noreferrer&quot;&gt;ANSYS website&lt;/a&gt; describes the capabilities of their solver to take advantage of multiple nodes and multiple cores for computational electromagnetics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would attribute the following advantages of open-source software vs. commercial:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;providing an ability to simulate basic-complexity models in physics, where support of parallel architectures is lagging by commercial software&lt;/li&gt;&#xA;&lt;li&gt;allow for experimentation with new different models/methods of simulation that are not supported by commercial software, but require parallel computing to be solved in a reasonable time&lt;/li&gt;&#xA;&lt;li&gt;provide a platform for developing new techniques and methods in parallelization itself&lt;/li&gt;&#xA;&lt;li&gt;serve as a reference&lt;/li&gt;&#xA;&lt;li&gt;cost. Usually, commercial software will ask a separate license for every core/node or a special HPC-licence. It will be expensive. Very expensive if you are actually going true exascale HPC.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-23T17:31:29.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29127" PostTypeId="2" ParentId="26971" CreationDate="2018-03-23T18:12:05.827" Score="1" Body="&lt;p&gt;SGE has been used on the Lonestar4 supercomputer (Texas Advanced Computing Center). Lonestar4 has been decommissioned in 2015; however, a very extensive &lt;a href=&quot;https://portal.tacc.utexas.edu/archives/lonestar4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lonestar4 user guide&lt;/a&gt; has been preserved on TACC website archive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://portal.tacc.utexas.edu/archives/lonestar4#running&quot; rel=&quot;nofollow noreferrer&quot;&gt;SGE: Running&lt;/a&gt; subsection of this guide provides a pretty detailed description of the variables and recipes on setting them up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your question, you should look into &quot;wayness&quot; and the command will have a syntaxis:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;-pe &amp;lt;TpN&amp;gt;way &amp;lt;NoN x NumCoresPerNode&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;code&gt;NumCoresPerNode&lt;/code&gt; - number of cored on the nodes of your cluster, &lt;code&gt;NoN&lt;/code&gt; - number of nodes, &lt;code&gt;TpN&lt;/code&gt; - tasks per node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say, you have a computing cluster consisting of nodes with two 6-Core processors. Then, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;-pe 12 way 48&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;requests 4 nodes with 12 tasks per node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding, if the whole node is blocked if you requested only part of its cores, that probably depends on the configuration of the SGE. On Lonestar4, the node was blocked and reserved only for you even if you requested only 1 core out of the 12 available.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-23T18:12:05.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29128" PostTypeId="2" ParentId="29120" CreationDate="2018-03-23T21:48:51.470" Score="1" Body="&lt;p&gt;No this is not possible. And actually, it is a good practice to not to rely on this type of inferences. Well before anything they tend to get in the way and certain operations are not structure preserving such as balancing and model reduction. Unit wise they are not that useful either. For example, a noise on the measurement signal shouldn't have any units since it operates on the signal itself but not necessarily the original physical phenomenon. Hence an input from disturbance $w_i$ to output $y_j$ which happen to have Kelvins as a unit doesn't really make sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such state representations especially the &quot;physically meaningful&quot; ones are often the worst in terms of ill-conditioning. Not only that, they either encode modal information to be &quot;meaningful&quot; or encode the characteristic polynomial e.g., companion form. It is best to have algebraic relations abstracted away. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if your system is MIMO then the resulting state space representation is very likely to be nonminimal thus you need to be extra careful when claiming &quot;describing the same dynamics&quot;. &lt;/p&gt;&#xA;" OwnerUserId="21617" LastActivityDate="2018-03-23T21:48:51.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29129" PostTypeId="2" ParentId="10029" CreationDate="2018-03-23T22:54:47.843" Score="1" Body="&lt;p&gt;Generating exactly the same elements is not entirely straightforward, and algorithms &lt;em&gt;marching cubes&lt;/em&gt;-like algorithms might be needed to make the mesh-generation process completely automatic. However, as mentioned in the comments, you will not be able to capture arbitrary boundaries with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on your goals, you might decide to help the mesher software to generate the conformal mesh that satisfies both requirements: as little sizes/orientations of the elements as possible, and conformal to the boundary. I will use GMSH as a software of choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider, the following 2-D example that you want to mesh: &lt;a href=&quot;https://i.stack.imgur.com/pZLjd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pZLjd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, we have a challenge of capturing the boundary at the bottom, but a lot of the potential to have a &quot;structured mesh&quot; in the top half (which is a square).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Attempt 1: just use the characteristic length at the points of the geometry and rely on the mesher's best effort:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8duRi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8duRi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cl=0.1; z=0.;&#xA;&#xA;Point(1) = {0.5,0.5,z,cl};&#xA;Point(2) = {-0.5,0.5,z,cl};&#xA;Point(3) = {-0.5,-0.5,z,cl};&#xA;Point(4) = {0.5,-0.5,z,cl};&#xA;Point(6) = {0.0, -0.5, z, cl};&#xA;&#xA;Line(1)={1,2};&#xA;Line(2)={2,3};&#xA;Line(4)={4,1};&#xA;Circle(5) = {3,6,4};&#xA;Line Loop(1) = {1,2,5,4};&#xA;Plane Surface(1) = {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the mesher created a mesh that certainly does not satisfy the requirement of &quot;same-size&quot; triangles.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Attempt 2: specify structured mesh in an obvious region:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HtMBA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HtMBA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cl=0.1; z=0.;&#xA;Point(1) = {0.5,0.5,z,cl};&#xA;Point(2) = {-0.5,0.5,z,cl};&#xA;Point(3) = {-0.5,-0.5,z,cl};&#xA;Point(4) = {0.5,-0.5,z,cl};&#xA;&#xA;Line(1)={1,2};  Transfinite Line{1} = 10;&#xA;Line(2)={2,3};  Transfinite Line{2} = 10;&#xA;Line(3)={3,4};  Transfinite Line{3} = 10;&#xA;Line(4)={4,1};  Transfinite Line{4} = 10;&#xA;Line Loop(1) = {1,2,3,4};   &#xA;Plane Surface(1) = {1}; Transfinite Surface{1};&#xA;&#xA;Point(6) = {0.0, -0.5, z, cl};&#xA;Circle(5) = {3,6,4};&#xA;Line Loop(2) = {5,-3};&#xA;Plane Surface(2) = {2};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, in one region, the mesh consists of exactly the same triangles and is conformal to the mesh in the other region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we do better?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Attempt 3: manually add another region with a specified mesh:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zo6eP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zo6eP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cl=0.1; z=0.;&#xA;&#xA;Point(1) = {0.5,0.5,z,cl};&#xA;Point(2) = {-0.5,0.5,z,cl};&#xA;Point(3) = {-0.5,-0.5,z,cl};&#xA;Point(4) = {0.5,-0.5,z,cl};&#xA;&#xA;Line(1)={1,2};  Transfinite Line{1} = 11;&#xA;Line(2)={2,3};  Transfinite Line{2} = 11;&#xA;Line(3)={3,4};  Transfinite Line{3} = 11;&#xA;Line(4)={4,1};  Transfinite Line{4} = 11;&#xA;Line Loop(1) = {1,2,3,4};   &#xA;Plane Surface(1) = {1}; Transfinite Surface{1};&#xA;&#xA;Point(10) = {-0.4,-0.5,z,cl};&#xA;Point(11) = {0.4,-0.5,z,cl};&#xA;Point(12) = {0.4,-0.7,z,cl};&#xA;Point(13) = {-0.4,-0.7,z,cl};&#xA;&#xA;Line(10) = {10,13}; Transfinite Line{10} = 3;&#xA;Line(11) = {13,12}; Transfinite Line{11} = 9;&#xA;Line(12) = {12,11}; Transfinite Line{12} = 3;&#xA;Line(13) = {11,10}; Transfinite Line{13} = 9;&#xA;Line Loop(3) = {10,11,12,13};   &#xA;Plane Surface(3) = {3}; Transfinite Surface{3};&#xA;&#xA;Point(6) = {0.0, -0.5, z, cl};&#xA;Circle(5) = {3,6,4};&#xA;Line(21) = {4,11};&#xA;Line(22) = {10,3};&#xA;&#xA;Line Loop(2) = {5,21,-12,-11,-10,22};&#xA;Plane Surface(2) = {2};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Effectively, what I have done -&gt; is applied marching cubes manually. I am not aware if GMSH contains something built-in to perform this kind of meshing. However, one would certainly be able to write a script that performs a marching cube-type algorithm to generate a GMSH-compliant geometry file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is also possible to have structured mesh regions with different triangle sizes to connect with the help of a relatively small number of triangles (unstructured mesh regions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: I used a very primitive way to instruct GMSH on the mesh, there are better ways to do it.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-23T22:54:47.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29130" PostTypeId="2" ParentId="25459" CreationDate="2018-03-23T23:27:33.913" Score="2" Body="&lt;p&gt;I'm a little late to reply but I found &lt;a href=&quot;https://arxiv.org/pdf/1206.1901.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this review&lt;/a&gt; to be really informative. One particularly nice feature of Hamiltonian flows is that they preserve volume in phase space. If we take an infinitesimal box of size $(\delta q, \delta p)$ around a point $(q, p)$ in phase space, then propagate every point inside the box forward in time by $\delta t$ under the Hamiltonian flow, the volume of the transformed box will stay the same. If you were to generate new proposals using some arbitrary transformation $\Psi$ that does not necessarily preserve volume in phase space, you'd also have to keep track of Jacobian factors $|\det\nabla\Psi|$ that may or may not be feasible to compute. Hamiltonian flows are volume-conserving, so this Jacobian factor is always equal to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just because Hamiltonian systems are nice doesn't necessarily mean that whatever discretization scheme you come up with shares all of these properties though. If you use a naive ODE solver like forward or backward Euler, then you lose the phase volume invariance property altogether.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately, there's a very rich theory for how to construct time-stepping schemes for Hamiltonian systems that do preserve a lot of invariants. These are called &lt;a href=&quot;https://en.wikipedia.org/wiki/Symplectic_integrator&quot; rel=&quot;nofollow noreferrer&quot;&gt;symplectic integrators&lt;/a&gt;. Provided that you use a symplectic integrator, such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Semi-implicit_Euler_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;semi-implicit Euler&lt;/a&gt;, the numerical flow is an exact sampling from the trajectories of a slightly perturbed Hamiltonian. You can even use some fancy Lie algebra theory to figure out what the perturbed Hamiltonian is to arbitrary order in the timestep. This backwards stability in the Hamiltonian means that all of the nice structural properties of Hamiltonian systems are preserved when you discretize.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-03-23T23:27:33.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29131" PostTypeId="1" CreationDate="2018-03-24T04:57:02.073" Score="1" ViewCount="79" Body="&lt;p&gt;I'm trying to simulate Stokes flow in 2D around an arbitrary polygon (representing a rigid body).  I'd like to get both the effect of the body on the flow velocity and the forces on the body by the fluid.  Basically, like how panel methods are used to examine inviscid flow around airfoils, I'm trying to examine &quot;airfoils&quot; in a Stokes flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm reading it right, in &lt;a href=&quot;https://epubs.siam.org/doi/10.1137/S106482750038146X&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Method of Regularized Stokeslets&lt;/a&gt;, section 2.2, it suggests doing this by discretizing the surface of the body and solving for the &quot;external forcing&quot; given the velocities of the fluid at the surface of the body.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is, the velocity of a Stokes flow at $\vec{x}$ given point forces $\vec{f_k}$ acting at $\vec{x_k}$ is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x) = \sum_{k=1}^N \frac{-\vec{f_k}}{4 \pi \mu} \ln|r| + \vec{f_k} \cdot (\vec{x} - \vec{x_k}) \frac{\vec{x} - \vec{x_k}}{4 \pi \mu |r|^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $r = |\vec{x} - \vec{x_k}|$.  The paper suggests inverting that to find the forces given the velocities.  However, I don't entirely understand what is meant by &quot;external forcing&quot; in the case of flow around a body. Is it the same as the fluid's force on the body (equal but opposite)? If so, it doesn't seem to match the pressure forces, given that the paper says the pressure is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p(x) = \sum_{k=1}^N \frac{\vec{f_k} \cdot (\vec{x} - \vec{x_k})}{2 \pi (\vec{x} - \vec{x_k})^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I misunderstanding something?&lt;/p&gt;&#xA;" OwnerUserId="1365" LastEditorUserId="20688" LastEditDate="2018-03-24T10:54:08.907" LastActivityDate="2018-03-24T10:54:08.907" Title="Stokes flow around rigid body" Tags="|fluid-dynamics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29133" PostTypeId="1" CreationDate="2018-03-24T16:16:54.480" Score="2" ViewCount="3655" Body="&lt;p&gt;Consider the heat equation&#xA;$$u_t = \kappa u_{xx}$$&#xA;with boundary conditions of&#xA;$$u(x,0)=0\\&#xA;u(0,t)=100\\&#xA;u(l,t)=0$$&#xA;Numerical analysis by pyton can be done with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.sparse import diags&#xA;&#xA;def Crank_Nicolson(dy,ny,dt,nt,k,T,ntout):&#xA;    Tout = []&#xA;    T0 = T[0]&#xA;    T1 = T[-1]&#xA;    s = k*dt/dy**2&#xA;    A = diags([-0.5*s, 1+s, -0.5*s], [-1, 0, 1], shape=(ny-2, ny-2)).toarray() &#xA;    B1 = diags([0.5*s, 1-s, 0.5*s],[-1, 0, 1], shape=(ny-2, ny-2)).toarray()&#xA;    for n in range(1,nt):&#xA;        Tn = T&#xA;        B = np.dot(Tn[1:-1],B1) &#xA;        B[0] = B[0]+0.5*s*(T0+T0)&#xA;        B[-1] = B[-1]+0.5*s*(T1+T1)&#xA;        T[1:-1] = np.linalg.solve(A,B)&#xA;        if n % int(nt/float(ntout)) == 0 or n==nt-1:&#xA;            Tout.append(T.copy())&#xA;    return Tout,s&#xA;&#xA;dt = 0.01&#xA;dy = 0.001&#xA;k = 10**(-4)&#xA;y_max = .1&#xA;y = np.arange(0,y_max+dy,dy) &#xA;ny = len(y)&#xA;nt = 1000&#xA;&#xA;T = np.zeros((ny,))&#xA;T[0] = 100&#xA;Tout,s = Crank_Nicolson(dy,ny,dt,nt,k,T,10)&#xA;&#xA;for T in Tout:&#xA;    plt.plot(y,T)    &#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How a Neuman boundary such as&#xA;$$u_x(l,t)=0$$&#xA;can be implemented in this code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This a case of keeping one end of a bar at temperature 100 °C while the other end is insulated.&lt;/p&gt;&#xA;" OwnerUserId="2115" LastEditorUserId="20688" LastEditDate="2018-03-24T16:23:59.790" LastActivityDate="2021-07-23T13:06:22.213" Title="Applying Neumann boundaries to Crank-Nicolson solution in python" Tags="|numerical-analysis|finite-difference|python|boundary-conditions|crank-nicolson|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29134" PostTypeId="2" ParentId="29133" CreationDate="2018-03-24T18:49:18.513" Score="4" Body="&lt;p&gt;Boundary conditions are often an annoyance, and can frequently take up a surprisingly large percentage of a numerical code. How to implement them depends on your choice of numerical method. Finite difference schemes often find Dirichlet conditions more natural than Neumann ones, whereas the opposite is often true for finite element and finite methods applied to diffusive problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One standard technique, which will work well in your case is to introduce an additional conceptually virtual degree of freedom or &quot;ghost&quot; point lying at $x=l+\Delta x$, then apply the model equation at $x = l$ in the form &#xA;$$ \frac{du(l)}{dt}=\kappa(l)\frac{u(l+\Delta x)-2u(l)+u(l-\Delta x)}{{\Delta x}^2}$$&#xA;as well as boundary condition at $x = l$, in the form $$\frac{u(l+\Delta x,t)-u(l-\Delta x,t)}{2\Delta x} = 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that there is no explicit equation applied at $x=l+\Delta x$, so there are still as many equations as unknowns, so the system remains well posed. In fact, most practical implementations rearrange the boundary condition into the form&#xA;$$u(l+\Delta x)=u(l-\Delta x),$$ and just substitute back into the previous equation to get $$ \frac{du(l)}{dt}=\kappa(l)\frac{2u(l-\Delta x)-2u(l)}{{\Delta x}^2}.$$.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2018-03-24T18:49:18.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29135" PostTypeId="1" CreationDate="2018-03-24T18:55:47.347" Score="7" ViewCount="4644" Body="&lt;p&gt;I am using Python, and I have a Pandas dataframe with hundreds of thousands, if not millions, of $(x,y,z)$ coordinates. I am looking to find an efficient method to index the original dataframe so that I'm only left with entries that are within some distance of a given point $P$ that's not necessarily in the set of coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trivial, but surely most expensive, method is to construct a vector between each $(x,y,z)$ point and the point $P$, take the norm, and only keep those that are within the cutoff distance. For millions of points though, as I have to do it dozens of times, this seems terribly inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are optimal but still somewhat intuitive methods to do this?&lt;/p&gt;&#xA;" OwnerUserId="22245" LastEditorUserId="22245" LastEditDate="2018-03-27T01:30:28.247" LastActivityDate="2018-04-17T11:37:54.077" Title="Efficiently finding all (x,y,z) points within certain distance of point P" Tags="|python|computational-physics|computational-geometry|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29136" PostTypeId="2" ParentId="29135" CreationDate="2018-03-24T21:14:11.977" Score="7" Body="&lt;p&gt;You can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Z-order_curve&quot; rel=&quot;nofollow noreferrer&quot;&gt;Morton keying&lt;/a&gt; to sort the coordinate locations by binning them into cubes of some specified size $d$. This is an $\mathcal{O}(N\log N)$ operation. Then, given any point P, you can use its Morton key to search only in a small number $(\mathcal{O}(1))$ of nearby boxes, bounded by your distance criterion. The search for each box is $\mathcal{O}(\log N)$.&lt;/p&gt;&#xA;" OwnerUserId="7241" LastEditorUserId="7241" LastEditDate="2018-04-17T11:37:54.077" LastActivityDate="2018-04-17T11:37:54.077" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29137" PostTypeId="2" ParentId="29135" CreationDate="2018-03-25T00:11:59.767" Score="4" Body="&lt;p&gt;You can use any &lt;a href=&quot;https://en.wikipedia.org/wiki/Nearest_neighbor_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;data structure for nearest neighbor search&lt;/a&gt;; there are many possibilities, with different tradeoffs.  You can trade off space vs query time vs how efficiently you can update the data structure with new points.  Specifically, your problem is &lt;a href=&quot;https://en.wikipedia.org/wiki/Nearest_neighbor_search#Fixed-radius_near_neighbors&quot; rel=&quot;nofollow noreferrer&quot;&gt;fixed-radius near neighbor search&lt;/a&gt;.  Most data structures for nearest neighbor search support this kind of query.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2018-03-25T00:11:59.767" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29140" PostTypeId="2" ParentId="29103" CreationDate="2018-03-25T15:16:12.330" Score="1" Body="&lt;p&gt;Finite element method is used to change the boundary value problem, with an infinite number of unknowns, into a system of algebraic equations with a finite number of degrees of freedom (DOFs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the response of the deformable body,  which conveniently is described by partial diffrenetial equation (PDE), using finite element method is transformed into a system of algebraic equations with the finite number of DOFs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you write equations for rigid body or system of rigid bodies, you do not have PDE; you have immediately a system of algebraic equations. Thus no need for finite element method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with the system only rigid bodies is that is very often over-constrained, since only you have equations of equilibrium to work with. However, this is another story.  &lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2018-03-25T17:12:44.240" LastActivityDate="2018-03-25T17:12:44.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29141" PostTypeId="1" CreationDate="2018-03-25T19:21:59.090" Score="0" ViewCount="123" Body="&lt;p&gt;I am developing a code where I am using the least squares method to compute gradients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, we use least squares to obtain some model based on a set of data (${q_1 \cdots q_N}$) at locations (${x_1 \cdots x_N}$), and use this model to predict $q_c$ at $x_c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, I am looking for $\nabla{q}$ at $x_c$ using least squares. Should the $q$ value at $x_c$ be included in a least squares model for its gradient at the location, or only the neighboring values? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like use the weighted least squares method. It seems the weighting matrix is defined as a diagonal matrix with the inverse of the variance squared along the main diagonal. The popular choice for the variance seems to be the distance between the neighboring data point and the point in question. So if I include the $q_c$ value at $x_c$ in my least squares model, then the variance would be zero for this data point, which would be problematic unless I add some $\epsilon$ value to the variance before taking it's inverse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The least squares equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\nabla q = q^TZM^T\nabla\phi(x) \\&#xA;\\&#xA;M=(Z^TZ)^{-1}\\&#xA;\\&#xA;\phi = \begin{bmatrix}&#xA;1 &amp;amp; x &amp;amp; y &amp;amp; z &amp;amp; xy &amp;amp; xz &amp;amp; yz &amp;amp; 0.5x^2 &amp;amp; 0.5y^2 &amp;amp; 0.5z^2 \\&#xA;\end{bmatrix}\\&#xA;\\&#xA;Z_{unweight} = \begin{bmatrix}&#xA;1 &amp;amp; x_1 &amp;amp; y_1 &amp;amp; z_1 &amp;amp; x_1y_1 &amp;amp; x_1z_1 &amp;amp; y_1z_1 &amp;amp; 0.5x_1^2 &amp;amp; 0.5y_1^2 &amp;amp; 0.5z_1^2 \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp;\vdots &amp;amp; \vdots \\&#xA;1 &amp;amp; x_N &amp;amp; y_N &amp;amp; z_N &amp;amp; x_Ny_N &amp;amp; x_Nz_N &amp;amp; y_Nz_N &amp;amp; 0.5x_N^2 &amp;amp; 0.5y_N^2 &amp;amp; 0.5z_N^2 \\&#xA;\end{bmatrix} \\&#xA;\\&#xA;q = \begin{bmatrix}&#xA;q_1 &amp;amp; \cdots &amp;amp; q_N&#xA;\end{bmatrix}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$q_c$ is known, but I am not sure if this should be included in the above model. &lt;/p&gt;&#xA;" OwnerUserId="23915" LastEditorUserId="23915" LastEditDate="2018-03-26T00:30:55.187" LastActivityDate="2018-04-25T05:49:08.933" Title="Using least squares for computing gradients" Tags="|least-squares|" AnswerCount="1" CommentCount="11" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29142" PostTypeId="1" AcceptedAnswerId="29144" CreationDate="2018-03-25T21:35:33.087" Score="3" ViewCount="403" Body="&lt;p&gt;Given a triangulated surface in $\Bbb{R}^3$ we can simply project it on a plane. This will result in a family of triangles which do not form a mesh of the projection for the following reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;each point of the projection is covered by at least two triangles (see the picture).&lt;/li&gt;&#xA;&lt;li&gt;the orientation of the triangles is not right&lt;/li&gt;&#xA;&lt;li&gt;things may get more complicated for more complex surfaces&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Even if the projection is not a mesh, every point of the projected surface is inside one of the triangles, so this could give a good idea of the projection. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a way to extract/construct a mesh of the projection, starting from the projection of the surface? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9UTCd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9UTCd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3548" LastActivityDate="2018-03-26T03:44:49.087" Title="Compute mesh of the projection of a 3D surface triangulation" Tags="|computational-geometry|mesh-generation|mesh|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29143" PostTypeId="2" ParentId="10834" CreationDate="2018-03-25T23:49:06.657" Score="4" Body="&lt;p&gt;The expressions you are using for eigenvalues and eigenfunctions are wrong (as per Wolfgang Bangerth's comment); therefore the results you are getting are not meaningful at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are analytic expressions for eigenvalues for both &lt;em&gt;continuous&lt;/em&gt; and &lt;em&gt;discrete&lt;/em&gt; cases for $d^2/dx^2$ on a circular segment – periodic BCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a continuous case on $S^1=[0,L]$, $j$th eigenvalue $\lambda_j$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\lambda_j^{\text{cont}} =&#xA;\begin{cases}&#xA;-\frac{\pi^2}{L^2}j^2,&amp;amp;\quad j \text{ is even}\\&#xA;-\frac{\pi^2}{L^2}(j-1)^2,&amp;amp;\quad j \text{ is odd}&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a discrete case with $N$ points distributed uniformly around a circle and central differencing scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\lambda_j^{\text{disc}} =&#xA;\begin{cases}&#xA;-\frac{4}{h^2}\sin^2\left(\frac{\pi j^2}{2N}\right),&amp;amp;\quad j \text{ is even}\\&#xA;-\frac{4}{h^2}\sin^2\left(\frac{\pi (j-1)^2}{2N}\right),&amp;amp;\quad j \text{ is odd}&#xA;\end{cases}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is effectively the analytical expression for the eigenvalues of your matrix $A$. Here, $h=L/N.$ for a circular segment $S^1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a simple Python script that compares the eigenvalues of the continuous operator and discrete (obtained from the actual matrix using &lt;code&gt;numpy.linalg.eigvals&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $N=5$, the relative error for the first eigenvalues was:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;[0.1248598 0.1248598 0.4272133 0.4272133]&lt;/code&gt; (first eigenvalue is 0 and is discarded, eigenvalues have multiplicity 2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For $N=100$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;[0.00032894 0.00032894 0.00131525 0.00131525]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the scheme &quot;converges&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/local/bin/python&#xA;import numpy as np&#xA;&#xA;# defining problem&#xA;L = 1.      # length of the circular interval S = [0,L] (starting is assumed to be fixed&#xA;N = 100       # Number of points discretizing circular interval S&#xA;h = L/N     # FD step-size (note, it is a circle; thus, N vs. N-1&#xA;&#xA;# forming discretized d^2 / dx^2 operator&#xA;A = np.zeros((N, N))&#xA;d = np.arange(N)&#xA;A[d, d] = -2&#xA;A[d, (d+1) % N] = 1&#xA;A[d, (d-1) % N] = 1&#xA;A *= 1./(h**2)&#xA;&#xA;# calculating eigenvalues&#xA;A_eig = np.linalg.eigvals(A)&#xA;A_eig = np.flip(np.sort(A_eig), 0)  # decreasing order&#xA;&#xA;# comparing to analytical expressions for eigenvalues in continuous and discrete cases&#xA;eig_cont_anal = np.zeros(N)     # continuous problem with periodic BC&#xA;eig_disc_anal = np.zeros(N)     # discrete problem with periodic BC&#xA;&#xA;for j in range(1, N+1):&#xA;    if j % 2 == 0:  # even eigenvalue&#xA;        arg = j&#xA;    else:           # odd eigenvalue&#xA;        arg = j-1&#xA;    eig_cont_anal[j-1] = arg**2&#xA;    eig_disc_anal[j-1] = np.sin(np.pi*arg/(2*N))**2&#xA;&#xA;# scaling eigenvalues according to the corresponding analytical expressions&#xA;factor_cont = -(np.pi/L)**2&#xA;factor_disc = -(4./h**2)&#xA;eig_cont_anal *= factor_cont&#xA;eig_disc_anal *= factor_disc&#xA;&#xA;# will calculate the relative error between the eigenvalues (except first lambda_0=0) obtained from&#xA;# a) analytic eigenvalues for the continuous  problem&#xA;# b) numerical eigenvalues for discrete problem with N points&#xA;&#xA;error = np.zeros(N-1)&#xA;for j in range(1, N):&#xA;    error[j-1] = abs(A_eig[j]-eig_cont_anal[j])/abs(eig_cont_anal[j])&#xA;&#xA;# print(eig_cont_anal); print(A_eig)&#xA;K = 4  # number of eigenvalues to print out a relative error&#xA;print(error[0:K])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-25T23:49:06.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29144" PostTypeId="2" ParentId="29142" CreationDate="2018-03-26T03:44:49.087" Score="2" Body="&lt;p&gt;If you just project the vertices to the plane, you can then construct a proper triangulation from that, for example using a Voronoi or Delaunay procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does not take into account the triangles of the original mesh. If you'd like to be more faithful to the original mesh, then you could take the collection of projected vertices and split all crossing line segments into non-crossing ones. The result will be something that does not always consist of triangles, but you can always split polygons into triangles. You'd then get a triangulation that is a refinement/further subdivision of the two-dimensional mesh you show in your picture.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-03-26T03:44:49.087" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29145" PostTypeId="1" AcceptedAnswerId="29151" CreationDate="2018-03-26T04:21:17.017" Score="0" ViewCount="73" Body="&lt;p&gt;I've been  trying to implement the CPU GS-PCA algorithm in &lt;a href=&quot;https://arxiv.org/pdf/0811.1081.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code starts on page 28 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a program written a script in python which gives the same output as the C++ program in the paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My function looks like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[T, P, R, L, U] = IterativePCA(theData, numDimensions, 10000, 1.0e-7)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;theData&lt;/code&gt; is the big matrix with the data vectors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;numDimensions&lt;/code&gt; is the number of dimensions the PCA algorithm projects onto&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using 10000 max iterations &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and an error tolerance of 1.0e-7&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I still have a question&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the matricies &lt;code&gt;[T, P, R, L, U]&lt;/code&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I get from them to the new data that has been projected onto the PCA basis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can compute this data using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.decomposition import PCA&#xA;&#xA;PCA(n_components=n_numDimensions).fit_transform(theData)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I want to be able to do it using the iterative method in the paper&lt;/p&gt;&#xA;" OwnerUserId="4680" LastEditorUserId="20688" LastEditDate="2018-05-17T00:10:18.620" LastActivityDate="2018-05-17T00:10:18.620" Title="Wanting an explanation of the variables in Iterative PCA algorithm" Tags="|linear-algebra|python|c++|projection|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29147" PostTypeId="2" ParentId="29141" CreationDate="2018-03-26T05:28:19.467" Score="1" Body="&lt;p&gt;You are asking the following question. Given $(q_0,q_1,q_2,\ldots,q_N)$ find $\nabla q$ at $\vec{r}=\vec{r}_0$. You can do this two ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I. Assume&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;q(x,y,z) = a + \vec{b} \cdot (\vec{r} - \vec{r}_0)&#xA;$$&#xA;and find $a$ and $\vec{b}$ by &#xA;$$&#xA;\min_{a,\vec{b}}\sum_{i=0}^N(q(x_i,y_i,z_i) - q_i)^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;II. Assume&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;q(x,y,z) = q_0 + \vec{b} \cdot (\vec{r} - \vec{r}_0)&#xA;$$&#xA;and find $\vec{b}$ by &#xA;$$&#xA;\min_{\vec{b}}\sum_{i=1}^N(q(x_i,y_i,z_i) - q_i)^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either case, $\vec{b}$ will give an approximation to $\nabla q$ at $\vec{r}_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In (II), you have $q(x_0,y_0,z_0)=q_0$ but in (I) you only have $q(x_0,y_0,z_0) \approx q_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to put in weights based on some inverse distance in above norm, then even in (I) you are forcing $q(x_0,y_0,z_0)=q_0$ to hold. In this case, you can just use approach (II) after putting in the weights appropriately in the norm.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-03-26T05:28:19.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29148" PostTypeId="2" ParentId="23323" CreationDate="2018-03-26T05:56:51.840" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This answer is an addition to the one from Wolfgang Bangerth.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is certainly not worth to bother with the initial guess for the iterative linear solver &lt;strong&gt;if there is any work to be done&lt;/strong&gt;: coding, solution, interpolation, etc. As the effect of the &lt;em&gt;proper&lt;/em&gt; initial guess compared to the &lt;em&gt;zero-vector&lt;/em&gt; will be very little.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in certain situations, a decent initial guess is actually already available. One example would be a frequency sweep with a constant (over frequency) mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say, one computes a scattering pattern from an object due to some excitation in the frequency domain and desires to have solutions for a set of frequencies ($\{f_1,\ldots,f_N\}$ – sorted). Now, while solving for the $i$th frequency $f_i$, it totally makes sense to use a solution for a previous frequency $f_{i-1}$ as an initial guess. The solution for $f_{i-1}$ has already been computed and no interpolation needs to be done – thus, saving several iterations for a literally zero cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side Notes: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it usually makes sense to start from $f_N$ descending to a lower frequency $f_1$ during a frequency sweep such that a solution for a higher frequency becomes an initial guess for a lower frequency.&lt;/li&gt;&#xA;&lt;li&gt;certainly, if the frequencies in $\{f_1,\ldots,f_N\}$ are very much apart and solution for $f_i$ has nothing to do with the $f_{i-1}$th – it will be hard to benefit from such initial guess choice.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-26T05:56:51.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29149" PostTypeId="1" AcceptedAnswerId="29154" CreationDate="2018-03-26T05:59:24.400" Score="48" ViewCount="12400" Body="&lt;p&gt;In this &lt;a href=&quot;https://space.stackexchange.com/questions/26199/calculating-j2-perturbations-in-cartesian-coordinates#comment77332_26199&quot;&gt;comment&lt;/a&gt; I wrote:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...default SciPy integrator, which I'm assuming only uses symplectic methods.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;in which I am refering to SciPy's &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;odeint&lt;/code&gt;&lt;/a&gt;, which uses either a &quot;non-stiff (Adams) method&quot; or a &quot;stiff (BDF) method&quot;. According to the &lt;a href=&quot;https://github.com/scipy/scipy/blob/v1.0.0/scipy/integrate/odepack.py#L25-L230&quot; rel=&quot;noreferrer&quot;&gt;source&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def odeint(func, y0, t, args=(), Dfun=None, col_deriv=0, full_output=0,&#xA;           ml=None, mu=None, rtol=None, atol=None, tcrit=None, h0=0.0,&#xA;           hmax=0.0, hmin=0.0, ixpr=0, mxstep=0, mxhnil=0, mxordn=12,&#xA;           mxords=5, printmessg=0):&#xA;    &quot;&quot;&quot;&#xA;    Integrate a system of ordinary differential equations.&#xA;&#xA;    Solve a system of ordinary differential equations using lsoda from the&#xA;    FORTRAN library odepack.&#xA;&#xA;    Solves the initial value problem for stiff or non-stiff systems&#xA;    of first order ode-s::&#xA;        dy/dt = func(y, t0, ...)&#xA;    where y can be a vector.&#xA;    &quot;&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is an &lt;a href=&quot;https://space.stackexchange.com/a/22170/12102&quot;&gt;example&lt;/a&gt; where I propagate a satellite's orbit around the earth for three months just to show that it precesses as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that &lt;em&gt;non-symplectic&lt;/em&gt; integrators have the undesirable property that they will tend not to conserve energy (or other quantities) and so are undesirable in orbital mechanics for example. But I'm not exactly sure what it is that makes a symplectic integrator symplectic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to explain what the property is (that makes a symplectic integrator symplectic) in a straightforward and (fairly) easy to understand but not inaccurate way? I'm asking from the point of view of &lt;em&gt;how the integrator functions internally&lt;/em&gt;, rather than how it performs in testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And is my suspicion correct that &lt;code&gt;odeint&lt;/code&gt; does use only symplectic integrators?&lt;/p&gt;&#xA;" OwnerUserId="17869" LastEditorUserId="20417" LastEditDate="2018-03-26T19:33:48.140" LastActivityDate="2022-02-09T23:32:14.303" Title="What does &quot;symplectic&quot; mean in reference to numerical integrators, and does SciPy's odeint use them?" Tags="|ode|time-integration|integration|differential-equations|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29151" PostTypeId="2" ParentId="29145" CreationDate="2018-03-26T06:56:41.397" Score="1" Body="&lt;p&gt;Section 2.1 in the paper explains that PCA takes a list of vectors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X = [X^{(0)} | X^{(1)} | ... | X^{(N-1)}]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and maps it to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$T = [T^{(0)} | T^{(1)} | ... | T^{(N-1)}]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I presume $T$ contains the data in the new coordinate system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It then explains that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X = T P^T$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I gather that $P$ contains the principle components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the &lt;a href=&quot;https://www.semanticscholar.org/paper/Parallel-GPU-Implementation-of-Iterative-PCA-Andrecut/d3cfbca16f734e5737ef0d292557222ad7b22076/read&quot; rel=&quot;nofollow noreferrer&quot;&gt;published source code&lt;/a&gt; now (instead of the preprint version)&lt;/p&gt;&#xA;" OwnerUserId="4680" LastActivityDate="2018-03-26T06:56:41.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29154" PostTypeId="2" ParentId="29149" CreationDate="2018-03-26T15:11:46.887" Score="84" Body="&lt;p&gt;Let me start off with corrections. No, &lt;code&gt;odeint&lt;/code&gt; doesn't have any symplectic integrators. No, symplectic integration doesn't mean conservation of energy.&lt;/p&gt;&#xA;&lt;h2&gt;What does symplectic mean and when should you use it?&lt;/h2&gt;&#xA;&lt;p&gt;First of all, what does symplectic mean? Symplectic means that the solution exists on a symplectic manifold. A symplectic manifold is a solution set which is defined by a 2-form. The details of symplectic manifolds probably sound like mathematical nonsense, so instead the gist of it is there is a direct relation between two sets of variables on such a manifold. The reason why this is important for physics is because Hamiltonian's equations naturally have that the solutions reside on a symplectic manifold in phase space, with the natural splitting being the position and momentum components. For the true Hamiltonian solution, that phase space path is constant energy.&lt;/p&gt;&#xA;&lt;p&gt;A symplectic integrator is an integrator whose solution resides on a symplectic manifold. Because of discretization error, when it is solving a Hamiltonian system it doesn't get exactly the correct trajectory on the manifold. Instead, that trajectory itself is perturbed &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\Delta t^n)$&lt;/span&gt; for the order &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; from the true trajectory. Then there's a linear drift due to numerical error of this trajectory over time. Normal integrators tend to have a quadratic (or more) drift, and do not have any good global guarantees about this phase space path (just local).&lt;/p&gt;&#xA;&lt;p&gt;What this tends to mean is that symplectic integrators tend to capture the long-time patterns better than normal integrators because of this lack of drift and this almost guarantee of periodicity. &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqTutorials.jl/blob/master/PhysicalModels/KeplerProblem.ipynb&quot; rel=&quot;noreferrer&quot;&gt;This notebook displays those properties well on the Kepler problem&lt;/a&gt;. The first image shows what I'm talking about with the periodic nature of the solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/paHNZ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/paHNZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This was solved using the &lt;a href=&quot;https://tutorials.sciml.ai/html/models/05-kepler_problem.html&quot; rel=&quot;noreferrer&quot;&gt;6th order symplectic integrator from Kahan and Li from DifferentialEquations.jl&lt;/a&gt;. You can see that the energy isn't exactly conserved, but its variation is dependent on how far the perturbed solution manifold is from the true manifold. But since the numerical solution itself resides on a symplectic manifold, it tends to be almost exactly periodic (with some linear numerical drift that you can see), making it do very nicely for long term integration. If you do the same with RK4, you can get disaster:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PB8SI.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PB8SI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can see that the issue is that there's no true periodicity in the numerical solution and therefore overtime it tends to drift.&lt;/p&gt;&#xA;&lt;p&gt;This highlights the true reason to choose symplectic integrators: &lt;strong&gt;symplectic integrators are good on long-time integrations on problems that have the symplectic property (Hamiltonian systems)&lt;/strong&gt;. So let's walk through a few things. Note that you don't always need symplectic integrators even on a symplectic problem. For this case, an adaptive 5th order Runge-Kutta method can do fine. Here's &lt;a href=&quot;https://diffeq.sciml.ai/latest/solvers/ode_solve/#Explicit-Runge-Kutta-Methods-1&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;Tsit5&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WGPX9.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WGPX9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Notice two things. One, it gets a good enough accuracy that you cannot see the actual drift in the phase space plot. However, on the right side you can see that there is this energy drift, and so if you are doing a long enough integration this method will not do as well as the solution method with the periodic properties. But that raises the question, how does it fare efficiency-wise vs just integrating extremely accurately? Well, this is a bit less certain. In &lt;a href=&quot;https://github.com/SciML/SciMLBenchmarks.jl&quot; rel=&quot;noreferrer&quot;&gt;SciMLBenchmarks.jl&lt;/a&gt; you can find some benchmarks investigating this question. For example, &lt;a href=&quot;https://benchmarks.sciml.ai/html/DynamicalODE/Quadrupole_boson_Hamiltonian_energy_conservation_benchmark.html&quot; rel=&quot;noreferrer&quot;&gt;this notebook&lt;/a&gt; looks at the energy error vs runtime on a Hamiltonian equation system from a quadruple Boson model and shows that if you want really high accuracy, then even for quite long integration times it's more efficient to just use a high order RK or Runge-Kutta Nystrom (RKN) method. This makes sense because to satisfy the symplectic property the integrators give up some efficiency and pretty much have to be fixed time step (there is some research making headway into the latter but it's not very far along).&lt;/p&gt;&#xA;&lt;p&gt;In addition, notice from both of these notebooks that you can also just take a standard method and project it back to the solution manifold each step (or every few steps). This is what the examples using the DifferentialEquations.jl &lt;a href=&quot;https://diffeq.sciml.ai/latest/features/callback_library/#Manifold-Conservation-and-Projection-1&quot; rel=&quot;noreferrer&quot;&gt;ManifoldProjection callback&lt;/a&gt; are doing. You see that guarantees conservation laws are upheld but with an added cost of solving an implicit system each step. You can also use a fully-implicit ODE solver or singular mass matrices to add on conservation equations, but the end result is that these methods are more computationally-costly as a tradeoff.&lt;/p&gt;&#xA;&lt;p&gt;So to summarize, the class of problems where you want to reach for a symplectic integrator are those that have a solution on a symplectic manifold (Hamiltonian systems) where you don't want to invest the computational resources to have a very exact (tolerance &lt;code&gt;&amp;lt;1e-12&lt;/code&gt;) solution and don't need exact energy/etc. conservation. This highlights that it's all about long-term integration properties, so you shouldn't just flock to them all willy-nilly like some of the literature suggests. But they are still a very important tool in many fields like Astrophysics where you do have long time integrations that you need to solve sufficiently fast without having absurd accuracy.&lt;/p&gt;&#xA;&lt;h2&gt;Where do I find symplectic integrators? What kind of symplectic integrators exist?&lt;/h2&gt;&#xA;&lt;p&gt;There are generally two classes of symplectic integrators. There are the symplectic Runge-Kutta integrators (which are the ones shown in the above examples) and there are implicit Runge-Kutta methods which have the symplectic property. As @origimbo mentions, the symplectic Runge-Kutta integrators require that you provide them with a partitioned structure so they can handle the position and momentum parts separately. However, counter to the comment, the implicit Runge-Kutta methods are symplectic without requiring this, but instead require solving a nonlinear system. This isn't too bad because if the system is non-stiff this nonlinear system can be solved with functional iteration or Anderson acceleration, but the symplectic RK methods should still probably be preferred for efficiency (it's a general rule that the more information you provide to an integrator, the more efficient it is).&lt;/p&gt;&#xA;&lt;p&gt;That said, &lt;strong&gt;odeint does not have methods from either of these families&lt;/strong&gt;, so it is not a good choice if you're looking for symplectic integrators. In Fortran, &lt;a href=&quot;https://www.unige.ch/%7Ehairer/software.html&quot; rel=&quot;noreferrer&quot;&gt;Hairer's site has a small set you can use&lt;/a&gt;. &lt;a href=&quot;http://reference.wolfram.com/language/tutorial/NDSolveSPRK.html&quot; rel=&quot;noreferrer&quot;&gt;Mathematica has a few built in&lt;/a&gt;. The GSL ODE solvers &lt;a href=&quot;https://www.gnu.org/software/gsl/doc/html/ode-initval.html&quot; rel=&quot;noreferrer&quot;&gt;have implicit RK Gaussian point integrators&lt;/a&gt; which IIRC are symplectic, but that's about the only reason to use the GSL methods.&lt;/p&gt;&#xA;&lt;p&gt;But &lt;strong&gt;the most comprehensive set of symplectic integrators can be found in &lt;a href=&quot;http://diffeq.sciml.ai/latest/&quot; rel=&quot;noreferrer&quot;&gt;DifferentialEquations.jl in Julia&lt;/a&gt;&lt;/strong&gt; (recall this was used for the notebooks above). The list of available symplectic Runge-Kutta methods is found &lt;a href=&quot;https://diffeq.sciml.ai/latest/solvers/dynamical_solve/#Symplectic-Integrators-1&quot; rel=&quot;noreferrer&quot;&gt;on this page&lt;/a&gt; and you'll notice that the &lt;a href=&quot;https://diffeq.sciml.ai/latest/solvers/ode_solve/#SDIRK-Methods-1&quot; rel=&quot;noreferrer&quot;&gt;implicit midpoint method is also symplectic&lt;/a&gt; (the implicit Runge-Kutta Trapezoid method is considered &amp;quot;almost symplectic&amp;quot; because it's reversible). Not only does it have the largest set of methods, but it's also open-source (you can see the code and its tests in a high-level language) and has a lot of &lt;a href=&quot;https://github.com/SciML/SciMLBenchmarks.jl&quot; rel=&quot;noreferrer&quot;&gt;benchmarks&lt;/a&gt;. A good introductory notebook for using it to solve physical problems &lt;a href=&quot;https://tutorials.sciml.ai/html/models/01-classical_physics.html&quot; rel=&quot;noreferrer&quot;&gt;is this tutorial notebook&lt;/a&gt;. But of course it's recommended you get started with the package through &lt;a href=&quot;https://diffeq.sciml.ai/latest/tutorials/ode_example/&quot; rel=&quot;noreferrer&quot;&gt;the first ODE tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In general you can find a detailed analysis of numerical differential equation suites at &lt;a href=&quot;http://www.stochasticlifestyle.com/comparison-differential-equation-solver-suites-matlab-r-julia-python-c-fortran/&quot; rel=&quot;noreferrer&quot;&gt;this blog post&lt;/a&gt;. It's quite detailed but since it has to cover a lot of topics it does each at less detail than this, so feel free to ask for it to be expanded in any way.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2022-02-09T23:32:14.303" LastActivityDate="2022-02-09T23:32:14.303" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="29155" PostTypeId="2" ParentId="29149" CreationDate="2018-03-26T15:17:22.053" Score="18" Body="&lt;p&gt;To complement Chris Rackauckas answer, to state some of the mathematical nonsense as well as some stuff you almost certainly know, a dynamical system is &lt;a href=&quot;https://en.wikipedia.org/wiki/Hamiltonian_system&quot; rel=&quot;noreferrer&quot;&gt;Hamiltonian&lt;/a&gt; if there is a description with coordinates &lt;span class=&quot;math-container&quot;&gt;$\mathbf{p}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{q}$&lt;/span&gt; and a functional, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H(\mathbf{p},\mathbf{q})}$&lt;/span&gt; such that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d\mathbf{q}}{dt}=+\frac{\partial \mathcal{H}}{\partial\mathbf{p}}$$&lt;/span&gt; and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d\mathbf{p}}{dt}=-\frac{\partial \mathcal{H}}{\partial\mathbf{q}}.$$&lt;/span&gt;&#xA;This motion conserves the value of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{H}$&lt;/span&gt; along trajectories, but it also has an additional property, namely that if we define a mapping&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \mathbf{p}(t),\mathbf{q}(t)= \phi_t(\mathbf{p}(t_0),\mathbf{q}(t_0))$$&lt;/span&gt;&#xA;then this mapping conserves the two form &lt;span class=&quot;math-container&quot;&gt;$d\mathbf{p}\wedge d\mathbf{q}$&lt;/span&gt;. For a problem in which &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; are one dimensional you can think of this as saying that the area inside closed curves on the phase space are conserved. This ensures all kinds of nice stability properties, since &quot;balls&quot; of trajectories have to stay &quot;close&quot; to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of numerics, a symplectic integrator acts in the same way, also conserving this area/two form. In turn this means that there is a conserved &quot;numerical Hamiltonian&quot; (which may not be [read 'is not'] the same as the exact one). Note that stability is not the same as accuracy, so that most of the advantages of symplectic methods come when integrating for very long times (e.g. your method may rapidly place a satellite on the wrong side of the Earth, while never allowing it to decay into it).&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="29895" LastEditDate="2019-02-05T15:05:51.377" LastActivityDate="2019-02-05T15:05:51.377" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29158" PostTypeId="1" CreationDate="2018-03-26T16:57:28.550" Score="2" ViewCount="127" Body="&lt;p&gt;In the context of a scientific computing application, using data coming from (powerful) embedded systems, acquiring raw data (but from calibrated acquisition electronics), I have been asked to integrate these data (in RT), with an empirical sum, instead of making use of usual numerical methods. (Important note : even if we are dealing with embedded systems, computational power is not an issue).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I was asking why not using these usual numerical methods (e.g. Simpson's, midpoint rule, trapezoidal rule, etc.), I could not get any answer. Then, I am wondering why someone would want to perform it this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The acquisition application on the embedded system is in C++03, compiled with GCC 4.4.7, the average being performed looking somewhat like this (this code is executed each time the acquisition is performed) :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;float sum = 0.0; &#xA;for (int i = 0; i &amp;lt; DATA_BLOCK_LENGTH; i++) {&#xA;    sum += data[i]&#xA;}&#xA;sum /= DATA_BLOCK_LENGTH;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This snippet gives you an idea of the code performing the average from a trace of the signal received ; the signal usually looking like this most of the time (one or two spikes along the whole signal trace).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HO5ZN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HO5ZN.jpg&quot; alt=&quot;Acquisition trace example&quot;&gt;&lt;/a&gt;&#xA;The previous constant, DATA_BLOCK_LENGTH, is usually set to the maximum (2048, which is the size of electronics buffer). The measures are equally-spaced samples.&lt;/p&gt;&#xA;" OwnerUserId="20852" LastActivityDate="2018-03-26T16:57:28.550" Title="Why would someone use empirical sum instead of numerical integration methods?" Tags="|numerical-analysis|integration|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29162" PostTypeId="1" AcceptedAnswerId="29166" CreationDate="2018-03-27T11:11:06.090" Score="3" ViewCount="50" Body="&lt;h3&gt;Background&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;A neutral delay differential equation is one where the derivative does not only depend on its past state, but also the derivative at a past point:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \dot{y}(t) = f\big(t, y(t), y(t-τ_1), y(t-τ_2), …, \dot{y}(t-τ_1), \dot{y}(t-τ_2), …\big).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A clear problem when numerically integrating such DDEs is the initial discontinuity, i.e., discrepancies between $\dot{y}(t_0)$ as defined through the initial past and $f(y(t_0))$, where $t_0$ is the starting time of the numerical integration. For regular (retarded) DDEs this discontinuity quickly smooths out and thus can be defused with a few targeted steps in the beginning. For neutral DDEs, this does not happen.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Actual Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;When talking about numerically integrating neutral DDEs, all the literature I surveyed so far only mentions initial discontinuities as a new problem (in comparison to retarded DDEs). Is there any other problem that makes these kind of DDEs require special attention?&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2018-03-27T18:48:53.763" Title="Apart from initial discontinuities, what is tricky about neutral DDEs?" Tags="|differential-equations|dde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29164" PostTypeId="2" ParentId="29090" CreationDate="2018-03-27T16:41:31.043" Score="8" Body="&lt;p&gt;As the &quot;&lt;em&gt;No Free Lunch Theorems for Search&lt;/em&gt;&quot; [&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.5447&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;] states that &lt;strong&gt;there is no one particular optimization algorithm that works best for every problem&lt;/strong&gt;, i.e., citing the authors:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;all algorithms that search for an extremum of a cost&#xA;  function perform exactly the same, when averaged over all possible&#xA;  cost functions. In particular, if algorithm A outperforms algorithm B&#xA;  on some cost functions, then loosely speaking there must exist exactly&#xA;  as many other functions where B outperforms A.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, for a really good guidance, I would suggest to look at the recent article [&lt;a href=&quot;https://link.springer.com/article/10.1007%2Fs10898-012-9951-y&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;] (&lt;em&gt;written by two well-known optimizers&lt;/em&gt;), which presents a review and comparison of 22 derivative-free algorithms performed on a test set consisting of 502  &lt;strong&gt;convex/nonconvex&lt;/strong&gt;, &lt;strong&gt;smooth/nonsmooth&lt;/strong&gt; optimization problems.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;References&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;1: &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.5447&quot; rel=&quot;noreferrer&quot;&gt;Wolpert, D. H., &amp;amp; Macready, W. G. (1995). No free lunch theorems for search (Vol. 10). Technical Report SFI-TR-95-02-010, Santa Fe Institute.&#xA;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2: &lt;a href=&quot;https://link.springer.com/article/10.1007%2Fs10898-012-9951-y&quot; rel=&quot;noreferrer&quot;&gt;Rios, L. M., &amp;amp; Sahinidis, N. V. (2013). Derivative-free optimization: a review of algorithms and comparison of software implementations. Journal of Global Optimization, 56(3), 1247-1293.&#xA;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19438" LastActivityDate="2018-03-27T16:41:31.043" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29165" PostTypeId="2" ParentId="29090" CreationDate="2018-03-27T18:38:40.580" Score="2" Body="&lt;p&gt;As @Remis says, it's very problem-dependent. I think the best thing to do is use a nice package with a suite of methods and just test them all. &lt;a href=&quot;https://github.com/JuliaOpt/NLopt.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;NLopt.jl&lt;/a&gt; makes it easy to try &lt;a href=&quot;https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;quite a large number of algorithms&lt;/a&gt; (including a lot of derivative-free methods). The one that's best for your problem is the one that performs best.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-03-27T18:38:40.580" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29166" PostTypeId="2" ParentId="29162" CreationDate="2018-03-27T18:48:53.763" Score="2" Body="&lt;p&gt;Nope, that is pretty much the issue. With &quot;normal DDEs&quot; you know that the order of the discontinuity will decrease each time it is hit, so after $n$ times you can drop it (if $n$ is the order of the discretization). But for neutral DDEs, the order of the discontinuity doesn't necessarily drop after each time, so you have to continue to track them. This can get extremely costly if you have multiple delays because then you have to track every combination which adds up quickly and drives the stepsize down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other methods, of course you can get around that by relying purely on residual control but that gets far too costly when the discontinuities pile up so I haven't seen someone actually recommend doing this. There is an alternative approach by Shampine for the solve ddesnd &lt;a href=&quot;http://faculty.smu.edu/shampine/current.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;which can be found at his website&lt;/a&gt; (interestingly, this one was never added into the MATLAB ODE Suite) and does so by a &quot;dissipative&quot; transformation to a retarded DDE. &lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-03-27T18:48:53.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29167" PostTypeId="1" AcceptedAnswerId="29176" CreationDate="2018-03-27T23:04:48.673" Score="3" ViewCount="716" Body="&lt;p&gt;I am trying to solve/optimize &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; in the least squares sense subject to&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;box constraints;&lt;/li&gt;&#xA;&lt;li&gt;a few (less than 5) equality/inequality constraints; and&lt;/li&gt;&#xA;&lt;li&gt;an absolute function penalty (or some other piecewise linear convex function).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;That is, min &lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{x})=|A\mathbf{x}-b|^2+\mathbf{c}\cdot|\mathbf{x}|$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{c}\cdot|\mathbf{x}|=c_1|x_1|+...+c_n|x_n|$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In particular, &lt;span class=&quot;math-container&quot;&gt;$A\mathbf{x}=b$&lt;/span&gt; is massively underdetermined (i.e. &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; has several thousand columns but hundreds of rows) and any valid solution will do if many exist. This code is performance critical. I've exhausted everything I can think of. Is there any obvious way to solve this efficiently (either a clever formulation or appropriate choice of algorithm)?&lt;/p&gt;&#xA;&lt;p&gt;Language agnostic solutions would be appreciated.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; - Skip the rest. Here's what I have tried so far&lt;/p&gt;&#xA;&lt;h3&gt;Approach 1 - Keep it simple&lt;/h3&gt;&#xA;&lt;p&gt;Use Bounded BFGS and evaluate &lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{x})$&lt;/span&gt; directly. I am trying to improve on this as it is too slow.&lt;/p&gt;&#xA;&lt;h3&gt;Approach 2 - QP:&lt;/h3&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{x})=\mathbf{x^T}A^TA\mathbf{x}-2\mathbf{x^T}A^Tb+\mathbf{c}\cdot|\mathbf{x}|=\frac{1}{2}\mathbf{x^T}Q\mathbf{x}+\mathbf{d}\cdot\mathbf{x}+\mathbf{c}\cdot|\mathbf{x}|$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So this is basically standard quadratic form but &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is now enormous and positive semi-definite making it difficult to work with. Slightly slower.&lt;/p&gt;&#xA;&lt;h3&gt;Approach 2 - Use QR decomposition:&lt;/h3&gt;&#xA;&lt;p&gt;The idea is to reduce the size of the quadratic matrix using QR decomposition and a co-ordinate transformation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A^T=QR$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x^T}Q=\mathbf{y^T}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{x})=\mathbf{x^T}A^TA\mathbf{x}-2\mathbf{x^T}A^Tb+\mathbf{c}\cdot|\mathbf{x}|=\mathbf{y^T}RR^T\mathbf{y}+\mathbf{d}\cdot Q\mathbf{y}+\mathbf{c}\cdot|Q\mathbf{y}|$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This makes &lt;span class=&quot;math-container&quot;&gt;$A^TA$&lt;/span&gt; very small and fast to execute. However, the transformation forces the thousands of (quick) box constraints to become linear inequality constraints and this hammers the performance. A lot slower.&lt;/p&gt;&#xA;&lt;p&gt;Any other ideas? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="27309" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-03-28T22:21:03.027" Title="What is the fastest way to solve Ax=b (subject to constraints and an absolute term)" Tags="|optimization|constrained-optimization|performance|quadratic-programming|semidefinite-programming|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29169" PostTypeId="2" ParentId="23323" CreationDate="2018-03-28T06:55:19.777" Score="4" Body="&lt;p&gt;It can be even harmful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Liesen/Strakos &lt;em&gt;Krylov subspace methods principles and analysis&lt;/em&gt; (Chapter 5.8.3) it is reported that a nonzero initial &lt;code&gt;x0&lt;/code&gt; makes a GMRes iteration first remove unwanted components of &lt;code&gt;x0&lt;/code&gt; before it starts the approximation of the wanted solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a nonzero initial guess is used, it should be rescaled by the &lt;em&gt;Hegedus trick&lt;/em&gt;; see the explanation and implementation in the &lt;a href=&quot;http://krypy.readthedocs.io/en/latest/krypy.utils.html#krypy.utils.hegedus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Krypy package.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: Nonetheless I made good experience with a nonzero initial guess, when solving the linear systems within a time stepping scheme.  &lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2018-03-28T06:55:19.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29172" PostTypeId="2" ParentId="27496" CreationDate="2018-03-28T13:03:08.967" Score="3" Body="&lt;p&gt;If you use &lt;code&gt;qutip&lt;/code&gt;, partial trace operations are already built-in. For example, here is how you can compute the partial trace of a random density matrix over three qubits (that is, an hermitian, trace-1 matrix living in a tensor product space of dimensions $(2, 2, 2)$), and then trace out the last space:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import qutip&#xA;dm = qutip.rand_dm_hs(8, dims=[[2] * 3] * 2)&#xA;dm.ptrace([0, 1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On the other hand, there are circumstances in which you may not want to use &lt;code&gt;qutip&lt;/code&gt;.&#xA;Say therefore that you have a matrix stored in a numpy array, which represents a tensor in a tensor product space width dimensions $(d_1,...,d_n)$.&#xA;The matrix has therefore shape $(d_1\cdots d_n, d_1\cdots d_n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A convenient way to compute the partial trace is, how the other answer suggests, to first reshape the matrix and then to sum the appropriate axes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose you have a matrix representing an object in a space $V= V_1\otimes V_2$, with $V_1$ of dimension $2$ and $V_2$ of dimension $4$.&#xA;Here is how partial trace the first space:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import qutip&#xA;# generate test matrix (using qutip for convenience)&#xA;dm = qutip.rand_dm_hs(8, dims=[[2, 4]] * 2).full()&#xA;# reshape to do the partial trace easily using np.einsum&#xA;reshaped_dm = dm.reshape([2, 4, 2, 4])&#xA;# compute the partial trace&#xA;reduced_dm = np.einsum('ijik-&amp;gt;jk', reshaped_dm)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while to partial trace with respect to the &lt;em&gt;second&lt;/em&gt; space you just change the last line to&#xA;&lt;code&gt;reduced_dm = np.einsum('jiki-&amp;gt;jk', reshaped_dm)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the same thing as above, including also a consistency check with the partial trace given by &lt;code&gt;qutip&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import qutip&#xA;# generate test matrix (using qutip for convenience)&#xA;dm = qutip.rand_dm_hs(8, dims=[[2, 4]] * 2).full()&#xA;# reshape to do the partial trace easily using np.einsum&#xA;reshaped_dm = dm.reshape([2, 4, 2, 4])&#xA;# partial trace the second space&#xA;reduced_dm = np.einsum('jiki-&amp;gt;jk', reshaped_dm)&#xA;# check results with qutip&#xA;qutip_dm = qutip.Qobj(dm, dims=[[2, 4]] * 2)&#xA;reduced_dm_via_qutip = qutip_dm.ptrace([0]).full()&#xA;# check consistency of results&#xA;np.allclose(reduced_dm, reduced_dm_via_qutip)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="13137" LastActivityDate="2018-03-28T13:03:08.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29173" PostTypeId="1" CreationDate="2018-03-28T13:04:00.227" Score="2" ViewCount="146" Body="&lt;p&gt;I am basically asking about &lt;em&gt;eigenvalue problems&lt;/em&gt; of differential equations using some finite difference method (FDM). Usually the system is subject to some boundary conditions (BC), e.g., Dirichlet or Neumann. Often it is homogeneous BC like $u(0)=0,u'(R)=0$, which seems to be fine to deal with. One way is to introduce and then eliminate ghost points with the BC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if it is &lt;strong&gt;&lt;em&gt;inhomogeneous BC&lt;/em&gt;&lt;/strong&gt; (e.g., for a Dirichlet $u(0)=a,u(R)=b$), the nonzero constants have a sting in the tail. Effectively they're put into an inhomogeneous column as shown below, which is also clearly introduced in &lt;a href=&quot;http://www.math.toronto.edu/mpugh/Teaching/Mat1062/notes2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this note&lt;/a&gt;. For instance, the eigenvalue probelm $Lu=\lambda u$ of the operator $L=-\frac{\partial^2}{{\partial x}^2}$ becomes an &lt;em&gt;inhomogeneous eigenvalue problem&lt;/em&gt; $Mu+R=\lambda u$ with&#xA;$$M=\frac{1}{h^2}\begin{bmatrix}&#xA;2 &amp;amp;-1 &amp;amp; &amp;amp; &amp;amp; \\&#xA;-1 &amp;amp;2 &amp;amp;-1 &amp;amp; &amp;amp; \\&#xA;&amp;amp;\ddots  &amp;amp;\ddots &amp;amp;\ddots &amp;amp; \\&#xA;&amp;amp; &amp;amp; -1 &amp;amp;2 &amp;amp;-1\\&#xA; &amp;amp; &amp;amp; &amp;amp; -1 &amp;amp;2&#xA;\end{bmatrix},&#xA;R=\frac{1}{h^2}\begin{bmatrix}&#xA;a \\&#xA;0 \\&#xA;\vdots \\&#xA;0\\&#xA; b&#xA;\end{bmatrix}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;But after this, practically how to proceed to solve the original problem?&lt;/strong&gt; Any other formulation? Or we simply don't need to think about such problem for some reason?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may not be a problem for usual DE solving (I naively assume one-sided stencil often suffices.). But for eigenvalue case, I can't find any way out. The &lt;em&gt;inhomogeneous eigenvalue problem&lt;/em&gt; doesn't have any standard or general solution as far as I've searched.&lt;/p&gt;&#xA;" OwnerUserId="9408" LastEditorUserId="9408" LastEditDate="2018-03-29T01:11:06.903" LastActivityDate="2018-03-29T01:11:06.903" Title="Finite differenced eigenvalue prob. of inhomogeneous boundary conditions?" Tags="|finite-element|finite-difference|ode|boundary-conditions|eigenvalues|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29174" PostTypeId="1" CreationDate="2018-03-28T13:16:44.097" Score="4" ViewCount="136" Body="&lt;p&gt;Let $M$ be a manifold and $UT(M)$ its unit tangent bundle. I have a PDE which looks something like&#xA;$$&#xA;\frac{\partial f}{\partial t}(x,v) = (v\cdot \nabla_x) f(x,v) + \Delta f(x,v)&#xA;$$&#xA;where $x\in M$, $v \in T_xM$, i.e. $x$ is a point on the surface and $v$ is an unit vector at that point, $\nabla_x$ is gradient on the surface $M$ and $\Delta$ is laplacian on the unit tangent bundle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any software which would allow me to solve such equation numerically?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main concern is how do I obtain discretization of the unit tangent bundle? I would like to have a triangular mesh of my surface and then somehow get a discretization of its unit tangent bundle. Once I have the mesh I can probably use some standard library to solve the PDE.&lt;/p&gt;&#xA;" OwnerUserId="4561" LastActivityDate="2018-03-30T21:38:42.267" Title="Numerically solving a PDE on an unit tangent bundle" Tags="|pde|mesh-generation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29175" PostTypeId="1" AcceptedAnswerId="29194" CreationDate="2018-03-28T13:51:41.773" Score="1" ViewCount="494" Body="&lt;p&gt;I am trying to solve this problem where we have a 1D-lattice of size 100 and the particle can start from any position in the lattice and moves randomly on it(with equal probability of moving to either direction). When the particle reaches either of the ends of the lattice it stops. The problem is to plot the time average(to reach the boundary) vs initial starting position of the particle when for each starting position we take 10000 trails to compute the time average.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(1,size):&#xA;    timearr=[]&#xA;    for j in range(trails):&#xA;        time=0&#xA;        x=i&#xA;        while True:&#xA;            r = random.randint(-1,1)&#xA;            time+=1&#xA;            x+=r&#xA;            if x==0 or x==size:&#xA;                timearr.append(time)&#xA;                break&#xA;    timemean.append(np.mean(timearr))&#xA;&#xA;&#xA;xarr= np.arange(1,size)&#xA;plt.plot(xarr, timemean)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, taking size as 100 and trials as 10000(or even 1000) it takes a humongous amount of time to run this. Is there any way to solve this faster using Python. Ideally, I would like to do this using a vectorised approach using numpy arrays. But I can't think of how to implement control-flow in that case.&lt;/p&gt;&#xA;" OwnerUserId="27314" LastActivityDate="2018-03-30T13:07:06.543" Title="Performing a random walk on a lattice that traps the particles" Tags="|python|computational-physics|numpy|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29176" PostTypeId="2" ParentId="29167" CreationDate="2018-03-28T13:59:48.973" Score="3" Body="&lt;p&gt;This problem can be formulated as a standard positive semidefinite QP with bounded variables.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, deal with the absolute value terms in the objective by letting $x=u-v$, where $u\geq 0$ and $v \geq 0$.  Then write the problem as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \;\; (A(u-v)-b)^{T}(A(u-v)-b)+c^{T}u+c^{T}v$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subject to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B(u-v)=d$                (equality constraints)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upper bounds on u and v.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u,v \geq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then use an &lt;em&gt;active set method&lt;/em&gt; for QP to solve this QP.  In this approach, most of the variables are fixed at their lower or upper bounds, which dramatically reduces the size of the QP.  You solve a simple linear programming feasibility problem with the constraints $B(u-v)=d$ and the bounds constraints to find an initial feasible solution and an initial active set of variables.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In each iteration of the active set method you solve the reduced size QP&#xA;over the current set of active variables, and then check optimality conditions to see if any of the fixed variables should be released from their bounds and whether any of the free variables should be pinned to their upper or lower bounds.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Active set methods for QP are widely available- don't reinvent the wheel.    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2018-03-28T22:21:03.027" LastActivityDate="2018-03-28T22:21:03.027" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29178" PostTypeId="2" ParentId="28822" CreationDate="2018-03-28T14:50:59.360" Score="2" Body="&lt;p&gt;Please have a look at the really great review and comparison of 22 derivative-free algorithms [&lt;a href=&quot;https://link.springer.com/article/10.1007%2Fs10898-012-9951-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] (&lt;em&gt;written by two well-known and respected optimizers&lt;/em&gt;). Here, the authors use the terms &quot;&lt;em&gt;derivative-free&lt;/em&gt;&quot; and &quot;&lt;em&gt;black box optimization&lt;/em&gt;&quot; synonymously (emphasis is mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In addition, what we refer to as &lt;strong&gt;derivative-free optimization is often also referred to as optimization over black boxes&lt;/strong&gt;. The literature on these terms is often inconsistent and confusing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h2&gt;Reference&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;1: &lt;a href=&quot;https://link.springer.com/article/10.1007%2Fs10898-012-9951-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rios, L. M., &amp;amp; Sahinidis, N. V. (2013). Derivative-free optimization: a review of algorithms and comparison of software implementations. Journal of Global Optimization, 56(3), 1247-1293.&#xA;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19438" LastActivityDate="2018-03-28T14:50:59.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29179" PostTypeId="1" CreationDate="2018-03-28T15:42:32.913" Score="5" ViewCount="80" Body="&lt;p&gt;The Padua points $\mathrm{Pad}_{n} \subset [-1,1]^{2}$ are a unisolvent pointset with optimal growth of Lebesgue constant, described in detail &lt;a href=&quot;https://www.researchgate.net/profile/Stefano_De_Marchi/publication/220493368_Algorithm_886_Padua2D---Lagrange_Interpolation_at_Padua_Points_on_Bivariate_Domains/links/0912f50c24818e3388000000/Algorithm-886-Padua2D---Lagrange-Interpolation-at-Padua-Points-on-Bivariate-Domains.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. With some work they can be used to generate a multivariate Chebyshev series representation of the function they attempt to interpolate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to use the Padua points to generate a multivariate Chebyshev series of a function singular along the line $x=y$, but this is impossible because two of the Padua points are consecutive vertices of the square.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to generate a multivariate Chebyshev polynomial interpolant of a function $f\colon [-1,1]^2\to \mathbb{R}$ which is singular on the line $x=y$, perhaps using a modification of the Padua points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Update&lt;/em&gt;: The goal is to write a piece of code which projects as many 2D functions defined on rectangular regions as possible onto Chebyshev polynomials and allow for interpolation and recovery of the coefficients for perhaps other uses. This is similar to the goals of chebfun in 2D.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2018-03-31T02:46:05.867" LastActivityDate="2018-03-31T02:46:05.867" Title="Padua-type pointset for functions singular on line $x=y$" Tags="|algorithms|interpolation|projection|chebyshev|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="29180" PostTypeId="1" AcceptedAnswerId="29190" CreationDate="2018-03-28T16:35:32.167" Score="1" ViewCount="157" Body="&lt;p&gt;I wrote a code on Python 2.7.5 to solve numerically the following differential equation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$\frac{\partial^2f}{\partial x^2}=-S$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$S=\pi^{2}\sin(\pi x)$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;S is chosen that way in order to have $f= \sin(\pi x)$ as the exact solution, that can be compared with the constructed solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The differential operator is discretized using a second-order central finite difference method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When running the code, the order of magnitude of the evolution of the discretization error &lt;code&gt;Err&lt;/code&gt; is correct (2nd order) for the first batch of points of &lt;code&gt;N&lt;/code&gt; (10,20,40,80,100,800,2000,5000) (goes down to 10e-11).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I found that for very small &lt;code&gt;dx&lt;/code&gt; meaning, very big &lt;code&gt;Nx&lt;/code&gt; (10000,50000,100000,600000,900000), &lt;code&gt;Err&lt;/code&gt; doesn't stagnate at the machine precision error (as expected) but goes up for several magnitude orders, and even worse, the discretization error &lt;code&gt;Err&lt;/code&gt; deteriorates (up to 10e-7).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it a problem in the precision/round managment of Python for very small numbers, or is it normal to have even worse Error for that amount of points  ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to write the problem in different ways (Matrix, Loops, Matrix inversion technices) but the same phenomenon happened.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import scipy.sparse as sp&#xA;from scipy.sparse.linalg.dsolve import spsolve&#xA;&#xA;Err=[]&#xA;N=np.array([10,20,40,80,100,800,2000,5000,10000,50000,100000,600000,900000])&#xA;&#xA;#Loop on the number of points used for the discretization of the 1D domain   &#xA;for Nx in N:&#xA;&#xA;    dx=(1.0)/Nx# fixed distance between two consecutive points in the domain&#xA;    x = np.linspace(dx,1.0-dx,Nx-1)# points of the 1D domain&#xA;&#xA;&#xA;    #contruction of the diff operator discretization Matrix&#xA;    data = [np.ones(Nx-1), -2*np.ones(Nx-1), np.ones(Nx-1)]# Diagonal terms&#xA;    offsets = np.array([-1, 0, 1])# Their positions&#xA;    LAP = sp.dia_matrix((data, offsets), shape=(Nx-1, Nx-1))#Sparse matrix&#xA;    S = np.pi**2*np.sin(np.pi*x)*dx**2# Second member &#xA;    f = spsolve(LAP,-S)#Resolution of the matrix system &#xA;&#xA;    f_ana=np.sin(np.pi*x)#The exact solution&#xA;&#xA;    Err.append((np.absolute((f_ana-f))).max())&#xA;&#xA;plt.figure()&#xA;plt.plot(N,Err,N,N**float(-2),'--')&#xA;plt.legend(['Err','Theoretical asymptotic behaviour'])&#xA;plt.xlabel('dx')&#xA;plt.ylabel('Err')&#xA;plt.xscale('log')&#xA;plt.yscale('log')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27315" LastActivityDate="2018-03-29T18:01:32.760" Title="Discretization Error amplification instead of stagnation to machine precision" Tags="|numerical-analysis|finite-difference|python|discretization|error-estimation|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29181" PostTypeId="2" ParentId="29180" CreationDate="2018-03-28T23:20:28.647" Score="3" Body="&lt;p&gt;Yes. This behavior is to be expected and normal. When you are computing with a small value for $\mathrm dx$ then, to compute the difference quotient, you are subtracting two numbers that are nearly the same. In this situation, you observe a so called catastrophic cancellation, meaning nearly all significant digits are lost (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Loss_of_significance&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2018-03-28T23:20:28.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29182" PostTypeId="1" AcceptedAnswerId="29189" CreationDate="2018-03-29T00:15:37.090" Score="5" ViewCount="290" Body="&lt;p&gt;A nonlinear least-squares problem with $F:\mathbb{R}^m\to\mathbb{R}^n$,&#xA;$$&#xA;F(x) \to \min_x \quad (\text{in the least-squares sense})&#xA;$$&#xA;really means minimizing&#xA;$$&#xA;\frac{1}{2} \|F(x)\|^2 \to \min_x.&#xA;$$&#xA;In principle, one could throw any old minimization method at it and get something. In turn, given any minimization problem $f(x)\to\min_x$, one could solve the gradient system $\nabla f(x)=0$ ($\nabla f: \mathbb{R}^m\to\mathbb{R}^m$) using least-squares methods. It almost appears that minimization and nonlinear least-squares algorithms are solving the same problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still, there are specific nonlinear least-squares methods &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;available in software&lt;/a&gt;: Trust-Region Reflective, Dogbox, Levenberg-Marquardt etc. How do they exploit the structure of the problem?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2018-03-29T08:40:16.997" LastActivityDate="2018-03-29T15:30:03.630" Title="Nonlinear least-squares solvers vs. generic minimization" Tags="|optimization|nonlinear-equations|least-squares|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29183" PostTypeId="2" ParentId="29180" CreationDate="2018-03-29T01:56:06.367" Score="1" Body="&lt;p&gt;@H.Rittich states one reason in their answer. Another reason is that when you go to more and more points, you have to do more and more computations, each of which adds its own amount of round-off error. Consequently, the smaller your $\delta x$ gets, the more computations and the more accumulated round-off -- it doesn't just stay constant, but actually grows.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-03-29T01:56:06.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29184" PostTypeId="2" ParentId="29174" CreationDate="2018-03-29T02:15:11.033" Score="3" Body="&lt;p&gt;The equation you are trying to solve in in essence the analogue of the &quot;radiative transfer&quot; equation when posed on a manifold. There is a vast literature in this area -- both in astrophysics and in nuclear physics, two areas in which this equation is important --, with the obvious difficulty being that your domain is pretty high-dimensional: if your manifold is three-dimensional, then you have a five-dimensional problem. (For 2d manifolds, the problem is three-dimensional.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typical ways to address this are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To approximate the equation by a sequence of equations that resemble diffusion equations in $x$&lt;/li&gt;&#xA;&lt;li&gt;To discretize the angular direction $v$ on a separate mesh from the spatial mesh, and then obtain a tensor product mesh in both $x$ and $v$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are many other approaches, such as ray tracing or Monte Carlo methods. Search the literature! &lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2018-03-29T23:57:06.207" LastActivityDate="2018-03-29T23:57:06.207" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="29187" PostTypeId="1" CreationDate="2018-03-29T11:59:35.877" Score="1" ViewCount="61" Body="&lt;p&gt;If $x_k$ are the Chebyshev nodes, that is for $n \in \mathbb{N}^*$, we have $x_k = \cos(\pi \frac{2k + 1}{n})$. Now suppose you have approximated values of $x_k$ and $x_{k'}$ for $k,k' \leq n$. In the worst case, what is the number of significant values we may loose when we calculate $x_{k'} - x_k$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, taking into account the fact that each real is represented as $x \approx m \times b^e$, we then have a formula that  for $x'$ the approximated value of $x$ and $y'$ approximated value of $y$, $\Delta (x' + y') \leq \epsilon(\Delta| x| + \Delta |y| + |x| + |y|)$ where $\epsilon = b^{1-N}$, where $N$ is there number of significant numbers after the comma.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for a fixed $N \in \mathbb{N}^*$ we have $\Delta (x_{k'}' - x_k') \leq \epsilon (\Delta |x_{k'}| + \Delta |x_k| +  |x_{k'}| + |x_k| ) \leq 4 \epsilon $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus in worst case we have a loss of precision of $\frac{4}{b^{N-1}}$. Thus we lose one significant term?&lt;/p&gt;&#xA;" OwnerUserId="27320" LastEditorUserId="4392" LastEditDate="2018-03-29T13:00:34.573" LastActivityDate="2018-03-29T13:00:34.573" Title="Numerical analysis: Chebyshev coefficient representation error" Tags="|numerical-analysis|error-estimation|polynomials|chebyshev|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29189" PostTypeId="2" ParentId="29182" CreationDate="2018-03-29T15:30:03.630" Score="5" Body="&lt;p&gt;If we let &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\phi(x)=\sum_{i=1}^{m} F_{i}(x)^{2}$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we could compute $\nabla \phi(x)$ by finite difference approximation.  However, it is generally smart to make use of the special structure of $\phi(x)$ and derivatives of $F_{i}(x)$.  Using the chain rule and exploiting the sum of squares structure of $\phi(x)$ we obtain that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \phi(x)=2J(x)^{T}F(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $J(x)$ is the Jacobian of $F(x)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, we can obtain the Hessian of $\phi(x)$ as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^{2} \phi(x)=2J(x)^{T}J(x)+2\sum_{i=1}^{m} F_{i}(x)Q_{i}(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q_{i}(x)=\nabla^{2}F_{i}(x)$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These formulas for $\nabla \phi(x)$ and $\nabla^{2} \phi(x)$ can be used within whatever general purpose nonlinear optimization algorithm you want to use- you'll get a version specialized for least squares problems.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Gauss-Newton method, an approximation to the Hessian of $\phi(x)$,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^{2} \phi(x) \approx 2J(x)^{T}J(x)$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;is used.  This approximation drops the second order terms $\sum_{i=1}^{m} F_{i}(x)Q(x)$ which tend to be small, particularly when $F_{i}(x)$ is close to 0.  Only the Jacobian (first derivative) of $F(x)$ is used.  This leads to a Newton's method step equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ J(x^{(k)})^{T}J(x^{(k)}) \Delta x = -J(x^{(k)})^{T}F(x^{(k)})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Levenberg-Marquadrt algorithm takes this one step further, by regularizing the Gauss-Newton methods to ensure improvement in $\phi(x)$ at each iteration.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-03-29T15:30:03.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29190" PostTypeId="2" ParentId="29180" CreationDate="2018-03-29T18:01:32.760" Score="2" Body="&lt;p&gt;Putting a more systematic point of view to the particular problems in the answer of H. Rittich, the matrices &lt;code&gt;LAP&lt;/code&gt; are nearly singular for large &lt;code&gt;N&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigensystem &#xA;$$&#xA;v_{k-1}-2v_k+v_{k+1}=\lambda v_k,~~v_0=v_{N+1}=0&#xA;$$&#xA;has basis solutions $v_k=q^k-q^{-k}$ where $q^{2N+2}=1$ for $v_{N+1}=0$. Then the eigenvalue computes as $\lambda=q+q^{-1}-2$, which means &#xA;$$&#xA;q_m=\exp(i\frac{m\pi}{N+1}),\\&#xA;v_{m,k}=2\cos(\frac{km\pi}{N+1}),\\&#xA;\lambda_m=2(\cos(m\frac{\pi}{N+1})-1)=-4\sin^2(\frac{m\pi}{2(N+1)})&#xA;$$&#xA;for $m=1,...,N$. Thus the ratio of largest to smallest eigenvalue, in other words the condition number of &lt;code&gt;LAP&lt;/code&gt;, is&#xA;$$&#xA;\kappa({\sf LAP})=\frac{\sin^2(\frac{N\pi}{2(N+1)})}{\sin^2(\frac{\pi}{2(N+1)})}\approx \frac{4(N+1)^2}{\pi^2}.&#xA;$$&#xA;Floating point noise gets magnified by that number in the relative error formula, see the general perturbation theory of linear systems. This already includes effects of catastrophic cancellation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JnbI4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JnbI4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one can see, the truncation error stops to be close to the observed error at the point that the (theoretically possible magnitude of the) magnified floating point noise starts to dominate the truncation error.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2018-03-29T18:01:32.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29192" PostTypeId="2" ParentId="23222" CreationDate="2018-03-30T04:20:26.360" Score="0" Body="&lt;p&gt;My heuristic method is that the line with the most intersections will be a line joining points on the convex hull. You could use a variant of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Rotating_calipers&quot; rel=&quot;nofollow noreferrer&quot;&gt;rotating calipers algorithm&lt;/a&gt; and test the lines between antipodal points on the hull. The number of intersections depends on the points &quot;between&quot; the antipodal pair. The exact number depends on how many points are on each side of the line joining the antipodal pair. The number of points on one side multiplied by the number of points on the other determines the maximum number of intersections possible. The exact number may be less because multiple lines may intersect in the same place. Thus, actual evaluations will be needed but this basic heuristic can narrow down the number of lines to check.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the given example, line AD are basically antipodal points. Since points B and C are on one side and points E and F are on the other, then there are 2 x 2 = 4 lines between them and thus 4 intersections maximum (which agrees with the diagram). If B was on the other side of AD, then there would be 3 and 1 points on either side, respectively and BF and BE wouldn't intersect AD, but BC would leaving 3 intersections. The intersections Q and R are close to each other, and even if the points are not collinear Q and R could be coincident resulting in fewer intersections than the maximum given by this heuristic. If 3 or more points are collinear, then there will also be fewer intersections than the maximum.&lt;/p&gt;&#xA;" OwnerUserId="21131" LastEditorUserId="21131" LastEditDate="2018-03-30T16:38:24.380" LastActivityDate="2018-03-30T16:38:24.380" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29193" PostTypeId="2" ParentId="23222" CreationDate="2018-03-30T04:57:07.167" Score="-2" Body="&lt;p&gt;You simply take each $n\choose 2$ line segment and then for each other line segment see if they intersect. This is tallied for every single line segment. The line segment with the most intersection is your segment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;^ The above part is easy... It's like this is somebody's idea of a joke. Anyways...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the method to see if there is an intersection has occurred is the more technical part. I'm not well versed in Computer Graphics, so I wouldn't know conclusively right away the best method.&lt;/p&gt;&#xA;" OwnerUserId="27328" LastActivityDate="2018-03-30T04:57:07.167" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29194" PostTypeId="2" ParentId="29175" CreationDate="2018-03-30T13:07:06.543" Score="4" Body="&lt;p&gt;This can be solved much more efficiently if you recast the problem as finding the mean first passage time (MFPT) of a Markov chain for a given starting state. You can easily represent the random walk on your lattice by a transition matrix and find the MFPT for each of the states. The problem can be solved using (numerical) linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A recent survey of methods to accomplish this can be found in: &lt;a href=&quot;https://arxiv.org/abs/1701.07781&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1701.07781&lt;/a&gt; Good books dealing with the topic are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;J. G. Kemeny and J. L. Snell, Finite Markov chains. D. Van Nostrand Co., Inc., Princeton, NJ-Toronto-London-New York, 1960.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;D. A. Levin, Y. Peres, and E. L. Wilmer, Markov chains and mixing times. American Mathematical Society, Providence, RI, 2009.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2018-03-30T13:07:06.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29196" PostTypeId="1" CreationDate="2018-03-30T16:24:27.577" Score="0" ViewCount="1275" Body="&lt;p&gt;I am using cvxpy to do a simple portfolio optimization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented the following dummy code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from cvxpy import *&#xA;import numpy as np&#xA;&#xA;np.random.seed(1)&#xA;n = 10&#xA;&#xA;Sigma = np.random.randn(n, n) &#xA;Sigma = Sigma.T.dot(Sigma)&#xA;&#xA;orig_weight = [0.15,0.25,0.15,0.05,0.20,0,0.1,0,0.1,0]&#xA;w = Variable(n)&#xA;&#xA;mu = np.abs(np.random.randn(n, 1))&#xA;ret = mu.T*w&#xA;&#xA;lambda_ = Parameter(sign='positive')&#xA;lambda_ = 5&#xA;&#xA;risk = quad_form(w, Sigma)&#xA;&#xA;constraints = [sum_entries(w) == 1, w &amp;gt;= 0, sum_entries(abs(w-orig_weight)) &amp;lt;= 0.750]&#xA;&#xA;prob = Problem(Maximize(ret - lambda_ * risk), constraints)&#xA;&#xA;prob.solve()&#xA;&#xA;print 'Solver Status : ',prob.status&#xA;&#xA;print('Weights opt :', w.value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am constraining on being fully invested, long only and to have a turnover of &amp;lt;= 75%. However I would like to use turnover as a &quot;soft&quot; constraint in the sense that the solver will use as little as possible but as much as necessary, currently the solver will almost fully max out turnover. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I basically want something like this which is convex and doesn't violate the DCP rules&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sum_entries(abs(w-orig_weight)) &amp;gt;= 0.05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would assume this should set a minimum threshold (5% here) and then use as much turnover until it finds a feasible solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried rewriting my objective function to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prob = Problem(Maximize(lambda_ * ret - risk - penalty * max(sum_entries(abs(w-orig_weight))+0.9,0))  , constraints)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where penalty is e.g. 2 and my constraint object still looks like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;constraints = [sum_entries(w) == 1, w &amp;gt;= 0, sum_entries(abs(w-orig_weight)) &amp;lt;= 0.9]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have never used soft-constraints and any explanation would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="24987" LastEditorUserId="24987" LastEditDate="2018-04-03T15:02:21.387" LastActivityDate="2018-04-03T15:02:39.943" Title="Defining a soft constraint in cvxpy" Tags="|optimization|constrained-optimization|constraints|cvxpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29197" PostTypeId="1" AcceptedAnswerId="29200" CreationDate="2018-03-30T19:35:57.220" Score="2" ViewCount="760" Body="&lt;p&gt;I have understood how &lt;a href=&quot;https://en.wikipedia.org/wiki/CUR_matrix_approximation&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUR&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular-value_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;SVD&lt;/a&gt; works, but have not been able to understand the following.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can we use CUR in place of the SVD decomposition?&lt;/li&gt;&#xA;&lt;li&gt;Do the $C$ and $R$ matrices in the CUR follow the same properties as those of $U$ and $V$ matrices in the SVD decomposition?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If we want to reduce the dimension of the original matrix, say, from $n$ to $k$, which matrix of CUR we can use to project original matrix, so that we will get $k$-dimensional data points?&lt;/p&gt;&#xA;" OwnerUserId="27334" LastEditorUserId="20688" LastEditDate="2018-03-31T01:37:11.797" LastActivityDate="2018-03-31T01:37:11.797" Title="How we can use CUR decomposition in place of SVD decomposition?" Tags="|linear-algebra|svd|matrix-factorization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29198" PostTypeId="2" ParentId="29174" CreationDate="2018-03-30T21:38:42.267" Score="2" Body="&lt;p&gt;I looked around a bit and found &lt;a href=&quot;http://www.imr.sandia.gov/papers/imr21/RNYin.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (featuring no less than Shing-Tung Yau!) about the problem of generating meshes for the unit tangent bundle $UT(S^2)$ on the 2-sphere $S^2$. As you point out, $UT(M)$ probably isn't diffeomorphic to $M \times S^n$. For example in Yin's paper they point out that $UT(S^2)$ is non-trivial and can't be embedded in $\mathbb{R}^3$. However, locally the bundle is trivializable, and Yin's paper suggests that one approach could be to divide the manifold into trivializable patches, then explicitly compute the transition maps between patches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be able to implement their idea to generate turn a mesh for $M$ into a mesh for $UT(M)$, which would have to be embedded into a much higher-dimensional Euclidean space. While many finite element libraries (&lt;a href=&quot;https://www.dealii.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;deal.II&lt;/a&gt; among them) can solve PDEs on surfaces, I think they usually assume that it's a 1D or 2D surface embedded in 2D or 3D respectively. If you were working with something as simple as $UT(S^2)$, you'd have to embed it in $\mathbb{R}^4$, which probably falls outside the purview of many libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try to work only with a mesh for $M$ or, more in the spirit of the paper, meshes for a set of overlapping patches $M_\alpha$ such that $UT(M)$ is trivializable on each $M_\alpha$. Provided you have a way to calculate the transition maps, you can represent $f$ within each patch as a tensor product of a Galerkin basis function $\phi$ defined on $M_\alpha$ with the spherical harmonics of the appropriate dimension. You could then try and solve your PDE using, say, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Schwarz_alternating_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Schwarz alternating method&lt;/a&gt; or some other &lt;a href=&quot;https://en.wikipedia.org/wiki/Domain_decomposition_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;domain decomposition&lt;/a&gt; method. Of course you'll also have to partition the mesh first. &lt;a href=&quot;http://glaros.dtc.umn.edu/gkhome/metis/metis/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;METIS&lt;/a&gt; is a great tool for this, but you could very well get a partition with a patch that isn't trivializable. It might be possible to then recursively partition non-trivial patches. Or it might be easiest to implement this yourself.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-03-30T21:38:42.267" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29199" PostTypeId="1" AcceptedAnswerId="29225" CreationDate="2018-03-30T23:53:55.010" Score="1" ViewCount="5141" Body="&lt;p&gt;I am working on a implementing a simple quadratic optimisation problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min _x \; {\underline{x}}^T Q {\underline{x}}$$&#xA;$$s.t. \,\quad {\underline{\mu}}^T{\underline{x}} = R^*$$&#xA;$$ \quad \quad \underline{1}^T\underline{x} = 1 $$&#xA;I also expect to go on to include the inequality constraint, as an additional complexity, once the above is working.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x_i \geq 0$$&#xA;The method I think is simplest, and which I understand best for implementing &#xA;these constraints, is the penalty function method, where we modify the objective function to 'steer' the optimisation away from forbidden regions. By carefully parameterising the size of the penalties, I have achieved good results using SciPy's built-in Nelder-Mead Simplex algorithm, using the objective function below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def objective(x):&#xA;    Q = DF.cov()     # Covariance matrix&#xA;&#xA;    # Penalty Function method&#xA;&#xA;    penalty1 =  0.0005 * abs(np.sum(x)-1)                             # Large for sum(x) &amp;lt;&amp;gt; 1&#xA;    penalty2 =  0.05 * abs(R_min - np.matmul(Mus.transpose(), x))     # Large for returns &amp;lt;&amp;gt; R_min&#xA;&#xA;    return np.matmul(x.transpose(),np.matmul(Q,x)) + penalty1 + penalty2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I wish to use other optimisation algorithms (in particular BFGS and Newton-CG), which require the gradient and Hessian of the objective function. &lt;a href=&quot;https://scicomp.stackexchange.com/questions/29080/how-to-define-the-derivative-for-scipy-optimize-minimize&quot;&gt;I have implemented the derivative functions in the unconstrained case&lt;/a&gt;, but by adding the penalty terms to the objective (and the derivatives of the penalties to the gradient function) optimisation fails with the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Warning: Desired error not necessarily achieved due to precision loss.&#xA;     Current function value: 0.000056&#xA;     Iterations: 0&#xA;     Function evaluations: 780&#xA;     Gradient evaluations: 96&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Previously Iterations would be a few hundred). This strictly occurs with penalty1, but not penalty2 by itself, so either my derivative is wrong for penalty1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;penalty1_der = np.sign(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or can I not use the L1 norm in this way? I also tried replacing the constraint with a smoother, quadratic approximation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;penalty1 = np.matmul((x - vector_ones).transpose(), (x - vector_ones))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but unfortunately, although this prevents the error, Minimize() seems to completely ignore my penalty functions (even with vastly increased parameters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I implement my constraints such that I can solve the problem using BFGS/Newton-CG?&lt;/p&gt;&#xA;" OwnerUserId="25837" LastEditorUserId="25837" LastEditDate="2018-04-03T01:21:22.290" LastActivityDate="2018-04-03T01:23:11.137" Title="How to Solve Optimisation Problems using Penalty Functions in Python" Tags="|convex-optimization|constrained-optimization|scipy|penalty-method|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29200" PostTypeId="2" ParentId="29197" CreationDate="2018-03-31T01:35:06.973" Score="1" Body="&lt;p&gt;In SVD decomposition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A_{m\times n}= U_{m\times m} \Sigma_{m\times n}V^{H}_{n\times n}&#xA;$$&#xA;Here, $U$ and $V$ are unitary matrices of left- and right singular vectors, and $\Sigma$ is a diagonal matrix containing real singular values in a decreasing order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your application, you are using a &lt;em&gt;truncated&lt;/em&gt; SVD (rank $k$), thus&#xA;$$&#xA;A_{m\times n}\approx U_{m\times k} \Sigma_{k\times k}V^{H}_{k\times n}&#xA;$$&#xA;where $U$ and $V$ are parts of unitary matrices containing left and right singular vectors corresponding to $k$ largest singular values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare that to a $k$-rank CUR decomposition:&#xA;$$&#xA;A_{m\times n}\approx C_{m\times k} U_{k\times k}R_{k\times n}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$C$ and $R$ in CUR decomposition contain $k$ columns and $k$ rows of $A$. Since rank-$k$ CUR decomposition is not unique, different CUR decomposition will feature different rows\columns.&lt;/li&gt;&#xA;&lt;li&gt;Entries of $C$ and $R$ &lt;em&gt;come directly from&lt;/em&gt; $A$ and have an easy interpretation, as opposed to entries of $U$ and $V$ in SVD decomposition.&lt;/li&gt;&#xA;&lt;li&gt;$C$ and $R$ are &lt;em&gt;not unitary&lt;/em&gt; in any way, as opposed to $U$ and $V$ in SVD decomposition.&lt;/li&gt;&#xA;&lt;li&gt;$U$ in CUR decomposition is a &lt;em&gt;full $k\times k$ matrix&lt;/em&gt;, as opposed to a &lt;em&gt;real diagonal $k\times k$ matrix&lt;/em&gt; $\Sigma$.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, taking into account that, I would not say that $C$ and $R$ in CUR decomposition follow the same properties as $U$ and $V$ in SVD decomposition. I would say, the only property that is the same between them is their dimensions ($m\times k$ and $k\times n$ in the rank-$k$ case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how can you use CUR instead of SVD. Instead of using a &lt;strong&gt;unique&lt;/strong&gt; (not discussing repeating singular values) rank-$k$ SVD decomposition, that provides you with the best rank-$k$ approximation (in the Frobenius-norm sense), you will have to use &lt;strong&gt;one-of-many&lt;/strong&gt; CUR decomposition. Depending on your purpose, you may want to choose one subset of CUR decomposition over the other, that is very problem- and algorithm dependent.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-03-31T01:35:06.973" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29201" PostTypeId="1" CreationDate="2018-03-31T14:49:23.597" Score="1" ViewCount="24" Body="&lt;p&gt;I'm implementing a function $f(x_1, x_2, \dots x_n)$ given by the ratio of two closed form equations.&#xA;$$ f(x_1, x_2, \dots, x_n) = \frac{g(x_1, x_2, \dots, x_n)}{h(x_1, x_2, \dots, x_n)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, when $x_1 = 0$, both $g$ and $h$ are $0$. However there is a well-defined limit&#xA;$$ k (x_2, x_3, \dots x_n) = \lim_{x_1 \to 0} f(x_1, x_2, \dots x_n) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a sensible way of implementing this? I was thinking of&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if |x_1| &amp;gt; cutoff:&#xA;   return f(x_1, ..., x_n)/g(x_1, ..., x_n)&#xA;else:&#xA;   return k(x_2, ... x_n)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I determine a suitable cutoff?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to research this question and I came across Higham's book on Accuracy and Stability of Numerical Algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a relevant example in section 1.14.1 where he discusses&#xA;$f(x) = (e^x-1)/x$. His solution to the instability is different from my naive approach above; he uses a change of coordinates $y = e^x$ that somehow reduce the inaccuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any suggestions/references on how to tackle this problem.&lt;/p&gt;&#xA;" OwnerUserId="21499" LastActivityDate="2018-03-31T15:22:36.883" Title="Implementation of a ratio with a well-defined limit" Tags="|reference-request|stability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29202" PostTypeId="2" ParentId="29201" CreationDate="2018-03-31T15:06:44.097" Score="1" Body="&lt;p&gt;Expand your function $f$ in powers of $x_1$ and compute the error term. Use the expansion when error is under your desired threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine you linearise the function $f$. You will have that:&#xA;$$f=f_0+f'x_0+\frac{1}{2}f''(\xi)x_0^2$$&#xA;Being the last term the error $\epsilon$. Chose $\xi$ as the value that maximises the absolute value of $f''(x_0)$ which is obtained from $f'''(\xi))=0$ &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if epsilon&amp;lt;threshold&#xA;    f=f_0+f'x_0&#xA;else&#xA;    f=g/h&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-03-31T15:22:36.883" LastActivityDate="2018-03-31T15:22:36.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29203" PostTypeId="1" AcceptedAnswerId="29216" CreationDate="2018-03-31T15:37:49.213" Score="4" ViewCount="574" Body="&lt;p&gt;I have a problem that I have been attempting to solve for a few days now. I was wondering if I would be able to get some assistance from the community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to detect if a point is in a polygon, I am using the winding number method. The algorithm is based on what I found on this site here: &lt;a href=&quot;http://geomalgorithms.com/a03-_inclusion.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://geomalgorithms.com/a03-_inclusion.html&lt;/a&gt;. In short, the algorithm will return a result that will indicate if the point is to the left or right of a line segment. For CCW orientation of the line, positive results mean that the point is to the left of the line and negative values indicate that the point is to the right of the line. The signs are switched for CW orientation. If the number of &quot;lefts&quot; is greater then the number of &quot;rights&quot;, then the point is inside of the polygon. Lines whose endpoints are between the point are only considered in order to speed up the algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my polygons (I use this term loosely) are composed of both arcs and line segments. These polygons are simple and do not loop back on themselves. The polygons can either be convex or concave. As a side note, arcs in my program are represented as circular arcs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The winding number algorithm works perfectly for line segments. But when we get to arcs, things can be a little bit more complicated. My current implementation is to consider any point that is underneath the curve of the arc would be inside of the arc. (Note: Source code for the implementation of my algorithm is given at the end of the post)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This implementation mostly works but, from the picture below, the red dot would be given as a false positive. The winding number algorithm would indicate that the red dot is inside the polygon when it clearly is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VgxRI.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VgxRI.jpg&quot; alt=&quot;False Positive&quot;&gt;&lt;/a&gt;&#xA;From the algorithm, line 1 will indicate that the point is to the left and line 4 will indicate that the point is to the right. But arc segment 2 will indicate that the point is also to the left and thus the point is inside of the polygon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the implementation with the arcs has another fault where the algorithm will return a false negative. Below is a picture of geometry where the point in the center would be outside of the polygon and the point off to the side is inside. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8FyH2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8FyH2.jpg&quot; alt=&quot;False Negative example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This polygon is composed of only arcs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a few days now, I have been considering different implementations but so far, nothing has panned out. Another thing about the arcs is that they are all in the CCW orientation. Whether the arc is concave/convex depends on which node is selected as the first node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am considering moving back to the ray crossing algorithm but I would like to see if I can get this to work first. I am fine with if I have 2 different algorithms for lines and arcs (in fact, I think that I will). I ideally, I would like to utilize the winding number algorithm but I am open to additional solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am posting a link to the source file for the code in case you want to see a more complete solution. I am also posting snippets that are relevant to convince. If you have any questions regarding the code, please do not hesitate to ask. Thank you&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Point in polygon test: &lt;a href=&quot;https://github.com/philm001/Omni-FEM/blob/master/Include/common/ClosedPath.h&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/philm001/Omni-FEM/blob/master/Include/common/ClosedPath.h&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bool pointInContour(wxRealPoint point)&#xA;    {&#xA;        int windingNumber = 0;&#xA;        bool isFirstRun = true;&#xA;        bool reverseWindingResult = false;&#xA;&#xA;        double additionTerms = 0;&#xA;        double subtractionTerms = 0;&#xA;&#xA;        // First, the algorithm will need to ensure that all of the lines are oriented in either CCW or CW.&#xA;        // Each contour, the order of the lines is CCW or CW but the order of the nodes is different.&#xA;        // For example, the start node of the 1st line could be &quot;connected&quot; to the end node of the next line.&#xA;        // In which case, we need to swap the nodes of the 1st line.&#xA;&#xA;        for(auto lineIterator = p_closedPath.begin(); lineIterator != p_closedPath.end(); lineIterator++)&#xA;        {&#xA;            auto nextLineIterator = lineIterator + 1;&#xA;&#xA;            if(nextLineIterator == p_closedPath.end())&#xA;                nextLineIterator = p_closedPath.begin();&#xA;&#xA;            if(isFirstRun)&#xA;            {&#xA;                if(*(*lineIterator)-&amp;gt;getSecondNode() != *(*nextLineIterator)-&amp;gt;getFirstNode())&#xA;                {&#xA;                    if(*(*lineIterator)-&amp;gt;getFirstNode() == *(*nextLineIterator)-&amp;gt;getFirstNode())&#xA;                        (*lineIterator)-&amp;gt;swap();&#xA;                    else if(*(*lineIterator)-&amp;gt;getFirstNode() == *(*nextLineIterator)-&amp;gt;getSecondNode())&#xA;                    {&#xA;                        (*lineIterator)-&amp;gt;swap();&#xA;                        (*nextLineIterator)-&amp;gt;swap();&#xA;                    }&#xA;                    else if(*(*lineIterator)-&amp;gt;getSecondNode() == *(*nextLineIterator)-&amp;gt;getSecondNode())&#xA;                        (*nextLineIterator)-&amp;gt;swap();&#xA;                }&#xA;&#xA;                isFirstRun = false;&#xA;            }&#xA;            else&#xA;            {&#xA;                if(*(*lineIterator)-&amp;gt;getSecondNode() == *(*nextLineIterator)-&amp;gt;getSecondNode())&#xA;                    (*nextLineIterator)-&amp;gt;swap();&#xA;            }&#xA;&#xA;            /* &#xA;             * This is the actual shoelace algorithm that is implemented. In short, we have two terms, addition terms&#xA;             * and subtraction terms. THe addition terms are the summation of the Xpoint of the first node multiplied by&#xA;             * the Ypoint of the second node for all edges (arcs are a special case)&#xA;             * &#xA;             * The subtraction term is the summation of the Xpoint of the second node multiplied by the&#xA;             * Ypoint of the first node (arcs are a special case)&#xA;             */ &#xA;            if((*lineIterator)-&amp;gt;isArc())&#xA;            {&#xA;                /* This algorithm is not being used to accurately determine the arc within a closed contour&#xA;                 * with that said, we only need to approximately determine the area. For arcs, we shall draw&#xA;                 * a line from the first node to the mid point and then from the mid point to the second node.&#xA;                 * We will use these two lines as the calculation for the shoelace algorithm.&#xA;                 */ &#xA;                additionTerms += ((*lineIterator)-&amp;gt;getFirstNode()-&amp;gt;getCenter().x) * ((*lineIterator)-&amp;gt;getMidPoint().y);&#xA;                subtractionTerms += ((*lineIterator)-&amp;gt;getMidPoint().x) * ((*lineIterator)-&amp;gt;getFirstNode()-&amp;gt;getCenter().y);&#xA;&#xA;                additionTerms += ((*lineIterator)-&amp;gt;getMidPoint().x) * ((*lineIterator)-&amp;gt;getSecondNode()-&amp;gt;getCenter().y);&#xA;                subtractionTerms += ((*lineIterator)-&amp;gt;getSecondNode()-&amp;gt;getCenter().x) * ((*lineIterator)-&amp;gt;getMidPoint().y);&#xA;            }&#xA;            else&#xA;            {&#xA;                additionTerms += ((*lineIterator)-&amp;gt;getFirstNode()-&amp;gt;getCenter().x) * ((*lineIterator)-&amp;gt;getSecondNode()-&amp;gt;getCenter().y);&#xA;                subtractionTerms += ((*lineIterator)-&amp;gt;getSecondNode()-&amp;gt;getCenter().x) * ((*lineIterator)-&amp;gt;getFirstNode()-&amp;gt;getCenter().y);&#xA;            }&#xA;        }&#xA;&#xA;        // Next, we need to run the shoe-lace algorithm to determine if the ordering is CCW or CW. If CW, then we will need to swap&#xA;        // the start node and end node of all of the edges in order to ensure the polygon edge is in CCW&#xA;        double shoelaceResult = additionTerms - subtractionTerms;&#xA;&#xA;        if(shoelaceResult &amp;lt; 0)&#xA;            reverseWindingResult = true;&#xA;&#xA;        // Now we can perform the winding number algorithm&#xA;        for(auto lineIterator = p_closedPath.begin(); lineIterator != p_closedPath.end(); lineIterator++)&#xA;        {&#xA;&#xA;            if((*lineIterator)-&amp;gt;isArc() &amp;amp;&amp;amp; (*lineIterator)-&amp;gt;getSwappedState())&#xA;                (*lineIterator)-&amp;gt;swap();&#xA;&#xA;            double isLeftResult = (*lineIterator)-&amp;gt;isLeft(point);&#xA;&#xA;            if(!(*lineIterator)-&amp;gt;isArc())&#xA;            {&#xA;                if(reverseWindingResult)&#xA;                    isLeftResult *= -1;&#xA;&#xA;                if((*lineIterator)-&amp;gt;getFirstNode()-&amp;gt;getCenterYCoordinate() &amp;lt;= point.y)&#xA;                {&#xA;                    if((*lineIterator)-&amp;gt;getSecondNode()-&amp;gt;getCenterYCoordinate() &amp;gt; point.y)&#xA;                    {&#xA;                        if(isLeftResult &amp;gt; 0)&#xA;                            windingNumber++;&#xA;                        else if(isLeftResult &amp;lt; 0)&#xA;                            windingNumber--;&#xA;                    }&#xA;                }&#xA;                else if((*lineIterator)-&amp;gt;getSecondNode()-&amp;gt;getCenterYCoordinate() &amp;lt;= point.y)&#xA;                {&#xA;                    if(isLeftResult &amp;lt; 0)&#xA;                        windingNumber--;&#xA;                    else if(isLeftResult &amp;gt; 0)&#xA;                        windingNumber++;&#xA;                }&#xA;&#xA;            }&#xA;            else&#xA;            {&#xA;                if((*lineIterator)-&amp;gt;getSwappedState())&#xA;                {&#xA;                    isLeftResult *= -1;&#xA;                    // For arcs, we need to preserve the orientation of the first and second&#xA;                    // node for drawing purposes. Lines do not matter as much for drawing but arcs,&#xA;                    // it matters&#xA;                    (*lineIterator)-&amp;gt;swap();&#xA;                }&#xA;&#xA;                if(isLeftResult &amp;gt; 0)&#xA;                    windingNumber++;&#xA;                else&#xA;                    windingNumber--;&#xA;            }&#xA;        }&#xA;&#xA;        if(windingNumber &amp;gt; 0)&#xA;            return true;&#xA;        else&#xA;            return false;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;IsLeft function implantation (code begins on line 896): &lt;a href=&quot;https://github.com/philm001/Omni-FEM/blob/master/Include/UI/geometryShapes.h&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/philm001/Omni-FEM/blob/master/Include/UI/geometryShapes.h&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double isLeft(wxRealPoint point)&#xA;    {&#xA;        if(!p_isArc)&#xA;        {&#xA;            return ((_secondNode-&amp;gt;getCenterXCoordinate() - _firstNode-&amp;gt;getCenterXCoordinate()) * (point.y - _firstNode-&amp;gt;getCenterYCoordinate()) &#xA;                        - (point.x - _firstNode-&amp;gt;getCenterXCoordinate()) * (_secondNode-&amp;gt;getCenterYCoordinate() - _firstNode-&amp;gt;getCenterYCoordinate())); &#xA;        }&#xA;        else&#xA;        {&#xA;            double result = 0;&#xA;&#xA;            if(isSameSign(crossProduct(point - _firstNode-&amp;gt;getCenter(), _secondNode-&amp;gt;getCenter() - _firstNode-&amp;gt;getCenter()), &#xA;                            crossProduct(this-&amp;gt;getCenter() - _firstNode-&amp;gt;getCenter(), _secondNode-&amp;gt;getCenter() - _firstNode-&amp;gt;getCenter())))&#xA;            {&#xA;                result = dotProduct(point - _firstNode-&amp;gt;getCenter(), _secondNode-&amp;gt;getCenter() - _firstNode-&amp;gt;getCenter()) / &#xA;                            dotProduct(_secondNode-&amp;gt;getCenter() - _firstNode-&amp;gt;getCenter(), _secondNode-&amp;gt;getCenter() - _firstNode-&amp;gt;getCenter());&#xA;&#xA;                if(result &amp;gt;= 0 &amp;amp;&amp;amp; result &amp;lt;= 1)&#xA;                    return 1.0;&#xA;                else&#xA;                    return -1.0;&#xA;            }&#xA;            else&#xA;            {&#xA;                result = dotProduct(point - this-&amp;gt;getCenter(), point - this-&amp;gt;getCenter());&#xA;&#xA;                if(result &amp;lt;= pow(_radius, 2))&#xA;                    return 1.0;&#xA;                else&#xA;                    return -1.0;&#xA;            }&#xA;&#xA;&#xA;        }&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18230" LastEditorUserId="20688" LastEditDate="2018-05-17T07:48:27.790" LastActivityDate="2018-05-17T07:48:27.790" Title="Expanding Winding Number algorithm to arcs" Tags="|computational-geometry|mesh|geometry|modeling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29204" PostTypeId="1" AcceptedAnswerId="29205" CreationDate="2018-03-31T17:58:30.750" Score="4" ViewCount="224" Body="&lt;p&gt;As a model for a nonlinear least-squares problem with a large linear part problem, consider&#xA;$$&#xA;\Delta u = 0 \quad\text{in } \Omega,\\&#xA;n\cdot\nabla u = 0 \quad \text{on } \Gamma,\\&#xA;(u(x_i) - u(y_i))^2 = 1 \quad \text{ for } i \in\{1,\dots,10\},&#xA;$$&#xA;i.e., on a domain $\Omega$ we're looking for a function $u$ that's somewhat smooth and for a bunch of given coordinate pairs, we want the function values to differ by 1. This system is overdetermined and will generally have no exact solution, but one could treat it with your favorite nonlinear least-squares method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, when discretizing the problem with, e.g., FEM and quite a fine mesh, the linear operator $\Delta_h$ will be &lt;em&gt;much&lt;/em&gt; larger than the 10 nonlinear equations at the end, and also one can solve linear systems with $\Delta_h$ really fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to exploit this fact?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2018-03-31T21:47:40.223" LastActivityDate="2018-04-02T00:17:38.410" Title="treating &quot;almost linear&quot; nonlinear least-squares problems" Tags="|pde|nonlinear-equations|least-squares|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29205" PostTypeId="2" ParentId="29204" CreationDate="2018-03-31T18:51:04.383" Score="6" Body="&lt;p&gt;If you change variables to optimize for the residual of the linear part, then the Hessian will be a low-rank update to the identity. Then L-BFGS would work very well. Specifically, your problem takes the form&#xA;$$&#xA;\min_x \frac{1}{2}\|Ax-b\|^2 + \frac{\mu}{2}\|g(x)\|^2&#xA;$$&#xA;where $Ax=b$ is the linear PDE and $g$ is the nonlinear part, and $\mu$ is a tradeoff parameter. Let $r:=Ax-b$.  Then the problem becomes&#xA;$$&#xA;\min_r \frac{1}{2}\|r\|^2 + \frac{\mu}{2}\|g(A^{-1}(r+b))\|^2.&#xA;$$&#xA;The gradient of the problem is&#xA;$$&#xA;\nabla f(r) = r + \mu A^{-T} G^T g(A^{-1}(r+b));&#xA;$$&#xA;the (Gauss-Newton) Hessian for this problem is&#xA;$$&#xA;H = I + \mu A^{-T}G^TG A^{-1},&#xA;$$&#xA;where $G$ is the Jacobian of $g$, which has rank $10$. Hence the Hessian is a rank-10 update to the identity. So, on this modified problem L-BFGS will probably converge in around 10 iterations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each iteration requires solving a system of the form $Ax=z$ to evaluate the objective function and $A^Tx=w$ to compute the gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Heres a simple python mockup. Seems to work.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import ones, zeros, arange, dot, sqrt&#xA;from numpy.random import choice, randn&#xA;from numpy.linalg import norm&#xA;from scipy.sparse import eye, diags, kron, coo_matrix, bmat&#xA;from scipy.sparse.linalg import factorized&#xA;from scipy.optimize import minimize&#xA;from matplotlib.pyplot import matshow, show&#xA;&#xA;# See: https://scicomp.stackexchange.com/a/29205/1502&#xA;&#xA;n=40&#xA;N=n*n&#xA;dx = 1./n&#xA;&#xA;m=10&#xA;inds = choice(N,2*m)&#xA;xx_inds = inds[:m]&#xA;yy_inds = inds[m:]&#xA;&#xA;mu = 1./(m*(dx**2)) # how strongly to force the differences being 1 vs forcing the PDE being solved&#xA;&#xA;# A: 2D Neumann Laplacian&#xA;A1D = diags([-ones(n - 1), 2*ones(n), -ones(n - 1)], [-1, 0, 1], shape=(n, n)).tocsr()&#xA;A1D[0,0] = 1&#xA;A1D[-1,-1] = 1&#xA;I=eye(n)&#xA;A0 = (kron(A1D, I) + kron(I, A1D)).tocsr()&#xA;A = bmat([[A0, ones([N,1])],[ones([1,N]), zeros(1)]])&#xA;solve_A = factorized(A)&#xA;solve_At = solve_A # symmetric&#xA;&#xA;B = (coo_matrix((ones(m),(arange(m),xx_inds)), shape=(m,N+1))&#xA;     - coo_matrix((ones(m),(arange(m),yy_inds)), shape=(m,N+1))).tocsr()&#xA;def g(u):&#xA;    return (B*u)**2 - 1.&#xA;&#xA;def G(u):&#xA;    return 2*diags(B*u,0)*B&#xA;&#xA;def objective(r):&#xA;    return 0.5*norm(r)**2 + mu*0.5*norm(g(solve_A(r)))**2&#xA;&#xA;def gradient(r):&#xA;    u = solve_A(r)&#xA;    return r + mu*solve_At(G(u).T * g(u))&#xA;&#xA;# Check gradient with finite differences&#xA;r0 = randn(N+1)&#xA;s = 1e-7&#xA;dr = randn(N+1)&#xA;r1 = r0 + s*dr&#xA;&#xA;j0 = objective(r0)&#xA;j1 = objective(r1)&#xA;dj_diff = (j1-j0)/s&#xA;dj = dot(gradient(r0),dr)&#xA;gradient_err = norm(dj - dj_diff)/norm(dj_diff)&#xA;print 's=', s, ', gradient_err=', gradient_err&#xA;&#xA;# Solve with L-BFGS&#xA;result = minimize(objective, ones(N+1), jac=gradient, method='L-BFGS-B', options={'maxiter':50})&#xA;r_optimal = result['x']&#xA;u_optimal = solve_A(r_optimal)&#xA;g_optimal = g(u_optimal)&#xA;num_iter = result['nit']&#xA;print 'num_iter=', num_iter&#xA;print 'norm(r_optimal*(dx**2))=', norm(r_optimal*(dx**2))&#xA;print 'norm(g_optimal)/m=', norm(g_optimal)/m&#xA;print 'u_optimal[xx_inds] - u_optimal[yy_inds]=', u_optimal[xx_inds] - u_optimal[yy_inds]&#xA;matshow(u_optimal[:N].reshape([n,n]))&#xA;show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit2:&lt;/strong&gt; Enforcing Neumann condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\{\phi\}_{i=1}^N$ be a set of finite element basis functions, let&#xA;$$A^0_{ij}:= \int_\Omega \nabla\phi_i \cdot \nabla \phi_j ~dx,$$&#xA;and let&#xA;$$\mathbb{1}_j:= \int_\Omega \phi_j ~dx.$$&#xA;Then at the discrete level solving the pure Neumann Laplacian problem with average value zero constraint is equivalent to solving the following saddle point system:&#xA;$$\underbrace{\begin{bmatrix}A^0 &amp;amp; \mathbb{1} \\ \mathbb{1}^T &amp;amp; 0\end{bmatrix}}_{A}\underbrace{\begin{bmatrix}u \\ \lambda\end{bmatrix}}_{x} = 0.$$&#xA;The matrix $A$ is invertible since $A^0$ is invertible on $\text{ker}(\mathbb{1}^T)$. For some background on invertibility of saddle point systems I recommend this: &lt;a href=&quot;https://arxiv.org/abs/1202.3330&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1202.3330&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have some specialized or black box solver that solves the problem&#xA;$$A^0 \widehat{u} = f, \quad\quad \text{and} \quad\quad \mathbb{1}^T \widehat{u}=c^0,$$&#xA;you can use it to solve problems of the form&#xA;$$\begin{bmatrix}A^0 &amp;amp; \mathbb{1} \\ \mathbb{1}^T &amp;amp; 0\end{bmatrix}\begin{bmatrix}u \\ \lambda\end{bmatrix} = \begin{bmatrix}f \\ c\end{bmatrix}$$&#xA;by setting $u = \widehat{u}+(c-c^0)$ and $\lambda = (A^0 u - f)_j/\mathbb{1}_j$, any component $j$ will do; Since the first row block equation is satisfied, the quantity $(A^0 u - f)_j/\mathbb{1}_j$ will be the same for all $j$.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2018-04-01T18:45:03.710" LastActivityDate="2018-04-01T18:45:03.710" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="29206" PostTypeId="2" ParentId="23286" CreationDate="2018-04-01T03:06:14.263" Score="5" Body="&lt;p&gt;I will try to give my thought on the first question regarding &lt;em&gt;fast $3\times 3$ inverse&lt;/em&gt;. Consider&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;A=\left[&#xA;\begin{array}{ccc}&#xA;a &amp;amp; d &amp;amp; g\\&#xA;b &amp;amp; e &amp;amp; h\\&#xA;c &amp;amp; f &amp;amp; i&#xA;\end{array}\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the matrices are small and very general (do not feature any known structure, zeroes, relative scales of the elements), I think it would be impossible to give an algorithm for arbitrary scale (without $1/\det(A)$) inverse that is faster than 18 fused flops, as each out of 9 elements requires 2 fused flops, and all products are unique, provided no prior info on $A$'s entries $a,\ldots,i$.&#xA;$$&#xA;A^{-1}\det(A)=\text{adj}(A)=&#xA;\left[\begin{array}{ccc}&#xA;ei-fh &amp;amp; di-fg &amp;amp; ge-dh\\&#xA;bi-ch &amp;amp; ai-cg &amp;amp; ah-bg\\&#xA;ce-bf &amp;amp; af-cd &amp;amp; ae-bd&#xA;\end{array}\right]&#xA;$$&#xA;Here, $\text{adj}(A)$ denotes the adjugate (transpose of cofactors), which essentially is an inverse with &quot;arbitrary scale&quot; (provided the inverse exists).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, some calculation can be reused for calculation of the $\det(A)$.&#xA;If I expand it over the first column (5 more choices are there):&#xA;$$&#xA;\begin{aligned}&#xA;\det(A)&amp;amp;=a(ei-fh)+b(fg-di)+c(dh-ge)\\&#xA;&amp;amp;=a\underbrace{(ei-fh)}_*-b\underbrace{(di-fg)}_*-c\underbrace{(ge-dh)}_*&#xA;\end{aligned}&#xA;$$&#xA;Notice, that (*) has been already computed during evaluation of $\text{adj}(A)$. So, the reciprocal of determinant can be computed in 4 additional fused flops (if $1/\det(A)$ reciprocal is considered as 1 flop). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, each 9 elements of the $\text{adj}(A)$ should be scaled by already obtained reciprocal of the determinant, adding another 9 fused flops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Calculate $\text{adj}(A)$ in 18 fused flops&lt;/li&gt;&#xA;&lt;li&gt;Calculate $\det(A)$ in 3 fused flops using entries of already computed $\text{adj}(A)$&lt;/li&gt;&#xA;&lt;li&gt;Find $\frac{1}{\det(A)}$ (assuming 1 flop).&lt;/li&gt;&#xA;&lt;li&gt;Scale each elelemt of already computed $\text{adj}(A)$ by $\frac{1}{\det(A)}$ in another 9 fused flops.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Resulting in 18+3+1+9=&lt;strong&gt;31 fused flops&lt;/strong&gt;. You did not describe your way of computing the determinant, but I guess 1 additional flop can be saved. Or it can be used to perform the check $|\det(A)|&amp;gt;\epsilon$ in step 3, where $\epsilon$ is the tolerance for &lt;em&gt;degenerate&lt;/em&gt; (not invertible) case, resulting in &lt;strong&gt;32 fused flops&lt;/strong&gt; (assuming &lt;code&gt;if&lt;/code&gt; is 1 flop).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think there is a faster way to compute the inverse of a $3\times 3$ general matrix as all remaining calculations are unique. Using Cayley-Hamilton should not help from the speed perspective, as in general, it will require calculation of $A^2$ for a $3\times 3$ matrix besides some other operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;this answer does not deal with the  numerical stability&lt;/li&gt;&#xA;&lt;li&gt;the possible potential for vectorization and optimizing memory access pattern is also not discussed&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-04-03T10:40:26.467" LastActivityDate="2018-04-03T10:40:26.467" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29207" PostTypeId="2" ParentId="28102" CreationDate="2018-04-01T06:42:32.653" Score="4" Body="&lt;p&gt;As noted in the comments by Kirill, the y-axes of the two plots are very different. And if they are rescaled accordingly, the boxes will certainly look very similar, if not identical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, it is very reasonable to conclude that the raw simulation result in &lt;em&gt;data.dat&lt;/em&gt; coming from your C++ code is correct, no matter what seed has been used for random-number generation. The suspect is on the exponential fit from &lt;code&gt;scipy&lt;/code&gt; that was unable to fit the function correctly and possibly had a bug in the implementation. That incorrect fit causes the orange line to skew the appearance of the plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was very curious about this behavior, but &lt;strong&gt;was totally unable to reproduce it&lt;/strong&gt; on my machine. So I would conclude with a high level of confidence that this bug in non-existent in my &lt;code&gt;scipy+numpy+python&lt;/code&gt; toolchain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My configuration (I did about 150 runs with different seeds visually checking the resultant PDFs):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GCC 6.4 (MacPorts gcc6 6.4.0_0)&lt;/li&gt;&#xA;&lt;li&gt;compiled with &lt;code&gt;-O3&lt;/code&gt;: &lt;code&gt;/opt/local/bin/g++-mp-6 -openmp -O3 monte.cpp -lgomp&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Python 2.7.10, native from Mac OS 10.13.3&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;numpy&lt;/code&gt;: 1.8.0rc1&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;scipy&lt;/code&gt;: 0.13.0b1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-04-01T06:42:32.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29208" PostTypeId="2" ParentId="8886" CreationDate="2018-04-01T13:58:05.960" Score="-1" Body="&lt;p&gt;We are currently working on a Massive Open Online Course, &quot;LAFF-On High-Performance Computing&quot; that uses dgemm as the example that leads one through different levels of parallelization: instruction level, OpenMP, MPI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not a reference implementation for the BLAS, but it is a reference for how to code the BLAS (for performance).  To be kept informed, visit &lt;a href=&quot;http://www.ulaff.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.ulaff.net&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="320" LastActivityDate="2018-04-01T13:58:05.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29209" PostTypeId="1" CreationDate="2018-04-01T17:42:25.847" Score="2" ViewCount="59" Body="&lt;p&gt;I'm looking for the best way of implementing the following algorithm: consider the set of all polynomials with a high degree (say, degree 30) whose coefficients ranges from a given set of values (say, {-1,0,1}). To any polynomial of this set, I want to apply some test (say, to verify if all its roots are real). If the polynomial pass in the test, I want to print it and its roots (numerically evaluated). If it does not pass, do nothing. (actually, my test is somewhat more complicated than this, but this is enough to clarify my point).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example above, I do not need to evaluate the roots of each polynomial in the set to verify if its roots are all real or not, since I can use instead, for instance, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sturm%27s_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sturm theorem&lt;/a&gt;, or some numerical algorithm for isolating the roots intervals, and also other intermediate filter-tests as Descartes's rule of sign and so on. It is only when the polynomial pass on all these tests—so that I'm sure that all its roots are real—that I want to print the polynomial and its roots. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is the following: since the degree is very high, there are too many polynomials to test for. This takes a high computational time even to run the list only. Therefore, I want to know what is the best language to implement the code and what is the best way of generating the list of polynomials for run the tests. Currently, I implemented the code on Maple, through a nested loop structure (one for loop for each polynomial coefficient), but I believe that there is a smarter way to do that. Maybe some way of parallelizing the loops would be desirable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not master any programing language, but I can learn fast. I'm comfortable with Maple, Mathematica and I'm learning SageMath. I know a little of Python as well and I think that Cython is a good idea to implement the code, but I'm not sure … Any help will be very welcome and I thank you all in advance.&lt;/p&gt;&#xA;" OwnerUserId="27351" LastEditorUserId="9667" LastEditDate="2018-04-01T18:26:05.377" LastActivityDate="2018-04-01T19:12:14.313" Title="Test on a set of high degree polynomials whose coefficients in {-1,0,1}" Tags="|numerical-analysis|parallel-computing|polynomials|roots|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29210" PostTypeId="2" ParentId="29083" CreationDate="2018-04-01T18:12:59.153" Score="1" Body="&lt;p&gt;The answer to your question is benchmarking. Both &lt;a href=&quot;http://arma.sourceforge.net/speed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Armadillo&lt;/a&gt; and &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=Benchmark&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigen&lt;/a&gt; provide some benchmarks in their documentation. The only problem is that they don't compare to the same libraries, but you can still get an idea. Maybe there are more comprehensive benchmarks elsewhere...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, the overhead of wrapping LAPACK in a C++ library will be much smaller compared to Python/Numpy.&lt;/p&gt;&#xA;" OwnerUserId="24110" LastActivityDate="2018-04-01T18:12:59.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29211" PostTypeId="2" ParentId="29209" CreationDate="2018-04-01T18:30:45.387" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Backtracking&quot; rel=&quot;nofollow noreferrer&quot;&gt;backtracking&lt;/a&gt; to generate your polynomials. This can be parallelized in pretty much any language you choose. You may also want to call the Sturm sequence finder in &lt;a href=&quot;http://www.flintlib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the FLINT library&lt;/a&gt; from your script for speed.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2018-04-01T19:12:14.313" LastActivityDate="2018-04-01T19:12:14.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29212" PostTypeId="1" CreationDate="2018-04-01T19:47:29.553" Score="3" ViewCount="1703" Body="&lt;p&gt;I'm having trouble figuring out how to implement boundary conditions for this problem:&#xA;\begin{align}&#xA;\frac{\partial n}{\partial t} &amp;amp;= D_n\nabla^2n - \nabla\cdot\left(\frac{\chi}{1+\alpha c}n\nabla c\right) - \nabla\cdot (pn\nabla f) \\&#xA;\frac{\partial f}{\partial t} &amp;amp;= \beta n-\gamma nf \\&#xA;\frac{\partial c}{\partial t} &amp;amp;= -\eta nc&#xA;\end{align}&#xA;Where the physical domain is a simple square, $\Omega = [0,100]\times[0,100]$ and the time domain is $T = [0,t]$. The paper I'm working from says the no-flux boundary condition is:&#xA;$$\left[D_n\nabla n - \left(\frac{\chi}{1+\alpha c}n\nabla c\right) - (pn\nabla f)\right]\cdot \hat{N} = 0$$&#xA;Where $\hat{N}$ is the outward unit normal vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I've done:&lt;/strong&gt;&#xA;I've set up the central difference versions of the first 3 equations (&lt;a href=&quot;https://www.overleaf.com/15244940hbpymwcpgkbk#/57668374/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Overleaf PDF&quot;&gt;Overleaf PDF here&lt;/a&gt;), but I'm not sure how to set up $\frac{\partial n}{\partial t}$ on the boundary using the BC above. I've tried converting the BC into a finite difference version and using it to cancel some terms, but I'm not sure if this is the right way to go about this. Any advice is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="26334" LastActivityDate="2018-11-29T06:01:20.233" Title="Implementing no-flux boundary condition reaction-diffusion PDE" Tags="|pde|numerical-analysis|finite-difference|boundary-conditions|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29214" PostTypeId="2" ParentId="29212" CreationDate="2018-04-01T23:58:18.917" Score="0" Body="&lt;p&gt;The equations for $f$ and $c$ do not have any spatial derivatives, and consequently they do not require (and cannot have, in fact) any boundary conditions. Thus, only $n$ has boundary conditions, and these are in your case of Neumann type. It is convenient to rewrite the equation you have as&#xA;$$&#xA;  \left(\frac{\chi}{1+\alpha c}n\nabla c\right)\cdot\hat{N}&#xA;  =&#xA;  \left[D_n\nabla n - (pn\nabla f)\right]\cdot \hat{N} &#xA;$$&#xA;to make clear what condition has to be satisfied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a finite element context, the term on the left hand side of this appears naturally as a result of integration by parts, and dealing with the boundary condition is therefore a trivial matter of substituting the term by what you have here from the boundary condition. I am not enough of a finite difference person to say how best to do this in that context.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-01T23:58:18.917" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29215" PostTypeId="2" ParentId="29204" CreationDate="2018-04-02T00:17:38.410" Score="4" Body="&lt;p&gt;If you want to solve such problems with least squares, you need to first address two things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You are interested in a continuous problem, not one particular discretization of it. Your statement &quot;quite a fine mesh, the linear operator $\Delta$_h will be much larger than the 10 nonlinear equations at the end&quot; shows that you are somehow aware of the problem, but you are not drawing the right conclusion. The correct conclusion is that any formulation for the least squares minimization must be posed in a way that is &lt;em&gt;independent&lt;/em&gt; of the discretization. In particular, this will mean that you need to pose it in a functional analytic, rather than a finite dimensional setting. An example would be to describe the residual $r=\Delta u$ in term of the $H^{-1}$ norm; you'd then have to find a way to compute this norm for your finite dimensional approximation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In this context, it is important to understand that the solution of the Laplace equation are in $H^1$, and consequently evaluating the solution at individual points is not a well-defined operation. This means that your least-squares problem is not well posed: it is conceivable that you can find a sequence of functions that all satisfy the Laplace equation and for which the constraint becomes smaller and smaller, but the limit is not in $H^1$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-02T00:17:38.410" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29216" PostTypeId="2" ParentId="29203" CreationDate="2018-04-02T00:25:49.563" Score="1" Body="&lt;p&gt;Enclose each one of your arcs by a box that connects the two end points on one diagonal and choose the other two points to make things into a box. Then replace the original polygon-with-arcs $P$ by the polygon $P'$ that contains all of the straight line segments plus, say, the inside two sides of all of the boxes. (Or the outsides, or choose randomly.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, given a point to test, you can first figure out whether the point is inside one of the boxes. If it is not, then if it is inside (outside) $P'$ (easily testable) then it must also be inside (outside) $P$. On the other hand, if the point is in one of the boxes that enclose the arcs, then you'll have to test whether it is on one side of the arc or the other -- but that's easy because an arc has such a simple structure.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-02T00:25:49.563" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="29217" PostTypeId="1" CreationDate="2018-04-02T00:47:03.230" Score="1" ViewCount="461" Body="&lt;p&gt;I am trying to solve the following diffusion equation problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial f}{\partial t}=\frac{\partial (D\frac{\partial f}{\partial x})}{\partial x}+S$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D=1+x^{2}+\sin(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(x,0)=1  ,  f(0,t)=f(1,t)=1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S $ is choosen adequately in order to have $f_{ana} = 1+\sin(\pi x)\sin(\pi t)$ as the analytical solution of the equation to be compared with the numerical one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation was discretized using a finite difference/ Central method in space (2nd order in space) and Euler implicit in time (1st order in time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The global error behavior should follow asymptotically $\dfrac{A}{dx^{2}}+\dfrac{B}{dt}$ with $A$ and $B$ that do not depend on $dt$ and $dx$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now after implementing the resolution algorithm on Python, and constructing the error plot variation in terms of $dx$  (2nd norm and $\infty $ norm ) for a fixed $dt$, I was expecting a behavior following $C+\dfrac{A}{dx^{2}}$, with a &quot;smooth&quot; decrease in the convergence rate as C would become more predominant for smaller $dx$ till reaching the constant. What I got instead (and what I don't understand) is a small increase in the convergence rate, before recovering to the constant value (&quot;reversed bell&quot; behavior).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it normal to have an &quot;acceleration&quot; of the convergence rate in this case ?&#xA;Is it a problem related to rounding effect of the very small space error ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the ready-to-use Python code :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import scipy as sci&#xA;from scipy.sparse.linalg.dsolve import spsolve&#xA;import scipy.sparse as sp&#xA;import sympy as sym&#xA;&#xA;#Symbolic calculation&#xA;x, t= sym.symbols('x t')&#xA;Dxx_1=1+x**2+sym.sin(x)&#xA;&#xA;fi_1=1+sym.sin(sym.pi*x)*sym.sin(sym.pi*t) #Analytical solution&#xA;Soi_1=-sym.diff(Dxx_1*sym.diff(fi_1,x),x)+sym.diff(fi_1,t)#Source&#xA;&#xA;Dxx_2=sym.lambdify((x),Dxx_1,&quot;numpy&quot;)&#xA;&#xA;#Conversion from symbolic to function&#xA;Soi=sym.lambdify((x,t),Soi_1,&quot;numpy&quot;)&#xA;fi=sym.lambdify((x,t),fi_1,&quot;numpy&quot;)    &#xA;&#xA;&#xA;#Lists that will contain the trucation errors for plot&#xA;Err00=[]&#xA;Err2=[]&#xA;&#xA;#list of the number of points in the domain &#xA;N=np.array([10,20,40,60,80,100,120,140,160,180,200,320,640])&#xA;&#xA;#Loop on Nx&#xA;for Nx in N:&#xA;&#xA;    #Space domain definition&#xA;    lx=1.0&#xA;    ox=0.0&#xA;    dx=(lx-ox)/(Nx)&#xA;&#xA;    x1=np.linspace(ox+dx,lx-dx,Nx-1)#Points of the domain where f is unknown&#xA;    x2=np.linspace(ox,lx,Nx+1)#All points of the domain &#xA;&#xA;    #Diffusion coefficient&#xA;    Dxx=Dxx_2(x2)&#xA;&#xA;    #Initialisation&#xA;    f=fi(x1,0)&#xA;&#xA;    #Time grid&#xA;    t1=0.5&#xA;    dt=0.0001&#xA;    Nt=int(t1/dt)&#xA;&#xA;    #Implicit matrix definition - Sparse method&#xA;    diag0= -(2*Dxx[1:-1]/(dx**2))&#xA;    diag1= ((Dxx[2:-1]-Dxx[:-3])/(4*dx**2))+((Dxx[1:-2])/(dx**2))&#xA;    diag_1= (-((Dxx[3:]-Dxx[1:-2])/(4*dx**2))+((Dxx[2:-1])/(dx**2)))&#xA;    data = [-dt*np.append(diag_1,[0]),1-dt*diag0,-dt*np.append([0],diag1)]     # Diagonal terms&#xA;    offsets = np.array([-1, 0, 1])                     # Their positions&#xA;    J = sp.dia_matrix((data, offsets), shape=(Nx-1,Nx-1))&#xA;&#xA;    #Iteration and resolution&#xA;    for k in range(1,Nt+1,1):&#xA;        #Boundary condition vector&#xA;        S1=np.zeros(Nx-1)&#xA;        S1[0]=fi(0.0,k*dt)*(-(Dxx[2]-Dxx[0])/(4*dx**2)+(Dxx[1])/(dx**2))&#xA;        S1[-1]=fi(1.0,k*dt)*((Dxx[-1]-Dxx[-3])/(4*dx**2)+(Dxx[-2])/(dx**2))&#xA;        #2nd member vector&#xA;        S=Soi(x1,k*dt)&#xA;        f=spsolve(J,f+dt*S+dt*S1)&#xA;        sol_ana=fi(x1,k*dt)&#xA;        print('iteration '+str(k)+' - t = '+str(k*dt)+' - Implicit - Nx ='+str(Nx))&#xA;&#xA;&#xA;    print('Nx = '+str(Nx))&#xA;    Err2.append(100*np.linalg.norm(f-sol_ana)/np.linalg.norm(sol_ana))&#xA;    print('Err2 = ' + str(Err2[-1]))&#xA;    Err00.append(100*(abs(f-sol_ana).max()/abs(sol_ana).max()))&#xA;    print('Err00 = '+ str(Err00[-1]))&#xA;&#xA;&#xA;plt.figure()&#xA;plt.plot(N,Err00,'o',N,Err2,'o',N,N[0]**2*Err00[0]*N**float(-2),N,N[0]*Err00[0]*N**float(-1))&#xA;plt.title(' Err = f(Nx) - 1D - DF - Dirichlet  - Impl - dt='+str(dt)+' - t= '+str(t1))&#xA;&#xA;plt.xscale('log')&#xA;plt.yscale('log')&#xA;plt.legend(['$Err_{\infty}$','$Err_{2}$','$\dfrac{1}{Nx^{2}}$','$\dfrac{1}{Nx}$'])&#xA;plt.xlabel('Nx')&#xA;plt.ylabel('Err')&#xA;plt.annotate('$f_{ana}='+sym.latex(fi_1)+'$', xy=(0.4, 0.14), xycoords='axes fraction')&#xA;plt.annotate('$D_{xx}='+sym.latex(Dxx_1)+'$', xy=(0.4, 0.1), xycoords='axes fraction')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9ukP7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9ukP7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27315" LastEditorUserId="27315" LastEditDate="2018-04-02T14:32:24.080" LastActivityDate="2018-04-02T14:32:24.080" Title="Global truncation error behavior at fixed time step" Tags="|numerical-analysis|finite-difference|python|discretization|error-estimation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29218" PostTypeId="2" ParentId="29212" CreationDate="2018-04-02T08:09:23.057" Score="1" Body="&lt;p&gt;A finite volume viewpoint would be very covenient for your problem. It is not too different from finite difference. Sticking to just 1-d, your first cell would be the interval $[0,h]$ and you store the solution at the center of this cell, say $n_0$. Then the semi-discrete scheme for this cell would be&#xA;$$&#xA;\frac{d n_0}{dt} + \frac{F_{1/2} - F_{-1/2}}{h} + ... = 0&#xA;$$&#xA;From your boundary condition $F_{-1/2} = 0$. The other fluxes can be computed by a finite difference, e.g.,&#xA;$$&#xA;F_{1/2} = D_n \frac{n_1 - n_0}{h} - \frac{\chi n_{1/2}}{1+\alpha c_{1/2}} \frac{c_1 - c_0}{h} - p n_{1/2} \frac{f_1 - f_0}{h}&#xA;$$&#xA;where $n_{1/2}=(n_0 + n_1)/2$, etc.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2018-04-03T02:49:31.700" LastActivityDate="2018-04-03T02:49:31.700" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29220" PostTypeId="1" CreationDate="2018-04-02T12:09:44.593" Score="1" ViewCount="286" Body="&lt;p&gt;I am trying to solve this diffusion equation :&#xA;$\dfrac{\partial D\dfrac{\partial f}{\partial x}}{\partial x}+S = \dfrac{\partial f}{\partial t}$ ($D$ is not constant and varies according to $x$) with the following BC: $f(x,0)=1 , f(0,t)=0, \dfrac{\partial f}{\partial x}(1,t)=0$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a central finite difference scheme (2nd order) for space and Euler explicit for time (1st order). &#xA;The discretized $[0..1]$ domain contains the $x_{i}$ points, $i \in [0..N]$ &#xA; ($x_{0}=0 , x_{N}=1$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implementation of Dirichlet condition at $x=0$ is simple.&#xA;For the Neumann boundary condition, I saw a lot of references that treated simple cases, adequate for constant diffusion coefficients and stationnary cases, like the ghost point, that I can't consider in my case(Diffusion coefficient can't be evaluated out of the domain, ghost point envolves evaluation the equation on a point that is not inside the interior domain... ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am trying something at my own and I want confirmation or correction if I omitted an important aspect in my work:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discretized equation evaluated at $x_{N-1}$ involves the value of $f_{N}$. If we had Diricihlet condition at $x_{N}$, the problem would have been solved directly. But in our case, $f_{N}$ is unknown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I did is that I treated the Neumann BC  with a backward scheme (at the 2nd order to preserve the general order of the whole scheme) in order to have an expression of $f^{k}_{N}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\dfrac{\partial f}{\partial x}(x_{N},k\Delta t)=\dfrac{-3f^{k}_{N}+4f^{k}_{N-1}-f^{k}_{N-2}}{2\Delta x}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f^{k}_{N}=4/3 f^{k}_{N-1}-1/3 f^{k}_{N-2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then I injected the new expression of $f^{k}_{N}$ in the discretized equation at $x_{N-1}$. Meaning that I did not add an equation to the problem as I would do for a stationnary problem but I injected the expression of $f^{k}_{N}$ inside the last equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the unknown vector $f^{k}$ in a case of matrix writing would only contain $f^{k}_{i}, i\in [1..N-1]$ (like in a full Dirichlet situation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it correct ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For information, the matrix form of the discretized problem is :&#xA;$f^{k+1}=Af^{k}+S+B$, with $B$ the vector containing the B.C , and $S$ the 2nd member vector&lt;/p&gt;&#xA;" OwnerUserId="27315" LastEditorUserId="27315" LastEditDate="2018-04-02T16:44:56.240" LastActivityDate="2018-04-02T16:44:56.240" Title="Neumann boundary condition FD implementation for instationnary diffusion equation" Tags="|finite-difference|boundary-conditions|diffusion|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="29221" PostTypeId="1" AcceptedAnswerId="29223" CreationDate="2018-04-02T16:14:11.057" Score="0" ViewCount="119" Body="&lt;p&gt;&lt;strong&gt;TLDR:&lt;/strong&gt; What references and pre-requisites are necessary for a Computer Science Undergraduate to get ready for a Masters and Career in Scientific Computation / Computational &lt;strong&gt;X&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a Computer Science Undergraduate, Programming and Mathematics are not a problem but there is definitely a void left in Basic Sciences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking an example of Physics, Computer Science and Engineering Syllabus only contain Basic Courses in Physics - Optics and Material Science. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keeping that in mind as a Computational Scientist what would you recommend a Computer Science major to start referring to, to cover the lost ground.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References can include :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Massively Open Online Courses&lt;/li&gt;&#xA;&lt;li&gt;Textbooks&lt;/li&gt;&#xA;&lt;li&gt;Topics to be covered both in Computing (e.g: HPC) or in Basic Sciences (FEM)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27263" LastActivityDate="2018-04-02T18:50:34.187" Title="References in Scientific Computation for a Computer Science Undergrad?" Tags="|reference-request|education|" AnswerCount="1" CommentCount="0" ClosedDate="2018-04-02T23:48:20.653" ContentLicense="CC BY-SA 3.0" />
  <row Id="29222" PostTypeId="2" ParentId="23131" CreationDate="2018-04-02T18:03:18.660" Score="2" Body="&lt;p&gt;Currently there is a GMSH API in the works:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gitlab.onelab.info/gmsh/gmsh/tree/master/api&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gitlab.onelab.info/gmsh/gmsh/tree/master/api&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there are rumors that there will be a fully documented API by version 4.0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, there is no documentation on the functions for GMSH (except for what is in the source all ready). But, they are planning on creating a documented version by GMSH 4.0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://onelab.info/pipermail/gmsh/2017/011767.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://onelab.info/pipermail/gmsh/2017/011767.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2018-04-02T18:03:18.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29223" PostTypeId="2" ParentId="29221" CreationDate="2018-04-02T18:50:34.187" Score="1" Body="&lt;p&gt;I would have a look at this course here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.coursera.org/learn/finite-element-method&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.coursera.org/learn/finite-element-method&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took it a few years back and it was a very nice intro into scientific computing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also brush up on your PDE skills and Computational Geometry. I started a wiki like answer on this page asking for resources on Meshing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/28388/resources-on-mesh-generation-for-finite-element-methods&quot;&gt;Resources on mesh generation for finite element methods&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Creating meshes is a core skill to have in scientific computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And last but not least, I highly recommend looking into DealII. It has extensive code documentation, video tutorials on programming with DealII for finite element simulations and written tutorials. It is quite literally a treasure trove for this subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dealii.org/8.5.0/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dealii.org/8.5.0/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dealii.org/8.5.0/doxygen/deal.II/Tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dealii.org/8.5.0/doxygen/deal.II/Tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should keep you busy for awhile and also prepare you for what is ahead.&lt;/p&gt;&#xA;" OwnerUserId="18230" LastActivityDate="2018-04-02T18:50:34.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29225" PostTypeId="2" ParentId="29199" CreationDate="2018-04-03T01:17:41.230" Score="1" Body="&lt;p&gt;Whilst I agree with the general consensus of responders that this is not the best way to solve the minimisation problem in the question, I have now resolved the challenge and can answer my own question to share the way one might overcome similar issues in using penalty methods to resolve optimisation problems in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key mathematical issue is indeed the non-differentiability of the penalty functions; it seems that best practice is to use a polynomial of the same order as the objective function; in this way you ensure the behaviour of the penalty is complementary to your objective function. Considering Taylor series, it seems conceptually clear that you should be able to form a sum of these polynomials, of different coefficients, to well approximate almost any penalty you might desire.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key programming issue (which will cause the SciPy error in the question) comes from providing incorrect Jacobian (and/or Hessian) functions to scipy.optimize.minimize(). The addition of the penalty function makes the calculation of the gradient vector and Hessian matrix considerably more difficult, and I had to calculate these by hand. Fortunately SciPy provides a function to test your gradient function: &lt;code&gt;check_grad(F, dF, x_k))&lt;/code&gt;, which compares the norm of your gradient function at x_k against an inbuilt finite difference approximation over a small region around x_k. Typically if this returns something $&amp;lt;10^{-4}$ then your function is likely correct (well, correct enough). This doesn't hold for the Hessian matrix, so more careful calculation is required. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for this question, I swapped the absolute, linear terms in the both penalty functions above with quadratic approximations: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def objective(x):&#xA;    Q = DF.cov()                                  # Covariance matrix&#xA;    var = np.matmul(W.transpose(),np.matmul(Q,x)) # Variance vector&#xA;&#xA;    # Penalty Function method&#xA;    penalty1 =  (np.sum(x)-1)**2                    # Large for sum(x) &amp;lt;&amp;gt; 1&#xA;    penalty2 =  100*(R_min - np.dot(Mus, x))**2     # Large for returns &amp;lt;&amp;gt; minR&#xA;&#xA;    return var + penalty1 + penalty2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are therefore twice differentiable and, after some painstaking matrix and vector derivative calculations, yielded gradient and Hessian functions as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def der_objective(x):&#xA;    Q = DF.cov()&#xA;&#xA;    der = 2 * np.matmul(Q,x)&#xA;&#xA;    penalty1_der = np.array([2*np.sum(x)-2 for i,xi in enumerate(x)])&#xA;    penalty2_der = 100*np.array([ 2*Mus[i]*(np.dot(Mus,x) - R_min) for i,xi in enumerate(x)])&#xA;&#xA;    return der + penalty1_der + penalty2_der&#xA;&#xA;&#xA;def hess_objective(x):&#xA;    Q = DF.cov()&#xA;    hess = 2 * Q.values&#xA;&#xA;    # Assemble Hessian terms for penalty1 function&#xA;    penalty1_hess = 2*np.ones_like(hess)&#xA;&#xA;    # Assemble Hessian terms for penalty2 function&#xA;    bcast = np.broadcast(Mus.reshape(len(Mus),1),Mus.reshape(1,len(Mus)))&#xA;    penalty2_hess = np.empty(bcast.shape)&#xA;    penalty2_hess.flat = 100*np.array([2*a*b for (a,b) in bcast])&#xA;&#xA;    return hess + penalty1_hess + penalty2_hess&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are then typically passed to the simple interface provided by SciPy to select a minimisation routine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result = scipy.optimize.minimize(fun=objective, x0=x_k, method='BFGS',&#xA;         jac=der_objective, hess=hess_objective, &#xA;         options={'tol':1e-6,'maxiter':1e3})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="25837" LastEditorUserId="25837" LastEditDate="2018-04-03T01:23:11.137" LastActivityDate="2018-04-03T01:23:11.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29226" PostTypeId="1" CreationDate="2018-04-03T01:32:55.240" Score="1" ViewCount="209" Body="&lt;p&gt;I have a function $F(x)$ which drops exponentially (like differential QCD cross section vs. Invariant mass). I want to perform Monte-Carlo integration. The problem is that only small $x$'s which have large $F(x)$ are accounted. I mean if the range of $x$ is $[1, 8]$ TeV, the maximum value of $F$ is $F(1)$ and the minimum value of $F$ is $F(8)$ which differ by 8 orders of magnitude. Thus, only random generated $x$'s which are close to 1 are accepted. How can I generate large $x$? Thanks&lt;/p&gt;&#xA;" OwnerUserId="27358" LastEditorUserId="20688" LastEditDate="2018-04-03T09:49:39.037" LastActivityDate="2018-04-05T21:09:43.477" Title="Weighted Monte Carlo Integration" Tags="|monte-carlo|integration|random-number-generation|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="29227" PostTypeId="1" CreationDate="2018-04-03T02:46:22.767" Score="3" ViewCount="238" Body="&lt;p&gt;I've tried my best to look through the relevant suggested similar questions when posting this, and hopefully this contains enough new material to not be considered a duplicate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently trying to draw up a semiconductor drift-diffusion simulation framework, and the work that I'm supposed to be working off of, even though it's called 'finite element' in the section heading, is control-volume finite element, or vertex-centered finite volume (depending on which field you come from, one may be more common notation than the other).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As best as I understand it right now, the finite volume method preserve fluxes by construction, and is therefore preferred for implementation of conservation laws. It also allows the Scharfetter-Gummel Finite Box method to be implemented for current densities across the finite volume elements. The SG-FB method is a generalization of the original SG formulation, where the finite differences between carrier densities (the $\nabla n$ term below) are calculated using a special exponential smoothing scheme. This is done because $n$ can often change discontinuously between two points in space, and the SG method is a stable difference scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would like to implement this in finite elements, not volume, almost entirely because of the libraries available for FEM (deal.II, libMesh, FEniCS, ...). The relevant current density equations for electrons are below (these, combined with an identical equation for holes, and a nonlinear Poisson equation, make up the system)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial n}{\partial t} = \frac{1}{q} \nabla \cdot \mathbf{J_n} + G_n - R_n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\mathbf{J_n} = qD_n\nabla n - q\mu_ nn\nabla \phi&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there any flux/mass-conserving schemes in FEM that have properties similar/identical to the flux/mass-conserving properties of FVM? I keep seeing in a few places that discontinuous-Galerkin conserves flux/mass, but I can't quite get a consensus on it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What would be appropriate stabilization scheme(s) to apply to this system, given the rapid fluctuations in $n$ (and $p$, identically, for the hole continuity equation)? I keep reading about streamline-upwind Petrov-Galerkin, Galerkin least Squares, and shock-capturing terms, but this seems to be a generally open question.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Given that I keep seeing shock capturing terms in my research, is it reasonable to claim that rapid, discontinuous changes in carrier density are identical to 'static' shock waves/fronts?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm still quite new to these equations, having dealt with them only for the past few months. I'd be happy to provide any more information, if required.&lt;/p&gt;&#xA;" OwnerUserId="17947" LastActivityDate="2018-04-03T02:46:22.767" Title="Finite Element Stabilization for Drift-Diffusion/Advection-Diffusion Equations" Tags="|finite-element|fluid-dynamics|finite-volume|hyperbolic-pde|advection-diffusion|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29228" PostTypeId="1" AcceptedAnswerId="29236" CreationDate="2018-04-03T06:34:27.453" Score="4" ViewCount="269" Body="&lt;p&gt;I am attempting to implement an $h$-adaptive FEM scheme for the following simple 1D problem:&#xA;$$&#xA;-u''(x) = f(x) \;\text{in}\;(0,1)\\&#xA;u(0) = u(1) = 0.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To this end, I begin with a coarse, uniform mesh of say, 4 elements and compute an error indicator on each element. I'm looking at an explicit a posteriori indicator based on the residual:&#xA;$$&#xA;\eta_{K} = h_{K}^{2}|| f + u_{h}'' ||_{K}^{2}.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen such indicators in many papers in CFD, for example, and I've been reading up on the details in the survey paper by Ainsworth and Oden&lt;a href=&quot;https://www.ices.utexas.edu/sites/oden/wp-content/uploads/2013/06/1997-006.a_posteriori.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt; [1]&lt;/a&gt;. My question is how to deal with the $u''$ term in the residual. In a typical finite element scheme, we usually work with the basis functions and their (first) spatial derivatives. If we are using linear basis functions, then $u_{h}'' \equiv 0$, yet I've seen AMR papers where linear basis functions are used with this residual error indicator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something obvious that I'm missing here? I know that in the weak formulation, we would typically &quot;pass&quot; the second derivative to a test function. I can compute something like $\int_{I_{k}} (f+u_{h}'')v\;dx$ where $v \in V_{h}$, and indeed some sources I've looked at mention an $L^{2}$ representation of the residual, but it's not immediately clear to me if this can be used to compute $\eta_{K}$.&lt;/p&gt;&#xA;" OwnerUserId="6643" LastActivityDate="2018-04-04T03:02:33.717" Title="Computation of residual error indicator in adaptive mesh refinement (FEM)" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29230" PostTypeId="1" CreationDate="2018-04-03T08:24:13.433" Score="1" ViewCount="87" Body="&lt;p&gt;I want to calculate the stream function $\psi$ starting from a velocity field $(u,v)$ (such that $u=-\frac{\partial\psi}{\partial y}$ and $v=\frac{\partial\psi}{\partial x}$). I thus calculate the vorticity $\omega$ (such that $\omega=\frac{\partial v}{\partial x}-\frac{\partial u}{\partial y}$) and then solve the Poisson equation $\nabla^2 \psi=\omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To easily specify the boundary conditions, I choose a successive over-relaxation (SOR) approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code seemingly runs smoothly. However, when I check if the solution is correct by differentiating $\psi$, its derivatives do not match the initial velocities $(u,v)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this due to some intrinsic limitation of the SOR?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mAXp7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mAXp7.png&quot; alt=&quot;u&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WiLx5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WiLx5.png&quot; alt=&quot;v&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;close all&#xA;clearvars&#xA;clc&#xA;&#xA;&#xA;%coordinates&#xA;x=1:10:600; x = repmat(x',1,50); &#xA;y=1:10:500; y = repmat(y,60,1);&#xA;dx = 10*ones(60,50); %x spacings&#xA;dy = 5*ones(60,50); %y spacings&#xA;&#xA;%define velocities&#xA;u = y.^3;&#xA;v = x.^3;&#xA;figure;pcolor(x,y, u);title('u')&#xA;figure;pcolor(x,y, v);title('v')&#xA;&#xA;%vorticity&#xA;[dudy,~] = gradient(u, dy(1,1),dx(1,1));&#xA;[~,dvdx] = gradient(v, dy(1,1),dx(1,1));&#xA;figure;pcolor(x,y, dudy);title('dudy')&#xA;figure;pcolor(x,y, dvdx);title('dvdx')&#xA;&#xA;omega = dvdx - dudy;&#xA;&#xA;figure;pcolor(x,y, omega);title('omega');colormap(jet(10));colorbar&#xA;&#xA;&#xA;%iterative solver (SOR)&#xA;psi=rand(62,50); %initial guess&#xA;&#xA;beta=dx./dy; %spacing ratio&#xA;&#xA;%boundary conditions in y&#xA;omega(:,1:2)=0;&#xA;omega(:,end-1:end)=0;&#xA;&#xA;%periodic boundary conditions in x -- append/prepend values on the other side&#xA;omega = [omega(end-1:end,:); omega; omega(1:2,:)]; &#xA;beta = [beta(end-1:end,:); beta; beta(1:2,:)];&#xA;&#xA;[M,N]=size(psi);&#xA;&#xA;w=1.4; %relaxation factor&#xA;&#xA;figure&#xA;for iter=1:1000&#xA;  for ii=3:M-2&#xA;    for jj=3:N-2&#xA;&#xA;%%fourth order formula &#xA;psi(ii,jj) = (1-w)*psi(ii,jj) + w*...&#xA;                    ( 16*( psi(ii+1,jj) + psi(ii-1,jj) ) - psi(ii+2,jj) - psi(ii-2,jj) +... &#xA;                    beta(ii,jj)^2*( -psi(ii,jj+2) + 16*(psi(ii,jj+1) + psi(ii,jj-1)) -psi(ii,jj-2) ) +...&#xA;                    12*dx(ii,jj)^2*omega(ii,jj) )...&#xA;                   /(30*(1+(beta(ii,jj)^2))); &#xA;&#xA;%%second order formula               &#xA;%          psi(ii,jj) = (1-w)*psi(ii,jj) + ...&#xA;%              w*( (psi(ii+1,jj)+beta(ii,jj)^2*psi(ii,jj+1)+psi(ii-1,jj)+beta(ii,jj)^2*psi(ii,jj-1)) -...&#xA;%              2*dx(ii,jj)^2*omega(ii,jj) )...&#xA;%              /(2*(1+(beta(ii,jj)^2)));&#xA;&#xA;    end&#xA;  end&#xA;&#xA;      if mod(iter,10)==0&#xA;        pcolor(x,y, psi(2:end-1,:))&#xA;        colorbar&#xA;        title(num2str(iter))&#xA;        drawnow&#xA;      end&#xA;&#xA;end&#xA;psi = psi(2:end-1,:); %resize back&#xA;&#xA;figure;pcolor(x,y, psi);title('psi')&#xA;&#xA;&#xA;%%%test&#xA;[dpsidy,dpsidx] = gradient(u, dy(1,1),dx(1,1));&#xA;&#xA;&#xA;figure;hold on&#xA;subplot(1,2,1)&#xA;pcolor(x,y, u); colorbar; title('u orig')&#xA;subplot(1,2,2)&#xA;pcolor(x,y, -dpsidy); colorbar; title('u reconstructed')&#xA;&#xA;figure;hold on&#xA;subplot(1,2,1)&#xA;pcolor(x,y, v); colorbar; title('v orig')&#xA;subplot(1,2,2)&#xA;pcolor(x,y, dpsidx); colorbar; title('v reconstructed')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27064" LastEditorUserId="20688" LastEditDate="2018-04-03T09:56:46.837" LastActivityDate="2018-04-03T09:56:46.837" Title="fourth order Poisson iterative solver --in Matlab" Tags="|fluid-dynamics|iterative-method|differential-equations|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29233" PostTypeId="2" ParentId="29196" CreationDate="2018-04-03T15:02:39.943" Score="1" Body="&lt;p&gt;EDIT: Intermediate solution&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from cvxpy import *&#xA;import numpy as np&#xA;&#xA;np.random.seed(1)&#xA;n = 10&#xA;&#xA;Sigma = np.random.randn(n, n) &#xA;Sigma = Sigma.T.dot(Sigma)&#xA;w = Variable(n)&#xA;&#xA;mu = np.abs(np.random.randn(n, 1))&#xA;ret = mu.T*w&#xA;&#xA;risk = quad_form(w, Sigma)&#xA;&#xA;orig_weight = [0.15,0.2,0.2,0.2,0.2,0.05,0.0,0.0,0.0,0.0]&#xA;&#xA;min_weight = [0.35,0.0,0.0,0.0,0.0,0,0.0,0,0.0,0.0]&#xA;max_weight = [0.35,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]&#xA;&#xA;lambda_ret = Parameter(sign='positive')&#xA;lambda_ret = 5&#xA;&#xA;lambda_risk = Parameter(sign='positive')&#xA;lambda_risk = 1&#xA;&#xA;penalty = Parameter(sign='positive')&#xA;penalty = 100&#xA;&#xA;penalized = True&#xA;&#xA;if penalized == True:&#xA;    print '-------------- RELAXED ------------------'&#xA;    constraints = [sum_entries(w) == 1, w &amp;gt;= 0, w &amp;gt;= min_weight, w &amp;lt;= max_weight]&#xA;    prob = Problem(Maximize(lambda_ * ret - lambda_ * risk - penalty * max_entries(sum_entries(abs(w-orig_weight)))-0.01), constraints)&#xA;else:&#xA;    print '--------------   HARD  ------------------'&#xA;    constraints = [sum_entries(w) == 1, w &amp;gt;= 0, w &amp;gt;= min_weight, w &amp;lt;= max_weight, sum_entries(abs(w-orig_weight)) &amp;lt;= 0.40]&#xA;    prob = Problem(Maximize(lambda_ret * ret - lambda_risk * risk ),constraints)&#xA;&#xA;prob.solve()&#xA;&#xA;print 'Solver Status : ',prob.status&#xA;print('Weights opt :', w.value)&#xA;&#xA;all_in = []&#xA;for i in range(n):&#xA;    all_in.append(np.abs(w.value[i][0] - orig_weight[i]))&#xA;&#xA;print 'Turnover : ', sum(all_in)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above code will force a specific increase in weight for item[0], here +20%, in order to maintain the sum() =1 constraint that has to be offset by a -20% decrease, therefore I know it will need a minimum of 40% turnover to do that, if one runs the code with penalized = False the &amp;lt;= 0.4 have to be hardcoded, anything smaller than that will fail. The penalized = True case will find the minimum required turnover of 40% and solve the optimization. What I haven't figured out yet is how I can set a minimum threshold in the relaxed case, i.e. do at least 45% (or more if required). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found some explanation around the problem here, in chapter 4.6 page 37.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://web.stanford.edu/~boyd/papers/pdf/cvx_portfolio.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boyed Paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24987" LastActivityDate="2018-04-03T15:02:39.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29234" PostTypeId="1" AcceptedAnswerId="29235" CreationDate="2018-04-03T16:46:57.097" Score="1" ViewCount="271" Body="&lt;p&gt;I am currently working on solving a positive definite symmetric systems in parallel. The parallel direct solver I used is MUMPS. However, the performance and efficiency of the parallel direct solver are quite disappointing compared to the sequential direct solver (Spooles is the one I used). The number of unknowns are 100k and 200k respectively on the figure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering that if the parallel direct solver always has this kind of efficiency or I might make some mistakes on setting up the MUMPS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, I also consider using iterative solver from PETSc but I cannot find a good example of using the block jacobi preconditioner, so it will be very much appreciate if someone is able to provide an example for PCBJACOBI and PCASM (I heard that the direct solver can be used in the subdomains and combined by using this PCASM preconditioner). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/j8FEI.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j8FEI.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5XPMO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5XPMO.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26225" LastActivityDate="2018-04-04T02:54:00.067" Title="Efficiency of parallel direct linear solver" Tags="|linear-solver|parallel-computing|petsc|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29235" PostTypeId="2" ParentId="29234" CreationDate="2018-04-04T02:54:00.067" Score="6" Body="&lt;p&gt;Your problem is too small. You have to consider that to get good efficiency, each processor has to have enough work to offset the cost of communication. In other words, there is a threshold how many degrees of freedom there have to be below which efficiency deteriorates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know where that threshold is for MUMPS. But, to give just one example, for iterative solvers in the context of finite element methods, you typically need at least 50,000 to 100,000 unknowns per processor. So, if you only have 200,000 unknowns, then you will get reasonable scalability to maybe 4 processors, but probably not beyond. It is quite possible that that threshold is even large for direct solvers, but I do not really know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really wanted to test this, I would suggest you look at a rather large problem -- say, 10 million unknowns -- and see how this scales. This will tell you something about strong scalability. Or, you take a sequence of problems -- say, of size 100,000 times the number of processors -- and measure how long it takes to solve these. This will tell you something about weak scalability.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-04T02:54:00.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29236" PostTypeId="2" ParentId="29228" CreationDate="2018-04-04T03:02:33.717" Score="3" Body="&lt;p&gt;You're not missing anything: $u_h''$ is indeed zero in the interior of cells when you use linear elements. This would suggest that for linear elements, the error estimator reduces to&#xA;$$&#xA;  \eta_K^2 = h_K^2 \|f\|_K^2.&#xA;$$&#xA;(I guess you were missing the square on the left hand side of your definition of $\eta_K$.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This turns out to be a poor choice, because locally it does not have the correct order of convergence. A better choice would be&#xA;$$&#xA;  \eta_K^2 = h_K^2 \|f-\bar f\|_K^2&#xA;$$&#xA;where $\bar f$ is the average of $f$ on $K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, either way, this estimator is pretty bad and not something you should use with linear elements. Rather, you should recognize that $u_h''$ is a function that consists of a sequence of delta functions at the interfaces of cells. So, if you wanted to derive an estimator based on that, you'd consider how large each of these delta functions are at the two end points of the interval. I won't write down the whole derivation here, but this leads exactly to the Kelly error estimator (Kelly, Gago, Babuska, et al., early 1980s).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-04T03:02:33.717" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29237" PostTypeId="2" ParentId="5072" CreationDate="2018-04-04T06:51:00.683" Score="0" Body="&lt;p&gt;To add up to Nathan's great answer with the variational reasoning, one often needs algorithmic details when implementing finite elements. For example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dseWR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dseWR.png&quot; alt=&quot;Algorithm 1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also have a more detailed explanation on the subject in &lt;a href=&quot;https://github.com/hos/implementing-finite-elements&quot; rel=&quot;nofollow noreferrer&quot;&gt;my personal notes&lt;/a&gt;. Please see the chapter &quot;Constrained Linear Systems&quot;.&lt;/p&gt;&#xA;" OwnerUserId="16947" LastActivityDate="2018-04-04T06:51:00.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29240" PostTypeId="1" AcceptedAnswerId="29243" CreationDate="2018-04-04T18:21:26.830" Score="2" ViewCount="493" Body="&lt;p&gt;I am implementing a finite difference method in solving the diffusive-advective equation:&#xA;$$&#xA;u_t + v \cdot u_x = D\cdot u_{xx}&#xA;$$&#xA;(v, D are constants). Planning to use the operator splitting method (see below), I am now &lt;strong&gt;focusing on the advective part&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I chose staggered leapfrog method (from &lt;a href=&quot;http://numerical.recipes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Recipes book&lt;/a&gt;)&#xA;$$&#xA;u^{n+1}_j = u^{n-1}_j - \frac{v\cdot\Delta t }{\Delta x}(u^n_{j+1} - u^n_{j-1}) \;\;\;\;&#xA;$$&#xA;because it should be Courant-stable in case of conserved flux; this turns out to be the case, since it works perfectly using periodic conditions.&#xA;The problems show when trying to change the boundaries.&#xA;My question, in brief, is: how to explicitly implement these conditions in such a method? Below, I am going to provide more details on my failings. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;No-flux conditions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Let us start from a more general case: as it is pointed out &lt;a href=&quot;https://scicomp.stackexchange.com/a/5440/27377&quot;&gt;here&lt;/a&gt; for diffusion-advection equation, Robin conditions are required to achieve closed boundaries&#xA;$$&#xA; v\cdot u - Du_x = 0&#xA;$$&#xA;I tried to discretize them:&#xA;$$&#xA;v\cdot u_j - \frac{D}{\Delta x} (c_j - c_{j-1}) = 0&#xA;$$&#xA;However, when D=0 the only condition remaining is that &#xA;$$&#xA;v\cdot u = 0&#xA;$$&#xA;thus it is to me unclear how to interpret this: imposing u=0 would result in the (failing) strategy for absorbing conditions (see below), while v is constant (and I don't understand where I could impose it equals to zero). &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Absorbing conditions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Leapfrog method is meant for flux-conserving situations. In fact, imposing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u(j=0,n) = 0 &#xA;u(jmax,n) = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;creates artificial instabilities. &#xA;The leapfrog updating has got three parts: one for the previous (two steps before) value in the point, one increasing this value proportionally to the uphill point, the other reducing proportionally to the downhill one. This is also what one could expect from a physical point of view: some quantity enters, some goes out. The reason why the simple imposition of null boundaries fails is now evident: if the downhill point is null, there is no reducing term, so the point before the last explodes; the point before it has now a big reducing term, so goes to zero; the point before then has a small reducing term and so on, resulting in an alternance of increasing and decreasing terms (see figure &lt;a href=&quot;https://i.stack.imgur.com/WaDj7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WaDj7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WaDj7.png&quot; alt=&quot;fig.1&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Operator splitting&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As explained in the Numerical Recipes book (cited above), an equation like&#xA;$$&#xA;u_t = \mathcal{L} u&#xA;$$&#xA;where a decomposition in linear operators &#xA;$$&#xA;\mathcal{L}=\sum^m_{i}\mathcal{L}_i&#xA;$$&#xA;is possible can be resolved applying, for each step, the sequence &#xA;$$&#xA;\mathcal{U}_1(\mathcal{U}_2(\dots\mathcal{U}_m(u^n)\dots)) = u^{n+1}&#xA;$$&#xA;provided that each operator U solves for a term in the sum and is stable.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Even if I know alternatives are possible, which contemplate diffusion-advection function in general, before digging into them I am willing to solve this doubt of mine.&lt;/p&gt;&#xA;" OwnerUserId="27377" LastEditorUserId="20688" LastEditDate="2018-04-05T14:45:00.417" LastActivityDate="2018-04-05T14:47:44.017" Title="Closed boundary conditions in finite difference method for diffusive-advective equation" Tags="|finite-difference|boundary-conditions|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29242" PostTypeId="2" ParentId="29226" CreationDate="2018-04-05T10:11:46.617" Score="1" Body="&lt;p&gt;What about dividing the interval $[0,0.6]$ in sub-intervals $[x_i,x_{i-1}]$ so that in each one the function value has the same order of magnitude?&#xA;For example you could choose $x_0 =0$, $x_{m}=0.6$&#xA;$$&#xA;x_i \;|\; 0&amp;lt;i&amp;lt;m \; \wedge \;F(x_i) = 10^i &#xA;$$&#xA;Then you can integrate in each one and sum the partial results to get the total integral. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never done this, but it is what I would try first. The contributions at the extreme right of each interval will probably be underestimated, but at least this effect is reduced. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be also sure to properly consider errors (they depend on your specific method): if the bump gives a contribution which is smaller than the uncertainty, either you have to reduce the error (which I suppose is your case, since you are interested in the bump) or you can ignore the bump.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you try this, let me know if it works!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Detailed implementation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;(pseudo-code)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hit = 0&#xA;miss = 0&#xA;x = generate_random(x_interval)&#xA;y = generate_random([0,max_of_function])&#xA;if(F(x)&amp;gt;y) hit++ &#xA;else if(F(x)&amp;lt;y) miss++&#xA;percentage_hits = hit/(hit+miss)&#xA;total_area = (x_interval*max_of_function)&#xA;function_area = percentage_hits * total_area&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you have the area in the x_interval selected. Compute the same for each interval, then sum all the partial areas.&lt;/p&gt;&#xA;" OwnerUserId="27377" LastEditorUserId="27377" LastEditDate="2018-04-05T21:09:43.477" LastActivityDate="2018-04-05T21:09:43.477" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29243" PostTypeId="2" ParentId="29240" CreationDate="2018-04-05T13:16:26.833" Score="1" Body="&lt;p&gt;If you consider only the pure &lt;strong&gt;advective&lt;/strong&gt; part and let, say, $v&amp;gt;0$, then you can obtain &quot;no flux condition&quot; at the right point only by setting (redefining) $v=0$ there. Of course, the solution can then develop a &quot;boundary layer&quot; there and it can become discontinuous at the right point as the solution will preserve there the value given by initial condition, i.e. $u_{jmax}^{n+1} = u^0_{jmax}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A natural boundary condition for constant speed $v&amp;gt;0$ is that you let the solution &quot;flow out&quot; through the right point of interval that can be viewed as &lt;strong&gt;outflow&lt;/strong&gt; boundary. It can be obtained by applying your leapfrog method for $u_{jmax}^{n+1}$ with the auxiliary value $u_{jmax+1}^n= u_{jmax}^n$ (a constant extrapolation) or $u_{jmax+1}^n= 2 u_{jmax}^n - u_{jmax-1}^n$ (a linear extrapolation). One can see such treatment as &quot;do nothing boundary conditions&quot; that is appropriate for outflow boundary and pure advection. It is like you are solving the pure advection (the differential equation) also at the right point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a remark - if you use a proper upwind method instead of leapfrog method, you even need no auxiliary values outside of your domain to compute $u_{jmax}^{n+1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;strong&gt;inflow&lt;/strong&gt; boundary, i.e. the left point in the case $v&amp;gt;0$, you must prescribe the values $u_0^{n+1}$ directly by &quot;Dirichlet boundary conditions&quot; and not solving the differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the case of &lt;strong&gt;advection-diffusion&lt;/strong&gt; equation you can in theory define the no-flux boundary conditions as you write with $D \neq 0$, but it means for $v&amp;gt;0$ that you prescribe an &quot;inward diffusion&quot;, consequently a corresponding gradient, that will cancel the advection flux, so the total flux will be zero. Such situation is hard to find in practice. More typical is to prescribe something only for the diffusion flux and to use the previous ``do nothing'' approach (or an extrapolation) for the advective flux.&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2018-04-05T14:47:44.017" LastActivityDate="2018-04-05T14:47:44.017" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29245" PostTypeId="1" CreationDate="2018-04-06T01:44:53.103" Score="1" ViewCount="66" Body="&lt;p&gt;I'm trying to solve the fully coupled drift-diffusion system using Newton's Method. Although I eventually plan to potentially use a Jacobian-Free Newton-Krylov approach, this is still something that I want to implement and look into.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bit of notation for the equations presented below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$d_{ij}$ and $l_{ij}$ are control area/volume constants, not physical entities. The $i$ and $j$ indices are for the central point being considered ($i$), and all neighbouring points that are connected ($j$'s). $A_{\Omega_i}$ is the control area/volume associated to each vertex/node, and is also constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All quantities that are explicitly required during calculation are at node points. In the equations $B(\Delta_{ij}) \equiv B(\phi_i - \phi_j)$, the Bernoulli function, used as a smoothing method for the differences. This difference should &lt;em&gt;technically&lt;/em&gt; exist between node points, but that's for another time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the equations: $\phi$ is electrostatic potential, $n$ and $p$ are carrier concentrations. These are the independent variables to solve for. Everything else is either a constant, or a function of one of these variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j\ne i} \frac{d_{ij}}{l_{ij}}(\phi_i - \phi_j) = -\frac{q}{\epsilon}(p_i - n_i - N_{D_i}^+ - N_{A_i}^-)A_{\Omega_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial{n_i}}{\partial t}A_{\Omega_i} = D_n \sum_{j \ne i} \left[\frac{d_{ij}}{l_{ij}} \{n_iB(\Delta_{ij}) - n_j B(-\Delta_{ij})\}\right] + (G_i - R_i)A_{\Omega_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial{p_i}}{\partial t}A_{\Omega_i} = D_p \sum_{j \ne i} \left[\frac{d_{ij}}{l_{ij}} \{p_iB(-\Delta_{ij}) - p_j B(\Delta_{ij})\}\right] + (G_i - R_i)A_{\Omega_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In residual form, and assuming equilibrium conditions so all time derivatives go to 0, these equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Phi = \sum_{j\ne i} \frac{d_{ij}}{l_{ij}}(\phi_i - \phi_j) +\frac{q}{\epsilon}(p_i - n_i - N_{D_i}^+ - N_{A_i}^-)A_{\Omega_i} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$N = D_n \sum_{j \ne i} \left[\frac{d_{ij}}{l_{ij}} \{n_iB(\Delta_{ij}) - n_j B(-\Delta_{ij})\}\right] + (G_i - R_i)A_{\Omega_i} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P = D_p \sum_{j \ne i} \left[\frac{d_{ij}}{l_{ij}} \{p_iB(-\Delta_{ij}) - p_j B(\Delta_{ij})\}\right] + (G_i - R_i)A_{\Omega_i} = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my question is whether I have these Jacobian elements correct. Here is the Jacobian element formula that I've tried to come up with. Only a few equations have been derived, since the remainining elements are just trivial variations of these 3 formulations. This post is mostly about getting more experienced eyes to look at these equations, and potentially see something that I haven't quite seen yet. Are these formulations for the Jacobian elements correct, or is there something wrong with them that I'm missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial\Phi}{\partial\phi_i} = \sum_{j\ne i} \frac{d_{ij}}{l_{ij}} +\frac{q}{\epsilon}(\frac{\partial p_i}{\partial \phi_i} - \frac{\partial n_i}{\partial \phi_i})A_{\Omega_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial{N}}{\partial \phi_i} =  D_n \sum_{j \ne i} \left[\frac{d_{ij}}{l_{ij}} \left\{\frac{\partial{n_i}}{\partial \phi_i}B(\Delta_{ij}) + n_i \frac{\partial B(\Delta_{ij})}{\partial \phi_i})\right\}\right] &#xA;+ \frac{\partial(G_i - R_i)}{\partial \phi_i}A_{\Omega_i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial N}{\partial n_i} = D_n \sum_{j \ne i} \left[\frac{d_{ij}}{l_{ij}} B(\Delta_{ij})\right] + \frac{\partial (G_i - R_i)}{\partial n_i}A_{\Omega_i}$$&lt;/p&gt;&#xA;" OwnerUserId="17947" LastActivityDate="2018-04-06T01:44:53.103" Title="Jacobian Elements for Coupled Drift-Diffusion System using Vertex-Centered Finite Volume" Tags="|pde|computational-physics|nonlinear-equations|finite-volume|advection-diffusion|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29246" PostTypeId="1" AcceptedAnswerId="29250" CreationDate="2018-04-06T03:58:59.453" Score="3" ViewCount="1368" Body="&lt;p&gt;If given a displacement gradient tensor, we can easily obtain the stress tensor (using Hooke's law and the strain-displacement relationship), as well as the traction vector. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If given a traction, and knowing that traction $\boldsymbol{T} = \boldsymbol{\sigma n}$ ,where $\boldsymbol{n}$ is the normal vector, we can &lt;strong&gt;not&lt;/strong&gt; uniquely determine the 6 unique components of the stress tensor without some additional condition(s). The same scenario exists for going from stress to displacement gradients, where we know 6 unique stress components, but cannot uniquely determine the 9 displacement gradients without some other condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of solving the linear elasticity equations, Neumann boundary conditions are typically imposed in terms of tractions. Would it make the problem more well-posed if Neumann boundary conditions were imposed in terms of stress, or even better, in terms of a displacement gradient? &lt;/p&gt;&#xA;" OwnerUserId="23915" LastEditorUserId="20688" LastEditDate="2018-04-06T04:49:40.527" LastActivityDate="2018-04-06T14:58:28.113" Title="Traction -&gt; stress; stress-&gt;displacement gradient" Tags="|boundary-conditions|solid-mechanics|well-posedness|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29247" PostTypeId="2" ParentId="5526" CreationDate="2018-04-06T05:29:52.663" Score="1" Body="&lt;p&gt;The python &lt;code&gt;PyMOL&lt;/code&gt; library seems to be pretty close to your needs. For example, see &lt;a href=&quot;https://pymolwiki.org/index.php/Mutagenesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;example on mutagenesis&lt;/a&gt;. It is Python, not C++; however, you can &lt;a href=&quot;https://pymolwiki.org/index.php/Advanced_Scripting&quot; rel=&quot;nofollow noreferrer&quot;&gt;get the best of both worlds&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you are more interested in the &lt;em&gt;process&lt;/em&gt; rather than the result, I would suggest taking some PDB IO library, say, &lt;a href=&quot;http://www.cgl.ucsf.edu/Overview/software.html#pdbio&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB Record IO Libraries from UCSF, open-source, C/C++&lt;/a&gt;, read your PDB data and write a mutation function according to your needs. With that, you would avoid the development of PDB IO and storage and focus on the interesting parts.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-04-06T05:29:52.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29248" PostTypeId="2" ParentId="18688" CreationDate="2018-04-06T07:32:51.210" Score="2" Body="&lt;p&gt;As per the &lt;a href=&quot;http://savannah.gnu.org/bugs/?44004&quot; rel=&quot;nofollow noreferrer&quot;&gt;submitted bug report&lt;/a&gt;, there was a bug in &lt;code&gt;ARPACK&lt;/code&gt; &amp;lt; &lt;strong&gt;3.1.5&lt;/strong&gt; regarding &lt;code&gt;dneupd&lt;/code&gt;, the subroutine that &quot;returns the converged approximations to eigenvalues&quot;. In &lt;code&gt;ARPACK&lt;/code&gt; &lt;strong&gt;3.2.0&lt;/strong&gt;, this particular issue has been resolved. Current distributions of &lt;code&gt;octave&lt;/code&gt; are usually linked with a much higher version of &lt;code&gt;ARPACK&lt;/code&gt;; thus, such behaviour is not expected to happen anymore. But checking that your &lt;code&gt;octave&lt;/code&gt; is linked against a new (technically, I want to say &lt;em&gt;stable&lt;/em&gt; version of &lt;code&gt;ARPACK&lt;/code&gt;) is worth a shot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tested the code above in &lt;code&gt;octave 4.2.2&lt;/code&gt; with &lt;code&gt;ARPACK 3.5.0&lt;/code&gt; installed using MacPorts. The results seem to be stable with a max difference in the eigenvalues between the two around &lt;code&gt;1E-12&lt;/code&gt;, which is, I think, more than satisfactory. &lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-04-06T07:32:51.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29249" PostTypeId="1" CreationDate="2018-04-06T09:11:39.133" Score="3" ViewCount="93" Body="&lt;p&gt;For my PhD Thesis, my advisor asked me to build a solver inspired from the article &quot;&lt;a href=&quot;https://journals.ametsoc.org/doi/abs/10.1175/1520-0426(1990)007%3C0269%3AOCTATA%3E2.0.CO%3B2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optimal Control Theory Applied to an Objective Analysis of a Tidal Current Mapping by HF Radar, J-L Devenon, 1989&lt;/a&gt;&quot;. In this article, in the numerical fluid-solving part, the author linearize the shallow water 2-D equations by removing the advection terms and using a linear friction coefficient. Then he uses a Fourier transform on the equation to finally end up with the system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;-i\sigma\vec{U}+\vec{f}\times\vec{U}+g\vec{\nabla}h&amp;amp;=-r\vec{U}\\\\&#xA;i\sigma h+\vec{\nabla}\cdot\left(H\vec{U}\right)&amp;amp;=0&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $i$ is the basis for the imaginary number, $\sigma$ the frequency of the given Fourier component, $\vec{f}$ the Coriolis force vector, $g$ the gravitational acceleration, $h$ the local sea-level elevation compared to &quot;mean local level&quot;, $r$ the linear friction coefficient, $H$ the local depth and $\vec{U}$ the local velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, this system is simplified in a singled vector equation of the form :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F(\vec{V})=\lambda\vec{V}+\vec{f}\times\vec{V}+\vec{\nabla}\left(\vec{\nabla}\cdot\left(\gamma\vec{V}\right)\right)=\vec{0}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\lambda = rH-i\sigma,\quad \gamma=\frac{-igH}{\sigma}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And $\vec{V}$ is the transport, given as the product of the speed (supposed constant on the entire water-column) and the local height of the water-column( supposing the fluctuations $h$ are small compared to the overall depth).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He then solves the Dirichlet problems with specified boundary conditions. For this, he uses an iterative relaxation method (from Smith, G.D, Numerical solution of partial differential equations: Finite differences methods, 1978). And use only centered discretisation schemes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then wrote my own code for solving it. My first attempts where on a rectangular bassin with fixed boundary conditions. However It did not seem to work with any condition that differed from 0. I also tried to figure out analytical solutions but the only (very obvious) solution I found was the null solutions everywhere. Here is an example of the code I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;############################################&#xA;# First test of implementing Devenon's     #&#xA;# Equation system and solving them in a clo#&#xA;# -sed basin                               #&#xA;############################################&#xA;&#xA;#*****Packages import**********************#&#xA;import numpy as np&#xA;&#xA;&#xA;&#xA;#****Functions*****************************#&#xA;&#xA;&#xA;def  relaxation_scheme_Jean_Luc_overrelaxed(U,V,dy,dx,nx,ny,gamma,lamda,omega):&#xA; &quot;&quot;&quot;Function use to solve 2D linearized equation&#xA; of Navier-Stokes for a rigid lid. From Devenon 1989&#xA; . We're trying an overrelaxation technique with omega&#xA;  as the relaxation parameter.&#xA;&#xA; INPUT:&#xA; -----&#xA; U,V: Zonal and meridional speed, size nx*ny, also the output,complex valued&#xA; nx: Number of grid cell in x direction&#xA; ny: Number of grid cell in y direction&#xA; dx: Size of a grid cell in x direction&#xA; dy: Size of a grid cell in y direction&#xA; b: Forcing source function, of size nx x ny&#xA; gamma: Local complex phase speed&#xA; lamda : Local complex wave vector&#xA; omega: Relaxation factor, must belong to [1:2]&#xA;&#xA; OUTPUT:&#xA; -------&#xA; U,V: Already specified earlier &quot;&quot;&quot;&#xA;&#xA; # First we compute the non-homogeneous terms&#xA;&#xA; #U_b=f*V[1:-1,1:-1]+ (gamma[:-2,:-2]*V[:-2,:-2] + &#xA; #       gamma[2:,2:]*V[2:,2:]-gamma[2:,:-2]*V[2:,:-2]-&#xA; #       gamma[:-2,2:]*V[:-2,2:])/(4*dy*dx) &#xA;&#xA; #V_b= -f*U[1:-1,1:-1]+ (gamma[:-2,:-2]*U[:-2,:-2] +&#xA; #         gamma[2:,2:]*U[2:,2:]- gamma[2:,:-2]*U[2:,:-2]-&#xA; #         gamma[:-2,2:]*U[:-2,2:])/(4*dx*dy)&#xA;&#xA; U_b=V[1:-1,1:-1]*0.&#xA; V_b=V[1:-1,1:-1]*0.&#xA; # Now we can use relaxation technique with an external term&#xA; U[1:-1,1:-1] = 1/lamda[1:-1,1:-1] * (omega*gamma[2:,1:-1]*U[2:,1:-1]+&#xA;                2*(1-omega)*gamma[1:-1,1:-1]*U[1:-1,1:-1]+(omega-1)*gamma[:-2,1:-1]*U[:-2,1:-1]) \&#xA;                /dx**2  +1/lamda[1:-1,1:-1]*U_b&#xA;&#xA; V[1:-1,1:-1] = 1/lamda[1:-1,1:-1] * (omega*gamma[1:-1,2:]*V[1:-1,2:]+&#xA;                 2*(1-omega)*(gamma[1:-1,1:-1]*V[1:-1,1:-1])+(omega-1)*gamma[1:-1,:-2]*V[1:-1,:-2]) \&#xA;                 /dy**2 +1/lamda[1:-1,1:-1]*V_b&#xA;&#xA; return U,V&#xA;&#xA;#*****Parameters and variables*************#&#xA;nx = 40&#xA;ny = 40 # Number of grid points in each direction&#xA;nt  = 100 # Number of iteration of the relaxation method&#xA;xmin = 0.&#xA;xmax = 12000.&#xA;ymin = 0.&#xA;ymax = 12000. # Dimension (in meters) of the grid&#xA;&#xA;dx = (xmax - xmin) / (nx - 1)&#xA;dy = (ymax - ymin) / (ny - 1)&#xA;&#xA;# Initialization&#xA;# I understood that each speed would be complex-valued&#xA;# However, I wondered if U and V could only be the real and imaginary&#xA;# Part of the same variable in this case&#xA;U,V  = np.zeros((nx,ny),dtype=complex),np.zeros((nx,ny),dtype=complex)&#xA;x  = np.linspace(xmin, xmax, nx)&#xA;y  = np.linspace(xmin, xmax, ny)&#xA;X, Y = np.meshgrid(x, y)&#xA;&#xA;&#xA;H=10 # Setting constant depth for testing&#xA;sigma=2.31484e-5 # Approximately semidiurnal frequency,  for tide (in hertz)&#xA;g=9.81 # Gravity acceleration&#xA;K=2.5e-2 #Friction coefficient&#xA;f=2e-4&#xA;lamda=np.ones((nx,ny),dtype=complex)*(K-sigma*1j)# (Friction and wavenumber)&#xA;gamma=np.ones((nx,ny),dtype=complex)*-1j*g*H/sigma #(Gravity and wavenumber)&#xA;&#xA;#Initial conditions&#xA;U[0,:]=1.&#xA;U[-1,:]=1.&#xA;U[:,0]=1.&#xA;U[:,-1]=1.&#xA;&#xA;V[0,:]=1.&#xA;V[-1,:]=1.&#xA;V[:,0]=1.&#xA;V[:,-1]=1.&#xA;&#xA;#********Solving****************************#&#xA;&#xA;for i in range(nt):    &#xA;    U,V=relaxation_scheme_Jean_Luc_overrelaxed(U,V,dy,dx,nx,ny,gamma,lamda,2.)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For any conditions, I used except for the constant ones I found whether a very stable behaviour (the system stayed exactly in its initial position) or a very unstable one (I have instabilities growing from the sides that propagate, very quickly giving me aberrant values). This behaviour seems to be driven by the parameters lambda, gamma as well as the size of the cells. However no combination I have tried yet helped me finding a solution. This leads me to believe that I did a very bad job at coding this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried searching the site ( and the web in general for answers) and found many links for nice EDP and even Navier-Stokes solvers. Though I did not get anything that I could link to my problem, may be due to a poor comprehension of it. This being said, I have another constraint that my advisor would like this to become an educational and simple problem. I, therefore, have to understand the internal functioning of my solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I have a few questions for you :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) The first is obvious: Why doesn't it work? Did I do a bad job at coding? What should I correct to have a correct working algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) The second is less obvious: Can I write my vector V in a complex manner, using one component of the transport as the real part and the second as the imaginary part? This seems obvious to my advisor but it does not look obvious to me at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) The paper refers to it as an &quot;elliptic&quot; problem, close to a Helmholtz one. However, I just see two ODE's with a coupling term. Can I talk about an elliptic problem in this case?&lt;/p&gt;&#xA;" OwnerUserId="27395" LastEditorUserId="20688" LastEditDate="2018-04-06T14:11:13.593" LastActivityDate="2018-04-06T14:11:13.593" Title="Solving numerically a linearized system of elliptic (?) Navier-Stokes equation (Shallow Water Derived)" Tags="|finite-difference|navier-stokes|elliptic-pde|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29250" PostTypeId="2" ParentId="29246" CreationDate="2018-04-06T14:58:28.113" Score="6" Body="&lt;p&gt;In short, no. Neumann boundary conditions should be specified in terms of traction. This is clear when you move from a strong-form statement of the boundary value problem to a weak-form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strong form:&#xA;\begin{align}&#xA;div(\sigma(\mathbf{u})) + \mathbf{b} &amp;amp;= \mathbf{0} &amp;amp;&amp;amp;\forall \mathbf{x} \in \Omega \\&#xA;\mathbf{u} &amp;amp;= \hat{\mathbf{u}}(\mathbf{x}) &amp;amp;&amp;amp;\forall \mathbf{x} \in \partial\Omega_D \\&#xA;\sigma \mathbf{n} &amp;amp;= \mathbf{g}(\mathbf{x}) &amp;amp;&amp;amp;\forall \mathbf{x} \in \partial\Omega_N \\&#xA;\partial\Omega_N \cup\partial\Omega_D &amp;amp;= \partial\Omega&amp;amp;&amp;amp; \\&#xA;\partial\Omega_N \cap\partial\Omega_D &amp;amp;= \emptyset&amp;amp;&amp;amp;&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking $\mathbf{w}$ to be an appropriate test function that vanishes on $\partial\Omega_D$, the weak form is written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\int_\Omega \left(\mathbf{w}\cdot\mathbf{b} - \nabla\mathbf{w}:\sigma \right)\,\mathrm{d}V + \int_{\partial\Omega_N}\mathbf{w}\cdot(\sigma\mathbf{n}) \,\mathrm{d}A = \mathbf{0}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the Neumann boundary condition appears directly as a traction in the weak form, so it is quite natural to specify it in terms of traction rather than a displacement gradient. In fact, it is not possible to impose the entire displacement gradient or stress tensor at a boundary, only the traction.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2018-04-06T14:58:28.113" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29251" PostTypeId="1" CreationDate="2018-04-06T20:18:16.593" Score="1" ViewCount="449" Body="&lt;p&gt;I am trying to solve the Helmholtz equation with Robin boundary conditions with complex coefficients and the weak formulation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\iint_\limits\Omega\nabla p_0(x,y)\nabla\left(\overline{v(x,y)}\right)dxdy-\frac{w^2}{c^2}\iint\limits_{\Omega}p_0(x,y)\left(\overline{v(x,y)}\right)dxdy\\&#xA;+ik\int\limits_{\partial\Gamma_R}p_0(x,y)\left(\overline{v(x,y)}\right)dx=0, \quad \forall v\in V&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I do not know how can I prove the existence and uniqueness of the solution because the coercitive in the Lax-Milgram theorem is a big problem.&lt;/p&gt;&#xA;" OwnerUserId="27401" LastEditorUserId="20688" LastEditDate="2018-11-19T05:59:59.033" LastActivityDate="2022-04-02T14:05:20.663" Title="Proving solution existence and uniqueness of the Helmholtz equation with Robin boundary conditions with complex coefficients" Tags="|finite-element|boundary-conditions|helmholtz-equation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29253" PostTypeId="1" AcceptedAnswerId="30816" CreationDate="2018-04-07T16:58:24.070" Score="0" ViewCount="48" Body="&lt;p&gt;I have a time iteration function looked on a 2D surface like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KLv7H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KLv7H.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Since the numbers wee very small i.e. hbar=6.6260700404e-34./(2*pi), my professor told me to use our own &quot;scaled unites&quot; during the calculation. ie. instead of using $1e-9$ (meter), scale the unit and use $1e-6$ ($\mu m$) during the calculation. e.t.c.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It worked very well when I was only changing the space parameter, (m) However, when I adjust the space scale and time scale together. I encounter a problem, where the function stopped being evolving smoothly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JKHPd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JKHPd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked my substitution, and they were correct. But what happened here? Can we really save time of calculation by using scaled units?&lt;/p&gt;&#xA;" OwnerUserId="26929" LastEditorUserId="26929" LastEditDate="2018-04-07T20:27:03.723" LastActivityDate="2018-12-31T16:38:45.713" Title="Time iteration no longer smooth after using scaled units" Tags="|numerical-analysis|iterative-method|precision|scaling|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="29254" PostTypeId="2" ParentId="28676" CreationDate="2018-04-07T17:26:47.200" Score="0" Body="&lt;p&gt;Here is a piece of code I developed in order to fit polinomials using regularization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;pre&gt;&#xA;/* x_tr and y_tr is trainig data, x_te and y_te test data */&#xA;degree = 4&#xA;identity_size = degree + 1&#xA;x_tilda = np.array([x_tr, np.ones(len(x_tr))]).transpose()&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for degree in range(2, degree + 1):&#xA;    x_tilda = np.append(np.array(np.power(x_tr, degree))[np.newaxis].transpose(), x_tilda, axis=1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;y_reg = y_tr&lt;/p&gt;&#xA;&#xA;&lt;p&gt;identidad = np.identity(identity_size)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;def get_theta(rho):&#xA;    mul1 = np.linalg.inv(&#xA;        np.add(&#xA;            np.matmul(x_tilda.transpose(), x_tilda), &#xA;            np.dot(identidad, rho)))&#xA;    mul2 = np.matmul(x_tilda.transpose(), y_reg)&#xA;    return np.matmul(mul1, mul2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;def poly(x, rho): &#xA;    theta = get_theta(rho)&#xA;    print(&quot;Theta polyfit: &quot;, np.polyfit(x_tr,y_reg, degree))&#xA;    print(&quot;Theta formula: &quot;, get_theta(rho))&lt;/p&gt;&#xA;&#xA;&lt;code&gt;    p = np.poly1d(theta)&#xA;    return p(x)&#xA;&#xA;rho = 0  // this is the regularization param&#xA;plt.figure()&#xA;plt.scatter(x_te,poly(x_te, rho), c ='r')&#xA;plt.scatter(x_te,y_te, c ='b')&amp;lt;code&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Above solution based on the following formula for polynomials (instead of X, it is used poly(X) for a general analytical solution with a squared-error loss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\hat{\beta} = (X^TX)^{-1}X^TY&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="27409" LastEditorUserId="20688" LastEditDate="2018-04-07T19:31:59.353" LastActivityDate="2018-04-07T19:31:59.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29256" PostTypeId="2" ParentId="25562" CreationDate="2018-04-08T00:39:05.413" Score="1" Body="&lt;p&gt;As nicoguaro mentioned, &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARPACK&lt;/a&gt; has routines that will naturally handle banded matrices that are stored in &lt;a href=&quot;http://www.netlib.org/lapack/lug/node124.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAPACK band format&lt;/a&gt;. Moreover, since ARPACK uses matrix-vector products (MVP) to find eigenvalues, you might &quot;connect&quot; it to your own MVP subroutine without the need to reorganize matrix storage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a primarily C++ user, at some point, I tried using &lt;a href=&quot;https://eigen.tuxfamily.org/dox/unsupported/group__ArpackSupport__Module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigen unsupported ARPACK module&lt;/a&gt;, but did not check it for a while. Eigen will provide you with convenient wrappers for your matrix storage and basic linear algebra operations. Though you can use ARPACK via Matlab, Python, and many other ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The python scipy page on &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/tutorial/arpack.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sparse Eigenvalue Problems with ARPACK&lt;/a&gt; gives an idea of what to do for your problem of finding $n$ smallest eigenvalues by switching to a corresponding &lt;a href=&quot;http://www.caam.rice.edu/software/ARPACK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;computation mode&lt;/a&gt;. See references on the &lt;code&gt;which&lt;/code&gt; parameter: &lt;code&gt;SM&lt;/code&gt; (smallest magnitude), &lt;code&gt;SR&lt;/code&gt; (smallest real part), &lt;code&gt;SI&lt;/code&gt; (smallest imaginary part), &lt;code&gt;SA&lt;/code&gt; (smallest algebraic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$2500 \times 2500$ does not sound like a large problem; however, if the scaling to larger sizes becomes an issue, one might also look into &lt;a href=&quot;http://slepc.upv.es/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SLEPc&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-04-08T00:39:05.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29258" PostTypeId="2" ParentId="25419" CreationDate="2018-04-08T01:12:23.200" Score="1" Body="&lt;p&gt;I would suggest that a slightly modified version of @GoHokies suggestion &lt;strong&gt;array layout&lt;/strong&gt; or even a more precise, but a bit wordish &lt;strong&gt;array memory layout&lt;/strong&gt; should suffice and be unambiguous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, this term is the clearest one, describing what &lt;em&gt;row-major&lt;/em&gt; and &lt;em&gt;column-major&lt;/em&gt; (and possibly some other variants) is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Google search (Apr 7, 2018, google.ca):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&quot;array memory layout&quot; - 2080 results with 100% of all first 10 results and images being to the topic&lt;/li&gt;&#xA;&lt;li&gt;&quot;array layout&quot; - 67500 results with only 3/10 directed at different areas of science (say, &quot;array layout methodology for VLSI circuits&quot;). So I would say it is safe to use &lt;/li&gt;&#xA;&lt;li&gt;&quot;memory layout&quot; results in a much larger #hits, but only 1/10 is relevant for the topic of the discussion&lt;/li&gt;&#xA;&lt;li&gt;&quot;storage order&quot; results in more than 100000  hits; however, a lot of them lead to a different (slightly or completely) areas.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://dl.acm.org/citation.cfm?id=2747875&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; uses a term &quot;data layout&quot; to generalize, but talks particularly about the ways arrays are stored in memory.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://engineering.purdue.edu/~mithuna/pubs/ics99.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; particularly discusses the &quot;array layouts&quot;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Though I doubt that any computational scientist or an average programmer would be confused by a term &lt;strong&gt;storage order&lt;/strong&gt;, that is used wider.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-04-08T01:12:23.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29260" PostTypeId="1" AcceptedAnswerId="29270" CreationDate="2018-04-09T02:42:25.963" Score="1" ViewCount="2250" Body="&lt;h1&gt;The problem&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I'm trying to make a metropolis simulation of the 2D Ising model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, it's the following, for each monte-carlo step: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Visit each lattice site, &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute energy required to flip the spin: &lt;code&gt;dE&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If $exp(-\frac{dE} {kT}) &amp;gt; p$ where &lt;code&gt;p&lt;/code&gt; is a random number in [0,1], flip the spin, else leave it.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Heat capacity I retrieve from the Fluctuation dissipation theorem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ C_v = \frac{\sigma_E^2}{k_B T^2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code reproduces some of the expected behaviour: i.e. there's a critical transition at vaguely the right temperature, but:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Energy has a smoothed-out transition. Magnetisation is a sharp step down from 1 to 0, while energy continues to rise, and doesn't show any change of gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The heat capacity shows a peak, but instead of rising when I increase the lattice size, the peak gets smaller.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The heat capacity is noisy: the peak is often doubled, and fitting it to a Lorentzian is a nightmare. Worse yet, running the simulation multiple times gets me the same data (as evidenced by the small errorbar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My project is a c++ program that simulates a single lattice, and a python script that analyses the output. As of now the script pipes the program stdout and assigns it to internal variables. It's inefficient, but that's for debugging purposes.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;The code&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The full code can be found here: &lt;a href=&quot;https://github.com/appetrosyan/Ising-Model-simulation.git&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt;. The code is a c++ program that runs a simulation on a single lattice and a python script that analyses the data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code snippets of importance (i.e. that don't contain the irrelevant housekeeping), are &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;simulation.cpp&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#include &quot;include/simulation.h&quot;&#xA;&#xA;using namespace std;&#xA;&#xA;/*&#xA;Advances the simulation a given number of steps, and updates/prints the statistics&#xA;into the given file pointer.&#xA;&#xA;Defaults to stdout.&#xA;&#xA;The number of time_steps is explcitly unsigned, so that linters/IDEs remind&#xA;the end user of the file that extra care needs to be taken, as well as to allow&#xA;advancing the simulation a larger number of times.&#xA;*/&#xA;void simulation::advance(unsigned int time_steps, FILE *output) {&#xA;  unsigned int area = spin_lattice_.get_size() * spin_lattice_.get_size();&#xA;  for (unsigned int i = 0; i &amp;lt; time_steps; i++) {&#xA;    /*&#xA;    To simulate an adiabatic Ising lattice, uncomment this&#xA;    */&#xA;    // double temperature_delta = total_energy_/area - mean_energy_;&#xA;    // if (abs(temperature_delta) &amp;lt; 1/area){&#xA;    //   cerr&amp;lt;&amp;lt;temperature_delta&amp;lt;&amp;lt;&quot;! Reached equilibrium &quot;&amp;lt;&amp;lt;endl;&#xA;    // }&#xA;    // temperature_ += temperature_delta;&#xA;&#xA;    if (time_ % print_interval_ == 0) {&#xA;      total_magnetisation_ = spin_lattice_.total_magnetisation();&#xA;      mean_magnetisation_ = total_magnetisation_ / area;&#xA;      total_energy_ = compute_energy(spin_lattice_);&#xA;      mean_energy_ = total_energy_ / area;&#xA;      print_status(output);&#xA;    }&#xA;    advance();&#xA;  }&#xA;}&#xA;&#xA;/*&#xA;Advances the simulation a single step.&#xA;&#xA;DOES NOT KEEP TRACK OF STATISTICS. Hence private.&#xA;*/&#xA;void simulation::advance() {&#xA;  // #pragma omp parallel for collapse(2)&#xA;  for (unsigned int row = 0; row &amp;lt; spin_lattice_.get_size(); row++) {&#xA;    for (unsigned int col = 0; col &amp;lt; spin_lattice_.get_size(); col++) {&#xA;      double dE = compute_dE(row, col);&#xA;      double p = r_.random_uniform();&#xA;      // float rnd = rand() / (RAND_MAX + 1.);&#xA;      if (dE &amp;lt;0 || exp(-dE / temperature_) &amp;gt; p) {&#xA;        spin_lattice_.flip(row, col);&#xA;      }&#xA;    }&#xA;  }&#xA;  time_++;&#xA;}&#xA;&#xA;&#xA;&#xA;/*&#xA;Computes the total energy associated with spins in the spin_lattice_.&#xA;&#xA;I originally used this function to test the code that tracked energy as the lattice&#xA;itself was modified, but that code turned out to be only marginally faster, and&#xA;not thread-safe. This is due to a race condition: when one thread uses a neighborhood&#xA;of a point, while another thread was computing the energy of one such point in&#xA;the neighborhood of (row, col).&#xA;*/&#xA;double simulation::compute_energy(lattice &amp;amp;other) {&#xA;  double energy_sum = 0;&#xA;  unsigned int max = other.get_size();&#xA;  #pragma omp parallel for reduction(+ : energy_sum)&#xA;  for (unsigned int i = 0; i &amp;lt; max; i++) {&#xA;    for (unsigned int j = 0; j &amp;lt; max; j++) {&#xA;      energy_sum += other.compute_point_energy(i, j);&#xA;    }&#xA;  }&#xA;  return energy_sum/2;&#xA;}&#xA;&#xA;&#xA;void simulation::set_to_chequerboard(int step){&#xA;  if (time_ !=0){&#xA;    return;&#xA;  }else{&#xA;    for (unsigned int i=0; i&amp;lt; spin_lattice_.get_size(); ++i){&#xA;      for (unsigned int j=0; j&amp;lt;spin_lattice_.get_size(); ++j){&#xA;        if ((i/step)%2-(j/step)%2==0){&#xA;          spin_lattice_.flip(i, j);&#xA;        }&#xA;      }&#xA;    }&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;## investigator.py &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#! /usr/local/bin/python3&#xA;from os import mkdir&#xA;from os.path import exists&#xA;from subprocess import run, PIPE&#xA;from numpy import loadtxt, linspace, log, sqrt, append, exp,  \&#xA;    array, shape, inf, std, mean, argmax&#xA;from scipy.optimize import curve_fit&#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;class Simulation:&#xA;    def __init__(self, magnetic_field=0.0, exchange_energy=1.0,&#xA;                 lattice_size=64, temperature=1.0, duration=50,&#xA;                 grain_size=0):&#xA;        self.magnetic_field = magnetic_field&#xA;        self.exchange_energy = exchange_energy&#xA;        self.lattice_size = lattice_size&#xA;        self.temperature = temperature&#xA;        self.grain_size = grain_size&#xA;        self.duration = duration&#xA;        file_path = 'data/' + str(self) + '.csv'&#xA;        if use_disk:&#xA;            self.data = self.load_from_disk(file_path)&#xA;        else:&#xA;            self.data = self.run()&#xA;&#xA;    def times(self):&#xA;        return self.data[:, 0]&#xA;&#xA;    def mean_magnetizations(self):&#xA;        return self.data[:, 1]&#xA;&#xA;    def mean_energies(self):&#xA;        return self.data[:, 2]&#xA;&#xA;    def load_from_disk(self, file_path):&#xA;        if not exists(path=file_path):&#xA;            self.run(file_path)&#xA;        data = loadtxt(file_path)&#xA;        data_duration = data[-1, 0] + self.duration/200 + 10&#xA;        if self.duration &amp;gt; data_duration:&#xA;            print('duration mismatch, ' + str(self.duration) + '!=' + str(&#xA;                data_duration), end='. re-', flush=True)&#xA;            self.run(file_path)&#xA;            data = loadtxt(file_path)&#xA;        return data&#xA;&#xA;    def run(self, file_path=None):&#xA;        if file_path is None:&#xA;            file_path = 'data/' + str(self) + '.csv'&#xA;        # print('running simulation... ', self.duration, end=' ', flush=True)&#xA;        command = ['./main', '-d', str(self.duration), '-t',&#xA;                   str(self.temperature), '-n', str(self.lattice_size),&#xA;                   '-j', str(self.exchange_energy), '-H',&#xA;                   str(self.magnetic_field), '-c', str(self.grain_size)]&#xA;        if self.duration &amp;gt; 500:&#xA;            command.append('-p')&#xA;            command.append(str(int(self.duration/200)))&#xA;        if use_disk:&#xA;            command.append('-f')&#xA;            command.append(file_path)&#xA;            # print('Done! ')&#xA;            run(command)&#xA;        else:&#xA;            r = run(command, stdout=PIPE)&#xA;            raw_data = loadtxt(r.stdout.decode().split(sep='\n'),&#xA;                               delimiter='\t')&#xA;            # print('Done! ')&#xA;            return raw_data&#xA;&#xA;    def __str__(self):&#xA;        return '(H=' + str(self.magnetic_field) + ')(J=' + str(&#xA;            self.exchange_energy) + ')(T=' + str(&#xA;            self.temperature) + ')(N=' + str(&#xA;            self.lattice_size) + ')' + str(self.grain_size)&#xA;&#xA;# ------------------------------------------------------------------------&#xA;&#xA;def save_plot(title):&#xA;    plt.legend(loc='best')&#xA;    plt.suptitle(title)&#xA;    file_name = title.replace(' ', '_')&#xA;    if not exists('figures/'):&#xA;        mkdir('figures/')&#xA;    plt.savefig('figures/%s.png'%file_name)&#xA;    plt.show()&#xA;&#xA;&#xA;# -------------------------------------------------------------------------&#xA;&#xA;&#xA;def smart_duration(temperature, multiplier=1.):&#xA;    return int(((10**5)*base_duration*multiplier)/(&#xA;                (temperature - t_c)**2 + breadth))&#xA;&#xA;&#xA;&#xA;&#xA;def theoretical_capacity(x, a, b, c, d):&#xA;    return c + (b*x)/((x - a)**2 + d)&#xA;&#xA;&#xA;def investigate_heat_capacity(lattice_sizes=None, temps=None, **kwargs):&#xA;    if temps is None:&#xA;        temps = linspace(1.5, 3, 10)&#xA;    if lattice_sizes is None:&#xA;        lattice_sizes = [16,32, 34, 36]&#xA;&#xA;    fig, axes = plt.subplots(nrows=len(lattice_sizes), sharex='col')&#xA;    crit_temps = []&#xA;    for l, ax in zip(lattice_sizes, axes):&#xA;        print('===',l)&#xA;        crit_temps.append(fit_and_plot_capacity(ax, l, temps, **kwargs))&#xA;&#xA;    fig.set_size_inches(10.5, 10.5)&#xA;    plt.xlabel('temperature / arb. u.')&#xA;    save_plot('Heat capacity')&#xA;    return crit_temps&#xA;&#xA;&#xA;def fit_and_plot_capacity(ax, l, temps, **kwargs):&#xA;    &quot;&quot;&quot;&#xA;    Plot the heat capacity of simulations at given temperature and lattice size.&#xA;    Afterwards fit a lorentzian and plot.&#xA;&#xA;    Parameters:&#xA;    -----------&#xA;    ax : pyplot.axis&#xA;    what to plot to.&#xA;&#xA;    l : int&#xA;    lattice size&#xA;&#xA;    temps: numpy.array&#xA;    Array of temperatures where to evaluate heat capacity.&#xA;&#xA;    Returns:&#xA;    -------&#xA;    popt[0]: float&#xA;    most likely critical temperature.&#xA;    &quot;&quot;&quot;&#xA;    global use_disk&#xA;    use_disk = False&#xA;    simulations = [Simulation(lattice_size=l, temperature=t, duration=2)&#xA;                   for t in temps]&#xA;    # sigmas = [stdev(s.mean_energies[:]) for s in simulations]&#xA;    sigmas = array([multi_run(s, **kwargs) for s in simulations]).T[3]&#xA;    meta_sigmas = array([multi_run(s, **kwargs) for s in simulations]).T[4]&#xA;    # print(meta_sigmas)&#xA;    Cs = [sigma**2/(temp**2)*10**3 for temp, sigma in zip(temps, sigmas)]&#xA;    C_errs = Cs[:]*meta_sigmas[:]&#xA;    try:&#xA;        # popt, pcov = curve_fit(theoretical_capacity, temps*10, Cs,&#xA;                               # sigma=meta_sigmas, bounds=(&#xA;                # [min(temps) - .2, 0, 0, 0],&#xA;                # [max(temps) + .2, inf, inf, .7]))&#xA;        pass&#xA;    except RuntimeError:&#xA;        print('I\'m too dumb to fit')&#xA;        # popt = [temps[argmax(Cs)], (max(Cs) - min(Cs))/4, min(Cs),&#xA;                # (max(temps) - min(temps))/4]&#xA;&#xA;&#xA;    # ax.axvline(x=popt[0], ls='--', color='g')&#xA;    # ax.plot(temps, theoretical_capacity(temps, *popt), 'g-',&#xA;            # label='N = ' + str(l) + 'd = ' + str(popt[3]) + ' fit')&#xA;    ax.errorbar(temps, Cs, fmt='b.', yerr=C_errs, label='N = ' + str(l))&#xA;    ax.set_ylabel(r'C $\cdot 10^3$/ arb. u.')&#xA;    ax.axvline(x=t_c, ls='-.', color='k')&#xA;    ax.set_xticks(&#xA;        append(linspace(min(temps), t_c, 6), linspace(t_c, max(temps), 6)))&#xA;    ax.legend(loc='best')&#xA;    # return popt[0]&#xA;&#xA;&#xA;def multi_run(sim, re_runs:int=1, take_last:int=300):&#xA;    &quot;&quot;&quot;&#xA;    Re run the Ising model simulation multiple times, and gather statistics.&#xA;&#xA;    Parameters:&#xA;    ----------&#xA;    sim: simulation&#xA;    re_runs: int&#xA;    number of times to repeat simulation&#xA;    take_last: int&#xA;    How many of the final points to take statistics over.&#xA;&#xA;    Returns:&#xA;    list:&#xA;&#xA;    &quot;&quot;&quot;&#xA;    global use_disk&#xA;    use_disk = False&#xA;    sim.duration = smart_duration(sim.temperature)&#xA;    print(sim.duration)&#xA;    magnetizations = []&#xA;    sigma_magnetizations = []&#xA;    energies = []&#xA;    sigma_energies = []&#xA;    for i in range(re_runs):&#xA;        sim.data = sim.run()&#xA;        # Make each run take 100% longer, so that we&#xA;        # can see if a system is still settling&#xA;&#xA;        last_magnetizations = sim.mean_magnetizations()[-take_last:]&#xA;        magnetizations.append(abs(mean(last_magnetizations)))&#xA;        sigma_magnetizations.append(std(last_magnetizations))&#xA;&#xA;        last_energies = sim.mean_energies()[-take_last:]&#xA;        energies.append(mean(last_energies))&#xA;        sigma_energies.append(std(last_energies))&#xA;&#xA;    return [mean(magnetizations), mean(sigma_magnetizations),&#xA;            mean(energies), mean(sigma_energies),&#xA;            std(sigma_energies)/mean(sigma_energies)]&#xA;&#xA;&#xA;# -------------------------------------------------------------------------&#xA;&#xA;def finite_size_scale(N, t_inf, a, v):&#xA;    return t_inf + a*(N**(-1/v))&#xA;&#xA;&#xA;def investigate_finite_size_scaling(critical_temperatures, lattice_sizes,&#xA;                                    **kwargs):&#xA;    if critical_temperatures is None:&#xA;        critical_temperatures = investigate_heat_capacity(lattice_sizes,&#xA;                                                          **kwargs)&#xA;    args, cov = curve_fit(finite_size_scale, lattice_sizes,&#xA;                                  critical_temperatures)&#xA;    plt.plot(lattice_sizes, critical_temperatures, 'b+', label='data')&#xA;    plt.plot(lattice_sizes, finite_size_scale(lattice_sizes, *args), 'r-',&#xA;             label='fit')&#xA;    plt.ylabel('critical temperature / arb. u.')&#xA;    plt.xlabel('Lattice size')&#xA;    save_plot('Finite size scaling')&#xA;    return args[0], sqrt(cov[0, 0])&#xA;&#xA;&#xA;# -------------------------------------------------------------------------&#xA;&#xA;t_c = 2/log(1 + sqrt(2))&#xA;use_disk = False&#xA;breadth = 1&#xA;base_duration = 10&#xA;sizes = [16, 32]&#xA;# investigate_time_evolution()&#xA;# investigate_temperature_dependence(lattice_sizes=sizes, re_runs=4)&#xA;&#xA;critical_temps = investigate_heat_capacity(lattice_sizes=sizes,&#xA;                                           take_last=10000)&#xA;&#xA;# temp_inf = investigate_finite_size_scaling(critical_temps, sizes)&#xA;&#xA;#  print(temp_inf)&#xA;# print((t_c - temp_inf[0])/ temp_inf[1], ' Standard errors away')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the heat capacity, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bQlOs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bQlOs.png&quot; alt=&quot;heat cap&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;a href=&quot;https://i.stack.imgur.com/PuIT8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PuIT8.png&quot; alt=&quot;mean magnetisation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27418" LastEditorUserId="27418" LastEditDate="2018-04-09T22:59:03.930" LastActivityDate="2019-08-21T09:51:04.417" Title="2D Ising Model, heat capacity decreases with lattice size" Tags="|python|c++|monte-carlo|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29261" PostTypeId="2" ParentId="21062" CreationDate="2018-04-09T03:01:56.270" Score="2" Body="&lt;p&gt;If you decide to go with a direct solver, you may consider the following options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Intel MKL Pardiso, which you have already tried&lt;/li&gt;&#xA;&lt;li&gt;MUMPS, that is another very popular choice for parallel direct solvers and has an out-of-core capability&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.hsl.rl.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HSL&lt;/a&gt;, free for academic use, consider HSL_MA78, sparse out-of-core solver&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Several years ago, I added to my collection the following paper that compares the performance of those three libraries for Navier-Stokes FEM: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/376e/18e903f39532fb9976ec4e9e95160da251c6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mandhapati P. Raju and Siddhartha Khaitan, &quot;High Performance Computing Using Out-of-Core Sparse Direct Solvers&quot;, &lt;em&gt;Int. J. Math. Computat. Phys. Electr. Comput. Eng.&lt;/em&gt;, vol. 3, no. 9, Sep. 2009&lt;/a&gt;. This is a questionable publisher &lt;a href=&quot;https://en.wikipedia.org/wiki/World_Academy_of_Science,_Engineering_and_Technology&quot; rel=&quot;nofollow noreferrer&quot;&gt;WASET&lt;/a&gt;, so take this paper with an extra grain of salt; however, I haven't found anything criminal with that particular paper myself.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-04-09T03:26:28.443" LastActivityDate="2018-04-09T03:26:28.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29265" PostTypeId="1" AcceptedAnswerId="29266" CreationDate="2018-04-09T16:04:13.700" Score="0" ViewCount="288" Body="&lt;p&gt;so hopefully I'll be using &lt;code&gt;gmsh&lt;/code&gt; to make meshes out of 2-D cross sections with o thickness. I tried to make a structured mesh with quad elements of a rectangular cross-section, but when I checked the .mesh file to see if it had the correct number of elements and nodes, it hadn't and I can't seem to figure out why... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll post the .geo and .mesh files here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;.geo file:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;`// Gmsh project created on Mon Apr 09 15:44:32 2018&#xA;Point(1) = {0, 0, 0, 1};&#xA;Point(2) = {6, 0, 0, 1};&#xA;Point(3) = {6, 2, 0, 1};&#xA;Point(4) = {0, 2, 0, 1};&#xA;Line(1) = {1, 2};&#xA;Line(2) = {2, 3};&#xA;Line(3) = {3, 4};&#xA;Line(4) = {4, 1};&#xA;Physical Line(&quot;left&quot;) = {4};&#xA;Physical Line(&quot;right&quot;) = {2};&#xA;Physical Line(&quot;top&quot;) = {3};&#xA;Physical Line(&quot;bottom&quot;) = {1};&#xA;Line Loop(1) = {4, 1, 2, 3};&#xA;Plane Surface(1) = {1};&#xA;Transfinite Surface {1} = {1, 2, 3, 4};&#xA;Transfinite Line {4, 2} = 3 Using Progression 1;&#xA;Transfinite Line {1, 3} = 7 Using Progression 1;&#xA;Recombine Surface {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;.mesh file:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$MeshFormat&#xA;2.2 0 8&#xA;$EndMeshFormat&#xA;$PhysicalNames&#xA;4&#xA;1 1 &quot;left&quot;&#xA;1 2 &quot;right&quot;&#xA;1 3 &quot;top&quot;&#xA;1 4 &quot;bottom&quot;&#xA;$EndPhysicalNames&#xA;$Nodes&#xA;16&#xA;1 0 0 0&#xA;2 6 0 0&#xA;3 6 2 0&#xA;4 0 2 0&#xA;5 0.999999999995645 0 0&#xA;6 1.99999999999205 0 0&#xA;7 2.999999999988572 0 0&#xA;8 3.999999999992324 0 0&#xA;9 4.999999999996161 0 0&#xA;10 6 0.9999999999973842 0&#xA;11 5.000000000006331 2 0&#xA;12 4.000000000003875 2 0&#xA;13 3.000000000008324 2 0&#xA;14 2.000000000005607 2 0&#xA;15 1.000000000002804 2 0&#xA;16 0 1.000000000004119 0&#xA;$EndNodes&#xA;$Elements&#xA;16&#xA;1 1 2 4 1 1 5&#xA;2 1 2 4 1 5 6&#xA;3 1 2 4 1 6 7&#xA;4 1 2 4 1 7 8&#xA;5 1 2 4 1 8 9&#xA;6 1 2 4 1 9 2&#xA;7 1 2 2 2 2 10&#xA;8 1 2 2 2 10 3&#xA;9 1 2 3 3 3 11&#xA;10 1 2 3 3 11 12&#xA;11 1 2 3 3 12 13&#xA;12 1 2 3 3 13 14&#xA;13 1 2 3 3 14 15&#xA;14 1 2 3 3 15 4&#xA;15 1 2 1 4 4 16&#xA;16 1 2 1 4 16 1&#xA;$EndElements&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;`&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was supposed to get 12 elements and 21 nodes, but I'm getting 16 elements and 16 nodes... Would be really grateful if anyone can help me with this.&#xA;Thanks in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I just need a quad-only mesh generator, if you know any open source one besides gmsh let me know!&lt;/p&gt;&#xA;" OwnerUserId="27425" LastEditorUserId="20688" LastEditDate="2018-04-09T17:45:26.413" LastActivityDate="2018-04-09T17:45:26.413" Title="Gmsh exporting wrong mesh DATA" Tags="|finite-element|computational-geometry|mesh|mesh-generation|gmsh|" AnswerCount="1" CommentCount="0" ClosedDate="2018-04-13T15:33:56.363" ContentLicense="CC BY-SA 3.0" />
  <row Id="29266" PostTypeId="2" ParentId="29265" CreationDate="2018-04-09T16:23:31.137" Score="1" Body="&lt;p&gt;You have specified the wrong things to be &lt;code&gt;Physical&lt;/code&gt;. If you just want to obtain 2-D elements, there is no need to specify the lines to be physical. However, you certainly need to specify your surface to be physical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you may want to specify the #points in the structured mesh for transfinite lines before announcing the &lt;code&gt;Surface&lt;/code&gt; to be transfinite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, see the corrected code attached that produces exactly 12 2-D elements and 21 nodes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Point(1) = {0, 0, 0, 1};&#xA;Point(2) = {6, 0, 0, 1};&#xA;Point(3) = {6, 2, 0, 1};&#xA;Point(4) = {0, 2, 0, 1};&#xA;Line(1) = {1, 2};&#xA;Line(2) = {2, 3};&#xA;Line(3) = {3, 4};&#xA;Line(4) = {4, 1};&#xA;Line Loop(1) = {4, 1, 2, 3};&#xA;Plane Surface(1) = {1};&#xA;Transfinite Line {4, 2} = 3 Using Progression 1;&#xA;Transfinite Line {1, 3} = 7 Using Progression 1;&#xA;Transfinite Surface {1} = {1, 2, 3, 4};&#xA;Recombine Surface {1};&#xA;Physical Surface(1) = {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-04-09T16:31:51.270" LastActivityDate="2018-04-09T16:31:51.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29267" PostTypeId="1" CreationDate="2018-04-10T06:24:02.493" Score="2" ViewCount="62" Body="&lt;p&gt;I have a probability distribution function that I don't have its analytical form (so I can't determine its CDF). How can I generate random numbers based on this distribution function? I'm looking for a way that is similar to that of using CDF. &lt;/p&gt;&#xA;" OwnerUserId="27358" LastActivityDate="2018-04-10T07:33:28.217" Title="Generating a random number based on a numerical distribution function" Tags="|monte-carlo|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29268" PostTypeId="2" ParentId="29267" CreationDate="2018-04-10T07:11:34.297" Score="1" Body="&lt;p&gt;That is exactly what the Monte Carlo method is designed to do. One example of a method to draw samples from your distribution is the Metropolis-Hastings sampler.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-10T07:11:34.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29269" PostTypeId="2" ParentId="29267" CreationDate="2018-04-10T07:33:28.217" Score="2" Body="&lt;p&gt;Your goal is to obtain the inverse function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the probability distribution $\rho(x)$ you are looking for the point $y$ such that the cumulative probability up to this point coincides with the result of a uniform distribution $U$  (you are converting a uniform distribution into your own distribution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore you have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{-\infty}^y \rho(x)\,dx=U$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value $y$ can be obtained iteratively, for example, using Newton Raphson method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y^{n+1}=y^n-\frac{\int_{-\infty}^{y^n} \rho(x)\,dx-U}{\rho(y^n)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting $y$ series will be arranged with a probability according to your distribution given the pair $(U,y)$ obtained from each uniform evaluation $U\rightarrow y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is more, you must solve the integrals with numerical methods since you have no analytic expression for the density.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-04-10T07:33:28.217" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29270" PostTypeId="2" ParentId="29260" CreationDate="2018-04-10T09:24:09.563" Score="1" Body="&lt;p&gt;It looks like you are computing the specific heat by using the fluctuations of the energy per site rather than the total energy of the system. If you multiply the y axes of your plots by $N^2$ you should recover the expected behaviour.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, you should not take into account &quot;self energies&quot; (that is, $e(i,i)$) in the &lt;code&gt;compute_energy&lt;/code&gt; method.&lt;/p&gt;&#xA;" OwnerUserId="27040" LastEditorUserId="27040" LastEditDate="2018-04-10T12:37:57.843" LastActivityDate="2018-04-10T12:37:57.843" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29271" PostTypeId="2" ParentId="7525" CreationDate="2018-04-10T09:54:49.210" Score="2" Body="&lt;p&gt;In the meantime, I created&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/nschloe/pygmsh&quot; rel=&quot;nofollow noreferrer&quot;&gt;pygmsh&lt;/a&gt; as a Python front end for gmsh and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/nschloe/pygalmesh&quot; rel=&quot;nofollow noreferrer&quot;&gt;pygalmesh&lt;/a&gt; as a Python front end for CGAL's 3D meshing capabilities.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both are fairly well received, so I'm guessing they are helpful to others as well.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2018-04-10T09:54:49.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29274" PostTypeId="1" CreationDate="2018-04-10T17:54:52.467" Score="3" ViewCount="134" Body="&lt;p&gt;Suppose we want to find $x$ such that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x=b+Ax$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is a large sparse square matrix with eigenvalues in the unit circle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two representations of the solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)&#xA;$$x=(I-A)^{-1}b,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2)&#xA;$$x=\sum_{k=0}^\infty{A^k b}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since we cannot practically store powers of the matrix $A$ (due to memory limits: indeed in practice, we just have a function that computes $Ax$ given $x$), using the representation 2) means evaluating the recurrence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_t=b+A x_{t-1}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since $A$ may have eigenvalues close to $1$ in magnitude, this will converge slowly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a multivariate series acceleration method which would handle such recursions? (E.g. a multivariate version of the Aitken &quot;delta-squared&quot; process/Steffensen's method.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is there a technique for solving 1) directly that is guaranteed to have good performance? Do our assumptions on $A$ imply anything about the performance of GMRES / CGS / BICGSTAB(L) / QMR etc.?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note: This question: &quot;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/18815/is-there-a-faster-method-to-compute-the-geometric-series-of-a-matrix&quot;&gt;Is there a faster method to compute the geometric series of a matrix?&lt;/a&gt;&quot; is similar but seems to be interested in finite sums of small dense matrices, rather than infinite sums of large sparse matrices.&lt;/p&gt;&#xA;" OwnerUserId="27429" LastEditorUserId="27429" LastEditDate="2018-04-10T18:10:50.527" LastActivityDate="2018-04-10T18:10:50.527" Title="Acceleration of matrix geometric series" Tags="|linear-algebra|matrix|linear-solver|sparse-matrix|acceleration|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29275" PostTypeId="1" AcceptedAnswerId="29276" CreationDate="2018-04-10T18:15:03.647" Score="2" ViewCount="126" Body="&lt;p&gt;If you solve a given PDE (Navier stoke's, Euler, heat eqn, advection eqn, etc...) using FVM, is this PDE supposed to be valid at every cell in the discretized domain, or only in the global domain as a whole? &lt;/p&gt;&#xA;" OwnerUserId="27430" LastActivityDate="2018-04-10T21:12:10.903" Title="Does the PDE hold at every cell in a FVM mesh?" Tags="|pde|finite-volume|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29276" PostTypeId="2" ParentId="29275" CreationDate="2018-04-10T21:12:10.903" Score="4" Body="&lt;p&gt;As with many questions that are more about philosophy than science, there are at least three different ways of answering your question, each with a fairly solid argument:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Given that the finite volume method requires discretising the problem, there is nowhere where the exact continuum PDE holds.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;However, we do create one equation representing a discretised version of the original PDE for each cell, so in this sense there is an equation &quot;valid&quot; in each cell.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Unless your original system is very simple, or you make some very unusual choices, then the final system will couple the degrees of freedom representing each cell. It's in this sense, when attempting to solve the problem, that there are global solutions only.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Lets work through producing a finite difference method and see when each bit kicks in. Since we're choosing to use the FVM, I'm going to assume that the original PDE looks like $$\frac{\partial \tau }{\partial t} +\nabla\cdot \mathbf{F}(\tau) = 0.$$ Fortunately all the equations you ask about can be written in this conservative form. Now &lt;strong&gt;for each cell&lt;/strong&gt; we write an integral equation, $$\int_{e_i}\frac{\partial \tau }{\partial t} +\nabla\cdot \mathbf{F}(\tau)d\mathbf{x} = 0.$$ Integrating by parts, and pulling the differentiation outside the integral gives  $$\frac{d \tau }{dt}  \int_{e_i}\tau d\mathbf{x} =\int_{\delta e_i} \mathbf{F}(\tau)\cdot d\mathbf{S}.$$&#xA;This is the last time we have an (integral) equation morally the same as the original PDE,(q.v. answer [1]).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this point onwards we choose to express the integral of the flux, $$\int_{f(e_i,e_j)} \mathbf{F}(\tau)\cdot d\mathbf{S},$$ directed through a facet, $f(e_i,e_j)$, bordering cells $e_i$ and $e_j$, in terms of our cell integrals, $\int_{e_i}\tau d\mathbf{x}$ and $\int_{e_j}\tau d\mathbf{x}$. Now the equation has been discretised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we choose (weirdly) to have terms in $f(e_i,e_j)$ only given as functions of $\int_{e_i}\tau d\mathbf{x}$, then the individual degrees of freedom are uncoupled, and we just have one independently soluble equation for each of the $\int_{e_i}\tau d\mathbf{x}$ (strongly suggesting answer [2] for your original question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this means giving up local conservation of $\tau$, one of the desirable properties of the FVM, which needs terms in $f(e_i,e_j)$ and $f(e_j,e_i)$ to match. Once we do this, the variables are coupled, and we can only find a solution by attacking the global solution. At this point we could argue for answer [3] instead.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2018-04-10T21:12:10.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29277" PostTypeId="1" AcceptedAnswerId="29281" CreationDate="2018-04-11T03:23:11.103" Score="2" ViewCount="594" Body="&lt;p&gt;I simulated the compression problem in ANSYS and compared to the analytical solution and found some discrepancies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classical solution to the 1-D compression problem is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;u(x) = Cx &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;subjecting to the following BCs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;u(x=0) = 0 \\&#xA;\frac{du}{dx}(x=L) = C&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solution is derived for 1-D in the absence of any shear phenomena. &#xA;For a 10 m long rod subject to a compressive load of $\sigma$ at $x=L$, we can use the stress relationship to determine $\frac{du}{dx}$. In 1-D, we can write the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\sigma = (2\mu+\lambda)\frac{du}{dx}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{du}{dx} = \frac{\sigma}{2\mu+\lambda} =C&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the purpose of comparing this analytical solution to ANSYS, I will give some numbers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\mu =    7.6923 \times 10^{10} \text{ Pa}\\&#xA;\lambda =    1.153 \times10^{11} \text{ Pa}\\&#xA;\sigma = 10^6 \text{ Pa} \\&#xA;L = 10 \text{ m}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analytical solution gives us &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;u(x=L) = 3.7143\times 10^{-5} \text{ m}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I simulate an analogous problem in ANSYS of a 10m rod with a 1m by 1m cross section subjected to the same load, I obtain a maximum displacement of $u(x=L) = 5 \times 10^{-5}\text{ m}$. The full 3-D equations are solved in ANSYS, so we have shear effects, displacements in other 2 directions, and intercomponent coupling among the displacement components. Are these the reasons why the solution is so different? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;EDIT&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In response to KNL's comment:&#xA;So I understand that using Hooke's law, and the stress-strain relationship for small displacements, we get the following in 3-D:&#xA;\begin{align}&#xA;\boldsymbol{\sigma}=&#xA;\begin{bmatrix}&#xA;\sigma_{xx} &amp;amp; \sigma_{xy} &amp;amp; \sigma_{xz} \\&#xA;\sigma_{yx} &amp;amp; \sigma_{yy} &amp;amp; \sigma_{yz} \\&#xA;\sigma_{zx} &amp;amp; \sigma_{zy} &amp;amp; \sigma_{zz} \\&#xA;\end{bmatrix}&#xA; = \mu&#xA;\begin{bmatrix}&#xA;\frac{du}{dx} &amp;amp; \frac{du}{dy} &amp;amp; \frac{du}{dz} \\&#xA;\frac{dv}{dx} &amp;amp; \frac{dv}{dy} &amp;amp; \frac{dv}{dz} \\&#xA;\frac{dw}{dx} &amp;amp; \frac{dw}{dy} &amp;amp; \frac{dw}{dz} \\&#xA;\end{bmatrix}&#xA;+ \mu \begin{bmatrix}&#xA;\frac{du}{dx} &amp;amp; \frac{du}{dy} &amp;amp; \frac{du}{dz} \\&#xA;\frac{dv}{dx} &amp;amp; \frac{dv}{dy} &amp;amp; \frac{dv}{dz} \\&#xA;\frac{dw}{dx} &amp;amp; \frac{dw}{dy} &amp;amp; \frac{dw}{dz} \\&#xA;\end{bmatrix}^T&#xA;+ \lambda&#xA;(\frac{du}{dx} + \frac{dv}{dy} + \frac{dw}{dz}) \boldsymbol{I}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In 1-D, and absent of shear terms, we get:&#xA;\begin{align}&#xA;\sigma_{xx} = (2\mu+\lambda)\frac{du}{dx} &#xA;\end{align}&#xA;Which is the relationship that I originally used, but I don't understand what is wrong? &lt;/p&gt;&#xA;" OwnerUserId="27430" LastEditorUserId="27430" LastEditDate="2018-04-11T13:44:33.677" LastActivityDate="2018-04-12T20:01:16.240" Title="1D analytical solution vs FEM solution for a bar under compression" Tags="|finite-element|solid-mechanics|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="29278" PostTypeId="1" CreationDate="2018-04-11T09:31:21.443" Score="2" ViewCount="88" Body="&lt;p&gt;I am working on a problem that involves semidefinite programming (constrained optimization of fairly large positive definite matrices). The software is written in C++ and calls &lt;a href=&quot;http://www.mcs.anl.gov/hs/software/DSDP/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSDP 5.8&lt;/a&gt; to solve the SDP problem. It works as advertised on small/medium test problems (up to 200 x 200 matrices, ~$10^5$ constraints), but dies due to indexing errors on larger tasks. The reason is, of course, that the authors of &lt;code&gt;DSDP&lt;/code&gt; used &lt;code&gt;int&lt;/code&gt; variables for indices (and &lt;code&gt;int[]&lt;/code&gt;-s for indexing arrays) instead of e.g. &lt;code&gt;size_t&lt;/code&gt; or &lt;code&gt;long&lt;/code&gt; and with the larger problems the 4 byte &lt;code&gt;int&lt;/code&gt;-s eventually overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before I start modifying a C source that has not been updated since 2006, I thought I might ask the community here: does anyone have, by any chance, a modified version of &lt;code&gt;DSDP&lt;/code&gt; that uses &lt;code&gt;size_t&lt;/code&gt; or &lt;code&gt;long&lt;/code&gt; for indices (and &lt;code&gt;size_t[]&lt;/code&gt; for the index arrays)? These types are usually 8 bytes long on modern 64-bit platforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My other option would be to swap out &lt;code&gt;DSDP&lt;/code&gt; and use another SDP solver instead. Please understand that I am reluctant to do this because I would have to adapt my software to a new API. Having said that, feel free to recommend open-source SDP solvers for first-order cone programming that can deal with large problems. I am aware of &lt;a href=&quot;http://cvxopt.org/userguide/coneprog.html#semidefinite-programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVXOPT&lt;/a&gt; but that is written in Python; and &lt;a href=&quot;https://www.mosek.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;MOSEK&lt;/a&gt; is not FOSS as far as I can tell.&lt;/p&gt;&#xA;" OwnerUserId="7024" LastEditorUserId="7024" LastEditDate="2018-04-12T13:19:22.940" LastActivityDate="2018-04-12T13:19:22.940" Title="Looking for a version of DSDP that is less prone to integer overflows than the original" Tags="|c++|c|semidefinite-programming|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29281" PostTypeId="2" ParentId="29277" CreationDate="2018-04-11T14:48:31.350" Score="4" Body="&lt;p&gt;It is a matter of boundary conditions on the longitudinal faces.&#xA;As you noted, the axial stress and strain for a linear isotropic material will satisfy the following relation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\sigma_{xx} = \lambda(e_x + e_y + e_z) + 2\mu e_x.&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you assume zero transverse strains (for example in the case of sliding contact boundary conditions), you will obtain the result that you did, $\sigma_{xx}=(2\mu+\lambda)e_x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the more commonly encountered case of free boundaries, the Poisson effect comes into play.&#xA;Just substitute $e_y = e_z = -\nu e_x = -\frac{\lambda}{2(\lambda+\mu)}e_x$ in the above relation, to obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\sigma_{xx} = \frac{\mu(3\lambda+2\mu)}{\lambda+\mu}e_x = Ee_x&#xA;\end{equation}&lt;/p&gt;&#xA;" OwnerUserId="6775" LastEditorUserId="6775" LastEditDate="2018-04-12T20:01:16.240" LastActivityDate="2018-04-12T20:01:16.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29283" PostTypeId="1" AcceptedAnswerId="29298" CreationDate="2018-04-11T23:42:05.053" Score="8" ViewCount="520" Body="&lt;p&gt;I recently came across &lt;a href=&quot;https://dl.acm.org/citation.cfm?id=1366244&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt;, which describes a scheme for saving memory in representing sparse matrices. A 32-bit integer can store numbers up to ~4 billion. But when you solve a PDE you've gone parallel and partitioned the problem long before the number of unknowns is anywhere close to 4 billion. Their idea is to reorder the matrix for small bandwidth. Rather than store the indices &lt;code&gt;j&lt;/code&gt; of all non-zero columns in a given row &lt;code&gt;i&lt;/code&gt;, they store the offset &lt;code&gt;j - i&lt;/code&gt;, which tends to be small in magnitude by virtue of the reordering. The offsets can then be stored using fewer bits than a 32-bit integer would. There's more arithmetic to do when iterating over the non-zero entries of the matrix, but the savings in fewer cache misses more than make up for it. In &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/5715269/&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt; they were looking specifically at using 16-bit indices in a hierarchical matrix format, but ultimately it's a similar idea. There's also the library &lt;a href=&quot;https://github.com/LLNL/zfp&quot; rel=&quot;noreferrer&quot;&gt;zfp&lt;/a&gt;, which is more for compressing floating-point data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since &quot;flops are free&quot; now and the bottleneck is memory access, these kinds of tricks seem really promising for making better use of CPU cache.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've scoured most of the cited/citing works of these two papers. &lt;strong&gt;I'm looking for any other references on the effectiveness of bit-packing, for sparse matrix-vector multiplication and for any other problems in computational science&lt;/strong&gt;. For example, I imagine you could design much more efficient data structures for graphs or for unstructured meshes using this idea.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-04-13T22:19:47.560" Title="bit-packing and compression of data structures in scientific computing" Tags="|sparse-matrix|data-structures|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29286" PostTypeId="1" CreationDate="2018-04-12T16:57:31.180" Score="3" ViewCount="119" Body="&lt;p&gt;Given square matrix $A_0$, vector $b$, vector $A_0^{-1}b$ and matrices $A_1, A_2, \dots, A_k$, in which each $A_i$ is generated from $A_{i-1}$ by replacing one single column, I would like to find an efficient method to compute all the results of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A_1^{-1}b, A_2^{-1}b, \dots, A_k^{-1}b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions are welcome.&lt;/p&gt;&#xA;" OwnerUserId="2131" LastEditorUserId="20417" LastEditDate="2018-04-12T17:55:08.143" LastActivityDate="2018-04-12T20:26:49.167" Title="Solve $A^{-1} b$ when one column is replaced" Tags="|linear-algebra|algorithms|matrix|linear-solver|efficiency|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29287" PostTypeId="1" AcceptedAnswerId="29290" CreationDate="2018-04-12T19:38:32.530" Score="7" ViewCount="2042" Body="&lt;p&gt;I need to find the first $n$ roots of the transcendental equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;F(k) = J_m'(kr)Y_m'(k)-J'_m(k)Y'_m(kr)&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for integer values of $m$ and any $r \in [0,1)$ where $J'$ and $Y'$ are derivatives of Bessel function of first and second kind.  This is a (standard?) problem encountered in eigenfunction expansions for annular cylinders. Using the recurrence properties of the Bessel functions we can rewrite this equation without the derivatives, as shown in the code below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the roots I'm using standard root finding algorithm &lt;code&gt;fsolve&lt;/code&gt; in python which requires initial guesses. I have gathered that a decent good guess for the first root is going to be $m$, the order of the Bessel function we want to find roots for. From the plot I also gather that the roots are somewhat evenly spaced so that after I find the second root I can find the other guesses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The spacing of the roots is nonlinear in $r$. Im sure there is a robust method out there that can help with the root guessing, or maybe there is a better way to go about doing this. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.special import jn, yn &#xA;from scipy.optimize import fsolve&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;from scipy.signal import argrelextrema&#xA;&#xA;&#xA;# Recurrence relations to relate derivative of bessels to normal bessel&#xA;Jp = lambda m,x : 0.5*(jn(m-1,x)-jn(m+1,x))&#xA;Yp = lambda m,x : 0.5*(yn(m-1,x)-yn(m+1,x))&#xA;&#xA;&#xA;# r is the ratio of inner/ outer radii and is [0,1)&#xA;F = lambda k,m,r : Jp(m,k*r)*Yp(m,k)-Jp(m,k)*Yp(m,k*r)&#xA;&#xA;plt.figure()&#xA;k_array = np.linspace(0.1,80,5000)&#xA;m = 5&#xA;&#xA;r = 0.9&#xA;&#xA;F_array = F(k_array,m,r)&#xA;&#xA;&#xA;&#xA;guesses = np.asarray([m,m/(1-r)])&#xA;&#xA;roots = fsolve(F,guesses,args=(m,r,))&#xA;#iroots = argrelextrema(F_array**2,np.less)[0]&#xA;#print iroots&#xA;&#xA;&#xA;&#xA;&#xA;plt.plot(k_array,F_array)&#xA;#plt.plot(k_array[iroots],np.zeros(len(iroots)),&quot;ro&quot;)&#xA;plt.plot(roots,np.zeros(len(roots)),&quot;ro&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7864" LastEditorUserId="127" LastEditDate="2018-04-13T06:56:41.877" LastActivityDate="2018-04-13T15:27:50.533" Title="Finding the first N roots of transcendental equation" Tags="|optimization|python|special-functions|rootfinding|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29288" PostTypeId="2" ParentId="29287" CreationDate="2018-04-12T20:10:06.457" Score="11" Body="&lt;p&gt;This is a root-finding problem for an analytic function over a single dimension. One standard technique is to approximate $F(k)$ as a polynomial $F(k) \approx c_0 + c_1 k + c_2 k^2 + \cdots$ using a Chebyshev approximation, and to compute the roots of the polynomial semi-analytically, e.g. by setting up the companion matrix and computing its eigenvalues. (Given that $F(k)$ is analytic, it is well-approximated by a polynomial, and the convergence of a Chebyshev polynomial approximation is exponential.) The roots found using this process can be optionally refined using Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above algorithm is implemented in the MATLAB package Chebfun. The reference on this algorithm is due to John P. Boyd.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Boyd, John P. &quot;Computing zeros on a real interval through Chebyshev expansion and polynomial rootfinding.&quot; &lt;em&gt;SIAM Journal on Numerical Analysis&lt;/em&gt; 40.5 (2002): 1666-1682.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="11716" LastActivityDate="2018-04-12T20:10:06.457" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29289" PostTypeId="2" ParentId="29286" CreationDate="2018-04-12T20:20:44.820" Score="6" Body="&lt;p&gt;This can be done using a technique called the $\eta$ (eta) factorization.  This method is commonly used in implementations of the simplex method for linear programming and can be found in many textbooks on linear programming.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The procedure is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find an LU factorization of $A_{1}$, &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$PA_{1}=LU$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this factorization, you can easily solve $A_{1}x=b$.  &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Suppose that $A_{2}$ is constructed from $A_{1}$ by replacing column &#xA;$p$ of $A_{1}$ by a vector $v$.  Then we can write &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$A_{2}=A_{1}E_{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $E_{2}$ is a so-called $\eta$ matrix that is an identity matrix with column $p$ replaced by the solution to $A_{1}u=v$.  When we multiply $A_{1}$ times the columns of $E_{2}$ taken from $I$, we get the original $A_{1}$ columns copied into $A_{2}$.  When we multiply $A_{1}$ times $u$ in the $p$th column of $E_{2}$, we get the desired vector $v$. &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;To solve $A_{2}x=b$, we use the following procedure.  First, solve $A_{1}w=b$ using the original LU factorization of $A_{1}$.  Then, solve $E_{2}x=w$ by first solving for $x_{p}=w_{p}/u_{p}$ and then using back substitution to solve for the remaining entries in $x$.  Thus &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$A_{2}x=A_{1}E_{2}x=A_{1}w=b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This process is continued to get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{k}=A_{1}E_{2}E_{3}E_{4}\cdots E_{k}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The worst-case computational complexity of the procedure is as follows. Computing the LU factorization of $A_{1}$ takes $O(n^{3})$ time.  Computing an $\eta$ matrix takes $O(n^{2})$ time.  The solution of the system $A_{1}w=b$ takes $O(n^{2}$ time.  The solution of a system $E_{j}v_{j}=w_{j-1}$ takes $O(n)$ time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the matrix is usually refactorized once the number of $\eta$ matrices hits a configurable limit.  In linear programming, refactorizing the basis matrix after 30 $\eta$ updates is a common rule of thumb.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this method is commonly employed with a direct factorization method to solve $A_{1}x=b$, you could apply the same approach with an iterative method for solving $A_{1}x=b$.  Finding each $\eta$ matrix $E_{k}$ would require a full iterative solution plus $k-1$ $\eta$ updates.  The solution of $A_{k}x=b$ would require one full iterative solution, plus $k$ $\eta$ updates, each of which takes only $O(n)$ time.  This really wouldn't save any computational effort compared to solving $A_{k}x=b$ using the iterative method.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2018-04-12T20:26:49.167" LastActivityDate="2018-04-12T20:26:49.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29290" PostTypeId="2" ParentId="29287" CreationDate="2018-04-12T20:47:29.167" Score="8" Body="&lt;p&gt;I will complement @Richard Zhang 's answer (+1) with a python implementation of his suggested approach. The MATLAB package &lt;code&gt;Chebfun&lt;/code&gt; has been partially ported in &lt;code&gt;python&lt;/code&gt;. Actually there are two versions available: &lt;a href=&quot;https://github.com/chebpy/chebpy&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;chebpy&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://github.com/pychebfun/pychebfun&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;pychebfun&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an implementation of the root finding procedure with &lt;code&gt;pychebfun&lt;/code&gt; (the approach is similar with &lt;code&gt;chebpy&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.special import jn, yn &#xA;import pychebfun&#xA;&#xA;# Recurrence relations to relate derivative of bessels to normal bessel&#xA;Jp = lambda m,x : 0.5*(jn(m-1,x)-jn(m+1,x))&#xA;Yp = lambda m,x : 0.5*(yn(m-1,x)-yn(m+1,x))&#xA;&#xA;&#xA;# r is the ratio of inner/ outer radii and is [0,1)&#xA;F = lambda k,m,r : Jp(m,k*r)*Yp(m,k)-Jp(m,k)*Yp(m,k*r)&#xA;&#xA;&#xA;m, r = 5, 0.9&#xA;&#xA;# obtain the chebyshev approximation of the function for a specific domain &#xA;f_cheb = pychebfun.Chebfun.from_function(lambda x: F(x, m, r), domain = (0.1,80))&#xA;&#xA;# obtain the roots of the function in this domain&#xA;roots = f_cheb.roots()&#xA;&#xA;&#xA;#plot function and roots&#xA;k_array = np.linspace(0.1,80,5000)&#xA;plt.plot(k_array, F(k_array, m, r), label = '$F$')&#xA;plt.plot(f_cheb.roots(), F(f_cheb.roots(), m, r), 'o', label = 'roots')&#xA;plt.ylim(ymin=-.1, ymax = 0.1)&#xA;plt.legend()&#xA;plt.grid()&#xA;plt.xlabel('$k$');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BI38M.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BI38M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21460" LastActivityDate="2018-04-12T20:47:29.167" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29291" PostTypeId="2" ParentId="29260" CreationDate="2018-04-12T21:22:40.427" Score="1" Body="&lt;p&gt;I have done this simulation before. I think you can solve your problem by first increasing the number averaging (Monte-Carlo method) and then increasing the size of the lattice. Since when you are doing in small sizes, the model feels the effect of size.&#xA;You can take a look at the following paper:&#xA;&lt;a href=&quot;http://iopscience.iop.org/article/10.1088/1742-5468/2009/07/P07030/pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://iopscience.iop.org/article/10.1088/1742-5468/2009/07/P07030/pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which talks about the effects of size.&lt;/p&gt;&#xA;" OwnerUserId="27358" LastActivityDate="2018-04-12T21:22:40.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29292" PostTypeId="2" ParentId="29287" CreationDate="2018-04-13T06:14:54.960" Score="5" Body="&lt;p&gt;Complementing the answers, I would like to say that directly discretizing the differential equation might work really well. I used that approach in &lt;a href=&quot;https://scicomp.stackexchange.com/q/24338/9667&quot;&gt;a previous question&lt;/a&gt;. I used Finite Differences and used an eigensolver to find the roots. That, besides a perturbation analysis followed by Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The suggestion there was to use Chebyshev approximation, as well.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-04-13T15:27:50.533" LastActivityDate="2018-04-13T15:27:50.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29293" PostTypeId="1" AcceptedAnswerId="29294" CreationDate="2018-04-13T16:02:30.127" Score="1" ViewCount="180" Body="&lt;p&gt;Suppose I have a reference quadrilateral on $[-1, -1] \times [-1, 1]$ with reference coordinates $\xi, \eta$ and a mapping to an isoparametric quadrilateral in 'physical space' described by coordinates $x, y$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a triangular element, the isoparametric mapping is handled the following way for the global nodal basis functions $\phi_i$ in 'physical space':&#xA;\begin{align}&#xA;\begin{bmatrix}&#xA; \frac{\partial \phi_i}{\partial x} \\  \frac{\partial \phi_i}{\partial y}&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA; \frac{\partial \xi}{\partial x} &amp;amp; \frac{\partial \eta}{\partial x} \\ &#xA; \frac{\partial \xi}{\partial y} &amp;amp; \frac{\partial \eta}{\partial y} \\   &#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA; \frac{\partial \phi_i}{\partial \xi} \\  \frac{\partial \phi_i}{\partial \eta}&#xA;\end{bmatrix}&#xA;= J^{-1}&#xA;\begin{bmatrix}&#xA; \frac{\partial \phi_i}{\partial \xi} \\  \frac{\partial \phi_i}{\partial \eta}&#xA;\end{bmatrix}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Via the chain rule, so that I can calculate&#xA;\begin{align}&#xA; \frac{\partial \phi_i}{\partial x} &amp;amp;= \frac{\partial \phi_i}{\partial \xi}J^{-1}_{11} + \frac{\partial \phi_i}{\partial \eta}J^{-1}_{12} \\&#xA; \frac{\partial \phi_i}{\partial y} &amp;amp;= \frac{\partial \phi_i}{\partial \xi}J^{-1}_{21} + \frac{\partial \phi_i}{\partial \eta}J^{-1}_{22}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would then build $J$ by first constructing &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;J =&#xA;\begin{bmatrix}&#xA; \frac{\partial x}{\partial \xi} &amp;amp; \frac{\partial y}{\partial \xi} \\ &#xA; \frac{\partial x}{\partial \eta} &amp;amp; \frac{\partial y}{\partial \eta} \\   &#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA; \sum x_j^K\frac{\partial \psi_j}{\partial \xi}  &amp;amp; \sum y_j^K\frac{\partial \psi_j}{\partial \xi} \\ &#xA; \sum x_j^K\frac{\partial \psi_j}{\partial \eta}  &amp;amp; \sum y_j^K\frac{\partial \psi_j}{\partial \eta} &#xA;\end{bmatrix} &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x^K$ are the interpolation points on the 'physical space' element $K$, and &#xA; $\psi_i$ are the nodal shape functions on the master element. I then invert $J$ to find $J^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does this procedure also work for calculating $J^{-1}$ for isoparametric quads? Or does the chain rule expression become more complicated / nonlinear?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&#xA;The idea is to be able to calculate integrals like &#xA;$$&#xA;\int_K \nabla \phi_j \cdot \nabla \phi_i \, dx = \int_K \left( \frac{\partial \phi_i}{\partial x} \frac{\partial \phi_j}{\partial x} + \frac{\partial \phi_i}{\partial y} \frac{\partial \phi_j}{\partial y}\right)\, dx&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="20148" LastEditDate="2018-04-13T16:08:31.873" LastActivityDate="2018-04-13T16:51:32.520" Title="Integral transformations for isoparametric quadrilateral elements" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29294" PostTypeId="2" ParentId="29293" CreationDate="2018-04-13T16:51:32.520" Score="3" Body="&lt;p&gt;The rule is exactly the same. The only difference is that the matrix $J$ still depends on the coordinates $\xi,\eta$ because the functions $\psi_j$ are not linear but bilinear. Other than that, everything remains the same.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-13T16:51:32.520" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29296" PostTypeId="1" AcceptedAnswerId="29297" CreationDate="2018-04-13T19:50:16.897" Score="1" ViewCount="45" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;There are $n$ sources with the following positive volumes: $p_1, ..., p_n$ and there are $m$ destinations with the following positive volumes: $q_1, ..., q_m$. It is known that $p_1+ ...+ p_n=q_1+ ...+ q_m$. Each source is going to use its volume to fill up one or more destination(s). If a source fills one destination, we are paying $1$ for it. If it fills $k$ destinations we are paying $k$ for it. The problem is to fill destinations using the sources so that the price is the smallest possible.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I need a help coming up with the algorithm for the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Frankly speaking I am very lost here and can not come up with the algorithm which would be at least constantly better than going over all the possible fillings and choosing the one with the smallest price.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing I pondered over was sorting the sources in the ascending order and then taking the sources one by one, but I can not prove the correctness of such an approach.&lt;/p&gt;&#xA;" OwnerUserId="27464" LastEditorUserId="27464" LastEditDate="2018-04-13T20:08:20.947" LastActivityDate="2018-04-13T21:08:48.190" Title="Distribute sources among destinations" Tags="|optimization|algorithms|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29297" PostTypeId="2" ParentId="29296" CreationDate="2018-04-13T21:08:48.190" Score="3" Body="&lt;p&gt;Let your decision variables be $X \in \mathbb{R}^{n\times m}$ and $Z \in \{0,1\}^{n\times m}$, where $X_{ij}$ is the amount produced by producer $i$ sent to consumer $j$. $Z_{ij}$ is a binary variable which is 1 if producer $i$ sends any thing to consumer $j$, 0 otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your objective is to minimize the cost function:&#xA;$$&#xA;c(X,Z) = \sum_{i,\,j}Z_{ij}&#xA;$$&#xA;Subject to the following constraints:&#xA;\begin{align}&#xA;\sum_i X_{ij} &amp;amp;= q_j \;\; \forall j\in\{1...m\} \;\; \text{(Demand satisfied)} \\&#xA;%&#xA;\sum_j X_{ij} &amp;amp;\le p_i \;\; \forall i\in\{1...n\}\;\;\text{(Producer supply constraint)} \\&#xA;%&#xA;X_{ij} &amp;amp;\ge 0 \;\;\text{(Nonnegative production)}\\&#xA;X_{ij} &amp;amp;\le Z_{ij}p_i \;\;  \text{($Z_{ij}$ must equal 1 if $X_{ij} \neq 0$)}\\ &#xA;Z_{ij} &amp;amp;\in \{0,1\} &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, integer programs require you to use a branch-and-bound algorithm. However, since the only integer variables in this problem are restricted to be &lt;em&gt;binary&lt;/em&gt; variables, I believe that you can treat $Z_{ij}$ as continuous and solve the problem using a linear programming solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're specifically interested in algorithms used to solve linear programs, you should google around for the &quot;Dantzig's simplex algorithm,&quot; which is the most common way to solve modest-to-moderate-sized linear programs. For larger scale problems, &quot;interior point&quot; solvers tend to perform better than simplex-based solvers.&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2018-04-13T21:08:48.190" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29298" PostTypeId="2" ParentId="29283" CreationDate="2018-04-13T22:19:47.560" Score="7" Body="&lt;h2&gt;Reducing Memory for Sparse Matrices&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;One method (that they mention in the first paper you linked, but is worth emphasizing) is the Block Compressed Sparse Row (BCSR) storage format. If your problem creates dense $n \times n$ blocks (common in e.g. FEM with multiple DoFs per node), you modify the typical CSR (CSC) storage scheme to store only a single column (row) index for the block. This reduces the storage requirement of the index vector by a factor of $n^2$. If you're storing $3\times 3$ blocks, this reduces the memory required for the index vector by ~90%, which is far more than you'll save using any sort of bit-packing tricks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A fuller description of the format can be found here:&#xA;&lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/etemplates/node375.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.netlib.org/utk/people/JackDongarra/etemplates/node375.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, storing dense blocks in the sparse matrix allows you to vectorize the matrix-vector multiplication with SSE or AVX instructions, but you probably won't see a huge speedup here because, as you noted, you'll still be memory-bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your problem doesn't produce a structure that is as perfect for BCSR as FEM, then you can still use the technique without incurring the memory overhead of unaligned blocks. This paper shows an implementation that does not require that the blocks start on a multiple of $n$ column (and also explains what I meant by memory overhead of unaligned blocks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bebop.cs.berkeley.edu/pubs/vuduc2005-ubcsr-split.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://bebop.cs.berkeley.edu/pubs/vuduc2005-ubcsr-split.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, there is nothing preventing you from playing the bit-packing games with a re-ordered BCSR matrix to further reduce the memory requirement, but you should probably pick the low-hanging fruit first.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Bit-packing in general&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you're specifically interested in applications of bit packing, this is done aggressively in the LLVM project. One of its (clever?) ideas is to use what they call a PointerIntPair, which stores a pointer and a small integer together in the space of a single pointer. The idea here is to take advantage of the fact that (on linux x64 systems), pointers returned by malloc are 16-byte aligned, which gives you 4 bits at the bottom to do something with. They do all sorts of crazy things using this idea, building up complex data structures that live in the space of a single pointer. Here's an interesting conference talk given a few years ago by one of the LLVM optimizer guys. He starts talking about this at 22:00. Fair warning, though: this was at a C++ conference, so there is a large amount of unabashedly-complicated C++ involved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=vElZc6zSIXM&amp;amp;t=22m&amp;amp;ab_channel=CppCon&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=vElZc6zSIXM&amp;amp;t=22m&amp;amp;ab_channel=CppCon&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2018-04-13T22:19:47.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29299" PostTypeId="1" CreationDate="2018-04-14T16:43:18.883" Score="4" ViewCount="865" Body="&lt;p&gt;I have difficulties in understanding the role of the weight function $w(x)$ that occurs in the solution of PDEs via the Galerkin approach. Consider a linear differential equation of the form&#xA;$$&#xA;\partial_t \,u(x,t) \ = \ L \, u(x,t)&#xA;$$&#xA;where $L= L[x,\partial_x,\partial_{xx}]$ is a differential operator.&#xA;In the Galerkin approach one uses the ansatz&#xA;$u(x,t) = \sum_k a_k(t) \,p_k(x)$&#xA;to obtain&#xA;$$&#xA;\sum_k {\dot a}_k(t) \,p_k(x) \ = \ \sum_k a_k(t) L\, p_k(x)&#xA;$$&#xA;The next step is an application of the functional $\int p_i(x) w(x) dx$:&#xA;$$&#xA;\sum_k {\dot a}_k(t) \,(p_i,p_k)_w \ = \ \sum_k a_k(t) \;(p_i, L \, p_k)_w&#xA;$$&#xA;The resulting equation is solved by the usual methods for time-propagation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to the point: the functions $p_k(x)$ are often chosen as orthogonal polynomials (with appropriately built-in boundary conditions via basis recombination). For this, the book of Hesthaven and Gottlieb states (e.g. page 118)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since we are using (orthogonal) polynomials $p_k(x)$ as a basis it is natural to choose the weight function $w(x)$ such that $(p_k,p_j)_w =\gamma_k \delta_{kj}$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Isn't it so that for each different $w(x)$, one solves a different problem? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why should one &lt;em&gt;a priory&lt;/em&gt; attribute particular importance to different regions of  $[-1,1]$ and relatively neglect others (as it is done for all weights $w(x)\neq1$). Put differently, why should one ever use the Chebyshev polynomials and weight function $w_\text{Cheb}(x) = 1/\sqrt{1-x^2}$ if I can use the Legendre polynomials and $w_{\text{Leg}}(x)=1$ (--one reason is the FFT, but please neglect that here, as it's meant more as a general question)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And if one does so, i.e. applies a weighting in $[-1,1]$, which weight function is &quot;optimal&quot; (in the sense, that the original PDE is approximated &quot;best&quot;)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And, just to confirm my understanding: if I use Chebyshev polynomials with a weight $w(x)=1$ (which is effectively Clenshaw-Curtis integration), I lose all the advantages of the Gaussian grid, yes?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'd appreciate also partial answers, thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2020-01-29T16:50:48.763" Title="Role of weight function in Galerkin methods" Tags="|spectral-method|galerkin|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29300" PostTypeId="2" ParentId="29299" CreationDate="2018-04-14T17:26:23.250" Score="2" Body="&lt;p&gt;The basis functions $\{p_n(x): n \in \mathbb{N}\}$ can be orthogonal in $[a, b]$ with respect to a weight function $w(x)$. For example, Hermite polynomials are orthogonal in $(-\infty, \infty)$ with respect to $e^{-x^2}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{-\infty}^\infty H_m(x) H_n(x)\, e^{-x^2}\, \mathrm{d}x = \sqrt{ \pi}\, 2^n n! \delta_{nm}\,  .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The weight function is needed for the orthogonality. In this case the integral would not even be bounded without the weight function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let's just consider [-1,1], I'll repeat the question: how could one ever come up with a different weight function than w(x)=1? What is the motivation a priori?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's an interesting question, but this is somewhat the reverse thought that one normally has. I would say that you don't want to have a different (from $w(x)=1$) weight function, it just happens to be the case that for some set of functions in a certain vector space to be orthogonal you have to have it. There might be several reasons for picking one set of functions over other as basis to approximate the solution of a PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first time I ended up using orthogonal functions was when I was using separation of variables to solve PDEs, where was common to write the spatial part as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Sturm%E2%80%93Liouville_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sturm-Liouville&lt;/a&gt;. When you have a differential equation written in this form you know that you have a set of orthogonal functions with a certain weight.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-04-15T00:58:34.683" LastActivityDate="2018-04-15T00:58:34.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29303" PostTypeId="1" CreationDate="2018-04-15T01:29:54.227" Score="2" ViewCount="141" Body="&lt;p&gt;I am a complete beginner in numerical simulation and I am pretty lost about how to tackle this problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been trying for some time to find the steady state (or simulate), the following system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \dot S = D_S \Delta S + \rho_S \frac{ F^n } { F^n + 1} - \delta_S S -\delta_C P^2 S $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \dot P  = D_P \Delta P + \rho_P - \delta_P P + (v - 2 \delta_C) P ^ 2 S $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \dot F = \Delta + \rho_F \frac{ 1 } { (P^2S)^n + 1 } - \delta_F F $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on a mesh that looks like this (with no boundary conditions given):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1yMqQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1yMqQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize that this systems contains the temporal derivatives, so I'm not explicitly going for the steady state, but when I took out the &lt;code&gt;TransientTerm&lt;/code&gt; it complained that it was &lt;code&gt;Exactly singular&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S = CellVariable(mesh = mesh, hasOld=True)&#xA;P = CellVariable(mesh = mesh, hasOld=True)&#xA;F = CellVariable(mesh = mesh, hasOld=True)&#xA;&#xA;sdot = TransientTerm(var=S) == DiffusionTerm(coeff=Ds, var=S) + \&#xA;rs * pow(F,2) / (pow(F,2) + 1) - ds * S - dc * pow(P, 2) * S&#xA;&#xA;pdot = TransientTerm(var=P) == DiffusionTerm(coeff=Dp, var=P) + \&#xA;rp  - dp * P  + (v - 2. * dc) * S * pow(P, 2)&#xA;&#xA;fdot = TransientTerm(var=F) == DiffusionTerm(coeff=1, var=F) + \&#xA;rf / (pow(S * pow(P, 2), 2)  + 1.0 ) - df * F&#xA;&#xA;dt = 0.005&#xA;coupled.sweep(dt = dt)&#xA;&#xA;for i in range(0, 100000):&#xA;&#xA;    v1.plot('plots/SHH_'+str(dt * i)+'.png')&#xA;    v2.plot('plots/Ptc_' + str(dt * i)+'.png')&#xA;    v3.plot('plots/FGF_'+str(dt * i)+'.png')&#xA;&#xA;    S.updateOld()&#xA;    P.updateOld()&#xA;    F.updateOld()&#xA;&#xA;&#xA;dt = 0.005&#xA;&#xA;for i in range(0, 100000):&#xA;&#xA;    v1.plot('plots/SHH_'+str(dt * i)+'.png')&#xA;    v2.plot('plots/Ptc_' + str(dt * i)+'.png')&#xA;    v3.plot('plots/FGF_'+str(dt * i)+'.png')&#xA;&#xA;    S.updateOld()&#xA;    P.updateOld()&#xA;    F.updateOld()&#xA;    coupled.sweep(dt = dt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The simulation initially appears to be proceeding as normal, but eventually, regardless of the size of my time-step, the $S$ values and $P$ values drop below zero and subsequently $S$ blows up do its large production rate $\rho_s = 1600$, causing an overflow error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should mention that when I simulated this problem in 1D using &lt;code&gt;pdepe&lt;/code&gt;, the simulation was find and converged to a steady state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I debug this, and is there other problems with my simulation? Should I try a different package such as Firedrake? If so, how do I solve the convergence problems there? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks very much in advance.&lt;/p&gt;&#xA;" OwnerUserId="27469" LastActivityDate="2018-04-15T01:29:54.227" Title="2D reaction-diffusion system simulation" Tags="|finite-element|numerical-analysis|python|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29304" PostTypeId="2" ParentId="29299" CreationDate="2018-04-15T06:11:57.683" Score="4" Body="&lt;p&gt;Chebyshev are orthogonal wrt to a weight function which is singular at the end-points. When you approximate a function f(x) with Chebyshev, the convergence of the approximations is not affected by the values of f or its derivatives at the end points. The convergence rate depends only on the smoothness of the function f and not on its boundary values. This is a general property of any orthogonal basis that comes as eigenfunctions of a sturm-liouville problem with singular weight functions. See ([1], section 3). This would not be true with uniform weight.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Gottlieb and Orszag: Numerical analysis of spectral methods &lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-04-15T06:11:57.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29305" PostTypeId="1" AcceptedAnswerId="29342" CreationDate="2018-04-15T13:00:55.910" Score="3" ViewCount="1062" Body="&lt;p&gt;How to numerically minimize a functional, for example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J[y]=\int_{x_1}^{x_2}L(x,y(x),y'(x))dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An equivalent problem is to solve the Euler equation for this functional as a differential equation. But is there a numerical method to directly minimize this functional without solving a differential equation?&lt;/p&gt;&#xA;" OwnerUserId="27472" LastActivityDate="2018-04-20T18:58:26.600" Title="How to numerically minimize a functional?" Tags="|optimization|pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29307" PostTypeId="1" CreationDate="2018-04-16T00:18:02.363" Score="4" ViewCount="1980" Body="&lt;p&gt;I'm working with the tight-binding model, and I'm trying to learn the basics of how to compute the Density of States (DOS) $N(E)$ numerically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The DOS is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$N(E) = \frac{1}{N}\sum_k \delta(E-\epsilon_k)\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\epsilon_k$ is the dispersion relation - in 1D, it has the form $-2t\cos(k)$, where $t$ is a parameter I choose to be 1. We can also write this as an integral, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$N(E) = \frac{1}{2\pi} \int_k \delta(E - \epsilon(k)) dk $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a couple ways of estimating the DOS. I am taking two approaches, both of which seem highly sensitive to numerical parameters with convergence.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the Lorentzian approximation to the Delta &quot;function&quot;, $\delta(x) \equiv \lim_{\epsilon \rightarrow 0} \frac{\epsilon}{\epsilon^2 + x^2} $. The code is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Approximate the delta function&#xA;def delta(x):&#xA;    return (1/pi)*(eps/(x**2 + eps**2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then simply compute the above sum over my range of $k$s, as shown below for the 1D case.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Use summation form of density of states for numeric calculation&#xA;def N(E):&#xA;    D = sum([delta(E - disp_e(k1)) for k1 in ks])&#xA;    # Minimum D for every E should be pi/4 for the 1D case. Unfortunately,   it's going to 0 mostly. Why? How? &#xA;    dos = (1/Num_sites) * D&#xA;    return dos&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute the DOS of states as the imaginary part of the Green's function of the system at its poles, $ N(E) =-\frac{1}{N\pi} \Im{\sum_k \frac{1}{E - \epsilon_k + i\epsilon}} $, where $\epsilon$ again is supposed to be a small parameter. Here is the code for the functions in 1D, though I've done them for 2D and 3D as well:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Try to get DOS by summing over k-points of the lattice Green's function&#xA;def Greens(E, e_k):&#xA;    return (1/Num_sites)*(1/(E - e_k + 0.05*1.0j))&#xA;&#xA;def N2(E):&#xA;    D = (-1/(N*pi))*(sum([Greens(E, disp_e(k1)) for k1 in ks])).imag&#xA;    return D&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Finally, onto my question.&lt;/strong&gt; I spent quite a while (10 hours) trying to figure out why my code wasn't converging to the 1D analytic solution, despite adding more k-points or having a finer resolution over the energies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turned out that &lt;strong&gt;despite the definitions&lt;/strong&gt;, the parameter $\epsilon$ I was choosing was WAY too small. I was choosing numbers on the order of $10^{-4}$, but eventually I figured out that $\epsilon$ only showed correct numerical results when it was around $\epsilon \in [0.1, 0.5]$ for the Green's function method, and between 0.01-0.05 for the delta function method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the correct solution in 1D, both analytic and numerical, that I finally got. This is for epsilon = 0.01 for the delta function method. &lt;a href=&quot;https://i.stack.imgur.com/V3TkQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V3TkQ.png&quot; alt=&quot;Hubbard U = 0 1D model, epsilon = 0.01.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I get when I vary the epsilon parameter to be smaller. The oscillations become divergences at even smaller epsilons and I don't get ANY numerical solution. &lt;a href=&quot;https://i.stack.imgur.com/adjRB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/adjRB.png&quot; alt=&quot;Epsilon = 0.001. At even smaller values the solution diverges.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is - why? The definitions imply a small $\epsilon$, and furthermore, I don't see any reason why two separate numerical methods would both not converge for the same reason. Is there stability/convergence analysis for these approximations anywhere? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I want to know is because I want to study systems without analytic solutions, and if the quality of my solution varies erratically with a supposedly arbitrary parameter, I can't be sure of my numerical solution!&lt;/p&gt;&#xA;" OwnerUserId="24690" LastEditorUserId="9667" LastEditDate="2018-04-17T15:54:38.323" LastActivityDate="2020-01-27T22:56:54.090" Title="Numerical Sensitivity in Density of States of Tight-binding model" Tags="|python|computational-physics|convergence|stability|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29308" PostTypeId="1" CreationDate="2018-04-16T05:03:34.013" Score="2" ViewCount="3866" Body="&lt;p&gt;I have three-dimensional radar reflectivity data obtained as voxels (scans, rays, altitudes). The data has been sampled at irregular spacings and I want to convert this into a regular grid. In addition, I have 3-D arrays for the latitude, longitude, and altitude with the same shape(scans, rays, altitudes). Python is my programming language of choice. Is using scipy's &lt;code&gt;RegularGridInterpolator&lt;/code&gt; the best way to obtain a regular grid? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A colleague suggested that I first start with a 2-D interpolation  (scans and rays for every altitude) and then proceed to do a 3-D interpolation. A bulk of the reflectivity data contains zeros(zero reflectivity) and NaNs(no data available) and whether any filter would be applicable to remove the zeros/NaNs would be nice to know. Any recommendations and references on this topic will be appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for a sample data set - I have total of 22163680 points  and out of which 266111 are finite values(greater than zero). The rest are zeros and NaNs. &lt;/p&gt;&#xA;" OwnerDisplayName="user16555" LastEditorDisplayName="user16555" LastEditDate="2018-05-06T09:10:34.820" LastActivityDate="2023-06-29T03:05:40.677" Title="Three dimensional irregular grid data interpolation to regular grid" Tags="|python|interpolation|scipy|grid|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29313" PostTypeId="1" CreationDate="2018-04-16T20:21:35.917" Score="0" ViewCount="1123" Body="&lt;p&gt;I am looking for numerical packages (ideally Python) to solve second kind Volterra integral equations, such as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(t)=g(t)+\int_0^tK(t,s)u(s) ds$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or Volterra-Fredholm integral equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x,t)=g(t,x)+c\int_0^t\int_\Omega K(t,s,x,\xi)u(s,\xi) d\xi ds$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any callable functions in Python to solve such equations? if not, are there any standard algorithm to solve such equations?&lt;/p&gt;&#xA;" OwnerUserId="27484" LastEditorUserId="9667" LastEditDate="2018-04-16T21:35:36.493" LastActivityDate="2020-07-19T18:10:09.033" Title="Numerical packages to solve Volterra integral equations" Tags="|python|integral-equations|software-recommendation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29314" PostTypeId="2" ParentId="29313" CreationDate="2018-04-16T21:06:42.853" Score="2" Body="&lt;p&gt;You can usually solve these kinds of equations via a transformation. &lt;a href=&quot;https://academic.oup.com/imajna/article/8/1/37/802496&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shampine&lt;/a&gt; discusses how Volterra integral equations can be transformed into an ODE which is then solved with a stiff ODE solver. If you discretize &lt;code&gt;u(x)&lt;/code&gt; into a system of ODEs first then you can maybe do something similar in that case.&lt;/p&gt;&#xA;&lt;p&gt;If you want to handle a general functional ODEs directly, then there are two approaches. One approach is to develop specific solvers for each type of subproblem, and there is research in this if you look around but there's not much software to go along with it. On the other hand, general state-dependent delay differential equation solvers can do this kind of thing because they give you the full &lt;code&gt;u(s)&lt;/code&gt; for &lt;code&gt;s&amp;lt;t&lt;/code&gt;. This means you can actually take that interval using that interpolation. But given Shampine's analysis you can see that these things can easily end up stiff so the DDE solver needs to be able to handle stiffness. &lt;a href=&quot;http://diffeq.sciml.ai/latest/tutorials/dde_example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl's DDE solvers&lt;/a&gt; have that kind of flexibility, and I believe Maple's would work here as well. I believe &lt;a href=&quot;https://www.unige.ch/%7Ehairer/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RADAR5&lt;/a&gt; has that flexibility too. I'm not sure of a similar Python offering. You need to make sure you setup some kind of caching structure to not re-solve that integral in full every step (for efficiency), but it shouldn't be too hard to build software with this route. That said, it likely won't be as efficient as the transformation or more direct solver approaches.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T18:10:09.033" LastActivityDate="2020-07-19T18:10:09.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29315" PostTypeId="1" CreationDate="2018-04-17T02:33:21.410" Score="1" ViewCount="336" Body="&lt;p&gt;I am attempting to numerically integrate the equation &#xA;$$\frac{\mathrm{dP} }{\mathrm{d} r}=-\left ( P+\rho\left ( r \right ) \right )\frac{m\left ( r \right )+4\pi r^{3}P}{r\left [ r-2m\left ( r \right ) \right ]}$$&#xA;to find $P\left ( r \right )$. I have defined the following:&#xA;$$\rho\left ( r \right )=\rho_{c}\left [ 1-\left ( \frac{r}{R} \right )^{2} \right ]$$&#xA;$$m\left ( r \right )=\frac{4\pi \rho_{c}\left ( 5r^{3}R^{2}-3r^{5} \right )}{15R^{2}}.$$&#xA;My code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.integrate import odeint&#xA;&#xA;def dP_dr(x, r):&#xA;    return -(x+density(r, p_c, R))*(mass(r, p_c, R)+4*np.pi*r**3*x)/(r*(r- &#xA;        2*mass(r, p_c, R)))&#xA;&#xA;def mass(r, p_c, R):&#xA;    return 4*np.pi*p_c*(5*r**3*R**2-3*r**5)/(15*R**2)&#xA;&#xA;def density(r, p_c, R):&#xA;    return p_c*(1-(r/R)**2)&#xA;&#xA;p_c = 7*10**17      #central density&#xA;P_c = 1.6*10**34    #central pressure&#xA;R = 10              #radius&#xA;&#xA;deltaR = 0.01&#xA;r = np.linspace(deltaR, R, int(R/deltaR))&#xA;&#xA;P = odeint(dP_dr, P_c, r)&#xA;&#xA;plt.figure()&#xA;plt.plot(r, P, label='Pressure')&#xA;plt.xlabel('Radius (km)')&#xA;plt.title('Neutron Star')&#xA;plt.legend(loc=0)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm unsure what is wrong, but I am getting an overflow error. The graphed result also is garbage. The pressure, $P\left ( r \right )$ should also decrease with $r$, but it jumps in very sharp spikes. Any help with what I am doing wrong would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="27488" LastEditorUserId="27488" LastEditDate="2018-04-17T12:04:47.510" LastActivityDate="2018-11-13T05:02:45.510" Title="Scipy odeint Unexpected Results" Tags="|ode|scipy|integration|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29317" PostTypeId="2" ParentId="18785" CreationDate="2018-04-17T06:17:56.623" Score="-2" Body="&lt;p&gt;For MILP the best solvers are CPLEX offered by GAMS, LINGO, and Gurobi in Python. You should have the full version of these solvers. &#xA;There is no single solver is superior among these three solvers.&lt;/p&gt;&#xA;" OwnerUserId="27490" LastEditorUserId="20688" LastEditDate="2018-04-17T13:34:56.537" LastActivityDate="2018-04-17T13:34:56.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29318" PostTypeId="2" ParentId="29299" CreationDate="2018-04-17T07:46:22.220" Score="2" Body="&lt;p&gt;I will answer you in order:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) The problem to solve is independent of the weighing function you choose, e.g. you could solve for a PDE using standard finite element basis (Lagrange) without using special functions that has special properties (orthogonal functions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) The weight function $w(x)$ for certain polynomials to be orthogonal arise naturally, e.g. the functions $f(x) = 1$ and $g(x) = x$ are orthogonal in the interval $[-1,1]$ w.r.t. the weights $w(x) = 1$ and $w(x) = 1/\sqrt{1-x^2}$, but if sequence continues this is no longer true. For example, basis functions like $f_n(x) = \cos(nx)$ are orthogonal for different $n$ in $[0,2\pi]$:&#xA;$$\int_0^{2\pi}{f_nf_mdx}=\int_0^{2\pi}{\cos{(nx)}\cos{(mx)}dx}=\pi\delta_{nm}$$&#xA;This functions, are orthogonal w.r.t. the weight function $w(x) = 1$. But, if we change variables to $y=\cos{x}$ you will discover actually that:&#xA;$$\int_0^{2\pi}{\cos{(nx)}\cos{(mx)}dx}=\int_{-1}^{1}\frac{T_n(y)T_m(y)}{\sqrt{1-y^2}}dy = \pi\delta_{nm}$$&#xA;Being $T_n(x) = \cos{(n\cos^{-1}{(y)})}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) The optimal basis is a basis that spans the whole space (your interval) and its &quot;basis vectors&quot; are as independent as possible among them. The perfect basis is of course an orthogonal one (think about cartesian axis, where everything is easy e.g. scalar products). If you choose this kind of basis, your solution will be best representated. There micht be reasons to choose a basis before another, e.g. Legendre polynomials are preferred in situations when extremal points of the interval are of importance and thus they are nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) The only advantage of the use of an orthogonal polynomial is that the scalar product leads to a diagonal matrix mass matrix, and inversion is then trivial.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-04-17T07:46:22.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29319" PostTypeId="1" AcceptedAnswerId="29324" CreationDate="2018-04-17T11:04:11.920" Score="4" ViewCount="120" Body="&lt;p&gt;First of all sorry if this is the wrong place to ask this question, I went to a few stack sites and thought here it would be more suitable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a physical quantity $F$ that depends on $x$. At each $x$ I have an error $dF$ in the measurement of $F$. &#xA;The quantity I want to measure is defined as an integral: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;M = \int\limits_{x_1}^{x_2} F(x)dx&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to be able to compute the error $dM$ in $M$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about estimating the same integral replacing $F(x)$ by $F(x)+dF(x)$ for the upper error and $F(x)-dF(x)$ for the lower error but I would like to know if there is a better way to make this error estimation.&lt;/p&gt;&#xA;" OwnerUserId="27491" LastEditorUserId="20688" LastEditDate="2018-04-17T13:38:10.610" LastActivityDate="2018-04-18T09:38:10.383" Title="Error on a integral quantity with noise" Tags="|error-estimation|integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29320" PostTypeId="1" AcceptedAnswerId="29552" CreationDate="2018-04-17T14:39:05.120" Score="11" ViewCount="1610" Body="&lt;p&gt;Let $A, G, Q$ be real, square, dense matrices. $G$ and $Q$ are symmetric. Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H = \begin{bmatrix} A &amp;amp; -G \\ -Q &amp;amp;-A^T \end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;be a Hamiltonian matrix. I want to compute the matrix exponential of $H$. I need the full matrix exponential, $e^{tH}$, not only the matrix-vector product. Are there any specialized algorithms or libraries available to compute the exponential of a Hamiltonian matrix?&lt;/p&gt;&#xA;" OwnerUserId="17283" LastEditorUserId="20417" LastEditDate="2018-05-18T19:59:05.147" LastActivityDate="2018-05-18T19:59:05.147" Title="Matrix exponential of a Hamiltonian matrix" Tags="|linear-algebra|matrix|dense-matrix|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="29323" PostTypeId="1" CreationDate="2018-04-17T17:43:52.900" Score="2" ViewCount="40" Body="&lt;p&gt;I think it is well known that 2nd order NURBS curve can describe any segment of a conic section. Such a segment has 7 degrees of freedom, take an ellipse arc for example: the ellipse itself has 5 degrees of freedom (center, half-axes and rotation), and the two points on the ellipse (2 angles) give the remaining 2 degrees of freedom. A 2nd order NURBS curve segment has less than 9 degrees of freedom: 3 control points and 3 weights, but the scaling of the weights is irrelevant. Can one discuss away the remaining additional degree of freedom?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I think I can show that a planar 2nd order B-SPLINE segment is a segment of a parabola or of a line. The argument that both have 6 degrees of freedoms applies, but the algebra is also easy to work out.)&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2018-04-17T17:43:52.900" Title="Are planar 2nd order NURBS curve segments exactly the segments of conic sections?" Tags="|interpolation|b-spline|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29324" PostTypeId="2" ParentId="29319" CreationDate="2018-04-17T23:00:25.727" Score="6" Body="&lt;p&gt;Assuming that you mean the following inequality in your prompt&#xA;$$ |F_\mathrm{true}(x) - F(x)| \le |dF(x)| \qquad \forall x,$$&#xA;a simple bound for $dM$ is the following&#xA;$$ dM \equiv | M_\mathrm{true} - M | = \left|\int_{x_1}^{x_2} F_\mathrm{true}(x) - F(x) dx\right| \le \int_{x_1}^{x_2} |dF(x)| dx. $$&#xA;This also assumes that $dF$ is absolute integrable.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastActivityDate="2018-04-17T23:00:25.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29325" PostTypeId="2" ParentId="29320" CreationDate="2018-04-18T00:16:23.933" Score="1" Body="&lt;p&gt;You might have an option to use hierarchical matrices ($\mathcal H$-matrices) and the corresponding functionality of the libraries that support them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Effectively, if every matrix $A$, $G$, and $Q$ are well &amp;amp; efficiently represented in $\mathcal H$-matrix format, then the block Hamiltonian matrix $H$ is effectively a block $\mathcal H$-matrix. The question about the representation of $A$, $G$, and $Q$ in the hierarchical form comes down to their origin: if one can find low-rank structures within them (possible row/col indices permutations applied), then this approach is viable. A plausible example would be if $A$, $G$, and $Q$ come from an integral equation which will also explain their dense structure and potential for compression (depending on the kernel). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formal requirement for this method to work, will be if $(H-\lambda I)^{-1}$ is representable in $\mathcal H$-matrix format; however, I would start directly from constructing $\mathcal H$-matrix representation of $A$, $G$, and $Q$ and hope for the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Application of $\mathcal H$-matrices to matrix exponentiation is well discussed in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1007/s002110100360&quot; rel=&quot;nofollow noreferrer&quot;&gt;I. Gavrilyuk, W. Hackbusch, and B. Khoromskij, &quot;$\mathcal{H}$-matrix approximation for the operator exponential with applications.&quot; &lt;em&gt;Numerische Mathematik&lt;/em&gt;, 92, 83–111, 2002.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There &lt;a href=&quot;https://github.com/gchavez2/awesome-hierarchical-matrices&quot; rel=&quot;nofollow noreferrer&quot;&gt;are several libraries&lt;/a&gt; that support $\mathcal H$-matrices. I know that the old &lt;a href=&quot;http://hlib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;$\mathcal H$Lib&lt;/a&gt; supported some form matrix exponentiation and had the necessary pieces to build one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Downsides of this approach:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;relies on the efficient representation of  $A$,$G$, and $Q$&lt;/li&gt;&#xA;&lt;li&gt;does not take advantage of the Hamiltonian structure&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Positives:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the compressed representation of the matrix exponential, though it is still a matrix, not just a way to do an MVP&lt;/li&gt;&#xA;&lt;li&gt;linear-logarithmic complexity (provided the low-rank assumption is there)&lt;/li&gt;&#xA;&lt;li&gt;the library might take advantage of the transposition and symmetry in the blocks &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-04-18T01:44:20.153" LastActivityDate="2018-04-18T01:44:20.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29327" PostTypeId="2" ParentId="5682" CreationDate="2018-04-18T07:42:11.580" Score="1" Body="&lt;p&gt;It's good to be aware of &lt;a href=&quot;http://www.inf.usi.ch/hormann/papers/Floater.2007.BRI.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Floater-Hormann interpolants&lt;/a&gt; when you &lt;em&gt;have&lt;/em&gt; to (or want to) work with equidistant points $\{x_i\}_{i=1\ldots n}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the integer $d$ with $0 \le d \le n$, let $p_i$ be the polynomial interpolant of $\{ x_i, \ldots x_{i+d} \}$. Then the FH interpolant of a function $f$ at $\{x_i\}_{i=1\ldots n}$ has the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ r_n(x) := \frac{\sum_{i=0}^{n-d} \lambda_i(x) \, p_i(x)}{\sum_{i=0}^{n-d} \lambda_i(x)} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the &quot;blending functions&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \lambda_i(x) = \frac{(-1)^i}{(x-x_i) \ldots (x - x_{i+d})} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some properties of these interpolants:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;they are barycentric rational interpolants &lt;em&gt;with no real poles&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;achieve arbitrary approximation orders ${\cal O}(h^{d+1})$ for $f \in C^{d+2}[a,b]$, regardless of the distribution of points;&lt;/li&gt;&#xA;&lt;li&gt;are somewhat similar to splines, in that they blend (local) polynomial interpolants $p_0, \ldots p_{n-d}$ with the $\lambda$'s acting as the blending functions;&lt;/li&gt;&#xA;&lt;li&gt;they reproduce polynomials of degree at most $d$ (or $d+1$ if $n-d$ is odd);&lt;/li&gt;&#xA;&lt;li&gt;can be written in barycentric form (see section 4 of Floater and Hormann's paper).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Caveat emptor&lt;/em&gt;: As expected (see the paper referenced by @Reid.Atcheson), increasing $d$ quickly degrades the conditioning of the approximation process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some fairly recent work done by Klein to alleviate this problem. He modified the original Floater-Hormann approach by adding $2 d$ &lt;em&gt;new&lt;/em&gt; data values corresponding to points outside the original interpolation interval $[a,b]$ constructed from a smooth extension of $f$ outside $[a,b]$ using only the given data $f_0, \ldots f_n$. This &quot;global&quot; data set is then interpolated by a new FH rational function $r_{n+2d}$ and evaluated &lt;em&gt;only&lt;/em&gt; inside $[a,b]$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The details are nicely laid out in Klein's paper (linked below), where it is shown that these &lt;em&gt;extended rational interpolants&lt;/em&gt; have Lebesgue constants that grow &lt;em&gt;logarithmically&lt;/em&gt; with $n$ and $d$ (whereas for the original FH scheme, said growth is &lt;em&gt;exponential&lt;/em&gt; in $d$, see Bos &lt;em&gt;et al.&lt;/em&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Chebfun library uses FH interpolants when building &lt;code&gt;chebfuns&lt;/code&gt; out of equispaced data, as explained &lt;a href=&quot;http://www.chebfun.org/examples/approx/EquispacedData.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;M. S. Floater and K. Hormann, Barycentric rational interpolation with&#xA;  no poles and high rates of approximation, &lt;em&gt;Numerische Mathematik&lt;/em&gt; 107&#xA;  (2007).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;G. Klein, An Extension of the Floater–Hormann Family of&#xA;  Barycentric Rational Interpolants, &lt;em&gt;Mathematics of Computation&lt;/em&gt;, 82 (2011) - &#xA;  &lt;a href=&quot;http://homeweb2.unifr.ch/kleing/pub/resources/Klein_final.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;L. Bos, S. De Marchi, K. Hormann, and G. Klein, On the Lebesgue&#xA;  constant of barycentric rational interpolation at equidistant nodes,&#xA;  Numer. Math. 121 (2012)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2018-04-18T12:05:26.737" LastActivityDate="2018-04-18T12:05:26.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29332" PostTypeId="1" CreationDate="2018-04-19T13:14:48.567" Score="0" ViewCount="366" Body="&lt;p&gt;I am trying to get some code working for the 1D Poisson equation using the textbook: Nodal Discontinuous Galerkin Methods Algorithms, Analysis, and Applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the following formulation (for a homogeneous case):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6eyVr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6eyVr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to account for inhomogeneous cases, the textbook proposes the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0ACvi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0ACvi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathcal A$ is given from the homogeneous case. When I attempt to implement the added term on the right-hand side of the equation above, I obtain the following response, which seems to almost respond correctly; however, the first and last terms deviate greatly from their specified b.c:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/92PUh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/92PUh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implement the extra rhs term with the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;e = zeros((basisdegree+1),numelements);&#xA;enp = e; enp(end) = 1; e1 = e; e1(1) = 1;&#xA;extraterm = Dr*(enp*b - e1*a);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;e&lt;/code&gt; is just a vector the size of the solution &lt;code&gt;u&lt;/code&gt;, with 1 located in the first and last element. &lt;code&gt;Dr&lt;/code&gt; is the same as in the homogeneous equations which are $\mathcal{M}^{-1} \mathcal{S}$.&lt;/p&gt;&#xA;" OwnerUserId="27505" LastEditorUserId="20688" LastEditDate="2018-04-19T14:30:17.220" LastActivityDate="2018-04-19T19:52:25.480" Title="Discontinuous Galerkin - Inhomogeneous Dirichlet B.C. for 1D Poisson Equation" Tags="|finite-element|matlab|poisson|discontinuous-galerkin|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29334" PostTypeId="1" AcceptedAnswerId="29336" CreationDate="2018-04-19T18:10:12.123" Score="0" ViewCount="298" Body="&lt;p&gt;Let $V$ a Hilbert space, $a:V\times V\rightarrow \mathbb{R}$ a bounded, symmetric and positive bilinear form and $f:V\rightarrow\mathbb{R}$ bounded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is well known that problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\lbrace\begin{array}{l}\textrm{Find }u\textrm{ such that}\\ a(u,v)=f(v)\quad\forall v\in V\end{array}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is equivalent to minimize the following (energy) functional:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$J(v)=\dfrac{1}{2}a(v,v)-f(v).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If now I have some extra condition I get the problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\lbrace\begin{array}{l}\textrm{min }J(v)\textrm{ with }v\in V\\ \textrm{ subjetc to }u=g\quad\forall v\in \tilde V\subset V\end{array}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking in how to get the corresponding Lagrange multipliers in &lt;strong&gt;Stokes&lt;/strong&gt;-like problems, for example, where the problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left\lbrace\begin{array}{l}\textrm{min }J(u,p)\textrm{ with }(u,q)\in V\times Q\\ \textrm{ subjetc to }\int_\Omega p=0\quad\end{array}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but when I try to use Lagrange multipliers (learned in my first course of calculus in several variables) I need to calculate things like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla f=\lambda\nabla g$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but what is &quot;$\nabla$&quot; operator in my context (with bilinear forms)? In other words, How can I use Lagrange multipliers in this context?&lt;/p&gt;&#xA;" OwnerUserId="17630" LastActivityDate="2018-04-20T07:25:52.533" Title="Lagrange multipliers in minimization problem with bilinear forms" Tags="|finite-element|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29335" PostTypeId="2" ParentId="29332" CreationDate="2018-04-19T19:52:25.480" Score="1" Body="&lt;p&gt;You did everything right but forgot that the way you impose boundary values leads to a solution that has zero boundary values (when applied strongly) and for which you have to set the first and last degree of freedom to their correct value &lt;em&gt;after solving the linear system&lt;/em&gt;. The solution will then be correct/as expected.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-19T19:52:25.480" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29336" PostTypeId="2" ParentId="29334" CreationDate="2018-04-20T07:25:52.533" Score="3" Body="&lt;p&gt;Let's assume that your functions $u$ and $p$ can be expanded using a finite discrete basis in the subspaces $V$ and $Q$, e.g. $\phi_i\in V$ and $\psi_i \in Q$&#xA;$$u = \sum_iu_i\phi_i\qquad p = \sum_ip_i\psi_i \tag{*}$$.&#xA;The Stokes problem fits into the frame of the following stationary problem:&#xA;$$\left\{\begin{array}{l}%&#xA;find \quad stationary \quad point \quad of\quad J(v)=\frac{1}{2}a(v,v)-f(v) \\&#xA;s.t.\qquad \textrm{div}(v) = 0 \end{array}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with appropiate B.C.for $v$ and where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$a(u,v) = \int_\Omega \nabla u\cdot \nabla v dV$$&#xA;$$f(v) = \int_\Omega{fv\,dV}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bearing in mind that $a(\cdot,\cdot)$ is a symmetric bilinear form and $f(\cdot)$ is a linear form  inserting the subspace projections $(*)$ you'll find that the previous problem my be rewritten a:&#xA;$$\left\{\begin{array}{l}%&#xA; J(v)=\frac{1}{2}\sum_{i,k}a(\phi_i,\phi_k)u_iu_k-\sum_{i}f(\phi_i)u_i=\frac{1}{2}u^TAu-f^Tu \\&#xA;s.t.\qquad \sum_i \textrm{div}(u_i\phi_i) =0 \end{array}\right.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the restriction is supposed to be fulfilled in every point, we define a lagrange multiplier $p$ such as $(*)$ to impose this restriction as:&#xA;$$\sum_i\int_\Omega p\textrm{div}(u_i\phi_i)\,dV =\sum_{i,k}\int_\Omega p_k\psi_k\textrm{div}(u_i\phi_i)\,dV =- \sum_{i,k}\int_\Omega \textrm{grad}(p_k\psi_k)\cdot u_i\phi_i\,dV =-p^TBu $$ &#xA;Where $B$ coincides with&#xA;$$B = \int_\Omega \textrm{grad}(\psi_k)\phi_i\,dV$$&#xA;Therefore your augmented lagrangian will be a function of $v_i$ and the Lagrange multipliers $p_i$:&#xA;$$L(v,p)=\frac{1}{2}u^TAu-f^Tu-p^TBu$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you can apply the usual rule for Lagrange multipliers, that permitts yuo to obtain the equations that solve the Stokes problem:&#xA;$$\frac{\partial L}{\partial u}=Au-f-B^Tp=0$$&#xA;You can see this expression as (using a slightly different notation for $\nabla F = \lambda \nabla G$):&#xA;$$\nabla F = Au-f, \qquad \lambda = p^T, \qquad \nabla G = B$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And you restriction equations:&#xA;$$\frac{\partial L}{\partial p} = Bu = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this resultsin the following saddle point optimisation problem:&#xA;$$\left[\begin{matrix} %&#xA;A &amp;amp; B^T\\&#xA;B &amp;amp; 0\end{matrix}\right]\left[\begin{matrix} u\\p\end{matrix}\right]=\left[\begin{matrix}{f\\0}\end{matrix}\right]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: This does not pretent to be rigurous, only is a shallow explanation of which is inside the problem you asked for. &lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-04-20T07:25:52.533" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29338" PostTypeId="2" ParentId="10876" CreationDate="2018-04-20T10:04:23.857" Score="3" Body="&lt;p&gt;A few answers and comments here conflate confusingly the TDSE with a wave equation; perhaps a semantics issue, to some extent. The TDSE is the quantized version of the classical non-relativistic hamiltonian $$H=\frac{p^2}{2m} + V(x)= E.$$ With the rules $$p \rightarrow i\hbar\partial_x,\ \ E\rightarrow i\hbar\partial_t, \ \ x\rightarrow x,$$ (as discussed in chapter 1 of d'Espagnat, Conceptual foundations of quantum mechanics, &lt;a href=&quot;https://philpapers.org/rec/ESPCFO&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://philpapers.org/rec/ESPCFO&lt;/a&gt;), it therefore reads $$\left[-\frac{\hbar^2}{2m}\partial_{xx} + V(x)\right]\psi = i\hbar\partial_t\psi,$$&#xA;so it is clearly a diffusion-like equation. If one used the   relativistic energy,  which contains a E$^2$ term, then a wave-like equation such as $$ \partial_{xx}\psi = \partial_{tt}\psi +\dots $$ would obtain (for V=0 for simplicity), such as the Pauli or Klein-Gordon equations. But that is, of course, a completely different matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, back to the TDSE,   the obvious method is Crank-Nicolson as has been mentioned, because it is a small-time expansion that conserves the unitarity of the evolution (FTCS, e.g., does not). For the 1-space-D case, it can be treated as a matrix iteration, reading&#xA;$$&#xA;\vec{\psi}^{n+1}=({\cal I} + \frac{i\tau}{2\hbar}\tilde{H})^{-1}&#xA;({\cal I} - \frac{i\tau}{2\hbar}\tilde{H})\vec{\psi}^{n}&#xA;$$&#xA;with ${\cal I}$ the identity matrix and&#xA;$$&#xA;H_{jk}=(\tilde{H})_{jk}=\frac{-\hbar^{2}}{2m}\left[\frac{\delta_{j+1,k}+\delta_{j-1,k}-2\delta_{jk}}{h^{2}}\right]+V_{j}\delta_{jk}.&#xA;$$&#xA;(Details e.g. in Numerical methods for physics, &lt;a href=&quot;http://algarcia.org/nummeth/nummeth.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://algarcia.org/nummeth/nummeth.html&lt;/a&gt;,  by A. L. Garcia).&#xA;As most clearly seen in periodic boundary conditions, a space-localized $\psi$ spreads out in time: this is expected, because the initial localized $\psi$ is not an eigenstate of the stationary Schroedinger equation, but a superposition thereof. The (classical massive free particle) eigenstate with fixed momentum (for the non-relativistic kinetic operator) is simply $\psi_s=e^{ikx}/\sqrt{L}$, i.e. fully delocalized as per Heisenberg principle, with constant probability density 1/L everywhere (note that I am avoiding normalization issues with continuum states by having my particle live on a finite, periodically repeated line). Using C-N,  the norm &#xA;$$\int |\psi|^2 dx$$ is conserved, thanks to unitarity (this is not the case in other schemes, such as FTCS, e.g.). Incidentally, notice that starting from an energy espression such as &#xA;$$cp=E$$  with $c$ fixed, you'd get&#xA;$$ic\hbar\partial_x=i\hbar\partial_t$$&#xA;i.e. the advection equation, which has no dispersion (if integrated properly with  Lax-Wendroff methods), and your wavepacket will not spread in time in that case. The quantum analog is the massless-particle Dirac equation. &lt;/p&gt;&#xA;" OwnerUserId="27513" LastActivityDate="2018-04-20T10:04:23.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29339" PostTypeId="1" CreationDate="2018-04-20T10:49:05.873" Score="3" ViewCount="204" Body="&lt;p&gt;&lt;strong&gt;Context:&lt;/strong&gt; I am working on implementing this &lt;a href=&quot;http://jes.ecsdl.org/content/152/1/A255.short&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and I am struggling to come up with a variational formulation for the Butler-Volmer interface conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To simplify my question I consider the following problem:&#xA;\begin{align}&#xA;-\Delta u &amp;amp;= f  &amp;amp;\text{on } \Omega \\&#xA;u &amp;amp;= 0  &amp;amp;\text{on } \partial\Omega \\&#xA;\frac{\partial u}{\partial n} &amp;amp;= g(u)  &amp;amp;\text{on } \Sigma&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the following domain, where $\Omega = \Omega_1 \cup \Omega_2$ and $\Sigma$ denotes the interface between the domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6vyMB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6vyMB.png&quot; alt=&quot;A square domain Omega 1 with a smaller square domain Omega 2 inside. The interface between the two domains is called Sigma.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to derive a variational formulation for this problem, so I can implement it in the FEM software I use (NGSolve). I researched for keywords including mortar and Nitsche methods, transmission conditions and interface conditions, but I'm unsure about the appropriate method and how to adapt it to my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My approach so far&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I write the equation separated for the two domains:&#xA;\begin{align}&#xA;-\Delta u_1 &amp;amp;= f  &amp;amp;\text{on } \Omega_1 \\&#xA;-\Delta u_2 &amp;amp;= f  &amp;amp;\text{on } \Omega_2 \\&#xA;u_1 &amp;amp;= 0  &amp;amp;\text{on } \partial\Omega \\&#xA;u_1 &amp;amp;= u_2  &amp;amp;\text{on } \Sigma \\&#xA;\frac{\partial u_1}{\partial n_1} &amp;amp;= g(u_1)  &amp;amp;\text{on } \Sigma \\&#xA;\frac{\partial u_2}{\partial n_2} &amp;amp;= -g(u_2)  &amp;amp;\text{on } \Sigma&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I derive the variational formulation for the outer domain as:&#xA;\begin{align}&#xA;  \int_{\Omega_1} \nabla u_1 \cdot \nabla v_1 \, dx - \int_{\Sigma} g(u_1) v_1\, ds&#xA;  &amp;amp;= \int_{\Omega_1} f v_1 \, dx \quad \forall v_1 \in H^1_0(\Omega_1)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get a solution for the inner domain I impose the continuity condition on the interface via a mixed formulation:&#xA;\begin{align}&#xA; \int_{\Omega_2} \nabla u_2 \cdot \nabla v_2 \, dx + \int_{\Sigma} \lambda v_2 \, ds&#xA;&amp;amp;= \int_{\Omega_2} f v_2 \, dx \\&#xA; \int_{\Sigma} u_2 \mu \, ds &amp;amp;= \int_{\Sigma} u_1 \mu \, ds&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Am I on the right track? Are there any papers/books considering this problem and outlining different methods?&lt;/p&gt;&#xA;" OwnerUserId="27512" LastActivityDate="2019-08-15T17:01:30.170" Title="Non-linear flux interface condition - variational formulation" Tags="|finite-element|numerical-modelling|computational-chemistry|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29341" PostTypeId="1" AcceptedAnswerId="29348" CreationDate="2018-04-20T17:36:41.390" Score="0" ViewCount="90" Body="&lt;p&gt;This is a continuation of my previous post, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/29277/1d-analytical-solution-vs-fem-solution-for-a-bar-under-compression?noredirect=1#comment53401_29277&quot;&gt;1D analytical solution vs FEM solution for a bar under compression&lt;/a&gt;. For some reason, I cannot comment in it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analytical solution to the 1-D static compression problem is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;u = \frac{du}{dx}x&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x=0$ is at the fixed end, and $x=L$ is at the loaded end. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question this time deals with meshing. In FLUENT, you cannot simulate a structural problem with only 1 column of elements. I believe you need at least 2 elements in each direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you simulate a problem with 9 columns for a square rod, where the square cross-section is divided into 9 smaller square elements. Would you expect that the displacement solution in the compression direction to be the same in all 9 elements or would you expect it to vary? &lt;/p&gt;&#xA;" OwnerUserId="27504" LastEditorUserId="20688" LastEditDate="2018-04-20T17:44:27.793" LastActivityDate="2018-04-22T18:51:07.040" Title="Structural mechanics simulation using FLUENT compared to analytical solution" Tags="|finite-element|mesh-generation|mesh|solid-mechanics|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29342" PostTypeId="2" ParentId="29305" CreationDate="2018-04-20T18:58:26.600" Score="8" Body="&lt;p&gt;Variational problems like this are special cases of optimal control problems, for which there is a huge literature on solution methods and also a good amount of available software. To express it as in the standard form used in optimal control, we may write $t$ rather than $x$ and consider &quot;system dynamics&quot;&#xA;$$ \dot{y} = u $$&#xA;where $u$ is a (fictitious) system input. We can then write the cost functional as&#xA;$$ J = \int_{t_1}^{t_2}L(t,y,u)~dt $$&#xA;which is to be minimised by choosing our system input, $u$. It's common in optimal control problems to have more complex problems such as inequality constraints on $y$ or $u$, so this is actually significantly more general than the original calculus of variations problem you posed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The are many numerical solution methods, but the most commonly used in practice are &quot;direct&quot; methods that express the cost as a sum rather than an integral by some process of discretisation. The most common approaches are known as direct collocation (see e.g. &lt;a href=&quot;https://doi.org/10.2514/3.20223&quot; rel=&quot;noreferrer&quot;&gt;https://doi.org/10.2514/3.20223&lt;/a&gt;) and multiple shooting (e.g. &lt;a href=&quot;https://doi.org/10.1016/S1474-6670(17)61205-9&quot; rel=&quot;noreferrer&quot;&gt;https://doi.org/10.1016/S1474-6670(17)61205-9&lt;/a&gt;). After this we get a nonlinear programming (NLP) problem, which is an optimisation problem that can be solved using standard solvers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're interested in solving practical problems, I would recommend taking a look at the open source toolkit ACADO, which solves optimal control problems by multiple shooting, or CasADi, which is an automatic differentiation toolkit designed for dynamic optimisation that can interface with common NLP solvers such as IPOPT.&lt;/p&gt;&#xA;" OwnerUserId="27138" LastActivityDate="2018-04-20T18:58:26.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29344" PostTypeId="1" CreationDate="2018-04-20T23:50:15.673" Score="3" ViewCount="10006" Body="&lt;p&gt;I wrote a UEL (User Element in Abaqus) for one element and compared to a reference UEL which used standard FEM, where the results agreed satisfactorily, except the reaction force. The stress, strain, strain energy all were pretty accurate.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unable to understand this, how the reaction force is calculated? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am simply restraining the element in the bottom (two nodes) and applying displacement (incrementally) at the top (two nodes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, the stiffness matrix is used to calculate reaction forces in these cases, the stiffness values are quite similar in my UEL and the FEM UEL. Also if this was the case, the other parameters wouldn't have had the same values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would be grateful if anybody could help me figure this out.&lt;/p&gt;&#xA;" OwnerUserId="22448" LastEditorUserId="9667" LastEditDate="2018-04-21T15:58:56.223" LastActivityDate="2018-04-23T07:29:29.787" Title="How does one calculate reaction force in FEA?" Tags="|finite-element|computational-physics|numerical-modelling|newton-method|solid-mechanics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29345" PostTypeId="2" ParentId="29344" CreationDate="2018-04-21T10:54:59.537" Score="6" Body="&lt;p&gt;To calculate the reaction forces at a node, Abaqus (or any structural FE code) simply sums the internal forces for all elements attached to that node. The reaction forces are the negative of that sum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an Abaqus user element, the internal forces for the element are returned from subroutine &lt;code&gt;UEL&lt;/code&gt; in the &lt;code&gt;RHS&lt;/code&gt; array. The returned stiffness matrix (Jacobian), &lt;code&gt;AMATRX&lt;/code&gt;,  is not used &#xA;in the reaction force calculations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, for a simple linear element, the internal force vector will equal the negative of the product of the element stiffness matrix and the nodal displacement vector.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2018-04-21T10:54:59.537" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29346" PostTypeId="1" AcceptedAnswerId="29347" CreationDate="2018-04-21T15:05:38.420" Score="3" ViewCount="2829" Body="&lt;p&gt;What is the complexity (in flops, floating-point operations) of taking the $l_2$-norm of vector $\mathbf{v}\in\mathbb{R}^n$ (or $\mathbf{v}\in\mathbb{C}^n$ if a difference exists).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have the following definition of the $l_2$-norm of $\mathbf{v}$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\lVert\mathbf{v}\rVert_2= \sqrt{|v_1|^2+\dots+|v_n|^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $|v_i|$ is the absolute value of $v_i$ (or complex modulus if $v_i$ complex).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess that what bothers me the most is the square-root computation.&#xA;I found that the square-root is considered one flop, and hence the complexity is $2n$ flops. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isn't that wrong since the square-root should take more time to compute?&lt;/p&gt;&#xA;" OwnerUserId="27396" LastEditorUserId="27396" LastEditDate="2018-04-21T15:26:38.657" LastActivityDate="2018-04-21T16:26:50.973" Title="Time complexity of $l_2$-norm of a vector" Tags="|linear-algebra|floating-point|complexity|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29347" PostTypeId="2" ParentId="29346" CreationDate="2018-04-21T16:26:50.973" Score="4" Body="&lt;p&gt;The square root is counted as a single floating point operation in the &lt;a href=&quot;https://standards.ieee.org/findstds/standard/754-2008.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;IEEE 754-2008 standard&lt;/a&gt; although that does not mean that it takes the same time as other operations. On Intel Xeon processors, the latency (the number of cycles an instruction takes to complete) of a square root might be between 5 to 7 times longer than a sum or a multiplication. You can check this out by comparing the latency reports for the instructions FADD, FMUL, and FSQRT in the &lt;a href=&quot;https://software.intel.com/sites/default/files/managed/ad/dc/Intel-Xeon-Scalable-Processor-throughput-latency.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intel Xeon Scalable Processor throughput latency document&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above assumes that you are implementing the $\ell^2$ norm in the most straight-forward way without using fused-multiply additions, vectorized operations, etc.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2018-04-21T16:26:50.973" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29348" PostTypeId="2" ParentId="29341" CreationDate="2018-04-21T20:06:03.987" Score="0" Body="&lt;p&gt;Let's consider an orthohedric bar defined by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(x,y,z)\in [-w/2,w/2]\times[-d/2,d/2]\times[0, h]\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A vertical load $P$ is applied on the top ($z=h$) surface and the following boundary conditions on the bottom surface:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;u_x(0, y, z) = 0\\&#xA;&amp;amp;u_y(x, 0, z) = 0\\&#xA;&amp;amp;u_z(x, y, 0) = 0\, ,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that would imply that your bottom surface can slide freely (except for the midpoint), the solutions for the differential equations are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;u =\frac{P}{E}(\nu x, \nu y, -z)\, ,\\&#xA;&amp;amp;\epsilon = \frac{P}{E}\begin{bmatrix}\nu &amp;amp;0 &amp;amp;0\\ 0 &amp;amp;\nu &amp;amp;0\\ 0 &amp;amp;0 &amp;amp;-1\end{bmatrix}\, ,\\&#xA;&amp;amp;\sigma = P\begin{bmatrix}0 &amp;amp;0 &amp;amp;0\\ 0 &amp;amp;0 &amp;amp;0\\ 0 &amp;amp;0 &amp;amp;-1\end{bmatrix}\, ,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $P$ is the load, $E$ the Young modulus and $\nu$ the Poisson's ratio. Based on this solution we can see that the displacement is larger for points further away from the center due to the Poisson's effect (except for $\nu=0$).&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-04-22T18:51:07.040" LastActivityDate="2018-04-22T18:51:07.040" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29350" PostTypeId="2" ParentId="29344" CreationDate="2018-04-22T07:19:05.030" Score="4" Body="&lt;p&gt;Once the solution of the problem is known, i.e. you know displacement vector, to calculate reaction/internal forces an integral is evaluated   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\mathbf{f}^\textrm{int} = \sum_{e=1}^{n_e} \int_\Omega \mathbf{B}^\textrm{T}({\boldsymbol\sigma}(\boldsymbol\varepsilon)) \textrm{d}\Omega&#xA;= \sum_{i=1}^{n_e} \sum_{i=1}^{n_g} j_i w_i \left( \mathbf{B}_i^\textrm{T}{\boldsymbol\sigma}(\boldsymbol\varepsilon_i) \right)  &#xA;\end{equation} &#xA;where $j_i$ and $w_i$ is jacobian and integration weight respectively,  $\mathbf{B}$ is differential operator evaluated at integration point $i$, such that&#xA;\begin{equation}&#xA;\boldsymbol\varepsilon_i = \mathbf{B}_i\mathbf{u}^e &#xA;\end{equation}&#xA;where $\mathbf{u}^e$ is vector of nodal degrees of freedom on element $e$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Elements of the vector $\mathbf{f}^\textrm{int}$ which are at constrained degrees of freedom are reaction forces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equations above are general, apply to the linear and nonlinear problem. To make it work you need to provide a physical equation for example in UMAT, i.e. &#xA;\begin{equation}&#xA;{\boldsymbol\sigma} = {\boldsymbol\sigma}( \boldsymbol\varepsilon )&#xA;\end{equation} &lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2018-04-23T07:29:29.787" LastActivityDate="2018-04-23T07:29:29.787" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29351" PostTypeId="2" ParentId="29339" CreationDate="2018-04-22T08:07:11.183" Score="1" Body="&lt;p&gt;Additional terms like yours appear for example when you have some osmotic pressure, and the solution is more straightforward as it looks at the beginning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case I will have edge element, which evaluates the normal gradient magnitude at integration points on Σ. Note Σ has to have an orientation such that $\mathbf{g}=g(u)\mathbf{n}$, where n is the orientation of internal edge element. Having orientation/direction of edge elements consistent is essential. With that at hand, you can evaluate vector&#xA;\begin{equation}&#xA;\mathbf{f}^\Sigma = \sum_{e=1}^{n^e_\Sigma} \int_\Sigma \boldsymbol\varphi^\textrm{T} \left(\mathbf{n}^e \cdot \mathbf{g}(u)\right)&#xA;\textrm{d}\Sigma&#xA;\end{equation}&#xA;where $\boldsymbol\varphi$ is a vector of base functions. $\mathbf{f}^\Sigma$ is added it to the right-hand side. Summation term is to indicate that you loop over edge elements on the internal surface. You have to have assemble $\mathbf{f}^\Sigma$ on the right positions.  If you do not like to implement this from scratch, there are couple codes which can do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need also add terms to your tangent matrix to have quadratic convergence.&#xA;\begin{equation}&#xA;\mathbf{K}^\Sigma = \sum_{e=1}^{n^e_\Sigma} \int_\Sigma \boldsymbol\varphi^\textrm{T} \mathbf{n}^e \cdot \frac{\partial \mathbf{g}}{\partial{u}} \frac{\partial u}{\partial \overline{u}}&#xA;\textrm{d}\Sigma =&#xA;\sum_{e=1}^{n^e_\Sigma} \int_\Sigma \boldsymbol\varphi^\textrm{T} \left(\mathbf{n}^e \cdot \frac{\partial \mathbf{g}}{\partial{u}} \right) \boldsymbol\varphi&#xA;\textrm{d}\Sigma&#xA;\end{equation}&#xA;where $\overline{u}$ are  element edge nodal values and $\mathbf{n}^e$ is element edge unit normal. Field $u$ is evaluated using vector of base functions $\boldsymbol\varphi$ as follows&#xA;\begin{equation}&#xA;u = \boldsymbol\varphi \cdot \overline{u}&#xA;\end{equation}&lt;/p&gt;&#xA;" OwnerUserId="13205" LastEditorUserId="13205" LastEditDate="2018-04-22T11:31:32.340" LastActivityDate="2018-04-22T11:31:32.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29352" PostTypeId="1" AcceptedAnswerId="29353" CreationDate="2018-04-22T16:53:29.887" Score="1" ViewCount="1774" Body="&lt;p&gt;I have a large sparse matrix which is symmetric for the location of non zero values, but the values are different. Could I still use the CG method? I don't have much knowledge of linear algebra, the matrix comes form the discretization of some PDEs and is very large, GMRES method is not appealing because of the need of the calculation with Arnoldi algorithm and storage of the basis.&lt;/p&gt;&#xA;" OwnerUserId="27531" LastEditorUserId="9667" LastEditDate="2018-04-22T18:37:09.873" LastActivityDate="2018-04-22T18:37:09.873" Title="Conjugate Gradient for non symmetric matrix" Tags="|pde|linear-solver|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29353" PostTypeId="2" ParentId="29352" CreationDate="2018-04-22T17:04:30.553" Score="3" Body="&lt;p&gt;The BiConjugate Gradient method (BiCG) or its stabilized variant (BiCGSTAB)  are Krylov subspace methods that are designed to work on non-symmetric matrices. Given a choice between the two, you should use BiCGSTAB, since it tends to converge more quickly. &lt;a href=&quot;https://en.wikipedia.org/wiki/Biconjugate_gradient_stabilized_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Biconjugate_gradient_stabilized_method&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7378" LastActivityDate="2018-04-22T17:04:30.553" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29354" PostTypeId="1" AcceptedAnswerId="29406" CreationDate="2018-04-23T06:09:07.720" Score="2" ViewCount="193" Body="&lt;p&gt;I have to calculate the normal derivative of a function $f(i,j)$ on a domain with an irregular boundary. Let's say something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x x x x&#xA;0 x 0 0&#xA;0 0 0 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;x&lt;/code&gt; indicates a solid boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the correct way to calculate the normal derivative of $f$ at the grid point (1,2) and (3,2)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I simply sum the two partial derivatives?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for the grid point (1,2), would this work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial f}{\partial \hat{n}}\bigg\rvert_{(i=1,~j=2)} = \frac{\partial f}{\partial x}\bigg\rvert_{(i=1,~j=2)} + \frac{\partial f}{\partial y}\bigg\rvert_{(i=1,~j=2)} $$&lt;/p&gt;&#xA;" OwnerUserId="27064" LastEditorUserId="27064" LastEditDate="2018-04-26T00:54:50.717" LastActivityDate="2018-04-28T07:52:23.683" Title="normal derivatives where normal vector is ill-defined" Tags="|finite-difference|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29355" PostTypeId="1" AcceptedAnswerId="29361" CreationDate="2018-04-23T08:29:44.103" Score="6" ViewCount="644" Body="&lt;p&gt;Solving a linear PDE like&#xA;$$&#xA;\Delta u = f \quad\text{on } \Omega,\\&#xA;n\cdot \nabla u = 0 \quad\text{on } \Gamma,&#xA;$$&#xA;with Finite Elements usually goes like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Create the discretization $Au=b$ via&#xA;$$&#xA;\int \nabla u \cdot \nabla v_i = \int f v_i&#xA;$$&#xA;(With the $u$ and $v_i$ from some finite-dimensional trial and test spaces; let's say they the spaces are equal.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Forget everything about the continuous problem and simply solve $Au=b$ with a method of your choosing, e.g., some Krylov method.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;When solving the discretized equation system with an iterative solver, often some norm of the discrete residual is minimized; in the case of MINRES and GMRES, for example, the discrete 2-norm,&#xA;$$&#xA;\|r|\|_2^2 = \|b - Au\|_2^2 = \sum_i \left(\int f v_i - \nabla u \cdot \nabla v_i\right)^2.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Does this norm have any correspondence in the non-discretized world, i.e., is there an entity in terms of $u$ and $f$ to which $\|r\|_2$ is an approximation?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2023-05-03T06:12:38.247" Title="Residual norm of PDE discretization: correspondence in the continuous problem?" Tags="|finite-element|pde|discretization|l2-norm|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29357" PostTypeId="1" CreationDate="2018-04-23T16:34:54.143" Score="4" ViewCount="556" Body="&lt;p&gt;I'm trying to detect dirty floor areas in a series of images, using MATLAB and its Image Processing Toolbox, like the one that follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8TZJl.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8TZJl.jpg&quot; alt=&quot;Clean/Dirty Floor Sample&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the image above, there are two distinct areas, the whiter floor, which is considered clean, and the dirty floor, which is the area beyond the yellow warning sign. I'm looking for ways to detect these differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A relevant problem is that lighting intensity and colour may not be constant along the paths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already tried 2D cross-correlation between a smaller sample image (taken from the image below), but it was not very effective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2GpcH.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2GpcH.jpg&quot; alt=&quot;Clean Floor Sample&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My method was something like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Mask out irrelevant features in input image (corridors other than&#xA;the white ones, for example);&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;RGB to Grayscale conversion of input image (analysis target) and&#xA;full sample image;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Extract a portion of the sample image and extract its histogram;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use histogram equalization on input image, using the sample's&#xA;histogram (histeq);&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run normxcorr2 between input and sample images;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Plot all points below an arbitrary constant (xCorr &amp;lt; 0.8, for&#xA;example) over input image to detect &quot;dirty&quot; areas.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Maybe deep learning for image segmentation would be the most effective method, but I currently don't have the time to implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other suggestions that would be any more effective on this situation?&lt;/p&gt;&#xA;" OwnerUserId="27534" LastActivityDate="2021-11-30T20:08:45.887" Title="Methods to implement floor dirt detection algorithm" Tags="|matlab|image-processing|computer-vision|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29360" PostTypeId="1" CreationDate="2018-04-24T08:56:38.730" Score="3" ViewCount="91" Body="&lt;p&gt;In the context of adaptive mesh in FEM, after a new mesh is created, the data on the old mesh are to be transferred to the new mesh. For the data on the integration points (IPs), it seems the usual way is to (1) transfer the data on IPs of the old mesh to the nodes of the old mesh, (2) use the shape function to transfer the data from the nodes of the old mesh to the nodes of the new mesh, (3) then use the shape function to transfer the data on the nodes of the new mesh to the IPs of the new mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are different techniques for the first step (see (1) above), and the most popular one seems to be the patch method or superconvergent patch recovery. In this method the IPs of the old mesh are used as the sampling points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question would be why not just simply use the patch method to transfer the data on the IPs of the old mesh to the IPs of the new mesh directly, as the IPs of the old mesh are used as sampling points anyway.&lt;/p&gt;&#xA;" OwnerUserId="27545" LastEditorUserId="27545" LastEditDate="2018-04-24T10:12:39.380" LastActivityDate="2018-04-24T10:12:39.380" Title="Data transfer in the context of adaptive mesh in finite element method (FEM)" Tags="|finite-element|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29361" PostTypeId="2" ParentId="29355" CreationDate="2018-04-24T12:30:02.080" Score="5" Body="&lt;h4&gt;TLDR&lt;/h4&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If you're using scalar products in FEM/FVM discretizations, use the mass-matrix scalar product, not &lt;span class=&quot;math-container&quot;&gt;$\ell_2.$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If you're solving FEM/FVM systems with Krylov methods, precondition with the inverse of the mass-matrix.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h4&gt;Some details&lt;/h4&gt;&#xA;&lt;p&gt;It's easy to think that the 2-norm (or in fact any scalar product) &lt;span class=&quot;math-container&quot;&gt;$\|x\|_2$&lt;/span&gt; of a finite-something discretization is an approximation of the &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt;-norm of the corresponding function. That is not the case.&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; be the basis functions of the finite-dimensional space &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt;, and let the coefficients be denoted by Greek letters such that all functions in &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; can be written as &lt;span class=&quot;math-container&quot;&gt;$v = \sum_i \alpha_i v_i$&lt;/span&gt;. Then the natural &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; scalar product is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\langle u, v\rangle_2 = \int u v = \sum_i\sum_j \int (\alpha_i v_i) (\beta_j v_j) = \alpha^T M \beta = \langle\alpha, \beta\rangle_M&#xA;$$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; being the &lt;em&gt;mass matrix&lt;/em&gt;, &lt;span class=&quot;math-container&quot;&gt;$M_{ij} = \int v_i v_j$&lt;/span&gt;. &lt;strong&gt;&lt;em&gt;This&lt;/em&gt; is the scalar product that Krylov methods of discretizations should be formulated in, not &lt;span class=&quot;math-container&quot;&gt;$\alpha^T \beta$&lt;/span&gt; as it's usually done.&lt;/strong&gt; This has multiple consequences:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Symmetry is no longer the determining propery of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, it's self-adjointness in &lt;span class=&quot;math-container&quot;&gt;$\langle\cdot,\cdot\rangle_M$&lt;/span&gt;: Instead of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, one needs to look at the system matrix &lt;span class=&quot;math-container&quot;&gt;$M^{-1}A$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The residual is &lt;span class=&quot;math-container&quot;&gt;$r=M^{-1} F_h(u)$&lt;/span&gt;; calling &lt;span class=&quot;math-container&quot;&gt;$\tilde{r}=F_h(u)$&lt;/span&gt; our &amp;quot;old&amp;quot; residual, one has&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \langle r, r\rangle_M = \langle M^{-1} \tilde{r}, M^{-1} \tilde{r}\rangle_M = \tilde{r}^T M^{-1} \tilde{r}.&#xA;  $$&lt;/span&gt;&#xA;In the case of finite volumes, it's easy to see that this quantity actually approximates the continuous resisdual &lt;span class=&quot;math-container&quot;&gt;$\int F(u)^2$&lt;/span&gt;: The mass matrix is &lt;span class=&quot;math-container&quot;&gt;$M_ij = \delta_{i,j} |\Omega_i|$&lt;/span&gt; (a diagonal matrix with the finite volumes). Since for small &lt;span class=&quot;math-container&quot;&gt;$\Omega_i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$F(u)$&lt;/span&gt; changes very little across &lt;span class=&quot;math-container&quot;&gt;$\Omega_i$&lt;/span&gt; (assuming it's continuous), it can be taken out of the integral&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{split}&#xA;  \tilde{r}^T M^{-1} \tilde{r}&#xA;  &amp;amp;= \sum_i |\Omega_i|^{-1} \left(\int F(u) v_i\right)^2\\&#xA;  &amp;amp;= \sum_i |\Omega_i|^{-1} \left(\int_{\Omega_i} F(u)\right)^2\\&#xA;  &amp;amp;\approx \sum_i |\Omega_i|^{-1} F(u)(x_i)^2 \left(\int_{\Omega_i} 1 \right)^2\\&#xA;  &amp;amp;= \sum_i |\Omega_i| F(u)(x_i)^2\\&#xA;  &amp;amp;\approx \int F(u)^2.&#xA;  \end{split}$$&lt;/span&gt;&#xA;For finite elements, it's harder to see. (Perhaps someone else can fill that in.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The article &lt;a href=&quot;http://etna.mcs.kent.edu/volumes/2011-2020/vol41/abstract.php?vol=41&amp;amp;pages=13-20&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://etna.mcs.kent.edu/volumes/2011-2020/vol41/abstract.php?vol=41&amp;amp;pages=13-20&lt;/a&gt; explains how replacing the scalar product is equivalent to preconditioning.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2023-05-03T06:12:38.247" LastActivityDate="2023-05-03T06:12:38.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29362" PostTypeId="1" CreationDate="2018-04-24T14:05:27.820" Score="0" ViewCount="79" Body="&lt;p&gt;In this problem I am trying to solve, I have a messy set of points distributed in 3D space, each with a defined temperature. If I would want to calculate the heat transfer scenario in this system, how would you recommend me an approach (numerically and with each computational tool)? I thought about triangulating the points and using a unstructured grid Finite Volume Method. Considering the maths around this is a bit harder than that of a structured grid, I assumed the possibility of interpolating the unstructured grid into a structured grid, solving the discretized diffusion model and then interpolating back to the unstructured grid.&lt;/p&gt;&#xA;" OwnerUserId="26600" LastActivityDate="2018-04-24T16:50:35.080" Title="Solving the diffusion/heat equation for a randomly distributed set of points in 3D" Tags="|finite-volume|diffusion|unstructured-mesh|heat-transfer|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29363" PostTypeId="1" CreationDate="2018-04-24T16:04:29.377" Score="1" ViewCount="482" Body="&lt;p&gt;Here is the URL: &lt;a href=&quot;https://github.com/avaneev/biteopt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/avaneev/biteopt&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tested it on numerous global optimization benchmarking functions (included), and on real-world hyperparameter optimization problems I have. Seems to be working quite well except in comparison to deterministic methods it's necessary to make several attempts at different random seeds, so the iteration budget may be high. But stochasticity of this method gives a chance to solve a problem which can't be sufficiently solved by deterministic methods. Anyway, most benchmarking functions solve in 1 attempt given enough iteration budget.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Works best for non-convex problems, and can also solve convex problems, but of course slower than deterministic methods. Can also solve non-linear constrained problems, but such constraints increase convergence time considerably (though, this application was not tested thoroughly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still working on the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to hear comments from users that have some practical models (e.g. black-box hyperparameter optimization) which are still needed to be solved acceptably - whether this method works or not for their models, possibly with the description of the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the description of the method. The algorithm consists of the following elements:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A cost-ordered population of previous solutions is maintained. A solution is an independent parameter vector which is evolved towards a better solution. On every iteration, the best solution is evolved.&#xA;$$x_\text{new}=x_\text{best}$$&#xA;Below, &lt;em&gt;i&lt;/em&gt; is either equal to rand(1, N) or in the range [1; N], depending on the &lt;code&gt;AllpProb&lt;/code&gt; probability. Probabilities are defined in the range [0; 1] and in many instances in the code were replaced with simple resetting counters for more efficiency. Parameter values are internally normalized to [0; 1] range and, to stay in this range, are wrapped in a special manner before each function evaluation. Algorithm's hyper-parameters (probabilities) were pre-selected and should not be changed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Depending on the &lt;code&gt;RandProb&lt;/code&gt; probability, a single (or all) parameter value randomization is performed using &quot;bitmask inversion&quot; operation.&#xA;$$mask= 2^{1+\lfloor(0.999999997-rand(0\ldots1)^4 )\cdot MantSize\rfloor}-1$$&#xA;$$MantMult=2^{MantSize}$$&#xA;$$x_\text{new}[i] = \frac{\lfloor x_\text{new}[i]\cdot MantMult \rfloor \bigotimes mask }{MantMult}$$&#xA;Plus, with &lt;code&gt;CentProb&lt;/code&gt; probability the random &quot;step in the right direction&quot; operation is performed using the centroid vector, twice.&#xA;$$m_1=\text{rand}(0\ldots1)\cdot CentSpan$$&#xA;$$x_\text{new}[i]=x_\text{new}[i]-m_1(x_\text{new}[i]-x_\text{cent}[i])$$&#xA;$$m_2=\text{rand}(0\ldots1)\cdot CentSpan$$&#xA;$$x_\text{new}[i]=x_\text{new}[i]-m_2(x_\text{new}[i]-x_\text{cent}[i])$$&#xA;With &lt;code&gt;RandProb2&lt;/code&gt; probability an alternative randomization method is used.&#xA;$$x_\text{new}[i]=x_\text{new}[i]+(-1)^{s}(x_\text{cent}[i]-x_\text{new}[i]), \quad i=1,\ldots,N,\\ \quad s\in\{1,2\}=(\text{rand}(0\ldots1)&amp;lt;0.5 ? 1:2)$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(Not together with N.2) the &quot;step in the right direction&quot; operation is performed using the random previous solution, current best and worst solutions. This is conceptually similar to Differential Evolution's &quot;mutation&quot; operation.&#xA;$$x_\text{new}=x_\text{best}-\frac{(x_\text{worst}-x_\text{rand})}{2}$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With &lt;code&gt;ScutProb&lt;/code&gt; probability a &quot;short-cut&quot; parameter vector change operation is performed.&#xA;$$z=x_\text{new}[\text{rand}(1,N)]$$&#xA;$$x_\text{new}[i]=z, \quad i=1,\ldots,N$$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;After each objective function evaluation, the highest-cost previous solution is replaced using the cost constraint.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can find this algorithm implemented in the &lt;code&gt;optimize()&lt;/code&gt; function in &lt;code&gt;biteopt.h&lt;/code&gt; on lines 284-395, it does not involve any higher-order math.&lt;/p&gt;&#xA;" OwnerUserId="27549" LastEditorUserId="27549" LastEditDate="2018-04-26T11:37:21.187" LastActivityDate="2018-04-26T11:37:21.187" Title="I've developed a derivative-free optimization method, looking for comments" Tags="|optimization|c++|c|libraries|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="29364" PostTypeId="2" ParentId="28822" CreationDate="2018-04-24T16:30:14.407" Score="0" Body="&lt;p&gt;You may give a try to my method: &lt;a href=&quot;https://github.com/avaneev/biteopt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/avaneev/biteopt&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tested it on a lot of functions, including those with non-linear constraints (but constraint programming requires a special approach which is demonstrated in examples).&lt;/p&gt;&#xA;" OwnerUserId="27549" LastActivityDate="2018-04-24T16:30:14.407" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29366" PostTypeId="2" ParentId="29362" CreationDate="2018-04-24T16:37:32.657" Score="1" Body="&lt;p&gt;If you have a cloud of points and you don't want to use mesh-based methods like FEM or FVM, a possibility is to use a mesh-free method like the Finite Point Method. For instance, you could have a look at this article:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1007/s00466-011-0613-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tatari, M., Kamranian, M., &amp;amp; Dehghan, M. (2011). The finite point method for the p-Laplace equation. Computational Mechanics, 48(6), 689-697.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many other &lt;a href=&quot;https://www.wikiwand.com/en/Meshfree_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;mesh-free methods&lt;/a&gt;, Finite Point Method is only one of them.&lt;/p&gt;&#xA;" OwnerUserId="27547" LastEditorUserId="27547" LastEditDate="2018-04-24T16:50:35.080" LastActivityDate="2018-04-24T16:50:35.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29367" PostTypeId="2" ParentId="26261" CreationDate="2018-04-24T16:48:34.020" Score="0" Body="&lt;p&gt;It may not be an easy thing to do if you are dealing with particle positions (as you've noted in comments above). Example is a Lennard-Jones atom clustering problem which is very hard to solve for even just 150 particles. Such problems are indirectly combinatorial or continuous-combinatorial. You may look for existing solutions to Lennard-Jones problem, maybe you can find something suitable for your application. Generally-available solvers won't acceptably solve such problems.&lt;/p&gt;&#xA;" OwnerUserId="27549" LastActivityDate="2018-04-24T16:48:34.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29368" PostTypeId="2" ParentId="29355" CreationDate="2018-04-24T17:08:09.550" Score="1" Body="&lt;p&gt;I am digressing slightly to address how the equation is discretized before going to the solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;General methods for numerically solving PDEs:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use pseudospectral methods: Approximate the solution as a series expansion (often Fourier or Chebyshev). Most of these series form an orthonormal basis set. Discretize the PDE as a set of matrix equations. This is possible because the differentiation of series expansion is rather straightforward. Solve the matrix equation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is an example using the strong form of the differential equation. $T_n(x)$ represents the Chebyshev polynomial. You could replace that with any other basis set of your choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^2u = f \\&#xA;Let\ u = \sum_{i=1}^Na_iT_{i-1}(x)&#xA;$$&#xA;Here, $a_i$ are the coefficients we need to find. Substituting the value of $u$ in the differential equation, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla^2\sum_{i=1}^Na_iT_{i-1}(x) = f \\&#xA;\sum_{i=1}^Na_i\nabla^2T_{i-1}(x) = f&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, we discretize $x$ and $f$ on a set of collocation points ($0$ to $N$). This leads us to the matrix equation&#xA;$$&#xA;L.a = f&#xA;$$&#xA;where, $L$ is the discretization operator, which contains the second derivatives of the Chebyshev series. In essence, we are choosing a set of $N$ points, where we make the residual zero.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Use Galerkin methods: In addition to approximating the solution as a series expansion, we project our solution onto another vector space, by multiplying the equation with a test function and integrating over the domain. For Galerkin methods, the trial function used to approximate the solution is the same as the test function used for projection. The use of variational form instead of the strong form of PDE is somewhat historical, and somewhat out of convenience. The main advantage is that by integrating and using the Green's theorem, you can reduce one order of the differential equation. Therefore, you do not have to calculate higher order derivatives. The matrix equation you would get is the same as the equation above.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The only difference between finite element (Galerkin) and pseudospectral methods is that the test function in the pseudospectral method is a Dirac delta function. Since you already have the equations for the finite element approximation in the question, I am not writing them here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the general matrix equation we get from any method is:&#xA;$$&#xA;Lu = f&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember that $u$ here is not the solution, but the coefficients of the solution approximated as series expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most direct way to solve this equation using iterative methods is to solve the time-dependent equation:&#xA;$$&#xA;u_t = -Lu + f&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a generalized diffusion equation. Here the time is actually the number of iterations, and we use the analogy of a diffusion equation to iteratively solve the matrix equation. The analogy of iteration = diffusion is very useful because all iterative methods &quot;diffuse&quot; the error until the solution settles down to a steady state value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any stationary, one step iterative method will have the form of &#xA;$$&#xA;u_{n+1} = Gu_n + k\\&#xA;G = I - R^{-1}L \\&#xA;k = R^{-1}f&#xA;$$&#xA;where $I$ is the identity matrix, and $R$ is some easily invertible square matrix (a.k.a preconditioner). The values of $G, k$ would vary depending on the iterative method used. In principle, matrix functions can always be evaluated by using an expansion in terms of the eigenfunctions of the matrix. For the respective eigenvectors and eigenvalues $\phi_i, \lambda_i$ of the square matrix $G$, the stationary, one step method above can be written as:&#xA;$$&#xA;u_{n+1} = \sum_{j=1}^{N}\lambda_ju^n_j\phi_j + k&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We define the error at the nth step as:&#xA;$$&#xA;e_n = u - u_n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting this back to the matrix equation, we get:&#xA;$$&#xA;e_{n+1} \equiv Ge_n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The magnitude of the largest eigenvalue of the iteration matrix $G$ is called the spectral radius of the matrix. Optimizing an iteration is equivalent to minimizing the spectral radius. You can easily see how the spectral radius or the matrix $G$ is related to the coefficients of expansion of our original differential operator. It is related to the set of basis functions, the specific linear combination of which is our solution. What you are minimizing is the coefficient of our approximate solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Source: Chapter 15 &lt;a href=&quot;http://depts.washington.edu/ph506/Boyd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://depts.washington.edu/ph506/Boyd.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27521" LastEditorUserId="27521" LastEditDate="2018-04-24T17:25:04.573" LastActivityDate="2018-04-24T17:25:04.573" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29371" PostTypeId="1" AcceptedAnswerId="29375" CreationDate="2018-04-25T02:02:34.883" Score="1" ViewCount="1500" Body="&lt;p&gt;In Gmsh, I have a test.geo file. The question is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;if I run &lt;code&gt;gmsh -3 test.geo&lt;/code&gt; in the command line, I get the below error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error   : Surface mesh is wrong, cannot do the 3D mesh&#xA;Info    : Done meshing 3D (0.002836 s)&#xA;Info    : 927 vertices 2064 elements&#xA;Error   : ------------------------------&#xA;Error   : Mesh generation error summary&#xA;Error   :     0 warnings&#xA;Error   :     2 errors&#xA;Error   : Check the full log for details&#xA;Error   : ------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;if I partition the &lt;code&gt;test.geo&lt;/code&gt; in Gmsh GUI, sometimes I get the same error as above. But if I reload &lt;code&gt;test.geo&lt;/code&gt; and then &lt;code&gt;3D&lt;/code&gt; it many times, I am sure I can get the right 3-D partition.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;test.geo&lt;/code&gt; file can be obtained from &lt;a href=&quot;https://github.com/fanronghong/connect_local_github/blob/master/test.geo&quot; rel=&quot;nofollow noreferrer&quot;&gt;test.geo file&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how does this happen?&lt;/p&gt;&#xA;" OwnerUserId="23237" LastEditorUserId="20688" LastEditDate="2018-04-25T12:50:01.600" LastActivityDate="2018-04-25T12:50:01.600" Title="A difference running Gmsh in command line and in GUI for 3-D mesh generation" Tags="|mesh-generation|gmsh|" AnswerCount="1" CommentCount="2" ClosedDate="2018-04-25T14:07:35.757" ContentLicense="CC BY-SA 3.0" />
  <row Id="29372" PostTypeId="1" CreationDate="2018-04-25T05:58:19.787" Score="1" ViewCount="1175" Body="&lt;p&gt;I am interested in knowing what the time complexity is (in Big-$\mathcal O$ notation) for solving system of $N$ differential equations? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using &lt;code&gt;ode15s&lt;/code&gt; in MATLAB for this but have no clue about time complexity.&lt;/p&gt;&#xA;" OwnerUserId="27560" LastEditorUserId="20688" LastEditDate="2018-04-25T11:58:14.463" LastActivityDate="2018-04-25T11:58:14.463" Title="System of ordinary differential equations - time complexity of initial value problem" Tags="|matlab|ode|complexity|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29373" PostTypeId="2" ParentId="29372" CreationDate="2018-04-25T07:13:10.877" Score="2" Body="&lt;p&gt;I'm going to assume that you're specifically talking about Initial Value Problems (IVPs) for Ordinary Differential Equations (ODEs), since that's what ode15s() is used for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, a disclaimer: Good ODE solvers have stepsize control procedures that automatically take smaller or larger steps to achieve a desired accuracy, which means that the actual amount of time taken to solve an IVP can vary a lot depending on the details of the problem. Also, the big-O notation hides constants which are often important when comparing performance of different methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, we can easily comment on the time complexity of &lt;em&gt;each step&lt;/em&gt; taken by a solver. Supposing we have an IVP&#xA;$$&#xA;\dot{x} = f(x,t), \qquad x(0) = x_0&#xA;$$&#xA;with $ x \in \mathbb{R}^N $, then for &lt;em&gt;explicit&lt;/em&gt; solvers like Matlab's ode45(), the complexity per step is the same as the complexity of evaluating $f(x,t)$. This is because explicit solvers evaluate $f(x,t)$ a fixed number of times per step. Typically this means a complexity of $O(N^2)$, since each component of $f(x,t)$ can depend on all components of $x$, but is less in some cases. For example, if $f(x,t)$ is 'sparse' in the sense that each component only depends on at most $m$ components of $x$, then the time complexity per step drops to $O(mN)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you use an &lt;em&gt;implicit&lt;/em&gt; solver like ode15s() then a linear system of equations is solved at each step. This involves an $N \times N$ matrix, and the complexity of solving them is $O(N^3)$. This is asymptotically greater than the time required to evaluate $f(x,t)$, so is also the time complexity of the step. Once again though, sparsity can sometimes reduce this for a specific set of ODEs. If each component of $f(x,t)$ only depends on at most $m$ components of $x$, we can often construct the linear system such that it involves a banded matrix of bandwidth $m$, and the time complexity per step reduces to $O(m^2N)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now here's where it gets complicated. If we were to fix a stepsize $\delta t$ and take $M$ steps of that fixed size to solve a particular IVP, then we could get the time complexity of the whole algorithm by multiplying the above step complexities by $M$, for example $O(MN^3)$ in the implicit case. But usually solvers take steps $\delta t$  of different sizes to achieve a desired accuracy. This makes it impossible to say anything about the overall time complexity without also considering details of the specific IVP. Despite the fact that implicit methods like ode15s() have a worse time complexity per step than explicit methods like ode45(), for some problems (called &lt;em&gt;stiff&lt;/em&gt; problems) we find that implicit methods outperform explicit methods as they can take larger steps while maintaining accuracy. Attempts to analyse this usually consider the 'stability' of the numerical method used on a particular ODE for a particular stepsize, as well as the 'order of accuracy' of the method, and there's a very large literature on the subject. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, the usual procedure is to try an explicit solver like ode45() (which is the Dormand-Prince method) and then if the problem is taking a long time to solve, try an implicit method like ode15s(). There's a huge number of different methods and software for solving IVPs, so usually the only way to find out the best method for a particular problem is to benchmark.&lt;/p&gt;&#xA;" OwnerUserId="27138" LastEditorUserId="27138" LastEditDate="2018-04-25T08:11:53.670" LastActivityDate="2018-04-25T08:11:53.670" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29374" PostTypeId="1" CreationDate="2018-04-25T10:02:57.623" Score="2" ViewCount="73" Body="&lt;p&gt;I want to partially diagonalise real sparse symmetric positive-definite matrices, that are of dimension $n = 10^5$ and I need on the order of $k = 500$ of the smallest eigenvalues and eigenvectors. The number of non-zero elements scales with $n$, and optionally I can arrange the matrices to be band-diagonal with the number of bands $\sim200$, independent of $n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am using ARPACK (making no use of the banded structure), but the run-time is becoming prohibitive. I'm wondering whether the &lt;a href=&quot;http://www.feast-solver.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEAST algorithm&lt;/a&gt; might be a good alternative, since having a good idea of what the spectrum will look like, I can confidently supply it with $[\lambda_{\min}, \lambda_{\max}]$. I have looked into other algorithms but am a bit overwhelmed by the number of choices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is twofold. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Could the banded structure make a significant difference?&lt;/li&gt;&#xA;&lt;li&gt;Are there specific algorithms that will outperform ARPACK? (Since as far as I know, it only makes use of the fact that $A$ is real and symmetric, not the other properties. Furthermore, I am not sure that $k = 500$ constitutes &quot;a few eigenvalues&quot; anymore, hence I am wondering whether other algorithms can outperform it.)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27564" LastEditorUserId="20688" LastEditDate="2018-04-25T11:56:25.957" LastActivityDate="2018-04-25T13:19:20.177" Title="Partial diagonalisation of large symmetric positive-definite band-diagonal matrices" Tags="|sparse-matrix|eigenvalues|arpack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29375" PostTypeId="2" ParentId="29371" CreationDate="2018-04-25T12:42:26.503" Score="1" Body="&lt;p&gt;I was totally able to replicate your problem in GMSH 3.0.2.&#xA;In GUI, I would get an error about self-intersecting surface mesh and inability to perform a 3-D meshing about 1 out of 3 times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I don't see anything wrong with your GMSH script, I fixed the normals for the surfaces and surface loop definitions, sometimes that can make a difference (it did not in this case)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I decided to change the meshing algorithm from the default ones to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;MeshAdapt&lt;/code&gt; for generation of the 2-D mesh&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;New Delaunay&lt;/code&gt; for generation of the 3-D mesh&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zZYnX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zZYnX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, I was not able to replicate the problem in 3-D mesh generation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I do not have a concrete answer on what happens with the default options; however, I think that should help you in the immediate term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, find below the script with the fixed normals (it might have contributed) and specified meshing algorithms.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mesh.RandomFactor=1.e-8;&#xA;Mesh.Algorithm = 1; //MeshAdapt&#xA;Mesh.Algorithm3D = 2; //New Delaunay&#xA;&#xA;Point(1) = {0.0, 0.0, 0.0, 0.05};&#xA;Point(2) = {0.5, 0.0, 0.0, 1.0};&#xA;Point(5) = {0.5, 1.0, 0.0, 1.0};&#xA;Point(4) = {0.0, 1.0, 0.0, 0.05};&#xA;Point(10) = {0.0, 1.0, 0.5, 1.0};&#xA;Point(7) = {0.0, 0.0, 0.5, 1.0};&#xA;Point(8) = {0.5, 0.0, 0.5, 1.0};&#xA;Point(11) = {0.5, 1.0, 0.5, 1.0};&#xA;Point(19) = {0.5, 0.2, 0.5, 0.05};&#xA;Point(20) = {0.5, 0.6, 0.5, 0.05};&#xA;Point(3) = {1.0, 0.0, 0.0, 0.05};&#xA;Point(6) = {1.0, 1.0, 0.0, 0.05};&#xA;Point(9) = {1.0, 0.0, 0.5, 1.0};&#xA;Point(12) = {1.0, 1.0, 0.5, 1.0};&#xA;Point(16) = {0.0, 1.0, 1.0, 1.0};&#xA;Point(13) = {0.0, 0.0, 1.0, 1.0};&#xA;Point(14) = {0.5, 0.0, 1.0, 1.0};&#xA;Point(17) = {0.5, 1.0, 1.0, 1.0};&#xA;Point(15) = {1.0, 0.0, 1.0, 1.0};&#xA;Point(18) = {1.0, 1.0, 1.0, 1.0};&#xA;&#xA;&#xA;Line(1) = {1, 2};&#xA;Line(5) = {2, 5};&#xA;Line(9) = {4, 5};&#xA;Line(2) = {1, 4};&#xA;Line(10) = {4, 10};&#xA;Line(15) = {7, 10};&#xA;Line(3) = {1, 7};&#xA;Line(6) = {2, 8};&#xA;Line(14) = {7, 8};&#xA;Line(12) = {5, 11};&#xA;Line(18) = {8, 19};&#xA;Line(34) = {19, 20};&#xA;Line(36) = {20, 11};&#xA;Line(22) = {10, 11};&#xA;Line(4) = {2, 3};&#xA;Line(7) = {3, 6};&#xA;Line(11) = {5, 6};&#xA;Line(8) = {3, 9};&#xA;Line(17) = {8, 9};&#xA;Line(13) = {6, 12};&#xA;Line(20) = {9, 12};&#xA;Line(24) = {11, 12};&#xA;Line(23) = {10, 16};&#xA;Line(28) = {13, 16};&#xA;Line(16) = {7, 13};&#xA;Line(19) = {8, 14};&#xA;Line(27) = {13, 14};&#xA;Line(25) = {11, 17};&#xA;Line(30) = {14, 17};&#xA;Line(32) = {16, 17};&#xA;Line(21) = {9, 15};&#xA;Line(29) = {14, 15};&#xA;Line(26) = {12, 18};&#xA;Line(31) = {15, 18};&#xA;Line(33) = {17, 18};&#xA;Line(35) = {19, 1};&#xA;Line(37) = {4, 20};&#xA;Line(38) = {19, 3};&#xA;Line(39) = {6, 20};&#xA;&#xA;&#xA;Line Loop(1) = {1, 5, -9, -2};&#xA;Plane Surface(1) = {-1};&#xA;Line Loop(2) = {2, 10, -15, -3};&#xA;Plane Surface(2) = {-2};&#xA;Line Loop(3) = {1, 6, -14, -3};&#xA;Plane Surface(3) = {3};&#xA;Line Loop(5) = {5, 12, -18, -34, -36, -6};&#xA;Plane Surface(5) = {5};&#xA;Line Loop(8) = {9, 12, -22, -10};&#xA;Plane Surface(8) = {-8};&#xA;Line Loop(10) = {14, 18, 34, 36, -22, -15};&#xA;Plane Surface(10) = {10};&#xA;Line Loop(4) = {4, 7, -11, -5};&#xA;Plane Surface(4) = {-4};&#xA;Line Loop(6) = {4, 8, -17, -6};&#xA;Plane Surface(6) = {6};&#xA;Line Loop(7) = {7, 13, -20, -8};&#xA;Plane Surface(7) = {7};&#xA;Line Loop(9) = {11, 13, -24, -12};&#xA;Plane Surface(9) = {-9};&#xA;Line Loop(13) = {17, 20, -24, -18, -34, -36};&#xA;Plane Surface(13) = {13};&#xA;Line Loop(11) = {15, 23, -28, -16};&#xA;Plane Surface(11) = {-11};&#xA;Line Loop(12) = {14, 19, -27, -16};&#xA;Plane Surface(12) = {12};&#xA;Line Loop(14) = {18, 34, 36, 25, -30, -19};&#xA;Plane Surface(14) = {14};&#xA;Line Loop(17) = {22, 25, -32, -23};&#xA;Plane Surface(17) = {-17};&#xA;Line Loop(19) = {27, 30, -32, -28};&#xA;Plane Surface(19) = {19};&#xA;Line Loop(15) = {17, 21, -29, -19};&#xA;Plane Surface(15) = {15};&#xA;Line Loop(16) = {20, 26, -31, -21};&#xA;Plane Surface(16) = {16};&#xA;Line Loop(18) = {24, 26, -33, -25};&#xA;Plane Surface(18) = {-18};&#xA;Line Loop(20) = {29, 31, -33, -30};&#xA;Plane Surface(20) = {20};&#xA;Line Loop(21) = {35, 2, 37, -34};&#xA;Plane Surface(21) = {21};&#xA;Line Loop(22) = {38, 7, 39, -34};&#xA;Plane Surface(22) = {22};&#xA;&#xA;&#xA;Surface Loop(1) = {1, 2, 3, 5, 8, 10};&#xA;Volume(1) = {1};&#xA;Surface Loop(2) = {4, -5, 6, 7, 9, 13};&#xA;Volume(2) = {2};&#xA;Surface Loop(3) = {-10, 11, 12, 14, 17, 19};&#xA;Volume(3) = {3};&#xA;Surface Loop(4) = {13, -14, 15, 16, 18, 20};&#xA;Volume(4) = {4};&#xA;&#xA;&#xA;Line{35} In Volume{1};&#xA;Line{37} In Volume{1};&#xA;Line{38} In Volume{2};&#xA;Line{39} In Volume{2};&#xA;Surface{21} In Volume{1};&#xA;Surface{22} In Volume{2};&#xA;&#xA;&#xA;&#xA;Physical Volume(&quot;Darcy&quot;) = {1};&#xA;Physical Volume(&quot;Darcy&quot;) += {2};&#xA;Physical Volume(&quot;Darcy&quot;) += {3};&#xA;Physical Volume(&quot;Darcy&quot;) += {4};&#xA;&#xA;Physical Surface(&quot;BC&quot;) = {1};&#xA;Physical Surface(&quot;BC&quot;) += {2};&#xA;Physical Surface(&quot;BC&quot;) += {3};&#xA;Physical Surface(&quot;BC&quot;) += {4};&#xA;Physical Surface(&quot;MHMSurface&quot;) = {5};&#xA;Physical Surface(&quot;BC&quot;) += {6};&#xA;Physical Surface(&quot;BC&quot;) += {7};&#xA;Physical Surface(&quot;BC&quot;) += {8};&#xA;Physical Surface(&quot;BC&quot;) += {9};&#xA;Physical Surface(&quot;MHMSurface&quot;) += {10};&#xA;Physical Surface(&quot;BC&quot;) += {11};&#xA;Physical Surface(&quot;BC&quot;) += {12};&#xA;Physical Surface(&quot;MHMSurface&quot;) += {13};&#xA;Physical Surface(&quot;MHMSurface&quot;) += {14};&#xA;Physical Surface(&quot;BC&quot;) += {15};&#xA;Physical Surface(&quot;BC&quot;) += {16};&#xA;Physical Surface(&quot;BC&quot;) += {17};&#xA;Physical Surface(&quot;BC&quot;) += {18};&#xA;Physical Surface(&quot;BC&quot;) += {19};&#xA;Physical Surface(&quot;BC&quot;) += {20};&#xA;&#xA;Physical Surface(&quot;f27&quot;) = {21};&#xA;Physical Surface(&quot;f26&quot;) = {22};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-04-25T12:42:26.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29376" PostTypeId="2" ParentId="29374" CreationDate="2018-04-25T13:14:33.503" Score="1" Body="&lt;p&gt;Generally, taking advantage of $A$'s structure (sparsity/bandedness) has no bearing on convergence rates but it will always improve the time to apply a matvec via the &quot;reverse communication API&quot; of ARPACK. Actually, it is not very clear what you might be doing, as $10^5$ is too big to store as an explicit dense matrix, you must be exploiting some kind of structure just to store/apply it, right? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are looking at the smallest eigenpairs, I'd also consider inverse iteration to accelerate convergence to that end of the spectrum (using some flavor of Cholesky decomposition, either banded or sparse, to apply the inverse). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A matrix $10^5$ is not especially large for, say, a 2-D or 3-D FE/FD problem. Computing the Cholesky decomposition of such an A is a matter of routine. Bandedness is a more sensitive matter, as the bandwidth can have strong (superlinear) effect on the time/memory. A $10^5$ tridiagonal matrix is trivial to solve, but a $10^5$ banded matrix with bandwidth on the order $\sqrt N$ is a different animal. Where did your $A$ come from?&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="20688" LastEditDate="2018-04-25T13:19:20.177" LastActivityDate="2018-04-25T13:19:20.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29377" PostTypeId="1" CreationDate="2018-04-25T14:07:10.983" Score="12" ViewCount="423" Body="&lt;p&gt;&lt;em&gt;Only the first two sections of this long question are essential. The others are just for illustration.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Background&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Advanced quadratures such as higher-degree composite Newton–Cotes, Gauß–Legendre, and Romberg seem to be mainly intended for cases where one can finely sample the function but not integrate analytically. However, for functions with structures finer than the sampling interval (see Appendix A for an example) or measurement noise, they cannot compete with simple approaches such as the midpoint or trapezoid rule (see Appendix B for a demonstration). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is somewhat intuitive as, e.g., the composite Simpson rule essentially “discards” one quarter of the information by assigning it a lower weight. The only reason such quadratures are better for sufficiently boring functions is that properly handling border effects outweighs the effect of discarded information.&#xA;From another point of view, it is intuitively clear to me that for functions with a fine structure or noise, samples that are remote from the borders of the integration domain must be almost equidistant and have almost the same weight (for a high number of samples).&#xA;On the other hand, quadrature of such functions may benefit from a better handling of border effects (than for the midpoint method).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Assume that I wish to numerically integrate noisy or fine-structured one-dimensional data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of sampling points is fixed (due to function evaluation being costly), but I can freely place them. However, I (or the method) cannot place sampling points interactively, i.e., based on results from other sampling points. I also do not know potential problem regions beforehand. So, something like Gauß–Legendre (non-equidistant sampling points) is okay; adaptive quadrature is not since it requires interactively placed sampling points.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Have any methods going beyond the midpoint method been suggested for such a case?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or: Is there any proof that the midpoint method is best under such conditions?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;More generally: Is there any existing work on this problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Appendix A: Specific example of a fine-structured function&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I wish to estimate $\int_0^1f(t)\, \mathrm{d}t$ for:&#xA;$$ f(t) = \sum_{i=1}^{k} \frac{\sin(ω_i t-φ_i)}{ω_i},$$&#xA;with $φ_i∈ [0,2π]$ and $\log{ω_i} ∈ [1,1000]$. A typical function looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V50RR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V50RR.png&quot; alt=&quot;superimposed sines&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I chose this function for the following properties:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It can be integrated analytically for a control result.&lt;/li&gt;&#xA;&lt;li&gt;It has fine structure on a level that makes it impossible to capture all of it with the number of samples I am using ($&amp;lt;10^2$).&lt;/li&gt;&#xA;&lt;li&gt;It is not dominated by its fine structure.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Appendix B: Benchmark&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For completeness, here is a benchmark in Python:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;from numpy.random import uniform&#xA;from scipy.integrate import simps, trapz, romb, fixed_quad&#xA;&#xA;begin = 0&#xA;end   = 1&#xA;&#xA;def generate_f(k,low_freq,high_freq):&#xA;    ω = 2**uniform(np.log2(low_freq),np.log2(high_freq),k)&#xA;    φ = uniform(0,2*np.pi,k)&#xA;    g = lambda t,ω,φ: np.sin(ω*t-φ)/ω&#xA;    G = lambda t,ω,φ: np.cos(ω*t-φ)/ω**2&#xA;    f = lambda t: sum( g(t,ω[i],φ[i]) for i in range(k) )&#xA;    control = sum( G(begin,ω[i],φ[i])-G(end,ω[i],φ[i]) for i in range(k) )&#xA;    return control,f&#xA;&#xA;def midpoint(f,n):&#xA;    midpoints = np.linspace(begin,end,2*n+1)[1::2]&#xA;    assert len(midpoints)==n&#xA;    return np.mean(f(midpoints))*(n-1)&#xA;&#xA;def evaluate(n,control,f):&#xA;    &quot;&quot;&quot;&#xA;    returns the relative errors when integrating f with n evaluations&#xA;    for several numerical integration methods.&#xA;    &quot;&quot;&quot;&#xA;    times = np.linspace(begin,end,n)&#xA;    values = f(times)&#xA;    results = [&#xA;            midpoint(f,n),&#xA;            trapz(values),&#xA;            simps(values),&#xA;            romb (values),&#xA;            fixed_quad(f,begin,end,n=n)[0]*(n-1),&#xA;        ]&#xA;&#xA;    return [&#xA;            abs((result/(n-1)-control)/control)&#xA;            for result in results&#xA;        ]&#xA;&#xA;method_names = [&quot;midpoint&quot;,&quot;trapezoid&quot;,&quot;Simpson&quot;,&quot;Romberg&quot;,&quot;Gauß–Legendre&quot;]&#xA;&#xA;def med(data):&#xA;    medians = np.median(np.vstack(data),axis=0)&#xA;    for median,name in zip(medians,method_names):&#xA;        print(f&quot;{median:.3e}   {name}&quot;)&#xA;&#xA;print(&quot;superimposed sines&quot;)&#xA;med(evaluate(33,*generate_f(10,1,1000)) for _ in range(100000))&#xA;&#xA;print(&quot;superimposed low-frequency sines (control)&quot;)&#xA;med(evaluate(33,*generate_f(10,0.5,1.5)) for _ in range(100000))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(I here use the median to reduce the influence of outliers due to functions that have only high-frequency content. For the mean, the results are similar.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The medians of the relative integration errors are:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;superimposed sines&#xA;6.301e-04   midpoint&#xA;8.984e-04   trapezoid&#xA;1.158e-03   Simpson&#xA;1.537e-03   Romberg&#xA;1.862e-03   Gauß–Legendre&#xA;&#xA;superimposed low-frequency sines (control)&#xA;2.790e-05   midpoint&#xA;5.933e-05   trapezoid&#xA;5.107e-09   Simpson&#xA;3.573e-16   Romberg&#xA;3.659e-16   Gauß–Legendre&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note: After two months and one bounty without result, &lt;a href=&quot;https://mathoverflow.net/q/304024/38453&quot;&gt;I posted this on MathOverflow&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2018-09-07T09:54:58.140" LastActivityDate="2019-10-03T15:02:20.340" Title="For noisy or fine-structured data, are there better quadratures than the midpoint rule?" Tags="|reference-request|quadrature|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="29378" PostTypeId="1" AcceptedAnswerId="29381" CreationDate="2018-04-25T18:03:48.493" Score="13" ViewCount="1042" Body="&lt;p&gt;Physics, biology, chemistry etc. have different sets of rules for making experiments: what events are considered relevant, how to avoid contamination of samples, how to create and fix a reproducing process and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are standards, protocols and best practices in ensure accuracy and reproducibility in numerical experiments?&lt;/p&gt;&#xA;" OwnerUserId="22974" LastActivityDate="2018-04-26T12:57:57.300" Title="What are the guidelines for conducting computational experiments?" Tags="|benchmarking|reproducibility|experiment|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29379" PostTypeId="2" ParentId="29363" CreationDate="2018-04-25T19:20:42.777" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I would like to hear comments from users that have some practical models (e.g. black-box hyperparameter optimization) which are still needed to be solved acceptably - whether this method works or not for their models, possibly with the description of the model.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Looks like you want somebody to invest what may be considerable time and energy in trying out your new method on their models, &lt;em&gt;and&lt;/em&gt; report on the results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a modeler, I have to ask myself the question: &lt;em&gt;why&lt;/em&gt; would I do that? It'd take me some time to change my code and link in your method. &lt;em&gt;What&lt;/em&gt; does your optimization method have to offer that may be better than my current method of choice? Now, we can't possibly answer this question because (a) you don't know what models I'm working with, and (b) I don't know what kind of problems your method is good for - beyond some &quot;real-life optimization problems&quot; where it &quot;seems to be working quite well&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottom line is you have to do a heck of a better job &quot;selling&quot; your method to the community at large. You could start by:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;testing the method on a representative sample of problems where you expect it to do well, and report on the results;&lt;/li&gt;&#xA;&lt;li&gt;compare the results with those achieved by similar, known methods (to show us that you're at least competitive with the state of the art);&lt;/li&gt;&#xA;&lt;li&gt;show that your method converges, and under which conditions - if you can't come up with a formal proof, then at least give some heuristic arguments (backed up by some solid experimental data);&lt;/li&gt;&#xA;&lt;li&gt;advise on the choice of hyper-parameters, like &lt;code&gt;RandProb&lt;/code&gt;, &lt;code&gt;CentProb&lt;/code&gt;, etc.&lt;/li&gt;&#xA;&lt;li&gt;tell us what problems we should &lt;em&gt;not&lt;/em&gt; be using your method on, and why.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2018-04-25T19:20:42.777" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29380" PostTypeId="1" AcceptedAnswerId="29389" CreationDate="2018-04-25T19:57:08.467" Score="1" ViewCount="68" Body="&lt;p&gt;Having shape functions $N_i(\xi,\eta), i = 1,...,N_n$ and, a normal vector $n = (n_x,n_y,n_z)$, a thickness function $F_\tau (\zeta), \tau = 1,...,N_\tau$ and nodal variables $\mathbf{Q}_u = (Q_u,Q_v,Q_w)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &#xA;$$&#xA; \int_{\Omega} u \delta u dV  = \int_{\Omega} F_\tau(\zeta) N_i(\xi,\eta) \mathbf{Q}_{u}  F_z(\zeta) N_j(\xi,\eta) \delta \mathbf{Q}_{u} |J| d \xi d\eta d\zeta = &amp;lt;F_\tau F_z&amp;gt; &amp;lt;N_i N_j&amp;gt;  \mathbf{Q}_{u} \delta \mathbf{Q}_{u} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $&amp;lt;F_\tau F_z&amp;gt;$ is a $N_\tau \times N_\tau$ matrix and $&amp;lt;N_i N_j&amp;gt;$ is a $N_n \times N_n$ matrix. This later shape function matrix will interpolate $u$, $v$, and $w$ separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which form should the shape function matrix $N_{un}$ have such that $&amp;lt;N_i n N_{un}&amp;gt;$ is $N_n \times N_n$ and interpolates $u$, $v$, and $w$ separetely in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_{\Omega} u \mathbf{n} \delta u dV = \int_{\Omega} F_\tau(\zeta) N_i(\xi,\eta) \mathbf{Q}_{u} F_z(\zeta) \mathbf{n} N_{un}(\xi,\eta) \delta \mathbf{Q}_{u} |J| d \xi d\eta d\zeta = &amp;lt;F_\tau F_z&amp;gt; &amp;lt;N_i  n N_{un}&amp;gt;  \mathbf{Q}_{u} \delta \mathbf{Q}_{u}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;having in mind that &#xA;$$&#xA;N_{ve} = \begin{bmatrix} N_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; N_2 &amp;amp; ... &amp;amp; N_n &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; N_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; ... &amp;amp; 0 &amp;amp; N_n &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; N_1 &amp;amp; 0 &amp;amp; ... &amp;amp; 0 &amp;amp; 0 &amp;amp; N_n \end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is not a desirable candidate because $&amp;lt;N_i nN_{ve}&amp;gt;$, which interpolates the $u$, $v$, and $w$ conjointly, is $N_n\times 3 N_n$.&lt;/p&gt;&#xA;" OwnerUserId="25099" LastEditorUserId="20688" LastEditDate="2018-04-26T12:43:46.290" LastActivityDate="2018-04-26T21:42:26.130" Title="FE discretisation of normal to displacement vector" Tags="|finite-element|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29381" PostTypeId="2" ParentId="29378" CreationDate="2018-04-25T22:25:22.837" Score="19" Body="&lt;p&gt;A lot has been written about how to design, execute, and report the results of computational experiments.  This has obvious connections with open source software and the broader &quot;open science&quot; movement.  Another important issue is the difference between &quot;my code is faster than yours&quot; research and research that helps us to better understand the properties of algorithms.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some references on computational experiments and testing that you might want to look at include:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coffin, Marie, and Matthew J. Saltzman. 2000. “&lt;a href=&quot;https://pubsonline.informs.org/doi/pdf/10.1287/ijoc.12.1.24.11899&quot; rel=&quot;noreferrer&quot;&gt;Statistical Analysis of Computational Tests of Algorithms and Heuristics&lt;/a&gt;.” INFORMS Journal on Computing 12 (1): 24–44.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Crowder, Harlan, Ron S. Dembo, and John M. Mulvey. 1979. “&lt;a href=&quot;https://dl.acm.org/citation.cfm?id=355833&quot; rel=&quot;noreferrer&quot;&gt;On Reporting Computational Experiments with Mathematical Software&lt;/a&gt;.” ACM Transactions on Mathematical Software (TOMS) 5 (2): 193–203.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feitelson, Dror G. 2006. “&lt;a href=&quot;http://www.cs.huji.ac.il/~feit/papers/exp05.pdf&quot; rel=&quot;noreferrer&quot;&gt;Experimental Computer Science: The Need for a Cultural Change&lt;/a&gt;.”  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hooker, John N. 1995. “&lt;a href=&quot;http://repository.cmu.edu/cgi/viewcontent.cgi?article=1194&amp;amp;context=tepper&quot; rel=&quot;noreferrer&quot;&gt;Testing Heuristics: We Have It All Wrong&lt;/a&gt;.” Journal of Heuristics 1 (1): 33–42.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;McGeoch, Catherine C. 2012. &lt;a href=&quot;https://doi.org/10.1017/CBO9780511843747&quot; rel=&quot;noreferrer&quot;&gt;A Guide to Experimental Algorithmics.&lt;/a&gt; Cambridge University Press.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="20688" LastEditDate="2018-04-26T12:57:57.300" LastActivityDate="2018-04-26T12:57:57.300" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29383" PostTypeId="1" CreationDate="2018-04-26T07:46:12.280" Score="2" ViewCount="91" Body="&lt;p&gt;I have inherited code that solves &lt;code&gt;Eigen::Matrix&lt;/code&gt; problems using the code &lt;a href=&quot;https://eigen.tuxfamily.org/dox/group__LeastSquares.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;shown on this page&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(A.transpose() * A).ldlt().solve(A.transpose() * b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works, but often my matrix size can be $[200 \times 5]$. The $A^T.A$ expression gives me a huge square matrix with 400 times as much data, so I suspect that the solver wastes a lot of time handling that extra data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a &quot;better&quot; solver that avoids creating a square matrix, or can the problem be approached by a &quot;divide and conquer strategy, where perhaps I could solve 40 50x5 problems, then solve the 40x5 results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: For further information, my problem is I have a point cloud from after doing a disparity check on a stereo image, and according to the function name, it is trying to detect a road surface. We solve the matrix, then use the calculated coefficients to weed out noise in each group of points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also strikes me that trying to accurately solve for all 2000 points seems like a lot of effort; perhaps picking each 10th point would give a roughly similar answer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, the 5 row values are something like (IIRC, I'm away from my computer for a week!) $X$, $Y$, $Z$, $X^2$ and $Y^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, note that one operation in itself is not that inefficient, it's that I have over 300 groups of points for a grand total of over 100,000 points per frame.&lt;/p&gt;&#xA;" OwnerUserId="27577" LastEditorUserId="27577" LastEditDate="2018-04-28T04:43:56.730" LastActivityDate="2020-06-19T21:29:47.783" Title="Dense decomposition of very non-square matrices" Tags="|matrix|c++|eigen|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="29384" PostTypeId="2" ParentId="7338" CreationDate="2018-04-26T09:03:57.800" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/nschloe/meshio&quot; rel=&quot;nofollow noreferrer&quot;&gt;meshio&lt;/a&gt; (a small project of mine) has as command-line tool that can do it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;meshio-convert cylinder6.msh out.xml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3980" LastActivityDate="2018-04-26T09:03:57.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29385" PostTypeId="1" AcceptedAnswerId="29386" CreationDate="2018-04-26T10:59:48.217" Score="4" ViewCount="7840" Body="&lt;p&gt;I'm trying to make a 2-D quad-only mesh of an &quot;I&quot;-shaped cross-section, with elements of size 5. So far I've managed to make it quad only, but some elements are very distorted when they shouldn't be. Also, the total number of elements is incorrect. Can you please help me out?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my .geo code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Point(1) = {0, 0, 0, 1.0};&#xA;Point(2) = {45, 0, 0, 1.0};&#xA;Point(3) = {55, 0, 0, 1.0};&#xA;Point(4) = {100, 0, 0, 1.0};&#xA;Point(5) = {100, 10, 0, 1.0};&#xA;Point(6) = {55, 10, 0, 1.0};&#xA;Point(7) = {45, 10, 0, 1.0};&#xA;Point(8) = {0, 10, 0, 1.0};&#xA;Point(9) = {0, 90, 0, 1.0};&#xA;Point(10) = {45, 90, 0, 1.0};&#xA;Point(11) = {55, 90, 0, 1.0};&#xA;Point(12) = {100, 90, 0, 1.0};&#xA;Point(13) = {100, 100, 0, 1.0};&#xA;Point(14) = {55, 100, 0, 1.0};&#xA;Point(15) = {45, 100, 0, 1.0};&#xA;Point(16) = {0, 100, 0, 1.0};&#xA;&#xA;Line(1) = {1, 2};&#xA;Line(2) = {2, 3};&#xA;Line(3) = {3, 4};&#xA;Line(4) = {4, 5};&#xA;Line(5) = {5, 6};&#xA;Line(6) = {6, 11};&#xA;Line(7) = {11, 12};&#xA;Line(8) = {12, 13};&#xA;Line(9) = {13, 14};&#xA;Line(10) = {14, 15};&#xA;Line(11) = {15, 16};&#xA;Line(12) = {16, 9};&#xA;Line(13) = {9, 10};&#xA;Line(14) = {10, 7};&#xA;Line(15) = {7, 8};&#xA;Line(16) = {8, 1};&#xA;&#xA;Line Loop(1) = {14, 15, 16, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13};&#xA;&#xA;Plane Surface(1) = {1};&#xA;&#xA;Recombine Surface {1};&#xA;&#xA;Transfinite Line {2, 4, 16, 12, 10, 8} = 3 Using Progression 1;&#xA;Transfinite Line {14, 6} = 17 Using Progression 1;&#xA;Transfinite Line {1, 15, 3, 5, 7, 9, 13, 11} = 10 Using Progression 1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27425" LastEditorUserId="20688" LastEditDate="2018-04-26T12:45:36.437" LastActivityDate="2019-06-20T14:30:20.697" Title="Creating a proper quad-mesh in GMSH for an &quot;I&quot;-shaped geometry" Tags="|mesh-generation|gmsh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29386" PostTypeId="2" ParentId="29385" CreationDate="2018-04-26T12:18:39.017" Score="6" Body="&lt;p&gt;So, by default and without recombination to quads (comment out &lt;code&gt;Recombine Surface {1}&lt;/code&gt;), GMSH creates a mesh like the one in the left part of the picture. And by recombining surfaces into quad-elements you will obtain the mesh in the right part of the picture. Naturally, if your recombination comes from the original triangular mesh, you would expect the &quot;distorted&quot; quad elements you mentioned, as well as, &quot;incorrect&quot; number of elements in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default, GMSH uses &lt;code&gt;Blossom&lt;/code&gt; recombination algorithm that will be unable to produce your desired mesh from a &quot;messy&quot; mesh like on the left. Notice, that the original mesh on the left satisfies all your requirements in terms of triangles: it actually places the required number of elements over each side (specified by your &lt;code&gt;Transfinite Line&lt;/code&gt; definitions); however, it is free to do whatever it wants inside your I-shaped geometry - and it does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A1Pp6.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A1Pp6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of your options, is to change the meshing algorithm for the original triangular mesh. Now, GMSH supports (experimentally) &lt;code&gt;Delaunay for quads&lt;/code&gt; 2-D meshing algorithm that should produce the mesh you desire (if I undesrstood your intentions correctly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gBtGH.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gBtGH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can select this algorithm either in the GUI: &lt;code&gt;Tools-&amp;gt;Options-&amp;gt;Mesh-&amp;gt;General-&amp;gt;2-D Meshing Algorithm&lt;/code&gt; or by adding the following line to the beginning of your GEO-file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mesh.Algorithm = 8; // Delaunay for quads&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To have the correct output mesh, you should also add the physical surface in the end of your file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Physical Surface(1) = {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-04-26T12:37:28.607" LastActivityDate="2018-04-26T12:37:28.607" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29387" PostTypeId="1" AcceptedAnswerId="29396" CreationDate="2018-04-26T18:03:15.513" Score="7" ViewCount="260" Body="&lt;p&gt;Given a real function $f$, how can one efficiently evaluate $\int_0^{a_i}f(x)dx$ for millions of different $a_i$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying a standard &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_integration&quot; rel=&quot;noreferrer&quot;&gt;quadrature&lt;/a&gt; method (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Simpson%27s_rule&quot; rel=&quot;noreferrer&quot;&gt;Simpson's rule&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Gaussian_quadrature&quot; rel=&quot;noreferrer&quot;&gt;Gaussian quadrature&lt;/a&gt;) will incur an independent cost for each $a_i$, since in general $f(x)$ will need to be evaluated at an entirely different set of points for each $a_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would prefer to pre-generate/calculate a data structure at substantial initial cost, if the integral can then be evaluated from this data structure cheaply, for different $a_i$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possibility I see is to evaluate the integral at a regular array of points $x_j=a_{max}j/n$ (which would allow reuse of points in Simpson's method), and then to &quot;correct&quot; this integral on final evaluation with a small $\int_{a_{max}j/n}^{a_i}f(x)dx$ which can be evaluated with a very low-order Gaussian quadrature because $j$ can be chosen to make this correction interval small. Although probably accurate, it seems like this method may give rise to discontinuities at the half-way points between subsequent $j$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility may be to fit cubic splines to a regular array of points (evaluated from Simpson's method) and then rely on these to interpolate the integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would rather follow an established approach if there is one.&lt;/p&gt;&#xA;" OwnerUserId="9650" LastEditorUserId="20688" LastEditDate="2018-04-26T18:34:10.307" LastActivityDate="2018-04-28T20:01:46.490" Title="Evaluating an integral numerically at many points" Tags="|interpolation|quadrature|integration|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29389" PostTypeId="2" ParentId="29380" CreationDate="2018-04-26T21:42:26.130" Score="0" Body="&lt;p&gt;One can use matrix $N_{ve}$ and program accordingly, i.e., you have to be attentive that in this case, you have the nodal values conjoint and when coding the stiffness matrix these entries should be taken in consideration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You end up with two different kinds of nodal values:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;in the case of $&amp;lt;N_i N_j&amp;gt;, i,j = 1,...,3$ you will have $Q_u = \begin{Bmatrix} u_1 &amp;amp; u_2 &amp;amp; u_3 &amp;amp; v_1 &amp;amp; v_2 &amp;amp; v_3 &amp;amp; w_1 &amp;amp; w_2 &amp;amp; w_3 \end{Bmatrix}$,&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;in the case of $&amp;lt;N_i n N_{ve}&amp;gt;, i,ve = 1,...,3$ you will have $Q_u = \begin{Bmatrix} u_1 &amp;amp; v_1 &amp;amp; w_1 &amp;amp; u_2 &amp;amp; v_2 &amp;amp; w_2 &amp;amp; u_3 &amp;amp; v_3 &amp;amp; w_3 \end{Bmatrix}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25099" LastActivityDate="2018-04-26T21:42:26.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29390" PostTypeId="2" ParentId="29387" CreationDate="2018-04-27T01:18:38.543" Score="3" Body="&lt;p&gt;Let's say you've ordered the points so that $0&amp;lt;a_1&amp;lt;a_2&amp;lt;\ldots&amp;lt;a_n$. Then you have of course that&#xA;$$&#xA;  \int_0^{a_2} f(x) dx&#xA;  =&#xA;  \int_0^{a_1} f(x) dx +&#xA;  \int_{a_1}^{a_2} f(x) dx.&#xA;$$&#xA;In other words, if you have previously computed the integrals from zero to $a_1$ through some numerical method, then computing the integral from zero to $a_2$ can reuse this information, and all you have to do is now compute the integral from $a_1$ to $a_2$, which you can do independently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same applied for all of the other integrals, of course.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-27T01:18:38.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29392" PostTypeId="2" ParentId="29387" CreationDate="2018-04-27T03:11:37.343" Score="1" Body="&lt;p&gt;The standard method is &lt;a href=&quot;http://www.ams.org/journals/mcom/2005-74-250/S0025-5718-04-01724-7/S0025-5718-04-01724-7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;double-exponential sinc quadrature&lt;/a&gt;. The fast convergence requires some regularity assumptions on the integrand, namely that it is the restriction to the real line of a complex differentiable function bounded on a Bernstein ellipse.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2018-04-27T03:11:37.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29393" PostTypeId="1" AcceptedAnswerId="29397" CreationDate="2018-04-27T03:52:13.697" Score="3" ViewCount="1664" Body="&lt;p&gt;With a rather basic knowledge in C++, I have written my own MD simulation code. Currently, I calculate forces in the most naive way: I go through all the atoms and account for their interactions. This of course is an $\mathcal{O}(N^2)$ algorithm, and very inefficient. It wouldn't be an issue, except for my research purposes, I need to simulate 1000 or more atoms, and currently this would take a very very long time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to learn how to implement the neighbor's list method. I have read some of the standard MD books, but they are often in Fortran or some language I cannot decipher. I'm looking for a rather modern and easy-to-follow guide on how to implement this algorithm. It's not an important part of my research so I would like to spend as little time on it as possible. I have to admit, I'm basically asking if there's a quick and dirty way I can learn and implement this algorithm! Does anyone know of such a guide? Or has any experience with implementing the neighbor's list method? &lt;/p&gt;&#xA;" OwnerUserId="26366" LastActivityDate="2022-09-29T19:10:14.957" Title="MD Simulation: Reference for the Neighbor's List Method" Tags="|algorithms|c++|simulation|molecular-dynamics|nearest-neighbors|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29396" PostTypeId="2" ParentId="29387" CreationDate="2018-04-27T06:36:59.427" Score="6" Body="&lt;p&gt;The point of @WolfgangBangerth is exactly what I mentionend in my comment, so I'd always try this first. In the best case, with millions of partitions $[a_{i},a_{i+1}]_{i\in \{0,\ldots,N-1\}}$ (where $a_{i} &amp;lt; a_{i+1}$), the length of these is quite small such that some low-order method could be appropriate. This will require roughly $N \times (s-1)$ function evaluations, where $s$ is the number of evaluations required to integrate one subinterval (e.g., $s=2$ for the trapezoidal rule), and $N \approx 10^6$ should be no problem for a modern PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, in order to map it on a standard problem, you can consider the integral $I(x) = \int_0^x f(t) dt$ in differential form,&#xA;$$&#xA;\frac{\partial I(x)}{\partial x} = f(x), \quad \ I(0) = 0.&#xA;$$&#xA;One can then apply Runge-Kutta or some other standard ODE solver to integrate for $I(x)$ from $x=0$ to $x=a_N$. The steps that are taken in the solution in general won't meet your specified gridpoints $a_i$, thus you need to apply a method called &lt;em&gt;dense output&lt;/em&gt; which is basically a polynomial interpolation with an order that is appropriate for your solver. Dense output usually requires no or only a few additional function evaluations per step, and thus can save you a lot of effort. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's apply this approach to the idea you mentioned: one can integrate the equation using fourth order Runge-Kutta (which in this case directly corresponds to Simpson's rule) with a constant stepsize $h = a_N / M$. Here, the number of steps $M$ should be chosen such that the final solution $I(a_N)$ is accurate, what should be possible with $M \ll N$ in order to justify the overhead over the simple integration mentioned in the beginning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, in each subinterval $[x_{n},\,x_{n+1}]_{n\in \{0,\ldots,N-1\}}$, where $x_{n}=nh$, $I_n=I(x_n)$, $f_n=f(x_n)$, you can apply Hermite interpolation as [see Numerical recipes, 3rd edition, formula (17.2.15)]:&#xA;$$I(x_n+\theta h)=(1-\theta)I_n + \theta I_{n+1} + \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\;\;\\ \theta(\theta-1)\Big((1-2\theta)(I_{n+1}-I_n)+(\theta-1)h f_n + \theta hf_{n+1}\Big)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a cubic interpolation polynomial which is appropriate for the fourth-order Runge-Kutta method -- one order less than the local order is ok, as you don't use the interpolant for taking further steps, see e.g. &lt;a href=&quot;http://homepage.divms.uiowa.edu/~ljay/publications.dir/EACM_Dense_Output.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. All that remains to do is to adjust the parameter $\theta$ such that $a_i = x_n+\theta h$ (you have to do that repeatedly for each $a_i \in [x_n,x_{n+1}]$).&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2018-04-28T20:01:46.490" LastActivityDate="2018-04-28T20:01:46.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29397" PostTypeId="2" ParentId="29393" CreationDate="2018-04-27T10:01:49.450" Score="6" Body="&lt;p&gt;I'd recommend &amp;quot;The Art of Molecular Dynamics Simulation&amp;quot; by D. C. Rapaport. The code samples are written in C. I'm not a huge fan of the programming style of the book, but at least it's not FORTRAN.&#xA;Having said that, my advice would be to take any book where neighbour lists are explained (for instance the Frenkel &amp;amp; Smit book, which I guess it's what you are using now) and just implement the algorithm, which is usually written in pseudo-code.&lt;/p&gt;&#xA;&lt;p&gt;If you want a TL;DR, there are several types of neighbour lists. All of these rely on your interaction potential being 0 outside a certain range &lt;span class=&quot;math-container&quot;&gt;$r_c$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Verlet_list&quot; rel=&quot;nofollow noreferrer&quot;&gt;Verlet lists&lt;/a&gt; (&lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N^2)$&lt;/span&gt; - &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N^{3/2})$&lt;/span&gt; but with a small-ish prefactor):&#xA;&lt;ol&gt;&#xA;&lt;li&gt;do an &lt;span class=&quot;math-container&quot;&gt;$N^2$&lt;/span&gt; sweep where for each particle you fill an array of neighbours that are closer than &lt;span class=&quot;math-container&quot;&gt;$r_c + r_v$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$r_v$&lt;/span&gt; is a parameter and save the current position of each particle (let's call it &lt;span class=&quot;math-container&quot;&gt;$\vec{r}_i(t_0)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Evolve your simulation by using the list of neighbours to compute the forces.&lt;/li&gt;&#xA;&lt;li&gt;After each integration step check whether, for any particle &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$|\vec{r}_i(t) - \vec{r}_i(t_0)| \geq r_v / 2$&lt;/span&gt;. If it is then update &lt;em&gt;all&lt;/em&gt; the lists (see step 1).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cell_lists&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cell lists&lt;/a&gt; (&lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N)$&lt;/span&gt;):&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Divide your simulation box in cells of linear size &lt;span class=&quot;math-container&quot;&gt;$l &amp;gt; r_c$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Use linked lists to assign particles to the cells.&lt;/li&gt;&#xA;&lt;li&gt;For each particle &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, the list of neighbours is given by all the particles that are in &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;'s cells and in each of the (8 in 2D and 26 in 3D) neighbouring cells. Calculate the forces between &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and its neighbours.&lt;/li&gt;&#xA;&lt;li&gt;After each integration step check whether particles have crossed cell boundaries and update the data structure accordingly&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Verlet lists built with cell lists  (usually the best choice, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N)$&lt;/span&gt; with a smaller prefactor than just using cells):&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Step 1 of the &amp;quot;Verlet lists&amp;quot; section is carried out by using cell lists. The linked lists that store the cell data structures are kept updated throughout the simulation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The reason why using Verlet lists instead of cell lists is (in MD simulations) better is that the average number of neighbours is smaller for the former than the latter. The difference is due to the fact that, for each particle, cell lists give you a list of neighbours that are in a volume &lt;span class=&quot;math-container&quot;&gt;$(3l)^3$&lt;/span&gt;, which, if &lt;span class=&quot;math-container&quot;&gt;$r_v$&lt;/span&gt; is chosen wisely, is much larger than the volume of the Verlet sphere (&lt;span class=&quot;math-container&quot;&gt;$4/3 \pi (r_c + r_v)^3$&lt;/span&gt;). Therefore, on average, you compute much fewer distances with Verlet lists.&lt;/p&gt;&#xA;" OwnerUserId="27040" LastEditorUserId="27040" LastEditDate="2022-09-29T19:10:14.957" LastActivityDate="2022-09-29T19:10:14.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29400" PostTypeId="1" CreationDate="2018-04-27T15:05:34.717" Score="2" ViewCount="469" Body="&lt;p&gt;For structural mechanics, such as linear elasticity, I am aware of BCs such as a prescribed displacement (Dirichlet) or a prescribed traction (Neumann). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible that a boundary can have a mixture of prescribed displacement and prescribed traction, i.e., a mixture of Dirichlet and Neumann BCs for the same boundary? It would seem that this would make the problem over-specified? &lt;/p&gt;&#xA;" OwnerUserId="27504" LastEditorUserId="20688" LastEditDate="2018-04-27T17:44:27.610" LastActivityDate="2018-04-30T05:06:29.030" Title="Are mixed boundary conditions possible in structural mechanics?" Tags="|boundary-conditions|solid-mechanics|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="29401" PostTypeId="2" ParentId="83" CreationDate="2018-04-27T16:34:10.720" Score="9" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dkamt.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dkamt.png&quot; alt=&quot;GEKKO Python&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We recently released (2018) the &lt;a href=&quot;http://gekko.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;GEKKO Python package&lt;/a&gt; for nonlinear programming with solvers such as IPOPT, APOPT, BPOPT, MINOS, and SNOPT with active set and interior point methods. One of the issues with using these solvers is that you normally need to provide at least first derivatives and optionally second derivatives. There are several nice modeling languages that can do this for you, as mentioned with other answers. GEKKO compiles the equations to byte code so that it is like you wrote the model in Fortran or C++ in terms of speed. Automatic differentiation provides the 1st and 2nd derivatives in sparse form to the gradient based solvers. We designed GEKKO for optimal control problems but it can also solve problems similar to fmincon. Below is a quick example of a nonlinear programming problem with equality and inequality constraints. First, you'll need to add the GEKKO package with pip install:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install gekko&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://gekko.readthedocs.io/en/latest/quick_start.html&quot; rel=&quot;noreferrer&quot;&gt;Hock Schittkowski problem #71&lt;/a&gt; is shown below as an example of an objective function, inequality constraint, equality constraint, and four variables with upper and lower bounds.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from gekko import GEKKO&#xA;m = GEKKO() # Initialize gekko&#xA;# Initialize variables&#xA;x1 = m.Var(value=1,lb=1,ub=5)&#xA;x2 = m.Var(value=5,lb=1,ub=5)&#xA;x3 = m.Var(value=5,lb=1,ub=5)&#xA;x4 = m.Var(value=1,lb=1,ub=5)&#xA;# Equations&#xA;m.Equation(x1*x2*x3*x4&amp;gt;=25)&#xA;m.Equation(x1**2+x2**2+x3**2+x4**2==40)&#xA;m.Obj(x1*x4*(x1+x2+x3)+x3) # Objective&#xA;m.options.IMODE = 3 # Steady state optimization&#xA;m.solve() # Solve&#xA;print('Results')&#xA;print('x1: ' + str(x1.value))&#xA;print('x2: ' + str(x2.value))&#xA;print('x3: ' + str(x3.value))&#xA;print('x4: ' + str(x4.value))    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;GEKKO works on all platforms (Windows, MacOS, Linux, ARM processors) and with Python 2.7 and 3+. A fully local option is available without an Internet connection by setting the option &quot;remote=False&quot;. The local option is currently only available for Windows and we are working on other versions such as Linux, MacOS, ARM processors to run locally without an Internet connection. The local version includes only free solvers that do not require a license. By default, the problem is sent to a public server where the solution is computed and returned to Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although this question is specifically about solving nonlinear programming in Python, I'll also highlight a few other types of problems that GEKKO can solve and some resources for learning optimization. GEKKO also solves mixed-integer and differential algebraic equations and has several pre-programmed objects for advanced controls (similar to DMC, RMPCT, etc). Modes of operation include data reconciliation, real-time optimization, dynamic simulation, and nonlinear predictive control. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I teach two courses on optimization (&lt;a href=&quot;https://apmonitor.com/me575&quot; rel=&quot;noreferrer&quot;&gt;design optimization&lt;/a&gt; and &lt;a href=&quot;https://apmonitor.com/do&quot; rel=&quot;noreferrer&quot;&gt;dynamic optimization&lt;/a&gt;) and have posted the course material online. The dynamic optimization course is offered each year starting in January and we use the GEKKO Python package (and MATLAB) for the course. GEKKO is an extension of the APMonitor Optimization Suite but has integrated the modeling and solution visualization directly within Python. &lt;a href=&quot;http://apmonitor.com/wiki/index.php/Main/APMonitorReferences&quot; rel=&quot;noreferrer&quot;&gt;APMonitor and GEKKO references&lt;/a&gt; give a sample of the types of applications that can be solved with this package. GEKKO is developed under the &lt;a href=&quot;https://www.nsf.gov/awardsearch/showAward?AWD_ID=1547110&quot; rel=&quot;noreferrer&quot;&gt;National Science Foundation Research Grant #1547110&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="906" LastEditorUserId="906" LastEditDate="2018-05-09T00:07:37.133" LastActivityDate="2018-05-09T00:07:37.133" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29402" PostTypeId="1" AcceptedAnswerId="29421" CreationDate="2018-04-27T19:38:22.070" Score="2" ViewCount="765" Body="&lt;p&gt;I am studying the reverse mode of&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Automatic_differentiation&quot; rel=&quot;nofollow noreferrer&quot;&gt;automatic differentiation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reverse mode of automatic differentiation allows the efficient computation of a the derivative of a single dependent variable $y$ with respect to as many independent variables $x_i$ as you want.&#xA;One assigns to each intermediate variable $v$ an adjoint variable $\bar{v}$ which is the derivative of a chosen dependent variable with respect to the subexpression $$\bar{v} \rightarrow \bar{v} = \frac{\partial y}{\partial v}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so the assignment $$y = v_1 \sin(v_2)$$ corresponds to the adjoint assignments&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\bar{v_1} &amp;amp;=&amp;amp; \bar{y} \sin(v_2) \\&#xA;\bar{v_2} &amp;amp;=&amp;amp; \bar{y} v_1 \cos(v_2)&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\bar{y} = 1$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in the situation where you solve a linear system in the program:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Ax = b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $y$ might be another function of $x$: $$y(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to this &lt;a href=&quot;http://www.scicomp.uni-kl.de/codi/d0/db5/TutorialA1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; of the &lt;em&gt;CoDiPack&lt;/em&gt; software, the corresponding adjoint statements are&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\bar{A} &amp;amp;=&amp;amp; - \lambda x^T \\&#xA;\bar{b} &amp;amp;=&amp;amp; \lambda \\&#xA;\end{eqnarray}&#xA;where $\lambda$ is the the solution of the adjoint equation &#xA;$$A^T \lambda = \bar{x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the same algorithm in several other documents, for example&#xA;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0377042700004222&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; in section 7, &lt;em&gt;Iteration and equation solving&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not clear to me how to arrive at these statements. &#xA;I think the derivation must be similar as in the case where one wants to optimize $w(x)$ where $x$ is subject to the constraint $$Ax = b$$&#xA;See for example this &lt;a href=&quot;http://www.piercelab.caltech.edu/assets/papers/ftc00.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;document&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="23586" LastEditorUserId="23586" LastEditDate="2018-05-04T16:37:00.483" LastActivityDate="2018-05-04T16:52:39.390" Title="Automatic Differentiation - reverse accumulation of linear system solve" Tags="|linear-solver|linear-system|automatic-differentiation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29403" PostTypeId="1" CreationDate="2018-04-27T19:55:33.237" Score="2" ViewCount="183" Body="&lt;p&gt;Let's consider the FEM discretization of the Laplace operator &lt;em&gt;without&lt;/em&gt; boundary conditions, i.e.,&#xA;$$&#xA;a(u,v) = \int_\Omega \nabla u \cdot \nabla v - \int_\Gamma (n\cdot \nabla u) v.&#xA;$$&#xA;For one-dimensional domains, the corresponding matrix has two empty rows (the first and the last), generally in $n$ dimensions the matrix has a nullspace of dimension $n+1$. This appears to suggest that, for example in 2D, Dirichlet conditions in only three separate points are necessary to make the operator nonsingular. Numerical experiments with various domains support this suspicion, e.g. a triangular domain with Dirichlet conditions at the corners:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wzINC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wzINC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hfq0m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hfq0m.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This is with linear elements; quadratic elements require more points.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any results on this?&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2018-04-27T20:03:26.677" LastActivityDate="2018-04-29T08:41:01.193" Title="FEM-Laplace with Dirichlet in only a few points: Nonsingular operator?" Tags="|finite-element|boundary-conditions|eigenvalues|discretization|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="29405" PostTypeId="2" ParentId="26484" CreationDate="2018-04-28T05:53:23.730" Score="2" Body="&lt;p&gt;This problem is rather non-trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The low-rank representation of $X_{n\times n}=A_{n\times r}B^T_{r\times n}$ with the rank $r$ does not lead to an easy way of finding the minimum and maximum entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for a particular case of $r=1$, $X_{n\times n}=uv^T$, $u,v\in \mathbb R^n$, you can find $k$ smallest and $k$ largest entries in $v$ in $\mathcal O(kn)$ and then taking into account the sign of $u_i$ find $k$ smallest or largest entries in the $i$th row in $\mathcal O(k)$, for $i=1,\ldots,n$. So, for $r=1$ you certainly can cut down the calculations to $\mathcal O(kn)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, when $r&amp;gt;1$, the things become tricky. The &lt;a href=&quot;https://mathoverflow.net/questions/76122/finding-minimum-or-maximum-element-of-a-low-rank-matrix&quot;&gt;following question on MathOverflow&lt;/a&gt; discusses options to compute the max\min element of the whole matrix, which is a very similar problem. &lt;a href=&quot;https://mathoverflow.net/a/85501/121862&quot;&gt;One of the answers&lt;/a&gt; gives an exact algorithm for $r=2,3$ that uses known techniques to find a convex hull in 2 and 3 dimensions that can be also adopted for your problem. But I doubt that this particular approach can be successfully generalized to higher dimensions in an efficient way ($r&amp;gt;3$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://mathoverflow.net/a/86065/121862&quot;&gt;The other answer&lt;/a&gt; mentions a pretty interesting idea of using an $\epsilon$-net of an $r$-dimensional sphere to map each of the rows/columns of matrices $A$ and $B$ to compute the minimum\maximum in $\mathcal O(nr+(1/\epsilon)^r)$ with $\epsilon$ error. I am yet to try this option, but it technically makes sense and should be extendible to finding $k$ values one row at a time (by mapping only 1 row of $A$ to the net at a time, while keeping records of $B$ there). But if any decent accuracy is required, the method cost increases very fast, as it is exponential in terms of $r$, which will make it practical only for $r\ll\ll\ll n$ for any decent $\epsilon$.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-04-29T14:27:35.567" LastActivityDate="2018-04-29T14:27:35.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29406" PostTypeId="2" ParentId="29354" CreationDate="2018-04-28T07:46:42.633" Score="2" Body="&lt;p&gt;First of all, there is a distinction between a continuous boundary having regions with ill-defined normal derivative - and its discretization having regions will ill-defined derivatives. The question describes a bit confusing discretized boundary with one point sticking out. I have a hard time imagining a boundary that could be discretized this way to solve some problem with just one point sticking out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A classic example of a continuous boundary in 2D with such problems would be a &lt;em&gt;corner&lt;/em&gt;, where at a &lt;em&gt;corner point&lt;/em&gt; the direction of the normal will be undefined; thus, resulting in an ill-defined normal derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if the normal is ill-defined -&gt; the normal derivative is ill-defined as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, a lot depends on what you actually intend to do with your normal derivatives. For example, you might want to enforce some boundary conditions on them or use their values in some other method that requires them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think that taking the definition of a normal derivative at such bad point as a sum of two partial derivatives has any physical meaning. For a crude approximation, you might consider your corner normal to be defined either as $\hat{n}=\hat{x}$ or $\hat{n}=\hat{y}$, effectively assigning a corner to a certain part of the boundary having a well-defined normal. I have also seen averaging out the values of those derivatives (which will be 0.5 times your proposed result) being done. However, that's a decision not to handle the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some approaches would avoid having the corner point at all (having only points very close to the point with the undefined normal) or instead of a normal derivative, a reformulation with adding additional equations might be required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1016/0377-0427(90)90043-Y&quot; rel=&quot;nofollow noreferrer&quot;&gt;L. J. Gray, E. Lutz, &quot;On the treatment of corners in the boundary element method,&quot; &lt;em&gt;J. Comput. Appl. Math.&lt;/em&gt;, vol. 32, no. 3, Dec. 1990, pp. 369–386&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;features a nice overview (or references) of different techniques to handle those situations for corners, which might be useful for your needs.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-04-28T07:52:23.683" LastActivityDate="2018-04-28T07:52:23.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29407" PostTypeId="2" ParentId="29403" CreationDate="2018-04-28T14:48:51.723" Score="2" Body="&lt;p&gt;You are solving the following equation:&#xA;$$-\textrm{div}(\vec{grad}\, u) = -\Delta u= 0 \quad \forall x \in\Omega\tag{P}$$&#xA;in a bounded domain $\Omega$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore we &lt;strong&gt;must&lt;/strong&gt; have the following BC:&#xA;$$u=u_D\quad\forall x\in\partial\Omega_D\qquad \frac{\partial u}{\partial n}=u_N \quad\forall x \in\partial\Omega_N$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to note that differential operators applied to bounded domains do not make sense without BC's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But for now, let's abandon the BC's&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying Gauss Th. one sees that &#xA;$$\int_{\partial \Omega}\frac{\partial u}{\partial n}\,d\sigma = 0\tag{*}$$&#xA;This equation is the equilibrium equation that ensures the net &quot;heat&quot; flux through the boundary adds to $0$. This equation is always present in every conservation principle that does not depend on time. This flux is already &quot;fixed&quot;. This restriction is the price one has to pay for $\Omega$ to be bounded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, at least, if the normal derivative $\partial_n u$ is specified along the whole boundary $\partial\Omega$, the condition $(*)$ must be fulfilled for the problem $(P)$ to be meaningful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We notice that the solution to the problem $(P)$ with the restriction $(*)$ has a unique solution up to a constant, because the problem is invariant w.r.t. the transformation $u\to u+c$, and therefore we have only one DOF that has to be fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Infinite domain&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here we have no restrictions and the equation $(P)$ holds everywhere. For example, every solution $u$ is of the type of the real or imaginary part of $f(z)$ with $z=x+iy$ being $f$ analytic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Another point of view&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the 1D problem and play with matrices. For the moment assign the matrix &#xA;$$A=\frac{d}{dx}$$&#xA;and its transpose &#xA;$$A^T = -\frac{d}{dx}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The operator $A$ (rectangular, because it does not have any imposed BC) is applied to the vector $u$ to obtain the fluxes $q=Au$. In physics there is always an equilibrium of sth. e.g. in heat problems the equilibrium means that at every node fluxes must add to 0: $A^Tq=0$. This is the Laplace equation $(P)$ in discrete space:&#xA;$$A^TAu=Lu=0$$&#xA;Since $L$ is symmetric, you can calculate its eigenvalues easily. You will find that only one eigenvalue is $0$ and the kernel of $L$ is equal to the vector $u=1$. here is your DOF.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-04-29T08:41:01.193" LastActivityDate="2018-04-29T08:41:01.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29409" PostTypeId="1" CreationDate="2018-04-29T03:18:16.090" Score="1" ViewCount="213" Body="&lt;p&gt;I am attempting to use the method of manufactured solutions (MMS) for code verification for linear elasticity. However, this is more of a general question regarding the general use of MMS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MMS, the manufacturer makes up some solution that satisfies the PDE, and then we obtain values for BCs and sources that we feed into the numerical code and solve. Does the manufacturer have the freedom in choosing whether a boundary is a Dirichlet or Neumann boundary condition, and then evaluating the manufactured solution at the boundary in accordance with their choice of the boundary condition type? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in 1-D steady heat-conduction, I manufacture some solution $T(x)$. Do I have the freedom of using either Dirichlet BC at $x=0$ or Neumann BC? For a Dirichlet BC, I simply evaluate $T(x)$ at $x=0$ and that's my BC. On the other hand, I could instead use a Neumann BC at $x=0$ so I would then evaluate $\frac{dT}{dx}$ at $x=0$. Is the manufacturer in full of control of what type of BC is to be prescribed? &lt;/p&gt;&#xA;" OwnerUserId="27504" LastEditorUserId="20688" LastEditDate="2018-04-29T04:17:09.967" LastActivityDate="2018-04-29T04:17:09.967" Title="Method of manufactured solutions - choice of type of boundary conditions" Tags="|boundary-conditions|verification|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29410" PostTypeId="2" ParentId="26484" CreationDate="2018-04-29T15:31:39.180" Score="2" Body="&lt;p&gt;A couple of possible approaches:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;N. J. Higham and S. D. Relton, &lt;a href=&quot;https://doi.org/10.1137/15M1053645&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Estimating the largest elements of a matrix&quot;, &lt;em&gt;SIAM J. Sci. Comput.&lt;/em&gt;, vol. 38, no. 5, pp. C584–C601.&lt;/a&gt; Alternative link for a &lt;a href=&quot;http://eprints.maths.manchester.ac.uk/2424/1/paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;S. A. Goreinov, I. V. Oseledets, D. V. Savostyanov, E. E. Tyrtyshnikov, and N. L. Zamarashkin, &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary;jsessionid=09D8CC8B7977922A73590DDD45F59667?doi=10.1.1.505.6808&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;How to find a good submatrix,&quot; Research Report 08-10, ICM HKBU, Kowloon Tong, Hong Kong.&lt;/a&gt; Alternatively, link to a chapter in &lt;a href=&quot;https://doi.org/10.1142/9789812836021_0015&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matrix Methods&lt;/a&gt; book.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="20688" LastEditDate="2018-04-30T05:01:53.793" LastActivityDate="2018-04-30T05:01:53.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29411" PostTypeId="1" CreationDate="2018-04-29T15:41:12.477" Score="1" ViewCount="806" Body="&lt;p&gt;I have several questions regarding suitable boundary conditions for linear elasticity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read that in order for modeling linear elasticity to be well-posed, the entire boundary cannot be traction/Neumann boundary conditions, and that at least part of it needs to be specified as a fixed/Dirichlet boundary condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first question is: Does at least part of the boundary need to be &quot;fixed,&quot; as in &quot;zero&quot; displacement, or could the boundary have a specified non-zero displacement Dirichlet condition and still have the problem be well-posed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My second question is: Physically, couldn't you have applied loadings to all parts of a boundary? For example you could have a rod and apply a compression/load on all surfaces? Computationally, to model this, all of the boundaries would then be a Neumann traction condition? &lt;/p&gt;&#xA;" OwnerUserId="27504" LastActivityDate="2018-04-30T11:13:04.490" Title="Well-posedness of linear elasticity boundary conditions" Tags="|boundary-conditions|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29412" PostTypeId="2" ParentId="29402" CreationDate="2018-04-29T17:02:19.170" Score="1" Body="&lt;p&gt;The identity connecting forward and reverse mode for $x=\text{solve}(A,b)$ is $$\bar x^⊤\dot x=\bar A(\dot A)+\bar b^⊤\dot b.$$ Where $\bar A$ is a linear functional on matrices which may be realized as the $trace(\bar A^⊤\dot A)$ which is related to the (Frobenius) scalar product for matrices $\langle X,Y\rangle=trace(X^⊤Y)=trace(XY^⊤)$, which is kind-of the euclidean scalar product as it is the sum over the element-wise product of $X$ and $Y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With $b=Ax$ the forward differentiation gives&#xA;$\dot b=\dot A x+A\dot x$, inserting you get &#xA;$$&#xA;\bar x^⊤\dot x=\bar A(\dot A)+\bar b^⊤(\dot A x+A\dot x). \tag{1}&#xA;$$ &#xA;As that has to be valid for all choices of $\dot A, \dot x$, one gets&#xA;\begin{align}&#xA;0&amp;amp;=trace(\bar A^⊤\dot A)+ \bar b^⊤\dot A x&#xA;\\&#xA;&amp;amp;=trace(\bar A^⊤\dot A)+ trace(\bar b^⊤\dot A x)&#xA;\\&#xA;&amp;amp;=trace(\bar A^⊤\dot A)+ trace(x\bar b^⊤\dot A)&#xA;\\&#xA;&amp;amp;=trace((\bar A+\bar bx^⊤)^⊤\dot A),&#xA;\\\text{and }~\bar b A\dot x&amp;amp;=\bar x\dot x \tag{2}&#xA;\end{align}&#xA;which implies &#xA;$$&#xA;\bar b^⊤A=\bar x^⊤\iff \bar b=\text{solve}(A^⊤,\bar x),\tag3&#xA;\\&#xA;\bar A = -\bar bx^⊤.&#xA;\\&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2018-05-04T16:52:39.390" LastActivityDate="2018-05-04T16:52:39.390" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="29413" PostTypeId="2" ParentId="29411" CreationDate="2018-04-29T19:57:11.533" Score="2" Body="&lt;p&gt;That's a really nice question :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main equation that is being solved here is:&#xA;$$\rho \frac{\partial^2 \vec{u}}{\partial t^2}-\textrm{div}(\sigma(\vec{u}))=\vec{f}\tag{1}$$&#xA;with $\sigma(\vec{u})$ the elastic tensor that describes the behaviour of the material that is being considerated. This is given by a constitutive law. And $\vec{f}$ the internal forces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation has (as is natural) two kind of BC:&#xA;$$\vec{u}=\vec{u}_D \quad \forall \vec{x}\in\partial \Omega\qquad \sigma\vec{n}=\vec{t}\quad \forall \vec{x}\in\partial\Omega\tag{BC}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that are imposed in the Dirichlet boudary where the displacements are fixed and in the Neumann boundary where tractions are known, respectively. There are as well two initial conditions, but they are not so important here...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In dynamics (equation $(1)$ you can impose whatever, because it is not a requirements that the body is at rest... but what happens if the body does?&#xA;This is statics... and its behaviour is described by the next equation:&#xA;$$-\textrm{div}(\sigma(\vec{u}))=\vec{f}\tag{2}$$&#xA;subjected to the same BC as before $(BC)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now everything change...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will ask you using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Physics&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could a body be at rest without moving? The answer is yes!&#xA;If the resultant of the applied forces adds to 0, then the body is in equilibrium. For this to happen, there is no need to specify any Dirichlet BC  but in one node (to specify the spatial reference).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, equilibrium of forces must be present:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\vec{F}=\int_{\Omega}\vec{f}\,dV=-\int_{\partial\Omega}\vec{t}\,dS=\vec{T}\tag{*}$$&#xA;This means that the resultant of the body forces $\vec{F}$ must exert a force equal but in oposite direction respect to the external forces that are applied to the body (as a boundary condition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this occurs we don't have anything to be afraid of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, a rope, whose extrema are being equally pulled does not remains at equilibrium  even the fact that in their boundaries two traction BC are imposed. Of course we must specify in which spatial point the rope is, otherwise there would be infinite solutions to the problem in which the rope would be translated in space. In this example, if the rope is hung under gravity, in the lower extremum the BC would be $\vec{t}_l=\sigma \vec{n}=\vec{0}$ (no applied force), while in the upper $\vec{T}=\vec{t}_u-\vec{t}_l=\vec{F}=\int_0^{L}\rho \vec{g}\,dz$ (force exerted by the whole mass that it supports). This problem will have a unique solution if one point of the rope is known, otherwise the rope could be in equilibrium wherever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mathematics&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is so easy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apply the Gauss Th. to $(2)$ and apply the $(BC)$. If to Dirichlet BC are imposed, we obtain the restriction:&#xA;$$\int_{\Omega}\left[-\textrm{div}(\sigma(\vec{u}))\right]dV=-\int_{\partial\Omega}\sigma\vec{n}\,dS=-\int_{\partial\Omega}\vec{t}\,dS=\vec{T}=\int_{\Omega}\vec{f}\,dV=\vec{F}$$&#xA;Which is the same equation deduced above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to @origimbo to remember me the rotations!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, apart from the equilibrium of forces, there must be an equilibrium of torques! Therefore, the provided boundary conditions must be consistent with this equilibrium:&#xA;$$\int_{\Omega}\vec{x}\times\vec{f}\,dV=-\int_{\partial\Omega}\vec{x}\times\vec{t}\,dS\tag{**}$$&#xA;with respect to an arbitrary origin of coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, this last statement may be deduced taking the cross product of $(2)$ with an arbitrary vector $\vec{x}$, integrating by parts and noting that $\sigma = \sigma^{T}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the provided $BC$ (neumann only) given by $(BC)$ fulfills $(*)$ and $(**)$ then your problem is well posed once fixed one node.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-04-30T11:13:04.490" LastActivityDate="2018-04-30T11:13:04.490" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="29414" PostTypeId="1" AcceptedAnswerId="29416" CreationDate="2018-04-30T01:14:01.540" Score="1" ViewCount="47" Body="&lt;p&gt;Does it really matter which techniques we use in the process of GA optimization? For instance, if I use the Roulette Wheel Technique instead of Tournament Method for &lt;em&gt;selection&lt;/em&gt;, two-point crossover instead of one-point for &lt;em&gt;crossover&lt;/em&gt;, and let’s say a single-point mutation instead of inversion method for &lt;em&gt;mutation&lt;/em&gt;, will I still get the same optimum solutions? Why does it matter then and how do we decide on which methods to use in each step? &lt;/p&gt;&#xA;" OwnerUserId="27618" LastActivityDate="2018-04-30T03:32:39.857" Title="How to decide on what techniques to adopt in Genetic Algorithm optimization" Tags="|optimization|genetic-algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29416" PostTypeId="2" ParentId="29414" CreationDate="2018-04-30T03:32:39.857" Score="1" Body="&lt;p&gt;GA optimization is considered as a meta-heuristic algorithm and depending on the algorithms chosen for &lt;em&gt;initialization&lt;/em&gt;, &lt;em&gt;selection&lt;/em&gt;, &lt;em&gt;mutation&lt;/em&gt;, and &lt;em&gt;crossover&lt;/em&gt; and the parameters for each of the steps, a different optimum solution might be obtained in a particular run. With GA, one rarely performs it as a single-pass; unless any feasible solution (not necessarily very close to optimal) is a goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Though for a lot of problems, the obtained solutions will be close to each other, the GAs tend to converge to local optimums rather than the global one. Multiple techniques exist to correct/exploit this behaviour, from the choice of the methods for GA fundamental operations and their parameters to simple multiple runs of GA optimization with seeding the initial populations in different regions of likelihood optimums.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the convergence of GA will be very different for different techniques; thus, the number of populations to converge, required population samples will vary quite a lot. Moreover, different techniques perform very differently for various problems and fitness function types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A large variety of methods and parameters for GA optimization is explained by their metaheuristic nature. If there are no known studies for a performance of GA for a particular problem, one has to try those options one-by-one in order to get a feasible solution of a required quality in decent time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For particular applications, you might find existing literature discussing different options and their performance – and with experience, one can start making connections and find similarities between different application areas. But, in general, the choice of the methods for fundamental GA steps - is trial and error, unless known otherwise.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-04-30T03:32:39.857" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29417" PostTypeId="1" CreationDate="2018-04-30T04:05:21.570" Score="6" ViewCount="1013" Body="&lt;p&gt;From what I understand, the Gauss-Newton method is used to find a search direction, then the step size, etc., can be determined by some other method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to that, are the following statements also true?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;the Gauss-Newton method always results in a direction of strict descent&lt;/li&gt;&#xA;  &lt;li&gt;the Gauss-Newton method only requires that the directional derivative of the objective function exist – but you do not need to&#xA;  compute it.&lt;/li&gt;&#xA;  &lt;li&gt;the Gauss-Newton method does not require the Hessian.&lt;/li&gt;&#xA;  &lt;li&gt;When you’re faced with an optimization problem of the form $\min ||F(x)||$, and $F(x)$ is non-linear, then Gauss-Newton is a good choice&#xA;  because it doesn’t require you to compute the directional derivative.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any other help is appreciated. My colleagues are in disagreement about those statements.&lt;/p&gt;&#xA;" OwnerUserId="27864" LastEditorUserId="20688" LastEditDate="2018-04-30T04:18:40.683" LastActivityDate="2018-05-05T03:20:54.420" Title="What is required of the objective function in order to use Gauss Newton method?" Tags="|optimization|convex-optimization|constrained-optimization|conjugate-gradient|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29418" PostTypeId="2" ParentId="19180" CreationDate="2018-04-30T04:36:55.553" Score="1" Body="&lt;p&gt;I would definitely advise going FFTW-route anyway. FFTW is a high-quality library with good documentation that can be used in a convenient way in your project without requiring many modifications or lots of additional code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since you explicitly asked about an FFTW alternative, you can go Intel MKL route. Now, Intel Math Kernel Library offers &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-fourier-transform-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fourier Transform functions&lt;/a&gt; both for &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-fft-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;serial/shared memory&lt;/a&gt;, as well as &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-cluster-fft-functions&quot; rel=&quot;nofollow noreferrer&quot;&gt;for distributed memory&lt;/a&gt; cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, Intel MKL supports &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-appendix-d-fftw-interface-to-intel-math-kernel-library&quot; rel=&quot;nofollow noreferrer&quot;&gt;FFTW3 interfaces&lt;/a&gt; (at least for shared memory), so that you have an option of using FFTW through Intel MKL anyway.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-04-30T04:36:55.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29419" PostTypeId="2" ParentId="29400" CreationDate="2018-04-30T05:06:29.030" Score="2" Body="&lt;p&gt;You can't prescribe &lt;em&gt;both&lt;/em&gt; Dirichlet and Neumann conditions, but you can prescribe a Robin-type boundary condition in which the normal stress (traction) is proportional to the displacement. You can think of this as a case where each point at the boundary is tethered by a little spring to its undeformed location, with the springs producing a force that is proportional to the displacement from the undeformed configuration. An example of how this could be done is when your body is in contact with another body.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-04-30T05:06:29.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29420" PostTypeId="1" AcceptedAnswerId="29423" CreationDate="2018-04-30T09:00:39.083" Score="1" ViewCount="2307" Body="&lt;p&gt;I am trying to derive the formula for the second-order second derivative of the function $f(z)$ in the case of non-uniform spacings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start by considering that, around $z=\zeta_k$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(z)=f(\zeta_k)+(z-\zeta_k)~f'(\zeta_k)+\frac{(z-\zeta_k)^2}{2!}f''(\zeta_k)+O((z-\zeta_k)^3)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the primes indicate a differentiation with respect to $z$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I evaluate this formula at $\zeta_{k-1}$ and $\zeta_{k+1}$ and take the difference of the two to have an expression for $f''(\zeta_{k})$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(\zeta_{k-1}) = f(\zeta_{k-1})+ (\zeta_{k-1}-\zeta_{k})~f'(\zeta_{k})+\frac{(\zeta_{k-1}-\zeta_{k})^2}{2}f''(\zeta_{k})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(\zeta_{k+1}) = f(\zeta_{k-1})+ (\zeta_{k+1}-\zeta_{k})~f'(\zeta_{k})+\frac{(\zeta_{k+1}-\zeta_{k})^2}{2}f''(\zeta_{k})$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f''(\zeta_{k}) = 2\frac{f(\zeta_{k+1})-f(\zeta_{k-1})}{(\zeta_{k+1}-\zeta_{k})^2 (\zeta_{k-1}-\zeta_{k})^2} +2\frac{\mathbf{f'(\zeta_k)}\cdot(\zeta_{k+1}-\zeta_{k})(\zeta_{k-1}-\zeta_{k})}{(\zeta_{k+1}-\zeta_{k})^2 (\zeta_{k-1}-\zeta_{k})^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am not sure now on how to get rid of the first derivative on the rhs of the last equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use the Taylor expansion a the beginning to derive an expression for $f'(\zeta_k)$, by stopping at $O((z-\zeta_k)^2)$, to plug into my last equation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I see with this is that I would be mixing up two different levels of accurancy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there another way of doing this?&lt;/p&gt;&#xA;" OwnerUserId="27064" LastEditorUserId="27064" LastEditDate="2018-05-02T01:06:26.713" LastActivityDate="2018-05-02T01:06:26.713" Title="second derivative with non-uniform spacing" Tags="|finite-difference|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29421" PostTypeId="2" ParentId="29402" CreationDate="2018-04-30T09:17:43.937" Score="1" Body="&lt;p&gt;Starting with $b=Ax$ the forward differentiation gives&#xA;$\dot b=\dot A x+A\dot x$, or, with $\dot y = \dot A x$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dot x = -A^{-1} \dot y + A^{-1} \dot b$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's write this in terms of the local transformation Jacobian for the tangent linear variables $\left( \dot A, \dot y, \dot b, \dot x\right)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ &#xA; \begin{matrix}&#xA;  \dot A \\ \dot y\\ \dot b\\ \dot x\\&#xA; \end{matrix}&#xA;\right] &#xA;=&#xA;\left[ &#xA;  \begin{matrix}&#xA;    1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    x &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0 &amp;amp; 0 &amp;amp; 1 &amp;amp; 0 \\&#xA;    0 &amp;amp; -A^{-1} &amp;amp; A^{-1} &amp;amp; 0 \\&#xA;    \end{matrix}&#xA;\right]&#xA;\left[ &#xA; \begin{matrix}&#xA;  \dot A \\ \dot y\\ \dot b\\ \dot x\\&#xA; \end{matrix}&#xA;\right] &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The adjoint operator is then the transpose matrix acting on the (local) adjoint variables $\left( \bar A, \bar y, \bar b, \bar x\right)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[ &#xA; \begin{matrix}&#xA;  \bar A \\ \bar y\\ \bar b\\ \bar x\\&#xA; \end{matrix}&#xA;\right] &#xA;=&#xA;\left[ &#xA;  \begin{matrix}&#xA;    1 &amp;amp; x^T &amp;amp; 0 &amp;amp; 0 \\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -A^{-T} \\&#xA;    0 &amp;amp; 0 &amp;amp; 1 &amp;amp; A^{-T} \\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    \end{matrix}&#xA;\right]&#xA;\left[ &#xA; \begin{matrix}&#xA;  \bar A \\ \bar y\\ \bar b\\ \bar x\\&#xA; \end{matrix}&#xA;\right], &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align}&#xA;\bar y &amp;amp;= -A^{-T} \bar x \\&#xA;\bar A &amp;amp;\mathrel{+}= x^T \bar y \\&#xA;\bar b &amp;amp;\mathrel{+}= A^{-T} \bar x \\&#xA;\bar x &amp;amp;= 0.&#xA;\end{align} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reference:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Giering and Kaminski, &lt;a href=&quot;http://www.fastopt.com/papers/racc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Recipes for Adjoint Code Construction&lt;/em&gt;&lt;/a&gt;, ACM&#xA;  TOMS, 1998&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2018-04-30T09:17:43.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29422" PostTypeId="2" ParentId="20166" CreationDate="2018-04-30T10:23:53.150" Score="0" Body="&lt;p&gt;You should use the discrete counterpart of the pressure-correction method so that no boundary conditions are needed for pressure (like in the original problem). Then this problem is avoided.&lt;/p&gt;&#xA;" OwnerUserId="27621" LastActivityDate="2018-04-30T10:23:53.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29423" PostTypeId="2" ParentId="29420" CreationDate="2018-04-30T10:47:59.590" Score="2" Body="&lt;p&gt;The resulting scheme, once the system is solved, is first order accurate. Try to add more nodes to your stencil.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use a 4 node stencil, you will have your desired accuracy level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine, your function $f(x)$ can be written as a polynomial of 3rd order:&#xA;$$f(x) = f_{0}L_0(x)+f_1L_1(x)+f_2L_2(x)+f_3L_3(x)+\epsilon(\xi)$$&#xA;Where $L_i$ are the Lagrange basis polynomials, and for some $\epsilon$ within the interval defined by the interpolation support.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interpolation error $\epsilon$ is written as follows:&#xA;$$\epsilon(\xi) = \frac{\prod_{i=0}^{3}{(x_i-\xi)}}{4!}f^{4)}(\xi)$$&#xA;If the function $f(x)$ is smooth enough the error is of the order of $\epsilon\sim \bar{h}^4$, where $\bar{h}$ is a kind of average of the spacings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While computing the second derivative of $f(x)$ we'll find that the error is of the order of $\epsilon''(\xi)\sim \bar{h}^2$, as desired.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-04-30T10:47:59.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29424" PostTypeId="1" CreationDate="2018-04-30T11:28:03.933" Score="4" ViewCount="1209" Body="&lt;p&gt;I have a 1st order ODE system. One of the equations is piecewise function of 2 of the dependent variables. I try to solve it in Python environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}\dot x_1 &amp;amp;= x_2 - x_3\\&#xA;\dot x_2 &amp;amp;= x_1 + x_4\\&#xA;\dot x_3 &amp;amp;= \left\{&#xA;\begin{array}{ll}&#xA;(x_2-x_{4})^{(0.5)} &amp;amp; x_2\ge x_{4}\\&#xA;0, &amp;amp; x_2\lt x_{4}&#xA;\end{array}&#xA;\right.\\&#xA;\dot x_4 &amp;amp;= x_3 - x_1\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Actually this is not my real ODE system. I just fabricate it to sketch my problem. &lt;a href=&quot;https://math.stackexchange.com/questions/2752803/what-type-of-differential-equation-system-is-this-and-how-to-solve-them&quot;&gt;Here is my real ODE system&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read carefully the book A Primer of Scientific Programming with Python. In that book it is stated that we must write ODEs like that : \begin{align}\dot u &amp;amp;= f(u,t)\end{align} For scalar ODEs u and f correspond to float objects, and to arrays for system of ODEs. But didn't mention if the right hand side is piecewise function of unknown variables.&lt;/p&gt;&#xA;" OwnerUserId="27594" LastEditorUserId="9667" LastEditDate="2018-04-30T14:35:44.920" LastActivityDate="2018-05-04T01:31:29.783" Title="ODE system with discontinuous right-hand-side" Tags="|python|ode|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="29425" PostTypeId="1" AcceptedAnswerId="29428" CreationDate="2018-04-30T15:39:44.250" Score="1" ViewCount="222" Body="&lt;p&gt;The stiffness matrix of $Ax=B$ system of linear equations, where $A$ is an $n\times n$ symmetric matrix stored in the form of symmetric skyline matrix, that is associated with a finite element model of 3-D elastic solids, when reordered with the reverse Cuthill–McKee algorithm doesn't result in a skyline matrix with limited RAM storage space requirements as would happen with a 2-D problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best algorithm for the 3-D case?&lt;/p&gt;&#xA;" OwnerUserId="27625" LastEditorUserId="20688" LastEditDate="2018-04-30T16:24:24.750" LastActivityDate="2018-04-30T19:09:20.713" Title="Reordering algorithm for minimization of ram usage of a skyline matrix" Tags="|finite-element|linear-solver|sparse-matrix|memory-management|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29426" PostTypeId="1" CreationDate="2018-04-30T17:48:30.150" Score="1" ViewCount="49" Body="&lt;p&gt;I have set up a mathematical model describing the diffusion of ozone out of a gas bubble. The bubble is surrounded by a thin gas film. So actually, the model describes the diffusion of ozone through this gas film. &#xA;The mathematical model is created by discretizing the volume from $V_0$ to $V_1$. Where $V_0$ represents the outer volume of the bubble, and $V_1$ represents the outer volume of the gas film (surrounding the bubble). The discretized scheme consists of $N$ equally spaced volume elements from $V_0$ to $V_1$ (finite difference method): &#xA;\begin{equation}&#xA;    \Delta V = \frac{V_1 - V_0}{N}&#xA;\end{equation}&#xA;The volume of the bubble changes as a function of the amount of ozone leaving the bubble, which in turn changes as a function of time. &#xA;The volume of the bubble and the amount of ozone inside the bubble is linked by the ideal gas law: &#xA;\begin{equation}&#xA;    V_0 = \frac{n_\text{total}(t) \cdot R \cdot T}{P}&#xA;\end{equation}&#xA;$n_\text{total}$: the total amount of gas inside the bubble, $T$: the temperature, $R$: the gas constant, and $P$: the pressure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bubble does not only contain ozone. It also contains inert gases, so that: &#xA;\begin{equation}&#xA;    n_\text{total} = n_\text{ozone}(t) + n_\text{inert}&#xA;\end{equation}&#xA;$n_\text{inert}$ will remain constant and only $n_\text{ozone}(t)$ will change over time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There should be a limit for how much the volume can change before numerical errors will start to occur. Beyond this limit, the discretization scheme should break down and cause errors. How do I express this limit? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the limit given by: &#xA;\begin{equation}&#xA;    Ratio = \frac{V_{0,initial}}{V_{1,initial}}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So that the volume change must not exceed the ratio of the two initial volumes of the bubble and the gas film?&lt;/p&gt;&#xA;" OwnerUserId="26970" LastEditorUserId="26970" LastEditDate="2018-04-30T19:21:33.100" LastActivityDate="2018-04-30T19:21:33.100" Title="Limit to volume change in a discretized mathematical model?" Tags="|finite-difference|discretization|numerical-modelling|diffusion|numerical-limitations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29427" PostTypeId="1" CreationDate="2018-04-30T18:15:49.243" Score="2" ViewCount="788" Body="&lt;p&gt;I am trying to perform an Integral of Hypergeometric function &lt;code&gt;2F1(a,b,c,x)&lt;/code&gt; from -1 to 1 for some good values of $a,b,c$ (lets say $a=1,b=2,c=3$) . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did it in Python and Mathematica and the answer is finite although it is divergent at the point $x=1$. Anyway, I need to do this integration as a part of my very long code in c++. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using GSL (thank you! creators of this amazing library) to do so. Since this function is divergent at the upper bound of my integration ($x=1$), I used the &lt;code&gt;CQUAD&lt;/code&gt; to calculate the integral; however because the output of Hypergeometric &lt;code&gt;2F1&lt;/code&gt; for $x=1$ is neither &lt;code&gt;NaN&lt;/code&gt; nor &lt;code&gt;Infinity&lt;/code&gt;, I get the following error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gsl: hyperg_2F1.c:685: ERROR: domain error &#xA;Default GSL error handler invoked.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any of you guys can help me to resolve this problem? &lt;/p&gt;&#xA;" OwnerUserId="27626" LastEditorUserId="20688" LastEditDate="2018-09-25T13:27:00.867" LastActivityDate="2018-09-27T15:17:46.113" Title="Integration of a diverge function in c++ GSL Library" Tags="|c++|integration|special-functions|gsl|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="29428" PostTypeId="2" ParentId="29425" CreationDate="2018-04-30T19:09:20.713" Score="2" Body="&lt;p&gt;For direct solution in 3D, you should probably be using some flavor of nested dissection (ND) or minimum degree (MD). These attack the storage requirements of A=LL' factorization directly, not the bandwidth (which has only an indirect effect on fill-in). On it's own, bandwidth reduction is just not strong enough to make 3D direct solve tractable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good ND codes to try are METIS or Scotch, while AMD is easiest to access MD code. Many sparse direct solvers will expose/reference these reordering packages directly among their input parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a good &quot;turn-key&quot; package that hides/handles all this detail for you, I'd recommend Intel's MKL PARDISO. Although it is closed-source, binaries are available under pretty liberal license terms.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2018-04-30T19:09:20.713" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="29429" PostTypeId="2" ParentId="29424" CreationDate="2018-04-30T19:27:57.700" Score="3" Body="&lt;p&gt;Most (if not all) ODE solvers have certain continuity requirements on the right-hand side. If this is not the case, an adaptive solver will usually regulate the step size smaller and smaller (and eventually fail due to hitting the minimum step size or similar). The main exception is if the integration step exactly hits the point of discontinuity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, your function is only discontinuous in the derivative of the right-hand side, but this discontinuity is pretty severe (being a jump from 0 to ∞).&#xA;You may try just implementing the discontinuity verbatim, but this will likely fail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The better alternatives are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Smooth the discontinuity by using a very sharp &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;sigmoid&lt;/a&gt; instead. This is probably also more realistic anyway.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use event-detection to locate the point of discontinuity and swap the right-hand side at this point.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2018-04-30T19:27:57.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="29430" PostTypeId="1" CreationDate="2018-04-30T20:00:19.873" Score="3" ViewCount="179" Body="&lt;p&gt;I have a sparse (density = 0.2%), ill-conditioned system that I am trying to solve, with no luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a sequence of sampled data, where two of every 8 samples have been zeroed due to a bug. The sequence is two sinusoids in the presence of white noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'd like to do is determine the Signal to Noise Ratio of the original sequence. I understand that there is information that is forever lost, and so I won't get an exact answer, but that's acceptable, as long as I can get something reasonably close using the existing data, without any filtering (i.e. no interpolation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Strategy&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having a sequence with missing samples as I do is like multiplying the original sequence with a square wave (75% duty cycle, in my case). In the frequency domain, this is equivalent to a periodic convolution of regularly-spaced impulses with the DFT of my original sequence. Essentially, its like doing circular shifts on the input sequence, multiplying particular points by certain weights according to the square wave DFT, and summing them together. This can be represented as a linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I've generated the sparse matrix representing the convolution, and am now trying to solve &lt;code&gt;Ax=b&lt;/code&gt; where &lt;code&gt;A&lt;/code&gt; is the sparse matrix, &lt;code&gt;x&lt;/code&gt; is the DFT of the original sequence, and &lt;code&gt;b&lt;/code&gt; is the DFT of the corrupt sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the sparse matrix is ill conditioned. Using matlab, neither &lt;code&gt;mldivide&lt;/code&gt; nor &lt;code&gt;svds&lt;/code&gt; provided useful answers. The thing is, I know what the input sequence (and its DFT) look like. It's a DC component, two bin-centered sinusoids, and white noise. So I have an excellent initial guess, I just can't figure out a way to iterate on that. Also worth noting is that although the DFT is complex-valued, and these operations are all on complex numbers, I only care about the magnitudes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I solve &lt;code&gt;mag(A)x=mag(b)&lt;/code&gt; I get a decent result, but its slightly too optimistic, so I'm hoping an iterative solution on the complex values will be more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My criteria for an acceptable solution is one that minimizes the mean square error between the magnitudes of  &lt;code&gt;Ax&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated! I have little experience in these sorts of linear systems, but am doing my best to learn the appropriate methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code I'm using to test my method: &lt;a href=&quot;https://pastebin.com/F3bArgnR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/F3bArgnR&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27627" LastEditorUserId="27627" LastEditDate="2018-04-30T20:21:18.490" LastActivityDate="2018-04-30T20:21:18.490" Title="Iteratively solving a sparse, ill-conditioned system" Tags="|linear-algebra|matlab|linear-solver|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="29431" PostTypeId="2" ParentId="29427" CreationDate="2018-05-01T02:29:10.863" Score="3" Body="&lt;p&gt;I am not a frequent user of GSL, but I think I made it work by wrapping the hypergeometric function &lt;code&gt;2F1&lt;/code&gt; provided by GSL in such a way, that it would return &lt;code&gt;NaN&lt;/code&gt; for $x=1$. Then, the numerical integration will use this information for its purposes and the &lt;code&gt;domain error&lt;/code&gt; will not be thrown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, (following the example provided in GSL docs) in function &lt;code&gt;double f&lt;/code&gt; I compared the passed argument $x$ with a domain bound &lt;code&gt;sing_value&lt;/code&gt; (don't judge the naming) in the floating-point sense. If they are sufficiently close, I return &lt;code&gt;NaN&lt;/code&gt;, otherwise, I call the provided by GSL &lt;code&gt;2F1&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_integration.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_sf_hyperg.h&amp;gt;&#xA;#include &amp;lt;limits&amp;gt;&#xA;&#xA;&#xA;double f (double x, void * params) {&#xA;  double alpha = *(double *) params;&#xA;  double sing_value = 1.0;&#xA;  double diff = std::abs(x-sing_value);&#xA;  bool singular_point = (diff &amp;lt;= std::numeric_limits&amp;lt;double&amp;gt;::epsilon()*std::abs(x+sing_value)*2);&#xA;  // as per Kirill's comment, the comparison &#xA;  // (diff &amp;lt; std::numeric_limits&amp;lt;double&amp;gt;::min()) &#xA;  // is unnecessary for any practical sense.&#xA;  if (singular_point)&#xA;    return std::numeric_limits&amp;lt;double&amp;gt;::quiet_NaN();&#xA;  else&#xA;    return gsl_sf_hyperg_2F1(1,2,3,alpha*x);&#xA;}&#xA;&#xA;int&#xA;main (void)&#xA;{&#xA;  gsl_integration_cquad_workspace * wcquad = gsl_integration_cquad_workspace_alloc (10000);&#xA;&#xA;  double result, error;&#xA;  size_t nevals;&#xA;  double expected = log(16);&#xA;  double alpha = 1.0;&#xA;&#xA;  gsl_function F;&#xA;  F.function = &amp;amp;f;&#xA;  F.params = &amp;amp;alpha;&#xA;&#xA;  gsl_integration_cquad (&amp;amp;F, -1.0, 1.0, 0, 1e-14, wcquad, &amp;amp;result, &amp;amp;error,&amp;amp;nevals);&#xA;&#xA;  printf (&quot;result          = % .18f\n&quot;, result);&#xA;  printf (&quot;exact result    = % .18f\n&quot;, expected);&#xA;  printf (&quot;estimated error = % .18f\n&quot;, error);&#xA;  printf (&quot;actual error    = % .18f\n&quot;, result - expected);&#xA;  printf (&quot;# evaluations   = %zu\n&quot;, nevals);&#xA;&#xA;  gsl_integration_cquad_workspace_free (wcquad);&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;According to Wolfram Alpha,&#xA;$$&#xA;\int_{-1}^1 {}_2F_1(1,2,3,x)dx = \ln 16&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which seems to be confirmed by the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result          =  2.772588722239701653&#xA;exact result    =  2.772588722239781145&#xA;estimated error =  0.000000000000023683&#xA;actual error    = -0.000000000000079492&#xA;# evaluations   = 2553&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code is not optimized in terms of tolerances, subdivisions and, especially, it being an awful mixture of C/C++ for no reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tested on gcc 6.4 and GSL 2.4.2&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-05-02T14:59:49.157" LastActivityDate="2018-05-02T14:59:49.157" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29436" PostTypeId="1" AcceptedAnswerId="29504" CreationDate="2018-05-02T07:54:55.390" Score="0" ViewCount="253" Body="&lt;p&gt;I've done an extensive web search, but could not find an option. Conversion between model formats is available, but conversion of objective function and constraint equations into C code seems absent. This is needed to run complex models a little bit more effectively utilizing processor-specific compiler optimizations.&lt;/p&gt;&#xA;" OwnerUserId="27549" LastActivityDate="2018-05-12T10:00:03.997" Title="Looking for a converter of AMPL/GAMS models into C code" Tags="|c++|c|libraries|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29437" PostTypeId="2" ParentId="29417" CreationDate="2018-05-02T14:43:14.647" Score="11" Body="&lt;p&gt;Just to clarify notation, I'll be discussing the Gauss-Newton method for the problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min \phi(x)=(1/2) \| F(x) \|_{2}^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the search direction $p$ computed as the solution to the linear system of equations &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$J(x^{(k)})^{T}J(x^{(k)}) p = - J(x^{(k)})^{T} F(x^{(k)})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $J(x)$ is the matrix of partial derivatives of components of $F(x)$ with respect to the components of $x$.  In the following, I'll drop the superscript $(k)$ notation to save some typing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that $\phi(x)$ and the individual components of $F(x)$ are  continuously differentiable, then by some elementary vector calculus, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \phi(x)=J(x)^{T}F(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and assuming that $\phi(x)$ and the individual components of $F(x)$ are twice continuously differentiable,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla^{2} \phi(x)= J(x)^{T}J(x)+ \sum_{i=1}^{m} F_{i}(x) \nabla^{2}F_{i}(x)$.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From what I understand, the Gauss-Newton method is used to find a&#xA;  search direction, then the step size, etc., can be determined by some&#xA;  other method.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the simplest version of the Gauss-Newton method, there is no line search.  The iteration is simply $x^{(k+1)}=x^{(k)}+p$.  There is no guaranteed convergence result for this simple method, just as there is no guarantee of convergence for Newton's method with the full Hessian and unit step size. However, it does often work in practice.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding an approximate line search to the algorithm improves its convergence properties.  If a safeguarded approximate line search is used, and if the individual $F_{i}(x)$ function have Lipschitz continuous gradients, and $J(x)^{T}J(x)$ is always nonsingular, then convergence to a stationary point can be guaranteed.  See the Nocedal and Wright book cited at the end of this answer.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Gauss-Newton method always results in a direction of strict&#xA;  descent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is basically true.  If the matrix $J(x)^{T}J(x)$ in the Gauss-Newton method is non-singular, then it is symmetric and positive definite and $(J(x)^{T}J(x))^{-1}$ is also symmetric and positive definite.  The GN direction is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$p=-(J(x)^{T}J(x))^{-1}J(x)^{T}F(x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and assuming that $J(x)^{T}F(x) \neq 0$, the directional derivative in the direction $p$ is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \phi(x)^{T}p=-(J(x)^{T}F(x))^{T} \; (J(x)^{T}J(x))^{-1}\; (J(x)^{T}F(x)) &amp;lt; 0$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $J(x)^{T}F(x)=0$, then you're at a stationary point (and the GN method would stop.)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $J(x)^{T}J(x)$ is singular, then the Gauss-Newton step is not well defined.  Actually, because of the structure of the system of equations, there will be infinitely many directions $p$ that satisfy the equations.  If $p$ is chosen arbitrarily from among these directions, the method can fail to converge.    &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the Gauss-Newton method only requires that the directional derivative&#xA;  of the objective function exist – but you do not need to compute it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This question is unclear.  Directional derivative of what function? &#xA;In what direction?  However, if what you mean by &quot;directional derivative&quot; is &quot;the gradient of $\phi(x)$&quot;, then &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla \phi(x)=J(x)^{T}F(x)$, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this is computed during each iteration of the Gauss-Newton method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want the directional derivative of $\phi(x)$ in some particular direction, you can obtain it by taking the dot product of $\nabla \phi(x)$ and that direction.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the Gauss-Newton method does not require the Hessian.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Undoubtedly true.  The matrix $J(x)^{T}J(x)$ used in the GN method is not the Hessian of $\phi(x)$.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When you’re faced with an optimization problem of the form&#xA;  min||F(x)||, and F(x) is non-linear, then Gauss-Newton is a good&#xA;  choice because it doesn’t require you to compute the directional&#xA;  derivative.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Gauss-Newton method works well in practice on many problems, but it can fail when $J(x)^{T}J(x)$ is singular or nearly singular.  Stabilization of the Gauss-Newton method is advisable for a more robust algorithm.  The Levenberg-Marquardt algorithm is an example of a stabilized version of Gauss-Newton.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, it's not clear what you mean by &quot;directional derivative&quot; (of what function in what direction?)  If you simply mean the gradient of $\phi(x)$, then you do compute $\nabla \phi(x)$ in the Gauss-Newton method as discussed above. In any case, this isn't a very good &quot;because&quot; reason. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This material is discussed in many textbooks on nonlinear optimization and nonlinear regression.  An authoritative source is &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0387303030&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nocedal and Wright, &lt;em&gt;Numerical Optimization, 2nd ed.&lt;/em&gt;  2006.&lt;/a&gt;    &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2018-05-05T03:20:54.420" LastActivityDate="2018-05-05T03:20:54.420" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="29438" PostTypeId="2" ParentId="26734" CreationDate="2018-05-02T21:45:00.483" Score="3" Body="&lt;p&gt;There are a couple of resources I know worth trying; however, in my opinion, they either lack the focus or do not have the required audience.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;One of the subreddits: &lt;a href=&quot;https://www.reddit.com/r/compsci/&quot; rel=&quot;nofollow noreferrer&quot;&gt;/r/CompSci&lt;/a&gt;. Though it is mainly used for computer science and applications, it should be appropriate for discussion on computational science topics. It is pretty popular (234K subscribers), but I doubt it is easy to find the right crowd there.&lt;/li&gt;&#xA;&lt;li&gt;Other thematic subreddits (&lt;a href=&quot;https://www.reddit.com/r/chemistry/&quot; rel=&quot;nofollow noreferrer&quot;&gt;/r/Chemistry&lt;/a&gt;, &lt;a href=&quot;https://www.reddit.com/r/science/&quot; rel=&quot;nofollow noreferrer&quot;&gt;/r/Science&lt;/a&gt;, &lt;a href=&quot;https://www.reddit.com/r/math/&quot; rel=&quot;nofollow noreferrer&quot;&gt;/r/Math&lt;/a&gt;, &lt;a href=&quot;https://www.reddit.com/r/Physics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;/r/Physics&lt;/a&gt;, &lt;a href=&quot;https://www.reddit.com/r/MachineLearning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;/r/MachineLearning&lt;/a&gt;) might be also appropriate for particular discussions devoted to computational modeling topics. There are also smaller communities there, some of them more active than the others.&lt;/li&gt;&#xA;&lt;li&gt;Although I am not aware of a particular Facebook group for computational modeling/science, one can find reasonable communities on Facebook. (For example, &lt;a href=&quot;https://www.facebook.com/search/str/data+science/keywords_groups&quot; rel=&quot;nofollow noreferrer&quot;&gt;Facebook: Data Science&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;One can also find relevant groups on LinkedIn. (&lt;a href=&quot;https://www.linkedin.com/groups/95317&quot; rel=&quot;nofollow noreferrer&quot;&gt;High-Performance &amp;amp; Supercomputing&lt;/a&gt;, &lt;a href=&quot;https://www.linkedin.com/groups/87791&quot; rel=&quot;nofollow noreferrer&quot;&gt;HPC&lt;/a&gt;, &lt;a href=&quot;https://www.linkedin.com/groups/2126533&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational electromagnetics&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Some activity in forum-like discussions on relevant topics can be found on &lt;a href=&quot;https://www.researchgate.net/home.Home.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResearchGate&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There is also a classic example of a science-focused forum in Russian that is still very alive:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;6&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://dxdy.ru/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DxDy&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Generally, the era of social media and SE &quot;killed&quot; a lot of forums. So, the activity on typical forums right now is very limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there are good reasons, why forum-like discussions for certain areas are not too common nowadays. Forums (IMHO) tend to lose a discussion topic, become a place for flamewars and are generally counterproductive. That, in a way, gave rise to SE with stricter QA-rules. &lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-02T21:45:00.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29439" PostTypeId="2" ParentId="23314" CreationDate="2018-05-03T00:04:36.277" Score="5" Body="&lt;p&gt;The following paper&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scholar.google.com/scholar?q=S.D.%20Prestwich%2C%20Local%20search%20and%20backtracking%20vs%20non-systematic%20backtracking%2C%20in%3A%20AAAI%202001%20Fall%20Symposium%20on%20Using%20Uncertainty%20within%20Computation%20%282001%29.&quot; rel=&quot;noreferrer&quot;&gt;S. D. Prestwich, &quot;Local search and backtracking vs non-systematic backtracking,&quot; in &lt;em&gt;AAAI 2001 Fall Symp. Uncertainty Computation&lt;/em&gt;, 2001&lt;/a&gt;. Alternative link to a &lt;a href=&quot;https://pdfs.semanticscholar.org/b163/a64dfcd8aeb3ac61e3d1e171c0f1109620ce.pdf&quot; rel=&quot;noreferrer&quot;&gt;PDF&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;has a thorough comparison of local search vs. backtracking-like algorithm.&#xA;In the introduction, it even features a question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;What is the essential difference between local search and&#xA;  backtracking, that sometimes enables local search causes to&#xA;  scale far better than backtracking&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;While describing his variation of a hybrid local search + backtracking algorithm, Prestwich tests it on &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;classic $N$ queens problem&lt;/li&gt;&#xA;&lt;li&gt;graph coloring with &quot;real-world applications to timetabling, scheduling, frequency assignment, computer register allocation, printed circuit board testing, and pattern matching&quot;&lt;/li&gt;&#xA;&lt;li&gt;satisfiability (SAT)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of the above (with possible specialization) can serve as examples for your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other reference that is directly related to a comparison of local search algorithm to backtracking-like is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ijcai.org/Proceedings/95-2/Papers/128.pdf&quot; rel=&quot;noreferrer&quot;&gt;E. C. Freuder &lt;em&gt;et al.&lt;/em&gt;,&quot;Systematic versus stochastic constraint&#xA;satisfaction,&quot; in _Proc. 14th Int. Joint Conf. Artificial Intelligence, Montreal, QC, Canada, pp. 2027–2032, 1995.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which will also talk about the possible reasons for local search to perform better than backtracking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the possible reasons can be summarized as (Prestwich, 2001):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;randomness&lt;/li&gt;&#xA;&lt;li&gt;incompleteness&lt;/li&gt;&#xA;&lt;li&gt;scalability&lt;/li&gt;&#xA;&lt;li&gt;different search space&lt;/li&gt;&#xA;&lt;li&gt;ability to follow gradients&lt;/li&gt;&#xA;&lt;li&gt;backtracking's commitment to early variable assignments&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The last reference that came to my mind is a nice PhD Thesis&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://tspace.library.utoronto.ca/bitstream/1807/34957/1/Ward_Justin_D_201210_PhD_thesis.pdf&quot; rel=&quot;noreferrer&quot;&gt;J. Ward, &quot;Oblivious and non-oblivious local search for combinatorial&#xA;optimization,&quot; PhD. Thesis, Univ. Toronto, Dept. Comput. Sci., 2012.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;that will feature a couple of other (depends on categorization) applications for local search, where backtracking suffers.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-03T00:04:36.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29441" PostTypeId="1" CreationDate="2018-05-03T03:54:41.077" Score="1" ViewCount="57" Body="&lt;p&gt;I am using the method of manufactured solutions to perform the order of accuracy testing. I am using a cube for the testing. The cube is size 1m on all sides. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used 5 refinements: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$dx = dy = dz = 0.5$ ($8$ cells)&lt;/li&gt;&#xA;&lt;li&gt;$dx = dy = dz = 0.5/2$ ($8^2$ cells)&lt;/li&gt;&#xA;&lt;li&gt;$dx = dy = dz = 0.5/2/2$ ($8^3$ cells)&lt;/li&gt;&#xA;&lt;li&gt;$dx = dy = dz = 0.5/2/2/2$ ($8^4$ cells)&lt;/li&gt;&#xA;&lt;li&gt;$dx = dy = dz = 0.5/2/2/2/2$ ($8^5$ cells)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I first performed the analysis for a pure Dirichlet BC problem and observed 2nd order accuracy, as expected. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/r1CF7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r1CF7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I changed one of the cube surfaces to be a Neumann condition, the other 5 surfaces are still Dirichlet conditions. The results for this are shown below. &lt;a href=&quot;https://i.stack.imgur.com/esuyr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/esuyr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the slope here goes from about 2 to 1. I am still expecting 2nd order accuracy with the Neumann condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that my math is correct and the scheme that I am using is 2nd order accurate, does the decrease in slope suggest that I have a problem with my implementation? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could it also signify the onset of some other sort of error (perhaps algebraic?) taking over the discretization error? I would think not given that we did not observe this in the pure Dirichlet case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am puzzled as to why the first segment of this 2nd figure indicates 2nd order accuracy and then gradually decreases. Any insight would be greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="27504" LastEditorUserId="20688" LastEditDate="2018-05-03T03:59:13.397" LastActivityDate="2018-05-03T03:59:13.397" Title="Decrease in slope during convergence analysis" Tags="|boundary-conditions|convergence|discretization|error-estimation|accuracy|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29444" PostTypeId="1" AcceptedAnswerId="29445" CreationDate="2018-05-03T15:17:08.460" Score="2" ViewCount="219" Body="&lt;p&gt;In a much larger MINLP problem, I have set of variables $\{a_{ij}\}_{m,n}$, such that $0 \leq a_{ij} \leq 1 $ for all $i$, $j$, which I could think of as a matrix, for which I have two requirements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;there is exactly one non-zero element per row;&lt;/li&gt;&#xA;&lt;li&gt;there is at most one non-zero element per column.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This can easily be achieved by letting $a_{ij} = x_{ij}\,b_{ij}$ and:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;&amp;amp;\min f(x_{ij}\,b_{ij},\ldots)\\&#xA;&amp;amp;\mbox{s.t.}\\&#xA;&amp;amp;\sum_j x_{ij}\,b_{ij} = 1\quad \forall i,\\&#xA;&amp;amp;\sum_i x_{ij}\,b_{ij} \leq 1\quad \forall j,\\&#xA;&amp;amp;\vdots\\&#xA;&amp;amp;x_{ij} \in \{0,1\}, \ 0 \leq b_{ij} \leq 1\quad \forall i,j.&#xA;\end{align*}&#xA;( $f$ is some non-linear function in multiple variables, and constraints not relevant for this question are omitted)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for computational reason, I'd prefer to do &lt;em&gt;without&lt;/em&gt; all those extra binary variables $x_{ij}$. That is, I am looking for a formulation of the constraints without introducing extra binary variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to construct linear constraints on the $a_{ij}$, that ensure that the two requirements (as mentioned on the top of this post) are satisfied? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions are much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="27655" LastActivityDate="2018-05-03T16:10:13.073" Title="Constraints 'exactly/at most one non-zero element' without binary variables" Tags="|nonlinear-programming|constraints|mixed-integer-programming|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29445" PostTypeId="2" ParentId="29444" CreationDate="2018-05-03T16:10:13.073" Score="4" Body="&lt;p&gt;No, this is not possible.  There is a standard way of showing this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The feasible region of your constraints is not convex.  For example, $x_{1,1}=1$, $x_{1,2}=0$ is feasible, $x_{1,1}=0$, $x_{1,2}=1$ is feasible, but the midpoint $x_{1,1}=1/2$, $x_{1,2}=1/2$ is not feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The feasible set of a system of linear equality and inequality constraints is always convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus your desired constraint can’t be expressed with linear constraints.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-05-03T16:10:13.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29446" PostTypeId="2" ParentId="943" CreationDate="2018-05-03T18:38:25.850" Score="-1" Body="&lt;p&gt;From studying many Engineering ideas, I woulds say a gpu is a form of focusing of tasks,of memory-management,of repeatable-calculation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many formulas might be simple to write but painful to calculate such as in matrix mathematics you don't get a single answer but many values.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is important in computing as how fast a computer is calculating values and running formulas as some formulas can't run without all calculated values (hence slow down). A computer doesn't know very well what order to run formulas or calculate values to use in these programs. It mainly brute forces through at fast speeds and breaks formulas into chucks to calculate but many programs these days require these calculated chucks right now and wait in ques (and ques of ques and more ques of ques). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in a simulation game which should be calculated first in collisions the damage of the collision, the position of the objects, the new velocity? How much time should this take?  How can any cpu handle this load? Also, most programs are very abstract requiring more time to handle data and aren't always designed for multi-threading or aren't any good ways in abstract programs to do this effectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As cpu became better and better people became sloppy in programming and we must program for many different types of computers as well. A gpu is designed to brute force through many simple calculations at the same time (not mention memory (secondary/ram) and heating cooling are the main bottle necks in computing). A cpu is managing many many ques at the same time or being pulled in to many directions it is figuring out what to do not being able to do it. (hey it's almost human)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A gpu is grunt worker the tedious work. A cpu is managing complete chaos and can't handle every detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what do we learn? A gpu does detail tedious work all at once and a cpu is a multi-task machine that can't focus very well with too many tasks to do.   (It's like it has attention disorder and autism at the same time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Engineering there is the ideas, design, reality, and a lot of grunt work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I leave remember to start simple, start quickly, fail-quickly, fail-fast, and never stop trying.&lt;/p&gt;&#xA;" OwnerUserId="27657" LastActivityDate="2018-05-03T18:38:25.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29447" PostTypeId="1" AcceptedAnswerId="29449" CreationDate="2018-05-03T23:59:37.987" Score="2" ViewCount="95" Body="&lt;p&gt;I'm reading a paper where they use a discrete approximation of a logarithmic mass growth rate as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{d \log M}{d \log t} \approx \frac{(t_B + t_A)(M_B - M_A)}{(t_B - t_A)(M_B + M_A)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to derive this approximation, but wasn't successful thus far. I tried forward, centered and backward differencing, using Taylor expansions and $\log(x) \approx (x-1) - \frac{(x-1)^2}{2} + ...$ for $x \rightarrow 1$, but haven't gotten that result. In particular, I don't see how the factors $(M_A + M_B)$ and $(t_A + t_B)$ come into play.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me out? &lt;/p&gt;&#xA;" OwnerUserId="26461" LastEditorUserId="20688" LastEditDate="2018-05-04T01:30:55.953" LastActivityDate="2018-05-04T02:12:19.517" Title="Discretisation of logarithmic derivative: Deriving the formula" Tags="|finite-difference|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29448" PostTypeId="2" ParentId="22031" CreationDate="2018-05-04T00:30:38.710" Score="1" Body="&lt;p&gt;So, since $V\in\mathbb R^{m\times n}$ has full column rank $n$, implying also $m\geq n$, we might obtain the pseudoinverse using&#xA;$$&#xA;V^+=(V^TV)^{-1}V^T&#xA;$$&#xA;since $V^TV$ is invertible and $V^*=V^T$ in $\mathbb R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's denote by $i$ the index of the row which is scaled and by $d_i$ - the scaling factor.&#xA;Thus, a perturbed matrix $V^\prime$ can be expressed as&#xA;$$&#xA;V^\prime = D_i V= \text{diag}(\underbrace{1,\ldots,1}_{i-1},d_i,\underbrace{1,\ldots,1}_{m-i})V&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how can we express it as a low-rank update?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider&#xA;$$&#xA;u=e_i=\left[&#xA;\begin{array}{c}&#xA;0\\&#xA;\vdots\\&#xA;1 \\&#xA;\vdots\\&#xA;0&#xA;\end{array}\right], \quad\quad&#xA;v=(V_{i,*})^T&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is the $i$th elementary unit vector in $m$ dimensional space and $v$ is the transposed $i$th row of the original matrix $V$. Then:&#xA;$$&#xA;V^\prime = V+(d_i-1)uv^T&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's express the update to the &quot;correlation&quot; matrix $V^T V$:&#xA;$$&#xA;\begin{aligned}&#xA;{V^\prime}^T{V^\prime}&amp;amp;=(V+(d_i-1)uv^T)^T(V+(d_i-1)uv^T)\\&#xA;&amp;amp;=V^TV+(d_i-1)v\underbrace{u^TV}_{v^T}+(d_i-1)\underbrace{V^Tu}_{v}v^T+(d_i-1)(d_i-1)v\underbrace{u^Tu}_{1}v^T\\&#xA;&amp;amp;=V^TV+\left((d_i-1)+(d_i-1)+(d_i-1)^2\right)vv^T\\&#xA;&amp;amp;=V^TV+(d_i^2-1)vv^T&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that, we expressed a rank-1 update to the correlation matrix $V^T V$ for which we already know the inverse. For compactness, let's denote &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\gamma=d_i^2-1$$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let's use Sherman-Morrison to find an update to the inverse of the correlation matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;({V^\prime}^T{V^\prime})^{-1}&amp;amp;=(V^TV+\gamma v v^T)^{-1}\\&#xA;&amp;amp;=(V^TV)^{-1}-\frac{\gamma (V^TV)^{-1}vv^T(V^TV)^{-1}}{1+\gamma v^T (V^TV)^{-1}v}&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's denote the following matrix-vector product by&#xA;$$&#xA;w = (V^TV)^{-1}v&#xA;$$&#xA;Then, since $V^T V$ is symmetric&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{aligned}&#xA;({V^\prime}^T{V^\prime})^{-1}&amp;amp;=(V^TV)^{-1}-\frac{\gamma ww^T}{1+\gamma v^T w}\\&#xA;&amp;amp;=(V^TV)^{-1}-\tau ww^T, \quad \tau=\frac{d_i^2-1}{1+(d_i^2-1)v^Tw}&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in order to update the inverse of the correlation matrix (provided you have the computation for the original matrix done):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute matrix-vector product $w = (V^TV)^{-1}v$ in $n^2$ operations.&lt;/li&gt;&#xA;&lt;li&gt;Calculate dot-product of $w$ and original row $v$: $v^T w$ in $n$ operations.&lt;/li&gt;&#xA;&lt;li&gt;Calculate $\tau$ in constant time.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the scaled by $\tau$ outer product $\tau ww^T$ in $n^2$ operations&lt;/li&gt;&#xA;&lt;li&gt;Calculate the update to the inverse of the correlation matrix by subtracting the outer product in $n^2$ operations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So, you achieved $({V^\prime}^T{V^\prime})^{-1}$ in $3n^2+n$ operations. Probably, can be done in $2n^2+n$ if fused flops are used combining steps 4 and 5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you can calculate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;{V^\prime}^+ = ({V^\prime}^T{V^\prime})^{-1}V^TD_i&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or leave it in the unassembled form depending on your future usage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if that will lead to significant speed-up for the requested $\mathbb R^{4\times 3}$ case but is worth trying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, this particular technique relies on $V$ having full-column rank, otherwise, the pseudoinverse cannot be found this way and SVD decomposition has to be used -&gt; where updates are also possible, but much trickier.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-04T00:30:38.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29449" PostTypeId="2" ParentId="29447" CreationDate="2018-05-04T01:27:06.883" Score="3" Body="&lt;p&gt;This should be coming from a chain rule (assuming $M$ is a function of $t$, $t_A&amp;lt;t_B$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(t)=\frac{d \log\big(M(t)\big)}{d \log t} = \frac{d\log\big(M(t)\big)}{dt}t=\frac{dM(t)}{dt}\frac{t}{M(t)}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, consider the backward difference,&#xA;$$&#xA;f(t_B) \approx \frac{M_B-M_A}{t_B-t_A}\frac{t_B}{M_B}&#xA;$$&#xA;Second, consider the forward difference,&#xA;$$&#xA;f(t_A) \approx \frac{M_B-M_A}{t_B-t_A}\frac{t_A}{M_A}&#xA;$$&#xA;Clearly, we are getting expressions similar to the given in the paper, but something is a little bit off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if we are looking for the central difference,&#xA;$$&#xA;f\left(\frac{t_A+t_B}{2}\right) \approx \frac{M_B-M_A}{t_B-t_A}\frac{\frac{t_A+t_B}{2}}{M_*}&#xA;$$&#xA;Here, $M_*$ is the value of $M$ at $t=\frac{t_A+t_B}{2}$.&#xA;Let $M_*=\frac{M_A+M_B}{2}$ - an average, which is exact if $M$ is linear between $t_A$ and $t_B$ or a very close approximation for a fine enough grid step. Then, for central differencing&#xA;$$&#xA;f\left(\frac{t_A+t_B}{2}\right) \approx \frac{M_B-M_A}{t_B-t_A}\frac{\frac{t_A+t_B}{2}}{\frac{M_A+M_B}{2}}=\frac{(M_B-M_A)}{(t_B-t_A)}\frac{(t_A+t_B)}{(M_A+M_B)}&#xA;$$&#xA;that is exactly the original expression from the paper.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-05-04T02:12:19.517" LastActivityDate="2018-05-04T02:12:19.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29450" PostTypeId="2" ParentId="29424" CreationDate="2018-05-04T01:31:29.783" Score="2" Body="&lt;p&gt;Typical ODE solvers simply need to provide you a function that describes the evaluates the right hand side. That is, they will call this function with a given $x,t$ and you simply have to return what $f(x,t)$ is for these arguments. In your case, the implementation of that function would simply contain an &lt;code&gt;if&lt;/code&gt; statement. That is a totally common thing in practice and any good ODE solver will not bat a lash over it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bigger issue in your case is that your right hand side is not Lipschitz. The convergence proof of most ODE solver algorithms requires that the right hand side if Lipschitz to guarantee the usual convergence order. The right hand side you have here does not satisfy this requirement -- which may or may not mean that your method of choice converges at the expected rate. But that's a question separate from what you were asking.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-05-04T01:31:29.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29451" PostTypeId="2" ParentId="18710" CreationDate="2018-05-04T05:21:23.537" Score="1" Body="&lt;p&gt;I guess, the final result is suffering from instability (relative) of Gram-Schmidt, even in modified/stabilized forms. In this case, instead of going from the start: modifying the orthogonalization process, we can start with the final result instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, after solving a non-Hermitian eigenvalue problem (with a reliable algorithm), we obtained $X_L$ and $X_R$, matrices of left and right eigenvectors, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We desire &#xA;$$X_LX_R=I$$ &#xA;however, &#xA;$$X_LX_R=Y$$ &#xA;where matrix $Y$ is unfortunately different from the identity $I$.&#xA;Now, we can compute the LU decomposition (which is supposed to be much more numerically stable than Gram-Schmidt) for $Y=L_YU_Y$.&#xA;$$&#xA;X_LX_R=L_YU_Y&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, by solving two triangular systems with multiple RHSs via forward- (for $L_Y$) and back-substitution (for $U_Y$):&#xA;$$&#xA;L_YX_L^\prime=X_L\\&#xA;X_R^\prime U_Y = X_R&#xA;$$&#xA;we can solve for the biorthogonalized $X_L^\prime$ and $X_R^\prime$ via an also relatively stable numerical process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the algorithm (definitely not the best in terms of computational complexity):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute $Y=X_LX_R$&lt;/li&gt;&#xA;&lt;li&gt;Compute LU-decomposition of $Y$&lt;/li&gt;&#xA;&lt;li&gt;Solve 2 triangular systems with multiple RHSs to obtain the final result of biorthogonolized matrices.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There is a small question of what happens if a set of eigenvectors is degenerate, but I don't see why LU should fail in that case either.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-04T05:21:23.537" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29452" PostTypeId="2" ParentId="26119" CreationDate="2018-05-04T06:26:17.440" Score="2" Body="&lt;p&gt;You got two questions. I can answer the first one about real coding and design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stability is for your algorithm. If the algorithm is not sensitive to the approximations made at every step, then the stability is implied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you have a lot of steps (iterations) to deal your data, and you make an approximate to the result of every step. At the end of your algorithm, you will get a huge deviation through enough steps. Especially in the iteration process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This kind of error produced by your algorithm has nothing to do with your data.&#xA;The deviation is acceptable if it is under your control, thus the algorithm is stable.&#xA;Here is some advice to avoid such problems in your experiment:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Deduce calculation times.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Avoid addition of a small number to a big number.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Avoid subtraction of two numbers of similar magnitude&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Avoid division by a small number and multiplication by a large number.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27659" LastEditorUserId="20688" LastEditDate="2018-05-04T06:51:19.303" LastActivityDate="2018-05-04T06:51:19.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29453" PostTypeId="2" ParentId="26734" CreationDate="2018-05-04T06:26:56.320" Score="2" Body="&lt;p&gt;The Julia community has a lot of people who are focused on different aspects of scientific computing. Most of its &lt;a href=&quot;https://discourse.julialang.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Discourse Forum&lt;/a&gt; is discussions of numerical issues, performance, simulation methods, and scientific computing packages. There's a lot of &lt;a href=&quot;https://julialang.org/community/&quot; rel=&quot;nofollow noreferrer&quot;&gt;other associated channels&lt;/a&gt; like the JuliaLang slack (&lt;a href=&quot;https://slackinvite.julialang.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here for an invite&lt;/a&gt;). The differential equations channel from there is bridged to the &lt;a href=&quot;https://gitter.im/JuliaDiffEq/Lobby&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffEq Gitter&lt;/a&gt; with daily discussions on all things differential equations related. There's lot of Slack channels which are highly active like &lt;code&gt;#gpu&lt;/code&gt;, &lt;code&gt;#machine-learning&lt;/code&gt;, &lt;code&gt;#parallel&lt;/code&gt;, &lt;code&gt;#linalg&lt;/code&gt;, etc. where people are discussing algorithms and their implementations. The Julialang/julia Github is very lively as well, though more purpose-driven and requires a bit more prerequisite knowledge to join in. Altogether, it's a great community to discuss with if you're interested in all things scientific computing related.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-05-04T06:26:56.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29455" PostTypeId="1" CreationDate="2018-05-04T18:57:51.973" Score="0" ViewCount="57" Body="&lt;p&gt;This question was raised at a different place without sufficient answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Definition 1:&#xA;We say that a vector $x \in R^n$ is a limit point of a sequence $\{x_k\}$ in $R^n$ if there exists a subsequence of $\{x_k\}$ that converges to $x$. This definition appears frequently in the optimization literature, for instance, see Bertsekas, Nonlinear Programming, 2nd edition, page 666. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But a definition of limit point in real analysis is different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Definition 2: A point $z_0$  is a limit  point for a set of point if  every neighborhood of $z_0$ contains  points, other  than $z_0$ of set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accordingly, $a_n=\{5, 4, 3, 2, 1, 0, 0, ...,\}$ has a limit point of $0$ based on the first definition, but $0$ is not a limit point based on the second definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that in the literature, limit point is frequently used but without a definition. So what is the default definition in computer science or optimization? Is Definition 1 the default? For instance, what is the presumed definition of limit point in &lt;a href=&quot;https://arxiv.org/pdf/1209.2385&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1209.2385&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="27664" LastActivityDate="2018-05-04T19:25:59.433" Title="Conflicting definition of limit point" Tags="|optimization|convergence|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29456" PostTypeId="2" ParentId="29455" CreationDate="2018-05-04T19:25:59.433" Score="2" Body="&lt;p&gt;The definition of the limit of a sequence is really rather uncontroversial and can be found at &lt;a href=&quot;https://en.wikipedia.org/wiki/Limit_of_a_sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Limit_of_a_sequence&lt;/a&gt; . Intuitively, a sequence $\{a_1,a_2,\ldots\}$ converges to a limit $a$ if the $a_k$ come closer and closer to $a$. Sitting on top of $a$ is allowed. So $\{4, 3, 2,1, 0,0,0,0,\ldots\}$ converges to zero just fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The concepts you are confused by are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;In compact sets, &lt;em&gt;every&lt;/em&gt; sequence has one or more &lt;em&gt;subsequences&lt;/em&gt; that converges to something. For example, the sequence $\{1,0,1,0,1,0,\ldots\}$ does not converge, but it has subsequences that converge to zero or one. This seems to be what you suggest in your definition 2 (though I don't think that it is quite correct as stated).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In the paper you reference, the sequence is indexed by something else. For example, let there be a sequence $\{a_1(x),a_2(x),a_3(x),\ldots\}$ for each choice of $x$. Then depending on $x$, these sequences may converge to different limits $a(x)$. Statements such as those in the preprint you reference say that for &lt;em&gt;any choice of $x$&lt;/em&gt;, the limit point $a(x)=\lim_{k\rightarrow \infty} a_k(x)$ has certain properties. (In the example of the preprint, $x$ is the starting point and the statement says that no matter where you start, you will always converge to a stationary point. &lt;em&gt;Which&lt;/em&gt; stationary point you converge to may be different, though!)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-05-04T19:25:59.433" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="29457" PostTypeId="1" CreationDate="2018-05-04T20:18:02.500" Score="1" ViewCount="291" Body="&lt;p&gt;I am using Runge-Kutta to solve a $3 \times 3$ 2&lt;sup&gt;nd&lt;/sup&gt; order linear ODE $$M x'' + C x' + K x = 0$$ and initial conditions are all zeros. Then I prescribe the 2nd variable to follow a given path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for prescribing displacement, I can choose two ways:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;[1] One is to use Lagrangian multiplier $$M x'' + C x' + K x + P x = P x_p,$$ where $x_p$ is prescribed path, it works perfectly. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;[2] Another way is to replace the variables with given values after Runge-Kutta updating, which will only work if the matrix $M$ is diagonal.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I really appreciate if anyone can offer any insights about this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if I am prescribing only the displacement, it ends up that in Runge-Kutta, I have to prescribe the displacement rate as well, to match results with FD and other schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want to use the Lagrangian multiplier in Runge-Kutta or explicit methods, because it will significantly reduce my stability region and I have to use a very small time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please feel free to jump with any suggestions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;=====================================================================&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My codes are attached below in case anyone needs to try. You can change $M$ matrix to compare the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is my main code. I put some parameters in a data structure so it's easier to access. And I have a prescribed displacement profile defined as &lt;code&gt;ExtDisp&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc;&#xA;clear variables;&#xA;&#xA;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% MODEL %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;% spring damper system&#xA;&#xA;M = [3, 0.2 0.2;&#xA;    0.2, 2  0;&#xA;    0.2,    0,  5];&#xA;&#xA;K = [10, 8 0.2;&#xA;    8, 10, 0.2;&#xA;    0.2, 0.2, 15];&#xA;&#xA;C = zeros(3, 3);&#xA;&#xA;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% MODEL %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;% tspan&#xA;dt = 0.005;&#xA;t_span = [0, 8];&#xA;time = t_span(1): dt: t_span(2);&#xA;&#xA;Ndt = length(time);&#xA;dim = length(M);&#xA;&#xA;% which displacement to prescribe&#xA;index_prescribed = 2;&#xA;index_free = (1:dim)';&#xA;index_free(index_prescribed) = [];&#xA;&#xA;% excitation path, a toneburst signal&#xA;freq = 1;&#xA;nCnt = 3.0;&#xA;ExtDisp_time_lim = nCnt / freq;&#xA;ExtDisp = zeros(1, Ndt);&#xA;for j = 1: Ndt&#xA;    if time(j) &amp;lt;= ExtDisp_time_lim&#xA;        ExtDisp(j) = (1-cos(2*pi*time(j)*freq/nCnt)) * sin(2*pi*time(j)*freq);&#xA;    end&#xA;end&#xA;&#xA;dExtDisp = diff(ExtDisp) / dt;&#xA;ddExtDisp = diff(dExtDisp) / dt;&#xA;&#xA;force_param_cells.time = time;&#xA;force_param_cells.ExtDisp_lim = ExtDisp_time_lim;&#xA;force_param_cells.ExtDisp = ExtDisp;&#xA;force_param_cells.dExtDisp = dExtDisp;&#xA;force_param_cells.ddExtDisp = ddExtDisp;&#xA;force_param_cells.index_prescribed = index_prescribed;&#xA;&#xA;% MAIN COMPUTATIONS&#xA;&#xA;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;% use Lagrangian multiplier&#xA;&#xA;% initial BCs&#xA;x0 = [0, 0, 0]';&#xA;v0 = [0, 0, 0]';&#xA;[tt2, yy2] = rk_lagrangian_multiplier(dt, t_span, M, C, K, force_param_cells, x0, v0);&#xA;&#xA;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;% just by replacing&#xA;&#xA;% initial BCs&#xA;x0 = [0, 0, 0]';&#xA;v0 = [0, 0, 0]';&#xA;[tt4, yy4] = rk_ordinary(dt, t_span, M, C, K, force_param_cells, x0, v0);&#xA;&#xA;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;% plot all variables&#xA;figure;hold on;&#xA;for k = 1 : 6&#xA;    subplot(2, 3, k);hold on;&#xA;    plot(tt2, yy2(k,:), 'k-', 'LineWidth', 2);&#xA;    plot(tt4, yy4(k,:), 'r--', 'LineWidth', 2);&#xA;&#xA;    if k &amp;gt; 3&#xA;        ylabel(['$\dot{x}$', int2str(k-3)], 'FontSize', 15 ,'interpreter','latex');&#xA;    else&#xA;        ylabel(['${x}$', int2str(k)], 'FontSize', 15 ,'interpreter','latex');&#xA;    end&#xA;&#xA;end&#xA;legend('RK-Multiplier', 'RK-Replace');&#xA;xlabel('Time (s)','FontSize',15);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;==================================================================&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other scripts are given as well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Runge-Kutta Lagrangian multiplier)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [tt, yy] = rk_lagrangian_multiplier(dt, t_span, M, C, K, force_param_cells, x0, v0)&#xA;&#xA;time = t_span(1): dt: t_span(2);&#xA;&#xA;% system dimensions&#xA;dim = length(M);&#xA;Ndt = length(time);&#xA;&#xA;index_prescribed = force_param_cells.index_prescribed;&#xA;&#xA;% penalty on prescribed displacement&#xA;lamda = max(max(K)) * 1E4;&#xA;PenaltyMatrix = zeros(3);&#xA;PenaltyMatrix(index_prescribed, index_prescribed) = lamda;&#xA;&#xA;% save model parameters as cells&#xA;model_cells{1} = M;&#xA;model_cells{2} = C;&#xA;model_cells{3} = K;&#xA;model_cells{4} = PenaltyMatrix;&#xA;&#xA;&#xA;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Runge-Kutta %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;&#xA;step = 1;&#xA;tt(:,step) = time(step);&#xA;yy(:,step) = [x0; v0];&#xA;&#xA;for step = 1 : Ndt-1          % calculation loop&#xA;    time_t = time(step);&#xA;    yy_t = yy(:, step);&#xA;&#xA;    k_1 = model_rk_spring_mass_full(time_t,              yy_t,                   model_cells,    force_param_cells);&#xA;    k_2 = model_rk_spring_mass_full(time_t + 0.5 * dt,  yy_t + 0.5*dt * k_1,    model_cells,    force_param_cells);&#xA;    k_3 = model_rk_spring_mass_full(time_t + 0.5 * dt,  yy_t + 0.5*dt * k_2,    model_cells,    force_param_cells);&#xA;    k_4 = model_rk_spring_mass_full(time_t + dt,         yy_t + dt * k_3,        model_cells,   force_param_cells);&#xA;&#xA;    tt(step + 1)    =   time_t + dt;&#xA;    yy(:, step + 1) =   yy(:, step) + dt * (k_1 + 2*k_2 + 2*k_3 + k_4) / 6.0;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Runge-Kutta - just replacing displacements after updating)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [tt, yy] = rk_ordinary(dt, t_span, M, C, K, force_param_cells, x0, v0)&#xA;&#xA;time = t_span(1): dt: t_span(2);&#xA;&#xA;% system dimensions&#xA;dim = length(M);&#xA;Ndt = length(time);&#xA;&#xA;% save model parameters as cells&#xA;model_cells{1} = M;&#xA;model_cells{2} = C;&#xA;model_cells{3} = K;&#xA;&#xA;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Runge-Kutta %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%&#xA;&#xA;step = 1;&#xA;tt(:,step) = time(step);&#xA;yy(:,step) = [x0; v0];&#xA;yy(force_param_cells.index_prescribed, step) = force_param_cells.ExtDisp(:, step);&#xA;&#xA;for step = 1 : Ndt-1          % calculation loop&#xA;&#xA;    time_t = time(step);&#xA;    yy_t = yy(:, step);&#xA;&#xA;    k_1 = model_rk_spring_mass_no_penalty(time_t,               yy_t,                   model_cells);&#xA;    k_2 = model_rk_spring_mass_no_penalty(time_t + 0.5 * dt,    yy_t + 0.5*dt * k_1,    model_cells);&#xA;    k_3 = model_rk_spring_mass_no_penalty(time_t + 0.5 * dt,    yy_t + 0.5*dt * k_2,    model_cells);&#xA;    k_4 = model_rk_spring_mass_no_penalty(time_t + dt,          yy_t + dt * k_3,        model_cells);&#xA;&#xA;    tt(step + 1)    =   time_t + dt;&#xA;    yy(:, step + 1) =   yy(:, step) + dt * (k_1 + 2*k_2 + 2*k_3 + k_4) / 6.0;&#xA;&#xA;    % prescribing displacement and rate&#xA;    if time_t &amp;lt;= force_param_cells.ExtDisp_lim&#xA;        yy(force_param_cells.index_prescribed, step+1) = force_param_cells.ExtDisp(step+1);&#xA;        yy(force_param_cells.index_prescribed + dim, step+1) = force_param_cells.dExtDisp(step+1);&#xA;    end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Below are the two model functions. The first one is the one used with the Lagrangian multiplier,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dy = model_rk_spring_mass_full(t, y, model_cells, force_param_cells)&#xA;&#xA;M = model_cells{1};&#xA;C = model_cells{2};&#xA;K = model_cells{3};&#xA;Penalty = model_cells{4};&#xA;&#xA;time = force_param_cells.time;&#xA;ExtDisp = force_param_cells.ExtDisp;&#xA;ExtDisp_lim = force_param_cells.ExtDisp_lim;&#xA;index_prescribed = force_param_cells.index_prescribed;&#xA;&#xA;dim = length(M);&#xA;dy = zeros(dim * 2, 1);&#xA;&#xA;% dealing with extra prescribed displacement&#xA;&#xA;disp_prescribed = interp1(time', ExtDisp', t)';&#xA;disp_vec = zeros(dim, 1);&#xA;disp_vec(index_prescribed) = disp_prescribed;&#xA;&#xA;if t &amp;lt;= ExtDisp_lim&#xA;    P_m = Penalty;&#xA;    force_penalty = P_m * disp_vec;&#xA;else&#xA;    P_m = zeros(dim, dim);&#xA;    force_penalty = zeros(dim, 1);&#xA;end&#xA;&#xA;% updating procedures&#xA;dy(1:dim)       = y(dim+1:end);&#xA;dy(dim+1:end)   = M \ (force_penalty - C * y(dim+1: end) - (K + P_m) * y(1:dim));&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the other model is used for replacing method,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dy = model_rk_spring_mass_no_penalty(t, y, model_cells)&#xA;&#xA;M = model_cells{1};&#xA;C = model_cells{2};&#xA;K = model_cells{3};&#xA;&#xA;dim = length(M);&#xA;dy = zeros(dim * 2, 1);&#xA;&#xA;% updating procedures&#xA;dy(1:dim)       = y(dim+1:end);&#xA;dy(dim+1:end)   = M \ (- C * y(dim+1: end) - K * y(1:dim));&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27666" LastEditorUserId="20688" LastEditDate="2018-05-05T15:58:15.667" LastActivityDate="2023-04-11T04:07:15.900" Title="Prescribing variables as an excitation in Runge-Kutta method" Tags="|matlab|ode|runge-kutta|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29459" PostTypeId="1" CreationDate="2018-05-05T15:35:09.663" Score="1" ViewCount="118" Body="&lt;p&gt;Suppose I have some time dependent PDE, which can be written in the strong form as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} + \mathcal{L}(u) = f&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\mathcal{L}$ is some differential operator. If I write the weak form of the problem without discretizing the time term, I end up with the standard weak form problem of finding $u_h \in X_h$ such that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;m\left(\frac{\partial u_h}{\partial t},\, v\right) + a(u_h,\,v) = f(v), \qquad \forall v \in X_h &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point, I usually discretize in time in the following way (using backward-euler here) as  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;m\left(\frac{u_h^{k+1} - u_h^k}{\Delta t},\, v\right) + a(u_h^{k+1&#xA;},\,v) = f(v), \qquad \forall v \in X_h &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;at which point, I know how to solve the problem by putting the $m(u_h^k, v)$ term to the right hand side, choosing a test, trial space, discretizing and solving. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But what if I wanted to start from the second equation and structure my code to use a black box integrator?&lt;/strong&gt; Is there a standard way of doing this? I know that the right hand side terms can become much more complicated with higher-order time discretizations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the answer is to first discretize in space, and express the discrete system as a matrix system $B\dot{U_h} + AU_h = F$. However, if the black box time integrator library does not support DAE systems (allowing user to input $B$) matrix, you're out of luck (this seems to be the case with &lt;code&gt;scipy.integrate.ode&lt;/code&gt;). If $B$ can be inverted cheaply, then the black box integrator can handle the equivalent problem $\dot{U_h} = -B^{-1}(AU_h - F)$. Alternatively, mass lumping can be used, but this seems ridiculous if you were interested in implicit time-marching schemes to begin with.&lt;/p&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="20148" LastEditDate="2018-05-07T18:13:28.647" LastActivityDate="2018-05-07T18:13:28.647" Title="How to implement Galerkin Method of Lines / FEM with black box integrators in scipy" Tags="|finite-element|time-integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29460" PostTypeId="2" ParentId="29459" CreationDate="2018-05-05T21:59:11.727" Score="1" Body="&lt;p&gt;Do you have a particular black box ode integrator in mind to solve?&#xA;$$&#xA;m\left(\frac{\partial u_h}{\partial t},\, v\right) + a(u_h,\,v) - f(v) = 0, \qquad \forall v \in X_h &#xA;$$&#xA;and initial conditions, those I will skip also in the rest of these notes.&#xA;In general, this is just a &lt;a href=&quot;https://en.wikipedia.org/wiki/Differential-algebraic_system_of_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;differential-algebraic system of equations&lt;/a&gt; (DAE)$$&#xA;F(t, u, u_{t}) = 0.&#xA;$$ that could also be solved with a solver for DAEs, such as &lt;a href=&quot;https://de.mathworks.com/help/matlab/ref/ode15i.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ode15i&lt;/a&gt; in &lt;em&gt;MATLAB&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27670" LastActivityDate="2018-05-05T21:59:11.727" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29461" PostTypeId="1" CreationDate="2018-05-05T23:40:46.113" Score="1" ViewCount="159" Body="&lt;p&gt;&lt;strong&gt;I'm trying to write the following function in python:&lt;/strong&gt; &#xA;$$&#xA;f_\mu(\mathcal X) = f_0(\mathcal X) + \sum_{i = 1}^n \max_{||\mathcal Y_{i(i)}|| \leq1} \alpha_i\langle \mathcal X_{(i)},\mathcal Y_{i(i)} \rangle - \frac{\mu_i}{2}||\mathcal Y_{i(i)}||_F^2 \\ \mathcal X, \mathcal Y \ \text{are tensors (MxNx3), } \\ \mathcal X_{(i)} \ \text{is the tensor unfolded along mode i (a matrix), } \\||.||_F \ \text{ is the Frobenius Norm}, \\ \alpha, \mu \in {\mathbb R}, \\ n = 3&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What i have so far:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import tensorly as tl&#xA;import numpy as np&#xA;LA = np.linalg&#xA;&#xA;def f_m(a,X,Y,m):&#xA;   result = np.zeros((3,3))&#xA;   zz = []&#xA;   for i in range(3):&#xA;      for j in range(3):&#xA;        yy = tl.unfold(Y[j],j)&#xA;        xx = a[j]*tl.unfold(X,j)&#xA;        zz = np.multiply(yy,xx)&#xA;        zz = LA.norm(zz, 'nuc')&#xA;        result[i,j] = zz-0.5*m[j] * LA.norm(tl.unfold(Y[j], j), 'fro') ** 2&#xA;   F1 = max(result[0,:])&#xA;   F2 = max(result[1,:])&#xA;   F3 = max(result[2,:])&#xA;   F = max(F1,F2,F3)&#xA;return F&#xA;&#xA;def grad_f_m(m,a,X):&#xA;   grad = a*truncate_op(a/m * X, 1)&#xA;return grad&#xA;&#xA;def truncate_op(X,tau):&#xA;   u , sig, v = LA.svd(X, full_matrices=False)&#xA;   t = np.zeros(len(sig))+tau&#xA;   np.minimum(sig,t,sig)&#xA;   trunk = np.dot(u[:,:], np.dot(np.diag(sig[:]), v[:,:]))&#xA;return trunk&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does $ f_\mu(\mathcal X)$ return? A scalar? A matrix? A tensor? I'm thinking its a scalar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second question:&lt;/strong&gt; &#xA;$$&#xA;g(X) := ||X||_* = \max_{||Y||\leq1} \langle X,Y \rangle&#xA;\\||.||_* \ \text{ is the Trace Norm}&#xA;$$&#xA;If i compute the trace norm like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;numpy.linalg.norm(zz,'nuc')&lt;/code&gt; &#xA;(which computes the nuclear norm according to the &lt;a href=&quot;https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.linalg.norm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that equivalent to $$ \max_{||\mathcal Y_{(i)}||\leq1}\alpha_i\langle \mathcal X_{(i)},\mathcal Y_{(i)} \rangle $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Third question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla f_\mu(\mathcal X) = \nabla f_0(\mathcal X) + \sum_{i = 1}^n \alpha_i T_1(\frac{\alpha_i}{\mu_i}\mathcal X_{(i)})&#xA;$$&#xA;$$&#xA;T_{\tau} = U\sigma_{\overline \tau}V&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sigma_{\overline \tau} = diag(\min(\sigma,\tau))&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;U,\sigma, V = SVD(\mathcal X_{(i)})&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\ T_{\tau}$ will result in a different sized matrix for each $ \mathcal X_{(i)}$ how can I sum them across $ n $ ?&lt;/p&gt;&#xA;" OwnerUserId="27671" LastEditorUserId="27671" LastEditDate="2018-05-06T05:54:46.117" LastActivityDate="2018-05-06T05:54:46.117" Title="computing dual matrix trace norm and tensor gradient in python" Tags="|python|numpy|image-processing|tensor-decomposition|l2-norm|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29463" PostTypeId="2" ParentId="10003" CreationDate="2018-05-06T04:29:46.050" Score="6" Body="&lt;p&gt;One of the commercial packages that offer element-free Galerkin (EFG) is &lt;a href=&quot;http://www.lstc.com/products/ls-dyna&quot; rel=&quot;noreferrer&quot;&gt;LS-DYNA&lt;/a&gt;, a multiphysics simulation software. You can find several papers that feature numerical examples with timing and accuracy using LS-DYNA's EFG implementation &lt;a href=&quot;https://www.dynalook.com/&quot; rel=&quot;noreferrer&quot;&gt;in DYNALOOK&lt;/a&gt;. Some of the problems being solved might satisfy your requirements on being comparable to FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.dynalook.com/european-conf-2009/I-I-01.pdf&quot; rel=&quot;noreferrer&quot;&gt;C. T. Wu, &quot;New Features in LS-DYNA EFG Method for Solids and Structures Analysis,&quot; in &lt;em&gt;7th Eu. LS-DYNA User's Conf.&lt;/em&gt;, Salzburg, Austria, 2009.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.dynalook.com/international-conf-2008/MetalForming4-3.pdf&quot; rel=&quot;noreferrer&quot;&gt;H. Lu, C. T. Wu, and J. Xu, &quot;Forging and extrusion analysis with LS-DYNA® using 3D adaptive EFG method,&quot; in &lt;em&gt;10th Int. LS-DYNA User's Conf.&lt;/em&gt;, Detroit, MI, 2008.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.dynalook.com/13th-international-ls-dyna-conference/fluid-structure-interaction/an-adaptive-meshfree-galerkin-method-for-the-three-dimensional-thermo-mechanical-flow-simulation-of-friction-stir-welding-process&quot; rel=&quot;noreferrer&quot;&gt;C. T. Wu, W. Hu, and H. P. Wang, &quot;An adaptive meshfree Galerkin method for the three-dimensional thermo-mechanical flow simulation of friction stir welding process,&quot; in &lt;em&gt;13th Int. LS-DYNA User's Conf.&lt;/em&gt;, Detroit, MI, 2014.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-06T04:29:46.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29464" PostTypeId="2" ParentId="29308" CreationDate="2018-05-06T08:46:33.210" Score="1" Body="&lt;p&gt;For scattered data at points with no structure, try&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/3104781/inverse-distance-weighted-idw-interpolation-with-python/3119544#3119544&quot;&gt;inverse-distance-weighted-idw-interpolation-with-python&lt;/a&gt;.&#xA;This combines&#xA;scipy's fast &lt;a href=&quot;https://en.wikipedia.org/wiki/K-d_tree&quot; rel=&quot;nofollow noreferrer&quot;&gt;K-d_trees&lt;/a&gt;&#xA;with inverse-distance aka radial kernels:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\qquad interpol( x ) = \sum w_i \, f_i$, sum over say 10 points nearest $x$&lt;br&gt;&#xA;$\qquad \qquad \qquad \qquad w_i = {1 \over |x - x_i|^p}$ normalized so $\sum w_i = 1$ .&lt;br&gt;&#xA;It's only 40 lines of code, so easy to understand and modify.&#xA;Runtimes on a stock iMac:&lt;br&gt;&#xA;0.2 sec to build a Kd tree for 170k points in 3d&lt;br&gt;&#xA;25 sec to query 100k points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For rectangular grids aka box grids in 2d, 3d and up, data at e.g.&#xA;$m \times n \times k$ points $(x_i, y_j, z_k)$ with no NAs, see&#xA;&lt;a href=&quot;http://denis-bz.github.io/docs/intergrid.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Intergrid&lt;/a&gt;,&#xA;a wrapper for &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/tutorial/ndimage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.ndimage.map_coordinates&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Needless to say, it's a good idea to check any interpolation.&#xA;An easy method is to split the data into say 90 % $train$ + 10 % $test$ points,&#xA;and compare $interpolate( \, train \to test \, )$ with $true( \, test \, )$ .)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also:&lt;br&gt;&#xA;&lt;a href=&quot;https://gis.stackexchange.com/questions/tagged/interpolation+python&quot;&gt;gis.stackexchange.com/questions/tagged/interpolation+python&lt;/a&gt;&#xA;on gis.stack&lt;br&gt;&#xA;&lt;a href=&quot;https://earthscience.stackexchange.com/questions/12057/how-to-interpolate-scattered-data-to-a-regular-grid-in-python&quot;&gt;how-to-interpolate-scattered-data-to-a-regular-grid-in-python&lt;/a&gt;&#xA;on earthscience.stack&lt;br&gt;&#xA;&lt;a href=&quot;https://pyresample.readthedocs.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyresample&lt;/a&gt;,&#xA;&lt;a href=&quot;http://jswhit.github.io/pyproj/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyproj&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2018-05-11T10:02:08.487" LastActivityDate="2018-05-11T10:02:08.487" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29465" PostTypeId="1" CreationDate="2018-05-06T13:31:50.513" Score="1" ViewCount="98" Body="&lt;p&gt;Typically an ODE System which involves 2 different physical problems such as diffusion and advection can be numerically approached by the well known Strang operator splitting scheme. I'm wondering if there's a generalization of this scheme to an arbitrary number of subproblems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, there's an additional issue: I got 4 subproblems, two of which have a smaller time-scale (Maxwell equations with time-varying medium) than the remaining 2 (advection/diffusion).&#xA;Any ideas, keywords, hints?&lt;/p&gt;&#xA;" OwnerUserId="21797" LastEditorUserId="20688" LastEditDate="2018-05-06T21:13:03.787" LastActivityDate="2018-05-07T06:48:54.450" Title="Operator splitting for 4 subproblems" Tags="|numerical-modelling|operator-splitting|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29467" PostTypeId="2" ParentId="29457" CreationDate="2018-05-06T15:36:06.877" Score="1" Body="&lt;p&gt;In the friction-free case the Lagrangian formulation for the system with the given constraints would be&#xA;$$&#xA;L(t,x,\dot x)=\frac12\dot x^TM\dot x-\frac12x^TKx+\lambda^T\Pi(x-x_p)&#xA;$$&#xA;where the Lagrangian parameter $λ$ is also a function of $t$. $\Pi$ is the partial isometry that reduces the position to the components that get prescribed. The Euler Lagrange equations then give&#xA;$$&#xA;0=\frac{d}{dt}\frac{\partial L}{\partial \dot x}-\frac{\partial L}{\partial  x}&#xA;= M\ddot x+Kx-Π^Tλ.&#xA;$$&#xA;The constraint $Πx=Πx_p$ leads to $Π\dot x=Π\dot x_p$ and&#xA;$$&#xA;Π\ddot x_p=Π\ddot x=ΠM^{-1}(Π^Tλ-Kx)\impliesλ= (ΠM^{-1}Π^T)^{-1}Π(\ddot x_p+M^{-1}Kx)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what the &quot;physical&quot; way would be to incorporate the friction term.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2018-05-06T15:36:06.877" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29468" PostTypeId="2" ParentId="29465" CreationDate="2018-05-06T21:20:37.193" Score="1" Body="&lt;p&gt;Sure, see in Numerical Recipes &quot;Operator Splitting Methods Generally&quot; and follow the references there...&lt;/p&gt;&#xA;" OwnerUserId="4325" LastActivityDate="2018-05-06T21:20:37.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29469" PostTypeId="2" ParentId="29465" CreationDate="2018-05-06T22:00:21.573" Score="1" Body="&lt;p&gt;In general, a problem with different parts can be split, then each part is solved separately, then the solutions for each flow are composed back to obtain the final solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a description of these methods you can refer to &lt;em&gt;A concise introduction to Geometric Numerical Integration&lt;/em&gt; by Blanes &amp;amp; Casas and to &lt;em&gt;Geometric Numerical Integration&lt;/em&gt; by Hairer, Lubich, Wanner. But, as far as I remember, they do not give example of 3D problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a more specific example, maybe &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0377042716304101&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A splitting approach for the magnetic Schrödinger equation&lt;/em&gt;&lt;/a&gt;, where kinetic-potential-advection splitting is performed, will be of use.&lt;/p&gt;&#xA;" OwnerUserId="22974" LastEditorUserId="22974" LastEditDate="2018-05-07T06:48:54.450" LastActivityDate="2018-05-07T06:48:54.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29471" PostTypeId="1" CreationDate="2018-05-08T06:14:17.957" Score="0" ViewCount="597" Body="&lt;p&gt;I was using Metatrader5 and have designed a strategy for trading using MQL5 programming language.&lt;br&gt;&#xA;While I was running a Strategy Optimization process, I saw the it will need 10,00= iterations or possibilities. My CPU core were not capable of processing it faster. So I thought of why not to have GPU cores. i have GTX Titan Xp which has almost 4000 CUDA Cores.&lt;br&gt;&#xA;But the optimization only works with CPU cores.&lt;br&gt;&#xA;Hence, thought of whether it is possible to make the CUDA core to act like a CPU core for the optimization. But I could not get any way out. I tried using OpenCL which supported in the MQL5. But all my effort are wasted.&lt;br&gt;&#xA;Hence, need help to convert the GPU to CPU for temporary purpose, o that the process get executed effortlessly and my CPU-GPU combinedly can be helpful.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me.&lt;/p&gt;&#xA;" OwnerUserId="27579" LastActivityDate="2018-05-08T17:46:02.080" Title="Can I convert CUDA core to CPU core and use it as cpu core while running any program?" Tags="|optimization|algorithms|convergence|gpu|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29472" PostTypeId="1" CreationDate="2018-05-08T07:10:05.620" Score="3" ViewCount="53" Body="&lt;p&gt;Suppose I want to approximate the integral $\int_0^1 x^2\,dx$ using Riemann Sums or Darboux sums over &lt;strong&gt;random&lt;/strong&gt; partitions of the interval $[0,1]$, Like in the image below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/upA50.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/upA50.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, A &lt;em&gt;&quot;random&quot;&lt;/em&gt; &lt;em&gt;partition&lt;/em&gt; of $[0,1]$ containing $16$ partition points, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$P=\{0, 0.048499, 0.097571, 0.25324, 0.28515, 0.4087, 0.45946, 0.53824,  0.62423     0.67352, 0.72305, 0.84254, 0.85201, 0.87255, 0.91006, 1\}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;was created randomly with corresponding lower Darboux sum (showed in red) = $0.292536$ and upper Darboux sum (showed in blue) = $0.377436$, And we've got an approximation of  $\int_0^1 x^2\,dx = \frac{1}{3} = 0.333333\ldots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which Books or other resources do you suggest to read that treats rigourously problems like this, Both theoretical (Just the mathematical theory used to address such problems) and practical (Approximating integrals on a computer in such manner)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some challenges I was thinking about that need addressing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First&lt;/strong&gt;, What is a &lt;strong&gt;random partition&lt;/strong&gt;? It needs to be rigorously defined...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second&lt;/strong&gt;, What about partitions that their points are distributed according to other probability distributions such as the Gaussian distribution or others, And how can we know which probability distribution will give the best result (probabilistically) to approximate an integral?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Third&lt;/strong&gt;, How can we answer questions such as, &lt;em&gt;&quot;What is the probability that the approximated value of the integral is within $0.5$ from the exact value of the integral?&quot;&lt;/em&gt;, i.e. How can we calculate probabilities like $$P\left\{\int_0^1 x^2 \,dx - 0.5 &amp;lt; \text{LOWER SUM}&amp;lt;\int_0^1 x^2\,dx &amp;lt; \text{UPPER SUM} &amp;lt; \int_0^1 x^2\,dx + 0.5\right\} \text{?}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks...&lt;/p&gt;&#xA;" OwnerUserId="27686" LastActivityDate="2018-05-08T07:10:05.620" Title="Book Suggestion for Approximating Integrals using Random Partitions" Tags="|integration|probability|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29473" PostTypeId="1" AcceptedAnswerId="29474" CreationDate="2018-05-03T16:48:10.863" Score="1" ViewCount="45" Body="&lt;p&gt;Given a system $Ax = b$, I'm coding a linear solver in Java that takes a triangular matrix $A$ (with 500 to 3,000 rows and columns) and a vector $b$ and solves for $x$. However, the rows and columns in $A$ are unordered and I'm looking for the quickest way to solve this system without pre-arranging such rows and columns to do back substitution. I have tried third party solvers but they are very slow for my application. What is the fastest way you guys know of for this special case? I don't think LU decomposition would help.&#xA;Note: Matrix $A$ is sparse!&lt;/p&gt;&#xA;" OwnerDisplayName="wilbertonunez" LastActivityDate="2018-05-08T15:55:28.397" Title="Solve system involving unordered triangular matrix" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29474" PostTypeId="2" ParentId="29473" CreationDate="2018-05-04T00:18:37.957" Score="1" Body="&lt;p&gt;You say that $A$ is triangular. That means you don't have to decompose it into anything. It is already in $LU$ form where $L = I$ and $U = A$ (or vice versa, depending on whether $A$ is upper or lower triangular).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or would be, I presume, if someone hadn't dropped the card deck. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think your best bet is effectively to sort the rows based on index of the lowest non-zero column, thus converting into an actual upper triangular matrix. Once sorted, start with the last row, and back-substitute into previous rows. You might speed things up a bit by doing some of the substitution while you are sorting, but I don't think you will see more than minor gains this way. 3000 entries doesn't take long for a good sorting algorithms to handle.&lt;/p&gt;&#xA;" OwnerDisplayName="Paul Sinclair" LastActivityDate="2018-05-04T00:18:37.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29475" PostTypeId="1" AcceptedAnswerId="29501" CreationDate="2018-05-08T16:18:59.410" Score="0" ViewCount="67" Body="&lt;p&gt;I have this data, and I need to detect the outliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The outliers are clearly visible on the borders (start and end, in red). And I only care for outliers present at the start or end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jOYgb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jOYgb.png&quot; alt=&quot;Screen capture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could easily make a rule as &quot;all values over 10 are outliers&quot;, but I do not have any assurance that the distance will always be over 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could also say &quot;a big distance between points means an outlier&quot;, but again, it is arbitrary to say what is a &quot;big distance&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, part of the licit data has (in green) some outliers characteristics, compared to the mainstream data (in blue).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had been checking some algorithms for detection of outliers, but they always assume something wrong, like a gaussian distribution, or just declaring &quot;outliers&quot; to all quartiles over 75%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any more general criteria for detection of outliers?&lt;/p&gt;&#xA;" OwnerUserId="27690" LastActivityDate="2018-05-11T21:16:13.713" Title="Is there any robust criteria for this kind of outlier?" Tags="|statistics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29476" PostTypeId="2" ParentId="29471" CreationDate="2018-05-08T17:46:02.080" Score="4" Body="&lt;p&gt;I assume, that you have a code that works on a standard CPU. I am not particularly familiar with MQL and Metatrader, but I don't think the answer will be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For compilable languages, the compiler creates an executable that is tailored to use the computational resources available to it for a given architecture (or many architectures). CPUs and GPUs are very different, and the computing cores in them offer different sets of instructions. So, without &lt;strong&gt;actually coding specifically for GPUs&lt;/strong&gt; (or using a library that already provides the GPU acceleration for certain functions), I don't think you can take advantage of your GPU cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ways to code for GPUs, you might be interested in OpenCL as it is featured on the &lt;a href=&quot;https://www.mql5.com/en/articles/690&quot; rel=&quot;nofollow noreferrer&quot;&gt;MQL5 website&lt;/a&gt; with a description of what to do.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-08T17:46:02.080" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29477" PostTypeId="1" AcceptedAnswerId="29483" CreationDate="2018-05-09T01:18:18.423" Score="3" ViewCount="643" Body="&lt;p&gt;For example, the convex function $f(t)=e^{-t}$ doesn't achieve its minimum 0 on the real line.  In a linear regression with $p$ predictors $X$, the loss function $f(\beta)=||Y-X\beta||^2$ achieves its minimum. But for a penalized objective $g(\beta)= ||Y-X\beta||^2+\lambda\sum_{j=1}^p|\beta_j|$ with $\lambda &amp;gt; 0$, does $g(\beta)$ achieve its minimum? How do I prove/disprove this in general?&lt;/p&gt;&#xA;" OwnerUserId="27664" LastEditorUserId="1804" LastEditDate="2018-05-09T15:27:20.130" LastActivityDate="2018-05-10T07:33:29.987" Title="How do I check if a loss function can achieve its minimum?" Tags="|optimization|convex-optimization|constrained-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29479" PostTypeId="2" ParentId="29477" CreationDate="2018-05-09T07:56:35.103" Score="0" Body="&lt;p&gt;The only thing you can in general say about convex optimisation problems is: If there is a local minimum, this is the global minimum. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be relaxed to: If the function has a stationary point, and is convex, this stationary point is a global minimum. &#xA;Stationary points $\beta^*$ are such that fulfill $\nabla f(\beta^*) =0$.&#xA;Your function is a sum of two convex functions, which makes it convex.&#xA;Your function is non-differentiable in $0$. However, it may still have a stationary point $\beta^*$ that is not $0$. If you find such a stationary point, you are done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, we have &#xA;$$g(\beta)= \|Y-X\beta\|^2+\lambda\sum_{j=1}^p|\beta_j|,$$&#xA;this implies $\nabla g(\beta) = -2(Y-X\beta)X + \nabla h(\beta)$,&#xA;where the $h(\beta) =  \lambda\sum_{j=1}^p|\beta_j|$ and &#xA;$$&#xA;(\nabla h(\beta))_j = \begin{cases}\lambda, &amp;amp;\beta_j \geq 0 \\ -\lambda, &amp;amp;\beta_j &amp;lt; 0\end{cases}.&#xA;$$&#xA;Now you &quot;just&quot; have to find $\beta^*$, which may be easier with a semi-smooth Newton method.&lt;/p&gt;&#xA;" OwnerUserId="27670" LastEditorUserId="27670" LastEditDate="2018-05-09T11:57:51.900" LastActivityDate="2018-05-09T11:57:51.900" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29481" PostTypeId="1" CreationDate="2018-05-09T08:53:20.563" Score="3" ViewCount="298" Body="&lt;p&gt;I have the following question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We have two face image files (JPEG), a Matrix of $128\times 128$ with values between 0-255.&lt;/li&gt;&#xA;&lt;li&gt;We would like to hash both image files using a function $f(x, key)$. Where I know $x$, but don't know $key$. The $key$ is being sent by another authority (a).&lt;/li&gt;&#xA;&lt;li&gt;It should not be possible to revert $f(x, key)$ back to the images. Both should be hashes now.&lt;/li&gt;&#xA;&lt;li&gt;Is there a function to compare the similarity of both face images which are now hashed, but not revertable?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27695" LastEditorUserId="20688" LastEditDate="2018-05-09T15:57:04.507" LastActivityDate="2018-05-12T23:50:44.797" Title="Image hash similarity matching possible?" Tags="|matrix|algorithms|approximation|image-processing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29482" PostTypeId="1" AcceptedAnswerId="29484" CreationDate="2018-05-09T09:30:14.533" Score="0" ViewCount="301" Body="&lt;p&gt;I am struggling with the following exercise from the book of &lt;a href=&quot;http://doi.org/10.1007/978-0-387-40065-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nocedal, Numerical optimization&lt;/a&gt;, chapter 2, exercise 2.12:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that a function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; of two variables is poorly scaled at the solution &lt;span class=&quot;math-container&quot;&gt;$x^*$&lt;/span&gt;. Write two Taylor expansions of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; around &lt;span class=&quot;math-container&quot;&gt;$x^*$&lt;/span&gt; - one along each coordinate direction - and use them to show that the Hessian &lt;span class=&quot;math-container&quot;&gt;$\nabla^2 f (x^*)$&lt;/span&gt; is ill-conditioned.&lt;/p&gt;&#xA;" OwnerUserId="27696" LastEditorUserId="20688" LastEditDate="2019-06-22T21:26:46.620" LastActivityDate="2019-06-22T21:26:46.620" Title="ill-conditioning" Tags="|optimization|conditioning|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29483" PostTypeId="2" ParentId="29477" CreationDate="2018-05-09T10:19:29.597" Score="8" Body="&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; The property you are looking for is &lt;strong&gt;coercivity&lt;/strong&gt;. This is satisfied for the example in your question (as are properties 1 and 3 below), and hence &lt;strong&gt;yes, the penalized objective attains its minimum.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;hr/&gt;&#xA;&#xA;&lt;p&gt;The classical proof of existence of minimizers of (a very general class of) functionals is the so-called &lt;em&gt;direct method&lt;/em&gt; of the calculus of variations (originally due to Hilbert and developed in its current form by Tonelli). For the sake of completeness, I give the full version in infinite-dimensional spaces, but you can ignore the fine details if you are only interested in finite-dimensional problems. If you &lt;em&gt;are&lt;/em&gt; interested in the full details, I have a set of lecture notes on nonsmooth optimization at &lt;a href=&quot;https://arxiv.org/abs/1708.04180&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1708.04180&lt;/a&gt;. (Not that this is the best reference; it's just that I &lt;em&gt;know&lt;/em&gt; it's in there.)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let $X$ be a reflexive Banach space (in particular, $\mathbb{R^N}$) and $F:X\to \mathbb{R}\cup \{+\infty\}$ (this allows including hard constraints into the functional). If $F$ is&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;strong&gt;proper&lt;/strong&gt; (meaning that it is not equal to $\infty$ everywhere),&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;strong&gt;coercive&lt;/strong&gt; (meaning $\|x\|\to\infty$ implies $F(x)\to \infty$), and&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;&lt;strong&gt;weakly lower semicontinuous&lt;/strong&gt; (meaning if $x_n \to x$ weakly, then $F(x)\leq \liminf_{n\to\infty} F(x_n)$),&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;  &#xA;  &lt;p&gt;then the minimum of $F$ over $X$ is attained in some $\bar x\in X$&#xA;  (for which $F(\bar x)&amp;lt;\infty$).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The proof is fairly simple and uses these properties in a straightforward way:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Since $F$ is proper, the set $\{F(x):x\in X\}\subset\mathbb{R}$ attains an infimum $M:=\inf_{x\in X} F(x)&amp;lt;\infty$ (although $M=-\infty$ may be possible). By the definition of the infimum, there exists a sequence $\{x_n\}_{n\in\mathbb{N}}\subset X$ with&#xA;       $$&#xA;            F(x_n) \to M = \inf_{x\in X} F(x).&#xA;       $$&#xA;        Such a sequence is called &lt;em&gt;minimizing sequence&lt;/em&gt;. Note that from the convergence of $\{F(x_n)\}_{n\in\mathbb{N}}$ we cannot conclude the convergence of $\{x_n\}_{n\in\mathbb{N}}$ (yet).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Suppose now that $\{x_n\}_{n\in\mathbb{N}}$ were unbounded, i.e., that $\|{x_n}\|_X\to\infty$ for $n\to \infty$. The coercivity of $F$ then implies that $F(x_n)\to \infty$ as well, in contradiction to $F(x_n)\to M&amp;lt;\infty$ by definition of the minimizing sequence.&#xA;        Hence, the sequence is bounded and thus contains a weakly (in finite-dimensional spaces, strongly) converging subsequence $\{x_{n_k}\}_{k\in\mathbb{N}}$ with limit $\bar x\in X$. (This limit is a candidate for the minimizer.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;From the definition of the minimizing sequence, we also have $F(x_{n_k})\to M$ for $k\to\infty$. Together with the weak lower semicontinuity of $F$ and the definition of the infimum we thus obtain &#xA;        \begin{equation*}&#xA;            \inf_{x\in X} F(x) \leq F(\bar x) \leq \liminf_{k\to\infty} F(x_{n_k}) = M = \inf_{x\in X} F(x)&amp;lt;\infty.&#xA;        \end{equation*}&#xA;        This implies that $F(\bar x)&amp;lt;\infty$ and that $\inf_{x\in X} F(x)=F(\bar x)&amp;gt; -\infty$. Hence, the infimum is attained in $\bar x$ which is therefore the desired minimizer.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note how these three properties serve to exclude the obvious counterexamples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;excludes the obviously degenerate case that there's nothing to minimize over and is clearly satisfied for any real-valued functional;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;excludes exactly your counterexample $f$; for $g$, you have&#xA;$$g(\beta) = \|{Y-X\beta}\|_2^2 + \lambda \|\beta\|_1 \geq C\|\beta\|_1,$$&#xA;and hence the penalized objective is indeed coercive.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;excludes functions such as &#xA;$$f(t) = \begin{cases} t^2+1 &amp;amp; t\leq 0 \\ t^2 &amp;amp; t&amp;gt;0 \end{cases} $$&#xA;for which the infimum is $0$, but this is not attained for any $t\in \mathbb{R}$. In $\mathbb{R}^N$, you don't need to worry about the difference of weak and strong convergence; in particular, the property is satisfied for any continuous $F$ (such as all your examples).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Note that convexity didn't play any role here; what this buys you is that any minimizer is a global minimizer (which is unique if the function is strictly convex). Also, convex functions are weakly lower semicontinuous if they are strongly lower semicontinuous even in infinite-dimensional spaces, where the former is a stronger property than the latter.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2018-05-10T07:33:29.987" LastActivityDate="2018-05-10T07:33:29.987" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="29484" PostTypeId="2" ParentId="29482" CreationDate="2018-05-09T14:22:30.690" Score="2" Body="&lt;p&gt;It might help to start with a concrete example of a function, like $f(x,y)=10^9 x^2+y^2$ (the example they use on p.26). I'm trying to guess what went wrong with your analysis, because you haven't given much information.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am mostly struggling with cross-terms of hessian.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Consider&#xA;$$f_1(x,y) = 10^9 x^2+y^2$$&#xA;and its rotation by $\theta=\frac\pi4$&#xA;$$f_2(x,y) = 10^9 (x\cos\theta + y\sin\theta)^2 + (-x\sin\theta + y\cos\theta)^2.$$&#xA;These are equivalent, and both are poorly scaled, but in different directions. If you try to analyze the Hessian in a way that just assumes that the function is poorly scaled &lt;em&gt;specifically in the coordinate directions&lt;/em&gt; (only the diagonal terms are important, and so you don't care about cross terms), it's not going to work for the second function. I think you just need a different way of proving ill-conditionedness of the Hessian.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-05-09T14:22:30.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29485" PostTypeId="1" CreationDate="2018-05-09T14:38:22.757" Score="1" ViewCount="122" Body="&lt;p&gt;For example, I have a function to optimize:&#xA;$$f_1(x,y) = x^2+y^2, \quad x_{lb}\le x\le x_{ub},\quad y_{lb}\le y\le y_{ub}$$&#xA;Then I apply rotation by $\theta$ plus translation by $x_0$ and $y_0$:&#xA;$$f_2(x,y) = ((x+x_0)\cos\theta + (y+y_0)\sin\theta)^2 + (-(x+x_0)\sin\theta + (y+y_0)\cos\theta)^2$$&#xA;Intuition tells these functions are equivalent given bounds are far enough from $x_0$ and $y_0$.&#xA;However, practical optimization methods, e.g. Differential Evolution, usually have problems with rotated and translated functions - they optimize original functions well, but no so good the rotated versions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I'd like to know some formal general proof that for optimization (especially non-convex case) these functions should be equivalent. And it's also interesting to hear reasoning why DE optimization methods may fail.&lt;/p&gt;&#xA;" OwnerUserId="27549" LastEditorUserId="27549" LastEditDate="2018-05-09T14:59:04.177" LastActivityDate="2020-04-05T11:01:00.123" Title="Some proof that linear translations and rotations of a bound-constrained function are equivalent" Tags="|optimization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29489" PostTypeId="1" AcceptedAnswerId="29493" CreationDate="2018-05-10T04:50:02.510" Score="7" ViewCount="273" Body="&lt;p&gt;Are there any drawbacks to using the Method of Manufactured Solutions for convergence testing and verification studies? I really can't think of any.&lt;/p&gt;&#xA;" OwnerUserId="27504" LastActivityDate="2018-05-10T15:55:40.343" Title="Are there any drawbacks to using the Method of Manufactured Solutions for convergence testing?" Tags="|verification|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29490" PostTypeId="1" AcceptedAnswerId="29581" CreationDate="2018-05-10T09:55:14.483" Score="0" ViewCount="185" Body="&lt;p&gt;At the Section 4.2 of &lt;a href=&quot;http://acoustics.ae.illinois.edu/pdfs/lele-1992.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper (which is very well known in the computational fluid dynamic community), the author claims that it is enough, for the &lt;strong&gt;compact finite difference&lt;/strong&gt; formulation in eq. 4.2.3 (as applied to problem 4.2.1) to be conservative, that the matrix $B$ (note, the derivation matrix is $A^{-1}B$) have all columns but first and last which sum to zero, but gives no proof or reference for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well I can't get this obvious proof.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've searched on the web for other papers (citing or not this one) and found some &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0021999112005712&quot; rel=&quot;nofollow noreferrer&quot;&gt;results&lt;/a&gt; (see Section 2.1 and Appendix A) and proofs for cases which are much simpler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author first considers the model equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_t f + \partial_x F = 0, \quad x \in [a,b] \subseteq \mathbb R, t \in \mathbb R^+$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for the quantity $f(x,t)$, which has flux $F(x,t) = \hat{F}(f(x,t))$. By integrating over $[a,b]$, it is shown that $f$ is conserved, in that the total $f$ in the domain $[a,b]$ changes in time only due to the flux $F$ at the boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d}{dt}\int^b_a f(x,t) dx = - (F(b,t) - F(a,t))\tag{1}\label{integral}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(All this is clear to me so far.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point the author seeks a discrete analog for Eq. \eqref{integral}.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I would do is the following.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To keep it simple and get the basic idea, I would consider a linear version of the model equation, &lt;em&gt;i.e.&lt;/em&gt; the linear advection equation with $F = f$,&#xA;$$\partial_t f + \partial_x f = 0$$&#xA;then I would discretize it in space,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\text d}{\text dt} \textbf f + \textbf D \textbf f = 0 \tag{2}\label{discretized}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\textbf f$ contains nodal values of $f$ and $\textbf D$ is a derivation matrix. A reasonable discrete analog for the integral $\int^b_a f(x,t) dx$ is, in my opinion, the sum of all nodal values of $\textbf f$, &lt;em&gt;i.e.&lt;/em&gt; $\textbf 1^{\text T} \textbf f$, being $\textbf 1$ the unitary column, so I'd multiply Eq. \eqref{discretized} by $\textbf 1^{\text T}$ (and take it in the time derivative), obtaining&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\text d}{\text dt} \textbf 1^{\text T} \textbf f = - \textbf 1^{\text T}\textbf D \textbf f \tag{3}\label{guilty}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point I'd conclude that, for the LHS of Eq. \eqref{guilty} to depend only on first and last elements of $\textbf f$ in the way as Eq. \eqref{integral}, the row vector $\textbf 1^{\text T}\textbf D$ should be $(-1,0,0,\dots,0,0,1)$, &lt;em&gt;i.e.&lt;/em&gt; all columns of $\mathbf D$ should sum up to zero, except the first and last (which should sum up to $-1$ and $1$ respectively).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Turning point&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper in question is about compact finite difference schemes, in which case the matrix $\mathbf D$ can be written as $\mathbf A^{-1} \mathbf B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author claims that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In order to satisfy the global conservation constraint it is&#xA;  sufficient to require that the columns $2$ through $N-1$ of the matrix&#xA;  $\mathbf B$ sum exactly to zero. &lt;strong&gt;This ensures that only the boundary&#xA;  nodes contribute to the boundary fluxes&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which is clearly not consistent with what I wrote just after Eq. \eqref{guilty}.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So either the author is wrong (unlikely, since the paper has been cited more than five thousands times since 1992), or I misunderstood what he means by &lt;em&gt;global conservation constraint&lt;/em&gt; as applied to the space-discrete equation or, in other words, multiplying Eq. \eqref{discretized} by $\mathbf 1^\text{T}$ is not the intended discrete analog of integrating $f$ over the domain.&lt;/p&gt;&#xA;" OwnerUserId="18846" LastEditorUserId="18846" LastEditDate="2018-05-14T12:58:32.777" LastActivityDate="2018-05-22T13:49:31.100" Title="Conservative formulation for compact finite difference schemes" Tags="|finite-difference|fluid-dynamics|banded-matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29491" PostTypeId="2" ParentId="29490" CreationDate="2018-05-10T10:45:40.947" Score="1" Body="&lt;p&gt;The equation:&#xA;$$\partial_t f + \partial_xF = 0$$  with $f_{x=0} = f_0(t)$ (which is known) and appropiate initial conditions can be discretised as follows:&#xA;$$\partial_t\vec{f}+D\vec{F}=\vec{b}\tag{*}$$&#xA;Where $\vec{f} = (f_1,f_2,...,f_{N+1})^T$ and $D$ is the discrete operator which replaces $\partial_x$, and is equal to:&#xA;$$D = \frac{1}{h}\left[\begin{matrix}1 &amp;amp;  &amp;amp;  &amp;amp; &amp;amp;\\&#xA;                          -1 &amp;amp;1&amp;amp;&amp;amp;&amp;amp;\\&#xA;                             &amp;amp; -1&amp;amp;1&amp;amp;&amp;amp;\\&#xA;                             &amp;amp;   &amp;amp;-1&amp;amp;1&amp;amp;&amp;amp;\\&#xA;                             &amp;amp;   &amp;amp;  &amp;amp; \ddots&amp;amp; \ddots&#xA;\end{matrix}\right] $$&#xA;and $\vec{b} = (F(f_0(t))/h,0,...,0)^T$ the boundary condition (inflow at first node).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conservation implies that if we sum all fluxes of equation $(*)$ multiplying by a vector $\vec{1}=(1,1,...)$ we must obtain the total variation of fluxes $f_T = \sum_if_i$ &#xA;$$\partial_t f_T+\vec{1}^TD\vec{F}=F(f_0(t))/h$$&#xA;since $\vec{1}^TD = (0,0,...,1)$&#xA;$$\partial_t f_T = F(f_0(t))/h-\vec{F}(L)/h$$&#xA;which is the conservation requirement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this applies for both cases, in which node $0$ or $N+1$ are inflow boundaries (matrix $D$ changes accordingly) we can say that all the sum of all the columns must add to $0$, as shown ($\vec{1}^TD$ must be equal to $(1,0,0,...)$ or $(0,0,....,1)$ if the inflow is at the last or first node respectively).&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-05-10T10:45:40.947" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29493" PostTypeId="2" ParentId="29489" CreationDate="2018-05-10T15:49:17.747" Score="9" Body="&lt;p&gt;I think MMS is generally a good technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were to offer criticism, however, it would be that in my discipline (electromagnetic radiation and scattering), it leads one to spending time/money on implementing mathematics that really don't correspond to actual use cases. MMS requires injecting arbitrary dirichlet/neumann/robin data on boundaries and arbitrary volume sources, but real physical models don't actually use these (they use port boundary conditions, modal expansion boundary conditions, total-field/scattered-field decompositions for sources.. all of these are weirder constructions that fall outside the scope of basic MMS). Overall this makes MMS feel rather tautological: it insists of me that I write extra code that is only of use/coveraged by MMS itself, and yet MMS remains rather silent when it comes to bigger/harder questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an alternative to implementing MMS for electromagnetism, I recommend instead to implement cavity eigenproblem tests (ie compute resonant frequencies of a closed cavity of canonical shape, subject to zero-valued dirichlet or neumann conditions). It can tell you similar things as MMS: am I obtaining the correct rates of convergence as I fiddle with geometry/basis orders and mesh density. Unlike MMS, this doesn't require any additional nonzero boundary-data/body-load terms. And yet, it actually solves a physical problem of genuine engineering interest! (resonant frequencies of general shapes). It does carry the drawback of more complicated algebraic machinery (solving a sparse eigenproblem instead of sparse linear system), but there are good black-box libraries for either case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that said, my opinion is shaped by my domain of interest (linear electromagnetism), so other PDE's might not have such a tidy/useful alternative to MMS. I certainly won't criticize other practitioners for thorough testing! It's more a sentiment that, in my use cases, MMS entails additional work but doesn't give very much additional information.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-05-10T15:55:40.343" LastActivityDate="2018-05-10T15:55:40.343" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29495" PostTypeId="1" AcceptedAnswerId="29509" CreationDate="2018-05-10T20:50:18.327" Score="6" ViewCount="2980" Body="&lt;p&gt;I am looking for an C/C++/Python algorithm implementation that calculates eigenvalues and eigenvectors of a symmetric, positive semidefinite covariance matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A general-purpose eigen-decomposition algorithm has about $O(n^3)$ complexity, but maybe a faster method exists for symmetric, positive semidefinite covariance matrices.&lt;/p&gt;&#xA;" OwnerUserId="27549" LastEditorUserId="20417" LastEditDate="2018-05-14T08:13:25.947" LastActivityDate="2021-04-29T17:48:27.843" Title="Efficient eigen-decomposition of covariance matrix" Tags="|linear-algebra|python|c++|eigenvalues|c|" AnswerCount="4" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29496" PostTypeId="2" ParentId="29495" CreationDate="2018-05-10T21:48:33.170" Score="3" Body="&lt;p&gt;TL;DR: I don't think you will be able to beat any real/vendor eigensolver, but it's fun to think about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deeper cut: One classic algorithm for symmetric eigendecomposition is tridiagonalizing A=Q&lt;em&gt;T&lt;/em&gt;Q' via householder methods, followed up by QR iteration upon T. The QR iteration is (very) loosely based on the iteration: [Q,R] = A; A = R*Q. That is, alternating between QR decomposition and then multiplying them out in reverse order. In the limit of many iterations, A will converge to a diagonal matrix (thus displaying the eigenvalues) and is also similar (same eigenvalues) to the original input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For symmetric positive definite A, I think you could in theory beat this algorithm using a treppeniteration-like method based on Cholesky decomposition [Consult Golub &amp;amp; Van Loan 3rd ed, chapter 8, problem 8.2.1]. It would be (very) loosely based on the iteration [G,G'] = chol(A); A = G'*G. That is, computing a Cholesky decomposition and then multiplying them in reverse order. Remarkably, this converges to a diagonal matrix too, which is similar to the original input. The departure from orthogonal iterations is mild cause for concern, but fortunately the Cholesky decomposition  is quite stable, too. You would also want to &quot;frontend&quot; this algorithm using householder tridiagonalization, so that all the A's in question become tridiagonal and the Cholesky's are all banded ones, with band=1. Fundamentally, band=1 Cholesky is less flops/simpler than band=1 QR via givens rotations, so that's how you could (possibly) come out on top.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you could also &quot;accumulate&quot; all these similarity transforms as you go (essentially, banded backsolution steps), to build the eigenvectors. This is much how the givens rotations in are accumulated in the classic QR iteration. If this is unworkable for some reason, you can always use inverse iteration at the end once you have the eigenvalues in hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that said, real/vendor solvers are not just performing the dumb QR/RQ/QR/RQ iteration .. there's (at the very least) shifting, plus a whole wider class of (faster!) algorithms in the field (divide and conquer, MRR, bisection, etc). This is a ridiculous amount of machinery/cumulative improvements to try to compete against .. many (!!) man-years of effort have been put into the development of these algorithms and their implementations (EISPACK/LAPACK/MKL/etc). I think it's an interesting thought experiment (wow, an eigensolver built from such a simple decomposition) but not a very practical one.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2018-05-10T21:48:33.170" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29497" PostTypeId="1" AcceptedAnswerId="29498" CreationDate="2018-05-11T01:49:05.310" Score="1" ViewCount="114" Body="&lt;p&gt;I've been discretizing PDEs and formulating $Ax=b$ systems, and yet I don't really know what the $A$ and $b$ are in words.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I occasionally call the $A$ matrix the &quot;Jacobian matrix,&quot; but for linear PDEs, &quot;Jacobian&quot; doesn't make sense, so I call it the &quot;differential operator matrix.&quot; Is there a standard name for this $A$ matrix? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the $b$ vector, I call it the forcing vector, but I don't know if that is correct either. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there standard names for both of these variables after applying a discretization to a PDE (whether linear or nonlinear)? &lt;/p&gt;&#xA;" OwnerUserId="27504" LastEditorUserId="20688" LastEditDate="2018-05-11T02:42:40.163" LastActivityDate="2018-05-11T07:30:36.133" Title="What are the names of the variables in the linear system $Ax=b$" Tags="|linear-system|terminology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29498" PostTypeId="2" ParentId="29497" CreationDate="2018-05-11T02:42:13.467" Score="3" Body="&lt;p&gt;$A$ is a discretized version of your differential operator + enforced boundary conditions. The names for $A$ can vary depending on the way the PDE is being discretized. For example, in FEM, it will be &lt;em&gt;stiffness matrix&lt;/em&gt;. For integral equation methods (technically not a PDE) applied to Maxwell equations, such a matrix is usually called &lt;em&gt;impedance matrix&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I assume that you are using finite differences to discretized PDE. I don't know a special name, but I've seen in multiple papers such notions as &lt;em&gt;finite difference matrix&lt;/em&gt;, &lt;em&gt;discretized operator&lt;/em&gt;, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the RHS, &lt;em&gt;forcing vector&lt;/em&gt; is something that I've seen a lot around; though, usually in the meaning of the &lt;em&gt;forcing function&lt;/em&gt; that produces the RHS in a discretized form. Another way to name $b$ is &lt;em&gt;excitation&lt;/em&gt;, which is more intuitive, as it describes the way the problem is setup and &quot;excited&quot; (sorry for the tautology).&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-11T02:42:13.467" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29499" PostTypeId="2" ParentId="29495" CreationDate="2018-05-11T07:25:06.003" Score="0" Body="&lt;p&gt;I think, for $100 \times 100$, you should be happy with the eig method in Matlab, or numpy.linalg.eig in Numpy/Python. Since it is a covariance operator, it is symmetric positive semidefinite. Probably you can speed up your implementation by passing this information through the solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For such small matrices there is little chance to solve it faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are indeed methods to solve the problem faster, if your covariance matrix has some structure - and is much bigger than $100 \times 100$:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If your matrix is Toeplitz or Block-Toeplitz with Toeplitz blocks, &lt;strong&gt;circulant embedding&lt;/strong&gt; can be used to sample in $O(N \log N)$. &lt;a href=&quot;https://www.nag.co.uk/doc/techrep/pdf/tr1_14.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to a tutorial.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are multipole methods that can be used to speed-up eigendecompositions of covariance operators. See this &lt;a href=&quot;ftp://ftp.math.ethz.ch/pub/sam-reports/reports/reports2006/2006-01.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;technical report&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You need to solve the eigendecomposition for several matrices. If there is some underlying structure, you could solve the problem on a reduced basis. &lt;a href=&quot;https://arxiv.org/abs/1804.11157&quot; rel=&quot;nofollow noreferrer&quot;&gt;There is a preprint&lt;/a&gt; from me and my colleagues that may help you.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27670" LastActivityDate="2018-05-11T07:25:06.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29500" PostTypeId="2" ParentId="29497" CreationDate="2018-05-11T07:30:36.133" Score="2" Body="&lt;p&gt;Here is some generic (application-independent) terminology I've seen in papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$A$: The ``coefficient matrix''&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$b$: The ``right hand side''&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$x$: The ``unknown''&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, sometimes $A$ may be called the ``coefficient operator'' if it is considered as a linear operator rather than a matrix.&lt;/p&gt;&#xA;" OwnerUserId="1502" LastActivityDate="2018-05-11T07:30:36.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29501" PostTypeId="2" ParentId="29475" CreationDate="2018-05-11T21:16:13.713" Score="1" Body="&lt;p&gt;If the histogram you're displaying is representative than you could methods which are used for binarization in Image Processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, using &lt;a href=&quot;https://en.wikipedia.org/wiki/Otsu%27s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Otsu's Method&lt;/a&gt; will probably be a robust way to set the threshold you're after.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/19HDr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/19HDr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7951" LastActivityDate="2018-05-11T21:16:13.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29502" PostTypeId="2" ParentId="26530" CreationDate="2018-05-11T21:20:53.067" Score="1" Body="&lt;p&gt;In Compressed Sensing the real problem is the $ {\left\| \cdot \right\|}_{0} $ norm (Well, Pseudo Norm).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the great discoveries are conditions under which using $ {\left\| \cdot \right\|}_{1} $ will yield the same results as using $ {\left\| \cdot \right\|}_{0} $.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can find geometrical intuition why $ {L}_{1} $ promotes sparsity yet real sparse cost function is the $ {\left\| \cdot \right\|}_{0} $ norm.&lt;/p&gt;&#xA;" OwnerUserId="7951" LastActivityDate="2018-05-11T21:20:53.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29504" PostTypeId="2" ParentId="29436" CreationDate="2018-05-12T10:00:03.997" Score="1" Body="&lt;p&gt;I have found a solution which converts NL format to C program. NL format can be obtained from AMPL input easily.&#xA;&lt;a href=&quot;https://ampl.com/netlib/ampl/solvers/nlc/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ampl.com/netlib/ampl/solvers/nlc/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27549" LastActivityDate="2018-05-12T10:00:03.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29505" PostTypeId="1" CreationDate="2018-05-12T18:03:15.177" Score="1" ViewCount="119" Body="&lt;p&gt;In the case of partial occupancies (for example, this &lt;a href=&quot;http://www.crystallography.net/cod/1009065.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIF&lt;/a&gt;), a supercell should be made for DFT calculations. However, numerous supercells can be built from a lattice (in the above example, 200 different supercells).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Which supercell should be used for the DFT calculations?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many DFT works (if not most) are based on partial occupancies (e.g., dopants). How is a typical supercell chosen for the DFT calculation?&lt;/p&gt;&#xA;" OwnerUserId="2115" LastActivityDate="2018-05-12T18:03:15.177" Title="How to make supercell of partial occupancies for DFT?" Tags="|density-functional-theory|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29506" PostTypeId="1" AcceptedAnswerId="29510" CreationDate="2018-05-12T20:13:57.227" Score="3" ViewCount="1358" Body="&lt;p&gt;I have to write a little finite elements code in C. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was asked to implement the conjugate gradients method, which I have done. Now, I am looking to improve further the efficiency of my program by using the Incomplete Cholesky Decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, instead of solving the $ Ax = B $ system, I have to solve the $ M^{-1} Ax = M^{-1}B$ system where matrix $M$ is the incomplete Cholesky decomposition of matrix $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I tried to implement this, I found that computing $M$ and then inverting it was more expensive in time than simply using the CG method.&#xA;I suppose I must have done something wrong, but I really don't see how PCG with Cholesky is implemented.&lt;/p&gt;&#xA;" OwnerUserId="27723" LastEditorUserId="20688" LastEditDate="2018-05-12T23:53:49.473" LastActivityDate="2018-05-14T13:50:01.200" Title="Finite Elements: using preconditioned conjugate gradients with incomplete cholesky decomposition" Tags="|linear-algebra|linear-solver|iterative-method|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="29507" PostTypeId="2" ParentId="29481" CreationDate="2018-05-12T20:23:35.010" Score="1" Body="&lt;p&gt;Assuming you are talking about a cryptographic hash (as opposed to a hash used for speeding lookups, say), I think you are posing contradictory requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of a hash is to remove any information from the input values, and essentially look like a random mapping &lt;a href=&quot;https://simple.wikipedia.org/wiki/Cryptographic_hash_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cryptographic Hash&lt;/a&gt;.  Thus using it to compare whether 2 inputs are close (similar) in some way should not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you are asking whether you could compare the similarity of 2 faces, you could think of a mapping face-&gt;vector of facial measurements, which you could then compare in some vector space definition of distance.  But you would not have a 'key' involved.  What is the purpose of the key?  You want to hide information about the nature of the faces involved?  However, I don't think that can work, because then closeness measures will fail.&lt;/p&gt;&#xA;" OwnerUserId="27724" LastActivityDate="2018-05-12T20:23:35.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29508" PostTypeId="2" ParentId="29481" CreationDate="2018-05-12T23:50:44.797" Score="0" Body="&lt;p&gt;Disclaimer: cryptography and hashing are not my major areas of expertise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, say we have two images $A$ and $B$ with corresponding hashes $h_A$ and $h_B$, respectively. Here, $h_A=f(A,key)$ and $h_B=f(B,key)$, where $f$ is the hashing function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also want that $f^{-1}(h_A,[key])$ does not exist, implying that it involves extremely computationally expensive procedures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we want to know, if there is a function $g(h_A,h_B)$ that is going to measure a similarity between $A$ and $B$ given their hashes $h_A$ and $h_B$. However, that directly contradicts with the requirements on $f^{-1}$ being computationally expensive as it gives me an option to construct $f^{-1}$ procedure by constructing a sequence of more-and-more similar images that will iteratively converge to the original.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I would say that even if such function exists, such process severely compromises non-recoverability of the image by having only its hash.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;h2&gt;Appendix&lt;/h2&gt; (construction of $f^{-1}$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input: $h_A$, $f$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Procedure:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take initial image guess $B^{(0)}$ (random, or based on something known)&lt;/li&gt;&#xA;&lt;li&gt;Calculate $h_B^{(0)}$ using $f$&lt;/li&gt;&#xA;&lt;li&gt;Calculate $g(h_A,h_B^{(0)}$)&lt;/li&gt;&#xA;&lt;li&gt;If value of $g$ is less than tolerance -&gt; stop. Otherwise continue.&lt;/li&gt;&#xA;&lt;li&gt;Find new image guess $B^{(1)}$ based on some procedure (genetic algorithms, black-box optimization, etc.). Return to step 2.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Of course, it is hard to find an efficient procedure of finding a new image guess $B^{(i+1)}$ based on $B^{(i)}$ and $g(h_A,h_B^{(i)})$; however, it is still significantly better than trying to crack a good hashing function.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-12T23:50:44.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29509" PostTypeId="2" ParentId="29495" CreationDate="2018-05-13T08:39:12.927" Score="2" Body="&lt;p&gt;There &lt;em&gt;are&lt;/em&gt; specialized methods for the eigendecomposition of symmetric matrices. LAPACK has &lt;code&gt;DSYEV&lt;/code&gt;, Numpy has &lt;code&gt;numpy.linalg.eigh&lt;/code&gt;. They are still $O(n^3)$, but overall cheaper and more accurate. You should use them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not familiar with JAMA, but from what I could Google and understand in a minute it is a pure-java port of an old pre-LAPACK implementation of the QR/Francis algorithm. That might be what is slowing you down. Try replacing it with native code: find whatever library wraps LAPACK in your language, and use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, you can't get any further speedup from the fact that your matrix is positive semidefinite nor from the fact that it has diagonal 1 (a property that is sometimes assumed when people say &quot;covariance matrix&quot;). If you could, cheap scaling and shifting tricks would permit to apply the same speedups to all symmetric matrices, I think.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2018-05-13T08:46:07.680" LastActivityDate="2018-05-13T08:46:07.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29510" PostTypeId="2" ParentId="29506" CreationDate="2018-05-14T08:54:31.163" Score="6" Body="&lt;p&gt;While preconditioned CG with incomplete Cholesky (ICC) is reasonably straightforward to formulate mathematically, writing an &lt;em&gt;efficient&lt;/em&gt; implementation is a non-trivial matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's some of the things that would make for an efficient implementation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the ICC implementation exploits the sparsity of your matrix $A$ (you'll want to store $A$ and $M$ in some sparse matrix format)&lt;/li&gt;&#xA;&lt;li&gt;$M$ is computationally &quot;easy&quot; to invert (you won't compute the actual inverse $M^{-1}$, just its action on a vector)&lt;/li&gt;&#xA;&lt;li&gt;you set appropriate hyper-parameters for the ICC algorithm (e.g., block size, levels of fill-in, zero threshold) and (pre-)allocate memory space for $M$ accordingly&lt;/li&gt;&#xA;&lt;li&gt;miscellaneous optimizations (e.g., the &lt;em&gt;Eisenstat trick&lt;/em&gt;, see the paper by Chan)&lt;/li&gt;&#xA;&lt;li&gt;code optimizations that target your specific hardware platform (I won't go into specifics here).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;At the end of the day, what you're after is a preconditioned iteration that converges &lt;em&gt;faster&lt;/em&gt; (in some objective measure) than its un-preconditioned counterpart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References to get you started:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Yousef Saad, &lt;a href=&quot;http://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iterative methods for sparse linear&#xA;  systems&lt;/a&gt; -&#xA;  in particular chapters 9.2, 9.2.1 and 9.2.2 deal with the preconditioned&#xA;  CG iteration (there is some pseudocode that you can base your initial implementation on).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Chan and van der Vorst, &lt;a href=&quot;https://www.researchgate.net/publication/2291679_Approximate_And_Incomplete_Factorizations&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Approximate and Incomplete&#xA;  Factorizations&lt;/em&gt;&lt;/a&gt;&#xA;  (they discuss incomplete LU and its variants, but the ideas there&#xA;  apply to ICC as well).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Chih-Jen Lin and Jorge J. Moré, &lt;a href=&quot;https://epubs.siam.org/doi/abs/10.1137/S1064827597327334&quot; rel=&quot;nofollow noreferrer&quot;&gt;Incomplete Cholesky Factorizations&#xA;  with Limited&#xA;  Memory&lt;/a&gt;, SIAM SISC 21(1), 1999&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Some high-quality C/C++ ICC implementations (these libraries have a fairly steep learning curve, but the source code is available for inspection):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;PETSc's &lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/pc/impls/factor/icc/icc.c.html#PCICC&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICC routines&lt;/a&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://trilinos.org/packages/ifpack&quot; rel=&quot;nofollow noreferrer&quot;&gt;IFPACK&lt;/a&gt; (part of the Trilinos&#xA;  suite). The user manual can be found &lt;a href=&quot;http://trilinos.org/oldsite/packages/ifpack/IfpackUserGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Eigen's &lt;a href=&quot;http://eigen.tuxfamily.org/dox/classEigen_1_1IncompleteCholesky.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;C++ ICC preconditioners for CG&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2018-05-14T13:50:01.200" LastActivityDate="2018-05-14T13:50:01.200" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29512" PostTypeId="1" CreationDate="2018-05-14T16:25:43.510" Score="1" ViewCount="69" Body="&lt;p&gt;In a book for solving a nonlinear differential equation with $N+1$ points, $u_{xx} = e^{u}, u(-1)=u(1) = 0$, in $[-1,1]$ with homogeneous Dirichlet boundary conditions, the fixed point iteration is used. The MATLAB code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u = zeros(N-1,1);&#xA;change = 1; it = 0;&#xA;while change &amp;gt; 1e-15&#xA;       unew = D2\exp(u);&#xA;       change1 = change;&#xA;       change = norm(unew-u,inf);&#xA;       change/change1&#xA;       u = unew; it = it+1;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where $D^2$ is a differential matrix. In each step the error is reduced by a factor of (change/change1) 0.2943. In the book it is asked how this factor is obtained? and what is its relationship with the eigenvalues of the $D^2$ matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I evaluated the eigenvalues of the matrix $D^2$, but could not find a relation. How is the reduction factor in fixed point method obtained?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="22763" LastActivityDate="2018-05-14T16:25:43.510" Title="Fixed point iteration reduction factor" Tags="|iterative-method|fixed-point|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29518" PostTypeId="1" CreationDate="2018-05-15T03:49:26.650" Score="0" ViewCount="115" Body="&lt;p&gt;I have a set of points which forms a closed loop, and I want to perform cutting/clipping a 3D model using this loop. I have used VTK but in some cases, it has a &quot;Cannot follow edges&quot; problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any free library I can use to do such operation?&lt;/p&gt;&#xA;" OwnerUserId="27741" LastEditorUserId="20688" LastEditDate="2018-05-16T20:19:18.993" LastActivityDate="2018-10-14T00:00:03.330" Title="Which free library should I use to perform cutting/clipping operation?" Tags="|computational-geometry|software|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29521" PostTypeId="2" ParentId="27351" CreationDate="2018-05-15T12:10:49.453" Score="0" Body="&lt;p&gt;For the simple case where the matrix $A$ is symmetric positive-definite,&#xA;calculate its say 20 biggest eigenvalues,&#xA;and see if they're $\to 0$, or compare norms.&#xA;ARPACK is fast for this; more important,&#xA;it needs only a function $x \to A \, x$ .&#xA;So for general $A$, look at the eigenvalues of $A^T A$&#xA;(as a LinOp, without instantiating it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.svds.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.sparse.linalg.svds&lt;/a&gt;&#xA;does this: LinOp$( A^T A ) \to$ Arpack, for $A$ of any size:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.sparse.linalg import svds&#xA;sing = svds( A, k=20, tol=1e-4, return_singular_vectors=False )  # v0=random&#xA;# runtimes on random-normal n x n:&#xA;# n = 100, 1k, 2k&#xA;#       5, 130, 770 ms&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2018-05-16T08:16:39.133" LastActivityDate="2018-05-16T08:16:39.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29523" PostTypeId="1" CreationDate="2018-05-15T20:11:48.367" Score="1" ViewCount="179" Body="&lt;p&gt;I have 20-80 users and 5-10 events with varying ranges of minimum and maximum number of free seats (2-4, 3-5, 2-6...). For example, with a range of 3-5 it is acceptable to only assign three users to the event. The users can choose weighted preferences (1,2,3) for 1-3 events. The problem is to assign as many as possible users to the events in an optimal way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using &lt;a href=&quot;https://en.wikipedia.org/wiki/Hungarian_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hungarian algorithm&lt;/a&gt; and the JavaScript implementation is done using &lt;a href=&quot;https://github.com/addaleax/munkres-js&quot; rel=&quot;nofollow noreferrer&quot;&gt;munkres-js&lt;/a&gt;. If there is no solution, events and finally users can be dropped until there is a solution. This produces satisfactory non-optimal solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current problem is how to allow the users to form predefined groups of varying sizes (2-8) before the optimization? The group cannot select an event if the maximum number of seats is smaller than the size of the group. This can be handled before the algorithm execution. The group can choose three weighted preferences in the same way as individual users. For example, out of 40 users there are three predefined groups of 4, 5, and 6 people and the remaining 25 users will be assigned individually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My initial idea was to assign all users to predefined groups and then assign the groups to the events but I could not figure how to do it with varying groups sizes and event seats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Non-optimal approximations and constraints that don't change the scope too much are also fine. There is a time constraint of ~30 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;// EDIT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proposed constraints:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Limit group size to two people. This is feasible because common scenario is that people want to join an event with one friend.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;// EDIT2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;List of requirements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Goal&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The goal is to assign as many users as possible to the events. &lt;/li&gt;&#xA;&lt;li&gt;User preferences must be honored.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Events&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Events have minimum and maximum number of seats (for example: 3-5).&lt;/li&gt;&#xA;&lt;li&gt;If an event doesn't receive the minimum number of users, the event must be cancelled.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Individual users&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Users can choose up to three (1-3) weighted preferences (1,2,3) for as many different events (1-3).&lt;/li&gt;&#xA;&lt;li&gt;Users can only be assigned to events they have chosen. &lt;/li&gt;&#xA;&lt;li&gt;If there are more users than free seats, the excessive users must be removed fairly.&lt;/li&gt;&#xA;&lt;li&gt;If users cannot be matched to their preferences, they must be removed. This should be the last measure.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Groups&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Users can form groups of varying sizes (2-8).&lt;/li&gt;&#xA;&lt;li&gt;Groups can choose three weighted preferences like individual users.&lt;/li&gt;&#xA;&lt;li&gt;The group cannot select an event if the maximum number of seats is smaller than the size of the group. This can be handled before the algorithm execution.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Other&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Algorithm must be able to execute in under 30 seconds.&lt;/li&gt;&#xA;&lt;li&gt;Non-optimal approximations are fine. &lt;/li&gt;&#xA;&lt;li&gt;New constraints can be introduced if they don't change the scope too much.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27751" LastEditorUserId="27751" LastEditDate="2018-05-20T18:27:22.273" LastActivityDate="2018-05-20T18:27:22.273" Title="Algorithm for group forming: as individual or in a preformed group" Tags="|algorithms|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29524" PostTypeId="2" ParentId="27351" CreationDate="2018-05-15T20:24:15.270" Score="4" Body="&lt;p&gt;Another approach worth trying is to use Adaptive Cross Approximation (ACA). It is a pretty popular algorithm that has many implementations available online. For the reference, you can see the original paper:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1016/S0024-3795(96)00301-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;S. A. Goreinov, E. E. Tyrtyshnikov, N. L. Zamarashkin, &quot;A theory of pseudoskeleton approximations,&quot; &lt;em&gt;Linear Algebra Appl.&lt;/em&gt;, vol. 261, no. 1–3,  pp. 1–21, Aug. 1997.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;ACA and its variations (say, ACA+, hybrid cross approximation HCA) can be used in different scenarios. You, already having the whole dense matrix computed is one of the favorable, as you will be able to calculate residuals exactly if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If heuristical residuals (see the algorithm) suffice, I believe your complexity will be $\mathcal O(Nr)$, where $N$ is the size of the square matrix and $r(\epsilon)$ is the rank. Note, that rank $r$ is a function of the prescribed truncation tolerance $\epsilon$. While the exact and guaranteed error bounds will require $\mathcal O(N^2r)$.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-15T20:24:15.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29525" PostTypeId="1" AcceptedAnswerId="29533" CreationDate="2018-05-15T21:23:34.240" Score="0" ViewCount="204" Body="&lt;p&gt;I frequently work with Keras and tensorflow-gpu on my Lenovo P50 workstation which has 64GB ram and a 2GB Nvidia QUadro M1000M GPU. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today while training my model on GPU, it ran into memory error and it exit at this line:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;da_ta = np.zeros((len(data),sequence_length, cls.get_vector_size(model)), dtype=K.floatx())&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is how to compare 64GB ram vs 2 GB gpu?&lt;br&gt;&#xA;Is one better than the other?&lt;br&gt;&#xA;Kindly migrate my question to the appropriate forum if this is not the right place. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.  &lt;/p&gt;&#xA;" OwnerUserId="27753" LastActivityDate="2018-05-16T16:57:08.873" Title="Compare 64gb dd3 ram with 2gb Quadro M1000M gpu on Lenovo P50" Tags="|gpu|memory-management|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29526" PostTypeId="1" AcceptedAnswerId="29537" CreationDate="2018-05-15T21:36:53.980" Score="6" ViewCount="900" Body="&lt;p&gt;While answering one of the questions on meshing process, I encountered a lack of understanding on my end for the comparison of the mesh quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, consider an unstructured mesh created in GMSH for a 2-D shape (I am not interested in this particular shape, just here for the illustration). The elements are controlled only through the characteristic length, which basically dictates, how many elements are created along the geometry boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Bz5O4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Bz5O4.png&quot; alt=&quot;Fig. 1 Unstructured mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I can detect the regions in the mesh that are &quot;kind of suitable&quot; for a structured mesh. That could be done automatically, say, using marching cubes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9UjGI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9UjGI.png&quot; alt=&quot;Fig. 1 Hybrid structured/unstructured mesh mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, both meshes satisfy the original criteria for the size of the element. But the second mesh has a lot of elements that are exactly the same - allowing for significant savings in the required amount of computations while featuring a similar number of triangular elements (even a little bit less).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the second mesh bad in general? Does a lot depend on the kind of basis functions that will be used?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I am particularly interested in using the meshes in frequency-domain computational electromagnetics, where the common meshing is done at $\lambda/10$, $\lambda$ is the wavelength. Say, both meshes satisfy $\lambda/10$ requirement (largest triangular edge is smaller than $\lambda/10$). Is there something fundamentally bad by using this type of a hybrid structured/unstructured mesh?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: I am usually using integral-equation methods; however, don't mind having an understanding of what happens, in general, and for FEM and others, in particular.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-05-16T14:20:25.057" LastActivityDate="2018-09-15T03:59:48.380" Title="Unstructured mesh vs hybrid structured/unstructured for numerical simulations" Tags="|finite-element|mesh-generation|integral-equations|unstructured-mesh|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29527" PostTypeId="2" ParentId="29526" CreationDate="2018-05-16T01:17:34.680" Score="3" Body="&lt;p&gt;I think that both meshes are good. But depending of the problem at hand, one might be better than the other. In the domain that you are mentioning, one thing to consider is that you probably don't want element sizes to be too different to satisfy the sampling criterion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding mesh quality I would suggest that you check this post:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/27090/9667&quot;&gt;Commonly-used metrics to quantify the irregularity of a triangular mesh&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And, if you check reference 1, you would see that depending on the problem at hand some criteria would be better than others, anisotropy in your might be a good thing , for example. You would like to have a finer discretization in directions with higher derivatives (gradients).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the meshing of the geometry in your example. I would probably go with something like the following&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wVUKH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wVUKH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I have fine tuning over discretization levels for different regions. Thus, I can refine wherever I find it needed according to the solution for a coarser mesh.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Jonathan Richard Shewchuk (2002). &lt;a href=&quot;https://people.eecs.berkeley.edu/~jrs/papers/elemj.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;What is a Good Linear Finite Element?&quot;&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Pébay, Philippe, and Timothy Baker. &lt;a href=&quot;http://www.ams.org/journals/mcom/2003-72-244/S0025-5718-03-01485-6/S0025-5718-03-01485-6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Analysis of triangle quality measures.&quot;&lt;/a&gt; Mathematics of Computation 72.244 (2003): 1817-1839.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-05-16T01:17:34.680" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29528" PostTypeId="2" ParentId="29485" CreationDate="2018-05-16T03:32:02.770" Score="1" Body="&lt;p&gt;Let $f: \mathbb R^N→\mathbb R$, $x \in \mathbb R^N$ and $z=t(x)$ where $ t$ is a bijective mapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x^* = \text{argmax}_x f(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x_1^*= \text{argmax}_x f(t(x))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using $z=t(x)$, and applying $t$ on both sides,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$t(x_1^*)= t\left(\text{argmax}_{t^{-1}(z)} f(z)\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$t(x_1^*)= \text{argmax}_{z} f(z) = x^*$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For intuition, we can think of the transformation as doing some kind of rearrangement of points in the domain. So the max value of the function remains unchanged. The argument which maximizes will be different. We get the original solution by transforming the argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are already doing the argument transformation, then may be you need to transform the initial points also to get similar results as that of the original optimization. &lt;/p&gt;&#xA;" OwnerUserId="24653" LastEditorUserId="20688" LastEditDate="2018-05-16T03:57:03.367" LastActivityDate="2018-05-16T03:57:03.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29529" PostTypeId="1" AcceptedAnswerId="29546" CreationDate="2018-05-16T04:19:37.600" Score="3" ViewCount="688" Body="&lt;p&gt;I've posted the &lt;a href=&quot;https://gamedev.stackexchange.com/q/158017/1087&quot;&gt;same question&lt;/a&gt; at GameDev Stack Exchange, but unfortunately I am not getting any response. So I am going to post ( and reword) it here. Hopefully I can get an answer!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a terrain ( in the form of mesh/element), and I want to indicate how the water flows in the terrain, subjected to gravity force and the terrain condition ( of course).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A crude way of doing this is to take every single mesh element, then draw the water flow direction from the highest point to the lowest, but I think this will result in a lot of ugly arrows that although can roughly point towards the correct directions, but really don't look nice when view as a whole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm, or even better, an existing code implementation ( in the form of well-documented APIs with Windows DLLs, or open source soure code), that allows me to generate the water &quot;flow map&quot; in an elegant, intuitive, look-nice way?&lt;/p&gt;&#xA;" OwnerUserId="690" LastEditorUserId="690" LastEditDate="2018-05-18T06:56:48.747" LastActivityDate="2018-05-18T06:56:48.747" Title="Algorithm to generate water flow map, given a terrain" Tags="|fluid-dynamics|computational-physics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29531" PostTypeId="2" ParentId="29529" CreationDate="2018-05-16T05:41:07.437" Score="0" Body="&lt;p&gt;There is one program, called IBER, that fits into your request:&#xA;&lt;a href=&quot;http://www.iberaula.es/space/53/modules&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.iberaula.es/space/53/modules&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can download it for free here:&#xA;&lt;a href=&quot;http://www.iberaula.es/space/54/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.iberaula.es/space/54/downloads&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-05-16T05:41:07.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29532" PostTypeId="1" CreationDate="2018-05-16T16:43:28.003" Score="1" ViewCount="3522" Body="&lt;p&gt;&lt;strong&gt;Why do scientists bother with the &lt;code&gt;np&lt;/code&gt; for &lt;code&gt;numpy&lt;/code&gt;&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note this is more a &quot;philosophical&quot;/reasoning question - let us assume that you get your code to perform as intended regardless of choice of syntax)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I write code for scientific applications, mathematical functions such as &lt;code&gt;sqrt&lt;/code&gt;, as well as arrays and the many other features of Numpy are &quot;bread and butter&quot; - ubiquitous and taken for granted. For this reason, I always use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;instead of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;despite nearly every online example I see. Indeed, a colleague of mine who found my code useful said that they had to go through it and insert &lt;code&gt;np&lt;/code&gt; everywhere. And yet, why bother? I can honestly say that in the thousands of lines of Python I've written, those three characters would have been redundant and a waste of my time. And that time is not trivial: as stated in many SE discussions, programmer time is more valuable than CPU time. If you want a super optimized program, use &lt;code&gt;C&lt;/code&gt; or &lt;code&gt;Fortran&lt;/code&gt; for HPC instead of &lt;code&gt;python&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import sqrt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is also an option, but also has drawbacks for rapidly making a script where you might want to use a new function out of the blue (as you would in &lt;code&gt;C&lt;/code&gt;, &lt;code&gt;Fortran&lt;/code&gt;, &lt;code&gt;Matlab&lt;/code&gt;, whatever).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that there is a library for accurately calculating the square root of a complex number - then it's still faster to &lt;code&gt;import complxlib as cplx&lt;/code&gt;, use &lt;code&gt;cplx.sqrt&lt;/code&gt; when necessary and &lt;code&gt;sqrt&lt;/code&gt; the other 99% of the time. So is there a real example or argument as to why my approach is bad practice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that the question is specifically about &lt;code&gt;numpy&lt;/code&gt; - a staple in scientific application of python, rather than e.g. &lt;code&gt;super_esoteric_library8472&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27763" LastEditorUserId="27763" LastEditDate="2018-05-18T14:08:25.150" LastActivityDate="2018-05-18T14:08:25.150" Title="Why `import numpy as np` for scientific computing?" Tags="|python|numpy|" AnswerCount="1" CommentCount="8" ClosedDate="2018-05-16T21:07:05.607" ContentLicense="CC BY-SA 4.0" />
  <row Id="29533" PostTypeId="2" ParentId="29525" CreationDate="2018-05-16T16:57:08.873" Score="1" Body="&lt;p&gt;In my opinion there is not a memory type better than other. Simply they are different things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normal ram are used only by the cpu, and the gpu ram is used onnly by the gpu.&#xA;This is quite clear when you code direct in CUDA, i.e. if you want that the gpu use some data in normal ram you must move them to the gpu memory (host -&gt; device). And if you want use some gpu results inside the cpu you must move them (device -&gt; host). Note that in some languages there are &quot;tricks&quot; to avoid explic passage, for example &lt;a href=&quot;https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-unified-memory-programming-hd&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unified Memory Programming&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The underlying system manages data access and locality within a CUDA program without need for explicit memory copy calls.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Behind the scene there are movement (note is better this or manual movement this is another question....).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keras and tensorflow  are made to hide this level of detail, so you do not see it direct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the above motivation is not correct to compare the two memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to have an idea about a size comparison, here is heavy dependent by the problem. I try to explain with CUDA. A  gpu works with a parallel execution of a N times of the same function (read &lt;a href=&quot;https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#kernels&quot; rel=&quot;nofollow noreferrer&quot;&gt;kernel&lt;/a&gt;) with different data. See &lt;a href=&quot;https://en.wikipedia.org/wiki/SIMD&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIMD&lt;/a&gt; every new data is assigned to a different thread. Now  normally you have got a number of cases to elaborate &gt;&gt; of N so the gpu groups and organize the threads, see &lt;a href=&quot;https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#thread-hierarchy&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, in pool of N and these pools are execute sequentially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With some simplification you can consider how many memory every thread use (here heavy dependency by the problem) and multiply for N. In this way you have got a rough estimate of the memory used by the gpu and you can understand if 2 GB of gpu are enough for 64 gp of cpu ram.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2018-05-16T16:57:08.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29534" PostTypeId="2" ParentId="29532" CreationDate="2018-05-16T17:29:36.010" Score="8" Body="&lt;p&gt;I would say, that it can be explained by the following famous programming principle:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Explicit is better than implicit&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Usually, that is applied to types; however, it can be applied to namespaces (as mentioned by @Mauro Vanzetto) as well as particular libraries/packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I, personally, like having &lt;code&gt;np.sqrt&lt;/code&gt; or &lt;code&gt;std::cout&lt;/code&gt; (as opposed to just &lt;code&gt;sqrt&lt;/code&gt; and &lt;code&gt;cout&lt;/code&gt;) because typing that does not cost me a lot and I can feel sure that no tricky variable/function hiding can happen and explicitly see if I am using a function coming from a certain namespace/library/package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In scientific computing, in particular, when even the order of summation can change the result significantly in some cases, I want to avoid a certain type of problems, to begin with – and simplify my life in debugging by explicitly pointing to where each function I am using is coming from.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-16T17:29:36.010" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="29535" PostTypeId="2" ParentId="29518" CreationDate="2018-05-16T20:18:06.077" Score="1" Body="&lt;p&gt;It is a bit suspicious that VTK (and supposedly Paraview) has troubles with your mesh: there is a slight chance that there are some troubles with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution would be to try using &lt;a href=&quot;http://gmsh.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMSH&lt;/a&gt;. It supports STL input and certainly has clipping capability. Now, with the newest versions, it also supports access to its functionality via API for at least C/C++/Python.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-16T20:18:06.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29537" PostTypeId="2" ParentId="29526" CreationDate="2018-05-17T05:51:10.570" Score="8" Body="&lt;p&gt;In my opinion, it is not a good neither a bad mesh. It clearly depends on the PDE you are considering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite space to which the PDE is projected is your mesh, where your operators, e.g. $\vec{\textrm{grad}}$ (gradients), $\textrm{div}$ (divergences), $\triangle$ (Laplacians)... strongly depend  on that mesh and become matrices:&#xA;$$ \vec{\textrm{grad}}\rightarrow\vec{\textrm{grad}}_h=G\quad \textrm{div}\rightarrow\textrm{div}_h=D \quad \triangle\rightarrow\triangle_h=L$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Elliptic problems do not suffer from problems associated with the mesh, in terms of isotropy/anisotropy because the nature of the associated operator (Laplacian) is isotropic (invariant under translations and rotations), therefore your mesh would be suitable for elliptic problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, other operators like the $\vec{\textrm{grad}}$ depend strongly on the mesh, because they are anisotropic (depend on the direction). This fact is important if you are going to project a hyperbolic equation (Fluid dynamics) onto your mesh, where a prevailing direction arises and thus the solution is affected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example of these matters try to compute the following problems with these two meshes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Mesh a) Square orthogonal mesh.&lt;/li&gt;&#xA;&lt;li&gt;Mesh b) Split mesh a) into triangles. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Problem 1) Solve, for example, the elliptic problem $-\triangle u=f\; \text{in}\;\Omega$ with $u=0\,\forall x\in\partial \Omega$ and a smooth $f$ in the two meshes. Use for example Q1 finite elements for the mesh a) and P2 FE in your mesh b). Compare them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problem 2) Solve, for example, the hyperbolic problem $\partial_t u-(\vec{v}^{T}\vec{\textrm{grad}})u = 0$ with $u=g(t)$ and $u(0)=0$ in the inflow boundary, in the two meshes (with $\vec{v}$ in the orthogonal/parallel direction of the boundaries). Use for example finite volumes of second order. Compare them. Now solve both problems with $\vec{v}$ entering the domain with a certain angle. Compare them again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In problem 1) you won't see great changes in your solution, while in problem 2, the orthogonal mesh is really suitable when solving for $\vec{v}$ oriented perpendicularly to the inflow boundary but misbehaves for an arbitrary angle (clearly the problem is not isotropic). For the triangular mesh, you shouldn't note such a great difference, because a certain anisotropy is already introduced by the mesh.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="20688" LastEditDate="2018-09-15T03:59:48.380" LastActivityDate="2018-09-15T03:59:48.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29538" PostTypeId="2" ParentId="28456" CreationDate="2018-05-17T09:37:04.563" Score="4" Body="&lt;p&gt;Late answer but the matrix can be thought of the following sum of rank-$2$ matrices which allows for having smaller block sizes as the number of blocks increase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\begin{align*}&#xA;c_{20}=\left(&#xA;\begin{array}{cc}&#xA;\frac{a}2 &amp;amp; 1 \\&#xA;b &amp;amp; 0\\&#xA;c &amp;amp; 0\\&#xA;d &amp;amp; 0\\&#xA;e &amp;amp; 0\\&#xA;\vdots&amp;amp;\vdots\\&#xA;r &amp;amp; 0\\&#xA;s &amp;amp; 0\\&#xA;t &amp;amp; 0&#xA;\end{array}&#xA;\right)&#xA;\left(&#xA;\begin{array}{cccccccccccccccc}&#xA;1&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;\cdots&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;\\\frac{a}2 &amp;amp; b &amp;amp; c &amp;amp; d &amp;amp; e &amp;amp;\cdots &amp;amp; r &amp;amp; s &amp;amp; t&#xA;\end{array}&#xA;\right)\begin{pmatrix}x_1\\\vdots\\x_{20}\end{pmatrix}+\\&#xA;\left(&#xA;\begin{array}{cc}&#xA;\frac{e}2 &amp;amp; 1 \\&#xA;f &amp;amp; 0\\&#xA;h &amp;amp; 0\\&#xA;k &amp;amp; 0\\&#xA;l &amp;amp; 0\\&#xA;m &amp;amp; 0\\&#xA;o &amp;amp; 0\\&#xA;p &amp;amp; 0\\&#xA;r &amp;amp; 0&#xA;\end{array}&#xA;\right)&#xA;\left(&#xA;\begin{array}{cccccccccccccccc}&#xA;1&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;0&amp;amp;\\\frac{e}2 &amp;amp; f &amp;amp; h &amp;amp; k &amp;amp; l &amp;amp;m &amp;amp; o &amp;amp; p &amp;amp; r&#xA;\end{array}&#xA;\right)\begin{pmatrix}x_2\\\vdots\\x_{10}\end{pmatrix}+\\\textrm{etc.} \\&#xA;\end{align*}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that, by performing the second multiplication at each term, most of the calculations are vector times scalar products and &quot;get the first element&quot; type operations hence allows for significant reduction. Thus this has one inner product of decreasing size, two lookups and one vector times scalar and two scalar addition to an element operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect this matrix could have been formulated as an iterative solution instead of the linear set.&lt;/p&gt;&#xA;" OwnerUserId="21617" LastEditorUserId="21617" LastEditDate="2018-05-17T09:48:21.207" LastActivityDate="2018-05-17T09:48:21.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29539" PostTypeId="1" AcceptedAnswerId="29612" CreationDate="2018-05-17T11:09:50.400" Score="0" ViewCount="1832" Body="&lt;p&gt;I've converted a MATLAB code to C++ to speed it up, using the Armadillo library to handle matrix operations in C++, but surprisingly it is 10 times slower than the MATLAB code!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I test the Armadillo library to see if it's the cause. The below code is a simple test code that initializes two matrices, adds them together and saves the result to a new matrix. One section of code uses the Armadillo library and other one doesn't. The section using Armadillo is too slow (notice the elapsed times). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it really slow down the execution (though it is supposed to speed it up) or am I missing some thing?&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;      #include&amp;lt;iostream&amp;gt;&#xA;      #include&amp;lt;math.h&amp;gt;&#xA;      #include&amp;lt;chrono&amp;gt;&#xA;      #include&amp;lt;armadillo&amp;gt;&#xA;      #include&amp;lt;time.h&amp;gt;&#xA;      #include&amp;lt;conio.h&amp;gt;&#xA;     using namespace std;&#xA;     using namespace arma;&#xA;     int main()&#xA;       {   &#xA;        srand((unsigned int)time(NULL));&#xA;        auto start = std::chrono::high_resolution_clock::now();&#xA;        double a[100][100];&#xA;        double b[100][100];&#xA;        double c[100][100];&#xA;        for (int i = 0; i &amp;lt; 100; i++)&#xA;           {&#xA;             for (int j = 0; j &amp;lt; 100; j++)&#xA;                {   &#xA;                 a[i][j] = rand() % 10;&#xA;                 b[i][j] = rand() % 10;&#xA;                 c[i][j] = a[i][j] + b[i][j];&#xA;                 }&#xA;            }&#xA;&#xA;        auto finish = std::chrono::high_resolution_clock::now();&#xA;        std::chrono::duration&amp;lt;double&amp;gt; elapsed = finish - start;&#xA;        std::cout &amp;lt;&amp;lt; &quot;Elapsed time: &quot; &amp;lt;&amp;lt; elapsed.count() &amp;lt;&amp;lt; &quot; s\n&quot;;&#xA;        auto start1 = std::chrono::high_resolution_clock::now();&#xA;        mat a1= randi&amp;lt;mat&amp;gt;(100, 100, distr_param(1, 10));&#xA;        mat b1= randi&amp;lt;mat&amp;gt;(100, 100, distr_param(1, 10));&#xA;        mat c1=zeros(100,100);&#xA;        c1 = a1 + b1;&#xA;        auto finish1 = std::chrono::high_resolution_clock::now();&#xA;        std::chrono::duration&amp;lt;double&amp;gt; elapsed1 = finish1 - start1;&#xA;        std::cout &amp;lt;&amp;lt; &quot;Elapsed time: &quot; &amp;lt;&amp;lt; elapsed1.count() &amp;lt;&amp;lt; &quot; s\n&quot;;&#xA;        return 0;&#xA;&#xA;&#xA;    }&#xA;&#xA;        }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the answer I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Elapsed time: 0.00217959 s&#xA;Elapsed time: 0.00946147 s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you see, Armadillo is significantly slower! Is it better not to use the Armadillo library?&lt;/p&gt;&#xA;" OwnerUserId="27656" LastEditorUserId="27656" LastEditDate="2018-05-21T17:26:14.580" LastActivityDate="2018-05-27T16:32:52.390" Title="Does armadillo library slow down the execution of matrix operations?" Tags="|linear-algebra|matrix|c++|lapack|armadillo|" AnswerCount="2" CommentCount="14" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29541" PostTypeId="2" ParentId="29539" CreationDate="2018-05-17T11:38:12.387" Score="0" Body="&lt;p&gt;Have you tried initialising c1 to memory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;c1=zeros(100,100);&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it doesn't allocate the memory to the variable until you fill it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result when I run your code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Elapsed time: 0.0002079 s&#xA; Elapsed time: 0.0003315 s&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Difference not as bad... I'm no expert but maybe the way your armadillo is installed?&lt;/p&gt;&#xA;" OwnerUserId="27769" LastEditorUserId="27769" LastEditDate="2018-05-17T12:13:49.860" LastActivityDate="2018-05-17T12:13:49.860" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="29542" PostTypeId="1" AcceptedAnswerId="29543" CreationDate="2018-05-17T12:38:49.110" Score="0" ViewCount="35" Body="&lt;p&gt;Suppose I have a spherical object projected onto a discrete square mesh. The dicretised circle can be represented by filling a logical matrix such that voxels in the interior of the sphere are filled with 1's where outside of the sphere the voxels are filled with 0's. Calculating the volume is easy as you simply add up the voxels filled with 1's to get the volume.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the phase field (diffuse interface) formalism however, the edges of the sphere are diffuse interfaces, ie. the value varies smoothly  0 &amp;lt; value &amp;lt; 1 over several mesh points. How does one go about calculating the volume for such an object?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way I could think of was defining an artificial discrete boundary at value=0.5 and then summing as in the first case. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="27769" LastActivityDate="2018-05-17T13:04:55.957" Title="Calculating volume of a discretised diffuse interface object" Tags="|finite-volume|discretization|diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29543" PostTypeId="2" ParentId="29542" CreationDate="2018-05-17T13:04:55.957" Score="0" Body="&lt;p&gt;What I have understood is that you have a solid phase (sphere within a cube, represented by $\Omega$) with a continuous solid fraction called $\alpha\in[0,1]$, $\alpha=1$ when solid $\alpha=0$ when void.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The volume of the solid $V$ is written as:&#xA;$$V=\int_{\Omega}\alpha\,dV\approx\sum_{e=1}^{Ne}\int_{\Omega^{e}}\alpha^{e}\,dV\tag{*}$$&#xA;Where $Ne$ is the number of elements and $\alpha^{e}$ is a function that interpolates the function from the vertices of the $e-th$ cubic element:&#xA;$$\alpha^{e}=\sum_{i=1}^{8}\alpha^{e}_i\varphi_i(\vec{x})\tag{**}$$&#xA;Where $\varphi(\vec{x})$ are the interpolant basis functions ($\varphi_i(\vec{x}_k) = \delta_{ik}$), and $\alpha^{e}_i$ are the nodal values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral in $(*)$ applying $(**)$ is then:&#xA;$$V \approx \sum_{e=1}^{Ne}\sum_{i=1}^{8}\alpha^{e}_i\int_{\Omega^{e}}\varphi(\vec{x})\,dV$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of simplicity you can take (if they are cubes) $\varphi_{i}(\vec{x}) = \frac{1}{8}$, therefore:&#xA;$$V \approx \sum_{e=1}^{Ne}\frac{V^{e}}{8}\sum_{i=1}^{8}\alpha^{e}_i$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-05-17T13:04:55.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29544" PostTypeId="1" CreationDate="2018-05-17T13:06:09.673" Score="1" ViewCount="54" Body="&lt;p&gt;I have the following problem: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the space $E=\{1, 2, \dots, N_x\} \times \{1, 2, \dots, N_y\}$, I want to define $N_R$ rectangles $R_k=\{x_k^0, \dots, x_k^1\}\times\{y_k^0, \dots, y_k^1\}$ which cover part of my space $E$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The total size of the union of all rectangles has a upper bound $C$ (constraint i). And the rectangle size is of course lower-bounded to $s_x$ on the 1st dimension (constraint ii) and $s_y$ on the 2nd one (constraint iii).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each point of the space $E$ is associated with a score value by the function $f: E \rightarrow \mathbb R_+$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to maximize this objective function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\max_{N_R}\max_{R_1, \dots, R_{N_R}\\ s.t. |\bigcup_{k=1}^N R_k|\le C \&amp;gt; (i) \\ |x_k^1 - x_k^0| \ge s_x \quad (ii) \\ |y_k^1 - y_k^0| \ge s_y \quad  (iii)} \sum_{z\in E}\sum_{k=1}^{N_R}1_{z\in R_k}f(z)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To express it in a non-mathematical way, I want to find a set of rectangles such that the sum of the scores of the points they contain is maximal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, $E$ is quite big and I must do that for several different score functions f, so that it is impossible to brute-force all possibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know how to wisely optimize this function. Do you have some pieces of advice? Or could you lead me to existent algorithms to do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thought of (and implemented) suboptimal processes such that fixing a rectangle size (10 x 10 for instance) and solving the problem in this simpler setting.&lt;/p&gt;&#xA;" OwnerUserId="27770" LastActivityDate="2018-05-17T13:06:09.673" Title="Space covering optimization" Tags="|optimization|algorithms|constrained-optimization|geometry|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29546" PostTypeId="2" ParentId="29529" CreationDate="2018-05-17T19:34:10.073" Score="3" Body="&lt;p&gt;This is a well-studied problem in geomorphological and hydrological modeling, as well as the fields of geomorphometry and terrain analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For visualization, &lt;a href=&quot;https://richdem.readthedocs.io/en/latest/flow_accumulation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;flow accumulation&lt;/a&gt; is usually calculated. If every element of your terrain is taken as contributing a value of 1 to the flow, then flow accumulation is the sum of all the flow that passes through a given element. A depiction of this for a landscape might look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1AfsS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1AfsS.png&quot; alt=&quot;Flow accumulation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several different &lt;a href=&quot;https://richdem.readthedocs.io/en/latest/flow_metrics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;flow routing methods&lt;/a&gt;. The one you describe is known as D8. A few additional ones are depicted below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WtV34.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WtV34.png&quot; alt=&quot;Different flow routing methods&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithms for performing the above, along with citations to appropriate references, are available in my library, &lt;a href=&quot;https://richdem.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RichDEM&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2018-05-17T19:34:10.073" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29548" PostTypeId="1" AcceptedAnswerId="29669" CreationDate="2018-05-17T22:43:38.367" Score="1" ViewCount="117" Body="&lt;p&gt;I am using the METIS to partition a matrix and then using domain decomposition to solve the subdomains in parallel using the Restricted Additive Schwarz method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently trying to solve some SPD matrices. Using METIS' &lt;code&gt;METIS_PartGraphKway(...)&lt;/code&gt;, I partition the matrix.  I think this function minimizes edge cuts. I am sorry, my knowledge on graph partitioning is limited. So my question is, after the partition, is the SPD-ness of each of the sub-domain matrix preserved ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simple cases of Laplacian 2d or other matrices such as bcsstk10 , bcsstk16 from the Matrix market which have more or less quite regular structure and non-zero entries near the diagonals, this property seems to hold, but for slightly more irregular matrices such as bcsstk38 (which is still SPD) it seems that the local solver does not converge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="16601" LastEditorUserId="16601" LastEditDate="2018-06-05T08:01:48.540" LastActivityDate="2018-09-03T09:02:35.777" Title="Partitioning SPD matrix with METIS to preserve block SPD-ness" Tags="|linear-solver|domain-decomposition|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29549" PostTypeId="1" AcceptedAnswerId="29567" CreationDate="2018-05-18T02:01:35.170" Score="0" ViewCount="1688" Body="&lt;p&gt;I'm trying to convert and process GrADS (grib) files into netcdf files. The files have values every 3 hours (2890 per year), 12 z levels, 120 variables, 241 columns and 236 rows. The data is almost 80 gb per year and I have to process a total of 50 years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before converting into netcdf I'm trying to calculate daily averages and sums for a subset of the variables. So far I've tried with GrADS but I still haven't been able to make the script to work. Going through the GrADS users forum to find some examples of a similar task I got the feeling that GrADS is not as commonly used as it used to be years ago so I'm curious about what people currently use when they have to deal with data as large as these (I come from a GIS background, so I'm not used to data as big).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First hand I'll say that I won't be able to use R or Python due to RAM limitation, the relational database option is also an issue due to not being able to find a tool to import from grib to SQL tables. What other options I have for such a &quot;simple&quot; task?&lt;/p&gt;&#xA;" OwnerUserId="27773" LastActivityDate="2018-06-12T10:01:54.310" Title="reading and processing large GRIB files" Tags="|python|simulation|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29550" PostTypeId="2" ParentId="29549" CreationDate="2018-05-18T05:55:32.243" Score="0" Body="&lt;p&gt;Looks like wgrib2 could turn the data into csv format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/csv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/csv.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once it’s csv, you have a variety of options. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any etl tool could pull in a csv to do the calculations. Given I work for Talend, I would use Talend Studio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The  Australian Ocean Data Network has a set of Talend routines for netcdf. I’ve not tried them, but that seems worth checking out. &lt;a href=&quot;https://github.com/aodn&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aodn&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27774" LastActivityDate="2018-05-18T05:55:32.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29551" PostTypeId="1" AcceptedAnswerId="29556" CreationDate="2018-05-18T08:44:47.283" Score="1" ViewCount="295" Body="&lt;p&gt;I am using an HPC cluster to launch parallel simulations. At the moment, there are 48 &lt;strong&gt;free&lt;/strong&gt; cores over 4 nodes but not each node do not have the same number of free cores. The distribution is something like this :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;node1: 8 cores free&lt;/p&gt;&#xA;&#xA;&lt;p&gt;node2: 8 cores free&lt;/p&gt;&#xA;&#xA;&lt;p&gt;node3: 8 cores free&lt;/p&gt;&#xA;&#xA;&lt;p&gt;node4: 24 cores free&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to utilize all of these resources to launch a simulation. I am using PBS as the job scheduler and normally I assign resources using &#xA;&lt;code&gt;qsub -l nodes=x:ppn=xxxx&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I assign resources in this case?&lt;/p&gt;&#xA;" OwnerUserId="27776" LastEditorUserId="20688" LastEditDate="2018-05-18T20:06:59.197" LastActivityDate="2018-05-18T20:21:37.127" Title="PBS: Using different computational resources on different nodes" Tags="|parallel-computing|hpc|pbs|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29552" PostTypeId="2" ParentId="29320" CreationDate="2018-05-18T11:13:01.587" Score="6" Body="&lt;p&gt;Very quick answer...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exponential of a Hamiltonian matrix is symplectic, a property that you probably wish to preserve, otherwise you would simply use a non-structure-preserving method. Indeed, there is no real speed advantage in using structured method, just structure preservation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible way to solve your problem is the following. First find a symplectic matrix such that $\hat{H}=M^{-1}HM=\begin{bmatrix} \hat{A} &amp;amp; -\hat{G}\\ 0 &amp;amp; -\hat{A}^T \end{bmatrix}$ is Hamiltonian and block upper triangular, and $\hat{A}$ has eigenvalues in the left half-plane. You get this matrix for instance by taking $\begin{bmatrix}I &amp;amp; 0\\ X &amp;amp; I\end{bmatrix}$, where $X$ solves the Riccati equation associated to $H$, or (more stable since it's orthogonal) by reordering the Schur decomposition of $H$ and applying the Laub trick (i.e., replacing the unitary Schur factor $\begin{bmatrix}U_{11} &amp;amp; U_{12} \\ U_{21} &amp;amp; U_{22}\end{bmatrix}$ with $\begin{bmatrix}U_{11} &amp;amp; -U_{12} \\ U_{12} &amp;amp; U_{11}\end{bmatrix}$). You may have trouble doing it if the Hamiltonian has eigenvalues on the imaginary axis, but that's a long story and for now I will suppose it doesn't happen in your problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have $M$, you have $\exp(H)=M\exp(\hat{H})M^{-1}$, and you can compute &#xA;$$&#xA;\exp(\hat{H}) = \begin{bmatrix} \exp(\hat{A}) &amp;amp; X\\ 0 &amp;amp; \exp(-\hat{A}^T) \end{bmatrix},&#xA;$$&#xA;where $X$ solves a certain Lyapunov equation, I believe something like $$&#xA;\hat{A} X + X \hat{A}^T = -\exp(\hat{A}) \hat{G} - \hat{G} \exp(-\hat{A}^T)&#xA;$$  (signs may be wrong; impose $\exp(\hat{H})\hat{H}=\hat{H}\exp(\hat{H})$ and expand blocks to get the correct equation. Look up &quot;Schur-Parlett method&quot; for a reference to this trick).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the three factors are exactly symplectic. Just use them separately: do not compute the product or you will lose this property numerically.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2018-05-18T11:13:01.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29553" PostTypeId="1" CreationDate="2018-05-18T14:27:34.567" Score="1" ViewCount="140" Body="&lt;p&gt;I have a system of coupled nonlinear PDEs that I cannot figure out how to solve in a smart way using FDM, so I was hoping someone here might have a clue.&#xA;The equations go as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;\frac{1}{n_\alpha}\frac{\partial}{\partial t} \left[n_\alpha-A_\alpha\left(\nabla^2\phi+\nabla^2 p_\alpha\right) \right] &amp;amp;=\text{RHS}_{n_\alpha} \\&#xA;\frac{\partial}{\partial t} \left[\ln(p_\alpha)- B_\alpha\left(\nabla^2\phi+\nabla^2 p_\alpha\right) \right] &amp;amp;=\text{RHS}_{p_\alpha}\\&#xA;\frac{\partial}{\partial t} \left[\sum_\alpha C_\alpha\left(\nabla^2\phi+\nabla^2 p_\alpha\right)\right] &amp;amp;= \text{RHS}_\text{vort}\\&#xA;\end{align*}&#xA;Where $A_\alpha, B_\alpha, C_\alpha$ are constants and $\alpha$ indicates the &quot;species&quot;. I.e. it could range from $1$ to $n$ and so there would in the end be $2n+1$ equations that need to solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose one would start with time stepping the equations, so one &quot;only&quot; needs to solve what seems to be coupled with nonlinear Helmholtz-type equations. But then what?&lt;/p&gt;&#xA;" OwnerUserId="21323" LastEditorUserId="20688" LastEditDate="2018-05-18T20:28:19.637" LastActivityDate="2018-05-18T20:28:19.637" Title="Finite difference methods for coupled 2nd order nonlinear pdes" Tags="|pde|finite-difference|computational-physics|nonlinear-equations|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29554" PostTypeId="1" CreationDate="2018-05-18T15:13:56.020" Score="1" ViewCount="66" Body="&lt;p&gt;&lt;strong&gt;INTRO&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within the last 5 days I was able to generate a block structured Cartesian grid &#xA;generation system with a combination of Fortran,C++ and Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am running intersection tests of the imported geometry with the bounding box &#xA;it is surrounded by and if the intersection test succeeds then I am subdividing the parent bounding box into 4 children boundary box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it can be understood for this 2D case, I am using Quad Tree decomposition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And now I am planning to code a FVM / FEM solver for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PROBLEM&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am unable to think of an algorithm that would help me identify cells which are SOLID (i.e inside the body) and FLUID ( i.e outside the body) at a fast speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can run a double ray intersection test by casting two opposite rays out of all the cells and see if both of the rays intersect with the boundary of the geometry or not to identify the solid domain. However such a test will be computationally very expensive to run after we have decomposed the quad tree &#xA;to a certain mesh level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The figure given below has 40,577 quad elements, so it is not very high. &#xA;But future meshes will have more elements. And hence this particular problem can become a performance bottleneck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FIG 1 : NACA 2412 Aerofoil meshed&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0d1KW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0d1KW.png&quot; alt=&quot;Cartesian grid for a NACA 2412 aerofoil&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FIG 2 : Zoomed view of trailing edge demonstrating mesh distribution&#xA;&lt;a href=&quot;https://i.stack.imgur.com/uUtMf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uUtMf.png&quot; alt=&quot;Zoomed view of trailing edge&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;QUESTIONS&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;i) Can I perform any kind of initial tests to reduce my computational load ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ii) Is there any other algorithm that would be more suited for this problem ?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="27783" LastEditorUserId="27783" LastEditDate="2018-05-19T06:52:54.873" LastActivityDate="2018-05-19T06:52:54.873" Title="Efficient initial identification of solid or liquid domains for a block structured Cartesian grid generation system" Tags="|algorithms|computational-geometry|mesh-generation|unstructured-mesh|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29556" PostTypeId="2" ParentId="29551" CreationDate="2018-05-18T20:21:37.127" Score="0" Body="&lt;p&gt;When you write/launch a PBS script, you are not supposed to know &lt;em&gt;what resources are available&lt;/em&gt;, you are supposed to describe what &lt;em&gt;resources do you need&lt;/em&gt;. And while the efficient use of HPC resources is important, you should not take the work of the job scheduler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I guess you are interested in the ability to request different &lt;code&gt;ppn&lt;/code&gt; for different nodes. This can be done using the following type of request (with &lt;code&gt;+&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#PBS -l nodes=1:ppn=3+1:ppn=8+24&#xA;#PBS -l naccesspolicy=SINGLEJOB -n&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That is supposed to request 3 nodes with 8 ppn and 1 node with 24 ppn. The question is if your scheduler is configured to process such requests is open. More about that &lt;a href=&quot;https://wiki.anl.gov/cnm/HPC/Submitting_and_Managing_Jobs/Advanced_node_selection#Different_PPN_by_node&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or in the official PBS manual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, since the resource availability is changing and dynamic with time, your PBS scripts have to be generated using some kind of a shell script that will check the available resources and generate the corresponding PBS file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a summary: I, personally, do not advise going this route and certainly check with HPC cluster administrators if such dynamic resource requests are even compliant with &quot;good citizen&quot; policy. And even if they are, I still think that you are taking a chunk of the scheduler work into your hands unnecessarily.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-18T20:21:37.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29557" PostTypeId="2" ParentId="21537" CreationDate="2018-05-18T23:51:07.527" Score="1" Body="&lt;p&gt;I think that this can be achieved using &lt;a href=&quot;https://www.mathworks.com/help/matlab/math/ode-event-location.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ODE Events&lt;/a&gt;. You can define your custom function(s)/expression(s) that will be evaluated and can terminate the solution process – thus serving as a custom convergence criterion/criteria.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-05-18T23:51:07.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29558" PostTypeId="1" CreationDate="2018-05-19T12:37:21.157" Score="4" ViewCount="179" Body="&lt;p&gt;I'm in the process of constructing an algorithm which computes the Voronoi diagram of a set of points, but I now need a method to decompose each Voronoi cell into simplices. The information we have is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The vertices of the polytope&lt;/li&gt;&#xA;&lt;li&gt;The faces of the polytope and their vertices&lt;/li&gt;&#xA;&lt;li&gt;The location of a central point&lt;/li&gt;&#xA;&lt;li&gt;The radial lines passing through the faces&lt;/li&gt;&#xA;&lt;li&gt;The fact that the polytope forms a convex hull&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The main issues are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The faces of the polytope can be non-simplicial&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We want the total algorithm to scale well with increasing dimension, but each Voronoi polytope has an average number of vertices which increases with dimension. Unfortunately, I can't find useful information on exactly how.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: While I have no formula for the increase in the number of polytope vertices with dimension, testing shows it to be $\mathcal O(5^d)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all faces were simplicial I'd draw simplices from the central point to each face. I can triangulate the non-simplicial faces and apply the above algorithm, but I still have the same question, just in a lower dimension, as each face has the above properties, except that we lack knowledge of &lt;em&gt;its&lt;/em&gt; faces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the Delaunay triangulation via the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bowyer%E2%80%93Watson_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bowyer-Watson&lt;/a&gt; algorithm at the moment, as this was the method I used for computing the Voronoi diagram. There are clearly much faster approaches, but before I spend too long implementing one of them, I felt it wise to check whether there's a well-known method which I'm missing.&lt;/p&gt;&#xA;" OwnerUserId="27789" LastEditorUserId="20688" LastEditDate="2018-05-21T19:15:15.893" LastActivityDate="2018-05-21T19:15:15.893" Title="Fast Algorithms for the Simplicial Decomposition of a Convex Polytope in N-Dimensions" Tags="|algorithms|computational-geometry|delaunay-triangulation|convex-hull|voronoi-diagrams|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29560" PostTypeId="2" ParentId="28645" CreationDate="2018-05-20T04:20:11.857" Score="0" Body="&lt;p&gt;A numerical scheme can be unconditionally unstable, conditionally stable and unconditionally stable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would want the later two. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend that you learn to do &lt;strong&gt;Von-Neumann  stability analysis&lt;/strong&gt; for numerical schemes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You learn it once. And you can find yourself the stability status of any arbitrary governing differential equation for any arbitrary numerical scheme. &lt;/p&gt;&#xA;" OwnerUserId="27783" LastActivityDate="2018-05-20T04:20:11.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29563" PostTypeId="2" ParentId="10362" CreationDate="2018-05-20T22:22:19.550" Score="3" Body="&lt;p&gt;Suppose $$z_{i} = x_{i} \times y_{i}$$&#xA;The constraint $$ \sum_{i=1}^{n} x_{i}y_{i}\le 1 $$ can be reformulated as linear constraints:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) $$ \sum_{i=1}^{n} z_{i}\le 1 $$ &#xA;2) $$ z_{i}\le x_{i} $$ &#xA;3) $$ z_{i}\le y_{i} $$ &#xA;4) $$ z_{i}\ge x_{i} + y_{i} -1 $$ &lt;/p&gt;&#xA;" OwnerUserId="27797" LastActivityDate="2018-05-20T22:22:19.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29564" PostTypeId="1" AcceptedAnswerId="29565" CreationDate="2018-05-20T23:45:52.200" Score="1" ViewCount="157" Body="&lt;p&gt;I am having trouble with a C program that uses the function &lt;code&gt;dsysv&lt;/code&gt; from LAPACK. Everything compiles and works without any errors, my only problem is, that the routine doesn't chance any of the arrays it is supposed to.&#xA;The code (only hardcode for test purposes):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;using namespace std;&#xA;&#xA;extern &quot;C&quot; {&#xA;extern int dsysv_( char*, int* , int* , double* ,int*,int*, double*,int*,             double* ,int* ,int *); &#xA;}  &#xA;&#xA;int main() {&#xA;&#xA;int n=3;&#xA;int nrhs=1;&#xA;int lda= 3;&#xA;int ldb=3;&#xA;int ipiv[3]={0,0,0}; &#xA;    double a[3*3] = {&#xA;    1,2,3,&#xA;    0,4,5,&#xA;    0,0,6&#xA;    };&#xA;double b[3]={1,2,3};&#xA;int info=42;&#xA;int lwork=-1;&#xA;double work[1]={0};&#xA;char uplo='U';&#xA;&#xA;dsysv_(&amp;amp;uplo,  &amp;amp;n, &amp;amp;nrhs, a, &amp;amp;lda,  ipiv, b,  &amp;amp;ldb, work, &amp;amp;lwork, &amp;amp;info);&#xA;&#xA;cout&amp;lt;&amp;lt;&quot;info: &quot;&amp;lt;&amp;lt;info&amp;lt;&amp;lt;endl;&#xA;for(int i=0; i&amp;lt;3; i++){&#xA;    cout&amp;lt;&amp;lt;b[i]&amp;lt;&amp;lt;&quot; &quot;;&#xA;}&#xA;cout &amp;lt;&amp;lt;endl&amp;lt;&amp;lt;&quot;and A: &quot;&amp;lt;&amp;lt;endl;&#xA;for(int i=0; i&amp;lt;9; i++){&#xA;    cout&amp;lt;&amp;lt;a[i]&amp;lt;&amp;lt;&quot; &quot;;&#xA;}&#xA;&#xA;return 1;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;info: 0&#xA;&#xA;1 2 3 &#xA;&#xA;and A: &#xA;&#xA;1 2 3 0 4 5 0 0 6 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Although the function is supposed to overwrite &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;.&#xA;I think I miss something really obvious here, which I am not aware of since I just started working with LAPACK and would appreciate any help with it.&lt;/p&gt;&#xA;" OwnerUserId="27798" LastEditorUserId="20688" LastEditDate="2018-05-21T01:31:19.520" LastActivityDate="2018-05-21T01:31:19.520" Title="Problem with 'dsysv' from LAPACK" Tags="|c|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29565" PostTypeId="2" ParentId="29564" CreationDate="2018-05-21T01:17:52.993" Score="2" Body="&lt;ul&gt;&#xA;&lt;li&gt;Set &lt;code&gt;lwork&lt;/code&gt; to &lt;code&gt;1&lt;/code&gt;.&#xA;(By using &lt;code&gt;-1&lt;/code&gt;, LAPACK does not do any computation. It returns the optimal size for &lt;code&gt;work&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Set &lt;code&gt;uplo&lt;/code&gt; to &lt;code&gt;'L'&lt;/code&gt;&#xA;(Your definition of &lt;code&gt;a&lt;/code&gt; actually fills the lower part. &#xA;You have in C&#xA;&lt;code&gt;double a[3*3] = {a_{11}, a_{21}, a_{31}, a_{12}, a_{22}, a_{32}, a_{13}, a_{23}, a_{33}};&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7440" LastEditorUserId="20688" LastEditDate="2018-05-21T01:30:26.957" LastActivityDate="2018-05-21T01:30:26.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29566" PostTypeId="2" ParentId="29251" CreationDate="2018-05-21T02:51:04.517" Score="1" Body="&lt;p&gt;Here are two options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Prove the inf-sup condition.&lt;/li&gt;&#xA;&lt;li&gt;Use the Fredholm alternative and the result that says &quot;only the trivial function can satisfy a second-order PDE with zero right hand side, zero Dirichlet condition, and zero normal derivative on the same part of boundary&quot;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The PhD thesis of Prof. Melenk contains a lot of information regarding the Helmholtz equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.asc.tuwien.ac.at/~melenk/publications/diss.ps.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.asc.tuwien.ac.at/~melenk/publications/diss.ps.gz&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7440" LastEditorUserId="20100" LastEditDate="2018-11-19T04:24:32.860" LastActivityDate="2018-11-19T04:24:32.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29567" PostTypeId="2" ParentId="29549" CreationDate="2018-05-21T09:23:18.470" Score="1" Body="&lt;p&gt;I would suggest &lt;a href=&quot;https://code.mpimet.mpg.de/projects/cdo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cdo&lt;/a&gt;. CDO can read GRIB files (though I think it only supports GRIB2 files) and has a host of options for computing means, subsetting, regridding etc. If you add the &lt;code&gt;-f nc&lt;/code&gt; option as well it will output to NetCDF automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ECMWF's &lt;a href=&quot;https://software.ecmwf.int/wiki/display/ECC/ecCodes+Home&quot; rel=&quot;nofollow noreferrer&quot;&gt;ecCodes library&lt;/a&gt; will probably also come in handy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I typically use a combination of CDO and ecCodes. As an example, if I want to process a GRIB file containing global meteorological fields from a climate model containing only one timestep, I would use the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# This ecCodes command splits the GRIB file &quot;model_output+000000&quot; into different&#xA;# vertical levels. In this case, the file only contains one kind of vertical level&#xA;# - pressure (hPa), but it could conceivably contain output on sigma levels, hybrid&#xA;# sigma coordinates, altitude etc. The &quot;+000000&quot; refers to the timestep contained&#xA;# in this file (the zeroth timestep).&#xA;grib_copy model_output+000000 model_output+000000_[typeOfLevel]&#xA;&#xA;# The &quot;*_isobaricInhPa&quot; file was created by the previous command.&#xA;cdo -f nc setgridtype,regular model_output+000000_isobaricInhPa model_output+000000.nc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27801" LastEditorUserId="27801" LastEditDate="2018-06-12T10:01:54.310" LastActivityDate="2018-06-12T10:01:54.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29568" PostTypeId="1" AcceptedAnswerId="29569" CreationDate="2018-05-21T09:38:51.113" Score="8" ViewCount="1071" Body="&lt;p&gt;I often see the terms &quot;low arithmetic intensity&quot; and &quot;memory-bound&quot; associated with sparse matrix operations. However, my intuition is that a sparse matrix operation should be &lt;em&gt;less&lt;/em&gt; memory-bound, if anything. After all, for a highly sparse matrix, you only have to transfer a fraction of the data required for a dense matrix operation (i.e. only the non-zero, actually meaningful entries of the matrix have to be stored and transferred).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: why are sparse matrix operations usually considered memory-bound?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose there must be some level of &quot;denseness&quot; at which a sparse matrix computation becomes compute-bound.&lt;/p&gt;&#xA;" OwnerUserId="27801" LastActivityDate="2018-05-21T21:38:54.120" Title="Why does sparse linear algebra have a low arithmetic intensity?" Tags="|linear-algebra|sparse-matrix|hpc|dense-matrix|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29569" PostTypeId="2" ParentId="29568" CreationDate="2018-05-21T12:06:56.680" Score="13" Body="&lt;p&gt;BLAS1-operations, BLAS2-operations, and sparse-operations share the same curse of low arithmetic intensity, that they perform $O(1)$ flops for each memory read (contrast this to a BLAS3-operation like &lt;code&gt;gemm&lt;/code&gt;, which performs $O(N^3)$ flops over $O(N^2)$ reads and only becomes more and more arithmetic-intensive/compute-bound for large $N$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, sparse matvecs have even further complications, because of indirect addressing. This adds (at the very least) additional memory pressure to load the integer indices. Even worse, it results in much poorer cache utilization, since a cache typically fetches a large run of data nearby any single read, but a sparse matvec will generally only use just the one value. (BLAS1/BLAS2-operations are low arithmetic intensity, but at least cache-friendly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are sparse matrix formats that attempt to mitigate some of this cache-unfriendliness, typically they are like blocked variants of the traditional formats (CSR/CSC/etc), where each &quot;nonzero&quot; of the matrix is instead a small dense matrix. This lets you cut out some of the indirect addressing and improve the locality of access. They are a good fit for high-order FEM's, where many dof's are often collocated on the same geometrical entity and share the same pattern. But they are not a good fit for low-order methods (there's just not enough local similarity in such a matrix to effectively &quot;build-up&quot; a large block).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this said, the upshot to &quot;low arithmetic intensity&quot; is &quot;low arithmetic&quot; ..  sparsity-exploiting methods are generally fast even though they're not particularly efficient at the hardware level. They're just innately doing &quot;less work&quot;... not necessarily a bad thing from a user perspective.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="20688" LastEditDate="2018-05-21T16:13:04.677" LastActivityDate="2018-05-21T16:13:04.677" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29570" PostTypeId="2" ParentId="27126" CreationDate="2018-05-21T12:55:27.780" Score="1" Body="&lt;p&gt;You need to use the &quot;normalized&quot; Legendre Polynomials instead of Legendre Polynomials. Where you divide the Legendre polynomial by $\sqrt{\frac{2}{2n+1}}$. &lt;/p&gt;&#xA;" OwnerUserId="27802" LastEditorUserId="27802" LastEditDate="2018-06-05T15:36:11.203" LastActivityDate="2018-06-05T15:36:11.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29571" PostTypeId="2" ParentId="27126" CreationDate="2018-05-21T14:42:32.227" Score="1" Body="&lt;p&gt;You're using the orthonormalized version of the Legendre polynomials, while Hesthaven is not. The polynomials in your matrix are normalized by a factor of $\sqrt{\frac{2n+1}{2}}$, i.e. $\sqrt{\frac{2(1)+1}{2}}=1.2247$ and $\sqrt{\frac{2(2)+1}{2}}=1.5811$.&lt;/p&gt;&#xA;" OwnerUserId="27171" LastActivityDate="2018-05-21T14:42:32.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29575" PostTypeId="1" CreationDate="2018-05-21T18:58:38.887" Score="1" ViewCount="345" Body="&lt;p&gt;I have a system of conservation laws of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial \mathbf{q}}{\partial t} + \nabla \cdot \mathbf{F}\!\left(\mathbf{q}\right) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use finite elements to solve this system. As an initial choice, I am using Legendre polynomials $\phi$ for my function basis. Let $j$ be a polynomial index. If I substitute the decomposition of my unknown function $\mathbf{q}$ and the flux function $\mathbf{F}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{q}\!\left(t, x\right) = \sum\limits_j \hat{\mathbf{q}}_j\!\left(t\right) \phi_j\!\left(x\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf{F}\!\left(\mathbf{q}\right) = \sum\limits_j \hat{\mathbf{F}}\!\left(t\right) \phi_j\!\left(x\right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and finally write out the weak form (given some test volume $\Omega$ with boundary $\partial \Omega$)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum\limits_j \left(\frac{\partial \hat{\mathbf{q}}_j}{\partial t} \int\limits_\Omega \phi_i \phi_j + \hat{\mathbf{h}}_j \oint\limits_{\partial \Omega} \phi_i \phi_j - \hat{\mathbf{F}}_j \cdot \int\limits_\Omega \phi_j \nabla \phi_i\right) = 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, I replaced $\mathbf{F} \cdot \hat{\mathbf{n}} = \mathbf{h}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem occurs when I actually try to evaluate the surface second and third terms inside the parentheses. Because my problem is 1-dimensional, the second term amounts to just evaluating $\phi_i \phi_j$ at 1 and -1 and taking the difference ($\hat{\mathbf{n}} = \pm \hat{\mathbf{x}}$ is the outward-facing normal). In analogy with the stiffness matrix, we could write this as a matrix $\mathbb{B} = \{b_{ij}\}$. Any given element $b_{ij}$ will be either 0 or 2 because the Legendre polynomials take the value of 1 or -1 at the edges of the domain. This matrix is sparse but singular. The third term exhibits a similar problem if we try to write it as a matrix $\mathbb{C} = \{c_{ij}\}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way I know to solve for the $\hat{\mathbf{F}}_j$ and $\hat{\mathbf{h}}_j$ coefficients is by solving a matrix equation, but all my attempts to solve them (using PETSc) converge slowly and may not even be correct. I would like to use a preconditioner, but I get errors any time I try (&lt;code&gt;PCSETUP_FAILED due to SUBPC_ERROR&lt;/code&gt;), even though I have specified the null spaces of these matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have I approached this problem incorrectly altogether? Or is there a common way of overcoming this challenge, in PETSc or otherwise?&lt;/p&gt;&#xA;" OwnerUserId="25718" LastActivityDate="2018-05-21T18:58:38.887" Title="Getting started with FEM: Ill-conditioned matrix when evaluating flux terms in conservation law?" Tags="|finite-element|petsc|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29576" PostTypeId="1" AcceptedAnswerId="37653" CreationDate="2018-05-21T20:38:26.280" Score="2" ViewCount="139" Body="&lt;h1&gt;Problem description&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Given data at many time instance $t$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min _{\alpha, \Lambda, \beta} \lVert y(t) - \alpha e^{\Lambda t} \beta \rVert_F$$&#xA;with $$ \lVert \alpha \rVert_2^F = 1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $y(t) \in \mathbb{R}^{n \times M}$, $\Lambda \in \mathbb{R}^{r\times r}$ is a diagonal matrix, $\alpha \in \mathbb{R}^{n \times r}$, $\beta \in \mathbb{R}^{r \times M}$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Special case&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;if $\alpha = I$, the problem becomes a classical problem in &lt;strong&gt;variable projection for nonlinear least square&lt;/strong&gt;, which has been studied for decades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The benefit of $\alpha = I$ is that, the nonlinear and linear part becomes separable through pseudo-inverse.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What I currently do&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I can simply (blindly) use an optimization package to &lt;strong&gt;generally solve&lt;/strong&gt; this problem by do naive gradient descent, it looks like a neural network training process, without leveraging the structure of the problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;It works for toy problem, like when n,r less than 3.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside is that, since it doesn't assume any structure to leverage, the computational time is costly.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What I want to know&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I want to know if any one know which category does this problem belongs to, and if there is better algorithm existing to solve this.&lt;/p&gt;&#xA;" OwnerUserId="16859" LastEditorUserId="16859" LastEditDate="2018-06-03T17:57:32.783" LastActivityDate="2021-06-24T15:28:48.887" Title="Nonlinear least square optimization" Tags="|optimization|convex-optimization|nonlinear-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29577" PostTypeId="2" ParentId="29568" CreationDate="2018-05-21T21:38:54.120" Score="5" Body="&lt;p&gt;This really depends on the operations you are including in your question. If you took the sparse equivalent of any level 1 BLAS or level 2 BLAS algorithm, then yes they are memory bound (not compute bound), but that is also true of level 1 BLAS and level 2 BLAS for dense matrices. To answer this we need to understand what makes an algorithm compute bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compute bound algorithms are those for which their performance relies largely on the capability of the underlying system to perform arithmetic, often measured in FLOPS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an algorithm to have this property it must do significantly more arithmetic operations than memory operations, because memory operations have very high latency. In the context of linear algebra algorithms this means there must be significant data re-use in the course of the algorithm. Data re-use means we can avoid resorting to main memory. Level 3 BLAS operations often have this property.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example to do a sparse matrix-vector product you scan through its list of nonzeros only once. There is no way to cache any values in any way that could help performance. The same is true of &lt;em&gt;dense&lt;/em&gt; matrix-vector product for that matter. These are not examples of compute bound algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately most compute architectures today are very good at arithmetic and only decent at memory performance. Thus A lot of sparse linear algebra algorithms are specifically designed to reduce to dense linear algebra algorithms from level 3 BLAS at its most demanding points. Multifrontal factorization is an example of an algorithm like this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the overall answer to you is that: Sparse linear algebra is memory bound for many operations, but so is dense linear algebra. We can choose/design our algorithms to match the capabilities of the system at hand however, so this is not necessarily a bad thing.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2018-05-21T21:38:54.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29579" PostTypeId="1" CreationDate="2018-05-22T03:33:32.393" Score="0" ViewCount="453" Body="&lt;p&gt;I am looking into the four-point bend test, such as one in this &lt;a href=&quot;https://www.youtube.com/watch?v=Shz2wogrqIY&quot; rel=&quot;nofollow noreferrer&quot;&gt;YouTube video&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample screenshot from the video illustrating the problem:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xC0po.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xC0po.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a little confused as to how the loads are prescribed numerically as boundary conditions. My intuition tells me that the object has to have a Dirichlet BC somewhere. But it appears that the 4 supports (2 on top and 2 on bottom) exert the vertical loads, hence they should be all Neumann BCs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I thinking about this incorrectly?&lt;/p&gt;&#xA;" OwnerUserId="27504" LastEditorUserId="20688" LastEditDate="2018-05-22T10:58:46.157" LastActivityDate="2020-04-10T18:41:37.080" Title="Boundary conditions in a four point bend test" Tags="|boundary-conditions|solid-mechanics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29580" PostTypeId="2" ParentId="27700" CreationDate="2018-05-22T07:24:37.560" Score="2" Body="&lt;p&gt;I will try to answer the part of the question regarding the accuracy of the calculation, which certainly affects all the other things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integrals of the type ($T_j$ denoting the $j$th triangle in the discretization and $\vec{x}$ some arbitrary observation point)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int\limits_{T_j}\frac{1}{||\vec{x}-\vec{y}||}dy,\quad \vec{y}\in T_j&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;have a singularity at $\vec{x}=\vec{y}$ which implies that a special care should be taken when $\vec{x}$ is close to $\vec{y}$. As I understood, your special treatment is limited to choosing even/odd Gauss quadrature rules, so that $\vec{x}$ and $\vec{y}$ never collide exactly. While such ad-hoc method will provide some result, it is avoiding the problem of handling the singularity rather than solving it. Moreover, increasing accuracy using this approach is nearly impossible; thus, another technique should be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are special quadrature rules designed to handle such singularities:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1137/0733048&quot; rel=&quot;nofollow noreferrer&quot;&gt;J. Ma, V. Rokhlin, and S. Wandzura, &quot;Generalized Gaussian quadrature rules for systems of arbitrary functions,&quot; &lt;em&gt;SIAM J. Numer. Anal.&lt;/em&gt;, vol. 33, no. 3, pp. 971–996.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But, I would propose handling such singularity analytically.&#xA;The following paper:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1109/TAP.1984.1143304&quot; rel=&quot;nofollow noreferrer&quot;&gt;D. R. Wilton, S. M. Rao, A. W. Glisson, D. H. Schaubert, O. M. Al-Bundak, and C. M. Butler, &quot;Potential integrals for uniform and linear source distributions of polygonal and polyhedral domains,&quot; &lt;em&gt;IEEE Trans. Antennas Propag.&lt;/em&gt;, vol. AP–32, no. 3, pp. 276–281, Mar. 1984.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;describes how to take those integrals analytically for planar polygons (certainly works for triangles). The theory is not too complicated and the implementation should be faster than using quadrature rules with a large number of quadrature points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a must when $\vec{x}$ and $\vec{y}$ can lie in the same triangle, but is recommended for close enough interactions. You will choose your &lt;em&gt;singularity treatment&lt;/em&gt; region yourself based on accuracy consideration. A good rule of thumb is to choose &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d_\text{sing. treatment}=4\max_\limits{j}{\max_\limits{s=1,2,3}e_{j,s}},$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $e_{j,s}$ is the length of the $s$th edge of the $j$th triangle. Then, if $||\vec{x}-\vec{y}||&amp;lt;d_\text{sing. treatment}$, you apply the singularity treatment of your choice (analytic or &lt;em&gt;special&lt;/em&gt; quadrature rule); otherwise, proceed with regular cheap Gaussian quadrature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also discussed a bit this paper and the technique in &lt;a href=&quot;https://scicomp.stackexchange.com/q/29111/20688&quot;&gt;this question&lt;/a&gt; for the case of a non-static integral (with $e^{jkR}$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a proper singularity treatment technique should increase the accuracy of your results dramatically which might eliminate a lot of problems with local errors and field lines near the interfaces.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-06-19T13:46:15.957" LastActivityDate="2018-06-19T13:46:15.957" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29581" PostTypeId="2" ParentId="29490" CreationDate="2018-05-22T13:49:31.100" Score="0" Body="&lt;p&gt;Matrix $\mathbf B$ having columns summing up to $(-1,0,0,...,0,0,+1)$ does not imply that the same happens for $\mathbf A^{-1}\mathbf B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, &lt;strong&gt;if&lt;/strong&gt; $\mathbf A^{-1}\mathbf B$ has that property, than it is correct to use $\mathbf 1^{\text{T}}$ to discretize the integral over the domain. If the scheme used is such that $\mathbf B$ has that property, than the row vector $\textbf 1^{\text{T}} \textbf A$ should be used to discretize the integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This appears clearer in the light of R. Knikker's 2009 paper on the Journal of Computational Physics.&lt;/p&gt;&#xA;" OwnerUserId="18846" LastActivityDate="2018-05-22T13:49:31.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29582" PostTypeId="1" CreationDate="2018-05-22T23:25:55.130" Score="2" ViewCount="134" Body="&lt;p&gt;Take the system $$A^TCAx=b$$&#xA;where $$A\in\mathbb{R}^{n\times m},\;C\in\mathbb{R}^{n\times n},\;x,b\in\mathbb{R}^{m}, \;m\leq n$$ and $$A^TA=I$$ and $$Cy=d$$ can be solved efficiently in general (specifically $C$ is SPD and it/it's inverse already has Cholesky decomposition). Clearly, if $m\ll n$, then it probably best to just generate and factor the new matrix, but if $m$ and $n$ are both large and $A,C$ are sparse/structured, generating this (typically dense) matrix may not be possible. While I could just use iterative techniques (CG, etc.) at this point, I was curious if there is any to use the knowledge of the inverse of $C$ to develop a more direct (and potentially more efficient) method? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more context, $A,C$ are Kronecker products in my particular problem. &lt;/p&gt;&#xA;" OwnerUserId="23072" LastEditorUserId="1047" LastEditDate="2018-05-23T12:27:21.867" LastActivityDate="2018-05-23T12:27:21.867" Title="Efficiently solve linear system with matrix quadratic form" Tags="|linear-algebra|linear-solver|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29583" PostTypeId="1" AcceptedAnswerId="29662" CreationDate="2018-05-23T07:17:04.497" Score="0" ViewCount="1120" Body="&lt;p&gt;I am trying to model 2D thermal expansion of a square area inside another square using &lt;a href=&quot;https://www.featool.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEATool&lt;/a&gt;. I have simulated plane strain by incorporating forces pointing along the $[1 \,\,\, -1]^T$ direction inside the smaller square . However, what I really want to simulate is this &quot;shrinking&quot; of the material as a thermal expansion process. What kind of PDE should I use to model the subtraction of heat from that small square?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LLfSr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LLfSr.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27807" LastEditorUserId="9667" LastEditDate="2018-06-03T16:22:22.597" LastActivityDate="2024-02-21T20:53:11.450" Title="How to simulate thermal expansion in a 2D plane using FEA?" Tags="|finite-element|pde|matlab|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29585" PostTypeId="1" AcceptedAnswerId="29591" CreationDate="2018-05-23T19:56:36.677" Score="2" ViewCount="95" Body="&lt;p&gt;If we have a continuous time state-equation, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \dot{x}(t) = A x(t) + B u(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A \in \mathbb{R}^{n \times n}, x \in \mathbb{R}^{n\times1}, B \in \mathbb{R}^{n \times m}, u \in \mathbb{R}^{m \times 1}$, the analytical solution is obtained as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x(t) = e^{A t} x(0) + \int_0^t e^{A(t-\tau)}B u(\tau)\, d\tau$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Converting this to discrete-time, with sampling period $T_s$, and assuming that all eigenvalues of $A$ are non-zero, real &amp;amp; distinct and $u$ remains constant within each sample interval, $k$ to $k+1$, we get the following &lt;strong&gt;difference&lt;/strong&gt; equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x[k+1] = e^{A T_s} x[k] + \left[\int_0^{T_s} e^{A\tau}B \, d\tau\right] u[k]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$x[k+1] = A_d  x[k] + B_d u[k]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the original matrix $A$ is invertible, the integral term corresponding to $B_d$ can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$B_d = A^{-1}(A_d - I)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is, &lt;strong&gt;What about the case when A is non-invertible&lt;/strong&gt;? In particular, if &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$A=\begin{bmatrix}a_1 &amp;amp; 0 &amp;amp; 0 \\ 0 &amp;amp; a_2 &amp;amp; 0 \\ 0 &amp;amp; 0 &amp;amp; 0  \end{bmatrix}$$ and $$B=\begin{bmatrix}b_1 \\ b_2 \\ b_3 \end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then one of the eigenvalues of $A$ is zero. In this case, how do I obtain the corresponding $B_d$?&lt;/p&gt;&#xA;" OwnerUserId="16900" LastEditorUserId="20417" LastEditDate="2018-05-24T17:33:15.073" LastActivityDate="2018-05-24T17:39:32.043" Title="Discrete-time input matrix when one of the eigenvalues of the system matrix is zero" Tags="|linear-algebra|matrix|differential-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29586" PostTypeId="2" ParentId="29585" CreationDate="2018-05-24T09:25:31.660" Score="1" Body="&lt;p&gt;That equivalence is only valid if $A$ is invertible, hence it is a result but not a definition. This is typically done via the following relation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\exp\left(\begin{bmatrix}A &amp;amp; B\\0 &amp;amp;0\end{bmatrix}t\right) = \begin{bmatrix}e^{At} &amp;amp;\int e^{At}B\\0&amp;amp;I\end{bmatrix}=\begin{bmatrix}A_d &amp;amp; B_d\\0 &amp;amp;I\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;exp&lt;/code&gt; is the matrix exponential, and $t$ is the sampling period. $C, D$ matrices remain unchanged. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shameless plug: I've recently finished these conversion tools in my python package. The code for other methods can be found &lt;a href=&quot;https://github.com/ilayn/harold/blob/master/harold/_discrete_funcs.py#L139&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ilayn/harold/blob/master/harold/_discrete_funcs.py#L139&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21617" LastEditorUserId="21617" LastEditDate="2018-05-24T09:33:19.013" LastActivityDate="2018-05-24T09:33:19.013" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29587" PostTypeId="1" CreationDate="2018-05-24T09:30:50.557" Score="3" ViewCount="882" Body="&lt;p&gt;Currently I am facing the problem to solve numerically the following equation for a double well harmonic potential:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$iℏ\frac{\partial}{\partial t}ψ(x,t)=−\frac{ℏ}{2m}\frac{\partial ^2}{∂x^2}ψ(x,t)+V(x,t)ψ(x,t)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V(x,t)=\frac{mω^2_0}{2}[a(t)x^4−b(t)x^2]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where both $a(t)$ and $b(t)$ are of the form $\frac{1}{2*\sqrt{const_1}}(const_1-const_2 \cos(\omega t))$. I started to use the split-step Fourier method algorithm and here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;tfinal=10000#total simulation time&#xA;dt=0.001#time step&#xA;M=np.round(tfinal/dt)&#xA;time_steps=M.astype(int)# number of time steps&#xA;&#xA;b=10.# final x&#xA;a=-10.# initial x&#xA;N=512# num of pieces&#xA;&#xA;dx=(b-a)/N` # x-space step size&#xA;dk=2*np.pi/(b-a)&#xA;n=np.linspace(-N/2,N/2 -1,N) # this is gonna be my n's in x_n=a+dx*n&#xA;&#xA;x=n*dx&#xA;k=n*dk&#xA;t=0&#xA;&#xA;alpha=0.0005&#xA;beta=0.0001&#xA;acoef=(alpha-beta*np.cos(t))/(2*alpha**0.5)&#xA;bcoef=(alpha-beta*np.cos(t))**0.5 / (2*alpha**0.5)&#xA;V=0.5*(acoef*x**4 -bcoef*x**2)&#xA;&#xA;u0=np.exp(-(x+4.72)**2 /2)*(1/np.pi)**0.25&#xA;u=u0&#xA;&#xA;for i in xrange(1,time_steps):&#xA;    u=np.exp(-1j*dt*V/2)*u&#xA;    c=np.fft.fftshift(np.fft.fft(u))&#xA;    c=np.exp(-1j*dt*k**2 /2)*c&#xA;    u=np.fft.ifft(np.fft.fftshift(c))&#xA;    u=np.exp(-1j*dt*V/2)*u&#xA;    t=t+dt&#xA;&#xA;plt.plot(x,u*np.conjugate(u),'r--')&#xA;plt.xlim(-10,10)&#xA;plt.ylim(0,0.6)&#xA;plt.show&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However the problem that arises it that it takes a very long time to execute in python. I am not sure what am I doing wrong since I am new in python or if I am implementing correctly the algorithm. I would appreciate any insight or suggestion.  &lt;/p&gt;&#xA;" OwnerUserId="27826" LastEditorUserId="22025" LastEditDate="2018-05-24T13:17:24.800" LastActivityDate="2018-05-24T13:17:24.800" Title="Numerical solution to Time-dependent Schrodinger equation with time dependent hamiltonian" Tags="|quantum-mechanics|numerics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29589" PostTypeId="1" CreationDate="2018-05-24T15:58:05.477" Score="1" ViewCount="148" Body="&lt;p&gt;Is there any way to multiply a symmetric matrix by a dense one in armadillo(and use the fact that we have a symmetric matrix)? I know about &lt;code&gt;DSYMM&lt;/code&gt; Routine in BLAS,but the matrices I'm dealing with are &lt;code&gt;mat&lt;/code&gt; type(they are not arrays),So I wonder if there is some thing in armadillo using &lt;code&gt;DSYMM&lt;/code&gt; Routine from BLAS .&#xA;I couldn't find any thing in armadillo documentation.&lt;/p&gt;&#xA;" OwnerUserId="27656" LastEditorUserId="27656" LastEditDate="2018-05-24T17:50:56.573" LastActivityDate="2018-05-24T17:50:56.573" Title="$ A * B $ computation when B is a symmetric matrix in armadillo" Tags="|c++|blas|intel-mkl|armadillo|" AnswerCount="0" CommentCount="2" ClosedDate="2018-11-19T15:19:42.640" ContentLicense="CC BY-SA 4.0" />
  <row Id="29590" PostTypeId="1" CreationDate="2018-05-24T16:46:23.370" Score="2" ViewCount="35" Body="&lt;p&gt;The perspective of a function $f : \mathbb{R}^n \to \mathbb{R}$ is the function $g: \mathbb{R}^n \times \mathbb{R} \to \mathbb{R}$ where $g$ is defined as $$g(x,t) = tf(x/t)$$ with $$\mathbf{\text{dom}}\, g = \{(x, t) : x/t \in \mathbf{\text{dom}}\, f, \,t &amp;gt; 0\}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I'm modeling a convex constraint of the form $g(x,t) \leq 0$ in a convex solver software, how do I ensure the domain constraint $t&amp;gt;0$ is satisfied? I tried $t \geq 0$, but for some problems, my $t^* = 0$.&lt;/p&gt;&#xA;" OwnerUserId="22292" LastEditorUserId="9667" LastEditDate="2018-06-03T16:20:03.283" LastActivityDate="2018-06-03T16:20:03.283" Title="Domain for convex perspective function" Tags="|convex-optimization|constraints|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29591" PostTypeId="2" ParentId="29585" CreationDate="2018-05-24T17:33:12.553" Score="1" Body="&lt;p&gt;$$\begin{array}{rl} \mathrm B_d &amp;amp;= \displaystyle\int_{0}^{T_s} \exp(\mathrm A t) \mathrm B \, \mathrm d t\\ &amp;amp;= \displaystyle\int_{0}^{T_s} \left(\sum_{k=0}^{\infty} \frac{t^k}{k!} \mathrm A^k \right)\mathrm B \, \mathrm d t\\ &amp;amp;= \displaystyle \sum_{k=0}^{\infty} \left( \int_{0}^{T_s} \frac{t^k}{k!} \, \mathrm d t \right) \mathrm A^k \mathrm B\\ &amp;amp;= \displaystyle \sum_{k=0}^{\infty} \frac{T_s^{k+1}}{(k+1)!} \mathrm A^k \mathrm B\\ &amp;amp;\approx T_s \left( \mathrm I + \frac{T_s}{2} \mathrm A \right) \mathrm B\end{array}$$&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2018-05-24T17:39:32.043" LastActivityDate="2018-05-24T17:39:32.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29592" PostTypeId="1" CreationDate="2018-05-24T17:33:21.483" Score="1" ViewCount="197" Body="&lt;p&gt;I need to solve the following problem:&#xA;$$ \begin{cases}&#xA;\dot{\vec{x(t)}} = A\vec{x(t)} + u(t)D\vec{x(t)} + u(t)\vec{b}, &amp;amp; x \in (0, T), \\&#xA;\vec{x(0)} = \vec{0},&#xA;\end{cases}$$&#xA;where $u(t)$ is known function, $A$ and $D$ are known constant matrices and $b$ is known constant vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solve this problem with explicit Euler and accuracy of the grid of $10^5$ nodes is quite enough for me. However, I need to solve such systems a lot of times, so my computations takes about a week. I am looking for a way of parallelizing numerical integration of this ODE. Can you advice some papers concerning this topic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found the approach called &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.112.6768&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parareal&lt;/a&gt;, but I am not sure that it is the most efficient way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Great thanks for any help, advice or papers!&lt;/p&gt;&#xA;" OwnerUserId="27832" LastActivityDate="2020-07-19T17:57:32.790" Title="Parallel integration of dynamical systems" Tags="|ode|parallel-computing|numerics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29594" PostTypeId="2" ParentId="24363" CreationDate="2018-05-24T19:56:53.143" Score="0" Body="&lt;p&gt;I completely agree with the answers posted so far. I would like to point out that the matrix &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ A = (I \otimes A_1 + A_2 \otimes I)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;never needs to be formed explicitly. As other answers correctly point out that one only needs to form matrix-vector products (matvecs) with A, or its inverse. Matvecs of the form $Ax$ can be done using the identity &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (B^T \otimes C)x = \text{vec}(CXB), $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\text{vec}(X) = x$. So, in your case &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ Ax = \text{vec}(A_1X) + \text{vec}(XA_2^T).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For applying the inverse operator to a vector (as needed for the smallest generalized eigenvalues), you could use a Krylov subspace method. You could use the same trick for the matvecs.&lt;/p&gt;&#xA;" OwnerUserId="4558" LastActivityDate="2018-05-24T19:56:53.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29595" PostTypeId="1" CreationDate="2018-05-24T21:45:03.610" Score="6" ViewCount="2129" Body="&lt;p&gt;I am translating some Matlab code into Python and I having some problems &lt;strong&gt;regarding matrix multiplication accuracy&lt;/strong&gt;. Assuming we have following data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A: 6x6 matrix&#xA;B: 5x5 matrix&#xA;C: 2x2 matrix&#xA;D: 5x5 matrix&#xA;E: 6x5 matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In Matlab, my operation looks as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R1 = A * (-( B*C(1,1) + D*C(2,1) ) * E.').'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Previous set of operations produces a 6x5 matrix (R1 matrix). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my Python code I have the same matrices and the operation looks as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R2 = np.matmul(A, np.matmul(-(np.multiply(B, C[0,0]) + np.multiply(D, C[1,0])), E.transpose()).transpose())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the same operation on same data produces different results in terms of &lt;code&gt;norm-2&lt;/code&gt;, namely: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;norm-2(R1-R2) = 7.4506e-09&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not able to understand why the results are different. Does anyone know what the reason could be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the sake of clarity, I attach the data and the scripts &lt;a href=&quot;https://www.dropbox.com/s/uayle64r81vbzfc/Matrix_multiplication.zip?dl=0&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. The instructions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Run the Run_python.py script. This will perform the operations and will generate a python_results.mat file.&lt;/li&gt;&#xA;&lt;li&gt;Run the Run_matlab script. This will perform the operations and will compare both results in terms of its norm-2. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Finally, my Python version is 3.6.3, with numpy (1.14.3) and scipy (1.1.0).&lt;/p&gt;&#xA;" OwnerUserId="27836" LastActivityDate="2018-05-25T08:40:59.700" Title="Matrix multiplication accuracy Matlab vs Python" Tags="|matlab|matrix|python|floating-point|accuracy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29596" PostTypeId="1" CreationDate="2018-05-24T21:57:02.113" Score="7" ViewCount="338" Body="&lt;p&gt;Consider the Schroedinger equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left(-\frac12\frac{\partial^2}{\partial x^2} + V(x) \right) \psi(x) = E \psi(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usual way to solve it is to introduce a discretization of $\psi(x)$. This yields an eigenvalue problem which can be solved by standard methods from linear algebra. When using a discretization that leads to hermitian (resp. symmetric) matrices, one obtains real eigenvalues which determine the energies of the eigenstates of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when one uses a non-symmetric discretization for the second-derivative matrix, such as those from a Chebyshev or Legendre collocation, or simply from an equidistant grid with forward or backward differences at the boundaries, one in general obtains complex eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that mean one can't use non-symmetric discretizations for quantum mechanics? Are there any workarounds?&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2018-05-28T06:27:35.030" LastActivityDate="2019-04-03T22:39:38.193" Title="Non-hermitian discretizations in quantum mechanics" Tags="|finite-difference|quantum-mechanics|collocation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29597" PostTypeId="2" ParentId="29595" CreationDate="2018-05-24T22:49:46.627" Score="14" Body="&lt;p&gt;Here is &lt;code&gt;R1&lt;/code&gt;, as computed in MATLAB:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   1.0e+07 *&#xA;  -7.382605957465515  -9.599867106092937  -2.830412177259742  -0.000000000002830  -0.000000000002830&#xA;  -1.230434326244253  -1.599977851015490  -0.471735362876624  -0.000000000000472  -0.000000000000472&#xA;   3.691302978732758   4.799933553046468   1.415206088629871   0.000000000001415   0.000000000001415&#xA;  -5.056592907133381  -6.575268976533256  -1.938643647277773  -0.000000000001939  -0.000000000001939&#xA;  -2.842303293624223  -3.695948835845781  -1.089708688245001  -0.000000000001090  -0.000000000001090&#xA;  -2.522882542531216  -3.280594585722160  -0.967246188042229  -0.000000000000967  -0.000000000000967&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;norm(R1) = 1.76e8&lt;/code&gt; (largest singular value)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remaining singular values of &lt;code&gt;R1&lt;/code&gt;, per MATLAB double precision, range from &lt;code&gt;1.48e-8&lt;/code&gt; down to &lt;code&gt;1.57e-37&lt;/code&gt;. So &lt;code&gt;R1&lt;/code&gt; is extremely ill-conditioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given &lt;code&gt;norm(R1) = 1.76e8&lt;/code&gt;, you can't expect &lt;code&gt;norm(R1)&lt;/code&gt; to be more accurate than your observed MATLAB to PYTHON &lt;code&gt;norm(R1-R2) = 7e-9&lt;/code&gt; (which already could be double the error in &lt;code&gt;norm(R1)&lt;/code&gt; due to triangle inequality). All told, you've done about an order of magnitude better than worst case for double precision, which has machine precision &lt;code&gt;2.2e-16&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="20688" LastEditDate="2018-05-25T02:00:19.647" LastActivityDate="2018-05-25T02:00:19.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29598" PostTypeId="2" ParentId="29592" CreationDate="2018-05-25T06:25:30.950" Score="6" Body="&lt;p&gt;First of all, don't even consider &amp;quot;optimizing&amp;quot; before you're using the right integration method. Chucking more computers at the problem may sound like the easiest way to solve it, but in reality it will be more difficult than you'd think (due to implicit BLAS multithreading explained later). If you're using explicit Euler, you can do a whole lot better just by changing the method. Proper adaptive time stepping (possibly implicit depending on the stiffness) algorithms can be orders of magnitude more efficient than explicit Euler. Don't even consider parallelizing and keeping explicit Euler unless you have a very good reason (i.e. for hyperbolic PDEs so you're concerned about SSP, but in that case &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html#Explicit-Strong-Stability-Preserving-Runge-Kutta-Methods-for-Hyperbolic-PDEs-(Conservation-Laws)-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;just use an SSP-optimized integrator&lt;/a&gt;... you're not going to have a good reason).&lt;/p&gt;&#xA;&lt;p&gt;Next, your derivative calculation function is by far the most expensive part of the calculation. First of all, you should make sure that it's optimized. Language won't matter much here since all of the time will be in the BLAS kernel for matrix multiplication, but make sure it's non-allocating via in-place operations, using BLAS or SIMD'd loops as much as possible, etc.&lt;/p&gt;&#xA;&lt;p&gt;Once that's all good, consider whether you still need to parallelize. With the right ODE methods and with a decently performant derivative implementation, 10^5 can be reasonably speedy. If you ended up using an implicit method, you'll want to do things like swap out the linear solver to some iterative solver, and that will give you another nice speedup.&lt;/p&gt;&#xA;&lt;p&gt;But since this is a post about parallelism, I'll keep going, assuming you want to do something extreme.&lt;/p&gt;&#xA;&lt;h2&gt;How to parallelize this: the details to know&lt;/h2&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; are matrices, if you're using a language with a standard BLAS setup then you're already parallelizing via multithreading. Just check your CPU usage (ex: &lt;code&gt;htop&lt;/code&gt; on Linux) and you'll see that large enough matrix multiplications with multithread in Python, R, Julia, MATLAB, etc. So, getting more performance out isn't as simple as just parallelizing it since if you're on a single computer it's likely you're using all of your cores optimally already.&lt;/p&gt;&#xA;&lt;p&gt;Parallel in time algorithms like parareal only make sense when the ODE is cheap since there's no other available parallelism. Parallel in time algorithms are not efficient, but they try to make up for it by allowing parallelism. The overhead cutoff can be quite high for these algorithms, I've read that 32 cores is a good low estimate. These are definitely not suitable to the way you've described your problem.&lt;/p&gt;&#xA;&lt;p&gt;However, there's two other levels of parallelism. If your solving is expensive, then solving at multiple systems at the same time in parallel will be almost optimal speedup. i.e. do a parallel for loop or map that solves 16 systems at a time in a 16 core machine and it will be about 16x speedup since the data transfer times are almost 0 compared to the full integration. For example, if you're using Julia, you can write this up yourself &lt;a href=&quot;https://diffeq.sciml.ai/en/stable/manual/parallel-computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;quite easily&lt;/a&gt;, or use what's &lt;a href=&quot;http://diffeq.sciml.ai/latest/features/monte_carlo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;built into DifferentialEquations.jl&lt;/a&gt;, or find the right resource for whatever other language. Remember, BLAS is already multithreading (and most likely those matrix multiplications are the most expensive operation in the loop) so this only makes sense if you can farm out to other computers/nodes. Julia's native parallelism does that, Python's multiprocessing can do it, you can use MPI in C/Fortran, etc. If you have a big cluster and want lots of gains for little work, this is a way to solve your problem.&lt;/p&gt;&#xA;&lt;p&gt;Finally, if you have an optimized derivative function with most of the time in matrix operations but don't have extra nodes hanging around (or if you do, you can still apply this), another level of parallelism you can exploit is within-method. You're already doing within-level parallelism via the multithreaded BLAS operations, but you can do better. GPUs are essentially parallel linear algebra machines, so if you have one hanging around then you can just perform the matrix multiplications on then it's a good idea. Setting it up isn't hard since things like CuBLAS are just BLAS on GPUs, so you just have to call it. However, &lt;a href=&quot;http://www.wolframalpha.com/input/?i=10%5E5+*+10%5E5+*+32+bits&quot; rel=&quot;nofollow noreferrer&quot;&gt;I think you'll have memory issues if you do it naively&lt;/a&gt;, so this could be difficult.&lt;/p&gt;&#xA;&lt;h2&gt;TL;DR&lt;/h2&gt;&#xA;&lt;p&gt;Stop using explicit Euler: the first thing you should do is really optimize your code and your method. If that's not enough, solve the system on different nodes at the same time. If that's not enough, accelerate the linear algebra with GPUs/Xeon Phis/etc. since the limiting step is the matrix operations.&lt;/p&gt;&#xA;&lt;p&gt;For reference, &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqTutorials.jl/blob/master/Introduction/OptimizingDiffEqCode.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's a notebook explaining the optimizing step&lt;/a&gt; and &lt;a href=&quot;http://www.stochasticlifestyle.com/solving-systems-stochastic-pdes-using-gpus-julia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's a blog post explaining within-method GPU parallelism&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T17:57:32.790" LastActivityDate="2020-07-19T17:57:32.790" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29599" PostTypeId="2" ParentId="29595" CreationDate="2018-05-25T08:22:59.807" Score="15" Body="&lt;p&gt;First, see Mark L. Stone's answers, which is completely correct. Second, realize that this is the reason why people told you to use relative errors in your numerical analysis class. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, the real question here is why the results do not coincide exactly, since both languages call some BLAS library functions for their computations. There are several very BLAS-specific reasons why this could happen:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Matlab and Python are probably linked to different versions of the BLAS libraries. They are not guaranteed to yield the exact same result up to the last bit. Different optimization could rearrange sums in different ways (using associativity), and give slightly different results. See &lt;a href=&quot;https://bebop.cs.berkeley.edu/reproblas/&quot; rel=&quot;noreferrer&quot;&gt;https://bebop.cs.berkeley.edu/reproblas/&lt;/a&gt; for more details.&lt;/li&gt;&#xA;&lt;li&gt;Matlab includes special tricks to detect multiplications of the form &lt;code&gt;A*B'&lt;/code&gt;, &lt;code&gt;A*B.'&lt;/code&gt;, &lt;code&gt;A*A'&lt;/code&gt; and map them to a single BLAS call. I'm not sure Python does that, too, and if it does it might map them to a slightly different call when something like &lt;code&gt;(A*B.').'&lt;/code&gt; is used.&lt;/li&gt;&#xA;&lt;li&gt;Both languages may include optimizations to map &lt;code&gt;A*B&lt;/code&gt; where &lt;code&gt;A&lt;/code&gt; and/or &lt;code&gt;B&lt;/code&gt; are symmetric to a specialized BLAS call (&lt;code&gt;DSYMM&lt;/code&gt;). Again, this might be implemented in slightly different ways in the two languages, and possibly also depend on the result of previous computations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2018-05-25T08:40:59.700" LastActivityDate="2018-05-25T08:40:59.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29601" PostTypeId="2" ParentId="29596" CreationDate="2018-05-25T14:37:42.210" Score="10" Body="&lt;p&gt;If your discrete equations represent your problem properly you should have the right eigenvalues, although numerically you can get a small imaginary part when solving the eigenvalue problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, let us consider the axisymmetric case of a cylindrical well. The Schrödinger equation would be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial^2 \psi}{\partial r^2} + \frac{1}{r}\frac{\partial \psi}{\partial r} = E \psi\, .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a naive finite difference we can use the following approximations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\frac{\partial^2 \psi(r_i)}{\partial r^2} \approx \frac{\psi_{i+1} - 2\psi_i + \psi_{i-1}}{\Delta r^2}\, ,\\&#xA;&amp;amp;\frac{\partial \psi(r_i)}{\partial r} \approx \frac{\psi_{i+1} - \psi_{i-1}}{2\Delta}\, ,&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the second approximation leads to a skew-symmetric matrix. The discrete problem would be then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$[D_{2r} + R_\text{inv} D_{r}] \{\Psi\} = E \{\Psi\}\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;being &lt;span class=&quot;math-container&quot;&gt;$D_{2r}$&lt;/span&gt; the matrix of second derivatives,  &lt;span class=&quot;math-container&quot;&gt;$D_{r}$&lt;/span&gt; the matrix of first derivatives and &lt;span class=&quot;math-container&quot;&gt;$R_\text{inv}$&lt;/span&gt; a diagonal matrix with &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{r_i + \Delta r/2}$&lt;/span&gt; as entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following snippet solves this problem&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division, print_function&#xA;from scipy.special import j0, jn_zeros&#xA;import numpy as np&#xA;from scipy.sparse import diags&#xA;from scipy.sparse.linalg import eigs&#xA;import matplotlib.pyplot as plt&#xA;&#xA;# Parameters&#xA;r_max = 1.0&#xA;dr = 0.01&#xA;r = np.arange(0, r_max, dr)&#xA;npts = r.shape[0]&#xA;nvals = 10&#xA;&#xA;# Matrices&#xA;D2 = diags([1, -2, 1], [-1, 0, 1], shape=(npts - 1, npts - 1))/dr**2&#xA;D1 = diags([-0.5, 0, 0.5], [-1, 0, 1], shape=(npts - 1, npts - 1))/dr&#xA;r_new = r[0:npts-1] + dr/2&#xA;R_inv = diags(1/r_new, 0)&#xA;A = D2 + R_inv.dot(D1)&#xA;&#xA;# Finite differences&#xA;vals, vecs = eigs(-A, k=nvals, which=&quot;SM&quot;) &#xA;vecs = np.vstack((vecs, np.zeros(vecs.shape[1])))&#xA;&#xA;# Analytic results&#xA;vals_anal = jn_zeros(0, nvals)&#xA;vecs_anal = j0(np.outer(r, vals_anal)/r_max)&#xA;&#xA;&#xA;## Plots&#xA;&#xA;# Eigenvalues&#xA;plt.figure()&#xA;plt.plot(vals_anal**2)&#xA;plt.plot(vals, &quot;ok&quot;)&#xA;plt.legend([&quot;Analytic eigenvalues&quot;,&#xA;            &quot;Numeric eigenvalues&quot;])&#xA;plt.xlabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;&quot;)&#xA;plt.ylabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$E_n$&lt;/span&gt;&quot;)&#xA;&#xA;&#xA;# Eigenvectors&#xA;plt.figure()&#xA;plt.subplot(121)&#xA;plt.plot(r, np.abs(vecs[:, 0:4]/vecs[0, 0:4])**2)&#xA;plt.title(&quot;Finite differences&quot;)&#xA;plt.xlabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;&quot;)&#xA;plt.ylabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$|\psi|^2$&lt;/span&gt;&quot;)&#xA;&#xA;plt.subplot(122)&#xA;plt.plot(r, np.abs(vecs_anal[:, 0:4])**2)&#xA;plt.title(&quot;Analytic solution&quot;)&#xA;plt.xlabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;&quot;)&#xA;plt.ylabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$|\psi|^2$&lt;/span&gt;&quot;)&#xA;&#xA;plt.tight_layout()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the following eigenvalues&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xFb4e.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xFb4e.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and these eigenvectors&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hdDjw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hdDjw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2019-04-03T22:39:38.193" LastActivityDate="2019-04-03T22:39:38.193" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29602" PostTypeId="1" CreationDate="2018-05-25T15:59:51.873" Score="3" ViewCount="179" Body="&lt;p&gt;I have to diagonalize, within a Fortran-written code, a block tridiagonal Toeplitz Hermitian matrix, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left[&#xA;\begin{array}{ccccc}&#xA;\ddots &amp;amp; \hat{A} &amp;amp;  &amp;amp;  &amp;amp;  \\&#xA;\hat{A}^\dagger &amp;amp; \hat{B} &amp;amp; \hat{A} &amp;amp; &amp;amp; \\&#xA;&amp;amp; \hat{A}^\dagger &amp;amp; \hat{B} &amp;amp; \hat{A} &amp;amp; \\&#xA;&amp;amp; &amp;amp; \hat{A}^\dagger &amp;amp; \hat{B} &amp;amp; \hat{A} \\&#xA;&amp;amp; &amp;amp; &amp;amp; \hat{A}^\dagger &amp;amp; \ddots&#xA;\end{array}&#xA;\right]&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $B$ is a Hermitian matrix. For the moment, I am just using standard Lapack routine &lt;code&gt;ZHEEV&lt;/code&gt; for Hermitian matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any suggestion on how to take advantage of one or more of the properties of this matrix to get a faster computation?&lt;/p&gt;&#xA;" OwnerUserId="27815" LastEditorUserId="20688" LastEditDate="2018-05-25T16:25:57.300" LastActivityDate="2018-06-28T13:22:09.473" Title="Diagonalizing a block tridiagonal Toepliz Hermitean matrix" Tags="|linear-algebra|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29603" PostTypeId="2" ParentId="29602" CreationDate="2018-05-25T16:49:04.800" Score="2" Body="&lt;p&gt;There are morally equivalent routines that seek the same results via different algorithms: ZHEEVD (divide and conquer) and ZHEEVR (multiply relatively robust representations). Perhaps one of them can outperform ZHEEV? (it uses QR iteration).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also want to try ZHBEV (hermitian eigensolver that  exploits bandedness). I would think that bandedness would mainly speedup the frontend, band-to-tridiagonal reduction, so it might not be all that faster in the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think another experiment worth running, is to apply ZHBTRD (reduce band hermitian matrix to real tridiagonal form) to your matrix. If the tridiagonal form is also Toeplitz**, you can compute its eigenvalues analytically (see the wikipedia page on tridiagonal matrices). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't need the whole spectrum, you might be better off using Krylov techniques (reduction to tridiagonal form may still be useful in this setting as a frontend to speedup the matvec action).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;** Unfortunately I don't have sufficient experience with Toeplitz systems to say for certain, whether or not Toeplitz-ness is preserved by tridiagonalization.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-05-29T13:00:23.943" LastActivityDate="2018-05-29T13:00:23.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29604" PostTypeId="2" ParentId="29579" CreationDate="2018-05-25T17:09:39.150" Score="2" Body="&lt;p&gt;You could also take advantage of the symmetry of the problem. As an added advantage you end up with a mesh with half the elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would just consider half of the beam and add roller constraints on D and also on the midplane of the beam, as presented in the following schematic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L8B5p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L8B5p.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This represents a mixed boundary condition in D. Horizontally, it has prescribed zero traction, and vertically a prescribed displacement of zero..&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2020-04-10T18:41:37.080" LastActivityDate="2020-04-10T18:41:37.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29608" PostTypeId="1" CreationDate="2018-05-27T01:03:18.440" Score="3" ViewCount="423" Body="&lt;p&gt;This concerns algorithms for searching for the minimum of a continuous unidimensional unimodal function.  The simplest method is three points, and bi-sectioning.  The golden-section search seems more natural.  Although Brent seems a little better yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to find a reference as to how much more efficient these searches are relative to one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need something like an &quot;expected number of function samples to reach tolerance xtol.&quot;  One can assume a uniform distribution of the minimum, and some distribution of the brackets, perhaps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;does someone have a good reference?&lt;/p&gt;&#xA;" OwnerUserId="27854" LastEditorUserId="27854" LastEditDate="2018-05-27T01:48:26.367" LastActivityDate="2018-05-27T01:48:26.367" Title="Quantifying Superiority of Golden Section vs Binary Search vs Brent on Continuous Function" Tags="|search|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29609" PostTypeId="1" AcceptedAnswerId="29613" CreationDate="2018-05-27T03:51:23.993" Score="3" ViewCount="562" Body="&lt;p&gt;This is really a general question, but my interest is primarily for structural mechanics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say you are modeling an elastic solid material that is orthotropic. You have the orthotropic properties (elastic modulus in 3 directions, the 6 poissons ratios, and the shear modulus in the 3 directions), but you only have a code that can take in isotropic properties. So if you want to model an orthotropic material with an isotropic code, you'd have to assume the orthotropic material exhibits isotropic properties. In doing so, would you just be averaging the properties to obtain a hypothetical isotropic property? For example, you would average the 3 elastic moduli in the 3 principal directions to obtain one single modulus for use in the isotropic code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is probably not a great example because it is pretty easy to change a linear elastic solver code to take in orthotropic properties as opposed to only isotropic properties. &lt;/p&gt;&#xA;" OwnerUserId="27504" LastActivityDate="2018-05-27T17:51:25.430" Title="Modeling orthotropic materials with isotropic assumption" Tags="|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29611" PostTypeId="1" CreationDate="2018-05-27T15:41:18.577" Score="4" ViewCount="299" Body="&lt;p&gt;I'm a high-schooler building a small vehicle for an independent study. I've had finite element modelling recommended to me as a way to save time during the design process, and I'd like to try it out. Unfortunately, the person who recommended FEM to me claims his knowledge is quite outdated. I have some coding experience and I'm a quick learner. How should I get started with FEM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I have a budget of zero dollars.&lt;/p&gt;&#xA;" OwnerUserId="27859" LastActivityDate="2018-05-28T20:26:26.233" Title="Getting started with finite element modelling" Tags="|finite-element|software-recommendation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29612" PostTypeId="2" ParentId="29539" CreationDate="2018-05-27T16:32:52.390" Score="4" Body="&lt;p&gt;You're comparing statically-defined arrays which live on the stack:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    double a[100][100];&#xA;    double b[100][100];&#xA;    double c[100][100];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to heap allocated arrays:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    mat a1= randi&amp;lt;mat&amp;gt;(100, 100, distr_param(1, 10));&#xA;    mat b1= randi&amp;lt;mat&amp;gt;(100, 100, distr_param(1, 10));&#xA;    mat c1=zeros(100,100);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course, &lt;a href=&quot;https://www.gribblelab.org/CBootCamp/7_Memory_Stack_vs_Heap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;stack-allocated variables are more constrained but have faster access than heap allocated variables&lt;/a&gt;. Additionally, this loop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;             a[i][j] = rand() % 10;&#xA;             b[i][j] = rand() % 10;&#xA;             c[i][j] = a[i][j] + b[i][j];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;doesn't actually need &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; if those are not returned from the function, so as @Kirill mentions in the comments some compilers will just completely remote those arrays and put it all into &lt;code&gt;c&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(also make sure you're compiling correctly. Use higher optimization level flags like &lt;code&gt;-O3&lt;/code&gt;, make sure it's not in debug mode, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there's a few morals to this story. First of all, using stack-allocated arrays is awesome. You should do it whenever possible. But secondly, you shouldn't expect MATLAB vs Armadillo to be very different if all of the time is spent in MATLAB operations since MATLAB is essentially a library for calling things like Armadillo. Writing out the MATLAB code as a for loop will be slow, but its vectorized operations are very optimized and &lt;a href=&quot;https://www.mathworks.com/matlabcentral/answers/95958-which-matlab-functions-benefit-from-multithreaded-computation&quot; rel=&quot;nofollow noreferrer&quot;&gt;many times implicitly multithreaded&lt;/a&gt;. While you don't give MATLAB code here to actually show what's going on, that implicit multithreading could be the distinguishing factor in the timings. Of course, you can always multithread and parallelize your own code in C and get it just as good, but the advantage of vectorized programming languages like MATLAB is that the hard work on simple functions has already been done for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what this is really saying is that MATLAB vectorization is fine/good when dealing with operations on the heap, or if you want to write out more complicated loops then you should try to use static arrays whenever possible and only go to heap-allocated array operations when necessary. Unfortunately, when working in something like MATLAB/Python/R your operations are with heap-allocated variables which is why they focus so much on vectorization and large array operations. Using C or Fortran gives you the choice between heap vs stack that these languages don't have, while libraries like Armadillo add on the MATLAB-like functionality. From the other direction, something like &lt;a href=&quot;https://julialang.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Julia&lt;/a&gt; does have libraries like &lt;a href=&quot;https://github.com/JuliaArrays/StaticArrays.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;StaticArrays.jl&lt;/a&gt; to utilize static arrays (and &lt;a href=&quot;https://docs.julialang.org/en/stable/manual/parallel-computing&quot; rel=&quot;nofollow noreferrer&quot;&gt;parallelism&lt;/a&gt;), so that's one option to both have the flexibility to mix the designs while staying in a higher level language. But no matter where you look, optimized vectorized heap operations will likely end up the same speed as MATLAB: it's the more complicated operations and loops which you would have to write yourself that make the speed difference.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-05-27T16:32:52.390" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29613" PostTypeId="2" ParentId="29609" CreationDate="2018-05-27T17:51:25.430" Score="3" Body="&lt;p&gt;First, I would like to mention that orthotropic materials have 9 parameters, and not 12, as can be implied from your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest approach would be to compute the arithmetic/geometric mean between your parameters. Another one would identify which of the moduli is more important in your problem of interest, although that means that you should know your solution already, and that it can vary from one problem to another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An approach that is more theoretically sound is to find the closest isotropic tensor to your orthotropic one. That implies the following optimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min_{\lambda, \mu}\quad d(C_\text{iso}, C_\text{ortho})\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that you can use three different metrics, that I know of. Let's consider two tensors $C_1$ and $C_2$, the different metrics are defined by:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The Frobenius metric&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$d_F(C_1, C_2) = \Vert C_1 - C_2 \Vert\, .$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The log-Euclidean metric&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$d_L(C_1, C_2) = \left\Vert \log(C_1) - \log(C_2)\right\Vert\, .$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;The Riemannian metric&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$$d_R(C_1, C_2) = \left\Vert(C_1^{-1/2} C_2 C_1^{-1/2}\right\Vert\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The logarithm and square root used above are the matrix logarithm and the square root of a matrix. Also, it should be noted that the log-Euclidean and Riemmanian norms that induce these metrics are better suited in the sense that they are invariant under inversion and return the same result for stiffness and compliance tensors distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a last comment, in the case of Frobenius metric you can obtain an analytic solution easily, and the solution is (I would double check these)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;\lambda + 2\mu = \frac{1}{33}[9(C_{11} + C_{22} + C_{33}) + 4(C_{44} + C_{55} + C_{66}) + 2(C_{12} + C_{13} + C_{23})]\, ,\\&#xA;&amp;amp;\mu = \frac{1}{33}[4(C_{11} + C_{22} + C_{33}) + 3(C_{44} + C_{55} + C_{66}) - 4(C_{12} + C_{13} + C_{23})]\, .&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Moakher, Maher, and Andrew N. Norris. &quot;The closest elastic tensor of arbitrary symmetry to an elasticity tensor of lower symmetry.&quot; Journal of Elasticity 85.3 (2006): 215-263. Preprint: &lt;a href=&quot;https://arxiv.org/pdf/cond-mat/0608311&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/cond-mat/0608311&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Norris, Andrew. &quot;The isotropic material closest to a given anisotropic material.&quot; Journal of Mechanics of Materials and Structures 1.2 (2006): 223-238. Preprint: &lt;a href=&quot;https://arxiv.org/pdf/cond-mat/0509705&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/cond-mat/0509705&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-05-27T17:51:25.430" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29615" PostTypeId="1" CreationDate="2018-05-28T04:08:33.120" Score="1" ViewCount="34" Body="&lt;p&gt;Suppose we have a primal problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;p^{*}=\min_x f(x), \\\text{s.t.}~~ h_i(x) \leq 0,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f(.)$ and $h_i(.)$ are possibly non-convex. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then its Lagrangian is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathcal{L}(x,z_i)= f(x) + \sum_i z_i h_i(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the dual problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d^{*}=\max_{z_i\geq 0} \min_x \mathcal{L}(x,z_i)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now it is well-known that $p^{*}\geq d^{*}$ and that the dual problem is convex. Is this the best possible convex lower bound in general? Are there instances of well-known problems where we know that there exists a better lower bound which also comes from a convex optimization problem.&lt;/p&gt;&#xA;" OwnerUserId="3255" LastActivityDate="2018-05-28T04:08:33.120" Title="Best Possible Convex bounds for optimization problems" Tags="|optimization|convex-optimization|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="29616" PostTypeId="1" CreationDate="2018-05-28T06:09:40.490" Score="2" ViewCount="763" Body="&lt;p&gt;I have implemented BFGS myself from scratch in order to solve minimization problems.  Part of BFGS, as I understand it, is that the approximation to the Hessian is supposed to be positive definite, which means it is supposed to be invertible.   I have been following the algorithm outlined here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm#Algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm#Algorithm&lt;/a&gt; .  This is how we update our approximation to the Hessian:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ B_{k+1} = B_k + \frac{ y_k y_k ^T }{y_k^T s_k} – \frac{ B_k s_k s_k^T B_k}{s_k^T B_k s_k} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &#xA;$ y_k = \nabla f(x_k) - \nabla f(x_{k-1}) $ and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$s_k = -1 * B_k^{-1} \nabla f (x_k)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made a few slight modifications to the standard BFGS algorithm in my implementation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)  I check to see if $y_k^T s_k$ is negative.  If it is, then this has the potential to make the next update $B_k$ negative, too.  So, I switch to an alternative update when that term is negative:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ B_{k+1} = B_k + \frac{ y_k y_k ^T }{y_k^T y_k} – \frac{ B_k s_k s_k^T B_k}{s_k^T B_k s_k} $$&#xA;2)  I have a check for when the direction it is about to step $d_k$ might be too large or too small compared to the magnitude of the gradient:  if either $||d_k|| \ge 100*||\nabla f|| $ or $||\nabla f^T * d_k|| \le 10^{-4} * ||\nabla f||*||d_k||$.  If either one of those are true, then I just use the gradient step, $d_k = \nabla f $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am running into an issue where the gradient somehow becomes very large:  $\nabla f$ becomes very large.  So, when I go to update $s_k$, it becomes very large:  $s_k = -1 * B_k^{-1} \nabla f$.  Then, this causes the next iterate of $B_k$ to be very large.  Eventually, the matrix becomes so large that &lt;code&gt;np.linalg.inv(B_k)&lt;/code&gt; throws an error and will not be inverted.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I protect against this happening?  Do I need another check, and why isn’t my existing preventative measure working for this?&lt;/p&gt;&#xA;" OwnerUserId="27864" LastActivityDate="2018-05-30T08:40:40.810" Title="How to prevent BFGS from getting stuck on astronomically large gradient?" Tags="|optimization|convex-optimization|newton-method|quasi-newton|numerics|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="29620" PostTypeId="2" ParentId="29611" CreationDate="2018-05-28T14:41:04.673" Score="3" Body="&lt;p&gt;In my experience, learning FEM is less about coding and more about learning the math that constitutes the foundation of the method. Essentially coding FEM simulations boils down to physics and calculus, and you didn't mention your background in either of those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option for you would be to use a software package that has some prebuilt models included. I saw some comments suggesting dealii and FEniCS - I'll add some more free tools to the list: Onelab (a gui/solver package built on top of gmsh), and OpenCASCADE, an open-source CAE/CAD simulation tool. These two are both very friendly to new CAD/FEM users relative to other open-source FEM software packages.&lt;/p&gt;&#xA;" OwnerUserId="19791" LastEditorUserId="19791" LastEditDate="2018-05-28T20:26:26.233" LastActivityDate="2018-05-28T20:26:26.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29621" PostTypeId="2" ParentId="29611" CreationDate="2018-05-28T15:20:50.780" Score="5" Body="&lt;p&gt;I would suggest that you start with &lt;a href=&quot;https://freecadweb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FreeCAD&lt;/a&gt;. It is a CAD software and you can do Finite Element Analysis using the graphic interface pretty straightforward. FreeCAD provides you all the stages needed in a Finite Element Analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you would need the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A CAD for the geometries, FreeCAD is a good option.&lt;/li&gt;&#xA;&lt;li&gt;Besides FreeCAD, I can suggest you &lt;a href=&quot;http://gmsh.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gmsh&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;FEM solvers. Two good options are:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.csc.fi/web/elmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Elmer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.agros2d.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Agros2D&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;A software for visualization, &lt;a href=&quot;https://www.paraview.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ParaView&lt;/a&gt; is a good option.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-05-28T15:31:10.043" LastActivityDate="2018-05-28T15:31:10.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29623" PostTypeId="1" AcceptedAnswerId="29624" CreationDate="2018-05-28T20:36:14.083" Score="1" ViewCount="2143" Body="&lt;p&gt;I get following error after implementing the attached code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error Message &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;fmincon stopped because the size of the current step is less than&#xA;the default value of the step size tolerance but constraints are not&#xA;satisfied to within the selected value of the constraint tolerance.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rho = (1/3)*eye(3);&#xA;&#xA;&#xA;v1 = sdpvar(3,1);&#xA;v2 = sdpvar(3,1);&#xA;v3 = sdpvar(3,1);&#xA;v4 = sdpvar(3,1);&#xA;v5 = sdpvar(3,1);&#xA;&#xA;obj2 = trace((v1*v1' + v2*v2'+v3*v3'+v4*v4'+v5*v5')*rho);&#xA;&#xA;&#xA;cons2 = [sum(v1.*v1) == 1; sum(v2.*v2) == 1;sum(v3.*v3) == 1;sum(v4.*v4) == 1;sum(v5.*v5) == 1; sum(v1.*v2) == 0;sum(v2.*v3) == 0;sum(v3.*v4) == 0;sum(v4.*v5) == 0;sum(v5.*v1) == 0];&#xA;&#xA;ops2 = sdpsettings('solver','fmincon');&#xA;&#xA;optimize(cons2, -obj2, ops2);&#xA;&#xA;obj2 = double(obj2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried changing tolerances by adding following code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options = optimset('TolX',1e-30,'TolFun', 1e-6)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried different values, but it doesn't work. Any suggestions to debug it will be helpful.Suggestions to reformulate the problem or use another solver will also be appreciated (in case fmincon is not the best solver for this problem).&lt;/p&gt;&#xA;" OwnerUserId="7832" LastEditorUserId="16756" LastEditDate="2018-05-29T06:05:46.563" LastActivityDate="2018-05-29T06:05:46.563" Title="FMINCON Step Size Tolerance" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29624" PostTypeId="2" ParentId="29623" CreationDate="2018-05-28T23:29:51.723" Score="2" Body="&lt;p&gt;FMINCON is not able to find a feasible point starting at what I think is the default value provided by YALMIP of all variables being zero vectors.  Local solvers, such as FMINCON, may have trouble finding feasible points to non-convexly-constrained problems such as this, annd may be dependent on having an adequate starting point in order to even find a feasible point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, I just picked somewhat arbitrarily starting values (which are infeasible, by the way) for all variables being vectors of ones, and FMINCON quickly found what turned out to be the globally optimal solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, I used YALMIP's BMIBNB global optimizer, with FMINCON as upper solver, and even without providing any non-default starting value, BMIBNB solved the problem to global optimality almost immediately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I used instead of your optimize command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% FMINCON with non-default starting values&#xA;assign(v1,ones(3,1));&#xA;assign(v2,ones(3,1));&#xA;assign(v3,ones(3,1));&#xA;assign(v4,ones(3,1));&#xA;assign(v5,ones(3,1));&#xA;optimize(cons2, -obj2,sdpsettings('solver','fmincon','usex0',1))&#xA;&#xA;% BMIBNB, with FMINCON as upper solver, default starting values --&amp;gt; produces global optimum&#xA;optimize(cons2, -obj2,sdpsettings('solver','bmibnb','bmibnb.uppersolver','fmincon')&#xA;&#xA;disp(value(obj2))&#xA;  1.6667&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BTW, I don't think your non-default tolerance settings would have helped, but they were not used. Given that you are using YALMIP, you need to specify any non-default solver options in sdpsettings as discussed at &lt;a href=&quot;https://yalmip.github.io/command/sdpsettings/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://yalmip.github.io/command/sdpsettings/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16756" LastEditorUserId="16756" LastEditDate="2018-05-28T23:48:28.217" LastActivityDate="2018-05-28T23:48:28.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29628" PostTypeId="1" CreationDate="2018-05-29T20:20:57.530" Score="7" ViewCount="149" Body="&lt;p&gt;Simple hard-sphere dynamical systems can exhibit chaotic dynamics. Due to finite-precision arithmetic when implemented on a computer, the presence of chaos implies that for a given set of initial data, simulations of such systems on different machines or run with different arithmetic precisions can yield wildly different dynamics for sufficiently long times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, for e.g. systems of many hard spheres in a box with reflecting boundaries, can it be mathematically proven that qualitative and/or statistical aspects of the system's finite-precision-simulated behavior will approximate its infinite-precision behavior?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm particularly curious about simple event-driven simulations of such systems such as that devised by Alder and Wainwright in 1959¹.  I would imagine that the answer to the question is &lt;em&gt;yes&lt;/em&gt;, because otherwise what's the use of these methods which, as far as I can tell, are ubiquitous?  I can't seem to find a good, reasonably specific discussion of this anywhere.  References appreciated.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Alder, B.J. and Wainwright, T.E., 1959. Studies in molecular dynamics. I. General method. The Journal of Chemical Physics, 31(2), pp.459–466.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14624" LastEditorUserId="7574" LastEditDate="2018-05-29T21:37:30.117" LastActivityDate="2018-05-30T08:17:26.810" Title="What good are hard-sphere event-driven molecular dynamics simulations in the face of chaos?" Tags="|reference-request|molecular-dynamics|precision|numerics|chaotic-systems|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29629" PostTypeId="2" ParentId="29628" CreationDate="2018-05-29T21:35:24.507" Score="3" Body="&lt;p&gt;Something that is at least very close to what your are asking for is the &lt;a href=&quot;http://www.scholarpedia.org/article/Shadowing_lemma_for_flows&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shadowing Lemma&lt;/a&gt;, which roughly states that near any numerical solution of a dynamical system, you will find a real solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am no expert on this, so I can only provide this a starting point for further research.&#xA;In particular I do not know whether this translates to non-smooth dynamical systems (such as hard-sphere molecular dynamics) or whether any statement can be made about statistical properties.&#xA;Still, it should provide a starting point for a literature search.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2018-05-30T08:17:26.810" LastActivityDate="2018-05-30T08:17:26.810" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29630" PostTypeId="2" ParentId="29628" CreationDate="2018-05-29T22:08:42.793" Score="6" Body="&lt;p&gt;Yes, it is possible to show that the statistical behavior of the approximate system will reach that of the &quot;exact&quot; system. (This is true even though hard-sphere dynamics do not accurately describe molecular systems!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic premise underlying molecular dynamics is the &lt;strong&gt;ergodic theorem&lt;/strong&gt;, which states that, in the limit of long times, the time average of a statistical measure of the system is equivalent to its ensemble average:&#xA;$$\lim_{t \to \infty} \bar{A} = \left&amp;lt; A \right&amp;gt;,$$&#xA;where $\bar{A}$ is the time average and $\left&amp;lt;A\right&amp;gt;$ is the ensemble average of the quantity $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in principle, the time average must be measured for an infinite period of time to get exact equality, but in practice, the results will begin to converge within experimental error as the time and the system size increases. (The exact amount of time this will take can be measured, as is the decorrelation time required to achieve statistical independence between time samples, which can itself be used as a proxy to determine how long a simulation is required.) It can be shown using &lt;a href=&quot;https://dx.doi.org/10.1137/S0036142997329797&quot; rel=&quot;noreferrer&quot;&gt;backward error analysis&lt;/a&gt; that the error of the numerical integration of a Hamiltonian can be made exponentially small; this applies to the &lt;a href=&quot;http://www.sandia.gov/~sdbond/preprints/2007_Acta_Numerica_16_1.pdf&quot; rel=&quot;noreferrer&quot;&gt;integrators in molecular dynamics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While numerical errors will take you on eventually exponentially divergent trajectories, in principle, so long you're generating the states in a manner consistent with the ensemble one is trying to observe, this should not ultimately impact the results, so long as a sufficient number of samples are collected. This means, for example, if we use a thermostat to ensure constant temperature, we can only equate the results with the results of the canonical ensemble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An excellent discussion of the ergodic theorem can be found in &lt;a href=&quot;https://archive.org/details/ThePrinciplesOfStatisticalMechanicsTolmanOxfordAtTheClarendonPress1938&quot; rel=&quot;noreferrer&quot;&gt;Tolman's &lt;em&gt;Principles of Statistical Mechanics&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="348" LastEditorUserId="348" LastEditDate="2018-05-30T00:00:55.797" LastActivityDate="2018-05-30T00:00:55.797" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29632" PostTypeId="1" AcceptedAnswerId="29672" CreationDate="2018-05-28T23:49:17.050" Score="1" ViewCount="288" Body="&lt;p&gt;I am trying to use &lt;a href=&quot;https://doi.org/10.1080/10556789908805765&quot; rel=&quot;nofollow noreferrer&quot;&gt;CSDP&lt;/a&gt; and am struggling with it. Consider, for example, the following semidefinite program&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; 0\\ \text{subject to} &amp;amp; Q - A' Q A - \varepsilon I \succeq 0\\ &amp;amp; Q \succ 0\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is a given matrix with spectral radius $\rho(A) &amp;lt; 1$. I need a Lyapunov function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The condition $\rho(A) &amp;lt; 1$ the existence of a matrix norm $\| \cdot \|$ such that $\|A\| &amp;lt; 1$. Let $Q$ such that $Q \succ 0$ and $Q - A' Q A \succ 0$ (exists since $\rho(A) &amp;lt; 1$). I am wondering how one represents it in &lt;a href=&quot;https://cran.r-project.org/web/packages/Rcsdp/Rcsdp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the standard CSDP format&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{maximize} &amp;amp; \mbox{tr} (C X)\\ \text{subject to} &amp;amp; \mbox{tr} (A_i X) = b_i, \qquad i \in \{1,2,\dots,n\}\\ &amp;amp; X \succeq 0\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider, for example, the semidefinite program proposed by Prof. &lt;a href=&quot;https://pdfs.semanticscholar.org/7eb8/c4429f323b64318586841b23e92441ceed64.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Borchers&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27794" OwnerDisplayName="RàMi" LastEditorUserId="27794" LastEditDate="2018-06-03T22:52:34.827" LastActivityDate="2018-07-23T14:02:47.513" Title="How to use CSDP to express a semidefinite program?" Tags="|linear-algebra|optimization|linear-solver|convex-optimization|constrained-optimization|" AnswerCount="1" CommentCount="19" ContentLicense="CC BY-SA 4.0" />
  <row Id="29633" PostTypeId="2" ParentId="16033" CreationDate="2018-05-30T19:20:18.063" Score="0" Body="&lt;p&gt;I couldn't resist to try out the Hungarian method to prove that 88 is the maximal value that could be achieved. I found a description here: &lt;a href=&quot;http://www.math.harvard.edu/archive/20_spring_05/handouts/assignment_overheads.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bruff, Derek, &quot;The Assignment Problem and the Hungarian Method&quot;&lt;/a&gt;, without any explanation why the method terminates an what is its running time. The algorithm transforms a cost matrix of an assignment problem to a new cost matrix by adding numbers to rows or columns of a cost matrix. &#xA;A theorem from the document says &quot;If a number is added to or subtracted from all of the entries of any one row or column of a cost matrix, then an optimal assignment for the resulting cost matrix is also an optimal assignment for the original cost matrix&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because a number $a$ is added to all entries of a row or a column then the value of any assignment is increased by $a$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method described in the document finds a minimum. To find  maximum we multiply the cost matrix by $-1$ and find a minimum of this matrix,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we start with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  16  16  18  18  18&#xA;  20  18  16  12  10&#xA;  20  18  18  16  16&#xA;  18  18  16  16   8&#xA;  10  12  14  14  14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and multiply it by $-1$ &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; -16 -16 -18 -18 -18&#xA; -20 -18 -16 -12 -10&#xA; -20 -18 -18 -16 -16&#xA; -18 -18 -16 -16  -8&#xA; -10 -12 -14 -14 -14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To avoid negative numbers we add $20$ to each element. hen for each row we subtract its minimum.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   4   4   2   2   2   | -2&#xA;   0   2   4   8  10   |&#xA;   0   2   2   4   4   |&#xA;   2   2   4   4  12   | -2&#xA;  10   8   6   6   6   | -6&#xA;&#xA;   2   2   0   0   0&#xA;   0   2   4   8  10&#xA;   0   2   2   4   4&#xA;   0   0   2   2  10&#xA;   4   2   0   0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now for each row the minimum should be subtracted but this is 0 for every row so there is nothing to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following matrix can be ignored if you haven't read the linked document.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   +----------------&#xA;   |   2   4   8  10&#xA;   |   2   2   4   4&#xA;   +----------------&#xA;   +----------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We continue to transform the matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   2   2   0   0   0   |&#xA;   0   2   4   8  10   | -2&#xA;   0   2   2   4   4   | -2&#xA;   0   0   2   2  10   |&#xA;   4   2   0   0   0   |&#xA;  ------------------&#xA;  +2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and finally, arrive at&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   4   2   0   0   0&#xA;   0   0   2   6   8&#xA;   0   0   0   2   2&#xA;   2   0   2   2  10&#xA;   6   2   0   0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now we select the following positions containing &lt;code&gt;0&lt;/code&gt; and mark them by &lt;code&gt;x&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   4   2   0   x  0&#xA;   x   0   2   6   8&#xA;   0   0   x   2   2&#xA;   2   x   2   2  10&#xA;   6   2   0   0   x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These &lt;code&gt;x&lt;/code&gt;-positions show an admissible assignment in this cost matrix, and it is minimal because a value smaller than &lt;code&gt;0&lt;/code&gt; cannot be achieved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This assignement is also optimal for the initial cost matrix. TSo we have proven that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;18+20+18+18+14=88&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is the maximum value.&lt;/p&gt;&#xA;" OwnerUserId="1047" LastEditorUserId="1047" LastEditDate="2018-05-31T06:04:38.717" LastActivityDate="2018-05-31T06:04:38.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29634" PostTypeId="1" CreationDate="2018-05-30T20:11:42.947" Score="5" ViewCount="393" Body="&lt;p&gt;I am xposting this from my original stackoverflow question where I was presented with a coding challenge that I have been able to narrow down extensively and I think it lies with Euler's Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my initial &lt;a href=&quot;https://stackoverflow.com/questions/50515888/unity-c-sharp-firing-a-projectile-at-a-target-rotating-around-moving-axis&quot;&gt;post&lt;/a&gt; with my previous assumptions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has been a while since I took differential equations and linear algebra so I am struggling to remember/keep up with the annotations of this and I am hoping someone can help me out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{aligned}I_{1}{\dot  {\omega }}_{{1}}+(I_{3}-I_{2})\omega _{2}\omega _{3}&amp;amp;=M_{{1}}\\I_{2}{\dot  {\omega }}_{{2}}+(I_{1}-I_{3})\omega _{3}\omega _{1}&amp;amp;=M_{{2}}\\I_{3}{\dot  {\omega }}_{{3}}+(I_{2}-I_{1})\omega _{1}\omega _{2}&amp;amp;=M_{{3}}\end{aligned}&#xA;&lt;strong&gt;The information I have:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Angular Velocity of the asteroid from the center of mass given in radians/second [wX,wY,wZ].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current (Instantaneous) Position of the point on the asteroid at time(t)=0. This is with respect to the center of mass of the asteroid. Just a vector pointing away from the center of mass to the point I am wanting to hit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A rough approximation of &quot;t&quot;, such as 5.8 seconds. I am deriving the time (t) from the distance of the shooter to the center of mass of the asteroid. I may be able to get a better approximation of t if I can figure out the above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am developing a space shooter that works in 3D space. I am wanting the AI to be able to fire a projectile at a point on a tumbling asteroid so that when the projectile hits the asteroid, it's on that point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the asteroid is not rotating, it's a pretty simple kinematics equation with no differentials needed. However in some cases the asteroid is rotating around the X, Y, and Z axis. I am wanting to find a way to describe the motion of the asteroid and project where that point will be at a given time(t).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried using 4x4 rotation/translation matricies for the above. This works if the asteroid is rotating around a single axis (say the Y axis), but the second it is rotating around more than 1 axis it looks like the rotational matricies are off (almost looks like its the opposite direction of what it should be)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The Information I am wanting to get&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wanting to have a simple model where I can essentially plug in these values (angular velocity, current rotation, current position of the point, and time(t)) to get a point in space where that point will be at time(t).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am struggling to teach myself this method. I'm sure it is more simple than I am thinking it is, I guess I'm struggling to understand what numbers go where and how.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the initial values were (arbitrarily):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Angular Velocity = (1,-2,3) (in radians/second)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current Point Position (with respect to center of mass of asteroid) = (5,4,6)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time(t)=7 seconds&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I use this method to predict the location of the point at time (t) with respect to the asteroid's center of mass? Is it as simple as plugging in these values into the equations? Maybe that's where I'm overthinking this. As of now I am not accounting for angular acceleration, would it be exponentially more difficult if I did? (Say the asteroid had a weird thrusting booster on it)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much for any insight you might help me with,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chris&lt;/p&gt;&#xA;" OwnerUserId="27887" LastActivityDate="2019-02-24T01:57:31.180" Title="How can I apply Euler's Method to predict a point in time rotating around multiple axis'" Tags="|linear-algebra|matrix|computational-physics|runge-kutta|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29635" PostTypeId="1" AcceptedAnswerId="29640" CreationDate="2018-05-30T20:51:40.843" Score="4" ViewCount="138" Body="&lt;p&gt;I asked this question previously on the physics board (LINK: &lt;a href=&quot;https://physics.stackexchange.com/questions/404539/linear-elasticity-governing-equation&quot;&gt;https://physics.stackexchange.com/questions/404539/linear-elasticity-governing-equation&lt;/a&gt;), but I thought I'd post it here and see if anyone else can offer me another perspective since there seem to be multiple users on here familiar with modeling structural mechanics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transient linear elasticity is governed by the following hyperbolic PDE:&#xA;\begin{align}&#xA;\frac{\partial^2(\rho \boldsymbol{u})}{\partial t^2} - \nabla \cdot \boldsymbol{\sigma} = \boldsymbol{F}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation is hyperbolic and and has no dampening term, so it will never dampen and won't reach equilibrium, i.e., it doesn't have a steady state form. If this equation doesn't have a steady state form, then why is the following equation frequently referred to as the steady state (or equilibrium) equations? &#xA;\begin{align}&#xA;- \nabla \cdot \boldsymbol{\sigma} = \boldsymbol{F}&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These equations are commonly referred to, in journal/conference papers, by people and/or on the web, as the &quot;steady state&quot; or &quot;equilibrium&quot; equations, and I don't understand why. Note that when I refer to papers, I mean computational science papers. I'm not sure what experimentalists refer to the equations as as I have not read any papers dealing with structural experiments. &lt;/p&gt;&#xA;" OwnerUserId="27504" LastActivityDate="2018-05-31T19:40:09.857" Title="Why are the linear elasticity equations referred to as the &quot;equilibrium&quot; equations?" Tags="|solid-mechanics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29636" PostTypeId="1" AcceptedAnswerId="29641" CreationDate="2018-05-30T21:46:15.557" Score="2" ViewCount="244" Body="&lt;p&gt;I'm writing some code to solve problems in nonlinear elasticity using finite element methods.  I have been following Bathe's book but I am having trouble with some nagging details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is related to which configuration variables to use for calculating different quantities.  I'm working on static problems in the updated Lagrangian formulation, so for each &quot;time&quot; $t$ I have a loading coefficient $\lambda_t$, and a displacement $\sideset{^t}{}{U}$.  In chapter 6, Bathe derives a linearization of the principle of virtual work, resulting in the equation (6.103):&#xA;\begin{equation}&#xA;\left(\sideset{^t_{t\,}}{_\text{L}}{K}+\sideset{^t_{t\,}}{_\text{NL}}{K}\right)U=\sideset{^{t+\Delta t}}{}R-\sideset{^t_t}{}F&#xA;\end{equation}&#xA;where $\sideset{^t_{t\,}}{_\text{L}}{K}$ is the &quot;linear&quot; part of the incremental stiffness matrix in the configuration with displacement $\sideset{^t}{}{U}$, $\sideset{^t_{t\,}}{_\text{NL}}{K}$ is the &quot;nonlinear&quot; part, $U$ is the displacement increment, $\sideset{^{t+\Delta t}}{}R$ is the externally applied load at &quot;time&quot; $t+\Delta t$, and $\sideset{^t_t}{}F$ is the nodal point force resulting from the stress, at &quot;time&quot; $t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about actually implementing this as a system of nonlinear equations to be solved via Newton's method.  As I understand it, the idea is to consider a function $H(\sideset{^{t}}{}U)=\sideset{^{t+\Delta t}}{}R-\sideset{^t_t}{}F$, where $\sideset{^t_t}{}F$ is a function of $\sideset{^{t}}{}U$ via the stress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to find a displacement $\sideset{^{t+\Delta t}}{}U$ s.t. the unbalanced force vector is zero:&#xA;\begin{equation}&#xA;0=H(\sideset{^{t+\Delta t}}{}U)\approx H(\sideset{^t}{}U)+\nabla H(\sideset{^t}{}U)\left(\sideset{^{t+\Delta t}}{}U-\sideset{^t}{}U\right)=\sideset{^{t+\Delta t}}{}R-\sideset{^t_t}{}F - \left(\sideset{^t_{t\,}}{_\text{L}}{K}+\sideset{^t_{t\,}}{_\text{NL}}{K}\right)U&#xA;\end{equation}&#xA;so that $\left(\sideset{^t_{t\,}}{_\text{L}}{K}+\sideset{^t_{t\,}}{_\text{NL}}{K}\right)$ is minus the Jacobian of the unbalanced force vector $\sideset{^{t+\Delta t}}{}R-\sideset{^t_t}{}F$.  Is this interpretation correct?  &lt;strong&gt;Is the sum of the incremental stiffness matrices derived by Bathe exactly equal to minus the Jacobian of the unbalanced force vector, when all are evaluated in the same configuration?&lt;/strong&gt;  In implementing Newton's method, an iteration is introduced, so that we can write $U^{(k)}$ for the incremental displacement at the $k$th iteration.  How does this affect the terms in the system of equations?  At iteration $k$, are the integrals still calculated over the volume $\sideset{^t}{}V$, or over $\sideset{^{t+\Delta t}}{}V^{(k-1)}$?  What about the stress, and the stiffness matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that a full implementation of Newton's method, with no modifications, would involve calculating the unbalanced force vector AND the stiffness matrices in the most recently calculated configuration $\sideset{^{t+\Delta t}}{^{(k-1)}}U$:&#xA;\begin{equation}&#xA;0=H(\sideset{^{t+\Delta t}}{^{(k)}}U)\approx H(\sideset{^{t+\Delta t}}{^{(k-1)}}U)+\nabla H(\sideset{^{t+\Delta t}}{^{(k-1)}}U)\left(\sideset{^{t+\Delta t}}{^{(k)}}U-\sideset{^{t+\Delta t}}{^{(k-1)}}U\right)=\sideset{^{t+\Delta t}}{}R-\sideset{^{t+\Delta t}_{t+\Delta t}\,}{^{(k-1)}}F - \left(\sideset{^{t+\Delta t}_{t+\Delta t\,}}{^{(k-1)}_\text{L}}{K}+\sideset{^{t+\Delta t}_{t+\Delta t\,}}{^{(k-1)}_\text{NL}}{K}\right)U&#xA;\end{equation}&#xA;and solving the linear system for the incremental displacement $U$.  However, when I set things up this way, the Jacobian of $H$ at displacement $\sideset{^{t+\Delta t}}{^{(k-1)}}U$, as calculated by finite difference, is not equal to $-\left(\sideset{^{t+\Delta t}_{t+\Delta t\,}}{^{(k-1)}_\text{L}}{K}+\sideset{^{t+\Delta t}_{t+\Delta t\,}}{^{(k-1)}_\text{NL}}{K}\right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One clue is that the coded Jacobian matches the finite difference Jacobian almost exactly, for all iterations at $t=0$.  My first instinct was to look for errors coming from the &quot;nonlinear&quot; stiffness matrix $K_{\text{NL}}$, but that matrix has entries which are far too small to explain the disparity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am fairly confident that I've implemented the calculation properly, provided my understanding is correct.  I can provide details if anyone is willing to help me check them, and help me debug the many steps in this calculation - Green-Lagrange strain, deformation gradient, second Piola-Kirchhoff stress, Cauchy stress, etc.  Suggestions for sanity checks are much appreciated as well.  Thanks!&lt;/p&gt;&#xA;" OwnerUserId="25235" LastEditorUserId="25235" LastEditDate="2018-05-30T23:07:26.280" LastActivityDate="2018-05-31T13:13:20.467" Title="Help debugging finite element solution in nonlinear elasticity" Tags="|finite-element|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29637" PostTypeId="1" CreationDate="2018-05-31T00:26:30.647" Score="2" ViewCount="55" Body="&lt;p&gt;I am trying to compute the CDF of a Multivariate Cumulative Normal Distribution in the 1000th dimension (I have a 1000 vectors and their covariance matrix). I haven't been able to find a fast way to compute this, numpy and scipy all take take too long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any libraries (platform and language doesn't matter) that can compute this in a reasonable amount of time? If not, are there any numerical methods that can achieve this?&lt;/p&gt;&#xA;" OwnerUserId="27888" LastActivityDate="2018-05-31T00:26:30.647" Title="Computing Multivariate Cumulative Normal Distribution" Tags="|numerics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29640" PostTypeId="2" ParentId="29635" CreationDate="2018-05-31T09:10:18.260" Score="7" Body="&lt;p&gt;The general idea comes from this example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine a rigid pendulum of length $l_0$ hanging a mass $m_0$. The equation of motion can be derived from a minimum principle (Lagrangian).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denoting with $\theta$ to the angle which the pendulum forms with respect to the vertical direction we have:&#xA;$$\mathcal{L}(\theta) = \int_{t_1}^{t_2}\left[\frac{1}{2}m_0l_0^2\dot{\theta}^2-m_0gl_0(1-\cos{(\theta))}\right]dt\tag{1}$$&#xA;The first term is the kinetic energy while the second is the potential energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is obvious that this dynamic system (derived from $(1)$) expressed by the dynamic equation:&#xA;$$\frac{d^2\theta}{dt^2}+\frac{g}{l_0}\sin(\theta)=0\tag{2}$$&#xA;will never reach the &quot;equilibrium&quot; (like your hyperbolic equations). But what is remarkable, is that certain special positions provide the system with sufficient &quot;stability&quot; to remain steady. These special positions are defined by the extrema of the potential $V(\theta)=m_0gl_0(1-\cos({\theta}))$:&#xA;$$V'(\theta) = m_0gl_0\sin(\theta) = 0$$&#xA;being $\theta_1=0$ or $\theta_2=\pi$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\theta_1$ is a minimum the equilibrium will be stable (small perturbation in position leads to small responses), while $\theta_2$ offers an unstable equilibrium because it is a maximum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These points could have been obtained just from $(2)$ setting $d_t^2\theta=0$ and solving for equilibrium. As a final remark, if we set the initial conditions for $(2)$ either $\theta(0) = \theta_1$ or $\theta(0)=\theta_2$ the system no longer moves (check it!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This frame can be exactly applied to our case&lt;/strong&gt;, in which we have the equation (now a PDE):&#xA;$$\rho\partial_t^2\vec{u} =\textrm{div}(\sigma)+\vec{f}\tag{3}$$&#xA;in which the positions $\vec{u}_0$ given by solving &quot;the equilibrium equation&quot;:&#xA;$$-\textrm{div}(\sigma(\vec{u}_0))=\vec{f}$$&#xA;leads to a system that remains steady for any time if we use them as initial conditions $\vec{u}(0) = \vec{u}_0$ for the transient problem $(3)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is the explanation (in my opinion) to the question of why these equations (without time derivatives) are named &quot;equilibrium equations&quot;.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-05-31T19:40:09.857" LastActivityDate="2018-05-31T19:40:09.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29641" PostTypeId="2" ParentId="29636" CreationDate="2018-05-31T13:13:20.467" Score="4" Body="&lt;p&gt;I find Bathe's notation and presentation confusing. First, it is hard&#xA;to keep track of all the superscripts and subscripts. And second, writing&#xA;the equations in an incremental form introduces a confusion between load&#xA;increments and Newton iteration increments. &#xA;Here is the&#xA;way I like to think about the nonlinear FE equations that seems simpler to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming we want to solve for the displacements, $U$, of a body under some set of loads&#xA;we start with Bathe's equation 6.2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;^tR - {^tF} = 0&#xA;$$&#xA;I'll drop the superscript $t$ for simplicity. Note, also, that I'm defining&#xA;$U$ as the total displacements of the body, NOT a displacement increment.&#xA;We assume that $F$, but not $R$, is a function of the displacements so the&#xA;equation is nonlinear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to solve this equation by Newton iteration as follows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;K^{(i-1)}\Delta U^{(i)} = R - F^{(i-1)}&#xA;$$&#xA;$$&#xA;U^{(i)} = U^{(i-1)} + \Delta U^{(i)}&#xA;$$&#xA;where $i$ is the Newton iteration and&#xA;$$&#xA;K = \frac{\partial F}{\partial U}&#xA;$$&#xA;This is Bathe's equation 6.8.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One more point concerns the &quot;updated&quot; versus total Lagrangian formulations.&#xA;Bathe presents the equations using both formulations but states that, theoretically&#xA;they are equivalent but that the updated Lagrangian may, in some cases, be more &quot;numerically effective.&quot; But, as far as I am aware, he gives no specific examples to show this. For simplicity, I suggest using the total Lagrangian formulation where the reference configuration is always the initial configuration.  That is, integrals over the element volume required for evaluating the internal forces and tangent stiffness matrices are always the initial, undeformed volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your test of making sure the tangent stiffness matrix calculated in closed form is equal to the one calculated using finite differences of the internal force vector is a good one.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2018-05-31T13:13:20.467" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="29642" PostTypeId="1" AcceptedAnswerId="29646" CreationDate="2018-05-31T17:19:05.133" Score="7" ViewCount="1509" Body="&lt;p&gt;I am new to numerical linear algebra, so i came to know that condition number in 2-norm case will be ratio of largest to smallest singular value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another concept &quot;Nearness To Singularity&quot; is measured using this number being large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But consider the case where singular values are close to 0 but smallest and largest singular values are also close to each other. In this case condition number will not be large enough to indicate that matrix is very near to being singular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is it reliable always to use condition number's magnitude as a measure of nearness to singularity.&lt;/p&gt;&#xA;" OwnerUserId="27893" LastEditorUserId="17294" LastEditDate="2018-05-31T20:50:19.973" LastActivityDate="2018-06-03T18:48:52.203" Title="Is large condition number good measure of nearness to singularity for a matrix?" Tags="|linear-algebra|matrix|condition-number|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29643" PostTypeId="2" ParentId="29642" CreationDate="2018-05-31T19:34:08.510" Score="1" Body="&lt;p&gt;For a normal matrix $A$, the value of the smallest eigenvalue tells us whether $A$ is singular or not:&#xA;Imagine that the eigenvectors of $A$ can form a complete set that span the space of the columns of $A$, it can be decomposed:&#xA;$$A=S^{-1}\Lambda S$$&#xA;Where $S$ are the eigenvectors gathered in a matrix and $\Lambda=\textrm{diag}(\lambda_1,\lambda_2,...,\lambda_n)$ their corresponding eigenvalues with $\lambda_1\leq \lambda_2\leq...\leq\lambda_n$&#xA;The smaller $\lambda_1$ the more singular $A$, because $A^{-1}=S^{-1}\Lambda^{-1}S$ with $\Lambda^{-1}=\textrm{diag}(\lambda_1^{-1},\lambda_2^{-1},...,\lambda_n^{-1})$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the accuracy when solving a system of equations, for example: $Ax=f$, the condition number of the matrix $\kappa(A)$ measures the error in the solution $x$ when $f$ is perturbed:&#xA;$$\delta x=A^{-1}\delta f$$&#xA;whose upper threshold coincides when $\delta f$ is in the direction of the eigenvector whose eigenvalue is the largest: &#xA;$$|\delta x|=|A^{-1}\delta f|\leq \frac{1}{\lambda_1}|\delta f|$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If now we divide the last equation by $|x| $ to obtain the relative error we find:&#xA;$$\frac{|\delta x|}{| x|}\leq\frac{1}{\lambda_1}\frac{|\delta f|}{|x|}$$&#xA;and the lower threshold for $|x|$ is obtained from:&#xA;$|f|=|Ax|\leq \lambda_n|x|$. Introduced in the last equation:&#xA;$$\frac{|\delta x|}{| x|}\leq\kappa(A)\frac{|\delta f|}{|f|}$$&#xA;Where $\kappa(A) = \lambda_n/\lambda_1$, tells us the upper threshold of the error in $x$ when $f$ is perturbed.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-06-03T18:48:52.203" LastActivityDate="2018-06-03T18:48:52.203" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29644" PostTypeId="1" CreationDate="2018-06-01T00:16:51.127" Score="8" ViewCount="136" Body="&lt;p&gt;I would like to find the smallest real root of a 1-D real-valued function $f(x)$ on the domain $x\in [0,\infty)$. In this problem, I can make the following guarantees on $f$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$f$ does have a root at some finite, positive $x$.&lt;/li&gt;&#xA;&lt;li&gt;$f$ is differentiable at all finite, positive $x$ and $f'(x)$ is either known or can be approximated accurately.&lt;/li&gt;&#xA;&lt;li&gt;$f$ may have multiple roots (and usually does).&lt;/li&gt;&#xA;&lt;li&gt;At its smallest root, $f$ changes sign from positive to negative.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Further, I can estimate&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;an approximate lower bound on the spacing between roots, call it $\Delta$.&lt;/li&gt;&#xA;&lt;li&gt;an upper bound on the location of the smallest root, call it $x_{\max}$. If the root is beyond this point, I can compute its approximate location analytically.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My current algorithm for this problem is to bracket the root via a naive linear search. Starting from $x=0$, I march along in increments of $\Delta$ until $f(x)$ and $f(x+\Delta)$ differ in sign. With the root bracketed, I defer to a standard root-finding algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the knowledge I have of $f$ and the estimates I am willing to make about its root structure, can I bracket the smallest root more quickly than a simple linear search?&lt;/p&gt;&#xA;" OwnerUserId="14469" LastEditorUserId="1047" LastEditDate="2018-06-01T13:44:19.810" LastActivityDate="2018-06-01T13:44:19.810" Title="Finding the smallest root of a function on $[0, \infty)$" Tags="|algorithms|roots|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="29646" PostTypeId="2" ParentId="29642" CreationDate="2018-06-01T15:59:09.340" Score="6" Body="&lt;p&gt;The condition number measures the &lt;em&gt;relative&lt;/em&gt; distance from singularity of a matrix $A$: that is,&#xA;$$&#xA;\min_{\text{$X$ is singular}} \frac{\|A-X\|}{\|A\|} = \frac{1}{\kappa(A)}&#xA;$$&#xA;(the norm here is the Euclidean / induced / spectral norm --- i.e., $\|A\|=\sigma_1(A)$). This property follows from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Low-rank_approximation#Basic_low-rank_approximation_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eckart-Young theorem&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your example with a small $\|A\|$ highlights the fact that &lt;em&gt;relative&lt;/em&gt; and &lt;em&gt;absolute&lt;/em&gt; distance aren't the same thing. Often, measuring relative errors and distances makes more sense in numerical analysis, because it fits better with how errors appear in the data (due to inexact arithmetic, but not only).&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2018-06-02T07:36:14.673" LastActivityDate="2018-06-02T07:36:14.673" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29648" PostTypeId="1" AcceptedAnswerId="29649" CreationDate="2018-06-02T06:53:54.007" Score="5" ViewCount="146" Body="&lt;p&gt;Given indices $i,j$ s.t. $0\leq i \leq j &amp;lt;n$, the function $f(i,j)=i+j(j+1)/2$ maps 2d indices to linear indices in column major order. What is the fastest way to invert this function? My first inclination is binary search on a list of the first $n$ triangle numbers but is there a faster (perhaps $O(1)$, other than a lookup table with quadratic storage complexity) solution?&lt;/p&gt;&#xA;" OwnerUserId="23072" LastEditorUserId="23072" LastEditDate="2018-06-02T07:01:42.223" LastActivityDate="2018-06-02T11:32:05.237" Title="Transform from linear index of a packed triangular matrix to dense indices" Tags="|linear-algebra|matrix|algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29649" PostTypeId="2" ParentId="29648" CreationDate="2018-06-02T11:32:05.237" Score="4" Body="&lt;p&gt;Given the linear index $f$, you can invert&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f(i,j) = i+\frac{j(j+1)}{2}$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;by first calculating the $j$-index via&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$j = \left\lfloor \frac{-1+\sqrt{1+8f}}{2} \right\rfloor$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\lfloor \cdot \rfloor$ is the floor function. &#xA;Then obviously $i$ follows trivially as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$i = f-\frac{j(j+1)}{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula for $j$ is based on solving the quadratic equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\hat{j}^2+\hat{j}-2f=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\hat{j}$ is the real counterpart to the integer $j=\lfloor \hat{j} \rfloor$. The choice of branch cut is obvious.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if $f$ is a triangle number, $\hat{j}$ is an integer. &lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2018-06-02T11:32:05.237" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29650" PostTypeId="1" AcceptedAnswerId="29651" CreationDate="2018-06-02T22:27:35.283" Score="2" ViewCount="177" Body="&lt;p&gt;I'm trying to solve physical problem in quantum mechanics of helium atoms, the solution require numerical integration over 2 variables. However when i'm trying to run the next code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;e = 1.6e-19;&#xA;eps0 = 8.85e-12;&#xA;a0 = 0.53e-10;&#xA;c = e^2/(4*pi*eps0);&#xA;psi1 =@(r) ((2^1.5)/(sqrt(pi)*a0^1.5))*exp(-2*r/a0);&#xA;psi2 =@(r) ((2^1.5)/(4*sqrt(2*pi)*a0^1.5))*(2-r/a0)*exp(-r/a0);&#xA;I =@(r1,r2) c*(psi2(r1).*psi1(r2).*(1./abs(r1-r2)).*psi2(r1).*psi1(r2));&#xA;J =@(r1,r2) c*(psi2(r1).*psi1(r2).*(1./abs(r1-r2)).*psi1(r1).*psi2et(r2));&#xA;integral2(I,0,inf,0,inf)&#xA;integral2(J,0,inf,0,inf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;matlab saying&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error using  * &#xA;Inner matrix dimensions must agree.&#xA;&#xA;Error in qmex7&amp;gt;@(r)((2^1.5)/(4*sqrt(2*pi)*a0^1.5))*(2-r/a0)*exp(-r/a0)&#xA;&#xA;Error in qmex7&amp;gt;@(r1,r2)c*(psi2(r1).*psi1(r2).*(1./abs(r1-r2)).*psi2(r1).*psi1(r2))&#xA;&#xA;Error in integral2Calc&amp;gt;@(y)fun(xi*ones(size(y)),y) (line 18)&#xA;    @(y)fun(xi*ones(size(y)),y),y1i,y2i,opstruct.integralOptions), ...&#xA;&#xA;Error in integralCalc/iterateScalarValued (line 314)&#xA;                fx = FUN(t);&#xA;&#xA;Error in integralCalc/vadapt (line 132)&#xA;            [q,errbnd] = iterateScalarValued(u,tinterval,pathlen);&#xA;&#xA;Error in integralCalc (line 83)&#xA;        [q,errbnd] = vadapt(@AToInfInvTransform,interval);&#xA;&#xA;Error in integral2Calc&amp;gt;@(xi,y1i,y2i)integralCalc(@(y)fun(xi*ones(size(y)),y),y1i,y2i,opstruct.integralOptions)&#xA;&#xA;Error in integral2Calc&amp;gt;@(x)arrayfun(@(xi,y1i,y2i)integralCalc(@(y)fun(xi*ones(size(y)),y),y1i,y2i,opstruct.integralOptions),x,ymin(x),ymax(x)) (line 17)&#xA;innerintegral = @(x)arrayfun(@(xi,y1i,y2i)integralCalc( ...&#xA;&#xA;Error in integralCalc/iterateScalarValued (line 314)&#xA;                fx = FUN(t);&#xA;&#xA;Error in integralCalc/vadapt (line 132)&#xA;            [q,errbnd] = iterateScalarValued(u,tinterval,pathlen);&#xA;&#xA;Error in integralCalc (line 83)&#xA;        [q,errbnd] = vadapt(@AToInfInvTransform,interval);&#xA;&#xA;Error in integral2Calc&amp;gt;integral2i (line 20)&#xA;[q,errbnd] = integralCalc(innerintegral,xmin,xmax,opstruct.integralOptions);&#xA;&#xA;Error in integral2Calc (line 7)&#xA;    [q,errbnd] = integral2i(fun,xmin,xmax,ymin,ymax,optionstruct);&#xA;&#xA;Error in integral2 (line 106)&#xA;    Q = integral2Calc(fun,xmin,xmax,yminfun,ymaxfun,opstruct);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;any idea why?&lt;/p&gt;&#xA;" OwnerUserId="27468" LastActivityDate="2018-06-03T00:10:40.387" Title="Two variables integration matlab" Tags="|computational-physics|integration|quantum-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29651" PostTypeId="2" ParentId="29650" CreationDate="2018-06-03T00:10:40.387" Score="2" Body="&lt;p&gt;The error message is because you forgot the &lt;code&gt;.&lt;/code&gt; before &lt;code&gt;*&lt;/code&gt; before &lt;code&gt;exp&lt;/code&gt; in &lt;code&gt;psi2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you fix that, MATLAB will be none too happy about the singularity you get dividing by &lt;code&gt;abs(r1-r2)&lt;/code&gt; in &lt;code&gt;I&lt;/code&gt; and &lt;code&gt;J&lt;/code&gt;. If &lt;code&gt;(1./abs(r1-r2))&lt;/code&gt; is changed, for instance, to &lt;code&gt;(1./(abs(r1-r2) + 1e-20))&lt;/code&gt; , MATLAB will produce an answer of 0 for &lt;code&gt;integral2(I,0,inf,0,inf)&lt;/code&gt;. I'll let you figure out how to fix that properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have horrible scaling to the max, by having &lt;code&gt;a0 = 0.53e-10&lt;/code&gt; in the denominator of the argument of &lt;code&gt;exp&lt;/code&gt;. Hence, for instance, &lt;code&gt;psi1(1e-8) = 5.39e-149&lt;/code&gt;, and only gets smaller as its argument increases. And this despite a multiplicative factor &lt;code&gt;(2^1.5)/(sqrt(pi)*a0^1.5) = 4.14e15&lt;/code&gt;, which, though, gets overwhelmed by the exponential of a large negative number. All the real action takes place at even smaller argument values than &lt;code&gt;1e-8&lt;/code&gt;. For instance, &lt;code&gt;psi1(1e-10) = 9.60e+13,  psi1(1e-9) = 0.17&lt;/code&gt;.  So I think you have some reformulation needed to get this to the point where the integral can be evaluated in a numerically sound way.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2018-06-03T00:10:40.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29652" PostTypeId="1" AcceptedAnswerId="29653" CreationDate="2018-06-03T14:49:50.190" Score="2" ViewCount="1471" Body="&lt;p&gt;I'd like to know if is possible to make a Von Neumann's stability analysis on an system of coupled equations, featuring quadratics:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{aligned}&#xA;\frac{\partial u_1}{\partial t}&amp;amp;=D_1\Delta u_1 - s_1\left(\left(\frac{\nabla u}{(k+a)^2}-2u\frac{\nabla a}{(k+a)^3}\right)\nabla(a) + \frac{u}{(k+a)^2}\left(\Delta(a) + 2\frac{\partial^2a}{\partial x \partial y}\right)\right) &#xA;\\&amp;amp;\quad+ r_1u_1 \left(\frac{n^2}{\beta + n^2}-u_1\right)\\&#xA;\frac{\partial a}{\partial t} &amp;amp;= D_a\Delta a + c_1n\frac{u_1^2}{\alpha_1 + u_1^2} - \gamma a\\&#xA;\frac{\partial n}{\partial t}&amp;amp;= D_s\Delta n - l_1u_1\frac{n^2}{\beta + n^2}&#xA;\end{aligned}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea was to make different analysis for each of terms of the equations. &#xA;There are no problems for most of the terms, but I don't know how to deal with the coupled ones and the nonlinear one. Can someone give me hints or link me a paper dealing with these?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the numerical scheme :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;    &amp;amp;\frac{U_{i,j}^{k+1}-U_{i,j}^k}{\Delta t}= &#xA;    D_1 \frac{U_{i+1,j}^k+U_{i-1,j}^k+U_{i,j+1}^k+U_{i,j-1}^k-4U_{i,j}^k}{{\Delta z}^2} + r_1 U_{i,j}^k\bigg(\frac{{N_{i,j}^k}^2}{\beta + {N_{i,j}^k}^2} - U_{i,j}^k\bigg)\\&#xA;    &amp;amp;\qquad\qquad\qquad-s_1\bigg( \frac{U_{i+1,j}^k-U_{i-1,j}^k+U_{i,j+1}^k-U_{i,j-1}^k}{2\big(k_1+A_{i,j}^k\big)^2\Delta z} &#xA;    -2U_{i,j}^k\frac{A_{i+1,j}-A_{i-1,j}^k+A_{i,j+1}^k-A_{i,j-1}^k}{2\Delta z(k_1+A_{i,j}^k)^3} \bigg)\\&#xA;    &amp;amp;\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\times\frac{A_{i+1,j}^k-A_{i-1,j}^k+A_{i,j+1}^k-A_{i,j-1}^k}{2\Delta z}\\&#xA;    &amp;amp;+ \frac{s_1U_{i,j}^k}{(k_1+{A_{i,j}^k}^2)}\times \bigg( \frac{A_{i+1,j}^k+A_{i-1,j}^k+A_{i,j+1}^k+A_{i,j-1}^k-4A_{i,j}^k}{{\Delta z}^2} + \frac{A_{i+1,j+1}^k-A_{i+1,j-1}^k-A_{i-1,j+1}^k+A_{i+1,j+1}^k}{2{\Delta z}^2}\bigg) \\&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;    &amp;amp;\frac{A_{i,j}^{k+1}-A_{i,j}^k}{\Delta t}= D_a \frac{A_{i+1,j}^k+A_{i-1,j}^k+A_{i,j+1}^k+A_{i,j-1}^k-4A_{i,j}^k}{{\Delta z}^2}-\gamma A_{i,j}^k+c_1N_{i,j}^k\frac{{U_{i,j}^k}^2}{\alpha_1+{U_{i,j}^k}^2}+c_2N_{i,j}^k\frac{{V_{i,j}^k}^2}{\alpha_2+{V_{i,j}^k}^2}&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;    &amp;amp;\qquad\frac{N_{i,j}^{k+1}-N_{i,j}^k}{\Delta t}= D_a \frac{N_{i+1,j}^k+N_{i-1,j}^k+N_{i,j+1}^k+N_{i,j-1}^k-4N_{i,j}^k}{{\Delta z}^2}-\frac{{N_{i,j}^k}^2}{\beta + {N_{i,j}^k}^2}\big(l_1{U_{i,j}^k}^2+l_2{V_{i,j}^k}^2\big)\\&#xA;\end{align*}&lt;/p&gt;&#xA;" OwnerUserId="27904" LastEditorUserId="27904" LastEditDate="2018-06-03T16:11:22.430" LastActivityDate="2018-06-03T18:40:59.990" Title="Von Neumann's stability analysis on non linear and coupled equations" Tags="|pde|stability|numerics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29653" PostTypeId="2" ParentId="29652" CreationDate="2018-06-03T18:40:59.990" Score="2" Body="&lt;p&gt;Von Neumann stability analysis does not only apply to FD schemes... it can always be applied to finite volume, finite element... or whatever method you use. The only requirement that must be fulfilled is the linearity of the equations and the orthogonality/uniformity of the mesh to apply Fourier methods for the following linear discretised system:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$d_t\vec{u}+A\vec{u}=\vec{f}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where in your case $\vec{u}=(\vec{u}_1,\vec{a},\vec{n})^{T}$ and $\vec{f}=\vec{0}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a nonlinear system, i.e. $A=A(u)$, you can always linearise it $A\approx A({\vec{u}_0})$ and see what occurs in the vicinity of a selected solution $\vec{u}_0$, usually the one that makes the system stationary:&#xA;$$A(\vec{u}_0)\vec{u}_0=\vec{f}$$&#xA;and perform the stability analysis.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-06-03T18:40:59.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29655" PostTypeId="1" AcceptedAnswerId="29664" CreationDate="2018-06-03T20:35:37.010" Score="5" ViewCount="154" Body="&lt;p&gt;In $\mathbb{R}^n$, I would like to solve a Poisson equation (given $f$, solve for $u$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2 u = f$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;assuming Neumann boundary condition (i.e. $\partial u = 0$ at boundaries).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solved it in spectral domain. Using the 3x3 Laplacian stencil (so Neumann boundary = symmetric boundary), DCT &amp;amp; IDCT will be good. So the direct solver is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u_{\text{ours}} = \mathcal{IDCT}\{ \frac{\mathcal{DCT}\{ f \}}{\mathcal{DCT\{\nabla^2\} }} \}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a test MATLAB code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc;close all;&#xA;% Solve for u from f: nabla^2 u = f &#xA;% (under Neumann boundary condition)&#xA;&#xA;% for reproduction&#xA;rng(0);&#xA;&#xA;% generate u&#xA;Udim = [256 256];&#xA;coeff = 0.2;&#xA;[~, X] = meshgrid(coeff*(1:Udim(2)),coeff*(1:Udim(1)));&#xA;u = sin(X);&#xA;&#xA;% generate f&#xA;lambda = 2e1;&#xA;f = lambda*imfilter(u, fspecial('laplacian',0), 'symmetric');&#xA;&#xA;% add Gaussian noise to f&#xA;noise_level = 1e-2;&#xA;f = f + noise_level*randn(size(f));&#xA;&#xA;&#xA;%%% Now solve for u from noisy f&#xA;% generate the DCT of f&#xA;numerator = dct2(f);&#xA;&#xA;% generate the DCT denominator&#xA;[H,W] = size(f);&#xA;[x,y] = meshgrid(0:W-1,0:H-1);&#xA;denominator = 2*cos(pi*x/W) + 2*cos(pi*y/H) - 4;&#xA;denominator(1) = 1;     % set DC to be 1; does not matter&#xA;&#xA;% inversion in DCT domain&#xA;numerator = idct2( numerator./( lambda*denominator) );&#xA;&#xA;% zero-mean normalization&#xA;u = u - min(u(:));&#xA;u_ours = numerator - min(numerator(:));&#xA;&#xA;% show results&#xA;figure;&#xA;subplot(131);imshow(u, [],'i','f');colormap(jet);title('u (real)');&#xA;subplot(132);imshow(u_ours, [],'i','f');colormap(jet);title('u (ours)');&#xA;subplot(133);imshow(u - u_ours, [],'i','f');&#xA;colormap(jet);   title('error');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Issues&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The denominator $\mathrm{DCT}\{\nabla^2\}$ is ill-condition. Below figure shows that near the origin, the values are close to zero.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/eWG8j.png&quot; width=&quot;400&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Increasing noise level (the &lt;code&gt;noise_level&lt;/code&gt; variable) leads to reconstruction artifacts. You may notice the artifacts are of low frequency: i.e. the consequence of almost-zero-division.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/o12NC.png&quot; width=&quot;335&quot; /&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/mFtJh.png&quot; width=&quot;320&quot; /&gt;&#xA;&lt;em&gt;Figure: Left: noise_level = 1e-2; Right: noise_level = 1e-1&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/WlCEJ.png&quot; width=&quot;321&quot; /&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/x7nxP.png&quot; width=&quot;334&quot; /&gt;&#xA;&lt;em&gt;Figure: Error plots&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why this happens? Is it because of the ill conditioning of the spectral method?&lt;/li&gt;&#xA;&lt;li&gt;If so, how to avoid it? What are other alternative methods for this issue?&lt;/li&gt;&#xA;&lt;li&gt;Any possible regularization suggestions?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="18751" LastEditorUserId="18751" LastEditDate="2018-06-04T11:40:22.593" LastActivityDate="2018-08-03T23:00:43.767" Title="Numerical error of a spectral-domain Poisson solver" Tags="|poisson|inverse-problem|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29656" PostTypeId="1" AcceptedAnswerId="29678" CreationDate="2018-06-04T05:30:38.987" Score="2" ViewCount="123" Body="&lt;p&gt;There is an article describing how to detect a pupil from an eye photo.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.3389/fneng.2015.00004&quot; rel=&quot;nofollow noreferrer&quot;&gt;A.-H. Javadi, Z. Hakimi, M. Barati, V. Walsh, and L. Tcheang, &quot;SET: a pupil detection method using sinusoidal approximation,&quot; &lt;em&gt;Front. Neuroeng.&lt;/em&gt;,  vol. 8, pp. 1–10, Apr. 2015&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;On the 4th step of pupil detection they use &#xA;&lt;em&gt;Decomposition of each segment into sinusoidal components method&lt;/em&gt; that I do not understand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They say that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We fit Equation 1 to the extracted points along the border to extract the free parameters $\omega$, $(h,k)$, $a$ and $b$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But I do not understand how exactly they came from polygon segments to the ellipse free parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use some obvious computational algorithms for these step to get them? &lt;/p&gt;&#xA;" OwnerUserId="27908" LastEditorUserId="20688" LastEditDate="2018-06-04T10:48:40.870" LastActivityDate="2018-06-06T01:55:56.450" Title="Polygon approximation with a circle" Tags="|computational-geometry|image-processing|computer-vision|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29658" PostTypeId="1" CreationDate="2018-06-04T14:04:07.997" Score="2" ViewCount="55" Body="&lt;p&gt;Primal-dual interior point methods for linear programming require interior primal and dual starting points. I am looking for a good reference containing a description for modifying a given linear program (without solving an auxiliary LP and without doubling the number of variables or constraints) to one with a primal and dual interior point. The linear program is given in the form  $\min c^Tx$ s.t. $Ax\ge b$ (or $Ax=b$, $x\ge 0$) and may possibly be primal or dual infeasible.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastEditorUserId="1117" LastEditDate="2018-06-05T09:02:43.207" LastActivityDate="2018-06-05T09:02:43.207" Title="Transforming an arbitrary linear program into one with an interior point" Tags="|linear-programming|interior-point-method|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29659" PostTypeId="2" ParentId="29634" CreationDate="2018-06-04T15:43:03.707" Score="3" Body="&lt;p&gt;This came out a bit long, so first you may want to scroll down to&#xA;the algorithm, skipping the explanations and the justifications. I&#xA;have written a discrete dynamical model of the case when the&#xA;asteroid rotates freely in space, twisting and turning, with&#xA;precessing angular velocity (so there is angular acceleration),&#xA;without any thrusting boosters. As you will see, the dynamics is non-trivial and interesting enough. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can suggest a sketch but you may have to try to figure out the&#xA;details. I will use standard &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; dimensional vectors, orthogonal&#xA;and skew symmetric matrices but one can do the absolutely the same&#xA;thing using quaternions. I am not sure which one is&#xA;computationally cheaper or more convenient but maybe there is not&#xA;that much difference. Ether way, the philosophy is always the same&#xA;and conceptually everything looks the same regardless of the usage&#xA;of quaternions or &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;D vectors and matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will be working a lot with the set of all orthogonal matrices&#xA;(also known as the orthogonal group) &lt;span class=&quot;math-container&quot;&gt;$$\text{SO}(3) = \Big\{ \, U&#xA;\in \mathbb{R}^{3 \times 3} \,\, \big{|} \,\, U\, U^T = U^T U = I&#xA;\, \Big\}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$U^T$&lt;/span&gt; is the transpose matrix of &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;.&#xA;Furthermore, we will be working with the set of all skew-symmetric&#xA;matrices (also known as the Lie algebra of &lt;span class=&quot;math-container&quot;&gt;$\text{SO}(3)$&lt;/span&gt;)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\text{so}(3) = \Big\{  \, A \in \mathbb{R}^{3 \times 3} \,\, \big{|}&#xA;\,\, A^T = -\, A \, \Big\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is skew-symmetric if and only if&#xA;&lt;span class=&quot;math-container&quot;&gt;$$A = \begin{pmatrix} \, 0  &amp;amp;  z &amp;amp; -y \\&#xA;                        -z  &amp;amp;  0 &amp;amp;  x \\&#xA;                         y  &amp;amp; -x &amp;amp;  0 \, \end{pmatrix}$$&lt;/span&gt;&#xA;Thus, for the sake of clarity, I introduce two functions:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\overrightarrow{V} = \text{vector}(V)$$&lt;/span&gt; which turns a skew-symmetric matrix&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V = \begin{pmatrix} \, 0  &amp;amp;  V_3 &amp;amp; -V_2 \\&#xA;                          -V_3  &amp;amp;   0  &amp;amp;  V_1 \\&#xA;                           V_2  &amp;amp; -V_1 &amp;amp;  0 \, \end{pmatrix}$$&lt;/span&gt;&#xA;                          into the vector column&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\overrightarrow{V} = \begin{pmatrix} V_1 \\&#xA;                             V_2 \\&#xA;                             V_3 \end{pmatrix} $$&lt;/span&gt;&#xA;and its inverse function&#xA;&lt;span class=&quot;math-container&quot;&gt;$$V = \text{Matrix}(\overrightarrow{V})$$&lt;/span&gt;&#xA;which turns a vector column &lt;span class=&quot;math-container&quot;&gt;$\overrightarrow{V}$&lt;/span&gt; into the&#xA;corresponding skew-symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; with. We also will use the&#xA;notation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$|V| = |\overrightarrow{V}| = \sqrt{V_1^2 + V_2^2 + V_3^2}$$&lt;/span&gt; which is the&#xA;length of the vector &lt;span class=&quot;math-container&quot;&gt;$\overrightarrow{V}$&lt;/span&gt;, and we can also think&#xA;of it as the magnitude &lt;span class=&quot;math-container&quot;&gt;$|V|$&lt;/span&gt; of the skew-symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The shape of the asteroid and the distribution of its mass is&#xA;encoded in a &lt;span class=&quot;math-container&quot;&gt;$3 \times 3$&lt;/span&gt; matrix &lt;span class=&quot;math-container&quot;&gt;${\bf \mathcal{I}}$&lt;/span&gt;, called the&#xA;inertia tensor of the asteroid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you have the coordinate system &lt;span class=&quot;math-container&quot;&gt;$S = O\,&#xA;\vec{e}_1\,\vec{e}_2\,\vec{e}_3$&lt;/span&gt; fixed in space, according to&#xA;which you determine the positions of the point &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; that fires upon&#xA;the asteroid and the position of the center of mass of the&#xA;asteroid &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Furthermore, you have another coordinate system&#xA;&lt;span class=&quot;math-container&quot;&gt;$S_A=A\, \vec{E}_1\,\vec{E}_2\,\vec{E}_3$&lt;/span&gt;, firmly attached to the&#xA;asteroid. Thus, the coordinate system &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt; rotates together with&#xA;the asteroid. Despite the fact that the system &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt; is&#xA;non-inertial, it is the right choice of system with respect to&#xA;which to describe the dynamics of the motion because in &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt; the&#xA;inertia tensor is constant, i.e. doesn't change with time, while&#xA;in &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; it does, making things complicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the idea is to study the (discrete) time evolution (i.e.&#xA;evolution in small time-increments) of the quantities of motion in&#xA;&lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt; and then transform them into the system &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$\vec{X}$&lt;/span&gt; is&#xA;the position vector of a point on the asteroid with respect to&#xA;system &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt;, the position vector &lt;span class=&quot;math-container&quot;&gt;$\vec{x}(n)$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; of the&#xA;same point with respect to &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; can be calculate by applying the&#xA;following transformation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{x}(n) = g(n) \,\vec{X} + {a}(n)$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n=0, 1, 2, 3, ...$&lt;/span&gt;&#xA;is the discrete time, &lt;span class=&quot;math-container&quot;&gt;$g(n) \in \text{SO}(3)$&lt;/span&gt; is a time-dependent&#xA;orthogonal matrix, representing rotation at time &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, and&#xA;&lt;span class=&quot;math-container&quot;&gt;$\vec{a}(n)$&lt;/span&gt; is the position of the center of mass &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; of the&#xA;asteroid at time &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. Furthermore, let &lt;span class=&quot;math-container&quot;&gt;$\vec{v}$&lt;/span&gt; be a fixed free&#xA;vector with respect to &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; (free vector is not a position vector).&#xA;Then the same free vector, observed in &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt; will change with time&#xA;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, so in &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt; vector &lt;span class=&quot;math-container&quot;&gt;$\vec{v}$&lt;/span&gt; will be &lt;span class=&quot;math-container&quot;&gt;$\vec{V}(n)$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;&#xA;and the two are connected by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{v} = g(n) \, \vec{V}(n)$$&lt;/span&gt;&#xA;Next, after one time step (i.e. one time increment) time changes&#xA;from &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt; so we get&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{v} = g(n+1)\, \vec{V}(n+1)$$&lt;/span&gt; Thus,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{V}(n+1) = g(n+1)^{-1} \, \vec{v} = g(n+1)^{-1} \,g(n) \, \vec{V}(n)$$&lt;/span&gt;&#xA;Denote by &lt;span class=&quot;math-container&quot;&gt;$W(n) = g(n)^{-1}\, g(n+1)\, \in \, \text{SO}(3)$&lt;/span&gt;. Then&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{V}(n+1) = W(n)^{-1} \, \vec{V}(n) = W(n)^T \, \vec{V}(n)$$&lt;/span&gt;&#xA;because &lt;span class=&quot;math-container&quot;&gt;$W(n)^{-1} = W(n)^T$&lt;/span&gt; for any orthogonal matrix. The&#xA;orthogonal operator &lt;span class=&quot;math-container&quot;&gt;$W(n)^T$&lt;/span&gt; describes the one time step rotation&#xA;of the vector &lt;span class=&quot;math-container&quot;&gt;$\vec{V}(n)$&lt;/span&gt; to the vector &lt;span class=&quot;math-container&quot;&gt;$\vec{V}(n+1)$&lt;/span&gt; in the&#xA;coordinate system &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt;. The matrix &lt;span class=&quot;math-container&quot;&gt;$W(n)$&lt;/span&gt; is the discrete time&#xA;analogue of angular velocity. If one finds the matrices &lt;span class=&quot;math-container&quot;&gt;$W(n)$&lt;/span&gt;&#xA;then one can generate the matrices &lt;span class=&quot;math-container&quot;&gt;$g(n)$&lt;/span&gt; by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$g(n+1) =  g(n) \, W(n)$$&lt;/span&gt;&#xA;If &lt;span class=&quot;math-container&quot;&gt;$h&amp;gt;0$&lt;/span&gt; is the small time step (the time increment), then the&#xA;actual time is &lt;span class=&quot;math-container&quot;&gt;$t_{n} = n\, h$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_{n+1} = t_{n} + h = (n+1)h$&lt;/span&gt;.&#xA;Then, during the time period &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; between &lt;span class=&quot;math-container&quot;&gt;$t_n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_{n+1}$&lt;/span&gt; the&#xA;rotation &lt;span class=&quot;math-container&quot;&gt;$W(n)$&lt;/span&gt; is around an axis determined by a vector&#xA;&lt;span class=&quot;math-container&quot;&gt;$\vec{\Omega}(n)$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt; for which &lt;span class=&quot;math-container&quot;&gt;$\Omega(n) =&#xA;\text{Matrix}(\vec{\Omega}(n))$&lt;/span&gt; and so&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align} W(n) = e^{h\, \Omega(n)} = {\bf 1} \, + \,&#xA;\frac{\sin\Big(h\,&#xA;\big|\Omega(n)\big|\Big)}{\big|\Omega(n)\big|}\,\, \Omega(n) \, +&#xA;\,  \frac{1 - \cos\Big(h\,&#xA;\big|\Omega(n)\big|\Big)}{\big|\Omega(n)\big|^2}\,\, \Omega(n)^2\end{align}&lt;/span&gt;&#xA;The vector &lt;span class=&quot;math-container&quot;&gt;$\vec{\Omega}(n)$&lt;/span&gt; is the angular velocity written in&#xA;the coordinate system &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;${\bf 1}$&lt;/span&gt; is the identity matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With analogy of the time continuous case of Euler's differential&#xA;equations of motion, the latter are based on two laws of physics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(i)&lt;/strong&gt; The angular momentum &lt;span class=&quot;math-container&quot;&gt;$\vec{m}$&lt;/span&gt; of the asteroid with&#xA;respect to &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is constant, i.e. it doesn't change with time. This&#xA;is the so called conservation of angular momentum law. Thus,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{m} = g(n) \, \vec{M}(n)$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\vec{M}(n)$&lt;/span&gt; is the same&#xA;angular momentum but written with respect to the moving coordinate&#xA;system &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$\vec{M}(n)$&lt;/span&gt; must change with time &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. But for&#xA;&lt;span class=&quot;math-container&quot;&gt;$n+1$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{m} = g(n+1) \, \vec{M}(n+1)$$&lt;/span&gt; Thus&#xA;&lt;span class=&quot;math-container&quot;&gt;$$g(n) \, \vec{M}(n) = \vec{m} = g(n+1) \, \vec{M}(n+1)$$&lt;/span&gt; which&#xA;means that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{M}(n+1) = g(n+1)^{-1}\, g(n) \, \vec{M}(n)$$&lt;/span&gt; and hence&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{M}(n+1) = W(n)^T \, \vec{M}(n) = \Big( e^{-\,h\, \Omega(n)} \Big)\, \vec{M}(n)$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$g(n)^{-1}\, g(n+1) = W(n) =  e^{h\, \Omega(n)}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W(n)^T&#xA;= e^{-\, h\, \Omega(n)}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(ii)&lt;/strong&gt; The inertia matrix &lt;span class=&quot;math-container&quot;&gt;$\mathcal{I}$&lt;/span&gt; gives the link between&#xA;the angular velocity and the angular momentum&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{M}(n) = \mathcal{I} \, \vec{\Omega}(n)$$&lt;/span&gt; so the discrete&#xA;time analogue of Euler's differential equations are the following&#xA;discrete time difference equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\vec{M}(n+1) = \left( e^{-\,h\, \text{Matrix}\big(\mathcal{I}^{-1} \, \vec{M}(n)\big)} \right)\, \vec{M}(n) $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$M(n) = \text{Matrix}(\vec{M}(n)) \,\,\, \text{ and } \,\,\,&#xA; \vec{M}(n) = \text{Vector}({M}(n)) $$&lt;/span&gt;&#xA;Generating &lt;span class=&quot;math-container&quot;&gt;$\vec{M}(n)$&lt;/span&gt; leads to &lt;span class=&quot;math-container&quot;&gt;$W(n) = e^{h\, \mathcal{I}^{-1}&#xA;\, {M}(n)}$&lt;/span&gt; and consequently to&#xA;&lt;span class=&quot;math-container&quot;&gt;$$g(n) = g_0\, W(1)\, W(2) \, ... \, W(n)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is a point on the asteroid with coordinates (i.e. position&#xA;vector) &lt;span class=&quot;math-container&quot;&gt;$$\overrightarrow{Q} = \overrightarrow{AQ} = \begin{pmatrix}Q_1 \\&#xA;                                      Q_2\\&#xA;                                      Q_3 \end{pmatrix}$$&lt;/span&gt;&#xA;in the coordinate system &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt;, the point &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; has coordinates&#xA;(i.e. position vector) that change with time&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{q}(n) = \overrightarrow{OQ}(n)  = \begin{pmatrix} q_1(n) \\&#xA;                                      q_2(n)\\&#xA;                                      q_3(n) \end{pmatrix}$$&lt;/span&gt;&#xA;                                          with respect to the&#xA;                                          coordinate system &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;&#xA;                                          and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{q}(n) = g(n) \, \overrightarrow{Q} + \vec{a}(n) =&#xA;\Big( g_0\, W(1)\, W(2) \, ... \, W(n) \Big) \, \overrightarrow{Q}&#xA;+ \vec{a}(n)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Algorithm&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; Start with initial position of asteroid's center of&#xA;gravity &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; given by a position vector &lt;span class=&quot;math-container&quot;&gt;$\vec{a}(0)$&lt;/span&gt; with respect&#xA;to the inertial coordinate system &lt;span class=&quot;math-container&quot;&gt;$S = O\, \vec{e}_1\, \vec{e}_2\,&#xA;\vec{e}_3$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; Furthermore, fix an initial orientation of the asteroid&#xA;with respect to &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;, given by a choice of three vectors forming&#xA;the coordinate system &lt;span class=&quot;math-container&quot;&gt;$S_A =  A\, \vec{E}_1\, \vec{E}_2\,&#xA;\vec{E}_3$&lt;/span&gt; attached to the asteroid at time &lt;span class=&quot;math-container&quot;&gt;$n=0$&lt;/span&gt;. The latter&#xA;three vectors are written as &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;D vector columns with respect to&#xA;&lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; Form the orthogonal matrix &lt;span class=&quot;math-container&quot;&gt;$g(0) = \Big(\vec{E}_1\,&#xA;\vec{E}_2\, \vec{E}_3\Big)^T \, \in \, \text{SO}(3)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4.&lt;/strong&gt; Choose a point &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; on the asteroid, which is then given&#xA;by a position vector &lt;span class=&quot;math-container&quot;&gt;$\overrightarrow{Q}$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;5.&lt;/strong&gt; Take a diagonal inertia matrix &lt;span class=&quot;math-container&quot;&gt;$\mathcal{I} =&#xA;\text{diag}(I_1, I_2, I_3)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;6.&lt;/strong&gt; Fix a time step &lt;span class=&quot;math-container&quot;&gt;$h&amp;gt;0$&lt;/span&gt;, say &lt;span class=&quot;math-container&quot;&gt;$h = \frac{1}{25}$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$25$&lt;/span&gt;&#xA;frames per second is a reasonable choice to simulate smooth&#xA;continuous motion on the screen, but the rate of change of frames&#xA;can be chosen even faster if you want). Time is then &lt;span class=&quot;math-container&quot;&gt;$t = n \, h$&lt;/span&gt;&#xA;seconds, where &lt;span class=&quot;math-container&quot;&gt;$n = 0, 1, 2, ...$&lt;/span&gt;. Choose an integer &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; so that&#xA;time runs from &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t = N\, h$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;7.&lt;/strong&gt; Give the asteroid an initial angular momentum&#xA;&lt;span class=&quot;math-container&quot;&gt;$\vec{M}(0)$&lt;/span&gt; written as a vector column with respect to &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt;.&#xA;Alternatively, you can choose initial angular velocity&#xA;&lt;span class=&quot;math-container&quot;&gt;$\vec{\Omega}(0) = \mathcal{I}^{-1} \, \vec{M}(0)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;8.&lt;/strong&gt; Execute the following loop: For &lt;span class=&quot;math-container&quot;&gt;$n =  0\, ... \, N$&lt;/span&gt; do&#xA;the following&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;&amp;amp;\vec{\Omega}(n) =  \mathcal{I}^{-1} \, \vec{M}(n) \\&#xA;&amp;amp;{\Omega}(n) = \text{Matrix}\Big( \vec{\Omega}(n) \Big) \\&#xA;&amp;amp;W(n) = e^{h\, {\Omega}(n)} = {\bf 1} \, + \, \frac{\sin\Big(h\,&#xA;\big|\Omega(n)\big|\Big)}{\big|\Omega(n)\big|}\,\, \Omega(n) \, +&#xA;\,  \frac{1 - \cos\Big(h\,&#xA;\big|\Omega(n)\big|\Big)}{\big|\Omega(n)\big|^2}\,\Omega(n)^2\\&#xA;&amp;amp;\vec{M}(n+1) = W(n)^T\, \vec{M}(n)\\&#xA;&amp;amp;g(n+1) = g(n) \, W(n)&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result is a sequence of matrices &lt;span class=&quot;math-container&quot;&gt;$g(n) \,  : \, n = 0\, ... \,&#xA;N$&lt;/span&gt; and so the motion of point &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; with respect to the inertial&#xA;non-moving frame &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{q}(n) = g(n) \, \overrightarrow{Q} + \vec{a}(n) =&#xA;g(n) \, \overrightarrow{Q} + \vec{a}(0) + nh\, \vec{v}_A $$&lt;/span&gt; where&#xA;I have assumed that the center of mass &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; of the asteroid moves&#xA;uniformly with constant velocity &lt;span class=&quot;math-container&quot;&gt;$\vec{v}_A$&lt;/span&gt; along a straight-line&#xA;trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be very interested to get some feed back from you whether&#xA;you have been able to realize the simulation of this kind of&#xA;motion, regardless of the involvement of shooting projectiles at&#xA;point &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; from the asteroid. So. please let me know whether it&#xA;works.&lt;/p&gt;&#xA;" OwnerUserId="20964" LastEditorUserId="20964" LastEditDate="2019-02-24T01:57:31.180" LastActivityDate="2019-02-24T01:57:31.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29660" PostTypeId="1" AcceptedAnswerId="29665" CreationDate="2018-06-04T16:40:43.980" Score="3" ViewCount="194" Body="&lt;p&gt;I am trying to solve a non-linear time-dependent heat equation&#xA;$$\partial_tT=\nabla \left(k_T(T)\nabla T\right) + f$$&#xA;using the galerkin method, with neumann boundary conditions. For linearization of the nonlinear part I am using the newton approach.  The source term $f$ always stays positive, thus I would expect that &#xA;$$T\geq T_0\forall t\geq t_0$$&#xA;but now I notice that at some nodes my value for $T$ drops below the initial value, and even going into negative ranges. I do not understand why that happens, after it is independent of the time step, but not independent of the value for $f$.&lt;br&gt;&#xA;What could be the reason for this behaviour?&lt;br&gt;&#xA;I notice that the range for the negative values is approximately five to six orders smaller than the maximum value for $T$ at that time, thus could it be an accuracy problem?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2018-06-04T23:35:56.743" Title="Solving a non-linear heat equation with the galerkin method gives negative values" Tags="|nonlinear-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29662" PostTypeId="2" ParentId="29583" CreationDate="2018-06-04T18:44:23.037" Score="2" Body="&lt;p&gt;In the most general case you want to solve elasticity and heat conduction equations simultaneously, what is normally termed as a strongly coupled model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More commonly people use a weakly coupled model, where you first compute the temperature increase ($\Delta T$) for each point in your body and then you solve the following set of equations, where I am using index notation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Equilibrium equations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$\sigma_{ij, j} + b_i = 0\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sigma$ is the stress tensor, $b$ is the body forces vector.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Constitutive equations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$ \sigma_{ij} = \frac{E}{1 + \nu}\left[\epsilon_{ij} + \frac{\nu}{1 - 2\nu}\epsilon_{kk} \delta_{ij}\right] - \frac{E \alpha \Delta T}{1 - 2\nu}\delta_{ij}\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$E$ is the Young modulus, $\nu$ the Poisson coefficient, $\epsilon$ the strain tensor, $\Delta T$ is the temperature increase, $\alpha$ the linear coefficient of thermal expansion, and $\delta_{ij}$ is the Kronecker delta.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Infinitesimal strain definition&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;$$\epsilon_{ij} = \frac{1}{2}\left(u_{i,j} + u_{j, i}\right)\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$u$ is the displacement vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can write the equations in terms of displacements, and this is the most common formulation in FEM software. In that case you can think that the change in temperature add a body force to your differential equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final comment, the simplest case is to consider $\Delta T$ to be uniform over your whole domain.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-06-04T18:44:23.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29664" PostTypeId="2" ParentId="29655" CreationDate="2018-06-04T21:52:48.147" Score="2" Body="&lt;p&gt;The Laplacian is $\sim k^2$ for low wavenumbers; it is a general property and it does not depend of your discretization. You are solving the Poisson equation for two different RHS so the solutions are going to be different. You cannot call that difference &quot;error&quot;, the solutions &lt;strong&gt;have&lt;/strong&gt; to be different.&lt;/p&gt;&#xA;" OwnerUserId="25205" LastActivityDate="2018-06-04T21:52:48.147" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29665" PostTypeId="2" ParentId="29660" CreationDate="2018-06-04T23:35:56.743" Score="4" Body="&lt;p&gt;It turns out that a variation of this question was recently asked on the deal.II mailing list, so I'll copy my answer here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your misunderstanding is that you believe that a property of the exact solution of the equation is supposed to also hold for the discrete one. But there is really no reason for this to be the case other than the fact that you know it must be true for the limit of your numerical solutions as $h\rightarrow 0$ in some sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give you just one example, if you take the Heaviside function&#xA;$$&#xA;  H(x) = \left\{&#xA;{0  \qquad\text{if $x\le0$}} \atop&#xA;         {1 \qquad  \text{if $x&amp;gt;1$}} \right.&#xA;$$&#xA;then it is obviously non-negative. But if you take the Fourier transform and truncate after a finite number of basis functions, then you get Gibbs phenomenon -- there are numerous examples of the over- and undershots here:&#xA;  &lt;a href=&quot;https://en.wikipedia.org/wiki/Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Fourier_transform&lt;/a&gt;&#xA;So the continuous property is also not satisfied for the finite-dimensional case. In fact, it is not even satisfied in a &lt;em&gt;pointwise&lt;/em&gt; sense if you let $N\rightarrow \infty$: the Fourier transform only converges in $L_2$, not in $L_\infty$!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-06-04T23:35:56.743" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29666" PostTypeId="2" ParentId="1556" CreationDate="2018-06-05T03:58:18.963" Score="2" Body="&lt;p&gt;This one is quite old and not well known, but it is a good one:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.complex-a5.ru/polyboolean/downloads/polybool_eng.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;M. V. Leonov and A. G. Nikitin. An Efficient Algorithm for a Closed Set of Boolean Operations on Polygonal Regions in the Plane (draft English translation). A. P. Ershov Institute of Informatics Systems, Preprint 46, 1997&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27908" LastActivityDate="2018-06-05T03:58:18.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29667" PostTypeId="1" AcceptedAnswerId="29677" CreationDate="2018-06-05T04:21:37.343" Score="3" ViewCount="223" Body="&lt;p&gt;Focusing on following least squares problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min\limits_{V} \lVert Z - WV \rVert _{_F}^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$Z∈{R}^{m\times n},\quad W∈{R}^{m\times k},\quad V∈{R}^{k\times n},\quad k\lt m\lt n $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem can be easily solved given $Z$ and $W$. I assume the solution of this problem as $\overline{V}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is that,  how can we recover $W$ by $\overline{V}$ and $Z$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found something called &lt;em&gt;inverse optimization&lt;/em&gt;, but cannot figure out the relationship between them.&lt;/p&gt;&#xA;" OwnerUserId="27919" LastEditorUserId="20688" LastEditDate="2018-06-05T13:36:00.717" LastActivityDate="2018-06-07T12:22:47.547" Title="How to solve the inverse problem of least-squares?" Tags="|linear-algebra|optimization|least-squares|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29668" PostTypeId="1" CreationDate="2018-06-05T06:28:24.157" Score="3" ViewCount="99" Body="&lt;p&gt;How to use PETSc SNES (&lt;a href=&quot;http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/SNES/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scalable nonlinear equation solver&lt;/a&gt;), when the solution is not a vector but a user defined state?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am implementing a non-linear mechanics problem (geometrically exact shell 5-paramter model), the update &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$X^{(k+1)} = X^{(k)} + \Delta_X$$  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;is not additive and requires the use of exponential maps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, the state I mentioned above stores $ X $ :&#xA;$$ X = \left \{ \Phi , \Lambda  \right \} $$ &#xA;$ \Phi $ (vector of size 3) is mid-surface coordinates and have direct updates i.e  $$ \Phi _{n+1} = \Phi_{n} + \Delta \Phi $$&#xA;While $ \Lambda $, the rotation matrix (correpondiong to director at mid surface), is updated through exponential map (the update increment is a vector of size 2, which is obtained after exploiting the one to one connection between tangent space of SO3 and S2 and fixing a reference vector).&#xA;The residual at  each point of mid surface(node) is a vector of size 5 and consists of $ \Lambda $ , $ \Phi $ and their derivatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Newton-Raphson and many times the solution couldn't converge using a complete step size. Consequently, I thought of resorting to PETSc SNES (line search). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At any state, I know the residual vector and stiffness matrix. &#xA;I could not figure how to make sense of solution vector that I get from SNES solver (or to send data to SNES), given that I have my own update procedure.&lt;/p&gt;&#xA;" OwnerUserId="27920" LastEditorUserId="27920" LastEditDate="2018-08-12T14:29:26.210" LastActivityDate="2018-08-12T14:29:26.210" Title="PETSc SNES for user defined state" Tags="|nonlinear-equations|iterative-method|petsc|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29669" PostTypeId="2" ParentId="29548" CreationDate="2018-06-05T08:08:53.320" Score="1" Body="&lt;p&gt;The block SPD-ness is preserved because any partitioning does not change the SPD property of the matrix as it would not alter the eigenvalues as partitioning would involve row and column swaps which would only re-arrange the matrix but not change the fact that the eigenvalues of its block be greater than 0.&lt;/p&gt;&#xA;" OwnerUserId="16601" LastActivityDate="2018-06-05T08:08:53.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29671" PostTypeId="1" CreationDate="2018-06-05T16:20:05.567" Score="4" ViewCount="221" Body="&lt;p&gt;In this code, all processes post a barrier, sleep a while for good measure, then first &lt;code&gt;Test&lt;/code&gt; and then &lt;code&gt;Wait&lt;/code&gt; for the barrier. The &lt;code&gt;Test&lt;/code&gt; says no and the wait succeeds. A test should be like a non-blocking wait, so it should succeed. What am I missing?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  printf(&quot;[%d] posting barrier, then sleeping\n&quot;,procid);&#xA;  MPI_Request final_barrier;&#xA;  MPI_Ibarrier(comm,&amp;amp;final_barrier);&#xA;  sleep(2);&#xA;&#xA;  int all_done_flag=0;&#xA;  MPI_Test(&amp;amp;final_barrier,&amp;amp;all_done_flag,MPI_STATUS_IGNORE);&#xA;  printf(&quot;[%d] all done: %d\n&quot;,procid,all_done_flag);&#xA;&#xA;  MPI_Wait(&amp;amp;final_barrier,MPI_STATUS_IGNORE);&#xA;  printf(&quot;[%d] concluded\n&quot;,procid);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4087" LastEditorUserId="4087" LastEditDate="2018-06-06T00:13:52.237" LastActivityDate="2018-06-06T00:13:52.237" Title="Can I MPI_Test for an Ibarrier" Tags="|mpi|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29672" PostTypeId="2" ParentId="29632" CreationDate="2018-06-05T16:56:03.633" Score="1" Body="&lt;p&gt;The question is a bit unclear here, so I'll start by restating the problem.  The OP has a matrix &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A=\left[&#xA;\begin{array}{cc}&#xA;0.5 &amp;amp; -0.4 \\&#xA;1   &amp;amp; -0.5 &#xA;\end{array}&#xA;\right]$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and wants to find a symmetric matrix $Q$ such that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q-A^{T}QA \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q \succ 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem here is that $Q \succ 0$ isn't an SDP constraint.  We could put in $Q \succeq 0$, but $Q=0$ is a solution to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q-A^{T}QA \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we want to constrain $Q$ to not be singular.  In the comments, the OP suggested solving the SDP &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min t$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q-A^{T}QA \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$tI-Q-A^{T}QA \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q \succeq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but $t=0$, $Q=0$ is an optimal solution to this problem too!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative SDP formulation that does ensure that $Q \succ 0$ is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\min 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q-A^{T}QA - \epsilon I \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\epsilon$ is a small positive constant (I'll use $\epsilon=0.01$ in the following.)  Because $A^{T}QA$ is positive semidefinite and $\epsilon I$ is positive definite, these constraints enforce $Q \succ 0$ (In fact, $Q \succeq \epsilon I$, so the smallest eigenvalue of $Q$ will be greater than or equal to $\epsilon$.)    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modeling packages like CVX and Yalmip can easily turn this formulation into an SDP that can be solved by a variety of solvers such as SDPT3, SeDuMi, CSDP, SDPA, etc.  However, the OP wants to see how to turn this into a standard form SDP &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \mbox{tr}(CX) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(A_{i}X)=b_{i}\;\; i=1, 2, \ldots, m$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step is to introduce a slack variable $S$, and write the &#xA;problem as  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q-A^{T}QA-S=\epsilon I$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q \succeq 0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraint $Q-A^{T}QA-S=\epsilon I$ is linear in the elements of $Q$, although this might not be immediately obvious.  The key observation is that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A^{T}QA=\sum_{i=1}^{2} \sum_{j=1}^{2} Q_{i,j} A_{i,:}^{T}A_{j,:}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the matrices in $Q-A^{T}QA-S$ are all symmetric, we need 3 linear equality constraints for the (1,1), (1,2) and (2,2) elements of the matrix equality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0.75Q_{1,1}-0.5Q_{1,2}-0.5Q_{2,1}-1.0Q_{2,2}-S_{1,1}=\epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0.2Q_{1,1}+1.25Q_{1,2}+0.4Q_{2,1}+0.5Q_{2,2}-S_{1,2}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$-0.16Q_{1,1}-0.2Q_{1,2}-0.2Q_{2,1}+0.75Q_{2,2}-S_{2,2}=\epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The constraints must be in symmetric form, so we rewrite the second constraint as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$0.2Q_{1,1}+0.825Q_{1,2}+0.825Q_{2,1}+0.5Q_{2,2}-0.5S_{1,2}-0.5S_{2,1}=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we embed $Q$ and $S$ into a block diagonal matrix &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X=\left[&#xA;\begin{array}{cc}&#xA;Q &amp;amp; 0 \\&#xA;0 &amp;amp; S \\&#xA;\end{array}&#xA;\right].$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem becomes &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\max \mbox{tr}(CX)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(A_{1}X)=\epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(A_{2}X)=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mbox{tr}(A_{3}X)=\epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$X \succeq 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C=0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{1}=\left[&#xA;\begin{array}{cccc}&#xA;0.75 &amp;amp; -0.5 &amp;amp; 0 &amp;amp; 0 \\&#xA;-0.5 &amp;amp; -1.0 &amp;amp; 0 &amp;amp; 0 \\&#xA;0    &amp;amp;   0  &amp;amp; -1.0 &amp;amp; 0 \\&#xA;0    &amp;amp;   0  &amp;amp;  0   &amp;amp; 0 \\&#xA;\end{array}&#xA;\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{2}=\left[&#xA;\begin{array}{cccc}&#xA;0.2 &amp;amp; 0.825 &amp;amp; 0 &amp;amp; 0 \\&#xA;0.825 &amp;amp; 0.5 &amp;amp; 0 &amp;amp; 0 \\&#xA;0    &amp;amp;   0  &amp;amp;  0 &amp;amp; -0.5 \\&#xA;0    &amp;amp;   0  &amp;amp;  -0.5   &amp;amp; 0 \\&#xA;\end{array}&#xA;\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$A_{3}=\left[&#xA;\begin{array}{cccc}&#xA;-0.16 &amp;amp; -0.2 &amp;amp; 0 &amp;amp; 0 \\&#xA;-0.2 &amp;amp; +0.75 &amp;amp; 0 &amp;amp; 0 \\&#xA;0    &amp;amp;   0  &amp;amp;  0 &amp;amp; 0 \\&#xA;0    &amp;amp;   0  &amp;amp;  0  &amp;amp; -1.0 \\&#xA;\end{array}&#xA;\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In SDPA format, the problem is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;3&#xA;2&#xA;2 2&#xA;0.01 0 0.01&#xA;1 1 1 1 0.75&#xA;1 1 1 2 -0.5&#xA;1 1 2 2 -1.0&#xA;1 2 1 1 -1.0&#xA;2 1 1 1 0.2&#xA;2 1 1 2 0.825&#xA;2 1 2 2 0.5&#xA;2 2 1 2 -0.5&#xA;3 1 1 1 -0.16&#xA;3 1 1 2 -0.2&#xA;3 1 2 2 +0.75&#xA;3 2 2 2 -1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I solved this problem using CSDP and obtained &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Q=\left[&#xA;\begin{array}{cc}&#xA;   38.5170   &amp;amp; -11.8166 \\&#xA;  -11.8166    &amp;amp; 24.3371 \\&#xA;\end{array}&#xA;\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$S=\left[&#xA;\begin{array}{cc}&#xA;   16.3572   &amp;amp;   0.3746 \\&#xA;0.3746   &amp;amp;  16.8067 \\&#xA;\end{array}&#xA;\right]$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's easy to verify that $Q$ has all of the required properties.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are infinitely many solutions to this problem- there is no reason to expect that all solvers will return the same solution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also adjust the objective function or add additional constraints to push the solution in some desired direction.  For example, you might want to minimize the maximum eigenvalue of $Q$ or minimize the sum of the eigenvalues of $Q$.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-06-05T16:56:03.633" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29673" PostTypeId="1" AcceptedAnswerId="29674" CreationDate="2018-06-05T17:31:56.863" Score="6" ViewCount="169" Body="&lt;p&gt;I'm interested in numerically diagonalizing a class of structured, symmetric eigenvalue problems with potentially extremely small eigenvalue gaps. The question I have is how to design a numerically stable way to get the eigenvalues and eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While usually small gaps destroy precision in numerical eigenproblems, for reasons I will explain below the structure of these problems allow for accurate solutions in principal, i.e. the eigenvalues and eigenvectors don't change dramatically as a function of perturbations of the non-zero matrix elements. An explicit example will follow, but the reason for the stability is the following: &#xA;the structure of the small parameters is such that they can always be brought to the block off-diagonal form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H = \left(&#xA;\begin{array}{c|c}&#xA;D_0 &amp;amp; \epsilon \\&#xA;\epsilon^T &amp;amp; D_1&#xA;\end{array}&#xA;\right),$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $D_0$, $D_1$ are diagonal and $\epsilon$ is small compared to the level spacing in $D_0$ and $D_1$. In this case, perturbation theory says that the eigenvalues $\lambda_i$ are close to those of $D_0$ or $D_1$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$|\lambda_i - d_i | &amp;lt; \frac{||\epsilon||^2}{d_i - d_j} $. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigenproblems of interest come from Hamiltonians of very short quantum spin chains, for example&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H = -h_1 \sigma^z_1 - h_2\sigma^z_2 - h_3\sigma^z_3 - J_1 \sigma^x_1 \sigma^x_2 - J_2 \sigma^x_2 \sigma^x_3,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sigma^z_1 = kron(\sigma^z, I, I),$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\sigma^x_2 = kron(I, \sigma^x, I),$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the full matrix takes a form like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$H = \left(&#xA;\begin{array}{cccccccc}&#xA; -d_0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -J_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; -J_2 &amp;amp; 0 \\&#xA; 0 &amp;amp; -d_1 &amp;amp; -J_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -J_2 \\&#xA; 0 &amp;amp; -J_1 &amp;amp; -d_2 &amp;amp; 0 &amp;amp; -J_2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA; -J_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; -d_3 &amp;amp; 0 &amp;amp; -J_2 &amp;amp; 0 &amp;amp; 0 \\&#xA; 0 &amp;amp; 0 &amp;amp; -J_2 &amp;amp; 0 &amp;amp; d_3 &amp;amp; 0 &amp;amp; 0 &amp;amp; -J_1 \\&#xA; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -J_2 &amp;amp; 0 &amp;amp; d_2 &amp;amp; -J_1 &amp;amp; 0 \\&#xA; -J_2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -J_1 &amp;amp; d_1 &amp;amp; 0 \\&#xA; 0 &amp;amp; -J_2 &amp;amp; 0 &amp;amp; 0 &amp;amp; -J_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; d_0 \\&#xA;\end{array}&#xA;\right),$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $d_0 = h_1 + h_2 + h_3$, $d_1 = -h_1 + h_2 + h_3$, $d_2 = h_1 - h_2 + h_3$, $d_3 = h_1 + h_2 - h_3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ratios of the parameters $h_1, h_2, h_3, J_1, J_2$ are potentially much smaller than machine precision. This leads to very small gaps in the eigenvalues and trouble for the typical numerical eigensolvers working at finite precision. &#xA;However, I do not think it should be impossible to determine the eigenvectors, because when the ratios are large we have precisely the situation where perturbation theory can be used to approximately determine the eigenvalues and eigenvectors in the following manner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if $h_2$ is the largest scale, then perturbation theory in the small parameters $J_i/h_2$ shows that the eigenvalues are very close to the eigenvalues of these smaller matrices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \tilde{H}_{\pm} =  \pm h_2 I - h_1 \sigma^z_1 - h_3 \sigma^z_2 \pm \frac{J_1 J_2}{h_2} \sigma^x_1 \sigma^x_2 + \mathcal{O}(\frac{J_i^4}{h_2^3}).$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This argument can be repeated again if one of the remaining coefficients $h_1, h_3, \frac{J_1 J_2}{h_2}$ is large compared to the others. For example, if $\frac{J_1 J_2}{h_2}$ is the largest remaining scale, the eigenvalues will be very close to  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ \lambda_{\pm, \pm, \pm} \approx \pm h_2 \pm \frac{J_1 J_2}{h_2} \pm \frac{h_1 h_2 h_3}{J_1 J_2}.$&#xA;The eigenvectors are also accurately determined by perturbation theory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to accurately determine a single eigenvector of these matrices. When using numerical eigensolvers, the issue is as follows: when the small gaps (in the example $2\frac{h_1 h_2 h_3}{J_1 J_2}$) are less than machine precision, a numerical eigensolver will mix the eigenstates. I could just use perturbation theory in this case.&#xA;In the situations where the ratios of the parameters are not so small, perturbation theory fails to be accurate but numerical eigensolvers have no problem. Is it possible to make an eigensolver that captures both cases accurately?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. For the specific test problem chosen here, a mapping to free fermions shows that the exact eigenvalues are given by the formula &#xA;$ \lambda_{\pm, \pm, \pm} = \pm \sigma_1 \pm \sigma_2 \pm \sigma_3$&#xA;where $\sigma_i$ are the singular values of the matrix &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M = \left(&#xA;\begin{array}{ccc}&#xA; h_1 &amp;amp; J_1 &amp;amp; 0  \\&#xA; 0 &amp;amp; h_2 &amp;amp; J_2 \\&#xA; 0 &amp;amp; 0 &amp;amp; h_3 \\&#xA;\end{array}&#xA;\right).$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't true for the general case I am interested in but can serve as a useful check that the eigensolver is actually accurate.&lt;/p&gt;&#xA;" OwnerUserId="27918" LastActivityDate="2018-06-05T19:01:59.020" Title="Eigenvalue problems with extremely small gaps" Tags="|computational-physics|eigensystem|preconditioning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29674" PostTypeId="2" ParentId="29673" CreationDate="2018-06-05T18:09:59.930" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it possible to make an eigensolver that captures both cases accurately?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In general no, for the following reason. When a typical eigensolver produces approximate eigenvalues of a matrix $A$, the answer is usually backward stable: the output eigenvalues are the exact eigenvalues of a nearby matrix $A+\delta A$ where $\|\delta A\|\leq \epsilon_{\mathrm{mach}}\|A\|$. This is also what causes them to produce inaccurate almost-double eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you have a special matrix $H$ that can only be perturbed in specific ways: as far as an eigensolver is concerned, the nearby matrix can contain 64 independent perturbation, one per matrix element, but in your matrix only the 5 parameters can be perturbed and the space of potential perturbations $\delta H$ is much smaller than what is allowed by $\|\delta H\| \leq \epsilon_{\mathrm{mach}}\|H\|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that if you did have an eigensolver that could handle this, it would have to be designed in advance with knowledge of the special structure of your matrix. This is in fact what you've done already: you've described an eigensolver that dispatches either to a standard solver or to perturbation theory. I think this is probably the best you can do anyway, short of having some closed form solution or something.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2018-06-05T19:01:59.020" LastActivityDate="2018-06-05T19:01:59.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29675" PostTypeId="2" ParentId="23680" CreationDate="2018-06-06T00:06:06.853" Score="1" Body="&lt;p&gt;Please check this line, the A[2][2] should be squared:&#xA;auto diag_sq = A[0][0]*A[0][0] + A[1][1]*A[1][1] + A[2][2];&lt;/p&gt;&#xA;" OwnerUserId="27927" LastActivityDate="2018-06-06T00:06:06.853" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29676" PostTypeId="2" ParentId="8308" CreationDate="2018-06-06T00:11:54.487" Score="0" Body="&lt;p&gt;You are right that there are situations where you can determine by reasoning that an &lt;code&gt;Isend&lt;/code&gt; or &lt;code&gt;Irecv&lt;/code&gt; has been concluded, and so you don't have to do a &lt;code&gt;Wait&lt;/code&gt; semantically. In that case, the only reason for doing the wait is that the &quot;I&quot; operation allocates an &lt;code&gt;MPI_Request&lt;/code&gt; object and you need to deallocate it. So the wait prevents memory leaks. You can also use &lt;code&gt;MPI_Request_free&lt;/code&gt; to deallocate the request object.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastEditorUserId="4087" LastEditDate="2018-06-11T16:47:33.753" LastActivityDate="2018-06-11T16:47:33.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29677" PostTypeId="2" ParentId="29667" CreationDate="2018-06-06T00:18:17.793" Score="2" Body="&lt;p&gt;The solution to the initial least squares problem can be written in terms of the pseudoinverse&#xA;$${\bar V} = W^+Z$$&#xA;Then several days later, you still have $({\bar V},Z)$ but have somehow lost the definition of $W$. &#xA;&lt;br&gt;You can try the following &#xA;$$\eqalign{&#xA;W^+ &amp;amp;= {\bar V}Z^+ \cr&#xA;W &amp;amp;= ({\bar V}Z^+)^+ \cr&#xA;}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&lt;/p&gt;&#xA;" OwnerUserId="27795" LastEditorUserId="4405" LastEditDate="2018-06-07T12:22:47.547" LastActivityDate="2018-06-07T12:22:47.547" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="29678" PostTypeId="2" ParentId="29656" CreationDate="2018-06-06T01:55:56.450" Score="2" Body="&lt;p&gt;They just curve fit the two equations&#xA;$$&#xA;  x(\omega) = h - b \sin(\omega) ~,~~&#xA;  y(\omega) = k + a \cos(\omega)&#xA;$$&#xA;using a nonlinear curve fitting algorithm.  However, the accuracy of the fit depends on the choice of algorithm and you have to be careful to make sure that $a$ and $b$ are always positive (by adding constraints).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way of approaching the problem is shown below (using Octave).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1) Set up the points&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fig1 = figure()&#xA;gca&#xA;[x, y, buttons] = ginput(10)&#xA;plot(x, y, 'x-');&#xA;hold on;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;2) Compute convex hull&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = convhull(x,y)&#xA;plot(x(k), y(k), &quot;r-&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;3) Remove points inside hull&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xx = zeros(length(k)-1)&#xA;yy = zeros(length(k)-1)&#xA;for i=1:length(k)-1&#xA;  xx(i) = x(k(i));&#xA;  yy(i) = y(k(i));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;4) Compute centroid of hull&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x_cen = mean(xx);&#xA;y_cen = mean(yy);&#xA;plot(x_cen, y_cen, 'mo');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;5) Compute angles and plot x,y vs. angle&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i=1:length(xx)&#xA;  theta(i) = atan2(yy(i)-y_cen, xx(i)-x_cen);&#xA;end&#xA;angle = theta*180/pi&#xA;fig2 = figure();&#xA;plot(angle, yy, 'r-'); hold on;&#xA;plot(angle, xx, 'b-');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;6) Load optimization package&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pkg load optim&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;7) Set up curve fit&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;indep_omega = theta(1:length(theta))&#xA;obs_x = xx(1:length(theta)) - x_cen&#xA;obs_y = yy(1:length(theta)) - y_cen&#xA;&#xA;% Model functions&#xA;fx_omega = @ (p, omega) p(1) - p(2) * sin(omega)&#xA;fy_omega = @ (p, omega) p(1) + p(2) * cos(omega)&#xA;&#xA;% Initial values&#xA;init_p_x = [0; 0.5]&#xA;init_p_y = [0; 0.5]&#xA;&#xA;% Constraints (A.' * p + B &amp;gt;= 0)&#xA;A = [0; 1] ; B = 0;&#xA;settings = optimset (&quot;inequc&quot;, {A, B});&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;8) Do curve fit&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[p_x, model_values, cvg, outp] = nonlin_curvefit(fx_omega, init_p_x, indep_omega, obs_x', settings)&#xA;[p_y, model_values, cvg, outp] = nonlin_curvefit(fy_omega, init_p_y, indep_omega, obs_y', settings)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2018-06-06T01:55:56.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29679" PostTypeId="2" ParentId="26570" CreationDate="2018-06-06T02:07:41.073" Score="5" Body="&lt;p&gt;You say that you want an MPI version. Then you need to study the literature, as the distributed memory variant of matrix-matrix product are not a simple parallellization of the sequential version.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The Cannon algorithm is pretty cute if you're on a square processor grid. In each step you rotate the input matrix rows and columns, so that in the end each processor contains a sum of &lt;code&gt;A(i,k)B(k,j)&lt;/code&gt; products.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Summa algorithm is more or less a bunch of rank k updates. This is more general in shapes of the matrices and processor grids.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2018-06-06T02:07:41.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29680" PostTypeId="2" ParentId="26886" CreationDate="2018-06-06T02:34:41.297" Score="3" Body="&lt;p&gt;You may please refer chapter two of Advanced Topics in Finite Element Analysis of Structures: With Mathematica and MATLAB Computations by M. Asghar Bhatti.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Section 2.4.5 of the book deals with the evaluation of surface integrals. A workout example of evaluating the surface pressure loads with Mathematica code is also given in the book.&#xA;A Matlab code is also given in the book site for 8 node brick element.&lt;/p&gt;&#xA;" OwnerUserId="27928" LastEditorUserId="27928" LastEditDate="2018-06-08T17:58:29.833" LastActivityDate="2018-06-08T17:58:29.833" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29683" PostTypeId="1" CreationDate="2018-06-06T11:49:39.793" Score="1" ViewCount="190" Body="&lt;p&gt;I need your help to solve a problem I'm working on for school.&#xA;My goal is to approximate the coefficients of a weight matrix so that they check particular properties.&#xA;So I have $W$ the weight matrix of my assets and $\Omega$ the covariance matrix of these assets. The global risk of my portfolio, $\sigma$, is defined by : $\sigma  = \sqrt{^tW.\Omega.W}$ and the marginal risk of the asset number $j$ by : &#xA;$\sigma_j = (W_j*(\Omega.W)_j)/\sigma^{2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to find a way to approximate the weights so that if $N$ is the number of assets, for each $j$, $\sigma_j=1/N$ and $\sum_{j=1}^{N}W_j=1$, the covariance matrix being known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use the &lt;code&gt;fsolve&lt;/code&gt; function from &lt;code&gt;scipy.optimize&lt;/code&gt; package in python, but the answers I get don't matches my criteria as I need the coefficients of $W$ to be between 0 and 1 and their sum to be 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My program:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def CalcPoids(A,t):&#xA;    n=len(A)&#xA;    W=MatriceCov(A,t-30,30)&#xA;    C=[(1/n) for i in range(n)]&#xA;    def g(x,i):&#xA;        M=np.dot(W,x)&#xA;        d=np.dot((x.T),np.dot(W,x))&#xA;        return(x[i]*M[i])/d - (1/n)&#xA;    def f(x):&#xA;        return [g(x,i) for i in range(n)]&#xA;    return op.fsolve(f,C)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;MatriceCov&lt;/code&gt; is just calculating the covariance matrix of my assets for the previous 30 values.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def MatriceCov(A,j,t):&#xA;    n=len(A)&#xA;    M=np.zeros((n,n))&#xA;    for i in range(n):&#xA;        M[i][i]=Variancet(A[i],j,t)&#xA;    for j in range(n):&#xA;        i=0&#xA;        while i&amp;lt;j:&#xA;            M[i][j]=M[j][i]=Covt(A[i],A[j],j,t)&#xA;            i+=1&#xA;    return M&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you have any ideas about how solving this problem, any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="27933" LastEditorUserId="27933" LastEditDate="2018-06-06T14:26:58.973" LastActivityDate="2018-06-06T14:26:58.973" Title="Approximation of a non linear problem with python" Tags="|python|nonlinear-equations|approximation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29684" PostTypeId="2" ParentId="29683" CreationDate="2018-06-06T12:23:41.020" Score="1" Body="&lt;p&gt;A very common reason that iteration can fail to converge is that you've given it an invalid system of equation because of a coding error. I suspect that's what happened here: in &lt;code&gt;op.fsolve(f,C)&lt;/code&gt;, the parameter &lt;code&gt;C&lt;/code&gt; is the starting guess, so you end up solving $f(x)=0$ starting with $x_0=C$ instead of solving $f(x)=C$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the documentation at &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.fsolve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.fsolve.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-06-06T12:23:41.020" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29685" PostTypeId="2" ParentId="27446" CreationDate="2018-06-06T16:50:15.823" Score="4" Body="&lt;p&gt;As Wolfgang observed, you are basically transposing a connectivity matrix: every process $i$ knows row $i$ that states which $j$ values you're sending to. What you're interested in is the columns: the description in column $j$ of everyone that sends to $j$. However, implementing this with an &lt;code&gt;Allgather&lt;/code&gt; is overkill: you only need to know the sum of the entries in a column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MPI has just the routine for this! It's called &lt;code&gt;MPI_Reduce_scatter&lt;/code&gt;. Every process declares an array of values as long as the number of ranks; MPI then does a pointwise reduction on the components of this array (that's where you do the summing), and then scatters it, sending a single scalar to each process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This routine has two major applications: the setup phase of the sparse matrix-vector product, and the execution of a 2D-distributed dense matrix-vector product. (Look for reduce-scatter in the index of &lt;a href=&quot;http://pages.tacc.utexas.edu/~eijkhout/istc/istc.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;My HPC book, freely downloadable&quot;&gt;my HPC book&lt;/a&gt;) And it sounds like it's just what you need: because of your hash code everyone knows who they are sending to, not who they are receiving from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Small note: in your receive call I would use &lt;code&gt;MPI_ANY_SOURCE&lt;/code&gt; to prevent any deadlock or parallel performance problems.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastEditorUserId="4087" LastEditDate="2018-06-11T14:05:36.043" LastActivityDate="2018-06-11T14:05:36.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29686" PostTypeId="2" ParentId="23311" CreationDate="2018-06-06T17:10:37.490" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;Everything Bill Barth wrote is correct.&lt;/li&gt;&#xA;&lt;li&gt;You are confused by MPI processes (or &quot;ranks&quot;) and computers. If you do &quot;mpirun -np 18 yourprogram&quot; on a 4-core machine that is not connected to anything, then MPI_Comm_size will still report that there are 18 MPI processes. Not 1, or 4. To an extent you could say that MPI processes are a software construct. How that ties to the hardware is not (as Bill states) a concern of the MPI standard.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2018-06-06T17:10:37.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29688" PostTypeId="2" ParentId="10516" CreationDate="2018-06-07T10:23:25.717" Score="5" Body="&lt;p&gt;Generating C++ code and use Eigen/Armadillo is a possibility, but this depends on the application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution for us was to just write out the result explicitly for $N&amp;lt;8$. Without the loops, the code is very fast with modern compilers and vector support (sse2, avx2, and avx512 in 64 bit).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do take care of memory alignment of your data (up to 64 bytes aligned) and do restrict the pointers to make life easier for the compiler. No need to use multicore support with these matrix sizes, the overhead is larger than the gain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use scripting to automatically generate separate functions for each possible combination, and cache the function pointers for consecutive calls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a nice cheap library for small matrix operations that is highly optimized by hand, called &lt;a href=&quot;http://www.optivec.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OptiVec&lt;/a&gt;, and it works very well in our case. We use it for $N \geq 8$.&lt;/p&gt;&#xA;" OwnerUserId="27943" LastEditorUserId="20688" LastEditDate="2018-06-07T13:34:06.503" LastActivityDate="2018-06-07T13:34:06.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29689" PostTypeId="1" CreationDate="2018-06-07T16:51:40.113" Score="2" ViewCount="1473" Body="&lt;p&gt;I am trying to find interatomic distance considering periodic boundary conditions for hexagon cubic cells (graphite). I tried to follow the answers to these two questions here but am unable to get the right results&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/20165/periodic-boundary-conditions-for-triclinic-box&quot;&gt;periodic boundary conditions for triclinic box&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/3107/minimum-image-convention-for-triclinic-unit-cell&quot;&gt;Minimum image convention for triclinic unit cell&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code, implementing the answer at first post is here&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;positions = initialConfig.get_positions()&#xA;nAtoms = positions.shape[0]&#xA;xMin = zMin = 0.0&#xA;xMax = LX&#xA;zMax = LZ&#xA;yCoord = positions[:,0:2]&#xA;u2 = UCell[1][0:2,]&#xA;modYPos = np.zeros((0,2))&#xA;for i in range(nAtoms):&#xA;    currAtom = yCoord[i,:]&#xA;    if currAtom[1]&amp;gt;LY:&#xA;        modPos = currAtom -LY*u2&#xA;    elif currAtom[1]&amp;lt;LY:&#xA;        modPos = currAtom +LY*u2&#xA;    else:&#xA;        modPos = currAtom&#xA;    modYPos = np.vstack((modYPos, modPos))&#xA;xCoord = modYPos[:,0].reshape(-1,1)&#xA;xCoord = xCoord - xMax*np.rint(xCoord/xMax)&#xA;zCoord = positions[:,2].reshape(-1,1)&#xA;zCoord = zCoord - zMax*np.rint(zCoord/zMax)&#xA;app5rMat = np.zeros((nAtoms,coordinates))&#xA;app5rMat[:,0] = xCoord[:,0]&#xA;app5rMat[:,1] = yCoord[:,1]&#xA;app5rMat[:,2] = zCoord[:,0]&#xA;app5RIJ = app5rMat.reshape(nAtoms,1,coordinates)-app5rMat.reshape(1,nAtoms,coordinates)&#xA;app5Dist = np.linalg.norm(app5RIJ, axis =2 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code for the second post is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#S1 create A matrix &#xA;UCell = initialConfig.get_cell()&#xA;cellParams = initialConfig.get_cell_lengths_and_angles()&#xA;a = cellParams[0]&#xA;b = cellParams[1]&#xA;c = cellParams[2]&#xA;&#xA;alpha= math.radians(90)#math.pi/2#math.radians(cellParams[3])&#xA;beta= math.radians(90)#math.pi/3#math.radians(cellParams[4])&#xA;gamma= math.radians(60)#math.pi/2#math.radians(cellParams[5])&#xA;&#xA;A = UCell.T&#xA;&#xA;#S2 Invert A to bet B&#xA;B = np.linalg.inv(A)&#xA;np.matmul(B,A)&#xA;&#xA;#S3 -- get fractional coordinates &#xA;&#xA;positions = initialConfig.get_positions()&#xA;nAtoms = positions.shape[0]&#xA;fracCood = np.zeros((0,3))&#xA;for i in range(nAtoms):&#xA;    currAtom = positions[i,:].reshape(-1,1)&#xA;    fracCood = np.vstack((fracCood, np.dot(B,currAtom).T))&#xA;fracCood.shape&#xA;&#xA;#s4 -- translate into reference cell&#xA;onesArray = np.ones((nAtoms,coordinates))&#xA;g = fracCood - np.rint(2*fracCood - onesArray)#np.floor(fracCood)&#xA;&#xA;#s4-- translate into real space &#xA;realCood = np.zeros((0,3))&#xA;for i in range(nAtoms):&#xA;    currAtom = g[i,:].reshape(-1,1)&#xA;    realCood = np.vstack((realCood, np.dot(A,currAtom).T))&#xA;algoRij = g.reshape(nAtoms,1,coordinates)-g.reshape(1,nAtoms,coordinates)&#xA;secondPart = np.rint(2*algoRij.copy() - onesArray.reshape(nAtoms,1,coordinates))&#xA;algoRij = algoRij-secondPart&#xA;algoRijPer = algoRij #-np.rint(algoRij)&#xA;algoRijDis = np.linalg.norm(algoRijPer, axis =2 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27946" LastEditorUserId="20688" LastEditDate="2018-06-07T17:09:04.417" LastActivityDate="2018-10-24T13:00:46.370" Title="Interatomic distance-periodic boundary conditions-non cubic unit cell" Tags="|python|boundary-conditions|molecular-dynamics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29690" PostTypeId="1" CreationDate="2018-06-07T18:10:01.947" Score="1" ViewCount="111" Body="&lt;p&gt;I have a code that uses both spatial and time discretization/integration. For convergence analysis, I am wondering how one would test the order of accuracy of their ${time}$ integration scheme? I assume you would have to isolate the time discretization from the spatial discretization somehow, but I am not sure how this can be done on a discretized mesh. I suppose you could choose some really really small spatial step size such that the spatial error is less than the time error, but this seems very inelegant and complicated.&lt;/p&gt;&#xA;" OwnerUserId="27504" LastActivityDate="2018-06-08T11:51:53.593" Title="How to isolate and test time discretization order of accuracy" Tags="|discretization|time-integration|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29693" PostTypeId="2" ParentId="29690" CreationDate="2018-06-08T11:43:07.323" Score="3" Body="&lt;p&gt;If you fix the spatial discretization, then you are solving a fixed system of ODEs. The exact solution of that system is not the same as the exact solution of the PDE, of course, but you can test convergence to it just like you would with any ODE system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the exact solution is usually too complicated to work out by hand, one typically uses a highly refined (in time) solution as a substitute. &lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2018-06-08T11:51:53.593" LastActivityDate="2018-06-08T11:51:53.593" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29694" PostTypeId="2" ParentId="12951" CreationDate="2018-06-08T18:29:56.450" Score="1" Body="&lt;p&gt;Let's say for simplicity that you have a 1D problem, and you discretize it over N grid points. Your code will have a bunch of loops from 1 to N that do stencil operations -- assuming you're not doing implicit time stepping because solving linear systems directly complicates the story considerable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this setup, a serial, OpenMP, and MPI version look remarkably similar.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Your serial version will have the loops from 1 to N.&lt;/li&gt;&#xA;&lt;li&gt;Your OpenMP version has the exact same loops, except with a directive that they are parallel.&lt;/li&gt;&#xA;&lt;li&gt;Interestingly, the MPI version will still have the same loops, but now your 1..N space is divided over your processors&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Ok? So the bulk of your code will look the same in all three cases. Now for the MPI case, which is the hard one. Your big loop gets divided over the processes, so you'd think that one process gets indices 100-200 for instance. Yes, but only in a metaphoric sense. With MPI every process is independent, so it thinks it has addresses 0-100, and you need to translate this back to the global numbering as needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real problem with MPI is that when you apply a stencil you may need some array components that live on another process. Elements 99 and 201 in the above example. That's where you need to do some send and receive operations before you execute your stencil code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you have only one code. Can you compile it three ways? Yeah, just sprinkle some &lt;code&gt;#ifdef&lt;/code&gt; directives through it. That part is simple.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2018-06-08T18:29:56.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29695" PostTypeId="1" CreationDate="2018-06-08T18:50:22.747" Score="1" ViewCount="130" Body="&lt;p&gt;I am interested in solving ODE systems of the form&#xA;\begin{align}&#xA;\frac{\partial \vec{u}}{\partial t} = F(\vec{u}) &#xA;\end{align}&#xA;where $F$ is a nonlinear operator, $\vec{u}$ is a vector valued function of $t$, and we have initial conditions $\vec{u}(0) = \vec{u}_0$. I am specifically interested in equations of this form that come from (finite element) spatial discretizations of initial-boundary value problems involving systems of PDEs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An accurate way of doing this would be to further discretize $F(\vec{u})$ and take small time steps with high order, but this may be too costly for the application. Instead, one may perform the ``operator split''&#xA;\begin{align}&#xA;F(\vec{u}) = A(\vec{u}) + B(\vec{u}),&#xA;\end{align}&#xA;and solve the following two subproblems in an alternating fashion for a single time step $t \mapsto t + \Delta t$:&#xA;\begin{align}&#xA;\begin{cases}&#xA;\frac{\partial \vec{v}}{\partial t} = A(\vec{v}), \\ &#xA;\frac{\partial \vec{w}}{\partial t} = B(\vec{w}).&#xA;\end{cases}&#xA;\end{align}&#xA;To start, the initial condition $\vec{u_0}$ is used to solve the first equation for $\vec{v_1}$, and then $\vec{v_1}$ is used as a condition to solve the second eqution for $\vec{w_1}$. We then set $\vec{u_1} := \vec{w_1}$, and then repeat this process to obtain $\vec{u_2}, \vec{u_3}, ... $ at each time step. This operator splitting method is well known to be first-order accurate, i.e., the operator error grows like $\mathcal{O}(\Delta t)$. The advantage here is the subproblems should somehow be easier to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am somewhat familiar with CFL conditions for explicit time-stepping schemes. They basically tell you how small you need to make your time step, for a specific problem, relative to your mesh size, and are generally only applicable for simple (linear) problems. For implicit time-stepping schemes applied to simple (linear) problems, one can expect unconditional stability, allowing for large time steps to be used. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My questions are the following: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When solving the two subproblems $\vec{v}_t = A\vec{v}$ and $\vec{w}_t = B \vec{w}$ in an alternating fashion, is one required to use the same time-integration scheme for both subproblems? For example, would I be able solve the first equation for time $v_{t_{n+1}}$ using a first-order implicit method, and then the second equation for time $w_{t_{n+1}}$ using a second-order explicit method? &lt;/li&gt;&#xA;&lt;li&gt;Do the time-step sizes for the two subproblems need to be the same? Or could one, for example, compute $v_{t_{n+1}}$ using the entire $\Delta t$ time step, and compute $w_{t_{n+1}}$ using four smaller $\frac{\Delta t}{4}$ time steps? Isn't this what higher order operator splitting basically is? Also, is this how you would ensure minimum step-size stability if one of the subproblems was &quot;stiff&quot;?&lt;/li&gt;&#xA;&lt;li&gt;If the two subproblems are nonlinear, how does one determine a good timestep size for each problem to guarantee stability? &lt;/li&gt;&#xA;&lt;li&gt;Is there any known relationship between the timestep size needed to guarantee stability of the original problem and timestep sizes needed to guarantee stability for the two subproblems?&lt;/li&gt;&#xA;&lt;li&gt;Should I probably be using second-order and higher operator splitting schemes, and if so, how do I decide which one to use (e.g., from this list &lt;a href=&quot;http://www.asc.tuwien.ac.at/~winfried/splitting/index.php?rc=0&amp;amp;ab=strang-ab&amp;amp;name=Strang&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.asc.tuwien.ac.at/~winfried/splitting/index.php?rc=0&amp;amp;ab=strang-ab&amp;amp;name=Strang&lt;/a&gt;)?&lt;/li&gt;&#xA;&lt;li&gt;Does it matter which subproblem you solve first?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Partial answers or references to literature are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2018-06-11T15:42:42.533" LastActivityDate="2018-06-11T15:42:42.533" Title="General questions regarding stability for time-integration of operator-split PDE systems" Tags="|pde|time-integration|operator-splitting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29696" PostTypeId="2" ParentId="29695" CreationDate="2018-06-08T23:55:45.723" Score="3" Body="&lt;p&gt;Yes, you can use different time steps and time stepping methods for the two sub-problems. In fact, that is one of the reasons why operator splitting methods are so popular: if you had two physical effects -- one slow, and one fast -- then you can use long time steps for the slow process and only need to use small time steps for the fast process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, finding the correct time step size is often easier for the separated sub-problems than for the coupled problem because you can more easily do things like a time step stability analysis for each process individually. How exactly you have to choose the time step of course depends on what exactly the two sub-processes are, and there is no guidance possible if you don't tell us what they are. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the question regarding whether you should use a second order method: Yes, you should! Learning about the first order splitting is often a good idea because it's more intuitive what is happening, but implementing the second order splitting is only marginally more complicated but has substantial advantages as far as accuracy is concerned. So yes, you should definitely do that!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally: From a convergence order perspective, it makes no difference which of the two processes you solve first and which second. That is also true for the second order splitting, though there one often chooses the simpler-to-solve process for the two half-steps because then you only have to solve the more-complicated one once per time step.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-06-08T23:55:45.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29697" PostTypeId="1" CreationDate="2018-06-09T12:52:20.837" Score="0" ViewCount="77" Body="&lt;p&gt;W have a complicated structure on which we do some FEM calculations regarding electrical potentials and heat distribution. The equations have the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla\kappa\nabla u = f + g\rvert_{N}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $u$ is the temperature or electrical potential, $\kappa$ is a material parameter and $f$ and $g$ are heat or current densities that enter the investigated object over the whole volume given as a density-per-volume or over the surface $N$, given as density-per-area, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we'd like to calculate the mechanical stress induced by heating of the material. In literature, this is usually described with vector valued functions and using nontrivial tensors. We wonder whether it is possible to formulate a meaningful calculation of the mechanical stress in the same form as the equation above, in order to re-use our code that we wrote to solve it. We are aware that this is not possible for all materials and larger stresses/displacements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is it possible to formulate a meaningful calculation of the mechanical stress in the same form as the equation above? For which type of materials and other restrictions would this form be valid?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="8138" LastActivityDate="2018-06-09T12:52:20.837" Title="Simplest meaningful PDE/FEM calculation for mechanical stress due to heat" Tags="|finite-element|pde|solid-mechanics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29699" PostTypeId="2" ParentId="1671" CreationDate="2018-06-10T09:32:20.920" Score="1" Body="&lt;p&gt;Possible methods, that can be used for floating-point compression:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Transpose 4xN for float and 8xN for double + lz77&lt;br&gt;&#xA;Implementation: &lt;a href=&quot;https://github.com/powturbo/TurboTranspose&quot; rel=&quot;nofollow noreferrer&quot;&gt;Floating point compression in TurboTranspose&lt;/a&gt;&lt;br&gt;&#xA;see also error-bounded lossy compression&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Predictor (ex. Finite Context Method) + encoding (ex. &quot;integer compression&quot;).&lt;br&gt;&#xA;Implementation: &lt;a href=&quot;https://github.com/powturbo/TurboPFor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Floating point compression in TurboPFor&lt;/a&gt;&lt;br&gt;&#xA;including special compression for time series.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;when possible, convert all floating point numbers to integers (ex. 1.63 -&gt; 163), then use integer compression&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can test all these methods with your data using the &lt;a href=&quot;https://sites.google.com/site/powturbo/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;icapp&lt;/a&gt;&#xA;tool for linux and windows.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27964" LastEditorUserId="27964" LastEditDate="2018-06-10T09:52:30.043" LastActivityDate="2018-06-10T09:52:30.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29701" PostTypeId="1" CreationDate="2018-06-10T18:49:23.273" Score="1" ViewCount="2162" Body="&lt;p&gt;This is my first post in this forum so please forgive me if it is not the way it should be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is about implementing &quot;Boole's rule&quot; into python. I have succesfully implementet trapezoidal and simpsons, but there is a difference in the results between the first 2 and Boole's rule when I calculate an integral.  The results differ between 1 and 10. Here it's my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; # implementation of 3 different integration techniques&#xA; from math import *&#xA; from scipy import *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;1. : Trapezoidal rule&lt;/h1&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# the trapezodial rule approximates the area under the function as a trapezoid. Mostly it averages the left and right&#xA;# sums of 2 sums given by the Riemann-Integral.&#xA;&#xA;# the approximation is as follows:&#xA; f= lambda x: e**(-x**2)&#xA; g=  lambda x: x**6-3*x**5+10*x**4-20*x**3&#xA; h= lambda x:1/(x**2-3*x+5)&#xA;&#xA;def trapezoidal (f,a,b,n):&#xA;   dx= ((b-a)/2)/n                                       &#xA;   x=a&#xA;   sum = 0&#xA;   while x&amp;lt;b:&#xA;      sum += f(x)*dx&#xA;      x += dx&#xA;   return sum&#xA;&#xA; print(&quot;trapezoidal f1:&quot;,trapezoidal(f,0,1,10000))&#xA; print(&quot;trapezoidal f2:&quot;,trapezoidal(g,0,4,10000))&#xA; print(&quot;trapezoidal f3:&quot;,trapezoidal(h,0,10,10000))&#xA;&#xA;&#xA;#2. : Simpson´s rule&#xA;&#xA;def simpson(f, a, b, n):&#xA;if n % 2: raise ValueError(&quot;n must be even (received n=%d)&quot; % n)&#xA;&#xA;dx = (b - a) / n&#xA;sum = f(a) + f(b)&#xA;&#xA;for i in range(1, n, 2):&#xA;    sum += 4 * f(a + i * dx)&#xA;for i in range(2, n-1, 2):&#xA;    sum += 2 * f(a + i * dx)&#xA;&#xA;return sum * dx / 3&#xA;&#xA;print(&quot;simpson f1:&quot;,simpson(f,0,1,10000))&#xA;print(&quot;simpson f2:&quot;,simpson(g,0,4,10000))&#xA;print(&quot;simpson f3:&quot;,simpson(h,0,10,10000))&#xA;&#xA;# 3. : Boole´s rule:&#xA;&#xA;def boole (f,a,b,n):&#xA;&#xA;       dx = (b - a) / (n-1)&#xA;       sum = 7*(f(a) + f(b))&#xA;       sum += 32*(f(a+dx)+f(b-dx))&#xA;       sum += 12*(f(a+2*dx))&#xA;       return 2*sum * dx / 45&#xA;&#xA;print(&quot;boole f1:&quot;,boole(f,0,1,5))&#xA;print(&quot;boole f2:&quot;,boole(g,0,4,5))&#xA;print(&quot;boole f3:&quot;,boole(h,0,10,5))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used the formula given on Wikipedia but without the error term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\int_{x_1}^{x_5} f(x)\,dx = \frac{2 h}{45}\left( 7f(x_1) + 32 f(x_2) + 12 f(x_3) + 32 f(x_4) + 7f(x_5) \right) + \text{error term}\, .$$&lt;/p&gt;&#xA;" OwnerUserId="27966" LastEditorUserId="9667" LastEditDate="2018-06-20T13:15:52.510" LastActivityDate="2018-06-20T13:15:52.510" Title="Boole's Rule in python" Tags="|python|computational-physics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29702" PostTypeId="1" CreationDate="2018-06-10T19:48:59.213" Score="2" ViewCount="149" Body="&lt;p&gt;I’m trying to figure out how to model the rings of Saturn using a particle system for a gravity simulator that I’m making. Using the code below, I’ve managed to create a, if not perfect, decent ring that rotates around Saturn under the force of the gravity of Saturn and its moons. However, my ring has an axial tilt of 0 whereas in reality the axial tilt of Saturn’s rings is 27 degrees. The problem is that I can’t get the z position and velocity vectors right for the particles that make up the particle system. Would anybody be able to help me figure out how to accomplish this? The code that I'm using to generate the initial state vectors for each particle in the ring can be found below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (let i = 0; i &amp;lt; this.numberOfParticles; i++) {&#xA;&#xA;  const rad = Math.PI * 2 * Math.random();&#xA;  const dist = (25 + 20 * Math.random()) / 32000;&#xA;&#xA;  this.particles.push({&#xA;    x: Math.cos(rad) * dist,&#xA;    y: Math.sin(rad) * dist,&#xA;    z: 0,&#xA;    vx: (Math.cos(rad + Math.PI / 2 + (Math.PI / 180 * 6 - Math.PI / 180 * 12) * 0) * Math.sqrt(500 / dist)) / 120,&#xA;    vy: (Math.sin(rad + Math.PI / 2 + (Math.PI / 180 * 6 - Math.PI / 180 * 12) * 0) * Math.sqrt(500 / dist)) / 120,&#xA;    vz: 0&#xA; });&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27969" LastActivityDate="2018-06-16T16:19:06.977" Title="Simple model of Saturn's rings" Tags="|computational-physics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29706" PostTypeId="2" ParentId="1671" CreationDate="2018-06-11T15:20:28.857" Score="0" Body="&lt;p&gt;Why not just save float32 / float16 ? In numpy,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A.astype( np.float32 )  # 100M: 200 msec imac&#xA;A.astype( np.float16 )  # 100M: 700 msec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These won't do if you're simulating the&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Butterfly_effect&quot; rel=&quot;nofollow noreferrer&quot;&gt;Butterfly effect&lt;/a&gt; in chaos theory,&#xA;but they're understandable, portable, &quot;do not require any work on my part&quot;.&#xA;And compression 2:1 / 4:1 over float64 is hard to beat :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Array type float16 is unsupported in np.linalg&quot;;&#xA;you'll have to expand it to 32 or 64 after reading it in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see how floating-point parameters differ,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;for f in [np.float64, np.float32, np.float16]:&#xA;    print np.finfo(f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a plot of a trivial test case comparing float 64 32 and 16, see&#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/26763/how-much-regularization-to-add-to-make-svd-stable/28865#28865&quot;&gt;here&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="1841" LastActivityDate="2018-06-11T15:20:28.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29707" PostTypeId="2" ParentId="1671" CreationDate="2018-06-12T06:19:42.603" Score="2" Body="&lt;p&gt;We have been using &lt;a href=&quot;https://computation.llnl.gov/projects/floating-point-compression&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;ZFP&quot;&gt;ZFP&lt;/a&gt; with HDF5 for our medical imaging data. It is made for lossy, floating point compression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're running it on literally everything, and have more than 40TB of data stored (and being used!). &#xA;It is fast enough to save our data real-time, and we can specify the required precision, so while the format is lossy, we're not seeing any differences in our final outputs.  &lt;/p&gt;&#xA;" OwnerUserId="7522" LastActivityDate="2018-06-12T06:19:42.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29708" PostTypeId="2" ParentId="18691" CreationDate="2018-06-12T07:40:18.350" Score="2" Body="&lt;p&gt;MIDACO is available in plain C/C++ and many other languages:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.midaco-solver.com/index.php/download/c&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.midaco-solver.com/index.php/download/c&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="27981" LastActivityDate="2018-06-12T07:40:18.350" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29710" PostTypeId="1" CreationDate="2018-06-12T10:09:39.807" Score="6" ViewCount="1001" Body="&lt;p&gt;I'm trying to solve a nonlinear eigenvalue problem in MATLAB, still without success. It's a problem about graphene plasmonics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nonlinear eigenvalue problem is given below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\frac{\epsilon_1}{\kappa_{1||n}}E_{x||n}+\frac{\epsilon_2}{\kappa_{2||n}}E_{x||n}+ \frac{i}{\omega \epsilon_0} \sum_{p=-M}^{M}\hat{\sigma}_{n-p} E_{x||p}=0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\kappa_{m||n}=\sqrt{(q+nG)^2-\epsilon_m (\omega/c)^2}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit&lt;/em&gt;: For those ones who are familiar with Physics, in order to have a clear understanding of the problem, this is a dispersion relation about a graphene sheet between two dielectrics, supporting plasmonic modes of electromagnetic waves. The graphene sheet has a spatially-periodically modulated conductivity. So $\hat{\sigma_{j}}$ are the Fourier-series coefficients of the surface conductivity, $G=2\pi/R$ is the so-called reciprocal lattice vector, when $R$ is the spatial period, $\epsilon_0$ is the vacuum permittivity, $\epsilon_m$ is the relative permittivity of the dielectrics above and below the graphene sheet, $c$ is the speed of light in vacuum, $\omega$ is the angular frequency, $q$ is the wavenumber, $E_{x||n}$ is the $n$-th diffracted order amplitude of the $x$-component electric field (when it is expander in Fourier-Floquet series). $E_{x||n}$ is in a sense the eigenvector of the problem. Below I give a relevant paper from which I saw this relation (page 46, equation 217):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1302.2317.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://arxiv.org/pdf/1302.2317.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameters $\epsilon_1$, $\epsilon_2$, $\epsilon_0$, $G$, $c$, $\hat{\sigma}_{j}$ are known. The problem stands for determining $q$ (wavenumber), $\omega$ (frequency), so as for the system of the first equation to has nontrivial solution. This should lead to a solution of the dispersion relation. This system of course is a $(2M+1) \times (2M+1)$ square system of the form $Ax=0$, where $A=A(\omega, q)$. Let's consider for simplicity that $M=1$. Then the initial equation can be written as below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;\begin{bmatrix}&#xA;\frac{\epsilon_1}{\kappa_{1||-1}}+ \frac{\epsilon_2}{\kappa_{2||-1}} + \frac{j}{\omega \epsilon_0} \hat{\sigma}_0 &amp;amp; \frac{j}{\omega \epsilon_0} \hat{\sigma}_{-1} &amp;amp; \frac{j}{\omega \epsilon_0} \hat{\sigma}_{-2} \\ \frac{j}{\omega \epsilon_0} \hat{\sigma}_{1} &amp;amp; \frac{\epsilon_1}{\kappa_{1||0}}+ \frac{\epsilon_2}{\kappa_{2||0}} + \frac{j}{\omega \epsilon_0} \hat{\sigma}_0 &amp;amp; \frac{j}{\omega \epsilon_0} \hat{\sigma}_{-1} \\ \frac{j}{\omega \epsilon_0} \hat{\sigma}_{2} &amp;amp; \frac{j}{\omega \epsilon_0} \hat{\sigma}_{1} &amp;amp; \frac{\epsilon_1}{\kappa_{1||1}}+ \frac{\epsilon_2}{\kappa_{2||1}} + \frac{j}{\omega \epsilon_0} \hat{\sigma}_0&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;E_{x||-1} \\ E_{x||0} \\ E_{x||1}&#xA;\end{bmatrix} = 0&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is clear that $A$ depends non-linearly from $\omega$, $q$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, I should give a specific value to $\omega$, and then solve the equation $det(A)=0$ (numerically) to find $q$. That's how one should find the exact dispersion relation (which is a curve that matches a wavenumer $q$ to every frequency $\omega$, in a certain frequency spectrum). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should mention that, in the paper above, it is clearly said that for sharp conductivity profiles (which is the case of my interest, for example $\sigma=\begin{cases} \sigma_1, \quad 0&amp;lt;x&amp;lt;d \\ \sigma_2, \quad d&amp;lt;x&amp;lt;R \end{cases} $), this process is poorly convergent. But I want to study how slowly does it converge. I also cannot solve the problem for a sinusodial conductivity profile either, which is the uttermost case of smooth profile, so I guess the problem is computational.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my MATLAB code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  global w % angular frequency - it is global because I pass it in the function fun&#xA;c0=299792458; %speed of light&#xA;f=linspace(25*10^12,45*10^12,1000); %frequency spectrum&#xA;guess=32*2*pi*25*10^12/c0;; % initial guess for the wavenumber&#xA;for x=1:1000 % scanning for every frequency in the spectrum&#xA;w=2*pi*f(x); &#xA;q(x)=fsolve(@fun,guess); % find the wavenumber q via fsolve&#xA;guess=q(x); % next guess is the previous solution (wavenumber)&#xA;end&#xA;plot(q,f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The main function (heart of the code) is below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function F=fun(q)&#xA;global w&#xA;hbc=3.161*10^-26; % hbar*c0, SI units, [J*m]&#xA;c0=299792458; %speed of light&#xA;e0=8.854*10^-12; %vacuum permittivity [F/m]&#xA;e1=1; %dielectric permittivity, air&#xA;e2=2.1; %dielectric permittivity, spacer&#xA;elcharge=1.602*10^-19; % elementary charge, [C] &#xA;kbol=1.38*10^-23; %Boltzmann constant, SI units [J/K]&#xA;hbar=1.054*10^-34; %reduced Planck constant, SI units, [J*s]    &#xA;h=hbar*2*pi; % Planck constant&#xA;t=0.5*10^-12; %[sec]&#xA;T=300; % Temperature [K]&#xA;mc1=0.2*elcharge; %chemical potential of 1st region, [J]&#xA;mc2=0.5*elcharge; %chemical potential of 2nd region, [J]&#xA;w1=10*10^-9; %width of 1st region, [m]&#xA;w2=60*10^-9; %width of 2nd region, [m]&#xA;R=w1+w2; %spatial period&#xA;G=2*pi/R; %reciprocal lattice wavevector&#xA;M=1; % M is the cutoff of the infinite terms of the system, -M&amp;lt;=n&amp;lt;=M&#xA;hbw=hbar*w; %hbar * w&#xA;sg1=i*((kbol*T*elcharge^2)/(pi*(w+i/t)*hbar^2))*((mc1/(kbol*T))+2*log(exp(-mc1/(kbol*T))+1))+i*(elcharge^2/(4*pi*hbar))*log((2*mc1-hbar*(w+i/t))/(2*mc1+hbar*(w+i/t))); %1st region's conductivity (Kubo formula for graphene conductivity)&#xA;sg2=i*((kbol*T*elcharge^2)/(pi*(w+i/t)*hbar^2))*((mc2/(kbol*T))+2*log(exp(-mc2/(kbol*T))+1))+i*(elcharge^2/(4*pi*hbar))*log((2*mc2-hbar*(w+i/t))/(2*mc2+hbar*(w+i/t))); %2nd region's conductivity (Kubo formula)&#xA;for n=-M:M&#xA;        k1(n+M+1)=sqrt((hbc*(q+n*G))^2-e1*hbw^2);&#xA;    k2(n+M+1)=sqrt((hbc*(q+n*G))^2-e2*hbw^2);&#xA;end&#xA;sf0=(sg1*w1/R)+(sg2*w2/R);; %zeroth Fourier component&#xA;%fill the matrix of Fourier components, sf[]&#xA;for l=-2*M:2*M&#xA;        if l~=0&#xA;                sf(l+2*M+1)=sg1*sin(l*G*w1)/(R*l*G)+j*sg1*(cos(l*G*w1)-1)/(R*l*G)-sg2*sin(l*G*w1)/(R*l*G)-j*sg2*(cos(l*G*w1)-1)/(R*l*G); &#xA;        else&#xA;                sf(l+2*M+1)=sf0;&#xA;        end&#xA;end&#xA;%forming the main matrix of the problem, of which the determinant we want to set to zero&#xA;for n=-M:M&#xA;    v0(n+M+1)=(e1/k1(n+M+1))+(e2/k2(n+M+1))+j*sf0/(hbw*e0*c0); %diagonal elements&#xA;end&#xA;A0=diag(v0);&#xA;A=A0;&#xA;for n=1:2*M&#xA;    vu=ones(1,2*M+1-n)*j*sf((4*M+2)/2-n)/(hbw*e0*c0); %upper diagonal elements&#xA;        vd=ones(1,2*M+1-n)*j*sf((4*M+2)/2+n)/(hbw*e0*c0); %down diagonal elements&#xA;    Au=diag(vu,n);&#xA;    Ad=diag(vd,-n);&#xA;    A=A+Au+Ad;&#xA;end&#xA;F=det(A);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, I'm trying to solve this with &lt;code&gt;fsolve&lt;/code&gt;.&#xA;For large cutoff integer $M$, the determinant becomes extremely big (or small, if I scale properly the equations with a constant number). Anyway, the problem seems unstable. Is there a way to fix my code? Or is there a way to treat the problem in a different mathematical way? My knowledge in computational linear algebra is poor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="27982" LastEditorUserId="27982" LastEditDate="2018-06-13T12:14:01.240" LastActivityDate="2019-10-31T10:52:51.473" Title="Nonlinear eigenvalue problem - MATLAB code" Tags="|linear-algebra|matlab|eigenvalues|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="29711" PostTypeId="2" ParentId="25005" CreationDate="2018-06-12T12:48:08.153" Score="2" Body="&lt;p&gt;A full log of the Dekker method run as in [1] confirms your observation and shows that the example is indeed a bad situation for the Dekker method, a combination of a multiple root inside the start interval and just the right interval. Letting it run for much more steps, the output reads&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;       ,        ,         -&amp;gt; initial   : x[ 0] =  -4.000000000000000,  f(x[ 0]) = -25.000000000000000&#xA;a=x[ 0],        ,         -&amp;gt; initial   : x[ 1] =   1.333333333333333,  f(x[ 1]) =   0.481481481481481&#xA;a=x[ 0], b=x[ 1], c=x[ 0] -&amp;gt; secant    : x[ 2] =   1.232558139534884,  f(x[ 2]) =   0.228910661954293&#xA;a=x[ 1], b=x[ 2], c=x[ 0] -&amp;gt; secant    : x[ 3] =   1.141223295849618,  f(x[ 3]) =   0.082592637299225&#xA;a=x[ 2], b=x[ 3], c=x[ 0] -&amp;gt; secant    : x[ 4] =   1.089667203324025,  f(x[ 4]) =   0.032881772315203&#xA;a=x[ 3], b=x[ 4], c=x[ 0] -&amp;gt; secant    : x[ 5] =   1.055564885924540,  f(x[ 5]) =   0.012521380362105&#xA;a=x[ 4], b=x[ 5], c=x[ 0] -&amp;gt; secant    : x[ 6] =   1.034592397360872,  f(x[ 6]) =   0.004827930257963&#xA;a=x[ 5], b=x[ 6], c=x[ 0] -&amp;gt; secant    : x[ 7] =   1.021431369375937,  f(x[ 7]) =   0.001847057878276&#xA;a=x[ 6], b=x[ 7], c=x[ 0] -&amp;gt; secant    : x[ 8] =   1.013276313630701,  f(x[ 8]) =   0.000707382104210&#xA;a=x[ 7], b=x[ 8], c=x[ 0] -&amp;gt; secant    : x[ 9] =   1.008214575350145,  f(x[ 9]) =   0.000270471306102&#xA;a=x[ 8], b=x[ 9], c=x[ 0] -&amp;gt; secant    : x[10] =   1.005081086844448,  f(x[10]) =   0.000103400954756&#xA;a=x[ 9], b=x[10], c=x[ 0] -&amp;gt; secant    : x[11] =   1.003141749908864,  f(x[11]) =   0.000039513380892&#xA;a=x[10], b=x[11], c=x[ 0] -&amp;gt; secant    : x[12] =   1.001942302905629,  f(x[12]) =   0.000015097489725&#xA;a=x[11], b=x[12], c=x[ 0] -&amp;gt; secant    : x[13] =   1.001200628613113,  f(x[13]) =   0.000005767766984&#xA;a=x[12], b=x[13], c=x[ 0] -&amp;gt; secant    : x[14] =   1.000742115041459,  f(x[14]) =   0.000002203347648&#xA;a=x[13], b=x[14], c=x[ 0] -&amp;gt; secant    : x[15] =   1.000458684641088,  f(x[15]) =   0.000000841662903&#xA;a=x[14], b=x[15], c=x[ 0] -&amp;gt; secant    : x[16] =   1.000283495152734,  f(x[16]) =   0.000000321500791&#xA;a=x[15], b=x[16], c=x[ 0] -&amp;gt; secant    : x[17] =   1.000175214373124,  f(x[17]) =   0.000000122805685&#xA;a=x[16], b=x[17], c=x[ 0] -&amp;gt; minimal   : x[18] =   1.000075214373124,  f(x[18]) =   0.000000022629233&#xA;a=x[17], b=x[18], c=x[ 0] -&amp;gt; minimal   : x[19] =   0.999975214373124,  f(x[19]) =   0.000000002457294&#xA;a=x[18], b=x[19], c=x[ 0] -&amp;gt; minimal   : x[20] =   0.999875214373124,  f(x[20]) =   0.000000062283868&#xA;a=x[19], b=x[20], c=x[ 0] -&amp;gt; midpoint  : x[21] =  -1.500062392813438,  f(x[21]) =   9.375077977391534&#xA;a=x[20], b=x[21], c=x[ 0] -&amp;gt; midpoint  : x[22] =  -2.750031196406719,  f(x[22]) =   3.515244786737265&#xA;a=x[21], b=x[22], c=x[ 0] -&amp;gt; midpoint  : x[23] =  -3.375015598203360,  f(x[23]) =  -7.178084118186108&#xA;a=x[23], b=x[22], c=x[23] -&amp;gt; secant    : x[24] =  -2.955483883754270,  f(x[24]) =   0.696492599989121&#xA;a=x[22], b=x[24], c=x[23] -&amp;gt; secant    : x[25] =  -3.006249707390058,  f(x[25]) =  -0.100308033086949&#xA;a=x[24], b=x[25], c=x[24] -&amp;gt; secant    : x[26] =  -2.999858874218545,  f(x[26]) =   0.002257853174204&#xA;a=x[25], b=x[26], c=x[25] -&amp;gt; secant    : x[27] =  -2.999999560010752,  f(x[27]) =   0.000007039826415&#xA;a=x[26], b=x[27], c=x[25] -&amp;gt; minimal   : x[28] =  -3.000099560010753,  f(x[28]) =  -0.001593039470593&#xA;exit by interval length&#xA;a=x[28], b=x[27], c=x[28] -&amp;gt; best value: x[27] =  -2.999999560010752,  f(x[27]) =   0.000007039826415&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The interpretation is that the method follows first the secant method in &lt;span class=&quot;math-container&quot;&gt;$b_i, a_i=b_{i-1}$&lt;/span&gt; towards the double root &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;. Because of the multiplicity the convergence is linear.&lt;/p&gt;&#xA;&lt;p&gt;As there is no exit condition based on the function value, at the moment that the points &lt;span class=&quot;math-container&quot;&gt;$a_i,b_i$&lt;/span&gt; are close enough so that the function values are almost equal the secant root computation fails and Dekker's method nudges the active point over the root with minimal steps. Remember that the method is constructed to find the root by enclosing it in an interval with a sign change in the function values.&lt;/p&gt;&#xA;&lt;p&gt;Finally, the secant is so horizontal or sloped the wrong way that the secant root falls outside of the bracketing interval &lt;span class=&quot;math-container&quot;&gt;$[b_i,c_i]$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; is here still constant &lt;span class=&quot;math-container&quot;&gt;$-4$&lt;/span&gt;, &lt;code&gt;c=x[0]&lt;/code&gt; in the table).&lt;/p&gt;&#xA;&lt;p&gt;After a while the secant root that is still computed close to the double root is safely away from the bracketing interval that the method now leaves the &amp;quot;emergency&amp;quot; mode and switches to the midpoint of the bisection method, until the value at the midpoint changes sign, ending the stalling at &lt;span class=&quot;math-container&quot;&gt;$c_i=-4$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now the double root is completely &amp;quot;forgotten&amp;quot; and the sign-switching root is found in a few secant steps.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;def dekker(f, a, b, tol=1e-7):&#xA;    fa, fb = f(a), f(b)&#xA;    if fa*fb &amp;gt; 0: &#xA;        error(&amp;quot;need sign change in initial interval&amp;quot;)&#xA;    # [a,b] is bracketing interval, establish triangle formation, [b,c] now also bracketing&#xA;    c,fc = a,fa&#xA;    while True:&#xA;        # make b the best. In superlinear mode nothing should happen here&#xA;        if abs(fc) &amp;lt; abs(fb): &#xA;            # A, B, C = B, C, B    # triangle formation&#xA;            a,fa, b,fb, c,fc = b,fb, c,fc, b,fb&#xA;        # atomic step size&#xA;        db = tol&#xA;        if abs(c-b)&amp;lt;2*db: print(&amp;quot;exit by interval length&amp;quot;); break&#xA;        # minimal change in case of stalling&#xA;        h = b + (db if c&amp;gt;b else -db);&#xA;        # elements of the secant step&#xA;        p = fb*(a-b); q = (fb-fa);&#xA;        if p&amp;lt;0: p,q = -p,-q&#xA;        # default for midpoint step&#xA;        v = m = 0.5*(b+c); &#xA;        # secant step if step update is large enough&#xA;        if p &amp;gt; abs(db*q):&#xA;            s = b+p/q&#xA;            # step goes not beyond midpoint, else remain at midpoint&#xA;            if p &amp;lt; q*(v-b): v = s;&#xA;        else: v = h; &#xA;        # shift the secant sequence&#xA;        a,fa, b,fb = b,fb, v,f(v);&#xA;        # Now a &amp;lt; b=v &amp;lt;= m &amp;lt; c or in the other direction. Between a and c there is a sign change&#xA;        # If the sign also changes between b and c, do nothing, linear formation remains.&#xA;        # Else copy a to c, establishing a triangle formation&#xA;        if fb*fc &amp;gt; 0:&#xA;            c,fc = a,fa&#xA;        # Now [b,c] is ensured to be bracketing and (a,b) is ready for the next secant root computation&#xA;    return b, c&#xA;        &#xA;dekker(lambda x: (x+3)*(x-1)**2,-4,4./3, tol=1e-4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2023-11-28T07:20:15.587" LastActivityDate="2023-11-28T07:20:15.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29712" PostTypeId="2" ParentId="29710" CreationDate="2018-06-12T14:50:57.560" Score="9" Body="&lt;p&gt;Given a nonlinear eigenvalue problem of the form $A(\lambda)x = 0$, reducing it to a real equation $\det(A(\lambda))=0$ is known to be a poor method for just the reason you've discovered yourself. The determinant is too ill-behaved for this to work on non-trivial problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even for an ordinary eigenvalue problem $Ax=\lambda x$, if you have a good approximate guess at an eigenvalue $x_0$, a typical method would be something like inverse iteration, rather than Newton's method applied to $\det(A-\lambda I)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is quite some literature on this topic already, you could start by trying some of the different methods listed in [1]. In particular, they discuss different ways to treat $A(\lambda)x=0$ other than through $A$'s determinant, such as through Rayleigh quotient, and list multiple methods starting on p.10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;em&gt;Nonlinear Eigenvalue Problems&lt;/em&gt;, Heinrich Voss, Chapter 115 of the &lt;em&gt;Handbook of Linear Algebra&lt;/em&gt; &lt;a href=&quot;https://www.mat.tuhh.de/forschung/rep/rep164.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://www.mat.tuhh.de/forschung/rep/rep164.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2018-06-12T16:13:41.680" LastActivityDate="2018-06-12T16:13:41.680" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29713" PostTypeId="1" CreationDate="2018-06-13T08:45:25.073" Score="5" ViewCount="153" Body="&lt;p&gt;Using sine and cosine transforms to solve Poisson's equation with Dirichlet boundary conditions seem quite standard nowadays (see, e.g., &lt;a href=&quot;https://doi.org/10.1016/0021-9991(88)90102-7&quot; rel=&quot;noreferrer&quot; title=&quot;here&quot;&gt;here&lt;/a&gt; or Table 2 in &lt;a href=&quot;https://arxiv.org/pdf/1409.8116.pdf&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt;). In the case of Poisson's equation, the multiplication with $-k^2$ preserves the symmetry of the Fourier coefficients such that the same transform can be used for going back to real space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what happens if the differential operator in Fourier space changes the symmetry? E.g., in the case of the transport equation, multiplication with $ik$ would transform even symmetry into odd, such that another transform would be needed for the backtransformation to real space. This should happen for all first or mixed derivatives. Are there any papers/codes around for these cases?&lt;/p&gt;&#xA;" OwnerUserId="27170" LastActivityDate="2018-06-13T08:45:25.073" Title="Discrete sine and cosine transform for mixed derivatives" Tags="|pde|fourier-analysis|fourier-transform|fftw|numerics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29714" PostTypeId="1" AcceptedAnswerId="29715" CreationDate="2018-06-13T09:07:38.663" Score="0" ViewCount="292" Body="&lt;p&gt;I need to conduct some FEM calculations and I am wondering whether parallelization would be a good idea. &#xA;The trouble is that my model is not especially large so it takes few seconds to solve a single problem on my laptop.&#xA;However I need to do optimization (there's a lot of different geometric parameters to test), so I need to run solver many times for slightly different geometry models.&#xA;To my knowledge, most available free software (Elmer, Fenics, OpenFOAM, ...) enable parallel computation only after partitioning domain into subdomains. In this way assembling matrices and solving system of equations for separate subdomains are the tasks for separate processors/cores.&#xA;I doubt whether parallelization at this stage would compensate the overhead. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question no 1.: Is my thinking correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since in the end there will be a lot of calculations (different parameters, different frequencies) I think some kind of parallelization would be of benefit, after all. I'm thinking about embedding FE code in other language to make a parallel split for separate tasks on a higher level. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question no 2.: Is this workflow correct? Does there exist any better approach? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question no 3.: If there is no better approach, then what would be the preferable language to do such things? Python? C? Something else? I'm thinking about using Elmer for FE stuff but I'm not decided yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mainly program in Octave/Matlab and I've never written parallel program before so I don't have enough experience in this field. Unfortunately I don't have enough time do experiments on my own, so I would really appreciate your hints.&lt;/p&gt;&#xA;" OwnerUserId="27985" LastActivityDate="2018-06-13T10:05:12.680" Title="Parallelization of FEM calculations" Tags="|finite-element|parallel-computing|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29715" PostTypeId="2" ParentId="29714" CreationDate="2018-06-13T10:05:12.680" Score="4" Body="&lt;p&gt;Since your computations only take a few seconds, there will be hardly any benefit from parallelizing you FEM computation itself. But since you need to do many individual simulations you can parallelize that part. Instead of running each simulation in series you can run the simulations in parallel $\rightarrow$ one simulation per available core.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python is a good choice to handle your processes. I recently had to implement something similar. The subprocess package came in very handy.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;&#xA;# parallel process&#xA;processes = []&#xA;for i in range(0, N):&#xA;    command = &quot;yourCommand&quot;&#xA;    processes.append(subprocess.Popen(command, shell=True))&#xA;    print(&quot;process started: &quot;, command)&#xA;for p in processes[:]:&#xA;    processes[i].wait()&#xA;print(&quot;done.&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code above starts N processes in parallel and waits for all of them to finish.&lt;/p&gt;&#xA;" OwnerUserId="23586" LastActivityDate="2018-06-13T10:05:12.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29716" PostTypeId="1" CreationDate="2018-06-13T14:25:02.923" Score="2" ViewCount="110" Body="&lt;p&gt;I was reading a book about Matlab (&lt;a href=&quot;https://www.elsevier.com/books/accelerating-matlab-with-gpu-computing/suh/978-0-12-408080-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Accelerating Matlab with GPU computing - A Primer with Examples&quot; by Jung Suh and Youngming Kim&lt;/a&gt;, 2013. Chapter 1.7 Examples). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read an example where it said that: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;by using elemental calculation, matrix calculation, and function call&#xA;  instead of for loops, the speed can be dramatically accelerated.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the case I read, it reduced calculation time from 50s to 0.5s.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;which syntax is more &quot;resources saving&quot; and &quot;faster&quot;: &lt;code&gt;function call&lt;/code&gt;, &lt;code&gt;for&lt;/code&gt;, &lt;code&gt;while&lt;/code&gt;, &lt;code&gt;if&lt;/code&gt;, &lt;code&gt;switch&lt;/code&gt;, and (i.e. search elements in a matrix or set) etc.?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How fast is the element-wise calculation, addition, and matrix calculation etc.?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For the convince of understanding, a short example was given below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%%% First case&#xA;tic;&#xA;B = 0;&#xA;A = 1:1e8;&#xA;for i=1:1000&#xA;    B = B + A(i);&#xA;end&#xA;toc;&#xA;&#xA;%%% Second case&#xA;tic;&#xA;B = 0;&#xA;A = 1:1e8;&#xA;B = sum(A);&#xA;toc;&#xA;&#xA;%%% Third case&#xA;C = ones(1e8,1);&#xA;tic;&#xA;B = 0;&#xA;A = 1:1e8;&#xA;B = A*C;&#xA;toc;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with the following timing results&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Elapsed time is &lt;strong&gt;0.463159&lt;/strong&gt; seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Elapsed time is &lt;strong&gt;0.501524&lt;/strong&gt; seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Elapsed time is &lt;strong&gt;0.532470&lt;/strong&gt; seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above example, all three methods had different but comparable speed.&lt;/p&gt;&#xA;" OwnerUserId="26929" LastEditorUserId="9667" LastEditDate="2018-06-14T16:21:01.703" LastActivityDate="2018-06-14T16:21:01.703" Title="Accelerate computation speed by using a different syntax" Tags="|matlab|gpu|acceleration|" AnswerCount="0" CommentCount="7" ClosedDate="2018-06-13T18:56:12.043" ContentLicense="CC BY-SA 4.0" />
  <row Id="29717" PostTypeId="2" ParentId="29701" CreationDate="2018-06-14T06:59:59.697" Score="1" Body="&lt;p&gt;I think your implementation is right. But the way you implemented it, will give large errors if you integrate over a large interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the way you have implemented the Trapezoidal rule and Simpson's rule, you are able to $n$ number of steps. So even tho the Trapezoidal rule is a two point integration and Simpson's rule is a three point integration, they will give better results, because you apply them $n$ number of times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your implementation of Boole's rule, $n$ can only be equal to five. In this implementation you integrate the entire interval in one go, instead of splitting it up (like you do with the two other methods). Even tho Boole's rule is a five point integration, it cannot outperform the two other methods, because they are applied $n$ times for a given integral, where Boole's rule is only applied once.&lt;/p&gt;&#xA;" OwnerUserId="25082" LastActivityDate="2018-06-14T06:59:59.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29718" PostTypeId="1" CreationDate="2018-06-14T08:30:27.830" Score="1" ViewCount="94" Body="&lt;p&gt;I am looking to solve 1D burgers equation with various random initial conditions. What is the best algorithm to find the exact solution? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One method that is covered in literature is the equal area principle (R. J. LeVeque, Numerical Methods for Conservation Laws). Is this the standard procedure employed in numerics papers on limiters (for ex: &lt;a href=&quot;http://www.sci.utah.edu/~zhongxh/paper/wenolimiter.pdf&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;weno limiter&quot;&gt;this paper)&lt;/a&gt; for the exact solutions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone provide a pseudo-code or algorithm to implement this method to find the shock location and hence the exact solution? &lt;/p&gt;&#xA;" OwnerUserId="11644" LastActivityDate="2018-06-14T08:30:27.830" Title="Equal area algorithm to find shock location" Tags="|hyperbolic-pde|numerics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29721" PostTypeId="1" CreationDate="2018-06-14T18:02:35.713" Score="6" ViewCount="2095" Body="&lt;p&gt;I am writing a finite element code for structural analysis, and I want to implement rigid bodies. How is this usually done? Say that I have a square mesh, with one half of the mesh being defined rigid and the other deformable. How are the forces on the &quot;boundary-nodes&quot; (the nodes shared between the deformable and rigid part of the mesh) transfered to the 6 (3 in 2D) dofs of the rigid body? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read somewhere that you could loop through all the nodes that are shared between the deformable and rigid parts, and sum all the forces (or express them as generalized forces) to get the total force acting on the rigid body. I can see how this would work for explicit finite element, but not for implicit/static since this does not contribute to the stiffness matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any insight in how to implement rigid bodies in my code is appreciated. &lt;/p&gt;&#xA;" OwnerUserId="28001" LastEditorUserId="28001" LastEditDate="2022-09-05T20:21:55.500" LastActivityDate="2022-09-05T20:21:55.500" Title="How are rigid bodies implemented in finite element codes" Tags="|finite-element|computational-physics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29723" PostTypeId="2" ParentId="29721" CreationDate="2018-06-14T21:57:14.433" Score="2" Body="&lt;p&gt;The best way to think of so-called &quot;rigid elements&quot; is as a set of&#xA;constraint equations. Here is what I mean by that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In three dimensions each rigid body has one independent node with&#xA;six degrees-of-freedom-- three translations and three rotations.&#xA;In general, the rigid body can connect to other elements in the&#xA;model at an arbitrary collection of attachment points, each located&#xA;at some distance from the independent node. Since the body is rigid,&#xA;the motion of these attachment nodes can be obtained from simple&#xA;kinematic relations from the independent node. In other words, the&#xA;motion of all the attachment nodes is constrained to the motion of the&#xA;independent node with sets of kinematic equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how does one implement these constraint equations in a finite element&#xA; code? The literature on this topic often refers to these types of constraints&#xA; as multi-point or multi-freedom constraints because the relations &#xA; involve degrees of freedom&#xA; at two or more points (nodes) in the model. There are several ways these&#xA; constraints can be implemented and they are discussed in these two introductory&#xA; notes by Felippa:&#xA;&lt;a href=&quot;https://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch08.d/IFEM.Ch08.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MultiFreedom Constraints I&lt;/a&gt; and &lt;a href=&quot;https://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.Ch09.d/IFEM.Ch09.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MultiFreedom&#xA;Constraints II&#xA;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2018-06-14T21:57:14.433" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29724" PostTypeId="2" ParentId="25014" CreationDate="2018-06-14T23:57:51.163" Score="2" Body="&lt;p&gt;The key component of using FEAST algorithm for computing eigenvalues is specifying an appropriate &lt;em&gt;search contour&lt;/em&gt;. In FEAST, the user can specify circular or elliptical contours (in the newest versions, also custom-shaped ones, which can be important for targeting specific eigenvalues of non-Hermitian problems). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if the size of the contour is too large (not even talking about semi-infiniteness), one would require a very large number of contour points (thus, significantly sacrificing the computation speed). For the whole quadrant, additional issues arise as one has to make a contour finite somehow (probably using some crude estimates for eigenvalue locations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Limiting the search space to &lt;strong&gt;only a quarter of the complex plane&lt;/strong&gt; (2nd quadrant) does not seem enough to justify using Feast algorithm. I don't see the benefits of using FEAST for computing all eigenvalues that lie in the whole quadrant – a more restrictive &lt;em&gt;search contour&lt;/em&gt; is required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References: &lt;a href=&quot;https://arxiv.org/abs/1203.4031&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEAST Manual. Section 2.1 The FEAST algorithm; Section 4.2 &amp;amp; 4.3 Defining a search contour&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-06-14T23:57:51.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29725" PostTypeId="1" AcceptedAnswerId="29728" CreationDate="2018-06-15T01:29:21.523" Score="2" ViewCount="266" Body="&lt;p&gt;I am trying to build a simple solver for the incompressible fluid in a periodic box to learn the spectral method. I am following the textbook (Peyret R. Spectral methods for incompressible viscous flow[M]. Springer Science &amp;amp; Business Media, 2013.) the author presents a Fourier method for the calculation of spatially periodic solutions. The first step is applying Fourier transform to momentum and continuity equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;($A$ is nonlinear term A(U,U), $f$ is source term)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KIUws.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KIUws.jpg&quot; alt=&quot;fft_ns_eqns&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then apply divergence operator in the Fourier space to momentum equation, using continuity equation in Fourier space to eliminate $p$ in momentum equation:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9NbrW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9NbrW.jpg&quot; alt=&quot;eliminate_p&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like it eliminates the necessity of projecting and solving pressure for each time step because the transformed equation satisfied the continuity automatically. However, when I tried to implement the code in Matlab for the 2d periodic domain, I didn't get a reasonable result and I have no idea about how to fix it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question here is, did I miss something nontrivial for implementing this method? Or just some bugs in my code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(this is only part of the code to keep brevity)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%% Construct k&#xA;k1 = 1i*[0:N/2-1 0 -N/2+1:-1];&#xA;k2 = -([0:N/2 -N/2+1:-1]).^2;&#xA;&#xA;% expand to 2d&#xA;k1 = repmat(k1,N,1);&#xA;k2 = repmat(k2,N,1);&#xA;&#xA;%% Solve PDE&#xA;for n = 1:plotgap&#xA;&#xA;    %% Time advance&#xA;    t = t+dt;&#xA;&#xA;    %% ifft velocity field&#xA;    u_r = real(ifft2(u));v_r = real(ifft2(v));&#xA;&#xA;    % gradient&#xA;    ux_r = real(ifft2(k1.*u)); uy_r = real(ifft2(k1'.*u));&#xA;    vx_r = real(ifft2(k1.*v)); vy_r = real(ifft2(k1'.*v));&#xA;&#xA;    % Advection&#xA;    A1 = fft2(u_r.*ux_r + v_r.*uy_r);&#xA;    A2 = fft2(u_r.*vx_r + v_r.*vy_r);&#xA;&#xA;    % Mass conservation&#xA;    C1 = (k1.*A1 + k1'.*A2)./(k2 + k2') .*k1;&#xA;    C2 = (k1.*A1 + k1'.*A2)./(k2 + k2') .*k1';&#xA;    C1(1,1) = 0;C2(1,1) = 0;&#xA;&#xA;    % Diffusion&#xA;    D1 = nu*(k2 + k2') .* u;&#xA;    D2 = nu*(k2 + k2') .* v;&#xA;&#xA;    %% Construct operator&#xA;    operator1 = -A1 + D1 + C1;&#xA;    operator2 = -A2 + D2 + C2;&#xA;&#xA;    %% Update velocity&#xA;&#xA;    if ~exist('tmp1','var'), tmp1 = operator1; end&#xA;    if ~exist('tmpp1','var'), tmpp1 = operator1; end&#xA;    if ~exist('tmppp1','var'), tmppp1 = operator1; end&#xA;    if ~exist('tmp2','var'), tmp2 = operator2; end&#xA;    if ~exist('tmpp2','var'), tmpp2 = operator2; end&#xA;    if ~exist('tmppp2','var'), tmppp2 = operator2; end&#xA;&#xA;    u = u + dt*(55/24*operator1 - 59/24*tmp1 + 37/24*tmpp1 - 3/8*tmppp1);  % 4th -order Adam - bashforth&#xA;    tmppp1 = tmpp1;tmpp1 = tmp1;tmp1 = operator1; % update f values&#xA;    v = v + dt*(55/24*operator2 - 59/24*tmp2 + 37/24*tmpp2 - 3/8*tmppp2);&#xA;    tmppp2 = tmpp2;tmpp2 = tmp2;tmp2 = operator2; % update f values&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28005" LastEditorUserId="20688" LastEditDate="2018-06-15T21:21:48.237" LastActivityDate="2018-06-15T21:21:48.237" Title="build a simple incompressible solver with spectral method" Tags="|fluid-dynamics|simulation|spectral-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29727" PostTypeId="1" CreationDate="2018-06-15T13:45:11.587" Score="4" ViewCount="406" Body="&lt;p&gt;I want to calculate the following expression&#xA;$$&#xA;\exp(-i\Delta t\mathbf{H})&#xA;$$&#xA;where $\mathbf{H}\in\mathbb{C}^{n\times n}$ is a hermitian matrix. Since I have a highly optimized eigensolver in the code I'm working with (ScaLAPACK &amp;amp; Fortran 90) I was thinking about utilizing it by calculating the matrix exponential with the eigenvector ansatz&#xA;$$&#xA;\exp(-i\Delta t\mathbf{H}) = \mathbf{C}\,\mathrm{diag}\Big[\exp(-i\Delta t \epsilon)\Big]\,\mathbf{C}^{-1}&#xA;$$&#xA;where $\mathbf{C}\in\mathbb{C}^{n\times n}$ contains the eigenvectors of $\mathbf{H}$ and $\epsilon$ indicate its eigenvalues. In this case, solving the generalized eigenvalue problem&#xA;$$&#xA;\mathbf{H}\mathbf{C} = \epsilon\mathbf{S}\mathbf{C}&#xA;$$&#xA;yields the eigenvectors and -values. The generalized EVP is reduced to an ordinary EVP via a Cholesky transform of $\mathbf{S}=\mathbf{U}^T\mathbf{U}$,&#xA;\begin{align}&#xA;&amp;amp;~\mathbf{H}\mathbf{C}=\epsilon\mathbf{U}^T\mathbf{U}\mathbf{C}\\&#xA;\Leftrightarrow  &amp;amp; ~\mathbf{U}^{-T}\mathbf{H}\mathbf{C} = \epsilon\mathbf{U}\mathbf{C}\\&#xA;\Leftrightarrow &amp;amp; ~\mathbf{U}^{-T}\mathbf{H}\mathbf{U}^{-1}\mathbf{U}\mathbf{C} = \epsilon\mathbf{U}\mathbf{C}&#xA;\end{align}&#xA;and by setting $\mathbf{H}'\equiv \mathbf{U}^{-T}\mathbf{H}\mathbf{U}^{-1}$ and $\mathbf{C}'\equiv \mathbf{U}\mathbf{C}$:&#xA;$$&#xA;\mathbf{H}'\mathbf{C}'=\epsilon\mathbf{C}'.&#xA;$$&#xA;The eigenvectors are backtransformed via $\mathbf{C}=\mathbf{U}^{-1}\mathbf{C}'$ when the eigenproblem is solved. Not sure about the general case, but here I have non-orthogonal eigenvectors so I can not make use of &#xA;$$&#xA;\mathbf{C}^{-1} = \mathbf{C}^{T}.&#xA;$$&#xA;Considering the paper of Moler and Van Loan (19 dubious ways ...), the eigenvector approach should always work out for a hermitian matrix if I understood it right. But I am not sure how to handle this situation. My next step would be the straightforward numerical calculation of $\mathbf{C}^{-1}$ but I am a bit afraid that I may miss a point here or that there is a better (i.e. cheaper) solution. &lt;/p&gt;&#xA;" OwnerUserId="25510" LastEditorUserId="9667" LastEditDate="2018-06-15T15:51:02.133" LastActivityDate="2018-06-16T22:34:21.233" Title="Matrix exponential of hermitian matrix with eigenvectors from generalized eigenvalue problem" Tags="|linear-algebra|eigensystem|quantum-mechanics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29728" PostTypeId="2" ParentId="29725" CreationDate="2018-06-15T14:53:39.847" Score="2" Body="&lt;p&gt;Your algebra is fine. Periodic boxes are particularly nice for imposing incompressibility: as you found out, you can easily get rid of pressure. [This intimately has to do with the fact that differential operators are diagonal on the Fourier basis, so that solving an otherwise horrendous Laplace equation for pressure is straightforward here.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple remarks on the implementation: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The Nyquist frequency &lt;code&gt;N/2&lt;/code&gt; should be present in &lt;em&gt;both&lt;/em&gt; &lt;code&gt;k2&lt;/code&gt; and &lt;code&gt;k1&lt;/code&gt; (or absent from both)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;k1 = 1i*[0:N/2 -N/2+1:-1]; [KX,KY] = meshgrid (k1); K2=KX.^2+KY.^2&lt;/code&gt; would be a much cleaner way to define all your wavenumber arrays: the wavenumber along x, &lt;code&gt;KX&lt;/code&gt;, along y &lt;code&gt;KY&lt;/code&gt;, and its square &lt;code&gt;K2&lt;/code&gt;. This is not only a question of taste, if would clean your code &lt;em&gt;a lot&lt;/em&gt; by eliminating the need to use transposes.&lt;/li&gt;&#xA;&lt;li&gt;Speaking of transposes, &lt;code&gt;'&lt;/code&gt; is a complex conjugate transpose whereas &lt;code&gt;.'&lt;/code&gt; is a regular transpose. Computing &lt;code&gt;k1'&lt;/code&gt; does not do what you think it does in your code.&lt;/li&gt;&#xA;&lt;li&gt;Your time-stepping scheme seems to be a 4th order Runge-Kutta. I would start with a naive Euler, or Backward Euler, debug that, and then ramp up the complexity of the time-stepping scheme. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I find &lt;a href=&quot;http://www.staff.uni-oldenburg.de/hannes.uecker/pre/030-mcs-hu.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt; to be a goldmine as an initiation to spectral methods and computational fluid dynamics. All the fundamentals are there. If you take the time to read it, understand it, and write your own programs following the guidance it provides, you will have a very solid foundation for writing spectral codes.&lt;/p&gt;&#xA;" OwnerUserId="28007" LastActivityDate="2018-06-15T14:53:39.847" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29729" PostTypeId="1" CreationDate="2018-06-15T18:55:45.420" Score="2" ViewCount="285" Body="&lt;p&gt;Suppose I have two images that contain many objects that are identical, or nearly so. Specifically, say the objects are shifted and scaled, but by differing amounts, from one image to the other. Is there an efficient algorithm for matching the objects? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The prototypical example would be subsequent images from a video stream, where different objects move by a different amount relative to the camera, and we'd like to know which object went were. Coming from an astronomy, where the images are filled with point sources, one method of identifying them is to convolve the base image with the point spread function (the average point source) and look for peaks in the resulting convolution image. So, I imagine it should be possible to shift, rotate, and scale one image, compute the inner product of the results (i.e. multiply pixels, then sum them), and then look for points in the shift/rotate/scale space that are local maxima to say that something in the two images match at that point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this, or something like it, be done efficiently? For instance, would it be more efficient or less efficient to locate candidate objects, cut them out, and then do the shift/rotate/scale space search than just doing it with the whole image (assuming most objects are a small shift/rotate/scale away from their prior position)?&lt;/p&gt;&#xA;" OwnerUserId="8313" LastActivityDate="2018-08-01T14:44:59.427" Title="Is there an efficient algorithm for matching objects in one image to objects in another?" Tags="|image-processing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29730" PostTypeId="1" CreationDate="2018-06-16T07:11:22.610" Score="5" ViewCount="560" Body="&lt;p&gt;I am trying to solve an equation in Python. Basically what I want to do is to solve the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{x^2}\frac{d}{dx}\left(Gam \frac{dL}{dx}\right)+L\left(\frac{a^2x^2}{Gam}-m^2\right)=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the Klein-Gordon equation for a massive scalar field in a Schwarzschild spacetime. We know $m$ and $Gam=x^2-2x$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial/boundary condition that I know are $L\rvert_{2+\epsilon}=1$ and $L\rvert_{\infty}=0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the asymptotic behavior of the equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;L(x\to\infty)\to \frac{e^{\pm(m^2-a^2)x}}{x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, if $a^2&amp;gt;m^2$ we will have oscillatory solutions, while if $a^2 &amp;lt; m^2$ we will have a divergent and a decay solution. What I am interested in is  the decay solution; however when I am trying to solve the above equation transforming it as a system of first-order differential equations and using the shooting method in order to find the $a$ that can give me the behavior that I am interested about, I am always having a divergent solution (for $0&amp;lt;a^2&amp;lt;m^2$). I suppose that it is happening because &lt;code&gt;odeint&lt;/code&gt; is always finding the divergent asymptotic solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to avoid or tell to &lt;code&gt;odeint&lt;/code&gt; that I am interested in the decay solution? If not, do you know a way that I could solve this problem? Maybe using another method for solving my system of differential equations? If yes, which method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically what I am doing is to add a new system of equations for $a$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2a}{dx^2}=0,\\ &#xA;\frac{da}{dx}(2+\epsilon)=0,\\&#xA;a(2+\epsilon)=a_0)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;in order to have $a$ as a constant. Then I am considering different values for $a_0$ and asking if my boundary conditions are fulfilled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hello, let me try to explain a little more my issue&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am incorporating the value at infinity considering the assimptotic behavior, it means that I will have a relation between the field and its derivative. I will post the code for you if it is helpful:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from IPython import get_ipython&#xA;get_ipython().magic('reset -sf')&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.integrate import odeint&#xA;from math import *&#xA;from scipy.integrate import ode&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are initial conditions for Schwarzschild. The field is invariant under reescaling, then I can use $L(2+\epsilon)=1$&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def init_sch(u_sch):&#xA;    om = u_sch[0]&#xA;    return np.array([1,0,om,0]) #conditions near the horizon, [L_c,dL/dx,a,da/dx]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are our system of equations&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def F_sch(IC,r,rho_c,m,lam,l,j=0,mu=0):&#xA;    L = IC[0]&#xA;    ph = IC[1]&#xA;    om = IC[2]&#xA;    b = IC[3]&#xA;&#xA;    Gam_sch=r**2.-2.*r&#xA;&#xA;    dR_dr = ph&#xA;    dph_dr = (1./Gam_sch)*(2.*(1.-r)*ph+L*(l*(l+1.))-om**2.*r**4.*L/Gam_sch+(m**2.+lam*L**2.)*r**2.*L)&#xA;    dom_dr = b&#xA;    db_dr = 0.&#xA;    return [dR_dr,dph_dr,dom_dr,db_dr]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I try for different values of &quot;om&quot; and ask if my boundary conditions are fulfilled. p_sch are the parameters of my model. In general what I want to do is a little more complicated and in general I will need more parameters that in the just massive case. Howeve I need to start with the easiest which is what I am asking here&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p_sch = (1,1,0,0) #[rho_c,m,lam,l], lam and l are for a more complicated case &#xA;ep = 0.2&#xA;ep_r = 0.01&#xA;r_end = 500&#xA;n_r = 500000&#xA;n_omega = 1000&#xA;omega = np.linspace(p_sch[1]-ep,p_sch[1],n_omega)&#xA;r = np.linspace(2+ep_r,r_end,n_r)&#xA;tol = 0.01&#xA;a = 0&#xA;&#xA;for j in range(len(omega)): &#xA;    print('trying with $omega =$',omega[j])&#xA;    omeg = [omega[j]]&#xA;    ini = init_sch(omeg)&#xA;    Y = odeint(F_sch,ini,r,p_sch,mxstep=50000000)&#xA;    print Y[-1,0]&#xA;    #This is my boundary condition that I am implementing. Basically this should be my conditions at &quot;infinity&#xA;    if abs(Y[-1,0]*((p_sch[1]**2.-Y[-1,2]**2.)**(1/2.)+1./(r[-1]))+Y[-1,1]) &amp;lt; tol: &#xA;        print(j,'times iterations in omega')&#xA;        print(&quot;R'(inf)) = &quot;, Y[-1,0])        &#xA;        print(&quot;\omega&quot;,omega[j])&#xA;        omega_1 = [omega[j]] &#xA;        a = 10&#xA;        break           &#xA;    if a &amp;gt; 1:&#xA;        break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically what I want to do here is to solve the system of equations giving different initial conditions and find a value for &quot;a=&quot; (or &quot;om&quot; in the code) that should be near to my boundary conditions. I need this because after this I can give such initial guest to a secant method and try to fiend a best value for &quot;a&quot;. However, always that I am running this code I am having divergent solutions that it is, of course, a behavior that I am not interested. I am trying the same but considering the scipy.integrate.solve_vbp, but when I run the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from IPython import get_ipython&#xA;get_ipython().magic('reset -sf')&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from math import *&#xA;from scipy.integrate import solve_bvp&#xA;&#xA;def bc(ya,yb,p_sch):&#xA;    m = p_sch[1]&#xA;    om = p_sch[4]&#xA;    tol_s = p_sch[5]&#xA;    r_end = p_sch[6]&#xA;&#xA;    return np.array([ya[0]-1,yb[0]-tol_s,ya[1],yb[1]+((m**2-yb[2]**2)**(1/2)+1/r_end)*yb[0],ya[2]-om,yb[2]-om,ya[3],yb[3]])&#xA;&#xA;def fun(r,y,p_sch):&#xA;    rho_c = p_sch[0]&#xA;    m = p_sch[1]&#xA;    lam = p_sch[2]&#xA;    l = p_sch[3]&#xA;&#xA;    L = y[0]&#xA;    ph = y[1]&#xA;    om = y[2]&#xA;    b = y[3]&#xA;&#xA;    Gam_sch=r**2.-2.*r&#xA;&#xA;    dR_dr = ph&#xA;    dph_dr = (1./Gam_sch)*(2.*(1.-r)*ph+L*(l*(l+1.))-om**2.*r**4.*L/Gam_sch+(m**2.+lam*L**2.)*r**2.*L)&#xA;    dom_dr = b&#xA;    db_dr = 0.*y[3]&#xA;    return np.vstack((dR_dr,dph_dr,dom_dr,db_dr))&#xA;&#xA;eps_r=0.01&#xA;r_end = 500&#xA;n_r = 50000&#xA;r = np.linspace(2+eps_r,r_end,n_r)&#xA;y = np.zeros((4,r.size))&#xA;y[0]=1&#xA;tol_s = 0.0001&#xA;p_sch= (1,1,0,0,0.8,tol_s,r_end)&#xA;&#xA;&#xA;sol = solve_bvp(fun,bc, r, y, p_sch)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am obtaining this error: ValueError: &lt;code&gt;bc&lt;/code&gt; return is expected to have shape (11,), but actually has (8,).&#xA;ValueError: &lt;code&gt;bc&lt;/code&gt; return is expected to have shape (11,), but actually has (8,).&lt;/p&gt;&#xA;" OwnerUserId="28010" LastEditorUserId="28010" LastEditDate="2018-06-17T05:05:06.453" LastActivityDate="2018-06-17T05:05:06.453" Title="Avoiding divergent solutions with `odeint`? shooting method" Tags="|python|ode|quantum-mechanics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29731" PostTypeId="1" CreationDate="2018-06-16T13:26:45.810" Score="2" ViewCount="78" Body="&lt;p&gt;I am currently studying eigenvalue problems. I already worked through the &lt;a href=&quot;https://en.wikipedia.org/wiki/Minimax&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimax-principles&lt;/a&gt; and seen why &lt;span class=&quot;math-container&quot;&gt;$\lambda_{h, m} \geq \lambda_m$&lt;/span&gt;, when comparing the eigenvalues of a discretization and the continuous ones.&lt;/p&gt;&#xA;&lt;p&gt;The next lemma tries to establish conditions when we can get convergence between these two, finally arriving at the theorem that eigenvalues converge twice as fast as eigenvectors do. The lemma states the following with &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; being the space we are looking for eigenvectors in, &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; being a discretation of that and &lt;span class=&quot;math-container&quot;&gt;$V_m$&lt;/span&gt; being the space which is spanned by the first &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; eigenvectors:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Define&#xA;&lt;span class=&quot;math-container&quot;&gt;$$C_{h,m} := \inf_{v \in V_m} \frac{\| P_h v \|}{\|v\|}, \quad 1 \leq m \leq N.$$&lt;/span&gt;&#xA;If there exists a &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$C_{h,m} &amp;gt; 0$&lt;/span&gt;, we get&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lambda_m \leq \lambda_{h,m} \leq C_{h,m}^{-2} \, \lambda_m.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is a remark stating that the &lt;span class=&quot;math-container&quot;&gt;$1-C^{-1}_{h,m}$&lt;/span&gt; is a measure for how close &lt;span class=&quot;math-container&quot;&gt;$V_m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P_h V_m$&lt;/span&gt; are and that it is the norm of the inverse of the operator &lt;span class=&quot;math-container&quot;&gt;$P_h$&lt;/span&gt;.  Can you help me how to arrive at this conclusion?&lt;/p&gt;&#xA;&lt;p&gt;My thoughts so far were that the &lt;span class=&quot;math-container&quot;&gt;$C_{h,m}$&lt;/span&gt; are measuring the minimal &lt;span class=&quot;math-container&quot;&gt;$\cos(\theta)$&lt;/span&gt; between &lt;span class=&quot;math-container&quot;&gt;$V_m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$P_h V_m$&lt;/span&gt;. As we want to get &lt;span class=&quot;math-container&quot;&gt;$C_{h,m} \to 1$&lt;/span&gt; later on for &lt;span class=&quot;math-container&quot;&gt;$h \to 0$&lt;/span&gt;, it means that &lt;span class=&quot;math-container&quot;&gt;$P_h V_m$&lt;/span&gt; would become parallel to &lt;span class=&quot;math-container&quot;&gt;$V_m$&lt;/span&gt;. What is the connection to the inverse of the operator &lt;span class=&quot;math-container&quot;&gt;$P_h$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="22586" LastEditorUserId="23791" LastEditDate="2022-02-28T21:22:08.620" LastActivityDate="2022-02-28T21:22:08.620" Title="Geometric interpretation of lemma" Tags="|eigenvalues|computational-geometry|projection|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29732" PostTypeId="2" ParentId="29702" CreationDate="2018-06-16T16:19:06.977" Score="3" Body="&lt;p&gt;A couple of directions and notes to make it work:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Now, you have points &amp;amp; vectors in the XY-plane with the normal $\hat{n}_{xy}=(0,0,1)$.&lt;/li&gt;&#xA;&lt;li&gt;Instead of constructing the points and vector initially in the &lt;em&gt;tilted plane&lt;/em&gt;, you can simply rotate your points &amp;amp; vectors (created in the XY-plane) accordingly&lt;/li&gt;&#xA;&lt;li&gt;For that you will have to find the normal of the &lt;em&gt;tilted plane&lt;/em&gt;: $\hat{n}_{\theta}$ which you can find using basic geometry and $\theta=27^\circ$.&lt;/li&gt;&#xA;&lt;li&gt;Since you will also need a point for rotation, you might notice, that for both the original and tilted planes, the origin $O=(0,0,0)$ stays where it is.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;After figuring out that, you will have enough information to make a proper rotation from the plane with $\hat{n}_{xy}$ to the one with $\hat{n}_{\theta}$.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-06-16T16:19:06.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29733" PostTypeId="2" ParentId="29727" CreationDate="2018-06-16T22:34:21.233" Score="3" Body="&lt;p&gt;I guess your problem is the following. Your working in a non-orthogonal basis $\phi_i$, and solved the time-independent Schrödinger equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf H \mathbf C = E \mathbf S \mathbf C$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here $\mathbf H$ is the Hamiltonian, and $\mathbf S$ is the (positive-definite) overlap matrix with $S_{ij} = \langle{\phi_i},\phi_j\rangle$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you want to propagate the thus obtained initial bound state via the time-dependent Schrödinger equation, which is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \mathbf S \, i\partial_t \mathbf C = \mathbf H \mathbf C$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the overlap matrix $\mathbf S$ is also present in the time-dependent equation. By employing the same Cholesky decomposition as in your OP, the previous equation becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ i\partial_t \mathbf C^\prime = \underbrace{(U^{-T}\mathbf H U^{-1})}_{=:\mathbf H^\prime}\mathbf C^\prime$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to this equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathbf C^\prime(t) = \exp(-i\mathbf H^\prime t) \, \mathbf C^\prime(0)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By solving the time-independent equation, you already have the eigenvalues and eigenvectors of $\mathbf H^\prime$. Use these to calculate the exponential (--I guess you know how), and finally use another backsubstitution step to obtain $\mathbf C(t)$ from $\mathbf C^\prime(t)$ as $\mathbf C(t) = \mathbf U^{-1} \mathbf C^\prime(t)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as in the time-independent problem, the clue is again not to work in the original, non-orthogonal basis, but in the Cholesky-transformed orthogonal basis.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastActivityDate="2018-06-16T22:34:21.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29734" PostTypeId="1" CreationDate="2018-06-17T14:17:54.957" Score="6" ViewCount="334" Body="&lt;p&gt;Consider an $M\times N$ grid with periodicity in both dimensions upon which I want to do some local stencil-like computations using distributed-memory parallelism (MPI). I have two options for decomposing this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) split the grid along one dimension so that each of the $P$ processes gets about $M/P\times N$ or $M\times N/P$ grid points, or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) divide the grid in both dimensions so that each process gets a rectangle of roughly $(M\times N)/\sqrt{P}$ grid points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, approach (2) has a much lower communication bound than (1), but for rectangular grids of arbitrary size on arbitrary numbers of processes, finding a partitioning in 2D that balances the load seems difficult. I cannot restrict myself to allowing only $P=4^n$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the state of the art in terms of these arbitrary 2D decompositions?&lt;/p&gt;&#xA;" OwnerUserId="7241" LastActivityDate="2018-06-21T14:23:07.903" Title="Parallel decomposition of 2D grid with minimal communication" Tags="|parallel-computing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29735" PostTypeId="2" ParentId="29734" CreationDate="2018-06-17T16:35:21.660" Score="7" Body="&lt;p&gt;The basic approach that I am using is &lt;em&gt;weighted orthogonal recursive bisection&lt;/em&gt; (weighted ORB).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(which is usually applied to unstructured meshes or N-body simulations; however, it is still naturally applied to grids in $d$-dimensions)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say (for example), you have a $N^x_0=100\times N^y_0=45$ rectangular grid that has $N^x_0*N^y_0=N_0=4500$ and you want to distribute it among $P=5$ processors. Here, $N_0$ stands for the total number of grid points at the $0$th partitioning level. The numbers are definitely arbitrary but should be illustrative enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Set initial weight at the $0$th partitioning level as $W_0=P=5$.&#xA;We will partition, until $W_\ell=1$ for each partition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First partitioning level ($100\times 45$)&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine the largest dimension that we are going to cut along: $x$.&lt;/li&gt;&#xA;&lt;li&gt;Determine weights and number of grid points for the left and right parts.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Left part &lt;strong&gt;00&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;W_{00}=\left\lfloor{\frac{W_{0}}{2}}\right\rfloor=\left\lfloor{\frac{5}{2}}\right\rfloor=2\\&#xA;\tilde{N}_{00}=N_{0}\frac{W_{00}}{W_{0}}=4500\frac{2}{5}=1800&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Right part &lt;strong&gt;01&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;W_{01}=\left\lceil{\frac{W_{0}}{2}}\right\rceil=\left\lceil{\frac{5}{2}}\right\rceil=3\\&#xA;\tilde{N}_{01}=N_{0}\frac{W_{01}}{W_{0}}=4500\frac{3}{5}=2700&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Now, cut along the $y$ dimension at $n^x_{01}=1800/45=40$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Resulting in &lt;strong&gt;00&lt;/strong&gt;th partition being $40\times45$ (starting at $n^x_{00}=0$) and &lt;strong&gt;01&lt;/strong&gt;th partitioning being $60\times 45$ (starting at $n^x_{01}=40$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since both $W_{00}, W_{01}&amp;gt;1$, the partitioning will continue on both left and right sides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, $\tilde{N}_{\ell}$ is the estimated number of grid points in the partitioning before the cut is done. Actual ${N}_{\ell}$ is very close to $\tilde{N}_{\ell}$, but depends on the grid shape (and its dimensions being even or odd - &lt;em&gt;since bisection is being used&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second partitioning level for &lt;em&gt;00&lt;/em&gt; ($40\times45$):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine the largest dimension to cut along: now $y$&lt;/li&gt;&#xA;&lt;li&gt;Determine weights and number of grid points for the left and right parts.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Left part &lt;strong&gt;000&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;W_{000}=\left\lfloor{\frac{W_{00}}{2}}\right\rfloor=\left\lfloor{\frac{2}{2}}\right\rfloor=1\\&#xA;\tilde{N}_{000}=N_{00}\frac{W_{000}}{W_{00}}=1800\frac{1}{2}=900&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Right part &lt;strong&gt;000&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;W_{001}=\left\lceil{\frac{W_{00}}{2}}\right\rceil=\left\lceil{\frac{2}{2}}\right\rceil=1\\&#xA;\tilde{N}_{001}=N_{00}\frac{W_{001}}{W_{00}}=1800\frac{1}{2}=900&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Now cut along the $y$ dimension at $n^y_{001}=\lfloor900/40\rfloor=22$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Resulting in &lt;strong&gt;000&lt;/strong&gt;th partitioning being $40\times 22$ (starting at $n^x_{000}=0$) and &lt;strong&gt;001&lt;/strong&gt;th partitioning being also $40\times 23$ (starting at $n^x_{001}=22$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since both $W_{000}, W_{001}=1$, the partitioning on this branch tree stops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, in this step $\hat{N}_{00}=900$ is different from the actual $N_{00}=880$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second partitioning level for &lt;em&gt;01&lt;/em&gt; ($60\times45$):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine the largest dimension to cut along: $x$&lt;/li&gt;&#xA;&lt;li&gt;Determine weights and number of grid points for the left and right parts.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Left part &lt;strong&gt;010&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;W_{010}=\left\lfloor{\frac{W_{01}}{2}}\right\rfloor=\left\lfloor{\frac{3}{2}}\right\rfloor=1\\&#xA;\tilde{N}_{010}=N_{01}\frac{W_{010}}{W_{01}}=2700\frac{1}{3}=900&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Right part &lt;strong&gt;011&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;W_{011}=\left\lceil{\frac{W_{01}}{2}}\right\rceil=\left\lceil{\frac{3}{2}}\right\rceil=2\\&#xA;\tilde{N}_{011}=N_{01}\frac{W_{011}}{W_{01}}=2700\frac{2}{3}=1800&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Now cut along the $x$ dimension at $n^x_{011}=900/45+40=60$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Resulting in &lt;strong&gt;010&lt;/strong&gt;th partitioning being $20\times 45$ (starting at $n^x_{010}=n^x_{01}=40$) and &lt;strong&gt;011&lt;/strong&gt;th partitioning being $40\times 45$ (starting at $n^x_{011}=60$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $W_{010}=1$, the partitioning of this branch is finished; however, the right branch has to go to the third level as $W_{011}=2&amp;gt;1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Third partitioning level for &lt;em&gt;011&lt;/em&gt; ($40\times45$):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine the largest dimension to cut along: $y$&lt;/li&gt;&#xA;&lt;li&gt;Determine weights and number of grid points for the left and right parts.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Left part &lt;strong&gt;0110&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;W_{0110}=\left\lfloor{\frac{W_{011}}{2}}\right\rfloor=\left\lfloor{\frac{2}{2}}\right\rfloor=1\\&#xA;\tilde{N}_{0110}=N_{011}\frac{W_{0110}}{W_{011}}=1800\frac{1}{2}=900&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Right part &lt;strong&gt;0111&lt;/em&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;W_{0111}=\left\lceil{\frac{W_{011}}{2}}\right\rceil=\left\lceil{\frac{2}{2}}\right\rceil=1\\&#xA;\tilde{N}_{0111}=N_{011}\frac{W_{0111}}{W_{011}}=1800\frac{1}{2}=900&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Now cut along the $y$ dimension at $n^y_{0111}=\lfloor900/40\rfloor=22$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Resulting in &lt;strong&gt;0110&lt;/strong&gt;th partitioning being $40\times 22$ and &lt;strong&gt;0111&lt;/strong&gt;th partitioning being $40\times 23$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alltogether, we have 5 leaf partitinings:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$N_{000}=880$ corresponding to processor with id $000_2=0$ (binary $\to$ dec)&lt;/li&gt;&#xA;&lt;li&gt;$N_{001}=920$ corresponding to processor with id $001_2=1$&lt;/li&gt;&#xA;&lt;li&gt;$N_{010}=900$ corresponding to processor with id $010_2=2$&lt;/li&gt;&#xA;&lt;li&gt;$N_{0110}=880$ corresponding to processor with id $0110_2=6$&lt;/li&gt;&#xA;&lt;li&gt;$N_{0111}=920$ corresponding to processor with id $0111_2=7$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Rearranging (renumbering) of the processor IDs is required to get to $0\ldots P-1$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the final partitioning visualized:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7AjZc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7AjZc.png&quot; alt=&quot;Partitioning visualization for 5 processors&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The total length of the boundaries between the partitionings is $40+45+45+40=170$ which is smaller than $45*4=180$ in case of simple partitioning along one largest dimension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the references that (as far as I know) presented this method is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://dl.acm.org/citation.cfm?id=145731&quot; rel=&quot;nofollow noreferrer&quot;&gt;J. K. Salmon, &lt;em&gt;Parallel hierarchical N-body methods&lt;/em&gt;, Ph.D. Thesis, Caltech, 1992.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To balance the load even more (920 vs 880), one can assign half of the grid points on the 22/23 border to the left (&lt;strong&gt;000&lt;/strong&gt;) and the other half to the right (&lt;strong&gt;001&lt;/strong&gt;). The case for &lt;strong&gt;0110&lt;/strong&gt; and &lt;strong&gt;0111&lt;/strong&gt; is handled the same way. That introduces a little bit more complicated borders (with certain consequences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/b4hdl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b4hdl.png&quot; alt=&quot;Partitioning visualization for 5 processors also using border-balancing&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-06-21T14:23:07.903" LastActivityDate="2018-06-21T14:23:07.903" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29737" PostTypeId="1" CreationDate="2018-06-18T13:39:23.890" Score="4" ViewCount="233" Body="&lt;p&gt;I need to compute a bilinear form for a set of left and right vectors&#xA;$$ w_k = \sum_{i,j} V_{ik}^*A_{ij}U_{jk},$$&#xA;where $A_{ij}\in\mathbb{R}$ and $U_{jk}, V_{ik} \in \mathbb{C}$ (Assume that all the matrix sizes make sense). This can be obtained in one line of MATLAB code&#xA;$$ \texttt{w=diag(V'*A*U);} $$&#xA;or&#xA;$$ \texttt{w=sum(conj(V).*(A*U), 1);} $$&#xA;My question is, what would be the most straight forward FORTRAN implementation of this operation using proper LAPACK/BLAS, considered that $A_{ij}$ is originally given as a real array and $U_{jk}, V_{ik}$ are stored as complex arrays?&lt;/p&gt;&#xA;" OwnerUserId="17612" LastActivityDate="2018-07-21T03:46:17.507" Title="Compute bilinear form with LAPACK" Tags="|linear-algebra|fortran|lapack|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29738" PostTypeId="2" ParentId="29710" CreationDate="2018-06-18T18:02:21.507" Score="4" Body="&lt;p&gt;I would say that you can find the dispersion equations in the form $\omega = f(k)$ or vice-versa $k = g(\omega)$, they are not exactly equivalent. In the second form you can find complex wavenumbers. They are real for a region with propagation modes, imaginary for a region without propagation (bandgap), and they are complex for regions with attenuation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that @Kirill answer is the best approach. Nevertheless, I have used another approach in the past. We can write our dispersion equation as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(\omega, k) = 0\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where we want to find all the points in a region $[0, \omega_\max] \times [0, \pi/R]$ that satisfy this relationship. If we consider a grid over this region, we can use an algorithm like &lt;a href=&quot;https://en.wikipedia.org/wiki/Marching_squares&quot; rel=&quot;nofollow noreferrer&quot;&gt;marching squares&lt;/a&gt; for this purpose. Similar algorithms are used in contour plotting routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would illustrate this approach for the dispersion relation of a bilayer material:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\cos(2dk) = \cos\left(\frac{\omega d}{c_1}\right)\cos\left(\frac{\omega d}{c_2}\right) - \frac{(\rho_1 c_1)^2 + (\rho_2 c_2)^2}{2\rho_1 \rho_2 c_1 c_2}\sin\left(\frac{\omega d}{c_1}\right)\sin\left(\frac{\omega d}{c_2}\right)\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $2d = R$, $c_i$ the speed for layer $i$, $\rho_i$ the density for layer $i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following (Python) snippet does the job.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division, print_function&#xA;from numpy import linspace, sqrt, pi, sin, cos, meshgrid&#xA;import matplotlib.pyplot as plt&#xA;&#xA;plt.rcParams[&quot;mathtext.fontset&quot;] = &quot;cm&quot;&#xA;plt.rcParams[&quot;font.size&quot;] = 12&#xA;&#xA;&#xA;def lhs_dispersion_bilayer(omega, d, c1, c2, rho_1, rho_2):&#xA;    factor = ((rho_1*c1)**2 + (rho_2*c2)**2)/(2*rho_1*rho_2*c1*c2)&#xA;    return cos(omega*d/c1)*cos(omega*d/c2) - factor*sin(omega*d/c1)*sin(omega*d/c2)&#xA;&#xA;&#xA;d = 0.5&#xA;mu_1 = 2.76e10&#xA;mu_2 = 3.47e10&#xA;rho_1 = 2770&#xA;rho_2 = 8270&#xA;&#xA;c1 = sqrt(mu_1/rho_1)&#xA;c2 = sqrt(mu_2/rho_2)&#xA;&#xA;omega = linspace(0, 13, 101)*c1/(2*d)&#xA;k = linspace(0, pi, 101)&#xA;k, omega = meshgrid(k, omega)&#xA;FS = cos(2*d*k) - lhs_dispersion_bilayer(omega, d, c1, c2, rho_1, rho_2)&#xA;plt.contour(k/pi, 2*d*omega/c1, FS, [0])&#xA;plt.xlabel(&quot;$2dk/\pi$&quot;, fontsize=16)&#xA;plt.ylabel(&quot;$2d\omega/c_1$&quot;, fontsize=16)&#xA;plt.ylim(0, 13)&#xA;plt.savefig(&quot;bilayer_disp.png&quot;, dpi=150)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And we obtain the following, that is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hyc5w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hyc5w.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same approach can be done in MATLAB, or other programming languages. The solution depend on how fine is the mesh. Also, the results from this approach can be used as seeds for a Newton-Raphson iteration, if high precission is required.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-06-18T18:02:21.507" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29739" PostTypeId="1" CreationDate="2018-06-19T04:54:02.477" Score="1" ViewCount="38" Body="&lt;p&gt;Suppose $F(u,v)$ is the center frequency Fourier representation of some $f(x,y)$ in 2D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;f(x,y)=\int\limits_{-\infty}^{\infty}\int\limits_{-\infty}^{\infty}F(u,v)e^{2\pi i (xu+yv)}dudv&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle for the same $f(x,y)$, the representation could also be $F(u-u_0, v-v_0)$, where $u_0$ and $v_0$ are arbitrarily shifted frequencies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now assume the fact is we already have a representation $F(u-u_0, v-v_0)$ and coefficients(arranged as a matrix) and also we know the shift $u_0$ and $v_0$, how can we derive the coefficients of $F(u,v)$ corresponding to the same $f(x,y)$?&lt;/p&gt;&#xA;" OwnerUserId="4532" LastEditorUserId="20688" LastEditDate="2018-06-19T13:14:11.393" LastActivityDate="2018-06-19T13:14:11.393" Title="How to numerically transform a 2D Fourier spectrum with arbitrary frequency shift to center frequency?" Tags="|computational-physics|fourier-analysis|image-processing|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29740" PostTypeId="1" CreationDate="2018-06-19T12:57:44.673" Score="1" ViewCount="84" Body="&lt;p&gt;I have a non-linear least-squares problem to solve and with my current modeling, the solver is either very slow or does not converge to a correct solution.&lt;/p&gt;&#xA;&lt;p&gt;For the problem, I need to minimize energy on a set of triangles, which is defined by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_f E_d(f) + \sum_e E_s(e)$$&lt;/span&gt;&#xA;with faces given by their vertices &lt;span class=&quot;math-container&quot;&gt;$f=((u_1,v_1), (u_2,v_2), (u_3,v_3))$&lt;/span&gt;&#xA;and shared edges given by the vertices of both adjacent triangles &lt;span class=&quot;math-container&quot;&gt;$e=((u_1, v_1)_{T_{1}}, (u_2, v_2)_{T_{1}}, (u_2, v_2)_{T_{2}}, (u_2, v_2)_{T_{2}})$&lt;/span&gt;, where 1,2,3 are local indices for the triangle/edge and the vertices are from a common set &lt;span class=&quot;math-container&quot;&gt;$X = \{(u_1, v_1), \dots, (u_{|V|}, v_{|V|})\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The edge indices are defined such that for the inner edges of a manifold mesh holds &lt;span class=&quot;math-container&quot;&gt;$(u_1, v_1)_{T_{1}}=(u_1, v_1)_{T_{2}},\ (u_2, v_2)_{T_{1}}=(u_2, v_2)_{T_{2}}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am using the &lt;a href=&quot;http://ceres-solver.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ceres solver&lt;/a&gt; using the line search algorithm and modeling the problem as two residual blocks of size &lt;span class=&quot;math-container&quot;&gt;$|F|$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|E|$&lt;/span&gt;, defined by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{E}_f(\vec{X}) = (E_f(f_1), \dots, E_f(f_{|F|})) \\&#xA;\vec{E}_s(\vec{X}) = (E_s(e_1), \dots, E_s(e_{|E|}))&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This works for many inputs but is converging slowly and sometimes to local minima.&lt;/p&gt;&#xA;&lt;p&gt;To model it using more blocks, I tried to create one block per triangle and one block per edge, leading to one block per face and one block per edge having only one residual&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_{f_i}(\vec{X}) = E_f(f_i) \\&#xA;E_{e_j}(\vec{X}) = E_s(e_j) \\&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$E_{f_i}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E_{e_j}$&lt;/span&gt; are only depending on the 6 respectively 8 values which are relevant for their energy, which should result in a sparse problem which is hopefully fast to solve.&lt;/p&gt;&#xA;&lt;p&gt;As far as I understood the problem modeling in the ceres solver, modeling the problem this way should be possible and probably the faster way to solve it, but I get totally different results and the result of the modeling using separate blocks seems completely wrong.&lt;/p&gt;&#xA;&lt;p&gt;What can be my problem here and what is the best way to model such a problem for fast solving?&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;(The problem is based on the methods in the &lt;a href=&quot;http://igl.ethz.ch/projects/autocuts/autocuts-SA-2017-poranne-et-al.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;autocuts&lt;/a&gt; parametrization paper. The solving process there is iterated a few times sharpening a smoothing term in &lt;span class=&quot;math-container&quot;&gt;$E_s$&lt;/span&gt;, but the question is only about a single iteration.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="20688" LastEditDate="2021-12-25T08:39:11.960" LastActivityDate="2021-12-25T08:39:11.960" Title="How to model a non-linear least-squares problem for triangles" Tags="|nonlinear-equations|least-squares|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29741" PostTypeId="2" ParentId="24428" CreationDate="2018-06-19T13:40:01.990" Score="3" Body="&lt;p&gt;In &lt;a href=&quot;https://scicomp.stackexchange.com/q/10842/20688&quot;&gt;this question, regarding fast eigenvalue/SVD solver for structured matrices&lt;/a&gt;, it was adviced by @GoHookies to look into the paper:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1016/j.laa.2007.05.027&quot; rel=&quot;nofollow noreferrer&quot;&gt;W. Xu, S. Qiao, &quot;A fast symmetric SVD algorithm for square Hankel matrices&quot;, &lt;em&gt;Lin. Alg. Appl.&lt;/em&gt;, vol. 428, no. 2–3, pp. 550–563, Jan. 2008&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;that also has a follow-up&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1016/j.laa.2008.01.012&quot; rel=&quot;nofollow noreferrer&quot;&gt;K. Brown, S. Qiao, and Y. Wei, &quot;A Lanczos bidiagonalization algorithm for Hankel matrices,&quot;  &lt;em&gt;Lin. Alg. Appl.&lt;/em&gt;, vol. 430, no. 5–6, pp. 1531–1543, Mar. 2009&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Those papers describe techniques (or parts of the required algorithm) to speed-up the calculation of SVD for structured matrices. The application to Hankel matrices naturally extends to the Toeplitz kernel. In general, they use fast FFT-based matrix-vector products for structured matrices applied to Lanczos iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also a &lt;a href=&quot;http://www.cas.mcmaster.ca/~qiao/software/gfhsvd/matlab/readme.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matlab package&lt;/a&gt; from the same group of authors available online (with some additional explanation and references) that implements their algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally played with this package for my applications, but I am still yet to confirm the benefits and claimed complexity savings (slow side work in progress).&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-06-19T13:40:01.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29742" PostTypeId="1" CreationDate="2018-06-19T15:27:31.907" Score="5" ViewCount="288" Body="&lt;p&gt;How could I go about solving the continuity equation below in 2D?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \rho}{\partial t} + \nabla \cdot \left(\rho u\right)=0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw that a similar question was posted here: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/1960/a-good-finite-difference-for-the-continuity-equation&quot;&gt;A good finite difference for the continuity equation&lt;/a&gt;, but that only discussed the 1-D case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In it, one of the answers defines incoming, and outgoing fluxes, as&#xA;$$\Phi_{i+1/2} = \dfrac{u_{i+1/2}+|u_{i+1/2}|}{2}\rho_{i} &#xA;              + \dfrac{u_{i+1/2}-|u_{i+1/2}|}{2}\rho_{i+1}\\&#xA;\Phi_{i-1/2} = \dfrac{u_{i-1/2}+|u_{i-1/2}|}{2}\rho_{i-1} &#xA;              + \dfrac{u_{i-1/2}-|u_{i-1/2}|}{2}\rho_{i}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With these, one could use the finite difference scheme&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dfrac{\partial(\rho)}{\partial t}=-\dfrac{\partial(\rho u)}{\partial x} = -\dfrac{\partial \Phi}{\partial x} \approx -\dfrac{\Delta \Phi}{\Delta x} \approx -\dfrac{\Phi_{i+1/2}-\Phi_{i-1/2}}{\Delta x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that when I use a naive generalization of this, namely defining &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Phi_{i+1/2,j;x} = \dfrac{u_{i+1/2,j;x}+|u_{i+1/2,j;x}|}{2}\rho_{i,j} &#xA;              + \dfrac{u_{i+1/2,j;x}-|u_{i+1/2,j;x}|}{2}\rho_{i+1,j}\\&#xA;\Phi_{i,j+1/2;y} = \dfrac{u_{i,j+1/2;y}+|u_{i,j+1/2;y}|}{2}\rho_{i,j} &#xA;              + \dfrac{u_{i,j+1/2;y}-|u_{i,j+1/2;y}|}{2}\rho_{i,j+1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and taking&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\dfrac{\partial (\rho u_x)}{\partial x}  \approx \frac{\Phi_{i+1/2,j;x}-\Phi_{i-1/2,j;x}}{\Delta x}$$&#xA;and&#xA;$$\dfrac{\partial (\rho u_y)}{\partial y}  \approx \frac{\Phi_{i,j+1/2;y}-\Phi_{i,j-1/2;y}}{\Delta y}$$&#xA;my density diverges. What scheme could I use to solve this equation properly in 2D?&lt;/p&gt;&#xA;" OwnerUserId="20177" LastEditorUserId="20177" LastEditDate="2018-06-20T07:42:39.183" LastActivityDate="2018-06-20T07:42:39.183" Title="A good 2D finite difference for the continuity equation" Tags="|finite-difference|fluid-dynamics|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="29743" PostTypeId="1" AcceptedAnswerId="29744" CreationDate="2018-06-19T16:07:02.137" Score="4" ViewCount="514" Body="&lt;h1&gt;Background&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;We know how to solve the following minimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\min_{X} \lVert AX - B \rVert_F^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;But what about the extended version?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\min_{X} \lVert A&#xA;\begin{bmatrix}&#xA;X &amp;amp; X^2 &#xA;\end{bmatrix} - \begin{bmatrix}&#xA;B_1 &amp;amp; B_2 &#xA;\end{bmatrix} \rVert_F^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any analytical solution to this? Or we have to use gradient descent like method?&lt;/p&gt;&#xA;" OwnerUserId="16859" LastEditorUserId="20417" LastEditDate="2018-06-21T16:04:24.603" LastActivityDate="2018-06-21T16:13:24.697" Title="How to solve the following Frobenius norm-minimization problem?" Tags="|linear-algebra|optimization|nonlinear-programming|least-squares|matrix-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29744" PostTypeId="2" ParentId="29743" CreationDate="2018-06-20T13:03:42.997" Score="4" Body="&lt;p&gt;$$\big\| \mathrm A \begin{bmatrix} \mathrm X &amp;amp; \mathrm X^2\end{bmatrix} - \begin{bmatrix} \mathrm B_1 &amp;amp; \mathrm B_2\end{bmatrix} \big\|_{\text{F}}^2 = \underbrace{\| \mathrm A \mathrm X - \mathrm B_1 \|_{\text{F}}^2}_{=: f_1 (\mathrm X)} + \underbrace{\| \mathrm A \mathrm X^2 - \mathrm B_2 \|_{\text{F}}^2}_{=: f_2 (\mathrm X)}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everybody knows that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla f_1 (\mathrm X) = 2 \, \mathrm A^\top \big( \mathrm A \, \mathrm X - \mathrm B_1 \big)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge is to compute $\nabla f_2 (\mathrm X)$. Since $f_2$ is quartic in $\mathrm X$, the gradient $\nabla f_2$ should be cubic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_2 (\mathrm X + h \mathrm V) = \cdots = f_2 (\mathrm X) + 2 h \, \mbox{tr} \big( \left( \mathrm X \mathrm V + \mathrm V \mathrm X \right)^\top \mathrm A^\top \left( \mathrm A \mathrm X^2 - \mathrm B_2 \right) \big) + o (h^2)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing the &lt;strong&gt;directional derivative&lt;/strong&gt; of $f_2$ in the direction of $\mathrm V$ at $\mathrm X$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\lim_{h \to 0} \frac{f_2 (\mathrm X + h \mathrm V) - f_2 (\mathrm X)}{h} = 2 \, \mbox{tr} \big( \left( \mathrm X \mathrm V + \mathrm V \mathrm X \right)^\top \mathrm A^\top \left( \mathrm A \mathrm X^2 - \mathrm B_2 \right) \big)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extracting the &lt;strong&gt;gradient&lt;/strong&gt; $\nabla f_2 (\mathrm X)$ from the (Frobenius) inner product,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\boxed{\nabla f_2 (\mathrm X) = 2 \, \mathrm X^\top \mathrm A^\top \big( \mathrm A \,\mathrm X^2 - \mathrm B_2 \big) + 2 \, \mathrm A^\top \big( \mathrm A \,\mathrm X^2 - \mathrm B_2 \big) \mathrm X^\top}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is indeed cubic in $\rm X$. Using the &lt;a href=&quot;http://mathworld.wolfram.com/Anticommutator.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;anticommutator&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla f_2 (\mathrm X) = \big\{ \mathrm X^\top, 2 \, \mathrm A^\top \big( \mathrm A \,\mathrm X^2 - \mathrm B_2 \big) \big\}$$&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2018-06-21T16:13:24.697" LastActivityDate="2018-06-21T16:13:24.697" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29745" PostTypeId="1" CreationDate="2018-06-20T22:44:17.533" Score="2" ViewCount="641" Body="&lt;p&gt;I am writing a 3d linear finite element code to solve the standard linear elasticity equation on a tetrahedron mesh of a gearbox. Notice that, the two rectangular plates above the gearbox are fixed, which means their displacement are zeros.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VaNdE.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VaNdE.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard linear elasticity model is &#xA;$$&#xA;-\mu\Delta \bf u - (\lambda + \mu)\nabla \text{div} \bf u = \bf f &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\bf u$ is the displacement, $\mu$ and $\lambda$ are the Lamé parameters. Then I get the weak formulation as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\int_\Omega \sigma(u):\varepsilon \rm(v) dx = \int_\Omega fv\rm d x&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\sigma$ is the stress and $\varepsilon$ is the strain. I use linear finite element to discrete above weak formulation and get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;KU=F&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $K$ is the stiffness matrix, $U$ is the displacement vector, and $F$ is the loads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At last, I want to use RBE2 element to add some constraints to the above finite element model. I add an extra node $A$ to the tetrahedron mesh, and connect it to a boundary node sets $\{B_i\}_{i=1}^N$ of the tetrahedron mesh. Then I apply a load on node $A$, include a force $F$ and a moment $M$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried my best to search a document which can explain the details of the mathematical and physical of RBE2 element clearly but got nothing. So I need your suggestion and help, thanks very much.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can take a look at &lt;a href=&quot;http://www.aersys.com/AERSYS-7015.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt;, which explain RBE3 element very clearly but the RBE2 element is still not clear for my case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;For RBE2 element, the $A$ is an independent node, which displacements (transition and rotation) are unknown. Furthermore, the displacements of $\{B_i\}_{i=1}^N$ are also unknown. I just know the concentrated load $F$ and moment $M$ on $A$, so my question is how to apply the $F$ and $M$ into $KU=F$?&lt;/p&gt;&#xA;" OwnerUserId="28027" LastEditorUserId="28027" LastEditDate="2018-06-22T03:20:13.773" LastActivityDate="2018-06-22T03:20:13.773" Title="What is the mathematical and physical principle behind of RBE2 element？" Tags="|finite-element|solid-mechanics|constraints|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29746" PostTypeId="2" ParentId="29737" CreationDate="2018-06-21T02:54:00.857" Score="1" Body="&lt;p&gt;I would use &lt;code&gt;?gemm&lt;/code&gt; for the matrix product &lt;code&gt;*&lt;/code&gt; &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-fortran-gemm#90EAA001-D4C8-4211-9EA0-B62F5ADE9CF0&quot; rel=&quot;nofollow noreferrer&quot;&gt;(MKL reference)&lt;/a&gt; and &lt;code&gt;v?mul&lt;/code&gt; for the Hadamard product &lt;code&gt;.*&lt;/code&gt; &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-fortran-v-mul&quot; rel=&quot;nofollow noreferrer&quot;&gt;(MKL reference)&lt;/a&gt;. As said before, you would have to cast everything to complex to the best of my knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supposing you work in double precision, something along the lines of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Integer, Parameter :: dp = kind(1.0d0)&#xA;Real(dp), Dimension(:,:), Allocatable :: V_real&#xA;Complex(dp), Dimension(:,:), Allocatable :: A, U, V&#xA;Complex(dp), Dimension(:,:), Allocatable :: C, D ! some auxilliary arrays&#xA;Complex(dp), Dimension(:), Allocatable :: w&#xA;!allocate and define everything&#xA;!convert V from real to complex:&#xA;V = Cmplx(V_real, 0._dp, kind=dp)&#xA;!start by computing C = A*U with ?gemm&#xA;Call zgemm(..., A, ..., U, ... C, ...)&#xA;!Then compute w = sum(conj(V).*C) with vzmul&#xA;V = Conjg (V)&#xA;Call vzmul (n, V, C, D)&#xA;w = sum(D,1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You may not need the auxilliary arrays &lt;code&gt;C&lt;/code&gt; and &lt;code&gt;D&lt;/code&gt; if you can overwrite some of the original arrays &lt;code&gt;A&lt;/code&gt;, &lt;code&gt;U&lt;/code&gt;, &lt;code&gt;V&lt;/code&gt;, and if the shape are compatible (but I assumed that you could not do that).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course for both routine/function there are quite some book-keeping arguments that you need to figure out using the reference (essentially to indicate the logical and physical shape of your arrays). I mentionned intel's BLAS because I am familiar with it, it is free, optimized for intel machines, and the documentation is easy to browse. Further, it features &lt;code&gt;v?mul&lt;/code&gt;, which I am not sure exists in the (non intel) regular BLAS.&lt;/p&gt;&#xA;" OwnerUserId="28007" LastActivityDate="2018-06-21T02:54:00.857" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29747" PostTypeId="1" CreationDate="2018-06-21T05:05:35.260" Score="11" ViewCount="226" Body="&lt;p&gt;&lt;strong&gt;My situation.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a function of a complex variable $f(z)$ defined through a complicated integral. What I am interested in is the value of this function on the imaginary axis. I have numerical access to this function on the following ribbon: $z=(x,y)\in (-\infty,\infty)\times[-1,1]$. Formally the integral expression is divergent outside this domain, and therefore I need an analytic continuation. To sum up my situation in a picture,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cJcyd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cJcyd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I know about $f(z)$ on this ribbon from numerics:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It is simultaneously symmetric about the imaginary and real axes. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It decays to zero at $Re(z)\rightarrow\infty$. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It blows up near $z=\pm i$. It could be pole or a branch point, I don't know. I suspect the nature of this singularity (and possibly all other isolated singularities of the analytic continuation) depends on the specific parameterization $\xi$ of this function (see integral below for details)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In fact it looks very similar to a $\text{sech}^2(z)$ or a $1/(1+z^2)^{2n}$ when plotted. Here is a plot of the real part:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/In1jo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/In1jo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, given the sheer amount of information I have about the function (total numerical access to it on that ribbon), is there some way for me to numerically calculate an approximation to this function along the imaginary axis? I am using Mathematica by the way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I am interested in the values along the imaginary axis is because I need to evaluate the following Fourier transform of this function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\bar{f}(t)=\int_{-\infty}^{\infty} dx\, e^{itx}\frac{1}{x^2+x^2_0} f(x) \tag{1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for &lt;strong&gt;large&lt;/strong&gt; values of $t$, which in my case is actually on the order of $10$. Although I know the integrand well, this Fourier transform is formidably oscillatory, so the only other way I know how to calculate this is by a Contour integration.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I have Tried.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I have actually tried to calculate the ultimate highly oscillatory integral, eq. (1). Evaluating eq. (1) for a single value of 't' takes a few hours to compute. I have carried out a few of these integrals already and the results actually make sense, but I would like an alternative approach.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have tried analytically continuing with Pade approximants, but this is also computationally expensive, but not as much as direct evaluation. More importantly, I could not establish convergence with increasing order of the approximants (nor the average of their partial sums!), which is in contrast with how my tests with simple functions like $\text{sech}^2(z)$ went (I could easily get very quick convergence on wide ranges of the complex $z$-plane with simple test functions).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have tried symbolic integration to no avail. I have tried massaging the integrand into a more digestible form for Mathematica, but my attempts have not succeeded.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The offending integral.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $k_4$,$k_{\perp}$, $\xi$, and $\alpha$ be positive real numbers while $E$ is the complex number we're interested in (plays the role of $z$ in the previous discussion). Define:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;p_1^2&amp;amp;=\left(k_4+\frac{1}{2}E\right)^2+k_{\perp}^2+\alpha^2\\&#xA;p_2^2&amp;amp;=\left(k_4-\frac{1}{2}E\right)^2+k_{\perp}^2+(1-\alpha)^2&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The integral I'm interested in is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{align}&#xA;f(E;\,\alpha,\xi)&amp;amp;=\int_{-\infty}^{\infty} dk_4 \int_{0}^{\infty} d(k_{\perp}^2)\left[\frac{\alpha (1+p_1^2)^{3\xi/2}(1+p_2^2)^{\xi/2}}{(1+p_1^2(1+p_1^2)^{2\xi})(1+p_2^2(1+p_2^2)^{2\xi})}+\\&#xA;\,\,\,\,\,\,\,\,\,\,\,\,\,+(p_1\leftrightarrow p_2)\right]&#xA;\end{align}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I have suppressed functional dependence notation in the integrand for brevity. I'm particularly interested in the values $\xi=1,2,3$, the range $0&amp;lt;\alpha&amp;lt;1$, and (as stated above) the Fourier transform (1) for $t~10$.&lt;/p&gt;&#xA;" OwnerUserId="9562" LastEditorUserId="9562" LastEditDate="2018-06-24T23:26:23.453" LastActivityDate="2018-06-24T23:26:23.453" Title="Numerically Recovering Imaginary Part of Analytic Continuation from Real Part" Tags="|integration|mathematica|complex-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29749" PostTypeId="1" CreationDate="2018-06-21T13:59:01.580" Score="5" ViewCount="352" Body="&lt;p&gt;Recently, I started to develop some codes that use NURBS (general things I intend to use/already using: spline generation, interpolation, grids, isolines, closest-point find, and many others), both for curves and surfaces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first practical problem I want to solve is to find a NURBS surface given a set of B-splines (Bezier patches, etc) in 3-D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In one of the classic books about B-splines, &lt;a href=&quot;https://www.springer.com/gp/book/9783642973857&quot; rel=&quot;noreferrer&quot;&gt;L. Piegl and W. Tiller, &lt;em&gt;The NURBS Book&lt;/em&gt;, Springer, 1995&lt;/a&gt;, Chapter 10 &quot;Advanced Surface Construction techniques&quot; describes a lot of computational algorithms, possibly including the ones I need (judging by the intro paragraph). However, I definitely do not want to invent the bicycle and would like to re-use as much as I can (provided the library development is not completely dead and has some userbase). I have no intention to write &lt;em&gt;yet another NURBS library&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, I am using &lt;a href=&quot;https://www.rhino3d.com/opennurbs&quot; rel=&quot;noreferrer&quot;&gt;openNURBS&lt;/a&gt; as a basic IO and storage format, but this library &lt;a href=&quot;http://developer.rhino3d.com/guides/opennurbs/what-is-opennurbs/#limitations&quot; rel=&quot;noreferrer&quot;&gt;supports only very basic operations&lt;/a&gt; – and I am very reluctant to go into &lt;a href=&quot;http://developer.rhino3d.com/&quot; rel=&quot;noreferrer&quot;&gt;&quot;full Rhino mode&quot;&lt;/a&gt; to get access to the required functionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already looked into:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/SINTEF-Geometry/SISL&quot; rel=&quot;noreferrer&quot;&gt;SISL&lt;/a&gt;, not entirely sure how I feel about the GNU license in this particular case.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceforge.net/projects/libnurbs/&quot; rel=&quot;noreferrer&quot;&gt;libnurbs++&lt;/a&gt; or &lt;a href=&quot;https://github.com/yiskang/libnurbs&quot; rel=&quot;noreferrer&quot;&gt;libnurbs++ GitHub&lt;/a&gt;, the development seems to have stopped at least in 2013, possibly 2011.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sourceforge.net/projects/nurbana/&quot; rel=&quot;noreferrer&quot;&gt;Nurbana&lt;/a&gt;, the development seemed to stop in 2003 (!)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://verbnurbs.com/&quot; rel=&quot;noreferrer&quot;&gt;verb&lt;/a&gt;, that I doubt I am the target audience of.&lt;/li&gt;&#xA;&lt;li&gt;and several other projects that lack any sort of documentation/development/user base.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, I wonder, what are right now the common grounds for integrating &lt;strong&gt;NURBS-related calculations&lt;/strong&gt; into your codes? Is there a widely used library that I missed (being relatively new to this topic)?&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-07-02T01:04:04.230" Title="Integrating/Implementing NURBS-related calculations" Tags="|computational-geometry|software|libraries|b-spline|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29752" PostTypeId="1" CreationDate="2018-06-22T07:30:30.287" Score="1" ViewCount="1174" Body="&lt;p&gt;I am trying to solve a boundary value problem with Python. I have been using scipy.integrate.solve_bvp but the result that it is giving me is completely wrong. Basically my code is as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from IPython import get_ipython&#xA;get_ipython().magic('reset -sf')&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from math import *&#xA;from scipy.integrate import solve_bvp&#xA;&#xA;r_in = 0.0&#xA;r_end = 500.&#xA;l = 0.&#xA;rho_c = 1.&#xA;m = 1.&#xA;&#xA;def system_DE(r,y,p_self,l=l,rho_c=rho_c):    &#xA;     R = y[0]&#xA;     ph = y[1]&#xA;     a = y[2]&#xA;     al = y[3]&#xA;&#xA;     om = p_self[0]&#xA;     u = p_self[1]&#xA;&#xA;    dR_dr = ph&#xA;    da_dr = a*(((2.*l+1.)*r/2.)*(-om**2.*a**2.*R**2./al**2.+ph**2.+l*(l+1.)*a**2.*R**2./r**2.+m**2.*a**2.*R**2.)-(a**2.-1.)/(2.*r))&#xA;    dal_dr = al*(da_dr/a-l*(l+1.)*(2.*l+1.)*a**2.*R**2./r-(2.*l+1.)*m**2.*a**2.*r*R**2.+(a**2.-1.)/r)&#xA;    dph_dr = -2.*ph/r+(l*(l+1.)*(2.*l+1.)*a**2.*R**2./r+(2.*l+1.)*m**2*a**2*r*R**2-(a**2-1)/r)*ph-om**2.*a**2.*R/al**2.+l*(l+1.)*a**2.*R/r**2.+m**2.*a**2.*R&#xA;&#xA;     return [dR_dr,dph_dr,da_dr,dal_dr]&#xA;&#xA;def bc(ya,yb,p_sch,l=l,rho_c=rho_c,r_in=r_in,r_end = r_end,m=m):&#xA;&#xA;    om = p_self[0]&#xA;    u = p_self[1]&#xA;&#xA;    if l==0.:&#xA;        return np.array([ya[0]-rho_c,ya[1]-rho_c*r_in,yb[0]*((m**2.-om**2.)**(1./2.)+1./(r_end))+yb[1],ya[2]-1.,ya[3]-u,yb[3]*yb[2]-1.])&#xA;    else:&#xA;        return np.array([rho_c*r**l,rho_c*l*r**(l-1.),1.,u])&#xA;&#xA;n_r = 1000&#xA;r = np.linspace(r_in,r_end,n_r)&#xA;y = np.zeros((4,r.size))&#xA;y[0,0] = rho_c&#xA;y[0,1] = rho_c&#xA;&#xA;y[2,:] = 1.&#xA;&#xA;p_self=[0.8,0.85] #[om,u]&#xA;&#xA;sol = solve_bvp(system_DE,bc, r, y, p=p_self, tol=0.01,max_nodes=100000)&#xA;&#xA;x_plot = np.linspace(r_in, 200  , 5000)&#xA;L = sol.sol(x_plot)[0]&#xA;Ph = sol.sol(x_plot)[1]&#xA;a = sol.sol(x_plot)[2]&#xA;alpha = sol.sol(x_plot)[3]&#xA;&#xA;plt.subplot(211)&#xA;plt.plot(x_plot, L)&#xA;plt.xlabel(&quot;x&quot;)&#xA;plt.ylabel(&quot;y&quot;)&#xA;plt.subplot(212)&#xA;plt.plot(x_plot, a)&#xA;plt.plot(x_plot, alpha)&#xA;plt.xlabel(&quot;x&quot;)&#xA;plt.ylabel(&quot;y&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The idea is that with the parameters that I am giving to the code I should obtain something similar to the figure [1] (up) on this article: &lt;a href=&quot;https://arxiv.org/pdf/0908.2435.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/0908.2435.pdf&lt;/a&gt;, however if I run the code I am obtaining something completely wrong. Is there another solver that I could use? Is there something wrong with my code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your time. Regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Luis P.&lt;/p&gt;&#xA;" OwnerUserId="28010" LastActivityDate="2018-06-22T07:30:30.287" Title="Which solvers for BVP in python are the best? Is there something better that scipy.integrate.solve_bvp?" Tags="|python|ode|computational-physics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29753" PostTypeId="1" AcceptedAnswerId="29754" CreationDate="2018-06-22T10:22:24.300" Score="2" ViewCount="918" Body="&lt;p&gt;In a FEM mesh, does the order of node numbering in an element has any importance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently trying to code my own FEM solver, which seems to work fine with quadrilateral elements, however I'm having difficulties whith triangular meshes, which yield different results when renumbering the nodes in the elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given the following mesh:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3vObM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3vObM.png&quot; alt=&quot;example mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solver yields different results if I define e.g. element $[1]$ as $(1) (5) (4)$ or $(5) (4) (1)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a known issue, or is there something wrong with my code? &lt;/p&gt;&#xA;" OwnerUserId="28047" LastEditorUserId="28047" LastEditDate="2018-06-22T10:53:10.070" LastActivityDate="2021-05-02T20:09:54.700" Title="Influence of node numbering in a FEM problem?" Tags="|finite-element|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29754" PostTypeId="2" ParentId="29753" CreationDate="2018-06-22T11:48:10.120" Score="1" Body="&lt;p&gt;There are a couple of places inside your code to look at:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Check $S^e_{1}$, the local stiffness matrix, and $T^e_{1}$, the local mass matrix, for element [1] for both cases of the node numbering. – there can be a bug in how the local matrices $S^e_{1}$ and $T^e_{1}$ are created (as noted in comments, one common bug is the sign for the area $A_1$ for CW and CCW node numbering).&lt;/li&gt;&#xA;&lt;li&gt;Make sure that the numbering for the element [1] does not influence $S^e_{2,3,4}$ and $T^e_{2,3,4}$, the local stiffness and mass matrices for the remaining untouched elements.&lt;/li&gt;&#xA;&lt;li&gt;Assemble the global stiffness matrix $S$ starting from elements [2], [3], and [4]. Then, compare, how your local stiffness matrix $S^e_{1}$ is added to the global stiffness matrix for both numberings.&lt;/li&gt;&#xA;&lt;li&gt;Compare the RHSs for both numberings of the element [1].&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-06-22T11:48:10.120" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29756" PostTypeId="1" CreationDate="2018-06-23T03:45:28.417" Score="4" ViewCount="481" Body="&lt;p&gt;According to the equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial y}{\partial t} = -a\frac{\partial y}{\partial x} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I simulated this in python. I used center differentiation, and I determined step size based on Von-Neumann stability criterion for a linear equation &#xA;$ \Delta t = \frac{\Delta x^2}{2a}$ . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I get:&lt;a href=&quot;https://gyazo.com/86024db42f71a6b5cb34eca7eb0d115f&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gyazo.com/86024db42f71a6b5cb34eca7eb0d115f&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;import numpy as np&#xA;&#xA;import matplotlib as mpl&#xA;import matplotlib.pyplot as plt&#xA;import matplotlib.animation as animation&#xA;&#xA;alpha = 1&#xA;seconds = 1000&#xA;dx = 1&#xA;dt = .1/alpha*dx**2&#xA;iter = int(np.round(seconds/dt))&#xA;print(iter, dt)&#xA;&#xA;&#xA;x = np.linspace(0,1000,1000)&#xA;# z = np.linspace(0,990, 10, dtype= int)&#xA;y = np.zeros([iter,1000])&#xA;y[0,:] = 50*np.sin(x[:]*np.pi/1000)&#xA;dy_t = np.zeros(1000)&#xA;&#xA;&#xA;for t in range(0,iter-1):&#xA;&#xA;    for x in range(1, 998):&#xA;&#xA;        dy_t[x] = -alpha*(y[t, x+1]-y[t,x-1])/(2*dx)&#xA;&#xA;&#xA;    for x in range(1,998):&#xA;        y[t+1,x] = y[t,x]+dy_t[x]*dt&#xA;&#xA;&#xA;# animation&#xA;fig, ax = plt.subplots()&#xA;line, = ax.plot(y[0,:])&#xA;def init():&#xA;    line.set_ydata([np.nan] * 1000)&#xA;    return line,&#xA;&#xA;def animate(i):&#xA;    line.set_ydata(y[i,:])&#xA;    return line,&#xA;&#xA;ani = animation.FuncAnimation(&#xA;fig, animate, range(0, iter), init_func=init, interval=5, blit=True, save_count=50)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28053" LastEditorUserId="20688" LastEditDate="2018-06-23T11:30:39.967" LastActivityDate="2018-06-24T04:53:15.620" Title="Why is my simulation of a first-order wave equation not stable?" Tags="|finite-difference|stability|wave-propagation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29757" PostTypeId="2" ParentId="29747" CreationDate="2018-06-23T11:16:43.823" Score="5" Body="&lt;p&gt;Note: &lt;em&gt;I'm somewhat worried at this point that the integral values Mathematica gives me are bogus. I thought it was working because it gave a sensible-looking result in a short time, but it might be the case that the method it tries to use is buggy or that I did something wrong. So it might be that the code below isn't working at all, I don't know, sorry.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note 2: &lt;em&gt;It bothered me, so I wrote another version (&lt;a href=&quot;https://gist.github.com/ikirill/f26a58fa0f18fc8754b976a163f055e8&quot; rel=&quot;nofollow noreferrer&quot;&gt;code here&lt;/a&gt;, sorry about the code quality) using Julia and GSL, and it evaluates &lt;code&gt;g&lt;/code&gt; in 2 seconds to the same answer that Mathematica gives below. So I think the code is probably okay.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main reason I was suspicious about numerical analytic continuation is that your integral actually looked kind of nice in my limited testing. In particular, the integrands of both $f$ and $\bar f$ decay polynomially and rather quickly, and this is exactly the sort of thing that conventional quadrature routines are designed to handle well. There are also no tricky singularities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My past experience with numerical integration leads me to believe that the fancier mathematical methods can sometimes be spectacularly helpful, but also that evaluating numerical Fourier transforms and integrating rational and algebraic functions are the bread-and-butter of numerical integration algorithms, so one can often make easy progress by picking algorithms carefully and playing with their parameters. This is usually the easier option if it's hard to see how to make the mathematical technique work right.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ClearAll[ξ, α, p1, p2, fi, f, g];&#xA;ξ = 1;&#xA;α = 1/2;&#xA;fi[e_, k4_, kp_] := Module[{&#xA;   p1 = (k4^2 + e/2)^2 + kp^2 + α^2,&#xA;   p2 = (k4^2 - e/2)^2 + kp^2 + (1 - α)^2},&#xA;  2 * (* because integrate k4 over (0,∞) *)&#xA;   2 kp * (* because d(kp^2) *)&#xA;   (α (1 + p1)^(3 ξ/2) (1 + p2)^(ξ/2)) /&#xA;     ((1 + p1 (1 + p1)^(2 ξ)) (1 + p2 (1 + p2)^(2 ξ)))&#xA;  ]&#xA;f[e_?NumericQ] := NIntegrate[&#xA;   fi[e, k4, kp], {k4, 0, ∞}, {kp, 0, ∞},&#xA;   Method -&amp;gt; {Automatic, &quot;SymbolicProcessing&quot; -&amp;gt; 0}];&#xA;&#xA;(* !!! This gives a bogus result: *)&#xA;gBogus[t_?NumericQ, e0_?NumericQ] := &#xA; NIntegrate[&#xA;  Exp[I t e]/(e^2 + e0^2) f[e], {e, -∞, ∞}, &#xA;  Method -&amp;gt; {&quot;DoubleExponentialOscillatory&quot;, &#xA;    &quot;SymbolicProcessing&quot; -&amp;gt; 0}]&#xA;&#xA;(* This gives *a* result, different from above despite being equivalent *)&#xA;g[t_?NumericQ, e0_?NumericQ] := &#xA; NIntegrate[Exp[I t e]/(e^2 + e0^2) f[e], {e, -\[Infinity], 0}, &#xA;   Method -&amp;gt; {&quot;DoubleExponentialOscillatory&quot;, &#xA;     &quot;SymbolicProcessing&quot; -&amp;gt; 0},&#xA;   EvaluationMonitor :&amp;gt; Print[&quot;e=&quot;, e]] +&#xA;  NIntegrate[Exp[I t e]/(e^2 + e0^2) f[e], {e, 0, \[Infinity]}, &#xA;   Method -&amp;gt; {&quot;DoubleExponentialOscillatory&quot;, &#xA;     &quot;SymbolicProcessing&quot; -&amp;gt; 0},&#xA;   EvaluationMonitor :&amp;gt; Print[&quot;e=&quot;, e]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In[18]:= Timing@g[10,1]&#xA;Out[18]= {78.0828, 0.0000704303 + 9.78009*10^-6 I}&#xA;&#xA;In[338]:= Timing@g[1,1]&#xA;Out[338]= {14.3125,0.389542 +0.024758 I}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I made Mathematica spend zero time on symbolically preprocessing the integrands, because in this case it wouldn't have been able to figure out anything useful about it anyway. I also told it to specifically use an oscillatory quadrature method for the second integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess for why randomly fiddling with integration strategies (see &lt;a href=&quot;https://reference.wolfram.com/language/tutorial/NIntegrateIntegrationStrategies.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NIntegrateIntegrationStrategies&lt;/a&gt;) works &lt;em&gt;at all&lt;/em&gt; is that sometimes Mathematica might accidentally pick a bad strategy automatically, killing performance, whereas anything I ask it to do is at least a little bit meaningful even if suboptimal. You could also consider getting help at &lt;a href=&quot;https://mathematica.stackexchange.com&quot;&gt;https://mathematica.stackexchange.com&lt;/a&gt;, they might know more about Mathematica's internals over there.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2018-06-23T14:00:38.660" LastActivityDate="2018-06-23T14:00:38.660" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29758" PostTypeId="2" ParentId="29756" CreationDate="2018-06-23T21:34:05.027" Score="8" Body="&lt;p&gt;Numerical solution of the advection equation with centered differences in space and forward Euler in time is unconditionally unstable. So the behavior you are seeing is expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a nice lecture by Gil Strang ( &lt;a href=&quot;https://ocw.mit.edu/courses/mathematics/18-086-mathematical-methods-for-engineers-ii-spring-2006/video-lectures/lecture-3-the-one-way-wave-equation-and-cfl-von-neumann-stability/&quot; rel=&quot;noreferrer&quot;&gt;MIT 18.086&lt;/a&gt;) where he discusses the &#xA;instability of this method. He also shows how the simple centered difference&#xA;method can be modified to yield the Lax-Friedrichs and Lax-Wendroff methods &#xA;which are conditionally stable.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2018-06-23T21:34:05.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29759" PostTypeId="2" ParentId="29756" CreationDate="2018-06-24T02:57:08.343" Score="1" Body="&lt;p&gt;I am going to answer this assuming you really want to use central difference scheme in space and forward difference in time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the problem&#xA;$$&#xA;u_t + a u_x = 0, \quad x \in (x_L,x_R), \quad t \in [0,T]&#xA;$$&#xA;and a numerical scheme&#xA;$$&#xA;u_j^{n+1} = H_j(u^n; a, \Delta t, h)&#xA;$$&#xA;The scheme is said to be weakly stable if there exists $C = C(T)$ such that&#xA;$$&#xA;\| u^n \| \le C \| u^0 \|, \qquad n \Delta t \le T&#xA;$$&#xA;An equivalent definition is&#xA;$$&#xA;\| u^{n+1} \| \le ( 1 + C \Delta t) \| u^n \|&#xA;$$&#xA;If $C=1$ then we say it is stable (strongly stable). The FTCS scheme&#xA;$$&#xA;\frac{u_j^{n+1} - u_j^n}{\Delta t} + \frac{u_{j+1}^n - u_{j-1}^n}{2h} = 0&#xA;$$&#xA;is never strongly stable. It is weakly stable under the condition&#xA;$$&#xA;\Delta t \le \left( \frac{h}{a} \right)^2&#xA;$$&#xA;This is not a useful scheme: (1) The time step is too small. (2) It is not useful for long time integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seem to have some bugs in your code which you have to fix yourself. I give my own implementation below where time step is chosen as&#xA;$$&#xA;\Delta t = CFL \left( \frac{h}{a} \right)^2&#xA;$$&#xA;Run this with Python2; for Python3 you have to make some changes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;def smooth(x):&#xA;    return np.sin(2*np.pi*x)&#xA;&#xA;def update_ftcs(nu, u):&#xA;    unew = np.empty_like(u)&#xA;    unew[0] = u[0] + 0.5*nu*(u[-2] - u[1])&#xA;    unew[1:-1] = u[1:-1] + 0.5*nu*(u[0:-2] - u[2:])&#xA;    unew[-1] = unew[0]&#xA;    return unew&#xA;&#xA;def solve():&#xA;    scheme     = 'FTCS'&#xA;    N          = 100&#xA;    cfl        = 0.95&#xA;    Tf         = 1.0&#xA;    uinit      = smooth&#xA;    xmin, xmax = 0.0, 1.0&#xA;    a          = 1.0&#xA;&#xA;    h = (xmax - xmin)/N&#xA;    dt= cfl * (h / np.abs(a))**2&#xA;    nu= a * dt / h&#xA;&#xA;    x = np.linspace(xmin, xmax, N+1)&#xA;    u = uinit(x)&#xA;&#xA;    fig = plt.figure()&#xA;    ax = fig.add_subplot(111)&#xA;    line1, = ax.plot(x, u, 'r')&#xA;    line2, = ax.plot(x, u, 'b')&#xA;    ax.set_xlabel('x'); ax.set_ylabel('u')&#xA;    plt.legend(('Numerical','Exact'))&#xA;    plt.title('N='+str(N)+', CFL='+str(cfl)+', Scheme='+scheme)&#xA;    plt.draw(); plt.pause(0.1)&#xA;    wait = raw_input(&quot;Press enter to continue &quot;)&#xA;&#xA;    t, it = 0.0, 0&#xA;    while t &amp;lt; Tf:&#xA;        u = update_ftcs(nu, u)&#xA;        t += dt; it += 1&#xA;        if it%100 == 0:&#xA;            line1.set_ydata(u)&#xA;            line2.set_ydata(uinit(x-a*t))&#xA;            plt.draw(); plt.pause(0.1)&#xA;    plt.show()&#xA;&#xA;solve()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2018-06-24T04:53:15.620" LastActivityDate="2018-06-24T04:53:15.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29760" PostTypeId="1" CreationDate="2018-06-25T09:31:01.900" Score="2" ViewCount="80" Body="&lt;p&gt;The &lt;a href=&quot;https://www.top500.org/green500/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Green500 List&lt;/a&gt; lists the top 500 most energy efficient supercomputers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what is the minimum FLOPS that computers need to be considered for the Green500 list?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do they just take the TOP500 list and rank them according to FLOPS/watt (cut-off = FLOPS of the 500th computer of TOP500)? Or is there some other cut-off value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there were no such minimum, then it would make no sense, otherwise, say, cell phones, might as well be on the list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could compare both lists to see if they contain the same computers, but lazy.&lt;/p&gt;&#xA;" OwnerUserId="4020" LastEditorUserId="4020" LastEditDate="2018-06-25T09:53:17.363" LastActivityDate="2018-06-25T09:53:17.363" Title="What is the minimum raw FLOPS that supercomputers need to be included in the Green500 list?" Tags="|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29761" PostTypeId="2" ParentId="29760" CreationDate="2018-06-25T09:42:55.507" Score="3" Body="&lt;p&gt;According to the &lt;a href=&quot;https://www.top500.org/green500/&quot; rel=&quot;nofollow noreferrer&quot;&gt;main page of Green500 List&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;About the Green500 List&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;The Green500 list ranks the top 500 supercomputers in the world by energy efficiency&lt;/strong&gt;. ... To address this trend, the Green500 list puts a premium on energy-efficient performance for sustainable supercomputing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also, if you look at the list itself, say &lt;a href=&quot;https://www.top500.org/green500/lists/2018/06/&quot; rel=&quot;nofollow noreferrer&quot;&gt;June 2018&lt;/a&gt;, the first column is Green500 rank, while the second one lists Top500 rank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, one can say that the overall procedure is the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First, the Top500 list is created based on the performance.&lt;/li&gt;&#xA;&lt;li&gt;Then, power efficiency (GFlops/watts) is measured or derived. (the systems with derived efficiencies are shaded in the Green500 list).&lt;/li&gt;&#xA;&lt;li&gt;Green500 list is created by sorting the participants of Top500 list by power efficiency.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So yeah, the cut-off is the 500th computer of TOP500.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-06-25T09:42:55.507" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29763" PostTypeId="1" AcceptedAnswerId="29791" CreationDate="2018-06-25T20:39:00.070" Score="7" ViewCount="380" Body="&lt;p&gt;I need to solve a least squares problem that takes the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$p = \arg \min_{x}\Vert J V x - y \Vert_2, $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $J \in \mathbb{R}^{N \times N}$ is a general dense matrix, and $V \in \mathbb{R}^{N \times m}$ is a block-diagonal matrix with $m &amp;lt;&amp;lt; N$ (that is, this is a typical overdetermined least-squares problem). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if there is a way to take advantage of that block-diagonal matrix.  When I form the product $JV$, now it is a dense matrix that I can use a dense least-squares solver to solve, but it feels like there should be a better solution.  Does anyone have an idea to do this faster?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edits for additional details:&#xA;Typical values of $V$ for my current problem are about 100,000 rows and 100 columns, though I'd like to scale that up to many more rows and probably a few more columns.  The block diagonal structure means that this matrix could be written as a block matrix that is diagonal, but the blocks are not necessarily square or the same size.&lt;/p&gt;&#xA;" OwnerUserId="28070" LastEditorUserId="28070" LastEditDate="2018-06-30T06:05:07.360" LastActivityDate="2018-06-30T21:58:36.837" Title="Least Squares with Dense-Block Diagonal Structure" Tags="|linear-algebra|least-squares|matrix-equations|matrix-factorization|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29764" PostTypeId="1" AcceptedAnswerId="29769" CreationDate="2018-06-25T22:30:10.153" Score="6" ViewCount="1217" Body="&lt;p&gt;I'd like to know whether there are any publicly available tools for solving QCQP with complex variables (and constraints therefore expressed through Hermitian matrices). What I have found so far is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A recently created CVXPY &lt;a href=&quot;https://github.com/cvxgrp/qcqp&quot; rel=&quot;noreferrer&quot;&gt;package&lt;/a&gt; for QCQP with real variables, as well as a &lt;a href=&quot;http://homepages.cae.wisc.edu/~linderot/papers/Linderoth-05.pdf&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt; mentioning the ability of &lt;a href=&quot;https://www.gams.com/latest/docs/S_BARON.html#BARON_VARIABLE_AND_EXPESSION_BOUNDS&quot; rel=&quot;noreferrer&quot;&gt;BARON&lt;/a&gt; to handle those problems.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/pdf/1203.5599.pdf&quot; rel=&quot;noreferrer&quot;&gt;Two&lt;/a&gt; &lt;a href=&quot;https://arxiv.org/pdf/1705.09057.pdf&quot; rel=&quot;noreferrer&quot;&gt;papers&lt;/a&gt; that implement solvers for some QCQP problems with complex variables. Their code seems at first glance not to be available online, though.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3760" LastEditorUserId="20417" LastEditDate="2018-06-27T21:50:26.373" LastActivityDate="2018-06-27T21:50:26.373" Title="Solvers for Quadratically Constrained Quadratic Programs (QCQP) with complex variables" Tags="|optimization|constrained-optimization|qcqp|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29765" PostTypeId="1" AcceptedAnswerId="29776" CreationDate="2018-06-26T01:29:56.143" Score="4" ViewCount="118" Body="&lt;p&gt;this might be a silly question, but if we’re trying to numerically solve Hamilton’s equations with some discrete scheme, sometimes when the scheme preserves phase space volume (Hamilton’s eqns are understood to be some flow on this phase space) it leads to stability, even if the algorithm isn’t symplectic (for example the nonrelativistic Boris Pusher algorithm for electrodynamics). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m wondering, what is there to be gained in the numerical analysis sense for a discrete, symplectic scheme for Hamilton’s equations? Some of the obvious desiderata like better error bounds might just come from the algorithm more closely resembling the Hamilton flow itself, but in general, can it be said exactly what is gained when an algorithm is symplectic and not just volume-preserving?&lt;/p&gt;&#xA;" OwnerUserId="28071" LastActivityDate="2018-06-28T01:08:26.060" Title="Symplectic Algorithms for Hamilton’s Equations as opposed to just Volume-Preserving" Tags="|computational-physics|time-integration|differential-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29767" PostTypeId="2" ParentId="29689" CreationDate="2018-06-26T11:57:15.533" Score="1" Body="&lt;p&gt;I think that you may be overcomplicating things, trying to use general triclinic periodic boundary conditions for a system with 2D hexagonal periodicity. It is quite possible to use a rhombic cell in the $x$ and $y$ directions (with regular boundaries in $z$ if needed, making it a rhombic prism in 3D) or indeed a rectangular cell (i.e. a cuboidal box in 3D). Both these are illustrated in the figure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/llkkB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/llkkB.png&quot; alt=&quot;hexagonal lattice overlaid with rectangular and rhombic boxes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the blue box illustrated here, if the side of the hexagon is one unit, then $L_x=12$ and $L_y=6\sqrt{3}\approx 10.392$, but there is plenty of flexibility depending on how many unit cells (in each direction) you wish to include. Then the Python code for periodic boundary corrections is the same as for any cuboidal box, for instance &lt;code&gt;xij=xij-np.rint(xij/Lx)*Lx&lt;/code&gt; and similarly for &lt;code&gt;yij&lt;/code&gt; and &lt;code&gt;zij&lt;/code&gt;, where &lt;code&gt;(xij,yij,zij)&lt;/code&gt; is an uncorrected pair vector.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-06-26T11:57:15.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29768" PostTypeId="1" AcceptedAnswerId="29771" CreationDate="2018-06-26T16:02:55.163" Score="6" ViewCount="250" Body="&lt;p&gt;I want to know whether the equation : a-b+b = a is always true for a, b belongs to double precision floating-point number and |a|&gt;=|b|. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the equation is true, how can I proof it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, what precondition can make the equation right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A counterexample shows the equation is not true, so I now want to know,&#xA;the inequality&#xA;|((a-b)+b)-a|&amp;lt;=ulp(a) is true or not, how can I proof the inequality?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested the python code below for half hours, the inequality |a-b+b-a|&amp;lt;=ulp(a) seems hold.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def getulp(x):&#xA;x = float(x)&#xA;k = frexp(x)[1]-1&#xA;if x == 0.0:&#xA;    return pow(2, -1074)&#xA;if (k&amp;lt;1023)&amp;amp;(k&amp;gt;-1022):&#xA;    return pow(2,k-52)&#xA;else:&#xA;    return pow(2,-1074)&#xA;&#xA;&#xA;&#xA;while 1&amp;gt; 0:&#xA;k = np.random.randint(-100, 100)&#xA;ub = np.float_power(2,k-1)&#xA;db = np.float_power(2,k)&#xA;j = np.random.randint(-100, 100)&#xA;ub2 = np.float_power(2, j - 1)&#xA;db2 = np.float_power(2, j)&#xA;x = np.random.uniform(ub,db,1)&#xA;y = np.random.uniform(ub2,db2,1)&#xA;if x&amp;gt;y:&#xA;    temp = x-y+y&#xA;    if np.fabs(temp-x)&amp;gt;getulp(x):&#xA;        print x&#xA;        print y&#xA;        print &quot;The inequality false&quot;&#xA;else:&#xA;    temp = y-x+x&#xA;    if np.fabs(temp-y)&amp;gt;getulp(y):&#xA;        print x&#xA;        print y&#xA;        print &quot;The inequality false&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks for the counterexample by @Kirill, I will explain why I need to proof the equation. I want to proof, image the a, b are two end points of a line, then if I know the value b and a-b, I want to proof that I can get the value of a without error larger than one ULP(a) error.&#xA;So I can give the condition |a|&gt;=|b| or |a|&amp;lt;=|b|, but I can not limit their actual value. &lt;/p&gt;&#xA;" OwnerUserId="22224" LastEditorUserId="22224" LastEditDate="2018-06-30T01:35:42.350" LastActivityDate="2018-06-30T01:35:42.350" Title="Is it possible to proof a-b+b = a for all double floating-point numbers?" Tags="|floating-point|precision|accuracy|numerics|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="29769" PostTypeId="2" ParentId="29764" CreationDate="2018-06-26T20:55:28.293" Score="7" Body="&lt;p&gt;Let's consider the following minimization problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp; \text{minimize} &amp;amp;&amp;amp; \tfrac12 z^\mathrm{H} P_0 z + q_0^\mathrm{H} z\\&#xA;&amp;amp; \text{subject to} &amp;amp;&amp;amp; \tfrac12 z^\mathrm{H} P_i z + q_i^\mathrm{H} z + r_i \leq 0 \quad \text{for } i = 1,\dots,m , \\&#xA;&amp;amp;&amp;amp;&amp;amp; Az = b, &#xA;\end{align} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $P_i$ are Hermitian matrices. The terms $q_i^H z$ can be complex in general, so they should not appear for us to have real-valued objective function and inequality constraints. We have then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp; \text{minimize} &amp;amp;&amp;amp; \tfrac12 z^\mathrm{H} P_0 z\\&#xA;&amp;amp; \text{subject to} &amp;amp;&amp;amp; \tfrac12 z^\mathrm{H} P_i z + r_i \leq 0 \quad \text{for } i = 1,\dots,m , \\&#xA;&amp;amp;&amp;amp;&amp;amp; Az = b, &#xA;\end{align} &#xA;with $r_i$ real numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, since $P_i$ is Hermitian, the following holds&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\mathrm{Re}(P_i) = \mathrm{Re}(P_i)^T\, ,\quad \mathrm{Im}(P_i)=-\mathrm{Im}(P_i)\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we can write&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$z^H P_i z = \begin{pmatrix} x &amp;amp;y\end{pmatrix}&#xA;\begin{bmatrix} \mathrm{Re}(P_i) &amp;amp;-\mathrm{Im}(P_i)\\ \mathrm{Im}(P_i) &amp;amp;\mathrm{Re}(P_i)\end{bmatrix}&#xA;\begin{pmatrix} x \\y\end{pmatrix} \equiv X^T Q_i X\, ,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $z = x + iy$. The crossed terms cancel out due to skew-symmetry in the imaginary part and symmetry in the real part of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rewriting the original problem, we are left with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp; \text{minimize} &amp;amp;&amp;amp; \tfrac12 X^\mathrm{T} Q_0 X\\&#xA;&amp;amp; \text{subject to} &amp;amp;&amp;amp; \tfrac12 X^\mathrm{T} Q_i X + r_i \leq 0 \quad \text{for } i = 1,\dots,m , \\&#xA;&amp;amp;&amp;amp;&amp;amp; MX = B, &#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;M = \begin{bmatrix} \mathrm{Re}(A) &amp;amp;-\mathrm{Im}(A)\\ \mathrm{Im}(A) &amp;amp;\mathrm{Re}(A)\end{bmatrix}\, ,\\&#xA;&amp;amp;B = \begin{pmatrix} \mathrm{Re}(b) \\-\mathrm{Im}(b)\end{pmatrix}\, .&#xA;\end{align}&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-06-27T14:03:19.647" LastActivityDate="2018-06-27T14:03:19.647" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29770" PostTypeId="1" CreationDate="2018-06-27T04:17:52.987" Score="9" ViewCount="141" Body="&lt;p&gt;I'm hoping someone can recommend recent literature concerning homotopy methods for solving systems of nonlinear equations.  Already by the time of Layne Watson's 1986 paper there were a lot of methods, and I'm sure there have been developments since then as well as practical shifts in the field due to hardware advancements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking to apply globally convergent methods to problems in nonlinear elasticity with ~$10^6$ degrees of freedom.  The Jacobians are sparse but local methods like Newton-Raphson or quasi-Newton with line search or trust region tend to get trapped at points where the Jacobian is singular.  I've had some success with a crude implementation of a homotopy method but I'm looking to get better performance and better familiarity with modern methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I'm mainly interested in the benefits and drawbacks of the three methods for curve tracking that Watson describes: the ODE algorithm, the normal flow algorithm, and the augmented Jacobian algorithm.  Watson's papers are very nice but they are a bit outdated for assessing my problem.  For example, it seems that the limited memory version of BFGS had not been developed yet, which made it impossible to fairly test a sparse version of the augmented Jacobian algorithm.  For my problem, I am only interested in the solution, i.e. $x$ s.t. $F(x,\lambda)=0$ for $\lambda=1$, but I am very concerned with stability and efficiency.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear algebra steps seem quite straightforward, and I can test different preconditioners, etc.&lt;/p&gt;&#xA;" OwnerUserId="25235" LastEditorUserId="25235" LastEditDate="2018-06-28T18:58:56.583" LastActivityDate="2018-06-28T18:58:56.583" Title="Review of modern homotopy methods and practical techniques" Tags="|parallel-computing|nonlinear-equations|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29771" PostTypeId="2" ParentId="29768" CreationDate="2018-06-27T12:11:36.497" Score="9" Body="&lt;p&gt;You can sometimes prove such results (or get counterexamples) using an SMT solver such as &lt;a href=&quot;https://github.com/Z3Prover/z3&quot; rel=&quot;noreferrer&quot;&gt;Z3&lt;/a&gt; that supports floating point arithmetic. Here is a proof of a version of your theorem that says $|((x+y)-y)-x| \leq 2^{-23}|x|$ when $x&amp;gt;y&amp;gt;1$ and $x+y\neq\infty_{32}$ in 32-bit floating point arithmetic:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;λ&amp;gt; import Data.SBV&#xA;λ&amp;gt; :set -XScopedTypeVariables&#xA;λ&amp;gt; prove $ \(x::SFloat) (y::SFloat) -&amp;gt; fpIsNormal (x+y) &amp;amp;&amp;amp;&amp;amp; x .&amp;gt; y &amp;amp;&amp;amp;&amp;amp; y .&amp;gt; 1 ==&amp;gt; fpAbs (((x+y)-y)-x) .&amp;lt; fpAbs x * 1.1920928955078125e-7&#xA;Q.E.D.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here I used the sbv package in Haskell. If it holds for 32-bit floats, it should hold for 64-bit floats also.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-06-27T12:11:36.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29774" PostTypeId="5" CreationDate="2018-06-27T19:23:51.297" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-06-27T19:23:51.297" LastActivityDate="2018-06-27T19:23:51.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29775" PostTypeId="4" CreationDate="2018-06-27T19:23:51.297" Score="0" Body="A quadratically constrained quadratic program (QCQP) is an optimization problem in which both the objective function and the constraints are quadratic." OwnerUserId="20417" LastEditorUserId="20417" LastEditDate="2018-06-27T21:50:40.353" LastActivityDate="2018-06-27T21:50:40.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29776" PostTypeId="2" ParentId="29765" CreationDate="2018-06-28T00:27:56.067" Score="4" Body="&lt;p&gt;Backward error analysis comes to mind. For example, for the Verlet scheme, you can say that the numerical solution turns out to be the exact solution for a perturbed Hamiltonian system (also known as the shadow Hamiltonian, the resulting system of differential equations are called the modified equations). See also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1137/S0036142997329797&quot; rel=&quot;nofollow noreferrer&quot;&gt;S. Reich, “Backward error analysis for numerical integrators,” SIAM Journal on Numerical Analysis, vol. 36, no. 5, pp. 1549–1570, 1999.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="20688" LastEditDate="2018-06-28T01:08:26.060" LastActivityDate="2018-06-28T01:08:26.060" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29777" PostTypeId="1" CreationDate="2018-06-28T08:59:01.133" Score="3" ViewCount="163" Body="&lt;p&gt;What are the main differences between finite element and finite difference approach for incompressible flow simulations? I have a vague idea about how FE methods rely on minimizing the residual over elements and FD methods consider Taylor series approximation of different terms in NS equations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on my little experience, I have observed that most of the codes in fluid dynamics community are based on finite volume/finite difference methods rather than FE methods. Are there any subtle reasons regarding why FD/FV methods are much more prevalent than the FE methods in flow simulations?&lt;/p&gt;&#xA;" OwnerUserId="28075" LastEditorUserId="20688" LastEditDate="2019-06-06T19:57:10.890" LastActivityDate="2019-06-06T19:57:10.890" Title="Comparison between FEM and FDM methods for flow simulations" Tags="|finite-element|finite-difference|fluid-dynamics|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29779" PostTypeId="1" CreationDate="2018-06-28T17:26:57.703" Score="3" ViewCount="116" Body="&lt;p&gt;In a posteriori error control for solving ODEs, one typically computes two different approximate solutions, one of which being &quot;more accurate&quot; and one of which being &quot;less accurate&quot;. If $y_q^{n+1}$ is the more accurate approximation to the solution at the next time step $y^{n+1}$, and $y_p^{n+1}$ is the less accurate approximation, one can estimate the error for the less accurate approximation, e.g., $$\text{error}(y_p^{n+1}) \approx y_q^{n+1} - y_p^{n+1}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This error estimate can then be used to determine if, for example, the time-step size should change according to some prescribed accuracy requirements by the user. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q&lt;/strong&gt;:  Could one use &quot;sequence acceleration&quot; techniques and &quot;error control&quot; at the same time using just $y_q^{n+1}$ and $y_p^{n+1}$? Or would this be &quot;having my cake and eating it&quot;?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Some cases I had in mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$y_q^{n+1}$ could come from a formally higher-order approximation and $y_p^{n+1}$ from a lower-order approximation as in embedded Runge-Kutta schemes. &lt;/li&gt;&#xA;&lt;li&gt;$y_q^{n+1}$ could come using a small time-step size when compared to $y_p^{n+1}$.  Should I do &quot;Richardson Extrapolation&quot; in this case?&lt;/li&gt;&#xA;&lt;li&gt;$y_q^{n+1}$ and $y_p^{n+1}$ could be &quot;Milne pairs&quot; which have the same order of accuracy, but provide a &quot;Milne error estimate&quot; of the form $\text{error}(y_p^{n_1}) \approx C (y_q^{n+1} - y_p^{n+1})$ where $C$ is known. Should I take some combination of $y_q^{n+1}$ and $y_p^{n+1}$ to obtain a higher-order estimate?&lt;/li&gt;&#xA;&lt;li&gt;$y_q^{n+1}$ could come from an implicit &quot;corrector&quot; step (perhaps of higher-order) and $y_p^{n+1}$ could come from an explicit &quot;predictor&quot; step in a predictor-corrector scheme. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q&lt;/strong&gt;: In all these cases, what would be a smart choice for $y^{n+1}$? Should I set $y^{n+1} := y_p^{n+1} + \text{error}(y_p^{n+1})$? Should I set $y^{n+1}:= y_q^{n+1}$?. Should I do something else entirely? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also seen that one can choose  $y_q^{n+1}$ and $y_p^{n+1}$ to have the same leading error terms but with opposite signs, so that their average is higher order. I am not sure how applicable this technique is in general. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by &quot;sequence acceleration&quot; is: obtaining local solutions (at the next time step) $y^{n+1}$ which are more accurate than the solution $y^{n+1}_p$ for which we have an error estimate of. &lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2018-08-01T19:38:45.157" LastActivityDate="2018-08-01T19:38:45.157" Title="Error control and sequence acceleration at the same time" Tags="|ode|time-integration|error-estimation|adaptive-timestepping|acceleration|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29780" PostTypeId="1" CreationDate="2018-06-28T19:19:31.800" Score="1" ViewCount="55" Body="&lt;p&gt;I am using a central-difference scheme to solve the eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2}{dx^2}u = \lambda u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on a unit interval with periodic boundary conditions. My understanding is the eigenvectors $u_i$ have the analytic form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u_i(x) = e^{in\pi x}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that, for example, with dirichlet boundary conditions $u_0 = u_{N+1} = 0$ and $N = 6$, the central-difference representation of the Laplacian is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{h^2}\begin{bmatrix}&#xA;    2       &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    -1       &amp;amp; 2 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0 &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  0 &amp;amp; 0\\&#xA;    0 &amp;amp;0 &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  0\\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 \\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 2 \\&#xA;\end{bmatrix}&#xA;$$&#xA;But I'm not sure about with periodic boundary conditions. My best guess so far is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{1}{h^2}\begin{bmatrix}&#xA;    2       &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 \\&#xA;    -1       &amp;amp; 2 &amp;amp; -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 \\&#xA;    0 &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  0 &amp;amp; 0\\&#xA;    0 &amp;amp;0 &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 &amp;amp;  0\\&#xA;    0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 2 &amp;amp; -1 \\&#xA;    -1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0 &amp;amp; -1 &amp;amp; 2 \\&#xA;\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this doesn't give me the expected eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: Is my form above correct?&lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2018-06-28T19:19:31.800" Title="What is the finite-difference representation of the Laplacian operator with periodic boundary conditions?" Tags="|finite-difference|boundary-conditions|laplacian|" AnswerCount="0" CommentCount="1" ClosedDate="2018-06-29T10:27:01.277" ContentLicense="CC BY-SA 4.0" />
  <row Id="29781" PostTypeId="1" AcceptedAnswerId="29807" CreationDate="2018-06-29T02:52:20.843" Score="5" ViewCount="172" Body="&lt;p&gt;I have a set of matrices $\{(A_i,D_i)\}$ for $i\in\{1,\ldots,n\}$, where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Each $D_j\in\mathbb{R}^{S\times S}$ is diagonal, and every entry on the main diagonal is non-negative.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Each $A_j\in\mathbb{R}^{m\times m}$ is symmetric and positive definite, with every entry being non-negative.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want to find a matrix $B\in\mathbb{R}^{m\times S}$ such that:&#xA;$$&#xA;A_i \approx B D_i B^T \;\;\;\forall\;\;\;1\leq i \leq n \\&#xA;B^TB\approx I&#xA;$$&#xA;However, everything is noisy (meaning $A_i$ and $D_i$ always have some random perturbation). This means (I assume) that I need something like:&#xA;$$&#xA;B^* = \arg\min_B \;\alpha||B^TB - I|| + \beta\sum_{i=1}^n || A_i - BD_iB^T||&#xA;$$&#xA;for some matrix norm (e.g. Frobenius) and (hyper-)parameters $\alpha,\beta\in\mathbb{R}$. I am not too picky about the exact formulation, however, so feel free to tweak it. For instance, perhaps there is a way to make the orthogonality a hard constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: how do I solve an optimization problem like the one above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What have I tried: well, this reminds me of an eigenvalue decomposition problem (if $n=1$ especially), but I have a set of problems I'd like to simultaneously satisfy.&#xA;One odd thing though is that the $D_j$ are &lt;em&gt;known&lt;/em&gt;, or at least estimated (albeit &lt;em&gt;with noise&lt;/em&gt;). My first thought was to rearrange this into a linear system somehow, but I have not been able to do so (so far). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is some literature I can look into relating to this problem, that would be a more than good enough answer. My apologies for the lack of optimization/numerical linear algebra knowledge.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note: I have &lt;a href=&quot;https://math.stackexchange.com/questions/2828164/optimizing-a-set-of-matrix-equations&quot;&gt;already tried to post this on math stack exchange&lt;/a&gt;, but to no avail.&lt;/p&gt;&#xA;" OwnerUserId="24492" LastEditorUserId="20417" LastEditDate="2018-07-04T20:52:16.337" LastActivityDate="2018-07-04T20:52:16.337" Title="Generate approximately semi-orthogonal tall matrix approximately satisfying constraints" Tags="|linear-algebra|optimization|matrix|eigensystem|matrix-equations|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29783" PostTypeId="2" ParentId="29779" CreationDate="2018-06-29T12:51:42.170" Score="3" Body="&lt;p&gt;You should state clearly what you mean by &lt;em&gt;sequence acceleration&lt;/em&gt;.  But if I understand you correctly, what you're asking about is exactly what extrapolation codes do.  A sequence of low-order approximations of the new solution are computed and then combined (via extrapolation) to produce high-order approximations.  The highest-order and second-highest-order approximations are also used to produce an error estimate.  The canonical reference for this is the 2-volume series by Hairer, Norsett, &amp;amp; Wanner.  If you don't have the books and are looking for a straightforward explanation, you might look at &lt;a href=&quot;http://www.davidketcheson.info/assets/papers/2014_hork.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper of mine&lt;/a&gt;.  A canonical implementation is Hairer's &lt;a href=&quot;https://www.unige.ch/~hairer/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ODEX code&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the early days of error estimation, people used the lower-order estimate as starting point for the next step, but since at least the 1980's everyone uses the higher-order estimate; this choice is sometimes referred to as &lt;em&gt;local extrapolation&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2018-06-29T12:51:42.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29785" PostTypeId="1" AcceptedAnswerId="29786" CreationDate="2018-06-29T19:02:00.450" Score="2" ViewCount="87" Body="&lt;p&gt;I am trying to take a Fourier transform of a density field estimated from mock galaxy survey catalogs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you start with a list of galaxy positions, then you bin these positions over some grid spacing, getting an array with a weight for each grid position in the relevant volume of space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I am trying to preserve the information about small spatial scales, I would like to avoid using large bins. Unfortunately, this makes a full array representation of the density field quite memory expensive, as there are huge numbers of empty bins which are for most purposes irrelevant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thus far been binning without the array representation, instead keeping the data in a list of coordinates with their respective weights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my understanding is that this is not sufficient to perform a standard FFT routine. I need evenly spaced data, which I do not have in this representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A natural option could then be something like &lt;a href=&quot;https://github.com/jyhmiinlin/pynufft&quot; rel=&quot;nofollow noreferrer&quot;&gt;pynufft&lt;/a&gt; (since I am using Python primarily) or &lt;a href=&quot;https://github.com/jakevdp/nfft&quot; rel=&quot;nofollow noreferrer&quot;&gt;nfft&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, my situation seems somewhat different from the standard non-uniform FFT one; I am not dealing with &quot;nonuniform sampling&quot; per se, rather, my sampling is perfectly uniform but just too large to store with explicit zeroes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any implementation of FFT which would allow me to not create an explicit array of zeros, or at least leverage the fact that I implicitly know the &quot;missing&quot; bins in the sparse representation of my data are zero-valued?&lt;/p&gt;&#xA;" OwnerUserId="28105" LastActivityDate="2018-06-29T20:34:06.447" Title="FFT of &quot;implicitly&quot; uniform data" Tags="|python|sparse-matrix|fourier-transform|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29786" PostTypeId="2" ParentId="29785" CreationDate="2018-06-29T20:34:06.447" Score="2" Body="&lt;p&gt;Since your histogram is very sparse, your density field is approximately a sum of weighted delta functions:&#xA;$$\rho(\vec r) = \sum_{j=1}^{N_{\mathrm{bins}}} w_j\, \delta(\vec r - \vec r_j)$$&#xA;where $r_j$ is the center of each bin, and $w_j$ is the weight.&#xA;Its Fourier transform is then simply&#xA;$$\begin{align}&#xA;\hat\rho(\vec k) &#xA;&amp;amp;= \int \exp(-i\vec k\cdot\vec r) \rho(\vec r)\,d^3r&#xA;\\&#xA;&amp;amp;= \sum_{j=1}^{N_{\mathrm{bins}}} w_j \exp(-i\vec k\cdot\vec r_j)&#xA;\end{align}$$&#xA;Alternatively, you could skip the binning process, treating each galaxy as being in its own &quot;bin&quot; with weight equal to one. In principle, this gives you access to arbitrarily high-$k$ information (rather than being limited by the physical width of the bin), but it will be noisier than the binned density.&lt;/p&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2018-06-29T20:34:06.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29787" PostTypeId="2" ParentId="29779" CreationDate="2018-06-30T03:22:36.547" Score="3" Body="&lt;p&gt;What you're talking about is local extrapolation. Local extrapolation is the idea of getting an error estimate between two methods and then continuing (accepting the new value) from the higher order method. Dormand-Prince is an influential paper which created a locally extrapolating 5th order method where the 5th order method minimizes its truncation error coefficients (&lt;a href=&quot;https://linkinghub.elsevier.com/retrieve/pii/0771050X80900133&quot; rel=&quot;nofollow noreferrer&quot;&gt;A family of embedded Runge-Kutta methods&lt;/a&gt;). This Dormand-Prince method was found to be modestly more efficient (for free) than the original adaptive Fehlberg algorithm which uses a 4/5 pair but optimizes the coefficients on the 4th order method and then uses that to advance. This then became the &lt;code&gt;dopri5&lt;/code&gt; method of Hairer, and it went on to be the standard method in most places for a long time (for example, it's &lt;code&gt;ode45&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why local extrapolation was used here is because it's stable. The Runge-Kutta methods were built in such a way that they had similar stability regions, and so the error estimator was stable when continuing with either one. I think stability of adaptive algorithms is covered in &lt;a href=&quot;https://www.springer.com/us/book/9783540604525&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer II&lt;/a&gt;, where there's also details about using PI-controlled stepsizes to further increase the stability with adaptivity. This isn't true for every algorithm though. But &lt;a href=&quot;https://www.jstor.org/stable/2005249&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shampine studied the effects on the most popular algorithms&lt;/a&gt; and local extrapolation was shown to either not be harmful or in some cases it increases the stability of the adaptive algorithm, so local extrapolation is used in most modern codes for this reason.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-06-30T03:22:36.547" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29788" PostTypeId="1" CreationDate="2018-06-30T09:01:44.800" Score="7" ViewCount="170" Body="&lt;p&gt;By a barycentric rational interpolant we understand a function of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align*}&#xA;r(t) := \frac{\sum_{i=0}^{n-1} \frac{w_i y_i}{t-t_i} }{ \sum_{i=0}^{n-1} \frac{w_i}{t-t_i}}&#xA;\end{align*}&#xA;In &lt;a href=&quot;http://www.ams.org/journals/mcom/1986-47-175/S0025-5718-1986-0842136-8/S0025-5718-1986-0842136-8.pdf&quot; rel=&quot;noreferrer&quot;&gt;Schneider and Werner&lt;/a&gt;, (proposition 12) a stable algorithm for computation of the derivatives of $r^{(k)}$ is presented. (This algorithm is reviewed &lt;a href=&quot;https://www.researchgate.net/profile/Richard_Baltensperger/publication/226684962_Recent_Developments_in_Barycentric_Rational_Interpolation/links/00b4951518a41531de000000.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;  in more modern notation, in equation 5.1a.) However, this algorithm requires two passes through the data vectors $\{w_{i}\}_{i=0}^{n-1}$, $\{y_{i}\}_{i=0}^{n-1}$ and $\{t_{i}\}_{i=0}^{n-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I only require the first derivative $r'$, I am curious as to whether I can get the first derivative via automatic differentiation, in one pass, perhaps increasing the speed of evaluation. So I have two questions: What is the automatic differentiation formula for $r$, and by using it, will I lose the stability guaranteed by Werner and Schneider's algorithm?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2018-07-01T03:11:16.780" LastActivityDate="2023-07-02T01:25:18.557" Title="Automatic differentiation of barycentric rational functions" Tags="|interpolation|stability|automatic-differentiation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29789" PostTypeId="2" ParentId="15907" CreationDate="2018-06-30T09:58:34.157" Score="15" Body="&lt;p&gt;The question has two very different subquestions. I will address the first one only.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Matlab's version runs on average &lt;strong&gt;24 times faster&lt;/strong&gt; than my python&#xA;  equivalent!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The second one is subjective. I would say that letting know the user that there is some problem with the integral is a good thing and this SciPy behavior outperforms the Matlab`s one to keep it silent and somehow try to deal with it internally the way only known by Matlab engineers who decided it to be the best. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I changed integration span to be from &lt;em&gt;0&lt;/em&gt; to &lt;em&gt;30&lt;/em&gt; (instead of from &lt;em&gt;0&lt;/em&gt; to &lt;em&gt;np.inf&lt;/em&gt;) to avoid NaN waring and added a JIT compilation. To benchmark the solution I repeated the integration for 300 times, results are from my laptop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without JIT compilation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./test_integrate.py&#xA;34.20992112159729&#xA;(0.2618828053067563+0.24474506983644717j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With JIT compilation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./test_integrate.py&#xA;0.8560323715209961&#xA;(0.261882805306756+0.24474506983644712j)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This way adding two lines of code leads to the &lt;strong&gt;speedup factor of about 40 times&lt;/strong&gt; of Python code compared to a non-JIT version. I have no Matlab on my laptop to provide a better comparison, however, if it scales well to your PC than 24/40 = 0.6, so &lt;strong&gt;Python with JIT should be almost twice as fast as Matlab&lt;/strong&gt; for this particular user algorithm. Full code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import numpy as np&#xA;from scipy import integrate&#xA;from numba import complex128,float64,jit&#xA;import time&#xA;&#xA;def integral(integrand, a, b,  arg):&#xA;    def real_func(x,arg):&#xA;        return np.real(integrand(x,arg))&#xA;    def imag_func(x,arg):&#xA;        return np.imag(integrand(x,arg))&#xA;    real_integral = integrate.quad(real_func, a, b, args=(arg))&#xA;    imag_integral = integrate.quad(imag_func, a, b, args=(arg))   &#xA;    return real_integral[0] + 1j*imag_integral[0]&#xA;&#xA;vintegral = np.vectorize(integral)&#xA;&#xA;&#xA;@jit(complex128(float64, float64), nopython=True, cache=True)&#xA;def f_integrand(s, omega):&#xA;    sigma = np.pi/(np.pi+2)&#xA;    xs = np.exp(-np.pi*s/(2*sigma))&#xA;    x1 = -2*sigma/np.pi*(np.log(xs/(1+np.sqrt(1-xs**2)))+np.sqrt(1-xs**2))&#xA;    x2 = 1-2*sigma/np.pi*(1-xs)&#xA;    zeta = x2+x1*1j&#xA;    Vc = 1/(2*sigma)&#xA;    theta =  -1*np.arcsin(np.exp(-np.pi/(2.0*sigma)*s))&#xA;    t1 = 1/np.sqrt(1+np.tan(theta)**2)&#xA;    t2 = -1/np.sqrt(1+1/np.tan(theta)**2)&#xA;    return np.real((t1-1j*t2)/np.sqrt(zeta**2-1))*np.exp(1j*omega*s/Vc);&#xA;&#xA;t0 = time.time()&#xA;omega = 10&#xA;for i in range(300): &#xA;    #result = integral(f_integrand, 0, np.inf, omega)&#xA;    result = integral(f_integrand, 0, 30, omega)&#xA;print (time.time()-t0)&#xA;print (result)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Comment out the @jit line to see the difference for your PC.&lt;/p&gt;&#xA;" OwnerUserId="28106" LastActivityDate="2018-06-30T09:58:34.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29791" PostTypeId="2" ParentId="29763" CreationDate="2018-06-30T21:58:36.837" Score="2" Body="&lt;p&gt;If $N$ is on the order of 100,000 and $m$ is on the order of $100$, Then &#xA;$J$ requires about 80 gigabytes to store in double precision and $V$ requires a trivial amount of storage.  The product $M=JV$ is of size $N$ by $m$ and would be fully dense, requiring about 80 megabytes to store.  You should have no trouble storing $V$ or $M$, but you may have to keep $J$ out on disk unless you've got a lot of RAM available.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have $M=JV$ as a matrix, solving the $100,000$ by $100$ linear least squares problem using a QR factorization is quite easy (takes about 1/2 of a second using LAPACK to do the QR factorization on my desktop machine.) Note that you want to use a compact or &quot;Q-less&quot; form of the QR factorization to avoid forming an $N$ by $N$ dense $Q$ matrix.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can take advantage of the structure of $V$ in multiplying out $M=JV$. If $V$ has $r$ rows and columns of blocks, with only the diagonal blocks being nonzero, then you can partition $J$ in corresponding fashion and do the block matrix multiplication as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$M=JV=\left[&#xA;\begin{array}{}&#xA;J_{1,1} V_{1,1} &amp;amp; J_{1,2} V_{2,2} &amp;amp; \ldots &amp;amp; J_{1,r}V_{r,r} \\&#xA;J_{2,1} V_{1,1} &amp;amp; J_{2,2} V_{2,2} &amp;amp; \ldots &amp;amp; J_{2,r}V_{r,r} \\&#xA;\vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots \\&#xA;J_{r,1} V_{1,1} &amp;amp; J_{r,2} V_{2,2} &amp;amp; \ldots &amp;amp; J_{r,r} V_{r,r}&#xA;\end{array}&#xA;\right]&#xA;$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done one block column at a time, and if $J$ is stored in a file would require only one pass through the file.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Iterative methods are not a good idea for this problem.  Although you could do multiplications of $V$ and $V^{T}$ times a vector easily, multiplications involving $J$ would be very expensive.  If you chose to just multiply $M=JV$ once, then you're faced with a dense 100,000 by 100 linear least squares problem which is faster to solve by QR factorization than by iterative methods.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-06-30T21:58:36.837" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29792" PostTypeId="1" AcceptedAnswerId="29798" CreationDate="2018-06-30T23:24:10.030" Score="1" ViewCount="66" Body="&lt;p&gt;Let $A$ be a finite (and small-ish) set of positive real numbers and 0. Let $B$ be a subset of $\mathbb N^0$, up to some (small-ish) bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a function $f(t)$, $A \rightarrow B$ that is strictly monotonically increasing. It is $f(0) = 0$ and $f$ shall cover all values of its codomain $B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In plain English: $f$ starts at (0,0) and always increases by one, but with varying $\Delta t$ in between the steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple continuous version of $f(t)$ is $c(t)$ which I can get by piecewise linear interpolation. However since my $\Delta t$ values are all different, the derivative $c'(t) = dc/dt$ will have discontinuities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for the easiest possible function $g(t)$ (stretching and squeezing $t$) such that $\frac {dc(g(t))}{dt}$ is smooth, with $g(t)=t$ $\forall t \in A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In plain English: How can I turn my simple linear interpolation into something that has a continuous time derivative (but still perfectly interpolates $f$)?&lt;/p&gt;&#xA;" OwnerUserId="740" LastEditorUserId="740" LastEditDate="2018-07-01T14:02:51.900" LastActivityDate="2018-07-02T15:41:42.397" Title="Wanted: smoothing time domain transform" Tags="|interpolation|time-integration|adaptive-timestepping|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29793" PostTypeId="1" AcceptedAnswerId="29794" CreationDate="2018-07-01T00:38:53.433" Score="6" ViewCount="171" Body="&lt;p&gt;I'm working on a fun &lt;a href=&quot;https://github.com/bcrowell/karl&quot; rel=&quot;noreferrer&quot;&gt;open-source project&lt;/a&gt; to calculate the trajectories of objects near black holes. This is obviously not the first time anyone has done this sort of thing, but I have some design goals that have led me to roll my own solution. (These considerations are spelled out in the project's README file, but briefly, I want to be able to construct browser-based educational apps, to model motion that crosses the horizon or hits the singularity, and to simulate things like rocket ships, for which the motion is not a geodesic.) Because of these considerations, I've written my own Runge-Kutta routine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically everything is working fine and passes various tests, except for the following. I want to be able to simulate motion that terminates at a singularity. When I do this, I want to be able to precisely determine the time (or technically the affine parameter) at which it terminates. So in the language of ODEs, I have a vector ODE of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$y'=f(t,y),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in which the function $f$ misbehaves at a known surface, and I want to determine accurately the time $t_0$ at which I hit the surface. &quot;Accurately&quot; would hopefully mean something like the precision with which a 4th-order Runge-Kutta can determine $y$ for a given $t$, i.e., with a reasonable step size I want to be able to get $t_0$ to within rounding errors. There is a &lt;a href=&quot;https://arxiv.org/abs/0705.1029&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt; by Lewis and Kwan that discusses something similar. They say they used software called odepack, but nothing about what routine they used or how they managed the singularity. They quote errors on the order of $10^{-4}$, which is good enough for their purposes but would be sad for a general-purpose library like the one I'm writing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give me pointers on how to proceed here, point me to references, or suggest keywords to search on? I'm not a whiz at numerical methods, although I took an undergraduate course a long time ago. I'm thinking that my ODE is not stiff. I ran across information about things called &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular_solution&quot; rel=&quot;noreferrer&quot;&gt;singular solutions&lt;/a&gt;, but the usage of this term doesn't seem to match up very well with my situation. What I'm dealing with is what a relativist would call an incomplete geodesic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is a description of what I've tried.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried reducing the step size according to a seemingly reasonable rule as I approached the singularity. This sort of worked, but basically I could never get it to behave better than the equivalent of first order, i.e., the relative error in $t$ seemed to scale like the step size $h$, so that I could get down to errors of about $10^{-6}$ with reasonable computational effort, but nowhere near machine precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried doing a change of variables. For solutions that approach the singularity, I had estimates that one of the components of the $y$ vector (it's actually the Schwarzschild radius $r$) should behave as $r\sim (t_0-t)^{2/3}$. So I tried a change of variable to $u=r^{3/2}$. This sort of helped, because with these new coordinates, $u'$ is nearly constant (should be exactly constant for trajectories that free fall from rest at infinity). However, I'm still not having much luck getting an algorithm to determine $t_0$ precisely. What tends to happen is that my techniques for predicting when I'm going to hit the singularity are a little off, so I overshoot, and I start getting garbage results, such as the particle flying back out of the black hole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some algorithm implemented in open-source software that I could try out, and possibly just reimplement? If it's of interest to see the equations concretely, the simplest example I can make up, omitting some degrees of freedom, is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$q''=-r^{-1}(r-1)^{-1} r'q'$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$r''=-\frac{1}{2}r^{-3}(r-1)(q')^2+\frac{1}{2}r^{-1}(r-1)^{-1}(r')^2,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $q$ is actually the Schwarzschild time coordinate, and the primes indicate differentiation with respect to proper time $t$. An example of initial conditions of interest would be $q=0$, $r=0.9$, $q'=9$, $r'=-1.0540925533894594$, for which there is an exact solution giving $t_0=0.5692099788303082$. (Physically, these conditions represent someone who has fallen into the black hole from rest at infinity, starting the simulation just inside the event horizon at $r=1$.)&lt;/p&gt;&#xA;" OwnerDisplayName="user28109" LastActivityDate="2018-07-02T10:50:58.737" Title="Methods for precise solution of an ODE whose solution terminates at a singularity" Tags="|ode|runge-kutta|numerics|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29794" PostTypeId="2" ParentId="29793" CreationDate="2018-07-01T12:09:10.023" Score="4" Body="&lt;p&gt;Here is an example of this can be done in Julia with an &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl&quot; rel=&quot;noreferrer&quot;&gt;ODE library&lt;/a&gt; that can do event location:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using OrdinaryDiffEq&#xA;&#xA;function main()&#xA;    function rhs(u, _p, _t)&#xA;        _q, r, qd, rd = u&#xA;        qdd = -rd*qd/(r*(r-1))&#xA;        rdd = -qd^2*(r-1)/2r^3 + rd^2/(2r*(r-1))&#xA;        [qd, rd, qdd, rdd]&#xA;    end&#xA;    cb = ContinuousCallback((t,u,i) -&amp;gt; u[2] - 1e-6, i -&amp;gt; terminate!(i))&#xA;    prob = ODEProblem(rhs, [0, 0.9, 9, -1.0540925533894594], (0.0, 1.0))&#xA;    sol = solve(prob, Vern9(), reltol=1e-12, abstol=1e-12)&#xA;    sol.t[end] - 0.5692099788303082&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The absolute error in $t_0$ comes out to &lt;code&gt;7.149836278586008e-14&lt;/code&gt;, compared with the value you gave. Like I said in the comment, what makes this problem nontrivial is that you don't have a change of sign in the equation $r(t)=0$, but you can work around that by solving $r(t_1)=\epsilon$, which only differs from $t_0$ by a small error of magnitude $|r(t_1)/r'(t_1)|$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, integrating an ODE next to a singularity should produce lots of extra steps as the solver tries to avoid the singularity itself, so this approach isn't all that efficient, if that matters to you.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-07-01T12:09:10.023" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29795" PostTypeId="2" ParentId="29793" CreationDate="2018-07-01T12:46:31.530" Score="1" Body="&lt;p&gt;Similar to the Julia example, here is some example code using the&#xA;&lt;code&gt;ode15i&lt;/code&gt; function in &lt;a href=&quot;https://www.gnu.org/software/octave&quot; rel=&quot;nofollow noreferrer&quot;&gt;Octave 4.4&lt;/a&gt;.&#xA;The Octave implementation of &lt;code&gt;ode15i&lt;/code&gt; is based on the &#xA;&lt;a href=&quot;https://computation.llnl.gov/projects/sundials/ida&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sundials &lt;/a&gt; IDA solver.&#xA;IDA uses a family of backward difference formulas of orders one through&#xA;five. &lt;code&gt;ode15i&lt;/code&gt; also includes a basic event location capability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Octave code is shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function black_hole_example&#xA;y0=[0 9 .9 -1.0540925533894594]';&#xA;yp0=[0 0 0 0]';&#xA;opts=odeset('Events', @eventFunc, 'RelTol', 1e-12, 'AbsTol', 1e-12);&#xA;[y0,yp0] = decic(@odeFunc,0,y0,[1 1 1 1]',yp0,[0 0 0 0]');&#xA;tspan=[0 1];&#xA;[t,y,te] = ode15i(@odeFunc,tspan,y0,yp0,opts);&#xA;err=te - 0.5692099788303082;&#xA;fprintf('Final time=%16.12f, final r=%10.2e, time error=%10.2e\n', &#xA;te, y(end,3), err);&#xA;figure; plot(t, y(:,3));&#xA;end&#xA;&#xA;function f=odeFunc(t,y,yp)&#xA;  q=y(1); qp=y(2); r=y(3); rp=y(4);&#xA;  qpp=-1/(r*(r-1))*rp*qp;&#xA;  rpp=-.5*r^-3*(r-1)*qp^2 + .5/(r*(r-1))*rp^2;&#xA;  f=yp-[qp qpp rp rpp]';&#xA;end&#xA;&#xA;function [res,isterminal,direction]=eventFunc(t,y,yp)&#xA;res=y(3)-1e-9;&#xA;isterminal=true;&#xA;direction=0;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When run in Octave the results are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Final time=  0.569209978864, final r= 9.99e-010, time error= 3.32e-011&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/owbRq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/owbRq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as with the Julia ODE solver, a relatively large number of time steps&#xA;are required to achieve this result but execution time is less than a&#xA;second on a fairly modest Windows PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I ran this code in Octave, it will also run in MATLAB with few, if&#xA;any, changes.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2018-07-02T10:50:58.737" LastActivityDate="2018-07-02T10:50:58.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29797" PostTypeId="2" ParentId="29749" CreationDate="2018-07-02T01:04:04.230" Score="1" Body="&lt;p&gt;I think that you should take a look to OpenCascade. It provides with Spline surfaces, and NURBS particularly. It is written in C++ and it is &lt;a href=&quot;http://git.dev.opencascade.org/gitweb/?p=occt.git&quot; rel=&quot;nofollow noreferrer&quot;&gt;released under LGPL license&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding FreeCAD, I don't think that they officially support NURBS, although you can import them (?). But there is a module &lt;a href=&quot;https://github.com/microelly2/freecad-nurbs&quot; rel=&quot;nofollow noreferrer&quot;&gt;in development&lt;/a&gt; (see also the &lt;a href=&quot;https://forum.freecadweb.org/viewtopic.php?t=16473&quot; rel=&quot;nofollow noreferrer&quot;&gt;Forum&lt;/a&gt;). The developer has published several &lt;a href=&quot;https://www.youtube.com/user/microelly/videos&quot; rel=&quot;nofollow noreferrer&quot;&gt;examples&lt;/a&gt; already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also suggest to look  at &lt;a href=&quot;https://www.blender.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blender&lt;/a&gt; since it also &lt;a href=&quot;https://en.wikipedia.org/wiki/Blender_(software)#Features&quot; rel=&quot;nofollow noreferrer&quot;&gt;supports NURBS&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both, FreeCAD and Blender have rich Python API, that might be useful if you are a Python user.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-07-02T01:04:04.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29798" PostTypeId="2" ParentId="29792" CreationDate="2018-07-02T08:46:38.877" Score="1" Body="&lt;p&gt;Look up monotone splines, e.g. Wikipedia &lt;a href=&quot;https://en.wikipedia.org/wiki/Monotone_cubic_interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monotone_cubic_interpolation&lt;/a&gt;.&lt;br&gt;&#xA;(Normal cubic splines, see e.g.&#xA;&lt;a href=&quot;http://apps.nrbook.com/empanel/index.html?pg=121&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Recipes pages 120-122&lt;/a&gt;,&#xA;are simpler but, surprisingly, may be non-monotone for monotone data.)&lt;/p&gt;&#xA;" OwnerUserId="1841" LastEditorUserId="1841" LastEditDate="2018-07-02T15:41:42.397" LastActivityDate="2018-07-02T15:41:42.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29799" PostTypeId="1" AcceptedAnswerId="29800" CreationDate="2018-07-02T19:54:39.787" Score="2" ViewCount="140" Body="&lt;p&gt;Imagine I want to consider the temperature for a process given several input varibales. The temperature can be anywhere between 400 and 500 K. Consider I have experimental data to train the network and then I want to predict the temperature for a test point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand the theory of NN activation function are needed to bound the value between 0 and 1. How could I proceed in my example? Should I just scale my data between 0 and 1? Are there good and bad methods to scale (of course there will be?). Or should I modify the activation function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what if one of the training point's is at 600 K. If I bound the values it would be impossible for the NN to reach this value. I hope someone can give me clarity on this issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gesetzt&lt;/p&gt;&#xA;" OwnerUserId="21748" LastActivityDate="2018-07-03T06:08:38.007" Title="Artificial neural networks for Temperature prediction" Tags="|python|computational-physics|machine-learning|neural-network|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29800" PostTypeId="2" ParentId="29799" CreationDate="2018-07-03T06:08:38.007" Score="1" Body="&lt;p&gt;NN activation functions don't need to be between 0 and 1. That's only done for classification problems. Many times you want them continuously differentiable and monotonic, though that isn't even required. RELU activation functions which are $\sigma(x)=max(0,x)$ are quite common for these kinds of scenarios.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-07-03T06:08:38.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29802" PostTypeId="1" CreationDate="2018-07-03T12:03:27.203" Score="8" ViewCount="521" Body="&lt;p&gt;I am trying to solve an APSP (All-Pair Shortest Path) problem on a weighted graph.&#xA;This graph is actually a 1, 2 or 3 dimensional grid, and the weights on each edge represent the distance between its two vertices.&#xA;What I want to have is the geodesic graph distance (shortest path through the graph), for every pair of vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want a diffusion-based method, because it is faster than a Dijkstra, or a Floyd-Warshall algorithm.&#xA;I'm trying to achieve this using the heat equation:&#xA;$$\frac{du}{dt} = \Delta u.$$&#xA;In the end, my application needs a kernel of the form $\exp(-d^2/\gamma)$ with $d$ the graph geodesic distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My hope is that since the solution is supposed to be the &lt;a href=&quot;https://en.wikipedia.org/wiki/Green&amp;#39;s_function#Table_of_Green&amp;#39;s_functions&quot; rel=&quot;noreferrer&quot;&gt;Green function for diffusion&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ u(t,x,y) = \left(\frac{1}{4\pi t}\right)^{-\frac{dim}{2}}\exp\left(\frac{-d^2(x,y)}{4t}\right),$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then I can directly use that solution (with a few adjustements to get rid of the factor in front) as my kernel, and the parameter $\gamma$ will be adjusted by adjusting $t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't been able to do something that works yet, and I would love some help. I have tried many things so far, and there are multiple problems that arise. It is difficult and long to explain all of them in one question, so I will first explain what I think is the beginning of a good approach, and then ask a few general questions.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the same way it is done in the first step of the Geodesic in Heat algorithm by Crane et al., with a backward Euler step, I can solve the linear system:&#xA;$$(Id - tL)u = u_0 \tag{1}$$&#xA;with $t$ the diffusion step, $L$ the laplacian matrix, and $u_0$ a Dirac at one of the vertices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving equation (1) actually gives a kernel of the form $\exp(-d/\gamma)$, which is not desired. Therefore I have to do K subiterations in time, and solve K times:&#xA;$$(Id - \frac{t}{k}L)u = u_0 \iff Mu = u_0$$&#xA;which gives $u = M^{-1}...M^{-1}u_0 \iff u = M^{-K}u_0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As K increases, the kernel is supposed to converge to a square one $\exp(-d^2/\gamma)$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now here come the questions :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Should I use a graph Laplacian, or a finite differences Laplacian ?&lt;/strong&gt; AFAIU, a graph laplacian is normalized to have 1 in the diagonal, whereas a FE Laplacian has the degree in the diagonal, and is multiplied by $\frac{1}{h^2}$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;How do I embed the graph weights in the Laplacian, so that the distance I get in the solution is the graph geodesic distance ?&lt;/strong&gt; I want to be able to predict what will be the range of values of $d(x,y)$ in the solution, with regard to the range of the weights, and the parameters $t$, $K$, and $n$ the number of points in one direction (total domain size: $N = n^{dim}$).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Which boundary conditions should I use in the Laplacian ?&lt;/strong&gt; I feel like I shouldn't impose the function values (Dirichlet) at the boundary, because that would mean imposing the highest distance. Or should I ? I tried homogeneous Neumann conditions, and homogeneous Dircihlet conditions, but in either cases I get some distortion near the boundaries of the parabola $d(x,y)^2$ (which I check by computing the log of the solution $u(t)$, and substracting the minimum).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Should I use a diffusion equation instead ?&lt;/strong&gt; : $\frac{\partial u}{\partial t}=\nabla\cdot\left(\kappa\nabla u\right)$, since the &lt;a href=&quot;https://physics.stackexchange.com/questions/165928/what-is-the-difference-between-the-diffusion-equation-and-the-heat-equation/165930#165930&quot;&gt;diffusion is spatially dependent&lt;/a&gt; ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;References :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Crane et al. 2013 : &lt;a href=&quot;http://ddg.math.uni-goettingen.de/pub/GeodesicsInHeat.pdf&quot; rel=&quot;noreferrer&quot;&gt;Geodesics in heat: A new approach to computing distance based on heat flow&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27975" LastEditorUserId="27975" LastEditDate="2018-07-03T13:59:44.217" LastActivityDate="2018-09-22T20:00:31.230" Title="Computing geodesic distances with diffusion" Tags="|linear-algebra|diffusion|heat-transfer|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29804" PostTypeId="5" CreationDate="2018-07-03T17:32:39.527" Score="0" Body="&lt;p&gt;Numba is an open-source NumPy-aware optimizing compiler for Python &lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-07-03T17:32:39.527" LastActivityDate="2018-07-03T17:32:39.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29805" PostTypeId="4" CreationDate="2018-07-03T17:32:39.527" Score="0" Body="NumPy-aware optimizing compiler for Python" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-07-03T17:32:39.527" LastActivityDate="2018-07-03T17:32:39.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29806" PostTypeId="2" ParentId="11049" CreationDate="2018-07-03T19:55:19.607" Score="6" Body="&lt;p&gt;For the Hankel transform, one can classify the methods into four major groups:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Numerical quadrature-based.&lt;/li&gt;&#xA;&lt;li&gt;Fourier-based ones.&lt;/li&gt;&#xA;&lt;li&gt;Asymptotic expansion of Bessel into sines and cosines.&lt;/li&gt;&#xA;&lt;li&gt;Projection-slice methods.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The following paper gives a nice overview of these methods (types of methods):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1016/0898-1221(93)90081-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;M. J. Cree and P. J. Bones, &quot;Algorithms to numerically evaluate the Hankel transform,&quot; &lt;em&gt;Comp. Math. Appl.&lt;/em&gt;, vol. 26, no. 1, pp. 1–12, Jul. 1993&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;According to this paper (p. 3, Sec. 4):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The main disadvantage of the trapezoidal rule is its poor computational efficiency. ... the trapezoidal method was found to be almost always as reliable as any other method tested. ...we found it useful as a benchmark to test more efficient algorithms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, two directions are possible: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Either find a more efficient numerical quadrature rule.&lt;/li&gt;&#xA;&lt;li&gt;Follow along with the Hankel transform-like direction.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In fact, this integral is very similar to the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;th order Hankel transform. However, I see as a major complication here that &lt;span class=&quot;math-container&quot;&gt;$f(r)$&lt;/span&gt; is not only extremely expensive, but &lt;em&gt;oscillatory&lt;/em&gt;, and, potentially, &lt;em&gt;highly oscillatory&lt;/em&gt;. In the given formulation, &lt;span class=&quot;math-container&quot;&gt;$f(r)$&lt;/span&gt; is almost a black-box with some known properties which are actually bad. A lot of the standard techniques for evaluation Hankel transforms use the fact that &lt;span class=&quot;math-container&quot;&gt;$f(r)$&lt;/span&gt; &lt;a href=&quot;https://doi.org/10.1016/0377-0427(95)00142-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;is eventually monotonic&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of that, I would look into the direction of Filon-type quadrature, that is used for highly-oscillatory integrals. But I think you would have to use some knowledge about the behaviour of &lt;span class=&quot;math-container&quot;&gt;$f(r)$&lt;/span&gt; to make it work. That is regarding the direction 1. The following reference (as well as the one for the next part) will be helpful for general understanding:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1016/0893-9659(96)00067-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;R. Barakat and E. Parshall, &quot;Numerical evaluation of the zero-order Hankel transform using Filon quadrature philosophy,&quot; &lt;em&gt;Appl. Math. Lett.&lt;/em&gt;, vol. 9, no. 5, pp. 21–26, Sep. 1996.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For direction 2, I would advise trying to develop some form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Projection-slice_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;projection-slice methods&lt;/a&gt;. I am personally not aware of a ready method of this type developed for your integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following references might be useful (for the Filon quadrature, as well):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1364/JOSAA.20.000621&quot; rel=&quot;nofollow noreferrer&quot;&gt;J. Markham and J.-A. Conchello, &quot;Numerical evaluation of Hankel transforms for oscillating functions,&quot; &lt;em&gt;J. Opt. Soc. America A&lt;/em&gt;, vol. 20, no. 4, pp. 621–630, 2003.&lt;/a&gt; Here, the authors discuss 6 methods for numerical calculation of the &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;th order Hankel transform of oscillating functions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Another interesting approach worth trying (me stumbling upon this paper was the main motivation to write this answer) is to apply Mellin transform. However, it might not be worth it if it required too many evaluations of &lt;span class=&quot;math-container&quot;&gt;$f(r)$&lt;/span&gt; in the first place:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1109/78.845927&quot; rel=&quot;nofollow noreferrer&quot;&gt;L. Knockaert, &quot;Fast Hankel transform by fast sine and cosine transforms: the Mellin connection,&quot; &lt;em&gt;IEEE Trans. Signal Processing&lt;/em&gt;, vol. 48, no. 6, pp. 1695–1701, Jun. 2000.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2019-11-14T16:55:26.257" LastActivityDate="2019-11-14T16:55:26.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29807" PostTypeId="2" ParentId="29781" CreationDate="2018-07-04T20:17:06.783" Score="2" Body="&lt;p&gt;Let us consider the simplest case, i.e., the case where &lt;span class=&quot;math-container&quot;&gt;$n = 1$&lt;/span&gt;. Rephrasing slightly:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;symmetric and positive definite matrix &lt;span class=&quot;math-container&quot;&gt;$\mathrm A \in \mathbb R^{m \times m}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;positive semidefinite diagonal matrix &lt;span class=&quot;math-container&quot;&gt;$\mathrm D \in \mathbb R^{p \times p}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;find a tall matrix &lt;span class=&quot;math-container&quot;&gt;$\mathrm X \in \mathbb R^{m \times p}$&lt;/span&gt; with orthonormal columns such that &lt;span class=&quot;math-container&quot;&gt;$\rm A \approx X D X^\top$&lt;/span&gt;, or, &lt;span class=&quot;math-container&quot;&gt;$\rm A X \approx X D$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Using the spectral norm, we have the following &lt;strong&gt;non-convex&lt;/strong&gt; optimization problem in &lt;span class=&quot;math-container&quot;&gt;$\mathrm X \in \mathbb R^{m \times p}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \| \mathrm A \mathrm X - \mathrm X \mathrm D \|_2\\ \text{subject to} &amp;amp; \mathrm X^\top \mathrm X = \mathrm I_p\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The feasible region, defined by &lt;span class=&quot;math-container&quot;&gt;$\mathrm X^\top \mathrm X = \mathrm I_p$&lt;/span&gt;, is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Stiefel_manifold&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stiefel manifold&lt;/a&gt;, whose convex hull is defined by &lt;span class=&quot;math-container&quot;&gt;$\mathrm X^\top \mathrm X \preceq \mathrm I_p$&lt;/span&gt;, or, equivalently, by the inequality &lt;span class=&quot;math-container&quot;&gt;$\| \mathrm X \|_2 \leq 1$&lt;/span&gt;. Therefore, a &lt;strong&gt;convex relaxation&lt;/strong&gt; of the original optimization problem is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \| \mathrm A \mathrm X - \mathrm X \mathrm D \|_2\\ \text{subject to} &amp;amp; \| \mathrm X \|_2 \leq 1\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$\rm {\bar X}$&lt;/span&gt; denote the solution of the relaxed problem. If &lt;span class=&quot;math-container&quot;&gt;$\rm {\bar X}^\top \rm {\bar X} \approx \mathrm I_p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rm A {\bar X} \approx \rm {\bar X} D$&lt;/span&gt;, then we may have something we could call a solution of the original problem.&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-07-04T20:48:02.667" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29808" PostTypeId="1" CreationDate="2018-07-05T07:34:22.507" Score="2" ViewCount="120" Body="&lt;p&gt;With the &lt;code&gt;guru&lt;/code&gt; interfaces of FFTW, I can apply transforms only to parts of a multidimensional array by modifying the &lt;code&gt;fftw_iodim&lt;/code&gt; parameter (e.g., with &lt;code&gt;fftw_plan_guru_r2r(...)&lt;/code&gt;). Unfortunately, there seems to be no MPI interface with the same arguments (see, e.g., &lt;code&gt;fftw_mpi_plan_r2r(...)&lt;/code&gt;). Does anybody know how I could apply one of the transforms of FFTW to only part of a multidimensional array (a subarray) using MPI?&lt;/p&gt;&#xA;" OwnerUserId="27170" LastActivityDate="2020-07-29T08:20:34.343" Title="FFTW on subarray with MPI" Tags="|c++|fourier-analysis|mpi|fftw|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29809" PostTypeId="2" ParentId="5526" CreationDate="2018-07-05T12:43:50.207" Score="1" Body="&lt;p&gt;This is typically done by the molecular simulation software that you use. For example, in NAMD this can be accomplished using the &lt;code&gt;psfgen&lt;/code&gt; command (see page 10 of the &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/plugins/psfgen/ug.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;user's guide&lt;/a&gt;). The source code for this command is written in C and exposes a TCL interface, so chances are that you can adapt it to your needs. Keep in mind, though, that after you mutate the residue you may need to do a short equilibration run to get a reasonable geometry for your system.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2018-07-05T12:43:50.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29810" PostTypeId="1" CreationDate="2018-07-05T13:47:40.817" Score="3" ViewCount="72" Body="&lt;p&gt;I'm trying to understand the staircase map.  Look at this map from the circle to itself:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ x \stackrel{F}{\mapsto} \big[\omega + x + \tfrac{\epsilon}{2\pi} \sin (2\pi x) \big] \pmod 1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such a map arises when applying Euler's method to solve a differential equation.  Then the rotation number as a function of $\omega$ is found by iterating $F$ many times:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \rho(\omega, \epsilon) = \lim_{n \to \infty} \frac{F^n(\omega, \epsilon, x_0)}{n}  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so I tried to find the rotation number in the instance provided in the &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0387971416&quot; rel=&quot;nofollow noreferrer&quot;&gt;textbook&lt;/a&gt; (Dynamics and bifurcations, Hale &amp;amp; Koçak, 1991)  the code is straightforward:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;w = np.arange(0,1,0.01)&#xA;x0  = 0&#xA;x   = 0*w + x0&#xA;eps = 0.5&#xA;&#xA;for n in 1+np.arange(100):&#xA;    x = (w + x + (eps/2/np.pi)*np.sin(2*np.pi*x))%1&#xA;&#xA;plt.plot(w,x/n)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In fact, I don't get a staircase at all.  Am I coding it correctly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lWOj1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lWOj1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4006" LastEditorUserId="9667" LastEditDate="2018-07-05T16:09:37.793" LastActivityDate="2018-07-05T16:15:58.563" Title="why am I not getting a staircase for the rotation number?" Tags="|python|iterative-method|numpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29811" PostTypeId="2" ParentId="4716" CreationDate="2018-07-05T15:49:37.597" Score="3" Body="&lt;p&gt;We have a system of two equations in &lt;span class=&quot;math-container&quot;&gt;$\mathrm X \in \mathbb R^{n \times n}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{aligned}\rm D X a &amp;amp;= \rm X b\\ \rm X^\top X &amp;amp;= \rm I_n\end{aligned}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The convex hull of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Orthogonal_group&quot; rel=&quot;nofollow noreferrer&quot;&gt;orthogonal group&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathrm O (n)$&lt;/span&gt; is defined by &lt;span class=&quot;math-container&quot;&gt;$\mathrm X^\top \mathrm X \preceq \mathrm I_n$&lt;/span&gt;, or, equivalently, by the inequality &lt;span class=&quot;math-container&quot;&gt;$\| \mathrm X \|_2 \leq 1$&lt;/span&gt;. Hence, we obtain the following (convex) feasibility problem&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; 0\\ \text{subject to} &amp;amp; \rm D X a = \rm X b\\ &amp;amp; \| \mathrm X \|_2 \leq 1\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let us try to minimize the trace of &lt;span class=&quot;math-container&quot;&gt;$\rm X$&lt;/span&gt; instead.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \mbox{tr} (\mathrm X)\\ \text{subject to} &amp;amp; \rm D X a = \rm X b\\ &amp;amp; \| \mathrm X \|_2 \leq 1\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Numerical experiment&lt;/h3&gt;&#xA;&lt;p&gt;Using NumPy to randomly generate &lt;span class=&quot;math-container&quot;&gt;$\rm a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\rm b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rm D$&lt;/span&gt; and &lt;a href=&quot;http://www.cvxpy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CVXPY&lt;/a&gt; to solve the convex program,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from cvxpy import *&#xA;import numpy as np&#xA;&#xA;np.set_printoptions(linewidth=250)&#xA;&#xA;n = 16&#xA;&#xA;a = np.random.rand(n)&#xA;b = np.random.rand(n)&#xA;D = np.diag(2 * np.random.randint(2,size=n) - 1)&#xA;I = np.identity(n)&#xA;&#xA;X = Variable(n,n)&#xA;&#xA;# define optimization problem&#xA;prob = Problem( Minimize(trace(X)), [ D*X*a == X*b, norm(X,2) &amp;lt;= 1 ] )&#xA;&#xA;# solve optimization problem&#xA;print prob.solve()&#xA;print prob.status&#xA;&#xA;# print results&#xA;print &amp;quot;X = \n&amp;quot;, X.value&#xA;print &amp;quot;Spectral norm of X = &amp;quot;, np.linalg.norm(X.value,2)&#xA;print &amp;quot;Round(X.T * X) = \n&amp;quot;, np.round((X.value).T * (X.value), 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which outputs the following&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-14.9470050407&#xA;optimal&#xA;X = &#xA;[[ -8.90251732e-01   2.33984981e-01   1.70258281e-01  -5.53391898e-02   2.34985256e-02   6.37982593e-02   2.18083930e-01   2.82596557e-02  -4.40395916e-03  -1.34227730e-03   1.50316724e-02   1.14689633e-01   9.74802430e-02   8.16953368e-02   6.42347352e-02   1.14863690e-01]&#xA; [ -2.48934037e-01  -8.34448790e-01   1.20541848e-01  -4.61428969e-02   1.10340611e-02  -1.38274027e-01   1.54718718e-01   1.58408510e-02   1.58655384e-02   4.16141469e-03   3.60515363e-03  -2.61841776e-01  -2.20909661e-01  -1.85873715e-01  -1.43558666e-01   7.68506567e-02]&#xA; [ -1.81207641e-01   1.20537511e-01  -9.12297754e-01  -3.37076829e-02   7.93129006e-03  -1.00006758e-01   1.12614657e-01   1.14552887e-02   1.21451662e-02   3.14253621e-03   2.49890970e-03  -1.90783327e-01  -1.60793846e-01  -1.35366758e-01  -1.04288378e-01   5.58544984e-02]&#xA; [  7.43758735e-02  -4.61861906e-02  -3.37397718e-02  -9.61551455e-01   1.74836317e-02  -8.98819614e-02  -4.44710999e-02   1.08682106e-02  -1.25474295e-01  -2.41955988e-02   2.48250432e-02   1.14611542e-01   6.31907668e-02   6.82729945e-02  -2.36503319e-04  -5.08569560e-03]&#xA; [ -1.25531195e-02   1.09967538e-02   7.90498685e-03   1.74965174e-02  -9.82789579e-01  -1.12302141e-01   9.21095895e-03   1.33248212e-02  -9.61305432e-02  -1.82172436e-02   2.09786819e-02   1.60028081e-02  -1.33975818e-02   8.53570200e-04  -4.18649149e-02   1.82271837e-02]&#xA; [  6.38057525e-02   1.24718862e-01   9.00934514e-02   1.04032856e-01   1.19729834e-01  -9.31499844e-01   1.09339291e-01   9.47648922e-02   2.64131010e-02   4.72762859e-03   1.42689857e-01   5.79300210e-02   5.73323636e-02   4.44262653e-02   4.76842316e-02   1.46438318e-01]&#xA; [ -2.32816373e-01   1.54719429e-01   1.12619209e-01  -4.44310509e-02   9.24567851e-03  -1.22537590e-01  -8.55441086e-01   1.40088873e-02   2.10810529e-02   5.07876003e-03   2.03310798e-03  -2.46769430e-01  -2.06461090e-01  -1.74497661e-01  -1.32033702e-01   7.09507849e-02]&#xA; [ -2.08115816e-02   1.58125336e-02   1.14355020e-02   1.08800825e-02   1.33265812e-02  -8.98683871e-02   1.39824856e-02  -9.89390841e-01  -7.07377866e-02  -1.33519272e-02   1.57628810e-02  -1.76591484e-04  -2.01471347e-02  -7.93638089e-03  -3.77446716e-02   1.70947859e-02]&#xA; [ -4.39615929e-03  -1.97439812e-02  -1.49717925e-02   1.27547846e-01   9.66702553e-02   2.64168741e-02  -2.47554289e-02   7.09595290e-02  -9.73213228e-01   5.12198340e-03   1.22666446e-01  -1.26261876e-02  -3.28268619e-03  -6.08434806e-03   6.95146187e-03   6.87239971e-02]&#xA; [ -1.34092041e-03  -4.83000977e-03  -3.62959570e-03   2.45018201e-02   1.82691521e-02   4.72827724e-03  -5.70953876e-03   1.33595657e-02   5.12198880e-03  -9.99045890e-01   2.32500551e-02  -2.92806032e-03  -1.07265164e-03  -1.53261064e-03   1.02586508e-03   1.25360928e-02]&#xA; [ -3.55257001e-04   3.55897148e-03   2.46608728e-03   2.48380511e-02   2.09761965e-02  -1.32517696e-01   1.99019610e-03   1.57588137e-02  -1.21746730e-01  -2.31431712e-02  -9.73903901e-01   3.63175408e-02  -3.15147533e-03   1.25820724e-02  -4.37034836e-02   1.81394397e-02]&#xA; [  1.14687468e-01   2.47573480e-01   1.80328301e-01  -9.55781671e-02  -4.86251696e-03   5.79213957e-02   2.32663340e-01   7.80107025e-03  -1.26341107e-02  -2.92943113e-03  -2.14388161e-02  -8.77792555e-01   1.01651871e-01   8.61957889e-02   6.42372713e-02   9.79062923e-02]&#xA; [  9.74807096e-02   2.07531845e-01   1.50995626e-01  -4.62218968e-02   2.31387972e-02   5.73256867e-02   1.93280542e-01   2.67724031e-02  -3.28989498e-03  -1.07390893e-03   1.62177826e-02   1.01654343e-01  -9.13451926e-01   7.24778004e-02   5.72604694e-02   1.03703534e-01]&#xA; [  8.16951801e-02   1.75222981e-01   1.27564138e-01  -5.44075733e-02   7.18739633e-03   4.44202822e-02   1.63985606e-01   1.34234104e-02  -6.09057287e-03  -1.53369672e-03  -1.82081989e-03   8.61973527e-02   7.24772812e-02  -9.38929859e-01   4.67971943e-02   7.77429043e-02]&#xA; [  6.42376700e-02   1.33212905e-01   9.67155701e-02   1.23871809e-02   4.86178123e-02   4.76808343e-02   1.21890605e-01   4.22879094e-02   6.94643031e-03   1.02498470e-03   5.28282150e-02   6.42416314e-02   5.72628819e-02   4.67996353e-02  -9.59038190e-01   9.34053301e-02]&#xA; [ -1.12307485e-01   7.68178288e-02   5.58331499e-02  -5.05776188e-03   1.82415250e-02  -1.46103311e-01   7.09199777e-02   1.71041618e-02  -6.98869555e-02  -1.27692448e-02   1.81593462e-02  -9.49159570e-02  -1.01458672e-01  -7.57289231e-02  -9.22890030e-02  -9.53958411e-01]]&#xA;Spectral norm of X =  1.0000572062&#xA;Round(X.T * X) = &#xA;[[ 1.  0.  0. -0. -0. -0.  0. -0. -0. -0. -0. -0. -0. -0. -0.  0.]&#xA; [ 0.  1. -0.  0.  0.  0. -0.  0.  0.  0.  0.  0.  0.  0.  0. -0.]&#xA; [ 0. -0.  1.  0.  0.  0. -0.  0.  0.  0.  0.  0.  0.  0.  0. -0.]&#xA; [-0.  0.  0.  1. -0. -0.  0. -0. -0. -0. -0. -0. -0. -0. -0.  0.]&#xA; [-0.  0.  0. -0.  1. -0.  0. -0. -0. -0. -0. -0. -0. -0. -0.  0.]&#xA; [-0.  0.  0. -0. -0.  1.  0. -0. -0. -0. -0. -0. -0. -0. -0.  0.]&#xA; [ 0. -0. -0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0. -0.]&#xA; [-0.  0.  0. -0. -0. -0.  0.  1. -0. -0. -0. -0. -0. -0. -0.  0.]&#xA; [-0.  0.  0. -0. -0. -0.  0. -0.  1. -0. -0. -0. -0. -0. -0.  0.]&#xA; [-0.  0.  0. -0. -0. -0.  0. -0. -0.  1. -0. -0. -0. -0. -0.  0.]&#xA; [-0.  0.  0. -0. -0. -0.  0. -0. -0. -0.  1. -0. -0. -0. -0.  0.]&#xA; [-0.  0.  0. -0. -0. -0.  0. -0. -0. -0. -0.  1. -0. -0. -0.  0.]&#xA; [-0.  0.  0. -0. -0. -0.  0. -0. -0. -0. -0. -0.  1. -0. -0.  0.]&#xA; [-0.  0.  0. -0. -0. -0.  0. -0. -0. -0. -0. -0. -0.  1. -0.  0.]&#xA; [-0.  0.  0. -0. -0. -0.  0. -0. -0. -0. -0. -0. -0. -0.  1.  0.]&#xA; [ 0. -0. -0.  0.  0.  0. -0.  0.  0.  0.  0.  0.  0.  0.  0.  1.]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I ran the Python script several times until I obtained a matrix &lt;span class=&quot;math-container&quot;&gt;$\rm X$&lt;/span&gt; that is approximately orthogonal. In other words, the algorithm does not produce such nice results for all choices of &lt;span class=&quot;math-container&quot;&gt;$\rm (a,b,D)$&lt;/span&gt;. Also, the larger the value of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, the better the results seem to be — which is why I picked &lt;span class=&quot;math-container&quot;&gt;$n=16$&lt;/span&gt;, not &lt;span class=&quot;math-container&quot;&gt;$n=3$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20417" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-07-06T16:11:26.303" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29812" PostTypeId="2" ParentId="29810" CreationDate="2018-07-05T16:15:58.563" Score="3" Body="&lt;p&gt;The problem is your explicitly taking the result of $F$ to be modulo 1. This messes up the calculation of the winding number. Simply removing &lt;code&gt;%1&lt;/code&gt; will yield the correct winding number. To see why you &lt;em&gt;want&lt;/em&gt; $F$ to be able to exceed 1, inspect the definition of the winding number&#xA;$$&#xA;\rho(\omega,\epsilon) = \lim_{n\to\infty} \frac{F^n(x_0;\omega, \epsilon)-x_0}{n}&#xA;$$&#xA;Since $x\in[0,1]$ parameterizes the unit circle,  $F(x_0)-x_0$ is the distance around the unit circle by which $F$ displaces $x_0$, and $(F^n(x_0)-x_0)/n$ is the mean displacement caused by $n$ applications of $F$. So the winding number is this mean displacement in the limit of infinitely many applications of $F$. But if the result of $F$ is always taken modulo 1, then larger and larger values of $n$ will result in ever smaller winding numbers, eventually becoming $0$, whatever the values of $\omega$ and $\epsilon$. This is obviously not capturing the intended meaning of the winding number.&lt;/p&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2018-07-05T16:15:58.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29813" PostTypeId="1" AcceptedAnswerId="29817" CreationDate="2018-07-05T17:17:39.710" Score="4" ViewCount="229" Body="&lt;p&gt;In the book &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A Multigrid Tutorial - Briggs, Henson. McCormick&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;in the beginning of Chapter 3, it is mentioned that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...because the convergence factor behaves as &lt;em&gt;1-$O(h^{2})$&lt;/em&gt;, the coarse grid will have a marginally improved convergence rate...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here, $h$ is the mesh spacing (say from Finite Difference Discretization). &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Question&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;How is this factor of $1-O(h^{2})$ derived ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it mean solution on low resolution grids converges faster than high resolution grids ? But if this is the case then say $h=\frac{1}{2}$, then $1-\frac{1}{4}=\frac{3}{4}$ is the convergence rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When $h=\frac{1}{4}$ then the convergence rate is $1-\frac{1}{16}=\frac{15}{16}$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly the latter is more and hence fine grids have better convergence rates with this approximation. What am I missing ?&lt;/p&gt;&#xA;" OwnerUserId="14303" LastActivityDate="2018-07-05T21:00:19.870" Title="Convergence rate Jacobi/Gauss-Seidel with mesh resolution" Tags="|finite-difference|iterative-method|convergence|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29815" PostTypeId="1" CreationDate="2018-07-05T20:13:25.193" Score="1" ViewCount="214" Body="&lt;p&gt;I have to deal with the following problem in my research:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\left[\frac{1}{D}F_{x}\right]_{x} + \frac{f D_{x}}{c D^{2}}F = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$F(0) = 0$$&#xA;$$F_{x}(L) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $f$ is a known constant and $D(x)$ is a known function of $x$.  I should clarify that $D(x)$ is known numerically for all $x$ (I have a vector of values), but it is &lt;em&gt;not&lt;/em&gt; known as a closed-form analytical function and is not readily approximated as such.  Subscripts denote derivatives.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I note that this is a Sturm-Liouville problem:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[p(x)F_{x}(x)]_{x} + q(x)F(x) = -\lambda r(x) F(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $p = 1/D$, $q = 0$, and $r = f D_{x}/D^{2}$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$F$ are the eigenvectors and $\lambda = 1/c$ are the eigenvalues.  Is there an easy way to solve for them numerically?  Thanks for any help.  &lt;/p&gt;&#xA;" OwnerUserId="28136" LastActivityDate="2018-07-07T16:39:16.467" Title="Numerical solution for eigenvectors and eigenvalues of a Sturm-Liouville problem" Tags="|ode|eigenvalues|numerics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29817" PostTypeId="2" ParentId="29813" CreationDate="2018-07-05T21:00:19.870" Score="5" Body="&lt;p&gt;The convergence rate that is mentioned here is in the sense that the error in iteration $k$ and $k-1$ are related by&#xA;$$&#xA;  \| x^{(k)} - x^\ast\| \le r \| x^{(k-1)} - x^\ast\|,&#xA;$$&#xA;which implies that &#xA;$$&#xA;  \| x^{(k)} - x^\ast\| \le r^k \| x^{(0)} - x^\ast\|.&#xA;$$&#xA;For this to converge at all, we need that $r&amp;lt;1$, which the statement you quote provides. But for fast convergence, we need that $r$ is not marginally smaller than one, but ideally close to zero -- say, $r=0.1$, in which case you'd reduce the error by a factor of ten in each iteration. On the other hand, $r=1-ch^2$ is close to one if $h$ is small enough, and so it will take a large number of iterations until the reduction $r^k$ over the initial residual is substantial.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-07-05T21:00:19.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29818" PostTypeId="1" AcceptedAnswerId="29866" CreationDate="2018-07-06T13:58:15.550" Score="2" ViewCount="180" Body="&lt;p&gt;I am working on an XY model and I'm trying to calculate the magnetization and direction for an anti-ferromagnetic (AF) system.  So I have a collection of spins in the $XY$ plane represented as vectors $s$ and when &lt;em&gt;cooled&lt;/em&gt; then they will order (anti-align).  The magnetization is normally calculated as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$M = \frac{1}{N}\sum_i^N s_i,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This formula does not describe how aligned the spins are (it's like it only works accurately for a ferro-type behavior). What suggestions are available for measuring the order in an AF system (I'm also interested in the invariance as well)?&lt;/p&gt;&#xA;" OwnerUserId="24134" LastEditorUserId="20688" LastEditDate="2018-07-16T14:34:17.447" LastActivityDate="2018-07-16T14:34:17.447" Title="Magnetization Vector from XY Model for an AntiFerromagnetic System" Tags="|quantum-mechanics|electromagnetism|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29820" PostTypeId="2" ParentId="29815" CreationDate="2018-07-07T16:39:16.467" Score="1" Body="&lt;p&gt;D'oh!  I didn't realize how easy this was.  My practical experience with numerical methods begins and ends with finite differences.  I got caught up by the fact that this is an eigenvalue problem.  I didn't realize that if I replace the derivatives with centered difference operators this reduces to a simple matrix eigenvalue problem!  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Letting $D1$ and $D2$ be centered difference matrices, and with $p$ and $r$ defined as above, I can write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[D2 + diag(p_{x}/p)D1]F = \lambda[diag(-r/p)]F$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$AF = \lambda BF$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Dirichlet condition at $x=0$ is easily built into the first rows of $A$ and $B$ and the Neumann condition at $x=L$ can be built in as well with a ghost point.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a test, the eigenvectors and eigenvalues have an analytical form when $D(x) = a \exp(2bx)$, and my finite difference method compares with sufficient accuracy for my needs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the comments.  Sorry this was so simple.  &lt;/p&gt;&#xA;" OwnerUserId="28136" LastActivityDate="2018-07-07T16:39:16.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29821" PostTypeId="1" CreationDate="2018-07-07T19:37:21.820" Score="4" ViewCount="176" Body="&lt;p&gt;I'm using Albany w/ Trilinos to solve an elasticity problem with thermal expansion mismatch. I'm using block GMRES with MueLu preconditioning. It works for problem size of several million dofs, but when the problem size gets bigger, the true residual won't converge even though the preconditioned residual converges. I've tried different parameters for MueLu (sym Gauss-Seidel, Jacobi, ...), but it won't work. Any suggestions on the convergence problem? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the problem size that converges with Block GMRES with MueLu, I tried Block GMRES with ILU preconditioner, and the true residual norm stays 1. Does that mean I have a singular preconditioner with ILU? Why did that happen?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="28147" LastActivityDate="2018-07-07T19:37:21.820" Title="Preconditioned residual converges, but true residual doesn't" Tags="|convergence|preconditioning|gmres|trilinos|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29823" PostTypeId="1" AcceptedAnswerId="30729" CreationDate="2018-07-07T15:52:27.337" Score="0" ViewCount="990" Body="&lt;p&gt;I want to plot the Poincare map for &lt;a href=&quot;https://en.wikipedia.org/wiki/Arnold%E2%80%93Beltrami%E2%80%93Childress_flow&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arnold-Beltrami-Childress&lt;/a&gt; magnetic field for parameters $A=1, B=0.816, C=0.5773$ in Python for the Poincare section $z=0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am not able to understand what initial conditions should I vary for $x$, $y$, and $z$ so as to plot the entire map? I am able to get only a subset of the plot for $x=0$, $y=0$, and $z=0$ but how should I vary $x$, $y$, and $z$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ODEs that I have used for the same:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d\mathbf{r}}{dt}=\frac{\bf{B}}{|\mathbf{B}|}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\mathbf{r}=(x,y,z)$ and &#xA;$&#xA;\mathbf{B}=\left(&#xA;\begin{array}{c}&#xA;A\sin(z)+C\cos(y)\\&#xA;B\sin(x)+A\cos(z)\\&#xA;C\sin(y)+B\cos(x)&#xA;\end{array}&#xA;\right)&#xA;$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please provide me with the Python code?&lt;/p&gt;&#xA;" OwnerUserId="28151" OwnerDisplayName="Nakul Aggarwal" LastEditorUserId="20688" LastEditDate="2018-07-07T23:01:23.677" LastActivityDate="2019-05-15T12:01:11.233" Title="Poincare map for Arnold-Beltrami-Childress Magnetic Field in Python" Tags="|python|fluid-dynamics|computational-physics|electromagnetism|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29824" PostTypeId="1" CreationDate="2018-07-07T22:52:35.017" Score="8" ViewCount="213" Body="&lt;p&gt;I have a function $g(x)$ defined numerically that is sort of in between a Gaussian and a Lorentzian. It decays much slower than a Gaussian, but still faster than a simple inverse power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pNhzF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pNhzF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to calculate its Fourier transform $f(t)\equiv \mathcal{F}[g(x)](t)$ for large $t$. Because function calls to $g(x)$ are computationally expensive, I define an interpolation of $g(x)$ - call it $g_{\text{int}}(x)$ - on some huge range of $x$, $-40&amp;lt;x&amp;lt;40$, and use that for my integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(t)=\int_{-\infty}^{\infty}\cos (tx)g(x)\,dx\,\,\underset{\approx}{\longrightarrow}\,\,\,\int_{-L}^{L}\cos(tx)g_{\text{int}}(x)\,dx$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I calculate an approximation to the Fourier transform, I get some odd oscillations that I would not initially expect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V86I9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V86I9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I have indicated in the picture above, the oscillations have a &quot;period&quot; of about 15.7. My first guess would be that this might be an artifact of the alternating nature of cancellation of the integral, but that would not explain the observed &quot;period&quot; of 15.7.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_{\text{guess}}=\frac{2\pi}{L}\approx 0.157\ldots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is exactly a factor of 100 different from what I observe (yes, I have checked that I defined my integrals and horizontal axes correctly). &lt;strong&gt;How could this be?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Edit #1: Interpolation Details&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am interpolating with Mathematica's built-in &lt;code&gt;Interpolation&lt;/code&gt;, which interpolates between successive points with a cubic curve (so at each point up to the 2$^{\text{nd}}$ derivative is defined). I am specifically interpolating the function $g(x)$ over the range $-40&amp;lt;x&amp;lt;40$ in steps of $dx=40/100=0.4$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, now that I write that, I realize it could very well be an artifact of my finite sampling, because:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$T_{\text{guess #2}}=\frac{2\pi}{dx}=\frac{2\pi}{0.4}=15.7\ldots$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any further help on this, in particular a good way to overcome this problem.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Edit #2: The function $g(x)$&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h[x_?NumericQ, En_?NumericQ, pz_?NumericQ] := &#xA; 1./(En^2 + pz^2 + 0.24^2)*&#xA;  NIntegrate[((Sqrt[&#xA;      0.316/(1. + &#xA;         1.2*((k4 + 0.5*En)^2 + kp + (x*pz)^2))^1.*0.316/(1. + &#xA;         1.2*((k4 - 0.5*En)^2 + kp + ((1. - x)*pz)^2))^1.])*((1. - &#xA;         x)*0.316/(1. + 1.2*((k4 + 0.5*En)^2 + kp + (x*pz)^2))^1. + &#xA;      x*0.316/(1. + &#xA;         1.2*((k4 - 0.5*En)^2 + kp + ((1. - x)*pz)^2))^1.))/(((k4 + &#xA;        0.5*En)^2 + &#xA;      kp + (x*pz)^2 + (0.316/(1. + &#xA;         1.2*((k4 + 0.5*En)^2 + kp + (x*pz)^2))^1.)^2)*((k4 - &#xA;        0.5*En)^2 + &#xA;      kp + ((1. - x)*&#xA;        pz)^2 + (0.316/(1. + &#xA;         1.2*((k4 - 0.5*En)^2 + &#xA;            kp + ((1. - x)*&#xA;              pz)^2))^1.)^2)), {k4, -\[Infinity], \[Infinity]}, {kp, &#xA;    0, \[Infinity]}, Method -&amp;gt; &quot;LocalAdaptive&quot;, &#xA;   MaxRecursion -&amp;gt; &#xA;    100]; (*LocalAdaptive seems to work slightly faster *)&#xA;&#xA;g[x_]:=h[0.5,x,2.]; (*this is the function*)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9562" LastEditorUserId="9562" LastEditDate="2018-07-08T18:55:03.813" LastActivityDate="2018-07-08T18:55:03.813" Title="What are these oscillations?" Tags="|interpolation|integration|mathematica|oscillations|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="29828" PostTypeId="2" ParentId="29824" CreationDate="2018-07-08T08:38:49.123" Score="5" Body="&lt;p&gt;I want to mention another idea in case it helps. The truncation error of replacing $\int_{-\infty}^{\infty}$ with $\int_{-L}^L$ is on the order of&#xA;$$\begin{aligned}&#xA; \int_L^\infty \cos(tx)g(x)\,\mathrm{d}x &amp;amp;= \frac{1}{t}\sin(tx)g(x)\big|_{L}^{\infty} - \int_L^\infty\frac{\sin tx}{t}g'(x)\,\mathrm{d}x&#xA;\\&amp;amp;= -\frac1L g(L)\sin(tL) + \text{asymptotically smaller terms}.&#xA;\end{aligned} $$&#xA;I took one omitted interval, used integration by parts, and assumed that $g'$ decays faster than $g$, which is usually reasonable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $g(L)$ decays like a power and not exponentially or like a Gaussian, then $g(L)$ might not decay fast enough to suppress the leading term in the truncation error of the Fourier integral, and you would then see an oscillating error. One important consequence would be that this error would remain even if the interpolation step is perfectly accurate.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-07-08T08:38:49.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29830" PostTypeId="1" AcceptedAnswerId="29846" CreationDate="2018-07-09T01:37:06.273" Score="-1" ViewCount="905" Body="&lt;p&gt;Here is the Schrodinger equation that is to be solved:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A 1D hard wall potential in $[0, 1]$. The potential within the potential well is given by a linear combination of Gaussian dips &#xA;$$v(x) = - \sum_{i=1}^3a_ie^{-\frac{(x-b_i)^{2}}{2c_i^2}}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the time-independent Schrodinger equation to be solved is&#xA;$$(-\frac{h^2}{2m}\partial_x^{2}+v(x))\psi(x)=E\psi(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the paper that I read, the interval $[0, 1]$ is discretized into dense grid points in the first place. How can I solve it numerically in such a way? &lt;/p&gt;&#xA;" OwnerUserId="28157" LastActivityDate="2018-08-14T20:12:36.960" Title="Solving Schrodinger equation numerically" Tags="|computational-physics|quantum-mechanics|" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29831" PostTypeId="2" ParentId="19522" CreationDate="2018-07-09T04:26:37.627" Score="5" Body="&lt;p&gt;Technically, the template Matlab code provided at &lt;a href=&quot;http://www.netlib.org/templates/matlab/gmres.m&quot; rel=&quot;noreferrer&quot;&gt;Netlib&lt;/a&gt; already calculates the obtained residual at every iteration. It's just not recorded in the way that this is an output of the subroutine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After each iteration &lt;code&gt;iter&lt;/code&gt; the residual &lt;code&gt;r&lt;/code&gt; is calculated, so one just needs to record it. I will do it right after it is calculated in the residual vector &lt;code&gt;resvec(iter)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the provided GMRES code is not optimal (for example, the residual &lt;code&gt;r&lt;/code&gt; is calculated both before and after the loop over &lt;code&gt;i&lt;/code&gt;-variable: construction of the orthonormal basis), so I am not going to optimize that behaviour, but just count matrix-vector products (MVP) every time they happen in &lt;code&gt;mvp_count&lt;/code&gt;. Then, at the end of each iteration, I will record the number of actually performed MVPs in &lt;code&gt;mvpvec(iter)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both &lt;code&gt;resvec&lt;/code&gt; and &lt;code&gt;mvpvec&lt;/code&gt; are added to the output parameters of the &lt;code&gt;gmres&lt;/code&gt; function which should provide enough information to do the plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;mvpvec(iter)&lt;/code&gt; counts MVPs performed &lt;strong&gt;BY&lt;/strong&gt; the end of the &lt;code&gt;iter&lt;/code&gt;th iteration by this particular implementation of &lt;code&gt;gmres&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;no normalization by the 2-norm of the RHS &lt;code&gt;b&lt;/code&gt; is done (to avoid confusion between residul, relative error, etc.). Should be easily changed either inside or outside of the function.&lt;/li&gt;&#xA;&lt;li&gt;only MVPs with the supposedly large original matrix $A$ are being counted&lt;/li&gt;&#xA;&lt;li&gt;the initial residual and the corresponding MVP (before iterative process starts) are not recorded.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is the slightly modified Matlab code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [x, error, iter, flag, resvec, mvpvec] = gmres( A, x, b, M, restrt, max_it, tol )&#xA;&#xA;%  -- Iterative template routine --&#xA;%     Univ. of Tennessee and Oak Ridge National Laboratory&#xA;%     October 1, 1993&#xA;%     Details of this algorithm are described in &quot;Templates for the&#xA;%     Solution of Linear Systems: Building Blocks for Iterative&#xA;%     Methods&quot;, Barrett, Berry, Chan, Demmel, Donato, Dongarra,&#xA;%     Eijkhout, Pozo, Romine, and van der Vorst, SIAM Publications,&#xA;%     1993. (ftp netlib2.cs.utk.edu; cd linalg; get templates.ps).&#xA;%&#xA;%     Sligtly modified by Anton Menshov to answer CompSci SE question in 2018.&#xA;%&#xA;% [x, error, iter, flag, resvec, mvpvec] = gmres( A, x, b, M, restrt, max_it, tol )&#xA;%&#xA;% gmres.m solves the linear system Ax=b&#xA;% using the Generalized Minimal residual ( GMRESm ) method with restarts .&#xA;%&#xA;% input   A        REAL nonsymmetric positive definite matrix&#xA;%         x        REAL initial guess vector&#xA;%         b        REAL right hand side vector&#xA;%         M        REAL preconditioner matrix&#xA;%         restrt   INTEGER number of iterations between restarts&#xA;%         max_it   INTEGER maximum number of iterations&#xA;%         tol      REAL error tolerance&#xA;%&#xA;% output  x        REAL solution vector&#xA;%         error    REAL error norm&#xA;%         iter     INTEGER number of iterations performed&#xA;%         flag     INTEGER: 0 = solution found to tolerance&#xA;%                           1 = no convergence given max_it&#xA;%         resvec   REAL (iter x 1) vector of norm2 solution resuduals at each iteration&#xA;%         mvpvec   INTEGER (iter x 1) vector of ACTUALLY computed MVPs at each iteration&#xA;&#xA;iter = 0;                                         % initialization&#xA;flag = 0;&#xA;mvp_count =0;&#xA;&#xA;bnrm2 = norm( b );&#xA;if  ( bnrm2 == 0.0 ), bnrm2 = 1.0; end&#xA;&#xA;r = M \ ( b-A*x );     % not recorded and MVP is not counted&#xA;error = norm( r ) / bnrm2;&#xA;if ( error &amp;lt; tol ) return, end&#xA;&#xA;[n,n] = size(A);                                  % initialize workspace&#xA;m = restrt;&#xA;V(1:n,1:m+1) = zeros(n,m+1);&#xA;H(1:m+1,1:m) = zeros(m+1,m);&#xA;cs(1:m) = zeros(m,1);&#xA;sn(1:m) = zeros(m,1);&#xA;e1    = zeros(n,1);&#xA;e1(1) = 1.0;&#xA;&#xA;&#xA;for iter = 1:max_it,                                    % begin iteration&#xA;     r = M \ ( b-A*x );   mvp_count = mvp_count+1;&#xA;     V(:,1) = r / norm( r );&#xA;     s = norm( r )*e1;&#xA;     for i = 1:m,                                       % construct orthonormal&#xA;         w = M \ (A*V(:,i));   mvp_count = mvp_count+1; % basis using Gram-Schmidt&#xA;         for k = 1:i,&#xA;             H(k,i)= w'*V(:,k);&#xA;             w = w - H(k,i)*V(:,k);&#xA;         end&#xA;         H(i+1,i) = norm( w );&#xA;         V(:,i+1) = w / H(i+1,i);&#xA;         for k = 1:i-1,                              % apply Givens rotation&#xA;             temp     =  cs(k)*H(k,i) + sn(k)*H(k+1,i);&#xA;             H(k+1,i) = -sn(k)*H(k,i) + cs(k)*H(k+1,i);&#xA;             H(k,i)   = temp;&#xA;         end&#xA;         [cs(i),sn(i)] = rotmat( H(i,i), H(i+1,i) ); % form i-th rotation matrix&#xA;         temp   = cs(i)*s(i);                        % approximate residual norm&#xA;         s(i+1) = -sn(i)*s(i);&#xA;         s(i)   = temp;&#xA;         H(i,i) = cs(i)*H(i,i) + sn(i)*H(i+1,i);&#xA;         H(i+1,i) = 0.0;&#xA;         error  = abs(s(i+1)) / bnrm2;&#xA;         if ( error &amp;lt;= tol ),                        % update approximation&#xA;            y = H(1:i,1:i) \ s(1:i);                 % and exit&#xA;            x = x + V(:,1:i)*y;&#xA;            break;&#xA;         end&#xA;     end&#xA;&#xA;     if ( error &amp;lt;= tol ), break, end&#xA;     y = H(1:m,1:m) \ s(1:m);&#xA;     x = x + V(:,1:m)*y;                            % update approximation&#xA;     r = M \ ( b-A*x ); mvp_count = mvp_count+1;    % compute residual&#xA;     resvec(iter) = r;                              % store the residual AT the iter's iteration&#xA;     mvpvec(iter) = mvp_count;                      % store ACTUALLY performed MVP count BY iter's iteration&#xA;     s(i+1) = norm(r);&#xA;     error = s(i+1) / bnrm2;                        % check convergence&#xA;     if ( error &amp;lt;= tol ), break, end;&#xA;end&#xA;&#xA;if ( error &amp;gt; tol ) flag = 1; end;                 % converged&#xA;% END of gmres.m&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-07-09T04:26:37.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29832" PostTypeId="1" AcceptedAnswerId="29844" CreationDate="2018-07-09T09:49:28.253" Score="0" ViewCount="384" Body="&lt;p&gt;I'm playing with the K-Means clustering (Documents Clustering) but i realised that the partitions are changing with each execution because the initial nodes are taken by the kmean++ algorithm. I would like to know if there is some method to find the &quot;Average&quot; partitioning scheme. I mean, after 100 executions, take the 100 observations, and calculate the average partitioning scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some method to do this?&lt;/p&gt;&#xA;" OwnerUserId="28162" LastActivityDate="2018-07-12T21:13:46.743" Title="K-Means Clustering &quot;Average&quot;?" Tags="|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29833" PostTypeId="2" ParentId="29729" CreationDate="2018-07-10T09:30:18.077" Score="2" Body="&lt;p&gt;&lt;em&gt;Disclaimer: This is most likely not the most efficient, promising way to go about it.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I was looking up the same topic from a similar interest (Analyzing microscopy images), I came across the following publication which peaked my interest, as it was rather straight forward to implement it in my coding environment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Image Registration Using Log Polar Transform and Phase Correlation to Recover Higher Scale&lt;/strong&gt;&lt;br&gt;&#xA;  JOURNAL OF PATTERN RECOGNITION RESEARCH 7 (2012) 90-105&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Testing this out, it worked sort of fine within the limits described in the paper as well (a not too extensive scaling factor.) I think your question will depend &lt;em&gt;enourmously&lt;/em&gt; on what application you have in mind, and what simplifications, assumptions, limitations are associated with it.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Edit: I did my tests with that algorithm in a scripting language within a scientific image/data processing software called &lt;em&gt;DigitalMicrograph&lt;/em&gt;. If anyone is interested, I can make that script code available (just comment below). &#xA;A free version of the software (with no timeout) can be downloaded from the &lt;a href=&quot;http://www.gatan.com/installation-instructions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gatan Inc.&lt;/a&gt; webpage.&lt;/p&gt;&#xA;" OwnerUserId="28168" LastEditorUserId="28168" LastEditDate="2018-08-01T14:44:59.427" LastActivityDate="2018-08-01T14:44:59.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29834" PostTypeId="2" ParentId="206" CreationDate="2018-07-10T11:01:45.353" Score="3" Body="&lt;p&gt;For bigger code bases, tests (not necessarily unit tests) for the highlevel stuff are useful. Unit tests for some simpler algorithms are useful as well to make sure your code isn't doing nonsense because your helper function is using &lt;code&gt;sin&lt;/code&gt; instead of &lt;code&gt;cos&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for the overall research code it is very hard to write and maintain tests. Algorithms tend to be large without meaningful intermediate results which can have obvious tests and often take a long time to run before there is a result. Of course you can test against reference runs which had nice results, but this is not a good test in the unit test sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Results often are approximations of the true solution. While you can test your simple functions if they are accurate up to some epsilon, it will be very hard to verify if i.e. some result mesh is correct or not, which was evaluated by visual inspection by the user (you) before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In such cases automated tests often have a too high cost/benefit ratio. I recommend something better: Write test programs.&#xA;For example I wrote a medium size python script to create data about results, like histograms of edge sizes and angles of a mesh, area of the biggest and smallest triangle and their ratio, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can both use it to evaluate input and output meshes during normal operation &lt;em&gt;and&lt;/em&gt; use it to have a sanity check after I changed the algorithm. When I change the algorithm I do not always know if the new result is better, because there often is no absolute measure which approximation is the best. But by generating such metrics I can tell about some factors what is better like &quot;The new variant eventually has a better angle ratio but a worse convergence rate&quot;.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastActivityDate="2018-07-10T11:01:45.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29835" PostTypeId="1" AcceptedAnswerId="29838" CreationDate="2018-07-10T11:15:07.393" Score="4" ViewCount="675" Body="&lt;p&gt;I have the value of a function in three dimensions, $f(x, y, z)$, which varies smoothly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to compute a 3D mesh where $f(x, y, z)$ has a particular value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there algorithms to do this? (And if so, any implementations?)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note: This is the 3D equivalent of finding a contour of a 2D function, $f(x, y)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tU5NG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tU5NG.png&quot; alt=&quot;2D contour examlpe&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28170" LastActivityDate="2018-07-10T15:19:26.313" Title="3D contour mesh computation" Tags="|computational-geometry|mesh-generation|mesh|visualization|3d|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29836" PostTypeId="1" CreationDate="2018-07-10T11:56:31.227" Score="1" ViewCount="138" Body="&lt;p&gt;I have written a Roe solver with Harten entropy fix code in Matlab to numerically solve the one-dimensional Shallow Water Equations. : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{eqnarray}&#xA;\dfrac{\partial h(x,t)}{\partial t} + \dfrac{\partial}{\partial x}(h(x,t).v(x,t)) &amp;amp; = &amp;amp; 0\\&#xA;\dfrac{\partial h(x,t).v(x,t)}{\partial t} + \dfrac{\partial}{\partial x}(h(x,t).v(x,t)^2 + \dfrac{1}{2}g.h(x,t)) &amp;amp; = &amp;amp; 0 \\&#xA;x \in [0, L], t \in \mathbb{R}&#xA;\end{eqnarray}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $h$ is the fluid height, $v$ its speed and $g$ the gravitational acceleration. The boundary conditions on the speed are $v(0,t) = v(L,t) = 0, \forall t$. Because of the lack of boundary condition on the height, I impose that the flux applied on the height is zero at the boundaries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the Roe solver as described in &quot;&lt;em&gt;Riemann Solvers and Numerical Methods for Fluid Dynamics&lt;/em&gt;&quot; by Eleuterio F. Toro (p. 357) and added an Harten entropy fix. My variables are $h(x,t)$ and $h(x,t).v(x,t)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To verify my code I implemented two method :&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I compute the total energy of the fluid E as :&#xA;$$E(t) = \dfrac{1}{2} \int_0^L \left(  h.v^2 + g.h^2\right)dx $$ and see if it remains constant.&lt;/li&gt;&#xA;&lt;li&gt;I compute the integrals of the equation on the whole &quot;volume&quot;: &#xA;$$\dfrac{\partial}{\partial t} \int_0^L h dx = 0$$&#xA;$$\dfrac{\partial}{\partial t} \int_0^L (h.v) dx + \dfrac{1}{2}g[h(L,t)^2 - h(0,t)^2] = 0$$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;After running my code, I see that the second method seems to works as both integrals are around $10^{-13}$. However by the plot of the total energy through time show a loss of energy that I can't explain. I try to increase the number of cells, to decrease the step-time and to change the Harten entropy fix parameter, without success. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compare my results I also used different codes online, such as James Adams' one on Mathworks (&lt;a href=&quot;https://fr.mathworks.com/matlabcentral/fileexchange/46475-1d-shallow-water-equations-dam-break&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fr.mathworks.com/matlabcentral/fileexchange/46475-1d-shallow-water-equations-dam-break&lt;/a&gt;). But they all shown this energy loss. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively I would explain this loss by a numerical viscosity, but I'm not sure as CFD is not my domain of competence. So I come here to have both an explanation and a solution please. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code I wrote is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;&#xA;close all;&#xA;clc;&#xA;warning off;&#xA;&#xA;%% ------------------------------------------------------------ PREPARATION&#xA;%% Parametres :&#xA;N     = 250;             % Nombre de cellules&#xA;rho   = 1000;            % Densite (kg/m^3);&#xA;g     = 9.81;            % acceleration gravitationnelle (kg/m/s^2)&#xA;L     = 1;               % Longueur du reservoir (m)&#xA;h_bar = 1;             % Niveau de fluide moyen&#xA;Dx    = L/N;             % Pas spatial (m)&#xA;t_fin = 5;               % Temps final de la simulation (s)&#xA;xx    = linspace(0,L,N); % Vecteur pour plot_fluid.m&#xA;x_vec = ((1:N)-1/2)*Dx;  % Vecteur des les centres des cellules&#xA;C     = 20;              % Fréquence d'affichage des plots pour la vidéo&#xA;&#xA;%% Switchs:&#xA;VIDEO = 0; % Sauvegarde une vidéo&#xA;&#xA;%% Initialisation/Allocation :&#xA;H     = NaN(N,1); % Hauteur du fluide&#xA;V     = NaN(N,1); % Moment du fluide&#xA;D     = 0;        % Position du réservoir&#xA;dot_D = 0;        % Vitesse du réservoir&#xA;&#xA;U_new  = NaN(N,2);&#xA;u      = NaN(N,1); % Vitesse du fluide dans le ref. du réservoir&#xA;X_f    = NaN(1);&#xA;lambda = NaN(N,2);&#xA;&#xA;time     = 0;&#xA;n        = 1;&#xA;telapsed = 0;&#xA;compteur = 1;&#xA;&#xA;newSubFolder = sprintf('./IMAGES');&#xA;if ~exist(newSubFolder, 'dir')&#xA;    mkdir(newSubFolder);&#xA;else&#xA;    rmdir(newSubFolder,'s');&#xA;    mkdir(newSubFolder);&#xA;end&#xA;&#xA;&#xA;%% -------------------------------------------------- SIMULATION TEMPORELLE&#xA;&#xA;fprintf('Début de la simulation temporelle\n')&#xA;&#xA;while (time(end) &amp;lt; t_fin)&#xA;&#xA;    tstart = tic;&#xA;    if(mod(n,200) == 0)&#xA;        if(telapsed &amp;gt; 60)&#xA;            fprintf('It. n°: %d - T. Simu: %.3f/%.3f s - T. Exec: %0.3f min\n',n,time(n),t_fin, telapsed/60)&#xA;        else&#xA;            fprintf('It. n°: %d - T. Simu: %.3f/%.3f s - T. Exec: %0.3f s\n',n,time(n),t_fin, telapsed)&#xA;        end&#xA;    end&#xA;&#xA;    if(n==1)&#xA;        %% Conditions initiales&#xA;        %H(:,n) = h_bar*ones(N,1);  % Fluide à plat&#xA;        H(:,n) = h_bar*(1+1*exp(-(x_vec-L/2).^2/(2*(0.1)^2))); % Hump&#xA;        %H(:,n)   = h_bar*(1+0.25*sin(x_vec*2*pi/L)); % Sin wave&#xA;        V(:,n)  = zeros(N,1); % Fluide au repos&#xA;    else&#xA;        %% Mise à jour des variables&#xA;        H(:,n)      = U_new(:,1);&#xA;        V(2:N-1,n)  = U_new(2:N-1,2);&#xA;    end&#xA;&#xA;    % Application des CL&#xA;    V(1,n) = dot_D;&#xA;    V(N,n) = dot_D;&#xA;&#xA;    % Calcul du vecteur vitesse&#xA;    u(:,n) = V(:,n) - dot_D;&#xA;&#xA;    %% Calcul du pas de temps&#xA;    % Valeurs propres de la matrice de Roe&#xA;    Dt = 0.99*Dx/max( abs(u(:,n)) + sqrt(g*H(:,n)) );&#xA;&#xA;    %% Calcul du vecteur [α1; α2] au temps suivant par un schéma Godunov-Roe&#xA;    U = [H(:,n), V(:,n)];&#xA;    U_new = Roe_Solver(U,dot_D,g,Dt,Dx,N);&#xA;&#xA;    %% Calcul de la position du centre de gravité du fluide:&#xA;    mf(n)  = rho*Dx*sum(H(:,n));&#xA;    X_f(n) = rho*Dx*sum(x_vec'.*H(:,n))/mf(n);&#xA;&#xA;    %% Calcul de l'énergie totale du fluide&#xA;    E_tot(n) = Dx*sum(0.5*H(:,n).*V(:,n).^2) + Dx*sum(0.5*g*H(:,n).^2);&#xA;&#xA;    %% Verification from Saint-Venant equations&#xA;    if (n&amp;gt;1)&#xA;        EQ1_Verif(n) = Dx*(sum(H(:,n))-sum(H(:,n-1)))/Dt;&#xA;        EQ2_Verif(n) = Dx*(sum(V(:,n))-sum(V(:,n-1)))/Dt + g*H(N,n) - g*H(1,n);&#xA;    end&#xA;&#xA;    %% Plots pour la vidéo:&#xA;    if(VIDEO==1 &amp;amp;&amp;amp; ( mod(n,C) == 0 || n == 1) )&#xA;        fig = figure('visible','off');&#xA;        set(fig, 'Position', [0 0 600 500])&#xA;&#xA;        titlefig = sprintf('Système ballotant (t=%.4f s)',time(n));&#xA;        plot_fluid(time,D,u(:,n),H,X_f,L,h_bar,titlefig,n,N,xx);&#xA;&#xA;        filename = sprintf('IMG-%d.png',compteur);&#xA;        saveas(fig,strcat(newSubFolder,'/',filename));&#xA;&#xA;        clf(fig);&#xA;&#xA;        compteur = compteur +1;&#xA;    end&#xA;&#xA;    % Incrementation&#xA;    n = n+1;&#xA;    % Vecteur temporel&#xA;    time(n) = time(end)+Dt;&#xA;    % Temps écoulé&#xA;    telapsed = toc(tstart) + telapsed;&#xA;end&#xA;&#xA;time = time(1:end-1);&#xA;&#xA;%% Calculs d'erreurs&#xA;figure(101)&#xA;plot(time,mf)&#xA;title('Fluid total mass');&#xA;&#xA;figure(102)&#xA;hold on&#xA;plot(time,EQ1_Verif,'r');&#xA;plot(time,EQ2_Verif,'b')&#xA;legend('H','V');&#xA;&#xA;figure(103)&#xA;hold on&#xA;plot(time,E_tot,'r');&#xA;title('Total Energy');&#xA;&#xA;fprintf('Fin d''exécution\n');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function U_new = Roe_Solver(U,dot_D,g,Dt,Dx,N)&#xA;&#xA;U_new = NaN(N,2);&#xA;&#xA;for i=1&#xA;    % Calcul de F_{i-1/2}&#xA;    F_m = [0 ; 0];&#xA;&#xA;    % Calcul de F_{i+1/2}&#xA;    UL  = U(1,:);&#xA;    UR  = U(2,:);&#xA;    F_p = Flux(UL,UR,dot_D,g);&#xA;&#xA;    U_new(1,:) =  U(1,:) - (Dt/Dx)*(F_p - F_m)';&#xA;end&#xA;&#xA;for i=N&#xA;    % Calcul de F_{i-1/2}&#xA;    UL  = U(N-1,:);&#xA;    UR  = U(N,:);&#xA;    F_m = Flux(UL,UR,dot_D,g);&#xA;&#xA;    % Calcul de F_{i+1/2}&#xA;    F_p = [0 ; 0];&#xA;&#xA;    U_new(N,:) =  U(N,:) - (Dt/Dx)*(F_p - F_m)';&#xA;end&#xA;&#xA;for i=2:N-1&#xA;    % Calcul de F_{i-1/2}&#xA;    UL  = U(i-1,:);&#xA;    UR  = U(i,:);&#xA;    F_m = Flux(UL,UR,dot_D,g);&#xA;&#xA;    % Calcul de F_{i+1/2}&#xA;    UL  = U(i,:);&#xA;    UR  = U(i+1,:);&#xA;    F_p = Flux(UL,UR,dot_D,g);&#xA;&#xA;    U_new(i,:) =  U(i,:) - (Dt/Dx)*(F_p - F_m)';&#xA;end&#xA;&#xA;end&#xA;&#xA;function F_p = Flux(UL,UR,dot_D,g)&#xA;% cf. E. Toro, &quot;Riemann Solvers and Numerical Methods for Fluid Dynamics&quot;, P. 357&#xA;&#xA;HL = UL(1);&#xA;VL = UL(2);&#xA;HR = UR(1);&#xA;VR = UR(2);&#xA;&#xA;% Roe average values&#xA;H_tilde = (HL + HR)/2;&#xA;V_tilde = (VL + VR)/2;&#xA;&#xA;A_tilde = [V_tilde - dot_D, H_tilde; g, V_tilde - dot_D];&#xA;[EVec, EVal] = eig(A_tilde);&#xA;&#xA;% Averaged eigenvalues&#xA;lambda_tilde_1 = EVal(1,1);&#xA;lambda_tilde_2 = EVal(2,2);&#xA;&#xA;% Averaged right eigenvectors&#xA;K_tilde_1 = EVec(:,1);&#xA;K_tilde_2 = EVec(:,2);&#xA;&#xA;% Wave strenghts&#xA;alpha_tilde_vec = inv(EVec)*[HR - HL; VR - VL];&#xA;alpha_tilde_1   = alpha_tilde_vec(1);&#xA;alpha_tilde_2   = alpha_tilde_vec(2);&#xA;% Flux&#xA;FR = [HR*(VR - dot_D); (VR^2)/2 - VR*dot_D + g*HR];&#xA;FL = [HL*(VL - dot_D); (VL^2)/2 - VL*dot_D + g*HL];&#xA;F_p = 0.5*(FL + FR) ...&#xA;      - 0.5*Harten_fix(lambda_tilde_1)*alpha_tilde_1*K_tilde_1 ... &#xA;      - 0.5*Harten_fix(lambda_tilde_2)*alpha_tilde_2*K_tilde_2;&#xA;&#xA;end&#xA;&#xA;function abs_lambda_Harten = Harten_fix(lambda)&#xA;&#xA;delta = 0.5; % Harten entropy fix parameter    &#xA;&#xA;for(i=1:2)&#xA;&#xA;    if(lambda &amp;lt; -delta || lambda &amp;gt; delta)&#xA;        abs_lambda_Harten = abs(lambda);&#xA;    else&#xA;        abs_lambda_Harten = 0.5*( (lambda^2)/delta + delta);&#xA;    end&#xA;end&#xA;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help.&lt;/p&gt;&#xA;" OwnerUserId="28171" LastActivityDate="2018-07-10T12:39:17.220" Title="Loss of energy when using Roe Solver for solving onedimensional Shallow Water Equations" Tags="|fluid-dynamics|computational-physics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29837" PostTypeId="2" ParentId="29836" CreationDate="2018-07-10T12:39:17.220" Score="1" Body="&lt;p&gt;Energy will not be conserved. You guessed it right that it is due to numerical viscosity. For smooth solutions, energy should be conserved to a better accuracy. You can use a high order method to get better energy conservation but it will never be perfectly conserved. The energy actually plays the role of entropy for this model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use a central difference scheme with say RK4 time integration which would be stable for smooth solutions and will have better energy conservation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will find more literature in numerical weather prediction papers regarding energy conservation. There are staggered grid schemes that can achieve this, see [1].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some FE methods that also conserve energy, see [2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Arakawa and Lamb, Monthly Weather Review, Vol. 109, 1980. &#xA;&lt;a href=&quot;https://doi.org/10.1175/1520-0493(1981)109%3C0018:APEAEC%3E2.0.CO;2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1175/1520-0493(1981)109%3C0018:APEAEC%3E2.0.CO;2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Mcrae and Cotter, QJRMS, 2013&#xA;&lt;a href=&quot;https://doi.org/10.1002/qj.2291&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1002/qj.2291&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-07-10T12:39:17.220" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29838" PostTypeId="2" ParentId="29835" CreationDate="2018-07-10T12:43:05.480" Score="7" Body="&lt;p&gt;I think you could use the &quot;marching cubes&quot; algorithm. If memory serves, it requires a grid of samples as input, so at the very least you should be able to sample your function and run the algorithm as-is. You also might be able to modify the algorithm to callback to f directly. There's a popular implementation at &lt;a href=&quot;http://paulbourke.net/geometry/polygonise/&quot; rel=&quot;noreferrer&quot;&gt;http://paulbourke.net/geometry/polygonise/&lt;/a&gt; that might help you get started.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2018-07-10T12:43:05.480" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29839" PostTypeId="2" ParentId="29835" CreationDate="2018-07-10T15:19:26.313" Score="5" Body="&lt;p&gt;In addition to the voxel-based approach that rchilton suggests, you could also look at Delaunay-type algorithms. For example, the Computational Geometry Algorithms Library (CGAL) has some built-in functionality for surface mesh generation with examples &lt;a href=&quot;https://doc.cgal.org/latest/Surface_mesher/index.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. You could also try &lt;a href=&quot;http://persson.berkeley.edu/distmesh/&quot; rel=&quot;noreferrer&quot;&gt;distmesh&lt;/a&gt;, the essential idea of which has been ported to a number of other libraries and programming languages. There's also an interesting &lt;a href=&quot;https://mirams.wordpress.com/2014/01/08/quick-and-easy-meshing/&quot; rel=&quot;noreferrer&quot;&gt;blog post&lt;/a&gt; about getting distmesh3d to work just right.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-07-10T15:19:26.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29840" PostTypeId="1" CreationDate="2018-07-11T06:35:11.233" Score="2" ViewCount="293" Body="&lt;p&gt;(note: I also posted this question on &lt;a href=&quot;https://stackoverflow.com/questions/51274069/convergence-of-a-very-large-non-linear-least-squares-optimization&quot;&gt;stackoverflow&lt;/a&gt;  before finding this community here, which seems a better place for it)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to solve the following problem: I have a lot (~80000) surface patches of an organ that's growing. I measure each of its areas over time (18 time-points) and want to fit a growth curve to it (bi-logistic model, eg. just the sum of two logistic functions bcs. there are two 'growth spurts' happening in the observed period).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have box constraints to ensure that the exponential terms don't explode and a linear constraint that one growth spurt has to happen after the other. Also, in order to enforce some sort of spatial continuity in the fitted parameters, I add a penalty term to the objective function (least squares) given as the squared sum of differences between (some of the) parameters of neighbouring patches, so the individual model fits are not independent anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have gradients and the hessian of the whole thing, and I solve it at increasingly finer scales, starting with the overall surface area, subdividing, mapping the parameters of the global model fit to each patch, running the solver again, subdividing ... until I get my desired resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, optimization using IPOPT works, but it is terribly slow, and since I'm a bit of a noob in all things concerning optimization theory I was wondering if I'm doing something terribly stupid in the settings. I'm using ma86 as linear solver compiled against openBLAS running on a machine with 256G RAM and 56 cores as well as metis reordering. The other settings I use are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% nlp scaling&#xA;solv_options.ipopt.nlp_scaling_method = 'gradient-based';&#xA;solv_options.ipopt.nlp_scaling_max_gradient = 1;&#xA;solv_options.ipopt.nlp_scaling_min_value = 1e-16;&#xA;&#xA;solv_options.ipopt.bound_mult_init_method='constant';&#xA;&#xA;% Barrier Parameter&#xA;solv_options.ipopt.mu_strategy = 'adaptive';&#xA;solv_options.ipopt.mu_oracle = 'quality-function';&#xA;solv_options.ipopt.fixed_mu_oracle = 'average_compl';&#xA;solv_options.ipopt.adaptive_mu_globalization = 'kkt-error';&#xA;solv_options.ipopt.corrector_type = 'affine';&#xA;&#xA;% linear solver&#xA;solv_options.ipopt.max_soc=0;&#xA;solv_options.ipopt.accept_every_trial_step='yes';&#xA;solv_options.ipopt.linear_system_scaling = 'none';&#xA;solv_options.ipopt.neg_curv_test_tol = 0;&#xA;solv_options.ipopt.neg_curv_test_reg = 'yes';&#xA;solv_options.ipopt.max_refinement_steps=0;&#xA;solv_options.ipopt.min_refinement_steps=0;&#xA;&#xA;% ma86 settings&#xA;solv_options.ipopt.linear_solver='ma86';&#xA;solv_options.ipopt.ma86_order='auto';&#xA;solv_options.ipopt.ma86_scaling='mc64';&#xA;solv_options.ipopt.ma86_small=1e-10;&#xA;solv_options.ipopt.ma86_static=1;&#xA;solv_options.ipopt.recalc_y='yes';&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this gives me something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;This is Ipopt version 3.12, running with linear solver ma86.&#xA;&#xA;Number of nonzeros in equality constraint Jacobian...:        0&#xA;Number of nonzeros in inequality constraint Jacobian.:     2560&#xA;Number of nonzeros in Lagrangian Hessian.............:   112280&#xA;&#xA;Total number of variables............................:     8960&#xA;                     variables with only lower bounds:     2560&#xA;                variables with lower and upper bounds:     3840&#xA;                     variables with only upper bounds:     2560&#xA;Total number of equality constraints.................:        0&#xA;Total number of inequality constraints...............:     1280&#xA;        inequality constraints with only lower bounds:        0&#xA;   inequality constraints with lower and upper bounds:        0&#xA;        inequality constraints with only upper bounds:     1280&#xA;&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;   0  9.8260736e-01 0.00e+00 1.19e-02   0.0 0.00e+00    -  0.00e+00 0.00e+00   0&#xA;   1  9.6288666e-01 0.00e+00 1.24e-02  -4.5 1.14e-02   0.0 9.91e-01 1.00e+00f  1&#xA;   2  9.1582880e-01 0.00e+00 1.16e-02  -4.4 2.72e-02  -0.5 1.00e+00 1.00e+00f  1&#xA;   3  8.2635857e-01 0.00e+00 1.01e-02  -4.6 1.39e-01  -1.0 9.99e-01 1.00e+00f  1&#xA;   4  7.8943781e-01 0.00e+00 9.40e-03  -4.9 2.70e-02  -0.5 1.00e+00 1.00e+00f  1&#xA;   5  7.2123624e-01 0.00e+00 8.12e-03  -5.3 8.70e-02  -1.0 1.00e+00 1.00e+00f  1&#xA;   6  6.9535003e-01 0.00e+00 7.56e-03  -6.1 2.20e-02  -0.6 1.00e+00 9.06e-01f  1&#xA;   7  6.6635914e-01 0.00e+00 7.00e-03  -6.6 6.21e-02  -1.1 1.00e+00 5.40e-01f  1&#xA;   8  6.5683787e-01 0.00e+00 6.78e-03  -7.6 2.14e-02  -0.6 1.00e+00 3.81e-01f  1&#xA;   9  6.4238130e-01 0.00e+00 6.53e-03  -7.7 1.53e-01  -1.1 1.00e+00 2.90e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  10  6.3440625e-01 0.00e+00 6.33e-03  -8.7 2.20e-02  -0.7 1.00e+00 3.30e-01f  1&#xA;  11  6.2431009e-01 0.00e+00 6.16e-03  -8.3 3.08e-01  -1.2 1.00e+00 2.04e-01f  1&#xA;  12  6.1872460e-01 0.00e+00 6.05e-03  -8.8 2.29e-02  -0.7 1.00e+00 2.32e-01f  1&#xA;  13  6.0753815e-01 0.00e+00 5.86e-03  -8.9 2.73e-01  -1.2 1.00e+00 2.31e-01f  1&#xA;  14  6.0575477e-01 0.00e+00 5.82e-03  -9.9 2.60e-02  -0.8 1.00e+00 7.44e-02f  1&#xA;  15  6.0089103e-01 0.00e+00 5.71e-03 -11.0 1.11e-02  -0.4 1.00e+00 4.48e-01f  1&#xA;  16  5.9426852e-01 0.00e+00 5.58e-03 -11.0 3.14e-02  -0.8 1.00e+00 2.68e-01f  1&#xA;  17  5.9175418e-01 0.00e+00 5.52e-03 -11.0 1.17e-02  -0.4 1.00e+00 2.26e-01f  1&#xA;  18  5.8756155e-01 0.00e+00 5.44e-03 -11.0 3.97e-02  -0.9 1.00e+00 1.65e-01f  1&#xA;  19  5.8651702e-01 0.00e+00 5.42e-03  -9.5 1.26e-02  -0.5 1.00e+00 8.89e-02f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  20  5.8278151e-01 0.00e+00 5.34e-03 -10.3 5.18e-02  -0.9 1.00e+00 1.40e-01f  1&#xA;  21  5.8100614e-01 0.00e+00 5.30e-03 -11.0 1.37e-02  -0.5 1.00e+00 1.42e-01f  1&#xA;  22  5.7827149e-01 0.00e+00 5.25e-03 -11.0 7.12e-02  -1.0 1.00e+00 9.77e-02f  1&#xA;  23  5.7564914e-01 0.00e+00 5.19e-03  -9.4 1.52e-02  -0.6 1.00e+00 1.97e-01f  1&#xA;  24  5.7340095e-01 0.00e+00 5.15e-03 -10.0 1.06e-01  -1.0 1.00e+00 7.66e-02f  1&#xA;  25  5.7134734e-01 0.00e+00 5.10e-03 -11.0 1.78e-02  -0.6 1.00e+00 1.45e-01f  1&#xA;  26  5.6897098e-01 0.00e+00 5.06e-03 -11.0 1.82e-01  -1.1 1.00e+00 7.65e-02f  1&#xA;  27  5.6722484e-01 0.00e+00 5.02e-03 -11.0 2.11e-02  -0.7 1.00e+00 1.16e-01f  1&#xA;  28  5.6402838e-01 0.00e+00 5.25e-03 -11.0 3.29e-01  -1.1 1.00e+00 9.57e-02f  1&#xA;  29  5.6299118e-01 0.00e+00 5.26e-03 -10.5 2.38e-02  -0.7 1.00e+00 6.53e-02f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  30  5.5753178e-01 0.00e+00 5.63e-03 -11.0 1.38e-01  -1.2 1.00e+00 1.62e-01f  1&#xA;  31  5.5730217e-01 0.00e+00 5.63e-03 -11.0 2.37e-02  -0.8 1.00e+00 1.38e-02f  1&#xA;  32  5.5363398e-01 0.00e+00 5.71e-03 -11.0 8.55e-02  -1.2 1.00e+00 1.06e-01f  1&#xA;  33  5.5223091e-01 0.00e+00 5.70e-03 -11.0 2.43e-02  -0.8 1.00e+00 7.99e-02f  1&#xA;  34  5.5163441e-01 0.00e+00 5.70e-03  -9.9 1.30e-01  -1.3 1.00e+00 1.65e-02f  1&#xA;  35  5.4699694e-01 0.00e+00 5.67e-03 -10.9 2.56e-02  -0.9 1.00e+00 2.48e-01f  1&#xA;  36  5.4559780e-01 0.00e+00 5.67e-03 -11.0 2.30e-01  -1.3 1.00e+00 3.71e-02f  1&#xA;  37  5.4248376e-01 0.00e+00 5.63e-03 -11.0 2.58e-02  -0.9 1.00e+00 1.60e-01f  1&#xA;  38  5.3955718e-01 0.00e+00 5.63e-03 -10.7 6.78e-01  -1.4 1.00e+00 6.97e-02f  1&#xA;  39  5.3606672e-01 0.00e+00 5.57e-03 -10.9 2.79e-02  -1.0 1.00e+00 1.73e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  40  5.3481638e-01 0.00e+00 5.55e-03 -11.0 1.32e-02  -0.5 1.00e+00 1.33e-01f  1&#xA;  41  5.3248177e-01 0.00e+00 5.50e-03 -11.0 3.23e-02  -1.0 1.00e+00 1.10e-01f  1&#xA;  42  5.3143109e-01 0.00e+00 5.48e-03 -10.4 1.37e-02  -0.6 1.00e+00 1.05e-01f  1&#xA;  43  5.2975978e-01 0.00e+00 5.45e-03 -11.0 3.92e-02  -1.1 1.00e+00 7.50e-02f  1&#xA;  44  5.2855042e-01 0.00e+00 5.42e-03 -11.0 1.50e-02  -0.6 1.00e+00 1.13e-01f  1&#xA;  45  5.2670676e-01 0.00e+00 5.38e-03 -11.0 4.80e-02  -1.1 1.00e+00 7.85e-02f  1&#xA;  46  5.2428812e-01 0.00e+00 5.31e-03 -11.0 1.64e-02  -0.7 1.00e+00 2.12e-01f  1&#xA;  47  5.2221881e-01 0.00e+00 5.27e-03 -11.0 5.84e-02  -1.2 1.00e+00 8.45e-02f  1&#xA;  48  5.2084516e-01 0.00e+00 5.23e-03 -10.8 1.79e-02  -0.8 1.00e+00 1.14e-01f  1&#xA;  49  5.1928549e-01 0.00e+00 5.19e-03 -11.0 7.06e-02  -1.2 1.00e+00 6.06e-02f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  50  5.1787195e-01 0.00e+00 5.15e-03 -11.0 1.95e-02  -0.8 1.00e+00 1.10e-01f  1&#xA;  51  5.1639782e-01 0.00e+00 5.12e-03 -10.9 8.49e-02  -1.3 1.00e+00 5.45e-02f  1&#xA;  52  5.1606363e-01 0.00e+00 5.11e-03 -11.0 2.13e-02  -0.9 1.00e+00 2.45e-02f  1&#xA;  53  5.1178863e-01 0.00e+00 5.01e-03 -11.0 1.04e-01  -1.3 1.00e+00 1.50e-01f  1&#xA;  54  5.1173718e-01 0.00e+00 5.01e-03 -11.0 2.33e-02  -0.9 1.00e+00 3.61e-03f  1&#xA;  55  5.0944130e-01 0.00e+00 4.96e-03 -11.0 1.40e-01  -1.4 1.00e+00 7.70e-02f  1&#xA;  56  5.0727659e-01 0.00e+00 4.89e-03 -11.0 2.55e-02  -1.0 1.00e+00 1.43e-01f  1&#xA;  57  5.0638483e-01 0.00e+00 4.85e-03 -11.0 1.11e-02  -0.5 1.00e+00 1.29e-01f  1&#xA;  58  5.0426908e-01 0.00e+00 4.78e-03 -11.0 2.79e-02  -1.0 1.00e+00 1.32e-01f  1&#xA;  59  5.0333858e-01 0.00e+00 4.74e-03 -11.0 1.22e-02  -0.6 1.00e+00 1.26e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  60  5.0161958e-01 0.00e+00 4.69e-03 -11.0 3.05e-02  -1.1 1.00e+00 1.02e-01f  1&#xA;  61  5.0037437e-01 0.00e+00 4.64e-03 -11.0 1.34e-02  -0.6 1.00e+00 1.58e-01f  1&#xA;  62  4.9971029e-01 0.00e+00 4.62e-03 -11.0 3.34e-02  -1.1 1.00e+00 3.72e-02f  1&#xA;  63  4.9914235e-01 0.00e+00 4.59e-03 -11.0 1.48e-02  -0.7 1.00e+00 6.69e-02f  1&#xA;  64  4.9664256e-01 0.00e+00 4.51e-03 -11.0 3.70e-02  -1.2 1.00e+00 1.32e-01f  1&#xA;  65  4.9621567e-01 0.00e+00 4.50e-03 -11.0 1.62e-02  -0.7 1.00e+00 4.72e-02f  1&#xA;  66  4.9453149e-01 0.00e+00 4.44e-03 -11.0 4.19e-02  -1.2 1.00e+00 8.39e-02f  1&#xA;  67  4.9304626e-01 0.00e+00 4.38e-03 -11.0 1.79e-02  -0.8 1.00e+00 1.53e-01f  1&#xA;  68  4.9163087e-01 0.00e+00 4.34e-03 -11.0 6.89e-02  -1.3 1.00e+00 6.69e-02f  1&#xA;  69  4.9031174e-01 0.00e+00 4.28e-03 -11.0 1.97e-02  -0.8 1.00e+00 1.28e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  70  4.8795240e-01 0.00e+00 4.22e-03  -8.8 2.01e-01  -1.3 9.38e-01 1.04e-01f  1&#xA;  71  4.8721723e-01 0.00e+00 4.18e-03  -9.8 2.16e-02  -0.9 1.00e+00 6.73e-02f  1&#xA;  72  4.8560412e-01 0.00e+00 4.10e-03 -11.0 9.08e-03  -0.5 1.00e+00 3.33e-01f  1&#xA;  73  4.8395604e-01 0.00e+00 4.03e-03 -11.0 2.38e-02  -0.9 1.00e+00 1.42e-01f  1&#xA;  74  4.8308417e-01 0.00e+00 3.99e-03 -11.0 9.99e-03  -0.5 1.00e+00 1.69e-01f  1&#xA;  75  4.8236398e-01 0.00e+00 3.96e-03 -10.8 2.63e-02  -1.0 1.00e+00 5.83e-02f  1&#xA;  76  4.7919513e-01 0.00e+00 3.79e-03 -10.9 1.11e-02  -0.6 1.00e+00 5.72e-01f  1&#xA;  77  4.7844811e-01 0.00e+00 3.76e-03 -11.0 3.12e-02  -1.0 1.00e+00 5.79e-02f  1&#xA;  78  4.7801338e-01 0.00e+00 3.74e-03 -11.0 1.22e-02  -0.6 1.00e+00 7.40e-02f  1&#xA;  79  4.7631089e-01 0.00e+00 3.67e-03 -11.0 4.30e-02  -1.1 1.00e+00 1.23e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  80  4.7516941e-01 0.00e+00 3.61e-03 -11.0 1.35e-02  -0.7 1.00e+00 1.81e-01f  1&#xA;  81  4.7405503e-01 0.00e+00 3.57e-03 -11.0 7.24e-02  -1.1 1.00e+00 7.54e-02f  1&#xA;  82  4.7394863e-01 0.00e+00 3.57e-03 -11.0 1.49e-02  -0.7 1.00e+00 1.57e-02f  1&#xA;  83  4.7209146e-01 0.00e+00 3.51e-03  -8.9 1.76e-01  -1.2 9.65e-01 1.15e-01f  1&#xA;  84  4.7199185e-01 0.00e+00 3.50e-03 -10.1 1.66e-02  -0.8 1.00e+00 1.36e-02f  1&#xA;  85  4.7051045e-01 0.00e+00 3.41e-03 -11.0 6.70e-03  -0.3 1.00e+00 4.74e-01f  1&#xA;  86  4.7028360e-01 0.00e+00 3.40e-03 -11.0 1.92e-02  -0.8 1.00e+00 2.87e-02f  1&#xA;  87  4.6892068e-01 0.00e+00 3.31e-03 -11.0 7.44e-03  -0.4 1.00e+00 4.02e-01f  1&#xA;  88  4.6754658e-01 0.00e+00 3.25e-03 -11.0 2.37e-02  -0.9 1.00e+00 1.62e-01f  1&#xA;  89  4.6723856e-01 0.00e+00 3.23e-03 -11.0 8.24e-03  -0.4 1.00e+00 8.47e-02f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA;  90  4.6625331e-01 0.00e+00 3.18e-03 -11.0 3.07e-02  -0.9 1.00e+00 1.08e-01f  1&#xA;  91  4.6621348e-01 0.00e+00 3.18e-03 -11.0 9.16e-03  -0.5 1.00e+00 1.01e-02f  1&#xA;  92  4.6511327e-01 0.00e+00 3.17e-03 -11.0 4.12e-02  -1.0 1.00e+00 1.11e-01f  1&#xA;  93  4.6363418e-01 0.00e+00 3.14e-03 -11.0 1.02e-02  -0.5 1.00e+00 3.45e-01f  1&#xA;  94  4.6314545e-01 0.00e+00 3.14e-03 -11.0 6.17e-02  -1.0 1.00e+00 4.60e-02f  1&#xA;  95  4.6235630e-01 0.00e+00 3.12e-03 -11.0 1.13e-02  -0.6 1.00e+00 1.70e-01f  1&#xA;  96  4.6143809e-01 0.00e+00 3.11e-03  -9.7 1.03e-01  -1.1 1.00e+00 7.93e-02f  1&#xA;  97  4.6069624e-01 0.00e+00 3.10e-03 -10.6 1.38e-02  -0.6 1.00e+00 1.47e-01f  1&#xA;  98  4.5824023e-01 0.00e+00 3.68e-03  -8.6 3.14e-01  -1.1 8.52e-01 1.81e-01f  1&#xA;  99  4.5822224e-01 0.00e+00 3.68e-03  -9.9 2.27e-02  -0.7 1.00e+00 3.30e-03f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 100  4.5747364e-01 0.00e+00 3.71e-03 -11.0 7.25e-03  -0.3 1.00e+00 3.33e-01f  1&#xA; 101  4.5651156e-01 0.00e+00 3.77e-03 -11.0 2.65e-02  -0.8 1.00e+00 1.62e-01f  1&#xA; 102  4.5624508e-01 0.00e+00 3.79e-03 -11.0 8.36e-03  -0.3 1.00e+00 1.09e-01f  1&#xA; 103  4.5460694e-01 0.00e+00 3.91e-03 -10.5 3.10e-02  -0.8 1.00e+00 2.54e-01f  1&#xA; 104  4.5415701e-01 0.00e+00 3.92e-03 -11.0 9.64e-03  -0.4 1.00e+00 1.69e-01f  1&#xA; 105  4.5379967e-01 0.00e+00 3.95e-03 -11.0 3.54e-02  -0.9 1.00e+00 5.13e-02f  1&#xA; 106  4.5282822e-01 0.00e+00 3.99e-03 -11.0 1.10e-02  -0.4 1.00e+00 3.33e-01f  1&#xA; 107  4.5236345e-01 0.00e+00 4.03e-03 -11.0 4.03e-02  -0.9 1.00e+00 6.16e-02f  1&#xA; 108  4.5158337e-01 0.00e+00 4.06e-03 -11.0 1.24e-02  -0.5 1.00e+00 2.45e-01f  1&#xA; 109  4.4983066e-01 0.00e+00 4.19e-03  -8.9 4.54e-02  -1.0 1.00e+00 2.15e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 110  4.4976166e-01 0.00e+00 4.20e-03 -10.3 1.39e-02  -0.5 1.00e+00 2.01e-02f  1&#xA; 111  4.4733643e-01 0.00e+00 4.34e-03  -8.3 4.72e-02  -1.0 9.30e-01 2.78e-01f  1&#xA; 112  4.4674741e-01 0.00e+00 4.36e-03  -9.7 1.51e-02  -0.6 1.00e+00 1.60e-01f  1&#xA; 113  4.4600496e-01 0.00e+00 4.39e-03  -8.6 4.25e-02  -1.1 1.00e+00 8.06e-02f  1&#xA; 114  4.4596146e-01 0.00e+00 4.39e-03 -10.0 1.64e-02  -0.6 1.00e+00 1.09e-02f  1&#xA; 115  4.4157951e-01 0.00e+00 4.49e-03  -8.4 4.50e-02  -1.1 1.00e+00 4.49e-01f  1&#xA; 116  4.4137682e-01 0.00e+00 4.48e-03  -9.6 1.60e-02  -0.7 1.00e+00 4.90e-02f  1&#xA; 117  4.3873388e-01 0.00e+00 4.47e-03  -7.5 3.70e-02  -1.2 9.19e-01 2.63e-01f  1&#xA; 118  4.3815153e-01 0.00e+00 4.45e-03  -8.7 1.61e-02  -0.7 1.00e+00 1.33e-01f  1&#xA; 119  4.3569743e-01 0.00e+00 4.41e-03  -7.5 3.98e-02  -1.2 1.00e+00 2.34e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 120  4.3444019e-01 0.00e+00 4.36e-03  -8.6 1.67e-02  -0.8 1.00e+00 2.71e-01f  1&#xA; 121  4.3145745e-01 0.00e+00 4.27e-03  -7.3 4.24e-02  -1.3 1.00e+00 2.74e-01f  1&#xA; 122  4.2975412e-01 0.00e+00 4.19e-03  -8.0 1.81e-02  -0.8 1.00e+00 3.55e-01f  1&#xA; 123  4.2433966e-01 0.00e+00 3.98e-03  -6.9 4.56e-02  -1.3 1.00e+00 4.92e-01f  1&#xA; 124  4.2414838e-01 0.00e+00 3.96e-03  -7.5 1.92e-02  -0.9 1.00e+00 4.01e-02f  1&#xA; 125  4.2194620e-01 0.00e+00 3.87e-03  -6.9 4.80e-02  -1.4 1.00e+00 1.96e-01f  1&#xA; 126  4.1778190e-01 0.00e+00 3.63e-03  -7.7 2.08e-02  -0.9 1.00e+00 8.34e-01f  1&#xA; 127  4.1737428e-01 0.00e+00 2.94e-03  -7.1 4.72e-02  -1.4 1.00e+00 3.66e-02f  1&#xA; 128  4.1315312e-01 0.00e+00 2.54e-03  -7.8 2.22e-02  -1.0 1.00e+00 8.46e-01f  1&#xA; 129  4.1173197e-01 0.00e+00 2.51e-03  -7.1 6.31e-02  -1.5 1.00e+00 1.27e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 130  4.0915489e-01 0.00e+00 2.46e-03  -7.7 2.32e-02  -1.0 1.00e+00 5.13e-01f  1&#xA; 131  4.0515277e-01 0.00e+00 2.38e-03  -6.9 9.97e-02  -1.5 1.00e+00 3.50e-01f  1&#xA; 132  4.0343660e-01 0.00e+00 2.34e-03  -7.3 2.38e-02  -1.1 1.00e+00 3.43e-01f  1&#xA; 133  3.9516682e-01 0.00e+00 2.17e-03  -6.8 1.68e-01  -1.6 1.00e+00 7.24e-01f  1&#xA; 134  3.9098427e-01 0.00e+00 2.06e-03  -7.2 2.34e-02  -1.1 1.00e+00 9.07e-01f  1&#xA; 135  3.9058959e-01 0.00e+00 2.05e-03  -7.0 5.50e-02  -1.6 1.00e+00 3.97e-02f  1&#xA; 136  3.8628795e-01 0.00e+00 1.93e-03  -7.7 2.39e-02  -1.2 1.00e+00 9.42e-01f  1&#xA; 137  3.8288703e-01 0.00e+00 1.84e-03  -7.2 6.76e-02  -1.7 1.00e+00 3.49e-01f  1&#xA; 138  3.8113960e-01 0.00e+00 1.79e-03  -7.8 2.37e-02  -1.2 1.00e+00 4.02e-01f  1&#xA; 139  3.7895865e-01 0.00e+00 1.74e-03  -7.3 8.98e-02  -1.7 1.00e+00 2.23e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 140  3.7630588e-01 0.00e+00 1.65e-03  -7.9 2.56e-02  -1.3 1.00e+00 6.04e-01f  1&#xA; 141  3.7040736e-01 0.00e+00 2.44e-03  -7.4 4.40e-01  -1.8 1.00e+00 5.52e-01f  1&#xA; 142  3.6744301e-01 0.00e+00 2.20e-03  -8.1 5.47e-02  -1.3 1.00e+00 6.87e-01f  1&#xA; 143  3.6668338e-01 0.00e+00 2.11e-03  -9.7 1.27e-02  -0.9 1.00e+00 4.33e-01f  1&#xA; 144  3.6563026e-01 0.00e+00 2.04e-03 -10.0 1.45e-01  -1.4 1.00e+00 2.33e-01f  1&#xA; 145  3.6473497e-01 0.00e+00 1.94e-03 -11.0 1.73e-02  -1.0 1.00e+00 4.85e-01f  1&#xA; 146  3.6423323e-01 0.00e+00 2.14e-03 -11.0 2.77e-01  -1.5 1.00e+00 1.00e-01f  1&#xA; 147  3.6397151e-01 0.00e+00 2.15e-03 -10.1 2.02e-02  -1.0 1.00e+00 1.32e-01f  1&#xA; 148  3.6196968e-01 0.00e+00 2.50e-03  -8.5 1.56e-01  -1.5 1.00e+00 3.92e-01f  1&#xA; 149  3.6176499e-01 0.00e+00 2.49e-03  -9.6 1.65e-02  -1.1 1.00e+00 1.00e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 150  3.6069639e-01 0.00e+00 2.45e-03  -8.4 3.49e-02  -1.6 1.00e+00 2.18e-01f  1&#xA; 151  3.6041434e-01 0.00e+00 2.43e-03  -9.4 1.63e-02  -1.1 1.00e+00 1.30e-01f  1&#xA; 152  3.5921303e-01 0.00e+00 2.37e-03  -8.5 3.78e-02  -1.6 1.00e+00 2.34e-01f  1&#xA; 153  3.5869485e-01 0.00e+00 2.33e-03  -9.4 1.64e-02  -1.2 1.00e+00 2.27e-01f  1&#xA; 154  3.5699552e-01 0.00e+00 2.25e-03  -7.9 4.06e-02  -1.7 1.00e+00 3.20e-01f  1&#xA; 155  3.5674149e-01 0.00e+00 2.23e-03  -8.7 1.77e-02  -1.2 1.00e+00 1.07e-01f  1&#xA; 156  3.5471902e-01 0.00e+00 2.11e-03  -7.5 4.33e-02  -1.7 1.00e+00 3.71e-01f  1&#xA; 157  3.5335745e-01 0.00e+00 1.99e-03  -8.3 1.91e-02  -1.3 1.00e+00 5.64e-01f  1&#xA; 158  3.5245420e-01 0.00e+00 1.94e-03  -7.5 4.53e-02  -1.8 1.00e+00 1.67e-01f  1&#xA; 159  3.5021073e-01 0.00e+00 1.73e-03  -8.3 2.05e-02  -1.3 1.00e+00 9.16e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 160  3.4610138e-01 0.00e+00 1.04e-03  -7.3 4.72e-02  -1.8 1.00e+00 7.94e-01f  1&#xA; 161  3.4469412e-01 0.00e+00 9.99e-04  -8.0 2.07e-02  -1.4 1.00e+00 6.40e-01f  1&#xA; 162  3.4399767e-01 0.00e+00 1.02e-03  -7.7 4.72e-02  -1.9 1.00e+00 1.42e-01f  1&#xA; 163  3.4260897e-01 0.00e+00 9.60e-04  -8.1 2.22e-02  -1.4 1.00e+00 6.21e-01f  1&#xA; 164  3.4129508e-01 0.00e+00 9.37e-04  -7.4 4.92e-02  -1.9 1.00e+00 2.68e-01f  1&#xA; 165  3.4016827e-01 0.00e+00 9.15e-04  -8.0 2.34e-02  -1.5 1.00e+00 5.07e-01f  1&#xA; 166  3.3825082e-01 0.00e+00 8.81e-04  -7.2 5.10e-02  -2.0 1.00e+00 3.98e-01f  1&#xA; 167  3.3610567e-01 0.00e+00 8.39e-04  -7.7 2.44e-02  -1.5 1.00e+00 1.00e+00f  1&#xA; 168  3.3460677e-01 0.00e+00 8.12e-04  -7.1 5.92e-02  -2.0 1.00e+00 3.31e-01f  1&#xA; 169  3.3266040e-01 0.00e+00 7.74e-04  -7.7 2.51e-02  -1.6 1.00e+00 9.50e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 170  3.3037125e-01 0.00e+00 7.34e-04  -7.1 8.30e-02  -2.1 1.00e+00 5.31e-01f  1&#xA; 171  3.2873890e-01 0.00e+00 7.02e-04  -7.7 2.53e-02  -1.6 1.00e+00 8.56e-01f  1&#xA; 172  3.2622239e-01 0.00e+00 6.57e-04  -7.2 5.31e-02  -2.1 1.00e+00 6.32e-01f  1&#xA; 173  3.2470308e-01 0.00e+00 6.28e-04  -7.8 2.53e-02  -1.7 1.00e+00 8.69e-01f  1&#xA; 174  3.2372187e-01 0.00e+00 6.27e-04  -7.5 5.26e-02  -2.2 1.00e+00 2.63e-01f  1&#xA; 175  3.2283458e-01 0.00e+00 5.92e-04  -8.2 2.60e-02  -1.7 1.00e+00 5.20e-01f  1&#xA; 176  3.2183252e-01 0.00e+00 6.17e-04  -7.5 5.47e-02  -2.2 1.00e+00 2.71e-01f  1&#xA; 177  3.2046827e-01 0.00e+00 5.47e-04  -8.1 2.70e-02  -1.8 1.00e+00 8.08e-01f  1&#xA; 178  3.1975003e-01 0.00e+00 6.03e-04  -7.1 5.85e-02  -2.3 1.00e+00 2.04e-01f  1&#xA; 179  3.1900035e-01 0.00e+00 5.19e-04  -7.6 2.77e-02  -1.8 1.00e+00 4.53e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 180  3.1615561e-01 0.00e+00 4.90e-04  -6.8 6.32e-02  -2.3 1.00e+00 8.53e-01f  1&#xA; 181  3.1524761e-01 0.00e+00 4.49e-04  -7.3 2.68e-02  -1.9 1.00e+00 5.85e-01f  1&#xA; 182  3.1326122e-01 0.00e+00 4.87e-04  -7.0 6.24e-02  -2.4 9.98e-01 6.26e-01f  1&#xA; 183  3.1296679e-01 0.00e+00 4.09e-04  -7.7 2.63e-02  -1.9 1.00e+00 2.01e-01f  1&#xA; 184  3.1055716e-01 0.00e+00 4.34e-04  -6.8 6.57e-02  -2.4 1.00e+00 8.22e-01f  1&#xA; 185  3.1020674e-01 0.00e+00 3.60e-04  -7.5 2.70e-02  -2.0 1.00e+00 2.54e-01f  1&#xA; 186  3.0929569e-01 0.00e+00 4.68e-04  -7.0 6.86e-02  -2.5 1.00e+00 3.18e-01f  1&#xA; 187  3.0819030e-01 0.00e+00 3.23e-04  -7.8 2.94e-02  -2.0 1.00e+00 8.02e-01f  1&#xA; 188  3.0763044e-01 0.00e+00 4.47e-04  -7.2 8.21e-02  -2.5 1.00e+00 2.02e-01f  1&#xA; 189  3.0725472e-01 0.00e+00 3.07e-04  -8.0 3.11e-02  -2.1 1.00e+00 2.85e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 190  3.0578606e-01 0.00e+00 4.05e-04  -7.5 2.33e-01  -2.6 1.00e+00 5.11e-01f  1&#xA; 191  3.0495222e-01 0.00e+00 3.43e-04  -8.1 2.91e-02  -2.2 1.00e+00 6.59e-01f  1&#xA; 192  3.0413633e-01 0.00e+00 1.17e-03  -7.6 4.93e-01  -2.6 1.00e+00 2.84e-01f  1&#xA; 193  3.0331078e-01 0.00e+00 9.60e-04  -7.7 3.14e-02  -2.2 1.00e+00 6.81e-01f  1&#xA; 194  3.0239991e-01 0.00e+00 8.65e-04  -6.7 7.67e-02  -2.7 9.99e-01 4.00e-01f  1&#xA; 195  3.0137954e-01 0.00e+00 6.77e-04  -7.5 3.36e-02  -2.3 1.00e+00 8.52e-01f  1&#xA; 196  3.0056119e-01 0.00e+00 6.24e-04  -7.5 8.05e-02  -2.7 1.00e+00 3.28e-01f  1&#xA; 197  3.0014178e-01 0.00e+00 5.65e-04  -8.3 3.58e-02  -2.3 1.00e+00 3.62e-01f  1&#xA; 198  2.9923633e-01 0.00e+00 5.17e-04  -8.2 8.53e-02  -2.8 1.00e+00 3.59e-01f  1&#xA; 199  2.9888005e-01 0.00e+00 4.75e-04  -8.9 3.83e-02  -2.4 1.00e+00 3.15e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 200  2.9848396e-01 0.00e+00 4.57e-04  -8.1 8.98e-02  -2.8 1.00e+00 1.58e-01f  1&#xA; 201  2.9821853e-01 0.00e+00 4.30e-04  -8.8 4.15e-02  -2.4 1.00e+00 2.29e-01f  1&#xA; 202  2.9738245e-01 0.00e+00 3.96e-04  -7.2 9.62e-02  -2.9 9.99e-01 3.39e-01f  1&#xA; 203  2.9665614e-01 0.00e+00 3.36e-04  -7.8 4.41e-02  -2.5 1.00e+00 6.30e-01f  1&#xA; 204  2.9578199e-01 0.00e+00 3.07e-04  -6.8 1.02e-01  -2.9 1.00e+00 3.87e-01f  1&#xA; 205  2.9465429e-01 0.00e+00 2.39e-04  -7.4 4.54e-02  -2.5 1.00e+00 1.00e+00f  1&#xA; 206  2.9301415e-01 0.00e+00 1.99e-04  -6.7 1.07e-01  -3.0 1.00e+00 8.01e-01f  1&#xA; 207  2.9292261e-01 0.00e+00 1.94e-04  -7.3 4.47e-02  -2.6 1.00e+00 8.17e-02f  1&#xA; 208  2.9132376e-01 0.00e+00 2.05e-04  -6.8 1.11e-01  -3.0 1.00e+00 7.08e-01f  1&#xA; 209  2.9068030e-01 0.00e+00 1.46e-04  -7.4 4.72e-02  -2.6 1.00e+00 5.83e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 210  2.8989341e-01 0.00e+00 2.15e-04  -6.8 1.19e-01  -3.1 1.00e+00 3.71e-01f  1&#xA; 211  2.8905846e-01 0.00e+00 1.25e-04  -7.4 5.00e-02  -2.7 1.00e+00 7.99e-01f  1&#xA; 212  2.8817010e-01 0.00e+00 2.35e-04  -6.9 1.32e-01  -3.1 1.00e+00 4.36e-01f  1&#xA; 213  2.8741077e-01 0.00e+00 1.25e-04  -7.5 5.21e-02  -2.7 1.00e+00 7.41e-01f  1&#xA; 214  2.8705020e-01 0.00e+00 2.96e-04  -7.2 1.57e-01  -3.2 1.00e+00 1.65e-01f  1&#xA; 215  2.8639044e-01 0.00e+00 1.29e-04  -8.0 5.54e-02  -2.8 1.00e+00 6.44e-01f  1&#xA; 216  2.8567025e-01 0.00e+00 2.13e-04  -7.4 4.30e-01  -3.2 1.00e+00 3.16e-01f  1&#xA; 217  2.8519078e-01 0.00e+00 1.22e-04  -8.1 6.05e-02  -2.8 1.00e+00 4.79e-01f  1&#xA; 218  2.8508336e-01 0.00e+00 1.93e-04  -7.5 3.80e-01  -3.3 1.00e+00 4.58e-02f  1&#xA; 219  2.8459190e-01 0.00e+00 1.16e-04  -8.3 6.70e-02  -2.9 1.00e+00 4.75e-01f  1&#xA;iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls&#xA; 220  2.8382163e-01 0.00e+00 1.63e-04  -7.2 3.30e-01  -3.3 1.00e+00 3.29e-01f  1&#xA; 221  2.8341130e-01 0.00e+00 1.28e-04  -7.8 7.27e-02  -2.9 1.00e+00 3.93e-01f  1&#xA; 222  2.8284739e-01 0.00e+00 1.56e-04  -7.0 2.21e-01  -3.4 1.00e+00 2.45e-01f  1&#xA; 223  2.8183563e-01 0.00e+00 1.19e-04  -7.7 7.96e-02  -3.0 1.00e+00 9.45e-01f  1&#xA; 224  2.8117611e-01 0.00e+00 1.40e-04  -6.9 4.77e-01  -3.4 1.00e+00 2.96e-01f  1&#xA; 225  2.8033847e-01 0.00e+00 1.04e-04  -7.6 8.59e-02  -3.0 1.00e+00 7.96e-01f  1&#xA; 226  2.8005033e-01 0.00e+00 9.73e-05  -9.1 3.26e-02  -2.6 1.00e+00 6.19e-01f  1&#xA;&#xA;Number of Iterations....: 226&#xA;&#xA;                                   (scaled)                 (unscaled)&#xA;Objective...............:   2.8005032523583379e-01    2.8005032523583379e-01&#xA;Dual infeasibility......:   9.7272674831558608e-05    9.7272674831558608e-05&#xA;Constraint violation....:   0.0000000000000000e+00    0.0000000000000000e+00&#xA;Complementarity.........:   3.2315149852880829e-07    3.2315149852880829e-07&#xA;Overall NLP error.......:   9.7272674831558608e-05    9.7272674831558608e-05&#xA;&#xA;&#xA;Number of objective function evaluations             = 227&#xA;Number of objective gradient evaluations             = 227&#xA;Number of equality constraint evaluations            = 0&#xA;Number of inequality constraint evaluations          = 228&#xA;Number of equality constraint Jacobian evaluations   = 0&#xA;Number of inequality constraint Jacobian evaluations = 1&#xA;Number of Lagrangian Hessian evaluations             = 226&#xA;Total CPU secs in IPOPT (w/o function evaluations)   =   3403.326&#xA;Total CPU secs in NLP function evaluations           =    214.977&#xA;&#xA;EXIT: Optimal Solution Found.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;.... the timing is ok at this resolution, but at higher ones, the linear solves become quite large and extremely slow. So I guess my question is: can I expect something like this (at a higher resolution, with ~500 000 parameters) to take a couple of days to solve or are there any settings I could tweak to increase convergence / reduce the number if iterations needed. The initial guesses are quite good actually btw, and a local optimum is actually all I'm looking for.&lt;/p&gt;&#xA;" OwnerUserId="28178" LastActivityDate="2018-07-11T06:35:11.233" Title="Convergence of a very large non-linear least squares optimization" Tags="|nonlinear-equations|constrained-optimization|nonlinear-programming|least-squares|interior-point-method|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29841" PostTypeId="1" CreationDate="2018-07-11T07:25:44.860" Score="3" ViewCount="287" Body="&lt;p&gt;What could be the reason that the solution of a Poisson equation is smooth when obtained by an iterative solver, only if the maximum residual is set to a high value (e.g. 0.1)? When the maximum residual is set as a typical lower value (e.g. 1e-6), then the solution oscillates - often (but not exclusively) near Neumann boundary conditions. The solution with the high residual is underpredicted, and with the low residual it oscillates around expected values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix is non-symmetric sparse, weakly diagonally-dominant (a diagonal entry equals the negative sum of other row entries). The system includes both Dirichlet and Neumann boundary conditions. Similar outcomes happen using various iterative solvers: BiCGStab, CGS, QMR and GMRES. The preconditioner is always a diagonal preconditioner, since I have to use matrix-free solving approach on the GPU. Using no preconditioner yields similar (though a bit smoother) results.&lt;/p&gt;&#xA;" OwnerUserId="28179" LastEditorUserId="28098" LastEditDate="2018-07-12T22:08:44.307" LastActivityDate="2018-07-12T22:08:44.307" Title="Unstable convergence of a Poisson equation" Tags="|linear-algebra|iterative-method|convergence|poisson|elliptic-pde|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29843" PostTypeId="2" ParentId="29729" CreationDate="2018-07-12T16:34:07.127" Score="2" Body="&lt;p&gt;I guess what you are referring to is the cross-correlation between the images. This is very good and efficient to find shifts, but not changes in scale and orientation. As you probably know, the cross-correlation simultaneously computes the inner product of one image with all possible shifts of the other, in time $O(n\log n)$, where $n$ is the number of pixels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an article by Morgan McGuire &lt;a href=&quot;http://cs.brown.edu/~morgan/papers/(McGuire%2098)%20Registration.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;An image registration technique for recovering rotation, scale and translation parameters&lt;/a&gt; that I've been meaning to study for some time but haven't gotten around yet. This does something similar, but also manages to capture changes in scale and rotations, in a time that essentially also is $O(n\log n)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very different class of algorithms are &lt;em&gt;feature matching&lt;/em&gt; algorithms, which analyze an image and identify special points in a way that is invariant under many different kinds of changes, like general projective transformations (including rotation, translation, scale), changes in lightness, etc. Some examples in OpenCV can be found &lt;a href=&quot;https://docs.opencv.org/3.4/dc/dc3/tutorial_py_matcher.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. One of the most famous (though not the most modern) is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Scale-invariant_feature_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFT&lt;/a&gt;, introduced by D. Lowe in &lt;em&gt;Distinctive Image Features from Scale-Invariant Keypoints&lt;/em&gt;. This is likely much slower than the previous method.&lt;/p&gt;&#xA;" OwnerUserId="5521" LastActivityDate="2018-07-12T16:34:07.127" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29844" PostTypeId="2" ParentId="29832" CreationDate="2018-07-12T21:13:46.743" Score="0" Body="&lt;p&gt;First of all, there is a big &lt;em&gt;definition question&lt;/em&gt; about what is an &lt;em&gt;average partitioning scheme&lt;/em&gt;. Nevertheless,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess, there are two possible intentions for this question:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Improve the quality of the partitioning that the &lt;em&gt;kmeans++&lt;/em&gt; provides by using $N$ runs with different initial and intermediate seeds.&lt;/li&gt;&#xA;&lt;li&gt;Have reproducible (at least on one machine) result.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If the latter is the concern, fixing the seed of the random number generator that is being used by your (or library) implementation should do the trick. You'll get the same result every run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a real &lt;em&gt;kmeans&lt;/em&gt; problem, it will be hard to know if you ended up with an optimal solution (as you are still finding local minima as opposed to global ones; however, &lt;em&gt;kmeans++&lt;/em&gt; improves the approximation to $\mathcal O (\log k)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, finding the average of the obtained results for multiple runs means that you are trying to find an average of local minimums, which I don't think can contribute in any way to improving the quality of the partitioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this question,&lt;a href=&quot;https://stats.stackexchange.com/a/186761/205311&quot;&gt; they discuss how we can &lt;strong&gt;compare&lt;/strong&gt; the partitionings obtained from several &lt;em&gt;kmeans&lt;/em&gt; runs&lt;/a&gt;. But this is in no way an average. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, it is very common to launch such heuristics with different starting criteria to obtain &lt;em&gt;a better result&lt;/em&gt;, I do not see why one might want to calculate &lt;em&gt;the average&lt;/em&gt;, no matter what that means. However, one might want to see how the obtained solutions differ from each other with each run and what kind of error do they provide.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-07-12T21:13:46.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29846" PostTypeId="2" ParentId="29830" CreationDate="2018-07-12T23:06:11.080" Score="3" Body="&lt;p&gt;I saw @BenCrowell comment and wanted to check how hard it was.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you write the variational formulation of the Schrödinger equation you would get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Pi[\psi] = \frac{1}{2} \langle \nabla \psi, \nabla\psi\rangle +&#xA;                 \langle \psi, V(x) \psi\rangle -&#xA;                  E\langle \psi, \psi\rangle \, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and, choosing the finite expansion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi = \sum_{n=0}^N c_n \sin(n\pi x)\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we obtain the following eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$[H]\lbrace c\rbrace = \frac{E^N}{2}\lbrace c\rbrace\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$H_{mn} = \frac{\pi^2 m^2}{4}\delta_{mn} + U_{mn}\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$U_{mn} = \frac{\sqrt{2 \pi}}{8} \sum_{i=1}^3 a_i c_i f_i(m, n, b_i, c_i)\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_i(m, n, b, c) = {\rm erf} \left(1/2\,{\frac { \left( i \left( m-n \right) \pi\,{c}^{2}&#xA;-b+1 \right) \sqrt {2}}{c}}\right){{\rm e}^{- \left( 1/2\, \left( m-n&#xA; \right) \pi\,{c}^{2}+ib \right)  \left( m-n \right) \pi}}-{\rm erf} &#xA;\left(1/2\,{\frac { \left( i \left( m-n \right) \pi\,{c}^{2}-b&#xA; \right) \sqrt {2}}{c}}\right){{\rm e}^{- \left( 1/2\, \left( m-n&#xA; \right) \pi\,{c}^{2}+ib \right)  \left( m-n \right) \pi}}+ \left( -&#xA;{\rm erf} \left(1/2\,{\frac { \left( i \left( m-n \right) \pi\,{c}^{2}&#xA;+b-1 \right) \sqrt {2}}{c}}\right)+{\rm erf} \left(1/2\,{\frac {\sqrt &#xA;{2} \left( i \left( m-n \right) \pi\,{c}^{2}+b \right) }{c}}\right)&#xA; \right) {{\rm e}^{ \left( -1/2\, \left( m-n \right) \pi\,{c}^{2}+ib&#xA; \right)  \left( m-n \right) \pi}}-{\rm erf} \left(1/2\,{\frac {\sqrt &#xA;{2} \left( i \left( m+n \right) \pi\,{c}^{2}-b+1 \right) }{c}}\right){&#xA;{\rm e}^{- \left( 1/2\, \left( m+n \right) \pi\,{c}^{2}+ib \right) &#xA; \left( m+n \right) \pi}}+{\rm erf} \left(1/2\,{\frac { \left( i&#xA; \left( m+n \right) \pi\,{c}^{2}-b \right) \sqrt {2}}{c}}\right){&#xA;{\rm e}^{- \left( 1/2\, \left( m+n \right) \pi\,{c}^{2}+ib \right) &#xA; \left( m+n \right) \pi}}-{{\rm e}^{ \left( m+n \right)  \left( -1/2\,&#xA; \left( m+n \right) \pi\,{c}^{2}+ib \right) \pi}} \left( {\rm erf} &#xA;\left(1/2\,{\frac { \left( i \left( m+n \right) \pi\,{c}^{2}+b&#xA; \right) \sqrt {2}}{c}}\right)-{\rm erf} \left(1/2\,{\frac { \left( i&#xA; \left( m+n \right) \pi\,{c}^{2}+b-1 \right) \sqrt {2}}{c}}\right)&#xA; \right)\, .$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following snippet solve the equation using this approach&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from __future__ import division&#xA;import numpy as np&#xA;from scipy.linalg import eigh&#xA;from scipy.special import erf&#xA;from numpy import sqrt, pi, exp&#xA;&#xA;&#xA;def Uterm(m, n, a, b, c):&#xA;    aux = (erf((1j * (m - n) * pi * c ** 2 - b + 1) * sqrt(2) / c / 2) *&#xA;           exp(-(m - n) * pi * ((m - n) * pi * c ** 2 / 2 + 1j * b)) -&#xA;           erf((1j * (m - n) * pi * c ** 2 - b) * sqrt(2) / c / 2) *&#xA;           exp(-(m - n) * pi * ((m - n) * pi * c ** 2 / 2 + 1j * b)) +&#xA;           (-erf((1j * (m - n) * pi * c ** 2 + b - 1) * sqrt(2) / c / 2) +&#xA;            erf((1j * (m - n) * pi * c ** 2 + b) * sqrt(2) / c / 2)) *&#xA;            exp((m - n) * pi * (-(m - n) * pi * c ** 2 / 2 + 1j * b)) -&#xA;            erf((1j * (m + n) * pi * c ** 2 - b + 1) * sqrt(2) / c / 2) *&#xA;            exp(-(m + n) * ((m + n) * pi * c ** 2 / 2 + 1j * b) * pi) +&#xA;            erf((1j * (m + n) * pi * c ** 2 - b) * sqrt(2) / c / 2) *&#xA;            exp(-(m + n) * ((m + n) * pi * c ** 2 / 2 + 1j * b) * pi) -&#xA;            exp((-(m + n) * pi * c ** 2 / 2 + 1j * b) * (m + n) * pi) *&#xA;            (erf((1j * (m + n) * pi * c ** 2 + b) * sqrt(2) / c / 2) -&#xA;             erf((1j * (m + n) * pi * c ** 2 + b - 1) * sqrt(2) / c / 2)))&#xA;    return np.real(sqrt(2*pi)*a*c/4 * aux)&#xA;&#xA;&#xA;N = 11&#xA;a = [1, 2, 3]&#xA;b = [0.3, 0.6, 0.8]&#xA;c = 0.001&#xA;T = 0.5*pi**2*np.diag(range(1, N + 1))**2&#xA;U = np.zeros_like(T)&#xA;for m in range(1, N + 1):&#xA;    for n in range(1, N + 1):&#xA;        U[m - 1, n - 1] = Uterm(m, n, a[0], b[0], c) +\&#xA;                             Uterm(m, n, a[1], b[1], c) +\&#xA;                             Uterm(m, n, a[2], b[2], c)&#xA;H = T + U&#xA;&#xA;vals, vecs = eigh(H)&#xA;print(np.round(np.real(vals[:11]), 6))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For $a=[1, 2, 3]$, $b=[0.3, 0.6, 08]$, $c=0.001$, the following are the results&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[  4.952339  19.760811  44.43076   78.972835 123.375073 177.668876  241.822857 315.848936 399.736522 493.48025  597.12861 ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That are really close to the ones computed using finite differences as proposed &lt;a href=&quot;https://scicomp.stackexchange.com/a/19679/9667&quot;&gt;here&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[  4.932655  19.681995  44.253232  78.656791 122.880544 176.955286   240.848923 314.57281  398.115649 491.471978 594.688728]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-08-14T20:12:36.960" LastActivityDate="2018-08-14T20:12:36.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29847" PostTypeId="1" CreationDate="2018-07-13T03:00:19.573" Score="2" ViewCount="67" Body="&lt;p&gt;Originally asked this on &lt;a href=&quot;https://math.stackexchange.com/q/2848196/332683&quot;&gt;math.stackexchange&lt;/a&gt;, but I figure it's also appropriate here. I'm reading through some finite difference code for a diffusion equation and came across something odd for the boundary conditions, and I was wondering about the validity of the method. I'm familiar with the finite difference method, but I've never seen this certain method before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation I am solving is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial f}{\partial t}=C(x)\frac{\partial^2 f}{\partial x^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary condition &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial f}{\partial x}=0\ \text{on}\ x=0,1 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;across time, $t$, and position, $x$, with position-dependent diffusion constant, $C(x)$. It's being solved in an explicit way using the forward difference for time and the second-order central difference for space&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{f^\text{new}_i-f^\text{old}_i}{\Delta t}=C_i\frac{f^\text{old}_{i-1}-2f^\text{old}_{i}+f^\text{old}_{i+1}}{(\Delta x)^2}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;along discretised space $\{x_1,\dots,x_n\}$. Rearranging the above equation gives &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;f^\text{new}_i=&amp;amp;f^\text{old}_i + \frac{C_i\Delta t}{(\Delta x)^2}\Big(f^\text{old}_{i-1}-2f^\text{old}_{i}+f^\text{old}_{i+1}\Big)\\&#xA;=&amp;amp;f^\text{old}_{i-1}(S_i)+f^\text{old}_{i}(1-2S_i)+f^\text{old}_{i+1}(S_i)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $S_i=\frac{C_i\Delta t}{(\Delta x)^2}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is simply represented in the code. For the boundary at $x_1$ (and at $x_n$, but I'll only talk about the first as they are similar), there are some strange things going on. Usually I would do a Taylor series expansion of function $f$ at points $x_2$ and $x_3$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ f_2=f_1+\frac{\partial f}{\partial x}\bigg\rvert_i \Delta x + \frac{1}{2} \frac{\partial^2 f}{\partial x^2}\bigg\rvert_i (\Delta x)^2 + \mathcal{O}((\Delta x)^3)$$&#xA;$$ f_3=f_1+\frac{\partial f}{\partial x}\bigg\rvert_i (2\Delta x) + \frac{1}{2} \frac{\partial^2 f}{\partial x^2}\bigg\rvert_i (2\Delta x)^2 + \mathcal{O}((\Delta x)^3)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which, due to boundary condition $\frac{\partial f}{\partial x}=0$ can be rearranged to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial^2 f}{\partial x^2}\bigg\rvert_i\approx\frac{2}{(\Delta x)^2}\Big(f_2 - f_1\Big)\tag{$A$}$$&#xA;$$ \frac{\partial^2 f}{\partial x^2}\bigg\rvert_i\approx\frac{1}{2(\Delta x)^2}\Big(f_3 - f_1\Big)\tag{$B$}.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, I'd average $A$ and $B$ to determine how the boundary conditions are applied in the eventual matrix solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{\partial^2 f}{\partial x^2}\bigg\rvert_i\approx\frac{1}{(\Delta x)^2}\Big(-\frac{5}{4}f_1 + f_2 + \frac{1}{4}f_3 \Big).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the code that I'm reading does something different and takes the boundary as a weighted sum of $A/5+4B/5$, as shown below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{A+4B}{5} \Rightarrow \frac{\partial^2 f}{\partial x^2}\bigg\rvert_i\approx\frac{1}{(\Delta x)^2}\Big(-\frac{4}{5}f_1 + \frac{2}{5}f_2 + \frac{2}{5}f_3 \Big).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think this as being inherently wrong, as the LHS is still $\frac{\partial^2 f}{\partial x^2}\big\rvert_i$, but I'm not sure. Is this a valid step to make? Is it ok to &quot;weight&quot; results from different nodes, and if so, in what situations would weighting or not weighting be useful? &lt;/p&gt;&#xA;" OwnerUserId="28191" LastEditorUserId="20688" LastEditDate="2018-07-15T15:34:59.923" LastActivityDate="2018-07-15T15:34:59.923" Title="Finite difference Neumann boundary conditions: uneven weighting of edge nodes?" Tags="|finite-difference|boundary-conditions|advection-diffusion|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29849" PostTypeId="2" ParentId="7954" CreationDate="2018-07-13T07:02:07.357" Score="1" Body="&lt;p&gt;if Nelder-Mead is consistent but the gradient-based methods aren't, it might be that the gradient is badly scaled. Eg. a unit change in parameter n affects the objective function very differently than a unit change in parameter m. Try scaling the parameters (what's f btw?) to (approximately) the same range, or if you can even better so that if you change any single parameter by a given amount the objective function will change similarly. (eg. &lt;a href=&quot;http://www.alglib.net/optimization/scaling.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.alglib.net/optimization/scaling.php&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="28178" LastEditorUserId="28178" LastEditDate="2018-07-13T07:13:59.480" LastActivityDate="2018-07-13T07:13:59.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29850" PostTypeId="1" AcceptedAnswerId="29856" CreationDate="2018-07-13T08:24:53.943" Score="2" ViewCount="67" Body="&lt;p&gt;Assume that in the a (residual type) posteriori error estimator of some PDE is a term of the form $h_T\|g\|_{L^2(\Omega)}$ involved where $h_T$ is the diameter of an element and $g$ is some &lt;em&gt;known&lt;/em&gt; data function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While proving the efficiency of the estimator, one wants to obtain&#xA;\begin{align}&#xA;h_T^2\|g\|_{L^2(T)}^2 \leq \|u-u_h\|^2 + h_T^2\|g-\bar{g}_T\|^2_{L^2(T)}&#xA;\end{align}&#xA;where $\bar{g}_T$ is the average of $g$ in $T$. Why do we need this oscillation term, since $g$ is known data and it is controlled by $h_T$? &lt;/p&gt;&#xA;" OwnerUserId="25965" LastActivityDate="2018-07-13T19:27:03.470" Title="Oscillation term in a posteriori error estimator" Tags="|finite-element|pde|adaptive-mesh-refinement|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29852" PostTypeId="1" AcceptedAnswerId="29854" CreationDate="2018-07-13T13:32:26.150" Score="3" ViewCount="256" Body="&lt;p&gt;A hypergraph $H = (V,E)$ consists of a finite set of vertices, say $V=\{1, \dots, n\}$ and a set of hyperedges $E \subseteq \mathcal{P}(V)$. We call $H$ a $k$-hypergraph if all $|e| = k$ for all $e\in E$. A vertex cover for $H$ is a set $V' \subseteq V$ such that for all $e \in E$ there is some $v \in V'$ such that $v \in e$. The problem I wish to solve computationally is finding a vertex cover of minimal size (a minimum vertex cover) for certain $k$-hypergraphs with $k&amp;gt;2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any software (e.g a Python package) that can solve this problem given a $k$-hypergraph $H$? For normal graphs (2-hypergraphs), &lt;em&gt;Mathematica&lt;/em&gt; has the function &lt;strong&gt;FindVertexCover&lt;/strong&gt;, but it does not seem to have support for hypergraphs with higher cardinality edges. I could write my own program to solve this, but, since the problem of finding a minimum vertex cover is hard, I would prefer it if anyone could find something where most of the currently known optimization is already implemented. &lt;/p&gt;&#xA;" OwnerUserId="28197" LastActivityDate="2018-07-13T16:00:27.090" Title="Software for finding a minimum vertex cover for a hypergraph" Tags="|optimization|graph-theory|combinatorics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29853" PostTypeId="1" CreationDate="2018-07-13T15:09:48.990" Score="3" ViewCount="103" Body="&lt;p&gt;I am trying to fit a time-dependent curve at each time step. I do so in minimizing along $x_c$ the quadratic error between the curve and a reference solution $ 1/(1 + \exp\left(\sqrt{S}(x-x_c)\right) $:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \frac{1}{N+2} \sum_{i=0}^{N+1} \left(y_i - \frac{1}{1 + \exp\left(\sqrt{S}(x_i-x_c)\right)}\right)^2 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the calculations are easy, I calculated the gradient so as to use a conjugate gradient method implemented in &lt;code&gt;optimize.minimize&lt;/code&gt; in &lt;code&gt;scipy&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So: fairly easy stuff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is my MWE and &lt;a href=&quot;https://we.tl/nXChsPcmFl&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is my data file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from matplotlib import pyplot as plt&#xA;from scipy import optimize&#xA;&#xA;def energie(centre, x, y, S):&#xA;    erreur = 0.&#xA;    for i in range(0, y.size):&#xA;        erreur += (y[i] - 1./(1.+np.exp(np.sqrt(S)*(x[i]-centre))))**2.&#xA;    return erreur/y.size&#xA;&#xA;def gradenergie(centre, x, y, S):&#xA;    grad = 0.&#xA;    for i in range(0, y.size):&#xA;        grad += - (y[i] - 1./(1.+np.exp(np.sqrt(S)*(x[i]-centre)))) * np.sqrt(S) * \&#xA;        np.exp(np.sqrt(S)*(x[i]-centre)) * (1.+np.exp(np.sqrt(S)*(x[i]-centre)))**-2.&#xA;    return 2.*grad/y.size&#xA;&#xA;T = np.arange(0., 36., 0.5/np.sqrt(2))&#xA;x = np.arange(301) * 0.5&#xA;q = np.load('data/data.npy')&#xA;centres = np.zeros(q.shape[0])&#xA;&#xA;for n in range(q.shape[0]):&#xA;    # initial guess&#xA;    centreq = centres[n-1]&#xA;    # call to optimize.minimize&#xA;    resultatq = optimize.minimize(energie, centreq, method = 'CG', jac = gradenergie, args = (x, q[n , : ], 0.1))&#xA;    # write to centres&#xA;    centres[n] = resultatq.x&#xA;&#xA;dcentresqdt = np.gradient(centres, T[1]-T[0])&#xA;plt.plot(dcentresqdt[2 : ])&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which produces the following graph:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mc4jI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mc4jI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;with obvious irregularities. Any idea why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in the code &lt;code&gt;centre&lt;/code&gt; or &lt;code&gt;centreq&lt;/code&gt; stands for $x_c$, each &lt;code&gt;q[n, :]&lt;/code&gt; stands for the $(y_i)$, and the $(x_i)$ are the same at each time step, they are in the array &lt;code&gt;x&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed I plot the derivative of &lt;code&gt;centres&lt;/code&gt; with respect to time, but if I plot &lt;code&gt;centres&lt;/code&gt; itself I have the same kind of problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realized my problem is so standard that there is actually a &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html#scipy.optimize.curve_fit&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy function&lt;/a&gt; that does the job in no time (101 seconds for Nelder Mead vs 0.39 for the version below):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.optimize import curve_fit&#xA;&#xA;def kink(x, centre):&#xA;    S = 0.1&#xA;    return 1./(1.+np.exp(np.sqrt(S)*(x-centre)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and later on in the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for n in range(q.shape[0]):&#xA;    centreq = centres[n-1]&#xA;    popt, pcov = curve_fit(kink, x, q[n , : ], p0 = centreq)&#xA;    centres[n] = popt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7784" LastEditorUserId="7784" LastEditDate="2018-07-15T12:11:47.633" LastActivityDate="2018-07-15T12:11:47.633" Title="Inconsistency in optimize.minimize" Tags="|optimization|python|scipy|numpy|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29854" PostTypeId="2" ParentId="29852" CreationDate="2018-07-13T15:54:07.370" Score="3" Body="&lt;p&gt;I usually use &lt;a href=&quot;http://www.sagemath.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SageMath&lt;/a&gt; for research work connected &lt;a href=&quot;http://doc.sagemath.org/html/en/reference/graphs/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;with graphs&lt;/a&gt;. However, I was not able to find there a ready-made algorithm to find a minimum vertex cover for &lt;a href=&quot;http://doc.sagemath.org/html/en/reference/graphs/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a hypergraph (see subsection with the corresponding name)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, using Sage together with Python should simplify your life a lot as it will give access to a lot of convenient and tested data structures and algorithms for graph theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, regarding the vertex cover. I stumbled upon the following publicly available Ph.D. thesis:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://sfx.cvt.dk/sfx_local?sid=pure.atira.dk:pure&amp;amp;pid=19933136-3253-455c-be54-4ea47da33945&amp;amp;atitle=Algebraic%20Varieties%20and%20System%20Design&amp;amp;aulast=Aabrandt&amp;amp;aufirst=Andreas&amp;amp;auinit=A&amp;amp;date=2016&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Aabrandt, &quot;Algebraic varieties and system design,&quot; Ph.D. Thesis, Dept. Electr. Comput. Eng., Tech. Univ. Denmark, 2016.&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There, in Appendix D, the author has the code (Python+Sage generated by C++) for vertex cover calculation, that should handle hypergraphs (as it is formulated as a cover ideal, see Section 6.1 &quot;Edge and Cover Ideals for Hypergraphs&quot;). I haven't used that code myself, but this is the best I have the knowledge about so far.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-07-13T16:00:27.090" LastActivityDate="2018-07-13T16:00:27.090" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29855" PostTypeId="1" CreationDate="2018-07-13T18:41:58.207" Score="2" ViewCount="72" Body="&lt;p&gt;According to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Set_cover_problem#Inapproximability_results&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia page for Set Cover&lt;/a&gt;, the greedy algorithm for weighted set cover achieves the polynomial-time approximation bound. There are other techniques for solving Set Cover, &lt;a href=&quot;https://en.wikipedia.org/wiki/Randomized_rounding#Lemma_(approximation_guarantee_for_rounding_scheme)&quot; rel=&quot;nofollow noreferrer&quot;&gt;such as randomized rounding of a linear programming relaxation&lt;/a&gt;, which I believe achieve the same asymptotic bound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there information comparing these asymptotically-equivalently-performing algorithms in practice?&lt;/p&gt;&#xA;" OwnerUserId="1217" LastActivityDate="2018-09-28T14:39:34.597" Title="Weighted Set Cover in practice, beyond the greedy algorithm" Tags="|linear-programming|approximation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29856" PostTypeId="2" ParentId="29850" CreationDate="2018-07-13T19:27:03.470" Score="2" Body="&lt;p&gt;In the efficiency proof you use bubble functions in order to get rid of the elementwise boundary terms. For a bubble function $b_T$ defined on an element $T$ it holds&#xA;$$c \|v_h\|_{0,T} \leq \|b_T v_h\|_{0,T} \leq C \| v_h \|_{0,T}$$&#xA;which is true for any discrete function $v_h$. In the proof you use this (or similar) inequality to bound the residual of the problem. This is not possible for an arbitrary (non-discrete) loading. Therefore in the residual, you replace $g$ by its projection through $g = \overline{g} + g - \overline{g}$ and use the triangle inequality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that for an arbitrary loading, it is not possible to prove the efficiency without any oscillation terms.&lt;/p&gt;&#xA;" OwnerUserId="10160" LastActivityDate="2018-07-13T19:27:03.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29857" PostTypeId="1" AcceptedAnswerId="29863" CreationDate="2018-07-13T20:37:44.013" Score="2" ViewCount="683" Body="&lt;p&gt;I'm trying to implement the following form of the cell/neighbor list method in my MD code. I have divided my simulation box into a fixed number of cells, and according to its positions, I have correctly assigned each atom to the appropriate cell. I build the neighbor list for each individual cell $i$. When performing calculations, I only consider the atoms within the neighboring cells; furthermore, on this subset, I perform the Verlet search where I only consider the atoms within radius $r_{cutoff}$ of atom $k$. This even smaller subset of atoms will be interacting. The algorithm I want to implement is found in this &lt;a href=&quot;http://www.ee.bgu.ac.il/~specmeth/EMT04/pdf/session_2/2_14_04.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, algorithm C 1. Algorithm 1 is the pseudocode for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jTx1s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jTx1s.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are regarding the building of the neighbors list. Pictured above is the simulation box represented in 3 layers (picture them on top of each other to form a 3D box). &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I know that we should not consider all the neighboring cells because that would lead to double counting the interactions. Is it correct to only consider the L shaped cells I have drawn? In the figure, we are considering the atoms in cell 14 and are interested in its neighbors.&lt;/li&gt;&#xA;&lt;li&gt;Currently, I can make a neighbors list if I use A LOT of &lt;code&gt;if/else&lt;/code&gt; statements to get all the boundary conditions correctly. I have been thinking about a more efficient/concise algorithm by detecting patterns but I couldn't come up with anything. Is there an algorithm that is more efficient than doing a bunch of &lt;code&gt;if&lt;/code&gt; statements? Perhaps one that automatically accounts for the periodic boundary conditions too? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Essentially, I'm asking how I can build the neighbors list. For this part, I'm looking to have something like &lt;code&gt;int neighbors[N_c][26]&lt;/code&gt;, where &lt;code&gt;N_c&lt;/code&gt; is the number of cells, and &lt;code&gt;26&lt;/code&gt; is how many neighbors each cell $i$ has. Data type is &lt;code&gt;int&lt;/code&gt; because cells are identified by an &lt;code&gt;int&lt;/code&gt;. &lt;code&gt;neighbors[1]&lt;/code&gt; returns the array of neighbors of cell 1.&lt;/p&gt;&#xA;" OwnerUserId="26366" LastEditorUserId="26366" LastEditDate="2018-07-14T16:49:14.657" LastActivityDate="2018-07-15T10:25:29.970" Title="Effective way to build the neighbor's list in MD" Tags="|algorithms|simulation|molecular-dynamics|nearest-neighbors|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29859" PostTypeId="2" ParentId="27538" CreationDate="2018-07-14T07:20:49.723" Score="1" Body="&lt;p&gt;At least in the deep learning community, GPU support in now the de facto standard. Common libraries (in alphabetical order) such as Caffe, Keras, PyTorch, Theano and Tensorflow are all GPU compatible with single-line commands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following table summarizes current GPU support for most deep learning libraries.&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15120" LastActivityDate="2018-07-14T07:20:49.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29860" PostTypeId="2" ParentId="19780" CreationDate="2018-07-14T17:35:32.097" Score="0" Body="&lt;p&gt;Try this :)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from math import sqrt,pi&#xA;&#xA;e = 1.602e-19 #Coloumbs&#xA;epsNaught = 8.85e-12 #F/m&#xA;M = 0 #Madelung constant, total potential felt by origin sodium atom&#xA;n = 0 #Number of atoms&#xA;&#xA;#lattice size&#xA;L = int(input(&quot;Enter lattice size: &quot;))&#xA;&#xA;#spacing of atoms&#xA;a = int(input(&quot;Distance between atoms: &quot;))&#xA;&#xA;for i in range(-L,L+1):&#xA;    for j in range(-L,L+1):&#xA;        for k in range(-L,L+1):&#xA;            n += 1&#xA;            distance = a * sqrt(i**2 + j**2 + k**2)&#xA;&#xA;            if (i == j == k == 0): #Case for soidium atom at origin&#xA;                continue&#xA;&#xA;            potential = e / (4 * pi * epsNaught * distance)&#xA;&#xA;            if (i+j+k)%2 == 1: #Odd, chlorine atoms, attraction = neg potential&#xA;                potential *= -1&#xA;&#xA;            M += potential&#xA;&#xA;print(&quot;Madelung constant:&quot;,M,&quot;due to&quot;,n-1,&quot;atoms.&quot;)`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28203" LastActivityDate="2018-07-14T17:35:32.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29861" PostTypeId="2" ParentId="29857" CreationDate="2018-07-14T19:51:34.323" Score="0" Body="&lt;p&gt;If this is a file that VMD recognizes, you can create the cell neighbor list with a tcl script that simply defines the total volume of the box you are working with and the cell side-length. Then in 3 nested loops (for x, y, z axis), shift along the whole volume of the box (working in layers) and select all the atoms within the loop-defined cell coordinates and writing them to an external list. &lt;/p&gt;&#xA;" OwnerUserId="28205" LastActivityDate="2018-07-14T19:51:34.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29862" PostTypeId="1" CreationDate="2018-07-15T05:10:16.327" Score="1" ViewCount="216" Body="&lt;p&gt;I have two slightly different but getting the same results MPI code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one is from an open-source package having several data exchange steps in between:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int main ( int argc, char **argv )&#xA;{&#xA;    int i,j,nx=600,nz=300,NP, MYID;&#xA;    int idum[2];&#xA;    float v[420][720];&#xA;&#xA;    for (i=0;i&amp;lt;420;i++){&#xA;      for (j=0;j&amp;lt;720;j++){&#xA;        if(i&amp;lt;161) { v[i][j] = 2800.0; }&#xA;        else { v[i][j] = 5200.0; }&#xA;      }&#xA;    }&#xA;&#xA;    MPI_Init ( &amp;amp;argc, &amp;amp;argv );&#xA;    MPI_Comm_size ( MPI_COMM_WORLD, &amp;amp;NP );&#xA;    MPI_Comm_rank ( MPI_COMM_WORLD, &amp;amp;MYID );&#xA;&#xA;    if(MYID==0){&#xA;        idum[0] = nx;&#xA;        idum[1] = nz;&#xA;    }&#xA;    MPI_Barrier(MPI_COMM_WORLD);&#xA;    MPI_Bcast(&amp;amp;idum,5,MPI_INT,0,MPI_COMM_WORLD);&#xA;    MPI_Bcast(&amp;amp;v,420*720,MPI_FLOAT,0,MPI_COMM_WORLD);&#xA;    MPI_Barrier(MPI_COMM_WORLD);&#xA;&#xA;    nx = idum[0];&#xA;    nz = idum[1];&#xA;&#xA;    for (i=0;i&amp;lt;5;i++){&#xA;      printf(&quot;id=%d,v[%d][350]=%f,\n&quot;,MYID,i*100+19,v[i*100+19][350]);&#xA;    }&#xA;    printf(&quot;nx=%d,nz=%d\n&quot;,nx,nz);&#xA;    MPI_Finalize();&#xA;    exit(0);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I run the code using &lt;code&gt;mpirun&lt;/code&gt; with &lt;code&gt;4&lt;/code&gt; cores. The results are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id=0,v[19][350]=2800.000000,&#xA;id=0,v[119][350]=2800.000000,&#xA;id=0,v[219][350]=5200.000000,&#xA;id=0,v[319][350]=5200.000000,&#xA;id=0,v[419][350]=5200.000000,&#xA;nx=600,nz=300&#xA;id=0,v[19][350]=2800.000000,&#xA;id=0,v[119][350]=2800.000000,&#xA;id=0,v[219][350]=5200.000000,&#xA;id=0,v[319][350]=5200.000000,&#xA;id=0,v[419][350]=5200.000000,&#xA;nx=600,nz=300&#xA;id=0,v[19][350]=2800.000000,&#xA;id=0,v[119][350]=2800.000000,&#xA;id=0,v[219][350]=5200.000000,&#xA;id=0,v[319][350]=5200.000000,&#xA;id=0,v[419][350]=5200.000000,&#xA;nx=600,nz=300&#xA;id=0,v[19][350]=2800.000000,&#xA;id=0,v[119][350]=2800.000000,&#xA;id=0,v[219][350]=5200.000000,&#xA;id=0,v[319][350]=5200.000000,&#xA;id=0,v[419][350]=5200.000000,&#xA;nx=600,nz=300&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I think the data exchange part is a little &quot;redundant?&quot;, so I simplify the above code as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int main ( int argc, char **argv )&#xA;{&#xA;    int i,j,nx=600, nz=300, NP=0, MYID;&#xA;    float v[420][720];&#xA;&#xA;    for (i=0;i&amp;lt;420;i++){&#xA;      for (j=0;j&amp;lt;720;j++){&#xA;        if(i&amp;lt;161) { v[i][j] = 2800.0; }&#xA;        else { v[i][j] = 5200.0; }&#xA;      }&#xA;    }&#xA;&#xA;    MPI_Init ( &amp;amp;argc, &amp;amp;argv );&#xA;    MPI_Comm_size ( MPI_COMM_WORLD, &amp;amp;NP );&#xA;    MPI_Comm_rank ( MPI_COMM_WORLD, &amp;amp;MYID );&#xA;&#xA;    for (i=0;i&amp;lt;5;i++){&#xA;      printf(&quot;id=%d,v[%d][350]=%f,\n&quot;,MYID,i*100+19,v[i*100+19][350]);&#xA;    }&#xA;&#xA;    printf(&quot;nx=%d,nz=%d\n&quot;,nx,nz);&#xA;    MPI_Finalize();&#xA;    exit(0);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the same results as the first code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which one of the two codes is correct? If both are correct, which one is better? Why do we need to have the data exchange lines in the first code, or don't have to?&lt;/p&gt;&#xA;" OwnerUserId="28208" LastActivityDate="2019-01-12T04:04:10.670" Title="MPI data broadcast or not in C" Tags="|parallel-computing|c|mpi|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29863" PostTypeId="2" ParentId="29857" CreationDate="2018-07-15T10:25:29.970" Score="1" Body="&lt;p&gt;There are open-source MD codes, and also text books, that describe how this can be done in general. I don't want to recommend any particular one, but you can search &quot;molecular simulation&quot; or something similar. It might save you some time in your program design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your diagram you are double counting cells 5 and 23. The rule is, you need to look at neighbouring cells with relative positions $(\Delta n_x,\Delta n_y,\Delta n_z)$ where the relative cell indices take all possible combinations of the values $+1$, $0$, $-1$; &lt;em&gt;except that&lt;/em&gt; if you look at $(\Delta n_x,\Delta n_y,\Delta n_z)$, you don't need to look at $(-\Delta n_x,-\Delta n_y,-\Delta n_z)$, so as to avoid double counting. This gives 13 neighbouring cells that need considering, and you can draw up a permanent table of the necessary relative index triplets in the computer code. And, as you realize, you need to look at the central cell, 14 in your diagram, for which $(\Delta n_x,\Delta n_y,\Delta n_z)=(0,0,0)$: when you start looking at the coordinates of pairs of atoms, it needs slightly different treatment to avoid double counting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leads to my main suggestion: give the cells a 3D integer index $(n_x,n_y,n_z)$, and store the cell contents (e.g. positions, or a pointer to a list of positions) in an array (or similar) using these indices. Then, identifying the neighbouring cells is a simple matter of adding $(\Delta n_x,\Delta n_y,\Delta n_z)$ to $(n_x,n_y,n_z)$ to examine each neighbouring cell in turn; the &lt;code&gt;modulo&lt;/code&gt; or &lt;code&gt;mod&lt;/code&gt; function will handle the wraparound of the indices associated with periodic boundary conditions. In other words, calculate &lt;code&gt;modulo(nx+delta_nx,nc)&lt;/code&gt; (or whatever the syntax is in your preferred language) where &lt;code&gt;nx&lt;/code&gt; runs from zero to &lt;code&gt;nc-1&lt;/code&gt; and there are &lt;code&gt;nc&lt;/code&gt; cells along each coordinate direction; similarly for &lt;code&gt;ny&lt;/code&gt; and &lt;code&gt;nz&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to applying the periodic boundary conditions to the vectors between pairs, it is possible to avoid &lt;code&gt;if&lt;/code&gt; statements by using arithmetic statements such as &lt;code&gt;xij = xij - rint(xij/box)*box&lt;/code&gt; where &lt;code&gt;xij&lt;/code&gt; is an uncorrected difference in $x$-coordinates between atoms &lt;code&gt;i&lt;/code&gt; and &lt;code&gt;j&lt;/code&gt;, &lt;code&gt;box&lt;/code&gt; is the box length and &lt;code&gt;rint&lt;/code&gt; is a function that gives the &lt;em&gt;nearest integer&lt;/em&gt; to its argument (again, this may be different for different programming languages).&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-07-15T10:25:29.970" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29864" PostTypeId="2" ParentId="29862" CreationDate="2018-07-15T15:29:42.837" Score="1" Body="&lt;p&gt;A couple of things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It is certainly a bit strange to start the computations before &lt;code&gt;MPI_INIT&lt;/code&gt;. Now, the fill of the array &lt;code&gt;v&lt;/code&gt; is performed before the MPI execution environment is initialized. At least according to MPICH, one should do &lt;a href=&quot;http://www.mpich.org/static/docs/v3.2/www3/MPI_Init.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;as little as possible before &lt;code&gt;MPI_INIT&lt;/code&gt; and after &lt;code&gt;MPI_FINILIZE&lt;/code&gt;&lt;/a&gt;, because the MPI standard does not specify what is actually allowed.&lt;/li&gt;&#xA;&lt;li&gt;I would assume, that the code was supposed to:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialize MPI&lt;/li&gt;&#xA;&lt;li&gt;Fill the array &lt;code&gt;v&lt;/code&gt; on the master node (&lt;code&gt;ID=0&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Broadcast the info about the &lt;code&gt;idum&lt;/code&gt; and &lt;code&gt;v&lt;/code&gt; from &lt;code&gt;ID=0&lt;/code&gt; to other nodes in the communicator (&lt;code&gt;MPI_COMM_WORLD&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Print out some elements of the array on all the nodes.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now a lot of the things go wrong here (if at least some of the assumptions I make are correct)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I doubt that this code is actually launched (properly) using MPI because according to your output, every processor outputs his &lt;code&gt;ID=0&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;If the arrays &lt;code&gt;v&lt;/code&gt; and &lt;code&gt;idum&lt;/code&gt; is supposed to be filled on ALL processors (not only the master one), of course, those broadcasts are useless. But then the MPI parallelization is useless.&lt;/li&gt;&#xA;&lt;li&gt;Your &quot;fixed version&quot; of the code is not practically using MPI at all.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Fix the initialization of the MPI and move it as close to the top of the &lt;code&gt;main&lt;/code&gt; function as possible&lt;/li&gt;&#xA;&lt;li&gt;Determine, where do you want to fill array &lt;code&gt;v&lt;/code&gt;: on all the nodes or only on the master&lt;/li&gt;&#xA;&lt;li&gt;Based on that, decide on the broadcasts if necessary.&lt;/li&gt;&#xA;&lt;li&gt;Make sure you are running it using proper MPI command and your &lt;code&gt;id&lt;/code&gt;s in the output are actually not all zeroes.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-07-15T15:29:42.837" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29866" PostTypeId="2" ParentId="29818" CreationDate="2018-07-16T08:14:31.540" Score="4" Body="&lt;p&gt;I believe that you need the &lt;em&gt;staggered magnetization&lt;/em&gt; (you can search for the term online). For a simple geometry, such as a square lattice, assuming that the ordered phase is like a checkerboard (spins on the black squares point one way and spins on the white squares point the opposite way) you can define it as&#xA;$$&#xA;\mathbf{M}^{\dagger} = \frac{1}{N_x N_y} \sum_{i_x=1}^{N_x} \sum_{i_y=1}^{N_y}&#xA;(-1)^{i_x+i_y} \mathbf{s}_{i_x,i_y}&#xA;$$&#xA;where the spins are labelled by their 2D lattice position $(i_x,i_y)$ (both integers) and there are $N=N_xN_y$ spins in total. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more complicated phases, the term multiplying the spin is $\exp[i \mathbf{q}\cdot(i_x,i_y)]$ where $\mathbf{q}$ is the ordering wave-vector in units consistent with the lattice spacing (usually this exponential factor will turn out to be $\pm 1$ at each lattice site).&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-07-16T08:14:31.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29867" PostTypeId="1" CreationDate="2018-07-16T14:16:20.840" Score="1" ViewCount="81" Body="&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work for a municipality and we are trying to figure out which sections of sewer lines to replace first or at least identify areas that should be looked at. It was suggested I use a Fast Messy Genetic Algorithm like &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-319-15536-4_6&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have records on the sewer lines like date installed, pipe type(clay, plastic, etc), life expectancy of pipe type, tree cover (roots penetrate the pipe causing breaks and blockages), ground water level(if there is a break in this area water gets into the pipe and then is unnecessarily treated), and visual inspections on 45 out of 3,500 pipes(number of breaks in pipe and how sever).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is some interaction between these factors like clay pipes can be broken by roots more easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Where I am suck&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unsure how to make a fitness function. I am thinking I would sum the values like percent of pipe covered by trees, percent of pipe blow ground water, age of pipe divided by its life expectancy. However, I am not sure how to account for clay pipes interacting with tree coverage (possibly multiply tree coverage for a some weight of each pipe type).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I set up a fitness function for the above parameters? &lt;/p&gt;&#xA;" OwnerUserId="28214" LastActivityDate="2018-09-23T22:07:53.267" Title="Ranking Sewer Lines worst to best condition using Genetic Algorithm?" Tags="|algorithms|genetic-algorithms|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29869" PostTypeId="1" CreationDate="2018-07-17T00:49:42.393" Score="6" ViewCount="509" Body="&lt;p&gt;I want to minimize a non-linear function $f(x)$ using Newton's method. At each optimization step, I compute a descent direction $d$ to update $x$ using a second-order approximation of $f(x)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \nabla^2f(x) \ d = -\nabla f(x) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I need to solve a linear system, but the Hessian matrix $\nabla^2f(x)$ might not be positive definite which could give me an invalid descent direction. I want to run conjugate gradient to solve the linear system. In the conjugate gradient method, one of the steps involves this operation $v^T \nabla^2\ f(x)\ v$ (computing $\alpha$ according to Shewchuk's notes, page 50 &lt;a href=&quot;https://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf&lt;/a&gt; , I'm using $v$ rather than $d$ because in my convention $d$ is the descent direction to update $x$). I could recycle this operation to know if the Hessian is not positive definite (if such operation is negative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I implemented my algorithm like that, so as soon as I detect that operation is negative, I stop the CG solver and return the solution iterated up to that point. I've actually seen it works pretty well in practice, but I have no rigorous justification for doing it. I remember hearing / reading people saying that each CG iteration takes care of different eigenvalues in order of &quot;importance&quot;, so by stopping early you end up exploring the positive definite part of the Hessian. Can anybody point me to a more rigorous justification of this? Is this a valid way to enforce positive definiteness using CG for non-linear optimization?&lt;/p&gt;&#xA;" OwnerUserId="28022" LastEditorUserId="28022" LastEditDate="2018-07-18T17:21:05.067" LastActivityDate="2018-07-18T17:21:05.067" Title="conjugate gradient for Newton's method with non positive definite Hessian matrix" Tags="|optimization|linear-solver|" AnswerCount="0" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29870" PostTypeId="1" CreationDate="2018-07-17T01:26:38.723" Score="2" ViewCount="787" Body="&lt;p&gt;I am using a (central) finite difference scheme to solve the eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\frac{d^2}{dx^2}u = \lambda u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with periodic boundary conditions on a unit interval.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use arpack's zndrv1 and the standard discretization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$-\frac{d^2}{dx^2} = \left(\begin{matrix}2t_0 &amp;amp; -t_0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; -t_0 \\&#xA;-t_0 &amp;amp; 2t_0 &amp;amp; -t_0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;\vdots &amp;amp;-t_0 &amp;amp; 2t_0 &amp;amp; -t_0 &amp;amp; \vdots \\&#xA;0 &amp;amp; \cdots &amp;amp;-t_0 &amp;amp; 2t_0 &amp;amp; -t_0 \\&#xA;-t_0 &amp;amp; \cdots &amp;amp; 0 &amp;amp; -t_0 &amp;amp; 2t_0 \\&#xA;\end{matrix}\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expected to obtain the eigenvalues with the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$u(x) = Ae^{in\pi x}\\ |u(x)|^2 = A$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, when I plot $|u(x)|^2$ I get periodic oscillations instead of $A$. E.g. The image below is the 3rd eigenfunction squared.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gXpkd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gXpkd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question: Should I expect eigenfunctions of the form $e^{in\pi x}$ or are more general eigenfunctions permitted?&lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2018-07-17T19:11:12.457" Title="Eigenvalue problem with periodic boundary conditions: Are my eigenvalues correct?" Tags="|finite-difference|eigenvalues|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29872" PostTypeId="1" AcceptedAnswerId="29874" CreationDate="2018-07-17T13:32:34.393" Score="4" ViewCount="556" Body="&lt;p&gt;In a quantum physics calculation, I have to deal with a matrix that has a lot of eigenvalues really close to each other ($10^{-6}$ relative difference for example) and I can't manage to obtain accurate enough eigenvectors and eigenvalues. Indeed, I would like to reach machine precision and I get only $10^{-9}$ with &lt;code&gt;zheev&lt;/code&gt; from LAPACK both with MKL and ATLAS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://diracprogram.org/doc/release-15/programmers/diag.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; implies that it is due to LAPACK itself and that it is possible to achieve better precision with other methods. I know that it is totally possible with my matrix since I achieve the expected precision with &lt;code&gt;numpy.linalg.eigh&lt;/code&gt; but I can't tell what is different between my use of LAPACK and what is done in numpy. &#xA;(I explain the comparison between the two &lt;a href=&quot;https://stackoverflow.com/questions/51322226/inaccurate-zheev-eigen-values-and-vectors&quot;&gt;there&lt;/a&gt; but my question here is different).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is an algorithm that is known to be efficient for really close eigenvalues. The restrictions are that I don't use really big matrices (&lt;strong&gt;less than 100 lines&lt;/strong&gt; for now) that are &lt;strong&gt;Hermitian&lt;/strong&gt; and I know that the eigenvalues are all comprised &lt;strong&gt;between 0 and 2&lt;/strong&gt; (both included).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to compute &lt;strong&gt;all&lt;/strong&gt; eigenvalues and vectors with the maximum precision.&lt;/p&gt;&#xA;" OwnerUserId="28227" LastEditorUserId="20688" LastEditDate="2018-07-17T14:36:41.917" LastActivityDate="2018-07-17T14:40:39.897" Title="Compute nearly degenerated eigen values and vectors" Tags="|matrix|eigenvalues|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29873" PostTypeId="1" AcceptedAnswerId="29875" CreationDate="2018-07-17T13:45:33.567" Score="5" ViewCount="4525" Body="&lt;p&gt;I want to calculate the carrier concentration of my semiconductor using this equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;n(x) = \frac{m^*}{\pi\hbar^2}\int_{E_k}^{\infty}\frac{1}{1+\exp\left(\frac{E-E_f}{k_BT}\right)} \mathrm{d}E&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using this straightforward scipy approach:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.constants as phys&#xA;import scipy.integrate as integrate&#xA;&#xA;eigenvalue = [0.9 * phys.electron_volt, 1.3 * phys.electron_volt]&#xA;fermi = 1.0 * phys.electron_volt&#xA;T = 300&#xA;&#xA;def fermi_integral(E, fermi, T):&#xA;    return 1 / (1 + np.exp((E - fermi) / (phys.Boltzmann * T)))&#xA;&#xA;for i in range(len(eigenvalue)):&#xA;    result = integrate.quad(fermi_integral, eigenvalue[i], np.inf, args=(fermi, T))&#xA;    print(result)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I'm running into &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RuntimeWarning: overflow encountered in exp&#xA;  return 1 / (1 + np.exp((E - fermi) / (phys.Boltzmann * T)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and my result is always &lt;code&gt;(0.0, 0.0)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess I have to use another approach, but I'm stuck and I hope you can give me some helpful input.&lt;/p&gt;&#xA;" OwnerUserId="28229" LastEditorUserId="20688" LastEditDate="2018-07-17T14:43:20.477" LastActivityDate="2022-01-26T13:02:52.970" Title="Integration of the Fermi distribution using Python" Tags="|python|scipy|integration|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29874" PostTypeId="2" ParentId="29872" CreationDate="2018-07-17T14:40:39.897" Score="3" Body="&lt;p&gt;Since your eigenvalues are very localized, I can suggest trying the &lt;a href=&quot;http://www.ecs.umass.edu/~polizzi/feast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEAST&lt;/a&gt; algorithm. It really benefits from a tight search interval and can be beneficial in terms of speed and accuracy (and v2.1 is readily available from Intel MKL, while you might opt in using the newer one by a separate library download).&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-07-17T14:40:39.897" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29875" PostTypeId="2" ParentId="29873" CreationDate="2018-07-17T15:24:45.883" Score="7" Body="&lt;p&gt;One of your problems is the system of units that you are using. Just changing the units improves the results&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.integrate as integrate&#xA;&#xA;eigenvalue = [0.9, 1.3]&#xA;fermi = 1.0&#xA;T = 300&#xA;kB = 8.6173303e-5&#xA;&#xA;&#xA;def fermi_integral(E, fermi, T):&#xA;    return 1 / (1 + np.exp((E - fermi) / (kB * T)))&#xA;&#xA;&#xA;for i in range(len(eigenvalue)):&#xA;    result = integrate.quad(fermi_integral, eigenvalue[i], np.inf,&#xA;                            args=( fermi, T))&#xA;    print(result)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are the results&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(0.10053464900138948, 1.2260194855049155e-09)&#xA;(2.3589139312840435e-07, 4.1489077967047375e-11)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I still get the overflow problem, but that is probably because you are evaluating the functions for really large numbers. I just replaced &lt;code&gt;np.inf&lt;/code&gt; for 10 and obtained the same results.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Edit&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Taking into account @gammatester suggestion the overflow warning does not appear anymore. The following works for me.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.integrate as integrate&#xA;&#xA;eigenvalue = [0.9, 1.3]&#xA;fermi = 1.0&#xA;T = 300&#xA;kB = 8.6173303e-5&#xA;&#xA;&#xA;def fermi_integral(E, fermi, T):&#xA;    if E &amp;lt; fermi:&#xA;        return 1 / (1 + np.exp((E - fermi) / (kB * T)))&#xA;    else:&#xA;        return np.exp(-(E - fermi) / (kB * T)) / (1 + np.exp(-(E - fermi) / (kB * T)))&#xA;&#xA;&#xA;for i in range(len(eigenvalue)):&#xA;    result = integrate.quad(fermi_integral, eigenvalue[i], np.inf,&#xA;                            args=( fermi, T))&#xA;    print(result)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-07-17T19:13:20.660" LastActivityDate="2018-07-17T19:13:20.660" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29876" PostTypeId="1" AcceptedAnswerId="29877" CreationDate="2018-07-17T17:07:53.783" Score="0" ViewCount="663" Body="&lt;p&gt;Say, I have this code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double start = MPI_Wtime();&#xA;ParallelMatrixVectorProductRotuine(...);&#xA;double end = MPI_Wtime();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I interpret the numbers I am getting. For instance, with 1 process I get something like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.7342&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For two processes I get &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.0023&#xA;1.6432&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What can I say about this? Do I take the average of these results?&lt;/p&gt;&#xA;" OwnerUserId="27791" LastEditorUserId="20688" LastEditDate="2018-07-17T17:17:07.310" LastActivityDate="2018-07-17T21:18:05.393" Title="How do I interpret the results from MPI_Wtime()?" Tags="|parallel-computing|mpi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29877" PostTypeId="2" ParentId="29876" CreationDate="2018-07-17T17:15:59.857" Score="4" Body="&lt;p&gt;That depends on your goal.&#xA;For one process it's obvious that $t$ is the wall-time of the entire program launch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For two processes, you got $t_1$ and $t_2$ which correspond for the walltime on processors 1 and 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, one is interested in the total time it took to solve the problem - aka overall walltime, which will be $t=\max(t_1,t_2)$. However, I would recommend some additional measures to do a proper job:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put an &lt;code&gt;MPI_BARRIER&lt;/code&gt; before and after the computation is done.&#xA;Measure the walltime only on the master node, which better be done right before and after the barrier. Then, you will obtain the accurate walltime of the &lt;code&gt;ParallelMatrixVectorProductRotuine(...);&lt;/code&gt; for $N$ processors.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-07-17T17:15:59.857" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29878" PostTypeId="2" ParentId="29870" CreationDate="2018-07-17T19:11:12.457" Score="-2" Body="&lt;p&gt;This Wiki page &lt;a href=&quot;https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors_of_the_second_derivative#Periodic_boundary_conditions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigenvalues and eigenvectors of the second derivative&lt;/a&gt; discusses the eigenpairs for the continuous operator and for the discretized operator. &lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2018-07-17T19:11:12.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29879" PostTypeId="2" ParentId="29876" CreationDate="2018-07-17T21:18:05.393" Score="1" Body="&lt;p&gt;There can be any number of reasons for different times on different processes.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If one process has lots of local work after the last MPI communication, it will report a longer running time.&lt;/li&gt;&#xA;&lt;li&gt;If one process concludes with an eager send, it will report a shorter running time than the other process, which has to perform the receive.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So the only thing you can say is that the processes are somehow unbalanced. What should you report as the objective overall time for the operation? Does such a concept make sense?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Maybe your code will go on to do local work, in which case your product seamlessly transitions into the next operation, and the concept of an overall time makes no sense.&lt;/li&gt;&#xA;&lt;li&gt;Maybe the next operation is something collective like an inner product, and the processor that finished early has idle time.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you really want to assign an objective global time, then I would use barriers, and put the timers immediately after the barriers. Even though a barrier is not a time synchronization semantically, in practice it often is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But bear in mind that your production code will be faster if you then remove the barriers again, meaning that the concept of &lt;em&gt;the&lt;/em&gt; time was ill-defined to begin with.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2018-07-17T21:18:05.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29881" PostTypeId="1" CreationDate="2018-07-18T14:13:49.327" Score="4" ViewCount="130" Body="&lt;p&gt;I am charged with maintaining a buildserver on Teamcity which is meant to test our in house FE software. Currently our test suite consists of a list of benchmarks which run every time a commit is made using svn as version control software. The benchmarks are fully meshed boundary value problems intended to test various features of the software. The features are tested by running the head revision of the kernel on each benchmark and then comparing an output file with a reference file. The reference file is a result for a previous run of the test which was deemed &quot;correct&quot; by an expert/academic. The tests are thus considered to be &quot;broken&quot; when the result of this output file is different from that generated by the initial run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to improve upon this system by having our test suite run comparisons based on physical/analytical solutions for each benchmark rather than an arbitrary output file. However, in this case, how does one judge when a test is &quot;broken&quot;? The analytical solution can and will never be exactly the same as the numerical one, so comparing two files is out of the question. Does there exist a general method/quality control process perhaps based on statistics which can ensure that our code remains scientifically viable? How do major software companies like ANSYS or Bentley ensure that their scientific software does not deviate from exact solutions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note I am in a commercial software development office setting and am looking for things that can be implemented in a straightforward manner within the Teamcity/tortoisesvn/Visual Studio environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone had such experience before? &lt;/p&gt;&#xA;" OwnerUserId="22736" LastActivityDate="2018-07-23T04:16:40.313" Title="Analytical testing/quality control for scientific software in professional setting" Tags="|finite-element|fortran|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29883" PostTypeId="2" ParentId="29881" CreationDate="2018-07-18T18:16:12.733" Score="4" Body="&lt;p&gt;There's a &lt;a href=&quot;https://www.osti.gov/servlets/purl/759450/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sandia technical report&lt;/a&gt; discussing the Method of Manufactured Solutions (a generalisation of the concept of exact solutions to PDEs) which goes to some lengths to discuss the concept of acceptance criteria on verification tests for numerical codes. They propose 4 classes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Expert Judgement,&lt;/li&gt;&#xA;&lt;li&gt;Percentage Error,&lt;/li&gt;&#xA;&lt;li&gt;Consistency,&lt;/li&gt;&#xA;&lt;li&gt;Order of accuracy,&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here it appears you're currently using a version of expert judgment, tied to a regression test on the output data. Bill Greene's comment encompasses the idea of percent error, with an acceptable tolerance in the absolute difference between the numerically calculated value and the analytic one. A &quot;good&quot; choice of value of this tolerance is to some degree still a judgement call, and is a function of the discretisation, the implementation and the computer architecture being used. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just testing values for a single discretisation of a given system is still less rigorous than convergence testing, in which runs are performed at multiple resolutions, and it is confirmed that the reduction in the error in the system scales with grid length in the manner theory suggests it should (of course this comparison of real numbers also requires a tolerance). The biggest annoyance with using this method under CI is that it can need moderately high resolution to sit in the region dominated by the leading order error term (rather than higher order terms above and machine precision below). This makes run times longer than comfortable at times when multiple commits are occuring.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2018-07-18T18:16:12.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29885" PostTypeId="1" AcceptedAnswerId="29886" CreationDate="2018-07-19T06:05:21.100" Score="0" ViewCount="81" Body="&lt;p&gt;I have an assignment where i got this definition - &quot;The dots velocities following the Maxwell distribution with typical velocity of $$v=&amp;lt;v&amp;gt;^{\frac{1}{2}}$$ and that mean that each of the velocity vector components vave normal distribution with mean zero and $$\sigma = \frac{v}{\sqrt{3}}$$  Someone can explain to me the meaning of that? how the mean of all the components can be zero if for example V=85 kmh? The dots are stand for stars in galaxy, my assignment is to do Barnes-Hut Galaxy simulation. &lt;/p&gt;&#xA;" OwnerUserId="27468" LastActivityDate="2018-07-19T11:28:46.330" Title="Barnes-Hut method for gravitational dynamics" Tags="|simulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29886" PostTypeId="2" ParentId="29885" CreationDate="2018-07-19T09:57:07.503" Score="2" Body="&lt;p&gt;Your question may need editing to be more clear. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your first equation is dimensionally incorrect (velocity on the left, square root of velocity on the right). &lt;em&gt;Perhaps&lt;/em&gt; the intention is to define a root-mean-square &lt;em&gt;speed&lt;/em&gt; $$v_{\text{rms}}=\langle v^2\rangle^{1/2}$$ where $v=|\mathbf{v}|$ and $\mathbf{v}=(v_x,v_y,v_z)$. But you need to check.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A normal probability distribution (for each of $v_x$, $v_y$ and $v_z$) can quite reasonably have zero mean velocity $\langle v_x\rangle=0$ and similarly for $v_y$ and $v_z$ (symmetric with respect to $v_x\rightarrow -v_x$ etc). It will have nonzero standard deviation $\sigma$. &#xA;In that case $\sigma^2=\langle v_x^2\rangle =\langle v_y^2\rangle=&#xA;\langle v_z^2\rangle$ since all directions are equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $v^2=v_x^2+v_y^2+v_z^2$ it would follow that &#xA;$$v_{\text{rms}}^2=\langle v_x^2\rangle +\langle v_y^2\rangle+&#xA;\langle v_z^2\rangle =3\sigma^2$$&#xA;This would be your second equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distribution is a product of independent Gaussian distributions for each Cartesian component. In your program, you just need to sample each component independently from a normal distribution with zero mean and standard deviation $\sigma$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look up &quot;Maxwell-Boltzmann&quot; distribution online you will find more details. But you will probably also see that the distribution is usually applied to atoms and molecules, with the particle mass $m$, temperature $T$, and Boltzmann's constant $k_{\text{B}}$ determining the value of $\sigma$. Those parameters might be a bit distracting in the context of setting up an astrophysical simulation, so it makes more sense to specify $\sigma$ in terms of a root-mean-square speed. I assume that is what is going on here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm just guessing. If this answer does not seem to explain things, please check the details of your assignment and clarify your question, so people can understand your problem better.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastEditorDisplayName="user28077" LastEditDate="2018-07-19T11:28:46.330" LastActivityDate="2018-07-19T11:28:46.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29887" PostTypeId="1" CreationDate="2018-07-19T11:37:02.090" Score="1" ViewCount="65" Body="&lt;p&gt;I did a liquid argon simulation at 100k. I forgot during analysis that I need to accommodate for the Periodic Boundary Conditions before doing any analysis which included distance. &lt;strong&gt;What are other such general things I should keep in mind during my analysis ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am rather novice to computational Science and might have missed even the most trivial things. &lt;/p&gt;&#xA;" OwnerUserId="25310" LastActivityDate="2018-07-20T09:43:47.287" Title="What are the things I should keep in mind before doing an analysis of my gromacs simulation?" Tags="|computational-physics|simulation|computational-chemistry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29888" PostTypeId="1" AcceptedAnswerId="29891" CreationDate="2018-07-19T13:43:53.420" Score="5" ViewCount="670" Body="&lt;p&gt;I have a matrix, whose size scales as $2^N$ (assume even $N$). In each row of the matrix, only about $2^{N/2}$ of the entries are filled ($N$ can be somewhere between 10 and 40, depending on what's feasible computationally). I'm not sure whether using sparse matrices would be suitable for storage and computation - there aren't that many non-zero entries in a &lt;em&gt;relative&lt;/em&gt; sense compared to the overall matrix size, but there are still a lot in an absolute sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an additional question, if using sparse matrices is indeed the way to go, what's an approximate order-of-magnitude size for which I could reasonably solve systems of linear equations, i.e. $Ax=b$, with a direct solver? I, unfortunately, don't have a good sense of what sort of size is reasonable, and this influences whether the method of solving the problem I'm considering is even remotely feasible (the exact number, of course, depends on the resources I have available - I was wondering in a more general sense what this dependency between size and resources/time was like, but specific examples would also be helpful, i.e. how big of a system size I could do on an average desktop in a few hours).&lt;/p&gt;&#xA;" OwnerUserId="28245" LastEditorUserId="20688" LastEditDate="2018-07-19T18:50:25.677" LastActivityDate="2018-07-19T18:50:25.677" Title="How &quot;sparse&quot; should a sparse matrix be to see benefits?" Tags="|linear-algebra|matrix|linear-solver|sparse-matrix|linear-system|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29889" PostTypeId="2" ParentId="29888" CreationDate="2018-07-19T14:45:46.527" Score="3" Body="&lt;p&gt;This really comes down to what methods you are intending to use. If you store the matrix dense then you will use dense factorization routines, the cost of which will scale cubically with the matrix size, and so $2^{3N}$ for your case. That limits the $N$ dramatically, even on today's best supercomputers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be able to push this slightly further with a sparse representation and a direct solver, but based on your comment about how many nonzeros you have per row I highly doubt a sparse direct solver will really outperform a dense one. The reason is that sparse direct solvers will try and compute fill-reducing reorderings of your unknowns and equations, but this becomes less effective with less sparse matrices. I suspect that in the end, even with an optimal reordering in your case, the resulting factorization will still be completely dense, and so leaving us with the same limitations as the dense case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the above comments in mind we might consider &lt;em&gt;iterative&lt;/em&gt; solvers like GMRES. The primary barrier in these methods for your case isn't computational cost, but memory. These will scale significantly better than the direct solvers mentioned above, but rarely work in a black-box fashion. You can try it black-box first to see if it works, but experience suggests it will not converge for larger $N.$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fix this you will need some preconditioner. There are a few black-box preconditioners to choose from that might work, but this requires some experimentation to find out. As a rule of thumb, a good preconditioner will take about as much memory to store as the matrix does itself. Thus in this case it might make sense to store your matrix sparse in order to make more space for preconditioners in memory. Also most available black-box preconditioners require you to have your matrix in a sparse format anyways, so if you don't want to rewrite those yourself a likely next step is to transition into a sparse format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is leaving out some other questions that might impact what you do: like how much memory you actually have available. (as suggested by comment: take &quot;memory&quot; here to mean whatever storage you are using, whether it be DRAM, Burst Buffer, Disk,... )&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2018-07-19T15:47:22.270" LastActivityDate="2018-07-19T15:47:22.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29890" PostTypeId="1" CreationDate="2018-07-19T16:23:37.620" Score="1" ViewCount="369" Body="&lt;p&gt;I want to verify and compare different Discretizations of the anisotropic diffusion equation in 2D / 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CwELG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CwELG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to both test the timestepping and the spatial discretisations I had a look at using the heat kernels as an analytical solution to diffusion equations. The one dimensional heat kernel (&lt;a href=&quot;https://en.wikipedia.org/wiki/Heat_kernel&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki-link&lt;/a&gt;) looks like this:&#xA;$$&#xA;u(x,t) = \frac{1}{(4\pi t)^{d/2}} e^{-x^2/4t}\&#xA;$$&#xA;and solves following diffusion equation of dimension $d$ analytically:&#xA;$$&#xA;\frac{\partial u}{\partial t} = -\Delta u&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that for an analytical solution I can verify my convergence both spatially and temporal. It is important in my case, that the diffusion is &lt;strong&gt;anisotropic&lt;/strong&gt;, and might not be aligned with the underlying grids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My &lt;strong&gt;anisotropic&lt;/strong&gt; diffusion equation reads:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial u}{\partial t} = \nabla (D \nabla u)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to expand the analytical solution to my case, where the diffusion is not scalar and unity, but a full (symmetric positive definite) tensor D, I had a look at multivariate gaußian distributions (&lt;a href=&quot;https://en.wikipedia.org/wiki/Multivariate_normal_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki-link&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there somewhere in the literature a heat kernel for anisotropic (but homogeneous) diffusion which I can use to verify my numerics?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;By analogy I would come up with something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u(\vec{x},\vec{\mu},t) = \frac{1}{(4\pi~ |\mathbf{D}| t)^{d/2}} \exp \large(\frac{-(\vec{x}-\vec{\mu})^{T} \mathbf{D}^{-1}(\vec{x}-\vec{\mu})}{4~|\mathbf{D}|~t}\large).&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Is the above equation a proper analytical solution of the anisotropic diffusion equation?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am not sure if the exponent $d/2$ is correct for the $|D|$. I am aware that the question might touch both physics, and numerics, but I thought this might be useful to other people trying to verify their codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hints or help is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="28246" LastActivityDate="2018-08-22T09:01:02.057" Title="Analytical testcase for 2D/3D anisotropic Diffusion (Heat Kernel)" Tags="|convergence|diffusion|heat-transfer|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29891" PostTypeId="2" ParentId="29888" CreationDate="2018-07-19T17:22:31.977" Score="5" Body="&lt;p&gt;This kind of scaling is fairly common and sparse direct factorization methods are commonly used on matrices of up to a few hundred thousand rows and columns.  By the time you get to $N=20$, you’ll have a million rows with a thousand nonzero entries per column.  This is getting into the range where sparse direct factorization becomes impractical and you’ll need to look at iterative methods.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At $N=20$, you would need about 10 gigabytes to store your matrix in double precision, which should be no problem on a contemporary PC.  However, by the time you get to $N=40$, you would need about 10^19 bytes of storage, which is beyond the capacity of current supercomputers.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-07-19T17:22:31.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29892" PostTypeId="2" ParentId="29887" CreationDate="2018-07-20T09:43:47.287" Score="4" Body="&lt;p&gt;I think that this question is too generic for a complete answer, as the latter would depend entirely on what you are simulating and what observables you are interested in.&#xA;The only things that come to mind are&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Remember to use relative velocities (relative to the system's centre of mass) when computing dynamical observables (such as the mean-squared displacement or the intermediate scattering function).&lt;/li&gt;&#xA;&lt;li&gt;The radial distribution function makes sense only up to $r = L/2$, where $L$ is the side of the box.&lt;/li&gt;&#xA;&lt;li&gt;If the system crystallises, be aware that you probably need NPT (that is, constant-pressure) simulations to get meaningful results.&lt;/li&gt;&#xA;&lt;li&gt;Very close to a critical point (or to some other transitions/crossovers such as the percolation line) some quantities become heavily size-dependent, meaning that you need to do some extrapolations with the system size in order to get the &quot;true&quot; values.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27040" LastActivityDate="2018-07-20T09:43:47.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29893" PostTypeId="1" AcceptedAnswerId="29894" CreationDate="2018-07-20T15:15:05.103" Score="3" ViewCount="194" Body="&lt;p&gt;Say if I wanted to parallelise an algorithm. Only certain parts of this algorithm can be parallelised. The other parts are trivial and not worth distributing. Do I do all of these trivial calculations on my master node and then broadcast the data to all remaining nodes OR do I conduct these calculations on ALL processors?&lt;/p&gt;&#xA;" OwnerUserId="27791" LastActivityDate="2018-07-21T15:04:16.270" Title="Writing a parallel version of an algorithm. Only which someparts are worth distributing" Tags="|mpi|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29894" PostTypeId="2" ParentId="29893" CreationDate="2018-07-20T19:31:30.550" Score="10" Body="&lt;p&gt;If the operation is as trivial as you say, and each node has all the information necessary to carry out the operation, then the communication will be substantially more expensive than recomputing locally on each node. That said, it's a good exercise to write both implementations and compare; experiment is better than the advice of strangers on the internet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The break-even point when a task becomes non-trivial will depend on how the problem has been parallelized. It's much lower for shared-memory parallelism on the same machine than it is for distributed-memory parallelism across a cluster. For that matter, it probably depends on whether your cluster is using ethernet or infiniband. A handy reference that I often come back to is &lt;a href=&quot;https://gist.github.com/jboner/2841832&quot; rel=&quot;noreferrer&quot;&gt;Latency Numbers Every Programmer Should Know&lt;/a&gt;. It's 6 years old at this point but the same principle still applies.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-07-20T19:31:30.550" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29895" PostTypeId="2" ParentId="29802" CreationDate="2018-07-20T19:40:37.753" Score="1" Body="&lt;p&gt;&lt;strong&gt;Combinatorial Laplacian ?&lt;/strong&gt; It depends on what you expect from your solution. Something reasonable to expect is that your solution should be independent of the way you mesh your surface (or as independent as possible). If you want that, then clearly a combinatorial graph Laplacian is not what you need, since the result would be completely different if you change the mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discrete or Continous ?&lt;/strong&gt; A possibility is to consider that your surface is not &lt;em&gt;discrete&lt;/em&gt;, it is rather a &lt;em&gt;continuous&lt;/em&gt; object. The surface is &lt;em&gt;parameterized&lt;/em&gt; by a set of discrete variables (points and function values at the vertice).  For instance, a triangle $T$ is defined by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ T = \{ \lambda_1 p_1 + \lambda_2 p_2 + \lambda_3 p_3\ |\ \lambda_1 + \lambda_2 + \lambda_3 = 1; \lambda_1 \ge 0; \lambda_2 \ge 0; \lambda_3 \ge 0\}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $p_1, p_2, p_3$ denote the three vertices of the triangle. Note that&#xA;the triangle contains an &lt;em&gt;infinity&lt;/em&gt; of points (it is continuous !) but it is &lt;em&gt;fully described&lt;/em&gt; by its three vertices. Similarly, when considering piecewise linear functions defined by their values at the vertices, you consider continuous functions (they are $C^0$) described by a &lt;em&gt;discrete&lt;/em&gt; vector of their values at the vertices.  It is the &lt;em&gt;function space&lt;/em&gt; that is discrete, not the &lt;em&gt;geometry&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discretizing with FEM&lt;/strong&gt; With this viewpoint (classical FEM), by projecting the equation onto the basis on piecewise linear functions (or other ones if you want), you can construct a FEM discretization of your equation and solve it. Refer to a classical textbook on how to do that, for instance [5] (also available on its author's webpage as pdf files in English and French), that fully explains the derivation for the Laplacian projected onto piecewise linear functions. Some care needs to be taken: too ofen computer graphics papers mix the discretization of the operator (stiffness matrix) with the &quot;dot product&quot; of the used function basis (mass matrix), by combining them into a single matrix (that they call &quot;discrete Laplacian&quot;). To fully understand what is going on, you will need to consider them separate, and use the mass matrix each time you want to project a function onto the function basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Varadhan's theorem and topology:&lt;/strong&gt; Something that needs to be mentioned about the &quot;heat method&quot; [3] that you cite: it is based on a theorem proved by Varadhan [1]. The original article by Varadhan proves the result by using a parameterization of the surface, and by &quot;pulling-back&quot; all computations in parameter space (using the chain rule).&#xA;Since it uses a parameterization, the proof is only valid for objects that have a non-degenerate parameterization. In particular this imposes that the object under consideration is homeomorphic to a disk. This restriction on Varadhan's theorem is also mentioned in [2], page 400: &quot;Varadhan's formula works within the injectivity radius, what happens when y moves to the cut locus of x ... dramatic changes take place ... strange events occur at antipodal points&quot;. The 'cut locus' is the zone where you would glue a toplogical disk to form an object of arbitrary topology (or put differently, if you grow a 'Voronoi cell' from a point, this is where the boundary of the Voronoi cell will self-collide). If you want to use something else than a topological disk, then you are on your own (no mathematical guarantee). However, [3] reports some empirical results tending to show that what you obtain is reasonable, so for practical applications it may do the job. Now if you want to prove theorems, you need to know that they will only work for topological disks (unless you find a way of extending them to more general cases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Varadhan 1967, Diffusion processes in a small time interval, Comm. Pure and Applied Math, 20, 659-685&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] A panoramic view of Riemannian geometry, Marcel Berger, Springer, 2003&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Geodesics in heat, K. Crane et.al, 2013&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5] &lt;a href=&quot;https://global.oup.com/academic/product/numerical-analysis-and-optimization-9780199205219?cc=fr&amp;amp;lang=en&amp;amp;&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://global.oup.com/academic/product/numerical-analysis-and-optimization-9780199205219?cc=fr&amp;amp;lang=en&amp;amp;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2018-07-24T18:19:33.520" LastActivityDate="2018-07-24T18:19:33.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29896" PostTypeId="2" ParentId="29893" CreationDate="2018-07-20T20:12:28.880" Score="5" Body="&lt;p&gt;An addition to @Daniel Shapero's answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might also be important to know if there are computations that can lead to different results depending on which machine they are launched (or just vary from launch to launch) – and &lt;strong&gt;how sensitive&lt;/strong&gt; is your code to those fluctuations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These will be even more pronounced if the cluster you are working on is inhomogeneous (nodes can be different).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possible examples:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Computations that depend on random number generator (with non-fixed seed)&lt;/li&gt;&#xA;&lt;li&gt;Floating-point calculations. Which might slightly differ from machine-to-machine (if the architecture/OS is different) or even on the same machine (say, OpenMP reduction has been performed).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, if this is the case and your future calculation &lt;strong&gt;are really sensitive&lt;/strong&gt; to that, you might consider broadcasting instead of everybody calculating this step individually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, the benchmarking will give you the ultimate answer.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-07-20T20:12:28.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29897" PostTypeId="2" ParentId="29893" CreationDate="2018-07-21T14:49:11.747" Score="1" Body="&lt;p&gt;Since you tagged this with &quot;MPI&quot;: there is no master node. The stuff that is too trivial should be done redundantly on every process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bear in mind that even sending a single byte through an MPI message is the equivalent of (tens of) thousands of operations.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastEditorUserId="4087" LastEditDate="2018-07-21T15:04:16.270" LastActivityDate="2018-07-21T15:04:16.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29898" PostTypeId="2" ParentId="28645" CreationDate="2018-07-21T17:39:03.957" Score="1" Body="&lt;p&gt;Oscillations is a natural result of higher-order approximations near discontinuities/shocks for hyperbolic conservation laws. Recall that the finite-difference approximations you have listed are generally derived using truncated Taylor expansions, which requires a degree of smoothness which is not present in your model problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you have observed, the single-point upwinding does not suffer from oscillations, but has low order. One solution is to use a limiter, which uses the low order approximation where the solution is not sufficiently smooth. Many possible choices are found in the literature, some of which are detailed on wikipedia: &lt;a href=&quot;https://en.wikipedia.org/wiki/Flux_limiter&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Flux_limiter&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1273" LastActivityDate="2018-07-21T17:39:03.957" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29900" PostTypeId="1" CreationDate="2018-07-22T13:19:50.730" Score="1" ViewCount="62" Body="&lt;p&gt;I read some books and scoured the web about design pattern,but almost all ways of expressing design patterns are the same. They define what is it,draw UML and give an example,but what I want is to suggest me any site or book that writes some bad code and refactor using one of the design patterns?&lt;/p&gt;&#xA;" OwnerUserId="28268" LastActivityDate="2018-07-22T13:19:50.730" Title="Understanding design patterns" Tags="|design-patterns|" AnswerCount="0" CommentCount="3" ClosedDate="2018-07-22T15:45:14.027" ContentLicense="CC BY-SA 4.0" />
  <row Id="29902" PostTypeId="1" CreationDate="2018-07-22T21:48:48.923" Score="0" ViewCount="211" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am using a neural network to calculate the potential energy of atoms in a configuration and then adding energy of all atoms to compare it with the true energy of the configuration(label) to update the weights. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing so for 10000 configurations of 100 atoms each for liquid Argon at 100K.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; I don't have the actual energy of each atom and only the energy of the configuration.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I need to calculate the forces on each atom to see if the sum of forces for all the atoms is zero in X,Y,Z axis. In case it is not I want to find out how off is it from zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I calculate the force on atoms using their position and potential energy for a given trajectory. I know I need to do $\frac{dE}{dr}$ but what should be my E and r. Also since I don,t a formula for E(r), what should my $dE$ and $dr$ be respectively?&lt;/p&gt;&#xA;" OwnerUserId="25310" LastActivityDate="2018-07-24T12:58:51.513" Title="Calculate forces on atoms from potential energy of system and position of atoms" Tags="|molecular-dynamics|differential-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29903" PostTypeId="1" CreationDate="2018-07-23T02:22:35.427" Score="5" ViewCount="193" Body="&lt;p&gt;I often see people using products of 1-D polynomials to do interpolation or projection of smooth multivariate functions over grids or cells because it is intuitive and simple to implement. What are the effects of using these product-type polynomials, which contain terms up to degree $kp$ where $k$ is the number of dimensions and $p$ is the maximal degree of the 1-D polynomial, as opposed to using a polynomial basis which merely spans the degree-$p$ polynomial space $P_p$? Obviously $P_p$ is a subspace of the product polynomial space, but it seems to me that the product polynomials just add degrees of freedom while adding bits and pieces of higher-degree polynomial spaces, like a kind of crappy low-pass filter. I'd appreciate your thoughts and good references to relevant reading material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: To clarify, consider a smooth function $f(x,y)$ projected into the tensor product polynomial space $P_3 \otimes P_3$. If I evaluate the projection $\tilde{f}(x,y)$ along a line parallel to the $x$-axis or $y$-axis the oscillation along that line is cubic, whereas if I evaluate it on a line at 45 degrees from the $x$ axis, i.e. $y(x)=x$, the oscillation is of degree 6 due to the $x^3y^3$ term. What are the ramifications of selectively including this oscillation but excluding other terms of the same combined degree, e.g. $x^4y^2$? Can we obtain a more accurate projection of $f$ in a space $P_k \otimes P_k$ with fewer degrees of freedom than in a polynomial space with elements $x^my^n, m+n \leq k'$, where $k' \approx k$?&lt;/p&gt;&#xA;" OwnerUserId="7241" LastEditorUserId="7241" LastEditDate="2018-07-24T23:03:15.427" LastActivityDate="2018-07-27T14:32:58.123" Title="Polynomial approximation spaces" Tags="|interpolation|error-estimation|polynomials|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29904" PostTypeId="1" CreationDate="2018-07-23T04:13:13.700" Score="1" ViewCount="2907" Body="&lt;p&gt;I am looking for a simple usage example/demo of OpenFOAM + CUDA and would like to understand how exactly OpenFOAM benefits from CUDA. The thing is I do not have any background in &lt;em&gt;fluid dynamics&lt;/em&gt; and so finding it hard to get a high-level understanding of the matter. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Appreciate if you can point me to a relevant link. I think a kind of &quot;Hello World&quot; would be sufficient.&lt;/p&gt;&#xA;" OwnerUserId="4932" LastEditorUserId="20688" LastEditDate="2018-07-25T13:34:30.070" LastActivityDate="2018-07-25T13:34:30.070" Title="Demo example for OpenFOAM with CUDA" Tags="|fluid-dynamics|gpu|openfoam|cuda|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29905" PostTypeId="2" ParentId="29881" CreationDate="2018-07-23T04:16:40.313" Score="3" Body="&lt;p&gt;I was once charged with solving this problem, in another setting. Here's what I did:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I found conservation laws. These are pure gold, as is well known to physicists. Conservation of energy is present in many FE settings, but conservation laws abound in almost all physics models. If you can find a conservation law and compute the associated conserved quantity at (say) each time step, then you have vastly higher confidence in the associated solution. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I manufactured solutions, as discussed in the other answer, and compared the numerical result to in in $L_{1}$, $L_2$ and $L_{\infty}$ norms (I like googletest for this purpose, but there are many good frameworks for the task). Of course these are never zero, but you can set some reasonable bound and when the tests break (and they will) you will have three good measures of whether that breakage is intolerable or not.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For linear models, you have superposition and scale invariance. So you don't even need an analytic solution, you can just rescale your rhs and test that the new solution matches your scaled solution. You'd think this isn't a powerful test, but guess what . . . (Some will argue that this is the same as my first point.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I worked to make sure the code compiled on clang, not just MSVC. This allowed me to use AddressSanitizer, UndefinedBehaviorSanitizer, and ThreadSanitizer. These caught many memory leaks and race conditions, and provided output that allowed me to fix the associated problems without undue effort.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I made the case that the output files which were being compared were forcing scientists to write parsers, which we know is exceptionally difficult. Once management became cognizant that most of the unit test failures were parser errors rather than code errors, my mandate and scope was expanded, and I had more authority to overhaul the code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2018-07-23T04:16:40.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29906" PostTypeId="2" ParentId="29890" CreationDate="2018-07-23T08:16:13.177" Score="1" Body="&lt;p&gt;Since Laplacian is an elliptic operator you are looking for the Cholesky decomposition of the assumed constant diffusion matrix $D$:&#xA;$$D = LL^T$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore the parabolic equation may be written as:&#xA;$$\partial_t u=-\nabla^T(D\nabla u)=-(L^T\nabla)^T(L^T\nabla u)=-\tilde{\nabla}^T(\tilde{\nabla} u)=-\tilde{\nabla}^2u$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naming $\tilde{\nabla}=L^T\nabla$ and therefore $\tilde{x}=L^{-1}x$. This last PDE is totally decoupled (using Fourier transform and separation of variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution for the unnormalised heat kernel will be (using $\tilde{x}=L^{-1}x$):&#xA;$$\Phi=\prod_{n=1}^d\frac{\exp{(-\tilde{x}_n^2/(4t))}}{\sqrt{4\pi t}}=\frac{\exp{(-\tilde{x}^T\tilde{x}/(4t))}}{(4\pi t)^{d/2}}=\frac{\exp{(-x^TD^{-1}x/(4t))}}{(4\pi t)^{d/2}}$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-07-23T08:43:52.563" LastActivityDate="2018-07-23T08:43:52.563" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29907" PostTypeId="2" ParentId="29904" CreationDate="2018-07-23T11:49:04.130" Score="2" Body="&lt;p&gt;I suggest to start form &lt;a href=&quot;https://openfoamwiki.net/index.php/GPGPU&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;, where you can find different project related to OpenFoam that use GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that OpenFoam was born with the intent to resolve practical case in applied problems, think its typical user as an engineer. In this kind of program is normal to develop a layer that hide the implementation, so the user principal work is to set options. In fact if you try OpenFoam essentially you set option: describe the mesh proprieties, choose solver, ecc.. in its syntax, but it is not a real programming task as you do in normal language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine OpenFoam as a front-end and the implementation, over cpu or gpu, as a back-end. So when you use a gpu the goal is to not note difference from the front-end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another consideration is that in this kind of program is important to know what are you doing, i.e. know the software that has got an entry level barrier, but more important know physics and CFD. For example there are a lot of solver developed along the years and every solver is for a specific case. In addition, to use it with awareness it would be important to know the &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_volume_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;finite volume method&lt;/a&gt; that it is what OpenFoam uses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end I report the link to &lt;a href=&quot;https://github.com/Atizar/RapidCFD-dev&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rapid CFD&lt;/a&gt; a gpu version of OpenFoam, so you can navigate into the code.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2018-07-23T11:49:04.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29909" PostTypeId="2" ParentId="29903" CreationDate="2018-07-23T14:13:54.680" Score="2" Body="&lt;p&gt;Let me answer you using a more general concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We seek a space of functions (not only polynomials) in which a function may be described. This space of functions $V$ is defined by a basis. The number of elements $d$ within this basis coincides with the dimension of the space of functions $V$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let these functions in one dimension be $\phi_i$ and therefore $V_1=\textrm{span}\left\{\phi_i\right\}_{i=1}^{d}$. As an example, in the one-dimensional case, the monomials $1,x,x^2,...$ will form a basis of $V$:&#xA;$$V_x=\textrm{span}\left\{1,x,x^2,...\right\}$$&#xA;On the other hand, the functions $1,\cos{(x)},\cos{(2x)},...$ form another basis:&#xA;$$V_{\sin{x}}=\textrm{span}\left\{1,\cos{(x)},\cos{(2x)},...\right\}$$&#xA;with the particularity that its elements are orthogonal to each other within $[0,2\pi]$ and therefore it has better properties than the polynomial basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now it is clear that we can form any basis once we now we have distict elements, $\textit{i.e.}$ our basis is linearly independent. For example, the basis given by $\textrm{span}\left\{1,\cos{(\pm x)}\right\}$&#xA;will have a dimension of $d=2$ despite it having 3 elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine now we move to two-dimensional space, in which we must form a two-dimensional basis. We know that if, for example, the basis $V_x$ defines a complete basis in 1D, the tensor product space $V_{xy}=V_x\otimes V_y$ will also form a complete basis in 2D. Formally the two-dimensional space $V_2=\textrm{span}\left\{\phi_i(x)\phi_k(y)\right\}_{i,k=1}^{d}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore we only need two complete one-dimensional function spaces to form one complete two-dimensional space. It is natural to form the function spaces:&#xA;$$V_{xy} = \textrm{span}\left\{1,x,y,xy,x^2,x^2y,...\right\}$$&#xA;or&#xA;$$V_{\cos{x}\cos{y}} = \textrm{span}\left\{1,\cos{(x)},\cos{(y)},\cos{(x)}\cos{(y)},\cos{(x)}^2,\cos{(x)}^2\cos{(y)},...\right\}$$&#xA;But no one has forbidden us to build spaces of the kind&#xA;$$V_{x\cos{y}}= \textrm{span}\left\{1,x,\cos{(y)},x\cos{(y)},x^2,x^2\cos{(y)},...\right\}$$&#xA;But the nice properties that $V_{\cos{x}}$ had are not present any more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final remark, you can also form a basis from a linear combination of its elements. If the initial monomial basis is given by:&#xA;$$V_{xy}=\textrm{span}\left\{1,x,y,xy\right\}$$&#xA;you can also form variants of this basis from basic operations among its elements:&#xA;$$V_{xy}'=\textrm{span}\left\{1+x+y,x(1+y),y(1-x),y\right\}$$&#xA;As @njuffa pointed out, there are special linear combinations of these basis that lead to orthogonal polynomials. For example if one forces that $(\phi_i,\phi_k)=\delta_{ik}$ (by means for example Gramm-Smith) for the monomial basis one will obtain the Legendre polynomials, which are orthogonal to each other in the considered interval (usually $[-1,1]$ or $[0,1]$) with respect to the unit weighing function. What is more, if one restrict the values of $x=\cos^{-1}{(y)}\in[-1,1]$ in the basis $V_{\cos{x}}$ one obtains the Tschebycheff polynomials:&#xA;$$V_{T}=\textrm{span}\left\{1,\cos{(\cos^{-1}{(y)})},\cos{(2\cos^{-1}{(y)})},...\right\}=\textrm{span}\left\{1,y,2y^2-1,...\right\}$$&#xA;Which are orthogonal in the interval $[-1,1]$ to each other with respect to the weighing function that appears after a change of variables ($x=\cos^{-1}{(y)}$) in the following integral:&#xA;$$\int_{-\pi}^{\pi}\cos{(nx)}\cos{(mx)}\,dx=\pi\delta_{nm}$$&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-07-24T06:40:00.233" LastActivityDate="2018-07-24T06:40:00.233" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29910" PostTypeId="1" CreationDate="2018-07-23T17:56:40.930" Score="4" ViewCount="97" Body="&lt;p&gt;I am trying to minimize a function where my initial guess is quite close to the minimum. &#xA;I'm trying to minimize &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(q) = \text{angle}(qw_1q*, v_1) + \text{angle}(qw_2q*, v_2) + \text{angle}(qw_3q*, v_3)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $q$ being a quaternion and it's conjugate $q*$ and $w$ and $v$ being vectors. What algorithms are there that can find my minimum without requiring expensive derivatives? Speed is my biggest concern.&lt;/p&gt;&#xA;" OwnerUserId="28275" LastEditorUserId="28275" LastEditDate="2018-07-24T14:55:57.293" LastActivityDate="2018-07-24T14:55:57.293" Title="What Derivative-free optimization method should I use when my initial guess is very good?" Tags="|optimization|algorithms|computational-geometry|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29911" PostTypeId="2" ParentId="29867" CreationDate="2018-07-23T18:17:17.627" Score="1" Body="&lt;p&gt;You could create a support vector machine (SVM) using python's &lt;code&gt;scikit-learn&lt;/code&gt; library. Specifically, support vector regression (SVR) might work well if you assigned all your non-numerical values to a numerical value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could then &quot;train&quot; the model using the 45 inspection results. Then, for the remainder of your 3500 pipes, you could predict their condition. Once you have a prediction of their condition, then you could use a genetic algorithm or some other global optimization technique (I like to use python's &lt;code&gt;scipy.optimize&lt;/code&gt; library with differential evolution for these types of things) to find the optimal replacement strategy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't read through the entirety of your paper, but it looks like you just need an objective function to apply the method to. The SVR technique works pretty well for creating an objective function.&lt;/p&gt;&#xA;" OwnerUserId="28276" LastEditorUserId="20688" LastEditDate="2018-07-23T18:48:46.470" LastActivityDate="2018-07-23T18:48:46.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29913" PostTypeId="1" CreationDate="2018-07-23T22:51:52.147" Score="0" ViewCount="985" Body="&lt;p&gt;I'm trying to write a program in python which doesn't need to use extra packages like numpy and scipy. In one part of the project, if I can interpolate a function to a set of data, I can save processing time. Is there a way to fit a function to a set of data with built-in functions of python only?&lt;/p&gt;&#xA;" OwnerUserId="27358" LastActivityDate="2018-07-24T08:18:40.027" Title="Python - Fitting a function to data without using Scipy" Tags="|python|interpolation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29914" PostTypeId="1" AcceptedAnswerId="29919" CreationDate="2018-07-24T00:26:05.553" Score="6" ViewCount="683" Body="&lt;p&gt;I would like to solve a linear system of the form $ABx=b$ in parallel, where $A$ and $B$ are large, sparse matrices. Currently, I am forming the system matrix $AB$ explicitly, but I would like to avoid the expensive sparse matrix-matrix product. I can think of two alternatives, but I have questions about each of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I could implement the mapping $x\mapsto ABx$ as a sequence of two sparse matrix-vector products and then apply a Krylov method. While this is easy to set up, I am worried that out-of-the-box preconditioners like (S)SOR or ILU preconditioners may no longer work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, I could solve the system $ABx=b$ by first solving $Ay=b$ followed by $Bx=y$, since both matrices are square. However, I am not sure how to choose the tolerances in this case. It seems that any tolerance $\varepsilon$ for the original system can be achieved by choosing tolerances $\varepsilon_1$ for the system $Ay=b$ and $\varepsilon_2$ for the system $Bx=y$ such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\|b-ABx\| &amp;amp;=\|b-Ay+Ay-ABx\|\\&#xA;&amp;amp;\le\|b-Ax\|+\|A\|\|y-Bx\|\\&#xA;&amp;amp;&amp;lt; \varepsilon_1+\|A\|\varepsilon_2\\&#xA;&amp;amp;&amp;lt; \varepsilon,&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but this does not seem practical, because $\|A\|$ is unknown, and $\varepsilon_2$ would likely be very small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are you aware of any standard methods to solve this kind of linear system?&lt;/p&gt;&#xA;" OwnerUserId="28277" LastEditorUserId="9667" LastEditDate="2018-07-24T01:27:06.403" LastActivityDate="2018-07-24T10:28:11.263" Title="Solving linear system of the form $ABx=b$" Tags="|linear-solver|parallel-computing|sparse-matrix|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29915" PostTypeId="2" ParentId="29867" CreationDate="2018-07-24T00:43:24.453" Score="1" Body="&lt;p&gt;I'm not sure why you would use a genetic algorithm here. It sounds to me as though you want to predict which pipes will break by training a model based on the inspection of the 45 pipes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're just getting started on this sort of thing, random forests might be a good way to go. The theory behind them is pretty easy to understand and they are relatively easy to interpret versus other models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find an introduction &lt;a href=&quot;https://medium.com/@williamkoehrsen/random-forest-simple-explanation-377895a60d2d&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and a step-by-step application of random forests to some data from the Titanic &lt;a href=&quot;https://www.kaggle.com/sshadylov/titanic-solution-using-random-forest-classifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In many ways genetic algorithms are a last resort for solving optimization problems when there is no good theory you can leverage to get something with better/faster convergence. If you're skipping straight to a GA, you're probably not thinking hard enough about your problem.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastEditorUserId="17088" LastEditDate="2018-09-23T22:07:53.267" LastActivityDate="2018-09-23T22:07:53.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29916" PostTypeId="1" AcceptedAnswerId="29917" CreationDate="2018-07-24T06:58:10.543" Score="1" ViewCount="317" Body="&lt;p&gt;I am trying to run my matlab script on a Xeon Phi node with 68 physical cores. Using &lt;em&gt;parpool&lt;/em&gt; I allocate 16 cores to enable SMD parallelization with a &lt;em&gt;parfor&lt;/em&gt; loop. The script looks something like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;parpool('local',16)&#xA;parfor i=1:N&#xA;    foo&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run the script on Intel Xeon Phi processor (with MIC architecture), each MATLAB task uses only 6.5% CPU. There are no issues when I run on another machine with Intel Xeon processor (20 cores).&#xA;How can I get the maximum out of my computing resources?&lt;/p&gt;&#xA;" OwnerUserId="19254" LastActivityDate="2018-09-22T10:01:32.777" Title="Matlab parpool on Intel Xeon-Phi (KNL)" Tags="|matlab|parallel-computing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29917" PostTypeId="2" ParentId="29916" CreationDate="2018-07-24T08:01:49.327" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.mathworks.com/help/distcomp/parpool.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;parpool&lt;/code&gt;&lt;/a&gt; is a part of &lt;a href=&quot;https://www.mathworks.com/help/distcomp/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matlab Parallel Computing Toolbox&lt;/a&gt;. According to &lt;a href=&quot;https://www.mathworks.com/matlabcentral/answers/243220-gpu-or-intel-xenon-phi-coprocessor&quot; rel=&quot;nofollow noreferrer&quot;&gt;this discussion from 2015&lt;/a&gt; and &lt;a href=&quot;https://www.mathworks.com/matlabcentral/answers/293911-is-intel-xeon-phi-hardware-supported-by-the-parallel-computing-toolbox-in-the-last-releases-matlab&quot; rel=&quot;nofollow noreferrer&quot;&gt;this discussion about Matlab2016a&lt;/a&gt;, Parallel Computing Toolbox did not have a support for Intel Xeon Phi. However, at that time only Intel Xeon Phis on KNC architecture were available (that allowed only co-processor mode of operation).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also was not able to find mentioning of Intel Xeon Phi in the current (R2018a) documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And while KNL architecture is a huge step towards &quot;a standalone CPU&quot; (as compared to KNC), I guess Parallel Computing Toolbox might still have troubles with its full usage.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-07-24T08:01:49.327" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29918" PostTypeId="2" ParentId="29913" CreationDate="2018-07-24T08:18:40.027" Score="2" Body="&lt;p&gt;Yes a way there is. You must implement a interpolation what you do by hand when interpolate. There are different method, for example &lt;a href=&quot;https://en.wikipedia.org/wiki/Lagrange_polynomial&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lagrangian interpolation&lt;/a&gt; or &lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/barycentric.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barycentric Lagrange Interpolation&lt;/a&gt;. This is an argument cover in all the books of numerical analysis for university level. Oblivious you will use the pit hon data structures and so on, but is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I agree with @AntonMenshov, I  do not think it is the best way. To be honest install numpy and scipy are these two lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install numpy&#xA;pip install scipy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Eventually I suggest to consider to write an your package that hide&#xA;numpy and scipy and the interface/api for the final user is oriented to&#xA;your specific case. Think your package as a sort of &lt;a href=&quot;https://en.wikipedia.org/wiki/Domain-specific_language&quot; rel=&quot;nofollow noreferrer&quot;&gt;Domain Specific Language&lt;/a&gt;. In this way your user have got an api more near they needs, but with a strong (and more tested) back-end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you built your package you can indicate the dependency, you can find the python documentation, so the you lunch &lt;em&gt;pip install my_package&lt;/em&gt; it will install the dependency indicated.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2018-07-24T08:18:40.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29919" PostTypeId="2" ParentId="29914" CreationDate="2018-07-24T09:58:47.900" Score="12" Body="&lt;p&gt;Defining the auxiliary variable $y=Bx$ yields the following algebraically equivalent expanded system,&#xA;$$\underbrace{\begin{bmatrix}&#xA;0 &amp;amp; A \\ B &amp;amp; -I&#xA;\end{bmatrix}}_{K}&#xA;\underbrace{\begin{bmatrix}&#xA;x \\ y&#xA;\end{bmatrix}}_{u} =&#xA;\underbrace{\begin{bmatrix}&#xA;b \\ 0&#xA;\end{bmatrix}}_{f},$$&#xA;which you could solve with GMRES or another nonsymmetric Krylov method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To build a preconditioner, notice that&#xA;$$\begin{bmatrix}A^{-1} &amp;amp; 0\\ 0 &amp;amp; I\end{bmatrix}\begin{bmatrix}&#xA;0 &amp;amp; A \\ B &amp;amp; -I&#xA;\end{bmatrix} \begin{bmatrix}B^{-1} &amp;amp; 0 \\ 0 &amp;amp; I\end{bmatrix}= \begin{bmatrix}0 &amp;amp; I \\ I &amp;amp; -I\end{bmatrix},$$&#xA;which is extremely well-conditioned (condition number: $\phi^2\approx 2.618$). So if you have good preconditioners $\widetilde{A}$ and $\widetilde{B}$ for $A$ and $B$, you can create good left and right preconditioners for the expanded system as follows:&#xA;$$P_L:=\begin{bmatrix}\widetilde{A} &amp;amp; 0 \\ 0 &amp;amp; I\end{bmatrix}, \quad\quad P_R:=\begin{bmatrix}\widetilde{B} &amp;amp; 0 \\ 0 &amp;amp; I\end{bmatrix}.$$&#xA;Then you solve the preconditioned system:&#xA;$$Ku=f \quad \Leftrightarrow \quad (P_L^{-1} K P_R^{-1})P_R u = P_L^{-1} f.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you can proceed in the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve $P_L w = f$ for $w$. (apply inverse of left preconditioner)&lt;/li&gt;&#xA;&lt;li&gt;Solve $(P_L^{-1} K P_R^{-1})v = w$ for $v$. (GMRES)&lt;/li&gt;&#xA;&lt;li&gt;Solve $P_R u = v$ for $u$. (apply inverse of right preconditioner)&lt;/li&gt;&#xA;&lt;li&gt;Extract first component of $u$ to get $x$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1502" LastEditorUserId="1502" LastEditDate="2018-07-24T10:28:11.263" LastActivityDate="2018-07-24T10:28:11.263" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29921" PostTypeId="2" ParentId="29902" CreationDate="2018-07-24T12:58:51.513" Score="4" Body="&lt;p&gt;Generally speaking, the force acting on particle $i$ is just $-\nabla E(\vec{r}_i)$ (note the minus sign), where $\nabla$ is the gradient operator and $\vec{r}_i$ is the position of particle $i$. However, note that this expression assumes that we are dealing with &lt;em&gt;conservative&lt;/em&gt; forces. But if that's the case, the energy is conserved &lt;em&gt;by definition&lt;/em&gt;, which means that the forces &lt;em&gt;must&lt;/em&gt; average to 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, to answer your question, I guess you could try to numerically estimate the force acting on each particle by displacing it along each axis, recomputing the energy and taking the numerical derivative by calculating &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F_\alpha \approx - \frac{E(\vec{r}_i + \delta \vec{\alpha}) - E(\vec{r}_i)}{\delta \alpha}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha = x$, $y$ or $z$. You should first do this for several values of $\delta$ in order to choose the optimal value and then compute the force acting on &lt;em&gt;each&lt;/em&gt; particle for a given configuration.&lt;/p&gt;&#xA;" OwnerUserId="27040" LastActivityDate="2018-07-24T12:58:51.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29924" PostTypeId="2" ParentId="29904" CreationDate="2018-07-25T07:02:28.727" Score="0" Body="&lt;p&gt;I would prefer to re-write a finite volume code directly using a library compatible with GPU. In fact, OpenFOAM has been written in C++ and optimized for CPU parallel usage. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, it is better either to use an existing finite volume code optimized for GPU usage or to re-write one from zero while paying attention to the possible optimizations.&lt;/p&gt;&#xA;" OwnerUserId="27080" LastEditorUserId="20688" LastEditDate="2018-07-25T13:31:28.303" LastActivityDate="2018-07-25T13:31:28.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29925" PostTypeId="1" CreationDate="2018-07-25T07:18:53.080" Score="2" ViewCount="274" Body="&lt;p&gt;For spatial accuracy in 2-D Laplace equation, a 9-point stencil is better than a 5-point one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\partial_tq= r\left(\partial^2_x q + \partial^2_y q\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for FTCS (forward-time, central-space) scheme to converge for 5 point stencil, $r/2\ge \delta t/ \delta x^2$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the condition for 9 point stencil?&lt;/p&gt;&#xA;" OwnerUserId="28289" LastEditorUserId="20688" LastEditDate="2018-07-25T13:39:39.230" LastActivityDate="2018-07-25T13:39:39.230" Title="Is there Von Neumann stability analysis for 9-point laplacian like we have for the 5-point Laplacian?" Tags="|pde|finite-difference|discretization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29926" PostTypeId="2" ParentId="29925" CreationDate="2018-07-25T10:21:23.580" Score="4" Body="&lt;p&gt;There are actually a few different 9 point stencils in use, but they can all be written as a linear combination of the standard and skewed 5 point stencils. Performing the usual von Neumann analysis for a general Fourier mode, $e^{ikx+ily}$, produces an equation like $$a_{n+1} = a_n \left(1-\lambda+\frac{\alpha\lambda}{2}\left[\cos\theta_1+\cos\theta_2\right]+\lambda\frac{1-\alpha}{2}\left[\cos(\theta_1+\theta_2)+\cos(\theta_1-\theta_2)\right]\right)$$&#xA;where $\lambda=\frac{r\Delta t}{\Delta x^2}$. Hence the stability criterion is still $\lambda\leq \frac{1}{2}$, with the derivation and proof left as an exercise to the reader. This shouldn't be too much of a surprise, since for the &quot;worst case&quot; of a plane wave aligned with the coordinate system, both the 5 point and 9 point stencils collapse back down to the 1D case.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="19263" LastEditDate="2018-07-25T10:37:46.457" LastActivityDate="2018-07-25T10:37:46.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29927" PostTypeId="1" CreationDate="2018-07-25T13:28:39.997" Score="1" ViewCount="98" Body="&lt;p&gt;I'm looking for some guidance on how to write a program to numerically solve a PDE. As an example for comparison in 1D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{d^2u}{dx^2} = f\;\;\;\;u(0) = 0\;\;\;\;u(1) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We could try 6 equally spaced nodes between 0 and 1, interpolate within each element with a quadratic, and ensure continuity of $u$ and $du/dx$ at the boundaries, and to solve within each element we can integrate the differential equation over each element. I believe this constant-weight integration makes this FVM. Is that correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, there were 3 DOFs for each of the 5 elements, 1 integral equation per element, 2 compatibility conditions for each of the 4 internal boundaries, and 2 boundary conditions given as part of the problem. So that's 15 equations in 15 variables which can then be solved. When trying to make the equivalent 2D method, I cannot work out how to generate the right number of compatibility equations. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\nabla^2{u} = f\;\;\;\;u(0,y) = u(1,y) = u(x,0) = u(x,1) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine we mesh with a grid of 5 by 5 squares with each square divided into two triangles. Each triangular element can be represented with a quadratic in $x$ and $y$ (6 DOFs). The DE is integrated over for each element. Continuity of $u$ and $\nabla{u}$ could be ensured at the boundaries, though already this might be over/under-constrained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 50 elements, 65 internal boundaries and 20 external boundaries. There are 300 DOFs. Compatibility of $u$ and $\nabla{u}$ on a single internal boundary yields 5 equations, but either the sets of 5 equations per internal boundary are not independent of each other or the system is over-constrained because already there are too many equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, representing the DOFs with the value of $u$ at the corners and edge midpoints of the triangles is a way of intrinsically satisfying the continuity of $u$ condition and reducing the number of DOFs, but it does not help intrinsically satisfy the other compatibility conditions. But I mention this in case proper representation of the DOFs is key to helping understand the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am thinking about this in roughly the right way, how do I determine the compatibility conditions and generate the right number of equations to solve for the given number of DOFs? If I'm not thinking about the problem in the right way but you can help me funnel this into a more specific, generally-relevant question(s), please do.&lt;/p&gt;&#xA;" OwnerUserId="28292" LastActivityDate="2018-07-25T13:28:39.997" Title="Basic approach for numerical solution of PDE" Tags="|finite-element|pde|finite-volume|numerics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29928" PostTypeId="1" CreationDate="2018-07-25T19:36:04.967" Score="2" ViewCount="602" Body="&lt;p&gt;I want to solve the Poisson PDE for heat flow in a 3-D solid cube with given dimensions $x$, $y$, and $z$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\rho C\frac{\partial T}{\partial t} = k \Delta T$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cube is irradiated with a constant heat flux $I(x,y)$ at the $z=0$ surface. The boundaries should meet the Neumann condition for radiative cooling given by $\frac{\partial T}{\partial x_i} = \pm\epsilon\sigma(T^4-300^4)$, where the sign depends on wether you look at $x_i=$max or $x_i=0$. Due to the irradiation the $z=0$ surface is an exception, where the condition should rather be $\frac{\partial T}{\partial x_i} = \epsilon\sigma(T^4-300^4) + I(x,y)$. The last term represents the intensity distribution of the radiative flux. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I decided to go for a numerical approach I found in &lt;a href=&quot;https://stackoverflow.com/questions/49463985/solving-heat-equation-with-python-numpy&quot;&gt;this&lt;/a&gt; thread using Python. As a result, I came up with the following piece of code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from mpl_toolkits import mplotn3d&#xA;&#xA;dt = 3&#xA;di = 0.01&#xA;&#xA;#Thermal conductivity&#xA;k = 1.38&#xA;#Density&#xA;rho = 2202&#xA;#Specific Heat capacity&#xA;cp = 745&#xA;#Thermal diffusivity&#xA;alpha = k/(rho*cp)&#xA;#Emissivity&#xA;emissivity = 0.79&#xA;&#xA;sigma = 5.670367e-8&#xA;&#xA;y_max = 0.3&#xA;x_max = 0.3&#xA;z_max = 0.3&#xA;t_max = 60&#xA;&#xA;P = 1&#xA;omega = 0.02&#xA;&#xA;#Function that generates a gaussian intensity pattern I(x,y) with power P and beam waist omega&#xA;&#xA;def get_beam(P, omega, di, x_max, ymax, z_max):&#xA;  x = np.arange(0,x_max+di,di)&#xA;  y = np.arange(0,y_max+di,di)&#xA;  z = np.arange(0,z_max+di,di)&#xA;  t = np.arange(0,t_max+dt,dt)&#xA;  r = len(t)&#xA;  cx = len(x)&#xA;  cy = len(y)&#xA;  cz = len(z)&#xA;  I = np.zeros([r, cx, cy, cz])&#xA;  for jx in range(0,cx-1):&#xA;      for jy in range(0,cy-1):&#xA;          I[:,jx,jy,0] = 2*P/(np.pi*omega**2)*np.exp(-2*(abs(di*(jx-cx/2))**2 + abs(di*(jy-cy/2))**2)/(omega**2))&#xA;  return I&#xA;&#xA;def FTCS(dt,dy,t_max,x_max,y_max,z_max,k,T0, I):&#xA;  s = alpha*dt/di**2&#xA;  x = np.arange(0,x_max+di,di)&#xA;  y = np.arange(0,y_max+di,di)&#xA;  z = np.arange(0,z_max+di,di)&#xA;  t = np.arange(0,t_max+dt,dt)&#xA;  r = len(t)&#xA;  cx = len(x)&#xA;  cy = len(y)&#xA;  cz = len(z)&#xA;  #Initialize mesh&#xA;  T = np.ones([r,cx, cy, cz])*T0&#xA;  #time loop&#xA;  for n in range(0,r-1):  &#xA;      print(n)&#xA;      #y loop&#xA;      for jy in range(0,cy-1):   &#xA;          #x loop &#xA;          for jx in range(0,cx-1):   &#xA;              #z loop &#xA;              for jz in range(0,cz-1):  &#xA;                  #Compute T at next time step: Tn+1 = Tn + d^2T/dx^2 + d^2T/dy^2 + d^2T/dz^2  &#xA;                  T[n+1,jx,jy,jz] = T[n,jx,jy,jz] + s*(T[n,jx-1,jy,jz] - 2*T[n,jx,jy,jz] + T[n,jx+1,jy,jz]) + s*(T[n,jx,jy-1,jz] - 2*T[n,jx,jy,jz] + T[n,jx,jy+1,jz]) + s*(T[n,jx,jy,jz-1] - 2*T[n,jx,jy,jz] + T[n,jx,jy,jz+1]) &#xA;&#xA;&#xA;      #Boundary Condition&#xA;      #Set boundary condition for x on both sides   &#xA;      T[n+1,cx-1,:,:] = T[n,cx-1,:,:] + s*(T[n,cx-2,:,:] - 2*T[n,cx-1,:,:] + T[n,cx-2,:,:] - 2*di*emissivity*sigma/k*(T[n,cx-1, :,:]**4-300**4))#- 2*di*4*emissivity*sigma/k*(T[n,cx-1,:,:]*300**3))&#xA;      T[n+1,0,:,:] = T[n,0,:,:] + s*(T[n,1,:,:] - 2*T[n,0,:,:] + T[n,1,:,:] - 2*di*emissivity*sigma/k*(T[n,0,:,:]**4-300**4))#- 2*di*4*emissivity*sigma/k*(T[n,0,:,:]*300**3))&#xA;&#xA;      #Set boundary condition for y on both sides &#xA;      T[n+1,:,cy-1,:] = T[n,:,cy-1,:] + s*(T[n,:,cy-2,:] - 2*T[n,:,cy-1,:] + T[n,:,cy-2,:] - 2*di*emissivity*sigma/k*(T[n,:,cy-1,:]**4-300**4))#- 2*di*4*emissivity*sigma/k*(T[n,:,cy-1,:]*300**3))&#xA;      T[n+1,:,0,:] = T[n,:,0,:] + s*(T[n,:,1,:] - 2*T[n,:,0,:] + T[n,:,1,:] - 2*di*emissivity*sigma/k*(T[n,:,0,:]**4-300**4))#- 2*di*4*emissivity*sigma/k*(T[n,:,0,:]*300**3))&#xA;&#xA;      #Set boundary condition for z on both sides &#xA;      T[n+1,:,:,cz-1] = T[n,:,:,cz-1] + s*(T[n,:,:,cz-2] - 2*T[n,:,:,cz-1] + T[n,:,:,cz-2] - 2*di*emissivity*sigma/k*(T[n,:,:,cz-1]**4-300**4))#- 2*di*4*emissivity*sigma/k*(T[n,:,:,cz-1]*300**3))&#xA;      T[n+1,:,:,0] = T[n,:,:,0] + s*(T[n,:,:,1] - 2*T[n,:,:,0] + T[n,:,:,1] - 2*di*emissivity*sigma/k*(T[n,:,:,0]**4-300**4) + 2*di/k*I[n,:,:,0])#- 2*di*4*emissivity*sigma/k*(T[n,:,:,0]*300**3) + 2*di*I[n,:,:,0])&#xA;&#xA;  return x,y,z,t,T,r,s &#xA;&#xA;I = get_beam(P, omega, di, x_max, y_max, z_max)&#xA;x,y,z,t,T,r,s = FTCS(dt,di,t_max,x_max,y_max,z_max,k,300,I)&#xA;&#xA;&#xA;fig = plt.figure()&#xA;X, Y = np.meshgrid(x, y)&#xA;ax = plt.axes(projection='3d')&#xA;ax.plot_surface(X, Y, T[-1,:,:,0])&#xA;ax.set_xlabel('x-axis')&#xA;ax.set_ylabel('y-axis')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result looks promising but is not equal to &lt;a href=&quot;https://hal.archives-ouvertes.fr/jpa-00212444/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;references&lt;/a&gt; I found. Furthermore, the result scales rapidly with the mesh size $d_i$ and crashes for huge point densities. I couldn't figure out whether and if so where I did a mistake. Can you help me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;//-----------------------------------//&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I updated the code to include the correct boundaries. The loop looks like this now. I know its a bit clunky, but I thought I take care of it later. Nevertheless, the dependence from the mesh size is still there. I am afraid that I just forgot a $di$ somewhere, but I couldn't find out where.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    for n in range(0,r-1):&#xA;    print(n)&#xA;    for jy in range(0,cy-1):&#xA;        for jx in range(0,cx-1):&#xA;            for jz in range(0,cz-1):&#xA;                T[n+1,jx,jy,jz] = T[n,jx,jy,jz] + s*(T[n,jx-1,jy,jz] - 2*T[n,jx,jy,jz] + T[n,jx+1,jy,jz]) + s*(T[n,jx,jy-1,jz] - 2*T[n,jx,jy,jz] + T[n,jx,jy+1,jz]) + s*(T[n,jx,jy,jz-1] - 2*T[n,jx,jy,jz] + T[n,jx,jy,jz+1]) &#xA;&#xA;                if jx == cx-1:&#xA;                    T[n+1,jx,jy,jz] = T[n,jx,jy,jz] + s*(T[n,jx-1,jy,jz] - 2*T[n,jx,jy,jz] + T[n,jx-1,jy,jz] + 2*di*emissivity*sigma/k*(T[n,jx, jy, jz]**4-300**4)) + s*(T[n,jx,jy-1,jz] - 2*T[n,jx,jy,jz] + T[n,jx,jy+1,jz]) + s*(T[n,jx,jy,jz-1] - 2*T[n,jx,jy,jz] + T[n,jx,jy,jz+1])&#xA;                elif jx == 0:&#xA;                    T[n+1,jx,jy,jz] = T[n,jx,jy,jz] + s*(T[n,jx-1,jy,jz] - 2*T[n,jx,jy,jz] + T[n,jx-1,jy,jz] - 2*di*emissivity*sigma/k*(T[n,jx, jy, jz]**4-300**4)) + s*(T[n,jx,jy-1,jz] - 2*T[n,jx,jy,jz] + T[n,jx,jy+1,jz]) + s*(T[n,jx,jy,jz-1] - 2*T[n,jx,jy,jz] + T[n,jx,jy,jz+1]) &#xA;                elif jy == cy-1:&#xA;                    T[n+1,jx,jy,jz] = T[n,jx,jy,jz] + s*(T[n,jx-1,jy,jz] - 2*T[n,jx,jy,jz] + T[n,jx+1,jy,jz]) + s*(T[n,jx,jy-1,jz] - 2*T[n,jx,jy,jz] + T[n,jx,jy-1,jz] + 2*di*emissivity*sigma/k*(T[n,jx,jy,jz]**4-300**4)) + s*(T[n,jx,jy,jz-1] - 2*T[n,jx,jy,jz] + T[n,jx,jy,jz+1]) &#xA;                elif jy == 0:&#xA;                    T[n+1,jx,jy,jz] = T[n,jx,jy,jz] + s*(T[n,jx-1,jy,jz] - 2*T[n,jx,jy,jz] + T[n,jx+1,jy,jz]) + s*(T[n,jx,jy-1,jz] - 2*T[n,jx,jy,jz] + T[n,jx,jy-1,jz] - 2*di*emissivity*sigma/k*(T[n,jx,jy,jz]**4-300**4)) + s*(T[n,jx,jy,jz-1] - 2*T[n,jx,jy,jz] + T[n,jx,jy,jz+1]) &#xA;                elif jz == cz-1:&#xA;                    T[n+1,jx,jy,jz] = T[n,jx,jy,jz] + s*(T[n,jx-1,jy,jz] - 2*T[n,jx,jy,jz] + T[n,jx+1,jy,jz]) + s*(T[n,jx,jy-1,jz] - 2*T[n,jx,jy,jz] + T[n,jx,jy+1,jz]) + s*(T[n,jx,jy,jz-1] - 2*T[n,jx,jy,jz] + T[n,jx,jy,jz-1] + 2*di*emissivity*sigma/k*(T[n,jx,jy,jz]**4-300**4)) &#xA;                elif jz == 0:&#xA;                    T[n+1,jx,jy,jz] = T[n,jx,jy,jz] + s*(T[n,jx-1,jy,jz] - 2*T[n,jx,jy,jz] + T[n,jx+1,jy,jz]) + s*(T[n,jx,jy-1,jz] - 2*T[n,jx,jy,jz] + T[n,jx,jy+1,jz]) + s*(T[n,jx,jy,jz-1] - 2*T[n,jx,jy,jz] + T[n,jx,jy,jz-1] - 2*di*emissivity*sigma/k*(T[n,jx,jy,jz]**4-300**4) + 2*di*I[n,jx,jy,0]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28296" LastEditorUserId="28296" LastEditDate="2018-07-26T15:13:49.560" LastActivityDate="2018-10-25T11:00:42.750" Title="Solve 3-D Heat equation with Neumann boundaries" Tags="|finite-difference|python|poisson|numerics|heat-transfer|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="29931" PostTypeId="1" AcceptedAnswerId="30024" CreationDate="2018-07-26T18:50:15.180" Score="5" ViewCount="741" Body="&lt;p&gt;I'm using the Convection(/advection)-Diffusion(-Reaction) equation to calculate the temperature over time in different hydraulic parts like a pipe or a heat exchanger.&lt;br&gt;&#xA;The flow/convection is &lt;strong&gt;always 1D&lt;/strong&gt;, while the diffusion, in this case, heat conduction, &lt;strong&gt;can be 1D, 2D or 3D&lt;/strong&gt;.&lt;br&gt;&#xA;Let's consider an example fully in 1D for the sake of easiness:&lt;br&gt;&#xA;$$ \frac{\delta T}{\delta t} = \frac{\lambda}{c_p \rho}\frac{\delta^2 T}{\delta x^2} + v\frac{\delta T}{\delta x}$$&#xA;$$\alpha = \frac{\lambda}{c_p \rho}$$&#xA;where $\lambda$ is the heat conductivity, $c_p$ the specific heat capacity, $\rho$ the density and $v$ the velocity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To discretize this equation, I use the &lt;strong&gt;forward-time central-space scheme for the diffusion&lt;/strong&gt; (second order central) and the &lt;strong&gt;forward-time forward-space scheme for the convection&lt;/strong&gt; (first order upwind), as shown here:&#xA;$$\frac{T^{n+1} - T^n}{\Delta t} \approx \alpha \frac{T^n_{i-1} - 2T^n_{i} + T^n_{i+1}}{\Delta x^2} + v \frac{T^n_{i\pm1} - T^n_{i}}{\Delta x}$$&#xA;where the sign $\pm$ in $T^n_{i\pm1}$ is depending on the sign of $v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now there are two common cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$v \neq 0$ and the resulting Peclet-Number is (in most cases) $|Pe| &amp;gt; 2$&lt;/li&gt;&#xA;&lt;li&gt;$v = 0$ and the resulting Peclet-Number is thus $Pe = 0$ (and the PDE becomes fully parabolic)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Having a relatively small timestep is ok in most cases since time continuous systems like PID controllers interact with the simulation results in between each step. These, for example, control the massflows, internal heat gains etc. Thus I've been using an embedded Heun and/or RK method with adaptive stepsize control to get the explicit time results.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;From this multiple questions arise:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is using this kind of mixed discretization ok?&lt;/li&gt;&#xA;&lt;li&gt;Should I use central differencing scheme for convection for $|Pe| &amp;lt; 2$&lt;/li&gt;&#xA;&lt;li&gt;Could I implement a QUICK-scheme for the convective term to reduce numeric errors?&lt;/li&gt;&#xA;&lt;li&gt;Is implementing higher accuracy schemes for these derivative orders, as shown in this &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_coefficient&quot; rel=&quot;nofollow noreferrer&quot;&gt;table&lt;/a&gt;, useful?&lt;/li&gt;&#xA;&lt;li&gt;When having mixed kinds of &quot;parts&quot; or parts with a big differences in the  size of the grid spacing (and thus of the resulting mass/volume of the cells around each node) this results in the parts with the smallest cells never (or really slow) reaching the temperature of the inflowing 1D massflow (neglecting cell heat loss by conduction). Is this called numeric diffusion or dissipation? And can I reduce this by using other differencing schemes like QUICK?&lt;/li&gt;&#xA;&lt;li&gt;I'd like to solve a few parts with implicit schemes, Crank-Nicholson is preferred. Is Crank-Nicholson space discretization centered scheme for &lt;em&gt;both&lt;/em&gt; convection AND diffusion &lt;em&gt;or&lt;/em&gt; can I use centered scheme for diffusion and upwind/QUICK for convection?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit:&lt;/strong&gt; Is there any additional information I could provide to make it easier to answer this question?&lt;/p&gt;&#xA;" OwnerUserId="27868" LastEditorUserId="27868" LastEditDate="2018-08-07T11:06:00.710" LastActivityDate="2018-08-11T04:36:43.310" Title="Finite Differencing schemes for Convection-Diffusion equation" Tags="|pde|finite-difference|discretization|advection-diffusion|crank-nicolson|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29932" PostTypeId="1" CreationDate="2018-07-26T19:02:21.893" Score="0" ViewCount="109" Body="&lt;p&gt;I have a self-written MD code in C++. I am simulating atoms using the Langevin dynamics model. I use the linked-cell method to speed up the simulation; as a result, atomic interactions are only happening among the nearest neighboring atoms. That means, when calculating the distances between atom $i$ and the rest of the system, only those atoms within the neighboring cells are considered. This naturally affects the calculation of the radial distribution function $g_r(r)$. For example, $g_r(r&amp;gt;r_c)=0$ uniquely for some cutoff distance $r_c$ (due to a lack of interactions). This isn't good since you want to simulate all the way up to larger distances to ensure $g_r \rightarrow 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to use the cell method and calculate $g_r(r)$ correctly? If not, am I stuck with using the traditional $\mathcal{O}(N^2)$ algorithm for this calculation, in which all atomic interactions are considered? (I simulate about 1000 atoms). &lt;/p&gt;&#xA;" OwnerUserId="26366" LastActivityDate="2018-07-27T07:49:21.950" Title="How to calculate $g_r$ when using the cell method algorithm" Tags="|computational-physics|molecular-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29933" PostTypeId="2" ParentId="29932" CreationDate="2018-07-26T21:32:37.013" Score="1" Body="&lt;p&gt;I believe that you can make a small improvement, but not a dramatic one. Assume that your force loop consists of an outer loop over cells, and an inner one over nearest-neighbour cells (and then, more loops over the atoms within those cells). For the pair distribution function calculation, the inner loop can go over not just the nearest-neighbour cells, but over all cells covering a roughly spherical region out to the desired distance, relative to the cell being considered in the outer loop. You can construct a list of these cells (with indices relative to the central cell) at the start of the program. How you do this depends on how you handle the cell indexing, as you mentioned &lt;a href=&quot;https://scicomp.stackexchange.com/questions/29857/effective-way-to-build-the-neighbors-list-in-md&quot;&gt;in your earlier question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the maximum distance of interest is not as large as half the box length $L$, this can make a reasonable saving of time over the all-pairs method. However, if you want to calculate out to half the box length, the maximum saving is the ratio of the spherical volume you are looking at, $\frac{4}{3}\pi (L/2)^3$ to the volume $L^3$ that you would look at with all pairs: around 50%. Essentially, you are just avoiding looking in the &quot;corners&quot; of the periodic box surrounding a given cell. For a system of size $N=1000$, I would think that it's not worth it, and you would do best to stick to all-pairs (discarding any for which the separation is larger than $L/2$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The silver lining here is that the statistics of the calculation improve along with the extra work done. In a sense, you get what you pay for: you look at more pairs, for the higher values of $r$, so the cost goes up, but the precision of the calculation is better for large $r$. If you wish, you can offset this by doing the calculation less often (certainly not every step!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another approach, which should give you the long-range part of $g(r)$ more cheaply, but again it is probably only worth doing if you have much larger systems. You can create a 3D histogram of particle density, $\rho(\mathbf{r})$ from a snapshot of the configuration, by counting the atoms in a cubic grid of cells, but on a smaller length scale than for the force calculation. The cell size should be a fraction of the particle diameter, typically. Then perform a 3D discrete Fourier transform, using an FFT algorithm, giving the instantaneous $\hat{\rho}(\mathbf{k})$ and take the square modulus. This function of wave-vector $|\hat{\rho}(\mathbf{k})|^2$ can be averaged over snapshots taken at intervals, and eventually gives the structure factor $S(\mathbf{k})$. Inverse-Fourier-transforming this would give you $g(r)$. Again, however, I think this would be too much trouble for $N=1000$ atoms.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastEditorDisplayName="user28077" LastEditDate="2018-07-26T21:39:54.447" LastActivityDate="2018-07-26T21:39:54.447" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29934" PostTypeId="2" ParentId="29932" CreationDate="2018-07-27T07:49:21.950" Score="0" Body="&lt;p&gt;That is not how it should work. Simulations using linked-list cells (and/or Verlet lists) or $\mathcal{O}(N^2)$ calculations should (must!) give the exact same results. The only side effect should be a speed-up. You need to decouple the operations, using linked-list cells to evaluate forces and a $\mathcal{O}(N^2)$ loop to calculate the radial distribution function. This should not slow down the simulation too much, since you don't really need to update the $g(r)$ every time step. Depending on the system, doing it every $10^3$ or even $10^4$ time steps shouldn't worsen the quality of the $g(r)$ too much.&lt;/p&gt;&#xA;" OwnerUserId="27040" LastActivityDate="2018-07-27T07:49:21.950" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29935" PostTypeId="2" ParentId="29928" CreationDate="2018-07-27T09:23:56.307" Score="1" Body="&lt;p&gt;Thanks for asking your question. It really made me delve alot deeper into the topic of boundary conditions. I am still a rookie in this topic, but I'll try to answer your question anyways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your major problem seems to be that your units are not correct. Your equation for radiative heat flux has the unit $[\frac{\text{W}}{\text{m}^2}]$, while the Neumann boundary condition needs a unit of $[\frac{\text{K}}{\text{m}}]$.          Your equation for the heat flux should say:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dq}{dt} = \epsilon \sigma \left(T^4 - 300^4 \right) + I(x,y)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You almost get your units correctly by multiplying this with $di/k$, resulting in $[\text{K}]$ for the radiative Stefan-Boltzmann heat transfer, but for the heat flux based on constant heat flux, you forgot the $1/k$ while still multiplying with $di$, resulting in $[\frac{\text{W}}{\text{m}}]$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am not sure about is the factor of $2$. I guess this stems from a central difference scheme and thus shouldn't be correct here. But if anyone knows an explanation why this is correct/incorrect, I'd really like to know it.&lt;br&gt;&#xA;Furthermore I am not sure if multiplying with $s$ is correct here. I guess you need to try all cases to and compare it with the reference solution, unless someone else with more knowledge in this topic answers.&lt;/p&gt;&#xA;" OwnerUserId="27868" LastActivityDate="2018-07-27T09:23:56.307" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29936" PostTypeId="1" AcceptedAnswerId="29970" CreationDate="2018-07-27T09:48:37.183" Score="2" ViewCount="340" Body="&lt;p&gt;I have $q$ $n$-dimensional vectors $\vec y_i$ and a matrix $\hat B$ of shape $n\times m$. I'm looking for $q$ $m$-dimensional vectors $\vec x_i$ such that:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$\vec y_i=\hat B \vec x_i$ &lt;/li&gt;&#xA;&lt;li&gt;each vector $\vec x_i$ represents probability distribution (each entry is non-negative and the $L_1$ norm is 1)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Associated issues are that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;vectors $\vec y_i$ are very noisy&lt;/li&gt;&#xA;&lt;li&gt;matrix $\hat B$ can be estimated with bigger accuracy, but it may be rather sparse&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How does one approach such problem?&lt;/p&gt;&#xA;" OwnerUserId="28309" LastActivityDate="2018-08-02T06:27:14.113" Title="Finding probability vectors from an implicit equation" Tags="|matrix|probability|inverse-problem|matrix-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29937" PostTypeId="2" ParentId="29903" CreationDate="2018-07-27T14:32:58.123" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;the $x^3y^3$ term... but excluding other terms of the same combined degree, e.g. $x^4y^2$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's because for a constant $y$ the approximant must be a cubic polynomial at most. &lt;em&gt;Cubic&lt;/em&gt; was our chosen model along each dimension. Which dictates the corresponding basis for the tensor product polynomial subspace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bilinear approximation similarly must produce a line along each coordinate, not a quadric. That's why there can be a $xy$ term but not $x^2y^0$ etc.&lt;/p&gt;&#xA;" OwnerUserId="10287" LastActivityDate="2018-07-27T14:32:58.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29938" PostTypeId="1" AcceptedAnswerId="29939" CreationDate="2018-07-27T14:35:33.433" Score="5" ViewCount="440" Body="&lt;p&gt;I've got a system of several PDEs for a multitude of parts which represent real hydraulic parts like pipes or thermal energy storages. Each of these parts may have an arbitrary number of nodes and/or or dimensions.&lt;br&gt;&#xA;The PDEs of most parts are non-stiff, while a few parts tend to have quite stiff PDEs.&lt;br&gt;&#xA;Thus filling one sparse matrix with all of these parts and solving this matrix using one and the same solver, like embedded RK(4)5/Dormand-Prince or embedded Heun methods is not easily feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I solve the PDEs of each part for itself. For this example let's say using an embedded Heun method with adaptive stepsize control. Therefore each part has a function (in the simulation program) which returns its derivative for the given set of variables. This derivative is then&#xA;used to calculate the intermediate steps of the Heun method for each part separately. Values shared between parts, like temperatures of contact areas, are updated in between stages AND steps. This results in the following programm control flow with the stepsize $h$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine predictor step input variables for all parts&lt;/li&gt;&#xA;&lt;li&gt;Predictor step:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get derivative of part 1 and calculate the intermediate result of it with $y_{interm,i+1} = y_i + h*deriv_{pred}(y_i)$&lt;/li&gt;&#xA;&lt;li&gt;Do the same for part 2&lt;/li&gt;&#xA;&lt;li&gt;... until part n.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Update value array which stores the variables of contact areas&lt;/li&gt;&#xA;&lt;li&gt;Corrector step:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get derivative of part 1 and calculate the final step result of it with $y_{i+1} = y_i + h/2*(deriv_{pred}(y_i) + deriv_{corr}(y_{interm,i+1}))$&lt;/li&gt;&#xA;&lt;li&gt;... until part n.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Check if von Neumann stability was violated and if yes, reduce the stepsize $h$, reset value array for connection variables and go back to 2.&lt;/li&gt;&#xA;&lt;li&gt;Get error estimate and if error too big, reduce stepsize $h$, reset value array for connection variables and go back to 2.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This approach works really well for most parts. But as soon as there is only one stiff part out of 1000 non-stiff-parts, the stepsize $h$ may easily be reduced to about $1/100$ of the &quot;old&quot; stepsize.&lt;br&gt;&#xA;To deal with this, I thought that solving only the stiff parts with an implicit scheme may be a good approach, since there will be several hundred parts of which only 2 or three will be stiff. This would retain the fast calculation speed of the non-stiff parts as well as the &quot;big&quot; stepsize, at the cost of a higher computational cost for the stiff parts and an increased truncation error in these parts.&lt;br&gt;&#xA;The new control flow would be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine predictor step input variables for all parts&lt;/li&gt;&#xA;&lt;li&gt;Predictor step:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get derivative of part 1 and calculate the intermediate result of it with $y_{interm,i+1} = y_i + h*deriv_{pred}(y_i)$&lt;/li&gt;&#xA;&lt;li&gt;Do the same for part 2&lt;/li&gt;&#xA;&lt;li&gt;... until part n.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Get implicit PDE solution for stiff parts with half stepsize with $y_{imp,i+0.5} = y_i + h/2*deriv(y_{imp,i+0.5})$&lt;/li&gt;&#xA;&lt;li&gt;Update value array which stores the variables of contact areas&lt;/li&gt;&#xA;&lt;li&gt;Corrector step:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get derivative of part 1 and calculate the final step result of it with $y_{i+1} = y_i + h/2*(deriv_{pred}(y_i) + deriv_{corr}(y_{interm,i+1}))$&lt;/li&gt;&#xA;&lt;li&gt;... until part n.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Check if von Neumann stability was violated and if yes, reduce the stepsize $h$, reset value array for connection variables and go back to 2.&lt;/li&gt;&#xA;&lt;li&gt;Get error estimate and if error too big, reduce stepsize $h$, reset value array for connection variables and go back to 2.&lt;/li&gt;&#xA;&lt;li&gt;Get final implicit PDE solution for stiff parts with half stepsize with $y_{imp,i+1} = y_{imp,i+0.5} + h/2*deriv(y_{imp,i+1})$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In this example I used Euler implicit method, but I'd preferably integrate something like Crank-Nicolson.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Will this be ok from a numerical/mathematical point of view? Is this a good idea? Or is there any other way to do it? Any concerns?&lt;br&gt;&#xA;Thanks for your help in advance!&lt;/p&gt;&#xA;" OwnerUserId="27868" LastActivityDate="2020-07-19T17:53:06.470" Title="Solving PDE implicitly or explicitly depending on stiffness" Tags="|pde|implicit-methods|explicit-methods|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29939" PostTypeId="2" ParentId="29938" CreationDate="2018-07-27T16:32:17.413" Score="8" Body="&lt;p&gt;If you just slap together an implicit and an explicit method you will likely have order loss. You can do so with low order methods though, and &lt;a href=&quot;https://www.math.utah.edu/%7Evshankar/5620/IMEX.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Crank-Nicholson mixed with some other integrator is an easy way to get a decent second order integrator&lt;/a&gt;. Higher order IMEX integrators like &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0168927402001381&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Kennedy and Carpenter Additive Runge-Kutta methods&lt;/a&gt; or the &lt;a href=&quot;https://pdfs.semanticscholar.org/f3de/1e6ddc008fab31abd206761ca57bfa4f057c.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBDF schemes&lt;/a&gt; are specifically designed to solve &lt;span class=&quot;math-container&quot;&gt;$u' = f_{stiff} + f_{nonstiff}$&lt;/span&gt; at higher order (with adaptivity and all of that jazz). These are the kinds of methods you're looking for if you want efficiency.&lt;/p&gt;&#xA;&lt;p&gt;There are two open source implementations that I know of. The &lt;a href=&quot;https://computation.llnl.gov/projects/sundials/arkode&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sundials ARKODE&lt;/a&gt; library and the &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/split_ode_solve.html#OrdinaryDiffEq.jl-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl Split ODE Solvers&lt;/a&gt; both have these same higher order additive Runge-Kutta methods, and DifferentialEquations.jl also has the lower order ones plus the SBDF methods in development. I would recommend the DifferentialEquations.jl ones not just because I am the lead developer of the JuliaDiffEq organization (disclosure), but also because the ARKODE implementations tend to diverge without coefficient tweaks on any heavily stiff equation according to the benchmarks (for example, here's &lt;a href=&quot;http://nbviewer.jupyter.org/github/JuliaDiffEq/DiffEqBenchmarks.jl/blob/master/StiffODE/Orego.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;a test on OREGO&lt;/a&gt;, and there's &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;a lot more tests here&lt;/a&gt;. These same tweaks are shown in the &lt;a href=&quot;https://computation.llnl.gov/projects/sundials/download/arkode-2.1.1.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARKODE C++ example code&lt;/a&gt; in order to make ROBER not diverge, so even the developers know), so while it's a part of Sundials it's not as robust as the more well-known CVODE integrator.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T17:53:06.470" LastActivityDate="2020-07-19T17:53:06.470" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="29940" PostTypeId="1" AcceptedAnswerId="29966" CreationDate="2018-07-27T20:50:55.033" Score="1" ViewCount="148" Body="&lt;p&gt;My background is in applied mathematics and I'm trying to learn plasticity. I have successfully understood the theory and finite element implementation of: linear elasticity, hyperelasticity (Neo-Hookean etc.), elastic contact problems (penalty and Lagrange multiplier-type methods). Recently I have been studying plasticity and so far having a very painful experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried reading the following books:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Simo, Hughes. Computational inelasticity.&lt;/li&gt;&#xA;&lt;li&gt;Wriggers. Nonlinear finite elements.&lt;/li&gt;&#xA;&lt;li&gt;Han, Reddy. Plasticity.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm looking for a reference where one starts with the simplest possible setup, say perfect plasticity, and goes through all the steps from the equations to the Newton's linear systems, and only after that starts increasing complexity. Does anyone know good beginner references for plasticity modeling?&lt;/p&gt;&#xA;" OwnerUserId="10160" LastEditorUserId="6775" LastEditDate="2018-07-28T10:14:19.730" LastActivityDate="2018-08-31T16:00:49.283" Title="A reference request for computational plasticity" Tags="|finite-element|nonlinear-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29945" PostTypeId="1" CreationDate="2018-07-29T19:31:52.817" Score="2" ViewCount="72" Body="&lt;p&gt;Suppose a tiling is given in 2D (an embedding of a planar triangulated graph), with all faces convex.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose one moves each point, one by one, to the barycenter of its neighbors. I think that with such an update one can guarantee that the planarity is preserved at each step (after each point is moved). But could planarity be guaranteed in case whole configuration moves simultaneously?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understood that once such simultaneous position update converges (in case the positions of the points of the outer convex face are fixed), it can be guaranteed that the new tiling is also planar (Tutte's embedding). My question is: given no fixed positions, what happens if one performs a single simultaneous coordinate update from the initial configuration that is already planar (all faces convex)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case no guarantees can be made, are there special cases with which the simultaneous update would preserve planarity, i.e., special kinds of initial configurations, etc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In mathematical terms: The initial configuration is $Y_0$, and can be written as $Y_0 = T_0Y_0$, where $T_0$ is a transition matrix (each row/column having weight coefficients expressing each coordinate as a convex combination of its neighbors). The update is achieved as $Y_1 = T_bY_0$, where $T_b$ is a transition matrix that moves a configuration to the barycenter (a simultaneous update). Obviously, $Y_1 = T'Y_0$, where $T'$ is also a transition matrix $T' = T_bT_0$. But is that sufficient to guarantee that $Y_1 = T''Y_1$, for some transition matrix $T''$, i.e., each coordinate is a convex combination of its neighbors?&lt;/p&gt;&#xA;" OwnerUserId="28321" LastEditorUserId="28321" LastEditDate="2018-07-30T12:03:13.593" LastActivityDate="2018-07-30T12:03:13.593" Title="Simultaneous update to barycenters" Tags="|linear-algebra|computational-geometry|iterative-method|geometry|laplacian|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29948" PostTypeId="1" CreationDate="2018-07-31T08:51:10.317" Score="2" ViewCount="281" Body="&lt;p&gt;Given a set of $n$ points on which a triangulation is performed, it is possible to construct coefficients $\lambda_{ij}&amp;gt;0$ such that each point $x_i$ is a convex combination of the points connected to it. Suppose $N(i)$ is a set of points adjacent to $x_i$ in a triangulation; then, for each $x_i$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j\in N(i)} \lambda_{ij}x_j = x_i $$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j\in N(i)} \lambda_j = 1.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In matrix form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\Lambda x=x$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\Lambda$ is a square, $n\times n$, row stochastic matrix (non-negative entries, and no self-loops). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now suppose matrix $\Lambda$ is given. My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Can one reach configuration $x$ by the &lt;a href=&quot;https://en.wikipedia.org/wiki/Power_iteration&quot; rel=&quot;nofollow noreferrer&quot;&gt;power method&lt;/a&gt;, i.e., by repeatedly multiplying $\Lambda$ by a vector? The &quot;dominant&quot; eigenvector is $1_n$, so one would have to orthogonalize against it to the &quot;second dominant&quot;. I am asking because the above matrix form reminds me of the definition of eigenvalue/eigenvector. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If not, what would be a good approach to compute such a configuration? Is the solution unique?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I think that configuration $x$ is the solution to the problem of finding minimum of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\sum_{(i,j) \in E} (x_i - x_j)^2,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $E$ is the set of the edges of the triangulation. However, here one should compute the eigenvectors associated with the smallest eigenvalues. To establish the relation, I used the relation between the Laplacian and the adjacency matrix, as in the following paper.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Yehuda Koren, &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.3.2055&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;On Spectral Graph Drawing&lt;/a&gt; [PDF].&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One more interesting question: &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Can the coefficients $\lambda_{ij}$ be constructed in such a way that $\lambda_{ij} = \lambda_{ji}$, meaning that the matrix $\Lambda$ is symmetric? This way of constructing coefficients means that the &lt;a href=&quot;https://en.wikipedia.org/wiki/Power_iteration&quot; rel=&quot;nofollow noreferrer&quot;&gt;power method&lt;/a&gt; should converge to $x$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28321" LastEditorUserId="20417" LastEditDate="2018-08-04T17:02:01.177" LastActivityDate="2018-08-04T17:02:01.177" Title="Can the Power Method be used here?" Tags="|linear-algebra|optimization|linear-solver|computational-geometry|eigensystem|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29949" PostTypeId="1" AcceptedAnswerId="29956" CreationDate="2018-07-31T09:07:58.773" Score="2" ViewCount="101" Body="&lt;p&gt;We research the aerodynamics of the vehicle. Usually, there exists turbulence, which will cause the time-varying measurement of the force. The unsteady flow field determines we could not get a definite result. Often, we get a rough result. However, this time I calculate the force with CFD (computational fluid dynamics).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, due to the &lt;strong&gt;numerical error&lt;/strong&gt;, there are some apparent flaws in the results. For example:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The force coefficient is $0.00076$. However, in the physical world, we usually have two digits after the decimal point. Should I change $0.00076$ to $0$?&lt;/li&gt;&#xA;&lt;li&gt;One of the net forces should be $0$ because two forces $F_1$ and $F_2$ of the same size but opposite direction will cancel each out. However, because of numerical error again, $F_1 = 0.28$ and $F_2 = -0.27$ (after I keep only two digits after decimal point), which results in a net force of $0.01$. Should I leave it as $0.01$, or change it into $0$?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The numerical method produces too precise result, and they are too good to be true. In practice, it is impossible to get such a precise result. I have compared one of the results with the experimental results and found that there is 10% error between them.&lt;/p&gt;&#xA;" OwnerUserId="17631" OwnerDisplayName="PureLine" LastEditorUserId="7574" LastEditDate="2018-07-31T12:34:48.910" LastActivityDate="2018-07-31T16:32:41.933" Title="When writing a research article, how to deal with deviations from theoretical expectations due to numerical errors?" Tags="|fluid-dynamics|accuracy|writing|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29950" PostTypeId="5" CreationDate="2018-07-31T12:35:43.360" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-07-31T12:35:43.360" LastActivityDate="2018-07-31T12:35:43.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29951" PostTypeId="4" CreationDate="2018-07-31T12:35:43.360" Score="0" Body="Questions on how to communicate the outcome of computational science" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2018-07-31T19:15:41.873" LastActivityDate="2018-07-31T19:15:41.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29952" PostTypeId="1" CreationDate="2018-07-31T07:30:04.483" Score="1" ViewCount="112" Body="&lt;p&gt;Suppose I want to generate a vector &lt;code&gt;y&lt;/code&gt; which when circularly convoluted with a vector &lt;code&gt;h&lt;/code&gt; gives me a vector &lt;code&gt;x&lt;/code&gt;. I can find such a vector through the division of FFT as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y = ifft(fft(x)./fft(h))           // (1)                  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = [2,4,8];&#xA;h = [1,0.5,0];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then &lt;code&gt;y&lt;/code&gt; comes out to be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;y =  [-0.8889, 4.4444, 5.7778]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can take this &lt;code&gt;y&lt;/code&gt; and compute &lt;code&gt;x&lt;/code&gt; either through cyclic convolution or FFT method and both methods give me &lt;code&gt;x&lt;/code&gt; back:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z = ifft(fft(y).*fft(h))           // (2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;z = cconv(y,h,3)                   // (3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;z&lt;/code&gt; comes out to be &lt;code&gt;[2,4,8]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is that, similar to how (3) implements (2) as a linear operation (cyclic conv), is there a linear method which can implement (1) without using FFT. I would think such a method would be called cyclic deconvolution, but I can not understand how to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand how to implement linear/cyclic convolution and linear deconvolution - but not cyclic convolution.&lt;/p&gt;&#xA;" OwnerDisplayName="Abhi" LastEditorUserId="20688" LastEditDate="2018-08-01T12:33:17.860" LastActivityDate="2018-08-01T12:33:17.860" Title="Cyclic Deconvolution" Tags="|signal-processing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29953" PostTypeId="2" ParentId="29952" CreationDate="2018-07-31T12:57:45.707" Score="1" Body="&lt;p&gt;Since convolution can be written as a matrix-vector product $Ax=b$ of a circulant or Toeplitz matrix $A$ acting on a vector $x$, you can invert or pseudoinvert via SVD $A$ to obtain $x=A^{-1}b$. That said, FFT deconvolution will always be dramatically faster than this approach and should be preferred unless the kernel function ($h$ in your example above) has zeros or near-zeros in its spectrum to avoid dividing by zero. Even then, there may be methods for dealing with this particular case that I am unaware of.&lt;/p&gt;&#xA;" OwnerUserId="27171" LastActivityDate="2018-07-31T12:57:45.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29954" PostTypeId="5" CreationDate="2018-07-31T14:12:50.920" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-07-31T14:12:50.920" LastActivityDate="2018-07-31T14:12:50.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29955" PostTypeId="4" CreationDate="2018-07-31T14:12:50.920" Score="0" Body="For questions about words, phrases and definitions that are specific to computational science." OwnerUserId="22284" LastEditorUserId="22284" LastEditDate="2018-07-31T19:15:50.403" LastActivityDate="2018-07-31T19:15:50.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29956" PostTypeId="2" ParentId="29949" CreationDate="2018-07-31T14:46:20.343" Score="3" Body="&lt;p&gt;If you have theoretical expectations for some observables of simulations, I see two general ways of dealing with them:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You exploit them to get more accurate results, e.g., you make your algorithm use the fact that $F_1=F_2$. In this case, the value of your observables is not interesting since it they match your theoretical result by construction.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You use them as a sanity check or measure of simulation accuracy. In this case, there are two options for a publication:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You decide that the sanity check went well and is not of sufficient interest to be reported explicitly, i.e., it’s just you doing your homework.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You decide that the sanity is worthy to be reported. In that case you have to report how large your deviation from zero is. Depending on your kind of simulation, units, etc., 0.00076 could be large or small. That’s for you to decide.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is not so different from experimental results where you would theoretical knowledge (that you are not seeking to confirm) either to increase your accuracy, to obtain a measurement at all, as a sanity check, or to gauge your accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that 0, without any further specifications, is never a good way to present an empirical result¹, be it from experiment or simulations. Something can be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;zero within the measurement accuracy,&lt;/li&gt;&#xA;&lt;li&gt;numerically zero,&lt;/li&gt;&#xA;&lt;li&gt;0.00076 ± 0.01 or better 0.00 ± 0.01,&lt;/li&gt;&#xA;&lt;li&gt;complying with the null hypothesis of an expected value of zero,&lt;/li&gt;&#xA;&lt;li&gt;…,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;but not just 0.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;&#xA;¹ that is a real number, i.e., not the result of counting&#xA;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="7574" LastEditorUserId="7574" LastEditDate="2018-07-31T16:32:41.933" LastActivityDate="2018-07-31T16:32:41.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29957" PostTypeId="1" CreationDate="2018-07-31T18:15:49.060" Score="3" ViewCount="86" Body="&lt;p&gt;I am trying to solve a three-dimensional baroclinic transport problem. The hydrodynamic (three-dimensional shallow water) equations are:&#xA;\begin{align} &#xA;\nabla \cdot \vec{v} = 0, \tag{1} \\&#xA;\frac{\partial u}{\partial t} + \nabla \cdot (u \vec{v} ) + \frac{1}{\rho_0} \frac{\partial P}{\partial x} - f_{\text{coriolis}}v - \frac{1}{\rho_0}(\nabla \cdot T_{\text{mx}}) - S_x = 0, \tag{2} \\&#xA;\frac{\partial v}{\partial t} + \nabla \cdot (v \vec{v} ) + \frac{1}{\rho_0} \frac{\partial P}{\partial y} + f_{\text{coriolis}}u - \frac{1}{\rho_0}(\nabla \cdot T_{\text{my}}) - S_y = 0, \tag{3} \\&#xA;\frac{\partial H}{\partial t} + \frac{\partial (\bar{u}H)}{\partial x} + \frac{\partial (\bar{v}H)}{\partial y} = 0 \tag{4}.&#xA;\end{align}&#xA;The constituent (salinity or temperature) transport equation is:&#xA;\begin{align*}&#xA;\frac{\partial c}{\partial t} = \nabla \cdot (\vec{v} c) + \nabla \cdot (\bar{\bar{D}} \nabla c) = 0 \tag{5}.&#xA;\end{align*}&#xA;The problem is to solve for water velocity $\vec{v} = [u,v,w]^T$, river depth $H$, and constituent concentrate $c$ over a fluid domain $\Omega \subset \mathbb{R}^3$ and timescale $[0, t_{\text{end}}]$ of interest. The first equation is usually called the incompressible continuity equation. The second and third equations are called the horizontal momentum equations. The fourth equation is called the depth-averaged continuity equation. The fifth equation is the transport equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other important relationship to note is that the fluid pressure $P$ (which appears in the horizontal momentum equations) is related to the concentration $c$. Specifically, $P$ depends on the fluid density $\rho$, and $\rho$ is related to $c$ through an algebraic equation of state. So there is two-way coupling between the hydrodynamic equations and the transport equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution procedure used to run the simulation one time-step is as follows. Note that spatial discretization is done using a first-order SUPG FEM, and temporal discretization is done using a second-order (implicit) BDF method. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using $u,v,w,H,P$ from the old time-step, we solve discrete weak version of the horizontal momentum equations (2)-(3) and  the depth average continuity (4) monolithically to get $u, v, H$ at the new time-step. Since the horizontal momentum equations (2)-(3) are nonlinear, Newton's method with line-search is used. &lt;/li&gt;&#xA;&lt;li&gt;Using the new $u, v, H$ from step 1, a discrete weak version of incompressible continuity equation (1) is solved to get $w$ at the new time-step. This is a linear problem, so this is computationally easy compared to step 1.&lt;/li&gt;&#xA;&lt;li&gt;Using the new $u, v, w, H$ from steps 1 and 2, a discrete weak version of the transport equation (5) is solved to get $c$ at the new time-step. This is also linear, so this is computationally easy compared to step 1. Also, this new $c$ value is used in the calculation of a new $P$ value for use in step 1. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;After these three steps, I have all the variables we are looking for at the next discrete time value for all spatial coordinates in our grid. So I repeat these three steps to march in time.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This solution procedure might be seen as a first-order operator splitting schemes, since the hydrodynamics processes (equations (1)-(4) and steps 1-2) are solved separately from the transport process (equation (5) and step 3). Should one expect to gain any sort of advantage (e.g., in computational efficiency) by implementing  second-order splitting schemes (e.g., from this webpage: &lt;a href=&quot;http://www.asc.tuwien.ac.at/~winfried/splitting/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.asc.tuwien.ac.at/~winfried/splitting/&lt;/a&gt;)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have ran some computational experiments testing a few second-order schemes where the hydrodynamic equations (1)-(4) are split from equation (5), i.e., the coefficients $a_i$ correspond fractional step coefficients for hydrodynamic processes while the coefficients $b_i$ correspond to fractional step coefficients for transport processes; however, I do not seem to see any computational gains at the moment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The experiments are run on a lock-exchange problem that is similar to the one found in [Shin et al. 2014]. Initially, the left-half of the flume (with length 2 meters, width 0.2 meters, and depth 0.2 meters) is filled with saltwater, the right-half is filled with freshwater, so there is a sharp interface in the middle at $t = 0$. The barrier is instantly removed, so the dense saltwater flows beneath the less-dense freshwater. The fixed uniform mesh that I use to run my experiments is relatively coarse (grid spacing is 0.2 meters), but I don't think this matter since I am only interested in temporal convergence. I roughly measure computationally efficiency by plotting the discrete $L_2$ error versus the wall-clock time of the simulation. Below is an example of one such plot (split1 through split5 correspond to different implementations of second-order splitting schemes, while split0 corresponds to first-order splitting). Note that the &quot;reference solution&quot; used in this discrete $L_2$ error measurement is actually an approximate solution to the semi-discrete system, which is computed using a very small time-step size of 0.01 seconds. Also, I am specifically looking at the error in salt concentration $c$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qwgXI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qwgXI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Looking at points which are vertically aligned, you can see that the black points (first-order splitting) achieves an $L_2$ error which is roughly the same as the $L_2$ errors computed using second-order splitting (in fact, they are a little better). &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shin, J. O., S. B. Dalzieland, and P. F. Linden. 2004. Gravity currents produced by lock exchange. Journal of Fluid Mechanics 521:1–34.&lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2018-07-31T20:51:54.747" LastActivityDate="2018-07-31T20:51:54.747" Title="Should I expect computational gains using a second-order splitting method here?" Tags="|finite-element|fluid-dynamics|computational-physics|time-integration|operator-splitting|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29959" PostTypeId="1" AcceptedAnswerId="30014" CreationDate="2018-07-31T19:19:11.387" Score="1" ViewCount="3584" Body="&lt;p&gt;I'm trying to implement Barnes-Hut algorithm, with a binary tree. My initial conditions are a uniform mass distribution in a sphere with radius $R$. How can I create uniform dots distribution in a sphere?&lt;/p&gt;&#xA;" OwnerUserId="27468" LastEditorUserId="20688" LastEditDate="2019-06-22T22:37:15.253" LastActivityDate="2022-07-26T17:21:55.933" Title="Uniform dots distribution in a sphere" Tags="|computational-geometry|random-sampling|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29961" PostTypeId="1" AcceptedAnswerId="29972" CreationDate="2018-08-01T04:34:30.750" Score="5" ViewCount="1894" Body="&lt;p&gt;My question is about how I can solve a coupled system of ODE's, and print out the variables in a plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving for an q value and an e value, seen in this set of coupled ODE's below:&#xA;$$&#xA;\begin{aligned} \frac{dq}{dt} &amp;amp;= \frac{48}{5\pi M^2}(2\pi Mq)^{11/3}e\frac{1+\frac{73}{24}e^2+\frac{37}{96}e^4}{(1-e^2)^{7/2}},\\&#xA;\frac{de}{dt} &amp;amp;= -\frac{304}{15M}(2\pi Mq)^{8/3}e\frac{1+\frac{121}{304}e^2}{(1-e^2)^{5/2}}.&#xA;\end{aligned}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With my initial value for $e$ is 0.95 an my initial $q$ is 1, where constant $M$ = 1e9. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my results, I want to plot q vs. e. I expect the e value to go to zero while q increases.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def q_e(x,t):&#xA;    # M, a constant&#xA;    M = 1e9&#xA;&#xA;    # q and e input assignment&#xA;    q = x[0]&#xA;    e = x[1]&#xA;&#xA;    # Eq.&#xA;    dqdt = (48/(5*math.pi*M**2))*(2*math.pi*M*q)**(11/3)*((1+(73/24)*e**2+(37/96)*e**4)/(1-e**2)**(7/2))&#xA;    dedt = -1*(304/(15*M))*(2*math.pi*M*q)**(8/3)*e*((1+(121/304)*e**2)/(1-e**2)**(5/2))&#xA;    return [dfdt,dedt]&#xA;&#xA;x0 = [1,0.95]   # initial values q = 1, e = 0.99&#xA;t = np.linspace(0,100)&#xA;y = odeint(q_e,x0,t)&#xA;&#xA;q = x[:,0]      # output in sep. columns &#xA;e = x[:,1]&#xA;&#xA;plt.plot(q,e)&#xA;#plt.semilogy(q,e) # test&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assumed that if I printed my q and e values, I could see where I got wrong. So I tried it,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and got an output of&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[  1.00000000e+00   9.50000000e-01]&#xA;[  6.29120765e+05   3.44703571e-05]&#xA;[  1.39158341e+02   4.86785907e+02]&#xA;[  1.39158341e+02   4.86793360e+02]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the [  1.39158341e+02   4.86793360e+02] repeats itself for many columns after for the length of $t$. It is somewhat positive to notice that the q and e values of [  6.29120765e+05   3.44703571e-05] are normal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried messing around with my code, putting the e value instead of time t, but nothing is working. Looking for pointers and help, thanks!&lt;/p&gt;&#xA;" OwnerUserId="28344" LastEditorUserId="713" LastEditDate="2018-08-01T14:48:15.270" LastActivityDate="2018-08-02T09:38:22.077" Title="Solving for a set of coupled ODEs to get correct variable values" Tags="|python|ode|scipy|numerical-modelling|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="29963" PostTypeId="2" ParentId="15907" CreationDate="2018-08-01T09:20:52.030" Score="1" Body="&lt;p&gt;Sometimes the function to integrate cannot be JITed.&#xA;In that case, using another integration method would be the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend &lt;code&gt;scipy.integrate.romberg&lt;/code&gt; &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.integrate.romberg.html#scipy.integrate.romberg&quot; rel=&quot;nofollow noreferrer&quot;&gt;(ref)&lt;/a&gt;.&#xA;&lt;code&gt;romberg&lt;/code&gt; can integrate complex functions, and can evaluate the function with array.&lt;/p&gt;&#xA;" OwnerUserId="28347" LastEditorUserId="20688" LastEditDate="2018-08-01T12:12:07.457" LastActivityDate="2018-08-01T12:12:07.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29964" PostTypeId="1" AcceptedAnswerId="29965" CreationDate="2018-08-01T13:15:30.650" Score="6" ViewCount="499" Body="&lt;p&gt;I am using the &lt;a href=&quot;https://github.com/cusplibrary/cusplibrary/blob/develop/examples/Solvers/cg.cu&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUSP CG solver&lt;/a&gt; and I ran it on a couple of sparse matrices from the University of Florida sparse matrix collection. The solver was able to solve non positive definite sparse matrices. My understanding is that CG solvers can't solve non positive definite sparse matrices? So is how was CUSP's CG solver able to do it?&lt;/p&gt;&#xA;" OwnerUserId="6663" LastEditorUserId="20688" LastEditDate="2021-12-17T05:54:35.717" LastActivityDate="2021-12-18T04:36:21.967" Title="How can a CG solver solve a non positive definite sparse matrix" Tags="|linear-solver|sparse-matrix|iterative-method|conjugate-gradient|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29965" PostTypeId="2" ParentId="29964" CreationDate="2018-08-01T13:50:31.340" Score="10" Body="&lt;p&gt;I highly recommend the following read:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf&quot; rel=&quot;noreferrer&quot;&gt;J.R. Shewchuk, &quot;An Introduction to the Conjugate Gradient Method Without the Agonizing Pain&quot;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In short, if the matrix is non-positive definite, there is &lt;strong&gt;no guarantee that CG will fail&lt;/strong&gt;. It might be able to solve it (for some RHSs and certain tolerances), but it is just not supposed to and, probably, converges slower than the appropriate methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You definitely should look into the other iterative solvers (say, BiCGStab or something else, depending on your problem properties).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: I am not familiar with CUPS, so I would also check if there is a function inside that checks the property of the matrix and switches to some other algorithm (&lt;strong&gt;unlikely, but possible&lt;/strong&gt;) – especially if the solver log is ambiguous.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-08-01T13:50:31.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29966" PostTypeId="2" ParentId="29940" CreationDate="2018-08-01T15:38:14.550" Score="1" Body="&lt;p&gt;As mentioned by Biswajit Banerjee, the following reference explains well the basics, although I have not checked this &quot;new&quot; version.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Belytschko, Ted, et al. Nonlinear finite elements for continua and structures. John wiley &amp;amp; sons, 2013.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I asked a colleague who has worked in computational plasticity and suggested the following reference.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;E.A. de Souza Neto, D. Perić and D.R.J. Owen. Computational Methods for Plasticity: Theory and application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It  presents the following topics:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Linear algebra&lt;/li&gt;&#xA;&lt;li&gt;Finite elements from  a virtual works perspective&lt;/li&gt;&#xA;&lt;li&gt;Theory of plasticity&lt;/li&gt;&#xA;&lt;li&gt;Finite Elements in plasticity, centering in return mapping algorithm. It explains the basic models: Tresca, von Mises, Drucker-Prager and Mohr-Coulomb.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-08-01T15:38:14.550" CommentCount="0" CommunityOwnedDate="2018-08-01T15:38:14.550" ContentLicense="CC BY-SA 4.0" />
  <row Id="29967" PostTypeId="2" ParentId="29948" CreationDate="2018-08-01T15:40:44.033" Score="3" Body="&lt;p&gt;Your intuition is right, solving $\Lambda x = x$ for $x$ given $\Lambda$ is similar to an eigenvalue problem, except you already know the eigenvalue $\lambda_1 = 1$ and its corresponding eigenvector $x_1 = 1_n$. You also know that the rest of the eigenvalues have modulus less than one since $\Lambda$ is a left-stochastic matrix, so it seems that the power method will work if you are interested in the finding the rest of the eigenvalues/eigenvectors. The idea that you suggest of &quot;removing&quot; the largest eigenvalue $\lambda_1 = 1$ so that the power method converges to the second-dominant eigenvector $x_2$ is known as &lt;strong&gt;deflation&lt;/strong&gt;, and this would be necessary if you wanted to find the rest of the eigenvectors using the power method. There are some very simple ways to accelerate the power method, you should search up &lt;strong&gt;shifted inverse iteration&lt;/strong&gt;, and its improvement &lt;strong&gt;Rayleigh quotient iteration&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The power (shifted inverse, Rayleigh quotient) method may not work if you have multiple dominant eigenvalues (i.e., $\lambda_2$ and $\lambda_3$ have the same modulus). To overcome this, you can use &lt;strong&gt;subspace iteration&lt;/strong&gt;, which is basically the power method, except you replace your random initial $x$ vector with a matrix (whose columns are linearly independent), so that you get convergence to multiple eigenvectors at once. The most popular version of subspace iteration makes use of the QR factorization for normalization, and it is commonly known as the &lt;strong&gt;QR algorithm&lt;/strong&gt;. The advantage of the QR algorithm is that it can be set up to obtain all of the eigenvalues of $\Lambda$. Many of the ideas from the power method (e.g., using shifts, using deflation) can be extended to the QR algorithm to accelerate convergence. There are also other things you can do to make the QR algorithm faster (e.g., reduce your matrix $\Lambda$ into an upper-Hessenberg matrix using Householder transformations, or using Given's rotations if your matrix is sparse or you are interested in parallelism). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR; If you want all of the eigenvalues, you should probably use the QR algorithm instead. If you only want a few of the largest (or smallest, or middle) eigenvalues, you can use rayleigh quotient iteration + deflation. This might be advantageous if your matrix is huge and sparse, in which case you wouldn't need to compute an expensive QR factorization. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that all of these methods fall under the category of &quot;subspace iteration&quot; (their proof of convergence is essentially the same as the power method). You can conveniently look up most of the things I mentioned &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/etemplates/node94.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;here&quot;&gt;here&lt;/a&gt;. Also, these methods are extremely easy to understand and implement, which is why I recommend these. More sophisticated eigenvalue algorithms are commonly used nowadays, but many of them rely on the same principles.&lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2018-08-01T19:26:25.270" LastActivityDate="2018-08-01T19:26:25.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29970" PostTypeId="2" ParentId="29936" CreationDate="2018-08-02T06:27:14.113" Score="0" Body="&lt;p&gt;I have some ideas, you could try. I will fix one $i$, to simplify the notation. If the $(\vec{x}_i)_{i=1}^q$ are independent, this should be no issue.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Least squares&lt;/strong&gt;: Solve the minimisation problem &#xA;$$ \min \|B\vec{x} - \vec{y}\|_2^2,$$ $$\text{s.t.: }  \vec{x} \geq 0, \|\vec{x}\|_1 = 1.$$&#xA;This could be done with probably any standard non-linear optimisation library. &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Bayesian approach&lt;/strong&gt;: In the Bayesian approach, you would try to compute a posterior measure on the vector $\vec{x}$. You assume you have some prior information about the vector and then incorporate the information given by $\vec{y}$ into this prior information. The prior information you have is that $\vec{x}$ is a probability vector with $m$ entries. Luckily, there are probability measures representing such probability vectors, such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Dirichlet_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dirichlet distribution&lt;/a&gt;. Say we have a prior measure $p_0$ on $\mathbb{R}^m$ representing the prior knowledge.&#xA;For the Bayesian procedure you need to construct a likelihood. If we assume that the noisy data is generated by&#xA;$$\vec{y} \leftarrow B(\vec{x}_{\rm true}) +\eta,$$&#xA;where $\eta \sim \mathrm{N}(0, \Gamma)$ is Gaussian noise. The likelihood would be $$L(\vec{y}|\vec{x}) = \exp(-(1/2)\|\Gamma^{-1/2}(\vec{y} \leftarrow B\vec{x})\|_2^2).$$&#xA;You can now use Bayes' rule and for instance importance sampling or Markov chain Monte Carlo to compute the posterior measure. But for that, you should probably look at the Wikipedia pages: &lt;a href=&quot;https://en.wikipedia.org/wiki/Bayesian_inference&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian inference&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov chain Monte Carlo&lt;/a&gt;.&#xA;The posterior mean would give you the $L^2$-optimal estimator for $\vec{x}_{\rm true}$, the measure itself would quantify the uncertainty around the vectors $\vec{x}$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="27670" LastActivityDate="2018-08-02T06:27:14.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29971" PostTypeId="1" AcceptedAnswerId="29979" CreationDate="2018-08-02T09:15:14.813" Score="0" ViewCount="149" Body="&lt;p&gt;What is the difference between Simulation-based Optimization and PDE-constrained Optimization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would studying a text on Simulation-based optimization be sufficient to understand and apply both?&lt;/p&gt;&#xA;" OwnerUserId="28350" LastActivityDate="2018-08-02T22:50:16.483" Title="Simulation-based Optimization vs PDE-constrained Optimization" Tags="|optimization|pde|simulation|constrained-optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29972" PostTypeId="2" ParentId="29961" CreationDate="2018-08-02T09:38:22.077" Score="6" Body="&lt;p&gt;The function $q(e)$ satisfies a first order linear ODE&#xA;$$ \frac{\mathrm{d}q}{\mathrm{d}e} = \frac{111 e^4+876 e^2+288}{(e^2-1)&#xA;(121 e^2+304)} q(e), $$&#xA;which can be solved very easily by using an integrating factor:&#xA;$$ q(e) = &#xA;C_1 \exp\left(&#xA;\tfrac{111}{121} e-3\operatorname{arctanh}e+\tfrac{10440}{1331 \sqrt{19}} \operatorname{arctan}\left(\tfrac{11}{4&#xA;\sqrt{19}} e\right)&#xA;\right). $$&#xA;For the initial conditions you gave this gives&#xA;$$ q|_{t=\infty} = q(0)/q(e_0) = 38.561177784533462867. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To follow up on my comments, this is what rescaling does to the&#xA;numerical solution (I added a factor of $e$ to &lt;code&gt;dqdt&lt;/code&gt; to match the formula).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import *&#xA;from scipy.integrate import odeint, solve_ivp&#xA;&#xA;M = 1e9&#xA;scale = 1e22&#xA;&#xA;def rhs(t, u):&#xA;  q, e = u[0], u[1]&#xA;  dqdt = (48/(5*math.pi*M**2))*(2*math.pi*M*q)**(11/3)*((1+(73/24)*e**2+(37/96)*e**4)/(1-e**2)**(7/2)) * e&#xA;  dedt = -1*(304/(15*M))*(2*math.pi*M*q)**(8/3)*e*((1+(121/304)*e**2)/(1-e**2)**(5/2))&#xA;  return [dqdt/scale, dedt/scale]&#xA;&#xA;ic = [1.0, 0.95]&#xA;&#xA;sol = solve_ivp(rhs, (0.0, 10.0), ic, method=&quot;LSODA&quot;, atol=0, rtol=1e-8)&#xA;print(&quot;success: &quot;, sol.success)&#xA;print(&quot;final t: &quot;, sol.t[-1])&#xA;print(&quot;final y: &quot;, sol.y[:,-1])&#xA;print(&quot;y error: &quot;, abs(sol.y[0,-1] - 38.561177784533462867))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the rescaling, it succeeds with no warning. Since you say you are interested in plotting $q(e)$, it's only necessary to increase $t$ until $e(t)$ becomes really small, and that $t$ is substantially smaller than $100$. Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;success:  True&#xA;final t:  10.0&#xA;final y:  [3.85611807e+01 7.80753973e-19]&#xA;y error:  2.9253639937110165e-06&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-08-02T09:38:22.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29973" PostTypeId="1" AcceptedAnswerId="29974" CreationDate="2018-08-02T13:24:46.297" Score="3" ViewCount="73" Body="&lt;p&gt;I have 5 million items that have 10 features (all continuous and not categorical) each and would like to select a small subset of these items. Ideally, I want to manually specify 10 features of my own up front and find a subset of items in the pool that are all close to the features I specify. By closest, I mean the distance between the numbers I specify and the item features selected from the pool.&lt;/p&gt;&#xA;" OwnerUserId="28351" LastActivityDate="2019-06-05T11:30:41.567" Title="What is a good way to select a small subset (say 50) of items from a large pool of items (say 5 million) while minimizing an objective function?" Tags="|optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29974" PostTypeId="2" ParentId="29973" CreationDate="2018-08-02T13:43:56.423" Score="4" Body="&lt;p&gt;Simple way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compute the Euclidean distance between each item and the target feature specification. Sort by distance. Take the first 50 elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fast way for arbitrary target features:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your parameters are continuous, you may define a gridding of the 10-dimensional feature space with grid spacing $d_i,i=1,\ldots,10$. Find the grid coordinates of each item by dividing the feature values $x_i$ by each $d_i$ and taking the floor of the result. Bin together items with the same grid coordinates. Next, for each target feature such as you've described, you can quickly compute its grid coordinates in the same manner and select items from that cell or its neighbors to quickly find the subsets you're looking for without having to compute all 5 million (or however many) distances every time you want to specify a feature.&lt;/p&gt;&#xA;" OwnerUserId="27171" LastActivityDate="2018-08-02T13:43:56.423" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29975" PostTypeId="1" CreationDate="2018-08-02T15:55:07.043" Score="1" ViewCount="2285" Body="&lt;p&gt;It is so confusing, especially when I learns discontinuous galerkin method in broken Sobolev space and weak Dirichlet boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the trial function is chosen with homogeneous boundary conditions, how can I get a linear discrete equation corresponding to test basis functions with DOFs on the boundary? I mean if $v\in V_0 \subset V$, then the space of $u\in V$ would be &quot;larger&quot; than $V_0$. The final matrix would be rectangular rather than square. Extra conditions must be imposed to make it square and solvable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding now is that the homogeneous boundary condition is helpful in derivation of weak form from original differential equation, such as Poisson equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: When it comes to matrix, a $V_{0,h}\subset V_0$ and $V_{h} \subset V$ must be chosen first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit2: I am confused, sometimes the book may say bases of $V_0 $ are identical to $V,$ but the boundary condition is different.&lt;/p&gt;&#xA;" OwnerUserId="24589" LastEditorUserId="23712" LastEditDate="2018-08-03T15:02:58.570" LastActivityDate="2018-08-03T21:50:13.520" Title="Why is the test function space in FEM chosen with homogeneous boundary conditions?" Tags="|finite-element|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29976" PostTypeId="2" ParentId="29975" CreationDate="2018-08-02T16:33:37.050" Score="1" Body="&lt;p&gt;While it is true that $V_0 \subset V$, these spaces are still both infinite dimensional, and there are no matrix sizes to speak of since no discretization has happened yet. To form matrices, you must choose finite dimensional subspaces $V_{0,h}$ and $V_h$ which &quot;approximate&quot; $V_0$ and $V$, in some sense. The size of the resulting matrices just depends on the dimension of $V_{0,h}$ and $V_h$. Therefore, if you use the same number of basis functions so that your finite-dimensional approximate spaces $V_{0,h}$ and $V_h$ have the same dimension, your matrices will end up square. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Actually, the last sentence above is incorrect. You can get square matrices without having the exact same dimension for your subspaces; what matters is obviously the number of DOF you have, which also includes essential boundary conditions which are enforced in your problem. The following example should hopefully be more helpful than what I wrote above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We seek a function $u(x)$ such that&#xA;\begin{align}&#xA;-u'' +u = f, \: x \in \Omega, \\&#xA;u'|_{\Gamma_N} = g_N, \: u|_{\Gamma_D} = g_D,&#xA;\end{align}&#xA;where $\Omega$ is some open interval whose boundary is partitioned $\partial \Omega = \Gamma_N \cup \Gamma_D$.&#xA;One weak form for this problem is: find $u \in U$ such that&#xA;\begin{align*}&#xA;\int_{\Omega} v' u' + \int_{\Omega} vu = \int_{\Omega} vf - v|_{\Gamma_N} g_N , \: \forall v \in V,&#xA;\end{align*}&#xA;where &#xA;\begin{align}&#xA;V = \{ w \: | \: w \in H^1(\Omega), w|_{\Gamma_D} = 0  \}, \\&#xA;U = \{ w \: | \: w \in H^1(\Omega),  w|_{\Gamma_D} = g_D \}.&#xA;\end{align}&#xA;&lt;strong&gt;Note that we set $w|_{\Gamma_D} = 0$ in the test function space $V$ to eliminate the $v|_{\Gamma_D} u''|_{\Gamma_D}$ term since no information about $u''$ is known on that part of the boundary. This does not affect the DOF boundary conditions which we will enforce later, since we can enforce them directly without using our test function space $V$.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets proceed by a Petrov-Galerkin method: we make finite-dimensional subspaces by specifying bases&#xA;\begin{align*}&#xA;U_m = \text{span} \{\phi_j\}_{j = 1}^m, \\ &#xA;V_n =  \text{span} \{\psi_i \}_{i = 1}^n.&#xA;\end{align*}&#xA;The weak problem reduces to problem of seeking some $u_m \in U_M$ for which the following $n+1$ equations hold:&#xA;\begin{align*}&#xA;\int_{\Omega} \psi_i' u_m' + \int_{\Omega} \psi_i u_m = \int_{\Omega} \psi_i f - \phi_i |_{\Gamma_N} g_N, \: i = 1, ... ,n \\ &#xA;u_m|_{\Gamma_D} = g_D \text{ (enforce DOF bc directly without $V$) }.&#xA;\end{align*}&#xA;Expanding $u_m$ out in a basis, this becomes the linear system &#xA;\begin{align*}&#xA;\sum_{j = 1}^m \left( \int_{\Omega} \psi_i ' \phi_j ' + \psi_i \phi_j \right) u_j = \int_{\Omega} \psi_i f - \phi_i |_{\Gamma_N} g_N, \: i = 1, ..., n \\&#xA;\sum_{j = 1}^m u_j \phi_j = g_D.&#xA;\end{align*}&#xA;In matrix form (with m = 3, n = 2 for simplicity):&#xA;\begin{align*}&#xA;\begin{bmatrix}&#xA;\int_{\Omega} (\psi_1' \phi_1' + \psi_1 \phi_1) &amp;amp; \int_{\Omega} (\psi_1' \phi_2' + \psi_1 \phi_2) &amp;amp; \int_{\Omega} (\psi_1' \phi_3' + \psi_1 \phi_3) \\ &#xA;\int_{\Omega} (\psi_2' \phi_1' + \psi_2 \phi_1) &amp;amp; \int_{\Omega} (\psi_2' \phi_2' + \psi_2 \phi_2) &amp;amp; \int_{\Omega} (\psi_2' \phi_3' + \psi_2 \phi_3) \\&#xA;\phi_1 |_{\Gamma_D} &amp;amp; \phi_2 |_{\Gamma_D} &amp;amp;  \phi_3 |_{\Gamma_D} &#xA;\end{bmatrix} &#xA;\begin{bmatrix}&#xA;u_1 \\ u_2 \\ u_3&#xA;\end{bmatrix} =&#xA;\begin{bmatrix}&#xA;\int_{\Omega} \psi_1 f - \phi_1|_{\Gamma_N} g_N \\&#xA;\int_{\Omega} \psi_2 f - \phi_2|_{\Gamma_N} g_N \\&#xA;g_D&#xA;\end{bmatrix}.&#xA;\end{align*}&#xA;So, the last row to enforce essential boundary conditions extends the matrix to be square, as you mention. Once again, it was fine that we restricted our space $V$ to be zero on part of the boundary since the essential DOFs were enforced directly using the equation $\sum u_j \phi_j = g_D$, which does not involve any functions from $V$. &lt;/p&gt;&#xA;" OwnerUserId="23712" LastEditorUserId="23712" LastEditDate="2018-08-02T19:38:56.137" LastActivityDate="2018-08-02T19:38:56.137" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="29977" PostTypeId="1" AcceptedAnswerId="29987" CreationDate="2018-08-02T20:46:20.950" Score="5" ViewCount="365" Body="&lt;p&gt;I am struggling a bit with the concept of preconditioners for iterative solvers and how to implement them efficiently. The literature mostly provides methods to create a preconditioner matrix $M$ (incomplete LU, incomplete Cholesky etc.). But the conjugate gradient methods need $M^{-1}$ during the iterations. As inverting a matrix is not a so good idea, I do not know how to get $M^{-1}$. I looked at some libraries that use iterative solvers and was able to see that the equation where $M^{-1}$ is needed, is actually solved by forward and backward substitution. Is this really pratical? When I think that I need to create the preconditioner and do forward and backward substitution, I just can use direct methods? Doing it that way I would save the time of all the iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also have read somewhere (unfortunately cannot find it anymore) that the LDL Cholesky decomposition can be easily inverted. The same literature also provides information that it might be better to solve $Mx=b$ instead of $Ax=b$ as $M$ should be very similar to $A$. I tried a couple of matrices but the results of $Mx=b$ are way different than $Ax=b$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think? What options are available for solving with preconditioners?&lt;/p&gt;&#xA;" OwnerUserId="28358" LastEditorUserId="17294" LastEditDate="2018-08-03T05:39:54.070" LastActivityDate="2018-08-03T22:45:52.500" Title="Efficient implementation of preconditioners for iterative solvers" Tags="|iterative-method|preconditioning|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29978" PostTypeId="2" ParentId="29975" CreationDate="2018-08-02T22:27:49.317" Score="5" Body="&lt;p&gt;This is more of a conceptual argument but may be helpful in understanding why the math works out this way. As a bonus, no matrix will be involved (because the problem is actually already with the weak formulation before any discretization).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What FEM does is to numerically approximate the solution $y\in V$ to some abstract operator equation $Ay=f$ for $A:V\to V^*$ for $f\in V^*$ and some abstract vector space $V$ and its dual $V^*$ (it's not really necessary for this argument to understand what a dual space is; it's enough to accept that it's the space for which the following works). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step is to see that $Ay-f = 0$ if and only if $\langle Ay-f,v\rangle = 0$ for all $v\in V$. (Intuitively, $0$ is the only vector that is orthogonal to every other vector.) Note that it's the same space $V$ for $y$ and $v$. &lt;strong&gt;It is crucial for this equivalence that $V$ is a vector space, in particular, that for $v_1,v_2 \in V$ you also have $v_1+v_2\in V$.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next step is specific for PDEs. If $A$ is a differential operator such as the Laplacian $-\Delta$, it's easy to see that the solution $y$ is not unique unless you fix constants. The usual (but not the only) way is to fix the constant by fixing the value at the boundary. To include this in the above approach means only looking in the space of all functions that have the same values at the boundary (both for $y$ and for $v$, remember!) Now it should be fairly obvious that &lt;strong&gt;this doesn't work unless that value is $0$&lt;/strong&gt;, because otherwise if you add two functions with the same boundary values, their boundary value will be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, that's a too nasty restriction to take lying down. Luckily there's a trick: Pick a function $y_b$ that has the desired values at the boundary and satisfies $A y_b=0$, compute a solution $y_0$ to $Ay=f$ with zero boundary values as described above (including using test functions $v$ with zero boundary values), and set $y=y_0+y_b$. &lt;strong&gt;This is precisely what you are doing in computations&lt;/strong&gt;, with some short cuts to avoid explicitly constructing $y_b$ first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mention the weak Dirichlet boundary conditions: Here you do not prescribe boundary conditions, but &quot;penalize&quot; non-zero values so that the solution will actually be (close to) zero. Since you do not have boundary conditions for the solution, you also shouldn't have boundary conditions for the test functions. (And for DG, there's additional difficulties since the weak form is an operator equation $A:V\to W^*$ for $W\neq V$, so the test functions have to come from a different space of functions that don't even need to &lt;em&gt;have&lt;/em&gt; sensible values on the boundary.)&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2018-08-02T22:27:49.317" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29979" PostTypeId="2" ParentId="29971" CreationDate="2018-08-02T22:50:16.483" Score="5" Body="&lt;p&gt;Both approaches apply to the same problem (numerical minimization of functionals which involve the solution of a PDE, although both extend to a larger class of problems). The difficulty is that for all but academic examples, the numerical solution of the PDEs requires a huge number of degrees of freedom which a) means that it takes a long time and b) computing gradients and Hessians by finite differences is completely infeasible. There's two ways of dealing with this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can take the numerical solution of PDEs as a black box that spits out a solution given a specific choice of the design values. This allows you to evaluate the functional at a point, but not any derivatives. Luckily, there are a number of &lt;em&gt;derivative-free&lt;/em&gt; optimization methods that (usually) work (somewhat) better than blind guessing.&lt;sup&gt;1&lt;/sup&gt; &lt;strong&gt;This seems to be what you call simulation-based optimization.&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use mathematical tools such as the implicit function theorem or Lagrange multiplier calculus to give an analytical, exact, characterization of derivatives of the solution of a PDE, which often turn out to be solutions to suitably linearized PDEs that can be solved numerically using the same (if not simpler) tools as for the original models. These can then be used to characterize and numerically calculate gradients and Hessians of the functional to be minimized, meaning you can apply faster methods (since they use more information) such as steepest descent, nonlinear conjugate gradients, or (quasi-Newton) methods. &lt;strong&gt;This is what PDE-constrained optimization does.&lt;/strong&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;As you can see, the approaches are very different, and in particular require different mathematical backgrounds. It is thus not reasonable to expect that reading a book on simulation-based optimization will give you the tools to understand PDE-constrained optimization (nor will a book on PDE-constrained optimization tell you about derivative-free optimization methods). If you'd like to know more about the latter, a good introductory (albeit still mathematical) textbook is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;De los Reyes, Juan Carlos&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1007/978-3-319-13395-9&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Numerical PDE-constrained optimization&lt;/strong&gt;&lt;/a&gt;, SpringerBriefs in Optimization. Cham: Springer (ISBN 978-3-319-13394-2/pbk; 978-3-319-13395-9/ebook). x, 123&amp;nbsp;p. (2015). &lt;a href=&quot;https://zbmath.org/?q=an:1312.65100&quot; rel=&quot;noreferrer&quot;&gt;ZBL1312.65100&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also many excellent lecture notes to be found on the internet.&lt;/p&gt;&#xA;&#xA;&lt;hr/&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;Yes, I'm biased. Of course, if all you have is a software that simulates some process and nobody will tell you what PDE exactly it (tries) to solve, simulation-based optimization is pretty much the only game in town.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2018-08-02T22:50:16.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29980" PostTypeId="1" AcceptedAnswerId="29983" CreationDate="2018-08-03T00:57:04.333" Score="1" ViewCount="450" Body="&lt;p&gt;I have a 7-14 dimensional piecewise linear cost function I'd like to minimize with two quadratic terms of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;  f(X) = X^tCX + d \sum_i |x_i-x^*_i|^2 + \sum_i P_i(x_i-x^0_i)&#xA;$$&#xA;$$ \sum_i x_i = 1, 0 \leq x_i \leq 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $d&amp;gt;0$, $C$ is positive definite and $X^*,X^0$ are fixed.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The $P_i(x_i-x_i^0)$ are continuous and piecewise-linear with an &quot;almost nice&quot; form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jQiqa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jQiqa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are always $0$ when $x_i-x_i^0 \leq 0$ and convex piecewise linear when $x_i-x_i^0 \ge 0$, but note that the blue curve is not convex for all $x_i-x_i^0$.  So close!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried splitting this into $2^7$ to $2^{14}$ convex problems by restricting the domains of $x_i \leq x^0_i, x_i \geq x^0_i$ combinatorially by that was too computationally intensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good (python) solver for functions that are piecewise-linear &lt;em&gt;and&lt;/em&gt; quadratic &lt;em&gt;and&lt;/em&gt; non-convex?  Is there a more clever way to simplify this problem?&lt;/p&gt;&#xA;" OwnerUserId="16716" LastActivityDate="2018-08-03T07:37:52.593" Title="Piecewise-Linear Quadratic Optimization for an &quot;Almost Convex&quot; Problem" Tags="|python|constrained-optimization|nonlinear-programming|nonconvex|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29981" PostTypeId="1" CreationDate="2018-08-03T03:51:20.237" Score="4" ViewCount="771" Body="&lt;p&gt;I know Lax-Milgram theorem is fundamental to FEM. But it did not explain what will happen if coercivity is not met.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is if it is met, eigen value of the operator (or its corresponding matrix) is always positive (may be degenerated). If it is not, with the refinement of grid, more and more eigen value will be put into the possible range of eigen values (from positive to negative) and there will be some one very close to zero. Then the conditional number would be high enough to ruin the result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that a correct explanation? &lt;/p&gt;&#xA;" OwnerUserId="24589" LastActivityDate="2018-08-03T07:09:44.617" Title="Why Coercivity is so important in FEM framework?" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29982" PostTypeId="2" ParentId="29981" CreationDate="2018-08-03T06:38:10.397" Score="7" Body="&lt;p&gt;The intuition is correct, although I would amend your statement that the smallest eigenvalue of the operator and its matrix discretizations is always &lt;em&gt;strictly&lt;/em&gt; positive and hence &lt;em&gt;never&lt;/em&gt; degenerated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting it another way, this is why in infinite-dimensional spaces, it is not sufficient for a (self-adjoint) operator $A:V\to V^*$ to satisfy $\langle v,Av\rangle &amp;gt;0$ for every $v\in V$ but you need to have  $\langle v,Av\rangle \geq c\|v\|^2$ for some $c&amp;gt;0$ independent of $v$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But note that if what you want to show that a matrix $M$ is invertible (which is what the point of the Lax-Milgram is), positive-definiteness is not necessary, only sufficient -- you can have invertible matrices that are not positive definite. All that is required is that $M$ is injective and surjective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding theorem for infinite-dimensional operators is the &lt;em&gt;Banach-Necas-Babuska Theorem&lt;/em&gt; (see, e.g., Theorem 8.1 in &lt;a href=&quot;https://arxiv.org/abs/1709.08618&quot; rel=&quot;noreferrer&quot;&gt;my lecture notes&lt;/a&gt;), which replaces the Lax-Milgram Theorem when studying Petrov-Galerkin methods such as DG. Again, you need to make the assumption &quot;quantitative&quot; in infinite dimensions: instead of surjectivity, you need the so-called &lt;em&gt;inf-sup condition&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="1804" LastEditDate="2018-08-03T07:09:44.617" LastActivityDate="2018-08-03T07:09:44.617" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="29983" PostTypeId="2" ParentId="29980" CreationDate="2018-08-03T07:18:02.300" Score="3" Body="&lt;p&gt;You could implement the piecewise linear functions using SOS2 variables (SOS2=Special Ordered Sets of Type 2). This approach does not care about convexity (that is, convexity related to the piecewise linear functions; for most solvers it is still important that the quadratic terms are convex). MIQP (Mixed Integer Quadratic Programming) solvers with support for SOS2 variables and callable from Python are readily available. If an MIQP solver does not support SOS2 variables, we can simulate them with binary variables. Some solvers to consider are Cplex, Gurobi, SCIP.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the non-convexities only appear at $x=0$, we can optimize the formulation. We will still arrive at an MIQP model.&lt;/p&gt;&#xA;" OwnerUserId="19551" LastEditorUserId="19551" LastEditDate="2018-08-03T07:37:52.593" LastActivityDate="2018-08-03T07:37:52.593" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29985" PostTypeId="1" CreationDate="2018-08-02T19:08:44.363" Score="6" ViewCount="1074" Body="&lt;p&gt;I have read this question and answer – &lt;a href=&quot;https://earthscience.stackexchange.com/questions/12057/how-to-interpolate-scattered-data-to-a-regular-grid-in-python&quot;&gt;Interpolation of scattered data to a regular grid in python&lt;/a&gt; and I am doing something similar as I have temperature values of the atmosphere at different heights defined on a  coarse grid  and I want to interpolate this to a regular grid in Python and I also need to weight the matches using the Cressman weighting scheme as shown in this old paper – &lt;a href=&quot;ftp://ftp.library.noaa.gov/docs.lib/htdocs/rescue/mwr/087/mwr-087-10-0367.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cressman weighting scheme&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this regard, I have a couple of questions. Looking at equation (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C_h =  -W  E_h$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand what $W$ is and how to go about calculating it but it is not clear to me what the author means when he writes:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$E_h$ is the error of the interpolated value of the first guess field at the location of the observation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;An error is usually a difference between two values. If that is the case which two values are being referred to here? As a first guess, I have my temperature values at irregular grid intervals and I can easily find matches within a radius of influence for a particular grid point. But which two values are contributing to the error that is being talked about here? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also read papers (cf. &lt;a href=&quot;https://journals.ametsoc.org/doi/pdf/10.1175/1520-0493%281998%29126%3C0393%3ATGCKSD%3E2.0.CO%3B2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vertical and horizontal Cressman radii&lt;/a&gt;) in which it is mentioned that the radius of influence that Cressman talks about in the above paper sometimes can be treated as a &quot;deep layer&quot; in the atmospheric sciences. In other words, the interpolated value at a grid point can include matches from not only the horizontal radius of influence on a given horizontal surface but also matches from above and below the surface in question. Is that accurate? &lt;/p&gt;&#xA;" OwnerUserId="46217" OwnerDisplayName="user16555" LastEditorUserId="20688" LastEditDate="2018-08-05T03:19:06.173" LastActivityDate="2018-08-05T03:19:06.173" Title="Cressman interpolation and objective analysis" Tags="|interpolation|numerics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29986" PostTypeId="2" ParentId="29975" CreationDate="2018-08-03T21:50:13.520" Score="2" Body="&lt;p&gt;The important point, not mentioned in the answers above, is that the weak form is a &lt;em&gt;variational&lt;/em&gt; formulation in which we ask the question: Can we find a function $u$ that minimizes the energy functional (or is a stationary point of some other functional) in such a way that any &lt;em&gt;other potential solution&lt;/em&gt; $u+v$ does not minimize the energy (or is not a saddle point). The $v$ here becomes the test function in the weak formulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we seek solutions among the set of functions that have boundary values $u|_\Gamma=g$, then clearly the &lt;em&gt;perturbed&lt;/em&gt; functions also have to satisfy $(u+v)|_\Gamma=g$, which can only be the case if $v|_\Gamma=0$, i.e., satisfies homogeneous Dirichlet boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lengthier discussion of this issue can be found in video lecture 21.5 and following here: &lt;a href=&quot;http://www.math.colostate.edu/~bangerth/videos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math.colostate.edu/~bangerth/videos.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-08-03T21:50:13.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="29987" PostTypeId="2" ParentId="29977" CreationDate="2018-08-03T22:45:52.500" Score="8" Body="&lt;p&gt;I don't particularly care for the notation $M^{-1}$ precisely because of the confusion you find yourself in. I (and others) simply call the preconditioner $P$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point, however, is that for iterative solvers, you generally never need a matrix or its inverse &lt;em&gt;explicitly&lt;/em&gt;. All you need to be able to do is &lt;em&gt;apply&lt;/em&gt; it to a vector. So, let's say you want to solve&#xA;$$&#xA;  A x = b&#xA;$$&#xA;and you've computed something like an incomplete LU factorization (ILU) of $A$, i.e., you have two triangular matrices $\tilde L,\tilde U$ so that&#xA;$$&#xA;  A \approx \tilde L\tilde U&#xA;$$&#xA;It is important to remember that computing an &lt;em&gt;incomplete&lt;/em&gt; LU decomposition is &lt;em&gt;substantially&lt;/em&gt; cheaper than computing the full LU decomposition because in general the LU factors of $A$ are dense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then call $P=(\tilde L\tilde U)^{-1}=\tilde U^{-1}\tilde L^{-1}$. In iterative solvers, you then have to &quot;apply&quot; this preconditioner, i.e., you will be given a vector $z$ and asked to return $y=Pz=\tilde U^{-1}\tilde L^{-1}z$. You may think that for this, you will need to know the inverses of $\tilde U,\tilde L$, but that's not true. You can compute it by doing the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Compute $a=\tilde L^{-1}z$ by solving $\tilde La = z$, which is easy because $\tilde L$ is triangular and so this only requires (forward) substitution.&lt;/li&gt;&#xA;&lt;li&gt;Compute $y=Pz=\tilde U^{-1}\tilde L^{-1}z=\tilde U^{-1}a$ by solving $\tilde U y = a$, which is again easy because $\tilde U$ is also triangular and so only requires (backward) substitution.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So once you have $\tilde L, \tilde U$, computing $y=Pz$ is not actually that difficult. Now, whether you call the matrix $P=(\tilde L\tilde U)^{-1}$ and say that $P$ approximates $A^{-1}$, or whether you call it $M^{-1}=(\tilde L\tilde U)^{-1}$ and say that $M$ approximates $A$ does not make a difference -- that's just notation. In either case, we never need to actually know the entries or $P$ or $M$ or their inverses. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-08-03T22:45:52.500" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="29988" PostTypeId="1" AcceptedAnswerId="29992" CreationDate="2018-08-04T12:56:04.360" Score="0" ViewCount="4506" Body="&lt;p&gt;I want to solve a QCQP in Python. It is a problem from finance: maximise return (linear function) given some linear constraints and one quadratic constraint that turns it into a QCQP. Formally,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{ll} \text{maximize} &amp;amp; c^T x\\ \text{subject to} &amp;amp; x^T \Sigma x \le \sigma^2\\ &amp;amp; Ax \le b\end{array}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $\Sigma$ is referred to as the covariance matrix and it is a symmetric positive semidefinite matrix. However, matrix $A$ doesn't need to be. In $A$ we have constraints of the type to bound the sum of subsets of $x$, like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\sum_{j\in J} a_{ij}x_j\le b_i$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $J \subset\{1,\dots,n\}$ with $n$ indicating the dimension of $x$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to solve this is Python. What are reliable packages out there to achieve this? Important is it should be easy to solve it for generic $A$, i.e., one whose dimension (how many constraints) I don't know a priori. For convex problems, I've used &lt;code&gt;cvxopt&lt;/code&gt; which works like a charm for this kind of &quot;dynamic&quot; constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of &lt;code&gt;A&lt;/code&gt; with two rows is given below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    temp = np.asarray([[-0.02  , -0.025 , -0.0275, -0.0325, -0.035 , -0.0525, -0.05  ,&#xA;            -0.025 , -0.0525, -0.0625, -0.0525, -0.055 , -0.0675, -0.0625,&#xA;            -0.08  , -0.08  , -0.06  , -0.0725, -0.0625, -0.0425, -0.0375],&#xA;           [-1.    , -0.    , -0.    , -0.    , -0.    , -0.    , -0.    ,&#xA;            -0.    , -0.    , -0.    , -0.    , -0.    , -0.    , -0.    ,&#xA;            -0.    , -0.    , -0.    , -0.    , -0.    , -0.    , -0.    ]])&#xA;&#xA;and the vector `b`&#xA;&#xA;    In [367]: b = np.asarray([[-0.02],[0.]])&#xA;    Out[367]: &#xA;    array([[-0.02],&#xA;           [ 0.  ]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21291" LastEditorUserId="20417" LastEditDate="2018-08-05T10:04:11.527" LastActivityDate="2018-08-05T10:04:11.527" Title="How to efficiently solve a QCQP with &quot;dynamic&quot; constraints in Python?" Tags="|python|convex-optimization|qcqp|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="29989" PostTypeId="1" CreationDate="2018-08-04T14:14:11.807" Score="1" ViewCount="48" Body="&lt;p&gt;In Finite Difference method or Finite volume textbook. Dispersion/Dissipation can be analyzed by set $u = u_0\exp{\omega t +\mathbf{kx}}$。&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I cannot find something about this kind of analysis in FEM without convert it to a system of difference equations. And FEM is a little more complex because of there are two function spaces and some time they are not continuous. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there example to do it in the framework of FEM? I saw someone used the Eigen value distribution of assembled matrix. &lt;/p&gt;&#xA;" OwnerUserId="24589" LastActivityDate="2018-08-04T14:14:11.807" Title="How to analyze the dispersion and dissipation of a certain FEM?" Tags="|finite-element|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="29990" PostTypeId="2" ParentId="21448" CreationDate="2018-08-04T15:06:24.017" Score="0" Body="&lt;p&gt;Why don't you include constant function $\psi(x)=1,x\in\Omega$ in your test function space? Then if you integrate it with conservative form of your differential equation. You can recover the conservation laws.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think for continuous Galerkin, because the test function space must be zero at boundary, it cannot include constant function. So conservation might be in question. But for DG, the function space includes constant function and the BC is imposed weakly. Conservation is no longer a problem.&lt;/p&gt;&#xA;" OwnerUserId="24589" LastActivityDate="2018-08-04T15:06:24.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29992" PostTypeId="2" ParentId="29988" CreationDate="2018-08-04T20:31:54.247" Score="5" Body="&lt;p&gt;In order for a QCQP to be convex, the quadratic terms needs be be convex.  Linear terms are always convex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regard to your specific problem, the objective function is convex because it is linear.  The quadratic inequality constraint is convex because $\Sigma$ is symmetric positive semi-definite (psd).  All linear equality and inequality constraints are convex, regardless of the matrix involved (which may npot even be square, in which case it can't be symmetric or psd).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your problem is a convex QCQP.  Whether or not you enter it as such, the cvx &quot;family&quot; of modeling systems generally reformulate QCQPs as Second Order Cone Problems (SOCPs), and submit them to a solver as such. In your case, let $C$ = upper triangular Choelsky factor of $\Sigma$ such that $C^TC = \Sigma$, then your quadratic constraint is $\|Cx\|_2 \le \sigma$, which matches the form for an SOCP constraint in cvxopt as shown at &lt;a href=&quot;http://cvxopt.org/userguide/coneprog.html#second-order-cone-programming&quot; rel=&quot;noreferrer&quot;&gt;http://cvxopt.org/userguide/coneprog.html#second-order-cone-programming&lt;/a&gt;. Or you can call cvxopt through cvxpy , which is a higher-level and easier to use modeling interface.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2018-08-04T20:31:54.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29993" PostTypeId="1" AcceptedAnswerId="29994" CreationDate="2018-08-05T02:52:55.797" Score="1" ViewCount="1434" Body="&lt;p&gt;I am working on a 2D unstructured code in C++. I am using &lt;code&gt;gmsh&lt;/code&gt; to generate a 2D unstructured mesh and reading it into my program with a library called &lt;code&gt;GMSH_IO&lt;/code&gt;. Now that I have the data, I would like to write it to a VTK file for visualization in Paraview.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could do this manually, but I am curious if anyone knows of a library for writing unstructured meshes to VTK files for visualization?&lt;/p&gt;&#xA;" OwnerUserId="22717" LastEditorUserId="20688" LastEditDate="2018-08-05T03:07:56.803" LastActivityDate="2020-04-23T17:53:10.677" Title="C++ library unstructured mesh writer to VTK format (or similar)" Tags="|visualization|unstructured-mesh|paraview|gmsh|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29994" PostTypeId="2" ParentId="29993" CreationDate="2018-08-05T03:07:26.983" Score="4" Body="&lt;p&gt;I think you are looking for &lt;a href=&quot;https://github.com/Kitware/VTK&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kitware VTK&lt;/a&gt;, basically, the main library for interaction with VTK files. &lt;a href=&quot;https://www.vtk.org/Wiki/VTK/Examples/Cxx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Examples page&lt;/a&gt; will contain a lot of samples, including the one you are looking for: &lt;a href=&quot;https://lorensen.github.io/VTKExamples/site/Cxx/IO/WriteVTU/&quot; rel=&quot;nofollow noreferrer&quot;&gt;output of an unstructured grid&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an addition, GMSH itself (I am using 3.0.5) is also able to export the mesh into VTK without the need to go through IO procedure. That can be accessed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;File -&gt; Export -&gt; Save As -&gt; Mesh VTK (*.vtk)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bi26X.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bi26X.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-08-05T03:07:26.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29996" PostTypeId="1" CreationDate="2018-08-05T22:03:13.283" Score="8" ViewCount="218" Body="&lt;p&gt;What scientific problems can be simulated with computers that couldn’t be simulated 10 years ago?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m particularly interested in cutting-edge scientific simulations, which wouldn’t have been possible (due to computational constraints, or lack of knowledge) 10 years ago.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please provide links to software, if possible.&lt;/p&gt;&#xA;" OwnerUserId="28384" LastActivityDate="2018-08-06T00:30:16.570" Title="What scientific problems can be simulated that couldn’t be simulated 10 years ago?" Tags="|simulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="29997" PostTypeId="1" AcceptedAnswerId="30012" CreationDate="2018-07-31T16:53:35.257" Score="3" ViewCount="96" Body="&lt;p&gt;Given triples of $n$ floating point values&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(\min_1, \max_1, w_1), \dots, (\min_n, \max_n, w_n)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a value $V$, what is a good algorithhm to assign values $v_i$ to each of the triples such that the following conditions hold?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;$\min_i \le v_i \le \max_i$.&lt;/li&gt;&#xA;&lt;li&gt;$\displaystyle\sum_{i=1}^n v_i = V$.&lt;/li&gt;&#xA;&lt;li&gt;$\dfrac{v_i}{V}$ is as close as possible to $\dfrac{w_i}{W}$, where $W = \displaystyle\sum_{i=1}^n w_i$, i.e., minimize $\displaystyle\sum_{i=1}^n \left| \frac{v_i}{V} - \frac{w_i}{W} \right|$.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I understand the above can be solved via an LP solver but am looking for an algorithm that may not return the optimal assignment but that is deterministic, returns close to the optimum solutions, runs in $O(n)$ time, and returns solutions that are stable in the sense that if two instances of the problem definition are &quot;close to each other&quot; then the solutions tend to be close to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like there should be a greedy approach the performs sufficiently in which the first step is assigning the minima to $v_i$ and then proportioning out the &quot;slack&quot; but I am not sure how to deal with the maxima while doing this.&lt;/p&gt;&#xA;" OwnerUserId="28389" OwnerDisplayName="jwezorek" LastEditorUserId="20417" LastEditDate="2018-08-13T15:56:13.060" LastActivityDate="2018-08-13T15:56:13.060" Title="What is a &quot;good enough&quot; method of assigning values to n variables subject to basic bounding constraints while maintaining relative weights?" Tags="|optimization|algorithms|linear-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="29998" PostTypeId="2" ParentId="29996" CreationDate="2018-08-06T00:30:16.570" Score="5" Body="&lt;p&gt;This report on exascale computing from the DOE might be useful: &lt;a href=&quot;https://science.energy.gov/~/media/ascr/ascac/pdf/reports/Exascale_subcommittee_report.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://science.energy.gov/~/media/ascr/ascac/pdf/reports/Exascale_subcommittee_report.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22717" LastActivityDate="2018-08-06T00:30:16.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30002" PostTypeId="1" CreationDate="2018-08-06T15:13:16.927" Score="1" ViewCount="145" Body="&lt;p&gt;For example, for 1D Burgers equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u u_x  = 0 \\&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;equivalently,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{dF(u)}{dx} = 0\\&#xA;F(u)= \frac{u^2}{2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to obtain $A_{ij},i\ne j$ for two DOFs ($U_i$ and $U_j$) of two elements ($E_0$ and $E_1$) sharing an internal interface $\Gamma_0 $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to substitute $u =\phi_i$ and $v=\phi_j$ to &lt;code&gt;nonlinear-linear&lt;/code&gt; form &#xA;$$\begin{aligned}&#xA;a(u;v)  &amp;amp;= \int_{\Omega}{\nabla F(u)\cdot \nabla v dx} -  \int_{\Gamma_0}{[F(u) v]dS} \\&amp;amp;=\int_{\Omega}{\nabla F(u)\cdot \nabla v dx} -  \int_{\Gamma_0}{\tilde F(u^+,u^-) [v]dS}&#xA;\end{aligned} &#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand what $F(u)$ or $\tilde F(u^+,u^-)$ are in the sense of fluid mechanics and conservation laws. But I am really confusing what $F(\phi_i)$ or $\tilde F(\phi_i^+,\phi_j^-)$ is in the sense of physics and how it is integrated. They are so unnatural. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can read the program codes, but they are all using certain iterations over elements rather than iteration over basis functions. They are indeed correct but I am not sure they are correct for certain $A_{ij},i\ne j$. &lt;/p&gt;&#xA;" OwnerUserId="24589" LastEditorUserId="20688" LastEditDate="2018-08-06T16:22:59.380" LastActivityDate="2018-11-05T17:00:40.457" Title="How is nonlinear flux interface term assembled for Discontinuous Galerkin method for hyperbolic conservation laws?" Tags="|finite-element|discontinuous-galerkin|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30003" PostTypeId="1" CreationDate="2018-08-07T13:08:29.167" Score="5" ViewCount="991" Body="&lt;p&gt;Connected to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/29938/solving-pde-implicitly-or-explicitly-depending-on-stiffness?noredirect=1#comment54828_29938&quot;&gt;this question here on Computational Science&lt;/a&gt;, I've posted a follow-up question on how to solve a PDE using an implicit scheme like Crank-Nicholson in general &lt;a href=&quot;https://stackoverflow.com/questions/51613581/solving-a-system-of-pdes-using-implicit-methods?noredirect=1#comment90192557_51613581&quot;&gt;in this question on SO&lt;/a&gt;.&lt;br&gt;&#xA;But I guess it is better to ask here it is correct to solve a PDE with an implicit step in this way in general and ask the more specific coding issues on SO. So I'll try to split up the question I asked on SO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I've got a non-linear advection-diffusion(-reaction) PDE which I want to solve for &lt;strong&gt;one single timestep&lt;/strong&gt; using &lt;strong&gt;Crank-Nicolson&lt;/strong&gt;. In this example my 1D-PDE is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{T^{n+1} - T^n}{\Delta t} \approx \frac{1}{2} \left(\left(\alpha(T^n) \frac{T^n_{i-1} - 2T^n_{i} + T^n_{i+1}}{\Delta x^2} + v(T^n) \frac{T^n_{i\pm1} - T^n_{i}}{\Delta x}\right) + \left( \alpha(T^{n+1}) \frac{T^{n+1}_{i-1} - 2T^{n+1}_{i} + T^{n+1}_{i+1}}{\Delta x^2} + v(T^{n+1}) \frac{T^{n+1}_{i\pm1} - T^{n+1}_{i}}{\Delta x}\right)\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $\alpha$ is the temperature dependent diffusion coefficient and $v$ is the temperature dependent advection speed. The advection is discretized using an upwind scheme, since when there is advection $|Pe|\geq2$ can be assumed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current program code returns the &lt;strong&gt;derivatives of the PDE for given input temperatures, summing up the specific advection and diffusion derivatives&lt;/strong&gt;. Thus my solver can't &quot;see&quot; the diffusion coefficient and the advection speed. Using a matrix notation to solve the PDE is thus not applicable.&lt;br&gt;&#xA;Furthermore the &lt;strong&gt;derivatives and value (temperature) arrays have the same shape as the calculation grid&lt;/strong&gt;, meaning that for the simplest case of the 1D PDE with, for example 10 nodes, the derivatives and value arrays are vectors with 10 cells (of course neglecting the additional ghost cells I use for fast stencil solving and boundary conditions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivative terms in my PDE are thus in my program code replaced with the following vector derivative where each element in $derivative$ corresponds to the derivative of its respective temperature element in the value array:&#xA;$$derivative(T^n) = \alpha(T^n) \frac{T^n_{i-1} - 2T^n_{i} + T^n_{i+1}}{\Delta x^2} + v(T^n) \frac{T^n_{i\pm1} - T^n_{i}}{\Delta x}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in pseudo-code equation my PDE looks like this:&#xA;$$\frac{T^{n+1} - T^n}{\Delta t} \approx \frac{1}{2} \left(derivative(T^{n}) + derivative(T^{n+1})\right)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To solve this, I reorganize this equation so that &#xA;$$T^{n+1} - T^n - \frac{\Delta t}{2} \left(derivative\left(T^{n}\right) + derivative(T^{n+1})\right) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which in python looks like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def crank_nicolson(y, yprev, h):&#xA;    return (y - yprev - h / 2 * (diff(y) + diff(yprev)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with the stepsize $h = \Delta t$, $y=T^{n+1}$, $yprev=T^{n}$ and $diff = derivative(T)$.&lt;br&gt;&#xA;This can easily be fed to any root finding algorithm like the &lt;code&gt;scipy.optimize.root&lt;/code&gt; module or using any other self-written iterative solver for example like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy import optimize&#xA;&#xA;def diff(T):  # simply example differential function&#xA;    return np.sqrt(T)&#xA;&#xA;def iterate_cranknicolson(y0, h, rtol, diff):  # simple iterative solver&#xA;    y = y0&#xA;    diff_old = diff(y)  # save forward diff&#xA;    f = np.ones_like(y)&#xA;    i = 0&#xA;    while np.any(f &amp;gt; rtol):&#xA;        y_old = y&#xA;        y = y0 + h / 2 * (diff_old + diff(y))  # at the first step: euler forward predictor&#xA;        f = (y / y_old - 1)  # simple stop criterion&#xA;        i += 1&#xA;    return y&#xA;&#xA;# some test runs&#xA;T0 = np.arange(10, 20).astype(float)  # set starting temperature&#xA;&#xA;cn_hybr = optimize.root(crank_nicolson, T0, args=(T0, 1.), method='hybr', tol=1e-6)&#xA;cn_it = iterate_cranknicolson(T0, 1., 1e-6, diff)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;TL,DR. The question(s) summmarized:&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is this approach to iteratively solve an implicit discretization of a PDE correct? With &quot;this approach&quot; I mean especially reorganizing the PDE to $T^{n+1} - T^n - \frac{\Delta t}{2} \left(derivative\left(T^{n}\right) + derivative(T^{n+1})\right) = 0$ and finding the root of this equation.&lt;/li&gt;&#xA;&lt;li&gt;Is there any better way to do it without being able to use matrix notation?&lt;/li&gt;&#xA;&lt;li&gt;Is the derivative of $T^n$ the first column of the Jacobian matrix, so that I could use a modified Newtonian iteration, as proposed &lt;a href=&quot;https://scicomp.stackexchange.com/a/20711/27868&quot;&gt;here&lt;/a&gt; and does this work without having a matrix notation at all?&lt;/li&gt;&#xA;&lt;li&gt;(not really important, thus secondary: Is it ok to use upwind scheme for advection in Crank-Nicolson or should I stick to central differences?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thanks alot in advance! Please tell me if I should refine parts of my question and/or add more detail.&lt;/p&gt;&#xA;" OwnerUserId="27868" LastActivityDate="2023-11-05T22:02:10.313" Title="Solving a PDE implicitly by iteration in python" Tags="|pde|python|iterative-method|implicit-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30004" PostTypeId="2" ParentId="30002" CreationDate="2018-08-07T14:16:37.433" Score="1" Body="&lt;p&gt;I just figured out that the integration must be applied on linearized form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tilde F(u^+,u^-) =  \tilde F((u+\phi_i)^+,(u+\phi_i)^-)&#xA;$$&#xA;and &#xA;$$&#xA;\int_{\Gamma_0}\tilde F(u^+,u^-)[[v]]dx =  \int_{\Gamma_0}\tilde F((u+\phi_i)^+,(u+\phi_i)^-)[[v]]dx  \\&#xA;=  \int_{\Gamma_0}\left( \tilde F(u^+,u^-)+&#xA; \tilde F^{(1)}(u^+,u^-)\phi_i^+ + &#xA;\tilde F^{(2)}(u^+,u^-)\phi_i^-   \right) (\phi_j^+-\phi_j^-)dx&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Derivatives of flux function to each state are needed if the matrix coefficient is needed for implicit solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the integration is represented as $a(\phi_i,\phi_j)+L(\phi_j)$ and can be assembled in a native way.&lt;/p&gt;&#xA;" OwnerUserId="24589" LastActivityDate="2018-08-07T14:16:37.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30005" PostTypeId="2" ParentId="29959" CreationDate="2018-08-07T17:46:53.763" Score="1" Body="&lt;p&gt;There is a nice document:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cmu.edu/biolphys/deserno/pdf/sphere_equi.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;M. Deserno, &quot;How to generate equidistributed points on the surface of a sphere&quot;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;which basically summarizes &lt;strong&gt;the two easiest methods&lt;/strong&gt; to generate a distribution of points on a surface of a sphere:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Uniform random distribution (generate $N$ random $z$ and $\varphi$ in $[-R, R]$ and $[0,2\pi]$, respectively and calculate $(x,y)$ as $(\sqrt{r^2-z^2}\cos{\varphi},\sqrt{r^2-z^2}\sin{\varphi})$. However, with small enough $N$ the outpput does not look too uniform.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Generating circles of latitude with constant intervals with points at almost the same distance from each other (see the paper above). Does not generate EXACTLY the requested number of points, but tries to do it close enough.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A sample Python script I quickly wrote:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python&#xA;import numpy as np &#xA;&#xA;n = 5000&#xA;r = 1&#xA;generateRandom = True&#xA;generateFixed = True&#xA;&#xA;if (generateRandom):&#xA;    print(&quot;Generating randomly %d points on a sphere centered at the origin&quot; % (n))&#xA;    theta = np.random.uniform(0.0,2*np.pi,n)&#xA;    z = np.random.uniform(-1.0,1.0,n)&#xA;    for i in range (0,n):&#xA;        zp = z[i]&#xA;        xp = np.sqrt(r*r - zp*zp)* np.cos(theta[i])&#xA;        yp = np.sqrt(r*r - zp*zp)* np.sin(theta[i])&#xA;if (generateFixed):&#xA;    print(&quot;Generating fixed %d points on a sphere centered at the origin&quot; % (n))&#xA;    alpha = 4.0*np.pi*r*r/n&#xA;    d = np.sqrt(alpha)&#xA;    m_nu = int(np.round(np.pi/d))&#xA;    d_nu = np.pi/m_nu&#xA;    d_phi = alpha/d_nu&#xA;    count = 0&#xA;    for m in range (0,m_nu):&#xA;        nu = np.pi*(m+0.5)/m_nu&#xA;        m_phi = int(np.round(2*np.pi*np.sin(nu)/d_phi))&#xA;        for n in range (0,m_phi):&#xA;            phi = 2*np.pi*n/m_phi&#xA;            xp = r*np.sin(nu)*np.cos(phi)&#xA;            yp = r*np.sin(nu)*np.sin(phi)&#xA;            zp = r*np.cos(nu)&#xA;            count = count +1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IrOnp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IrOnp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-08-07T17:46:53.763" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30009" PostTypeId="1" AcceptedAnswerId="30021" CreationDate="2018-08-08T15:54:27.353" Score="4" ViewCount="205" Body="&lt;p&gt;Assume one has a triangular sparse matrix and want to solve $Lx=b$ where $b$ and $L$ are known. This can be done easily by using forward substitution when $L$ is a lower triangular matrix. Forward substitution is highly sequential and hard to implement in parallel.&#xA;I have read some articles about level scheduling where $L$ is reordered so that different levels appear which can be solved in parallel. A good reference seems to be the book of Saad where also the forward substitution with level scheduling is explained. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I do not really get how the level scheduling is performed and the vector $q$ is filled. Could someone please provide an example or something?&lt;/p&gt;&#xA;" OwnerUserId="28358" LastEditorUserId="20688" LastEditDate="2018-08-09T21:05:42.547" LastActivityDate="2018-08-10T19:58:59.473" Title="Level scheduling of triangular sparse matrices" Tags="|linear-algebra|parallel-computing|sparse-matrix|" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30010" PostTypeId="1" CreationDate="2018-08-08T16:11:18.487" Score="2" ViewCount="138" Body="&lt;p&gt;I'm interested to derive the total energy balance from Chapman-Enskog analysis of lattice Boltzmann equation (LBE). I know, I should go to the second moment of LBE (zeroth moment gives mass conservation and first moment gives momentum conservation). In fact, I derived the main equation for the second moment at convective ($t_{1}$) and diffusive ($t_{2}$) time scales as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From Chapman-Enskog analysis up to the second order:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convective time scale: $\frac{\partial f_{i}^{eq}}{\partial t_{1}} + \vec{e_{i}} \cdot \nabla_{1} f_{i}^{eq} = -\frac{f_{i}^{(1)}}{\tau}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Diffusive time scale: $\frac{\partial f_{i}^{eq}}{\partial t_{2}} + (1-\frac{1}{2 \tau}) \Big [ \frac{\partial f_{i}^{(1)}}{\partial t_{1}} + \vec{e_{i}} \cdot \nabla_{1} f_{i}^{(1)} \Big ] = -\frac{f_{i}^{(2)}}{\tau}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $f_{i}^{eq}$ is the equilibrium distribution function (expanded Maxwell-Boltzmann up to third order):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_{i}^{eq} = \rho \omega_{i} (1 + \frac{\vec{e_{i}} \cdot \mathbf{u}}{c_{s}^{2}} + \frac{(\vec{e_{i}} \otimes \vec{e_{i}} - \mathbf{I})\cdot \mathbf{u} \otimes \mathbf{u}}{2 c_{s}^{4}} + \frac{(\vec{e_{i}} \otimes \vec{e_{i}} \otimes \vec{e_{i}} - 3 \vec{e_{i}} \otimes \mathbf{I}) \cdot \mathbf{u} \otimes \mathbf{u} \otimes \mathbf{u}}{6 c_{s}^{6}})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_{i}^{(1)}$ and $f_{i}^{(2)}$ are the first and second order terms of distribution function Taylor's expansion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know the total energy and its flux could be derived as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho e = \rho (\frac{3}{2} U^{2} + \frac{1}{2} |\mathbf{u}|^{2}) = \sum_{i} \frac{1}{2} \vec{e_{i}} \cdot \vec{e_{i}} f_{i}^{eq}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\rho e \mathbf{u} = \rho (\frac{3}{2} U^{2} + \frac{1}{2} |\mathbf{u}|^{2}) \mathbf{u} = \sum_{i} \frac{1}{2} \vec{e_{i}} \cdot \vec{e_{i}} \vec{e_{i}} f_{i}^{eq}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convective time scale :$\frac{\partial}{\partial t_{1}} (\rho e) + \nabla_{1} \cdot (\rho e \mathbf{u}) = -\frac{1}{\tau} \sum_{i} \frac{1}{2} f_{i}^{(1)} \vec{e_{i}} \cdot \vec{e_{i}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Diffusive time scale: $\frac{\partial}{\partial t_{2}} (\rho e) + (1-\frac{1}{2 \tau}) \frac{\partial}{\partial t_{1}} (\sum_{i} \frac{1}{2} \vec{e_{i}} \cdot \vec{e_{i}} f_{i}^{(1)}) + (1-\frac{1}{2 \tau}) \nabla_{1} \cdot (\sum_{i} \frac{1}{2} \vec{e_{i}} \cdot \vec{e_{i}} \vec{e_{i}} f_{i}^{(1)}) = -\frac{1}{\tau} \sum_{i} \frac{1}{2} f_{i}^{(2)} \vec{e_{i}} \cdot \vec{e_{i}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The terms $\sum_{i} \frac{1}{2} f_{i}^{(1)} \vec{e_{i}} \cdot \vec{e_{i}}$ and $\sum_{i} \frac{1}{2} f_{i}^{(2)} \vec{e_{i}} \cdot \vec{e_{i}}$ are equal to zero because energy is a conserved quantity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convective time scale :$\frac{\partial}{\partial t_{1}} (\rho e) + \nabla_{1} \cdot (\rho e \mathbf{u}) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Diffusive time scale: $\frac{\partial}{\partial t_{2}} (\rho e) + (1-\frac{1}{2 \tau}) \nabla_{1} \cdot (\sum_{i} \frac{1}{2} \vec{e_{i}} \cdot \vec{e_{i}} \vec{e_{i}} f_{i}^{(1)}) = 0$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, my question is how can I simplify the term $(1-\frac{1}{2 \tau}) \sum_{i} \frac{1}{2} \vec{e_{i}} \cdot \vec{e_{i}} \vec{e_{i}} f_{i}^{(1)}$ to prove that its divergence is equal to $\nabla \cdot \mathbf{q} - \nabla \cdot (\sigma \cdot \mathbf{u})$, where $\mathbf{q}$ is the heat flux and $\sigma$ is the Cauchy stress tensor:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\nabla_{1} \cdot ((1-\frac{1}{2 \tau}) \sum_{i} \frac{1}{2} \vec{e_{i}} \cdot \vec{e_{i}} \vec{e_{i}} f_{i}^{(1)}) = \nabla \cdot \mathbf{q} - \nabla \cdot (\sigma \cdot \mathbf{u})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, we could find the total energy balance equation as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{\partial}{\partial t} (\rho e) + \nabla \cdot (\rho e \mathbf{u}) = -\nabla \cdot \mathbf{q} + \nabla \cdot (\sigma \cdot \mathbf{u})$&lt;/p&gt;&#xA;" OwnerUserId="28407" LastEditorUserId="28407" LastEditDate="2018-08-11T15:44:20.513" LastActivityDate="2018-08-11T15:44:20.513" Title="Derive total energy balance equation from Chapman-Enskog analysis of lattice Boltzmann equation" Tags="|navier-stokes|lattice-boltzmann-methods|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30011" PostTypeId="1" CreationDate="2018-08-08T16:26:01.827" Score="2" ViewCount="32" Body="&lt;p&gt;This is a problem I am encountering in a research project. I have thought of a naive approach to solve it which relies on many assumptions, is not guaranteed to work, and would be tedious and computationally expensive to implement. Hence, I am wondering if there would be cleverer or more general approaches. Below I will try to explain my problem as clearly as possible.&lt;/p&gt;&#xA;&lt;p&gt;Tree growth at day &lt;em&gt;d&lt;/em&gt; is thought of as a function of meteorology that has happened in the past. For instance, tree growth on a given day &lt;em&gt;d&lt;/em&gt; may depend, on the cumulative temperature between day &lt;em&gt;d-1&lt;/em&gt; and day &lt;em&gt;d-20&lt;/em&gt; but probably not on the temperature a year before. Likewise for all the other meteorological variables.&lt;/p&gt;&#xA;&lt;p&gt;I have daily meteorological data and daily tree growth data measured in number of cells created.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;For any given meteorological variable, I would like to know for which period in the past the variable influences tree growth the most (in terms of daily creation of cells).&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Naive approach:&lt;/h1&gt;&#xA;&lt;p&gt;Let's choose any meteorological variable, like temperature.&lt;/p&gt;&#xA;&lt;p&gt;I have daily mean temperature data &lt;span class=&quot;math-container&quot;&gt;$T(d)$&lt;/span&gt; and daily cell creation data &lt;span class=&quot;math-container&quot;&gt;$C(d)$&lt;/span&gt;. These are discrete functions but I can extend them to all time by interpolation.&lt;/p&gt;&#xA;&lt;p&gt;Solving the problem is equivalent to finding values &lt;span class=&quot;math-container&quot;&gt;$0 &amp;lt; \lambda \leq \mu$&lt;/span&gt; which maximise the correlation&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$K(\mu,\lambda) = cor(\: \frac{1}{2 \lambda}\int_{(d - \mu) - \lambda}^{(d - \mu) +\lambda} T(t) \: dt,\quad C(d)\:)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$2\lambda$&lt;/span&gt; would then be length of the time interval which most influences growth and &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; the number of days in the past at the center of the interval.&lt;/p&gt;&#xA;&lt;p&gt;We can refine this further. There is probably a sweet spot in the past for which temperature particularly influences present growth. Conversely, temperatures further away in time from that sweet spot influence present growth less. To take this into account we can weigh the integration with a normal distribution.&lt;/p&gt;&#xA;&lt;p&gt;Solving the problem then becomes maximising&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\tilde{K}(\mu, \sigma, \lambda) = cor( \: \frac{\int_{(d-\mu) - \lambda}^{(d-\mu) +\lambda} T(t)\:F_{d-\mu, \sigma}(t) \: dt}{\int_{(d-\mu) - \lambda}^{(d-\mu) +\lambda} F_{d-\mu, \sigma}(t) \: dt},\:  C(d)\:)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for &lt;span class=&quot;math-container&quot;&gt;$\sigma \geq 0, \: \mu \geq \lambda \geq 0 $&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$F_{d-\mu, \sigma}$&lt;/span&gt; is the density function of a normal distribution of mean &lt;span class=&quot;math-container&quot;&gt;$d-\mu$&lt;/span&gt; and standard deviation &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Lastly, I am coding in R and the correlation methods available are &amp;quot;Pearson&amp;quot; (the usual one), &amp;quot;Spearman&amp;quot; and &amp;quot;Kendall&amp;quot;. Pearson mostly measures linear relationships. However, I would not expect growth to be linearly dependent with the temperature. There could be a polynomial relation with optimum temperatures causing maximum growth and temperatures far from that optimum causing less or no growth. For that reason I was going to use Spearman correlation which better captures non-linear relationships.&lt;/p&gt;&#xA;&lt;p&gt;With the above approach I need to assume that the meteorological influences (e.g. temperature and rainfall) are not dependent, but they probably are. I am treating each meteorological factor separately for now.&lt;/p&gt;&#xA;&lt;p&gt;Going back to questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do you have any thoughts on how I could tackle this problem?&lt;/li&gt;&#xA;&lt;li&gt;Have you spotted significant flaws in this method that could be addressed?&lt;/li&gt;&#xA;&lt;li&gt;Do you know of better methods to solve this problem?&lt;/li&gt;&#xA;&lt;li&gt;Do you know of similar problems that have already been solved, which I could look into for a solution?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thanks a lot for your help!&lt;/p&gt;&#xA;" OwnerUserId="28408" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-08-08T16:26:01.827" Title="Identifying the time window at which a meteorological factor influences tree growth the most" Tags="|statistics|data-analysis|computational-biology|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30012" PostTypeId="2" ParentId="29997" CreationDate="2018-08-08T16:42:38.903" Score="2" Body="&lt;p&gt;(This answer is the work of &lt;a href=&quot;https://www.quora.com/profile/Michal-Fori%C5%A1ek&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michal Forišek&lt;/a&gt; ... I will paraphrase here)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First reformulate the problem by nomalizing the constraints and weights yielding:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\sum v_i = 1$&lt;/li&gt;&#xA;&lt;li&gt;$\forall i: \min_i \leq v_i \leq \max_i$&lt;/li&gt;&#xA;&lt;li&gt;minimize $\sum |v_i - w_i |$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that the problem is only solvable if  $\sum \min_i \leq 1 \leq \sum \max_i$.&#xA;Then find an optimal solution as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Set each $v_i$ equal to $w_i$. If this is a valid solution, we are done.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each $i$, if $v_i  &amp;lt; \min_i$, increment $v_i$ to $\min_i$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each $i$, if $v_i &amp;gt; \max_i$, decrement $v_i$ to $\max_i$. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;While $\sum v_i &amp;lt; 1$, find the smallest $x$ such that $v_x$ that can be incremented and increment it until you either get $\sum v_i = 1$ or $v_x = \max_x$.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;While $\sum v_i &amp;gt; 1$, do the same but decrement.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The above will be optimal but Michal notes that one might get intuitively better results by minimizing the sum of squares rather than of absolute values, which is possible to do by modifying 4. and 5. above.&lt;/p&gt;&#xA;" OwnerUserId="28389" LastEditorUserId="28389" LastEditDate="2018-08-08T19:05:27.677" LastActivityDate="2018-08-08T19:05:27.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30013" PostTypeId="1" CreationDate="2018-08-08T17:47:22.480" Score="4" ViewCount="4040" Body="&lt;p&gt;I am trying to solve numerically a constrained optimisation problem in MATLAB, and I am wondering how the &lt;code&gt;fmincon&lt;/code&gt; function calculates gradients when one isn't provided. Does anyone here know, or know how I might be able to find out?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running the optimisation problem takes more time than I'd like it to, so I was hoping to speed it up by providing the gradient analytically. However, when I do this, I end up with wildly different solutions that seem less plausible than the solution that MATLAB generates when I do not provide the gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a check, I used the &lt;code&gt;CheckGradients&lt;/code&gt; option in &lt;code&gt;fmincon&lt;/code&gt;. Predictably, the gradient I provided did not pass this test. The same happens even when I set &lt;code&gt;FiniteDifferenceType&lt;/code&gt; to &lt;code&gt;'central'&lt;/code&gt;. One obvious explanation is that the derivatives I provided truly are incorrect. However, I've gone over them several times and I'm fairly certain they are not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a sanity check, I tried to calculate the gradient of my objective numerically, using &lt;a href=&quot;https://uk.mathworks.com/help/matlab/ref/gradient.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;gradient&lt;/code&gt;&lt;/a&gt;, which the documentation suggests is calculated using finite differences. Unfortunately, the output of &lt;code&gt;gradient&lt;/code&gt; is &lt;em&gt;nowhere near&lt;/em&gt; the gradient calculated by &lt;code&gt;fmincon&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm really not sure what's going on, and I'd appreciate it if anyone can help shed light on this situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I'm more interested in why &lt;code&gt;fmincon&lt;/code&gt; and &lt;code&gt;gradient&lt;/code&gt; produce different numerical derivatives, despite ostensibly both being calculated using finite differences. Unless I've misunderstood the options, the difference persists even when I give them the same finite difference step size.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Also, in case it's relevant: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am actually using &lt;code&gt;GlobalSearch&lt;/code&gt; (which then calls &lt;code&gt;fmincon&lt;/code&gt;) to solve a constrained optimisation problem of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation*}&#xA;\begin{aligned}&#xA;&amp;amp; \underset{\mathbf p, \mathbf q}{\max}&#xA;&amp;amp; &amp;amp; V(\mathbf p, \mathbf q) - C(\mathbf p, \mathbf q) \\&#xA;&amp;amp; \ \ \text{s.t.}&#xA;&amp;amp; &amp;amp; \sum_{i=1}^n p_i = 1 \\&#xA;&amp;amp; &amp;amp; &amp;amp; \sum_{i=1}^n q_i = 1.&#xA;\end{aligned}&#xA;\end{equation*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$V(\mathbf p, \mathbf q)$ is actually the value function of some linear programming problem, and I've written a script that invokes &lt;code&gt;linprog&lt;/code&gt; to calculate the value of my objective. $(\mathbf p, \mathbf q)$ also enters linearly into the objective and constraints in that problem. $C$, however, is non-linear.&lt;/p&gt;&#xA;" OwnerUserId="22591" LastEditorUserId="22591" LastEditDate="2018-08-10T15:11:51.990" LastActivityDate="2018-08-10T15:11:51.990" Title="How does fmincon in MATLAB calculate gradients?" Tags="|optimization|matlab|constrained-optimization|numerics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30014" PostTypeId="2" ParentId="29959" CreationDate="2018-08-09T08:39:01.977" Score="4" Body="&lt;p&gt;To generate a specified number of random points in a 3D ball, one possibility is to use rejection sampling to have an initial point distribution, then improve the sampling with Lloyd relaxation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K3ary.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K3ary.png&quot; alt=&quot;Examples&quot;&gt;&lt;/a&gt;&#xA;&lt;em&gt;Examples with 30, 300 and 10000 points&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rejection sampling:&lt;/strong&gt;&#xA;Generate x,y,z coordinates using a uniform random number generator, ignore points that are outside the ball, and continue until you get the required number of points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Lloyd relaxation&lt;/strong&gt;&#xA;It is an iterative method that makes the distribution of points more regular. Each iteration works as follow: compute the Voronoi diagram of the points, then move each point to the barycenter of its Voronoi cell. See [1] for more detailed explanations.&#xA;There is an implementation in my Geogram software library [2]. With Geogram, you need to use a tetrahedralized version of the sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In volume / on surface&lt;/strong&gt;&#xA;There is also a version of the algorithm that works on a 3D surface (sphere or arbitrary surface), called Constrained Centroidal Voronoi Tesselation [3],&#xA;also implemented in Geogram [2].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also use my software Graphite [4] (graphic user interface around geogram). To sample a sphere in Graphite:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Scene-&amp;gt;Create object-&amp;gt;OGF::MeshGrob&#xA;Surface-&amp;gt;Shapes-&amp;gt;Create sphere&#xA;Points-&amp;gt;Sample volume or Points-&amp;gt;Sample surface&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0wx4u.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0wx4u.gif&quot; alt=&quot;minitutorial&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;minitutorial: sampling a ball in Graphite&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://en.wikipedia.org/wiki/Lloyd%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Lloyd%27s_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://alice.loria.fr/software/geogram/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/geogram/doc/html/index.html&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] Constrained Centroidal Voronoi Tesselation, Du, Gunzburger and Ju, SIAM J. on computing, 2003&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;http://alice.loria.fr/software/graphite/doc/html/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://alice.loria.fr/software/graphite/doc/html/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2018-08-09T09:27:22.093" LastActivityDate="2018-08-09T09:27:22.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30015" PostTypeId="2" ParentId="30013" CreationDate="2018-08-09T10:36:05.197" Score="3" Body="&lt;p&gt;The &lt;code&gt;fmincon&lt;/code&gt; &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/fmincon.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation &lt;/a&gt; is fairly clear on HOW it calculates gradients.&#xA;Specifically, the documentation for the &lt;code&gt;FiniteDifferenceType&lt;/code&gt; and &#xA;&lt;code&gt;FiniteDifferenceStepSize&lt;/code&gt; options explain this in some detail. &lt;code&gt;fmincon&lt;/code&gt; is using&#xA;either forward (default) or central difference formulas with the step size selected&#xA;according to the documentation for &lt;code&gt;FiniteDifferenceStepSize&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the relevant question is not HOW are they calculated but why do the gradients&#xA;calculated by finite difference differ so significantly from those calculated&#xA;from an analytical expression? Usually this is caused by the finite difference&#xA;step size being either too large or too small for the function being numerically&#xA;&quot;differentiated.&quot; The problem with a too-large step size is obvious. The&#xA;problem with a too-small step size is that roundoff error makes the calculation&#xA;unreliable. Some experimentation with different step sizes is often needed to&#xA;find a value that is appropriate for a particular function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is explained in more detail in this paper by&#xA;&lt;a href=&quot;https://ntrs.nasa.gov/search.jsp?R=19850025225&quot; rel=&quot;nofollow noreferrer&quot;&gt; Iott and Haftka &lt;/a&gt; where they&#xA;discuss an approach for step size selection. &lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2018-08-09T10:36:05.197" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30016" PostTypeId="1" CreationDate="2018-08-09T11:19:43.960" Score="5" ViewCount="162" Body="&lt;p&gt;The Navier--Stokes equations are axially symmetric, so with symmetric boundary conditions, how can features such as &lt;a href=&quot;https://en.wikipedia.org/wiki/K%C3%A1rm%C3%A1n_vortex_street&quot; rel=&quot;noreferrer&quot;&gt;Karman vortex streets&lt;/a&gt; develop?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that in reality symmetry does never completely hold due to minimal fluctuations of geometries, densities, etc., but in the linked article, vortex streets develop even in numerical simulations&lt;/p&gt;&#xA;" OwnerUserId="7789" LastActivityDate="2018-08-09T11:19:43.960" Title="How can Navier--Stokes equations have asymmetric solutions such as Karman vortex streets" Tags="|simulation|navier-stokes|symmetry|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="30017" PostTypeId="1" CreationDate="2018-08-09T14:41:53.720" Score="14" ViewCount="1667" Body="&lt;p&gt;I would like to use automatic differentiation to calculate gradients to function written in numpy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've come across a number of packages, including&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/HIPS/autograd&quot; rel=&quot;noreferrer&quot;&gt;autograd&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/google/tangent&quot; rel=&quot;noreferrer&quot;&gt;tangent&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/chainer/chainer&quot; rel=&quot;noreferrer&quot;&gt;chainer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But none of them seem to support things like numba and numexpr, which I'd normally use to accelerate my python code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What packages do people use for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. I know there's also stuff like tensorflow and pytorch, but I would like to keep my code in numpy&lt;/p&gt;&#xA;" OwnerUserId="21499" LastActivityDate="2021-10-27T20:52:28.893" Title="Fast Automatic Differentiation for numpy?" Tags="|numpy|automatic-differentiation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30018" PostTypeId="1" AcceptedAnswerId="30019" CreationDate="2018-08-09T18:50:07.700" Score="3" ViewCount="1679" Body="&lt;p&gt;I'm new to SOCP and want to try to get familiar with the format and how to solve it with &lt;code&gt;cvxopt&lt;/code&gt; in python. However, for a simple toy example I'm struggling to get the right input format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem I want to solve has the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \max_x c^T x$$&#xA;subject to&#xA;$$ \| A_ix + b_i\|_2\le c_i^Tx + d_i, i=1,\dots, m$$&#xA;$$Fx = g $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example I came up with has the following parameters: $c=(0.02, 0.06)$. Additionally I'm given a symmetric positive semi-definite matrix $$\Sigma=\begin{bmatrix}&#xA;    0.000025 &amp;amp; 0.000046\\&#xA;    0.000046&amp;amp; 0.024025 &#xA;\end{bmatrix}&#xA;$$ &#xA;and have the constraints:&#xA;$$ \sqrt{x^T\Sigma x} \le d_{max}=0.05$$&#xA;$$ 0\le x_i \le 1, i = 1, 2$$&#xA;$$ \sum_{i=1}^2 x_i = 1$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then first transformed the constrained into the proper SOCP format. In a next step I want to make them cvxopt acceptable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Transform to SOCP&lt;/strong&gt; The constraint $$ \sum_{i=1}^2 x_i = 1$$ is simple by taking &#xA;$F=\begin{bmatrix}&#xA;    1       &amp;amp; 1&#xA;\end{bmatrix}$ and $ g = 1$. Next, for the constraint $ \sqrt{x^T\Sigma x} \le d_{max}$ we use that for a positive semi-definite matrix $ \sqrt{x^T\Sigma x} = \|\Sigma^{\frac{1}{2}}x\|_2$ where $\Sigma^{\frac{1}{2}}$ denotes the Cholesky decomposition. This means, we have $A_1 = \Sigma^{\frac{1}{2}}$, $b_1=0$, $c_1 = 0$ and $d_1 = d_{max}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last constraints of the type $ 0\le x_i \le 1, i = 1, 2$ can be converted into a proper SOCP format by taking $A_2 = 0, b_2 =0, c_2=(-1, 0)$ and $d_2 = 1$. This gives $ x_1 \le 1$. To get $0 \le x_1$ we take $A_3 = 0, b_3 =0, c_3=(1, 0)$ and $d_3 = 0$. We get two additional constraints for $x_2$ in the same way with changing $c_4 = (0, -1)$ and $c_5=(0, 1)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Transform SCOP to &lt;code&gt;cvxopt&lt;/code&gt; format&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;cvxopt&lt;/code&gt; has slightly different format. The objective function and equality constraint is the same. However, the second order constraints have the &lt;a href=&quot;http://cvxopt.org/userguide/coneprog.html#second-order-cone-programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;form&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ G_kx + s_k = h_k$$&#xA;$$s_{k0}\ge \|s_{k1}\|_2$$&#xA;where $s_k=(s_{k0}, s_{k1})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solver needs as input the list of the matrices $G_k$ and vectors $h_k$. If I'm not wrong I transfer each of the constraints $\| A_ix + b_i\|_2\le c_i^Tx + d_i$ to this format by taking&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ G_i =\begin{bmatrix}&#xA;    -c_i       \\ &#xA;-A_i&#xA;\end{bmatrix}$$ and&#xA;$$ h_i = \begin{bmatrix}&#xA;    d_i      \\ &#xA;b_i&#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Python toy model&lt;/strong&gt; I've wanted to test this in python. Unfortunately, it doesn't work as expected and I don't see exactly what is wrong. I'm not sure if it is the implementation or one of the transformations above. Here is the python toy example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [201]: import numpy as np&#xA;&#xA;In [202]: import cvxopt as cpt&#xA;&#xA;In [203]: Sigma = np.asarray([[0.000025, 0.000046],[0.00046, 0.024025]])&#xA;&#xA;In [205]: np.linalg.eig(Sigma)&#xA;Out[205]: &#xA;(array([2.41183657e-05, 2.40258816e-02]), array([[-0.99981638, -0.00191659],&#xA;        [ 0.01916244, -0.99999816]]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see the the matrix is really positive semi-definite.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [37]: c = cpt.matrix(np.asarray([[0.02, 0.06]]).T)&#xA;&#xA;In [38]: F = cpt.matrix(np.asarray([[1.0, 1.]]), tc='d')&#xA;&#xA;In [39]: g = cpt.matrix(np.asarray([[1.0]]), tc='d')&#xA;&#xA;In [40]: A0 = cpt.matrix(np.linalg.cholesky(Sigma))&#xA;&#xA;In [41]: G1 = cpt.matrix(np.vstack(([[0, 0]],-A0)), tc='d')&#xA;&#xA;In [42]: h1 = cpt.matrix(np.asarray([[0.05],[0.0], [0.0]]), tc='d')&#xA;&#xA;In [43]: G2 = cpt.matrix(np.asarray([[-1, 0],[0, 0]]), tc='d')&#xA;&#xA;In [44]: h2 = cpt.matrix(np.asarray([[1.0],[0.0]]), tc='d')&#xA;&#xA;In [45]: G3 = cpt.matrix(np.asarray([[1, 0],[0, 0]]), tc='d')&#xA;&#xA;In [46]: h3 = cpt.matrix(np.asarray([[0.0],[0.0]]), tc='d')&#xA;&#xA;In [47]: G4 = cpt.matrix(np.asarray([[0, -1],[0, 0]]), tc='d')&#xA;&#xA;In [48]: h4 = cpt.matrix(np.asarray([[1.0],[0.0]]), tc='d')&#xA;&#xA;In [49]: G5 = cpt.matrix(np.asarray([[0, 1],[0, 0]]), tc='d')&#xA;&#xA;In [50]: h5 = cpt.matrix(np.asarray([[0.0],[0.0]]), tc='d')&#xA;&#xA;In [51]: Gq = [G1]&#xA;&#xA;In [52]: [Gq.append(Gi) for Gi in [G2, G3, G4, G5]]&#xA;&#xA;In [53]: hq = [h1]&#xA;&#xA;In [54]: [hq.append(hi) for hi in [h2, h3, h4, h5]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Gq&lt;/code&gt; and &lt;code&gt;hq&lt;/code&gt; are lists which is required by &lt;code&gt;cvxopt.solvers.scop&lt;/code&gt;. If I know call the solver I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [55]: cpt.solvers.socp(-c, Gq=Gq, hq=hq, A=F, b=g)&#xA;     pcost       dcost       gap    pres   dres   k/t&#xA; 0: -3.9965e-02 -2.1105e+00  1e+01  2e+00  3e-16  1e+00&#xA; 1: -3.9369e-02 -1.3687e-02  5e+00  8e-01  1e-15  1e+00&#xA; 2: -3.3610e-02  1.0744e+02  1e+03  2e+00  2e-13  1e+02&#xA; 3: -3.3610e-02  1.0935e+04  1e+05  2e+00  4e-12  1e+04&#xA; 4: -3.3610e-02  1.0936e+06  1e+07  2e+00  1e-09  1e+06&#xA;Certificate of primal infeasibility found.&#xA;Out[55]: &#xA;{'dual infeasibility': None,&#xA; 'dual objective': 1.0,&#xA; 'dual slack': 0.9204046569064762,&#xA; 'gap': None,&#xA; 'iterations': 4,&#xA; 'primal infeasibility': None,&#xA; 'primal objective': None,&#xA; 'primal slack': None,&#xA; 'relative gap': None,&#xA; 'residual as dual infeasibility certificate': None,&#xA; 'residual as primal infeasibility certificate': 5.782986313941418e-08,&#xA; 'sl': None,&#xA; 'sq': None,&#xA; 'status': 'primal infeasible',&#xA; 'x': None,&#xA; 'y': &amp;lt;1x1 matrix, tc='d'&amp;gt;,&#xA; 'zl': &amp;lt;0x1 matrix, tc='d'&amp;gt;,&#xA; 'zq': [&amp;lt;3x1 matrix, tc='d'&amp;gt;,&#xA;  &amp;lt;2x1 matrix, tc='d'&amp;gt;,&#xA;  &amp;lt;2x1 matrix, tc='d'&amp;gt;,&#xA;  &amp;lt;2x1 matrix, tc='d'&amp;gt;,&#xA;  &amp;lt;2x1 matrix, tc='d'&amp;gt;]}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't understand why. Even if I use $d_{max}=0.2$ we should get a solution of $x_1 = 0, x_2=1$. Any help would be much appreciated. Please note that I want to use &lt;code&gt;cvxopt&lt;/code&gt; and I'm thankful but not interested in answers / comments suggesting to use another module like &lt;code&gt;cvxpy&lt;/code&gt;. I have a restriction for real cases I want to work on, after finishing some of the basic examples, to use &lt;code&gt;cvxopt&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21291" LastActivityDate="2018-08-12T09:04:16.837" Title="How to transform this SOCP to the format required by cvxopt" Tags="|python|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30019" PostTypeId="2" ParentId="30018" CreationDate="2018-08-09T21:09:43.573" Score="5" Body="&lt;p&gt;Instead of direclty answering your question, I will propose to use a, so called, modeling language for optimization problems, which allows to formulate your problem in a natural way, and have the language translate it to the appropriate format required by solvers such as &lt;code&gt;CVXOPT&lt;/code&gt;. The code below solves your toy problem using the &lt;a href=&quot;http://www.cvxpy.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;cvxpy&lt;/code&gt; package&lt;/a&gt;, which is compatible with many solvers, including &lt;code&gt;CVXOPT&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import cvxpy&#xA;from scipy.linalg import sqrtm  # for finding the squared root of Sigma&#xA;&#xA;x = cvxpy.Variable(2) # optimization vector variable&#xA;c = cvxpy.Parameter(2) # placeholder for vector c&#xA;A = cvxpy.Parameter((2,2))  # placeholder for sqrtm(Sigma)&#xA;dmax = cvxpy.Parameter() # placeholder for parameter dmax&#xA;&#xA;obj = cvxpy.Maximize(cvxpy.sum(x*c))  #define objective function &#xA;cons = [cvxpy.norm(A@x,2)&amp;lt;=dmax,      # define set of constraints&#xA;        cvxpy.sum(x)==1, &#xA;        x[0]&amp;gt;=0, x[1]&amp;gt;=0,&#xA;        x[0]&amp;lt;=1, x[1]&amp;lt;=1]&#xA;&#xA;prob = cvxpy.Problem(obj, cons)  #setup the problem&#xA;&#xA;# instantiate the problem for a specific value of parameters&#xA;A.value = sqrtm(np.asarray([[0.000025, 0.000046], [0.000046, 0.024025]]))&#xA;c.value = np.asarray([0.02, 0.06])&#xA;dmax.value = 0.05&#xA;&#xA;prob.solve()  # solve the problem&#xA;&#xA;x_opt = x.value # the optimal variable&#xA;&#xA;print(x_opt, np.linalg.norm(A.value @ x_opt), np.sum(x_opt) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;[0.6795 0.3205] 0.049999998463030305 0.9999999999999976&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note how simple and natural is the problem definition. No need to know the technicalities of the solver(s). Here, &lt;code&gt;cvxopt&lt;/code&gt; chooses automatically the &quot;best&quot; solver (out of those available in your system and supported by &lt;code&gt;cvxpy&lt;/code&gt;) for the problem. You may explicitly indicate the solver (e.g., &lt;code&gt;CVXOPT&lt;/code&gt;), if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: &#xA;Here's how you can automatically generate the domain constraints instead of manually writing them (helpful when there are many of them and/or you don't know dimensions beforehand)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# define the `cons` python list with cvxpy constraints you can specify before runtime (it may as well be cons=[])&#xA;cons = [cvxpy.norm(A@x,2)&amp;lt;=dmax, cvxpy.sum(x)==1]&#xA;&#xA;# generate domain constraints on the fly:&#xA;# N is the dimension, possibly available only at runtime; for the toy example, N = 2&#xA;for i in range(N):&#xA;     cons = cons + [x[i]&amp;gt;=0, x[i]&amp;lt;=1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21460" LastEditorUserId="21460" LastEditDate="2018-08-10T10:08:21.527" LastActivityDate="2018-08-10T10:08:21.527" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="30020" PostTypeId="2" ParentId="28846" CreationDate="2018-08-10T19:49:56.947" Score="2" Body="&lt;p&gt;Have you tried any of the Coin-OR tools, like cbc or clp? They are pretty comparable to CPLEX for LP (not MIPs though), at least until a certain scale of the problem (your size should be solvable quickly using these). They claim to have support for Dantzig-Wolfe, although I haven't tried that.&lt;/p&gt;&#xA;" OwnerUserId="28429" LastActivityDate="2018-08-10T19:49:56.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30021" PostTypeId="2" ParentId="30009" CreationDate="2018-08-10T19:58:59.473" Score="2" Body="&lt;p&gt;You can try to discover parallelism in L and U, but a better approach is to reorder the original matrix. Luby &amp;amp; Jones/Plassman describe this scheme: you can generate a multi-colouring in parallel, and then in the solution every colour has all processors concurrently active.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2018-08-10T19:58:59.473" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30022" PostTypeId="1" AcceptedAnswerId="30043" CreationDate="2018-08-10T20:25:52.610" Score="1" ViewCount="232" Body="&lt;p&gt;I'd like to find an algorithm that can solve the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider 4 groups of numbers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Group 1: [10, 100, 1000],&lt;/li&gt;&#xA;&lt;li&gt;Group 2: [101, 15, 2000],&lt;/li&gt;&#xA;&lt;li&gt;Group 3: [20, 1500, 100],&lt;/li&gt;&#xA;&lt;li&gt;Group 4: [150, 3000, 13].&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I need to select one number from each group so that the difference between the maximum and minimum of the four selected numbers would be the smallest among all possible combinations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example, the numbers 10, 15, 20, 13 give the difference 10 and it is the smallest. The answer for the problem above is 10, 15, 20, 13.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is an algorithm that can solve this problem and also scales relatively well for a large number of groups and the number of entries in each group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: @aurelian-tutuianu proposed an algorithm of complexity $O(nk\log k)$ that involves sorting each group. I wonder what would be the best algorithm if it's impossible to sort those groups. For instance, for groups&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Group 1: [a1, a2, a3],&lt;/li&gt;&#xA;&lt;li&gt;Group 2: [b1, b2, b3],&lt;/li&gt;&#xA;&lt;li&gt;Group 3: [c1, c2, c3],&lt;/li&gt;&#xA;&lt;li&gt;Group 4: [d1, d2, d3],&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;where I can find a pairwise similarity between all elements but cannot sort them.&lt;/p&gt;&#xA;" OwnerUserId="28430" LastEditorUserId="1804" LastEditDate="2018-08-15T14:27:34.870" LastActivityDate="2018-08-15T14:27:34.870" Title="Algorithm to find most similar elements in several groups" Tags="|algorithms|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30023" PostTypeId="1" AcceptedAnswerId="30030" CreationDate="2018-08-10T21:48:30.350" Score="0" ViewCount="755" Body="&lt;p&gt;I am trying to sort data following a bivariate distribution into a numpy histogramdd, where each bin should contain the same number of data points (to the nearest whole sample).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect that some kind of quantile-approach is required, and have tried &lt;code&gt;scipy.stats.mstats.mquantiles&lt;/code&gt;, which according to the documentation takes up to 2D data. However, it seems to take the dimensions independently, splitting each dimension into to equal marginal probabilities, which doesn't achieve the desired result of equal-probability bins in 2D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a built-in way in scipy/numpy or another package to achieve this (in 2D or higher)? If not, are there algorithms designed to achieve this which I can implement myself directly? &lt;/p&gt;&#xA;" OwnerUserId="25837" LastActivityDate="2021-10-15T20:05:23.967" Title="Is it possible to partition 2D data into bins such that each bin contains the same number of samples?" Tags="|python|scipy|numpy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30024" PostTypeId="2" ParentId="29931" CreationDate="2018-08-11T04:36:43.310" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;it is ok for the discretization.&lt;/li&gt;&#xA;&lt;li&gt;CD scheme has some stability problem when Pe&gt;2, but we can decrease the mesh spacing to obtain a low mesh Pe number.&lt;/li&gt;&#xA;&lt;li&gt;QUICK-scheme is more stable and accurate than CD, so it is ok to implement it.&lt;/li&gt;&#xA;&lt;li&gt;you should increase the order of both convection term and diffusion term to obtain a high order solution.&lt;/li&gt;&#xA;&lt;li&gt;the mesh spacings should be changed gradually. I think you met some problems about the stability.&lt;/li&gt;&#xA;&lt;li&gt;yes, both convection and diffusion terms can be treated implicitly. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="22404" LastActivityDate="2018-08-11T04:36:43.310" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30025" PostTypeId="2" ParentId="28123" CreationDate="2018-08-11T04:44:15.563" Score="0" Body="&lt;p&gt;Here is my C++ code for the output of FVM data file in tecplot format.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;void OutPut()&#xA;{&#xA;    int nbPoints=PT-&amp;gt;Field.mesh().NodeNumber;&#xA;    int nbCells=PT-&amp;gt;Field.mesh().CellNumber;&#xA;    string name=&quot;Energy.dat&quot;;&#xA;    ofstream outfile(&quot;../&quot;+name);&#xA;    outfile&amp;lt;&amp;lt;&quot; VARIABLES=\&quot;X\&quot;,\&quot;Y\&quot;,\&quot;U\&quot;, \&quot;V\&quot;, \&quot;T\&quot; &quot;&amp;lt;&amp;lt;endl;&#xA;    outfile&amp;lt;&amp;lt; &quot;ZONE N=&quot;&amp;lt;&amp;lt;nbPoints&amp;lt;&amp;lt;&quot;,E=&quot;&amp;lt;nbCells&amp;lt;&amp;lt;&quot;,ZONETYPE=FEQUADRILATERAL&quot;    &amp;lt;&amp;lt;endl;&#xA;    outfile&amp;lt;&amp;lt; &quot;DATAPACKING=block &quot;&amp;lt;&amp;lt;endl;&#xA;    outfile&amp;lt;&amp;lt;&quot;  VARLOCATION=( [1,2]=NODAL,[3,4,5]=CELLCENTERED )&quot;&amp;lt;&amp;lt;endl;    for(int i=0; i&amp;lt;nbPoints;i++)&#xA;    {&#xA;        outfile&amp;lt;&amp;lt; setprecision(20)&amp;lt;&amp;lt;scientific&amp;lt;&amp;lt;PT-&amp;gt;Field.mesh().Coordinate[i][0]&amp;lt;&amp;lt;&quot;   &quot;&amp;lt;&amp;lt;endl;&#xA;    }&#xA;    for(int i=0; i&amp;lt;nbPoints;i++)&#xA;    {&#xA;        outfile&amp;lt;&amp;lt; setprecision(20)&amp;lt;&amp;lt;scientific&amp;lt;&amp;lt;PT-&amp;gt;Field.mesh().Coordinate[i][1]&amp;lt;&amp;lt;&quot;   &quot;&amp;lt;&amp;lt;endl;&#xA;    }&#xA;&#xA;    for(int i=0; i&amp;lt;nbCells;i++)&#xA;    {&#xA;        outfile&amp;lt;&amp;lt;this-&amp;gt;U.Field()[i]&amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt;endl;&#xA;    }&#xA;    for(int i=0; i&amp;lt;nbCells;i++)&#xA;    {&#xA;        outfile&amp;lt;&amp;lt;this-&amp;gt;V.Field()[i]&amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt;endl;&#xA;    }&#xA;    for(int i=0; i&amp;lt;nbCells;i++)&#xA;    {&#xA;        outfile&amp;lt;&amp;lt;this-&amp;gt;Field.Field()[i]&amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt;endl;&#xA;    }&#xA;&#xA;    for(int i=0; i&amp;lt;nbCells;i++)&#xA;    {&#xA;        if(PT-&amp;gt;Field.mesh().Connection[i].size()==3)&#xA;        {&#xA;            for(unsigned int j=0; j&amp;lt;PT-&amp;gt;Field.mesh().Connection[i].size();j++) &#xA;            {&#xA;                outfile&amp;lt;&amp;lt;PT-&amp;gt;Field.mesh().Connection[i][j]+1&amp;lt;&amp;lt;&quot; &quot;; &#xA;            }&#xA;            outfile&amp;lt;&amp;lt;PT-&amp;gt;Field.mesh().Connection[i][2]+1;&#xA;            outfile&amp;lt;&amp;lt;endl;      &#xA;         }&#xA;&#xA;         if(PT-&amp;gt;Field.mesh().Connection[i].size()==4)&#xA;         {&#xA;             for(unsigned int j=0; j&amp;lt;PT&amp;gt;Field.mesh().Connection[i].size();j++)&#xA;             {&#xA;                 outfile&amp;lt;&amp;lt;PT-&amp;gt;Field.mesh().Connection[i][j]+1&amp;lt;&amp;lt;&quot; &quot;; &#xA;             }&#xA;             outfile&amp;lt;&amp;lt;endl;     &#xA;          }&#xA;     }&#xA;     outfile.close();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22404" LastEditorUserId="20688" LastEditDate="2018-08-11T04:59:43.007" LastActivityDate="2018-08-11T04:59:43.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30026" PostTypeId="2" ParentId="26953" CreationDate="2018-08-11T13:07:02.380" Score="0" Body="&lt;p&gt;The first thing is to find the halo cells of the current processor and its owner-processor. We create a list to record it. Then we use the MPI_SEND and MPI_receive to conduct the communication. &lt;/p&gt;&#xA;" OwnerUserId="22404" LastActivityDate="2018-08-11T13:07:02.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30027" PostTypeId="2" ParentId="21581" CreationDate="2018-08-11T13:20:59.580" Score="2" Body="&lt;p&gt;I think you may create another vector to store all the non-zero elements in the solution vector. Then use MPI_ALLReduce. &lt;/p&gt;&#xA;" OwnerUserId="22404" LastActivityDate="2018-08-11T13:20:59.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30028" PostTypeId="1" CreationDate="2018-08-11T17:46:08.790" Score="1" ViewCount="465" Body="&lt;p&gt;A certain filter I'm writing uses two different kernels.  The Fejer kernel (which is common) and the Jackson kernel:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \Delta_T(x) = T \,\left(  \frac{\sin \pi T x}{\pi T x}\right)^2  \quad\text{and}\quad&#xA;J_T(x) = \frac{3T}{4} \left(  \frac{\sin \pi T x/2}{\pi T x/2}\right)^4&#xA; $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I had to remind myself how these two kernels behaved and what their properties were:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a fairly broad class of functions, convolving with the Fejer kernel is the same as integrating against a triangle:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \int_{-T}^T \left( 1 - \frac{|\xi|}{T} \right) \hat{f}(\xi) \,e^{2\pi i \, x \xi} \,d\xi = (f * \Delta_T)(x) $$&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Notably, the discrete &lt;a href=&quot;https://en.wikipedia.org/wiki/Fej%C3%A9r_kernel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fejér kernel&lt;/a&gt; (over $S^1$) is a limit of the continuous Fejer kernel (over $\mathbb{R}$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ F_N(x) = \frac{D_0(x) + \dots + D_{N-1}(x)}{N}  = \frac{1}{N} \left(\frac{\sin \left(Nx/2\right)}{\sin (x/2)}\right)^2 = \Delta_{1/T}(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here it emulates &lt;a href=&quot;https://en.wikipedia.org/wiki/Ces%C3%A0ro_summation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cesaro summabilition&lt;/a&gt; on the Fourier series:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \big(f*F_N\big)(x) = \frac{ S_0(f) (x) + \dots + S_{N-1}(f)(x)}{N} = \frac{1}{N} \sum_{n=0}^{N} \sum_{|x| \leq n} \widehat{f}(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The similarities continue that the $S^1$ Fejér kernel is the periodization of the $\mathbb{R}$ Fejér kernel:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \sum_{n \in \mathbb{Z}} \Delta_{\frac{1}{N}}(x + n) = F_N(x)  $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are taken from Fourier Analysis by Elias Stein.  My question is what similar properties should occur for the Jackson kernel?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Googling the Jackson kernel doesn't return much. E.g. this note on &lt;a href=&quot;http://www.damtp.cam.ac.uk/user/na/PartIIIat/b09.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;trigonometric polynomials&lt;/a&gt; and modulus of continuity.&lt;/p&gt;&#xA;" OwnerUserId="4006" LastActivityDate="2018-08-11T17:46:08.790" Title="What does the Jackson Kernel measure?" Tags="|fourier-analysis|numerics|signal-processing|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30030" PostTypeId="2" ParentId="30023" CreationDate="2018-08-12T14:03:39.740" Score="1" Body="&lt;p&gt;You obviously can't do this if you want the bins to be separated at the same $x_i$ and $y_i$ values. This is easy to see if you want to have, say, 4 bins and have 10 data points at $(0,0)$ and 10 points at $(1,1)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you can use a $kd$ tree data structure in which you recursively subdivide each bin so that it contains the same number of points. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-08-12T14:03:39.740" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30031" PostTypeId="1" AcceptedAnswerId="30035" CreationDate="2018-08-12T14:17:00.303" Score="1" ViewCount="568" Body="&lt;p&gt;I want to do somethings similar to the plot found at the bottom of &lt;a href=&quot;http://blog.nag.com/2010/12/paraview-vtk-files-and-endianness.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; blogpost, which I found via &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10910/visualization-of-3d-probability-flow&quot;&gt;this&lt;/a&gt; post in ParaView, which seems to have been the original Application used to generate the plot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a &lt;code&gt;.csv&lt;/code&gt;-file containing data points of the form $(x, y, z, a)$ where $a$ ranges from 0 to 1 and represents gives the probability density - or &quot;fog&quot; density that should be plottet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how to plot this datafile/which ParaView filters to use to create this plot.&lt;/p&gt;&#xA;" OwnerUserId="28437" LastEditorUserId="28437" LastEditDate="2018-08-12T14:50:06.543" LastActivityDate="2018-08-12T21:31:48.250" Title="Recreating this 3D density plot in ParaView" Tags="|visualization|paraview|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30033" PostTypeId="2" ParentId="29873" CreationDate="2018-08-12T17:17:53.137" Score="6" Body="&lt;p&gt;I know this is already answered satisfactorily, but let me just add this gentle reminder to check whether the integral can be done analytically, before crunching it with a numerical method! In this case the indefinite integral of &#xA;$$&#xA;f(E) = \frac{1}{1+\exp[(E-E_f)/k_B T]}&#xA;$$&#xA;is this:&#xA;$$&#xA;F(E) = -k_B T \ln \left\{ 1+\exp[-(E-E_f)/k_B T] \right\}&#xA;$$&#xA;and $F(\infty)=0$, so the desired answer is $-F(E_k)$. The following python code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;import numpy as np&#xA;&#xA;eigenvalue = np.array([0.9, 1.3])&#xA;fermi = 1.0&#xA;T = 300&#xA;kB = 8.6173303e-5&#xA;&#xA;result = kB*T*np.log(1+np.exp(-(eigenvalue-fermi)/(kB*T)))&#xA;print(result)&#xA;&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives the exact result(s), for comparison with the numerical one(s).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;[1.00534649e-01 2.35891393e-07]&#xA;&lt;/pre&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-08-12T17:17:53.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30035" PostTypeId="2" ParentId="30031" CreationDate="2018-08-12T21:31:48.250" Score="3" Body="&lt;p&gt;You will have to interpolate the points to a 3D grid (a regular grid will do) and then do the volume rendering.  See &lt;a href=&quot;https://www.visitusers.org/index.php?title=Volume_Rendering&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.visitusers.org/index.php?title=Volume_Rendering&lt;/a&gt; for some examples.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2018-08-12T21:31:48.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30036" PostTypeId="1" CreationDate="2018-08-12T21:40:17.977" Score="1" ViewCount="563" Body="&lt;p&gt;I would like to insert elements into a ring (circular) buffer one at a time and maintain a permutation array which keeps track of the sorted elements in ascending order. To do this, I have adapted the &quot;Insertion Sort&quot; algorithm to use circular array indexing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/* sort ringbuf entries with insertion sort algorithm */&#xA;void&#xA;ringbuf_isort(const ringbuf *rbuf, int *idx)&#xA;{&#xA;  const int n = ringbuf_n(rbuf);&#xA;  int i, j;&#xA;&#xA;  for (i = 1; i &amp;lt; n; i++)&#xA;    {&#xA;      int id = idx[i];&#xA;      double v = rbuf-&amp;gt;array[(rbuf-&amp;gt;head + id) % rbuf-&amp;gt;size];&#xA;&#xA;      for (j = i; j &amp;gt; 0; j--)&#xA;        {&#xA;          if (rbuf-&amp;gt;array[(rbuf-&amp;gt;head + idx[j-1]) % rbuf-&amp;gt;size] &amp;lt; v)&#xA;            break;&#xA;&#xA;          idx[j] = idx[j-1];&#xA;        }&#xA;&#xA;      idx[j] = id;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This algorithm works correctly - the &lt;code&gt;idx&lt;/code&gt; array contains the permutation of the buffer elements which tracks their sorted status. The problem is that the algorithm is extremely slow, due to the modular arithmetic needed for the circular indexing, i.e. &lt;code&gt;(head + idx) % size&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, I have found it is faster to simply copy the ring buffer into a linear array and use a quicksort algorithm each time an element is added, rather than use the code above. For a linear array, I have confirmed the insertion sort algorithm outperforms quicksort when only a single element is changed, so I am hoping I can find a better solution for the ring buffer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to optimize the code above to speed up the modular arithmetic (or avoid it altogether?)&lt;/p&gt;&#xA;" OwnerUserId="28440" LastEditorUserId="20688" LastEditDate="2018-08-12T22:43:10.343" LastActivityDate="2018-08-12T22:43:10.343" Title="Maintain sorted ring buffer" Tags="|algorithms|sorting|" AnswerCount="0" CommentCount="1" ClosedDate="2018-08-13T15:57:55.233" ContentLicense="CC BY-SA 4.0" />
  <row Id="30041" PostTypeId="1" CreationDate="2018-08-13T14:25:11.453" Score="2" ViewCount="231" Body="&lt;p&gt;There are generic methods for solving systems of ODEs numerically. Are there generic methods for PDEs? If so, what are they? If not, why not? To elaborate...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any set of ODEs can be written in standard form as a set of &lt;em&gt;n&lt;/em&gt; first order ODEs in &lt;em&gt;n&lt;/em&gt; variables. A single method can then be used. The number of boundary/initial conditions required is &lt;em&gt;n&lt;/em&gt;. Not absolutely every ODE with a solution can be solved by a single method and mathematicians will understand advantages of certain methods for certain problems, but science/engineering type of people can expect to use the method to get a solution to a reasonable problem if one exists. Example implementations include &lt;em&gt;bvp4c&lt;/em&gt; in MATLAB and &lt;em&gt;scipy.integrate.solve_bvp&lt;/em&gt; in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not aware of a single version of FEM/FVM or any other method that can be applied as reliably as the ODE methods. But systems of PDEs can also be &lt;a href=&quot;http://home.iitk.ac.in/~tmk/courses/mth424/main.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;reduced to a first order standard form&lt;/a&gt; and possibly other standard forms. Complications that may prevent a generic method I'm aware of include: behavioural differences based on elliptic/hyperbolic classification (definitely applies to second order PDEs but possibly also any system of PDEs); form of boundary condition required (eg whole-boundary conditions vs Cauchy conditions, which may be directly related to ellipticity); behavioural difference for even/odd derivatives(?) Do any of these things prevent a single generic method? Are there any other reasons?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB, &lt;em&gt;solvepde&lt;/em&gt; works on systems of second order PDEs, so I think this excludes any PDEs with odd (spatial) derivatives. I'm not sure if the C++/Python &lt;em&gt;FEniCS&lt;/em&gt; can represent any system of PDEs with its form language. If it can, my understanding is still that the method will not converge if certain conditions are not met on the choice of basis functions. So it seems these are fairly general methods but not completely generic from the point of view of a science/engineering programmer who just wants a solution without understanding lots of theory (as is the case for the mentioned ODE solvers).&lt;/p&gt;&#xA;" OwnerUserId="28292" LastEditorUserId="28292" LastEditDate="2018-08-13T17:20:25.480" LastActivityDate="2018-08-13T18:07:33.057" Title="Does a generic method for solving a system of PDEs exist?" Tags="|finite-element|pde|numerics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30042" PostTypeId="2" ParentId="30041" CreationDate="2018-08-13T18:07:33.057" Score="6" Body="&lt;p&gt;No. Different discretizations are stable/unstable on different PDEs. There is no one size fits all approach to the whole class of PDEs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Even for ODEs there are generic methods but which methods are good depends on the equation being solved)&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-08-13T18:07:33.057" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30043" PostTypeId="2" ParentId="30022" CreationDate="2018-08-14T06:39:34.797" Score="0" Body="&lt;p&gt;Here is a solution. We assume the numbers from groups are sorted. We start with the following observation: any candidate solution will have the smallest number and the biggest number. To simplify we fix the smallest number and check which is the solution if that number is the smallest one. The algorithm follows these steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take the smallest number from the first numbers in each group. Remember groups are sorted. In your case, the smallest is 10&lt;/li&gt;&#xA;&lt;li&gt;Find the biggest difference between the smallest number and all first numbers from each group. This is guaranteed to be the optimal solution for the given smallest number. &lt;/li&gt;&#xA;&lt;li&gt;Keep the solution if it is the best seen for now. If it is the first solution found, keep it anyway for later comparison. &lt;/li&gt;&#xA;&lt;li&gt;Remove the smallest element from its group. If there are multiple groups then remove first elements from those groups too.&lt;/li&gt;&#xA;&lt;li&gt;If you have an empty group then you stop. The best solution seen so far is the optimal one. If your groups are not empty then continue with step 1.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you put your groups in a priority queue or min heap at each step you spent $\log k$, where $k$ is the number of groups. Each solution verification takes k steps. For a total number of $n$ numbers from all groups, you will have $\mathcal O(nk \log k)$ running time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later edit: if at step 2 you find a difference bigger than best candidate so far you can avoid further computation because you know already you will not improve the solution&lt;/p&gt;&#xA;" OwnerUserId="4838" LastEditorUserId="20688" LastEditDate="2018-08-14T14:32:17.560" LastActivityDate="2018-08-14T14:32:17.560" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30048" PostTypeId="1" CreationDate="2018-08-14T18:42:34.280" Score="5" ViewCount="335" Body="&lt;p&gt;I am trying to do integrals of the type &#xA;$$&#xA;\int d^3\vec{p} \int d^3\vec{p}' e^{-p^2} e^{-{p'}^2}f(\vec{p}, \vec{p}')&#xA;$$&#xA;where $\vec{p}$ and $\vec{p}'$ are three dimensional vectors represented using spherical coordinates, $\vec{p} = \{p,\theta,\phi\}$, and $f$ is a non-trivial, potentially complex, function. The integrals over $\phi$ and $\phi'$ can be done analytically even though the answers are rather complicated. However that is not true for the other integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was wondering what would be the best method to approach this problem or if there are any packages (preferably for python) that do this kind of integrals. I plan to try SciPy's &lt;code&gt;nquad&lt;/code&gt; but I hear that it is not suggested for integrals weighted by $e^{-p^2}$.&lt;/p&gt;&#xA;" OwnerUserId="20647" LastEditorUserId="3481" LastEditDate="2018-08-15T01:13:33.487" LastActivityDate="2018-08-15T01:13:33.487" Title="What is the best numerical method for a six dimensional spherical integral?" Tags="|quadrature|integration|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="30049" PostTypeId="2" ParentId="1571" CreationDate="2018-08-14T19:36:19.840" Score="4" Body="&lt;p&gt;The accepted answer takes &lt;span class=&quot;math-container&quot;&gt;$O(kn)$&lt;/span&gt; multiplications and additions. There is also a method using &lt;span class=&quot;math-container&quot;&gt;$O(k\log n)$&lt;/span&gt; additions, multiplications and divisions, using the below identity.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;S(n,k)=\frac1{k!}\sum_{j=0}^k(-1)^j\binom{k}j(k-j)^n&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can use &lt;a href=&quot;https://dlmf.nist.gov/26.3#E6&quot; rel=&quot;nofollow noreferrer&quot;&gt;DLMF Equation 26.3.E6&lt;/a&gt; as a recurrence relation to greatly reduce the amount of computation necessary to compute the binomial coefficient.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\binom{m}{n} = \frac{m-n+1}{n}\binom{m}{n-1}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Pseudocode:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;accum ← 0&#xA;k_choose_j ← 1&#xA;for j = 0,1,...,k-1:&#xA;  accum ← accum + (-1)^j * k_choose_j * (k - j)^n&#xA;  k_choose_j ← k_choose_j * (k-j) / (j+1)&#xA;return accum / factorial(k)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that in the above pseudocode, we update &lt;code&gt;k_choose_j&lt;/code&gt; at the end of each iteration, so we need to substitute &lt;code&gt;j + 1&lt;/code&gt; for &lt;code&gt;n&lt;/code&gt; in the above relation.&lt;/p&gt;&#xA;" OwnerUserId="28461" LastEditorUserId="41597" LastEditDate="2021-10-23T20:34:43.367" LastActivityDate="2021-10-23T20:34:43.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30050" PostTypeId="2" ParentId="30048" CreationDate="2018-08-14T23:58:45.470" Score="5" Body="&lt;p&gt;The Genz-Malik algorithm [&lt;a href=&quot;https://github.com/stevengj/cubature&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;], as implemented in the &lt;a href=&quot;https://github.com/stevengj/cubature&quot; rel=&quot;noreferrer&quot;&gt;cubature library&lt;/a&gt;, works well for computing 6-dimensional integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[&lt;a href=&quot;https://github.com/stevengj/cubature&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;] A. C. Genz and A. A. Malik, “Remarks on algorithm 006: An adaptive algorithm for numerical integration over an N-dimensional rectangular region,” Journal of Computational and Applied Mathematics, vol. 6, no. 4, pp. 295–302, Dec. 1980.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2018-08-14T23:58:45.470" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30051" PostTypeId="2" ParentId="816" CreationDate="2018-08-15T13:15:25.883" Score="0" Body="&lt;p&gt;We're basically after:&#xA;$$ \arg \min_{m} \sum_{i = 1}^{N} \left| m - {x}_{i} \right| $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One should notice that $ \frac{\mathrm{d} \left | x \right | }{\mathrm{d} x} = \operatorname{sign} \left( x \right) $ (Being more rigorous would say it is a Sub Gradient of the non smooth $ {L}_{1} $ Norm function).&lt;br&gt;&#xA;Hence, deriving the sum above yields $ \sum_{i = 1}^{N} \operatorname{sign} \left( m - {x}_{i} \right) $.&lt;br&gt;&#xA;This equals to zero only when the number of positive items equals the number of negative which happens when $ m = \operatorname{median} \left\{ {x}_{1}, {x}_{2}, \cdots, {x}_{N} \right\} $.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One should notice that the &lt;code&gt;median&lt;/code&gt; of a discrete group is not uniquely defined.&lt;br&gt;&#xA;Moreover, it is not necessarily an item within the group.&lt;/p&gt;&#xA;" OwnerUserId="7951" LastActivityDate="2018-08-15T13:15:25.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30052" PostTypeId="1" AcceptedAnswerId="30057" CreationDate="2018-08-15T19:43:02.770" Score="2" ViewCount="1309" Body="&lt;p&gt;I have a python implementation of calculating the partial trace over select dimensions.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def partial_trace(rho, keep, dims):&#xA;    &quot;&quot;&quot;Calculate the partial trace&#xA;&#xA;    Parameters&#xA;    ----------&#xA;    ρ : 2D array&#xA;        Matrix to trace&#xA;    keep : array&#xA;        An array of indices of the spaces to keep after&#xA;        being traced. For instance, if the space is&#xA;        A x B x C x D and we want to trace out B and D,&#xA;        keep = [0,2]&#xA;    dims : array&#xA;        An array of the dimensions of each space.&#xA;        For instance, if the space is A x B x C x D,&#xA;        dims = [dim_A, dim_B, dim_C, dim_D]&#xA;&#xA;    Returns&#xA;    -------&#xA;    ρ_a : 2D array&#xA;        Traced matrix&#xA;    &quot;&quot;&quot;&#xA;    dims = np.asarray(dims)&#xA;    N = dims.size&#xA;    # Indices to trace&#xA;    rest = np.delete(np.arange(N), keep)&#xA;    # Reshape into tensor&#xA;    rho_a = rho.reshape(np.tile(dims, 2))&#xA;    # Trace indices&#xA;    for i,d in enumerate(rest[::-1]):&#xA;        rho_a = np.trace(rho_a, axis1=d, axis2=N-i+d)&#xA;    # Reshape into matrix&#xA;    N = np.prod(dims[keep])&#xA;    return rho_a.reshape(N,N)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to calculate the partial trace of $|u\rangle \langle u|$ but I run into an insufficient memory error when I try to construct the outer product.&#xA;Instead, I would like to directly calculate the partial trace from $|u \rangle$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I do this in Python?&lt;/p&gt;&#xA;" OwnerUserId="28473" LastEditorUserId="9667" LastEditDate="2018-08-15T20:00:57.850" LastActivityDate="2023-11-09T23:13:01.283" Title="Calculate partial trace of an outer product in Python?" Tags="|linear-algebra|python|quantum-mechanics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30053" PostTypeId="1" CreationDate="2018-08-15T22:58:53.270" Score="6" ViewCount="204" Body="&lt;p&gt;Given a fat matrix $B \in \mathbb{C}^{n \times m}$ (where $m &amp;gt; n$) with full row rank, I would like to find (numerically) a full-rank matrix $A$ that minimizes the Frobenius norm of the product $A B$. Formally,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\underset{A \in \mathbb{C}^{n \times n}}{\text{minimize}} \quad \frac{1}{2} \|AB\|_F^2 \quad \text{subject to} \quad \det (A) \neq 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The value of $m$ is typically an order of magnitude larger than $n$. The sizes ($n,m$) I am interested in may be on the order of hundreds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found the following &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19840/optimization-with-matrix-determinant-as-constraint&quot;&gt;discussion&lt;/a&gt;, which I guess could be generalized to the above case. I wonder if there is a simpler solution in this particular scenario.&lt;/p&gt;&#xA;" OwnerUserId="28475" LastEditorUserId="20417" LastEditDate="2018-08-17T17:31:48.620" LastActivityDate="2018-08-17T17:31:48.620" Title="An invertible matrix that minimizes the norm of the product with a given matrix" Tags="|matrix|constrained-optimization|" AnswerCount="0" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="30054" PostTypeId="1" CreationDate="2018-08-17T06:45:15.820" Score="2" ViewCount="497" Body="&lt;p&gt;I have a time dependent data set where each frame corresponds to a certain date. I need to show the time annotation, much like what Annotate Time Filter does, but show the date instead of time as a float value. How can I do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In python I can convert the time index to a date with &lt;code&gt;datetime&lt;/code&gt; for instance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;init_date = datetime.datetime(2018, 1, 1)&#xA;time_step = 3600.&#xA;itime = 20&#xA;date = init_date + datetime.timedelta(seconds=itime*time_step)&#xA;print(date.strftime('%Y-%m-%d %H:%M'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9503" LastActivityDate="2019-11-03T10:02:55.380" Title="Showing date in Paraview's Annotate Time Filter" Tags="|python|visualization|paraview|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30055" PostTypeId="1" CreationDate="2018-08-17T13:47:56.780" Score="10" ViewCount="282" Body="&lt;p&gt;Every real matrix $A$ can be reduce to real Schur form $T = U^T A U$ using an  orthogonal similiary transform $U$. Here the matrix $T$ is quasi-triangular form with 1 by 1 or 2 by 2 blocks on the main diagonal. Each 1 by 1 block corresponds to a real eigenvalue of $A$ and each 2 by 2 block corresponds to a pair of complex conjugate eigenvalues of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The eigenvalue reordering problem consists of finding an orthogonal similarity transformation $V$ such that the user's selection of eigenvalues of $A$ appears along the diagonal of the upper left corner of $S = V^T T V$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In LAPACK the relevant routine double precision routine is called DTRSEN. Daniel Kressner has written a blocked version by the name BDTRSEN. The ScaLAPACK routine is PDTRSEN. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am looking for applications and algorithms where advances in solving the eigenvalue reordering problem would have real benefits.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can easily generate test matrices in quasi triangular form, but we are having trouble deciding the shape of a realistic distribution of the user's selection of eigenvalues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my perspective, subspace iteration with Ritz acceleration is an ideal algorithm for testing improvements to the reordering algorithm. It needs (sparse) matrix vector multiplication, a tall QR algorithm and a reordering algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, is hard for me to find real life problems where it is clear that a particular set of eigenpairs is physically interesting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can do eigenvalue reordering for dense matrices of dimension 40,000 using a shared memory machine. The best performance is achieved when the user selects about 50% of all eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2021-09-01T11:08:00.807" Title="Benchmark problems for eigenvalue reordering algorithms sought" Tags="|eigenvalues|eigensystem|lapack|numerics|scalapack|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30056" PostTypeId="2" ParentId="30055" CreationDate="2018-08-17T20:26:09.627" Score="0" Body="&lt;p&gt;I'm sure I don't fully appreciate the utility of the eigenvalue reordering algorithm, but many answers come to mind for this part of your question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, is hard for me to find real life problems where it is clear that a particular set of eigenpairs is physically interesting.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For example, in certain hydrodynamic stability problems you will have unique eigenvalues that are associated with physical phenomena such as Kelvin--Helmholtz modes or Tollmien--Schlichting waves.  In fluid-structure-interaction problems, the resonant mode can be associated with a flapping instability. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this along the lines of what you are looking for? If so, I'm sure others will chirp in with examples from their fields; if not, can you sharpen up the question? &lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2018-08-17T20:26:09.627" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30057" PostTypeId="2" ParentId="30052" CreationDate="2018-08-17T20:39:08.637" Score="1" Body="&lt;p&gt;I modified my implementation of the partial trace to use einsum.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def partial_trace(rho, keep, dims, optimize=False):&#xA;    &quot;&quot;&quot;Calculate the partial trace&#xA;&#xA;    ρ_a = Tr_b(ρ)&#xA;&#xA;    Parameters&#xA;    ----------&#xA;    ρ : 2D array&#xA;        Matrix to trace&#xA;    keep : array&#xA;        An array of indices of the spaces to keep after&#xA;        being traced. For instance, if the space is&#xA;        A x B x C x D and we want to trace out B and D,&#xA;        keep = [0,2]&#xA;    dims : array&#xA;        An array of the dimensions of each space.&#xA;        For instance, if the space is A x B x C x D,&#xA;        dims = [dim_A, dim_B, dim_C, dim_D]&#xA;&#xA;    Returns&#xA;    -------&#xA;    ρ_a : 2D array&#xA;        Traced matrix&#xA;    &quot;&quot;&quot;&#xA;    keep = np.asarray(keep)&#xA;    dims = np.asarray(dims)&#xA;    Ndim = dims.size&#xA;    Nkeep = np.prod(dims[keep])&#xA;&#xA;    idx1 = [i for i in range(Ndim)]&#xA;    idx2 = [Ndim+i if i in keep else i for i in range(Ndim)]&#xA;    rho_a = rho.reshape(np.tile(dims,2))&#xA;    rho_a = np.einsum(rho_a, idx1+idx2, optimize=optimize)&#xA;    return rho_a.reshape(Nkeep, Nkeep)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This can be modified to take a vector as input.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def ptrace_outer(u, keep, dims, optimize=False):&#xA;    &quot;&quot;&quot;Calculate the partial trace of an outer product&#xA;&#xA;    ρ_a = Tr_b(|u&amp;gt;&amp;lt;u|)&#xA;&#xA;    Parameters&#xA;    ----------&#xA;    u : array&#xA;        Vector to use for outer product&#xA;    keep : array&#xA;        An array of indices of the spaces to keep after&#xA;        being traced. For instance, if the space is&#xA;        A x B x C x D and we want to trace out B and D,&#xA;        keep = [0,2]&#xA;    dims : array&#xA;        An array of the dimensions of each space.&#xA;        For instance, if the space is A x B x C x D,&#xA;        dims = [dim_A, dim_B, dim_C, dim_D]&#xA;&#xA;    Returns&#xA;    -------&#xA;    ρ_a : 2D array&#xA;        Traced matrix&#xA;    &quot;&quot;&quot;&#xA;    keep = np.asarray(keep)&#xA;    dims = np.asarray(dims)&#xA;    Ndim = dims.size&#xA;    Nkeep = np.prod(dims[keep])&#xA;&#xA;    idx1 = [i for i in range(Ndim)]&#xA;    idx2 = [Ndim+i if i in keep else i for i in range(Ndim)]&#xA;    u = u.reshape(dims)&#xA;    rho_a = np.einsum(u, idx1, u.conj(), idx2, optimize=optimize)&#xA;    return rho_a.reshape(Nkeep, Nkeep)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28473" LastActivityDate="2018-08-17T20:39:08.637" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30058" PostTypeId="1" AcceptedAnswerId="30097" CreationDate="2018-08-17T23:39:46.840" Score="6" ViewCount="317" Body="&lt;p&gt;I would like to determine the number of negative eigenvalues (inertia count) of the $(N \times N)$ symmetric real matrix $K - \sigma M$, with $K$ a positive-definite sparse matrix and $M$ a positive-semidefinite matrix such that $M = Z Z^T$ with $Z$ a dense $(N \times r)$ full column-rank matrix, $r \ll N$, $\sigma &amp;gt; 0$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix $K - \sigma M$ is a low-rank perturbation of $K$ and cannot be assembled into memory because $N$ is very large. If $M$ was sparse, the LDL factorization $K - \sigma M = L D L^T$ would be possible and would give the inertia count through the inertia count of $D$.&lt;/p&gt;&#xA;" OwnerUserId="28491" LastEditorUserId="28491" LastEditDate="2018-08-21T04:55:40.037" LastActivityDate="2018-09-05T18:41:10.573" Title="inertia count sparse matrix with dense low-rank perturbation" Tags="|sparse-matrix|eigenvalues|dense-matrix|matrix-factorization|update|" AnswerCount="2" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30059" PostTypeId="2" ParentId="8514" CreationDate="2018-08-18T11:58:05.087" Score="3" Body="&lt;p&gt;I know of one contest to &lt;a href=&quot;https://www.ccdc.cam.ac.uk/Community/initiatives/cspblindtests/&quot; rel=&quot;nofollow noreferrer&quot;&gt;predict crystal structure&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21198" LastActivityDate="2018-08-18T11:58:05.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30060" PostTypeId="2" ParentId="26478" CreationDate="2018-08-18T15:00:13.053" Score="0" Body="&lt;p&gt;If your data is measured at discrete equal time intervals then you can measure the autocorrelation at different lags. This is the standard correlation between your time series starting at 0 ending at t-p and your time series starting at p ending at t, where t is your number of data points.   You can go further  by studying partial autocorrelation. Where you have high peaks then you have a dominant period. For acf there are also statistical tests to tell you if the values for acf are significant or are made mostly from noise artifacts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those functions works on one dimensional data, but you can use any of x, y or z. This is because your data projected on any axis exhibit the same periods except the situation when your true 3d curve does not lie on a plane perpendicular on one of your axes. As such you can use your findings on some axis and confirm by measuring on other axes.&lt;/p&gt;&#xA;" OwnerUserId="4838" LastActivityDate="2018-08-18T15:00:13.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30062" PostTypeId="1" CreationDate="2018-08-20T06:46:43.397" Score="1" ViewCount="155" Body="&lt;p&gt;I'm trying to solve aerodynamics of whole aircraft by &lt;a href=&quot;https://en.wikipedia.org/wiki/Vortex_lattice_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;vortex lattice method&lt;/a&gt; (or deeper &lt;a href=&quot;http://www.dept.aoe.vt.edu/~mason/Mason_f/CAtxtChap6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). The problem is that sometimes &lt;a href=&quot;https://en.wikipedia.org/wiki/Wingtip_vortices&quot; rel=&quot;nofollow noreferrer&quot;&gt;trailing vortex filaments&lt;/a&gt; from &lt;a href=&quot;https://en.wikipedia.org/wiki/Horseshoe_vortex&quot; rel=&quot;nofollow noreferrer&quot;&gt;Horse Shoe Vortexes&lt;/a&gt; of main wing hits panels of rudder/elevator which leads to singularities (infinite coupling between the two panels).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v2Xuy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v2Xuy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there good way to damp those singularities? I was thinking something like adding imaginary part to expression for vortex filaments might help, but than vorticity is not exactly conserved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is related to my &lt;a href=&quot;https://scicomp.stackexchange.com/questions/28534/vortex-lattice-method-better-basis-than-horse-shoe-vortex&quot;&gt;previous question&lt;/a&gt; which was prehaps a bit long, unclear and unfocused. &lt;/p&gt;&#xA;" OwnerUserId="4696" LastEditorUserId="4696" LastEditDate="2018-08-20T14:43:15.777" LastActivityDate="2020-09-25T01:55:07.160" Title="How to remove sigularities from 3D Vortex Lattice Method" Tags="|fluid-dynamics|vorticity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30063" PostTypeId="1" CreationDate="2018-08-20T16:32:42.853" Score="5" ViewCount="43" Body="&lt;p&gt;My question concerns a specific statement in this paper: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1007/3-540-46084-5_218&quot; rel=&quot;nofollow noreferrer&quot;&gt;N. N. Schraudolph and T. Graepel, &quot;Conjugate Directions for Stochastic Gradient Descent,&quot; in &lt;em&gt;Int. Conf. Artificial Neural Networks&lt;/em&gt;, Berlin, 2002 pp. 1351–1356.&lt;/a&gt; The preprint is available &lt;a href=&quot;http://www.cse-lab.ethz.ch/wp-content/papercite-data/pdf/schraudolph2002b.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Schraudolph and Graepel address the problem that in stochastic settings of ill-conditioned problems the sampled estimate of the gradient is very unlikely to point in the right direction due to the ill-conditioning of the (estimated) Hessian.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Chapter 3, the authors claim that directions associated with large eigenvalues of the Hessian (directions of large curvature) can be identified by multiplying the stochastic estimates of Hessian $H$ and gradient $g$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand this argument. What point am I missing? &lt;/p&gt;&#xA;" OwnerUserId="28513" LastEditorUserId="9667" LastEditDate="2018-08-20T22:04:48.043" LastActivityDate="2018-08-20T22:04:48.043" Title="Stochastic conjugate directions to improve convergence in narrow valleys" Tags="|optimization|conjugate-gradient|stochastic|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30064" PostTypeId="1" CreationDate="2018-08-20T20:17:16.370" Score="2" ViewCount="94" Body="&lt;p&gt;I have a situation where I'm interested in solving the same ODE many times, from different initial locations. More precisely, I'm interested in solving an ODE of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{dx}{dt} &amp;amp;= f(x) \\&#xA;x(0) &amp;amp;= a \in \mathbb{R}^n&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I know that this ODE has a $T$-periodic orbit $\mathcal{A}$, i.e for $a \in \mathcal{A}$, we have that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;x(0) &amp;amp;= a \implies x(T) = a&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is such that at each iteration $k$, I am given a point $a_k \in \mathcal{A}$, and need to output where the ODE takes $a_k$ after time $t$, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{dx}{dt} &amp;amp;= f(x) \\&#xA;x(0) &amp;amp;= a_k \\&#xA;\implies b_k &amp;amp;\triangleq x(t)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I want to report $b_k$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is there a better way of doing this than using an ODE solver at every step $k$? For example, if possible, I would prefer to incur a one-off cost at the start of my algorithm which allows me to map directly from $a_k$ to $b_k$, rather than having to repeatedly solve the same problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Additional details&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system I'm interested in is a two-dimensional Hamiltonian system, given in coordinates as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;\frac{dq}{dt} &amp;amp;= |p|^{\alpha - 1} \cdot \text{sign} (p) \\&#xA;\frac{dp}{dt} &amp;amp;= -|q|^{\beta - 1} \cdot \text{sign} (q)&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $\alpha, \beta &amp;gt; 1$ such that $1/\alpha + 1/\beta = 1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard integrator for such systems is the leapfrog integrator, which I would use if I was solving for a single trajectory. As written, the system is non-smooth - I would rather not worry about that right now, and if it makes the question easier to answer, please feel free to assume that everything involved &lt;em&gt;is&lt;/em&gt; smooth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When $\alpha = \beta = 2$ things are much easier, and I can handle this situation fine - I'm asking about the other cases.&lt;/p&gt;&#xA;" OwnerUserId="28518" LastEditorUserId="28518" LastEditDate="2018-08-20T23:07:35.830" LastActivityDate="2018-08-20T23:07:35.830" Title="Solving an ODE from many starting points" Tags="|ode|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30065" PostTypeId="1" AcceptedAnswerId="30073" CreationDate="2018-08-20T20:33:40.660" Score="1" ViewCount="162" Body="&lt;p&gt;I'm trying to solve this equation (Windkessel equation) numerically as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C \frac{d P}{d t} + \frac{P}{R} = Q(t)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $C$ is compliance, $R$ is resistance, $P$ is pressure, and $Q(t)$ is a known input signal (i.e. volumetric flux). I tried explicit (i.e. Forward Euler Method) as well as semi-implicit (i.e. Backward Euler Method) and trapezoidal time integration techniques. But, all of these three well-known methods fail at some values for $C$ and $R$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I wanted to know how can I map the stability region of this ODE based on $R$ and $C$ values? Also, what's your suggestion in terms of the time integration method? For example, will the 4th order Runge-Kutta method help me to reach a broader stability zone or not? Any comment or idea is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="28407" LastEditorUserId="28407" LastEditDate="2018-08-21T15:19:53.660" LastActivityDate="2018-08-22T04:34:45.493" Title="Stability Criteria for Numerical Solution of Windkessel Ordinary Differential Equation" Tags="|finite-difference|time-integration|runge-kutta|implicit-methods|explicit-methods|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30066" PostTypeId="1" CreationDate="2018-08-20T22:52:45.190" Score="1" ViewCount="30" Body="&lt;p&gt;Lets say that one has a scalar field defined in 3D space for whose gradient he wants to find the Morse-Smale Complex for later performing an integration of several hexa-dimensional functions over attractor domains. The functions to be integrated are mostly concentrated around the attractor with relatively small contributions at the boundary. As well, the boundary can have a spiky shape. What approach (or algorithm or even software) would be appropriate to tackle this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently the problem is divided in two steps: first the boundary of each attractor is found by testing in many points in space if following the gradient drives the search to other attractor or not. The initial points are chosen along rays starting at the attractor. Those rays point in the direction of the points generated by a Lebedev grid on a sphere of unit radius. So later the integration is separated in radial and angular variables. For each point in the radial quadrature (that goes no further than the point of the boundary found furthest from the attractor) the corresponding sphere is integrated checking for each angular point if it is inside or outside of the region of integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if there is a more efficient approach around it that has enough resolution to treat the spiky boundaries and the maintains the benefits of the separation in angular and radial integration.&lt;/p&gt;&#xA;" OwnerUserId="21198" LastActivityDate="2018-08-20T22:52:45.190" Title="Algorithm for integrating a 6D function in a Morse-Smale 3D cell" Tags="|quadrature|integration|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30070" PostTypeId="2" ParentId="30058" CreationDate="2018-08-21T18:00:30.997" Score="5" Body="&lt;p&gt;Upon further examination, I do think the Woodbury identity can be used to solve this problem. With it we can write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1} = \mathbf K^{-1} - \mathbf K^{-1} \mathbf Z \left(-\frac{1}{\sigma}\mathbf I + \mathbf Z^{T} \mathbf K^{-1} \mathbf Z\right)^{-1}\mathbf Z^T \mathbf K^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $\mathbf K$ is symmetric positive definite, you can write $\mathbf K^{-1}$ in terms of the Cholesky triangle $\mathbf L$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathbf K^{-1} = \left(\mathbf L \mathbf L^T \right)^{-1} = \mathbf L^{-T} \mathbf L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Insert this representation, then simplify by introducing $\mathbf Y = \mathbf L^{-1} \mathbf Z$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1} = \mathbf L^{-T} \mathbf L^{-1} - \mathbf L^{-T} \mathbf Y \left(-\frac{1}{\sigma}\mathbf I + \mathbf Y^{T} \mathbf Y\right)^{-1}\mathbf Y^T \mathbf L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now regroup:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1} = \mathbf L^{-T} \left[ \mathbf I - \mathbf Y \left(-\frac{1}{\sigma}\mathbf I + \mathbf Y^{T} \mathbf Y\right)^{-1}\mathbf Y^T \right] \mathbf L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is progress. It's not quite an $\mathbf L \mathbf D \mathbf L^T$ decomposition, but it might be good enough in the sense that it's a symmetric real congruence transform of the perturbed $\mathbf K$. So the square-bracketed part, despite being non-diagonal, still embeds the inertia. To proceed, introduce the matrix $\mathbf S = \left(-\frac{1}{\sigma}\mathbf I + \mathbf Y^{T} \mathbf Y\right)$, which we should be able to tabulate explicitly because it's so small (only $r \times r$). We can also readily compute/apply its inverse (using dense $\mathbf L \mathbf D \mathbf L^T$):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1} = \mathbf L^{-T} \left[ \mathbf I - \mathbf Y \mathbf S^{-1}\mathbf Y^T \right] \mathbf L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continue teasing at the bracketed part to get it to display its eigenvalues. Compute the $\mathbf Q \mathbf R$ decomposition of the tall/skinny matrix $\mathbf Y$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1} = \mathbf L^{-T} \left[ \mathbf I - \left(\mathbf Q \mathbf R\right)\mathbf S^{-1} \left( \mathbf Q \mathbf R \right)^T \right] \mathbf L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regroup, then compute the eigendecomposition of the (symmetric) matrix $\mathbf R \mathbf S^{-1} \mathbf R^T = \mathbf V \mathbf \Lambda \mathbf V^T$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1} = \mathbf L^{-T} \left( \mathbf I - \left(\mathbf Q \mathbf V\right) \mathbf \Lambda \left( \mathbf Q \mathbf V \right)^T \right) \mathbf L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inner data structure, $\left(\mathbf Q \mathbf V\right) \mathbf \Lambda \left( \mathbf Q \mathbf V \right)^T$, is like a &quot;thin&quot; eigendecomposition. For sake of illustration, fatten it back up by including the implicit zero eigenpairs. Group the handful of non-null eigenvectors into the orthogonal matrix $\mathbf U = \mathbf Q \mathbf V$ and denote the null eigenvectors as $\mathbf N = \text{null}(\mathbf U^T)$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1} = \mathbf L^{-T} \left( \mathbf I - \left[\begin{array}{cc} \mathbf U &amp;amp; \mathbf N \end{array}\right] \left[\begin{array}{cc} \mathbf \Lambda &amp;amp; \mathbf 0 \\ \mathbf 0 &amp;amp; \mathbf 0 \end{array}\right] \left[\begin{array}{cc} \mathbf U &amp;amp; \mathbf N \end{array}\right]^T \right) \mathbf L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course $\mathbf I$ can be simultaneously diagonalized with any other matrix, so fold it right into this structure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1} = \mathbf L^{-T} \left( \left[\begin{array}{cc} \mathbf U &amp;amp; \mathbf N \end{array}\right] \left[\begin{array}{cc} \mathbf I - \mathbf \Lambda &amp;amp; \mathbf 0 \\ \mathbf 0 &amp;amp; \mathbf I \end{array}\right] \left[\begin{array}{cc} \mathbf U &amp;amp; \mathbf N \end{array}\right]^T \right) \mathbf L^{-1}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This displays eigenvalues/inertia. Since the perturbation was only low rank, most of them will stay positive as indicated the lower right $\mathbf I$ block. However, from any $\mathbf \Lambda_i$ that is greater than one, $\left( \mathbf K - \sigma \mathbf Z \mathbf Z^T \right)^{-1}$ will pick up a negative eigenvalue from the upper left $\mathbf I - \mathbf \Lambda$ block. Since the eigenvalues of $\mathbf K - \sigma \mathbf Z \mathbf Z^T$ are just the reciprocals, the signs/inertia are the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarize, here are the essential calculations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Factor $\mathbf K = \mathbf L \mathbf L^T$, using a sparse direct package.&lt;/li&gt;&#xA;&lt;li&gt;Form $\mathbf Y = \mathbf L^{-1} \mathbf Z$, basically a &quot;fat&quot; backsolve across multiple RHS's. But note that this is only &quot;half&quot; of the normal sparse forward/backward solution. A high quality sparse solver package will expose this as an independent calculation (MKL PARDISO does, and so does my own package, MyraMath).&lt;/li&gt;&#xA;&lt;li&gt;Initialize the small matrix $\mathbf S = \mathbf Y^T \mathbf Y$. This is basically a &quot;fat&quot; dot product, can be done with BLAS [dsyrk].&lt;/li&gt;&#xA;&lt;li&gt;Downdate $\mathbf S = \mathbf S -\frac{1}{\sigma} \mathbf I$.&lt;/li&gt;&#xA;&lt;li&gt;Form the $\mathbf L \mathbf D \mathbf L^T$ decomposition of $\mathbf S$. Can be done with LAPACK [dsytrf].&lt;/li&gt;&#xA;&lt;li&gt;Form the $\mathbf Q \mathbf R$ decomposition of $\mathbf Y$. Can be done with LAPACK [dgeqrf].&lt;/li&gt;&#xA;&lt;li&gt;Form the small matrix $\mathbf T = \mathbf R \mathbf S^{-1} \mathbf R^{T}$. You can use [dystrs] to apply $\mathbf S^{-1}$ to $\mathbf R$ into a temporary, then use [dgemm] to form $\mathbf R \cdot \left( \mathbf S^{-1} \mathbf R \right)$&lt;/li&gt;&#xA;&lt;li&gt;Compute the eigendecomposition of $\mathbf T = \mathbf V \mathbf \Lambda \mathbf V^T$, using LAPACK's [dsyevd].&lt;/li&gt;&#xA;&lt;li&gt;The number of negative eigenvalues of $\mathbf K - \sigma \mathbf Z \mathbf Z^T$ is the number of $\mathbf \Lambda_i$'s above 1. All the others eigenvalues are positive. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;... I think? It seems reasonably efficient to me, as long as you are able to explicitly store $\mathbf Z$ (and therefore $\mathbf Y$). As an aside, note that $\mathbf N$ is never needed/computed, it's only introduced for explanatory purposes.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-08-21T18:05:46.057" LastActivityDate="2018-08-21T18:05:46.057" CommentCount="19" ContentLicense="CC BY-SA 4.0" />
  <row Id="30072" PostTypeId="1" AcceptedAnswerId="30074" CreationDate="2018-08-22T01:35:17.887" Score="2" ViewCount="3047" Body="&lt;p&gt;I want to solve the time-dependent Schrodinger Equation using the Crank-Nicolson scheme. I end up with the following matrix equation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A_fixed = ...  # the matrix that does the finite difference method&#xA;B_fixed = ...  # the matrix that does the finite difference method&#xA;A = A_fixed - diags(V)  # Add the potential term V to the diagonal of A_fixed&#xA;B = B_fixed + diags(V)  # Add the potential term V to the diagonal of A_fixed  &#xA;&#xA;# solve for psi_new at each time step:&#xA;A.dot(psi_new) = B.dot(psi_old)   # psi_old is the solution at the previous time-step&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And $A$ and $B$ are sparse matrices with all the terms along the main/off-diagonals (is this called 'banded'?). What are the best ways to solve for the above matrix equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the scipy library &lt;code&gt;sp.sparse.linalg&lt;/code&gt;, where it provides a &lt;code&gt;solve&lt;/code&gt; method which uses LU decomposition. If the potential term $V$ is time-independent, then both $A$ and $B$ are actually fixed throughout the whole computation, so I can use &lt;code&gt;sp.sparse.linalg.factorized&lt;/code&gt; to pre-factorize $A$. This gives me like a 2x speed up. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if $V$ is time-varying, so $A$ and $B$ changes at each time step? Are there are algorithms which work for this specific problem (where both $A$ and $B$ consist of fixed terms and some time-varying terms which are only added to the main diagonal)?&lt;/p&gt;&#xA;" OwnerUserId="21426" LastEditorUserId="20688" LastEditDate="2018-08-22T22:38:16.413" LastActivityDate="2018-08-22T22:38:16.413" Title="Ways to solve $Ax=b$ for a sparse (banded) $A$ with updates" Tags="|linear-algebra|sparse-matrix|scipy|banded-matrix|matrix-factorization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30073" PostTypeId="2" ParentId="30065" CreationDate="2018-08-22T04:34:45.493" Score="2" Body="&lt;p&gt;Backward Euler and the implicit trapezoidal rule are both unconditionally stable for this problem. If you're seeing instability then you haven't implemented them correctly. &lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2018-08-22T04:34:45.493" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30074" PostTypeId="2" ParentId="30072" CreationDate="2018-08-22T14:45:25.500" Score="6" Body="&lt;p&gt;If the only non-zero entries of $A_{ij}$ have $j$ in $\{i - 1, i, i + 1\}$, then $A$ is a &lt;em&gt;banded&lt;/em&gt; matrix with &lt;em&gt;bandwidth&lt;/em&gt; 1.&#xA;More generally, you can talk about matrices of bandwidth $k$ where $k$ is any integer.&#xA;For example, if you were using a higher-order finite difference discretization that used more points to calculate a derivative, you'd get higher-bandwidth matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible to solve a system with a banded matrix in $O(\text{bandwidth}^2\cdot n)$ time, where $n$ is the size of the matrix; this is essentially the &lt;a href=&quot;https://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Thomas algorithm&lt;/a&gt;.&#xA;Since you're using scipy already, it has a built-in &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.solveh_banded.html#scipy.linalg.solveh_banded&quot; rel=&quot;noreferrer&quot;&gt;banded matrix solver&lt;/a&gt;, although you'll probably have to change how you store the matrices to the banded format.&#xA;A banded solver should be substantially faster than a general sparse factorization because it avoids having to store the factored form of the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the time-varying terms are concerned, your best bet is to just call the banded solver on the updated matrix on every timestep.&#xA;If you were using iterative methods and the time-varying terms were small in magnitude compared to the time-independent terms, there might be some advantage to be had if you were using an iterative linear solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While your matrix is banded, many people have asked about updating the LDU factorization of a general sparse or dense matrix after adding, say, a multiple of the identity matrix or a diagonal matrix.&#xA;For general matrices and full-rank updates, you are &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10630/full-rank-update-to-cholesky-decomposition/10631#10631&quot;&gt;out of luck&lt;/a&gt; and need to use iterative methods if you want to leverage information from past solves.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2018-08-22T14:45:25.500" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30076" PostTypeId="1" AcceptedAnswerId="30086" CreationDate="2018-08-22T15:59:45.117" Score="0" ViewCount="585" Body="&lt;p&gt;I want to see tetrahedron volumes generated from STL meshes in ParaView (5.5.2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a given STL (simple cube, for example), I can generate a volume mesh using gmsh, and export it as .vtk UnstructuredGrid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I open it with notepad, I can inspect that its properly generated, with all the surface and volume elements and their types (5,10). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when opening this file in ParaView, I can see the surface, but when I do &lt;code&gt;Representation-&amp;gt;Volume&lt;/code&gt;, there is just nothing there, it does not seem to load the volumetric data, only the surface data.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The information seems to suggest that the number of loaded cells is correct (matches surface+volume cells in vtk).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I see the volumetric tetrahedrons inside my mesh? Ideally I'd like to clip my mesh to be able to see inside.&lt;/p&gt;&#xA;" OwnerUserId="26665" LastActivityDate="2018-08-24T12:44:58.437" Title="Loading VTK tetrahedron volumes into ParaView" Tags="|paraview|vtk|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30077" PostTypeId="2" ParentId="30076" CreationDate="2018-08-22T22:08:29.953" Score="1" Body="&lt;p&gt;For volume rendering in ParaView your vtk file should contain a scalar, vector, or tensor data array. Otherwise, it does not show anything to you. You can create a test data array by going to Filters -&gt; Calculator. Then you should see something in your volume rendering.&lt;/p&gt;&#xA;" OwnerUserId="28407" LastActivityDate="2018-08-22T22:08:29.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30078" PostTypeId="2" ParentId="338" CreationDate="2018-08-23T07:17:34.290" Score="-1" Body="&lt;p&gt;I have made my own simple implementation and created short tutorial, hope it will be helpfull:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i just converted the gillespie's algorithm into python code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.Define total simulation time T and set current time t=0;&#xA;2.Initialize the state of the system x – choose randomly or set zero state;&#xA;3.Find the sum Q of all the possible transition rates from the current state x;&#xA;4.Simulate the time Δt until the next transition by drawing from an exponential distribution with parameter Q;&#xA;5.Generate a random number r from a uniform distribution;&#xA;6.Set the next state n as follows:&#xA;If 0 &amp;lt; r &amp;lt; q(x,0)/Q, choose transition 0; if q(x,0)/Q &amp;lt; r &amp;lt; (q(x,0) + q(x,1))/Q choose transition 1, and so on;&#xA;7.Update the current time: t=Δt+t;&#xA;8.Repeat 3-7 until t≤T.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source code in python: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mamedshahmaliyev/adhoctuts/blob/master/python/math/mc_gillespie.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mamedshahmaliyev/adhoctuts/blob/master/python/math/mc_gillespie.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;additionally, the result of simulation is compared with the exact result using mean squared error (MSE). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://adhoctuts.com/find-stationary-distribution-of-markov-chain-using-stochastic-simulation-gillespies-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://adhoctuts.com/find-stationary-distribution-of-markov-chain-using-stochastic-simulation-gillespies-in-python/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://youtu.be/3DRdoqhNFrk&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://youtu.be/3DRdoqhNFrk&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28534" LastEditorUserId="28534" LastEditDate="2018-08-28T08:08:49.463" LastActivityDate="2018-08-28T08:08:49.463" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30081" PostTypeId="1" CreationDate="2018-08-23T23:55:01.237" Score="1" ViewCount="248" Body="&lt;p&gt;I am trying to solve a set of coupled, nonlinear ODEs. The only dependent variable is a 1-dimensional spatial coordinate, let's call it $x$. For now, I've managed to approximate away some of the coupling between the equations, such that the first one doesn't depend on the second one at all. I solved it with FEM without major difficulty: boundary conditions met, residuals look good, physically meaningful result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now I'm left with one nonlinear, second-order ODE where the nonlinear coefficient functions are completely determined by the FEM solution, something like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \begin{cases}&#xA;A(x) + B(x) f(x) + C(x) f'(x) + D(x) f''(x) = 0 \\&#xA;f(0) = f(1) = 0&#xA;\end{cases} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$, $B$, $C$, and $D$ are known, at least at the resolution of the FEM mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that, using the same basic code as before, the solution appears to violate the boundary conditions entirely, blowing up as $x \rightarrow 0$ instead of approaching zero (or even any finite value, it doesn't seem to matter what I choose). Thinking this was a problem with my code, I experimented with a finer mesh and with finite differences instead of finite elements, with no success. It finds the same nonsensical result every time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I put my numerical data for the coefficients into Mathematica and attempted to solve the above BVP. I let it run for quite a while (maybe 15-30 minutes) before giving up. It never found a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So finally, my question: Is it possible that this equation simply can't be solved with these boundary conditions? For example, I tried changing $f(1) = 0 \rightarrow f'(0) = 0$ and the solution was found almost instantaneously by Mathematica, but it still violated the new boundary condition; somehow, that didn't trigger an error, but it made me suspicious that something &quot;bigger&quot; might be going on here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for some mathematical intuition behind why the numerical methods might fail in these cases and how to mitigate these problems.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Update&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;This equation is linear (correcting here so that comments still make sense). I think that's not the root of my problem, though; I tried using a linear solver with the proper stiffness matrix and load vector to solve $-f''(x) = g(x)$, which should be EASY; here, $g(x)$ is the FEM solution to the first equation. The solution meets boundary conditions but is &quot;jagged&quot; (think sawtooth patterns overlaid on a smooth function). Now I think perhaps there's something wrong with substituting the FEM solution for the first equation into the second equation, but I'm not sure why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not giving up yet.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Update 2&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I think I can safely rule out a problem with this code; I am generating the source files with SymPy, so there shouldn't be any human errors associated with changing functional forms, etc. of the equations I want to solve. As an example of what I'm seeing now, see figure below. The only intuition I have for this is that the code &quot;tries&quot; to meet boundary conditions (which are homogeneous Dirichlet), but the solution, for whatever reason, fundamentally wants to blow up to $-\infty$. So we end up with this &quot;compromise&quot; solution that straddles the two possibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this even remotely correct? It doesn't seem to matter what I change at this point, and refining the mesh just increases the frequency of the sawtooth form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AXUFZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AXUFZ.png&quot; alt=&quot;Solution doesn&amp;#39;t meet boundary condition&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="25718" LastEditorUserId="25718" LastEditDate="2018-08-31T03:41:52.457" LastActivityDate="2018-09-30T04:00:52.023" Title="Can a second-order ODE be &quot;inconsistent&quot; with its boundary conditions?" Tags="|finite-element|finite-difference|ode|differential-equations|mathematica|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30082" PostTypeId="2" ParentId="30081" CreationDate="2018-08-24T03:11:29.997" Score="1" Body="&lt;p&gt;It is impossible that finite element or finite difference violate their boundary condition that you imposed. It is not related to the physics of your problem and it's just an issue with your implementation. No matter your ODE is consistent with your boundary conditions or something else. The only thing matters is that you enforce your boundary conditions when you are trying to assemble your stiffness matrix and response vector in your finite element or finite difference methods and if you assemble them correctly it should recover the imposed boundary conditions. Otherwise maybe you have a bug in your matrix assembling which makes the stiffness matrix singular, which shows you some nonsense values at the boundaries. As a result, my suggestion is that check your matrix assembling function and also check if your linear solver or whatever could generate informative warnings/errors when something is wrong with your stiffness matrix or response vector. You need to get at least whatever you wanted to impose on your equation correctly and then that's an another story when you want to discuss about the physical meaning of your model, which makes sense or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also your ODE is not nonlinear because $A(x)$, $B(x)$, $C(x)$, and $D(x)$ do not depend on $f$ itself and as a result it should be solved easily by simple linear finite element or finite difference solvers (e.g. scipy BVP solver).&lt;/p&gt;&#xA;" OwnerUserId="28407" LastEditorUserId="28407" LastEditDate="2018-08-24T13:11:41.977" LastActivityDate="2018-08-24T13:11:41.977" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30083" PostTypeId="1" AcceptedAnswerId="30087" CreationDate="2018-08-24T04:01:39.803" Score="0" ViewCount="55" Body="&lt;p&gt;1) Which approach from the following approaches &quot;a&quot; and &quot;b&quot; is used while performing crossover?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) By selecting any 2 parents randomly from selected population, crossing them, forming just TWO offsprings, adding them in population and throwing out just TWO least fit individuals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, here, the new generation will have just 2 new individuals? Rest all individuals will be old?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) By selecting parents and forming offsprings equal to population size and replacing the entire old population with new one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, here , in new generation, all individuals will be the new offsprings that were generated by previous population?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Also, where will the crossover probability come into picture here?&lt;/p&gt;&#xA;" OwnerUserId="28541" LastEditorUserId="28541" LastEditDate="2018-08-24T04:07:07.663" LastActivityDate="2018-08-24T13:01:51.667" Title="Doubts in genetic algorithm" Tags="|genetic-algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30084" PostTypeId="1" CreationDate="2018-08-24T06:39:10.097" Score="1" ViewCount="58" Body="&lt;p&gt;I have an application that is somewhat similar to the situation of computing Gaussian quadrature nodes and weights: simply put, I need to compute the eigenvalues and the last two (normalized) components of the eigenvectors of a symmetric tridiagonal matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of the Golub-Welsch scheme that uses the implicit QL/QR algorithm for Gaussian quadrature rule generation; the difference between it and my situation is that one uses the first component of the eigenvectors for Golub-Welsch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can modify the relevant LAPACK routines for my specific situation, but I was wondering if there were any new developments in this area. From my limited searching, it does not seem that MRRR or divide-and-conquer allow for computing only selected components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Literature pointers would be extremely appreciated.&lt;/p&gt;&#xA;" OwnerUserId="28543" LastEditorUserId="-1" LastEditDate="2018-08-24T13:41:50.113" LastActivityDate="2018-08-24T13:41:50.113" Title="Methods to compute specific eigenvector components for a tridiagonal matrix" Tags="|linear-algebra|eigensystem|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30086" PostTypeId="2" ParentId="30076" CreationDate="2018-08-24T12:44:58.437" Score="1" Body="&lt;p&gt;What I want to do can be done by adding cell information to the VTK file, or in ParaView by using &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Filter-&gt;Clip with the &quot;crinkle cut&quot; activated.&lt;/p&gt;&#xA;" OwnerUserId="26665" LastActivityDate="2018-08-24T12:44:58.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30087" PostTypeId="2" ParentId="30083" CreationDate="2018-08-24T13:01:51.667" Score="2" Body="&lt;p&gt;Both (a) and (b) are valid options that appear in real algorithms. Option (a) is called a &quot;steady state genetic algorithm&quot; and option (b) a &quot;generational genetic algorithm&quot;, though (b) is less likely to be referred to by a specific name, since it's more like the standard way of doing things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Realistically, there are no &quot;right&quot; answers. There is almost a continuum of ways to do this kind of thing, with different points along that spectrum occasionally getting a name to refer to them (e.g., &quot;elitism&quot; is almost generational, but maybe you keep one or two of the best individuals from the parent population). Ultimately your job as a GA practitioner is to understand the implications of how you set these parameters so that you can choose effectively for a particular situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Crossover probability is just the probability that you perform the operator versus passing two parents down unmodified. Something like the following inside the core loop of your algorithm.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p1, p2 = select parents&#xA;if rand01() &amp;lt; crossover probability&#xA;    c1, c2 = crossover(p1, p2)&#xA;else&#xA;    c1, c2 = p1, p2&#xA;insert c1, c2 into child population&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28216" LastActivityDate="2018-08-24T13:01:51.667" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30088" PostTypeId="2" ParentId="24998" CreationDate="2018-08-24T16:47:11.583" Score="2" Body="&lt;p&gt;If you are interested in comparing two integrators solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dq}{dt} = p \\&#xA;\frac{dp}{dt} = -q$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial values $(q,p) = (1,0)$ and $dt = 0.1$ for both of them. Then plot the errors &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$dq = q - \cos(t) ; dp = p + \sin(t)$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;for a reasonable range of $t$ (0 to 100, one thousand time steps in all) and choose the integrator with the smaller error. This harmonic oscillator test will show you the phase and amplitude errors with very little effort.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;P J Channell and C Scovel. &lt;a href=&quot;http://iopscience.iop.org/article/10.1088/0951-7715/3/2/001/meta&quot; rel=&quot;nofollow noreferrer&quot;&gt;Symplectic integration of Hamiltonian systems&lt;/a&gt;. Journal of Computational Physics 73, 468 (1973).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28550" LastEditorUserId="9667" LastEditDate="2018-08-25T17:56:02.897" LastActivityDate="2018-08-25T17:56:02.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30089" PostTypeId="1" CreationDate="2018-08-23T20:27:18.790" Score="3" ViewCount="178" Body="&lt;p&gt;I am interested in solving the following semidefinite optimization problem:&#xA;\begin{equation}&#xA;\begin{split}&#xA;\underset{X,\lambda}{\rm maximize} \;\;\;\;&amp;amp;\lambda^Tc \\&#xA;&amp;amp;-\mathbb{I} \le X \le \mathbb{I} \\&#xA;&amp;amp;X - \sum_{k=1}^m \lambda_k A_k \ge 0,&#xA;\end{split}&#xA;\end{equation}&#xA;where $\lambda,c \in \mathbb{R}^m$ and $X,A_k$, for $k = 1,\ldots,m$, are symmetric $n\times n$ matrices, where $c$ and the $A_k$'s are fixed. Further, $\mathbb{I}$ denotes the $n \times n$ identity matrix. I would like to solve such optimizations for $n$ large ($\mathcal{O}(10^3)$ and larger) and $m \sim 100$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any algorithms that perform better than alternating projection for those dimensions? What is the state of the art algorithm for such a problem?&lt;/p&gt;&#xA;" OwnerUserId="28557" OwnerDisplayName="Marc" LastActivityDate="2018-08-24T18:14:25.773" Title="First order methods for a large scale semidefinite program" Tags="|algorithms|convex-optimization|semidefinite-programming|" AnswerCount="0" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30090" PostTypeId="1" AcceptedAnswerId="30091" CreationDate="2018-08-24T19:05:18.730" Score="4" ViewCount="148" Body="&lt;p&gt;Suppose I want to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\text{arg min}_x \frac{1}{2}\|\tilde{A}x - b\|_2^2 + \frac{1}{2}\|x - c\|_2^2$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$ is a wide sparse matrix and $\tilde{A} = A C_n  = A (I - \mathbf{1}\, \mathbf{1}^T/n)$ is the &quot;centered&quot; version of $A$, typically non-sparse. [1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am willing to materialize $\tilde{A}$, I know several efficient methods to solve this problem, but I'd like to avoid doing so if possible. ($\tilde{A}$ is small enough to fit on a single machine, but large enough that a sparsity-aware method may be worthwhile.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any literature on this problem? (It arises fairly often in statistics, where centering $A$ makes the intercept of a linear regression independent of the other coefficients.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it matters, I'll actually need to solve this problem repeatedly for different values of $c$, so I am willing to &quot;pay for&quot; a more expensive factorization and reuse it if possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://en.wikipedia.org/wiki/Centering_matrix&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Centering_matrix&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28552" LastActivityDate="2018-08-25T15:54:09.817" Title="Method to Efficiently Solve &quot;Centered&quot; Least Squares without centering &quot;A&quot;" Tags="|linear-algebra|sparse-matrix|least-squares|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30091" PostTypeId="2" ParentId="30090" CreationDate="2018-08-25T15:54:09.817" Score="5" Body="&lt;p&gt;How would you solve the problem if you didn't need to do the centering?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since $A$ is large and sparse, you'd probably pick an iterative method such as CGNE which depends on being able to perform matrix-vector multiplies $Ax$ and $A^{T}y$.  It turns out that you can still use the same iterative method for the centered version of the problem since matrix-vector multiplications with $\tilde{A}$ aren't really slower than matrix-vector multiplications with $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{A}x=A(I- 1\; 1^{T}/n)x=Ax-(A1/n)(1^{T}x)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{A}^{T}y=(A(I-1\;1^{T}/n)^{T}y=A^{T}y-1(1^{T}A^{T}/n)y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In these two expressions, you'd only have to compute $A1/n$ and its transpose $1^{T}A^{T}/n$ once.  The rest of the work takes trivial $O(n)$ time.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you were going to use a QR factorization of $A$, you'd have to deal with the fact that the Q matrix is typically fully dense even though $A$ is sparse.  You could use a rank-one update procedure (qrupdate in MATLAB) to update the factorization of $A$ to a factorization of $\tilde{A}$, but this would probably be no faster than centering $A$ and then finding its QR factorization.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-08-25T15:54:09.817" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30092" PostTypeId="1" CreationDate="2018-08-26T13:53:02.637" Score="6" ViewCount="170" Body="&lt;p&gt;I use interval arithmetic for reliable computing. Now, a procedure coded in a good implementation of interval arithmetic takes perhaps about eight times as much as the same procedure carried out without interval arithmetic (if the lion's share of runtime is taken by floating-point computations). I have sometimes been asked why I don't do rounding-error analysis by hand instead. To me, that does not look like a good option: it amounts to saving computer time at the expense of human time, and creates one more occasion for human error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what would make sense for me would be for there to be a system for bounding rounding errors with the help of a computer. It might not be realistic to expect a computer to analyse the total error in fairly complex procedures, but a human and a computer might be able to do a quicker and more reliable job than a human working alone, and produce a certificate of correctness to boot. Cf. how it is much easier to produce a formal proof with computer assistance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In particular: I would be interested in how error analysis in the sense of e.g. Wilkinson could be used in the context of a rigorous proof. I could see how it might be possible to implement it by means of a system for the computer-aided production of formal proofs. Has such a thing been done in practice?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: perhaps it shows, but -- I am a pure mathematician; feel free to tell me if what I say seems a little naïve.&lt;/p&gt;&#xA;" OwnerUserId="21512" LastActivityDate="2021-08-19T21:37:33.643" Title="Compile-time error control vs. interval arithmetic?" Tags="|precision|numerics|interval-arithmetic|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="30095" PostTypeId="1" AcceptedAnswerId="30098" CreationDate="2018-08-27T16:19:29.310" Score="2" ViewCount="96" Body="&lt;p&gt;I am trying to find the diffusion coefficient ($D$) and the partition coefficient ($KLP$) using experimental data of desorption of a pollutant from a film into a liquid. This process can be modelled, under certain assumptions, following Fick's law (related question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/28864/mass-conservation-in-1d-diffusion-by-method-of-lines&quot;&gt;here&lt;/a&gt; for more details on the model) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Matlab's command &lt;em&gt;lsqnonlin&lt;/em&gt; to find both parameters from experimental data. In general, $D$ has a range between $10^{-16}$ and $10^{-12} m^2/s$ whereas $KLP$ is comprehended between $10^{-3}$ and $10^{3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes I have been advised to multiply the parameters so that the lowest bounds that &lt;em&gt;lsqnonlin&lt;/em&gt; has to deal with are around 1, so that the absolute tolerances are acceptable. Hence, the parameters are to be found are transformed as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D^* = D·10^{16}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$KLP^* = KLP·10^{3}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My intuition is that using a log-transformation such as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$D^* = log(D)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$KLP^* = log(KLP)$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is a better solution. But I am not able to prove it or to see the disadvantages of the log-transformation (lower accuracy?)&#xA;What transformation would be more advisable? And what would be the pros and cons?&lt;/p&gt;&#xA;" OwnerUserId="18706" LastActivityDate="2018-08-28T00:24:59.713" Title="Log-transformation of decision variables in parameter estimation" Tags="|optimization|matlab|error-estimation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30097" PostTypeId="2" ParentId="30058" CreationDate="2018-08-27T22:03:11.347" Score="2" Body="&lt;p&gt;This answer is based on rchilton1980 answer and the comments that have followed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $K = L L^T$ be the Cholesky factorization of $K$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $Y = L^{-1} Z$ , computed by forward substitution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\mathcal{I} (A)$ denote the number of negative eigenvalues of a given matrix $A$ (negative index of inertia).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We seek to compute $\mathcal{I}_\sigma = \mathcal{I} (K - \sigma M) = \mathcal{I} (L L^T - \sigma Z Z^T)$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the inertia is invariant under a change of basis (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_congruence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Matrix_congruence&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Sylvester%27s_law_of_inertia&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Sylvester%27s_law_of_inertia&lt;/a&gt;), inertia count $\mathcal{I}_\sigma$ can be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal{I}_\sigma = \mathcal{I} \left( L^{-1} \left( L L^T - \sigma Z Z^T \right) L^{-T} \right) = \mathcal{I} \left( I_N - \sigma Y Y^T \right) = \mathcal{I} \left( \frac{1}{\sigma} I_N - Y Y^T \right)$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, it should be noted that $Y Y^T$ and $Y^T Y$ have the same nonzero eigenvalues (see for instance &lt;a href=&quot;https://en.wikipedia.org/wiki/Singular-value_decomposition&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Singular-value_decomposition&lt;/a&gt;). The eigendecomposition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\left( Y Y^T \right) \Phi_Y = \Phi_Y \begin{bmatrix} \Lambda_Y &amp;amp; 0\\ 0 &amp;amp; 0 \end{bmatrix}$ such that $Y Y^T = \Phi_Y \begin{bmatrix} \Lambda_Y &amp;amp; 0\\ 0 &amp;amp; 0 \end{bmatrix} \Phi_Y^T$ and $\Phi_Y \Phi_Y^T = I_N$  yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal{I}_\sigma = \mathcal{I} \left( \Phi_Y \left( \frac{1}{\sigma} I_N - \begin{bmatrix} \Lambda_Y &amp;amp; 0\\ 0 &amp;amp; 0 \end{bmatrix} \right) \Phi_Y^T \right)$ and consequently,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\mathcal{I}_\sigma = \mathcal{I} \left( \frac{1}{\sigma} I_N - \begin{bmatrix} \Lambda_Y &amp;amp; 0\\ 0 &amp;amp; 0 \end{bmatrix} \right)$ , which is given by $\mathcal{I}_\sigma = \mathcal{I} \left( \frac{1}{\sigma} I_r - \Lambda_Y \right)$ .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Matrix $\Lambda_Y$ is the diagonal matrix of the eigenvalues of $Y^T Y$. Once $\Lambda_Y$ is calculated (independent of $\sigma$), then $\mathcal{I}_\sigma$ can be computed at no cost for any $\sigma &amp;gt; 0$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, we now show how it is possible to compute all the $r$ finite eigenvalues and associated eigenvectors of the generalized eigenvalue problem (GEP) $K \Phi = M \Phi \Lambda$ , with the same cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recalling that $M = Z Z^T$, it can be deduced that the eigenvectors belong to the image (column space) of $(N \times r)$ real matrix $S = K^{-1} Z$ (see for instance &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2294/large-scale-generalized-eigenvalue-problem-with-low-rank-lhs-matrix?rq=1&quot;&gt;Large-scale generalized eigenvalue problem with low rank LHS matrix&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying Rayleigh-Ritz (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Rayleigh%E2%80%93Ritz_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Rayleigh%E2%80%93Ritz_method&lt;/a&gt;) to above GEP (Galerkin projection onto $S$), a $(r \times r)$ GEP is obtained,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$(S^T K S) X = (S^T M S) X \Lambda$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is to say, an eigenvector $\boldsymbol{\varphi}$ of the $(N \times N)$ GEP is written as $\boldsymbol{\varphi} = S \textbf{x}$ with $\textbf{x}$ an eigenvector of the $(r \times r)$ GEP (that is, $\Phi = S X$). This procedure yields exact eigenpairs since $S$ spans the sought eigenspace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, it can be observed that $S^T K S = Y^T Y$ and $S^T M S = {(Y^T Y)}^2$. Introducing the notation $A = Y^T Y$, the $(r \times r)$ GEP writes $A X = A^2 X \Lambda$, which is equivalent to the standard eigenvalue problem (SEP) $A X = X \Lambda^{-1}$ . It can be recognized that $\Lambda = \Lambda_Y^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As a conclusion, the proposed approach for inertia count is as costly as directly computing all the eigenvalues of the associated GEP. The cost (dense eigenvalue problem with dimension $r$) rapidly increases with $r$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be noted that the cost for computing $\mathcal{I}_\sigma$ for one given $\sigma &amp;gt; 0$ can be less than that of computing all the eigenvalues, as one also has $\mathcal{I}_\sigma = \mathcal{I} \left( \frac{1}{\sigma} I_r - Y^T Y \right)$, which can be obtained by LDL factorization of $\frac{1}{\sigma} I_r - Y^T Y$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My next question would be how to determine the inertia count of $K_\sigma - \sigma Z Z^T$ with $K_\sigma = K - \sigma M_0$ , in which $M_0$ is a sparse $(N \times N)$ symmetric positive-semidefinite matrix. The difference with the above is that $K_\sigma$ is not positive definite. I am presently using the above developments to answer this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: See below for the answer to this question.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;We seek to compute $\mathcal{I}_\sigma = \mathcal{I} \left( K - \sigma M \right)$ with $M = M_0 + Z Z^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have $\mathcal{I}_\sigma = \mathcal{I} \left( K_\sigma - \sigma Z Z^T \right)$ with $K_\sigma = K - \sigma M_0$ a full-rank sparse symmetric matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LDL factorization of $K_\sigma$ is performed, which yields $K_\sigma = L_\sigma D_\sigma L_\sigma^T$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, we obtain $\mathcal{I}_\sigma = \mathcal{I} \left( L_\sigma^{-1} \left( K_\sigma - \sigma Z Z^T \right) L_\sigma^{-T} \right) = \mathcal{I} \left( D_\sigma - \sigma Y_\sigma Y_\sigma^T \right)$ with $Y_\sigma = L_\sigma^{-1} Z$. Which gives $\mathcal{I}_\sigma = \left( \frac{D_\sigma}{\sigma} - Y_\sigma Y_\sigma^T \right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Introducing the $(N \times N)$ GEP $\left( \frac{D_\sigma}{\sigma} \right) \Phi = \left( Y_\sigma Y_\sigma^T \right) \Phi \Lambda$ such that $\Phi^T \left( \frac{D_\sigma}{\sigma} \right) \Phi = \Lambda$ and $\Phi^T \left( Y_\sigma Y_\sigma^T \right) \Phi = I_N $ (unit mass eigenvector normalization), we obtain $\mathcal{I}_\sigma = \mathcal{I} \left( \Phi^T \left( \frac{D_\sigma}{\sigma} - Y_\sigma Y_\sigma^T  \right) \Phi \right) = \mathcal{I} \left( \Lambda - I_N \right)$. This GEP cannot be solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly to before, the $r$ eigenvectors associated with the $r$ finite eigenvalues belong to the column space of $S_\sigma = {\left( \frac{D\sigma}{\sigma} \right)}^{-1} Y_\sigma$. The $(r \times r)$ GEP $\left( S_\sigma^T \left( \frac{D_\sigma}{\sigma} \right) S_\sigma \right) X = \left( S_\sigma^T \left( Y_\sigma Y_\sigma^T \right) S_\sigma \right) X \Lambda_f$ allows to obtain the finite eigenvalues $\Lambda_f$ of the $(N \times N)$ GEP. Let $A_\sigma = S_\sigma^T \left( \frac{D_\sigma}{\sigma} \right) S_\sigma$. One has $A_\sigma = Y_\sigma^T {\left( \frac{D_\sigma}{\sigma} \right)}^{-1} Y_\sigma$ and $S_\sigma^T \left( Y_\sigma Y_\sigma^T \right) S_\sigma = A_\sigma^2$. Thus, the $r$ finite eigenvalues $\Lambda_f$ can be deduced from the $(r \times r)$ SEP $A_\sigma X = X \Lambda_f^{-1}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $\Lambda_i$ denote the matrix of the $N - r$ infinite eigenvalues.&#xA;We have $\mathcal{I}_\sigma = \mathcal{I} \left( \Lambda - I_N \right) = \mathcal{I} \left( \Lambda_f - I_r \right) + \mathcal{I} \left( \Lambda_i - I_{N - r} \right)$. Let $\mathcal{I}_f = \mathcal{I} \left( \Lambda_f - I_r \right)$ and $\mathcal{I}_i = \mathcal{I} \left( \Lambda_i - I_{N - r} \right)$. We have $\mathcal{I}_f = \mathcal{I} \left( X^T A_\sigma^{-1} X - X^T X \right)$ because $X^{-1} = X^T$, and consequently, $\mathcal{I}_f = \mathcal{I} \left( A_\sigma^{-1} - I_r \right)$. Since for a real scalar $a$ we have $a^{-1} \in ] - \infty , 1 [ \,\,\, \Rightarrow \,\, a \in ] - \infty , 0 [ \, \cup  \,] 1 , + \infty [\,$, it can be seen that $\mathcal{I}_f = \mathcal{I} \left( A_\sigma^{-1} - I_r \right) = \mathcal{I} \left( A_\sigma \right) + \mathcal{I} \left( I_r - A_\sigma \right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, we have $\mathcal{I}_i = \mathcal{I} \left( \Lambda_i - I_{N - r} \right) = \mathcal{I} \left( \Lambda_i \right)$ ($\mathcal{I}_i$ is given by the number of negative infinite eigenvalues). Since $\mathcal{I} \left( \Lambda \right) = \mathcal{I} \left( \Lambda_f \right) + \mathcal{I} \left( \Lambda_i \right)$ and $\mathcal{I} \left( \Lambda \right) = \mathcal{I} \left( \Phi^T \left( \frac{D_\sigma}{\sigma} \right) \Phi \right) = \mathcal{I} \left( \frac{D_\sigma}{\sigma} \right) = \mathcal{I} \left( D_\sigma \right)$, we obtain $\mathcal{I}_i = \mathcal{I} \left( D_\sigma \right) - \mathcal{I} \left( \Lambda_f \right)$. In addition, we have $\mathcal{I} \left( \Lambda_f \right) = \mathcal{I} \left( \Lambda_f^{-1} \right) = \mathcal{I} \left( A_\sigma \right)$. Therefore, we obtain $\mathcal{I}_i = \mathcal{I} \left( D_\sigma \right) - \mathcal{I} \left( A_\sigma \right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a conclusion, it can be deduced that $\mathcal{I}_\sigma = \mathcal{I} \left( D_\sigma \right) + \mathcal{I} \left( I_r - A_\sigma \right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary of the operations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;LDL factorization of $(N \times N)$ sparse matrix $K_\sigma = L_\sigma D_\sigma L_\sigma^T$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Solve $Y_\sigma = L_\sigma^{-1} Z$ by forward substitutions&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Assemble $A_\sigma = Y_\sigma^T {\left( \frac{D_\sigma}{\sigma} \right)}^{-1} Y_\sigma$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;LDL factorization of $(r \times r)$ dense matrix $ I_r - A_\sigma$&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28491" LastEditorUserId="28491" LastEditDate="2018-09-05T18:41:10.573" LastActivityDate="2018-09-05T18:41:10.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30098" PostTypeId="2" ParentId="30095" CreationDate="2018-08-28T00:24:59.713" Score="4" Body="&lt;p&gt;Transformations are usually a good idea if they are done to impose domain constraints. Using the exponential of a parameter (so storing the log, which I think is what you mean here) is a good way to constrain to a positive domain, and using the sigmoid of a parameter can put it in a bounded domain (and you can dictate the sharpness between the ends). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually the libm (math library) with whatever language you have is able to do log/exp/sigmoid/etc. to 1 ulp so there really isn't much of a numerical issue. In fact, usually it's the opposite: if your parameters can range between 4 orders of magnitude, then taking a log transform can decrease that range and make the gradients stay numerically closer to each other which helps with controlling round-off error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this reason, many libraries are actually doing transformations behind the scenes even if users haven't explicitly stated it. For example, if you declare a Stan variable to be positive, it is holding the log and exponentiating as necessary so that the HMC can utilize $p \in \mathbb{R}$ which but still acts strictly positive (in this case it's quite necessary since small numerical errors could overshoot the parameter into the negative regime if this wasn't done!)&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-08-28T00:24:59.713" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30099" PostTypeId="1" CreationDate="2018-08-28T00:57:59.253" Score="1" ViewCount="694" Body="&lt;p&gt;I want to resolve numerically this equation using of difference finite method with Leapfrog Scheme&#xA;$$\frac{\partial{u}}{\partial t}+ v \frac{\partial{u}}{\partial x}= 0 $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to write a code that resolves that PDE wit $v=1$ in leapfrog scheme (&lt;a href=&quot;http://www4.ncsu.edu/~zhilin/TEACHING/MA584/Chapter5_fd_fem.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www4.ncsu.edu/~zhilin/TEACHING/MA584/Chapter5_fd_fem.pdf&lt;/a&gt; page 7), this scheme uses two level of time, but, I don't know how to write it in my code&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_j^{n+1}=u_j^{n-1}-\frac{\Delta t}{\Delta x}\left[u_{j+1}^{n}-u_{j-1}^{n}\right]+\mathcal O(\Delta t^3,\Delta x^2)&#xA;$$&#xA;So I guess this is how could be.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do l=1,Nt&#xA;t = t + dt&#xA;u_n_p_o=u_n_o&#xA;u_n_p=u_n&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I calculated $u^{n-1}$ to using FTCS(centering difference) Scheme&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do i=1,Nx-1 &#xA;u_n_o(i)= u_n_p_o(i)- (dt/dx)*(u_n_p_o(i+1)- u_n_p_o(i-1))&#xA;end do &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually &lt;code&gt;u_n_o&lt;/code&gt; is correct ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u_n_p_o(i)=u_n_o(i-1) !2do reciclado de variable&#xA;&#xA;&#xA;do i=1,Nx-1&#xA;u_n(i)= u_n_p_o(i)- (dt/dx)*(u_n_p(i+1)-u_n_p(i-1))&#xA;end do &#xA;&#xA;u_e = amp * exp( - ( x - x0 - t)**2 / sigma**2 ) + 1e-20&#xA;call save1Ddata(Nx,t,x,u_e,'u_e',1)&#xA;call save1Ddata(Nx,t,x,u_n,'u_n',1)&#xA;call save1Ddata(Nx,t,x,u_n,'u_n_o',1)&#xA;end do&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What are the features of Leapfrog? I know that in this case, it's no matter the Courant factor (because it is stable); however, the solution numerically dissipates in its amplitude. So how can you test your numeric solution? &lt;/p&gt;&#xA;" OwnerUserId="28581" LastEditorUserId="28581" LastEditDate="2018-08-29T03:56:16.887" LastActivityDate="2018-09-28T20:01:06.570" Title="How write a integration loop in fortran, leapfrog scheme to solvind PDE (advection)?" Tags="|finite-difference|fortran|advection-diffusion|differential-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30102" PostTypeId="1" CreationDate="2018-08-28T17:18:48.957" Score="1" ViewCount="225" Body="&lt;p&gt;I asked a question on Stack Overflow &lt;a href=&quot;https://stackoverflow.com/questions/51900138/performance-issues-with-nearest-neighbor-search-and-vptrees&quot;&gt;Performance Issue with VP Trees and Nearest Neighbors&lt;/a&gt;and I was not satisfied with the answer and so I thought I would reword my question for this site and post this question as a computational science question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the &lt;a href=&quot;https://pypi.org/project/vptree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VPTree&lt;/a&gt; to store the coordinates of my sparse array as data is on a sphere/ellipsoid and I want to use great circle distances  or geodesics to do nearest neighbor searches as asked in a related question - &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7622/fast-nearest-neighbor-search-latitude-longitude&quot;&gt;Fast nearest neighbor search with latitude and longitude &lt;/a&gt;. The creation of the VPTree is itself not a problem as that happens very fast( I have 7000 points approximately). I create a regular grid in the following way and the minLat,maxLat,minLon,maxLon are obtained from the coarse data grid that contains the sparse matrix of observations. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;latGrid = np.arange(minLat,maxLat,0.05)&#xA;lonGrid = np.arange(minLo,maxLo,0.05)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the total number of points in latGrid is 1800 and the total number of points in lonGrid is 7200. So the total number of grid points is 1800 * 7200 = 12960000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to do a &lt;a href=&quot;https://en.wikipedia.org/wiki/Nearest_neighbor_search#Fixed-radius_near_neighbors&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fixed radius nearest neighbor search&lt;/a&gt; and that is where I have my problem currently - &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for grid_point in (grid_points):&#xA;    indices = tree.get_all_in_range(grid_point,r=4)&#xA;    args.append(indices)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run this query against 12960000 grid points it takes 12 minutes and I have to run this 175 times. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I build the VPTree using the 7000 points of the coarse grid and I query it with 12960000 points of the regular grid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the number of grid points that is sent to the query &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;get_all_in_range &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;can be reduced substantially as many of the grid points  are not in the range of the sparse matrix coordinates. The question is how  does one do an &quot;intersection&quot; of the grid points and the coarse matrix coordinates using a mathematical approach ? I have seen the answer to this question - &lt;a href=&quot;https://scicomp.stackexchange.com/questions/29135/efficiently-finding-all-x-y-z-points-within-certain-distance-of-point-p&quot;&gt;Using Morton keys&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Geohash&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geohashes&lt;/a&gt; but I am not sure whether this would be applicable to my problem. At least I do not understand how to apply it to my problem. Similarly can a &lt;a href=&quot;https://stackoverflow.com/questions/9678624/convex-hull-of-longitude-latitude-points-on-the-surface-of-a-sphere&quot;&gt;geodesic convex hull&lt;/a&gt; be used to determine the intersection ? &lt;/p&gt;&#xA;" OwnerDisplayName="user16555" LastEditorDisplayName="user16555" LastEditDate="2018-08-29T05:05:53.627" LastActivityDate="2018-08-29T05:05:53.627" Title="Use of Morton Key to reduce number of grid points" Tags="|algorithms|python|computational-geometry|nearest-neighbors|spatial-data|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30105" PostTypeId="2" ParentId="28723" CreationDate="2018-08-29T06:29:14.117" Score="0" Body="&lt;p&gt;You consider a face parallel to the y-axis and z-axis, it's normal would be in the x-axis.&#xA;I don't think you can draw a face parallel to the y and z-axises simultaneously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gradient is a vector, it has three components in x y and z directions. You need to compute the gradient-x, gradient-y, and gradient-z, respectively in these 3 axes.&lt;/p&gt;&#xA;" OwnerUserId="22404" LastActivityDate="2018-08-29T06:29:14.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30107" PostTypeId="1" CreationDate="2018-08-29T08:53:38.850" Score="4" ViewCount="476" Body="&lt;p&gt;&lt;a href=&quot;https://su2code.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SU2&lt;/a&gt; is an open-source CFD suite that is built around a RANS-solver. The main PDE that is solved, is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\partial}{\partial t} \mathbf{U} + \nabla \cdot \mathbf{F^c} - \nabla \cdot \mathbf{F^v} = \mathbf{Q} \qquad \mathrm{in}\ \Omega, t &amp;gt; 0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this equation, $\mathbf{U}$ is the state vector and $\mathbf{F^c}$ and $\mathbf{F^v}$ represent the convective and viscous fluxes, respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://arc.aiaa.org/doi/10.2514/6.2013-287&quot; rel=&quot;nofollow noreferrer&quot;&gt;Palacios et al. (2013)&lt;/a&gt; state the following (p. 13):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Naturally, both the laminar Navier-Stokes and Euler equations are also available in the code as subsets of the RANS equations by disabling turbulence modeling and by completely removing viscosity, respectively.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This, however, is not so 'natural' to me. My thoughts/ideas so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Turbulent RANS to laminar NS&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the RANS-case, part of $\mathbf{F^v}$ is the term $v_j \tau_{ij} + \mu^*_{tot} C_p \frac{\partial}{\partial i} T$ (part of the energy equation). As $\mu_{tot}$ (as well as $\mu^*_{tot}$) is a combination of $\mu_{dyn}$ (dynamic viscosity) and $\mu_{turb}$ (turbulent viscosity). Given the quote on 'disabling turbulence modelling', my guess would be that that would mean setting $\mu_{turb} = 0$ (such that $\mu_{tot} = \mu_{dyn}$). Is that indeed correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Turbulent RANS to inviscid Euler&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get that setting $\mathbf{F^v} = 0$, the inviscid Euler equations are obtained from the main PDE above. However, taking a more theoretical approach, I would think that the process of averaging flow variables (into mean and time-fluctuating parts) that one uses to obtain RANS from NS, somehow interferes with that. Put differently: if we can go from RANS to Euler, that would imply that all viscous effects are contained in these time-fluctuating parts. On one hand that makes sense, on the other hand, I'm not so sure.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Follow-up question: does this hold for RANS formulations in general, or only the specific implementation used in SU2?&lt;/p&gt;&#xA;" OwnerUserId="28592" LastActivityDate="2018-08-29T10:01:12.057" Title="How to go from turbulent RANS to laminar Navier-Stokes and Euler" Tags="|fluid-dynamics|navier-stokes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30108" PostTypeId="2" ParentId="30107" CreationDate="2018-08-29T09:35:31.040" Score="2" Body="&lt;p&gt;The only difference between RANS and Navier Stokes equations is the Reynolds' stress tensor: $$\tau_R = -\overline{v'\otimes v'}$$&#xA;This term is modeled using a turbulent viscosity and therefore:&#xA;$$\tau_{R}\propto \mu_t$$&#xA;In the limit when $\mu_t\to 0$  the RANS equations converge to NS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Viscous effects are not within any part: nor fluctuating neither averaged. Viscous effects are owed by the nature of the fluid itself. They are relevant when the Reynolds number is high and negligible when low. In turbulence these are always relevant (dissipative effects).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this, it is also natural to set $\mu\to 0$ to obtain the Euler equations from the NS. Note that the process would be RANS$\to$ NS $\to$ Euler. But the natural process would have been: NS $\to$ RANS or NS$\to$ Euler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please ask whatever is not clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The regime of the fluid (Laminar-Transition-Turbulent) depends on the Reynolds number. I mean the term of study is:&#xA;$$\frac{1}{Re}\triangle \vec{v}$$&#xA;This term is always dissipative (it takes kinetik energy from the fluid and transforms it into heat) makes the fluid to turn, creating boundary layers and recirculation bubbles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The higher the reynolds the lower the scale of energy dissipation (more or less the size of the recirculation bubble or eddy. This length scale has a minimum threshold in which the all kinetic energy is dissipated into heat, called Kolmogorov length scale (check it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact in which $\mu_t\to 0$ yields to nothing. We agreed with that in this case we recovered the full Navier Stokes equations in which the effect of turbulence is implicitly defined and it depends on the value of the Reynolds number.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="22590" LastEditDate="2018-08-29T10:01:12.057" LastActivityDate="2018-08-29T10:01:12.057" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30109" PostTypeId="2" ParentId="30099" CreationDate="2018-08-29T17:16:20.307" Score="2" Body="&lt;p&gt;There are a number of ways to check if a code is correct.  The best way in this instance is to perform a convergence check on your code.  This requires that you perform the same computation with smaller and smaller values of $\Delta x$ and $\Delta t$ and see if the expected rates of convergence are demonstrated.  The usual way to go about this for PDEs is to establish a relationship between them, such as the Courant number.  I have some course notes at &lt;a href=&quot;https://github.com/mandli/numerical-methods-pdes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mandli/numerical-methods-pdes&lt;/a&gt; that might be useful, in particular 7-IVP and 10-Hyperbolic, which contain some Python code snippets and describes issues with Leapfrog.&lt;/p&gt;&#xA;" OwnerUserId="567" LastActivityDate="2018-08-29T17:16:20.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30113" PostTypeId="1" AcceptedAnswerId="31430" CreationDate="2018-08-30T14:23:27.573" Score="2" ViewCount="283" Body="&lt;p&gt;I am comparing the run-time of two algorithm by solving different instance of the problems. Sample of my data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; problem_id  |  algo_original_time  |  algo_improved_time&#xA;---------------------------------------------------------&#xA; prob_01     |  0.56                |  0.036&#xA; prob_02     |  0.26                |  0.005&#xA; prob_03     |  0.75                |  0.055&#xA; ....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that, all of the problem instances are distinct and they do not have any correlation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already used cross-validation plot to represent the data. Is there any better way/ statistical method to compare the run-time of this two algorithms.  &lt;/p&gt;&#xA;" OwnerUserId="20543" LastActivityDate="2019-09-13T20:41:07.563" Title="Which statistical method should I use for comparing machine run-time of two algorithms?" Tags="|statistics|visualization|data-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30114" PostTypeId="1" CreationDate="2018-08-30T14:56:14.110" Score="4" ViewCount="511" Body="&lt;p&gt;I am trying to visualize a higher dimensional vector field. Is there a way to do this. I asked this question &lt;a href=&quot;https://mathematica.stackexchange.com/questions/180866/plotting-or-visualizing-a-higher-dimensional-vector-field&quot;&gt;here&lt;/a&gt;, I was told to post it here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, one can use&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{eqnarray}&#xA;   \dot{x}&amp;amp;=&amp;amp; \left(J-R\right)x&#xA;   \end{eqnarray}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $x\in \mathbb{R}^5$ $J $ is a skew-symmetric matrix and $R$ is a positive semi-definite matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: The only idea I came so far is: somehow I have to use &lt;a href=&quot;https://en.wikipedia.org/wiki/T-distributed_stochastic_neighbor_embedding&quot; rel=&quot;nofollow noreferrer&quot;&gt;t-sne&lt;/a&gt;, but I really don't know how.&lt;/p&gt;&#xA;" OwnerUserId="28611" LastActivityDate="2018-08-30T17:57:37.023" Title="Plotting or Visualizing a Higher dimensional vector field" Tags="|fluid-dynamics|visualization|vector|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30115" PostTypeId="1" CreationDate="2018-08-30T16:25:03.140" Score="3" ViewCount="44" Body="&lt;p&gt;This is a continuation of the question asked &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10303/factorization-for-reweighted-least-squares&quot;&gt;here&lt;/a&gt;. I want to solve numerous least squares systems of the form&#xA;$$&#xA;D_i A x \approx D_i b&#xA;$$&#xA;where $D_i$ are $m \times m$ diagonal matrices with positive elements on the diagonal, $A$ is $m \times n$ with $m &amp;gt; n$, $x$ is $n \times 1$, and $b$ is $m \times 1$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer given in the previous question recommended to factor $A$ with an $LQ$ decomposition, noting that $D_i L$ would be lower triangular so that it wouldn't be necessary to factor $D_i A$ for each $D_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it turns out that $L$ in an $LQ$ factorization is only lower triangular when $m \le n$ (under-determined). When $m &amp;gt; n$ (over-determined), $L$ is in fact lower trapezoidal (zero above the $m=n$ diagonal, see the LAPACK routine &lt;a href=&quot;http://www.netlib.org/lapack/explore-3.1.1-html/dgelqf.f.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DGELQF&lt;/a&gt;). So $L$ would look like&#xA;$$&#xA;L = \pmatrix{ L_{11} \\ L_{21}}&#xA;$$&#xA;where $L_{11}$ is $n \times n$ lower triangular, and $L_{21}$ is $(m-n) \times n$ dense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, when using an $LQ$ factorization, we would need to solve&#xA;$$&#xA;D_i L z \approx D_i b&#xA;$$&#xA;where $z = Q x$. Now multiplying a diagonal matrix by a trapezoidal gives a trapezoidal matrix, so the problem becomes solving the trapezoidal least squares problem&#xA;$$&#xA;(D_i L) z = y&#xA;$$&#xA;&lt;strong&gt;My question is how can this trapezoidal system be solved efficiently?&lt;/strong&gt; It seems to me I'm back to square 1 because I would need to factor $D_i L$ with $QR$ or $SVD$ in order to solve this least squares system. I don't see any LAPACK/BLAS routines which can efficiently solve trapezoidal systems.&lt;/p&gt;&#xA;" OwnerUserId="28440" LastActivityDate="2018-08-30T16:25:03.140" Title="Reweighted least squares factorization" Tags="|matrix|least-squares|matrix-factorization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30116" PostTypeId="2" ParentId="30114" CreationDate="2018-08-30T17:26:31.440" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Somehow I have to use t-sne, but I really don't know how.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Since you have a PhD my answer will be brief, it's quite a lengthy subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aim of dimensionality reduction is to preserve as much of the significant structure of the high-dimensional data as possible in the low-dimensional map. For high-dimensional data that lies on or near a low-dimensional, non-linear manifold it is usually more important to keep the low-dimensional representations of very similar datapoints close&#xA;together, which is typically not possible with a linear mapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/T-distributed_stochastic_neighbor_embedding&quot; rel=&quot;nofollow noreferrer&quot;&gt;T-distributed Stochastic Neighbor Embedding&lt;/a&gt; (t-SNE) is a way of converting a high-dimensional data set into a matrix of pair-wise similarities. t-SNE is capable of capturing much of the local structure of the high-dimensional data very well, while also revealing global structure such as the presence of clusters at several scales.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Stochastic Neighbor Embedding (SNE) starts by converting the high-dimensional Euclidean distances between datapoints into conditional probabilities that represent similarities. The similarity of datapoint $x_j$ to datapoint $x_i$ is the conditional probability, $p_{j|i}$, that $x_i$ would pick $x_j$ as its neighbor if neighbors were picked in proportion to their probability density under a Gaussian centered at $x_i$.&#xA;For nearby datapoints, $p_{j|i}$ is relatively high, whereas for widely separated datapoints, $p_{j|i}$ will be almost infinitesimal (for reasonable values of the variance of the Gaussian, $σ_i$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cost function used by t-SNE differs from the one used by SNE in two ways: (1) it uses a symmetrized version of the SNE cost function with simpler gradients that was briefly introduced by Cook et al. (2007) and (2) it uses a Student-t distribution rather than a Gaussian to compute the similarity between two points in the low-dimensional space. t-SNE employs a heavy-tailed distribution in the low-dimensional space to alleviate both the crowding problem and the optimization problems of SNE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few t-SNE tutorials:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Laurens van der Maaten's (author of the original paper) &lt;a href=&quot;https://lvdmaaten.github.io/tsne/&quot; rel=&quot;nofollow noreferrer&quot;&gt;webpage&lt;/a&gt; has implications in: Matlab, CUDA, Python, Torch, Julia, Java, R, JavaScript and more ...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Distill's interactive t-SNE dimensionality reduction webpage: &lt;a href=&quot;https://distill.pub/2016/misread-tsne/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;How to Use t-SNE Effectively&quot;&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;GoogleTechTalks Video - &quot;&lt;a href=&quot;https://youtu.be/RJVL80Gg3lA?t=10m7s&quot; rel=&quot;nofollow noreferrer&quot;&gt;Visualizing Data Using t-SNE&lt;/a&gt;&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/123040/whats-wrong-with-t-sne-vs-pca-for-dimensional-reduction-using-r?noredirect=1&amp;amp;lq=1&quot;&gt;CrossValidated.SE&lt;/a&gt; is probably a better site to ask about statistics visualization.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28346" LastEditorUserId="28346" LastEditDate="2018-08-30T17:57:37.023" LastActivityDate="2018-08-30T17:57:37.023" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30117" PostTypeId="1" CreationDate="2018-08-30T21:49:17.550" Score="0" ViewCount="56" Body="&lt;p&gt;While playing around I managed to design a Cellular Automaton (CA) that behaves like a fluid. Very basic behavior, like diffusion, obstacle avoidance, pressure etc. And that got me wondering about it. Designing by hand doesn't seem like the way to go; is there any work that tried to explore the rule space using other methods? Like, combining different rules in different blocks, or maybe use machine learning to tweak the parameters of a set of rules while comparing with a traditional simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple of disclaimers: I noticed there are different types of cellular automata, and they can be vastly different from each other. For sake of generality, consider this question without restricting to any particular type of CA. I also understand that many CA rules are Turing complete, meaning they you could model anything that can run on a Turing computer, in principle. So, for the sake of this question, consider that we want the dynamics to show up as directly as possible.&lt;/p&gt;&#xA;" OwnerUserId="28617" LastEditorUserId="9667" LastEditDate="2018-08-30T21:53:09.330" LastActivityDate="2018-09-30T23:50:16.187" Title="Are there any studies on finding cellular automata rules for modeling specific systems?" Tags="|modeling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30118" PostTypeId="1" AcceptedAnswerId="30119" CreationDate="2018-08-30T22:51:49.890" Score="0" ViewCount="166" Body="&lt;p&gt;I have research work where I need to compute a matrix inversion. The matrix has a size $31300\times31300$. I am using a universal java matrix package to invert this matrix. But as the dimension of the matrix is huge, it throws error memory out of scope. Cpu cannot allocate huge memory space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to find out a solution I came to know about GPU(Graphical Processing Unit). But in every search about GPU, it is described that GPU is used to speed up a process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my doubt is that the computation that failed to compute by CPU, is it possible to be computed by GPU? Or GPU only used for those processes which take of time to compute in CPU, they can compute in GPU for speedup.&lt;/p&gt;&#xA;" OwnerUserId="28618" LastEditorUserId="20688" LastEditDate="2018-08-30T22:59:51.543" LastActivityDate="2018-08-30T23:07:30.167" Title="Use of GPU with respect to CPU" Tags="|linear-solver|gpu|memory-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30119" PostTypeId="2" ParentId="30118" CreationDate="2018-08-30T23:07:30.167" Score="2" Body="&lt;p&gt;Unfortunately, GPUs will be of no help to you in this particular situation. Your problem is in the memory limitation; thus, you just do not have enough RAM resources to allocate/factorize/solve the system. GPUs usually have a much smaller amount of memory available on them, so they are not used to help with memory-limited problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suggestions to try:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It looks like you are trying to allocate $31300\times 31300$ matrix which implies you are solving a &lt;em&gt;dense&lt;/em&gt; linear algebra problem. Make sure that the matrix you work with is not &lt;em&gt;sparse&lt;/em&gt; (in this case, there are efficient sparse linear algebra solvers).&lt;/li&gt;&#xA;&lt;li&gt;There are out-of-core linear algebra solvers (which are, of course, significantly slower) which will allow you to solve larger problems within your RAM limitations by using your hard drive to store intermediate/final results.&lt;/li&gt;&#xA;&lt;li&gt;Use efficient LAPACK linear algebra, say Intel MKL.&lt;/li&gt;&#xA;&lt;li&gt;Try using a machine with a larger amount of memory (Amazon EC2 has cheap options available).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-08-30T23:07:30.167" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30120" PostTypeId="1" AcceptedAnswerId="30128" CreationDate="2018-08-31T03:23:43.147" Score="3" ViewCount="1081" Body="&lt;p&gt;I try to implement some big matrix inversion. My system configuration is &lt;strong&gt;Hardware:- Memory: 62.8GiB, Processor: Intel Xeon(R)CPU E5-2670 v3 @2.30GHZ*48&lt;/strong&gt; To implement matrix inversion I am using universal Java matrix package. I want to invert a 31000*31000 dimension matrix. It is a Sparse matrix, most of the cell of this matrix is zero. I know that UJMP support sparse and dense matrix. So I am using it. But when I try to invert the matrix it cannot give any output also it cannot throw any error. The process runs for an infinite time without any output. As per my system configuration, there is enough memory space. Then why it did not give any output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, If UJMP not working then can anyone suggest me any other sparse matrix package by which I can invert a sparse matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thirdly, I came to know that LAPACk is a standard software library for numerical linear algebra. I also came to know that BLAS (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations. Are they compatible with Java? I am using &lt;strong&gt;Intellj Idea&lt;/strong&gt; for Java coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last, If LAPACK or BLAS are compatible with Java how could I configure them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One more discussion Is GPU is helpful for my problem. Actually, my CPU has a huge capacity so I never think about GPU. So, as per my belief, it is not a good option to use GPU. Still, for a second opinion, I discuss this topic.  &lt;/p&gt;&#xA;" OwnerUserId="28618" LastEditorUserId="20688" LastEditDate="2018-08-31T06:26:38.207" LastActivityDate="2018-09-02T10:00:25.970" Title="Lost on Matrix Inversion" Tags="|linear-algebra|matrix|linear-solver|gpu|java|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30121" PostTypeId="1" CreationDate="2018-08-31T03:39:59.260" Score="2" ViewCount="7285" Body="&lt;p&gt;When I open U for example, I find:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dimensions      [0 1 -1 0 0 0 0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That means m/s&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the other dimensions?&lt;/p&gt;&#xA;" OwnerUserId="28619" LastActivityDate="2018-08-31T04:48:41.787" Title="How are dimension units on openFoam?" Tags="|openfoam|" AnswerCount="1" CommentCount="6" ClosedDate="2018-09-02T18:50:52.490" ContentLicense="CC BY-SA 4.0" />
  <row Id="30122" PostTypeId="2" ParentId="30121" CreationDate="2018-08-31T03:39:59.260" Score="4" Body="&lt;p&gt;Units on the brackets follow the figure:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5Djmz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Djmz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Accordingly, [0 1 -1 0 0 0 0] = m/s&lt;/p&gt;&#xA;" OwnerUserId="28619" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-08-31T04:48:41.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30124" PostTypeId="1" CreationDate="2018-08-31T05:00:25.077" Score="-2" ViewCount="2103" Body="&lt;p&gt;For learning purposes, how do I create an scalar variable like temperature divided by 2 and outputs it on files generated?&lt;/p&gt;&#xA;" OwnerUserId="28619" LastEditorUserId="28619" LastEditDate="2018-08-31T05:55:21.110" LastActivityDate="2018-08-31T05:55:21.110" Title="openFoam - How to create an intermediate scalar variable and output it?" Tags="|openfoam|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30125" PostTypeId="2" ParentId="30124" CreationDate="2018-08-31T05:39:28.217" Score="0" Body="&lt;p&gt;Supposing that your variable is totally dependent of T, so you don't need to read it on &quot;0&quot; file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On createFields.h add:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Info&amp;lt;&amp;lt; &quot;Creating T2\n&quot; &amp;lt;&amp;lt; endl;&#xA;volScalarField T2&#xA;(&#xA;    IOobject&#xA;    (&#xA;        &quot;T2&quot;,&#xA;        runTime.timeName(),&#xA;        mesh,&#xA;        IOobject::NO_READ,&#xA;        IOobject::AUTO_WRITE&#xA;    ),&#xA;    mesh,dimensionedScalar(&quot;T2&quot;, dimensionSet(0,0,0,1,0,0,0),0.0 )&#xA;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;0.0 inside dimensioned scalar just initializes the variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you should add the equation to your runTimeLoop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T2=T/2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28619" LastEditorUserId="28619" LastEditDate="2018-08-31T05:54:38.530" LastActivityDate="2018-08-31T05:54:38.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30127" PostTypeId="2" ParentId="30120" CreationDate="2018-08-31T06:44:37.907" Score="8" Body="&lt;p&gt;Several points I want to mention (with an encouragement to other CompSci users that are more familiar with Java specifics to give additional, more Java related answers):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The solution of a system of linear equations and inversion of the matrix are two very different things. You &lt;strong&gt;almost never should explicitly invert the matrix&lt;/strong&gt;. One should use one form of the &lt;em&gt;direct&lt;/em&gt; or &lt;em&gt;iterative&lt;/em&gt; methods to solve the system. Since your matrix is sparse, you should look for the one that takes sparsity into account.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://ujmp.org/&quot; rel=&quot;noreferrer&quot;&gt;UJMP&lt;/a&gt; does not seem like an awful choice (unfortunately, I personally never used Java for numerical computing). However, in the brief documentation there I was not able to find a sparse factorization or iterative-based method. For sure, if you construct your matrix as sparse and then use &lt;code&gt;dense.inv();&lt;/code&gt; on it, you should not expect to get a solution in any reasonable time. So, I certainly suggest to spend some time in the UJMP documentation and understand how you are supposed to do a &lt;strong&gt;solution&lt;/strong&gt; to a sparse linear system with it.&lt;/li&gt;&#xA;&lt;li&gt;You can use &lt;a href=&quot;http://www.netlib.org/blas/faq.html#_6_a_id_where_can_i_find_java_blas_a_where_can_i_find_java_blas&quot; rel=&quot;noreferrer&quot;&gt;LAPACK and BLAS under Java&lt;/a&gt; and &lt;a href=&quot;http://math.nist.gov/javanumerics/&quot; rel=&quot;noreferrer&quot;&gt;JavaNumerics&lt;/a&gt; should give plenty of information on how to use it and set it up.&lt;/li&gt;&#xA;&lt;li&gt;Though direct solvers for sparse linear systems exist and can be efficient, in your case you might look into iterative solvers that would use multiple matrix-vector products (easy for sparse matrices) to converge to a solution.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-08-31T06:44:37.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30128" PostTypeId="2" ParentId="30120" CreationDate="2018-08-31T15:09:39.627" Score="7" Body="&lt;p&gt;Normally when you invert a sparse matrix the inverse is dense.&#xA;This imply to have enough memory to store the inverse, in your case the matrix is not so big for nowdays computers. In double precision (1 cell = 8 byte) you have&#xA;$$&#xA;31000 \times 31000 \times 8 \text{ byte } =  61504000000 \text{ byte }&#xA;\approx 7.7 \text{ Gygabyte }&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another problem is the time invert a matrix, because it is a very heavy task.&#xA;For example &lt;a href=&quot;https://it.mathworks.com/help/matlab/ref/inv.html&quot; rel=&quot;noreferrer&quot;&gt;Matlab inv&lt;/a&gt; describe the function so:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;inv performs an LU decomposition of the input matrix (or an LDL decomposition if the input matrix is Hermitian). It then uses the results to form a linear system whose solution is the matrix inverse inv(X). For sparse inputs, inv(X) creates a sparse identity matrix and uses backslash, X\speye(size(X)).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.inv.html&quot; rel=&quot;noreferrer&quot;&gt;Scipy linalg.inv&lt;/a&gt; recall the getry function of &lt;a href=&quot;http://www.netlib.org/lapack/double/&quot; rel=&quot;noreferrer&quot;&gt;Lapack&lt;/a&gt; where:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Purpose &#xA;    =======&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;DGETRI computes the inverse of a matrix using the LU factorization&#xA;   computed by DGETRF.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This method inverts U and then computes inv(A) by solving the system&#xA;   inv(A)*L = inv(U) for inv(A).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These for show the idea under the inversion functions, for the library UJMP I found problem to acces to the documentation. Note that solve the linaer system related to the action of the inverse is cheaper than calculate the inverse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if we wait the time for the inversion we will obtain, &lt;em&gt;in general&lt;/em&gt;, a result not so good from to use in numerical calculation. For detail see &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26423/practical-example-of-why-it-is-not-good-to-invert-a-matrix&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For these reason you must valuate what are your goal with the inverse, and if you really need the explicit inverse of the application of the inverse.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;BLAS &amp;amp; LAPACK&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case you need &lt;strong&gt;sparse&lt;/strong&gt; BLAS or LAPACK. For an example, not in java, see this &lt;a href=&quot;https://math.nist.gov/spblas/&quot; rel=&quot;noreferrer&quot;&gt;nist link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_numerical_libraries#Java&quot; rel=&quot;noreferrer&quot;&gt;this wikipedia page&lt;/a&gt; there are some java libraries, I cite some (note I do not use them):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Matrix_Toolkit_Java&quot; rel=&quot;noreferrer&quot;&gt;Matrix Toolkit Java&lt;/a&gt; this the &lt;a href=&quot;https://github.com/hwinkler/matrix-toolkits-java&quot; rel=&quot;noreferrer&quot;&gt;gitHub page&lt;/a&gt; note the follow &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;MTJ supports sparse matrix storage but does not provide solvers for sparse matrices. Have a look at Sparse Eigensolvers for Java or consider implementing your own and letting us know about it (e.g. by using the ARPACK backend which comes with netlib-java).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Parallel_Colt&quot; rel=&quot;noreferrer&quot;&gt;Parallel Colt&lt;/a&gt; interesting project but some outdate&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://commons.apache.org/proper/commons-math/&quot; rel=&quot;noreferrer&quot;&gt;Commons Apache&lt;/a&gt; in the java word Common Apache are a security of durable project. There is an &lt;a href=&quot;http://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/index.html&quot; rel=&quot;noreferrer&quot;&gt;interface for sparse matrix&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://ejml.org/wiki/index.php?title=Main_Page&quot; rel=&quot;noreferrer&quot;&gt;Efficient Java Matrix Library&lt;/a&gt; for dense case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://jblas.org/&quot; rel=&quot;noreferrer&quot;&gt;jblas&lt;/a&gt; dense blas&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;GPU&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before use gpu I suggest to understand if you need the explicit inverse&#xA;However gpu not always give you speed up, but is depends by your specif problem. Without information, i.e. in general, you can think to use the gpu for the linear algebra (for example iterative solver). Normally gpu is bounded by memory transfer and  it has got less ram of cpu.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt; Memory Test&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I replicate your test of creation of a matrix in octave. My pc has go 16 GB of ram plus 32 GB of hard disk space dedicate to ram. I use Octave 4.0 from Ubuntu 16.04 repository.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I launch the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A=zeros(31000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The ram used before the command 1.4 GB and at the end 8.6 GB, the measures come from  monitor of system. So we can consider in line with prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the middle what happened?&#xA;In the middle there are one or two minutes of work (to be honest I did not use tic toc functions) by Octave. The memory usage varies during this time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First the memory usage went to 9 GB quickly at as the same velocity it went to over the 16 GB of RAM and started to use the hard disc ram.&#xA;After a while the memory ram has decreased and started the moving from hard disc memory to normal ram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end the memory value was correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that Octave memory error message is like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thing that you do not see (according with the comment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure at 100% but knowing Octave I have an idea. Octave, similar Matlab, has got a memory management &lt;a href=&quot;https://octave.org/doc/v4.2.1/Call-by-Value.html&quot; rel=&quot;noreferrer&quot;&gt;by value&lt;/a&gt; this is a general approach that it uses. &lt;a href=&quot;https://octave.org/doc/v4.2.2/Miscellaneous-Techniques.html&quot; rel=&quot;noreferrer&quot;&gt;In documentation&lt;/a&gt; are explained some techniques that it use to improve performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ram behavior can be compatible with this explanation: during the call of function zeros the matrix of $31000 \times 31000$ elements are created a first time (inside the normal ram). Copied, for some motivation, to the variable A, saved some in normal ram and the surplus in hard disk ram. After the first  matrix (the first copy) is deleted and start the migration from the hard disk ram to normal ram, maybe with some cache techniques.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion you did not found a memory outbound (no messagge) but you were waiting the memory transfers. Obviously you have got a lot of ram memory, but How many free? (I think you need a check about this, remember that in my pc with less memory it runs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Different the case for UJMP, according with your question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But when I try to invert the matrix it cannot give any output also it cannot throw any error. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you arrived to this point the matrix was already created and, I think, you are wait for the inversion task, that is very very heavy (normally never use in numerical analysis).&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="22025" LastEditDate="2018-09-02T10:00:25.970" LastActivityDate="2018-09-02T10:00:25.970" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30129" PostTypeId="2" ParentId="30117" CreationDate="2018-08-31T21:18:43.973" Score="2" Body="&lt;p&gt;You should take a look at the book (or, really, &quot;tome&quot;) called &quot;A new kind of science&quot; by Steven Wolfram. It is about many questions like the one you state.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, more concretely, think of starting with, say, the Navier-Stokes equations and then discretizing it in space and time on a uniform mesh. So $U(i,j,k)$ represents, for example, the state of the fluid at location $(i,j)$ on the grid and at the $k$th time step. If you had used an explicit time stepping scheme, then this state clearly depends on the state of the fluid in the previous time step. If, furthermore, you use a simple nearest neighbor finite difference stencil for the spatial derivatives, then $U(i,j,k)$ also only depends on the states $U(i\pm 1, j\pm 1, k-1)$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, voila, you've just got yourself that looks a lot like a cellular automaton!&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-08-31T21:18:43.973" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30130" PostTypeId="1" CreationDate="2018-08-31T21:52:47.503" Score="2" ViewCount="229" Body="&lt;p&gt;In &lt;a href=&quot;https://www.doi.org/10.1007/978-1-4419-8853-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yurii Nesterov's Introductory Lectures on Convex Optimization&lt;/a&gt;, there is a description of the rate of convergence and corresponding upper bound for the analytical complexity of a minimization problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\min f(x),\quad x \in \mathbb R^n$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error at iteration $k$ is denoted by $r_k=||x_k-x^*||$ where $x_k$ is the approximate solution at iteration $k$ and $x^*$ is the true solution. Quadratic rate: $r_{k+1} \leq c r_k^2$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding complexity estimate depends on a double logorithm of the desired accuracy: $\ln\ln\frac{1}{\epsilon}$. Why is this the case?&lt;/p&gt;&#xA;" OwnerUserId="27664" LastEditorUserId="20688" LastEditDate="2018-09-01T00:26:56.527" LastActivityDate="2018-09-01T03:48:07.827" Title="Convergence rate and complexity for convex minimization problem" Tags="|optimization|convex-optimization|complexity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30132" PostTypeId="2" ParentId="30130" CreationDate="2018-09-01T03:48:07.827" Score="3" Body="&lt;p&gt;Unlike the first two cases on page 36, case 3, &quot;Quadratic rate&quot;, has a bound on $r_{k+1}$ that depends on $r_{k}$ rather than $k$.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_{k+1} \le cr_{k}^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_{k+1} \le c(cr_{k-1}^{2})^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_{k+1} \le c(c(cr_{k-2}^{2})^{2})^{2}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\vdots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$r_{k+1} \le c^{2^{k}-1} r_{0}^{2^{k+1}}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfolding this, you can see that $O(\log \log 1/\epsilon)$ iterations are needed to get $r$ down to less than $\epsilon$.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-09-01T03:48:07.827" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30133" PostTypeId="1" AcceptedAnswerId="30139" CreationDate="2018-09-01T10:12:11.280" Score="1" ViewCount="91" Body="&lt;p&gt;I am trying to implement the Thomas algorithm using OpenCL.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;__kernel void thomas(__global float *a_d, __global float *b_d, __global float *c_d, __global float *d_d, __global float *x_d, &#xA;                                     __local float *shared, int system_size, int num_systems, int iterations)&#xA;{&#xA;    int thid = get_local_id(0);&#xA;    int blid = get_group_id(0);&#xA;&#xA;    int delta = 1;&#xA;&#xA;    __local float* a = shared;&#xA;    __local float* b = &amp;amp;a[system_size+1];&#xA;    __local float* c = &amp;amp;b[system_size+1];&#xA;    __local float* d = &amp;amp;c[system_size+1];&#xA;    __local float* cs = &amp;amp;d[system_size+1];&#xA;    __local float* ds = &amp;amp;cs[system_size+1];&#xA;&#xA;    a[thid] = a_d[thid + blid * system_size];&#xA;    b[thid] = b_d[thid + blid * system_size];&#xA;    c[thid] = c_d[thid + blid * system_size];&#xA;    d[thid] = d_d[thid + blid * system_size];&#xA;&#xA;    float tmp;&#xA;&#xA;    barrier(CLK_LOCAL_MEM_FENCE);&#xA;&#xA;    cs[0] = c[0] / b[0];&#xA;    ds[0] = d[0] / b[0];&#xA;&#xA;&#xA;    // Forward &quot;warp&quot;&#xA;    barrier(CLK_LOCAL_MEM_FENCE);&#xA;&#xA;    if(thid&amp;gt;0)&#xA;    {&#xA;        tmp = (b[thid] - a[thid] * c[thid - 1]);&#xA;        cs[thid] = c[thid] / tmp;&#xA;        ds[thid] = (d[thid] - d[thid - 1] * a[thid]) / tmp;&#xA;    }&#xA;    barrier(CLK_LOCAL_MEM_FENCE);&#xA;    if(thid &amp;lt; system_size)&#xA;    {&#xA;        d[thid] = ds[thid] - cs[thid]*d[thid + 1];&#xA;    }&#xA;    barrier(CLK_LOCAL_MEM_FENCE);&#xA;&#xA;    x_d[thid + blid * system_size] = d[thid];&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is giving me an incorrect result. The host code is working perfectly fine. It has been used on several kernels that I have been experimenting with. I the problem is arising because the threads are not executed in any order (?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone got some advice?&lt;/p&gt;&#xA;" OwnerUserId="27791" LastEditorUserId="27791" LastEditDate="2018-09-01T10:27:13.313" LastActivityDate="2018-09-02T18:21:52.793" Title="Thomas Algorithm Kernel OpenCL" Tags="|linear-algebra|matrix|opencl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30134" PostTypeId="1" CreationDate="2018-09-01T16:38:39.060" Score="7" ViewCount="841" Body="&lt;p&gt;I need to put $N$ spheres with given radius $R$ randomly in a Volume $[-0.5,0.5]^3$, without any overlap of spheres.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I choose values so that all the spheres will occupy ~57% of the total volume, I find it difficult to get results.&#xA;The basic scheme to place the $i$-th particle after $i-1$ particles have been placed:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;choose random position&lt;/li&gt;&#xA;&lt;li&gt;check for overlap with any other sphere&lt;/li&gt;&#xA;&lt;li&gt;if(overlap): check the new randomly chosen position&lt;/li&gt;&#xA;&lt;li&gt;repeat until $i$ has been placed&lt;/li&gt;&#xA;&lt;li&gt;place next particle&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, the more particles have been placed, the less likely my random generator will find a position with enough space AND the more time the overlap check needs. For $N=500$, $R=0.5/7.7$, the algorithm is barely able to place 340 particles (overlap is called billions of times at this point).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++ code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using namespace std;&#xA;&#xA;//this function places N hard spheres of radius R on random coordinates in a [-0.5,0.5]^3 box&#xA;&#xA;vector &amp;lt;particle&amp;gt; random_positions_hard_spheres (int N, double R){&#xA;vector &amp;lt;particle&amp;gt; positions;&#xA;&#xA;static seed_seq seed_sequence { 100, 78639, 193, 55555, 3, 2348089, 6474367, 264441578 };&#xA;static mt19937 gen (seed_sequence);&#xA;uniform_real_distribution&amp;lt;double&amp;gt; random_double(-0.5, 0.5);&#xA;double x,y,z;&#xA;double d=2*R;&#xA;&#xA;for(size_t i=0; i&amp;lt;N; ++i){&#xA;&#xA;    x=random_double(gen);&#xA;    y=random_double(gen);&#xA;    z=random_double(gen);&#xA;&#xA;    positions.push_back({x,y,z});&#xA;&#xA;    //find position with no overlap&#xA;    int j=0;&#xA;    while(overlap_index(positions, i, d)){&#xA;&#xA;        positions[i]={random_double(gen), random_double(gen), random_double(gen)};&#xA;        ++j;&#xA;    }&#xA;    cout &amp;lt;&amp;lt;&quot;overlap was called &quot;+to_string(j)+&quot; times&quot;&amp;lt;&amp;lt;endl;&#xA;    cout &amp;lt;&amp;lt;&quot;particle &quot;+to_string(i)+&quot; was placed&quot;&amp;lt;&amp;lt;endl;&#xA;&#xA;}&#xA;&#xA;return positions;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the overlap function checks whether particle $i$ overlaps with any previously placed particles.&lt;/p&gt;&#xA;" OwnerUserId="28628" LastEditorUserId="20688" LastEditDate="2018-09-01T16:53:58.390" LastActivityDate="2018-09-01T18:04:25.923" Title="Putting N hard spheres randomly in given volume" Tags="|computational-physics|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30135" PostTypeId="2" ParentId="30134" CreationDate="2018-09-01T18:04:25.923" Score="10" Body="&lt;p&gt;Welcome to Computational Science StackExchange!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are trying to do is fairly difficult, and will need a more sophisticated method. That volume fraction of 0.57 lies just within the &lt;em&gt;crystalline&lt;/em&gt; branch of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hard_spheres&quot; rel=&quot;noreferrer&quot;&gt;hard sphere&lt;/a&gt; phase diagram. The generation of dense random packings is an active area of current research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle, a Monte Carlo simulation, or a collision-by-collision molecular dynamics program, can &quot;equilibrate&quot; an initial configuration prepared by randomly placing spheres which are significantly smaller than your desired size. Then, the idea is to steadily increase the sphere diameter, during the course of the simulation, until the desired volume fraction is reached. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Monte Carlo code would not be too difficult to write, based on what you have already:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Insert the desired number of &lt;em&gt;smaller&lt;/em&gt; spheres into the volume, at positions chosen at random, checking for overlap as you do at present.&lt;/li&gt;&#xA;&lt;li&gt;Conduct Monte Carlo: loop over all $N$ spheres sequentially. Attempt to move each sphere by a small amount in a random direction. Test for overlap with the other $N-1$ spheres. If there is any overlap, reject the move and leave the sphere in its original place.&lt;/li&gt;&#xA;&lt;li&gt;At intervals, attempt to increase the diameter of all the spheres by a very small amount, checking all $\frac{1}{2}N(N-1)$ overlaps. If there are any overlaps, keep the diameter unchanged and continue.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The alternative, of steadily increasing the diameter in the course of a molecular dynamics program, is one of the standard computational approaches to &quot;random packing&quot;. Due to &lt;a href=&quot;https://en.wikipedia.org/wiki/Lubachevsky%E2%80%93Stillinger_algorithm&quot; rel=&quot;noreferrer&quot;&gt;Lubachevsky and Stillinger&lt;/a&gt;, an example of its use can be found &lt;a href=&quot;https://arxiv.org/abs/cond-mat/0003416&quot; rel=&quot;noreferrer&quot;&gt;on arXiv&lt;/a&gt; (also published in &lt;a href=&quot;https://doi.org/10.1103/PhysRevLett.84.2064&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Phys Rev Lett,&lt;/em&gt; &lt;strong&gt;84&lt;/strong&gt; 2064 (2000)&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem, of course, is that in principle, at that volume fraction, the system should be stable in the crystalline state, not in a randomly packed state. In practice, this may not be a problem: the system is likely to get &quot;jammed&quot; in a randomly packed state, at a volume fraction that depends on the compression rate, as discussed in the &lt;em&gt;Phys Rev Lett&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MD programs for hard spheres can be found online, for example &lt;a href=&quot;http://dynamomd.org/&quot; rel=&quot;noreferrer&quot;&gt;DynamO&lt;/a&gt;, or you can find guidance on how to write one in various books and papers. There are some standard tricks to improve the program speed, basically by making lists of near neighbours of the spheres: these would be worthwhile, but would not give huge speedups, for a system size of order 500.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-09-01T18:04:25.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30136" PostTypeId="1" AcceptedAnswerId="30141" CreationDate="2018-09-02T06:45:49.763" Score="2" ViewCount="255" Body="&lt;h2&gt;Analytical&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Inside the box, the wavefunction is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;	\frac{\hbar^2}{2m} \frac{d^2 \psi(x)}{dx^2} = E \psi(x) \iff&#xA;	\frac{d^2 \psi(x)}{dx^2}  = k^2 \psi(x)&#xA;\end{equation}&#xA;where $k = \frac{\sqrt{2mE}}{\hbar}$. Outside the box:&#xA;\begin{equation}&#xA;	-\frac{\hbar^2}{m} \frac{\partial^2 \psi(x)}{\partial x^2} = (E - V_0) \psi(x) \iff \frac{\partial^2 \psi(x)}{\partial x^2} = \alpha^2 \psi(x)&#xA;\end{equation}&#xA;where $\alpha = \frac{\sqrt{2m(V_0 - E)}}{\hbar}$. The solutions for the Schrödinger equation for a particle in a finite potential well are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;	\psi(x) = &#xA;	\begin{cases}&#xA;		Ce^{\alpha x}, \text{if } x &amp;lt; -\frac{L}{2}\\&#xA;		A\sin(kx) + B\cos(kx), \text{if } -\frac{L}{2} \leq x \leq \frac{L}{2}\\&#xA;		Fe^{-\alpha x} , \text{if } x &amp;gt; \frac{L}{2} &#xA;	\end{cases}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, $\psi(x)$ must be continuous and differentiable, which means that the value of the functions and their derivatives must match at the boundaries, $-\frac{L}{2}$ and $\frac{L}{2}$. Depending on whether the solution is symmetric, $\psi(-x) = \psi(x)$, or anti-symmetric, $\psi(-x) = -\psi(x)$, $A = 0$, $C = F$ or $B = 0$. $C = -F$, respectively. For the symmetric case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;	\begin{array}{|l@{}}&#xA;		 Fe^{-\alpha L/2} = B\cos(k L/2)\\&#xA;		 -\alpha F e^{-\alpha L/2} = -\alpha B\cos(k L/2)&#xA;	\end{array}&#xA;\end{equation} &lt;/p&gt;&#xA;&#xA;&lt;p&gt;taking the ratio of the above two to give us $\alpha = k \tan(k L/2)$. Similarly, from the anti-symmetric case we get $\alpha = -k \cot(k L/2)$.Both $\alpha$ and $k$ depend on the energy, $E$, making the above equations transcendental. To solve them we make the following substitutions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;	\begin{array}{|l@{}}&#xA;		u = \frac{L}{2} \alpha \\&#xA;		v = \frac{L}{2} k  = \frac{L}{2}\frac{\sqrt{2mE}}{\hbar}\quad (1)&#xA;	\end{array}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, $u^2 = u_{0}^{2} - v^2$ where $u_0 = \frac{m L^{2} V_0}{2 \hbar^2}$. This leads to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{equation}&#xA;    \sqrt{u_{0}^{2} - v^2} =&#xA;	\begin{cases} &#xA;		 v \tan(v)\\&#xA;		-v \cot(v) &#xA;    \end{cases}&#xA;\end{equation}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the above I use the following Octave code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clc;&#xA;clear;&#xA;close all;&#xA;&#xA;x = 0 :0.1:5;  % coordinates  interval.&#xA;L = 1;         % well width.&#xA;U_0 = 40;      % well depth.&#xA;hbar = 1;      % Planck's constant.&#xA;m = 1;         % particle mass.&#xA;u_0sq = (m*(L^2)*U_0 ) / (2 * hbar^2);&#xA;&#xA;lhs = @(x) sqrt(u_0sq - x.^2);&#xA;rhs_s = @(x) x.*tan(x);&#xA;rhs_a = @(x) -x.*cot(x);&#xA;&#xA;plot(x, lhs(x), 'r', 'LineWidth', 2, ...,&#xA;     x, rhs_s(x), 'b', 'LineWidth', 2, ...,&#xA;     x, rhs_a(x), 'g', 'LineWidth', 2)&#xA;xlabel('v')&#xA;axis([0 5 0 5])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From $(1)$ and the three intesection values&lt;sup&gt;1&lt;/sup&gt; for $v$ from the below plot, the results for the energies are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;3.2724&lt;br&gt;&#xA;  12.8576&lt;br&gt;&#xA;  27.7294&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pYuHC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pYuHC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Numerical&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Now, for the numerical solution I use the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Parameters for solving problem in the interval -L &amp;lt; x &amp;lt; L.&#xA;L = 5;                                               % Interval Length.                                                                                                                                &#xA;N = 1000;                                            % No of points.&#xA;x = linspace(-L, L, N)';                             % Coordinate vector&#xA;dx = x(2) - x(1);                                    % Coordinate step&#xA;&#xA;% Finite square well of width 2w and depth given by U_0.&#xA;U_0 = 40;                           w = L / 10;                                          &#xA;U = -U_0*(heaviside(x + w) - heaviside(x - w));&#xA;&#xA;% Discretized (three-point finite-difference) representation of the Laplacian. &#xA;e = ones(N, 1);                                                      &#xA;Lap = spdiags([e  -2*e e], [-1 0 1], N, N)  &#xA;&#xA;% Total Hamiltonian&#xA;hbar = 1;                                                                  &#xA;m = 1;                                                                     &#xA;H = -1/2*(hbar^2 / m)*Lap + spdiags(U, 0, N, N);  % T = H + U.&#xA;&#xA;% Find (first 3) (smalest algebraic) eigenvalues.&#xA;nmodes = 3;             options.disp = 0;&#xA;[V, E] = eigs(H, nmodes, 'sa' , options);            &#xA;[E, ind] = sort(diag(E));                            &#xA;E&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results for the energies are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;36.72&lt;br&gt;&#xA;  27.14&lt;br&gt;&#xA;  12.27&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm expecting some error but in this case, the energies differ in an order of magnitude.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clearly, I am doing something wrong but I don't know whether the error is in the analytical or numerical solution. What am I doing wrong here? Any hint will be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;1. The vertical lines are discontinuity points.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="24654" LastEditorUserId="9667" LastEditDate="2018-09-04T14:15:01.970" LastActivityDate="2018-09-04T14:15:01.970" Title="Discrepancies between numerical and analytical solution for particle in a finite potential well?" Tags="|eigensystem|quantum-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30137" PostTypeId="1" AcceptedAnswerId="30140" CreationDate="2018-09-02T09:21:05.930" Score="4" ViewCount="239" Body="&lt;p&gt;When a timestep $h$ is rejected using a Runge-Kutta pair, such as Dormand–Prince, the algorithm resumes from the same initial point $t$ with a smaller timestep. A different idea is to resume at an intermediate point $t + kh$ where $1/7 &amp;lt; k &amp;lt; 4/5$, using the 4th order interpolation. The estimated error would be tested at intermediate points until an acceptable point is found. The FSAL property is lost when doing this type of recovery, thus costing an extra function evaluation during the next timestep. My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it expensive to calculate the error at these intermediate points?&lt;/li&gt;&#xA;&lt;li&gt;Will these intermediate points have an acceptable error that makes it worthwhile to do this procedure, or does high error make the interpolation useless throughout the entire interval?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28631" LastActivityDate="2018-09-02T20:38:34.680" Title="Estimating error at mid timesteps for Runge-Kutta methods" Tags="|ode|runge-kutta|numerics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30138" PostTypeId="1" CreationDate="2018-09-02T16:46:58.723" Score="10" ViewCount="167" Body="&lt;p&gt;Existing algorithms for solving ODEs handle functions $\frac{dy}{dt} = f(y, t)$, where $y \in \mathbb R^n$. But in many physical systems, the differential equation is autonomous, so $\frac{dy}{dt} = f(y)$, $y \in \mathbb R^n$, with the $t$ left out. With this simplifying assumption, what improvements can be seen in existing numerical methods? For example, if $n=1$, the problem turns into $t = \int \frac{dy}{f(y)}$ and we turn to a wholly different class of algorithms for integrating one-dimensional integrals. For $n&amp;gt;1$, the maximum possible improvement is reducing the dimension of $y$ by 1, because the time-dependent case can be simulated by appending $t$ to $y$, changing the domain of $y$ from $\mathbb R^n$ to $\mathbb R^{n+1}$.&lt;/p&gt;&#xA;" OwnerUserId="28631" LastActivityDate="2019-03-26T09:44:23.453" Title="Are there shortcuts for numerically approximating systems of ordinary differential equations when autonomous?" Tags="|ode|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30139" PostTypeId="2" ParentId="30133" CreationDate="2018-09-02T18:21:52.793" Score="4" Body="&lt;p&gt;The Thomas algorithm as proposed in most textbooks is inherently serial and anything that just executes the steps concurrently will in general lead to wrong results. You will need a clever modification to parallelize the algorithm which usually comes at the price of at least a logarithmic factor (w.r.t. the number of threads). However, I doubt that you will be able to ever beat the Thomas algorithm if the problem size does not justify parallelization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related discussion: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16359/parallel-vs-serial-thomas-algorithm&quot;&gt;Parallel vs Serial Thomas Algorithm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4260" LastActivityDate="2018-09-02T18:21:52.793" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30140" PostTypeId="2" ParentId="30137" CreationDate="2018-09-02T19:13:08.853" Score="3" Body="&lt;p&gt;Using the interpolation to come up with an error estimate of the continuous (dense) solution is known as residual control. This is done quite frequently when the Runge-Kutta methods are utilized for delay differential equations. There are smarter ways than just taking random points though. You can see some discussion &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0168927404001187&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; where a method for estimating the maximum residual error is done with a few points using the properties of the interpolant.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-09-02T19:13:08.853" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30141" PostTypeId="2" ParentId="30136" CreationDate="2018-09-02T19:42:36.873" Score="4" Body="&lt;p&gt;I think that your potential for the numerical case is wrong. The potential should be a big positive number, so the solution tends to zero outside the well when the value of the potential increases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you check the solutions in &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_potential_well#Finding_wavefunctions_for_the_bound_state&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; for the first three (bounded) states they should be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$E_n = \frac{2\hbar^2 v_n^2}{m L^2}\, ,$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with $v_1 = 1.28$, $v_2 = 2.54$ and $v_3=3.73$, or just&lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{align}&#xA;&amp;amp;E_1 = 3.2768\\&#xA;&amp;amp;E_2 = 12.9032\\&#xA;&amp;amp;E_3 = 27.8258&#xA;\end{align}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is your code with corrections, notice that you also miss the step size of the finite difference.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Parameters for solving problem in the interval -L &amp;lt; x &amp;lt; L.&#xA;L = 5;  % Interval Length.                                                                                                                                &#xA;N = 1000;  % No of points.&#xA;x = linspace(-L, L, N)';  % Coordinate vector&#xA;dx = x(2) - x(1);  % Coordinate step&#xA;&#xA;% Finite square well of width 2w and depth given by U_0.&#xA;U_0 = 40;&#xA;w = L / 10;                                          &#xA;U = U_0*(1 - heaviside(x + w) + heaviside(x - w));&#xA;&#xA;% Discretized (three-point finite-difference) representation of the Laplacian. &#xA;e = ones(N, 1);                                                  &#xA;Lap = spdiags([e  -2*e e], [-1 0 1], N, N)/dx^2;&#xA;&#xA;% Total Hamiltonian&#xA;hbar = 1;                                                           &#xA;m = 1;                                                    &#xA;H = -0.5*(hbar^2 / m)*Lap + spdiags(U, 0, N, N);  % T = H + U&#xA;&#xA;% Find (first 3) (smalest algebraic) eigenvalues.&#xA;nmodes = 6;&#xA;options.disp = 0;&#xA;[V, E] = eigs(H, nmodes, 'sm' , options);            &#xA;[E, ind] = sort(diag(E));                            &#xA;disp(E)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The following is the answer given by the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 3.2724&#xA;12.8576&#xA;27.7294&#xA;40.1985&#xA;40.2454&#xA;40.7874&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2018-09-02T19:42:36.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30142" PostTypeId="2" ParentId="30137" CreationDate="2018-09-02T20:38:34.680" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it expensive to calculate the error at these intermediate points?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;At a quick glance¹, you need 11 vector operations (scalar multiplications or vector additions) for each interpolation. For comparison, a Dormand–Prince step (without evaluations of the right-hand side) takes 45 vector operations¹. In many applications both of this is negligible as the 7 evaluations of the right-hand side of the differential equation are much more costly than this. Even if not, the interpolations are comparably cheap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;¹ Even if I am mistaken about this, at least the order of magnitude should be correct.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Will these intermediate points have an acceptable error that makes it worthwhile to do this procedure, or does high error make the interpolation useless throughout the entire interval?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For explicit Runge–Kutta methods of order higher than 3 (such as the Dormand–Prince method), the interpolation error is inevitably of higher order than the integration error. (This is why in cases where interpolation is required such as integrating DDEs, a third-order pair such as Bogacki–Shampine is often preferred.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even if your integration error has the same relative size as your integration error, there is another problem with what you propose, namely that you do not gain anything. For an example, suppose that the relative error you make during a full step is $ρ$ and your intermediate point is at $t+\tfrac{1}{2}h$, i.e., exactly in the middle of the step. Then the relative error of your interpolated step is $\tfrac{ρ}{2}$. If you now make two steps with that error, you have as a first-order approximation again an error of $ρ$, so you gain nothing. Now, of course, you could do your second step with a smaller $h$, but your first, interpolated step has still a higher error than a respective adapted step.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2018-09-02T20:38:34.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30145" PostTypeId="1" CreationDate="2018-09-03T13:47:59.207" Score="2" ViewCount="155" Body="&lt;p&gt;I want to implement a Bilinear Finite Volume discretisation of the anisotropic diffusion problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{du}{dt} = \nabla \cdot (\textbf{D} \nabla u)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both my degrees of freedom as well as the Diffusion Tensors are given on the vertices of my quadrilateral grid. I can, therefore, use bilinear test/trialfunctions on each cell to calculate the full gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering and am deeply puzzled as to how I am to approximate the Diffusion tensor on an internal sub-face of my control volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 1D case (TPFA), the diffusion coefficient is modeled to be constant over one cell and the degree of freedom is thought to be the same everywhere within the cell. The flux over an interface is made consistent by using the harmonic average of the two given diffusivities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f_{A-&amp;gt;B}= \frac{2}{\frac{1}{D_A}+ \frac{1}{D_B}} ~\nabla_x u$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, the gradient is then approximated by the simple finite difference of the values modeled to be in the cell centers.&#xA;sketch to clarify:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LGl4N.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LGl4N.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far so good. Now I am not in a setting where the Grid is aligned with the principal directions of the diffusion tensors in involved (not &quot;K-orthogonal&quot;). The Two-point flux approximation would, therefore, fail, because it can not capture the off-diagonal entries of the diffusion tensors involved, because only the x-derivative can be calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my 2D setting, &lt;strong&gt;I do reconstruct the full gradient&lt;/strong&gt;. I want to calculate four sub-fluxes within my cell as indicated by the following sketch:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9BcZQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9BcZQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the black dots indicate where my degrees of freedom lie. The dashed line shows the area where I model the diffusion tensor to be constant (dual grid), the friendly green dot marks the location where I want to evaluate the bottom-most of the four sub-fluxes to properly assemble. Following the procedure from the 1D case, I could use the harmonic average of the diffusion tensors of the involved dual cells A,B. Let's call it&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;option 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D_{eff} = \big(\frac{\textbf{A}^{-1 } + \textbf{B}^{-1} }{2}\big)^{-1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the &quot;-1&quot; in the exponents indicates the matrix inverse. What puzzles me so much is that by calculating the matrix inverse, the diagonal entries become coupled with the off-diagonal ones. To my understanding, these are different physical processes and I feel awkward to have, say, The component $D_{xx}$ depend on $D_{xy}$ etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second option I see is to do a harmonic average component-wise, to have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;option 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$D_{eff} = \begin{bmatrix}&#xA;\frac{2}{\frac{1}{D^{A}_{xx}} + \frac{1}{D^{B}_{xx}}} &amp;amp;&#xA;\frac{2}{\frac{1}{D^{A}_{xy}} + \frac{1}{D^{B}_{xy}}} \\&#xA;\frac{2}{\frac{1}{D^{A}_{yx}} + \frac{1}{D^{B}_{yx}}} &amp;amp;&#xA;\frac{2}{\frac{1}{D^{A}_{yy}} + \frac{1}{D^{B}_{yy}}} &#xA;\end{bmatrix}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If i do it this way, the diagonal component $D_{xx}$ is only dependend on the corresponding entries in $D_A$ and $D_B$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the correct way to do average the Diffusion Tensors at the sub-face marked with the green dot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&#xA;Investigating option two I found that as the offdiagonal entries of the Diffusion Matrices may be negative, I face the situation where I take the harmonic average of a negative and a positive number which, according to wikipdeia, is not defined properly. Help appreciated.&lt;/p&gt;&#xA;" OwnerUserId="28636" LastEditorUserId="28636" LastEditDate="2018-09-04T14:09:07.637" LastActivityDate="2018-09-04T14:09:07.637" Title="Harmonic average of Diffusion Tensors in Finite Volume Method" Tags="|finite-volume|diffusion|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30146" PostTypeId="2" ParentId="24005" CreationDate="2018-09-03T17:01:44.767" Score="6" Body="&lt;p&gt;The numerical flux is chosen to ensure that information in the problem travels in the direction of the characteristic curves of the equation (upwinding). As mentioned in the comments, the numerical flux is necessary in order to couple the subproblems defined on each element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to get an intuition for the role of the numerical flux is to consider the following simple example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider the scalar advection equation (where for simplicity &lt;span class=&quot;math-container&quot;&gt;$a=1$&lt;/span&gt;)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial u}{\partial t} + \frac{\partial u}{\partial x} = 0 \qquad\text{on $\Omega$},&#xA;$$&lt;/span&gt;&#xA;where the domain is given by &lt;span class=&quot;math-container&quot;&gt;$\Omega = [0,1]$&lt;/span&gt;. Because this is a hyperbolic equation, and information is propagating from left to right, we need to enforce a boundary condition at &lt;span class=&quot;math-container&quot;&gt;$x = 0$&lt;/span&gt; (but not at &lt;span class=&quot;math-container&quot;&gt;$x = 1$&lt;/span&gt;). For concreteness, suppose we enforce the Dirichlet condition &lt;span class=&quot;math-container&quot;&gt;$u(0,t) = g_D$&lt;/span&gt; for some given &lt;span class=&quot;math-container&quot;&gt;$g_D$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose now we discretize this equation using the DG method, and we use two elements, &lt;span class=&quot;math-container&quot;&gt;$D_1 = [0,1/2]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D_2 = [1/2,1]$&lt;/span&gt;. We could equally-well be discretizing the following set of two coupled PDEs,&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;\text{(PDE 1):}&amp;amp;&amp;amp; \quad v_t + v_x &amp;amp;= 0 \quad\text{on $D_1$},\\&#xA;\text{(PDE 2):}&amp;amp;&amp;amp; \quad w_t + w_x &amp;amp;= 0 \quad\text{on $D_2$},&#xA;\end{align*}&lt;/span&gt;&#xA;where we will couple these equations to make them equivalent to the original equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make the above equations well-posed, we need to enforce boundary conditions. As before, each equation is hyperbolic, and information is traveling from left to right. Therefore, we need to enforce a boundary condition for (PDE 1) on the left endpoint of &lt;span class=&quot;math-container&quot;&gt;$D_1$&lt;/span&gt;, and a boundary condition for (PDE 2) on the left endpoint of &lt;span class=&quot;math-container&quot;&gt;$D_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boundary condition on the left endpoint of &lt;span class=&quot;math-container&quot;&gt;$D_1$&lt;/span&gt; must be chosen to be &lt;span class=&quot;math-container&quot;&gt;$v(0,t) = g_D$&lt;/span&gt; in order to remain consistent with the original problem. We also look for smooth solutions, so the boundary condition on the left endpoint of &lt;span class=&quot;math-container&quot;&gt;$D_2$&lt;/span&gt; must be chosen to enforce continuity. This condition reads &lt;span class=&quot;math-container&quot;&gt;$w(1/2,t) = v(1/2,t)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The DG method in this case chooses the numerical fluxes precisely to enforce the above boundary conditions. If we multiply by a test function &lt;span class=&quot;math-container&quot;&gt;$\psi$&lt;/span&gt; and integrate by parts over each element &lt;span class=&quot;math-container&quot;&gt;$D_k$&lt;/span&gt;, we obtain boundary terms of the form&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;\int_{\partial D_1} \hat{n} \cdot v \psi \, dx &amp;amp;= \left[ v\psi \right]_0^{1/2}\\&#xA;\int_{\partial D_2} \hat{n} \cdot w \psi \, dx &amp;amp;= \left[ w\psi\right]_{1/2}^1 \\&#xA;\end{align*}&lt;/span&gt;&#xA;In order to &quot;weakly&quot; enforce the boundary conditions, we replace &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; with the prescribed values at those points where boundary conditions are specified (i.e. the left endpoints of &lt;span class=&quot;math-container&quot;&gt;$D_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D_2$&lt;/span&gt;). This means we replace &lt;span class=&quot;math-container&quot;&gt;$v(0,t)$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$g_D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w(1/2,t)$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$v(1/2,t)$&lt;/span&gt; in the boundary integrals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, we define &lt;span class=&quot;math-container&quot;&gt;$u_h^* = g_D$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$x = 0$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$u_h^* = v(1/2,t)$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$x=1/2$&lt;/span&gt;, and we recover exactly the standard upwind flux that is used in the DG method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at things this way, we can consider the numerical flux functions as weakly enforcing the boundary conditions on each element that are required to couple the equations in such a way that respects the characteristic structure of the equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For equations more complicated than constant-coefficient advection, information may not propagate always in the same direction, and so the numerical flux must be determined by solving (or approximating the solution to) a &lt;a href=&quot;https://en.wikipedia.org/wiki/Riemann_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Riemann problem&lt;/a&gt; at the interface. This is discussed for linear problems in Section 2.4 of Hesthaven's book.&lt;/p&gt;&#xA;" OwnerUserId="25388" LastEditorUserId="25388" LastEditDate="2019-02-03T23:11:36.367" LastActivityDate="2019-02-03T23:11:36.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30147" PostTypeId="1" AcceptedAnswerId="30148" CreationDate="2018-09-03T17:46:31.670" Score="3" ViewCount="471" Body="&lt;p&gt;I have a linear system of complex numbers. I am using &lt;a href=&quot;http://www.netlib.org/lapack/explore-3.1.1-html/zgesv.f.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAPACK' zgesv&lt;/a&gt; (actually I am using &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-gesv&quot; rel=&quot;nofollow noreferrer&quot;&gt;intel MKL LAPACKE&lt;/a&gt;, but I am assuming the algorithm is the same). No assumption can be made about the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to follow its subroutine calls, but I got lost. What algorithm does it use, how many operations will be needed according to the size $n$ of the system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick check on &lt;a href=&quot;https://en.wikipedia.org/wiki/LU_decomposition#Applications&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The cost of solving a system of linear equations is approximately&#xA;  $2n^3/3$&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Would that be the answer? I am interested in the performance of both large and small $n$.&lt;/p&gt;&#xA;" OwnerUserId="20992" LastActivityDate="2018-09-04T04:52:08.457" Title="How many operations are needed for LAPACK's zgesv to solve a linear system?" Tags="|linear-algebra|linear-solver|lapack|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30148" PostTypeId="2" ParentId="30147" CreationDate="2018-09-03T18:31:22.340" Score="5" Body="&lt;p&gt;The LAPACK routine &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d6/d10/group__complex16_g_esolve_ga531713dfc62bc5df387b7bb486a9deeb.html#ga531713dfc62bc5df387b7bb486a9deeb&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;zgesv&lt;/code&gt;&lt;/a&gt; first computes the LU factorization, and then solves the system making use of the factorization. It is a simple driver for calling the two routines &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d3/d01/group__complex16_g_ecomputational_ga5b625680e6251feb29e386193914981c.html#ga5b625680e6251feb29e386193914981c&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;zgetrf&lt;/code&gt;&lt;/a&gt; (compute the LU factorization) and &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d3/d01/group__complex16_g_ecomputational_ga3a5b88a7e8bf70591e521e86464e109d.html#ga3a5b88a7e8bf70591e521e86464e109d&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;zgetrs&lt;/code&gt;&lt;/a&gt; (solve the system).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LU factorization is computed using partial pivoting and row interchanges. Then, the system is solved using simple backwards and forwards substitution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the number of operations needed for &lt;code&gt;zgesv&lt;/code&gt; will be the sum of the operations for the two sub-calls. Appendix C of the &lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn41.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;installation guide&lt;/a&gt; for LAPACK by Blackford and Dongarra lists the operation counts needed for certain LAPACK routines. The operation counts are given for the real-valued, single precision versions. To obtain the operation counts for complex-valued versions, we notice that each complex addition is computed with two real additions, and each complex multiplication is computed with four real multiplications and two real additions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the installation guide:&#xA;\begin{align*}&#xA;\texttt{SGETRF} \qquad &amp;amp; \text{multiplications:}&amp;amp;&amp;amp; 1/2mn^2 - 1/6n^3+1/2mn - 1/2n^2 +2/3n \\&#xA;&amp;amp; \text{additions:} &amp;amp;&amp;amp; 1/2mn^2 - 1/6n^3 - 1/2mn + 1/6n \\[20pt]&#xA;\texttt{SGETRS} \qquad &amp;amp; \text{multiplications:}&amp;amp;&amp;amp; \texttt{NRHS} [n^2] \\&#xA;&amp;amp; \text{additions:} &amp;amp;&amp;amp; \texttt{NRHS}[n^2 - n]&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Converting to complex-valued operations, we obtain&#xA;\begin{align*}&#xA;\texttt{ZGETRF} \qquad &amp;amp; \text{multiplications:}&amp;amp;&amp;amp;&#xA;     2mn^2 - 2/3n^3 + 2mn - 2n^2 + 8/3n \\&#xA;&amp;amp; \text{additions:} &amp;amp;&amp;amp;&#xA;     2mn^2 - 2/3n^3 - n^2 + 5/3n \\[20pt]&#xA;\texttt{ZGETRS} \qquad &amp;amp; \text{multiplications:}&amp;amp;&amp;amp;\texttt{NRHS} [4n^2] \\&#xA;&amp;amp; \text{additions:} &amp;amp;&amp;amp; \texttt{NRHS}[4n^2 - 2n]&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Adding up:&#xA;\begin{align*}&#xA;\texttt{ZGESV} \qquad &amp;amp; \text{total:}&amp;amp;&amp;amp; 4mn^2 - 4/3n^3 + 2mn -3n^2 +13/3n + \texttt{NRHS}[8n^2 - 2n]&#xA;\end{align*}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the system is square ($m=n$) and there is only one right-hand side ($\texttt{NRHS}=1$), this reduces to&#xA;$$&#xA;8/3n^3 + 7n^2 + 7/3n.&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="25388" LastEditorUserId="25388" LastEditDate="2018-09-04T04:52:08.457" LastActivityDate="2018-09-04T04:52:08.457" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30150" PostTypeId="2" ParentId="29377" CreationDate="2018-09-04T14:30:15.077" Score="1" Body="&lt;p&gt;First of all, I think you misunderstand the concept of adaptive quadrature. Adaptive quadrature does not imply &quot;interactively placing sample points&quot;. The whole idea behind adaptive quadrature is to devise a scheme that will integrate a certain function to a certain (estimated) absolute or relative error with as little function evaluations as possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second remark: you write &quot;The number of sampling points is fixed (due to function evaluation being costly), but I can freely place them&quot;. I think the idea should be that the number of sampling points (or function evaluations in quadrature terminology) should be as small as possible (i.e. not fixed). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is the idea behind adaptive quadrature as implemented in &lt;a href=&quot;http://www.netlib.org/quadpack/&quot; rel=&quot;nofollow noreferrer&quot;&gt;QUADPACK&lt;/a&gt; for example? &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The basic ingredient is a &quot;nested&quot; quadrature rule: this is a combination of two quadrature rules where one has a higher order (or accuracy) as the other. Why? Based on the difference between these rules, the algorithm can estimate the quadrature error (of course, the algorithm will use the most accurate one as the reference result). Examples could be the trapezoid rule with $2^{n}$ nodes and $2^{n+1}$ nodes. In the case of QUADPACK, the rules are Gauss-Kronrod rules. These are interpolatory quadrature rules that use a Gauss-Legendre quadrature rule of a certain order $N$ and an optimal extension of this rule. This means that a higher quadrature order can be obtained by re-using the Gauss-Legendre nodes (i.e. the costly function evaluations) with different weights and adding a number of extra nodes. In other words, the original Gauss-Legendre rule of order $N$ will integrate all polynomials of degree $2N-1$ exactly while the extended Gauss-Kronrod rule will integrate some higher order polynomial exactly. A classic rule is the G7K15 (7th order Gauss-Legendre with 15th order Gauss-Kronrod). The magic is that the 7 nodes of the Gauss-Legendre are a subset of the 15 nodes of the Gauss-Kronrod so with 15 function evaluations, I have a quadrature evaluation together with an error estimate!&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Next ingredient is a &quot;divide-and-conquer&quot; strategy. Suppose you let loose this G7K15 on your integrand and you observe a quadrature error that is according to your taste too large. QUADPACK will then subdivide the original interval in two equally spaced subintervals. And then it will re-evaluate the two subintegrals using the basic rule, G7K15. Now, the algorithm has a global error estimate (which should be hopefully lower than the first one) but also two local error estimates. It picks the interval with the largest error and divides this one in two. Two new integrals are estimated and the global error is updated. And so on until the global error is below your requested target or the maximum number of subdivisions has been surpassed. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So I challenge you to update your code above using the &lt;code&gt;scipy.quad&lt;/code&gt; method. Maybe in case of an integrand with a lot of &quot;fine structure&quot; you might need to increase the maximum number of subdivisions (the &lt;code&gt;limit&lt;/code&gt; option). You could also play with the &lt;code&gt;epsabs&lt;/code&gt; and/or &lt;code&gt;epsrel&lt;/code&gt; parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you only have experimental data, I see two possibilities. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you have the opportunity to select the measurement points, i.e. values of $t$, I would select them equidistanly and preferably as a power of $2$ so that you can apply a nested trapezoidal rule (and profit from Romberg extrapolation). &lt;/li&gt;&#xA;&lt;li&gt;If you have no means of choosing the nodes, i.e. the measurements come at random times, the best option in my opinion still is the trapezoid rule. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2018-09-07T06:50:15.197" LastActivityDate="2018-09-07T06:50:15.197" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30152" PostTypeId="1" CreationDate="2018-09-04T20:47:54.860" Score="1" ViewCount="538" Body="&lt;p&gt;I'm trying to solve this advection-diffusion equation (ADE):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\partial \phi}{\partial t} + \nabla \cdot (-D \nabla \phi + \mathbf{u} \phi) = 0$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, this ADE framework is coupled to a Navier-Stokes (NS) solver and I get velocity field ($\mathbf{u}$) from NS solver. My question is about ADE boundary conditions on inflow and outflow planes. I postulate that because the flow is directed from inflow to outflow obviously, I put zero concentration ($\phi = 0$) boundary condition on inlet plane because my source of species is located far from inflow boundary. Also, I assumed that the dominant transport mechanism at the outlet plane is convection and as a result of that normal diffusive flux will be vanished at the outlet plane ($-D \frac{\partial \phi}{\partial \vec{n}} = 0$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to know how much these assumptions are true and could be justified physically? I should say that the outflow boundary condition is verified by comparing the simulation results with analytical solution of the Graetz problem. Any reference, suggestion, or idea is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="28407" LastActivityDate="2018-09-04T20:47:54.860" Title="Inflow and outflow boundary conditions for advection-diffusion equation" Tags="|boundary-conditions|navier-stokes|advection-diffusion|advection|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30153" PostTypeId="1" CreationDate="2018-09-05T08:26:49.777" Score="1" ViewCount="265" Body="&lt;p&gt;I'm trying to implement a model I found in a paper, but there is something I do not understand. The authors say they use TDMA to solve their equations; however, they use a 3rd order upwind biased approximation to discretize the convective terms. This leads to an almost tridiagonal matrix (with coeff. for the terms $i-1$, $i$, and $i+1$), but with an extra diagonal line for the $i-2$ terms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to use a TDMA solver with a matrix like this? Is there some procedure to tridiagolise?&lt;/p&gt;&#xA;" OwnerUserId="28651" LastEditorUserId="20688" LastEditDate="2018-10-17T05:08:18.210" LastActivityDate="2018-10-17T05:08:18.210" Title="TDMA with 3rd order upwind scheme" Tags="|linear-algebra|finite-volume|linear-system|banded-matrix|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30154" PostTypeId="1" CreationDate="2018-09-05T16:28:27.123" Score="1" ViewCount="51" Body="&lt;p&gt;I have a question for the format of Schrodinger equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\psi(x,t) = \int_0^\infty c_n e^{-iE_nt/\hbar} \psi_n(x)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do we have $i$ instead of $-i$?&lt;/p&gt;&#xA;" OwnerUserId="28653" LastEditorUserId="20688" LastEditDate="2018-09-05T18:24:25.093" LastActivityDate="2018-09-05T18:24:25.093" Title="The sign of Schrodinger equation" Tags="|eigensystem|fourier-transform|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30157" PostTypeId="1" CreationDate="2018-09-06T13:23:39.847" Score="16" ViewCount="2434" Body="&lt;p&gt;For teaching purposes I'd need a continuous function of a single variable that is &quot;difficult&quot; to approximate with polynomials, i.e. one would need very high powers in a power series to &quot;fit&quot; this function well. I intend to show my students the &quot;limits&quot; of what can be achieved with power series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought about concocting something &quot;noisy&quot;, but instead of rolling my own I am just wondering whether there is a kind of standard &quot;difficult function&quot; that people use for testing approximation / interpolation algorithms, somewhat similarly to those &lt;a href=&quot;https://en.wikipedia.org/wiki/Test_functions_for_optimization&quot; rel=&quot;noreferrer&quot;&gt;optimisation test functions&lt;/a&gt; that have numerous local minima where naive algorithms get stuck easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apologies if this question is not well-formed; please have mercy on a non-mathematician.&lt;/p&gt;&#xA;" OwnerUserId="7024" LastActivityDate="2018-09-12T09:09:59.497" Title="Example of a continuous function that is difficult to approximate with polynomials" Tags="|interpolation|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30158" PostTypeId="2" ParentId="30157" CreationDate="2018-09-06T14:01:10.873" Score="6" Body="&lt;p&gt;Approximation is not only made hard by the function to be approximated but by the interval in which the approximation should be a &quot;good fit&quot;. And you should define the measure for a &quot;good fit&quot;, i.e. what is the maximum (absolute or relative) error you wish to tolerate? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you will need a huge number of terms in the Taylor series of $\exp(x)$  to have a reasonable fit on the interval $[0,10]$. The same holds for periodic functions. Take $\sin(x)$, for example, on the interval $[0,2\pi]$. See pictures below...&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FFaQP.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FFaQP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/BmTdi.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BmTdi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="9667" LastEditDate="2018-09-06T15:48:22.250" LastActivityDate="2018-09-06T15:48:22.250" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30159" PostTypeId="2" ParentId="30157" CreationDate="2018-09-06T15:28:43.243" Score="10" Body="&lt;p&gt;It's a pathological case, but you can always resort to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Weierstrass_function&quot; rel=&quot;noreferrer&quot;&gt;Weierstrass monster function&lt;/a&gt;. It illustrates a broader point, namely that functions that are not smooth -- e.g., that have a kink -- are difficult to approximate because the interpolation error estimates require the function being interpolated to be differentiable a number of times. In other words, if you don't like the Weierstrass function too much, you can always just choose $|x|$.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-09-06T15:28:43.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30160" PostTypeId="2" ParentId="30157" CreationDate="2018-09-06T22:04:16.967" Score="14" Body="&lt;p&gt;Why not simply show the absolute value function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Approximation with e.g. Legendre-polynomial expansion works, but &lt;a href=&quot;https://en.wikipedia.org/wiki/Gibbs_phenomenon&quot; rel=&quot;noreferrer&quot;&gt;pretty badly&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LEnzg.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LEnzg.gif&quot; alt=&quot;Sequential approximation of the absolute-value function by polynomials&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taylor expansion is of course completely useless here, always giving only a linear function, either always decreasing or always increasing (depending on whether the point you expand around is negative or positive).&lt;/p&gt;&#xA;" OwnerUserId="562" LastActivityDate="2018-09-06T22:04:16.967" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30161" PostTypeId="2" ParentId="30157" CreationDate="2018-09-07T08:35:20.307" Score="2" Body="&lt;p&gt;$f(x) = \frac{1}{x^2+1}$ has this Maclaurin series expansion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{1}{x^2+1} = 1-x^2+x^4-x^6+x^8-x^{10}+x^{12} - \ldots$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This converges for $-1&amp;lt;x&amp;lt;1$, but it diverges everywhere else. A polynomial approximation around $x=0$ will never get you anywhere near the right answer for $x=2$.&lt;/p&gt;&#xA;" OwnerUserId="28671" LastActivityDate="2018-09-07T08:35:20.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30162" PostTypeId="1" CreationDate="2018-09-07T13:28:00.793" Score="1" ViewCount="45" Body="&lt;p&gt;I understand that the function &lt;code&gt;applyrule&lt;/code&gt; uses &lt;code&gt;patmatch&lt;/code&gt; to identify patterns and apply transformations accordingly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose, I want to write a rule for transforming spatial derivatives along $z-$direction of field components $E_i(x,y,z)$ to momentum space, so that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$\partial_{z}(E_i(x,y,z)) \to -ikE_i(x,y,z)$.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried to define a rule as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In: applyrule(diff(f::function,z)=-I*k*f,diff(E1(x, y, z), z))&#xA;Out: diff(E1(x, y, z), z)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It does not work. The reason is, patmatch is unable to match the above pattern:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In: patmatch(diff(E1(x, y, z), z), diff(f::function, z))&#xA;Out: false&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On the contrary, if there is no &lt;code&gt;diff()&lt;/code&gt; operator, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In: patmatch(E1(x, y, z), f::function)&#xA;Out: true&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As a workaround, I currently write the following command: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;applyrule(diff(dE1(X), z) = -I*k*dE1(X),expression)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for each field component in &lt;code&gt;expression&lt;/code&gt; and it works fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would like to know an easier way to do this so that I don't have to write the rules for each component separately. Specifically, the inability of &lt;code&gt;patmatch&lt;/code&gt; to identify pattern inside &lt;code&gt;diff()&lt;/code&gt; is troubling. Any workaround or solution to resolve this would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="28675" LastEditorUserId="20688" LastEditDate="2018-09-07T15:01:41.993" LastActivityDate="2018-09-07T15:01:41.993" Title="Maple: patmatch does not identify patterns inside diff() operator" Tags="|symbolic-computation|maple|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30163" PostTypeId="1" CreationDate="2018-09-07T13:51:02.927" Score="1" ViewCount="52" Body="&lt;p&gt;I have come across an optimization problem with the following objective function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(x_0,y_0,z_0,x_1,y_1,z_1,...,x_N,y_N,z_N) = \sum_{i=0}^N f_i(x_i,y_i,z_i, \alpha(x_{i+1}-x_i) + \beta(y_{i+1}-y_i))$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. the objective function is the sum of the functions $f_i$ that only depend on three variables $x_i,y_i,z_i$ and on a linear combination of the difference to the direct neighbors (for $x_i$ and $y_i$). So far I have tired an NLCG algorithm, but convergence is very slow. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a specialized solver that can exploit the structure of the optimization problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thought about introducing new variables $s_i = \alpha(x_{i+1}-x_i) + \beta(y_{i+1}-y_i)$ and writing&#xA;$$f(x_0,y_0,z_0,s_0,...,x_N,y_N,z_N,s_N) = \sum_{i=0}^N f_i(x_i,y_i,z_i,s_i)$$ because then the optimization could be conducted separately for each $f_i$ but I would have to introduce equality constraints to get back the original problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$f_i(x_i,y_i,z_i,s_i)$ is nonconvex (there is a $\cos(s_i)$ term in the function); however $f_i(x_i,y_i,z_i)$ is convex.&lt;/p&gt;&#xA;" OwnerUserId="25502" LastEditorUserId="20688" LastEditDate="2018-09-07T17:59:52.323" LastActivityDate="2018-09-07T17:59:52.323" Title="Efficient numerical optimization of an &quot;almost separable&quot; function" Tags="|optimization|constrained-optimization|numerics|nonconvex|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30164" PostTypeId="1" CreationDate="2018-09-07T14:41:24.280" Score="4" ViewCount="185" Body="&lt;p&gt;The discussion in &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5515/how-does-density-functional-theory-scale-with-system-size&quot;&gt;How does density functional theory scale with system size?&lt;/a&gt; already tells, how DFT scales with the number of electrons. &#xA;I want to simulate some clusters, so the simulation cell contains much vacuum and its size can be changed without affecting the number of electrons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that with a larger cell less k-points are needed in my plane-wave-code, but I am wondering which further scaling effects there are. &lt;/p&gt;&#xA;" OwnerUserId="28676" LastActivityDate="2018-09-08T00:59:24.237" Title="How does density-functional-theory scale with cell size, not with number of electrons" Tags="|density-functional-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30165" PostTypeId="1" AcceptedAnswerId="30206" CreationDate="2018-09-07T22:03:46.880" Score="3" ViewCount="160" Body="&lt;p&gt;I'm interested in running Hartree-Fock calculations on single atoms for a project, and I was trying to find more information about how to actually run them instead of just derivations of equations. Most of what I've found involves simulating molecules, but since I just want to do single atoms, can I use the hydrogen wavefunctions as basis functions? They seem nice because they're all orthogonal, and calculating the overlap integrals will be easy. Is there a reason not to use them?&lt;/p&gt;&#xA;" OwnerUserId="28677" LastActivityDate="2018-09-15T09:26:39.793" Title="Hartree-Fock Hydrogen Basis Functions" Tags="|hartree-fock|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30166" PostTypeId="2" ParentId="30164" CreationDate="2018-09-08T00:59:24.237" Score="2" Body="&lt;p&gt;In planewave calculations, we use a regularly spaced grid that fills the entire unit cell to perform calculations. The number of planewaves is proportional to the number of grid points used (as well as other system dependent parameters). Thus, when you add vacuum to your unit cell, you increase the number of planewave basis functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this in mind, you can use the link you posted to figure out the scaling. In the link, he has four parameters: $n_a$, $n_e$, $n_p$, and $n_v$. $n_v$ is the size of the basis, so when adding vacuum, only the value of $n_v$ will change. &lt;/p&gt;&#xA;" OwnerUserId="23968" LastActivityDate="2018-09-08T00:59:24.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30167" PostTypeId="1" CreationDate="2018-09-08T01:38:41.797" Score="3" ViewCount="460" Body="&lt;p&gt;I want the eigenvalues of the following generalized eigenvalue problem:&#xA;$$ Av = \lambda M v $$&#xA;where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;$A\in\mathbb{R}^{n\times n}$ is sparse, symmetric, and positive semi-definite&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;$M\in\mathbb{R}^{n\times n}$ is sparse, symmetric, and indefinite (i.e. neither positive nor negative definite) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want the eigenvalues $\lambda$ with the smallest absolute value, i.e. $|\lambda|$ closest to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that I am using &lt;code&gt;scipy&lt;/code&gt; and in particular &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.eigsh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scipy.sparse.linalg.eigsh&lt;/code&gt;&lt;/a&gt;. (Note that I have to use Python, but not necessarily &lt;code&gt;scipy&lt;/code&gt;, if someone knows a better library.) In this case, &lt;code&gt;eigsh&lt;/code&gt; forces me to use the &lt;code&gt;buckling&lt;/code&gt; option, since $M$ is indefinite. However, the &lt;code&gt;buckling&lt;/code&gt;  mode sorts based on eigenvalues transformed via:&#xA;$$ \widetilde{\lambda} = \frac{\lambda}{\lambda - \sigma} $$&#xA;where $\sigma$ is a user-defined constant. Unfortunately, I can't figure out how to get the eigenvalues with smallest magnitude from this.&#xA;If I choose $\sigma$ to be large, then ask &lt;code&gt;eigsh&lt;/code&gt; for the smallest magnitude (i.e. option &lt;code&gt;SM&lt;/code&gt;), it is too slow. Slightly better is using the option &lt;code&gt;BE&lt;/code&gt;, which returns values from both ends of the spectrum, with a small $\sigma$ close to zero. However, there's no reason to believe the smallest eigenvalues are equally distributed between positive and negative.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does anyone have any advice for dealing with &lt;em&gt;getting the smallest magnitude eigenvalues&lt;/em&gt; from a generalized symmetric eigenvalue problem where the mass matrix is indefinite?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;(As a separate note, I have to add $\xi I$ to $A$, and then subtract $\xi$ from $\lambda$ at the end (for small $\xi$), so that it goes from positive semi-definite to positive definite, as required by &lt;code&gt;buckling&lt;/code&gt; mode. I am aware that this is not an exact correction for the generalized eigenproblem as it is for the simpler case, but I could not find a better alternative so far, and it doesn't seem to be a big issue. I've asked a related question &lt;a href=&quot;https://math.stackexchange.com/questions/2909125/adding-a-multiple-of-the-identity-into-a-generalized-eigenvalue-problem&quot;&gt;here&lt;/a&gt;, specific to this aspect.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT (091018): I needed some preliminary results and it seems like the following works, but certainly may not be optimal in terms of speed. Simply solve $M^{-1}Av=\lambda v$ instead using &lt;code&gt;eigs&lt;/code&gt;. The trick to making it fast is using prefactorizations (i.e. sparse LUs) to make computing $M^{-1}A$ and $A^{-1}M$ fast. I'd love to know more about potentially better ways to do this!&lt;/p&gt;&#xA;" OwnerUserId="24492" LastEditorUserId="24492" LastEditDate="2018-09-10T17:01:29.173" LastActivityDate="2018-09-10T17:01:29.173" Title="Computing Small Eigenvalues with Sparse Symmetric Indefinite Mass Matrix" Tags="|linear-algebra|matrix|python|eigenvalues|scipy|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="30168" PostTypeId="2" ParentId="30157" CreationDate="2018-09-08T03:09:27.570" Score="6" Body="&lt;p&gt;Polynomials are surprisingly effective at function approximation [1]. If you have at least Lipschitz continuity, then Chebyshev approximations will converge. Of course, convergence may be slow, and that is the price we pay for dealing with a non-smooth function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Today, computers are much faster than the days in which many numerical analysis books were written, and clever algorithms have increased the speed further, so that having to use more terms may not be as bad as it used to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pathological examples like Weierstrass monster function are interesting from a theoretical point of view, but they are not representative of most real application contexts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think we should be careful not to give a pessimistic view to our students. Ok, there is no method that works for all problems. But we can build adaptive methods, say even based on polynomials, that can deal with such cases. For example, Chebfun [2] can easily approximate $|x|$ by automatically splitting the domain at $x=0$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to teach the difficulties in approximation with polynomials, but it also important to tell the students that we can build error estimates and adaptive algorithms that can deal with these issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://people.maths.ox.ac.uk/trefethen/mythspaper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://people.maths.ox.ac.uk/trefethen/mythspaper.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] &lt;a href=&quot;http://www.chebfun.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.chebfun.org&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="20688" LastEditDate="2018-09-08T13:37:05.567" LastActivityDate="2018-09-08T13:37:05.567" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30169" PostTypeId="1" AcceptedAnswerId="30171" CreationDate="2018-09-08T18:27:39.430" Score="0" ViewCount="2557" Body="&lt;p&gt;Imagine a simple triangular base mesh in finite element method with an unknown number of elements (varying by the user). How can connectivity matrix be coded to be generated automatically?&lt;/p&gt;&#xA;" OwnerUserId="5439" LastEditorUserId="20688" LastEditDate="2018-09-08T19:05:42.667" LastActivityDate="2018-09-08T20:19:34.540" Title="Connectivity matrix in Finite Element Method in Triangular elements" Tags="|finite-element|computational-geometry|mesh-generation|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30171" PostTypeId="2" ParentId="30169" CreationDate="2018-09-08T19:35:04.657" Score="1" Body="&lt;p&gt;A proper generated finite-element mesh comprises both the mesh vertices, as well as the elements that consist of those vertices – thus, a &lt;em&gt;connectivity matrix&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, see the basic &lt;a href=&quot;http://gmsh.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMSH&lt;/a&gt; geometry file that describes a 1-meter square in 2D centered at $(0,0)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Square.geo&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cl = 0.5;&#xA;Point(1) = {0.5,0.5,0.,cl};&#xA;Point(2) = {-0.5,0.5,0.,cl};&#xA;Point(3) = {-0.5,-0.5,0.,cl};&#xA;Point(4) = {0.5,-0.5,0.,cl};&#xA;&#xA;Line(1) = {1,2};&#xA;Line(2) = {2,3};&#xA;Line(3) = {3,4};&#xA;Line(4) = {4,1};&#xA;Line Loop(1) = {1,2,3,4};&#xA;Surface(1) = {1};&#xA;Physical Surface(1) = {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This geometry description results in the following mesh (density of the mesh in this example is controlled by characteristic length &lt;code&gt;cl&lt;/code&gt;). Notice, the mesh file has the following sections:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Header in &lt;code&gt;$MeshFormat&lt;/code&gt; (mesh file version and other auxiliary info)&lt;/li&gt;&#xA;&lt;li&gt;Mesh element vertices in &lt;code&gt;$Nodes&lt;/code&gt; (id, $x$, $y$, $z$)&lt;/li&gt;&#xA;&lt;li&gt;Triangles in &lt;code&gt;$Elements&lt;/code&gt; (id, 4 numbers that do not mean a lot for this discussion, vert1, vert2, vert3). This is pretty much the connectivity matrix you are looking for.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Square.msh&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$MeshFormat&#xA;2.2 0 8&#xA;$EndMeshFormat&#xA;$Nodes&#xA;13&#xA;1 0.5 0.5 0&#xA;2 -0.5 0.5 0&#xA;3 -0.5 -0.5 0&#xA;4 0.5 -0.5 0&#xA;5 1.376398994779038e-12 0.5 0&#xA;6 -0.5 1.376398994779038e-12 0&#xA;7 -1.376398994779038e-12 -0.5 0&#xA;8 0.5 -1.376398994779038e-12 0&#xA;9 0 0 0&#xA;10 -0.2499999999996558 0.2500000000003442 0&#xA;11 0.2500000000003441 0.2499999999996558 0&#xA;12 -0.2500000000003442 -0.2499999999996558 0&#xA;13 0.2499999999996559 -0.2500000000003442 0&#xA;$EndNodes&#xA;$Elements&#xA;16&#xA;1 2 2 1 1 10 5 2&#xA;2 2 2 1 1 10 6 9&#xA;3 2 2 1 1 6 10 2&#xA;4 2 2 1 1 5 10 9&#xA;5 2 2 1 1 5 11 1&#xA;6 2 2 1 1 8 11 9&#xA;7 2 2 1 1 11 8 1&#xA;8 2 2 1 1 11 5 9&#xA;9 2 2 1 1 12 6 3&#xA;10 2 2 1 1 12 7 9&#xA;11 2 2 1 1 7 12 3&#xA;12 2 2 1 1 6 12 9&#xA;13 2 2 1 1 13 7 4&#xA;14 2 2 1 1 13 8 9&#xA;15 2 2 1 1 8 13 4&#xA;16 2 2 1 1 7 13 9&#xA;$EndElements&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6jU4a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6jU4a.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, a proper finite-element generation utility (GMSH here is just an example and there are many other very good alternatives) gives you the connectivity matrix; otherwise, your mesh is not properly defined anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the process of how to write a mesh-generating software yourself (&lt;em&gt;which you should not do, unless you have very very good reasons&lt;/em&gt;) is out-of-topic for this discussion.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-09-08T20:19:34.540" LastActivityDate="2018-09-08T20:19:34.540" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30172" PostTypeId="1" CreationDate="2018-09-08T19:37:09.783" Score="1" ViewCount="99" Body="&lt;p&gt;Can someone recommend the best way to solve a least squares fitting problem with B-splines, with additional equality constraints? I want to solve:&#xA;$$&#xA;\min_x || b - A x ||^2, \textrm{subject to: } C x = d&#xA;$$&#xA;where $A$ is $m \times n$ and $C$ is $p \times n$. Specifically I want to fit B-splines and force the first derivative of the resulting spline to take prescribed values at a certain number of points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the unconstrained problem, the standard approach seems to be to solve the normal equations&#xA;$$&#xA;A^T A x = A^T b&#xA;$$&#xA;because for a basis of B-splines of order $k$, the normal equations matrix  $A^T A$ is symmetric and banded, with lower bandwidth $k-1$. So a banded Cholesky decomposition can be applied to efficiently solve this system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I can't find much literature on dealing with constraints and B-spline fitting. For linear equality constrained systems, LAPACK uses the Generalized RQ decomposition, which does not require forming the normal equations matrix. I could of course do this as well, but the Generalized RQ algorithm won't then take advantage of the sparse structure of the matrix $A$. It might be possible to adapt the RQ algorithm to deal with the sparsity of $A$, but there don't appear to be any existing libraries which do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just wondering if anyone has experience with this and/or recommendations?&lt;/p&gt;&#xA;" OwnerUserId="28440" LastActivityDate="2018-09-08T19:37:09.783" Title="B-splines least squares with equality constraints" Tags="|least-squares|efficiency|b-spline|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30173" PostTypeId="1" CreationDate="2018-09-09T00:25:20.807" Score="1" ViewCount="187" Body="&lt;p&gt;I am trying to simulate a robot manipulator dynamics in SciLab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I generated a step function that has constant acceleration for half of the time and then the same acceleration but negative for the other half, so I get a smooth transition between the manipulator positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code generates the velocity and position from the step function I mentioned:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [position,velocity,acceleration,time]=smoothTransition(initialPosition,finalPosition,resolution,timeSpan)&#xA;&#xA;    a=(finalPosition-initialPosition)/((timeSpan/2)**2);//magnitud of acceleration and deceleration so I get to final position in timespan&#xA;    acceleration=[ones(1,resolution/2)*a -ones(1,resolution/2)*a];&#xA;    if modulo(resolution,2) ~= 0 then&#xA;        acceleration=[acceleration -a];//case where time resolution is odd&#xA;    end&#xA;    time=linspace(0,timeSpan,resolution);&#xA;    function dx=f(t,x)&#xA;        dx(1)=x(2);&#xA;        dx(2)=linear_interpn(t,time,acceleration);&#xA;&#xA;    endfunction&#xA;    x=ode([0;0],time(1),time,f);&#xA;&#xA;    velocity=x(2,:);&#xA;    position=x(1,:);&#xA;endfunction&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, I integrate the step function twice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula to get the torque required is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\tau=gm_1s_{1x}\cos(q)+\ddot{q}m_1s_{1x}^2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(this is a simplified version with one link)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $g$ is the gravity magnitude, $s_{1x}$ is how far is the center of mass in the x-direction, $m_1$ is the mass of the link, and $q$ is the angle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am trying to do is generate a torque input with this equation an then do the numeric integration to get $q$ and its derivative back (mostly for testing purpose).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am trying to solve this numerically:&#xA;$$&#xA;\ddot{q}=\frac{\tau-gm_1s_{1x}\cos(q)}{m_1s_{1x}^2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I don't get the same behavior back when I integrate for more than 1 second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code to do this is as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m_1=1;&#xA;g=9.81;&#xA;s_1x=1;&#xA;[position,velocity,acceleration,time]=smoothTransition(0,%pi/2,100,10);&#xA;&#xA;tau=g*m_1*s_1x*cos(position)+acceleration*m_1*s_1x**2;&#xA;&#xA;function dx=f(t,x)&#xA;    torque=linear_interpn(t,time,tau);&#xA;    dx(1)=x(2);&#xA;    dx(2)=(-g*m_1*s_1x*cos(x(1))+torque)/(m_1*s_1x**2);&#xA;endfunction&#xA;q0=[0;0];&#xA;q=ode(q0,0,time,f);&#xA;plot(time,position,'r');&#xA;plot(time,q(1,:),'g');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this code I plug the $\tau$ input and integrate two times, I suspect the problem is in this part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this code I get the following figure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XausX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XausX.png&quot; alt=&quot;graph&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the red curve is the expected behavior and the green curve is the obtained one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, the problem persists if I increase the time resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize that when I crank up the resolution (let say 10000) the result curve (green) does approximate the the correct behavior (red). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the result with 10000 time resolution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ynvv4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ynvv4.png&quot; alt=&quot;plot with 10000 time resolution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do a more exact integration without so much time resolution?&lt;/p&gt;&#xA;" OwnerUserId="28685" LastEditorUserId="28685" LastEditDate="2018-09-11T23:15:17.010" LastActivityDate="2020-04-27T21:04:40.437" Title="Integrating direct dynamics form more than 1 second does not give back the correct result" Tags="|ode|numerical-modelling|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="30174" PostTypeId="1" AcceptedAnswerId="30180" CreationDate="2018-09-09T16:58:29.673" Score="4" ViewCount="80" Body="&lt;p&gt;I simply want to solve the elliptic equation:&#xA;$$&#xA;-\kappa \nabla^2 u + u = f&#xA;$$&#xA;where $f\in [0,1]$. When using continuous Galerkin with Lagrange elements, I have noticed that $\kappa$ has to be greater than a certain value related to the mesh resolution to avoid values of $u$ under 0 or over 1. I would like to avoid these over/undershoots without having to increase the mesh resolution. I was advised to solve this equation using finite volume, which solved the problem. However, its implementation with non-cartesian second-order meshes is onerous. Therefore I decided to switch to stabilized methods:&#xA;$$&#xA;(k\nabla u, \nabla v) + (u,v) +  (\alpha h^2 \nabla u, \nabla v)  = (f,v)&#xA;$$&#xA;This seems to work given that I am artificially increasing the diffusion coefficient which will vanish with refinement. However, I haven't been able to find literature on appropriate values for $\alpha$. Could anybody give me some pointers? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2018-09-11T12:40:25.957" Title="Stabilization parameter for an elliptic equation" Tags="|finite-element|elliptic-pde|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30176" PostTypeId="1" AcceptedAnswerId="30177" CreationDate="2018-09-10T13:52:52.287" Score="0" ViewCount="1261" Body="&lt;p&gt;You can set the Absolute or Relative ODE solver tolerances in Matlab with the options structure from an &lt;code&gt;odeset&lt;/code&gt; command. But how do I find the default values for the different ODE solvers?&lt;/p&gt;&#xA;" OwnerUserId="4575" LastEditorUserId="20688" LastEditDate="2018-09-10T14:02:31.890" LastActivityDate="2018-09-10T14:02:31.890" Title="How do I find the default ODE solver tolerances in Matlab?" Tags="|matlab|ode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30177" PostTypeId="2" ParentId="30176" CreationDate="2018-09-10T14:02:04.790" Score="2" Body="&lt;p&gt;According to the &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/odeset.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;odeset&lt;/code&gt; documentation in R2018a&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;relative tolerance has a default of $10^{-3}$&lt;/li&gt;&#xA;&lt;li&gt;absolute tolerance has a default of $10^{-6}$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You should also be able to see them by running &lt;code&gt;odeset&lt;/code&gt; command without any arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7ceAv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7ceAv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-09-10T14:02:04.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30178" PostTypeId="2" ParentId="22071" CreationDate="2018-09-11T03:07:23.737" Score="0" Body="&lt;p&gt;I recommend to &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialize  your transient simulation with the solution of steady-state one as this will make the vortices appear faster&lt;/li&gt;&#xA;&lt;li&gt;Activate the double precision in ANSYS fluent start dialog box&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28696" LastEditorUserId="20688" LastEditDate="2018-09-11T03:11:37.663" LastActivityDate="2018-09-11T03:11:37.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30179" PostTypeId="2" ParentId="1485" CreationDate="2018-09-11T09:24:18.260" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://mathoverflow.net/questions/80793/is-gauss-seidel-guaranteed-to-converge-on-semi-positive-definite-matrices&quot;&gt;Here is a proof&lt;/a&gt; that Gauss-Seidel fits your requirements, given that $b$ is in the image of $A$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same is very much not true of Jacobi; which is a shame since who wants to bother with Gauss-Seidel on modern computer hardware? If your problem can be split into diagonally-dominant blocks, you are in luck; you can apply Jacobi updates to those blocks in an incremental Gauss-Seidel fashion, and get the best of both for these type of semi-definite problems.&lt;/p&gt;&#xA;" OwnerUserId="28699" LastEditorUserId="20688" LastEditDate="2018-09-11T14:04:51.567" LastActivityDate="2018-09-11T14:04:51.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30180" PostTypeId="2" ParentId="30174" CreationDate="2018-09-11T12:40:25.957" Score="2" Body="&lt;p&gt;The problem you want to solve, with a small $\kappa$ is &quot;singularly perturbed&quot;, i.e., it generally has boundary and internal layers. The way you can think of it is that if $\kappa$ were zero, then you'd get $u=f$, and for $\kappa&amp;gt;0$, you get a smoothed version of $f$ with a smoothing length scale $L$ that is related to $\kappa$ in some way -- I believe that it is $L\propto\sqrt{\kappa}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works well if you have that the mesh size $h$ is on the order of or smaller than $L$, but if $h&amp;gt;L$, then you are effectively trying to resolve a discontinuous function. For these kinds of functions, you will get &lt;em&gt;Gibbs' phenomenon&lt;/em&gt; unless you use a scheme that either (i) is nonlinear or (ii) has sufficient artificial diffusion so that the length scale that results from the physical diffusion $\kappa$ plus the artificial diffusion is at least on the same order as the mesh size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both finite volume methods (which are basically piecewise constant &quot;discontinuous Galerkin&quot; schemes) and the addition of $\alpha h^2$ to $\kappa$ fall into the second class. An alternative is to use something like a nonlinear shock capturing method, though this makes the problem &lt;em&gt;substantially&lt;/em&gt; more complicated to solve.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-09-11T12:40:25.957" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30181" PostTypeId="1" CreationDate="2018-09-11T12:47:21.647" Score="3" ViewCount="1784" Body="&lt;p&gt;I have a nonlinear function &lt;code&gt;fun&lt;/code&gt; with &lt;em&gt;n&lt;/em&gt; inputs and &lt;em&gt;n&lt;/em&gt; outputs. I also have a function &lt;code&gt;jac&lt;/code&gt; which calculates the Jacobian, which is sparse. Which SciPy method, if any, can I use to find the root of &lt;code&gt;fun&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am surprised that I can't find a method in &lt;code&gt;scipy.optimize&lt;/code&gt; that takes a Jacobian in sparse matrix format. But I believe this can be simply solved by &lt;em&gt;the&lt;/em&gt; Newton's method, and so I am confused if such a method does not exist. There are many quasi-Newton methods in this package that estimate the Jacobian, but they do not seem quite right. The only thing that looks similar is &lt;code&gt;newton_krylov&lt;/code&gt; which documents something about sparse preconditioners. Is this what I should be looking at? If there is no such method, should I just do something like the following?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while not some_stopping_condition(x):&#xA;    x -= scipy.sparse.spsolve(jac(x), fun(x))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would prefer to use an existing SciPy function in case I do something stupid. Notably, as it happens, I believe my particular problem is poorly conditioned, and I think I would benefit from being able to use &lt;code&gt;scipy.sparse.lsqr&lt;/code&gt; or &lt;code&gt;lsmr&lt;/code&gt; which takes a damping parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Incidentally, the problem is solving large-displacement elasticity equations on a 2D membrane in 3D space with FEM. The problem is sparse because it is FEM. The problem is nonlinear because of the large displacements (and the corresponding Green-Lagrange strains). The poor conditioning comes from the fact that a small perturbation of any node in the direction perpendicular to the surface creates little change in strain energy (and the flatter the surface, the smaller the change).&lt;/p&gt;&#xA;" OwnerUserId="28292" LastEditorUserId="28292" LastEditDate="2018-09-11T13:49:52.430" LastActivityDate="2020-02-19T17:41:28.700" Title="Which SciPy nonlinear solver when Jacobian is analytically known and sparse?" Tags="|finite-element|nonlinear-equations|sparse-matrix|scipy|newton-method|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30183" PostTypeId="1" CreationDate="2018-09-11T18:53:44.033" Score="3" ViewCount="114" Body="&lt;p&gt;I'm interested in solving a differential equation in which I don't know the analytical form of the right hand side, I only know its numerical value for a finite set of values of the independent variable:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{dy}{dx}=f(x_i).$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't found a library that allows me to solve this problem and I want to make sure if there are any before trying to solve it myself, I figured one would have to interpolate the array to match the specific grid that the integrator uses, or fix the grid to the points $x_i$ for which I have the value $f(x_i)$.&lt;/p&gt;&#xA;" OwnerUserId="28595" LastEditorUserId="9667" LastEditDate="2018-09-12T02:40:37.960" LastActivityDate="2018-09-15T14:57:39.753" Title="Ordinary differenial equation with numerical right hand side" Tags="|ode|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30184" PostTypeId="2" ParentId="30157" CreationDate="2018-09-12T06:36:45.383" Score="0" Body="&lt;p&gt;$y = sin(x)$? Periodic functions should be difficult to approximate using polynomials, unless there is an option to restrict the polynomials to finite intervals.&lt;/p&gt;&#xA;" OwnerUserId="24653" LastEditorUserId="24653" LastEditDate="2018-09-12T09:09:59.497" LastActivityDate="2018-09-12T09:09:59.497" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30187" PostTypeId="1" CreationDate="2018-09-12T19:39:30.720" Score="2" ViewCount="121" Body="&lt;p&gt;I was doing my research/project using WENO as the limiter in finite volume methods to solve hyperbolic conservation law. I have no idea why the result in the characteristic wise method has a big error compared to component-wise. This is my current solution that I get from my solver, it shows that there is a numerical issue in characteristic wise reconstruction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Velocity in $x$ direction: solution of sod shock at $t=0.1$:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wEwRB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wEwRB.png&quot; alt=&quot;velocity in x direction: solution of sod shock at t=0.1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My algorithm of characteristic wise reconstruction (I am using 5th order WENO). Let we evaluate cell $i$:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using ROE averaging, get averaged value $ \bar q_{i+\frac{1}{2}} = f(q_{i},q_{i+1})$ and compute left and right eigenvector&lt;/li&gt;&#xA;&lt;li&gt;Transform variable $q$ to characteristic variable $v$ using left eigenvector, because 5 points is needed so we need $ q_{j} = [q_{i-2},q_{i-1},q_{i},q_{i+1},q_{i+2}]$, and then $v_j = L(\bar q_{i+\frac{1}{2}})*q_{j}$&lt;/li&gt;&#xA;&lt;li&gt;Perform WENO reconstruction&lt;/li&gt;&#xA;&lt;li&gt;Transform back to physical space using $ q^{\pm}_{i\mp\frac{1}{2}} = R(\bar q_{i+\frac{1}{2}})*v^{\pm}_{i\mp\frac{1}{2}} $, so we get $\ q^{+}_{i-\frac{1}{2}}$ and $q^{-}_{i+\frac{1}{2}}$&lt;/li&gt;&#xA;&lt;li&gt;While at cell interface interface $i+\frac{1}{2}:\ $ $q^{+}_{i+\frac{1}{2}}=q^{+}_{(i-\frac{1}{2})+ 1},\ q^{-}_{i+\frac{1}{2}}=q^{-}_{i+\frac{1}{2}}$.&lt;/li&gt;&#xA;&lt;li&gt;Compute flux $ F_{i+\frac{1}{2}} = f(q^{-}_{i+\frac{1}{2}},q^{+}_{i+\frac{1}{2}})$&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is there any inconsistency in my algorithm?&lt;/p&gt;&#xA;" OwnerUserId="28714" LastEditorUserId="20688" LastEditDate="2018-09-12T20:38:45.507" LastActivityDate="2018-09-12T20:38:45.507" Title="WENO methods: why the characteristic wise method resulting big errors?" Tags="|fluid-dynamics|finite-volume|hyperbolic-pde|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30188" PostTypeId="1" AcceptedAnswerId="30195" CreationDate="2018-09-12T20:57:05.323" Score="4" ViewCount="643" Body="&lt;p&gt;I am trying to attain the Moore-Penrose pseudoinverse of a very large, very sparse, rank-degenerate, singular, and square matrix. ($75000 \times 75000$, near rank). The matrix is a graph Laplacian and I need to find the &lt;a href=&quot;https://en.wikipedia.org/wiki/Resistance_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;resistance distance&lt;/a&gt; between a large number of nodes (LU determinates are too slow). I realize the inverse will be very dense, but I would like to persist it so I can query out distance metrics ad-hoc . I have been trying some brute approaches by converting it to a dense matrix (~25GB) on a large memory machine (124GB). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both &lt;code&gt;lstsq&lt;/code&gt; (&lt;code&gt;scipy.linalg.pinv&lt;/code&gt;) and some SVD approaches (&lt;code&gt;numpy.linalg.pinv&lt;/code&gt;) quickly exhaust the memory. The only approach that has worked is a slow but memory efficient SVD (&lt;code&gt;scipy.linalg.pinv2&lt;/code&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am exploring what other techniques are out there that can balance memory efficiency and speed, although I have yet to try &lt;code&gt;scipy.sparse.inv&lt;/code&gt; or &lt;code&gt;scipy.sparse.svds&lt;/code&gt; for a thorough comparison. I don't think any iterative approaches will work well because I can't provide a decent starting point. I think this leaves QR as a prime candidate. I have been exploring some of the methods listed in this &lt;a href=&quot;http://wang.hebmlc.org/UploadFiles/2017417194151138.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I want to sample selected entries of the MPI to calculate the&#xA;distance between a large number of graph nodes in a ad-hoc fashion.&#xA;Is there a better way to sample select elements of the MPI than&#xA;persisting the whole matrix?&lt;/li&gt;&#xA;&lt;li&gt;Are there any algorithms particularly well suited to my matrix type?&lt;/li&gt;&#xA;&lt;li&gt;Is QR factorization my best bet? Is there some other approaches I&#xA;should be considering?&lt;/li&gt;&#xA;&lt;li&gt;If QR, what memory efficient QR implementations exist? Or will I need to hand roll one from a paper?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28716" LastEditorUserId="28716" LastEditDate="2018-09-13T14:21:45.263" LastActivityDate="2018-09-13T14:54:41.733" Title="Moore-Penrose pseudoinverse of singular rank degenerate matrix" Tags="|linear-algebra|sparse-matrix|scipy|numpy|graph-theory|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30189" PostTypeId="2" ParentId="30188" CreationDate="2018-09-12T21:32:47.950" Score="4" Body="&lt;p&gt;75k x 75k double-precision entries is 45 gigabytes. That fits in memory, but barely; you need to be careful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linear algebra routines in most languages rely on Lapack as a backend, which is a highly optimized linear algebra library written in Fortran. Most Lapack routines operate in-place: for instance, its QR routine &lt;code&gt;xGEQRF&lt;/code&gt; does not allocate a second and a third matrix to store the results, but overwrites A with R, and then stores a compressed representation of Q in its lower triangular part (which would otherwise contain zeros). Typically, the &quot;user-friendly&quot; linear algebra packages like Numpy will make a copy of A, call &lt;code&gt;xGEQRF&lt;/code&gt;, and then reconstruct Q and R as full matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you have space in memory to store two matrices, but not three, so you have to avoid this. So &lt;code&gt;Q, R = numpy.linalg.qr(A)&lt;/code&gt; won't work, but you have (barely) the space to make a Lapack call for the QR factorization, copy R to another memory location, and compute its product in-place with the implicitly-represented Q that is still in memory (there's another Lapack routine for that, &lt;code&gt;xORMQR&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, computing the pseudoinverse with QR only works for full-rank matrices. If your matrix is rank-degenerate, you will have to work with the SVD, to which the same arguments apply. And, also, you will probably need to implement some form of regularization / truncation (for instance, truncated SVD, or Tikhonov, also known as ridge regression).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR: (1) don't use QR, use SVD (2) ditch the user-friendly libraries and use the Lapack interface directly.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2018-09-13T06:35:23.247" LastActivityDate="2018-09-13T06:35:23.247" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30190" PostTypeId="1" AcceptedAnswerId="30213" CreationDate="2018-09-12T22:13:36.030" Score="1" ViewCount="116" Body="&lt;p&gt;I am trying to solve a particular form of the Euler / Navier-Stokes equations in 1D, with very strong and non-linear diffusion coefficients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My system of equations is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;\begin{cases}&#xA;               \frac{\partial n}{\partial t} + \frac{\partial}{\partial x}\left(nv \right ) = S_n\\&#xA;               \frac{\partial \left( n v\right)}{\partial t} + \frac{\partial}{\partial x}\left(nv^2 + nT \right ) = S_M + \frac{\partial}{\partial x}\left(\nu(T)\frac{\partial v}{\partial x} \right)\\&#xA;               \frac{\partial}{\partial t}\left(\frac{3}{2}nT + \frac{1}{2}n v^2 \right ) + \frac{\partial}{\partial x}\left(\frac{5}{2}nTv + \frac{1}{2}n v^3 \right ) - \frac{\partial}{\partial x}\left(D(T)\frac{\partial T}{\partial x} \right) - \frac{\partial}{\partial x}\left(v\nu(T)\frac{\partial v}{\partial x} \right)  = S_E&#xA;\end{cases}&#xA;where $\nu(T)$ and $D(T)$ are &lt;em&gt;strongly&lt;/em&gt; nonlinear coefficients. For instance, in some versions of my model, &#xA;$\nu(T), D(T) \propto T^3$. This system is associated with a set of boundary conditions and initial conditions that I think are not essential to describe my issue. The temperature diffusion term ($D$) is by far (at least one order of magnitude) the fastest timescale in my system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried solving this using a purely time-explicit scheme, but this leads to very small time-steps (if the diffusion terms are removed from the system, I can use higher time-steps, so I guess they are the culprits). I also tried a fully implicit method (Crank-Nicholson or Backward-Euler, implemented with Petsc), but the non-linear Newton solve-step is quite slow. I would like to improve this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wondered if a scheme like the following could be more efficient:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Solve the advection part of the equation &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;using $Q = \begin{pmatrix}&#xA;n&#xA;\\ nv&#xA;\\ \frac{3}{2}nT + \frac{1}{2}nv^2&#xA;\end{pmatrix}$ and $F(Q) = \begin{pmatrix}&#xA;nv&#xA;\\ nv^2+nT&#xA;\\ \frac{5}{2}nTv + \frac{1}{2}nv^3&#xA;\end{pmatrix}$,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we solve $\frac{\partial Q}{\partial t} + \frac{\partial F}{\partial x} = S$ (where $S$ is a vector of the source terms). This is a quite &quot;classical&quot; problem, that can be solved explicitly (I've tried a FD-WENO3 that I was happy with). This gives us $Q^\star$, from which we can compute $n^\star$, $v^\star$ and $T^\star$.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;We &quot;push&quot; $v$ implicitly, assuming other variables are frozen. We write&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;$n^\star \frac{\partial v}{\partial t} = \frac{\partial}{\partial x}\left(\nu(T^\star)\frac{\partial v}{\partial x} \right)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is linear in $v$, so can be solved very effectively, and get $v^{\star\star}$&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;We &quot;push&quot; $T$ implicitly, assuming other variables are frozen.&#xA;$n^\star \frac{\partial T}{\partial t} = \frac{\partial}{\partial x}\left(D(T^\star)\frac{\partial T}{\partial x} \right) + \frac{\partial}{\partial x}\left(v^{\star\star}\nu(T^\star)\frac{\partial v^{\star\star}}{\partial x} \right) $.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;which again should not be complicated to solve. We get $T^{\star\star}$.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Solve again the explicit part using $n^\star, v^{\star\star}, T^{\star\star}$ as initial solutions. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is then the end of the iteration, and we have as a solution at $t=t+\Delta t$, using for steps 1, 2, 3, 4 the time-steps $\Delta t /2 $, $\Delta t$,$\Delta t$,$\Delta t /2$, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can guess, I am not an expert in numerical methods, and I have a hard time determining if this idea makes any sense. Is it consistent? I have the feeling that it is plain wrong. If it makes sense, is the order of the steps correct ? Otherwise, would you have any ideas of methods I could explore to solve this problem ? &lt;/p&gt;&#xA;" OwnerUserId="26815" LastEditorUserId="26815" LastEditDate="2018-09-15T11:26:15.483" LastActivityDate="2018-09-15T12:45:44.983" Title="Multi-steps method for Navier-stokes equations with strongly nonlinear diffusion" Tags="|pde|nonlinear-equations|simulation|numerical-modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30191" PostTypeId="1" AcceptedAnswerId="30200" CreationDate="2018-09-12T23:49:29.877" Score="0" ViewCount="161" Body="&lt;p&gt;I have a cube which is divided into 8 small cubes by bisecting each edge, I am trying to find out the surface area of each of the faces and the corresponding outward normals for them. This operation is done on a finite element mesh, so I have transformed the cube into the isoparametric form using shape(basis) functions and then tried to extract the area and normals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the part of the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; program polyhedron &#xA;&#xA;&#xA;DO INPT=1,8  !LOOP OVER 8 SMALL CUBES&#xA;&#xA;! GAUSS POINTS&#xA;XII(1,1) = MONE/THREE**HALF&#xA;XII(1,2) = MONE/THREE**HALF&#xA;XII(1,3) = MONE/THREE**HALF&#xA;XII(2,1) = ONE/THREE**HALF&#xA;XII(2,2) = MONE/THREE**HALF&#xA;XII(2,3) = MONE/THREE**HALF&#xA;XII(3,1) = ONE/THREE**HALF&#xA;XII(3,2) = ONE/THREE**HALF&#xA;XII(3,3) = MONE/THREE**HALF&#xA;XII(4,1) = MONE/THREE**HALF&#xA;XII(4,2) = ONE/THREE**HALF&#xA;XII(4,3) = MONE/THREE**HALF&#xA;XII(5,1) = MONE/THREE**HALF&#xA;XII(5,2) = MONE/THREE**HALF&#xA;XII(5,3) = ONE/THREE**HALF&#xA;XII(6,1) = ONE/THREE**HALF&#xA;XII(6,2) = MONE/THREE**HALF&#xA;XII(6,3) = ONE/THREE**HALF&#xA;XII(7,1) = ONE/THREE**HALF&#xA;XII(7,2) = ONE/THREE**HALF&#xA;XII(7,3) = ONE/THREE**HALF&#xA;XII(8,1) = MONE/THREE**HALF&#xA;XII(8,2) = ONE/THREE**HALF&#xA;XII(8,3) = ONE/THREE**HALF&#xA;&#xA;        XI(1) = XII(INPT,1)&#xA;        XI(2) = XII(INPT,2)&#xA;        XI(3) = XII(INPT,3) &#xA;&#xA;&#xA;DO I=1,8&#xA; DO J=1,3&#xA;    dNdxi(I,J) =  ZERO&#xA; END DO&#xA;END DO&#xA;&#xA;!HEXAHEDRAL SHAPE FUNCTION DERIVATIVES&#xA;dNdxi(1,1) =  MONE/EIGHT*(ONE-XI(2))*(ONE-XI(3))&#xA;dNdxi(1,2) =  MONE/EIGHT*(ONE-XI(1))*(ONE-XI(3))&#xA;dNdxi(1,3) =  MONE/EIGHT*(ONE-XI(1))*(ONE-XI(2))&#xA;dNdxi(2,1) =  ONE/EIGHT*(ONE-XI(2))*(ONE-XI(3))&#xA;dNdxi(2,2) =  MONE/EIGHT*(ONE+XI(1))*(ONE-XI(3))&#xA;dNdxi(2,3) =  MONE/EIGHT*(ONE+XI(1))*(ONE-XI(2))&#xA;dNdxi(3,1) =  ONE/EIGHT*(ONE+XI(2))*(ONE-XI(3))&#xA;dNdxi(3,2) =  ONE/EIGHT*(ONE+XI(1))*(ONE-XI(3))&#xA;dNdxi(3,3) =  MONE/EIGHT*(ONE+XI(1))*(ONE+XI(2))&#xA;dNdxi(4,1) =  MONE/EIGHT*(ONE+XI(2))*(ONE-XI(3))&#xA;dNdxi(4,2) =  ONE/EIGHT*(ONE-XI(1))*(ONE-XI(3))&#xA;dNdxi(4,3) =  MONE/EIGHT*(ONE-XI(1))*(ONE+XI(2))&#xA;dNdxi(5,1) =  MONE/EIGHT*(ONE-XI(2))*(ONE+XI(3))&#xA;dNdxi(5,2) =  MONE/EIGHT*(ONE-XI(1))*(ONE+XI(3))&#xA;dNdxi(5,3) =  ONE/EIGHT*(ONE-XI(1))*(ONE-XI(2))&#xA;dNdxi(6,1) =  ONE/EIGHT*(ONE-XI(2))*(ONE+XI(3))&#xA;dNdxi(6,2) =  MONE/EIGHT*(ONE+XI(1))*(ONE+XI(3))&#xA;dNdxi(6,3) =  ONE/EIGHT*(ONE+XI(1))*(ONE-XI(2))&#xA;dNdxi(7,1) =  ONE/EIGHT*(ONE+XI(2))*(ONE+XI(3))&#xA;dNdxi(7,2) =  ONE/EIGHT*(ONE+XI(1))*(ONE+XI(3))&#xA;dNdxi(7,3) =  ONE/EIGHT*(ONE+XI(1))*(ONE+XI(2))&#xA;dNdxi(8,1) =  MONE/EIGHT*(ONE+XI(2))*(ONE+XI(3))&#xA;dNdxi(8,2) =  ONE/EIGHT*(ONE-XI(1))*(ONE+XI(3))&#xA;dNdxi(8,3) =  ONE/EIGHT*(ONE-XI(1))*(ONE+XI(2))&#xA;&#xA;&#xA;&#xA;      dXdXi = zero&#xA;      dXdEta = zero&#xA;      dXdZeta = zero&#xA;      dYdXi = zero&#xA;      dYdEta = zero&#xA;      dYdZeta = zero&#xA;      dZdXi = zero&#xA;      dZdEta = zero&#xA;      dZdZeta = zero&#xA;&#xA;      do k=1,8&#xA;         dXdXi = dXdXi + dNdxi(k,1)*coords(1,k)&#xA;         dXdEta = dXdEta + dNdxi(k,2)*coords(1,k)&#xA;         dXdZeta = dXdZeta + dNdxi(k,3)*coords(1,k)&#xA;         dYdXi = dYdXi + dNdxi(k,1)*coords(2,k)&#xA;         dYdEta = dYdEta + dNdxi(k,2)*coords(2,k)&#xA;         dYdZeta = dYdZeta + dNdxi(k,3)*coords(2,k)&#xA;         dZdXi = dZdXi + dNdxi(k,1)*coords(3,k)&#xA;         dZdEta = dZdEta + dNdxi(k,2)*coords(3,k)&#xA;         dZdZeta = dZdZeta + dNdxi(k,3)*coords(3,k)&#xA;      enddo&#xA;&#xA;      DO face=1,6  !LOOP OVER ALL 6 FACES OF THE SMALL CUBE         &#xA;&#xA;      ! Jacobian of the mapping&#xA;      !&#xA;      if((face.eq.1).or.(face.eq.2)) then&#xA;         ! zeta = constant on this face&#xA;         dA = dsqrt((dYdXi*dZdEta - dYdEta*dZdXi)**two+ (dXdXi*dZdEta -&amp;amp;&#xA;         &amp;amp;dXdEta*dZdXi)**two+ (dXdXi*dYdEta - dXdEta*dYdXi)**two)&#xA;      elseif((face.eq.3).or.(face.eq.5)) then&#xA;         ! eta = constant on this face&#xA;         dA = dsqrt((dYdXi*dZdZeta - dYdZeta*dZdXi)**two+ (dXdXi*dZdZeta -&amp;amp;&#xA;         &amp;amp;dXdZeta*dZdXi)**two+ (dXdXi*dYdZeta - dXdZeta*dYdXi)**two)&#xA;      elseif((face.eq.4).or.(face.eq.6)) then&#xA;         ! xi = constant on this face&#xA;         dA = dsqrt((dYdEta*dZdZeta - dYdZeta*dZdEta)**two+ (dXdEta*dZdZeta -&amp;amp;&#xA;         dXdZeta*dZdEta)**two+ (dXdEta*dYdZeta - dXdZeta*dYdEta)**two)&#xA;&#xA;      endif&#xA;&#xA;  write(*,*) 'face',face,'area is dA', dA&#xA;&#xA;        if((face.eq.1).or.(face.eq.2)) then&#xA;         ! zeta = constant on this face&#xA;         normal(1,1) = dYdXi*dZdEta - dYdEta*dZdXi&#xA;         normal(2,1) = dXdXi*dZdEta - dXdEta*dZdXi&#xA;         normal(3,1) = dXdXi*dYdEta - dXdEta*dYdXi&#xA;         if(face.eq.1) normal = -normal&#xA;      elseif((face.eq.3).or.(face.eq.4)) then&#xA;         ! eta = constant on this face&#xA;         normal(1,1) = dYdXi*dZdZeta - dYdZeta*dZdXi&#xA;         normal(2,1) = dXdXi*dZdZeta - dXdZeta*dZdXi&#xA;         normal(3,1) = dXdXi*dYdZeta - dXdZeta*dYdXi&#xA;         if(face.eq.3) normal = -normal&#xA;      elseif((face.eq.5).or.(face.eq.6)) then&#xA;         ! xi = constant on this face&#xA;         normal(1,1) = dYdEta*dZdZeta - dYdZeta*dZdEta&#xA;         normal(2,1) = dXdEta*dZdZeta - dXdZeta*dZdEta&#xA;         normal(3,1) = dXdEta*dYdZeta - dXdZeta*dYdEta&#xA;         if(face.eq.6) normal = -normal&#xA;&#xA;      endif&#xA;      mag = dsqrt(normal(1,1)**two+normal(2,1)**two+normal(3,1)**two)&#xA;&#xA;      normal(1,1) = normal(1,1)/mag&#xA;      normal(2,1) = normal(2,1)/mag&#xA;      normal(3,1) = normal(3,1)/mag&#xA;&#xA;      write(*,*) 'face', face, 'normal is ',normal&#xA;&#xA;     END DO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The area of each surface of each small cube should be .25, for this case but this code it isn't returning so. The normals are also not correctly oriented, here the cube is aligned with the global axes so it should be either 1 or -1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: a. This might always not be a cube, it can be any irregular hexahedron, so the areas will not be equal always thus we need to compute each of them. b. The faces might be oriented along different directions, so the isoparametric transformation is sort of necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I have approched this using vectors of each face, without using the isoparametric transformation, where I use the diagonals of face of the cube and compute their cross product to find the area and outward unit normal, that approch also fails when the element is not regular. That part of the code looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IF (J==1) THEN&#xA;&#xA;        xsd1(1:4)=xsd(1:4)&#xA;        ysd1(1:4)=ysd(1:4)&#xA;        zsd1(1:4)=zsd(1:4)&#xA;        b(1)=xsd1(1)-xsd1(3)&#xA;        b(2)=ysd1(1)-ysd1(3)&#xA;        b(3)=zsd1(1)-zsd1(3)&#xA;        c(1)=xsd1(2)-xsd1(4)&#xA;        c(2)=ysd1(2)-ysd1(4)&#xA;        c(3)=zsd1(2)-zsd1(4)        &#xA;        call cross(c, b,crossproduct)&#xA;        norm(:)=crossproduct/(crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5&#xA;        area_isd=((crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5)/2        &#xA;        ELSE IF (J==2) THEN&#xA;&#xA;        xsd2(1:4)=xsd(5:8)&#xA;        ysd2(1:4)=ysd(5:8)&#xA;        zsd2(1:4)=zsd(5:8)&#xA;        b(1)=xsd2(4)-xsd2(2)&#xA;        b(2)=ysd2(4)-ysd2(2)&#xA;        b(3)=zsd2(4)-zsd2(2)&#xA;        c(1)=xsd2(3)-xsd2(1)&#xA;        c(2)=ysd2(3)-ysd2(1)&#xA;        c(3)=zsd2(3)-zsd2(1)&#xA;        call cross(c, b,crossproduct)   &#xA;        norm(:)=crossproduct/(crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5        &#xA;        area_isd=((crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5)/2            &#xA;        ELSE IF (J==3) THEN&#xA;&#xA;        xsd3(1:2)=xsd(1:2)&#xA;        xsd3(3)=xsd(6)&#xA;        xsd3(4)=xsd(5)&#xA;        ysd3(1:2)=ysd(1:2)&#xA;        ysd3(3)=ysd(6)&#xA;        ysd3(4)=ysd(5)      &#xA;        zsd3(1:2)=zsd(1:2)&#xA;        zsd3(3)=zsd(6)&#xA;        zsd3(4)=zsd(5)&#xA;        b(1)=xsd3(4)-xsd3(2)&#xA;        b(2)=ysd3(4)-ysd3(2)&#xA;        b(3)=zsd3(4)-zsd3(2)&#xA;        c(1)=xsd3(3)-xsd3(1)&#xA;        c(2)=ysd3(3)-ysd3(1)&#xA;        c(3)=zsd3(3)-zsd3(1)&#xA;        call cross(c, b,crossproduct)   &#xA;        norm(:)=crossproduct/(crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5        &#xA;        area_isd=((crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5)/2            &#xA;        ELSE IF (J==4) THEN&#xA;&#xA;        xsd4(1:2)=xsd(7:8)&#xA;        xsd4(3)=xsd(4)&#xA;        xsd4(4)=xsd(3)&#xA;        ysd4(1:2)=ysd(7:8)&#xA;        ysd4(3)=ysd(4)&#xA;        ysd4(4)=ysd(3)&#xA;        zsd4(1:2)=zsd(7:8)&#xA;        zsd4(3)=zsd(4)&#xA;        zsd4(4)=zsd(3)&#xA;        b(1)=xsd4(1)-xsd4(3)&#xA;        b(2)=ysd4(1)-ysd4(3)&#xA;        b(3)=zsd4(1)-zsd4(3)&#xA;        c(1)=xsd4(2)-xsd4(4)&#xA;        c(2)=ysd4(2)-ysd4(4)&#xA;        c(3)=zsd4(2)-zsd4(4)&#xA;        call cross(c, b,crossproduct)   &#xA;        norm(:)=crossproduct/(crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5        &#xA;        area_isd=((crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5)/2            &#xA;        ELSE IF (J==5) THEN&#xA;&#xA;        xsd5(1:2)=xsd(2:3)&#xA;        xsd5(3)=xsd(7)&#xA;        xsd5(4)=xsd(6)&#xA;        ysd5(1:2)=ysd(2:3)&#xA;        ysd5(3)=ysd(7)&#xA;        ysd5(4)=ysd(6)&#xA;        zsd5(1:2)=zsd(2:3)&#xA;        zsd5(3)=zsd(7)&#xA;        zsd5(4)=zsd(6)&#xA;        b(1)=xsd5(4)-xsd5(2)&#xA;        b(2)=ysd5(4)-ysd5(2)&#xA;        b(3)=zsd5(4)-zsd5(2)&#xA;        c(1)=xsd5(3)-xsd5(1)&#xA;        c(2)=ysd5(3)-ysd5(1)&#xA;        c(3)=zsd5(3)-zsd5(1)&#xA;        call cross(c, b,crossproduct)&#xA;        norm(:)=crossproduct/(crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5        &#xA;        area_isd=((crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5)/2        &#xA;        ELSE IF (J==6) THEN&#xA;&#xA;        xsd6(1)=xsd(1)&#xA;        xsd6(2)=xsd(4)&#xA;        xsd6(3)=xsd(8)&#xA;        xsd6(4)=xsd(5)&#xA;        ysd6(1)=ysd(1)&#xA;        ysd6(2)=ysd(4)&#xA;        ysd6(3)=ysd(8)&#xA;        ysd6(4)=ysd(5)&#xA;        zsd6(1)=zsd(1)&#xA;        zsd6(2)=zsd(4)&#xA;        zsd6(3)=zsd(8)&#xA;        zsd6(4)=zsd(5)&#xA;        b(1)=xsd6(3)-xsd6(1)&#xA;        b(2)=ysd6(3)-ysd6(1)&#xA;        b(3)=zsd6(3)-zsd6(1)&#xA;        c(1)=xsd6(4)-xsd6(2)&#xA;        c(2)=ysd6(4)-ysd6(2)&#xA;        c(3)=zsd6(4)-zsd6(2)&#xA;        call cross(c, b,crossproduct)   &#xA;        norm(:)=crossproduct/(crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5        &#xA;        area_isd=((crossproduct(1)**2+crossproduct(2)**2+crossproduct(3)**2)**0.5)/2            &#xA;        END IF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;J being the faces, I loop over the points in the respective faces and find cross product of the diagonals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know about what is wrong in both these approches and the way to correct it. I would be glad if some one could give me a way forward.&lt;/p&gt;&#xA;" OwnerUserId="22448" LastActivityDate="2018-09-14T02:21:36.160" Title="Compute outward normal and surface area for 8 noded brick element in FEA" Tags="|finite-element|computational-geometry|fortran|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30192" PostTypeId="1" CreationDate="2018-09-13T10:05:18.590" Score="1" ViewCount="448" Body="&lt;p&gt;I'm trying to get the normal modes of a system of springs and dasphots using &#xA;the basic dynamic equations for a linear, damped elastic structure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ M \ddot{u}(t) + C \dot{u}(t) + K u(t) = f(t) $&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to get the normal modes, the following QEP (quadratic eigenvalue problem) must be solved &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$ (\lambda^2M+\lambda C+K)\bar{\psi}$ = 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem can be solved by linearization, converting it in a generalized eigenvalue problem, that leads in general to complex eigenvectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I convert this complex eigenvectors to real ones, representing the real displacements of each mode of the structure?&#xA;Is there any library that implements this functionality?&lt;/p&gt;&#xA;" OwnerUserId="4458" LastActivityDate="2018-09-17T10:59:35.560" Title="Computing real normal modes from complex eigenvectors" Tags="|eigenvalues|complex-arithmetic|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30194" PostTypeId="2" ParentId="30183" CreationDate="2018-09-13T12:59:27.003" Score="5" Body="&lt;p&gt;Because the r.h.s. is independent of $y$, your case is already handled as a special case by some standard interpolation libraries. Here's an example of how to do with cubic splines in scipy (&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.CubicSpline.html&quot; rel=&quot;noreferrer&quot;&gt;docs link&lt;/a&gt;), I picked a nice function $(1+x^2)^{-1}$ just to illustrate:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x = array([0.0] + sorted(np.random.rand(16)) + [1.0])&#xA;y = 1/(1+x*x)&#xA;f = CubicSpline(x, y)&#xA;u = f.antiderivative()&#xA;&#xA;In [34]: u(0.5) - np.math.atan(0.5)&#xA;Out[34]: -5.874574474096228e-07&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So first you interpolate the r.h.s. function $f$, then its antiderivative $u$ is computed exactly for that interpolant, solving $u'(x)=f(x)$. It's a reasonably common basic feature for such a library.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-09-13T12:59:27.003" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30195" PostTypeId="2" ParentId="30188" CreationDate="2018-09-13T13:38:03.057" Score="2" Body="&lt;p&gt;Although I'm unfamiliar with &quot;distance metrics&quot;, there's no shortage of information about graph laplacians on the web because of their practical applicability to matrix reordering and classification/clustering problems. The nullspace of the undirected graph laplacian is well known (a vector of 1's, or one such vector per connected component if the graph is not strongly connected). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the structure of the laplacian is pretty well known, you might not need to resort to brute force techniques like computing a dense pseudoinverse. There are cheaper/faster tools [sparse cholesky, lanczos, lobpcg] that can interrogate these matrices in other ways [solve by it, extract the Fiedler vector or other important eigenvectors, sample selected entries of the inverse]. Maybe they (or something similar) could help? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you write a bit more about &quot;distance metrics&quot;, and how the pseudoinverse helps you find them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT Given your broader explanation, here's what I would try: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of seeking entries the pseudoinverse $\mathbf \Gamma$ of your (singular) Laplacian $\mathbf L$, consider the exact inverse $\mathbf \Gamma'$ of a nearby (nonsingular) matrix, $\mathbf L' = \mathbf L + \sigma \mathbf I$, where $\sigma$ is some small shift parameter (say, $1\times10^{-8}$). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would expect (hope?) that $\mathbf \Gamma'_{i,j}$ is very close to $\mathbf \Gamma_{i,j}$, but the former is vastly easier to compute, because you can just use sparse cholesky decomposition to represent $\mathbf \Gamma' = \left( \mathbf L + \sigma \mathbf I\right)^{-1}$, and then $\mathbf \Gamma'_{ij} = \mathbf e_i \left( \mathbf L + \sigma \mathbf I\right)^{-1} \mathbf e_j$. That is, direct backsolve by a vector containing a single 1 at entry $j$, then extract the $i$'th entry of the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be pretty easy to code up just in Matlab. On a small problem (N=5K?) you could compare the approximate $\mathbf \Gamma'_{i,j}$ to the exact ($\sigma=0$) $\mathbf \Gamma_{i,j}$ as computed via the pseudoinverse. If they're pretty close, this might be all you need. (I expect they'd differ by about $\sigma \cdot \kappa$, with $\kappa$ the condition number).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-09-13T14:54:41.733" LastActivityDate="2018-09-13T14:54:41.733" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30196" PostTypeId="1" AcceptedAnswerId="30197" CreationDate="2018-09-13T14:59:56.323" Score="0" ViewCount="124" Body="&lt;p&gt;I have just started with PETSc hence it might seem like a very stupid question but I couldn't find any answer in manual. &#xA;After Calling KSPSolve, where can I access the soluition for my linear system?&#xA;Example  for a matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left(\begin{array}{ccc}&#xA;2 &amp;amp;-1 &amp;amp; 0\\&#xA;-1 &amp;amp;2&amp;amp;-1\\&#xA;0&amp;amp;-1&amp;amp;2&#xA;\end{array}\right)&#xA;\left(\begin{array}{c}&#xA;x\\y\\z&#xA;\end{array}\right)=&#xA;\left(\begin{array}{c}&#xA;1\\0\\1&#xA;\end{array}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to get vector &#xA;$$&#xA;\left(\begin{array}{c}&#xA;3/4\\1/2\\1/4&#xA;\end{array}\right)&#xA;$$ &#xA;back. &#xA;The documentation for KSP Solve says &lt;code&gt;x - the solution (this may be the same vector as b, then b will be overwritten with answer)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I keep getting &lt;code&gt;x&lt;/code&gt; vector as &#xA;$$&#xA;\left(\begin{array}{c}&#xA;1\\1\\1&#xA;\end{array}\right)&#xA;$$ &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something wrong in my understanding?&lt;/p&gt;&#xA;" OwnerUserId="7333" LastEditorUserId="20688" LastEditDate="2018-09-13T16:09:36.407" LastActivityDate="2018-09-13T16:09:36.407" Title="How to access solution to linear system in PETSc?" Tags="|linear-solver|petsc|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30197" PostTypeId="2" ParentId="30196" CreationDate="2018-09-13T16:07:44.957" Score="0" Body="&lt;p&gt;&lt;strong&gt;Before&lt;/strong&gt; you solve the system in PETSc you store the matrix $A$ and RHS $b$.&#xA;The message from PETSc says, that &lt;strong&gt;after&lt;/strong&gt; the system is solved, the solution might be overwritten into $b$, where originally the RHS was stored. That's what it means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the original system of equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\left(\begin{array}{ccc}&#xA;2 &amp;amp;-1 &amp;amp; 0\\&#xA;-1 &amp;amp;2&amp;amp;-1\\&#xA;0&amp;amp;-1&amp;amp;2&#xA;\end{array}\right)&#xA;\left(\begin{array}{c}&#xA;x\\y\\z&#xA;\end{array}\right)=&#xA;\left(\begin{array}{c}&#xA;1\\0\\1&#xA;\end{array}\right)&#xA;$$&#xA;the correct solution is indeed (as pointed out by @JedBrown)&#xA;$$&#xA;\left(\begin{array}{c}&#xA;1\\1\\1&#xA;\end{array}\right)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you intend (for whatever good reason) to get &#xA;$$&#xA;\left(\begin{array}{c}&#xA;3/4\\1/2\\1/4&#xA;\end{array}\right)&#xA;$$&#xA;that only means that you have a typo in either your matrix or RHS. Assuming that your matrix is correct, you will get your desired solution for the following system (changed RHS slightly):&#xA;$$&#xA;\left(\begin{array}{ccc}&#xA;2 &amp;amp;-1 &amp;amp; 0\\&#xA;-1 &amp;amp;2&amp;amp;-1\\&#xA;0&amp;amp;-1&amp;amp;2&#xA;\end{array}\right)&#xA;\left(\begin{array}{c}&#xA;x\\y\\z&#xA;\end{array}\right)=&#xA;\left(\begin{array}{c}&#xA;1\\\color{red}{1/2}\\1&#xA;\end{array}\right)&#xA;$$&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-09-13T16:07:44.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30200" PostTypeId="2" ParentId="30191" CreationDate="2018-09-13T22:40:34.857" Score="2" Body="&lt;p&gt;To debug situations like this, it's easiest if you break up the problem into several parts.  Since you don't really need an isoparametric map to compute areas and normals, I'll ignore that aspect of your approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 1:&lt;/strong&gt;&#xA;Find the orientation of the nodes on the face.  They should follow a right-hand rule (counterclockwise sequence).  You can use the approach in &lt;a href=&quot;https://en.wikipedia.org/wiki/Curve_orientation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Curve_orientation&lt;/a&gt; (with the appropriate projection on to the xy-, yz-, or zx-planes).  For example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function isCounterClockwise(x_coords, y_coords, z_coords) &#xA;  real   :: isCounterClockwise&#xA;  real   :: x_coords(4)&#xA;  real   :: y_coords(4)&#xA;  real   :: z_coords(4)&#xA;&#xA;  isCounterClockwise = ....&#xA;  return&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 2:&lt;/strong&gt; If the points are not counter-clockwise, reverse the order.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine reverseOrder(x_coords, y_coords, z_coords)&#xA; ....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step 3:&lt;/strong&gt; Compute cross product&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subroutine computeAreaAndNormal(x_coords, y_coords, z_coords, area, normal)&#xA;&#xA;  u1 = x_coords(2) - x_coords(1)&#xA;  u2 = y_coords(2) - y_coords(1)&#xA;  u3 = z_coords(2) - z_coords(1)&#xA;&#xA;  v1 = x_coords(4) - x_coords(1)&#xA;  v2 = y_coords(4) - y_coords(1)&#xA;  v3 = z_coords(4) - z_coords(1)&#xA;&#xA;  normal = crossProduct(u1, u2, u3, v1, v2, v3)&#xA;  area = sqrt(normal(1)**2 + normal(2)**2 + normal(3)**2)&#xA;  normal = normal/area&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you write your code this way it will be much easier to debug.  &lt;strong&gt;Caveat:&lt;/strong&gt; Note that the above is not compileable Fortran.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For arbitrary planar quadrilaterals with nodes (1,2,3,4) arranged clockwise, find the areas of the triangles (1,2,3) and (1,3,4) (half of the norm of the cross product) and add them.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastEditorUserId="7626" LastEditDate="2018-09-14T02:21:36.160" LastActivityDate="2018-09-14T02:21:36.160" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30201" PostTypeId="1" CreationDate="2018-09-14T05:46:39.843" Score="0" ViewCount="137" Body="&lt;p&gt;If $u_1$ and $u_2$ are solutions of (weak-form) Laplace equation on a connected domain $\Omega$, with Dirichlet boundary values $u_{\partial\Omega, 1}$ and $u_{\partial\Omega, 2}$, respectively. If $$u_{\partial\Omega,1} \neq u_{\partial\Omega, 2},$$ we know $$\|u_1-u_2\|_{L^2(\Omega)} \neq 0.$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, for any open set $\tilde{\Omega} \subset \Omega$, is $$\|u_1-u_2\|_{L^2(\tilde{\Omega})} \neq 0 $$ always true?&lt;/p&gt;&#xA;" OwnerUserId="22780" LastEditorUserId="22780" LastEditDate="2018-09-15T00:09:57.843" LastActivityDate="2020-01-20T06:01:55.227" Title="seminorm of solutions of Laplace equation" Tags="|finite-element|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="30202" PostTypeId="1" CreationDate="2018-09-14T06:29:13.907" Score="3" ViewCount="437" Body="&lt;p&gt;What is the difference between the Dishonest Newton method and the Very Dishonest Newton method? Is there a difference or do they mean the same thing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried searching for this on the internet but I can't find anything. I understand that in the ordinary (&quot;honest&quot;) Newton method, the Jacobian is computed in each iteration.&#xA;If we reuse the same Jacobian for several iterations, this is referred to as the Dishonest Newton method (DN) by some sources and as the Very Dishonest Newton method (VDN) by others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I would think that the &lt;em&gt;Very&lt;/em&gt; Dishonest Newton method reuses the same Jacobian for more iterations than Dishonest Newton. Maybe in VDN the Jacobian is only computed once? But I have not been able to verify this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would prefer some credible sources with clear descriptions of both methods as I want to refer to them in my master's thesis. &lt;/p&gt;&#xA;" OwnerUserId="28728" LastActivityDate="2018-09-14T06:29:13.907" Title="Difference between Dishonest Newton method and Very Dishonest Newton method" Tags="|optimization|newton-method|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30203" PostTypeId="1" AcceptedAnswerId="30207" CreationDate="2018-09-14T14:47:12.170" Score="14" ViewCount="4753" Body="&lt;p&gt;Suppose I know the expected sparsity of a matrix (i.e. the number of non-zeros / total possible number of non-zeros). Is there a rule of thumb (perhaps approximate) for deciding whether to use sparse matrix storage (specifically, compressed row storage) vs. storing it as a dense matrix?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Speed is more important in my application than memory. But out of general curiosity, I'm interested in answers from both a speed and memory perspective.&lt;/li&gt;&#xA;&lt;li&gt;After generating the matrix, I only apply addition and multiplication operations on it.&lt;/li&gt;&#xA;&lt;li&gt;I have only been able to find qualitative answers, e.g. &lt;a href=&quot;https://math.stackexchange.com/questions/1643428/sparse-matrix-or-dense-matrix?rq=1&quot;&gt;this question&lt;/a&gt; and &lt;a href=&quot;https://scicomp.stackexchange.com/questions/29888/how-sparse-should-a-sparse-matrix-be-to-see-benefits?rq=1&quot;&gt;this question&lt;/a&gt; but I'm looking for something like &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...if the sparsity is more than approximately $x\%$, then use dense storage.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="25809" LastEditorUserId="20688" LastEditDate="2018-09-14T15:05:00.287" LastActivityDate="2020-07-28T04:55:35.807" Title="Rule of thumb for sparse vs dense matrix storage" Tags="|matrix|sparse-matrix|dense-matrix|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30204" PostTypeId="1" AcceptedAnswerId="30216" CreationDate="2018-09-14T17:55:49.480" Score="0" ViewCount="187" Body="&lt;p&gt;The goal is to display the relation between K and E in the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\cos(K) = 5 \text{sinc}( 5.12\sqrt{E} ) - \cos(5.12\sqrt{E} )$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The intended result is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Lzsud.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lzsud.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, using the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = linspace(0, 3*pi, 99);  % 0 &amp;lt; k &amp;lt; 3pi , crystal momentum.&#xA;E = linspace(0.1,  3.5, 99);% Energy.&#xA;[X, Y] = meshgrid(k, E);&#xA;&#xA;P = 5;                      % 2mU_0b/hbar^2.&#xA;Z = cos(X)  - ( P * sinc( 5.12 * sqrt(Y) ) )  - cos( 5.12 * sqrt(Y) );&#xA;&#xA;contour(X, Y, Z);&#xA;&#xA;xlabel('k')&#xA;ylabel('E(k)')&#xA;legend('cos(K) - P * sin(E) - cos(E)')&#xA;title('Dispersion relation for periodic potential in extended zone.')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;what I currently get unfortunately bears no resemblance at all:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8nqIJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8nqIJ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just don't see how to separate the graph into three (discontinuous) regions, $K = [0, \pi], [\pi, 2\pi], [2\pi, 3\pi]$, and what value of $E$ should be used for each of these regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to plot the equation the same way as in the first graph?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The first plot is from &lt;a href=&quot;https://mathematica.stackexchange.com/questions/20624/plotting-extended-zone-scheme&quot;&gt;here&lt;/a&gt;, where they do some modular arithmetic trick by adding $i*floor(k ./ \pi)$ on the l.h.s. and $i*floor(5.12*\sqrt{E} ./ \pi)$ on the r.h.s. There is something similar &lt;a href=&quot;https://mathematica.stackexchange.com/questions/147149/plotting-extended-zone-scheme-plot-3pi-to-3pi?noredirect=1&amp;amp;lq=1&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24654" LastEditorUserId="24654" LastEditDate="2018-09-15T09:31:17.290" LastActivityDate="2018-09-16T18:09:08.480" Title="Plot a multi-variable equation in (discontinuous) regions" Tags="|matlab|plotting|octave|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30205" PostTypeId="2" ParentId="30204" CreationDate="2018-09-14T21:46:14.367" Score="0" Body="&lt;p&gt;Here is something near to the intended goal:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BleYc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BleYc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the code is ugly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = linspace(0,  3*pi, 100);   &#xA;&#xA;hbar = 1;&#xA;m = 1;&#xA;E =  ( (hbar^2) .* (k.^2) ) ./  (2*m);&#xA;&#xA;E(33:66) = E(33:66) + 3;&#xA;E(66:100) = E(66:100) + 6;&#xA;&#xA;E(33) = E(32);&#xA;E(66) = E(65);&#xA;&#xA;E(34) = NaN;&#xA;E(67) = NaN;&#xA;&#xA;hold on&#xA;plot(k,  E, [pi pi], [0 60], '--k', [2*pi 2*pi], [0 60], '--k');&#xA;plot(k(34), E(33), 'ob', 'MarkerFaceColor','b',...&#xA;        k(67), E(66), 'ob',  'MarkerFaceColor', 'b',...&#xA;        k(34), E(35), 'ob', 'MarkerFaceColor', 'b',...&#xA;        k(67), E(68), 'ob' , 'MarkerFaceColor', 'b');&#xA;hold off&#xA;&#xA;xlabel('k')&#xA;ylabel('E(k)')&#xA;legend('E = hbar^2 k^2 / 2m', 'x = pi', 'x = 2pi')&#xA;title('Dispersion relation for periodic potential in extended zone.')&#xA;&#xA;txt = '&amp;lt;- Band Gap';&#xA;text(k(35), E(33) +2, txt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;" OwnerUserId="24654" LastActivityDate="2018-09-14T21:46:14.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30206" PostTypeId="2" ParentId="30165" CreationDate="2018-09-14T21:52:36.230" Score="4" Body="&lt;p&gt;Short answer: For a close-to-optimal representation, instead of Coulomb-wave functions, use spherical harmonics for the angular part and a grid-like representation in radial direction. (&quot;grid-like&quot; means finite-differences, collocation methods, discrete-variable-representations, or similar methods).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Long answer: The usual approach to solving the Hartree-Fock equations is implemented in terms of the electron integrals, i.e. the matrix representation of the single-particle Hamiltonian $h_{ij}$ and the two-particle Coulomb-interaction $g_{ijkl}$ (all indices $i,j,k,l$ run from $1$ up to the sp-basis size $N_b$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, the Coulomb-interaction tensor is required to calculate the Fock (or mean-field) matrix and thus determines a good part of the total run-time of your computation (together with the linear algebra task of solving the eigenvalue problem). It is therefore advantageous to choose a basis that leads to an as sparse representation of the two-electron integrals as possible. (In this regard, nothing beats a grid-like basis which reduces the size of the two-electron integrals to $N_b^2$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, one wants a basis that well represents the problem. For atoms, and particular for ground-state calculations, the Hamiltonian has spherical symmetry and therefore spherical harmonics are ideal. The sp-basis is represented as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\phi_{klm} = \sum_{klm} R_{klm}(r) Y_{lm}(\theta,\varphi) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the index $i$ becomes a multi-index $(klm)$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This leaves the radial part $R_{klm}(r)$ to determine. Coulomb functions make a specific choice here (that only depends on $kl$ but not on $m$). As you said, this leads to orthogonal basis functions, which is ok, but for the crucial two-particle integrals, nothing is gained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A grid like representation for $R_{klm}(r)$ does much better here, as it leads to two-electron integrals which depend only quadratically on the size of the grid (instead of quartically as for the index $k$ in the Coulomb wave functions). This is important, as  for ground-state calculations, the angular basis can be kept quite small, so the radial part dominates. Moreover, for large grids, as they're required for time-dependent calculations, this gives a huge advantage. Moreover, the flexibility to freely adjust the grid can be used to obtain a higher accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, this is basically the answer to the question what would be best. Anyways, one usually starts by implementing the general case, and once you're there, just go on and use the Coulomb basis. But then, as a next step, maybe consider to go on to grid-like representations. It's a nice example of how the single-particle basis affects the many-particle solution method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, a good reference for Hartree-Fock calculations (and also their implementation) is the book Molecular Electronic Structure Theory by Helgaker et al. Have a look.&lt;/p&gt;&#xA;" OwnerUserId="14942" LastEditorUserId="14942" LastEditDate="2018-09-15T09:26:39.793" LastActivityDate="2018-09-15T09:26:39.793" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30207" PostTypeId="2" ParentId="30203" CreationDate="2018-09-14T23:14:05.090" Score="15" Body="&lt;p&gt;All matrix operations are memory bound (and not compute bound) on today's processors. So basically, you have to ask which format stores fewer bytes. This is easy to compute:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For a full matrix, you store 8 bytes (one double) per entry&lt;/li&gt;&#xA;&lt;li&gt;For a sparse matrix, you store 12 bytes per entry (one double for the value, and one integer for the column index of the entry).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In other words, if your sparsity is below 67% -- i.e., for nearly any matrix any reasonable person would call sparse --, the sparse matrix format will not only yield better memory use but also better compute time.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-09-14T23:14:05.090" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="30208" PostTypeId="1" AcceptedAnswerId="30209" CreationDate="2018-09-12T21:43:26.390" Score="1" ViewCount="1369" Body="&lt;p&gt;As a simple exercise for a class in computational methods for physics, we've learned how to implement the RK4 (Runge-Kutta 4th-order) algorithm for a very simple exponential decay. E.g. the function y(t) is governed by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/qX8YQ.png&quot; alt=&quot;y and y dot definitions&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the RK4 definition from most sources, and specifically in this case Wikipedia, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/h8h27.png&quot; alt=&quot;wikipedia rk4 values&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;determining the 4 values &lt;em&gt;k&lt;/em&gt;₁, &lt;em&gt;k&lt;/em&gt;₂, &lt;em&gt;k&lt;/em&gt;₃, and &lt;em&gt;k&lt;/em&gt;₄ all depend on the function &lt;em&gt;f(t,y)&lt;/em&gt; for different increments of &lt;em&gt;t&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/knT6s.png&quot; alt=&quot;y dot = f(t, y)&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since I don't know &lt;em&gt;y(t)&lt;/em&gt; for arbitrary times (that's what the RK4 approximation is trying to calculate), I can't easily get &lt;em&gt;f(t+h/2, y+k₁/2)&lt;/em&gt; like in the Wikipedia notation for &lt;em&gt;k&lt;/em&gt;₂.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: When the time dependence of &lt;em&gt;f(t, y)&lt;/em&gt; is not known in an exact form, would it be possible to just use Euler’s method of approximation for &lt;em&gt;f(t, y)&lt;/em&gt; for &lt;strong&gt;all&lt;/strong&gt; of the &lt;em&gt;k&lt;sub&gt;n&lt;/sub&gt;&lt;/em&gt; values instead of just &lt;em&gt;k₁&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerDisplayName="WindowsNT" LastActivityDate="2018-09-14T23:40:50.020" Title="Runge-Kutta when time dependence not known" Tags="|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30209" PostTypeId="2" ParentId="30208" CreationDate="2018-09-12T22:14:17.913" Score="3" Body="&lt;p&gt;That's exactly what Runge-Kutta does: You have a differential equation y' = f (y, t), and you calculate those values and plug them into the Runge-Kutta formula. Of course you don't know y (t) exactly, that's what the formula produces for you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You say &quot;I can't easily get f(t+h/2, y+k1/2) like in the wikipedia notation for k2&quot;. Of course you can. You calculate t + h/2, y + k1/2 and use the results. That's what the formula says. It doesn't say &quot;use y(t)&quot;. It says &quot;use y + k1/2&quot;. You are totally overthinking this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. Keep your fingers away from the Euler approximation. The coefficients in the Runge-Kutta formula are very, very cleverly calculated to give you the best possible results. If you mess with it by trying to somehow put Euler approximations in there, you just mess it up and totally destroy your precision. &lt;/p&gt;&#xA;" OwnerUserId="15239" OwnerDisplayName="gnasher729" LastActivityDate="2018-09-12T22:14:17.913" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30210" PostTypeId="1" AcceptedAnswerId="30212" CreationDate="2018-09-15T00:04:36.117" Score="1" ViewCount="32" Body="&lt;p&gt;If I just need a chain of cartesian coordinates (as parameters/dimensions) that photon travels one after another until it reaches to a stop (hitting a wall or something), can GA minimize its path to get true ray tracing, if objective function returns total time of travel and whole 3D space is uniformly divided into nodes that a photon can travel?&lt;/p&gt;&#xA;" OwnerUserId="2014" LastActivityDate="2018-09-15T07:29:40.587" Title="Can a genetic algorithm solve shortest path of light (photons/waves) through different mediums (diamond, glass, water, air) to do raytracing?" Tags="|genetic-algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30211" PostTypeId="2" ParentId="30203" CreationDate="2018-09-15T02:55:41.147" Score="19" Body="&lt;p&gt;For what it is worth, for random sparse matrices of size 10,000 by 10,000 vs. dense matrices of the same size, on my Xeon workstation using MATLAB and Intel MKL as the BLAS, the sparse matrix-vector multiply was faster for densities of 15% or less. At 67% (as proposed by &lt;a href=&quot;https://scicomp.stackexchange.com/a/30207/20688&quot;&gt;another answer&lt;/a&gt;), the dense matrix-vector multiplication was about three time faster.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="20688" LastEditDate="2018-09-15T03:49:34.230" LastActivityDate="2018-09-15T03:49:34.230" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30212" PostTypeId="2" ParentId="30210" CreationDate="2018-09-15T07:29:40.587" Score="2" Body="&lt;p&gt;Genetic algorithm is an optimization technique, which can be used to find minima of an N-dimensional function. However, this function needs to be more or less smooth and “nice”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the paths can diverge very fast (after a few reflections or refractions) even if their initial conditions (i.e. position and direction) are almost the same, hence the fitness function would rather be like a random function at least in case of certain inputs. For other inputs, it could work, but it seems to be a quite inefficient way of ray path calculations.&lt;/p&gt;&#xA;" OwnerUserId="20475" LastActivityDate="2018-09-15T07:29:40.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30213" PostTypeId="2" ParentId="30190" CreationDate="2018-09-15T12:45:44.983" Score="1" Body="&lt;p&gt;If your diffusion is strong, then it is a strongly elliptic problem and I would have thought this should be easier to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can try operator splitting approach which is basically what you are trying to describe. To get second order accuracy, you can use Strang splitting. If you search for &quot;strang splitting&quot; on this forum you will find many posts. The idea is simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_t = A(u) + B(u)&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is solved in three steps. Let $E(A,\Delta t)$ and $E(B,\Delta t)$ denote the second order accurate numerical schemes to advance the split equations $u_t=A(u)$ and $u_t = B(u)$ by time step $\Delta t$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let $u^n$ be current solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve $u_t = A(u)$ for half time step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_1 = E(A, \Delta t/2) u^n&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve $u_t = B(u)$ for full time step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u_2 = E(B, \Delta t) u_1&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve $u_t = A(u)$ for half time step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;u^{n+1} = E(A, \Delta t/2) u_2&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where you now have the solution at next time step.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-09-15T12:45:44.983" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30214" PostTypeId="2" ParentId="30192" CreationDate="2018-09-15T20:31:19.097" Score="1" Body="&lt;p&gt;When you find the eigenvalues $\lambda_n$ and corresponding eigenvectors $u_n$, some of them might be pairs of complex-conjugated eigenvectors $u_n=v_n\pm iw_n$ that correspond to complex-conjugated eigenvalues $\lambda_n=a_n\pm ib_n$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the modes associated with an eigenvalue and its eigenvector can be found as &#xA;$$x_n(t)=e^{\lambda_n t}\,u_n$$&#xA;Now for the case of complex-conjugated pairs of eigenvalues and vectors, choose only one pair of them and substitute them in the above relation. For instance, let's take &#xA;$$\lambda_n=a_n+ib_n,$$&#xA;$$u_n=v_n+iw_n$$&#xA;Notice that you could just as well choose the eigenvalue and vector with the &quot;$-$&quot; instead of the &quot;$+$&quot;. Then plugging them in the relation for the mode $x_n(t)$ and simplifying (use Euler's formula for complex numbers and do some algebra) you find that &#xA;$$x_n(t)=e^{(a_n+ib_n)t}\,(v_n+iw_n)\Rightarrow$$&#xA;$$\Rightarrow x_n(t)=e^{a_nt}[\cos(b_nt)v_n-\sin(b_nt)w_n]+ie^{a_nt}[\sin(b_nt)v_n+\cos(b_nt)w_n]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way, you've decomposed the mode into its real and imaginary parts. For a linear eigenvalue problem, both the real and the imaginary part are each real solutions of the problem; so you find two real solutions that represent real displacement modes of your structure. You should redo the simplifying in the above relation, just to be sure I haven't missed anything there!  &lt;/p&gt;&#xA;" OwnerUserId="28737" LastEditorUserId="28737" LastEditDate="2018-09-17T10:59:35.560" LastActivityDate="2018-09-17T10:59:35.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30216" PostTypeId="2" ParentId="30204" CreationDate="2018-09-16T18:09:08.480" Score="3" Body="&lt;p&gt;First, you originally wrote that your equation is $\cos(K)=5 \text{ sinc}(5.12\sqrt{E})- \cos(5.12\sqrt{E})$, but you clearly meant $\cos(K)=5 \text{ sinc}(5.12\sqrt{E}) + \cos(5.12\sqrt{E})$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, the $\text{sinc}()$ function has differing conventions. &lt;a href=&quot;https://mathematica.stackexchange.com/questions/20624/plotting-extended-zone-scheme&quot;&gt;The other question&lt;/a&gt; is based on Mathematica, which uses the &lt;a href=&quot;https://reference.wolfram.com/language/ref/Sinc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;convention&lt;/a&gt; $\text{sinc}(x) \equiv \frac{\sin x}{x}$, while Matlab/Octave use the &lt;a href=&quot;https://octave.sourceforge.io/octave/function/sinc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;convention&lt;/a&gt; $\text{sinc}(x) \equiv \frac{\sin (\pi x)}{\pi x}$. (To be precise of course we need to separately define $\text{sinc}(0)\equiv 1$.) This is why your first contour plot shows &quot;no resemblance at all&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compensating for these differing conventions and altering your code slightly so that it only shows the contours for Z=0 (as suggested in a comment by Biswajit Banerjee) yields&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = linspace(0, 3*pi, 99);  % 0 &amp;lt; k &amp;lt; 3pi , crystal momentum.&#xA;E = linspace(0.1,  3.5, 99);% Energy.&#xA;[X, Y] = meshgrid(k, E);&#xA;&#xA;P = 5;                      % 2mU_0b/hbar^2.&#xA;Z = cos(X)  - ( P * pi * sinc( 5.12 * sqrt(Y)/pi ) )  - cos( 5.12 * sqrt(Y) );&#xA;contour(X, Y, Z,[0,0],'linewidth',2,'linecolor','k');&#xA;for i=1:3&#xA;  line(i*[pi,pi],[0,3.5],'color','k','linestyle','--')&#xA;end&#xA;&#xA;xlabel('k')&#xA;ylabel('E(k)')&#xA;legend('cos(K) - P * sin(E) - cos(E)')&#xA;title('Dispersion relation for periodic potential in extended zone.')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which produces the plot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lczLi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lczLi.jpg&quot; alt=&quot;contour plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way I could come up with to isolate sections of these curves is to simply run &lt;code&gt;contour&lt;/code&gt; for the separate regions and plot them together. The following code accomplishes this and generalizes it to &lt;code&gt;N&lt;/code&gt; sections. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P = 5;                      % 2mU_0b/hbar^2.&#xA;a=5.12;&#xA;N=3;&#xA;&#xA;hold on;&#xA;&#xA;&#xA;for i=1:N&#xA;    k = linspace((i-1)*pi, i*pi, 1000);  % 0 &amp;lt; k &amp;lt; 3pi , crystal momentum.&#xA;    E = linspace(((i-1)*pi/a)^2,  (i*pi/a)^2, 1000);% Energy.&#xA;    [X, Y] = meshgrid(k, E);&#xA;&#xA;    Z = cos(X)  - ( P * pi * sinc( a * sqrt(Y)/pi ) )  - cos( a * sqrt(Y) );&#xA;    contour(X, Y, Z,[0;0],'linewidth',2,'linecolor','k');&#xA;&#xA;    line(i*[pi,pi],[0,(N*pi/a)^2+0.2],'color','k','linestyle','--')&#xA;end&#xA;&#xA;xlabel('k')&#xA;xlim([0,N*pi+0.2])&#xA;ylim([0,(N*pi/a)^2+0.2])&#xA;ylabel('E(k)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V3wxM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V3wxM.jpg&quot; alt=&quot;contour plot sectioned&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2018-09-16T18:09:08.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30218" PostTypeId="2" ParentId="30203" CreationDate="2018-09-17T04:50:26.130" Score="7" Body="&lt;p&gt;Even if a matrix is very sparse, its matrix product with itself can be dense. Take for example a diagonal matrix and fill its first row and column with nonzero entries; its product with itself will be completely dense. Such a matrix can arise, for examle, as graph Laplacian of a graph in which there is a vertex that is connected to all other vertices. In practice, it suffices if there are few vertices with pretty high connectivity to the rest of the network. For matrix-vector multiplication, this phenomenon is less relevant although it may lead to imbalances when trying to parallelize the matrix-vector multiplication.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to highlight: It really depends on the sparsity pattern and on what you want to do with the matrix. So, the best definition of a sparse matrix that I can come up with (which is pretty useless at the same time) is as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A matrix is &lt;em&gt;sparse&lt;/em&gt; if it is advantageous to store only its nonzero values and their positions &lt;em&gt;and to invest the additional overhead&lt;/em&gt; that is coming from managing the arising data structure.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The lesson to learn: It really depends on &lt;em&gt;what you want to do with it&lt;/em&gt;, &lt;em&gt;which algorithm you use&lt;/em&gt;, and (as others have already pointed out) &lt;em&gt;which hard- and software you use&lt;/em&gt; whether a given matrix is sparse or not (read as: whether you should use a sparse or dense matrix data structure). There cannot be a purely percentage-based rule if it is not only about storing data or matrix-vector multiplication. The best way to find out if your matrices are sparse is just &lt;em&gt;to try it&lt;/em&gt; and compare with dense matrix methods.&lt;/p&gt;&#xA;" OwnerUserId="24976" LastEditorUserId="24976" LastEditDate="2018-09-17T05:04:10.747" LastActivityDate="2018-09-17T05:04:10.747" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30219" PostTypeId="1" CreationDate="2018-09-17T10:04:17.437" Score="2" ViewCount="189" Body="&lt;p&gt;I am trying to solve the viscous Burger equation&#xA;$$&#xA;\frac{\partial u}{\partial t} + u\frac{\partial u}{\partial x} = \mu \frac{\partial^2 u}{\partial x^2}&#xA;$$&#xA;with Neumann boundary conditions. I am&#xA;using the Maccormack predictor-corrector scheme:&#xA;$$&#xA;u_j^\overline{{n+1}} = u_j^n - \frac{\Delta t}{\Delta x}(F_{j+1} - F_j^n) + r(u_{j+1}^n - 2u_{j}^n + u_{j-1}^n )\\&#xA;u_j^{n+1} = \frac{1}{2}\Big[ u_j^n + u_j^\overline{{n+1}}  - \frac{\Delta t}{\Delta x}(F_{j}^\overline{{n+1}} - F_{j-1}^\overline{{n+1}} ) + r(u_{j+1}^\overline{{n+1}}  - 2u_{j}^\overline{{n+1}}  + u_{j-1}^\overline{{n+1}} ) \Big]&#xA;$$&#xA;for $F=0.5u^2$. I understand that the diffusive term has been centre-differenced, and I believe I implement the boundary conditions in this term by solving for a ghost node. For example, if 0 is my first real node, I say that the ghost node at $j=-1$ should satisfy $u_{-1}=u_{1}$, and I impose&#xA;$$&#xA;u_0^\overline{{n+1}} = \,... \,+ r(2 u_{1}^n - 2u_{0}^n ).&#xA;$$&#xA;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this correct?&lt;/li&gt;&#xA;&lt;li&gt;How do I implement the no-flux condition with the forwards (backwards) differenced terms in the predictor (corrector) step?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Simply setting everything involving $F$ to zero doesn't seem to work.&lt;/p&gt;&#xA;" OwnerUserId="28747" LastEditorUserId="20688" LastEditDate="2018-09-17T13:53:11.360" LastActivityDate="2018-09-17T13:53:11.360" Title="Neumann boundary conditions in the Maccormack scheme" Tags="|pde|finite-difference|boundary-conditions|nonlinear-equations|hyperbolic-pde|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30220" PostTypeId="1" CreationDate="2018-09-17T13:39:12.100" Score="2" ViewCount="46" Body="&lt;p&gt;I have asked this question already on &lt;a href=&quot;https://math.stackexchange.com/questions/2913987/lagrange-multiplier-sensitivity-implicit-constraint&quot;&gt;maths&lt;/a&gt; and &lt;a href=&quot;https://mathoverflow.net/questions/310766/sensitivity-of-lagrangian-solution-implicit-constraint&quot;&gt;mathoverflow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a question about a literature reference. I am writing a paper for engineers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually for the Lagrange multiplier problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\nabla f(x)+\lambda \nabla g(x)=0&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the sensitivity result that the multiplier $\lambda$ gives the sensitivity for changes in the constraint function is derived for the case $g(x)−h=0$ for varying $h$.&#xA;Is there somewhere a reference deriving this for $g(x;h)=0$?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked through the literature and asked one expert. No trace of such a proof. But there must be one, I am sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can do it myself, however, I would like to avoid to reinvent the wheel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am only looking for a hint for a reference, no explanations how to do it.&lt;/p&gt;&#xA;" OwnerUserId="28748" LastEditorUserId="20688" LastEditDate="2018-09-17T13:52:06.060" LastActivityDate="2018-09-17T13:52:06.060" Title="Sensitivity Lagrangian solution general case" Tags="|constrained-optimization|reference-request|numerics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30222" PostTypeId="1" AcceptedAnswerId="30223" CreationDate="2018-09-17T14:40:12.153" Score="2" ViewCount="146" Body="&lt;p&gt;In a traditional PDE solver on unstructured meshes, we know exactly the sparse pattern of the Jacobian matrix, and we know that the amount of data to communicate is far smaller than the amount of local data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then is it more efficient to use ghosted PETSC Vec (created with &lt;code&gt;VecCreateGhost&lt;/code&gt; or &lt;code&gt;VecCreateGhostWithArray&lt;/code&gt;) than the default Vec (creatd with &lt;code&gt;VecCreateMPI&lt;/code&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="22780" LastActivityDate="2018-09-17T15:12:09.337" Title="is it more efficient to use ghosted vector in PETSC for PDE solving on unstructured mesh?" Tags="|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30223" PostTypeId="2" ParentId="30222" CreationDate="2018-09-17T15:12:09.337" Score="2" Body="&lt;p&gt;&lt;code&gt;VecGhost&lt;/code&gt; can be useful if copying the interior values as part of a global-to-local (halo update) is expensive. The memory access pattern of a dedicated local vector can give better cache reuse, so perceived benefits of &lt;code&gt;VecGhost&lt;/code&gt; are often erased in traversal. Also, since &lt;code&gt;VecGhost&lt;/code&gt; contains storage for the ghost values, it typically increases the memory used by a Krylov space (relative to normal vectors, where there are a constant number of local vectors).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;VecGhost&lt;/code&gt; is a dubious performance optimization; I don't recommend using it in new code unless you have done the profiling for your application and are sure that it is beneficial. It exists so that users who desire that particular optimization don't need to roll their own implementation.&lt;/p&gt;&#xA;" OwnerUserId="119" LastActivityDate="2018-09-17T15:12:09.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30224" PostTypeId="1" CreationDate="2018-09-17T20:10:33.060" Score="0" ViewCount="202" Body="&lt;p&gt;I am applying &lt;code&gt;optim()&lt;/code&gt; function in R to obtain maximum likelihood estimates of the fixed effects and random effects in a model with bivariate random effects. The code works well but it is very slow in executing the line of code involving the &lt;code&gt;optim()&lt;/code&gt; function. I would like to know if there is/ are any specialized optimization function(s)/ package(s) in R for maximizing likelihoods in random effect models, instead of the general purpose optimization function &lt;code&gt;optim()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28726" LastEditorUserId="20688" LastEditDate="2018-11-11T22:14:43.727" LastActivityDate="2023-05-04T12:05:30.667" Title="R function or package for carrying out maximum likelihood techniques in random effect models" Tags="|optimization|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30226" PostTypeId="2" ParentId="30201" CreationDate="2018-09-18T02:42:49.727" Score="1" Body="&lt;p&gt;Let us note $w = u_1 - u_2$. We know that $-\Delta w = 0$ in $\Omega$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that there is $\tilde{\Omega} \subset \Omega$ such that $w = 0$ on $\tilde{\Omega}$ (without any loss of generality, we can assume that $\overline{ \tilde{\Omega} } \subset \Omega$). As $w$ satisfies the Laplace equation, there is no jump for $w$ across the boundary of $\tilde{\Omega}$: $[w]_{|\partial \tilde{\Omega}} = 0$ and similarly for the normal derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in $\Omega \backslash \tilde{\Omega}$, $w$ satisfies the Laplace equation and, on $\partial \tilde{\Omega}$, $w$ has a null trace and a null normal derivative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By a continuation argument (see R. Leis, &quot;Initial boundary value problems in mathematical physics&quot;, 1986), $w$ is 0 in $\Omega \backslash \tilde{\Omega}$.&lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2018-09-18T02:42:49.727" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30229" PostTypeId="1" CreationDate="2018-09-18T13:42:23.107" Score="2" ViewCount="163" Body="&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of a finite element code, I have many small (order of 30x30) matrix inverses (or LU factorizations), one per finite element. These matrix inverses never change and must be applied to vectors repeatedly in the context of a matrix-free solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Storing all these inverses is the memory bottleneck of the code, because they have to be applied for each finite element in the mesh. On the other hand, computing the inverses 'on the fly' (using LAPACK or Numpy) is not practical, because these inverses must be applied at each iteration of a sparse iterative solve, so each time-step would require &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(number of CG iterations) * (number of elements) * (time of inverse)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is too expensive, even for problems far smaller than those I am interested in. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(possible) solution&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am considering performing the matrix inversions or LU factorizations once, and then storing them to disk, rather than keeping them in memory. Then the idea is to read the inverses into memory either one at a time or by chunks, and apply them, avoiding the repeated inversions. But I know that reading from disk can be expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there good or accepted ways of doing this? Most out-of-core papers I've looked at are considering direct solves or much more complicated problems, so I would appreciate any pointers to relevant references as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: The code is a mix of Python and C, and I've found some posts indicating that perhaps the HDF5 library is a good alternative to rolling my own cache, as suggested above.&lt;/p&gt;&#xA;" OwnerUserId="20148" LastEditorUserId="20148" LastEditDate="2018-09-18T14:50:01.473" LastActivityDate="2018-10-10T01:52:42.363" Title="Memory/speed tradeoff for many small matrix inverses" Tags="|linear-algebra|linear-solver|io|matrix-free|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30230" PostTypeId="1" CreationDate="2018-09-18T15:25:45.237" Score="1" ViewCount="66" Body="&lt;p&gt;I'm thinking to couple my fluid flow solver based on lattice Boltzmann method with a mechanical deformation solver based on finite element method to take account for solid deformation in my models. In fact, I'm looking for a fluid-structure interaction framework, which its fluid solver was written based on lattice Boltzmann. I didn't find any opensource example by using this coupling idea. Can anyone suggest any opensource software or direction how it is possible to do it?&lt;/p&gt;&#xA;" OwnerUserId="28407" LastActivityDate="2018-09-18T15:25:45.237" Title="Combining fluid flow solver based on lattice Boltzmann method with a mechanical deformation solver based on finite element method" Tags="|navier-stokes|solid-mechanics|unstructured-mesh|coupling|lattice-boltzmann-methods|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30231" PostTypeId="2" ParentId="2583" CreationDate="2018-09-18T16:17:57.813" Score="6" Body="&lt;p&gt;Arnold Neumaier's answer is discussed in more detail in section 3.2 of the paper &lt;a href=&quot;https://doi.org/10.1137/130934283&quot; rel=&quot;noreferrer&quot;&gt;&quot;Approximating Spectral Densities of Large Matrices&quot; by Lin Lin, Yousef Saad and Chao Yang (2016)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other methods are also discussed but the numerical analysis at the end of the paper shows that the Lanczos method outperforms these alternatives. &lt;/p&gt;&#xA;" OwnerUserId="28721" LastEditorUserId="20688" LastEditDate="2018-09-18T16:22:07.900" LastActivityDate="2018-09-18T16:22:07.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30232" PostTypeId="1" CreationDate="2018-09-18T18:08:33.020" Score="0" ViewCount="1166" Body="&lt;p&gt;I am switching from OSX to Linux, which I am familiar with but have no greater experience. There are several different alternatives: archlinux, gnome, ubuntu etc. Which would you recommend for a PhD student in numerical analysis? I mainly use Julia, Fortran, C++, Python and occasionally Matlab. Thus it should be easy to get the associated compilers up and running, as installing libraries for numerical tool boxes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore I become more familiar with the Linux environment, that being said I want an OS that doesn't require to much of me as a user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I am switching from OSX since I got tired of getting open source codes to compile on my Mac. Most codes and packages in my field are intended for Linux anyway.&lt;/p&gt;&#xA;" OwnerUserId="19807" LastEditorUserId="19807" LastEditDate="2018-09-19T05:59:10.700" LastActivityDate="2018-09-22T10:51:08.450" Title="Which Linux OS for computer science and numerical analysis" Tags="|numerics|compiling|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30233" PostTypeId="2" ParentId="30232" CreationDate="2018-09-18T22:30:42.360" Score="6" Body="&lt;p&gt;There are lots of available Linux distributions and several widely used window managers, and most of them would be perfectly adequate for your needs.  It is important to make sure that the distribution and window manager that you pick will work with the software packages that matter to you.  It's also important to understand how the distribution you pick is supported and the supported lifetime of that particular distribution.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You want to be able to run MATLAB, so check the Mathworks list of supported Linux distributions at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mathworks.com/support/sysreq.html?sec=linux&quot; rel=&quot;noreferrer&quot;&gt;https://www.mathworks.com/support/sysreq.html?sec=linux&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll also want to understand the long-term plans for your Linux distribution and required upgrades that you'll have to do in the future.&#xA;You don't want to have to do a lot of work to upgrade to a new distribution, so consider using one of the LTS (Long Term Support) versions of Ubuntu.  Ubuntu 16.04 LTS will be supported by Canonical into the year 2021, and Ubuntu 18.04 LTS will be supported into the year 2023.  Not having to do a major upgrade to your Linux distribution before then is a big advantage.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-09-18T22:30:42.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30234" PostTypeId="2" ParentId="30229" CreationDate="2018-09-19T05:00:43.323" Score="4" Body="&lt;p&gt;There is no faster way to storing things than to put them into memory. If your problem is such that you can't store all of the element matrix inverses in memory, then you have two options: (i) buy more memory for the machines you use, (ii) use more machines. There is also (iii) solve smaller problems. Trying to externalize the matrices to disk will make things vastly slower because disks are at least an order of magnitude slower than memory. Disks are also vastly slower than communication between machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's worth stepping back. If your solver scheme requires storing lots of local matrix inverses, then maybe that's not a great solver. I'm making the assumption here that in your scheme, storing the inverses is more expensive than storing the local matrices to begin with -- for example, because the local matrices are sparse but the inverse are not. Generally, my experience from 20 years of finite element computations is that if you are running out of memory, then you've already run out of CPU time as well. So I'm a bit surprised that runtimes don't also show up on the list of your worries.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-09-19T05:00:43.323" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30235" PostTypeId="1" CreationDate="2018-09-19T06:27:03.697" Score="1" ViewCount="299" Body="&lt;p&gt;I'm new to numerical analysis, and have been learning root finding algorithms. I am a bit confused about the difference between Muller's method, and Newton's method using an n-degree interpolating polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the Muller's method, which approximates f(x) using a quadratic polynomial, different from the Newton's method, where lets say we use a 2 degree interpolating polynomial to find roots of f(x)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could not find clear and concise theory on using the Newton method with a 2 degree interpolating polynomial to identify if its the same as or is different than the Muller's method.&lt;/p&gt;&#xA;" OwnerUserId="28761" LastActivityDate="2018-09-19T11:38:08.577" Title="Muller's method is the same as Newton's method with a quadratic interpolating polynomial?" Tags="|newton-method|rootfinding|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30236" PostTypeId="2" ParentId="30235" CreationDate="2018-09-19T11:38:08.577" Score="1" Body="&lt;p&gt;One of the most basic techniques in numerical analysis, when solving a complicated problem, is to construct an approximately-similar easy problem, solve that to obtain an approximate solution, and restart the process using that solution as a new starting point. This is a very general technique, and it’s not even restricted to root finding methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Newton’s and Halley’s methods both share this construction, but that’s not enough to call them the “same” method: most iterative methods are constructed this way. So picking which specific approximation (a line, a quadratic, etc) to use and how is what defines the method, not the general approach.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-09-19T11:38:08.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30237" PostTypeId="1" CreationDate="2018-09-19T12:16:34.960" Score="1" ViewCount="488" Body="&lt;p&gt;I've been trying to learn about root finding using Newton's method, which uses a quadratic interpolating polynomial. I found this text, &lt;a href=&quot;https://www.elsevier.com/books/numerical-methods-for-roots-of-polynomials-part-ii/mcnamee/978-0-444-52730-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical methods for roots of polynomials, PART 2: McNamee &amp;amp; Pan&lt;/a&gt;, which has relevant theory, but there seems to be some error in the text.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/polv6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/polv6.jpg&quot; alt=&quot;Book section describing the quadratic part of Newton&amp;#39;s method&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While calculating $x_{i+1}$, shouldn't the second term in the RHS of that equation be reciprocal of what it is, or am I wrong? I don't understand what am I missing here. &lt;/p&gt;&#xA;" OwnerUserId="28761" LastEditorUserId="23791" LastEditDate="2018-09-19T22:57:06.757" LastActivityDate="2018-09-19T22:57:06.757" Title="Root finding using Newton's method with quadratic interpolation - Is there a mistake in this textbook?" Tags="|newton-method|rootfinding|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30238" PostTypeId="2" ParentId="30237" CreationDate="2018-09-19T14:57:17.657" Score="3" Body="&lt;p&gt;No, the book is right. I know that the second term on the right looks superficially like the standard solution of a quadratic equation, upside down, but look more closely: the denominator $2f(x_i)$ is $2c$, not the $2a$ which you might have expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Muller's method uses a less common expression for the roots of a quadratic equation:&#xA;$$&#xA;x = \frac{-2c}{b\pm\sqrt{b^2-4ac}}&#xA;$$&#xA;which is mentioned on &lt;a href=&quot;https://en.wikipedia.org/wiki/Quadratic_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia page for quadratic equations&lt;/a&gt;. They even mention there that it is used in Muller's method. And on &lt;a href=&quot;https://en.wikipedia.org/wiki/Muller%27s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Wikipedia page for Muller's method&lt;/a&gt; they say that the standard formula for a quadratic equation is not used because it may lead to a loss of significance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure that Muller's method is the go-to method for this kind of root finding. &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverse_quadratic_interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Inverse quadratic interpolation&lt;/a&gt; is used within the standard Brent method for locating roots. I don't know enough about the topic to explain in more detail, however.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-09-19T14:57:17.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30240" PostTypeId="2" ParentId="30232" CreationDate="2018-09-19T15:10:28.807" Score="2" Body="&lt;p&gt;I use Debian and OpenSuse Tumbleweed on our Linux desktops. Tubleweed is a rolling distribution, so there is no concept of OS version. You get very latest softwares but could be bit unstable. If your requirements are complex, then it can still be difficult to manage all the software installations using your OS package managers. My strategy is like this.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Install compilers from OS package manager&lt;/li&gt;&#xA;&lt;li&gt;Install all scientific software (including mpi) using Spack &lt;a href=&quot;https://spack.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://spack.io&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;For Python, use Anaconda&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using Spack has tremendously eased up the pain of installation of a ton of packages and their dependencies for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, the above works for Mac OSX also, and my laptop runs OSX.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-09-19T15:10:28.807" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30241" PostTypeId="2" ParentId="30232" CreationDate="2018-09-19T16:27:16.640" Score="1" Body="&lt;p&gt;I use Ubuntu 18.04 at work and Manjaro Arch Linux on my laptop. I use &lt;code&gt;MATLAB&lt;/code&gt;, &lt;code&gt;C++&lt;/code&gt;, and &lt;code&gt;Julia&lt;/code&gt; for my work. &lt;code&gt;MATLAB&lt;/code&gt; works fine on both operating systems, though you may have to play with the drivers on Ubuntu in order to get good-looking plots. Naturally, &lt;code&gt;C++&lt;/code&gt; and &lt;code&gt;Julia&lt;/code&gt; work great on both OSes. &lt;code&gt;MATLAB&lt;/code&gt; also runs fine, though if you intend to use the Coder or MEX compiler, be sure to set up the MEX compiler properly. Here's a &lt;a href=&quot;https://xolotl.readthedocs.io/en/latest/compilers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;short guide&lt;/a&gt; I wrote to troubleshooting that.&lt;/p&gt;&#xA;" OwnerUserId="28767" LastActivityDate="2018-09-19T16:27:16.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30247" PostTypeId="1" CreationDate="2018-09-21T11:38:31.810" Score="-2" ViewCount="82" Body="&lt;p&gt;I was doing error analysis of numerical scheme and I get $L_1$ error for each grid size with $N$ element. I was searching reference to compute accuracy order/rate from that error data but doesn't find any good reference. Anyone know?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error:&#xA;$e=|u-u_{exact}|$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$L_1$ error:&#xA;$L_1=1/N \sum_{i=1}^{N} e_i$&lt;/p&gt;&#xA;" OwnerUserId="28714" LastEditorUserId="28714" LastEditDate="2018-09-22T05:04:43.930" LastActivityDate="2018-09-22T08:46:46.733" Title="Calculate accuracy order/rate" Tags="|error-estimation|numerics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30250" PostTypeId="1" AcceptedAnswerId="30251" CreationDate="2018-09-21T21:30:00.143" Score="8" ViewCount="671" Body="&lt;p&gt;The Langevin function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(x) = \mathrm{coth}(x) - \frac{1}{x}$&lt;/span&gt; occurs in computations related to elastomers and paramagnetic materials. It is easily computed accurately and with high performance as long as the issue of subtractive cancellation near the origin is addressed, e.g. by using a minimax polynomial approximation in that region. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To eliminate the cancellation issue completely, choosing a good switch-over point to the polynomial approximation is important; some implementations that switch over at unity, which is suboptimal. A switchover point closer to &lt;span class=&quot;math-container&quot;&gt;$2.0$&lt;/span&gt; is advisable. With a faithfully-rounded implementation of the exponential function &lt;span class=&quot;math-container&quot;&gt;$\exp$&lt;/span&gt;, which is usually provided by modern math libraries, a maximum error of &lt;span class=&quot;math-container&quot;&gt;$1.3$&lt;/span&gt; ulp can be achieved in a straightforward manner. Exemplary C implementations are shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, many applications require the &lt;em&gt;inverse&lt;/em&gt; Langevin function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x)$&lt;/span&gt;, which has no closed-form representation; numeric approximation is required. Most of the approximations used in the literature are quite inaccurate, providing the relative error in the range of 0.05% to 2%. Useful overviews are provided by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1007/s00397-015-0880-9&quot; rel=&quot;noreferrer&quot;&gt;Benjamin C. Marchi and Ellen M. Arruda, &quot;An error-minimizing approach to inverse Langevin approximations.&quot; &lt;em&gt;Rheol Acta&lt;/em&gt; (2015) 54:887-902&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1016/j.jnnfm.2015.05.007&quot; rel=&quot;noreferrer&quot;&gt;Martin Kröger, &quot;Simple, admissible, and accurate approximants of the inverse Langevin and Brillouin functions, relevant for strong polymer deformations and flows.&quot; &lt;em&gt;Journal of Non-Newtonian Fluid Mechanics&lt;/em&gt; 223 (2015) 77-87&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recent work shows that a lack of accuracy in the computation of the inverse Langevin function has a negative impact on the overall accuracy of the scientific computations in which it used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1016/j.jnnfm.2016.02.008&quot; rel=&quot;noreferrer&quot;&gt;Amine Ammar, &quot;Effect of the inverse Langevin approximation on the solution of the Fokker-Planck equation of non-linear dilute polymer.&quot; &lt;em&gt;Journal of Non-Newtonian Fluid Mechanics&lt;/em&gt; 231 (2016): 1-5.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The accurate computation of the inverse Langevin function is therefore highly desirable. In addition to accuracy, some use cases benefit from specific functional forms to ease of substitution, differentiation, and integration. Leaving aside such additional requirements, and focusing on minimizing relative error, how can the inverse Langevin function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x)$&lt;/span&gt; be computed accurately (close to machine precision) and with good performance?&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;math.h&amp;gt;  // fabs, exp, copysign&#xA;&#xA;/* compute Langevin function L(a) = coth(a) - 1/a. Max error typically &amp;lt; 1.3 ulp */&#xA;float langevinf (float a)&#xA;{&#xA;    float r, t;&#xA;&#xA;    t = fabsf (a);&#xA;    if (t &amp;gt; 1.8125f) {&#xA;        float e;&#xA;        e = 1.0f / (expf (2.0f * t) - 1.0f);&#xA;        r = (1.0f - 1.0f / t) + 2.0f * e;&#xA;        r = copysignf (r, a);&#xA;    } else {&#xA;        float s;&#xA;        s = a * a;&#xA;        r =         7.70960469e-8f;&#xA;        r = r * s - 1.65101926e-6f;&#xA;        r = r * s + 2.03457112e-5f;&#xA;        r = r * s - 2.10521728e-4f;&#xA;        r = r * s + 2.11580913e-3f;&#xA;        r = r * s - 2.22220998e-2f;&#xA;        r = r * s + 8.33333284e-2f;&#xA;        r = r * a + 0.25f * a;&#xA;    }&#xA;    return r;&#xA;}&#xA;&#xA;/* compute Langevin function L(a) = coth(a) - 1/a. Max error typically &amp;lt; 1.3 ulp */&#xA;double langevin (double a)&#xA;{&#xA;    double r, t;&#xA;&#xA;    t = fabs (a);&#xA;    if (t &amp;gt; 1.8125) {&#xA;        double e;&#xA;        e = 1.0 / (exp (2.0 * t) - 1.0);&#xA;        r = (1.0 - 1.0 / t) + 2.0 * e;&#xA;        r = copysign (r, a);&#xA;    } else {&#xA;        double s, t, u;&#xA;        s = a * a;&#xA;        t = s * s;&#xA;        r = 2.6224020345921745e-16 * s - 9.0382507125284313e-15;&#xA;        u = 1.6058476918889783e-13 * s - 2.0544032928760129e-12;&#xA;        r = r * t + u;&#xA;        u = 2.2318295632954910e-11 * s - 2.2645716105162602e-10;&#xA;        r = r * t + u;&#xA;        u = 2.2484176131435534e-09 * s - 2.2212056151537571e-08;&#xA;        r = r * t + u;&#xA;        u = 2.1925751794205211e-07 * s - 2.1644032423757088e-06;&#xA;        r = r * t + u;&#xA;        u = 2.1377798795067705e-05 * s - 2.1164021156459166e-04;&#xA;        r = r * t + u;&#xA;        u = 2.1164021163937877e-03 * s - 2.2222222222221859e-02;&#xA;        r = r * t + u;&#xA;        r = r * s + 8.3333333333333329e-02;&#xA;        r = r * a + 0.25 * a;&#xA;    }&#xA;    return r;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20458" LastEditorUserId="20688" LastEditDate="2018-09-21T21:49:00.363" LastActivityDate="2023-06-12T23:44:43.217" Title="Accurate and efficient computation of the inverse Langevin function" Tags="|algorithms|efficiency|accuracy|special-functions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30251" PostTypeId="2" ParentId="30250" CreationDate="2018-09-21T21:30:00.143" Score="12" Body="&lt;p&gt;The inverse Langevin function &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x)$&lt;/span&gt; is an odd function. Therefore one needs to consider only approximation on the interval &lt;span class=&quot;math-container&quot;&gt;$[0, 1]$&lt;/span&gt;; the negative half-plane is treated via symmetry about the origin. Because of the singularity at unity, a polynomial approximation is not a fruitful direction of the investigation, although some authors have explored high-degree Taylor expansions around zero, for example&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1177%2F1081286511429886&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mikhail Itskov, Roozbeh Dargazany, and Karl Hörnes, &amp;quot;Taylor expansion of the inverse function with application to the Langevin function.&amp;quot; &lt;em&gt;Mathematics and Mechanics of Solids&lt;/em&gt; 17, No. 7 (2012): 693-701&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The 115-term series expansion from the paper is not practical due to the large amount of computation required. Rational approximations are well suited to approximating functions with singularities, and most work in the literature has focused on these, in particular in the form of Pade approximants. In recent years, the approach has been refined by adding compensation for the error terms, for example&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1177%2F1081286517754131&quot; rel=&quot;nofollow noreferrer&quot;&gt;Benjamin C. Marchi and Ellen M. Arruda. &amp;quot;Generalized error-minimizing, rational inverse Langevin approximations.&amp;quot; &lt;em&gt;Mathematics and Mechanics of Solids&lt;/em&gt; (2018): 1081286517754131.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some authors have tried to approximate the shape of the inverse Langevin function with trigonometric functions, in particular, &lt;span class=&quot;math-container&quot;&gt;$\tan$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sin$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://hdl.handle.net/1721.1/9794&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jörgen Stefan Bergström, &amp;quot;Large strain time-dependent behavior of elastomeric materials.&amp;quot; PhD diss., Massachusetts Institute of Technology, 1999.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Grant Keady, &amp;quot;The Langevin function and truncated exponential distributions.&amp;quot; &lt;em&gt;arXiv preprint arXiv:1501.02535&lt;/em&gt; (2015).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1007/s00397-016-0977-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rafayel Petrosyan, &amp;quot;Improved approximations for some polymer extension models.&amp;quot; &lt;em&gt;Rheologica Acta&lt;/em&gt; 56, no. 1 (2017): 21-26.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;While newer works push relative error down to about &lt;span class=&quot;math-container&quot;&gt;$10^{-5}$&lt;/span&gt;, I am aware of only one paper that tries to achieve close to machine precision. It does this  by using piece-wise approximation with thousands of intervals to each of which it fits a cubic spline. The storage requirements for tables of coefficients reach to the hundreds of KBytes. This exceeds the size of first level caches on most processors, so the scheme may not be suitable in practice:&lt;/p&gt;&#xA;&lt;p&gt;José María Benitez and Francisco Javier Montáns, &amp;quot;A simple and efficient numerical procedure to compute the inverse Langevin function with high accuracy&amp;quot;. &lt;em&gt;arXiv preprint arXiv:1806.08068&lt;/em&gt; (2018).&lt;/p&gt;&#xA;&lt;p&gt;For a different approach, I noted the similarity in overall shape between the inverse Langevin function and the inverse error function. As I pointed out in another &lt;a href=&quot;https://stackoverflow.com/a/49743348&quot;&gt;answer&lt;/a&gt;, the latter can be approximated efficiently by making use of the transform &lt;span class=&quot;math-container&quot;&gt;$t=\log(1-x^2)$&lt;/span&gt; and then generating approximations of the form &lt;span class=&quot;math-container&quot;&gt;$x \cdot P(t)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is a polynomial. Since &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}(x) \approx 1 - \frac{1}{x}$&lt;/span&gt; near unity, it follows that &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{x-1}$&lt;/span&gt; is a good approximation to &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x)$&lt;/span&gt; near the singularity. Experimentally, for IEEE-754 single precision, using that approximation is suitable on &lt;span class=&quot;math-container&quot;&gt;$[\frac{57}{64}, 1]$&lt;/span&gt;, maintaining the maximum error of less than &lt;span class=&quot;math-container&quot;&gt;$2.6$&lt;/span&gt; ulp.&lt;/p&gt;&#xA;&lt;p&gt;My experiments show that, as expected, the approximation of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x)$&lt;/span&gt; in the same manner as for the inverse error function requires just moderately more terms in the polynomial approximation. However,&#xA;generating a high-quality minimax approximation requires some effort as the approximation problem is somewhat poorly conditioned. Starting with the Remez algorithms to generate an initial approximation, I used a heuristic refinement (similar to simulated annealing) to arrive at the final result. My preliminary experiments showed that this approach is extensible for use cases requiring more than single-precision accuracy, but too computational expensive for reaching full double-precision accuracy, which would require polynomials of degree 40 or so. A rational approximation, as shown at the end of this answer, is more appropriate for a double-precision implementation.&lt;/p&gt;&#xA;&lt;p&gt;Most modern hardware platforms provide a fused multiply-add operation (FMA) that helps reduce rounding errors and provides protection against some instances of subtractive cancellation. Its use also tends to improve performance, as is execution time is often identical or just slightly higher than a floating-point multiplication. The operation is often exposed through a library function, such as &lt;code&gt;fma()&lt;/code&gt; in C and C++. However, this maps to slow emulation on hardware platforms without support for FMA, and some emulations have functional &lt;a href=&quot;https://stackoverflow.com/questions/42166563/is-my-fma-broken&quot;&gt;bugs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;An exemplary C implementation for IEEE-754 (2008) single precision (&lt;code&gt;binary32&lt;/code&gt;) is shown below. While the accuracy achieved will depend on the quality of the standard math library's implementation of &lt;span class=&quot;math-container&quot;&gt;$\log$&lt;/span&gt;, most modern math libraries provide a faithfully-rounded implementation. With the three different implementations of &lt;code&gt;logf&lt;/code&gt; I tried, the error bounds stated in the code comments were maintained.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;math.h&amp;gt; // fabs, log, copysign, fma&#xA;&#xA;/* &#xA;  Compute inverse Langevin function accurate to almost machine precision&#xA;&#xA;  USE_FMA == 0: max. ulp error &amp;lt; 4.27, max. relative error &amp;lt; 4.43e-7&#xA;  USE_FMA == 1: max. ulp error &amp;lt; 3.64, max. relative error &amp;lt; 3.84e-7&#xA;*/&#xA;float langevininvf (float x)&#xA;{&#xA;    float p, r, t;&#xA;    if ((fabsf (x) &amp;gt; 0.890625f) &amp;amp;&amp;amp; (fabsf (x) &amp;lt;= 1.0f)) {&#xA;        r = copysignf (1.0f / (fabsf (x) - 1.0f), x);&#xA;    } else {&#xA;#if USE_FMA&#xA;        t = fmaf (x, 0.0f - x, 1.0f); // compute 1-x*x accurately&#xA;        t = logf (t);&#xA;        p =              2.18808651e-4f;  //  0x1.cae000p-13&#xA;        p = fmaf (p, t, -7.90076610e-3f); // -0x1.02e46ep-7&#xA;        p = fmaf (p, t, -7.12909698e-2f); // -0x1.240200p-4&#xA;        p = fmaf (p, t, -2.40409270e-1f); // -0x1.ec5bb2p-3&#xA;        p = fmaf (p, t, -4.14386481e-1f); // -0x1.a854eep-2&#xA;        p = fmaf (p, t, -4.05752033e-1f); // -0x1.9f7d76p-2&#xA;        p = fmaf (p, t, -2.56382942e-1f); // -0x1.068940p-2&#xA;        p = fmaf (p, t, -1.22061931e-1f); // -0x1.f3f736p-4&#xA;        p = fmaf (p, t,  5.00488468e-2f); //  0x1.9a000ap-5&#xA;        p = fmaf (p, t, -1.84208602e-1f); // -0x1.79425cp-3&#xA;        p = fmaf (p, t,  3.98338169e-1f); //  0x1.97e5f6p-2 &#xA;        p = fmaf (p, t, -9.00006115e-1f); // -0x1.cccd9ap-1 &#xA;        p = fmaf (p, t,  5.00000000e-1f); //  0x1.000000p-1&#xA;        t = x + x;&#xA;        r = fmaf (p, t, t);&#xA;#else // USE_FMA&#xA;        t = (1.0f - x) * (1.0f + x); // compute 1-x*x accurately&#xA;        t = logf (t);&#xA;        p =         2.18868256e-4f; //  0x1.cb0000p-13&#xA;        p = p * t - 7.90085457e-3f; // -0x1.02e52cp-7&#xA;        p = p * t - 7.12914243e-2f; // -0x1.24027ap-4&#xA;        p = p * t - 2.40408897e-1f; // -0x1.ec5b80p-3&#xA;        p = p * t - 4.14384902e-1f; // -0x1.a85484p-2&#xA;        p = p * t - 4.05750901e-1f; // -0x1.9f7d2ap-2&#xA;        p = p * t - 2.56382436e-1f; // -0x1.06891ep-2&#xA;        p = p * t - 1.22061789e-1f; // -0x1.f3f710p-4&#xA;        p = p * t + 5.00487871e-2f; //  0x1.99ffeap-5&#xA;        p = p * t - 1.84208602e-1f; // -0x1.79425cp-3&#xA;        p = p * t + 3.98338020e-1f; //  0x1.97e5ecp-2&#xA;        p = p * t - 9.00006115e-1f; // -0x1.cccd9ap-1&#xA;        p = p * t + 5.00000000e-1f; //  0x1.000000p-1&#xA;        t = x + x;&#xA;        r = p * t + t;&#xA;#endif // USE_FMA&#xA;    }&#xA;    return r;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Computational platforms that provide fast hardware support for simple transcendental functions lend themselves to particularly efficient implementations of the proposed algorithm. The following shows an implementation for GPUs using CUDA that translates to only about twenty-five instructions.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;/* max. ulp error &amp;lt; 3.925, max. relative error &amp;lt; 3.95e-7 */&#xA;__device__ float langevininvf (float x)&#xA;{&#xA;    float fa, p, r, t;&#xA;    fa = fabsf (x);&#xA;    if ((fa &amp;gt; (57.0f / 64.0f)) &amp;amp;&amp;amp; (fa &amp;lt;= 1.0f)) {&#xA;        t = fa - 1.0f;&#xA;        asm (&amp;quot;rcp.approx.ftz.f32 %0,%0;&amp;quot; : &amp;quot;+f&amp;quot;(t));&#xA;        r = copysignf (t, x);&#xA;    } else {&#xA;        t = fmaf (x, -x, 1.0f);&#xA;        asm (&amp;quot;lg2.approx.ftz.f32 %0,%0;&amp;quot; : &amp;quot;+f&amp;quot;(t));&#xA;        p =              2.69152224e-6f;  //  0x1.694000p-19&#xA;        p = fmaf (p, t, -1.40199758e-4f); // -0x1.26052cp-13&#xA;        p = fmaf (p, t, -1.82510854e-3f); // -0x1.de70f6p-10&#xA;        p = fmaf (p, t, -8.87932349e-3f); // -0x1.22f52ap-7&#xA;        p = fmaf (p, t, -2.20804960e-2f); // -0x1.69c450p-6&#xA;        p = fmaf (p, t, -3.11916713e-2f); // -0x1.ff0b5ap-6&#xA;        p = fmaf (p, t, -2.84342300e-2f); // -0x1.d1ddcep-6&#xA;        p = fmaf (p, t, -1.95302274e-2f); // -0x1.3ffbb6p-6&#xA;        p = fmaf (p, t,  1.15530221e-2f); //  0x1.7a91c6p-7&#xA;        p = fmaf (p, t, -6.13459945e-2f); // -0x1.f68be0p-5&#xA;        p = fmaf (p, t,  1.91382647e-1f); //  0x1.87f3a0p-3&#xA;        p = fmaf (p, t, -6.23836875e-1f); // -0x1.3f678cp-1&#xA;        p = fmaf (p, t,  5.00000000e-1f); //  0x1.000000p-1&#xA;        t = x + x;&#xA;        r = fmaf (p, t, t);&#xA;    }&#xA;    return r;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As noted earlier, a rational approximation appears the best approach for a double-precision implementation. Noting the similarity of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathrm{tanh}(x)$&lt;/span&gt;, one approach is to transform the argument &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$t=\frac{\ln(1+x)}{\ln(1-x)}$&lt;/span&gt;, then compute &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x) \approx xR_{m,n}(t^2)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$R_{m,n}$&lt;/span&gt; is a rational minimax approximation comprising a polynomial &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; of degree &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; in the numerator and a polynomial &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; of degree &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in the denominator. Trying to numerically compute the coefficients of the minimax approximation with a variant of the Remez algorithm turns out to be extremely ill conditioned. When increasing &lt;span class=&quot;math-container&quot;&gt;$m, n$&lt;/span&gt; to achieve full double-precision accuracy, the solutions tend to degenerate: the number of peaks in the error curve is deficient and the equioscillation property is lost. A recent paper reports similar difficulties in approximating &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x)$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$[0,1)$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;Nicolas Brisebarre and Silviu-Ioan Filip, &amp;quot;Towards Machine-Efficient Rational&#xA;&lt;span class=&quot;math-container&quot;&gt;$L^∞$&lt;/span&gt;-Approximations of Mathematical Functions,&amp;quot; Draft publication hal-04093020, May 2023 (&lt;a href=&quot;https://hal.science/hal-04093020/&quot; rel=&quot;nofollow noreferrer&quot;&gt;online&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The best I have managed so far is to create an approximation &lt;span class=&quot;math-container&quot;&gt;$R_{14,14}$&lt;/span&gt; that results in &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}^{-1}(x)$&lt;/span&gt; being computed with a maximum relative error of &lt;span class=&quot;math-container&quot;&gt;$6 \cdot 10^{-15}$&lt;/span&gt; or, using a different error metric, 47 double-precision ulp. These error bounds are based on 200M random test vectors. My attempts of creating an approximation &lt;span class=&quot;math-container&quot;&gt;$R_{15,15}$&lt;/span&gt; have failed so far, with minute changes in the starting conditions of the Remez algorithm leading to vastly different sets of polynomial coefficients, none of them representing a non-degenerate solution.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;/* Compute inverse Langevin function with maximum relative error of 6e-15 */&#xA;double langevininv (double x)&#xA;{&#xA;    double fa, p, q, r, t;&#xA;    fa = fabs (x);&#xA;    if ((fa &amp;gt; 971.0/1024.0) &amp;amp;&amp;amp; (fa &amp;lt;= 1.0)) {&#xA;        // maximum error: 4 ulps&#xA;        t = fa - 1.0;&#xA;        t = 1.0 / t;&#xA;        r = copysign (t, x);&#xA;    } else {&#xA;        // maximum error: 47 ulps &#xA;        t = log1p (2.0 * fa / (1.0 - fa));&#xA;        t = t * t;&#xA;        p =             8.7442384822583792e-14;  //  0x1.89ce34b44b32ep-44&#xA;        p = fma (p, t, -1.0246877312258876e-12); // -0x1.206c8855594fap-40&#xA;        p = fma (p, t,  6.9112635465060234e-11); //  0x1.2ff5e8eabbf42p-34&#xA;        p = fma (p, t, -1.5954120724949512e-9);  // -0x1.b68b24dc5af04p-30&#xA;        p = fma (p, t,  3.7719321864398964e-8);  //  0x1.4401aa7c94e3bp-25&#xA;        p = fma (p, t, -5.9501169268478895e-7);  // -0x1.3f71c8cae9a0ap-21&#xA;        p = fma (p, t,  7.8764836653194943e-6);  //  0x1.084a7ac083ecfp-17&#xA;        p = fma (p, t, -7.3715838816829616e-5);  // -0x1.352fc77b8b1c1p-14&#xA;        p = fma (p, t,  5.3389955343705986e-4);  //  0x1.17eac8f88c356p-11&#xA;        p = fma (p, t, -2.3748559515852675e-3);  // -0x1.3746f14a1f408p-9 &#xA;        p = fma (p, t,  5.7661901230895048e-3);  //  0x1.79e49e004dd54p-8&#xA;        p = fma (p, t,  1.6354150376575827e-2);  //  0x1.0bf247415c4fdp-6&#xA;        p = fma (p, t, -1.0764052701852338e-1);  // -0x1.b8e545f44cd9bp-4&#xA;        p = fma (p, t,  2.9732462670549570e-1);  //  0x1.3075ddeffb679p-2&#xA;        p = fma (p, t,  1.0000000000000000e+0);  //  0x1.0000000000000p+0&#xA;        q =            -1.8576261900157100e-16;  // -0x1.ac56e54dc47acp-53&#xA;        q = fma (q, t,  3.7025672616258288e-14); //  0x1.4d7f636bcc935p-45&#xA;        q = fma (q, t, -3.8488081523689396e-12); // -0x1.0ed5f6be6f859p-38&#xA;        q = fma (q, t,  2.3670115378123239e-10); //  0x1.044173a5c110cp-32&#xA;        q = fma (q, t, -7.2533635216156031e-9);  // -0x1.f2728540cd2dcp-28&#xA;        q = fma (q, t,  1.6722199004755106e-7);  //  0x1.671b42e087ca4p-23&#xA;        q = fma (q, t, -2.8147043983109852e-6);  // -0x1.79c881b7b21b0p-19&#xA;        q = fma (q, t,  3.6736019004458165e-5);  //  0x1.342a0006dc6d1p-15&#xA;        q = fma (q, t, -3.6078759090487885e-4);  // -0x1.7a502e75b71c8p-12&#xA;        q = fma (q, t,  2.6901070708282545e-3);  //  0x1.609903c8ad1bap-9&#xA;        q = fma (q, t, -1.4214262366936270e-2);  // -0x1.d1c5e0005a453p-7&#xA;        q = fma (q, t,  4.9851609255380437e-2);  //  0x1.986266ecf45ddp-5&#xA;        q = fma (q, t, -7.0008037607361850e-2);  // -0x1.1ec0bf7fb12e8p-4&#xA;        q = fma (q, t, -1.5267537329450756e-1);  // -0x1.38adddb9a2d1dp-3&#xA;        q = fma (q, t,  1.0000000000000000e+0);  //  0x1.0000000000000p+0&#xA;        r = fma (p / q, x, x + x);&#xA;    }&#xA;    return r;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20458" LastEditorUserId="20458" LastEditDate="2023-06-12T23:44:43.217" LastActivityDate="2023-06-12T23:44:43.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30252" PostTypeId="2" ParentId="30247" CreationDate="2018-09-22T08:37:13.457" Score="0" Body="&lt;p&gt;After read some new reference, using first order interpolation, accuracy order/rate simply calculated as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Order=\frac{log(e_1/e_2)}{log(h_1/h_2)}$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's it&lt;/p&gt;&#xA;" OwnerUserId="28714" LastEditorUserId="28714" LastEditDate="2018-09-22T08:46:46.733" LastActivityDate="2018-09-22T08:46:46.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30253" PostTypeId="2" ParentId="30232" CreationDate="2018-09-22T10:51:08.450" Score="0" Body="&lt;p&gt;As you said you want to use &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Fortran, C++, Python (already in Linux) &lt;/li&gt;&#xA;&lt;li&gt;Matlab (can be installed easily).&lt;/li&gt;&#xA;&lt;li&gt;Julia you may need to install explicitly. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Further, you may want to use some &lt;strong&gt;scientific library&lt;/strong&gt;, such as of GNU. For writing reports you may want &lt;strong&gt;Latex&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I suggest to use &lt;strong&gt;&lt;a href=&quot;https://labs.fedoraproject.org/en/scientific/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scientific Fedora&lt;/a&gt;&lt;/strong&gt; it comes preloaded with several scientific packages e.g. Ipython, Pandas, GNU scientific library, latex, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, there are different flavors of distributions, aiming for different research communities, e.g., &lt;a href=&quot;http://caelinux.com/CMS/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CAE Linux&lt;/a&gt;  for Computer aided engineering and &lt;a href=&quot;http://environmentalomics.org/bio-linux/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bio Linux&lt;/a&gt; for biology. You should search for with your discipline name if any specific distribution exists to cater your requirements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other distros, like Ubuntu, you have to install such software by yourself. But Ubuntu has a pretty large community, so you will get better support.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: Specialized packages for PDE (e.g. Clawpack, PETSc, DEAL, etc.) can be installed later.  &lt;/p&gt;&#xA;" OwnerUserId="20015" LastActivityDate="2018-09-22T10:51:08.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30257" PostTypeId="1" AcceptedAnswerId="30258" CreationDate="2018-09-24T05:56:35.853" Score="1" ViewCount="142" Body="&lt;p&gt;I struggle with this problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o7c68.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o7c68.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand how to apply Newton's method to a already given function but I can't seem to find that function in this case from the given input n, t, &lt;strong&gt;d&lt;/strong&gt; and &lt;strong&gt;s&lt;/strong&gt;. I'm completely new to numerical method's so sorry if I don't see the obvious. It would be nice if someone could lead me in the right direction on how to solve this.&lt;/p&gt;&#xA;" OwnerUserId="28805" LastActivityDate="2018-09-24T14:43:18.927" Title="Newton's method problem" Tags="|newton-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30258" PostTypeId="2" ParentId="30257" CreationDate="2018-09-24T08:09:25.517" Score="5" Body="&lt;p&gt;The Netwon(-Raphson) method helps you finding the root(s) of a function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;, that is, those values of &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; for which &lt;span class=&quot;math-container&quot;&gt;$f(x_0) = 0$&lt;/span&gt;. In your case, the unknown value is &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, so we start by writing down the problem in terms of a function &lt;span class=&quot;math-container&quot;&gt;$f(c)$&lt;/span&gt; that is zero when &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; takes the &quot;correct&quot; value (which we call &lt;span class=&quot;math-container&quot;&gt;$c_0$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this looks like homework, I don't want to give you the solution right away, but just help you find it yourself.&#xA;On one hand, you have the total time it took to drive for the whole journey, &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. On the other hand, you have the distance and speed (to be corrected by &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;) associated with each segment. Now, the equation you need to solve looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;t = T(c_0)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$T(c)$&lt;/span&gt; is the time it takes to drive all the segments as a function of &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. Clearly, &lt;span class=&quot;math-container&quot;&gt;$T(c)$&lt;/span&gt; is a function of the sets of &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; (and its derivation is left to you!). Finding the correct value of &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; thus reduces to finding the root of the function &lt;span class=&quot;math-container&quot;&gt;$T(c) - t$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27040" LastEditorUserId="20688" LastEditDate="2018-09-24T14:43:18.927" LastActivityDate="2018-09-24T14:43:18.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30260" PostTypeId="1" AcceptedAnswerId="30273" CreationDate="2018-09-24T16:34:26.830" Score="1" ViewCount="342" Body="&lt;p&gt;I am trying to compute the following integration&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int_0^\infty e^{-y}y^{a/2}L_c^b(y)L_e^d(y)dy&#xA;$$&lt;/span&gt;&#xA;using the generalized Gauss-Laguerre quadrature routine in the GNU Scientific Library. Here the &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;'s are the generalized Laguerre functions, and &lt;span class=&quot;math-container&quot;&gt;$a&amp;gt;-1, b&amp;gt;0, d&amp;gt;0$&lt;/span&gt;. &#xA;I am not sure how to choose the number of quadrature points, different numbers give different values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Edit: MWE)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_math.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_integration.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_sf_laguerre.h&amp;gt;&#xA;&#xA;struct data { double b; int c; double d; int e; };&#xA;&#xA;double f(double y, void* userdata) {&#xA;    struct data *d = (struct data *) userdata;&#xA;    return gsl_sf_laguerre_n(d-&amp;gt;c, d-&amp;gt;b, y) * gsl_sf_laguerre_n(d-&amp;gt;e, d-&amp;gt;d, y);&#xA;}&#xA;&#xA;int main() {&#xA;    const double a = -0.5;&#xA;    const double b = 0.5;&#xA;    const int c = 0;&#xA;    const double d = 0.5;&#xA;    const int e = 0;&#xA;    const size_t num_nodes = (c + e + 1) / 2;&#xA;&#xA;    const gsl_integration_fixed_type *T = gsl_integration_fixed_laguerre;&#xA;    gsl_integration_fixed_workspace *w&#xA;        = gsl_integration_fixed_alloc(T, num_nodes, 0.0, 1.0, 0.5 * a, 0.0);&#xA;&#xA;    struct data params = { b, c, d, e };&#xA;    gsl_function F;&#xA;    F.function = &amp;amp;f;&#xA;    F.params = &amp;amp;params;&#xA;&#xA;    double result;&#xA;    gsl_integration_fixed(&amp;amp;F, &amp;amp;result, w);&#xA;    printf(&quot;%12.e\n&quot;, result);&#xA;&#xA;    gsl_integration_fixed_free(w);&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20647" LastEditorUserId="20647" LastEditDate="2018-09-27T12:53:42.233" LastActivityDate="2018-09-27T13:48:08.167" Title="How many quadrature points should I use?" Tags="|quadrature|integration|gsl|" AnswerCount="2" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="30261" PostTypeId="5" CreationDate="2018-09-24T18:11:46.763" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-24T18:11:46.763" LastActivityDate="2018-09-24T18:11:46.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30262" PostTypeId="4" CreationDate="2018-09-24T18:11:46.763" Score="0" Body="Computational science questions regarding GNU Scientific Library" OwnerUserId="20688" LastEditorUserId="9667" LastEditDate="2018-09-24T22:18:55.657" LastActivityDate="2018-09-24T22:18:55.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30265" PostTypeId="2" ParentId="1106" CreationDate="2018-09-26T04:21:56.607" Score="4" Body="&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$X\sim \text{Normal}(\mu,\sigma)$&lt;/span&gt;, then the &lt;span class=&quot;math-container&quot;&gt;$Y\sim \text{Half-Normal}$&lt;/span&gt; is obtainable via several approaches. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Absolute Value:&lt;/strong&gt;   &lt;span class=&quot;math-container&quot;&gt;$\quad Y = |X|\quad $&lt;/span&gt; (As pointed out by @DavidZ.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Truncation:&lt;/strong&gt;   &lt;span class=&quot;math-container&quot;&gt;$\quad Y = X_{(0,\infty)} \quad $&lt;/span&gt; MATLAB does this nicely with &lt;a href=&quot;https://www.mathworks.com/help/stats/prob.normaldistribution.truncate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;truncate()&lt;/code&gt;&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Conditioning (Logical Indexing):&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$\quad Y = (X|X&amp;gt;0)\quad $&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Graphical examples below with code.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ir6Al.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ir6Al.png&quot; alt=&quot;Truncation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% MATLAB R2018b&#xA;mu = 0; sigma = 1;&#xA;pdN = makedist('Normal',mu,sigma);&#xA;pdHN = truncate(pdN,0,inf);&#xA;&#xA;X = (-4:.01:4)';&#xA;Xt = X(X&amp;gt;0);&#xA;&#xA;figure, hold on, box on&#xA;p(1) = plot(X,pdf(pdN,X),'k-','DisplayName','Normal')&#xA;p(2) = plot(Xt,pdf(pdHN,Xt),'b-','DisplayName','HalfNormal')&#xA;YAxis = get(gca,'YTick');&#xA;plot([0;0],[0,YAxis(end)],'b--','HandleVisibility','off')&#xA;legend('show')&#xA;set(p,'LineWidth',1.8)&#xA;ylabel('Probability Density Function (PDF)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G6Cn5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G6Cn5.png&quot; alt=&quot;Logical Indexing&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y = random(pdN,50000,1);&#xA;Yt = Y(Y&amp;gt;0);&#xA;&#xA;figure, hold on, box on&#xA;histogram(Y,'Normalization','pdf','FaceColor','k','DisplayName','Normal')&#xA;histogram(Yt,'Normalization','pdf','FaceColor','b','DisplayName','HalfNormal')&#xA;legend('show')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28098" LastEditorUserId="28098" LastEditDate="2019-06-20T23:52:37.130" LastActivityDate="2019-06-20T23:52:37.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30267" PostTypeId="2" ParentId="30260" CreationDate="2018-09-26T16:47:53.753" Score="2" Body="&lt;p&gt;The following program will integrate your integral with GSL. I was unable to reproduce the result you quoted with Mathematica. Note that when &lt;span class=&quot;math-container&quot;&gt;$c = e = 0$&lt;/span&gt;, the Laguerre polynomials are always &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; for any value of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. The code below does a slightly more interesting case of &lt;span class=&quot;math-container&quot;&gt;$c = 2, e = 3$&lt;/span&gt;. The output is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result = 1.539250228072e+00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Mathematica's output is &lt;span class=&quot;math-container&quot;&gt;$1.539250228072043$&lt;/span&gt;. When &lt;span class=&quot;math-container&quot;&gt;$c = e = 0$&lt;/span&gt; as you originally posted, both GSL and Mathematica produce the result &lt;span class=&quot;math-container&quot;&gt;$1.225416702465178$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;&#xA;#include &amp;lt;gsl/gsl_math.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_specfunc.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_integration.h&amp;gt;&#xA;&#xA;struct data&#xA;{&#xA;  double b;&#xA;  int c;&#xA;  double d;&#xA;  int e;&#xA;};&#xA;&#xA;double&#xA;f(double x, void * params)&#xA;{&#xA;  struct data *d = (struct data *) params;&#xA;  double Lcb = gsl_sf_laguerre_n(d-&amp;gt;c, d-&amp;gt;b, x);&#xA;  double Led = gsl_sf_laguerre_n(d-&amp;gt;e, d-&amp;gt;d, x);&#xA;  return Lcb * Led;&#xA;}&#xA;&#xA;int&#xA;main()&#xA;{&#xA;  const gsl_integration_fixed_type * T = gsl_integration_fixed_laguerre;&#xA;  const double a = -0.5;&#xA;  const double b = 0.5;&#xA;  const int c = 2;&#xA;  const double d = 0.5;&#xA;  const int e = 3;&#xA;  const size_t n = (c + e + 1) / 2;&#xA;  gsl_integration_fixed_workspace * w =&#xA;    gsl_integration_fixed_alloc(T, n, 0.0, 1.0, 0.5 * a, 0.0);&#xA;  gsl_function F;&#xA;  double result;&#xA;  struct data data_params = { b, c, d, e };&#xA;&#xA;  F.function = &amp;amp;f;&#xA;  F.params = &amp;amp;data_params;&#xA;&#xA;  gsl_integration_fixed(&amp;amp;F, &amp;amp;result, w);&#xA;  fprintf(stderr, &quot;result = %.12e\n&quot;, result);&#xA;&#xA;  gsl_integration_fixed_free(w);&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28440" LastActivityDate="2018-09-26T16:47:53.753" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30269" PostTypeId="1" AcceptedAnswerId="30274" CreationDate="2018-09-27T07:25:02.370" Score="2" ViewCount="180" Body="&lt;p&gt;I am currently enjoying writing computational codes as a hobby. Right now I've worked out an Euler method and results are pretty good with up to &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;. Over &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;, instability starts to kick in. May I know why is this so? The equations are as below:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;df=-\frac{x^2}{y},\quad y(0)=1&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Euler Method&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;#Specify Step Size:&#xA;h=float(input(&quot;Size of h:&quot;))&#xA;#Key in X value you want to obtain y(x1):&#xA;x1=float(input(&quot;input X to get Y:&quot;))&#xA;#Number of steps:&#xA;n=int((x1)/h)+1&#xA;#Creation of array table&#xA;A=np.zeros((n,3))&#xA;A[0]=[1,0,1]&#xA;for i in range(1,n):&#xA;#Evaluating values of y:&#xA;    A[i,0]=i+1&#xA;    df=-(A[i-1,1]**2)/A[i-1,2]&#xA;    A[i,2]=A[i-1,2]+h*df&#xA;    A[i,1]=A[i-1,1]+h&#xA;print(A)&#xA;plt.plot(A[:,1], A[:,2], label='Approximation')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LN7SQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LN7SQ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NdX8G.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NdX8G.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jMJq7.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jMJq7.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28830" LastEditorUserId="20688" LastEditDate="2018-09-27T13:10:32.680" LastActivityDate="2018-09-27T14:39:58.120" Title="Euler Method Instability. Why?" Tags="|ode|runge-kutta|numerics|explicit-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30270" PostTypeId="1" CreationDate="2018-09-27T08:35:34.683" Score="2" ViewCount="158" Body="&lt;p&gt;I am trying to solve a non-linear time-dependent heat equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\partial_tT=\nabla \left(k_T(T)\nabla T\right) + f$$&lt;/span&gt; (similar to question &lt;a href=&quot;https://scicomp.stackexchange.com/q/29660/20545&quot;&gt;Solving a non-linear heat equation with the galerkin method gives negative values&lt;/a&gt;) with a FEM-approach, using the continuous Galerkin method for the FEM part, the Newton method for linearization of the non-linear part and the implicit Euler method for the time stepping.&lt;br&gt;&#xA;Initially, I had problems with parts of the results becoming negative, which could be solved (more or less) by clipping all parts which are below a certain value. Now I got the additional problem that the solution at some nodes becomes unstable, and due to a positive feedback loop increases into unrealistic values. This leads to results as shown in the following image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/txQXY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/txQXY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;The heat source is located at the bottom, approximately at 500, which can be seen due to the darker yellow color. Still, in the middle of the image, the solution shows some nodes with values significantly higher values than the surrounding area, leading to a non-smooth result.&lt;br&gt;&#xA;Thus I was wondering if there is a method to prevent those runaways, such as a slope limiter (and if yes, how large will the error be)? I can not use a lowpass here, after those values can be reached during regular calculations.&lt;br&gt;&#xA;Or is my approach in general flawed for such an equation, and I should switch to another method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To address the comment: In general I try to solve the equations from &lt;a href=&quot;https://research.utwente.nl/en/publications/two-temperature-model-for-pulsed-laser-induced-subsurface-modific&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://research.utwente.nl/en/publications/two-temperature-model-for-pulsed-laser-induced-subsurface-modific&lt;/a&gt;, with the equation for the electron temperature (&lt;span class=&quot;math-container&quot;&gt;$T_E$&lt;/span&gt;) resulting in node spikes, while the other two equations do not make any problems. The boundary conditions are Neumann conditions with &lt;span class=&quot;math-container&quot;&gt;$dT/dt=0$&lt;/span&gt;. The domain itself is defined as a square from (0, 0) to (1, 1), due to the problem being dimensionless, while the basis functions are polynomials.&lt;/p&gt;&#xA;" OwnerUserId="20545" LastEditorUserId="20545" LastEditDate="2018-09-29T17:50:02.770" LastActivityDate="2018-09-29T17:50:02.770" Title="Prevent single node spikes in a FEM-simulation (using continuous Galerkin)" Tags="|finite-element|nonlinear-equations|smoothing|galerkin|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30271" PostTypeId="1" CreationDate="2018-09-27T09:22:57.670" Score="2" ViewCount="743" Body="&lt;p&gt;I need to solve the following equation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{pmatrix} &#xA;\frac{\omega^2}{c^2}\varepsilon_x-\mu_z^{-1}k_y^2-\mu_y^{-1}k_z^2 &amp;amp; \mu_z^{-1}k_xk_y &amp;amp; \mu_y^{-1}k_xk_z\\&#xA;\mu_z^{-1}k_xk_y &amp;amp;\frac{\omega^2}{c^2}\varepsilon_y-\mu_z^{-1}k_x^2-\mu_x^{-1}k_z^2 &amp;amp;\mu_x^{-1}k_yk_z\\&#xA;\mu_y^{-1}k_xk_z &amp;amp; \mu_x^{-1}k_yk_z&amp;amp;\frac{\omega^2}{c^2}\varepsilon_z-\mu_y^{-1}k_x^2-\mu_x^{-1}k_y^2&#xA;\end{pmatrix}&#xA;\begin{pmatrix}E_x\\E_y\\E_z\end{pmatrix}=\begin{pmatrix}0\\0\\0\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All parameters are given except &lt;span class=&quot;math-container&quot;&gt;$k_z$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E_x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E_y$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$E_z$&lt;/span&gt;. I first find &lt;span class=&quot;math-container&quot;&gt;$k_z$&lt;/span&gt; that makes the determinant of the matrix vanishes, then I find vector &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; as the corresponding null space of the matrix. I have the following codes that generate the matrix and find the values of &lt;span class=&quot;math-container&quot;&gt;$k_z$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy as sp   # np.sqrt doesn't handle complex number?&#xA;&#xA;def maxwell_matrix(k0,kx,ky,kz,eps,mu):&#xA;    kx2,ky2,kz2,k02 = kx**2,ky**2,kz**2,k0**2&#xA;    mat = sp.empty((3,3),dtype=complex)&#xA;    mat[0,0] = k02*eps[0] - ky2/mu[2] - kz2/mu[1]&#xA;    mat[1,1] = k02*eps[1] - kz2/mu[0] - kx2/mu[2]&#xA;    mat[2,2] = k02*eps[2] - kx2/mu[1] - ky2/mu[0]&#xA;    mat[0,1] = kx*ky/mu[2]&#xA;    mat[1,0] = mat[0,1]&#xA;    mat[0,2] = kx*kz/mu[1]&#xA;    mat[2,0] = mat[0,2]&#xA;    mat[1,2] = kz*ky/mu[0]&#xA;    mat[2,1] = mat[1,2]&#xA;    return mat&#xA;&#xA;def compute_kz(k0,kx,ky,eps,mu):&#xA;    eps = sp.array(eps)&#xA;    mu = sp.array(mu)&#xA;    if len(eps.shape)==1:&#xA;        eps = eps[sp.newaxis,:]&#xA;    elif len(eps.shape)==0:&#xA;        eps = eps*sp.ones((1,3))&#xA;    if len(mu.shape)==1:&#xA;        mu = mu[sp.newaxis,:]&#xA;    elif len(mu.shape)==0:&#xA;        mu = mu*sp.ones((1,3))&#xA;    kx2, ky2, k02 = kx**2, ky**2, k0**2&#xA;&#xA;    A = kx2*(eps[:,2]*mu[:,0]+eps[:,0]*mu[:,2])\&#xA;            +ky2*(eps[:,2]*mu[:,1]+eps[:,1]*mu[:,2])\&#xA;            -k02*eps[:,2]*mu[:,2]*(eps[:,1]*mu[:,0]+eps[:,0]*mu[:,1])&#xA;    B = (kx2*mu[:,0] + mu[:,1]*(ky2-k02*eps[:,2]*mu[:,0]))\&#xA;        *(kx2*eps[:,0] + eps[:,1]*(ky2-k02*eps[:,0]*mu[:,2]))&#xA;&#xA;    epsz_muz = eps[:,2]*mu[:,2]&#xA;    C = sp.sqrt(A**2 - 4*epsz_muz*B)&#xA;&#xA;kz1 = sp.sqrt(0.5*(-A+C)/epsz_muz)&#xA;kz2 = sp.sqrt(0.5*(-A-C)/epsz_muz)&#xA;&#xA;return sp.transpose(sp.array([kz1,kz2,-kz1,-kz2]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The equations for calculating &lt;span class=&quot;math-container&quot;&gt;$k_z$&lt;/span&gt; are found using Mathematica and it should be correct as I have tested it many times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried two methods for finding the null space. The first method is SVD decomposition, and the second one is to find the eigenvector with eigenvalue zero. The following code does this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import scipy.linalg as spl&#xA;&#xA;k0 = 1.0&#xA;theta = 37*sp.pi/180  # just some random number&#xA;phi = 0*sp.pi/180&#xA;eps = (2*np.random.rand(3)+1) + 0.1j*np.random.rand(3)&#xA;mu = (np.random.rand(3)+1)&#xA;&#xA;kx = k0*sp.sin(theta)*sp.cos(phi)&#xA;ky = k0*sp.sin(theta)*sp.sin(phi)&#xA;&#xA;kz = compute_kz(k0,kx,ky,eps,mu)[0]&#xA;&#xA;&#xA;tol = 1e-9&#xA;for i in range(4):&#xA;    mat = maxwell_matrix(k0,kx,ky,kz[i],eps,mu)  &#xA;    print('det:',spl.det(mat))   # this confirms kz is correct&#xA;&#xA;    # SVD decomposition&#xA;    U, s, Vh = spl.svd(mat)&#xA;    idx = sp.argmin(sp.absolute(s))&#xA;    if sp.absolute(s[idx]) &amp;lt; tol:&#xA;        evect = Vh[idx]&#xA;        evect = evect/spl.norm(evect)&#xA;    else:&#xA;        raise ValueError(&quot;no solution found!&quot;)&#xA;    dot_prod = sp.dot(mat,evect)&#xA;    print('SVD vector:', evect)&#xA;    print('SVD dot prod: ({x[0]:.3e}  {x[1]:.3e}  {x[2]:.3e})    abs:{y:.3e}'.format(x=dot_prod, y=spl.norm(dot_prod)))&#xA;&#xA;&#xA;    # find eigenvector with zero eigenvalue&#xA;    w,v = sp.linalg.eig(mat)&#xA;    idx = sp.argmin(sp.absolute(w))&#xA;    if sp.absolute(w[idx]) &amp;lt; tol:&#xA;        evect = v[:,idx]&#xA;        evect = evect/spl.norm(evect)&#xA;    else:&#xA;        raise ValueError(&quot;no solution found!&quot;)&#xA;    dot_prod = sp.dot(mat,evect)&#xA;    print('method2 vector:', evect)&#xA;    print('method2 dot prod: ({x[0]:.3e}  {x[1]:.3e}  {x[2]:.3e})    abs:{y:.3e}'.format(x=dot_prod, y=spl.norm(dot_prod)))&#xA;    print()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The strange thing is that when &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt; is real, both methods seem to give the same answer, but when &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt; is complex, then SVD decomposition seems to fail. It agrees with method 2 for one set of &lt;span class=&quot;math-container&quot;&gt;$k_z$&lt;/span&gt;, but for the other set of &lt;span class=&quot;math-container&quot;&gt;$k_z$&lt;/span&gt; the computed dot product is clearly zero. What's wrong with SVD decomposition/my method for finding null space? What's the best way to find the null space of a &lt;span class=&quot;math-container&quot;&gt;$3\times 3$&lt;/span&gt; matrix, which is in general complex?&lt;/p&gt;&#xA;" OwnerUserId="21426" LastEditorUserId="20688" LastEditDate="2018-09-27T13:46:09.650" LastActivityDate="2018-09-27T13:46:09.650" Title="finding null space to a complex matrix" Tags="|linear-algebra|matrix|scipy|svd|nullspace|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30272" PostTypeId="2" ParentId="30270" CreationDate="2018-09-27T12:14:29.680" Score="1" Body="&lt;p&gt;I believe these issues arise because you are solving the transient form of the heat equation which, being parabolic, can lead to some instability for continuous galerkin method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are ways to circumvent that. GGLS approaches (Galerkin Gradient Least Square) introduce stabilization term that dampen these oscillations by minimizing the Error on temperature not only in the L2 space but in H1.&#xA;You can see this article by Ilinca and Hétu which is very detailed:&#xA;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0045782502002426&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0045782502002426&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also alternatives like the source stabilized petrov-galerkin stabilization (SSPG). There is also very good work by Ilinca on this :&#xA;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/abs/10.1002/nme.2324&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://onlinelibrary.wiley.com/doi/abs/10.1002/nme.2324&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer : F. Ilinca is someone I know.&lt;/p&gt;&#xA;" OwnerUserId="7452" LastActivityDate="2018-09-27T12:14:29.680" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30273" PostTypeId="2" ParentId="30260" CreationDate="2018-09-27T13:43:03.630" Score="1" Body="&lt;p&gt;The way printf formats work, a call like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;printf(&quot;%12.e\n&quot;, result);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;instructs printf to print a number with 12 digits/characters/spaces (of &lt;em&gt;width&lt;/em&gt; 12), with &lt;strong&gt;NO&lt;/strong&gt; digits after the decimal point, just like in the precision specifier &lt;code&gt;&quot;12.&quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing it to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;printf(&quot;%.12e\n&quot;, result);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;results in printing &lt;code&gt;1.225416702465e+00&lt;/code&gt;, which is probably what was intended, and which matches vibe's answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://en.cppreference.com/w/cpp/io/c/fprintf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.cppreference.com/w/cpp/io/c/fprintf&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(optional) . followed by integer number or *, or neither that specifies precision of the conversion. In the case when * is used, the precision is specified by an additional argument of type int. If the value of this argument is negative, it is ignored. &lt;strong&gt;If neither a number nor * is used, the precision is taken as zero.&lt;/strong&gt; See the table below for exact effects of precision. [Emphasis added]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2018-09-27T13:48:08.167" LastActivityDate="2018-09-27T13:48:08.167" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30274" PostTypeId="2" ParentId="30269" CreationDate="2018-09-27T14:39:58.120" Score="5" Body="&lt;p&gt;If we take your ODE, &lt;span class=&quot;math-container&quot;&gt;$$\frac{dy}{dx}=-\frac{x^2}{y},$$&lt;/span&gt; multiply both sides by &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and integrate up, we see that the solutions look like&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ y^2 = C-\frac{2}{3} x^3. $$&lt;/span&gt;&#xA;Taking your initial condition, your real trajectory is then &lt;span class=&quot;math-container&quot;&gt;$$ y = \sqrt{1- \frac{2}{3} x}. $$&lt;/span&gt; &#xA;This is only real valued for the domain &lt;span class=&quot;math-container&quot;&gt;$-\infty &amp;lt; x&amp;lt;=1.5 $&lt;/span&gt;, so it's no real surprise that you're seeing odd behaviour out past &lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;, since your &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; value is getting small and your derivative is getting singular (i.e. heading towards infinity). In practise, the forward Euler method overshoots, so that you cross the cut line at &lt;span class=&quot;math-container&quot;&gt;$y=0$&lt;/span&gt; between the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-positive and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-negative solutions, hence the oscillations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wanted to do better, you could try an implicit method with an adaptive time step, but you are generally trying to numerically approximate a not very nicely behaved function.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2018-09-27T14:39:58.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30275" PostTypeId="2" ParentId="29427" CreationDate="2018-09-27T15:17:46.113" Score="2" Body="&lt;p&gt;Here is a slighly simpler solution to Anton's method. The GSL &lt;code&gt;hyperg_2F1&lt;/code&gt; function will raise a domain error flag as well as set the return value to &lt;code&gt;NaN&lt;/code&gt; for input values it doesn't like. The integration routines know how to handle &lt;code&gt;NaN&lt;/code&gt; return values, so all you need to do is turn off the GSL error handler. This can be accomplished as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_integration.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_sf_hyperg.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_errno.h&amp;gt;&#xA;&#xA;double f (double x, void * params) {&#xA;  double alpha = *(double *) params;&#xA;  return gsl_sf_hyperg_2F1(1,2,3,alpha*x);&#xA;}&#xA;&#xA;int&#xA;main (void)&#xA;{&#xA;  gsl_integration_cquad_workspace * wcquad = gsl_integration_cquad_workspace_alloc (10000);&#xA;&#xA;  double result, error;&#xA;  size_t nevals;&#xA;  double expected = log(16);&#xA;  double alpha = 1.0;&#xA;&#xA;  gsl_set_error_handler_off();&#xA;&#xA;  gsl_function F;&#xA;  F.function = &amp;amp;f;&#xA;  F.params = &amp;amp;alpha;&#xA;&#xA;  gsl_integration_cquad (&amp;amp;F, -1.0, 1.0, 0, 1e-14, wcquad, &amp;amp;result, &amp;amp;error,&amp;amp;nevals);&#xA;&#xA;  printf (&quot;result          = % .18f\n&quot;, result);&#xA;  printf (&quot;exact result    = % .18f\n&quot;, expected);&#xA;  printf (&quot;estimated error = % .18f\n&quot;, error);&#xA;  printf (&quot;actual error    = % .18f\n&quot;, result - expected);&#xA;  printf (&quot;# evaluations   = %zu\n&quot;, nevals);&#xA;&#xA;  gsl_integration_cquad_workspace_free (wcquad);&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the output I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result          =  2.772588722239768266&#xA;exact result    =  2.772588722239781145&#xA;estimated error =  0.000000000000305493&#xA;actual error    = -0.000000000000012879&#xA;# evaluations   = 1565&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28440" LastActivityDate="2018-09-27T15:17:46.113" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30276" PostTypeId="1" AcceptedAnswerId="30281" CreationDate="2018-09-27T20:40:21.680" Score="4" ViewCount="651" Body="&lt;p&gt;How well do explicit Runge-Kutta &quot;tableau&quot; methods compare to the state of the art ODE solvers and when do they fail? I've been reading Butcher's ODE book and he does a good job at introducing tableaus to represent different Runge-Kutta methods. This includes methods with built-in error estimates for adaptive steps sizes like Fehlberg, Vergner, or Dorman-Prince. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't understand is how these methods compare to the current state-of-the-art in ODE solvers. Are these methods ever really used or are they simply discussed for historical reasons? If they are used, what's their general limitation or domain of use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a minor aside, I noticed that &lt;a href=&quot;http://juliadiffeq.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt; appeared to implement just about every known tableau method. That said, they don't appear to be the default methods used, which also caused me to question whether or not they're still of modern use.&lt;/p&gt;&#xA;" OwnerUserId="10094" LastEditorUserId="20688" LastEditDate="2018-09-28T16:11:51.967" LastActivityDate="2020-07-19T18:03:04.980" Title="How well do explicit Runge-Kutta &quot;tableau&quot; methods compare to the state of the art ODE solvers and when do they fail?" Tags="|ode|runge-kutta|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30277" PostTypeId="2" ParentId="8217" CreationDate="2018-09-27T22:16:13.547" Score="9" Body="&lt;p&gt;The basic difference is simply the meaning to be attached to the results. FDM predicts point values of any aspect of the solution. Interpolation between these values is often left to the imagination of the user. FVM predicts averages of conserved variables within specific control volumes. Hence it predicts the integrated conserved variables and can be shown to converge to weak (discontinuous) solutions. FEM gives a set of discrete values from which an approximate solution can be deduced unambiguously everywhere by invoking a set of basis functions. Usually, but not necessarily, the variables involved are conservative. It is possible to have finite difference methods that are conservative in some sense, according to a particular quadrature rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are matters of definition. There are many variations of all three methods. Not every method is cleanly of one type, and the details vary between application areas. Researchers inventing a new method employ those tools that will help to provide the properties that they are looking for. It is, as you seem to have found, difficult to find an authoritative discussion and it would be difficult for me to provide one. The best advice that I can give is to continue reading, without expecting a totally clear answer, but giving credence to the things that make sense to you.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2018-09-27T22:16:13.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30278" PostTypeId="2" ParentId="30276" CreationDate="2018-09-28T04:32:01.180" Score="3" Body="&lt;p&gt;DifferentialEquations.jl has the best survey of current solvers. It goes through every option and benchmarks them. Modern RK tableaus are the state-of-the-art in most non-stiff situations. Multistep methods are good with expensive non-stiff functions. For stiff problems, it varies. See:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://diffeq.sciml.ai/latest/solvers/ode_solve.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://devdocs.juliadiffeq.org/latest/internals/notes_on_algorithms.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://devdocs.juliadiffeq.org/latest/internals/notes_on_algorithms.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The two documents occasionally contradict each other, such as recommending different algorithms at order 7 and 8&lt;s&gt;, so you'll have to ask Chris Rackauckas about that&lt;/s&gt; (see Chris's comment below for the reason). On the second link, there's an implicit understanding that higher order methods are useful when the desired tolerance shrinks, but also come with speed drawbacks at looser tolerances.&lt;/p&gt;&#xA;&lt;p&gt;You'll also want to learn about stepsize control since many of the best algorithms rely on it and leave it as a separate problem. DiffEq has &lt;a href=&quot;http://diffeq.sciml.ai/latest/extras/timestepping.html#Proportional-Integral-Control-(PI-Control)-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this short description&lt;/a&gt; but I think &lt;a href=&quot;https://link.springer.com/article/10.1007/BF01934091&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper is more understandable&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28631" LastEditorUserId="18981" LastEditDate="2020-07-19T18:03:04.980" LastActivityDate="2020-07-19T18:03:04.980" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30279" PostTypeId="1" CreationDate="2018-09-28T06:31:02.190" Score="8" ViewCount="470" Body="&lt;p&gt;Writing &lt;a href=&quot;https://space.stackexchange.com/questions/30952/how-did-the-apollo-computers-evaluate-transcendental-functions-like-sine-arctan?noredirect=1#comment92890_30952&quot;&gt;this comment&lt;/a&gt; reminded me of something I noticed years ago about evaluating Bessel functions of the first kind &lt;span class=&quot;math-container&quot;&gt;$J_n(x)$&lt;/span&gt; in Excel. (&lt;a href=&quot;https://support.office.com/en-us/article/besselj-function-839cb181-48de-408b-9d80-bd02982d94f7&quot; rel=&quot;noreferrer&quot;&gt;BESSELJ&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't use Excel now but at the time I'd checked MacOS and Windows computers with several different released versions of Excel and they all had the same problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was circa 2015.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the argument passed through &lt;span class=&quot;math-container&quot;&gt;$x=8$&lt;/span&gt;, there was a step discontinuity of order &lt;span class=&quot;math-container&quot;&gt;$10^{-5}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$n&amp;lt;9$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotting the first difference will show that the Excel values are step-wise discontinuous at &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Is it possible to know which algorithm had this behavior?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/raq09.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/raq09.png&quot; alt=&quot;Excel Bessel Function Glitch&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pastebin.com/dq3kWhA6&quot; rel=&quot;noreferrer&quot;&gt;Python source file is available&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example from Excel:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   x         J1             J3            J5&#xA;7.9997,  0.234593634,  -0.291131046,  0.185841208,&#xA;7.9998,  0.234607873,  -0.291131434,  0.185819063,&#xA;7.9999,  0.234622108,  -0.291131819,  0.185796918,&#xA;8.0000,  0.234628387,  -0.291125242,  0.185775021,&#xA;8.0001,  0.234642617,  -0.291125622,  0.185752874,&#xA;8.0002,  0.234656846,  -0.291126,     0.185730726,&#xA;8.0003,  0.234671072,  -0.291126376,  0.185708577&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit:&lt;/strong&gt;  per request:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n    x=7.99999952316284            x=8             x=8.00000095367431&#xA;--   ------------------    ------------------      ------------------&#xA;0     0.17165091838403      0.171650807317694       0.171650583550973&#xA;1     0.2346362739686       0.234628386507074       0.234628522235498&#xA;2    -0.112991846395527    -0.112993710690925      -0.112993459984572&#xA;3    -0.291132200533783    -0.291125241852537      -0.112993459984572&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17869" LastEditorUserId="17869" LastEditDate="2020-06-15T09:28:48.563" LastActivityDate="2022-05-22T17:09:43.283" Title="What algorithm does (or did?) Excel use for Bessel functions that is discontinuous at x=8?" Tags="|approximation|special-functions|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="30280" PostTypeId="2" ParentId="30276" CreationDate="2018-09-28T07:01:28.820" Score="4" Body="&lt;p&gt;Runge-Kutta methods are arguably the most widely used methods for initial value ODEs (including also PDE semi-discretizations).  There are particular types of Runge-Kutta methods that are applicable to virtually every class of initial value problem.  They're certainly not just a historical artifact.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2018-09-28T07:01:28.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30281" PostTypeId="2" ParentId="30276" CreationDate="2018-09-28T16:22:58.370" Score="8" Body="&lt;p&gt;I think the confusion here is what exactly is a &quot;tableau method&quot; since it's used in our docs in a very specific way. A &quot;tableau method&quot; in DifferentialEquations.jl parlance is a method which is implemented by explicitly building the arrays for the tableaus and performing loops on said arrays. The tableaus are stored in DiffEqDevTools.jl in the highest precision published and transform at compile-time to the right precision before solving. Here is Tsitouras' 5th order method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqDevTools.jl/blob/master/src/ode_tableaus.jl#L866-L920&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/JuliaDiffEq/DiffEqDevTools.jl/blob/master/src/ode_tableaus.jl#L866-L920&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can directly use these with the ExplicitRK method. Here is its internal &lt;code&gt;perform_step&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/perform_step/explicit_rk_perform_step.jl#L72-L119&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/perform_step/explicit_rk_perform_step.jl#L72-L119&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as another post mentioned, when we first did this we did a literature review of all of the tableaus we could find, implemented them, tested them, looked at stability, etc. (that 8000 line page of tableaus :) ) and then specialized some of the algorithms. For example, for Tsitoras' method we pulled it apart into a stack-allocated tableau&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/tableaus/low_order_rk_tableaus.jl#L497-L601&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/tableaus/low_order_rk_tableaus.jl#L497-L601&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;specialized the equations on the form of the constants &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/perform_step/low_order_rk_perform_step.jl#L609-L656&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/perform_step/low_order_rk_perform_step.jl#L609-L656&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;added a special interpolant to it, modified some default magic numbers for adaptivity, etc. to specifically optimize this algorithm. We also have a trait system which for example &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/alg_utils.jl#L346&quot; rel=&quot;noreferrer&quot;&gt;hard codes specific constants about the stability region&lt;/a&gt; necessary for performing stiffness detection and automatic switching to implicit methods. And we did that for a few other RK methods. A lot of these optimizations only truly matter if &lt;code&gt;f&lt;/code&gt; is inexpensive, but since that shows up in a lot of cases (parameter estimation and searches, analysis of chaotic systems) we took the time to do this, tested it, and performance-wise would recommend these implementations over the pure tableau-based form. Note that it is mathematically the same method, but this formulation does not require loops over constants and instead is in a form that LLVM optimizes very well. When the equation is small, Julia will in fact in line the user's &lt;code&gt;f&lt;/code&gt; derivative function and the solver will construct something entirely different and efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that's just optimizations and just a DifferentialEquations.jl thing. Explicit RK methods, implicit RK methods, Rosenbrock methods, etc. can all be discussed as methods in a tableau form, and these are all quite good methods. The Tsitorious method which was shown above is actually one of the core defaults of the ecosystem, and a similar method is the Dormand-Prince method which you might know of as &lt;code&gt;ode45&lt;/code&gt;. These are still quite state-of-the-art (at least for non-stiff equations, and when paired with PI-adaptivity and their special interpolants). These are the methods you see recommended everywhere, along with a few non-tableau methods like multistep Adams and BDF. Due to the benchmarks we have we recommend the hand-tuned versions, but it's stepping behavior are mathematically the same as if you looped through arrays of constants (sans the specializations on constants and interpolations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your aside, our test showed that not every tableau is worth it. Some just aren't efficient. Over time, people have gotten better at choosing coefficients which reduce the leading truncation error term while keeping the stability region the same size. The older methods then take similar steps but result in a solution with more error. We keep them there for research purposes, and also it's a nice archive of the literature since some of those are hard to find.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-09-28T16:22:58.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30282" PostTypeId="1" CreationDate="2018-09-30T14:50:59.013" Score="0" ViewCount="189" Body="&lt;p&gt;I am developing a FEM fluid solver using the Taylor-Hood algorithm, i.e. quadratic interpolation for velocity, and linear for pressure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have developed the code for 2-D quadrilaterals and triangles, and everything worked fine.&#xA;I am trying to do it in 3D, using a combination of 20-noded and 8-noded hexahedrals, and I have the following problem. The velocity solution is &#xA;correct and converging, but the pressure solution is very unstable. Pressure is not converging. It doesn't diverge either. I tried with under-relaxation and the problem remained.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code looks alright (I checked it several times), but there's always a possibility to have a problem somewhere. Am I missing something in the physics? The pressure is zeroed at one point, and the problem I am trying to solve is the simple 3D lid driven cavity flow, where velocity is prescribed on all boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you suggest me any relative document?&lt;/p&gt;&#xA;" OwnerUserId="28848" LastEditorUserId="20688" LastEditDate="2018-09-30T16:12:22.917" LastActivityDate="2018-10-01T18:33:25.350" Title="Taylor-Hood finite hexahedral elements, pressure diverging" Tags="|finite-element|navier-stokes|incompressible|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30283" PostTypeId="2" ParentId="30282" CreationDate="2018-09-30T16:28:47.290" Score="2" Body="&lt;p&gt;It's of course hard to tell without knowing the code, the testcase, and your results. So I can't really help you out here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it &lt;em&gt;is&lt;/em&gt; worth pointing out that you are making yourself too much work. Any of the big, open source finite element libraries have these elements already built in and exceedingly well tested. They also have adaptive meshes, excellent solvers, and parallelism available. There is really no reason for you to implement all of this yourself -- save your time and energy to implement the things that are &lt;em&gt;specific&lt;/em&gt; to your problem, not generic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I am one of the authors of one of these large libraries -- &lt;a href=&quot;http://dealii.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;deal.II&lt;/a&gt;. You may be interested in looking at the &lt;a href=&quot;https://dealii.org/developer/doxygen/deal.II/step_22.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stokes equation tutorial program&lt;/a&gt;. You can make your way from there to the Navier-Stokes solvers.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-09-30T16:28:47.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30285" PostTypeId="2" ParentId="30117" CreationDate="2018-09-30T21:41:28.727" Score="1" Body="&lt;p&gt;&quot;A New Kind of Science&quot; (NKS) has been brought up, in this case for a very legitimate reason; this book provides a lot of information on the actual rules generation &amp;amp; analysis, universal computation, etc. However, I feel the need to add a link to the famous &lt;a href=&quot;http://bactra.org/reviews/wolfram/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Review of &quot;A New Kind of Science&quot; by C. Shalizi&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I bring it up &lt;strong&gt;not for the reason of yet another debate on the NKS&lt;/strong&gt; (only partially to add a perspective), but for the links to other (actually Wolfram's early papers are very valuable as well) people's works that are mentioned in it and should be valuable, most importantly:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1017/CBO9780511524714&quot; rel=&quot;nofollow noreferrer&quot;&gt;D. Rothman and S. Zaleski, &lt;em&gt;Lattice Gas Cellular Automata&lt;/em&gt;, Cambridge Univ. Press, 1997&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-09-30T23:50:16.187" LastActivityDate="2018-09-30T23:50:16.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30287" PostTypeId="1" CreationDate="2018-10-01T06:45:25.000" Score="7" ViewCount="223" Body="&lt;p&gt;I have the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{bmatrix}A &amp;amp;B \\C&amp;amp; D\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix} = \begin{bmatrix}\lambda I_m &amp;amp; 0 \\ 0&amp;amp; \mu I_n\end{bmatrix}\begin{bmatrix}x \\y\end{bmatrix}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $A$, $B$, $C$, $D$ are matrices, $I_m,I_n$ are identity matrices of sizes $m,n$, $x,y$ are vectors and $\lambda$ and $\mu$ are scalars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does one solve such a problem?&lt;/p&gt;&#xA;" OwnerUserId="1870" LastEditorUserId="9667" LastEditDate="2018-10-01T12:35:24.413" LastActivityDate="2018-10-01T12:35:24.413" Title="Solving a coupled eigen value problem" Tags="|eigenvalues|eigensystem|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="30288" PostTypeId="2" ParentId="30282" CreationDate="2018-10-01T11:18:21.297" Score="1" Body="&lt;p&gt;I resolved my problem. I added a small pressure coefficient in the continuity equation, let's say &lt;span class=&quot;math-container&quot;&gt;$10^{-10}$&lt;/span&gt;. In the weak form, this quantity is multiplied by the integral {&lt;span class=&quot;math-container&quot;&gt;$N^T$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;} (or mass matrix), where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; are the shape functions of the 8 noded hexahedral. The code now works fine.&lt;/p&gt;&#xA;" OwnerUserId="28848" LastEditorUserId="20688" LastEditDate="2018-10-01T18:33:25.350" LastActivityDate="2018-10-01T18:33:25.350" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30290" PostTypeId="1" AcceptedAnswerId="30973" CreationDate="2018-10-02T10:03:40.323" Score="5" ViewCount="466" Body="&lt;p&gt;Many dense linear algebra decompositions (QR, SVD...) on an &lt;span class=&quot;math-container&quot;&gt;$m\times n$&lt;/span&gt; matrix have cost&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;O(\max(m,n)\min(m,n)^2)&#xA;$$&lt;/span&gt;&#xA;when implemented in practice on a computer. Is there a colloquial name or a more compact notation for this complexity? For instance, something like &quot;linearithmic&quot;? Or, how would you write it in the most compact and understandable form? Think about writing it in an abstract, for instance.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2018-10-02T11:40:12.360" LastActivityDate="2021-08-20T19:42:16.210" Title="Term for the typical &quot;linear in the larger dimension, quadratic in the smaller&quot; cost for linear algebra" Tags="|linear-algebra|reference-request|matrix-factorization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30291" PostTypeId="1" AcceptedAnswerId="30293" CreationDate="2018-10-02T16:22:00.290" Score="0" ViewCount="889" Body="&lt;p&gt;I have a vague understanding of the compatibility equations for linear elasticity. They appear to be necessary in obtaining a unique displacement field. However, why is it that, in my papers I've come across, the compatibility equations aren't solved for in the linear elastic numerical formulation? Are they not a governing set of equations for linear elasticity? &lt;/p&gt;&#xA;" OwnerUserId="28865" LastActivityDate="2018-10-02T20:20:08.483" Title="Purpose of compatibility equations in linear elasticity" Tags="|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30292" PostTypeId="2" ParentId="30290" CreationDate="2018-10-02T17:57:58.960" Score="4" Body="&lt;p&gt;Many authors avoid the problem entirely using the terms &quot;tall and skinny&quot; matrix, as in following example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Let &lt;span class=&quot;math-container&quot;&gt;$m \gg n$&lt;/span&gt; and let &lt;span class=&quot;math-container&quot;&gt;$A \in \mathbb{R}^{m \times n}$&lt;/span&gt; be a tall and skinny matrix. The QR decomposition of &lt;span class=&quot;math-container&quot;&gt;$A = QR$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$R \in \mathbb{R}^{n \times n}$&lt;/span&gt;) can be computed using &lt;span class=&quot;math-container&quot;&gt;$O(mn^2)$&lt;/span&gt; arithmetic operations ...&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the assumption of &lt;span class=&quot;math-container&quot;&gt;$m \gg n$&lt;/span&gt;, is not strictly necessary to invoke the phrase &quot;tall and skinny&quot;, but there is no harm done and it enforces the message we are trying to convey. Let us continue with an SVD:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot; ... Now given &lt;span class=&quot;math-container&quot;&gt;$A = QR$&lt;/span&gt; we can compute the SVD of &lt;span class=&quot;math-container&quot;&gt;$R = U \Sigma V^T$&lt;/span&gt; using at most &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt; arithmetic operations. We arrive at &lt;span class=&quot;math-container&quot;&gt;$A = (QU)\Sigma V^T$&lt;/span&gt;, the SVD of the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not aware of any specialized term which covers your precise situation. It would not be wrong to write something along the lines of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The flop count is &lt;span class=&quot;math-container&quot;&gt;$O(pq^2)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$p=\max\{m,n\}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q=\min\{m,n\}$&lt;/span&gt;.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but this is not quite as illustrative as the above. I would hesitate to invent new terms unless absolutely necessary.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2018-10-02T17:57:58.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30293" PostTypeId="2" ParentId="30291" CreationDate="2018-10-02T20:20:08.483" Score="1" Body="&lt;p&gt;A reasonably complete discussion of compatibility can be found at &lt;a href=&quot;https://en.wikipedia.org/wiki/Compatibility_(mechanics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Compatibility_(mechanics)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are six strain-displacement relations and only three displacements in continuum mechanics.  The compatibility conditions will be needed if you wish to compute displacements given strains.  However, most numerical formulations of mechanics are displacement-driven and the need for those equations typically does not arise because one uses: displacement -&gt; strain -&gt; stress -&gt; force.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For force-driven formulations, one has to make sure that the displacements are compatible.  That is because the direction of computations goes as: force -&gt; stress -&gt; strain -&gt; displacement.&lt;/p&gt;&#xA;" OwnerUserId="7626" LastActivityDate="2018-10-02T20:20:08.483" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30294" PostTypeId="1" CreationDate="2018-10-02T21:39:20.183" Score="2" ViewCount="92" Body="&lt;p&gt;I have found that in unstructured mesh, discretizing the laplacian operator with finite volumes requires special care, as given in &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0131274988&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Introduction to Computational Fluid Dynamics: The Finite Volume Method&lt;/a&gt;, (section 11.8) but this is limited to constant normals along the element faces. What happens when these faces are curves as in high order geometries? I haven't been able to find methods to get around this. &lt;/p&gt;&#xA;" OwnerUserId="7023" LastActivityDate="2018-11-02T00:04:58.270" Title="Elliptic equation with finite volume and unstructured high order geometry" Tags="|reference-request|finite-volume|elliptic-pde|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30295" PostTypeId="2" ParentId="30294" CreationDate="2018-10-02T22:17:54.850" Score="0" Body="&lt;p&gt;In finite volume methods, you just have to figure out to parameterize the fluxes across the faces that are now curved. In finite element methods, this is a standard approach and not complicated at all. Most finite element libraries allow you to choose higher order geometry descriptions.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-10-02T22:17:54.850" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30296" PostTypeId="1" CreationDate="2018-10-03T20:57:21.717" Score="6" ViewCount="680" Body="&lt;p&gt;I am using a second-order finite difference in space and time approximation for the 1D wave equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;No source but initial data: &lt;span class=&quot;math-container&quot;&gt;$I(x)=\mathrm{e}^{-400 (x-0.5)^2}$&lt;/span&gt;.&#xA;Velocity &lt;span class=&quot;math-container&quot;&gt;$c=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$nx=501$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$nt=1001$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x \in[0,1]$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$t \in[0,1]$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dt=0.001$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After 0.2 seconds I get &lt;a href=&quot;https://i.stack.imgur.com/PUlHI.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PUlHI.png&quot; alt=&quot;response to a smooth Gaussian impulse&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I change the initial condition to a spike &lt;span class=&quot;math-container&quot;&gt;$I(x)=1$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$x=0.5$&lt;/span&gt;&#xA;and &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$x \ne 0.5$&lt;/span&gt;, I get a noisy, low amplitude response that I include next. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xG6KZ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xG6KZ.png&quot; alt=&quot;impulse response&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is my output (response to a spike) so noisy and where did my amplitude go? I expect to see a traveling impulse (perhaps with lots of small wiggles)&#xA;but distinctive spikes in the middle with amplitude 0.5 each (as in the smooth case). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe it is not a bug in my code. I tried other programs from the web and got the same results. I tested the accuracy of my program against an analytic solution and got an error smaller than &lt;span class=&quot;math-container&quot;&gt;$10^{-4}$&lt;/span&gt;. There is not an issue with the CFL condition since my &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt; is small enough for the velocity &lt;span class=&quot;math-container&quot;&gt;$c=1$&lt;/span&gt; and spatial sampling &lt;span class=&quot;math-container&quot;&gt;$dx$&lt;/span&gt;. I tried sampling rates 10 times smaller. Nothing in particular changed. I believe it is something inherent to the finite difference method but I still do not get it. I have the program in Python and can include it here if necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue has nothing to do with boundary conditions because the impulse has not even reached the boundary yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any hint about what could be going on here would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="28330" LastEditorUserId="20688" LastEditDate="2018-10-03T21:49:38.463" LastActivityDate="2018-10-14T17:12:41.167" Title="Finite difference for 1D wave equation: why the spike initial data results in a noisy output?" Tags="|finite-difference|wave-propagation|differential-equations|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30297" PostTypeId="2" ParentId="30296" CreationDate="2018-10-04T01:06:14.237" Score="10" Body="&lt;p&gt;I think you're probably seeing artifacts that are due to numerical dispersion. In brief, in the discrete case different (spatial) frequencies of a wave function will propagate at different phase/group velocities. This stands in contrast to the continuous case, where the all frequency components travel with exactly the same speed (c). In the first simulation, the initial condition is apparently sufficiently smooth that this effect isn't noticeable (it can be decomposed using only a narrow band of spatial frequencies, that have similar speeds). However, with impulsive initial conditions like the second simulation, every spatial frequency is (equally) present, leading to a &quot;spreading&quot; or dispersion of the signal due to their different propagation speeds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If memory serves, in 1D you can pick a &quot;magic&quot; timestep, exactly at the courant criterion, and this effect disappears (all spatial frequencies have same velocity c, no spreading/dispersion). Probably worth trying that, even if for no other reason than verifying the claim that this is a numerical dispersion artifact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately there is no such &quot;magic&quot; timestep in 2D or 3D. In fact, this problem is even more exaggerated in higher ND, because there is not only this chromatic dispersion, but also dispersion anisotropy (waves going in different directions exhibit different velocities). High order discretizations (in space and time) are the best way to combat these dispersion problems.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-10-04T12:07:13.327" LastActivityDate="2018-10-04T12:07:13.327" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30298" PostTypeId="1" AcceptedAnswerId="30330" CreationDate="2018-10-04T13:37:49.180" Score="1" ViewCount="292" Body="&lt;p&gt;I am working on a multi-commodity flow problem where for a graph &lt;span class=&quot;math-container&quot;&gt;$G=(V, E)$&lt;/span&gt;, some flows are permitted to be split and some flows should strictly follow one path. I have formulated this problem as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem Formulation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \min \sum_{(i,j) \in E} \sum_{f \in F} c_{ij}^fx^f_{ij}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \sum_{j \in V}x^f_{ij} - \sum_{j \in V}x_{ji}^f = \begin{cases}&#xA;d_f  &amp;amp;, &amp;amp; i = s_f \\&#xA;-d_f&amp;amp;, &amp;amp; i = t_f\\&#xA;0 &amp;amp;,&amp;amp; \text{otherwise}&#xA;\end{cases} $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ x_{ij}^f \ge \begin{cases}&#xA;0 &amp;amp; , &amp;amp; f \in S\\&#xA;d_f &amp;amp; , &amp;amp; f \in NS\\&#xA;\end{cases} \quad \forall (i, j) \in E$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \sum_{i,j \in E} x_{ij}^f \le u_{ij}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$d_f$&lt;/span&gt; is the demand for flow &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is the set of flows that are permitted to be split, and &lt;span class=&quot;math-container&quot;&gt;$NS$&lt;/span&gt; is the set of non-splittable flows. &lt;span class=&quot;math-container&quot;&gt;$u_{ij}$&lt;/span&gt; is the capacity of edge &lt;span class=&quot;math-container&quot;&gt;$(i, j)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a novice in linear programming. On paper, using two simple graphs, this formulation seems to work fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate it if someone could point out if there is any problem with this formulation.&lt;/p&gt;&#xA;" OwnerUserId="28884" LastEditorUserId="28884" LastEditDate="2018-10-23T20:52:28.110" LastActivityDate="2018-10-23T20:52:28.110" Title="Splittable and non-splittable flows in the network flow problem" Tags="|linear-programming|graph-theory|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30299" PostTypeId="1" AcceptedAnswerId="30300" CreationDate="2018-10-04T17:53:07.397" Score="2" ViewCount="504" Body="&lt;p&gt;What are common methods to choose pivot blocks in Block LU (for non-SPD/non-Diagonally Dominant Matrices)?&lt;/p&gt;&#xA;" OwnerUserId="16874" LastActivityDate="2018-10-04T19:41:08.787" Title="Pivoting in Block LU" Tags="|linear-algebra|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30300" PostTypeId="2" ParentId="30299" CreationDate="2018-10-04T18:43:32.033" Score="5" Body="&lt;p&gt;I think the best place to find for this information is in the LAPACK working notes or presentations about MAGMA's LU decomposition. See LAWN280 and LAWN282 in particular (links follow).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn280.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.netlib.org/lapack/lawnspdf/lawn280.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.netlib.org/lapack/lawnspdf/lawn282.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.netlib.org/lapack/lawnspdf/lawn282.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a couple of interesting strategies IMO:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the usual partial pivoting over entire panels, and only use block/tile updates on the trailing submatrix downdate. Although there's a limit to how parallel you can make this algorithm (the panel remains an unaddressed bottleneck), it has the advantage that it's basically identical to sequential partial pivoting (in accuracy and interface, all the way down to the ABI level).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Partial pivot over just the diagonal blocks. Cross your fingers for non-diagonally dominant matrices, but you can always statically permute the matrix prior to factorization, ie move big off-diagonal entries to within the diagonal blocks, in the hopes of them remaining large and suitable for use as a pivot at runtime. You can also combine this with thresholding/perturbation (replace a near zero pivot with something else). Since factoring a low-rank perturbation of the input gives a low rank perbutation of the desired factorization, it is likely you can recover later by using some sort of iteration (backwards refinement, or preconditioned krylov solution). This is a common strategy for sparse solvers, because they have limited pivoting choices due to fill-in and rigidity of the symbolic analysis.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Apply some randomized (but fast-to-apply/fast-to-invert) transform to the input matrix in an effort to make it easier to factor with limited/no pivoting. This is a bit heuristic, but is based on the practical observation that it's rare to find a random matrix where tile-pivoted (or even unpivoted) gaussian elimination fails. So transform your (bad) matrix randomly into another (hopefully better) one, factor it, then apply the inverse transformation at solve-time.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Mix tile-LU and panel-QR algorithms (they both work left to right), only using the latter when you detect breakdown in the tile-LU factorization. The QR steps are slower but more stable. Unfortunately tall/skinny QR of a panel is more complicated to implement in parallel, so it's quite a bit of legwork to get this algorithm off the ground.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2018-10-04T19:41:08.787" LastActivityDate="2018-10-04T19:41:08.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30302" PostTypeId="2" ParentId="30296" CreationDate="2018-10-05T03:06:44.243" Score="2" Body="&lt;p&gt;You can only expect to get a sensible answer if your data can be resolved by the grid. Shannons Sampling Theorem is central to all digital signal processing and states that no signal with wavelength less than 2h can be represented on a grid with spacing h (and for most methods the minimum wavelength is much more than this). Your spike is severely underresolved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The magic CFL number, unfortunately does not work for nonlinear problems with variable wavespeed or as rchilton observes, in higher dimensions, so it is of limited practical use use&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2018-10-05T03:06:44.243" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30303" PostTypeId="1" CreationDate="2018-10-05T09:45:17.327" Score="2" ViewCount="121" Body="&lt;p&gt;The partial differential equation,&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\dfrac{\partial f}{\partial x} + a(x)\dfrac{\partial f}{\partial y} = 0&#xA;\qquad &amp;amp; f(0,y) = f(L_1,y) = c_0e^{-y} \\&#xA;&amp;amp; f(x,0) = c_0 \;,\; f(x,L_2) = c_0e^{-L_2}&#xA;\end{align}&lt;/span&gt;&#xA;for &lt;span class=&quot;math-container&quot;&gt;$f=f(x,y)$&lt;/span&gt; has the nontrivial solution,&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;f(x,y) = Ae^{(-y-h(x))} \;,\quad a(x)=-\dfrac{dh(x)}{dx} \;.&#xA;\end{equation}&lt;/span&gt;&#xA;If I discretize the equation by box integration over &lt;span class=&quot;math-container&quot;&gt;$[y_{i},y_{i+1}]$&lt;/span&gt;, I get:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\dfrac{\Delta y}{2}&#xA;\left[\dfrac{\partial f(x,y_{i})}{\partial x} &#xA;+ \dfrac{\partial f(x,y_{i+1})}{\partial x}\right]&#xA;+ a(x)\big[f(x,y_{i+1}) - f(x,y_i)\big] = 0&#xA;\end{equation}&lt;/span&gt;&#xA;which converges to the nontrivial solution as &lt;span class=&quot;math-container&quot;&gt;$\Delta y\rightarrow 0$&lt;/span&gt;.&#xA;However, I have noticed such discretizations are numerically unstable for &lt;span class=&quot;math-container&quot;&gt;$a(x)\neq 0$&lt;/span&gt; and result in oscillations when solved on a computer.&#xA;How can I investigate the above equation in terms of numerical stability? &#xA;Does this equation belong to a specific class of PDEs, and are there techniques to improve its stability?&lt;/p&gt;&#xA;" OwnerUserId="17485" LastEditorUserId="17485" LastEditDate="2018-10-05T09:55:59.583" LastActivityDate="2018-10-17T11:49:57.323" Title="Stability Analysis" Tags="|pde|stability|discretization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30304" PostTypeId="2" ParentId="11276" CreationDate="2018-10-05T15:51:07.143" Score="6" Body="&lt;p&gt;XTensor is a modern approach and is getting more and more popular.&#xA;&lt;a href=&quot;https://github.com/QuantStack/xtensor&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/QuantStack/xtensor&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28890" LastActivityDate="2018-10-05T15:51:07.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30305" PostTypeId="1" CreationDate="2018-10-05T18:26:54.953" Score="2" ViewCount="593" Body="&lt;p&gt;I need to optimize affine transformations for of a set of triangles using energy function based on the connectivity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The energy of an edge &lt;span class=&quot;math-container&quot;&gt;$e_j$&lt;/span&gt; between triangles &lt;span class=&quot;math-container&quot;&gt;$T_a, T_b$&lt;/span&gt; is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_j = \left[(A_a v_{a,1} + t_a) - (A_b v_{1,b} + t_b) \right]+&#xA;\left[(A_a v_{a,2} + t_a) - (A_b v_{2,b} + t_b) \right]&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$(A_a, t_a)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(A_b, t_b)$&lt;/span&gt; are affine transformations on the triangles adjacent to the edge and &lt;span class=&quot;math-container&quot;&gt;$v_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_2$&lt;/span&gt; are the vertices of the edge. The solution &lt;span class=&quot;math-container&quot;&gt;$v_i \equiv 0$&lt;/span&gt; is prevented by an additional deformation energy on the triangles, like the conformal energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the optimization, we want to have &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{a,1}=v_{b,1}, \quad v_{a,2}=v_{b,2} \quad \Rightarrow E_j = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is, that this is a non-linear problem, which is hard for solvers. I am using Google Ceres with line-search, but it often fails to find a valid line-search stepsize when optimizing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{pmatrix}&#xA;a_1 &amp;amp; a_2 \\&#xA;a_3 &amp;amp; a_4&#xA;\end{pmatrix}, \begin{pmatrix}t_1 \\ t_2\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as a vector &lt;span class=&quot;math-container&quot;&gt;$(a_1, a_2, a_3, a_4, \dots, t_1, t_2, \dots)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For some transformations, there are more favorable parameterizations, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rotation&lt;/strong&gt; can be parameterizized as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{pmatrix}&#xA;\cos \alpha &amp;amp; -\sin\alpha \\&#xA;\sin \alpha &amp;amp; \cos \alpha&#xA;\end{pmatrix}, \begin{pmatrix}t_1 \\ t_2\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;optimizing the vector &lt;span class=&quot;math-container&quot;&gt;$(\alpha, \dots, t_1, t_2, \dots)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Shearing&lt;/strong&gt; can be optimized using&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{pmatrix}&#xA;1 &amp;amp; s_1 \\&#xA;s_2 &amp;amp; 1&#xA;\end{pmatrix}, \begin{pmatrix}t_1 \\ t_2\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;optimizing the vector &lt;span class=&quot;math-container&quot;&gt;$(s_1, s_2, \dots, t_1, t_2, \dots)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for &lt;strong&gt;general&lt;/strong&gt; deformations, I do not see which parameterization of the transformation matrix would fit and optimizing the coefficients directly probably lets the solver fail (or not converge) because it yields no good gradient.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So what is the best way to optimize affine transformations to find triangle deformations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here a graphic what should be optimized:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sEPjA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sEPjA.png&quot; alt=&quot;Optimization of the triangle positions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The final optimization objective is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a set of Triangles &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$T=\{t_1=[v_1, v_2, v_3], t_2=[v_4, v_5, v_6], ßdots\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;without shared vertices in the set and the information about the shared vertices of common edges as a separate set&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$E=\{[a,b,c,d], \dots|a\sim c, b\sim d\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\overline{ab}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\overline{cd}$&lt;/span&gt; are the same edges, when all triangles are re-assembled to a mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Find affine transformations &lt;span class=&quot;math-container&quot;&gt;$A_i+t_i$&lt;/span&gt; for all triangles, such that for two adjacent triangles &lt;span class=&quot;math-container&quot;&gt;$t_1=[a,b,x]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_2=[c,d,y]$&lt;/span&gt; holds&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A_1 a + t_1 = A_2 c + t_2 \quad\wedge\quad A_1 c + t_1 = A_2 d + t_2&#xA;$$&lt;/span&gt;&#xA;and for all transformations &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{trace} \frac{(A_i^T+A_i)}{2 \det A_i} \to \min&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first condition ensures that common edges are merged and the second condition minimizes the conformal deformation, what e.g. prevents the mesh form collapsing to a single vertex.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;When I for example optimize the triangles of a cut cube, then the only transformations I need are rotation and translation to arrange the triangles in the plane. Optimzing &lt;span class=&quot;math-container&quot;&gt;$(\alpha, t_1, t_2, \dots)$&lt;/span&gt; using line-search finds a good solution without problem.s&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for example a sphere requires quite a bit of deformation to be embedded in the plane. And to optimize the deformation needed to assemble the triangles into a planar patch, the optimization of the raw values &lt;span class=&quot;math-container&quot;&gt;$(a_1, a_2, a_3, a_4, t_1, t_2)$&lt;/span&gt; seems not to work very well, especially line-search with Google ceres fails to find a valid stepsize for which the wolfe condition holds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that in contrast to e.g. the gradient of the rotation, the gradient of this vector is not well-suited for finding the optimal transformation and it would need another local parameterization, which better suits the search space, just like the rotation/translation for rigid embeddings.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="14700" LastEditDate="2018-10-07T15:00:45.050" LastActivityDate="2018-10-07T15:00:45.050" Title="How to numerically optimize affine transformations?" Tags="|optimization|computational-geometry|numerics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30306" PostTypeId="1" CreationDate="2018-10-05T22:39:43.877" Score="0" ViewCount="613" Body="&lt;p&gt;I have a TTree in &lt;a href=&quot;https://root.cern.ch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ROOT&lt;/a&gt; with 1000 events and 15 variables associated to each of them. I would like to convert this in its entirety to an hdf5 dataset. How do I organise my data in HDF5 Groups such that I can access data both by event number and by variable (if I wanted all the data from the '&lt;em&gt;kinetic energy&lt;/em&gt;' variable, for example, all events)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I have already tried the &lt;a href=&quot;http://www.rootpy.org/commands/root2hdf5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;root2hdf5&lt;/a&gt; conversion tool but this does not work for branches with arrays/compound datatypes.&lt;/p&gt;&#xA;" OwnerUserId="28893" LastEditorUserId="20688" LastEditDate="2018-10-06T16:13:11.343" LastActivityDate="2020-05-19T06:00:55.207" Title="Converting ROOT Tree to HDF5" Tags="|data-analysis|data-sets|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30307" PostTypeId="1" CreationDate="2018-10-06T04:15:02.737" Score="9" ViewCount="233" Body="&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving a non-linear system of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of variables is the same as the number of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I fix a set of variables (say &lt;span class=&quot;math-container&quot;&gt;$\vec{y}$&lt;/span&gt;) and keep another set free (say &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt;), the system becomes an &lt;strong&gt;under-determined&lt;/strong&gt;, dense, and linear system of the subset of variables &#xA;&lt;span class=&quot;math-container&quot;&gt;$$A(\vec{y})\vec{x} = \vec{b}(\vec{y})\tag{1}\label{system1}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$A(\vec{y})$&lt;/span&gt; is a dense matrix, and &lt;span class=&quot;math-container&quot;&gt;$\vec{b}(\vec{y})$&lt;/span&gt; is a dense vector). Let's call this sub-system &lt;span class=&quot;math-container&quot;&gt;$\eqref{system1}$&lt;/span&gt; as &lt;em&gt;system 1&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I fix &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt; and keep &lt;span class=&quot;math-container&quot;&gt;$\vec{y}$&lt;/span&gt; free, the system becomes an &lt;strong&gt;over-determined&lt;/strong&gt;, sparse, and non-linear system of the subset of variables &#xA;&lt;span class=&quot;math-container&quot;&gt;$$F(\vec{x}, \vec{y}) = 0\tag{2}\label{system2}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobian &lt;span class=&quot;math-container&quot;&gt;$J(\vec{x}, \vec{y})$&lt;/span&gt; is available in a closed form. Let's call this sub-system &lt;span class=&quot;math-container&quot;&gt;$\eqref{system2}$&lt;/span&gt; as &lt;em&gt;system 2&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About half of the equations in &lt;span class=&quot;math-container&quot;&gt;$\eqref{system2}$&lt;/span&gt; are equality constraints that are linear in terms of &lt;span class=&quot;math-container&quot;&gt;$\vec{y}$&lt;/span&gt;. Each of the constraints is quite sparse and involves only about 5% of all variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I solve it with the following?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Algorithm 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Initialize &lt;span class=&quot;math-container&quot;&gt;$\vec{x} = \vec{x}_0$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\vec{y} = \vec{y}_0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Fix &lt;span class=&quot;math-container&quot;&gt;$\vec{y}_{n - 1}$&lt;/span&gt;, solve &lt;span class=&quot;math-container&quot;&gt;$A(\vec{y}_{n-1})\vec{x}_{n} = \vec{b}(\vec{y}_{n-1})$&lt;/span&gt; for one of all the possible &lt;span class=&quot;math-container&quot;&gt;$\vec{x}_{n}$&lt;/span&gt; because this system is &lt;em&gt;under-determined&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Fix &lt;span class=&quot;math-container&quot;&gt;$\vec{x}_{n - 1}$&lt;/span&gt;. Perform one iteration of Newton's method for solving &lt;span class=&quot;math-container&quot;&gt;$F(\vec{x}_{n-1}, \vec{y}_{n}) = 0$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\vec{y}_{n}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;If not converged, go to step 2.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Algorithm 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I replace step 2 in Algorithm 1 by a Newton's method iteration for solving &lt;em&gt;system 1&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\eqref{system1}$&lt;/span&gt;, then I guess the steps become a block Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't know if these two algorithms can work because system 1 &lt;span class=&quot;math-container&quot;&gt;$\eqref{system1}$&lt;/span&gt; is under-determined and system 2 &lt;span class=&quot;math-container&quot;&gt;$\eqref{system2}$&lt;/span&gt; is over-determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this work? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Related&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1016/j.jcp.2009.08.003&quot; rel=&quot;noreferrer&quot;&gt;A. Yeckel, L. Lun, and J. J. Derby, &quot;An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications,&quot; &lt;em&gt;J. Comp. Phys.&lt;/em&gt;, vol. 228, no. 23, pp. 8566–8588, Dec. 2009&#xA;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/1703.07810.pdf&quot; rel=&quot;noreferrer&quot;&gt;B. Polyak and A. Tremba, &quot;Solving underdetermined nonlinear equations&#xA;by Newton-like method&quot;&lt;/a&gt; &#xA;Solving over-determined non-linear equations with Newton's method and Moore-Penrose inverse is fine.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pdfs.semanticscholar.org/30ad/d70c4afaf2b17349ffe03524a78dae7d4e04.pdf&quot; rel=&quot;noreferrer&quot;&gt;V. Meñkov. &quot;Solving block linear system with low-rank off diagonal blocks is easily parallelizable&quot;&lt;/a&gt;&#xA;Block triangular form allows some blocks to be over-determined while other blocks to be under-determined. Newton's method is equivalent to solving the system linearized at current approximation of solution. The question is equivalent to if it is possible to solve a system of linear equations by alternatively performing iterations of Newton's method on two groups of variables.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0377042706003190&quot; rel=&quot;noreferrer&quot;&gt;Śmietański, M. J. &quot;A generalized Jacobian based Newton method for semismooth block-triangular system of equations&quot; &lt;em&gt;J. Comput. Appl. Math.&lt;/em&gt;. 2007.&lt;/a&gt;&#xA;Solving block triagular system by newton iteration on each block can work. &quot;We prove locally superlinear convergence of parameterized version method that is well defined even when the generalized Jacobian is singular.&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="28896" LastEditorUserId="28896" LastEditDate="2018-10-10T17:32:43.597" LastActivityDate="2018-12-18T20:01:55.400" Title="Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system" Tags="|linear-algebra|linear-solver|nonlinear-equations|newton-method|" AnswerCount="0" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="30309" PostTypeId="2" ParentId="30303" CreationDate="2018-10-06T14:02:25.593" Score="3" Body="&lt;p&gt;This is a first-order PDE but you are trying to impose Dirichlet BC on the whole boundary. This is not well-posed in general but maybe your data is such that you expect it to have a unique solution. You can write this PDE as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v \cdot \nabla f = \nabla\cdot (v f) = 0&#xA;$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;v(x,y) = (1, a(x))&#xA;$$&lt;/span&gt;&#xA;This is a hyperbolic PDE. If &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is outward normal to your domain boundary, then you can specify Dirichlet BC only on those portions of the boundary where &lt;span class=&quot;math-container&quot;&gt;$v \cdot n &amp;lt; 0$&lt;/span&gt;. So you can specify BC on the left boundary &lt;span class=&quot;math-container&quot;&gt;$(x=0)$&lt;/span&gt; but not the right boundary. On the bottom and top boundaries, you have to check the sign of &lt;span class=&quot;math-container&quot;&gt;$a(x)$&lt;/span&gt; to decide this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Draw the characteristic curves&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;dy/dx = a(x)&#xA;$$&lt;/span&gt;&#xA;to see how BC will flow into your domain. This can identify a time-like direction along which you can march to find the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if &lt;span class=&quot;math-container&quot;&gt;$a(x)=x$&lt;/span&gt; and your domain is &lt;span class=&quot;math-container&quot;&gt;$[0,L_1]\times[0,L_2]$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$a(x) \ge 0$&lt;/span&gt; in your domain, then you can specify BC on the left and bottom sides. You can march along &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. As Wolfgang wrote, for &lt;span class=&quot;math-container&quot;&gt;$\partial f/\partial y$&lt;/span&gt; you can use backward-difference type scheme. An FD scheme may look like this&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{f_{i+1,j}-f_{i,j}}{\Delta x} + a_i \frac{f_{i,j} - f_{i,j-1}}{\Delta y} = 0, \qquad a_i \ge 0&#xA;$$&lt;/span&gt;&#xA;Note that &lt;span class=&quot;math-container&quot;&gt;$f_{0,j}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_{i,0}$&lt;/span&gt; are the given boundary values and you determine the remaining values using the scheme above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is only first-order accurate but a good starting point.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="20688" LastEditDate="2018-10-06T15:48:35.260" LastActivityDate="2018-10-06T15:48:35.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30310" PostTypeId="2" ParentId="30306" CreationDate="2018-10-06T16:12:31.573" Score="1" Body="&lt;p&gt;You might want to give a try to &lt;a href=&quot;https://github.com/WolfgangWaltenberger/dataharvester&quot; rel=&quot;nofollow noreferrer&quot;&gt;DataHarvester&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This tool claims to support ROOT, HBOOK, HDF, XML, SQLITE and input/output between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the following publication:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://doi.org/10.1088/1742-6596/119/4/042033&quot; rel=&quot;nofollow noreferrer&quot;&gt;W. Waltenberger, G. Richter, and H. V. Riedel, &quot;DataHarvester—an abstraction layer for accessing scientific data from various sources,&quot; in &lt;em&gt;Int. Conf. Computing High Energy Nuclear Phys.&lt;/em&gt;, Victoria, Canada, Sep. 2007, pp. 1-4.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-11-06T21:09:35.287" LastActivityDate="2018-11-06T21:09:35.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30313" PostTypeId="1" AcceptedAnswerId="30319" CreationDate="2018-10-07T21:38:48.467" Score="1" ViewCount="272" Body="&lt;p&gt;I adapted &lt;a href=&quot;http://www.karlin.mff.cuni.cz/~hron/fenics-tutorial/discontinuous_galerkin/doc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this FEniCS code&lt;/a&gt; for my problem and I'm wondering if there is any good resource about how to choose the penalty parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;? Best case would be, if I can define it through some relation of the diffusion coefficient &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and element height &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28906" LastEditorUserId="28906" LastEditDate="2018-10-08T18:00:12.923" LastActivityDate="2018-10-09T12:56:46.757" Title="Penalization parameter for DG with jump penalization" Tags="|finite-element|fenics|discontinuous-galerkin|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30315" PostTypeId="1" CreationDate="2018-10-08T19:06:31.207" Score="2" ViewCount="212" Body="&lt;p&gt;I set up two one-way wave equations for constant velocity &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in one-dimension. When I implement them I get a highly unstable (divergent) solution. I wonder if someone could give me a suggestion about what could be wrong with my reasoning. Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea behind the PML method&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Perfectly_matched_layer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perfectly Matched Layer&lt;/a&gt;&#xA;is to attenuate the wavefield by doing analytic continuation of the real field into the complex in a way that the field in a buffer zone is damped exponentially. This is done by doing a change of variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;  x' = x +  \mathrm{i} \frac{f(x)}{\omega}.&#xA;  \tag{eq. 1}&#xA;\end{eqnarray}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$x, f(x), \omega \in  \mathbb{R}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; is a complex number with &lt;span class=&quot;math-container&quot;&gt;$\mathrm{i}=\sqrt{-1}$&lt;/span&gt;. The factor &lt;span class=&quot;math-container&quot;&gt;$1/\omega$&lt;/span&gt; is set up to avoid grid dispersion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We assume a wave equation in the complex plane and time with variables &lt;span class=&quot;math-container&quot;&gt;$(x', t)$&lt;/span&gt;&#xA;This is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  u_{x'x'} - \frac{1}{c^2} u_{tt}=0.&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After applyting the Fourier transform in time we find the Helmholtz equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{\partial^2 U(x',\omega)}{\partial x'^2} + \frac{\omega^2}{c^2} U(x',\omega)=0&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;which we can factor as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \left ( \frac{\partial }{\partial x'} + \mathrm{i} \frac{\omega}{c} \right )&#xA;  \left (\frac{\partial }{\partial x'} - \mathrm{i} \frac{\omega}{c} \right ) U(x', \omega)=0.&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;and find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;  \left ( \frac{\partial }{\partial x'} + \mathrm{i} \frac{\omega}{c} \right ) &#xA;  U(x', \omega) = 0 \quad , \quad &#xA;  \left ( \frac{\partial }{\partial x'} - \mathrm{i} \frac{\omega}{c} \right ) &#xA;  U(x',\omega) = 0.&#xA;  \tag{eq.2}&#xA;\end{eqnarray}&lt;/span&gt;&#xA;These are &#xA;&lt;a href=&quot;http://farside.ph.utexas.edu/teaching/329/lectures/node90.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;advection equations &lt;/a&gt;.&#xA;The first represents a wave traveling forward (say, to the right) with wavespeed &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and the second a wave traveling backward (say, to the left) with wave speed &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The change of variable &lt;span class=&quot;math-container&quot;&gt;$x'=x+\mathrm{i} \frac{f(x)}{\omega}$&lt;/span&gt; produces, by using the chain rule, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{\partial U}{\partial x'} &amp;amp;=&amp;amp; &#xA;  \frac{\partial U}{\partial x} \frac{d x}{d x'}  = &#xA;  \frac{1}{\left ( 1 + \frac{\mathrm{i} f_x}{\omega} \right )} \frac{\partial U}{\partial x} &#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;now from the first of equations eq.2 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  -\mathrm{i} \frac{\omega}{c}  U = \frac{1}{&#xA;    \left ( 1 + \mathrm{i} \frac{f_x}{\omega} \right )} U_{x}.&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  -\mathrm{i} \frac{\omega}{c} U + \frac{f_x}{c} U = U_x.&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;Clearly, if &lt;span class=&quot;math-container&quot;&gt;$f=0$&lt;/span&gt; (or constant) the equation in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; coincides &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; and&#xA;the wave in &lt;span class=&quot;math-container&quot;&gt;$x'$&lt;/span&gt; coincides with the classical wave equation for real domain &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. We can write the equation in time (by using the Fourier inverse transform) as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;  \frac{1}{c} u_t + \frac{f_x}{c} u = u_x&#xA;  \tag{eq.3}&#xA;\end{eqnarray}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We now verify that this equation fits our purposes. Let us assume a solution of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;u(x,t)=A \mathrm{e}^{\mathrm{i}(k x-\omega t)} \mathrm{e}^{-k f(x)/\omega}&#xA;\tag{eq.4}&#xA;\end{eqnarray}&lt;/span&gt;&#xA;If we substitute this solution in equation (eq.3) we find, for the left hand side &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \left ( -\frac{\mathrm{i} \omega }{c} + \frac{f_x}{c} \right ) u(x,t)&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;and, for the right hand side&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \left ( \mathrm{i k} - \frac{k f'(x)}{\omega}  \right ) u(x,t).&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;Since &lt;span class=&quot;math-container&quot;&gt;$k=\omega/c$&lt;/span&gt; we see that the function &lt;span class=&quot;math-container&quot;&gt;$u(x,t)$&lt;/span&gt; defined in (eq.4)&#xA;satisfies equation (eq.3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, instead of chosing the first of equations (eq.3), we pick the second&#xA;we find, following a similar procedure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;  -\frac{1}{c} u_t + \frac{f_x}{c} u = u_x.&#xA;  \tag{eq.5}&#xA;\end{eqnarray}&lt;/span&gt;&#xA;Similarly, we find  that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;  u(x,t)=A \mathrm{e}^{\mathrm{i}(k x+\omega t)} \mathrm{e}^{-k f(x)/\omega}&#xA;\tag{eq.6}&#xA;\end{eqnarray}&lt;/span&gt;&#xA;satisfies (eq.5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution (eq.4) represents a wave moving to the right with a speed &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;,&#xA;wave number &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, frequency &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; and attenuation factor &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;. &#xA;Likewise, solution (eq.6) represents a wave moving to the left with the&#xA;same parameters above. The superposition of all this &quot;plane-wave&quot; solutions&#xA;over all frequencies and wave numbers constrained with initial and boundary conditions is the solution of the initial/boundary value problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases (eq.4) and (eq.6), if &lt;span class=&quot;math-container&quot;&gt;$f(x)=0$&lt;/span&gt; we get back to the initial problem with no attenuation over the real variable &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us define &lt;span class=&quot;math-container&quot;&gt;$f_x=\sigma(x)$&lt;/span&gt;. That is &lt;span class=&quot;math-container&quot;&gt;$f(x)=\int_{x_0}^x \sigma(s) ds$&lt;/span&gt;&#xA;then, in (eq.3) we use forward differences in time and central differences in space to get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{ w_{i j+1}  - w_{ij}}{\Delta t} = \frac{c}{2 \Delta x} (w_{i+1 j} - w_{i-1 j})&#xA;  -  \sigma_i w_{ij},&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;using a forward difference in time and central in space and the CFL number &lt;span class=&quot;math-container&quot;&gt;$\gamma= c \Delta t/\Delta x$&lt;/span&gt;.&#xA;We find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  w_{i j+1} = \frac{\gamma}{2}( w_{i+1 j} - w_{i-1 j}) - w_{ij}(\sigma_i \Delta t -1)&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use equation (eq.5) for the left side of the working interval (model).&#xA;The corresponding finite difference system is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  w_{i j+1} = -\frac{\gamma}{2}( w_{i+1 j} - w_{i-1 j}) + w_{ij}(\sigma_i \Delta t-1). &#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use these finite difference equations in the edges of the model (last and first 20% or 10%....nothing works). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;----------------update------------------&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I changed the finite difference scheme to forward in both, space and time.&#xA;Now it is stable, but the the modeling acts as if the sides where shifted&#xA;and the reflections are taking place at &lt;span class=&quot;math-container&quot;&gt;$[a+d,b-d]$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$[a,b]$&lt;/span&gt; is&#xA;the spatial domain of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the buffer zone for the damping.&#xA;The field &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is zeroed in the intervals &lt;span class=&quot;math-container&quot;&gt;$[a,a+d]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[b-d,b]$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28330" LastEditorUserId="28330" LastEditDate="2018-10-09T00:26:52.777" LastActivityDate="2018-10-12T16:26:26.023" Title="PML boundary conditions" Tags="|finite-difference|boundary-conditions|differential-equations|numerics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30318" PostTypeId="2" ParentId="1749" CreationDate="2018-10-08T23:26:44.507" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=2ahUKEwjjre3W8PfdAhUKzIUKHYSQDzgQFjAAegQIChAC&amp;amp;url=https%3A%2F%2Fcran.r-project.org%2Fweb%2Fpackages%2Fminpack.lm%2Fminpack.lm.pdf&amp;amp;usg=AOvVaw2GJL44QiM_VH1mbFXgHJDb&quot; rel=&quot;nofollow noreferrer&quot;&gt;R minpack.lm CRAN package&lt;/a&gt; provides a Levenberg-Marquardt implementation with box constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, Levenberg-Marquardt is much better suited than L-BFGS-B for least-squares problems. It will converge (much) better on challenging problems. It will also be much faster than the general purpose IPOPT, as it is tailored to non-linear least-squares problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The R package chooses a very straightforward projection approach to enforce the constraints (see &lt;a href=&quot;https://github.com/cran/minpack.lm/blob/master/src/fcn_lmder.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;the source code&lt;/a&gt;). Depending on the LM implementation you are using, it could be simple to include.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the suggestion in the comments of using a transformation, (for example a sine transformation as in scipy) is also a good, simple alternative to transform your unconstrained LM algorithm into a constrained one. You will also need to include the transformation in the Jacobian if the Jacobian is analytic.&lt;/p&gt;&#xA;" OwnerUserId="28915" LastActivityDate="2018-10-08T23:26:44.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30319" PostTypeId="2" ParentId="30313" CreationDate="2018-10-09T00:05:19.717" Score="2" Body="&lt;p&gt;TL;DR: Let &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; be the diffusion coefficient, &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; the minimum angle between any two edges of the mesh, &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; the space dimension, and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; the polynomial degree of the finite element basis you're using. Then taking&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\alpha \ge \frac{2 \cdot d \cdot (p + d - 1)}{\cos\theta\cdot \tan\theta/2} \cdot \frac{\max k}{\min k}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;should guarantee that the discretized diffusion operator will be positive-definite. You'll still have to be careful about which numerical flux you use for the convective part in order to guarantee stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DG in part evolved from &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19910/what-is-the-general-idea-of-nitsches-method-in-numerical-analysis&quot;&gt;Nitsche's method&lt;/a&gt; for weakly imposing Dirichlet boundary conditions, so a lot of the relevant information for this is contained in the literature on Nitsche's method. When using either the symmetric interior penalty DG method or Nitsche's method, the penalty parameter &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; has to be greater than a certain lower bound in order to guarantee positivity of the resulting linear operator. It would be nice if there were a formula in terms of the diffusion coefficient and the element shape so that you could select a good value &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; without any guesswork.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately a lot of papers on DG or Nitsche's method never address this point.&#xA;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/abs/10.1002/gamm.201490018&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hansbo 2005&lt;/a&gt; has an analytical lower-bound in terms of the element shape, but only for linear finite elements. It has some discussion of Nitsche's method which you can apply equally well to DG, so although more limited in scope it's worth the read. &lt;a href=&quot;https://pdfs.semanticscholar.org/4ecd/23c30be74482c2a393ec1a920870ffdf3a3e.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Warburton and Hesthaven 2003&lt;/a&gt; has the full analysis for higher-order finite elements. I also have some &lt;a href=&quot;https://github.com/danshapero/firedrake-2018-06-07/blob/master/laplace.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;example code&lt;/a&gt; demonstrating Nitsche's method for the Poisson equation using firedrake which, if you're not familiar with it, shares a lot of features with fenics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another paper that I think is worth looking at is &lt;a href=&quot;http://math.aalto.fi/~rstenber/Publications/Venice95.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Freund and Stenberg 1995&lt;/a&gt;, in particular for the discussion of convection-diffusion problems like the one you're solving. In that case, the operator is non-symmetric, so it might arguably make more sense to use the asymmetric interior penalty method. This approach has no lower bound on the size of the penalty parameter, it just has to be positive.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="393" LastEditDate="2018-10-09T12:56:46.757" LastActivityDate="2018-10-09T12:56:46.757" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30320" PostTypeId="1" CreationDate="2018-10-09T08:48:04.703" Score="3" ViewCount="54" Body="&lt;p&gt;I am currently implementing classical density functional theory for a radially symmetric system.&#xA;In mathematical terms, I am searching for a function &lt;span class=&quot;math-container&quot;&gt;$f(r)$&lt;/span&gt; that minimizes a functional &lt;span class=&quot;math-container&quot;&gt;$\Omega[f]$&lt;/span&gt;.&#xA;The simplest part of &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; looks like this: &lt;span class=&quot;math-container&quot;&gt;$\int\limits_0^R r^2 f(r)(\ln(f(r))-1)\mathrm{d}r$&lt;/span&gt;, but there are others that involve convolutions of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&#xA;I am trying to minimize a numerical implementation of &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; using &lt;a href=&quot;https://nlopt.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;nlopt&lt;/a&gt;, which works fine in planar coordinates for the same functional.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(r)$&lt;/span&gt; is represented as a vector &lt;span class=&quot;math-container&quot;&gt;$f_i$&lt;/span&gt; which represents samples of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; at evenly spaced points &lt;span class=&quot;math-container&quot;&gt;$r_i$&lt;/span&gt;. The points start at &lt;span class=&quot;math-container&quot;&gt;$Δr$&lt;/span&gt; and go up to &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$f(0)$&lt;/span&gt; is irrelevant. There is a constant boundary condition &lt;span class=&quot;math-container&quot;&gt;$f(r&amp;gt;R)=f_∞$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in radial coordinates, the convergence for small &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is way worse than for large &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;, because of the factor &lt;span class=&quot;math-container&quot;&gt;$r^2$&lt;/span&gt; (the Jacobian determinant). If I only cared about the value of &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;, this would not be a problem, but in this case, &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; is only a vehicle to obtain &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there libraries that are better suited for this kind of problem? Or is there a different way to fix the problem and keep using nlopt?&lt;/p&gt;&#xA;" OwnerUserId="28919" LastEditorUserId="23791" LastEditDate="2018-10-09T23:39:04.353" LastActivityDate="2018-10-09T23:39:04.353" Title="Nonlinear functional optimization in radial coordinates" Tags="|optimization|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30321" PostTypeId="1" CreationDate="2018-10-09T11:38:20.977" Score="2" ViewCount="1169" Body="&lt;p&gt;I'm using Abaqus/CAE to create a cup deep drawing simulation and everything worked perfectly, but my objective is to run the same exact simulation on Calculix (I am new to using this program) and compare the results.&lt;/p&gt;&#xA;&lt;p&gt;So I used the same geometry and mesh and converted all the keywords from the input file of Abaqus to match the format of the input file for Calculix (it is a simple simulation so no big changes are made).&lt;/p&gt;&#xA;&lt;p&gt;However, when I run the CCX solver, it stops at &lt;span class=&quot;math-container&quot;&gt;$t=0.09$&lt;/span&gt; (total time is &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt;) and displays the error message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Too many cutbacks.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;refining the mesh of the slave part&lt;/li&gt;&#xA;&lt;li&gt;lowering the initial time increment&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I don't know where the problem is in my input, although it seems to me that is in the contact definition because every other thing defined is very simple (like mesh, boundary condition, and surface definition, etc...) and no complicated parameters are needed.&lt;/p&gt;&#xA;&lt;p&gt;This is how I defined the contact:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;** contact &#xA;*Contact Pair, Interaction=SI1, Type=Surface To Surface&#xA; Slave_tole, Master_poin&#xA;*Surface Interaction, Name=SI1&#xA;*Surface Behavior, Pressure-Overclosure=Linear&#xA; 1290000&#xA;*Friction&#xA; 0.25, 350000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used the same values as in the Abaqus simulation.&lt;/p&gt;&#xA;" OwnerUserId="26742" LastEditorUserId="20688" LastEditDate="2022-01-02T05:35:32.600" LastActivityDate="2022-01-02T05:35:32.600" Title="What is the difference between Abaqus and Calculix contact input?" Tags="|finite-element|numerics|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30322" PostTypeId="2" ParentId="29028" CreationDate="2018-10-09T23:33:57.927" Score="1" Body="&lt;p&gt;Here is a systematic way of doing it.  Numerically solve an optimization problem to find a matrix &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;, which is smallest in some sense, let's say Frobenius norm, such that &lt;span class=&quot;math-container&quot;&gt;$$(A+E)x = b$$&lt;/span&gt; &#xA; &lt;span class=&quot;math-container&quot;&gt;$$x_1 \ge y_1 + d$$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$$x_2 = y_2$$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$$x_3 \le y_3 - d$$&lt;/span&gt;where &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is some specified moimnimum amount of separation between the old and new solution elements, and is needed because numerical optimization solvers don't deal with strict inequalities for continuous variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I show here an implementation in CVX (under MATLAB) for 3 by 3 &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. But this easily generalizes to higher dimensions and many variations.  &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; are the input data to the optimization problem, and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; are the (decision) variables being solved for in the optimization.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cvx_begin&#xA;variables x(3) E(3,3)&#xA;minimize(norm(E,'fro'))&#xA;subject to&#xA;(A+E)*x == b&#xA;x(1) &amp;gt;= y(1) + d&#xA;x(1) == y(2) &#xA;x(3) &amp;lt;= y(3) - d&#xA;cvx_end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At the conclusion of which E will be the matrix having smallest Frobenius norm which satisfies the constraints.  Note that if &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is singular and there is a solution satisfying all the constraints with &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; being the zero matrix (i.e., not  &quot;changing&quot; &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;), then such a solution will be found by this optimization approach without any special logic being required.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2018-10-09T23:33:57.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30323" PostTypeId="2" ParentId="30229" CreationDate="2018-10-10T01:52:42.363" Score="2" Body="&lt;p&gt;One idea would be to store the low-rank approximation of the inverse of the matrix &lt;span class=&quot;math-container&quot;&gt;$A^{-1}\approx UV$&lt;/span&gt; using SVD. If &lt;span class=&quot;math-container&quot;&gt;$A\in \mathbb{R}^{30\times 30}$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$U\in \mathbb{R}^{30\times k}\  \&amp;amp;  \ V\in \mathcal{R}^{k\times 30} $&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k\approx 4,5$&lt;/span&gt; is the rank. Use the &lt;span class=&quot;math-container&quot;&gt;$UV$&lt;/span&gt; approximation as a preconditioner for solving the system &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; using GMRES. You can use vary &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; as a tuning parameter that trades off memory with the convergence rate. &lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2018-10-10T01:52:42.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30324" PostTypeId="1" CreationDate="2018-10-10T07:55:52.867" Score="1" ViewCount="117" Body="&lt;p&gt;Consider the implicit ODE&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;M(y)\dot{y} = F(t,y)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is non-singular for all &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How to use the forward-Euler method to numerically solve for &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; without inverting &lt;span class=&quot;math-container&quot;&gt;$M(y)$&lt;/span&gt;?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only came out following method:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Forward Euler, &lt;span class=&quot;math-container&quot;&gt;$y_{n+1} = y_n + h  \dot{y_n}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Multiply by &lt;span class=&quot;math-container&quot;&gt;$M(y_n)$&lt;/span&gt; to obtain &lt;span class=&quot;math-container&quot;&gt;$M(y_n) y_{n+1} = M(y_n) y_n + h M(y_n) \dot{y_n}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$M(y_n) y_{n+1} = M(y_n) y_n + h F(t, y_n) = \mathrm{RHS}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Solve &lt;span class=&quot;math-container&quot;&gt;$M(y_n) y_{n+1} = \mathrm{RHS}$&lt;/span&gt; by iterative matrix solver from start point &lt;span class=&quot;math-container&quot;&gt;$y_n$&lt;/span&gt; and get &lt;span class=&quot;math-container&quot;&gt;$y_{n+1}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But I think the procedure is not efficient since it needs to iterative solve &lt;span class=&quot;math-container&quot;&gt;$M(y_n) y_{n+1} = \mathrm{RHS}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone tell me a practical or more efficient method (need to use forward-Euler)?&lt;/p&gt;&#xA;" OwnerUserId="28929" LastEditorUserId="20688" LastEditDate="2018-10-10T14:34:23.623" LastActivityDate="2018-10-10T14:34:23.623" Title="How to solve an implicit ODE with forward Euler?" Tags="|ode|differential-equations|numerics|dae|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30325" PostTypeId="1" CreationDate="2018-10-10T10:27:08.623" Score="3" ViewCount="472" Body="&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; has eigenvalue &lt;span class=&quot;math-container&quot;&gt;$\lambda_A$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$B = I - c\frac{I-rA}{I-\bar{r}A}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to derive the eigenvalue &lt;span class=&quot;math-container&quot;&gt;$\lambda_B$&lt;/span&gt;?&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lambda_B=1-c\frac{1-r\lambda_A}{1-\bar{r}\lambda_A}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$c, r, \bar{r}$&lt;/span&gt; are constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This formula occurs at the derivation of speedup modified Newton iteration in solving DAE. There are no details in the paper for the derivation of this linear algebra piece.&lt;/p&gt;&#xA;" OwnerUserId="28929" LastEditorUserId="20688" LastEditDate="2018-10-10T14:37:07.777" LastActivityDate="2018-10-12T07:42:04.743" Title="Derive the formula for eigenvalues" Tags="|linear-algebra|eigenvalues|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="30327" PostTypeId="2" ParentId="30325" CreationDate="2018-10-10T11:45:06.530" Score="2" Body="&lt;p&gt;From a quick look at your problem, it seems to me that this should follow by writing &lt;span class=&quot;math-container&quot;&gt;$(I - \bar{r} A)^{-1}$&lt;/span&gt; as a geometric series, expanding its product with &lt;span class=&quot;math-container&quot;&gt;$I - r A$&lt;/span&gt;, and then applying the Spectral Mapping Theorem (which says that if &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a polynomial/power series, then the eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$B = p(A)$&lt;/span&gt; are the eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; transformed by &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2018-10-10T11:45:06.530" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30328" PostTypeId="1" CreationDate="2018-10-10T12:59:45.887" Score="1" ViewCount="41" Body="&lt;p&gt;I'm coding an iterative algorithm of constrained continuous optimisation. An augmented Lagrangian algorithm (outer) calls a bound-constrained L-BFGS-B algorithm (inner), which calls a line search algorithm. Optimisation problems have between 1 and 10,000 bound-constrained variables and 100–200 nonlinear constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The program outputs a solution vector (if found),  Lagrangian multipliers and termination reasons. Also, the output is a list of iterates, objective values and possibly Lagrangian multipliers per each inner and outer iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For problematic cases, I'd like to provide diagnostics info: warnings, numerical issues (matrix ill-conditioning, &lt;code&gt;NaN&lt;/code&gt;s, etc.) and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Scala to adapt some algorithms from the Breeze library, and also add a Lagrangian algorithm. Some algorithms in that library have logging statements everywhere, whereas others have very little logging. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What information is normally logged and what is output in numerical optimisation algorithms? &lt;/li&gt;&#xA;&lt;li&gt;How do I decide between logging vs outputting? Obviously, there's a trade-off between outputs complexity and the &quot;intrusiveness&quot; of logging.&lt;/li&gt;&#xA;&lt;li&gt;Will it make sense creating lean algorithm classes with diagnostics sub-classes containing logging? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Other advice, e.g., design tips, references and open-source code examples, are very welcome too. &lt;/p&gt;&#xA;" OwnerUserId="28930" LastEditorUserId="20688" LastEditDate="2018-10-10T14:32:46.873" LastActivityDate="2018-10-10T14:32:46.873" Title="Logging vs outputs in iterative optimisation" Tags="|optimization|constrained-optimization|iterative-method|design-patterns|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30329" PostTypeId="1" CreationDate="2018-10-10T14:59:32.603" Score="2" ViewCount="81" Body="&lt;p&gt;I am implementing the advection equation &lt;span class=&quot;math-container&quot;&gt;$u_x+(1/c)u_t=0$&lt;/span&gt; following a Crank-Nicholson finite difference scheme. The equation for this is&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  -\frac{\gamma}{4} w_{n-3 j+1} + w_{n-2 j+1} + \frac{\gamma}{4} w_{n-1 j+1}&#xA;  = \frac{\gamma}{4} w_{n-3 j} + w_{n-2 j} - \frac{\gamma}{4} w_{n-1 j}.&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In matrix form it can be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  A \bf{w}_{j+1} = \it B {\bf{w}_j } + \bf{b}&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;A = &#xA;\begin{pmatrix}&#xA;  1 &amp;amp; \frac{\gamma}{4} &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;  -\frac{\gamma}{4} &amp;amp; 1 &amp;amp; \frac{\gamma}{4} &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;  0 &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; 0 &amp;amp; 0  \\&#xA;  \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \vdots  \\&#xA;  0 &amp;amp; 0 &amp;amp; \cdots &amp;amp;  -\frac{\gamma}{4} &amp;amp; 1 &amp;amp; \frac{\gamma}{4} \\&#xA;  0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \cdots &amp;amp; -\frac{\gamma}{4} &amp;amp; 1&#xA;\end{pmatrix} &#xA;\; , \;&#xA;\bf{w}_{j+1} =&#xA;\begin{pmatrix}&#xA;  w_{1 j+1} \\&#xA;  w_{2 j+1} \\&#xA;  \vdots \\&#xA;  \vdots \\&#xA;  \vdots \\&#xA;  w_{n-2 j+1} \\&#xA;\end{pmatrix}&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;B = &#xA;\begin{pmatrix}&#xA;  1 &amp;amp; -\frac{\gamma}{4} &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;  \frac{\gamma}{4} &amp;amp; 1 &amp;amp; -\frac{\gamma}{4} &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;  0 &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; 0 &amp;amp; 0  \\&#xA;  \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \vdots  \\&#xA;  0 &amp;amp; 0 &amp;amp; \cdots &amp;amp;  \frac{\gamma}{4} &amp;amp; 1 &amp;amp; -\frac{\gamma}{4} \\&#xA;  0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; \cdots &amp;amp; \frac{\gamma}{4} &amp;amp; 1&#xA;\end{pmatrix} &#xA;\; , \;&#xA;\bf{w}_{j} =&#xA;\begin{pmatrix}&#xA;  w_{1 j} \\&#xA;  w_{2 j} \\&#xA;  \vdots \\&#xA;  \vdots \\&#xA;  \vdots \\&#xA;  w_{n-2 j} \\&#xA;\end{pmatrix}&#xA;\; , \; &#xA;\bf{b} = &#xA;\begin{pmatrix}&#xA;  \frac{\gamma}{4} w_{0 j+1} + \frac{\gamma}{4} w_{0j}  \\&#xA;   0 \\&#xA;   \vdots \\&#xA;   0 \\&#xA;   -\frac{\gamma}{4} w_{n-1 j+1}  - \frac{\gamma}{4} w_{n-1 j}&#xA;\end{pmatrix}&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an initial condition I have a Gaussian pulse &lt;span class=&quot;math-container&quot;&gt;$u(x,0)=\mathrm{e}^{-400(x-1/2)^2}$&lt;/span&gt;. The spacial domain is &lt;span class=&quot;math-container&quot;&gt;$x \in [0,1]$&lt;/span&gt;. Constant velocity &lt;span class=&quot;math-container&quot;&gt;$c=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dt=0.001$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dx=0.002$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\gamma=c dt/dx$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My implementation is in Python and I can provide it upon request.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attach a few screen captures of the &quot;wavefield&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TukEh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TukEh.png&quot; alt=&quot;Time 0.3 seconds&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gIJ3p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gIJ3p.png&quot; alt=&quot;time 0.4 seconds&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2j38F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2j38F.png&quot; alt=&quot;time 0.6 seconds&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tL1ff.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tL1ff.png&quot; alt=&quot;time 0.7 seconds&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please help me understand what is going on after the pulse hits the left boundary? I am using Dirichlet boundary conditions with 0 on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*****Update*****&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found someone else asking the same question and some interesting answers which I am still trying to digest.  Still I would like to have more discussion on this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/5425/strange-oscillation-when-solving-the-advection-equation-by-finite-difference-wit?rq=1&quot;&gt;link on this&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28330" LastEditorUserId="28330" LastEditDate="2018-10-15T13:10:30.167" LastActivityDate="2018-10-15T13:10:30.167" Title="Efects from the boundary in advection equation" Tags="|finite-difference|hyperbolic-pde|advection|numerics|" AnswerCount="0" CommentCount="6" ClosedDate="2018-10-16T21:45:34.050" ContentLicense="CC BY-SA 4.0" />
  <row Id="30330" PostTypeId="2" ParentId="30298" CreationDate="2018-10-10T17:57:49.030" Score="1" Body="&lt;p&gt;With your current model you are forcing your non splittable flows to be equal to your demand on all arcs ( &lt;span class=&quot;math-container&quot;&gt;$ x^f_{ij} \geq d_f, \forall f \in NS$&lt;/span&gt;). Unless there is a single path from your source to your target this is going to make your problem infeasible, as the flow conservation constraints are going to conflict with this. I am afraid that for non splittable flows you do not have many alternatives to model whether or not a flow goes through an arc using a binary variable, effectively turning your problem into a MIP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1:&lt;/strong&gt; To model the problem as a MIP, for the non-splittable flows you need to replace the &lt;span class=&quot;math-container&quot;&gt;$x_{ij}^f$&lt;/span&gt; linear variables by &lt;span class=&quot;math-container&quot;&gt;$y_{ij}^f$&lt;/span&gt; binary variables representing if the complete flow goes through arc &lt;span class=&quot;math-container&quot;&gt;$ij$&lt;/span&gt; or not. You should then rewrite your constraint and objective function by replacing for non splittable flows &lt;span class=&quot;math-container&quot;&gt;$x_{ij}^f$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$y_{ij}^fd_f$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="28933" LastEditorUserId="28933" LastEditDate="2018-10-12T17:47:58.097" LastActivityDate="2018-10-12T17:47:58.097" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="30332" PostTypeId="1" CreationDate="2018-10-11T16:18:07.213" Score="1" ViewCount="213" Body="&lt;p&gt;Ideally, I would like to get the symbolic/algebraic integral of the function and plot the resulting surface in 3d. I am not sufficiently versed in SciPy to know if this is even really possible.&lt;/p&gt;&#xA;" OwnerUserId="28942" LastActivityDate="2018-10-14T01:49:30.297" Title="SciPy 3d plotting Integral of $\int x^y dx$ for $y$ in $[-4,4]$" Tags="|scipy|plotting|calculus|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30333" PostTypeId="2" ParentId="30325" CreationDate="2018-10-11T18:50:51.153" Score="5" Body="&lt;p&gt;It's just some easy matrix algebra. If &lt;span class=&quot;math-container&quot;&gt;$Av=\lambda_A v$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$(I-rA)v= (1-r\lambda_A)v$&lt;/span&gt; and (multiplying by inverses on both sides) &lt;span class=&quot;math-container&quot;&gt;$(I-rA)^{-1}v= (1-r\lambda_A)^{-1}v$&lt;/span&gt;, which is valid also if you replace &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\bar{r}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you just expand &lt;span class=&quot;math-container&quot;&gt;$Bv = (I-c(I-rA)(I-\bar{r}A)^{-1})v=\dots$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT: wim's answer carries out this computation in more detail.]&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2018-10-12T07:42:04.743" LastActivityDate="2018-10-12T07:42:04.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30334" PostTypeId="2" ParentId="30325" CreationDate="2018-10-11T19:19:32.370" Score="4" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$(\lambda,x)$&lt;/span&gt; be an eigenpair of matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, so &lt;span class=&quot;math-container&quot;&gt;$A x=\lambda x$&lt;/span&gt;.&#xA;Now we compute &lt;span class=&quot;math-container&quot;&gt;$B x$&lt;/span&gt;, and see if it is equal to &#xA;&lt;span class=&quot;math-container&quot;&gt;$\left(1-c\frac{1-r\lambda}{1-\bar{r}\lambda}\right)x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \begin{align}&#xA;B x &amp;amp; = \left( I - c\frac{I-rA}{I-\bar{r}A} \right) x\\&#xA;    &amp;amp; =  x - c\frac{x-rAx}{I-\bar{r}A}  = x - c\frac{x-r \lambda x}{I-\bar{r}A}\\&#xA;    &amp;amp; = x - c\frac{1-r \lambda }{I-\bar{r}A}x\\&#xA;&amp;amp; = x - c\frac{1-r \lambda }{I-\bar{r}A} \, \frac{I-\bar{r}\lambda}{I-\bar{r}\lambda}x\\&#xA;&amp;amp; = x - c\frac{1-r \lambda }{I-\bar{r}A} \, \frac{x-\bar{r}Ax}{I-\bar{r}\lambda}\\&#xA;&amp;amp; = x - c\frac{1-r \lambda }{I-\bar{r}A} \, \frac{I-\bar{r}A}{I-\bar{r}\lambda}x\\&#xA;&amp;amp; = \left( 1 - c\frac{1-r \lambda }{I-\bar{r}\lambda}\right)x\\&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, we may conclude that following relationship between the eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; holds:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lambda_B = 1 - c\frac{1-r \lambda_A }{I-\bar{r}\lambda_A} ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as long as &lt;span class=&quot;math-container&quot;&gt;$I-\bar{r}A$&lt;/span&gt; is invertible. &#xA;Note that it is not necessary that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is diagonalizable.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the comments there has been some discussion whether or not &lt;span class=&quot;math-container&quot;&gt;$(I-\alpha A)^{-1}$&lt;/span&gt; commutes with &lt;span class=&quot;math-container&quot;&gt;$I-\beta A$&lt;/span&gt;. If we assume that &lt;span class=&quot;math-container&quot;&gt;$I-\alpha A$&lt;/span&gt; is invertible, then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \begin{align}&#xA;(I-\alpha A)^{-1}(I-\beta A) &#xA;&amp;amp; = (I-\alpha A)^{-1}(I-\beta A)(I-\alpha A)(I-\alpha A)^{-1}\\&#xA;&amp;amp; = (I-\alpha A)^{-1}(I-\beta A -\alpha A +\alpha \beta A^2)(I-\alpha A)^{-1}\\&#xA;&amp;amp; = (I-\alpha A)^{-1}(I-\alpha A)(I-\beta A)(I-\alpha A)^{-1}\\&#xA;&amp;amp; = (I-\beta A)(I-\alpha A)^{-1} ,\\&#xA;\end{align}&#xA;$$&lt;/span&gt;&#xA;which shows that indeed &lt;span class=&quot;math-container&quot;&gt;$(I-\alpha A)^{-1}$&lt;/span&gt; commutes with &lt;span class=&quot;math-container&quot;&gt;$I-\beta A$&lt;/span&gt;. Therefore &#xA;we can write  &lt;span class=&quot;math-container&quot;&gt;$\frac{I-\beta A}{I-\alpha A}$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$(I-\alpha A)^{-1}(I-\beta A)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$(I-\beta A)(I-\alpha A)^{-1}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5286" LastActivityDate="2018-10-11T19:19:32.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30336" PostTypeId="1" CreationDate="2018-10-12T11:48:54.633" Score="1" ViewCount="48" Body="&lt;p&gt;In &lt;a href=&quot;https://www.doi.org/10.1007/978-1-4419-8853-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yurii Nesterov's Introductory Lectures on Convex Optimization&lt;/a&gt;, there is a bound for the total number of iterations for some process. See page 109:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left[\frac{1}{\ln(2(1-\kappa))} \ln\frac{t_0-t^*}{(1-\kappa) \epsilon}+2\right]\cdot \left[1+\sqrt\frac{L}{\mu}\ln\frac{2(L-\mu)}{\kappa \mu}\right]\\+\sqrt\frac{L}{\mu}\cdot\ln\left(\frac{1}{\epsilon}\max_{1\leq i \leq m}\{f_0(x_0)-t_0; f_i(x_0)\}\right)\label{eq1}\tag{1}$$&lt;/span&gt;                     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the principal term in the above estimate is of the order&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\ln\frac{t_0-t^*}{\epsilon} \sqrt\frac{L}{\mu} \ln\frac{L}{\mu} \label{eq2}\tag{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How did we arrive at statement &lt;span class=&quot;math-container&quot;&gt;$\eqref{eq2}$&lt;/span&gt;? Is that true the second term &lt;span class=&quot;math-container&quot;&gt;$\sqrt\frac{L}{\mu}\cdot\ln\big(\frac{1}{\epsilon}\max_{1\leq i \leq m}\{f_0(x_0)-t_0; f_i(x_0)\}\big)$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\eqref{eq1}$&lt;/span&gt; is eliminated? I would appreciate any advice here.&lt;/p&gt;&#xA;" OwnerUserId="27664" LastEditorUserId="20688" LastEditDate="2018-10-14T12:25:43.603" LastActivityDate="2018-10-14T12:25:43.603" Title="Order of a principal term" Tags="|optimization|convex-optimization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30337" PostTypeId="2" ParentId="30315" CreationDate="2018-10-12T16:26:26.023" Score="3" Body="&lt;p&gt;I found that the solution of the PML problem by the method proposed here should not work. Next is a detailed analysis of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start with the von Neumann stability analysis for the FTCS system corresponding to (eq 3.) on the original question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{ w_{i j+1}  - w_{ij}}{\Delta t} = &#xA;    \frac{c}{2 \Delta x} (w_{i+1 j} - w_{i-1 j})&#xA;  -  \sigma_i w_{ij},&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;Call &lt;span class=&quot;math-container&quot;&gt;$\gamma=c \Delta t/\Delta x$&lt;/span&gt; (the CFL number) and so we have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  w_{i j+1} =  \frac{\gamma}{2} ( w_{i+1 j} - w_{i-1 j}) - w_{ij}(\sigma_i \Delta -1).&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the von Neumann analysis we assume &lt;span class=&quot;math-container&quot;&gt;$w_{ij}=\mathrm{e}^{\mathrm{i} \omega t_j} &#xA;\mathrm{e}^{\mathrm{i} k x_k}$&lt;/span&gt; and call &lt;span class=&quot;math-container&quot;&gt;$G=w_{i j+1}/w_{ij}=\mathrm{e}^{\mathrm{i} \omega \Delta t }$&lt;/span&gt;  the complex gain factor.&#xA;After replacing and dividing by &lt;span class=&quot;math-container&quot;&gt;$w_{ij}$&lt;/span&gt; we find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA; G = \mathrm{e}^{\mathrm{i} \omega \Delta t} =&#xA; \gamma \mathrm{i} \sin k \Delta x - \sigma_i \Delta t +1.&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;and, since &lt;span class=&quot;math-container&quot;&gt;$\gamma, \sigma_i k, \Delta x, \Delta t \in \mathbb{R}$&lt;/span&gt; we find that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  |G| = \sqrt{(1-\sigma_i \Delta t)^2 + \gamma^2 \sin^2 k \Delta x}&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;If we want &lt;span class=&quot;math-container&quot;&gt;$|G|&amp;lt;1$&lt;/span&gt; then we need&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \gamma^2 \sin^2 k \Delta x \le 1 - (1 -\sigma_i \Delta t)^2&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;so that, we need&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \gamma \le \sqrt{ \frac{1-(1-\sigma_i \Delta t)^2}{|\sin k \Delta x|}}&#xA;  \le \sqrt{ 1 - (1 - \sigma_i \Delta t)^2}&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;If &lt;span class=&quot;math-container&quot;&gt;$\sigma_i=0$&lt;/span&gt; (case  &lt;span class=&quot;math-container&quot;&gt;$x=x_0$&lt;/span&gt;, starting point of buffer zone where &lt;span class=&quot;math-container&quot;&gt;$f(x_0)=0$&lt;/span&gt;) then &#xA;&lt;span class=&quot;math-container&quot;&gt;$\gamma \le 0$&lt;/span&gt;, and this impedes the election of &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; which guarantee stability.&#xA;This idea then does not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us now try forward differences in both domain FTFS:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{ w_{i j+1}  - w_{ij}}{\Delta t} = \frac{c}{\Delta x} (w_{i+1 j} - w_{i j})&#xA;  -  \sigma_i w_{ij},&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;That is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  w_{i j+1} = \gamma w_{i+1 j} - (\sigma_i \Delta t+\gamma -1) w_{ij}.&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We perform von Neumann stability analysis:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  G = \mathrm{e}^{\mathrm{i} \omega \Delta t} &amp;amp;=&amp;amp; &#xA;  \gamma \mathrm{e}^{\mathrm{i} k \Delta x} -(\sigma_i \Delta t + \gamma -1)  \\&#xA;  &amp;amp;=&amp;amp; \gamma \cos k \Delta x -(\sigma_i \Delta t + \gamma -1) + \mathrm{i} \gamma \sin k&#xA;  \Delta x&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;La absolute value of  &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is given by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;|G| &amp;amp;=&amp;amp; \gamma^2 - 2 \gamma \cos k \Delta x(\sigma_i \Delta t + \gamma -1) +&#xA; (\sigma_i \Delta t + \gamma -1)^2 \\&#xA; &amp;amp;=&amp;amp; \gamma^2 (2 - 2 \cos k \Delta x ) + 2 \gamma ( \sigma_i \Delta t -1 - \cos k \Delta x&#xA; + \sigma_i \Delta t \cos k \Delta x) -2 \sigma_i \Delta t + 1 \\&#xA; &amp;amp;=&amp;amp; \gamma^2 (2 - 2 \cos k \Delta x ) + 2 \gamma &#xA; [ \sigma_i \Delta t -1 + \cos k \Delta x( \sigma_i \Delta t -1)] -2 \sigma_i \Delta t + 1 \\&#xA; &amp;amp;=&amp;amp; \gamma^2 (2 - 2 \cos k \Delta x ) + 2 \gamma &#xA; [ (\sigma_i \Delta t -1)(1 + \cos k \Delta x)] -2 \sigma_i \Delta t + 1 \\&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;If we want  &lt;span class=&quot;math-container&quot;&gt;$|G| \le 1$&lt;/span&gt; we need&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA; \gamma^2 (2 - 2 \cos k \Delta x ) + 2 \gamma &#xA; [ (\sigma_i \Delta t -1)(1 + \cos k \Delta x)] -2 \sigma_i \Delta t \le 0&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;The left hand-side is a parabola convex up, since &#xA;&lt;span class=&quot;math-container&quot;&gt;$2-2 \cos k \Delta x &amp;gt; 0$&lt;/span&gt;. &#xA;The discriminant &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; of this equation is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  D = 4 (\sigma_i \Delta t-1)^2 (1+ \cos k \Delta x)^2 +8 \sigma_i \Delta t&#xA;  (2 - 2 \cos k \Delta x) &amp;gt; 0&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;so that there exist a range of values of &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; where this parabola is under&#xA;the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; axis. This range is given by the interval:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  [ -2(\sigma_i \Delta t -1)(1 + \cos k \Delta x) - \sqrt{D} \, , \,&#xA;  -2(\sigma_i \Delta t -1)(1 + \cos k \Delta x) + \sqrt{D}]&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; can be as close as we want to&#xA; &lt;span class=&quot;math-container&quot;&gt;$2 ( \sigma_i \Delta t -1)(1+\cos k \Delta x)$&lt;/span&gt; by making  &lt;span class=&quot;math-container&quot;&gt;$k \Delta x$&lt;/span&gt; close to  &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&#xA;Hence we cannot guaranteed   &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;gt; 0$&lt;/span&gt; for stability. We see two classical methods&#xA;for which stability fails.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We now turn our attention to implicit methods such as the Crank-Nicholson method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us get back to equation  (eq.3) and write:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;       w_{i j+1} = &#xA;       \frac{\gamma}{4} ( w_{i+1 j+1} - w_{i-1 j+1} ) + \frac{\gamma}{4}&#xA;       ( w_{i+1 j} - w_{i-1 j}) + w_{i j-1} (1 -\sigma_i).&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;We do a von Neumann analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming: &lt;span class=&quot;math-container&quot;&gt;$G=w_{i j+1}/w_{ij}= \mathrm{e}^{\mathrm{i} \omega \Delta t}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  G &amp;amp;=&amp;amp; \frac{\gamma}{2} G \mathrm{i} \sin k \Delta x + &#xA;  \frac{\gamma}{2} \mathrm{i} \sin k \Delta x + (1-\sigma_i) \\&#xA;  \left ( 1 - \mathrm{i} \frac{\gamma}{2} \sin k \Delta x \right) G&#xA;  &amp;amp;=&amp;amp; \left ( 1 + \mathrm{i} \frac{\gamma}{2} \sin k \Delta x - \sigma_i \right )&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;that is,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  | G | = \frac{(1-\sigma_i)^2 + (\gamma^2/4) \sin^2 k \Delta x}&#xA;   {1 + (\gamma^2/4) \sin^2 k \Delta x}.&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;We find that if, &lt;span class=&quot;math-container&quot;&gt;$0&amp;lt;\sigma_i&amp;lt;2$&lt;/span&gt; then  &lt;span class=&quot;math-container&quot;&gt;$|G| \le 1$&lt;/span&gt; and the system is unconditionally convergent and if  &lt;span class=&quot;math-container&quot;&gt;$\sigma_i &amp;lt; 0$&lt;/span&gt; the system is unconditionally divergent. &#xA;The solution (eq.4) diverges for &lt;span class=&quot;math-container&quot;&gt;$f(x)&amp;gt;0$&lt;/span&gt; (assuming &lt;span class=&quot;math-container&quot;&gt;$k&amp;gt;0, \omega &amp;gt; 0$&lt;/span&gt;), and if,&#xA; &lt;span class=&quot;math-container&quot;&gt;$\sigma(x) &amp;gt;0$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$f(x)=\int_{x_0}^x \sigma(s) ds &amp;gt;0 $&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$x_0&amp;lt;x$&lt;/span&gt; and&#xA;&lt;span class=&quot;math-container&quot;&gt;$f(x) \le 0$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$x_0&amp;gt;x$&lt;/span&gt;. However the solution (eq.4) corresponds to a pulse traveling to the left, so that it diverges for &lt;span class=&quot;math-container&quot;&gt;$x&amp;lt;x_0$&lt;/span&gt; (left).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We conclude that all methods tried to solve the PML problem based on the &#xA;factorization of the two-way wave equation into two single-way way equations are not appropriate.&lt;/p&gt;&#xA;" OwnerUserId="28330" LastActivityDate="2018-10-12T16:26:26.023" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30338" PostTypeId="1" AcceptedAnswerId="32709" CreationDate="2018-10-12T17:10:00.443" Score="3" ViewCount="681" Body="&lt;p&gt;MKL and FFTW offer 1-D FFTs that can operate on many inputs simultaneously - in other words, they can batch-transform the columns of some input matrix. Is the performance of these multi-transforms significantly superior to just looping over the inputs (assuming they are stored as columns of the same matrix) and performing individual FFTs on each column? Google has turned up nothing, and I do not presently have the ability to test this out.&lt;/p&gt;&#xA;" OwnerUserId="27171" LastEditorUserId="27171" LastEditDate="2018-10-12T19:09:05.137" LastActivityDate="2019-05-21T07:21:48.450" Title="MKL/FFTW performance of batch 1-D FFTs" Tags="|performance|fftw|intel-mkl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30340" PostTypeId="1" AcceptedAnswerId="30342" CreationDate="2018-10-12T18:26:04.883" Score="3" ViewCount="178" Body="&lt;p&gt;I am cross posting this question to the mathermatics stack exchange. please find it either at this link, &lt;a href=&quot;https://math.stackexchange.com/q/2952989/430980&quot;&gt;https://math.stackexchange.com/q/2952989/430980&lt;/a&gt;, or below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a question concerning the derivatives of approximate matrix inverses. I have a system, &lt;span class=&quot;math-container&quot;&gt;$$Ax = b$$&lt;/span&gt; which I solve approximately (and with an iterative method) with:   &lt;span class=&quot;math-container&quot;&gt;$$\Delta x = x - \tilde{A}^{-1}b$$&lt;/span&gt;   I would like to take the derivative of this process, i.e. find &lt;span class=&quot;math-container&quot;&gt;$$\frac{d\Delta x}{dF} = \frac{dx}{dF} - \frac{d\tilde{A}^{-1}}{dF}b - \tilde{A}^{-1}\frac{db}{dF}$$&lt;/span&gt; I know that if I had an ideal inverse, my expression would be without tildes as I'd have the exact result: &lt;span class=&quot;math-container&quot;&gt;$$\frac{d\Delta x}{dF} = \frac{dx}{dF} - {A}^{-1}\frac{dA}{dF}{A}^{-1}b - \tilde{A}^{-1}\frac{db}{dF}$$&lt;/span&gt; But for this, I cannot start with the typical assumption that &lt;span class=&quot;math-container&quot;&gt;$$AA^{-1} = I$$&lt;/span&gt; and instead have &lt;span class=&quot;math-container&quot;&gt;$$A\tilde{A}^{-1} = C \neq I$$&lt;/span&gt; Does anyone have any ideas for resources or techniques for such problems?&#xA;Thank you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;To add some additional context, I'm using a Newton method in a CFD solver, so my linear system is &lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial R}{\partial u}\Delta u= -R(u)$$&lt;/span&gt; I'm attempting to differentiate the process. I solve the linear system using point Jacobi, and I break up the matrix into diagonal and off diagonal, so it looks like so: &lt;span class=&quot;math-container&quot;&gt;$$D \Delta u^{k+1}= -R(u) - \frac{\partial R(u) }{\partial u}\Delta u^{k}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="9446" LastEditorUserId="9446" LastEditDate="2019-02-11T17:22:40.600" LastActivityDate="2019-02-11T17:22:40.600" Title="Derivatives of Approximate Matrix inverses" Tags="|linear-algebra|linear-solver|numerics|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="30341" PostTypeId="2" ParentId="30332" CreationDate="2018-10-13T02:45:36.163" Score="3" Body="&lt;p&gt;As you may know:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \int x^y \, dx = \dfrac{x^{y+1}}{y+1},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$y \neq 1.$&lt;/span&gt; If you want to plot the function &lt;span class=&quot;math-container&quot;&gt;$ \dfrac{x^{y+1}}{y+1}$&lt;/span&gt; for a specific &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; using Matplotlib, here is the answer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;&#xA;x = np.linspace(-4, 4, 30)&#xA;y = np.linspace(-4, 4, 30)&#xA;&#xA;def func(x, y, expn):&#xA;    return x ** (1 + expn) / (1 + expn)&#xA;&#xA;X, Y = np.meshgrid(x, y)&#xA;&#xA;# Choose the exponent you want. In this case, you change expn from -4 to 4&#xA;Z = func(X, Y, -3.9)&#xA;&#xA;fig = plt.figure() &#xA;ax = Axes3D(fig) &#xA;ax.plot_surface(X, Y, Z) &#xA;ax.set_proj_type('ortho') &#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, I am plotting a surface instead of a line.&lt;/p&gt;&#xA;" OwnerUserId="27272" LastEditorUserId="27272" LastEditDate="2018-10-14T01:49:30.297" LastActivityDate="2018-10-14T01:49:30.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30342" PostTypeId="2" ParentId="30340" CreationDate="2018-10-13T03:09:00.257" Score="2" Body="&lt;p&gt;Call the approximate inverse &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, then its products with &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; are&#xA;&lt;span class=&quot;math-container&quot;&gt;$$AV = I+E_{a},\quad VA = I+E_{v}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$E_k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E_a$&lt;/span&gt; are &quot;small&quot; random error matrices which characterize your iterative method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error matrices are related since&#xA;&lt;span class=&quot;math-container&quot;&gt;$$VAV=V+VE_a=V+E_vV\implies VE_a=E_vV$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$AVA=A+E_aA=A+AE_v\implies AE_v=E_aA$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pick one of the two products, differentiate, and solve for &lt;span class=&quot;math-container&quot;&gt;$dV$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\eqalign{&#xA;V\,dA + dV\,A &amp;amp;= dE_v \cr&#xA;dV\,A &amp;amp;= dE_v - V\,dA \cr&#xA;dV\,AV &amp;amp;= dE_vV - V\,dA\,V \cr&#xA;dV &amp;amp;= (dE_vV - V\,dA\,V)\,(AV)^{-1} \cr&#xA;}$$&lt;/span&gt;&#xA;When &lt;span class=&quot;math-container&quot;&gt;$\,(dE_vV)\,$&lt;/span&gt; is negligible&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\eqalign{&#xA;dV &amp;amp;= -(V\,dA\,V)\,(AV)^{-1} \cr&#xA;}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="27795" LastEditorUserId="27795" LastEditDate="2019-02-10T17:20:25.850" LastActivityDate="2019-02-10T17:20:25.850" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30343" PostTypeId="1" AcceptedAnswerId="30344" CreationDate="2018-10-13T10:41:16.767" Score="4" ViewCount="203" Body="&lt;p&gt;I need to calculate a matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (at least some elements of it, see below) as defined by the following equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ A=B(\mathbb{1}-B)^{-1} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where B is a square matrix of dimension &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbb{1}$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$N \times N$&lt;/span&gt; identity matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inspired by this post:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.johndcook.com/blog/2010/01/19/dont-invert-that-matrix/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.johndcook.com/blog/2010/01/19/dont-invert-that-matrix/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if I really need to invert &lt;span class=&quot;math-container&quot;&gt;$\mathbb{1}-B$&lt;/span&gt; in my case, or if there's some easier way. Keep in mind that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; in my case is quite large, its order of magnitude can be ten thousand.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't need to know the full matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, I just need a few elements in the upper left corner, let's say &lt;span class=&quot;math-container&quot;&gt;$A_{00}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A_{01}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A_{11}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A_{02}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A_{12}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A_{22}$&lt;/span&gt; would be perfect.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7560" LastEditorUserId="20688" LastEditDate="2018-10-13T18:45:20.420" LastActivityDate="2018-10-13T18:45:20.420" Title="Do I really need to invert this matrix" Tags="|linear-algebra|matrix|inverse|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30344" PostTypeId="2" ParentId="30343" CreationDate="2018-10-13T12:21:32.533" Score="11" Body="&lt;p&gt;Since&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A = B(I-B)^{-1} = (I-B)^{-1}(I-B)B(I-B)^{-1} = (I-B)^{-1}B(I-B)(I-B)^{-1} =(I-B)^{-1}B&#xA;$$&lt;/span&gt;&#xA;So you want to solve&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(I-B)A=B&#xA;$$&lt;/span&gt;&#xA;You seem to need only the first three columns of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Solve the matrix problems&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(I-B)a_i = b_i, \qquad i=0,1,2&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$b_0,b_1,b_2$&lt;/span&gt; are first three columns of &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. Then &lt;span class=&quot;math-container&quot;&gt;$a_0,a_1,a_2$&lt;/span&gt; are the first three columns of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-10-13T12:21:32.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30346" PostTypeId="1" AcceptedAnswerId="30350" CreationDate="2018-10-13T16:05:11.167" Score="1" ViewCount="318" Body="&lt;p&gt;When solving a linear system &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$A=B^TCB$&lt;/span&gt; do I need to form &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; explicitly by two matrix-matrix multiplications or is there another more simple way?&#xA;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a NxN matrix and not always symmetric. &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is NxM and not symmetric.&lt;/p&gt;&#xA;" OwnerUserId="28358" LastEditorUserId="28358" LastEditDate="2018-10-15T05:51:39.000" LastActivityDate="2018-10-15T14:06:15.223" Title="Solving linear system with matrix multiplication" Tags="|linear-algebra|matrix|matrix-equations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30347" PostTypeId="2" ParentId="29315" CreationDate="2018-10-14T04:05:51.687" Score="1" Body="&lt;p&gt;I have changed your code to this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.integrate import odeint&#xA;&#xA;def dP_dr(P, r, p_c, R, G, c):&#xA;    return - G * (P + density(r, p_c, R) / c**2) * (mass(r, p_c, R) &#xA;             + 4 * np.pi * r**3 * P / c**2) / (r * (r - 2 * G* mass(r, p_c, R) / c**2))&#xA;&#xA;def mass(r, p_c, R):&#xA;    return 4 * np.pi * p_c * (5 * r**3 * R**2 - 3 * r**5) / (15 * R**2)&#xA;&#xA;def density(r, p_c, R):&#xA;    return p_c * (1 - (r/R)**2)&#xA;&#xA;p_c = 7 * 10**10     #central density&#xA;P_c = 1.6 * 10**15    #central pressure&#xA;R = 10              #radius&#xA;c = 3 * 10**8&#xA;G = 6.674 * 10**(-11)&#xA;&#xA;&#xA;deltaR = 0.0001&#xA;r = np.linspace(deltaR, R, int(R/deltaR))&#xA;&#xA;P, dic = odeint(dP_dr, P_c, r, args=(p_c, R, G, c), full_output=True, printmessg=True)&#xA;&#xA;num = 10000&#xA;plt.figure()&#xA;plt.plot(r[:num], P[:num], label='Pressure')&#xA;plt.xlabel('Radius (km)')&#xA;plt.title('Neutron Star')&#xA;plt.legend()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You have to play with &lt;code&gt;num&lt;/code&gt;, &lt;code&gt;p_c&lt;/code&gt;, &lt;code&gt;P_c&lt;/code&gt;. With the numbers you show, &lt;span class=&quot;math-container&quot;&gt;$P(r)$&lt;/span&gt; decays very fast. By the way, I have added two constants to the function &lt;code&gt;dP_dr&lt;/code&gt;, &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27272" LastActivityDate="2018-10-14T04:05:51.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30348" PostTypeId="2" ParentId="18987" CreationDate="2018-10-14T13:07:41.900" Score="1" Body="&lt;p&gt;Here's a very simple way to create pink noise in C#, which just sums lots of waves (spaced logarithmically apart) together! The functions outputs a double array with values from -1 to 1. This represents the lowest and highest points in the waveform respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 'quality' parameter represents the number of waves produced to make the sound. I find 5000 waves is just about the threshold where I can't detect any noticeable improvement with higher values, but to be on the safe side, you could (optionally) increase this to about 10,000 or higher. Also, according to Wikipedia, 20 hertz is around the lower limit of human perception in terms of what we can hear, but you can change this too if you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the sound gets quieter with a higher quality value due to technical reasons, so you may (optionally) want to adjust the volume via the volumeAdjust parameter.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;public double[] createPinkNoise(double seconds, int quality=5000, double lowestFrequency=20, double highestFrequency = 44100, double volumeAdjust=1.0)&#xA;{&#xA;    long samples = (long)(44100 * seconds);&#xA;    double[] d = new double[samples];&#xA;    double[] offsets = new double[samples];&#xA;    double lowestWavelength = highestFrequency / lowestFrequency;&#xA;    Random r = new Random();&#xA;    for (int j = 0; j &amp;lt; quality; j++)&#xA;    {&#xA;        double wavelength = Math.Pow(lowestWavelength, (j * 1.0) / quality)  * 44100 / highestFrequency;&#xA;        double offset = r.NextDouble() * Math.PI*2;     // Important offset is needed, as otherwise all the waves will be almost in phase, and this will ruin the effect!&#xA;        for (int i = 0; i &amp;lt; samples; i++)&#xA;        {&#xA;            d[i] += Math.Cos(i * Math.PI * 2 / wavelength + offset) / quality * volumeAdjust;&#xA;        }&#xA;    }&#xA;    return d;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28963" LastEditorUserId="28963" LastEditDate="2018-10-15T17:35:04.657" LastActivityDate="2018-10-15T17:35:04.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30349" PostTypeId="2" ParentId="30296" CreationDate="2018-10-14T17:12:41.167" Score="4" Body="&lt;p&gt;The credits of this should go to @rchilton1980.  I only want to validate the hypothesis of dispersion as the result of the response of a spike to the finite difference operator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\bf{  Basic \; Theory: }$&lt;/span&gt;&lt;br&gt;&#xA;Physical dispersion accounts for the phenomena of waves moving in a way&#xA;that different frequencies or wavenumbers travel at different velocities.&#xA;We can make an analogy with a group of cliclist traveling along a flat road&#xA;at constant speed. Once they hit the mountain it could happen that the heaviest&#xA;cliclists will ride slower and the group will spread (or disperse) along the road. If we&#xA;think of a wavelet as the supperposition (synthesis) of many frequencies (and wavenumbers)&#xA;it could happen that the velocity of each frequency is different. However we&#xA;need to be more precise on the meaning of velocity here.&#xA;We define two velocities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\bf{Phase \; velocity} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;        v_f= \frac{\omega}{k}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\bf {  Group \; Velocity: }$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;    v_g= \frac{\partial \omega}{\partial k}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The previous definitions indicate, in a way, that the angular frequency &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt;&#xA;should be a function of the wavenumber &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. That is, &lt;span class=&quot;math-container&quot;&gt;$\omega=\omega(k)$&lt;/span&gt;.&#xA;To better understand this let us consider the 1D wave equation with constant&#xA;wavespeed &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{\partial^2 u}{\partial x^2}  -\frac{1}{c^2} \frac{\partial^2 u}{\partial t^2} = 0.&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;If we take a double Fourier transform (from time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; to frequency &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt;, and&#xA;from space &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; to wavenumber &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;)  we find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \left ( -k^2 + \frac{\omega^2}{c^2} \right ) U(k,\omega) = 0,&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$U(k,\omega)$&lt;/span&gt; is the Fourier transform of &lt;span class=&quot;math-container&quot;&gt;$u(x,t)$&lt;/span&gt;.&#xA;We get, from here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \omega = \pm c k&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;The &quot;&lt;span class=&quot;math-container&quot;&gt;$+$&lt;/span&gt;&quot; sign indicates a wave traveling to the right, while&#xA;The ``&lt;span class=&quot;math-container&quot;&gt;$-$&lt;/span&gt;'' sign indicates a wave traveling to the left.&#xA;From now on, we will ignore the sign on this discussion. Then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{\omega}{k} = \frac{\partial \omega}{\partial k} = c&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;So we find that both phase and group velocity coincide with the wavespeed &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;.&#xA;So there is no physical dispersion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when we find a finite difference approximation we could introduce&#xA;dispersion (numerical dispersion). That is, both phase and group velocities&#xA;could change with frequency (or wave number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us consider the following finite difference scheme for the wave equation&#xA;without source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;  w_{i j+1} = \gamma^2 w_{i-1 j} + 2(1-\gamma^2) w_{ij} +&#xA;  \gamma^2 w_{i+1 j} - w_{i j-1},&#xA;\end{eqnarray}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is the CFL number &lt;span class=&quot;math-container&quot;&gt;$c \Delta t/\Delta x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We take the Fourier transform of this expression, and using the shift property of the Fourier &#xA;transform to find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  W_{ij} \left (  \mathrm{e}^{\mathrm{i} \omega \Delta t} + &#xA;   \mathrm{e}^{-\mathrm{i} \omega \Delta t} \right ) = \left ( &#xA;   \gamma^2 \mathrm{e}^{-\mathrm{i} k \Delta x}&#xA;  + 2(1 - \gamma^2) W_{ij} + \gamma^2 \mathrm{e}^{\mathrm{i} k \Delta x} W_{ij} \right )&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$W_{ij}$&lt;/span&gt;  is the Fourier transform of &lt;span class=&quot;math-container&quot;&gt;$w_{ij}$&lt;/span&gt;.  That is, we find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \left [ 2 \cos \omega \Delta t - 2 \gamma^2  \cos k \Delta x &#xA;  + 2(1 - \gamma^2)  \right ] W_{ij} = 0.&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;We isolate &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; from here so that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \omega \Delta t = \arccos \left (  \gamma^2 \cos k \Delta x - (1 - \gamma^2) \right ),&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;That is,&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  v_f = \frac{\omega}{k} = \frac{1}{k \Delta t} \arccos &#xA;    \left (  \gamma^2 \cos k \Delta x - (1 - \gamma^2) \right ),&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;Clearly &lt;span class=&quot;math-container&quot;&gt;$v_f$&lt;/span&gt; changes with wavenumber &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. That is there is numerical dispersion.&#xA;The group velocity is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{\partial \omega}{\partial k} = -\frac{1}{\Delta t} &#xA;  \frac{  -\gamma^2 \sin k \Delta x}&#xA;  { \sqrt{1 -  \gamma^2 \cos k \Delta x - (1 - \gamma^2) }}.&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;Again, here group velocity &lt;span class=&quot;math-container&quot;&gt;$v_g$&lt;/span&gt; changes with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a special case given by &lt;span class=&quot;math-container&quot;&gt;$\gamma=1$&lt;/span&gt;. Here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  w \Delta t = \arccos(\cos k \Delta x)= k \Delta x&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  \frac{\omega}{k} = \frac{\Delta x}{\Delta t} \\&#xA;  \frac{\partial \omega}{\partial k} = \frac{\Delta x}{\Delta t} \\&#xA;\end{eqnarray*}&lt;/span&gt;&#xA;and since &lt;span class=&quot;math-container&quot;&gt;$\gamma=c \Delta t/\Delta x =1$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$c=\Delta x/\Delta t$&lt;/span&gt; and&#xA;the phase and group velocities coincide with &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. So there is no dispersion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; 1$&lt;/span&gt; then we can expect dispersion as shown in the original question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I include the plots after a few fractions of second of propagation.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TlFbW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TlFbW.png&quot; alt=&quot;Initial data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LX9Tx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LX9Tx.png&quot; alt=&quot;Propagation at t=0.1sec&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oouEb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oouEb.png&quot; alt=&quot;Propagation at t=0.2sec&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NfiFs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NfiFs.png&quot; alt=&quot;Propagation at t=0.3sec&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GE3tv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GE3tv.png&quot; alt=&quot;Propagation at t=0.4 sec&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28330" LastActivityDate="2018-10-14T17:12:41.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30350" PostTypeId="2" ParentId="30346" CreationDate="2018-10-14T20:37:05.593" Score="4" Body="&lt;p&gt;The beauty of iterative methods is that all they require you to do is matrix-vector multiplications. In your case, the product of your matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; with a vector &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; can be written as &lt;span class=&quot;math-container&quot;&gt;$z=Ay = (B^TCB)y= B^T(C(By))$&lt;/span&gt; which shows that all you need is three matrix-vector products but no matrix-matrix products.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; happens to be symmetric and positive semi-definite, then so is &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and you can use the Conjugate Gradients method. Otherwise you probably want to think about GMRES, another iterative method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="17294" LastEditDate="2018-10-15T14:06:15.223" LastActivityDate="2018-10-15T14:06:15.223" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30352" PostTypeId="1" CreationDate="2018-10-15T13:10:58.700" Score="6" ViewCount="285" Body="&lt;p&gt;Consider Poisson’s equation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$- \Delta u = f{\rm\qquad{     in  }}\;\Omega $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;with following mixed boundary cconditions &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u = g{\rm\qquad{ on }}\;\Gamma  \subset \partial \Omega $$&lt;/span&gt; &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{{\partial u}}{{\partial n}} = h{\qquad\rm{        on  }}\;\partial \Omega \backslash \Gamma$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\frac{{\partial u}}{{\partial n}}$&lt;/span&gt; denotes the derivative of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; in the direction normal to the boundary, &lt;span class=&quot;math-container&quot;&gt;$\partial \Omega$&lt;/span&gt;. When &lt;span class=&quot;math-container&quot;&gt;$g=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h=0$&lt;/span&gt;, the problem becomes homogeneous boundary condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already know the &lt;em&gt;Aubin-Nitsche&lt;/em&gt; trick to get error estimates for &lt;span class=&quot;math-container&quot;&gt;$u - {u_h}$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; norm with homogeneous boundary condition. I learned the duality argument from Brenner and Scott's book &lt;em&gt;&lt;a href=&quot;https://link.springer.com/book/10.1007%2F978-0-387-75934-0&quot; rel=&quot;noreferrer&quot;&gt;The mathematical theory of finite element methods&lt;/a&gt;&lt;/em&gt;. But when the boundary condition is inhomogeneous, it is difficult for me to get error estimates for &lt;span class=&quot;math-container&quot;&gt;$u - {u_h}$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; norm using duality argument. Although the book says, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...inhomogeneous boundary conditions are easily treated' and the proof is analogous&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I encountered a problem arising from the Neumann boundary condition. When the boundary condition is homogeneous, I can get the dual problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a\left( {v,w} \right) = \left( {u - {u_h},v} \right)\qquad\forall v \in H_0^1\left( \Omega  \right)$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the elliptic regularity estimates &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$${\left| w \right|_{{H^2}}} \le C{\left\| {u - {u_h}} \right\|_{{L^2}}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is crucial to the proof. In the case of inhomogeneous boundary condition, however, I get the dual problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a\left( {v,w} \right) = \left( {r - {r_h},v} \right) + {\left( {h,v} \right)_{\partial \Omega \backslash \Gamma }}\qquad\forall v \in H_0^1\left( \Omega  \right)$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the elliptic regularity estimates &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$${\left| w \right|_{{H^2}}} \le C\left( {{{\left\| {r - {r_h}} \right\|}_{{L^2}}} + {{\left\| h \right\|}_{{H^{\frac{1}{2}}}}}} \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which &lt;span class=&quot;math-container&quot;&gt;$r = u - \tilde u$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\tilde u \in {H^1}\left( \Omega\right)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;${\left.{\tilde u} \right|_\Gamma } = g$&lt;/span&gt;. In the following steps of proof, I try to estimate &lt;span class=&quot;math-container&quot;&gt;${\left( {h,v} \right)_{\partial \Omega \backslash \Gamma }}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;${{{\left\| h \right\|}_{{H^{\frac{1}{2}}}}}}$&lt;/span&gt;, but I didn't get the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is my idea about the proof of the inhomogeneous boundary condition, such as the dual problem and the elliptic regularity estimate which I get, correct?&lt;/li&gt;&#xA;&lt;li&gt;If my idea about the proof is correct, how can I continue to prove the error estimates for &lt;span class=&quot;math-container&quot;&gt;$u - {u_h}$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; norm(my idea to estimate &lt;span class=&quot;math-container&quot;&gt;${\left( {h,v} \right)_{\partial \Omega \backslash \Gamma }}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;${{{\left\| h \right\|}_{{H^{\frac{1}{2}}}}}}$&lt;/span&gt;)?&lt;/li&gt;&#xA;&lt;li&gt;If my idea about the proof is wrong, how to obtain the error estimates for &lt;span class=&quot;math-container&quot;&gt;$u - {u_h}$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; norm(the case of inhomogeneous boundary condition)?Please tell me the proof or the main idea of the proof.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28937" LastEditorUserId="20688" LastEditDate="2018-10-15T14:37:55.630" LastActivityDate="2018-10-15T14:37:55.630" Title="$L^2$ norm error estimates of conforming FEM about Poisson’s equation with mixed boundary conditions" Tags="|finite-element|error-estimation|poisson|elliptic-pde|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30353" PostTypeId="1" AcceptedAnswerId="30356" CreationDate="2018-10-16T02:38:03.353" Score="10" ViewCount="9085" Body="&lt;p&gt;I have been working with an algorithm, which uses&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;additions of floating point vectors,&lt;/li&gt;&#xA;&lt;li&gt;(sparse matrix of floats)x(dense vector of floats) dot products&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I recently found out that I can get the same functionality form the algorithm if I set everything (vectors and matrices) to integers. The details are nontrivial and boring, but I now know it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still remember how floating point operations work (mantissa, exponents etc) and to me this seems much more complicated integer operations. I thus expected that modifying the algorithm to use pure integer operations would accelerate things. I am trying to ballpark by how much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are now my questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are integer operations generally faster than float? &lt;/li&gt;&#xA;&lt;li&gt;Has hardware been specialised to floats so much that floats are actually preferable?&lt;/li&gt;&#xA;&lt;li&gt;FLOPS can be used to specifically quantify the how good a piece of hardware can handle floats. What metric should/could we look at to find out the same thing about integers? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28970" LastActivityDate="2018-10-16T12:50:20.867" Title="Integer operations vs floating point operations" Tags="|sparse-matrix|performance|floating-point|complexity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30355" PostTypeId="2" ParentId="30353" CreationDate="2018-10-16T12:49:00.150" Score="5" Body="&lt;p&gt;Integer operations are generally faster than floating point operations, but the gap is far less than it was, say, 30 years ago when everyone was still counting FLOPS. The difference may be a factor of 3 or 5 now between a fp-fp operation and an integer-integer operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in both cases, at least if your problems become large enough that they don't fit into the cache of the processor, is that you are limited by how fast data can be loaded from memory onto the processor. On today's processors, floating point computations are so fast that a processor sits idle for more than 90% of the time if you are doing, for example, a matrix-vector product with data that needs to come from main memory. So the bigger question in your application is how much memory you need to dedicate to store your matrix and vectors. If you are thinking of replacing double precision floating point numbers (64-bit) by regular integers (32-bit) and if you only need to store one integer per matrix or vector entry, then you have just saved 50% of your memory needs and your algorithm will run approximately twice as fast because only half the memory needs to be loaded. But if you were planning on only storing single precision floating point numbers (32-bit) then there is likely not going to be a large difference between the two implementations.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-10-16T12:49:00.150" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30356" PostTypeId="2" ParentId="30353" CreationDate="2018-10-16T12:50:20.867" Score="8" Body="&lt;p&gt;There is &lt;a href=&quot;https://stackoverflow.com/q/2550281/2452869&quot;&gt;a nice discussion on StackOverflow&lt;/a&gt; regarding floating point vs integer operations. In short, the performance of the operations depends a lot on&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;processor architecture&lt;/li&gt;&#xA;&lt;li&gt;how the data is stored in memory and in which order it is accessed&lt;/li&gt;&#xA;&lt;li&gt;if (and which) &lt;a href=&quot;https://en.wikipedia.org/wiki/Streaming_SIMD_Extensions&quot; rel=&quot;noreferrer&quot;&gt;SSE/AVX/AVX2/etc&lt;/a&gt; instructions are used (and how efficiently)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This probably provides some insight into the questing #1 regarding the &lt;strong&gt;generality&lt;/strong&gt; of integer-types being faster than floats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;a href=&quot;https://en.wikipedia.org/wiki/FLOPS&quot; rel=&quot;noreferrer&quot;&gt;FLOPS&lt;/a&gt; are well known for a good reason as it is a standard measure in the performance of the numerical software. In numerical computing, it is generally not easy to go from floating types to integer without&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;changing the nature of mathematical operations (division, trigonometric functions, square roots, special functions, most of the linear algebra, etc)&lt;/li&gt;&#xA;&lt;li&gt;having to implement special (long) integer arithmetic to &quot;emulate&quot; those operations – which will definitely negate all the possible benefits of using integer types instead of floating types.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Of course, there are probably counter-examples, where the usage of integer types is natural (say, incidence matrices for graphs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, regarding the performance metrics. A slightly simplified formula from &lt;a href=&quot;https://en.wikipedia.org/wiki/FLOPS&quot; rel=&quot;noreferrer&quot;&gt;Wiki&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{FLOPS}=\text{cores}\times\frac{\text{cycles}}{\text{second}}\times\frac{\text{FLOPS}}{\text{cycle}}&#xA;$$&lt;/span&gt;&#xA;Also, see the discussion in &lt;a href=&quot;https://scicomp.stackexchange.com/q/11306/20688&quot;&gt;this question on CompSci&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/q/15655835/2452869&quot;&gt;this question on SO&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for judging integer operations, we can have &lt;a href=&quot;https://en.wikipedia.org/wiki/Instructions_per_second&quot; rel=&quot;noreferrer&quot;&gt;IPS&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{IPS}=\text{cores}\times\frac{\text{cycles}}{\text{second}}\times\frac{\text{Instructions}}{\text{cycle}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice the difference:&#xA;For FLOPS, we use FLOPS/cycle, while for  IPS, we use general &lt;a href=&quot;https://en.wikipedia.org/wiki/Instructions_per_cycle&quot; rel=&quot;noreferrer&quot;&gt;instructions per cycle&lt;/a&gt;. Again, it will depend on the processor, if it can actually do more integer-arithmetic operations per cycle (compared to FLOPS). Notice, that there is some averaging done on many levels:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;average # instructions per cycle&lt;/li&gt;&#xA;&lt;li&gt;hidden dependence on the order of instructions&lt;/li&gt;&#xA;&lt;li&gt;... and &lt;strong&gt;mainly on the data access pattern&lt;/strong&gt; - which is the common bottleneck for the modern codes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, in short, I would answer one question #2 that floating-point calculations have been optimized a lot, so the &lt;strong&gt;unnatural&lt;/strong&gt; use of integer types is very unlikely to bring you any dividends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Small extra note:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also look into &lt;a href=&quot;https://en.wikipedia.org/wiki/Half-precision_floating-point_format&quot; rel=&quot;noreferrer&quot;&gt;half-precision arithmetic&lt;/a&gt;. It may be helpful in your tasks provided that the architecture you are using can exploit it efficiently.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-10-16T12:50:20.867" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30357" PostTypeId="1" CreationDate="2018-10-16T15:35:40.203" Score="2" ViewCount="98" Body="&lt;p&gt;I am interested in solving a large number of small linear systems of equations, &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; either &lt;span class=&quot;math-container&quot;&gt;$2\times2$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$3\times3$&lt;/span&gt;. Assuming none of these systems are actually singular, is there anything to deter me from forming &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt; explicitly using Cramer's rule and computing &lt;span class=&quot;math-container&quot;&gt;$x=A^{-1}b$&lt;/span&gt;? The only things that comes to mind is possible loss of precision in the calculation of &lt;span class=&quot;math-container&quot;&gt;$\mathrm{det}A$&lt;/span&gt;. Can I expect any improvement over this naive approach by using a canned solver, e.g., LAPACK's &lt;code&gt;dgesv&lt;/code&gt;? Namely, would doing so reap numerical benefits that justify the overhead of calling out to LAPACK?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I am open to methodologies in between these two extremes as well, e.g., hard-coding Gaussian elimination).&lt;/p&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2018-10-16T17:48:19.100" Title="Inverting small matrices: canned factorization versus explicit formula" Tags="|linear-algebra|lapack|inverse|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30358" PostTypeId="2" ParentId="30153" CreationDate="2018-10-16T17:41:26.637" Score="1" Body="&lt;p&gt;I would assume, that your &lt;span class=&quot;math-container&quot;&gt;$N\times N$&lt;/span&gt; system has the following form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\underbrace{\begin{pmatrix}&#xA;b_1 &amp;amp; c_1 &amp;amp;     &amp;amp;     &amp;amp;     &amp;amp;   0     \\&#xA;a_1 &amp;amp; b_2 &amp;amp; c_2 &amp;amp;     &amp;amp;     &amp;amp;           \\&#xA;s_1 &amp;amp; a_2 &amp;amp; b_3 &amp;amp; c_3 &amp;amp;     &amp;amp;           \\&#xA;    &amp;amp; s_2 &amp;amp; a_3 &amp;amp; b_4 &amp;amp; \ddots &amp;amp;           \\&#xA;    &amp;amp;     &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \ddots &amp;amp; c_{N-1}\\&#xA;0   &amp;amp;     &amp;amp;        &amp;amp; s_{N-2} &amp;amp; a_{N-1} &amp;amp; b_N&#xA;\end{pmatrix}}_{M}&#xA;\begin{pmatrix}&#xA;x_1\\&#xA;x_2\\&#xA;x_3\\&#xA;x_4\\&#xA;\vdots\\&#xA;x_{N}&#xA;\end{pmatrix}=&#xA;\begin{pmatrix}&#xA;d_1\\&#xA;d_2\\&#xA;d_3\\&#xA;d_4\\&#xA;\vdots\\&#xA;d_{N}&#xA;\end{pmatrix}&#xA;\tag{1}&#xA;\label{fourDiagSystem}&#xA;$$&lt;/span&gt;&#xA;So, our matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is a four-diagonal matrix (lower bandwidth = 2, upper bandwidth = 1), where &lt;span class=&quot;math-container&quot;&gt;$\{s_1,\ldots,s_{N-2} \}$&lt;/span&gt; is the &quot;additional&quot; (to the classic triadiagonal case) lower diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if all &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; are zeroes, this matrix can be solved using the classical tridiagonal algorithm (Thomas).&#xA;There are several ways to look at this problem:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Explicitly derive the expressions for this four-diagonal case, similar to the derivation of the Thomas algorithm.&lt;/li&gt;&#xA;&lt;li&gt;The derivation in the previous &quot;approach&quot; can go along Crout's method. For example, see &lt;a href=&quot;http://www.aip.de/groups/soe/local/numres/bookfpdf/f2-3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Recepies&lt;/a&gt; (Sections 2.3, 2.4).&lt;/li&gt;&#xA;&lt;li&gt;Use block-tridiagonal algorithm (Thomas algorithm for block matrices) with block size of &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since the question asks directly about using Thomas algorithm, I would go only into the &lt;strong&gt;third point&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume (without loss of generality) &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; being even. Then, &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\eqref{fourDiagSystem}$&lt;/span&gt; can be rewritten, as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;M=\begin{pmatrix}&#xA;\underbrace{\begin{pmatrix} b_1 &amp;amp; c_1 \\ a_1 &amp;amp; b_2 \end{pmatrix}}_{B_{1}} &amp;amp;&#xA;\underbrace{\begin{pmatrix} 0 &amp;amp; 0 \\ c_2 &amp;amp; 0\end{pmatrix}}_{C_{1}} &amp;amp; &#xA;&amp;amp;&#xA;\\&#xA;\underbrace{\begin{pmatrix} s_1 &amp;amp; a_2 \\ 0 &amp;amp; s_2 \end{pmatrix}}_{A_{1}} &amp;amp;&#xA;\underbrace{\begin{pmatrix} b_3 &amp;amp; c_3 \\ a_3 &amp;amp; b_4 \end{pmatrix}}_{B_{2}} &amp;amp;&#xA;\underbrace{\begin{pmatrix} 0 &amp;amp; 0 \\ c_4 &amp;amp; 0\end{pmatrix}}_{C_{2}} &amp;amp;&#xA;\\&#xA;&amp;amp;&#xA;\underbrace{\begin{pmatrix} s_3 &amp;amp; a_4 \\ 0 &amp;amp; s_4 \end{pmatrix}}_{A_{2}} &amp;amp;&#xA;\underbrace{\begin{pmatrix} b_5 &amp;amp; c_5 \\ a_5 &amp;amp; b_6 \end{pmatrix}}_{B_{2}} &amp;amp;&#xA;\ddots&#xA;\\&#xA;&amp;amp;&#xA;\ddots &amp;amp;&#xA;\ddots &amp;amp;&#xA;\underbrace{\begin{pmatrix}&#xA;0 &amp;amp; 0\\&#xA;c_{N-2} &amp;amp; 0&#xA;\end{pmatrix}}_{C_{N/2-1}}&#xA;\\&#xA;&amp;amp;&#xA;&amp;amp;&#xA;\underbrace{\begin{pmatrix}&#xA;s_{N-3} &amp;amp; a_{N-2}\\&#xA;0 &amp;amp; s_{N-2}&#xA;\end{pmatrix}}_{A_{N/2-1}}&#xA;&amp;amp;&#xA;\underbrace{\begin{pmatrix}&#xA;b_{N-1} &amp;amp; c_{N-1}\\&#xA;a_{N-1} &amp;amp; b_{N}&#xA;\end{pmatrix}}_{B_{N/2}}&#xA;\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or, removing the clarifying annotations, &lt;span class=&quot;math-container&quot;&gt;$\eqref{fourDiagSystem}$&lt;/span&gt; becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\underbrace{\begin{pmatrix}&#xA;B_1 &amp;amp; C_1 &amp;amp; &amp;amp; &amp;amp;0\\&#xA;A_1 &amp;amp; B_2 &amp;amp; C_2 &amp;amp;\\&#xA;    &amp;amp; A_2 &amp;amp; B_3 &amp;amp; \ddots &amp;amp;\\&#xA;   &amp;amp;     &amp;amp; \ddots &amp;amp; \ddots &amp;amp; C_{N/2-1}\\&#xA;0 &amp;amp; &amp;amp; &amp;amp; A_{N/2-1} &amp;amp; B_{N/2}&#xA;\end{pmatrix}}_{M_\text{block}}&#xA;\begin{pmatrix}&#xA;Y_1\\&#xA;Y_2\\&#xA;Y_3\\&#xA;\vdots\\&#xA;Y_{N/2}&#xA;\end{pmatrix}=&#xA;\begin{pmatrix}&#xA;D_1\\&#xA;D_2\\&#xA;D_3\\&#xA;\vdots\\&#xA;D_{N/2}&#xA;\end{pmatrix}&#xA;\tag{2}&#xA;\label{triBlockDiagSystem}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in &lt;span class=&quot;math-container&quot;&gt;$\eqref{triBlockDiagSystem}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$M_\text{block}$&lt;/span&gt; is simply a &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt; block representation of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\eqref{fourDiagSystem}$&lt;/span&gt;.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;B_i=\begin{pmatrix}&#xA;b_{2i-1} &amp;amp; c_{2i-1}\\&#xA;a_{2i-1} &amp;amp; b_{2i}&#xA;\end{pmatrix},\quad &#xA;A_j=\begin{pmatrix}&#xA;s_{2j-1} &amp;amp; a_{2j}\\&#xA;0 &amp;amp; s_{2j}&#xA;\end{pmatrix},\quad&#xA;C_j=\begin{pmatrix}&#xA;0 &amp;amp; 0\\&#xA;c_{2j} &amp;amp; 0&#xA;\end{pmatrix},\\&#xA;i=1,\ldots,\frac{N}{2},\quad j=1,\ldots,\frac{N}{2}-1&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you can apply the block-Thomas algorithm to the &lt;span class=&quot;math-container&quot;&gt;$\eqref{triBlockDiagSystem}$&lt;/span&gt;, in which you can also &lt;strong&gt;take advantage of the special structure&lt;/strong&gt; of your &lt;span class=&quot;math-container&quot;&gt;$C_j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A_j$&lt;/span&gt; blocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the details on block-Thomas, you can refer to &lt;a href=&quot;http://www4.ncsu.edu/eos/users/w/white/www/white/ma580/chap2.5.PDF&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chapter 2.5&lt;/a&gt; of &lt;a href=&quot;http://www4.ncsu.edu/eos/users/w/white/www/white/ma580/ma580hp.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;MA/CSC 580:  Numerical Analysis I course by R. E. White from North Carolina&lt;/a&gt;, but they are effictively standard non-blocked Thomas where operations with scalars are changed to their counterparts with matrices (&lt;em&gt;scalar-scalar&lt;/em&gt; product and &lt;em&gt;scalar division&lt;/em&gt; become &lt;em&gt;matrix-matrix&lt;/em&gt; product and &lt;em&gt;inverse&lt;/em&gt;, or, more stable &lt;em&gt;small&lt;/em&gt;, &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt; in your case, &lt;em&gt;system solve&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, see the discussion in &lt;a href=&quot;https://scicomp.stackexchange.com/q/6701/20688&quot;&gt;this related CompSci question on block-tridiagonal algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-10-16T18:14:42.267" LastActivityDate="2018-10-16T18:14:42.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30359" PostTypeId="2" ParentId="30357" CreationDate="2018-10-16T17:48:19.100" Score="2" Body="&lt;p&gt;Cramer's rule using a more stable determinant:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://hal.archives-ouvertes.fr/hal-01500199/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.archives-ouvertes.fr/hal-01500199/document&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Implementation of the algorithm is consistently 2.5 times slower than LU factorization of matlab regardless of size of matrix.&lt;/p&gt;&#xA;" OwnerUserId="28896" LastActivityDate="2018-10-16T17:48:19.100" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30360" PostTypeId="1" AcceptedAnswerId="30364" CreationDate="2018-10-16T19:31:59.737" Score="6" ViewCount="2430" Body="&lt;p&gt;I'm looking for resources on fast, numerically stable pairwise euclidean distance algorithms. In particular, suppose &lt;span class=&quot;math-container&quot;&gt;$A \in \mathbb{R}^{M \times D}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B \in \mathbb{R}^{N \times D}$&lt;/span&gt; are two sets of row vectors. I would like to compute the matrix,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$X \in \mathbb{R}^{M \times N}, \quad X_{i,j} = \|A_i - B_j\|_2^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have found two methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method 1&lt;/strong&gt; - The simple approach is to loop over each vector &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; and each vector &lt;span class=&quot;math-container&quot;&gt;$B_j$&lt;/span&gt; and populate &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; with the corresponding sum of squares.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Method 2&lt;/strong&gt; - Take advantage of the fact that,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\|A_i - B_j\|_2^2 = \langle A_i - B_j, A_i - B_j \rangle = \|A_i\|_2^2 + \|B_j\|_2^2 - 2 \langle A_i, B_j \rangle.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taking advantage of efficient code for computing each of these three terms, Method 2 is almost an order of magnitude faster. However, it is less numerically stable than Method 1. For example, Method 2 can output negative distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Are there alternate approaches to computing pairwise distances which are faster than Method 1 but &lt;strike&gt;guarantee (at the very least) all non-negative distances?&lt;/strike&gt; have better numerical stability?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Code Demonstration&lt;/strong&gt; - Below is an example comparison written in Python.&#xA;Scipy's &lt;code&gt;cdist()&lt;/code&gt; function is, effectively, an implementation of Method 1 whereas &lt;code&gt;cdist_fast()&lt;/code&gt; below is an implementation of Method 2:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# experiment.py&#xA;import numpy as np&#xA;import time&#xA;from scipy.spatial.distance import cdist&#xA;&#xA;&#xA;def cdist_fast(XA, XB):&#xA;    XA_norm = np.sum(XA**2, axis=1)&#xA;    XB_norm = np.sum(XB**2, axis=1)&#xA;    XA_XB_T = np.dot(XA, XB.T)&#xA;    distances = XA_norm.reshape(-1,1) + XB_norm - 2*XA_XB_T&#xA;    return distances&#xA;&#xA;&#xA;def main():&#xA;    M,N = 5000, 128&#xA;    XA = np.random.randn(M,N)&#xA;&#xA;    t = time.time()&#xA;    distances_cdist = cdist(XA, XA, metric='sqeuclidean')&#xA;    time_cdist = time.time() - t&#xA;&#xA;    t = time.time()&#xA;    distances_cdist_fast = cdist_fast(XA, XA)&#xA;    time_cdist_fast = time.time() - t&#xA;&#xA;    print(f'time_cdist = {time_cdist:.3f} s')&#xA;    print(f'time_cdist_fast = {time_cdist_fast:.3f} s')&#xA;&#xA;    # check validity of results&#xA;    assert np.allclose(distances_cdist, distances_cdist_fast)&#xA;&#xA;    # check that the results are non-negative&#xA;    try:&#xA;        assert (distances_cdist &amp;gt;= 0.0).all()&#xA;    except AssertionError:&#xA;        print('Numerical instability in cdist()')&#xA;&#xA;    try:&#xA;        assert (distances_cdist_fast &amp;gt;= 0.0).all()&#xA;    except AssertionError:&#xA;        print('Numerical instability in cdist_fast()')&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Script output on a 3.1 GHz Intel Core i7:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ python experiment.py&#xA;time_cdist = 3.457 s&#xA;time_cdist_fast = 0.625 s&#xA;Numerical instability in cdist_fast()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6783" LastEditorUserId="6783" LastEditDate="2018-10-17T19:48:48.723" LastActivityDate="2018-10-17T19:48:48.723" Title="Fast and Numerically Stable Pairwise Distance Algorithms" Tags="|linear-algebra|performance|stability|floating-point|distance-measure|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30361" PostTypeId="2" ParentId="28764" CreationDate="2018-10-16T21:22:05.230" Score="2" Body="&lt;p&gt;We decided to go with C++, because it&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is free&lt;/li&gt;&#xA;&lt;li&gt;is fast&lt;/li&gt;&#xA;&lt;li&gt;has a freely available and widely supported numerical integration library (boost/odeint)&lt;/li&gt;&#xA;&lt;li&gt;is easy to integrate with our larger simulation driver/GUI&lt;/li&gt;&#xA;&lt;li&gt;is widely known&lt;/li&gt;&#xA;&lt;li&gt;makes standalone simulation easily available to biological scientists (because they can use RStudio as an interactive simulation environment via Rcpp)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Apart from C++, Python seems to be the other option that might meet these criteria. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;C# was rejected because there is no suitable numerical integration library and no interactive simulation environment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am translating the CSL code to C++ using R scripts to parse the CSL code. This is proving surprisingly effective. Running the model via Rcpp has also proved straightforward. I have made the project available on Github at&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/woodwards/csl2cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/woodwards/csl2cpp&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26265" LastActivityDate="2018-10-16T21:22:05.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30362" PostTypeId="2" ParentId="5425" CreationDate="2018-10-16T22:12:20.983" Score="0" Body="&lt;p&gt;I learned a lot from the answers above. I want to include this answer because I believe it offers different insights to the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us consider the equation&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray} &#xA;   u_{xx} + \frac{1}{c} u_{tt} = 0.&#xA;\end{eqnarray}&lt;/span&gt;&#xA;with constant wavespeed &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without initial and boundary conditions this equation has a solution of the form &#xA;&lt;span class=&quot;math-container&quot;&gt;$u(x,t)=f(x-ct)$&lt;/span&gt;. (a pulse moving to the right)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we impose an initial condition &lt;span class=&quot;math-container&quot;&gt;$u(x,t_0)=p(x)$&lt;/span&gt;, then the solution of&#xA;the equation in the interval &lt;span class=&quot;math-container&quot;&gt;$x \in (-\infty, \infty)$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$p[x-c(t-t_0)]$&lt;/span&gt;. There is no boundary and that is the solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, let us assume that we define a bounded domain &lt;span class=&quot;math-container&quot;&gt;$x \in [a,b]$&lt;/span&gt;, after&#xA;all computers have limited memory.  Then we need to specify the &#xA;values at &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, otherwise computationally we are stuck . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to define the boundary conditions is to use Dirichlet on the&#xA;left boundary and a condition that is consistent with the solution being&#xA;propagated. That is, we can define (assume initial time &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt;)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;   u(a,t_0)=0 \quad , \quad u(b, t_0) = p[b-c(t-t_0)].&#xA;\end{eqnarray}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This produces a pulse running to the right until it disappears on the right edge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/open?id=1g59d-sOdlmlK22lKrvO_qm5ehRCHYj89&quot; rel=&quot;nofollow noreferrer&quot;&gt;push here for animation on Dirichlet on   left  boundary&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still get some noise which I cannot understand (anybody could help here please?) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other option is to impose periodic boundary conditions. That is&#xA;instead of imposing the Dirichlet boundary condition on the left&#xA;we can impose the wave packet which is consistent with the boundary&#xA;on the left. That is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray}&#xA;   u(a,t_0)=p[a-c(t-t_0) ] \quad , \quad u(b, t_0) = p[b-c(t-t_0)].&#xA;\end{eqnarray}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However &lt;span class=&quot;math-container&quot;&gt;$a-c(t-t_0) &amp;lt;a$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$t&amp;gt;t_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c&amp;gt;0$&lt;/span&gt;, and since we need to put&#xA;data inside the interval &lt;span class=&quot;math-container&quot;&gt;$[a,b]$&lt;/span&gt; we add one &quot;period&quot; of length &lt;span class=&quot;math-container&quot;&gt;$b-a$&lt;/span&gt;&#xA;and then we find &lt;span class=&quot;math-container&quot;&gt;$a-c(t-t_0)+b-a=a+b(t-t_0)$&lt;/span&gt;, so that the condition&#xA;on the left would be &lt;span class=&quot;math-container&quot;&gt;$u(a,t_0)=p[b-c(t-t_0]$&lt;/span&gt; (same!) and we would have&#xA;a pulse exiting on the right and entering on the left. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &#xA;&lt;a href=&quot;https://drive.google.com/open?id=1jbqiagSYLY09uAGEYS5hiNo3rjDiIbat&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;  shows what I would call periodic boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made the animations in python and the scheme is a Crank-Nicholson scheme as indicated in the question here.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am still struggling with the noise pattern after the wave hits the first (right) boundary. &lt;/p&gt;&#xA;" OwnerUserId="28330" LastEditorUserId="28330" LastEditDate="2018-10-16T22:47:47.413" LastActivityDate="2018-10-16T22:47:47.413" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30363" PostTypeId="2" ParentId="30360" CreationDate="2018-10-17T04:37:00.307" Score="0" Body="&lt;p&gt;The computed values are negative only when &lt;span class=&quot;math-container&quot;&gt;$A_i = B_j + O(\varepsilon)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt; is the machine precision. The negative values are round-off errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you return the absolute value of your computation, &#xA;&lt;span class=&quot;math-container&quot;&gt;$\left| ||A_i||_2^2 + ||B_j||_2^2 - 2 &amp;lt;A_i,B_j&amp;gt; \right|$&lt;/span&gt;, you get the speed-up and positive numbers.&lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2018-10-17T04:37:00.307" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30364" PostTypeId="2" ParentId="30360" CreationDate="2018-10-17T05:06:06.977" Score="2" Body="&lt;p&gt;I would strongly advise against using the method 2:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\|A_i - B_j\|_2^2 = \langle A_i - B_j, A_i - B_j \rangle = \|A_i\|_2^2 + \|B_j\|_2^2 - 2 \langle A_i, B_j \rangle.$$&lt;/span&gt;&#xA;whether you are uing the absolute value on top of it or not. Because you are going to get your positive (but wrong) distances; however, you are still lacking any numerical stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for 5000 vectors in &lt;span class=&quot;math-container&quot;&gt;$\mathbb R^{128}$&lt;/span&gt;, your (scipy) stable version &lt;strong&gt;is only ~6 times slower&lt;/strong&gt; than the fast one. I don't think you are able to find a numerically stable version that is faster than provided by &lt;code&gt;scipy.cdist&lt;/code&gt;. I would make sure that it is actually a bottleneck in your code. Especially since method 1 &lt;strong&gt;naturally sums only positive entries&lt;/strong&gt;, an ideal and cheap scenario to not deal with the stability!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You definitely can try using &lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kahan&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Kahan_summation_algorithm#Further_enhancements&quot; rel=&quot;nofollow noreferrer&quot;&gt;Neumaier&lt;/a&gt; summation, but I would be surprised that if they are employed (note, Neumaier has to be applied for each entry individually), the aforementioned method 2 would be any faster than &lt;code&gt;scipy.cdist&lt;/code&gt;, most likely, at least tens time slower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if the positiveness of the distance is the only thing you care, you certainly can use an absolute value on top of your &lt;code&gt;cdist_fast(.,.)&lt;/code&gt; function. &lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-10-17T05:06:06.977" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30368" PostTypeId="1" CreationDate="2018-10-17T19:50:29.513" Score="1" ViewCount="93" Body="&lt;p&gt;Consider the integer lattice in &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt;, namely the set &lt;span class=&quot;math-container&quot;&gt;$\mathbb{Z}^2 = \{(x,y): x,y\in \mathbb{Z}\}$&lt;/span&gt;, and let &lt;span class=&quot;math-container&quot;&gt;$u:\mathbb{Z}^2 \to \mathbb{R} $&lt;/span&gt; be a function defined on some bounded subset of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{Z}^2$&lt;/span&gt;. The discrete (graph) Laplace of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, denote it by &lt;span class=&quot;math-container&quot;&gt;$\Delta^1$&lt;/span&gt;, at point &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; is defined as &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tag{1} \Delta^1 u(x,y) = u(x + 1, y) + u(x-1,y) + u(x, y+1) + u(x, y-1)  - 4u(x,y),&#xA;$$&lt;/span&gt;&#xA;i.e. we sum over all lattice neighbors of &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt;  and subtract the value of the function at the given point multiplied by &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; (which is the &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt; since dimension &lt;span class=&quot;math-container&quot;&gt;$d=2$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute the discrete Laplace numerically, I represent the lattice (part of it, of course) as a &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt;-array, and thus the values of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; are being stored in a &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt; matrix replicating the grid. &#xA;Given the nature of &lt;span class=&quot;math-container&quot;&gt;$\Delta^1$&lt;/span&gt;, when I need to compute it at some index &lt;span class=&quot;math-container&quot;&gt;$(i,j)$&lt;/span&gt; of my array, then I need to jump over a row of the array to access the values of neighbors of &lt;span class=&quot;math-container&quot;&gt;$(i,j)$&lt;/span&gt;. This is obviously not an efficient way of accessing the array, as we force the pointer to repeatedly jump over large chunks of memory locations. Indeed, for certain problems when computing the Laplace of a given function excessively (in some iterative process, for example), even if the size of the matrix is moderately small (say &lt;span class=&quot;math-container&quot;&gt;$512\times512$&lt;/span&gt; ) the computation time is enormous compared to the one with the same number of operations but without &lt;strong&gt;array access&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The question:&lt;/strong&gt; is there an efficient way (a data structure) to represent a &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt; grid to optimize for array access operation of the form described above? Say, specifically for access pattern as in &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of spacial and temporal optimization of arrays, as well as that repeatedly accessing the same memory locations the compiler might promote those pointers into registry. However, the access pattern in my case is not very regular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question is rather naive of course, but as a &lt;span class=&quot;math-container&quot;&gt;$2d$&lt;/span&gt; grid is a ubiquitous object, I'd much appreciate if you can share your insights on efficient data structures concerning it.&lt;/p&gt;&#xA;" OwnerUserId="28979" LastActivityDate="2018-10-18T02:03:53.250" Title="Access optimized data structure for representing integer lattice" Tags="|matrix|grid|memory-management|laplacian|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30369" PostTypeId="2" ParentId="30368" CreationDate="2018-10-17T20:17:51.440" Score="3" Body="&lt;p&gt;In essence, you are asking whether you can enumerate the integer lattice sites within your domain from &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; in such a way that accessing the east/west/north/south neighbors of a location &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; requires accessing positions &lt;span class=&quot;math-container&quot;&gt;$n_e,n_w,n_n,n_s$&lt;/span&gt; so that the distance between index &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and these four indices is minimal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem is equivalent to shuffling the rows and columns of a matrix so that its &lt;em&gt;bandwidth&lt;/em&gt; becomes minimal. Algorithms such as Cuthill-McKee are used to compute permutations of rows and columns to achieve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it can be shown that if your domain is, say, a square part of the integer lattice with maximal distance &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; between nodes in both &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; direction (i.e., you have a total of &lt;span class=&quot;math-container&quot;&gt;$N=(d+1)^2$&lt;/span&gt; lattice sites), then there is no ordering that reduces the bandwidth (i.e., &lt;span class=&quot;math-container&quot;&gt;$\max\{|n-n_E|,|n-n_W|,|n-n_N|,|n-n_S|\}$&lt;/span&gt;) to a value below &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;. In other words, you cannot avoid accessing elements that are stored far away in memory.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-10-17T20:17:51.440" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30372" PostTypeId="2" ParentId="30368" CreationDate="2018-10-18T02:03:53.250" Score="2" Body="&lt;p&gt;You can probably speed things up a little bit by storing the array in 4x4 square subarrays so that each of them fit in cache line (64 bytes = 4x4 32-bit integers). This changes the probability distribution of number of memory accesses from [0,0,1] (mean number of accesses = 3) (for 1,2,or 3 accesses) to [1/4,1/2,1/4] (mean number of accesses = 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe some space-filling curve can also help a bit &lt;a href=&quot;https://en.wikipedia.org/wiki/Hilbert_curve&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Hilbert_curve&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8207" LastActivityDate="2018-10-18T02:03:53.250" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30373" PostTypeId="2" ParentId="22074" CreationDate="2018-10-18T12:35:27.923" Score="5" Body="&lt;p&gt;Let me supply a belated answer from the POV of a cluster user and a cluster administrator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A well-designed cluster will, in general, be as homogeneous as possible, with login-nodes being generally a bit beefier due to being the single point of contact of users to the cluster. A well maintained cluster will also have a good documentation of the system, for users who want to get their hands really dirty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over time I have acquired a few rules of thumb for developing and using parallel software which I will share here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the goal is simply a running program try to compile it on the login node. If possible (and if you are using Make) use parallel compilation.&lt;/li&gt;&#xA;&lt;li&gt;Same as 1, but the compilation is progressing slowly. The impact on other users may not be negligible, especially if you use parallel make (&lt;code&gt;make -j &amp;lt;n&amp;gt;&lt;/code&gt;). In this case you should try to estimate (lavishly) the time required to compile the code and submit a job to the queue.&lt;/li&gt;&#xA;&lt;li&gt;If the goal is to have an optimized code but &lt;strong&gt;not&lt;/strong&gt; rely on architectural optimisations then you should submit the compilation job in exclusive mode, i.e. requesting an entire node for the compilation. In this way other users will not interfere with the optimisation process. An example of a code that i would compile this way would be NumPy, with its profiler guided optimisations and link-time optimisations. Simple &lt;code&gt;-O3&lt;/code&gt; optimisations can be used with the first two versions, too.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the goal is to have an optimised code &lt;strong&gt;and&lt;/strong&gt; use architectural optimisations (AVX, SSE etc.) then it is safer to compile on the nodes, since that is where the job will run. &lt;em&gt;Caveat&lt;/em&gt;: If the cluster you are running on consists of multiple CPU architectures (e.g. Intel Broadwell, Skylake), compare for instance &lt;a href=&quot;http://www.bwhpc-c5.de/wiki/index.php/BwUniCluster_Hardware_and_Architecture&quot; rel=&quot;noreferrer&quot;&gt;http://www.bwhpc-c5.de/wiki/index.php/BwUniCluster_Hardware_and_Architecture&lt;/a&gt; then you have two options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use only options common to both architectures. This generally entails optimising for the older one.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Compile on nodes with a specific architecture and optimise for that architecture. Generally job schedulers such as SLURM allow selection of resources via constraints. Ask your cluster-admin though. Afterwards, when you run the code make sure to limit the set of nodes it can be run on only to those with the fitting architecture, else the program will likely crash.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These are rough guidelines. Myself I prefer not to submit a compilation script to the queue if I am not 100% sure that the compilation will succeed. In such cases I submit a request for an interactive session, which is essentially like &lt;code&gt;ssh&lt;/code&gt;-ing to a node, only the batch system will kick you out once the requested time is up.&lt;/p&gt;&#xA;" OwnerUserId="28993" LastActivityDate="2018-10-18T12:35:27.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30374" PostTypeId="1" CreationDate="2018-10-18T22:07:40.047" Score="3" ViewCount="124" Body="&lt;p&gt;I'm looking for a comparison between the Jacobian-Free Newton-Krylov (JFNK) method performance compared to the conventional CFD nonlinear solution methodologies like SIMPLE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if such a document exists? Also, if anyone has any intuition on this without any direct evidence, I would like to hear about it.&lt;/p&gt;&#xA;" OwnerUserId="29000" LastEditorUserId="20688" LastEditDate="2018-10-19T03:07:07.600" LastActivityDate="2022-06-09T06:19:46.990" Title="Has there been a comparison bewteen SIMPLE/SIMPLER and JFNK for steady CFD?" Tags="|fluid-dynamics|nonlinear-equations|reference-request|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30375" PostTypeId="1" CreationDate="2018-10-19T09:20:15.970" Score="2" ViewCount="34" Body="&lt;p&gt;I have a sparse square matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; with size &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; and number of nonzero entries &lt;span class=&quot;math-container&quot;&gt;$nnz$&lt;/span&gt;.&#xA;The goal is making a sub-matrix &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; nonzeros which are randomly chosen from &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Duplicates are ok, so we need &lt;em&gt;at most&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; nonzeros in &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The probability of entry &lt;span class=&quot;math-container&quot;&gt;$A_{kl}$&lt;/span&gt; being chosen is &lt;span class=&quot;math-container&quot;&gt;$p_{ij} = \frac{A_{ij}^2}{\sum_{k, l} A_{kl}^2}$&lt;/span&gt;, in which, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; go through all the matrix entries. I have this algorithm in mind so far:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;generate a random number &lt;span class=&quot;math-container&quot;&gt;$rand$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$0\le rand \le 1$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;if &lt;span class=&quot;math-container&quot;&gt;$(rand &amp;lt; p_{11}) \implies$&lt;/span&gt; add &lt;span class=&quot;math-container&quot;&gt;$A_{11}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;go to the next entry of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and repeat from step &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; until &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; entries are chosen. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If I go through all entries of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; &lt;strong&gt;once&lt;/strong&gt;, on average how many entries will be chosen? In other words, how many times I should go through entries of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to have &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; entries chosen? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to change the probability &lt;span class=&quot;math-container&quot;&gt;$p_{ij}$&lt;/span&gt; so by going through entries of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; once, all &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; entries are chosen.&lt;/p&gt;&#xA;" OwnerUserId="18634" LastEditorUserId="20688" LastEditDate="2019-06-22T22:38:12.407" LastActivityDate="2019-06-22T22:38:12.407" Title="Randomized Submatrix of a Sparse Matrix" Tags="|linear-algebra|matrix|random-sampling|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30376" PostTypeId="1" CreationDate="2018-10-19T11:17:16.053" Score="2" ViewCount="35" Body="&lt;p&gt;What I have are 3D grayscale images of brains where dark pixels indicate blood vessels (see below, red arrows mark blood vessels). The objective is to obtain some descriptive statistics about them, such as number of vessels in the brain or volume fraction of the brain occupied by vessels.  Currently, some poor student would either manually count all the vessels in the brain, or we would just look at the histogram over a certain volume, to determine the volume fraction of blood vessels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a bit a general question, but what approach would you take to automatically detect these structures? Scikit image seems to have some promising methods for edge detection. The algorithm should work on a 3D matrix since one vessel can span multiple image slices. The grayscale values could also be used to improve the resolution: A voxel that is completely filled by a blood vessel has the lowest signal value, while a voxel that is only partially occupied by a vessel has a slightly higher signal value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s1mhK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s1mhK.jpg&quot; alt=&quot;Example Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29005" LastActivityDate="2018-10-19T11:17:16.053" Title="Detecting structures (blood vessels) in MR images" Tags="|computational-biology|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30377" PostTypeId="1" CreationDate="2018-10-19T11:58:17.460" Score="2" ViewCount="142" Body="&lt;p&gt;I am trying to solve  the 3D Poisson equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla\cdot(\epsilon(\mathbf{r})\nabla) u(\mathbf{r}) = f(\mathbf{r})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I notice intel advertises &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-fast-poisson-solver-routines&quot; rel=&quot;nofollow noreferrer&quot;&gt;routines&lt;/a&gt; that appear to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla^2u(\mathbf{r}) = f(\mathbf{r})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if I can find similar such routines for solving the more general electrostatics Poisson equation above?&lt;/p&gt;&#xA;" OwnerUserId="7808" LastEditorUserId="20688" LastEditDate="2018-10-19T14:51:13.733" LastActivityDate="2018-10-19T14:51:13.733" Title="Solving electrostatics Poisson equation with Intel MKL routines" Tags="|poisson|electromagnetism|intel-mkl|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30378" PostTypeId="1" CreationDate="2018-10-19T19:37:57.267" Score="1" ViewCount="1160" Body="&lt;p&gt;Goal: Optimize convex function &lt;span class=&quot;math-container&quot;&gt;$f(\vec{x})$&lt;/span&gt; subjected to constraint &lt;span class=&quot;math-container&quot;&gt;$A\vec{x} = \vec{b}$&lt;/span&gt; starting at a point &lt;span class=&quot;math-container&quot;&gt;$\vec{x}_0$&lt;/span&gt; that satisfies the constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem only has equality constraint. Why does the solution requires using the KKT condition, which is for inequality constraint?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.stat.cmu.edu/~ryantibs/convexopt-S15/lectures/14-newton.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; lecture note mention using KKT condition and quadratic approximation gives the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Newton's method with line search:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \begin{bmatrix}&#xA;\nabla^{2} f &amp;amp; A^{T} \\&#xA;A &amp;amp; 0 \\&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;u \\ w \\&#xA;\end{bmatrix}  &#xA;= &#xA;\begin{bmatrix}&#xA;- \nabla f \\ 0 \\&#xA;\end{bmatrix}  &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\vec{x}^{(n+1)} = \vec{x}^{(n)} + t\vec{u}$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the step size found by backtracking.&lt;/p&gt;&#xA;" OwnerUserId="28896" LastActivityDate="2018-10-21T16:24:46.047" Title="Why does Newton's method with Linear Equality Constraints use KKT condition?" Tags="|linear-algebra|optimization|constrained-optimization|convex-optimization|nonlinear-programming|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30382" PostTypeId="1" CreationDate="2018-10-20T08:05:27.990" Score="2" ViewCount="109" Body="&lt;p&gt;I am doing a comparison of different algebraic modeling langues (AMPL, AIMMS, GAMS, Pyomo) in both theoretical and practical terms. As a practical experiment I am trying to measure problem model instance creation time. I.e. taking optimization problem defined in one of the above languages I am trying to load the model and it export/convert it to LP solver input format. Model instance creation time is being measured and model instance characteristics are being examined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have only used sample problems provided by the authors of AMLs which are quite small. In order to have a meaningful benchmark I would like to find test data to build much larger problem model instance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe someone could guide me where I could find larger data sets for the models of linear programming problems  defined in the AMLs mentioned above?&lt;/p&gt;&#xA;" OwnerUserId="29012" LastActivityDate="2018-10-20T08:05:27.990" Title="Where can I find sample data for large linear programming optimization problems?" Tags="|optimization|convex-optimization|linear-programming|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30383" PostTypeId="1" AcceptedAnswerId="30385" CreationDate="2018-10-20T11:06:44.940" Score="2" ViewCount="696" Body="&lt;p&gt;I have done an experiment &lt;em&gt;Estimation of silver nitrate by potentiometric titration with standard KCl solution&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A plot of &lt;span class=&quot;math-container&quot;&gt;$\dfrac{\Delta E}{\Delta V}$&lt;/span&gt; versus Volume of KCl solution gives the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vF2Hs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vF2Hs.png&quot; alt=&quot;&lt;span class=&quot;math-container&quot;&gt;$\dfrac{\Delta E}{\Delta V}$&lt;/span&gt; versus Volume of KCl &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the possible equation which I can fit this data to?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I tried fitting it to a degree 5 polynomial, but it doesn't give a good fit:)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Xys07.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xys07.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I have used gnuplot for plotting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Here's the &lt;a href=&quot;https://drive.google.com/file/d/1l3KLcsYqjH_ixdbnKKgxcAKtfh8mHAor/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;data&lt;/a&gt;, if it helps.&lt;/p&gt;&#xA;" OwnerUserId="29014" LastEditorUserId="29014" LastEditDate="2018-10-21T10:12:02.913" LastActivityDate="2018-10-21T16:07:17.927" Title="What equation should I fit this set of data points to?" Tags="|regression|computational-chemistry|curve-fitting|gnuplot|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30385" PostTypeId="2" ParentId="30383" CreationDate="2018-10-20T17:15:24.907" Score="5" Body="&lt;p&gt;You could either fit a &lt;a href=&quot;https://en.wikipedia.org/wiki/Logistic_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;logistic function&lt;/a&gt; (possibly composing it with a linear function), use &lt;a href=&quot;https://en.wikipedia.org/wiki/Segmented_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;segmented regression&lt;/a&gt;, or &lt;a href=&quot;https://en.wikipedia.org/wiki/Decision_tree_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;classification and regression trees&lt;/a&gt;, among other options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original data, shown in the figure below,&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MzUeU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MzUeU.png&quot; alt=&quot;Original data and regression&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;was fitted in Gnuplot using the following commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h(x) = k * 0.5 * (1.0 - tanh(0.5 * (a * x + b))) + c * x + d&#xA;fit h(x) 'plot-EV.txt' via 'start.par'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the file &lt;code&gt;start.par&lt;/code&gt; contains an initial guess (with values in the ballpark of the optimal parameters) for the optimization process. The specific contents of the initial guess file are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = 100&#xA;a = 1&#xA;b = 1&#xA;c = 1&#xA;d = 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The reasoning behind the choice of initial values for &lt;code&gt;k&lt;/code&gt; (and &lt;code&gt;d&lt;/code&gt;) is that they should be of the same order of magnitude as the original function.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2018-10-21T16:07:17.927" LastActivityDate="2018-10-21T16:07:17.927" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30387" PostTypeId="1" AcceptedAnswerId="30389" CreationDate="2018-10-20T20:21:43.740" Score="1" ViewCount="88" Body="&lt;p&gt;Testing on an &lt;strong&gt;MPI scientific code&lt;/strong&gt; for compressible flow dynamics I noticed that the results may depend on the number of processors used for the calculation. In fact, comparing the binary files they differ around &lt;strong&gt;10.0E-10 ~ 10.0E-11&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I am &lt;strong&gt;optimising the application&lt;/strong&gt; and I am almost completely rewriting it from zero, I wonder if it is possible to make the results independent of the number of MPI processes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If yes, how to proceed?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What are the things to avoid and what are the things to do?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="26978" LastActivityDate="2018-10-22T22:40:05.770" Title="Guaranteed equality between binary results with increasing MPI processes" Tags="|fluid-dynamics|parallel-computing|computational-physics|mpi|numerics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30388" PostTypeId="2" ParentId="30383" CreationDate="2018-10-20T20:29:37.427" Score="3" Body="&lt;p&gt;An assortment of curves for fitting chemistry examples is presented in &lt;a href=&quot;https://www.colby.edu/chemistry/PChem/scripts/lsExamples.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;these Colby College class notes&lt;/a&gt;.  Of particular application is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sigmoid_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;sigmoid response curve&lt;/a&gt; with variable &quot;slope&quot; for the central part of the curve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ f(x) = \frac{a}{1 + e^{bx - c} } + d $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[This is similar to the suggested &lt;em&gt;logistic function&lt;/em&gt; proposed in the first Answer, but has four rather than three parameters allowing for the inflection points (changing sign of second derivative) to appear in a location other than at the origin.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such functions are inherently monotone (decreasing when &lt;span class=&quot;math-container&quot;&gt;$b\gt 0$&lt;/span&gt; and increasing when &lt;span class=&quot;math-container&quot;&gt;$b\lt 0$&lt;/span&gt;), so they will &quot;automatically&quot; possess that property if you fit your data to one of these models, avoiding the tendency of polynomial curve fits to &lt;em&gt;oscillations&lt;/em&gt; seen in your graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A disadvantage of this &lt;em&gt;nonlinear&lt;/em&gt; parameterization is that computing it requires an iterative process, rather than by the direct solvers available for a linear least squares fitting as you seem to have used for the fifth degree polynomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But nonlinear least squares fitting is often not difficult when reasonable initial estimates for the parameters are available.  There are online solvers (at a glance &lt;a href=&quot;http://statpages.info/nonlin.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nonlinear Least Squares Regression&lt;/a&gt; suggested in the class note above seems to be intact, although a second site mentioned there returns 404:Not found).  Most spreadsheets have either built-in or add-in solvers that would be capable of doing the fit.  But you may find it edifying to do the fit &quot;manually&quot; so that you understand the roles played by the parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would start with the two extremes where the curve levels off.  That is, for arguments well below the &quot;drop-off&quot; portion of the curve, the data appears to approach &lt;span class=&quot;math-container&quot;&gt;$f(0) = 360$&lt;/span&gt; more or less.  From this one might estimate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ a + d \approx 360 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;while the data well above the &quot;drop-off&quot; suggests &lt;span class=&quot;math-container&quot;&gt;$d \approx 130$&lt;/span&gt;.  So we can get initial estimates for parameters &lt;span class=&quot;math-container&quot;&gt;$a,d$&lt;/span&gt; easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Visually the inflection point (change from concave down to concave up) appears to occur at about &lt;span class=&quot;math-container&quot;&gt;$x=3.5$&lt;/span&gt;.  Solving &lt;span class=&quot;math-container&quot;&gt;$f''(x) = 0$&lt;/span&gt; gives us &lt;span class=&quot;math-container&quot;&gt;$bx = c$&lt;/span&gt;, thus eliminating one more parameter estimate.  Finally the first derivative of &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; at the inflection point can be estimated visually from the slope exhibited by the data (although this seems rather steep, circa &lt;span class=&quot;math-container&quot;&gt;$600$&lt;/span&gt;, so less firmly estimated by sight).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The iterative procedure (nonlinear least squares regression) will seek to adjust the parameters &lt;span class=&quot;math-container&quot;&gt;$a,b,c,d$&lt;/span&gt; to minimize the least squares measure of error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \sum_{i=1}^n (y_i - f(x_i))^2 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where summation is taken over the set of data &lt;span class=&quot;math-container&quot;&gt;$(x_i,y_i)$&lt;/span&gt; from your titration experiments.&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="651" LastEditDate="2018-10-21T15:01:30.883" LastActivityDate="2018-10-21T15:01:30.883" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30389" PostTypeId="2" ParentId="30387" CreationDate="2018-10-21T05:37:11.677" Score="2" Body="&lt;p&gt;That is a very common problem with parallel computations. Technically, if the communication pattern within your MPI-parallelized code is complicated enough, even runs with the same number of processors might end up having slightly different results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the strategy to deal with that will differ a lot depending on what and how is being parallelized. One of the common sources of &quot;results fluctuation&quot;, is the reduction (due to non-associativity of floating point operations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several things you can do about that.&#xA;First, look at &lt;a href=&quot;https://scicomp.stackexchange.com/q/2384/20688&quot;&gt;this CompSci question on MPI reduction reproducibility&lt;/a&gt;. &lt;a href=&quot;https://scicomp.stackexchange.com/a/2386/20688&quot;&gt;Jed's answer&lt;/a&gt; gives a nice overview of the problem regarding reduction and an advice to have stable results for the same and the different number of processors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, do your calculations in parallel, until you need to &quot;merge&quot; them together (reduction). To merge them, gather them on one processor (commonly referred to as the &lt;em&gt;master node&lt;/em&gt;) and perform the required set of operations. When doing the last step locally, you can be in charge of the operation order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the reduction is not the only possible source: as your problem partitioning (mesh partitioning, etc.) might already introduce another source of &quot;fluctuations&quot;. So, I would suggest &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;to reevaluate your criteria on independence on the number of MPI processes&lt;/li&gt;&#xA;&lt;li&gt;if you decide you need it, to have both &lt;em&gt;(more) stable, debug, but slower version&lt;/em&gt;, and &lt;em&gt;standard&lt;/em&gt; one.&lt;/li&gt;&#xA;&lt;li&gt;have some tolerance to which you would allow the results to be different.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-10-21T05:37:11.677" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30391" PostTypeId="1" AcceptedAnswerId="30411" CreationDate="2018-10-21T10:02:46.787" Score="0" ViewCount="157" Body="&lt;p&gt;I'm trying to solve multicomponent Euler equations for perfect gas mixture with Godunov-like scheme using exact Riemann solver. Of course, some approximate solver would probably be more cost-effective, but for now I'd like to make it work with exact solver. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the problematic part of algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have left and right conditions, for which we calculated relevant values, namely, density, velocity, pressure, heat capacity ratio and sound speed: &lt;span class=&quot;math-container&quot;&gt;$\rho_l, u_l, p_l, \gamma_l, a_l$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rho_r, u_r, p_r, \gamma_r, a_r$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we try to find the velocity and pressure &lt;span class=&quot;math-container&quot;&gt;$v, p$&lt;/span&gt; in the middle state that may be reached through single shock wave or rarefaction wave from left and right states at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We construct functions &lt;span class=&quot;math-container&quot;&gt;$\phi_l(p), \phi_r(p)$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\phi_l(p) = \begin{cases}&#xA;                    u_l + \frac{2a_l}{\sqrt{2\gamma_l(\gamma_l-1)}}\frac{\left(1-\frac{p}{p_l}\right)}{\sqrt{1+\beta\frac{p}{p_l}}} &amp;amp; \text{if} \;\; p \ge p_l \;\; \text{(shock wave)}  \\&#xA;                    u_l + \frac{2a_l}{\gamma_l-1}\left( 1-\left(\frac{p}{p_l}\right)^{\alpha} \right) &amp;amp; \text{if} \;\; p &amp;lt; p_l \;\; \text{(rarefaction)}  &#xA;                 \end{cases}&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\phi_r(p) = \begin{cases}&#xA;                    u_r - \frac{2a_r}{\sqrt{2\gamma_r(\gamma_r-1)}}\frac{\left(1-\frac{p}{p_r}\right)}{\sqrt{1+\frac{\gamma_r+1}{\gamma_r-1}\frac{p}{p_r}}} &amp;amp; \text{if} \;\; p \ge p_r \;\; \text{(shock wave)}  \\&#xA;                    u_r - \frac{2a_r}{\gamma_r-1}\left( 1-\left(\frac{p}{p_r}\right)^{\alpha} \right) &amp;amp; \text{if} \;\; p &amp;lt; p_r \;\; \text{(rarefaction)}  &#xA;                 \end{cases}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\alpha = \frac{\gamma-1}{2\gamma}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\beta = \frac{\gamma+1}{\gamma-1}$&lt;/span&gt;;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and their derivatives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\phi'_l(p) = \begin{cases}&#xA;                    \frac{2a_l}{\sqrt{2\gamma_l(\gamma_l-1)}} &#xA;\frac{   &#xA;\frac{-\sqrt{1+\beta\frac{p}{p_l}}}{p_l} - \frac{  \beta \left(1-\frac{p}{p_l}\right)} {2p_l \sqrt{1+\beta\frac{p}{p_l}}}} {1+\beta\frac{p}{p_l}} &amp;amp; \text{if} \;\; p \ge p_l \;\; \text{(shock wave)}  \\&#xA;                    - \alpha \frac{1}{p_l^{\alpha}} \frac{2a_l}{\gamma_l-1}\left(\frac{p}{p_l}\right)^{\alpha - 1} &amp;amp; \text{if} \;\; p &amp;lt; p_l \;\; \text{(rarefaction)}  &#xA;                 \end{cases}&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\phi'_r(p) = \begin{cases}&#xA;                    -\frac{2a_r}{\sqrt{2\gamma_r(\gamma_r-1)}} &#xA;\frac{   &#xA;\frac{-\sqrt{1+\beta\frac{p}{p_r}}}{p_r} - \frac{  \beta \left(1-\frac{p}{p_r}\right)} {2p_r \sqrt{1+\beta\frac{p}{p_r}}}} {1+\beta\frac{p}{p_r}} &amp;amp; \text{if} \;\; p \ge p_r \;\; \text{(shock wave)}  \\&#xA;                     \alpha \frac{1}{p_r^{\alpha}} \frac{2a_r}{\gamma_r-1}\left(\frac{p}{p_r}\right)^{\alpha - 1} &amp;amp; \text{if} \;\; p &amp;lt; p_r \;\; \text{(rarefaction)}  &#xA;                 \end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and try to find &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; so that &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\phi(p) = \phi_l(p) - \phi_r(p) = 0&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that we use Newton's method:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p_{n+1} = p_{n} - \frac{\phi(p_n)}{\phi'(p_n)}&#xA;$$&lt;/span&gt;&#xA;starting with &lt;span class=&quot;math-container&quot;&gt;$p_0 = (p_l+p_r)/2$&lt;/span&gt; until it converges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For single-component Euler equations it almost always converges extremely fast: typical Sod's problem &lt;span class=&quot;math-container&quot;&gt;$(\rho_l,u_l,p_l = 1,0,1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\rho_r,u_r,p_r = 0.125,0,0.1)$&lt;/span&gt; converges in 5-6 steps. But for my multi-component problems typical values are something like &lt;span class=&quot;math-container&quot;&gt;$p \approx 10^5-10^7 Pa$&lt;/span&gt;, while &lt;span class=&quot;math-container&quot;&gt;$u,a \approx 10^2-10^3 m/s$&lt;/span&gt; and also &lt;span class=&quot;math-container&quot;&gt;$\phi(p), \phi'(p) \approx 10^2-10^3$&lt;/span&gt;. Thus the pressure changes very little at every step of the Newton's algorithm (for example only by &lt;span class=&quot;math-container&quot;&gt;$\Delta p \approx 0.3 Pa$&lt;/span&gt; while &lt;span class=&quot;math-container&quot;&gt;$p_0 = 550000 Pa$&lt;/span&gt;) and the convergence is extremely slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases, I was able to achieve convergence within hundred of steps by artificially multiplying &lt;span class=&quot;math-container&quot;&gt;$\Delta p$&lt;/span&gt; by something like &lt;span class=&quot;math-container&quot;&gt;$10^4$&lt;/span&gt; and it gave the correct solution. But I wasn't able to construct stable algorithm for dynamic multiplier for &lt;span class=&quot;math-container&quot;&gt;$\Delta p$&lt;/span&gt; that would give at least reasonable convergence speed in general case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some more general way to speed up the convergence here? Or may some different way to determine middle state pressure?&lt;/p&gt;&#xA;" OwnerUserId="25060" LastEditorUserId="25060" LastEditDate="2018-10-22T06:06:37.183" LastActivityDate="2018-11-01T09:10:35.760" Title="Exact Riemann solver for perfect gas mixture: problem with Newton's method convergence" Tags="|finite-volume|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30392" PostTypeId="2" ParentId="30378" CreationDate="2018-10-21T10:05:33.253" Score="2" Body="&lt;p&gt;You know that you have to use the Lagrange multiplier,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(x,\mu)=f(x)+\mu^T(Ax-b).&#xA;$$&lt;/span&gt;&#xA;The derivatives of that function that have to be zero at a possible minimum are&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;0=\frac{\partial L}{\partial x}=f'(x)+\mu^TA\\&#xA;0=\frac{\partial L}{\partial \mu}=(Ax-b)^T&#xA;$$&lt;/span&gt;&#xA;To compute a Newton step &lt;span class=&quot;math-container&quot;&gt;$(u,w)$&lt;/span&gt; for the pair &lt;span class=&quot;math-container&quot;&gt;$x,\mu$&lt;/span&gt; you have to solve&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f'(x)+f''(x)u+(\mu+w)^TA=0\\&#xA;A(x+u)-b=0&#xA;$$&lt;/span&gt;&#xA;Converting first and second derivative to gradient and Hessean results in&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}&#xA;\nabla^2f(x)&amp;amp;A^T\\&#xA;A&amp;amp;0&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;u\\w&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;-\nabla f-A^T\mu\\-(Ax-b)&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&#xA;By the first assumption one can assume that in every step &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; exactly so that the right side in the second block reduces to zero. I'm not sure why you assume that &lt;span class=&quot;math-container&quot;&gt;$A^T\mu=0$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;It would of course now be useful to parametrize the kernel of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to remove the second block of equations so that only the (potentially) non-linear equations of the first block remain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$U=Cz$&lt;/span&gt; parametrizes the kernel, &lt;span class=&quot;math-container&quot;&gt;$AC=0$&lt;/span&gt;, then the reduced system has to minimize &lt;span class=&quot;math-container&quot;&gt;$f(x_0+Cz)$&lt;/span&gt;, thus &lt;span class=&quot;math-container&quot;&gt;$f'(x_0+Cz)C=0$&lt;/span&gt;, so that the Newton update &lt;span class=&quot;math-container&quot;&gt;$Δz$&lt;/span&gt; is obtained from &lt;span class=&quot;math-container&quot;&gt;$$C^T\nabla^2f(x_0+Cz)C\,\Delta z=-C^T\nabla f(x_0+Cz)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2018-10-21T16:24:46.047" LastActivityDate="2018-10-21T16:24:46.047" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30393" PostTypeId="1" CreationDate="2018-10-21T11:14:29.263" Score="2" ViewCount="127" Body="&lt;p&gt;I have working program for solving one-component 1D Euler equations with Roe's approximate Riemann solver constructed according to &lt;a href=&quot;https://perswww.kuleuven.be/~u0016541/Talks/roe.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this pdf&lt;/a&gt;. My implementation of the algorithm is as follows (&lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is density, &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is velocity, &lt;span class=&quot;math-container&quot;&gt;$h=\frac{e+p}{\rho}$&lt;/span&gt; is total specific enthalpy, &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is speed of sound):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, we compute Roe average values between the states:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\rho = \sqrt{\rho_l\rho_r} \\&#xA;u = \frac{\sqrt{\rho_l}u_l + \sqrt{\rho_r}u_r}{\sqrt{\rho_l}+\sqrt{\rho_r}} \\&#xA;h = \frac{\sqrt{\rho_l}h_l + \sqrt{\rho_r}h_r}{\sqrt{\rho_l}+\sqrt{\rho_r}} \\&#xA;c = \sqrt{(\gamma-1)(h-u^2/2)}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the right eigenvectors of the Jacobian matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left(&#xA;\begin{matrix} &#xA;r_0 &amp;amp; r_1 &amp;amp; r_2 \\&#xA;\end{matrix}&#xA;\right)&#xA;=&#xA;\left(&#xA;\begin{matrix} &#xA;1 &amp;amp; 1 &amp;amp; 1 \\&#xA;u-c &amp;amp; u &amp;amp; u+c \\&#xA;h-uc &amp;amp; u^2/2 &amp;amp; h+uc &#xA;\end{matrix}&#xA;\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Absolute values of wave speeds:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;|\lambda_0| = |u-c|, \; |\lambda_1| = |u|, \; |\lambda_2| = |u+c|&#xA;$$&lt;/span&gt;&#xA;and apply some entropy fix to them (not relevant here).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wave amplitudes (coefficients of the jump between states decomposition by eigenvectors):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\alpha_0 = \frac{(p_r-p_l) - \rho c (u_r-u_l)}{2c^2} \\&#xA;\alpha_1 = (\rho_r - \rho_l) - \frac{(p_r-p_l)}{c^2} \\&#xA;\alpha_2 = \frac{(p_r-p_l) + \rho c (u_r-u_l)}{2c^2} \\&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Flux values of the states:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;F_l = \left(\begin{array}{c} \rho_l u_l \\ p_l+\rho_l u_l^2 \\ (e_l+p_l)u_l \end{array}\right), \quad F_r = \left(\begin{array}{c} \rho_r u_r \\ p_r+\rho_r u_r^2 \\ (e_r+p_r)u_r \end{array}\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And finally flux values of the middle state:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;F = \frac{1}{2}(F_l+F_r) - \frac{1}{2} \sum\limits_{p=0}^2 |\lambda_p| \alpha_p r_p&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is &lt;strong&gt;how to generalize this algorithm for equilibrium multi-component perfect gas mixture&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm already able to calculate the eigenvectors for this case, here is an example for 3 components, corresponding to &lt;span class=&quot;math-container&quot;&gt;$(u,u,u,u+a,u-a)$&lt;/span&gt; eigenvalues:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left(&#xA;\begin{matrix} &#xA;r_0 &amp;amp; r_1 &amp;amp; r_2 &amp;amp; r_3 &amp;amp; r_4 \\&#xA;\end{matrix}&#xA;\right)&#xA;=&#xA;\left(&#xA;\begin{matrix} &#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \frac{\rho_0}{\rho} &amp;amp; \frac{\rho_0}{\rho} \\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; \frac{\rho_1}{\rho} &amp;amp; \frac{\rho_1}{\rho} \\&#xA;0 &amp;amp; 0 &amp;amp; 1 &amp;amp; \frac{\rho_2}{\rho} &amp;amp; \frac{\rho_2}{\rho} \\&#xA;u &amp;amp; u &amp;amp; u &amp;amp; u+c &amp;amp; u-c \\&#xA;u^2-\frac{1}{\gamma-1}\frac{\partial p}{\partial \rho_0} &amp;amp; u^2-\frac{1}{\gamma-1}\frac{\partial p}{\partial \rho_1} &amp;amp; u^2-\frac{1}{\gamma-1}\frac{\partial p}{\partial \rho_2} &amp;amp; h+uc &amp;amp; h-uc &#xA;\end{matrix}&#xA;\right)&#xA;$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial p}{\partial \rho_i} = \frac{R_0 T}{\mu_i} - (\gamma-1)(h_{0i}+c_{vi}T - u^2/2)&#xA;$$&lt;/span&gt;&#xA;and &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is temperature, &lt;span class=&quot;math-container&quot;&gt;$R_0$&lt;/span&gt; is universal gas constant, &lt;span class=&quot;math-container&quot;&gt;$\mu_i, h_{0i}, c_{vi}$&lt;/span&gt; are molar mass, heat of formation and heat capacity of &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th gas component. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These formulas are derived from &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/002199919090245V&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; by discarding of vibrational terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How to correctly apply Roe averaging to gas mixture? I.e. how to calculate &lt;span class=&quot;math-container&quot;&gt;$\rho_i, u, \gamma, T, h, c$&lt;/span&gt; for the matrix above?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How to calculate &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; for this case?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="25060" LastEditorUserId="25060" LastEditDate="2018-10-24T05:58:35.303" LastActivityDate="2018-10-24T05:58:35.303" Title="Roe Riemann solver for perfect gas mixture" Tags="|finite-volume|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30394" PostTypeId="1" AcceptedAnswerId="30400" CreationDate="2018-10-21T15:08:31.893" Score="4" ViewCount="354" Body="&lt;p&gt;I may be incorrect, but it seems like commercial graphics codes typically use smoothed particle hydrodynamics (SPH) to produce stunning simulations and not continuum based methods. Why is this? Is SPH much less costly than continuum methods?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is that case the SPH is much less expensive than direct numerical simulation (DNS), why do we use DNS at all?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have only ever worked with finite element method, finite volume method, and finite difference method for PDEs, so I'm curious how these methods compare to SPH.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="22717" LastActivityDate="2018-10-22T09:03:44.663" Title="Computational cost comparison of DNS and SPH" Tags="|computational-physics|numerical-modelling|sph|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30395" PostTypeId="1" AcceptedAnswerId="30397" CreationDate="2018-10-21T16:37:22.810" Score="3" ViewCount="1101" Body="&lt;p&gt;I'm trying to calculate the xyz velocity vectors for a circular orbit given a set of position vectors, but I can't seem to get it right. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula for obtaining the velocity magnitude is the square root of gravitational constant multiplied by the mass of the primary divided by the distance from the primary to the object orbiting it. I'm using solar masses for mass, astronomical units for distance and years for time. To simplify things, the primary is our sun, which means that G * M is equal to 39.5. The position of the Sun is 0, 0, 0 and the vectors you see for Earth are in au and were obtained from the JPL Horizons tool. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    function getDistanceParams(p1, p2) {&#xA;      const dx = p2.x - p1.x;&#xA;      const dy = p2.y - p1.y;&#xA;      const dz = p2.z - p1.z;&#xA;&#xA;      return { dx, dy, dz, dSquared: dx * dx + dy * dy + dz * dz };&#xA;    }&#xA;&#xA;    function getCircularOrbit() {&#xA;      const dParams = getDistanceParams(&#xA;        { x: 0, y: 0, z: 0 },&#xA;        {&#xA;          x: 0.9197324105567349,&#xA;          y: -0.4147318273536994,&#xA;          z: -1.750037390352759e-5&#xA;        }&#xA;      );&#xA;&#xA;      const vMag = Math.sqrt(39.5 / Math.sqrt(dParams.dSquared));&#xA;&#xA;      return {vx: dParams.dx * vMag, vy: dParams.dy * vMag, vz: dParams.dz * vMag};&#xA;    }&#xA;&#xA;    getCircularOrbit();&#xA;&#xA;/*&#xA;&#xA;The above outputs this:&#xA;&#xA;{ vx: 5.754832301294686,&#xA;  vy: -2.59500707927136,&#xA;  vz: -0.00010950110691846996 }&#xA;&#xA;The actual velocity vectors:&#xA;&#xA;  vx: 2.4645428337894026,&#xA;  vy: 5.7097644117945805,&#xA;  vz: -3.3177815766459033e-4,&#xA;&#xA;*/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where am I going wrong??? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update: Code that works&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case anyone is curious, here's the code that ended up doing the job:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;export function getDistanceParams(p1, p2) {&#xA;  const dx = p2.x - p1.x;&#xA;  const dy = p2.y - p1.y;&#xA;  const dz = p2.z - p1.z;&#xA;&#xA;  return { dx, dy, dz, dSquared: dx * dx + dy * dy + dz * dz };&#xA;}&#xA;&#xA;export function getIdealCircularOrbit(primary, secondary, g) {&#xA;  const dParams = getDistanceParams(primary, secondary);&#xA;&#xA;  const d = Math.sqrt(dParams.dSquared);&#xA;&#xA;  const vMag = Math.sqrt(g * primary.m / d);&#xA;&#xA;  return {&#xA;    x: secondary.x,&#xA;    y: secondary.y,&#xA;    z: secondary.z,&#xA;    vx: primary.vx + -dParams.dy * vMag / d,&#xA;    vy: primary.vy + dParams.dx * vMag / d,&#xA;    vz: primary.vz + dParams.dz * vMag / d&#xA;  };&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27969" LastEditorUserId="27969" LastEditDate="2018-10-22T14:34:25.183" LastActivityDate="2018-10-22T14:34:25.183" Title="Calculate xyz velocity vectors for circular orbit" Tags="|vector|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30396" PostTypeId="1" CreationDate="2018-10-21T17:22:47.613" Score="2" ViewCount="177" Body="&lt;p&gt;I have a somewhat large (20+ dimensional) root finding problem that I'm solving with Levenberg-Marquardt.  One of the functions has box constraints on [0, 2].  When it is against those bounds it will not be able to find a zero of the function.  Leaving that in the solution and letting the optimizer try to find a minimum does not find a good solution for the rest of the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried a few different techniques to insert zeros into that function, the most naive being:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if ( y0[m] &amp;lt;= 0 || y0[m] &amp;gt;= 2 )&#xA;  z[n] = 0&#xA;else&#xA;  z[n] = H0tf&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(where y0[m] is the independent variable that is bounded, z[n] is the zero   function being produce, and H0tf is a somewhat complicated function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried less naive ways of introducing those zeros, with various potentials introduced near 0 and 2 so that the function becomes smooth and not discontinuous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overall difficulty of all of those approaches is that now in the non-bounded case the optimizer needs to solve a global optimization problem where there may be a zero at 0 and 2 and also the correct 0 in H0tf somewhere in the [0, 2] range, which gets very 'hacky' trying to solve that with a local optimizer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to handle somehow deleting this function from consideration of the optimizer when it is at its bounds?  And what do you call such a technique?&lt;/p&gt;&#xA;" OwnerUserId="27129" LastEditorUserId="27129" LastEditDate="2018-10-21T20:28:36.977" LastActivityDate="2018-12-21T01:01:35.350" Title="Techniques to remove a function from Levenberg-Marquardt when it is against box constraints" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30397" PostTypeId="2" ParentId="30395" CreationDate="2018-10-21T17:30:15.993" Score="3" Body="&lt;p&gt;In your code, you seem to be calculating the magnitude &lt;span class=&quot;math-container&quot;&gt;$|\vec{v}|$&lt;/span&gt; of the velocity (the speed) from your gravitational formula and multiplying it by the components of the position vector &lt;span class=&quot;math-container&quot;&gt;$\vec{r}$&lt;/span&gt;. There are two errors here. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You need to multiply the speed by a &lt;em&gt;unit&lt;/em&gt; vector in the direction of the velocity, in order to get the velocity vector &lt;span class=&quot;math-container&quot;&gt;$\vec{v}$&lt;/span&gt;. In general the&#xA;position vector is not a unit vector (actually in this case, it is pretty close, because you chose astronomical units, but that is just luck).&lt;/li&gt;&#xA;&lt;li&gt;The position vector &lt;span class=&quot;math-container&quot;&gt;$\vec{r}$&lt;/span&gt; is pointing in a radial direction, whereas the velocity vector &lt;span class=&quot;math-container&quot;&gt;$\vec{v}$&lt;/span&gt; (for circular motion) &#xA;lies in a tangential direction.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You are not actually inputting to your program enough information to work out the direction of the velocity vector. You need to know the plane in which the rotation is taking place, and the sense of the rotation (clockwise or anticlockwise). The key equation is &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{v}=\vec{\omega}\times\vec{r}=\omega\hat{n}\times\vec{r}&#xA;$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$\vec{\omega}=\omega\hat{n}$&lt;/span&gt; is the angular velocity vector&#xA;and &lt;span class=&quot;math-container&quot;&gt;$\times$&lt;/span&gt; is the vector cross product.&#xA;Assuming that you know the unit vector &lt;span class=&quot;math-container&quot;&gt;$\hat{n}$&lt;/span&gt; &#xA;pointing in the direction of&#xA;&lt;span class=&quot;math-container&quot;&gt;$\vec{\omega}$&lt;/span&gt;,&#xA;you can deduce the direction of &lt;span class=&quot;math-container&quot;&gt;$\vec{v}$&lt;/span&gt; from the above formula.&#xA;Alternatively, you can calculate the magnitude &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; from the speed&#xA;and the radius of the orbit, and use it together with &lt;span class=&quot;math-container&quot;&gt;$\hat{n}$&lt;/span&gt; &#xA;directly in the above formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you are prepared to make the approximation that the rotation&#xA;is entirely in the &lt;span class=&quot;math-container&quot;&gt;$xy$&lt;/span&gt; plane (which is not exactly correct, from your numbers, but quite close) then you know &lt;span class=&quot;math-container&quot;&gt;$\hat{n}$&lt;/span&gt; and can easily work out the direction of &lt;span class=&quot;math-container&quot;&gt;$\vec{v}$&lt;/span&gt; from &lt;span class=&quot;math-container&quot;&gt;$\vec{r}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-10-21T17:30:15.993" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30398" PostTypeId="2" ParentId="30396" CreationDate="2018-10-21T20:36:50.217" Score="1" Body="&lt;p&gt;Did you try adding penalty regularisation scheme to your least-squares objective function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Levenburg-Marquardt algorithm with attempt to minimise the residual, &lt;span class=&quot;math-container&quot;&gt;$r(x_i)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; is your parameter vector. You could try adding a penalty term which penalizes the solver for suggesting to place the troublesome parameter near to a boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$r^\prime(x_i) = r(x_i) + p(x_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$p(x_i)$&lt;/span&gt; is the penalty term, which will be non-zero when one of your parameters gets stuck or approaches the boundary. Obviously, you should only accept solutions where &lt;span class=&quot;math-container&quot;&gt;$p(x_i)=0$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, have you tried Trust Region Reflection algorithm. These things can be problem specific, but I have been very impressed with how efficient and robust this can be. It is the default option in &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.optimize.least_squares.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3691" LastEditorUserId="3691" LastEditDate="2018-10-21T23:33:49.383" LastActivityDate="2018-10-21T23:33:49.383" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30399" PostTypeId="2" ParentId="30394" CreationDate="2018-10-21T21:24:20.097" Score="1" Body="&lt;p&gt;I don't think you can say that SPH is always much less costly than continuum methods. Less costly than implicit DNS methods, perhaps, but explicit ones can be faster. These have their own problems though, mainly that it can be tricky to predict how the time step will be chosen according to the CFL criterion. That makes them at least pretty useless for real-time applications – you can't rely on it that the needed frame rate will be fulfilled. SPH in principle also needs well-chosen time steps, but like with implicit DSL these aren't all-important in the sense that the solver goes completely berserk when you exceed it – you'll merely get a solution that doesn't really obey the physics equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is probably the main reason that the different approaches are differently popular in science and graphics: in science and engineering, you want results that can well be reasoned about. You want to be able to evaluate local quantities in absolute terms, confirm some theoretical prediction within given error bounds, etc.. DNS (even though it's often enough still rather shaky theoretically speaking, “we've tried two different resolutions and the result agrees, probably this is close enough to the exact solution”) can fulfill those goals pretty well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In graphics, you scarcely care about such quantitative concerns. What you care about is that it should &lt;em&gt;look good&lt;/em&gt; and in particular without obvious artifacts. Both DNS and SPH do cause artifacts, but in DNS they tend to be visually “obviously digital” blocking etc. effects. For SPH it'll be more of a randomly-scattered blurring. For science that's actually a bit of a problem because it can be tricky to predict and recognise, but for graphics that's an &lt;em&gt;advantage&lt;/em&gt; because you really don't want the viewers to notice it when the results are unphysical, unlike in science where physical correctness is the most important thing and it's actually &lt;em&gt;good&lt;/em&gt; if artifacts are visually obvious.&lt;/p&gt;&#xA;" OwnerUserId="562" LastEditorUserId="562" LastEditDate="2018-10-22T09:03:44.663" LastActivityDate="2018-10-22T09:03:44.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30400" PostTypeId="2" ParentId="30394" CreationDate="2018-10-21T22:51:49.857" Score="2" Body="&lt;p&gt;It is extremely difficult to achieve greater than second order accuracy using SPH. Hence once cannot compare SPH with DNS (FVM or FDM). Also the computational cost is much higher for SPH for the same order of accuracy. However, SPH produces realistic looking results even for low resolutions due to its ability to track interfaces implicitly. With FVM, one has to resort to more involved techniques such as the Volume of Fluids method to capture the surfaces of liquids. SPH is a meshless method. Hence one could avoid pre-processing steps such as mesh building and refinement. All these advantages of SPH outweighs the disadvantages with respect to computer graphics applications. When it comes to solving engineering problems, accuracy becomes important and SPH becomes an expensive approach, at least for single phase flow scenarios. &lt;/p&gt;&#xA;" OwnerUserId="29022" LastActivityDate="2018-10-21T22:51:49.857" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30404" PostTypeId="1" CreationDate="2018-10-22T20:33:56.500" Score="0" ViewCount="100" Body="&lt;p&gt;Let us think on the Poisson equation &lt;span class=&quot;math-container&quot;&gt;$\nabla^2 u(\bf{x})=\rho(x)$&lt;/span&gt; with Neumann boundary conditions, with &lt;span class=&quot;math-container&quot;&gt;$\bf{x}=\it (x,y)$&lt;/span&gt; in 2D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a stencil with central differences in both &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; (forgive my label &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; on the vertical, it is supposed to be &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;)  directions.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ytdGi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ytdGi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$\beta=\Delta x/\Delta y$&lt;/span&gt;. More specifically we have the&#xA;equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{eqnarray*}&#xA;  w_{i+1 j} -2 w_{ij} \left (  1 + \beta^2 \right)&#xA;  + w_{i-1 j} + \beta^2 (w_{i j+1} + w_{i j-1})&#xA;  = \Delta x^2 \rho(x,y)&#xA;\end{eqnarray*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can get a similar stencil in the (hyerbolic) wave equation in 1D where&#xA;time is the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; (vertical axis). We can always find, recursively &#xA;&lt;span class=&quot;math-container&quot;&gt;$w_{i j+1}$&lt;/span&gt; in terms of all other &lt;span class=&quot;math-container&quot;&gt;$w_{lm}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$l=i,i+1,i-1$&lt;/span&gt; and&#xA;&lt;span class=&quot;math-container&quot;&gt;$m=j,j-1$&lt;/span&gt;. The points 1 cell away from the boundary can be computed with&#xA;a ghost boundary condition (assuming Neumann BC with derivative equal to 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been searching for hours and all literature shows me an implicit method where I need to invert a huge &lt;span class=&quot;math-container&quot;&gt;$(n-1 \times m-1)$&lt;/span&gt; matrix. Is there a specific reason why the elliptic equation is not solved as we do the 1D wave explicit solution centered in space and time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or...if it could be solved, is there a place to look for that solution? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="28330" LastEditorUserId="28330" LastEditDate="2018-10-22T20:51:23.493" LastActivityDate="2018-10-22T20:51:23.493" Title="why I cannot find explicit finite difference for elliptic equation" Tags="|finite-difference|numerics|differential-equations|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30405" PostTypeId="2" ParentId="30387" CreationDate="2018-10-22T22:40:05.770" Score="0" Body="&lt;p&gt;The associativity issue is address in other answers (and see my comment); it mostly plays in collectives. Another thing to watch out for it not to use wildcard receives. If the order of incoming messages is indeterminate the associativity issue comes up again.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2018-10-22T22:40:05.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30406" PostTypeId="1" CreationDate="2018-10-22T23:07:07.033" Score="3" ViewCount="129" Body="&lt;p&gt;Recently, I have taken an interest in performing combinatorial analysis for the game of 21 (blackjack) and attempted to use my AMD APU to try and thread the program via the 4 cores on the chip. Generating the proper strategy and expectations has proven to be a success, as well as taking very few seconds (around 12) to compute the optimal strategy for any given rule(s) and deck composition. However, when it comes to computing other deck compositions, it would take 12.63 years to compute all deck states...for a single deck. So, another approach would be to simulate a select limit of random deck subsets via a Monte-Carlo method. This will work! However, the 12 seconds per analysis is a &lt;em&gt;huge&lt;/em&gt; bottleneck! Randomly selecting just 10E6 subsets would take around 139 days. Not feasible with my current CPU, even with threading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;A primer on how I am analysing the strategy chart(s):&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1.) I first start by computing what the overall expectation for standing is by taking a set of n_cards for the player and enumerating all possible dealer hand sets. I compare the outcome of each specific dealer total against the current hand total of the player hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.) I then compute the hitting expectation by taking a player hand set {23} and finding matching hand sets for x:2-&gt;A, so that we can compute the weighed expectation of hitting to {232}, {233}, {234}, ...,{23A}. We do this by computing all hard 21's first, then hard 20's...down to all hard 12's; then do the same by computing hitting all soft 21's to soft 13's; then compute all hard 11's to hard 5; and finally, we compute hitting all pair cards ({22}, {33}, ..., {TT}, {AA}.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3.) Doubling is the same as hitting, except we only draw one card and stand for any 2_card player hand. We return the expectation of drawing to a specific 3_card hand and multiply the expectation by 2 times the probability of that drawn card.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4.) For splitting, we will take steps 1.) to 3.), we will repeat 1-&gt;3 up to 11 times. Why? We are computing the expectation of splitting to a new hand by removing the rank for which we are splitting. (For example: We want to split {22}. We will compute standing, hitting, doubling for a given deck subset, minus one of the ranks we are splitting. If we originally split {22} for a deck subset of {4 4 4 4 4 4 4 4 16 4: 2-A} or {3 1 4 3 4 4 3 1 14 3: 2-A}, our new subsets would be {3 4 4 4 4 4 4 4 16 4: 2-A} or {2 1 4 3 4 4 3 1 14 3: 2-A}.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5.) After all of this, we should get back out optimal strategy with respect to the computed expectation(s) for each possible decision for each player hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I have been looking into seeing if there is a quicker way of analysing the strategy using either GPGPU or FPGA/ASIC. I first looked at GPU programming as a possible route, however, I am not sure if it is feasible as there are 3072 unique player hand states, meaning there will be 3072 unique data structures to compute for standing, hitting, doubling, and splitting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With all this said, I was hoping there was a way to perform some form of simultaneous processing, where the state of each expectation changes dependent on the change in the rules and/or the deck state. That is, rather than recompute each expectation for every change in rules/deck_state, there would be some dependence on the output for each data point. For example: We have three data points named A, B, and C. We also have state data {1 2 3}. For each data point, there is a state equation that we possess:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A = data[0]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B = A + data[1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C = B +data[2]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under serial processing, we would evaluate A first for 1, B for 2 plus A, and C for 3 plus B. If we change data to be {2 2 3}, then A would be 2, B would be 4, and C would be 7. Under serial processing, this would take several cycles per data point, for each data point. Is there a method under FPGA where the change in state is instantaneous? That is if we change any point of data, that A, B, and C will change after data changes, simultaneously?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, what benefits does FPGA/GPGPU processing offer to a programmer who wants to accurately and quickly compute large volumes of floating point data in a near instant? Basically, is there a way to rapidly compute the optimal strategy of a blackjack game that is faster than multi-threaded CPU processing that is not 12 seconds long?&lt;/p&gt;&#xA;" OwnerUserId="29034" LastEditorUserId="20688" LastEditDate="2018-10-22T23:46:41.787" LastActivityDate="2020-11-24T09:01:30.140" Title="GPGPU/FPGA programming for Combinatorial Analysis" Tags="|parallel-computing|data-analysis|probability|combinatorics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30407" PostTypeId="1" AcceptedAnswerId="30409" CreationDate="2018-10-23T00:44:14.913" Score="9" ViewCount="2235" Body="&lt;p&gt;I'm a noob into eigenvalues algorithms, but something call my attention. QR algorithm works with real/complex matrices producing real/complex eigenvalues. However, it can not produce &lt;em&gt;complex eigenvalues from a real matrix&lt;/em&gt;. Here a simplistic example written in Julia and derivated from &lt;a href=&quot;https://scicomp.stackexchange.com/questions/24622/how-to-improve-this-double-shift-qr-algorithm-for-non-symmetric-matrices&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.mathworks.com/matlabcentral/fileexchange/13813-determination-of-the-eigenvalues-using-the-qr-decomposition&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using LinearAlgebra&#xA;A = [7 3 4 11 -9 -2;&#xA;    -6 4 -5 7 1 12;&#xA;    -1 -9 2 2 9 1;&#xA;    -8 0 -1 5 0 8;&#xA;    -4 3 -5 7 2 10;&#xA;    6 1 4 -11 -7 -1]&#xA;M = copy(A)&#xA;&#xA;for i=1:100&#xA;    global M&#xA;    Q,R = LinearAlgebra.qr(M);&#xA;    M=R*Q;&#xA;end&#xA;&#xA;display(diag(M))&#xA;display(eigvals(A))&#xA;&#xA;6-element Array{Float64,1}:&#xA; -2.8415406888480472&#xA;  8.675063708533656&#xA;  3.658872985794657&#xA;  6.3411270142053695&#xA;  0.12201942568224483&#xA;  3.0444575546321087&#xA;6-element Array{Complex{Float64},1}:&#xA;  2.916761509842819 + 13.248032079355992im&#xA;  2.916761509842819 - 13.248032079355992im&#xA;  5.000000000000005 + 6.000000000000003im&#xA;  5.000000000000005 - 6.000000000000003im&#xA; 1.5832384901571723 + 1.4155521348117128im&#xA; 1.5832384901571723 - 1.4155521348117128im&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Defining matrix &lt;strong&gt;A&lt;/strong&gt; as complex, with only real components, makes no difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are : &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what is my conceptual misunderstanding on the subject ?  &lt;/li&gt;&#xA;&lt;li&gt;what step am I doing wrong ?  &lt;/li&gt;&#xA;&lt;li&gt;and how to fix it ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="29035" LastEditorUserId="1804" LastEditDate="2018-10-23T07:18:40.433" LastActivityDate="2019-07-16T03:11:36.730" Title="How does the QR algorithm applied to a real matrix returns complex eigenvalues?" Tags="|linear-algebra|eigenvalues|iterative-method|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30409" PostTypeId="2" ParentId="30407" CreationDate="2018-10-23T06:59:15.500" Score="14" Body="&lt;p&gt;In a nutshell, the QR algorithm applied to a matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is an iterative procedure that converges to the real Schur decomposition: a unitary matrix &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and a matrix &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; in &lt;em&gt;block upper triangular form&lt;/em&gt; (see below) such that &lt;span class=&quot;math-container&quot;&gt;$A = QRQ^T$&lt;/span&gt;. It follows that the columns of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; are the eigenvectors (which are the principal objects that are computed!) and that &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; has the same eigenvalues as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key point is the &lt;em&gt;block upper triangular form&lt;/em&gt;, which means that &#xA;&lt;span class=&quot;math-container&quot;&gt;$$  R = \begin{pmatrix}&#xA;            R_{11}&amp;amp;&amp;amp;*\\&amp;amp;\ddots\\0&amp;amp;&amp;amp;R_{mm}&#xA;        \end{pmatrix},&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$R_{ii}$&lt;/span&gt; are &lt;strong&gt;real&lt;/strong&gt; blocks of either&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;size &lt;span class=&quot;math-container&quot;&gt;$1\times 1$&lt;/span&gt;, in which case &lt;span class=&quot;math-container&quot;&gt;$R_{ii}$&lt;/span&gt; is a (real) eigenvalue of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, or&lt;/li&gt;&#xA;&lt;li&gt;size &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt;, in which case &lt;span class=&quot;math-container&quot;&gt;$R_{ii}$&lt;/span&gt; has a pair of complex conjugate eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (such as &lt;span class=&quot;math-container&quot;&gt;$2+i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$2-i$&lt;/span&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since you can compute eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt; matrices analytically (as roots of a quadratic polynomial), it is a cheap step to extract the complex eigenvalues from the computed (approximation of) &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; in the end -- and this is what &lt;code&gt;eigvals&lt;/code&gt; does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So your conceptual misunderstanding is the following: Every real &lt;span class=&quot;math-container&quot;&gt;$n\times n$&lt;/span&gt; matrix has &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; eigenvalues, but they don't have to be real (or distinct) -- look at Richard Zhang's now deleted example, &lt;span class=&quot;math-container&quot;&gt;$A=\begin{pmatrix} 0 &amp;amp; -1 \\ 1 &amp;amp; 0 \end{pmatrix}$&lt;/span&gt;, which has the eigenvalues &lt;span class=&quot;math-container&quot;&gt;$\pm i$&lt;/span&gt;. Only if the matrix is &lt;strong&gt;symmetric&lt;/strong&gt; are the eigenvalues guaranteed to be real (and the matrix &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; to be diagonal) -- so your code only works for symmetric matrices. If the input matrix is non-symmetric, you additionally have to extract the (complex) eigenvalues by identifying the &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt; blocks (e.g., by checking whether a subdiagonal element is greater than a tolerance) and if so, computing the eigenvalues by a formula.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a bit tedious, but if you're willing to cheat a bit and use &lt;code&gt;eigenvalues&lt;/code&gt; for the &lt;span class=&quot;math-container&quot;&gt;$2\times 2$&lt;/span&gt; blocks, the following modification of your code will do it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using LinearAlgebra&#xA;A = [7 3 4 11 -9 -2;&#xA;    -6 4 -5 7 1 12;&#xA;    -1 -9 2 2 9 1;&#xA;    -8 0 -1 5 0 8;&#xA;    -4 3 -5 7 2 10;&#xA;    6 1 4 -11 -7 -1]&#xA;M = copy(A)&#xA;&#xA;for i=1:100&#xA;    global M&#xA;    Q,R = LinearAlgebra.qr(M);&#xA;    M=R*Q;&#xA;end&#xA;&#xA;eig = Complex{Float64}[]&#xA;let&#xA;    i=1&#xA;    N=size(M,1)&#xA;    while i&amp;lt;N   &#xA;        if abs(M[i+1,i])&amp;lt;1e-10             &#xA;            append!(eig,M[i,i])         &#xA;            i+=1         &#xA;        else             &#xA;            append!(eig,eigvals(M[i:i+1,i:i+1]))         &#xA;            i+=2         &#xA;        end                 &#xA;    end&#xA;    if length(eig)&amp;lt;N&#xA;       append!(eig,M[N:N,N:N])&#xA;    end                 &#xA;end       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1804" LastEditorUserId="29035" LastEditDate="2018-10-23T17:34:27.070" LastActivityDate="2018-10-23T17:34:27.070" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30410" PostTypeId="2" ParentId="30407" CreationDate="2018-10-23T07:23:01.347" Score="5" Body="&lt;p&gt;Forget about the QR algorithm, and remember what eigenvalues are - they are the roots of the characteristic polynomial for the matrix (see e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Characteristic_polynomial&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Characteristic_polynomial&lt;/a&gt;). For a real matrix of order N this is a polynomial of order N with real coefficients. But real coefficients does not mean real roots necessarily, you may have complex conjugate pairs. Hence a general real matrix may have complex eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="17475" LastActivityDate="2018-10-23T07:23:01.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30411" PostTypeId="2" ParentId="30391" CreationDate="2018-10-23T09:42:33.927" Score="0" Body="&lt;p&gt;I have found very simple solution! At least, it seem to work well with my test case.&#xA;Instead of explicit calculation of &lt;span class=&quot;math-container&quot;&gt;$\phi'(p_n)$&lt;/span&gt;, I use it's simple approximation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\phi'(p_n) \approx \frac{\phi(p_n)-\phi(p_{n-1})}{p_n-p_{n-1}}&#xA;$$&lt;/span&gt;&#xA;To use this algorithm, we need to have two starting values (&lt;span class=&quot;math-container&quot;&gt;$p_0, p_1$&lt;/span&gt;) instead of one (&lt;span class=&quot;math-container&quot;&gt;$p_0$&lt;/span&gt;). Currently I calculate &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; via single step of original Newton's method with explicit &lt;span class=&quot;math-container&quot;&gt;$\phi'(p_0)$&lt;/span&gt;. As stated above, it gives very small difference between &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_0$&lt;/span&gt;, but following steps with approximate &lt;span class=&quot;math-container&quot;&gt;$\phi'(p_n)$&lt;/span&gt; produce very good increments and the method converges in usual 4-5 steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; As Praveen Chandrashekar mentioned below, I should have checked my derivatives again. There is a pretty simple error in derivative for rarefaction wave functions, it should contain &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{p_l}$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{p_l^\alpha}$&lt;/span&gt; (and similar thing for &lt;span class=&quot;math-container&quot;&gt;$p_r$&lt;/span&gt;). With correct derivatives, original Newton's method converges well.&lt;/p&gt;&#xA;" OwnerUserId="25060" LastEditorUserId="25060" LastEditDate="2018-11-01T09:10:35.760" LastActivityDate="2018-11-01T09:10:35.760" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30412" PostTypeId="1" CreationDate="2018-10-24T13:29:10.187" Score="8" ViewCount="4477" Body="&lt;p&gt;To implement maximum likelihood estimators, I am looking for a good C++ optimization library that plays nicely with Eigen's matrix objects. Eigen has some capability of interfacing of its own but if anyone here has experience of using Eigen with an optimizer library in tandem, that would be great!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of similar questions being asked &lt;a href=&quot;https://scicomp.stackexchange.com/questions/2969/recommendations-for-a-usable-fast-gpl-compatible-derivative-free-numerical-opti&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://scicomp.stackexchange.com/questions/24874/looking-for-c-function-for-performing-optimization-of-parameters-for-multivari&quot;&gt;here&lt;/a&gt; but those are specific for derivative-free methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think a 'good' library would be one that plays nicely with other libraries such as Eigen or Armadillo. So far, I have found &lt;a href=&quot;http://dlib.net/optimization.html&quot; rel=&quot;noreferrer&quot;&gt;dlibnet&lt;/a&gt; and &lt;a href=&quot;https://nlopt.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;nlopt&lt;/a&gt;, which seem promising.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, the answer to this question could be a longer one with having the pros and cons of each library listed as well. Thank you so much!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I have decided to go with &lt;a href=&quot;https://github.com/PatWie/CppNumericalSolvers/blob/master/README.md&quot; rel=&quot;noreferrer&quot;&gt;CppNumericalSolvers&lt;/a&gt; for now. There was also &lt;a href=&quot;http://ceres-solver.org/gradient_solver.html&quot; rel=&quot;noreferrer&quot;&gt;Ceres&lt;/a&gt; (linking to the relevant bit) but what put me off that was the emphasis on non-linear least squares, which requires the objective function to be of a certain format, which for more complex statistical models cannot always be guaranteed. Even iterated generalised least squares which can achieve certain maximum likelihood estimators would fail this format. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, CppNumericalSolvers supports bindings to TensorFlow, Eigen, and MATLAB. Note that both Ceres and CppNumericalSolvers ask you to formulate your function as a class rather than a lambda function, which may put off some beginners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, there is &lt;a href=&quot;https://root.cern.ch/&quot; rel=&quot;noreferrer&quot;&gt;ROOT&lt;/a&gt; which is an environment for C++ with tons of routines where C++ source files can be loaded as 'macros.' Elementary BFGS optimizers exist with plenty of examples such as &lt;a href=&quot;https://root.cern.ch/numerical-minimization&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. However, this is an interpreted environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am still hoping someone might answer my questions with some actual experience using a library.&lt;/p&gt;&#xA;" OwnerUserId="29045" LastEditorUserId="29045" LastEditDate="2018-10-24T16:34:43.667" LastActivityDate="2018-10-24T16:34:43.667" Title="Good C++ optimization library for BFGS" Tags="|optimization|c++|reference-request|libraries|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30413" PostTypeId="1" AcceptedAnswerId="30416" CreationDate="2018-10-24T17:15:35.763" Score="0" ViewCount="46" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vec{a}\cdot\vec{b} \approx c$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vec{\alpha} \cdot \vec{\beta} = c$&lt;/span&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\vec{\alpha}$&lt;/span&gt; is close to &lt;span class=&quot;math-container&quot;&gt;$\vec{a}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\vec{\beta}$&lt;/span&gt; is close to &lt;span class=&quot;math-container&quot;&gt;$\vec{b}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given &lt;span class=&quot;math-container&quot;&gt;$\vec{a}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\vec{b}$&lt;/span&gt; and c, how to find &lt;span class=&quot;math-container&quot;&gt;$\vec{\alpha}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\vec{\beta}$&lt;/span&gt; quickly? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If necessary, I can assume distance between &lt;span class=&quot;math-container&quot;&gt;$\vec{\alpha}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\vec{a}$&lt;/span&gt; is usually much smaller than distance between &lt;span class=&quot;math-container&quot;&gt;$\vec{b}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\vec{\beta}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each vector has about 10 - 15 elements but I have to do this for about 10^4 vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If probability should sum to 1 is the constraint, I can normalize the probabilty after each (or several) optimization step to control the numerical error. But I can't do the same thing for this constraint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lagrange multipliers and Linearization of the constraints give:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \vec{b} \cdot \vec{\alpha} + \vec{a} \cdot \vec{\beta} = \vec{\alpha} \cdot \vec{\beta} + \vec{a} \cdot \vec{b}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\vec {\alpha} + \lambda \vec{\beta} = \vec{a} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\vec {\beta} + \lambda \vec{\alpha} = \vec{b} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is still not a linear system because of the &lt;span class=&quot;math-container&quot;&gt;$\lambda \vec{\alpha}$&lt;/span&gt; terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to do this faster?&lt;/p&gt;&#xA;" OwnerUserId="28896" LastEditorUserId="28896" LastEditDate="2018-10-25T01:01:19.730" LastActivityDate="2018-10-25T01:01:19.730" Title="Slightly change two vectors to satisfy a constraint" Tags="|linear-algebra|nonlinear-programming|approximation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30416" PostTypeId="2" ParentId="30413" CreationDate="2018-10-24T22:51:18.197" Score="2" Body="&lt;p&gt;There are of course infinitely many vectors &lt;span class=&quot;math-container&quot;&gt;$\vec \alpha,\vec \beta$&lt;/span&gt; that satisfy &lt;span class=&quot;math-container&quot;&gt;$\vec \alpha\cdot\vec \beta=c$&lt;/span&gt;. So if you want to have a particular pair of vectors, you will have to be precise when stating what you mean that &lt;span class=&quot;math-container&quot;&gt;$\vec \alpha,\vec \beta$&lt;/span&gt; should be &quot;close&quot; to &lt;span class=&quot;math-container&quot;&gt;$\vec a,\vec b$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One pair that satisfies this is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \vec \alpha = \vec a,&#xA;  \qquad&#xA;  \qquad&#xA;  \vec \beta = \frac{c}{\vec a \cdot \vec b}\;\vec b.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-10-24T22:51:18.197" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30418" PostTypeId="1" AcceptedAnswerId="30425" CreationDate="2018-10-25T08:57:21.867" Score="4" ViewCount="138" Body="&lt;p&gt;Consider the least squares problem,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_{\mathbf{a},\mathbf{b}} || \mathbf{f}(\mathbf{a},\mathbf{b})||^2&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a},\mathbf{b}$&lt;/span&gt; represent the unknown parameters to be found. In my problem, the parameters &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}$&lt;/span&gt; appear linearly in the residual vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{f}$&lt;/span&gt; while the parameters &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}$&lt;/span&gt; are nonlinear. So my Jacobian matrix will look like:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J =&#xA;\pmatrix{J_a &amp;amp; J_b(\mathbf{b})}&#xA;$$&lt;/span&gt;&#xA;In other words, part of the Jacobian matrix (&lt;span class=&quot;math-container&quot;&gt;$J_a$&lt;/span&gt;) is fixed and will not change from one least squares iteration to the next. Can this somehow be exploited to speed up the iterative algorithm? On each iteration the following linear least squares problem needs to be solved for the step &lt;span class=&quot;math-container&quot;&gt;$(\delta \mathbf{a}, \delta \mathbf{b})$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\pmatrix{J_a &amp;amp; J_b(\mathbf{b})} \pmatrix{ \delta \mathbf{a} \\ \delta \mathbf{b}} = -\mathbf{f}(\mathbf{a},\mathbf{b})&#xA;$$&lt;/span&gt;&#xA;The only solution I've come up with so far is to use the normal equations approach, and the normal equations matrix is,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J^T J = \pmatrix{ J_a^T J_a &amp;amp; J_a^T J_b(\mathbf{b}) \\ J_b^T(\mathbf{b}) J_a &amp;amp; J_b^T(\mathbf{b}) J_b(\mathbf{b})}&#xA;$$&lt;/span&gt;&#xA;The upper left term &lt;span class=&quot;math-container&quot;&gt;$J_a^T J_a$&lt;/span&gt; can be precomputed once while the other terms need to be computed for each iteration. Then I need to solve&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(J^T J) \pmatrix{ \delta \mathbf{a} \\ \delta \mathbf{b}} = -J^T \mathbf{f}&#xA;$$&lt;/span&gt;&#xA;and I can save a little bit of computation when constructing &lt;span class=&quot;math-container&quot;&gt;$J^T J$&lt;/span&gt; using the precomputed &lt;span class=&quot;math-container&quot;&gt;$J_a^T J_a$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just wondering if anyone knows of any other clever methods to solve such a problem. Ideally I'm wondering if its possible to somehow remove the &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}$&lt;/span&gt; parameters and solve a smaller sub-problem involving only &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}$&lt;/span&gt; at each iteration?&lt;/p&gt;&#xA;" OwnerUserId="28440" LastActivityDate="2018-10-26T17:18:23.073" Title="Nonlinear least squares when some parameters are linear" Tags="|optimization|nonlinear-equations|least-squares|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30420" PostTypeId="1" CreationDate="2018-10-25T14:54:17.170" Score="8" ViewCount="455" Body="&lt;p&gt;The system in the title has a damper factor &lt;span class=&quot;math-container&quot;&gt;$\lambda &amp;gt; 0$&lt;/span&gt; and the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is sparse and rectangular, with a structure I can exploit to solve matrix vector products very fast. My current solver, LSMR, is trying to solve the normal equations &lt;span class=&quot;math-container&quot;&gt;$(A^TA + \lambda I) x = A^T b$&lt;/span&gt; associated to the original problem &lt;span class=&quot;math-container&quot;&gt;$\min \|Ax - b\|$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although each iteration is computed very fast, the algorithm uses the maximum number of iterations. I know this can be fixed with a good preconditioner. This is where lies my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A^TA + \lambda I$&lt;/span&gt; is a SPD matrix, which is a good property to have. On the other side, this matrix is no more sparse. I don't know how to choose and use a preconditioner for this dense matrix. I suppose this is already worked by someone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how to proceed in this case and, if possible, how to use the sparsity of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to obtain a good preconditioner. What are the common approaches?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; In order to be more complete, I'll briefly describe how the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is obtained. My problem at hand consists in minimizing the error associated to a &lt;em&gt;low tensor rank-&lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; approximation&lt;/em&gt;. You can consider a tensor &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; as being a multidimensional array. In this case, a 3-D multidimensional array with coordinates &lt;span class=&quot;math-container&quot;&gt;$T_{ijk}$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$1 \leq i \leq m, 1 \leq j \leq n, 1 \leq k \leq p$&lt;/span&gt;. I am considering an approximation &lt;span class=&quot;math-container&quot;&gt;$\tilde{T}_{ijk} = \sum_{\ell=1}^r X_{i \ell} \cdot Y_{j \ell} \cdot Z_{k \ell}$&lt;/span&gt;. The error in this approximation is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{1}{2} \sum_{i,j,k} \left( T_{ijk} - \tilde{T}_{ijk} \right)^2 = \frac{1}{2} \sum_{i,j,k} res_{i,j,k} (X,Y,Z)^2,$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$X, Y, Z$&lt;/span&gt; lists all components &lt;span class=&quot;math-container&quot;&gt;$X_{i \ell}, Y_{j \ell}, Z_{k \ell}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$res_{ijk}$&lt;/span&gt; is the residual of the component with index &lt;span class=&quot;math-container&quot;&gt;$i,j,k$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the components of &lt;span class=&quot;math-container&quot;&gt;$\tilde{T}$&lt;/span&gt; which minimize the error above, it is of interest to find the Jacobian matrix of &lt;span class=&quot;math-container&quot;&gt;$res = (res_{111}, res_{112}, \ldots, res_{mnp})$&lt;/span&gt;. We have the formulas below for the partial derivatives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial res_{ijk}}{\partial X_{I \ell}} = \left\{ &#xA;\begin{array}{c}&#xA;    - Y_{j \ell} Z_{k \ell},\quad \text{if } i = I,\\&#xA;    0, \quad \text{otherwise}&#xA;\end{array}\right.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial res_{ijk}}{\partial Y_{J \ell}} = \left\{ &#xA;\begin{array}{c}&#xA;    - X_{j \ell} Z_{k \ell},\quad \text{if } j = J,\\&#xA;    0, \quad \text{otherwise}&#xA;\end{array}\right.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial res_{ijk}}{\partial Z_{K \ell}} = \left\{ &#xA;\begin{array}{c}&#xA;    - X_{i \ell} Y_{j \ell},\quad \text{if } k = K,\\&#xA;    0, \quad \text{otherwise}&#xA;\end{array}\right.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give a sparse matrix, which becomes more sparse as we increase the dimensions. The structure follows a nested for loop pattern, from left to right. The figure below shows this structure for &lt;span class=&quot;math-container&quot;&gt;$m = 3, n = 5, p = 7, r = 10$&lt;/span&gt;. I hope this can be useful for someone to spot the &quot;right&quot; preconditioner, because at the moment I really don't know how to proceed. Keep im mind that I'm trying to use this structure to find a preconditioner for &lt;span class=&quot;math-container&quot;&gt;$A^TA + \lambda I$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is this sparse matrix just described.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AAReO.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AAReO.png&quot; alt=&quot;sparse Jacobian&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="6991" LastEditorUserId="6991" LastEditDate="2018-10-25T17:06:13.960" LastActivityDate="2020-02-15T08:54:41.200" Title="How to find a good preconditioner to the system $(A^T A + \lambda I) x = A^T b$?" Tags="|preconditioning|conjugate-gradient|krylov-method|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="30422" PostTypeId="1" AcceptedAnswerId="30471" CreationDate="2018-10-26T15:38:37.180" Score="0" ViewCount="993" Body="&lt;p&gt;Consider two sets of points &lt;span class=&quot;math-container&quot;&gt;$P = (P_1, ...,P_n), \ Q = (Q_1, ..., Q_m) $&lt;/span&gt; included in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^3$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking to compute an optimal affine transformation that &quot;maps&quot; &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, although the sets don't have the same number of elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A trivial case is one where &lt;span class=&quot;math-container&quot;&gt;$m=n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Q = (R \cdot P_1 + T, R\cdot P_2 + T, ..., R\cdot P_n+T)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that I'm not sure how to describe this optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out in comments, I should have been more specific. &#xA;In my application, both sets of points represent different closed curves in space. I'm trying to best position one curve against the other one (in the least squares sense, for example). &lt;/p&gt;&#xA;" OwnerUserId="16732" LastEditorUserId="16732" LastEditDate="2018-10-27T07:30:18.433" LastActivityDate="2018-11-03T16:34:26.890" Title="Compute affine transformation between two sets of points" Tags="|optimization|geometry|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30425" PostTypeId="2" ParentId="30418" CreationDate="2018-10-26T17:18:23.073" Score="2" Body="&lt;p&gt;One simple way is to pre-factorize the block associated with &lt;span class=&quot;math-container&quot;&gt;$J_{a}$&lt;/span&gt;.&#xA;For readability, I will slightly change your notations. Consider solving&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_{x,y}\|f(x,y)\|^{2}\text{ with Jacobian }\nabla f(x,y)=\begin{bmatrix}A &amp;amp; B(y)\end{bmatrix}.&#xA;$$&lt;/span&gt;&#xA;We can solve each Gauss-Newton step of the form&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}A^{T}A &amp;amp; A^{T}B(y)\\&#xA;B^{T}(y)A &amp;amp; B^{T}(y)B(y)&#xA;\end{bmatrix}\begin{bmatrix}\Delta x\\&#xA;\Delta y&#xA;\end{bmatrix}=\begin{bmatrix}c\\&#xA;d&#xA;\end{bmatrix},&#xA;$$&lt;/span&gt;&#xA;by block-LU factorization&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{bmatrix}A^{T}A &amp;amp; A^{T}B(y)\\&#xA;B^{T}(y)A &amp;amp; B^{T}(y)B(y)&#xA;\end{bmatrix}=\begin{bmatrix}A^{T}A &amp;amp; 0\\&#xA;B^{T}(y)A &amp;amp; S(y)&#xA;\end{bmatrix}\begin{bmatrix}I &amp;amp; (A^{T}A)^{-1}B(y)\\&#xA;0 &amp;amp; I&#xA;\end{bmatrix},\tag{1}&#xA;$$&lt;/span&gt;&#xA;where the Schur complement reads&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;S(y)=B^{T}(y)B(y)-B^{T}(y)A(A^{T}A)^{-1}A^{T}B(y).&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, let us precompute the permuted Cholesky of &lt;span class=&quot;math-container&quot;&gt;$A^{T}A$&lt;/span&gt;,&#xA;as in &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A^{T}A=\Pi(LL^{T})\Pi^{T}&#xA;$$&lt;/span&gt;&#xA;with permutation matrix &lt;span class=&quot;math-container&quot;&gt;$\Pi$&lt;/span&gt; and sparse lower-triangular &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; with&#xA;positive diagonals. Then we can invert &lt;span class=&quot;math-container&quot;&gt;$A^{T}A$&lt;/span&gt; via two sparse triangular&#xA;solves:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(A^{T}A)\xi=g\quad\iff L\eta=(\Pi^{T}g),\quad L^{T}(\Pi^{T}\xi)=\eta.\tag{2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Combined, we solve &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Delta y$&lt;/span&gt; by inverting the block&#xA;in (1) analytically, while using (2) to compute matrix-vector products&#xA;with &lt;span class=&quot;math-container&quot;&gt;$(A^{T}A)^{-1}$&lt;/span&gt;, as in&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align*}&#xA;z &amp;amp; =(A^{T}A)^{-1}c\\&#xA;\text{solve }S(y)\Delta y &amp;amp; =d-B^{T}(y)Az\text{ for }\Delta y\\&#xA;\Delta x &amp;amp; =z-(A^{T}A)^{-1}B(y)\Delta y&#xA;\end{align*}$$&lt;/span&gt;&#xA;Note that this is only efficient if &lt;span class=&quot;math-container&quot;&gt;$B(y)$&lt;/span&gt; is long-and-skinny, i.e.&#xA;has many more rows than columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can also apply the same idea to a QR-based technique. The&#xA;resulting complexity is the same, though in practice it is slower but much more accurate.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastActivityDate="2018-10-26T17:18:23.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30426" PostTypeId="1" CreationDate="2018-10-26T20:56:28.897" Score="3" ViewCount="482" Body="&lt;p&gt;Given&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;A_i=B+C_i&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i=1,\dotsc,N$&lt;/span&gt; are large square matrices, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is symmetric, &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; are zero matrices aside for a square block on the diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to factor the &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt;, preferably but not necessarily by &lt;span class=&quot;math-container&quot;&gt;$LU$&lt;/span&gt; decomposition, but I don't want to do this repeatedly &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; times, as &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; may be large. Is there a method of doing this?&lt;/p&gt;&#xA;" OwnerUserId="6775" LastActivityDate="2018-10-29T18:17:29.793" Title="Factoring the sum of two matrices" Tags="|linear-algebra|matrix-factorization|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30428" PostTypeId="2" ParentId="30426" CreationDate="2018-10-27T07:48:41.353" Score="4" Body="&lt;p&gt;The structure of &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; gives a hope that &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; can be described as &lt;strong&gt;a low-rank update&lt;/strong&gt;. If that is the case (size of the square block is much smaller than the size of the matrix &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;), one can apply classical methods for low-rank updates based on Sherman-Morrisson-Woodbury identity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While updating the LU factorization is not the easiest task (the required pivoting might be sensitive to the performed modification) it is certainly possible. To begin with, you might want to read the classic low-rank update section of QR in Golub, Van Loan (Section 12.6 in 2nd edition). Then, proceed with the intricacies of updating the &lt;span class=&quot;math-container&quot;&gt;$PA=LU$&lt;/span&gt; factorization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, since &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is symmetric, one can take advantage of Cholesky factorization whenever &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; is also symmetric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; is not (&lt;em&gt;enough&lt;/em&gt;) low-rank, I don't think there is much hope on any savings here.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-10-29T18:17:29.793" LastActivityDate="2018-10-29T18:17:29.793" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30431" PostTypeId="1" AcceptedAnswerId="30442" CreationDate="2018-10-27T14:32:12.817" Score="2" ViewCount="772" Body="&lt;p&gt;I was reading the documentation of &lt;a href=&quot;https://it.mathworks.com/help/matlab/ref/ode23t.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the MatLab ODE solver &lt;code&gt;ode23t&lt;/code&gt;&lt;/a&gt;, and I've seen that  the trapezoidal rule is used. &#xA;Moreover, the error is estimated by differentiating a cubic polynomial, and that's still fine to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My big question is about the &lt;em&gt;step size selection&lt;/em&gt;: most of ODE solver usually are embedded Runge-Kutta methods and the step-size selection is done (modulus lots of improvements made by MatLab developers) in a way that is presented in classical texts like &lt;strong&gt;&quot;Hairer-Wanner-Norseet (I and II volume)&quot;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can one do, in order to change the step-size, if only trapezoidal rule is used? I'm really curious about that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;Thanks @Chris&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still a doubt&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read the SPICE book (the interesting pages are from 308 to 312), and the algorithm at page 310 is also clear: it uses the number of iteration required to solve the non-linear system in order to change the step-size. &#xA;But...&#xA;as stated above, MatLab estimates the error by differentiating a cubic interpolant. What are the points (and the derivatives) used to build this interpolant?&#xA;Your code seems to use the solution just computed &lt;span class=&quot;math-container&quot;&gt;$u_{n+1}$&lt;/span&gt;, and the &lt;span class=&quot;math-container&quot;&gt;$u_{n}, u_{n-1}$&lt;/span&gt;, but still can't see where are the derivatives.&lt;/p&gt;&#xA;" OwnerUserId="26296" LastEditorUserId="26296" LastEditDate="2018-11-10T10:55:15.767" LastActivityDate="2020-07-19T17:52:52.490" Title="Step-size selection for an Trapezoidal Method ODE solver (ode23t)" Tags="|matlab|ode|stiffness|adaptive-timestepping|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30434" PostTypeId="2" ParentId="30426" CreationDate="2018-10-28T00:01:18.983" Score="2" Body="&lt;ul&gt;&#xA;&lt;li&gt;If you plan on solving &lt;span class=&quot;math-container&quot;&gt;$A_i x = f_i$&lt;/span&gt;, one alternative option is to use a Krylov iterative solver (like GMRES) with &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; as preconditioner and &lt;span class=&quot;math-container&quot;&gt;$B^{-1} f_i$&lt;/span&gt; as starting vector. Then you can prove that all the residuals will belong to the range of &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;. The iterations will be performed in the range of &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; (which reduces the dimension of vectors from &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; down to the rank of &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt;). &lt;/li&gt;&#xA;&lt;li&gt;For a reference with GMRES, you could check Section 4.2 of Heikkola, Rossi, Toivanen, &quot;A parallel fictitious domain method for three-dimensional Helmholtz equation&quot;, SISC, vol. 24, pp. 1567-1588 (2003).&lt;/li&gt;&#xA;&lt;li&gt;Finally, you could recycle the Krylov subspace at every step &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, as long as &lt;span class=&quot;math-container&quot;&gt;$C_{i+1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_{i+1}$&lt;/span&gt; are relatively &quot;close&quot; to &lt;span class=&quot;math-container&quot;&gt;$C_{i}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_i$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2018-10-28T00:01:18.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30435" PostTypeId="1" CreationDate="2018-10-28T06:30:14.817" Score="0" ViewCount="65" Body="&lt;p&gt;I'm extremely new to Autodesk CFD, and I'm working on a project that deals with diodes heating up and cooling down based on a fixed temperature regulated by a temperature switch. Basically, I have a string of 6 diodes soldered together hooked up to a temperature switch, and when the diode temperature reaches 150C, the current stops flowing through the diodes. Another parameter is that this diode chain is immersed in an aluminum bottle of oil, so convection currents come into play. My problem is, I'm trying to model this on Autodesk CFD, and I'm not really sure how to go about modeling something that is fluctuating in temperature, AND has convection currents surrounding it.&lt;/p&gt;&#xA;" OwnerUserId="29085" LastActivityDate="2018-10-31T16:13:42.553" Title="Modeling Diodes in Autodesk CFD" Tags="|fluid-dynamics|heat-transfer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30436" PostTypeId="1" AcceptedAnswerId="30437" CreationDate="2018-10-27T17:30:53.213" Score="2" ViewCount="147" Body="&lt;p&gt;I am trying to numerically solve the following PDE,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial u^A}{\partial t} = c_1\frac{\partial^2 u^A}{\partial^2x} \,,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$c_1$&lt;/span&gt; is a constant.  The above can be discretized using the numerical approximations&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;\frac{\partial u^A}{\partial t} &amp;amp; ~\Rightarrow~ \frac{u_{i}^{t+1} - u_i^t}{\Delta t } \\[5px]&#xA;\frac{\partial^2 u^A}{\partial^2x} &amp;amp;~\Rightarrow~ \frac{u_{i+1}^t -2u_i^t+ u_{i-1}^t}{\Delta x^2 }&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gives,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{u_{i}^{t+1} - u_i^t}{\Delta t} = c_1\frac{u_{i+1}^t -2u_i^t+ u_{i-1}^t}{\Delta x^2 }$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above can be rearranged,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;u_{i}^{t+1} &amp;amp;= u_{i}^t + \frac{c_1 \Delta t}{\Delta x}(u_{i+1}^t -2u_i^t+ u_{i-1}^t) \\[5px]&#xA;u_{i+1}^{t+1} &amp;amp;= u_{i+1}^t + \frac{c_1 \Delta t}{\Delta x}(u_{i+2}^t -2u_{i+1}^t+ u_{i}^t)&#xA;\end{align}&#xA;$$&lt;/span&gt; and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In matrix notation,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$U^{t+1} = [u_{i}^{t+1},u_{i+1}^{t+1}, \, \dots ,u_{i+n}^{t+1}]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;U_A^{t+1} = U_A^{t}   + c_1* \text{tridiagonal matrix} * U_A^t&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above can be solved to obtain &lt;span class=&quot;math-container&quot;&gt;$U_A^{t+1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now when there is another species B, the equation will be &#xA;&lt;span class=&quot;math-container&quot;&gt;$U_B^{t+1} = U_B^{t}   + c_2* \text{tridiagonal matrix} * U_B^t$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am solving for the time evolution of function &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; of species A and B as two separate matrix equations. Could someone suggest if there is a way to formulate this as a single matrix equation? I'm trying to create a diagonal matrix of the constants c and then combine the matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I'm trying an alternate approach. If I were to use the method of lines the equations would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{du^A}{dt} = c_1\frac{u_{i+1}^{t^A} -2u_i^{t^A}+ u_{i-1}^{t^A}}{\Delta x^2 }$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{du^B}{dt} = c_2\frac{u_{i+1}^{t^B} -2u_i^{t^B}+ u_{i-1}^{t^B}}{\Delta x^2 }$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the matrix form,&#xA;&lt;span class=&quot;math-container&quot;&gt;$[\frac{du^A}{dt}  \frac{du^B}{dt}]^T = \frac{1}{\Delta x^2}*diagonalmatrix*[u_{i+1}^{t^A} -2u_i^{t^A}+ u_{i-1}^{t^A} ; u_{i+1}^{t^B} -2u_i^{t^B}+ u_{i-1}^{t^B} ]$&lt;/span&gt; ,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where diagonalmatrix contains the constants &lt;span class=&quot;math-container&quot;&gt;$c_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c_2$&lt;/span&gt; along its diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would this be the right way to proceed? I have the initial condition and boundary conditions.  &lt;/p&gt;&#xA;" OwnerUserId="29087" OwnerDisplayName="Natasha" LastEditorUserId="29087" LastEditDate="2018-10-28T14:29:01.057" LastActivityDate="2018-10-28T14:29:01.057" Title="Numerically solving a partial differential equation" Tags="|fluid-dynamics|computational-physics|differential-equations|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30437" PostTypeId="2" ParentId="30436" CreationDate="2018-10-28T11:39:43.940" Score="2" Body="&lt;p&gt;Assuming that the time-step taken between the two species is the same, you should be able to just simply combine the two species into a single vector:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;U_\text{tot}=\left[\begin{array}{c}u_0^A \\ u_1^A\\ \vdots \\u_{n+1}^A \\ u_0^B \\ u_1^B \\ \vdots \\ u_{n+1}^B \end{array}\right]&#xA;$$&lt;/span&gt;&#xA;And turn the two individual matrix operators into the single matrix,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathsf{X}_\text{tot}=\left(\begin{array}{cc}\mathsf{X}_A &amp;amp; \mathbf 0 \\ \mathbf 0 &amp;amp; \mathsf{X}_B\end{array}\right)&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathsf{X}_A =c_1 * \text{tridiagonal matrix}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf 0$&lt;/span&gt; a matrix of 0's. Thus you still have a tridiagonal problem to solve:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$U^{n+1}_\text{tot}=\mathsf{X}_\text{tot}U^n_\text{tot}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a programming aspect, there probably won't be much of a difference in the timing between doing them separately and together like the above. Maybe a collective few microseconds due to the duplicate calls to the same update function with different vectors and matrices, but you'll probably lose those gains with populating &amp;amp; de-populating &lt;span class=&quot;math-container&quot;&gt;$U_\text{tot}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathsf {X}_\text{tot}$&lt;/span&gt;. Perhaps the combined method would allow you to add in some cross terms (in the operator), so that may be one advantage to running with this method?&lt;/p&gt;&#xA;" OwnerUserId="4762" OwnerDisplayName="Kyle Kanos" LastActivityDate="2018-10-28T11:39:43.940" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30438" PostTypeId="1" AcceptedAnswerId="30446" CreationDate="2018-10-28T14:22:03.880" Score="-1" ViewCount="81" Body="&lt;p&gt;Can anybody explain me the relation between the condition of a Matrix and the convergency of a problem. &#xA;For example how is the relation between the condition of the stiffness Matrix occuring in FEM to the convergency of the FEM method? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you! &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note:  For example I want to solve problem In weak formulation: find &lt;span class=&quot;math-container&quot;&gt;$u\in H^1_{\Gamma_D}(\Omega)$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$f\in L^2(\Omega)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g_D\in H^{\frac{1}{2}}(\Gamma_D)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g_N \in L^2(\Gamma_N)$&lt;/span&gt; such that for all &lt;span class=&quot;math-container&quot;&gt;$v\in H^{1}_0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation*}&#xA; a(u,v) :=  \int\limits_{\Omega} \nabla u A \nabla v  \, d\Omega  = \int\limits_{\Omega} v^T f \, d\Omega + \int\limits_{\Gamma_N} v^T g_N \,d\Gamma:= \ell(v)&#xA;\end{equation*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; a bounded self-adjoint and elliptic linear operator and &lt;span class=&quot;math-container&quot;&gt;$\partial\Omega = \Gamma_D \cup \Gamma_N$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Gamma_D \cap \Gamma_N = \emptyset$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Galerkin approach and defining &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;A:=(a(\varphi_i,\varphi_j))_{i,j = 1}^n,  \quad \quad&#xA;f := (\ell(\varphi_1),\ldots,\ell(\varphi_n))^T, \quad \quad  u :&amp;amp;= (u_1,\ldots,u_n)^T&#xA;\end{align*}&lt;/span&gt;&#xA;the problem can be written as&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}\label{eq:LGS}&#xA; A u = f.&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is what does &lt;span class=&quot;math-container&quot;&gt;$cond(A)$&lt;/span&gt; tells me about the convergency of FEM for this problem?&lt;/p&gt;&#xA;" OwnerDisplayName="user29088" LastEditorDisplayName="user29088" LastEditDate="2018-10-28T17:52:55.143" LastActivityDate="2018-10-29T11:52:32.370" Title="Relation of Condition of a Matrix and Convergency" Tags="|linear-algebra|finite-element|numerics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30439" PostTypeId="1" CreationDate="2018-10-28T16:30:13.157" Score="1" ViewCount="59" Body="&lt;ul&gt;&#xA;&lt;li&gt;I want to uniformly sample a point within each of &lt;span class=&quot;math-container&quot;&gt;$10^5$&lt;/span&gt; convex polytopes in each iteration of a solver.&lt;/li&gt;&#xA;&lt;li&gt;The polytopes in one iteration are completely different from the polytopes in another iteration.&lt;/li&gt;&#xA;&lt;li&gt;The dimension of the space is about 15. &lt;/li&gt;&#xA;&lt;li&gt;The vertices of each polytopes are known.&lt;/li&gt;&#xA;&lt;li&gt;I only want one point per polytope.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hit and run sampling would take quite a lot of steps to converge to a uniform distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Triangulating each polytope to simplex can work. But doing that in high dimension seems tedious (at a quick glance.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other methods to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="28896" LastEditorUserId="28896" LastEditDate="2018-10-29T15:42:11.300" LastActivityDate="2018-10-29T15:42:11.300" Title="Uniformly sample a point per polytope" Tags="|computational-geometry|probability|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30442" PostTypeId="2" ParentId="30431" CreationDate="2018-10-29T02:31:32.507" Score="6" Body="&lt;p&gt;The only documentation I know about for the implementation of &lt;code&gt;ode23t&lt;/code&gt; is in &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/0168927495001158&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper which documents the implementation of ode23tb, the TRBDF2 method in MATLAB&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As usually implemented, the trapezoidal rule is not strictly a one-step&#xA;method because the truncation error estimate makes use of two previously computed solution values. The method is not efficient for very stiff problems because it is not strongly stable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and it cites &lt;a href=&quot;http://s1.nonlinear.ir/epublish/book/The_SPICE_Book_0471609269.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The SPICE book&lt;/a&gt;. Two things here. First of all, the Trapezoidal rule, also known as Crank-Nicholson, is not L-stable and is generally not suited for highly stiff equations (but this lack of dampening can give undesirable outcomes in some models). This is exacerbated by the instability of the error estimate, which is from a truncation error estimate. Basically, you save some previous points and use derivative approximations in order to match the leading truncation error terms, and then use this as the input to standard P or PI adaptive algorithms.&lt;/p&gt;&#xA;&lt;p&gt;For an open source (MIT licensed) implementation of this algorithm, you can take a look at the &lt;code&gt;Trapezoid&lt;/code&gt; method from &lt;a href=&quot;http://diffeq.sciml.ai/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt;. The implementation can be found &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/v4.16.4/src/perform_step/sdirk_perform_step.jl#L208-L244&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the source code here&lt;/a&gt; and it was derived from the description in the SPICE book reference given by Shampine. Note that due to licensing issues this has not been directly checked against the MATLAB source code, only directly against these sources given by Shampine.&lt;/p&gt;&#xA;&lt;h2&gt;Responding to Edits&lt;/h2&gt;&#xA;&lt;p&gt;Continuing the conversation is kind of frowned upon in SO, but I'll do it since it makes sense. For part one, the linked method isn't implicit Euler in the first step. Instead, the adaptivity doesn't exist in the first step since it uses two steps to get a second derivative approximation. However, being a 1-step method, the Trapezoidal rule is still used even when it takes the non-adaptive step. This keeps it always second order (though you can have a single first order step and still be second order).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;But... as stated above, MatLab estimates the error by differentiating a cubic interpolant.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Where is it stated that MATLAB does this? MATLAB uses a cubic interpolant error for &lt;code&gt;dde23&lt;/code&gt;, but I have never seen it stated that it does this on &lt;code&gt;ode23t&lt;/code&gt;. In the comments I show the paper trail starting from MATLAB's documentation to Shampine's paper and to the SPICE book. In the SPICE book they derive the truncation error estimate which uses two steps to estimate the third derivative term which is the leading truncation error term. To the best of my knowledge, and not looking at the MATLAB source since I am an open source developer and do not want licensing issues, that is how it's done according to the sources given. If someone has the time, they can double change and confirm or deny this by looking at the MATLAB source.&lt;/p&gt;&#xA;&lt;p&gt;To answer the question of how you could come up with an error estimate using the interpolant, let me show you that Hermite won't work here. I am just going to point to the live code since I know it's correct and it's easier than TeXing it up. The Hermite interpolant is defined here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/v4.16.4/src/dense/generic_dense.jl#L389&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/v4.16.4/src/dense/generic_dense.jl#L389&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;with (&lt;span class=&quot;math-container&quot;&gt;$y_0$&lt;/span&gt;,k[1]) and (&lt;span class=&quot;math-container&quot;&gt;$y_1$&lt;/span&gt;,k[2]) as the values at t and t+dt respectively, calculating at the point t+&lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;dt. To get the derivative of the interpolant, you differentiate by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and get&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/v4.16.4/src/dense/generic_dense.jl#L418&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/v4.16.4/src/dense/generic_dense.jl#L418&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Notice that when &lt;span class=&quot;math-container&quot;&gt;$\theta=1$&lt;/span&gt;, the derivative approximated by the cubic Hermite polynomial is &lt;span class=&quot;math-container&quot;&gt;$k[2]$&lt;/span&gt;, which is the derivative estimate at &lt;span class=&quot;math-container&quot;&gt;$t+dt$&lt;/span&gt; calculated from &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; itself. So it doesn't have a natural error estimate at the end points of the interval against the derivative since it's made to perfectly hit those derivative values. When this estimate is used it's called a residual correction method, &lt;a href=&quot;http://faculty.smu.edu/shampine/residuals.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;discussed here&lt;/a&gt;, where this derivative estimate is used against extra &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; evaluations in the middle of the interval to give a continuous derivative error estimate. By using specific points you can estimate the maximum difference, and this done on RK4 is how &lt;code&gt;ddesd&lt;/code&gt; works. RK4 is simple, and an open source implementation of the two-point maximal residual error estimate for it &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/perform_step/fixed_timestep_perform_step.jl#L244-L258&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be found here&lt;/a&gt;. But while MATLAB does use this in a different integrator, Shampine writes about it, and you technically can apply this to the Trapezoid method, from what I can tell this is not what's done in &lt;code&gt;ode23t&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T17:52:52.490" LastActivityDate="2020-07-19T17:52:52.490" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30443" PostTypeId="1" CreationDate="2018-10-29T09:53:50.300" Score="2" ViewCount="43" Body="&lt;p&gt;I have implemented a small compressible Euler solver, discretizing in primitive variables (rho, u, v, p) with standard Galerkin FEM P1 triangular elements, and mixed isotropic and anisotropic/streamline diffusion for stabilization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have verified the solver on several test cases with shock tube, double shocks, and flow over bumps, so it seems to work fine except for some oscillations (as the artificial diffusion isn't shock capturing or monoticity preserving).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I'm trying to validate a simple 2D static oblique shock test case for a unit square which should be simple and has an analytical solution. With top/left boundary conditions Ma_in = 2, rho_in = 1, u_in = cos(10 deg), v_in = sin(10 deg), p_in = 0.1786, bottom slip (v = 0), the analytic solution should be a shock at 29.3 degrees and Ma = 1.64 at the outlet (right boundary). My solver converges just fine and resolves the shock, however the solution is inverted after the shock, Ma 2.4, density 0.65 instead of 1.46 etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot seem to find the error, especially as the dual shock problem solves fine. Could this be to using non-conservative/primitive variables? Any ideas at all would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="29094" LastActivityDate="2018-10-29T09:53:50.300" Title="Solving Compressible Euler in Primitive Variables with Galerkin P1 FEM" Tags="|finite-element|galerkin|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30445" PostTypeId="1" CreationDate="2018-10-29T11:06:23.750" Score="5" ViewCount="206" Body="&lt;p&gt;I would like to numerically solve the following system of coupled nonlinear differential equations: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  -\frac{\hbar^2}{2m_a} \frac{\partial^2}{\partial x^2}\psi_a + V_{ext}\psi_a  + &#xA;  \left( g_a |\psi_a|^2 + g_{ab} |\psi_b|^2 \right)\psi_a = \mu_a \psi_a&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;   -\frac{\hbar^2}{2m_b} \frac{\partial^2}{\partial x^2}\psi_b + V_{ext}\psi_b  &#xA;   + \left( g_b |\psi_b|^2 + g_{ab} |\psi_a|^2 \right)\psi_b=\mu_b\psi_b&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\hbar$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m_a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m_b$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g_a$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g_b$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g_{ab}$&lt;/span&gt; are known coefficients and &lt;span class=&quot;math-container&quot;&gt;$V_{ext}$&lt;/span&gt; is a known function of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, i.e.:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;   V_{ext}= -P \left[\cos\left(\frac{3}{2}\, \frac{x}{L}\, 2\pi \right)\right]^2&#xA;$$&lt;/span&gt; &#xA;The unknowns are eigenfunctions &lt;span class=&quot;math-container&quot;&gt;$\psi_a(x)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\psi_b(x)$&lt;/span&gt; and the eigenvalues &lt;span class=&quot;math-container&quot;&gt;$\mu_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mu_b$&lt;/span&gt;. Both &lt;span class=&quot;math-container&quot;&gt;$V_{ext}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\psi_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\psi_b$&lt;/span&gt; are defined on the domain &lt;span class=&quot;math-container&quot;&gt;$x\in[0,L]$&lt;/span&gt;. Functions &lt;span class=&quot;math-container&quot;&gt;$\psi_a(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\psi_b(x)$&lt;/span&gt; are complex functions. The boundary conditions are the periodic ones, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \psi_a(x+L)=\psi_a(x) \qquad  \psi_b(x+L)=\psi_b(x)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice that the period of the eigenfunctions should be &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; while the period of the external potential is &lt;span class=&quot;math-container&quot;&gt;$L/3$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eventually, there is a constraint on the norms of &lt;span class=&quot;math-container&quot;&gt;$\psi_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\psi_b$&lt;/span&gt;, namely:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;   \int_0^L |\psi_a|^2 \, \mathrm{d}x= N, \qquad  \int_0^L |\psi_b|^2 \, \mathrm{d}x= M&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please give me a good strategy to handle this problem and, possibly, a Matlab code?&lt;/p&gt;&#xA;" OwnerUserId="29095" LastEditorUserId="29095" LastEditDate="2018-11-04T09:45:13.807" LastActivityDate="2018-12-08T12:55:49.987" Title="Numerical solution of two coupled nonlinear eigenvalue problems" Tags="|ode|eigenvalues|differential-equations|numerics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30446" PostTypeId="2" ParentId="30438" CreationDate="2018-10-29T11:52:32.370" Score="1" Body="&lt;p&gt;I suppose there is an misunderstanding about the convergence, in my opinion it is about the convergence of the linear solver used to solve the linear system &lt;span class=&quot;math-container&quot;&gt;$Au=f$&lt;/span&gt; and no the convergence of the method FEM itself. I try to explain.&lt;/p&gt;&#xA;&lt;h1&gt;Galeriking - FEM convergence&lt;/h1&gt;&#xA;&lt;p&gt;Your objective if to find the unique &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; s.t.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tag{1}&#xA;u \in W \quad A(u,v) = l(v) \quad \forall v \in V &#xA;$$&lt;/span&gt;&#xA;where:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a bi-linear for associated to the differential operator&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; a linear functional in the dual&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$W,V$&lt;/span&gt; opportune functional spaces&#xA;I only rewrite your problem in a general form, but we are talking about the same things.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now find the exact solution for the problem &lt;span class=&quot;math-container&quot;&gt;$(1)$&lt;/span&gt; is difficult, also because &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; is an space infinite dimensional. So you approx it with a subspace finite dimensional.&lt;/p&gt;&#xA;&lt;p&gt;You go on with keep &lt;span class=&quot;math-container&quot;&gt;$W=V$&lt;/span&gt; and approximate &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; with the family &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; s.t.:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\forall v \in V \; \inf_{v_h \in V_h} ||v - v_h|| \longrightarrow 0 \; \text{for}&#xA;\; h \longrightarrow 0&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that the idea is &lt;em&gt;approximate the solution space&lt;/em&gt; and no the operator essentially this is the Galerking method.&lt;/p&gt;&#xA;&lt;p&gt;Now the problem is find:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tag{2}&#xA;u_h \in V_h \quad:\quad A(u_h,v_h) =l(v_h) \; \forall v_h \in V_h&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;h is the discretization index i.e. the grid step. More &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is small more &lt;span class=&quot;math-container&quot;&gt;$V_h$&lt;/span&gt; is big so you have got a best approximation.&lt;/p&gt;&#xA;&lt;p&gt;The motivation about the Galerking method works is essentially based on the &lt;a href=&quot;https://en.wikipedia.org/wiki/C%C3%A9a%27s_lemma&quot; rel=&quot;nofollow noreferrer&quot;&gt;Céa's lemma&lt;/a&gt;, it give us this estimate:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;||u -u_h||_V \leq c \inf_{v \in V_h}||u - v||_V&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the wiki page there is an example of estimate with the approx by piecewise-linear function.&lt;/p&gt;&#xA;&lt;p&gt;The choice of the base for the space is important, in the early time before computers often were used global or trigonometric polynomial.&lt;/p&gt;&#xA;&lt;p&gt;When you choose piecewise polynomial, i.e simple polinomial with compact support, you obtain FEM method. For example it has got the advantage that the stiff matrix is sparse.&lt;/p&gt;&#xA;&lt;p&gt;As you can see form the Céa's lemma and this estimates:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;||u - u_h||_{0,\infty} \leq C h^{l+1} |u|_{l+1, \infty} \; \forall u \in W^{l+1, \infty}&#xA;$$&lt;/span&gt;&#xA;, here &lt;span class=&quot;math-container&quot;&gt;$W^{l+1, \infty}$&lt;/span&gt; is a Sobolev's space, the convergence of the FEM method is related to the step &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;. For this I think that your question is about the second part.&lt;/p&gt;&#xA;&lt;h1&gt;Linear solver convergence&lt;/h1&gt;&#xA;&lt;p&gt;If you have got:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;data with infinite precision&lt;/li&gt;&#xA;&lt;li&gt;solve the linear system with infinite precision. For example  by hand gauss elimination with infinite precision, or conjugate gradient (CG) that with infinite precision is an exact solver and converge after &lt;span class=&quot;math-container&quot;&gt;$n = dim(A)$&lt;/span&gt; iteration&#xA;you are immune about &lt;span class=&quot;math-container&quot;&gt;$cond(A)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;But as the Spartans said, &lt;strong&gt;if&lt;/strong&gt;...&#xA;For point one you can not meet this requirement.&#xA;For point 2, it is impractical, to much time by hand and also if you want use CG and wait the final step the dimension of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; give you problem.&lt;/p&gt;&#xA;&lt;p&gt;For this is important to consider &lt;span class=&quot;math-container&quot;&gt;$cond(A)$&lt;/span&gt; because influence your convergence.&lt;/p&gt;&#xA;&lt;p&gt;For example, only to show a case, if you consider &lt;a href=&quot;https://en.wikipedia.org/wiki/Conjugate_gradient_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;CG&lt;/a&gt; that is often used yo have got this estimate:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{e_n}{e_0} = \frac{||x - x_n||_A}{||x -x_0||_A} &#xA;&amp;lt; 2 \left ( \frac{\sqrt{K_A} - 1}{\sqrt{K_A} + 1} \right )^n \;,&#xA;\quad K_A = \frac{\lambda_{max}}{\lambda_{min}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;as you can see &lt;span class=&quot;math-container&quot;&gt;$cond(A) = K_A$&lt;/span&gt; influence the convergence. Note this is a generic estimate for CG for a generic matrix.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-10-29T11:52:32.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30447" PostTypeId="1" AcceptedAnswerId="30448" CreationDate="2018-10-30T02:44:40.780" Score="5" ViewCount="899" Body="&lt;p&gt;Is there any software available which can directly render higher order finite element results? In particular, 3D finite elements would be preferable. It seems gmsh has some capability in this regard but am looking to explore more options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, when I say &quot;higher order&quot; I mean rendering of quadratic and above elements. Tools like Paraview and VisIt seem to only render linear elements, and require mesh refinement and subsequent interpolation to a linear space to see the higher order solution, from what I understand.&lt;/p&gt;&#xA;" OwnerUserId="29000" LastActivityDate="2018-10-30T20:03:03.950" Title="Recommended visualization tools for higher order finite element solutions?" Tags="|finite-element|visualization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30448" PostTypeId="2" ParentId="30447" CreationDate="2018-10-30T03:07:45.873" Score="4" Body="&lt;p&gt;New versions of VTK support what you are asking for, and new versions of Paraview can also display this. The current version of Visit can not yet, though, at the time of writing this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trick is figuring out the specifics of how this should be described in the VTK file format. You can take a look at these two links to see the discussion and implementation of this feature in deal.II, along with some example images:&#xA;&lt;a href=&quot;https://github.com/dealii/dealii/issues/6931&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dealii/dealii/issues/6931&lt;/a&gt;&#xA;&lt;a href=&quot;https://github.com/dealii/dealii/pull/6994&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dealii/dealii/pull/6994&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Disclaimer: I'm one of the principal developers of deal.II.)&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2018-10-30T12:49:18.877" LastActivityDate="2018-10-30T12:49:18.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30450" PostTypeId="2" ParentId="30321" CreationDate="2018-10-30T10:39:37.063" Score="1" Body="&lt;p&gt;I'm not sure but the problem behind the contact: in Abaqus if the friction is bigger than 0.2 the solver using an unsymmetric matrix storage, however in the input file probably symmetric. So was overridden by the solver. But I don't know how is it working in calculix. But the input file should modify: UNSYMM=YES&#xA;Check this:&lt;a href=&quot;https://www.sharcnet.ca/Software/Abaqus610/Documentation/docs/v6.10/books/rnb/default.htm?startat=abc06aqs05.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; it might be helpful. Let me know does it help.&lt;/p&gt;&#xA;" OwnerUserId="29100" LastEditorUserId="29100" LastEditDate="2018-10-30T11:16:59.943" LastActivityDate="2018-10-30T11:16:59.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30453" PostTypeId="2" ParentId="30447" CreationDate="2018-10-30T20:03:03.950" Score="2" Body="&lt;p&gt;MFEM &lt;a href=&quot;http://mfem.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mfem.org&lt;/a&gt; and the associated visualisation tool supports several types of high order elements.&lt;/p&gt;&#xA;" OwnerUserId="16910" LastActivityDate="2018-10-30T20:03:03.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30455" PostTypeId="2" ParentId="1098" CreationDate="2018-10-30T23:23:46.563" Score="0" Body="&lt;p&gt;I use the command ‘VecGather’ to gather all the vectors to the root node, then perform the output on the root node. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the other way is to write every local result by every node in the cluster into the NFS share system. Some visualization software such as Visit can support the parallel file format. &lt;/p&gt;&#xA;" OwnerUserId="22404" LastEditorUserId="20688" LastEditDate="2018-10-30T23:32:05.650" LastActivityDate="2018-10-30T23:32:05.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30456" PostTypeId="2" ParentId="453" CreationDate="2018-10-30T23:26:08.263" Score="1" Body="&lt;p&gt;In the source directory, there is a log file to record every output of the screen including the commands has been input.&lt;/p&gt;&#xA;" OwnerUserId="22404" LastActivityDate="2018-10-30T23:26:08.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30460" PostTypeId="2" ParentId="30435" CreationDate="2018-10-31T16:13:42.553" Score="1" Body="&lt;p&gt;The natural convection of the oil shouldn't be an issue and can be solved without any 'assumptions' to be made.  The challenge comes with the diodes temperature control. CFD doesn't currently have a temperature controlled heat flux, so some assumptions will have to be made. The simplest assumption would be to just assign a temperature of 150C and assume it is always running at the hottest.  This would assume that the diodes do not cool down much before current is passed back through again.  If we assume this fixed temperature then the analysis is a simple steady state and the setup can follow the guide in the help:&#xA;&lt;a href=&quot;http://help.autodesk.com/view/SCDSE/2019/ENU/?guid=GUID-A7E7C4B9-701B-4A4E-9F98-2E7C0BD1CB31&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://help.autodesk.com/view/SCDSE/2019/ENU/?guid=GUID-A7E7C4B9-701B-4A4E-9F98-2E7C0BD1CB31&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can get more complicated than that (transient, fixed volume of oil), but I would recommend taking your question to the Autodesk CFD forums.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://forums.autodesk.com/t5/cfd-forum/bd-p/330&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://forums.autodesk.com/t5/cfd-forum/bd-p/330&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29108" LastActivityDate="2018-10-31T16:13:42.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30465" PostTypeId="2" ParentId="30294" CreationDate="2018-11-02T00:04:58.270" Score="1" Body="&lt;p&gt;I have work in the FVM for many years. The second order FVM is very popular in the CFD area. The integration is approximated with the mid-point integration method. It means that the curved edge should be approximated with many line segments. If you want to work with the high-order FVM method, I suggest you try the spectral-FVM method. &lt;/p&gt;&#xA;" OwnerUserId="22404" LastActivityDate="2018-11-02T00:04:58.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30466" PostTypeId="1" CreationDate="2018-11-02T15:31:45.907" Score="1" ViewCount="579" Body="&lt;p&gt;I would like to extremize a certain function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with respect to a parameter &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, under constraints &lt;span class=&quot;math-container&quot;&gt;$g_1(x) = 0, ..., g_m(x)=0$&lt;/span&gt;. In order to achieve this, I consider the Lagrangian function &lt;span class=&quot;math-container&quot;&gt;$L(x, \lambda) = f(x) - \sum_{i=1}^m\lambda_i g_i(x)$&lt;/span&gt; and look for a stationary point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my particular case, this amounts to looking for a saddle point, which is not convenient for me because I have to perform several extremizations before I find a satisfactory point (for example by using Uzawa's algorithm). I'd rather have to look for a minimum or a maximum, that I can reach with a single gradient descent/BFGS run/whatever.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I was wondering: is there any way I can &quot;rephrase&quot; my optimization problem so that the stationary point of the Lagrangian is an extremum, or are there some cases where a saddle point is the best I can hope for?&lt;/p&gt;&#xA;" OwnerUserId="29123" LastActivityDate="2018-11-02T21:16:01.280" Title="Nature of stationary points of a Lagrangian fuction" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30467" PostTypeId="2" ParentId="30466" CreationDate="2018-11-02T21:16:01.280" Score="4" Body="&lt;p&gt;By the very definition of the Lagrangian, the extrema of the original, constrained problem are stationary points of the Lagrangian. That's just how the Lagrangian is defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are looking for are &quot;penalty methods&quot; or &quot;augmented Lagrangian methods&quot;. In essence, it adds something to either the objective function or to the Lagrangian to make sure that the solutions of the problem are really the &lt;em&gt;minima&lt;/em&gt; of the augmented function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As with all things optimization, I've found the book by Nocedal and Wright (&quot;Numerical Optimization&quot;) to be an excellent source of insight. It has long sections on both penalty methods and augmented Lagrangian methods, and how to choose the term(s) you will want to add.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-11-02T21:16:01.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30471" PostTypeId="2" ParentId="30422" CreationDate="2018-11-03T16:14:22.733" Score="1" Body="&lt;p&gt;Based on the problem description, and under the assumption you do not know which point in &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; some point in &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; should be best matched up with when computing errors, I am tempted to say you might be able to pose your optimization problem in a general sense as something along the lines of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\min_{R, T} \sum_{p \in P} \min_{q \in Q} \left \lVert R p + T - q\right \rVert^2&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to minimize the sum of nearest neighbor distances between each affine mapped vector &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and the points within the set &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;. This looks a bit nasty at first glance due to the minimum in the sum, but from here, we might approximate this objective function by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\min_{R, T} \; \;-\sum_{p \in P} \epsilon \log\left( \sum_{q \in Q} \exp \left(\frac{-\left \lVert R p + T - q\right \rVert^2 }{\epsilon}\right) \right)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for some &lt;span class=&quot;math-container&quot;&gt;$\epsilon &amp;gt; 0$&lt;/span&gt; that is close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;. Note that this approximation is based on the fact that &lt;span class=&quot;math-container&quot;&gt;$\lim_{\epsilon \rightarrow 0} \epsilon \log\left(\sum_{i=1}^n \exp\left(x_i/\epsilon\right)\right) = \max_{i} x_i$&lt;/span&gt;. Assuming you choose some value for &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, like say &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 10^{-3}$&lt;/span&gt;, you should be able to compute the gradients with respect to &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; of this objective function above and solve it in an unconstrained manner. The benefit to the construction above is it makes no assumption on the size of either set of points.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2018-11-03T16:34:26.890" LastActivityDate="2018-11-03T16:34:26.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30472" PostTypeId="1" CreationDate="2018-11-03T17:44:43.143" Score="1" ViewCount="112" Body="&lt;p&gt;I am attempting to solve for a displacement field where I know the thermal stresses on my discretized domain, which consists of hex cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first question is: is easier way to solve for the displacements besides integrating the linear elasticity equations:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla \cdot \sigma = 0&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; is stress (contains both the mechanical and thermal stresses). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently am using the finite volume method to solve the above equation, and the integral form is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int \int (\sigma \cdot \hat{n}) dA = 0 \ \ \ \ \text{   applied divergence theorem}\\&#xA;\int \int (\sigma \cdot \hat{n}) dA \approx \sum_{f=1}^N \sigma_fA_f\hat{n}_f&#xA;$$&lt;/span&gt;&#xA;where the integral is approximated using the midpoint rule that reads as the sum of the product of the face stress, face area, and face normal over all faces of a cell. The face stress is computed by taking an average of the neighboring cell's stress. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The stress uses the duhamel-neumann relationship for an anisotropic material (note  that I am currently testing the code on a simple isotropic material)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sigma = C:(\epsilon_m - \epsilon_t) &#xA;$$&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$\epsilon_m$&lt;/span&gt; is the mechanical strain that is computed using the linear strain displacement relationship and &lt;span class=&quot;math-container&quot;&gt;$\epsilon_t$&lt;/span&gt; is the thermal strain, computed using&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\epsilon_t = \alpha (T-T_0)&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is the CTE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can formulate the above system of equations as a linear system &lt;span class=&quot;math-container&quot;&gt;$Au=b$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; are the displacements to be solved for. &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the forcing and essentially all the thermal stress term is contained inside &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; (thermal stress has no impact on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The good news is that if I assign the same thermal stress to the entire domain (i.e., same temperature at all the cells), then I get a solution that is &quot;correct.&quot; By correct, I mean it matches commercial software (ANSYS). So something is correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if my temperature distribution across the domain is not the same, then I get solutions that not only do not come close to matching ANSYS, but also do not match my intuition (certain cells with larger temperature changes are expanding less or even contracting(!), as indicated by the computed displacement field, than parts with smaller or no temperature changes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have checked my code and it seems I am implementing everything correctly, so I am wondering if my numerics are incorrect. But I am getting the correct solution if the whole field is initialized to the same temperature, so something must be correct. e.g., does adding the thermal stress term somehow affect how I should approach the discretization? I would think not. This term is easy and it's really just a forcing term..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As previously mentioned, the thermal stress is manifested mathematically in the forcing vector &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; and has no impact on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. So changing the temperature just affects the magnitude of my forcing vector, and my &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; remains the same. So this tells me that my &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; matrix is formulated correctly, or that it could be formulated incorrectly, but in a way that has not affected my solutions for other problems. I think the latter is very unlikely because I have also tested my code on cantilever and various other bending/compression problems (with NO thermal stress), and I get solutions that match well with analytical solutions or with commercial software. I have also done some manufactured solutions studies to look at convergence and the tests succeeded. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's just this thermal stress term that's causing me issues, and only when the temperature is not uniform across the discretized domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions? I'm happy to provide more info if needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit #2: &#xA;Here is the particular geometry I am working with right now:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wOls2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wOls2.png&quot; alt=&quot;Geometry Mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The length is in the x direction. I apply a uniform temperature of &lt;span class=&quot;math-container&quot;&gt;$1900K$&lt;/span&gt; to all the cells. The following picture is the displacement contours for the y-direction, which is the more refined direction in the above picture. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/PmsPm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PmsPm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The max y-displacement from the contours for the +y and -y surfaces are about 0.0006m and -0.0006m. This material is steel, using CTE =  &lt;span class=&quot;math-container&quot;&gt;$1.2\times 10^{-5} K^{-1}$&lt;/span&gt;. The dimension of the block is about 0.6098m by 0.0047625m by 0.00508m (sorry multiple decimals. The original grid was created in inches) in the x,y,z directions. The mesh step size in the 3 directions is constant, at 0.03048m (1.2inches) by 0.004318m (0.17 inches) by 0.0254m (1 inch). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here I get a solution that matches what ANSYS gives me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next I proceeded to run a different simulation where I only specify a temperature of 1900K in the domain in the top two layers (see the region in the white boxes below) and the rest of the domain is at 298K. &lt;a href=&quot;https://i.stack.imgur.com/XilQj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XilQj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this simulation where only part of the domain experiences a temperature change, I get a solution that is way off from expected or from what ANSYS gives me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit #3: Adding displacement contours for the aforementioned test case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the 3 displacement contours for the x-,y-, and z-displacements for the second case. My intuition tells me that the x- and z- displacement contours show trends that are expected (not sure if the magnitudes are correct, but at least the trends are reasonable). But I would not expect the y-displacement to behave the way it is shown in the y-displacement contours. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hl1aB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hl1aB.png&quot; alt=&quot;Z-displacement contour&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W1RRw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W1RRw.png&quot; alt=&quot;X-displacement contours&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NB7K2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NB7K2.png&quot; alt=&quot;Y-Displacement contours&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24311" LastEditorUserId="24311" LastEditDate="2018-11-03T21:07:07.300" LastActivityDate="2018-11-03T21:07:07.300" Title="Thermal stress to displacement through finite volume method" Tags="|finite-volume|solid-mechanics|linear-system|" AnswerCount="0" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="30474" PostTypeId="2" ParentId="28489" CreationDate="2018-11-03T23:44:15.597" Score="5" Body="&lt;h3&gt;Short version&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In scientific computing, the notion of &lt;em&gt;relative error&lt;/em&gt; is way more popular than &lt;em&gt;accuracy to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; digits&lt;/em&gt;. Whenever we present the results, we usually plot the obtained (scaled) data and relative/absolute error in multiple different ways. Reporting the number of correct obtained digits (in addition to the relative error) is usually limited to the following scenarios:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In a table form, for the column &quot;number of digits&quot;.&lt;/li&gt;&#xA;&lt;li&gt;In a sentence of the type: &quot;...this method allowed to obtain a &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;-digit accurate solution...&quot;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For both cases, I can hardly imagine the value of providing a counter-intuitive, non-integer value.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Moreover (since we usually prefer to think in base 10), one finds the quantity &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;D = -\log_{10}\left(\frac{|x-y|}{|x|}\right)=-\log_{10}(\eta)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the test value, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the reference value, and &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt; is the relative error between the two. Without loss of generality, let's not touch cases when &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$xy&amp;lt;0$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; have different signs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, knowing &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; &lt;strong&gt;does not give me an immediate answer&lt;/strong&gt; to the question of &lt;em&gt;how many correct digits &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; has&lt;/em&gt;. Simply, because &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is not an integer, and the answer to this question (usually) &lt;strong&gt;implies&lt;/strong&gt; an integer answer. &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; can tell me that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; has at least &lt;span class=&quot;math-container&quot;&gt;$\lfloor D \rfloor$&lt;/span&gt; and at most &lt;span class=&quot;math-container&quot;&gt;$\lceil D\rceil$&lt;/span&gt; correct digits, reducing the uncertainty to &lt;strong&gt;two&lt;/strong&gt; possible answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more elaborate derivation &amp;amp; discussion on this part and how this is justified, see &lt;a href=&quot;https://math.stackexchange.com/a/626472/332683&quot;&gt;this question on Math SE&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to decide &lt;strong&gt;solely&lt;/strong&gt; on &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$\eta$&lt;/span&gt;), how many digits are correct? No.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comments provide counterexamples (explained by the above paragraph):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x_1 = 1.004,\quad y_1 = 1.001,\quad \eta_1 \approx 2.988\cdot10^{-3},\quad D_1 = 2.5246...\\&#xA;x_2 = 3.012,\quad y_2 = 3.003,\quad \eta_2 \approx 2.988\cdot10^{-3},\quad D_2 = 2.5246...&#xA;$$&lt;/span&gt;&#xA;Here, &lt;span class=&quot;math-container&quot;&gt;$D_1=D_2$&lt;/span&gt;, but &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In case of &lt;span class=&quot;math-container&quot;&gt;$(x_1,y_1)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$D_1$&lt;/span&gt; has to be ceiled to get the correct answer of 3-digit match&lt;/li&gt;&#xA;&lt;li&gt;In case of &lt;span class=&quot;math-container&quot;&gt;$(x_2,y_2)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$D_2$&lt;/span&gt; has to be floored to get the correct answer of 2-digit match&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, the definition of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; in &lt;em&gt;extended to a non-integer number of digits&lt;/em&gt; sense is counterintuitive and hard to rely upon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to always be correct one may answer that:&#xA;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is &lt;strong&gt;at least&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$\lfloor D \rfloor$&lt;/span&gt;-digit accurate wrt &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. That implies the following conclusion: if one has a method that provides a &lt;span class=&quot;math-container&quot;&gt;$10^{-3}$&lt;/span&gt;-level relative error, that leaves us with a 2-digit accurate solution. Even though, in some cases, it might be 3-digit accurate.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-03T23:44:15.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30476" PostTypeId="1" AcceptedAnswerId="30482" CreationDate="2018-11-05T02:27:00.850" Score="1" ViewCount="4969" Body="&lt;p&gt;I'm trying to understand the Schrödinger equation and solving it a bit better, and I'm running into some doubts while coding, even though I am adapting the code to this situation. Also I tried asking this on the physics stack exchange and I was directed here.&#xA;this is the situation I am trying to solve:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Pr106.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Pr106.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;It's supposed to be an infinite well with a barrier in the middle. alpha b is just b multiplied by some constant. I have a running code, but since I am basing it off of &lt;a href=&quot;https://helentronica.com/2014/09/04/quantum-mechanics-with-the-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;someone else's code&lt;/a&gt; there's one setting I don't seem to understand (probably some basic concept). I'm having trouble defining the Wave function initial states so how can I define the initial state of psi?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm adding the code in case if anyone wants to play with this code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from pylab import *&#xA;from scipy.integrate import odeint&#xA;from scipy.optimize import brentq&#xA;&#xA;a=1&#xA;B=4&#xA;L= B+a&#xA;Vmax= 50&#xA;Vpot = False&#xA;N = 1000                  # number of points to take&#xA;psi = np.zeros([N,2])     # Wave function values and its derivative (psi and psi')&#xA;psi0 = array([0,1])   # Wave function initial states&#xA;Vo = 50&#xA;E = 0.0                   # global variable Energy  needed for Sch.Eq, changed in function &quot;Wave function&quot;&#xA;b = L                     # point outside of well where we need to check if the function diverges &#xA;x = linspace(-B-a, L, N)    # x-axis&#xA;&#xA;def V(x):&#xA;    '''&#xA;    #Potential function in the finite square well.&#xA;    '''&#xA;    if -a &amp;lt;=x &amp;lt;=a:&#xA;        val = Vo&#xA;    elif x&amp;lt;=-a-B:&#xA;        val = Vmax&#xA;    elif x&amp;gt;=L:&#xA;        val = Vmax&#xA;    else:&#xA;        val = 0&#xA;    if Vpot==True:&#xA;          if -a-B-(10/N) &amp;lt; x &amp;lt;= L+(1/N):&#xA;             Ypotential.append(val)&#xA;             Xpotential.append(x)&#xA;    return val&#xA;&#xA;def SE(psi, x):&#xA;    &quot;&quot;&quot;&#xA;    Returns derivatives for the 1D schrodinger eq.&#xA;    Requires global value E to be set somewhere. State0 is first derivative of the&#xA;    wave function psi, and state1 is its second derivative.&#xA;    &quot;&quot;&quot;&#xA;    state0 = psi[1]&#xA;    state1 = 2.0*(V(x) - E)*psi[0]&#xA;    return array([state0, state1])&#xA;&#xA;def Wave_function(energy):&#xA;    &quot;&quot;&quot;&#xA;    Calculates wave function psi for the given value&#xA;    of energy E and returns value at point b&#xA;    &quot;&quot;&quot;&#xA;    global psi&#xA;    global E&#xA;    E = energy&#xA;    psi = odeint(SE, psi0, x)&#xA;    return psi[-1,0]&#xA;&#xA;def find_all_zeroes(x,y):&#xA;    &quot;&quot;&quot;&#xA;    Gives all zeroes in y = Psi(x)&#xA;    &quot;&quot;&quot;&#xA;    all_zeroes = []&#xA;    s = sign(y)&#xA;    for i in range(len(y)-1):&#xA;        if s[i]+s[i+1] == 0:&#xA;            zero = brentq(Wave_function, x[i], x[i+1])&#xA;            all_zeroes.append(zero)&#xA;    return all_zeroes&#xA;&#xA;def find_analytic_energies(en):&#xA;    &quot;&quot;&quot;&#xA;    Calculates Energy values for the finite square well using analytical&#xA;    model (Griffiths, Introduction to Quantum Mechanics, 1st edition, page 62.)&#xA;    &quot;&quot;&quot;&#xA;    z = sqrt(2*en)&#xA;    z0 = sqrt(2*Vo)&#xA;    z_zeroes = []&#xA;    f_sym = lambda z: tan(z)-sqrt((z0/z)**2-1)      # Formula 2.138, symmetrical case&#xA;    f_asym = lambda z: -1/tan(z)-sqrt((z0/z)**2-1)  # Formula 2.138, antisymmetrical case&#xA;&#xA;    # first find the zeroes for the symmetrical case&#xA;    s = sign(f_sym(z))&#xA;    for i in range(len(s)-1):   # find zeroes of this crazy function&#xA;       if s[i]+s[i+1] == 0:&#xA;           zero = brentq(f_sym, z[i], z[i+1])&#xA;           z_zeroes.append(zero)&#xA;    print (&quot;Energies from the analyitical model are: &quot;)&#xA;    print (&quot;Symmetrical case)&quot;)&#xA;    for i in range(0, len(z_zeroes),2):   # discard z=(2n-1)pi/2 solutions cause that's where tan(z) is discontinous&#xA;        print (&quot;%.4f&quot; %(z_zeroes[i]**2/2))&#xA;    # Now for the asymmetrical&#xA;    z_zeroes = []&#xA;    s = sign(f_asym(z))&#xA;    for i in range(len(s)-1):   # find zeroes of this crazy function&#xA;       if s[i]+s[i+1] == 0:&#xA;           zero = brentq(f_asym, z[i], z[i+1])&#xA;           z_zeroes.append(zero)&#xA;    print (&quot;(Antisymmetrical case)&quot;)&#xA;    for i in range(0, len(z_zeroes),2):   # discard z=npi solutions cause that's where ctg(z) is discontinous&#xA;        print (&quot;%.4f&quot; %(z_zeroes[i]**2/2))&#xA;&#xA;&#xA;def main():&#xA;    # main program        &#xA;&#xA;    en = linspace(0, Vo, 1000)   # vector of energies where we look for the stable states&#xA;&#xA;    psi_b = []      # vector of wave function at x = b for all of the energies in en&#xA;    for e1 in en:&#xA;        psi_b.append(Wave_function(e1))     # for each energy e1 find the the psi(x) at x = b&#xA;    E_zeroes = find_all_zeroes(en, psi_b)   # now find the energies where psi(b) = 0 &#xA;&#xA;    # Print energies for the bound states&#xA;    print (&quot;Energies for the bound states are: &quot;)&#xA;    for E in E_zeroes:&#xA;        print (&quot;%.2f&quot; %E)&#xA;    # Print energies of each bound state from the analytical model&#xA;    find_analytic_energies(en)   &#xA;&#xA;    # Plot wave function values at b vs energy vector&#xA;    figure()&#xA;    plot(en/Vo,psi_b)&#xA;    title('Values of the &lt;span class=&quot;math-container&quot;&gt;$\Psi(b)$&lt;/span&gt; vs. Energy')&#xA;    xlabel('Energy, &lt;span class=&quot;math-container&quot;&gt;$E/V_0$&lt;/span&gt;')&#xA;    ylabel('&lt;span class=&quot;math-container&quot;&gt;$\Psi(x = b)$&lt;/span&gt;', rotation='horizontal')&#xA;    for E in E_zeroes:&#xA;        plot(E/Vo, [0], 'go')&#xA;        annotate(&quot;E = %.2f&quot;%E, xy = (E/Vo, 0), xytext=(E/Vo, 30))&#xA;    grid()&#xA;&#xA;    # Plot the wavefunctions for first 4 eigenstates&#xA;    figure(2)&#xA;    for E in E_zeroes[0:4]:&#xA;        Wave_function(E)&#xA;        plot(x, psi[:,0], label=&quot;E = %.2f&quot;%E)&#xA;    legend(loc=&quot;upper right&quot;)&#xA;    title('Wave function')&#xA;    xlabel('x, &lt;span class=&quot;math-container&quot;&gt;$x/L$&lt;/span&gt;')&#xA;    ylabel('&lt;span class=&quot;math-container&quot;&gt;$\Psi(x)$&lt;/span&gt;', rotation='horizontal', fontsize = 15)&#xA;    grid()&#xA;&#xA;    figure(3)&#xA;    pot =[]&#xA;    for i in x:&#xA;        pot.append(V(i))&#xA;    plot(x,pot)&#xA;    show()&#xA;if __name__ == &quot;__main__&quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29144" LastEditorUserId="29144" LastEditDate="2018-11-15T17:08:52.177" LastActivityDate="2019-08-13T08:41:02.297" Title="Double potential well with Python" Tags="|python|quantum-mechanics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30477" PostTypeId="2" ParentId="28035" CreationDate="2018-11-05T04:06:36.420" Score="4" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;\text{Minimize}\quad&amp;amp;\frac{1}{2} x^T Q x + a^T x + c^T|x| \\&#xA;\text{subject to}\quad&amp;amp;Gx \leq b&#xA;\end{align}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is positive definite matrix, &lt;span class=&quot;math-container&quot;&gt;$c^T \gt 0$&lt;/span&gt; (element-wise) and the rest are vectors (assume standard QP notation).&lt;/p&gt;&#xA;&lt;p&gt;We begin by reformulating the problem in standard form without the absolute value sign, and we then make a further change to ensure a positive definite quadratic coefficient.&lt;/p&gt;&#xA;&lt;h3&gt;Step 1.&lt;/h3&gt;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$y = |x|$&lt;/span&gt;. Our original problem becomes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;\begin{array}{c}  \text{Minimize} \\ (x,y)  \end{array} \quad&#xA;&amp;amp;\frac{1}{2} \left[\begin{array}{c} x \\ y \end{array}\right]^T &#xA;\left[\begin{array}{cc} Q &amp;amp; 0 \\ 0 &amp;amp; 0 \end{array}\right]&#xA;\left[\begin{array}{c} x \\ y \end{array}\right] + &#xA;\left[\begin{array}{c} a \\ c \end{array}\right]^T \left[\begin{array}{c} x \\ y \end{array}\right] \\ \\&#xA;\text{subject to } \quad  &amp;amp;Gx \leq b \\ \\&#xA;\text{and } \quad &amp;amp; \left[\begin{array}{cc} &#xA;I &amp;amp; -I \\&#xA;-I &amp;amp; -I \\&#xA;\end{array}\right] &#xA;\left[\begin{array}{c} &#xA;x \\&#xA;y \\&#xA;\end{array}\right] &#xA;\leq&#xA;\left[\begin{array}{c} &#xA;0 \\&#xA;0&#xA;\end{array}\right] &#xA;\end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Although it is outside the scope of the original question, a sketch proof that the extra constraints are sufficient follows:&lt;/p&gt;&#xA;&lt;p&gt;Letting &lt;span class=&quot;math-container&quot;&gt;$y = |x|$&lt;/span&gt;, our objective becomes &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2} x^TQx + a^Tx + b^Ty$&lt;/span&gt;, the original contraints still hold and we require &lt;span class=&quot;math-container&quot;&gt;$x_i = y_i$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$x_i \geq 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-x_i = y_i$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$x_i \leq 0$&lt;/span&gt;. We need to write the new constraints in standard form.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$x_i \geq 0$&lt;/span&gt;, the constraint &lt;span class=&quot;math-container&quot;&gt;$x_i \leq y_i$&lt;/span&gt; is equivalent to &lt;span class=&quot;math-container&quot;&gt;$x_i = y_i$&lt;/span&gt;. This is because the optimizer will drive the value of &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt; as low as possible - drive it all the way to equality. Also note in this case &lt;span class=&quot;math-container&quot;&gt;$-x_i \leq y_i$&lt;/span&gt; is always true.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$x_i \leq 0$&lt;/span&gt;, we similarly have &lt;span class=&quot;math-container&quot;&gt;$-x_i \leq y_i$&lt;/span&gt; equivalent to &lt;span class=&quot;math-container&quot;&gt;$-x_i = y_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_i \leq y_i$&lt;/span&gt; is always true.&lt;/p&gt;&#xA;&lt;p&gt;Thus, the constraints &lt;span class=&quot;math-container&quot;&gt;$x_i \leq y_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-x_i \leq y_i$&lt;/span&gt; are sufficient, and can be written in the standard matrix form shown above.&lt;/p&gt;&#xA;&lt;h3&gt;Step 2.&lt;/h3&gt;&#xA;&lt;p&gt;The quadratic coefficient is clearly singular in the current form. Noting &lt;span class=&quot;math-container&quot;&gt;$\quad x_i^2 = |x_i|^2 = y_i^2$&lt;/span&gt; we can re-write the quadratic term as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;\left[\begin{array}{c} x \\ y \end{array}\right]^T &#xA;\left[\begin{array}{cc} Q - \delta I &amp;amp; 0 \\ 0 &amp;amp; \delta I \end{array}\right] &#xA;\left[\begin{array}{c} x \\ y \end{array}\right]^T &amp;amp;= &#xA;x^T Q x - x^T \delta I x + y^T \delta I y \\&#xA;&amp;amp;= x^T Q x - \delta \sum_i x_i^2 + \delta \sum_i y_i^2 \\&#xA;&amp;amp;= x^T Q x &#xA;\end{align}$$&lt;/span&gt;&#xA;as required.&lt;/p&gt;&#xA;" OwnerUserId="25658" LastEditorUserId="25658" LastEditDate="2022-02-23T15:42:52.280" LastActivityDate="2022-02-23T15:42:52.280" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30479" PostTypeId="1" CreationDate="2018-11-05T08:56:44.513" Score="6" ViewCount="1616" Body="&lt;p&gt;There is a lot of literature that indicates using &lt;code&gt;rand()&lt;/code&gt; for simulations might be harmful. A couple of them are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0167947398800029&quot; rel=&quot;noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0167947398800029&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://channel9.msdn.com/Events/GoingNative/2013/rand-Considered-Harmful&quot; rel=&quot;noreferrer&quot;&gt;https://channel9.msdn.com/Events/GoingNative/2013/rand-Considered-Harmful&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A common recommendation has been to use the &lt;code&gt;ran1&lt;/code&gt;/&lt;code&gt;ran2&lt;/code&gt;/&lt;code&gt;ran3&lt;/code&gt; functions as defined in &lt;em&gt;Numerical Recipes&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But these seem pretty old criticisms of the &lt;code&gt;rand&lt;/code&gt; function. So my questions is, how do its current implementations (GCC/clang) measure up for the purpose of being a good RNG in, say Monte Carlo simulations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the current implementation of the function used by rand() in glibc:&#xA;(rand calls random, which calls random_r)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceware.org/git/?p=glibc.git;a=blob;f=stdlib/rand.c&quot; rel=&quot;noreferrer&quot;&gt;https://sourceware.org/git/?p=glibc.git;a=blob;f=stdlib/rand.c&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceware.org/git/?p=glibc.git;a=blob;f=stdlib/random.c&quot; rel=&quot;noreferrer&quot;&gt;https://sourceware.org/git/?p=glibc.git;a=blob;f=stdlib/random.c&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceware.org/git/?p=glibc.git;a=blob;f=stdlib/random_r.c&quot; rel=&quot;noreferrer&quot;&gt;https://sourceware.org/git/?p=glibc.git;a=blob;f=stdlib/random_r.c&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="19445" LastEditorUserId="19445" LastEditDate="2018-11-06T08:02:31.173" LastActivityDate="2018-11-07T23:28:18.570" Title="How good are current implementations of rand() in C?" Tags="|c|random-number-generation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30481" PostTypeId="2" ParentId="30479" CreationDate="2018-11-05T12:31:55.003" Score="1" Body="&lt;p&gt;Though the question is about C, I still consider the following information regarding C++11 relevant.&#xA;In C++11, &lt;a href=&quot;http://www.cplusplus.com/reference/random/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;&amp;lt;random&amp;gt;&lt;/code&gt;&lt;/a&gt; was added that is both designed to fix many of C &lt;a href=&quot;http://www.cplusplus.com/reference/cstdlib/rand/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;rand()&lt;/code&gt;&lt;/a&gt; implementation flaws, as well as to provide additional capabilities. So, if you have an option of using C++11, I would point your attention to &lt;code&gt;&amp;lt;random&amp;gt;&lt;/code&gt; and &lt;a href=&quot;https://stackoverflow.com/a/18726221/2452869&quot;&gt;its benefits&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases, &lt;code&gt;&amp;lt;random&amp;gt;&lt;/code&gt; function family would provide sufficient quality random numbers (as opposed to &lt;code&gt;rand()&lt;/code&gt;, which is mentioned in the materials you linked in your question. The critique to &lt;code&gt;rand()&lt;/code&gt; is still relevant (as ANSI standard did not change); however, certainly exaggerated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice would be to first assess the need in higher quality random numbers and go step-by-step. There is a high chance, that for particular purposes &lt;code&gt;&amp;lt;random&amp;gt;&lt;/code&gt; or even &lt;code&gt;rand()&lt;/code&gt; will provide decent enough results. Otherwise (unlikely), you might have to go to special libraries or even hardware generators.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-05T12:31:55.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30482" PostTypeId="2" ParentId="30476" CreationDate="2018-11-05T14:02:05.923" Score="3" Body="&lt;p&gt;Although this site is not intended for questions of the &quot;please debug my program&quot; kind, I believe that what's needed here is a clarification of the scientific computing aspects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method you are adopting, from the link that you gave, is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Shooting_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;shooting method&lt;/em&gt;&lt;/a&gt;. The program uses a numerical ordinary differential equation solver for the Schrodinger equation, which takes initial conditions &lt;span class=&quot;math-container&quot;&gt;$\psi(x_0)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\psi'(x_0)$&lt;/span&gt;, for some chosen starting point &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt;, and integrates forward in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. The output is a wavefunction at discrete values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;; the program carries out this integration for a range of energies &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although your question refers to an infinite potential well,&#xA;your diagram, your program, and your original source&#xA;all deal with a &lt;em&gt;finite&lt;/em&gt; potential well.&#xA;So I'm going to stick with that.&#xA;(To be honest, in the original program given at &#xA;&lt;a href=&quot;https://helentronica.com/2014/09/04/quantum-mechanics-with-the-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://helentronica.com/2014/09/04/quantum-mechanics-with-the-python/&lt;/a&gt;,&#xA;I can't be sure that the sign of &lt;span class=&quot;math-container&quot;&gt;$V_0$&lt;/span&gt; is correct! but I haven't&#xA;looked closely).&#xA;If you really mean an infinite well, then there are only slight differences, which I return to at the end.&#xA;For simplicity I'm also going to stick with the symmetric case, &lt;span class=&quot;math-container&quot;&gt;$\alpha=1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main part of the program is mainly interested in checking the value of &lt;span class=&quot;math-container&quot;&gt;$\psi(b)$&lt;/span&gt; for a position &lt;span class=&quot;math-container&quot;&gt;$x=b$&lt;/span&gt; lying far outside the potential well; this is why the Wavefunction function only returns the last element of the computed &lt;span class=&quot;math-container&quot;&gt;$\psi$&lt;/span&gt; array. Only values of &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; which give non-divergent wavefunctions are wanted; in fact, the boundary condition that we want to enforce is &lt;span class=&quot;math-container&quot;&gt;$\psi(x)\rightarrow 0$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$x\rightarrow\infty$&lt;/span&gt;. The program uses the array of values of &lt;span class=&quot;math-container&quot;&gt;$\psi(b)$&lt;/span&gt; for different energies to determine intervals of &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\psi(b)$&lt;/span&gt; changes sign, and then uses Brent's method to refine the value of &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; for which &lt;span class=&quot;math-container&quot;&gt;$\psi(b)=0$&lt;/span&gt;. To a good approximation, this will be one of the allowable eigenvalues and the corresponding &lt;span class=&quot;math-container&quot;&gt;$\psi(x)$&lt;/span&gt; will be the eigenfunction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;state&quot; which is required by the ODE solver, calculated by the SE function, will always be an array having two elements: the first derivative and the second derivative of &lt;span class=&quot;math-container&quot;&gt;$\psi$&lt;/span&gt;. This is because the Schrodinger equation is a second-order differential equation, which is being tackled here as two first order equations:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;\frac{d\psi}{dx} &amp;amp;= \psi'(x) \\&#xA;\frac{d\psi'}{dx} &amp;amp;= 2[V(x)-E]\psi(x)&#xA;\end{align*}&lt;/span&gt;&#xA;in your units.&#xA;The psi array stores the values of &lt;span class=&quot;math-container&quot;&gt;$\psi(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\psi'(x)$&lt;/span&gt;;&#xA;the SE function returns the two quantities on the right hand side&#xA;of these equations.&#xA;So, the changes you have made to the SE function, returning 6 values, are on the wrong track. The value of the second derivative in the different regions of the potential well will depend on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; through the function &lt;span class=&quot;math-container&quot;&gt;$V(x)$&lt;/span&gt;, which you set up at the start of the program. So nothing in SE needs changing from the original code which you downloaded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other aspects needing attention are the ranges of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. In the original code you started working with, the width of the well is determined by &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; (which was set to 1 in the sample program) and the check on &lt;span class=&quot;math-container&quot;&gt;$\psi(b)$&lt;/span&gt; was made at &lt;span class=&quot;math-container&quot;&gt;$b\gg L$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; was set to 2 in the example). You need to do something similar. Various starting points &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; (for the shooting integration) were considered in the original example; for a symmetric potential well, there is something to be said for starting at the midpoint, &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt;, with either &lt;span class=&quot;math-container&quot;&gt;$\psi(0)=0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\psi'(0)=0$&lt;/span&gt; to produce, respectively, antisymmetric or symmetric solutions. But you can also start at &lt;span class=&quot;math-container&quot;&gt;$-b$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$-b\ll -L$&lt;/span&gt;, and this might be better in your case. Finally, as someone noted in this answer &lt;a href=&quot;https://physics.stackexchange.com/a/438299/197851&quot;&gt;https://physics.stackexchange.com/a/438299/197851&lt;/a&gt; to your question on Physics SE, the locations of the edges of the well in your diagram do not match the same locations in your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to summarize: there are really no fundamental changes needed to the original program, just changing &lt;span class=&quot;math-container&quot;&gt;$V(x)$&lt;/span&gt; should work. The SE routine does not need changing from the originally downloaded version. You need to pay more attention to the limits of the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; range used in the shooting method: you should end at a position which is outside the potential well on the right, and you might start from a position outside the well, on the left. I recommend starting again with the original program, where the well extends from &lt;span class=&quot;math-container&quot;&gt;$-L$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$+L$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$L=1$&lt;/span&gt;, and adding a symmetrical bump around the origin from &lt;span class=&quot;math-container&quot;&gt;$-a$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$+a$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$a&amp;lt;1$&lt;/span&gt;, keeping &lt;span class=&quot;math-container&quot;&gt;$b=2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to deal with an infinite potential well,&#xA;then you should set &lt;span class=&quot;math-container&quot;&gt;$b=L$&lt;/span&gt; and enforce the boundary condition &lt;span class=&quot;math-container&quot;&gt;$\psi(b)=0$&lt;/span&gt;.&#xA;In this case it also makes sense to start shooting at &lt;span class=&quot;math-container&quot;&gt;$x=-b$&lt;/span&gt;,&#xA;with &lt;span class=&quot;math-container&quot;&gt;$\psi(-b)=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\psi'(b)$&lt;/span&gt; nonzero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully this helps. As I indicated, this site isn't for program debugging, but if you don't follow the idea behind the method you are using, I'm happy to try and clarify.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastEditorDisplayName="user28077" LastEditDate="2018-11-05T15:19:34.897" LastActivityDate="2018-11-05T15:19:34.897" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30483" PostTypeId="1" CreationDate="2018-11-05T15:53:41.657" Score="3" ViewCount="89" Body="&lt;p&gt;Im just working on a preconditioner for the linear equation system &lt;span class=&quot;math-container&quot;&gt;$Ax = b$&lt;/span&gt; arising in FEM for elliptic PDE. &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a s.p.d Matrix with real valued entries. I read something about the element by element preconditioner and now I have a question about the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Defining a preconditioner &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$P \approx A$&lt;/span&gt;. One can solve the equation system (left preconditioning) instead: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;P^{-1}Ax = P^{-1}b&#xA;\end{equation}&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read that some people choosing the preconditioner as the Assembly of Element Preconditioners.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;P =\prod\limits_{e =1}^n P_e&#xA;\end{equation}&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$P_e$&lt;/span&gt; is defined for every element &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; as &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;P_e = I + diag(A_e)^{-1/2}\cdot(A_e - diag(A_e))\cdot diag(A_e)^{-1/2}&#xA;\end{equation}&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to calculate &lt;span class=&quot;math-container&quot;&gt;$P^{-1}$&lt;/span&gt; you just need to assembly all &lt;span class=&quot;math-container&quot;&gt;$P_e^{-1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is now: If I use a Cholesky decomposition would for &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$P = C\cdot C^T$&lt;/span&gt; would it be the same as if I use the decomposition for every element and assemble afterwards:??&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;P_e = C_e C_e^T \rightarrow C\cdot C^T = \prod\limits_{e = 1}^n C_e C_e^T&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dont have the feeling that this holds because of overlapping Elements. But Im not sure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way my goal is to use a decomposition to solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;C^{-1}AC^{-T}y &amp;amp;= C^{-1}b \\&#xA;x &amp;amp;= C^{-T}y&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user29088" LastActivityDate="2018-11-05T15:53:41.657" Title="Element Preconditioner" Tags="|linear-algebra|finite-element|preconditioning|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30489" PostTypeId="1" CreationDate="2018-11-06T13:27:42.980" Score="4" ViewCount="1935" Body="&lt;p&gt;I'm trying to simulate basic semiconductor models for pedagogical purposes--starting from the Drift-diffusion model. Although I don't want to use an off-the-shelf semiconductor simulator--I'll be learning other (common, recent or obscure) models, I do want to use an off-the-shelf PDE solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But even for the simple 1D case, the drift-diffusion model consists of a number of coupled nonlinear PDEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Current density equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$J_n = q n(x) \mu_n E(x) + q D_n \nabla n$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$J_p = q p(x) \mu_p E(x) - q D_p \nabla p$$&lt;/span&gt;        &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Continuity equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial{n}}{\partial{t}} = \frac{1}{q} \nabla \cdot J_n + U_n $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial{p}}{\partial{t}} = \frac{1}{q} \nabla \cdot J_p + U_p $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Poisson equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla \cdot (\epsilon \nabla V) = -(p - n + N_D^+ - N_A^-) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and a number of boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a package (preferably open source) that would take these equations, in that form, and solve them? Or perhaps is the variational form required by the tools is not as hard? In any case, what are my options?&lt;/p&gt;&#xA;" OwnerUserId="29163" LastEditorUserId="7681" LastEditDate="2018-12-16T08:52:45.103" LastActivityDate="2018-12-16T08:52:45.103" Title="How to simulate basic semiconductor models using the Drift-diffusion model on Python?" Tags="|pde|computational-physics|simulation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30490" PostTypeId="2" ParentId="30489" CreationDate="2018-11-06T15:37:12.437" Score="1" Body="&lt;p&gt;I can not provide you with a solver that will solve the equation for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will, however, provide a warning:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to employ finite element methods, especially so called continuous Galerkin &#xA;methods, you &lt;em&gt;will&lt;/em&gt; need a weak form of the equations. For the equations you have provided that form is not trivial to obtain, but it isn't especially difficult to obtain either.&#xA;If you opt for finite element methods an do derive the weak form of the equations you should be able to use FEniCS( &lt;a href=&quot;https://fenicsproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fenicsproject.org/&lt;/a&gt; )&#xA;for everything else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving the resulting coupled systems of linear equations is not difficult (use Newton or Newton-Kantorovich methods), but will require some thought if you want to have acceptable execution times.&#xA;So in summary: A bespoke implementation of a solver will be quite time consuming (you MUST test it) so try to avoid it unless you really&#xA;wan to understand what can go wrong with your solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My background: I have advised a friend (in numerical mathematics) during his master's thesis on how to interpret the numerical results he was&#xA;obtaining from solving an optimal control problem for the Nernst-Planck-Poisson equation (which is essentially the equation you have provided).&#xA;He was solving it in 2D though (or 2+1 D if you count time, too) and we were using my C++.&lt;/p&gt;&#xA;" OwnerUserId="28993" LastActivityDate="2018-11-06T15:37:12.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30491" PostTypeId="1" AcceptedAnswerId="30492" CreationDate="2018-11-06T17:08:17.823" Score="-4" ViewCount="185" Body="&lt;p&gt;Can someone please explain what is the Sherman-Morrison formula and it's specialities when it comes to matrix calculations? I'm a little bit confused on understanding how the preconditioning works with Sherman-Morrison formula.&lt;/p&gt;&#xA;" OwnerUserId="26940" LastEditorUserId="26940" LastEditDate="2018-11-07T14:06:37.283" LastActivityDate="2018-11-07T14:06:37.283" Title="what is Sherman-Morrison formula" Tags="|matrix|preconditioning|matrix-factorization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30492" PostTypeId="2" ParentId="30491" CreationDate="2018-11-06T18:17:01.070" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Sherman%E2%80%93Morrison_formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sherman-Morrison&lt;/a&gt; formula helps to find the inverse of the matrix cheap after a rank-1 update. &lt;a href=&quot;https://en.wikipedia.org/wiki/Woodbury_matrix_identity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sherman-Morrison-Woodbury&lt;/a&gt; does it for a low-rank update (not necessarily rank-1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You have a matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;strong&gt;you already have a computed&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;For some reason, you need to do a rank-1 update to &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. This rank-1 update can be described as an &lt;a href=&quot;https://en.wikipedia.org/wiki/Outer_product&quot; rel=&quot;nofollow noreferrer&quot;&gt;outer-product&lt;/a&gt; of two vectors &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;So, you want an inverse of &lt;span class=&quot;math-container&quot;&gt;$(A+uv^T)$&lt;/span&gt;, but you don't want to redo all the calculations. Can you reuse your knowledge of &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt;? Yes.&lt;/li&gt;&#xA;&lt;li&gt;Sherman-Morrison tells you that &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(A+uv^T)^{-1} = A^{-1}-\frac{A^{-1}uv^TA^{-1}}{1+v^tA^{-1}u}&#xA;$$&lt;/span&gt;&#xA;which allows &lt;a href=&quot;https://en.wikipedia.org/wiki/Sherman%E2%80%93Morrison_formula#Application&quot; rel=&quot;nofollow noreferrer&quot;&gt;huge savings&lt;/a&gt; since you are just performing relatively cheap manipulations with an already obtained &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I strongly encourage you to read at least the Wikipedia articles regarding this topic in order to get an understanding about &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;numerical stability&lt;/li&gt;&#xA;&lt;li&gt;requirements for the formula to be applicable&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Important notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;calculations of the inverse should be avoided at all costs&lt;/li&gt;&#xA;&lt;li&gt;same ideas (SM, SMW) usually can be applied to factorizations, where they are extremely useful&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-11-06T18:22:48.033" LastActivityDate="2018-11-06T18:22:48.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30493" PostTypeId="2" ParentId="20629" CreationDate="2018-11-06T21:22:24.310" Score="4" Body="&lt;p&gt;To expand on the difference between &lt;code&gt;log&lt;/code&gt; and &lt;code&gt;log1p&lt;/code&gt; it might help to recall the graph if the logarithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D2v3d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D2v3d.png&quot; alt=&quot;Logarithm&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data contains zeroes, then you probably don't want to use &lt;code&gt;log&lt;/code&gt; since it's undefined at zero. And as &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; approaches &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, the value of &lt;span class=&quot;math-container&quot;&gt;$\ln(x)$&lt;/span&gt; approaches &lt;span class=&quot;math-container&quot;&gt;$- \infty$&lt;/span&gt;. So if your &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; values are close to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, then the value of &lt;span class=&quot;math-container&quot;&gt;$\ln(x)$&lt;/span&gt; is potentially a large negative number. For example &lt;span class=&quot;math-container&quot;&gt;$\ln(\frac{1}{e}) = -1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\ln(\frac{1}{e^{10}}) = -10$&lt;/span&gt; and so on. This can be useful, but it can also distort your data towards large negative numbers, especially if your dataset also contains numbers much larger than zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, as &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; approaches &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, the value of &lt;span class=&quot;math-container&quot;&gt;$\ln(x+1)$&lt;/span&gt; approaches &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; from the positive direction. For example &lt;span class=&quot;math-container&quot;&gt;$\ln(1+\frac{1}{e}) \sim 0.31$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\ln(1+\frac{1}{e^{10}}) \sim 0.000045$&lt;/span&gt;. So &lt;code&gt;log1p&lt;/code&gt; produces only positive values and removes the 'danger' of large negative numbers. This generally insures a more homogeneous distribution when a dataset contains numbers close to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, if the dataset is all greater than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, then &lt;code&gt;log&lt;/code&gt; is usually fine. But, if the dataset has numbers between &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, then &lt;code&gt;log1p&lt;/code&gt; is usually better.&lt;/p&gt;&#xA;" OwnerUserId="27556" LastEditorUserId="27556" LastEditDate="2019-04-05T18:15:22.093" LastActivityDate="2019-04-05T18:15:22.093" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="30494" PostTypeId="2" ParentId="3437" CreationDate="2018-11-07T02:08:33.070" Score="2" Body="&lt;p&gt;This question is very related to &lt;a href=&quot;https://scicomp.stackexchange.com/q/365/20688&quot;&gt;another SE question on condition number estimates&lt;/a&gt; which contains relevant materials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @Jack_Poulson mentioned, the following paper contains a detailed discussion on the asked topic:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1137/0613066&quot; rel=&quot;nofollow noreferrer&quot;&gt;J. Kuczyński and H. Woźniakowski, &quot;Estimating the Largest Eigenvalue by the Power and Lanczos Algorithms with a Random Start,&quot; &lt;em&gt;SIAM J. Matrix Anal. Appl.&lt;/em&gt;, vol. 13, no. 4, pp. 1094–1122&lt;/a&gt;. Downloadable version of a slightly different version of this paper &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.4725&quot; rel=&quot;nofollow noreferrer&quot;&gt;is available here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-07T02:08:33.070" CommentCount="0" CommunityOwnedDate="2018-11-07T02:08:33.070" ContentLicense="CC BY-SA 4.0" />
  <row Id="30497" PostTypeId="2" ParentId="30479" CreationDate="2018-11-07T17:37:30.560" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;But these seem pretty old criticisms of the rand function&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This may be a nitpick, but I want to point out what I think is a flaw in this logic. Compilers are often extremely conservative about changing program behaviour, even when that behaviour (foolishly) depends on implementation details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may or may not be true for the big compilers you’re familiar with, but it is &lt;em&gt;conceivable&lt;/em&gt; that a compiler would keep a broken legacy RNG for compatibility reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following program on my macbook:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;cstdlib&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;using namespace std;&#xA;&#xA;int main() {&#xA;  srand(1);&#xA;  for (int i = 0; i &amp;lt; 20; ++i) {&#xA;    cout &amp;lt;&amp;lt; (i &amp;gt; 0 ? &quot;, &quot; : &quot;&quot;) &amp;lt;&amp;lt; rand();&#xA;  }&#xA;  cout &amp;lt;&amp;lt; endl;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;produces the output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;16807, 282475249, 1622650073, 984943658, 1144108930, 470211272, 101027544, 1457850878, 1458777923, 2007237709, 823564440, 1115438165, 1784484492, 74243042, 114807987, 1137522503, 1441282327, 16531729, 823378840, 143542612&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which you can look up in OEIS: &lt;a href=&quot;https://oeis.org/A096550&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://oeis.org/A096550&lt;/a&gt;, it is the sequence  &lt;span class=&quot;math-container&quot;&gt;$$16807^n\bmod(2^{31}-1).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How this does under the SmallCrush RNG test (in the TestU01 library):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;========= Summary results of SmallCrush =========&#xA;&#xA; Version:          TestU01 1.2.3&#xA; Generator:        ulcg_CreateLCG&#xA; Number of statistics:  15&#xA; Total CPU time:   00:00:08.05&#xA; The following tests gave p-values outside [0.001, 0.9990]:&#xA; (eps  means a value &amp;lt; 1.0e-300):&#xA; (eps1 means a value &amp;lt; 1.0e-15):&#xA;&#xA;       Test                          p-value&#xA; ----------------------------------------------&#xA;  1  BirthdaySpacings                 eps&#xA;  2  Collision                        eps&#xA;  6  MaxOft                           eps&#xA; ----------------------------------------------&#xA; All other tests were passed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as I know there is simply no good reason to assume that &lt;code&gt;rand()&lt;/code&gt; might be okay. If you assume otherwise, you'll usually end up writing some extremely non-portable code that will break under different compilers on different systems.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2018-11-07T18:07:25.347" LastActivityDate="2018-11-07T18:07:25.347" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30499" PostTypeId="1" AcceptedAnswerId="30501" CreationDate="2018-11-07T21:37:14.970" Score="1" ViewCount="262" Body="&lt;p&gt;I am having trouble with a C/C++ program that uses the BLAS routine &lt;code&gt;ddot&lt;/code&gt;.&#xA;I am running Linux and so far LAPACK routines worked without any problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get a wrong result, no matter if I choose the type &lt;code&gt;int&lt;/code&gt; for integer arguments of &lt;code&gt;ddot&lt;/code&gt; or if I choose &lt;code&gt;ptrdiff_t&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;using namespace std;&#xA;&#xA;extern &quot;C&quot; {&#xA;    extern int ddot_( int*, double* , int* , double* ,int*); &#xA;    // OR extern int ddot_( ptrdiff_t*, double* , ptrdiff_t* , double* ,ptrdiff_t*);&#xA;}&#xA;&#xA;int main() {&#xA;    int n=3; // OR ptrdiff_t  n=3;&#xA;    int inc=1; // OR ptrdiff_t inc=1;&#xA;    double result=0;&#xA;&#xA;    double *x = (double*) calloc (n, sizeof (double));&#xA;    double *y = (double*) calloc (n, sizeof (double));&#xA;&#xA;    result = ddot_(&amp;amp;n, x, &amp;amp;inc, y, &amp;amp;inc);&#xA;    cout&amp;lt;&amp;lt;result&amp;lt;&amp;lt;endl;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I choose &lt;code&gt;n&lt;/code&gt; and &lt;code&gt;inc&lt;/code&gt; to be of type &lt;code&gt;int&lt;/code&gt; everything compiles but I get the output that &lt;code&gt;result&lt;/code&gt; is equal to &lt;code&gt;4&lt;/code&gt;. Same effect with different number for &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;y&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to google my question and found out that the type of &lt;code&gt;n&lt;/code&gt; and &lt;code&gt;inc&lt;/code&gt; might cause wrong results and I should try &lt;code&gt;ptrdiff_t&lt;/code&gt; or &lt;code&gt;mwSignedIndex&lt;/code&gt;. &#xA;I didn't manage to include &lt;code&gt;mwSignedIndex&lt;/code&gt; in my program so far but if I choose &lt;code&gt;n&lt;/code&gt; and &lt;code&gt;inc&lt;/code&gt; to be of type &lt;code&gt;ptrdiff_t&lt;/code&gt; and change the &quot;extern C&quot; part in the beginning I get &lt;code&gt;4&lt;/code&gt;as a result aswell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something? Is my math wrong?&#xA;I appreciate any help or suggestions!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: in a former version of the question I didn't manage to properly use the &lt;code&gt;ptrdiff_t&lt;/code&gt;. Nevertheless the wrong result is still there.&lt;/p&gt;&#xA;" OwnerUserId="27798" LastEditorUserId="27798" LastEditDate="2018-11-07T22:12:48.560" LastActivityDate="2018-11-07T22:13:12.430" Title="Wrong result of 'ddot' from BLAS" Tags="|c|lapack|blas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30500" PostTypeId="2" ParentId="26145" CreationDate="2018-11-07T22:03:12.493" Score="0" Body="&lt;p&gt;For &lt;code&gt;pc_type mg&lt;/code&gt;, the default smoother is Chebyshev as is specified in &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPCHEBYSHEV.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSC documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The preconditioned Chebyshev iterative method&lt;br&gt;&#xA;  The Chebyshev method requires both the matrix and preconditioner to be symmetric positive (semi) definite. Only support for left preconditioning.&lt;br&gt;&#xA;  Chebyshev is configured as a smoother by default, targetting the &quot;upper&quot; part of the spectrum.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Worth to mention, that the comments suggesting using &lt;code&gt;-ksp-view&lt;/code&gt; advertise very good practices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, this is true that Chebyshev is designed to work only with SPD matrices. However, the fact that it will become unstable for &quot;very non-SPD matrices&quot; (having eigenvalues relatively far away from the real axis), does not mean that it will not work at all for &quot;slightly non-symmetric matrices&quot;. So, your problem might be not unsymmetric enough.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that low-order Chebyshev is very close to Richardson, so no wonder it might work surprisingly well for certain not necessarily SPD cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is not recommended to use Chebyshev (the default) for matrices that are known to have a significant imaginary component to their eigenvalues and switch to Richardson or GMRES.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-07T22:03:12.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30501" PostTypeId="2" ParentId="30499" CreationDate="2018-11-07T22:13:12.430" Score="3" Body="&lt;p&gt;The strange thing in your code is &lt;code&gt;ddot_&lt;/code&gt; being declared as &lt;code&gt;extern C int&lt;/code&gt;, while it is actually a &lt;code&gt;double&lt;/code&gt;-valued function. And its value is assigned to a &lt;code&gt;double&lt;/code&gt; variable &lt;code&gt;result&lt;/code&gt;. Those conversions are probably what is causing your issues (I am not entirely sure how did it compile and passed the &lt;code&gt;-Wall&lt;/code&gt; compiler flag – I hope you are using one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;n&lt;/code&gt; and &lt;code&gt;inc&lt;/code&gt; being &lt;code&gt;int&lt;/code&gt;s should be fine, unless you are not doing something very specific with a specifically configured version of BLAS on a particular architecture.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-07T22:13:12.430" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30502" PostTypeId="2" ParentId="30479" CreationDate="2018-11-07T23:28:18.570" Score="0" Body="&lt;p&gt;There is a finite number of correlation metrics relevant to your problem which you ideally &lt;em&gt;should&lt;/em&gt; know before trying to implement MC!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding &lt;code&gt;rand()&lt;/code&gt;: I have written (parallel) MC methods which could easily draw &lt;span class=&quot;math-container&quot;&gt;$&amp;gt;10^9$&lt;/span&gt; PRNs at which point you cycle the sequence of PRNs provided by &lt;code&gt;rand()&lt;/code&gt;, whose period is &lt;span class=&quot;math-container&quot;&gt;$2^{32}-1\approx 4.3\cdot 10^9$&lt;/span&gt; if I understand the comments in the code you link to correctly.&#xA;Now you either have to analyse whether potential correlations will impact your problem, or you can choose a PRNG with a longer period, thereby avoiding the problem (or at least pushing it off to larger problem sizes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially in parallel environments (the common way to run a MC simulation) the usage of &lt;code&gt;rand()&lt;/code&gt; harbors the additional problem that multiple processes may end up initializing the PRNG to either start at the same position, or at positions at which their sequences will eventually overlap. The latter will result in two processes using the same PRNs for (supposedly) independent simulations. Other PRNGs, like the Mersenne Twister engine used in C++11 suffer the same problem, but to a lesser degree since their (limited) parameter sets produce different sequences, so as long as you do not initialize them with the same seed they will yield non-overlapping sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A C code I have worked with used &lt;code&gt;rand()&lt;/code&gt; for a parallel MC method used to approximate an inverse of a (very large) matrix. I have empirically determined that replacing it by a TRNG ( &lt;a href=&quot;https://www.numbercrunch.de/trng/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.numbercrunch.de/trng/&lt;/a&gt; ) - had to write a C-wrapper for it, though - measurably reduced the errors of the final result. Especially for large matrices (e.g., those that needed a lot of PRNs).&#xA;&lt;strong&gt;Note here:&lt;/strong&gt; Only because you are limited to work with C does not mean you can not use C++ libraries, you just have to write wrappers for the functions you need.&lt;/p&gt;&#xA;" OwnerUserId="28993" LastActivityDate="2018-11-07T23:28:18.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30503" PostTypeId="1" CreationDate="2018-11-07T23:30:31.860" Score="1" ViewCount="284" Body="&lt;p&gt;I am dealing with a &lt;a href=&quot;https://cgns.github.io/CGNS_docs_current/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;*.cgns&lt;/code&gt; file&lt;/a&gt;. This mesh format, when saved as an unstructured grid, holds nodes coordinates, nodes connectivity per element and boundary conditions. It is a neat mesh format with good documentation. However, I am still responsible for building the rest of the data structures for the calculations. For example, the face table for cell-centered FV schemes. This is a very delicate work which has to be done for every solver somehow (for cell-vertex solvers it would be the points neighboring I think). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, since many folks out there had to develop this type of routines, I think it is reasonable to think someone smarter than me wrote a library which organize these data structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was reading &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DMPLEX/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc documentation for DMPlex&lt;/a&gt; but I am still trying to figure out how to give a &lt;code&gt;*.cgns&lt;/code&gt; file to it. From what I saw, even compiling with cgns support, the file I/O for this format is not well established for this library. I am not criticizing anyone, PETSc is an awesome library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is if there is any library out there which manages the creation of this type of data structures, not just for FV methods, but also for any type of scheme/solver? A quick bonus question... Is there anyone researching algorithms to create data structures for PDE solvers efficiently?&lt;/p&gt;&#xA;" OwnerUserId="21588" LastEditorUserId="3970" LastEditDate="2018-11-13T13:55:58.907" LastActivityDate="2018-11-13T13:55:58.907" Title="Libraries to deal with unstructured grids" Tags="|pde|numerical-modelling|libraries|unstructured-mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30504" PostTypeId="2" ParentId="25812" CreationDate="2018-11-08T00:30:40.727" Score="2" Body="&lt;p&gt;I prefer a bit different notation, but the workflow is certainly similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Denote the rectangular vertices, as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{v}_{00}=\left(\matrix{0\\0}\right),\quad \mathbf{v}_{10}=\left(\matrix{2\\-1}\right),\quad&#xA;\mathbf{v}_{11}=\left(\matrix{3\\2}\right),\quad&#xA;\mathbf{v}_{01}=\left(\matrix{1\\3}\right)&#xA;$$&lt;/span&gt;&#xA;Now, the position vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{r}$&lt;/span&gt; inside (or on the boundary) of the quadrilateral can be expressed in terms of barycentric coordinates &lt;span class=&quot;math-container&quot;&gt;$(\xi,\eta)$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{r}(\xi,\eta) = (1-\xi)(1-\eta)\mathbf{v}_{00} + \xi(1-\eta)\mathbf{v}_{10} + &#xA;(1-\xi)\eta\mathbf{v}_{01} + \xi\eta\mathbf{v}_{11}, \quad \xi,\eta\in[0,1]&#xA;$$&lt;/span&gt;&#xA;Now, the integral for the edge between &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}_{10}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}_{11}$&lt;/span&gt;, will correspond to &lt;span class=&quot;math-container&quot;&gt;$\xi=1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\eta \in[0,1]$&lt;/span&gt;. Now, in a slighlty sloppy mathematical notation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int\limits_{\mathbf{v}_{10}}^{\mathbf{v}_{11}}f(\mathbf{v})d\mathbf{v} = \int\limits_0^1d\eta f\big(\mathbf{r}(\xi=1,\eta)\big)J&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$f(\mathbf{v}) = (\mathbf{v}\cdot\hat{x})^3 +  (\mathbf{v}\cdot\hat{y})^3$&lt;/span&gt;, which is just a fancy way to write &lt;span class=&quot;math-container&quot;&gt;$f(x,y) = x^3+y^3$&lt;/span&gt; using position-vectors. For this integral, the Jacobian &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; would be the length of the line between &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}_{10}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}_{11}$&lt;/span&gt;. So, &lt;span class=&quot;math-container&quot;&gt;$J=\sqrt{10}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can just compute the integral and obtain that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J\int\limits_0^1d\eta f\big(\mathbf{r}(\xi=1,\eta)\big)=\sqrt{10}\frac{35}{2}\approx 55.3399&#xA;$$&lt;/span&gt;&#xA;which is the analytical result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also checked your Lagrange interpolant and Jacobian. Seems right to me. I guess the error is somewhere in the piping between your functions, main code and interpolants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, see below screenshot of Mathcad screen for reference that implements both the &quot;method&quot; described above, as well as your Lagrange interpolant (only for the integral over the requested edge).&lt;a href=&quot;https://i.stack.imgur.com/CiWCR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CiWCR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-08T00:30:40.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30505" PostTypeId="1" CreationDate="2018-11-08T02:06:38.030" Score="3" ViewCount="995" Body="&lt;p&gt;I am aware than Euler explicit is conditionally stable, and Euler implicit is unconditionally stable. And I am aware that it is probably pointless to use Euler implicit with a small computational step &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; for which Euler explicit is stable. But from a theoretical point of view, should one of the two be more accurate than the other, of course for &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; for which the explicit method is stable? It can be shown that the local truncation error for both Euler explicit and Euler Implicit scales linearly with &lt;span class=&quot;math-container&quot;&gt;$\Delta t^2$&lt;/span&gt;, and therefore the global truncation error should scale with &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt;, so they are both first order accurate. And from what I understand also the constant in front of &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; should be the same, or is here where I am doing wrong? &lt;a href=&quot;https://www.flow3d.com/resources/cfd-101/numerical-issues/implicit-versus-explicit-numerical-methods/&quot; rel=&quot;nofollow noreferrer&quot;&gt;In FLOW-3D technical manual on implicit vs explicit methods&lt;/a&gt; I read that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For an implicit method to have minimal under-relaxation (i.e., little damping), a time-step size much smaller than the stable, explicit value would have to be used. [...] To reduce this under-relaxation damping the time-step size would have to be much smaller than the explicit stability limit, but this makes little sense since an implicit method is not required.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is this true? Since they are both first order, are we saying that it is possible to prove that the proportionality constant in front of &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; is larger for the implicit scheme and depends on the step (larger damping for higher steps)?&#xA;Can the answer to this question be generalized to Runge-Kutta schemes?&lt;/p&gt;&#xA;" OwnerUserId="10032" LastEditorUserId="20688" LastEditDate="2018-11-08T02:19:47.770" LastActivityDate="2018-11-09T21:14:12.300" Title="ODE: should Euler implicit be more accurate than Euler explicit for a given computational step?" Tags="|ode|implicit-methods|explicit-methods|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30508" PostTypeId="2" ParentId="29038" CreationDate="2018-11-08T15:02:27.060" Score="1" Body="&lt;p&gt;The brief history that I was able to find regarding the SLEPc Matlab interface:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;3.2 (October 2011): Matlab interface + examples is introduced. Located at &lt;code&gt;/bin/matlab&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;3.5.4 (July 2014): the files are moved to &lt;code&gt;/share/slepc/matlab&lt;/code&gt;. In the documentation for 3.5.4, section 8.9 is devoted to the Matlab interface.&lt;/li&gt;&#xA;&lt;li&gt;3.6.3 (June 2015): Matlab interface disappears from &lt;code&gt;/share&lt;/code&gt; and &quot;8.9 Matlab Interface&quot; is removed from the documentation.&lt;/li&gt;&#xA;&lt;li&gt;This interface &lt;strong&gt;is not present&lt;/strong&gt; in the current version of SLEPc (3.10 September 2018).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's worth to note the phrasing in the documentation for this interface of SLEPc suggested its experimental status:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since version 3.2, slepc includes an interface intended to make most of slepc’s functionality available from Matlab. It is experimental and needs further development, so users planning to use it seriously are recommended to contact the authors. Below are some guidelines for using this interface...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was not able to find any SLEPc Matlab interface that was working right out-of-the-box with 3.10. You might be able to get some limited functionality yourself by reusing the old interface from 3.5.4 or calling your own C-functions that provide SLEPc wrapper via Matlab's &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/calllib.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;calllib&lt;/code&gt; functionality&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-08T15:02:27.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30510" PostTypeId="2" ParentId="30505" CreationDate="2018-11-09T21:14:12.300" Score="5" Body="&lt;p&gt;In general, just because method A provides certain guarantees (such as unconditional stability, energy conservation, being symplectic) does not imply that it is more accurate. In fact, a common observation is that the opposite may be true: For example, if a method is symplectic, then it guarantees that the error is zero with regard to certain quantities (e.g. angular momentum) but that in the norm of the solution vector, the error is often larger than other methods that are not symplectic. It's a bit like a water balloon: if you squeeze it in one direction, it has to expand in the other direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So no, there is no reason to believe that the implicit Euler method is, in general, more accurate than the explicit Euler method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-11-09T21:14:12.300" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30511" PostTypeId="2" ParentId="30503" CreationDate="2018-11-09T21:21:54.820" Score="1" Body="&lt;p&gt;Most finite element libraries have the ability to read a good number of mesh formats. The library I work on, deal.II (see &lt;a href=&quot;http://www.dealii.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.dealii.org&lt;/a&gt;) happens to have quite a number of mesh readers for VTK, UCD, GMSH, ... formats, see here: &#xA;&lt;a href=&quot;https://dealii.org/developer/doxygen/deal.II/classGridIn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dealii.org/developer/doxygen/deal.II/classGridIn.html&lt;/a&gt; and its implementation here&#xA;&lt;a href=&quot;https://github.com/dealii/dealii/blob/master/source/grid/grid_in.cc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dealii/dealii/blob/master/source/grid/grid_in.cc&lt;/a&gt; . These readers are generally not terribly complicated (with the exception of the UNV format, which for historical reasons is really just a memory dump of one particular program) and relatively easy to extend to a new format such as the one you're interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have that out of the way, the whole rest then happens automatically: building meshes, building connections between cells, querying cells for their neighbors, bounding edges or vertices, or building numerical methods on them. In other words, the problem is nicely separated: All you would have to do is to write a reader for a particular mesh format, and every functionality of these libraries then becomes available to you immediately.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-11-09T21:21:54.820" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30514" PostTypeId="1" AcceptedAnswerId="30569" CreationDate="2018-11-10T13:53:20.327" Score="1" ViewCount="132" Body="&lt;p&gt;What is the best approach to go about solving a PDE problem of the type&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} &#xA;  k^3\Delta u - k(\mathbf{1}\cdot\nabla u) = 0\, ,\\&#xA;  u=g\; \text{on}\; \Gamma_D\, ,\\&#xA;  mean(u) = u_\text{mean}&#xA;\end{equation}&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where one wants to find for which positive constant scalar coefficient &lt;span class=&quot;math-container&quot;&gt;$k&amp;gt;0$&lt;/span&gt; the mean of the solution &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; fulfills a prescribed value &lt;span class=&quot;math-container&quot;&gt;$u_\text{mean}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="29168" LastEditorUserId="29168" LastEditDate="2018-11-16T11:08:44.253" LastActivityDate="2018-11-24T02:55:48.660" Title="Method to find PDE equation coefficient satisfying mean solution?" Tags="|finite-element|optimization|pde|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30515" PostTypeId="1" CreationDate="2018-11-10T14:43:01.403" Score="0" ViewCount="46" Body="&lt;p&gt;I'm trying to solve&#xA;&lt;span class=&quot;math-container&quot;&gt;$&#xA;    \begin{cases}&#xA;        -u''=f \\&#xA;        u(0)=0 \\&#xA;        u(1)= \alpha &#xA;    \end{cases}&#xA;$&lt;/span&gt;&#xA;with FEM using reference elements and local coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we have the global matrix &lt;span class=&quot;math-container&quot;&gt;$K_{ij}=\int_\Omega N_i'(x) N_j'(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computing each local matrix for a 2noded element, I have &lt;span class=&quot;math-container&quot;&gt;$K^e(\xi)=\begin{pmatrix} 1/2 &amp;amp; -1/2 \\ -1/2 &amp;amp; 1/2 \end{pmatrix}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To compute its global equivalent, I use the substitution rule&#xA;&lt;span class=&quot;math-container&quot;&gt;$&#xA;\int_{\phi(a)=-1}^{\phi(b)=1} f(\xi)d\xi = \int_a^b f(\phi(x)) \phi'(x) dx&#xA;$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$\xi=\phi(x)=\frac{2}{h}(x-x_c)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\phi'(x)=\frac{2}{h}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So basically I have &lt;span class=&quot;math-container&quot;&gt;$K^e=K^g\frac{2}{h}$&lt;/span&gt; so &lt;span class=&quot;math-container&quot;&gt;$K^g=K^e\frac{h}{2}$&lt;/span&gt;.&#xA;Here this result is wrong, and I don't know where I missed up.&#xA;I'm supposed to have &lt;span class=&quot;math-container&quot;&gt;$K^g=K^e\frac{2}{h}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks :)&lt;/p&gt;&#xA;" OwnerUserId="29194" LastActivityDate="2018-11-11T00:59:18.573" Title="FEM 1D poisson substitution integral issue" Tags="|finite-element|integration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30516" PostTypeId="1" AcceptedAnswerId="30517" CreationDate="2018-11-10T15:34:06.583" Score="1" ViewCount="255" Body="&lt;p&gt;I am trying to run an MD simulation using this generalized version of Lennard Jones. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;U(r) = \left(\frac{r_0}{r}\right)^A -\frac{A}{B}\left( \frac{r_0}{r} \right)^B &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I do not know how to implement this potential in Lammps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been going through the manual and, as far as I cal tell, this potential isn't in there. All you can choose between is 9-6 or 12-6 for the exponents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully, one can get around the problem without extending any c++ classes. &lt;/p&gt;&#xA;" OwnerUserId="13098" LastActivityDate="2019-12-09T02:13:30.177" Title="Custom exponents for Lennard Jones in LAMMPS" Tags="|simulation|molecular-dynamics|lammps|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30517" PostTypeId="2" ParentId="30516" CreationDate="2018-11-10T17:31:57.990" Score="3" Body="&lt;p&gt;The Lennard-Jones potential is a special case of the &lt;a href=&quot;http://www.sklogwiki.org/SklogWiki/index.php/Mie_potential&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mie potential&lt;/a&gt;, which is &lt;a href=&quot;https://lammps.sandia.gov/doc/pair_mie.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;available in LAMMPS&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-11-10T17:31:57.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30518" PostTypeId="2" ParentId="30515" CreationDate="2018-11-11T00:59:18.573" Score="1" Body="&lt;p&gt;On &lt;span class=&quot;math-container&quot;&gt;$[x_i, x_{i+1}]$&lt;/span&gt;, you can write&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;N_i(x) = \frac{x_{i+1}-x}{h} = 1 - \xi = \phi_i(\xi) \quad \mbox{where } \;&#xA;\xi = \frac{x-x_i}{h}&#xA;\end{equation}&lt;/span&gt;&#xA;So the derivatives satisfy&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\frac{dN_i}{dx}(x) = - \frac{1}{h} = \frac{d\phi_i}{d\xi}\left(\xi (x) \right) \frac{d\xi}{dx}(x) = (-1) \left( \frac{1}{h} \right)&#xA;\end{equation}&lt;/span&gt;&#xA;So the integral becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\int_{x_i}^{x_i+h} N_i'(x) N_i'(x) dx = \int_{x_i}^{x_i+h} \left( - \frac{1}{h} \right) \left( - \frac{1}{h} \right) dx = \frac{1}{h}&#xA;\end{equation}&lt;/span&gt;&#xA;If we use the change of variables, we have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{multline}&#xA;\int_{x_i}^{x_i+h} N_i'(x) N_i'(x) dx = \int_{x_i}^{x_i+h} &#xA;\left( \frac{d\phi_i}{d\xi}\left(\xi (x) \right) \frac{d\xi}{dx}(x) \right)^2&#xA;dx&#xA;=&#xA;\int_{0}^{h} &#xA;\left( \frac{d\phi_i}{d\xi}\left( \xi \right) \frac{d\xi}{dx}( \xi ) \right)^2&#xA;\left( h d\xi \right)&#xA;\\&#xA;=&#xA;\int_{0}^{1} &#xA;\left[ \left( -1 \right) \left( \frac{1}{h} \right) \right]^2&#xA;\left( h d\xi \right)&#xA;=&#xA;\frac{1}{h}&#xA;\end{multline}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="7440" LastActivityDate="2018-11-11T00:59:18.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30524" PostTypeId="1" AcceptedAnswerId="30679" CreationDate="2018-11-11T19:11:15.633" Score="3" ViewCount="644" Body="&lt;p&gt;When solving &lt;span class=&quot;math-container&quot;&gt;$A*x=b$&lt;/span&gt; using preconditioned conjugate gradient methods one has to solve &lt;span class=&quot;math-container&quot;&gt;$z=K^{-1}*r$&lt;/span&gt; for the preconditioning where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is the preconditioner of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is the residual vector.&#xA;Instead of performing classical preconditioning like incomplete Cholesky or ILU one can also perform one V-cycle of algebraic multigrid method(AMG).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean that the V-cycle is applied to the equation &lt;span class=&quot;math-container&quot;&gt;$z=A^{-1}*r$&lt;/span&gt; to get &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have implemented for first testing a Ruge Stuben based AMG algorithm as a preconditioner for BiCGStab method. But I do not really see improvements in convergency compared to the classical preconditioners.&lt;/p&gt;&#xA;" OwnerUserId="28358" LastActivityDate="2018-12-09T18:28:25.647" Title="Multigrid preconditioner for conjugate gradient methods" Tags="|linear-algebra|preconditioning|multigrid|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30525" PostTypeId="2" ParentId="30224" CreationDate="2018-11-11T22:17:04.863" Score="1" Body="&lt;p&gt;I suggest trying &lt;a href=&quot;https://www.rdocumentation.org/packages/nloptr/versions/1.2.1/topics/bobyqa&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bound Optimization By Quadratic Approximation&lt;/a&gt; which is sometimes used for random-effect modeling.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-11T22:17:04.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30528" PostTypeId="2" ParentId="28111" CreationDate="2018-11-12T21:17:01.570" Score="7" Body="&lt;p&gt;The following paper suggests that the Jacobi-Davidson method can be used to target eigenvectors based on &quot;any property that can be computed from the eigenvector&quot;, which would seem to include overlap with a given vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://journals.aps.org/prb/abstract/10.1103/PhysRevB.66.245104&quot; rel=&quot;noreferrer&quot;&gt;https://journals.aps.org/prb/abstract/10.1103/PhysRevB.66.245104&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the paper, the key is just to reorder the QR decomposition by your desired property rather than by closeness to a target eigenvalue during each iteration of the Jacobi-Davidson algorithm. Thus, it seems like a minimal modification of existing Jacobi-Davidson codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of any codes that support this type of targeting built in, but would be quite interested in implementing/testing one.&lt;/p&gt;&#xA;" OwnerUserId="27918" LastActivityDate="2018-11-12T21:17:01.570" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30529" PostTypeId="1" AcceptedAnswerId="30534" CreationDate="2018-11-12T21:43:02.793" Score="6" ViewCount="411" Body="&lt;p&gt;Let's say, I have a compact area &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; (for example a circle, a square or some arbitrary polygon) and a function &lt;span class=&quot;math-container&quot;&gt;$f: S \rightarrow \mathbb{R}$&lt;/span&gt;. I want to numerically calculate the Integral&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \int_S f(\vec{x})\ \mathrm{d}x \approx \sum_i f(x_i)\Delta x_i $$&lt;/span&gt;&#xA;of the function over &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how to choose the sapling points &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; for &quot;complicated&quot; areas such as circles or shapes irregularly shaped borders. Can I simply overlay the aread with a equidistant lattice of sampling points and remove the ones not inside the area?&lt;/p&gt;&#xA;" OwnerUserId="28437" LastEditorUserId="28437" LastEditDate="2018-11-12T21:48:58.943" LastActivityDate="2018-11-13T19:42:55.370" Title="How do I integrate a function defined over an arbitrary area?" Tags="|discretization|quadrature|integration|numerics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30530" PostTypeId="2" ParentId="10708" CreationDate="2018-11-12T21:47:00.030" Score="4" Body="&lt;p&gt;You'd like to sort eigenvalues/eigenvectors in a way that is continuous as you move through momentum. This highly constrains the sorting - for most k-points, you must sort the eigenvalues to be a close as possible to the eigenvalues from a neighboring k-point. Only at eigenvalue crossings are you allowed to permute the order of bands. Furthermore, in many scenarios crossings don't actually occur - instead, with sufficient resolution, you would see an &quot;avoided crossing&quot;, and true continuity of the eigenvalues would require the eigenvalues to stay in increasing order throughout. In the case of avoided crossings, it may still make sense to allow the band assignment to swap, depending on your situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One situation where true crossings due occur is if your bands have different symmetry representations, and thus cannot mix. In this case, you can order your eigenvectors first by symmetry sector and then by increasing eigenvalues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you cannot use symmetry, then I'd suggest the following strategy:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pick a initial momentum point, and sort the eigenvectors at that point based on increasing energy.&#xA;For each additional momentum, match each eigenvector with one at the previous momentum point by finding the maximum overlap. If the momentum spacing is very small, these identifications should only allow changes in order at a level crossing.&lt;/p&gt;&#xA;" OwnerUserId="27918" LastActivityDate="2018-11-12T21:47:00.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30531" PostTypeId="1" CreationDate="2018-11-13T01:19:19.827" Score="1" ViewCount="252" Body="&lt;p&gt;The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a column matrix(im2col) and perform and perform Multiple Channel Multiple Kernel (MCMK) convolution using an existing parallel General Matrix Multiplication (GEMM) library. However im2col operation need load and store the image data, and also need another memory block to hold the intermediate data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I need to optimize the convolutional implementation, I may choose to direct implementation with SIMD instructions. Such method will not incur any memory operation overhead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;the benefits from the very regular patterns of memory access outweigh the wasteful storage costs.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the following link, at the end of the link&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I hope to know the reason. May floating point operations require more instruction cycle? or the input image is not much large, so it may residue in the cache and the memory operations don't need access DDR and consume less cycles.&lt;/p&gt;&#xA;" OwnerUserId="7846" LastActivityDate="2018-11-13T01:19:19.827" Title="Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN" Tags="|floating-point|efficiency|memory-management|neural-network|architecture|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30532" PostTypeId="1" AcceptedAnswerId="30549" CreationDate="2018-11-13T03:38:09.487" Score="4" ViewCount="216" Body="&lt;p&gt;I want to discretize the following problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{cases}&#xA;      \mu \nabla^2u+(\lambda+\mu)\nabla \nabla\cdot u = \rho \frac{\partial^2u  }{\partial t^2 } +  \beta \frac{\partial u}{\partial t}\\&#xA; u(x,y,t) = c, (x,y) \in \partial( [0,L_{x}]\times [0,L_{y}]), t \in [0,T]\\&#xA;u(x,y,0) = u_{0}(x,y), (x,y) \in [0,L_{x}]\times [0,L_{y}]\\&#xA;\frac{\partial u(x,y,0)}{\partial t} = u_{1}(x,y), (x,y) \in [0,L_{x}]\times [0,L_{y}]\\&#xA;\end{cases}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$T,L_{x},L_{y} &amp;gt;0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$c\in \mathbb{R}$&lt;/span&gt;&#xA;and  &lt;span class=&quot;math-container&quot;&gt;$$ u_{0},u_{1} \in C^2( [0,L_{x}]\times [0,L_{y}], \mathbb{R^2} ) $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$u \in C^2( [0,L_{x}]\times [0,L_{y}]\times [0,T], \mathbb{R^2} ) $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My attempt. &#xA;Let &lt;span class=&quot;math-container&quot;&gt;$u = ( u_{x} \ u_{y} )^T$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mu \nabla^2u = ( \mu \nabla^2 u_{x} \ \  \mu \nabla^2u_{y} )^T =  $$&lt;/span&gt; &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mu \begin{pmatrix}&#xA;\nabla^2 &amp;amp; 0 \\&#xA;0 &amp;amp; \nabla^2&#xA;\end{pmatrix}\begin{pmatrix} u_{x}\\ u_{y} \end{pmatrix} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the laplacian in the Kronecker product is: &lt;span class=&quot;math-container&quot;&gt;$$ \nabla^2 = L_{2}\otimes I_{n_{x}} + I_{n_{y}} \otimes L_{2} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} &#xA;   L_{2} = \frac{1}{h^2}\left[\begin{matrix}&#xA;-2 &amp;amp; 1 &amp;amp; &amp;amp; 0\\&#xA;1 &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \\&#xA;&amp;amp; \ddots &amp;amp; \ddots &amp;amp; 1 \\&#xA;0 &amp;amp; &amp;amp; 1 &amp;amp; -2 \end{matrix} \right]&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the step size or mesh size... Now &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ (\lambda+\mu)\nabla \nabla \cdot u =  $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \begin{pmatrix} (\lambda+\mu)\frac{\partial^2 u_{x}}{\partial x^2} + (\lambda+\mu)\frac{\partial^2 u_{y}}{\partial x \partial y}\\&#xA;(\lambda+\mu)\frac{\partial^2 u_{x}}{\partial y \partial x}+(\lambda+\mu)\frac{\partial^2 u_{y}}{\partial y^2} &#xA;\end{pmatrix} = $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ (\lambda+\mu) \begin{pmatrix} \frac{\partial^2 }{\partial x^2}  &amp;amp; \frac{\partial^2 }{\partial x \partial y} \\ &#xA;\frac{\partial^2 }{\partial y \partial x} &amp;amp;  \frac{\partial^2 }{\partial y^2} &#xA;   \end{pmatrix} \begin{pmatrix} u_{x} \\ u_{y}  \end{pmatrix} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial^2 }{\partial x^2} = L_{2}\otimes I_{n_{x}} $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial^2 }{\partial y^2} =  I_{n_{y}} \otimes L_{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how do I represent the following operators in terms of &lt;span class=&quot;math-container&quot;&gt;$L_{2}, I, \otimes$&lt;/span&gt; or is not possible ? if it is not possible, what is the matrix representation of...?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial^2 }{\partial x \partial y} = \textbf{ ? }$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial^2 }{\partial y \partial x} = \textbf{ ? }$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: PHYSICAL INTERPRETATION&#xA;This is known as the wave equation in elastodynamic (a more general form)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; are the Lame’s coefficients given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mu = \frac{E}{2(1+\upsilon)}  $$&lt;/span&gt; and &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \lambda = \frac{E\upsilon}{(1+\upsilon)(1-2\upsilon)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; is the modulus of elasticity (Young modulus) and &lt;span class=&quot;math-container&quot;&gt;$\upsilon$&lt;/span&gt; is the&#xA;Poisson’s ratio of the elastic material. &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is the linear density.&lt;/p&gt;&#xA;" OwnerUserId="28594" LastEditorUserId="28594" LastEditDate="2018-11-13T13:44:43.903" LastActivityDate="2018-11-16T14:39:49.497" Title="Matrix Representation of a Discretization for a Partial Differential Equation" Tags="|pde|finite-difference|discretization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30533" PostTypeId="1" CreationDate="2018-11-13T09:38:34.117" Score="-1" ViewCount="30" Body="&lt;p&gt;I am implementing an optimization problem using semi-definite approach. One of my constraints is of following form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ trace(A∗X)−(k∗trace(A∗X))+(k∗\sqrt {(trace(B∗X)} )==0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where k is a constant, A and B are constant complex hermitian matrices, &lt;span class=&quot;math-container&quot;&gt;$B=A^2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$m∗m$&lt;/span&gt; complex, hermitian semi-definite matrix optimization variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be seen that the last terms &lt;span class=&quot;math-container&quot;&gt;$\sqrt {(trace(B∗X)}$&lt;/span&gt; is a problematic term. This terms appears in the equation when I introduced one special electrical load (In the absence of that load, the constraint is reduced to first two terms which I have already solved successfully). Since, I am writing my code using YALMIP, which is an external toolbox of MATLAB, sqrt on complex variable is not allowed in that toolbox. Is there any way to rewrite that particular term using some convex optimization tools so that, I can get rid of square root and obtain a linear equation again in terms of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="28952" LastActivityDate="2018-11-18T19:12:53.437" Title="Semi-Definite relaxation of non-linear constraint?" Tags="|nonlinear-equations|convex-optimization|constraints|semidefinite-programming|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30534" PostTypeId="2" ParentId="30529" CreationDate="2018-11-13T15:59:28.750" Score="7" Body="&lt;p&gt;Instead of directly integrating over the area, it is often more convenient&#xA;to use the&#xA;divergence theorem to replace the area integral with an integral over the&#xA;boundary edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The divergence theorem in three dimensions for a&#xA;vector function &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; can be written&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int_V \nabla \cdot {\bf F} dv = \int_S {\bf F} \cdot ds&#xA;$$&lt;/span&gt;&#xA;That is, an integral over the volume of the region can be replaced by&#xA;an integral over the bounding surface.&#xA;For a scalar function, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, in two dimensions this can be simplified to&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int_A \frac{\partial f}{\partial x} dx dy = \int_C f ds&#xA;$$&lt;/span&gt;&#xA;relating an integral over the area to a line integral over the curve&#xA;bounding the area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you may have many separate edges bounding the area. You&#xA;integrate over each edge and sum the results. Each edge may have a&#xA;different geometric form. For simple analytic curves (e.g. a straight line&#xA;or circular arc), it may be possible to evaluate the integral analytically.&#xA;For more complicated parametric forms, it is necessary to use numerical&#xA;integration to evaluate the line integral.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2018-11-13T19:42:55.370" LastActivityDate="2018-11-13T19:42:55.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30539" PostTypeId="1" CreationDate="2018-11-14T17:23:38.573" Score="5" ViewCount="350" Body="&lt;p&gt;I am trying to factorize the following Laplacian matrix in terms of &lt;span class=&quot;math-container&quot;&gt;$ D^TD$&lt;/span&gt;, D is the first derivative matrix.&#xA;The tridiagonal form of the secon derivative matrix using Neumann boundary condition is given by,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\begin{bmatrix} -1 &amp;amp; 1 &amp;amp; 0  &amp;amp; 0 &amp;amp; 0 \\ 1 &amp;amp; -2 &amp;amp; 1 &amp;amp; 0 &amp;amp;0\\ 0 &amp;amp; 1 &amp;amp; -2 &amp;amp;1&amp;amp;0\\0&amp;amp;0&amp;amp;1&amp;amp;-2&amp;amp;1\\0&amp;amp;0&amp;amp;0&amp;amp;1&amp;amp;-1\end{bmatrix}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to write the above in terms of the gradient( first derivative) operator,  I tried factorizing the Laplacian using Cholesky factorization. But, I couldn't succeed in factorizing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to find a decomposition of the form &lt;span class=&quot;math-container&quot;&gt;$D^TD$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="29087" LastEditorUserId="29087" LastEditDate="2018-11-14T17:28:38.670" LastActivityDate="2024-02-08T08:14:03.937" Title="Factorize laplacian in terms of first derivative matrix" Tags="|finite-difference|graph-theory|matrix-factorization|laplacian|" AnswerCount="2" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="30540" PostTypeId="1" CreationDate="2018-11-14T17:14:40.320" Score="1" ViewCount="44" Body="&lt;p&gt;I'm trying to solve the following system of differential equations numerically. What are the available finite difference approaches and matlab solvers to solve such a system? Other approaches to solve the problem are welcome as well. The first two equations are quasi-steady and the rest of the equations have time derivatives in them. I'm trying using matlab's pdepe solver, but since the boundary conditions have to be specified even for the ODEs which have time-derivatives, I didn't think it as a viable option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$0= \rho \frac{\partial}{\partial z}\left(D \frac{\partial c_1}{\partial z} \right) - \it{source_1}\, $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ 0= \rho \frac{\partial}{\partial z}\left(D \frac{\partial c_2}{\partial z} \right) - \it{source_2}\,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial c_3}{\partial t} = source_3$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial c_4}{\partial t} = source_4$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$D = A_1 c_1  + A_2 c_2 + \dots + A_n c_n \,, &#xA;\rho = 1 (say)\,.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source terms for any particular equation (or concentration) contain dependence on the other concentrations. &lt;/p&gt;&#xA;" OwnerUserId="29233" OwnerDisplayName="mnatch" LastActivityDate="2018-11-14T19:54:31.500" Title="Numerically solving a system of parabolic PDEs and 1st order ODEs" Tags="|computational-physics|differential-equations|numerics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30542" PostTypeId="1" CreationDate="2018-11-15T16:14:18.407" Score="1" ViewCount="87" Body="&lt;p&gt;I need to compute a matrix-matrix product, &lt;span class=&quot;math-container&quot;&gt;$A^T B$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$n \times r$&lt;/span&gt; sparse, and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$n \times q$&lt;/span&gt; dense. The number of rows &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is far larger than both &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;. In fact &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is so large I cannot store the entire matrix &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; in memory, so I build it one row at a time and update the matrix product &lt;span class=&quot;math-container&quot;&gt;$A^T B$&lt;/span&gt; accordingly. The way I currently do this is to store &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; in CSR format and then apply the following algorithm:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loop i = 1, ..., n     // loop over rows of B&#xA;  bi = build_row(B, i) // construct row i of B&#xA;  loop p = A-&amp;gt;row_ptr[i], ..., A-&amp;gt;row_ptr[i+1] - 1 // loop over non-zero entries in row i of A&#xA;    DAXPY(A-&amp;gt;data[p], bi, ATB(A-&amp;gt;col_ind[p], :))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This algorithm seems to have complexity &lt;span class=&quot;math-container&quot;&gt;$O(nkq)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of non-zero entries in a row of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (this is the same for all rows of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;). This algorithm works fine, but is quite slow so I am looking for ideas on how to speed it up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dense BLAS algorithms often take advantage of blocks of data instead of operating on one row at a time, so my thinking was to do the same (instead of building B one row at a time, build a block of rows, where the block size is still small enough to fit in memory, and then apply a matrix multiplication algorithm). For this type of method, I don't think CSR format would work, since we would need the columns of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; in sequential memory locations, so CSC format might work better. A possible algorithm (with A in CSC format) is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loop i = 1, ..., num_blocks&#xA;  row0 = (i-1) * block_size           // row of start of block in B&#xA;  row1 = row0 + block_size            // row of end of block in B&#xA;  blockB = build_block(B, row0, row1) // build B(row0:row1,:)&#xA;  loop j = 1, ..., r // loop over rows of A^T&#xA;    loop p = A-&amp;gt;col_ptr[j], ..., A-&amp;gt;col_ptr[j+1]-1 // loop over non-zero entries in row j of A^T&#xA;      if (row0 &amp;lt;= A-&amp;gt;row_ind[p]) AND (A-&amp;gt;row_ind[p] &amp;lt;= row1) // if this row of A is in the right range, perform DAXPY operation&#xA;        DAXPY(A-&amp;gt;data[p], blockB(A-&amp;gt;row_ind[p]-row0, :), ATB(j, :))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think the above algorithm has complexity &lt;span class=&quot;math-container&quot;&gt;$O(k r q \times NB)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of non-zero entries in a column of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$NB$&lt;/span&gt; is the number of blocks (num_blocks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't yet coded up this method, since I'm not sure it will actually be faster. Does anyone have any suggestions on how to make a fast algorithm to do this? Perhaps there is another sparse matrix storage format which would be better to use?&lt;/p&gt;&#xA;" OwnerUserId="28440" LastActivityDate="2018-11-15T16:14:18.407" Title="Fast calculation of $A^T B$" Tags="|matrix|sparse-matrix|blas|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30545" PostTypeId="1" CreationDate="2018-11-15T19:14:56.287" Score="0" ViewCount="185" Body="&lt;p&gt;For an ODE: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{dy}{dt}=f(y(t),t)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Euler Explicit scheme reads:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y_{n+1}=y_{n}+\Delta tf_n$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and it can be easily shown with a Taylor expension that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y_{n+1}=y_{n}+\Delta t \frac{dy}{dt}|_n+\Delta t^2 \frac{d^2y}{dt^2}|_n$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and therefore by comparing the scheme with the Taylor expansion we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$LTE= O(\Delta t^2)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e. the local truncation error LTE for Euler explicit scales with &lt;span class=&quot;math-container&quot;&gt;$\Delta t^2$&lt;/span&gt;. Since we repeat this for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; times, with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; the number of grid points, the global truncation error is &lt;span class=&quot;math-container&quot;&gt;$n\Delta t^2$&lt;/span&gt;. Since &lt;span class=&quot;math-container&quot;&gt;$n=L/\Delta t$&lt;/span&gt;, with L the domain length, then the global truncation error scales with Δt, so Euler explicit is first order accurate.&#xA;For an elliptic PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial^2\phi}{\partial x^2}+\frac{\partial^2\phi}{\partial y^2}=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if I use the central difference scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial^2\phi}{\partial x^2}= \frac{\phi_{i-1,j}-2\phi_{i,j}+\phi_{i+1,j}}{dx^2}+O(\Delta x^2)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the error is second order as it can be shown by Taylor expanding &lt;span class=&quot;math-container&quot;&gt;$\phi_{i+1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\phi_{i-1}$&lt;/span&gt; in the neighborhood of &lt;span class=&quot;math-container&quot;&gt;$\phi_{i}$&lt;/span&gt;. Similar reasoning in &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial^2\phi}{\partial y^2}= \frac{\phi_{i,j-1}-2\phi_{i,j}+\phi_{i,j+1}}{dy^2}+O(\Delta y^2)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is were I am lost. The error in the derivative is second order, but what about the error in the function &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; I am looking for? If I replace the discretization into the equation (let's assume &lt;span class=&quot;math-container&quot;&gt;$\Delta y = \Delta x$&lt;/span&gt; for now) we have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\phi_{i,j-1}-2\phi_{i,j}+\phi_{i,j+1}}{dx^2}+ \frac{\phi_{i-$1,j}-2\phi_{i,j}+\phi_{i+1,j}}{dx^2}+2O(\Delta x^2)=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which, multiplying by &lt;span class=&quot;math-container&quot;&gt;$\Delta x^2$&lt;/span&gt; reads: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\phi_{i,j-1}-4\phi_{i,j}+\phi_{i,j+1}+ \phi_{i,i-1,j}+\phi_{i+1,j}+2O(\Delta x^4)=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;span class=&quot;math-container&quot;&gt;$O(\Delta x^4)$&lt;/span&gt; shows up. From here, how can I prove that &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; is obtained with second order accuracy? I guess the Local error for &lt;span class=&quot;math-container&quot;&gt;$\phi_{j}$&lt;/span&gt; is 4th order (if all the 4 neighbours were known exactly) , but the global error is second error, I just do not know how to prove it. Any suggestion?&lt;/p&gt;&#xA;" OwnerUserId="10032" LastActivityDate="2018-11-15T22:48:52.520" Title="Elliptic PDE: Proving that a 2nd order accurate discretization of the 2nd derivative of the unknown is 2nd order accurate for the unknown itself" Tags="|finite-difference|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30546" PostTypeId="2" ParentId="30545" CreationDate="2018-11-15T22:48:52.520" Score="2" Body="&lt;p&gt;Consider as a simple case the 1D Poisson problem (it should be simple to extend to the 2D case you describe):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \tag{$*$}\label{eq:poisson}&#xA;  \frac{d^2 \phi}{dx^2}(x) = f(x), \quad 0&amp;lt;x&amp;lt;1, \qquad \phi(0) = \phi(1) = 0,&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is given. We approximate&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{d^2 \phi}{dx^2}(x_i) = \frac{\phi_{i-1} - 2\phi_i + \phi_{i+1}}{h^2} + \mathcal{O}(h^2).&#xA;$$&lt;/span&gt;&#xA;Let &lt;span class=&quot;math-container&quot;&gt;$\phi(x)$&lt;/span&gt; denote the exact solution to the problem &lt;span class=&quot;math-container&quot;&gt;$(*)$&lt;/span&gt;.&#xA;Suppose &lt;span class=&quot;math-container&quot;&gt;$\tilde{\phi}$&lt;/span&gt; is the numerical solution obtained by solving the system of equations&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\tilde\phi_{i-1} - 2\tilde\phi_i + \tilde\phi_{i+1}}{h^2} = f_i.&#xA;$$&lt;/span&gt;&#xA;We now write everything in terms of linear algebra. Let &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol\phi$&lt;/span&gt; denote the vector of values &lt;span class=&quot;math-container&quot;&gt;$\phi(x_i)$&lt;/span&gt;, let &lt;span class=&quot;math-container&quot;&gt;$\tilde{\boldsymbol\phi}$&lt;/span&gt; denote the vector &lt;span class=&quot;math-container&quot;&gt;$\tilde\phi_i$&lt;/span&gt;, and let &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol f$&lt;/span&gt; denote the vector &lt;span class=&quot;math-container&quot;&gt;$f(x_i)$&lt;/span&gt;. Finally, let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; denote the matrix corresponding to the finite-difference Laplacian. For the discrete approximate solution, we have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A \tilde{\boldsymbol\phi} = \boldsymbol f.&#xA;$$&lt;/span&gt;&#xA;Using the truncation error for the finite-difference approximation, we have for the exact solution&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A \boldsymbol\phi = \boldsymbol f + \boldsymbol \tau, \qquad&#xA;\tau_i = \mathcal{O}(h^2).&#xA;$$&lt;/span&gt;&#xA;We want to measure the error in our approximate solution, which is given by &lt;span class=&quot;math-container&quot;&gt;$\| \tilde{\boldsymbol\phi} - \boldsymbol\phi \|$&lt;/span&gt; in some norm. We see that the difference &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol\varepsilon = \tilde{\boldsymbol\phi} - \boldsymbol\phi$&lt;/span&gt; satisfies&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A \boldsymbol\varepsilon = A (\tilde{\boldsymbol\phi} -\boldsymbol\phi)= -\boldsymbol \tau&#xA;$$&lt;/span&gt;&#xA;and so&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\boldsymbol\varepsilon = -A^{-1}\boldsymbol \tau.&#xA;$$&lt;/span&gt;&#xA;In particular&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\| \boldsymbol\varepsilon \| \leq \|A^{-1}\| \|\boldsymbol\tau\|&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\|A^{-1}\|$&lt;/span&gt; is the corresponding matrix norm. If we can bound the norm of &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt; by a constant, then we can conclude that &lt;span class=&quot;math-container&quot;&gt;$\|\boldsymbol\varepsilon\| = \mathcal{O}(h^2)$&lt;/span&gt;. If we use the 2-norm and using that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is symmetric, we see that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA; \| A^{-1} \|_2 = \frac{1}{\min |\lambda|}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is an eigenvalue of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Using an explicit expression for the eigenvalues of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, the smallest in magnitude is given by &lt;span class=&quot;math-container&quot;&gt;$\lambda = -\pi^2 + \mathcal{O}(h^2)$&lt;/span&gt; and the result follows.&lt;/p&gt;&#xA;" OwnerUserId="25388" LastActivityDate="2018-11-15T22:48:52.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30549" PostTypeId="2" ParentId="30532" CreationDate="2018-11-16T14:39:49.497" Score="2" Body="&lt;p&gt;In 1D,  if we define &lt;span class=&quot;math-container&quot;&gt;$u_{k} := u(x_{k}) $&lt;/span&gt;;    &lt;span class=&quot;math-container&quot;&gt;$\ \  x_{k} = kh $&lt;/span&gt;  and &lt;span class=&quot;math-container&quot;&gt;$ \ \ k = 0,1,2,...,N$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is known as the mesh size or step size.&lt;br&gt;&#xA;I want to approximate the first derivative using central difference:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{du_{k}}{dx} \approx \frac{ u_{k+1}-u_{k-1}  }{ 2h } = \frac{ -u_{k-1} +u_{k+1}  }{ 2h }   $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this problem (1D) if  &lt;span class=&quot;math-container&quot;&gt;$u(0) = u_{0} = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ u(L) = u(x_{N}) = u_{N} = 0 $&lt;/span&gt; we get the following matrix representation of the first derivative  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} &#xA;   \frac{d}{dx} \approx L_{1} = \frac{1}{2h}\left(\begin{matrix}&#xA;0 &amp;amp; 1 &amp;amp; &amp;amp; 0\\&#xA;-1 &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \\&#xA;&amp;amp; \ddots &amp;amp; \ddots &amp;amp; 1 \\&#xA;0 &amp;amp; &amp;amp; -1 &amp;amp; 0 \end{matrix} \right)&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now using &lt;span class=&quot;math-container&quot;&gt;$L_{1}$&lt;/span&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial^2 }{\partial x \partial y} = \Big((L_{1})_{n_{x}} \otimes I_{n_{y}} \Big)\Big( I_{n_{x}} \otimes (L_{1})_{n_{y}}   \Big) = { (L_{1})_{n_{x}} \otimes (L_{1})_{n_{y}}  }  $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used the mixed-product property for kronecker product. &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ (\mathbf{A} \otimes \mathbf{B})(\mathbf{C} \otimes \mathbf{D}) = (\mathbf{AC}) \otimes (\mathbf{BD})$$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a similar way we get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial^2 }{\partial y \partial x} =  (L_{1})_{ n_{y} }   \otimes  (L_{1})_{n_{x}}  $$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="28594" LastActivityDate="2018-11-16T14:39:49.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30550" PostTypeId="1" CreationDate="2018-11-16T23:53:48.760" Score="3" ViewCount="69" Body="&lt;p&gt;I am working on determining the spatial information of a set of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; points in &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;-dimensional space. It is well-known that the construction of Delaunay triangulation is expensive in high dimensional space. Thus, I would like to only find the largest simplex of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;-dimensional Delaunay triangulation (DT). I am going to sample a new point inside that simplex by each iteration, which means that the largest simplex would become smaller and smaller by iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any idea or references to computing the largest simplex in DT? It may not be exactly the largest simplex after computing DT. But it should at least (1) approximate the largest simplex within a tolerance; (2) the largest simplex that this method computed should become smaller and smaller by each iteration.&lt;/p&gt;&#xA;" OwnerUserId="29252" LastEditorUserId="20688" LastEditDate="2018-11-17T03:09:01.027" LastActivityDate="2018-11-17T03:09:01.027" Title="Approximate the largest simplex in N-dimensional Delaunay triangulation" Tags="|computational-geometry|delaunay-triangulation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30551" PostTypeId="1" AcceptedAnswerId="30552" CreationDate="2018-11-17T18:19:57.843" Score="3" ViewCount="2570" Body="&lt;p&gt;the classical Newton's method for non-linear systems of equations is &lt;span class=&quot;math-container&quot;&gt;$x_{k+1} =x_k-J_F(x_n)^{-1} F(x_n)$&lt;/span&gt;. In pratice, rather than compute the inverse of the Jacobian matrix, one solves the systems &lt;span class=&quot;math-container&quot;&gt;$J_F(x_k) (x_{k+1}-x_k)=-F(x_k)$&lt;/span&gt;, for the unknown &lt;span class=&quot;math-container&quot;&gt;$x_{k+1}-x_k$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my notes (about ODE) I found: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Newton's method requires the computation of the Jacobian matrix and its &quot;inversion&quot; at every step &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. This could be too expensive (&lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N^3)$&lt;/span&gt;), where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the dimension of the matrix.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My doubt is how to get that computational complexity. Is it talking about the way to invert a matrix using LU decomposition, which I know to be &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N^3)$&lt;/span&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then it states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A standard way to reduce computational complexity is to use always the same Jacobian matrix, compute its LU decomposition and use it to solve the linear systems. This is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N^2)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here I have still a question: the complexity of the &lt;em&gt;computation&lt;/em&gt; of the LU decomposition of &lt;span class=&quot;math-container&quot;&gt;$J_F$&lt;/span&gt; should be &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\frac{N^3}{3})$&lt;/span&gt;. While the computational complexity of the resolution of a triangular system is &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\frac{N^2}{2})$&lt;/span&gt;. Since there are two triangular systems, it amounts to &lt;span class=&quot;math-container&quot;&gt;$2 \mathcal{O}(\frac{N^2}{2})$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shouldn't it be, totally, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\frac{N^3}{3})$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(N^2)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="26296" LastActivityDate="2018-11-17T18:29:51.780" Title="Computational complexity of Newton's method" Tags="|newton-method|complexity|numerics|quasi-newton|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30552" PostTypeId="2" ParentId="30551" CreationDate="2018-11-17T18:29:51.780" Score="4" Body="&lt;p&gt;If you take &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; steps, and update the Jacobian every &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; steps, the time complexity will be &lt;span class=&quot;math-container&quot;&gt;$O(m N^2 + (m/t)N^3)$&lt;/span&gt;. So the time taken per step is &lt;span class=&quot;math-container&quot;&gt;$O(N^2+N^3/t)$&lt;/span&gt;. You're reducing the amount of work you do by a factor of &lt;span class=&quot;math-container&quot;&gt;$1/t$&lt;/span&gt;, and it's &lt;span class=&quot;math-container&quot;&gt;$O(N^2)$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$t\geq N$&lt;/span&gt;. But &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is determined adaptively by the behaviour of the loss function, so the point is just that you're saving some unknown, significant amount of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the quote, &quot;this&quot; probably refers to the immediately preceding sentence, the complexity of solving an already-factored linear system, not to the time taken for the whole step like in the paragraph before it.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-11-17T18:29:51.780" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30554" PostTypeId="1" CreationDate="2018-11-18T13:28:38.500" Score="3" ViewCount="117" Body="&lt;p&gt;Lanczos/Arnoldi/Rietz/CG-like algorithm share the same core strategy...&#xA;In each, a little miracle appears, most of the Gram-Schmidt inner products are zeroes! In others words, new direction need only get orthonormalized with the one previous updated search direction (or two in Lanczos case), no need to check other previous directions (because of zeros). Rather a big miracle when &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is sparse and &lt;span class=&quot;math-container&quot;&gt;$n = 10^6$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of non-linear inverse/eigenproblem, this property is not true, but depending of the &quot;nonlinearity rate&quot;, the values could be small and easily neglected or threshold since the iterative nature of the algorithm assumed that the residual must decrease in each step. So this property remains useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions are welcome for understanding why the new direction is &quot;already&quot; orthogonal to the set of all (previous+1) directions.&#xA;Math is ok, but there is a deep principle behind this thing.&#xA;It looks like a hidden tridiagonalization process that happens at the same time. Is there an easy way to handle the trick?&lt;/p&gt;&#xA;" OwnerUserId="17773" LastEditorUserId="20688" LastEditDate="2018-11-19T01:44:23.837" LastActivityDate="2018-11-19T01:44:23.837" Title="Why the MIRACLE of Lanczos/CG-like?" Tags="|linear-algebra|sparse-matrix|eigenvalues|convex-optimization|numerics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30555" PostTypeId="2" ParentId="30533" CreationDate="2018-11-18T19:12:53.437" Score="0" Body="&lt;p&gt;square both side ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(1-k)^2*trace(A*X)^2 == k^2*trace(B*X)&#xA;(1-k)^2*trace(A*X)^2 -  k^2*trace(B*X) == 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;move from SDP to QCQP.&#xA;Or move to unconstrained problem with last term penalized with quadratic error (so X^4 minimization) or abs error&lt;/p&gt;&#xA;" OwnerUserId="17773" LastActivityDate="2018-11-18T19:12:53.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30556" PostTypeId="2" ParentId="28177" CreationDate="2018-11-19T03:05:33.120" Score="3" Body="&lt;p&gt;This metric is pretty much as misleading (or useful, depending on your perspective) for GPUs as it is for CPUs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, a lot of applications/algorithm's implementations are limited more by memory throughput rather than FLOPs. Memory throughput (in GB/s) is also always listed for GPU specification and those two numbers together give a much better view on the expected GPU performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of that will also apply for CPU FLOPs metric. However, in my opinion, FLOPs metric critique is even more applicable to GPUs since they are much more specialized and oriented on parallelizable computations, which can produce even bigger dependence on the particular algorithm implementation and data organization patterns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, can FLOPS be misleading in terms of GPU performance? Sure. Is there a reason to have them? Yes, because it gives at least &lt;strong&gt;some&lt;/strong&gt; information which is even more complete with taking memory throughput into the subsequent analysis of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the underlying problem&lt;/li&gt;&#xA;&lt;li&gt;the chosen algorithm and implementation&lt;/li&gt;&#xA;&lt;li&gt;the chosen strategy of data organization and manipulation&lt;/li&gt;&#xA;&lt;li&gt;other GPU characteristics (possible parallelization mode across several GPU units, architecture and instruction support, etc.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-19T03:05:33.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30558" PostTypeId="1" CreationDate="2018-11-19T13:46:48.533" Score="1" ViewCount="464" Body="&lt;p&gt;I'm searching for lower bounds of bilinear forms arising in FEM for elliptic second order PDEs with mixed boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some research and found:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\max_{v_{h}\in\mathcal{V}_h(\mathcal{\Omega})}a(v_{h}, v_{h}) \geq Ch^{d-2}\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the Dimension of the domain, &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; a positive constant and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the mesh size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bilinear form is defined as &lt;span class=&quot;math-container&quot;&gt;$a(v_h,v_h):= \int_\Omega \nabla v_h A \nabla v_h \, d\Omega$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is an elliptic operator. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this bound quite often but not a single proof to it. Does anyone know some literature or how the proof works?&lt;/p&gt;&#xA;" OwnerDisplayName="user29088" LastEditorDisplayName="user29088" LastEditDate="2018-11-20T09:47:25.697" LastActivityDate="2022-08-20T02:07:45.787" Title="Lower bound for bilinear form in FEM" Tags="|finite-element|error-estimation|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="30559" PostTypeId="1" CreationDate="2018-11-19T14:40:19.633" Score="3" ViewCount="907" Body="&lt;p&gt;I work in Python 3 on astrophysics projects. I need to compute the smallest circumscribed circle of a set of points in the sky (so described by Right Ascension and Declination). I have found a code &lt;a href=&quot;https://www.nayuki.io/res/smallest-enclosing-circle/smallestenclosingcircle.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (real-time demo &lt;a href=&quot;https://www.nayuki.io/page/smallest-enclosing-circle&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It implements an algorithm in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(n)$&lt;/span&gt; instead of the naive &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(n^4)$&lt;/span&gt; one, which is very interesting for me as I have large datasets. But it is written in the Euclidean plane. I have transformed it as much as possible to match spherical trigonometry on the sky, but there is one part I don't know how to convert. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current code is then a mix of spherical geometry and plane approximation. Unfortunately, although my objects are very close in the sky (they are galaxies in compact groups), this approximation is not good enough and I need the fully-spherical computation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hereafter my modified code, with as many comments as possible (&lt;code&gt;Original code&lt;/code&gt;, &lt;code&gt;Spherical code&lt;/code&gt;, and so on) to describe what I have done and the part where I don't know what to do. It uses import &lt;code&gt;numpy as np&lt;/code&gt; and &lt;code&gt;import math&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Circumscribed circle - PARTIALLY ADAPTED TO SPHERICAL GEOMETRY! &#xA;# From https://www.nayuki.io/res/smallest-enclosing-circle/smallestenclosingcircle.py&#xA;# Demo on https://www.nayuki.io/page/smallest-enclosing-circle&#xA;def calc_sep1(a,b):&#xA;    (phi1,theta1)=(a[0],a[1])&#xA;    (phi2,theta2)=(b[0],b[1])&#xA;&#xA;    return math.acos(min(1,math.sin(theta1)*math.sin(theta2) + &#xA;                     math.cos(theta1)*math.cos(theta2)*math.cos(phi2 - phi1)) )&#xA;&#xA;#Defines barycenter of a triangle&#xA;def calc_bary1(a,b,c):&#xA;    phi = [a[0],b[0],c[0]]&#xA;    theta = [a[1],b[1],c[1]]&#xA;    x = np.cos(theta) * np.cos(phi)&#xA;    y = np.cos(theta) * np.sin(phi)&#xA;    z = np.sin(theta)&#xA;&#xA;    x_O, y_O, z_O = np.mean(x), np.mean(y), np.mean(z) &#xA;&#xA;    l = np.sqrt(x_O**2 + y_O**2 + z_O**2)&#xA;&#xA;    (x_M, y_M, z_M) = (x_O, y_O, z_O)/l&#xA;&#xA;    Dec_M = np.arcsin(z_M)&#xA;    if (y_M &amp;lt;0):&#xA;        RA_M = 2.0*np.pi - np.arccos(x_M / np.cos(Dec_M))&#xA;    else:&#xA;        RA_M = np.arccos(x_M / np.cos(Dec_M))&#xA;&#xA;    return (RA_M,Dec_M)&#xA;&#xA;def midpoint(a,b):&#xA;    (phi1,theta1)=(a[0],a[1])&#xA;    (phi2,theta2)=(b[0],b[1])&#xA;&#xA;    Bx = math.cos(theta2)*math.cos(phi2-phi1)&#xA;    By = math.cos(theta2)*math.sin(phi2-phi1)&#xA;    theta_m = math.atan2(math.sin(theta1) + math.sin(theta2), math.sqrt((math.cos(theta1)+Bx)**2 + By**2) )&#xA;    phi_m = phi1 + math.atan2(By, math.cos(theta1)+Bx)&#xA;&#xA;    return (phi_m,theta_m)&#xA;&#xA;# Data conventions: A point is a pair of floats (x, y). &#xA;#                   A circle is a triple of floats (center x, center y, radius).&#xA;&#xA;# Returns the smallest circle that encloses all the given points. Runs in expected O(n) time, randomized.&#xA;# Input: A sequence of pairs of floats or ints, e.g. [(0,5), (3.1,-2.7)].&#xA;# Output: A triple of floats representing a circle.&#xA;# Note: If 0 points are given, None is returned. If 1 point is given, a circle of radius 0 is returned.&#xA;# &#xA;# Initially: No boundary points known&#xA;def make_circle(points):&#xA;    # Randomize order&#xA;    # Carfull: inpuot and outpu in degrees, inside: radiants&#xA;    shuffled = [(d2r(row['RA']), d2r(row['Dec'])) for index, row in points.iterrows()]&#xA;    random.shuffle(shuffled)   &#xA;    # Progressively add points to circle or recompute circle&#xA;    c = None&#xA;    for (i, p) in enumerate(shuffled):&#xA;        if c is None or not is_in_circle(c, p):&#xA;            c = _make_circle_one_point(shuffled[ : i + 1], p)&#xA;    return (r2d(c[0]),r2d(c[1]),c[2])&#xA;&#xA;&#xA;# One boundary point known&#xA;def _make_circle_one_point(points, p):&#xA;    c = (p[0], p[1], 0.0)&#xA;    for (i, q) in enumerate(points):&#xA;        if not is_in_circle(c, q):&#xA;            if c[2] == 0.0:&#xA;                c = make_diameter(p, q)&#xA;            else:&#xA;                c = _make_circle_two_points(points[ : i + 1], p, q)&#xA;    return c&#xA;&#xA;&#xA;# Two boundary points known&#xA;def _make_circle_two_points(points, p, q):&#xA;    circ = make_diameter(p, q)&#xA;    left  = None&#xA;    right = None&#xA;    px, py = p&#xA;    qx, qy = q&#xA;&#xA;    # For each point not in the two-point circle&#xA;    for r in points:&#xA;        if is_in_circle(circ, r):&#xA;            continue&#xA;&#xA;        # Form a circumcircle and classify it on left or right side&#xA;        cross = _cross_product(px, py, qx, qy, r[0], r[1])&#xA;        c = make_circumcircle(p, q, r)&#xA;        if c is None:&#xA;            continue&#xA;        elif cross &amp;gt; 0.0 and (left is None or _cross_product(px, py, qx, qy, c[0], c[1]) &amp;gt; &#xA;                              _cross_product(px, py, qx, qy, left[0], left[1])):&#xA;            left = c&#xA;        elif cross &amp;lt; 0.0 and (right is None or _cross_product(px, py, qx, qy, c[0], c[1]) &amp;lt; &#xA;                              _cross_product(px, py, qx, qy, right[0], right[1])):&#xA;            right = c&#xA;&#xA;    # Select which circle to return&#xA;    if left is None and right is None:&#xA;        return circ&#xA;    elif left is None:&#xA;        return right&#xA;    elif right is None:&#xA;        return left&#xA;    else:&#xA;        return left if (left[2] &amp;lt;= right[2]) else right&#xA;&#xA;&#xA;def make_diameter(a, b):&#xA;    # Original code:&#xA;#     cx = (a[0] + b[0]) / 2.0&#xA;#     cy = (a[1] + b[1]) / 2.0&#xA;#     r0 = math.hypot(cx - a[0], cy - a[1])&#xA;#     r1 = math.hypot(cx - b[0], cy - b[1])&#xA;#     return (cx, cy, max(r0, r1))&#xA;&#xA;    # Sperical code:&#xA;    (phi_m,theta_m) = midpoint(a,b)&#xA;    return (phi_m,theta_m,calc_sep1(a,b)*.5)&#xA;&#xA;&#xA;def make_circumcircle(a, b, c):&#xA;    # Mathematical algorithm from Wikipedia: Circumscribed circle&#xA;&#xA;#---------------------------------------------------------------------------    &#xA;    # I don't know how to sphericalize this part&#xA;    ox = (min(a[0], b[0], c[0]) + max(a[0], b[0], c[0])) / 2.0&#xA;    oy = (min(a[1], b[1], c[1]) + max(a[1], b[1], c[1])) / 2.0&#xA;&#xA;    ax = a[0] - ox;  ay = a[1] - oy&#xA;    bx = b[0] - ox;  by = b[1] - oy&#xA;    cx = c[0] - ox;  cy = c[1] - oy&#xA;&#xA;    d = (ax * (by - cy) + bx * (cy - ay) + cx * (ay - by)) * 2.0&#xA;    if d == 0.0:&#xA;        return None&#xA;    x = ox + ((ax * ax + ay * ay) * (by - cy) + &#xA;              (bx * bx + by * by) * (cy - ay) + &#xA;              (cx * cx + cy * cy) * (ay - by)) / d&#xA;    y = oy + ((ax * ax + ay * ay) * (cx - bx) + &#xA;              (bx * bx + by * by) * (ax - cx) + &#xA;              (cx * cx + cy * cy) * (bx - ax)) / d&#xA;#---------------------------------------------------------------------------&#xA;&#xA;&#xA;&#xA;    #---------------    &#xA;    # Original code:&#xA;#     ra = math.hypot(x - a[0], y - a[1])&#xA;#     rb = math.hypot(x - b[0], y - b[1])&#xA;#     rc = math.hypot(x - c[0], y - c[1])&#xA;&#xA;    # Spherical code:&#xA;    ra = calc_sep1((x,y),a)&#xA;    rb = calc_sep1((x,y),b)&#xA;    rc = calc_sep1((x,y),c)&#xA;    #---------------&#xA;&#xA;    return (x, y, max(ra, rb, rc))&#xA;&#xA;&#xA;_MULTIPLICATIVE_EPSILON = 1 + 1e-14&#xA;&#xA;def is_in_circle(c, p):&#xA;#     return c is not None and math.hypot(p[0] - c[0], p[1] - c[1]) &amp;lt;= c[2] * _MULTIPLICATIVE_EPSILON&#xA;    return c is not None and calc_sep1(p,(c[0],c[1])) &amp;lt;= c[2] * _MULTIPLICATIVE_EPSILON&#xA;&#xA;&#xA;# Returns twice the signed area of the triangle defined by (x0, y0), (x1, y1), (x2, y2).&#xA;def _cross_product(x0, y0, x1, y1, x2, y2):&#xA;    # Original code:&#xA;    #return (x1 - x0) * (y2 - y0) - (y1 - y0) * (x2 - x0)&#xA;&#xA;    # Spherical code:&#xA;    # (Uses http://mathforum.org/library/drmath/view/65316.html &#xA;    # then adds sign accordingly to ordinary cross product)&#xA;&#xA;    side1 = calc_sep1((x0, y0), (x1, y1))&#xA;    side2 = calc_sep1((x1, y1), (x2, y2))&#xA;    side3 = calc_sep1((x2, y2), (x0, y0))&#xA;    s = (side1+side2+side3)/2&#xA;    Area = 4* math.atan(math.sqrt(math.tan(s/2)*&#xA;                                  math.tan((s-side1)/2)*&#xA;                                  math.tan((s-side2)/2)*&#xA;                                  math.tan((s-side3)/2)))&#xA;&#xA;    Eucl_cross_prod = (x1 - x0) * (y2 - y0) - (y1 - y0) * (x2 - x0)&#xA;    cross_prod_sign = 1 if (Eucl_cross_prod &amp;gt;=0) else -1&#xA;&#xA;    return 2*cross_prod_sign*Area&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26471" LastEditorUserId="26471" LastEditDate="2019-06-06T11:05:45.987" LastActivityDate="2022-09-15T17:55:50.023" Title="Smallest circumscribed circle in spherical geometry" Tags="|python|computational-geometry|" AnswerCount="1" CommentCount="23" ContentLicense="CC BY-SA 4.0" />
  <row Id="30560" PostTypeId="1" CreationDate="2018-11-19T23:14:53.390" Score="3" ViewCount="64" Body="&lt;p&gt;I need to compute a &lt;span class=&quot;math-container&quot;&gt;$n-$&lt;/span&gt;dimensional integral with &lt;span class=&quot;math-container&quot;&gt;$n&amp;lt;10$&lt;/span&gt; on a convex polytope. Since most numerical integration libraries (e.g. Cuba) expect the function to be integrated defined inside an axis aligned bounding box (AABB), I need to compute a (minimum volume) AABB enclosing the polytope. Since the polytope is defined by a set of linear inequalities (H-polytope), the easiest way to compute the AABB would be converting the polytope to the V-form using the Fourier–Motzkin elimination (FME) algorithm. The problem is that the number of vertices generated by FME grows so quickly to make the method not practical. So my question is: is there a way to compute the AABB directly from the H-polytope while avoiding the conversion to V-polytope?&lt;/p&gt;&#xA;" OwnerUserId="7084" LastActivityDate="2018-11-19T23:14:53.390" Title="Minimum axis aliged bounding box of convex polytope" Tags="|computational-geometry|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30562" PostTypeId="1" AcceptedAnswerId="30565" CreationDate="2018-11-20T09:13:33.877" Score="3" ViewCount="386" Body="&lt;p&gt;I'm trying to understand the asymptotic error behaviour of &lt;em&gt;forward Euler&lt;/em&gt; (finite difference method), as timesteps are decreased (refined), so I feel trust in the &lt;em&gt;method of manufactured solutions&lt;/em&gt; (MMS).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But looking at a simple model of a ball accelerating, I noticed you can make the error asymptote to whatever you want...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the ball begins with zero velocity, then forward euler estimates that it does not move in the first timestep. So the error is the same as the analytically determined position:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Constant acceleration gives velocity &lt;span class=&quot;math-container&quot;&gt;$v(t) = t$&lt;/span&gt; and position &lt;span class=&quot;math-container&quot;&gt;$p(t)=\frac{1}{2}t^2$&lt;/span&gt;. So that is the error, which is order &lt;span class=&quot;math-container&quot;&gt;$O( (\Delta t)^2 )$&lt;/span&gt;, for the first timestep to &lt;span class=&quot;math-container&quot;&gt;$t=\Delta t$&lt;/span&gt;. So a plot of error vs timestep, with timestep refinement, will reveal an asymptote of that order.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;An increasing acceleration, say &lt;span class=&quot;math-container&quot;&gt;$v(t) = t^2$&lt;/span&gt;, gives &lt;span class=&quot;math-container&quot;&gt;$p(t)=\frac{1}{3}t^3$&lt;/span&gt;. The error is now order &lt;span class=&quot;math-container&quot;&gt;$O( (\Delta t)^3 )$&lt;/span&gt;, which it again asymptotes to with timestep refinement.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is troubling, because MMS relies on error vs timestep having the same asymptotic behaviour of the estimation method (in this case, forward Euler, which is a first order method). In this simple example, it doesn't seem to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MMS is a well-established technique, so I must be doing something wrong or misunderstanding something...&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it that you should take many timesteps (accumulating error), not just the first one?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it that this example has only one variable &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, but fluid simulations have many (a grid of many cells, with many variables in each), and in a complicated scenario, with typical fluid equations, it kind of all evens out?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;maybe relevant: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/10106/numerically-determining-convergence-order-of-eulers-method&quot;&gt;Numerically determining convergence order of Euler's method&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To summarize my understanding (seeking feedbac!):&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;The error term expected to dominate evaluates to zero&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Forward Euler being &lt;em&gt;first order&lt;/em&gt; is confusing to me, because for estimating the function, there's &lt;em&gt;local truncation error&lt;/em&gt; (LTE) &lt;span class=&quot;math-container&quot;&gt;$O(h^2)$&lt;/span&gt; and &lt;em&gt;global trunction error&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$O(h)$&lt;/span&gt;, and for estimating the first derivative (used in forward Euler/forward differences) the order of accuracy is &lt;span class=&quot;math-container&quot;&gt;$O(h)$&lt;/span&gt;.  Here, I'll stick with LTE because it's applicable to the single timestep used here and (bonus!) I kinda understands it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Taylors series expansion of a function, the first two terms can be seen as forward Euler, &lt;span class=&quot;math-container&quot;&gt;$p(t+h) \approx p(t)+hp'(t)$&lt;/span&gt;, with the remaining terms being trunction error (just two error terms shown here):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;p(t+h) = p(t)+hp\prime(t) + \frac{h^2}{2} p\prime\prime(t) + \frac{h^3}{3} p\prime\prime\prime(t) + \ldots&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LTE order of the error is &lt;span class=&quot;math-container&quot;&gt;$O(h^2)$&lt;/span&gt;, which will be greater than the higher order terms, because &lt;span class=&quot;math-container&quot;&gt;$O(h^2)&amp;lt;O(h)$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$h \rightarrow 0$&lt;/span&gt; (similar to why &lt;span class=&quot;math-container&quot;&gt;$x^2&amp;lt;x$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$x&amp;lt;1$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying to the two examples above:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(t)=\frac{1}{2}t^2$&lt;/span&gt;, with derivatives &lt;span class=&quot;math-container&quot;&gt;$t, 1, 0$&lt;/span&gt;, at &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$0,1,0$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$p(t+h) = 0 + h.(0) + \frac{h^2}{2}.(1) + \frac{h^3}{6}.(0)$&lt;/span&gt;  Here, the truncation error is &lt;span class=&quot;math-container&quot;&gt;$\frac{h^2}{2}$&lt;/span&gt;, which is order &lt;span class=&quot;math-container&quot;&gt;$O(h^2)$&lt;/span&gt; - exactly as expected.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(t)=\frac{1}{3}t^3$&lt;/span&gt;, with derivatives &lt;span class=&quot;math-container&quot;&gt;$t^2, 2t, 2$&lt;/span&gt;, at &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$0,0,2$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$p(t+h) = 0 + h.(0) + \frac{h^2}{2}.(0) + \frac{h^3}{6}.(2)$&lt;/span&gt;  Unfortunately, the truncation error is &lt;span class=&quot;math-container&quot;&gt;$\frac{h^3}{3}$&lt;/span&gt;, which is order &lt;span class=&quot;math-container&quot;&gt;$O(h^3)$&lt;/span&gt; - NOT as expected.  The problem is the error term expected to dominate (&lt;span class=&quot;math-container&quot;&gt;$\frac{h^2}{2}.(2t)$&lt;/span&gt;) evaluates to zero (at &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is one danger warned of in the answer by @cpraveen: &lt;strong&gt;zero derivatives in the truncation error&lt;/strong&gt;. I read this at first as meaning the zero function (like the comment by @wolfgangbangerth, &lt;span class=&quot;math-container&quot;&gt;$p\prime(t)=0$&lt;/span&gt;), but it happens here because the derivative expected to dominate evaluates to zero. A derivative evaluating to zero is also a zero!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also (if I understand correctly) &lt;strong&gt;non-linearity&lt;/strong&gt; in the answer would be a problem because its higher derivatives are zero (similar again for &lt;strong&gt;non-trivial&lt;/strong&gt; in the answer). An advantage of &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sin$&lt;/span&gt; in manufactured solutions is they never differentiate to zero... though all &lt;span class=&quot;math-container&quot;&gt;$\sin$&lt;/span&gt; derivatives evaluate to zero twice per cycle, so that might cause the same problem, if that special input values come up too often (e.g. wavelength aligned with grid). This fits with &lt;strong&gt;variation&lt;/strong&gt; in the solution.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Avoiding it in the examples&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Simply evaluating at &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt; (instead of &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;) fixes it, because the error term expected to dominate no longer evaluates to zero.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(t)=\frac{1}{2}t^2$&lt;/span&gt;, with derivatives &lt;span class=&quot;math-container&quot;&gt;$t, 1, 0$&lt;/span&gt;, at &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$1,1,0$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$p(t+h) = 1/2 + h.1 + \frac{h^2}{2}.1 + \frac{h^3}{6}.0$&lt;/span&gt;  The truncation error is still order &lt;span class=&quot;math-container&quot;&gt;$O(h^2)$&lt;/span&gt; as before (good!)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(t)=\frac{1}{3}t^3$&lt;/span&gt;, with derivatives &lt;span class=&quot;math-container&quot;&gt;$t^2, 2t, 2$&lt;/span&gt;, at &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt;, are &lt;span class=&quot;math-container&quot;&gt;$1, 2, 2$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$p(t+h) = 0 + h.1 + \frac{h^2}{2}.2 + \frac{h^3}{6}.(2)$&lt;/span&gt;. The truncation error now has both terms non-zero, so the order is &lt;span class=&quot;math-container&quot;&gt;$O(h^2)$&lt;/span&gt;, which is what it should be. (fixed!)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;tl;dr Conclusion&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;It seems the cause is just these special cases, of zero error terms. One way to avoid this is to use &lt;span class=&quot;math-container&quot;&gt;$e^t$&lt;/span&gt; in the manufactured solution, because its derivatives never evaluate to zero. Another is to use &lt;span class=&quot;math-container&quot;&gt;$\sin(t)$&lt;/span&gt;, though its derivatives do evaluate to zero sometimes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a proper fluid simulation, the interactions between equations could well produce zero error terms from time to time, no matter what you do. It seems the only way to guard against this is to have varied manufactured solutions, so that zero error terms only occur rarely in the grid, and are overwhelmed by the error terms of all the other grid cells. (But careful of systemic zeros, like from cell-aligned &lt;span class=&quot;math-container&quot;&gt;$\sin$&lt;/span&gt; periods). (Using many steps instead of &lt;strong&gt;one timestep&lt;/strong&gt; would  also help, for the same reason.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We must remember that error is supposed to be random, so that this probabilistic averaging out is perfectly OK!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW It reminds me of the probablistic &lt;em&gt;Schwartz-Zippel lemma&lt;/em&gt;: (oversummarizing) you can test for the zero polynomial by evaluating with random arguments. But it could be zero by chance; that is, you hit a root - a special case. The lemma shows the probability of this happening, so if you try several (unrelated) random arguments, you can make the probability as small as you like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess a similar analysis might have been done for MMS - but it seems so much more difficult because of the complexity of the governing equations, over a grid, over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have I got all that about right?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I did some numerical tests,  which comfirmer the above. Surprisingly (to me), for &lt;span class=&quot;math-container&quot;&gt;$p(t)=\frac{1}{3}t^3$&lt;/span&gt;, at &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt;, I didn't get exactly &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; for the power law, but around &lt;span class=&quot;math-container&quot;&gt;$2\pm .02$&lt;/span&gt;. I only played around a little, but seemed to be that, even with much smaller refinements in &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$0.001$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have heard that that level of accuracy is pretty good for MMS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried &lt;span class=&quot;math-container&quot;&gt;$p(t)=\sin t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p(t) = e^t$&lt;/span&gt;, and got that &lt;span class=&quot;math-container&quot;&gt;$2\pm.02$&lt;/span&gt; power law again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I confirmed that &lt;span class=&quot;math-container&quot;&gt;$p(t)=\sin t$&lt;/span&gt; doesn't give the expected err vs timestep asymptote of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;, because the second derivative is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; there. But it works for &lt;span class=&quot;math-container&quot;&gt;$t=1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further note: the error term expected to domimate being zero is a particular case, of that term being smaller than following terms. So, for &lt;span class=&quot;math-container&quot;&gt;$\sin t$&lt;/span&gt; this would also happen in s small region &lt;em&gt;around&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; not just exactly at zero. How small a region depends on how small &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is, because the higher powers of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; in the following terms makes them smaller.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think we can see all this as a natural probabilistic distribution of random errors. &lt;span class=&quot;math-container&quot;&gt;$O(h^2)$&lt;/span&gt; is approximate, because there are still other error terms. Because special cases are rare, if they sometimes do occur, it doesn't really matter - provided there is adequate sampling of the overall distribution, to even them out. I guess that explains the &lt;span class=&quot;math-container&quot;&gt;$2\pm0.02$&lt;/span&gt;, and it's not just this special case, but general random variation in the errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems my strategy, of starting with the simplest possible case of a sample of one from the distribution, was bound to have problems! &lt;strong&gt;EDIT&lt;/strong&gt; but much easier to analyse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But we could attempt to minimize these special cases by choosing non-zero regions of &lt;span class=&quot;math-container&quot;&gt;$\sin$&lt;/span&gt; - especially for time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, which would be used throughout a simulation grid. It might also be helpful to similarly choose a non-zero region of &lt;span class=&quot;math-container&quot;&gt;$\sin$&lt;/span&gt; over the entire grid (i.e  so that &lt;span class=&quot;math-container&quot;&gt;$\sin x$&lt;/span&gt; is zero outside the grid, e.g. &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt; before the grid starts, and &lt;span class=&quot;math-container&quot;&gt;$x=\pi$&lt;/span&gt; after the grid ends).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I tried detecting mistakes with this, since that's  the purpose of MMS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For forward Euler, the derivative needs to be non-zero, or that term won't be exercised, and any mistakes in it won't be detected. For  &lt;span class=&quot;math-container&quot;&gt;$p(t+h) \approx p(t)+hp'(t)$&lt;/span&gt;,  we don't want &lt;span class=&quot;math-container&quot;&gt;$p\prime(t)=0$&lt;/span&gt;. This happens e.g. for &lt;span class=&quot;math-container&quot;&gt;$p(t) = \sin t, t=\pi/2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;span class=&quot;math-container&quot;&gt;$p(t) = e^t$&lt;/span&gt; avoids this problem, but won't detect using the wrong order  of derivative (because they're all the same). Can fix this with e.g. &lt;span class=&quot;math-container&quot;&gt;$p(t)=e^{2t}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found gnuplot fit had problems with &lt;span class=&quot;math-container&quot;&gt;$p(t) = e^{t^2}$&lt;/span&gt;, when &lt;span class=&quot;math-container&quot;&gt;$t&amp;gt;4$&lt;/span&gt;, with a power law &lt;span class=&quot;math-container&quot;&gt;$ax^b$&lt;/span&gt;, but was better if converting to logs before fitting (and using a linear law - how you'd do it by hand).&lt;/p&gt;&#xA;" OwnerUserId="24091" LastEditorUserId="24091" LastEditDate="2018-11-26T08:53:19.913" LastActivityDate="2018-11-26T10:41:21.510" Title="Asymptotic error of forward Euler" Tags="|finite-difference|fluid-dynamics|convergence|error-estimation|numerics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30563" PostTypeId="2" ParentId="30514" CreationDate="2018-11-20T09:40:09.143" Score="0" Body="&lt;p&gt;With &lt;span class=&quot;math-container&quot;&gt;$$k^3\Delta u-k(1\cdot\nabla u)=0,$$&lt;/span&gt; you can already remove 1 solution for &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt; which is not of interest for you. You are really looking at&#xA;&lt;span class=&quot;math-container&quot;&gt;$$k^2\Delta u-(1\cdot\nabla u)=0.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;:&#xA;The previous equations enforced for a given &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u_\text{mean}$&lt;/span&gt; a solution on the field &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and its constraint force &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to find &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; for a given &lt;span class=&quot;math-container&quot;&gt;$u_\text{mean}$&lt;/span&gt;, it should be rephrased to an optimization under constraints:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\min_{k&amp;gt;0} \frac{1}{2}||PDE(k,u)||^2+\frac{\lambda}{2}^2||u_\text{mean}-\text{mean}(u)||^2$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is a regularization parameter, basically how much weight you put on the constraint. In this case you end up with a similar equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$k^2\Delta u-(1\cdot\nabla u)+\lambda(u_\text{mean}-\text{mean}(u))=0.$$&lt;/span&gt; In this optimization problem you are no longer solving for &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; (this is input) but for &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In  case of optimization  I suggest to look for the adjoint formulation of PDEs with constraints, e.g. the mathematical description on the &lt;a href=&quot;http://www.dolfin-adjoint.org/en/latest/documentation/maths/1-foreword.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dolfin adjoint page&lt;/a&gt; and its references.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2018-11-20T13:33:00.247" LastActivityDate="2018-11-20T13:33:00.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30564" PostTypeId="1" AcceptedAnswerId="30568" CreationDate="2018-11-20T13:11:29.590" Score="1" ViewCount="315" Body="&lt;p&gt;I have a trouble of defining a Jacobian matrix for my problem. Basically, I have 4 differential equations to be solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{aligned}&#xA;\dot x_1(t)&amp;amp;=x_2(t)\\&#xA;\dot x_2(t)&amp;amp;=p_2(t)−\sqrt 2 x_1(t)e^{-αt}\\&#xA;\dot p_1(t)&amp;amp;=\sqrt 2p_2(t)e^{-αt}+x_1(t)\\&#xA;\dot p_2(t)&amp;amp;=−p_1(t)&#xA;\end{aligned}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial and boundary values of:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x_1(0)=1\\ p_2(0)=0\\p_1(1)=0\\p_2(1)=0$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following classic shooting method strategy, I suggest some values for &lt;span class=&quot;math-container&quot;&gt;$x_2(0), p_1(0)$&lt;/span&gt;, solve the system of diff equations using &lt;a href=&quot;https://en.wikipedia.org/wiki/Dormand%E2%80%93Prince_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dormand Prince&lt;/a&gt; method and obtain some values at the boundary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I check, if the solution on my boundaries meets the criteria of &lt;span class=&quot;math-container&quot;&gt;$p_1(1)=0,p_2(1)=0$&lt;/span&gt;. If not, I have to suggest new values for the initial values of &lt;span class=&quot;math-container&quot;&gt;$x_2(0), p_1(0)$&lt;/span&gt; to be iterated again. Since this is a two-parametric shooting, I can't use simple linear interpolation as in one-parametric shooting, so I have to use Newton's method for that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my question: How to implement Newton's method to iterate towards the right solution? I tried myself and it doesn't work. Can you at least help me with defining a Jacobian, for the given problem of iterative solution to &lt;span class=&quot;math-container&quot;&gt;$x_2(0), p_1(0)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="29281" LastEditorUserId="20688" LastEditDate="2018-11-20T17:37:11.470" LastActivityDate="2018-11-21T07:39:02.483" Title="Shooting method implementation" Tags="|ode|boundary-conditions|newton-method|jacobian|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30565" PostTypeId="2" ParentId="30562" CreationDate="2018-11-20T13:17:55.930" Score="4" Body="&lt;p&gt;When we say that Euler method is first order accurate, it means that for a class of ode with sufficiently smooth solutions, the error will be at most &lt;span class=&quot;math-container&quot;&gt;$O(h)$&lt;/span&gt; and there is at least one ode for which it cannot be smaller than &lt;span class=&quot;math-container&quot;&gt;$O(h)$&lt;/span&gt;. So it can happen that for some ode, the error will be smaller than &lt;span class=&quot;math-container&quot;&gt;$O(h)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example you give is somewhat special. But that also happens at the first time step only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence when using MMS, it is important to take non-trivial solutions which have a lot of variation and non-linearity and have all derivatives non zero that may appear in the truncation error. MMS is a way to verify theoretical convergence rates and code correctness, and it is not a proof of the convergence rate.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="20688" LastEditDate="2018-11-20T17:29:41.567" LastActivityDate="2018-11-20T17:29:41.567" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30566" PostTypeId="1" CreationDate="2018-11-20T14:59:45.027" Score="4" ViewCount="324" Body="&lt;p&gt;Is there any library or routine for high-performance matrix-matrix product, where the matrix elements are computed on-the-fly using a given function of &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, in the problem I am currently facing, I have to compute a matrix-matrix product&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{y} = \mathbf{A} \mathbf{b}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt; is a symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$N \times N$&lt;/span&gt;,  &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$N\times 3$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using LAPACK or BLAS routines requires the formation of the matrix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt;. The matrix elements, however, are a function that depends only on the matrix indices &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29282" LastEditorUserId="9667" LastEditDate="2018-11-20T15:05:01.130" LastActivityDate="2018-11-21T15:48:06.403" Title="Fast matrix multiplication with matrix elements computed on-the-fly (without forming the matrix)" Tags="|linear-algebra|lapack|blas|dense-matrix|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30568" PostTypeId="2" ParentId="30564" CreationDate="2018-11-21T07:39:02.483" Score="0" Body="&lt;p&gt;Imagine you have the following system:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\left\{\begin{array}{c}&#xA;\dot{y} = f(y,z,t)\\&#xA;\dot{z} = g(y,z,t)&#xA;\end{array}\right.&#xA;\end{equation}&lt;/span&gt;&#xA;being &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; vectors with appropiate BC &lt;span class=&quot;math-container&quot;&gt;$y(0) = y_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z(T) = z_T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; vector valued functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you don't have any initial condition for &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;, you have to solve the nonlinear equation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$F_1(z_0) = \dot{z}(z_0,t)-g(y,z(z_0,t),t)=0$$&lt;/span&gt;&#xA;subjected to &#xA;&lt;span class=&quot;math-container&quot;&gt;$$F_2(z_0) = z(z_0,T) - z_T$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you linearise last equations w.r.t. &lt;span class=&quot;math-container&quot;&gt;$z_0$&lt;/span&gt; you obtain a system for &lt;span class=&quot;math-container&quot;&gt;$\delta z_0$&lt;/span&gt; introducing a new variable:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\zeta(t) =\frac{\partial z(z_0,t)}{\partial z_0}$$&lt;/span&gt;&#xA;Which is a matrix if &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; has more than 1 component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equations are the following:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$F_1(z_0+\delta z_0)=F_1(z_0)+\frac{dF_1(z_0)}{dz_0}\delta z_0=\frac{dF_1(z_0)}{dz_0}\delta z_0=\left(\dot{\zeta}-\frac{\partial g}{\partial z}\zeta\right)\delta z_0=0$$&lt;/span&gt;&#xA;Since we solve &lt;span class=&quot;math-container&quot;&gt;$F_1(z_0) = 0$&lt;/span&gt; exactly with a guessed value &lt;span class=&quot;math-container&quot;&gt;$z_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We note that to solve the first linearised equation we need an initial value for &lt;span class=&quot;math-container&quot;&gt;$\zeta$&lt;/span&gt;, and according to the definition, in is exactly the identity matrix:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\zeta(0) = \frac{\partial z(z_0,0)}{\partial z_0}=\frac{\partial z_0}{\partial z_0}=I$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$F_2(z_0+\delta z_0)=F_2(z_0)+\frac{dF_2(z_0)}{dz_0}\delta z_0=z(T,z_0)-z_T+\zeta(T)\delta z_0=0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final equations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, you have to solve iteratively for &lt;span class=&quot;math-container&quot;&gt;$z_0$&lt;/span&gt; the following equations:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\left\{\begin{aligned}&#xA;\dot{y} &amp;amp;= f(y,z,t)\\&#xA;\dot{z} &amp;amp;= g(y,z,t)\\&#xA;\dot{\zeta} &amp;amp;= \frac{\partial g}{\partial z}\zeta&#xA;\end{aligned}\right.&#xA;\end{equation}&lt;/span&gt;&#xA;With the initial conditions &lt;span class=&quot;math-container&quot;&gt;$y(0) = y_0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$z(0) = z_0^{n}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\zeta(0) = I$&lt;/span&gt;, and the updating equations for &lt;span class=&quot;math-container&quot;&gt;$z_0$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$z_0^{n+1} = z_0^n + \delta z_0\qquad \zeta(T)\delta z_0=z_T-z(T,z_0)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$z_0^0$&lt;/span&gt; a value close enough to the actual one.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-11-21T07:39:02.483" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30569" PostTypeId="2" ParentId="30514" CreationDate="2018-11-21T08:58:58.913" Score="1" Body="&lt;p&gt;I would recommend an adjoint method also but slightly different. Following is the general idea. Whether this approach is well posed for this problem is left as an excercise :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solve this&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_k J(k) = (1/2)[u_{mean} - mean(u)]^2&#xA;$$&lt;/span&gt;&#xA;subject to the pde as constraint. If you change &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$k+k'$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; changes to &lt;span class=&quot;math-container&quot;&gt;$u+u'$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$J+J'$&lt;/span&gt;. Find a linear PDE for &lt;span class=&quot;math-container&quot;&gt;$u'$&lt;/span&gt; assuming changes are small, which should have following structure&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L(u',k)=F(u,k) k'&#xA;$$&lt;/span&gt;&#xA;and it will have &lt;span class=&quot;math-container&quot;&gt;$u'=0$&lt;/span&gt; on boundary since you have a Dirichlet bc on &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;. Then you get&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J' = -[u_{mean} - mean(u)] mean(u') + \int_\Omega v \cdot L(u',k) dx - \int_\Omega F(u,k) k' v dx&#xA;$$&lt;/span&gt;&#xA;and we just added a zero term. Integrate by parts so all derivatives are on the adjoint variable &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;. Then you get an equation of the form&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J' =  - k' \int_\Omega F(u,k) v dx + \int_\Omega L^*(v,k)\cdot u' dx + \textrm{boundary terms}&#xA;$$&lt;/span&gt;&#xA;Form  adjoint pde&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L^*(v,k)=0&#xA;$$&lt;/span&gt;&#xA;and choose bc for &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; so that boundary terms in &lt;span class=&quot;math-container&quot;&gt;$J'$&lt;/span&gt; vanish. Once you solve adjoint, you can update &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;k' = \int_\Omega F(u,k) v dx&#xA;$$&lt;/span&gt;&#xA;which ensures that &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; will decrease.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2018-11-24T02:55:48.660" LastActivityDate="2018-11-24T02:55:48.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30571" PostTypeId="2" ParentId="30566" CreationDate="2018-11-21T15:48:06.403" Score="3" Body="&lt;p&gt;If the matrix is small enough to fit into memory, then there is of course no cost associated with actually forming the elements: You will have to compute the elements at least once anyway to perform the matrix-vector product, so you might as well store the values you computed in memory and then run an optimized matrix-vector product function over it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are of course cases where you genuinely can't or don't want to compute and store the matrix entries. An example is if &lt;span class=&quot;math-container&quot;&gt;$A=BC$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; are other matrices and where it is cheaper to compute &lt;span class=&quot;math-container&quot;&gt;$y=Ab$&lt;/span&gt; via &lt;span class=&quot;math-container&quot;&gt;$B(Cb)$&lt;/span&gt; than &lt;span class=&quot;math-container&quot;&gt;$(BC)b$&lt;/span&gt;. Other cases are where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; does not fit into memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In these cases, you need a library that allows you to implement a matrix via its &lt;em&gt;action&lt;/em&gt;, i.e., its matrix-vector product. One example is PETSc where you can create a &lt;code&gt;MatShell&lt;/code&gt; object -- a matrix for which you need to provide a pointer to a function that can perform matrix-vector products. Most other linear algebra libraries have similar functionality -- I'm pretty sure Trilinos can do this, and deal.II (disclaimer: that's a project I'm associated with) does this all over the place by creating classes that provide a matrix-vector multiplication member function. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-11-21T15:48:06.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30573" PostTypeId="2" ParentId="10582" CreationDate="2018-11-22T06:09:17.243" Score="2" Body="&lt;p&gt;Unfortunately, the listed set of matrix properties does not give any hope on a closed-form of the matrix inverse or its &lt;strong&gt;reasonable&lt;/strong&gt; approximation (for this specific case). It is too general, and finding matrix inverse (or matrix factorization) is a non-trivial task. Otherwise, such fundamental problem as preconditioning would have been successfully solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have the following options, as far as I can see:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Perform exact factorization of the matrix and tolerate that it is memory consuming and slow.&lt;/li&gt;&#xA;&lt;li&gt;Find an accelerated method to speed up the factorization/inverse calculation. That would require domain (physics) knowledge from where this matrix is coming. Examples: &lt;a href=&quot;https://en.wikipedia.org/wiki/Hierarchical_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hierarchical matrices&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Fast_multipole_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;FMM-&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Fast_Fourier_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;FFT-&lt;/a&gt;based methods. Different application areas might have many more specialized methods that offer certain advantages. There is also an active field in randomized methods.&lt;/li&gt;&#xA;&lt;li&gt;Stick to very coarse approximations: diagonal(+possible near zone) matrix inverse. Here, you &lt;strong&gt;manually&lt;/strong&gt; select &quot;most significant&quot; parts of the matrix and choosing only them to form an approximation inverse. Commonly, it is the matrix main diagonal and possibly some near-zone around it (subject to row/col reordering).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;NB: I would omit discussion of the question of why explicit calculation of the matrix inverse should be avoided in the first place.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-22T06:09:17.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30574" PostTypeId="2" ParentId="13002" CreationDate="2018-11-22T06:19:24.463" Score="2" Body="&lt;p&gt;I would suggest &lt;a href=&quot;https://en.wikipedia.org/wiki/Barnes%E2%80%93Hut_simulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barnes-Hut type methods&lt;/a&gt; for this problem. It seems to fit the agenda perfectly and has a nice &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(N\log N)$&lt;/span&gt; complexity. You will have to augment it with your restrictions and conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Barnes-Hut is a member of FMM-family, but it is much simpler than others and still covers your needs as far as I can see.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2018-11-22T14:45:40.283" LastActivityDate="2018-11-22T14:45:40.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30575" PostTypeId="2" ParentId="28509" CreationDate="2018-11-22T06:31:44.790" Score="2" Body="&lt;p&gt;It seems like you are inventing &lt;a href=&quot;https://en.wikipedia.org/wiki/Barnes%E2%80%93Hut_simulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barnes-Hut&lt;/a&gt;-type algorithm, which is a fundamental accelerated algorithm for n-body simulations. It follows a similar logic: you combine masses on a grid. But, it is done in a more elaborate fashion:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;when particles are close, you still use direct calculations (no combination), otherwise, you would lose accuracy.&lt;/li&gt;&#xA;&lt;li&gt;multipole expansions (center of mass) are used on an octree (in 3D) to accelerate distant interactions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Since there are many more distant interactions when you handle your particles in a tree-fashion, this leads to the complexity reduction: &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(N\log N)$&lt;/span&gt; vs &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(N^2)$&lt;/span&gt; for direct computation.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-22T06:31:44.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30576" PostTypeId="1" AcceptedAnswerId="30584" CreationDate="2018-11-22T12:30:12.300" Score="1" ViewCount="1063" Body="&lt;p&gt;I am writing a model for an optimization problem. I need to write the following constraint: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum^{N - 1}_i \lvert (a_i - a_{i+1}) \rvert \leq 2\, .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How to write this constraint (or linearize)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it permissible to use absolute value constraints in an optimization problem (even for nonlinear programming)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After I searched, I found that absolute values can be modeled as the following (without the sum or difference):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\lvert a_i \rvert \leq 2 \: \text{ to }\:  a_i \leq 2 \: \text{ and }\:  - a_i \leq 2 $$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this type of transformation is applicable here?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29292" LastEditorUserId="9667" LastEditDate="2018-11-22T16:29:10.723" LastActivityDate="2018-11-25T23:09:30.857" Title="sum of absolute difference constraint in optimization problem" Tags="|optimization|constrained-optimization|constraints|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30580" PostTypeId="1" CreationDate="2018-11-22T21:14:48.640" Score="2" ViewCount="111" Body="&lt;p&gt;I have a quick questions which has been troubling me lately.&#xA;When reading the &lt;a href=&quot;http://launchpad.net/fenics-book/trunk/final/+download/fenics-book-2011-10-27-final.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;FENICS Finite Element Book&lt;/a&gt; they assess various approaches to solver the Stokes equation. Obviously, they discuss the notion of elements that satisfy the LBB conditions, but they also discuss stabilized method. They briefly explain that the core of stabilization method is to add a term to the pressure equation which alters the saddle point problem at the consequence of making the velocity &quot;less divergence free&quot; in layman's term. This is discussed at page 397 of the pdf.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting weak form reads :&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/68O1i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/68O1i.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;They noticeably insist on the fact that consistency is lost for orders q&gt;1 which makes sense because &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; is of order &lt;span class=&quot;math-container&quot;&gt;$h^2$&lt;/span&gt;, consequently it would appear to me that the convergence rate for pressure becomes bounded by this term if the order of the pressure interpolation is higher than 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Later on in the book they look at the order of convergence of the various scheme as well as the mass loss on a manufactured solution. They look at various polynomial order for the various elements and show the convergence order. For the stablized method, they observed that consistency is maintained for the &lt;strong&gt;L2 error on the velocity&lt;/strong&gt;, for instance using 4th order Lagrange polynomial interpolation, they obtain a order of convergence of 4+1.&#xA;This is shown in the following table (see the STAB column):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gbNk0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gbNk0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not understand how this occurs, considering that the term that is added to the continuity equation is by itself of order 2 in L2 space (due to it being &lt;span class=&quot;math-container&quot;&gt;$h^2$&lt;/span&gt;). What is wrong in my reasoning? I am obviously missing some details and something is wrong in my understanding of stabilization + high order interpolation. Is consistency maintained for velocity but lost for pressure (that is the order of convergence for pressure is reduce to second order, but not for velocity)?&#xA;Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="7452" LastEditorUserId="7452" LastEditDate="2018-11-23T13:48:09.713" LastActivityDate="2018-11-23T13:48:09.713" Title="Why do stabilized formulations for the Navier-Stokes equation maintain the convergence rate for high order polynomial interpolation?" Tags="|finite-element|fluid-dynamics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30581" PostTypeId="1" AcceptedAnswerId="30582" CreationDate="2018-11-23T04:28:50.880" Score="5" ViewCount="177" Body="&lt;p&gt;I have a system of nonlinear equations on the general form:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{M}(\bar{y})\dot{\bar{y}} =\bar{f}(\bar{y},t)&#xA;\end{align}&lt;/span&gt;&#xA;Where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{M}(\bar{y})$&lt;/span&gt; is a matrix and &lt;span class=&quot;math-container&quot;&gt;$\bar{f}$&lt;/span&gt; is a system of non-linear functions acting on &lt;span class=&quot;math-container&quot;&gt;$\bar{y}$&lt;/span&gt;. This system has many stiff elements and so i wish to solve it using the implicit BDF method with newton iterations for solving the non-linear system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is then related to how to treat the mass matrix in the BDF method when it depends on &lt;span class=&quot;math-container&quot;&gt;$\bar{y}$&lt;/span&gt;. When reading about the method in textbooks or e.g. wikipedia, the method is stated as:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\sum_{k=0}^{s}\alpha_k\bar{y}_{n-k} = h\bar{f}(\bar{y}_{n},t_{n})&#xA;\end{align}&lt;/span&gt;&#xA;But if a mass matrix is included, should it be treated as the lhs or rhs? I.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\sum_{k=0}^{s}\alpha_k\mathbf{M}(\bar{y}_{n-k})\bar{y}_{n-k} = h\bar{f}(\bar{y}_{n},t_{n})&#xA;\end{align}&lt;/span&gt;&#xA;Or&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{M}(\bar{y}_{n})\sum_{k=0}^{s}\alpha_k\bar{y}_{n-k} = h\bar{f}(\bar{y}_{n},t_{n})&#xA;\end{align}&lt;/span&gt;&#xA;I hope my question makes sense.&lt;/p&gt;&#xA;" OwnerUserId="21323" LastActivityDate="2018-11-23T05:23:40.133" Title="Mass matrix and BDF time integration" Tags="|computational-physics|numerics|implicit-methods|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30582" PostTypeId="2" ParentId="30581" CreationDate="2018-11-23T05:14:26.733" Score="5" Body="&lt;p&gt;Your second one is the correct form. BDF approximates derivative with backward difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Write&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;M(y) \dot{y} = f(y,t)&#xA;$$&lt;/span&gt;&#xA;as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\dot{y} = M^{-1}(y) f(y,t) =: F(y,t)&#xA;$$&lt;/span&gt;&#xA;write the BDF for this&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum_{k=0}^s \alpha_k y_{n-k} = h F(y_n,t) = h M^{-1}(y_n) f(y_n,t_n)&#xA;$$&lt;/span&gt;&#xA;Hence you get&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;M(y_n) \sum_{k=0}^s \alpha_k y_{n-k} = h f(y_n,t_n)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2018-11-23T05:23:40.133" LastActivityDate="2018-11-23T05:23:40.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30583" PostTypeId="1" CreationDate="2018-11-23T15:54:59.977" Score="3" ViewCount="111" Body="&lt;p&gt;I have ~30 non-negative variables and 24 equations and I want to find out the upper and lower bound for each variable. Feasible solutions are guaranteed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for each variable, I solve two LP problem, one for min and one for max&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{array}{rl}&#xA;\min &amp;amp; c^T x\\&#xA;\textrm{s.t.} &amp;amp; Ax = b\\&#xA;&amp;amp;x\geq0&#xA;\end{array}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$c=(0, 0, \dots, 0, 1, 0, \dots, 0)^T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$(0, 0, \dots, 0, -1, 0, \dots, 0)^T$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in total I have 60 LPs with exactly the same constraint, but slightly different objective. I have to do the same thing for each day's data for a few thousand days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to somehow reuse the intermediate results to speed up the calculation? Is it possible to implement using existing Python or R package?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Just to add that for each day, only the rhs &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is different, everything else stays the same.&lt;/p&gt;&#xA;" OwnerUserId="14305" LastEditorUserId="14305" LastEditDate="2018-11-24T22:42:51.963" LastActivityDate="2018-11-24T22:42:51.963" Title="Solving multiple linear programs with same constraints but different objective" Tags="|linear-programming|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30584" PostTypeId="2" ParentId="30576" CreationDate="2018-11-23T23:58:02.673" Score="5" Body="&lt;p&gt;The way to deal with this kind of constraint is to add &quot;slack variables&quot; to your system. In your case, let us say that you want to solve the problem&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \min_a c^T a \\&#xA;  \text{subject to}\; \sum_{i=1}^{N-1} |a_i - a_{i+1}| \le 2&#xA;$$&lt;/span&gt;&#xA;then you could introduce slack variables &lt;span class=&quot;math-container&quot;&gt;$y_1,\ldots y_{N-1}$&lt;/span&gt; so that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  y_i \ge |a_i-a_{i+1}|&#xA;$$&lt;/span&gt;&#xA;and then solve this problem instead:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \min_a c^T a \\&#xA;  \text{subject to}\; \sum_{i=1}^{N-1} y_i \le 2, \\&#xA;  \qquad \qquad \quad y_i \ge |a_i - a_{i+1}|&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, recognize that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  y_i \ge |a_i-a_{i+1}|&#xA;$$&lt;/span&gt;&#xA;is equivalent to the two conditions&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  y_i \ge +(a_i-a_{i+1}) \\&#xA;  y_i \ge -(a_i-a_{i+1}) \\&#xA;$$&lt;/span&gt;&#xA;and consequently your initial problem is equivalent to the following, which is now an entirely linearly constrained linear optimization problem:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \min_a c^T a \\&#xA;  \text{subject to}\; \sum_{i=1}^{N-1} y_i \le 2, \\&#xA;  \qquad \qquad \quad y_i \ge +(a_i - a_{i+1}), \\&#xA;  \qquad \qquad \quad y_i \ge -(a_i - a_{i+1}).&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2018-11-25T23:09:30.857" LastActivityDate="2018-11-25T23:09:30.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30586" PostTypeId="1" CreationDate="2018-11-24T11:18:34.843" Score="2" ViewCount="223" Body="&lt;p&gt;I wonder if there are any theorems which can help me to calculate an upper bound for the spectral norm of:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\left\| \left[ I + \sum_{i=1}^{\overline{n}\in\mathbb{N}}  \big( C_i - I\big)\right]^{-1}\right\|_2$$&lt;/span&gt; &#xA;with the identity matrix &lt;span class=&quot;math-container&quot;&gt;$I \in \mathbb{R}^{n\times n}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_i$&lt;/span&gt; are triangular matrices arising from a Cholesky decompostion of &lt;span class=&quot;math-container&quot;&gt;$A_i = C_i C_i^T $&lt;/span&gt;. The matrices &lt;span class=&quot;math-container&quot;&gt;$A_i \in \mathbb{R}^{n\times n}$&lt;/span&gt; are symmetric positive semidefinite. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first idea was to use that if &lt;span class=&quot;math-container&quot;&gt;$A\leq B$&lt;/span&gt;  the following relation holds for s.p.d. matrices &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\big\|(A + B)^{-1}\big\|_2 \leq \big\| A^{-1}\big\|_2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the problem is that the matrices &lt;span class=&quot;math-container&quot;&gt;$C_i - I$&lt;/span&gt; are negative definite if I'm right. &lt;/p&gt;&#xA;" OwnerDisplayName="user29088" LastEditorDisplayName="user29088" LastEditDate="2018-11-25T21:50:47.143" LastActivityDate="2019-11-06T20:00:40.530" Title="A Bound for the inverse of the sum of identity and triangular matrix" Tags="|linear-algebra|matrix|eigenvalues|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30587" PostTypeId="2" ParentId="30562" CreationDate="2018-11-24T12:14:17.843" Score="1" Body="&lt;h3&gt;MMS for first order ODE&lt;/h3&gt;&#xA;&lt;p&gt;The method of manufactured solutions takes some ODE &lt;span class=&quot;math-container&quot;&gt;$$\dot x(t)=f(t,x(t))+r(t)$$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\dot x=\frac{dx}{dt}$&lt;/span&gt;) and adapts the perturbation or control term &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; in such a way that some previously selected function &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a solution. This means setting &lt;span class=&quot;math-container&quot;&gt;$$r(t)=\dot p(t)-f(t,p(t))$$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x(t_0)=p(t_0)$&lt;/span&gt;. For instance, for &lt;span class=&quot;math-container&quot;&gt;$$\dot x+\sin(x)=r~~\text{ and }~~p(t)=\sin(2t)$$&lt;/span&gt; one would have to set &lt;span class=&quot;math-container&quot;&gt;$$x(0)=0~~\text{ and }~~r(t)=2\cos(2t)+\sin(\sin(2t)).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The error of the Euler method has the form &lt;span class=&quot;math-container&quot;&gt;$O(t\cdot Δt)$&lt;/span&gt; for small &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; or a little more globally &lt;span class=&quot;math-container&quot;&gt;$O((e^{Lt}-1)⋅Δt)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is a Lipschitz constant close to the initial point (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Gr%C3%B6nwall%27s_inequality&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grönwall lemma&lt;/a&gt; and that the numerical solution can be considered an &lt;span class=&quot;math-container&quot;&gt;$O(Δt)$&lt;/span&gt; perturbation of the original ODE).&lt;/p&gt;&#xA;&lt;p&gt;Using the above example over the interval &lt;span class=&quot;math-container&quot;&gt;$[0,3\pi]$&lt;/span&gt; and dividing the (absolute) error of the Euler method by &lt;span class=&quot;math-container&quot;&gt;$Δt$&lt;/span&gt; to get comparable results for the different step sizes result in the following plot&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YtizO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YtizO.png&quot; alt=&quot;error plot&quot; /&gt;&lt;/a&gt;&#xA;The zero crossings of the error function are caused by the oscillating nature of the (signed) error. However, apart from these points, one can see the convergence to a constant (relative to &lt;span class=&quot;math-container&quot;&gt;$Δt$&lt;/span&gt;) coefficient of the leading error term &lt;span class=&quot;math-container&quot;&gt;$C(t)Δt$&lt;/span&gt;. And the global error is related to the worst or maximimum local error, however that is measured.&lt;/p&gt;&#xA;&lt;h3&gt;What is wrong in the approach in the question&lt;/h3&gt;&#xA;&lt;p&gt;If you do an asymptotic error analysis, you do not observe the error in the first step of the varying &lt;span class=&quot;math-container&quot;&gt;$Δt$&lt;/span&gt;, but the error at some fixed time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; that requires &lt;span class=&quot;math-container&quot;&gt;$n=n(Δt)=(t-t_0)/Δt$&lt;/span&gt; integration steps to reach.&lt;/p&gt;&#xA;&lt;p&gt;The ODE you seem to solve as test case is simply a quadrature problem &lt;span class=&quot;math-container&quot;&gt;$\dot x(t)=r(t)$&lt;/span&gt;, where then consequently for MMS &lt;span class=&quot;math-container&quot;&gt;$r(t)=\dot p(t)$&lt;/span&gt; for some given test function &lt;span class=&quot;math-container&quot;&gt;$p(t)$&lt;/span&gt;. Then for all &lt;span class=&quot;math-container&quot;&gt;$Δt$&lt;/span&gt; chosen such that exactly &lt;span class=&quot;math-container&quot;&gt;$t=t_0+nΔt$&lt;/span&gt; for some integer &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; the Euler method is simply the Riemann sum with left sample points, where one can find a more concrete form of the leading error term (see also the &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler%E2%80%93Maclaurin_formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler-Maclaurin summation formula&lt;/a&gt;)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;x_n&amp;amp;=p(t_0)+\sum_{k=0}^{n-1}\dot p(t_0+kΔt)Δt&#xA;\\&#xA;&amp;amp;=\sum_{k=0}^{n-1}[p(t_0+(k+1)Δt)-p(t_0+kΔt)+\frac12\ddot p(t_0+(k+\theta_k)Δt)Δt^2]&#xA;\\&#xA;&amp;amp;=p(t_0+nΔt)+\frac12\sum_{k=0}^{n-1}\ddot p(t_0+(k+\theta_k)Δt)Δt^2&#xA;\\&#xA;&amp;amp;=p(t_0+nΔt)+\frac12[\dot p(t_0+nΔt)-\dot p(t_0)]Δt +O(Δt^2)&#xA;\end{align}&lt;/span&gt;&#xA;That last expression tells us that the error is always &lt;span class=&quot;math-container&quot;&gt;$O(Δt)$&lt;/span&gt;, with the leading error term vanishing only for &lt;span class=&quot;math-container&quot;&gt;$\dot p(t)=\dot p(t_0)$&lt;/span&gt;. This can be true for all &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; only if &lt;span class=&quot;math-container&quot;&gt;$p(t)$&lt;/span&gt; is a linear function where the Euler method integrates the constant derivative exactly. In testing the method, one should avoid the roots of &lt;span class=&quot;math-container&quot;&gt;$\dot p(t)-\dot p(t_0)=0$&lt;/span&gt; where the errors approximately cancel systematically, as that may be specific only for quadrature problems and not for more general, proper ODE.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-11-26T10:41:21.510" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="30592" PostTypeId="1" AcceptedAnswerId="30593" CreationDate="2018-11-25T16:29:41.337" Score="1" ViewCount="140" Body="&lt;h2&gt;Statement of the problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Suppose, we consider the following model&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{array}{l}&#xA;(1)~\mathbf{u}_t + \mathbf{F(u)}_x = \mathbf{S}(\mathbf{u},\mathbf{w}), \\&#xA;(2)~\mathbf{w}_x = \mathbf{P}(\mathbf{w},\mathbf{u}).&#xA;\end{array}&#xA;$$&lt;/span&gt;&#xA;Computational domain to be as follows &lt;span class=&quot;math-container&quot;&gt;$(x,t)=[0,L]\times[0,T]$&lt;/span&gt;. Let's consider Neumann boundary conditions (BC) for (1) and initial value problem (IVP) for (2). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Algorithm of simulation as follows: on each time step &lt;span class=&quot;math-container&quot;&gt;$t^n$&lt;/span&gt; solve (2) at first and then solve (1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could assume that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{u} = [n_d\,v_d]^T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{w} = [n_e\,E]^T$&lt;/span&gt; and obtaion the corresponding plot&lt;a href=&quot;https://i.stack.imgur.com/DSHAL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DSHAL.png&quot; alt=&quot;Output of simultation&quot;&gt;&lt;/a&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathrm{\color{blue}{blue}}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathrm{\color{red}{red}}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathrm{\color{green}{green}}$&lt;/span&gt; lines correspond grids with &#xA;&lt;span class=&quot;math-container&quot;&gt;$(1024,\,512,\,256)$&lt;/span&gt; poins respectively.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;One could realize that the shape of the solution depends on the number of grid points, i.e. &lt;em&gt;grid depence&lt;/em&gt; achieved. &#xA;How to avoid it?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;P.S.&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Eq. (1) was solved by smoothed Lax-Wendroff scheme (LxW) and eq. (2) was solved by Runge-Kutta scheme of 4th order.&lt;/p&gt;&#xA;" OwnerUserId="21795" LastActivityDate="2018-11-25T19:08:10.300" Title="Grid dependence of a numerical model" Tags="|finite-difference|nonlinear-equations|finite-volume|hyperbolic-pde|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30593" PostTypeId="2" ParentId="30592" CreationDate="2018-11-25T18:50:08.483" Score="4" Body="&lt;p&gt;Your numerical solution is probably just getting more accurate as you increase the number of grid points. Do you know or have you tried to derive the analytic (exact) solution for this problem? By looking at your plots, it seems like the exact solution has a shock (discontinuity) occurring at around x = 4.5, and the numerical method is resolving it with greater accuracy as you increase the number of grid points. The numerical solution will always depend on the grid you use.&lt;/p&gt;&#xA;" OwnerUserId="29276" LastEditorUserId="29276" LastEditDate="2018-11-25T19:08:10.300" LastActivityDate="2018-11-25T19:08:10.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30594" PostTypeId="1" CreationDate="2018-11-26T08:24:01.910" Score="3" ViewCount="48" Body="&lt;p&gt;I need to solve a simple, low-dimensional Hamilton-Jacobi-Bellman equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simple guide for doing this numerically using finite-difference schemes? I found a few research articles discussing higher order methods or discussing theoretical properties of numerical schemes but nothing that really helped me, as a newcomer without academic aspiration, to implement an actual algorithm. I really don't care about runtime and am perfectly fine with something that takes a whole day for 5% relative accuracy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I tried a simple forward Euler method, I got bizzare oscillations even though I tried to evaluate derivatives upstream. What upstream is depends on the solution of the Hamiltonian, of course, which itself requires evaluation of the derivatives; hence, there is some circularity here and I guess I got it wrong. Alternatively, I might have messed up the boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example of what I need to solve:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u\colon [0,T]\times \mathbb{R}^d\to \mathbb{R}&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_t-H(u,\nabla u)=0&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H(u,\nabla u)=\min_{\alpha_i\geq 0\\ a\cdot \alpha=1} b\cdot \alpha+(c-\alpha)\cdot \nabla u  &#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u(0,\cdot)=0&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following iteration over finite two-dimensional arrays &lt;span class=&quot;math-container&quot;&gt;$(u_n, \nabla u_n)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0\leq n\leq N$&lt;/span&gt; (with &lt;span class=&quot;math-container&quot;&gt;$\Delta t:= T/N$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;(For each array index) solve the Hamiltonian minimization problem to get &lt;span class=&quot;math-container&quot;&gt;$H(u_{n},\nabla u_{n})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(For each array index) define &lt;span class=&quot;math-container&quot;&gt;$u_{n+1}:=u_n+\Delta t H(u_{n},\nabla u_n)$&lt;/span&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(For each array index) define &lt;span class=&quot;math-container&quot;&gt;$\nabla u_{n+1}$&lt;/span&gt; by differencing &lt;span class=&quot;math-container&quot;&gt;$u_{n+1}$&lt;/span&gt; in upstream direction, as defined by &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;At the boundary, I used &lt;span class=&quot;math-container&quot;&gt;$u_{x_ix_i}:=0$&lt;/span&gt;, i.e., I whenever the upstream method required me to access a non-existent entry of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, I evaluated downstream instead. &lt;/p&gt;&#xA;" OwnerUserId="7789" LastActivityDate="2018-11-26T08:24:01.910" Title="Guide for finite-difference schemes for Hamilton-Jacobi-Bellman Equations" Tags="|pde|finite-difference|reference-request|optimal-control|dynamic-programming|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30595" PostTypeId="1" CreationDate="2018-11-26T16:45:48.907" Score="1" ViewCount="35" Body="&lt;p&gt;I am using 3D FEM with 8-node brick elements to model a certain type of growth/expansion, which is not a plastic deformation. First, I apply a pressure/load and I get displacement and the new position of nodes and calculate stress/strain. Next, I would like to change the 'edges' relaxing length (different edges by different factors) of an 8-node hexahedral element. Such change in relaxing length would be due to addition of a new material, resulting in expansion of a mesh. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, whether in such a model I can calculate some sort of relaxing length for a brick element edges after application of pressure and consequently change it to get new nodal positions. I can imagine doing so for a 1D string, but I am not sure if it makes any sense for a 3D hexahedral element.&lt;/p&gt;&#xA;" OwnerUserId="29315" LastEditorUserId="20688" LastEditDate="2018-11-26T20:36:10.263" LastActivityDate="2018-11-26T20:36:10.263" Title="Is it possible to obtain a 'relaxing lengths' for 8 node hexaedral element" Tags="|finite-element|solid-mechanics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30597" PostTypeId="1" AcceptedAnswerId="30606" CreationDate="2018-11-26T19:47:02.480" Score="6" ViewCount="929" Body="&lt;p&gt;The condition number of function evaluation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathrm{cond}(f,x) := \left| \frac{x f'(x)}{f(x)} \right|&#xA;$$&lt;/span&gt;&#xA;is infinite at a root of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. Hence it is useless for rescaling a tolerance which defines an exit condition from Newton's method. For instance, naively, you would exit the Newton iteration&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x_{k+1} := x_{k} - f(x_{k})/f'(x_{k})&#xA;$$&lt;/span&gt;&#xA;when (say) &lt;span class=&quot;math-container&quot;&gt;$|f(x_{k+1})| &amp;lt; \epsilon$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is (again, say) the double epsilon. However, even if &lt;span class=&quot;math-container&quot;&gt;$x^{*}$&lt;/span&gt; is the correct floating point approximation of the root, we have no guarantee that &lt;span class=&quot;math-container&quot;&gt;$|f(x^{*})| &amp;lt; \epsilon$&lt;/span&gt;, due to the aformentioned ill-conditioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to define a generalized condition number &lt;span class=&quot;math-container&quot;&gt;$\kappa(f, x)$&lt;/span&gt; for Newton's method so that a reasonable termination condition can written in the form &lt;span class=&quot;math-container&quot;&gt;$|f(x_{k})| &amp;lt; \kappa(f, x_{k}) \epsilon$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, even the proposed form of the termination condition is problematic, because the sequence &lt;span class=&quot;math-container&quot;&gt;$\{x_{k}\}$&lt;/span&gt; is invariant under rescaling &lt;span class=&quot;math-container&quot;&gt;$f\mapsto \lambda f$&lt;/span&gt;, but the proposed form of the termination condition is not. Can we write a termination condition which is scale invariant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function which lead me to these questions was&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(z) := 601080390z -185961388.136908293 + 141732493.98435241i&#xA;$$&lt;/span&gt;&#xA;Applying Newton's iteration to this leads to convergence to the true root in just three iterations starting with the guess of &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$|f(z^{*})| \approx 3\times 10^{-8}$&lt;/span&gt;, which somehow feels &quot;far from zero&quot; in double precision, so my termination condition was never satisfied.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2018-11-26T19:57:00.903" LastActivityDate="2018-11-28T15:09:48.860" Title="Defining a condition number and termination criteria for Newton's method" Tags="|newton-method|condition-number|roots|rootfinding|" AnswerCount="1" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="30598" PostTypeId="1" CreationDate="2018-11-27T09:55:39.397" Score="1" ViewCount="963" Body="&lt;p&gt;I want to minimize a cost function of the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_{q,t}\left(q^T\left(\mathcal A + \mathcal B\right)q + t^T\mathcal C t+\delta t+\varepsilon Q(q)^TW(q)t+\lambda\left(1-q^Tq\right)^2\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with the Levenberg-Marquart method using &lt;code&gt;scipy.optimize.least_squares&lt;/code&gt; function. But I do not see how to formulate it in terms of residuals so that I can use such a method. Otherwise, I get the error message:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Method &lt;code&gt;lm&lt;/code&gt; doesn't work when the number of residuals is less than the number of variables.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My cost function is defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def canonical_cost(qv, t, A, B, C, delta, epsilon, lam):&#xA;    assert(type(qv) is np.ndarray and len(qv) == 4)&#xA;    # assert(type(t) is np.ndarray and len(t) == 3)&#xA;&#xA;    q = Quaternion(*qv)&#xA;    qv, tv = qv.reshape(-1, 1), np.vstack(([0], t.reshape(-1, 1)))&#xA;&#xA;    f1 = qv.T @ (A + B) @ qv&#xA;    f2 = tv.T @ C @ tv + delta @ tv + epsilon @ (q.Q.T @ q.W) @ tv&#xA;    qnorm = (1 - qv.T @ qv)**2&#xA;    return np.squeeze(f1 + f2 + lam*qnorm)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I try to optimize with,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def cost(x):&#xA;    qv, t = x[:4], x[4:]&#xA;    return canonical_cost(qv, t, A, B, C, delta, epsilon, lam)&#xA;&#xA;result = opt.least_squares(cost, initial_conditions, method='lm',&#xA;                               **kwargs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3595" LastEditorUserId="20688" LastEditDate="2018-11-27T11:31:41.903" LastActivityDate="2019-09-24T04:05:35.870" Title="Minimize cost with Levenberg-Marquart method" Tags="|optimization|scipy|least-squares|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30600" PostTypeId="1" CreationDate="2018-11-27T16:47:24.940" Score="0" ViewCount="6286" Body="&lt;p&gt;For the diffusion equation &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial u(x,t)}{\partial t} = D \frac{\partial ^2 u(x,t)}{\partial x^2} + Cu(x,t)&#xA;$$&lt;/span&gt;&#xA;with the boundary conditions &lt;span class=&quot;math-container&quot;&gt;$u(-\frac{L}{2},t)=u(\frac{L}{2},t)=0$&lt;/span&gt; I've programmed the numerical solution into python correctly (I think).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;L=np.pi # value chosen for the critical length&#xA;s=101 # number of steps in x&#xA;t=10002 # number of timesteps&#xA;ds=L/(s-1) # step in x&#xA;dt=0.0001 # time step&#xA;D=1 # diffusion constant, set equal to 1&#xA;C=1 # creation rate of neutrons, set equal to 1&#xA;Alpha=(D*dt)/(ds*ds) # constant for diffusion term&#xA;Beta=C*dt # constant for u term&#xA;&#xA;x = np.linspace(-L/2, 0, num=51)&#xA;x = np.concatenate([x, np.linspace(x[-1] - x[-2], L/2, num=50)]) # setting x in the specified interval&#xA;&#xA;u=np.zeros(shape=(s,t)) #setting the function u&#xA;u[50,0]=1/ds # delta function&#xA;for k in range(0,t-1):&#xA;    u[0,k]=0 # boundary conditions&#xA;    u[s-1,k]=0&#xA;    for i in range(1,s-1):&#xA;        u[i,k+1]=(1+Beta-2*Alpha)*u[i,k]+Alpha*u[i+1,k]+Alpha*u[i-1,k] # numerical solution  &#xA;    if k == 50 or k == 100 or k == 250 or k == 500 or k == 1000 or k == 10000: # plotting at times&#xA;        plt.plot(x,u[:,k])&#xA;&#xA;plt.title('Numerical Solution of the Diffusion equation over time')&#xA;plt.xlabel('x')&#xA;plt.ylabel('u(x,t)')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However now I have to change the right boundary condition into &lt;span class=&quot;math-container&quot;&gt;$u_x(\frac{L}{2})=0$&lt;/span&gt; and I'm not really sure how to change my code to reflect this. If I do this the critical length should decrease and the function should start increasing exponentially, but everything I've tried usually does nothing to my plot - is there something wrong with my original code possibly? Any help is really appreciated, I've been trying for ages but can't seem to get it! Thanks!&lt;/p&gt;&#xA;" OwnerUserId="29330" LastActivityDate="2018-11-28T00:24:37.083" Title="Applying neumann boundary conditions to diffusion equation solution in python" Tags="|finite-difference|python|boundary-conditions|numpy|diffusion|" AnswerCount="1" CommentCount="1" ClosedDate="2018-11-28T16:56:18.063" ContentLicense="CC BY-SA 4.0" />
  <row Id="30601" PostTypeId="2" ParentId="30600" CreationDate="2018-11-27T23:06:24.217" Score="0" Body="&lt;p&gt;If you approximate &lt;span class=&quot;math-container&quot;&gt;$u_x(\frac{L}{2},t) = 0$&lt;/span&gt; with a second-order difference, you get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u_x(\frac{L}{2},t) \approx \frac{1}{2\Delta x}\left(u_s^k - u_{s-2}^k \right) = 0$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which simplifies to &lt;span class=&quot;math-container&quot;&gt;$u_s^k = u_{s-2}^k$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, write the equation for the last node, &lt;span class=&quot;math-container&quot;&gt;$u_{s-1}$&lt;/span&gt;, just like you would for an interior node:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{1}{\Delta t} \left(u_{s-1}^{k+1} - u_{s-1}^{k} \right) = \frac{D}{\Delta x^2} \left(u_s^k - 2 u_{s-1}^k + u_{s-2}^k \right) + C u_{s-1}^k$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the above expression involves &lt;span class=&quot;math-container&quot;&gt;$u_s$&lt;/span&gt;, which is &lt;strong&gt;outside&lt;/strong&gt; the domain (a &lt;em&gt;ghost&lt;/em&gt; node), so we need to eliminate it by using the Neumann BC equation obtained earlier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substituting &lt;span class=&quot;math-container&quot;&gt;$u_s^k = u_{s-2}^k$&lt;/span&gt; into the equation for &lt;span class=&quot;math-container&quot;&gt;$u_{s-1}$&lt;/span&gt; and simplifying gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u_{s-1}^{k+1} = u_{s-1}^{k} + \frac{D \Delta t}{\Delta x^2} \left(-2 u_{s-1}^k + 2 u_{s-2}^k \right) + C \Delta t\ u_{s-1}^k$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in your code, you need to replace the right Dirichlet BC with the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u[s-1,k+1]=(1+Beta-2*Alpha)*u[s-1,k] + 2*Alpha*u[s-2,k]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29276" LastEditorUserId="29276" LastEditDate="2018-11-28T00:24:37.083" LastActivityDate="2018-11-28T00:24:37.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30602" PostTypeId="1" AcceptedAnswerId="30615" CreationDate="2018-11-28T01:51:21.373" Score="2" ViewCount="497" Body="&lt;p&gt;In my numerical linear algebra class, I learned that for some matrices, it could have an element that is a very small number that is approximately 0 (and many orders of magnitude different from all the other non-zero elements), can destroy the conditioning of a matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am wondering how this works practically. When we formulate the linear system &lt;span class=&quot;math-container&quot;&gt;$$Ax=b$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of math goes into forming the operator &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. In the case of solving PDEs, we run into a lot of round-off issues during computations, and this, will certainly affect &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. In some of the matrices I've formed, my operator is a sparse matrix. I notice that some elements of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; end up being approximately 0, but not actual zero, decreasing the sparsity, and perhaps destroying the conditioning of the matrix? Theoretically, those computations should have evaluated zero, but due to roundoff, they end up non-zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be a good approach to go through the non-zero elements of a sparse matrix and filter out all the values close to zero and set them as zero? I have thought about doing this, but I am unsure if this is the right approach. What if those numbers that I am filtering are not theoretically zero, and they are actually very small? &lt;/p&gt;&#xA;" OwnerUserId="28865" LastActivityDate="2018-11-30T01:08:10.283" Title="Condition number of matrix and effects of round off errors" Tags="|linear-algebra|linear-solver|condition-number|linear-system|conditioning|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="30604" PostTypeId="1" CreationDate="2018-11-28T06:20:16.403" Score="-1" ViewCount="121" Body="&lt;p&gt;I recently moved towards computational biophysics from an experimental science background. As of now, I am learning the fundamentals and doing some basic monte carlo simulations of LJ fluid on my desktop PC. However, when I tried to run my code on the cluster, it was equally or more slower than my desktop. Later, my lab mates told me to write my code suitable for parallel computation. I have no idea how to do that. Is there any resource, book aimed for people who are not very proficient in computer science, that can teach them about basics of such programming and necessary tools. &#xA;I wrote my code on c++, I am willing to move to fortran as well.&lt;/p&gt;&#xA;" OwnerUserId="29337" LastActivityDate="2018-11-28T16:44:09.700" Title="Writing parallel code for molecular computation" Tags="|parallel-computing|simulation|monte-carlo|" AnswerCount="1" CommentCount="0" ClosedDate="2018-11-28T16:56:30.930" ContentLicense="CC BY-SA 4.0" />
  <row Id="30605" PostTypeId="1" AcceptedAnswerId="30614" CreationDate="2018-11-28T09:02:32.683" Score="3" ViewCount="1666" Body="&lt;p&gt;when solving the advection equation in 1D that is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$  \frac{\partial u}{\partial t} + c\frac{\partial u}{\partial x} = 0  $$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$ u'(t,0) = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u(t,L) = 0$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$u(0,x) = u_{0} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one numerical scheme is the FTCS (Forward time-centered space), but this numerical scheme is unstable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{u_{j}^{n+1}-u_{j}^{n}}{ h_{t}} = c \frac{u_{j+1}^{n}-u_{j-1}^{n}}{ 2h_{x}} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when solving &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$  \frac{\partial u}{\partial t} + c\frac{\partial u}{\partial x} = \alpha \frac{\partial^2 u}{\partial x^2}   $$&lt;/span&gt;&#xA;the advection-diffusion equation in 1D with &lt;span class=&quot;math-container&quot;&gt;$ u'(t,0) = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u(t,L) = 0$&lt;/span&gt; , &lt;span class=&quot;math-container&quot;&gt;$u(0,x) = u_{0} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the advection-diffusion equation is a second order equation I'd like to use a second order approximation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;if we define &lt;span class=&quot;math-container&quot;&gt;$u_{k}^{n} := u(t_{n},x_{k}) $&lt;/span&gt;;    &lt;span class=&quot;math-container&quot;&gt;$\ \  x_{k} = kh $&lt;/span&gt;  and &lt;span class=&quot;math-container&quot;&gt;$ \ \ k = 0,1,2,...,N$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is known as the mesh size or step size.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second derivative:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial^2u_{k}^{n}}{\partial x^2} \approx \frac{u_{k+1}^{n}-2u_{k}^{n}+u_{k-1}^{n}}{h^2}  = \frac{u_{k-1}^{n}-2u_{k}^{n}+u_{k+1}^{n}}{h^2}   $$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$k=0,1,...,N-1$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$u'(t,0) =  0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ u(t_{n},L) = u(t_{n},x_{N}) = u_{N}^{n} = 0 $&lt;/span&gt; we get the following matrix representation of the second derivative  operator&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} &#xA;   \frac{\partial^2}{\partial x^2} \approx L_{2} = \frac{1}{h^2}\left(\begin{matrix}&#xA;-2 &amp;amp; 1 &amp;amp; &amp;amp; 0\\&#xA;1 &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \\&#xA;&amp;amp; \ddots &amp;amp; \ddots &amp;amp; 1 \\&#xA;0 &amp;amp; &amp;amp; 1 &amp;amp; -2 \end{matrix} \right)&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for &lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt; , we get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial u_{0}^{n}}{\partial x} = \frac{u_{0+1}^{n}-u_{0-1}^{n}}{ 2h} = 0 $$&lt;/span&gt; this implies that  &lt;span class=&quot;math-container&quot;&gt;$ u_{1}^{n}=u_{-1}^{n} $&lt;/span&gt; and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial^2u_{0}^{n}}{\partial x^2} \approx \frac{u_{0+1}^{n}-2u_{0}^{n}+u_{0-1}^{n}}{h^2}  = \frac{u_{0-1}^{n}-2u_{0}^{n}+u_{0+1}^{n}}{h^2}  = \frac{-2u_{0}^{n}+2u_{1}^{n}}{h^2}   $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thus we have to modify the entry &lt;span class=&quot;math-container&quot;&gt;$1,2$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$L_{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} &#xA;    L_{2} = \frac{1}{h^2}\left(\begin{matrix}&#xA;-2 &amp;amp; 2 &amp;amp; &amp;amp; 0\\&#xA;1 &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \\&#xA;&amp;amp; \ddots &amp;amp; \ddots &amp;amp; 1 \\&#xA;0 &amp;amp; &amp;amp; 1 &amp;amp; -2 \end{matrix} \right)&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have done, is &lt;span class=&quot;math-container&quot;&gt;$\mathbf{impose \  the \ Neumann \ boundary \ condition}$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$L_{2}$&lt;/span&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to approximate the first derivative using central difference(Second order approximation):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial u_{k}^{n}}{\partial x} = \frac{u_{k+1}^{n}-u_{k-1}^{n}}{ 2h} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix representation is:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation} &#xA;   \frac{\partial}{\partial x} \approx L_{1} = \frac{1}{2h}\left(\begin{matrix}&#xA;0 &amp;amp; 1 &amp;amp; &amp;amp; 0\\&#xA;-1 &amp;amp; \ddots &amp;amp; \ddots &amp;amp; \\&#xA;&amp;amp; \ddots &amp;amp; \ddots &amp;amp; 1 \\&#xA;0 &amp;amp; &amp;amp; -1 &amp;amp; 0 \end{matrix} \right)&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for &lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt; , we get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\partial u_{0}^{n}}{\partial x} = \frac{u_{0+1}^{n}-u_{0-1}^{n}}{ 2h} = 0 $$&lt;/span&gt; this implies that  &lt;span class=&quot;math-container&quot;&gt;$ u_{1}^{n}=u_{-1}^{n} $&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm stuck when I try to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{impose \ the \ neumann \ boundary \ condition \ in}$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$L_{1}$&lt;/span&gt;.&#xA;I don't know how to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we solve that problem, we can solve the differential equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{ \partial u }{\partial t} = \Big(-cL_{1} +\alpha L_{2} \Big)u  $$&lt;/span&gt;&#xA;integrating in time( C-N, Back-Euler, RK4 )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{please \ help!\  \ How \ do \ you \ impose \ the \ Neumann \ Boundary \ Condition \ in \ L_{1}?}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="28594" LastActivityDate="2018-11-30T04:14:12.173" Title="Impose Neumann Boundary Condition in advection-diffusion equation 1D" Tags="|finite-difference|boundary-conditions|advection-diffusion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30606" PostTypeId="2" ParentId="30597" CreationDate="2018-11-28T15:09:48.860" Score="2" Body="&lt;p&gt;Any reasonable convergence criterion must be invariant to scaling of the function. A decent stopping criterion is therefore if &lt;span class=&quot;math-container&quot;&gt;$\|f(x_k)\|≤ \epsilon\|f(x_0)\|$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; is the starting point of the iteration. It's slightly annoying that the stopping criterion now depends on the starting point, but because Newton's iteration converges so quickly close to the solution, this criterion in practice works pretty well. &lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-11-28T15:09:48.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30607" PostTypeId="2" ParentId="30604" CreationDate="2018-11-28T16:44:09.700" Score="0" Body="&lt;p&gt;As was mentioned in the comments, &lt;a href=&quot;https://scicomp.stackexchange.com/questions/26681/a-good-simple-book-resource-on-parallel-programming-in-c-for-scientific-compu&quot;&gt;A good, simple book/resource on Parallel Programming in C++ for scientific computing&lt;/a&gt; already gives an answer to the more general question of learning about parallel programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, a simple Lennard-Jones Monte Carlo program may not be a bad project to attempt, if you decide to go ahead and learn about parallel programming. The OpenMP approach is relatively easy to understand and implement. Many compilers have a -fopenmp flag, which will switch on the processing of any OpenMP &lt;code&gt;pragma&lt;/code&gt;s in the code. A typical &lt;code&gt;pragma&lt;/code&gt; is one that is applied to a &lt;code&gt;for&lt;/code&gt; loop (or in Fortran, a &lt;code&gt;do&lt;/code&gt; loop), to divide the iterations amongst the parallel threads. As an example, if you wanted to sum up &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; numbers, and you have &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; threads available, each thread could sum up (approximately) &lt;span class=&quot;math-container&quot;&gt;$N/P$&lt;/span&gt; of the numbers, in parallel, and then the results can be combined at the end. You would need to learn some basic OpenMP concepts first: the difference between &lt;code&gt;private&lt;/code&gt; and &lt;code&gt;shared&lt;/code&gt; variables, and crucially the idea of a &lt;code&gt;reduction&lt;/code&gt; variable, which would hold the result of the summation. But it doesn't take too long to master OpenMP at this level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point is that, in the simplest Monte Carlo program, the time consuming part is exactly such a loop. When you consider moving an atom, you need to compute the sum of Lennard-Jones pair potentials between it and all the other atoms, before and after the move. This assumes that you have a relatively small system, and are not using sophisticated techniques such as domain decomposition / neighbour lists to speed things up. Basically, if you can parallelize a simple summation loop, it is not a huge stretch to parallelize the single-particle potential energy calculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On your cluster, it is quite possible that each node consists of several cores with access to the same memory: if so, then you could use OpenMP to run on a single node, with each thread corresponding to one of those cores. So, in principle, best-case scenario, you might get a speedup of however many cores there are per node.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For larger-scale Monte Carlo, a more sophisticated approach may be needed. Parallelizing large-scale Monte Carlo codes can be done (e.g. by using domain decomposition and writing a program that uses MPI), but is more fiddly and needs more care (because of the need to ensure detailed balance) than this simple OpenMP example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you decide to proceed in this way, you should definitely follow up some of the resources mentioned above first. But after you've learned the basics, it's actually just a question of inserting a very few extra lines in a simple program to sum up a set of numbers, as a test, and then doing a similar thing to your code, and compiling with the appropriate option.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-11-28T16:44:09.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30609" PostTypeId="1" CreationDate="2018-11-29T14:41:13.580" Score="5" ViewCount="93" Body="&lt;p&gt;I'm using SIMD processor arrays to compute the eigen-decomposition for large numbers of small (up to &lt;span class=&quot;math-container&quot;&gt;$32\times 32$&lt;/span&gt;) matrices. For assorted technical reasons, Jacobi iteration maps well to the SIMD architecture and gives excellent results for diagonalisation of symmetric, Hermitian and normal matrices using unitary similarity transformations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've extended the technique to compute the Schur triangular form of general complex matrices using the same unitary similarity transformations in order to obtain the eigenvalues from the diagonal. Again, convergence to the Schur form is typically excellent for most random complex matrices up to &lt;span class=&quot;math-container&quot;&gt;$32\times 32$&lt;/span&gt; size but I observe very poor convergence for a small number of random matrices. As I increase the size of the number of random matrices tested to thousands, tens of thousands, millions, the number of Jacobi iterations required increases indefinitely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some form of pre-conditioning that can improve convergence? I've tried an initial reduction to upper Hessenberg (i.e. almost triangular) with no improvement.&lt;/p&gt;&#xA;" OwnerUserId="29351" LastEditorUserId="20688" LastEditDate="2018-11-29T15:35:32.817" LastActivityDate="2018-11-29T15:35:32.817" Title="Improving convergence of Jacobi iteration to Schur form" Tags="|linear-algebra|eigenvalues|iterative-method|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30610" PostTypeId="2" ParentId="20643" CreationDate="2018-11-29T15:11:00.610" Score="1" Body="&lt;p&gt;Due to the potential presence of elliptic regions in your solution, it's not clear to me if equation you are considering has a well-posed initial value problem (IVP). In general for mixed-type PDE one does not have a well-posed IVP; one typically needs to impose other types of boundary conditions. If you are able to find boundary conditions for the boundary of some region in the t-x plane, you may be able to use a numerical procedure outlined in sec. IV of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arc.aiaa.org/doi/10.2514/3.6131&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arc.aiaa.org/doi/10.2514/3.6131&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that section, the authors numerically solve and equation of the form (their Eq. 33)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ 0 = (A - \partial_x\phi) \partial_x^2\phi + \partial_y^2\phi $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is elliptic when &lt;span class=&quot;math-container&quot;&gt;$\partial_x\phi&amp;lt;A$&lt;/span&gt; and hyperbolic when &lt;span class=&quot;math-container&quot;&gt;$\partial_x\phi &amp;gt; A$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking for more work on numerical solutions to mixed-type pde, hyperbolic-elliptic type pde seem to appear somewhat frequently in, for example, aeronautics. Unfortunately the specific method one uses depends a lot on the PDE in question.&lt;/p&gt;&#xA;" OwnerUserId="23388" LastActivityDate="2018-11-29T15:11:00.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30613" PostTypeId="1" AcceptedAnswerId="30625" CreationDate="2018-11-29T22:08:23.463" Score="3" ViewCount="121" Body="&lt;p&gt;I am repeatedly solving quadratic program,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x^T Q x$&lt;/span&gt; with time dependent linear constraints &lt;span class=&quot;math-container&quot;&gt;$Ax=b_t$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dimension of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is around 10000 and there are around 50 constraints. I want to solve the system as fast as possible at each time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. I know that &lt;span class=&quot;math-container&quot;&gt;$b_t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b_{t+1}$&lt;/span&gt; are pretty close, say, in L2-norm. I also know, that the optimal solution &lt;span class=&quot;math-container&quot;&gt;$x^*|b_t$&lt;/span&gt; is pretty close to &lt;span class=&quot;math-container&quot;&gt;$x^*|b_{t+1}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how can I best employ this knowledge to speed up the computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I am thinking about iterative minimization of the corresponding Lagrangian with providing the previous solution as an initial guess. I can also do a lot of precomputations and evaluate expensive inversions such as &lt;span class=&quot;math-container&quot;&gt;$Q^{-1}A^{T}$&lt;/span&gt;. The dependence of the solution on &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; also leads me to some sensitivity analysis, but I’m clueless here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is obtaining the solution in 5 s, which as I am inexperienced in this area, don’t know if it’s even possible.&lt;/p&gt;&#xA;" OwnerUserId="29353" LastEditorUserId="29353" LastEditDate="2018-12-01T08:20:56.467" LastActivityDate="2018-12-01T20:10:16.890" Title="Improve optimization speed for a set of similar problems: Quadratic programming with a warm start" Tags="|nonlinear-equations|constrained-optimization|quadratic-programming|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30614" PostTypeId="2" ParentId="30605" CreationDate="2018-11-30T01:02:09.797" Score="5" Body="&lt;p&gt;First, write out the semi-discrete equation for &lt;span class=&quot;math-container&quot;&gt;$u_0$&lt;/span&gt;, assuming it's an &lt;em&gt;interior&lt;/em&gt; node:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \frac{d}{dt}(u_0) = - c \frac{\left( u_1 - u_{-1} \right)}{2h} + \alpha \frac{\left(u_1 - 2u_0 + u_{-1}\right)}{h^2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, eliminate the ghost node &lt;span class=&quot;math-container&quot;&gt;$u_{-1}$&lt;/span&gt; by using the equation you obtained from the Neumann BC: &lt;span class=&quot;math-container&quot;&gt;$u_{-1} = u_1$&lt;/span&gt;. Substituting this condition into the above equation gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \frac{d}{dt}(u_0) = \alpha \frac{\left(- 2u_0 + 2u_1 \right)}{h^2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the advection term vanishes, so the first row of your &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt; matrix should be all zeros. The first row of the &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; matrix should have coefficients of &lt;span class=&quot;math-container&quot;&gt;$-2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; in the first two columns, as you have already derived.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also something to keep in mind, using a central difference approximation for the first derivative in the advection term will cause stability issues if your problem is advection dominant. In particular, this central difference scheme may develop oscillations if the grid Peclet number is greater than &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;, where the Peclet number is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Pe = \frac{c h}{\alpha}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can avoid these oscillations by using an &lt;em&gt;upwinded&lt;/em&gt; (biased) approximation, such as this first-order upwinded scheme:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial u_k}{\partial x} \approx \frac{u_k - u_{k-1}}{h}$&lt;/span&gt;, if &lt;span class=&quot;math-container&quot;&gt;$c &amp;gt; 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Higher-order upwinded approximations also do exist, and typically involve wider stencils.&lt;/p&gt;&#xA;" OwnerUserId="29276" LastEditorUserId="29276" LastEditDate="2018-11-30T04:14:12.173" LastActivityDate="2018-11-30T04:14:12.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30615" PostTypeId="2" ParentId="30602" CreationDate="2018-11-30T01:08:10.283" Score="2" Body="&lt;p&gt;For sparse matrices &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, that are a discretized version of operators in PDEs in FE, FV, or FD, you do know your sparsity pattern &lt;em&gt;before you compute the actual entries&lt;/em&gt;. So, you usually&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;compute matrix elements of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; that are not zeros according to the sparsity pattern&lt;/li&gt;&#xA;&lt;li&gt;do not do anything with your known zeros.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In your question, it seems that you want to go one step further and analyze what is computed on step 1 and preemptively zero it out according to some tolerance criteria. Effectively, you are going to mess up with your discretized operator and solve some &lt;em&gt;approximate version&lt;/em&gt; of that when certain values are filtered. I am not aware of this technique being practically applied and I can think of several reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the fact that some matrix element is relatively small does not necesserily imply that the condition number will be ruined. The condition number of the matrix is determined by singular values that certainly &quot;come from matrix entries&quot;, but not in a such straigtforward manner. As @Spencer correctly pointed out in the comments, your small matrix entry should add a near-zero singular value.&lt;/li&gt;&#xA;&lt;li&gt;filtering out matrix entries is not the way to go. One can try improve the solution by filtering out singular values. In short: compute SVD decomposition of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, truncate it by some tolerance, solve the problem. However, that is usually done for ill-posed problems (say inverse problems), and I am not aware of this approach being applied for forward solution of PDEs.&lt;/li&gt;&#xA;&lt;li&gt;filtering out of matrix entries can technically be used to form a preconditioner. But it is used together with your unfiltered matrix within an iterative solution of the system of linear equations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, I would certainly suggest to check if those entries actually influence the condition number of the matrix (which is unlikely). And choosing a different discretization scheme (geometry/operator) might be a much more promising route.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-11-30T01:08:10.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30616" PostTypeId="1" CreationDate="2018-11-30T09:01:51.980" Score="0" ViewCount="68" Body="&lt;p&gt;I bought three boxes of tea bags with different flavors (A, B, C).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to mix them in such a way that &#xA;- there is never two consecutive bags of the same flavor (ABCCAB is avoided) ;&#xA;- the mixing is the &quot;most&quot; random, i.e. avoid patterns such as ABCABCABC... or &#xA;ABABAB...BCBCBC...CACACA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a known algorithm for this mix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently I randomly shuffle many &quot;ABC&quot; and concatenate the results, swapping the first letters if the latest letter of the previous shuffle is the same than the beginning of the new shuffle (...ABCCAB =&gt; ...ABCACB).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess I could improve this algorithm by pre-computing the permutations of ABC, and draw one permutation among the ones who do not begin with the same letter than the previous permutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to &quot;google&quot; this problem but as a French native speaker, I probably miss the appropriate key-words.&lt;/p&gt;&#xA;" OwnerUserId="29356" LastEditorUserId="20688" LastEditDate="2018-12-01T07:26:58.847" LastActivityDate="2018-12-01T07:26:58.847" Title="tea bag flavors mixing algorithm" Tags="|algorithms|combinatorics|" AnswerCount="1" CommentCount="3" ClosedDate="2018-12-02T18:30:05.847" ContentLicense="CC BY-SA 4.0" />
  <row Id="30617" PostTypeId="2" ParentId="30616" CreationDate="2018-11-30T16:54:17.497" Score="2" Body="&lt;p&gt;The first idea that would come to my mind naturally is: after each bag, choose with 50% probability one of the remaining two. Does this avoid &quot;special&quot; patterns such as ABABAB? No, they just have a low chance of appearing. And that's how it should be, in a true random number generator. Avoiding special patterns give the &lt;em&gt;impression&lt;/em&gt; of randomness, but true random numbers may have them (exactly as you have already noticed with equal consecutive numbers).&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2018-11-30T16:54:17.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30618" PostTypeId="1" AcceptedAnswerId="30622" CreationDate="2018-11-30T17:36:21.187" Score="1" ViewCount="258" Body="&lt;p&gt;I am looking for some intuition in making comparisons of FEM simulations to experimental measurements. In particular, I am interested in comparisons to strain gauge readings, and perhaps even LVDTs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an FEM simulation, the strains you compute would typically be at a certain node's location. For an FVM simulation, the strain you compute would typically be at a cell centered location. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you make a strain gauge measurement at a location where your simulation is outputting a strain computation, could you make a direct comparison with the experimental measurement, or is some manipulation of the simulation results needed prior to making a direct comparison? &lt;/p&gt;&#xA;" OwnerUserId="28865" LastActivityDate="2018-12-01T15:36:52.473" Title="Strain from FEM simulations to strain gauge measurements" Tags="|finite-element|finite-volume|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30620" PostTypeId="1" AcceptedAnswerId="30623" CreationDate="2018-12-01T00:01:10.647" Score="6" ViewCount="284" Body="&lt;p&gt;If you compute &lt;code&gt;1+x&lt;/code&gt; for &lt;code&gt;x&lt;/code&gt; less than the machine precision, the answer will be &lt;code&gt;1&lt;/code&gt; which is the exact result for &lt;code&gt;x = 0&lt;/code&gt;. But this would then imply that the relative backward error is &lt;code&gt;|0 - x| / |x| = 1&lt;/code&gt;, i.e. this method is not backwards stable and no method based on standard floating point arithmetic can ever be. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It feels like this finding must be wrong, because how can what is literally the simplest possible operation not be backwards stable? So where is fault in the above reasoning?&lt;/p&gt;&#xA;" OwnerUserId="21342" LastActivityDate="2018-12-02T17:07:31.793" Title="1+x not backwards stable?" Tags="|stability|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30621" PostTypeId="1" CreationDate="2018-12-01T11:15:57.063" Score="2" ViewCount="238" Body="&lt;p&gt;In my full 3D solutions I am solving for the potential throughout a &lt;span class=&quot;math-container&quot;&gt;$100\times 200\times 200$&lt;/span&gt; grid. Inside is a ring electrode set to -5V via a Dirichlet boundary condition, and surrounded on all sides by Dirichlet boundary conditions at 0V.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I solved for this setup in full 3D vs solving for the same setup in 2D axisymmetric form, I get different results. In the images below, one can denote the darker blue color, in the very center of each of the two solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the 2D axisymmetric setup, the left, top, and right all have the same 0V Dirichlet boundaries as the 3D setup except for the Neumann boundary on the bottom where the axis of symmetry for the problem resides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below you can see two images. In both images, the object on the left side is a 'slice' of the 3D solution, and the object(s) on the right is the solution for the 2D axisymmetric solution. The first image shows the 2D axisymmetric solution, whereas the second image has a 'mirror' image of that solution placed just below its counterpart, to make comparing the two solutions (3D vs 2D axisymmetric) that much easier to see.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is it in any way common for such a marked difference in the results of 3D vs 2D axisymmetric results for elliptic PDEs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t3JO8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t3JO8.png&quot; alt=&quot;3D (left, sliced) vs 2D axisymmetric solutions&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/K8joE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K8joE.png&quot; alt=&quot;3D (left, sliced) vs 2D axisymmetric solutions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt; I would like to thank Bill and Wolfgang for their constructive questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing that the 2D Axisymmetric and 3D solutions should be the same, I modified my 3D setup and used a &lt;span class=&quot;math-container&quot;&gt;$100\times199\times199$&lt;/span&gt; grid instead, with &lt;span class=&quot;math-container&quot;&gt;$(i, 99, 99)$&lt;/span&gt; as the axis for the ring electrode, and setting Dirichlet boundaries at a radius of &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; from said axis. The solutions in 3D now match my 2D axisymmetric solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are the images showing the now matching solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vZuDv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vZuDv.png&quot; alt=&quot;Matching 2D axisymmetric and 3D solutions&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/V6baV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V6baV.png&quot; alt=&quot;Matching 2D axisymmetric and 3D solutions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29362" LastEditorUserId="29362" LastEditDate="2018-12-02T18:29:01.763" LastActivityDate="2018-12-02T18:29:01.763" Title="Are there known accuracy issues between 2D axisymmetric and 3D solutions?" Tags="|boundary-conditions|numerical-modelling|elliptic-pde|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30622" PostTypeId="2" ParentId="30618" CreationDate="2018-12-01T15:36:52.473" Score="2" Body="&lt;p&gt;Your physical strain gauge gives you a single number, which means that it is represented by a functional &lt;span class=&quot;math-container&quot;&gt;$\varphi(\cdot)$&lt;/span&gt; applied to the solution and its gradient (strain) everywhere. Let's say that it only depends on the strain; then the Riesz representation theorem tells us that the operation the strain gauge performs must be of the form&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \varphi(\varepsilon(\mathbf u))&#xA;  =&#xA;  \int_\Omega a(\mathbf x) : \varepsilon(\mathbf u(\mathbf x)) \, \text{d}x &#xA;$$&lt;/span&gt;&#xA;for some &lt;span class=&quot;math-container&quot;&gt;$a(\mathbf x)\in L^2(\Omega)^{d\times d}$&lt;/span&gt;. (&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; must be in &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; because the strain is in &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; because the solution is in &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So then you have to model how exactly this looks. Since strain gauges have a finite size, your &lt;span class=&quot;math-container&quot;&gt;$a(\mathbf x)$&lt;/span&gt; is likely a function that is constant in some small subdomain and zero outside. Then what you will need to evaluate for your numerical solution is the approximate value of the predicted strain:&#xA;&lt;span class=&quot;math-container&quot;&gt;$&#xA;  \varphi(\varepsilon(\mathbf u_h))&#xA;$&lt;/span&gt;. If the area where &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is nonzero is smaller than a cell, then you can approximate the integral by taking the value of the (computed approximation to the) strain at an individual node or face, but in general you will want to actually compute this integral: if your mesh is fine enough, then this will involve integration over more than one cell, and you will want to approximate the integral using quadrature.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-12-01T15:36:52.473" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30623" PostTypeId="2" ParentId="30620" CreationDate="2018-12-01T18:16:59.900" Score="3" Body="&lt;p&gt;It is indeed true that this function does not allow for a backwards stable algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The confusion arose because the lecture notes that I was reading seemed to imply that backwards stability is what every algorithm should aim for, however as the above example clearly shows there are algorithms for which no backwards stable algorithms exist. I did some more more reading in the meantime and I learnt that it is not backwards stability, but rather mixed stability that you should aim for. According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_stability&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The usual definition of numerical stability uses a more general concept, called mixed stability, which combines the forward error and the backward error. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="21342" LastActivityDate="2018-12-01T18:16:59.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30624" PostTypeId="1" CreationDate="2018-12-01T19:58:48.430" Score="1" ViewCount="97" Body="&lt;p&gt;I am (naively) trying to solve the 2d wave equation with finite differences. But the system blows up instantly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity I set the constant &lt;span class=&quot;math-container&quot;&gt;$c=1$&lt;/span&gt;, then I am left with&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Delta u =u_{tt}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I set up the FDM with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 9&#xA;u = np.zeros((N**2,1))&#xA;u.reshape(N,N)[N//2,N//2] = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, at the time &lt;code&gt;t=0&lt;/code&gt; the function looks like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u_0 = &#xA;[[0 0 0 0 0 0 0 0 0]&#xA; [0 0 0 0 0 0 0 0 0]&#xA; [0 0 0 0 0 0 0 0 0]&#xA; [0 0 0 0 0 0 0 0 0]&#xA; [0 0 0 0 1 0 0 0 0]&#xA; [0 0 0 0 0 0 0 0 0]&#xA; [0 0 0 0 0 0 0 0 0]&#xA; [0 0 0 0 0 0 0 0 0]&#xA; [0 0 0 0 0 0 0 0 0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To which I apply the five point diffence stencil &lt;code&gt;Lh&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.sparse as sp&#xA;import numpy as np&#xA;h = 1/(N+1)&#xA;T = np.diag(N*[-2]) + np.diag((N-1)*[1],k=-1) +np.diag((N-1)*[1],k=1)    &#xA;spt = sp.coo_matrix(T)&#xA;spi = sp.eye(N)&#xA;Lh = (sp.kron(spi,spt) + sp.kron(spt,spi) ) / h**2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since &lt;code&gt;Lh&lt;/code&gt; is my discretized Laplacian, for every timestep I add the acceleration that it gives me to some book-keeping vector &lt;code&gt;v&lt;/code&gt; to change the velocity accordingly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;v = np.zeros_like(u)&#xA;print(u.reshape(N,N).astype(int))&#xA;for i in range(3):&#xA;    v += Lh.dot(u)&#xA;    u += v&#xA;    print(u.reshape(N,N).astype(int))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is what happens:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[   0    0    0    0    0    0    0    0    0]&#xA; [   0    0    0    0    0    0    0    0    0]&#xA; [   0    0    0    0    0    0    0    0    0]&#xA; [   0    0    0    0   99    0    0    0    0]&#xA; [   0    0    0   99 -398   99    0    0    0]&#xA; [   0    0    0    0   99    0    0    0    0]&#xA; [   0    0    0    0    0    0    0    0    0]&#xA; [   0    0    0    0    0    0    0    0    0]&#xA; [   0    0    0    0    0    0    0    0    0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the next timestep:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[     0      0      0      0      0      0      0      0      0]&#xA; [     0      0      0      0      0      0      0      0      0]&#xA; [     0      0      0      0   9999      0      0      0      0]&#xA; [     0      0      0  19999 -79699  19999      0      0      0]&#xA; [     0      0   9999 -79699 198800 -79699   9999      0      0]&#xA; [     0      0      0  19999 -79699  19999      0      0      0]&#xA; [     0      0      0      0   9999      0      0      0      0]&#xA; [     0      0      0      0      0      0      0      0      0]&#xA; [     0      0      0      0      0      0      0      0      0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the problem here? I have not set up a boundary condition to my solution. Maybe I should?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;greetings&lt;/p&gt;&#xA;" OwnerUserId="29367" LastEditorUserId="29367" LastEditDate="2018-12-01T20:09:12.710" LastActivityDate="2018-12-01T20:09:12.710" Title="2d wave equation with finite differences blowing up" Tags="|pde|finite-difference|python|" AnswerCount="0" CommentCount="4" ClosedDate="2019-05-21T20:39:20.867" ContentLicense="CC BY-SA 4.0" />
  <row Id="30625" PostTypeId="2" ParentId="30613" CreationDate="2018-12-01T20:10:16.890" Score="2" Body="&lt;p&gt;Here's an approach for dense &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; that does some relatively expensive preliminary computations that make the individual solutions very fast.  I doubt that any iterative approach would be faster for the case where &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; are dense.  If &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; are very sparse, you might be better off with an iterative approach.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We'll assume that &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is symmetric (any quadratic form can easily be symmetrized) and positive definite (if not, then the problem could be unbounded.)  In this case, the optimal solution is unique, and the Karush-Kuhn-Tucker conditions can be written as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;\left[&#xA;\begin{array}{cc}&#xA;Q &amp;amp; A^{T} \\&#xA;A &amp;amp; 0     &#xA;\end{array}&#xA;\right]&#xA;\left[&#xA;\begin{array}{c}&#xA;x \\&#xA;\lambda &#xA;\end{array}&#xA;\right]=&#xA;\left[&#xA;\begin{array}{c}&#xA;0 \\&#xA;b&#xA;\end{array}&#xA;\right]&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be reduced to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;AQ^{-1}A^{T}\lambda=-b&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;x=-Q^{-1}A^{T}\lambda.&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before solving any of these systems, we can precompute the Cholesky factorization&#xA;of &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;Q=R^{T}R&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then use the Cholesky factorization to get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;B=Q^{-1}A^{T}&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and let&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;C=AQ^{-1}A^{T}=AB&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you want to avoid computing &lt;span class=&quot;math-container&quot;&gt;$Q^{-1}$&lt;/span&gt;, since this is much more expensive than computing &lt;span class=&quot;math-container&quot;&gt;$Q^{-1}A^{T}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also compute the Cholesky factorization of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;AQ^{-1}A^{T}=C=S^{T}S.&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for each right hand side &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, we use &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; to solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;AQ^{-1}A^{T}\lambda=-b&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then use &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; to obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&#xA;x=-Q^{-1}A^{T}\lambda&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On my desktop machine, for a random dense problem with &lt;span class=&quot;math-container&quot;&gt;$n=10000$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$m=50$&lt;/span&gt;, the setup computations take about 6 seconds, and then each individual solve runs in less than 0.1 seconds.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2018-12-01T20:10:16.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30627" PostTypeId="1" CreationDate="2018-12-02T08:38:52.387" Score="-1" ViewCount="187" Body="&lt;p&gt;I am running simulations of a cantilever beam where it is fixed on one end and negative force applied to the other end. The first simulation is with 4-node linear quadrilateral elements and the other is simulation with 9-node quadratic quadrilaterial elements. At first I ran the simulations with a poisson's ratio of 0.3, then with a Poisson's ratio of 0.499 and calculated the convergence rate.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got the convergence rate for the 4-node as 1.7 but it should be 2 and for the 9-node it is 3. My question is why they are different than each other is it because 4-node is linear and 9-node is quadratic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, what would be the reasons that cause the convergence rate to be 1.7 not 2? when I asked my professor he said your algorithm is right but you need to know why its not 2.&#xA; When I changed the Poisson's ratio to 0.499. everything messed up it the 4node simulation but it in it is fine for the 9node, why is that? why the 9-node can handle 0.499 material and 4-node can' &lt;/p&gt;&#xA;" OwnerUserId="29370" LastEditorUserId="9667" LastEditDate="2018-12-02T15:08:22.883" LastActivityDate="2018-12-03T01:33:53.460" Title="Finite element convergence rate and possion's ratio" Tags="|finite-element|convergence|poisson|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30628" PostTypeId="2" ParentId="30620" CreationDate="2018-12-02T17:07:31.793" Score="1" Body="&lt;p&gt;There is I think another reason why this result is surprising, apart from the fact that we're used to aiming for backward-stable algorithms like you said in your answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is the result of an approximate computation, that &lt;span class=&quot;math-container&quot;&gt;$y\approx1$&lt;/span&gt;, and we're trying to compute &lt;span class=&quot;math-container&quot;&gt;$y+x$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$x\approx0$&lt;/span&gt;. With this trivial change, the computation is now backward stable: any small absolute perturbation in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is equivalent to a small relative perturbation in &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, giving a small relative backward error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think the second reason why &lt;span class=&quot;math-container&quot;&gt;$1+x$&lt;/span&gt; not being backward stable is surprising is that it is legitimately surprising. Its backward stability depends on some fairly small details of how you define the problem. In particular, in a real numerical program, it might not be a priori obvious whether &lt;span class=&quot;math-container&quot;&gt;$1+x$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$y+x$&lt;/span&gt; is the right way to think about your program's behaviour. Different people might end up modelling program behaviour differently.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-12-02T17:07:31.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30631" PostTypeId="1" AcceptedAnswerId="30643" CreationDate="2018-12-02T23:35:23.673" Score="10" ViewCount="7589" Body="&lt;p&gt;I'm given a matrix. How do I find the nearest (or a near) positive definite from it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix can have complex eigenvalues, not be symmetric, etc. However, all its entries are real valued. The resulting matrix from the algorithm must be positive definite, with all its entries real valued only. Symmetry is a plus, but not necessary.&#xA;I have no preference toward the metric used. I prefer a pragmatic(relatively easy to programme)  approach. It doesn't have to be optimal.&lt;/p&gt;&#xA;" OwnerUserId="24970" LastEditorUserId="24970" LastEditDate="2018-12-03T09:55:02.483" LastActivityDate="2021-11-27T14:37:49.660" Title="How to find the nearest/a near positive definite from a given matrix?" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30633" PostTypeId="2" ParentId="30627" CreationDate="2018-12-03T01:33:53.460" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/esmq2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/esmq2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/b7KRo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b7KRo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hWf3y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hWf3y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here some plot of the deformation and how it converge to the exact solution as the size of mesh increase, the last plot shows the convergence rate of the solution. I am not sure why it is 1.7 but in theory it should be 2 for some reason of how the material behavior to the 4node could you help me out with this&lt;/p&gt;&#xA;" OwnerUserId="29370" LastActivityDate="2018-12-03T01:33:53.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30635" PostTypeId="1" AcceptedAnswerId="30636" CreationDate="2018-12-03T10:43:48.203" Score="0" ViewCount="1851" Body="&lt;p&gt;The original post was on stackoverflow : I transfert it here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to solve numerically the advection equation with periodic boundaries conditions : u(t,0) = u(t,L) with L the length of system to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start also &lt;code&gt;with u(0,x) = uexacte(0,x) = sin(2*pi*x/L)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the main part of the code with loop time (we use here &lt;code&gt;FTCS scheme&lt;/code&gt;) :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;V=1&#xA;L=1&#xA;# analytical solution --------------------------&#xA;def uexacte(t,x):&#xA;    return sin(2*pi*(x-V*t)/L)&#xA;&#xA;# 1. Centre FTCS (Forward Time Centered Space)&#xA;cfl = 0.25&#xA;nx = 10&#xA;tend = 1&#xA;&#xA;#&#xA;dx = L/(nx-1.)&#xA;dt = cfl*dx/V&#xA;nt = int(tend/dt)+1&#xA;print &quot;CFL=%5.2f tend=%4.1f --&amp;gt; %i iterations en temps&quot;%(cfl,tend,nt)&#xA;&#xA;# Arrays&#xA;x = linspace(0,L,nx)&#xA;# Bounadry condition&#xA;u0 = uexacte(0,x)&#xA;&#xA;# Starting solution&#xA;t=0.0 ; u=copy(u0)&#xA;&#xA;# Time loop&#xA;for i in range(1,nt):&#xA;    # FTCS&#xA;    #u[1:nx-1] = u[1:nx-1] - cfl/2*(u[2:nx] - u[0:nx-2])&#xA;    # Using roll&#xA;    u = u + - cfl/2*(roll(u,-1)- roll(u,1))&#xA;    # Update time&#xA;    t = t+dt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't understand the solution given by teacher who uses the python function &lt;code&gt;roll&lt;/code&gt; in this way :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Using roll&#xA;u = u - cfl/2*(roll(u,-1)- roll(u,1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One says that with the using of &lt;code&gt;roll&lt;/code&gt;, we are sure to respect the periodic boundary conditions but I don't understand why ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, my first approach was to do :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u[0] = u[nx-1]&#xA;u[1:nx-1] = u[1:nx-1] - cfl/2*(u[2:nx] - u[0:nx-2])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this doesn't work and I don't know how to implement this periodic conditions in this way (without using &lt;code&gt;roll&lt;/code&gt; function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If someone could explain this matter and the trick with &lt;code&gt;roll function&lt;/code&gt;, this would be nice to tell it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 1 :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried with classical approach (simple recurrence formula ) like this :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Time loop&#xA;for i in range(1,nt):&#xA;    # FTCS&#xA;    u[1:nx-1] = u[1:nx-1] - cfl/2*(u[2:nx] - u[0:nx-2])&#xA;&#xA;    # Try to impose periodic boundary conditions but without success&#xA;    u[0] = u[0] - cfl/2*(u[0] - u[nx-1])&#xA;&#xA;    # Update time&#xA;    t = t+dt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Indeed, the result is bad (values for each side are not the same). I could impose the theorical values at each step but in practise, we don't know always the analytical solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the trick to impose this periodic boundary condition on the numerical solution at each step of time ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with periodic boundary conditions is formulated as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GUkiX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GUkiX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 2 :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution given by LonelyProf below works fine. Thanks&lt;/p&gt;&#xA;" OwnerDisplayName="user14831" LastEditorDisplayName="user14831" LastEditDate="2018-12-11T12:18:50.347" LastActivityDate="2018-12-11T12:18:50.347" Title="Solving advection equation - periodic conditions - using roll python function" Tags="|finite-difference|python|boundary-conditions|" AnswerCount="1" CommentCount="3" ClosedDate="2018-12-11T16:10:55.353" ContentLicense="CC BY-SA 4.0" />
  <row Id="30636" PostTypeId="2" ParentId="30635" CreationDate="2018-12-03T11:56:09.307" Score="1" Body="&lt;p&gt;Similar issues arise in other languages, so this is not Python-specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The NumPy function &lt;code&gt;roll&lt;/code&gt; applies a cyclic shift to the array given as its first argument. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;roll(u,-1)&lt;/code&gt; returns an array with elements &lt;code&gt;u[1],u[2],u[3]...,u[nx-1],u[0]&lt;/code&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;roll(u,1)&lt;/code&gt; returns an array with elements &lt;code&gt;u[nx-1],u[0],u[1]...,u[nx-3],u[nx-2]&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is exactly what you want on the right hand side of your assignment, accessing the elements on either side of every element of &lt;code&gt;u&lt;/code&gt;, using periodic boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could do this by array slices, but then you would need to cope with the end effects by hand. So you can write a single statement to update all of &lt;code&gt;u[1:nx-2]&lt;/code&gt; in terms of &lt;code&gt;u[0:nx-3]&lt;/code&gt; and &lt;code&gt;u[2:nx-1]&lt;/code&gt;, and write separate statements to update &lt;code&gt;u[0]&lt;/code&gt; in terms of &lt;code&gt;u[nx-1]&lt;/code&gt; and &lt;code&gt;u[1]&lt;/code&gt;, and to update &lt;code&gt;u[nx-1]&lt;/code&gt; in terms of &lt;code&gt;u[nx-2]&lt;/code&gt; and &lt;code&gt;u[0]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, you are making mistakes in the way you try to do this by hand (using an illegal index &lt;code&gt;nx&lt;/code&gt;, for instance, or simply writing the wrong indices on the right). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue is that using a function such as &lt;code&gt;roll&lt;/code&gt; enables you to do the update effectively simultaneously for all the elements of &lt;code&gt;u&lt;/code&gt;. In some applications, perhaps yours, it may be an error (and is certainly different) to update any of the elements, and then use the updated value on the right-hand side to update other elements. You can avoid this by defining temporary variables to hold the updated values, if you are sticking with the &quot;by hand&quot; approach. Or define an array &lt;code&gt;du&lt;/code&gt; to hold the &lt;em&gt;change&lt;/em&gt; in &lt;code&gt;u&lt;/code&gt;, calculated from the current values, and actually perform the update &lt;code&gt;u=u+du&lt;/code&gt; in a separate statement, all at once. Using the &lt;code&gt;roll&lt;/code&gt; function (or similar) allows the code to be more compact and simpler.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[EDIT following OP comments]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of what I mean by using temporary variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;utemp1=u[0] - cfl/2*(u[nx-1] - u[1])&#xA;utemp2=u[nx-1] - cfl/2*(u[nx-2] - u[0])&#xA;u[1:nx-2] = u[1:nx-2] - cfl/2*(u[0:nx-3] - u[2:nx-1])&#xA;u[0] = utemp1&#xA;u[nx-1] = utemp2&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastEditorDisplayName="user28077" LastEditDate="2018-12-03T22:01:29.733" LastActivityDate="2018-12-03T22:01:29.733" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30637" PostTypeId="1" AcceptedAnswerId="32690" CreationDate="2018-12-03T19:59:06.680" Score="2" ViewCount="97" Body="&lt;p&gt;I am poking at reproducing some fundamental research in group theory.  In particular, I want to reproduce the &lt;a href=&quot;https://oeis.org/A000001&quot; rel=&quot;nofollow noreferrer&quot;&gt;OEIS sequence #1&lt;/a&gt;.  The crux of the problem is not generating potential groups, this can be accomplished by a sat-solver and about 20 lines of code, but determining which of those groups so generated are isomorphic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The finite group isomorphism algorithms are tractable, but computationally a bit hairy, so I am keen to determine what heuristics can be employed to save some compute on clearly not isomorphic groups.  A few obvious examples are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Groups with different order&lt;/li&gt;&#xA;&lt;li&gt;Groups with different Caylee skeleton (number of non-self identity elements)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In short, I would love some form or precomputation or fingerprint that I can compute to avoid comparing obviously different groups.&lt;/p&gt;&#xA;" OwnerUserId="53" LastEditorUserId="20688" LastEditDate="2020-01-26T12:17:06.440" LastActivityDate="2020-01-26T12:17:06.440" Title="Finite Group (Non-)Isomorphism" Tags="|abstract-algebra|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30640" PostTypeId="1" CreationDate="2018-12-04T02:12:03.853" Score="2" ViewCount="60" Body="&lt;p&gt;There are a lot of algorithms which seem to have been supplied an official number, such as Algorithm 76, Hierarchical clustering using the minimum spanning tree. Another example is Algorithm 123, Single Transferable Vote by Meek's Method. These numbers seem to refer to some official registry, but I haven't been able to find out what this registry is or how numbers are assigned. It does not seem to be related to the journal publishing the algorithm. Does anyone know how this system works and where one can read more about it?&lt;/p&gt;&#xA;" OwnerUserId="21459" LastActivityDate="2018-12-04T11:34:15.300" Title="Where does the seemingly official number of certain algorithms come from?" Tags="|algorithms|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30641" PostTypeId="1" AcceptedAnswerId="30646" CreationDate="2018-12-04T06:17:23.360" Score="4" ViewCount="713" Body="&lt;p&gt;The question is pretty much the title. Note that I have lots of experience using open-source meshing tool, e.g. Gmsh and OpenFoam blockMesh &amp;amp; snappyHexMesh. Nevertheless, I have no idea on how to generate the 2D mesh shown below when doing some simple test in SU2 (CFD code).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe some commercial meshing kits can do this job, but no experience using any commercial codes as I mainly work with open-source mesh/CFD codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea which mesh libraries (e.g. CGAL, OpenMesh) can make hybrid 2D tri/quad mesh?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 1:&lt;/strong&gt; The quad-mesh around the rectangle seems like beautiful O-Grid style. Any idea achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n24i4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n24i4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28844" LastEditorUserId="28844" LastEditDate="2018-12-05T02:43:18.993" LastActivityDate="2021-10-21T23:02:30.450" Title="Is there any open-source code for a hybrid 2D mesh (triangles and quadrilaterals)?" Tags="|mesh-generation|unstructured-mesh|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30643" PostTypeId="2" ParentId="30631" CreationDate="2018-12-04T15:32:50.363" Score="21" Body="&lt;p&gt;Quick sketch of an answer for the Frobenius norm:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Replace &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; with the closest symmetric matrix, &lt;span class=&quot;math-container&quot;&gt;$Y=\frac12 (X+X^\top)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Take an eigendecomposition &lt;span class=&quot;math-container&quot;&gt;$Y=QDQ^\top$&lt;/span&gt;, and form the diagonal matrix &lt;span class=&quot;math-container&quot;&gt;$D_+=\max(D,0)$&lt;/span&gt; (elementwise maximum).&lt;/li&gt;&#xA;&lt;li&gt;The closest symmetric positive &lt;em&gt;semi&lt;/em&gt;definite matrix to &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$Z=QD_+Q^\top$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The closest positive definite matrix to &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; does not exist; any matrix of the form &lt;span class=&quot;math-container&quot;&gt;$Z+\varepsilon I$&lt;/span&gt; is positive definite for &lt;span class=&quot;math-container&quot;&gt;$\varepsilon&amp;gt;0$&lt;/span&gt;. There is no minimum, just an infimum. Positive definite matrices are not a closed set.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To prove (1) and (3), you can use the fact that the decomposition of a matrix into a symmetric and antisymmetric part is orthogonal. In particular, this implies that we can minimize in two succesive steps like we did.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To prove (2), use the Wielandt-Hoffmann theorem.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2018-12-04T15:32:50.363" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30644" PostTypeId="1" CreationDate="2018-12-04T16:34:06.563" Score="0" ViewCount="197" Body="&lt;p&gt;I have an array of length L with N zeros, and L-N non-zero values. I have another array of length N. I would like to put the values of the shorter array into the positions of the longer array which are zero, keeping them in order. For example, if I had,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L = \, [1,2,3,0,0,4,5,6,0]&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;N = \, [10,0,20]&#xA;$$&lt;/span&gt;&#xA;I would want,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L' = \, [1,2,3,10,0,4,5,6,20]&#xA;$$&lt;/span&gt;&#xA;I will do this repeatedly with the same &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; and many different &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; (all of the same length). What is the best way to accomplish this using Numpy?&lt;/p&gt;&#xA;" OwnerUserId="29396" LastActivityDate="2018-12-04T17:16:03.677" Title="Numpy: How to permute array into indices of larger array?" Tags="|python|numpy|" AnswerCount="1" CommentCount="0" ClosedDate="2018-12-04T23:32:44.157" ContentLicense="CC BY-SA 4.0" />
  <row Id="30645" PostTypeId="2" ParentId="30644" CreationDate="2018-12-04T17:16:03.677" Score="1" Body="&lt;pre&gt;&lt;code&gt;L[L==0]=N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unless I’m missing something in your question. In numpy the main constraint is that you want to work with built-in array objects as much as possible. But taking a subset of an array, with indices given by a logical array, is a built-in operation so I expect this to be sufficient. This modifies L in-place, so you’ll need to create a copy for every new N.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-12-04T17:16:03.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30646" PostTypeId="2" ParentId="30641" CreationDate="2018-12-04T17:22:22.223" Score="7" Body="&lt;p&gt;You can use Gmsh for this purpose. I show an example below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Points&#xA;Point(1) = {-2, -2, 0, 1.0};&#xA;Point(2) = {2, -2, 0, 1.0};&#xA;Point(3) = {2, 2, 0, 1.0};&#xA;Point(4) = {-2, 2, 0, 1.0};&#xA;Point(5) = {-10, -10, 0, 2.0};&#xA;Point(6) = {10, -10, 0, 2.0};&#xA;Point(7) = {10, 10, 0, 2.0};&#xA;Point(8) = {-10, 10, 0, 2.0};&#xA;&#xA;// Lines&#xA;Line(1) = {1, 2};&#xA;Line(2) = {2, 3};&#xA;Line(3) = {3, 4};&#xA;Line(4) = {4, 1};&#xA;Line(5) = {5, 6};&#xA;Line(6) = {6, 7};&#xA;Line(7) = {7, 8};&#xA;Line(8) = {8, 5};&#xA;&#xA;// Surfaces&#xA;Line Loop(1) = {3, 4, 1, 2};&#xA;Plane Surface(1) = {1};&#xA;Line Loop(2) = {7, 8, 5, 6};&#xA;Plane Surface(2) = {-1, -2};&#xA;&#xA;// Meshing&#xA;Transfinite Line {3, 4, 1, 2} = 10 Using Progression 1;&#xA;Transfinite Surface {1};&#xA;Recombine Surface {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obtaining the following mesh.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VTMFV.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VTMFV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Points&#xA;Point(1) = {0, 0, 0, 1.0};&#xA;Point(2) = {-1, -1, 0, 1.0};&#xA;Point(3) = {1, -1, 0, 1.0};&#xA;Point(4) = {1, 1, 0, 1.0};&#xA;Point(5) = {-1, 1, 0, 1.0};&#xA;Point(6) = {-5*Sqrt(2)/2, -5*Sqrt(2)/2, 0, 1.0};&#xA;Point(7) = {5*Sqrt(2)/2, -5*Sqrt(2)/2, 0, 1.0};&#xA;Point(8) = {5*Sqrt(2)/2, 5*Sqrt(2)/2, 0, 1.0};&#xA;Point(9) = {-5*Sqrt(2)/2, 5*Sqrt(2)/2, 0, 1.0};&#xA;Point(10) = {-10, -10, 0, 2.0};&#xA;Point(11) = {10, -10, 0, 2.0};&#xA;Point(12) = {10, 10, 0, 2.0};&#xA;Point(13) = {-10, 10, 0, 2.0};&#xA;&#xA;// Lines&#xA;Line(1) = {2, 3};&#xA;Line(2) = {3, 4};&#xA;Line(3) = {4, 5};&#xA;Line(4) = {5, 2};&#xA;Line(5) = {6, 2};&#xA;Line(6) = {7, 3};&#xA;Line(7) = {8, 4};&#xA;Line(8) = {9, 5};&#xA;Line(9) = {10, 11};&#xA;Line(10) = {11, 12};&#xA;Line(11) = {12, 13};&#xA;Line(12) = {13, 10};&#xA;Circle(13) = {8, 1, 9};&#xA;Circle(14) = {9, 1, 6};&#xA;Circle(15) = {6, 1, 7};&#xA;Circle(16) = {7, 1, 8};&#xA;&#xA;// Surfaces&#xA;Line Loop(1) = {11, 12, 9, 10};&#xA;Line Loop(2) = {13, 14, 15, 16};&#xA;Plane Surface(1) = {1, 2};&#xA;Line Loop(3) = {7, 3, -8, -13};&#xA;Plane Surface(2) = {-3};&#xA;Line Loop(4) = {14, 5, -4, -8};&#xA;Plane Surface(3) = {4};&#xA;Line Loop(5) = {1, -6, -15, 5};&#xA;Plane Surface(4) = {-5};&#xA;Line Loop(6) = {6, 2, -7, -16};&#xA;Plane Surface(5) = {-6};&#xA;&#xA;// Meshing&#xA;Transfinite Line {8, 7, 6, 5} = 10 Using Progression 1;&#xA;Transfinite Line {14, 13, 16, 15, 1, 4, 3, 2} = 10 Using Progression 1;&#xA;Transfinite Surface {3};&#xA;Transfinite Surface {2};&#xA;Transfinite Surface {5};&#xA;Transfinite Surface {4};&#xA;Recombine Surface {2, 3, 4, 5};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gNPgA.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gNPgA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2018-12-04T19:51:36.810" LastActivityDate="2018-12-04T19:51:36.810" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30647" PostTypeId="1" AcceptedAnswerId="30656" CreationDate="2018-12-04T17:24:37.980" Score="2" ViewCount="204" Body="&lt;p&gt;Compared to the transient PDE for linear elasticity, the steady equations appear to less well-conditioned. Are they inherently ill-conditioned without the transient term? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The condition number for the operator matrix seems to be fairly large (at least &lt;span class=&quot;math-container&quot;&gt;$O(10^4)$&lt;/span&gt;) for all the problems that I have solved. Is this fairly normal for linear elasticity? I don't usually see condition numbers this large for some other computational science problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not tried a FEM discretization for linear elasticity; would FEM yield a better conditioned system than, say, FVM/FD? &lt;/p&gt;&#xA;" OwnerUserId="28865" LastActivityDate="2018-12-05T18:09:30.020" Title="Is steady linear elasticity inherently ill-conditioned?" Tags="|linear-solver|solid-mechanics|condition-number|linear-system|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30651" PostTypeId="1" CreationDate="2018-12-05T07:45:23.280" Score="5" ViewCount="1214" Body="&lt;p&gt;In an attempt to create three-dimensional volumes with inclusions in Gmsh I stumble upon a problem which was non-existent in the two-dimensional case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the OpenCASCADE geometry kernel because this facilitates convenient boolean operations to remove parts of the inclusions outside my considered domain box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a nutshell the problem boils down to: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;whenever I incorporate inclusions I perform a unite step, such that the different geometrical features are recognized but the interfaces&#xA;are joined in the mesh. This works like a charm in 2D, however, when&#xA;I do the equivalent in 3D, my volumetric geometrical information is&#xA;lost and volumes are actually united completely removing the interface.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I can graphically depict this by performing the 2D mesh step, in both 2D and 3D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o8oA4.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o8oA4.png&quot; alt=&quot;2D united mesh&quot;&gt;&lt;/a&gt;  &lt;a href=&quot;https://i.stack.imgur.com/YCmsK.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YCmsK.png&quot; alt=&quot;3D united mesh&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What the above showcases is the intended behaviour in 2D where (continuous) interfaces are retained, but although in 3D the sphere locations are still visible from the outside, the internal surface is no longer existent, i.e. the sphere outside is not meshed, only the cut section.&#xA;I should note that I believe I need the unite step to tie the different geometrical features together, that I've tried many different minor variations to produce 3D results like the one depicted above but none resulted in the intended mesh and that my web search was unhelpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope someone has a concrete idea or even a hunch on where to find the solution for the issue. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For completeness I'm adding the simplified .geo examples used to obtain the above graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 2D:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SetFactory(&quot;OpenCASCADE&quot;);&#xA;&#xA;Point(1) = {0, 0, 0};&#xA;Point(2) = {2, 0, 0};&#xA;Point(3) = {2, 2, 0};&#xA;Point(4) = {0, 2, 0};&#xA;&#xA;Line(1) = {1, 2};&#xA;Line(2) = {2, 3};&#xA;Line(3) = {3, 4};&#xA;Line(4) = {4, 1};&#xA;&#xA;Line Loop(1) = {1, 2, 3, 4};&#xA;&#xA;Plane Surface(1) = {1};&#xA;&#xA;Periodic Line{1} = {-3};&#xA;Periodic Line{2} = {-4};&#xA;&#xA;Disk(2) = {0, 0, 0, 0.5};&#xA;Disk(3) = {2, 0, 0, 0.5};&#xA;Disk(4) = {0, 2, 0, 0.5};&#xA;Disk(5) = {2, 2, 0, 0.5};&#xA;Disk(6) = {1, 1, 0, 0.5};&#xA;&#xA;f1() = BooleanDifference{ Surface{1}; }{ Surface{2:6}; };&#xA;f2() = BooleanIntersection{ Surface{1}; Delete; }{ Surface{2:6}; Delete; };&#xA;BooleanUnion{ Surface{ f1() }; Delete; }{ Surface{ f2() }; Delete; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For 3D:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SetFactory(&quot;OpenCASCADE&quot;);&#xA;&#xA;incRadi = 0.5;&#xA;boxSize = 1.5;&#xA;&#xA;Box(1) = {0,0,0, boxSize,boxSize,boxSize};&#xA;&#xA;Periodic Surface{2} = {1} Translate{boxSize,0,0};&#xA;Periodic Surface{4} = {3} Translate{0,boxSize,0};&#xA;Periodic Surface{6} = {5} Translate{0,0,boxSize};&#xA;&#xA;Sphere(2) = {0, 0.5*boxSize, 0.5*boxSize, incRadi, -Pi/2, Pi/2, 2*Pi};&#xA;Sphere(3) = {boxSize, 0.5*boxSize, 0.5*boxSize, incRadi, -Pi/2, Pi/2, 2*Pi};&#xA;&#xA;f1() = BooleanDifference{ Volume{1}; }{ Volume{2:3}; };&#xA;f2() = BooleanIntersection{ Volume{1}; Delete; }{ Volume{2:3}; Delete; };&#xA;BooleanUnion{ Volume{ f1() }; Delete; }{ Volume{ f2() }; Delete; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17418" LastEditorUserId="17418" LastEditDate="2018-12-05T07:53:45.423" LastActivityDate="2018-12-05T07:53:45.423" Title="Gmsh for 3D volume with inclusions" Tags="|finite-element|mesh-generation|mesh|gmsh|" AnswerCount="0" CommentCount="0" ClosedDate="2018-12-17T01:15:04.870" ContentLicense="CC BY-SA 4.0" />
  <row Id="30653" PostTypeId="1" CreationDate="2018-12-05T13:08:48.433" Score="3" ViewCount="166" Body="&lt;p&gt;I am trying to understand the meshless methods as described in &lt;a href=&quot;https://arxiv.org/pdf/1409.7395.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1409.7395.pdf&lt;/a&gt;. I'm having trouble understanding the following step: (Page 7, just after equation 17)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Now, rather than take the flux functions F directly at the particle location and time of i or j, in which case the scheme would&#xA;  require some ad-hoc artificial dissipation terms (viscosity and con-&#xA;  ductivity) to be stable, we can replace the fluxes with the solution&#xA;  of an appropriate time-centered Riemann problem between the particles/cells i and j,  which  automatically  includes  the  dissipation terms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What I can't find informations about is: &lt;strong&gt;What is a time-centered Riemann problem?&lt;/strong&gt; I wasn't able to find any details on this so far, not even in Toro's &quot;Riemann Solvers and Numerical Methods for Fluid Dynamics&quot;. From what I know, a Riemann problem is a special initial value problem for a conservation law&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathbf{U}_t + \mathbf{A}(\mathbf{U}) \mathbf{U}_x = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \mathbf{U}(x, 0) = \mathbf{U}_0(x) = \cases{ \mathbf{U}_L \text{ if } x&amp;lt;0\\ \mathbf{U}_R \text{ if } x&amp;gt;0} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how does time-centering come into play here?&lt;/p&gt;&#xA;" OwnerUserId="26461" LastActivityDate="2020-12-24T21:02:35.170" Title="What's a time centered Riemann problem?" Tags="|fluid-dynamics|meshless-methods|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30654" PostTypeId="2" ParentId="30653" CreationDate="2018-12-05T14:17:13.527" Score="1" Body="&lt;p&gt;Some schemes have a two step update. First update solution &lt;span class=&quot;math-container&quot;&gt;$u^n$&lt;/span&gt;  to middle of time interval &lt;span class=&quot;math-container&quot;&gt;$[t_n,t_{n+1}]$&lt;/span&gt; to get &lt;span class=&quot;math-container&quot;&gt;$u^{n+1/2}$&lt;/span&gt;. Then use this predicted solution &lt;span class=&quot;math-container&quot;&gt;$u^{n+1/2}$&lt;/span&gt; to do the final update, where you make use of Riemann solvers. This is what they probably mean by time centered Riemann solver. In particular, search for MUSCL-Hancock scheme for more details on this approach.&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2018-12-05T14:17:13.527" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30656" PostTypeId="2" ParentId="30647" CreationDate="2018-12-05T18:09:30.020" Score="6" Body="&lt;p&gt;The condition number for the stiffness matrix of any method (finite elements, finite volumes, finite differences) applied to a second order differential operator always grows as &lt;span class=&quot;math-container&quot;&gt;${\cal O}(h^{-2})$&lt;/span&gt; where the exponent equals the order of the differential operator. As a consequence, if you just make the mesh fine enough, you can make the condition number arbitrarily large -- and in fact, &lt;span class=&quot;math-container&quot;&gt;$10^4$&lt;/span&gt; is a pretty small value that probably results from the fact that you have a relatively small number of cells or mesh points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why you don't observe this issue is because for time dependent (dynamic) problems, the matrix that needs to be inverted is either of the form &lt;span class=&quot;math-container&quot;&gt;$M+\Delta t\, A$&lt;/span&gt; (parabolic problems with one time derivative) or &lt;span class=&quot;math-container&quot;&gt;$M+\Delta t^2\, A$&lt;/span&gt; (hyperbolic problems with two time derivatives). The mass matrix has condition number &lt;span class=&quot;math-container&quot;&gt;${\cal O}(1)$&lt;/span&gt; and so the total matrix you need to invert has condition number of either &lt;span class=&quot;math-container&quot;&gt;${\cal O}(1+\Delta t\,h^{-2})$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;${\cal O}(1+\Delta t^2\,h^{-2})$&lt;/span&gt; in the two cases above. In other words, the multiplication of the bad matrix by a small number &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; makes the problem better conditioned.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2018-12-05T18:09:30.020" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30657" PostTypeId="1" CreationDate="2018-12-05T23:13:05.113" Score="-1" ViewCount="2940" Body="&lt;p&gt;&lt;strong&gt;Nonlinear ODE Statement&lt;/strong&gt;&#xA;I would like to use scipy to solve the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u'' + (u')^2 = sin(x)&#xA;u(0)=0,&#xA;u(1)=1    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where u = u(x). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach&lt;/strong&gt;&#xA;I am looking at the following link&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.integrate.solve_bvp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.integrate.solve_bvp.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem - Not sure about variable wrt x or How to input BC's&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example given in the link does not include a term wrt x, in particular not sure how to include &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sin(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;from the right hand side of my problem, nor the &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u(1)=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Work Done&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following I try to modify the example from the above link to conform to my ODE above&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   from scipy.integrate import *&#xA;import numpy as np&#xA;&#xA;from numpy import *&#xA;def fun(x, y):&#xA;    return np.vstack((y[1],np.sin(x) -(y[0])**2))&#xA;def bc(ya, yb):&#xA;    return np.array([ya[0], yb[0]-1])&#xA;x = np.linspace(0, 1, 5)&#xA;y_a = np.zeros((2, x.size))&#xA;y_b = np.zeros((2, x.size))&#xA;y_b[0] = 3&#xA;&#xA;from scipy.integrate import solve_bvp&#xA;res_a = solve_bvp(fun, bc, x, y_a)&#xA;res_b = solve_bvp(fun, bc, x, y_b)&#xA;x_plot = np.linspace(0, 1, 100)&#xA;y_plot_a = res_a.sol(x_plot)[0]&#xA;y_plot_b = res_b.sol(x_plot)[0]&#xA;import matplotlib.pyplot as plt&#xA;plt.plot(x_plot, y_plot_a, label='y_a')&#xA;plt.plot(x_plot, y_plot_b, label='y_b')&#xA;plt.legend()&#xA;plt.xlabel(&quot;x&quot;)&#xA;plt.ylabel(&quot;y&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="27900" LastEditorUserId="27900" LastEditDate="2018-12-06T00:20:24.130" LastActivityDate="2019-01-05T16:01:25.677" Title="Scipy Two-point Boundary value Problem" Tags="|ode|scipy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30659" PostTypeId="2" ParentId="30657" CreationDate="2018-12-06T15:49:51.827" Score="1" Body="&lt;p&gt;I think in NumPy, if &lt;code&gt;y&lt;/code&gt; is a matrix, &lt;code&gt;y[0]&lt;/code&gt; gets you the 0-th row, but it usually makes the code clearer if you use &lt;code&gt;y[0,:]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary problem I see is that in &lt;code&gt;fun&lt;/code&gt; you're defining the ODE&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ u''+u^2=\sin x, $$&lt;/span&gt;&#xA;using &lt;code&gt;y[0,:]&lt;/code&gt; instead of &lt;code&gt;y[1,:]&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-12-06T15:49:51.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30663" PostTypeId="1" CreationDate="2018-12-07T05:19:22.253" Score="1" ViewCount="443" Body="&lt;p&gt;I'm having two sets of grids. One is uniform and another one is not uniform. I will calculate the derivative in uniform grid points and I like to transfer(map)  the derivative to the non-uniform grid points. I have the relationship between uniform and non-uniform grid points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/24780/derivatives-approximation-on-non-uniform-grid&quot;&gt;A similar question was already asked.&lt;/a&gt; I'd like to get one answer with an example. For the time sake, I consider the following simple example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relation between my uniform and the non-uniform grid is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$x= \zeta^2$$&lt;/span&gt; Let &lt;span class=&quot;math-container&quot;&gt;$y(x)$&lt;/span&gt; be the function for which I'd like to calculate the derivative on the non-uniform grid and &lt;span class=&quot;math-container&quot;&gt;$y = x^2$&lt;/span&gt;. So, &lt;span class=&quot;math-container&quot;&gt;$\frac{dy}{dx} =2x$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The uniform grid and derivative information are given, as follows: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B9MPN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B9MPN.png&quot; alt=&quot;uniform grid and derivative&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The non-uniform grid: &lt;a href=&quot;https://i.stack.imgur.com/ti0zC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ti0zC.png&quot; alt=&quot;figure&quot;&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find the derivative information in the non-uniform grid by mapping.&lt;/p&gt;&#xA;" OwnerUserId="16930" LastEditorUserId="20688" LastEditDate="2019-06-22T19:19:21.777" LastActivityDate="2019-06-22T19:19:21.777" Title="Mapping derivative information in uniform to non-uniform grid" Tags="|finite-difference|grid|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30664" PostTypeId="1" CreationDate="2018-12-07T17:26:47.293" Score="0" ViewCount="120" Body="&lt;p&gt;I am working on an FEA mesh of hexahedron elements. The elemental level calculations involve finding the surface normals and area for each surface of a hex element. I preferred the vector cross product approach. But that doesn't seem to work when the faces are non-planar.  Will the isoparametric transformation of each surface be a better option? Since I am already in a finite element setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pzNyF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pzNyF.png&quot; alt=&quot;Irregular Hex&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="22448" LastActivityDate="2018-12-07T20:44:29.623" Title="Given co-ordinates of 8 vertices, how to calculate the outward normal and surface area for each face of a irregular hexahedron?" Tags="|finite-element|numerical-modelling|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30665" PostTypeId="2" ParentId="30664" CreationDate="2018-12-07T20:44:29.623" Score="3" Body="&lt;p&gt;If your vertices aren't coplanar, then your bounding surfaces can't be planes, and your volume is actually &quot;worse&quot; than an irregular hexahedron. Indeed, it isn't actually a polyhedron at all. At this point, having four vertices to a face stops being an over-specification of the surface normals, and starts being too little, in that there are an infinite number of curved surfaces passing through the four points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where you need to take a step backwards and look at what caused you to ask the question in the first place. Since you're doing finite element analysis, it's very likely that the isoparametric trilinear basis through each vertex is actually the &lt;em&gt;definition&lt;/em&gt; of the shape you're working with. As such, it's probable that the normals and surface areas you want are the ones arising for that object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a bonus, that probably means you can calculate the surface normals (in physical space) at locations given by parametric coordinates (the quadrature points for your method) rather than by physical coordinates. That would save you an inversion for every point on the surface where you need the normal (note again that since the surface isn't planar, its normal isn't constant).&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2018-12-07T20:44:29.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30666" PostTypeId="2" ParentId="30663" CreationDate="2018-12-08T01:18:53.850" Score="2" Body="&lt;p&gt;Use the chain rule to get the derivative on the non-uniform grid, &lt;span class=&quot;math-container&quot;&gt;$\frac{dy}{d\zeta}$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;\begin{align}&#xA;\frac{dy}{d\zeta} &amp;amp;= \frac{dy}{dx} \frac{dx}{d\zeta}\\&#xA;&amp;amp;= \frac{dy}{dx} \cdot 2 \zeta \\&#xA;&amp;amp;= \frac{dy}{dx} \cdot 2 \sqrt{x} \\&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you have a way of computing &lt;span class=&quot;math-container&quot;&gt;$\frac{dy}{dx}$&lt;/span&gt; on the uniform grid, you can simply scale it by &lt;span class=&quot;math-container&quot;&gt;$2\sqrt{x}$&lt;/span&gt; to get the corresponding &lt;span class=&quot;math-container&quot;&gt;$\frac{dy}{d\zeta}$&lt;/span&gt; derivative.&lt;/p&gt;&#xA;" OwnerUserId="29276" LastActivityDate="2018-12-08T01:18:53.850" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30668" PostTypeId="2" ParentId="29710" CreationDate="2018-12-08T11:35:58.257" Score="3" Body="&lt;p&gt;You are trying to solve an eigenvalue problem by directly finding zeros of the determinant. This has many numerical difficulties, e.g. overflow and small convergence basin. This is the same for standard eigenvalue problems. You need techniques from numerical linear algebra. Your problem is referred to as a nonlinear eigenvalue problem which is an active field of research within numerical linear algebra. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem should be reasonably easy to solve with the Julia package &lt;a href=&quot;https://nep-pack.github.io/NonlinearEigenproblems.jl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEP-PACK&lt;/a&gt;. (I am a developer of NEP-PACK.) Your code is a bit involved so here is a simplified problem which should be easy generalize:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar to your &lt;a href=&quot;https://math.stackexchange.com/questions/2816478/nonlinear-eigenvalue-problem-matlab-code/3030933#3030933&quot;&gt;post in mathexchange&lt;/a&gt;, you can solve &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ M(\lambda)q=0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ M(\lambda)=A_0+A_1\lambda+A_2\sqrt{c+\lambda^2}+A_3\sqrt{d+\lambda^2} $$&lt;/span&gt;&#xA;as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using NonlinearEigenproblems&#xA;A0=randn(10,10);&#xA;A1=randn(10,10);&#xA;A2=randn(10,10);&#xA;A3=randn(10,10); &#xA;c=4; d=6;&#xA;f0= S-&amp;gt;one(S)&#xA;f1= S-&amp;gt;S&#xA;f2= S-&amp;gt;sqrt(c*one(S)+S^2)&#xA;f3= S-&amp;gt;sqrt(d*one(S)+S^2)&#xA;nep=SPMF_NEP([A0,A1,A2,A3],[f0,f1,f2,f3]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then solve it with any of the available methods, e.g., quasinewton&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(omega,q)=quasinewton(nep,λ=-3.0);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it will give you a solution if the starting guess is sufficiently close to a solution since&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;norm((A0+A1*f1(omega)+A2*f2(omega)+A3*f3(omega))*q)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will be small. If you use this solution, please cite &lt;a href=&quot;https://arxiv.org/abs/1811.09592&quot; rel=&quot;nofollow noreferrer&quot;&gt;the package&lt;/a&gt;. If it leads to a publication, it would be nice to post a reference to the paper where it is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See a similar problem/solution in &lt;a href=&quot;https://nep-pack.github.io/NonlinearEigenproblems.jl/movebc_tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29434" LastEditorUserId="29434" LastEditDate="2018-12-08T11:44:05.500" LastActivityDate="2018-12-08T11:44:05.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30670" PostTypeId="2" ParentId="11515" CreationDate="2018-12-08T11:58:39.590" Score="2" Body="&lt;p&gt;The problem of finding singular points of a parameter-dependent matrix is called the nonlinear eigenvalue problem in the field of numerical linear algebra. I think you could benefit from using developed methods. It is a quite active area of research. You can consider solving it with the Julia software package &lt;a href=&quot;https://nep-pack.github.io/NonlinearEigenproblems.jl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NEP-PACK&lt;/a&gt; (I am a lead developer of NEP-PACK). NEP-PACK contains an implementation of the method of Beyn cited in the previous post. There are some methods also in &lt;a href=&quot;http://slepc.upv.es/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SLEPc&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29434" LastActivityDate="2018-12-08T11:58:39.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30672" PostTypeId="2" ParentId="30445" CreationDate="2018-12-08T12:55:49.987" Score="3" Body="&lt;p&gt;I would call your problem a eigenvector nonlinear two-parameter eigenvalue problem. It is essentially a combination of two difficult problems: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Eigenvector nonlinear eigenproblem &#xA;&lt;span class=&quot;math-container&quot;&gt;$$A(v)v=\lambda v.$$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Two-parameter eigenvalue problem &#xA;&lt;span class=&quot;math-container&quot;&gt;$$(A_1+A_2\lambda +A_3\mu)u=0$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$(B_1+B_2\lambda +B_3\mu)v=0$$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These problems have received separate attention in numerical linear algebra, e.g., 2 is studied in a &lt;a href=&quot;https://arxiv.org/abs/1802.07386&quot; rel=&quot;nofollow noreferrer&quot;&gt;this recent paper&lt;/a&gt;. Problem 2 can be reduced to a standard eigenvalue problem by using Kronecker products, which squares the dimension of the problem, but works for small problems. There are not so many methods for 1. We proposed an algorithm in &lt;a href=&quot;https://arxiv.org/abs/1212.0417&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; which may be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge the combination of 1 and 2 as in your case has not been studied. So an approach based on just applying Newtons method for nonlinear systems is probably a reasonable starting point.&lt;/p&gt;&#xA;" OwnerUserId="29434" LastActivityDate="2018-12-08T12:55:49.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30673" PostTypeId="1" CreationDate="2018-12-08T18:38:57.687" Score="2" ViewCount="99" Body="&lt;p&gt;I am trying to identify the weak form of PDE in structural mechanics. I read a lot of papers where they are using the elliptic boundary value problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA; \int\limits_{\Omega} \delta \epsilon : \sigma \,d\Omega - \Big[\int\limits_{\Omega} \delta u \cdot b \, d\Omega + \int\limits_{\Gamma_N} \delta u \cdot t \, d\Gamma\Big] = 0 &#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; represent the body forces, &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is the traction vector acting on the Neumann boundary, stress &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; the displacement. With &lt;span class=&quot;math-container&quot;&gt;$\delta \epsilon$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\delta u$&lt;/span&gt; they define virtual strain/displacement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be honest, I am not used to mechanics. For example I am using the general elliptic PDE: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}\left\{\begin{aligned}&#xA;div(A\nabla u) = &amp;amp; \,f &amp;amp;&amp;amp; \text{ in } \Omega   \\&#xA;u  = &amp;amp; \,g_D &amp;amp;&amp;amp; \text{ on }\Gamma_D \\&#xA;A \nabla_n u = &amp;amp;\, g_N &amp;amp;&amp;amp; \text{ on }\Gamma_N \\&#xA;  \end{aligned}\right.&#xA;\end{align}&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; a bounded self-adjoint and elliptic linear operator. The weak form this PDE reads: find &lt;span class=&quot;math-container&quot;&gt;$u\in H^1_{\Gamma_D}(\Omega)$&lt;/span&gt; for given functions &lt;span class=&quot;math-container&quot;&gt;$f\in L^2(\Omega)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g_N \in L^2(\Gamma_N)$&lt;/span&gt; such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\int\limits_{\Omega} \nabla u A \nabla v  \, d\Omega = \int\limits_{\Omega} v^T f \, d\Omega + \int\limits_{\Gamma_N} v^T g_N \,d\Gamma \qquad \forall v \in H^1_{\Gamma_D}(\Omega).&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal now is to identify the weak form in structural mechanics such that my general weak form will be the same. As I can see, first I need to define &lt;span class=&quot;math-container&quot;&gt;$v^T = \delta u$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$t = g_N$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b = f$&lt;/span&gt;. But I am not sure about the other terms. Is it correct that &lt;span class=&quot;math-container&quot;&gt;$\delta \epsilon = \nabla v = \nabla \delta u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\nabla u = \sigma$&lt;/span&gt;? I cannot see the connection between these definitions. And how do I use the elliptic operator &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; in this context? Can someone bring light into the darkness? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My new Idea is: Use that strain is equal to the derivatives of the displacement such that &lt;span class=&quot;math-container&quot;&gt;$\epsilon = \nabla u$&lt;/span&gt;. Next is to define the virtual strain as the derivative of the virtual displacement such that &lt;span class=&quot;math-container&quot;&gt;$\nabla v = \nabla \delta u = \delta \epsilon$&lt;/span&gt;. By Hookes law it is now: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\nabla v : A \nabla u = \nabla \delta u : A \nabla u = \delta \epsilon : A \epsilon = \delta \epsilon : \sigma&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user29088" LastEditorDisplayName="user29088" LastEditDate="2018-12-08T22:37:43.453" LastActivityDate="2018-12-08T22:37:43.453" Title="Identify the components of the (weak form) PDE in structural mechanics" Tags="|pde|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="30676" PostTypeId="1" CreationDate="2018-12-09T08:55:22.607" Score="0" ViewCount="225" Body="&lt;p&gt;I have a set of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; keywords. Each of this keywords can have set of bids from &lt;span class=&quot;math-container&quot;&gt;$1\$,\dots,N\$$&lt;/span&gt;. For each bid for a keyword, it will get a specific amount of clicks and a specific cost. Clicks and Cost are monotonic in bids. Now, the problem is this &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Select a bid for each of the keyword such that overall clicks across the keywords is maximized such that the cost is still within a given budget &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$\text{Clicks}(W_i)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{Cost}(W_i)$&lt;/span&gt; denote the clicks and cost on the keyword &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. Then (informally) the problem is to select a value of bid for each keyword such that &lt;span class=&quot;math-container&quot;&gt;\begin{align}\max\sum_{i=1}^{K}&amp;amp;\text{Clicks}(W_i) \\s.t.&amp;amp;~~\sum_{i=1}^{K}\text{Cost}(W_i)\leq B\end{align}&lt;/span&gt; Is this a type of knapsack problem? Are there known heuristic in literature.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Following is not important for the question (and thus can be skipped), but is for the curious ones to know how I am solving this currently.&#xA;Let &lt;span class=&quot;math-container&quot;&gt;$S_{ij}$&lt;/span&gt; denote the cost for using bid &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; for keyword &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and let &lt;span class=&quot;math-container&quot;&gt;$C_{ij}$&lt;/span&gt; be the clicks for the same. Let &lt;span class=&quot;math-container&quot;&gt;$x_{ij}$&lt;/span&gt; be the indicator variable for using bid &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; for keyword &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. Then among variables &lt;span class=&quot;math-container&quot;&gt;$x_{i1},\dots,x_{iN}$&lt;/span&gt; only one can be non-zero (i.e. &lt;span class=&quot;math-container&quot;&gt;$x_{ij}=1$&lt;/span&gt;) whereas the rest is zero (since only one bid can chosen for keyword). Thus the whole problem can be formulated mathematically as &lt;span class=&quot;math-container&quot;&gt;\begin{align}\max_{x_{ij}}~&amp;amp;\sum_{i=1}^{K}\sum_{j=1}^{N}x_{ij}C_{ij} \\s.t.~~\sum_{i=1}^{K}\sum_{j=1}^{N}&amp;amp;x_{ij}S_{ij}\leq B\\\sum_{j=1}^{N}x_{ij}=1&amp;amp;~~\forall i\in \{1,\dots,K\}~~,~~x_{ij}\in\{0,1\}\end{align}&lt;/span&gt;Last two constraints ensures they are indicator variables and only one bid is chosen for a given keyword. Solving this should solve the original problem. &lt;/p&gt;&#xA;" OwnerUserId="3255" LastActivityDate="2020-02-08T05:01:20.177" Title="Is this a knapsack problem?" Tags="|optimization|constrained-optimization|heuristics|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30679" PostTypeId="2" ParentId="30524" CreationDate="2018-12-09T18:28:25.647" Score="1" Body="&lt;p&gt;The main problem why the AMG preconditioner does not show much improvements in convergence is the structure of the system matrix. The scalar AMG approach is not suitable for the used matrices. Another approach might show better results.&lt;/p&gt;&#xA;" OwnerUserId="28358" LastActivityDate="2018-12-09T18:28:25.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30681" PostTypeId="1" CreationDate="2018-12-10T05:25:25.130" Score="2" ViewCount="1639" Body="&lt;p&gt;One can use GMRES as it is, but there is also a version of GMRES called k-step restarted GMRES, which is used for large matrices, where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is some fixed number of steps after which we take a new &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; and restart the algorithm to save memory storage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to count the number of flops and memory storage required in either case. Concerning the flops, as far as I understand, we have the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; steps in the main loop, &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; steps for Arnoldi iteration, and &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; steps for matrix multiplication in the least squares solution. So the total comes to &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt;. Is this true?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the restarted algorithm, we have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$O(k^3)$&lt;/span&gt; steps until convergence for each restart. Is this true?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now, I don't really know how to calculate memory storage. Could someone please help me with this part?&lt;/p&gt;&#xA;" OwnerUserId="29427" LastActivityDate="2018-12-11T18:37:02.093" Title="Operation count for GMRES" Tags="|linear-algebra|iterative-method|numerics|gmres|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30683" PostTypeId="1" CreationDate="2018-12-10T10:16:27.540" Score="5" ViewCount="88" Body="&lt;p&gt;I am currently working on a numerical algorithm involving a lot of floating point arithmetic, involving some badly conditioned problem sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the relation &lt;span class=&quot;math-container&quot;&gt;$|x - y| / (\max(|x|, |y|, 1)) \leq \epsilon$&lt;/span&gt;. Unfortunately, the corresponding relation &lt;span class=&quot;math-container&quot;&gt;$=\sim$&lt;/span&gt; is not invariant with respect to additions, i.e. &lt;span class=&quot;math-container&quot;&gt;$x =\sim y$&lt;/span&gt; does not imply &lt;span class=&quot;math-container&quot;&gt;$x - y =\sim 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a counter example I have &lt;span class=&quot;math-container&quot;&gt;$x = 1e+20$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y = 1.0000000000000082e+20$&lt;/span&gt;, and therefore &lt;span class=&quot;math-container&quot;&gt;$x - y = -819200$&lt;/span&gt; (wrt. floating point arithmetic). I think that this is a pretty typical case of cancellation...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a suitable relation?&lt;/li&gt;&#xA;&lt;li&gt;How should I compare &lt;span class=&quot;math-container&quot;&gt;$x + y =\sim z$&lt;/span&gt;? I am using the statement &#xA;in an assertion which consequently fails on certain instances.&#xA;Is it advisable to use (&lt;span class=&quot;math-container&quot;&gt;$x + y =\sim z$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$x =\sim z - y$&lt;/span&gt;)&#xA;instead?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I trying to achieve: I am implementing a solver for nonlinear programming problems, and running the implementation on the cutest suite of test problems. In the specific step, I am asserting that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ c(x) + J(x)d = \sim 0, $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$c(x)$&lt;/span&gt; is a vector of constraint values, and &lt;span class=&quot;math-container&quot;&gt;$J(x)$&lt;/span&gt; is the constraint Jacobian. The problems in the test set vary widely in terms of size and type. Therefore, I'm afraid that there is not much specific knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is more, the test suite is known to be badly scaled, i.e. containing constraints of the type &lt;span class=&quot;math-container&quot;&gt;$1000*x^2 + 0.001 * y^2$&lt;/span&gt;. Therefore, the Jacobian is likely ill-conditioned.&lt;/p&gt;&#xA;" OwnerUserId="29447" LastEditorUserId="29447" LastEditDate="2018-12-10T12:55:40.140" LastActivityDate="2018-12-10T12:55:40.140" Title="Comparing sum of floating points" Tags="|floating-point|numerics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30684" PostTypeId="1" CreationDate="2018-12-10T11:33:16.333" Score="1" ViewCount="305" Body="&lt;p&gt;I want to obtain the &lt;em&gt;algebraic&lt;/em&gt; values of, for example, Hermite polynomials using SciPy but in a recursive manner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Maple, for example, these can be defined as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;H(0,x):=1;&#xA;H(1,x):=2*x;&#xA;for n from 1 to 6 do&#xA;    H(n_1,x):= 2*x*H(n,x) - 2*n*H(n-1,x);&#xA;    print( simplify(%) );&#xA;end do:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can find no way in SciyPy to do this algebraically. Please can you help?&lt;/p&gt;&#xA;" OwnerUserId="21287" LastEditorUserId="21287" LastEditDate="2018-12-10T14:07:34.697" LastActivityDate="2018-12-10T14:07:34.697" Title="Algebraic recursion of Hermite polynomials in SymPy" Tags="|python|" AnswerCount="0" CommentCount="5" ClosedDate="2018-12-11T00:05:51.810" ContentLicense="CC BY-SA 4.0" />
  <row Id="30686" PostTypeId="1" CreationDate="2018-12-10T19:01:03.623" Score="0" ViewCount="116" Body="&lt;p&gt;I have an explicit FD (Finite Difference) code for diffusion/heat on a PDE in a cuboid domain, and it works fine. I would like to update the discretized equations and change the code so as to solve the problem on a sphere. From some preliminary searches, I can't seem to find any examples of FD on spheres using rectangular coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen some in spherical coordinates, but that may require too many changes to the code (if this is the only way, please advice on how best to proceed)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial U}{\partial t} = \alpha \nabla^2U\quad in\enspace\mathbb{R}^3$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$U(x,y,z,0)=T$$&lt;/span&gt;&#xA;where T is a constant, and is also the boundary condition applied on all faces of the cube / the surface area of the sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: Is it possible to adapt my code for a cube, to work on a sphere, and remain in rectangular coordinates? If so, how do I approach this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: (more context)&lt;/strong&gt; This is a homogenous heat equation, where the boundary condition is a constant temperature (say T=100 degrees) applied on the entire surface area of the sphere (for the cube it was also applied on all 6 faces). The initial condition is the same heat source on the surface, while the entire insides of the sphere was initially at zero.&lt;/p&gt;&#xA;" OwnerUserId="20775" LastEditorUserId="20775" LastEditDate="2018-12-10T20:34:34.323" LastActivityDate="2018-12-10T20:34:34.323" Title="Changing the domain of a 3D Finite Difference code from cube to sphere" Tags="|pde|finite-difference|discretization|numerical-modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30687" PostTypeId="2" ParentId="30686" CreationDate="2018-12-10T20:23:31.927" Score="1" Body="&lt;p&gt;That is certainly possible. You can reformulate your problem to use &lt;span class=&quot;math-container&quot;&gt;$\Delta + V$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; as an operator, where &lt;span class=&quot;math-container&quot;&gt;$V=V(x,y,z)=V(\vec{r})$&lt;/span&gt; is a potential defined in a way suitable to tackle your problem. For instance, if I were looking for eigenmodes of a disc or a sphere by solving the Helmholtz equation using finite differences I'd define the potential as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$V(\vec{r}) = \begin{cases}0 &amp;amp; \vec{r}&amp;lt; r_0\\ V_0 &amp;amp; else\end{cases}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$V_0\rightarrow\infty$&lt;/span&gt;  being essentially a (very) large number. This should, in the case of a Helmholtz problem, force the resulting solution to be ~0 outside of the sphere with radius &lt;span class=&quot;math-container&quot;&gt;$r_0$&lt;/span&gt;. For a heat equation you can do something similar (depending on the boundary condition).&lt;/p&gt;&#xA;" OwnerUserId="28993" LastActivityDate="2018-12-10T20:23:31.927" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30688" PostTypeId="1" CreationDate="2018-12-11T13:07:51.233" Score="2" ViewCount="2098" Body="&lt;p&gt;I have 5 at most 4th order polynomials in 5 variables,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p_i(x_1,x_2,x_3,x_4,x_5) \qquad i = 1, \ldots, 5$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where all coefficients are either rational or floating point. I'd would like to get the roots of the simultaneous equations &lt;span class=&quot;math-container&quot;&gt;$p_1 = 0, p_2 = 0, p_3 = 0, p_4 = 0, p_5 = 0$&lt;/span&gt; and I'm partially able to do this quite easily with non-linear solvers using python scipy. Of course these non-linear solvers don't give me all roots just a couple of them if I choose the starting points carefully. In any case there is no guarantee that I'm getting all of them. But since we are talking about polynomials there are known algorithms for reducing them to polynomials of a single variable and the full set of roots of those can be found easily (see &lt;a href=&quot;https://en.wikipedia.org/wiki/System_of_polynomial_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wiki&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm wondering is if there was any of this implemented in Python/SciPy. What I've found so far is finding the roots of a polynomial (single variable) and general non-linear solvers.&lt;/p&gt;&#xA;" OwnerUserId="29458" LastEditorUserId="9667" LastEditDate="2018-12-11T16:18:13.547" LastActivityDate="2018-12-11T16:18:13.547" Title="Solve system of polynomial equations with Python" Tags="|python|nonlinear-equations|scipy|polynomials|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30690" PostTypeId="2" ParentId="30681" CreationDate="2018-12-11T17:26:32.443" Score="8" Body="&lt;p&gt;Performing &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; steps of GMRES uses &lt;span class=&quot;math-container&quot;&gt;$O(n k^2)$&lt;/span&gt; time and &lt;span class=&quot;math-container&quot;&gt;$O(n k)$&lt;/span&gt; memory. In other words, the algorithm gets more and more expensive with each additional iteration. In theory, the algorithm terminates in &lt;span class=&quot;math-container&quot;&gt;$k \le n$&lt;/span&gt; steps (ignoring round-off), thereby yielding a worst-case cubic &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt; time and quadratic &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt; memory figure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, we manually terminate the algorithm after &lt;span class=&quot;math-container&quot;&gt;$k=O(1)$&lt;/span&gt; steps to yield a linear &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; time and memory figure. We do this with the hope that convergence is sufficiently rapid that the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-th iterate will already satisfy error tolerances. In some special cases, we will be able to prove this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, when &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is large, the cost of the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-th iteration may still be too large. We can reduce this by restarting GMRES after every &lt;span class=&quot;math-container&quot;&gt;$p \le k$&lt;/span&gt; steps. In this case, the algorithm has a fixed &lt;span class=&quot;math-container&quot;&gt;$O(n p^2) = O(n)$&lt;/span&gt; time and &lt;span class=&quot;math-container&quot;&gt;$O(n p) = O(n)$&lt;/span&gt; memory complexity, independent of the final iterate count &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. The reduction in complexity comes at a price though: restarted GMRES (provably) converges slower than GMRES, and may also stagnate. &lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2018-12-11T18:37:02.093" LastActivityDate="2018-12-11T18:37:02.093" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30692" PostTypeId="1" AcceptedAnswerId="30693" CreationDate="2018-12-11T19:52:33.063" Score="0" ViewCount="216" Body="&lt;p&gt;I am trying to solve a simple optimal control problem using the Hamilton-Jacobi-Bellman equation, numerically in Python. This is proving to be rather difficult as I end up having to solve the following:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J_t - (J_x)^2 + x\cdot J_x = 0&#xA;$$&lt;/span&gt;&#xA;I believe this to be a non-linear first order PDE. Being the HJB, we are given boundary condition at terminal time.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mbox{BC:} \quad J(x_f) = \dfrac{1}{4} x_f^2&#xA;$$&lt;/span&gt;&#xA;I have attempted this problem myself by simply putting in central Euler approximations for &lt;span class=&quot;math-container&quot;&gt;$J_x$&lt;/span&gt; and using backward difference for &lt;span class=&quot;math-container&quot;&gt;$J_t$&lt;/span&gt;. I then solve backwards in time by making &lt;span class=&quot;math-container&quot;&gt;$J_{i}^{j-1}$&lt;/span&gt; the subject of the formula.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J_{i}^{j-1} = J_{i}^{j}+\frac{k}{4h^2}\big[ J_{i+1}^{j}-J_{i-1}^{j} \big]^2 + \frac{k}{h}\big[ J_{i+1}^{j}-J_{i-1}^{j}   \big]x_i&#xA;$$&lt;/span&gt;&#xA;This only works for a few computations before the scheme becomes very unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you please help guide me on how to solve this. I don't quite know whether my approach is correct. Please suggest resources or help me to develop a scheme that would work.&lt;/p&gt;&#xA;" OwnerUserId="29463" LastEditorUserId="29463" LastEditDate="2018-12-11T20:50:56.073" LastActivityDate="2018-12-12T04:49:16.553" Title="Numerical solution of non-linear first order partial differential equation (HJB)" Tags="|pde|finite-difference|nonlinear-equations|numerics|optimal-control|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30693" PostTypeId="2" ParentId="30692" CreationDate="2018-12-12T03:58:14.570" Score="1" Body="&lt;p&gt;I haven't solved the HJB equations myself, but I can think of two things you can try based on my experience with other PDEs. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Use an &lt;a href=&quot;https://en.wikipedia.org/wiki/Upwind_scheme&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;upwinded&quot; approximation&lt;/a&gt; for &lt;span class=&quot;math-container&quot;&gt;$J_x$&lt;/span&gt;, instead of a central difference scheme.&lt;/strong&gt; The &lt;span class=&quot;math-container&quot;&gt;$x J_x$&lt;/span&gt; term in the equation suggests that information is flowing from left to right in your system as you move forward in time (assuming &lt;span class=&quot;math-container&quot;&gt;$x &amp;gt; 0$&lt;/span&gt;). Therefore it makes more sense to construct the &lt;span class=&quot;math-container&quot;&gt;$J_x$&lt;/span&gt; approximation using &lt;span class=&quot;math-container&quot;&gt;$J_i$&lt;/span&gt; and the nodes to its left, since that's where the information is coming from. The simplest such approximation is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$J_x \approx \frac{1}{h}\left(J_i - J_{i-1}\right)$$&lt;/span&gt;&#xA;However, I'm not sure how this argument applies to the nonlinear &lt;span class=&quot;math-container&quot;&gt;$(-J_x^2)$&lt;/span&gt; term. The negative sign on that term suggests a right-to-left direction, so you could probably experiment with a right-biased approximation for &lt;span class=&quot;math-container&quot;&gt;$J_x$&lt;/span&gt; there.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Use an implicit time-marching scheme, instead of an explicit scheme.&lt;/strong&gt; Explicit time-marching schemes, such as the one you are using, have stability conditions that severely restrict the size of the time-step you can take. In fact, there's a good chance your method will work (or survive for longer) if you reduce the time-step size by some large factor. But for many practical problems, this time-step constraint is simply too restrictive since it increases the cost/time required to obtain the final solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, implicit methods are unconditionally stable for many problems, meaning that the solutions remain bounded for any time-step size you choose. However, there's no free lunch; the extra stability comes at the cost of having to solve a system of equations at each time step. This is because all solution unknowns (&lt;span class=&quot;math-container&quot;&gt;$J_i$&lt;/span&gt;'s) involved in the spatial discretization are now evaluated at time &lt;span class=&quot;math-container&quot;&gt;$j-1$&lt;/span&gt;, instead of at time &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;. To make it more clear, here's how your discrete equation would look like with an implicit time-marching method,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$J_i^{j-1} - J_i^{j} = \frac{k}{4h^2} \left[J_{i+1}^{j-1} - J_{i-1}^{j-1} \right]^2 + \frac{k}{\textbf{2}h} \left[J_{i+1}^{j-1} - J_{i-1}^{j-1} \right] x_{i} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that all terms on the right-hand side are evaluated at time &lt;span class=&quot;math-container&quot;&gt;$(j-1)$&lt;/span&gt;, so your solution unknowns &lt;span class=&quot;math-container&quot;&gt;$J_{i-1}^{j-1}, J_i^{j-1}, J_{i+1}^{j-1}..$&lt;/span&gt; etc appear on &lt;strong&gt;both&lt;/strong&gt; sides of the equation. The above equation represents a nonlinear system of equations that needs to solved at each time-step, probably using something like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%27s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Newton's method&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;P.S. I think you are missing a factor of 2 in the last term of your equation. The central difference for &lt;span class=&quot;math-container&quot;&gt;$x J_x$&lt;/span&gt; needs to be divided by &lt;span class=&quot;math-container&quot;&gt;$2h$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29276" LastEditorUserId="29276" LastEditDate="2018-12-12T04:49:16.553" LastActivityDate="2018-12-12T04:49:16.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30695" PostTypeId="1" CreationDate="2018-12-12T04:34:49.833" Score="4" ViewCount="345" Body="&lt;p&gt;Consider the nonlinear least-squares minimization of a vector of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; residuals &lt;span class=&quot;math-container&quot;&gt;$\mathbf{f}$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; parameters &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2&#xA;$$&lt;/span&gt;&#xA;This can be done with a standard Gauss-Newton or Levenberg-Marquardt approach, using the &lt;span class=&quot;math-container&quot;&gt;$n \times p$&lt;/span&gt; Jacobian matrix&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;J_{ij} = \frac{\partial f_i}{\partial x_j}&#xA;$$&lt;/span&gt;&#xA;Now suppose I want to regularize the solution, so I want to solve&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2 + \lambda^2 x^T C x&#xA;$$&lt;/span&gt;&#xA;for some symmetric &lt;span class=&quot;math-container&quot;&gt;$p \times p$&lt;/span&gt; matrix &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. It seems to me there are two ways to proceed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Option 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; happens to be SPD, then we can do a Cholesky decomposition &lt;span class=&quot;math-container&quot;&gt;$C = U^T U$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; is upper triangular. Then we get&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2 + \lambda^2 || U \mathbf{x}||^2&#xA;$$&lt;/span&gt;&#xA;Here we can form the augmented residual vector and Jacobian:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\tilde{\mathbf{f}} &amp;amp;= \pmatrix{ \mathbf{f} \\ \lambda U \mathbf{x}} \\&#xA;\tilde{J} &amp;amp;= \pmatrix{ J \\ \lambda U}&#xA;\end{align}&lt;/span&gt;&#xA;and the problem becomes&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_{\mathbf{x}} || \tilde{\mathbf{f}}(\mathbf{x}) ||^2&#xA;$$&lt;/span&gt;&#xA;for which we can apply a standard Gauss-Newton or LM code to solve.&#xA;The augmented system will have &lt;span class=&quot;math-container&quot;&gt;$n + p$&lt;/span&gt; residuals and a &lt;span class=&quot;math-container&quot;&gt;$(n+p) \times p$&lt;/span&gt; Jacobian matrix. For a large number of parameters &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (say several thousand), this could add significant cost to the nonlinear iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Option 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, suppose &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is symmetric but not positive definite, then we can't perform a Cholesky factorization. In this case we can still proceed by forming the augmented system:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\tilde{\mathbf{f}} &amp;amp;= \pmatrix{ \mathbf{f} \\ \lambda \sqrt{x^T C x}} \\&#xA;\tilde{J} &amp;amp;= \pmatrix{ J \\ \lambda \frac{x^T C}{x^T C x}}&#xA;\end{align}&lt;/span&gt;&#xA;In this case, we now have &lt;span class=&quot;math-container&quot;&gt;$n + 1$&lt;/span&gt; residuals and the Jacobian is &lt;span class=&quot;math-container&quot;&gt;$(n+1) \times p$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is whether Option 2 is superior to Option 1? Option 2 requires factoring a smaller Jacobian matrix which could result in significant speed improvements when &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is very large. However I wonder if Option 1 might converge faster to the solution, since more &quot;information&quot; is stored in the residual vector and Jacobian. Does anyone have any insights into this?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a long time I used Option 1, but recently I encountered a problem where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is symmetric but not positive definite, so I am forced to consider Option 2. Just wondering if Option 2 might have worse convergence properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: it seems there is a third option which may or may not be useful:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Option 3&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimization problem can be put in the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\min_{\mathbf{x}} \tilde{\mathbf{f}}^T W \tilde{\mathbf{f}}&#xA;$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W = \pmatrix{ I &amp;amp; 0 \\ 0 &amp;amp; C}&#xA;$$&lt;/span&gt;&#xA;and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\tilde{\mathbf{f}} = \pmatrix{ \mathbf{f} \\ \lambda x}&#xA;$$&lt;/span&gt;&#xA;In this form, it may not be possible to use an off-the-shelf LM software due to the &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; matrix, but certainly a specialized algorithm could be developed to solve this problem. Again I am not clear if this offers any advantage over Option 1 or 2.&lt;/p&gt;&#xA;" OwnerUserId="28440" LastEditorUserId="28440" LastEditDate="2018-12-12T17:06:57.790" LastActivityDate="2018-12-12T17:06:57.790" Title="Nonlinear least squares and regularization" Tags="|optimization|nonlinear-equations|iterative-method|least-squares|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="30696" PostTypeId="1" AcceptedAnswerId="30698" CreationDate="2018-12-12T08:59:39.347" Score="2" ViewCount="97" Body="&lt;p&gt;Is there a faster way to check whether &lt;span class=&quot;math-container&quot;&gt;$A\in\mathbb{R}^{n\times n}$&lt;/span&gt; is positive definite on &lt;span class=&quot;math-container&quot;&gt;$b^{\bot}:=\{x\in \mathbb{R}^{n}: x\cdot b=0\}$&lt;/span&gt; than &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function foo(A,b)&#xA;    N = null(b);&#xA;    e = eig(N'*A*N);&#xA;    return all(e&amp;gt;0);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7789" LastActivityDate="2018-12-13T07:35:42.367" Title="Checking positive definiteness on a hyperplane" Tags="|linear-algebra|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30697" PostTypeId="1" AcceptedAnswerId="33096" CreationDate="2018-12-12T09:41:26.557" Score="1" ViewCount="712" Body="&lt;p&gt;In my calculation (of a simple heat equation, for testing) using the Newton method, I tried to replace the full Jacobian matrix with an approximation vector, i.e. replacing &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; in &#xA;&lt;span class=&quot;math-container&quot;&gt;$$J(u)\delta u=-F(u)$$&lt;/span&gt;&#xA;with &#xA;&lt;span class=&quot;math-container&quot;&gt;$$J\vec{v}\approx\frac{F(\vec{u}+\varepsilon \vec{v}) - F(\vec{u})}{\varepsilon}$$&lt;/span&gt;&#xA;which should be possible, after I solve the system using a GMRES-solver. Here &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt; is calculated using &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\varepsilon=\frac{1}{n\vert\vert\vec{v}\vert\vert_2}\sum_{i=1}^n\left(b\vert u_i\vert+b\right)\text{, }b=10^{-6}$$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$F(u)$&lt;/span&gt; is the function itself, i.e. &#xA;&lt;span class=&quot;math-container&quot;&gt;$$F(u)=\nabla^2u+f$$&lt;/span&gt;&#xA;Now I encountered two problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Usually in the first iteration &lt;span class=&quot;math-container&quot;&gt;$\vert\vert\vec{v}\vert\vert_2=0$&lt;/span&gt;, resulting in infinity for &lt;span class=&quot;math-container&quot;&gt;$\varepsilon$&lt;/span&gt;. My current solution is to set &lt;span class=&quot;math-container&quot;&gt;$\varepsilon=10^{-6}$&lt;/span&gt; in that case, but is that correct? I could not find any solution for that in the literature&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When considering the value for &lt;span class=&quot;math-container&quot;&gt;$F(u)$&lt;/span&gt; at each step, it decreases (as it should) when using the full version of &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt;, but increases when using the approximation. Is there a way how can I narrow down the possible problems here?&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How can I solve those problems (or is there literature about them which I could not find yet)?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastEditorUserId="20688" LastEditDate="2019-07-22T17:36:21.087" LastActivityDate="2019-07-22T17:36:21.087" Title="Implementation of the Jacobian-free Newton method" Tags="|newton-method|jacobian|matrix-free|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30698" PostTypeId="2" ParentId="30696" CreationDate="2018-12-12T13:14:17.430" Score="5" Body="&lt;p&gt;Instead of &lt;code&gt;e = eig(N'*A*N);&lt;/code&gt;, you can use &lt;code&gt;[R,p]=chol(N'*A*N);&lt;/code&gt;, and test for &lt;code&gt;p==0&lt;/code&gt;. Matlab function &lt;a href=&quot;https://www.mathworks.com/help/matlab/ref/chol.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;chol&lt;/code&gt;&lt;/a&gt; returns the cholesky factor &lt;code&gt;R&lt;/code&gt; and  a &lt;code&gt;p&lt;/code&gt; which is zero if &lt;code&gt;N'*A*N&lt;/code&gt; is positive definite. &lt;code&gt;p&lt;/code&gt; is a positive integer if the matrix is not positive definite. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually &lt;code&gt;chol&lt;/code&gt; is much faster than &lt;code&gt;eig&lt;/code&gt;. For example, with dimension &lt;code&gt;n=2000&lt;/code&gt;, I found that &lt;code&gt;chol(N'*A*N)&lt;/code&gt; is about  10 times faster than &lt;code&gt;eig(N'*A*N)&lt;/code&gt; (tested with Octave instead of Matlab).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix product &lt;code&gt;N'*A*N&lt;/code&gt; is relatively expensive to compute. It might be faster to use a sparse basis &lt;code&gt;N&lt;/code&gt; for &lt;span class=&quot;math-container&quot;&gt;$b^{\bot}$&lt;/span&gt;: Matlab/Octave code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function r = foo2(A,b)&#xA;    [dummy,p] = sort(abs(b),&quot;descend&quot;);&#xA;    A = A(p,p);&#xA;    m = size(A,1);&#xA;    b = b(p);&#xA;    N = [b(2:m)/-b(1);speye(m-1,m-1)];   % now b*N = 0&#xA;    [L,p] = chol(N'*A*N);&#xA;    r = (p==0);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5286" LastEditorUserId="5286" LastEditDate="2018-12-13T07:35:42.367" LastActivityDate="2018-12-13T07:35:42.367" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30699" PostTypeId="2" ParentId="30676" CreationDate="2018-12-12T14:48:23.890" Score="0" Body="&lt;p&gt;It appears that the original statement is essentially a slight generalization of the Knapsack Problem since you assume your functions &lt;span class=&quot;math-container&quot;&gt;$\text{Clicks}(\cdot)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{Cost}(\cdot)$&lt;/span&gt; can be any monotonic functions, instead of the linear functions seen in the traditional Bounded Knapsack Problem assumes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think one can be sure that this is at least as hard as the Knapsack Problem. I don't know what values for &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; you have, but this can certainly be solved exactly using dynamic programming (DP). The main issues you may run into is if &lt;span class=&quot;math-container&quot;&gt;$N = O(2^K)$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is instead polynomial in &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;, you should be able to solve this problem in polynomial time and space using DP pretty readily.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2018-12-12T14:48:23.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30701" PostTypeId="2" ParentId="30697" CreationDate="2018-12-12T15:24:23.770" Score="6" Body="&lt;p&gt;Not sure where you get your equation for &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;, but ultimately your approximation for the Jacobian matvec operation is a finite difference approximation to the directed derivative of &lt;span class=&quot;math-container&quot;&gt;$F(\cdot)$&lt;/span&gt;. This means that you will want as accurate of an approximation to this directed derivative as possible while being sure to avoid numerical round-off issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this situation, I have personally used the following method referred to as the &lt;strong&gt;Complex-Step Derivative Approximation&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\nabla F(\boldsymbol{x}) \cdot \boldsymbol{v} = \frac{\text{Im}(F(\boldsymbol{x} + i \epsilon \boldsymbol{v}))}{\epsilon} + O(\epsilon^2)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\text{Im}(\cdot)$&lt;/span&gt; maps a complex vector in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{C}^n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; by just taking the imaginary part of each component. Note that if &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{v}$&lt;/span&gt; is not a unit vector, you should compute the above quantity with the direction of &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{v}$&lt;/span&gt; and then multiply the result by &lt;span class=&quot;math-container&quot;&gt;$\left\lVert \boldsymbol{v}\right\rVert$&lt;/span&gt;. The obvious thing to note here is that the above assumes you can modify your code to allow for feeding in complex numbers into &lt;span class=&quot;math-container&quot;&gt;$F(\cdot)$&lt;/span&gt;. If you can do this, this approximation has great numeric precision since it does not have round-off issues related to subtractions, like typical Finite Difference approximations. You can readily choose &lt;span class=&quot;math-container&quot;&gt;$\epsilon = 10^{-8}$&lt;/span&gt; and get close to machine precision.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2018-12-12T22:20:51.937" LastActivityDate="2018-12-12T22:20:51.937" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="30703" PostTypeId="1" CreationDate="2018-12-13T01:10:50.520" Score="3" ViewCount="223" Body="&lt;p&gt;I'd like to know how to deal with a divergence when trying to solve the Poisson equation for electrostatics with a simple spectral method. I'm not sure how to best state my problem, so I'll explain how I got to the problem. I'm sorry that this makes the post long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that I have two conducting ground planes at &lt;span class=&quot;math-container&quot;&gt;$z=\pm L_z$&lt;/span&gt;, an infinite domain in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, and some surface charge at &lt;span class=&quot;math-container&quot;&gt;$z=0$&lt;/span&gt; (i.e. &lt;span class=&quot;math-container&quot;&gt;$\rho\left(x,y,z\right)=\sigma\left(x,y\right)\delta\left(z\right)$&lt;/span&gt;). I can take Poisson's equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\nabla^2\phi\left(x,y,z\right)=-\frac{\rho\left(x,y,z\right)}{\epsilon_0}=\frac{\sigma\left(x,y\right)\delta\left(z\right)}{\epsilon_0}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and Fourier transform both sides of the equation in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; (but not &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;) to get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$-k^2\tilde{\phi}\left(k_x, k_y, z\right) + \frac{\partial^2}{\partial z^2}\tilde{\phi}\left(k_x, k_y, z\right)=\frac{\tilde{\sigma}\left(k_x,k_y\right)\delta\left(z\right)}{\epsilon_0},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$k^2=k_x^2+k_y^2$&lt;/span&gt;, and for brevity, I'll use &lt;span class=&quot;math-container&quot;&gt;$\mathbf{k}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\left(k_x,k_y\right)$&lt;/span&gt;. When &lt;span class=&quot;math-container&quot;&gt;$z\neq0$&lt;/span&gt;, the above reduces to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$-k^2\tilde{\phi}\left(\mathbf{k}, z\right) + \frac{\partial^2}{\partial z^2}\tilde{\phi}\left(\mathbf{k}, z\right)=0,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which has the solution of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\tilde{\phi}\left(\mathbf{k},z\right)=A e^{-kz} + B e^{kz}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I can look for two solutions &lt;span class=&quot;math-container&quot;&gt;$\tilde{\phi}_1\left(\mathbf{k},z\right)=A_1 e^{-kz} + B_1 e^{kz}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\tilde{\phi}_2\left(\mathbf{k},z\right)=A_2 e^{-kz} + B_2 e^{kz}$&lt;/span&gt; for above and below &lt;span class=&quot;math-container&quot;&gt;$z=0$&lt;/span&gt;, respectively. I can solve for &lt;span class=&quot;math-container&quot;&gt;$A_{1,2},B_{1,2}$&lt;/span&gt; using four boundary conditions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Top ground plane: &lt;span class=&quot;math-container&quot;&gt;$\quad\tilde{\phi}_1\left(\mathbf{k},L_z\right)=0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Bottom ground plane: &lt;span class=&quot;math-container&quot;&gt;$\quad\tilde{\phi}_2\left(\mathbf{k},-L_z\right)=0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Continuity of potential at &lt;span class=&quot;math-container&quot;&gt;$z=0$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\quad\tilde{\phi}_1\left(\mathbf{k},0\right)=\tilde{\phi}_2\left(\mathbf{k},0\right)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Effect of surface charge at &lt;span class=&quot;math-container&quot;&gt;$z=0$&lt;/span&gt;: &lt;span class=&quot;math-container&quot;&gt;$\quad\left.\frac{d\tilde{\phi}_1}{dz}\right|_{z=0} - \left.\frac{d\tilde{\phi}_2}{dz}\right|_{z=0} = -\frac{\tilde{\sigma}\left(\mathbf{k}\right)}{\epsilon_0}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For example, I find&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A_1 = \frac{\tilde{\sigma}\left(\mathbf{k}\right)}{2k\epsilon_0\left(e^{-2kL_z} + 1\right)}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once I know &lt;span class=&quot;math-container&quot;&gt;$\tilde{\phi}\left(\mathbf{k},z\right)$&lt;/span&gt;, I can do some inverse Fourier transforms to get &lt;span class=&quot;math-container&quot;&gt;$\phi\left(x,y,z\right)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose now that I want to do this for a system that is periodic in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; (say, with a square domain), and I want to do transforms numerically. So, I discretize &lt;span class=&quot;math-container&quot;&gt;$\sigma\left(x,y\right)$&lt;/span&gt; on a square mesh and replace the Fourier transforms with discrete Fourier transforms, and almost everything works. The only problem is the &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{k}$&lt;/span&gt; that show up in &lt;span class=&quot;math-container&quot;&gt;$A_{1,2},B_{1,2}$&lt;/span&gt;. It causes &lt;span class=&quot;math-container&quot;&gt;$\tilde{\phi}\left(\mathbf{k},z\right)$&lt;/span&gt; to blow up at &lt;span class=&quot;math-container&quot;&gt;$\mathbf{k}=0$&lt;/span&gt;. I can just set &lt;span class=&quot;math-container&quot;&gt;$\tilde{\phi}\left(\mathbf{0},z\right)=0$&lt;/span&gt;, but that sets the average value of &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; to zero, and I lose information I'd like to have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to get around that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'll note that, regardless of the value of &lt;span class=&quot;math-container&quot;&gt;$\sigma\left(x,y\right)$&lt;/span&gt;, each unit cell has zero net charge due to the conducting ground planes, which will have a net charge equal and opposite to the net charge from &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt;. I'll also note that I can solve this problem numerically with FEM, and it produces good results, but I'd like to solve the problem with a lighter-weight method.)&lt;/p&gt;&#xA;" OwnerUserId="29476" LastEditorUserId="29476" LastEditDate="2018-12-13T14:15:47.830" LastActivityDate="2022-12-22T22:08:46.020" Title="Average value divergence in spectral method for Poisson equation" Tags="|poisson|electromagnetism|spectral-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30704" PostTypeId="2" ParentId="29862" CreationDate="2018-12-13T03:59:59.433" Score="3" Body="&lt;p&gt;The question I think you're asking, based on your code, is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Better to have each node do redundant work&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B. Better to have one node do the work distribute the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is &lt;strong&gt;usually A&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see why, consider these Latency Numbers Every Programmer Should Know:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ILW2S.png&quot; alt=&quot;Latency Numbers Every Programmer Should Know]&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that CPU operations take essentially no time (~1ns) and that doing a decently complex operation on 1KB of local memory takes 3 microseconds. On the other hand, sending 1KB of data over a network takes 10 microseconds. Therefore, you're moving (at least) three times faster by doing the operation locally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's even better than that. In parallel computing we call the &quot;critical path&quot; the longest series of serial operations we are obliged to do. In a program where we calculate then broadcast, the total time is the sum of these two operations. In this example, that's 13 microseconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Communication also introduces unpredictability: message transmission time can depend on message length in nonlinear ways:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6ttAr.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6ttAr.gif&quot; alt=&quot;MPI packet size versus bandwidth&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Communication bandwidth is also a function of the number of processes communicating:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wgTfc.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wgTfc.gif&quot; alt=&quot;MPI bandwidth versus number of tasks&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is: do you want to delegate the speed of your computation to a slow network with unknown timing properties? Or do you want to do redundant computation locally, where you know it will be fast?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if you compare trends in network versus memory bandwidth, the push to avoid communication is only increasing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Az8Qn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Az8Qn.jpg&quot; alt=&quot;Network versus memory bandwidth&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is why &lt;a href=&quot;https://en.wikipedia.org/wiki/Communication-avoiding_algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;communication avoiding algorithms&lt;/a&gt; are a bit of a hot topic. Where that means avoiding not just communication between network nodes, but also between RAM and CPU cache, and even different levels of the CPU cache. The speeds of various parts of computers are diverging from each other exponentially, so it is more and more important to avoid slow parts of your system as much as possible.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2018-12-13T03:59:59.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30705" PostTypeId="1" AcceptedAnswerId="30737" CreationDate="2018-12-13T10:54:13.913" Score="9" ViewCount="558" Body="&lt;p&gt;I'm trying to calculate higher order (e.g., &lt;code&gt;m=0&lt;/code&gt;, &lt;code&gt;n=46&lt;/code&gt;) Zernike moments for some image. However, I'm running into a problem regarding the radial polynomial (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Zernike_polynomials&quot; rel=&quot;noreferrer&quot;&gt;wikipedia&lt;/a&gt;). This is a polynomial defined on the interval [0 1]. See the MATLAB code below&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;function R = radial_polynomial(m,n,RHO)&#xA;    R = 0;&#xA;    for k = 0:((n-m)/2)        &#xA;        R = R + (-1).^k.*factorial(n-k) ...&#xA;            ./ ( factorial(k).*factorial((n+m)./2-k) .* factorial((n-m)./2-k) ) ...&#xA;            .*RHO.^(n-2.*k);&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, this obviously runs into numerical issues near &lt;code&gt;RHO &amp;gt; 0.9&lt;/code&gt;. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/gHC2g.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gHC2g.png&quot; alt=&quot;A polynomial with lots of noise&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried refactoring it to &lt;code&gt;polyval&lt;/code&gt; thinking it might have some better behind-the-scenes algorithms but that didn't didn't solve anything. Converting it to a symbolic calculation did create the desired graph but was mindbogglingly slow even for a simple graph such as shown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a numerically stable way of evaluating such high-order polynomials?&lt;/p&gt;&#xA;" OwnerUserId="22561" LastActivityDate="2018-12-17T18:22:42.483" Title="Numerical stability of higher order Zernike polynomials" Tags="|matlab|polynomials|numerical-limitations|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="30706" PostTypeId="2" ParentId="30703" CreationDate="2018-12-13T14:14:49.080" Score="2" Body="&lt;p&gt;I have an answer, but I'm still curious if there are simpler answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that we break the charge density into two parts: one with an average density of zero, and one with a constant value. We use the above procedure on the part with zero average charge density, and we know the answer for a constant charge density. Then we just add the two potentials together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The surface charge density &lt;span class=&quot;math-container&quot;&gt;$\sigma\left(x,y\right)$&lt;/span&gt; can be broken into two parts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sigma\left(x,y\right) = \left(\sigma\left(x,y\right) - \bar{\sigma}\right) + \bar{\sigma},$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\bar{\sigma}$&lt;/span&gt; is the average value of &lt;span class=&quot;math-container&quot;&gt;$\sigma\left(x,y\right)$&lt;/span&gt; over the unit cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, deal with the term in parenthesis. Define &lt;span class=&quot;math-container&quot;&gt;$\sigma_1\left(x,y\right) = \sigma\left(x,y\right) - \bar{\sigma}$&lt;/span&gt;. Now, &lt;span class=&quot;math-container&quot;&gt;$\sigma_1$&lt;/span&gt; has zero average value, so &lt;span class=&quot;math-container&quot;&gt;$\tilde{\sigma}_1\left(\mathbf{0}\right)=0$&lt;/span&gt;. That makes it fair to set &lt;span class=&quot;math-container&quot;&gt;$\tilde{\phi}_1\left(\mathbf{0},z\right)=0$&lt;/span&gt;, and we can solve for &lt;span class=&quot;math-container&quot;&gt;$\phi_1$&lt;/span&gt; using the above method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, &lt;span class=&quot;math-container&quot;&gt;$\sigma_2\left(x,y\right) = \bar{\sigma}$&lt;/span&gt; is just a uniform sheet charge, and we know that it will produce &lt;span class=&quot;math-container&quot;&gt;$\mathbf{E}=\pm \frac{\bar{\sigma}}{2\epsilon_0}$&lt;/span&gt; (+ for &lt;span class=&quot;math-container&quot;&gt;$z&amp;gt;0$&lt;/span&gt;, - for &lt;span class=&quot;math-container&quot;&gt;$z&amp;lt;0$&lt;/span&gt;). This results in &lt;span class=&quot;math-container&quot;&gt;$\phi_2=\frac{\bar{\sigma}}{2\epsilon_0}\left(\pm L_x-z\right)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we put everything together to find &lt;span class=&quot;math-container&quot;&gt;$\phi = \phi_1 + \phi_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd be interested in any other approaches, because I'm interested in tackling more complicated problems (multiple charged sheets and dielectric layers) with a similar method, and extending the above method will certainly work, but maybe it's not the simplest answer.&lt;/p&gt;&#xA;" OwnerUserId="29476" LastEditorUserId="29476" LastEditDate="2018-12-13T16:13:58.470" LastActivityDate="2018-12-13T16:13:58.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30707" PostTypeId="1" CreationDate="2018-12-13T17:15:09.937" Score="5" ViewCount="857" Body="&lt;p&gt;Homework disclaimer...&lt;/p&gt;&#xA;&lt;p&gt;The task:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We are using the following algorithm to solve the quadratic equation &lt;span class=&quot;math-container&quot;&gt;$x^2+2px+q=0$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_1=|p|+\sqrt{p^2-q}\mathtt{;}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathtt{if}\,p&amp;gt;0\,\mathtt{then}\,x_1=-x_1\mathtt{;}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_2=q/x_1\mathtt{;}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Point out two places where running this algorithm can meet significant problems. How to mitigate these problems?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That's my guess: The first place is this: &lt;span class=&quot;math-container&quot;&gt;$p^2-q$&lt;/span&gt;, we're risking catastrophic cancellation here; and the second place is this: &lt;span class=&quot;math-container&quot;&gt;$q/x_1$&lt;/span&gt;, we're getting a NaN if &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The second one seems kind of easy: the only way for the absolutey larger root to be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; is when both roots are &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt;, which entails so &lt;span class=&quot;math-container&quot;&gt;$p=q=0$&lt;/span&gt;; we can test for this case.&lt;/p&gt;&#xA;&lt;p&gt;I don't know how to mitigate the first problem, though. I can't think of a way to solve a quadratic equation without computing its discriminant, and computing its discriminant does require subtraction. Which, of course, risks catastrophic cancellation.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to avoid catastrophic cancellation when computing the discriminant of a quadratic function?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: As per request from comments, moving additional info from answer to question:&lt;/p&gt;&#xA;&lt;p&gt;Hah. Regarding &lt;span class=&quot;math-container&quot;&gt;$x_1=|p|+\sqrt{p^2-q}$&lt;/span&gt;. Another student was arguing today that this cannot be an issue: According to him, catastrophic cancellation can only occur if &lt;span class=&quot;math-container&quot;&gt;$p^2\approx q$&lt;/span&gt;; but then &lt;span class=&quot;math-container&quot;&gt;$\sqrt{p^2-q}\approx 0$&lt;/span&gt; and moreover &lt;span class=&quot;math-container&quot;&gt;$|p|\gg\sqrt{p^2-q}$&lt;/span&gt;; so the discriminant itself and any errors introduced by computing it loose significance when computing &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt;. The Professor said he would have to think about it; I'll post an update here if/when he's done thinking.&lt;/p&gt;&#xA;&lt;p&gt;However, there is one more problem I failed to notice. Namely, if &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is large, then &lt;span class=&quot;math-container&quot;&gt;$p^2$&lt;/span&gt; may overflow; but then &lt;span class=&quot;math-container&quot;&gt;$\sqrt{p^2-q}$&lt;/span&gt; will also overflow even if its accurate value would fit in the range of the number format; so &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; will be spuriously set to &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;. To avoid this, the Professor said, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; needs to be rescaled if its large enough.&lt;/p&gt;&#xA;" OwnerUserId="26311" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-12-14T21:46:22.253" Title="Any way to avoid catastrophic cancellation when computing the discriminant of a quadratic function?" Tags="|stability|floating-point|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30713" PostTypeId="2" ParentId="30705" CreationDate="2018-12-14T12:40:03.333" Score="8" Body="&lt;p&gt;A possible solution (suggested by @gammatester) is to use Jacobi polynomials. This circumvents the problem of catastrophic cancellation in adding the large polynomial coefficients by 'naive' polynomial evaluation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The radial Zernike polynomial can be expressed by Jacobi polynomials as follows (see &lt;a href=&quot;http://mathworld.wolfram.com/ZernikePolynomial.html&quot; rel=&quot;noreferrer&quot;&gt;equation (6)&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ R^m_n(\rho) = (-1)^{(n-m)/2}\rho^m \cdot P^{(m,0)}_{(n-m)/2} \Big(1-2\rho^2 \Big)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB however, the use of &lt;code&gt;jacobiP(n,a,b,x)&lt;/code&gt; is unacceptably slow for large vectors/matrices of &lt;code&gt;x=rho&lt;/code&gt;. The &lt;code&gt;jacobiP&lt;/code&gt; function is actually part of the Symbolic Toolbox, and evaluation of the polynomial is deferred to the symbolic engine, which trades speed for arbitrary precision. A manual implementation of the Jacobi polynomials is thus necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the parameters to the Jacobi function are all nonnegative (&lt;span class=&quot;math-container&quot;&gt;$\alpha=m$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\beta=0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$n^*=(n-m/2)$&lt;/span&gt;), we can use the following expression (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Jacobi_polynomials#Definitions&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia&lt;/a&gt;, note that I filled in the values for &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{multline} P_n^{(\alpha,\beta)}(\rho) = (n+\alpha)!(n+\beta)! \quad \cdot\\&#xA;\sum_{s={0}}^n \left[\frac{1}{s!(n+\alpha-s)!(\beta+s)!(n-s)!} \left(\frac{x-1}{2}\right)^{n-s}\left(\frac{x+1}{2}\right)^{s} \right]\end{multline}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In MATLAB, this translates to (Jacobi &lt;s&gt;&lt;strong&gt;p&lt;/strong&gt;olice &lt;strong&gt;d&lt;/strong&gt;epartment&lt;/s&gt; &lt;strong&gt;P&lt;/strong&gt;olynomial, '&lt;strong&gt;D&lt;/strong&gt;ouble' implementation)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function P = jacobiPD(n,a,b,x)&#xA;    P = 0;&#xA;    for  s  0:n&#xA;        P = P + ...&#xA;            1/(factorial(s)*factorial(n+a-s)*factorial(b+s)*factorial(n-s)) * ...&#xA;            ((x-1)/2).^(n-s).*((x+1)/2).^s;&#xA;    end&#xA;    P = P*factorial(n+a) * factorial(n+b);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The actual radial Zernike polynomial is thus (for &lt;code&gt;m=abs(m)&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Z = @(m,n,rho) (-1)^((n-m)/2) * rho.^m .* jacobiPD((n-m)/2,m,0,1-2*rho.^2);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note: this self-answer is only a practical solution; feel free to tag on another answer that explains &lt;em&gt;why&lt;/em&gt; this works.&lt;/p&gt;&#xA;" OwnerUserId="22561" LastEditorUserId="22561" LastEditDate="2018-12-14T22:53:11.600" LastActivityDate="2018-12-14T22:53:11.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30714" PostTypeId="1" AcceptedAnswerId="30723" CreationDate="2018-12-14T15:15:35.010" Score="0" ViewCount="56" Body="&lt;p&gt;Supposing that one molecule is static and one is dynamic, &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;can the dynamic one be solved with LINCS for its shape (angle, bond length) constraints and also keep collisions with static molecule off, using single-pass (with order-4) LINCS?&lt;/li&gt;&#xA;&lt;li&gt;if single pass LINCS is not enough to keep both molecules non-interfered state and dynamic molecule's original shape, then how many passes of LINCS(order-4) would be needed on average for N atoms of dynamic molecule and M atoms of static molecule? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2014" LastActivityDate="2018-12-15T21:51:44.117" Title="Can LINCS algorithm be used for colliding molecules?" Tags="|linear-solver|constrained-optimization|molecular-dynamics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30715" PostTypeId="1" CreationDate="2018-12-14T15:26:33.860" Score="0" ViewCount="198" Body="&lt;p&gt;I have the following integral:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\int_{1}^{Xd} \dfrac{(X^{z_i}-1)}{[X^2 \sum_{l=1}^{N}c_l(X^{z_l}-1)]^{1/2}}dX = \int_{1}^{Xd} h(X) dX$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Xd is a real that can be either negative, positive or even 1.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; is a integer (positive or negative)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$z_l$&lt;/span&gt; also a integer (&lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; will be in the vector that contain all the  &lt;span class=&quot;math-container&quot;&gt;$z_l$&lt;/span&gt; of length N)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;&lt;span class=&quot;math-container&quot;&gt;$c_l$&lt;/span&gt; is a  vector of positive real&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can observe that there is a singularity at the limit value 1. Where I got division of 0/0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Spyder idle (so python + libraries) and I have tried with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.integrate as integrate&#xA;&#xA;partB = integrate.quad(self.integrand_fun_Borkovec_1983_eqn_11, 1, Xd, arg = (zi,zl_vec, cl_vec))&#xA;&#xA;def integrand_fun_Borkovec_1983_eqn_11 (self, x, zi,z_vec, cb):&#xA;    a = (x**zi)-1&#xA;    b= 0&#xA;    for i in range(0, len(z_vec)):&#xA;        b = b + cb[i]*((x**z_vec[i])-1)&#xA;    b = x*x*b&#xA;    return a/b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for the following values of Xd = 0.07544956110914136, zi = 1, zl_vec = [1,1,-1,-1] and cl_vec = [1.02882947e-07, 8.43609299e-05, 8.43566150e-05, 9.71978375e-08]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the following result in part B: (166085.34873353728, 6428.115530877825). The value 6428.115530877825 is an estimation of the error, as you can seen the estimated error is pretty high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read in the paper that I am using Borkovec(1983) the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Expand the integrand to second order in X-1 for values of X near 1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What would mean (If I am not wrong):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h(x) \approx h(X-1)+h'(X-1)+(1/2)h''(X-1)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, I will need to calculate h'(X-1) and h''(X-1). I wonder if there is an easier way to solve this singularity. I have tried using the points argument of the quad function, but the estimated error is still high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;partB = integrate.quad(self.integrand_fun_Borkovec_1983_eqn_11, 1, Xd, arg = (zi,zl_vec, cl_vec), points=[0.95, 1])&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="22125" LastEditorUserId="22125" LastEditDate="2018-12-17T09:24:13.977" LastActivityDate="2018-12-17T10:38:48.907" Title="How to cope with the following singularity" Tags="|python|quadrature|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30716" PostTypeId="2" ParentId="30715" CreationDate="2018-12-14T22:08:33.583" Score="2" Body="&lt;p&gt;Do not compute the expansion of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, compute the expansions of the simpler terms. For this purpose you can use a series approximation of &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;X^z-1&amp;amp;=(1+(X-1))^z-1=\sum_{k=0}^\infty\binom{z}{k}(X-1)^k\\&#xA;&amp;amp;=z(X-1)+\frac{z(z-1)}2(X-1)^2+\frac{z(z-1)(z-2)}6(X-1)^3+...&#xA;\end{align}&lt;/span&gt;&#xA;where the terms in the expansion get rapidly small if &lt;span class=&quot;math-container&quot;&gt;$X-1$&lt;/span&gt; is small enough.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def term(X,z):&#xA;    if abs(X-1)&amp;gt;1e-6: return X**z-1;&#xA;    return z*(X-1)*(1+(z-1)*(X-1)/2.0*(1+(z-2)*(X-1)/3.0*(1+(z-3)*(X-1)/4.0)));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The integral under consideration is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int_1^{X_d}\frac{X^{z_i}-1}{\sqrt{X^2\sum_{l=1}^N c^B_l(X^{z_l}-1)}}&#xA;$$&lt;/span&gt;&#xA;under the condition &lt;span class=&quot;math-container&quot;&gt;$\sum_{l=1}^N c^B_l z_l=0$&lt;/span&gt;. This has the consequence that close to &lt;span class=&quot;math-container&quot;&gt;$X=1$&lt;/span&gt; the integrand is approximately&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{&#xA;  z_i+\frac{z_i(z_i-1)}2(X-1)+...&#xA;}{&#xA;  X\sqrt{\left[\sum_{l=1}^N c^B_l \frac{z_l(z_l-1)}2\right]&#xA;  +\left[\sum_{l=1}^N c^B_l \frac{z_l(z_l-1)(z_l-2)}6\right](X-1)+...}&#xA;}&#xA;$$&lt;/span&gt;&#xA;which does no longer have a singularity at &lt;span class=&quot;math-container&quot;&gt;$X=1$&lt;/span&gt;. To compensate for the rounding error in the given coefficients, compute the defect &lt;span class=&quot;math-container&quot;&gt;$d=\sum_{l=1}^N c^B_l z_l$&lt;/span&gt; and remove &lt;span class=&quot;math-container&quot;&gt;$d\,(X-1)$&lt;/span&gt; from the sum in the denominator. This then clears all negative values that previously occurred and thus removes the necessity to &quot;sanitize&quot; the value of &lt;code&gt;b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def integrand_fun_Borkovec_1983_eqn_11 (x, zi,z_vec, cb):&#xA;    a = term(x,zi)&#xA;    b = sum(cB*term(x,z) for z,cB in zip(z_vec,cb))&#xA;    d = sum(cB*z for z,cB in zip(z_vec,cb))&#xA;    b = x*x*(b-d*(x-1))&#xA;    if b&amp;lt;0: print x,b&#xA;    return a/b**0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now running the integration with the routines of scipy gives a value with a low error bound&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import scipy.integrate as integrate&#xA;Xd = 0.07544956110914136&#xA;zi = 1&#xA;zl_vec = [1,1,-1,-1] &#xA;cl_vec = [1.02882947e-07, 8.43609299e-05, 8.43566150e-05, 9.71978375e-08]&#xA;print &quot;sum z[i]*cB[i] should be zero, but is&quot;,sum(cB*z for z,cB in zip(zl_vec,cl_vec))&#xA;&#xA;partB = integrate.quad(integrand_fun_Borkovec_1983_eqn_11, 1, Xd, args = (zi,zl_vec, cl_vec))&#xA;print &quot;partB =&quot;, partB&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives as result&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sum z[i]*cB[i] should be zero, but is 1.00000095e-08&#xA;partB = (157.85179247575064, 2.291846657454834e-06)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is good enough relative to the accuracy of the provided data.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2018-12-17T10:38:48.907" LastActivityDate="2018-12-17T10:38:48.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30717" PostTypeId="1" CreationDate="2018-12-14T23:38:48.293" Score="5" ViewCount="87" Body="&lt;p&gt;The Sokhotski-Plemelj theorem states,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lim_{\epsilon\rightarrow 0^+}\int_a^b\frac{f(x)dx}{x+i\epsilon} = \mathcal P \int_a^b \frac{f(x)dx}{x} - i\pi f(0). $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a numerically stable way to take this limit, without explicitly using the above theorem? I find that if I use &lt;span class=&quot;math-container&quot;&gt;$f(x)=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a=-b$&lt;/span&gt;, I almost converge to &lt;span class=&quot;math-container&quot;&gt;$-i\pi$&lt;/span&gt;, but not before things become unstable.&lt;/p&gt;&#xA;" OwnerUserId="29489" LastEditorUserId="29489" LastEditDate="2018-12-15T04:08:39.060" LastActivityDate="2018-12-15T04:08:39.060" Title="Is there a numerically stable way to take $\epsilon \rightarrow 0$ in integrals of the form $\int \frac{f(x)dx}{x+i\epsilon}$?" Tags="|integration|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="30718" PostTypeId="1" CreationDate="2018-12-15T11:50:06.863" Score="1" ViewCount="58" Body="&lt;p&gt;I have a question regarding the condition number of two different perturbation matrices. To start with let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; be a spd matrix with elements defined by &lt;span class=&quot;math-container&quot;&gt;$a_{i,j} = \int\limits_{\Omega\subset \mathbb{R}^d} \nabla \varphi_i\cdot \nabla\varphi_j \,d\Omega$&lt;/span&gt; arising in Finite Element Methods with &lt;span class=&quot;math-container&quot;&gt;$\varphi_i$&lt;/span&gt; basis functions of the approximation space. Furthermore let &lt;span class=&quot;math-container&quot;&gt;$cond_2(A)$&lt;/span&gt; be the condition number of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;My first question is regarding the numerical integration of the matrix elements and how do they affect the condition number. My Idea was to write &lt;span class=&quot;math-container&quot;&gt;$\tilde A = A + E$&lt;/span&gt; where the elements of &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; illustrate the error between the numerical integration and the exact integration. Is there any way to give a inequality relation between &lt;span class=&quot;math-container&quot;&gt;$cond_2(\tilde{A})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$cond_2(A)$&lt;/span&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;My second question is about the limit inside the condition number. Assuming a matrix &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; with elements &lt;span class=&quot;math-container&quot;&gt;$e_{i,j} = \int\limits_{\tilde{\Omega}\subset \mathbb{R}^d} \nabla \tilde{\varphi}_i\cdot \nabla\tilde{\varphi}_j \,d\Omega$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\tilde{\Omega}\cap\Omega = \emptyset$&lt;/span&gt;. How does the condition number reacts to limit of matrices i.e. does it hold that &lt;span class=&quot;math-container&quot;&gt;$cond_2(A + \lim_{\alpha\to 0} \alpha E) = cond_2(A)$&lt;/span&gt; or is there any inequality relation between these two? I first tried to use &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ cond_2(A + \lim_{\alpha\to 0} \alpha E) \leq cond_2(A) + cond_2(\lim_{\alpha\to 0} \alpha E)$$&lt;/span&gt;&#xA;but I realized that the condition number of zero Matrix is defined as &lt;span class=&quot;math-container&quot;&gt;$\infty$&lt;/span&gt;. And therefore the inequality is useless. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user29088" LastActivityDate="2018-12-15T14:30:26.460" Title="Condition number of two perburbation matrix regarding limit and quadtrature integration rules" Tags="|condition-number|conditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30719" PostTypeId="2" ParentId="30718" CreationDate="2018-12-15T14:30:26.460" Score="2" Body="&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; For a matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; with distinct eigenvalues, adding a perturbation &lt;span class=&quot;math-container&quot;&gt;$\delta A$&lt;/span&gt; results [1] in a change to eigenvalues of magnitude (to first order)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \delta\lambda_i = (X^{-1}\delta A X)_{ii}, $$&lt;/span&gt;&#xA;so &lt;span class=&quot;math-container&quot;&gt;$|\delta \lambda_i| \leq \kappa(X)\|\delta A\|$&lt;/span&gt;. The change in condition number &lt;span class=&quot;math-container&quot;&gt;$\kappa(A)$&lt;/span&gt; is then given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \delta(\kappa(A)) = \kappa(A)\big(\delta(\log \lambda_n) - \delta(\log \lambda_1)\big) \leq \kappa(A)\|\delta A\|\big(|\lambda_n|^{-1} + |\lambda_1|^{-1}\big),$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\lambda_1,\lambda_n$&lt;/span&gt; are the smallest and largest eigenvalues by absolute value. Since this is proportional to &lt;span class=&quot;math-container&quot;&gt;$\|\delta A\|$&lt;/span&gt;, the only issue you can get is if your matrix is almost singular and the perturbation turns it singular, then &lt;span class=&quot;math-container&quot;&gt;$\delta(\kappa(A))$&lt;/span&gt; would be large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; It isn't true that &lt;span class=&quot;math-container&quot;&gt;$\kappa(A+B)\leq \kappa(A) + \kappa(B)$&lt;/span&gt;. Consider &lt;span class=&quot;math-container&quot;&gt;$A=\mathrm{diag}(1,1+\epsilon)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B=\mathrm{diag}(1,-1)$&lt;/span&gt;, then both are well-conditioned, &lt;span class=&quot;math-container&quot;&gt;$\kappa(A)=1+\epsilon$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\kappa(B)=1$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$\kappa(A+B)=\epsilon^{-1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[1] &lt;em&gt;An extended collection of matrix derivative results for forward and reverse mode automatic differentiation&lt;/em&gt;, M. Giles, &lt;a href=&quot;https://people.maths.ox.ac.uk/gilesm/files/NA-08-01.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(pdf)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-12-15T14:30:26.460" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30720" PostTypeId="1" AcceptedAnswerId="30738" CreationDate="2018-12-15T17:13:08.767" Score="3" ViewCount="229" Body="&lt;p&gt;I have a matrix which I want to assembly quickly, which is in block form:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A = \pmatrix{ A_{11} &amp;amp; A_{12} &amp;amp; A_{13} \\ A_{21} &amp;amp; A_{22} &amp;amp; A_{23} \\ A_{31} &amp;amp; A_{32} &amp;amp; A_{33}}&#xA;$$&lt;/span&gt;&#xA;with:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A_{ij} = \sum_{k=1}^N \alpha_{ijk} v_k v_k^T&#xA;$$&lt;/span&gt;&#xA;and my current implementation is quite slow, so I am looking for advice on how to speed it up. For a given &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, the matrix &lt;span class=&quot;math-container&quot;&gt;$v_k v_k^T$&lt;/span&gt; appears in each block &lt;span class=&quot;math-container&quot;&gt;$A_{ij}$&lt;/span&gt; so it can be precomputed and then copied into each block. I use OpenMP to distribute the calculation of &lt;span class=&quot;math-container&quot;&gt;$v_k v_k^T$&lt;/span&gt; across multiple threads, which is quite fast. The bottleneck of the code is when I need to update the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. I give each thread its own matrix A to update and then sum them all together at the end for the final A. My pseudo code is below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# pragma omp parallel for&#xA;loop from k = 1 to N&#xA;  v_k = buildv(k);   // v_k and G_k are stored in thread-specific storage&#xA;  G_k = v_k v_k^T;   // using DSYR - this step is fast&#xA;  loop i = 1 to 3    // this part is really SLOW&#xA;    loop j = 1 to 3&#xA;      alpha_ijk = ...; // this is fast&#xA;      A[thread_id](i,j) += alpha_ijk * G_k; // I wrote a matrix DAXPY routine for this&#xA;&#xA;sum A[thread_id] over all threads to build final A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I wrote the code, I had thought that the calculation of &lt;span class=&quot;math-container&quot;&gt;$v_k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$G_k$&lt;/span&gt; would be the slow parts, since they involve computations (BLAS calls). However I was suprised that the slow part is just copying already computed results to the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (even if each thread has its own &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;). So I guess I need to focus on parallizing the copy operation somehow. Does anyone have any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is quite large (about 5 million), which is why I focused on parallezing the computation of &lt;span class=&quot;math-container&quot;&gt;$v_k v_k^T$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As requested, I have coded up a minimal working example (available &lt;a href=&quot;https://ufile.io/exocl&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). I compiled the code below with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gcc -g -Wall -O2 -fopenmp -o matrix matrix.c -lm -lgsl -lcblas -latlas&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the prerequesites to run this are the GSL library and ATLAS BLAS - though you could replace ATLAS with any cblas library (-lgslcblas for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I added a #if flag to enable/disable the matrix copy operation which is the slow part. When the matrices are copied into the larger thread-specific matrices, the total runtime on my machine is &lt;strong&gt;81 seconds&lt;/strong&gt;. When this code is disabled, the total run time is &lt;strong&gt;4.1 seconds&lt;/strong&gt;. I have 24 cores on the machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, I use a random number generator to specify the vectors &lt;span class=&quot;math-container&quot;&gt;$v_k$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;math.h&amp;gt;&#xA;#include &amp;lt;omp.h&amp;gt;&#xA;#include &amp;lt;sys/time.h&amp;gt;&#xA;&#xA;#include &amp;lt;gsl/gsl_math.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_matrix.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_blas.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_rng.h&amp;gt;&#xA;&#xA;static void&#xA;random_vector(const double a, const double b, gsl_vector * v, gsl_rng * r)&#xA;{&#xA;  size_t i;&#xA;&#xA;  for (i = 0; i &amp;lt; v-&amp;gt;size; ++i)&#xA;    {&#xA;      double vi = (b - a) * gsl_rng_uniform(r) + a;&#xA;      gsl_vector_set(v, i, vi);&#xA;    }&#xA;}&#xA;&#xA;static int&#xA;matrix_daxpy(const double alpha, const gsl_matrix * X, gsl_matrix * Y)&#xA;{&#xA;  const size_t M = Y-&amp;gt;size1;&#xA;  const size_t N = Y-&amp;gt;size2;&#xA;&#xA;  if (X-&amp;gt;size1 != M || X-&amp;gt;size2 != N)&#xA;    {&#xA;      GSL_ERROR (&quot;matrices must have same dimensions&quot;, GSL_EBADLEN);&#xA;    }&#xA;  else&#xA;    {&#xA;      if (alpha != 0.0)&#xA;        {&#xA;          const size_t tda_X = X-&amp;gt;tda;&#xA;          const size_t tda_Y = Y-&amp;gt;tda;&#xA;          size_t i, j;&#xA;&#xA;          for (i = 0; i &amp;lt; M; ++i)&#xA;            {&#xA;              for (j = 0; j &amp;lt; N; ++j)&#xA;                {&#xA;                  Y-&amp;gt;data[tda_Y * i + j] += alpha * X-&amp;gt;data[tda_X * i + j];&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;      return GSL_SUCCESS;&#xA;    }&#xA;}&#xA;&#xA;int&#xA;main(int argc, char *argv[])&#xA;{&#xA;  const size_t max_threads = (size_t) omp_get_max_threads();&#xA;  const size_t N = 500000;  /* my real N is around 5 million */&#xA;  const size_t m = 300;&#xA;  const size_t nblocks = 3; /* number of block matrices in A */&#xA;  const double a = -10.0;&#xA;  const double b = 10.0;&#xA;  gsl_vector **v = malloc(max_threads * sizeof(gsl_vector *));&#xA;  gsl_matrix **GTG = malloc(max_threads * sizeof(gsl_matrix *));&#xA;  gsl_rng **r = malloc(max_threads * sizeof(gsl_rng *));&#xA;  gsl_matrix **A = malloc(max_threads * sizeof(gsl_matrix *));&#xA;  size_t k;&#xA;  struct timeval tv0, tv1;&#xA;&#xA;  for (k = 0; k &amp;lt; max_threads; ++k)&#xA;    {&#xA;      v[k] = gsl_vector_alloc(m);&#xA;      GTG[k] = gsl_matrix_calloc(m, m);&#xA;      A[k] = gsl_matrix_calloc(nblocks * m, nblocks * m);&#xA;      r[k] = gsl_rng_alloc(gsl_rng_default);&#xA;    }&#xA;&#xA;  gettimeofday(&amp;amp;tv0, NULL);&#xA;&#xA;#pragma omp parallel for&#xA;  for (k = 0; k &amp;lt; N; ++k)&#xA;    {&#xA;      int thread_id = omp_get_thread_num();&#xA;      size_t i, j;&#xA;&#xA;      /* construct v_k */&#xA;      random_vector(a, b, v[thread_id], r[thread_id]);&#xA;&#xA;      /* GTG += v_k v_k^T */&#xA;      gsl_blas_dsyr(CblasLower, 1.0, v[thread_id], GTG[thread_id]);&#xA;&#xA;      /* copy lower triangle to upper triangle */&#xA;      gsl_matrix_transpose_tricpy('L', 0, GTG[thread_id], GTG[thread_id]);&#xA;&#xA;#if 1&#xA;      /* SLOW PART: now copy alpha_{ijk} v_k v_k^T into thread-specific final A */&#xA;      for (i = 0; i &amp;lt; nblocks; ++i)&#xA;        {&#xA;          for (j = 0; j &amp;lt;= i; ++j)&#xA;            {&#xA;              double alpha_ijk = 1.0; /* for simplicity */&#xA;              gsl_matrix_view Av = gsl_matrix_submatrix(A[thread_id], i * m, j * m, m, m); /* A_{ij} m-by-m block */&#xA;&#xA;              /* copy alpha_ijk v_k v_k^T into appropriate spot */&#xA;              matrix_daxpy(alpha_ijk, GTG[thread_id], &amp;amp;Av.matrix);&#xA;            }&#xA;        }&#xA;#endif&#xA;    }&#xA;&#xA;  gettimeofday(&amp;amp;tv1, NULL);&#xA;  fprintf(stderr, &quot;elapsed time = %.4f [sec]\n&quot;,&#xA;          (tv1.tv_sec + tv1.tv_usec * 1.0e-6) - (tv0.tv_sec + tv0.tv_usec * 1.0e-6));&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28440" LastEditorUserId="28440" LastEditDate="2018-12-16T22:48:01.910" LastActivityDate="2018-12-17T15:59:13.900" Title="Parallel assembly of matrix" Tags="|matrix|parallel-computing|blas|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30721" PostTypeId="1" AcceptedAnswerId="30730" CreationDate="2018-12-15T20:17:53.677" Score="1" ViewCount="54" Body="&lt;p&gt;I'm asked to find the conditions on the coefficients &lt;span class=&quot;math-container&quot;&gt;$a_1,a_2,b_1,b_2$&lt;/span&gt; in the RK method&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_{n+1} = y_n + ha_1f(t_n,y_n) + ha_2f(t_n + b_1h, y_n + b_2hf(t_n,y_n))$$&lt;/span&gt;&#xA;such that is consistent of &lt;strong&gt;(a)&lt;/strong&gt; first order and &lt;strong&gt;(b)&lt;/strong&gt; second order. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first assumption I make is that &lt;span class=&quot;math-container&quot;&gt;$b_1 = b_2 := b$&lt;/span&gt; otherwise I have no idea how to solve it. I know it should be like this, but why is this reasonable? I know it follows if you look at the butcher tablaeu, but why is a method with &lt;span class=&quot;math-container&quot;&gt;$b_1 \neq b_2$&lt;/span&gt; &quot;impossible&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I taylor expand &lt;span class=&quot;math-container&quot;&gt;$y(t_{n+1})$&lt;/span&gt;,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y(t_{n+1}) = y(t_n) + hy'(t_n) + O(h^2).$$&lt;/span&gt;&#xA;The order of consistency is &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$l = O(h^{p+1})$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; is the local residual. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The local residual takes the form&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y(t_n) + hy'(t_n) + O(h^2) - y(t_n) - ha_1f(t_n,y_n) - ha_2f(t_{n+b}, y_{n+b}) $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$= hy'(t_n) - ha_1y'(t_n) - ha_2y'(t_{n+b}) + O(h^2).$$&lt;/span&gt;&#xA;So I need to find &lt;span class=&quot;math-container&quot;&gt;$a_1,a_2,b$&lt;/span&gt; so that every term except the &lt;span class=&quot;math-container&quot;&gt;$O(h^2)$&lt;/span&gt; remains. I get for &lt;strong&gt;(a)&lt;/strong&gt; &#xA;&lt;span class=&quot;math-container&quot;&gt;$$a_1 + a_2 = 1, b=0.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;strong&gt;(b)&lt;/strong&gt; I expand one term more &#xA;&lt;span class=&quot;math-container&quot;&gt;$$y(t_{n+1}) = y(t_n) + hy'(t_n) + \frac{h^2y''(t_n)}{2} + O(h^3)$$&lt;/span&gt;&#xA;The local residual becomes &#xA;&lt;span class=&quot;math-container&quot;&gt;$$y(t_n) + hy'(t_n) + \frac{h^2y''(t_n)}{2} + O(h^3) - y(t_n) - ha_1y(t_n) - ha_2y(t_{n+b}) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But here I am stuck. How am I supposed to choose the coefficients as to remove the &lt;span class=&quot;math-container&quot;&gt;$\frac{h^2y''(t_n)}{2}$&lt;/span&gt; term?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I solve (b)? And is my answer on (a) correct?&lt;/p&gt;&#xA;" OwnerUserId="29218" LastActivityDate="2018-12-16T16:18:10.107" Title="Determine conditions on parameters (for consistency) on RK method $y_{n+1} = y_n + ha_1f(t_n,y_n) + ha_2f(t_n + b_1h, y_n + b_2hf(t_n,y_n))$" Tags="|runge-kutta|numerics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30722" PostTypeId="2" ParentId="30489" CreationDate="2018-12-15T20:47:54.597" Score="1" Body="&lt;p&gt;I develop an open source that uses a Python front end to specify the PDEs in a scripted form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://devsim.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://devsim.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A key point in all of this is that the Scharfetter-Gummel approach has historically been the most successful means of solving the drift diffusion equations.  This approach is most compatible with using Finite Volume methods.  There are hybrid approaches which use a combined form of Finite Element and Finite Volume.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please see accepted answer here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/4812/pde-solvers-for-drift-diffusion-and-related-models?rq=1&quot;&gt;PDE solvers for Drift-diffusion and related models&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Diode simulation results are presented here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://devsim.com/1d-diode-junction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://devsim.com/1d-diode-junction/&lt;/a&gt;&#xA;&lt;a href=&quot;https://devsim.com/1d-diode-junction-part-ii/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://devsim.com/1d-diode-junction-part-ii/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether you want to write your solution, or try my project, please feel free to reach out on the project mailing list.&lt;/p&gt;&#xA;" OwnerUserId="7681" LastEditorUserId="7681" LastEditDate="2018-12-15T20:54:43.033" LastActivityDate="2018-12-15T20:54:43.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30723" PostTypeId="2" ParentId="30714" CreationDate="2018-12-15T21:51:44.117" Score="2" Body="&lt;p&gt;Constrained simulation methods such as LINCS are there to keep certain degrees of freedom fixed throughout the simulation, so you can use them to make each of your molecules behave like a rigid body.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to the interactions between the two molecules, I suggest you replace the non-bonded forces between the two molecules by something like a &lt;a href=&quot;http://www.sklogwiki.org/SklogWiki/index.php/Weeks-Chandler-Andersen_perturbation_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weeks-Chandler-Andersen potential&lt;/a&gt;. You can do that in GROMACS using &lt;a href=&quot;http://www.gromacs.org/Documentation/How-tos/Tabulated_Potentials&quot; rel=&quot;nofollow noreferrer&quot;&gt;tabulated potentials&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2018-12-15T21:51:44.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30724" PostTypeId="2" ParentId="30721" CreationDate="2018-12-15T22:08:50.140" Score="1" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y''(t_n)$&lt;/span&gt; must be expressed in terms of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$f'(t_n,y_n)$&lt;/span&gt;, because of the identity &lt;span class=&quot;math-container&quot;&gt;$y'(t)=f(t,y(t))$&lt;/span&gt;. Think about it this way: you want to find the residual using only the available information at time &lt;span class=&quot;math-container&quot;&gt;$t_n$&lt;/span&gt;, and the only available values are &lt;span class=&quot;math-container&quot;&gt;$t_n,y_n,f(t_n,y_n)$&lt;/span&gt; and all the derivatives of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$(t_n,y_n)$&lt;/span&gt;. So you can't have terms involving, say, derivatives of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. &lt;a href=&quot;https://www.math.auckland.ac.nz/~butcher/ODE-book-2008/Tutorials/RK-methods.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is another example of how to handle these methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think your answer to &lt;em&gt;(a)&lt;/em&gt; is correct, it should be as in this list: &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods#Generic_second-order_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods#Generic_second-order_method&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-12-15T22:08:50.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30725" PostTypeId="2" ParentId="30720" CreationDate="2018-12-16T05:03:12.130" Score="3" Body="&lt;p&gt;The first suggestion (without seeing the code) is to check if your memory access pattern is reasonable or, preferably, optimal for the critical parts of the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One needs to keep in mind how the data is stored in RAM and minimize cache misses during your operations. In your case, you should check&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;whether your large matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, its sub-blocks &lt;span class=&quot;math-container&quot;&gt;$A_{ij}$&lt;/span&gt; are stored column-wise or row-wise. Whichever way they are stored, it should be reflected in the way they are accessed for read/write operations.&lt;/li&gt;&#xA;&lt;li&gt;try to avoid &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d3/d60/dsyr_8f.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;DSYR&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d9/dcd/daxpy_8f.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;DAXPY&lt;/code&gt;&lt;/a&gt; calls to have&lt;code&gt;INCX&lt;/code&gt; and &lt;code&gt;INCY&lt;/code&gt; different from &lt;code&gt;1&lt;/code&gt;. If the storage spacing increments are different from 1 (non-contiguous memory access pattern), it will probably be beneficial to reorganize the data. This is unlikely the problem for &lt;code&gt;DSYR&lt;/code&gt; since you mentioned it to be fast.&lt;/li&gt;&#xA;&lt;li&gt;you mentioned that you've written your own &lt;code&gt;DAXPY&lt;/code&gt; for the matrix. I am not sure if you need one of your own since you can call regular &lt;a href=&quot;http://www.netlib.org/lapack/explore-html/d9/dcd/daxpy_8f.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;DAXPY&lt;/code&gt;&lt;/a&gt;, just with &lt;code&gt;N&lt;/code&gt; now being not the length of the vector, but the size of the matrix. Your custom implementation of &lt;code&gt;DAXPY&lt;/code&gt; is highly likely to slow down your computations.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;changing the order of operations might also help. Right now, you have the loop over &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; on top. I would try looping over the blocks first and then have an OpenMP-parallelized loop for &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; inside. The number of blocks is very small, and precomputing &lt;span class=&quot;math-container&quot;&gt;$G_k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_k$&lt;/span&gt; (shared by 9 blocks) can be traded for data locality.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;you may try to manually unroll the &quot;block-loop&quot; for &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt;. It's hard to say how you are storing the individual blocks and what compiler options are used, but changing from 2-D indexing &lt;span class=&quot;math-container&quot;&gt;$i,j = 1,\ldots,3$&lt;/span&gt; to 1-D indexing &lt;span class=&quot;math-container&quot;&gt;$b=1,\ldots,9$&lt;/span&gt; can be beneficial. It will certainly remove one level of folded loops without too much hassle.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-12-16T05:03:12.130" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30727" PostTypeId="2" ParentId="10481" CreationDate="2018-12-16T07:22:37.543" Score="1" Body="&lt;p&gt;This is a generalization of the scalar case &lt;span class=&quot;math-container&quot;&gt;$f(x) = a x^2 + b x + c = 0$&lt;/span&gt;. The derivative &lt;span class=&quot;math-container&quot;&gt;$f'(x) = 2 a x + b$&lt;/span&gt; may be zero at a solution, which happens if and only if &lt;span class=&quot;math-container&quot;&gt;$b^2 = 4 a c$&lt;/span&gt; (in which case the graph of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; only touches the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-axis but does not cross it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likewise, in your quadratic system the Jacobian may be singular (i. e. not invertible) at a solution under some conditions. In this case Newton's method will fail to converge.&lt;/p&gt;&#xA;" OwnerUserId="29495" LastActivityDate="2018-12-16T07:22:37.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30729" PostTypeId="2" ParentId="29823" CreationDate="2018-12-16T10:52:25.670" Score="3" Body="&lt;p&gt;One first observation is that as &lt;span class=&quot;math-container&quot;&gt;$x,y,z$&lt;/span&gt; are angles, one should take their values mod &lt;span class=&quot;math-container&quot;&gt;$2\pi$&lt;/span&gt;. Without that, the trajectory of the initial point &lt;span class=&quot;math-container&quot;&gt;$[0,0,0]$&lt;/span&gt; exhibits this interesting pattern&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def ABC_ode(u,t):&#xA;    A, B, C = 1, 0.816, 0.5773&#xA;    x, y, z = u&#xA;    B = np.array([A*np.sin(z)+C*np.cos(y), B*np.sin(x)+A*np.cos(z), C*np.sin(y)+B*np.cos(x)])&#xA;    return B/(1e-12+sum(B**2))**0.5&#xA;&#xA;def mysolver(u0, tspan): return odeint(ABC_ode, u0, tspan, atol=1e-10, rtol=1e-11)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Rk9a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Rk9a.png&quot; alt=&quot;trajectory of the solution starting in origin&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So one has to reduce the coordinates by removing multiples of &lt;span class=&quot;math-container&quot;&gt;$2\pi$&lt;/span&gt;, but then also exclude the so-introduced &quot;long jumps&quot; by about &lt;span class=&quot;math-container&quot;&gt;$2\pi$&lt;/span&gt; from the root computation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fill the cross section at &lt;span class=&quot;math-container&quot;&gt;$z=0$&lt;/span&gt; maximally in a reasonable time, that is without computing similar trajectories, divide the square into smaller squares and remember for each small square in the grid if it was already visited. Only start trajectories from squares without visit. Here linear interpolation is used to find the intersection points, one could also use cubic interpolation with the derivative values or an additional integration step with smaller step size to get a more accurate intersection location. But at the level of the plot that is not necessary.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u0 = [0,0,0]&#xA;t = np.arange(0, 1500, 0.2)&#xA;&#xA;N=54&#xA;grid = np.zeros([N,N], dtype=int)&#xA;for i in range(N):&#xA;    for j in range(N):&#xA;        if grid[i,j]&amp;gt;0: continue;&#xA;        x0, y0 = (2*i+1)*pi/N-pi, (2*j+1)*pi/N-pi &#xA;        u0 = [x0,y0,0]&#xA;        px = []&#xA;        py = []&#xA;&#xA;        u = mysolver(u0, t); u0 = u[-1]&#xA;        u = np.mod(u+pi,2*pi)-pi&#xA;        x,y,z = u.T&#xA;&#xA;        for k in range(len(z)-1): &#xA;            if z[k]&amp;lt;=0 and z[k+1]&amp;gt;=0 and z[k+1]-z[k]&amp;lt;pi:&#xA;                s = -z[k]/(z[k+1]-z[k])  # 0 = z[k] + s*(z[k+1]-z[k])&#xA;                rx, ry = (1-s)*x[k]+s*x[k+1], (1-s)*y[k]+s*y[k+1]&#xA;                px.append(rx); &#xA;                py.append(ry);&#xA;                m, n = int((rx+pi)*N/(2*pi)), int((ry+pi)*N/(2*pi))&#xA;                grid[m,n]=1&#xA;&#xA;        #plt.plot(px, py, '-k', lw=0.2)&#xA;        plt.plot(px, py, '.', ms=3)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aG42t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aG42t.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2018-12-16T10:52:25.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30730" PostTypeId="2" ParentId="30721" CreationDate="2018-12-16T16:12:47.797" Score="1" Body="&lt;p&gt;Apply the integration method to the cases &lt;span class=&quot;math-container&quot;&gt;$f(t,x)=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f(t,x)=2t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f(t,x)=x$&lt;/span&gt; all with initial conditions &lt;span class=&quot;math-container&quot;&gt;$(t_0,x_0)=(0,1)$&lt;/span&gt;. This gets us&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f(t,x)=1&amp;amp;:&amp;amp; 1+h &amp;amp;= 1+ha_1+ha_2         &amp;amp;\implies&amp;amp; a_1+a_2=1\\&#xA;f(t,x)=2t&amp;amp;:&amp;amp; 1+h^2 &amp;amp;= 1+ha_2(2(b_1h))  &amp;amp;\implies&amp;amp; 2a_2b_1=1\\&#xA;f(t,x)=x&amp;amp;:&amp;amp; e^h &amp;amp;= 1+ha_1+ha_2(1+b_2h)) + O(h^3)&amp;amp;\implies&amp;amp; a_2b_2=\frac12&#xA;\end{align}&lt;/span&gt;&#xA;so necessarily &lt;span class=&quot;math-container&quot;&gt;$$a_1=1-a_2 ~~ \text{ and } ~~ b_1=b_2=\frac1{2a_2}.$$&lt;/span&gt; Now insert these conditions into the Taylor expansion to find that they are also sufficient for all continuously differentiable &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;x_{+1}&amp;amp;=x+(1-a)hf+ah\left(f+f_t\frac{h}{2a}+f_x\frac{h}{2a}f+O(h^2)\right)\\&#xA;&amp;amp;=x+hf+\frac{h^2}2(f_t+f_xf)+O(h^3)\\&#xA;\text{while }~~&#xA;x(t+h)&amp;amp;=x+hf+\frac{h^2}2\frac{d}{dt}f(t,x(t))+O(h^3)\\&#xA;&amp;amp;=x+hf+\frac{h^2}2(f_t+f_xf)+O(h^3)&#xA;\end{align}&lt;/span&gt;&#xA;so that indeed the local error is of size &lt;span class=&quot;math-container&quot;&gt;$O(h^3)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2018-12-16T16:18:10.107" LastActivityDate="2018-12-16T16:18:10.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30731" PostTypeId="1" CreationDate="2018-12-16T16:41:23.193" Score="5" ViewCount="88" Body="&lt;p&gt;Can someone point me in a direction to solve this kind of integral constrained system of ODEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\int_0^{1/2}\dot{y}^2(t)=p\\&#xA;&amp;amp;2\lambda_1\ddot{y}(t)+\pi cos(\pi y(t))=0\\&#xA;&amp;amp;y(0)=0,y(1/2)=1/2&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have reduced it to 1st order:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\int_0^{1/2}x^2(t)=p\\&#xA;&amp;amp;\dot{y}=x \\&#xA;&amp;amp;2\lambda_1\dot{x}(t)+\pi cos(\pi y(t))=0\\&#xA;&amp;amp;y(0)=0,y(1/2)=1/2&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but its still not suitable for an ODE solver. Any help will be appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a known constant and &lt;span class=&quot;math-container&quot;&gt;$\lambda_1$&lt;/span&gt; is an unknown constant.&lt;br&gt;&#xA;Disclaimer: This is a cross-post and has some good answers &lt;a href=&quot;https://math.stackexchange.com/questions/3042833/system-of-odes-with-integral-constrains&quot;&gt;here on MathSE&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29498" LastEditorUserId="29498" LastEditDate="2018-12-17T19:02:43.913" LastActivityDate="2018-12-17T19:02:43.913" Title="Solution of constrained system of ODEs" Tags="|ode|constrained-optimization|numerics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30733" PostTypeId="1" AcceptedAnswerId="30735" CreationDate="2018-12-16T22:18:35.580" Score="1" ViewCount="69" Body="&lt;p&gt;In my textbook it is stated that we make a &quot;simplifying assumption&quot; &#xA;&lt;span class=&quot;math-container&quot;&gt;$$c_i = \sum_{j=1}^sa_{i,j}, $$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$c_i, a_{i,j}$&lt;/span&gt; are the constants in the butcher tableau. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the relevancy of this assumption? Is it only for explicit RK-methods? &lt;/p&gt;&#xA;" OwnerUserId="29218" LastActivityDate="2018-12-17T13:57:12.520" Title="Why is it assumed that $c_i = \sum_{j=1}^sa_{i,j}$ in the butcher tableau of a RK-method?" Tags="|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30735" PostTypeId="2" ParentId="30733" CreationDate="2018-12-17T00:43:35.613" Score="3" Body="&lt;p&gt;You want that the method produces on the autonomous version of a non-autonomous system the same values in both versions. Remember that you get the autonomous version by enlarging the state vector &lt;span class=&quot;math-container&quot;&gt;$y=(y_1,...,y_n)$&lt;/span&gt; to &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\bar y=(y_0,y_1,...,y_n)$$&lt;/span&gt; with the added equation &lt;span class=&quot;math-container&quot;&gt;$\dot y_0=\bar {f_0}(\bar y)=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y_0(t_0)=t_0$&lt;/span&gt;, that is, &lt;span class=&quot;math-container&quot;&gt;$$\frac{d\bar y}{dt}=\bar f(\bar y)=(1, f_1(y_0,y),...,f_1(y_0,y)),\\&#xA;\bar y(t_0)=(t_0, y_{1,0},...,y_{n,0}).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now insert into the Runge-Kutta mechanism to find that in the first component of the intermediate direction vectors&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\bar k_i=f(\bar y+h\sum_j a_{ij}\bar k_j)&#xA;$$&lt;/span&gt;&#xA;for the autonomous system you have &lt;span class=&quot;math-container&quot;&gt;$\bar k_{i,0}=1$&lt;/span&gt;, so that the first component of the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th intermediate point in the argument of above formula&#xA;is &lt;span class=&quot;math-container&quot;&gt;$$y_0+h\sum_ja_{ij}k_{j,0}=y_0+h\sum_ja_{ij}.$$&lt;/span&gt; This compares in the non-autonomous implementation to the evaluation of&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;k_i=f(t+hc_i,y+\sum_j a_{ij}k_j)&#xA;$$&lt;/span&gt;&#xA;to the first argument &lt;span class=&quot;math-container&quot;&gt;$t+c_ih$&lt;/span&gt;, so that for both to be equal one needs&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;c_i=\sum_ja_{ij}.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2018-12-17T13:57:12.520" LastActivityDate="2018-12-17T13:57:12.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30736" PostTypeId="1" CreationDate="2018-12-17T08:33:18.110" Score="2" ViewCount="116" Body="&lt;p&gt;Consider a nonlinear system of the form &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{f}(\boldsymbol{x}) = \boldsymbol{0}_{\mathbb{R}^n}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{x} \in \mathbb{R}^n$&lt;/span&gt;, where the function &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{f}$&lt;/span&gt; is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\boldsymbol{f}(\boldsymbol{x}) := \boldsymbol{g}(\boldsymbol{x}) + \boldsymbol{\underline{A}} \boldsymbol{x} + \boldsymbol{b} \in \mathbb{R}^n, \quad \boldsymbol{x} \in \mathbb{R}^n.&#xA;\end{equation}&lt;/span&gt;&#xA;Here, the function &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{g}$&lt;/span&gt; is differentiable. Moreover, the functions &lt;span class=&quot;math-container&quot;&gt;$g_i$&lt;/span&gt; depend only on &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;, i. e. &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial g_i}{\partial x_j} \equiv 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i \neq j$&lt;/span&gt;. The matrix &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\underline{A}} \in \mathbb{R}^{n \times n}$&lt;/span&gt; is large, sparse, non-symmetric, and invertible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we take advantage of the purely diagonal nonlinearity in &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{f}$&lt;/span&gt; somehow when solving the nonlinear system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The assembly of the Jacobian matrix &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\underline{J}_f}(\boldsymbol{x}) = \boldsymbol{\underline{J}_g}(\boldsymbol{x}) + \boldsymbol{\underline{A}}$&lt;/span&gt; is cheap since &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\underline{J}_g}(\boldsymbol{x})$&lt;/span&gt; is a diagonal matrix. Therefore, Newton's method appears to be a natural choice, but other methods are also acceptable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we use Newton's method, then I guess my question boils down to whether we can take advantage of the special form of &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\underline{J}_f}$&lt;/span&gt; when computing the updates &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\Delta x}$&lt;/span&gt; in each Newton step, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\underline{J}_f}(\boldsymbol{x}_k) \boldsymbol{\Delta x} = - \boldsymbol{f}(\boldsymbol{x}_k)$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="29495" LastActivityDate="2018-12-17T08:33:18.110" Title="Nonlinear system with diagonal nonlinearity" Tags="|nonlinear-equations|newton-method|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30737" PostTypeId="2" ParentId="30705" CreationDate="2018-12-17T14:20:37.173" Score="7" Body="&lt;p&gt;In &lt;a href=&quot;https://www.researchgate.net/publication/255789472_Recursive_formula_to_compute_Zernike_radial_polynomials&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, &#xA;Honarvar and Paramesran derive an interesting method to compute the radial Zernike polynomials in a very nice recursive way. &#xA;The recursion formula is surprisingly straightforward, without division or multiplication by large integers:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ R^m_n(\rho) = \rho &#xA;\left(R^{|m-1|}_{n-1}(\rho)+R^{m+1}_{n-1}(\rho)\right) - R^{m}_{n-2}(\rho)$$&lt;/span&gt;&#xA;I'd recommend to have a look at figure 1 in the Honarvar and Paramesran paper, which clearly illustrates the dependencies between the different Zernike polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is implemented in the following Octave script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear                                     % Tested with Octave instead of Matlab&#xA;N = 120;&#xA;n_r = 1000;&#xA;R = cell(N+1,N+1);&#xA;rho = [0:n_r]/n_r;&#xA;rho_x_2 = 2*[0:n_r]/n_r;&#xA;&#xA;R{0+1,0+1} = ones(1,n_r+1);               % R^0_0  Unfortunately zero based cell indexing is not possible&#xA;R{1+1,1+1} = R{0+1,0+1}.*rho;             % R^1_1  ==&amp;gt;  R{...+1,...+1} etc.&#xA;for n = 2:N,&#xA;    if bitget(n,1) == 0,                  % n is even&#xA;        R{0+1,n+1} = -R{0+1,n-2+1}+rho_x_2.*R{1+1,n-1+1};                % R^0_n&#xA;        m_lo = 2;&#xA;        m_hi = n-2;&#xA;    else&#xA;        m_lo = 1;&#xA;        m_hi = n-1;&#xA;    end&#xA;    for m = m_lo:2:m_hi,&#xA;        R{m+1,n+1} = rho.*(R{m-1+1,n-1+1}+R{m+1+1,n-1+1})-R{m+1,n-2+1};  % R^m_n&#xA;    end&#xA;    R{n+1,n+1} = rho.*R{n-1+1,n-1+1};                                    % R^n_n&#xA;end;&#xA;&#xA;&#xA;Z = @(m,n,rho) (-1)^((n-m)/2) * rho.^m .* jacobiPD((n-m)/2,m,0,1-2*rho.^2);&#xA;m = 22;&#xA;n = 112;&#xA;figure&#xA;plot(rho,Z(m,n,rho))&#xA;hold on&#xA;plot(rho,R{m+1,n+1},'r');&#xA;xlabel(&quot;rho&quot;)&#xA;ylabel(&quot;R^{22}_{112}(rho)&quot;)&#xA;legend(&quot;via Jacobi&quot;,&quot;recursive&quot;);&#xA;%print -djpg plt.jpg&#xA;&#xA;m = 0;&#xA;n = 46;&#xA;max_diff_m_0_n_46 = norm(Z(m,n,rho)-R{m+1,n+1},inf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example, the figure produced by this code shows that with &lt;span class=&quot;math-container&quot;&gt;$m = 22$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$n = 112$&lt;/span&gt;, catastrophic cancellation occurs near &lt;span class=&quot;math-container&quot;&gt;$\rho = 0.7$&lt;/span&gt;, if Zernike radial polynomials are computed via Jacobi polynomials. &#xA;Therefore, one also has to worry about the accuracy of the lower-degree Zernike &#xA;polynomials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cww34.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cww34.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The recursive method seems to be much more suitable for computing these higher-order Zernike polynomials in a stable way.&#xA;Nevertheless, for &lt;span class=&quot;math-container&quot;&gt;$m = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n = 46$&lt;/span&gt;, the maximum difference between the Jacobi and the recursive method is (only?) &lt;code&gt;1.4e-10&lt;/code&gt;, which might be accurate enough for your application.&lt;/p&gt;&#xA;" OwnerUserId="5286" LastEditorUserId="5286" LastEditDate="2018-12-17T18:22:42.483" LastActivityDate="2018-12-17T18:22:42.483" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="30738" PostTypeId="2" ParentId="30720" CreationDate="2018-12-17T15:45:43.760" Score="1" Body="&lt;p&gt;You are updating &lt;span class=&quot;math-container&quot;&gt;$m^2\times\mathit{nblocks}^2$&lt;/span&gt; memory locations at each iteration, 8 bytes each, a total of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; times, so you're getting memory bandwidth &lt;span class=&quot;math-container&quot;&gt;$N\times m^2\times\mathit{nblocks}^2\times 8\mathrm{B}\,/\,81\mathrm{s} = 40\mathrm{GB}/\mathrm{s}$&lt;/span&gt;. (When you disable the slow part, it's the same formula with &lt;span class=&quot;math-container&quot;&gt;$\mathit{nblocks}=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$t=4.1\mathrm{s}$&lt;/span&gt;, giving &lt;span class=&quot;math-container&quot;&gt;$82\mathrm{GB}/\mathrm{s}$&lt;/span&gt;.) This doesn't count updates to &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, so might be off by a small factor, but that doesn't matter. It seems likely to me that your code is memory-limited and that you are already within a small integer factor of max possible memory bandwidth (whatever that is on your machine).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible solution in such cases is to reduce the matrix blocks further until they fit in cache, and pay extra attention to only manipulate values that are already in cache. This is very similar to what a typical &lt;code&gt;gemm&lt;/code&gt; routine does with matrix blocks. So make the blocks smaller so that &lt;span class=&quot;math-container&quot;&gt;$8\mathrm{B}\times m^2$&lt;/span&gt; fits into your per-thread cache: if it's 256KB, that's &lt;span class=&quot;math-container&quot;&gt;$\sqrt{256\times 2^{10}/8}\sim 180$&lt;/span&gt; give or take (it's often easier to just experiment with setting this value directly by hand instead of trying to compute it on your own). For each such block, you would still accumulate &lt;span class=&quot;math-container&quot;&gt;$\alpha_{ijk}v_k v_k^\top$&lt;/span&gt; but using only a part of each &lt;span class=&quot;math-container&quot;&gt;$v_k$&lt;/span&gt;, depending on the block, so it's the &lt;code&gt;ger&lt;/code&gt; non-symmetric rank-1 update. &lt;em&gt;Note: I haven't tested this idea in code.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I think you can also save a factor of two by only doing the triangular-transpose-copy at the very end, once per block, instead of at every iteration.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2018-12-17T15:59:13.900" LastActivityDate="2018-12-17T15:59:13.900" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30739" PostTypeId="2" ParentId="22022" CreationDate="2018-12-17T17:49:37.643" Score="1" Body="&lt;p&gt;Blog post on an experimentation using Markov to recreate art:&#xA;&lt;a href=&quot;https://magenta.as/using-machine-learning-to-make-art-84df7d3bb911&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://magenta.as/using-machine-learning-to-make-art-84df7d3bb911&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code is on github made by @william-index:&#xA;&lt;a href=&quot;https://github.com/william-index/markov-fun&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/william-index/markov-fun&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29505" LastActivityDate="2018-12-17T17:49:37.643" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30742" PostTypeId="2" ParentId="28290" CreationDate="2018-12-18T05:15:58.387" Score="1" Body="&lt;p&gt;It is easy to see that the function&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;P(t) := \int \limits_0^1 p(x,t) \, \mathrm{d}x, \quad t \geq 0,&#xA;\end{equation}&lt;/span&gt;&#xA;is constant over time (i. e. &lt;span class=&quot;math-container&quot;&gt;$P(t) = P(0)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\forall \, t \geq 0$&lt;/span&gt;), if &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is a solution of your initial-boundary-value problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you discretize the problem in space as you described, then you obtain a system of linear ordinary differential equations in time of the form&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\frac{d\boldsymbol{p}}{dt} = \boldsymbol{\underline{T}} \boldsymbol{p}&#xA;\end{equation}&lt;/span&gt;&#xA;for the vector-valued function &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{p}(t) \in \mathbb{R}^{N+1}$&lt;/span&gt;, with the tridiagonal matrix &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\underline{T}} \in \mathbb{R}^{(N+1) \times (N+1)}$&lt;/span&gt; that you have specified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you define some function &lt;span class=&quot;math-container&quot;&gt;$\tilde{P}(t) := \boldsymbol{c}^{\top} \boldsymbol{p}(t)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$t \geq 0$&lt;/span&gt;, with a vector &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{c} \in \mathbb{R}^{N+1}$&lt;/span&gt;, then this function will be constant over time (i. e. &lt;span class=&quot;math-container&quot;&gt;$\tilde{P}(t) = \tilde{P}(0)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\forall \, t \geq 0$&lt;/span&gt;) if and only if &lt;span class=&quot;math-container&quot;&gt;$\frac{d\tilde{P}}{dt} = 0$&lt;/span&gt;. The time derivative of &lt;span class=&quot;math-container&quot;&gt;$\tilde{P}$&lt;/span&gt; is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\frac{d\tilde{P}}{dt} = \frac{d}{dt}(\boldsymbol{c}^{\top}\boldsymbol{p}) = \boldsymbol{c}^{\top} \frac{d\boldsymbol{p}}{dt} = \boldsymbol{c}^{\top} \boldsymbol{\underline{T}} \boldsymbol{p},&#xA;\end{equation}&lt;/span&gt;&#xA;and the right-hand side is zero for any &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{p}$&lt;/span&gt; if and only if &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{c}^{\top} \boldsymbol{\underline{T}} = \boldsymbol{0}_{\mathbb{R}^{N+1}}^{\top}$&lt;/span&gt;. For your tridiagonal matrix &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\underline{T}}$&lt;/span&gt; this is satisfied if and only if&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\boldsymbol{c}^{\top} = \left( \begin{array}{ccccc}&#xA;\frac{1}{2} &amp;amp; 1 &amp;amp; \cdots &amp;amp; 1 &amp;amp; \frac{1}{2}&#xA;\end{array}&#xA;\right) \quad \Rightarrow \quad \tilde{P} = \boldsymbol{c}^{\top} \boldsymbol{p} = \frac{1}{2} p_0 + \sum_{i=1}^{N-1} p_i + \frac{1}{2} p_N.&#xA;\end{equation}&lt;/span&gt;&#xA;This corresponds to using the trapezoidal rule for the approximation of the integral in &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;. If you calculate your &lt;span class=&quot;math-container&quot;&gt;$\tilde{P}$&lt;/span&gt; in any other way then its value will not be constant over time for this discretization.&lt;/p&gt;&#xA;" OwnerUserId="29495" LastActivityDate="2018-12-18T05:15:58.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30744" PostTypeId="1" AcceptedAnswerId="30750" CreationDate="2018-12-18T06:23:52.283" Score="0" ViewCount="2000" Body="&lt;p&gt;I am looking for the analytical solution of 1-dimensional advection-diffusion equation with Neumann boundary condition at both the inlet and outlet of a cylinder through which the fluid flow occurs.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial t} =  D\frac{\partial ^2 C}{\partial x^2}-v\frac{\partial C}{\partial x}$$&lt;/span&gt;&#xA;with initial condition&#xA;&lt;span class=&quot;math-container&quot;&gt;$$c(x,0) = C_i$$&lt;/span&gt;&#xA;and with Neumann boundary condition&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial x}=0\text{ at }t&amp;gt;0.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone suggest a reference?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had a chance to look at the answer posted &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7873/does-the-time-dependent-advection-diffusion-equation-have-an-analytical-solution&quot;&gt;here&lt;/a&gt;. Out of the solutions listed, I couldn't find the analytical solution for the transport equation with Neumann boundary condition at both the ends.&lt;/p&gt;&#xA;" OwnerUserId="29087" LastEditorUserId="713" LastEditDate="2018-12-18T11:39:38.630" LastActivityDate="2018-12-19T18:50:59.563" Title="Analytical solution of 1D advection -diffusion equation" Tags="|advection-diffusion|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30745" PostTypeId="2" ParentId="5478" CreationDate="2018-12-18T07:04:26.557" Score="3" Body="&lt;p&gt;Five years later, this question is still valid.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ecirtam.net/autoblogs/autoblogs/zythomblogspotfr_2530d81ebaf87a9d212b9c420da28b772b9e5cb4/media/445d3ada.bms-basic-NLP_120609.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; (pages 16 and 17) you can find a great explanation, including an Algorithm.&lt;/p&gt;&#xA;" OwnerUserId="29510" LastEditorUserId="29510" LastEditDate="2021-05-19T05:33:16.027" LastActivityDate="2021-05-19T05:33:16.027" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30750" PostTypeId="2" ParentId="30744" CreationDate="2018-12-19T18:50:59.563" Score="1" Body="&lt;p&gt;Given the initial conditions&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;C(x,0) = C_i \quad \forall x&#xA;$$&lt;/span&gt;&#xA;and the boundary conditions &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left . \frac{\partial C(x,t)}{\partial x} \right|_{x=0} = &#xA;\left . \frac{\partial C(x,t)}{\partial x} \right|_{x=L} = 0 \quad \forall t&#xA;$$&lt;/span&gt;&#xA;the right-hand side of the PDE vanishes at &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;,&#xA;and hence the left-hand side does as well.&#xA;So the initial concentration never evolves in time, &#xA;&lt;span class=&quot;math-container&quot;&gt;$\partial C(x,t)/\partial t=0$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;,&#xA;and the exact solution is the steady state &lt;span class=&quot;math-container&quot;&gt;$C(x,t) = C_i$&lt;/span&gt; for all &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2018-12-19T18:50:59.563" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30751" PostTypeId="2" ParentId="10003" CreationDate="2018-12-19T20:58:55.203" Score="1" Body="&lt;p&gt;You could consider Monte Carlo neutral particle transport to be mesh-free, and it's the dominant method for solving particle transport problems of that type, e.g., the Monte Carlo programs on &lt;a href=&quot;https://en.wikipedia.org/wiki/Neutron_transport&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Wikipedia page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29000" LastActivityDate="2018-12-19T20:58:55.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30753" PostTypeId="1" CreationDate="2018-12-20T11:37:44.810" Score="2" ViewCount="57" Body="&lt;p&gt;I am doing a project related to robotics where I am using &lt;code&gt;fmincon&lt;/code&gt; function from matlab to minimize the distance between the points &lt;code&gt;x_0&lt;/code&gt; and &lt;code&gt;x_f&lt;/code&gt;. Basically task is to make manipulator move from &lt;code&gt;x_0&lt;/code&gt; to &lt;code&gt;x_f&lt;/code&gt; such that it never goes out of torus region. So I think it is a constrained optimization problem. &#xA;I am having difficulty in figuring out what is the strategy to find the two intermediate points (as shown in figure below) so that given &lt;code&gt;x_0&lt;/code&gt; and &lt;code&gt;x_f&lt;/code&gt;, it gives us two points [x1,y1,z1] and [x2,y2,z2] to use as via points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please figure out how should I write the &lt;em&gt;objective&lt;/em&gt; function. &#xA;Code so far is below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    function [x, fval, history] = solver( x0, xn, R, r )&#xA;&#xA;% x0 = initial position, xn = final position, R = distance from the center of the tube to the center of the torus&#xA;% r is the radius of the tube.&#xA;% plotting torus&#xA;th=linspace(0,2*pi,60); % e.g. 60 partitions along perimeter of the tube&#xA;phi=linspace(0,pi,60); % e.g. 60 partitions along azimuth of torus&#xA;[Phi,Th]=meshgrid(phi,th);&#xA;x=0+(R+r.*cos(Th)).*cos(Phi); &#xA;y=0+(R+r.*cos(Th)).*sin(Phi);&#xA;z=0+r.*sin(Th);&#xA;s=surface(x,y,z);&#xA;s.EdgeColor='none'&#xA;axis equal;&#xA;daspect([1 1 1 ])&#xA;hold off &#xA;alpha(s,0.2);&#xA;% defining constraints and objective function&#xA;history = [];&#xA;options = optimset('OutputFcn', @myoutput);&#xA;[x, fval] = fmincon(@(x)objfun(x,xn),x0,[],[],[],[],[],[],@noncol,options);&#xA;&#xA;    function stop = myoutput(x,optimvalues,state)&#xA;        stop = false;&#xA;        if isequal(state,'iter')&#xA;            history = [history; x];&#xA;        end&#xA;    end&#xA;&#xA;    function z = objfun(x,p)   &#xA;        z = sqrt((p(1)-x(1))^2+(p(2)-x(2))^2+(p(3)-x(3))^2);&#xA;    end&#xA;    function [y, yeq] = noncol( x ) % x ==== [ x(1)   x(2)    x(3) ]&#xA;        y = (sqrt(x(1)^2+x(2)^2)-R)^2 + x(3)^2-r^2; &#xA;        yeq =[];&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yJnX7.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yJnX7.jpg&quot; alt=&quot;Aim is to go from &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$x_f$&lt;/span&gt; within this toroid region&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29526" LastActivityDate="2018-12-20T11:37:44.810" Title="How to find two points within defined region in this constrained optimization problem?" Tags="|matlab|constrained-optimization|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30755" PostTypeId="2" ParentId="28061" CreationDate="2018-12-21T05:00:21.857" Score="5" Body="&lt;p&gt;Regarding point 1: generally, applying a discretized derivative (stencil) twice is not equivalent to applying an equivalent (i.e. same number of points) stencil for the second derivative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example in matrix representation: A three point stencil for the first derivative would correspond to a tridiagonal matrix. The same is true for the three point stencil for the second derivative. Taking the square of the matrix representing the first derivative leads to a matrix where also the second sub-diagonal is non-zero (the corresponding five-point form is different from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Five-point_stencil&quot; rel=&quot;nofollow noreferrer&quot;&gt;five-point stencil&lt;/a&gt; obtained via Taylor expansion and  thus, unlike the latter, does not have the (generally optimal) &lt;span class=&quot;math-container&quot;&gt;$O\!\left({\rm gridspacing}^4\right)$&lt;/span&gt; error for the second derivative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason for squaring these discretized representations of the first derivative not being equivalent to the direct discretization of the second derivative is that the corresponding basis set is incomplete. Only in the limit of infinitely small grid spacing between the stencil points will the square of the approximated first derivative equal the second derivative approximation. Infinitely small grid spacing also corresponds to a complete basis for (the considered interval of) &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An exception to this incomplete basis set problem are &lt;a href=&quot;https://en.wikipedia.org/wiki/Chebyshev_polynomials#As_a_basis_set&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chebyshev polynomial basis sets&lt;/a&gt;: here the second derivative is equivalent to squaring the first derivative operator in the Chebyshev basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding point 3: Since &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; depends on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, I don't follow the steps presented (assuming constant media would mean &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is a constant and not &lt;span class=&quot;math-container&quot;&gt;$a(x)$&lt;/span&gt;?).&#xA;I would transform according to the product rule for differentiation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial}{\partial x}\left[a(x)\frac{\partial}{\partial x}\right] \psi = \frac{\partial a(x)}{\partial x} \frac{\partial \psi}{\partial x} + a(x) \frac{\partial^2 \psi}{\partial x^2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The representations of &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial}{\partial y}$&lt;/span&gt; can be applied in any order (the corresponding matrices commute). Care only has to be taken in the case of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; being the same variable as discussed above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a specific example for a differential operator in real space representation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In real space, the three point stencil would correspond to the first derivative operator&#xA;&lt;span class=&quot;math-container&quot;&gt;$$D_{ij} = -\frac{1}{2} \delta_{i,j+1} + \frac{1}{2} \delta_{i,j-1}.$$&lt;/span&gt;&#xA;Taking the square&#xA;&lt;span class=&quot;math-container&quot;&gt;$$D_{ij}^2 = \frac{1}{4} \delta_{i,j+2} - \frac{1}{2} \delta_{i,j} + \frac{1}{4} \delta_{i,j-2}.$$&lt;/span&gt;&#xA;This is different from (and a worse approximation than) starting from Taylor expansion and using the three point stencil for the second derivative directly:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$D^{\rm second}_{ij} = \delta_{i,j+1} - 2 \delta_{i,j} + \delta_{i,j-1}.$$&lt;/span&gt;&#xA;Was the grid spacing infinitely small (meaning making the basis set complete), &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; would go over to the analytical differential operator, and applying &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; twice and second derivative would be identical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, generally, if the basis set in which &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is represented is finite (or really incomplete), repeated application of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is not equivalent to creating higher differential operators with the same technique as used for &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; in this finite basis set. In a complete basis set (infinitely small grid spacing limit as an example), repeated application of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is equivalent to higher order derivative operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final example, there can also be finite basis sets, in which powers of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; are identical to higher derivatives. In Fourier space, e.g., derivatives simply become multiplication with the frequency variable &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; and thus repeated multiplication by &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; is identical to higher derivatives, no matter how many values of &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; are included in the finite basis.&lt;/p&gt;&#xA;" OwnerUserId="29530" LastEditorUserId="29530" LastEditDate="2019-01-24T18:42:17.780" LastActivityDate="2019-01-24T18:42:17.780" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30756" PostTypeId="2" ParentId="10582" CreationDate="2018-12-22T08:48:10.163" Score="1" Body="&lt;p&gt;&lt;strong&gt;Truncated Neumann series&lt;/strong&gt;: let &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; be the diagonal of your matrix, and &lt;span class=&quot;math-container&quot;&gt;$-N$&lt;/span&gt; be the non-diagonal part. Then, &lt;span class=&quot;math-container&quot;&gt;$M=D-N=D(I-D^{-1}N)$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$$M^{-1}=(I-D^{-1}N)D^{-1}=(I+D^{-1}N + D^{-1}ND^{-1}N + D^{-1}ND^{-1}ND^{-1}N + \dots)D^{-1}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Truncate the infinite sum where you prefer. Note that &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; contain non-negative elements, so what you get is guaranteed to be an approximation from below.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2018-12-22T08:48:10.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30757" PostTypeId="1" CreationDate="2018-12-22T10:27:05.030" Score="4" ViewCount="3008" Body="&lt;p&gt;I need to use a Discrete-time Algebraic Riccati Equation (DARE) solver for an embedded controller (with limited processing power) in a research project and sadly, I can't find any implementation of it in C++ that I can understand. I use C++ since it is faster than higher-level languages.&lt;/p&gt;&#xA;&lt;p&gt;I found various libraries online that offer Riccati equation or LQR solvers such as the &lt;a href=&quot;https://adrlab.bitbucket.io/ct/v2.3/ct_doc/doc/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Control Toolbox&lt;/a&gt; or &lt;a href=&quot;https://drake.mit.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drake&lt;/a&gt;, but I can't understand the language and I think they are full of unnecessary functions and things for the simple job of solving DARE.&lt;/p&gt;&#xA;&lt;p&gt;I would like the implementation to use basic open-source C++ libraries or header files such as Eigen, Armadillo, etc. with a lot of documentation, tutorials, or an active online community using them so that I can learn the syntax on my own. I'm looking for a simple code like Arash's C++ implementation of the Continuous-time Algebraic Riccati Equation (CARE) solver on &lt;a href=&quot;https://math.stackexchange.com/questions/679989/analytic-solution-to-structured-algebraic-riccati-equatio&quot;&gt;Math SE&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you don't have a personal C++ code, can you please refer me to a library or something that can help me implement DARE in C++? I attempted to write my own solver but when I read papers of the Riccati solver, I was dumbfounded by the math and terminologies since I am an undergraduate engineering student with only basic knowledge in linear algebra.&lt;/p&gt;&#xA;" OwnerUserId="29538" LastEditorUserId="20688" LastEditDate="2021-12-28T22:28:37.980" LastActivityDate="2021-12-28T22:28:37.980" Title="Discrete-time Algebraic Riccati Equation (DARE) solver in C++" Tags="|algorithms|nonlinear-equations|quadratic-programming|optimal-control|discrete-optimization|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30761" PostTypeId="2" ParentId="10003" CreationDate="2018-12-22T22:23:41.510" Score="0" Body="&lt;p&gt;Autodyn (SPH) is pretty much the standard for simulating ballistics/impacts/explosions. Used a lot in the industry.&lt;/p&gt;&#xA;" OwnerUserId="471" LastActivityDate="2018-12-22T22:23:41.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30762" PostTypeId="1" CreationDate="2018-12-22T22:41:51.483" Score="4" ViewCount="69" Body="&lt;p&gt;I am in need to optimize the performance of the 3D solver that integrates a system of equations for seismic wave propagation.&#xA;Unsurprisingly, the function that implements the finite-difference stencil computations, is the bottleneck of the computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.407.3398&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, where in Section 3.2 the authors say that the register blocking allowed them to obtain substantial speedup of the 3D stencil computations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not understand how to implement this technique.&#xA;Is it possible to find somewhere the description of register blocking in clear form?&lt;/p&gt;&#xA;" OwnerUserId="3254" LastActivityDate="2018-12-22T22:41:51.483" Title="How to implement register blocking for 3D finite-difference stencil computations" Tags="|finite-difference|hpc|3d|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30764" PostTypeId="1" CreationDate="2018-12-23T21:49:07.727" Score="4" ViewCount="75" Body="&lt;p&gt;Start with 1d case. Say I have lots of 1d intervals &lt;span class=&quot;math-container&quot;&gt;$[s_i, e_i]$&lt;/span&gt; and I want to find an interval &lt;span class=&quot;math-container&quot;&gt;$[s^*, e^*]$&lt;/span&gt; to maximise the count of interval &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$[s_i, e_i]\supseteq [s^*, e^*]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1d case is easy. All I need to do is to sort the &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$e_i$&lt;/span&gt; together, eg &lt;span class=&quot;math-container&quot;&gt;$[s_1, s_2, e_1, s_3, e_3, e_2]$&lt;/span&gt;. Then go from left to right, &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; if it's &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; if it's &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;. And locate the maximum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I deal with the case in 3d? Here intervals are 3d balls with L1 distance from a point &lt;span class=&quot;math-container&quot;&gt;$\{(x,y,z): |x-x_i|+|y-y_i|+|z-z_i|\leq d_i\}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can cast it as binary linear programming, but I don't think it's solvable with 1000 binary variables plus 6000 auxiliary variables to deal with absolute value. I wonder if there is a polynomial time solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ps: I don't need the entire region, any point inside the max region is good.&lt;/p&gt;&#xA;" OwnerUserId="14305" LastEditorUserId="14305" LastEditDate="2018-12-24T13:15:20.027" LastActivityDate="2018-12-26T14:26:51.917" Title="Given a list of intervals, find region that is contained by the largest number of those intervals" Tags="|optimization|interval-arithmetic|discrete-optimization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30765" PostTypeId="1" AcceptedAnswerId="30774" CreationDate="2018-12-24T10:14:35.897" Score="0" ViewCount="79" Body="&lt;p&gt;I found an old lecture on YouTube given by Walter Rudin (1990, in Wisconsin), and towards the beginning he mentions that if &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; were not integrable, on some interval, it would be obvious that it doesn't have a Fourier Series on that interval, since if you looked at the Fourier coefficient given by &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{1}{\pi}\int_0^\pi f(x) \cos(nx) dx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it wouldn't be meaningful, given that &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is not integrable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was a bit confused because I'm wondering whether the &lt;strong&gt;product&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$f(x)\cos(nx)$&lt;/span&gt; could be integrable, and so one could indeed compute the coefficients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: if &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; is not integrable, how do we know that the product &lt;span class=&quot;math-container&quot;&gt;$f(x)\cos(nx)$&lt;/span&gt; is not integrable?&lt;/p&gt;&#xA;" OwnerUserId="29552" LastEditorUserId="20688" LastEditDate="2018-12-24T13:17:27.197" LastActivityDate="2018-12-25T21:16:30.343" Title="Rudin lecture -- if f(x) is not integrable on some interval, does it not have a Fourier Series expansion on that interval?" Tags="|fourier-analysis|integration|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30766" PostTypeId="2" ParentId="30765" CreationDate="2018-12-24T13:05:55.997" Score="2" Body="&lt;p&gt;Applying Cauchy-Schwartz inequaility, we have:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{aligned}&#xA;\left(\int_{0}^{\pi}f(x)\cos{(x)}\,dx \right)^2&amp;amp;\leq \left(\int_{0}^{\pi}|f(x)\cos{(x)}|\,dx \right)^2\\ &amp;amp;\leq \int_{0}^{\pi}f(x)^2\,dx \int_{0}^{\pi}\cos{(x)}^2\,dx=\frac{\pi}{2}\int_{0}^{\pi}f(x)^2\,dx &#xA;\end{aligned}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if &lt;span class=&quot;math-container&quot;&gt;$f(x)\notin {L}^2(x)$&lt;/span&gt; then the Fourier coefficients are not bounded.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastEditorUserId="20688" LastEditDate="2018-12-24T14:46:40.733" LastActivityDate="2018-12-24T14:46:40.733" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30767" PostTypeId="1" AcceptedAnswerId="30770" CreationDate="2018-12-24T18:00:36.710" Score="1" ViewCount="63" Body="&lt;p&gt;I need to fix a code to utilise the &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; stage multistep method :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{n+2} - y_n = h\left[(1/3)f_{n+2} + (4/3)f_{n+1} + (1/3)f_n\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this is an implicit method, I used a Newton-Raphson approach for the final determination of &lt;span class=&quot;math-container&quot;&gt;$y_{n+2}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below, is my attempt at a code implementing the already given rk4 method (runge-kutta 4) for the first approximation steps :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function [tout, yout] = newcorrect(FunFcn,t0,tfinal,step,y0)&#xA;maxiter = 1000;&#xA;tolnr   = 1e-9;&#xA;diffdelta = 1e-6;&#xA;stages=2;&#xA;[tout,yout]=rk4(FunFcn,t0,t0+(stages-1)*step,step,y0);&#xA;tout=tout(1:stages);&#xA;yout=yout(1:stages);&#xA;t = tout(stages);&#xA;y = yout(stages).';&#xA;% The main loop&#xA; while abs(t- tfinal)&amp;gt; 1e-6 &#xA;    if t + step &amp;gt; tfinal, step = tfinal - t; end&#xA;    t = t + step;&#xA;    yn0 = y;&#xA;    ynf = yn0;&#xA;    yn = inf;&#xA;    iter = 0;&#xA;    while (abs(yn - ynf)&amp;gt;= tolnr) &amp;amp;&amp;amp; (iter &amp;lt; maxiter)&#xA;        df = 1/diffdelta * (feval(FunFcn,t, yn0+diffdelta) - feval(FunFcn, t, yn0));&#xA;        yn = yn0 - 1/(1/3*step*df - 1) * (4/3*step*feval(FunFcn,tout(end),yout(end)) + 1/3*step*feval(FunFcn,tout(end-1),yout(end-1)) + 1/3*step*feval(FunFcn, t, yn0) -yn0 + yout(end-1));&#xA;        ynf = yn0;&#xA;        yn0 = yn;&#xA;        iter = iter + 1;&#xA;    end&#xA;    y = yn;&#xA;    tout = [tout; t];&#xA;    yout = [yout; y.'];&#xA; end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is to be used for showing experimentally that when you divide the step by two, the fraction of the maximum absolute errors per consecutive different steps &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is approximately equal to &lt;span class=&quot;math-container&quot;&gt;$2^{-p}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the order of the method. The method is proven to be of order &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;. The exercise whishes the initial starting point to be &lt;span class=&quot;math-container&quot;&gt;$k0=2$&lt;/span&gt; thus I created the following script :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all;&#xA;t0 = 1;&#xA;tfinal = 3;&#xA;y1 = 2;&#xA;tout  =t0:0.01:tfinal;&#xA;k0 = 2;&#xA;kf = input('enter final k:')&#xA;for k = k0:kf&#xA;  h(k-1) = 2^(-k);&#xA;  [tout,yout] = newcorrect('f0', t0, tfinal, h(k-1), y1);&#xA;  outputs{k-1} = [tout,yout];&#xA;end&#xA;for i = 1:(kf-1)&#xA;   maxabserror(i) = max(abs(outputs{1,i}(:,2)-f0true(outputs{1,i}(:,1))));&#xA;end&#xA; for i = 1:(kf-2)&#xA;   consmax(i) = maxabserror(i+1)/maxabserror(i);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The function f0 and f0true are :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function yprime = f0(t,y)&#xA;yprime = (t.^2 + y.^2)/(2.*t.*y);&#xA;end&#xA;&#xA;function y = f0true(t);&#xA;y = sqrt(t.*(t+3));&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run the method, though, the problem is that despite providing very good approximations for the solution values, the experimental conclusion of the order cannot be carried out (the fraction vary and do not converge to &lt;span class=&quot;math-container&quot;&gt;$2^{-p}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;**After running the script for &lt;span class=&quot;math-container&quot;&gt;$kf = 10$&lt;/span&gt;, I get the consecutive absolute max errors fraction vector :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\textbf{consmax} = \big[0.0704, \; 0.0657, \; 0.0639, \; 0.0632, \; 0.0628, \; 0.0627, \; 0.0656, \; 0.2400\big] $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it is easy to see, every term of it tends to &lt;span class=&quot;math-container&quot;&gt;$1/16 = 2^{-4}$&lt;/span&gt; which indeed is what we want as &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; is the order of the method. &lt;strong&gt;BUT&lt;/strong&gt; what is it going on with that last value ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Where is my mistake ? Is this something that can be explained ?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="29547" LastEditorUserId="29547" LastEditDate="2018-12-24T18:38:29.580" LastActivityDate="2018-12-24T23:50:57.400" Title="Wrong results for $2$ stage multistep method $y_{n+2} - y_n = h\left[(1/3)f_{n+2} + (4/3)f_{n+1} + (1/3)f_n\right]$" Tags="|matlab|ode|iterative-method|numerics|differential-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30769" PostTypeId="2" ParentId="30765" CreationDate="2018-12-24T21:36:05.250" Score="4" Body="&lt;p&gt;The point is to &lt;em&gt;guarantee&lt;/em&gt; that the Fourier coefficients exist, not to make really fine distinctions about what might happen if the guarantee is violated. Furthermore, you would be hard pressed to find a reasonable function for which this is an issue, if it exists at all. The class of integrable functions is already so wide as to include everything you might want to compute the Fourier coefficients of anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statement as you quoted it doesn’t seem right because &lt;span class=&quot;math-container&quot;&gt;$1/\cos x$&lt;/span&gt; isn’t integrable but has some finite Fourier coefficients.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2018-12-24T21:36:05.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30770" PostTypeId="2" ParentId="30767" CreationDate="2018-12-24T23:50:57.400" Score="4" Body="&lt;p&gt;The local error in each integration step is composed of 3 parts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the discretization error of the method&lt;/li&gt;&#xA;&lt;li&gt;the floating point error of actually performing the method in precision-limited data types, and&lt;/li&gt;&#xA;&lt;li&gt;specifically for implicit methods, the error in solving the implicit equation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The first error is of size &lt;span class=&quot;math-container&quot;&gt;$O(h^{p+1})$&lt;/span&gt;, the second of size &lt;span class=&quot;math-container&quot;&gt;$\sim \mu$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; the machine constant of the number type. The last error depends on the implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually one uses a predictor-corrector scheme with an explicit method step as predictor, often of the same order &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; so that ensuing corrector steps towards the solution of the implicit equation only influence the coefficient of the leading power in &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; of the error. Here one easy non-trivial predictor is the midpoint step &lt;span class=&quot;math-container&quot;&gt;$y_{n+1}^{[0]}=y_{n-1}+2hf(t_n,y_n)$&lt;/span&gt; with error &lt;span class=&quot;math-container&quot;&gt;$O(h^3)$&lt;/span&gt;.&#xA;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every Newton-like corrector implementation (like the simplified Newton method) will reduce that error by a factor of &lt;span class=&quot;math-container&quot;&gt;$\approx Ch^2$&lt;/span&gt; so that in general after the first corrector iteration the order of the method is reached. If &lt;span class=&quot;math-container&quot;&gt;$Ch^2\ll1$&lt;/span&gt;, the update of the corrector step may be taken as proxy for the remaining distance to the exact solution, so instead of a fixed number of 1 to 3 corrector iterations one can also stop the corrector loop if the update is smaller than for example &lt;span class=&quot;math-container&quot;&gt;$0.1\,h^4\|y_{n+1}^{[0]}-y_n\|$&lt;/span&gt; (it is &lt;span class=&quot;math-container&quot;&gt;$O(h^5)$&lt;/span&gt; and scales with the scaling of the problem).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, setting this last tolerance independent of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; will introduce a lower bound for the local and thus global error. You used the previous value as predictor, &lt;span class=&quot;math-container&quot;&gt;$y_{n+1}^{[0]}=y_n$&lt;/span&gt;. This has error &lt;span class=&quot;math-container&quot;&gt;$O(h)$&lt;/span&gt;. One Newton step is always performed, so that the first error that is really compared to &lt;code&gt;tolnr   = 1e-9;&lt;/code&gt; has magnitude &lt;span class=&quot;math-container&quot;&gt;$O(h^3)$&lt;/span&gt;. A second step down to error &lt;span class=&quot;math-container&quot;&gt;$O(h^5)$&lt;/span&gt; and thus the order of the method is not performed if &lt;span class=&quot;math-container&quot;&gt;$h^3\lesssim 10^{-9}\implies h\lesssim 10^{-3}\sim 2^{-10}$&lt;/span&gt;, which is about what you observed.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2018-12-24T23:50:57.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30772" PostTypeId="1" CreationDate="2018-12-25T14:18:09.730" Score="2" ViewCount="132" Body="&lt;p&gt;I have an ultrasonic image represented by a point cloud as shown below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TTcv5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TTcv5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I need to extract features from this image. These features can be easily recognised by a human brain since they form continuous lines/curves. I am currently using a modified density based clustering algorithm to extract the features. A result is shown below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GNEmB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GNEmB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;where points inside those black borders are extracted.&lt;/p&gt;&#xA;&lt;p&gt;However, the result is not perfect. If I zoom in at one of the clusters, you can see the points actually form two curves/lines in the cluster, one inside the red rectangle and the other inside the blue rectangle.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XIDTd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XIDTd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For this image, I can apply some different parameters for db-clustering algorithm locally to sub-divide the cluster but if I were to process hundreds of images, this became impossible.&lt;/p&gt;&#xA;&lt;p&gt;Hence, I am asking if there is any algorithm that can recognise the lines/curves in a point cloud like such.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;Update:&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://1drv.ms/u/s!ArdKO5IJqtSdgfdb9-q6qbym_tasnQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to sample data&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29559" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2018-12-27T02:17:56.037" Title="Find points that form curve segments in a point cloud" Tags="|image-processing|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30773" PostTypeId="1" CreationDate="2018-12-25T18:49:37.053" Score="4" ViewCount="79" Body="&lt;p&gt;Does there exist a library (in any programming language) for solving (numerically) systems of multidimensional first-order linear PDEs in the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{u}_{t}+\hat{A}(\mathbf{x})\mathbf{u}_{\mathbf{x}}=\mathbf{f}(\mathbf{u}),$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathbf{u}(\mathbf{x},0)=\mathbf{g}(\mathbf{x}),$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{u}=(u_1(\mathbf{x}),\ldots,u_m(\mathbf{x})),\mathbf{x}=(x_1,\ldots,x_n),\mathbf{f}(\mathbf{u})=(f_{1}(\mathbf{u}),\ldots,f_{m}(\mathbf{u})),$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat{A}(\mathbf{x})$&lt;/span&gt; is a matrix of variable coefficients?&lt;/p&gt;&#xA;" OwnerUserId="29560" LastEditorUserId="29560" LastEditDate="2018-12-30T23:18:32.210" LastActivityDate="2018-12-30T23:18:32.210" Title="Library for solving multidimensional (n &gt; 3) hyperbolic PDE systems" Tags="|hyperbolic-pde|libraries|numerics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30774" PostTypeId="2" ParentId="30765" CreationDate="2018-12-25T21:16:30.343" Score="1" Body="&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is not integrable, then the first fourier coefficient obtained by considering &lt;span class=&quot;math-container&quot;&gt;$n=0$&lt;/span&gt; (i.e., the mean) is not defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other Fourier coefficients are likely to be undefined as well (but some could very well exist, as Kirill already pointed out). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, the existence of the Fourier Series for such a function is doomed.&lt;/p&gt;&#xA;" OwnerUserId="28007" LastActivityDate="2018-12-25T21:16:30.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30775" PostTypeId="2" ParentId="25777" CreationDate="2018-12-25T21:32:56.800" Score="0" Body="&lt;p&gt;It's been a while since this question was posted and looking at it again, I have some new insights into tackling the problem. Let us first define the set &lt;span class=&quot;math-container&quot;&gt;$V = \lbrace V_1, V_2, \cdots, V_k \rbrace$&lt;/span&gt; as the set of possible key values we care to consider. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us also assume the time series data is sorted in time so we can model it as some sequence &lt;span class=&quot;math-container&quot;&gt;$D(n): \mathbb{N}^{+} \rightarrow \mathbb{R}$&lt;/span&gt; where we have &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; pieces of data. Let us also compute the prefix sum of the time series and denote it &lt;span class=&quot;math-container&quot;&gt;$p(n) = \sum_{k=1}^n D(k) = p(n-1) + D(n)$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$p(1) = D(1)$&lt;/span&gt;, which can be computed in &lt;span class=&quot;math-container&quot;&gt;$O(M)$&lt;/span&gt; time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will now compute the minimum error, using the key set, for some subset of the data ranging from the indices &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in the following manner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;e(i,j) &amp;amp;= \min_{v \in V} d\left(p(j) - p(i) + D(i), (j-i+1) v \right)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$d(u,v)$&lt;/span&gt; is some metric, perhaps &lt;span class=&quot;math-container&quot;&gt;$d(u,v) = |u-v|$&lt;/span&gt;. This above computation must be done for all &lt;span class=&quot;math-container&quot;&gt;$1 \leq i &amp;lt; j$&lt;/span&gt;, so this computation is ultimately &lt;span class=&quot;math-container&quot;&gt;$O\left(k M^2\right)$&lt;/span&gt;. With this information, we then wish to try and segment the data in a manner that minimizes the error across each segment of the data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us define some positive scalar &lt;span class=&quot;math-container&quot;&gt;$\tau \in \mathbb{R}$&lt;/span&gt; that we will use to softly penalize each segment we generate, meaning that the larger &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is, the less segments we will strive to have to balance that penalty with the error of the segments. Using this, we can write the following recursive relationship:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f(j) = \min_{1 \leq i \leq j} \left(e(i,j) + \tau + f(i-1)\right)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$f(0) = 0$&lt;/span&gt;. It is trivial to see we can find &lt;span class=&quot;math-container&quot;&gt;$f(j)$&lt;/span&gt; in increasing order of &lt;span class=&quot;math-container&quot;&gt;$j=1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$j=M$&lt;/span&gt; where each minimization will take &lt;span class=&quot;math-container&quot;&gt;$O(M)$&lt;/span&gt; computation, making the computation of &lt;span class=&quot;math-container&quot;&gt;$f(j)$&lt;/span&gt; an &lt;span class=&quot;math-container&quot;&gt;$O\left(M^2\right)$&lt;/span&gt; process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To back out the segment indices and keys associated with each segment, we can use the following algorithm:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;S = \emptyset\\&#xA;&amp;amp;j^* \leftarrow M\\&#xA;&amp;amp;\textbf{while}(j^* \geq 1)\\&#xA;&amp;amp;\hspace{0.5cm} \cdot i^* \leftarrow \arg \min_{1 \leq i \leq j^*} \left(e(i,j^*) + \tau + f(i-1)\right)\\&#xA;&amp;amp;\hspace{0.5cm} \cdot v^* \leftarrow \arg \min_{v \in V} d\left(p(j^*) - p(i^*) + D(i^*), (j^* - i^* + 1) v \right)\\&#xA;&amp;amp;\hspace{0.5cm} \cdot S \leftarrow S \cup \left\lbrace \left(i^*,\; j^*,\; v^*\right) \right\rbrace\\&#xA;&amp;amp;\hspace{0.5cm} \cdot j^* \leftarrow \left(i^* - 1\right) \\&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is also at worst case &lt;span class=&quot;math-container&quot;&gt;$O\left(M^2 + k M\right)$&lt;/span&gt;. Looking at the overall algorithm, we can see that the time complexity is &lt;span class=&quot;math-container&quot;&gt;$O \left(k M^2 \right)$&lt;/span&gt; and the space complexity is &lt;span class=&quot;math-container&quot;&gt;$O \left( M^2 \right)$&lt;/span&gt;. In this algorithm, &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; can be tuned to help impact how large of segments you'll allow, though this may take some experimentation.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To validate that this algorithm described above works, below is a code to perform the computation in C++. The primary source code for the algorithm is shown below. I should note that to minimize space requirements down to &lt;span class=&quot;math-container&quot;&gt;$O(M)$&lt;/span&gt;, I slightly modified how some of the above quantities are stored and computed such that the time complexity is still &lt;span class=&quot;math-container&quot;&gt;$O\left(k M^2 \right)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;stdint.h&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;limits.h&amp;gt;&#xA;&#xA;namespace opt {&#xA;&#xA;    struct segment {&#xA;        segment(size_t s, size_t e, double k):start(s),end(e),key(k) {}&#xA;        size_t start, end;&#xA;        double key;&#xA;        void print() const {&#xA;            std::cout &amp;lt;&amp;lt; &quot;(&quot; &amp;lt;&amp;lt; start &amp;lt;&amp;lt; &quot;, &quot; &amp;lt;&amp;lt; end &amp;lt;&amp;lt; &quot;, &quot; &amp;lt;&amp;lt; key &amp;lt;&amp;lt; &quot;)&quot; &amp;lt;&amp;lt; std::endl;&#xA;        }&#xA;    };&#xA;&#xA;    using segment_list = std::vector&amp;lt;struct segment&amp;gt;;&#xA;&#xA;    void comp_prefixsum(const std::vector&amp;lt;double&amp;gt;&amp;amp; D,&#xA;                        std::vector&amp;lt;double&amp;gt;&amp;amp; p)&#xA;    {&#xA;        p[0] = D[0];&#xA;        for(size_t i = 1; i &amp;lt; D.size(); ++i){ p[i] = p[i-1] + D[i]; }&#xA;    }&#xA;&#xA;    double err_metric(double u, double v){&#xA;        return std::abs(u - v);&#xA;    }&#xA;&#xA;    double min_error(size_t i, size_t j,&#xA;                     const std::vector&amp;lt;double&amp;gt;&amp;amp; D,&#xA;                     const std::vector&amp;lt;double&amp;gt;&amp;amp; p,&#xA;                     const std::vector&amp;lt;double&amp;gt;&amp;amp; V )&#xA;    {&#xA;        double min_err = std::numeric_limits&amp;lt;double&amp;gt;::max();&#xA;        for(auto v: V){&#xA;            double err = err_metric(p[j] - p[i] + D[i],&#xA;                                    static_cast&amp;lt;double&amp;gt;(j-i+1)*v );&#xA;            if( err &amp;lt; min_err ){ min_err = err; }&#xA;        }// end for v&#xA;&#xA;        return min_err;&#xA;    }&#xA;&#xA;    double argmin_error(size_t i, size_t j,&#xA;                     const std::vector&amp;lt;double&amp;gt;&amp;amp; D,&#xA;                     const std::vector&amp;lt;double&amp;gt;&amp;amp; p,&#xA;                     const std::vector&amp;lt;double&amp;gt;&amp;amp; V )&#xA;    {&#xA;        double argmin_key, min_err = std::numeric_limits&amp;lt;double&amp;gt;::max();&#xA;        for(auto v: V){&#xA;            double err = err_metric(p[j] - p[i] + D[i],&#xA;                                    static_cast&amp;lt;double&amp;gt;(j-i+1)*v );&#xA;            if( err &amp;lt; min_err ){ min_err = err; argmin_key = v; }&#xA;        }// end for v&#xA;&#xA;        return argmin_key;&#xA;&#xA;    }&#xA;&#xA;    void segmentation(const std::vector&amp;lt;double&amp;gt;&amp;amp; time_series,&#xA;                      const std::vector&amp;lt;double&amp;gt;&amp;amp; key_set,&#xA;                      double tau,&#xA;                      std::vector&amp;lt;segment&amp;gt;&amp;amp; soln_segmentation )&#xA;    {&#xA;        // compute size of data&#xA;        size_t M = time_series.size();&#xA;&#xA;        // allocate data arrays&#xA;        std::vector&amp;lt;double&amp;gt; p(M,0.0), f(M,0.0);&#xA;        std::vector&amp;lt;size_t&amp;gt; arg_fmin(M, 0);&#xA;&#xA;        // compute prefix sum&#xA;        comp_prefixsum(time_series, p);&#xA;&#xA;        // compute sub problem values&#xA;        for(size_t j = 0; j &amp;lt; M; ++j){&#xA;            double ft, f_min = std::numeric_limits&amp;lt;double&amp;gt;::max();&#xA;            for(size_t i = j; i != -1; --i){&#xA;&#xA;                // compute minimum key-based error&#xA;                double e_ij = min_error(i, j, time_series, p, key_set);&#xA;&#xA;                // compute recurrence value&#xA;                if( i != 0 ){   ft = e_ij + tau + f[i-1]; }&#xA;                else{           ft = e_ij + tau; }&#xA;&#xA;                // update minimum recurrence value, if necessary&#xA;                // and the argmin index of this optimization&#xA;                if( ft &amp;lt;= f_min ){&#xA;                    f_min = ft;&#xA;                    f[j]  = f_min;&#xA;                    arg_fmin[j] = i;&#xA;                }&#xA;&#xA;            }// end for i&#xA;        }// end for j&#xA;&#xA;        // compute segmentation solution&#xA;        soln_segmentation.clear();&#xA;        soln_segmentation.reserve(M);&#xA;        size_t j = M-1;&#xA;        while(j != -1){&#xA;            size_t i_opt = arg_fmin[j];&#xA;            double v_opt = argmin_error(i_opt, j, time_series, p, key_set);&#xA;            soln_segmentation.emplace_back(i_opt, j, v_opt);&#xA;            j = i_opt - 1;&#xA;        }// end while&#xA;&#xA;    }&#xA;} // end namespace opt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A sample code to test that the above algorithms work is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int main(int argc, const char** argv) {&#xA;&#xA;    // initialize the data&#xA;    std::vector&amp;lt;double&amp;gt; data { 6, 6, 4, 4, 4, 4, 4, 1, 1, 1, 1 };&#xA;    std::vector&amp;lt;double&amp;gt; keys { 1, 4, 6 };&#xA;&#xA;    // declare the output segment solution list&#xA;    typename opt::segment_list segments;&#xA;&#xA;    // define the tuning tau variable&#xA;    double tau = 0;&#xA;&#xA;    // compute the optimal segmentation&#xA;    opt::segmentation(data, keys, tau, segments);&#xA;&#xA;    // print the segmentation results based on&#xA;    // 3-tuple defined in mathematical formulation&#xA;    std::cout &amp;lt;&amp;lt; &quot;Print segmentation 3-tuples:&quot; &amp;lt;&amp;lt; std::endl;&#xA;    size_t num_segs = segments.size();&#xA;    for(size_t i = (num_segs-1); i != -1; --i){&#xA;        segments[i].print();&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; std::endl;&#xA;&#xA;    // print the actual segmentations&#xA;    std::cout &amp;lt;&amp;lt; &quot;Print actual segmentations:&quot; &amp;lt;&amp;lt; std::endl;&#xA;    size_t count = 0;&#xA;    for(size_t i = (num_segs-1); i != -1; --i){&#xA;        std::cout &amp;lt;&amp;lt; &quot;s&quot; &amp;lt;&amp;lt; count &amp;lt;&amp;lt; &quot; = [ &quot;;&#xA;        for(size_t j = segments[i].start; j &amp;lt;= segments[i].end; ++j){&#xA;            std::cout &amp;lt;&amp;lt; &quot;(&quot; &amp;lt;&amp;lt; j &amp;lt;&amp;lt; &quot;, &quot; &amp;lt;&amp;lt; data[j] &amp;lt;&amp;lt; &quot;) &quot;;&#xA;        }&#xA;        std::cout &amp;lt;&amp;lt; &quot; ]&quot; &amp;lt;&amp;lt; std::endl;&#xA;        ++count;&#xA;    }&#xA;&#xA;    // finish work&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code prints the &lt;span class=&quot;math-container&quot;&gt;$(i,j,v)$&lt;/span&gt; tuples and their associated segmentations of the actual data. The above example outputs the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Print segmentation 3-tuples:&#xA;(0, 1, 6)&#xA;(2, 6, 4)&#xA;(7, 10, 1)&#xA;&#xA;Print actual segmentations:&#xA;s0 = [ (0, 6) (1, 6)  ]&#xA;s1 = [ (2, 4) (3, 4) (4, 4) (5, 4) (6, 4)  ]&#xA;s2 = [ (7, 1) (8, 1) (9, 1) (10, 1)  ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thus, this algorithm is correct and runs in polynomial time!&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2018-12-25T22:27:24.967" LastActivityDate="2018-12-25T22:27:24.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30776" PostTypeId="1" AcceptedAnswerId="30791" CreationDate="2018-12-26T05:30:30.860" Score="2" ViewCount="187" Body="&lt;p&gt;I am familiarizing myself with CVXPY, and encountered a strange problem. I have the following simple toy optimization problem:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import cvxpy as cp&#xA;&#xA;A=np.array([[1,0,0],[0,1,0], [0,0,1]])&#xA;y=np.array([1,1,1])&#xA;&#xA;# Upper bound for the constraint term&#xA;upper=1&#xA;# Solve the optimization problem using CVXPY&#xA;x = cp.Variable(3)&#xA;objective = cp.Minimize(cp.sum_squares(x))&#xA;constraint = [cp.sum_squares(A*x - y) &amp;lt;= upper]&#xA;prob = cp.Problem(objective, constraint)&#xA;prob.solve()&#xA;optimal_x = x.value&#xA;&#xA;print('Value of constraint at optimal x:' + str(np.linalg.norm(A*optimal_x - y)**2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I expect my output number to be samller than &lt;code&gt;upper=1&lt;/code&gt;, but what I get is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Value of constraint at optimal x:3.0000000068183947&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am very confused about how this could be true. Am I using the function &lt;code&gt;cp.sum_squares&lt;/code&gt; incorrectly? Am I just setting up the optimization in a wrong way? Any help is appreciated!!&lt;/p&gt;&#xA;" OwnerUserId="29563" LastActivityDate="2018-12-27T22:31:04.713" Title="Question about strange outputs from the CVXPY solver" Tags="|optimization|python|constrained-optimization|convex-optimization|cvxpy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30783" PostTypeId="1" CreationDate="2018-12-27T13:17:53.020" Score="3" ViewCount="466" Body="&lt;p&gt;I'm trying to model the temperature distribution over a curved surface. Apart from the heat equation, I need to take into account the energy emission/absorption through electromagnetic radiation. The surface is divided into small triangles that will exchange energy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we focus on the radiation part, one way to start is to compute form factors that take into account how much one surface sees another one. Then, it is easy to derive a formula for the energy exchange between the triangles.&#xA;However, this formula implies calculating powers of a matrix &lt;span class=&quot;math-container&quot;&gt;$n\times n$&lt;/span&gt; where n is the number of triangles. This can be done in &lt;span class=&quot;math-container&quot;&gt;$n^{2.8}$&lt;/span&gt;. Besides that, form factors are also expensive to calculate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is quite expensive. For a mesh divided in 10,000 triangles, which is not a lot, it's more than &lt;span class=&quot;math-container&quot;&gt;$10^{11}$&lt;/span&gt; calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know any reference to some algorithm that solves the same problem with better complexity? Note that game-like algorithms aren't likely to work well because here there are as many light sources and as many observers as there are triangles...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not aware of any good deterministic algorithm but know about &lt;strong&gt;heuristic&lt;/strong&gt; ones it can also be useful. &lt;/p&gt;&#xA;" OwnerUserId="29581" LastActivityDate="2019-01-26T14:02:55.577" Title="Radiation heat transfer between surfaces" Tags="|pde|algorithms|simulation|performance|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30784" PostTypeId="2" ParentId="30783" CreationDate="2018-12-27T13:40:15.897" Score="2" Body="&lt;p&gt;Disclaimer: I am not an expert in heat transfer, but have a lot of computational electromagnetics experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you are looking for a so-called &lt;em&gt;fast algorithm&lt;/em&gt; for heat transfer problems. One family worth considering is a family of &lt;a href=&quot;https://en.wikipedia.org/wiki/Multipole_expansion&quot; rel=&quot;nofollow noreferrer&quot;&gt;multipole algorithms&lt;/a&gt;. &lt;a href=&quot;https://en.wikipedia.org/wiki/Fast_multipole_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast Multipole Method (FMM)&lt;/a&gt; is extremely popular in the modeling of electromagnetic phenomena. But I've seen its application to heat transfer problems as well. FMM, in general, allows reducing the complexity to &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(N\log N)$&lt;/span&gt; (subject to the problem statement, preconditioning, etc.) There probably are certain limitations/details in applying FMM to heat transfer, but I would consider this approach very viable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;J. Bennet, J. Claesson, and G. Hellström, &quot;&lt;a href=&quot;http://lup.lub.lu.se/record/1894632&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multipole method to compute the conductive heat flows to and between pipes in a composite cylinder&lt;/a&gt;,&quot; &lt;em&gt;Notes Heat Transfer&lt;/em&gt;, vol. 3, Jan. 1987.&lt;/li&gt;&#xA;&lt;li&gt;Y. Han, Y. Nie, and H. Dong, &quot;&lt;a href=&quot;https://doi.org/10.1016/j.jqsrt.2018.09.002&quot; rel=&quot;nofollow noreferrer&quot;&gt;A fast multipole algorithm for radiative heat transfer in 3D semitransparent media,&lt;/a&gt;&quot; &lt;em&gt;J. Quant. Spectroscopy Radiative Transfer&lt;/em&gt;, vol. 221, pp. 8–17, Dec. 2018.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These are some publications which I found first. I think that they will allow you to get to maybe more foundational/advanced papers on this topic.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-12-27T13:40:15.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30788" PostTypeId="1" CreationDate="2018-12-27T21:22:56.150" Score="2" ViewCount="1960" Body="&lt;p&gt;I have a system of fairly large set of linear equations (approximately 30K equations). I am using &lt;code&gt;scipy.sparse.spsolve&lt;/code&gt; to solve these equations.  Initially, I tried with 10K equations, but my program got killed due to memory issue. In general, I would like to know the memory and time requirements of the &lt;code&gt;spsolve&lt;/code&gt; algorithm and exactly how the algorithm solves the system of equations. Is it based on factorization? Anyone can help me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB* - system of equations does not have a predefined structure like symmetricity etc, only thing is it is very sparse.&lt;/p&gt;&#xA;" OwnerUserId="4396" LastEditorUserId="20688" LastEditDate="2018-12-27T22:07:50.613" LastActivityDate="2018-12-27T22:17:36.257" Title="Memory and time requirements of the scipy sparse spsolve" Tags="|linear-solver|sparse-matrix|scipy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30790" PostTypeId="2" ParentId="30788" CreationDate="2018-12-27T22:17:36.257" Score="4" Body="&lt;p&gt;First of all, you need to make sure that you construct &amp;amp; store your original matrix already in a sparse format. Otherwise, your problems can start already at this stage, since storing such a large matrix in dense format and only then converting it to sparse can easily blow up the memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.sparse.linalg.spsolve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.sparse.linalg.spsolve&lt;/a&gt; documentation, by default it uses &lt;a href=&quot;https://en.wikipedia.org/wiki/UMFPACK&quot; rel=&quot;nofollow noreferrer&quot;&gt;UMFPACK&lt;/a&gt;, which would be multifrontal LU factorization. &lt;a href=&quot;https://scicomp.stackexchange.com/a/8972/20688&quot;&gt;This answer&lt;/a&gt; will give you some insight on what may actually happen in your case (fill-ins). In short, there is a chance that even though your non-factorized matrix is sparse, its structure (and reorderings used in the factorization algorithm) result in a lot of fill-ins, resulting in a much denser factorized matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, it's quite hard to give time&amp;amp;memory estimates for sparse matrix factorization since a lot depends on the structure of the matrix, used algorithms, and &lt;a href=&quot;https://www.mathworks.com/help/matlab/math/sparse-matrix-reordering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;strategies to reduce fill-ins&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2018-12-27T22:17:36.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30791" PostTypeId="2" ParentId="30776" CreationDate="2018-12-27T22:31:04.713" Score="2" Body="&lt;p&gt;You are doing a broadcast (&lt;code&gt;A*x&lt;/code&gt;), rather than matrix multiplication (&lt;code&gt;A@x&lt;/code&gt;), so your code should look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import numpy as np&#xA;import cvxpy as cp&#xA;&#xA;A=np.array([[1,0,0],[0,1,0], [0,0,1]])&#xA;y=np.array([1,1,1])&#xA;&#xA;# Upper bound for the constraint term&#xA;upper=1&#xA;# Solve the optimization problem using CVXPY&#xA;x          = cp.Variable(3)&#xA;objective  = cp.Minimize(cp.sum_squares(x))&#xA;constraint = [cp.sum_squares(A@x - y) &amp;lt;= upper]&#xA;prob       = cp.Problem(objective, constraint)&#xA;prob.solve()&#xA;optimal_x = x.value&#xA;&#xA;print('Value of constraint at optimal x: {0}'.format(np.linalg.norm(A@optimal_x - y)**2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which gives&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Value of constraint at optimal x: 1.000000002699704&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2018-12-27T22:31:04.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30792" PostTypeId="1" AcceptedAnswerId="30803" CreationDate="2018-12-27T23:52:49.220" Score="3" ViewCount="1616" Body="&lt;p&gt;I'm trying to implement the aforementioned procedure using &lt;a href=&quot;http://www.netlib.org/utk/people/JackDongarra/etemplates/node198.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website&lt;/a&gt; as a reference. At the end of the page the algorithm is described as follows:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DxqVf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DxqVf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I've mapped the given algorithm to code correctly. However when I use the U and V obtained in the procedure to compute &lt;span class=&quot;math-container&quot;&gt;$U^{T}AV$&lt;/span&gt; I don't get a bidiagonal matrix. The values along the diagonals indeed are the ones computed within the procedure but the rest of the values are not zeros. Here's my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def golub_kahan(a):&#xA;  n = a.shape[1]&#xA;  v = np.ones(n, dtype=&quot;float32&quot;) / np.sqrt(n)&#xA;  u = np.zeros(a.shape[0], dtype=&quot;float32&quot;)&#xA;  beta = 0&#xA;  U, V = np.zeros_like(a, dtype=&quot;float32&quot;), np.zeros((n,n), dtype=&quot;float32&quot;)&#xA;&#xA;  for i in range(n):&#xA;    V[:, i] = v&#xA;    u = a @ v - beta * u&#xA;    alpha = np.linalg.norm(u)&#xA;    u /= alpha&#xA;    U[:, i] = u&#xA;    v = a.T @ u - alpha * v&#xA;    beta = np.linalg.norm(v)&#xA;    v /= beta&#xA;&#xA;  return U, V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not sure where my mistake is, did I misunderstand the algorithm somehow?&lt;/p&gt;&#xA;" OwnerUserId="29587" LastActivityDate="2018-12-29T01:19:04.127" Title="Golub-Kahan-Lanczos Bidiagonalization Procedure implementation doesn't produce bidiagonal matrix" Tags="|python|numerics|svd|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30793" PostTypeId="2" ParentId="30598" CreationDate="2018-12-28T02:59:19.603" Score="1" Body="&lt;p&gt;Perhaps you could augment your output to include sufficient fake residuals of value 0 such that &lt;code&gt;lm&lt;/code&gt; accepts your function.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2018-12-28T02:59:19.603" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30794" PostTypeId="1" CreationDate="2018-12-28T08:18:06.410" Score="1" ViewCount="88" Body="&lt;p&gt;I'm studying the transport of species A in the blood vessels,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial t} =  D\frac{\partial ^2 C}{\partial x^2}-v\frac{\partial C}{\partial x}$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At x=0, I want to use the Neumann boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial C}{\partial x} =$&lt;/span&gt; &lt;em&gt;constant&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the literature, I have the average concentration of the species A in blood.&#xA;But, I am not sure what would be a reasonable value of the &lt;em&gt;constant&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to ask for suggestions on how to define the concentration gradient at the inlet.&lt;/p&gt;&#xA;" OwnerUserId="29087" LastActivityDate="2018-12-28T08:18:06.410" Title="Implementing boundary condition" Tags="|pde|boundary-conditions|numerics|computational-biology|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30795" PostTypeId="1" CreationDate="2018-12-28T08:39:16.920" Score="1" ViewCount="110" Body="&lt;p&gt;I have come across schemes where TVD (with flux limiters) is used for spatial discretisation along with Runge-kutta for Temporal discretisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can TVD be used for Temporal discretisation? If so could you please point me to some references? I have been trying to look for it but couldn't find anything.&lt;/p&gt;&#xA;" OwnerUserId="29590" LastActivityDate="2018-12-28T21:45:10.107" Title="TVD for temporal dicretisation" Tags="|finite-difference|fluid-dynamics|finite-volume|runge-kutta|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30796" PostTypeId="1" AcceptedAnswerId="30798" CreationDate="2018-12-28T14:37:11.903" Score="2" ViewCount="132" Body="&lt;p&gt;If I have a RK method &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; of order &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and a RK method &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; of order &lt;span class=&quot;math-container&quot;&gt;$p-1$&lt;/span&gt; I have read I can estimate the local error as &lt;span class=&quot;math-container&quot;&gt;$r_{n+1} = y_{n+1} - z_{n+1}$&lt;/span&gt;. First of all I don't see how this estimates the local error defined as the error we make in a single step using correct previous values. What does a lower order method have anything to do with previous correct values? I can accept that this is the error we make using a lower order method instead of a higher order method, but what is the relevance of this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, I read that the step size should be updated like &#xA;&lt;span class=&quot;math-container&quot;&gt;$$h_{n+1} = \left( \frac{TOL}{r_{n+1}}\right)^{\frac{1}{p}}h_n.$$&lt;/span&gt;&#xA;But I don't understand why this would work since we update our step size so that &lt;span class=&quot;math-container&quot;&gt;$r_{n+1} = TOL$&lt;/span&gt;, this would just result in &lt;span class=&quot;math-container&quot;&gt;$h_{n+1} = h_n$&lt;/span&gt; after a while. &lt;strong&gt;What am I missing?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="29218" LastActivityDate="2018-12-28T16:36:34.250" Title="Step size updating scheme adaptive embedded RK methods" Tags="|error-estimation|runge-kutta|adaptive-timestepping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30798" PostTypeId="2" ParentId="30796" CreationDate="2018-12-28T16:36:34.250" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;First of all I don't see how this estimates the local error defined as the error we make in a single step using correct previous values. What does a lower order method have anything to do with previous correct values? I can accept that this is the error we make using a lower order method instead of a higher order method, but what is the relevance of this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Both methods are converging to the same true solution, but at slightly different weights. You can think of convergence of rate &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; as matching the first &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; terms of the Taylor series. The is because when proving convergence to the true solution, you expand &lt;span class=&quot;math-container&quot;&gt;$u(t_{n+1})$&lt;/span&gt; in its Taylor series of &lt;span class=&quot;math-container&quot;&gt;$h = t_{n+1} - t_n$&lt;/span&gt; and show that you match the first &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; terms. So given that, in Taylor series form, &lt;span class=&quot;math-container&quot;&gt;$r_{n+1}$&lt;/span&gt; is some polynomial &lt;span class=&quot;math-container&quot;&gt;$r_{n+1} = \alpha_p h^p + \mathcal{O}(h^{p+1})$&lt;/span&gt; since both of the methods have exactly the same expansion for the first &lt;span class=&quot;math-container&quot;&gt;$p-1$&lt;/span&gt; terms, but only one of the methods matches the &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;th term. Dropping the higher order terms since &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is considered small enough, we have &lt;span class=&quot;math-container&quot;&gt;$r_{n+1} \approx \alpha_p h^p$&lt;/span&gt;. But understand what this term means. Since the &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;th order method has the correct Taylor series expansion, this term is exactly the error of the &lt;span class=&quot;math-container&quot;&gt;$p-1$&lt;/span&gt;th order method from the true Taylor term, meaning that &lt;span class=&quot;math-container&quot;&gt;$\alpha_p$&lt;/span&gt; is the coefficient of the leading (largest) term in the local truncation error of the &lt;span class=&quot;math-container&quot;&gt;$p-1$&lt;/span&gt;th method and thus a good approximation for the method's local truncation error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now we have two things, &lt;span class=&quot;math-container&quot;&gt;$r_{n+1}$&lt;/span&gt; is a good approximation to one of the method's local truncation error, and &lt;span class=&quot;math-container&quot;&gt;$r_{n+1}\approx \alpha_p \Delta t^p$&lt;/span&gt;. The true path to the formula you have up there is a result for the optimal stepsize due to Cechino where you prove that would give you the error/time optimal stepsize in the next step if the error estimate is exact. This is also known as proportional control but doesn't take into account stability concerns &lt;strong&gt;so this is not the method actually used in a lot of ODE solver software&lt;/strong&gt;, but it's a good teaching tool. You can get to this result without the optimal stepsize proof just through a bit of reasoning though. Proportional control makes sense because what it means is that you want to correct the stepsize proportional to to size of the error: the larger the error, the more correction you want. So you could take your &lt;span class=&quot;math-container&quot;&gt;$TOL/r_{n+1}$&lt;/span&gt; as the coefficient, that's TOL divided by your error estimate, meaning that if your error estimate is twice the tolerance, you would halve your stepsize. That's reasonable, but recall that &lt;span class=&quot;math-container&quot;&gt;$r_{n+1}\approx \alpha_p \Delta t^p$&lt;/span&gt;, so dimensionally it makes sense to take the &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;th root so that way this term actually scales like &lt;span class=&quot;math-container&quot;&gt;$1/h$&lt;/span&gt;. Of course, this is heuristic and the actual proof is more in depth. So there you go, now you have a reasonable error estimate and a reasonable way to make that produce a new stepsize.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But I don't understand why this would work since we update our step size so that &lt;span class=&quot;math-container&quot;&gt;$r_{n+1}=TOL$&lt;/span&gt;, this would just result in &lt;span class=&quot;math-container&quot;&gt;$h_{n+1}=h_n$&lt;/span&gt; after a while.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the error of the methods were independent of the stepsize or the location in time for the ODE, this would happen. But that's never true. Instead, the local truncation error of the methods very depending on many factors, such as the derivatives of the derivative function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, the size of the current state values, etc., and thus &lt;span class=&quot;math-container&quot;&gt;$\alpha_p$&lt;/span&gt; is always changing, causing &lt;span class=&quot;math-container&quot;&gt;$r_{n+1}$&lt;/span&gt; and the optimal stepsize to always change. But for some simple enough equations, when adaptivity is working correctly, the stepsize will stability around some &quot;optimal&quot; value.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2018-12-28T16:36:34.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30801" PostTypeId="2" ParentId="30795" CreationDate="2018-12-28T20:07:31.913" Score="1" Body="&lt;p&gt;TVD is used only for spacial dicretization(as per my knowledge), using a TVD(total variation diminishing) scheme the total variation in a solution in transient state reduces over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a spacial discretization method that is intended to be stable(without 'wiggles'), and the variation in this system diminishes over time, it is not temporal discretization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if you want to read on TVD 'An Introduction to Computational Fluid Dynamics' by H.K. Versteeg is good.&lt;/p&gt;&#xA;" OwnerUserId="29593" LastEditorUserId="29593" LastEditDate="2018-12-28T20:15:52.073" LastActivityDate="2018-12-28T20:15:52.073" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30802" PostTypeId="2" ParentId="30795" CreationDate="2018-12-28T21:45:10.107" Score="2" Body="&lt;p&gt;For SSP time integrators, a good reference is the book by Gottlieb, Ketcheson, and Shu&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/9814289264&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.amazon.com/stability-preserving-runge-kutta-multistep-discretizations/dp/9814289264&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5503" LastActivityDate="2018-12-28T21:45:10.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30803" PostTypeId="2" ParentId="30792" CreationDate="2018-12-29T01:19:04.127" Score="3" Body="&lt;p&gt;This algorithm suffers from similar numerical stability problems as the &#xA;symmetric Lanczos tridiagonalization algorithm, &#xA;see &lt;a href=&quot;https://en.wikipedia.org/wiki/Lanczos_algorithm#Numerical_stability&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In exact arithmetic, after &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; steps the following holds: &lt;span class=&quot;math-container&quot;&gt;$U_k^T U_k =I$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_k^T V_k =I$&lt;/span&gt;,&#xA;and &lt;span class=&quot;math-container&quot;&gt;$U_k^T A V_k = B_k$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$B_k$&lt;/span&gt; is the bidiagonal matrix with diagonal elements &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; and superdiagonal elements &lt;span class=&quot;math-container&quot;&gt;$\beta_i$&lt;/span&gt;. &#xA;However, in floating point arithmetic, even for modest &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, the matrices &#xA;&lt;span class=&quot;math-container&quot;&gt;$U_k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V_k$&lt;/span&gt; may become far from orthogonal. &#xA;Moreover, &lt;span class=&quot;math-container&quot;&gt;$U_k^* A V_k$&lt;/span&gt; may have relatively large entries not only on the bidiagonals, &#xA;but also at positions far away from the diagonal.&#xA;Nevertheless, the largest singular value of &lt;span class=&quot;math-container&quot;&gt;$B_k$&lt;/span&gt; is often a quite good approximation of &#xA;the largest singular value of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, even for relatively small &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A much more stable bidiagonalization algorithm is the Householder bidiagonalization&#xA;algorithm, which is described in Paragraph 5.4.8 of &lt;em&gt;Matrix computations&lt;/em&gt; by &lt;em&gt;Golub and &#xA;Van Loan, 4th edition&lt;/em&gt;. This method uses Householder reflections&#xA;to bidiagonalize matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, which is relatively expensive but also very stable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to improve the Lanczos bidiagonalization algorithm,&#xA;and to maintain the orthogonality, &#xA;by &lt;a href=&quot;https://tidsskrift.dk/daimipb/article/view/7070/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reothogonalization&lt;/a&gt;&#xA;See also: &lt;a href=&quot;http://www2.cs.cas.cz/mweb/download/publi/Ples2006.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some remarks on bidiagonalization and its &#xA;implementation&lt;/a&gt;.&#xA;Note that Lanczos bidiagonalization only needs matrix vector products with &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A^T$&lt;/span&gt;,&#xA;while the Householder bidiagonalization algorithm needs explicit access to the matrix entries of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, which is a disadvantage&lt;/p&gt;&#xA;" OwnerUserId="5286" LastActivityDate="2018-12-29T01:19:04.127" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30806" PostTypeId="1" AcceptedAnswerId="30820" CreationDate="2018-12-29T08:29:36.073" Score="4" ViewCount="1303" Body="&lt;p&gt;I am trying to do some work on some math conjecture. I am testing the conjecture numbers using very large math numbers (100+ digits ). I am currently using python to test these numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the conjecture's calculation, I quickly need to multiply and divide different large numbers several times. Python can do this quickly(~3000 calculations per minute). Is there any other faster software to achieve this.&lt;/p&gt;&#xA;" OwnerUserId="29600" LastActivityDate="2019-05-07T10:39:04.937" Title="Best software to do big number calculations quickly" Tags="|linear-algebra|optimization|c++|software|computer-arithmetic|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30807" PostTypeId="1" AcceptedAnswerId="30808" CreationDate="2018-12-30T00:14:32.357" Score="-2" ViewCount="150" Body="&lt;pre&gt;&lt;code&gt;x0=[10 5]';&#xA;tol=1e-6;&#xA;syms x y&#xA;f= (1/2)*(x^2)+(9/2)*(y^2);&#xA;g=gradient(f, [x, y]);&#xA;g0=subs(g, x, x0(1,1));&#xA;g0=subs(g0, y, x0(2,1));&#xA;while norm(g0)&amp;gt;tol&#xA;    s0=-g0;&#xA;    %perform a linesearch to find an acceptable step size&#xA;    alpha=1/2;&#xA;    L0=1; &#xA;    f0=subs(f,x, x0(1,1));&#xA;    f0=subs(f0,y,x0(2,1));&#xA;&#xA;    v=f0;&#xA;    xnew=x0+L0*s0;&#xA;&#xA;    f1=subs(f,x, xnew(1,1));&#xA;    f1=subs(f,y,xnew(2,1));&#xA;&#xA;    u=f1;&#xA;    w=alpha*L0*g0'*s0;&#xA;&#xA;    while u &amp;gt; v+w&#xA;&#xA;        L0=L0/2;&#xA;&#xA;    end&#xA;&#xA;    L0;&#xA;    x0=x0+L0*s0;&#xA;&#xA;end&#xA;&#xA;double(x0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;When I try to run this script in Matlab. It gets stuck. I don't know why!&#xA;is there something wrong with the code?&lt;/p&gt;&#xA;" OwnerUserId="29598" LastEditorUserId="29598" LastEditDate="2018-12-31T09:35:21.167" LastActivityDate="2018-12-31T09:35:21.167" Title="Gradient line-search Matlab code gets stuck" Tags="|optimization|matlab|nonlinear-programming|" AnswerCount="1" CommentCount="0" ClosedDate="2018-12-30T16:54:13.923" ContentLicense="CC BY-SA 4.0" />
  <row Id="30808" PostTypeId="2" ParentId="30807" CreationDate="2018-12-30T01:16:42.153" Score="1" Body="&lt;p&gt;The while loop only changes &lt;code&gt;L0&lt;/code&gt;, but &lt;code&gt;u&lt;/code&gt;, &lt;code&gt;v&lt;/code&gt; and &lt;code&gt;w&lt;/code&gt; stay the same. So the condition for the loop never changes.&lt;/p&gt;&#xA;" OwnerUserId="7031" LastActivityDate="2018-12-30T01:16:42.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30810" PostTypeId="1" CreationDate="2018-12-30T13:48:21.723" Score="-2" ViewCount="91" Body="&lt;pre&gt;&lt;code&gt;x0=[0.8, 0.8, 0.2, 0.2]';&#xA; m0=[0.5,0.5]'; &#xA;tol=1e-6;&#xA;syms x y z p m1 m2&#xA;h1=y-(x^3)-(z^2);&#xA;h2=(x^2)-y-p^2;&#xA;h=[h1;h2];&#xA;f=-x; &#xA;L=f+ m1*h1+m2*h2;&#xA;h0 = subs(h, {x,y,z,p}, [x0(1,1), x0(2,1), x0(3,1), x0(4,1)]); &#xA;h0=double(h0);&#xA;%the system h evaluated at the intial vector&#xA;g1=gradient(h1, [x, y, z, p]);&#xA;g2=gradient(h2, [x, y, z, p]);&#xA;J=[g1 g2];&#xA;J0=subs(J, x, x0(1,1)); &#xA;J0=subs(J0, y, x0(2,1));&#xA;J0=subs(J0, z, x0(3,1));&#xA;J0=subs(J0, p, x0(4,1));&#xA;%J evaluated at the intial vector&#xA;J0=double(J0)&#xA;n=size(J0,1); %number of rows in J0&#xA;m=size(J0,2); %number of columns in J0&#xA;DL=gradient(L, [x,y,z,p,m1,m2]); &#xA;DL=subs(DL, y, x0(2,1));&#xA;DL=subs(DL, z, x0(3,1));&#xA;DL=subs(DL, p, x0(4,1));&#xA;DL=subs(DL, x, x0(1,1));&#xA;DL=subs(DL,m1, m0(1,1));&#xA;DL=subs(DL,m2, m0(2,1));&#xA;DL=double(DL)           &#xA;DLd=DL(1:n,1);&#xA;H=hessian(L, [x,y,z,p]); &#xA;H0 =subs(H,x,x0(1,1));&#xA;H0=subs(H0,y,x0(2,1));&#xA;H0=subs(H0,z, x0(3,1));&#xA;H0=subs(H0,p, x0(4,1));&#xA;H0=subs(H0,m1, m0(1,1));&#xA;H0=subs(H0,m2, m0(2,1));&#xA;H0=double(H0)&#xA;[Q, R]= qr(J0);&#xA;Y=Q(1:n,1:m);&#xA;Z=Q(1:n, m+1:n);&#xA;qz=Z'*DLd; qh=h0;&#xA;while norm(qz)+ norm(qh) &amp;gt; tol&#xA;E=[Z'*H0;J0'];&#xA;V=[Z'*DLd;h0];&#xA;s0=E\-V&#xA;x0=x0+s0 %the new point&#xA;J0=subs(J, x, x0(1,1)); &#xA;J0=subs(J0, y, x0(2,1));&#xA;J0=subs(J0, z, x0(3,1));&#xA;J0=subs(J0, p, x0(4,1));&#xA;J0=double(J0);&#xA;[Q, R]= qr(J0);&#xA;Y=Q(1:n,1:m);&#xA;Z=Q(1:n, m+1:n);&#xA;&#xA;r=R(1:m,1:m);&#xA;T0=[-1 0 0 0]';&#xA;%T evaluated at the new vector x0&#xA;m0=r\-(Y'*T0); &#xA;qz=Z'*DLd;&#xA;h0 = subs(h, {x,y,z,p}, [x0(1,1), x0(2,1), x0(3,1), x0(4,1)]); &#xA;h0=double(h0);&#xA;qh=h0;&#xA;end&#xA;newvector=[double(x0); double(m0)];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT: Made some corrections in the code and updated qz and qh inside the loop. However, the code seems to loop forever. &#xA;the stopping criteria is never violated and x0 continues to change but diverging from the actual solution which is x=(1,1,0,0). I changed the tol to a relatively larger value which is 0.7 and the code generated an output that is near the solution.&#xA;Could the double precision be causing that problem when the tol=1e-6? &lt;/p&gt;&#xA;" OwnerUserId="29598" LastEditorUserId="29598" LastEditDate="2018-12-31T09:32:57.573" LastActivityDate="2018-12-31T09:32:57.573" Title="Stuck in infinite loop" Tags="|matlab|" AnswerCount="1" CommentCount="2" ClosedDate="2019-01-01T20:42:38.220" ContentLicense="CC BY-SA 4.0" />
  <row Id="30811" PostTypeId="2" ParentId="30810" CreationDate="2018-12-30T16:25:08.693" Score="4" Body="&lt;p&gt;The issue seems similar to the &lt;a href=&quot;https://scicomp.stackexchange.com/questions/30807/gradient-line-search-matlab-code-gets-stuck&quot;&gt;other question&lt;/a&gt; you have asked. You are not updating the &lt;code&gt;qz&lt;/code&gt; and &lt;code&gt;qh&lt;/code&gt; variables inside the loop, so the termination condition of the while loop is never reached.&lt;/p&gt;&#xA;" OwnerUserId="29276" LastActivityDate="2018-12-30T16:25:08.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30812" PostTypeId="2" ParentId="24099" CreationDate="2018-12-30T20:28:30.497" Score="1" Body="&lt;p&gt;You can go check the book &lt;a href=&quot;https://www.springer.com/gp/book/9783319446479&quot; rel=&quot;nofollow noreferrer&quot;&gt;'&lt;em&gt;The Lattice Boltzmann method: Principles and Practice&lt;/em&gt;'&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;em&gt;chapter 2.4 - Outlook: Why Lattice Boltzmann?&lt;/em&gt;, it talks about the simplicity, efficiency, geometry and the applications in multiphase and multicomponent flows, thermal flow, sound wave, and compressible flow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might not appropriate to copy and paste them here. So I would suggest you download the book, either from the Springer or the Genesis:).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: &#xA; 1. If your research is related to hydrodynamics, I would suggest Smoothed Particle Hydrodynamics (SPH).&#xA; 2. Palabos did the parallelization based on CPU instead of GPU. But generally, one of the advantages of LBM, as stated by many researchers, is the ease of parallelization.&lt;/p&gt;&#xA;" OwnerUserId="23469" LastActivityDate="2018-12-30T20:28:30.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30813" PostTypeId="2" ParentId="30806" CreationDate="2018-12-30T23:09:24.160" Score="1" Body="&lt;p&gt;Using something such as &lt;a href=&quot;https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.divide.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;numpy.divide&lt;/code&gt;&lt;/a&gt;, you most likely will see no considerable difference from doing it in C++. Although casting might be something to consider. &#xA;C++ may see some improvement in this area as Python will have to deal with this in real-time. With C++ this will not pose an issue. &#xA;(Python will now cast any &lt;code&gt;ints&lt;/code&gt; to &lt;code&gt;floats&lt;/code&gt; for doing division as well). &lt;/p&gt;&#xA;" OwnerUserId="28966" LastEditorUserId="28966" LastEditDate="2019-03-22T21:37:32.607" LastActivityDate="2019-03-22T21:37:32.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30814" PostTypeId="1" AcceptedAnswerId="30815" CreationDate="2018-12-31T10:47:35.687" Score="0" ViewCount="340" Body="&lt;p&gt;I am trying to solve the following, &quot;test problem&quot; involving a vectorial Poisson equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$-\nabla^2 \vec{A}=\vec{J} \quad \forall x\in\Omega=[-1,1]^3$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \vec{A}=\vec{0} \quad \forall x\in\partial\Omega$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The inhomogenity &lt;span class=&quot;math-container&quot;&gt;$\vec{J}$&lt;/span&gt; is given as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\vec{J}(x)=\vec{e_{x}} \quad \mathrm{if} \quad \mathrm{max}(\mathrm{abs}(x_1,x_2,x_3))&amp;lt;0.1$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. &lt;span class=&quot;math-container&quot;&gt;$\vec{J}$&lt;/span&gt; is a vector pointing in the x-Direction in a cube of length 0.2 units centered around the origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to solve this problem using the Python code listed below. Unfortunately, the code always produces an error message (also listed below), saying that &lt;span class=&quot;math-container&quot;&gt;$a(\vec{A},\vec{v})$&lt;/span&gt;, defined as &lt;code&gt;a=inner(grad(A),grad(v))*dx&lt;/code&gt;, is not a bilinear form. Does anyone have an idea on how I could resolve this issue?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Python-Code:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from fenics import *&#xA;from mshr import *&#xA;&#xA;#define the domain&#xA;domain=Box(Point(-1,-1,-1),Point(1,1,1))&#xA;mesh = generate_mesh(domain, 32)&#xA;V=VectorFunctionSpace(mesh,&quot;P&quot;,1)&#xA;#plot(mesh)&#xA;&#xA;#define the boundary conditions&#xA;def boundary(x,on_boundary):&#xA;    return on_boundary&#xA;A_D=Constant((0,0,0))&#xA;bc=DirichletBC(V,A_D,boundary)&#xA;&#xA;#define the inhomogenity J&#xA;codex=&quot;x[0]&amp;gt;-ext&amp;amp;&amp;amp;x[0]&amp;lt;ext&amp;amp;&amp;amp;x[1]&amp;gt;-ext&amp;amp;&amp;amp;x[1]&amp;lt;ext&amp;amp;&amp;amp;x[2]&amp;gt;-ext&amp;amp;&amp;amp;x[2]&amp;lt;ext?1:0&quot;&#xA;codex=codex.replace(&quot;ext&quot;,&quot;0.1&quot;)&#xA;codey=codez=&quot;0&quot;&#xA;&#xA;#define variational problem&#xA;A=TrialFunction(V)&#xA;v=TestFunction(V)&#xA;J=Expression((codex,codey,codez),degree=1)&#xA;a=inner(grad(A),grad(v))*dx&#xA;L=dot(J,v)*dx&#xA;&#xA;#solve the system&#xA;A=Function(V)&#xA;solve(L==a,A,bc)&#xA;plot(A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Error Message:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;*** -------------------------------------------------------------------------&#xA;*** DOLFIN encountered an error. If you are not able to resolve this issue&#xA;*** using the information listed below, you can ask for help at&#xA;***&#xA;***     fenics-support@googlegroups.com&#xA;***&#xA;*** Remember to include the error message listed below and, if possible,&#xA;*** include a *minimal* running example to reproduce the error.&#xA;***&#xA;*** -------------------------------------------------------------------------&#xA;*** Error:   Unable to define linear variational problem a(u, v) == L(v) for all v.&#xA;*** Reason:  Expecting the left-hand side to be a bilinear form (not rank 1).&#xA;*** Where:   This error was encountered inside LinearVariationalProblem.cpp.&#xA;*** Process: 0&#xA;*** &#xA;*** DOLFIN version: 2018.1.0&#xA;*** Git changeset:  unknown&#xA;*** -------------------------------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29610" LastActivityDate="2018-12-31T12:30:19.567" Title="Solving Vectorial Poisson Equation in FENICS" Tags="|poisson|fenics|vector|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30815" PostTypeId="2" ParentId="30814" CreationDate="2018-12-31T12:30:19.567" Score="1" Body="&lt;p&gt;Ok I will answer my own question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is in the line &lt;code&gt;solve(L==a,A,bc)&lt;/code&gt; which needs to be replaced by &lt;code&gt;solve(a==L,A,bc)&lt;/code&gt;. The two versions seem to non interchangeable. Doing so will result in the magnetic field &lt;code&gt;B=project(curl(A),V)&lt;/code&gt; which is shown in the picture below:&lt;a href=&quot;https://i.stack.imgur.com/OcROU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OcROU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29610" LastActivityDate="2018-12-31T12:30:19.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30816" PostTypeId="2" ParentId="29253" CreationDate="2018-12-31T16:38:45.713" Score="2" Body="&lt;p&gt;To close this question, as @AntonMenshov said in a comment I will put two of mine toguether:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These peaks result when some iterative method becomes unstable, like when using euler forwards scheme the Courant number defined as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$C = \frac{v \Delta t}{\Delta x}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\Delta x/v$&lt;/span&gt; must be the minimum residence time in a computation cell defined as the characteristic length of this cell &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; divided by its local velocity &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; and this number must be compared with the simulation timestep &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt;. For an explicit time-scheme the Courant number is usually under 1 for a pure convection equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you can improve the handling by using nondimensional numbers: scaling your dimensional quantities such as the potential or the time allows you to control better the numerical aspects.&lt;/p&gt;&#xA;" OwnerUserId="22590" LastActivityDate="2018-12-31T16:38:45.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30817" PostTypeId="1" AcceptedAnswerId="30824" CreationDate="2019-01-01T10:41:30.080" Score="0" ViewCount="157" Body="&lt;p&gt;I want to simulate Brownian motion in 3-D for the following conditions:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$p(x=0,y=0,z=0,t=0)=1$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$p(x,y,z=c,t)=0$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is the probability of finding molecules in the 3-D environment. I want to find the number of particles getting absorbed on a certain region on the absorbing wall in 5 secs. As can be inferred the absorbing boundary is at &lt;span class=&quot;math-container&quot;&gt;$z=c$&lt;/span&gt;, and I want the probability of molecules hitting a circular area centered at &lt;span class=&quot;math-container&quot;&gt;$(0,0,2.5\times10^{-4})$&lt;/span&gt; with a radius &lt;span class=&quot;math-container&quot;&gt;$5\times10^{-6}$&lt;/span&gt;m. The fluid environment has drift,i.e. &lt;span class=&quot;math-container&quot;&gt;$u,v,w$&lt;/span&gt; in three directions. The particles in &lt;span class=&quot;math-container&quot;&gt;$x,y,z$&lt;/span&gt; directions are moving as&#xA;    &lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;	\text{d}X_j(t)=u\text{d}t+\sqrt{2D}\text{d}B_j(t),\\&#xA;\text{d}Y_j(t)=v\text{d}t+\sqrt{2D}\text{d}B_j(t),\\&#xA;\text{d}Z_j(t)=w\text{d}t+\sqrt{2D}\text{d}B_j(t),&#xA;	\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\text{d}B$&lt;/span&gt; is the derivative of Brownian process which I am simulating through normal random varaible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using following MATLAB code to simulate particle diffusion:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;close all; &#xA;clear all;&#xA;clc;&#xA;TotalNumberReceivedUp=0;&#xA;Ts=5;&#xA;TotalSimulations1=100000;&#xA;flg=0;&#xA;for particlesNum=1:TotalSimulations1&#xA;flg=flg+1&#xA;D=4*10^-9; &#xA;c=250*10^-6;u=5*10^-6;v=2*10^-6;w=3*10^-6; &#xA;e=5*10^-6;&#xA;X=zeros;Y=zeros;Z=zeros;&#xA;X(1)=0;Y(1)=0;Z(1)=0;&#xA;j=1;&#xA;for i=5*10^-4:5*10^-4:Ts&#xA;t2=i;    &#xA;t1=t2-5*10^-4;&#xA;j=j+1;&#xA;r = normrnd(0,1);&#xA;q= normrnd(0,1);&#xA;p= normrnd(0,1);&#xA;X(j)=X(j-1)+(sqrt(2*D))*sqrt(t2-t1)*p+u*(t2-t1);&#xA;Y(j)=Y(j-1)+(sqrt(2*D))*sqrt(t2-t1)*q+v*(t2-t1);&#xA;Z(j)=Z(j-1)+(sqrt(2*D))*sqrt(t2-t1)*r+w*(t2-t1);&#xA;if ((Z(j)&amp;gt;=c) &amp;amp;&amp;amp; ...&#xA;    (sqrt((X(j))^2+(Y(j))^2)&amp;lt;=e))&#xA;TotalNumberReceivedUp=TotalNumberReceivedUp+1;&#xA;break&#xA;end&#xA;end&#xA;end&#xA;ProbabilityUp=TotalNumberReceivedUp/TotalSimulations1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But my results are not coming good, the probable particles hitting are more than those I am getting through analysis. I think it may be because of the excess over the boundary problem which has been discussed in &quot;A comparison of four methods for simulating the diffusion process&quot; for 1-D movement with two absorbing barriers. I have tried to use the same factor in 3-D but it doesn't work well. &lt;/p&gt;&#xA;" OwnerUserId="23559" LastEditorUserId="23559" LastEditDate="2019-01-01T10:47:32.427" LastActivityDate="2019-01-02T10:40:04.803" Title="Simulating Brownian motion in 3-D for first hitting time?" Tags="|simulation|advection-diffusion|diffusion|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30818" PostTypeId="1" AcceptedAnswerId="30822" CreationDate="2019-01-01T13:04:09.693" Score="2" ViewCount="779" Body="&lt;p&gt;I'm looking at the analytical solution of the convection-diffusion equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial t} =  D\frac{\partial ^2 C}{\partial x^2}-v\frac{\partial C}{\partial x}$$&lt;/span&gt;&#xA;with initial condition&#xA;&lt;span class=&quot;math-container&quot;&gt;$$c(x,0) = 4000$$&lt;/span&gt;&#xA;and with Dirichlet boundary condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C(x=0,t&amp;gt;0) = 4100$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neumann boundary condition&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial x}=0\text{ at } x=L ; t&amp;gt;0.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8xQzi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8xQzi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when the above analytical solution is coded I obtain negative values of C which is unrealistic.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function AnalyticalSoln2()&#xA;format long&#xA;R = 1;&#xA;D = 900;&#xA;v = 200;&#xA;L = 60; &#xA;co = 4100;&#xA;ci = 4000;&#xA;X = linspace(0,60,10);&#xA;t = 0:0.001:2;&#xA;sol=[];&#xA;for pos = 1:length(X)&#xA;    x = X(pos);&#xA;A1 = 0.5*erfc((x.*R-t.*v)./(2*(t.*D*R).^0.5));&#xA;A2 = 0.5*exp(x.*v/D).*erfc((x.*R+t.*v)./(2*(D*t.*R).^0.5));&#xA;A31 = 0.5*(2+v*(2*L-x)/D + (t.*v^2)/(D*R))*exp(v*L/D);&#xA;A32 = erfc((R*(2*L-x)+t.*v)./(2*(D*t.*R).^0.5));&#xA;A41 = -((t.*v^2)./(pi*D*R)).^0.5;&#xA;A42 = exp((v*L/D)-(R./(4*t.*D)).*(2*L -x + t.*v/R).^2);&#xA;A = A1 + A2 + A31.*A32 + A41.*A42;&#xA;if t==0 &#xA;C = ci + (co - ci)*A'&#xA;else&#xA;C = ci + (co - ci)*A' - co*A';&#xA;end&#xA;sol = horzcat(sol,C);&#xA;end&#xA;sol(1:100:end,:)&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Whereas, the numerical solution obtained from the pdepe solver is non-negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the solution obtained using pdepe&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function DiffusionConvectionMATLAB&#xA;format short&#xA;global D&#xA;m = 0;&#xA;x = linspace(0,60,10);&#xA;t = 0.1:0.1:2;&#xA;D = 900;&#xA;sol = pdepe(m,@pdefun,@icfun,@bcfun,x,t)&#xA;    function [g,f,s] = pdefun(x,t,c,DcDx)&#xA;    v = 200;&#xA;    g = 1;&#xA;    f = D*DcDx;&#xA;    s = -v*DcDx;&#xA;    end&#xA;&#xA;    function c0 = icfun(x)&#xA;    c0 = 4000;&#xA;    end&#xA;&#xA;    function [pl,ql,pr,qr] = bcfun(xl,cl,xr,cr,t)&#xA;    pl = cl-4100;&#xA;    ql = 0;&#xA;    pr = 0;&#xA;    qr = 1;&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could someone suggest if there is any issue with the way I'm computing the solution form the analytical expression?&lt;/p&gt;&#xA;" OwnerUserId="29087" LastEditorUserId="29087" LastEditDate="2019-01-02T06:20:59.220" LastActivityDate="2019-01-02T06:20:59.220" Title="Analytical Solution of Transport Equation" Tags="|pde|matlab|finite-difference|numerical-modelling|advection-diffusion|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30819" PostTypeId="2" ParentId="29307" CreationDate="2019-01-01T16:21:35.773" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Exact Density of states&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DosE[x_] := 1/(2 \[Pi]) *1/Sin[ArcCos[-x/2]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Define delta function  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Delta[e0_, \[Sigma]_] := &#xA;1/(Sqrt[2 \[Pi] \[Sigma]^2]) *Exp[-0.5*e0^2/\[Sigma]^2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;let  &lt;code&gt;evals&lt;/code&gt; be the known eigenvalues of the Hamiltonian matrix, then&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DOS[e0_, \[Sigma]0_] := Module[{\[Sigma] = \[Sigma]0, sum},&#xA;sum = 0.;&#xA;Clear[k];&#xA;For[k = 1, k &amp;lt;= Length[evals], k++,&#xA;If[Abs[e0 - evals[[k]]] &amp;lt; 6*\[Sigma],&#xA; sum = sum + Delta[(e0 - evals[[k]]), \[Sigma]]&#xA; ];&#xA;];&#xA;Return[sum/L] (*L=system size*)&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Plot&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Plot[{DosE[x], DOS[x, 0.01]}, {x, -1.99, 1.99}, PlotRange -&amp;gt; All,PlotStyle -&amp;gt; {Black, Red}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The numerical result (using Mathematica) 100%  agree with analytical for L=1024 with 0.01 variance of the Gaussian function (delta function).&lt;/p&gt;&#xA;" OwnerUserId="29616" LastActivityDate="2019-01-01T16:21:35.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30820" PostTypeId="2" ParentId="30806" CreationDate="2019-01-01T19:59:30.720" Score="6" Body="&lt;p&gt;Why not give &lt;a href=&quot;https://gmpy2.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;GMPY2&lt;/a&gt; a try? From the introduction:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;gmpy2 is a C-coded Python extension module that supports multiple-precision arithmetic. gmpy2 is the successor to the original gmpy module. The gmpy module only supported the GMP multiple-precision library. gmpy2 adds support for the MPFR (correctly rounded real floating-point arithmetic) and MPC (correctly rounded complex floating-point arithmetic) libraries. gmpy2 also updates the API and naming conventions to be more consistent and support the additional functionality.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;GMPY2 supports the integer arithmetic I assume you need and I doubt you can find anything faster.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On *nix based systems, you'll need to make sure that the latest version of GMP is installed (&lt;a href=&quot;https://gmplib.org/&quot; rel=&quot;noreferrer&quot;&gt;GNU MP Bignum Library&lt;/a&gt;. On Windows, you can find a Python wheel courtesy of Christophe Gohlke &lt;a href=&quot;https://www.lfd.uci.edu/~gohlke/pythonlibs/#gmpy&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2019-01-01T19:59:30.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30822" PostTypeId="2" ParentId="30818" CreationDate="2019-01-02T04:51:51.837" Score="1" Body="&lt;p&gt;Here's one way to implement the analytic solution. However, I don't seem to be getting the correct plots if I use &lt;code&gt;t0 = 0&lt;/code&gt; (as you mentioned in the comment). It works if you set &lt;code&gt;t0&lt;/code&gt; to something greater than the final time you have (like &lt;code&gt;t0 = 10&lt;/code&gt; for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also used function handles so that &lt;code&gt;A(x,t)&lt;/code&gt; can be evaluated at any &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;t&lt;/code&gt; value.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;format long&#xA;R = 1;&#xA;D = 900;&#xA;v = 200;&#xA;L = 60; &#xA;co = 4100;&#xA;ci = 4000;&#xA;x = linspace(0,60,100);&#xA;tvec = 0.0:0.01:2;&#xA;sol=[];&#xA;&#xA;%Use function handles so that A(x,t) can be evaluated for any x and t&#xA;A1 = @(x,t) 0.5*erfc((x.*R-t.*v)./(2*(t.*D*R).^0.5));&#xA;A2 = @(x,t) 0.5*exp(x.*v/D).*erfc((x.*R+t.*v)./(2*(D*t.*R).^0.5));&#xA;A31 = @(x,t) 0.5*(2+v*(2*L-x)/D + (t.*v^2)/(D*R))*exp(v*L/D);&#xA;A32 = @(x,t) erfc((R*(2*L-x)+t.*v)./(2*(D*t.*R).^0.5));&#xA;A41 = @(x,t) -((t.*v^2)./(pi*D*R)).^0.5;&#xA;A42 = @(x,t) exp((v*L/D)-(R./(4*t.*D)).*(2*L -x + t.*v/R).^2);&#xA;A = @(x,t) A1(x,t) + A2(x,t) + A31(x,t).*A32(x,t) + A41(x,t).*A42(x,t);&#xA;&#xA;t0 = 0; %this doesn't seem to give the right solution&#xA;%t0 = 10; %this seems to work&#xA;&#xA;for t = tvec %loop over time&#xA;    if (t &amp;lt;= t0)&#xA;        C = ci + (co - ci)*A(x,t);&#xA;    else&#xA;        C = ci + (co - ci)*A(x,t) - co*A(x,t-t0);&#xA;    end&#xA;&#xA;    sol = [sol; C];&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29276" LastActivityDate="2019-01-02T04:51:51.837" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30824" PostTypeId="2" ParentId="30817" CreationDate="2019-01-02T10:40:04.803" Score="1" Body="&lt;p&gt;If the whole wall is absorbing, then you should replace:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    if ((Z(j)&amp;gt;=c) &amp;amp;&amp;amp; ...&#xA;        (sqrt((X(j))^2+(Y(j))^2)&amp;lt;=e))&#xA;        TotalNumberReceivedUp=TotalNumberReceivedUp+1;&#xA;        break&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    if Z(j) &amp;gt;= c&#xA;        if sqrt(X(j)^2 + Y(j)^2) &amp;lt;= e&#xA;            TotalNumberReceivedUp = TotalNumberReceivedUp + 1;&#xA;        end&#xA;&#xA;        break&#xA;    end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Otherwise, the current absorbing boundary is the cylinder &lt;span class=&quot;math-container&quot;&gt;$\{ (x, y, z) \in \mathbb{R}^3 \, | \, x^2 + y^2 = e^2, z \ge c \}$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2019-01-02T10:40:04.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30826" PostTypeId="2" ParentId="10838" CreationDate="2019-01-02T16:40:35.073" Score="0" Body="&lt;p&gt;The following code will give you the expected zero&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;static void Main(string[] args)&#xA;{&#xA;    int a = 24;&#xA;    double result = alglib.gammafunction(5) - a;&#xA;    Console.WriteLine(result);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The library used is here &lt;a href=&quot;http://www.alglib.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;alglib&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29626" LastActivityDate="2019-01-02T16:40:35.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30827" PostTypeId="1" CreationDate="2019-01-02T19:08:01.860" Score="0" ViewCount="56" Body="&lt;p&gt;I'm trying to use the gradient ascent method on a convex function like the multivariate-Normal density function with respect to its parameters (the original is a bit more complicated), something similar to maximum likelihood estimation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a_{n+1}=a_n+\gamma_n\nabla F(a_n)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if for the gradient ascent method will misbehave if I use a constant value for the step size, i.e. &lt;span class=&quot;math-container&quot;&gt;$\gamma_n=\gamma$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if I wanted to use an adaptive step size, what would be the simplest, and fastest computationally-wise way to define it?&lt;/p&gt;&#xA;" OwnerUserId="24970" LastActivityDate="2019-01-02T23:41:40.250" Title="Gradient ascent method with a constant step size?" Tags="|optimization|convex-optimization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30828" PostTypeId="2" ParentId="30827" CreationDate="2019-01-02T23:41:40.250" Score="2" Body="&lt;p&gt;It's not hard to couple gradient ascent with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Line_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;line search method&lt;/a&gt; to determine the step size on the fly. Otherwise, for a fixed step size &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;, your algorithm might require too many steps to maximize &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is too small or it may not find the local maximum &#xA;if &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is too big.&lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2019-01-02T23:41:40.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30829" PostTypeId="1" CreationDate="2019-01-03T04:05:37.133" Score="4" ViewCount="234" Body="&lt;p&gt;The codes Quantum Espresso (QE) and Vienna Ab initio Simulation Package (VASP) both use plane wave basis sets and psuedopotentials. Most of the codes in both implementation of DFT uses Fortran code. Do they use similar self-consistent-field procedure to find the solutions of Kohn-Sham equations? &lt;/p&gt;&#xA;" OwnerUserId="11626" LastActivityDate="2019-01-03T04:05:37.133" Title="Does quantum espresso and VASP use same same self-consistent-field procedure?" Tags="|density-functional-theory|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30830" PostTypeId="1" CreationDate="2019-01-03T17:07:51.063" Score="1" ViewCount="307" Body="&lt;p&gt;Examine a dynamic 2D heat equation &lt;span class=&quot;math-container&quot;&gt;$\dot{u} = \Delta u$&lt;/span&gt; with zero boundary temperature. A standard finite difference approach is used on a rectangle using a &lt;span class=&quot;math-container&quot;&gt;$n\times n$&lt;/span&gt; grid. For the resulting linear systems a banded Cholesky solver is used. Compare the three methods explicit, implicit and Crank-Nicolson for the time stepping.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Explicit method&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ &#xA;\begin{gathered}&#xA;\frac{u_{i+1,j} - u_{i,j}}{\Delta t} = \kappa \frac{u_{i,j-1} - 2u_{i,j} + u_{i,j+1}}{(\Delta x)^2} \\&#xA;u_{i+1,j} = u_{i,j} + \frac{\kappa \Delta t}{(\Delta x)^2}(u_{i,j-1} - 2u_{i,j} + u_{i,j+1})\\&#xA;\vec{u}_{i+1} = \vec{u}_i - \kappa \Delta t \textbf{A}_n \cdot \vec{u}_{i}\\&#xA;\end{gathered} &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Implicit method&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{gathered}&#xA;\frac{u_{i+1,j} - u_{i,j}}{\Delta t} = \kappa \frac{u_{i+1,j-1} - 2u_{i+1,j} + u_{i+1,j+1}}{(\Delta x)^2} \\&#xA;\vec{u}_{i+1} = \vec{u}_i - \kappa \Delta t \textbf{A}_n \cdot \vec{u}_{i+1}\\&#xA;u_i = (\mathbb{I}_n + \kappa \Delta t \textbf{A}_n)^{-i}\cdot \vec{u}_0 &#xA;\end{gathered}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Crank-Nicolson&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{gathered}&#xA;\frac{u_{i+1, j}-u_{i,j}}{\kappa \Delta t}  = \frac{u_{i,j-1} - 2u_{i,j} + u_{i,j + 1}}{2(\Delta x)^2} + \frac{u_{i+1,j-1} - 2u_{i+1,j} + u_{i+1,j + 1}}{2(\Delta x)^2}\\&#xA;\vec{u}_{i+1} - \vec{u}_i = -\frac{\kappa \Delta t}{2}(\textbf{A}_n \cdot \vec{u}_{i+1} + \textbf{A}_n \cdot \vec{u}_i)&#xA;\end{gathered}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the &lt;strong&gt;question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Does the implicit method require the least memory?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my &lt;strong&gt;proposition for an answer:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would say that the statement if &lt;strong&gt;FALSE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The implicit method requires more computational effort to give an answer, because the matrix &lt;span class=&quot;math-container&quot;&gt;$\textbf{A}_n$&lt;/span&gt; needs to be inverted. However, I would say that this is not the reason why the statement is false: for the implicit method there is &lt;span class=&quot;math-container&quot;&gt;$\textbf{no extra/less storage needed}$&lt;/span&gt; (compared to the explicit method for example) because there is no extra data generated from the computation (for example no other matrix generated). Is the answer correct as well as the reasoning behind it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;N.B. This question has been originally asked in &lt;a href=&quot;https://math.stackexchange.com/questions/3060678/finite-difference-explicit-implicit-crank-nicolson-does-the-implicit-met?noredirect=1#comment6311192_3060678&quot;&gt;Mathematics&lt;/a&gt; and has been posted here following comments&lt;/p&gt;&#xA;" OwnerUserId="29635" LastActivityDate="2019-01-03T18:12:10.407" Title="Finite difference - Explicit / Implicit / Crank Nicolson - Does the implicit method require the least memory?" Tags="|finite-difference|implicit-methods|crank-nicolson|memory-management|explicit-methods|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30831" PostTypeId="2" ParentId="30830" CreationDate="2019-01-03T17:57:59.977" Score="1" Body="&lt;p&gt;You seem to have given the 1D equations for the discretizations, even though the problem is in 2D. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless, the explicit method requires the least memory since you don't even have to form a matrix to compute the solution at the next time step. If you have the solution vector at time &lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt;, you can apply simple stencil operations and directly obtain the solution at time &lt;span class=&quot;math-container&quot;&gt;$t_{i+1}$&lt;/span&gt;, without forming a matrix. Your storage requirement will be just two solution vectors: one for the current solution, and one for the solution at the next time step.&lt;/p&gt;&#xA;" OwnerUserId="29276" LastEditorUserId="29276" LastEditDate="2019-01-03T18:12:10.407" LastActivityDate="2019-01-03T18:12:10.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30833" PostTypeId="1" CreationDate="2019-01-04T20:30:13.883" Score="0" ViewCount="29" Body="&lt;p&gt;I want to understand Binary Search for 2 element list made of 1,2. I draw a tree as below. Is it correct?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IV7ZA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IV7ZA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I want to search for an element 2, it will make 2 comparisons. If I want to search for an element 22, it will also make 2 comparisons. But according to formula Θ(lg n), with n=2, it should make only one comparison in worst case. I'm not able to connect these 2 facts. Please help me.&lt;/p&gt;&#xA;" OwnerUserId="29647" LastActivityDate="2019-01-05T02:31:48.117" Title="Binary tree for 2 elements" Tags="|algorithms|" AnswerCount="1" CommentCount="1" ClosedDate="2019-01-05T13:42:26.060" ContentLicense="CC BY-SA 4.0" />
  <row Id="30834" PostTypeId="2" ParentId="30833" CreationDate="2019-01-05T02:31:48.117" Score="1" Body="&lt;p&gt;In your example, the binary tree is correct in that the right node is greater than the parent node, but it is not optimal because the search tree will not be balanced. The fact that the root node is 1 will mean that for the first comparison, the search algorithm will navigate right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is happening is the binary search tree in your example is acting as a linked list with a time complexity of Θ(n) instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can been seen in the &quot;Searching&quot; here: &lt;a href=&quot;https://www.cs.cmu.edu/~adamchik/15-121/lectures/Trees/trees.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.cmu.edu/~adamchik/15-121/lectures/Trees/trees.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29649" LastActivityDate="2019-01-05T02:31:48.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30835" PostTypeId="1" CreationDate="2019-01-05T05:52:13.640" Score="1" ViewCount="96" Body="&lt;p&gt;I would like to do simulations using cellular automata to describe the behavior of influenza. What software do you recommend?&lt;/p&gt;&#xA;" OwnerUserId="29650" LastActivityDate="2022-05-09T06:23:10.027" Title="Software for cellular automota" Tags="|simulation|software|software-recommendation|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30837" PostTypeId="2" ParentId="30835" CreationDate="2019-01-05T11:56:51.357" Score="3" Body="&lt;p&gt;Well &lt;em&gt;Mathematica&lt;/em&gt; provides a reasonable amount of intrinsic functionality for working with Cellular Automata.  It's not an area of the system I have extensive experience of and I don't know how it compares with other software with similar functionality.  You might learn more about it's CA capabilities before breaking open your piggy-bank (maybe digging into your trust fund too, &lt;em&gt;Mathematica&lt;/em&gt;'s not cheap) over at the sister site &lt;a href=&quot;https://mathematica.stackexchange.com&quot;&gt;https://mathematica.stackexchange.com&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond that, I'd have thunk that programming CA's isn't tremendously difficult in many programming languages.&lt;/p&gt;&#xA;" OwnerUserId="1003" LastActivityDate="2019-01-05T11:56:51.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30839" PostTypeId="1" AcceptedAnswerId="30840" CreationDate="2019-01-05T22:28:10.960" Score="1" ViewCount="13926" Body="&lt;p&gt;&lt;strong&gt;Hello all,&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've recently been introduced to Python and Numpy, and am still a beginner in applying it for numerical methods. I've been performing simple 1D diffusion computations. I suppose my question is more about applying python to differential methods. I'm asking it here because maybe it takes some diff eq background to understand my problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;My overall question :&lt;/em&gt;&lt;/strong&gt; &lt;em&gt;Pretty beginner question, I have a picture of a specific example case below.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the benefits/cons of using built-in numpy functions vs using for loops in numerical computational methods (CFD &amp;amp; FEA), is it absolutely necessary to write explicit for loops for time stepping sometimes? &lt;em&gt;(no pun intended)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attached a link to the short excercise I've been practicing below. Thanks for your time! &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;&lt;strong&gt;Discretization Scheme&lt;/strong&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;*&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Discretization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*&#xA;&lt;a href=&quot;https://i.stack.imgur.com/uBlSx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uBlSx.png&quot; alt=&quot;Discretized 1D diffusion eqn, FWD time, Centered Space&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My specific code example&lt;/strong&gt;: How could you get rid of the for loop below ?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dD4nU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dD4nU.png&quot; alt=&quot;for loop&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The link for the excercise : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/04_spreadout/04_01_Heat_Equation_1D_Explicit.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/04_spreadout/04_01_Heat_Equation_1D_Explicit.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29459" LastActivityDate="2019-01-06T01:07:59.030" Title="Python Finite Difference Schemes for 1D Heat Equation: How to express for loop using numpy expression" Tags="|finite-difference|python|fluid-dynamics|numpy|heat-transfer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30840" PostTypeId="2" ParentId="30839" CreationDate="2019-01-06T01:02:32.143" Score="2" Body="&lt;p&gt;numpy arrays and methods are incredibly helpful. They are usually optimized and much faster than looping in python. Always look for a way to use an existing numpy method for your application.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;np.roll()&lt;/code&gt; will allow you to shift and then you just add.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I learned to use &lt;code&gt;convolve()&lt;/code&gt; from comments on &lt;a href=&quot;https://stackoverflow.com/q/32195525/3904031&quot;&gt;How to np.roll() faster?&lt;/a&gt;. I haven't checked if this is faster or not, but it may depend on the number of dimensions. See &lt;a href=&quot;https://physics.stackexchange.com/a/363868/83380&quot;&gt;this answer&lt;/a&gt; for a 2D relaxation of the Laplace equation (electrostatics, a different problem)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this kind of relaxation you'll need a bounding box, so the boolean &lt;code&gt;do_me&lt;/code&gt; is &lt;code&gt;False&lt;/code&gt; on the boundary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that for Jacobi relaxation solutions to the Laplace equation, there are two speed-up methods. I don't know if they can be extended to solving the Heat Diffusion equation, but I'm sure something can be done:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Multigrid_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multigrids&lt;/a&gt;; solve on a coarse (fast) grid, then interpolate to a fine grid and iterate a little longer&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Successive_over-relaxation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Successive Overrelaxation&lt;/a&gt;; &quot;overshoot&quot; the averaging by changing the values by an amount larger than 100% of what you would have with simple averaging. Try values like 110% to 150%, it can go unstable if they are too large.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ic1cb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ic1cb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.ndimage import convolve&#xA;&#xA;T0             = np.zeros(50, dtype=float)&#xA;T0[17:25]      = 1.&#xA;&#xA;T1             = T0.copy()  # method 1  np.roll()&#xA;T2             = T0.copy()  # method 2  convolve()&#xA;&#xA;do_me          = np.ones_like(T0, dtype=bool)&#xA;do_me[[0, -1]] = False    #  keep the boundaries of your bounding box fixed&#xA;&#xA;a              = 0.01&#xA;&#xA;hold_1         = [T0.copy()]&#xA;for i in range(10001):&#xA;    Ttemp      = T1 + a*(np.roll(T1, +1) + np.roll(T1, -1) - 2*T1)&#xA;    T1[do_me]  = Ttemp[do_me]&#xA;    if not i%1000:&#xA;        hold_1.append(T1.copy())&#xA;&#xA;hold_2         = [T0.copy()]&#xA;kernel         = np.array([a, (1 - 2.*a), a])&#xA;&#xA;for i in range(10001):&#xA;    Ttemp      = convolve(T2, kernel)&#xA;    T2[do_me]  = Ttemp[do_me]&#xA;    if not i%1000:&#xA;        hold_2.append(T2.copy())&#xA;&#xA;if True:&#xA;    plt.figure()&#xA;    plt.subplot(1, 2, 1)&#xA;    for thing in hold_1:&#xA;        plt.plot(thing)&#xA;&#xA;    plt.subplot(1, 2, 2)&#xA;    for thing in hold_2:&#xA;        plt.plot(thing)&#xA;&#xA;    plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17869" LastEditorUserId="17869" LastEditDate="2019-01-06T01:07:59.030" LastActivityDate="2019-01-06T01:07:59.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30845" PostTypeId="1" CreationDate="2019-01-06T17:12:48.413" Score="3" ViewCount="1673" Body="&lt;p&gt;I have been reading upon numerical techniques that are used to solve stiff ordinary differential equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the description given &lt;a href=&quot;http://www.scholarpedia.org/article/Backward_differentiation_formulas&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, I could follow the steps till equation (5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am finding it difficult to understand how equation 6 is obtained and the calculations involved in computing the coefficients of BDF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone recommend a reference in which these steps have been detailed?&lt;/p&gt;&#xA;" OwnerUserId="29087" LastEditorUserId="29087" LastEditDate="2019-01-09T03:12:20.427" LastActivityDate="2019-01-09T03:12:20.427" Title="Derivation of backward differentiation formulas(BDF)" Tags="|ode|reference-request|numerics|stiffness|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30847" PostTypeId="1" CreationDate="2019-01-07T11:36:38.440" Score="3" ViewCount="140" Body="&lt;p&gt;I have a following problem : I want to transform a function &lt;span class=&quot;math-container&quot;&gt;$g(\omega) \exp(i C \omega^2)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$g(\omega)$&lt;/span&gt; is real and limited. It changes slowly compered to &lt;span class=&quot;math-container&quot;&gt;$\exp(i C \omega^2)$&lt;/span&gt;. I have a black box that I can ask for the values of that function at any specific point in frequency domain. I want to get &lt;span class=&quot;math-container&quot;&gt;$g(\omega) \exp(i C \omega^2)$&lt;/span&gt; representation in time domain. If I try to apply a uniform DFT then (if I understand DFT correctly) I have to sample my &lt;span class=&quot;math-container&quot;&gt;$g(\omega)$&lt;/span&gt; with sampling frequency that correspond to the highest frequency of &lt;span class=&quot;math-container&quot;&gt;$\exp(i C \omega^2)$&lt;/span&gt;. And that would be &lt;span class=&quot;math-container&quot;&gt;$C \omega_{max}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\omega_{max}$&lt;/span&gt; is the largest &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; for which &lt;span class=&quot;math-container&quot;&gt;$g(\omega)$&lt;/span&gt; is non-zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately doing it like that requires too much memory since &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is extremely large. I calculated that sampling would required &lt;span class=&quot;math-container&quot;&gt;$\approx 10^{14}$&lt;/span&gt; points. I have no idea how to proceed. Would NUFFT-II be better then FFT? &lt;/p&gt;&#xA;" OwnerUserId="29660" LastActivityDate="2019-01-07T17:53:53.793" Title="DFT of $g(\omega) \exp(i C \omega^2)$. How to do it ,if uniform sampling requires too much memory?" Tags="|fourier-transform|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30848" PostTypeId="2" ParentId="30847" CreationDate="2019-01-07T12:14:30.960" Score="1" Body="&lt;p&gt;You could try the logarithmic Fourier transform.&#xA;The idea is that if one desires the time values in a log-spaced fashion one also only needs the frequency values in a log-spaced fashion. &#xA;Moreover, if one plugs these log spaced assumptions into the Fourier transform one obtains a convolution type relationship. &#xA;Or in other words, in log spaced coordinates the Fourier transform becomes a convolution.&#xA;This convolution is calculable with the normal Fourier transform, but typically needs much less datapoints. It could be of benefit for your problem as well, since there is the &lt;span class=&quot;math-container&quot;&gt;$\omega^2$&lt;/span&gt; term. However due to the log-spacing some limitations apply, e.g. the &lt;span class=&quot;math-container&quot;&gt;$\omega=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt; value can not be included.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A classical paper on the log-spaced Fourier transform can be found &lt;a href=&quot;https://homepages.dias.ie/~ajones/publications/28.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Since then it has been re-invented several times (and perhaps it is also not the first on this topic).&lt;/p&gt;&#xA;" OwnerUserId="29662" LastActivityDate="2019-01-07T12:14:30.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30850" PostTypeId="2" ParentId="30847" CreationDate="2019-01-07T17:46:14.230" Score="7" Body="&lt;p&gt;Expanding upon my comment above, if you only need a few digits of accuracy you can probably use the method of stationary phase. We can follow the procedure &lt;a href=&quot;https://en.wikipedia.org/wiki/Stationary_phase_approximation&quot; rel=&quot;noreferrer&quot;&gt;on Wikipedia&lt;/a&gt;. We can write the transform as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(t) = \int_{-\infty}^{\infty} g(\omega)e^{i\phi(\omega,t)}d\omega,&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\phi(\omega,t)=C\omega^2-\omega t$&lt;/span&gt;. The principal contribution of the integrand to the value of the integral comes in the vicinity of the stationary point of the phase function &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt;, which can be found by solving&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial\phi}{\partial\omega}=0 \rightarrow \omega_0=t/2C.&#xA;$$&lt;/span&gt;&#xA;The Taylor series expansion of &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; in the vicinity of the stationary point &lt;span class=&quot;math-container&quot;&gt;$\omega_0$&lt;/span&gt; is given, to quadratic degree, by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\phi(\omega,t) \approx \frac{t^2}{4C} + C (\omega-\omega_0)^2.&#xA;$$&lt;/span&gt;&#xA;Substituting this into the transform integral, we find&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(t) \approx \left|g\left(\frac{t}{2C}\right)\right| e^{it^2/4C} \int_{-\infty}^{\infty} e^{iC(\omega-\omega_0)^2}d\omega.&#xA;$$&lt;/span&gt;&#xA;Substituting an approximation with an error term that goes as &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(C^{-1})$&lt;/span&gt; for the remaining highly-oscillatory integral, we find that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(t) \approx \left|g\left(\frac{t}{2C}\right)\right| e^{i(t^2/4C+\pi/4)} \sqrt{\frac{\pi}{C}}.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;span class=&quot;math-container&quot;&gt;$g(\omega)=e^{-\omega^2}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C=10^{12}$&lt;/span&gt;, this approximation is accurate to 6 digits up to &lt;span class=&quot;math-container&quot;&gt;$t=1000$&lt;/span&gt; (verified with Wolfram Alpha).&lt;/p&gt;&#xA;" OwnerUserId="27171" LastEditorUserId="27171" LastEditDate="2019-01-07T17:53:53.793" LastActivityDate="2019-01-07T17:53:53.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30851" PostTypeId="1" AcceptedAnswerId="30857" CreationDate="2019-01-07T23:23:35.910" Score="0" ViewCount="852" Body="&lt;p&gt;I have these three differential equations in which I need to solve numerically:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{dn_0}{dt}= -n_0(t)W_{01}(t) + n_1(t)K_{10}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{dn_1}{dt}= -n_1(t)W_{12}(t) - n_1(t)K_{10} + n_2(t)K_{21} + n_0(t)W_{01}(t)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{dn_2}{dt}= n_1(t)W_{12}(t) - n_2(t)K_{21}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ n_0(0)=1 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ n_0(N)=0 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ n_1(0)=0 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ n_1(N)=1 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ n_2(0)=0 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ n_2(N)=0 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the central finite difference formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{n_{0}(t + \Delta t) - n_{0}(t - \Delta t)}{2\Delta t}=-n_0(t)W_{01}(t) + n_1(t)K_{10}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{n_{1}(t + \Delta t) - n_{1}(t - \Delta t)}{2\Delta t}=-n_1(t)W_{12}(t) - n_1(t)K_{10} + n_2(t)K_{21} + n_0(t)W_{01}(t)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{n_{2}(t + \Delta t) - n_{2}(t - \Delta t)}{2\Delta t}=n_1(t)W_{12}(t) - n_2(t)K_{21}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I determine the functions &lt;span class=&quot;math-container&quot;&gt;$n0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$n1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n2$&lt;/span&gt; knowing that &lt;span class=&quot;math-container&quot;&gt;$n0 + n1 + n2 =1$&lt;/span&gt;, and that the three equations are coupled?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I could not understand how to calculate the derivatives, how can I determine their value with the finite difference method without knowing the functions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please help me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hmm, so I did it now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{n_{0_{i+1}} - n_{0_{i-1}}}{2\Delta t}=&#xA;-n_0(t_i)W_{01}(t_i) + n_1(t_i)K_{10}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$$ t_i= (i-1)\Delta t $$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$$ i= 2,3,... N-1 $$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$$ \Delta t=\frac{b-a}{N-1} $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;knowing that this gives a system of the type &lt;span class=&quot;math-container&quot;&gt;$$ Au=B$$&lt;/span&gt;&#xA;I can not think of a way to fill the array. I say, on the right side of the equation, what values &lt;span class=&quot;math-container&quot;&gt;$ n_0(t_i)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_1(t_i)$&lt;/span&gt; assume as &lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt;  varies?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to implement in scilab, but to no avail&lt;/p&gt;&#xA;" OwnerUserId="29661" LastActivityDate="2020-07-19T18:08:27.363" Title="coupled equations with finite difference method" Tags="|matlab|computational-physics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30855" PostTypeId="2" ParentId="30845" CreationDate="2019-01-08T14:00:13.277" Score="4" Body="&lt;p&gt;In BDF schemes for &lt;span class=&quot;math-container&quot;&gt;$\dot y = f$&lt;/span&gt;, one uses &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(t_n)=\dot y(t_n)&#xA;$$&lt;/span&gt;&#xA;and tries to approximate &lt;span class=&quot;math-container&quot;&gt;$\dot y(t_n)\approx \sum_{j=0}^k\alpha_k y_{n-j}$&lt;/span&gt; by the current value &lt;span class=&quot;math-container&quot;&gt;$y_n$&lt;/span&gt; (that is to be computed) and the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; previously computed approximations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the presented &lt;a href=&quot;http://www.scholarpedia.org/article/Backward_differentiation_formulas&quot; rel=&quot;nofollow noreferrer&quot;&gt;approach&lt;/a&gt;, in &lt;span class=&quot;math-container&quot;&gt;$(5)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is approximated as a polynomial &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; fitted to &lt;span class=&quot;math-container&quot;&gt;$y_{n-j}$&lt;/span&gt;, so that the time derivative of the polynomial at &lt;span class=&quot;math-container&quot;&gt;$t_n$&lt;/span&gt; approximates &lt;span class=&quot;math-container&quot;&gt;$\dot y(t_n)$&lt;/span&gt; as desired. With that, for a given approximation order &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, one can read of the coefficients &lt;span class=&quot;math-container&quot;&gt;$\alpha_j$&lt;/span&gt; by evaluating &lt;span class=&quot;math-container&quot;&gt;$\dot p$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$t_n$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\quad \dot y(t_n) \approx \dot p(t_n) = \frac{1}{h}(y_n - y_{n-1})&#xA;$$&lt;/span&gt;&#xA;which gives that &lt;span class=&quot;math-container&quot;&gt;$h\dot y(t_n)$&lt;/span&gt; is approximated by &lt;span class=&quot;math-container&quot;&gt;$$1\cdot y_n + (-1)\cdot y_{n-1}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$k=2$&lt;/span&gt; the terms read:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;k=2: \quad \dot y(t_n) \approx \dot p(t_n) = \frac{1}{h}(y_n - y_{n-1})+\frac{1}{2h^2}[(t_n-t_n)\nabla^2y_n + (t_n-t_{n-1})\nabla^2y_n]&#xA;$$&lt;/span&gt;&#xA;which, with &lt;span class=&quot;math-container&quot;&gt;$t_n-t_{n-1}=h$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\nabla^2y_n = y_n - 2y_{n-1} + y_{n-2}$&lt;/span&gt; gives that &lt;span class=&quot;math-container&quot;&gt;$h\dot y(t_n)$&lt;/span&gt; is approximated by &lt;span class=&quot;math-container&quot;&gt;$$\frac{3}{2}\cdot y_n + (-2)\cdot y_{n-1} + \frac{1}{2}y_{n-2}$$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And so on...&lt;/p&gt;&#xA;" OwnerUserId="4244" LastActivityDate="2019-01-08T14:00:13.277" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30856" PostTypeId="1" CreationDate="2019-01-09T16:33:00.403" Score="2" ViewCount="290" Body="&lt;p&gt;I am trying to understand how stiff differential equations are solved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance the equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial y}{\partial t} =  \alpha\frac{\partial ^2 y}{\partial z^2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can be solved using an ode solver by discretizing z-direction at i th node&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{dy_i}{dt} =  \alpha\frac{y_{i+1}-2y_i + y_{i-1}}{\Delta z^2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This ode is often solved using ode15s. From what I understand, stiff solver uses numerical differentiation formals like backward differentiation formula(BDF) to approximate the derivative.&#xA;Considering a two-step BDF formula,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\dot y(t_n)= \frac{\frac{3}{2}\cdot y_n + (-2)\cdot y_{n-1} + \frac{1}{2}y_{n-2}}{\Delta t }$$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to proceed from here.&#xA;Instead of using the solver, I am trying to implement the above BDF to understand how the solver works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to ask for suggestions on how to proceed from here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Equating (2) and (3) equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{\frac{3}{2}\cdot y^n_i + (-2)\cdot y^{n-1}_i + \frac{1}{2}y^{n-2}_i}{\Delta t }=  \alpha\frac{y^n_{i+1}-2y^n_i + y^n_{i-1}}{\Delta z^2}$$&lt;/span&gt;&#xA;Here, n represents time and i respresents space.&lt;/p&gt;&#xA;" OwnerUserId="29087" LastEditorUserId="29087" LastEditDate="2019-01-10T08:47:00.243" LastActivityDate="2019-02-11T18:00:43.527" Title="How does a stiff equation solver work?" Tags="|pde|finite-difference|numerics|stiffness|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30857" PostTypeId="2" ParentId="30851" CreationDate="2019-01-09T18:06:19.833" Score="2" Body="&lt;p&gt;You need to think of your coupled equations as one equation.&lt;/p&gt;&#xA;&lt;p&gt;Be n=&lt;span class=&quot;math-container&quot;&gt;$\left(\begin{array}{c}n_0\\n_1\\n_2\end{array}\right)$&lt;/span&gt;, you can write it as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d}{dt}n= A n$$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$$A=\left(\begin{array}{ccc}-W_{01}(t)&amp;amp;K_{10} &amp;amp;0\\W_{01}(t)&amp;amp;-W_{12}(t) - K_{10}&amp;amp;K_{21}\\0&amp;amp;W_{12}(t)&amp;amp;-K_{21}\end{array}\right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the constraint you could either derive a new set of equations by explicitly  solving the constraint for &lt;span class=&quot;math-container&quot;&gt;$n_2=1-n_0-n_1$&lt;/span&gt; and plug it in or if you want to be flexible with linear constraints of such type, you could do the following.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Linear Constraints&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can rewrite the linear constraint equation as &lt;span class=&quot;math-container&quot;&gt;$$Bn=l$$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$$B=\left(\begin{array}{ccc}1&amp;amp;1&amp;amp;1\end{array}\right)$$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$$l=1.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now be &lt;span class=&quot;math-container&quot;&gt;$I_B$&lt;/span&gt; a matrix of a basis of the image space of &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$N_B$&lt;/span&gt; a matrix of a basis of the null space of B, we can do a change of basis by writing &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$$n=I_B n_{I}+ N_B n_{N}.$$&lt;/span&gt; The matrices &lt;span class=&quot;math-container&quot;&gt;$I_B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$N_B$&lt;/span&gt; can be obtained by using a singular value decomposition on &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. Our new variables are &lt;span class=&quot;math-container&quot;&gt;$n_I$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_N$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Using this basis in the constraint equation leads to &lt;span class=&quot;math-container&quot;&gt;$$B I_B n_I+B N_B n_N=l.$$&lt;/span&gt; By construction &lt;span class=&quot;math-container&quot;&gt;$B N_B n_N= 0$&lt;/span&gt;,  we solve the linear system for &lt;span class=&quot;math-container&quot;&gt;$n_I$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$$B I_B n_I =  l.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The same basis can be used in the ODE system&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d}{dt}\left(I_B n_I + N_B n_N \right)= A \left(I_B n_I + N_B n_N \right)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{d}{dt}I_B n_I=0$&lt;/span&gt; because it is a constant of motion due to the constraint equation.&lt;/p&gt;&#xA;&lt;p&gt;Now multiplying both sides with &lt;span class=&quot;math-container&quot;&gt;$N_B^T$&lt;/span&gt; leads to the reduced final system&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d}{dt} n_N = N_B^T A I_B n_I + N_B^T A N_B n_N .$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can apply your favorite time integrator on the vectorized version of the equation for the variables &lt;span class=&quot;math-container&quot;&gt;$n_N$&lt;/span&gt;, since we haven taken care of the constraints. Remember, the first term on the rhs &lt;span class=&quot;math-container&quot;&gt;$N_B^T A I_B n_I$&lt;/span&gt; is a known function of &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, since we solved already for &lt;span class=&quot;math-container&quot;&gt;$n_I$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Once you have obtained a solution for &lt;span class=&quot;math-container&quot;&gt;$n_N$&lt;/span&gt;, you can use our change of basis to obtain the solution in &lt;span class=&quot;math-container&quot;&gt;$n_0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$n_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Time Integration&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The simplest method is for sure the &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler method&lt;/a&gt;, which is in this case leads to &lt;span class=&quot;math-container&quot;&gt;$$n_N^{i+1}=n_N^i+\Delta t \left(N_B^T A(t_i) I_B n_I + N_B^T A(t_i) N_B n_N^i\right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Due to the dependency of the next step from the previous one, you cannot simply build up a system and solve it. Additionally, since &lt;span class=&quot;math-container&quot;&gt;$W_{01}(t)$&lt;/span&gt; is a function of time, it is possible to a have a stiff or non stiff ODE system. So a recommendation is difficult, a good overview of possible solvers and there application can be found &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Also you mention too many initial conditions. You only need at one point in time, e.g. the initial conditions at &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$$n_N(0)=N_B^T\left(\left(\begin{array}{c}n_0(0)\\n_1(0)\\n_2(0)\end{array}\right)-I_B n_I\right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't know scilab, but it seems that they have &lt;a href=&quot;https://help.scilab.org/docs/6.0.1/en_US/ode.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;time integrators for ODEs&lt;/a&gt;, too. So check their documentation how to use them.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="18981" LastEditDate="2020-07-19T18:08:27.363" LastActivityDate="2020-07-19T18:08:27.363" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30859" PostTypeId="1" CreationDate="2019-01-10T06:17:19.770" Score="0" ViewCount="1532" Body="&lt;p&gt;I have many STL files and I want to reduce their size, so I use Gmsh in this way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gmsh -2 -bin -format vtk -o file.vtk file.stl -0&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It reduces the size from 7 MB to 1.2 MB, but if I use the option &quot;Recombine 2D&quot; the size is reduced to 800 KB. I cannot find a way to use recombine in the command line, so I try with the next script file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Merge &quot;malla.stl&quot;;&#xA;Recombine Surface {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it doesn't work. The only way that works Recombine 2D is in the graphical interface. How can I automate the command &quot;Recombine 2D&quot;?&lt;/p&gt;&#xA;" OwnerUserId="29688" LastEditorUserId="20688" LastEditDate="2019-01-10T12:36:17.157" LastActivityDate="2019-01-10T12:36:17.157" Title="Gmsh: Recombine 2D in script file or command line" Tags="|vtk|gmsh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30860" PostTypeId="2" ParentId="30856" CreationDate="2019-01-10T06:23:26.997" Score="0" Body="&lt;p&gt;The newest time is supposed to be the unknown, so the implicit method can be reshaped into a root finding equation system by moving the lhs to the rhs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems you discretize y wrong though, the spatial discretization needs to be done separately from the temporal discretization, so the answer is a matrix if you keep the whole history. (Note: you have only shown the expression for the time derivative, not an expression for an integral or the taylor expansion, and shouldn't the lhs for the spatial stencil be the spatial derivative?)&lt;/p&gt;&#xA;" OwnerUserId="19947" LastEditorUserId="19947" LastEditDate="2019-01-10T07:01:08.610" LastActivityDate="2019-01-10T07:01:08.610" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30861" PostTypeId="2" ParentId="30856" CreationDate="2019-01-10T06:31:12.123" Score="-2" Body="&lt;p&gt;Using &lt;code&gt;ode15s&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[t,y] = ode15s( @(t,z) fun(t,z,alpha,dz),tspan,y0);&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dy = fun(t,y,alpha,dz)&#xA;dy = zeros(size(y));&#xA;dy(1) =  % b.c.&#xA;dy(2:end-1) = alpha*(y(3:end) -2*y(2:end-1) + y(1:end-2) )/dz^2&#xA;dy(end) = % b.c.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;dy(1)&lt;/code&gt; and &lt;code&gt;dy(end)&lt;/code&gt; depend of the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to implement your own numeric method, you can search of any method to solve system of linear ODEs.&lt;/p&gt;&#xA;" OwnerUserId="29688" LastActivityDate="2019-01-10T06:31:12.123" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30864" PostTypeId="2" ParentId="30859" CreationDate="2019-01-10T12:30:50.953" Score="1" Body="&lt;p&gt;&lt;code&gt;Recombine Surface{1}&lt;/code&gt; - recombines only the triangles for the surface #1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To allow recombination of all the surfaces, you need either &lt;code&gt;Mesh.RecombineAll=1&lt;/code&gt; or &lt;code&gt;Recombine Surface {:};&lt;/code&gt;. Those commands will perform it on all the surfaces.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I find it a bit strange that you are using &lt;code&gt;Recombine&lt;/code&gt; commands for that purpose. They are usually used to create a structured rectangular/quad mesh in certain regions of the model rather than reduce file size.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-01-10T12:30:50.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30867" PostTypeId="1" CreationDate="2019-01-10T17:08:11.643" Score="2" ViewCount="51" Body="&lt;p&gt;I want to minimize the following with respect to parameters &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{k = 1}^{K} f(A_{k}, B)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$A_k$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; different data-sets and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is a matrix of parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I do this by a gradient descent where:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;randomly pick &lt;em&gt;one&lt;/em&gt; matrix &lt;span class=&quot;math-container&quot;&gt;$A_{i}$&lt;/span&gt; among &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; matrices.&lt;/li&gt;&#xA;&lt;li&gt;perform a gradient step that minimizes &lt;span class=&quot;math-container&quot;&gt;$f(A_{i}, B)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;repeat steps 1 and 2 until convergence.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is the logarithm of a linear function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After searching online, this looks like a stochastic gradient descent.&#xA;Is this really an SGD?&lt;/p&gt;&#xA;" OwnerUserId="28896" LastEditorUserId="20688" LastEditDate="2019-06-22T22:35:35.933" LastActivityDate="2019-06-22T22:35:35.933" Title="Is this a form of stochastic gradient descent?" Tags="|optimization|stochastic|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30868" PostTypeId="1" CreationDate="2019-01-10T17:19:53.590" Score="7" ViewCount="96" Body="&lt;p&gt;Consider&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;I = \int_{-L}^L f(x)dx,&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; is real-valued and analytic on &lt;span class=&quot;math-container&quot;&gt;$[-L,L]$&lt;/span&gt;, but it has a pole in the complex plane whose real part lies in &lt;span class=&quot;math-container&quot;&gt;$[-L,L]$&lt;/span&gt;. Call it &lt;span class=&quot;math-container&quot;&gt;$z_0$&lt;/span&gt;, and assume it is a simple pole. If the imaginary part of &lt;span class=&quot;math-container&quot;&gt;$z_0$&lt;/span&gt; is small compared to the real part, then &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; is sharply peaked in the vicinity of &lt;span class=&quot;math-container&quot;&gt;$x_0=\mathrm{Re}\{z_0\}$&lt;/span&gt;. If this peak is sufficiently sharp and &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; is slowly varying elsewhere, then basic adaptive quadrature methods (e.g., recursive Simpson or Gauss-Legendre) will spend considerable effort attempting to resolve the peak. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An obvious thing to try is to approximate the value of &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; and break the integral up &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;I = I_-+I_+ = \int_{-L}^{x_0}f(x)dx+\int_{x_0}^Lf(x)dx&#xA;$$&lt;/span&gt;&#xA;But doing so only pushes the problem to the endpoints of the integration domain instead of the interior. I wonder if there is a standard change of variables to alleviate this behavior, though?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option could be to try to evaluate &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; by means of contour integration in the complex plane and use of the residue theorem. However, this will not fly if the software implementation of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is only defined for real arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other specialized quadrature methods for these kinds of &quot;peaky&quot; integrands? Ideally I'd like to make no further assumptions on &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; execpt that it's real, analytic, and slowly varying away from &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt;. But I am also open to answers that require more assumptions on &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; if there is no general answer.&lt;/p&gt;&#xA;" OwnerUserId="14469" LastActivityDate="2019-01-10T17:19:53.590" Title="Quadrature methods for peaky integrands" Tags="|quadrature|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30870" PostTypeId="1" AcceptedAnswerId="35912" CreationDate="2019-01-10T21:20:10.987" Score="5" ViewCount="401" Body="&lt;p&gt;I am trying to find the roots of a monotonic function with as few function evaluations as possible.&lt;/p&gt;&#xA;&lt;p&gt;I have approximated a manifold with a piece-wise defined polynomial. The manifold is periodic and so I am only considering its unit cell (one period). I split the unit cell, in general, a parallelogram but in this case a square, into triangles.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2INtF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2INtF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I then approximate each sheet of the manifold within each triangle with a unique quadratic polynomial.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n3KMP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n3KMP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would like to find the root that satisfies the equation&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\sum_{i}^{\mathrm{sheets}} \sum_{j}^{\mathrm{triangles}} \int p_{i,j}(x,y) \, \mathrm{d}C_{i,j} - A = 0&#xA;\end{equation}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is a sum over the sheets of the manifold, &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; is a sum over the triangular tiles, &lt;span class=&quot;math-container&quot;&gt;$p_{i,j}$&lt;/span&gt; is the second-degree polynomial approximation of the manifold's &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th sheet within the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th tile, and &lt;span class=&quot;math-container&quot;&gt;$C_{i,j}$&lt;/span&gt; is the region within a level curve of the polynomial approximation of the manifold's &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th sheet within the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th tile. Here is a plot of the &lt;span class=&quot;math-container&quot;&gt;$C_{i,j}$&lt;/span&gt; for each triangle and sheet for some estimate of the root.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fz8uK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fz8uK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Said another way, I would like to find an isovalue where the area within the level curves of the polynomials, regions where the polynomials are less than the isovalue, is some predetermined value &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;At the moment I am using the bisection method, which is very slow because at each iteration it takes a significant amount of time to interpolate the manifold and then calculate the level curves and their containing areas. I may have hundreds of triangles and tens of sheets.&lt;/p&gt;&#xA;&lt;p&gt;I also tried the &lt;em&gt;regula falsi&lt;/em&gt; method but ran into cases where its convergence was worse than the bisection method.&lt;/p&gt;&#xA;" OwnerUserId="29528" LastEditorUserId="43139" LastEditDate="2023-08-03T20:06:34.763" LastActivityDate="2023-08-03T20:06:34.763" Title="Efficient root finding algorithm for monotonic function" Tags="|computational-geometry|roots|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="30871" PostTypeId="2" ParentId="338" CreationDate="2019-01-10T21:23:21.670" Score="1" Body="&lt;p&gt;I wrote a small Python package called &lt;a href=&quot;https://github.com/BPSB/OOGillespie&quot; rel=&quot;nofollow noreferrer&quot;&gt;OOGillespie&lt;/a&gt; whose main feature is that it is rather flexible and embraces object-oriented programming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Efficiency was not the main priority when writing OOGillespie, but I tried to keep things fast, where it did not lead to a major slowdown.&#xA;OOGillespie will probably not be able to compete with other implementations for standard tasks, but it may be the only implementation that can handle some non-standard ones.&#xA;If your bottleneck is computing transition rates from large matrices, OOGillespie may be competitive again by allowing you to use NumPy matrix operations for this purpose.&#xA;Finally, depending on how experienced you are with classes, OOGillespie may cut down your programming time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of the day, I do not think that you can simply rank Gillespie implementations without knowing the specific application as a major point is what kind of events you have and how you can calculate tehm.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2019-01-10T21:23:21.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30872" PostTypeId="1" CreationDate="2019-01-10T21:56:10.653" Score="4" ViewCount="104" Body="&lt;p&gt;I want to compute the matrix&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A = \sum_{i=1}^N v_i v_i^T&#xA;$$&lt;/span&gt;&#xA;where each &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; is a given vector of length &lt;span class=&quot;math-container&quot;&gt;$2500$&lt;/span&gt;, so that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$2500 \times 2500$&lt;/span&gt;, and my &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is about 2 million. Rather than call DSYR 2 million times, I found I can speed up the calculation significantly, by storing the &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; in block matrices, and then calling DSYRK when the block matrix is full. I define a &lt;span class=&quot;math-container&quot;&gt;$\textrm{block_size} \times 2500$&lt;/span&gt; matrix &lt;span class=&quot;math-container&quot;&gt;$W_k$&lt;/span&gt; as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;W_k = \pmatrix{ v_1^T \\ \vdots \\ v_b^T }&#xA;$$&lt;/span&gt;&#xA;so that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A = \sum_{k=1}^{\textrm{nblocks}} W_k^T W_k&#xA;$$&lt;/span&gt;&#xA;The reason I set the rows of &lt;span class=&quot;math-container&quot;&gt;$W_k$&lt;/span&gt; equal to the vectors &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt; and not the columns is because I use C and this way the incX of the rows will be 1 which I think helps to speed up the BLAS operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am finding this procedure is still slower than I would like, and &lt;strong&gt;my question is how do I choose the &lt;span class=&quot;math-container&quot;&gt;$\textrm{block_size}$&lt;/span&gt; (i.e. number of rows of &lt;span class=&quot;math-container&quot;&gt;$W_k$&lt;/span&gt;) to get the best performance?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am choosing &lt;span class=&quot;math-container&quot;&gt;$\textrm{block_size} = 50000$&lt;/span&gt;, which I only chose so that the total size of the &lt;span class=&quot;math-container&quot;&gt;$W_k$&lt;/span&gt; storage matrix is 1GB (&lt;span class=&quot;math-container&quot;&gt;$50000 \times 2500 \times 8 = 1 GB$&lt;/span&gt;). Is there a better choice of &lt;span class=&quot;math-container&quot;&gt;$\textrm{block_size}$&lt;/span&gt; (number of rows of &lt;span class=&quot;math-container&quot;&gt;$W_k$&lt;/span&gt;) to get optimal performance?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it matters I am using the ATLAS CBLAS library on a machine with 24 cores.&lt;/p&gt;&#xA;" OwnerUserId="28440" LastActivityDate="2019-01-10T21:56:10.653" Title="Block matrix and DSYRK" Tags="|linear-algebra|matrix|blas|dense-matrix|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30874" PostTypeId="1" AcceptedAnswerId="30877" CreationDate="2019-01-11T13:33:53.467" Score="4" ViewCount="121" Body="&lt;p&gt;Is it possible to construct a sequence that converges in theory but when computed numerically with a computer program is diverging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel that today our computer programs doesn't allow such pathological cases so I think I might found such pathological sequence working in simple precision.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course I want to avoid the case of overflows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, what would be an ill-conditioned sequence ? I have some examples about ode, matrix, ... but does there exist an ill-conditioned notion for sequences (defined by a recurrence relation) ?&lt;/p&gt;&#xA;" OwnerUserId="29701" LastEditorUserId="29701" LastEditDate="2019-01-11T14:11:37.467" LastActivityDate="2019-01-11T21:54:49.890" Title="Analytical convergent sequence and numerical divergent sequence" Tags="|convergence|numerics|precision|conditioning|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30876" PostTypeId="2" ParentId="30874" CreationDate="2019-01-11T19:44:07.093" Score="6" Body="&lt;p&gt;In my opinion, an example could be the calculation of the minimal solution of a three term recurrence relation (TTRR) &lt;span class=&quot;math-container&quot;&gt;$$y_{n+1} +a_{n}y_{n}+b_{n}y_{n-1} = 0, \quad n=1,2,3,\ldots$$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the Bessel &lt;span class=&quot;math-container&quot;&gt;$J_{n}(x)$&lt;/span&gt; function for a fixed &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; satisfies the TTRR &lt;span class=&quot;math-container&quot;&gt;$$y_{n+1}-\frac{2n}{x}y_{n}+y_{n-1} = 0$$&lt;/span&gt; Suppose you know &lt;span class=&quot;math-container&quot;&gt;$y_{0} = J_{0}(1)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y_{1} = J_{1}(1)$&lt;/span&gt; and you wish to calculate &lt;span class=&quot;math-container&quot;&gt;$y_{100} = J_{100}(1)$&lt;/span&gt; by just applying the TTRR in the forward direction (i.e. increasing &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;), you will see that in finite precision arithmetic your solution will be perturbed by the smallest round-off error and instead of a decreasing function, you will notice an increasing function. See the figure below for the relative error between &lt;span class=&quot;math-container&quot;&gt;$J_n(1)$&lt;/span&gt; calculated using the increasing TTRR and the exact value using state-of-the-art algorithms. Even if you would do this in fixed precision using a very high precision you will always run into trouble for high &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would this be an example of what you are looking for? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/diKPN.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/diKPN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2019-01-11T19:44:07.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30877" PostTypeId="2" ParentId="30874" CreationDate="2019-01-11T20:49:56.650" Score="10" Body="&lt;p&gt;Jean-Michel Muller, et. al., &quot;Handbook of Floating-Point Arithmetic 2nd ed.&quot;, Birkhäuser 2018, gives the following example due to Muller, specifically constructed to deliver incorrect results with floating-point evaluation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;{u_{0} = 2,\\  &#xA;u_{1} = -4,\\  &#xA;u_{n} = 111 - \frac{1130}{u_{n-1}} + \frac{3000}{u_{n-1}u_{n-2}},\&amp;gt;\&amp;gt;\&amp;gt;\&amp;gt;n \ge 2.}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, this sequence converges to &lt;span class=&quot;math-container&quot;&gt;$6$&lt;/span&gt;. However, evaluated numerically it appears to approach &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paul Zimmermann, et. al. &quot;Computational Mathematics with SageMath&quot;, SIAM 2018, gives the following example due to Marc Deléglise:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;{&#xA;u_{0} = \frac{1}{3},\\&#xA;u_{n+1} = 4u_{n}-1&#xA;}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically, this sequence is stationary, but evaluated in floating-point arithmetic it diverges to &lt;span class=&quot;math-container&quot;&gt;$-\infty$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20458" LastEditorUserId="20458" LastEditDate="2019-01-11T21:54:49.890" LastActivityDate="2019-01-11T21:54:49.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30880" PostTypeId="2" ParentId="24175" CreationDate="2019-01-13T15:25:03.487" Score="0" Body="&lt;p&gt;This question has appeared a long time, anyway I cite the book:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Rauber, Thomas; Rünger, Gudula&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1007/978-3-642-37801-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Parallel programming for multicore and cluster systems&lt;/strong&gt;&lt;/a&gt;, Berlin: Springer (ISBN 978-3-642-37800-3/hbk; 978-3-642-37801-0/ebook). xiii, 516&amp;nbsp;p. (2013). &lt;a href=&quot;https://zbmath.org/?q=an:1295.68012&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL1295.68012&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cite two extracts from the &lt;a href=&quot;https://www.springer.com/us/book/9783642378003#aboutBook&quot; rel=&quot;nofollow noreferrer&quot;&gt;book description&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Rauber and Rünger take up these recent developments in processor architecture by giving detailed descriptions of parallel programming techniques that are necessary for developing efficient programs for multicore processors as well as for parallel cluster systems and supercomputers.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The main goal of the book is to present parallel programming techniques that can be used in many situations for a broad range of application areas and which enable the reader to develop correct and efficient parallel programs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is not related to a specific library but I found it a good book for the parallel part of the HPC world.&lt;/p&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2019-01-13T15:25:03.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30884" PostTypeId="1" CreationDate="2019-01-14T16:16:00.147" Score="5" ViewCount="178" Body="&lt;p&gt;I need to compute elliptic integrals of the third kind with complex arguments, preferably in C++. Is there code out there to do this? I have discovered the Arb library, but that does much more than I would like to do, and would be much slower for my particular application.&lt;/p&gt;&#xA;" OwnerUserId="29489" LastEditorUserId="20458" LastEditDate="2019-01-15T13:52:14.513" LastActivityDate="2019-01-15T13:52:14.513" Title="Best way to numerically compute elliptic integrals of the third kind with complex arguments?" Tags="|integration|special-functions|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="30885" PostTypeId="1" AcceptedAnswerId="30886" CreationDate="2019-01-14T18:04:11.233" Score="1" ViewCount="6549" Body="&lt;p&gt;I have to compute &lt;span class=&quot;math-container&quot;&gt;$CA^{-1}B$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$CA^{-1}x$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A,B,C$&lt;/span&gt; are conformable matrices and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read that the a very computationally stable way to compute these inverses is by computing the Cholesky Decomposition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't see how one can use that decomposition to compute Both terms I'm interested in...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="24970" LastActivityDate="2019-01-14T20:32:09.750" Title="Computing the Inverse of a matrix, using the Cholesky decomposition" Tags="|linear-algebra|inverse|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30886" PostTypeId="2" ParentId="30885" CreationDate="2019-01-14T20:32:09.750" Score="6" Body="&lt;p&gt;The Cholesky decomposition [the function dpotrf() in LAPACK] factors &lt;span class=&quot;math-container&quot;&gt;$\mathbf A = \mathbf L \mathbf L^{\mathrm T}$&lt;/span&gt;, or alternatively &lt;span class=&quot;math-container&quot;&gt;$\mathbf A^{-1} = \left(\mathbf L \mathbf L^\mathrm T \right)^{-1} = \mathbf L^{-\mathrm T}\mathbf L^{-1}$&lt;/span&gt;. If you insert the latter representation your other expressions you'll see how you can compute them efficiently:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf C \mathbf A^{-1} \mathbf B = \mathbf C \mathbf L^{-T}\mathbf L^{-1} \mathbf B$&lt;/span&gt;: if you allow yourself to overwrite &lt;span class=&quot;math-container&quot;&gt;$\mathbf B$&lt;/span&gt;, the quantity &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\tilde B} = \mathbf L^{-T}\mathbf L^{-1} \mathbf B$&lt;/span&gt; can be computed efficiently using two consecutive triangle\matrix backsolutions [the function dtrsm() in BLAS]. Then the sought quantity &lt;span class=&quot;math-container&quot;&gt;$\mathbf C \mathbf{\tilde B}$&lt;/span&gt; can be computed using matrix-matrix multiplication [the function dgemm() in BLAS]. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf C \mathbf A^{-1} \mathbf x = \mathbf C \mathbf L^{-T}\mathbf L^{-1} \mathbf x$&lt;/span&gt;: similar approach, overwrite &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\tilde x} = \mathbf L^{-T}\mathbf L^{-1} \mathbf x$&lt;/span&gt; using two triangle\vector backsolutions [dtrsv() in BLAS]. Then &lt;span class=&quot;math-container&quot;&gt;$\mathbf C \mathbf {\tilde x}$&lt;/span&gt; follows from matrix-vector multiplication [dgemv() in BLAS].&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It is worth mentioning that a symmetric product involving &lt;span class=&quot;math-container&quot;&gt;$\mathbf A^{-1}$&lt;/span&gt; leads to an especially efficient algorithm. For example, the quantity &lt;span class=&quot;math-container&quot;&gt;$\mathbf B^{\mathrm T} \mathbf A^{-1} \mathbf B = \mathbf B^{\mathrm T} \mathbf L^{-T}\mathbf L^{-1} \mathbf B $&lt;/span&gt; can be found by introducing &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\tilde B} = \mathbf L^{-1} \mathbf B$&lt;/span&gt; [requires only one dstrm() call] and then using symmetric-rank-k multication to form &lt;span class=&quot;math-container&quot;&gt;$\mathbf {\tilde B}^{\mathrm T} \mathbf {\tilde B}$&lt;/span&gt; [the function dsyrk() in BLAS]. This is a factor of 2 savings. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A similar idea can be applied to computing the quadratic form &lt;span class=&quot;math-container&quot;&gt;$\mathbf x^{\mathrm T}\mathbf A^{-1} \mathbf x$&lt;/span&gt; [one dtrsv() call, followed by ddot()]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2019-01-14T20:32:09.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30887" PostTypeId="1" CreationDate="2019-01-14T22:35:47.893" Score="1" ViewCount="577" Body="&lt;p&gt;I am currently looking into post-processing of simulation data using Paraview. I would like to compute certain integrals of field quantities. As an example, consider the following surface integral of an electric field in a simple situation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\iint_{S}^{} \vec{E}\cdot\vec{\omega}=\Phi $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The surface &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is shown in the following picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8N7VL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8N7VL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can define a surface in ParaView and I can compute the surface normals &lt;span class=&quot;math-container&quot;&gt;$\vec{w}$&lt;/span&gt; (white arrows in he picture). In order to compute the integral in question, I guess I would have to use a calculator to compute &lt;span class=&quot;math-container&quot;&gt;$\vec{E}\cdot\vec{\omega}$&lt;/span&gt; and then use the &quot;Integrate Variables&quot; Filter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is, that I can only add the calculator to the &quot;case0001.vtu tree&quot; or the &quot;plane tree&quot; in the pipeline browser such that it's impossible to select both the vectors &quot;electric fields&quot; and &quot;normals&quot; in the calculator to compute the integrand &lt;span class=&quot;math-container&quot;&gt;$\vec{E}\cdot\vec{\omega}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how I could use a calculator to compute the integrand in Paraview?&lt;/p&gt;&#xA;" OwnerUserId="29610" LastActivityDate="2019-01-14T22:35:47.893" Title="Computing a Flux Integral in Paraview" Tags="|paraview|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30888" PostTypeId="1" CreationDate="2019-01-15T07:45:59.423" Score="1" ViewCount="412" Body="&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; How do I simulate a hole, at the bottom of a (full) water tank?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am attempting to &lt;strong&gt;simulate water, flowing out of a hole/slit, at the bottom of a tank (Water Domain) (under the influence of gravity); entering into a domain (vertically downward), that contains air (Air Domain), and impinging on a curved surface&lt;/strong&gt;. I am using COMSOL Multiphysics 5.4/5.3a. The idea is to simulate (in 2D) and obtain data on:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;1. The distribution of water, as it flows along the surface, on two sides of the surface (for a 2D Simulation), for different initial points of impingement.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;2. The (distribution of) stress produced, due to the impinging water, on the (different points of the) curved surface.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked through a plethora of papers, as well as tutorials and none of them include, &lt;strong&gt;“liquid/gas, flowing out, &lt;em&gt;under gravity&lt;/em&gt;, into another domain, containing a different liquid/gas”&lt;/strong&gt;. I am assuming &lt;em&gt;Laminar Flow&lt;/em&gt; and &lt;em&gt;No Slip Boundary Condition&lt;/em&gt;, at the hole &amp;amp; walls and the curved surface.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, as far as I know, I cannot define an Inlet at the interface of the two domains. I have tried Multiphase (Phase Field), with no inlet and 1 outlet in the Air Domain (situated vertically downwards), but it did not converge. A Surface Plot of Fluid Velocity showed water, entering the Air Domain, but after a few more time steps, COMSOL threw a &lt;strong&gt;Did not converge error&lt;/strong&gt;, thereby preventing the water from reaching the curved surface. I have tried the following, hoping to fix the issue:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Turning the “hole”, into a faucet or nozzle (of sorts), with a smooth downward curve: No change in outcome observed - Still not converging.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Defining a Pressure Point Constraint (1 atm) at the top of the Fluid Domain - No change in outcome observed - Still not converging.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any help or hint is much appreciated. Also, if this problem (First paragraph - in Bold text) can be done in any other way (preferably in COMSOL), I’d like to hear that, as well.&lt;/p&gt;&#xA;" OwnerUserId="29716" LastEditorUserId="9667" LastEditDate="2019-01-16T16:24:37.313" LastActivityDate="2019-01-16T16:24:37.313" Title="How to simulate water, falling under gravity, and impinging on a curved surface, which is kept/present in a domain, containing air?" Tags="|fluid-dynamics|computational-physics|comsol|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30889" PostTypeId="1" AcceptedAnswerId="30890" CreationDate="2019-01-15T15:13:36.370" Score="2" ViewCount="264" Body="&lt;p&gt;&lt;strong&gt;Which C standard should be used for computational science code ?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should we keep compatibility with C89/90/ANSI or jump to C99 or C11 ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Context: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Code will use third-party : BLAS, LAPACK, MKL, OpenBlas, Direct Sparse Solvers, ...&lt;/li&gt;&#xA;&lt;li&gt;Parallel computation | HPC | OpenMP | MPI&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="29632" LastEditorUserId="29632" LastEditDate="2019-01-16T12:55:09.063" LastActivityDate="2020-12-11T12:21:54.073" Title="C standard for computational science" Tags="|parallel-computing|sparse-matrix|c|blas|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30890" PostTypeId="2" ParentId="30889" CreationDate="2019-01-15T16:00:05.683" Score="9" Body="&lt;p&gt;In theory, as the original authors, you're free to pick and name a standard, then expect others to follow it. In practise, if you're supporting an HPC system, then your choice is likely to be restricted among the &lt;strong&gt;compiler&lt;/strong&gt; standards that the given system you're using for testing has for its tool stack (you are testing your software, aren't you?). This might involve having to roll your own code to get around various compiler bugs which have slipped into the various implementations out in the wild. This all ought to be documented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of following the herd, then following a moderately recent version of clang's implementation of C11 is now relatively unlikely to leave potential users out in the cold, but don't be too surprised to be contacted by someone trying to build on outlandish hardware with a hand-rolled compiler forked from an ancient version of gcc. It's then up to your project how much free customer support you want to attempt to give them.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2019-01-15T16:00:05.683" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30893" PostTypeId="2" ParentId="30889" CreationDate="2019-01-15T23:54:35.140" Score="7" Body="&lt;p&gt;You should definitely jump to C99, or newer(!). The C99 standard introduced &#xA;the &lt;a href=&quot;https://en.wikipedia.org/wiki/Restrict&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;restrict&lt;/code&gt;&lt;/a&gt; keyword. Loosely speaking, &#xA;with this keyword you can inform the compiler that &lt;code&gt;A[i]&lt;/code&gt; and &lt;code&gt;B[j]&lt;/code&gt; do not access &#xA;the same memory location. In that case the compiler can generate better optimized code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, it makes it easier for the compiler to auto-vectorize code. This is&#xA;essential for achieving high performance on today's CPUs, which usually support SIMD instruction sets &#xA;(such as Intel &lt;a href=&quot;https://en.wikipedia.org/wiki/Streaming_SIMD_Extensions&quot; rel=&quot;noreferrer&quot;&gt;SSE&lt;/a&gt;/&lt;a href=&quot;https://en.wikipedia.org/wiki/Advanced_Vector_Extensions&quot; rel=&quot;noreferrer&quot;&gt;AVX&lt;/a&gt;, Arm NEON, PowerPC Altivec, etc.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Citing the &#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Restrict&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;restrict&lt;/code&gt; Wikipedia page:&lt;/a&gt; &lt;em&gt;The use of &#xA;the restrict keyword in C, in principle, allows non-obtuse C to achieve the &#xA;same performance as the same program written in Fortran.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="5286" LastEditorUserId="5286" LastEditDate="2019-01-16T07:16:27.747" LastActivityDate="2019-01-16T07:16:27.747" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30894" PostTypeId="2" ParentId="593" CreationDate="2019-01-16T08:56:14.003" Score="3" Body="&lt;p&gt;Addition to the accepted answer. Here is a link to an open source implementation of Greengard's and Lee's method: &lt;a href=&quot;https://finufft.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://finufft.readthedocs.io/en/latest/&lt;/a&gt; &#xA;It has wrappers to C, fortran, MATLAB, octave, and python. I believe the FINUFFT is written in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is maintained and used at NYU Courant institute, SFU, Flatiron institute (obviously), University of Texas Austin and Florida state university. At least these are the ones I know of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am myself using an older version, because I am lazy. See: &lt;a href=&quot;https://cims.nyu.edu/cmcl/nufft/nufft.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cims.nyu.edu/cmcl/nufft/nufft.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="19807" LastActivityDate="2019-01-16T08:56:14.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30896" PostTypeId="1" CreationDate="2019-01-17T10:59:12.337" Score="3" ViewCount="847" Body="&lt;p&gt;How can I generate the higher &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; quantum harmonic oscillator wavefunction (in position space) numerically? Here, higher means around &lt;span class=&quot;math-container&quot;&gt;$n=500$&lt;/span&gt;, or say &lt;span class=&quot;math-container&quot;&gt;$n=2000$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;th oscillator wavefunction. &lt;hr/&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The position-space wavefunction of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;th state involves Hermite polynomials of order n (see Griffith's book for the detailed form of the solution). I can generate wavefunction up to &lt;span class=&quot;math-container&quot;&gt;$n\approx 160$&lt;/span&gt; only using 'gsl' library in C because the library doesn't offer Hermite polynomials of order higher than &lt;span class=&quot;math-container&quot;&gt;$\approx 170$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get rid of this limitation, I found that the higher &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; wavefunctions can be achieved by using 'raising operator', and then repeatedly acting it on the previous wavefunction to get the next wavefunction, and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Raising operator:  &lt;span class=&quot;math-container&quot;&gt;$a^{\dagger} = \frac{1}{\sqrt{2}} \left(x - \frac{d}{dx}\right)$&lt;/span&gt;&lt;/strong&gt;. &lt;em&gt;(Consider, constants in the expression to be 1.)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the expression, the derivative term, i.e., &lt;span class=&quot;math-container&quot;&gt;$d/dx$&lt;/span&gt; needs to be evaluated. It can be solved either by the finite-difference method or by using Fourier transform to get 1st derivative (which is more accurate than the finite-difference method).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a Matlab program for &lt;span class=&quot;math-container&quot;&gt;$a^{\dagger}$&lt;/span&gt; (&lt;em&gt;shown below&lt;/em&gt;) using the 2nd method, i.e., Fourier transform. After evaluation of states greater than &lt;span class=&quot;math-container&quot;&gt;$n=10$&lt;/span&gt;, the program is becoming unstable and becoming even worse for higher &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (&lt;em&gt;instability plot is shown below in the link&lt;/em&gt;). The same issue had appeared when I used the finite-difference method to evaluate the derivative (code is not attached).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution to the issue or an alternate approach to the problem will be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt; I'm also interested to know the reason for the development of instability while using 'fft' in the code.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sigma = 1.0;&#xA;xmin = -10.0;&#xA;xmax = 10.0;&#xA;npts = 512;&#xA;nstates = 14;&#xA;&#xA;dx = (xmax-xmin)/npts;&#xA;x = xmin + dx*(0:npts-1);&#xA;&#xA;% -- initial state/wavefunction&#xA;psi_init = exp(-0.5*x.^2/sigma^2)*(pi*sigma^2)^(-0.25);&#xA;&#xA;psi = zeros(nstates,npts);             % -- list to store oscillator states&#xA;psi(1,:) = psi_init;&#xA;for nn=2:nstates  &#xA;     psi(nn,:) = raising_psi(psi(nn-1,:),xmin,xmax,npts,sigma); &#xA;end&#xA;&#xA;function adag_fn = raising_psi(previous_fn,a,b,n,sigma)     % -- raising_operator&#xA; dx = (b-a)/n;&#xA; x = a + dx*(0:n-1);&#xA;&#xA; % -- going into fourier space using 'fft' library &#xA; fwd_fft = fft(previous_fn);&#xA; k = (2*pi/(b-a))*[0:n/2-1,0,-n/2+1:-1];&#xA; dfk_dx = 1i*k.*fwd_fft;                     % -- 1st derivative in fourier space&#xA; df_dx = ifft(dfk_dx);                       % -- back into position space&#xA;&#xA; % -- a^dagger acting on the previous state&#xA; adag_fn = (x.*previous_fn/sigma - sigma*real(df_dx))/sqrt(2);&#xA; norm_fn = adag_fn*transpose(adag_fn);&#xA; adag_fn = adag_fn/sqrt(norm_fn);            % -- normalization&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$n=10$&lt;/span&gt;:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CZJyK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CZJyK.jpg&quot; alt=&quot;wavefunction for n=10&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$n=11$&lt;/span&gt;:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/6TDI5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6TDI5.jpg&quot; alt=&quot;wavefunction for n=11&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29729" LastEditorUserId="29729" LastEditDate="2019-01-19T06:26:33.197" LastActivityDate="2019-01-28T22:56:01.943" Title="Generate high n quantum harmonic oscillator states numerically" Tags="|matlab|stability|fourier-analysis|numerics|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="30900" PostTypeId="2" ParentId="30896" CreationDate="2019-01-18T12:58:26.597" Score="6" Body="&lt;p&gt;I'm posting my comment as an answer, partly to guard against link rot, and partly because I noticed a couple of minor errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this at &lt;a href=&quot;https://www.numbercrunch.de/blog/2014/08/calculating-the-hermite-functions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heiko Bauke's blog&lt;/a&gt;, which also refers to a paper by B Bunck &lt;a href=&quot;https://doi.org/10.1007/s10543-009-0216-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;BIT Numerical Mathematics,&lt;/em&gt; &lt;strong&gt;49,&lt;/strong&gt; 281 (2009)&lt;/a&gt; where more details are given (and yet another method involving evaluation of contour integrals is proposed). So the credit really belongs to them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aim is to calculate the normalized Hermite functions&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;h_n(x) = c_n H_n(x) \exp(-x^2/2), \quad&#xA;c_n = \frac{1}{\pi^{1/4}}\,\frac{1}{\sqrt{2^n \, n!}}&#xA;$$&lt;/span&gt;&#xA;and the problem is that at high &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, the Hermite polynomials &lt;span class=&quot;math-container&quot;&gt;$H_n(x)$&lt;/span&gt; become very large, and the normalizing factors &lt;span class=&quot;math-container&quot;&gt;$c_n$&lt;/span&gt; become very small, as well as the function &lt;span class=&quot;math-container&quot;&gt;$\exp(-x^2/2)$&lt;/span&gt; when &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is large. The starting point is the recurrence relation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H_n(x) = 2x H_{n-1}(x) -2(n-1) H_{n-2}(x)&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$H_0(x)=1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$H_1(x)=2x$&lt;/span&gt;. The corresponding recurrence relation for the functions &lt;span class=&quot;math-container&quot;&gt;$h_n(x)$&lt;/span&gt; is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;h_n(x) = \sqrt{\frac{2}{n}} \, x \, h_{n-1}(x) - \sqrt{\frac{n-1}{n}} h_{n-2}(x)&#xA;$$&lt;/span&gt;&#xA;and the problem of large values partly goes away.&#xA;Here I think there is a typo in eqn (3) of the blog post (a spurious factor &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; in the second term) but the typo is not repeated in the code. Then, in the paper, a method is described for handling the evaluation of the functions at large &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Basically a scaling factor is progressively introduced at higher &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to keep the scaled function within reasonable bounds. This method is mentioned on the blog, and is implemented in the code provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To guard against link rot, I'm copying the code here from Bauke's blog, but I emphasize that credit really belongs to Bunck and Bauke. I have made a couple of changes. Firstly, there was a typo in the statement evaluating &lt;span class=&quot;math-container&quot;&gt;$h_1$&lt;/span&gt; (an undefined variable &lt;code&gt;norm&lt;/code&gt;) which I've corrected. Secondly, I've inserted a guard against taking the log of zero, which can happen when one of the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; points coincides exactly with a zero of &lt;span class=&quot;math-container&quot;&gt;$h_n(x)$&lt;/span&gt;: the most obvious instance is &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt; for odd &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def h(n, x):&#xA;    import numpy as np&#xA;    if n==0:&#xA;        return np.ones_like(x)*np.pi**(-0.25)*np.exp(-x**2/2)&#xA;    if n==1:&#xA;        return np.sqrt(2.)*x*np.pi**(-0.25)*np.exp(-x**2/2)&#xA;    h_i_2=np.ones_like(x)*np.pi**(-0.25)&#xA;    h_i_1=np.sqrt(2.)*x*np.pi**(-0.25)&#xA;    sum_log_scale=np.zeros_like(x)&#xA;    for i in range(2, n+1):&#xA;        h_i=np.sqrt(2./i)*x*h_i_1-np.sqrt((i-1.)/i)*h_i_2&#xA;        h_i_2, h_i_1=h_i_1, h_i&#xA;        abs_h_i=np.abs(h_i)&#xA;        log_scale=np.log(abs_h_i,out=np.zeros_like(abs_h_i),where=abs_h_i&amp;gt;1).round()&#xA;        scale=np.exp(-log_scale)&#xA;        h_i=h_i*scale&#xA;        h_i_1=h_i_1*scale&#xA;        h_i_2=h_i_2*scale&#xA;        sum_log_scale+=log_scale&#xA;    return h_i*np.exp(-x**2/2+sum_log_scale)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any other changes are cosmetic. I hope I haven't introduced any new errors. You should check this for errors if you use it or convert it to Matlab. This approach using the recursion relation is most suitable when you want to calculate the &lt;span class=&quot;math-container&quot;&gt;$h_n(x)$&lt;/span&gt; for many values of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; up to some maximum, in which case this function can be re-written to return all of them. I've done some preliminary tests, here for instance is the result for &lt;span class=&quot;math-container&quot;&gt;$n=800$&lt;/span&gt; at high &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, pretty much copying what was on Bauke's blog.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oR30A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oR30A.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT following OP edit to, and comments on, question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Hermite_polynomials#Recursion_relation_2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hermite function recursion relations&lt;/a&gt; are&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;h_n'(x) &amp;amp;= \sqrt{\frac{n}{2}} h_{n-1}(x) - \sqrt{\frac{n+1}{2}} h_{n+1}(x)&#xA;\tag{1}&#xA;\\&#xA;xh_n(x) &amp;amp;= \sqrt{\frac{n}{2}} h_{n-1}(x) + \sqrt{\frac{n+1}{2}} h_{n+1}(x)&#xA;\tag{2}&#xA;\end{align*}&lt;/span&gt;&#xA;Subtracting (2) - (1) gives the equation you are using to calculate &lt;span class=&quot;math-container&quot;&gt;$h_{n+1}(x)$&lt;/span&gt;, with the &quot;raising operator&quot; appearing on the left, and &lt;span class=&quot;math-container&quot;&gt;$h_n'(x)$&lt;/span&gt; approximated using FFT. Eqn (2) is used in the algorithm given above in my answer. I have used this to generate &lt;span class=&quot;math-container&quot;&gt;$h_n(x)$&lt;/span&gt; functions as a reference, and hence obtained accurate derivatives through eqn (1). Comparing with your Fourier evaluation of &lt;span class=&quot;math-container&quot;&gt;$h_n'(x)$&lt;/span&gt;, from the same accurate functions &lt;span class=&quot;math-container&quot;&gt;$h_n(x)$&lt;/span&gt;, shows that &lt;a href=&quot;https://en.wikipedia.org/wiki/Gibbs_phenomenon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gibbs-type oscillations&lt;/a&gt; do indeed start appearing, due to the discontinuities at the edges. Iterating your algorithm would be expected to amplify these effects at each stage. I cannot be sure that this is the complete answer to the problems with the FFT approach, but that's as far as I'm willing to go. If you are continuing the analysis, I suggest that you look at the accuracy of each derivative step in the way I just described, rather than simply examining the output of the whole algorithm, which will show the accumulated errors.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastEditorDisplayName="user28077" LastEditDate="2019-01-27T19:23:47.060" LastActivityDate="2019-01-27T19:23:47.060" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30901" PostTypeId="1" AcceptedAnswerId="30916" CreationDate="2019-01-18T14:40:11.370" Score="3" ViewCount="602" Body="&lt;p&gt;I wanted to verify my FEM-program by applying the method of manufactured solutions, while solving the Poisson equation in two dimensions using the continuous Galerkin method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$-\nabla^2u=f$$&lt;/span&gt;&#xA;with&#xA;&lt;span class=&quot;math-container&quot;&gt;$$u=\sin(\pi x)\sin(\pi y)$$&lt;/span&gt;&#xA;resulting in &#xA;&lt;span class=&quot;math-container&quot;&gt;$$f=2\pi^2\sin(\pi x)\sin(\pi y)$$&lt;/span&gt;&#xA;My system is defined on a square with &lt;span class=&quot;math-container&quot;&gt;$\Omega=[0,1]^2$&lt;/span&gt;, and dirichlet boundary conditions: &lt;span class=&quot;math-container&quot;&gt;$u_{\partial\Omega}=0$&lt;/span&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I looked at the convergence rate of the &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt;-seminorm for elements from second to eighth order at different grid densities, resulting in&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+--------------+---------+----------+----------+------------+-------------+--------------+---------------+&#xA;| cell numbers  |  P2     |  P3      |  P4      |  P5        |  P6         |  P7          |  P8           |&#xA;+--------------+---------+----------+----------+------------+-------------+--------------+---------------+&#xA;| 4            |  11.04  |  339     |  852.46  |  10595.67  |  137926.51  |  6300733.33  |  69993038.13  |&#xA;| 16           |  3.97   |  62.5    |  12.8    |  14.29     |  56.35      |  335.63      |  295.60       |&#xA;| 64           |  4      |  125.47  |  12.81   |  14.02     |  56.95      |  350.12      |  299.23       |&#xA;| 256          |  4      |  251.15  |  12.81   |  13.96     |  57.10      |  303.92      |  12.77        |&#xA;| 1024         |  4      |  502.4   |  12.8    |  13.94     |  57.24      |  6.82        |  1.16         |&#xA;+--------------+---------+----------+----------+------------+-------------+--------------+---------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Some things here I (think I can) explain, others not. On the one hand, the large value in the first line results from the fact that I start from &lt;span class=&quot;math-container&quot;&gt;$u_0=0$&lt;/span&gt;, while for every other step I reuse the result from the last step, which has been interpolated onto the new grid. Thus I expect the correct convergence rate showing up there, except in the first step.&lt;br&gt;&#xA;Furthermore, I assume that the decreasing values for elements of seventh and eighth order show that the interpolation already is quite good for low grid densities. Is that correct?&lt;br&gt;&#xA;The things I do not understand here:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is the convergence rate for elements of third order so much higher (and increasing for increasing density) than for the elements of fourth and fifth order (partially also for higher orders)? Is that by accident, i.e. third-order elements simply interpolate the test function quite well, or something else?  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is there a big step in convergence rates from second order to fourth order (skipping third order), but almost no difference between the fourth and fifth order? Again, a step from fifth to sixth order and from sixth to seventh, but almost no difference is visible between the seventh and eighth order.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How can I explain those things? (I hope there is no error in the program itself...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The values of the &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt;-seminorm itself are&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+--------------+----------+----------+----------+-----------+-----------+-----------+-----------+&#xA;| cell numbers |    P2    |    P3    |    P4    |    P5     |    P6     |    P7     |    P8     |&#xA;+--------------+----------+----------+----------+-----------+-----------+-----------+-----------+&#xA;|            4 | 3.485e-1 | 1.135e-2 | 4.514e-3 | 3.631e-4  | 2.790e-5  | 6.107e-7  | 5.497e-8  |&#xA;|           16 | 8.820e-2 | 7.335e-4 | 2.897e-4 | 1.172e-5  | 4.451e-7  | 4.343e-9  | 2.131e-10 |&#xA;|           64 | 2.210e-2 | 4.624e-5 | 1.823e-5 | 3.684e-7  | 6.994e-9  | 3.293e-11 | 8.274e-13 |&#xA;|          256 | 5.527e-3 | 2.896e-6 | 1.141e-6 | 1.157e-8  | 1.094e-10 | 2.981e-13 | 7.630e-14 |&#xA;|         1024 | 1.382e-3 | 1.811e-7 | 7.135e-8 | 3.617e-10 | 1.708e-12 | 1.183e-13 | 1.428e-13 |&#xA;+--------------+----------+----------+----------+-----------+-----------+-----------+-----------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using the library deal.II for calculating the values, and use the element &lt;code&gt;FE_Q&lt;/code&gt; as basis element. It switches from lagrange polynomials (up to order 2) with equidistant support points to Gauss-Lobatto-polynomials for higher orders (also refer to &lt;a href=&quot;https://www.dealii.org/developer/doxygen/deal.II/classFE__Q.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dealii.org/developer/doxygen/deal.II/classFE__Q.html&lt;/a&gt;). Could that be the reason for the observed convergence behaviour? &lt;/p&gt;&#xA;" OwnerUserId="20545" LastEditorUserId="20545" LastEditDate="2019-01-22T10:19:44.427" LastActivityDate="2019-01-22T15:12:16.150" Title="$H^1$-convergence rate of finite element method for Poisson equation, depending on element order" Tags="|finite-element|convergence|galerkin|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30904" PostTypeId="1" AcceptedAnswerId="30921" CreationDate="2019-01-18T21:19:31.523" Score="0" ViewCount="75" Body="&lt;p&gt;This is my first post on here, so please excuse mistakes if any.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to plot out the difference between two ODE solvers based on Taylor series: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1st order acccurate:  &lt;span class=&quot;math-container&quot;&gt;$x(t_0 + h) = x(t_0) + h x'(t_0)$&lt;/span&gt; and &lt;/li&gt;&#xA;&lt;li&gt;2nd order accurate:   &lt;span class=&quot;math-container&quot;&gt;$x(t_0 + h) = x(t_0) + h x'(t_0) + \frac{h^2}{2} x''(t_0)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;ODE: x'= -x + e^-t  x0 = 0; t0 = 0&#xA;     x'' = x - 2e^-t&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some reason real solution and both 1st and 2nd order approximation are the same. Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from math import exp&#xA;from numpy import arange&#xA;from numpy import zeros&#xA;from matplotlib import pyplot as plt&#xA;&#xA;def f(t,x):&#xA;    dx = -x + exp(-t)&#xA;    return dx&#xA;&#xA;def df(t,x):&#xA;    df = x - 2*exp(-t)&#xA;    return df&#xA;&#xA;def real(h):&#xA;    t = 0&#xA;    xk_list = []&#xA;    tk_list = []&#xA;    def func(t):&#xA;        x = t*exp(-t)&#xA;        return x&#xA;&#xA;    while(True):&#xA;        xk_list.append(func(t))&#xA;        tk_list.append(t)&#xA;        t += h&#xA;        if (t &amp;gt; 3.0):&#xA;            return tk_list, xk_list&#xA;&#xA;def forward_euler_1(x0, t0, h):&#xA;    xk = x0&#xA;    tk = t0&#xA;    tk_list = []&#xA;    xk_list = []&#xA;    counter = 0&#xA;    while(True):&#xA;        xk1 = xk + h*f(tk,xk)&#xA;        xk_list.append(xk)&#xA;        tk_list.append(tk)&#xA;        if(tk &amp;gt; 3.0):&#xA;            print(counter,tk)&#xA;            return tk_list, xk_list&#xA;        xk = xk1&#xA;        tk = tk + h&#xA;        counter += 1&#xA;&#xA;def forward_euler_2(x0,t0,h):&#xA;    xk = x0&#xA;    tk = t0&#xA;    tk_list = []&#xA;    xk_list = []&#xA;    counter = 0 &#xA;&#xA;    while(True):&#xA;        xk1 = xk + h*f(tk, xk) + (h**2)/2*df(tk, xk)&#xA;        xk_list.append(xk1)&#xA;        tk_list.append(tk)    &#xA;        if(tk &amp;gt; 3.0):&#xA;            print(counter,tk)&#xA;            return tk_list, xk_list        &#xA;        xk = xk1&#xA;        tk = tk + h&#xA;        counter += 1&#xA;&#xA;x, y = forward_euler_1(0.0, 0.0,0.001)&#xA;x2, y2 = forward_euler_2(0.0, 0.0, 0.001)&#xA;x3, y3 = real(0.001)&#xA;&#xA;plt.plot(x,y)&#xA;plt.plot(x2,y2,'--')&#xA;plt.plot(x3, y3)&#xA;plt.show()       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29741" LastEditorUserId="9667" LastEditDate="2019-01-18T21:38:37.550" LastActivityDate="2019-01-23T14:35:24.560" Title="Approximation of ODE solution using Taylor series methods" Tags="|ode|numerics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30905" PostTypeId="1" CreationDate="2018-12-14T03:46:50.347" Score="1" ViewCount="227" Body="&lt;p&gt;I have a matrix that is extremely singular, but I am still interested in computing the exact condition number, which is the ratio between the largest and smallest singular values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to do it in Python? I am curious because if we stick with machine precision 10^-16, then any condition number beyond 10^16 could be inaccurate.&lt;/p&gt;&#xA;" OwnerUserId="16859" OwnerDisplayName="ArtificiallyIntelligence" LastActivityDate="2019-01-20T16:14:37.597" Title="How to compute large condition number of a matrix in Python?" Tags="|python|precision|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30906" PostTypeId="1" AcceptedAnswerId="30910" CreationDate="2019-01-21T09:28:13.913" Score="0" ViewCount="94" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$r,\epsilon &amp;gt; 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a, b \in \mathbb R^n$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\|a\|_2 \le r$&lt;/span&gt;. Define &lt;span class=&quot;math-container&quot;&gt;$C(a) := \{x \in \mathbb R^p | \|x+a\|_2 \le r,\;\|x\|_\infty \le \epsilon\}$&lt;/span&gt;, and assume it is non-empty.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Question&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;(A)&lt;/em&gt; How to formulate and solve the problem problem &lt;span class=&quot;math-container&quot;&gt;$\sup_{x \in C(a)} b^Tx$&lt;/span&gt; in the cvxpy language ?&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;(B)&lt;/em&gt; Same question with &lt;span class=&quot;math-container&quot;&gt;$\|a\|_2 = r$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C(a) := \{x \in \mathbb R^p | \|x+a\|_2 = r,\;\|x\|_\infty \le \epsilon\}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Related to:&lt;/strong&gt; &lt;a href=&quot;https://math.stackexchange.com/q/3080805/168758&quot;&gt;https://math.stackexchange.com/q/3080805/168758&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I've never done cvxopt / cvxpy before. I plan to learn the syntax later. For now, I just want something to plug-and-play. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16807" LastEditorUserId="16807" LastEditDate="2019-01-22T02:04:33.380" LastActivityDate="2019-01-22T02:04:33.380" Title="Formulate and solve a simple conic programs in cvxpy language" Tags="|constrained-optimization|convex-optimization|cvx|cvxpy|" AnswerCount="1" CommentCount="1" ClosedDate="2019-01-25T14:35:23.863" ContentLicense="CC BY-SA 4.0" />
  <row Id="30907" PostTypeId="1" CreationDate="2019-01-21T13:38:10.050" Score="1" ViewCount="52" Body="&lt;p&gt;I've been looking into radial solvers for quantum wave equations (Schroedinger and Dirac). In both cases, the suggestion seems to be to go with the &quot;shooting method&quot;, with integration schemes of various precision such as the Numerov one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for a shooting method, one needs to impose starting values coming from boundary conditions. Many texts mention asymptotic behaviours such as Bessel functions near &lt;span class=&quot;math-container&quot;&gt;$r=0$&lt;/span&gt; etc. Since my interest is in solving perturbed versions of the basic Coulomb potential, and the solution for this problem is already known,  however, couldn't I just start with the known hydrogen-like solution and work from there? Or is there some reason why this is not indicated?&lt;/p&gt;&#xA;" OwnerUserId="25184" LastEditorUserId="20688" LastEditDate="2019-01-21T15:04:38.720" LastActivityDate="2019-01-21T15:04:38.720" Title="Hydrogen-like wavefunction as starting guess for atomic solver?" Tags="|finite-difference|boundary-conditions|quantum-mechanics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30908" PostTypeId="2" ParentId="30901" CreationDate="2019-01-21T15:27:23.867" Score="4" Body="&lt;p&gt;There is a bug in your code :-) First, going down in each column, you have cases where the error becomes larger again, and this should clearly not happen because the finite element spaces are nested going from top to bottom. Secondly, moving left to right the error should also decrease because the finite element spaces are nested, but again this is not happening. Finally, in most columns the error does not appear to converge to zero but to some finite value -- this should also clearly not happen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if the table shows &lt;em&gt;errors&lt;/em&gt;, then there is a bug somewhere. If, in contrast, the table shows the &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt; seminorm of the &lt;em&gt;solution&lt;/em&gt; rather than the error, then there is still a bug somewhere because then going top to bottom and left to right should still result in convergence towards the exact value, but your values are all over the place.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-01-21T15:27:23.867" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30909" PostTypeId="1" AcceptedAnswerId="30948" CreationDate="2019-01-21T19:13:08.650" Score="3" ViewCount="1563" Body="&lt;p&gt;I am trying to solve the following coupled PDEs: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C_e\frac{\partial  u(x,t)}{\partial t} = k_{ed}\frac{\partial^2u(x,t)}{\partial^2x} - G_{el}(u(x,t) - v(x,t)) + S(x,t)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$C_l\frac{\partial  v(x,t)}{\partial t} = k_{ed}\frac{\partial^2v(x,t)}{\partial^2x} + G_{el}(u(x,t) - v(x,t)) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$k_{ed}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$G_{el}$&lt;/span&gt; are constants and &lt;span class=&quot;math-container&quot;&gt;$C_e = \gamma u(x,t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_l$&lt;/span&gt; are specific heats and &lt;span class=&quot;math-container&quot;&gt;$S(x,t) = I_0 exp(-\frac{x}{\delta}-\frac{(t - 2t_p)^2}{t_p^2})$&lt;/span&gt; is the laser pulse source term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use finite difference methods to solve the above equations as follows: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u^{f+1}_i = u^f_i + \frac{k_{ed}dt}{\Delta x^2}(u^f_{i+1} -2u^f_i + u^f_{i-1}) + dt(-G_{el}(u^f_i - v^f_i) + S^f_i)$$&lt;/span&gt; &#xA;and &#xA;&lt;span class=&quot;math-container&quot;&gt;$$v^{f+1}_i = v^f_i + \frac{k_{ed}dt}{\Delta x^2}(v^f_{i+1} -2v^f_i + v^f_{i-1}) + dt(G_{el}(u^f_i - v^f_i))$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$(f,i)$&lt;/span&gt; are mesh in time and space as &#xA;&lt;span class=&quot;math-container&quot;&gt;$$F\Delta t = T_{total}, f = 0,1,2 ... F$$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$$N\Delta x = L, i = 0,1,2,... N $$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$T_{total}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; are the total integration time and length respectively. &#xA;I have implemented the above in python: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;# sample parameter §§ Gold &#xA;la0 = 429 # conductivity in W/mK &#xA;gma = 62.8 # thermal constant J/m^3K^2&#xA;Cl = 2.6*10**6 # phonon specific heat  in J/m^3K&#xA;Gel = 3.5*10**16 # lelectron phonon coupling constant &#xA;tau_e = 0.04 # e relaxation t const &#xA;tau_l = 0.6 # l relaxation t const&#xA;Tm = 300&#xA;L = 100 # sample thickness in nm &#xA;# Laser pulse parameters &#xA;T = 6&#xA;&#xA;R = 0.93 # reflectivity &#xA;I0 = 100 # fulence in J/m^2&#xA;tp = 0.1 # in ps &#xA;zz0 = 15.3 # penetration depth in nm&#xA;&#xA;&#xA;#F = 0.5&#xA;&#xA;def las(x,t):&#xA;    pt = 0.93*((1-R)/(zz0*tp))*I0*np.exp(-(x/zz0)-2.772*((t-2*tp)**2/tp**2))&#xA;    return pt&#xA;&#xA;def I(x):&#xA;    tin = Tm&#xA;    return tin&#xA;&#xA;a = la0 # Ked&#xA;&#xA;&#xA;&#xA;def solver_FE(I, a, las, L, T):&#xA;    &quot;&quot;&quot;&#xA;&#xA;    &quot;&quot;&quot;&#xA;    import time;  t0 = time.clock()  # For measuring the CPU time&#xA;&#xA;    dt = 0.001&#xA;    Nt = int(round(T/float(dt)))&#xA;    t = np.linspace(0, Nt*dt, Nt+1)   # Mesh points in time&#xA;&#xA;    dx = 1&#xA;    Nx = int(round(L/dx))&#xA;    x = np.linspace(0, L, Nx+1)       # Mesh points in space&#xA;&#xA;&#xA;    F = a*(dt/dx**2)&#xA;    u   = np.zeros(Nx+1)&#xA;    u_n = np.zeros(Nx+1)&#xA;    v   = np.zeros(Nx+1)&#xA;    v_n = np.zeros(Nx+1)    &#xA;&#xA;&#xA;&#xA;    # Set initial condition u(x,0) = I(x)&#xA;    for i in range(0, Nx+1):&#xA;        u_n[i] = I(x[i])&#xA;        v_n[i] = I(x[i])&#xA;&#xA;&#xA;&#xA;    for n in range(0, Nt):&#xA;        # Compute u, v  at inner mesh points&#xA;        for i in range(1, Nx):&#xA;            u[i] = u_n[i] + (-F*(u_n[i-1] - 2*u_n[i] + u_n[i+1]) + dt*(-Gel*(u_n[i] - v_n[i]) + las(x[i], t[n])))/(gma*u_n[i])&#xA;            v[i] = v_n[i] + (-F*(v_n[i-1] - 2*v_n[i] + v_n[i+1]) + dt*(Gel*(u_n[i] - v_n[i])))/(Cl)&#xA;&#xA;        # Insert boundary conditions&#xA;        u[0] = 0;  u[Nx] = 0&#xA;        v[0] = 0;  v[Nx] = 0&#xA;&#xA;       # this step is to save each value of v and u in each time step before updating it, such that I can plot u and v with respect to time. &#xA;         ut = np.array([])&#xA;         vt = np.array([])  &#xA;        for m in range(0, Nt):&#xA;            ut = np.append(ut,u)&#xA;            vt = np.append(vt,v)  &#xA;&#xA;&#xA;        # Switch variables before next step&#xA;&#xA;        u_n, u = u, u_n&#xA;        v_n, v = v, v_n &#xA;&#xA;&#xA;&#xA;    t1 = time.clock()&#xA;    return u_n, v_n, x, t, t1-t0  # u_n holds latest u&#xA;&#xA;&#xA;u, v, x, t, cpu = solver_FE(I, a, las, L, T)&#xA;&#xA;#fig = plt.figure(1)&#xA;##t = np.linspace(0, T, h)&#xA;#plt.plot(t,u,'r',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_e$&lt;/span&gt;')&#xA;#plt.plot(t,v,'b',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_l$&lt;/span&gt;')&#xA;##plt.plot(t,sol0[:,2],'g',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_s$&lt;/span&gt;')&#xA;#plt.ylabel('&lt;span class=&quot;math-container&quot;&gt;$Temperature$&lt;/span&gt;',fontsize=20)&#xA;#plt.xlabel('Delay',fontsize=20) &#xA;#plt.legend(loc='best')&#xA;#plt.show()&#xA;##&#xA;fig = plt.figure(2)&#xA;#t = np.linspace(0, T, h)&#xA;plt.plot(x,u,'r',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_e$&lt;/span&gt;')&#xA;plt.plot(x,v,'g',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_l$&lt;/span&gt;')&#xA;#plt.plot(t,sol0[:,2],'g',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_s$&lt;/span&gt;')&#xA;plt.ylabel('&lt;span class=&quot;math-container&quot;&gt;$Temperature$&lt;/span&gt;',fontsize=20)&#xA;plt.xlabel('length',fontsize=20) &#xA;plt.legend(loc='best')&#xA;plt.show()&#xA;&#xA;fig = plt.figure(3)&#xA;#t = np.linspace(0, T, h)&#xA;plt.plot(t,las(0,t),'r',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_e$&lt;/span&gt;')&#xA;#plt.plot(x,v,'g',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_l$&lt;/span&gt;')&#xA;#plt.plot(t,sol0[:,2],'g',label=r'&lt;span class=&quot;math-container&quot;&gt;$T_s$&lt;/span&gt;')&#xA;plt.ylabel('&lt;span class=&quot;math-container&quot;&gt;$Intensity$&lt;/span&gt;',fontsize=20)&#xA;plt.xlabel('Delay',fontsize=20) &#xA;plt.legend(loc='best')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while running it I get the following errors: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RuntimeWarning: overflow encountered in double_scalars&#xA;  u[i] = u_n[i] + F*(u_n[i-1] - 2*u_n[i] + u_n[i+1]) + dt*(-Gel*(u_n[i] - v_n[i]) + las(x[i], t[n]))&#xA;C:/Users/jayas/OneDrive/Documents/Python Scripts/llg/diff_ttm_v_test.py:90: RuntimeWarning: overflow encountered in double_scalars&#xA;  v[i] = v_n[i] + F*(v_n[i-1] - 2*v_n[i] + v_n[i+1]) + dt*(Gel*(u_n[i] - v_n[i]))&#xA;C:/Users/jayas/OneDrive/Documents/Python Scripts/llg/diff_ttm_v_test.py:89: RuntimeWarning: invalid value encountered in double_scalars&#xA;  u[i] = u_n[i] + F*(u_n[i-1] - 2*u_n[i] + u_n[i+1]) + dt*(-Gel*(u_n[i] - v_n[i]) + las(x[i], t[n]))&#xA;C:/Users/jayas/OneDrive/Documents/Python Scripts/llg/diff_ttm_v_test.py:90: RuntimeWarning: invalid value encountered in double_scalars&#xA;  v[i] = v_n[i] + F*(v_n[i-1] - 2*v_n[i] + v_n[i+1]) + dt*(Gel*(u_n[i] - v_n[i]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I guess that I am doing somewhere wrong while calculating the inner mesh points. Can anyone lemme know whats going wrong in here and ideas to rectify it?  &lt;/p&gt;&#xA;" OwnerUserId="14602" LastEditorUserId="14602" LastEditDate="2019-01-21T19:24:17.977" LastActivityDate="2019-01-28T12:09:24.807" Title="Is this the correct way for solving coupled 1d PDEs using finite difference methods?" Tags="|pde|python|hyperbolic-pde|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30910" PostTypeId="2" ParentId="30906" CreationDate="2019-01-21T23:15:08.937" Score="2" Body="&lt;p&gt;You can solve Question A as a second-order cone program like so:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import cvxpy as cp &#xA;import numpy as np&#xA;&#xA;##########################&#xA;#Question A&#xA;##########################&#xA;&#xA;n = 50                          #Arbitrary number of dimensions&#xA;r = 10                          #Arbitrary radius&#xA;e = 3                           #Epsilon value&#xA;a = np.random.random(size=50)   #Generate random a vector&#xA;a = a/np.linalg.norm(a, ord=2)  #Scale to unit length&#xA;a = r*a                         #Scale to radius&#xA;&#xA;x = cp.Variable(shape=n)        #x, a variable to be optimized&#xA;cons = []                       #List of constraints&#xA;&#xA;#See cvxpy's atoms here: https://www.cvxpy.org/api_reference/cvxpy.atoms.html&#xA;cons += [cp.norm(x+a,2)&amp;lt;=r]     #Note that we are using cvxpy's `norm` function!&#xA;cons += [cp.norm_inf(x)&amp;lt;=e]&#xA;&#xA;#Objective&#xA;obj = cp.Maximize(cp.sum(a*x))&#xA;&#xA;#Formulate problem&#xA;prob = cp.Problem(obj, cons)&#xA;&#xA;#Solve problem&#xA;optval = prob.solve()&#xA;&#xA;print(&quot;Optimum value = {0}&quot;.format(optval))&#xA;print(&quot;x = {0}&quot;.format(x.value))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Question B you can't solve using plug-and-play with cvxpy because the problem is non-convex (you're optimizing over the surface of an ellipsoid).&lt;/p&gt;&#xA;" OwnerUserId="17088" LastEditorUserId="17088" LastEditDate="2019-01-22T00:04:59.053" LastActivityDate="2019-01-22T00:04:59.053" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30911" PostTypeId="1" AcceptedAnswerId="30914" CreationDate="2019-01-22T06:26:24.727" Score="1" ViewCount="215" Body="&lt;p&gt;I'm having some difficulties understanding the Langevin thermostat (MD).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my notes, there is written that the Langevin equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ m\dot{v} = F - m\gamma v + f_R, \tag{1} \label{1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$f_R$&lt;/span&gt; is the random force. Then, to prove that by using this thermostat we are actually sampling from the canonical ensemble, there is written that the starting point of the proof are the overdamped Langevin equations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \dot{r}_i(t) = \beta D F_i (r_1(t), \dots,r_N(t)) + \eta_i(t), \tag{2} \label{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\eta_i(t)$&lt;/span&gt; is a random velocity. Where do &lt;span class=&quot;math-container&quot;&gt;$\eqref{2}$&lt;/span&gt; come from? Are they derived from &lt;span class=&quot;math-container&quot;&gt;$\eqref{1}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="22185" LastEditorUserId="20688" LastEditDate="2019-06-22T19:18:17.323" LastActivityDate="2019-06-22T19:18:17.323" Title="Langevin Thermostat and overdamped Langevin Equations" Tags="|molecular-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30913" PostTypeId="2" ParentId="30901" CreationDate="2019-01-22T08:13:59.220" Score="1" Body="&lt;p&gt;How are you actually computing error norm and convergence rate ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use your error table, I get the expected result. E.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P2, 256 and 1024 cells&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [1]: log(5.527e-3/1.382e-3)/log(2)                                           &#xA;Out[1]: 1.9997389968259023&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;P4, 256 and 1024 cells&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [4]: log(1.141e-6/7.135e-8)/log(2)                                           &#xA;Out[4]: 3.9992415517002806&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is something weird going on with your P3 results. You must double check your code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I have a deal.II code that solves almost the problem here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/cpraveen/fembook/tree/master/deal.II/ex04&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/cpraveen/fembook/tree/master/deal.II/ex04&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modifying this for your particular problem with degree = 3, I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cells dofs       L2             H1       &#xA;    4   49 1.359e-03    - 2.668e-02    - &#xA;   16  169 8.812e-05 3.95 3.376e-03 2.98 &#xA;   64  625 5.564e-06 3.99 4.233e-04 3.00 &#xA;  256 2401 3.486e-07 4.00 5.295e-05 3.00 &#xA; 1024 9409 2.180e-08 4.00 6.620e-06 3.00 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3970" LastEditorUserId="3970" LastEditDate="2019-01-22T14:02:55.997" LastActivityDate="2019-01-22T14:02:55.997" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30914" PostTypeId="2" ParentId="30911" CreationDate="2019-01-22T08:22:48.770" Score="3" Body="&lt;p&gt;Eqn (2) basically comes from setting &lt;span class=&quot;math-container&quot;&gt;$m\dot{v}=0$&lt;/span&gt; in eqn (1), on the assumption that the velocities are strongly damped. Then the &lt;span class=&quot;math-container&quot;&gt;$m\gamma v$&lt;/span&gt; term is taken over to the left side, and both sides are divided by &lt;span class=&quot;math-container&quot;&gt;$m\gamma$&lt;/span&gt;. The diffusion coefficient is &lt;span class=&quot;math-container&quot;&gt;$D=k_BT/m\gamma$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said all that, this is &lt;em&gt;not&lt;/em&gt; a proof that the thermostat generates the canonical ensemble. Such a proof usually (I believe) starts by considering the Fokker-Planck equation (in both &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;) which is equivalent to the Langevin equation, and showing that the canonical ensemble (with a potential from which &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is derived) is a stationary solution of that equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eqn (2) can lead to a proof that the &lt;em&gt;positions&lt;/em&gt; are sampled from the canonical ensemble, in this overdamped limit, but to show that both positions and velocities are correctly sampled in general, one should keep the velocity evolution of eqn (1).&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastEditorDisplayName="user28077" LastEditDate="2019-01-22T08:41:14.307" LastActivityDate="2019-01-22T08:41:14.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30916" PostTypeId="2" ParentId="30901" CreationDate="2019-01-22T15:12:16.150" Score="0" Body="&lt;p&gt;I found the explanation for that behaviour: For the calculation of the integral over the differences I used an element of third order, which was fixed. This explains the high convergence rate for P3, and the lower rate for all other elements. When replacing the fixed order with a more suitable order (such as FE_degree * 2 + 1), my convergence rate becomes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P2    4&#xA;P3    8&#xA;P4    16&#xA;P5    32&#xA;P6    64&#xA;P7    128&#xA;P8    256&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as expected.&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2019-01-22T15:12:16.150" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30920" PostTypeId="1" CreationDate="2019-01-23T13:11:25.943" Score="2" ViewCount="105" Body="&lt;p&gt;I am working on a 1D drift-driffusion problem in a finite-difference (FD) approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hade 3 equations per node (&lt;span class=&quot;math-container&quot;&gt;$3N$&lt;/span&gt; in total): electron continuity &lt;span class=&quot;math-container&quot;&gt;$E_i$&lt;/span&gt;, Poisson &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt;, hole continuity &lt;span class=&quot;math-container&quot;&gt;$H_i$&lt;/span&gt;. With corresponding variables: electron concentration &lt;span class=&quot;math-container&quot;&gt;$n_j$&lt;/span&gt;, potential &lt;span class=&quot;math-container&quot;&gt;$V_j$&lt;/span&gt;, hole concentration &lt;span class=&quot;math-container&quot;&gt;$p_j$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By rearranging the problem has a banded matrix, the Jacobian &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; takes the following form, with shape &lt;span class=&quot;math-container&quot;&gt;$(3N\times 3N)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A = \begin{pmatrix}&#xA;A_{1,1} &amp;amp; A_{1,2} &amp;amp;         &amp;amp;         &amp;amp;         &amp;amp;         &amp;amp;       \\&#xA;A_{2,1} &amp;amp; A_{2,2} &amp;amp; A_{2,3} &amp;amp;         &amp;amp;         &amp;amp;         &amp;amp;       \\&#xA;        &amp;amp; A_{3,2} &amp;amp; A_{3,3} &amp;amp; A_{3,4} &amp;amp;         &amp;amp;         &amp;amp;       \\&#xA;        &amp;amp;         &amp;amp; \ddots  &amp;amp; \ddots  &amp;amp; \ddots  &amp;amp;         &amp;amp;       \\&#xA;        &amp;amp;         &amp;amp;         &amp;amp; A_{N-2,N-3} &amp;amp; A_{N-2,N-2} &amp;amp; A_{N-2,N-1} &amp;amp;       \\&#xA;        &amp;amp;         &amp;amp;         &amp;amp;         &amp;amp; A_{N-1,N-2} &amp;amp; A_{N-1,N-1} &amp;amp; A_{N-1,N} \\&#xA;        &amp;amp;         &amp;amp;         &amp;amp;         &amp;amp;         &amp;amp; A_{N,N-1} &amp;amp; A_{N,N} \\&#xA;\end{pmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A_{i, j} = \begin{pmatrix} &#xA;{\partial E_i}/{\partial n_j} &amp;amp; {\partial E_i}/{\partial V_j} &amp;amp; {\partial E_i}/{\partial p_j} \\&#xA;{\partial P_i}/{\partial n_j} &amp;amp; {\partial P_i}/{\partial V_j} &amp;amp; {\partial P_i}/{\partial p_j} \\&#xA;{\partial H_i}/{\partial n_j} &amp;amp; {\partial H_i}/{\partial V_j} &amp;amp; {\partial H_i}/{\partial p_j}&#xA;\end{pmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Owing to that data structure, I have been using &lt;code&gt;scipy.linalg.solve_banded&lt;/code&gt; to solve for &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$AX = b$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$(3N \times 1)$&lt;/span&gt; shaped vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem comes from that I want to add an equation that controls the potential at one end (say on the left) based on the value of total fluxes of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; &lt;strong&gt;at both ends&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result the additional equation depends on &lt;span class=&quot;math-container&quot;&gt;$(n_1, V_1, p_1, n_2, V_2, p_2)$&lt;/span&gt; but also &lt;span class=&quot;math-container&quot;&gt;$(n_{N-1}, V_{N-1}, p_{N-1}, n_N, V_N, p_N)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, if I was to include this new equation in &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to form a &lt;span class=&quot;math-container&quot;&gt;$(3N+1 \times 3N+1)$&lt;/span&gt; Jacobian &lt;span class=&quot;math-container&quot;&gt;$A'$&lt;/span&gt;, the bandwith would suddenly become much larger. Though I can optimize this by inserting the new equation in the middle row of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, the banded representation would remain very sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is then: is there a way I can solve for &lt;span class=&quot;math-container&quot;&gt;$X'$&lt;/span&gt; in &#xA;&lt;span class=&quot;math-container&quot;&gt;$$A' X' = b'$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;while keeping the advantages of the banded structure of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;hr&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;EDIT addressing comments&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Here is a visual example of the Jacobian before (left) and after reordering (right). The last column/line, highlighted in yellow, is the additional equation that fails to fit the banded structure.&#xA;Elements in dark gray background are non-zero, elements in white are zeros, elements in light gray background are zero, but a future version of my physical could in principle make them non-zero so an ideal solution would not exclude that possibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qnGJt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qnGJt.png&quot; alt=&quot;visual sparsity representation&quot;&gt;&lt;/a&gt;&#xA;(click to enlarge)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I do not pretend to be the author of this ordering method. I have read it a few years ago in a software manual I believe ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="29422" LastEditorUserId="29422" LastEditDate="2019-01-24T23:30:17.257" LastActivityDate="2019-01-24T23:30:17.257" Title="Can I take advantage of a nearly banded A in AX=b?" Tags="|finite-difference|banded-matrix|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="30921" PostTypeId="2" ParentId="30904" CreationDate="2019-01-23T14:35:24.560" Score="0" Body="&lt;p&gt;There is nothing wrong with the code, I am just using an h resoution that is too high, if one put's the time discretization at 0.1 you see what I expected to see. &lt;/p&gt;&#xA;" OwnerUserId="29741" LastActivityDate="2019-01-23T14:35:24.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30922" PostTypeId="1" CreationDate="2019-01-23T16:05:31.007" Score="5" ViewCount="230" Body="&lt;p&gt;I need to calculate a basis of the nullspace of large (up to a thousand columns and rows) matrices. For my application, it is very important that no round-off errors occur during the computation, so I need software that computes the nullspace symbolically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entries of my matrix are fractions but can of course be converted to (large) integers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me, naively, this seems like a very straightforward problem that should have been solved somewhere already. However, the available methods that I have tried either solve harder problems, introduce round-off errors, or cannot cope with the size of my matrices. Amongst others, I already tried linbox and Mathematica.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment, my input matrices are arrays of fractions in Python, but it is no problem to use different open-source programs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question: is there readily-usable software that I overlooked to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="29768" LastActivityDate="2019-01-26T22:44:26.023" Title="Nullspace calculation of large matrix with rational numbers without round-off errors (exact)" Tags="|linear-algebra|symbolic-computation|nullspace|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30924" PostTypeId="1" CreationDate="2019-01-24T11:51:46.720" Score="1" ViewCount="1313" Body="&lt;p&gt;I have a quite large algebraic equation system to solve, the system is so large, I can't post the example here, so I am posting it to &lt;a href=&quot;https://pastebin.com/aWSNZ7dS&quot; rel=&quot;nofollow noreferrer&quot;&gt;pastebin&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;sympy.solve&lt;/code&gt; is taking ages: I started it this morning and it is still running on a single core. I have tried to use the optional arguments &lt;a href=&quot;https://docs.sympy.org/latest/modules/solvers/solvers.html#algebraic-equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;from the sympy documentation&lt;/a&gt;, like &lt;code&gt;quick=True&lt;/code&gt; and &lt;code&gt;warn=True&lt;/code&gt;, to speed things up and get warnings if the solver fails, but there was no warning given, and the solver is still dying. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way multiprocessing can be used or maybe &lt;a href=&quot;https://pypi.org/project/mpi4py/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mpi4py&lt;/a&gt; with the &lt;code&gt;solve&lt;/code&gt; function in &lt;code&gt;sympy&lt;/code&gt;. I have access to HPC resources, but I'm not sure how &lt;code&gt;solve&lt;/code&gt; can be separated into different MPI processes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way in &lt;code&gt;sympy&lt;/code&gt; to get an idea if the system is solvable or if the solver is diverging? Something like runtime solver information, but without running in debug mode? &lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2019-01-24T11:51:46.720" Title="Speeding up the solution of a large set of nonlinear algebraic equations in `sympy`" Tags="|python|nonlinear-equations|sympy|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30925" PostTypeId="1" CreationDate="2019-01-24T14:39:22.817" Score="3" ViewCount="165" Body="&lt;p&gt;I'm trying to use the finite-difference WENO scheme to solve the 2D density conservation law with axial symmetry (coordinates &lt;span class=&quot;math-container&quot;&gt;$r,z$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial \rho}{\partial t}+\nabla \cdot (\rho \vec{v}) = \frac{\partial \rho}{\partial t}+ \frac{1}{r}\partial_r (r \rho v_r)+\partial_z (\rho v_z) = 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is defined at grid points &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$v_r$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$i+1/2,j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_z$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$i,j+1/2$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would like use a WENO reconstruction procedure to estimate &lt;span class=&quot;math-container&quot;&gt;$f_r = r \rho v_r$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$i\pm 1/2,j$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$f_z = \rho v_z$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$i,j \pm 1/2$&lt;/span&gt;, from their values at &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt; (flux-splitted), and use those estimates to update the density at &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial \rho^{i,j}}{\partial t} = -\frac{1}{r}\frac{1}{dr} \left[f_r^{i+1/2,j}-f_r^{i-1/2,j} \right]-\frac{1}{dz} \left[f_z^{i,j+1/2}-f_z^{i,j-1/2} \right]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, since the velocities aren't defined at &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt;, I use a simple interpolation (i.e, &lt;span class=&quot;math-container&quot;&gt;$v_r^{i,j} = (v_r^{i+1/2,j}+v_r^{i-1/2,j})/2$&lt;/span&gt;) to estimate their values at those points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this, however, does not yield good results. I've tried to do a WENO interpolation of the densities, but that also isn't working as well as I need it to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to handle this issue? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionaly, any recomendations on how to properly flux-split in this case? Most flux-splitting procedures seem to reduce to a trivial case when &lt;span class=&quot;math-container&quot;&gt;$f(\rho) \propto \rho$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20177" LastActivityDate="2019-01-26T14:54:43.890" Title="WENO5 scheme in a staggered grid" Tags="|finite-difference|interpolation|grid|conservation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30927" PostTypeId="2" ParentId="11492" CreationDate="2019-01-24T21:34:02.913" Score="0" Body="&lt;p&gt;Multigrid loops even on single core are vectorizable by the optimizer.&#xA;So while operation counts can help, we should not forget that even in the serial world, processors have vector parallelism, and hence time-to-solution might not be exactly what we predict from the cost analysis. &lt;/p&gt;&#xA;" OwnerUserId="26771" LastActivityDate="2019-01-24T21:34:02.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30933" PostTypeId="1" AcceptedAnswerId="30940" CreationDate="2019-01-25T10:27:07.897" Score="4" ViewCount="857" Body="&lt;p&gt;I have a symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; which I want to numerically project onto the positive semi definite cone.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do so, I decompose it into &lt;span class=&quot;math-container&quot;&gt;$M = QDQ^T$&lt;/span&gt; and transform all negative eigenvalues to zero. (according to this post for example &lt;a href=&quot;https://scicomp.stackexchange.com/questions/30631/how-to-find-the-nearest-a-near-positive-definite-from-a-given-matrix&quot;&gt;How to find the nearest/a near positive definite from a given matrix?&lt;/a&gt;)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I numerically do this (double precision), if M is quite large (say 100*100), the matrix I obtain is not PSD, (according to me, due to numerical imprecision) and I'm obliged to repeat the process a long time to finally get a PSD matrix. My question is : is it a normal side-effect ? If so, is there a trick I missed or a better way to do it ?   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for the help. &lt;/p&gt;&#xA;" OwnerUserId="29789" LastEditorUserId="29789" LastEditDate="2019-01-25T12:51:44.440" LastActivityDate="2019-01-26T12:37:36.907" Title="Numerically find the nearest positive semi definite matrix to a symmetric matrix" Tags="|linear-algebra|eigenvalues|numerics|matrix-factorization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30936" PostTypeId="1" AcceptedAnswerId="30937" CreationDate="2019-01-25T14:51:11.327" Score="1" ViewCount="77" Body="&lt;p&gt;I am trying to fit some data points &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; to a non-linear model function &lt;span class=&quot;math-container&quot;&gt;$m_i$&lt;/span&gt;, which depends on a number of fit parameters &lt;span class=&quot;math-container&quot;&gt;$f_k$&lt;/span&gt; (I want to determine these) and also on some known, constant values &lt;span class=&quot;math-container&quot;&gt;$c_j$&lt;/span&gt;. For this I am using the Levenberg–Marquardt (LM) algorithm, which iteratively minimizes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\chi^2=\sum_i\left[\frac{d_i-m_i\left(f_1,f_2,\ldots,c_1,c_2,\ldots\right)}{\sigma_i\left(d_i\right)}\right]^2\,,\tag{1}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\sigma_i^2\left(d_i\right)$&lt;/span&gt; is the variance of &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt; (I'll call &lt;span class=&quot;math-container&quot;&gt;$\sigma_i$&lt;/span&gt; the &lt;em&gt;error&lt;/em&gt; of &lt;span class=&quot;math-container&quot;&gt;$d_i$&lt;/span&gt;). I have an estimate for the errors, so let's say they are known.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I want to deal with the &quot;constant&quot; values &lt;span class=&quot;math-container&quot;&gt;$c_j$&lt;/span&gt;, which actually come from different measurements and thus have some (known) error themselves. My first idea is to adjust the term in the denominator, with the simplest type of error propagation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\chi^2=\sum_i\left[\frac{d_i-m_i\left(f_1,f_2,\ldots,c_1,c_2,\ldots\right)}{\sqrt{\sigma_i^2\left(d_i\right)+\sum_j\left(\frac{\partial{}m_i}{\partial{}c_j}\sigma\left(c_j\right)\right)^2}}\right]^2\,.\tag{2}&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how to calculate the partial derivatives &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial{}m_i}{\partial{}c_j}$&lt;/span&gt; correctly, since they depend on the current value of the fit parameters &lt;span class=&quot;math-container&quot;&gt;$f_k$&lt;/span&gt;. I see two solutions, but I don't find either very satisfying:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Apply the LM algorithm multiples times. For example: Calculate the derivatives with a starting guess &lt;span class=&quot;math-container&quot;&gt;$f_k^0$&lt;/span&gt; and execute the LM fit, giving the solution &lt;span class=&quot;math-container&quot;&gt;$f_k^1$&lt;/span&gt;. Then, use &lt;span class=&quot;math-container&quot;&gt;$f_k^1$&lt;/span&gt; to calculate the derivatives and execute a second LM fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Adjust the LM algorithm code to actually recalculate the denominator after each LM-iteration with the current best estimates of &lt;span class=&quot;math-container&quot;&gt;$f_k$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would you recommend I approach this problem? Any hints in general or reading suggestions are also appreciated.&lt;/p&gt;&#xA;" OwnerUserId="21811" LastEditorUserId="21811" LastEditDate="2019-01-26T11:02:23.020" LastActivityDate="2019-01-26T11:02:23.020" Title="Correct weighting in least squares fitting" Tags="|nonlinear-equations|error-estimation|least-squares|regression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30937" PostTypeId="2" ParentId="30936" CreationDate="2019-01-25T21:13:41.200" Score="2" Body="&lt;p&gt;So the correct way is to use the probability distribution of your &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in the modeling process. Imho, the most natural way of describing and solving this type of problem is the &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Bayesian approach&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P(f|X,m)=\frac{P(X|f,m)P(f|m)}{P(X|m)}.$$&lt;/span&gt;&#xA;&lt;strong&gt;Noise free &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have the data points &lt;span class=&quot;math-container&quot;&gt;$X_i=(x_i,d_i)$&lt;/span&gt;. The model &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; predicts the values of &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$d(x)=m(x,f)+\epsilon$$&lt;/span&gt; with parameters &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and e.g. &lt;span class=&quot;math-container&quot;&gt;$\epsilon =d-m=\mathcal{N}(0,\sigma)$&lt;/span&gt;. You assume &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is noise free but &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is not. The likelihood for observing one data point &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; is &#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(X_i|f,m)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{\left(d_i-m(x_i,f)\right)^2}{2\sigma^2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the log likelihood for all data points is&#xA; &lt;span class=&quot;math-container&quot;&gt;$$\ln(P(X|f,m))=\sum_i \ln\left(P(d_i|x_i,f,m)\right),$$&lt;/span&gt; which leads to your minimization expression &lt;span class=&quot;math-container&quot;&gt;$$\ln(P(X|f,m)) \propto \sum_i\left[\frac{d_i-m_i\left(f_1,f_2,\ldots,c_1,c_2,\ldots\right)}{\sigma}\right]^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the parameters &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; you are interested in the posterior pdf &lt;span class=&quot;math-container&quot;&gt;$$P(f|X,m)= \frac{P(X|f,m)P(f|m)}{P(X|m)}.$$&lt;/span&gt;&#xA;This can be solved by assuming some prior for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and doing markov chain monte carlo (MCMC) sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model evidence &lt;span class=&quot;math-container&quot;&gt;$P(X|m)$&lt;/span&gt; is the likelihood averaged over the model parameters and tells how well the data are predicted given the model.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(X|m)=\int P(X,f|m)df=\int P(X|f,m)P(f|m)df$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Noisy &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you do the same exercise again with the additional parameter &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and construct the posterior pdf for &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; given &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, also the parameter distribution &lt;span class=&quot;math-container&quot;&gt;$P(c)=\mathcal{N}(c_m,\sigma_c)$&lt;/span&gt; is known and independent of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The posterior pdf &lt;span class=&quot;math-container&quot;&gt;$P(f|X,m)$&lt;/span&gt; is then the marginalization of &lt;span class=&quot;math-container&quot;&gt;$P(f|X,c,m)$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(f|X,m)=\int P(f|X,c,m)P(c)dc.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Bayesian modeling there are several frameworks which provide MCMC sampling also for hierarchical models. e.g. &lt;a href=&quot;https://pymc-devs.github.io/pymc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyMC&lt;/a&gt; is capable of doing the MCMC calculation together with an additional distribution for &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. There is however a drawback. If the model &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is very expensive, sampling can quickly get out of hands.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2019-01-25T21:13:41.200" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30940" PostTypeId="2" ParentId="30933" CreationDate="2019-01-26T12:37:36.907" Score="7" Body="&lt;p&gt;After you compute &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, form &lt;span class=&quot;math-container&quot;&gt;$D'=\max(D,0)$&lt;/span&gt;, and compute &lt;span class=&quot;math-container&quot;&gt;$A'=QD'Q^\top$&lt;/span&gt;, the algorithms involved in multiplying those matrices do not promise that &lt;span class=&quot;math-container&quot;&gt;$A'$&lt;/span&gt; will be exactly &lt;span class=&quot;math-container&quot;&gt;$QD'Q^\top$&lt;/span&gt;. Most commonly, they are backward stable, and promise that the actual floating-point output will be &lt;span class=&quot;math-container&quot;&gt;$(Q+\delta Q)(D'+\delta D')(Q+\delta Q)^\top$&lt;/span&gt;, for some small perturbations &lt;span class=&quot;math-container&quot;&gt;$\delta Q,\delta D'$&lt;/span&gt;. The actual perturbations don't matter here, but they should be small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when you replace the diagonal entries of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; with zeros, backward stability may still allow roundoff errors to make it look like those zeros were replaced by very small &quot;random&quot; numbers. In effect you can think of it as computing &#xA;&lt;span class=&quot;math-container&quot;&gt;$$Q\max(D, \epsilon\times\mathrm{randn})Q^\top.$$&lt;/span&gt;&#xA;If you replace the zeros of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; with a small number &lt;span class=&quot;math-container&quot;&gt;$x&amp;gt;0$&lt;/span&gt; instead, then perturbing &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; will no longer change its sign and the eigenvalues will come out all positive. &#xA;This is a common issue with basically any floating point comparison where you compare &lt;span class=&quot;math-container&quot;&gt;$u&amp;gt;0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is computed approximately with roundoff errors.&lt;/p&gt;&#xA;" OwnerUserId="713" LastActivityDate="2019-01-26T12:37:36.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30941" PostTypeId="1" CreationDate="2019-01-26T15:21:46.737" Score="10" ViewCount="11175" Body="&lt;p&gt;I am developing a 2D CFD solver for fluid-particle interaction. To solve Navier-Stokes equations on a grid of size &lt;span class=&quot;math-container&quot;&gt;$10000\times 10000$&lt;/span&gt; cells (or &gt;1 million cells), a large linear system &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; being the &lt;span class=&quot;math-container&quot;&gt;$10000\times10000$&lt;/span&gt; sparse coefficient matrix needs to be solved efficiently in each time-step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am looking for is a high-performance/parallel C++ linear algebra library to solve this large sparse linear system. An iterative approach such as biconjugate gradient stabilized method is preferred.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of existing libraries out there like: &lt;strong&gt;&lt;em&gt;Eigen3, PETSc, Trilinos, MLT4, GNU GSL, Armadillo, LAPACK++&lt;/em&gt;&lt;/strong&gt;, and the list goes on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among the well-known libraries, which one should I choose for my project, in terms of high-performance (better with OpenMP/MPI support), and easy-to-use in c++.&lt;/p&gt;&#xA;" OwnerUserId="28844" LastEditorUserId="20688" LastEditDate="2019-01-26T23:38:03.007" LastActivityDate="2019-01-27T18:41:43.670" Title="Which C++ linear algebra library is probably the fastest on solving huge sparse [square matrix] linear system?" Tags="|linear-algebra|linear-solver|sparse-matrix|conjugate-gradient|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30942" PostTypeId="2" ParentId="30922" CreationDate="2019-01-26T22:44:26.023" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://www.sagemath.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sagemath&lt;/a&gt; looks fast enough. Here is an example created by taking a 999x1000 random rational matrix and appending its column sums at the bottom, so that [1, 1, ... , 1, -1] is the kernel.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sage: A = random_matrix(QQ, 999, 1000)&#xA;sage: B = A.stack(sum(A))&#xA;sage: %time v = B.kernel()&#xA;CPU times: user 5.21 s, sys: 36.4 ms, total: 5.24 s&#xA;Wall time: 5.24 s&#xA;Vector space of degree 1000 and dimension 1 over Rational Field&#xA;Basis matrix:&#xA;1 x 1000 dense matrix over Rational Field&#xA;sage: v.basis()&#xA;[&#xA;(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1)&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but, as stated in @Kirill's comment, all depends on the growth of the numerators and denominators of the rationals involved in your specific example. Exact computations with rationals can blow up pretty quick.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2019-01-26T22:44:26.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30944" PostTypeId="2" ParentId="30941" CreationDate="2019-01-27T18:41:43.670" Score="11" Body="&lt;p&gt;Eigen 3 is a nice C++ template library some of whose routines are parallelized.&#xA;&lt;a href=&quot;http://eigen.tuxfamily.org/dox/TopicMultiThreading.html&quot; rel=&quot;noreferrer&quot;&gt;c.f. Eigen documentation&lt;/a&gt;&#xA;The parallelization is OMP only, so if you intend to parallelise using MPI (and OMP) it is probably not suitable for your purpose.&#xA;The nice feature of Eigen is that you can swap in a high performance BLAS library (like MKL or OpenBLAS) for some routines by simply using &lt;code&gt;#define EIGEN_USE_BLAS&lt;/code&gt; (and other macros).&lt;/p&gt;&#xA;&lt;p&gt;Similarly Armadillo allows for node-level parallelism only. In my experience it is better to use Eigen since it is easier to interface with the raw C++ arrays in Eigen, which facilitates use of other libraries (e.g. ARPACK++).&lt;/p&gt;&#xA;&lt;p&gt;In my experience I would advise against using GSL for linear algebra. I have found its performance to be lacking and the usability to be worse than that of Eigen.&lt;/p&gt;&#xA;&lt;p&gt;If you plan to execute linear solvers (e.g. BiCGstab) on multiple nodes I would advise you to use Trilinos. I have used it in my research codes with fairly little delving into its documentation due to the good examples available on the Trilinos homepage. Furthermore its performance is decent and can be fine-tuned by including good BLAS/LAPACK libraries during the compilation.&lt;/p&gt;&#xA;&lt;p&gt;Similar should hold for PETSc, although I have never actively used its LA routines. In my experience PETSc is dependency hell, if you want to use performance-optimized (e.g. optimized for the CPU architecture you're using) versions of the libraries it requires. The performance should be fairly decent, too, I think, since PETSc relies on common LA libraries (BLAS, LAPACK, SCALAPACK etc.).&lt;/p&gt;&#xA;&lt;p&gt;Long story short: For interoperability and good performance on a single node (using OpenMP) I advise to use &lt;strong&gt;Eigen with OpenBLAS&lt;/strong&gt;. If you want to use multiple nodes via MPI and let the library figure out how to solve a system using multiple nodes then use &lt;strong&gt;Trilinos&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28993" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2019-01-27T18:41:43.670" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30945" PostTypeId="1" CreationDate="2019-01-27T20:24:30.297" Score="2" ViewCount="78" Body="&lt;p&gt;I'm trying to solve the following PDE by Matlab,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_t-\Delta u = 0, \quad \text{in}\quad \Omega\times (0,T)&#xA;\tag{1}&#xA;$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_t-\Delta_\Gamma u + \partial_\nu u=0,\quad \text{on}\quad \Gamma\times(0,T)&#xA;\tag{2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; is a bounded domain of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\Gamma =\partial \Omega$&lt;/span&gt; is the boundary of &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\partial_\nu$&lt;/span&gt; is the normal derivative, and &lt;span class=&quot;math-container&quot;&gt;$\nu$&lt;/span&gt; is the outer unit vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For simplicity, I considered the one dimensional case when &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; is the interval &lt;span class=&quot;math-container&quot;&gt;$(0,1)$&lt;/span&gt;, and discretized the equation following the Finite Difference method (we can neglect the Laplace-Beltrami operator &lt;span class=&quot;math-container&quot;&gt;$\Delta_\Gamma$&lt;/span&gt; in this case), but I didn't find how to solve such PDE with the dynamic boundary conditions (2) in a simple way using &lt;code&gt;pdepe&lt;/code&gt; or PDE toolbox. Usual Matlab functions use the usual boundary conditions (Dirichlet or Neumann), but the dynamic boundary conditions are not included. So I'm asking how to solve this PDE with Matlab in a simple way.&#xA;For the 2D heat equation can we define &lt;span class=&quot;math-container&quot;&gt;$\Delta_\Gamma$&lt;/span&gt; by Matlab tools.&lt;/p&gt;&#xA;" OwnerUserId="29814" LastEditorUserId="29814" LastEditDate="2019-01-27T21:18:11.707" LastActivityDate="2019-01-27T21:18:11.707" Title="Unusual boundary conditions on Matlab" Tags="|matlab|finite-difference|boundary-conditions|parabolic-pde|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30946" PostTypeId="1" AcceptedAnswerId="30965" CreationDate="2019-01-28T00:53:39.960" Score="4" ViewCount="749" Body="&lt;p&gt;The problem I am trying to solve involves minimising a function with respect to a large number (probably 10,000+) of parameters. I can cheaply compute both its Jacobian and its Hessian. The Hessian is very helpful; I am not entirely sure the function is convex, but using the &lt;code&gt;Newton-CG&lt;/code&gt; algorithm in &lt;code&gt;scipy.optimize&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;result = minimize(to_minimize, f, jac=jac, hess=hess, method='Newton-CG')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;converges quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far so good, but unfortunately, &lt;code&gt;scipy.optimize.minimize&lt;/code&gt; is unable to deal with sparse Hessians, so I have to convert the extremely sparse Hessian to a dense matrix. Internally, I believe the &lt;code&gt;Newton-CG&lt;/code&gt; method multiplies that Hessian with other things, which I assume will also be much slower than using its sparseness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, my question is: does anyone know of a library that can exploit the sparse Hessian, preferably using the &lt;code&gt;Newton-CG&lt;/code&gt; algorithm, or do I have to write one myself? It would be nice if it were in python, but I'm happy to use C++ if that is more fruitful.&lt;/p&gt;&#xA;" OwnerUserId="27250" LastActivityDate="2019-01-30T22:30:00.060" Title="Minimize a function with sparse Hessian" Tags="|optimization|sparse-matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="30947" PostTypeId="1" CreationDate="2019-01-28T11:01:20.997" Score="0" ViewCount="64" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0QslT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0QslT.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a sequence of points which was obtained from an iterative algorithm, and I computed the order of convergence &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; of the method using the formula &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;p \approx \frac{\log({\rm err}(k+2))-\log({\rm err}(k+1))}{\log({\rm err}(k+1)) - \log({\rm err}(k))}.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first computed ratio is much larger compared to the other ones. Without removing this value from the complete list of values, I obtained &lt;span class=&quot;math-container&quot;&gt;$p = 1.2615$&lt;/span&gt; as the order of convergence of my algorithm. However, when I computed the order of convergence after removing the outlier (the first calculated ratio), I got &lt;span class=&quot;math-container&quot;&gt;$p = 1.0495$&lt;/span&gt; (see figure). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, which of the two is the correct order of convergence of my algorithm? &lt;/p&gt;&#xA;" OwnerUserId="29818" LastActivityDate="2019-01-28T11:01:20.997" Title="Correct order of convergence" Tags="|convergence|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30948" PostTypeId="2" ParentId="30909" CreationDate="2019-01-28T12:09:24.807" Score="5" Body="&lt;p&gt;This doesn't answer your question directly but instead suggests&#xA;an alternate strategy. In general, unless you are interested in&#xA;experimenting with numerical methods, I recommend using an existing&#xA;PDE solver instead of trying to write one yourself. This advice&#xA;is particularly true for numerically-challenging problems, like&#xA;yours appears to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are interested in Python, I have written a &lt;a href=&quot;https://github.com/usnistgov/fipy&quot; rel=&quot;noreferrer&quot;&gt;FiPy&lt;/a&gt; version of your&#xA;problem, as shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I can't say that FiPy easily obtained a solution.&#xA;But I have done some numerical experimentation and made a few &#xA;observations. First, I should acknowledge that I don't understand&#xA;much about the physics of these equations.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I changed your boundary conditions to zero-flux at the ends to&#xA;be consistent with your initial conditions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The laser pulse doesn't seem to be adding much heat to the system;&#xA;the temperatures as a function of time change very little from the&#xA;initial values. The temperature changes are so small, I have been plotting the difference in temperature compared with the initial conditions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The large value of &lt;code&gt;Gel&lt;/code&gt; appears to make the problem very numerically&#xA;stiff and more difficult to solve. I've been experimenting with values&#xA;several orders of magnitude less.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The Python code using the FiPy PDE solver is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import fipy&#xA;# sample parameter Gold &#xA;la0 = 429 # conductivity in W/mK &#xA;gma = 62.8 # thermal constant J/m^3K^2&#xA;Cl = 2.6e6 # phonon specific heat  in J/m^3K&#xA;&#xA;Gel = 3.5e16 # lelectron phonon coupling constant &#xA;#Gel=1e10 # reduce value to improve numerics&#xA;tau_e = 0.04 # e relaxation t const &#xA;tau_l = 0.6 # l relaxation t const&#xA;&#xA;Tm = 300.&#xA;L = 100 # sample thickness in nm &#xA;# Laser pulse parameters &#xA;T = 6.&#xA;T=1&#xA;&#xA;R = 0.93 # reflectivity &#xA;I0 = 100 # fulence in J/m^2&#xA;tp = 0.1 # in ps &#xA;zz0 = 15.3 # penetration depth in nm&#xA;&#xA;&#xA;#F = 0.5&#xA;&#xA;def las(x,t):&#xA;    pt = 0.93*((1-R)/(zz0*tp))*I0*np.exp(-(x/zz0)-2.772*((t-2*tp)**2/tp**2))&#xA;    return pt&#xA;&#xA;def I(x):&#xA;    tin = Tm&#xA;    return tin&#xA;&#xA;a = la0 # Ked&#xA;&#xA;nx = 100&#xA;dx = L/nx&#xA;&#xA;mesh = fipy.Grid1D(nx = nx, dx = dx)&#xA;x = mesh.cellCenters[0]&#xA;#pt25 = mesh._getNearestCellID(([25.],))&#xA;u = fipy.CellVariable(name=&quot;u&quot;, mesh=mesh)&#xA;v = fipy.CellVariable(name=&quot;v&quot;, mesh=mesh)&#xA;valueLeft = 0&#xA;valueRight = 0&#xA;applyDirichletConstraint=False&#xA;if(applyDirichletConstraint):&#xA;  u.constrain(valueRight, mesh.facesRight)&#xA;  u.constrain(valueLeft, mesh.facesLeft)&#xA;  v.constrain(valueRight, mesh.facesRight)&#xA;  v.constrain(valueLeft, mesh.facesLeft)&#xA;else:&#xA;  u.faceGrad.constrain(valueRight, mesh.facesRight)&#xA;  u.faceGrad.constrain(valueLeft, mesh.facesLeft)&#xA;  v.faceGrad.constrain(valueRight, mesh.facesRight)&#xA;  v.faceGrad.constrain(valueLeft, mesh.facesLeft)   &#xA;pi=np.math.pi&#xA;#Gumv = Gel*(u-v) # explicit source terms are not appropriate for this stiff system&#xA;Gumv = fipy.ImplicitSourceTerm(Gel, var=u) - fipy.ImplicitSourceTerm(Gel, var=v)&#xA;time = fipy.Variable()&#xA;&#xA;eqn0 = fipy.TransientTerm(gma*u, var=u) == fipy.DiffusionTerm(a, var=u) - Gumv + las(x,time)&#xA;eqn1 = fipy.TransientTerm(Cl, var=v) == fipy.DiffusionTerm(a, var=v) + Gumv&#xA;eqn = eqn0 &amp;amp; eqn1&#xA;steps = 5000&#xA;dt = T/steps&#xA;&#xA;ival = I(x)&#xA;&#xA;u.setValue(ival)&#xA;v.setValue(ival)&#xA;centerX=([L/2.],)&#xA;uCenter = [u(centerX)[0]]&#xA;vCenter = [v(centerX)[0]]&#xA;ts  = [0.]&#xA;t = 0&#xA;for step in range(steps):&#xA;  time.setValue(t+dt/2.)&#xA;  eqn.solve(dt=dt)&#xA;  uCenter.append(u(centerX)[0])&#xA;  vCenter.append(v(centerX)[0])&#xA;  t += dt&#xA;  ts.append(t)&#xA;&#xA;# plot difference from intial value  &#xA;uCenter = np.asarray(uCenter) - Tm&#xA;vCenter = np.asarray(vCenter) - Tm&#xA;&#xA;import matplotlib.pyplot as plot&#xA;plot.subplot(211);&#xA;plot.title('nx=%d, dt=%12.3e, Gel=%g' % (nx, dt, Gel))&#xA;p1=plot.plot(ts, uCenter)&#xA;plot.grid(b='on')&#xA;plot.xlabel('Time')&#xA;plot.ylabel('u at center')&#xA;&#xA;plot.subplot(212);&#xA;p2=plot.plot(ts, vCenter)&#xA;plot.grid(b='on')&#xA;plot.xlabel('Time')&#xA;plot.ylabel('v at center')&#xA;&#xA;plot.savefig('plot.pdf')&#xA;&#xA;plot.show()  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2019-01-28T12:09:24.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30949" PostTypeId="1" AcceptedAnswerId="30950" CreationDate="2019-01-28T14:19:13.377" Score="1" ViewCount="172" Body="&lt;p&gt;What is difference beetwen this two method? Are these methods far from each other or are these methods complement each other? Could you take an example?&lt;/p&gt;&#xA;" OwnerUserId="17965" LastEditorUserId="17965" LastEditDate="2019-01-28T18:07:52.853" LastActivityDate="2019-01-28T18:07:52.853" Title="Relation between conjugate gradient method and finite elements method" Tags="|finite-element|numerics|conjugate-gradient|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30950" PostTypeId="2" ParentId="30949" CreationDate="2019-01-28T16:34:03.217" Score="5" Body="&lt;p&gt;The methods are unrelated. The finite element method is a way to convert a partial differential equation into a finite dimensional problem (&quot;discretization&quot;) so it can be solved on a computer. The Conjugate Gradient method is a method to solve linear systems such as those that arise from the finite element method.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-01-28T16:34:03.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30951" PostTypeId="1" AcceptedAnswerId="30953" CreationDate="2019-01-28T16:35:15.627" Score="7" ViewCount="653" Body="&lt;p&gt;It is well-known that Newton's method can converge quadratically, if initial guess is close enough and if the arising linear systems are solved accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am applying Newton's method to highly ill-conditioned systems (with condition number around &lt;span class=&quot;math-container&quot;&gt;$10^{14}$&lt;/span&gt;). I am sure that my Hessian is computed correctly, however I never obtain quadratic convergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that the problem comes from the inability to solve linear systems with sufficient accuracy (I am using LU decomposition with iterative refinement). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know about any reference, which discusses how speed of convergence of Newton's method depends on accuracy of the obtained correction? &lt;/p&gt;&#xA;" OwnerUserId="29268" LastEditorUserId="9667" LastEditDate="2019-01-28T16:53:42.857" LastActivityDate="2019-02-10T17:51:02.997" Title="Lack of quadratic convergence in Newton's method" Tags="|optimization|convergence|newton-method|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="30952" PostTypeId="1" AcceptedAnswerId="30958" CreationDate="2019-01-28T18:39:33.300" Score="3" ViewCount="213" Body="&lt;p&gt;I am looking for examples of poorly conditioned sums which can rapidly be evaluated, for the purposes of unit testing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently using the series representation for &lt;span class=&quot;math-container&quot;&gt;$\ln(2)$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum_{n=1}^{\infty} \frac{(-1)^{n+1}}{n}&#xA;$$&lt;/span&gt;&#xA;which has an infinite summation condition number, but in double precision, I can only get the condition number to ~22 after an obscene number of terms (terrible for a unit test), and the error in the naive summation is not large. So I guess that this sum is numerically well-conditioned!&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2019-01-30T10:46:12.897" Title="Poorly conditioned, easily evaluated sum for unit testing" Tags="|numerics|condition-number|testing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30953" PostTypeId="2" ParentId="30951" CreationDate="2019-01-28T19:46:45.630" Score="9" Body="&lt;p&gt;Yes: See Higham's book &quot;Accuracy And Stability of Numerical Algorithms&quot;, second edition, chapter 25: Nonlinear Systems and Newton's Method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, see the section on the &quot;limiting residual&quot; in terms of a condition number for the Jacobian. It may well be that since your system is ill-conditioned, that you quickly hit the limiting residual and your iteration meanders aimlessly. I would try long double or quad precision, if that's available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;: Higham's treatment of this subject is very advanced. It may be advisable to first read Corless's &quot;A Graduate Introduction to Numerical Methods&quot;, section 3.2.1, The Condition Number of A Root, which treats conditioning of the 1D rootfinding problem, along with a discussion of the 1D Newton's method.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2019-02-10T17:51:02.997" LastActivityDate="2019-02-10T17:51:02.997" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30954" PostTypeId="2" ParentId="30896" CreationDate="2019-01-28T22:56:01.943" Score="4" Body="&lt;p&gt;I did some more investigating about the accuracy of the standard recurrence relation method vs. the newer Bunck algorithm. It seems that in fact the Bunck algorithm is generally more accurate for all &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. I made a python script using the mpmath arbitrary precision library to calculate the Hermite functions to lots of decimal places, and then compared that result with the GSL recurrence relation approach and the Bunck method as implemented in LonelyProf's answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a plot for &quot;small&quot; &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt;.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/v68o3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v68o3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;The blue dots are mpmath-GSL (i.e. the recurrence relation result). The green dots are mpmath-Bunck (labeled NEW). Here I used &lt;span class=&quot;math-container&quot;&gt;$n = 10,000$&lt;/span&gt;. The red lines show &lt;span class=&quot;math-container&quot;&gt;$\pm \epsilon_{mach} = \pm 2.22e-16$&lt;/span&gt;. For this range of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, there isn't much difference in accuracy, and both algorithms give an absolute error close to the machine epsilon. The Bunck algorithm gives a slightly smaller rms of &lt;span class=&quot;math-container&quot;&gt;$2.50e-16$&lt;/span&gt; compared with &lt;span class=&quot;math-container&quot;&gt;$2.59e-16$&lt;/span&gt; for GSL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to explore larger &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; ranges, I have the below figure:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gDIc8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gDIc8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the horizontal axis is plotted as &lt;span class=&quot;math-container&quot;&gt;$1/x$&lt;/span&gt;, corresponding to &lt;span class=&quot;math-container&quot;&gt;$x \in [-\infty,-1]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x \in [1,\infty]$&lt;/span&gt;. You can see that near &lt;span class=&quot;math-container&quot;&gt;$1/x \approx 0$&lt;/span&gt;, the error in the GSL algorithm based on recurrences blows up substantially. This occurs near &lt;span class=&quot;math-container&quot;&gt;$1/x \approx 0.025$&lt;/span&gt; which corresponds to &lt;span class=&quot;math-container&quot;&gt;$x = 40$&lt;/span&gt;, which as the previous authors note, is where &lt;span class=&quot;math-container&quot;&gt;$\exp{-x^2/2}$&lt;/span&gt; underflows in double precision. Reducing the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; scale in this figure gives us:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/K07vQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K07vQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which shows that both methods give similar accuracy until the &quot;large x&quot; region in the middle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made similar plots for a wide range of different &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; values and found basically the same behavior overall. My conclusion is that the new Bunck algorithm is as accurate as recurrences for small &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, and more accurate for large &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. If Bunck's claim is true that his method is also faster than recurrences, which he says in his paper (I have not done any benchmarks), then I would conclude his method is superior to the recurrence method in general.&lt;/p&gt;&#xA;" OwnerUserId="28440" LastActivityDate="2019-01-28T22:56:01.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30957" PostTypeId="1" AcceptedAnswerId="30962" CreationDate="2019-01-29T13:10:17.590" Score="1" ViewCount="1286" Body="&lt;p&gt;I have a little background in writing toy finite volume CFD codes. In 2D Cartesian scenarios, I typically take &lt;span class=&quot;math-container&quot;&gt;$x_{\min}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x_{\max}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y_{\min}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$y_{\max}$&lt;/span&gt;, and the number of points in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; to calculate &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Delta y$&lt;/span&gt;. &#xA;Then, I compute cell centers and assemble a 1D array of cells. &#xA;To perform field operations, I loop over this array of cells, storing items at node centers and interpolating to faces as necessary. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In researching more advanced projects, I see that a face-based data structure is advantageous in terms of conservation, flexibility, and application of boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question:&lt;/strong&gt; For a Cartesian 2D domain, how does one generate a list of faces from arrays of vertex coordinates? &#xA;These faces should compose quadrilateral cells.&#xA;Can this be done explicitly? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, I could create an extremely simple 2D Cartesian mesh in something like &lt;code&gt;Gmsh&lt;/code&gt;, but then I'd have to read in a mesh and parse the file.&#xA;I can write this functionality eventually, but for my own edification, I want to quickly explore a face-based code.  &lt;/p&gt;&#xA;" OwnerUserId="10256" LastEditorUserId="20688" LastEditDate="2019-01-30T19:23:30.250" LastActivityDate="2019-01-30T19:23:30.250" Title="How to generate a face list from vertices?" Tags="|computational-geometry|finite-volume|mesh-generation|data-structures|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30958" PostTypeId="2" ParentId="30952" CreationDate="2019-01-29T19:10:42.293" Score="10" Body="&lt;p&gt;The condition number of sum &lt;span class=&quot;math-container&quot;&gt;$s(x) = \sum_{j=1}^n x_j$&lt;/span&gt; is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \kappa(x) = \frac{\sum_{j=1}^n |x_j|}{|\sum_{j=1}^n x_j|} = \frac{s(|x|)}{|s(x)|}$$&lt;/span&gt; and reflects the sums sensitivity to small changes in the input. Specifically, we have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \underset{\epsilon \rightarrow 0_+}{\lim}\sup \left\{ \frac{1}{\epsilon} \left|\frac{s(x+\Delta x) - s(x)}{s(x)} \right| \: : \: |\Delta x_j| \leq \epsilon |x_j|\right\} = \kappa(x).$$&lt;/span&gt;&#xA;If all &lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt; have the same sign, then the sum is well-conditioned. If the &lt;span class=&quot;math-container&quot;&gt;$s(x) \approx 0$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$s(|x|)$&lt;/span&gt; is large, then the sum is ill-conditioned. A good example of a sum whose conditioning is well understood is the polynomial &lt;span class=&quot;math-container&quot;&gt;$p_n$&lt;/span&gt; given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ p_n(x) = \sum_{j=0}^n \frac{x^j}{j!}$$&lt;/span&gt;&#xA;This is the Taylor polynomial of order &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; for the natural exponential function at the point &lt;span class=&quot;math-container&quot;&gt;$x_0 = 0$&lt;/span&gt;. The sum is ill-conditioned for large negative values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and well-conditioned for all positive values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is positive and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is so large that &lt;span class=&quot;math-container&quot;&gt;$p_n(x) \approx e^x$&lt;/span&gt; is a good approximation, then &lt;span class=&quot;math-container&quot;&gt;$1/e^{x} \approx p_n(-x)$&lt;/span&gt; is a good approximation as well. In short, you have an ill-conditioned sum for which a good approximation is easy to obtain.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2019-01-30T10:46:12.897" LastActivityDate="2019-01-30T10:46:12.897" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30961" PostTypeId="1" AcceptedAnswerId="30977" CreationDate="2019-01-30T15:26:50.800" Score="1" ViewCount="649" Body="&lt;p&gt;I want to calculate L2 norm of a 3D unstructured grid to compare my simulation results in two different mesh sizes as coarse and fine. I read &lt;a href=&quot;https://scicomp.stackexchange.com/a/5130&quot;&gt;this answer&lt;/a&gt; and it seems in three-dimensional space, I should use this formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L^{2}-norm = \sqrt{\sum_{\Omega} (\phi_{coarse}-\phi_{fine})^{2} \Delta x^{3}}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My grid is a uniform grid with size &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; but it is unstructured and as a result, &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; is stored as a 1D vector based on ids of each point in unstructured grid instead of having &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; as a three-dimensional matrix as: &lt;span class=&quot;math-container&quot;&gt;$\phi(i,j,k)$&lt;/span&gt;, which is common for structured grids. My question is: &lt;strong&gt;Should I use the above formula with power 3 of &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; or cause I stored my results as 1D vectors, drop the power 3 and use this formula instead?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L^{2}-norm = \sqrt{\sum_{\Omega} (\phi_{coarse}-\phi_{fine})^{2} \Delta x}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really appreciate any suggestion or answer.&lt;/p&gt;&#xA;" OwnerUserId="28872" LastEditorUserId="28872" LastEditDate="2019-02-01T19:22:14.070" LastActivityDate="2019-07-02T19:01:45.220" Title="How I could calculate L2 norm of an unstructured grid?" Tags="|numerics|unstructured-mesh|l2-norm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30962" PostTypeId="2" ParentId="30957" CreationDate="2019-01-30T15:35:19.900" Score="1" Body="&lt;p&gt;As with many questions in computation, a lot of this comes down to what you are trying to achieve. For structured meshes on quadrilaterals, the best and most efficient way to deal with this is not to bother and work implicitly, as you've (kind of suggested). If your cell centred variable has index pair (i,j), then you can label faces to the left/right (depending on your numbering and  boundary conditions) as (i,j) and faces up/down as (i,j) and keep the two lists separate. If you are working on what is sometimes known as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Arakawa_grids&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arakawa C grid&lt;/a&gt;, then this also gives you the indices for your U and V variables respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This becomes a little bit more interesting when your mesh is truly unstructured. In that case, it's likely that you are building your entire topology from a set of mappings from an element/cell index onto the vertex indices. Indeed this is what a meshing program such as GMSH fundamentally gives you. For a structured mesh, you could cheat if necessary by e.g. numbering both cells and vertices outwards  in a spiral. A fairly simple algorithm to get face neighbour pairs is then something like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Build a mapping &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; from vertices/nodes to cells/elements, by sweeping through your cell list once.&lt;/li&gt;&#xA;&lt;li&gt;For cell &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, work around the edges in order. &lt;/li&gt;&#xA;&lt;li&gt;For each edge, identify both vertices, &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;For each cell in &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; that &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; belongs to, check if &lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt; also belongs to it. The first one you find which isn't &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the neighbour . If &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;&amp;lt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, then this is the first time you've found that face.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To repeat though, this is overkill for structured meshes.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2019-01-30T15:35:19.900" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30963" PostTypeId="1" AcceptedAnswerId="30967" CreationDate="2019-01-30T19:18:18.257" Score="1" ViewCount="265" Body="&lt;p&gt;I have a 2D domain which is discretized using Q4 elements. I have the nodal positions and the element connectivity matrix. I would now like to renumber the nodes in such a way that all the interior nodes are labeled first followed the boundary nodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At present, the bandwidth of the stiffness matrix is not of concern. Hence the node numbers can be arbitrary as long as all the interior nodes are numbered first. Such kind of numbering makes the implementation of certain domain decomposition methods easy to write. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone suggest an efficient algorithm to accomplish this?&lt;/p&gt;&#xA;" OwnerUserId="23598" LastEditorUserId="20688" LastEditDate="2019-01-30T19:22:11.400" LastActivityDate="2019-01-31T17:17:16.187" Title="Node renumbering in a 2D mesh" Tags="|finite-element|computational-geometry|mesh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30964" PostTypeId="1" AcceptedAnswerId="30968" CreationDate="2019-01-30T22:04:12.393" Score="8" ViewCount="326" Body="&lt;p&gt;If one has a sufficiently smooth function &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; that is approximated by a piecewise constant function &lt;span class=&quot;math-container&quot;&gt;$u_h=\Pi^0_h u$&lt;/span&gt; on a mesh of cell size &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; (where &lt;span class=&quot;math-container&quot;&gt;$\Pi^0_h$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; projection onto the constants on each cell), then it is not hard to see graphically (or from the first term of the Taylor expansion) that the size of jumps must be &lt;span class=&quot;math-container&quot;&gt;${\cal O}(h)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But what if I use piecewise linear functions? Taylor expansion gives me that I should expect the difference between &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt; at the end points of each cell to be &lt;span class=&quot;math-container&quot;&gt;${\cal O}(h^2)$&lt;/span&gt;. But it is not inconceivable that the &quot;jump&quot; of &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt; at these end points (i.e., the difference &lt;span class=&quot;math-container&quot;&gt;$|u_h(x_+)-u_h(x_-)|$&lt;/span&gt; when approaching the point &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; that separates two cells from the left and right) might be of higher order than just two. How about projections &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt; into even higher order discontinuous spaces — say, of polynomial degree &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;: Is the jump then of size &lt;span class=&quot;math-container&quot;&gt;${\cal O}(h^{k+1})$&lt;/span&gt;, or is it even better?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am certain this is a pretty standard question for those who do error estimation for Discontinuous Galerkin finite element methods, but it's a bit outside my theoretic knowledge — and so help would be appreciated!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As a side note, here's what I'm really after: I have an estimate that involves on the right hand side the term &lt;span class=&quot;math-container&quot;&gt;$h^{s+1} \|u_h\|_{H^s}$&lt;/span&gt;. Since &lt;span class=&quot;math-container&quot;&gt;$u_h=\Pi^k u$&lt;/span&gt; is piecewise constant, it's not in &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt; or higher, but only &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt;. So &lt;span class=&quot;math-container&quot;&gt;$s=0$&lt;/span&gt; is really the only choice. That's likely the best I can do for piecewise constants (&lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But at least for higher degree piecewise polynomials, the jump becomes pretty small, and so while &lt;span class=&quot;math-container&quot;&gt;$\Pi_h^k u\not\in H^s$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$s&amp;gt;0$&lt;/span&gt;, one would expect that one could find an asymptotically equivalent norm &lt;span class=&quot;math-container&quot;&gt;$\|\cdot\|_\ast$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$\|\Pi_h^k u\|_{\ast,H^s} \le C h^{-p} \|u\|_{H^s}$&lt;/span&gt; or some such, for a power &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; that is probably related to the polynomial degree &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; of the space into which we project and, I believe, the space dimension. This estimate blows up, of course, as &lt;span class=&quot;math-container&quot;&gt;$h\rightarrow 0$&lt;/span&gt;, but at a rate that is predictable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want this because then I could replace my poor estimate &lt;span class=&quot;math-container&quot;&gt;$h\|u_h\|_{L_2}$&lt;/span&gt; by something more like &lt;span class=&quot;math-container&quot;&gt;$h^{s-p} \|u\|_{H^s}$&lt;/span&gt;. The question is what &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; needs to be: If the blow-up happens sufficiently slowly, then the power &lt;span class=&quot;math-container&quot;&gt;$s-p$&lt;/span&gt; may be substantially better than the order 1 one gets by just taking the simple route and estimating via &lt;span class=&quot;math-container&quot;&gt;$h\|\Pi_h^k u\|_{L_2}$&lt;/span&gt;. Calculating &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; clearly involves the size of the jump of &lt;span class=&quot;math-container&quot;&gt;$\Pi_hu$&lt;/span&gt; from one cell to the next, thus my question.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="9667" LastEditDate="2019-02-01T22:15:53.073" LastActivityDate="2019-02-06T16:03:23.980" Title="Size of jump for piecewise discontinuous approximations" Tags="|finite-element|discontinuous-galerkin|approximation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30965" PostTypeId="2" ParentId="30946" CreationDate="2019-01-30T22:30:00.060" Score="1" Body="&lt;p&gt;Thanks for everyone's comments!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I managed to solve this for my problem by writing my own Newton optimisation:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;from sksparse.cholmod import cholesky&#xA;&#xA;f = np.zeros(n_data)&#xA;&#xA;difference = 999&#xA;&#xA;while difference &amp;gt; 1e-5:&#xA;&#xA;    cur_hess = hess(f)&#xA;    cur_jac = jac(f)&#xA;&#xA;    cur_chol = cholesky(cur_hess)&#xA;    sol = cur_chol.solve_A(cur_jac)&#xA;&#xA;    new_f = f - sol&#xA;    difference = np.linalg.norm(f - new_f)&#xA;&#xA;    f = new_f&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm using &lt;code&gt;scikit-sparse&lt;/code&gt; to do a sparse Cholesky decomposition with &lt;code&gt;CHOLMOD&lt;/code&gt; here. This works really well for my problem and typically converges within a few iterations. Hope it's of use to some!&lt;/p&gt;&#xA;" OwnerUserId="27250" LastActivityDate="2019-01-30T22:30:00.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30967" PostTypeId="2" ParentId="30963" CreationDate="2019-01-31T15:34:49.543" Score="1" Body="&lt;p&gt;The first idea that comes to my mind:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multimap&quot; rel=&quot;nofollow noreferrer&quot;&gt;multimap&lt;/a&gt; &lt;code&gt;Edge2Quad&lt;/code&gt;, where edge numbers serve as keys, and quadrilateral element numbers as values.&lt;/li&gt;&#xA;&lt;li&gt;Traverse all of your edges and fill the &lt;code&gt;Edge2Quad&lt;/code&gt; multimap. &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(N_\text{edges})$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Now, all interior edges will happen to belong to two quads, while the boundary ones will belong to exactly one.&lt;/li&gt;&#xA;&lt;li&gt;By traversing your multimap, you will be able to mark the nodes as interior or boundary. &lt;span class=&quot;math-container&quot;&gt;$O(N_\text{edges})$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Perform node reordering. &lt;span class=&quot;math-container&quot;&gt;$O(N_\text{nodes})$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Of course, you can simplify this by using only arrays and just counting up.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2019-01-31T17:17:16.187" LastActivityDate="2019-01-31T17:17:16.187" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30968" PostTypeId="2" ParentId="30964" CreationDate="2019-01-31T18:56:39.827" Score="5" Body="&lt;p&gt;I believe that for odd polynomial degrees you can get superconvergence by one order at the jump. For even degrees I don't think that's the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the odd case, we consider a smooth function &lt;span class=&quot;math-container&quot;&gt;$u:\mathbb{R}\to\mathbb{R}$&lt;/span&gt;. Consider two adjacent intervals &lt;span class=&quot;math-container&quot;&gt;$I^-=[x_{i-1},x_i]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$I^+=[x_i,x_{i+1}]$&lt;/span&gt; both of length &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, and denote the &lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt; projection of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; on each interval by &lt;span class=&quot;math-container&quot;&gt;$u_h^-$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u_h^+$&lt;/span&gt; respectively. We write these projections as truncated Legendre expansions of degree &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_h^\pm(x) = \sum_{j=0}^k c_j^\pm P_j(\xi_\pm(x)),&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\xi_\pm$&lt;/span&gt; simply maps from &lt;span class=&quot;math-container&quot;&gt;$I^\pm$&lt;/span&gt; to the reference interval &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt;. Note that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u(x) = u_h^\pm(x) + \sum_{j=k+1}^\infty c_j^\pm P_j(\xi_\pm(x))&#xA;$$&lt;/span&gt;&#xA;and the coefficients &lt;span class=&quot;math-container&quot;&gt;$c_j^\pm$&lt;/span&gt; scale like &lt;span class=&quot;math-container&quot;&gt;$h^j$&lt;/span&gt;. So, the jump term is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_h^+(x_i) - u_h^-(x_i) = \sum_{j=k+1}^\infty \left(c_j^-P_j(1) - c_j^+ P_j(-1) \right) \\&#xA;= \left(c_{k+1}^- P_{k+1}(1) - c_{k+1}^+ P_{k+1}(-1)\right) + \mathcal{O}(h^{k+2}).&#xA;$$&lt;/span&gt;&#xA;Note that since &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is odd, &lt;span class=&quot;math-container&quot;&gt;$P_{k+1}(\pm 1) = 1$&lt;/span&gt;. So the jump is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_h^+(x_i) - u_h^-(x_i) = (c_{k+1}^- - c_{k+1}^+) + \mathcal{O}(h^{k+2}).&#xA;$$&lt;/span&gt;&#xA;By &lt;a href=&quot;https://en.wikipedia.org/wiki/Rodrigues%27_formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rodrigues' formula&lt;/a&gt;, we have that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;c_{k+1}^\pm = C \int_{-1}^1 \left(\frac{\partial^{k+1}}{\partial x^{k+1}} u(\xi_\pm^{-1}(x)) \right) (1 - x^2)^{k+1} \, dx \\&#xA;= C h^{k+1} \int_{-1}^1  u^{(k+1)}(\xi_\pm^{-1}(x)) (1 - x^2)^{k+1} \, dx.&#xA;$$&lt;/span&gt;&#xA;Thus, the difference between the coefficients in neighboring cells is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;c_{k+1}^- - c_{k+1}^+ = C h^{k+1}\int_{-1}^1  \left(u^{(k+1)}(\xi_-^{-1}(x)) - u^{(k+1)}(\xi_+^{-1}(x)) \right) (1 - x^2)^{k+1} \, dx.&#xA;$$&lt;/span&gt;&#xA;But, &lt;span class=&quot;math-container&quot;&gt;$\xi_+^{-1}(x) = \xi_-^{-1}(x) + h$&lt;/span&gt;, and so a Taylor argument gives that the integrand scales like &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(h)$&lt;/span&gt;. Thus &lt;span class=&quot;math-container&quot;&gt;$c_{k+1}^- - c_{k+1}^+ = \mathcal{O}(h^{k+2})$&lt;/span&gt; and we have for the jump&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_h^+(x_i) - u_h^-(x_i) = \mathcal{O}(h^{k+2}).&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="25388" LastEditorUserId="393" LastEditDate="2019-02-06T16:03:23.980" LastActivityDate="2019-02-06T16:03:23.980" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30969" PostTypeId="2" ParentId="3159" CreationDate="2019-01-31T19:04:14.367" Score="2" Body="&lt;p&gt;I also write a benchmark. For matrix of small size (&amp;lt;100*100), the performance is similar for vector&amp;lt; vector&amp;lt; double&gt;&gt; and wrapped 1D vector. For matrix of large size (~1000*1000), wrapped 1D vector is better. The Eigen matrix behaves worse. It is surprise to me that the Eigen is the worst.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;iomanip&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;sstream&amp;gt;&#xA;#include &amp;lt;algorithm&amp;gt;&#xA;#include &amp;lt;map&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;#include &amp;lt;cmath&amp;gt;&#xA;#include &amp;lt;numeric&amp;gt;&#xA;#include &quot;time.h&quot;&#xA;#include &amp;lt;chrono&amp;gt;&#xA;#include &amp;lt;cstdlib&amp;gt;&#xA;#include &amp;lt;Eigen/Dense&amp;gt;&#xA;&#xA;using namespace std;&#xA;using namespace std::chrono;    // namespace for recording running time&#xA;using namespace Eigen;&#xA;&#xA;int main()&#xA;{&#xA;    const int row = 1000;&#xA;    const int col = row;&#xA;    const int N = 1e8;&#xA;&#xA;    // 2D vector&#xA;    auto start = high_resolution_clock::now();&#xA;    vector&amp;lt;vector&amp;lt;double&amp;gt;&amp;gt; vec_2D(row,vector&amp;lt;double&amp;gt;(col,0.));&#xA;    for (int i = 0; i &amp;lt; N; i++)&#xA;    {&#xA;        for (int i=0; i&amp;lt;row; i++)&#xA;        {&#xA;            for (int j=0; j&amp;lt;col; j++)&#xA;            {&#xA;                vec_2D[i][j] *= vec_2D[i][j];&#xA;            }&#xA;        }&#xA;    }&#xA;    auto stop = high_resolution_clock::now();&#xA;    auto duration = duration_cast&amp;lt;microseconds&amp;gt;(stop - start);&#xA;    cout &amp;lt;&amp;lt; &quot;2D vector: &quot; &amp;lt;&amp;lt; duration.count()/1e6 &amp;lt;&amp;lt; &quot; s&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;    // 2D array&#xA;    start = high_resolution_clock::now();&#xA;    double array_2D[row][col];&#xA;    for (int i = 0; i &amp;lt; N; i++)&#xA;    {&#xA;        for (int i=0; i&amp;lt;row; i++)&#xA;        {&#xA;            for (int j=0; j&amp;lt;col; j++)&#xA;            {&#xA;                array_2D[i][j] *= array_2D[i][j];&#xA;            }&#xA;        }&#xA;    }&#xA;    stop = high_resolution_clock::now();&#xA;    duration = duration_cast&amp;lt;microseconds&amp;gt;(stop - start);&#xA;    cout &amp;lt;&amp;lt; &quot;2D array: &quot; &amp;lt;&amp;lt; duration.count() / 1e6 &amp;lt;&amp;lt; &quot; s&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;    // wrapped 1D vector&#xA;    start = high_resolution_clock::now();&#xA;    vector&amp;lt;double&amp;gt; vec_1D(row*col, 0.);&#xA;    for (int i = 0; i &amp;lt; N; i++)&#xA;    {&#xA;        for (int i=0; i&amp;lt;row; i++)&#xA;        {&#xA;            for (int j=0; j&amp;lt;col; j++)&#xA;            {&#xA;                vec_1D[i*col+j] *= vec_1D[i*col+j];&#xA;            }&#xA;        }&#xA;    }&#xA;    stop = high_resolution_clock::now();&#xA;    duration = duration_cast&amp;lt;microseconds&amp;gt;(stop - start);&#xA;    cout &amp;lt;&amp;lt; &quot;1D vector: &quot; &amp;lt;&amp;lt; duration.count() / 1e6 &amp;lt;&amp;lt; &quot; s&quot; &amp;lt;&amp;lt; endl;&#xA;&#xA;    // eigen 2D matrix&#xA;    start = high_resolution_clock::now();&#xA;    MatrixXd mat(row, col);&#xA;    for (int i = 0; i &amp;lt; N; i++)&#xA;    {&#xA;        for (int j=0; j&amp;lt;col; j++)&#xA;        {&#xA;            for (int i=0; i&amp;lt;row; i++)&#xA;            {&#xA;                mat(i,j) *= mat(i,j);&#xA;            }&#xA;        }&#xA;    }&#xA;    stop = high_resolution_clock::now();&#xA;    duration = duration_cast&amp;lt;microseconds&amp;gt;(stop - start);&#xA;    cout &amp;lt;&amp;lt; &quot;2D eigen matrix: &quot; &amp;lt;&amp;lt; duration.count() / 1e6 &amp;lt;&amp;lt; &quot; s&quot; &amp;lt;&amp;lt; endl;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20711" LastActivityDate="2019-01-31T19:04:14.367" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30971" PostTypeId="1" AcceptedAnswerId="31007" CreationDate="2019-01-31T20:38:11.097" Score="2" ViewCount="403" Body="&lt;p&gt;The Fokker-Planck equation (a.k.a Kolmogorov forward equation or Smoluchowski equation) describes the evolution of a probability density function and numerical integration of the FPE should conserve probability and ensure probabilites remain non-negative. The Chang-Cooper method is a popular finite-difference scheme which satisfies these constraints, however it requires that in the FPE of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial u}{\partial t} = \frac{1}{A(x)}\frac{\partial}{\partial x} \big[B(x,t)u + C(x,t)\frac{\partial u}{\partial x}\big]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the functions &lt;span class=&quot;math-container&quot;&gt;$A(x), B(x,t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C(x,t)$&lt;/span&gt; are positive. For some problems, however, such as the Fokker-Planck equation describing Brownian motion of a particle inside a potential, &lt;span class=&quot;math-container&quot;&gt;$B(x,t)$&lt;/span&gt; may not be a positive function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a method that exists for integrating the FPE that allows for a negative &lt;span class=&quot;math-container&quot;&gt;$B(x,t)$&lt;/span&gt; term? &lt;/p&gt;&#xA;" OwnerUserId="29849" LastActivityDate="2019-02-06T14:43:58.153" Title="Numerical integration of Fokker-Planck equation allowing for negative drift?" Tags="|finite-difference|reference-request|probability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30972" PostTypeId="1" AcceptedAnswerId="30986" CreationDate="2019-01-31T23:35:48.180" Score="1" ViewCount="69" Body="&lt;p&gt;I implemented a simple simulation of a comet flying through space being deflected by the gravity of randomly generated planets for art purposes. My problem with that simulation is, that there are many &quot;boring&quot; simulations in which the comet instantly collides with a planet or is deflected to leave the viewport without an interesting trajectory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Is there an efficient way to compute the initial values (gravitiy of the planets and initial velocity of the comet) for a simulation, in which the comet does not collide (early) with planets and possibly has an interesting trajectory (being deflected more often, slingshots, etc.)?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I want to use it for art, there is no problem if the algorithm has constraints like only working for integer valued planet gravity or similar restrictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current approach is using brute-force by simulating sets of parameters until I get an acceptable trajectory, which is rather slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I probably need (please comment to suggest others):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Points the comet should not cross (i.e planets)&lt;/li&gt;&#xA;&lt;li&gt;Points with a specified velocity (e.g. near planets to deflect the trajectory from a collision course)&lt;/li&gt;&#xA;&lt;li&gt;Maybe way points to guide the trajectory&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The parameters should not be arbitrary (the comet does not have a motor), but boundary conditions for choosing planet gravity and the initial velocity of the comet. If it's useful, the comet's mass can be chosen as well. Currently I assume the comet to be massless, which probably approximates the reality quite good.&lt;/p&gt;&#xA;" OwnerUserId="14700" LastEditorUserId="14700" LastEditDate="2019-02-03T09:51:18.837" LastActivityDate="2019-02-03T10:00:23.063" Title="How to choose the parameters for a collision free comet trajectory" Tags="|simulation|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="30973" PostTypeId="2" ParentId="30290" CreationDate="2019-02-01T09:49:14.063" Score="7" Body="&lt;p&gt;The book &amp;quot;&lt;a href=&quot;http://vmls-book.stanford.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Applied Linear Algebra&lt;/a&gt;&amp;quot; by Boyd and Vandenberghe has an appendix about complexity of basic operations in linear algebra and they call this case&lt;/p&gt;&#xA;&lt;h3 id=&quot;big-times-small-squared-complexity-q8np&quot;&gt;big-times-small-squared complexity.&lt;/h3&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xiRGf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xiRGf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="302" LastEditDate="2021-08-20T19:42:16.210" LastActivityDate="2021-08-20T19:42:16.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30974" PostTypeId="1" AcceptedAnswerId="30985" CreationDate="2019-02-01T10:30:29.840" Score="0" ViewCount="558" Body="&lt;p&gt;Thank you for looking at this problem. Is this the correct approach to include neumann boundary conditions? With this solution temperature is not correct, and there´s no diffusion. The model seems unbalanced. &lt;strong&gt;However if the area is included in km2 instead of m2, temperature is ok and there is diffusion...&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I´m trying to aplly the following 1D heat diffusion equation to a lake:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{∂T}{∂t}= \frac{1}{A(z)}\  \frac{∂}{∂z}\ ((A(z)K(z)\frac{∂T}{∂z}) $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where: &#xA;k(z)= Eddy diffusion coefficient, 1.43x10&lt;sup&gt;-3&lt;/sup&gt; (m&lt;sup&gt;2&lt;/sup&gt;/s),&#xA;A(z)= lake area at depth z, m&lt;sup&gt;2 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The surface boundary condition is the following:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ ρcK(z,t)\frac{∂T}{∂z}= -Q $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where: &#xA;ρ= water density, 1000 kg/m&lt;sup&gt;3&lt;/sup&gt;&#xA;c= water specific heat, 4186 J/kgºC&#xA;Q= radiation, w/m&lt;sup&gt;2&lt;/sup&gt; (or J.s&lt;sup&gt;-1&lt;/sup&gt;.m-2&lt;/sup&gt;).&#xA;&lt;strong&gt;The lake bottom condition is the following:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{∂T}{∂z}= 0 $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I´m considering the Crank-Nicolson scheme with centered differences in space and time and I have used Thomas Algorithm to solve the system of equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation discretization is included in the picture below. But to simplify the analysis I´m including here the equations for each coefficient (f = k(z); n is time and i is space):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ α_{i}=\frac{1}{A_{i,n+1}+A_{i,n}} $$&lt;/span&gt; &#xA;  &lt;span class=&quot;math-container&quot;&gt;$$ s=\frac{∆t}{2(∆z)^{2}} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a_{i} = −α_{i}s(f_{i−1,n+1}+ f_{i,n+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$b_{i} = 1+α_{i}s(f_{i−1,n+1}+ 2f_{i,n+1}+f_{i+1,n+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$c_{i} = −α_{i}s(f_{i,n+1}+ f_{i+1,n+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$d_{i,n} = α_{i}s(f_{i,n}+ f_{i-1,n})T_{i−1,n}+α_{i}s(f_{i+1,n}+ f_{i,n})T_{i+1,n}+[1−α_{i}s(f_{i−1,n}+ 2f_{i,n}+f_{i+1,n})]T_{i,n}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;for the surface boundary:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$b_{0}=1+α_{0}s(f_{−1,n+1}+ 2f_{0,n+1}+f_{1,n+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$c_{0}=−α_{0}s(f_{-1,n+1}+ 2f_{0,n+1}+f_{1,n+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$d_{0,n}=[1-α_{0}s(f_{-1,n}+ f_{0,n}+f_{1,n}]T_{0,n}+α_{0}s(f_{-1,n}+2 f_{0,n}+f_{1,n})T_{1,n}+α_{0}s(f_{−1,n}+ 2f_{0,n})\frac{2Q_{n}∆z}{ρcK_{0,n}}+α_{0}s(f_{-1,n+1}+ f_{0,n+1})\frac{2Q_{n+1}∆z}{ρcK_{0,n+1}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;for the bottom boundary:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a_{NZ} = −α_{NZ}s(f_{NZ−1,n+1}+ 2f_{NZ,n+1}+f_{NZ+1,n+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$b_{NZ} = 1+α_{NZ}s(f_{NZ−1,n+1}+ 2f_{NZ,n+1}+f_{NZ+1,n+1})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$d_{NZ,n} = α_{NZ}sT_{NZ−1,n}(f_{NZ-1,n}+ 2f_{NZ,n}+f_{NZ+1,n})+[1−α_{NZ}s(f_{NZ−1,n}+ 2f_{NZ,n}+f_{NZ+1,n})]T_{NZ,n}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also include a simplification of the &lt;strong&gt;Python code&lt;/strong&gt;. I have removed all the loops and I have considered a constante value for eddy diffusivity K(z).&#xA;It´s a daily simulation (N=10) for a lake with 6 layers. Each layer is 2m deep (z=2m). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;np.set_printoptions(threshold=np.inf)&#xA;import pylab as plt&#xA;&#xA;&#xA;&#xA;#------------------------------------------------------------------------------------------------------------------------------------&#xA;# Tri Diagonal Matrix Algorithm (Thomas algorithm) solver (source:https://gist.github.com/cbellei/8ab3ab8551b8dfc8b081c518ccd9ada9)&#xA;#-------------------------------------------------------------------------------------------------------------------------------------&#xA;&#xA;def TDMAsolver(a, b, c, d):&#xA;&#xA;    nf = len(d)&#xA;    ac, bc, cc, dc = map(np.array, (a, b, c, d))&#xA;    for it in xrange(1, nf):&#xA;        mc = ac[it-1]/bc[it-1]&#xA;        bc[it] = bc[it] - mc*cc[it-1] &#xA;        dc[it] = dc[it] - mc*dc[it-1]&#xA;&#xA;    xc = bc&#xA;    xc[-1] = dc[-1]/bc[-1]&#xA;&#xA;    for il in xrange(nf-2, -1, -1):&#xA;        xc[il] = (dc[il]-cc[il]*xc[il+1])/bc[il]&#xA;&#xA;    del bc, cc, dc&#xA;&#xA;    return xc&#xA;&#xA;#-------------------------------------------------------------------------------------------------------------------------------------&#xA;#Crank Nicolson scheme&#xA;#-------------------------------------------------------------------------------------------------------------------------------------&#xA;#Simulation time period (10 days)&#xA;N=10&#xA;&#xA;Nlayers=6&#xA;&#xA;#Space, m&#xA;DZ=2.0&#xA;&#xA;#Radiation, J.s-1.m-2 (or W/m2)&#xA;Q=300.0&#xA;&#xA;#Eddy diffusion coefficient at 25 C (m2/s)&#xA;K=1.43*10**-3&#xA;&#xA;#Water temperature array&#xA;TempLake=np.zeros((N+1,Nlayers))&#xA;&#xA;# Initial condition, C&#xA;TempLake[0]=[25,25,25,25,25,25]&#xA;&#xA;#Lake surface area, m2&#xA;A1=23992000.0 #m2&#xA;A2=23992000.0 #m2&#xA;&#xA;alfa=1.0/(A1+A2)&#xA;&#xA;#time interval, seconds, s&#xA;DT=86400&#xA;#Water density, kg/m3&#xA;DH2O=1000.0&#xA;#Water specific Heat j/kg C&#xA;Cp=4186&#xA;&#xA;s=DT/(2*DZ**2.0)&#xA;&#xA;#Initializing coefficients np.arrays&#xA;a=np.zeros(Nlayers-1)&#xA;b=np.zeros(Nlayers)&#xA;c=np.zeros(Nlayers-1)&#xA;d=np.zeros(Nlayers)&#xA;&#xA;for i in xrange(1,N+1):  # 1,2,3,4,5,6,7,8,9,10        &#xA;    #Surface&#xA;    #b0, c0, d0&#xA;    b[0]=1+alfa*s*(K+2.0*K+K)&#xA;    c[0]=-alfa*s*(K+2.0*K+K)&#xA;    d[0]=((1.0-alfa*s*(K+2.0*K+K))*TempLake[i-1][0])+(alfa*s*(K+2.0*K+K)*TempLake[i-1][1])+alfa*s*(K+K)*((2*Q*DZ)/(Cp*DH2O*K))+alfa*s*(K+K)*((2.0*Q*DZ)/(DH2O*Cp*K)) # 0,1,2,3,4,5,6,7,8,9  &#xA;&#xA;    #Bottom&#xA;    #an, bn, dn&#xA;    a[-1]=-alfa*s*(K+2.0*K+K)&#xA;    b[-1]=1+alfa*s*(K+2.0*K+K)&#xA;    d[-1]=alfa*s*TempLake[i-1][-2]*(K+2.0*K+K)+(1-alfa*s*(K+2*K+K))*TempLake[i-1][-1] # 0,1,2,3,4,5,6,7,8,9 &#xA;&#xA;    #ai, bi, ci, di&#xA;    a[0]=-alfa*s*(K+K)&#xA;    a[1]=-alfa*s*(K+K)&#xA;    a[2]=-alfa*s*(K+K)&#xA;    a[3]=-alfa*s*(K+K)&#xA;&#xA;    b[1]=1.0+alfa*s*(K+2.0*K+K)&#xA;    b[2]=1.0+alfa*s*(K+2.0*K+K)  &#xA;    b[3]=1.0+alfa*s*(K+2.0*K+K)&#xA;    b[4]=1.0+alfa*s*(K+2.0*K+K)&#xA;&#xA;    c[1]=-alfa*s*(K+K)&#xA;    c[2]=-alfa*s*(K+K)&#xA;    c[3]=-alfa*s*(K+K)&#xA;    c[4]=-alfa*s*(K+K)&#xA;&#xA;    d[1]=alfa*s*(K+K)*TempLake[i-1][0]+alfa*s*(K+K)*TempLake[i-1][2]+(1-alfa*s*(K+2.0*K+K))*TempLake[i-1][1]&#xA;    d[2]=alfa*s*(K+K)*TempLake[i-1][1]+alfa*s*(K+K)*TempLake[i-1][3]+(1-alfa*s*(K+2.0*K+K))*TempLake[i-1][2]&#xA;    d[3]=alfa*s*(K+K)*TempLake[i-1][2]+alfa*s*(K+K)*TempLake[i-1][4]+(1-alfa*s*(K+2.0*K+K))*TempLake[i-1][3]&#xA;    d[4]=alfa*s*(K+K)*TempLake[i-1][3]+alfa*s*(K+K)*TempLake[i-1][5]+(1-alfa*s*(K+2.0*K+K))*TempLake[i-1][4]&#xA;&#xA;    TempTDMA=TDMAsolver(a, b, c, d)&#xA;&#xA;    TempLake[i]=TempTDMA&#xA;&#xA;print TempLake&#xA;&#xA;Z=TempLake &#xA;&#xA;X,Y=np.meshgrid(range(Z.shape[0]+1),range(Z.shape[1]+1)) &#xA;&#xA;im = plt.pcolormesh(X,Y,Z.transpose(), cmap='jet') &#xA;ax = plt.gca()&#xA;ax.set_ylim(ax.get_ylim()[::-1])&#xA;plt.colorbar(im, orientation='horizontal') &#xA;&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WJ8Wi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WJ8Wi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TEST 1 : simulation code included above&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TEST 2 : simulation equal to Test 1 but lake area is included in km2 instead of m2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uJrEE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uJrEE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uJrEE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uJrEE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29853" LastEditorUserId="29853" LastEditDate="2019-02-04T17:14:51.123" LastActivityDate="2019-02-04T17:14:51.123" Title="Heat diffusion - Is this the correct approach to include Newmann boundary conditions?" Tags="|python|fluid-dynamics|boundary-conditions|heat-transfer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30975" PostTypeId="1" AcceptedAnswerId="30978" CreationDate="2019-02-01T12:35:21.587" Score="0" ViewCount="125" Body="&lt;p&gt;I am currently proof-checking my program, which is intended to use Newton's method for solving nonlinear equations, using a continuous galerkin approach. Thus, as first step I checked it using a time-independent equation (&lt;a href=&quot;https://scicomp.stackexchange.com/q/30901/20545&quot;&gt;$H^1$-convergence rate of finite element method for Poisson equation, depending on element order&lt;/a&gt;), with positive results. As next step I intended to check the time-dependent heat equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\partial_tu=\nabla^2u\text{, }u_0=\sin(\pi x)\sin(\pi y)$$&lt;/span&gt;&#xA;on the domain&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Omega=[0,1]\times[0,1]$$&lt;/span&gt;&#xA;and the boundary conditions&#xA;&lt;span class=&quot;math-container&quot;&gt;$$u=0\text{ on }\partial\Omega$$&lt;/span&gt;&#xA;As time discretization I use&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{u_{new}-u_{old}}{dt}=\nabla^2u_{new}$$&lt;/span&gt;&#xA;This should give me &#xA;&lt;span class=&quot;math-container&quot;&gt;$$u(x,y,t)=\exp(-2\pi^2t)\sin(\pi x)\sin(\pi y)$$&lt;/span&gt;&#xA;The jacobian then is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$F'(u)(\delta u)=\partial_t\delta u-\nabla^2\delta u$$&lt;/span&gt;&#xA;or in the weak form (while considering the boundary conditions)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$=\partial_t\delta u\varphi+\nabla u\nabla\varphi$$&lt;/span&gt;&#xA;The optimal step size should be 1, after the problem is linear.&lt;br&gt;&#xA;I then compare the result for each time step with the expected results, on the one hand by calculating the &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt;-seminorm, and on the other hand the highest value in the solution (which should be equivalent to &lt;span class=&quot;math-container&quot;&gt;$\exp(-2\pi^2t)$&lt;/span&gt;). Still, my results are not as expected:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+-------+-------------------+-----------------+&#xA;| time  | Calculated result | Expected result |&#xA;+-------+-------------------+-----------------+&#xA;|     0 | 0.336254          | 1               |&#xA;|   0.1 | 0.113069          | 0.138911        |&#xA;|   0.2 | 0.0380201         | 0.0192963       |&#xA;|   0.3 | 0.0127845         | 0.00268047      |&#xA;|   0.4 | 0.00429886        | 0.000372347     |&#xA;|   0.5 | 0.00144552        | 5.17232e-05     |&#xA;|   0.6 | 0.000486064       | 7.18493e-06     |&#xA;|   0.7 | 0.000163442       | 9.98066e-07     |&#xA;|   0.8 | 5.49583e-05       | 1.38643e-07     |&#xA;|   0.9 | 1.84801e-05       | 1.9259e-08      |&#xA;|     1 | 6.21403e-06       | 2.67529e-09     |&#xA;+-------+-------------------+-----------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that the program works for &lt;span class=&quot;math-container&quot;&gt;$$\partial_t u=f$$&lt;/span&gt; but currently I am out of ideas about how to check why my calculated results are not the same as I would expect them. I checked the Jacobian (assumed my calculated equation above is correct), so I assume the bug is somewhere else.&lt;/p&gt;&#xA;" OwnerUserId="20545" LastEditorUserId="20545" LastEditDate="2019-02-01T12:53:41.953" LastActivityDate="2019-02-01T21:04:46.693" Title="Debugging Newton-method used in a CG-approach" Tags="|newton-method|jacobian|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30977" PostTypeId="2" ParentId="30961" CreationDate="2019-02-01T18:41:40.813" Score="1" Body="&lt;p&gt;How are you on a uniform unstructured grid? Are you in 1-D or 2-D? You're missing a lot of detail. This expression of the norm that you found is area weighted. If you're 1D then multiplying by &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; makes sense, if you're 2D  then it should be squared, etc, assuming you want to be area weighted. The dimension of the array doesn't matter, its the dimension of the domain that matters.&lt;/p&gt;&#xA;" OwnerUserId="9446" LastActivityDate="2019-02-01T18:41:40.813" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30978" PostTypeId="2" ParentId="30975" CreationDate="2019-02-01T21:04:46.693" Score="2" Body="&lt;p&gt;You're using a backward Euler finite difference time stepping method. This is stable, but only first order accurate, so I suspect explains the discrepancy in the reduction factor. More explicitly, assuming a perfect spatial representation and substituting your initial condition into your difference equation gives &lt;span class=&quot;math-container&quot;&gt;$$(1+2\pi^2\Delta t)u^{n+1} = u^n.$$&lt;/span&gt;&#xA;For a timestep of 0.1, that makes each step 0.3363 of the last which (to several significant figures) is exactly what you seeing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only outstanding question is why your metric is giving 0.336254 rather than 1.0 for the initial condition. Based on how close that is to the number calculated above, I suspect this is actually the value for your &lt;em&gt;first&lt;/em&gt;, rather than &lt;em&gt;zeroth&lt;/em&gt; step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want closer results, you could try using a second order (or higher) time stepping method like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Midpoint_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;midpoint method&lt;/a&gt; (which may or may not need surgery to your code). Note that in exchange you'll be sacrificing simplicity, stability or both.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2019-02-01T21:04:46.693" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="30979" PostTypeId="1" AcceptedAnswerId="30980" CreationDate="2019-02-02T10:28:23.007" Score="0" ViewCount="215" Body="&lt;p&gt;So, I've written a short python program simulating diffusion in 2D; in essence each particle is behaving like a random walker. The output is a 3D tensor, which for 4 particles and 10 timesteps, looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[[ 0.          0.        ]&#xA;  [-0.16043999 -0.9870456 ]&#xA;  [-1.09597489 -1.34027986]&#xA;  [-2.08225903 -1.17522353]&#xA;  [-1.76925893 -0.22547044]&#xA;  [-2.01800797 -1.19403841]&#xA;  [-2.15515569 -2.18458902]&#xA;  [-1.32309938 -1.62989786]&#xA;  [-2.27406193 -1.9392039 ]&#xA;  [-1.90350731 -1.01039322]]&#xA;&#xA; [[ 0.          0.        ]&#xA;  [-0.6938323  -0.72013662]&#xA;  [-0.36299996 -1.66382617]&#xA;  [ 0.5186036  -1.19183553]&#xA;  [ 0.34714146 -0.20664482]&#xA;  [ 1.31071192 -0.47409941]&#xA;  [ 2.25711798 -0.1511202 ]&#xA;  [ 1.58434992 -0.89097365]&#xA;  [ 1.3892911   0.0898179 ]&#xA;  [ 0.94147109 -0.80430583]]&#xA;&#xA; [[ 0.          0.        ]&#xA;  [ 0.99886494  0.04763229]&#xA;  [ 1.5346788   0.89196842]&#xA;  [ 0.81263989  1.58382086]&#xA;  [-0.18631285  1.62957497]&#xA;  [ 0.75609885  1.96403003]&#xA;  [-0.06263107  1.38985122]&#xA;  [-0.1323122   0.3922819 ]&#xA;  [ 0.8338982   0.13452725]&#xA;  [ 1.05742482  1.10922508]]&#xA;&#xA; [[ 0.          0.        ]&#xA;  [ 0.04440308  0.9990137 ]&#xA;  [ 0.86574242  0.42857379]&#xA;  [ 1.16274132 -0.52630403]&#xA;  [ 1.43873896  0.43485428]&#xA;  [ 2.10741306 -0.30870133]&#xA;  [ 1.74826957  0.62458103]&#xA;  [ 1.44620452  1.57786834]&#xA;  [ 1.31735411  2.56953238]&#xA;  [ 0.73397918  1.75732939]]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, each block is 1 particle, the columns are x- and y-coordinates, and the rows display successive timesteps. Now, I would like to plug this in into VMD to visualize the particle trajectories. How should the VMD movie format look?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thank you all for your help!&lt;/p&gt;&#xA;" OwnerUserId="29843" LastActivityDate="2019-02-02T14:40:39.140" Title="Visualizing a trajectory in VMD" Tags="|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30980" PostTypeId="2" ParentId="30979" CreationDate="2019-02-02T14:40:39.140" Score="1" Body="&lt;p&gt;I'd recommend that you use the &lt;a href=&quot;https://en.wikipedia.org/wiki/XYZ_file_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;XYZ file format&lt;/a&gt;. Once you have a XYZ trajectory file, go into VMD, load the molecule by clicking File &gt; New molecule, and display the particles by clicking Graphics &gt; Representations &gt; Drawing methods, then choose Points or VDW, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a quick script to produce the desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def write_xyz(xyz: np.array, filename: str) -&amp;gt; None:&#xA;    with open(filename, 'w') as f:&#xA;        for j in range(xyz.shape[1]):&#xA;            print(f'{xyz.shape[0]}\n', file=f)&#xA;            for i in range(xyz.shape[0]):&#xA;                print(f'p{i} {xyz[i, j, 0]} {xyz[i, j, 1]} 0', file=f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15" LastActivityDate="2019-02-02T14:40:39.140" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30983" PostTypeId="2" ParentId="10479" CreationDate="2019-02-02T21:02:42.283" Score="0" Body="&lt;p&gt;I think this paper by H.L. Atkins and C.-W. Shu is exactly what you are looking for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arc.aiaa.org/doi/abs/10.2514/2.436&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quadrature-Free Implementation of Discontinuous Galerkin Method for Hyperbolic Equations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20711" LastActivityDate="2019-02-02T21:02:42.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30985" PostTypeId="2" ParentId="30974" CreationDate="2019-02-03T05:35:30.067" Score="1" Body="&lt;p&gt;The choice of the boundary conditions is dependent on the physics you want to model. If you're doing heat diffusion through a lake, and you want constant temp on top you'll have a different boundary condition than for constant heat flux out of the top (which appears to be what you want). I'm assuming that you want the bottom of the lake to have no heat flux (as that's what you wrote). as far as I can tell from your break down you have no mistakes that jump out at me, but I didn't check too closely. When you say the temperature is not correct, what do you mean? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9446" LastActivityDate="2019-02-03T05:35:30.067" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="30986" PostTypeId="2" ParentId="30972" CreationDate="2019-02-03T10:00:23.063" Score="3" Body="&lt;p&gt;You're trying to solve the &lt;a href=&quot;https://en.wikipedia.org/wiki/N-body_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;n-body problem&lt;/a&gt;, which has some known approximations that can be used to accelerate computation; however, it's still unclear how you can turn these into artistically interesting orbits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some thoughts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Place the comet near the edge of one of your planet's &lt;a href=&quot;https://en.wikipedia.org/wiki/Hill_sphere&quot; rel=&quot;nofollow noreferrer&quot;&gt;hill spheres&lt;/a&gt; with a velocity just slightly above the planet's &lt;a href=&quot;https://en.wikipedia.org/wiki/Escape_velocity&quot; rel=&quot;nofollow noreferrer&quot;&gt;escape velocity&lt;/a&gt; and, possibly, near, but under, your solar system's escape velocity. This will give you a comet with a relatively stable orbit capable of traversing, and being perturbed by, most of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could set up an &lt;a href=&quot;https://en.wikipedia.org/wiki/Hohmann_transfer_orbit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hohmann transfer orbit&lt;/a&gt; between two planets in your system, but perturb the velocity so that your comet &quot;misses&quot; the intended orbit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more complex patterns, you could figure out how to use the &lt;a href=&quot;http://gmatcentral.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;General Mission Analysis Tool&lt;/a&gt; to multi-planet assists. You could also look into the &lt;a href=&quot;https://en.wikipedia.org/wiki/Interplanetary_Transport_Network&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interplanetary Transport Network&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2019-02-03T10:00:23.063" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="30989" PostTypeId="1" CreationDate="2019-01-31T10:05:34.520" Score="11" ViewCount="1393" Body="&lt;p&gt;LAPACK's QR routine stores Q as Householder reflectors. It scales the reflection vector &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$1/v_1$&lt;/span&gt;, so the first element of the result becomes &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, so it doesn't have to be stored. And it stores a separate &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; vector, which contains the needed scale factors. So a reflector matrix is like this: &lt;span class=&quot;math-container&quot;&gt;$$H=I-\tau v v^T,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is not normalized. While, in textbooks, the reflector matrix is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$H = I-2vv^T,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is normalized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why does LAPACK scale &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$1/v_1$&lt;/span&gt;, instead of normalizing it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Needed storage is the same (instead of &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$v_1$&lt;/span&gt; has to be stored), and afterwards, applying &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; can be done faster, as there is no need to multiply with &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; (multiplication with &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; in the textbook version can be optimized, if instead of simple normalization, &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is scaled by &lt;span class=&quot;math-container&quot;&gt;$\sqrt 2/\|v\|$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The reason of my question is that I'm writing an QR and SVD routine, and I'd like to know the reason of this decision, whether I need to follow it or not)&lt;/p&gt;&#xA;" OwnerUserId="29873" OwnerDisplayName="geza" LastEditorUserId="29873" LastEditDate="2019-02-04T07:35:12.010" LastActivityDate="2019-02-06T11:41:30.220" Title="What is the reason that LAPACK uses $\tau$ in QR decomposition (instead of normalizing the reflection vector)?" Tags="|linear-algebra|matrix|lapack|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30990" PostTypeId="2" ParentId="30989" CreationDate="2019-02-03T06:24:20.300" Score="1" Body="&lt;p&gt;My suggestion is based on the documentation for Intel MKL &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-geqrf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.intel.com/en-us/mkl-developer-reference-c-geqrf&lt;/a&gt;. It looks like the values on and above the diagonal of the output store R so there is only lower triangle left for Q. The it seems natural to use additional storage for the scaling factors.&lt;/p&gt;&#xA;" OwnerUserId="21958" OwnerDisplayName="VorKir" LastActivityDate="2019-02-03T06:24:20.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30991" PostTypeId="2" ParentId="3159" CreationDate="2019-02-03T17:03:11.723" Score="5" Body="&lt;p&gt;I don't recommend it, but not because of performance issues. It will be a little less performant than a traditional matrix, which are usually allocated as a big chunk of contiguous data that is indexed using a single pointer dereference and integer arithmetic . The reason for the performance hit is mostly caching differences, but once your matrix size gets big enough this effect will be amortized and if you use a special allocator for the inner vectors so that they are aligned to cache boundaries then this further mitigates the caching issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That by itself isn't reason enough not to do it, in my opinion. The reason for me is that it creates a lot of coding headaches. Here's a list of headaches this will cause long-term&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Use of HPC libraries&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you want to use most HPC libraries you'll need to iterate over your vector and place all their data in a contiguous buffer, because most HPC libraries expect this explicit format. BLAS and LAPACK come to mind, but also the ubiquitous HPC library MPI would be much harder to use.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;More potential for coding error&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;std::vector&lt;/code&gt; doesn't know anything about its entries. If you fill a &lt;code&gt;std::vector&lt;/code&gt; with more &lt;code&gt;std::vector&lt;/code&gt;s then it's entirely your job to make sure that they all have the same size, because remember that we want a matrix and matrices don't have variable number of rows (or columns). Thus you'll have to call all the correct constructors for every entry of your outer vector, and anybody else who uses your code must resist the temptation to use &lt;code&gt;std::vector&amp;lt;T&amp;gt;::push_back()&lt;/code&gt; on any of the inner vectors, which would cause all following code to break. Of course you can disallow this if you write your class correctly, but it's much easier to enforce this simply with a big contiguous allocation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;HPC culture and expectations&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;HPC programmers simply expect low level data. If you give them a matrix there is an expectation that if they grabbed the pointer to the first element of the matrix and a pointer to the last element of the matrix, then all pointers in between these two are valid and point to elements of that same matrix. This is similar to my first point, but different because it may not be related so much to libraries but rather team members or anyone you share your code with.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Easier to reason about performance of lower level data&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Dropping to the lowest level representation of your desired data structure makes your life easier in the long run for HPC. Using tools like &lt;code&gt;perf&lt;/code&gt; and &lt;code&gt;vtune&lt;/code&gt; will give you very low level performance counter measurements which you will try to combine with traditional profiling results in order to improve the performance of your code. If your data structure uses a lot of fancy containers it will be hard to understand that cache misses are coming from a problem with the container or an inefficiency in the algorithm itself. For more complicated code containers are necessary, but for matrix algebra they really aren't - you can get by with just &lt;code&gt;1&lt;/code&gt; &lt;code&gt;std::vector&lt;/code&gt; to store the data rather than &lt;code&gt;n&lt;/code&gt; &lt;code&gt;std::vector&lt;/code&gt;s, so go with that.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2019-02-03T17:03:11.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30992" PostTypeId="2" ParentId="27490" CreationDate="2019-02-03T18:48:33.617" Score="4" Body="&lt;p&gt;It turns out that &lt;code&gt;scipy&lt;/code&gt; does indeed support this type of overloading. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One simply needs to write a class inheriting from &lt;code&gt;scipy.sparse.linalg.LinearOperator&lt;/code&gt; and implement the &lt;code&gt;matvec&lt;/code&gt; method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing so allows use of all the &lt;code&gt;scipy.sparse&lt;/code&gt; linear solvers, but with a matrix vector routine that can be implemented as the user desires (in my case, which applies each contribution independently and takes advantage of symmetry).&lt;/p&gt;&#xA;" OwnerUserId="20148" LastActivityDate="2019-02-03T18:48:33.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30994" PostTypeId="2" ParentId="25203" CreationDate="2019-02-04T13:02:04.700" Score="9" Body="&lt;p&gt;HDF5 is, to some extent, a filesystem on its own. By introducing B-Trees and by the way it manages blocks, it duplicates the functionality of a filesystem. When you are running your code, you are probably running it on an operating system with a proven and scalable filesystem. Hence, I would suggest to write your numerical raw data into a single file using raw file access or MPI-IO and write the meta-data (endianess, size, attributes, etc.) into a separate JSON or XML file. If you have multiple datasets you can organize them into a directory or a hierarchy of directories. When you want to distribute the dataset, you just have to pack it into a ZIP file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only downside is that you have to deal with &lt;a href=&quot;https://en.wikipedia.org/wiki/Endianness&quot; rel=&quot;noreferrer&quot;&gt;Endianness&lt;/a&gt; yourself, which is, however, &lt;a href=&quot;https://stackoverflow.com/questions/8978935/detecting-endianness&quot;&gt;not hard&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an inspiration on how this can be done see &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fninf.2018.00016/full&quot; rel=&quot;noreferrer&quot;&gt;Dragly, et. al. &quot;A. Experimental Directory Structure (Exdir): An Alternative to HDF5 Without Introducing a New File Format&quot; Front. Neuroinform., 2018, 12&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastEditorUserId="414" LastEditDate="2019-02-04T13:07:26.647" LastActivityDate="2019-02-04T13:07:26.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30995" PostTypeId="1" AcceptedAnswerId="30997" CreationDate="2019-02-04T13:34:17.147" Score="0" ViewCount="105" Body="&lt;p&gt;I'm trying compute the heat capacity &lt;span class=&quot;math-container&quot;&gt;$C_v$&lt;/span&gt; out of my simulation for the 2D-Ising model which is given by &#xA;&lt;span class=&quot;math-container&quot;&gt;$C_v = \frac{\langle E^2 \rangle - \langle E \rangle^2}{T^2N^2}$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt;: Energy, &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;: Temperature, &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;: Number of points).&#xA;I implemented the Ising simulation using the Metropolis-method which seems to be the easiest way to simulate this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ising[n_, count_, B_, J_, T_] := Module[{}, &#xA;e1 = 0; e2 = 0; &#xA;dE[a_, b_]= a*2*(B - b*J)/T;&#xA;Lat = 2*Table[Random[Integer], {n}, {n}] - 1;&#xA;&#xA;flip =&#xA;  (&#xA; L = #;&#xA; {i1, i2} = {RandomInteger[{1, n}], RandomInteger[{1, n}]};&#xA;&#xA; If[i1 == n, down = 1, down = i1 + 1];&#xA; If[i2 == n, rechts = 1, rechts = i2 + 1];&#xA; If[i1 == 1, up = n, up = i1 - 1];&#xA; If[i2 == 1, links = n, links = i2 - 1];&#xA; Es = L[[down, i2]] + L[[up, i2]] + L[[i1, rechts]] + L[[i1, links]];&#xA; If[ dE[ L[[i1, i2]], Es ] &amp;lt; 0 || &#xA;   RandomReal[] &amp;lt; Exp[-dE[ L[[i1, i2]], Es ]], &#xA;  L[[i1, i2]] = -L[[i1, i2]]; e1 = e1 + dE[ L[[i1, i2]], Es ]; &#xA;  e2 = e2 + dE[ L[[i1, i2]], Es ]*dE[ L[[i1, i2]], Es ], L; &#xA;  e1 = e1; e2 = e2];&#xA; L&#xA; ) &amp;amp;;&#xA;&#xA;&#xA;fliplist = NestList[flip, Lat, count];&#xA;  ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This simulation is working for the plots of the lattice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as I try computing the terms &lt;span class=&quot;math-container&quot;&gt;$\langle E \rangle$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\langle E \rangle^2$&lt;/span&gt; and using them to compute &lt;span class=&quot;math-container&quot;&gt;$C_v$&lt;/span&gt; the result is not what I am expecting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For large numbers of iterations (in this case equivalent to time units &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;) the mean value of the energy can be computed by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\langle E \rangle = \frac{1}{count} \sum_{t=0}^{count} E_t$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sum over &lt;span class=&quot;math-container&quot;&gt;$E_t$&lt;/span&gt; is what i thought i was doing with the e1 and e2 in the code. &#xA;As i repeat the simulation for several temperatures and then use the values to calculate &lt;span class=&quot;math-container&quot;&gt;$C_v$&lt;/span&gt; the result is not the heat capacity.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Do[steps = 50000;&#xA;  Ising[64, steps, 0, -1, i]; Subscript[epts, i] = 1/steps e1; &#xA;  Subscript[esq, i] = 1/steps e2,&#xA;  {i, 1, 3, 0.1}];&#xA;&#xA;Do[Subscript[c, i] = &#xA;  1/(2*i*i) (Subscript[esq, &#xA;      i] - (Subscript[epts, i])*(Subscript[epts, i])), {i, 1, 3, 0.1}];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The plot of those points looks like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9TfsC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9TfsC.png&quot; alt=&quot;C_v vs Temperature&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not the expected plot for &lt;span class=&quot;math-container&quot;&gt;$C_v$&lt;/span&gt;. There should be a peak at around 2.3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anybody help me to calculate the real &lt;span class=&quot;math-container&quot;&gt;$C_v$&lt;/span&gt; because I can't find the mistake. &#xA;I'm very thankful for any help!&lt;/p&gt;&#xA;" OwnerUserId="29881" LastActivityDate="2019-08-21T09:47:24.053" Title="ISING2D with Mathematica. Searching a correct way to compute the heat capacity (mean values over several iterations)" Tags="|computational-physics|mathematica|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="30996" PostTypeId="2" ParentId="30989" CreationDate="2019-02-04T15:08:47.167" Score="7" Body="&lt;p&gt;It's the blocked variant of Householder-QR that is driving this design. If you look in Golub and Van Loan's book (Ch 5.2 or so) they talk about how k-iterations of the algorithm can be blocked together by accumulating the individual reflectors into a rank-k reflector of the form &lt;span class=&quot;math-container&quot;&gt;$\mathbf I + \mathbf W \mathbf Y^{\mathrm T}$&lt;/span&gt;, where both &lt;span class=&quot;math-container&quot;&gt;$\mathbf W$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf Y$&lt;/span&gt; are &quot;tall-skinny&quot; matrices with size &lt;span class=&quot;math-container&quot;&gt;$n \times k$&lt;/span&gt;. This algorithm does more work but is faster in practice because it's rich in gemm() calls. Unfortunately it is wasteful in storage due to the need to represent &lt;span class=&quot;math-container&quot;&gt;$\mathbf W$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf Y$&lt;/span&gt; independently. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a later paper (cited below), Van Loan describes a more efficient &quot;symmetrized&quot; data structure, a block reflector of the form &lt;span class=&quot;math-container&quot;&gt;$\mathbf I + \mathbf Y \mathbf T \mathbf Y^{\mathrm T}$&lt;/span&gt;. Here &lt;span class=&quot;math-container&quot;&gt;$\mathbf Y$&lt;/span&gt; is still &lt;span class=&quot;math-container&quot;&gt;$n \times k$&lt;/span&gt;, but the flop/storage requirement for forming &lt;span class=&quot;math-container&quot;&gt;$\mathbf W$&lt;/span&gt; has been eliminated by introducing &lt;span class=&quot;math-container&quot;&gt;$\mathbf T$&lt;/span&gt;, a small &lt;span class=&quot;math-container&quot;&gt;$k \times k$&lt;/span&gt; upper triangular matrix. Although the need to multiply by &lt;span class=&quot;math-container&quot;&gt;$\mathbf T$&lt;/span&gt; introduces a small amount of extra work, it's typically a net gain because &lt;span class=&quot;math-container&quot;&gt;$k &amp;lt;&amp;lt; n$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within LAPACK, the non-blocked algorithm is really just a limiting &lt;span class=&quot;math-container&quot;&gt;$k \rightarrow 1$&lt;/span&gt; case of the block algorithm, all the way down to the choice of symbols (which leads us to &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt;, a little &lt;span class=&quot;math-container&quot;&gt;$1\times1$&lt;/span&gt; version of the &lt;span class=&quot;math-container&quot;&gt;$\mathbf T$&lt;/span&gt; triangle).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Citation: Schreiber, Robert, and Charles Van Loan. &quot;A storage-efficient WY representation for products of Householder transformations.&quot; SIAM Journal on Scientific and Statistical Computing 10.1 (1989): 53-57.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2019-02-05T03:09:03.790" LastActivityDate="2019-02-05T03:09:03.790" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30997" PostTypeId="2" ParentId="30995" CreationDate="2019-02-04T16:34:33.090" Score="1" Body="&lt;p&gt;Unfortunately I am not a Mathematica whiz. and in general do not encourage the use of Mathematica for extensive numerical simulations like Ising model or the likes. So I will post some answers pertaining to C++ and the general equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I am going to assume, that your &lt;span class=&quot;math-container&quot;&gt;$E_t$&lt;/span&gt; is actually the spin-exchange energy &lt;span class=&quot;math-container&quot;&gt;$-JS_iS_j$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt; are the spins and &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt; the interaction parameter.&#xA;Heat capacity &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is then computed, as you'Ve stated via&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$C=\frac{\langle E^2\rangle - \langle E\rangle^2}{k_bT^2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the following holds&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&amp;lt;E&amp;gt;=-\frac{J}{N}\sum_{&amp;lt;i,j&amp;gt;}S_iS_j$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$&amp;lt;E&amp;gt;^2 = \frac{1}{N}\sum_{i=1}^N\left(-J\sum_{j\in n.N.} S_iS_j\right)^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where in the first equation &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;i,j&amp;gt;$&lt;/span&gt; indicates summation over nearest-neighbours, what has been split in the second one into a sum over &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and for each &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; a sum over its nearest neighbours. &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the total number of sites (&lt;span class=&quot;math-container&quot;&gt;$=n^2$&lt;/span&gt; for a square grid of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; cells in each direction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I'm pretty sure you are not supposed to average over multiple iterations when you are computing the observables (also:Label your axes!).&lt;/p&gt;&#xA;" OwnerUserId="28993" LastActivityDate="2019-02-04T16:34:33.090" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="30999" PostTypeId="1" AcceptedAnswerId="31003" CreationDate="2019-02-04T18:26:41.320" Score="20" ViewCount="11201" Body="&lt;p&gt;I am developing a code to simulate fluid flow with biological substances present in the flow. This involves the standard Navier-Stokes equations coupled to some additional biological models. There are many parameters/constants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written functions to handle the major computations, but a problem I am having is the large number of constants/parameters that these computations depend on. It seems cumbersome to pass 10-20 arguments to a function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One alternative is to make all the constants global variables, but I know this is frowned upon in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the standard way of handling many inputs to a function? Should I make a struct and pass that instead?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="22717" LastActivityDate="2019-06-19T02:43:41.797" Title="C++ Best practices for dealing with many constants, variables in scientific codes" Tags="|c++|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31001" PostTypeId="1" AcceptedAnswerId="31027" CreationDate="2019-02-04T20:05:00.057" Score="6" ViewCount="687" Body="&lt;p&gt;I need to solve a matrix equation which contains a Hadamard product and a standard matrix multiplication: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$A\odot X + BX = C$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$A, C, X \in \mathbb{R}^{m \times n}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B \in \mathbb{R}^{m \times m}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\odot$&lt;/span&gt; denotes the element-wise (Hadamard) matrix product. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this sort of system have a name? I've checked the standard resources (Golub and van Loan, Horn and Johnson), but I didn't find any discussion of this type of problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we let &lt;span class=&quot;math-container&quot;&gt;$a = \text{vec}(A)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x = \text{vec}(X)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$c = \text{vec}(C)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\tilde{a} = \text{diag}(a)$&lt;/span&gt;, then this becomes: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$a \odot x + (I \otimes B)x = \tilde{a}x + (I \otimes B)x = c$$&lt;/span&gt; which has solution &lt;span class=&quot;math-container&quot;&gt;$$x = (\tilde{a} + (I \otimes B))^+c \implies X = \text{unvec}\left[\left(\text{diag}(\text{vec}(A)) + I\otimes B\right)^+\text{vec}(C)\right]$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\otimes$&lt;/span&gt; is the Kronecker product and &lt;span class=&quot;math-container&quot;&gt;$(\cdot)^+$&lt;/span&gt; is the (Moore-Penrose) pseudo-inverse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it's not too expensive to use an iterative solver (since the &lt;span class=&quot;math-container&quot;&gt;$\tilde{a} + (I \otimes B)$&lt;/span&gt; operator can be evaluated more-or-less easily), but I'd like to avoid that if possible. Is it possible to derive an analytical solution for this class of problems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar problem is discussed at &lt;a href=&quot;https://math.stackexchange.com/questions/2583719/how-to-solve-the-linear-equation-a-circ-xb-cx-d&quot;&gt;https://math.stackexchange.com/questions/2583719/how-to-solve-the-linear-equation-a-circ-xb-cx-d&lt;/a&gt;, but they stop at the &quot;vec'd-out&quot; solution. &lt;/p&gt;&#xA;" OwnerUserId="28552" LastActivityDate="2019-02-08T04:05:19.530" Title="Solving a Hadamard + Linear System" Tags="|linear-algebra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31002" PostTypeId="2" ParentId="30999" CreationDate="2019-02-04T21:30:22.583" Score="1" Body="&lt;p&gt;One way that I do is to use singleton. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you start your program you initiate your singleton and fill it with the constant data (probably from a properties file that you have for the run). You get this in every class that you need the values and just use it. &lt;/p&gt;&#xA;" OwnerUserId="17327" LastActivityDate="2019-02-04T21:30:22.583" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31003" PostTypeId="2" ParentId="30999" CreationDate="2019-02-04T23:34:26.523" Score="16" Body="&lt;p&gt;If you have constants that will not change before runs, declare them in a header file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//constants.hpp&#xA;#ifndef PROJECT_NAME_constants_hpp&#xA;#define PROJECT_NAME_constants_hpp&#xA;namespace constants {&#xA;  constexpr double G        = 6.67408e-11;&#xA;  constexpr double M_EARTH  = 5.972e24;&#xA;  constexpr double GM_EARTH = G*M_EARTH; &#xA;}&#xA;#endif&#xA;&#xA;//main.cpp&#xA;using namespace constants;&#xA;auto f_earth = GM_EARTH*m/r/r;  //Good&#xA;auto f_earth = G*M_EARTH*m/r/r; //Also good: compiler probably does math here too&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The reason why you would want to do this is that it allows the compiler to calculate constant values ahead before run-time, which is good if you have a lot of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use a simple class to pass values around:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class Params {&#xA; public:&#xA;  double a,b,c,d;&#xA;  Params(std::string config_file_name){&#xA;    //Load configuration here&#xA;  }&#xA;};&#xA;&#xA;void Foo(const Params &amp;amp;params) {&#xA;  ...&#xA;}&#xA;&#xA;int main(int argc, char **argv){&#xA;  Params params(argv[1]);&#xA;  Foo(params);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17088" LastEditorUserId="17088" LastEditDate="2019-02-05T21:17:45.467" LastActivityDate="2019-02-05T21:17:45.467" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31004" PostTypeId="1" CreationDate="2019-02-05T02:53:58.790" Score="4" ViewCount="130" Body="&lt;p&gt;&lt;strong&gt;Some Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on a C++ translation of a SLATEC routine, &lt;code&gt;R1UPDT&lt;/code&gt;, which performs a Givens rotation: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$r = \frac{1.0}{\sqrt{a^2 + b^2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, this equation is put in a slightly different form, depending upon whether &lt;span class=&quot;math-container&quot;&gt;$|a| &amp;lt; |b|$&lt;/span&gt;. A temporary variable, &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, is used such that &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is smaller than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$|a| &amp;lt;|b| \to c =\frac{a}{b}$&lt;/span&gt; and the equation is written &lt;span class=&quot;math-container&quot;&gt;$r = \frac{b}{\sqrt{c^2 + 1}}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$|a| &amp;gt;|b| \to c =\frac{b}{a}$&lt;/span&gt; and the equation is written &lt;span class=&quot;math-container&quot;&gt;$r = \frac{a}{\sqrt{1+c^2}}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either way, the potential risk of overflow is avoided because the term squared is less than 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, so good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;On to the Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the Fortran code in &lt;code&gt;R1UPDT&lt;/code&gt; is written such that &lt;span class=&quot;math-container&quot;&gt;$r = \frac{0.5}{\sqrt{0.25 + 0.25c^2}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r = 0.5 / SQRT(0.25 + 0.25 * C**2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have to ask: why was the computation coded like this? The RHS of the equation could be factored by hand simply enough and coded as such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r = 1.0 / SQRT(1.0 + C**2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My first thought is that one multiplication could be eliminated and program execution made quicker. However, perhaps this is one more technique for avoiding overflow. Or perhaps ensuring that the radicand is in the range &lt;span class=&quot;math-container&quot;&gt;$[0.25 - 0.5]$&lt;/span&gt; makes it quicker to calculate or yields a more accurate result. I am just speculating; perhaps somebody here can clarify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anybody here knows why the common factor was not taken out of this code?&lt;/p&gt;&#xA;" OwnerUserId="13202" LastEditorUserId="20688" LastEditDate="2019-02-17T20:31:11.397" LastActivityDate="2019-02-17T20:31:11.397" Title="SLATEC Routine Computes Givens Rotation in Unexpected Way" Tags="|floating-point|numerics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31005" PostTypeId="2" ParentId="30999" CreationDate="2019-02-05T03:56:07.680" Score="14" Body="&lt;p&gt;Another alternative that may be in line with your train of thought is to use a namespace (or nested namespaces) to properly group constants. An example might be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;namespace constants {&#xA;   namespace earth {&#xA;      constexpr double G = 6.67408e-11;&#xA;      constexpr double Mass_Earth = 5.972e24;&#xA;      constexpr double GM = G*Mass_Earth;&#xA;   }// constant properties about Earth&#xA;&#xA;   namespace fluid {&#xA;      constexpr double density = 0.999; // g/cm^3&#xA;      constexpr double dyn_viscosity = 1.6735; //mPa * s&#xA;   }// constants about fluid at 2C&#xA;&#xA;   // ... &#xA;&#xA;} // end namespace for constants&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the above technique, you can localize reference constants to some desired files and namespaces, making them more controlled than global variables while getting some of the similar benefits. When you use the constants, it is as simple as doing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;constexpr double G_times_2 = 2.0*constants::earth::G;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you dislike long chains of nested namespaces, you can always shorten things when necessary by using a namespace alias:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;namespace const_earth = constants::earth;&#xA;constexpr double G_times_2 = 2.0*const_earth::G;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2019-06-19T02:43:41.797" LastActivityDate="2019-06-19T02:43:41.797" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31006" PostTypeId="1" CreationDate="2019-02-05T08:29:24.017" Score="1" ViewCount="120" Body="&lt;p&gt;For the polynomial&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p(x) = (x-1)(x-2) \cdots (x-20) - 2^{-23}x^{19}\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use &lt;code&gt;fzero()&lt;/code&gt; in MATLAB, and I set the interval to be &lt;span class=&quot;math-container&quot;&gt;$[0.5,1.5]\cdots [19.5,20.5]$&lt;/span&gt;. It did work for &lt;span class=&quot;math-container&quot;&gt;$[0.5, 1.5]$&lt;/span&gt; to about &lt;span class=&quot;math-container&quot;&gt;$[7.5, 8.5]$&lt;/span&gt;. However, when it goes to larger numbers it seems not to work, due to the sign.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to fix that? Or some other methods?&lt;/p&gt;&#xA;" OwnerUserId="29891" LastEditorUserId="20688" LastEditDate="2019-02-05T16:06:47.373" LastActivityDate="2019-02-05T21:37:11.910" Title="Find the roots of a complicated polynomial" Tags="|polynomials|roots|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31007" PostTypeId="2" ParentId="30971" CreationDate="2019-02-05T15:07:08.883" Score="4" Body="&lt;p&gt;I am familiar with the equation in a different application, it is a so called convection-diffusion equation. There is no problem to solve it for negative values of B. The scheme you mention (Chang-Cooper method) is known in my field as the &lt;strong&gt;exponential upwind scheme&lt;/strong&gt; and it is valid for all nonzero values of B. Due to a lack of time I put here screen shots of my paper with a slightly different notations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Frolkovic, P. &quot;Maximum principle and local mass balance for numerical solutions of transport equation coupled with variable density flow.&quot; Acta Mathematica Universitatis Comenianae 1.68 (1998): 137-157.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that you can find easily on internet. I do not know which paper you use to define the Chang-Cooper method, but I hope you can understand it using my explanation below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the screenshot with the stationary form of the equation where the unknown function is &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;. If you take a single representative constant values of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; (your &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$d&amp;gt;0$&lt;/span&gt; (your &lt;span class=&quot;math-container&quot;&gt;$C&amp;gt;0$&lt;/span&gt;), you obtain an exact solution in a form of exponential interpolation, see (40), where two values of numerical solution &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$c_j$&lt;/span&gt; in two grid points are interpolated (you can think in 1D that &lt;span class=&quot;math-container&quot;&gt;$j=i+1$&lt;/span&gt;): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LKnUE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LKnUE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important role is played by so called Peclet number &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, see above, that can be positive or negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use this interpolation you can derive the exponential upwind scheme, see the next screenshot. The idea can be explained by using a standard linear interpolation for the convection term and standard finite difference for the diffusion term where the diffusion coefficient is multiplied by a value of &lt;span class=&quot;math-container&quot;&gt;$K \ge 1$&lt;/span&gt;, it means that the effective numerical diffusion is enlarged, see (42) below (and you should recognize the formula (43) in your definition of the Chang-Cooper method)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MptO1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MptO1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally I add screenshots of an output from Mathematica where you see that the exponential interpolation is working for positive and negative values of &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; (your &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;) for &lt;span class=&quot;math-container&quot;&gt;$P=-100,-10,-1,1,10,100$&lt;/span&gt; and where you see a plot of different values of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; for different values of Peclet number &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T4OFc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T4OFc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7545" LastEditorUserId="7545" LastEditDate="2019-02-06T14:43:58.153" LastActivityDate="2019-02-06T14:43:58.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31008" PostTypeId="1" AcceptedAnswerId="31020" CreationDate="2019-02-05T15:53:46.030" Score="0" ViewCount="112" Body="&lt;p&gt;I have an ODE system defining a mathematical model of a biological system, say&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{da}{dt}=f_1(a,b,\ldots,z,p)\\&#xA;\frac{db}{dt}=f_2(a,b,\ldots,z,p)\\&#xA;\cdots\\&#xA;\frac{dz}{dt}=f_n(a,b,\ldots,z,p)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with state variables, &lt;span class=&quot;math-container&quot;&gt;$a,b,\ldots,z$&lt;/span&gt;, and parameter vector, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, I need to calculate a scalar model response, &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(p)=\int_0^{t_\text{end}}dt \left(a(t)+b(t)\right)+\int_0^{t_\text{end}}dt \left(x(t)+z(t)\right)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: what is the best way to calculate &lt;span class=&quot;math-container&quot;&gt;$f(p)$&lt;/span&gt; apart from numerical integration once the time-courses are calculated?&lt;/p&gt;&#xA;" OwnerUserId="28411" LastEditorUserId="20688" LastEditDate="2019-02-05T16:03:19.233" LastActivityDate="2019-02-07T14:19:43.253" Title="calculating integral for an ODE system" Tags="|ode|integration|numerics|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31009" PostTypeId="2" ParentId="31006" CreationDate="2019-02-05T17:16:56.690" Score="3" Body="&lt;p&gt;One technique is to use a library with arbitrary large integers and use fixed point arithmetic. Just scale up x by some integer factor, then compute your polynomial in fixed-point, keeping only the sign. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From two values, that you know positive and negative, you can then perform a binary search. At the very end, divide back by the fixed point factor and get the value you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The below is still work-in-progress, but should give you the right idea. (Using vanilla python, because it supports large integers)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It finds the following roots. It has not been debugged thoroughly, but hopefully, this gives you an idea of a way to do it. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1.0&#xA;2.0&#xA;2.999999999999805&#xA;4.000000000261023&#xA;4.999999927551538&#xA;6.000006943952296&#xA;6.999697233936014&#xA;8.007267603450376&#xA;8.917250248517071&#xA;20.846908101482256&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finding the complex roots is left as an exercise to the reader :-)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# an arbitrary fixed-point precision factor&#xA;factor = 2 ** 80&#xA;&#xA;def poly(x):&#xA;    # this function computes 2^23*(x−1)(x−2)...(x−20) − x^19&#xA;    # it has the same sign as (x−1)(x−2)...(x−20) − x^19 / 2^23&#xA;&#xA;    pos = 1&#xA;    for i in range(20):&#xA;        pos = pos * (x - (i + 1) * factor)&#xA;    pos = (2 ** 23) * pos&#xA;    # pos is (2 ** 23) * (x−1)(x−2)...(x−20) (factor^20)&#xA;&#xA;    neg = (x ** 19) * factor&#xA;    # neg is x^19 (factor^19) (factor)&#xA;&#xA;    #print(pos,neg)&#xA;&#xA;    ret = pos - neg &#xA;    # ret is now 2^23*(x−1)(x−2)...(x−20) (factor^20) - x^19 (factor^19) (factor)&#xA;    # which is 2^23*(x−1)(x−2)...(x−20) (factor^20) - x^19 (factor^20)&#xA;    # which is (factor^20)( 2^23*(x−1)(x−2)...(x−20) - x^19)&#xA;    # which has the same sign as (2^23*(x−1)(x−2)...(x−20) - x^19)&#xA;&#xA;    return ret&#xA;&#xA;def search(x,y):&#xA;&#xA;    # print(&quot;Searching between {} and {}&quot;.format(x/factor,y/factor))&#xA;    while((y - x) &amp;gt; 1):&#xA;        z = (x + y) // 2&#xA;&#xA;        if (poly(z) &amp;lt; 0) == (poly(x) &amp;lt; 0):&#xA;            x = z&#xA;        else:&#xA;            y = z&#xA;&#xA;    print(y / factor) # here, we go back to floating point and incur loss of precision&#xA;&#xA;&#xA;def main():&#xA;&#xA;    poly(2 * factor)&#xA;&#xA;    last_v = None&#xA;    last_sign = None&#xA;&#xA;    steps = 100000 # number of steps&#xA;    lower = -10   # range from&#xA;    upper = 30    # range to&#xA;&#xA;    for i in range(steps):&#xA;        v = (((steps - i) * lower) + (i * upper)) * factor // steps&#xA;&#xA;        sign = poly(v) &amp;gt; 0&#xA;&#xA;        if (last_sign is not None) and (sign != last_sign):&#xA;            search(last_v,v)&#xA;&#xA;        last_v = v&#xA;        last_sign = sign&#xA;&#xA;main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29905" LastEditorUserId="29905" LastEditDate="2019-02-05T21:37:11.910" LastActivityDate="2019-02-05T21:37:11.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31010" PostTypeId="1" AcceptedAnswerId="31011" CreationDate="2019-02-05T21:03:52.533" Score="5" ViewCount="4964" Body="&lt;p&gt;I want to preprocess a relatively large dataset using python.  I implemented some Dask parallelization and was stunned by the time reduction.  I figure there are other libraries or frameworks I could use : multiprocessing, Numba, joblib, maybe even PyOpenCL (I don't have a CUDA GPU)…&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are those libraries alternatives of each other, or can one expect a significant improvement by mixing them all ?  Are there common guidelines about when to use one or the other ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already read mixing Dask and Numba was way more efficient than using either of them.&lt;/p&gt;&#xA;" OwnerUserId="29915" LastEditorUserId="29915" LastEditDate="2019-02-05T21:17:16.583" LastActivityDate="2019-02-06T03:26:51.953" Title="How to choose a python parallelization library?" Tags="|python|parallel-computing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31011" PostTypeId="2" ParentId="31010" CreationDate="2019-02-06T03:26:51.953" Score="9" Body="&lt;p&gt;Dask schedules tasks across processes and across nodes, so it is appropriate for use on a single computer, supercomputer, or cloud. Dask also provides specialized data structures to aid in this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiprocessing manages tasks only within a single computer. It may be easier to use than Dask for some tasks, but is less flexible and less scalable. If you're using Dask, you probably don't want multiprocessing. The two will interfere with each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numba is a just-in-time compiler for Python that can provide significant acceleration to code. If most of your work is in Numpy, this is not likely to offer much advantage because Numpy is already highly-optimized. If you're using Python for loops or other high-level, interpreted language features to do the bulk of your computation, then Numba is good for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;joblib has a lot of overlap with Dask, though you might benefit from its memoization features if you use a lot of recursion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PyOpenCL offloads array computation to a GPU. This can probably be used in conjunction with Dask and Numba; however, you likely have only one GPU per machine so using PyOpenCL indiscriminately will create contention for that GPU and, essentially, limit you to only a few processes per node.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2019-02-06T03:26:51.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31012" PostTypeId="2" ParentId="24005" CreationDate="2019-02-06T04:37:55.910" Score="3" Body="&lt;p&gt;Very loosely speaking there are two things most discretization techniques need in order to converge to the actual solution of your PDE as you increase their approximation quality, regardless if you're using DG or not:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Consistency (If a function &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; satisfies the PDE, then it also satisfies your weak formulation)&lt;/li&gt;&#xA;&lt;li&gt;Stability (small changes in data result in small changes in answer)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The first steps of a DG derivation where you integrate by parts on each mesh element preserves (1) because you are starting with the PDE and only applying legal operations from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does not give you (2) though. You can see this yourself by trying to assemble the matrix of a partially formulated DG weak form and looking at its eigenvalues - for time dependent problem we want them all on the left half plane, but without a proper numerical flux they will be everywhere. This leads to a solution that explodes exponentially in time even if the physical problem does not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus you need to add terms to your formulation so that (2) is satisfied, but without hurting (1). This is tricky to do, but not impossible. You can replace function values with cell wall averages without hurting consistency, and you can always add the jump in at cell walls without hurting consistency (because for a solution &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; with suitable smoothness property, the jump is just 0!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trick is to take combinations of jumps and averages and combine them in a way that your scheme is still consistent but also stable. After that a convergence theorem usually reveals itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the basics, but you can also often bring in additional physics into the numerical flux so that it doesn't simply satisfy these mathematical requirements but also plays nicely with conservation principles.&lt;/p&gt;&#xA;" OwnerUserId="547" LastActivityDate="2019-02-06T04:37:55.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31014" PostTypeId="2" ParentId="30989" CreationDate="2019-02-06T11:15:50.913" Score="2" Body="&lt;p&gt;You don't have to store &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt;, you can re-compute it from the rest of the vector. (You can recompute &lt;span class=&quot;math-container&quot;&gt;$v_1$&lt;/span&gt; from the other entries also in the normalized version, but it's clearly an unstable computation because of those subtractions.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, you can reuse the lower-triangular part of &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; to store &lt;span class=&quot;math-container&quot;&gt;$v_2,...v_n$&lt;/span&gt;, so that the factorization is computed fully in-place. Lapack cares a lot abut these in-place versions of algorithms.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2019-02-06T11:41:30.220" LastActivityDate="2019-02-06T11:41:30.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31015" PostTypeId="1" AcceptedAnswerId="31026" CreationDate="2019-02-07T02:18:19.503" Score="1" ViewCount="344" Body="&lt;p&gt;I'm a software engineer and I've just started to fly rc-planes. I'm currently building a self learning auto-pilot for my plane and I'm really interested in CFD. I installed OpenFOAM and ran through some examples. I see there's A LOT to learn but before I start learning OpenFOAM I wonder whether it is actually capable of doing what I want to achieve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately I want to design a new airplane and then let my autopilot simulate a flight using CFD. That means that while the simulation runs, I want to be able to control the plane by adjusting the flaps. This could be used to develop the plane but also to train the auto-pilot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I've got tons to learn before I am there, but my question is; is this actually possible with OpenFOAM? Or is OpenFOAM only for predefined simulations which cannot be altered during the simulation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All tips are welcome!&lt;/p&gt;&#xA;" OwnerUserId="29926" LastActivityDate="2019-02-07T21:08:03.760" Title="Can OpenFoam be used for interactive simulations (flying a plane)?" Tags="|fluid-dynamics|simulation|openfoam|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="31016" PostTypeId="1" CreationDate="2019-02-07T08:41:37.480" Score="1" ViewCount="447" Body="&lt;p&gt;I saw some definition that if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$M=\max_{x\not=0} \tfrac{\|Ax\|}{\|x\|}&#xA;\quad\text{and}\quad&#xA;m=\min_{x\not=0} \tfrac{\|Ax\|}{\|x\|} \,,$$&lt;/span&gt;&#xA;the condition number of &lt;span class=&quot;math-container&quot;&gt;$A=M/m$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion, If we have a matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and a vector &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, shouldn't &lt;span class=&quot;math-container&quot;&gt;$\tfrac{\|Ax\|}{\|x\|}$&lt;/span&gt; just be a constant number? Why are there &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;? Can anyone give a specific example?&lt;/p&gt;&#xA;" OwnerUserId="29891" LastEditorUserId="24953" LastEditDate="2019-02-07T10:14:05.290" LastActivityDate="2019-02-07T16:49:34.480" Title="Condition number of a matrix" Tags="|linear-algebra|condition-number|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31017" PostTypeId="2" ParentId="31016" CreationDate="2019-02-07T09:10:21.070" Score="3" Body="&lt;p&gt;There are different &quot;matrix condition numbers&quot; relative to the problem to be solved. I assume that you are inquiring about the matrix condition number to solving a linear system &lt;span class=&quot;math-container&quot;&gt;$Ax = b$&lt;/span&gt;. The condition of a problem is a measure for the sensitivity of the solution on small perturbations in the problem statement. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the approachin &quot;Matrix Computations, 4th Ed&quot; by Golub and Van Loan (MC4), we can write a perturbed version of the initial problem as &lt;span class=&quot;math-container&quot;&gt;$$(A + \epsilon F) x(\epsilon) = b + \epsilon f$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is a small quantity and &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; are perturbations of the original matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and right-hand-side &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; respectively. Clearly, &lt;span class=&quot;math-container&quot;&gt;$x(0) = x$&lt;/span&gt;: for a zero perturbation, the solution coincides with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a Taylor expansion, one can shown (see MC4) that &lt;span class=&quot;math-container&quot;&gt;$$\frac{\parallel x(\epsilon)-x \parallel}{\parallel x\parallel} \leq \kappa(A)\left(\mid\epsilon\mid \frac{\parallel F\parallel}{\parallel A\parallel} + \mid \epsilon \mid \frac{\parallel f\parallel}{\parallel b\parallel}\right) +O(\epsilon^{2})$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\kappa(A)$&lt;/span&gt;, the matrix condition number (without specifying the exact matrix norm) is given by &lt;span class=&quot;math-container&quot;&gt;$$\kappa(A) = \parallel A \parallel \cdot\parallel A^{-1}\parallel$$&lt;/span&gt; and is a measure for the sensitivty of the problem &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt;. The special case of the matrix 2-norm &lt;span class=&quot;math-container&quot;&gt;$$\kappa_{2}(A) = {\parallel A \parallel}_{2}\cdot {\parallel A^{-1}\parallel}_{2} = \frac{\sigma_{\text{max}}(A)}{\sigma_{\text{min}}(A)}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\sigma_{\text{max}}(A)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma_{\text{min}}(A)$&lt;/span&gt; denote the largest and smallest singular value of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; respectively (in your nomenclature &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, but the definitions you gave for them are not correct).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You misinterpret the definition of a matrix norm: in this defition, the vector &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is not fixed; the matrix norm &lt;span class=&quot;math-container&quot;&gt;${\parallel A\parallel}_{p}$&lt;/span&gt; is defined as &lt;span class=&quot;math-container&quot;&gt;$${\parallel A\parallel}_{p} = \sup_{x \neq 0} \frac{{\parallel Ax\parallel}_{p}}{{\parallel x\parallel}_{p}}$$&lt;/span&gt; so it is in fact the supremum of a ratio of two vector norms where the supremum is searched for taking &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; as &quot;variable&quot;.You can then shown that &lt;span class=&quot;math-container&quot;&gt;${\parallel A\parallel}_{2} = \sigma_{\text{max}}$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final note and rule of thumb: in floating point, having &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; significant digits (like 16 in IEEE double precision), solving a linear problem &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\log_{10}\left(\kappa_{2}(A)\right) \approx p$&lt;/span&gt;, you can only &quot;count&quot; on &lt;span class=&quot;math-container&quot;&gt;$d-p$&lt;/span&gt; digits to be correct in your solution &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. So solving a problem with &lt;span class=&quot;math-container&quot;&gt;$\kappa_{2}(A) \approx 10^{12}$&lt;/span&gt; will only get you at maximum 4 correct digits in the solution &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;em&gt;no matter what algorithm you apply&lt;/em&gt; (in fact, you will only get the 4 digits if you use the most stable algorithm available).   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For in-depth reading, I suggest &quot;Matrix Computations&quot; by Golub and Van Loan and &quot;Accuracy and stability of numerical algorithms&quot; by Higham. Enjoy linear algebra, it's a wonderful world !&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2019-02-07T10:30:16.923" LastActivityDate="2019-02-07T10:30:16.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31018" PostTypeId="2" ParentId="31016" CreationDate="2019-02-07T10:57:10.663" Score="2" Body="&lt;p&gt;First of all, the term &lt;span class=&quot;math-container&quot;&gt;$\frac{\| A x \|}{\| x \|}$&lt;/span&gt; is not constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider a &lt;span class=&quot;math-container&quot;&gt;$2 \times 2$&lt;/span&gt; matrix and let &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt; be the columns of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Then,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ A x = x_1 w_1 + x_2 w_2 . $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, observe that for &lt;span class=&quot;math-container&quot;&gt;$e_1 = (1, 0)^T$&lt;/span&gt; we have that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\| A e_1 \|}{\| x \|} = \frac{\| w_1 \|}{\| x \|} = \| w_1 \|$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and for &lt;span class=&quot;math-container&quot;&gt;$e_2 = (0,1)^T$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\| A e_2 \|}{\| x \|} = \frac{\| w_2 \|}{\| x \|} = \| w_2 \| \,.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, &lt;span class=&quot;math-container&quot;&gt;$\| w_1 \|$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\| w_2 \|$&lt;/span&gt; do not need to be the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, assume that you want to compute &lt;span class=&quot;math-container&quot;&gt;$A e_1$&lt;/span&gt;; the solution is &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;. Furthermore, let &lt;span class=&quot;math-container&quot;&gt;$\| w_2 \| \gg \| w_1 \|$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a computer, however, we are in general not able to perform an exact computation; we have to deal with round-off errors. Hence, consider the perturbed matrix vector product,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ A (e_1 + \epsilon e_2) = w_1 + \epsilon w_2 \,. $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between the true solution and the perturbed one, which is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ w_1 + \epsilon w_2  - w_1 = \epsilon w_2 \,, $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is large (in comparison to the size of &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;) even if &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is small, since &lt;span class=&quot;math-container&quot;&gt;$\| w_2 \|$&lt;/span&gt; is large. Hence, the value of the perturbed matrix vector product is very different from the true value, while the perturbation of the input data was small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that it can cause problems, when a matrix sends vectors of the same magnitude to vectors of very different magnitudes, and the quotient of the largest and smallest possible magnitude is exactly the definition of the condition number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The condition number of a matrix appears in many different contexts. One can, e.g., imagine that a large condition number also causes trouble, when solving a linear system. Take a look at @GertVdE great answer for that matter.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastEditorUserId="24953" LastEditDate="2019-02-07T16:49:34.480" LastActivityDate="2019-02-07T16:49:34.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31019" PostTypeId="1" AcceptedAnswerId="31024" CreationDate="2019-02-07T13:32:55.473" Score="1" ViewCount="780" Body="&lt;p&gt;I would like to solve system &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is SPD, but very ill-conditioned (&lt;span class=&quot;math-container&quot;&gt;$\text{cond}(A)&amp;gt;10^{11}$&lt;/span&gt;). I am interested in using UNpreconditioned version of the conjugate gradient method.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any estimate on norm of residual to which method can actually converge in double precision arithmetic? I am not interested in solving system numerically. I am looking for error bounds that incorporate limited precision floating point arithmetic.&lt;/p&gt;&#xA;" OwnerUserId="29268" LastEditorUserId="29268" LastEditDate="2019-02-07T16:50:06.183" LastActivityDate="2019-02-07T20:10:01.207" Title="Conjugate gradient - ill-conditioning and numerical tolerance" Tags="|linear-solver|iterative-method|conjugate-gradient|precision|condition-number|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31020" PostTypeId="2" ParentId="31008" CreationDate="2019-02-07T14:19:43.253" Score="3" Body="&lt;p&gt;The common way of tracking such properties is indeed to add an additional ODE to the system, here: &lt;span class=&quot;math-container&quot;&gt;$$\frac{df}{dt}=a(t)+b(t)+x(t)+z(t)$$&lt;/span&gt; with initial condition &lt;span class=&quot;math-container&quot;&gt;$$f(0)=0.$$&lt;/span&gt; If this is not possible, then a smart choice of output times e.g. quadrature points in t for an integration afterwards is recommend.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2019-02-07T14:19:43.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31024" PostTypeId="2" ParentId="31019" CreationDate="2019-02-07T20:10:01.207" Score="3" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; denote the solution of &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt; and let &lt;span class=&quot;math-container&quot;&gt;$\hat{x}$&lt;/span&gt; denote the computed solution. We cannot hope to do better than &lt;span class=&quot;math-container&quot;&gt;$$\hat{x} = \text{fl}(x),$$&lt;/span&gt; i.e., the floating point representation of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. In this, the most favorable case, we have &lt;span class=&quot;math-container&quot;&gt;$\hat{x}_j = x_j(1+\delta_j)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$|\delta_j| \leq u$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is the unit roundoff. It follows, that &lt;span class=&quot;math-container&quot;&gt;$\|x-\hat{x}\|_2 \leq u \|x\|_2$&lt;/span&gt;. Now, let &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; denote the residual given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ r = b - A\hat{x} = A(x-\hat{x}).$$&lt;/span&gt; We have &lt;span class=&quot;math-container&quot;&gt;$$\|r\|_2 \leq \|A\|_2 \|x-\hat{x}\|_2 \leq u \|A\|_2 \|x\|_2 \leq u \|A\|_2 \| A^{-1} \|_2 \|b\|_2.$$&lt;/span&gt; We conclude that the relative residual satisfies &lt;span class=&quot;math-container&quot;&gt;$$ \frac{\|r\|_2}{\|b\|_2} \leq u \, \kappa_2(A), $$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\kappa_2(A) = \|A\|_2 \| A^{-1} \|_2$&lt;/span&gt; denotes the 2-norm condition number of the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. &#xA;&lt;hr&gt;&#xA;The estimate above is true for a general matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. In practice, you will find that the relative residual of iterative methods stagnates at the level of &lt;span class=&quot;math-container&quot;&gt;$u \kappa_2(A)$&lt;/span&gt;. &#xA;There is no hope of the CG algorithm doing better in general.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2019-02-07T20:10:01.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31025" PostTypeId="1" CreationDate="2019-02-07T20:40:26.807" Score="1" ViewCount="547" Body="&lt;p&gt;&lt;strong&gt;Aim&lt;/strong&gt;: I am trying to solve a system of SDEs, while using the &lt;code&gt;SDEint&lt;/code&gt; package in python 3.x. It is a system of SDEs adapted from and inspired by the Zombie Apocalypse model, published by &lt;a href=&quot;https://mysite.science.uottawa.ca/rsmith43/Zombies.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Munz, 2009&lt;/a&gt;. I tried to add an explanation of the terms within the code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;: The problem I would like to solve occurs in the example below (overflow encoutered in double_scalar). Note that the noice term is currently 0, so it is basically a system of ODEs and not yet a system of SDEs.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;import sdeint&#xA;from scipy.integrate import odeint&#xA;&#xA;p, f, e, k = 0.7, 0.0002, 0.05, 0.02    # human birth rate, confilt occurence rate, conflict end rate, zombie killing rate&#xA;&#xA;tspan = np.linspace(0, 20., 100)&#xA;y0 = np.array([100000., 0., 0., p])&#xA;&#xA;def ff(y, t):&#xA;    Hi = y[0]&#xA;    Ci = y[1]&#xA;    Zi = y[2]&#xA;&#xA;    # Human are created (exponential)                       --&amp;gt; y[3] * Hi&#xA;    # Humans and zombies engage in conflict at rate (f)     --&amp;gt; - f * Hi * Zi&#xA;    # the conficts end at rate (e) without a victor         --&amp;gt; + e * Ci&#xA;    f0 = y[3] * Hi - f * Hi * Zi + e * Ci&#xA;&#xA;    # Humans and zombies engage in conflict at rate (f)     --&amp;gt; f * Hi * Zi&#xA;    # the conficts end at rate (e) without a victor         --&amp;gt; - e * Ci&#xA;    # and zombies kill humans in conflict at rate (k)       --&amp;gt; - k * Ci&#xA;    f1 = f * Hi * Zi - e * Ci - k * Ci&#xA;&#xA;    # Zombies arise from the earth at a fixed rate          --&amp;gt; 10**7&#xA;    # they arise as victor from a conflict                  --&amp;gt; + k * Ci&#xA;    # enter conflicts                                       --&amp;gt; - f * Hi * Zi&#xA;    # and exit conflicts without killing                    --&amp;gt; + e * Ci&#xA;    f2 = 10**7 + k * Ci - f * Hi * Zi + e * Ci&#xA;&#xA;    f3 = 0&#xA;    print(f0, f1 ,f2, f3)&#xA;    return np.array([f0, f1, f2, f3])&#xA;&#xA;def GG(y, t):&#xA;    return np.diag([0, 0, 0, 0])&#xA;&#xA;result = sdeint.itoint(ff, GG, y0, tspan)&#xA;&#xA;fig = plt.figure()&#xA;ax = fig.add_subplot(1,1,1)&#xA;ax.set_yscale('log')&#xA;ax.set_ylim(bottom=1, top=10**13)&#xA;plt.plot(result)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried to solve the same system of ODEs (without noise) using Scipy's &lt;code&gt;odeint&lt;/code&gt; and this seems to work well due to the possibility of adding several arguments to &lt;code&gt;odeint&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result = odeint(ff, y0, tspan, mxstep=10 ** 9, rtol=10 ** (-3),&#xA;                atol=10 ** (-3), hmax=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I thought maybe &lt;code&gt;SDEint&lt;/code&gt; uses too large integration steps, however there is no possibility to manually add additional arguments to &lt;code&gt;sdeint&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Potential Solution(?)&lt;/strong&gt;: So I was wondering, if a solution might be to manually set the set size of the integration. I tried to do so by increasing the amount of steps in &lt;code&gt;np.linspace&lt;/code&gt; e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tspan = np.linspace(0, 20., 100000) # 100 --&amp;gt; 100000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, this does not seem to work properly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Since I am not really familiar with ODEs/SDEs, I am not sure if I am addressing this problem properly. Is this a common way to address the problem of too large integration steps? And might the solution be to increase the step size or am I completely mistaken?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="29936" LastEditorUserId="29936" LastEditDate="2019-02-10T11:14:24.830" LastActivityDate="2019-02-10T11:14:24.830" Title="SDE solver in python: manual determination of integrator step size (dt)" Tags="|python|ode|stochastic-ode|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31026" PostTypeId="2" ParentId="31015" CreationDate="2019-02-07T21:04:00.727" Score="2" Body="&lt;p&gt;OpenFOAM doesn't have the degree of prescribed mesh motion you are looking for. It has 6 DOF, but thats not sufficient. What you need is a CFD tool which is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A) 3D&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B) Unsteady&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C) Has strong mesh motion capability&lt;/p&gt;&#xA;&#xA;&lt;p&gt;D) has control surface capabilities (can read meshes that define ailerons and other control surfaces, move them (the surface mesh) in accordance with your controller and deform the (CFD) mesh accordingly)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E) can be partnered with your code&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot think of a code that would work for you, and you might find the computational cost to be prohibitive. There are some universities and companies that have specialized capabilities that can do this sort of thing, but those would not be open source.&lt;/p&gt;&#xA;" OwnerUserId="9446" LastEditorUserId="9446" LastEditDate="2019-02-07T21:08:03.760" LastActivityDate="2019-02-07T21:08:03.760" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31027" PostTypeId="2" ParentId="31001" CreationDate="2019-02-08T01:01:33.617" Score="3" Body="&lt;p&gt;If you multiply a Hadamard product (from the left or right) by a vector from the standard basis, it distributes over the terms in the product like so&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\eqalign{&#xA;e_k^T(A\odot B) &amp;amp;= (e_k^TA)\odot(e_k^TB) \cr&#xA;(A\odot B)e_k &amp;amp;= (Ae_k)\odot(Be_k) \cr&#xA;}$$&lt;/span&gt;&#xA;We can use this to solve for the &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; matrix, one column at a time&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\eqalign{&#xA;(A\odot X)e_k + (BX)e_k &amp;amp;= Ce_k \cr&#xA;(Ae_k)\odot(Xe_k) + B(Xe_k) &amp;amp;=Ce_k \cr&#xA;\Big({\rm Diag}(a_k)+B\Big)\,x_k &amp;amp;= c_k \cr&#xA;x_k &amp;amp;= \Big({\rm Diag}(a_k)+B\Big)^+c_k \cr&#xA;}$$&lt;/span&gt;&#xA;Here is some Julia code (version 1.1.0) which demonstrates the technique.&#xA;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;using LinearAlgebra&#xA;&#xA;m,n = 30,50;&#xA;X,A,C,B = zeros(m,n), randn(m,n), randn(m,n), randn(m,m);&#xA;&#xA;for k = 1:n&#xA;  X[:,k] = (Diagonal(A[:,k]) + B) \ C[:,k];&#xA;end&#xA;&#xA;norm(A.*X + B*X - C)&#xA;1.3032451036976518e-12&#xA;&lt;/pre&gt;&#xA;" OwnerUserId="27795" LastEditorUserId="27795" LastEditDate="2019-02-08T04:05:19.530" LastActivityDate="2019-02-08T04:05:19.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31029" PostTypeId="1" AcceptedAnswerId="31030" CreationDate="2019-02-08T09:07:53.940" Score="2" ViewCount="788" Body="&lt;p&gt;I would like to know the algorithm asymptotic complexity with Complete Pivoting. With partial pivoting, it is known to be &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it the same for complete pivoting?&lt;/p&gt;&#xA;" OwnerUserId="29248" LastEditorUserId="20688" LastEditDate="2019-02-08T15:51:49.167" LastActivityDate="2019-02-08T15:51:49.167" Title="Asymptotic Complexity of Gaussian Elimination using Complete Pivoting" Tags="|linear-algebra|linear-solver|complexity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31030" PostTypeId="2" ParentId="31029" CreationDate="2019-02-08T09:41:09.087" Score="4" Body="&lt;p&gt;Yes. Searching the entire trailing submatrix for the next pivot instead of only the current column merely replaces the time spent on finding pivots from &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt;. While the total runtime is increased, the overall asymptotic complexity remains &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2019-02-08T09:41:09.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31031" PostTypeId="1" CreationDate="2019-02-08T14:24:35.267" Score="3" ViewCount="313" Body="&lt;p&gt;I need to find the smallest eigenvalue and the corresponding eigenvector of a sparse matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; whose dimension is &lt;span class=&quot;math-container&quot;&gt;$\approx 10^4$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within Matlab enviroment, I use the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[Evec, Eval]= eigs(M,1,'sa');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The eigenvalue I obtain, &lt;span class=&quot;math-container&quot;&gt;$\lambda_0$&lt;/span&gt;, is very reasonable, but I suspect that the corresponding eigenvector, &lt;span class=&quot;math-container&quot;&gt;$\vec{v}_0$&lt;/span&gt;, is wrong. I also suspect that this possibly wrong eigenvector comes from the fact that matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is such that the lower part of its spectrum is quasi-degenerate, i.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;   \lambda_0 \approx \lambda_1\approx\lambda_2\approx \dots&#xA;$$&lt;/span&gt;&#xA;Notice that the spectrum is &lt;em&gt;quasi&lt;/em&gt; degenerate and not degenerate, meaning that, in any case, one has that &lt;span class=&quot;math-container&quot;&gt;$\lambda_0&amp;lt;\lambda_1&amp;lt;\lambda_2&amp;lt;\dots$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Are my suspects reasonable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is this a well-known problem in Computational Science?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Is there a way to circumvent it, i.e. to  obtain the exact eigenvector &lt;span class=&quot;math-container&quot;&gt;$\vec{v}_0$&lt;/span&gt; associated to the smallest eigenvale &lt;span class=&quot;math-container&quot;&gt;$\lambda_0$&lt;/span&gt; of a matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; whose lower part of the spectrum is quasi-degenerate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional info: matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; represents a quantum Hamiltonian. It is symmetric but neither positive definite nor negative definite. Nevertheless, I am interested just in the lower part of its spectrum, i.e to the most negative eigenvalues.&lt;/p&gt;&#xA;" OwnerUserId="29095" LastEditorUserId="29095" LastEditDate="2019-02-09T10:42:43.547" LastActivityDate="2019-11-06T12:01:55.277" Title="Eigenvectors associated to two quasi-degenerate eigenvalues" Tags="|matlab|eigenvalues|eigensystem|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31032" PostTypeId="2" ParentId="31029" CreationDate="2019-02-08T15:44:52.647" Score="4" Body="&lt;p&gt;Carl's answer is correct, I upvoted it too. The growth in pivot searches from taking &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt; steps to &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt; steps is unfortunate, but doesn't jeopardize the overall complexity. But I think the usual caveats about big-&lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; notation should be doubly repeated, that omitting the constants can obscure important details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hidden problem here is that full pivoting appears to spoil the opportunity to use BLAS3 operations to do deferred updates on the trailing columns. Because any column might be holding the next pivot at any time, they always have to be to kept up to date using BLAS2 kernels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In contrast, partial pivoting only requires BLAS2 to update a thin &quot;leading panel&quot; of upcoming columns, and can update all the trailing columns later using BLAS3 once the panel is done. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bottom line of all this is that full pivoting runs vastly slower than partial pivoting on practical computers, despite having the same asymptotic complexity. &lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2019-02-08T15:50:32.753" LastActivityDate="2019-02-08T15:50:32.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31033" PostTypeId="2" ParentId="21230" CreationDate="2019-02-08T17:10:51.370" Score="3" Body="&lt;p&gt;In Numpy v1.17 (which is, as of writing, the non-release development version) there is an &lt;a href=&quot;https://www.numpy.org/devdocs/reference/generated/numpy.lcm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;lcm&lt;/code&gt;&lt;/a&gt; function that can be used for two numbers with, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;np.lcm(12, 20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or for multiple numbers with, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.lcm.reduce([40, 12, 20])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There's also a &lt;a href=&quot;https://www.numpy.org/devdocs/reference/generated/numpy.gcd.html#numpy.gcd&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;gcd&lt;/code&gt;&lt;/a&gt; function.&lt;/p&gt;&#xA;" OwnerUserId="24565" LastActivityDate="2019-02-08T17:10:51.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31034" PostTypeId="1" CreationDate="2019-02-08T17:14:15.007" Score="0" ViewCount="689" Body="&lt;p&gt;I'm want to load a mesh from a file, generated by &lt;a href=&quot;https://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;triangle&lt;/a&gt;, and I want to use the boundary marks of its nodes (boundary marks of the file). It is possible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I can change the mesh generator (or convert its output file), but I want to use the marks of the nodes of the file, is it possible?&lt;/p&gt;&#xA;" OwnerUserId="17630" LastEditorUserId="20688" LastEditDate="2019-02-11T02:50:49.093" LastActivityDate="2019-02-11T02:50:49.093" Title="Load the mesh file with boundary marks" Tags="|mesh-generation|fenics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31035" PostTypeId="2" ParentId="31031" CreationDate="2019-02-09T00:35:21.843" Score="2" Body="&lt;p&gt;As a first comment, I would mention that you are indeed discretizing, although not spatially, your differential equation. That happens when you choose your base, you could pick more or less elements and different kind of basis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding, your question I doubled checked Matlab's documentation and it seems to be using Lanczos algorithm. So, I would first increase the size of your Krylov subspace and see if this improves your eigenvector. Then, I would try with the number of iterations and tolerance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what is suggested in the documentation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If eigs fails to converge for a given matrix, increase the number of Lanczos basis vectors by increasing the value of 'SubspaceDimension'. As secondary options, adjusting the maximum number of iterations, 'MaxIterations', and the convergence tolerance, 'Tolerance', also can help with convergence behavior.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2019-02-09T00:35:21.843" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31039" PostTypeId="2" ParentId="31034" CreationDate="2019-02-11T02:47:41.517" Score="2" Body="&lt;p&gt;Since GMSH was mentioned in the comments as a possible meshing tool, the common way to deal with it there is to create a &lt;code&gt;Physical Line&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest &lt;code&gt;.geo&lt;/code&gt; (modeling and meshing a square):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cl=0.5;&#xA;Point(1) = {1.,1.,0.,cl};&#xA;Point(2) = {-1.,1.,0.,cl};&#xA;Point(3) = {-1.,-1.,0.,cl};&#xA;Point(4) = {1.,-1.,0.,cl};&#xA;&#xA;Line(1) = {1,2};&#xA;Line(2) = {2,3};&#xA;Line(3) = {3,4};&#xA;Line(4) = {4,1};&#xA;&#xA;Line Loop (1) = {1,2,3,4};&#xA;Surface(1) = {1};&#xA;Physical Surface(1) = {1};   // that is &quot;1&quot; the tag for your 2-D elements, by default, triangles&#xA;Physical Line(2) = {1,2,3,4}; // that is the tag &quot;2&quot; for your contour lines; thus, boundary nodes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After meshing, it will create both 2-D and 1-D mesh elements. Looking inside the generated &lt;code&gt;*.msh&lt;/code&gt;, you will be able to easily distinguish between them. In that way, it would also be easy to assign different boundary conditions to different lines (simply by creating several &lt;code&gt;Physical Line&lt;/code&gt;'s).&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-02-11T02:47:41.517" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31040" PostTypeId="2" ParentId="31004" CreationDate="2019-02-11T06:48:13.360" Score="3" Body="&lt;p&gt;The square root is a relatively tough mathematical operation. Thus, it is reasonable to assume that in &lt;strong&gt;pre-IEEE-754 dominated world&lt;/strong&gt;, the argument range &lt;em&gt;could&lt;/em&gt; significantly influence the accuracy/performance of &lt;code&gt;SQRT&lt;/code&gt;. Especially, as @njuffa mentioned in the comment with&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...wobbling precision of IBM System/360 hexadecimal floating point format&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, I &lt;strong&gt;can not find&lt;/strong&gt; a confirmation to that in this case of a relatively benign rescaling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of research has been done on accurate, fast, and efficient algorithms for square-root &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the past:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1145/365278.365556&quot; rel=&quot;nofollow noreferrer&quot;&gt;C.T. Fike, &quot;Starting approximation for square root calculation on IBM system/360,&quot; &lt;em&gt;Comm. ACM&lt;/em&gt;, vol. 9, no. 4, Apr. 1966.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or even in the recent future,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1016/j.tcs.2005.09.056&quot; rel=&quot;nofollow noreferrer&quot;&gt;P. Kornerup, J.-M. Muller, &quot;Choosing starting values for certain Newton-Raphson iterations,&quot; &lt;em&gt;Theoretical Comput. Sc.&lt;/em&gt;, vol. 351, no. 1, pp. 101–110, Feb. 2006.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://doi.org/10.1109/SIES.2007.4297353&quot; rel=&quot;nofollow noreferrer&quot;&gt;C.-P. Jeannerod &lt;em&gt;et al.&lt;/em&gt;, &quot;Faster floating-point square root for integer processors,&quot; in &lt;em&gt;Int. Symp. Industrial Embedded Syst.&lt;/em&gt;, Lisbon, Portugal, Jul. 2007&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, I do not see why the range &lt;span class=&quot;math-container&quot;&gt;$[0.25,0.5]$&lt;/span&gt; would be preferable to &lt;span class=&quot;math-container&quot;&gt;$[1,2]$&lt;/span&gt;. Especially, from the point of &lt;em&gt;guideline on using &lt;code&gt;SQRT&lt;/code&gt;&lt;/em&gt;, even in pre-IEEE-754 era.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, see the manual &lt;a href=&quot;http://www.bitsavers.org/pdf/ibm/360/model44/C28-6596-2_FortranIV_LibrarySubrs_1966.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;IBM System/360 FORTRAN IV Library Subprograms, 1966&lt;/a&gt;. There, Table 12 &quot;Performance Statistics&quot; on page 39 gives accuracy/speed performance for various math functions in different IBM System/360 models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While (in this table) certain math functions have varying accuracy/speed performance, &lt;code&gt;DSQRT&lt;/code&gt; is listed as a single entry for the full range of argument values. Concluding, there are possibly no recommendations to limit the range of &lt;code&gt;SQRT&lt;/code&gt; for IBM System/360 (btw, even for complex &lt;code&gt;CSQRT&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mVnrv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mVnrv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are infinitely many other platforms, I would conclude that there is not too much to read into that peculiar detail of SLATEC code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your purposes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;with IEEE-754 it should not matter for your C++ translation&lt;/li&gt;&#xA;&lt;li&gt;definitely take a look at modern BLAS/LAPACK implementations that usually have Givens rotations implemented. (For example, Intel MKL &lt;a href=&quot;https://software.intel.com/node/1a7e7c3a-e209-4c7a-b8ea-00c60e103e1f#1A7E7C3A-E209-4C7A-B8EA-00C60E103E1F&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;rotg&lt;/code&gt;&lt;/a&gt;,&lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-fortran-lartg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;lartg&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://software.intel.com/en-us/mkl-developer-reference-c-larot&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;larot&lt;/code&gt;&lt;/a&gt;. It's quite unlikely you actually have to implement this functionality yourself.&lt;/li&gt;&#xA;&lt;li&gt;The following technical note: &lt;a href=&quot;https://www.netlib.org/lapack/lawnspdf/lawn148.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;D. Bindel &lt;em&gt;et al.&lt;/em&gt;, &quot;On computing Givens rotations reliably and efficiently&quot;&lt;/a&gt; gives a lot of information on Givens rotations and numerical issues with them. Quite an interesting read.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-02-11T06:48:13.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31041" PostTypeId="2" ParentId="31004" CreationDate="2019-02-11T10:44:52.470" Score="3" Body="&lt;p&gt;The motivation for arranging the computation in this fashion was almost certainly the issue of wobbling precision when computing with the hexadecimal floating-point format of the IBM System/360. There, the mantissa is normalized such that &lt;span class=&quot;math-container&quot;&gt;$num = \pm 16^{e} \cdot f$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{16} \le f \lt 1$&lt;/span&gt;. This means that up to three leading bits of the mantissa can be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;R1UPTD&lt;/code&gt;, &lt;span class=&quot;math-container&quot;&gt;$c = \max (|v_{n}|, |v_{j}|) / \min (|v_{n}|, |v_{j}|) \le 1$&lt;/span&gt;, which means that &lt;span class=&quot;math-container&quot;&gt;$1 \le 1 + c^{2} \le 2$&lt;/span&gt;, and therefore &lt;span class=&quot;math-container&quot;&gt;$1 \le \sqrt{1 + c^{2}} \le \sqrt{2} \lt 2$&lt;/span&gt;. Thus &lt;span class=&quot;math-container&quot;&gt;$\sqrt{1 +c^{2}} = 16^{1} \cdot f$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{16} \le f \lt \frac{1}{8}$&lt;/span&gt;. This means the leading three bits of the square root result will be zero, resulting in a computation with reduced accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scaling with &lt;span class=&quot;math-container&quot;&gt;$0.25$&lt;/span&gt; halves the result of the square root, so &lt;span class=&quot;math-container&quot;&gt;$\sqrt{0.25 + 0.25 \cdot c^{2}} = 16^{0} \cdot f$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{2} \le f \lt 1$&lt;/span&gt;, meaning there are no leading zero bits in the mantissa and full accuracy is preserved.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: W. J. Cody, &quot;The Construction of Numerical Subroutine Libraries&quot;, &lt;em&gt;SIAM Review&lt;/em&gt;, Vol. 16., No. 1, January 1974, pp. 36-46.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On modern platforms with binary floating-point arithmetic based on the IEEE-754 standard there is no need for this scaling, and most programming environments include a standard function &lt;code&gt;hypot(x,y)&lt;/code&gt; that computes &lt;span class=&quot;math-container&quot;&gt;$\sqrt{x^{2} + y^{2}}$&lt;/span&gt; with optimal accuracy and eliminating issues with overflow or underflow in intermediate computation. Some environments also offer a function &lt;code&gt;rhypot(x,y)&lt;/code&gt; which computes &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{\sqrt{x^{2}+y^{2}}}$&lt;/span&gt; and which perfectly fits the needs of Givens rotation.&lt;/p&gt;&#xA;" OwnerUserId="20458" LastEditorUserId="20458" LastEditDate="2019-02-11T21:03:28.787" LastActivityDate="2019-02-11T21:03:28.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31042" PostTypeId="1" CreationDate="2019-02-11T12:34:12.833" Score="1" ViewCount="3643" Body="&lt;p&gt;I am new to Gmsh and I am having trouble creating a circle with a coarse mesh. I use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Geometry-&amp;gt;Elementary entities-&amp;gt;Add-&amp;gt;Circle&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to create the geometry and then&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mesh-&amp;gt;1D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to create a mesh. Then I can can refine the mesh using Mesh-&gt;Refine by splitting but there seems to be no option for coarsening the mesh?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other shapes/dimensions I can easily coarsen the mesh using the .geo file directly. For example, I created a planar rectangle in 3D and use the 'h' parameter in my .geo file to make the mesh finer/coarser. Here is that .geo file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h = 1;&#xA;//+&#xA;Point(1) = {-1, -1, 0, h};&#xA;//+&#xA;Point(2) = {1, -1, 0, h};&#xA;//+&#xA;Point(3) = {1, 1, 0, h};&#xA;//+&#xA;Point(4) = {-1, 1, 0, h};&#xA;//+&#xA;Line(1) = {1, 2};&#xA;//+&#xA;Line(2) = {2, 3};&#xA;//+&#xA;Line(3) = {3, 4};&#xA;//+&#xA;Line(4) = {4, 1};&#xA;//+&#xA;Line Loop(1) = {3, 4, 1, 2};&#xA;//+&#xA;Plane Surface(1) = {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But my .geo mesh for the circle doesn't to have any place I can use a variable such as 'h' above to refine/coarsen the mesh for the circle:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;//+&#xA;SetFactory(&quot;OpenCASCADE&quot;);&#xA;Circle(1) = {0, 0, 0, 1, 0, 2*Pi};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So how can I create coarser meshes for the circle either in Gmsh or directly using some parameter in the .geo file?&lt;/p&gt;&#xA;" OwnerUserId="26915" LastActivityDate="2019-02-11T13:43:12.010" Title="How can I coarsen a mesh in Gmsh when 'Mesh options' include 'Refine by splitting' but nothing about coarsening?" Tags="|mesh-generation|mesh|gmsh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31043" PostTypeId="2" ParentId="31042" CreationDate="2019-02-11T13:43:12.010" Score="4" Body="&lt;p&gt;To create a coarser mesh, you can set the characteristic length globally to a larger value, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SetFactory(&quot;OpenCASCADE&quot;);&#xA;Mesh.CharacteristicLengthFactor = 2;&#xA;&#xA;Circle(1) = {0, 0, 0, 1, 0, 2*Pi};&#xA;Line Loop(1) = {1};&#xA;Surface(1) = {1};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Increasing the value of &lt;code&gt;Mesh.CharacteristicLengthFactor&lt;/code&gt; results in a coarser mesh; decreasing the value results in a finer mesh.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also set the characteristic length factor in the gmsh options. You can find it under &lt;code&gt;Tools -&amp;gt; Options -&amp;gt; Mesh -&amp;gt; General -&amp;gt; Element size factor&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need a finer control over the element sizes, you can set the element size for different parts of your domain, e.g., by defining a box in which a certain mesh size is required. See &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#Specifying-mesh-element-sizes&quot; rel=&quot;nofollow noreferrer&quot;&gt;Specifying mesh element sizes&lt;/a&gt; in the gmsh manual.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2019-02-11T13:43:12.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31044" PostTypeId="1" CreationDate="2019-02-11T14:47:30.393" Score="2" ViewCount="565" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$M \in \mathbb{R}^{n \times n}$&lt;/span&gt; and denote the set of Orthogonal matrices by&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\mathcal{O}_{n} = \left\lbrace Q \in \mathbb{R}^{n \times n} \colon QQ^{T} = \mathbb{I}_{n} \right\rbrace .&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is an efficient way to find the projection of an arbitrary matrix &lt;span class=&quot;math-container&quot;&gt;$M \in \mathbb{R}^{n \times n}$&lt;/span&gt; onto &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}_{n}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One approach I can find is to compute the Singular Value Decomposition of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, namely,&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;M = U \Sigma V^{T} .&#xA;\end{equation}&lt;/span&gt;&#xA;Then we have&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;Q = UV^{T} .&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other approach besides this SVD?&lt;/p&gt;&#xA;" OwnerUserId="29958" LastEditorUserId="29958" LastEditDate="2019-02-12T11:11:23.117" LastActivityDate="2019-02-18T21:29:14.653" Title="Projection onto the set of Orthogonal matrices" Tags="|linear-algebra|matrix|projection|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31045" PostTypeId="1" CreationDate="2019-02-11T17:15:03.223" Score="3" ViewCount="254" Body="&lt;p&gt;&lt;strong&gt;Aim:&lt;/strong&gt; I am trying to numerically solve a Lotka-Volterra ODE in R, using de &lt;code&gt;sde.sim()&lt;/code&gt; function in the &lt;a href=&quot;https://cran.r-project.org/web/packages/sde/sde.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sde package&lt;/a&gt;. I would like to use the &lt;code&gt;sde.sim()&lt;/code&gt; function in order to eventually transform this system into an SDE. So initially, I started with a simple ODE system (Lotka-Volterra model) without a noise term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Lotka%E2%80%93Volterra_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lotka-Volterra&lt;/a&gt; ODE system:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left\{\begin{matrix}\frac{dx}{dt}= \alpha x -\beta xy. &#xA;\\ \frac{dy}{dt}= \delta xy -\gamma y&#xA;\end{matrix}\right.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with initial values for x = 10 and y = 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parameter values for alpha, beta, delta and gamma are 1.1, 0.4, 0.1 and 0.4 respectively (mimicking this &lt;a href=&quot;https://en.wikipedia.org/wiki/Lotka%E2%80%93Volterra_equations&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Attempt to solve the problem:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(sde)&#xA;d &amp;lt;- expression((1.1 * x[0] - 0.4 * x[0] * x[1]), (0.1 * x[0] * x[1] - 0.4 * x[1]))&#xA;s &amp;lt;- expression(0, 0)&#xA;X &amp;lt;- sde.sim(X0=c(10,10), T = 10, drift=d, sigma=s) &#xA;plot(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, this does not seem to generate a nice cyclic behavior of the predator and prey population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is going wrong in the attempt to solve this system of ODEs in &lt;code&gt;sde.sim()&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="29936" LastEditorUserId="20688" LastEditDate="2019-02-13T17:58:05.947" LastActivityDate="2019-02-13T17:58:05.947" Title="Numerical solving Lotka-Volterra ODE in R" Tags="|ode|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31046" PostTypeId="2" ParentId="30856" CreationDate="2019-02-11T18:00:43.527" Score="1" Body="&lt;p&gt;The way you have set the equation up in your edit, allows you to solve it with thomas algorithm. If you look up an algorithm for TDMA, you can see that this equation will create a tridiagonal matrix A of coefficients of your (n) timestep, and a right hand side b depending on your previous timesteps (n-1, n-2) solution values. And this will allow for an easy solve of your BDF1 timestepping. The wiki page for thomas algorithm is here:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case you would like an explanation for how BDF1 works, you can think of it like so, I'll explain it for a steady state euler CFD code. In steady state euler we have our vector of conservative variables, &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;, and our residual operator corresponding to the discretized and integrated PDE, which must equal zero. &lt;span class=&quot;math-container&quot;&gt;$$R(u) = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We begin with &lt;span class=&quot;math-container&quot;&gt;$u_n$&lt;/span&gt;, for which &lt;span class=&quot;math-container&quot;&gt;$R(u_n)$&lt;/span&gt; does not equal zero, and a &lt;span class=&quot;math-container&quot;&gt;$R(u_{n+1})$&lt;/span&gt; which we want to be equal to zero. If we take the taylor series expansion of &lt;span class=&quot;math-container&quot;&gt;$R(u_{n+1})$&lt;/span&gt; about &lt;span class=&quot;math-container&quot;&gt;$u_n$&lt;/span&gt; we get:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$R(u_{n+1}) = R(u_n) + \frac{\partial R}{\partial u_n}\Delta u = 0$$&lt;/span&gt;&#xA;We can then bring the left term to the RHS and get:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial R}{\partial u_n}\Delta u = -R(u_n)$$&lt;/span&gt;&#xA;Now, by solving this linear system, we will not actually solve the nonlinear governing equations in one step, because a linear taylor series expansion is not sufficient to solve the nonlinear equation that quickly, but we can repeat this iteration many times to solve the steady state problem, and you may notice this bears more that a passing resemblance to Newton's Method. Hope this was helpful!&lt;/p&gt;&#xA;" OwnerUserId="9446" LastActivityDate="2019-02-11T18:00:43.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31047" PostTypeId="2" ParentId="5509" CreationDate="2019-02-11T18:19:08.380" Score="1" Body="&lt;p&gt;I can't think of a generalized kutta condition for general shapes. But I can tell you it is useful for bluff bodies, such as cylinders and the like. The thing to realize is that the kutta condition is used to enforce a unique solution in terms of circulation and stagnation point location in place of the infinitely many theoretical ones. I would suggest sticking to relatively &quot;real&quot; aerodynamic shapes, such as airfoils, squares, circles, diamonds, half bodies, and then developing a way to determine your desired stagnation point in the code and enforcing the kutta condition there. Otherwise you can end up with infinite solutions and/or garbage. Sorry if this restricts your plans, but hope it was helpful!&lt;/p&gt;&#xA;" OwnerUserId="9446" LastActivityDate="2019-02-11T18:19:08.380" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31048" PostTypeId="1" AcceptedAnswerId="31049" CreationDate="2019-02-11T20:46:33.897" Score="6" ViewCount="133" Body="&lt;p&gt;I need to solve a linear regression problem &lt;span class=&quot;math-container&quot;&gt;$$Ax=y$$&lt;/span&gt; which is hugely underdetermined. I have around &lt;span class=&quot;math-container&quot;&gt;$10^6$&lt;/span&gt; features but only &lt;span class=&quot;math-container&quot;&gt;$10^3$&lt;/span&gt; equations. So &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$1,000\times 1,000,000$&lt;/span&gt; matrix and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; a vector of length &lt;span class=&quot;math-container&quot;&gt;$1,000$&lt;/span&gt;, both are given and I need to find &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; (of length &lt;span class=&quot;math-container&quot;&gt;$1,000,000$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution is of course &lt;span class=&quot;math-container&quot;&gt;$x^*=A^\dagger y$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$A^\dagger$&lt;/span&gt; is the pseudo inverse. &lt;span class=&quot;math-container&quot;&gt;$x^*$&lt;/span&gt; minimizes the least-square expression &lt;span class=&quot;math-container&quot;&gt;$(Ax-y)^T(Ax-y)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All this is well known and I can do very efficiently using standard libraries (&lt;code&gt;lstsq&lt;/code&gt; of &lt;code&gt;numpy&lt;/code&gt; for example). The problem is that I want to add a regularization term of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\min_x\Big\{ (Ax-y)^T(Ax-y)+x^T R x\Big\}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is some &lt;span class=&quot;math-container&quot;&gt;$10^6\times 10^6$&lt;/span&gt; matrix that I have (it is of course very sparse). The analytic solution this equation is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x=\left(A^TA+R\right)^\dagger y\ ,$$&lt;/span&gt;&#xA;but it is completely impractical to even instantiate the matrix &lt;span class=&quot;math-container&quot;&gt;$A^TA+R$&lt;/span&gt; (a &lt;span class=&quot;math-container&quot;&gt;$10^6\times10^6$&lt;/span&gt; matrix). &lt;strong&gt;What's the best way to obtain the solution in a numerically stable manner?&lt;/strong&gt; If that helps, I know how to express &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$R=B^TB$&lt;/span&gt; with b is a matrix of the same shape as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--- &lt;em&gt;Preemtive apology::&lt;/em&gt;&#xA;I realize that this might be a very basic question, but I can't find a standard way to do it in any of the packages I use (scikit and the likes). I also know that when &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; is the identity this is the usual Tikhonov regularization, and there is a trick there of evaluating &lt;span class=&quot;math-container&quot;&gt;$A A^T$&lt;/span&gt; instead of &lt;span class=&quot;math-container&quot;&gt;$A^TA$&lt;/span&gt;, but I'm not sure if that trick applies for arbitrary &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="753" LastActivityDate="2019-02-13T18:24:05.887" Title="numerical solution of an under-determined linear equation in high dimensions" Tags="|linear-algebra|linear-solver|convex-optimization|inverse-problem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31049" PostTypeId="2" ParentId="31048" CreationDate="2019-02-11T23:44:11.903" Score="9" Body="&lt;p&gt;You want to minimize&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\min \| Ax -y \|_{2}^{2} + x^{T}B^{T}Bx=\| Ax -y \|_{2}^{2} + \| Bx \|_{2}^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\| u \|_{2}^{2} + \| v \|_{2}^{2}=&#xA;\left\| \left[ \begin{array}{c}&#xA;u \\&#xA;v&#xA;\end{array} \right] \right\|_{2}^{2}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus your problem can be written as  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\min \| Hx - g \|_{2}^{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$H=\left[ \begin{array}{c}&#xA;A \\&#xA;B&#xA;\end{array}&#xA;\right]$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g=\left[&#xA;\begin{array}{c}&#xA;y \\&#xA;0 &#xA;\end{array}&#xA;\right]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In using an iterative algorithm such as LSQR to solve the least squares problem, you'll need to be able to compute products of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$w=Hv$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$w=\left[&#xA;\begin{array}{c}&#xA;Av \\&#xA;Bv &#xA;\end{array}&#xA;\right]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, you'll need to be able to compute products of the form &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v=H^{T}v$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be done as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$v=\left[ A^{T}  B^{T} \right]w$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2019-02-13T18:24:05.887" LastActivityDate="2019-02-13T18:24:05.887" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31050" PostTypeId="1" CreationDate="2019-02-12T09:48:18.333" Score="0" ViewCount="422" Body="&lt;p&gt;I am trying to solve a system of 8 coupled differential equations using scipy's odeint. I have already written my code and it runs fine, but the solutions I get are completely different from what I expected. Originally, I would write all these equations within a loop, but since I was having problems with them I wrote them all separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Zd argument that I need to pass is an exponentially decreasing variable the same size as my time vector. However, since odeint only calculates one sample at a time, I need to give it the right Zd sample for each time, so I use a copy of t to get it inside the function. The differential equations have four terms each and they need to be turned on/off at different times, so I use the step function there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This system of equations is supposed to represent a sort of energy going through adjoining systems, so the eight Z functions are the evolution of energy within each one of these systems over time. At t=0, the energy enters the first of these systems and the others have no energy whatsoever. Zd represents the input energy into the system, which changes over time (it decays exponentially as a result of scattering) and needs to be turned on/off as energy enters/leaves each layer.  By the end of my time window, I expected to have little or no energy at all. However, my resulting Zs grow almost exponentially and this doesn't make sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have checked it over and over again but I can't see what is wrong in it. The signs and times in the equations are correct. Any idea on what is wrong here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code and a plot of the results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def ODE_solver(Z,t,Q,Zd,Z0,dts,ts,cf,t_copy):&#xA;&#xA;    import numpy as np&#xA;&#xA;    # Define functions to solve for:&#xA;    [Z1, Z2, Z3, Z4, Z5, Z6, Z7, Z8] = Z&#xA;    # Define time intervals:&#xA;    [dt1, dt2, dt3, dt4, dt5, dt6, dt7, dt8] = dts    &#xA;    # Define cumulative times:&#xA;    [t1, t2, t3, t4, t5, t6, t7, t8] = ts    &#xA;    # Q factors:&#xA;    [Q1, Q2, Q3, Q4, Q5, Q6, Q7, Q8] = Q&#xA;&#xA;    # Set some boundary conditions:&#xA;    Z9=0; dt9=1/4 # (It doesn't matter which value I have here, since Z9 = 0)&#xA;    t0=0; dt0=1/4; &#xA;&#xA;    # Find index of the time sample that is being used at the moment:&#xA;    t_ind=(np.abs(t-t_copy)).argmin()&#xA;    Zd=Zd[t_ind]&#xA;&#xA;    # j=1&#xA;    dZ1dt= ((1/(4*dt0)) * Z0 * step_fun (t,t0)) + ((1/(4*dt2)) * Z2 * step_fun (t,t0)) &#xA;    - ((1/(4*dt1)) * Z1 * step_fun (t,t1)) - ((1/(4*dt1)) * Z1 * step_fun(t,t0)) &#xA;    + (((2*np.pi*cf)*Q1) * Zd * step_fun(t,t0) * inv_step_fun(t,t1)) &#xA;&#xA;    # j=2&#xA;    dZ2dt= ((1/(4*dt1)) * Z1 * step_fun (t,t1)) + ((1/(4*dt3)) * Z3 * step_fun (t,t1)) &#xA;    - ((1/(4*dt2)) * Z2 * step_fun (t,t2)) - ((1/(4*dt2)) * Z2 * step_fun(t,t1)) &#xA;    + (((2*np.pi*cf)*Q2) * Zd * step_fun(t,t1) * inv_step_fun(t,t2)) &#xA;&#xA;    # j=3&#xA;    dZ3dt= ((1/(4*dt2)) * Z2 * step_fun (t,t2)) + ((1/(4*dt4)) * Z4 * step_fun (t,t2)) &#xA;    - ((1/(4*dt3)) * Z3 * step_fun (t,t3)) - ((1/(4*dt3)) * Z3 * step_fun(t,t2)) &#xA;    + (((2*np.pi*cf)*Q3) * Zd * step_fun(t,t2) * inv_step_fun(t,t3))     &#xA;&#xA;    # j=4&#xA;    dZ4dt= ((1/(4*dt3)) * Z3 * step_fun (t,t3)) + ((1/(4*dt5)) * Z5 * step_fun (t,t3)) &#xA;    - ((1/(4*dt4)) * Z4 * step_fun (t,t4)) - ((1/(4*dt4)) * Z4 * step_fun(t,t3)) &#xA;    + (((2*np.pi*cf)*Q4) * Zd * step_fun(t,t3) * inv_step_fun(t,t4))   &#xA;&#xA;    # j=5&#xA;    dZ5dt= ((1/(4*dt4)) * Z4 * step_fun (t,t4)) + ((1/(4*dt6)) * Z6 * step_fun (t,t4)) &#xA;    - ((1/(4*dt5)) * Z5 * step_fun (t,t5)) - ((1/(4*dt5)) * Z5 * step_fun(t,t4)) &#xA;    + (((2*np.pi*cf)*Q5) * Zd * step_fun(t,t4) * inv_step_fun(t,t5))  &#xA;&#xA;    # j=6&#xA;    dZ6dt= ((1/(4*dt5)) * Z5 * step_fun (t,t5)) + ((1/(4*dt7)) * Z7 * step_fun (t,t5)) &#xA;    - ((1/(4*dt6)) * Z6 * step_fun (t,t6)) - ((1/(4*dt6)) * Z6 * step_fun(t,t5)) &#xA;    + (((2*np.pi*cf)*Q6) * Zd * step_fun(t,t5) * inv_step_fun(t,t6))  &#xA;&#xA;    # j=7&#xA;    dZ7dt= ((1/(4*dt6)) * Z6 * step_fun (t,t6)) + ((1/(4*dt8)) * Z8 * step_fun (t,t6)) &#xA;    - ((1/(4*dt7)) * Z7 * step_fun (t,t7)) - ((1/(4*dt7)) * Z7 * step_fun(t,t6)) &#xA;    + (((2*np.pi*cf)*Q7) * Zd * step_fun(t,t6) * inv_step_fun(t,t7))  &#xA;&#xA;    # j=8&#xA;    dZ8dt= ((1/(4*dt7)) * Z7 * step_fun (t,t7)) + ((1/(4*dt9)) * Z9 * step_fun (t,t7)) &#xA;    - ((1/(4*dt8)) * Z8 * step_fun (t,t8)) - ((1/(4*dt8)) * Z8 * step_fun(t,t7)) &#xA;    + (((2*np.pi*cf)*Q8) * Zd * step_fun(t,t7) * inv_step_fun(t,t8))  &#xA;&#xA;&#xA;    return [dZ1dt,dZ2dt,dZ3dt,dZ4dt,dZ5dt,dZ6dt,dZ7dt,dZ8dt]&#xA;&#xA;def step_fun(x,a):&#xA;&#xA;    if x==a:  return 0.5&#xA;    elif x&amp;lt;a: return 0&#xA;    elif x&amp;gt;a: return 1 &#xA;&#xA;&#xA;def inv_step_fun(x,a):&#xA;&#xA;    if x==a:  return 0.5&#xA;    elif x&amp;lt;a: return 1&#xA;    elif x&amp;gt;a: return 0 &#xA;&#xA;####################################################################################################&#xA;&#xA;import copy&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from scipy.integrate import odeint&#xA;&#xA;# Number of equations:&#xA;N=8&#xA;&#xA;# Define t:&#xA;t=np.arange(0,50,0.025)&#xA;t_copy=copy.deepcopy(t)&#xA;Nt=len(t)&#xA;&#xA;# Define other parameters/arguments:&#xA;Q=(0.008017307188069223, 0.008117153569161557, 0.008081825341069762, 0.00803759248462624, 0.00803759248462624, 0.008081825341069762, 0.008117153569161557, 0.008017307188069223)&#xA;Z0=np.array([22050, 0., 0., 0., 0., 0., 0., 0.])&#xA;Zd0=Z0[0]&#xA;dts=np.array([20.67120623,  1.43225437,  1.55738981,  1.63132137,  1.63132137, 1.55738981,  1.43225437, 20.67120623])&#xA;ts=np.array([20.67120623, 22.10346059, 23.66085041, 25.29217178, 26.92349315, 28.48088296, 29.91313733, 50.58434356])&#xA;cf=1.5&#xA;&#xA;# Define Zd: this is a vector of size Nt, but I will need to give the function only the right sample&#xA;# at each time t.&#xA;Zd=1/(1+np.exp(t))*5e4&#xA;&#xA;# Solve system of differential equations: &#xA;Z=odeint(ODE_solver,Z0,t,args=(Q,Zd,Zd0,dts,ts,cf,t_copy))&#xA;&#xA;# Let's rearrange this results so it is easier to read in the future:&#xA;Zs=[]&#xA;for n in range(N):&#xA;    Zs.append([])&#xA;    for vec in Z:&#xA;        Zs[n].append(vec[n])&#xA;    Zs[n]=np.array(Zs[n])&#xA;&#xA;plt.figure()&#xA;for vec in Zs:&#xA;    plt.plot(t,vec)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/79Bbq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/79Bbq.png&quot; alt=&quot;I was expecting almost this exact result, but reversed in time (functions that decrease almost exponentially over time&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29963" LastActivityDate="2019-02-12T15:14:43.317" Title="Unexpected solutions solving an ODE using odeint" Tags="|python|ode|differential-equations|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="31051" PostTypeId="2" ParentId="31044" CreationDate="2019-02-12T14:19:50.297" Score="2" Body="&lt;p&gt;Some thoughts on &lt;strong&gt;a particular case&lt;/strong&gt; of low-rank matrices (&lt;span class=&quot;math-container&quot;&gt;$k=\text{rank}(M)\ll n)$&lt;/span&gt;. Here, I can suggest some economical version of SVDs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rand-SVD&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the &lt;a href=&quot;https://www.rdocumentation.org/packages/RGeode/versions/0.1.0/topics/randSVD&quot; rel=&quot;nofollow noreferrer&quot;&gt;R package documentation&lt;/a&gt; that also contains references for canonical papers of V. Rokhlin and P. Martinsson on this topic. That should reduce the complexity to &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(nk^2+k^3)$&lt;/span&gt; as opposed to &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(n^3)$&lt;/span&gt; for dense SVD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ACA-based&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find (or if you start with one) low-rank representation of matrix &lt;span class=&quot;math-container&quot;&gt;$M=AB^T$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A,B\in\mathbb R^{n\times k}$&lt;/span&gt;. This might come from &quot;physics&quot; (if the matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; comes from the discretization) or, by using a pure linear algebra approach like &lt;a href=&quot;https://doi.org/10.1007/PL00005410&quot; rel=&quot;nofollow noreferrer&quot;&gt;ACA (adaptive cross approximation)&lt;/a&gt; (usually at &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(nk^2)$&lt;/span&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Find QR of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(nk^2)$&lt;/span&gt; resulting in &lt;span class=&quot;math-container&quot;&gt;$A=Q_AR_A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B=Q_BR_B$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$Q_A, Q_B\in\mathbb R^{n\times k}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$R_A,R_B\in\mathbb R^{k\times k}$&lt;/span&gt;, upper triangular.&lt;/li&gt;&#xA;&lt;li&gt;Find &lt;span class=&quot;math-container&quot;&gt;$P = R_A R_B$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(k^3)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Compute SVD of &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(k^3)$&lt;/span&gt; resulting in &lt;span class=&quot;math-container&quot;&gt;$P=U_P\Sigma V_P^T$&lt;/span&gt;. Here, &lt;span class=&quot;math-container&quot;&gt;$U_P, \Sigma, V_P \in \mathbb R^{k\times k}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt; - diagonal. Notice, &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt; is the matrix of singular values not only for &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, but for the original &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, as well.&lt;/li&gt;&#xA;&lt;li&gt;Find &lt;span class=&quot;math-container&quot;&gt;$U=Q_AU_P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V=Q_BV_P$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(nk^2)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Now you have an SVD of &lt;span class=&quot;math-container&quot;&gt;$M=U\Sigma V^T$&lt;/span&gt; in total of &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(nk^2+k^3)$&lt;/span&gt; operations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2019-02-18T21:29:14.653" LastActivityDate="2019-02-18T21:29:14.653" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31052" PostTypeId="2" ParentId="31050" CreationDate="2019-02-12T15:14:43.317" Score="1" Body="&lt;p&gt;For the interpretation where the left and right &quot;virtual&quot; compartments are infinite sinks at level &lt;code&gt;Z0=Z9=0&lt;/code&gt;, and the walls between the compartments &lt;code&gt;j&lt;/code&gt; and &lt;code&gt;j+1&lt;/code&gt; are removed at time &lt;code&gt;ts[j]&lt;/code&gt; where also the external filling of compartment &lt;code&gt;j&lt;/code&gt; starts, I get the shortened implementation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def ODE_func(Z,t,Q,Zd,Z0,dts,ts,cf,t_copy):&#xA;&#xA;    # Declare some boundary conditions:&#xA;    Z9=0; dt9=1/4; t9=ts[-1]+dt9 # (It doesn't matter which value I have here, since Z9 = 0)&#xA;    Z0=0; dt0=1/4; t0=0;&#xA;    # Define functions to solve for:&#xA;    Z = np.concatenate([[Z0],Z,[Z9]]);&#xA;    # Define time intervals:&#xA;    dt = np.concatenate([[dt0],dts,[dt9]])&#xA;    # Define cumulative times:&#xA;    ts = np.concatenate([[t0],ts,[t9]]);&#xA;&#xA;&#xA;    # Find index of the time sample that is being used at the moment:&#xA;    Zd=np.interp(t,t_copy,Zd);&#xA;&#xA;    dZdt = [ (Z[j-1]/dt[j-1]-Z[j]/dt[j])*step_fun(t,ts[j-1])/4 &#xA;             + (Z[j+1]/dt[j+1]-Z[j]/dt[j])*step_fun(t,ts[j])/4&#xA;             + 2*np.pi*cf*Q[j-1]*Zd*step_fun(t,ts[j-1])*step_fun(ts[j],t) for j in range(1,9)]&#xA;&#xA;    return dZdt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the plots of the &quot;fill heights&quot; &lt;code&gt;Z[j]/dt[j], j=1,...,8&lt;/code&gt; over the long term as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h08Ja.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h08Ja.png&quot; alt=&quot;long-term evolution of energy levels&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where indeed the levels start to fall after some time.&lt;/p&gt;&#xA;" OwnerUserId="6839" LastActivityDate="2019-02-12T15:14:43.317" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31054" PostTypeId="1" AcceptedAnswerId="31104" CreationDate="2019-02-13T02:26:30.537" Score="1" ViewCount="242" Body="&lt;p&gt;I was curious if someone could share their opinion on this matter.  I have noticed that some people in literature normalize their Legendre polynomials, i.e. divide or multiply the polynomial by &lt;span class=&quot;math-container&quot;&gt;$$\sqrt{\frac{2n+1}{2}},$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the order of the polynomial.  I am quite new to DG, but I am experiencing better results when I normalize my Legendre polynomial than when I don't.  Is there a particular reason why people normalize?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Density at &lt;span class=&quot;math-container&quot;&gt;$T = 0.3$&lt;/span&gt; with normalized Legendre polynomial.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NLQkF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NLQkF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Density at &lt;span class=&quot;math-container&quot;&gt;$T = 0.3$&lt;/span&gt; with unnormalized Legendre Polynomial&#xA;&lt;a href=&quot;https://i.stack.imgur.com/aqL1M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aqL1M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of the resolution, I am running on a &lt;span class=&quot;math-container&quot;&gt;$50\times 50$&lt;/span&gt; grid points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Polynomial subroutine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function legendre (x,n)&#xA;integer :: n&#xA;real(kind=8) :: x&#xA;real(kind=8) :: legendre&#xA;x = min(max(x,-1.0),1.0)&#xA;select case(n)&#xA;case(0)&#xA; legendre = 1.0&#xA;case(1)&#xA; legendre = x&#xA;case(2)&#xA; legendre = 0.5*(3*x**2-1)&#xA;case(3)&#xA; legendre = 0.5*(5.0*x**3-3.0*x)&#xA;case(4)&#xA; legendre = 0.125*(35.0*x**4-30.0*x**2+3.0)&#xA;case(5)&#xA; legendre = 0.125*(63.0*x**5-70.0*x**3+15.0*x)&#xA;case(6)&#xA; legendre = 1.0/16.0*(231.0*x**6-315.0*x**4+105.0*x**2-5.0)&#xA;end select&#xA;legendre = sqrt((2.0*dble(n)+1.0)/2.0)*legendre&#xA;return&#xA;end function legendre&#xA;&#xA;function legendre_prime (x,n)&#xA;integer :: n&#xA;real(kind=8) :: x&#xA;real(kind=8) :: legendre_prime&#xA;x = min(max(x,-1.0),1.0)&#xA;&#xA;select case(n)&#xA;case(0)&#xA;  legendre_prime = 0.0&#xA;case(1)&#xA;  legendre_prime = 1.0&#xA;case(2)&#xA;  legendre_prime = 3.0*x&#xA;case(3)&#xA;  legendre_prime = 0.5*(15.0*x**2-3.0)&#xA;case(4)&#xA;  legendre_prime = 0.125*(140.0*x**3-60.0*x)&#xA;case(5)&#xA;  legendre_prime = 0.125*(315.0*x**4-210.0*x**2+15.0)&#xA;case(6)&#xA;  legendre_prime = 1.0/16.0*(1386.0*x**5-1260.0*x**3+210.0*x)&#xA;end select&#xA;legendre_prime = sqrt((2.0*dble(n)+1.0)/2.0)*legendre_prime&#xA;return&#xA;end function legendre_prime&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="21337" LastEditorUserId="20688" LastEditDate="2019-06-22T19:08:21.213" LastActivityDate="2019-06-22T19:08:21.213" Title="Normalization of polynomials for discontinuous Galerkin methods (DGM)" Tags="|finite-element|fluid-dynamics|discontinuous-galerkin|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31055" PostTypeId="2" ParentId="31045" CreationDate="2019-02-13T08:59:16.167" Score="3" Body="&lt;p&gt;I did not yet succeed to use the &lt;code&gt;sde.sim()&lt;/code&gt; function of the SDE package, however, I succeeded to solve the system (with and without noise) using the suggestions of &lt;a href=&quot;https://scicomp.stackexchange.com/users/18981/chris-rackauckas&quot;&gt;Chris&lt;/a&gt; and the &lt;code&gt;diffeqr&lt;/code&gt; package in &lt;code&gt;R&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(plotly)&#xA;&#xA;# Lotka-Volterra Model (SDE without noise)&#xA;f &amp;lt;- function(u,p,t) {&#xA;  du1 = p[1]*u[1]-p[2]*u[1]*u[2]&#xA;  du2 = p[3]*u[1]*u[2]-p[4]*u[2]&#xA;  return(c(du1,du2))&#xA;}&#xA;&#xA;g &amp;lt;- function(u,p,t) {&#xA;  return(c(0*u[1],0*u[2]))&#xA;}&#xA;&#xA;u0 = c(10.0, 10.0)&#xA;tspan &amp;lt;- list(0.0,100.0)&#xA;p = c(1.1, 0.4, 0.1, 0.4)&#xA;sol = diffeqr::sde.solve(f,g,u0,tspan,p=p,saveat=0.005)&#xA;udf = as.data.frame(sol$u)&#xA;&#xA;plot_ly(udf, x = sol$t, y = ~V1, name = 'Prey', type = 'scatter', mode = 'lines') %&amp;gt;%&#xA;  add_trace(y = ~V2, name = 'Predator', mode = 'lines')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, not the solution to the problem, but a step in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="29936" LastActivityDate="2019-02-13T08:59:16.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31058" PostTypeId="1" CreationDate="2019-02-15T08:27:50.863" Score="3" ViewCount="88" Body="&lt;p&gt;When doing LU decomposition, the algorithm will break down if any of the diagonal element &lt;span class=&quot;math-container&quot;&gt;$x_{ii}$&lt;/span&gt; is zero.  Therefore, we can use pivoting on the matrix such that &lt;span class=&quot;math-container&quot;&gt;$x_{ii}$&lt;/span&gt; is no longer zero. That is instead of looking at the &lt;span class=&quot;math-container&quot;&gt;$x_{ii}$&lt;/span&gt; we look at another element &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; in the matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One problem with LU decomposition is that when our matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is sparse, we would like to keep the sparsity pattern of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, something that will not happen when we do LU decomposition. Therefore, we can use incomplete LU decomposition, where if &lt;span class=&quot;math-container&quot;&gt;$x_{ij}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is zero, we also skip &lt;span class=&quot;math-container&quot;&gt;$x_{ij}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; depending on where &lt;span class=&quot;math-container&quot;&gt;$x_{ij}$&lt;/span&gt; is in &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: will we do pivoting for incomplete LU decomposition? If we do, how do we do it? If we don’t, how do we make sure that our algorithm doesn't break?&lt;/p&gt;&#xA;" OwnerUserId="29985" LastEditorUserId="29985" LastEditDate="2019-02-16T10:33:27.780" LastActivityDate="2019-02-16T10:33:27.780" Title="Numerical analysis, pivoting and incomplete LU decomposition" Tags="|linear-solver|sparse-matrix|preconditioning|matrix-factorization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31059" PostTypeId="1" AcceptedAnswerId="31060" CreationDate="2019-02-15T08:30:57.117" Score="0" ViewCount="711" Body="&lt;p&gt;I'm working with Gmsh to generate &lt;strong&gt;1D meshes of polygon edges&lt;/strong&gt;. &#xA;I have noticed that drawing a polygon by &quot;vertices and lines&quot; the program always assigns to the first N nodes of the mesh the coordinates of the N vertices of the polygon; all the other nodes instead are ordered clockwise. Is there a way to order all the nodes clockwise, including the vertices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="26978" LastEditorUserId="26978" LastEditDate="2019-02-15T10:21:23.350" LastActivityDate="2020-05-01T18:10:58.793" Title="Node re-numbering in 1D mesh GMSH" Tags="|mesh|gmsh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31060" PostTypeId="2" ParentId="31059" CreationDate="2019-02-15T13:01:05.593" Score="3" Body="&lt;p&gt;I'm not aware of any way to do this in a .geo file, under the GMSH application, or using the libgmsh interfaces. Somewhat annoyingly, the last operation gmsh does before a write tends to be a renumbering to sweep over 0D, 1D, 2D, and 3D objects sequentially. There has been some work on renumbering options recently, but it's targeted renumbering to make the indices dense rather than to allow other options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the GMSH msh formats are &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#Legacy-formats&quot; rel=&quot;nofollow noreferrer&quot;&gt;very well documented&lt;/a&gt; and the ASCII text based ones are very easy to parse and rewrite. For what you want to do, the algorithm would go:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;parse the file once, storing an array of node values, and the element-node mappings.&lt;/li&gt;&#xA;&lt;li&gt;find the angles of each node from the centre of your polygon. &lt;/li&gt;&#xA;&lt;li&gt;Generate an integer array of the ordered node indices&lt;/li&gt;&#xA;&lt;li&gt;output a new file, outputting the node list in the new order and applying your index mapping to the element list.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This isn't totally trivial, but if you know a scripting language, wouldn't take forever to write.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastEditorUserId="35788" LastEditDate="2020-05-01T18:10:58.793" LastActivityDate="2020-05-01T18:10:58.793" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31061" PostTypeId="1" CreationDate="2019-02-15T15:14:35.170" Score="0" ViewCount="1808" Body="&lt;p&gt;I have the following code which takes a DataFrame and plot the pdist matrix. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.spatial.distance import squareform, pdist&#xA;res = pdist(df, 'euclidean')&#xA;df1 = pd.DataFrame(squareform(res), index=df.index, columns= df.index)&#xA;plt.imshow(df1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TyCvO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TyCvO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to reorder the columns such that the adjacency matrix will cluster together rows/columns with higher interactions. To make it look something like &lt;a href=&quot;http://sociograph.blogspot.com/2012/11/visualizing-adjacency-matrices-in-python.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y61vZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y61vZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23048" LastEditorUserId="23048" LastEditDate="2019-02-16T01:46:36.720" LastActivityDate="2021-04-22T13:08:08.173" Title="How to reorder/cluster adjacency matrix to maximize the interaction along the super diagonal?" Tags="|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31062" PostTypeId="2" ParentId="31061" CreationDate="2019-02-15T15:35:37.053" Score="2" Body="&lt;p&gt;As far as I understand, you are looking for a family of algorithms that are used to reorder sparse matrices. Usually, it is used to reduce fill-in during sparse factorization; however, it's certainly not the only use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first candidate would be &lt;a href=&quot;https://en.wikipedia.org/wiki/Cuthill%E2%80%93McKee_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;(reverse) Cuthill-McKee algorithm&lt;/a&gt;. Also, take a look at &lt;a href=&quot;https://www.mathworks.com/help/matlab/math/sparse-matrix-reordering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matlab sparse matrix reordering page&lt;/a&gt; that would demonstrate the work of &quot;column-count&quot; and &quot;minimum degree reorderings&quot;. Many others exist. Also, &lt;a href=&quot;https://scicomp.stackexchange.com/q/5666/20688&quot;&gt;this question on SciComp&lt;/a&gt; might be useful. The particular choice of strategy will depend on its intended use and requirements on the reordering phase heaviness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice, that the picture you've shown as an example is produced by sorting the adjacency matrix using additional information about the matrix entries in order to create the block-wise structure. The algorithms I listed use only the matrix itself.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-02-15T15:35:37.053" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="31064" PostTypeId="1" CreationDate="2019-02-15T20:45:53.143" Score="2" ViewCount="42" Body="&lt;p&gt;It seems that both &lt;a href=&quot;https://en.wikipedia.org/wiki/Piecewise_linear_continuation&quot; rel=&quot;nofollow noreferrer&quot;&gt;piecewise-linear continuation&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Marching_squares&quot; rel=&quot;nofollow noreferrer&quot;&gt;marching squares&lt;/a&gt; are methods to produce iso-contours of a scalar function given the function's values on a grid. It seems that piecwise-linear continuation works for any triangular mesh, not just a grid.  Marching squares also has a variation which works on triangles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the connection between these methods?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;They both seem to use linear interpolation to position the contour at the edges of the grid. Are the two methods equivalent under some conditions?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1217" LastEditorUserId="9667" LastEditDate="2019-12-01T15:33:42.677" LastActivityDate="2019-12-01T15:33:42.677" Title="Piecewise-linear Continuations vs Marching Squares/Cubes" Tags="|algorithms|visualization|rootfinding|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31066" PostTypeId="1" CreationDate="2019-02-16T05:07:22.847" Score="1" ViewCount="29" Body="&lt;p&gt;In many quadrature problems, we are interested in computing &lt;span class=&quot;math-container&quot;&gt;$\int_a^b f(x) \, \mathrm{d}x$&lt;/span&gt; via a quadrature sum. However, most software packages precompute the quadrature nodes and weights for use with integrals on &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt; (Gaussian, Chebyshev and tanh-sinh quadratures are examples). Hence it is common to use a variable transformation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int_{a}^{b} f(x) \, \mathrm{d}x = \frac{(b-a)}{2} \int_{-1}^{1} f(((b-a)x+(b+a))/2) \, \mathrm{d}x&#xA;$$&lt;/span&gt;&#xA;The quadrature nodes are mapped from &lt;span class=&quot;math-container&quot;&gt;$x_{i} \in (-1,1) \mapsto t_{i} := ((b-a)x+(b+a))/2$&lt;/span&gt;. However, in floating point arithmetic, we can have &lt;span class=&quot;math-container&quot;&gt;$x_0 &amp;gt; -1$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt; bitwise equal to &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is singular at &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, then this is catastrophic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to give a useful characterization of when interior nodes in the &lt;span class=&quot;math-container&quot;&gt;$(-1,1)$&lt;/span&gt; domain will be squashed into boundary in the &lt;span class=&quot;math-container&quot;&gt;$[a,b]$&lt;/span&gt; domain using this transformation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a well-known method of mitigating this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;: If &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is singular at &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, I've been using &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int_{\mathrm{floatafter}(a)}^{b} f \, \mathrm{d}x,&#xA;$$&lt;/span&gt;&#xA;but certainly this is not always necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The goal here is for a quadrature routine to do the best with what it is given. I also assume the quadrature is adaptive.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2019-02-16T06:47:36.007" LastActivityDate="2019-02-16T06:47:36.007" Title="Domain transformation squashing interior quadrature nodes into boundary" Tags="|quadrature|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31067" PostTypeId="2" ParentId="31066" CreationDate="2019-02-16T05:56:59.857" Score="2" Body="&lt;p&gt;In short:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, the vanilla quadrature-based integration is used for well-behaved functions that do not have singularities on the integration domain. While some functions could sometimes still be integrated with that method relatively well, the singularity requires special treatment. When you are integrating a singular function, having floating point error for &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; in your notation is the least of your problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you have a singularity at &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;, but you have to directly evaluate the function at the &quot;floating point distance&quot; close to it – you already have a problem. There is a lot of literature devoted to singularity treatment of different types for various applications.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now, regarding the singularity treatment in more detail. The usual suspects:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Extract the singularity and split the integral into two parts: singular and non-singular. The non-singular part can be integrated to the desired precision using standard quadratures, and non-singular part evaluated analytically.&lt;/li&gt;&#xA;&lt;li&gt;Alternatively, one can use specially developed quadrature rules for particular singular integrals. The nicest reference I have and commonly use is &lt;a href=&quot;https://doi.org/10.1137/0733048&quot; rel=&quot;nofollow noreferrer&quot;&gt;J. H. Ma, V. Rokhlin, and S. Wandzura, &quot;Generalized Gaussian quadrature rules for systems of arbitrary functions,&quot; &lt;em&gt;SIAM J. Numer. Anal.&lt;/em&gt;, vol. 33, no. 3, pp. 971–996, Jun. 1996.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Adaptive integration. The brute-force approach is usually the easiest to implement and often gives decent accuracy. Unfortunately, for highly singular functions it might never converge and is commonly way slower compared to the correctly formulated and implemented previous approaches.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since I am from computational electromagnetics community, my examples and references are also from this area that is quite rich in singularities.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Example of the first approach in 2-D (common for computational electromagnetics):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int\limits_{C}H_0^{(2)}(k|\vec{\rho}-\vec{\rho}^\prime|)dl^\prime&#xA;\tag{1}&#xA;$$&lt;/span&gt;&#xA;In (1), we want to integrate the Hankel function of zero order and second kind over line segment &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the wavenumber (for the purpose of this discussion can be considered as some constant), &lt;span class=&quot;math-container&quot;&gt;$\vec{\rho}$&lt;/span&gt; is the constant position-vector to some observation point and &lt;span class=&quot;math-container&quot;&gt;$\vec{\rho}^\prime$&lt;/span&gt; is the position vector on the line &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; varying with &lt;span class=&quot;math-container&quot;&gt;$dl\prime$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, if &lt;span class=&quot;math-container&quot;&gt;$\vec{\rho}$&lt;/span&gt; is far away from the contour &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, Hankel function behaves nicely; however, if &lt;span class=&quot;math-container&quot;&gt;$|\vec{\rho}-\vec{\rho}^\prime|\to0$&lt;/span&gt; we have a singularity. The common treatment for this problem is to apply the small argument expansion (2) to &lt;span class=&quot;math-container&quot;&gt;$H_0^{(2)}$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;H_0^{(2)}(k\underbrace{|\vec{\rho}-\vec{\rho}^\prime|}_{\to0})\to1-\frac{2j}{\pi}\left(\ln\frac{k|\vec{\rho}-\vec{\rho}^\prime|}{2}+\gamma\right)&#xA;\tag{2}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; is the imaginary unity and &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is Euler's constant. Now we can subtract expansion (2) from (1) and arrive to an expression that is ready for the numerical evaluation even if the observation point &lt;span class=&quot;math-container&quot;&gt;$\vec{\rho}$&lt;/span&gt; lies on the integration contour &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\int\limits_{C}H_0^{(2)}(k|\vec{\rho}-\vec{\rho}^\prime|)dl^\prime=\int\limits_{C}\underbrace{\left[H_0^{(2)}(k|\vec{\rho}-\vec{\rho}^\prime|)+\frac{2j}{\pi}\ln|\vec{\rho}-\vec{\rho}^\prime|\right]}_\text{*, non-singular}dl^\prime-\frac{2j}{\pi}\underbrace{\int\limits_C\ln|\vec{\rho}-\vec{\rho}^\prime|dl^\prime}_\text{**, singular}&#xA;\tag{3}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In (3), the non-singular part (*) can be evaluated using regular quadrature, while for the singular logarithmic part the integral is evaluated analytically (the expression for analytic log-integration as well as the presented example is available from &lt;a href=&quot;https://doi.org/10.1109/TAP.1984.1143304&quot; rel=&quot;nofollow noreferrer&quot;&gt;D. Wilton &lt;em&gt;et al.&lt;/em&gt;, &quot;Potential integrals for uniform and linear source distributions on polygonal and polyhedral domains,&quot; &lt;em&gt;IEEE Trans. Antennas Propag.&lt;/em&gt;, vol. 32, no. 3, pp. 276–281, Mar. 1984.&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in many cases integration of (1) is possible using standard quadratures even for observation points &lt;span class=&quot;math-container&quot;&gt;$\vec{\rho}$&lt;/span&gt; lying exactly on the line segment &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. But one would certainly be required to use an expensive quadrature and suffer for the convergence.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-02-16T05:56:59.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31069" PostTypeId="1" AcceptedAnswerId="31076" CreationDate="2019-02-16T11:02:42.867" Score="2" ViewCount="1634" Body="&lt;p&gt;Let &lt;code&gt;v_{0},...,v_{N-1}&lt;/code&gt; be N points in a Cartesian xy plane defining the vertices of closed polygon (i.e. v_{N} = v_{0}).&#xA;Let &lt;code&gt;P_{0}&lt;/code&gt; a point lying inside the polygon, point that is not known &lt;em&gt;a priori&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a robust algorithm to sort the list &lt;code&gt;v_{0},...,v_{N-1}&lt;/code&gt; clockwise/counterclockwise in respect of &lt;code&gt;P_{0}&lt;/code&gt;. &#xA;I know that, &lt;strong&gt;if the polygon is convex&lt;/strong&gt;, the following algorithm works well:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;compute &lt;code&gt;P_{0}&lt;/code&gt; as the mean of &lt;code&gt;v_{0},...,v_{N-1}&lt;/code&gt;;&lt;/li&gt;&#xA;&lt;li&gt;compute the angle defined by &lt;code&gt;atan2 = (y_i-y_0/x_i-x_0)&lt;/code&gt; for every vertex and save it into an array;&lt;/li&gt;&#xA;&lt;li&gt;sort the vertices using the array of the angle as the key.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am wondering if exist an algorithm able to deal with all kind of polygon, also &lt;strong&gt;non-convex&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="26978" LastEditorUserId="26978" LastEditDate="2019-02-17T20:28:19.233" LastActivityDate="2019-02-17T20:53:20.113" Title="A robust algorithm to sort a non-convex polygon vertices" Tags="|computational-geometry|fortran|nonconvex|sorting|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31070" PostTypeId="1" AcceptedAnswerId="31073" CreationDate="2019-02-16T19:58:28.053" Score="1" ViewCount="94" Body="&lt;p&gt;I am recently dealing with a diffusion simulation project and I have come up with the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;N = 10000000&#xA;num_steps = 100&#xA;dim = 3&#xA;&#xA;particles2 = npr.uniform(-1, 1, (N, num_steps, dim))&#xA;particles = np.cumsum(particles2, axis=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To briefly explain: the first 3 lines are the simulation parameters; the number of particles (&lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;), the number of simulation timesteps (&lt;code&gt;num_step&lt;/code&gt;) and the number of dimensions (&lt;code&gt;dim&lt;/code&gt;) of the simulation. I am simulating diffusion by considering each particle as a random walker, so I used the &lt;code&gt;numpy.random&lt;/code&gt; module to generate a random translation vector for each timestep, such that each coordinate is simply a random number between &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Now, in the &lt;code&gt;particles2&lt;/code&gt; line, I generate a tensor filled with random numbers between &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and then in the next line use cumulative sum to sum just over different timesteps. This is exactly the same as the following (more intuitive) code, but much, much more efficient:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;particles = np.zeros((N, num_steps, dim))&#xA;&#xA;for i in range(N):  &#xA;    prev_vec = np.zeros((dim))&#xA;&#xA;    for t in range(num_steps):&#xA;        trans_vec = npr.uniform(-1, 1, dim)&#xA;        particles[i, t, :] = prev_vec&#xA;        prev_vec = prev_vec + trans_vec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;prev_vec&lt;/code&gt; stands for &quot;previous vector&quot; and &lt;code&gt;trans_vec&lt;/code&gt; stands for the random translation vector of each timestep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a problem, however - the memory usage. Since I am storing my entire trajectory for each particle, I (relatively) quickly hit the available memory cap. For the analysis, I only need the last timestep, so I don't really need the whole trajectory. I see how to get around this in the bottom code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;particles = np.zeros((N, dim))&#xA;&#xA;for i in range(N):&#xA;    prev_vec = np.zeros((dim))&#xA;&#xA;    for t in range(num_steps):&#xA;        trans_vec = npr.uniform(-1, 1, dim)&#xA;        prev_vec = prev_vec + trans_vec&#xA;&#xA;    particles[i,:] = prev_vec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I don't know how I would get around this in the above, a lot more efficient code. I would like to do this since the 2 for-loops are really making the calculation slow.&lt;/p&gt;&#xA;" OwnerUserId="29843" LastEditorUserId="20688" LastEditDate="2019-02-16T21:41:09.343" LastActivityDate="2019-02-16T22:09:12.923" Title="Minimizing the used memory in diffusion simulation using Python" Tags="|python|numpy|diffusion|memory-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31071" PostTypeId="1" AcceptedAnswerId="31075" CreationDate="2019-02-16T21:53:36.407" Score="6" ViewCount="284" Body="&lt;p&gt;In &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;, equation 1.6a, Higham gives the following update formula for the mean:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;M_{1} := x_1, \quad&#xA;M_{k+1} := M_{k} + \frac{x_k - M_k}{k}&#xA;$$&lt;/span&gt;&#xA;Ok, one benefit of this update is that it doesn't overflow, which is a potential problem when we naively compute &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=1}^{k} x_k$&lt;/span&gt; and then divide by &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; before any use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the naive sum requires &lt;span class=&quot;math-container&quot;&gt;$N-1$&lt;/span&gt; adds and 1 division, and potentially vectorizes very nicely, whereas the update proposed by Higham requires &lt;span class=&quot;math-container&quot;&gt;$2N-2$&lt;/span&gt; additions and &lt;span class=&quot;math-container&quot;&gt;$N-1$&lt;/span&gt; divisions, and I'm unaware of any assembly instruction that can vectorize all this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it Higham's update formula worth using? Are there benefits I'm not seeing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;: Higham gives the following generic advice (Section 1.18 &quot;Designing Stable Algorithms):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;It is advantageous to express update formulae as &lt;code&gt;newvalue = oldvalue + &#xA;  smallcorrection&lt;/code&gt; if the small correction can be computed with many correct significant figures.&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Update 1.6a does take this form, but it's not clear to me that the small correction can be given to many significant figures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit&lt;/em&gt;: I found an empirical study of the performance of various methods of computing means, and 1.6a came highly recommended; see&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://doi.org/10.1080/01621459.1974.10480219&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robert F. Ling (1974) Comparison of Several Algorithms for Computing Sample Means and Variances, Journal of the American Statistical Association, 69:348, 859-866, DOI: 10.1080/01621459.1974.10480219&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it still wasn't clear to me after reading that paper that the update was worth the price; and in any case, I was hoping for a worst and average case bound by accumulating rounding errors.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="20688" LastEditDate="2019-02-17T20:04:28.920" LastActivityDate="2019-02-17T20:32:26.800" Title="Is Highams' computation of mean worth the price?" Tags="|floating-point|numerics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31073" PostTypeId="2" ParentId="31070" CreationDate="2019-02-16T22:09:12.923" Score="1" Body="&lt;p&gt;It seems that you are going from one extreme to the other: you probably want to generate all &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; particles at once without the for-loop; however, you don't want to generate all the &lt;code&gt;num_steps&lt;/code&gt; at once since you only need two last ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I think you are looking for something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import numpy.random as npr&#xA;&#xA;N = 10000000&#xA;dim = 3&#xA;num_steps = 100&#xA;&#xA;# First, generating N initial particle positions (or you can initialize them to zeros)&#xA;particles = npr.uniform(-1, 1, (N, dim)) &#xA;&#xA;# start looping over time&#xA;for t in range(num_steps):&#xA;    # store the previous particles position&#xA;    prev_particles = particles&#xA;    # overwrite positions with the addition of random translation (no additional storage!)&#xA;    particles = np.add(particles,npr.uniform(-1,1,(N, dim)))  &#xA;    # notice, I could have saved the translation in the following way (commented out)&#xA;    # trans_vec = npr.uniform(-1,1,(N, dim)); particles = np.add(particles,trans_vec)&#xA;    # but that would have required additional storage.&#xA;    #   &#xA;    # do whatever you need with particles and prev_particles that store positions at the two preceding time_steps.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I might have a little bit misunderstood what goes where inside your code, but with that structure, you are doing the for-loop only for the time-dimension, where you need a lot of steps and you want to get the memory savings by getting rid of things you don't need to be storing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code above runs on my machine in less than a minute consuming less than 500 MB of storage (which makes sense, I am effectively storing 2 real vectors of the size &lt;span class=&quot;math-container&quot;&gt;$10\cdot10^6\times 1$&lt;/span&gt; which would result in around 160 MB  of storage at least + whatever overhead Python adds).&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-02-16T22:09:12.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31074" PostTypeId="2" ParentId="31071" CreationDate="2019-02-17T00:37:49.513" Score="3" Body="&lt;p&gt;Suppose you have &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; numbers and the vector length &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; divides &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, i.e., &lt;span class=&quot;math-container&quot;&gt;$m=ln$&lt;/span&gt;. The you can apply Higham's scheme to each group of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; numbers. This vectorizes nicely, especially if the numbers are interleaved, i.e., all the first elements are contiguous in memory, followed by all the second elements, etc. When you have computed the average of each group you have &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; numbers left. You apply a sequential implementation of Higham's scheme and then you are done. If the vector length does not divide &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, then pad the data with zeros, split the data into &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; groups with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; numbers each and proceed as before. When you have the average &lt;span class=&quot;math-container&quot;&gt;$\mu_j$&lt;/span&gt; of each group, you compute &lt;span class=&quot;math-container&quot;&gt;$(n/m)\mu_j$&lt;/span&gt; and simply add these numbers using the naive summation algorithm.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Whether Higham's formula is worth using depends on the situation. When overflow is an issue, then the naive formula is useless. Both formulas have low arithmetic intensity, and I expect Higham's formula to run very slowly simply because the division has very high latency. Personally, I find that speed is &lt;em&gt;very&lt;/em&gt; secondary compared with reliability and accuracy.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Higham's primary goal in this very first chapter is simply to alert the reader to the fact that computing with floating point numbers is profoundly different from real arithmetic. His advice about corrections is quoted correctly, but I would have written it a bit differently. Specifically, it does not matter if the correction is computed with a large relative error as long as the correction is small relative to the original value.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="-1" LastEditDate="2019-02-17T20:32:26.800" LastActivityDate="2019-02-17T20:32:26.800" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31075" PostTypeId="2" ParentId="31071" CreationDate="2019-02-17T01:35:56.977" Score="3" Body="&lt;p&gt;Higham's algorithm seems very useful for online algorithms or algorithms in which you have limited storage capacity such as edge processing. In these situations it is probably always worth the cost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, to address somewhat your question as posed, I implemented an SSE2 version which I think captures your question:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;chrono&amp;gt;&#xA;#include &amp;lt;cstdlib&amp;gt;&#xA;#include &amp;lt;iomanip&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;xmmintrin.h&amp;gt;&#xA;&#xA;int main(){&#xA;  const int num_count = 100000;&#xA;&#xA;  alignas(16) float data[num_count];&#xA;  for(int i=0;i&amp;lt;num_count;i++)&#xA;    data[i] = rand()%100000+rand()/(double)RAND_MAX;&#xA;&#xA;&#xA;  {&#xA;    const auto t1 = std::chrono::high_resolution_clock::now();&#xA;    float sum=0;&#xA;    for(int i=0;i&amp;lt;num_count;i++){&#xA;      sum += data[i];&#xA;    }&#xA;    const float mean=sum/num_count;&#xA;&#xA;    const auto t2 = std::chrono::high_resolution_clock::now();&#xA;    const auto time_span1 = std::chrono::duration_cast&amp;lt;std::chrono::duration&amp;lt;double&amp;gt;&amp;gt;(t2 - t1);&#xA;    std::cout &amp;lt;&amp;lt; &quot;Exec time:\t&quot; &amp;lt;&amp;lt; time_span1.count() &amp;lt;&amp;lt; &quot; s\n&quot;;&#xA;    std::cout &amp;lt;&amp;lt; &quot;Mean = &quot;&amp;lt;&amp;lt;std::setprecision(20)&amp;lt;&amp;lt;mean&amp;lt;&amp;lt;std::endl;&#xA;  }&#xA;&#xA;  {&#xA;    const auto t1 = std::chrono::high_resolution_clock::now();&#xA;    __m128 mean = _mm_load_ps(&amp;amp;data[0]);&#xA;    for (int i=4;i&amp;lt;num_count;i+=4){&#xA;      const __m128 x    = _mm_load_ps(&amp;amp;data[i]);&#xA;      const __m128 diff = _mm_sub_ps(x,mean);&#xA;      const __m128 k    = _mm_set1_ps(i/4);&#xA;      const __m128 div  = _mm_div_ps(diff, k);&#xA;      mean              = _mm_add_ps(mean, div);&#xA;    }&#xA;    float result[4];&#xA;    _mm_store_ps(result, mean);&#xA;    const float tmean = (result[0] + result[1] + result[2] + result[3])/4; //I'm suspicious about this step: probably throws away precision&#xA;    const auto t2 = std::chrono::high_resolution_clock::now();&#xA;    const auto time_span1 = std::chrono::duration_cast&amp;lt;std::chrono::duration&amp;lt;double&amp;gt;&amp;gt;(t2 - t1);&#xA;    std::cout &amp;lt;&amp;lt; &quot;Exec time:\t&quot; &amp;lt;&amp;lt; time_span1.count() &amp;lt;&amp;lt; &quot; s\n&quot;;&#xA;    std::cout &amp;lt;&amp;lt; &quot;Mean = &quot;&amp;lt;&amp;lt;std::setprecision(20)&amp;lt;&amp;lt;tmean&amp;lt;&amp;lt;std::endl;&#xA;  }&#xA;&#xA;&#xA;}&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and observe that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Exec time:  0.000225851 s&#xA;Simple Mean = 49891.23046875&#xA;Exec time:  0.0003759360000000000002 s&#xA;Higham Mean = 49890.26171875&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Higham's mean takes a little longer to calculate and the values differ by what might be a non-negligible amount, though the accuracy you need really depends on your implementation.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2019-02-17T01:35:56.977" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31076" PostTypeId="2" ParentId="31069" CreationDate="2019-02-17T20:53:20.113" Score="4" Body="&lt;p&gt;As @HighPerformanceMark mentioned in the comments if the set of vertices &lt;span class=&quot;math-container&quot;&gt;$V=\{v_0,\ldots,v_{N-1}\}$&lt;/span&gt; is the only information you have about your &lt;em&gt;non-convex polygon&lt;/em&gt;, you did not define it uniquely. One can find &lt;strong&gt;a&lt;/strong&gt; non-convex polygon defined by &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;, but not &lt;strong&gt;the&lt;/strong&gt; one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below, is the example (certainly non-minimal, but that is the first I drew) of two different six-sided polygons defined by exactly the same set of vertices. In polygon &lt;span class=&quot;math-container&quot;&gt;$S_A$&lt;/span&gt;, the vertices are ordered counter-clockwise, while &lt;span class=&quot;math-container&quot;&gt;$S_B$&lt;/span&gt; is not so lucky.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7WBSB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7WBSB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in order for an algorithm to work, you will either get some non-convex polygon (in this example, &lt;span class=&quot;math-container&quot;&gt;$P_0\in S_A, S_B$&lt;/span&gt;; however, &lt;span class=&quot;math-container&quot;&gt;$P_1\in S_B$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$P_1\notin S_A$&lt;/span&gt;), but I can hardly imagine an application that allows for non-uniqueness. Or provide some additional information about your &quot;desired&quot; non-convex polygon – and list of edges come to mind right away. When the edge list is present, you probably don't need that algorithm in the first place, or the reordering becomes trivial by traversing the edge list.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-02-17T20:53:20.113" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31081" PostTypeId="1" CreationDate="2019-02-18T17:55:19.113" Score="4" ViewCount="6919" Body="&lt;p&gt;I've been running various simulations with C++, and doing so has often involved saving lots of data to file (real/complex matrices, arrays, etc) and then reading them into other programs later. However, I have been unable to figure out a consistent and simple way of doing so. My solutions often involve several dozen lines of difficult-to-read code for each different kind of data structure that I want to save/read from a file. Are there any known libraries that can do this, or at least aid in the process? If there are any sort of matrix libraries that allow automatic saving/reading of matrices/vectors to file, that would also be helpful (Eigen does not have this option, as far as I can tell).&lt;/p&gt;&#xA;" OwnerUserId="28245" LastActivityDate="2019-02-20T10:19:05.487" Title="Simple way to store/read data from file in C++" Tags="|c++|data-management|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31083" PostTypeId="1" CreationDate="2019-02-18T20:22:56.287" Score="1" ViewCount="42" Body="&lt;p&gt;This is a particular case of a &lt;a href=&quot;https://math.stackexchange.com/q/3117513/660&quot;&gt;question&lt;/a&gt; I asked on Mathematics Stackexchange, question which got no answer so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; be integers with &lt;span class=&quot;math-container&quot;&gt;$n\ge1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$k\ge0$&lt;/span&gt;, and let &lt;span class=&quot;math-container&quot;&gt;$a(n,k)$&lt;/span&gt; be the number of orbits of the symmetric group &lt;span class=&quot;math-container&quot;&gt;$S_n$&lt;/span&gt; on the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-th iterated power set &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P^k(\{1,\dots,n\})&#xA;$$&lt;/span&gt; &#xA;of the set &lt;span class=&quot;math-container&quot;&gt;$\{1,\dots,n\}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have &lt;span class=&quot;math-container&quot;&gt;$a(n,0)=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a(n,1)=n+1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a(1,k+1)=2^{a(1,k)}$&lt;/span&gt; and &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{b(n,k)}{n!}\le a(n,k)\le b(n,k),&#xA;$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$b(n,k)$&lt;/span&gt; is the cardinality of &lt;span class=&quot;math-container&quot;&gt;$P^k(\{1,\dots,n\})$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also have &lt;span class=&quot;math-container&quot;&gt;$a(2,2)=12$&lt;/span&gt;. Indeed, the twelve orbits of the two-element group &lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$P^2(\{1,2\})$&lt;/span&gt; can be described as follows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The set &lt;span class=&quot;math-container&quot;&gt;$P^2(\{1,2\})$&lt;/span&gt; having sixteen elements, it suffices to give the eight fixed points. The complement of a fixed point being a fixed point, it suffices to give four fixed points &lt;span class=&quot;math-container&quot;&gt;$F_1,F_2,F_3,F_4$&lt;/span&gt; such that, for all &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt;, the subset &lt;span class=&quot;math-container&quot;&gt;$F_i$&lt;/span&gt; is not the complement of &lt;span class=&quot;math-container&quot;&gt;$F_j$&lt;/span&gt;. A possible choice of the &lt;span class=&quot;math-container&quot;&gt;$F_i$&lt;/span&gt; is &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\varnothing,\ \{\varnothing\},\ \{\{1,2\}\},\ \{\varnothing,\{1,2\}\}.&#xA;$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can anybody compute, say, &lt;span class=&quot;math-container&quot;&gt;$a(2,3),a(3,2)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a(3,3)$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you don't feel like computing these three integers, I'd be most grateful if you could nevertheless explain to me a simple (perhaps not efficient) way of computing them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sadly, I know nothing about programming languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1.&lt;/strong&gt; Here are more details about the equality &lt;span class=&quot;math-container&quot;&gt;$a(2,2)=12$&lt;/span&gt;. We have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(\{1,2\})=\{\varnothing,\{1\},\{2\},\{1,2\}\}.&#xA;$$&lt;/span&gt; &#xA;So &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P^2(\{1,2\})=P(\{\varnothing,\{1\},\{2\},\{1,2\}\})&#xA;$$&lt;/span&gt; &#xA;has &lt;span class=&quot;math-container&quot;&gt;$16$&lt;/span&gt; elements, one of cardinality &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (that is, with zero elements), which is the empty set &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\varnothing,&#xA;$$&lt;/span&gt; &#xA;four of cardinality &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, which are &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\{\varnothing\},\{\{1\}\},\{\{2\}\},\{\{1,2\}\},&#xA;$$&lt;/span&gt; &#xA;six of cardinality &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;, which are &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\{\varnothing,\{1\}\}, \{\varnothing,\{2\}\}, \{\varnothing,\{1,2\}\},&#xA;$$&lt;/span&gt; &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\{\{1\},\{2\}\}, \{\{1\},\{\{1,2\}\}, \{2\},\{\{1,2\}\}&#xA;$$&lt;/span&gt; &#xA;four of cardinality &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;, which are &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\{\varnothing\}^c,\{\{1\}\}^c,\{\{2\}\}^c,\{\{1,2\}\}^c,&#xA;$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$A^c$&lt;/span&gt; denotes the complement of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$P(\{1,2\})$&lt;/span&gt;, and one of cardinality &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;, which is &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P(\{1,2\}).&#xA;$$&lt;/span&gt; &#xA;Now let &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; be one of the sixteen above expressions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you swap the symbols &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, you get either &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; itself, or a different expression &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, and if you perform the same operation &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, you get &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; back. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the non-invariant expressions (the &quot;non-fixed points&quot;) come in pairs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By inspection one sees that there are &lt;span class=&quot;math-container&quot;&gt;$8$&lt;/span&gt; invariant expressions and &lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt; pairs of non-invariant expressions, that is &lt;span class=&quot;math-container&quot;&gt;$12$&lt;/span&gt; &quot;orbits&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2.&lt;/strong&gt; An inductive formula for &lt;span class=&quot;math-container&quot;&gt;$a(2,k)$&lt;/span&gt; is given in an edit to this Mathematics Stackexchange &lt;a href=&quot;https://math.stackexchange.com/q/3117513/660&quot;&gt;post&lt;/a&gt;, but I'm still unable to compute &lt;span class=&quot;math-container&quot;&gt;$a(3,2)$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="659" LastEditorUserId="659" LastEditDate="2019-02-19T12:22:04.690" LastActivityDate="2019-02-19T12:22:04.690" Title="Number of $S_n$-orbits in $P^k(\{1,\dots,n\})$" Tags="|algorithms|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31084" PostTypeId="1" CreationDate="2019-02-18T21:43:45.857" Score="0" ViewCount="70" Body="&lt;p&gt;Hello I am in a basic numerical methods class and our teacher has given us an algorithm which can compute the inverse of a matrix other than using MATLAB's built in library function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = [1 2 4; 1 3 9; 1 4 16];&#xA;b = [7; 13; 21];&#xA;[L, U, P] = lu(A);&#xA;n = 3;&#xA;X = zeros(n,n);&#xA;for i=1:n&#xA;   e = zeros(n,1); &#xA;   e(i) = 1;&#xA;    X(:,i) = U\(L\(P*e))&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am a bit new to MATLAB, so I Am a bit unsure as to what this code is actually doing to find the inverse. I naively set n = 3 because it was the only value i could find to get the algorithm to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I know this algorithm finds the LU factorization of A then uses the \ operator repeatedly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what I do not understand is what &lt;strong&gt;x = zeros(n,n)&lt;/strong&gt; what is the point of this line as well as the inner for loop&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;e = zeros(n,1); //creates a 3 x 1 matrix of zeroes?&#xA;e(i) = 1;       //No idea what this line is doing&#xA;X(:,i) = U\(L\(P*e)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As for the last line of the for loop, This is the repeated &quot;\&quot; operation but I have no idea what the X(:,i) stands for as well as well using &quot;\&quot; by P*e&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone help me understand this algorithm better? Because I am only understanding fragments of it at the moment.&lt;/p&gt;&#xA;" OwnerUserId="30020" LastEditorUserId="9667" LastEditDate="2019-02-18T21:55:39.983" LastActivityDate="2019-02-19T17:17:54.977" Title="Finding Matrix inverse with LU and repeted left division calls" Tags="|linear-algebra|matlab|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31086" PostTypeId="2" ParentId="31084" CreationDate="2019-02-19T00:15:34.797" Score="4" Body="&lt;p&gt;The inverse of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the solution to the linear system of equations  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$AX=I$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we write &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; in terms of its columns as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X=\left[ x^{(1)}, x^{(2)}, \ldots, x^{(n)} \right]$&lt;/span&gt;,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;then we can break up &lt;span class=&quot;math-container&quot;&gt;$AX=I$&lt;/span&gt; into &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; separate systems of equations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Ax^{(1)}=\left[ \begin{array}{c}&#xA;1 \\&#xA;0 \\&#xA;\vdots \\&#xA;0&#xA;\end{array} \right]$&lt;/span&gt;,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Ax^{(2)}=\left[ \begin{array}{c}&#xA;0 \\&#xA;1 \\&#xA;\vdots \\&#xA;0&#xA;\end{array} \right]$&lt;/span&gt;,&#xA;&lt;span class=&quot;math-container&quot;&gt;$\vdots$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Ax^{(n)}=\left[ \begin{array}{c}&#xA;0 \\&#xA;0 \\&#xA;\vdots \\&#xA;0 \\&#xA;1&#xA;\end{array} \right]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common notation for a vector of all zeros except for a 1 in the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th position is &lt;span class=&quot;math-container&quot;&gt;$e^{(i)}$&lt;/span&gt;.  With that notation, these systems are &lt;span class=&quot;math-container&quot;&gt;$Ax^{(i)}=e^{(i)}$&lt;/span&gt;, for &lt;span class=&quot;math-container&quot;&gt;$i=1, 2, \ldots, n$&lt;/span&gt;. In MATLAB you can create a vector &lt;span class=&quot;math-container&quot;&gt;$e^{(i)}$&lt;/span&gt; with &lt;code&gt;e=zeros(n,1); e(i)=1&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code you were given solves these &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; linear systems of equations using the LU factorization of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The LU factorization (with pivoting) is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$PA=LU$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to solve &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt;, multiply by &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; on both sides of the equation to get &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$PAx=Pb$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$LUx=Pb$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, if we solve &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Lw=Pb$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Ux=w$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$PAx=LUx=Lw=Pb$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Ax=b$&lt;/span&gt;.  The MATLAB &lt;code&gt;\&lt;/code&gt; function is used in code to solve the triangular systems. In particular, we get &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; by &lt;code&gt;w=L\(P*b)&lt;/code&gt; and then get &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; by &lt;code&gt;x=U\(L\(P*b))&lt;/code&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="20688" LastEditDate="2019-02-19T17:17:54.977" LastActivityDate="2019-02-19T17:17:54.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31087" PostTypeId="1" CreationDate="2019-02-19T04:15:07.160" Score="0" ViewCount="39" Body="&lt;p&gt;I'm looking for an open source software package that provides a fast evaluation of continuous Daubechies/Symmlet wavelet/scaling functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GSL only has the discrete wavelets, and PyWavelets comes close to what I want, in that it gives a gives the values of the functions on an equispaced grid of whatever refinement you desire.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2019-02-19T04:15:07.160" Title="Open Source Packages Implementing Continuous Wavelet and Scaling Functions" Tags="|open-source|wavelet|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="31088" PostTypeId="2" ParentId="31081" CreationDate="2019-02-19T13:25:26.767" Score="2" Body="&lt;p&gt;You can use boost's serialization facilities, which can be straightforwardly extended to support custom data structures. If you use Eigen you can adapt something like &lt;a href=&quot;https://github.com/artivis/boost_serialization_helper/blob/master/save_load_eigen.h&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; in order to suit your needs.&lt;/p&gt;&#xA;" OwnerUserId="27040" LastActivityDate="2019-02-19T13:25:26.767" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31089" PostTypeId="1" AcceptedAnswerId="31266" CreationDate="2019-02-19T14:49:46.580" Score="1" ViewCount="77" Body="&lt;p&gt;I'm looking for papers dealing with multi-objective non-linear programming which could help me implement an algorithm to solve my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maximize &lt;span class=&quot;math-container&quot;&gt;$f(x) = c \cdot x$&lt;/span&gt;, while minimizing &lt;span class=&quot;math-container&quot;&gt;$g(x) = r \cdot x$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\cdot$&lt;/span&gt; is the scalar product.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Constrained by &lt;span class=&quot;math-container&quot;&gt;$h_1(x) = \frac{v \cdot x}{p \cdot x} \geq 0.5$&lt;/span&gt; and  &lt;span class=&quot;math-container&quot;&gt;$h_2(x) = b \cdot x = B$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that &lt;span class=&quot;math-container&quot;&gt;$b, c, p, r, v \in (\mathbb{N}^*)^n, B \in \mathbb{N}^*,  x \in [0, 1]^n$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For me, &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; will be in the order of &lt;span class=&quot;math-container&quot;&gt;$1000$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there is no unique solution so I'm looking for the set of Pareto optimal solutions, therefore I am not interested in scalarizing. I could use interactive methods though.&lt;/p&gt;&#xA;" OwnerUserId="30024" LastEditorUserId="20688" LastEditDate="2019-02-21T20:14:40.267" LastActivityDate="2019-07-08T09:34:22.503" Title="Research articles on MultiObjective Non-Linear Programming (MONLP)" Tags="|constrained-optimization|reference-request|nonlinear-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31090" PostTypeId="1" CreationDate="2019-02-19T15:33:22.610" Score="1" ViewCount="61" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;Problem Background&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to find a solution/model to the following problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's consider a cellular network (mobile network, ie., hexagonal cells) denoted &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; composed of &lt;span class=&quot;math-container&quot;&gt;$|N|$&lt;/span&gt; cells. Each cell &lt;span class=&quot;math-container&quot;&gt;$i \in N$&lt;/span&gt; has a resources capacity &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$D_{i,j}$&lt;/span&gt; denote the distance between the centers of two adjacent cells &lt;span class=&quot;math-container&quot;&gt;$i \in N$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j \in N$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, let &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; be a set of vehicles composed of &lt;span class=&quot;math-container&quot;&gt;$|V|$&lt;/span&gt; vehicles. Each vehicle &lt;span class=&quot;math-container&quot;&gt;$v \in V$&lt;/span&gt; has a resources demand &lt;span class=&quot;math-container&quot;&gt;$r_v$&lt;/span&gt;. Furthermore, at the beginning, each vehicle &lt;span class=&quot;math-container&quot;&gt;$v \in V$&lt;/span&gt; is located in a starting cell &lt;span class=&quot;math-container&quot;&gt;$S_v \in N$&lt;/span&gt; and has a mission to complete, i.e., traveling to a destination cell &lt;span class=&quot;math-container&quot;&gt;$D_v \in N$&lt;/span&gt; via some intermediate cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mission of the vehicle &lt;span class=&quot;math-container&quot;&gt;$v \in V$&lt;/span&gt; must start at a predefined time denoted &lt;span class=&quot;math-container&quot;&gt;$T^v_S$&lt;/span&gt; and must end no later than the mission complete time denoted &lt;span class=&quot;math-container&quot;&gt;$T^v_E$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a vehicle &lt;span class=&quot;math-container&quot;&gt;$v \in V$&lt;/span&gt; passes through a cell &lt;span class=&quot;math-container&quot;&gt;$i \in N$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r_v$&lt;/span&gt; resources will be reserved from the &lt;span class=&quot;math-container&quot;&gt;$R_i$&lt;/span&gt; resources of that cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When a vehicle &lt;span class=&quot;math-container&quot;&gt;$v \in V$&lt;/span&gt; leaves a cell &lt;span class=&quot;math-container&quot;&gt;$i \in N$&lt;/span&gt;, the &lt;span class=&quot;math-container&quot;&gt;$r_v$&lt;/span&gt; resources will be released.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Assumptions&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We assume that the vehicles travel between the centers of the adjacent cells.&lt;/li&gt;&#xA;&lt;li&gt;We assume that at the beginning, the vehicles are located in cells with sufficient resources.&lt;/li&gt;&#xA;&lt;li&gt;We assume that each vehicle has a maximum speed of &lt;span class=&quot;math-container&quot;&gt;$S^v_{max}$&lt;/span&gt;   &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Objective&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective is to find the shortest path from the cell  &lt;span class=&quot;math-container&quot;&gt;$S_v$&lt;/span&gt; to the cell &lt;span class=&quot;math-container&quot;&gt;$D_v$&lt;/span&gt; for all the vehicles while respecting the time and resources constraints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Question&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was exploring some vehicles routing LP models and Dynamic flow network, but none of them seems to be suitable for my problem. If anyone can help me with some related problems, models, techniques or hints to solve this problem I would be very gratefull.&lt;/p&gt;&#xA;" OwnerUserId="30027" LastActivityDate="2019-07-22T08:01:50.193" Title="Vehicle Route assignment with capacity constraint" Tags="|optimization|constrained-optimization|graph-theory|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31091" PostTypeId="1" CreationDate="2019-02-19T16:45:18.083" Score="0" ViewCount="60" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$G(V,E)$&lt;/span&gt; is a weighted simple graph, where &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; are the set of vertices and Edges. The graph is undirected. Let &lt;span class=&quot;math-container&quot;&gt;$A \in \{0,1\}^{n\times n}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W \in R_+^{n\times n}$&lt;/span&gt; be the adjacency matrix and the weight matrix of the graph. Note that both both &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; are symmetric. My target is to output a sparse representation of this graph. I am focusing on edge sparsification that means deleting edges from the graph. In particular I am concentrating on degree constraint subgraph of the original graph as a representation of sparsity. Let the new graph is &lt;span class=&quot;math-container&quot;&gt;$G_s(V,E_s)$&lt;/span&gt;, where the &lt;span class=&quot;math-container&quot;&gt;$E_s$&lt;/span&gt; is the edge set of the new graph while the vertex set remains unchanged. My goal in sparsification is to find a degree constraint subgraph &lt;span class=&quot;math-container&quot;&gt;$G_s$&lt;/span&gt; of the original graph &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, optimizing some distance criteria between them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A vertex &lt;span class=&quot;math-container&quot;&gt;$v \in V$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; can be thought of a vector, say &lt;span class=&quot;math-container&quot;&gt;$W_v$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$R_+^{|V|}$&lt;/span&gt;. Here &lt;span class=&quot;math-container&quot;&gt;$W_v$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;'th row of the matrix &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; which represents the weights of the adjacent edges of &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;. We define a binary matrix &lt;span class=&quot;math-container&quot;&gt;$X \in \{0,1\}^{n\times n}$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; would be the adjacency matrix of the new graph &lt;span class=&quot;math-container&quot;&gt;$G_s$&lt;/span&gt;. We want to solve the following optimization problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;	\text{min} \quad &#xA;    &amp;amp;\sum_{v \in V}||W_v-W_v \circ X_v||_2  \\&#xA;    \text{s.t.,}\quad &#xA;    &amp;amp;\sum_{i \in V} X_{vi} \leq b_v  \quad \forall v \in V \\&#xA;    &amp;amp;X_{vi} \leq A_{vi}  \quad \forall v \in V, \forall i \in V \\&#xA;    &amp;amp;X_{vi} = X_{iv}  \quad \forall v \in V, \forall i \in V \\&#xA;    &amp;amp;X_{vv} = 0  \quad \forall v \in V\\&#xA;    &amp;amp;X_{vi} =\{0,1\}, \quad \forall v \in V, \forall i \in V&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$(W_v-W_v\circ X_v)$&lt;/span&gt; is the difference vector between the original incident vector and the new incident vector for vertex &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;. Note that &lt;span class=&quot;math-container&quot;&gt;$W_v\circ X_v$&lt;/span&gt; is the pairwise multiplication between these two vector also known as hadamard operation. The objective function is the summation of the norm of this difference vector over all the vertices in the graph. We wish to minimize this quantity subject to several constraints. The first constraint bounds the degree of the new graph by a user input positive bound defined for each vertex. The second constraint ensure that the new graph is a subgraph of the original graph that is no new edges are created. The third and fourth constraint is necessary to output a loop free symmetric graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now what I need is the simplification of the objective function. Ideally I want to show equivalence of the objective function to something like this&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;{\rm max} \sum_{v \in V} ||W_v \circ X_v||_2&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea of how to simplify the objective function? &lt;/p&gt;&#xA;" OwnerUserId="30026" LastActivityDate="2019-02-19T16:45:18.083" Title="Simplification of an optimization objective" Tags="|optimization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31092" PostTypeId="2" ParentId="31081" CreationDate="2019-02-19T18:55:27.073" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://uscilab.github.io/cereal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cereal&lt;/a&gt; is a straight-forward and easy-to-use serialization library for C++ data structures. It knows all of the STL data structures and adapting custom classes for use with it is easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;cereal/types/unordered_map.hpp&amp;gt;&#xA;#include &amp;lt;cereal/types/memory.hpp&amp;gt;&#xA;#include &amp;lt;cereal/archives/binary.hpp&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;&#xA;struct MyRecord&#xA;{&#xA;  uint8_t x, y;&#xA;  float z;&#xA;&#xA;  template &amp;lt;class Archive&amp;gt;&#xA;  void serialize( Archive &amp;amp; ar )&#xA;  {&#xA;    ar( x, y, z );&#xA;  }&#xA;};&#xA;&#xA;struct SomeData&#xA;{&#xA;  int32_t id;&#xA;  std::shared_ptr&amp;lt;std::unordered_map&amp;lt;uint32_t, MyRecord&amp;gt;&amp;gt; data;&#xA;&#xA;  template &amp;lt;class Archive&amp;gt;&#xA;  void save( Archive &amp;amp; ar ) const&#xA;  {&#xA;    ar( data );&#xA;  }&#xA;&#xA;  template &amp;lt;class Archive&amp;gt;&#xA;  void load( Archive &amp;amp; ar )&#xA;  {&#xA;    static int32_t idGen = 0;&#xA;    id = idGen++;&#xA;    ar( data );&#xA;  }&#xA;};&#xA;&#xA;int main()&#xA;{&#xA;  std::ofstream os(&quot;out.cereal&quot;, std::ios::binary);&#xA;  cereal::BinaryOutputArchive archive( os );&#xA;&#xA;  SomeData myData;&#xA;  archive( myData );&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2019-02-19T18:55:27.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31093" PostTypeId="2" ParentId="31081" CreationDate="2019-02-19T21:52:21.623" Score="6" Body="&lt;p&gt;Numpy has a file format that is pretty simple, which makes it perfectly compatible with basically every other high level language. (&lt;a href=&quot;https://www.numpy.org/devdocs/reference/generated/numpy.lib.format.html&quot; rel=&quot;noreferrer&quot;&gt;https://www.numpy.org/devdocs/reference/generated/numpy.lib.format.html&lt;/a&gt;) It looks like the format is much lighter than boost or hdf5. The docs say it should be easy enough to write a parser yourself, if necessary, which I would tend to believe. Cursory googling also suggests there are a number of existing lightweight libraries in C++ for &lt;code&gt;.npy&lt;/code&gt; files (such as &lt;a href=&quot;https://github.com/rogersce/cnpy&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/rogersce/cnpy&lt;/a&gt;). As as added benefit, you'd always be able to load your files in python and examine them by hand in a nice python interpreter like ipython, which can be a little tricky to achieve in C++ otherwise.&lt;/p&gt;&#xA;" OwnerUserId="713" LastEditorUserId="713" LastEditDate="2019-02-20T10:19:05.487" LastActivityDate="2019-02-20T10:19:05.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31094" PostTypeId="1" CreationDate="2019-02-20T04:55:48.783" Score="1" ViewCount="344" Body="&lt;p&gt;In the formulation of Euler-Bernoulli Beam Theory, there are two degrees of freedom at a point, &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\frac{dw}{dx}$&lt;/span&gt;. Typically, the finite element model of this theory uses cubic polynomial for interpolation of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; using a two noded element as given in Chapter 5 of this &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0072466855&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt;. This element is a subparametric because the geometry is represented by low-order elements than those used to approximate the dependent variable. I need a reference where I can find the FE model for this problem using isoparametric elements. &lt;/p&gt;&#xA;" OwnerUserId="30014" LastActivityDate="2019-02-20T04:55:48.783" Title="Finite Element Model of Euler-Bernoulli Beam Theory with Isoparametric Element" Tags="|finite-element|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31095" PostTypeId="1" AcceptedAnswerId="31099" CreationDate="2019-02-20T09:16:17.673" Score="3" ViewCount="91" Body="&lt;p&gt;I'm working on an inverse problem where the solution is the values of a 2D scalar field at the vertices of a 2D triangular mesh, such that the field can be defined continuously inside the mesh via barycentric interpolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is however quite sensitive to noise, and without any kind of regularisation to impose spatial smoothness on the solution we obtain a very non-smooth field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can anyone suggest methods for imposing the required smoothness of the field?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Importantly, whatever cost function is used to promote smooth solutions, I need to be able to find the vector-derivative of its value with respect to the field values on each vertex, as gradient-based optimisation is being used to solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case it's important - the meshes are &lt;em&gt;mostly&lt;/em&gt; made up of equilateral triangles, but not exclusively.&lt;/p&gt;&#xA;" OwnerUserId="6700" LastEditorUserId="9667" LastEditDate="2019-02-20T20:28:15.663" LastActivityDate="2019-02-22T02:33:44.033" Title="Smoothness regularisation of a 2D field on a triangular mesh?" Tags="|computational-geometry|mesh|inverse-problem|smoothing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31096" PostTypeId="1" CreationDate="2019-02-16T11:14:44.497" Score="5" ViewCount="218" Body="&lt;p&gt;I have a linear system of equations &#xA;&lt;span class=&quot;math-container&quot;&gt;$$Ax=b$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$N\times N$&lt;/span&gt; matrix with integer values, and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$N\times 1$&lt;/span&gt; vector with integer values. Due to prior knowledge, I know that I am guaranteed that there exists exactly one rational solution &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$x=A^{-1}b$&lt;/span&gt; (as well as that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; has full rank). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am now searching for an algorithm in Matlab (or any other language which I can rewrite into Matlab) which provides the exact rational solution to this problem for a given &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;, for example in the form&#xA;&lt;span class=&quot;math-container&quot;&gt;$\vec{x}=\frac{r}{s}\vec{t}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; are scalar integers and &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; is an &lt;span class=&quot;math-container&quot;&gt;$N\times 1$&lt;/span&gt; vector of integers, or in the form (Matlab notation) &lt;code&gt;x=r./s&lt;/code&gt; where &lt;code&gt;r&lt;/code&gt; and &lt;code&gt;s&lt;/code&gt; are &lt;span class=&quot;math-container&quot;&gt;$N\times 1$&lt;/span&gt; integer vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried &lt;code&gt;[r,s]=rat(A\b)&lt;/code&gt;, which works for simple enough cases but quickly results in rounding problems (i.e. &lt;code&gt;r./s&lt;/code&gt; only approximates &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, but I need the exact solution). Using symbolic calculations works; however, I have to compile the program to stand-alone, and the Matlab Compiler does not support the Symbolic Toolbox as far as I know and as far as I tried.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some stats: &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; should be allowed to be at least around &lt;span class=&quot;math-container&quot;&gt;$100$&lt;/span&gt; or higher. The values in &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; can become rather large, such that I am already now using &lt;code&gt;int64&lt;/code&gt;. I understand that at one point I will run into numeric problems, and it's sufficient for me to postpone these kinds of problems as long as possible (I am grateful for any solution, even if it only works for smaller &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;). However, I need the precise solution, and I prefer an error message over any approximation, no matter how good. Runtime of the algorithm is secondary, and it's OK if it needs an hour or so for &lt;span class=&quot;math-container&quot;&gt;$N=100$&lt;/span&gt; (being more efficient is however a plus).&lt;/p&gt;&#xA;" OwnerUserId="30070" OwnerDisplayName="Engineer trying math" LastEditorUserId="20688" LastEditDate="2019-02-20T22:02:51.227" LastActivityDate="2019-07-01T08:02:15.633" Title="Finding exact rational solution to linear integer equations in Matlab" Tags="|linear-algebra|matlab|symbolic-computation|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31097" PostTypeId="2" ParentId="31096" CreationDate="2019-02-16T11:36:03.543" Score="0" Body="&lt;p&gt;The problem can be solved using the Matlab symbolic toolbox:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;N = 100; %Dimensionality&#xA;A = randi(1000, [N, N]);&#xA;assert(rank(A) == N);&#xA;B = randi(1000, [N, 1]);&#xA;&#xA;As = sym(A);&#xA;Bs = sym(B);&#xA;&#xA;tic&#xA;xs = linsolve(As, Bs); % is a rational number&#xA;toc&#xA;&#xA;disp(xs);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;xs&lt;/code&gt; is represented as a vector of rational numbers. The runtime on an i5 is about 9 seconds for the &lt;code&gt;linsolve&lt;/code&gt; call. &lt;/p&gt;&#xA;" OwnerUserId="30001" OwnerDisplayName="Lukas Koestler" LastActivityDate="2019-02-16T11:36:03.543" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31098" PostTypeId="2" ParentId="31096" CreationDate="2019-02-20T20:40:22.463" Score="1" Body="&lt;p&gt;We have the linear system &lt;span class=&quot;math-container&quot;&gt;$\rm A x = b$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathrm A \in \mathbb Z^{n \times n}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathrm b \in \mathbb Z^n$&lt;/span&gt;. We would like to find rational solutions &lt;span class=&quot;math-container&quot;&gt;$\mathrm x \in \mathbb Q^n$&lt;/span&gt;. Computing the &lt;a href=&quot;https://en.wikipedia.org/wiki/Smith_normal_form&quot; rel=&quot;nofollow noreferrer&quot;&gt;Smith normal form&lt;/a&gt; of matrix &lt;span class=&quot;math-container&quot;&gt;$\rm A$&lt;/span&gt; in &lt;a href=&quot;https://www.mathworks.com/help/symbolic/smithform.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MATLAB&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[U, V, S] = smithForm(A) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;we obtain &lt;a href=&quot;https://en.wikipedia.org/wiki/Unimodular_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;unimodular&lt;/a&gt; matrices &lt;span class=&quot;math-container&quot;&gt;$\rm U$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rm V$&lt;/span&gt; (i.e., integer matrices that have integer inverses) and diagonal matrix &lt;span class=&quot;math-container&quot;&gt;$\rm S = U A V$&lt;/span&gt;. Hence, &lt;span class=&quot;math-container&quot;&gt;$\rm A x = b$&lt;/span&gt; can be rewritten as &lt;span class=&quot;math-container&quot;&gt;$\rm S V^{-1} x = U b$&lt;/span&gt;. Let &lt;span class=&quot;math-container&quot;&gt;$\rm y := V^{-1} x$&lt;/span&gt;. We then obtain the linear system &lt;span class=&quot;math-container&quot;&gt;$\rm S y = U b$&lt;/span&gt;, which is easy to solve because &lt;span class=&quot;math-container&quot;&gt;$\rm S$&lt;/span&gt; is diagonal. Since both &lt;span class=&quot;math-container&quot;&gt;$\rm S$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rm U b$&lt;/span&gt; are integer-valued, the solution will be a rational vector. Lastly, we compute &lt;span class=&quot;math-container&quot;&gt;$\rm x = V y$&lt;/span&gt;, which is also a rational vector because &lt;span class=&quot;math-container&quot;&gt;$\rm V$&lt;/span&gt; is also integer-valued.&lt;/p&gt;&#xA;" OwnerUserId="20417" OwnerDisplayName="Rodrigo de Azevedo" LastEditorUserId="20417" LastEditDate="2019-02-21T06:26:31.733" LastActivityDate="2019-02-21T06:26:31.733" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="31099" PostTypeId="2" ParentId="31095" CreationDate="2019-02-20T23:31:42.560" Score="3" Body="&lt;p&gt;In microwave imaging, a great chunk of literature is devoted to regularization and its effect on the solution process and inversion results. One of the common methods for microwave imaging is the &lt;a href=&quot;https://doi.org/10.1088/0266-5611/13/6/013&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contrast Source Inversion (CSI) method&lt;/a&gt;, which is essentially a gradient-based optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In CSI, one would formulate the inverse problem as the optimization of the functional &lt;span class=&quot;math-container&quot;&gt;$\mathcal F(\chi)$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal F_n(\chi) = F_{1,n}(\chi)+\ldots+ F_{m,n}(\chi)&#xA;\tag{1}&#xA;$$&lt;/span&gt;&#xA;In (1), &lt;span class=&quot;math-container&quot;&gt;$F_{i,n}$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th cost function (in particular for CSI, &lt;span class=&quot;math-container&quot;&gt;$m=2$&lt;/span&gt;) for the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;th iteration. Now, there are several ways to perform a regularization on (1).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;additive regularization with a penalty function &lt;span class=&quot;math-container&quot;&gt;$F^R(\chi)$&lt;/span&gt; and weighting parameter &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt;:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal F_n(\chi) = F_{1,n}(\chi)+\ldots+ F_{m,n}(\chi)+\gamma^2F^R_n(\chi)&#xA;\tag{2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;multiplicative regularization:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathcal F_n(\chi) = \big(F_{1,n}(\chi)+\ldots+ F_{m,n}(\chi)\big)F^R_n(\chi)&#xA;\tag{3}&#xA;$$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem with additive regularization is that looking for good &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; usually requires a lot of numerical experimentation and, possibly, prior information about the desired reconstruction of &lt;span class=&quot;math-container&quot;&gt;$\chi$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiplicative regularization stems from the idea that &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; should depend on the &lt;span class=&quot;math-container&quot;&gt;$\mathcal F_{n-1}(\chi)$&lt;/span&gt;, the functional value at the previous iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest looking into &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1029/2001RS002555&quot; rel=&quot;nofollow noreferrer&quot;&gt;P. M. van den Berg, A. Abubakar, J. T. Fokkema, &quot;Multiplicative regularization for contrast profile inversion,&quot; &lt;em&gt;Radio Sci.&lt;/em&gt;, vol. 38, no. 2, Apr. 2003.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In this free-access paper, the authors present a multiplicative regularizer that fits into the gradient-optimization framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, regarding the usual suspects from the regularizers' family:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$L^1$&lt;/span&gt;-norm of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Total_variation&quot; rel=&quot;nofollow noreferrer&quot;&gt;total variation&lt;/a&gt; - prefers piecewise constants&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;F^R_n(\chi) = \frac{\int\limits_S ds(\vec{\rho}) \sqrt{\left|\nabla \chi(\vec{\rho})\right|^2+\delta_n^2}}{\int\limits_S ds(\vec{\rho}) \sqrt{\left|\nabla \chi_n(\vec{\rho})\right|^2+\delta_n^2}}&#xA;\tag{4}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$L^2$&lt;/span&gt;-norm of the total variation - &lt;strong&gt;prefers smooth profile&lt;/strong&gt; (that's probably what you are looking for), but you will lose the distinct object edges. Although, one can employ some edge-preserving algorithm, - but that's a topic for another discussion.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;F^R_n(\chi) = \frac{\int\limits_S ds(\vec{\rho}) \left(\left|\nabla \chi(\vec{\rho})\right|^2+\delta_n^2\right)}{\int\limits_S ds(\vec{\rho}) \left(\left|\nabla \chi_n(\vec{\rho})\right|^2+\delta_n^2\right)}&#xA;\tag{5}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In (4) and (5), &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; is the 2-D position vector (wrt to which we perform spatial differentiation and integration), &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is your 2-D domain, and &lt;span class=&quot;math-container&quot;&gt;$\delta_n$&lt;/span&gt; can be defined, as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\delta_n^2 = h^2\sum\limits_{j}F_{j,n}(\chi)&#xA;$$&lt;/span&gt;&#xA;By choosing &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; one can control, which cost functions from (1) participate in regularization. If you have just one functional (&lt;span class=&quot;math-container&quot;&gt;$m=1$&lt;/span&gt;), you have a simpler formula. &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the reciprocal of your mesh element size.&#xA;Now, choosing non-zero &lt;span class=&quot;math-container&quot;&gt;$\delta_n$&lt;/span&gt; helps to restore differentiability of regularizers &lt;span class=&quot;math-container&quot;&gt;$F_n^R$&lt;/span&gt; in (4) and (5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this answer, I simplified notation from the cited paper to only one inverse function &lt;span class=&quot;math-container&quot;&gt;$\chi$&lt;/span&gt; (CSI uses two different ones) and 2-D. The paper (and especially references in it) should help in finding regularizers that work better for your particular inverse problem or are just simpler versions of them.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2019-02-22T02:33:44.033" LastActivityDate="2019-02-22T02:33:44.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31102" PostTypeId="1" CreationDate="2019-02-21T18:29:45.333" Score="2" ViewCount="47" Body="&lt;p&gt;Given two solid bodies of masses &lt;span class=&quot;math-container&quot;&gt;$m_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$m_2$&lt;/span&gt; with invariable charge distributions &lt;span class=&quot;math-container&quot;&gt;$\rho_1(\mathbf{r_1})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\rho_2(\mathbf{r_2})$&lt;/span&gt; in them,* respectively, I would like to model the electrical interaction of the bodies using a user-specified electrical force law.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;* &lt;span class=&quot;math-container&quot;&gt;$\mathbf{r_1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{r_2}$&lt;/span&gt; are measured with respect to a coordinate system fixed to body 1 and body 2, respectively.&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What open source multiphysics software packages could I use to model the motion of these bodies?&lt;/p&gt;&#xA;" OwnerUserId="20904" LastEditorUserId="20904" LastEditDate="2019-02-21T20:29:12.700" LastActivityDate="2019-02-21T20:29:12.700" Title="Open source multiphysics software that can model the interaction of two insulators with a user-specified electrical force law?" Tags="|electromagnetism|open-source|multiphysics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31104" PostTypeId="2" ParentId="31054" CreationDate="2019-02-22T04:57:22.130" Score="2" Body="&lt;p&gt;You have to careful while applying TVD limiter and account for the normalization of your basis functions. With reference to [1], if your solution is as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_h(x,y) = \bar{u} + u_x \phi_i(x) + u_y \psi_j(y)&#xA;$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\phi_i(x) = \frac{x - x_i}{\Delta x_i/2}, \qquad \psi_j(y) = \frac{y - y_j}{\Delta y_j/2}&#xA;$$&lt;/span&gt;&#xA;Here &lt;span class=&quot;math-container&quot;&gt;$\bar{u}, u_x, u_y$&lt;/span&gt; are the dofs or solution variables and &lt;span class=&quot;math-container&quot;&gt;$\bar{u}$&lt;/span&gt; is the cell average value. You limit the slope as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_x = minmod(u_x, \bar{u}_{i,j} - \bar{u}_{i-1,j}, \bar{u}_{i+1,j} - \bar{u}_{i,j})&#xA;$$&lt;/span&gt;&#xA;Due to your normalization, your solution is of the form&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_h(x,y) = \bar{u} \sqrt{1/2} + u_x \sqrt{3/2} \phi_i(x) + u_y \sqrt{3/2} \psi_j(y)&#xA;$$&lt;/span&gt;&#xA;Now &lt;span class=&quot;math-container&quot;&gt;$\bar{u}$&lt;/span&gt; is not the cell average value. You have to account for these differences and properly modify the limiter step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Bernardo Cockburn and Chi-Wang Shu, The Runge–Kutta Discontinuous Galerkin Method for Conservation Laws V, JCP, 141, 1998&#xA;Multidimensional Systems&lt;/p&gt;&#xA;" OwnerUserId="3970" LastActivityDate="2019-02-22T04:57:22.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31105" PostTypeId="1" CreationDate="2019-02-22T07:37:26.393" Score="2" ViewCount="146" Body="&lt;p&gt;I have a smooth function &lt;span class=&quot;math-container&quot;&gt;$f(x) = \frac{g(x)}{h(x)}$&lt;/span&gt; that is the ratio of two smooth convex functions &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h(x)$&lt;/span&gt;. It is known that &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; has a global minimum, achieved at the unique point &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt;. Also is known that &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; has countably many local minima, and function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; value at no two local minima, is the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: Is there any optimization algorithm for this scenario, which converges to the global minimum. I am hoping there is some variant of gradient descent or the likes. Appreciate any references/solutions.&lt;/p&gt;&#xA;" OwnerUserId="13214" LastEditorUserId="20688" LastEditDate="2019-02-22T12:08:13.623" LastActivityDate="2019-02-23T02:31:17.693" Title="Is there an optimization scheme/algorithm that converges, for this non-convex scenario but with some special properties" Tags="|optimization|reference-request|nonconvex|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31106" PostTypeId="2" ParentId="31090" CreationDate="2019-02-22T07:45:47.753" Score="1" Body="&lt;p&gt;I assume that the variables &lt;span class=&quot;math-container&quot;&gt;$D_{i, j}, T^v_E, S^v_{max} \in \mathbb{R}_{&amp;gt;0}$&lt;/span&gt;. Here are my pointers for modeling this problem in discrete time steps,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define a binary variable &lt;span class=&quot;math-container&quot;&gt;$y_{vi} (t) \in \{0, 1\}$&lt;/span&gt;: Vehicle &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is in space &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; at time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. Then, the resource constraint can be expressed as &lt;span class=&quot;math-container&quot;&gt;$\sum_{v \in V} y_{vi} (t) \leq R_i, \, \forall i \in N, \forall t \geq 0$&lt;/span&gt;. The time constraint can be expressed as,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{v i} (t = T^v_E) = &#xA;     \begin{cases}&#xA;       1 &amp;amp;\quad\text{if } i = D_v\\&#xA;       0 &amp;amp;\quad\text{otherwise.} \\ &#xA;     \end{cases}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define &lt;span class=&quot;math-container&quot;&gt;$Y(t) = [y_{v i} (t)]_{v \in V, i \in N} \in \{0, 1\}^{|V|\times |N|}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$T = \max_{v\in V} T^v_E$&lt;/span&gt;.&#xA;Since the maximum speed is fixed, the problem can be modeled as distance minimization. In this case, the objective function is,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\min_{Y} \sum_{t=2}^T \sum_{v=1}^V \left(D_{\arg_{i}(y_{v i} (t-1) == 1), \arg_{j}(y_{v j} (t) == 1)} \right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you assume discrete time formulation, then we can use these definitions for a solution. We further need to add constraints on the reachable nodes in each time step for every vehicle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question needs more structure about the types of variables, specifically the space in which they live. For example, is the time discrete or continuous? Is &lt;span class=&quot;math-container&quot;&gt;$D_{i, j}$&lt;/span&gt; defined only on neighboring nodes in &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;? Can this be represented as a graph? If it is defined for all nodes, what is the meaning of vehicle path intersecting another node space. &lt;/p&gt;&#xA;" OwnerUserId="21919" LastEditorUserId="21919" LastEditDate="2019-02-22T07:55:48.730" LastActivityDate="2019-02-22T07:55:48.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31107" PostTypeId="1" AcceptedAnswerId="31112" CreationDate="2019-02-22T07:57:01.183" Score="0" ViewCount="1712" Body="&lt;p&gt;I'm currently working with the following Poisson equation with mixed boundary conditions, including a Neumann boundary condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Delta u = f\\ u(x,0) = g_1(x), 0&amp;lt;x&amp;lt;1\\u(0,y) = g_2(y), 0&amp;lt;y&amp;lt;1\\&#xA;\partial_n u(x, 1-x) = 0, 0&amp;lt;x&amp;lt;1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;on the domain &lt;span class=&quot;math-container&quot;&gt;$\Omega := \{(x,y):\mathbb{R}^2_+:x+y&amp;lt;1\}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, &lt;span class=&quot;math-container&quot;&gt;$\partial_n u(x, 1-x)=\nabla u \cdot n$&lt;/span&gt;. Now I'm somewhat confused as to how this should be discretized. How does one obtain the normal vector &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;? I think I've seen a discretization using centered finite differences, which looked like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\partial_n u_{i,j}= \frac{u_{i+1,j}-u_{i-1,j}+u_{i, j+1}-u_{i, j-1}}{2\Delta x}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;but how does this discretization account for the normal vector &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;&lt;/strong&gt; in the Neumann BC?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, here I have not just &lt;span class=&quot;math-container&quot;&gt;$\partial_n u(x, y) = 0$&lt;/span&gt; but &lt;span class=&quot;math-container&quot;&gt;$\partial_n u(x, 1-x) = 0$&lt;/span&gt;. So I'm puzzled as to how to discretize this. It is suggested to do so using the Shortley-Weller approximation, but I don't yet quite understand how it works (haven't seen it in action and couldn't find good literature on how exactly this method works mathematically).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in my understanding, this can look like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\partial_n u(x, 1-x)_{|\Delta x}=\partial_n u_{i,1-i}=\frac{u_{i+1,1-i}-u_{i-1,1-i}}{2\Delta x}+\frac{u_{i, 2-i}-u_{i, -i}}{2(1-\Delta x)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this correct? How can this be implemented in an algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would appreciate some help.&lt;/p&gt;&#xA;" OwnerUserId="29427" LastActivityDate="2019-02-22T21:17:54.347" Title="Discretization Neumann boundary condition" Tags="|finite-difference|boundary-conditions|discretization|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31109" PostTypeId="1" CreationDate="2019-02-22T10:44:09.277" Score="0" ViewCount="310" Body="&lt;p&gt;I'm refering here to Taflove's &quot;computational electrodynamcis, 3rd ed.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;He says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let us assume that the structure being modeled extends to infinity in&#xA;  the z-direction with no change in the shape or position in its&#xA;  transverse cross section. If the incident wave is also uniform in the&#xA;  z-direction, then all partial derivatives of the fields with respect&#xA;  to z must equal zero...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't quite understand why the wave should be uniform along z? How could you justify this assumption? Wave's have a finite wavelength, even within media which implies that they vary with respect to position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW: It seems like this paragraph is also cited in another textbook:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://books.google.de/books?id=uC70r-R_wF8C&amp;amp;pg=PA208&amp;amp;lpg=PA208&amp;amp;dq=tmz+mode&amp;amp;source=bl&amp;amp;ots=HPALrm6rKU&amp;amp;sig=ACfU3U1ghG1rU2p7F6EAFLJr6PISFtRbGw&amp;amp;hl=de&amp;amp;sa=X&amp;amp;ved=2ahUKEwjKlIHClM_gAhWFzKQKHUswCtwQ6AEwEHoECAMQAQ#v=onepage&amp;amp;q=tmz%20mode&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://books.google.de/books?id=uC70r-R_wF8C&amp;amp;pg=PA208&amp;amp;lpg=PA208&amp;amp;dq=tmz+mode&amp;amp;source=bl&amp;amp;ots=HPALrm6rKU&amp;amp;sig=ACfU3U1ghG1rU2p7F6EAFLJr6PISFtRbGw&amp;amp;hl=de&amp;amp;sa=X&amp;amp;ved=2ahUKEwjKlIHClM_gAhWFzKQKHUswCtwQ6AEwEHoECAMQAQ#v=onepage&amp;amp;q=tmz%20mode&amp;amp;f=false&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21797" LastEditorUserId="21797" LastEditDate="2019-02-22T14:20:01.950" LastActivityDate="2019-02-22T14:20:01.950" Title="TMZ TME modes, clarification" Tags="|electromagnetism|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31110" PostTypeId="2" ParentId="31109" CreationDate="2019-02-22T12:13:40.453" Score="1" Body="&lt;p&gt;I posit that this is better asked on physics.SE, but nevertheless I shall give my take on the answer to this question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem appears to be the understanding of &quot;uniform wave&quot;. Here &quot;uniform&quot; relates to, e.g., uniform plane wave and does not say that each point along the z axis will see the &lt;em&gt;same&lt;/em&gt; field configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a translationally invariant (along z) waveguide each xy-plane is equivalent to any other such plane. You can use Helmholtz-Hodge decomposition for the wave equation and what you will obtain using Fourier's method is that you will have a simple plane wave in the z-direction.&#xA;The transverse modes (i.e., x,y-modes) will be dependent on the wavenumber &lt;span class=&quot;math-container&quot;&gt;$k_z$&lt;/span&gt; though.&lt;/p&gt;&#xA;" OwnerUserId="28993" LastActivityDate="2019-02-22T12:13:40.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31111" PostTypeId="1" CreationDate="2019-02-22T14:08:09.707" Score="1" ViewCount="163" Body="&lt;p&gt;Given the 2D Poisson equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Delta u = f\\ u(x,0) = g_1(x), 0&amp;lt;x&amp;lt;1\\u(0,y) = g_2(y), 0&amp;lt;y&amp;lt;1\\&#xA;\partial_n u (x, 1-x) =0, 0&amp;lt;x&amp;lt;1$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;defined on the domain &lt;span class=&quot;math-container&quot;&gt;$\Omega := \{(x,y) \in \mathbb{R}^2_+: x+y&amp;lt;1\}$&lt;/span&gt;, discretize the equation using Shortley-Weller approximation with 5-point stencils.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On this domain, however, it appears that all near-boundary points use standard 5-point stencils. Is my understanding correct? I.e., with a uniform mesh non-boundary points' descritization uses the same step &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; in all directions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm, however, puzzled about the Neumann condition. Is this where Shortley-Weller approximation actually applied? I would appreciate some help with pointing me in the right direction towards discretizing the Neumann condition, and this equation as a whole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; I figure the Neumann condition for this domain can be discretized as follows: since &lt;span class=&quot;math-container&quot;&gt;$\partial_n u = \nabla u \cdot n$&lt;/span&gt;, we have &lt;span class=&quot;math-container&quot;&gt;$\partial_{n} u_{i,j} = \frac{u_{i, j+1}-u_{i,j-1}+u_{i+1,j}-u_{i-1,j}}{h}=0$&lt;/span&gt; (because &lt;span class=&quot;math-container&quot;&gt;$n=(1, 1)^T$&lt;/span&gt;). Now it looks like we encounter two ghost shells in each case. Am I on the right track?&lt;/p&gt;&#xA;" OwnerUserId="29427" LastEditorUserId="29427" LastEditDate="2019-02-22T14:44:37.073" LastActivityDate="2019-02-22T14:44:37.073" Title="PDE discretization on triangular domain" Tags="|pde|boundary-conditions|discretization|poisson|elliptic-pde|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31112" PostTypeId="2" ParentId="31107" CreationDate="2019-02-22T21:09:14.563" Score="2" Body="&lt;p&gt;The formula you show is not a general formula for the Neumann Boundary Condition, it is already projected onto &lt;span class=&quot;math-container&quot;&gt;$\mathbf{n}$&lt;/span&gt;. To see this, note that the gradient in finite difference approximation (with central differences) is given by:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\boldsymbol{\nabla}f=\left(\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}\right)\approx\left(\frac{f_{i+1,j}-f_{i-1,j}}{2\Delta x}, \frac{f_{i,j+1}-f_{i,j-1}}{2\Delta y}\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the boundary over which you are taking the NBC is: &lt;span class=&quot;math-container&quot;&gt;$$\Omega=\{(x,y)\in\mathbb{R}^2: y=1-x, 0\leq x\leq 1\}$$&lt;/span&gt; (Note: I am taking the set to be closed because boundaries of sets in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt; are closed and the set you used is neither open nor closed). The plot of this boundary is shown below. As you can see, the vector &lt;span class=&quot;math-container&quot;&gt;$\mathbf{n}=(1,1)$&lt;/span&gt; is normal to the line, and it is easy to see that in that case &lt;span class=&quot;math-container&quot;&gt;$\Delta y = \Delta x$&lt;/span&gt; (because it is a line with slope 1).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eaq3y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eaq3y.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, we verify the formula you posted:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\boldsymbol{\nabla}u\cdot\mathbf{n}\approx \left(\frac{u_{i+1,j}-u_{i-1,j}}{2\Delta x}, \frac{u_{i,j+1}-u_{i,j-1}}{2\Delta y}\right)\cdot(1,1)=\left(\frac{u_{i+1,j}-u_{i-1,j}}{2\Delta x}, \frac{u_{i,j+1}-u_{i,j-1}}{2\Delta x}\right)\cdot(1,1)=\frac{u_{i+1,j}-u_{i-1,j}+u_{i,j+1}-u_{i,j-1}}{2\Delta x}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user30058" LastEditorUserId="-1" LastEditorDisplayName="user30058" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2019-02-22T21:17:54.347" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31113" PostTypeId="1" AcceptedAnswerId="31126" CreationDate="2019-02-22T22:19:50.153" Score="2" ViewCount="101" Body="&lt;p&gt;This is a problem from projective geometry. Suppose I have a vector &lt;span class=&quot;math-container&quot;&gt;$z \in R^k$&lt;/span&gt; of unit length &lt;span class=&quot;math-container&quot;&gt;$\| z \| =1$&lt;/span&gt; inside a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-dimensional hypercube. I don't know its value but do know its projection upto an unknown face-dependent scaling factor &lt;span class=&quot;math-container&quot;&gt;$\lambda_j$&lt;/span&gt; onto &lt;span class=&quot;math-container&quot;&gt;$(k-1)$&lt;/span&gt; orthogonal faces, &lt;span class=&quot;math-container&quot;&gt;$j = 1 \ldots (k-1)$&lt;/span&gt; of the cube. My question is how best can I reconstruct the direction of &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; from the &lt;span class=&quot;math-container&quot;&gt;$(k-1)$&lt;/span&gt; projections? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation is complicated by the fact that my information about the projections is only approximate and potentially over-specified if I have estimates from more than &lt;span class=&quot;math-container&quot;&gt;$(k-1)$&lt;/span&gt; faces. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an initial solution using &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt;-optimisation but wondered if there is any established algorithm for this problem. It is obviously related to more general projective reconstructions where the views are not necessarily observed from orthogonal planes, and the orientation of those planes is typically unknown.  &lt;/p&gt;&#xA;" OwnerUserId="5261" LastEditorUserId="5261" LastEditDate="2019-02-24T14:03:51.793" LastActivityDate="2019-02-25T09:50:33.243" Title="projective reconstruction from orthogonal views" Tags="|computational-geometry|geometry|projection|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31115" PostTypeId="2" ParentId="31105" CreationDate="2019-02-23T02:31:17.693" Score="4" Body="&lt;p&gt;Despite your claim that these functions have &quot;special properties&quot;, the properties you've supplied still leave &lt;code&gt;f&lt;/code&gt; and &lt;code&gt;g&lt;/code&gt; extremely general. This means the answers you're going to get must be correspondingly general. If, for instance, you know that &lt;code&gt;g&lt;/code&gt; is always positive or &lt;code&gt;f&lt;/code&gt; is quadratic, or &lt;code&gt;g&lt;/code&gt; is some kind of positive semi-definite matrix, and so on, then you should ask a new question specifying this critical information from the start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you're trying to do is convex-convex fractional programming. Schaible's 1976 paper &quot;&lt;a href=&quot;https://link.springer.com/article/10.1007/BF00934101&quot; rel=&quot;noreferrer&quot;&gt;Minimization of ratios&lt;/a&gt;&quot; explains how such a problem can be transformed into a &lt;a href=&quot;https://en.wikipedia.org/wiki/Quasiconvex_function&quot; rel=&quot;noreferrer&quot;&gt;quasi-convex problem&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/f75O6.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f75O6.png&quot; alt=&quot;Schaible 1976&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you can make this into a quasi-convex problem. That means that you can use bisection to solve the problem to &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; tolerance in &lt;span class=&quot;math-container&quot;&gt;$\lceil log_2((u-l)/\epsilon)\rceil$&lt;/span&gt; steps where &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; bound the range of the &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; variable shown above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the functions involved, you may be able to use cvxpy to solve the problem. Some examples showing bisection of quasiconvex functions are &lt;a href=&quot;https://www.cvxpy.org/examples/applications/ant_array_min_beamwidth.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.cvxpy.org/examples/applications/maximise_minimum_SINR_BV4.20.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2019-02-23T02:31:17.693" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="31117" PostTypeId="1" CreationDate="2019-02-23T15:38:23.683" Score="3" ViewCount="136" Body="&lt;p&gt;Do there exist &quot;frameworks&quot; as to how computational scientific experiments claim validity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like &quot;scientific method for computed science&quot;?&lt;/p&gt;&#xA;" OwnerUserId="17316" LastActivityDate="2019-02-24T00:02:11.773" Title="Do there exist &quot;frameworks&quot; as to how computational scientific experiments claim validity? Scientific method for computed science?" Tags="|computational-physics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31118" PostTypeId="2" ParentId="31117" CreationDate="2019-02-24T00:02:11.773" Score="4" Body="&lt;p&gt;There is a lot of research on techniques for verification and validation of codes that use numerical approximations to solve partial differential equations arising in fluid dynamics and other areas of engineering and science.  The focus here is on verifying that the code solves the discretization of the problem correctly and that the discretization is doing an adequate job of approximating the PDE so that the resulting solution will be good enough for engineering use.  See for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;W. L. Oberkampf and C. J. Roy.  Verification and Validation in Scientific Computing 1st Edition.  Cambridge University Press, 2010.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P. J. Roache.  Verification and Validation in Computational Science and Engineering.  1998.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2019-02-24T00:02:11.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31119" PostTypeId="1" CreationDate="2019-02-24T02:02:07.790" Score="0" ViewCount="57" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$m\times m$&lt;/span&gt; diagonal, with real non negative elements&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$n \times m$&lt;/span&gt; complex&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$n \times 1$&lt;/span&gt; complex&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$m \times 1$&lt;/span&gt; complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimize &lt;span class=&quot;math-container&quot;&gt;$\Gamma(A)$&lt;/span&gt;, with respect to &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Gamma(A) = \frac{m^2(DA-P)^H (DA-P) + (FA)^H(FA)}{A^HA}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;It is known&lt;/strong&gt; that both numerator and denominator of &lt;span class=&quot;math-container&quot;&gt;$\Gamma(A)$&lt;/span&gt; are &lt;strong&gt;convex&lt;/strong&gt; and non negative. Also both the terms of the numerator are individually convex and non negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question:  An numerical optimization algorithm to find global minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from a reagular solution, I am also interested in a gradient descent based method if possible as  the matrices are large. Also &lt;span class=&quot;math-container&quot;&gt;$m &amp;gt;&amp;gt; n$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS : This question is a specific version of &lt;a href=&quot;https://scicomp.stackexchange.com/q/31105/13214&quot;&gt;this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; : more known information&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No constraints on problem but &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum P = 0$&lt;/span&gt;, I mean sum of elements of matrix &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is zero.&lt;/li&gt;&#xA;&lt;li&gt;Diagonal elements of &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; are not all zeros.&lt;/li&gt;&#xA;&lt;li&gt;Also &lt;span class=&quot;math-container&quot;&gt;$P^HP \ne 0$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Rows of &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; are orthogonal to each other. Also they are linearly independent. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="13214" LastEditorUserId="13214" LastEditDate="2019-02-24T08:30:40.910" LastActivityDate="2019-02-24T08:30:40.910" Title="Minimizing the ratio of two specific non negative quadratic convex functions" Tags="|optimization|convex-optimization|nonconvex|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31120" PostTypeId="1" AcceptedAnswerId="31121" CreationDate="2019-02-24T03:35:22.363" Score="-1" ViewCount="101" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; be a &lt;span class=&quot;math-container&quot;&gt;$m \times n$&lt;/span&gt; matrix, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; a &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-vector, &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; a &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;-vector, and &lt;span class=&quot;math-container&quot;&gt;$\|\cdot\|_2$&lt;/span&gt; represent the &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; norm (i.e., Euclidean norm).  Given &lt;span class=&quot;math-container&quot;&gt;$M,y$&lt;/span&gt;, the goal is to find &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; that minimizes the expression&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Psi(x) = \|x\|^2 + \|Mx-y\|^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a closed-form expression for the optimal &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;?  Without the &lt;span class=&quot;math-container&quot;&gt;$\|x\|^2$&lt;/span&gt; term, this would be an instance of ordinary least squares linear regression and there would be a nice solution, but I'm not sure happens with this additional term.&lt;/p&gt;&#xA;" OwnerUserId="4274" LastActivityDate="2019-02-24T03:45:42.020" Title="Minimize squared error of linear function" Tags="|optimization|least-squares|regression|quadratic-programming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31121" PostTypeId="2" ParentId="31120" CreationDate="2019-02-24T03:45:42.020" Score="1" Body="&lt;p&gt;Your problem is still a linear least squares problem.  You can write &lt;span class=&quot;math-container&quot;&gt;$\Psi(x)$&lt;/span&gt; as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Psi(x)=\| Hx - g \|_{2}^{2}$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$H=\left[&#xA;\begin{array}{c}&#xA;I \\&#xA;M&#xA;\end{array}&#xA;\right]&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g=\left[&#xA;\begin{array}{c}&#xA;0 \\&#xA;y &#xA;\end{array}&#xA;\right]$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the normal equations, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x=(H^{T}H)^{-1}H^{T}g=(M^{T}M+I)^{-1} M^{T}y$&lt;/span&gt;.   &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2019-02-24T03:45:42.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31123" PostTypeId="2" ParentId="26105" CreationDate="2019-02-24T12:08:06.323" Score="6" Body="&lt;p&gt;I was surprised for not receiving a satisfactory answer to the question above and my investigations showed me that this, indeed is an unexplored area. Hence, I put some effort in developing solutions to this problem, and published the following manuscripts:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;T. Birdal, B. Busam, N. Navab, S. Ilic and P. Sturm. &quot;A Minimalist&#xA;  Approach to Type-Agnostic Detection of Quadrics in Point Clouds.&quot;&#xA;  Proceedings of the IEEE Conference on Computer Vision and Pattern&#xA;  Recognition. 2018.&#xA;  &lt;a href=&quot;http://openaccess.thecvf.com/content_cvpr_2018/html/Birdal_A_Minimalist_Approach_CVPR_2018_paper.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://openaccess.thecvf.com/content_cvpr_2018/html/Birdal_A_Minimalist_Approach_CVPR_2018_paper.html&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;T. Birdal, B. Busam, N. Navab, S. Ilic and P. Sturm, &quot;Generic&#xA;  Primitive Detection in Point Clouds Using Novel Minimal Quadric Fits,&quot;&#xA;  in IEEE Transactions on Pattern Analysis and Machine Intelligence.&#xA;  &lt;a href=&quot;https://arxiv.org/abs/1901.01255&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1901.01255&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I will briefly touch the main idea here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is similar to &lt;a href=&quot;http://perso.lcpc.fr/tarel.jean-philippe/brown/jpt-cvpr99.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;gradient-one fitting&lt;/a&gt; (&lt;span class=&quot;math-container&quot;&gt;$\nabla 1$&lt;/span&gt;). We align the gradient vector of the quadric &lt;span class=&quot;math-container&quot;&gt;$\nabla \mathbf{Q}(\mathbf{x}_i)$&lt;/span&gt; with the normal of the point cloud &lt;span class=&quot;math-container&quot;&gt;$\mathbf{n}_i \in \mathbb{R}^3$&lt;/span&gt;. However, unlike &lt;span class=&quot;math-container&quot;&gt;$\nabla 1$&lt;/span&gt;-fits, we opt to use a linear constraint to increase the rank rather than regularizing the solution. This is seemingly non-trivial as the vector-vector alignment brings a non-linear constraint either of the form:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;\frac{\nabla \mathbf{Q}(\mathbf{x}_i)}{\|\nabla \mathbf{Q}(\mathbf{x}_i)\|} - \mathbf{n}_i\, = \,0\,\quad\text{or}\,\quad&#xA;\frac{\nabla \mathbf{Q}(\mathbf{x}_i)}{\|\nabla \mathbf{Q}(\mathbf{x}_i)\|} \,\cdot\, \mathbf{n}_i\, = \,1.&#xA;\end{align}&#xA;$$&lt;/span&gt;&#xA;The non-linearity is caused by the normalization as it is hard to know the magnitude and thus the homogeneous scale in advance. We solve this issue by introducing a per normal homogeneous scale &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; among the unknowns and write: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;\nabla \mathbf{Q}( \mathbf{x}_i) = \nabla \mathbf{v}^T_i \mathbf{q} = \alpha_i \mathbf{n}_i&#xA;\end{align}&#xA;$$&lt;/span&gt;&#xA;where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{v} = \begin{bmatrix}&#xA;			x^2 &amp;amp; y^2 &amp;amp; z^2 &amp;amp; 2xy &amp;amp; 2xz &amp;amp; 2yz &amp;amp; 2x &amp;amp; 2y &amp;amp; 2z &amp;amp; 1&#xA;         \end{bmatrix}^T&#xA;$$&lt;/span&gt;&#xA;Stacking this up for all &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; points &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i$&lt;/span&gt; and normals &lt;span class=&quot;math-container&quot;&gt;$\mathbf{n}_i$&lt;/span&gt; leads to a system of the form &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}^{\prime}\mathbf{q}=\mathbf{0}$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{equation}&#xA;\begin{bmatrix}&#xA;  \mathbf{v}^T_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;  \mathbf{v}^T_2 &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;  \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;  \mathbf{v}^T_n &amp;amp; 0 &amp;amp; 0 &amp;amp; \cdots &amp;amp; 0 \\&#xA;  {\nabla \mathbf{v}_1^T} &amp;amp; -\mathbf{n}_1 &amp;amp; \mathbf{0}_3 &amp;amp; \cdots &amp;amp; \mathbf{0}_3 \\&#xA;  {\nabla \mathbf{v}_2^T} &amp;amp; \mathbf{0}_3 &amp;amp; -\mathbf{n}_2 &amp;amp; \cdots &amp;amp; \mathbf{0}_3 \\&#xA;  \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \ddots &amp;amp; \vdots \\&#xA;  {\nabla \mathbf{v}_n^T} &amp;amp; \mathbf{0}_3 &amp;amp; \mathbf{0}_3 &amp;amp; \cdots &amp;amp; -\mathbf{n}_n&#xA;\end{bmatrix}&#xA;\begin{bmatrix} A \\ B \\ \vdots \\ I \\ J \\ \alpha_1 \\ \alpha_2 \\ \vdots \\ \alpha_n \end{bmatrix}&#xA;= \mathbf{0}&#xA;\end{equation}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;${\nabla \mathbf{v}_i^T}= {\nabla \mathbf{v}(\mathbf{x}_i)^T \in \mathbb{R}^{3\times 10}}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{0}_3$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$3\times 1$&lt;/span&gt; column vector of zeros, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}^\prime$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$4N\times(N+10)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\alpha}=\{\alpha_i\}$&lt;/span&gt; are the unknown homogeneous scales.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While the solution of this formulation lying in the nullspace of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}^{\prime}$&lt;/span&gt; produces acceptable results, the system is quite unconstrained in what it could do (the scale factors are too free). It is better to find an appropriate regularizer that is also not too complicated to implement.&#xA;In practice, once again analogous to gradient-one fitting, we could prefer unit-norm polynomial gradients, and thus, can write &lt;span class=&quot;math-container&quot;&gt;$\alpha_i=1$&lt;/span&gt; or equivalently &lt;span class=&quot;math-container&quot;&gt;$\alpha_i \gets \bar{\alpha}$&lt;/span&gt;,  one common scale factor. This &lt;strong&gt;soft constraint&lt;/strong&gt; will try to force zero set of the polynomial respect the local continuity of the data. Such regularization also saves us from solving the sensitive homogeneous system, and lets us re-write the system in a more compact form &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}\mathbf{q}=\mathbf{n}$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\small&#xA;\begin{bmatrix}&#xA;x_1^2 &amp;amp; y_1^2 &amp;amp; z_1^2 &amp;amp; 2x_1y_1 &amp;amp; 2x_1z_1 &amp;amp; 2y_1z_1 &amp;amp; 2x_1 &amp;amp; 2y_1 &amp;amp; 2z_1 &amp;amp; 1\\&#xA;x_2^2 &amp;amp; y_2^2 &amp;amp; z_2^2 &amp;amp; 2x_2y_2 &amp;amp; 2x_2z_2 &amp;amp; 2y_2z_2 &amp;amp; 2x_2 &amp;amp; 2y_2 &amp;amp; 2z_2 &amp;amp; 1\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp; \vdots &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;\\&#xA;2x_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 2y_1 &amp;amp; 2z_1 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 2y_1 &amp;amp; 0 &amp;amp; 2x_1 &amp;amp; 0 &amp;amp; 2z_1 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 2z_1 &amp;amp; 0 &amp;amp; 2x_1 &amp;amp; 2y_1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0\\&#xA;2x_2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 2y_2 &amp;amp; 2z_2 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 2y_2 &amp;amp; 0 &amp;amp; 2x_2 &amp;amp; 0 &amp;amp; 2z_2 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 2z_2 &amp;amp; 0 &amp;amp; 2x_2 &amp;amp; 2y_2 &amp;amp; 0 &amp;amp; 0 &amp;amp; 2 &amp;amp; 0\\&#xA;&amp;amp;&amp;amp;&amp;amp;&amp;amp; \vdots &amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&#xA;\end{bmatrix}&#xA;\begin{bmatrix}&#xA;A \\ B \\ C \\ D \\ E \\ F \\ G \\ H \\ I \\ J&#xA;\end{bmatrix}&#xA;=&#xA;\begin{bmatrix}&#xA;0 \\ 0 \\ \dots \\ {n}^1_x \\ {n}^1_y \\ {n}^1_z \\ {n}^2_x \\ {n}^2_y \\ {n}^2_z \\ \dots&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All in all, solving this system of equations will simultaneously guide the quadric to be incident to the point cloud while aligning its gradients towards the normals. It is also possible to weigh the contributions of points and normals differently.&#xA;In certain cases, to obtain a type-specific fit, a minor redesign of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt; tailored to the desired primitive suffices. For all these details as well as some theoretical analysis and pseudo-code, I refer you to the aforementioned publications.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2019-11-30T16:27:46.217" LastActivityDate="2019-11-30T16:27:46.217" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31125" PostTypeId="2" ParentId="31113" CreationDate="2019-02-24T14:57:40.410" Score="1" Body="&lt;p&gt;The problem seems quite ill posed so I will assume some level of over-determinism is there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, note that your vector, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; lives on a hypersphere. Thus, it is indeed on a &lt;span class=&quot;math-container&quot;&gt;$(k-1)$&lt;/span&gt; dimensional manifold, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{S}^{k-1}$&lt;/span&gt;. Then, because you don't know the scales, we could always write the problem as linear combination of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; unit vectors, &lt;span class=&quot;math-container&quot;&gt;$\{\mathbf{v}_i\in \mathbb{R}^{k}\}$&lt;/span&gt;, absorbing the unknowns into the linear coefficients:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{z}= \lambda_1\mathbf{v}_1 + \lambda_2\mathbf{v}_2 + \cdots + \lambda_{k}\mathbf{v}_{k}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you said you could overspecify &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;, I will, for the moment, assume that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{v}_{k}$&lt;/span&gt; is available. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us write those vectors in a matrix:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{V} = \begin{bmatrix}&#xA;\mathbf{v}_1 &amp;amp; \mathbf{v}_2 &amp;amp; \cdots &amp;amp; \mathbf{v}_{k-1} &amp;amp; \mathbf{v}_{k}&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since both &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\{\mathbf{v}_i\}$&lt;/span&gt; are unit length, &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\lambda}=\{\lambda_i\}$&lt;/span&gt; must be unit length: &lt;span class=&quot;math-container&quot;&gt;$\|\boldsymbol{\lambda}\|=1$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; can then be written as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{z} = \mathbf{V} \boldsymbol{\lambda} &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\lambda}$&lt;/span&gt; are the unknowns. If we have &lt;em&gt;multiple projection sets&lt;/em&gt; (different subsets can reconstruct &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt;) then &lt;span class=&quot;math-container&quot;&gt;$i=1\dots L$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L&amp;gt;&amp;gt;k$&lt;/span&gt; growing &lt;span class=&quot;math-container&quot;&gt;$\mathbf{V}$&lt;/span&gt; in width, while keeping each column unit length. In contrast, we will have a set of &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\lambda}$&lt;/span&gt; vectors, for each projection set: &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\Lambda}=\{\boldsymbol{\lambda}_i\}$&lt;/span&gt;. We can then write the above relation as: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{Z}_{k\times D}=\mathbf{V}_{k \times L}\boldsymbol{\Lambda}_{L \times D}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Z}=\begin{bmatrix} \mathbf{z} &amp;amp; \mathbf{z} &amp;amp; \cdots &amp;amp; \mathbf{z}\end{bmatrix}$&lt;/span&gt; as each of the &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; projection sets should recover the same &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this stage, I propose to pick two paths:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Alternating Optimization&lt;/strong&gt;&#xA;Start with initializing &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; to the expected value. To do that we assume &lt;span class=&quot;math-container&quot;&gt;$\lambda_i=1/L\,\,\forall i$&lt;/span&gt; and compute &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}_i = \mathbf{V}\boldsymbol{\lambda}_0\,\,\forall i$&lt;/span&gt; (At this stage you could also use your initialization). We can then iterate as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a. &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b. Fix &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}=\mathbf{z}_t$&lt;/span&gt; and solve:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\boldsymbol{\Lambda}^{\star}=\arg\min_{\boldsymbol{\Lambda}} \|\mathbf{Z}_t-\mathbf{V}\boldsymbol{\Lambda} \|_F&#xA;$$&lt;/span&gt;&#xA;  c. Fix &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\Lambda}_t=\boldsymbol{\Lambda}$&lt;/span&gt; and solve:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{z}^{\star}=\arg\min_{\mathbf{z}} \|\mathbf{Z}_t-\mathbf{V}\boldsymbol{\Lambda} \|_F&#xA;$$&lt;/span&gt;&#xA;  d. &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}_t\gets \mathbf{z}^{\star}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\,\,t\gets t+1$&lt;/span&gt;. Iterate to (b)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Sparse Coding&lt;/strong&gt;&#xA;Note that under the observation that the number of projections (in high dimensions) grow with factorial, and the desired vector can be reconstructed from a minimal subset of projections (thus sparse &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\lambda}$&lt;/span&gt;), this can potentially (and sorry if I made a mistake already) formulated as a &lt;a href=&quot;https://en.wikipedia.org/wiki/Sparse_dictionary_learning&quot; rel=&quot;nofollow noreferrer&quot;&gt;sparse coding&lt;/a&gt; problem, that tries to learn &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\Lambda}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}$&lt;/span&gt; simultaneously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this perspective &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Z}$&lt;/span&gt; denotes a dictionary, and &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\Lambda}$&lt;/span&gt; a representation that we simultaneously like to learn to recover the input, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{V}$&lt;/span&gt;. The result of this will hopefully be multiple &lt;span class=&quot;math-container&quot;&gt;$\mathbf{z}_i$&lt;/span&gt; corresponding to the columns of the &lt;em&gt;learned&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Z}$&lt;/span&gt;. We could for instance average those to get the most probable solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will take a look at the details of this and update soon.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastActivityDate="2019-02-24T14:57:40.410" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31126" PostTypeId="2" ParentId="31113" CreationDate="2019-02-25T00:13:46.000" Score="1" Body="&lt;p&gt;Thinking about it even more, I feel like the problem resembles triangulation of lines under orthographic camera model. It is possible and useful to visualize the setting in 3D:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zD0UP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zD0UP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The blue vector is what we like to recover. The identity element for the orthographic camera and projection matrices are defined respectively as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;K=\begin{bmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 \\&#xA;0 &amp;amp; 1 &amp;amp; 0 \\&#xA;0 &amp;amp; 0 &amp;amp; 0 \\&#xA;\end{bmatrix}\quad&#xA;P=\begin{bmatrix}&#xA;1 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 1 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 0\\&#xA;0 &amp;amp; 0 &amp;amp; 0 &amp;amp; 1\\&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will base my explanations on the following papers, which you should consult in case this method is to be implemented:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Wu, Fuchao, et al. &quot;Algebraic Error Based Triangulation and Metric of&#xA;  Lines.&quot; PloS one 10.7 (2015): e0132354.&#xA;  &lt;a href=&quot;https://pdfs.semanticscholar.org/5f79/2b04f318c6923496ab475168f6e94adfe93e.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pdfs.semanticscholar.org/5f79/2b04f318c6923496ab475168f6e94adfe93e.pdf&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Bartoli, Adrien, and Peter Sturm. &quot;Structure-from-motion using lines: Representation, triangulation, and bundle adjustment.&quot; Computer vision and image understanding 100.3 (2005): 416-441.&#xA;  &lt;a href=&quot;https://hal.archives-ouvertes.fr/hal-00092589/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hal.archives-ouvertes.fr/hal-00092589/document&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To benefit from these well studied tools, it is possible to think of this scenario as a &lt;span class=&quot;math-container&quot;&gt;$k-D$&lt;/span&gt; line imaged from &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; cameras. It is also good that we have a large baseline. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in the examplary figure provided above, one could think that each 2D line is composed of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; end-points &lt;span class=&quot;math-container&quot;&gt;$l_i=\{(x_1,y_1), (x_2,y_2)\}$&lt;/span&gt;. This is the standard two-point parameterization. Note that the precise location of the end points only effects the precision and has no theoretical implications. This will be clearer later. I will also parameterize the unknown 3D line &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; as a homogeneous &lt;a href=&quot;https://en.wikipedia.org/wiki/Pl%C3%BCcker_coordinates&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pluecker vector&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$L \in \mathbb{R}^6$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L \triangleq \begin{bmatrix} u \\ v \end{bmatrix}&#xA;$$&lt;/span&gt;&#xA;where the 3D vectors &lt;span class=&quot;math-container&quot;&gt;$u^Tv=0$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is defined only up to a scale, and hence it follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L^T \begin{bmatrix} &amp;amp; I_3 \\ I_3 &amp;amp; \end{bmatrix} L = 0&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can also define a &lt;span class=&quot;math-container&quot;&gt;$3 \times 6$&lt;/span&gt; line-projection matrix &lt;span class=&quot;math-container&quot;&gt;$\hat{P}$&lt;/span&gt; from a standard &lt;a href=&quot;https://en.wikipedia.org/wiki/Camera_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;camera projection matrix&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$P=K [R | t]$&lt;/span&gt; by the following equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\hat{P} =  \begin{bmatrix} det(P)R &amp;amp; [t]_xR \end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solution to the &lt;em&gt;line triangulation problem&lt;/em&gt; can be obtained through a closed form solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;L^{\star} = \arg\min_{L} L^T A L \quad \text{s.t. } L^T \begin{bmatrix} &amp;amp; I_3 \\ I_3 &amp;amp; \end{bmatrix} L = 0 \,\, \text{ and } \,\, L^T L = 1&#xA;$$&lt;/span&gt;&#xA;The second condition only avoids the trivial solution. &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; can be defined as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A = \sum\limits_{i=1}^{\text{# views}} \hat{P}_i (\sum\limits_{j=1}^{2}l_{ij}^Tl_{ij}) \hat{P}_i&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; may not satisfy the Pluecker constraint and hence it might need a correction (projection onto the Pluecker vectors or the Klein quadric). This operation can be found &lt;a href=&quot;https://hal.archives-ouvertes.fr/hal-00092589/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://arxiv.org/pdf/1602.05990.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The obtained line does not specify the sign of the direction of course, but I guess this can easily be imposed as prior or found from the orientation of 2D projections (e.g. sample two points on the line, project and if vectors do not align, reverse).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is presented above minimizes an algebraic error term that is not always the most desired energy. Of course for this problem a true geometric fit in this sense of MLE cannot be written in closed form. Bartoli and Sturm (see above) provide at least two alternatives, a quasi-linear and a non-linear refinement of this initial fit. If you are keen on getting the most accuracy, then a subsequent non-linear optimization is definitely worth looking into.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The computer vision tools are generally devised for 2D/3D scenarios. For lower dimensions, e.g. 2D, this algorithm can be simplified. However, it might not be easy to generalize the aforementioned approach to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;-dimensions. Though, the idea and the essence of the approach would be identical. &lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2019-02-25T09:50:33.243" LastActivityDate="2019-02-25T09:50:33.243" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31127" PostTypeId="1" AcceptedAnswerId="31129" CreationDate="2019-02-25T05:57:20.223" Score="0" ViewCount="357" Body="&lt;p&gt;I wanted to solve a periodic elliptic equation of the form &lt;span class=&quot;math-container&quot;&gt;$$-\nabla\cdot(A\nabla u)=-\nabla\cdot F$$&lt;/span&gt; on &lt;span class=&quot;math-container&quot;&gt;$Y=[0,2\pi)^d$&lt;/span&gt; using FreeFem++, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; are &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;-periodic. The space of solutions is &lt;span class=&quot;math-container&quot;&gt;$H^1_{per}(Y)/\mathbb{R}$&lt;/span&gt;, which can also be seen as the space of periodic &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt; functions with zero mean. How does one typically impose the zero mean condition in FEM?&lt;/p&gt;&#xA;" OwnerUserId="30072" LastActivityDate="2019-02-25T10:16:16.773" Title="Imposing zero mean condition in FEM" Tags="|finite-element|elliptic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31128" PostTypeId="1" CreationDate="2019-02-25T09:05:33.920" Score="0" ViewCount="317" Body="&lt;p&gt;I'm having serious troubles with solving translating 3 coupled differential equations into python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 3 DE's stem from a 4th order DE used to calculate the bending moment of an underwater pipeline that has been substituted/transformed into 1st order equations and are coupled to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4th order:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{d^4 w(x)}{dx^4} - \frac{Q_{grnd}+ Q_{fric}}{EI} w(x) = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3x 1st order:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$w_1(x_1) = A x_1^3  + B x_1^2 + C x_1 + D + \frac{(q_{subm} x_1^4)}{(24 EI)}$&lt;/span&gt;&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$w_2(x_2) = E x_2^3  + F x_2^2 + G x_2 + H + \frac{(q_{subm} x_2^4)}{(24 EI)}$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$w_3(x_3) = K e^{(\sqrt2 \beta x_3)}+ L e^{(-\sqrt2 \beta x_3)} + M \cos(\sqrt2 \beta x_3)+N \sin(\sqrt2 \beta x_3)- \frac{(q_{subm})}{(Q_{grnd}+Q_{fric})} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$0 \leq x_1 \leq I_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$0 \leq x_2 \leq I_2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$0 \leq x_3 \leq I_3$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The system has 15 unknown constraints and 15 boundary constraints conditions for which i want to solve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Boundary conditions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_1(0) = 0$&lt;/span&gt; (deflection at &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; = 0 is 0), &lt;em&gt;D=0&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_1'(0) = 0$&lt;/span&gt; (slope at &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; = 0 is 0), &lt;em&gt;C=0&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_1''(0) = 0$&lt;/span&gt; (bending moment at &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt; = 0 is 0), &lt;em&gt;B=0&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_1(I_1) = -p$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_2(0) = p$&lt;/span&gt;, &lt;em&gt;H=p&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_1'(I_1) = w_2'(0)$&lt;/span&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_1''(I_1) = w_2''(0)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_2(I_2) = 0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_3(0) = 0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_2'(I_2) = w_3'(0)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_2''(I_2) = w_3''(0)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_2'''(I_2) = w_3'''(0)$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_3(I_3) = d$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_3(I_3)' = 0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w_3(I_3)'' = 0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I've looked online and tried some codes myself, but until now I unfortunately haven't got any useful results. An example of my code is given below, where 15 start / 'guess' values are given for the unknown variables. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;I1 = 46.483&#xA;I2 = 5.916&#xA;I3 = 21.90&#xA;&#xA;&#xA;&#xA;A = -3.858*10**(-5)&#xA;B = 0&#xA;C = 0&#xA;D = 0&#xA;E = -1.928*10**(-4)&#xA;F = 4.270*10**(-3)&#xA;G = 0.049&#xA;H = p&#xA;K = 0.052&#xA;L = 0.1&#xA;M = -0.021&#xA;N = 0.787&#xA;&#xA;&#xA;def f(w, t):&#xA;     x1 = w[0]&#xA;     x2 = w[1]&#xA;     x3 = w[2]&#xA;     # the model equations (see Munz et al. 2009)&#xA;&#xA;     F0 = A*x1**3 + B*x1**2 + C*x1 + D + (q_subm*x1**4)/(24*EI)&#xA;     F1 = E*x2**3 + F*x2**2 + G*x2 + H + (q_subm*x2**4)/(24*EI)&#xA;     F2 = K*np.exp(np.sqrt(2)*beta*x3)+L*np.exp(-np.sqrt(2)*beta*x3)+M*np.cos(np.sqrt(2)*beta*x3)+N*np.sin(np.sqrt(2)*beta*x3)-(q_subm/(Q_grnd+Q_fric)) &#xA;     return [F0, F1, F2]&#xA;&#xA;w0 = [0, 0, 0]&#xA;t = np.linspace(0, 80, 1000) &#xA;soln = odeint(f,w0,t)&#xA;&#xA;w1 = soln[:,0]&#xA;w2 = soln[:,1]&#xA;w3 = soln[:,2]&#xA;&#xA;plt.figure()&#xA;plt.plot(t, w1, label='w1')&#xA;plt.plot(t, w2, label='w2')&#xA;plt.plot(t, w3, label='w3')&#xA;plt.xlabel('x')&#xA;plt.ylabel('deflection')&#xA;plt.title('def')&#xA;plt.legend(loc=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could anybody please be so kind to assist me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With kind regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ronald &lt;/p&gt;&#xA;" OwnerUserId="30074" LastEditorUserId="30074" LastEditDate="2019-02-25T10:16:19.357" LastActivityDate="2019-03-27T18:01:06.350" Title="Combining multiple coupled 1st order equations in python" Tags="|python|ode|boundary-conditions|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31129" PostTypeId="2" ParentId="31127" CreationDate="2019-02-25T10:16:16.773" Score="2" Body="&lt;p&gt;There are two approaches to solve this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first approach is given by Lagrange multipliers and is well explained &lt;a href=&quot;https://math.stackexchange.com/questions/2112280/zero-average-in-solution-to-laplace-problem-with-lagrange-multipliers&quot;&gt;here&lt;/a&gt;. You have to solve an extended linear system where you impose the solution to be zero-mean. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second approach, which may be easier, is to modify the system so that &lt;em&gt;one nodal value&lt;/em&gt; inside the domain is clamped to zero, i.e., you impose &lt;span class=&quot;math-container&quot;&gt;$u_{i^*} = 0$&lt;/span&gt; for some index &lt;span class=&quot;math-container&quot;&gt;$i^*$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{u} = (u_1, \ldots, u_N)^\top$&lt;/span&gt; is the vector of nodal values of your discrete solution &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt;. This is obtained by replacing an equation &lt;span class=&quot;math-container&quot;&gt;$u_{i^*} = 0$&lt;/span&gt; on the &lt;span class=&quot;math-container&quot;&gt;$i^*$&lt;/span&gt;-th line of your FEM linear system. Then, your final solution &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt; is obtained as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_h \leftarrow u_h - \frac{1}{|Y|}\int_{Y} u_h,&#xA;$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$|Y| = (2\pi)^d$&lt;/span&gt;, which is trivially zero-mean.&lt;/p&gt;&#xA;" OwnerUserId="30073" LastActivityDate="2019-02-25T10:16:16.773" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31131" PostTypeId="2" ParentId="31128" CreationDate="2019-02-25T11:30:07.963" Score="1" Body="&lt;p&gt;I think there are some additional assumptions in your equations. A fourth order equation reduces to four first order equations.&#xA;Given the fourth order equation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d^4 w(x)}{dx^4} - \frac{Q_{grnd}+ Q_{fric}}{EI} w(x) = 0$$&lt;/span&gt;&#xA;the corresponding set of first order equations&#xA;with &lt;span class=&quot;math-container&quot;&gt;$w(x)=\left(\begin{array}{c}w_1(x)\\w_2(x)\\w_3(x)\\w_4(x)\end{array}\right)$&lt;/span&gt; reads:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{dw}{dx}=\left(\begin{array}{ccc}0&amp;amp;1&amp;amp;0&amp;amp;0\\0&amp;amp;0&amp;amp;1&amp;amp;0\\0&amp;amp;0&amp;amp;0&amp;amp;1\\\frac{Q_{grnd}+ Q_{fric}}{EI}&amp;amp;0&amp;amp;0&amp;amp;0\end{array}\right)w$$&lt;/span&gt;&#xA;You can solve this numerical and the solution will be in &lt;span class=&quot;math-container&quot;&gt;$w_1(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there is no need for a numerical solution, if &lt;span class=&quot;math-container&quot;&gt;$S=\frac{Q_{grnd}+ Q_{fric}}{EI}$&lt;/span&gt; is not dependent on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Given the matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; of the first order system, you simply want to calculate the eigenvalues and eigenvector of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; to obtain the solution of the system. The solution is a linear combination of the four eigenvectors &lt;span class=&quot;math-container&quot;&gt;$w(x)=\sum_{i=1}^4a_i \phi_i(x)$&lt;/span&gt;. The unknown &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; are given by the boundary conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the clarification. The domain &lt;span class=&quot;math-container&quot;&gt;$[x_0,x_3]$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$x_i=\sum_j^i I_j$&lt;/span&gt; of the fourth order ODE is decomposed in three subdomains,  &lt;span class=&quot;math-container&quot;&gt;$[x_0,x_1],[x_1,x_2],[x_2,x_3]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do two things, either implement the given ode with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;-dependent &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; together with four boundary conditions and obtain the solution on the entire domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you demand that the solutions in different subdomains are matched along their common boundary by requiring that &lt;span class=&quot;math-container&quot;&gt;$w(x)$&lt;/span&gt; and a finite number of its derivatives are equal along the interface. &lt;strong&gt;&lt;em&gt;The number of matching conditions is the number that is necessary to give a unique solution in each subdomain&lt;/em&gt;&lt;/strong&gt;, here four.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So possible choices  for the interface conditions are e.g. symmetric &lt;span class=&quot;math-container&quot;&gt;$x_j\in\{x_1,x_2\}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$i\in\{1,2\}$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$$\begin{eqnarray}w_1(x_0)&amp;amp;=&amp;amp;\text{left bc1}\\w'_1(x_0)&amp;amp;=&amp;amp;\text{left bc2}\\w_i(x_j)&amp;amp;=&amp;amp;w_j(x_j)\\ \frac{dw_i}{dx}(x_j)&amp;amp;=&amp;amp;\frac{dw_j}{dx}(x_j)\\w_3(x_3)&amp;amp;=&amp;amp;\text{right bc1}\\w'_3(x_3)&amp;amp;=&amp;amp;\text{right bc2}\end{eqnarray}$$&lt;/span&gt;&#xA;or asymmetric:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{eqnarray}w_1(x_0)&amp;amp;=&amp;amp;\text{left bc1}\\w'_1(x_0)&amp;amp;=&amp;amp;\text{left bc2}\\w''_1(x_0)&amp;amp;=&amp;amp;\text{left bc3}\\w_1(x_1)&amp;amp;=&amp;amp;w_2(x_1)\\\\w_2(x_2)&amp;amp;=&amp;amp;w_3(x_2)\\w'_2(x_2)&amp;amp;=&amp;amp;w'_3(x_2)\\w''_2(x_2)&amp;amp;=&amp;amp;w''_3(x_2)\\w_3(x_3)&amp;amp;=&amp;amp;\text{right bc}\end{eqnarray}$$&lt;/span&gt;&#xA;As you can see, there are always four conditions for each &lt;span class=&quot;math-container&quot;&gt;$w_i$&lt;/span&gt;. You state six conditions for &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt;, seven for &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt; and seven for &lt;span class=&quot;math-container&quot;&gt;$w_3$&lt;/span&gt;. These are simply too many and cannot give you a solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you need to think which of your stated boundary conditions are the ones you want, with this equipped you need to implement a system of linear equations for the coefficients &lt;span class=&quot;math-container&quot;&gt;$A \dots N$&lt;/span&gt; defined by the conditions, given that the solutions &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$w_3$&lt;/span&gt; are valid for each domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is, that you want to keep &lt;span class=&quot;math-container&quot;&gt;$w_1(0)=0$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$w'_1(0)=0$&lt;/span&gt;,&lt;span class=&quot;math-container&quot;&gt;$w''_1(0)=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_3(x_3)=d$&lt;/span&gt; and use it together with the asymmetric conditions.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastEditorUserId="3989" LastEditDate="2019-02-25T16:51:25.330" LastActivityDate="2019-02-25T16:51:25.330" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31135" PostTypeId="2" ParentId="21106" CreationDate="2019-02-25T22:03:27.610" Score="1" Body="&lt;p&gt;It is possible to use any group &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; for isomorphism pruning with Margot's isomorphism pruning (ISOP 1.1) solver. One way of doing this is by creating a different integer linear program (ILP) whose symmetry group is &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; and submitting this ILP to Margot's solver for finding the symmetry group. Then you can replace this ILP with the original ILP and submit that as the problem that needs to be solved by using &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;. The different ILP that needs to be created can have the zero function as its objective. The constraints of this new ILP can be obtained by combining all the constraints in the original problem with the constraints that can be obtained from the original constraints by letting &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; act on them. You may end up having to use GAP to generate the new constraints by calculating orbits. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this approach you can also get rid of all the redundant constraints in the original problem and submit that with &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; to Margot's ISOP 1.1 solver. (Getting rid of redundant constraints speeds up solution times for the LP relaxations.) If you do this, then the set of all isomorphism classes of solutions you will get will remain the same.&lt;/p&gt;&#xA;" OwnerUserId="30078" LastActivityDate="2019-02-25T22:03:27.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31137" PostTypeId="1" AcceptedAnswerId="31139" CreationDate="2019-02-26T06:23:58.683" Score="1" ViewCount="154" Body="&lt;p&gt;I am using an Adam Bashforth 4 method to solve an IVP problem so I need other numerical method to estimate the first 3 values. I am &lt;em&gt;very&lt;/em&gt; much interested in finding a way to estimate the first 3 values and that particular method should have order of accuracy of 4. &lt;strong&gt;Which method would you suggest?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From a theoretical point of view, is it even possible to estimate the first 3 values with order of accuracy at 4?&lt;/p&gt;&#xA;" OwnerUserId="30032" LastActivityDate="2019-02-26T08:56:23.250" Title="Adam Bashforth 4 method: how to determine starting values and stil keep the the order of accuracy" Tags="|ode|numerics|accuracy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31139" PostTypeId="2" ParentId="31137" CreationDate="2019-02-26T08:56:23.250" Score="5" Body="&lt;p&gt;What you are looking for is called &quot;bootstrapping&quot;. It is a common problem of all multistep ODE integrators and is discussed in many books on the topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among your options are to use a lower-order method with smaller time step, or to use a one-step method of higher order for the first few steps (e.g., a Runge-Kutta method).&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-02-26T08:56:23.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31140" PostTypeId="1" AcceptedAnswerId="31168" CreationDate="2019-02-26T17:24:18.767" Score="5" ViewCount="810" Body="&lt;p&gt;Let us consider a problem of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$(\mathcal{L} + k^2) u(\mathbf{x})=0\, ,\quad \forall \mathbf{x} \in \Omega$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with Dirichlet boundary conditions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u(\mathbf{x}) = 0, \quad \forall \mathbf{x} \in \partial\Omega\, .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We discretize this problem using, for example, the finite element method, to obtain the following generalized eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$[K]\{U\} = k^2 [M]\{U\} \, .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To impose boundary conditions one can do one of the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Reduce the system by deleting rows/columns corresponding to boundary degrees of freedom in matrices &lt;span class=&quot;math-container&quot;&gt;$[K]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[M]$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Zero row/column values for the boundary degrees of freedom of both matrices &lt;span class=&quot;math-container&quot;&gt;$[K]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[M]$&lt;/span&gt;, and fix the diagonal values in &lt;span class=&quot;math-container&quot;&gt;$[K]$&lt;/span&gt; to any finite number. The problem with this approach is that the matrix &lt;span class=&quot;math-container&quot;&gt;$[M]$&lt;/span&gt; loses its positive-definiteness (if it was) and eigensolvers do not like this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Zero row/column values for the boundary degrees of freedom of both matrices &lt;span class=&quot;math-container&quot;&gt;$[K]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[M]$&lt;/span&gt;, and fix the diagonal values in &lt;span class=&quot;math-container&quot;&gt;$[M]$&lt;/span&gt; to any finite number.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For eigenvalue problems I have (almost) exclusively used 1, while I have used the other methods (or equivalent) for solving linear systems of equations.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Questions&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the most common method used for imposing Dirichlet boundary conditions?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any other method that I have not mentioned?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This question has been partially answered in &lt;a href=&quot;https://scicomp.stackexchange.com/a/24848/9667&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2019-03-02T20:22:02.197" Title="Dirichlet boundary conditions in generalized eigenvalue problem" Tags="|finite-element|pde|eigenvalues|eigensystem|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31145" PostTypeId="1" AcceptedAnswerId="31147" CreationDate="2019-02-27T14:26:09.330" Score="7" ViewCount="149" Body="&lt;p&gt;Assume one uses the classical AMG with Ruge-Stuben coarsening and direct interpolation for solving real valued problems. How can this approach be recycled to also solve complex valued problems like every harmonic simulation?&#xA;I have read in papers that it is possible to split the real and imaginary part and create an equivalent real matrix and solve this one.&#xA;But I have also read that it is possible to use the complex matrix as it is. In that case the c/f splitting is only performed using the real part of the matrix and that the prolongation and restriction needs to be only real as well. And here my main questions arises for which I cannot find any paper or so. What needs to be considered for creating the prolongation and restriction matrix using direct interpolation? Also only the real parts like in c/f splitting or should the amplitude of the complex values be used?&#xA;Any hint, papers etc. would be nice.&lt;/p&gt;&#xA;" OwnerUserId="28358" LastActivityDate="2019-02-27T21:50:30.780" Title="Algebraic multigrid for complex valued matrices" Tags="|numerics|multigrid|complex-arithmetic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31146" PostTypeId="2" ParentId="31145" CreationDate="2019-02-27T17:15:20.533" Score="3" Body="&lt;p&gt;Check &lt;a href=&quot;https://epubs.siam.org/doi/abs/10.1137/070687232?journalCode=sjoce3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algebraic Multigrid Solvers for Complex-Valued Matrices&lt;/a&gt; by MacLachlan and Osterlee.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the implementation side of things, PyAMG supports complex-valued matrices. I've used it before and it works well.&lt;/p&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="3980" LastEditDate="2019-02-27T21:50:30.780" LastActivityDate="2019-02-27T21:50:30.780" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31147" PostTypeId="2" ParentId="31145" CreationDate="2019-02-27T19:10:09.787" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How can this approach be recycled to also solve complex valued problems like every harmonic simulation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The difference of an harmonic simulation to a static simulation are bigger than just replacing real valued matrices by complex valued matrices. You will normally get matrices which can have a few significantly negative eigenvalues in addition the positive eigenvalues clustered around zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we look at this in the picture of geometric multigrid, this normally means that you need a relatively fine coarse mesh, which is already able to resolve the corresponding waves more or less (say &gt;4 points per wave cycle). You can still use multigrid to solve on the really fine grid. If using the classical AMG with Ruge-Stuben coarsening, then you probably have take care yourself that it doesn't use a too coarse mesh, for example by restricting the number of coarsening steps to a very small number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article by MacLachlan and Osterlee mentioned in Nico Schlömer's answer probably has more relevant details how to actually adapt classical AMG to complex valued matrices. However, the issue of the negative eigenvalues and their impact on an appropriate coarse mesh is not discussed in that article. The discussed Helmholz intentionallly avoids that isssue, despite that fact that the article also used the harmonic simulation as one example to motivate the study of AMG of complex valued matrices.&lt;/p&gt;&#xA;" OwnerUserId="534" LastActivityDate="2019-02-27T19:10:09.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31149" PostTypeId="1" AcceptedAnswerId="31171" CreationDate="2019-02-28T16:35:09.730" Score="4" ViewCount="747" Body="&lt;p&gt;I'm interested in a good reference/paper about how to handle &lt;em&gt;numerically&lt;/em&gt; a mass-matrix system as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{M}(t,y)\dot{y} =F(y,t)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that such a problem can be solved by using classical ODEsolvers in MatLab such as &lt;code&gt;ode15s&lt;/code&gt;, and others. Of course, due to copiright issues, it's not possible to read the source code and, IMHO, there's a lack of reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found something interesting in Hairer &amp;amp; Wanner's books (both volumes), but honestly there's not so much stuff. &#xA;In such problems, the non-singularity of &lt;span class=&quot;math-container&quot;&gt;$M(t,y)$&lt;/span&gt; plays a fundamental role, and I'm quite interested in some techniques. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Suppose to have a &lt;em&gt;not-time-dependent&lt;/em&gt; and &lt;em&gt;non-singular&lt;/em&gt; mass matrix &lt;span class=&quot;math-container&quot;&gt;$M(t,y)=M(y)$&lt;/span&gt; and a system &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;M(y)\dot{y} =F(y,t)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and say I want to compute the numerical solution with Backward Euler (or any implicit method, in order to avoid numerical instabilities)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, one can write &lt;span class=&quot;math-container&quot;&gt;$\dot{y} = M^{-1}(y) f(y,t):= \tilde{F}(y,t)$&lt;/span&gt; and formally we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;y_{n+1}=y_n + \Delta t \tilde{F}(t_{n+1},y_{n+1})=y_n+ M(y_{n+1})^{-1} \Delta t F(t,y_{n+1})&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and hence&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;M(y_{n+1}) [y_{n+1}-y_n]=\Delta t F(t_{n+1},y_{n+1})&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another interesting problem is how to solve with Newton's method this non-linear system of equations. As far as I know, a simplified Newton's method is used, but I can't find any reference about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd be very grateful if someone has references/hints or something else !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; (after Bill's comment)&#xA;[REMARK: I'd like not to copy explicitely MatLab's approach]&#xA;For the simplfied Newton's method, a possible approach could be the following. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting from the functional we have to set to zer written above, I'd like to differentiate w.r.t. &lt;span class=&quot;math-container&quot;&gt;$y_{n+1}$&lt;/span&gt;, namely&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;J_n=\frac{\partial (M(y_{n+1}) \cdot y_{n+1})}{\partial y_{n+1}}- \Delta t \frac{\partial F(t_{n+1},y_{n+1})}{\partial y_{n+1}}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and consider the &lt;em&gt;constant matrix&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$M(y_{n+1})$&lt;/span&gt; at each step, so the jacobian would become&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;J_n=M(y_{n+1})-\Delta t \frac{\partial F(t_{n+1},y_{n+1})}{\partial y_{n+1}}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could it be a good starting point?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;span class=&quot;math-container&quot;&gt;$^2$&lt;/span&gt; [02/03/19']&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use the previous approach to solve simple autonomous and non-singular problems like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\begin{bmatrix}&#xA;    y(1) &amp;amp; 0 \\&#xA;   0 &amp;amp; y(2)&#xA;\end{bmatrix}&#xA;\dot{y} = [y(1),\sin(y(2))]&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$y(1)(0)=y(2)(0)=1$&lt;/span&gt; and it worked in the right way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also with other non-linear and more complicated problems, the numerical solution was ok. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to generalize this approach to the trapezoidal rule:&#xA;startinf from &lt;span class=&quot;math-container&quot;&gt;$M(y) \dot{y}=f(y)$&lt;/span&gt;, in the hypotesis that &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is non-singular, I have &lt;span class=&quot;math-container&quot;&gt;\begin{align} \dot{y}= M^{-1}(y)f(y)=\tilde{f}(y) \end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the standard trapezoidal rule leads to : &lt;span class=&quot;math-container&quot;&gt;\begin{align} y_{n+1}=y_{n} +\frac{\Delta t}{2}(M^{-1}(y_{n+1})f(y_{n+1})+M^{-1}(y_n)f(y_n)) \end{align}&lt;/span&gt; and multiplying first for one and then for the other inverse, one gets &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;M(y_n)M(y_{n+1})[y_{n+1}-y_n]=\frac{\Delta t}{2} (M(y_n)f(y_{n+1})+M(y_{n+1})f(y_n))&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The jacobian has been computed with the same logic as before, &quot;freezing&quot; the &lt;span class=&quot;math-container&quot;&gt;$M(y_{n+1})$&lt;/span&gt; term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A rapid numerical experiment with the same problem above shows the correct order of convergence.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Any other hints, suggestions, comments, are really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="26296" LastEditorUserId="26296" LastEditDate="2019-03-02T15:19:37.403" LastActivityDate="2019-04-03T08:01:45.360" Title="Mass Matrix and how to handle it (ODEs) - References" Tags="|ode|numerics|matrix-equations|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31151" PostTypeId="1" CreationDate="2019-02-28T17:42:57.123" Score="0" ViewCount="531" Body="&lt;p&gt;I have series of points extracted from a regular grid, with their X/Y coordinates. A previous algorithm (that I cannot modified!) output a list of these coordinates, but the ordering of these point is random in regard to the scope of this problem as expressed byt the following python-style notation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ [X_O,Y_O], [X_M, Y_M], [X_D, Y_D], ... ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where X Y are the X and Y coordinates of each point. I am now trying to &quot;order&quot; them with the following logic (from point A to point R on the grid visualisation):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ [X_A,Y_A], [X_B, Y_B], [X_C, Y_C], ... ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mc5BX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mc5BX.png&quot; alt=&quot;Point ordering aim&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final aim being to plot a profile with the actual value on the Z grid. I have tried a &quot;hacky&quot; way by just identifying the neighbours of each point and ordering the one that have not been processed yet, but it is not reliable enough as I faced many artifacts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm I missed that is designed for that problem?&#xA;I have done some research but I feel I am missing out something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="30110" LastEditorUserId="30110" LastEditDate="2019-03-01T09:00:11.247" LastActivityDate="2019-04-12T11:08:35.643" Title="Ordering points from X Y coordinates" Tags="|computational-geometry|geometry|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31152" PostTypeId="1" CreationDate="2019-02-28T21:48:27.950" Score="0" ViewCount="52" Body="&lt;p&gt;Have Schoenberg's cardinal B-splines been extended to accept derivative information at each knot, similar to how Lagrange interpolation can be improved by Hermite interpolation?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2019-02-28T21:48:27.950" Title="Cardinal B-Splines with derivative information" Tags="|reference-request|interpolation|b-spline|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31154" PostTypeId="1" CreationDate="2019-03-01T02:54:03.383" Score="1" ViewCount="207" Body="&lt;p&gt;I have been working on a hybrid dimensional model using the mixed FEM formulation, in which 3D elements and 2D elements are combined by certain relationships between the degrees of freedom (DOFs) between face flux for 3D elements and pressure DOFs for 2D elements. I'm doing this for Raviart Thomas elements of various orders, i.e., RT0-RT1 etc. However, I'm struggling to find the 1D equivalent of the RT elements. I have not found any reference about 1D mixed formulation elements, let alone 1D RT elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that the lowest 1D mixed element would be quite simple: 1 DOF for the constant pressure and 2 DOFs for the velocity field, one for each end node of the element. But what about higher order elements? Pressure DOFs will surely be point evaluations along the interval, but how can the DOFs for the velocity be defined?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or hint will be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="23183" LastActivityDate="2019-03-01T17:16:08.073" Title="Mixed formulation in 1D" Tags="|finite-element|pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31157" PostTypeId="2" ParentId="31151" CreationDate="2019-03-01T16:58:34.780" Score="1" Body="&lt;p&gt;I think the easiest way in C++ would be to make a vector of tuple.&#xA;&lt;code&gt;std::vector&amp;lt; std::tuple&amp;lt; int, int, double&gt;&gt;&lt;/code&gt;&#xA;This vector than contains the X and Y coordinate as well as the Z value as tuples. After that you can simply use &lt;code&gt;sort()&lt;/code&gt;. If you store the Z value at the third place of the tuple, you need an extra function for sorting. Therefore I would suggest to put the value for which you need the sorting at the first place of the tuple. Have a look at this tutorial &lt;a href=&quot;https://www.geeksforgeeks.org/sorting-vector-tuple-c-ascending-order/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;Hope this was what you were looking for.&lt;/p&gt;&#xA;" OwnerUserId="28358" LastActivityDate="2019-03-01T16:58:34.780" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31158" PostTypeId="2" ParentId="31154" CreationDate="2019-03-01T17:16:08.073" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;http://www.logg.org/anders/pub/papers/KirbyLoggEtAl2012a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;RT elements&lt;/a&gt; for a given dimension span &lt;span class=&quot;math-container&quot;&gt;$H(div)$&lt;/span&gt;, the space of vector fields for which the components and the weak divergence are square-integrable. In one dimension the distinctions between vectors and scalars break down, and this is just the space of square-integrable functions with square integrable weak derivatives, otherwise known as &lt;span class=&quot;math-container&quot;&gt;$H^1$&lt;/span&gt;. As such, all your 1D elements are likely to be common Lagrange elements of various polynomial degrees, with or without continuity across element boundaries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what you describe, it's possible you may be interested in the &lt;span class=&quot;math-container&quot;&gt;$P2-P1$&lt;/span&gt; discontinuous element pair, where velocities are piecewise quadratic and continuous (for the nodal representation, think a DOF at each end of the interval, and one in the middle) and pressure is piecewise linear, but discontinuous (you can put the DOFs where you like for that one). This is &lt;a href=&quot;http://www-users.math.umn.edu/~arnold/papers/imacs92.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sometimes described in 2D for the incompressible Stokes problem&lt;/a&gt;, but a 1D description isn't very interesting, since the only incompressible 1D velocity fields are constant everywhere.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2019-03-01T17:16:08.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31159" PostTypeId="1" CreationDate="2019-03-01T18:14:08.767" Score="2" ViewCount="82" Body="&lt;p&gt;Are there efficient public solvers for the time-dependent Schrödinger equation with time-independent Hamiltonian and 2 or 3 degrees of freedom?&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2019-03-11T14:23:58.540" Title="public solvers for the time-dependent Schrödinger equation?" Tags="|hyperbolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31160" PostTypeId="1" CreationDate="2019-03-01T20:11:43.713" Score="3" ViewCount="117" Body="&lt;p&gt;Can anyone recommend a structural analysis library that satisfies the following requirements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;C++ API&lt;/li&gt;&#xA;&lt;li&gt;Simulate both beam elements and shell (slab) elements&lt;/li&gt;&#xA;&lt;li&gt;Both static and dynamic analysis&lt;/li&gt;&#xA;&lt;li&gt;Free and/or open-source&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We're essentially looking to find a library that can do analysis similar to e.g. &lt;a href=&quot;https://www.csiamerica.com/products/sap2000&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAP2000&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However we are looking to incorporate this analysis in our larger software product, so we'd like the library to have an API that we can include and link against (i.e. has header(s) and dll(s)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of, at least, &lt;a href=&quot;https://libmesh.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;libMesh&lt;/a&gt;. Are there other popular libraries I should be considering?&lt;/p&gt;&#xA;" OwnerUserId="16853" LastActivityDate="2019-03-01T20:11:43.713" Title="Structural Analysis Library" Tags="|finite-element|c++|solid-mechanics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31162" PostTypeId="1" CreationDate="2019-03-01T21:33:09.773" Score="0" ViewCount="21" Body="&lt;p&gt;I am working on an interesting algorithm: Its absolute error is exponential in a parameter &lt;span class=&quot;math-container&quot;&gt;$j \in \mathbb{N}$&lt;/span&gt;, and for a given &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, I have complete freedom to choose between an &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(1)$&lt;/span&gt; time-complexity algorithm with memory consumption &lt;span class=&quot;math-container&quot;&gt;${\sim}2^{j}$&lt;/span&gt;, or a &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(2^j)$&lt;/span&gt; time-complexity algorithm with &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(1)$&lt;/span&gt; memory consumption. I can also &quot;meet in the middle&quot; and consume any intermediate amount of memory and compute that I wish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having trouble deciding on how make the compute/memory/accuracy tradeoffs. Have there been cost functions developed that could help answer this question?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, I could write something like&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathrm{cost} = c_1 t + c_2 m + c_3 e&#xA;$$&lt;/span&gt;&#xA;for evaluation time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;, memory consumption &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and error &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;, and just &lt;em&gt;pick&lt;/em&gt; three positive constants &lt;span class=&quot;math-container&quot;&gt;$c_1, c_2, c_3$&lt;/span&gt;. But I would prefer to use a cost function that someone has already put some thought into.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2019-03-01T21:33:09.773" Title="Cost functions to judge time/memory/accuracy tradeoffs" Tags="|optimization|complexity|accuracy|memory-management|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31164" PostTypeId="1" CreationDate="2019-03-01T22:47:54.967" Score="9" ViewCount="1373" Body="&lt;p&gt;I am very interested in optimizing the hell out of linear system solving for small matrices (10x10), sometimes called &lt;em&gt;tiny&lt;/em&gt; matrices. Is there a ready solution for this? The matrix can be assumed nonsingular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solver is to be executed in excess of 1 000 000 times in microseconds on an Intel CPU. I am talking to the level of optimization used in computer games. No matter if I code it in assembly and architecture-specific, or study precision or reliability tradeoffs reductions and use floating point hacks (I use the -ffast-math compile flag, no problem). The solve can even fail for about 20% of the time!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eigen's partialPivLu is the fastest in my current benchmark, outperforming LAPACK when optimized with -O3 and a good compiler. But now I am at the point of handcrafting a custom linear solver. Any advice would be greatly appreciated. I will make my solution open source and I'll akcnowledge key insights in publications, etc..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related: &lt;a href=&quot;https://scicomp.stackexchange.com/questions/16327/speed-of-solving-linear-system-with-block-diagonal-matrix&quot;&gt;Speed of solving linear system with block diagonal matrix&lt;/a&gt; &#xA;&lt;a href=&quot;https://scicomp.stackexchange.com/questions/29043/what-is-the-fastest-method-to-invert-millions-of-matrices?noredirect=1&amp;amp;lq=1&quot;&gt;What is the fastest method to invert millions of matrices?&lt;/a&gt;&#xA;&lt;a href=&quot;https://stackoverflow.com/q/50909385/1489510&quot;&gt;https://stackoverflow.com/q/50909385/1489510&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30112" LastEditorUserId="30112" LastEditDate="2019-03-04T17:17:20.477" LastActivityDate="2020-02-23T18:34:53.467" Title="fastest linear system solve for small square matrices (10x10)" Tags="|linear-algebra|linear-solver|compiling|open-source|assembly|" AnswerCount="4" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="31165" PostTypeId="2" ParentId="31164" CreationDate="2019-03-02T00:22:37.170" Score="7" Body="&lt;p&gt;Using an Eigen matrix type where the number of rows and columns is encoded into the type &lt;em&gt;at compile time&lt;/em&gt; gives you an edge over LAPACK, where the matrix size is known only at runtime.&#xA;This extra information allows the compiler to do full or partial loop unrolling, eliminating lots of branch instructions.&#xA;If you're looking at using an existing library rather than writing your own kernels, having a data type where the matrix size can be included as C++ template parameters will probably be essential.&#xA;The only other library I know of that does this is &lt;a href=&quot;https://bitbucket.org/blaze-lib/blaze&quot; rel=&quot;nofollow noreferrer&quot;&gt;blaze&lt;/a&gt;, so that might be worth benchmarking against Eigen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you decide to roll your own implementation, you might find what PETSc does for its block CSR format to be a useful example, although PETSc itself probably won't be the right tool for what you have in mind.&#xA;Rather than write a loop, they write out every single operation for small matrix-vector multiplies explicitly (see &lt;a href=&quot;https://github.com/petsc/petsc/blob/master/src/mat/impls/baij/seq/baij2.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;this file&lt;/a&gt; in their repository).&#xA;This guarantees that there are no branch instructions like you might get with a loop.&#xA;The versions of the code with AVX instructions are a good example of how to actually use vector extensions.&#xA;For example, &lt;a href=&quot;https://github.com/petsc/petsc/blob/master/src/mat/impls/baij/seq/baij2.c#L1686&quot; rel=&quot;nofollow noreferrer&quot;&gt;this function&lt;/a&gt; uses the &lt;code&gt;__m256d&lt;/code&gt; data type to simultaneously operate on four doubles at the same time.&#xA;You could get an appreciable performance boost by explicitly writing out all the operations using vector extensions, only for LU factorization instead of matrix-vector multiply.&#xA;Rather than actually write the C code by hand, you'd be better off using a script to generate it.&#xA;It might also be fun to see if there's an appreciable performance difference when you reorder some of the operations to better take advantage of instruction pipelining.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also get some mileage out of the tool &lt;a href=&quot;http://stoke.stanford.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;STOKE&lt;/a&gt;, which will randomly explore the space of possible program transformations to find a faster version.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastEditorUserId="3481" LastEditDate="2019-03-08T17:06:09.853" LastActivityDate="2019-03-08T17:06:09.853" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31166" PostTypeId="1" CreationDate="2019-03-02T07:23:33.410" Score="2" ViewCount="98" Body="&lt;p&gt;&lt;strong&gt;Context:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a fluid solver, I need to compute areas and volumes of curved elements. My curved elements are quadratic triangles and quadratic tetrahedra, defined by their Lagrange nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider a 3d quadratic &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}_2$&lt;/span&gt; tetrahedron defined by the positions of its 10 Lagrange nodes. Its volume is the integral of the Jacobian. Since the Jacobian is a degree 3 polynomial, a degree 3 quadrature will give the exact answer. Is there a smart way of obtaining an expression that is as simple as possible (and as fast as possible to compute) ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Consider now a 3d quadratic &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}_2$&lt;/span&gt; triangle defined by the positions of its 6 Lagrange nodes. Now the Jacobian to be integrated has a square root (length of the cross product of the two gradient vectors). Is there another way of doing the computation that gives a polynomial ? In my specific case, the &lt;span class=&quot;math-container&quot;&gt;$\mathbb{P}_2$&lt;/span&gt; triangle is on a sphere, can we make use of that to simplify the computation ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16910" LastEditorUserId="16910" LastEditDate="2019-03-02T08:52:10.037" LastActivityDate="2019-03-06T14:13:01.767" Title="Area and volume of P2 elements" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31168" PostTypeId="2" ParentId="31140" CreationDate="2019-03-02T20:22:02.197" Score="2" Body="&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is described spectrally, then there are other methods as I mentioned in that old answer. If not, I am unsure of additional options for you. This is a well-known problem that often results in spurious eigenvalues, with plenty of literature to back it up (e.g. &lt;a href=&quot;http://web.physics.ucsb.edu/~complex/pubs/siam_spurious.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;see here&lt;/a&gt;). Most of it lies in the area of hydrodynamic stability, which might help guide your search, or at least assure yourself you are in good company.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I have seen, most people use approach 1. from your question if possible, though  I prefer approach 2 as it is more general. If you're keen on maintaining positive-definiteness of &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, then you might be able to modify option 2. to accommodate this: instead of zero-ing all the boundary values/rows you care about, you can constrain those rows to be such that &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; remains PD, then modify the associated row of &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; to be whatever it needs to be to satisfy your boundary condition. I have some concern that this might only be possible in the spectral space, since in that case all of the DOFs of &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; would be associated with the BC (and so you have more options to play around with).&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2019-03-02T20:22:02.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31169" PostTypeId="2" ParentId="20178" CreationDate="2019-03-03T05:22:56.770" Score="3" Body="&lt;p&gt;I realize this question is old, but I just saw it and find it interesting. In the past, I have followed the suggestions found in this question's comments, coupled with some slightly more complicated cases that I'm familiar with in the literature (Orr--Sommerfeld is always handy). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm also aware of some literature on the inhomogeneous eigenvalue problems that arise when constructing a manufactured solution. There is some discussion of such problems here: &lt;a href=&quot;https://www.researchgate.net/publication/326679335_The_application_of_Method_of_Manufactured_Solutions_to_method_of_characteristics_in_planar_geometry&quot; rel=&quot;nofollow noreferrer&quot;&gt;DOI: 10.1016&lt;/a&gt;. These authors also suggest a so-called Method of Manufactured Cross Sections (MXS, I guess) to avoid this issue altogether, which I won't pretend to understand at the moment, but could very well be useful.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2019-03-03T05:22:56.770" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31171" PostTypeId="2" ParentId="31149" CreationDate="2019-03-03T22:16:57.980" Score="4" Body="&lt;p&gt;Ignoring Newton's method here is the wrong approach! The fact that you're using Newton's method is what makes this cheap to add, and is what makes singular mass matrices possible. Essentially look at Implicit Euler. You'll notice that the only matrix you need to use in your quasi-Newton steps is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ W = (I - \gamma J) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and if you have a mass matrix, this becomes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ W = (M - \gamma J) $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is true for any BDF/SDIRK/Rosenbrock method, since you can simplify any of their rootfinding equations to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ M u_{n+1} - \gamma f(u_{n+1},p,t) - C = 0 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is all of the stuff that doesn't depend on &lt;span class=&quot;math-container&quot;&gt;$u_{n+1}$&lt;/span&gt;. So with that small change to your Newton method, you now converge to the solution with the mass matrix for any of these (semi-)implicit schemes. Newton's method just uses this for a linear solve, so you only ever have to LU it, never inverse. If &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; is singular, you can still have that &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; isn't, and also just use something robust like QR-factorization, and then this will still converge. A singular mass matrix is another representation of a DAE (use the terms without a derivative to define a constraint relation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note that for higher order methods, you may have to handle FSAL separately with an &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; linear solve.)&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2019-03-03T22:16:57.980" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31173" PostTypeId="1" CreationDate="2019-03-04T18:33:12.963" Score="0" ViewCount="170" Body="&lt;p&gt;I am trying to solve a 2D elliptic PDE (see complete electrode model for electrical impedance tomography) using the finite element method (FEM) over a circular region &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt;. I have discretized the model, and got the following surface integral&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_e \phi_i(x,y) \phi_j(x,y) \mathrm{d}s, \mbox{ for } i,j = 1,...,n\, .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of mesh points, &lt;span class=&quot;math-container&quot;&gt;$\phi_i$&lt;/span&gt; are the basis functions and &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; is an electrode placed on the boundary connecting two adjacent boundary points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I use polar coordinates combined with the standard basis functions in FEM to evaluate these integrals? If so, how can I do this? &lt;/p&gt;&#xA;" OwnerUserId="30145" LastEditorUserId="9667" LastEditDate="2019-03-04T22:07:34.453" LastActivityDate="2019-03-04T22:07:34.453" Title="Finite element method for Surface integrals using polar coordinates" Tags="|finite-element|elliptic-pde|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31174" PostTypeId="1" AcceptedAnswerId="31176" CreationDate="2019-03-04T20:33:03.853" Score="1" ViewCount="152" Body="&lt;p&gt;I am a little bit confused over the concept of translational and rotational degrees of freedom (DoFs) in structures, and their relation to displacement/traction BCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do displacement boundary conditions constrain translational DoFs, only, or also rotational DoFs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for a 3-D case, if I specify the 3 translational components to be zero, I am effectively constraining all 3 translational DoFs. What does this do to the rotational DoFs? Intuitively, all 3 rotational DoFs would also be constrained. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does traction BCs play into translational and rotational DoFs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of my confusion is when I look at open-source or commercial structural mechanics solvers, I only see the ability to specify displacement or traction boundary conditions, not anywhere to specify translational or rotational constraints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some illustrations would be very helpful in my understanding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematical expressions would be helpful as well. In particular, I am wondering how rotational and translation DoFs are related to displacements and displacement gradients mathematically. &lt;/p&gt;&#xA;" OwnerUserId="29583" LastActivityDate="2019-03-05T11:39:12.540" Title="Structural boundary conditions - rotational/translational DoFs and displacement/tractions BCs" Tags="|boundary-conditions|solid-mechanics|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="31175" PostTypeId="1" AcceptedAnswerId="31193" CreationDate="2019-03-05T05:49:33.957" Score="2" ViewCount="277" Body="&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;. Consider the one-dimensional adjoint Euler equations for &lt;span class=&quot;math-container&quot;&gt;$(x,t) \in \Omega \times [0,T]$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$\Omega \subset \mathbb{R}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$T &amp;gt; 0$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \varphi_t + \Big(\frac{\mathrm{d}F}{\mathrm{d} U}(x)\Big)^{\top}\ \varphi_x = 0 \quad \text{in} \quad \Omega \times [0,T] \tag{$*$} \label{eq:adj}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; is the flux vector of the 1D Euler equations, &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; the conservative variable vector and &lt;span class=&quot;math-container&quot;&gt;$\varphi_x$&lt;/span&gt; the derivative of the adjoint vector with respect to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, \eqref{eq:adj} is the adjoint equation derived from the conservative form of Euler equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question I&lt;/strong&gt;. Now, what scheme, having a cell-centered finite volume (FV) discretization and artificial dissipation (similar to the JST scheme), would be appropriate to numerically solve this equation (considering that it is cast in a nonconservative form)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Remark.&lt;/strong&gt;&#xA;If I am not mistaken, the above homogeneous and nonconservative form can be recasted to a nonhomogeneous and conservative form using the product rule of the derivative&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \varphi_t + \Big[\Big(\frac{\mathrm{d}F}{\mathrm{d} U}\Big)^{\top}(x)\ \varphi \Big]_x = \Big[\Big(\frac{\mathrm{d}F}{\mathrm{d} U}(x)\Big)^{\top}\Big]_x\ \varphi$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question II&lt;/strong&gt;. Is the above method correct? Also, considering that artificial dissipation is used, is there any faster and simpler way to discretize the system \eqref{eq:adj} avoiding numerical pitfalls?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Remark&lt;/strong&gt;. The coefficients &lt;span class=&quot;math-container&quot;&gt;$\frac{\mathrm{d}F}{\mathrm{d} U}(x)$&lt;/span&gt; are frozen in time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and they depend only on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29031" LastEditorUserId="29031" LastEditDate="2019-03-07T16:28:40.677" LastActivityDate="2019-03-09T05:04:20.080" Title="Finite volume discretization of non-conservative linear hyperbolic equation" Tags="|fluid-dynamics|computational-physics|finite-volume|discretization|hyperbolic-pde|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31176" PostTypeId="2" ParentId="31174" CreationDate="2019-03-05T11:39:12.540" Score="3" Body="&lt;p&gt;In classical elasticity theory, the kinematics of the body are described in&#xA;terms of translational displacements. It is certainly not required that only&#xA;translational displacements are used. The less-frequently-used &lt;a href=&quot;http://silver.neep.wisc.edu/~lakes/Coss.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;micropolar&#xA;elasticity&lt;/a&gt;&#xA;theory also includes local rotations to describe the kinematics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In constructing a finite element model for a solid body, it may be useful&#xA;or even essential to include rotational degrees of freedom as nodal unknowns.&#xA;This is most common in elements that describe bending-- i.e. beam, plate, and&#xA;shell elements. In structural mechanics, the classical simplification of&#xA;elasticity theory to describe this bending behavior results in higher-order&#xA;derivatives of the translations. Mathematically-correct, finite element &#xA;approximation of these&#xA;terms frequently requires rotational degrees of freedom at the nodes.&#xA;But rotational degrees of freedom are not limited to bending elements &#xA;Higher-order finite element formulations for general elasticity have occasionally&#xA;used rotational degrees of freedom simply to improve the approximation of the&#xA;translations within the element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kinematic boundary conditions in classical elasticity are described in terms&#xA;of translations, say, on a face of a 3D body. In a finite element model,&#xA;prescribing a particular translation on a face may require also prescribing&#xA;the rotations at the nodes on this face simply because the finite element&#xA;approximation for this translation on this face includes those rotations.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2019-03-05T11:39:12.540" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31178" PostTypeId="1" AcceptedAnswerId="31185" CreationDate="2019-03-05T14:00:37.000" Score="5" ViewCount="119" Body="&lt;p&gt;I have what would be a straightforward mixed-integer linear programming problem, except for the fact that some of the constraints are of the form &lt;span class=&quot;math-container&quot;&gt;$f(x_1,x_2,x_3,\ldots,x_n) &amp;lt; c$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is '&lt;em&gt;take the maximum of the largest coordinate and the sum of all the smaller ones&lt;/em&gt;'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In lisp:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(defn f [&amp;amp; l]&#xA;  (let [sl (reverse (sort l))]&#xA;    (max (first sl) (reduce + (rest sl)))))&#xA;&#xA;(f 1 2 3 4 5) -&amp;gt; 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In three dimensions &lt;em&gt;e.g.&lt;/em&gt; I think I can rephrase this for e.g. &lt;span class=&quot;math-container&quot;&gt;$f&amp;lt;10$&lt;/span&gt; as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(defn constraint [x,y,z]&#xA;  (or&#xA;   (and (&amp;lt;= x 10)&#xA;        (&amp;lt;= (+ y z) 10))&#xA;   (and (&amp;lt;= y 10)&#xA;        (&amp;lt;= (+ x z) 10))&#xA;   (and (&amp;lt;= z 10)&#xA;        (&amp;lt;= (+ x y) 10))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which is clearly the union of &lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;) convex objects (prisms).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a name for this type of constraint? Are there techniques and packages for solving these kinds of problems?&lt;/p&gt;&#xA;" OwnerUserId="30090" LastEditorUserId="30090" LastEditDate="2019-03-05T17:59:39.553" LastActivityDate="2019-03-05T20:58:51.877" Title="What is the name for this type of constraint?" Tags="|optimization|linear-programming|mixed-integer-programming|terminology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31179" PostTypeId="1" AcceptedAnswerId="31214" CreationDate="2019-03-05T14:17:34.833" Score="6" ViewCount="1803" Body="&lt;p&gt;Most CFD codes use FVM. Most computational structures codes use FEM.&#xA;Why is the FEM not frequently used in CFD, and why is FVM not frequently used in FEM? &lt;/p&gt;&#xA;" OwnerUserId="29583" LastActivityDate="2019-03-11T12:19:13.190" Title="Why is the FVM traditionally used in CFD, and FEM in computational structures?" Tags="|finite-element|fluid-dynamics|finite-volume|solid-mechanics|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31182" PostTypeId="1" CreationDate="2019-03-05T16:05:06.310" Score="3" ViewCount="36" Body="&lt;p&gt;When solving systems of non-linear equations using Newton's method, it is often observed that the system has an independent sub-system, e.g. :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ f(x,y) = 0 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ g(x,y) = 0 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ h(x,y,z) = 0 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am not mistaken, you can (and should) solve first for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; and then subsequently solve for &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;.&#xA;Now when the dimension of the system increases, it is not so easy to spot such independent sub-systems. Are there algorithms adapted to this issue ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my specific case, the Jacobian of the system has a fixed structure so even a slow algorithm for detection would be largely beneficial when solving the non-linear system using Newton's method. I feel that this is a common topic in solving systems of equations, so I am open to be redirected to additional ressources that are relevant.&lt;/p&gt;&#xA;" OwnerUserId="30153" LastEditorUserId="9667" LastEditDate="2019-03-07T13:26:03.253" LastActivityDate="2019-03-07T13:26:03.253" Title="Detecting blocks in non-linear system of equations" Tags="|nonlinear-equations|jacobian|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31183" PostTypeId="1" AcceptedAnswerId="31194" CreationDate="2019-03-05T16:44:12.003" Score="0" ViewCount="206" Body="&lt;p&gt;I have implemented an Adams Bashforth 4 method to solve an Initial Value Problem for an ODE and I am testing it against the test equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$y'=\lambda y$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$y(0)=1$&lt;/span&gt; with the exact solution: &lt;span class=&quot;math-container&quot;&gt;$y(t)=e^{\lambda t}$&lt;/span&gt;.&#xA;I have set &lt;span class=&quot;math-container&quot;&gt;$\lambda =1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get decent results and decent rate of convergence because the loglog plot is linear with slope of 4 (AD4 is of order 4 so the error is &lt;span class=&quot;math-container&quot;&gt;$O(h^4)$&lt;/span&gt;). Here is the global error when I am testing it in the interval &lt;span class=&quot;math-container&quot;&gt;$t \in [0,3]$&lt;/span&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/UB5r3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UB5r3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is this consistent with theory?&lt;/strong&gt; Also, &lt;strong&gt;where will you plot the &lt;span class=&quot;math-container&quot;&gt;$O(h^4)-$&lt;/span&gt;line in this graph?&lt;/strong&gt; I know it needs to be parallel to the AD4 line but where? In general, &lt;strong&gt;I don't really know the connection between the global error and &lt;span class=&quot;math-container&quot;&gt;$O(h^4)$&lt;/span&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my data&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h        = ( 0.1, 0.05, 0.01, 0.005)&#xA;ad4error = (3.477e-03, 2.493e-04, 4.445e-07, 2.816e-08)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30032" LastEditorUserId="30032" LastEditDate="2019-03-06T13:32:21.477" LastActivityDate="2019-03-07T10:58:35.627" Title="Actual global error vs theoretical global error: How to combine theory with practice" Tags="|finite-difference|ode|error-estimation|numerics|differential-equations|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31184" PostTypeId="1" CreationDate="2019-03-05T16:47:08.870" Score="3" ViewCount="128" Body="&lt;p&gt;In the realm of scientific computing, there are a plethora of techniques developed to solve Partial Differential Equations (PDEs). Many of the popular methods are variants of common techniques such as Finite Differences, Finite Volume, Finite Elements and the theory behind where they work or not is pretty extensive. Each method has pretty clear pros and cons which might influence its use over another for any given research problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was just now contemplating the idea of non-parametrically modeling a solution for some system of PDEs, say some hyperbolic system of PDEs to keep things interesting. The idea could be to essentially having some cloud of spacetime points with some solution coefficients that represent the approximate solution to the PDEs. Then a separate cloud of spacetime points could be used as collocation points used to &quot;train&quot; the model. Whenever we wish to evaluate our model and compute gradients at some spacetime location, we could perhaps do some least square fit based on data that is &quot;close&quot; to that spacetime location. The problem of actually solving the PDE might then be viewed as an optimization of the solution coefficients at the model's spacetime points such that the residual of the PDEs and boundary conditions are minimized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of hyperbolic PDEs, do you see such an approach as described working out? Do you see problems arising due to characteristics of the system of hyperbolic PDEs that might influence how one should evaluate the non-parametric model in spacetime? Are there any papers you know of covering similar concepts? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This idea came about thinking about some paper I read a while back about someone using a similar collocation approach but with a neural network as the approximate PDE solution instead. It seems to me a non-parametric model would be easier to optimize and would still be capable of capturing potential nonlinearities, at the expense of having to keep track of solution data at a likely good number of spacetime locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't thought too hard about this so perhaps there is something obvious I am missing but any thoughts, comments, or references about similar material would be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="20688" LastEditDate="2019-06-22T19:19:56.917" LastActivityDate="2019-06-22T19:19:56.917" Title="Non-parametric models as solutions to Partial Differential Equations" Tags="|optimization|pde|hyperbolic-pde|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31185" PostTypeId="2" ParentId="31178" CreationDate="2019-03-05T20:42:36.497" Score="4" Body="&lt;p&gt;I would call the constraint &quot;upper- and lower-bounds on the maximum element.&quot; Note that you are actually dealing with two separate constraints. Define the&#xA;max element function as follows&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\max:\mathbb{R}^{n}\to\mathbb{R}\qquad\max(x)\equiv\max_{i\in\{1,\ldots,n\}}x_{n}.&#xA;$$&lt;/span&gt;&#xA;Your first constraint is &quot;take the max element and ensure that it&#xA;is less than &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;&quot;: &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\max(x)&amp;lt;c,\tag{1}&#xA;$$&lt;/span&gt;&#xA;while the second is &quot;take the sum, subtract the max element, and&#xA;ensure that the result is less than &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;&quot;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mathbf{1}^{T}x-\max(x)&amp;lt;c,\tag{2}&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{1}=[1,\ldots,1]^{T}$&lt;/span&gt; is the usual column vector of&#xA;ones. Observe that (1) is a convex upper-bound on &lt;span class=&quot;math-container&quot;&gt;$\max(x)$&lt;/span&gt; while (2) is a &lt;strong&gt;nonconvex&lt;/strong&gt; lower-bound. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such problems with a fixed upper-bound have a very elegant solution&#xA;via the Big-M method. Let &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; be an arbitrary large number that satisfies&#xA;&lt;span class=&quot;math-container&quot;&gt;$\max(x)\le M$&lt;/span&gt; for all possible choices of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. Then it is easy&#xA;to verify that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\alpha=\max(x)\le M\quad\iff\quad\alpha\ge x\ge\alpha-(1-z)M,\quad\mathbf{1}^{T}z=1,\quad z\in\{0,1\}^{n}.&#xA;$$&lt;/span&gt;&#xA;Using the above identity, we can implement (1) and (2) &lt;strong&gt;exactly&lt;/strong&gt;&#xA;as the following mixed integer constraints&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x)&amp;lt;c\quad\iff\quad\begin{array}{c}&#xA;\mathbf{1}^{T}x-c&amp;lt;\alpha&amp;lt;c\\&#xA;\alpha-(1-z)c\le x\le\alpha\\&#xA;\alpha\in\mathbb{R},\quad z\in\{0,1\}^{n}&#xA;\end{array}&#xA;$$&lt;/span&gt;&#xA;where we have conveniently used &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; as the Big-M parameter. &lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2019-03-05T20:58:51.877" LastActivityDate="2019-03-05T20:58:51.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31186" PostTypeId="2" ParentId="31175" CreationDate="2019-03-05T22:19:48.680" Score="1" Body="&lt;p&gt;There is some literature on this. Such methods are generally associated with a property called &quot;path-conservative&quot;. The main player in this area, as far as I know, is Carlos Pares. He has several nice papers, though the math is somewhat involved. Check &lt;a href=&quot;http://benasque.org/benasque/2005pde/2005pde-talks/017sinum2005-def.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one out&lt;/a&gt;, for example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An additional point is that the product rule trick doesn't entirely get you out of trouble, as you now have a nonhomogeneous equation with derivatives on the source terms. I assume you are using a finite-volume approach to deal with shocks? If so, as far as I know, there isn't a quick way to get yourself out of trouble. Another reference on the adjoint part, in particular, is Giles &amp;amp; Stefan Ulbrich. They have a &lt;a href=&quot;https://people.maths.ox.ac.uk/gilesm/files/SINUM_2010a.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;few papers&lt;/a&gt; together on this topic.&lt;/p&gt;&#xA;" OwnerDisplayName="user20857" LastActivityDate="2019-03-05T22:19:48.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31189" PostTypeId="1" CreationDate="2019-03-06T13:10:19.193" Score="3" ViewCount="114" Body="&lt;p&gt;For a little personal project, I am picking up my FEM skills again. I learned a lot about the theory back in university and I am able to implement a simple FEM solver for specific problems but I was curious about the different parallelization possibilities because we never went into those too deeply. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us focus on the Poisson equation for simplicity in some &quot;nice&quot;  domain for n &gt;1. From what I have seen, one can&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Parallelize the assembly of the stiffness matrix. This takes quite some time for larger problems and can be parallelized easily. &lt;/li&gt;&#xA;&lt;li&gt;Parallelize the linear solver. Here there are lots of possibilities to construct specific solvers for sparse systems based on CG, multigrid or Krylov-based methods. Also possible but a little harder. &lt;/li&gt;&#xA;&lt;li&gt;Domain Decomposition. If I understood that topic correctly one can solve some sub-areas of the domain and divide the solution of that onto several processors. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am not a researcher in the topic but would like to gain a deeper understanding in the topic — mainly not by using some available library (I am well aware of FEniCS et al.) but code the first simple examples for parallelization by hand. For example, I keep reading that DG FEM is supposed to scale a little bit better on a parallel level and from some slides I can guess why — you have smaller assumptions on continuity, so the faces of your elements do not need to be coupled strongly. I have however trouble finding some extensive discussion with relating examples about this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot find really well-explained material on the last topic either, I know it exists but I couldn't find comprehensive material on that topic either. Most jump from the formulation of the discretization to &quot;… and then I used METIS plus xyz to solve. Here are three graphs showing convergence&quot;. It is of course also really tiring to try to reverse-engineer code found somewhere that &quot;seems&quot; to do what I think it does.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Questions&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;From those who know, are there known books on that topic I could read?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Could someone point me in the right direction?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Or would someone with some experience in those topics give a small introduction into the topic?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="25036" LastEditorUserId="9667" LastEditDate="2019-03-07T13:18:02.400" LastActivityDate="2019-03-07T15:25:48.327" Title="Parallelizing FEM for elliptical PDEs with n &gt;1" Tags="|finite-element|parallel-computing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31190" PostTypeId="2" ParentId="31166" CreationDate="2019-03-06T14:13:01.767" Score="1" Body="&lt;p&gt;As you may be aware, this is a fairly common issue in finite element models, since these are all about calculating integrals on shapes including n-simplices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your first question, the typical general approach for the codes I'm used to is simply to differentiate the reference to physical transformation, &lt;span class=&quot;math-container&quot;&gt;$$ \mathbf{x} = \sum_i \mathbf{x}_i N_i(\xi,\eta, \zeta),$$&lt;/span&gt; where the &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i$&lt;/span&gt;s are the positions of the Lagrange nodes and the &lt;span class=&quot;math-container&quot;&gt;$N_i$&lt;/span&gt;s are the equivalent shape functions, which for P2 look like &lt;span class=&quot;math-container&quot;&gt;$\xi(2\xi-1)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$4\xi\zeta$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\ldots$&lt;/span&gt; etc. The full set can be fairly trivially generated by introducing a fourth coordinate, &lt;span class=&quot;math-container&quot;&gt;$\sigma = 1-\xi-\eta-\zeta$&lt;/span&gt;, then taking the four versions of the form &lt;span class=&quot;math-container&quot;&gt;$a(2a-1)$&lt;/span&gt; and 6 combinations &lt;span class=&quot;math-container&quot;&gt;$4ab$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Jacobian matrix is then &lt;span class=&quot;math-container&quot;&gt;$$\mathcal{J}=\left(\begin{array}\mathcal{J}_{x\xi} &amp;amp; \mathcal{J}_{x\eta} &amp;amp; \mathcal{J}_{x\zeta}\\ \mathcal{J}_{y\xi} &amp;amp; \mathcal{J}_{y\eta} &amp;amp; \mathcal{J}_{y\zeta} \\ \mathcal{J}_{y\xi} &amp;amp; \mathcal{J}_{z\eta} &amp;amp; \mathcal{J}_{z\zeta}\end{array}\right)$$&lt;/span&gt; consists of terms such as &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \mathcal{J}_{x\xi}=\sum_i x_i\frac{\partial N_i}{\partial \xi}$$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If hand optimising, this can be calculated more quickly by noting that eg. only the shape functions involving &lt;span class=&quot;math-container&quot;&gt;$\xi$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\sigma$&lt;/span&gt; vary with &lt;span class=&quot;math-container&quot;&gt;$\xi$&lt;/span&gt;, so only 6 terms appear in each sum, which are linear functions of the reference coordinates. Typically the most complicated part is making sure that the numbering is consistent everywhere. The final operation is calculating the determinant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of this is particularly cheap, so it makes sense (unless your mesh is changing every tilmestep, and frequently even then) to cache the results. If you do that, then the cost is amortised across your time steps, and there isn't actually that much to be gained by trying to be much cleverer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your second question, if you are genuinely working on spherical triangles and really only want the areas, then you can probably get a significant time saving using &lt;a href=&quot;http://www.maths.qmul.ac.uk/~whitty/Oxford/Tauvpi/Girard.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Girard's Theorem&lt;/a&gt;,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \mbox{Area of spherical triangle} = \mbox{radius}^2\left(\sum \mbox{angles}-\pi\right).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the angle at vertex &lt;span class=&quot;math-container&quot;&gt;$\mathbf{a}$&lt;/span&gt; can be found using &lt;span class=&quot;math-container&quot;&gt;$$\cos\alpha = \frac{|\mathbf{a}\times\mathbf{b}\cdot\mathbf{a}\times\mathbf{c}|}{\|\mathbf{a}\times\mathbf{b}\|\|\mathbf{a}\times\mathbf{c}\|} .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All told, you'll have three cross products, three normalisations, three dot products and three trig functions, which I suspect will still beat trying to use high order quadrature and affine transformations.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2019-03-06T14:13:01.767" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31193" PostTypeId="2" ParentId="31175" CreationDate="2019-03-07T06:49:50.563" Score="1" Body="&lt;p&gt;Because the problem is linear, you don't need to deal with the complicated theory referenced in the answer from @spencerbryngelson.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a very nice discussion of problems of this kind in LeVeque's &lt;a href=&quot;http://depts.washington.edu/clawpack/book.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite Volume Methods for Hyperbolic Problems&lt;/a&gt;, Chapter 9.  Specifically, your system is very similar to the variable-coefficient acoustics equations (when written in non-conservative form) as discussed in section 9.6-9.13.  An accurate and efficient method for this problem is implemented in &lt;a href=&quot;http://www.clawpack.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clawpack&lt;/a&gt;.  Specifically, the Riemann solver for the system is &lt;a href=&quot;https://github.com/clawpack/riemann/blob/master/src/rp1_acoustics_variable.f90&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  To implement something similar for your problem, you'll need to determine the eigenvalues and eigenvectors of the Jacobian &lt;span class=&quot;math-container&quot;&gt;$\left(\frac{\mathrm{d}F}{\mathrm{d} U}(x)\right)^T$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2019-03-09T05:04:20.080" LastActivityDate="2019-03-09T05:04:20.080" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31194" PostTypeId="2" ParentId="31183" CreationDate="2019-03-07T10:58:35.627" Score="1" Body="&lt;p&gt;I'm going to turn my comment into an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The error order tells you up to which order (exclusive) the discrete&#xA;solution corresponds to the exact solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(\Delta t^4)$&lt;/span&gt; method reproduces the orders 0-3 of the Taylor expansion of the&#xA;function around &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; (i.e. in the expression &lt;span class=&quot;math-container&quot;&gt;$f(t+\Delta t) = f(t) +\dots$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put bluntly (and mathematically imprecisely)&#xA;if you halve the size of the time step &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; the deviation will be smaller by &lt;span class=&quot;math-container&quot;&gt;$2^4$&lt;/span&gt;.&#xA;This, of course, assumes that the error is computed at the same final time.&#xA;Or, if you want to use the log-log plot effectively you can say that by reducing &#xA;the step size by one order of magnitude &lt;span class=&quot;math-container&quot;&gt;$\Delta t = 10^{-1} \rightarrow \Delta t = 10^{-2}$&lt;/span&gt;&#xA;the deviation from the analytic solution will be reduced by 4 orders of magnitude, i.e.,&#xA;instead of &lt;span class=&quot;math-container&quot;&gt;$10^{-2}$&lt;/span&gt; it will be &lt;span class=&quot;math-container&quot;&gt;$10^{-6}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To visualize the growth of an error one could plot curves of different orders (&lt;span class=&quot;math-container&quot;&gt;$\Delta  t^2, \Delta t^3$&lt;/span&gt; etc.)&#xA;and try and compare the result to these curves. This is very finnicky in a log-log plot and&#xA;results in a rather confusing plot.&#xA;A better way is to define a function that will draw a slope triangle with appropriately&#xA;labelled sides close to the curve(s) you want to show the behaviour of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used this in the analysis of the implementation of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{P}_2,\mathcal{P}_3$&lt;/span&gt; &#xA;finite elements in my thesis, as can be seen in the following plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sfAaE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sfAaE.png&quot; alt=&quot;Finite element errors for a Poisson equation on a rectangle, using different FE orders.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this plot would be really confusing if I were to add an entire curve for each growth order. Admittedly, the slope triangles could have been placed better, but they do show the growth order (one power of &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; of mesh width corresponds to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; powers of &lt;span class=&quot;math-container&quot;&gt;$10$&lt;/span&gt; on the error axis). &lt;/p&gt;&#xA;" OwnerUserId="28993" LastActivityDate="2019-03-07T10:58:35.627" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31195" PostTypeId="2" ParentId="31164" CreationDate="2019-03-07T10:58:40.640" Score="2" Body="&lt;p&gt;Your question leads to two different considerations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you need to pick the right algorithm. Hence, the question if the matrices have any structure, should be considered. E.g., when the matrices are symmetric, a Cholesky decomposition is more efficient than LU. When you only need a limited amount of accuracy an iterative method can be faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, you need to implement the algorithm efficiently. To do so, you need to know the bottleneck of your algorithm. Is your implementation bound by the speed of the memory transfer or by the speed of the computation. Since you consider only &lt;span class=&quot;math-container&quot;&gt;$10 \times 10$&lt;/span&gt; matrices, your matrix should fit into the CPU cache completely. Thus, you should make use of the SIMD units (SSE, AVX, etc.) and cores of your processor, to do as many computations per cycle as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all, the answer to your question heavily depends on the hardware and matrices that you consider. There probably is no definite answer and you have to try out a few things to find an optimal method.&lt;/p&gt;&#xA;" OwnerUserId="24953" LastActivityDate="2019-03-07T10:58:40.640" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31196" PostTypeId="1" CreationDate="2019-03-07T12:14:23.957" Score="2" ViewCount="1345" Body="&lt;p&gt;I have a large system of homogenous inequalities involving 33 real unknowns of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec{F}(z_i)^T \cdot \vec{X}&amp;gt;0\,&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\vec{X} = \left(x_1,...,x_{24}\right)^T$&lt;/span&gt; are the unknowns and &lt;span class=&quot;math-container&quot;&gt;$\vec{F}(z_i)$&lt;/span&gt; are functions depending on a real parameter &lt;span class=&quot;math-container&quot;&gt;$|z_i|&amp;lt;1$&lt;/span&gt; which parametrises the system, that is to each value of &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; is associated an inequality. An example might be &lt;span class=&quot;math-container&quot;&gt;$\vec{F}(z) = \left(z^2+z\,,\, z^3+1/z,...,0.1 z^4 + 1/\sqrt{z}\right)^T $&lt;/span&gt; but I am interested into more general cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in finding a point &lt;span class=&quot;math-container&quot;&gt;$\vec{X}_0$&lt;/span&gt; which satisfy the system of inequalities resulting from the very big set of parameters &lt;span class=&quot;math-container&quot;&gt;$\{z_i\}$&lt;/span&gt;. Let say, we select 100 different parameters &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt;, then we have 100 inequalities to solve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to solve this problem with &lt;code&gt;Mathematica&lt;/code&gt;, which has a function &lt;code&gt;FindInstance&lt;/code&gt; which does this job. This function works very well for smaller systems, whereas it fails for large system of inequalities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a way to study this problem by using Python or C++?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there already dedicated libraries to solve such kind of problems?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I want to find a set of points &lt;span class=&quot;math-container&quot;&gt;$\{\vec{X}\}$&lt;/span&gt; which solves the system.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8188" LastEditorUserId="8188" LastEditDate="2019-03-14T07:12:08.263" LastActivityDate="2019-03-14T07:12:08.263" Title="Find a solution of large system of inequalities" Tags="|optimization|python|c++|linear-programming|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="31197" PostTypeId="1" CreationDate="2019-03-07T13:32:53.847" Score="0" ViewCount="95" Body="&lt;p&gt;I would like to get the power from a Gaussian beam given a set of points at which electric field is evaluated. Please follow my reasoning and tell me what assumption maybe are wrong &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Power definition is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ P = \int |E|^2dA$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We integrate over area, then Power is a quantity that does not depend on the geometry. If we compute the integral over a cross section of a Gaussian Beam or over the spherical surface, both results should be the same (I think ??)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To test so, I wrote a code to get the electric field on both regions (code below), and I'm here looking for help of how to integrate electric field on both scenarios.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Gaussian Electric Field&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm using expression (3.11) from this &lt;a href=&quot;https://www.photonics.ethz.ch/fileadmin/user_upload/Courses/NanoOptics/focus3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf&lt;/a&gt; and not changing the direction of propagation, it is a Gaussian beam of waist &lt;span class=&quot;math-container&quot;&gt;$w_0$&lt;/span&gt; traveling at &lt;span class=&quot;math-container&quot;&gt;$\hat{z}$&lt;/span&gt; direction &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Cross Section&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Looking at Stackoverflow &lt;a href=&quot;https://stackoverflow.com/questions/28567166/uniformly-distribute-x-points-inside-a-circle&quot;&gt;results&lt;/a&gt; I am using Sunflower Seed Arrangements with radius equal to the Gaussian waist&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Sphere&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Nothing special, I have matrix of points uniformly distributed in &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\phi$&lt;/span&gt; spherical coordinates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, my source code wrote in Julia language :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using Plots&#xA;plotly()&#xA;&#xA;function E_gaussian(x,y,z, k₀, w₀, E₀)&#xA;    first_term = exp(im*k₀*z)/(1 + 2*im*z/(k₀*w₀^2))&#xA;    second_term = exp(  - ((x^2 + y^2)/w₀^2)*(1/(1 + 2*im*z/(k₀*w₀^2)))  )&#xA;    E = E₀*first_term*second_term&#xA;    return E&#xA;end&#xA;&#xA;function createGaussianSection(nPoints, α, w₀ )&#xA;    b = round(α*sqrt(nPoints)) # number of boundary points&#xA;    ϕ = (sqrt(5)+1)/2 # golden ratio&#xA;    screen = zeros(nPoints,3)&#xA;    for k=1:nPoints&#xA;        r = radius_sunflower(k,nPoints,b)&#xA;        θ = 2*pi*k/ϕ^2&#xA;        screen[k,1] = w₀*r*cos(θ)&#xA;        screen[k,2] = w₀*r*sin(θ)&#xA;        screen[k,3] = 0&#xA;    end&#xA;    return screen&#xA;end&#xA;&#xA;function radius_sunflower(k,n,b)&#xA;    if k&amp;gt;n-b&#xA;        r = 1 # put on the boundary&#xA;    else&#xA;        r = sqrt(k-1/2)/sqrt(n-(b+1)/2) # apply square root&#xA;    end&#xA;    return r&#xA;end&#xA;&#xA;function getE_GassianSection(screen, k₀,  w₀, E₀)&#xA;    nPoints = size(screen,1)&#xA;    E_laser = zeros(Complex{Float64}, nPoints )&#xA;&#xA;    for k =1:nPoints&#xA;        xₛ = screen[k,1]&#xA;        yₛ = screen[k,2]&#xA;        zₛ = screen[k,3]&#xA;        E_laser[k] = E_gaussian(xₛ, yₛ, zₛ, k₀,  w₀, E₀)&#xA;    end&#xA;    return E_laser&#xA;end&#xA;&#xA;&#xA;&#xA;function createSphere(Radius, θNintervals, φNintervals, w₀, k₀)&#xA;    θ_range = range(0, stop=π, length=θNintervals)&#xA;    φ_range = range(0, stop=2π, length=φNintervals)&#xA;&#xA;    xₛ = Radius.*[cos(φ)*sin(θ) for θ in θ_range, φ in φ_range]&#xA;    yₛ = Radius.*[sin(φ)*sin(θ) for θ in θ_range, φ in φ_range]&#xA;    zₛ = Radius.*[cos(θ) for θ in θ_range, φ in φ_range]&#xA;&#xA;    screen = zeros(size(xₛ,1),size(xₛ,2),3)&#xA;    screen[:,:,1] = xₛ&#xA;    screen[:,:,2] = yₛ&#xA;    screen[:,:,3] = zₛ&#xA;&#xA;    return screen, θ_range, φ_range&#xA;end&#xA;&#xA;&#xA;function getE_Sphere(screen, k₀,  w₀, E₀)&#xA;    nΘ = size(screen,1)&#xA;    nΦ = size(screen,2)&#xA;    E_sphere = zeros(Complex{Float64},nΘ, nΦ )&#xA;&#xA;    for θ in 1:nΘ, φ in 1:nΦ&#xA;        xₛ = screen[θ,φ,1]&#xA;        yₛ = screen[θ,φ,2]&#xA;        zₛ = screen[θ,φ,3]&#xA;        E_sphere[θ,φ] = E_gaussian(xₛ, yₛ, zₛ, k₀,  w₀, E₀)&#xA;    end&#xA;    return E_sphere&#xA;end&#xA;&#xA;k₀ =  1&#xA;w₀ = 30/k₀&#xA;E₀ = 1&#xA;&#xA;screenGaussin = createGaussianSection(5000, 0, w₀)&#xA;E_section = getE_GassianSection(screenGaussin, k₀,  w₀, E₀)&#xA;Intensity_section = real.( conj.(E_section).*E_section)&#xA;scatter(screenGaussin[:,1], screenGaussin[:,2], zcolor=Intensity_section, axis=:equal, label=&quot;&quot;, xlabel=&quot;x&quot;, ylabel=&quot;y&quot;)&#xA;&#xA;&#xA;&#xA;Radius = 50/k₀&#xA;θNintervals = φNintervals = 25&#xA;screen, θ_range, φ_range = createSphere(Radius, θNintervals, φNintervals, w₀, k₀)&#xA;E_sphere = getE_Sphere(screen, k₀,  w₀, E₀)&#xA;Intensity_sphere = real.( conj.(E_sphere).*E_sphere)&#xA;scatter3d(screen[:,:,1],screen[:,:,2],screen[:,:,3], zcolor =Intensity_sphere , label=&quot;&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results are the following, the color is the local intensity:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/6Xi1c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6Xi1c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/iJKp1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iJKp1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From everything that I've shown, I only have &lt;strong&gt;Points&lt;/strong&gt;. How do I find &lt;strong&gt;Areas&lt;/strong&gt; and &lt;em&gt;integrate&lt;/em&gt; them ???&lt;/p&gt;&#xA;" OwnerUserId="29035" LastEditorUserId="29035" LastEditDate="2019-03-07T13:47:07.847" LastActivityDate="2019-12-02T17:02:45.303" Title="How do I get power from gaussian beam numerically?" Tags="|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31198" PostTypeId="1" CreationDate="2019-03-07T14:17:19.853" Score="6" ViewCount="294" Body="&lt;p&gt;I have a simple but long function that takes a vector x[10], and outputs a vector y[100]. It is an automatically generated eval function for a multivariate polynomial, ie, there is only (complex) multiplications and additions involved (5000 of them), with many variables to cache intermediate results that are reused. The structure goes like this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;static inline void&#xA;evaluate(const complex x[10], complex y[100])&#xA;{&#xA;  const complex &amp;amp;X0 = x[0];   // shortcuts to input&#xA;  const complex &amp;amp;X1 = x[1];&#xA;  ...&#xA;  const complex &amp;amp;X9 = x[9];&#xA;&#xA;  static constexpr complex C0 = 1;   // constants&#xA;  static constexpr complex C1 = -1;&#xA;&#xA;  const complex G0 = C1 * X8;        // multiply. G standds for 'gate'&#xA;  const complex G1 = C0 + G0;        // add.&#xA;  const complex G2 = G1 * X9;&#xA;  ...&#xA;  const complex G5000 = G3427 + G3430 + G3433 + G3436 + G3438;&#xA;&#xA;  y[0] = G104; // output&#xA;  y[1] = G135;&#xA;  y[2] = G195;&#xA;  ....&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to go about optimizing this function to the max on an Intel processor? For instance, reducing the number of multiplications or approximating them would help, since complex multiplications are expensive. &#xA;What I've tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I use &lt;code&gt;-O3 -ffast-math -march=native&lt;/code&gt; and other flags to GCC 5-7&lt;/li&gt;&#xA;&lt;li&gt;I profiled this and it has mostly L1 data write miss, and some L1 data read miss.&lt;/li&gt;&#xA;&lt;li&gt;Idea: the gates (G's) form a DAG, so if I recursively pack each set of independent variables into vectors, then one might be able to exploit SIMD instructions. Compilers do this during SLP auto-vectorization, see recent work (&lt;a href=&quot;https://dl.acm.org/citation.cfm?id=3314897&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;)(&lt;a href=&quot;http://rcor.me/papers/cgo18lslp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;), but I am unsure how much of it they handle.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is a visualization of the DAG for a small example:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;  G2 = G1 * X9;&#xA;  G3 = X8 * X6;&#xA;  G5 = G1 * X0;&#xA;  G6 = X8 * X7;&#xA;  G8 = X4 * X4;&#xA;  G9 = X5 * X5;&#xA;  G11 = X6 * X6;&#xA;  G12 = G11 + X6;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zpuai.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zpuai.png&quot; alt=&quot;Example DAG with 8 gates&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example visualization of 100 of these gates:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kPhTL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kPhTL.png&quot; alt=&quot;Example DAG with 100 gates&quot;&gt;&lt;/a&gt;&#xA;Each rank/line (and more) can be vectorized at a time. Do modern systems optimize this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: the variables are &lt;code&gt;const&lt;/code&gt; meaning this straightline program (no loops) is in &lt;a href=&quot;https://homepages.dcc.ufmg.br/~fernando/classes/dcc888/ementa/slides/StaticSingleAssignment.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Static Single Assignment (SSA) form&lt;/a&gt;, a term used in compiler optimization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After ordering the nodes for vectorization from the DAG (selecting a suitable topological ordering), I get 21 layers/ranks of vectorizable operations (1 rank not in table corresponds to the seeds: constants or vector X):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wYFpR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wYFpR.png&quot; alt=&quot;layer&quot;&gt;&lt;/a&gt;&#xA;Each entry in this table can be vectorized with SIMD, running from left to right. I'll be executing this function 1 billion times in my application.&lt;/p&gt;&#xA;" OwnerUserId="30112" LastEditorUserId="20688" LastEditDate="2019-06-22T21:20:31.980" LastActivityDate="2019-11-14T19:26:04.080" Title="Fast evaluation functions given by straight-line programs" Tags="|polynomials|compiling|complex-arithmetic|assembly|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="31199" PostTypeId="2" ParentId="31179" CreationDate="2019-03-07T15:25:08.500" Score="0" Body="&lt;p&gt;The most obvious advantage of FVM is that the conservation law(the flux is canceled between the interface of two neighboring cells) is satisfied automatically during the discretization process, no other special mathematical treatment needed. This property is very important for the fluid flow problem to satisfy the mass continuity equation.&lt;/p&gt;&#xA;" OwnerUserId="22404" LastActivityDate="2019-03-07T15:25:08.500" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31200" PostTypeId="2" ParentId="31189" CreationDate="2019-03-07T15:25:48.327" Score="3" Body="&lt;p&gt;I think you've identified some of the major issues. Yes, Finite Element methods are very parallel, because you have all these elements that can often be treated in parallel, but the devil is in the details. To address some of your points:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Assembly is an interesting point. For Finite Difference methods you can pretty much assemble the whole thing perfectly in parallel. With FE you get the phenomenon that two elements contribute integrals to their shared nodes/edges/faces. And what if two elements are evaluated by different processors? Then you have to resolve that simultaneous access.&lt;/li&gt;&#xA;&lt;li&gt;Solvers. A simple Jacobi method, apart from its limited convergence conditions, is fully parallel. Methods such as CG converge much faster and in more cases, but they contain inner products, which are something to worry about, especially if you get in the hundreds of thousands of processors/cores. There is tons of literature about this. Usually, the first issue to worry about is the matrix-vector product. For regular grids that is easy to parallelize, for irregular grids programming becomes a headache, certainly in distributed memory.&lt;/li&gt;&#xA;&lt;li&gt;Domain Decomposition. That is both a metaphorical term for &quot;domain partitioning&quot; and the name of a certain mathematical method that relies on the better conditioning of interface systems. Feel free to read up on trace theorems in functional analysis. So let me just address the partitioning part: yes, fruitful parallel methods are based on solving subproblems. For instsance there is Block Jacobi and the Additive Schwarz method. Both of these are typically used as preconditioners in a CG (or BiCGstab or GMRES) method.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A lot of these topics are discussed in my HPC book:&#xA;&lt;a href=&quot;http://pages.tacc.utexas.edu/~eijkhout/istc/istc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pages.tacc.utexas.edu/~eijkhout/istc/istc.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2019-03-07T15:25:48.327" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31201" PostTypeId="2" ParentId="31197" CreationDate="2019-03-07T16:02:24.193" Score="1" Body="&lt;p&gt;Thank you for people on inbox to help me. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't need full sphere, only half of sphere contains the same energy of the cross section.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I can use my point grid and imagine a small area between them. I set a point and create a square based upon the nearest neighbors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.1 In spherical coordinates, an element of area is &lt;span class=&quot;math-container&quot;&gt;$dA = r^2sin(\theta)d\theta d\phi$&lt;/span&gt;. To covert this expression into discrete case, we use &lt;span class=&quot;math-container&quot;&gt;$d\theta = \theta[i+1]-\theta[i]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d\phi= \phi[i+1]-\phi[i]$&lt;/span&gt;, and also &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is the average of the nearest neighbors &lt;span class=&quot;math-container&quot;&gt;$\theta = (\theta[i+1]+\theta[i])/2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.2 For simplicity, the cross section area can be just a square grid of points&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This procedure works only for far field, that is, the radius of the sphere should be big if compared with the Gaussian waist ( I've used a wrong choice of values in my example)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For everybody that is interested, here the full working code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function E_gaussian(x,y,z, k₀, w₀, E₀)&#xA;    first_term = exp(im*k₀*z)/(1 + 2*im*z/(k₀*w₀^2))&#xA;    second_term = exp(  - ((x^2 + y^2)/w₀^2)*(1/(1 + 2*im*z/(k₀*w₀^2)))  )&#xA;    E = E₀*first_term*second_term&#xA;    return E&#xA;end&#xA;&#xA;function createGaussianSection(nPoints, w₀ )&#xA;    x_range = range(-w₀, stop=w₀, length=nPoints)&#xA;    y_range = range(-w₀, stop=w₀, length=nPoints)&#xA;    screen = zeros(nPoints, nPoints, 3)&#xA;    screen[:,:,1] = [x for x in x_range, y in y_range]&#xA;    screen[:,:,2] = [y for x in x_range, y in y_range]&#xA;    screen[:,:,3] .= 0&#xA;    return screen, x_range, y_range&#xA;end&#xA;&#xA;&#xA;&#xA;function createSphere(Radius, θNintervals, φNintervals, w₀, k₀)&#xA;    θ_range = range(0, stop=π/2, length=θNintervals)&#xA;    φ_range = range(0, stop=2π, length=φNintervals)&#xA;&#xA;    xₛ = Radius.*[cos(φ)*sin(θ) for θ in θ_range, φ in φ_range]&#xA;    yₛ = Radius.*[sin(φ)*sin(θ) for θ in θ_range, φ in φ_range]&#xA;    zₛ = Radius.*[cos(θ) for θ in θ_range, φ in φ_range]&#xA;&#xA;    screen = zeros(size(xₛ,1),size(xₛ,2),3)&#xA;    screen[:,:,1] = xₛ&#xA;    screen[:,:,2] = yₛ&#xA;    screen[:,:,3] = zₛ&#xA;&#xA;    return screen, θ_range, φ_range&#xA;end&#xA;&#xA;&#xA;function getE(screen, k₀,  w₀, E₀)&#xA;    nΘ = size(screen,1)&#xA;    nΦ = size(screen,2)&#xA;    E_sphere = zeros(Complex{Float64},nΘ, nΦ )&#xA;&#xA;    for θ in 1:nΘ, φ in 1:nΦ&#xA;        xₛ = screen[θ,φ,1]&#xA;        yₛ = screen[θ,φ,2]&#xA;        zₛ = screen[θ,φ,3]&#xA;        E_sphere[θ,φ] = E_gaussian(xₛ, yₛ, zₛ, k₀,  w₀, E₀)&#xA;    end&#xA;    return E_sphere&#xA;end&#xA;&#xA;k₀ =  1&#xA;w₀ = 30/k₀&#xA;E₀ = 1&#xA;&#xA;&#xA;screenGaussin, x_range, y_range = createGaussianSection(150, 2*w₀)&#xA;E_section = getE(screenGaussin, k₀,  w₀, E₀)&#xA;Intensity_section = real.( conj.(E_section).*E_section)&#xA;&#xA;I_section = 0&#xA;for x = 1:(length(x_range)-1), y = 1:(length(y_range)-1)&#xA;    global I_section&#xA;    E_mean = (E_section[x, y] + E_section[x+1, y] + E_section[x, y+1]+E_section[x+1, y+1])/4&#xA;    ΔA = (x_range[x+1]-x_range[x])*(y_range[y+1]-y_range[y])&#xA;    I_section += real(conj(E_mean)*E_mean)*ΔA&#xA;end&#xA;println(&quot;Total Power Section: &quot;, round(I_section,digits=3))&#xA;&#xA;&#xA;Radius = 1000/k₀&#xA;θNintervals = φNintervals = 200&#xA;screen, θ_range, φ_range = createSphere(Radius, θNintervals, φNintervals, w₀, k₀)&#xA;E_sphere = getE(screen, k₀,  w₀, E₀)&#xA;Intensity_sphere = real.( conj.(E_sphere).*E_sphere)&#xA;&#xA;I_sphere = 0&#xA;for θ = 1:(θNintervals-1), φ = 1:(φNintervals-1)&#xA;    global I_sphere&#xA;    E_medio = (E_sphere[θ, φ] + E_sphere[θ+1, φ] + E_sphere[θ, φ+1]+E_sphere[θ+1, φ+1])/4&#xA;    ΔA = (Radius^2)*sin(   (θ_range[θ]+θ_range[θ+1])/2  )*(θ_range[θ+1]-θ_range[θ])*(φ_range[φ+1]-φ_range[φ])&#xA;    I_sphere += real(conj(E_medio)*E_medio)*ΔA&#xA;end&#xA;println(&quot;Total Power Sphere: &quot;, round(I_sphere,digits=3))&#xA;&#xA;println(&quot;Relation:&quot;,round(I_section/I_sphere,digits=3))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Total Power Section: 1413.028&#xA;Total Power Sphere: 1413.567&#xA;Relation:1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29035" LastActivityDate="2019-03-07T16:02:24.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31202" PostTypeId="2" ParentId="31179" CreationDate="2019-03-07T19:07:28.370" Score="2" Body="&lt;p&gt;The most fundamental reason is that (static) structural problems often resolve themselves into finding minimum energy configurations and this is easily translated into minimizing the energy of a FE representation. This almost never happens for fluid flow, even if is steady, because of the convective terms. There are by now many FE codes for fluid flow (such as the discontinuous Galerkin method) that have their enthusiasts, but for most engineering purposes they are regarded as too expensive.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2019-03-07T19:07:28.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31203" PostTypeId="2" ParentId="31196" CreationDate="2019-03-07T19:19:34.297" Score="1" Body="&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$\vec{F}(z_i)$&lt;/span&gt; is a constant once the parameter &lt;span class=&quot;math-container&quot;&gt;$z_i$&lt;/span&gt; has been chosen, the problem amounts to solving a system &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}\vec x&amp;gt;0$&lt;/span&gt; for which &lt;span class=&quot;math-container&quot;&gt;$\vec x =0$&lt;/span&gt; is trivially a feasible point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that no optimization software implements strict inequalities. One reason for this is that the difference between say, 1e-999 and 0 is really, really small. So, operationally, a strict equality and an equality shakeout to the same answer. Let's explore this a bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cvxpy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cvxpy&lt;/a&gt; is a Python package that allows you to write a linear program in a generic way and then pass it to one of a large number solvers. Like most such programs, it does not allow for strict inequalities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've written a representation of your problem below:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import cvxpy as cp&#xA;import numpy as np &#xA;&#xA;def f(z):&#xA;  return np.array([&#xA;    z**2+z-3,&#xA;    z**3+1/z,&#xA;    0.1*z**4+1/np.sqrt(z),&#xA;    -4&#xA;  ])&#xA;&#xA;#Generate 100 random numbers in [-1,1)&#xA;zs = np.random.uniform(low=-1, high=1, size=100)&#xA;#Run the function on each number&#xA;zs = [f(z) for z in zs]&#xA;#Strip out results with a NaN&#xA;zs = [z for z in zs if not any(np.isnan(z))]&#xA;&#xA;zs = [[-1,-1,-1,-1],[1,1,1,1]]&#xA;&#xA;#x vector of length four to match length of vector from `f`&#xA;x = cp.Variable(4)&#xA;&#xA;#Multiply f(z)'s by x's and build constraints&#xA;cons = [z*x&amp;gt;=0 for z in zs]&#xA;&#xA;#Constant objective implies a problem in which we only want to find a feasible&#xA;#point&#xA;obj = cp.Maximize(1) &#xA;&#xA;#Create a problem with the objective and constraints&#xA;prob = cp.Problem(obj, cons)&#xA;&#xA;#Solve problem, get optimal value&#xA;val = prob.solve()&#xA;&#xA;if val==-np.inf:&#xA;  print(&quot;NO SOLUTION FOUND&quot;)&#xA;else:&#xA;  print(x.value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we run the above code, we find that the solver indeed finds the 0-vector as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, you would like your answer to be strictly greater than zero. Since there is no solver that will do this (that I know of), one way to achieve it is to ask the solver to find a solution to a different problem. Rather than solving &lt;span class=&quot;math-container&quot;&gt;$Ax&amp;gt;0$&lt;/span&gt;, we can ask the solver to find &lt;span class=&quot;math-container&quot;&gt;$Ax\ge\epsilon$&lt;/span&gt;. Where &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt; is some small value. Since all you want is a feasible point, this is a reasonable approach since any solution to &lt;span class=&quot;math-container&quot;&gt;$Ax\ge\epsilon$&lt;/span&gt; is also a solution to &lt;span class=&quot;math-container&quot;&gt;$Ax&amp;gt;0$&lt;/span&gt;. This is the simple solution you thought you were looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can implement this by changing the line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cons = [z*x&amp;gt;=0 for z in zs]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cons = [z*x&amp;gt;=0.0001 for z in zs]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running the code again, I get the answer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ 1.34351053e-09 -9.62045447e-10  5.11738862e-09 -3.99990907e-05]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(You may get a different answer because the solvers leverage stochasticity.) This looks promising! But there's a caveat here... What if we use this program?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import cvxpy as cp&#xA;import numpy as np &#xA;&#xA;#A system with no solution&#xA;zs = [[-1,-1,-1,-1],[1,1,1,1]]&#xA;&#xA;#x vector of length four to match length of vector from `f`&#xA;x = cp.Variable(4)&#xA;&#xA;#Multiply f(z)'s by x's and build constraints&#xA;cons = [z*x&amp;gt;=0.0001 for z in zs]&#xA;&#xA;#Constant objective implies a problem in which we only want to find a feasible&#xA;#point&#xA;obj = cp.Maximize(1) &#xA;&#xA;#Create a problem with the objective and constraints&#xA;prob = cp.Problem(obj, cons)&#xA;&#xA;#Solve problem, get optimal value&#xA;val = prob.solve()&#xA;&#xA;if val==-np.inf:&#xA;  print(&quot;NO SOLUTION FOUND&quot;)&#xA;else:&#xA;  print(x.value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's obvious from inspection that there can be no solution satisfying this system. Interestingly, however, when you run the program (using the default solver) the 0-vector is returned as an answer! If we modify the program to read:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cons = [z*x&amp;gt;=0.001 for z in zs]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the same thing happens. Only when we get to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cons = [z*x&amp;gt;=0.01 for z in zs]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;do we finally get the correct response: that there is no solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few reasons for this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Internally, the solver is using a floating-point representation whose limited precision results in it &lt;em&gt;thinking&lt;/em&gt; it's solved the problem when it really hasn't. (You could deal with this by using a rational-number solver that does its calculations using fractional representations.)&lt;/li&gt;&#xA;&lt;li&gt;More generally, the solver may not be numerically robust. The subfield of &quot;robust optimization&quot; can be leveraged to find disciplined ways of handling this.&lt;/li&gt;&#xA;&lt;li&gt;Philosophically, asking the solver to differentiate between small values of epsilon and 0 is silly. Say you're optimizing the floor plan of a house using numbers which represent metres and choose epsilon as 1e-10. You're asking for a solution that differs from zero by the width of an atom. Say you're calculating a solar trajectory with numbers representing astronomical units (1 AU is the distance from Earth to the sun - 93 million miles): the difference between 1e-10 and 0 is 50 feet (the width of a house).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Perhaps the simplest way of dealing with the problems above is to rescale your system so that small values &lt;em&gt;are&lt;/em&gt; unimportant. For instance, rather than measuring my floorplan in meters, I could measure it in millimeters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth noting that there is an entire class of problems for which your question has trivial and reliable solutions. If any column of your &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; matrix contains only positive values greater than zero, then setting that column's corresponding &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; value to 1 and all other &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; values to 0 provides an answer. Similarly, if any column contains only negative values then choosing -1 for the corresponding &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; value and 0 elsewhere provides an answer.&lt;/p&gt;&#xA;" OwnerUserId="17088" LastEditorUserId="17088" LastEditDate="2019-03-12T00:55:44.763" LastActivityDate="2019-03-12T00:55:44.763" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="31205" PostTypeId="1" CreationDate="2019-03-08T13:16:59.420" Score="2" ViewCount="635" Body="&lt;p&gt;I'm trying to simulate the scattering of a wave-packet at a potential barrier in Python. I'm using a Fourier Transform method (not sure if its the same as the Split-Step method), where I apply Fourier Transform on my initial wavepacket (plane wave in a gaussian envelope) to get the wavefunction in momentum-space:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\phi(k) = \int^{\infty}_{-\infty} \psi(x) e^{ikx} dx $$&lt;/span&gt;&#xA;I then calculate the dispersion term, &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt;, at each &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; value, &lt;span class=&quot;math-container&quot;&gt;$\omega = \frac{\hbar k^2}{2m}$&lt;/span&gt;. I can then incorporate the time-evolution operator &lt;span class=&quot;math-container&quot;&gt;$e^{-i\omega t}$&lt;/span&gt;. This allows me to obtain &lt;span class=&quot;math-container&quot;&gt;$\psi(x)$&lt;/span&gt; at a given time:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\psi(x,t) = \int^{\infty}_{-\infty} \phi(k) e^{-ikx}e^{-i\omega t}$$&lt;/span&gt;&#xA;which I achieve through an Inverse Fourier Transform of &lt;span class=&quot;math-container&quot;&gt;$\phi(k)e^{-i\omega t}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My wavepacket moves as expected, but I am now trying to incorporate a potential barrier. To start, I am trying to get my wave-packet to reflect off of a barrier (like a particle in a box). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a bit unsure of how to implement this into my code. This problem is usually described by having both a incoming and reflected wave in the region before the barrier, so I have implemented a 'reflected' wave-packet which is simply a mirror image of the initial wave-packet with its amplitudes reversed. This can be manipulated to simulate the wave reflecting off the barrier but I'm wondering if there is a more trivial solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a way to incorpate reflection/transmission at a barrier using my method of propagating the wave-packet?&lt;/li&gt;&#xA;&lt;li&gt;I've seen this problem solved using a finite difference method to solve the Schrodinger equation instead, is there an advantage of using that method?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30176" LastActivityDate="2021-04-09T20:03:40.473" Title="Incorporating a potential barrier in a wave-packet simulation (Fourier Transform method)" Tags="|python|fourier-analysis|quantum-mechanics|wave-propagation|fourier-transform|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31206" PostTypeId="2" ParentId="31198" CreationDate="2019-03-09T01:18:59.200" Score="3" Body="&lt;p&gt;Once you're at the level of a long list of expressions, there is little you can still do other than hope that the compiler finds opportunities for optimization at the assembly level. This may bring you 10 or 20% but not something that's going to make a meaningful difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you ought to do is go back to the original formula you're trying to implement. There may be more efficient ways to look at the whole problem, for example through the use of loops, iterations, reformulations of the mathematical statement, etc. This is the kind of thing that might be able to accelerate your function by a factor of ten -- factors that matter. But none of this is accessible any more in the code you show, and no compiler will be able to undo the transformations that were made to come up with the code you show.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-03-09T01:18:59.200" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31210" PostTypeId="1" CreationDate="2019-03-10T15:06:53.430" Score="1" ViewCount="35" Body="&lt;p&gt;I am trying to implement a simple logistic regression algorithm from scratch in python (for learning purposes). Every article I've seen online so far presents the following expression for &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; (argument of the sigmoid function):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$z = \theta^T\cdot x$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when they implement it in code they always use &lt;code&gt;np.dot(X, theta)&lt;/code&gt; which is not the same thing. I have carefully tried to trace the dimensions of all the arrays as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dot product property: &lt;span class=&quot;math-container&quot;&gt;$\theta^T\cdot x = x^T\cdot\theta$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is of dimension &lt;span class=&quot;math-container&quot;&gt;$l\times (d+1)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; is the number of records and &lt;span class=&quot;math-container&quot;&gt;$d+1$&lt;/span&gt; the number of features (including the &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; ones vector). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; is of dimensions &lt;span class=&quot;math-container&quot;&gt;$1 \times (d+1)$&lt;/span&gt; (every column in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; gets a weight, including the ones vector).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; is the dot product of &lt;span class=&quot;math-container&quot;&gt;$\theta^T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, it will have dimensions &lt;span class=&quot;math-container&quot;&gt;$((d+1)\times 1)\cdot(l \times 1) =  ??$&lt;/span&gt; --&gt; &lt;strong&gt;DOES NOT WORK&lt;/strong&gt;!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There appears to be no way to arrange the last statement such that it remains mathematically correct and yielding an array of dimension &lt;span class=&quot;math-container&quot;&gt;$l\times1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$1\times l$&lt;/span&gt; without fundamentally changing either:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the shape of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;the equation itself (as they do in the code)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So which is correct? The code or the mathematical statement? Why are they contradictory? Please help. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="22736" LastActivityDate="2019-04-10T06:01:30.267" Title="Want to make sense of array dimensions in logistic regression algorithms" Tags="|machine-learning|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31211" PostTypeId="1" AcceptedAnswerId="31212" CreationDate="2019-03-10T17:12:06.137" Score="1" ViewCount="268" Body="&lt;p&gt;I am trying to simulate multiple harmonic oscillators in periodic boundary conditions (subsequently visualizing the process in VMD). I have successfully simulated multiple HOs by using the Leapfrog algorithm, but when I try to impose the periodic boundary conditions on the simulations, unexplainable things start to happen. Below is my code - I am new to python, so I know it is definitely not the most efficient, but the way it is written makes to me a lot of intuitive sense and I cannot see, what is wrong.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import numpy.random as npr&#xA;&#xA;# distance vector between two particles&#xA;def distance_vec(pos_vec1, pos_vec2):&#xA;    return pos_vec1 - pos_vec2&#xA;&#xA;# norm of the above distance vector&#xA;def distance(dist_vec):&#xA;    return np.linalg.norm(dist_vec)&#xA;&#xA;# script for writing the .xyz file for visualization in VMD&#xA;def write_xyz(data, filename) -&amp;gt; None:&#xA;    with open(filename, 'w') as f:&#xA;        for j in range(data.shape[1]):&#xA;            print(f'{data.shape[0]}\n', file=f)&#xA;            for i in range(data.shape[0]):&#xA;                print(f'p{i}   {data[i, j, 0]}   {data[i, j, 1]}   0', file=f)&#xA;&#xA;############################ Simulation parameters #############################&#xA;&#xA;N = 2                                        # Number of particles in the HO&#xA;M = 5                                        # Number of HOs&#xA;num_step = 10000                             # Number of timesteps&#xA;dt = 0.001                                   # Timestep size (accuracy)&#xA;dim = 2                                      # Number of dimensions&#xA;&#xA;######################### Periodic boundary conditions #########################&#xA;&#xA;min_x = -10&#xA;max_x = 10&#xA;min_y = -10&#xA;max_y = 10&#xA;&#xA;############################# Spring properties ################################&#xA;&#xA;bond_length = 5                              # Bond length (equilibrial position)&#xA;k = 30                                       # Spring constant&#xA;&#xA;########################## Particle properties #################################&#xA;&#xA;m = np.array([1,1])                          # Particle masses&#xA;&#xA;######################## Randomized initial conditions ##########################&#xA;&#xA;x1 = npr.uniform(-10,10,(M, dim))              # Particle 1, initial position&#xA;x2 = npr.uniform(-10,10,(M, dim))              # Particle 2, initial position&#xA;&#xA;v1 = npr.uniform(-10,10,(M, dim))              # Particle 1, initial velocity&#xA;v2 = npr.uniform(-10,10,(M, dim))              # Particle 2, initial velocity&#xA;&#xA;################################ Simulation #####################################&#xA;&#xA;particles = np.zeros((N*M, num_step, dim))&#xA;&#xA;for i in range(M):&#xA;&#xA;    # initial distance calculation&#xA;    d_vec = distance_vec(x1[i, :], x2[i, :])        # not normalized&#xA;    d = distance(d_vec)&#xA;&#xA;    for j in range(num_step):&#xA;&#xA;        # Leapfrog steps 1 and 2&#xA;        F1 = -k * d_vec * (1 - (bond_length / d))&#xA;        F2 = k * d_vec * (1 - (bond_length / d))&#xA;&#xA;        # Leapfrog step 3&#xA;        v1[i, :] = v1[i, :] + (dt / m[0]) * F1&#xA;        v2[i, :] = v2[i, :] + (dt / m[1]) * F2&#xA;&#xA;        # Leapfrog step 4&#xA;        x1[i, :] = x1[i, :] + v1[i, :] * dt&#xA;        x2[i, :] = x2[i, :] + v2[i, :] * dt&#xA;&#xA;        #Periodic boundaries in x-axis&#xA;        if x1[i, 0] &amp;lt; min_x:&#xA;            x1[i, 0] = max_x - (min_x - x1[i, 0])&#xA;        elif x1[i, 0] &amp;gt; min_x:&#xA;            x1[i, 0] = min_x + (x1[i, 0] - max_x)&#xA;&#xA;        if x2[i, 0] &amp;lt; min_x:&#xA;            x2[i, 0] = max_x - (min_x - x1[i, 0])&#xA;        elif x2[i, 0] &amp;gt; min_x:&#xA;            x2[i, 0] = min_x + (x1[i, 0] - max_x)&#xA;&#xA;        #Periodic boundaries in y-axis&#xA;        if x1[i, 1] &amp;lt; min_y:&#xA;            x1[i, 1] = max_y - (min_y - x1[i, 1])&#xA;        elif x1[i, 1] &amp;gt; min_y:&#xA;            x1[i, 1] = min_y + (x1[i, 1] - max_y)&#xA;&#xA;        if x2[i, 1] &amp;lt; min_y:&#xA;            x2[i, 1] = max_y - (min_y - x1[i, 1])&#xA;        elif x2[i, 1] &amp;gt; min_y:&#xA;            x2[i, 1] = min_y + (x1[i, 1] - max_y)&#xA;&#xA;        # Saving positions/timestep&#xA;        particles[2 * i, j, :] = x1[i, :]&#xA;        particles[2 * i + 1, j, :] = x2[i, :]&#xA;&#xA;        # Distance calculation in periodic boundaries&#xA;        d_vec = distance_vec(x1[i, :], x2[i, :])&#xA;&#xA;        if d_vec[0] &amp;gt; ((max_x - min_x)/2):&#xA;            d_vec[0] = (max_x - min_x) - d_vec[0]                 &#xA;        elif d_vec[1] &amp;gt; ((max_y - min_y)/2):&#xA;            d_vec[1] = (max_y - min_y) - d_vec[1]&#xA;&#xA;        d = distance(d_vec)&#xA;&#xA;# Visualizing trajectory&#xA;write_xyz(particles, str(M) + &quot;xHO_2D.xyz&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If anyone has an idea about what is going wrong, I would really appreciate it. Thank you for your help!&lt;/p&gt;&#xA;" OwnerUserId="29843" LastActivityDate="2019-03-10T18:12:59.027" Title="Harmonic oscillators with periodic boundary conditions" Tags="|boundary-conditions|simulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31212" PostTypeId="2" ParentId="31211" CreationDate="2019-03-10T18:12:59.027" Score="2" Body="&lt;p&gt;You seem to be applying periodic boundary corrections wrongly to your dvec, as well as to your vector x2. The x2 problem is just that you've copied part of the formula for x1, without changing 1 to 2. Really, checking your code is not what this site is for, but I couldn't just let this slip by.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The formula for dvec should be very similar to the one you use to correct absolute positions. In words, you want to add or subtract &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, (or in general, multiples of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, but this is not essential if you are always correcting the positions), where &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the box length, so as to bring the components of the vector into the range &lt;span class=&quot;math-container&quot;&gt;$\pm \frac{1}{2}L$&lt;/span&gt;. That's not what you are doing: there are mistakes in the tests that you are doing (the if ... elif tests, which mix up what you are doing to the two independent components of dvec), and then the actual algebra performed on the components, which does not match the formula you are applying to x1 and x2. This is the main problem with your approach.&lt;/p&gt;&#xA;" OwnerDisplayName="user28077" LastActivityDate="2019-03-10T18:12:59.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31213" PostTypeId="2" ParentId="31210" CreationDate="2019-03-11T05:46:09.927" Score="1" Body="&lt;p&gt;They’re doing a dot product across each record using one call to &lt;code&gt;np.dot(.,.)&lt;/code&gt;. They are doing this to take advantage of fast numpy operations on the whole dataset instead of having you loop through computing &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; for each record individually using explicit Python code. They are essentially saying that &lt;span class=&quot;math-container&quot;&gt;$X = [x_1, x_2, \cdots, x_n]^T$&lt;/span&gt; and doing &lt;span class=&quot;math-container&quot;&gt;$Z = X \theta$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$Z = [z_1, \cdots, z_n]^T$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$z_k = \theta^T x_k$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2019-03-11T05:46:09.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31214" PostTypeId="2" ParentId="31179" CreationDate="2019-03-11T12:19:13.190" Score="9" Body="&lt;p&gt;The finite element method is actually quite widely used in fluid flow problems, for example for the Stokes and Navier-Stokes equations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The delineation between the methods is more along the following line:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The finite element method is very well suited for second order (in space) differential equations. That has something to do with the fact that trial and test spaces can be chosen the same in that case, and consequently the FEM is a tool that is well suited to problems such as elasticity and more generally structures, but also for the Stokes equation or the Navier-Stokes equations in fluids.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Many other fluid flow problems are described by first order differential equations (think, for example, the Euler equations of gas dynamics) or equations in which the first order terms are dominant (e.g., for the compressible Navier-Stokes equations in the transonic regime). For these cases, solutions are often discontinuous -- for example in the form of shocks -- and the Galerkin method (on which the FEM is based) produces oscillatory approximations that are not useful in practice. One can fix this through stabilization techniques developed over the past twenty or so years, but at least historically the preferred approach is to base methods on the conservation properties that underlie the equations -- i.e., to use finite volume methods.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-03-11T12:19:13.190" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="31215" PostTypeId="2" ParentId="31159" CreationDate="2019-03-11T14:23:58.540" Score="3" Body="&lt;p&gt;I found the following package&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.pci.uni-heidelberg.de//tc/usr/mctdh/doc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pci.uni-heidelberg.de//tc/usr/mctdh/doc/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;featuring the multiconfiguration time-dependent Hartree (MCTDH) method for distinguishable particles. The method is recommended for up to 12 degrees of freedom, though bigger systems have been treated , too.&lt;/p&gt;&#xA;" OwnerUserId="1117" LastActivityDate="2019-03-11T14:23:58.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31216" PostTypeId="1" CreationDate="2019-03-11T19:07:21.190" Score="2" ViewCount="3043" Body="&lt;p&gt;I am following a course of computational material physics. The professor uses fortran to code and uses lapack to solve eigenvalue problems.&#xA;So far I just know c++. There is an equivalent library that make the same as LAPACK but in c++? Which is the best one ?&#xA;Should I prefer to use fortran rather than c++ for my scope?&#xA;Thank you&lt;/p&gt;&#xA;" OwnerUserId="30200" LastActivityDate="2019-03-14T00:10:49.837" Title="LAPACK equivalent on c++ , which is the best one?" Tags="|c++|fortran|lapack|" AnswerCount="1" CommentCount="3" ClosedDate="2019-03-13T00:50:27.217" ContentLicense="CC BY-SA 4.0" />
  <row Id="31217" PostTypeId="1" CreationDate="2019-03-12T07:04:36.653" Score="-1" ViewCount="546" Body="&lt;p&gt;I am solving an advection-diffusion equation using the FEM and am having trouble calculating my fluxes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start with the equation,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial n}{\partial t} = \frac{\partial j_{n}}{\partial x}\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where my flux (current density) is given by,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$j_n = \mu E n + D \frac{\partial n}{\partial x}\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where  &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is my drift velocity and &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; the diffusion coefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this equation using FEM-FCT to ensure stability I obtain my nodal values of &lt;img src=&quot;https://latex.codecogs.com/gif.latex?n&quot; alt=&quot;n&quot;&gt;, which I then use to solve for my fluxes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue I am having currently is that my steady state current density should be zero, where an in-built field causes the drift and diffusion terms to balance, however I am obtaining non-zero current densities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to determine the cause of this, I plotted the two components of the current density in the following plot, and as is seen the two are not equal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PR50Y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PR50Y.png&quot; alt=&quot;Drift and Diffusion Flux at steady state.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering if it has to do with how the drift flux uses the calculated &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; directly while the diffusion flux requires a derivative which will introduce error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help in generating more accurate fluxes would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A few notes:&lt;/strong&gt; FEM-FCT limits me to linear basis functions, however I tried to create a higher order first derivative matrix just using finite differences however that did not seem to fix the problem.&lt;/p&gt;&#xA;" OwnerUserId="30203" LastEditorUserId="9667" LastEditDate="2019-03-12T13:53:41.730" LastActivityDate="2019-12-15T00:02:30.337" Title="Finite Element - Flux Calculation" Tags="|finite-element|fluid-dynamics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31219" PostTypeId="1" AcceptedAnswerId="31223" CreationDate="2019-03-12T10:02:31.793" Score="2" ViewCount="77" Body="&lt;p&gt;Given a linear system &lt;span class=&quot;math-container&quot;&gt;$A\textbf{x}=\textbf{b}$&lt;/span&gt;, we can express it into the easier-to-solve right-preconditioned form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ AM^{-1}\textbf{y}=\textbf{b}, \quad \textbf{y}= M\textbf{x} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, the left-preconditioned system is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ M^{-1}A\textbf{x}=M^{-1}\textbf{b} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the literature we know it's easy to show that the left-preconditioned linear system above is equivalent to a fixed-point linear iteration as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \textbf{x}^{(k+1)}=(I-M^{-1}A)\textbf{x}^{(k)}+M^{-1}\textbf{b} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or equivalently:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \textbf{x}^{(k+1)}=\textbf{x}^{(k)} + M^{-1}\textbf{r}^{(k)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how can I get an update scheme like those starting from the right-preconditioning formulation?&lt;/p&gt;&#xA;" OwnerUserId="30204" LastEditorUserId="17294" LastEditDate="2019-03-12T22:15:54.330" LastActivityDate="2019-03-12T22:15:54.330" Title="Right-preconditioning and fixed point linear iterations" Tags="|linear-solver|preconditioning|fixed-point|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31220" PostTypeId="1" AcceptedAnswerId="31253" CreationDate="2019-03-12T13:20:55.877" Score="3" ViewCount="359" Body="&lt;p&gt;&lt;strong&gt;EDIT: I moved the full code to &lt;a href=&quot;https://github.com/VogelAnt/Fab_approx/blob/master/Fahs_approach.m&quot; rel=&quot;nofollow noreferrer&quot;&gt;my github page&lt;/a&gt; so the post can be read more easily.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am writing a script to take the Faber approximation approach outlined in &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0307904X11008560&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hassan Fahs paper&lt;/a&gt; (free access)&#xA;and apply it to the Liouville-von-Neumann equation to propagate the density matrix &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; with some inspiration from the scripts found &lt;a href=&quot;https://arxiv.org/abs/1808.00416&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for general structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently the approach is stuck with the approximation of the exponential.&#xA;The idea behind the propagation scheme is to compute the update step as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\rho_{n+1}(\tau) = exp({\mathcal{L}\tau})\rho_n = \sum_{m=0}^{\infty}c_m(\tau) F_m(\mathcal{L})\rho_n.$$&lt;/span&gt;&#xA;As we can see the Faber polynomials are matrix valued, and perhaps -if you have taken a look into the script or the paper-you will see that the  the spectrum of &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; has to be scaled with a factor I will simply refer to as &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; in order to preserve stability, which in turn means the time step has to be adapted to &lt;span class=&quot;math-container&quot;&gt;$c*\tau =\tilde{\tau}$&lt;/span&gt;&#xA;So when we approximate the exponential we can see that our propagation scheme becomes:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\rho_{n+1}(\tau*c)=\rho_{n+1}(\tilde{\tau}) = exp({\frac{\mathcal{L}}{c}\tau c}\rho_n) = \sum_{m=0}^{\infty}c_m(\tilde{\tau}) F_m(\frac{\mathcal{L}}{c})\rho_n.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The matrix valued Faber polynomial recurrence relation for an elliptic domain in my case is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$F_{m+1}(\mathcal{L})\rho(0) = (\mathcal{L}-b_0\mathcal{I})F_m(\mathcal{L})\rho(0) - b_1 F_{m-1}(\mathcal{L})\rho(0).$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; denotes the Liouville superoperator.&#xA;Obviously, &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; is a function of the density matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After initialising a basic density matrix and computing the factors as in the paper above I tried to implement the algorithm as outlined by Dr . Fahs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fUIRM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fUIRM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But before trying to compute the full scheme I simply wanted to see if the approximation of the exponential&#xA;&lt;span class=&quot;math-container&quot;&gt;$$exp(\mathcal{L}_{sc}\tilde{\tau})= \sum_{m=0}^{\infty}c_m(\tilde{\tau}) F_m(\mathcal{L}_{sc})$$&lt;/span&gt;&#xA;was correct.&#xA;In order to do so tried to compare my solution for 100 iterations with the standard diagonalisation approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting picture is created by plotting the third line of a matrix whose columns are filled with the main diagonal of the respective matrix.&#xA;The blue line is the diagonalisation approach, the orange one mine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HDySX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HDySX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oddly, my approximation stays arround 1 for the entirety of the execution of the script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've traced the issue to the polynomial recurrence relation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$F_1...F_m$&lt;/span&gt; all depend on the factors of the Laurent expansion &lt;span class=&quot;math-container&quot;&gt;$b_0, b_1$&lt;/span&gt; and the scaled Liouvillian &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_sc$&lt;/span&gt;. They barely change, making me wonder if the algorithm for the approximation mentioned above is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code snippet for the computation of the Faber coefficients and Matrix valued polynomial recurrence relation, as well as the summing up of all the factors:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;% Faber approximation of the matrix exponential&#xA;function z = faber(L, dt, N)&#xA;&#xA;    % Faber coefficients for ellipic domain&#xA;    c_m = @(dt_tilde, m) (((-1i/sqrt(b_1))^m)*exp(dt_tilde*b_0)*...&#xA;    besselj(m, 2*dt_tilde*sqrt(-b_1)));&#xA;&#xA;    % establish polynomial truncation order so that M &amp;gt;e*sf*dt&#xA;    M = 1;&#xA;    while true&#xA;        orderinterator(M) = double(c_m(dt_tilde, M));&#xA;        if abs(orderinterator(M))&amp;lt;10e-15&#xA;            break&#xA;        else&#xA;        M = M+1;&#xA;        end&#xA;    end&#xA;&#xA;    % Compute time dependent Faber coefficients of order M for elliptic domain&#xA;    % initialize CM&#xA;    CM = zeros(M+1,1);&#xA;    for m = 0:M&#xA;        CM(m+1) = c_m(dt_tilde, m);&#xA;    end&#xA;&#xA;    % Compute matrix valued polynomials with initial value density matrix from&#xA;    % previous iteration&#xA;    I = ones(size(L_sc));&#xA;    P = zeros((M+1)*N, N);&#xA;&#xA;    % Compute matrix valued Faber polynomial recurrence relation&#xA;    % Compute initial value polynomials P_0, P_1, P_2:&#xA;    % F_0&#xA;    P(1:N, 1:N) = I;&#xA;&#xA;    % F_1&#xA;    P((N+1):(2*N), 1:N) = (L_sc-b_0*I);&#xA;&#xA;    % F_2&#xA;    P((2*N+1):(3*N), 1:N) = (L_sc-b_0*I)-2*b_1*I;&#xA;&#xA;    % c0*F_0+c1*F_1+c_2*F_2&#xA;    temp = CM(1)*P(1:N, 1:N)+CM(2)*P(N+1:2*N, 1:N)+CM(3)*P(2*N+1:3*N, 1:N); &#xA;&#xA;    % F_3 ... F_M = c2*F_2...cm*F_m&#xA;    for i = 3:M&#xA;        % F_{m+1} = L_sc(rho_fab)*F_m - b_0*F_m - b_1*F_{m-1}&#xA;        P((i*N)+1:(i+1)*N, 1:N) = L_sc*P((i-1)*N+1:(i)*N, 1:N)...&#xA;            -b_0*P((i-1)*N+1:(i)*N, 1:N)...&#xA;            -b_1*P((i-1-1)*N+1:(i-1)*N, 1:N);&#xA;&#xA;        temp = temp + CM(i+1)*P((i*N)+1:(i+1)*N, 1:N);&#xA;    end&#xA;    z = temp;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone perhaps elaborate on where it went wrong? Have I perhaps misunderstood the approach ?&lt;/p&gt;&#xA;" OwnerUserId="30177" LastEditorUserId="30177" LastEditDate="2019-03-16T13:22:37.217" LastActivityDate="2024-01-30T15:39:21.170" Title="MATLAB: Faber approximation of the matrix exponential to solve Liouville-von-Neumann equation" Tags="|matlab|quantum-mechanics|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="31221" PostTypeId="2" ParentId="31164" CreationDate="2019-03-12T13:49:16.363" Score="4" Body="&lt;p&gt;Another idea could be to use a generative approach (a program writing a program). Author a (meta)program that spits out the sequence of C/C++ instructions to perform unpivoted** LU on a 10x10 system.. basically taking the k/i/j loop nest and flattening it into O(1000) or so lines of scalar arithmetic. Then feed that generated program into whichever optimizing compiler. What I think is sort of interesting here, is removing the loops exposes every data dependency and redundant subexpression, and gives the compiler maximum opportunity to reorder instructions so that they map well to actual hardware (eg number of execution units, hazards/stalls, so on). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you happen to know all of the matrices (or even just a few of them), you can improve throughput by calling SIMD intrinsics/functions (SSE/AVX) instead of scalar code. Here you'd be exploiting the embarrassing parallelism across the instances, instead of chasing any parallelism within a single instance. For instance, you could perform 4 double precision LU's simultaneously using AVX256 intrinsics, by packing 4 matrices &quot;across&quot; the register and doing the same operations** on all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;** Hence the focus on unpivoted LU. Pivoting spoils this approach in two ways. First, it introduces branches due to pivot selection, meaning your data dependencies are not as perfectly known. Second, it means that different SIMD &quot;slots&quot; would have to do different things, because instance A might pivot differently than instance B. So if you pursue any of this, I'd suggest statically pivoting your matrices prior to computation (permute largest entry of each column to diagonal).&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2019-03-12T13:49:16.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31222" PostTypeId="2" ParentId="31196" CreationDate="2019-03-12T18:16:55.033" Score="3" Body="&lt;p&gt;The goal is to compute an &lt;span class=&quot;math-container&quot;&gt;$x\in\mathbb{R}^{n}$&lt;/span&gt; to satisfy the strict&#xA;inequalities&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(z_{i})^{T}x&amp;gt;0\qquad\forall z_{i}\in\{1,\ldots,m\}.\tag{1}&#xA;$$&lt;/span&gt;&#xA;If we write &lt;span class=&quot;math-container&quot;&gt;$\epsilon&amp;gt;0$&lt;/span&gt; as the smallest margin of feasibility, that&#xA;is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\epsilon=\min_{i}\{f(z_{i})^{T}x\},&#xA;$$&lt;/span&gt;&#xA;then (1) is equivalent to &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(z_{i})^{T}x\ge\epsilon&amp;gt;0\qquad\forall z_{i}\in\{1,\ldots,m\}.\tag{2}&#xA;$$&lt;/span&gt;&#xA;&lt;strong&gt;Proposition.&lt;/strong&gt; There exists a pair &lt;span class=&quot;math-container&quot;&gt;$(x,\epsilon)$&lt;/span&gt; satisfying&#xA;the strict inequality (2) if and only if there exists &lt;span class=&quot;math-container&quot;&gt;$y\in\mathbb{R}^{n}$&lt;/span&gt;&#xA;satisfying the nonstrict inequality (3) below&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(z_{i})^{T}y\ge1\qquad\forall z_{i}\in\{1,\ldots,m\}.\tag{3}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Proof.&lt;/em&gt; &lt;strong&gt;If.&lt;/strong&gt; Any &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; satisfying (3) also yields the&#xA;pair &lt;span class=&quot;math-container&quot;&gt;$(y,1)$&lt;/span&gt; satisfying (2). &lt;strong&gt;Only if.&lt;/strong&gt; By contradiction,&#xA;suppose that there exists no &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; satisfying (3) but there exists&#xA;a pair &lt;span class=&quot;math-container&quot;&gt;$(x,\epsilon)$&lt;/span&gt; satsifying (2). Then &lt;span class=&quot;math-container&quot;&gt;$y=x/\epsilon$&lt;/span&gt; satisfies&#xA;(3)--- a contradiction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, we can just go and find a choice of &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; satisfying (3). Any&#xA;standard off-the-shelf linear programming solver will solve&#xA;(3) in standard canonical form&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{gather*}&#xA;\min_{z}\quad c^{T}z\quad\text{subject to }\quad Az=b,\quad z\ge0,\\&#xA;\max_{y}\quad b^{T}y\quad\text{subject to }\quad A^{T}y\le c,&#xA;\end{gather*}&#xA;$$&lt;/span&gt;&#xA;with data&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;A=-\begin{bmatrix}f(z_{1}) &amp;amp; \cdots &amp;amp; f(z_{m})\end{bmatrix}^T,\qquad b=0,\qquad c=-\begin{bmatrix}1 &amp;amp; \cdots &amp;amp; 1\end{bmatrix}^T.&#xA;$$&lt;/span&gt;&#xA;A problem with &lt;span class=&quot;math-container&quot;&gt;$n\approx m\approx100$&lt;/span&gt; is extremely small.&#xA;In this case, you can also try 3 lines of MATLAB code using&#xA;ADMM. (See the &lt;a href=&quot;http://web.stanford.edu/~boyd/admm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boyd et al. survey&lt;/a&gt; for more details. At the risk of a little self-promotion, the specific derivations can be found in Section IV-C of our &lt;a href=&quot;https://arxiv.org/pdf/1709.08841.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial paper&lt;/a&gt;):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align*}&#xA;y^{k+1} &amp;amp; =(AA^{T})^{-1}A\left[c-s^{k}-z^{k}/t\right]\\&#xA;s^{k+1} &amp;amp; =\max\{0,c-A^{T}y^{k+1}-z^{k}/t\}\\&#xA;z^{k+1} &amp;amp; =z^{k}+t(A^{T}y^{k+1}+s^{k+1}-c)&#xA;\end{align*}&#xA;$$&lt;/span&gt;&#xA;Here, you can pick any step-size &lt;span class=&quot;math-container&quot;&gt;$t&amp;gt;0$&lt;/span&gt; and just set the initial points&#xA;to zero. &lt;a href=&quot;https://jemnz.com/lp75.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;See also Jacob Mattingley's page on ridiculously short LP solvers.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2019-03-12T18:25:11.837" LastActivityDate="2019-03-12T18:25:11.837" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31223" PostTypeId="2" ParentId="31219" CreationDate="2019-03-12T18:17:04.260" Score="5" Body="&lt;p&gt;Starting with &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$AM^{-1} \mathbf{y} = \mathbf{b}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y} \equiv M \mathbf{x}$&lt;/span&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we can manipulate to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{y} - (I-AM^{-1})\mathbf{y} = \mathbf{b}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replacing one instance of &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}^k$&lt;/span&gt; and the other by &lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}^{k+1}$&lt;/span&gt; yields the update equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}^{k+1} = (I-AM^{-1})\mathbf{y}^k + \mathbf{b}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can also be written as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathbf{y}^{k+1} = \mathbf{y}^k + (\mathbf{b}-AM^{-1}\mathbf{y}^k)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2019-03-12T18:17:04.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31224" PostTypeId="1" CreationDate="2019-03-12T19:09:29.547" Score="2" ViewCount="346" Body="&lt;p&gt;Richardson's iteration introduce a scalar &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; to the update formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \textbf{x}^{(k+1)} = \textbf{x}^{(k)} + \alpha \textbf{r}^{(k)} $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And compute &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; by minimizing the spectral radius:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \min_{\omega}\,\rho(B) = \min_{\omega}\, \rho(I-\omega A)\, . $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; do not change over the iterations, it might seem it exists only one (i.e. equal for all the iterations) &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; making the max eigenvalue minimal, hence the spectral radius minimal. Since I know gradient method and conjugate gradient perform better by setting &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; dynamically, I am wondering what am I missing here. Is there any way to see through the spectral radius expression that gradient and conjugate gradient iterations converge faster? &lt;/p&gt;&#xA;" OwnerUserId="30204" LastEditorUserId="9667" LastEditDate="2019-03-12T21:55:14.733" LastActivityDate="2019-03-12T21:55:14.733" Title="Richardson's Iteration, Gradient Method and Spectral Radius" Tags="|linear-solver|iterative-method|conjugate-gradient|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31225" PostTypeId="2" ParentId="31216" CreationDate="2019-03-13T00:34:53.543" Score="2" Body="&lt;p&gt;Definitely use C++ and &lt;a href=&quot;http://eigen.tuxfamily.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eigen&lt;/a&gt;. Reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sounds like it fits best for your case.&lt;/li&gt;&#xA;&lt;li&gt;Eigen beats LAPACK using optimization flags (&lt;code&gt;-O3&lt;/code&gt;) and a good compiler (GCC, Clang). At least for most tests I recently performed (dense linear algebra)&lt;/li&gt;&#xA;&lt;li&gt;Once you debug your application use the &lt;code&gt;-ffast-math&lt;/code&gt; compiler flag for a huge speedup, but test your output to see if it remains correct. I've noticed this is specially useful with Eigen.&lt;/li&gt;&#xA;&lt;li&gt;C++ is an actively evolving high-performance language. Eigen takes advantage of its advanced features to optimize your code and make most things compile-time and SIMD/vectorized.&lt;/li&gt;&#xA;&lt;li&gt;Virtually the only disadvantage of Eigen over LAPACK is that LAPACK has a standardized API. You can switch to another LAPACK library for higher performance in some cases.&lt;/li&gt;&#xA;&lt;li&gt;Still, keep an eye on LAPACK for any routines from Eigen that are not serving you well.&lt;/li&gt;&#xA;&lt;li&gt;Many production code in my field (computer vision, augmented reality, etc) that I've seen is using Eigen, and previously used LAPACK.&lt;/li&gt;&#xA;&lt;li&gt;You can always use LAPACK as well with your C++ code when you want to follow your professor more closely, and compare with Eigen to be a good student. But if you just wanted to follow the professor, without regard to the best alternative, you might as well just learn Fortran.&lt;/li&gt;&#xA;&lt;li&gt;See this: &lt;a href=&quot;http://eigen.tuxfamily.org/index.php?title=FAQ#How_does_Eigen_compare_to_BLAS.2FLAPACK.3F&quot; rel=&quot;nofollow noreferrer&quot;&gt;How does Eigen compare to BLAS/LAPACK&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That said, Fortran can be simpler and faster than C++ in common usage. See the beginning of &lt;a href=&quot;https://www.youtube.com/watch?v=mWK0ZO8xOoY&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/146159/is-fortran-easier-to-optimize-than-c-for-heavy-calculations&quot;&gt;the first answer to this stackoverflow post&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="30112" LastEditorUserId="30112" LastEditDate="2019-03-14T00:10:49.837" LastActivityDate="2019-03-14T00:10:49.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31226" PostTypeId="2" ParentId="31196" CreationDate="2019-03-13T04:25:05.350" Score="2" Body="&lt;p&gt;Here are two considerations that might help you figure out whether a solution exists:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;First, if there is a vector &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$F(z_i) \cdot X &amp;gt; 0$&lt;/span&gt;, then all vectors &lt;span class=&quot;math-container&quot;&gt;$Y=\alpha X$&lt;/span&gt; for any choice of &lt;span class=&quot;math-container&quot;&gt;$\alpha&amp;gt;0$&lt;/span&gt; are also solutions because &lt;span class=&quot;math-container&quot;&gt;$F(z_i) \cdot Y = \alpha \underbrace{(F(z_i)\cdot X)}_{\ge 0} &amp;gt; 0$&lt;/span&gt;. In other words, if a solution exists, then there are in fact infinitely many solutions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Second, let's forget that the &lt;span class=&quot;math-container&quot;&gt;$F(z_i)$&lt;/span&gt; vectors are really some kind of parameterized version. They are really just a bunch of vectors &lt;span class=&quot;math-container&quot;&gt;$F_i$&lt;/span&gt; that happen to be created in a special way. Then the inequality &lt;span class=&quot;math-container&quot;&gt;$F_i \cdot X&amp;gt;0$&lt;/span&gt; really just means that &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; must lie in the (open) half space cut out of all of &lt;span class=&quot;math-container&quot;&gt;${\mathbb R}^n$&lt;/span&gt; by the plane that is perpendicular to the vector &lt;span class=&quot;math-container&quot;&gt;$F_i$&lt;/span&gt;. A vector &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; that satisfies &lt;em&gt;all&lt;/em&gt; of the inequalities &lt;span class=&quot;math-container&quot;&gt;$F_i \cdot X&amp;gt;0, i=1,\ldots,n$&lt;/span&gt; must therefore lie in the &lt;em&gt;intersection&lt;/em&gt; of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; half spaces. The question of existence of a solution then boils down to the question whether the intersection of the &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; half spaces is non-empty. This is a geometric question. For example, if you were in 2d, then if you had vectors &lt;span class=&quot;math-container&quot;&gt;$F_1=(1,0), F_2=(-1,0)$&lt;/span&gt; then the (open) half-spaces are mutually exclusive and so no solution exists. The same would be true if you had &lt;span class=&quot;math-container&quot;&gt;$F_1=(1,0), F_2=(-1,1), F_3=(-1,-1)$&lt;/span&gt;, even though the intersection of each pair of half spaces is non-empty. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Out of this second consideration, I suspect that you can build an algorithm that determines whether the intersection of the first &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; half spaces with the &lt;span class=&quot;math-container&quot;&gt;$(k+1)$&lt;/span&gt;st half space is non-empty. I'd have to think about this in more depth to come up with an algorithm, but maybe you can do a literature search about the intersection of half spaces to come up with something useful. The point worth mentioning is that the intersection of your half spaces is a cone rooted at the origin that I suspect can be described by the intersection of the dividing planes of each half space; these are low dimensional objects easily amenable to to linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, there is also the possibility to use the special structure of the &lt;span class=&quot;math-container&quot;&gt;$F_i$&lt;/span&gt; to prove that a solution does or does not exist. For example, if you can show that there is a direction &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; so that all of the &lt;span class=&quot;math-container&quot;&gt;$F_i=F(z_i)$&lt;/span&gt; have an angle less than 90 degree from &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; (i.e., &lt;span class=&quot;math-container&quot;&gt;$Y\cdot F_i&amp;gt;0$&lt;/span&gt;), then it is geometrically clear that &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is a vector that satisfies the desired inequalities. Of course, you may not know &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, but maybe the statement is true for &lt;span class=&quot;math-container&quot;&gt;$Y=F(z_1)$&lt;/span&gt;, for example.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-03-13T04:25:05.350" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31229" PostTypeId="1" CreationDate="2019-03-13T15:59:57.877" Score="2" ViewCount="78" Body="&lt;p&gt;Modern augmented reality platforms such as &lt;a href=&quot;https://developers.google.com/ar/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google's ARCore&lt;/a&gt; and &lt;a href=&quot;https://developer.apple.com/arkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apple's ARKit&lt;/a&gt; seem to only operate on mobile devices, I'm guessing, because their underlying algorithms require specialized hardware that is typically available on these devices (accelerometers, gyroscopes, etc.). Since this hardware isn't available on laptops/desktops, I'm &lt;em&gt;guessing&lt;/em&gt; these libraries would never be able to work off of mobile platforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that I'm wondering if the following AR/image processing &quot;capabilities&quot; can be achieved via algorithms that &lt;em&gt;do not&lt;/em&gt; require such specialized hardware:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Flat surface detection inside an image or video; and&lt;/li&gt;&#xA;&lt;li&gt;Orientation/angle of the camera view inside an image or video&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Meaning, if I am given an image or video (set of sequential images), can I detect flat surfaces in that media without specialized hardware (as in, on a Linux box running on server or PC)? Same goes for orientation/angle determination (meaning, determining the &lt;code&gt;(x,y,z)&lt;/code&gt; coordinates of the camera within the &quot;domain&quot; of the image/video and determining which angles its pointed towards)?&lt;/p&gt;&#xA;" OwnerUserId="30214" LastActivityDate="2019-06-06T22:55:30.777" Title="Algorithm to determine flat surfaces and camera orientation without specialized hardware" Tags="|image-processing|computer-vision|3d|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31231" PostTypeId="1" CreationDate="2019-03-13T23:23:52.807" Score="0" ViewCount="120" Body="&lt;p&gt;Is it possible to view the data stored in the various PETSc data types from within totalview?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ordinarily, PETSc types are integers which act as pointers to the actual data (obviously my understanding of this is quite weak).&#xA;So when debugging and troubleshooting, my workflow looks like&#xA;1) Edit the source to save the PETSc data to a file&#xA;2) recompile&#xA;3) run and inspect output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is quite slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be quite nice to view this data directly from a GUI parallel debugger like totalview.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using fortran 90, by the way. &lt;/p&gt;&#xA;" OwnerUserId="25776" LastActivityDate="2020-10-03T12:03:20.380" Title="Access PETSc data in totalview?" Tags="|parallel-computing|fortran|petsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31232" PostTypeId="1" AcceptedAnswerId="31240" CreationDate="2019-03-14T01:20:50.970" Score="17" ViewCount="1032" Body="&lt;p&gt;I am reading Professor Saad's &lt;a href=&quot;https://www-users.cs.umn.edu/~saad/IterMethBook_2ndEd.pdf&quot; rel=&quot;noreferrer&quot;&gt;&quot;Iterative Methods for Sparse Linear Systems&quot; (2nd edition)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic algorithm for FOM is given on page 166 and the basic algorithm for GMRES is given on page 172. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both FOM and GMRES appear to build the same Krylov subspace and upper Hessenberg matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, at the very end of the algorithms, FOM solves a linear system to obtain &lt;span class=&quot;math-container&quot;&gt;$x_m$&lt;/span&gt; (seemingly discarding the last row of the Hessenberg matrix) while GMRES solves a least squares problem with the whole Hessenberg matrix to obtain &lt;span class=&quot;math-container&quot;&gt;$x_m$&lt;/span&gt;. Then, the solution to both problems is &lt;span class=&quot;math-container&quot;&gt;$y_m = V_m x_m + x_0$&lt;/span&gt; and I think the &lt;span class=&quot;math-container&quot;&gt;$V_m$&lt;/span&gt; are the same in both algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why does this (in my opinion, seemingly small) difference create two separate algorithms?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is GMRES used widely over FOM?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Clearly I seem to be missing something, but I don't know what. &lt;/p&gt;&#xA;" OwnerUserId="30219" LastEditorUserId="30219" LastEditDate="2019-03-14T21:01:17.190" LastActivityDate="2022-01-12T11:29:25.360" Title="What are the major differences between GMRES and FOM?" Tags="|krylov-method|gmres|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31234" PostTypeId="2" ParentId="27560" CreationDate="2019-03-14T15:14:07.533" Score="0" Body="&lt;p&gt;That Roe-averaged density guarantees the exact projection of the solution jump onto the right eigenvectors. See &lt;a href=&quot;http://ossanworld.com/cfdnotes/cfdnotes_roe_averaged_density.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ossanworld.com/cfdnotes/cfdnotes_roe_averaged_density.html&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="30222" LastActivityDate="2019-03-14T15:14:07.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31235" PostTypeId="1" CreationDate="2019-03-14T15:48:23.107" Score="2" ViewCount="62" Body="&lt;p&gt;I'm curious to know how QUADPACK's QAG routine works.  My understanding is that it begins by calculating on each subinterval the numerical quadrature with a Gaussian-Legendre rule and a nested Kronrod rule to provide estimates of the integral and the local error.  Then, globally the interval with the highest error is bisected and the process is repeated. However, since Gauss-Legendre abscissae do not nest, can any of the previous function evaluations be reused at the next iteration? Or are all previous function evaluations discarded when an interval is bisected and reevaluated? If the latter is true this seems incredibly inefficient, so why would anyone choose such an adaptive quadrature scheme like that in QUADPACK over one where the function evaluations may be reused, such as Newton-Cotes or Clenshaw-Curtis?&lt;/p&gt;&#xA;" OwnerUserId="20722" LastActivityDate="2019-03-14T15:48:23.107" Title="Does adaptive Gauss-Kronrod reuse function evaluations?" Tags="|quadrature|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31238" PostTypeId="5" CreationDate="2019-03-14T20:58:52.560" Score="0" Body="&lt;p&gt;Assume a linear system of the form:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Ax=b$$&lt;/span&gt;&#xA;where A is square and nonsingular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GMRES is an iterative Krylov subspace method that solves linear systems. GMRES does not require the system matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to be symmetric positive definite (as Conjugate Gradient does) or even symmetric (as MINRES does). GMRES is guaranteed not to break down as long as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is invertible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a given &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, define the residual vector &lt;span class=&quot;math-container&quot;&gt;$r(x)$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$ r(x) = b - Ax$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; be the initial solution (typically is the zero vector if no information is known about the solution).  Define &lt;span class=&quot;math-container&quot;&gt;$r_0 = r(x_0)$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;GMRES selects iterates by choosing &lt;span class=&quot;math-container&quot;&gt;$x_k$&lt;/span&gt; that solve the least squares problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align*} &#xA;    \min &amp;amp; \| r(x) \|_2 \\&#xA;         &amp;amp; \text{s.t. } x = x_0 + v, v \in \text{span} \lbrace r_0, Ar_0, \ldots A^{k-1} r_0 \rbrace&#xA;\end{align*}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that is, by minimizing &lt;span class=&quot;math-container&quot;&gt;$\ell^2$&lt;/span&gt; norm of the residual over a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-dimensional Krylov subspace. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seminal paper: &lt;a href=&quot;https://web.stanford.edu/class/cme324/saad-schultz.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.stanford.edu/class/cme324/saad-schultz.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other resource: &lt;a href=&quot;https://en.wikipedia.org/wiki/Generalized_minimal_residual_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Generalized_minimal_residual_method&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30219" LastEditorUserId="30219" LastEditDate="2019-03-15T21:38:43.803" LastActivityDate="2019-03-15T21:38:43.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31239" PostTypeId="4" CreationDate="2019-03-14T20:58:52.560" Score="0" Body="Refers to the (G)eneral (M)inimal (RES)idual algorithm, which is a popular Krylov subspace method for solving linear systems. " OwnerUserId="30219" LastEditorUserId="30219" LastEditDate="2019-03-15T21:38:32.790" LastActivityDate="2019-03-15T21:38:32.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31240" PostTypeId="2" ParentId="31232" CreationDate="2019-03-14T21:20:09.600" Score="16" Body="&lt;p&gt;There is one major difference between GMRES over FOM. It is also the reason why I would recommend GMRES over FOM.&lt;/p&gt;&#xA;&lt;p&gt;In exact arithmetic, the residuals obtained by GMRES form a decreasing sequence. You are certain that the GMRES residuals will not increase in the absence of rounding errors. Once the computed residual deviates from this simple pattern there is nothing to be gained from further iterations.&lt;/p&gt;&#xA;&lt;p&gt;In the case of FOM an arbitrary positive residual curve is possible. You can still detect when further iterations are pointless, but you must measure the size of the subdiagonal elements &lt;span class=&quot;math-container&quot;&gt;$h_{j+1,j}$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$V_j$&lt;/span&gt; is an invariant subspace for a matrix &lt;span class=&quot;math-container&quot;&gt;$A + \Delta A$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\|\Delta A\|_2 = h_{j+1,j}$&lt;/span&gt;. You will find an explicit formula for &lt;span class=&quot;math-container&quot;&gt;$\Delta A$&lt;/span&gt; in Saad's book. In other words, once &lt;span class=&quot;math-container&quot;&gt;$h_{j+1,j}$&lt;/span&gt; drops below &lt;span class=&quot;math-container&quot;&gt;$\tau \|A\|_2$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\tau$&lt;/span&gt; is the normwise relative error associated with computing &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, there is no point in doing further Arnoldi steps. At this stage it is entirely possible that we have solved the true problem exactly.&lt;/p&gt;&#xA;&lt;p&gt;Deciding when further FOM iterations are pointless requires more knowledge and insight than deciding when further GMRES iterations are pointless.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastEditorUserId="19136" LastEditDate="2022-01-12T11:29:25.360" LastActivityDate="2022-01-12T11:29:25.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31242" PostTypeId="1" CreationDate="2019-03-15T14:10:20.557" Score="2" ViewCount="169" Body="&lt;p&gt;In classical linear elasticity, when modeling a force/load boundary condition, it appears that we could either:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a pure Neumann boundary condition, where the 3 traction components are specified. In 3-D the 2 tangential traction components would be zero, with the non-zero component being the normal component.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a mixed boundary condition. Here, the 2 tangential traction components would be zero. The normal traction component is unknown. However, the normal displacement is now known, and the 2 tangential displacements are unknown.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it correct to say that 1 and 2 are both accurate methods of modeling this force boundary condition?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there an actual analytical expression for converting the 1 to 2?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If we know the normal traction component, could we actually uniquely convert this to a normal displacement value? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="24311" LastEditorUserId="24311" LastEditDate="2019-03-16T02:17:23.017" LastActivityDate="2019-03-16T15:30:06.090" Title="Linear elasticity modeling load using traction vs. mixed BC" Tags="|boundary-conditions|solid-mechanics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31243" PostTypeId="2" ParentId="24002" CreationDate="2019-03-15T15:06:40.730" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://wias-berlin.de/software/tetgen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TetGen&lt;/a&gt; is specifically designed to perform 3-D meshes -&gt; tetrahedralization. I do not see a reliable and direct way to use it explicitly as a 2-D mesher. It is well pointed out that &lt;a href=&quot;http://www.cs.cmu.edu/~quake/triangle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Triangle&lt;/a&gt; has a &quot;surprisingly&quot; similar API and specifically targets 2-D mesh generation. No surprise is there: both TetGen and Triangle are a part of &lt;a href=&quot;http://www.wias-berlin.de/software/pdelib/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdelib2&lt;/a&gt; software collection that contains necessary building blocks for solving PDEs. So, if you are already using TetGen, I would advise going to add Triangle to your library dependencies (maybe in addition to a couple of other &quot;helpers&quot; from pdelib2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding GMSH. Recently, &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#Gmsh-API&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMSH started to offer API&lt;/a&gt; to its functionality for C, C++, Python, and Julia. That would eliminate the need to use GMSH GUI or even command line interfaces. And certainly, &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#Mesh-options-list&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMSH offers Delaunay triangulation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-03-15T15:06:40.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31244" PostTypeId="1" AcceptedAnswerId="31249" CreationDate="2019-03-16T02:51:36.783" Score="0" ViewCount="132" Body="&lt;p&gt;I am working through an exercise in my textbook and implementing the code in Python to practice dynamic programming. I feel like I am right on the edge of figuring it out, but after many hours, I come here for help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, my code is going through a list of values &lt;code&gt;x&lt;/code&gt;, and given a &lt;code&gt;k&lt;/code&gt;, breaks that list into &lt;code&gt;k&lt;/code&gt; clusters based on calculating the minimum sum of squared errors (SSE) for a particular cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code creates a table, calculating the SSE for 1 cluster, 2 clusters, ..., k clusters, if we were to put the cluster parentheses around all variations of values within &lt;code&gt;list[0:1]&lt;/code&gt;, &lt;code&gt;list[0:2]&lt;/code&gt;, &lt;code&gt;list[0:3]&lt;/code&gt;, ..., &lt;code&gt;list[0:n]&lt;/code&gt;, and choosing the minimum SSE for that particular step in the table.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given &lt;code&gt;x= [7,6,9,15,18,17,30,28,29]&lt;/code&gt; and &lt;code&gt;k=3&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;we would return clusters &lt;code&gt;(7,6,9)(15,18,17)(30,28,29)&lt;/code&gt;, which would translate to sum of squared error equal to &lt;code&gt;(4.666)(4.666)(2)&lt;/code&gt; for each cluster. So our max SSE would be &lt;code&gt;4.666&lt;/code&gt; for that clustering on that list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now when I try it on my second list &lt;code&gt;x = [52, 101, 103, 101, 6, 5, 7]&lt;/code&gt;, I should get clustering &lt;code&gt;(52)(101, 103, 101)(6, 5, 7)&lt;/code&gt;, which should give &lt;code&gt;(0)(2.666)(2)&lt;/code&gt; or a max of &lt;code&gt;2.666&lt;/code&gt;, but am not getting that. I believe the error lives in the  &lt;code&gt;def f(s, j_down, t)&lt;/code&gt; for the 2nd return statement, and how I increment &lt;code&gt;s&lt;/code&gt; and &lt;code&gt;t&lt;/code&gt;. Hopefully, I have not made a silly mistake!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def mean(numbers):&#xA;    return float(sum(numbers)) / max(len(numbers), 1)&#xA;&#xA;def sum_square(x):&#xA;    if isinstance(x, (int,)):&#xA;        return 0&#xA;    w = 0&#xA;    for i in x:&#xA;        w += (i - mean(x))**2&#xA;    return w&#xA;&#xA;def f(s, j_down, t):&#xA;    if not r[s][j_down] and r[s][j_down] != 0:&#xA;        return sum_square(x[:t - s])&#xA;&#xA;    return max(r[s][j_down], sum_square(x[:t-s]))&#xA;&#xA;def get_min_f_and_s(j_down, t):&#xA;    &quot;&quot;&quot; range s from 1 to t-1 and set s to minimize f(s)&#xA;    &quot;&quot;&quot;&#xA;    items = [(s, f(s, j_down, t)) for s in range(t)]&#xA;    s, min_f = min(items, key=lambda x:x[1])&#xA;    return s, min_f&#xA;&#xA;def seq_out(n,k):&#xA;    for j in range(k):&#xA;        if j == 0:&#xA;            for t in range(n):&#xA;                r[t][j] = sum_square(x[:t+1])&#xA;&#xA;                c[t][j] = x[:t+1]&#xA;        else:&#xA;            for t in range(1, n):&#xA;                s, min_f = get_min_f_and_s(j - 1, t)&#xA;                r[t][j] = min_f&#xA;                c[t][j] = [c[s][j - 1]] + x[s+1:t+1]&#xA;&#xA;    print('the max SSE is: {}'.format(r[-1][-1]))&#xA;    print('the cluster centers are: {}'.format(c[-1][-1]))&#xA;&#xA;#x = [7,6,9,15,18,17,30,28,29]    &#xA;x = [52, 101, 103, 101, 6, 5, 7]&#xA;k = 3&#xA;n = len(x)&#xA;&#xA;r = [[[] for _ in range(k)] for _ in range(n)]&#xA;c = [[[] for _ in range(k)] for _ in range(n)]&#xA;&#xA;print(seq_out(n,k))&#xA;print(r)&#xA;print(c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: Question layout&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a sequence &lt;code&gt;X = [x_1, x_2, ... x_n]&lt;/code&gt; and integer &lt;code&gt;k &amp;gt; 1&lt;/code&gt;, partition &lt;code&gt;X&lt;/code&gt; into clusters &lt;code&gt;C_1,..., C_k&lt;/code&gt; of sizes &lt;code&gt;n_1, ..., n_k&lt;/code&gt;, so that the sum of squared errors is minimized. &lt;/p&gt;&#xA;" OwnerUserId="30233" LastEditorUserId="20688" LastEditDate="2019-03-16T19:58:03.703" LastActivityDate="2019-03-16T20:53:46.550" Title="Python sequence cluster exercise" Tags="|python|algorithms|dynamic-programming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31245" PostTypeId="1" CreationDate="2019-03-16T10:02:36.977" Score="1" ViewCount="97" Body="&lt;p&gt;What are the most sophisticated methods for interpolating a scalar field say Electric or Magnetic Field on a 3-D grid? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have scalar data on a meshgrid with equal spacing. I would like to use an interpolating method to interpolate those values onto random points I select on the same grid. I think my best bet is using tricubic splines. Am I right? Some of the scalars might not be linear with others being linear though. Any guidance would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30234" LastEditorUserId="20688" LastEditDate="2019-03-17T18:08:09.527" LastActivityDate="2019-03-17T18:08:09.527" Title="Grid Data Interpolation" Tags="|computational-physics|interpolation|b-spline|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31246" PostTypeId="2" ParentId="31242" CreationDate="2019-03-16T15:30:06.090" Score="2" Body="&lt;p&gt;The two descriptions you use describe different physical situations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first, you apply a given force (which you have assumed is normal to the surface, though it could of course also have a tangential component in general) and the displacement is then simply going to be determined by how the object to which the force is applied reacts to the force. An example is if you place a weight of known mass on top of a deformable object -- say, you park a truck on a bridge: it has a fixed weight, and the bridge deforms according to its elastic properties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second situation you describe is where you apply a certain displacement, regardless of how much force that requires. An example is if you push a steel tool into a soft object: the steel is going to hard enough to not deform noticeably so that you can achieve whatever deformation you desire regardless of the force necessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since these are two entirely different physical situations, there is no way to convert one description into the other. In other words, there is no easy way (other than to solve the equations of elasticity) to compute the displacement you obtain for the given force in situation 1 above, or to compute the necessary force to obtain a given displacement in situation 2. In other words, the answers to your second and third question is &quot;no, this is not possible without actually solving the equations of elasticity&quot;.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-03-16T15:30:06.090" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31249" PostTypeId="2" ParentId="31244" CreationDate="2019-03-16T20:53:46.550" Score="0" Body="&lt;p&gt;Until you define what recursion you used, it's hard to really be sure what you're trying to implement. I took a look at the fundamental problem you are solving, though, and defined a recurrence that will work. I assume here we have some sequence &lt;span class=&quot;math-container&quot;&gt;$d(i)$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; components that starts with indexing at &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Running with this, the minimum cost clustering such that you get exactly &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; clusters can be expressed with the recurrence&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;f(i,j) &amp;amp;= \begin{cases} &#xA;\infty &amp;amp; (i=0 \land j \neq k+1) \lor (i \neq 0 \land j = k+1) \\&#xA;0 &amp;amp; (i = 0 \land j = k+1) \\&#xA;\text{cost}(1,i) &amp;amp; j = k\\&#xA;\min_{1 \leq l \leq i} f(l-1, j+1) + \text{cost}(l,i) &amp;amp; \text{otherwise}&#xA;\end{cases}&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the &lt;span class=&quot;math-container&quot;&gt;$\text{cost}(i,j)$&lt;/span&gt; cost function returns the cluster cost of viewing elements &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; in the sequence as a cluster. We can define the cluster cost as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\text{cost}(i,j) &amp;amp;= \sum_{s = i}^j \left(d(s) - \mu(i,j)\right)^2 \\&#xA;\text{where }\mu(i,j) &amp;amp;= \frac{1}{j - i + 1}\sum_{t = i}^j d(t)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using these definitions, one can implement the recurrence however they want and to find &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. The simplest is a recursion with memoization built in but I tend to look for where the know elements are in &lt;span class=&quot;math-container&quot;&gt;$f(i,j)$&lt;/span&gt; off the bat and using those to fill in the other quantities more efficiently. Regardless of how you do it, you can fill in &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; and then use the table to find the optimal &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; clusters by using &lt;span class=&quot;math-container&quot;&gt;$\arg \min_{1 \leq l \leq i} f(l-1, j+1) + \text{cost}(l,i)$&lt;/span&gt; with the starting indices &lt;span class=&quot;math-container&quot;&gt;$i = n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j = 1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a code in Python that implements this solution, which you can compare against if desired&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#import useful libs&#xA;import numpy as np&#xA;&#xA;# problem is found here: &#xA;# https://scicomp.stackexchange.com/questions/31244/python-sequence-cluster-exercise&#xA;&#xA;# define efficient representation of cluster cost&#xA;class cluster_cost:&#xA;    def __init__(self, data):&#xA;        self.n = len(data)&#xA;        self.ss_x = np.cumsum(data)    # subset sum of x&#xA;        self.ss_x2= np.cumsum(data**2) # subset sum of x^2&#xA;&#xA;    def cost(self, i, j):&#xA;        if i == 0:&#xA;            dx = self.ss_x[j]&#xA;            dx2= self.ss_x2[j]&#xA;        else:&#xA;            dx = self.ss_x[j] - self.ss_x[i-1]&#xA;            dx2= self.ss_x2[j] - self.ss_x2[i-1]&#xA;        mean = dx/(j-i+1)&#xA;        return dx2  + (j-i+1)*mean**2 - 2.0*mean*dx&#xA;&#xA;&#xA;&#xA;def dp_soln(data, k):&#xA;&#xA;    # define cluster cost function&#xA;    cc = cluster_cost(data)&#xA;&#xA;    # get number of elements in data&#xA;    n = cc.n&#xA;&#xA;    # init the DP cost table&#xA;    f = np.full((n+1,k+2), np.inf)&#xA;&#xA;    # construct f[i,j] for all i and j&#xA;    # init for j = k+1&#xA;    f[0,k+1] = 0&#xA;&#xA;    # init for j &amp;lt;= k&#xA;    for j in range(k, 0, -1):&#xA;        for i in range(1,n+1):&#xA;&#xA;            # get the min cost option&#xA;            best = np.inf&#xA;            for l in range(1, i+1):&#xA;                best = np.min([best, f[l-1,j+1] + cc.cost(l-1,i-1)])&#xA;&#xA;            # set best cost to DP table&#xA;            f[i,j] = best&#xA;&#xA;    # solve for the optimal clustering&#xA;    # using cost table from DP algo&#xA;    partition_indices = []&#xA;    partition = []&#xA;&#xA;    # set initial state indices to (n,1)&#xA;    i = n&#xA;    j = 1&#xA;&#xA;    # loop until k clusters found&#xA;    while j &amp;lt;= k:&#xA;&#xA;        # find argmin that matches DP value&#xA;        best = np.inf&#xA;        best_idx = 1&#xA;        for l in range(1, i+1):&#xA;            bestn = np.abs( f[i,j] - f[l-1,j+1] - cc.cost(l-1,i-1) )&#xA;            if( bestn &amp;lt; best ):&#xA;                best_idx = l&#xA;                best = bestn&#xA;&#xA;        # get the next partition based on argmin&#xA;        partition_indices.append((i-1, best_idx-1))&#xA;        partition.append(data[(best_idx-1):i])&#xA;&#xA;        # update indices&#xA;        i = best_idx - 1&#xA;        j = j + 1&#xA;&#xA;    # return the partition indices and partition&#xA;    return (partition_indices, partition)&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    k = 3&#xA;    data = np.array([7, 6, 9, 15, 18, 17, 30, 28, 29])&#xA;    (indices, part) = dp_soln(data, k)&#xA;    print(indices)&#xA;    print(part)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above sample &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\text{data}$&lt;/span&gt; return the expected output when run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[(8, 6), (5, 3), (2, 0)]&#xA;[array([30, 28, 29]), array([15, 18, 17]), array([7, 6, 9])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9804" LastActivityDate="2019-03-16T20:53:46.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31250" PostTypeId="1" CreationDate="2019-03-16T23:43:12.827" Score="4" ViewCount="259" Body="&lt;p&gt;Suppose we have a conformal mapping from the unit disk in the &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; plane onto the exterior of a polygon in the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; plane.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The  &lt;strong&gt;Schwarz-Christoffel mapping&lt;/strong&gt; in this case is defined as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(u) = A - C \int^u w^{-2}\prod_{k=1}^{n} (1-\frac{\omega}{u_k})^{1-\alpha_k} d\omega.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meaning that we determine the vertex &lt;span class=&quot;math-container&quot;&gt;$z_k$&lt;/span&gt; with the mapping &lt;span class=&quot;math-container&quot;&gt;$f(u_k)$&lt;/span&gt; from the respective prevertex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have provided an image from Discroll and Trefethens' book on the subject (PDF sample with the image is freely available).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HmscL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HmscL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I would like to compute a factor from an integral over function analytic inside a polygon in the complex plane.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this I will use &lt;strong&gt;Gauss-Kronrod quadrature&lt;/strong&gt;, specifically MATLAB's &lt;strong&gt;quadgk&lt;/strong&gt; command with 'waypoints' &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ll just pick my function arbitrarily as &lt;span class=&quot;math-container&quot;&gt;$g = \exp(z)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to compute is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int{\frac{\exp(f(u))}{u}}du$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use &lt;strong&gt;quadgk&lt;/strong&gt; I have to define a function handle for &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;.&#xA;But I am confused on how to do that with the SC-Mapping f(u).&#xA;I know I can extract the prevertices and interior angles &lt;span class=&quot;math-container&quot;&gt;$\alpha_k$&lt;/span&gt; with the SC-toolbox for any polygon like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PSI = extermap(drawpoly)&#xA;p = polygon(PSI);&#xA;a = parameters(PSI);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I compute the integral &lt;span class=&quot;math-container&quot;&gt;$\int^u w^{-2}\prod_{k=1}^{n} (1-\frac{\omega}{u_k})^{1-\alpha_k} d\omega$&lt;/span&gt; ? It's a polygonal integral in the complex plane so should I use quadgk as well ?&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I set the complex constants &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;b = a.constant&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;seems to refer to the logarithmic capacity of the polygonal domain.&lt;/p&gt;&#xA;" OwnerUserId="30177" LastEditorUserId="22284" LastEditDate="2019-03-17T22:36:29.633" LastActivityDate="2019-03-17T22:36:29.633" Title="MATLAB: Compute the Schwarz-Christoffel transformation symbolically" Tags="|matlab|integration|approximation|integral-equations|complex-analysis|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31252" PostTypeId="1" CreationDate="2019-03-18T07:17:23.287" Score="0" ViewCount="339" Body="&lt;p&gt;I want to discretize the following equation using a Finite Volume Method&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\nabla \cdot (a(x)\nabla u)=f(x)\\x\in \Omega \subset \mathbb{R}^2&#xA;\\u_{|\partial\Omega}=g$$&lt;/span&gt;&#xA;I'm using &lt;a href=&quot;https://en.wikipedia.org/wiki/Voronoi_diagram&quot; rel=&quot;nofollow noreferrer&quot;&gt;Voronoi cells&lt;/a&gt; here: let &lt;span class=&quot;math-container&quot;&gt;$V_i \subset\Omega$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i=1,\dots,N$&lt;/span&gt;, such that &lt;span class=&quot;math-container&quot;&gt;$\bigcup\limits_{i=1}^N V_i=\Omega$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$V_i\cap V_j=\partial V_i \cap \partial V_j$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$i\ne j$&lt;/span&gt; be the mesh cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After applying the Divergence theorem and integrating the equation over each &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt;, obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_{V_i} \nabla \cdot (a(x)\nabla u) dx = \int_{\partial V_i} a(x)\nabla u \cdot n ds=\int_{V_i} fdx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; be the &quot;center&quot; of &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt; (this is a cell-cenetered method). Define &lt;span class=&quot;math-container&quot;&gt;$n_i\subset \{1,\dots,N\}$&lt;/span&gt; as the set of indices of the cells immediately neighbouring &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt;. Let the boundary between any two mesh cells be defined as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Gamma_{i,j}:=\partial V_i \cap \partial V_j$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define the length of the boundary segment between neighbouring cells &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V_j$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$$l_{i,j}:=\left|\Gamma_{i,j}\right|, j\in n_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the distance between two cell &quot;centers&quot; as &lt;span class=&quot;math-container&quot;&gt;$$h_{i,j}:=\|x_i-x_j\|_2, j\in n_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Approximate &lt;span class=&quot;math-container&quot;&gt;$a(x)$&lt;/span&gt; for each two neighbouring cells &lt;span class=&quot;math-container&quot;&gt;$V_i,V_j$&lt;/span&gt; as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$a_{i,j}:=a\left(\frac{x_i+x_j}{2}\right), j\in n_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we obtain the following discretization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\int_{\partial V_i} a(x)\nabla u \cdot n ds=\sum_{j\in n_i}\int_{\Gamma_{i,j}}a(x)\nabla u(x)\cdot n ds&#xA;\approx \sum_{j\in n_i} a_{i,j} \frac{u_j-u_i}{h_{i,j}}l_{i,j}=|V_i|f(x_i)\approx \int_{V_i} f(x)dx$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;To discretize the Dirichlet BC&lt;/strong&gt; using centered cells, as far as I understand, one needs to use outer ghost cells. So let &lt;span class=&quot;math-container&quot;&gt;$V_{G_j}$&lt;/span&gt; be such a cell, adjacent to &lt;span class=&quot;math-container&quot;&gt;$V_j$&lt;/span&gt; for some &lt;span class=&quot;math-container&quot;&gt;$j\in\{1,\dots, N\}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$x_{G_{j}}\in V_{G_{j}}$&lt;/span&gt; be its &quot;center&quot;. But this is where I'm stuck. Where do I go from here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I know there is a similar question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/8538/applying-dirichlet-boundary-conditions-to-the-poisson-equation-with-finite-volum&quot;&gt;here&lt;/a&gt;, but I couldn't quite follow the notation there.&lt;/p&gt;&#xA;" OwnerUserId="29427" LastActivityDate="2021-04-09T23:03:10.340" Title="Elliptic PDE finite volume method with Dirichlet boundary condition" Tags="|pde|finite-volume|discretization|elliptic-pde|voronoi-diagrams|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31253" PostTypeId="2" ParentId="31220" CreationDate="2019-03-18T11:01:13.307" Score="3" Body="&lt;p&gt;&lt;em&gt;Not sure whether this provides a full answer, but at least it provides some thoughts and hints.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Concept&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I think the expression &lt;span class=&quot;math-container&quot;&gt;$\exp(\mathcal{Lt})\rho$&lt;/span&gt; needs clarification. This is not necessarily a matrix exponential, but rather an abstract notation for the solution operator. For ODEs &lt;span class=&quot;math-container&quot;&gt;$\partial_t x = A x$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is a vector and &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a matrix (such as in the Fahs reference), the solution operator is indeed a matrix exponential. On the other hand, for the Liouville-von Neumann equation the solution operator is (roughly and without any relaxation terms) &lt;span class=&quot;math-container&quot;&gt;$\exp(\mathcal{L}t)\rho = U \rho U^\dagger$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$U = \exp(-\mathrm{i}\hbar^{-1} H t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is the Hamiltonian. In your code, &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; is calculated using MATLABs built-in expm function and may serve as reference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Now I am not sure what your plan is, but you could try to calculate &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; using your Faber approximation code and check whether it gives the same result as the expm method.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Implementation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I tried to do this using your code, but the code returned errors. If you write a function that performs a certain calculation, immediately think about ways to test it. Even during early stages of development this will be helpful to detect mistakes.&lt;/p&gt;&#xA;" OwnerUserId="30021" LastEditorUserId="30021" LastEditDate="2019-03-18T14:43:33.227" LastActivityDate="2019-03-18T14:43:33.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31254" PostTypeId="1" CreationDate="2019-03-18T14:47:33.010" Score="0" ViewCount="126" Body="&lt;p&gt;I have a diffusion problem which can be broken down to be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$-\Delta u = f(u) $&lt;/span&gt;    on &lt;span class=&quot;math-container&quot;&gt;$\Omega ~/~ \Omega_{int}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u = 1$&lt;/span&gt;  on &lt;span class=&quot;math-container&quot;&gt;$\Omega_{int}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is an &lt;strong&gt;internal&lt;/strong&gt; Dirichlet constraint to the value 1.0 in the center circle, unlike the standard textbook examples with homogeneous boundary conditions. (see picture) Think of it like a source in the middle, which diffuses into the domain with f(u) = u(1-u) being a logistic growthterm. I want that reaction to stop at a certain maximum total mass.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I want to impose an &lt;strong&gt;integral condition&lt;/strong&gt; which will stabilize the solution:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$P(u) = \int_{\Omega}  u  ~ dx =V_{tot}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I tried to read into the problem, and I find that it is possible to add penalty terms to the bilinear form of the finite element discretization, but I am deeply puzzled as to how to properly define the penalty term so that my Newton solver will behave correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multiplying with a test function v, integrating over the domain we may formulate the problem without the volume constraint like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;find u, v so that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ r(u,v) = a(u,v)+ l(v) = 0 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$r(u,v)$&lt;/span&gt; is a global residuum (To be minimized by the Newton solver), &lt;span class=&quot;math-container&quot;&gt;$a(u,v)$&lt;/span&gt; is the canonical discretisation of the diffusive term and &lt;span class=&quot;math-container&quot;&gt;$l(v)$&lt;/span&gt; the disc. of the reaction term.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Now in the language of Lagrange multipliers, we have a functional &lt;span class=&quot;math-container&quot;&gt;$r(u,v)$&lt;/span&gt; and a constraint &lt;span class=&quot;math-container&quot;&gt;$P(u)$&lt;/span&gt; on the volume so that the new Functional would look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$L(u,v,\lambda)= a(u,v)+ l(v) -\lambda(P(u)-V_{tot}) = 0 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do I apply this to my Finite Element Method in a clean way? &lt;/li&gt;&#xA;&lt;li&gt;How would I determine &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;li&gt;Is there any readable literature on how to construct volume-constraints via penalty terms?&lt;/li&gt;&#xA;&lt;li&gt;Is there a more readable/hands-on literature/Paper on Lagrange multipliers in the context of FE-Methods than the work of &lt;a href=&quot;https://gdz.sub.uni-goettingen.de/id/PPN362160546_0020?tify=%7B%22view%22%3A%22info%22%2C%22pages%22%3A%5B183%5D%7D&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ivo Babuska&lt;/a&gt;?&lt;/li&gt;&#xA;&lt;li&gt;How do I make sure that the penalty term, which will probably contain a domain-wide integral will not mess up my Newtonian (since the necessary calculations for the Jacobian are no longer local)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n8N4s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n8N4s.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="28636" LastEditorUserId="28636" LastEditDate="2019-03-19T17:16:44.657" LastActivityDate="2019-03-19T17:16:44.657" Title="Constraining the total volume in Finite Element Methods" Tags="|finite-element|constraints|weak-solution|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="31255" PostTypeId="1" CreationDate="2019-03-18T21:22:54.277" Score="3" ViewCount="125" Body="&lt;p&gt;I'm working with a finite element model in which I'm interested to &lt;strong&gt;minimize&lt;/strong&gt; the average temperature at a surface. I have &lt;strong&gt;15 independent variables&lt;/strong&gt; in my model, including geometry, materials, flows, process times, etc. Most of my variables are continuous but some are discrete; and it takes me &lt;strong&gt;~10 mins to run one simulation&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not well educated on what optimization algorithms are available and how they work, so I was wondering if you can help me out to find which optimization techniques are suitable for this kind of situation: in which I have a (relatively) expensive function to evaluate as well as (relatively) many variables.&lt;/p&gt;&#xA;" OwnerUserId="25649" LastEditorUserId="25649" LastEditDate="2019-03-19T00:13:09.143" LastActivityDate="2019-03-21T21:09:12.433" Title="Optimization techniques for expensive multi-variable functions" Tags="|optimization|simulation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31256" PostTypeId="2" ParentId="31255" CreationDate="2019-03-18T22:30:18.673" Score="8" Body="&lt;p&gt;One commonly used approach is the &quot;Response Surface Method&quot; in which you sample the feasible region, running the full simulation at the sample points, then use regression techniques to fit a surrogate model to these points.  You'll be assuming that the response in between your sample points is relatively smooth.  Once you've fit that surrogate model, you optimize over the surrogate model to find its minimum.  Then go back to the full simulation and verify that the surrogate model made a good prediction of the value at that point.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One advantage of this approach is that the simulations can be run at the various sample points in parallel, completely independent of each other.  &lt;/p&gt;&#xA;" OwnerUserId="2150" LastActivityDate="2019-03-18T22:30:18.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31257" PostTypeId="1" CreationDate="2019-03-19T03:51:13.913" Score="0" ViewCount="65" Body="&lt;p&gt;I'm discretizing the following Poisson equation using FVM where the domain &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; of the solution is a regular hexagon of side &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; centered about the origin.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Delta u =k,\text{ $k$ constant}\\&#xA;\partial_n u = 0\\&#xA;\Omega \subset \mathbb{R^2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not really quite sure if my discretization and especially the argument for existence and uniqueness are correct. So I would appreciate some feedback.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$V_i\subset \Omega$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i=1,\dots,6$&lt;/span&gt;, be one of the six equilateral triangles comprising the hexagon; so &lt;span class=&quot;math-container&quot;&gt;$\bigcup\limits_{i=1}^6 V_i=\Omega$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we obtain a system of six equations, that is a linear system after the following steps:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\int_{V_i} \Delta udx = \int_{\partial V_i} \nabla u(x)\cdot n ds =\int_{V_i}kdx$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\int_{partial V_i} \nabla u(x)\cdot nds=\int_{\partial V_i}\partial_n u ds$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\approx \sum\limits_{j\in n_i}\frac{u_j-u_i}{h_{i,j}}l_{i,j}+\int_{\Gamma_i}0ds=\sum\limits_{j\in n_i} \frac{u_j-u_i}{h_{i,j}}l_{i,j}+\text{constant $C$}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$=|V_i|k$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$|V_i|$&lt;/span&gt; means the area of &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; is the set of cells immediately neighboring &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt; (two such cells for a hexagon).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can use some geometry to deduce that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$|V_i| = \frac{\sqrt{3}}{4}; h_{i,j} = \frac{\sqrt{3}}{2}; l_{i,j}=1$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the system becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{2}{\sqrt3}\sum_{j\in n_i} (u_j-u_i) + C = \frac{\sqrt3}{4}k$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which can be represented in matrix-vector form as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$B\vec{u}=\vec{c}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is a square &lt;span class=&quot;math-container&quot;&gt;$6\times 6$&lt;/span&gt; matrix with the following components&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$b_{ij}=\frac{2}{\sqrt3}, i\ne j; b_{ii}=-\frac{2\sqrt2}{3}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$c_i = \frac{\sqrt3}{4}k-C$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the &quot;most interesting&quot; part: is it enough to argue that there is a unique solution and it exists because &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is non-singular?&lt;/p&gt;&#xA;" OwnerUserId="29427" LastEditorUserId="20688" LastEditDate="2019-03-19T15:51:56.973" LastActivityDate="2019-03-19T15:51:56.973" Title="Existence and uniquness of solution of FVM for Poisson equation" Tags="|boundary-conditions|discretization|poisson|elliptic-pde|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31258" PostTypeId="1" CreationDate="2019-03-19T15:56:02.603" Score="0" ViewCount="542" Body="&lt;p&gt;I am searching for an optimization solution, which is a 8d vector representing 4 complex elements, where each element is within the complex circle with maximal radius 1.2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objective function is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f: \left\|\mathbf{c}_{ref}-\mathbf{c}\right\|_{2}+\left|2-\|\mathbf{c}\|_{2}\right|$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the &lt;code&gt;scipy.optimize.basinhopping&lt;/code&gt; module.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy import optimize&#xA;&#xA;class MyBounds(object):&#xA;    def __init__(self, xmax=[360, 360, 360, 360, 1.2, 1.2, 1.2, 1.2], xmin=[0, 0, 0, 0, 0, 0, 0, 0] ):&#xA;        self.xmax = np.array(xmax)&#xA;        self.xmin = np.array(xmin)&#xA;&#xA;    def __call__(self, **kwargs):&#xA;        x    = kwargs[&quot;x_new&quot;]&#xA;        tmax = bool(np.all(x &amp;lt;= self.xmax))&#xA;        tmin = bool(np.all(x &amp;gt;= self.xmin))&#xA;        return tmax and tmin&#xA;&#xA;class MyTakeStep(object):&#xA;    def __init__(self, stepsize=45):&#xA;        self.stepsize = stepsize&#xA;    def __call__(self, x):&#xA;        s = self.stepsize&#xA;        x[0] += np.random.uniform(0, 1*s)&#xA;        x[1] += np.random.uniform(0, 2*s)&#xA;        x[2] += np.random.uniform(0, 3*s)&#xA;        x[3] += np.random.uniform(0, 4*s)&#xA;        s = 1&#xA;        x[4] += np.random.uniform(0, 1*s)&#xA;        x[5] += np.random.uniform(0, 1*s)&#xA;        x[6] += np.random.uniform(0, 1*s)&#xA;        x[7] += np.random.uniform(0, 1*s)&#xA;        return x&#xA;&#xA;def f(input_list):&#xA;    polar_form = [rphi for rphi in zip(input_list[4:], input_list[:4])]&#xA;    x          = np.array([z(e[0], e[1]) for e in polar_form])&#xA;    # objective function componenets&#xA;    result  = abs(2 - np.linalg.norm(x, 2)) + np.linalg.norm(cref-x, 2)&#xA;    return result&#xA;&#xA;def hopping_solver(min_f, min_x):&#xA;    minimizer_kwargs = {&quot;method&quot;:'Nelder-Mead'}&#xA;    comb             = np.array([0, 0, 0, 0, 0, 0, 0, 0])&#xA;    # define bounds&#xA;    mybounds   = MyBounds()&#xA;    mytakestep = MyTakeStep()&#xA;    optimal_c  = optimize.basinhopping(f, x0 = comb, niter=50000, T=45, stepsize=1, minimizer_kwargs=minimizer_kwargs, take_step=None, accept_test=mybounds, callback=None, interval=5000, disp=False, niter_success=None)&#xA;&#xA;    min_x, min_f = optimal_c['x'], optimal_c['fun']&#xA;    print(optimal_c)&#xA;    return min_x, min_f&#xA;&#xA;min_f = 10**10&#xA;min_x = []&#xA;min_x, min_f = hopping_solver(min_f, min_x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My questions relate to the step essentially: My first elements are angles in [0, 360] and the others are in [0, 1.2]. What is a suitable way to define a good step here to balance between precision and fast processing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that this code does not return the global minimum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also here I chose the center of my solutions space as &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt;, are there recommendations on how to choose the starting point?&lt;/p&gt;&#xA;" OwnerUserId="30253" LastEditorUserId="20688" LastEditDate="2019-03-20T14:58:08.457" LastActivityDate="2019-03-20T16:28:28.480" Title="Solve multi-dimensional optimization problem using basinhopping" Tags="|optimization|python|scipy|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="31259" PostTypeId="1" CreationDate="2019-03-19T16:37:20.403" Score="1" ViewCount="114" Body="&lt;p&gt;I'm trying to improve the speed of the following iteration to calculate &lt;span class=&quot;math-container&quot;&gt;$s_k$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$B_k^{-1} = \Bigg( I + \frac{s_{k}s_{k-1}^T}{||s_{k-1}||^2}\Bigg)...\Bigg(I+ \frac{s_1s_0^T}{||s_0||^2}\Bigg) B_0^{-1}\\s_{k+1} = -\frac{B_k^{-1}r_{k+1}}{1+\frac{s_k^TB_k^{-1}r_{k+1}}{||s_k||^2}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we calculate &lt;span class=&quot;math-container&quot;&gt;$B_k^{-1}$&lt;/span&gt; explicitly then each step is roughly &lt;span class=&quot;math-container&quot;&gt;$O(n^4)$&lt;/span&gt; to calculate &lt;span class=&quot;math-container&quot;&gt;$B_k^{-1}$&lt;/span&gt;. If instead we compute &lt;span class=&quot;math-container&quot;&gt;$B_k^{-1}r_{k+1}$&lt;/span&gt; by expanding the product from the right, we do &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; matrix-vector multiplications which is &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt;, a lot better. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to find &lt;span class=&quot;math-container&quot;&gt;$s_k$&lt;/span&gt; in the form of a low rank update using the value of previous &lt;span class=&quot;math-container&quot;&gt;$s_i$&lt;/span&gt;? &lt;/p&gt;&#xA;" OwnerUserId="30254" LastActivityDate="2019-08-17T03:04:20.573" Title="Product of rank one updates as a low rank update for quasi newton/BFGS" Tags="|linear-algebra|newton-method|quasi-newton|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31260" PostTypeId="1" AcceptedAnswerId="32935" CreationDate="2019-03-19T20:39:13.167" Score="4" ViewCount="115" Body="&lt;p&gt;We have the following generalized eigenvalue (set of) problem(s)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$[K_R(\kappa)]\{u_R\} = \omega^2[M_R(\kappa)]\{u_R\}\quad \forall \kappa \in [\kappa_0, \kappa_1]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;K_R(\kappa) = T^H(\kappa) K T(\kappa)\, ,\\&#xA;&amp;amp;M_R(\kappa) = T^H(\kappa) M T(\kappa)\, ,\\&#xA;&amp;amp;u = T(\kappa) u_R\, .&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; are sparse and symmetric and come from a PDE, and &lt;span class=&quot;math-container&quot;&gt;$T(\kappa)$&lt;/span&gt; is also sparse, non-square, complex and represents a set of multipoint constraints. Forming the matrix &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is relatively cheap compared with the other matrices since they have a fixed structure and are sparse.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we consider a regular mesh with &lt;span class=&quot;math-container&quot;&gt;$n^2$&lt;/span&gt; nodes, we have that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;K\in \mathbb{R}^{n^2\times n^2}\, ,&amp;amp;M\in \mathbb{R}^{n^2\times n^2}\, ,\\&#xA;&amp;amp;u\in \mathbb{C}^{n^2}\, , &amp;amp; &amp;amp;\\&#xA;&amp;amp;K_R\in \mathbb{C}^{(n^2 - 2n + 1)\times (n^2 - 2n + 1)}\, ,&amp;amp;M_R\in \mathbb{C}^{(n^2 - 2n + 1)\times (n^2 - 2n + 1)}\, ,&amp;amp;\\&#xA;&amp;amp;u_R\in \mathbb{C}^{(n^2 - 2n + 1)}\, , &amp;amp; &amp;amp;\\&#xA;&amp;amp;T(\kappa) \in \mathbb{C}^{(2n -1)\times(n^2 - 2n + 1)}\, .&amp;amp; &amp;amp;&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We normally handle the problem in one of the following ways:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Assemble &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; and for each value of &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; form the product matrices &lt;span class=&quot;math-container&quot;&gt;$K_R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$M_R$&lt;/span&gt;. The main advantage of this method is that we have to assemble once, but then we loose the sparse nature of the problem.&lt;/li&gt;&#xA;&lt;li&gt;Assemble &lt;span class=&quot;math-container&quot;&gt;$K_R$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$M_R$&lt;/span&gt; for each &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; value, conserving the sparse nature of the problem.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Is there any way of solving the generalized eigenvalue problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$[T^H(\kappa) K T(\kappa)]\{u_R\} = \omega^2[T^H(\kappa) M T(\kappa)]\{u_R\}\quad \forall \kappa \in [\kappa_0, \kappa_1]\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;conserving the sparse nature of the system and assembling the matrices &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; only once?&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2019-06-21T22:29:47.363" Title="Solver for generalized eigenvalue problem with multipoint constraints" Tags="|finite-element|pde|sparse-matrix|eigenvalues|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31261" PostTypeId="2" ParentId="31217" CreationDate="2019-03-19T22:29:57.777" Score="0" Body="&lt;p&gt;If you are using a (classical) finite element formulation you have &lt;span class=&quot;math-container&quot;&gt;$C^0$&lt;/span&gt; continuity, that is, your function is continuous but its derivatives are not. Thus, it is expected to have that behavior towards the end, since there the derivatives are larger. As you refine these two curves will get closer to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in my comment, you can do some &lt;em&gt;smoothing&lt;/em&gt; of your derivatives. I have used the following approach (see [1]).:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The variable is extrapolated from the Gauss point to nodes for each element. &lt;/li&gt;&#xA;&lt;li&gt;These values are averaged according to the number of elements that share that node.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;References&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Zienkiewicz, Olgierd Cecil, and Jian Zhong Zhu. &quot;The superconvergent patch recovery and a posteriori error estimates. Part 1: The recovery technique.&quot; International Journal for Numerical Methods in Engineering 33.7 (1992): 1331-1364.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2019-03-19T22:29:57.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31263" PostTypeId="2" ParentId="31259" CreationDate="2019-03-20T02:05:19.763" Score="1" Body="&lt;p&gt;You can do this using the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sherman%E2%80%93Morrison_formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sherman-Morrison Formula&lt;/a&gt; and some other tricks, detailed starting on page 124 of &lt;a href=&quot;https://archive.siam.org/books/textbooks/fr16_book.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kelley, Iterative Methods for Linear and Nonlinear Equations&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29512" LastActivityDate="2019-03-20T02:05:19.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31264" PostTypeId="1" AcceptedAnswerId="31281" CreationDate="2019-03-20T05:36:58.097" Score="8" ViewCount="252" Body="&lt;p&gt;Given a large matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; with eigenvalues &lt;span class=&quot;math-container&quot;&gt;$\sigma_1\ge \sigma_2 \ge \dotsc $&lt;/span&gt;, I want to determine only a subset of these values, say &lt;span class=&quot;math-container&quot;&gt;$\sigma_5,\sigma_8$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma_{19}$&lt;/span&gt;. Is there an algorithm that can do this, or is finding the top 19 eigenvalues the best that can be done?&lt;/p&gt;&#xA;" OwnerUserId="30257" LastEditorUserId="9667" LastEditDate="2019-03-20T15:08:54.050" LastActivityDate="2019-03-22T08:11:41.340" Title="Finding the $i$-th largest eigenvalue of a matrix" Tags="|eigenvalues|svd|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="31265" PostTypeId="1" CreationDate="2019-03-20T09:59:06.387" Score="2" ViewCount="100" Body="&lt;p&gt;I have a linear system of the type&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Ax = b$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to minimise &lt;span class=&quot;math-container&quot;&gt;$|b - Ax|^2$&lt;/span&gt;.&#xA;I know there are different approaches to directly solve the system (Normal equation + Cholesky, QR decomposition, SVD decomposition) that have different numerical stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to ask how it is possible to estimate the impact of the numerical error on the estimates of each parameter &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Can it be done solely on the base of the condition number? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Should it be done using Monte Carlo simulation?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How should such simulation be designed?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30259" LastEditorUserId="30259" LastEditDate="2019-03-22T08:17:38.597" LastActivityDate="2019-03-22T08:17:38.597" Title="Assessing numerical error in solving a least squares problem" Tags="|linear-solver|stability|condition-number|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31266" PostTypeId="2" ParentId="31089" CreationDate="2019-03-20T10:10:05.437" Score="1" Body="&lt;p&gt;In &lt;em&gt;Gandibleux, X. (Ed.). (2006). Multiple criteria optimization: state of the art annotated bibliographic surveys (Vol. 52). Springer Science &amp;amp; Business Media.&lt;/em&gt;, I found in chapter 5 many interactive nonlinear multiobjective procedures explained. According to Kaisa Miettinen, none is better than another and they all have their pros and cons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can  cite :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Interactive Surrogate Worth Trade-Off Method [1]&lt;/li&gt;&#xA;&lt;li&gt;Geoffrion-Dyer-Feinberg Method [2]&lt;/li&gt;&#xA;&lt;li&gt;Tchebycheff Method [3]&lt;/li&gt;&#xA;&lt;li&gt;Step Method [4]&lt;/li&gt;&#xA;&lt;li&gt;Reference Point Method [5]&lt;/li&gt;&#xA;&lt;li&gt;GUESS Method [6]&lt;/li&gt;&#xA;&lt;li&gt;Satisficing Trade-Off Method [7]&lt;/li&gt;&#xA;&lt;li&gt;Light Beam Search [8]&lt;/li&gt;&#xA;&lt;li&gt;Reference Direction Approach [9]&lt;/li&gt;&#xA;&lt;li&gt;Reference Direction Method [10]&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;NIMBUS Method [11]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1]: V. Chankong and Y. Y. Haimes. Multiobjective Decision Making Theory and Methodology. Elsevier Science Publishing Co., New York, 1983&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2]: A. M. Geoffrion, J. S. Dyer, and A. Feinberg. An interactive&#xA;approach for multi-criterion optimization, with an application to&#xA;the operation of an academic department. Management Science,&#xA;19:357–368, 1972&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3]: R. E. Steuer. The Tchebycheff procedure of interactive multiple objective programming. In B. Karpak and S. Zionts, editors, Multiple Criteria Decision Making and Risk Analysis Using Microcomputers, pages 235–249. Springer-Verlag, Berlin, Heidelberg,&#xA;1989.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4]: R. Benayoun, J. de Montgolfier, J. Tergny, and O. Laritchev. Linear programming with multiple objective functions: Step method&#xA;(STEM). Mathematical Programming, 1:366–375, 1971&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[5]: A. P. Wierzbicki. The use of reference objectives in multiobjective optimization. In G. Fandel and T. Gal, editors, Multiple Criteria Decision Making Theory and Applications, pages 468–486.&#xA;Springer-Verlag, Berlin, Heidelberg, 1980&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[6]: J. T. Buchanan. A naïve approach for solving MCDM problems:&#xA;The GUESS method. Journal of the Operational Research Society,&#xA;48:202–206, 1997.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[7]: H. Nakayama. Aspiration level approach to interactive multi-&#xA;objective programming and its applications. In P. M. Pardalos,&#xA;Y. Siskos, and C. Zopounidis, editors, Advances in Multicriteria&#xA;Analysis, pages 147–174. Kluwer Academic Publishers, Dordrecht,&#xA;1995&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[8]: A. Jaszkiewicz The light beam search – outranking based interactive procedure for multiple-objective mathematical programming. In P. M. Pardalos, Y. Siskos, and C. Zopounidis,&#xA;editors, Advances in Multicriteria Analysis, pages 129–146. Kluwer&#xA;Academic Publishers, Dordrecht, 1995.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[9]: P. Korhonen. Reference direction approach to multiple objective linear programming: Historical overview. In M. H. Karwan,&#xA;J. Spronk, and J. Wallenius, editors, Essays in Decision Making:&#xA;A Volume in Honour of Stanley Zionts, pages 74–92. Springer-&#xA;Verlag, Berlin, Heidelberg, 1997&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[10]: S. C. Narula, L. Kirilov, and V. Vassilev. An interactive algorithm&#xA;for solving multiple objective nonlinear programming problems. In&#xA;G. H. Tzeng, H. F. Wand, U. P. Wen, and P. L. Yu, editors, Multiple Criteria Decision Making – Proceedings of the Tenth International Conference: Expand and Enrich the Domains of Thinking&#xA;and Application, pages 119–127. Springer-Verlag, New York, 1994&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[11]: K. Miettinen. Nonlinear Multiobjective Optimization. Kluwer Academic Publishers, Boston, 1999&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30024" LastEditorUserId="30024" LastEditDate="2019-07-08T09:34:22.503" LastActivityDate="2019-07-08T09:34:22.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31267" PostTypeId="1" AcceptedAnswerId="31276" CreationDate="2019-03-20T10:49:33.563" Score="0" ViewCount="125" Body="&lt;p&gt;I am trying to write on paper the piecewise polynomials given by the &lt;code&gt;splinefit&lt;/code&gt; function, but I am having some problems figuring out what the coefficients should be. So, the output from the script gives me the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  scalar structure containing the fields:&#xA;&#xA;form = pp&#xA;breaks =&#xA;&#xA; Columns 1 through 15:&#xA;&#xA;  -1.00000  -0.93750  -0.87500  -0.81250  -0.75000  -0.68750  -0.62500  -0.56250  -0.50000   0.00000   0.50000   0.56250   0.62500   0.68750   0.75000&#xA;&#xA; Columns 16 through 19:&#xA;&#xA;   0.81250   0.87500   0.93750   1.00000&#xA;&#xA;coefs =&#xA;&#xA;   2.5644e+03  -3.1686e+02   7.4332e+00  -1.0744e-01&#xA;  -2.5013e+02   1.6397e+02  -2.1226e+00  -2.5453e-01&#xA;  -2.4859e+03   1.1707e+02   1.5442e+01   1.9225e-01&#xA;   2.4278e+03  -3.4904e+02   9.4429e-01   1.0078e+00&#xA;   2.0742e+01   1.0618e+02  -1.4235e+01   2.9610e-01&#xA;  -9.0213e+02   1.1007e+02  -7.1939e-01  -1.7374e-01&#xA;   3.6900e+02  -5.9083e+01   2.4670e+00  -9.0075e-03&#xA;  -5.4428e+01   1.0105e+01  -5.9408e-01   4.4780e-03&#xA;   9.4209e-02  -1.0004e-01   3.1240e-02  -6.4667e-03&#xA;  -5.9852e-02   4.1271e-02   1.8542e-03  -4.0812e-03&#xA;  -5.3315e+01  -4.8507e-02  -1.7641e-03  -3.1792e-04&#xA;   3.5957e+02  -1.0045e+01  -6.3261e-01  -1.3634e-02&#xA;  -8.6341e+02   5.7374e+01   2.3254e+00  -4.6254e-03&#xA;  -4.0925e+01  -1.0452e+02  -6.2096e-01   1.5404e-01&#xA;   2.4543e+03  -1.1219e+02  -1.4165e+01  -3.0303e-01&#xA;  -2.4445e+03   3.4799e+02   5.7280e-01  -1.0274e+00&#xA;  -2.7185e+02  -1.1035e+02   1.5426e+01  -2.2903e-01&#xA;   2.2566e+03  -1.6132e+02  -1.5531e+00   2.3768e-01&#xA;&#xA;pieces =  18&#xA;order =  4&#xA;dim =  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I plot the function via ppval, I get what I expect. But, if I take the first line of the coeff matrix, as I interpret it, I can write the  me the first cubic polynomial as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$f(x) = 2.5644 \times 10^3 x^3 -3.1686\times10^2 x^2 + 7.4332 x - 1.0744\times10^{-1}, \ -1 &amp;lt; x &amp;lt; -0.9375$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just as a reference, &lt;span class=&quot;math-container&quot;&gt;$f(x=-1)\approx 0$&lt;/span&gt; in the data I have, but clearly from the equation just shown that's not the case.&lt;/p&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="9667" LastEditDate="2019-03-20T15:23:43.813" LastActivityDate="2019-04-20T18:00:08.363" Title="How to write in paper the equations given by splinefit?" Tags="|matlab|octave|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31268" PostTypeId="1" AcceptedAnswerId="31272" CreationDate="2019-03-20T10:51:38.980" Score="1" ViewCount="1375" Body="&lt;p&gt;I would like to solve the following differential equation numerically in 2D,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial z^-}{\partial t}+(\vec{B}\cdot\vec{\nabla})z^-=0,$$&lt;/span&gt;&#xA;see &lt;a href=&quot;https://en.wikipedia.org/wiki/Magnetohydrodynamic_turbulence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; if you are curious about what the symbols mean, where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\vec{B}=(B_x,B_y,0).$$&lt;/span&gt;&#xA;I have attached some code which can solve this to first order accuracy in python by following the corner transport upstream (CTU) algorithm outlined &lt;a href=&quot;https://ocw.mit.edu/courses/earth-atmospheric-and-planetary-sciences/12-950-atmospheric-and-oceanic-modeling-spring-2004/lecture-notes/lec10.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The problem is the code is too diffusive for my purposes. Does anyone know of an algorithm I could follow to make the code a higher order of accuracy while still remaining stable? Also, it would nice if the algorithm was an upstream algorithm i.e. each cell update only required information from cells in the upstream direction as it is easier to impose open boundary conditions if this is the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the python code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;import os&#xA;&#xA;nx = 64&#xA;x_min = -2&#xA;x_max = 2&#xA;dx = (x_max - x_min) / (nx - 1)&#xA;x = np.linspace(x_min, x_max, nx)&#xA;&#xA;ny = 64&#xA;y_min = -2&#xA;y_max = 2&#xA;dy = (y_max - y_min) / (ny - 1)&#xA;y = np.linspace(y_min, y_max, ny)&#xA;&#xA;bx = np.zeros((nx,ny)) + 1&#xA;by = np.zeros((nx,ny)) + 1&#xA;&#xA;t_max = 1&#xA;dt = dx * dy / np.max(abs(bx) * dy + abs(by) * dx) # CFL condition&#xA;nt = int(t_max / dt)&#xA;t = np.linspace(0, t_max, nt)&#xA;&#xA;zm = np.zeros((nx, ny, nt))&#xA;for i in range(nx):&#xA;    for j in range(ny):&#xA;        r = np.sqrt(x[i] ** 2 + y[j] ** 2)&#xA;        if r &amp;lt; 1:&#xA;            zm[i,j,0] = 2 * np.cos(np.pi * r / 2) ** 2&#xA;&#xA;print(nt)&#xA;zm1 = np.zeros((nx, ny))&#xA;for n in range(nt - 1):&#xA;&#xA;    if n % 100 == 0: print('n =', n)&#xA;&#xA;    cx = bx[1:,1:] * dt / dx&#xA;    cy = by[1:,1:] * dt / dy&#xA;    zm1[1:,1:]    = (1 - cx) * zm[1:,1:,n] + cx * zm[0:-1,1:,n]&#xA;    zm[1:,1:,n+1] = (1 - cy) * zm1[1:,1:]  + cy * zm1[1:,0:-1]&#xA;&#xA;    zm[0,:,n+1]  = 0&#xA;    zm[:,0,n+1]  = 0&#xA;&#xA;print('Computation finished, now generating figures...')&#xA;&#xA;os.makedirs('Figures/2d/zm', exist_ok = True)&#xA;&#xA;X = np.zeros((nx,ny))&#xA;Y = np.zeros((nx,ny))&#xA;for j in range(ny):&#xA;    X[:,j] = x&#xA;for i in range(nx):&#xA;    Y[i,:] = y&#xA;i = -1&#xA;for n in range(nt - 1):&#xA;    # if n % 10 == 0:&#xA;        i = i + 1&#xA;        fig = plt.figure()&#xA;        ax = fig.gca(projection = '3d')&#xA;        ax.plot_surface(X, Y, zm[:,:,n])&#xA;        plt.title('t = ' + str(t[n]))&#xA;        ax.set_xlabel('x')&#xA;        ax.set_ylabel('y')&#xA;        ax.set_zlabel('zp')&#xA;        ax.set_zlim(-1, 1)&#xA;        plt.savefig('Figures/2d/zm/' + &quot;{0:0=4d}&quot;.format(i) + '.png')&#xA;        plt.close(fig)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My attempt at Beam-Warming (see comments):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;import os&#xA;&#xA;nx = 68&#xA;x_min = 0&#xA;x_max = 2&#xA;dx = (x_max - x_min) / (nx - 1)&#xA;x = np.linspace(x_min - dx, x_max + dx, nx) # Add ghost cells&#xA;&#xA;ny = 68&#xA;y_min = -2&#xA;y_max = 0&#xA;dy = (y_max - y_min) / (ny - 1)&#xA;y = np.linspace(y_min - dy, y_max + dy, ny)&#xA;&#xA;bx = np.zeros((nx,ny)) + 1&#xA;by = np.zeros((nx,ny)) + 1&#xA;&#xA;t_max = 10&#xA;dt = dx * dy / np.max(abs(bx) * dy + abs(by) * dx) # CFL condition&#xA;nt = int(t_max / dt)&#xA;t = np.linspace(0, t_max, nt)&#xA;&#xA;zm = np.zeros((nx, ny, nt))&#xA;zp = np.zeros((nx, ny, nt))&#xA;for i in range(nx):&#xA;    for j in range(ny):&#xA;        r = np.sqrt((x[i] - 1) ** 2 + (y[j] + 1) ** 2)&#xA;        if r &amp;lt; 1:&#xA;            zm[i,j,0] = 2 * np.cos(np.pi * r / 2) ** 2&#xA;&#xA;print(nt)&#xA;for n in range(nt - 1):&#xA;&#xA;    if n % 100 == 0: print('n =', n)&#xA;&#xA;    zm[2:,2:,n+1] = zm[2:,2:,n] \&#xA;                  - 0.5 * dt * bx[2:,2:] / dx * \&#xA;                    (3 * zm[2:,2:,n] - 4 * zm[1:-1,2:,n] + zm[0:-2,2:,n]) \&#xA;                  + 0.5 * (dt * bx[2:,2:] / dx) ** 2 * \&#xA;                    (zm[2:,2:,n] - 2 * zm[1:-1,2:,n] + zm[0:-2,2:,n]) \&#xA;                  - 0.5 * dt * by[2:,2:] / dy * \&#xA;                    (3 * zm[2:,2:,n] - 4 * zm[2:,1:-1,n] + zm[2:,0:-2,n]) \&#xA;                  + 0.5 * (dt * by[2:,2:] / dy) ** 2 * \&#xA;                    (zm[2:,2:,n] - 2 * zm[2:,1:-1,n] + zm[2:,0:-2,n])&#xA;&#xA;    zm[0,:,n+1]  = 0&#xA;    zm[:,0,n+1]  = 0&#xA;&#xA;print('Computation finished, now generating figures...')&#xA;&#xA;os.makedirs('Figures/2d/zm', exist_ok = True)&#xA;&#xA;X = np.zeros((nx,ny))&#xA;Y = np.zeros((nx,ny))&#xA;for j in range(ny):&#xA;    X[:,j] = x&#xA;for i in range(nx):&#xA;    Y[i,:] = y&#xA;i = -1&#xA;for n in range(nt - 1):&#xA;    if n % 100 == 0:&#xA;        i = i + 1&#xA;        fig = plt.figure()&#xA;        ax = fig.gca(projection = '3d')&#xA;        ax.plot_surface(X, Y, zm[:,:,n])&#xA;        plt.title('t = ' + str(t[n]))&#xA;        ax.set_xlabel('x')&#xA;        ax.set_ylabel('y')&#xA;        ax.set_zlabel('zp')&#xA;        ax.set_zlim(-1, 1)&#xA;        plt.savefig('Figures/2d/zm/' + &quot;{0:0=4d}&quot;.format(i) + '.png')&#xA;        plt.close(fig)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Pictures of the instability:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/T900m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T900m.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sVc0A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sVc0A.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="26454" LastEditorUserId="26454" LastEditDate="2019-03-20T16:42:30.033" LastActivityDate="2021-12-13T08:34:31.207" Title="How to make a less diffusive code to solve 2D advection equation?" Tags="|finite-difference|finite-volume|advection-diffusion|" AnswerCount="3" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31269" PostTypeId="2" ParentId="31265" CreationDate="2019-03-20T15:20:10.793" Score="2" Body="&lt;p&gt;I think most solvers will give you a residual error. If you flip your equation to read:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Ax - b = r\approx 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solver will usually provide you with the residual error like: &lt;span class=&quot;math-container&quot;&gt;$|r|$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if you want to compare different solvers you might use that as a measurement of the error. As a rule of thumb: you will not find a solver which is optimal for every System A. Usually there are also speed/stability tradeofs or such.&lt;/p&gt;&#xA;" OwnerUserId="28636" LastActivityDate="2019-03-20T15:20:10.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31270" PostTypeId="2" ParentId="31258" CreationDate="2019-03-20T16:28:28.480" Score="3" Body="&lt;p&gt;Geometrically, you are trying to find a point that is (i) as close as possible to the point &lt;span class=&quot;math-container&quot;&gt;$\mathbf c_{ref}$&lt;/span&gt; and (ii) as close as possible to the sphere of radius 2. Your objective function is the sum of these two distances. The solution is that point &lt;span class=&quot;math-container&quot;&gt;$\mathbf c$&lt;/span&gt; that is half-way between &lt;span class=&quot;math-container&quot;&gt;$\mathbf c_{ref}$&lt;/span&gt; and the (closest point on the) sphere of radius 2, i.e., the solution if the vector&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \mathbf c  = \frac 12 \left( \mathbf c_{ref} + 2 \frac{\mathbf c_{ref}}{\|\mathbf c_{ref}\|}\right).&#xA;$$&lt;/span&gt;&#xA;Written differently, you get that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \mathbf c  = \left( \frac 12  + \frac{1}{\|\mathbf c_{ref}\|}\right)&#xA;  \mathbf c_{ref}.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-03-20T16:28:28.480" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31272" PostTypeId="2" ParentId="31268" CreationDate="2019-03-21T15:21:35.927" Score="1" Body="&lt;p&gt;I am unfamiliar with the CTU scheme, but if you are running into problems of numerical dissipation in advection problems, one easy approach might be to do higher order upwinding. That basically means that when you want to calculate the flux at a certain point, you adjust your derivative to be calculated in the &quot;upwind&quot; direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are on a structured grid like your images suggest, that might be an easy fix.&#xA;The right/left discretisations for your gradient should keep simple form like &lt;a href=&quot;https://en.wikipedia.org/wiki/Upwind_scheme&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;left&quot;-leaning discretization:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA; \frac{du}{dx} = \frac{3u_i^n - 4u_{i-1}^n + u_{i-2}^n}{2\Delta x}&#xA;$$&lt;/span&gt;&#xA;&quot;right&quot;-leaning discretsation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;  \frac{du}{dx}  = \frac{-u_{i+2}^n + 4u_{i+1}^n - 3u_i^n}{2\Delta x}&#xA;$$&lt;/span&gt;&#xA;Then at every point you check where the wind is blowing from and discretise in the suggested fashion. You will have to be a bit carefull at the border of your domains, as you might not have first and second neighbours in the direction of the border. That should somewhat alleviate your numerical dissipation problems, but some of it will persist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might also want to check whether your time-stepping size might be too big. The relevant keyword here is the CFL-condition. Just check that the advection is not so strong, that the transported distance in the time dt exceedes your cell-size. Because if that happens then the discretisation will fail miserably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="28636" LastActivityDate="2019-03-21T15:21:35.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31273" PostTypeId="2" ParentId="31205" CreationDate="2019-03-21T15:46:04.730" Score="2" Body="&lt;p&gt;The neat thing about Fourier codes is that you can achieve very high order derivatives and if you are describing physics which is about waves, then the trigonometric base functions are a good choice. You also have a reasonably fast transform in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(n\log(n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bad thing about Fourier codes is, that they can &lt;em&gt;only&lt;/em&gt; really work well in &lt;strong&gt;periodic settings&lt;/strong&gt;, and &lt;strong&gt;they refuse to work wherever you have jumps&lt;/strong&gt; in your profile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way to describe this is &lt;a href=&quot;https://en.wikipedia.org/wiki/Gibbs_phenomenon&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki: Fourier rect&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Gibbs_phenomenon&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki: Gibbs phenomenon&lt;/a&gt;. If you transform a rect-function into Fourier space you get a nasty sinc function in your spectrum. If your barrier should look something like this, then your Fourier spectrum will deteriorate rapidly and you will not be able to fix that with increasing resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might at least be advisable to have your barrier be 'smooth' so that the Fourier base functions have a chance at estimating them. It might be advisable to move away from the Fourier-base towards FEM/FV altogether.&lt;/p&gt;&#xA;" OwnerUserId="28636" LastEditorUserId="20688" LastEditDate="2019-03-21T15:55:16.057" LastActivityDate="2019-03-21T15:55:16.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31274" PostTypeId="2" ParentId="31252" CreationDate="2019-03-21T16:00:43.847" Score="1" Body="&lt;p&gt;In a cell centered finite volume setting your are discretising the fluxes over your cell boundary as you indicated. Lets say we look at one face between the cubes A and B, the calculated flux would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_{AB} = \frac{1}{\frac{1}{a_{A}} + \frac{1}{a_{B}} } ~\frac{u_B-u_A}{h} ~ F&#xA;$$&lt;/span&gt;&#xA;with the fraction is the harmonic average (!) between the diffusivities in the two cells, the discretized gradient and &lt;span class=&quot;math-container&quot;&gt;$F$&lt;/span&gt; being the area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know what software/code you are using, but whenever you are at a boundary face, simply swap the value u_B with the value you want to enforce at the boundary! Then the flux is just as if the border is another neighbour cell, with the distinction that it does not change. If you want to impose no flux conditions (i.e. du/dx = 0 at the border) then you set the whole flux to zero, as there should be no flux if the derivative is zero!&lt;/p&gt;&#xA;" OwnerUserId="28636" LastActivityDate="2019-03-21T16:00:43.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31275" PostTypeId="2" ParentId="31198" CreationDate="2019-03-21T16:19:42.373" Score="1" Body="&lt;p&gt;Would it be possible to do the 1 billion runs method-parallel? Even if there is no way to parallelize the interior of the method body, depending on your problem, you might reach some form of concurrency by evoking the method in parallel. If you do not re-use your output y[100] as an input x[10] to the function then nobody stops you from executing it within different threads/cores/machines etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;pseudocode:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;open_one_zillion_threads();&#xA;  x = get_threadlocal_x();&#xA;  y = get_threadlocal_y();&#xA;  evaluate(const  x[10],  y[100]);&#xA;  store_y_in_proper_place(y[100]);&#xA;rejoin_threads();&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;also:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are your precision constraints concerning the output? Is the result necessary in double precision? Do you really need all aspects of the output? In many problems there is a reasonable tradeoff between precision and speed and even switching from double to single might boost performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;another idea:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be possible to express the operations in a form using matrices and vectors. If it is all simple operations you might interpret your input array as a vector, and apply a series of matrix ops to it which represent the layers. These matrices will not be square matrices:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;y[100] = A( B( C( D ~x[10]) ) )&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might be counter intuitive as there will be loads of zeros in your matrix, but it opens your problem up to use powerful and well optimized LA-Packages (Lapack / Blas etc.) which possibly bring you out-of-the-box parallelization on those matrix vector products and they might be better at using SIMD / AVX etc. If you somehow can make full use of AVX registers in the matrix-vector operations, then having a couple of them contain zeros does not hurt you too much. The Matrix approach would put you in a place where there is loads of online material and libraries and you are more flexible if you want to change certain parts of that method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are on an intel cpu, then using their homemade intel compiler might give you another &lt;a href=&quot;https://colfaxresearch.com/compiler-comparison/#sec-3-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;50%&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="28636" LastEditorUserId="28636" LastEditDate="2019-03-22T10:07:15.307" LastActivityDate="2019-03-22T10:07:15.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31276" PostTypeId="2" ParentId="31267" CreationDate="2019-03-21T16:25:12.040" Score="1" Body="&lt;p&gt;I found the answer! Octave/MATLAB's function &lt;code&gt;splinefit&lt;/code&gt; differs from &lt;code&gt;polyfit&lt;/code&gt; in the sense that the polynomial in the former is proposed as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{n=0}^k a_n(x-x_o)^n$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and in the latter&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{n=0}^k a_n x^n$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt; refers to the range &lt;span class=&quot;math-container&quot;&gt;$[x_0,x_f]$&lt;/span&gt; of the absissa.&lt;/p&gt;&#xA;" OwnerUserId="14659" LastActivityDate="2019-03-21T16:25:12.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31277" PostTypeId="2" ParentId="31255" CreationDate="2019-03-21T21:09:12.433" Score="0" Body="&lt;p&gt;You're asking for a method, but let me point you to a specific software package - &lt;a href=&quot;https://www.mat.univie.ac.at/~neum/software/snobfit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNOBFIT&lt;/a&gt;. I don't think it can handle discrete variables, but it does handle noisy cost function evaluations, which is critical when optimizing using numerical methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a Matlab package. It doesn't look like it's been updated recently. There is a Python port as well as a link to a paper describing the method and implementation.&lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2019-03-21T21:09:12.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31279" PostTypeId="1" CreationDate="2019-03-21T21:40:44.380" Score="3" ViewCount="89" Body="&lt;p&gt;I have read a lot about numerical integrators (ode solvers) lately and tried reading a few papers but I have stumbled upon something that I can't understand and it's something called ABA and BAB. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example in this paper &lt;a href=&quot;https://arxiv.org/pdf/1208.0689.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;New families of symplectic splitting methods for numerical integration in dynamical astronomy&lt;/a&gt; they have tables with coefficients but I haven't figured out how to interpret those and code an algorithm with them because I haven't found what ABA and BAB schemes are, Google isn't helping me a lot sadly. How are those tables meant to be interpreted?&lt;/p&gt;&#xA;" OwnerUserId="30266" LastEditorUserId="20688" LastEditDate="2019-03-21T22:26:15.277" LastActivityDate="2019-03-22T10:58:39.013" Title="What is ABA and BAB schemes when talking about numerical integrators" Tags="|ode|terminology|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31280" PostTypeId="1" AcceptedAnswerId="31284" CreationDate="2019-03-21T22:14:58.963" Score="3" ViewCount="161" Body="&lt;p&gt;I'm trying to read papers on numerical ODE algorithms and I always seem to stumble upon huge amounts of exponentials multiplied by each other. For example in &lt;a href=&quot;https://arxiv.org/pdf/1208.0689.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;New families of symplectic splitting methods for numerical integration in dynamical astronomy&lt;/a&gt; there are a few of them. Is it the ordinary exponential that is meant or is it just like a way of writing something else (just like with complex numbers)?&lt;/p&gt;&#xA;" OwnerUserId="30266" LastEditorUserId="22284" LastEditDate="2019-03-26T15:31:02.147" LastActivityDate="2019-03-26T15:31:02.147" Title="What does the exponential function mean in numerical ODE solving formulas?" Tags="|ode|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31281" PostTypeId="2" ParentId="31264" CreationDate="2019-03-22T08:11:41.340" Score="6" Body="&lt;p&gt;No, there is nothing, as far as I know, unless you know approximately the location of these eigenvalues. As for methods that can compute a subset of the spectrum of a matrix, I know only of methods that can produce:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;eigenvalues from &quot;the extremes of the spectrum&quot;, e.g., the ones with the largest absolute value, or with the most negative real/imaginary part. For instance Arnoldi / subspace iteration (suitably shifted).&lt;/li&gt;&#xA;&lt;li&gt;eigenvalues closest to one given point in the complex plane, e.g., the smallest ones in modulus (closest to zero). For instance (rational) shift-and-invert Arnoldi or the inverse power iteration.&lt;/li&gt;&#xA;&lt;li&gt;eigenvalues in a particular region of the plane, for instance &lt;span class=&quot;math-container&quot;&gt;$[-1,1] \times [-1,1]$&lt;/span&gt;. I think FEAST-type methods do this, but I know very little about them.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;From what I understand, with FEAST-type methods you can also get a count of the number of eigenvalues in a specified region of the complex plane (via a contour integral), so in case you need a specified eigenvalue from the middle of the spectrum, for instance the 300th, you can run a sort of bisection search: assume (shifting and scaling) that the spectrum of your matrix is in &lt;span class=&quot;math-container&quot;&gt;$[-1,1]$&lt;/span&gt; (and that is real, for simplicity). Compute the eigenvalues &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; closest to &lt;span class=&quot;math-container&quot;&gt;$\frac12$&lt;/span&gt;. Compute the number of eigenvalues in &lt;span class=&quot;math-container&quot;&gt;$[\frac12,1]$&lt;/span&gt;. If it is larger than 300, look for the eigenvalue closest to the midpoint of &lt;span class=&quot;math-container&quot;&gt;$[\mu,1]$&lt;/span&gt;, otherwise for the eigenvalue closest to the midpoint of &lt;span class=&quot;math-container&quot;&gt;$[-1,\mu]$&lt;/span&gt;. Repeat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think any of this stuff would be competitive with vanilla &lt;code&gt;LA&lt;/code&gt; Arnoldi if you replace 300 with 19, as in your example. And maybe not even with 300.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2019-03-22T08:11:41.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31283" PostTypeId="2" ParentId="31279" CreationDate="2019-03-22T10:58:39.013" Score="2" Body="&lt;p&gt;You may want to check out Section 3 in chapter III of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E. Hairer, C. Lubich, and G. Wanner, Geometric numerical integration: structure-preserving algorithms for ordinary differential equations, Second., vol. 31. Berlin: Springer-Verlag, 2006.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and/or:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.560.2136&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Murua, “The Hopf Algebra of Rooted Trees, Free Lie Algebras, and Lie Series,” Foundations of Computational Mathematics, vol. 6, no. 4, pp. 387–426, Nov. 2006.&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="15" LastActivityDate="2019-03-22T10:58:39.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31284" PostTypeId="2" ParentId="31280" CreationDate="2019-03-22T11:05:01.263" Score="7" Body="&lt;p&gt;This is a way of writing the flow of a vector field. In other words, if you have an ordinary differential equation (ODE) given by &lt;span class=&quot;math-container&quot;&gt;$\dot{x}(t) = v(x(t))$&lt;/span&gt; with initial value &lt;span class=&quot;math-container&quot;&gt;$x(0) = x_0$&lt;/span&gt;, you could &lt;strong&gt;formally&lt;/strong&gt; write its solution as &lt;span class=&quot;math-container&quot;&gt;$x(t) = \mathrm{e}^{t v} x_0$&lt;/span&gt;. The formal solution coincides with the actual solution when &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is a linear mapping (think of the case in which &lt;span class=&quot;math-container&quot;&gt;$v(x) = A x$&lt;/span&gt; for some matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, if we have another ODE &lt;span class=&quot;math-container&quot;&gt;$\dot{y}(t) = w(y(t))$&lt;/span&gt; and an initial value &lt;span class=&quot;math-container&quot;&gt;$y(0) = y_0$&lt;/span&gt;, then we could write &lt;span class=&quot;math-container&quot;&gt;$\mathrm{e}^{t v} \mathrm{e}^{s w} y_0$&lt;/span&gt; to represent the curve resulting from solving the second initial value problem up to time &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, then using &lt;span class=&quot;math-container&quot;&gt;$y(s; y_0)$&lt;/span&gt; as the initial value of the first ODE &lt;span class=&quot;math-container&quot;&gt;$\dot{x} = v(x)$&lt;/span&gt; and solving that initial value problem up to time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. To reiterate, you can also think about the product of exponentials as meaning that you must first obtain the integral curve of the vector field &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; that passes through &lt;span class=&quot;math-container&quot;&gt;$y_0$&lt;/span&gt;, follow said curve until it reaches &lt;span class=&quot;math-container&quot;&gt;$y(s)$&lt;/span&gt;, and then follow the integral curve of the vector field &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; that starts at &lt;span class=&quot;math-container&quot;&gt;$y(s)$&lt;/span&gt; and goes on for another &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; units of time. I hope this helps clarify things a little bit more.&lt;/p&gt;&#xA;" OwnerUserId="15" LastEditorUserId="15" LastEditDate="2019-03-23T20:11:29.193" LastActivityDate="2019-03-23T20:11:29.193" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31285" PostTypeId="1" AcceptedAnswerId="31286" CreationDate="2019-03-22T12:56:12.963" Score="2" ViewCount="327" Body="&lt;p&gt;I have noticed arpack comes with a driver dsdrv1 that exploits symmetry of a real-valued matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to analogously exploit a Hermitian matrix in some way via z--- drivers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The manual discusses Hermitian matrices &lt;a href=&quot;http://li.mit.edu/Archive/Activities/Archive/CourseWork/Ju_Li/MITCourses/18.335/Doc/ARPACK/Lehoucq97.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;in section 3.2.1&lt;/a&gt;, but I'm not sure if this analysis and proposal is already present in the ARPACK suite.&lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2019-03-28T00:32:40.400" Title="Can ARPACK exploit hermiticity when diagonalising a complex matrix?" Tags="|eigenvalues|arpack|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31286" PostTypeId="2" ParentId="31285" CreationDate="2019-03-22T13:37:34.880" Score="6" Body="&lt;p&gt;No, there is no specialized ARPACK routine for complex Hermitian matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ARPACK authors recommend using the &lt;code&gt;znaupd&lt;/code&gt; routine for &lt;em&gt;both Hermitian and non-Hermitian problems&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.caam.rice.edu/software/ARPACK/UG/node43.html#SECTION00790000000000000000&quot; rel=&quot;noreferrer&quot;&gt;https://www.caam.rice.edu/software/ARPACK/UG/node43.html#SECTION00790000000000000000&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reasoning (direct quote from the ARPACK manual):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Occasionally, when using &lt;code&gt;znaupd&lt;/code&gt; on a complex Hermitian problem,&#xA;  eigenvalues will be returned with small but non-zero imaginary part&#xA;  due to unavoidable round-off errors. These should be ignored unless&#xA;  they are significant with respect to the eigenvalues of largest&#xA;  magnitude that have been computed. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;There is little computational&#xA;  penalty for using the non-Hermitian routines in this case. The only&#xA;  additional cost is to compute eigenvalues of a Hessenberg rather than&#xA;  a tridiagonal matrix. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For the problem configurations this software is&#xA;  designed to solve, the size of these matrices are small enough that&#xA;  the differences in computational cost are negligible compared to the&#xA;  major &lt;span class=&quot;math-container&quot;&gt;${\cal O}(n)$&lt;/span&gt; work that is required.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The relevant driver routines are &lt;code&gt;zndrvX&lt;/code&gt;, with &lt;span class=&quot;math-container&quot;&gt;$X = 1 \dots 4 $&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another library that &lt;em&gt;appears&lt;/em&gt; to implement sparse Hermitian eigensolvers is PRIMME, &lt;a href=&quot;http://www.cs.wm.edu/~andreas/software/&quot; rel=&quot;noreferrer&quot;&gt;http://www.cs.wm.edu/~andreas/software/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet another, bigger beast, is &lt;a href=&quot;https://slepc.upv.es/&quot; rel=&quot;noreferrer&quot;&gt;SLEPc&lt;/a&gt; (built on top of petsc). They, too, offer support for sparse Hermitian eigensolvers (&lt;a href=&quot;http://slepc.upv.es/documentation/slepc.pdf&quot; rel=&quot;noreferrer&quot;&gt;user's manual&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="17294" LastEditorUserId="17294" LastEditDate="2019-03-22T13:49:15.817" LastActivityDate="2019-03-22T13:49:15.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31288" PostTypeId="1" CreationDate="2019-03-23T12:10:43.320" Score="7" ViewCount="573" Body="&lt;p&gt;An optimization problem has 3 main components: decision variables, constraints, and an objective function. Such a problem can be mathematically modelled and solved using an optimization solver. For example, a SUDOKU puzzle can be modelled as such and solved using a solver.&lt;/p&gt;&#xA;&lt;p&gt;What are examples of famous real-life problems or puzzles that &lt;strong&gt;cannot&lt;/strong&gt; be modelled as optimization problems? I reviewed relevant literature, but could not find any discussion on problems that cannot be modelled as optimization problems.&lt;/p&gt;&#xA;" OwnerUserId="24966" LastEditorUserId="20688" LastEditDate="2021-12-25T02:29:23.610" LastActivityDate="2023-02-10T17:28:12.563" Title="Examples of problems that cannot be formulated as optimization problems" Tags="|optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31289" PostTypeId="1" AcceptedAnswerId="31304" CreationDate="2019-03-21T20:53:52.307" Score="6" ViewCount="223" Body="&lt;p&gt;When solving a system of ODEs, as part of a preconditioner, I get the system&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(A + D(t))x = b(t)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a sparse matrix and &lt;span class=&quot;math-container&quot;&gt;$D(t)$&lt;/span&gt; is diagonal. I'm currently solving this by taking the LU-decomposition of the system for each &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is constant throughout the solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an algorithm that would allow me to solve this efficiently without recomputing a decomposition at every step?&lt;/p&gt;&#xA;" OwnerUserId="30287" OwnerDisplayName="Sevenless" LastEditorUserId="9667" LastEditDate="2019-03-23T15:44:25.207" LastActivityDate="2019-03-25T21:06:16.667" Title="Efficient approach for solving matrix plus diagonal matrix system that varies in time" Tags="|ode|linear-solver|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31290" PostTypeId="2" ParentId="31289" CreationDate="2019-03-23T13:33:21.017" Score="1" Body="&lt;p&gt;No (unless &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; has few nonzeros). Variants of this question get asked a lot here, and the answer is invariably the same.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2019-03-23T13:33:21.017" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31291" PostTypeId="1" CreationDate="2019-03-23T18:35:33.250" Score="1" ViewCount="32" Body="&lt;p&gt;I am writing some simple code to raise a base value to a power&#xA;then perform some operations on the output.  I have included the code I am&#xA;using now, below ( see end of this message ), also see immediately below&#xA;what this looks like when I run my script from the console ;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C:\PYTHON34&amp;gt;python tst.py&#xA;INPUT Value: 431.1&#xA;INPUT Power Value; 1.9907&#xA;RESULT 1739.554505641426658257842063903808593750000000000000000000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I need to be able to do is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Store whole number value to left of the decimal point without decimals to a variable = x&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Grab entire decimal value to the right of the decimal point and store in a variable = y&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Parse (y) according to some simple rules. Here it is where it gets trick for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to do is examine (y) to see if there are any leading leading and trailing zeros&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example &lt;code&gt;1739.554505.....&lt;/code&gt;, if this value was instead, something like any of the following&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1739.0554505&#xA;1739.00554505&#xA;1739.000554505&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I want to extract the whole number less any continuous leading zeroes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I want to cut the number so that when there are four continuous trailing zeros, all zeros starting from the first one that started off the first four continuous zeros and all following zeroes are truncated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in our example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1739.554505641426658257842063903808593750000000000000000000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(y) becomes &lt;code&gt;55450564142665825784206390380859375&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I want to take the mod of above value by %1999. This returns &lt;code&gt;1407&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I want to join this value AS the decimal value to the right as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1739.1407&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Ok, so now why am I stuck at this point?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm still fairly new to learning Python so I don't actually know what the best ways are to do this.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thinking I could maybe just do a trim on the value to the right of the decimal, take that, convert the whole thing to a string, then chop up each digit and store each individually to an array then code a rule set for stepping through strings in array cells, but this also seems like it could take more CPU  time than I'd like.  I'm hoping to perform this sort of operation fast, as fast as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I do not even know how to best approach this even if I had to store this as a string into an array, and if I understand correctly string processing tends to takes more CPU time than number calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any insight, direction, code suggestions greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code I am currently using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import time&#xA;e0=time.time()&#xA;c0=time.clock()&#xA;&#xA;import sys&#xA;import math&#xA;from sympy import mpmath&#xA;from mpmath import *&#xA;mpmath.mp.dps = 10000&#xA;&#xA;&#xA;inputx = float(input('Enter inputx Value: '))&#xA;powerx =float(input('Enter Power Value; '))&#xA;&#xA;inputx_squared = float((mpmath.power(inputx, powerx)))%1999&#xA;&#xA;print('\r')&#xA;print('RESULT ', '%.60f' % inputx_squared)&#xA;&#xA;elapsed_time=time.time() -e0&#xA;cpu_time=time.clock() -c0&#xA;&#xA;print('\r')&#xA;print(&quot;CPU TIME ;&quot;, cpu_time)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="28375" LastEditorUserId="9667" LastEditDate="2019-03-25T21:23:07.177" LastActivityDate="2019-03-25T21:23:07.177" Title="Paring Large Numbers Question" Tags="|python|" AnswerCount="0" CommentCount="1" ClosedDate="2019-05-13T23:37:08.023" ContentLicense="CC BY-SA 4.0" />
  <row Id="31292" PostTypeId="2" ParentId="31198" CreationDate="2019-03-23T19:03:51.883" Score="5" Body="&lt;p&gt;OK, you have a very nice problem, I tried to run some benchmarks. &#xA;First, I don't have your parameters so I used your small example. &#xA;Second, since you do not specify the language, I used C + GSL (since I'm not that familiar with C++)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#if __STDC_VERSION__ &amp;gt;= 199901L&#xA;#define _XOPEN_SOURCE 700&#xA;#else&#xA;#define _XOPEN_SOURCE 600&#xA;#endif /* __STDC_VERSION__ */&#xA;#include &amp;lt;time.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_vector.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_rng.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_randist.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_complex.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_complex_math.h&amp;gt;&#xA;#include &amp;lt;gsl/gsl_blas.h&amp;gt;&#xA;&#xA;gsl_vector_complex *A;&#xA;gsl_vector_complex *B;&#xA;gsl_vector_complex *C;&#xA;&#xA;void evaluate(gsl_vector_complex *in, gsl_vector_complex *out)&#xA;{&#xA;    gsl_complex n_unity;&#xA;    gsl_complex unity;&#xA;    gsl_complex zero;&#xA;&#xA;    GSL_SET_COMPLEX (&amp;amp;n_unity, -1, 0);&#xA;    GSL_SET_COMPLEX (&amp;amp;unity, 1, 0);&#xA;    GSL_SET_COMPLEX (&amp;amp;zero, 0, 0);&#xA;&#xA;    const gsl_complex neg_x8 = gsl_complex_mul (n_unity, gsl_vector_complex_get (in, 8));&#xA;&#xA;    gsl_vector_complex_set (A, 0, neg_x8);&#xA;    gsl_vector_complex_set (A, 1, neg_x8);&#xA;    gsl_vector_complex_set (A, 2, neg_x8);&#xA;    gsl_vector_complex_set (A, 3, gsl_vector_complex_get (in, 8));&#xA;    gsl_vector_complex_set (A, 4, gsl_vector_complex_get (out, 4));&#xA;    gsl_vector_complex_set (A, 5, neg_x8);&#xA;    gsl_vector_complex_set (A, 6, gsl_vector_complex_get (in, 8));&#xA;    gsl_vector_complex_set (A, 7, gsl_vector_complex_get (out, 7));&#xA;    gsl_vector_complex_set (A, 8, gsl_vector_complex_get (in, 4));&#xA;    gsl_vector_complex_set (A, 9, gsl_vector_complex_get (in, 5));&#xA;    gsl_vector_complex_set (A, 10, gsl_vector_complex_get (out, 10));&#xA;    gsl_vector_complex_set (A, 11, gsl_vector_complex_get (in, 6));&#xA;    gsl_vector_complex_set (A, 12, gsl_vector_complex_get (in, 6));&#xA;&#xA;    gsl_vector_complex_set (B, 0, unity);&#xA;    gsl_vector_complex_set (B, 1, unity);&#xA;    gsl_vector_complex_set (B, 2, gsl_vector_complex_get (in, 9));&#xA;    gsl_vector_complex_set (B, 3, gsl_vector_complex_get (in, 6));&#xA;    gsl_vector_complex_set (B, 4, unity);&#xA;    gsl_vector_complex_set (B, 5, gsl_vector_complex_get (in, 0));&#xA;    gsl_vector_complex_set (B, 6, gsl_vector_complex_get (in, 7));&#xA;    gsl_vector_complex_set (B, 7, unity);&#xA;    gsl_vector_complex_set (B, 8, gsl_vector_complex_get (in, 4));&#xA;    gsl_vector_complex_set (B, 9, gsl_vector_complex_get (in, 5));&#xA;    gsl_vector_complex_set (B, 10, unity);&#xA;    gsl_vector_complex_set (B, 11, gsl_vector_complex_get (in, 6));&#xA;    gsl_vector_complex_set (B, 12, gsl_vector_complex_get (in, 6));&#xA;&#xA;    gsl_vector_complex_set_basis (C, 1);&#xA;    gsl_vector_complex_set (C, 2, gsl_vector_complex_get (in, 9));&#xA;    gsl_vector_complex_set (C, 5, gsl_vector_complex_get (in, 0));&#xA;    gsl_vector_complex_set (C, 12, gsl_vector_complex_get (in, 6));&#xA;&#xA;    gsl_vector_complex_mul (A, B);&#xA;    gsl_blas_zaxpy (unity, C, A);&#xA;    gsl_vector_complex_memcpy (out, A);&#xA;&#xA;}&#xA;&#xA;void evaluate_naive(gsl_vector_complex *in, gsl_vector_complex *out)&#xA;{&#xA;    gsl_complex n_unity;&#xA;    gsl_complex unity;&#xA;    gsl_complex zero;&#xA;&#xA;    GSL_SET_COMPLEX (&amp;amp;n_unity, -1, 0);&#xA;    GSL_SET_COMPLEX (&amp;amp;unity, 1, 0);&#xA;    GSL_SET_COMPLEX (&amp;amp;zero, 0, 0);&#xA;&#xA;    const gsl_complex neg_x8 = gsl_complex_mul (n_unity, gsl_vector_complex_get (in, 8));&#xA;&#xA;    gsl_vector_complex_set (out, 0, neg_x8);&#xA;    gsl_vector_complex_set (out, 1, gsl_complex_add (unity, neg_x8));&#xA;    gsl_vector_complex_set (out, 2, gsl_complex_sub (gsl_vector_complex_get (in, 9), gsl_complex_mul (neg_x8, gsl_vector_complex_get (in, 9))));&#xA;    gsl_vector_complex_set (out, 3, gsl_complex_mul (gsl_vector_complex_get (in, 8), gsl_vector_complex_get (in, 6)));&#xA;    gsl_vector_complex_set (out, 5, gsl_complex_sub (gsl_vector_complex_get (in, 0), gsl_complex_mul (neg_x8, gsl_vector_complex_get (in, 0))));&#xA;    gsl_vector_complex_set (out, 6, gsl_complex_mul (gsl_vector_complex_get (in, 8), gsl_vector_complex_get (in, 7)));&#xA;    gsl_vector_complex_set (out, 8, gsl_complex_mul (gsl_vector_complex_get (in, 4), gsl_vector_complex_get (in, 4)));&#xA;    gsl_vector_complex_set (out, 9, gsl_complex_mul (gsl_vector_complex_get (in, 5), gsl_vector_complex_get (in, 5)));&#xA;    gsl_vector_complex_set (out, 11, gsl_complex_mul (gsl_vector_complex_get (in, 6), gsl_vector_complex_get (in, 6)));&#xA;    gsl_vector_complex_set (out, 12, gsl_complex_add (gsl_vector_complex_get (out, 11), gsl_vector_complex_get (in, 6)));&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    struct timespec start, end;&#xA;    size_t time_spent;&#xA;&#xA;    gsl_vector_complex *x;&#xA;    gsl_vector_complex *y;&#xA;&#xA;    double x_,y_;&#xA;    gsl_complex val;&#xA;&#xA;    const gsl_rng_type * T;&#xA;    gsl_rng * r;&#xA;&#xA;    gsl_rng_env_setup();&#xA;&#xA;    T = gsl_rng_default;&#xA;    r = gsl_rng_alloc (T);&#xA;&#xA;    x = gsl_vector_complex_calloc (13);&#xA;    y = gsl_vector_complex_calloc (13);&#xA;    A = gsl_vector_complex_calloc (13);&#xA;    B = gsl_vector_complex_calloc (13);&#xA;    C = gsl_vector_complex_calloc (13);&#xA;&#xA;&#xA;    for (size_t iteration = 0; iteration &amp;lt; 100000000; iteration++)&#xA;    {&#xA;        for (size_t index = 0; index &amp;lt; 13; index++)&#xA;        {&#xA;            x_ = gsl_ran_gaussian_ziggurat (r, 1.0);&#xA;            y_ = gsl_ran_gaussian_ziggurat (r, 1.0);&#xA;            GSL_SET_COMPLEX(&amp;amp;val, x_, y_);&#xA;&#xA;            gsl_vector_complex_set (x, index, val);&#xA;        }&#xA;&#xA;        for (size_t index = 0; index &amp;lt; 13; index++)&#xA;        {&#xA;            x_ = gsl_ran_gaussian_ziggurat (r, 1.0);&#xA;            y_ = gsl_ran_gaussian_ziggurat (r, 1.0);&#xA;            GSL_SET_COMPLEX(&amp;amp;val, x_, y_);&#xA;&#xA;            gsl_vector_complex_set (y, index, val);&#xA;        }&#xA;&#xA;&#xA;        clock_gettime (CLOCK_MONOTONIC, &amp;amp;start);&#xA;#ifdef NAIVE&#xA;        evaluate_naive (x,y);&#xA;#else&#xA;        evaluate (x, y);&#xA;#endif&#xA;        clock_gettime (CLOCK_MONOTONIC, &amp;amp;end);&#xA;        time_spent = (end.tv_sec - start.tv_sec) * 1000000000 + (end.tv_nsec - start.tv_nsec);&#xA;        printf(&quot;%e\n&quot;, 1.0 * time_spent);&#xA;&#xA;    }&#xA;&#xA;    gsl_rng_free (r);&#xA;    gsl_vector_complex_free(x);&#xA;    gsl_vector_complex_free(y);&#xA;    gsl_vector_complex_free(A);&#xA;    gsl_vector_complex_free(B);&#xA;    gsl_vector_complex_free(C);&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried both vectorized and naive implementations and here are my results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gcc, no optimizations, naive version, mean = 270.6; std = 190.6 ns&#xA;gcc, no optimizations, vector version, mean = 318.1; std = 381.1 ns&#xA;gcc, optimized, naive version, mean = 106.9; std = 160.3 ns&#xA;gcc, optimized, vector version, mean = 154.5; std = 109.6 ns&#xA;gcc, optimized, machine-specific, naive version, mean = 105.2; std = 96.2 ns&#xA;gcc, optimized, machine-specific, vector version, mean = 154.8; std = 144.8 ns&#xA;&#xA;icc, optimized, naive version, mean = 75.6; std = 99.4 ns&#xA;icc, optimized, vector version, mean = 164.6; std = 209.9 ns&#xA;icc, optimized, machine-specific, naive version, mean = 75.8; std = 189.5 ns&#xA;icc, optimized, machine-specific, vector version, mean = 166.9; std = 1850.2 ns // this std is somehow bad, however, I used my laptop to run the tests&#xA;icc, no optimizations, naive version, mean = 130.9; std = 153.6 ns&#xA;icc, no optimizations, vector version, mean = 266.6; std = 238.8 ns&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Q.E.D. &#xA;If you're not sure what to do, let the compiler (icc) do its dark magic. &#xA;This case can be platform-specific, moreover, timings can change when you increase the number/size of arrays. &#xA;P.S. &#xA;There is one more possible solution to try: Intel provides its own functions for element-wise vector operations, that can improve the results, however, you have to rewrite the code. &#xA;P.P.S. Actually, I completely forgot that icc by default doesn't follow IEEE standards (equivalent of -ffast-math) and can give you extremely incorrect results extremely quickly. Be aware. &lt;/p&gt;&#xA;" OwnerUserId="28575" LastEditorUserId="28575" LastEditDate="2019-11-14T19:26:04.080" LastActivityDate="2019-11-14T19:26:04.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31293" PostTypeId="1" CreationDate="2019-03-24T03:16:04.360" Score="1" ViewCount="127" Body="&lt;p&gt;I am using ARPACK's zndrv1 to diagonalise a matrix (the context is quantum chemistry). While all vectors have a norm 1, as expected, vectors corresponding to degenerate eigenstates aren't always orthogonal (often their inner product will be roughly 0.01).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any parameters I can adjust to improve orthonormality (even if it means longer compute time)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand there are alternative libraries and routines out there. I am specifically curious about ARPACK.&lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2019-03-24T03:16:04.360" Title="How can I maximise orthonormality between degenerate eigenvectors using ARPACK?" Tags="|eigensystem|arpack|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31294" PostTypeId="1" CreationDate="2019-03-24T07:09:00.837" Score="1" ViewCount="105" Body="&lt;p&gt;The equation is &lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial}{\partial x}\left(u\frac{\partial u}{\partial x}\right)=f(x)\\ 0&amp;lt;x&amp;lt;1, u(0)=u(1)=0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm discretizing this PDE using FVM as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$0=x_0=x_{1/2}&amp;lt;x_1&amp;lt;x_{3/2}&amp;lt;\dots&amp;lt;x_{N+1/2}=x_{N+1}=1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h_i:=x_{i+1/2}-x_{i-1/2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$h_{i+1/2}:=x_{i+1}-x_i$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int\limits_{x_{i-1/2}}^{x_{i+1/2}} \frac{\partial}{\partial x}\left(u\frac{\partial u}{\partial x}\right)dx=(uu_x)(x_{i+1/2})-(uu_x)(x_{i-1/2})=\int\limits_{x_{i-1/2}}^{x_{i+1/2}}f(x)dx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using finite differences, obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u_{i+1/2}\frac{u_{i+1}-u_i}{h_{i+1/2}}-u_{i-1/2}\frac{u_{i}-u_{i-1}}{h_{i-1/2}}=h_if_i$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$u_0=u_{N+1}=0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f_i:=\frac1h_i\int\limits_{x_{i-1/2}}^{x_{i+1/2}}f(x)dx$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now comes the part where I got really stuck. I want to determine the truncation error of this discretization. For this I substitute the Taylor expansion of the exact solution into the equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r_i:=\frac1h_i\left(u(x_{i+1/2})\frac{u(x_{i+1})-u(x_i)}{h_{i+1/2}}-u(x_{i-1/2})\frac{u(x_{i})-u(x_{i-1})}{h_{i-1/2}}\right)-f_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$=\frac{1}{h_i}\left(u(x_{i+1/2})\left(u_x(x_i)+\frac{h_{i+1/2}}{2}u_{xx}(x_i)+O(h^2_{i+1/2})\right)-u(x_{i-1/2})\left(u_x(x_i)+\frac{h_{i-1/2}}{2}u_{xx}(x_i)+O(h^2_{i-1/2})\right)\right)-f_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what to do with the terms &lt;span class=&quot;math-container&quot;&gt;$u(x_{i+1/2})$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u(x_{i-1/2})$&lt;/span&gt; to arrive at the truncation error, which I think will be &lt;span class=&quot;math-container&quot;&gt;$O(h^2)$&lt;/span&gt;. I'd appreciate some help with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already tried to expand these terms about &lt;span class=&quot;math-container&quot;&gt;$x_{i-1/2}$&lt;/span&gt; but that wasn't quite useful.&lt;/p&gt;&#xA;" OwnerUserId="29427" LastActivityDate="2019-03-24T07:09:00.837" Title="Determine truncation error of PDE discretization" Tags="|pde|finite-volume|discretization|error-estimation|elliptic-pde|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31297" PostTypeId="1" AcceptedAnswerId="31326" CreationDate="2019-03-24T16:31:10.747" Score="0" ViewCount="639" Body="&lt;p&gt;Consider a Boundary Value Problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\delta u''+u(u'-1) =0 \Leftrightarrow u''=\frac{-u(u'-1)}{\delta}=:f(t,u',u), \\&#xA;u(0)=a, u(1)=b&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\delta,a,b$&lt;/span&gt; are known parameters. I want to implement Newton's method on a uniform grid. &lt;strong&gt;How do I derive the Jacobian?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="30281" LastEditorUserId="20688" LastEditDate="2019-03-24T17:12:10.810" LastActivityDate="2019-03-28T18:07:01.490" Title="Non-linear Boundary Value Problem. How to compute the Jacobian?" Tags="|finite-difference|ode|boundary-conditions|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31298" PostTypeId="1" CreationDate="2019-03-24T21:07:07.153" Score="1" ViewCount="493" Body="&lt;p&gt;I developed the code below for performing gaussian elimination in order to evaluate the determinant of a matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;program det3&#xA;&#xA;&#xA;REAL, DIMENSION(3,3) :: matriz = RESHAPE([1,2,3,-7,13,5,4,-21,8],[3,3])&#xA;REAL :: temp, m&#xA;INTEGER :: n = 3&#xA;INTEGER :: i, j, k&#xA;INTEGER :: fator_cor = 1&#xA;REAL :: det&#xA;&#xA;DO k=1, n-1&#xA;&#xA;    DO i = k+1, n&#xA;&#xA;                IF (matriz(k,k) == 0) THEN&#xA;                    linha_aux = matriz(k,0:n)&#xA;                    matriz(k,0:n)= matriz(k+1, 0:n)&#xA;                    matriz(k+1,0:n)= linha_aux&#xA;                    fator_cor = fator_cor*(-1)&#xA;&#xA;&#xA;&#xA;                END IF&#xA;&#xA;    END DO&#xA;&#xA;&#xA;END DO&#xA;&#xA;&#xA;    DO j = k+1, n&#xA;            m = matriz(j,k)/matriz(k,k)&#xA;&#xA;            DO i = k+1, n&#xA;                matriz(j,i) = matriz(j,i) - m*matriz(k,i)&#xA;&#xA;&#xA;&#xA;            END DO&#xA;&#xA;&#xA;&#xA;    END DO&#xA;&#xA;&#xA;&#xA;&#xA;det = l&#xA;    DO i = 1, n&#xA;        det = det * matriz(i,i)&#xA;        det = det*fator_cor&#xA;    END DO&#xA;&#xA;WRITE (*,*) det&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;end program&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm trying to apply it to the 3x3 matrix declared in the beginning but I'm getting the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Error: Incompatible ranks 0 and 1 in assignment at (1)&quot;&#xA;  linha_aux = matriz(k,0:n)&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm a bit new to fortran so I'm not sure what it is refering to. I'm unsure if I should declare linha_aux at all, and if so if I should declare it as an allocatable array. Any help is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="30236" LastEditorUserId="30236" LastEditDate="2019-03-25T01:30:22.687" LastActivityDate="2019-03-25T08:12:51.837" Title="Gaussian Elimination Using Fortran" Tags="|linear-algebra|fortran|" AnswerCount="1" CommentCount="1" ClosedDate="2019-03-25T21:14:48.680" ContentLicense="CC BY-SA 4.0" />
  <row Id="31299" PostTypeId="1" AcceptedAnswerId="31302" CreationDate="2019-03-25T03:32:48.627" Score="0" ViewCount="80" Body="&lt;p&gt;I am recently starting with OpenFoam. I have a Mac as my personal laptop, but I would have to use OpenFoam on linux in my lab. So my questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is the OpenFoam software independent of OS, so that I can use interchangeably on both Mac and linux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2)I am fairly good with C programming(I am not really comfortable with the OOPS concept). So is that sufficient or would I have to get better at it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3)I have taken a fluid mechanics course and a basic numerical course that deals with solving matrices. So is that sufficient or would I have to take a basic CFD course online?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;&#xA;I do know the choice of installing VM or BootCamp. But I want to keep that option as last resort. &lt;/p&gt;&#xA;" OwnerUserId="30285" LastActivityDate="2019-03-25T17:28:16.480" Title="Is open foam Mac version compatible with Linux version" Tags="|openfoam|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31300" PostTypeId="1" CreationDate="2019-03-25T04:01:44.967" Score="1" ViewCount="65" Body="&lt;p&gt;I want to discretize the following parabolic PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u_t = \nabla\cdot(\alpha(x)\nabla u)- \beta u\\&#xA;x\in\Omega \subset \mathbb{R}^2\\&#xA;\partial_n u = 0\\&#xA;u(t,0) = u_0(x)\ge 0, \alpha(x)&amp;gt;0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the Neumann boundary condition above, one can integrate this PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{alignat}{2}&#xA;\frac{\mathrm d}{\mathrm dt}\int_\Omega \! u\,\mathrm dx&amp;amp;=\phantom{-}\int_\Omega \! \nabla\cdot (a(x)\nabla u)\,\mathrm dx \:\:&amp;amp;-\int_\Omega \! \beta(x) u \,\mathrm dx \\&#xA;&amp;amp;=\phantom{-}\int_{\partial\Omega}\! a(x)\nabla u\cdot n \,\mathrm ds&amp;amp;-\int_\Omega \! \beta(x) u \, \mathrm dx \\&#xA;&amp;amp;=\phantom{-}\int_{\partial\Omega} a(x)\partial_n u \,\mathrm ds&amp;amp;-\int_\Omega \! \beta(x) u \, \mathrm dx \\&#xA;&amp;amp;=-\int_\Omega \! \beta(x) u \, \mathrm dx&#xA;\end{alignat}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The form of the discretized linear system I want to get is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d\vec{u}}{dt}=A\vec{u}-\vec{c}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I don't think this is exactly what I'm obtaining, despite I do think I use a correct discretization technique. To this end, please consider my steps below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I discretize the two integrals using the finite volume method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d}{dt}\sum\limits_{j\in n_i} u_j l_{ij}=-\sum\limits_{j\in n_i} \beta_{ij} u_j l_{ij},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; is the set of neighboring cells to the cell &lt;span class=&quot;math-container&quot;&gt;$V_j\subset \Omega$&lt;/span&gt; and where &lt;span class=&quot;math-container&quot;&gt;$l_{ij}$&lt;/span&gt; is the length of the boundary of &lt;span class=&quot;math-container&quot;&gt;$V_j$&lt;/span&gt; between &lt;span class=&quot;math-container&quot;&gt;$u_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u_j$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now one can discretize it in time to obtain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sum\limits_{j\in n_i} \frac{u_j^{n+1}-u_j^n}{\Delta t}l_{ij}=-\sum\limits_{j\in n_i} \beta_{ij} u_j^n l_{ij}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we obtain the linear system&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L\frac{d\vec{u}}{dt} = -A\vec{u},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the matrix containing &lt;span class=&quot;math-container&quot;&gt;$l_{ij}$&lt;/span&gt; and each entry of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; contains &lt;span class=&quot;math-container&quot;&gt;$-\beta_{ij}l_{ij}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I correct? I think I'm quite confused as I don't see where each term of the matrix should go, and where the vector &lt;span class=&quot;math-container&quot;&gt;$\vec{c}$&lt;/span&gt; is supposed to be. Moreover, what does one do with the &lt;span class=&quot;math-container&quot;&gt;$l_{ij}$&lt;/span&gt; on the LHS?&lt;/p&gt;&#xA;" OwnerUserId="29427" LastEditorUserId="29427" LastEditDate="2019-03-25T10:33:29.770" LastActivityDate="2019-03-25T10:33:29.770" Title="Discretizing a parabolic PDE with finite volume method" Tags="|pde|finite-volume|discretization|parabolic-pde|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31301" PostTypeId="2" ParentId="31298" CreationDate="2019-03-25T08:12:51.837" Score="0" Body="&lt;p&gt;Some (bit to to long for a comment) initial remarks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You try to assign an array &lt;code&gt;matriz(k,0:n)&lt;/code&gt; to a scalar integer variable &lt;code&gt;linha_aux&lt;/code&gt;. &lt;code&gt;linha_aux&lt;/code&gt; is a scalar integer variable as it is implicitly declared.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;declare all variables&lt;/li&gt;&#xA;&lt;li&gt;use IMPLICIT NONE so you have to declare all variables&lt;/li&gt;&#xA;&lt;li&gt;Fortran starts counting elements at 1, so &lt;code&gt;matriz(k,0:n)&lt;/code&gt; has a&#xA;problem as &lt;code&gt;n&lt;/code&gt; is declared as 3 and the matrix has size 3 times 3.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the nested loops:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DO k=1, n-1&#xA;&#xA;   DO i = k+1, n &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as the content of the loop does not use &lt;code&gt;i&lt;/code&gt; and also in &lt;code&gt;matriz(k+1,0:n)&lt;/code&gt;the matrix would go out of bounds on the first index (and and the second due to the 0).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;note that Fortran has column major order (so the first index runs&#xA;fasts), this ordering is other than e.g. C that has row major&#xA;ordering.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="29137" LastActivityDate="2019-03-25T08:12:51.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31302" PostTypeId="2" ParentId="31299" CreationDate="2019-03-25T17:28:16.480" Score="0" Body="&lt;p&gt;You can install OpenFOAM on a variety on operating systems, see e.g. &lt;a href=&quot;https://openfoam.org/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you set up a simulation on your private computer, running macOS, and copy it to your lab computer, running Linux, there should be no problem, as OpenFOAM cases are largely based on ASCII text files. Only the big data files, such as fields and certain files of the mesh can be written in a binary format to save disk space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When working on different machines, remove any run-time compiled files from the case. They are likely to cause trouble when they were compiled on machine A and you try to run them on machine B.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Yes, the cases are independent of the OS.&lt;/li&gt;&#xA;&lt;li&gt;Depending on what you want to do, you should get familiar with C++ and the OOP concepts as well as templates. OpenFOAM makes heavy use of these, and knowledge about them is vital if you want to understand OpenFOAM's source code&lt;/li&gt;&#xA;&lt;li&gt;Having attended a CFD course, when you want to do CFD is never a bad thing.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3510" LastActivityDate="2019-03-25T17:28:16.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31304" PostTypeId="2" ParentId="31289" CreationDate="2019-03-25T21:06:16.667" Score="3" Body="&lt;p&gt;I do not believe that a direct solve is possible in your situation, there are simply &lt;em&gt;too&lt;/em&gt; many applications with &lt;em&gt;too&lt;/em&gt; many users which would benefit &lt;em&gt;too&lt;/em&gt; much if it was possible. In particular, the problem of computing, say, the transfer function &lt;span class=&quot;math-container&quot;&gt;$T(s) = C(A - sI)^{-1}B+D$&lt;/span&gt; of a linear time invariant system &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \begin{align} x'(t) &amp;amp;= Ax(t) + Bu(t) \\ y (t) &amp;amp;= Cx(t) + Du(t) \end{align}$$&lt;/span&gt;&#xA;or solving the Lyapunov matrix equation &#xA;&lt;span class=&quot;math-container&quot;&gt;$$AX + XA^T = BB^T$$&lt;/span&gt; &#xA;using the alternating direction implicit (ADI) method would be greatly facilitated by any algorithm which would allows to pass cheaply from a factorization of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; to a factorization of &lt;span class=&quot;math-container&quot;&gt;$A - sI$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the off chance that &lt;span class=&quot;math-container&quot;&gt;$D(t)$&lt;/span&gt; has low rank, then the Sherman-Morrison-Woodbury formula would allow you to quickly solve systems with coefficient matrix &lt;span class=&quot;math-container&quot;&gt;$A + D(t)$&lt;/span&gt; provided you can quickly solve system with coefficient matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. However, I gather that this is not likely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest that you view the solution &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; of your linear system &lt;span class=&quot;math-container&quot;&gt;$$(A + D(t)) x = b(t)$$&lt;/span&gt; as a function of &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;. It is likely that &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; can be differentiate a few times with respect to &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; which suggests that future values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; can be extrapolated from past values. In particular, if &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is small enough, then &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ x_0 = x(t), \quad \text{or}\quad x_0 = 2x(t) - x(t-h)$$&lt;/span&gt;&#xA;represent simple and easy to implement initial guesses for the value of &lt;span class=&quot;math-container&quot;&gt;$x(t+h)$&lt;/span&gt;. More sophisticated approximation are possible, but they are not necessarily worth the effort.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest further that your solve your linear system using a preconditioned Krylov subspace method. Instead of computing a new preconditioner for every timestep, I suggest that you recycle the preconditioner a few times and compute a new preconditioner only when the current one appears ineffective, i.e., the iteration count becomes too large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot promise that these options will give you the speed you are looking for, but at least they provide some parameters which can be tuned: the choice of the initial guess, the preconditioner and the number of time-steps before you compute a new preconditioner.&lt;/p&gt;&#xA;" OwnerUserId="19136" LastActivityDate="2019-03-25T21:06:16.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31306" PostTypeId="1" AcceptedAnswerId="31308" CreationDate="2019-03-26T03:22:01.147" Score="0" ViewCount="3535" Body="&lt;p&gt;I am writing some functions to interpolate data. While using interp2D, somehow, a sample matrix works but when I change the size of the matrix, it returns an error.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/python3&#xA;&#xA;# &#xA;# Useful interpolation functions in python.&#xA;# &#xA;&#xA;import numpy as np&#xA;import scipy.interpolate&#xA;&#xA;def interp_2d(x,y,fxy,x_desired,y_desired):&#xA;&#xA;    interp_func = scipy.interpolate.interp2d(x, y, fxy, kind='cubic')&#xA;&#xA;    return interp_func(x_desired,y_desired)&#xA;&#xA;#&#xA;# Main function.&#xA;#&#xA;&#xA;def main():&#xA;&#xA;    x   =  np.array([0.0,1.0,2.0,3.0])&#xA;    y   =  np.array([0.0,1.0,2.0,3.0])&#xA;    fxy =  np.array([[50.0,60.0,70.0,80.0],&#xA;                     [50.0,60.0,70.0,80.0],&#xA;                     [50.0,60.0,70.0,80.0],&#xA;                     [50.0,60.0,70.0,80.0]])&#xA;&#xA;    # With the current definition of x,y and fxy works.&#xA;&#xA;    x_desired = 1.0&#xA;    y_desired = 1.0&#xA;&#xA;    print(interp_2d(x,y,fxy,x_desired,y_desired)[0]) # returns 60.0, ok !&#xA;&#xA;    # With this definition, it crashes.&#xA;&#xA;    x   =  np.array([0.0,1.0])&#xA;    y   =  np.array([0.0,1.0])&#xA;    fxy =  np.array([[50.0,60.0],[50.0,60.0]])&#xA;&#xA;    x_desired = 1.0&#xA;    y_desired = 1.0&#xA;&#xA;    print(interp_2d(x,y,fxy,x_desired,y_desired)[0]) # Crashes, should be 60 as well.&#xA;&#xA;&#xA;if __name__== &quot;__main__&quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is still a squared matrix, however, it do not handles the problem properly.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The error it returns is:&#xA;Traceback (most recent call last):&#xA;  File &quot;./interp_ask.py&quot;, line 49, in &amp;lt;module&amp;gt;&#xA;    main()&#xA;  File &quot;./interp_ask.py&quot;, line 45, in main&#xA;    print(interp_2d(x,y,fxy,x_desired,y_desired)[0]) # Crashes&#xA;  File &quot;./interp_ask.py&quot;, line 12, in interp_2d&#xA;    interp_func = scipy.interpolate.interp2d(x, y, fxy, kind='cubic')&#xA;  File &quot;/usr/lib/python3.7/site-packages/scipy/interpolate/interpolate.py&quot;, line 249, in __init__&#xA;    kx=kx, ky=ky, s=0.0)&#xA;dfitpack.error: (mx&amp;gt;kx) failed for hidden mx: regrid_smth:mx=2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you all !&lt;/p&gt;&#xA;" OwnerUserId="21588" LastActivityDate="2019-03-26T10:43:52.243" Title="Problems with python's interp 2D" Tags="|python|interpolation|scipy|numpy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31307" PostTypeId="2" ParentId="41" CreationDate="2019-03-26T05:55:30.863" Score="1" Body="&lt;p&gt;Today I come across a thesis &lt;a href=&quot;https://tigerprints.clemson.edu/all_dissertations/2243/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The EMAC Scheme for Navier-Stokes Simulations, and Application to Flow Past Bluff Bodies&lt;/em&gt;&lt;/a&gt; and notice Section 1.2 of it answers OP's question, at least partially. The relevant parts are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is widely believed in the computational fluid dynamics (&lt;strong&gt;CFD&lt;/strong&gt;)&#xA;  community that the more physics is built into the discretization, the&#xA;  more accurate and stable the discrete solutions are, especially over&#xA;  longer time intervals. N. Phillips in 1959&#xA;  &lt;a href=&quot;http://twister.caps.ou.edu/CFD2007/Phillips_NLInstablity.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;[42]&lt;/a&gt;&#xA;  constructed an example for the barotropic nonlinear vorticity equation&#xA;  (using a finite-difference scheme), where the long-time integration of&#xA;  the convection terms results in a failure of numerical simulations for&#xA;  any time step. In&#xA;  &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/0021999166900155&quot; rel=&quot;nofollow noreferrer&quot;&gt;[4]&lt;/a&gt;&#xA;  Arakawa showed that one can avoid instability issues with integration&#xA;  over long time if kinetic energy and enstrophy (in 2D) are conserved&#xA;  by a discretization scheme. … . In 2004, Liu and Wang developed that&#xA;  conserves helicity and energy for three-dimensional flows. In&#xA;  &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0021999104001287&quot; rel=&quot;nofollow noreferrer&quot;&gt;[35]&lt;/a&gt;,&#xA;  they present an energy and helicity-preserving scheme for axisymmetric&#xA;  flows. They also show that their dual conservation scheme eliminates&#xA;  the need for large nonphysical numerical viscosity. …&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;… It has been known for decades in CFD, that the more physical&#xA;  quantities are conserved by a finite element scheme, the more accurate&#xA;  the prediction, especially over the long time intervals. Thus the&#xA;  solutions provided by a more physically accurate scheme are also more&#xA;  physically relevant. If one could afford a fully resolved mesh and&#xA;  infinitely small time step, all commonly used finite element schemes&#xA;  are believed to provide the same numerical solutions. However, in&#xA;  practice one cannot afford a fully resolved mesh in 3D-simulations,&#xA;  especially for time-dependent problems. For example in chapter 2 we&#xA;  need 50-60 thousand time steps, where each time step requires solving&#xA;  a sparse linear system with 4 million unknowns. This required 2-3&#xA;  weeks of computational time with highly parallel code on 5 nodes with&#xA;  24 cores each.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5331" LastEditorUserId="5331" LastEditDate="2019-04-30T05:42:29.040" LastActivityDate="2019-04-30T05:42:29.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31308" PostTypeId="2" ParentId="31306" CreationDate="2019-03-26T07:34:35.217" Score="2" Body="&lt;p&gt;According to the &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp2d.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;, the minimum number of data points in the matrix should be &lt;span class=&quot;math-container&quot;&gt;$(k+1)^2$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$k=3$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k=5$&lt;/span&gt; for linear, cubic and quintic interpolations, respectively. Indeed, changing &lt;code&gt;kind='cubic'&lt;/code&gt; to &lt;code&gt;kind='linear'&lt;/code&gt; in your code gets rid of the error. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You probably want to check the size of the matrix at the beginning of the &lt;code&gt;interp_2d&lt;/code&gt; function to either change the interpolation type and/or return an error if the matrix size is too small.&lt;/p&gt;&#xA;" OwnerUserId="27040" LastEditorUserId="27040" LastEditDate="2019-03-26T10:43:52.243" LastActivityDate="2019-03-26T10:43:52.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31311" PostTypeId="2" ParentId="30138" CreationDate="2019-03-26T09:44:23.453" Score="2" Body="&lt;p&gt;I would say one significant improvement is that in the scope of time-stepping approaches, where you propagate &lt;span class=&quot;math-container&quot;&gt;$y_n \rightarrow y_{n+1}=\mathcal{U}(y_n)$&lt;/span&gt; using a solution map &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}$&lt;/span&gt;, you can determine the propagator (or at least parts of it) once and then re-use it at every time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in the linear case you would have &lt;span class=&quot;math-container&quot;&gt;$\partial_t y = A y$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is a matrix. The solution operator &lt;span class=&quot;math-container&quot;&gt;$\mathcal{U}(y) = \exp(A \Delta t)y$&lt;/span&gt; consists mainly of a matrix exponential. For autonomous systems, this costly matrix exponential evaluation is required only once for the complete propagation -- in contrast to a time-dependent system, where you have to perform this evaluation at every time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For nonlinear systems it is not that easy, but depending on the algorithm certain costly evaluations can be re-used.&lt;/p&gt;&#xA;" OwnerUserId="30021" LastActivityDate="2019-03-26T09:44:23.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31312" PostTypeId="1" CreationDate="2019-03-26T10:20:38.260" Score="1" ViewCount="20" Body="&lt;p&gt;I am putting into MATLAB code the equations that describe a plastic extrusion process. From a paper, I found I should use a spatial grid that translates with the extrusion speed, being the reference coordinate system the centerline of the extrudate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, &lt;span class=&quot;math-container&quot;&gt;$s = 0$&lt;/span&gt; is the extruder die and the length of the domain is time dependent as the extrudate gets longer and longer. In the paper, new grid points are continually added to the solution domain at &lt;span class=&quot;math-container&quot;&gt;$s = 0$&lt;/span&gt; (the extruder die) to account for the extrusion of fresh fluid. How would I do this?&lt;/p&gt;&#xA;" OwnerUserId="28900" LastEditorUserId="20688" LastEditDate="2019-03-26T10:50:32.947" LastActivityDate="2019-03-26T10:50:32.947" Title="Translating grid with extrusion speed" Tags="|matlab|ode|grid|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31313" PostTypeId="1" CreationDate="2019-03-26T11:11:58.970" Score="0" ViewCount="196" Body="&lt;p&gt;Are there any circumstances under which using a value &lt;span class=&quot;math-container&quot;&gt;$w &amp;lt; 0$&lt;/span&gt; would help us find a solution in over-relaxation faster than we can with the ordinary relaxation method?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over Relaxation Method:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$x'= [1 + w]f(x) - wx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Calculating &lt;span class=&quot;math-container&quot;&gt;$x = 1-e^{-3x}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take x = 1 as initial value, and w as 0.2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x' = (1+0.2)f(1)-0.2(1) = 0.94025551795&#xA;&#xA;x' = (1+0.2)f(0.94025551795)-0.2(0.94025551795) = 0.94047657354&#xA;&#xA;x' = (1+0.2)f(0.94047657354)-0.2(0.94047657354) = 0.94047974478&#xA;&#xA;x' = (1+0.2)f(0.94047974478)=0.2(0.94047974478) = 0.94047979005&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We stop until the value get to a certain accuracy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why would the over-relaxation reach the solution faster if we consider &lt;span class=&quot;math-container&quot;&gt;$w &amp;lt; 0$&lt;/span&gt; in non-linear function such as &lt;span class=&quot;math-container&quot;&gt;$x = 1 - e^{(1 - x^2)}$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="30296" LastEditorUserId="9667" LastEditDate="2019-04-01T15:18:58.763" LastActivityDate="2019-04-01T20:59:47.423" Title="Overrelaxation with w &lt; 0" Tags="|computational-physics|iterative-method|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31314" PostTypeId="1" AcceptedAnswerId="31316" CreationDate="2019-03-26T11:39:30.347" Score="4" ViewCount="847" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4ilkM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4ilkM.png&quot; alt=&quot;The kind of flow plot I&amp;#39;m trying to get&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially I want to use paraview to recreate a flow visualization like the one shown in the picture above. I am able to create the 3d flow lines using a pipeline that looks like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data (csv)-&amp;gt; TableToStructuredGrid -&amp;gt; Calculator-&amp;gt; ExtractSubset -&amp;gt; StreamTracerwithCustomSource&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where I have used the subset as my seed. However I have no idea how to project the 3d flow onto the walls as shown in the picture. There is an option in streamlines called &quot;surface streamlines&quot; but this restricts streamlines to live in a plane whereas I want to project my 3d streamlines down into a plane and display them on the back 3 faces of the bounding cube.&lt;/p&gt;&#xA;" OwnerUserId="30297" LastEditorUserId="9667" LastEditDate="2019-03-26T12:15:11.360" LastActivityDate="2019-03-26T16:43:29.427" Title="Visualization of 3D streamlines in ParaView" Tags="|visualization|paraview|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31315" PostTypeId="2" ParentId="31313" CreationDate="2019-03-26T14:39:47.243" Score="2" Body="&lt;p&gt;You don't say which method you are using (Jacobi? SSOR?) but in general, if you choose the weight negative, the iteration you are using is no longer a contraction and you will not converge at all.&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-03-26T14:39:47.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31316" PostTypeId="2" ParentId="31314" CreationDate="2019-03-26T16:43:29.427" Score="3" Body="&lt;p&gt;StackOverflow has a similar &lt;a href=&quot;https://stackoverflow.com/questions/45544709/paraview-view-a-2d-projection-of-a-3d-object&quot;&gt;question&lt;/a&gt; to yours. Although, it is not exactly the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is a pipeline for what you want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;StreamTracer1&#xA; |— Transform1&#xA; |— Transform2&#xA; |— Transform3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where each transformation correspond to a projection and a translation, namely:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$P_x = \begin{pmatrix}&#xA;0 &amp;amp;0 &amp;amp;0 &amp;amp;\Delta x\\&#xA;0 &amp;amp;1 &amp;amp;0 &amp;amp;0\\&#xA;0 &amp;amp;0 &amp;amp;1 &amp;amp;0\\&#xA;0 &amp;amp;0 &amp;amp;0 &amp;amp;1\\\end{pmatrix}\, ,\quad&#xA;P_y = \begin{pmatrix}&#xA;1 &amp;amp;0 &amp;amp;0 &amp;amp;0\\&#xA;0 &amp;amp;0 &amp;amp;0 &amp;amp;\Delta y\\&#xA;0 &amp;amp;0 &amp;amp;1 &amp;amp;0\\&#xA;0 &amp;amp;0 &amp;amp;0 &amp;amp;1\\\end{pmatrix}\, ,\quad&#xA;P_z = \begin{pmatrix}&#xA;1 &amp;amp;0 &amp;amp;0 &amp;amp;0\\&#xA;0 &amp;amp;1 &amp;amp;0 &amp;amp;0\\&#xA;0 &amp;amp;0 &amp;amp;0 &amp;amp;\Delta z\\&#xA;0 &amp;amp;0 &amp;amp;0 &amp;amp;1\\\end{pmatrix}\, ,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I used homogeneous coordinates, and &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\Delta y$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\Delta z$&lt;/span&gt; are the location of the planes where you want your projection to be placed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In ParaView, the transformation correspond to the field in &lt;code&gt;Transform &amp;gt; Translate&lt;/code&gt; and &lt;code&gt;Tranform &amp;gt; Scale&lt;/code&gt;, as shown below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y9Ad9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y9Ad9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this approach I get the following result&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GQThg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GQThg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is close to what you want.&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2019-03-26T16:43:29.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31319" PostTypeId="1" AcceptedAnswerId="31322" CreationDate="2019-03-27T01:45:44.727" Score="0" ViewCount="744" Body="&lt;p&gt;I am trying to finish a series of interpolation functions.&#xA; The problem is more related with organizing the data than how to do the interpolations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the RegularGridInterpolator, I created this function:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def interp_3d(x,y,z,fxyz,x_desired,y_desired,z_desired):&#xA;    interp_func = RegularGridInterpolator((x,y,z), fxyz)&#xA;    return interp_func([x_desired,y_desired,z_desired])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It gets three one dimensional arrays (x,y,z) and the function answer (f(x,y,z)) to return the interpolating function. The problem I am having is related to the format the data is given to me and how the function receives it. The data comes to me as a table:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x   y   z   f(x,y,z)&#xA;26600   5000    0.05    0.01&#xA;26600   5000    0.10    0.02&#xA;26600   5000    0.15    0.03&#xA;26600   5000    0.20    0.04&#xA;26600   5000    0.25    0.05&#xA;26600   10000   0.05    0.01&#xA;26600   10000   0.10    0.02&#xA;26600   10000   0.15    0.03&#xA;26600   10000   0.20    0.04&#xA;40000   5000    0.00    0.00&#xA;40000   5000    0.05    0.01&#xA;40000   5000    0.10    0.02&#xA;40000   5000    0.15    0.03&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I have to organize the f(x,y,z) function in an array of fxyz = [[[],[],[],[]]].&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I do it?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you guys recommend another method to create this interpolation function assuming that the data comes in this form?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Any recommendations?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="21588" LastEditorUserId="9667" LastEditDate="2019-03-27T02:49:35.053" LastActivityDate="2019-03-27T14:29:51.400" Title="Creating an Interpolation of a w = f(x,y,z) function" Tags="|python|interpolation|scipy|numpy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31321" PostTypeId="1" CreationDate="2019-03-27T11:22:18.803" Score="0" ViewCount="71" Body="&lt;p&gt;So I am trying to do some further tests on a 2nd-order code Incompressible Navier Stokes equations, by studying transition to turbulence in a Poiseuille flow. Specifically, I'm interested to see whether the code is energy-conserving by modelling a 3D TS wave in a Poiseuille flow. Note that I have tested the code for a 2d inviscid Taylor vortex and the code holds. Now, I have calculated the perturbation velocities via the Orr-Sommerfeld/Squire equations, and I get the Eigenspectrum and corresponding Eigenfunctions (velocities/wall-normal vorticity) for the following Poiseuille flow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$U^0_1 = 1-y^2,\,\, -1 \le y \le 1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a Reynolds number based on the centreline velocity and the channel's half-height &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Re_{cl}=\frac{U_{max}(h/2)}{\nu}=5000$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here, the driving pressure gradient is just:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Pi_1 = -\frac{2}{Re}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the linear perturbation I choose a streamwise and spanwise wavenumbers of &lt;span class=&quot;math-container&quot;&gt;$\alpha = 1.12, \beta= 2.1$&lt;/span&gt;, respectively. For the eigenvectors, the most unstable mode has been chosen (that where Real(Lambda) is the closest to zero). All the results are the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KTN9n.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KTN9n.jpg&quot; alt=&quot;Eigenspectrum &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kgtGT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kgtGT.jpg&quot; alt=&quot;Eigenvectors for &lt;span class=&quot;math-container&quot;&gt;$\lambda \approx -0.07 - 0.36i$&lt;/span&gt;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have imposed the velocity/vorticity perturbations into my code, without success (meaning not getting the transition), so clearly there's something I'm not quite getting right. Now, my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Orr-Sommerfeld Modelling: Are my results obtained (meaning perturbation velocities/vorticity) reasonable? Are these correct?&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Equations to solve for temporal transition modelling: In my opinion, the equations to be solved when studying spatio-temporal transitions are the incompressible Navier-Stokes equations (INSE) for the perturbation velocities, that is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ u_{i,t} +u_j u_{i,j} + u_j U^0_{i,j} + U^0_j u_{i,j} = -p_{,i}+\frac{1}{Re}u_{i,kk}+\Pi_i$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which results from the splitting of the instantaneous fields into base  and perturbation fields, &lt;span class=&quot;math-container&quot;&gt;$\widetilde{\phi_i} = \phi^0_i + \phi_i$&lt;/span&gt; in the INS, and then substracting the INSE of the base flow. Now my questions are: Is the previous equation correct? I'm having doubts about the pressure gradient &lt;span class=&quot;math-container&quot;&gt;$\Pi_i$&lt;/span&gt;. Second: Should be these the equations I should be solving for simulating temporal transitions? Or, Should I go just with the classic INSE for the poiseuille flow and superposed fluctuations?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14659" LastEditorUserId="14659" LastEditDate="2019-03-27T14:35:34.103" LastActivityDate="2019-03-27T14:35:34.103" Title="3D Tollmien-Schlichting Waves Imposed in a Channel Flow (Are Physics correct?, etc)" Tags="|fluid-dynamics|computational-physics|navier-stokes|singular-perturbation|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31322" PostTypeId="2" ParentId="31319" CreationDate="2019-03-27T14:29:51.400" Score="2" Body="&lt;p&gt;As I mentioned in my comment, your data is not defined over a regular grid and that's why you should not be using that function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option available in &lt;code&gt;scipy.interpolate&lt;/code&gt; is &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.griddata.html#scipy.interpolate.griddata&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;griddata&lt;/code&gt;&lt;/a&gt;, where you pass your points and values and interpolate in another set of points that you pass. None of them need to be regular. One caveat is that you are going to obtain the interpolation on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Convex_hull&quot; rel=&quot;nofollow noreferrer&quot;&gt;convex hull&lt;/a&gt; of your data (unless you use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nearest neighbor method&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is an example.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.interpolate import griddata&#xA;import matplotlib.pyplot as plt&#xA;from mpl_toolkits.mplot3d import Axes3D&#xA;&#xA;# Data&#xA;data = np.array([&#xA;        [2.66e+04, 5.00e+03, 5.00e-02, 1.00e-02],&#xA;        [2.66e+04, 5.00e+03, 1.00e-01, 2.00e-02],&#xA;        [2.66e+04, 5.00e+03, 1.50e-01, 3.00e-02],&#xA;        [2.66e+04, 5.00e+03, 2.00e-01, 4.00e-02],&#xA;        [2.66e+04, 5.00e+03, 2.50e-01, 5.00e-02],&#xA;        [2.66e+04, 1.00e+04, 5.00e-02, 1.00e-02],&#xA;        [2.66e+04, 1.00e+04, 1.00e-01, 2.00e-02],&#xA;        [2.66e+04, 1.00e+04, 1.50e-01, 3.00e-02],&#xA;        [2.66e+04, 1.00e+04, 2.00e-01, 4.00e-02],&#xA;        [4.00e+04, 5.00e+03, 0.00e+00, 0.00e+00],&#xA;        [4.00e+04, 5.00e+03, 5.00e-02, 1.00e-02],&#xA;        [4.00e+04, 5.00e+03, 1.00e-01, 2.00e-02],&#xA;        [4.00e+04, 5.00e+03, 1.50e-01, 3.00e-02]])&#xA;x, y, z, f = data.T&#xA;&#xA;# Interpolation&#xA;grid_x, grid_y, grid_z = np.mgrid[26600:40000:5j, 5000:10000:5j, 0.05:0.20:5j]&#xA;grid_f = griddata(data[:, :3], f, (grid_x, grid_y, grid_z))&#xA;&#xA;# Visualization&#xA;fig = plt.figure(figsize=(7, 3))&#xA;ax0 = fig.add_subplot(121, projection='3d')&#xA;ax0.scatter(x, y, z, c=f)&#xA;ax1 = fig.add_subplot(122, projection='3d')&#xA;ax1.scatter(grid_x.flatten(), grid_y.flatten(), grid_z.flatten(),&#xA;            c=grid_f.flatten())&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where you obtain the following result. At the left is your original dataset and the interpolated values are on the right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZOORn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZOORn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2019-03-27T14:29:51.400" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31324" PostTypeId="1" AcceptedAnswerId="31328" CreationDate="2019-03-27T17:00:32.470" Score="3" ViewCount="643" Body="&lt;p&gt;My friend in the statistics department asked me how to do the following calculation efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose we have data &lt;span class=&quot;math-container&quot;&gt;$X\in\mathbb{R}^{N\times 2}$&lt;/span&gt;. He needs to do the following calculation:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$C_{i,j}=\sigma^2\exp\left(-\frac{||X_i-X_j||_2}{\theta}\right)$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt; is a tuple with two elements and &lt;span class=&quot;math-container&quot;&gt;$\sigma,\theta$&lt;/span&gt; are scalars. So &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a &lt;span class=&quot;math-container&quot;&gt;$N\times N$&lt;/span&gt; matrix computed by taking every possible combination of rows from &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fancy myself a MATLAB aficionado, but I could not figure out how to efficiently compute this using the built in MATLAB matrix operations and functions. The fastest implementation I have uses the standard for-loops:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all; close all; clc;&#xA;&#xA;n=10;&#xA;X=rand(n,2);&#xA;sigma=1;&#xA;theta=1;&#xA;&#xA;tic&#xA;C=zeros(n,n);&#xA;for i=1:n&#xA;    for j=1:n&#xA;        C(i,j)=sigma*sigma*exp(-norm(X(i,:)-X(j,:),2)/theta);&#xA;    end&#xA;end&#xA;toc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a faster way?&lt;/p&gt;&#xA;" OwnerUserId="22717" LastEditorUserId="22717" LastEditDate="2019-03-27T17:53:33.533" LastActivityDate="2019-03-28T23:17:21.477" Title="Efficient covariance matrix calculation MATLAB (every combination of rows from data)" Tags="|matlab|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31326" PostTypeId="2" ParentId="31297" CreationDate="2019-03-27T17:57:13.237" Score="4" Body="&lt;p&gt;since you're dealing with a BVP, it's not a good choice to reduce to a first order system. That's because you should use a &lt;em&gt;finite difference&lt;/em&gt; approach. Given a uniform grid from &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; equally-spaced knots, i.e. &lt;span class=&quot;math-container&quot;&gt;$x_i=x_{i-1}+nh$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the discretization step, you can discretize the second and the first derivative (assuming knowledge of enough regularity of your solution) with finite differences, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u'(x_i) \approx \frac{u_{i+1}-u_{i-1}}{2h}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u''(x_i)\approx \frac{u_{i+1}-2u_i+u_{i-1}}{h^2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for &lt;span class=&quot;math-container&quot;&gt;$2 \leq i \leq N-1$&lt;/span&gt;, with the &quot;convenction&quot; that &lt;span class=&quot;math-container&quot;&gt;$u_i \approx u(x_i)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't care for a moment about the boundary conditions, you can easily see (just put some values for &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;) that you can build a &lt;em&gt;tridiagonal&lt;/em&gt; matrix which discretize the second derivative: so with a matrix-vector multiplication, you can get an approximation of &lt;span class=&quot;math-container&quot;&gt;$u''(x_i)$&lt;/span&gt; for every &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. Similarly for the first derivative. The entries will be made by 1,-2,1, for &lt;span class=&quot;math-container&quot;&gt;$u''(x_i)$&lt;/span&gt;, and -1,1 for &lt;span class=&quot;math-container&quot;&gt;$u'(x_i)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can look &lt;a href=&quot;http://www.maths.lth.se/na/courses/FMN130/media/material/Chapter4.08a__.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to see how they are built.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a MatLab enviroment, you can build them with the commands &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = toeplitz(sparse([1,2],[1,1],[-2,1]/(h^2),N,1));&#xA;B = toeplitz(sparse(2,1,-1/(2*h),m,1), sparse(2,1,1/(2*h),N,1));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(I used sparse because you should use a sufficient number of knots), but it's not important in order to understand how the matrices are)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this way the original problem, after the above discretization, can be rewritten as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A \vec{u} + \vec{u} (B \vec{u}-\vec{1})=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$A,B$&lt;/span&gt; are the matrices defined above (A for second derivative, B for first derivative). Of course the above equation is not much formal since I should give a meaning to what &lt;span class=&quot;math-container&quot;&gt;$\vec{u} (B-\vec{1})$&lt;/span&gt; is. But if you look component-wise the discretization, you can see that it's just a &lt;em&gt;component-wise multiplication&lt;/em&gt;. So, the correct way to write the discretized problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$A \vec{u} + diag(\vec{u}) (B \vec{u} -\vec{1})=0$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;where you see immediately that the dimensions are OK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming you have imposed in the right way the boundary conditions (it's quite easy: just change first and last rows), in order to find &lt;span class=&quot;math-container&quot;&gt;$\vec{u}$&lt;/span&gt; you have to solve a system of non-linear equations defined as &lt;span class=&quot;math-container&quot;&gt;$F(u)=A \vec{u} + diag(\vec{u}) (B \vec{u}-\vec{1})=0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The solutions &lt;span class=&quot;math-container&quot;&gt;$\bar{u}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$F(u)=0$&lt;/span&gt; will be the numerical solution of you BVP&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As you stated in your answer, Newton's method is the right choice, but we have to compute the Jacobian matrix of &lt;span class=&quot;math-container&quot;&gt;$F: \mathbb{R}^N \rightarrow \mathbb{R}^N$&lt;/span&gt;, defined as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$(JF)_{ij}=\frac{\partial F_i}{\partial u_j}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The Jacobian of &lt;span class=&quot;math-container&quot;&gt;$A\vec{u}$&lt;/span&gt; is just &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (think about the scalar case).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The term &lt;span class=&quot;math-container&quot;&gt;$\vec u$&lt;/span&gt; has as Jacobian the identity matrix &lt;span class=&quot;math-container&quot;&gt;$I_N$&lt;/span&gt; of dimension &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; (differentiate component-wise the vector &lt;span class=&quot;math-container&quot;&gt;$(u_1,\ldots,u_n)$&lt;/span&gt; w.r.t &lt;span class=&quot;math-container&quot;&gt;$u_1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$u_n$&lt;/span&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The last (and more difficult) term is &lt;span class=&quot;math-container&quot;&gt;$u u'$&lt;/span&gt;, where we have &lt;span class=&quot;math-container&quot;&gt;$diag(\vec u) (B\vec{u})$&lt;/span&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here it's more easy to look at what is the i-th component of this vector:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u_i \cdot (\frac{u_{i+1}-u_{i-1}}{2h})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to compute the Jacobian, we need to differentiate each component wrt the variables &lt;span class=&quot;math-container&quot;&gt;$u_1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$u_n$&lt;/span&gt;. But it's easy to see that here the only &lt;em&gt;non vanishing&lt;/em&gt;  terms are the ones which corresponds to positions &lt;span class=&quot;math-container&quot;&gt;$i-1,i,i+1$&lt;/span&gt;, and then the resulting matrix is tridiagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely we have this entries &lt;span class=&quot;math-container&quot;&gt;$\frac{-u_{i}}{2h}, \frac{u_{i+1}-u_{i-1}}{2h}, \frac{u_{i}}{2h}$&lt;/span&gt;. I just differentiate the above expression w.r.t &lt;span class=&quot;math-container&quot;&gt;$u_{i-1},u_i,u_{i+1}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I like to build matrices, the resulting &lt;em&gt;tridiagonal matrix&lt;/em&gt; can be written as &lt;span class=&quot;math-container&quot;&gt;$diag(B \cdot \vec{u})+diag(\vec{u})\cdot B$&lt;/span&gt; (you can check).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting together all the terms, we have the following Jacobian matrix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$JF(\vec{u})=A-I_N+diag(B \cdot \vec{u})+diag(\vec{u})\cdot B$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and we can perform our Newton's method with a suitable initial guess &lt;span class=&quot;math-container&quot;&gt;$u_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, boundary conditions are really important and &lt;em&gt;need&lt;/em&gt; to be imposed before the application of Newton's method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT [28/3/19]&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following &lt;em&gt;runable&lt;/em&gt; Octave code solves your problem. Actually, I set &lt;span class=&quot;math-container&quot;&gt;$a=4,b=1,\delta=0.05$&lt;/span&gt;, but you only need to replace them with their effective values.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m=200; %grid points&#xA;h=1/(m-1); &#xA;x=linspace(0,1,m)';&#xA;                %parameters corresponding to a,b,delta&#xA;&#xA;a=4;&#xA;b=1;&#xA;delta=0.05;&#xA;&#xA;&#xA;A = toeplitz(sparse([1,2],[1,1],[-2,1]/(h^2),m,1));&#xA;B = toeplitz(sparse(2,1,-1/(2*h),m,1), sparse(2,1,1/(2*h),m,1));&#xA;&#xA;&#xA;F=@(u) [u(1)-a;(delta*A*u+diag(u)*(B*u-1))(2:m-1);u(m)-b];&#xA;&#xA;JF=@(u) [[1,zeros(1,m-1)];(delta*A+diag(B*u)+diag(u)*B)(2:m-1,1:m);[zeros(1,m-1),1]];&#xA;&#xA;                %Newton's method&#xA;u0=linspace(a,b,m)'; %initial guess&#xA;tol=h^2/100; %second order FD&#xA;res= -JF(u0)\F(u0); %residue&#xA;&#xA;while(norm(res,inf)&amp;gt;tol)&#xA;  u0+=res;&#xA;  res=-JF(u0)\F(u0);  %main loop of Newton's method&#xA;endwhile&#xA;u0+=res;&#xA;&#xA;plot(x,u0,'*')&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also checked the right order of convergence, you can see it in the following plot&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LgKpJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LgKpJ.png&quot; alt=&quot;Correct order of convergence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I have been helpful.&lt;/p&gt;&#xA;" OwnerUserId="26296" LastEditorUserId="26296" LastEditDate="2019-03-28T15:10:14.377" LastActivityDate="2019-03-28T15:10:14.377" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31328" PostTypeId="2" ParentId="31324" CreationDate="2019-03-27T18:19:25.297" Score="1" Body="&lt;p&gt;Try the snippet below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear all; &#xA;close all; &#xA;&#xA;n=500;&#xA;rand('seed',0)&#xA;X=rand(n,2);&#xA;sigma=1;&#xA;theta=1;&#xA;&#xA;% Original method, A&#xA;tic&#xA;Ca=zeros(n,n);&#xA;% DX1a = zeros(n,n); &#xA;% DX2a = zeros(n,n); &#xA;for i=1:n&#xA;    for j=1:n&#xA;        Ca(i,j)=sigma*sigma*exp(-norm(X(i,:)-X(j,:),2)/theta);&#xA;        % DX1a(i,j) = X(i,1) - X(j,1);&#xA;        % DX2a(i,j) = X(i,2) - X(j,2);&#xA;    end&#xA;end&#xA;toc&#xA;&#xA;% Alternative method, B&#xA;tic&#xA;X1 = X(:,1);&#xA;X2 = X(:,2);&#xA;[MX1, MX2] = meshgrid(X1,X2);&#xA;DX1b = MX1'-MX1;&#xA;DX2b = MX2-MX2';&#xA;Cb = sigma*sigma*exp(-sqrt(DX1b.^2 + DX2b.^2)/theta);&#xA;toc&#xA;&#xA;C_error = norm(Ca-Cb,'fro')&#xA;&#xA;% DX1_error = norm(DX1b-DX1a,'fro')&#xA;% DX2_error = norm(DX2b-DX2a,'fro')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The trick here is using meshgrid to spill/splay the coordinates into a pair of NxN arrays (here MX1 for coordinate 1, and MX2 for coordinate 2) so that you can compute the coordinate-difference matrices, DX1 and DX2, all in one go. Then it's just a few elementwise ops (squares, sqrts, exp's) to form C. Although the original method didn't compute DX1 or DX2, I put some commented-out code in there, that you can turn back on if you wish to probe more deeply. Method B is considerably faster, on my box anyway:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Elapsed time is 7.594 seconds.&#xA;Elapsed time is 0.0380321 seconds.&#xA;C_error =   1.9128e-014&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I encourage you to test some more. The meshgrid() function is a little idiosyncratic, so double check on your datasets / larger program just to make sure.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2019-03-27T18:19:25.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31329" PostTypeId="1" CreationDate="2019-03-27T18:19:33.793" Score="2" ViewCount="98" Body="&lt;p&gt;I am solving the following biconvex problem:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\min_{x,y} f(y)$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$s.t. ~~ g(x) \leq 0$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$~~~~~h(x,y) = 0$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$x \in X, y \in Y$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; are compact convex sets, &lt;span class=&quot;math-container&quot;&gt;$g(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f(y)$&lt;/span&gt; are convex and &lt;span class=&quot;math-container&quot;&gt;$h(x,y)$&lt;/span&gt; is affine for fixed &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. &#xA;I wonder if there is any efficient algorithm to solve this type of optimization problem, not restricted to algorithms for general biconvex optimization problems. &lt;/p&gt;&#xA;" OwnerUserId="23323" LastActivityDate="2019-03-27T18:19:33.793" Title="Biconvex problem whose objective function depends on only one variable" Tags="|optimization|constrained-optimization|convex-optimization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31331" PostTypeId="2" ParentId="31285" CreationDate="2019-03-28T00:32:40.400" Score="3" Body="&lt;p&gt;If there are no algorithms specifically for sparse Hermitian matrices, a practical alternative is to convert the problem into double-sized real matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Write the complex matrix-vector products in your problem, for example &lt;span class=&quot;math-container&quot;&gt;$Ax$&lt;/span&gt;, in real an imaginary parts &lt;span class=&quot;math-container&quot;&gt;$(A_r + i A_i)(x_r + i x_i)$&lt;/span&gt;, and then convert to real symmetric matrix products like &lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix}A_r &amp;amp; -A_i \\ A_i &amp;amp; A_r \end{bmatrix}&#xA;\begin{bmatrix}{x_r \\ y_i}\end{bmatrix}.$$&lt;/span&gt; Note that since in a Hermitian matrix &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; is skew-symmetric, the above real matrix is indeed symmetric, despite the minus sign.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will double the storage for the matrices and multiply the solution cost by &lt;span class=&quot;math-container&quot;&gt;$2^k$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is a small integer (typically 2 or 3) but it should always work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some applications of Hermitian eigenproblems, it turns out that &lt;span class=&quot;math-container&quot;&gt;$A_i$&lt;/span&gt; is significantly more sparse than &lt;span class=&quot;math-container&quot;&gt;$A_r$&lt;/span&gt;, and this approach can take advantage of that situation using sparse matrix algorithms.&lt;/p&gt;&#xA;" OwnerUserId="20246" LastActivityDate="2019-03-28T00:32:40.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31332" PostTypeId="1" CreationDate="2019-03-28T15:19:46.453" Score="1" ViewCount="110" Body="&lt;p&gt;ReLU activation is defined as follows&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sigma(x)=\max(0, x).$$&lt;/span&gt;&#xA;Let's assume that I have deep network of 1 hidden layer, than output from my layer has form &#xA;&lt;span class=&quot;math-container&quot;&gt;$$ f(x)= \sigma(Wx +b), $$&lt;/span&gt;&#xA;where matrix W represents the weights, b is bias and x stands for input data (output of previous layer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is connection  between &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and piecewise linear function, defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$g(x_j)=\sum_{i=N}^N \alpha_i \psi_i(x_j),$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; stands for the coefficients and &lt;span class=&quot;math-container&quot;&gt;$\psi_i$&lt;/span&gt; for the basis function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, I would like to identify basis functions and coefficients in the definition of &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;. &lt;/p&gt;&#xA;" OwnerUserId="29268" LastEditorUserId="29268" LastEditDate="2019-03-31T21:35:07.563" LastActivityDate="2019-03-31T21:35:07.563" Title="Connection between piecewise linear basis functions and RELU activation function" Tags="|linear-algebra|nonlinear-programming|machine-learning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31333" PostTypeId="2" ParentId="31297" CreationDate="2019-03-28T18:07:01.490" Score="3" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$G(u)=\delta u''+u(u'-1) =0 \\&#xA;u(0)=a, u(1)=b$$&lt;/span&gt;&#xA;If you wish to calculate implicitly the Jacobian before applying any discretization scheme on your PDE first (e.g. so you can reuse the code of a linear solver already available to you),  you need the linearization of the nonlinear operator &lt;span class=&quot;math-container&quot;&gt;$G(u)$&lt;/span&gt;. This is the Frechet derivative of &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt;, defined as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$G_u \Delta\equiv\lim_{\epsilon\rightarrow0}\frac{G(u+\epsilon \Delta)-G(u)}{\epsilon}$$&lt;/span&gt;&#xA;Here, &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; is also a function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its derivation is straight forward, plug in &lt;span class=&quot;math-container&quot;&gt;$G(u)$&lt;/span&gt; into the definition of the Frechet derivative and take the limit in &lt;span class=&quot;math-container&quot;&gt;$\epsilon$&lt;/span&gt;.&#xA;This gives you &lt;span class=&quot;math-container&quot;&gt;$G_u \Delta = \delta \Delta''+ u \Delta' + (u'-1) \Delta$&lt;/span&gt;, now a linear operator in &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Newton-Kantorovich iteration one uses the generalized Taylor series &lt;span class=&quot;math-container&quot;&gt;$G(u+\Delta)=G(u)+G_u\Delta+O(\Delta^2)$&lt;/span&gt; at a solution  &lt;span class=&quot;math-container&quot;&gt;$u^{(i)}$&lt;/span&gt; of some previous iteration step which solves already the boundary conditions, and solves a linear PDE for &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; with homogeneous boundary conditions &lt;span class=&quot;math-container&quot;&gt;$\Delta(0)=\Delta(1)=0$&lt;/span&gt;, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$0=G^{(i)}+G_u^{(i)} \Delta$$&lt;/span&gt;&#xA;Once you have found the solution for &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; you update &lt;span class=&quot;math-container&quot;&gt;$u^{(i)}$&lt;/span&gt; to &#xA;&lt;span class=&quot;math-container&quot;&gt;$$u^{(i+1)}=u^{(i)}+\Delta$$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is repeated until &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; is small in some sense, e.g. &lt;span class=&quot;math-container&quot;&gt;$\int_\Omega\Delta^2 &amp;lt;\varepsilon$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the Newton part is more or less identical to VoB answer. We simply skip the all the error prone component wise calculations and go right to the correct definition of the derivative, so we can reuse any well tested well written linear PDE solver.&lt;/p&gt;&#xA;" OwnerUserId="3989" LastActivityDate="2019-03-28T18:07:01.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31335" PostTypeId="2" ParentId="31324" CreationDate="2019-03-28T23:08:45.490" Score="1" Body="&lt;p&gt;A slightly faster variant of @rchilton1980's method that uses singleton expansion rather than &lt;code&gt;meshgrid&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Cc = sigma * sigma * exp(-sqrt((X(:,1) - X(:,1)').^2 + (X(:,2) - X(:,2)').^2) / theta);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And yet another variant that uses &lt;code&gt;vecnorm&lt;/code&gt; (introduced in R2017b). Its speed is on par with method B, but I guess it's going to be the most efficient solution when &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$n\times k$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$k&amp;gt;2$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Cd = sigma * sigma * exp(-vecnorm(reshape(X, [500,1,2]) - reshape(X, [1,500,2]), 2, 3)/theta);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On my machine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Elapsed time is 0.575712 seconds. % Method A&#xA;Elapsed time is 0.013994 seconds. % Method B&#xA;Elapsed time is 0.006800 seconds. % Method C&#xA;Elapsed time is 0.016893 seconds. % Method D&#xA;Cb_error =&#xA;   1.3146e-14&#xA;Cc_error =&#xA;   1.3146e-14&#xA;Cd_error =&#xA;     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2019-03-28T23:17:21.477" LastActivityDate="2019-03-28T23:17:21.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31337" PostTypeId="1" AcceptedAnswerId="31339" CreationDate="2019-03-29T15:08:31.857" Score="0" ViewCount="103" Body="&lt;p&gt;I wanted to test the numerical accuracy of my program. For that I wanted to interpolate the function &lt;span class=&quot;math-container&quot;&gt;$$f=I_0\exp\left(-100x^2\right)\exp(-100y^2)$$&lt;/span&gt; onto a grid, defined on &lt;span class=&quot;math-container&quot;&gt;$$\Omega=[0,1]^2$$&lt;/span&gt; by using the equation &lt;span class=&quot;math-container&quot;&gt;$$u=f$$&lt;/span&gt; and the continuous galerkin method.&lt;br&gt;&#xA;In addition I compared the result with the expected (interpolated) function. Boundary conditions are &lt;span class=&quot;math-container&quot;&gt;$$\vec{n}\cdot u=0$$&lt;/span&gt;&lt;br&gt;&#xA;Now I would expect the lowest value to be &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$\exp(-100)\exp(-100)\approx1.3\cdot10^{-87}$&lt;/span&gt;), regardless of grid size, but instead I get (for &lt;span class=&quot;math-container&quot;&gt;$I_0=1$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+---------------+---------------+&#xA;| cell number   |  Lowest value |&#xA;+---------------+---------------+&#xA;| 16            |  -0.003398    |&#xA;| 64            |  -1.434e-5    |&#xA;| 256           |  -5.822e-10   |&#xA;| 1024          |  -4.559e-32   |&#xA;| 4096          |  0            |&#xA;+---------------+---------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Those values are not problematic for small values for &lt;span class=&quot;math-container&quot;&gt;$I_0$&lt;/span&gt;, but as soon as &lt;span class=&quot;math-container&quot;&gt;$I_0$&lt;/span&gt; increases I have to use a finer mesh, else my values will go into negative values. It also does not help by refining the mesh around the origin, the values at &lt;span class=&quot;math-container&quot;&gt;$(1,1)$&lt;/span&gt; will still be negative, unless I use the critical mesh density everywhere.&lt;br&gt;&#xA;Are those numerical errors, or errors resulting from the method itself? And is there a way to have the density of the grid below the critical density at places with a low gradient (as for example at &lt;span class=&quot;math-container&quot;&gt;$(1,1)$&lt;/span&gt;), without getting &quot;incorrect&quot; results?&lt;/p&gt;&#xA;" OwnerUserId="20545" LastActivityDate="2019-03-30T02:40:34.717" Title="Interpolation of function onto mesh gives different results, depending on mesh density" Tags="|interpolation|galerkin|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31338" PostTypeId="1" CreationDate="2019-03-29T17:16:19.630" Score="1" ViewCount="37" Body="&lt;p&gt;Reading ARPACK documentation, I see that ARPACK will return an &quot;orthogonal basis for the invariant subspace corresponding to the eigenvalues in D&quot; if eigenvectors are not requested. Can this subspace be exploited in some kind of restart capacity? Documentation discusses restarts but I'm not sure how these have been implemented in the routines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know you can set the residue vector to an initial guess, but I'd like to know if a pre-generated subspace can be used.&lt;/p&gt;&#xA;" OwnerUserId="7808" LastActivityDate="2019-03-29T17:16:19.630" Title="Can I use the Schur basis returned by ARPACK in a restart capacity?" Tags="|arpack|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31339" PostTypeId="2" ParentId="31337" CreationDate="2019-03-30T01:32:08.850" Score="5" Body="&lt;p&gt;You assume that a property that holds for the original function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is also true for the &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; projection &lt;span class=&quot;math-container&quot;&gt;$u_h=P_h f$&lt;/span&gt;. In your case, the property is that if &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is non-negative, then &lt;span class=&quot;math-container&quot;&gt;$u_h$&lt;/span&gt; should also be non-negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this assumption is not in general correct. You already know this from taking a few terms of a Fourier or Taylor series: Just because a function is non-negative, there is really no guarantee that its (truncated) Fourier series or (truncated) Taylor series is non-negative. In fact, if you think of the Fourier series of the step (&quot;Heaviside&quot;) function, then we know that this yields in Gibbs phenomenon which means that the maximum and minimum of the truncated Fourier series exceeds the maximum and minimum of the original function. The &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; project onto a finite dimensional finite element space is really of the same kind: it has over- and undershots, and that's what you are observing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will add that if you &lt;em&gt;interpolate&lt;/em&gt; the function onto the mesh, the minima and maxima of the interpolated function do &lt;em&gt;not&lt;/em&gt; exceed the extrema of the original function. That is a property of the interpolation operation that is not shared by the projection operation.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2019-03-30T02:40:34.717" LastActivityDate="2019-03-30T02:40:34.717" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31340" PostTypeId="1" CreationDate="2019-03-31T08:57:34.537" Score="3" ViewCount="112" Body="&lt;p&gt;Assume I have a system of at least (but generally only) &lt;span class=&quot;math-container&quot;&gt;$N+1$&lt;/span&gt; points in an &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;-dimensional space (&lt;span class=&quot;math-container&quot;&gt;$N &amp;gt; 3$&lt;/span&gt; is possible). At each of these points &lt;span class=&quot;math-container&quot;&gt;$x_i, i=1,...,N+1$&lt;/span&gt; I know an initial potential/concentration/quantity &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt;. I can evaluate the possible connections between all points. I also have an isotropic diffusion coefficient &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;. I am now interested in calculating diffusion between these three points (with mass conservation). I have sketched an example case in 2-D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This problem sounds rather easy, but I quickly realized that it isn't. Classic finite difference approaches require (to my knowledge) a structured gid, which I don't have. Finite volume could handle unstructured grids, but I cannot afford tessellation in high dimensions (say, &lt;span class=&quot;math-container&quot;&gt;$N &amp;gt; 100$&lt;/span&gt;). For both approaches I would have calculated the first potential derivative along each line (e.g., &lt;span class=&quot;math-container&quot;&gt;$x_1$&lt;/span&gt;--&gt;&lt;span class=&quot;math-container&quot;&gt;$x_2$&lt;/span&gt;), and assumed that the same derivative half a distance outside (dashed lines) is zero. Unfortunately, I don't think this assumption works if I have more than &lt;span class=&quot;math-container&quot;&gt;$N+1$&lt;/span&gt; points. Finite elements might be an option, but I have yet to find a tutorial which would cover such a case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any idea how to solve this system?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Some additional clarification concerning my goal: I want to simulate the equilibration between the &lt;span class=&quot;math-container&quot;&gt;$N+1$&lt;/span&gt; vertices. I am only interested in solutions at the vertices themselves, and diffusion would occur along the edges. The specific equation I want to solve is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Heat_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;heat equation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{dP_i}{dt}=K\Delta P_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4fHau.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4fHau.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="24382" LastEditorUserId="24382" LastEditDate="2019-04-02T09:52:43.237" LastActivityDate="2019-04-02T09:52:43.237" Title="Numerical solution to N-dimensional diffusion on simplex?" Tags="|finite-element|finite-difference|diffusion|high-dimensional|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31341" PostTypeId="2" ParentId="31313" CreationDate="2019-04-01T20:51:06.343" Score="0" Body="&lt;p&gt;No, it won't work. Kahan's theorem assures you that it will converge if w is between 0 and 2&lt;/p&gt;&#xA;" OwnerUserId="34236" LastEditorUserId="34236" LastEditDate="2019-04-01T20:59:47.423" LastActivityDate="2019-04-01T20:59:47.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31342" PostTypeId="1" CreationDate="2019-04-02T01:08:34.483" Score="0" ViewCount="38" Body="&lt;p&gt;&lt;em&gt;Please note:&lt;/em&gt; this question is not a duplicate of &lt;a href=&quot;https://scicomp.stackexchange.com/questions/31343/analytic-vs-discrete-understanding-of-pde&quot;&gt;this question&lt;/a&gt; since, while the PDE is the same, the nature of this question is different, i.e. the other question treats a different aspect of this PDE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the PDE system I am working with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\partial_tu = \nabla \cdot (a(x)\nabla u)-\beta(x)u\\&#xA;\partial_nu=0, x \in \Omega \subset \mathbb{R}^2\\&#xA;\beta(x)&amp;gt;0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Integrate the PDE:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_\Omega \partial_t u=\int_\Omega \nabla\cdot (a(x)\nabla u)dx-\int_\Omega \beta(x)udx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Apply the Divergence theorem:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_\Omega \partial_t u=\int_{\partial\Omega} a(x)\nabla u\cdot nds-\int_\Omega \beta(x)udx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Apply the Neuman boundary condition:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_\Omega \partial_t u=\int_{\partial\Omega} a(x)\partial_n uds-\int_\Omega \beta(x)udx=-\int_\Omega \beta(x)udx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Obtain a new PDE:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\partial_t u+\beta(x)u=0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This PDE is in essence an ODE, so we can get the following solution for this ODE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u(x,t)=e^{-\beta(x)t}D(x),$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$D(x)$&lt;/span&gt; is an integration &quot;constant&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;Discretize the ODE:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{u_{ij}^{n+1}-u_{ij}^n}{\Delta t}=-\beta_{ij} u_{ij}^n$$&lt;/span&gt;&#xA;or&#xA;&lt;span class=&quot;math-container&quot;&gt;$$u^{n+1}_{ij} = u^n_{ij}(1-\Delta t \beta_{ij})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now we can assign lexicographical ordering to each &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt; and denote it by a function &lt;span class=&quot;math-container&quot;&gt;$p(i,j)$&lt;/span&gt;, thus rewriting the discretized PDE about as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u^{n+1}_{p(i,j)} = u^n_{p(i,j)}(1-\Delta t \beta_{p(i,j)})$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I correct to deduce that for each &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; we have a linear system of the form&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$U^{n+1}=AU^n$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is constant and &lt;span class=&quot;math-container&quot;&gt;$A = diag(1-\beta_{ij},\dots , 1-\beta_{ij})$&lt;/span&gt;? If this is true then is it also correct that there are thus as many such linear systems as there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;'s (say, &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;) and to solve this discretized system we need to solve &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; such linear systems?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think, alternatively, we can probably extend the system to include not only &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt; in the lexicographical ordering, but also to include &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; in it, so as to get only one linear system to solve.&lt;/p&gt;&#xA;" OwnerUserId="29427" LastEditorUserId="29427" LastEditDate="2019-04-02T14:37:03.783" LastActivityDate="2019-04-02T14:37:03.783" Title="What will PDE discretization matrix look like for time and space?" Tags="|discretization|time-integration|parabolic-pde|linear-system|" AnswerCount="0" CommentCount="3" ClosedDate="2019-04-02T17:27:19.233" ContentLicense="CC BY-SA 4.0" />
  <row Id="31343" PostTypeId="1" CreationDate="2019-04-02T02:12:55.290" Score="0" ViewCount="84" Body="&lt;p&gt;The PDE I am working with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\partial_tu = \nabla \cdot (a(x)\nabla u)-\beta(x)u\\&#xA;\partial_nu=0, x \in \Omega \subset \mathbb{R}^2\\&#xA;\beta(x)&amp;gt;0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Integrate the PDE:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_\Omega \partial_t u=\int_\Omega \nabla\cdot (a(x)\nabla u)dx-\int_\Omega \beta(x)udx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Apply the Divergence theorem:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_\Omega \partial_t u=\int_{\partial\Omega} a(x)\nabla u\cdot nds-\int_\Omega \beta(x)udx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Apply the Neuman boundary condition:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_\Omega \partial_t u=\int_{\partial\Omega} a(x)\partial_n uds-\int_\Omega \beta(x)udx=-\int_\Omega \beta(x)udx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Obtain a new PDE:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\partial_t u+\beta(x)u=0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This PDE is in essence an ODE, so we can get the following solution for this ODE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u(x,t)=e^{-\beta(x)t}D(x),$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$D(x)$&lt;/span&gt; is an integration &quot;constant&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;Discretize the ODE. After discretization I should obtain a linear system of the following form:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{du}{dt}=Au - c$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I discretize this system I obtain the following:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{du_{ij}}{dt}=-\beta_{ij} u_{ij}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So one could say that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; in this case has the form diag&lt;span class=&quot;math-container&quot;&gt;$(-\beta_{ij}, \dots, -\beta_{ij})$&lt;/span&gt;, but then where is the vector &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; coming from?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, one could observe that, since &lt;span class=&quot;math-container&quot;&gt;$\beta(x)&amp;gt;0$&lt;/span&gt;, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\lim\limits_{t\to \infty} u(x,t) = 0$$&lt;/span&gt;.&#xA;But how can we deduce the same from the discretized system?&lt;/p&gt;&#xA;" OwnerUserId="29427" LastActivityDate="2019-04-02T02:12:55.290" Title="Analytic vs discrete understanding of PDE" Tags="|discretization|numerics|parabolic-pde|linear-system|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="31345" PostTypeId="1" AcceptedAnswerId="31351" CreationDate="2019-04-02T14:38:35.700" Score="6" ViewCount="288" Body="&lt;p&gt;With a truncated Whitaker-Shannon series (cardinal series)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(t) = \sum_{j = 0}^{n-1} y_{j} \frac{\sin\left(\pi( \frac{t-t_0}{h} -j)\right)}{\pi\left(\frac{t-t_0}{h}-j\right)}&#xA;$$&lt;/span&gt;&#xA;we can naively evaluate the sum by repeated calls to sinc routines, such as the following code does:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;   Real naive_sum(Real t) const {&#xA;        using boost::math::constants::pi;&#xA;        Real f = 0;&#xA;        for (size_t i = 0; i &amp;lt; m_y.size(); ++i) {&#xA;            Real arg = pi&amp;lt;Real&amp;gt;()*( (t-m_t0)/m_h - i);&#xA;            f += m_y[i]*boost::math::sinc_pi(arg);&#xA;        }&#xA;        return y;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, repeated calls to sinc are expensive, so we can use the identity &lt;span class=&quot;math-container&quot;&gt;$\sin(\theta - j\pi) = (-1)^{j}\sin(\theta)$&lt;/span&gt; to write this as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(t) = \frac{\sin(\pi(t-t_0)/h)}{\pi} \sum_{j=0}^{n-1} (-1)^{j}\frac{y_{j}}{(t-t_0)/h -j}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which can be implemented as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;    Real operator()(Real t) const {&#xA;        using boost::math::constants::pi;&#xA;        using std::sin;&#xA;        Real y = 0;&#xA;        Real x = (t - m_t0)/m_h;&#xA;&#xA;        for (size_t i = 0; i &amp;lt; m_y.size(); ++i)&#xA;        {&#xA;            Real denom = (x - i);&#xA;            if (denom == 0) {&#xA;                return m_y[i];&#xA;            }&#xA;            if (i &amp;amp; 1) {&#xA;                y -= m_y[i]/denom;&#xA;            }&#xA;            else {&#xA;                y += m_y[i]/denom;&#xA;            }&#xA;        }&#xA;        return y*sin(pi&amp;lt;Real&amp;gt;()*x)/pi&amp;lt;Real&amp;gt;();&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I have observed a &lt;em&gt;vast&lt;/em&gt; decrease in accuracy using the fast method over the slow method. Can the speed of the fast method be preserved without a massive decrease in accuracy?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Working code, for those that care:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#ifndef BOOST_MATH_INTERPOLATORS_WHITAKKER_SHANNON_HPP&#xA;#define BOOST_MATH_INTERPOLATORS_WHITAKKER_SHANNON_HPP&#xA;#include &amp;lt;boost/math/special_functions/sinc.hpp&amp;gt;&#xA;#include &amp;lt;boost/math/constants/constants.hpp&amp;gt;&#xA;&#xA;namespace boost::math::interpolators {&#xA;&#xA;template&amp;lt;class RandomAccessContainer&amp;gt;&#xA;class whittaker_shannon {&#xA;public:&#xA;&#xA;    using Real = typename RandomAccessContainer::value_type;&#xA;    whittaker_shannon(RandomAccessContainer&amp;amp;&amp;amp; y, Real t0, Real h) : m_y{std::move(y)}, m_t0{t0}, m_h{h}&#xA;    {&#xA;    }&#xA;&#xA;    Real operator()(Real t) const {&#xA;        using boost::math::constants::pi;&#xA;        using std::sin;&#xA;        Real y = 0;&#xA;        Real x = (t - m_t0)/m_h;&#xA;&#xA;        for (size_t i = 0; i &amp;lt; m_y.size(); ++i)&#xA;        {&#xA;            Real denom = (x - i);&#xA;            if (denom == 0) {&#xA;                return m_y[i];&#xA;            }&#xA;            if (i &amp;amp; 1) {&#xA;                y -= m_y[i]/denom;&#xA;            }&#xA;            else {&#xA;                y += m_y[i]/denom;&#xA;            }&#xA;        }&#xA;        return y*sin(pi&amp;lt;Real&amp;gt;()*x)/pi&amp;lt;Real&amp;gt;();&#xA;    }&#xA;&#xA;&#xA;    Real naive_sum(Real t) const {&#xA;        using boost::math::constants::pi;&#xA;        Real y = 0;&#xA;        Real s = pi&amp;lt;Real&amp;gt;()*(t-m_t0)/m_h;&#xA;        for (size_t i = 0; i &amp;lt; m_y.size(); ++i) {&#xA;            Real arg = pi&amp;lt;Real&amp;gt;()*( (t-m_t0)/m_h - i);&#xA;            y += m_y[i]*sinc_pi(arg);&#xA;        }&#xA;        return y;&#xA;    }&#xA;&#xA;&#xA;    Real operator[](size_t i) const {&#xA;        return m_y[i];&#xA;    }&#xA;&#xA;&#xA;private:&#xA;    RandomAccessContainer m_y;&#xA;    Real m_t0;&#xA;    Real m_h;&#xA;};&#xA;}&#xA;#endif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's a test that reproduces the phenomenon:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;template&amp;lt;class Real&amp;gt;&#xA;void test_bump()&#xA;{&#xA;    using std::exp;&#xA;    using std::abs;&#xA;    auto bump = [](Real x) { if (abs(x) &amp;gt;= 1) { return Real(0); } return exp(-Real(1)/(Real(1)-x*x)); };&#xA;&#xA;    Real t0 = -1;&#xA;    size_t n = 2049;&#xA;    Real h = Real(2)/Real(n-1);&#xA;&#xA;    std::vector&amp;lt;Real&amp;gt; v(n);&#xA;    for(size_t i = 0; i &amp;lt; n; ++i) {&#xA;        Real t = t0 + i*h;&#xA;        v[i] = bump(t);&#xA;    }&#xA;&#xA;&#xA;    auto ws = whittaker_shannon(std::move(v), t0, h);&#xA;&#xA;    std::mt19937 gen(323723);&#xA;    std::uniform_real_distribution&amp;lt;long double&amp;gt; dis(-0.95, 0.95);&#xA;&#xA;    size_t i = 0;&#xA;    while (i++ &amp;lt; 1000)&#xA;    {&#xA;        Real t = static_cast&amp;lt;Real&amp;gt;(dis(gen));&#xA;        Real expected = bump(t);&#xA;        if(!CHECK_MOLLIFIED_CLOSE(expected, ws(t), 50*std::numeric_limits&amp;lt;Real&amp;gt;::epsilon())) {&#xA;            std::cerr &amp;lt;&amp;lt; &quot;  Problem occured at abscissa &quot; &amp;lt;&amp;lt; t &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;        }&#xA;    }&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2019-04-02T15:38:49.513" LastActivityDate="2019-04-03T13:23:25.180" Title="Whittaker-Shannon interpolation: Accuracy dies with speedup; can it be fixed?" Tags="|c++|performance|accuracy|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31346" PostTypeId="2" ParentId="27649" CreationDate="2019-04-02T15:19:48.910" Score="1" Body="&lt;p&gt;When the points belong to more than one curve, it will first be necessary to cluster them into curves. A possible approach is described together with a reference implementation in&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dalitz, Wilberg, Aymans: &lt;a href=&quot;http://www.ipol.im/pub/art/2019/234/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TriplClust: An Algorithm for Curve Detection in 3D Point Clouds.&lt;/a&gt; IPOL 2019.234 (2019)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When your points have an implicit parameter representing an approximate time of the point on the &lt;em&gt;x(t)&lt;/em&gt; curve, Moving Least Squares might be applicable (beware that the algorithm commonly associated with this term is &lt;em&gt;not&lt;/em&gt; applicable for 1D manifolds, i.e. curves):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Amirfakhrian, Mafikandi: &quot;Approximation of parametric curves by Moving Least&#xA;  Squares method.&quot; Applied Mathematics and Computation 283, pp. 290-298 (2016)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When no time is knwon, a possible option is to &lt;em&gt;select&lt;/em&gt; some representative points and fit a &lt;em&gt;perfect&lt;/em&gt; spline through this representative points:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Rupniewski: &quot;Curve Reconstruction from Noisy and Unordered Samples.&quot;&#xA;  ICPRAM, pp. 183-188 (2014)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In any case, I would greatly apprciate, if you keep me informed which method works for you and how you have solved your problem.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-04-02T15:19:48.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31348" PostTypeId="1" CreationDate="2019-04-02T19:32:18.307" Score="1" ViewCount="85" Body="&lt;p&gt;I'm wondering if someone could help me out, or point me in a direction of how I can understand the following oscillations that occur when I solve the Porous Medium Equation &lt;span class=&quot;math-container&quot;&gt;$$u_t = u_{xx}^{m+1}$$&lt;/span&gt; using FTCS method. The following plots occur for a the following initial condition&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f(x) = &#xA;\begin{cases}&#xA;1 &amp;amp; \text{if}\ |x| \geq 1 \\&#xA;1 - e^{-1/(1-|x|^2)} &amp;amp; \text{if}\ |x| &amp;lt; 1&#xA;\end{cases}&#xA;$$&lt;/span&gt;&#xA;with Dirichlet boundary conditions as &lt;span class=&quot;math-container&quot;&gt;$u(x_0, t)= u(x_M,t) = 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By adjusting the step size for space and time, I end up with the given plots below. The surprising result for me, who just learned about finite difference methods, is that the numerical solution does not converge for values where &lt;span class=&quot;math-container&quot;&gt;$$\frac{k\ \text{max}\{u^{m}\}}{h^2} &amp;lt; \frac{1}{2}.$$&lt;/span&gt;&#xA;How come? It seems to converge for values &lt;span class=&quot;math-container&quot;&gt;$&amp;lt; \frac{1}{4}$&lt;/span&gt; instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NpqLj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NpqLj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/EZ31d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EZ31d.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29932" LastEditorUserId="123" LastEditDate="2019-04-03T08:00:28.420" LastActivityDate="2019-04-03T08:00:28.420" Title="Oscillations when solving parabolic heat equation with FTCS" Tags="|finite-difference|nonlinear-equations|parabolic-pde|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="31349" PostTypeId="1" AcceptedAnswerId="31361" CreationDate="2019-04-02T22:00:07.420" Score="1" ViewCount="412" Body="&lt;p&gt;First Part: (First-order derivative)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is an infinitely differential function everywhere, the Taylor series of &lt;span class=&quot;math-container&quot;&gt;$f(x + h)$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}\tag{1}&#xA;f(x + h) = f(x) + hf'(x) + \frac{1}{2}h^2f''(\xi)&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\xi$&lt;/span&gt; is a number between &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x+h$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After rearrangment of terms in (1), we can write&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f'(x) = \frac{f(x+h) - f(x)}{h} - \frac{1}{2}hf''(\xi).&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we define a finite difference approximation of &lt;span class=&quot;math-container&quot;&gt;$f'(x)$&lt;/span&gt; by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f'_h(x) = \frac{f(x+h) - f(x)}{h},&#xA;$$&lt;/span&gt; and we express&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f'(x) = f'_h(x) + E_1&#xA;$$&lt;/span&gt;&#xA;where approximation error &lt;span class=&quot;math-container&quot;&gt;$E_1$&lt;/span&gt; satisfy&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;|E_1| &amp;amp;= |- 0.5 hf''(\xi)| \\&#xA;    &amp;amp;\leq Ch&#xA;\end{align}&lt;/span&gt;&#xA;assuming &lt;span class=&quot;math-container&quot;&gt;$|- 0.5f''(\xi)| \leq C$&lt;/span&gt;. Now, using the definition of Big-O notation, we can say&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}\tag{2}&#xA;f'(x) = f'_h(x) + O(h)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a very standard result. However, I have a question for clarification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 1:&#xA;It seems that the constant &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; can be based on the local behavior of function between &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x+h$&lt;/span&gt;. Can I say that &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; depends on &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;? Moreover, can I comment on the behavior of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$h \to 0$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second Part: (Square of the first-order derivative)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using (2), the square of &lt;span class=&quot;math-container&quot;&gt;$f'(x)$&lt;/span&gt; can be expressed as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\Big(f'(x)\Big)^2 = \Big(f'_h(x) + O(h)\Big)^2&#xA;= \Big(f'_h(x)\Big)^2 + 2f'_h(x)O(h) + O(h^2) = \Big(f'_h(x)\Big)^2 + E_2&#xA;$$&lt;/span&gt;&#xA;where the approximation error &lt;span class=&quot;math-container&quot;&gt;$E_2$&lt;/span&gt; is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;E_2 = 2f'_h(x)O(h) + O(h^2).&#xA;$$&lt;/span&gt;&#xA;It seems that, &lt;span class=&quot;math-container&quot;&gt;$E_2$&lt;/span&gt; depends on the local approximation quantity &lt;span class=&quot;math-container&quot;&gt;$f'_h(x)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 2:&#xA;How can we estimate the leading order term for the error &lt;span class=&quot;math-container&quot;&gt;$E_2$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="13142" LastEditorUserId="13142" LastEditDate="2019-04-02T22:05:30.953" LastActivityDate="2019-04-04T02:09:07.780" Title="Approximation Error in a Finite Difference Approximation of the Square of Derivative" Tags="|pde|finite-difference|discretization|numerics|optimal-control|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31350" PostTypeId="2" ParentId="19870" CreationDate="2019-04-03T00:21:35.483" Score="6" Body="&lt;p&gt;I agree with all of the answers provided here but I wanted to supplement with a Python implementation. We developed the Python GEKKO package for solving similar problems. We're also working on &lt;a href=&quot;https://apmonitor.com/do/index.php/Main/DeepLearning&quot; rel=&quot;noreferrer&quot;&gt;machine learning functions&lt;/a&gt; that may be able to combine a convolutional neural network with this constrained mixed-integer problem as a single optimization. Here is a potential solution with &lt;a href=&quot;https://gekko.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;Python GEKKO (&gt;0.2rc4)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from gekko import GEKKO&#xA;m = GEKKO()&#xA;ni = 8&#xA;nj = 3&#xA;x = [[m.Var(lb=0,integer=True) for j in range(nj)] for i in range(ni)]&#xA;s = 0&#xA;for i in range(ni):&#xA;    for j in range(nj):&#xA;        s += x[i][j]&#xA;m.Equation(s==10)&#xA;m.Equations([x[2][j]+x[4][j]&amp;gt;=x[0][j] for j in range(nj)])&#xA;m.Equations([x[3][j]+x[5][j]&amp;gt;=x[1][j] for j in range(nj)])&#xA;for j in range(nj):&#xA;    x[6][j].upper=15&#xA;    x[7][j].upper=15&#xA;m.Equations([(m.sign3(x[6][j])==m.sign3(x[2][j])) for j in range(nj)])&#xA;m.Equations([(m.sign3(x[7][j])==m.sign3(x[3][j])) for j in range(nj)])&#xA;m.Obj(sum([m.tanh(x[i][0]) for i in range(ni)]))&#xA;m.options.SOLVER=1&#xA;m.solve()&#xA;print(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When you switch to IPOPT with &lt;strong&gt;m.options.SOLVER=3&lt;/strong&gt;, it gives a non-integer solution, as expected:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;---------------------------------------------------&#xA;Solver         :  IPOPT (v3.12)&#xA;Solution time  :   0.109899999995832      sec&#xA;Objective      :   7.933869530630442E-009&#xA;Successful solution&#xA;---------------------------------------------------&#xA;&#xA;[[0.0], [0.2838769309], [0.2838769309]]&#xA;[[0.0], [0.2838769309], [0.2838769309]]&#xA;[[3.9669348007e-09], [0.99734110398], [0.99734110398]]&#xA;[[3.96693473e-09], [0.99734110398], [0.99734110398]]&#xA;[[0.0], [0.52439022369], [0.52439022369]]&#xA;[[0.0], [0.52439022369], [0.52439022369]]&#xA;[[0.0], [0.69439174379], [0.69439174379]]&#xA;[[0.0], [0.69439174379], [0.69439174379]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when you switch to the &lt;a href=&quot;https://en.wikipedia.org/wiki/APOPT&quot; rel=&quot;noreferrer&quot;&gt;APOPT&lt;/a&gt; MINLP solver with &lt;strong&gt;m.options.SOLVER = 1&lt;/strong&gt;, it gives an integer solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;---------------------------------------------------&#xA;Solver         :  APOPT (v1.0)&#xA;Solution time  :   3.900000000430737E-002 sec&#xA;Objective      :   0.000000000000000E+000&#xA;Successful solution&#xA;---------------------------------------------------&#xA;&#xA;[[0.0], [2.0], [1.0]]&#xA;[[0.0], [1.0], [1.0]]&#xA;[[0.0], [0.0], [0.0]]&#xA;[[0.0], [0.0], [0.0]]&#xA;[[0.0], [2.0], [1.0]]&#xA;[[0.0], [1.0], [1.0]]&#xA;[[0.0], [0.0], [0.0]]&#xA;[[0.0], [0.0], [0.0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the solution is fast but it is likely because the problem size is small and the objective function is not correct. For small problems, an exhaustive search may be the right solution. However, if you scale-up to larger problems with more integer variables or include the convolutional neural network model, each NLP branch and bound sub-problem will take longer to solve or may not converge.&lt;/p&gt;&#xA;" OwnerUserId="906" LastActivityDate="2019-04-03T00:21:35.483" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31351" PostTypeId="2" ParentId="31345" CreationDate="2019-04-03T04:30:25.740" Score="14" Body="&lt;p&gt;I was able to reproduce the behavior reported in the question, and traced the observed inaccuracies to the following line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;return y*sin(pi&amp;lt;Real&amp;gt;()*x)/pi&amp;lt;Real&amp;gt;();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The explicit multiplication with a floating-point approximation of π introduces a small error into the argument to &lt;code&gt;sin&lt;/code&gt;, which comprises the representational error in the constant and the rounding error added by the multiply and is on the order of 1 ulp. This small error in the argument represents a phase error in &lt;code&gt;sin&lt;/code&gt; which &lt;em&gt;grows with the magnitude&lt;/em&gt; of &lt;code&gt;x&lt;/code&gt;. In this case |x| is on the order of 1000, resulting in quite a bit of error magnification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because this is a relatively frequent scenario, various platforms offer a &lt;code&gt;sinpi&lt;/code&gt; function that computes sin (π x) with the multiplication by π happening inside &lt;code&gt;sinpi&lt;/code&gt; after argument reduction, minimizing the phase error. In the specific case of Boost the desired function is &lt;code&gt;boost::math::sin_pi&lt;/code&gt;. Often the &lt;code&gt;sinpi&lt;/code&gt; function is also more efficient than the regular &lt;code&gt;sin&lt;/code&gt; function, since its internal argument reduction is simpler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replacing the original source line with the line below should fix the observed accuracy issue completely:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;return y*boost::math::sin_pi(x)/pi&amp;lt;Real&amp;gt;();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20458" LastEditorUserId="20458" LastEditDate="2019-04-03T13:23:25.180" LastActivityDate="2019-04-03T13:23:25.180" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31353" PostTypeId="2" ParentId="14671" CreationDate="2019-04-03T08:18:45.257" Score="6" Body="&lt;p&gt;It is quite straightforward to demonstrate this for implicit Euler (aka backward Euler) on a scalar example. Consider the initial value problem&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\dot{y}(t) = i \alpha y(t), \ y(0) = 1$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with solution&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y(t) = \exp(i \alpha t) = \cos(\alpha t) + i \sin(\alpha t)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The solution is a harmonic oscillation with amplitude 1 and this amplitude does not change over time.&lt;/p&gt;&#xA;&lt;p&gt;Now apply implicit Euler to the ODE, which gives&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{n+1} = y_{n} + h i \alpha y_{n+1}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; being your time step or, after rearranging,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{n+1} = \frac{y_n}{1 - h i \alpha} = \frac{1}{\alpha^2 h^2+1} \left(1 + i \alpha h \right) y_n$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Because I assume &lt;span class=&quot;math-container&quot;&gt;$y(0) = 1$&lt;/span&gt;, that means&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y_{n+1} = \left( \frac{1}{\alpha^2 h^2 + 1} \right)^n \left( 1 + i \alpha h \right)^n$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;While the first term goes to one &lt;em&gt;in the limit&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$h \to 0$&lt;/span&gt;, we have&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{1}{\alpha^2 h^2 + 1} &amp;lt; 1$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;for any finite value of &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;. Of course, any actual numerical solution will involve a finite &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;. Therefore, we get&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left( \frac{1}{\alpha^2 h^2 + 1} \right)^n \to 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;as &lt;span class=&quot;math-container&quot;&gt;$n \to \infty$&lt;/span&gt;, meaning that in contrast to what should happen, the amplitude of the numerical solution will decrease over time. This effect is referred to as &lt;em&gt;numerical damping&lt;/em&gt; because the decrease in amplitude is purely an artifact of the numerical method.&lt;/p&gt;&#xA;&lt;p&gt;Numerical damping in implicit Euler is very strong. If you implement the scalar problem above, you will see that unless you use very small time steps &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, after a few oscillations your numerical solution will have almost decayed to zero. Methods like BDF-2 also show numerical damping but much less pronounced than implicit Euler. I'd assume this is why Comsol recommends them in the text you posted.&lt;/p&gt;&#xA;" OwnerUserId="16921" LastEditorUserId="20688" LastEditDate="2021-12-31T03:23:55.690" LastActivityDate="2021-12-31T03:23:55.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31354" PostTypeId="1" CreationDate="2019-04-02T20:24:11.063" Score="0" ViewCount="130" Body="&lt;p&gt;I am currently trying to solve some PDEs with FiPy. At page 56, the manual mentions (&lt;a href=&quot;https://www.ctcms.nist.gov/fipy/download/fipy-3.0.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ctcms.nist.gov/fipy/download/fipy-3.0.pdf&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The largest stable timestep that can be taken for this explicit 1D diffusion problem is &lt;span class=&quot;math-container&quot;&gt;$∆t ≤ ∆x^ 2/(2D)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A few questions that might appear basic but I'm having a really hard time getting my head around:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Why is that inequality? How does that scale for 2D and 3D problems?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Shouldn't units come into play here? For example, if I converted by time-step from hours to minutes, then clearly the magnitude of the value would change, even though the actual time represented would remain the same? Or does the fact I am also scaling the Diffusivity (which depends both on the units of space and time) ensures the inequality remains sound?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="28524" OwnerDisplayName="DarioP" LastActivityDate="2019-05-03T18:03:05.570" Title="Stability of PDEs" Tags="|computational-physics|differential-equations|diffusion|stability|algorithms|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="31357" PostTypeId="1" CreationDate="2019-04-03T15:05:54.427" Score="1" ViewCount="126" Body="&lt;p&gt;Now I have wave functions &lt;span class=&quot;math-container&quot;&gt;$\psi_a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\psi_b$&lt;/span&gt; of two states in Gaussian CUBE format. I'd like to evaluate the transition dipole moment integral &lt;span class=&quot;math-container&quot;&gt;$\pmb\mu$&lt;/span&gt; between these two states. As my simulation cell is cubic periodic with length of &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;, I will need the Berry phase formula to calculate &lt;span class=&quot;math-container&quot;&gt;$\pmb\mu$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\pmb\mu= \rm{IMAG}\; \rm{LOG}\; \langle \psi_a|\exp(\it i\frac{2\pi}{\it L}\pmb r)|\psi_b\rangle\; $$&lt;/span&gt; &#xA;My implementation is as follows: &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of grid points of CUBE file in each direction. &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the step length. So the lattice constant &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$nb$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\pmb r$&lt;/span&gt; is the coordinate. psi_a and psi_b are the volumetric data which are changed to 1D arrays. &lt;span class=&quot;math-container&quot;&gt;$dd=\exp(i2\pi \pmb r/L)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vz3ji.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vz3ji.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;However, the above implementation didn't give me correct numbers as I expected. So please can anyone help me point out the problems of the implementation?&lt;/p&gt;&#xA;" OwnerUserId="30377" OwnerDisplayName="xmW" LastEditorUserId="-1" LastEditDate="2019-04-04T03:04:51.380" LastActivityDate="2019-04-04T03:04:51.380" Title="How to numerically calculate the transition dipole integral in periodic systems?" Tags="|quantum-mechanics|computational-physics|algorithms|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31358" PostTypeId="2" ParentId="31354" CreationDate="2019-04-03T17:48:15.903" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;This comes from von Neumann stability analysis. You discretize the partial differential equation and look at the error equation, which is the difference between the exact solution to the finite difference approximation and the actual equation. You need the error to shrink in time, and if you assume that the error behaves like a fourier series, you can evaluate the growth of the error, which must have a magnitude less than one for stability. The page for von Neumann analysis on wikipedia can explain the exact process to follow for your question. In 2D or 3D it is more complicated, but as I recall it is possible in some cases. One thing to note is that von Neumann error analysis is developed for linear PDEs.&lt;/li&gt;&#xA;&lt;li&gt;Yes, your diffusivity should be on the RHS of the inequality and will scale with your units appropriately&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9446" LastActivityDate="2019-04-03T17:48:15.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31359" PostTypeId="2" ParentId="30173" CreationDate="2019-04-03T18:42:45.930" Score="1" Body="&lt;p&gt;According to &lt;a href=&quot;https://brage.bibsys.no/xmlui/handle/11250/260376&quot; rel=&quot;nofollow noreferrer&quot;&gt;this master thesis&lt;/a&gt;, the Dynamic Model obtained from the Newton Euler Method for articulated arms cannot be controlled (as in control theory) by applying torque directly without feedback. I suppose the floating point lack of precision is sufficient to make the robot deviate from the wanted behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the best answer I could conclude, somebody with knowledge of control theory and robotics can confirm this.&lt;/p&gt;&#xA;" OwnerUserId="28685" LastActivityDate="2019-04-03T18:42:45.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31360" PostTypeId="1" CreationDate="2019-04-03T21:33:01.807" Score="2" ViewCount="354" Body="&lt;p&gt;I have a 32 cores machine, and I need to run 16 different dynamics simulations in parallel on it. I want the 16 jobs to run in parallel, not sequentially, on the same machine. The 16 dynamics input are in different folders and they requires the same time to complete. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;pbs script: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash -l&#xA;#PBS -l nodes=...:ppn=32,walltime=...&#xA;#&#xA;#PBS -N dynamics&#xA;&#xA;export OMP_NUM_THREADS=32&#xA;&#xA;export PATH=mypath:$PATH&#xA;&#xA;export DIR=&quot;my folder&quot;&#xA;&#xA;export WORK=/scratch/Work/&#xA;&#xA;cp -r &lt;span class=&quot;math-container&quot;&gt;$DIR/dyn1 $&lt;/span&gt;WORK/.&#xA;&#xA;cp -r &lt;span class=&quot;math-container&quot;&gt;$DIR/dyn2 $&lt;/span&gt;WORK/.&#xA;&#xA;...&#xA;&#xA;cp -r &lt;span class=&quot;math-container&quot;&gt;$DIR/dyn3 $&lt;/span&gt;WORK/.&#xA;&#xA;cd $WORK&#xA;&#xA;my_dynamic_program1.x &#xA;my_dynamic_program2.x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30376" LastEditorUserId="30376" LastEditDate="2019-04-04T07:54:24.090" LastActivityDate="2020-04-28T18:03:55.397" Title="How to compute 16 different simulations on parallel with pbs script on the same machine" Tags="|python|parallel-computing|pbs|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31361" PostTypeId="2" ParentId="31349" CreationDate="2019-04-04T02:09:07.780" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;It is true that &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; depends on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;. This implies that if your function has a very poorly behaved second derivative at &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, this method will be inaccurate. The dependance on &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is also to be expected, since finite differences are based on Taylor series, which converge locally, so our error estimate essentially depends on how well a Taylor series works out to &lt;span class=&quot;math-container&quot;&gt;$x+h$&lt;/span&gt;, which is where that dependency comes from. To your second part of the question, as &lt;span class=&quot;math-container&quot;&gt;$h\to0$&lt;/span&gt;, we have that if &lt;span class=&quot;math-container&quot;&gt;$\xi_h\in[x,x_h]$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\forall h$&lt;/span&gt;, then &lt;span class=&quot;math-container&quot;&gt;$\xi_h\to x$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$C^2$&lt;/span&gt;, then we can say that &lt;span class=&quot;math-container&quot;&gt;$u''(\xi)\to u''(x)$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$h\to 0$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For this, this really is the best you can do by simply squaring the forward difference derivative. Try it for yourself on something like &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt; vs. &lt;span class=&quot;math-container&quot;&gt;$1000x^2$&lt;/span&gt; at &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt; vs. &lt;span class=&quot;math-container&quot;&gt;$x=1,10,20$&lt;/span&gt;. At &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f'(x)=0$&lt;/span&gt; so you actually get second order convergence. As you increase &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, your will get first order convergence at each &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, but the prefactor of &lt;span class=&quot;math-container&quot;&gt;$f'(x)$&lt;/span&gt; gets larger and the results will be worse and worse. &lt;span class=&quot;math-container&quot;&gt;$1000x^2$&lt;/span&gt; will also do worse at every point because the derivative and second derivative are larger.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29512" LastActivityDate="2019-04-04T02:09:07.780" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31364" PostTypeId="2" ParentId="31360" CreationDate="2019-04-04T08:27:39.160" Score="0" Body="&lt;p&gt;There are a few options, but these two come to mind as the most straight-forward:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could submit 16 different jobs to your scheduler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could modify your script to start your programs in the background:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_dynamic_program1.x &amp;amp;&#xA;my_dynamic_program2.x &amp;amp;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(the &lt;code&gt;&amp;amp;&lt;/code&gt; says: start running this command and immediately run the next one as well)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could even do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd dir1&#xA;./prog1.exe &amp;amp;&#xA;cd dir2&#xA;./prog2.exe &amp;amp;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2019-04-04T08:27:39.160" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31365" PostTypeId="2" ParentId="14818" CreationDate="2019-04-04T11:00:26.400" Score="3" Body="&lt;p&gt;I found the &lt;a href=&quot;https://cran.r-project.org/web/packages/gramEvol/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;gramEvol&lt;/a&gt; R package flexible and easy to use.&#xA;They have a &lt;a href=&quot;https://cran.r-project.org/web/packages/gramEvol/vignettes/ge-intro.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;small tutorial&lt;/a&gt; in which they rederive Kepler's third law from data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that it relies on Genetic Programmic for its optimisation and thus might return different results if you run it twice.&lt;/p&gt;&#xA;" OwnerUserId="30380" LastActivityDate="2019-04-04T11:00:26.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31370" PostTypeId="2" ParentId="31360" CreationDate="2019-04-04T14:56:22.987" Score="1" Body="&lt;p&gt;If you system does not have a job launcher facility (or if PBS does not have that built in) you can do a couple of things. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Start all 16 processes, and hope that the OS will nicely spread them over the cores.&lt;/li&gt;&#xA;&lt;li&gt;Start all 16 processes, but prefix them with numactl to fix them to a certain core.&lt;/li&gt;&#xA;&lt;li&gt;Use mpirun but with the following script:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;mpi -np 16 parallel_script&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where your script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;if [ $PMI_RANK -eq 0 ] ; then &#xA;  ./program1&#xA;else if [ $PMI_RANK -eq 1 ] ; then&#xA;  ./program2&#xA;et cetera.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The variable &lt;code&gt;PMI_RANK&lt;/code&gt; may be system-dependent.&lt;/p&gt;&#xA;" OwnerUserId="4087" LastActivityDate="2019-04-04T14:56:22.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31373" PostTypeId="1" CreationDate="2019-04-04T18:54:27.503" Score="1" ViewCount="68" Body="&lt;p&gt;I currently have 2 different functions with options to vectorise them: &lt;code&gt;acc_rej_sine(max_iter, algorithm=None)&lt;/code&gt; and &lt;code&gt;analytical_sine(max_iter, algorithm=None)&lt;/code&gt; for which I'm trying to compare their run time against the number of iterations computed. i.e. compare all 4 methods; 2 looped, 2 vectorised. Essentially my code goes something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def analytical_sine(max_iter, algorithm=None):&#xA;    if algorithm is None:&#xA;        count = 0&#xA;        analytical_hist = []&#xA;        for i in range(max_iter):&#xA;            count += 1&#xA;            progress = round((count/max_iter)*100)&#xA;            sys.stdout.write('\r' + str(progress) + '%')&#xA;            uni_dist = np.random.uniform(0, 1)&#xA;            arccos = np.arccos(1 - 2*uni_dist)&#xA;            analytical_hist.append(arccos)&#xA;    elif algorithm is &quot;vectorise&quot;:&#xA;        analytical_hist = np.arccos(1 - 2*np.random.uniform(0, 1, max_iter))&#xA;&#xA;    return analytical_hist&#xA;&#xA;def acc_rej_sine(max_iter, algorithm=None):&#xA;    x = np.random.uniform(0, np.pi, max_iter)&#xA;    y = np.random.rand(max_iter)&#xA;    if algorithm is None:&#xA;        accepted_x = []&#xA;        j = count = 0&#xA;        for i in range(max_iter):&#xA;            count += 1&#xA;            progress = round((count/max_iter)*100)&#xA;            sys.stdout.write('\r' + str(progress) + '%')&#xA;            if y[i] &amp;lt;= np.sin(x[i]):&#xA;                accepted_x.append(x[i])&#xA;                j += 1&#xA;    elif algorithm is &quot;vectorise&quot;:&#xA;        accepted_x = np.extract((y &amp;lt;= np.sin(x)), x)&#xA;&#xA;    return accepted_x&#xA;&#xA;def runtime(func, runs, max_iter, algorithm=None):&#xA;    time = []&#xA;    for i in range(runs):&#xA;        start = timer()&#xA;        func()&#xA;        end = timer()&#xA;        time.append((end-start))&#xA;    error = np.std(time)&#xA;    time = sum(time)/runs&#xA;&#xA;    return time, error&#xA;&#xA;def time_analysis():&#xA;    time1, time2, time3, time4 = [], [], [], []&#xA;    error1, error2, error3, error4 = [], [], [], []&#xA;    for i in np.arange(1, 8, 1):&#xA;        max_iter = 10**i&#xA;        time, error = runtime(analytical_sine, 5, int(max_iter))&#xA;        time1.append(time)&#xA;        error1.append(error)&#xA;        time, error = runtime(analytical_sine, 5, int(max_iter), &quot;vectorise&quot;)&#xA;        time2.append(time)&#xA;        error2.append(error)&#xA;        time, error = runtime(acc_rej_sine, 5, int(max_iter))&#xA;        time3.append(time)&#xA;        error3.append(error)&#xA;        time, error = runtime(acc_rej_sine(max_iter), 5, int(max_iter), &quot;vectorise&quot;)&#xA;        time4.append(time)&#xA;        error4.append(error)&#xA;    return [time1, time2, time3, time4], [error1, error2, error3, error4]&#xA;&#xA;# to run the code I would probably do something like this&#xA;time, error = time_analysis()&#xA;#then if I wanna plot number of iterations vs run time with errors I would probably do something along the lines of&#xA;plt.plot(max_iter, time) # max_iter would be a list of [10**i for i in np.arange(1, 8, 1)]&#xA;plt.errorbar(error)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the idea is my &lt;code&gt;runtime()&lt;/code&gt; function would allow me to put in any of the 4 functions that I want to compare(which currently still isn't working yet and I can't/haven't figured out why), and run it for 5 times, work out the mean run time and standard deviation as the error. In return my &lt;code&gt;time_analysis()&lt;/code&gt; function would run &lt;code&gt;runtime()&lt;/code&gt; for different functions for different &lt;code&gt;max_iter&lt;/code&gt;(max iterations) which goes like &lt;code&gt;[10, 1E2, 1E3, 1E4, 1E5, 1E6, 1E7]&lt;/code&gt; so I can plot max iterations against run time. However, this whole method seems quite cumbersome and inelegant, as my &lt;code&gt;time_analysis()&lt;/code&gt; requires me to repeatedly work out time and error and append it to a list. Is there a better way of timing this?(Also my &lt;code&gt;runtime()&lt;/code&gt; doesn't work yet lol because my &lt;code&gt;algorithm&lt;/code&gt; argument seems to be making something not &lt;code&gt;callable&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="30385" LastActivityDate="2019-04-10T07:15:43.003" Title="Is there a better way to do run time analysis than this?" Tags="|python|numerics|data-analysis|random-number-generation|random-sampling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31374" PostTypeId="2" ParentId="20901" CreationDate="2019-04-04T19:46:18.247" Score="4" Body="&lt;p&gt;I think that there is a bit of confusion with change of variables in some of the previous answers as well as some errors. The integral of a log function is not the log of the integral. I think in general is difficult to write out the the integral of a function knowing the integral of its log. If anyone knows how to do that I would be interested.&lt;/p&gt;&#xA;&lt;p&gt;In the meanwhile, @Stefan's solution above is the way to get around integrating a function in log-log space. The starting point is that the function you're dealing is linear in log-log space for small enough segments.&lt;/p&gt;&#xA;&lt;p&gt;One can then write the equation of the line at the segment endpoints:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TNc0V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TNc0V.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\log(y_1)=m_1 \log(x_1) + n_1$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\log(y_2)=m_1 \log(x_2) + n_1$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$m_1$&lt;/span&gt; is the slope of the line and &lt;span class=&quot;math-container&quot;&gt;$n_1$&lt;/span&gt; is its &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;-intercept.&lt;/p&gt;&#xA;&lt;p&gt;By subtracting the two, one can find:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$m_1=\frac{\log(y_1) -\log(y_2)}{\log(x_1)-\log(x_2)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and from substitution:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$n_1=\log(y_1)-m_1 \log(x_1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If in the log-log space the equation of a segment is close to a line then in normal (linear) space the equation of the segment is close to an exponential:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$y(x)\simeq x^{m} e^{n}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If we have a analytical formulation for this segment it is easy to integrate:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_{x_1}^{x_2}y(x)dx = \frac{e^{n_1}}{m_1+1}(x_2^{m_1+1}-x_1^{m_1+1}), \,\, \text{for } \, m\neq -1$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\int_{x_1}^{x_2}y(x)dx = e^{n_1} \log \frac{x_2}{x_1},\, \,\text{for } \, m = -1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This feels a bit like cheating, but this is sampling in log-log space such that we can approximate the function in the linear space to an exponential with parameters derived from the log-log space.&lt;/p&gt;&#xA;" OwnerUserId="30382" LastEditorUserId="-1" LastEditDate="2022-08-04T15:34:19.957" LastActivityDate="2022-08-04T15:34:19.957" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31375" PostTypeId="1" AcceptedAnswerId="31381" CreationDate="2019-04-05T06:36:25.673" Score="1" ViewCount="244" Body="&lt;p&gt;Consider the advection equation (1D in space)&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial u}{\partial t} +  V\, \frac{\partial u}{\partial x}=0&#xA;$$&lt;/span&gt;&#xA;and we solve it numerically on &lt;span class=&quot;math-container&quot;&gt;$[0,1]\times [0,1]\ni (t,x)$&lt;/span&gt; using a forward scheme in time and a backward decentered in space.&#xA;This scheme is of order &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in space and of order &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in time and is stable iff &lt;span class=&quot;math-container&quot;&gt;$V\frac{dt}{dx}\le 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; be the number of points in the subdivision of &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt; be the time step.&#xA;Numerically, to observe the order, if I fix &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt; is varying the error between the analyitc solution and the numerical solution is represented in the log-plot figure:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red curve is the linear regression between the points, but useless here because we don't observe a straight line so the order can be checked. More importantly when &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt; is decreasing, the error is increasing ...&#xA;&lt;strong&gt;What is the reason for that ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;problem in the implementation ? maybe&lt;/li&gt;&#xA;&lt;li&gt;the fact is when we say of order &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in space and time it means we approximate the equation by an error which can be written:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ C_1\, dt + C_2\, dx$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt; are constant.&#xA;But &lt;span class=&quot;math-container&quot;&gt;$C_1$&lt;/span&gt; may depend on &lt;span class=&quot;math-container&quot;&gt;$dx$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt; may depend on &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt;. So when &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt; is decreasing, &lt;span class=&quot;math-container&quot;&gt;$C_2$&lt;/span&gt; is not constant and the error can increase, which could explain the plot.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GdtGP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GdtGP.png&quot; alt=&quot;(blue) Error when dt varies and N is fixed&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that when the ratio &lt;span class=&quot;math-container&quot;&gt;$V\frac{dt}{dx}$&lt;/span&gt; is constant equal to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dt,dx \to 0$&lt;/span&gt;, the error is constant and very small (no diffusion)&lt;/p&gt;&#xA;" OwnerUserId="29701" LastActivityDate="2019-04-06T18:56:32.477" Title="Error for the finite differences scheme -- Advection equation" Tags="|finite-difference|advection-diffusion|parabolic-pde|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31378" PostTypeId="1" CreationDate="2019-04-05T20:17:00.400" Score="1" ViewCount="1684" Body="&lt;p&gt;I am trying to implement a routine to solve a differential equation in Python. Basically the kind of equation that I am interested in solving is of the form:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\displaystyle \frac{d}{dx^2} \left(x y(x) \right) = 2 x ((U(x)-a) y(x)+ 2 b y(x)^3)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is an unknown constant and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is a known constant and &lt;span class=&quot;math-container&quot;&gt;$U(x)$&lt;/span&gt; is a function that depends on &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; but that I only know numerically (I mean, I don't have an explicit form of &lt;span class=&quot;math-container&quot;&gt;$U(x)$&lt;/span&gt; in terms of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I need to find the value of &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; that fulfills my initial and boundary conditions (&lt;span class=&quot;math-container&quot;&gt;$y(0)=y_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y(x\rightarrow \infty)=0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$a&amp;lt;1$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;For that purpose, I was considering using a shooting method (a secant method to be precise) by solving several times the above equation with RK4 (using &lt;code&gt;scipy.integrate.ode&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The problem that I have is that I don't know how to introduce the numerical value of &lt;span class=&quot;math-container&quot;&gt;$U(x)$&lt;/span&gt; in my equations given the fact that ode only asks for values at the initial conditions.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to solve my equation with &lt;code&gt;scipy.integrate.ode&lt;/code&gt; or with another solver?&lt;/p&gt;&#xA;" OwnerUserId="28010" LastEditorUserId="20688" LastEditDate="2022-01-03T06:48:41.007" LastActivityDate="2022-01-03T06:48:41.007" Title="Solving differential equation in Python with variable coefficients (I just know the coefficients numerically)" Tags="|python|ode|differential-equations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31380" PostTypeId="1" AcceptedAnswerId="31382" CreationDate="2019-04-06T01:13:59.917" Score="2" ViewCount="1055" Body="&lt;p&gt;I'm looking for current references on solving stiff ODEs. Most of what I know (say, BDF methods) apparently date back to the 1980's, and I feel like a lot of progress should have been made in that time, but I'm having trouble finding the &quot;right&quot; search terms to get what I need. What are the (current) best methods for solving stiff IVPs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Note: I'm not looking for software, but rather the methods themselves. I've seen a lot of people around the web recommending to &quot;just use MATLAB&quot;, but I need something for a very specific application that MATLAB would not be able to handle anyway.)&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="25718" LastActivityDate="2020-07-19T17:49:49.190" Title="What is the state of the art in solving stiff initial value problems?" Tags="|ode|reference-request|stiffness|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31381" PostTypeId="2" ParentId="31375" CreationDate="2019-04-06T12:23:00.643" Score="1" Body="&lt;p&gt;Let's define&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\lambda = V \frac{dt}{dx}.&#xA;$$&lt;/span&gt;&#xA;Can the error increase if you reduce &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt; but not &lt;span class=&quot;math-container&quot;&gt;$dx$&lt;/span&gt;, i.e. if &lt;span class=&quot;math-container&quot;&gt;$\lambda &amp;lt; 1$&lt;/span&gt;? It seems so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the initial data is smooth, the solution can be expanded into a Fourier series. We can investigate what happens to each component of the series by assigning the solution&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u = \exp{i(\tilde{\kappa} x - \tilde{\omega}t)},&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\tilde{\omega}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\tilde{\kappa}$&lt;/span&gt; denote a generic frequency and wavenumber respectively. This is a useful ansatz since the true solution will be a (probably infinite) sum of such Fourier modes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inserting &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; into the PDE, the result is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(-\tilde{\omega} + V \tilde{\kappa}) u = 0 \quad \Rightarrow \quad \tilde{\omega} = V \tilde{\kappa}.&#xA;$$&lt;/span&gt;&#xA;This expression is the dispersion relation for the advection equation and tells us the relation between the frequency and wavenumber of any Fourier mode of the solution. This relation tells us that a Fourier mode will retain its amplitude for all time (&lt;span class=&quot;math-container&quot;&gt;$\tilde{\omega}$&lt;/span&gt; is real if &lt;span class=&quot;math-container&quot;&gt;$\tilde{\kappa}$&lt;/span&gt; is real). It also tells us that the speed of each Fourier mode is &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; (the wave speed is given by &lt;span class=&quot;math-container&quot;&gt;$\tilde{\omega} / \tilde{\kappa}$&lt;/span&gt;). This is enough to characterise the behaviour of the exact solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, after discretisation there are two qualities of the numerical solution that will keep it accurate: correct amplitude (no growth or decay of the numerical solution) and correct wavespeed (it won't drift out of phase), both of which are encoded in the &lt;em&gt;numerical&lt;/em&gt; dispersion relation. The numerical scheme is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{u_j^{n+1} - u_j^n}{dt} + V \frac{u_j^n - u_{j-1}^n}{dx} = 0.&#xA;$$&lt;/span&gt;&#xA;We may derive the numerical dispersion relation much like we did for the PDE: Assign a solution&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;u_j^n = \exp{i(\kappa x_j - \omega t_n)},&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; are the numeric frequency and wavenumber respectively. Inserting &lt;span class=&quot;math-container&quot;&gt;$u_j^n$&lt;/span&gt; into the discretisation gives&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left( \frac{e^{-i\omega dt} - 1}{dt} + V \frac{1 - e^{-i\kappa dx}}{dx} \right) u_j^n = 0,&#xA;$$&lt;/span&gt;&#xA;which holds for any Fourier mode. Using &lt;span class=&quot;math-container&quot;&gt;$dt = \lambda dx / V$&lt;/span&gt; we obtain the discrete dispersion relation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{e^{-i \omega \lambda dx / V} - 1}{\lambda} + 1 - e^{-i \kappa dx} = 0.&#xA;$$&lt;/span&gt;&#xA;Now, if &lt;span class=&quot;math-container&quot;&gt;$\lambda = 1$&lt;/span&gt;, this reduces to&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;e^{-i(\omega dx / V)} = e^{-i(\kappa dx)} \quad \Rightarrow \quad \omega = V \kappa,&#xA;$$&lt;/span&gt;&#xA;which is exactly the same dispersion relation as we had for the PDE. In other words, the numerical solution will preserve its initial amplitude and it will travel with the wavespeed &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;. This is as close to a perfect numerical solution as we can hope for since there is not much else that can cause error for the linear advection equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;span class=&quot;math-container&quot;&gt;$\lambda \neq 1$&lt;/span&gt; we will get a different numerical dispersion error. I'll leave the calculation to you, but (ignoring the imaginary part) a Maclaurin expansion seems to give something like&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\omega = \frac{V \kappa}{\sqrt{\lambda}} + \mathcal{O}(dx).&#xA;$$&lt;/span&gt;&#xA;The point is that choosing &lt;span class=&quot;math-container&quot;&gt;$\lambda &amp;lt; 1$&lt;/span&gt;, i.e. &lt;span class=&quot;math-container&quot;&gt;$dt &amp;lt; dx / V$&lt;/span&gt; is detrimental to the approximation quality of the numerical dispersion relation. The smaller &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is, the worse the approximation becomes, leading to larger errors in amplitude and phase. This may explain your observations.&lt;/p&gt;&#xA;" OwnerUserId="19512" LastEditorUserId="19512" LastEditDate="2019-04-06T18:56:32.477" LastActivityDate="2019-04-06T18:56:32.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31382" PostTypeId="2" ParentId="31380" CreationDate="2019-04-06T12:33:13.693" Score="22" Body="&lt;p&gt;So there is a ton to say about this, and we will actually be putting a paper out that tries to summarize it a bit, but let me narrow it down to something that can be put into a quick StackOverflow post. I will make one statement really early and keep repeating it: you cannot untangle the efficiency of a method from the efficiency of a software. The details of software implementations are what actually matter in this field.&lt;/p&gt;&#xA;&lt;p&gt;Essentially, the state of the art methods completely depend on the problem that you're looking at. &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffEqBenchmarks.jl&lt;/a&gt; has quite a few benchmarks of some of the latest software (includes DifferentialEquations.jl, SUNDIALS, Hairer's FORTRAN stuff, Shampine's FORTRAN stuff, LSODA, etc.). It's based on Hairer's benchmarks from &lt;a href=&quot;https://www.springer.com/us/book/9783540604525&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hairer II&lt;/a&gt; but the DiffEqBenchmarks are more up-to-date with the software, and explores many of the methods mentioned here more thoroughly (while still including Hairer's FORTRAN methods). The full list of algorithms that are available to benchmark through this can be found &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the DifferentialEquations.jl documentation&lt;/a&gt;, which covers most known algorithms at this point (and getting more complete after this summer). Not all of the methods in the documentation are showing up in every plot, but you can download the benchmarks and add/subtract things plus change around options.&lt;/p&gt;&#xA;&lt;p&gt;Although you want to decouple &amp;quot;software&amp;quot; from the &amp;quot;methods&amp;quot;, the actual software implementation of many details matters as much if not more than the actual method that is used. So you can see in many of the benchmarks on stiff problems that the three different BDF implementations (ddebdf in Fortran, LSODA, CVODE_BDF) have very different efficiencies on different problems given how they choose time steps, re-use Jacobians, etc. Those more &amp;quot;engineering&amp;quot; aspects (especially CVODE's Jacobian re-use structure) is what makes a software efficient or inefficient. And as you can see from the benchmarks, changing around options matters a ton as well. For example, SUNDIALS ARKODE fails to solve most of the stiff benchmarks we threw at it without changing some options (This is noted in the SUNDIALS examples as well, so it's not just an error on our end)! So keep that in mind while talking about methods and whenever reading a paper that talks about the efficiency.&lt;/p&gt;&#xA;&lt;p&gt;Given all of those caveats, high order Rosenbrock methods seem to dominate when you have sufficiently small systems because they can take large time steps and don't require the stability of Newton iterations (which places the real stability bound on implicit methods). While previous literature mentioned that Jacobian accuracy issues would prevent Rosenbrock methods from converging well at high order, modern automatic differentiation techniques like those in &lt;a href=&quot;https://github.com/JuliaDiff/ForwardDiff.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;ForwardDiff.jl&lt;/a&gt; circumvent a lot of the traditional issues with Jacobian accuracy of numerical differentiation which has breathed new life into this field (and is one of the reasons why Julia is being used in this domain). Parallelization of BLAS seems to only get in the way at this size (Jacobian &amp;lt;50x50, see, engineering details matter!), so I suspect there is some gains to be had looking at parallel Rosenbrock methods (which allow for multiple lu-factorizations to be done in parallel by creating a DIRK-like Rosenbrock), there aren't any real software implementations to test this idea.We will have software coming out soon enough to start benchmarking with, but the literature has only created methods for dual core parallel Rosenbrock methods (that I know of), so methods specialized for modern 4-8 core CPUs of order 4 or 5 would be an interesting topic. Parallelized impicit extrapolation methods could also be an interesting method in this domain.&lt;/p&gt;&#xA;&lt;p&gt;One caveat I'll throw in here is that, as you ask for lower and lower tolerances (say around &lt;code&gt;1e-8&lt;/code&gt;), you'll always see Hairer's Radau implementation become the most efficient. It doesn't scale well to larger ODE sizes either, since it uses a Jacobian that is massively larger than the other methods as a FIRK method, but its higher order does become a factor that leads to impressive efficiency when high accuracy is required.&lt;/p&gt;&#xA;&lt;p&gt;As the system size increasing to &amp;gt;100 ODEs, other factors start asymptotically mattering more. Essentially, the cost of computing a Jacobian and factorizing the W matrix &lt;span class=&quot;math-container&quot;&gt;$W = I - \gamma f'$&lt;/span&gt; for some &lt;span class=&quot;math-container&quot;&gt;$\gamma \in \mathbb{R}$&lt;/span&gt; become the dominating factors in any semi-implicit or implicit method (so any implicit RK, implicit multistep, Rosenbrock). As this occurs, the ability to re-use a Jacobian that you've already computed, or re-use a factorized Jacobian you've already created, becomes very important. Thus, since the accuracy of a standard Rosenbrock method depends on the accuracy of its Jacobian, Rosenbrock methods begin to lag behind because they need to compute and invert a new Jacobian every step. Implicit methods do not have this issue since the Jacobian is only used as a line search for solving the implicit equations, and thus does not actually effect accuracy. &amp;quot;Bad Jacobians&amp;quot; do cause more linear solves to have to be done in order for the Newton method to converge, but a linear solve with a pre-factorized matrix is asymptotically trivial in comparison to the Jacobian calculation or the inversion. Backsubstitution is &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt;, Jacobian calculations are &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt;, while inversion is &lt;span class=&quot;math-container&quot;&gt;$O(n^3)$&lt;/span&gt;. That fact makes people generally consider only the latter to be an issue, actual timing and profiling proves that is an oversimplification since calculating the Jacobian requires performing the calculations in &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, the ODE derivative function, while the inversion is pure linear algebra (matrix coloring and analytical forms of sparse Jacobians reduce the complexity, but still require the extra calculations of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; in some form). Thus at this stage you have to segment the problems a little bit more based on how costly it is to compute &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;, since even if &lt;span class=&quot;math-container&quot;&gt;$N=100$&lt;/span&gt; you could be given a really really slow &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; calculation so &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; calls may dominate time over the matrix inversions (this is generally the case in MATLAB or Python ODE programs).&lt;/p&gt;&#xA;&lt;p&gt;So let's assume that you have an &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;-dominated semi-large ODE system, where we define semi-large to be a size that is large enough to care about these more asymptotic factors but small enough that dense/sparse factorizations still fit into memory. What we have found works well in these cases are things like SDIRK methods since they can stably step (L-stability and B-stability) with much higher order than BDF while re-using Jacobians to some extent. What happens is this minimizes &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; calculations (though currently does require a few more inversions, something we are investigating, again an engineering &amp;quot;post-math&amp;quot; challenge) because BDF becomes efficient by doing a lot of Jacobian re-use and slamming really small steps (ha, bet you didn't see that coming) with the same inverted &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; for multiple steps in a row. We need to formalize this into DiffEqBenchmarks.jl some more, but one good source on this is &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/pull/698#issuecomment-472199138&quot; rel=&quot;nofollow noreferrer&quot;&gt;this plot from a PR&lt;/a&gt; which is looking at a &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/issues/664&quot; rel=&quot;nofollow noreferrer&quot;&gt;stiff 1000 ODE model with costly f evals&lt;/a&gt;. In this case, &lt;code&gt;KenCarp4&lt;/code&gt; seems to be one of the better methods, though we are investigating alternative (E)SDIRK tableaus but haven't found a better one yet (none of the 5th order seem to do as well, a fact noted by Kennedy and Carpenter in their paper on ESDIRK methods as well). An alternative strategy which may prove useful here are RosW methods, which are Rosenbrock methods that do not lose order when the Jacobian is inaccurate. Comprehensive testing of these methods in this domain is set to occur during this summer. A lot of the efficiency will be dependent on developing efficient Jacobian re-use strategies though.&lt;/p&gt;&#xA;&lt;p&gt;When finally getting to the domain that is actually dominated by matrix inversions, BDF seems to do well for weirder reasons than one might expect. Not BDF, but specifically the VODE and CVODE line of implementations. The reason is because they take very small steps in comparison to other stiff solvers, in a way that constrains their &lt;code&gt;dt&lt;/code&gt; changes, which helps keep the &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; matrix the same (&lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is always proportional to &lt;code&gt;dt&lt;/code&gt;), and decreases the number of inversions which are required. This coupled with their special Fixed Leading Coefficient (FLC) form lets them re-use already inverted matrices as much as possible. However, alternative strategies here, such as relaxed Newton iterations, may prove successful. This is entirely engineering-dependent, dependent on how adaptivity is handled and smoothing that out.&lt;/p&gt;&#xA;&lt;p&gt;The next stage is when you get to ODEs which are large enough that the sparse factorizations can no longer fit into memory. At this point, you pretty much have to change your nonlinear solver strategy. There are two methods which are being investigated. One is to do Anderson Acceleration. The other is Newton-Krylov methods. Both of these methods cannot just keep something factorized to make proceeding stages in the same step trivial, which means that multi-staged methods (Rosenbrock, ESDIRK, etc.) now have a lot more nonlinear solving time that a single stage method since they have to perform the entire nonlinear solving from scratch every stage. That said, there are one again some engineering factors that get in there. For example, when you implement nonlinear solvers for something like an ESDIRK method, you can use the previous stages to do an extrapolation for the starting value of the nonlinear iterations in the next stage. In many cases, the &lt;span class=&quot;math-container&quot;&gt;$c_i$&lt;/span&gt; of the tableau are not monotonic, which in turn means that it's not an extrapolation but a (low order) interpolation, meaning that the nonlinear iteration at some stages converge in one or two goes. So it's not that straightforward to say that multi-staged methods are disadvantaged here without mentioning the accuracy of your stage predictors (which is the term used for this process), so again something &amp;quot;outside the method&amp;quot; and part of the software processes is influencing the practical results. But, in this domain BDF does seem to shine because it only has one stage. However, the fact that it uses such small steps is somewhat of a concern (lack of L-stability above order 2, and the lack of optimized leading truncation error coefficients gives this property) and does lead to quite a few linear solves, so I wouldn't be surprised if well-designed and well-implemented ESDIRKs can topple that sooner or later.&lt;/p&gt;&#xA;&lt;p&gt;This focused on the methods which have been the most benchmarked, but alternatively the EPRIK and exponential Rosenbrock methods are an entirely alternative branch to explore. There are some good results that have been reported in the literature, and DifferentialEquations.jl does have the full Krylov-expmv + Krylov adaptivity &lt;a href=&quot;https://github.com/JuliaDiffEq/ExponentialUtilities.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;setup in ExponentialUtilities.jl&lt;/a&gt; and implemented in &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html#Exponential-Propagation-Iterative-Runge-Kutta-Methods-(EPIRK)-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;5th order EPIRK methods&lt;/a&gt; and so if you want to join our benchmarking team we would love to see how these are turning out. And of course, since engineering is so important, these are quite new and will likely need a few rounds of profiling. &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl/blob/308d34d328af6a66047a75ec050a6cd493098943/StiffODE/Burgers.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some preliminary results are exciting&lt;/a&gt; (what's interesting is that while ARKODE fails on a lot of ODE benchmarks, it's quite good on discretized PDE benchmarks).&lt;/p&gt;&#xA;&lt;p&gt;What these benchmarks are also highlighting is that one of the ways to improve stiff solvers in 2019 is to not solve it as a single &lt;code&gt;f&lt;/code&gt;, but as a split &lt;code&gt;f = f1 + f2&lt;/code&gt; and treat only one part implicitly via IMEX integrators. IMEX BDF (called SBDF), IMEX ESDIRK, IMEX Rosenbrock, etc. all exist and are interesting routes that need more testing.&lt;/p&gt;&#xA;&lt;p&gt;Also, if you only have &amp;quot;semi-stiffness&amp;quot;, then stabilized explicit methods are explicit methods with enough adaptive stability to handle some stiff problems. &lt;a href=&quot;http://diffeq.sciml.ai/latest/solvers/ode_solve.html#Stabilized-Explicit-Methods-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;The implementations exist&lt;/a&gt; but need benchmarks. We are &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl/pull/32&quot; rel=&quot;nofollow noreferrer&quot;&gt;currently undergoing a benchmark overhaul to update to use Weave.jl files (for auto-running) and update all benchmarks to Julia v1.0&lt;/a&gt;, so stabilized RK, RosW, exponential integrators, and others will be joining the benchmarks once this is done. Our benchmarks also are missing the efficiency when GPUs and TPUs are integrated into the software: the native Julia methods are close to being able to use GPUs so it'll be interesting to test what happens when we make use of it. That puts a higher stress on memory but a lower stress on the cost of linear algebra routines, maybe bumping things more towards the &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;-dominated domain.&lt;/p&gt;&#xA;&lt;p&gt;Because this is a gigantic topic, there is a chance I also missed one of the infinite options you specifically wanted more discussion on. Oops I'm sorry but it would take &lt;span class=&quot;math-container&quot;&gt;$O(n^5)$&lt;/span&gt; time to review all methods. Ask again in 5 years and we'll have more benchmarks :).&lt;/p&gt;&#xA;&lt;p&gt;So in total, you wanted a short summary about the methods and not the software, but since you asked about what's actually state-of-the-art I had to respond with all of the different ways that the results are actually tied to the software. However, these are a bunch of different directions that seem to be fruitful in different domains, and the JuliaDiffEq team is continuing to implement, optimize, and benchmark. So any time now some of these results may be changing, and that's just how it is since asymptotic factors and pure mathematical constraints only matter at infinity. If you want to know more about the field, summaries like this can be helpful but you will never get a good idea until you get your hands dirty and start benchmarking and profiling. We'd be happy to have you submit problems from your domain to our benchmarks and see what does well!&lt;/p&gt;&#xA;&lt;p&gt;P.S. To track down the papers on the methods, see &lt;a href=&quot;https://github.com/JuliaDiffEq/DiffEqJump.jl/issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;our citing page&lt;/a&gt;. That's not completely up to date, so you may &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/issues/394&quot; rel=&quot;nofollow noreferrer&quot;&gt;have to dive into our issues in some cases&lt;/a&gt;. A &lt;a href=&quot;https://github.com/JuliaDiffEq/juliadiffeq.github.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;PR to improve our citations page&lt;/a&gt; is always welcome of course.&lt;/p&gt;&#xA;&lt;p&gt;Edit: Yingbo Ma shared this nice little script that highlights the stepping behavior difference of BDF vs the other methods:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;using OrdinaryDiffEq, Plots, ParameterizedFunctions, Sundials, ODEInterfaceDiffEq&#xA;&#xA;hires = @ode_def Hires begin&#xA;  dy1 = -1.71*y1 + 0.43*y2 + 8.32*y3 + 0.0007&#xA;  dy2 = 1.71*y1 - 8.75*y2&#xA;  dy3 = -10.03*y3 + 0.43*y4 + 0.035*y5&#xA;  dy4 = 8.32*y2 + 1.71*y3 - 1.12*y4&#xA;  dy5 = -1.745*y5 + 0.43*y6 + 0.43*y7&#xA;  dy6 = -280.0*y6*y8 + 0.69*y4 + 1.71*y5 -&#xA;           0.43*y6 + 0.69*y7&#xA;  dy7 = 280.0*y6*y8 - 1.81*y7&#xA;  dy8 = -280.0*y6*y8 + 1.81*y7&#xA;end&#xA;&#xA;u0 = zeros(8)&#xA;u0[1] = 1&#xA;u0[8] = 0.0057&#xA;prob = ODEProblem(hires,u0,(0.0,321.8122))&#xA;function plotdts(sols::Tuple, title; kwargs...)&#xA;    ts = map(s-&amp;gt;s.second.t, sols)&#xA;    dts = map(t-&amp;gt;[0; diff(t)], ts)&#xA;    xlim = max(map(maximum, ts)...)&#xA;    plt = plot(title=title, xlabel=&amp;quot;t&amp;quot;, ylabel=&amp;quot;dt&amp;quot;, ylims=(0, max(map(maximum, dts)...)*1.3), xlims=(0, xlim*1.5); kwargs...)&#xA;    vline!(plt, [xlim], lab=&amp;quot;t1&amp;quot;)&#xA;    for i in 1:length(ts)&#xA;        plot!(plt, ts[i], dts[i], lab=sols[i].first, marker=(2, ))&#xA;    end&#xA;    plt&#xA;end&#xA;&#xA;plotdts((&amp;quot;CVODE_BDF&amp;quot;=&amp;gt;solve(prob, CVODE_BDF()),&#xA;         &amp;quot;KenCarp4&amp;quot;=&amp;gt;solve(prob, KenCarp4()),&#xA;         &amp;quot;radau5&amp;quot;=&amp;gt;solve(prob, radau5())), &amp;quot;Time vs Step Size&amp;quot;, dpi=200)&#xA;savefig(&amp;quot;$(homedir())/tvsdt.png&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/r9gtc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r9gtc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18981" LastEditorUserId="18981" LastEditDate="2020-07-19T17:49:49.190" LastActivityDate="2020-07-19T17:49:49.190" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="31383" PostTypeId="1" CreationDate="2019-04-06T12:44:04.830" Score="0" ViewCount="174" Body="&lt;p&gt;I was trying to implement the algorithm from the paper &lt;a href=&quot;https://www.researchgate.net/publication/282588186_Adapting_a_Fourier_pseudospectral_method_to_Dirichlet_boundary_conditions_for_Rayleigh-Benard_convection&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Adapting a Fourier pseudospectral method to Dirichlet boundary conditions for Rayleigh–Benard convection&quot;.&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having a hard time to understand the way the boundary conditions are imposed.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The author rewrites the no-slip (on upper and lower boundaries) boundary conditions as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \sum_{q} \tilde f_{\bot,pq} = 0   \forall p$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; are horizontal and vertical wavenumbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do we impose that?&lt;/p&gt;&#xA;" OwnerUserId="30400" LastEditorUserId="20688" LastEditDate="2019-04-07T06:09:49.150" LastActivityDate="2019-05-11T14:00:56.617" Title="Imposition of Dirichlet BC for Fourier pseudospectral in this paper" Tags="|fluid-dynamics|boundary-conditions|spectral-method|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31387" PostTypeId="1" AcceptedAnswerId="31389" CreationDate="2019-04-07T07:46:59.153" Score="5" ViewCount="844" Body="&lt;p&gt;I have heard of space-time finite element methods. Although I was able to find some articles that describe the different possible methods from a mathematical point of view (thanks to &lt;a href=&quot;https://scicomp.stackexchange.com/questions/19355/space-time-finite-element-discretization-for-time-dependent-pdes/19361#19361&quot;&gt;Space-time finite element discretization for time-dependent PDEs&lt;/a&gt; for instance), I would like to know with more detail what are the main advantages (and disadvantages also) of such methods.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For instance, what are the improvements obtained when used in an elastodynamics problem, a fluid dynamics problem, or in a fluid-structure interaction problem over a traditional method of lines?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What can be achieved that could not be achieved before?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="27849" LastEditorUserId="9667" LastEditDate="2019-04-07T16:54:10.697" LastActivityDate="2019-04-07T18:06:32.530" Title="Advantages and disadvantages of space-time finite element methods" Tags="|finite-element|space-time-galerkin|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31388" PostTypeId="1" CreationDate="2019-04-07T12:55:29.233" Score="2" ViewCount="324" Body="&lt;p&gt;I have a system of 10 ordinary differential equations of the form,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{dy_1}{dt} = f1(V1,k1,y1,y2)\\&#xA;\vdots \\&#xA;\frac{dy_{10}}{dt} = f_{10}(V_{10},k_{10},y_{9},y_{10})&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to estimate the parameters &lt;span class=&quot;math-container&quot;&gt;$V_1 ,V_6,V_7,V_{10}$&lt;/span&gt; using a  global fit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the approaches suggested in the literature, I understand least-squares error minimization is commonly used. However, I'm not able to understand how optimization problem is actually formulated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cost function = &lt;span class=&quot;math-container&quot;&gt;$\Sigma_{i=1}^{10} (y_i^\text{experiment} - y_i^\text{model})^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where, &lt;span class=&quot;math-container&quot;&gt;$y_i^\text{experiment}$&lt;/span&gt; is the steady-state value obtained from experiments and not the time series data of &lt;span class=&quot;math-container&quot;&gt;$y_i$&lt;/span&gt;.&#xA;Could someone explain how &lt;span class=&quot;math-container&quot;&gt;$y_i^\text{model}$&lt;/span&gt; is expressed in terms of the parameters that are to be estimated?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is is differential equation,(say)&#xA;&lt;span class=&quot;math-container&quot;&gt;$\frac{dy_1}{dt} = f1(V1,k1,y1,y2)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;expanded using Taylor polynomial to find &lt;span class=&quot;math-container&quot;&gt;$y_1^\text{model}$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone provide an example?&lt;/p&gt;&#xA;" OwnerUserId="29087" LastEditorUserId="20688" LastEditDate="2019-04-23T23:23:51.370" LastActivityDate="2019-04-23T23:47:05.197" Title="Formulation of the least-squares parameter estimation problem" Tags="|optimization|ode|least-squares|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31389" PostTypeId="2" ParentId="31387" CreationDate="2019-04-07T18:00:26.020" Score="6" Body="&lt;p&gt;I am uniquely qualified to talk about this approach as I am a current student of Jeff Erickson (advisor to Shripad Thite mentioned in that first link) and Robert Haber doing research on the spacetime meshing side of this Spacetime Discontinuous Galerkin approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fundamentally, this Spacetime Discontinuous Galerkin approach is quite different relative to more common method of lines approaches. There's many advantages to this approach that make it very exciting relative to more traditional approaches, though it ultimately has the disadvantage of being way more complex to build. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Typical Approaches&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typical marching schemes of all varieties have the benefit of being quite simple when you approach stepping through time, though obviously spatial aspects of solving the PDEs can have a range of complexity depending on your solution technique (Finite Differences, Finite Volume, Finite Element, etc.) and whether your mesh is structured, unstructured, adaptive, and/or moving. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A key drawback with time marching like this is that in between two time-slices of your PDE solution, you don't &lt;em&gt;really&lt;/em&gt; know what the best interpolation scheme is to use between the solutions at &lt;span class=&quot;math-container&quot;&gt;$t_k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_k + \Delta t$&lt;/span&gt;. One might lazily choose linear interpolation because they assume &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; is small enough that such an approximation is good enough. Some might use more complicated post processing techniques by considering more information than what is found in just two time-slices of your solution. Either way, you are ultimately &lt;em&gt;guessing&lt;/em&gt; what is an adequate interpolation scheme. Obviously this could be problematic if your &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; isn't actually small enough and/or if your PDE is nonlinear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Traditional marching schemes do have a fairly simple approach to parallelizing the solution procedure though, namely by decomposing the mesh into disjoint chunks and using separate processes to process chunks of the PDE solution local to its mesh chunk, only communicating with other meshes to handle shared boundaries between different disjoint mesh chunks. Obviously distributed parallel solution procedures also exist for solving large sparse matrices that might be formed for large implicit time stepping problems, so there's infrastructure there as well people can leverage. Of course, there's a lot of synchronization overhead since a given process must wait for a given time step to complete across (essentially) the whole mesh.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Spacetime Discontinuous Galerkin Approach&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compared to the above techniques, this technique is actually quite a different beast. This technique is, first and foremost, an implicit method that has the usual benefits in terms of stability. In the most naive approach, it also has the typical downfall of a implicit method, in that it is computationally demanding per timestep. In this naive approach, one can essentially &quot;time march&quot; by taking the mesh &lt;span class=&quot;math-container&quot;&gt;$\Omega$&lt;/span&gt; (assume represented as a simplicial mesh) and extruding it into a &lt;em&gt;spacetime slab&lt;/em&gt; by doing &lt;span class=&quot;math-container&quot;&gt;$\Omega \times [t_k, t_k + \Delta t]$&lt;/span&gt;, ultimately being made up of spacetime simplicial prisms. Each prism &lt;span class=&quot;math-container&quot;&gt;$\omega_i \times [t_k, t_k + \Delta t] \subset \Omega \times [t_k, t_k + \Delta t]$&lt;/span&gt; would be coupled to a constant number of simplicial prisms in the previous spacetime slab &lt;span class=&quot;math-container&quot;&gt;$\Omega \times [t_k - \Delta t, t_k]$&lt;/span&gt;. Within each simplicial prism, you will have some local spacetime basis defined, meaning each simplicial prism will have some number of degrees of freedom it contributes to what will ultimately be a pretty big matrix solve for each spacetime slab. The benefit of this over typical time marching, though, is you now &lt;em&gt;know&lt;/em&gt; how to compute your solution at &lt;strong&gt;any&lt;/strong&gt; time! No need to guess how to interpolate between two time slices.. you have all the data you need to compute a time-slice yourself anywhere! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now of course, one might consider it not worthwhile to go through the complication of forming spacetime simplicial prisms and dealing with solving the large sparse system of equations that could come out of each spacetime slab solution. That is fair and as @Paul mentioned in the comments, that would be costly! But..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;There is actually a way more interesting way to do things!&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fundamentally, we turn our attention to properties of systems of Hyperbolic PDEs. We know that the nature of the characteristics of such systems is such that a given location in spacetime depends on a cone of information that comes &lt;em&gt;before&lt;/em&gt; it in time, something along the lines of the image below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1fhlJ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1fhlJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can use this information to do something much smarter. For simplicity, let us assume the characteristic wavespeeds are constant throughout spacetime. Let us also just consider solving a problem in 1D x Time to make things simple. Under these assumptions, let the initial spacetime mesh will simply be the initial space mesh, a 1D line, as shown below at &lt;span class=&quot;math-container&quot;&gt;$t = 0$&lt;/span&gt;. Notice that I show the characteristics at the top right, &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt;, and I define &lt;span class=&quot;math-container&quot;&gt;$u_2$&lt;/span&gt; to be a more limiting characteristic than &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt; since &lt;span class=&quot;math-container&quot;&gt;$u_2$&lt;/span&gt; has the faster wavespeed of &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; (since &lt;span class=&quot;math-container&quot;&gt;$\left\lVert\frac{dt}{dx}\right\rVert$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$w_1$&lt;/span&gt; is smaller than for &lt;span class=&quot;math-container&quot;&gt;$w_2$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3321K.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3321K.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here, what might we be able to do? Well, consider the &lt;em&gt;pitch&lt;/em&gt; of one of the vertices shown below. As is shown in the diagram, this pitch generated a set of spacetime simplices such that any point inside this &lt;em&gt;patch&lt;/em&gt; of spacetime triangles is only dependent in &lt;em&gt;known&lt;/em&gt; information (the initial condition at the bottom facets, in this case). When a spacetime simplex has this property, it is named as being &lt;em&gt;causal&lt;/em&gt;. A patch comprised of simplices that are causal is also a &lt;em&gt;causal patch&lt;/em&gt;. Indeed, by forming causal patches around a vertex, this propery actually allows you to solve this spacetime patch independently of the rest of the spacetime mesh (which we have yet to explicitly construct)! This actually induces an algorithm that only requires you to solve systems of equations for &lt;em&gt;patches&lt;/em&gt;, which are constant sized relative to the size of the input spatial mesh!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XfdfQ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XfdfQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there's more! Consider the sample spacetime mesh that has been constructed and solved so far shown below. Gray and green triangles are causal spacetime simplices that have already been solved for their degrees of freedom and the purple triangles are a recently pitched patch that is about to be solved, which we will term as &lt;em&gt;active&lt;/em&gt; spacetime simplices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Afk0D.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Afk0D.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First notice that this causal pitching behavior allows different parts of the domain to capture much more of spacetime in one patch than other parts. This is important! This technique automatically adapts the &quot;time step&quot; based on the geometric and causal properties of the hyperbolic system without requiring &lt;strong&gt;all&lt;/strong&gt; of the mesh to be penalized for, say, having a really tiny simplex in some part of the mesh. This means you can make tons of progress in your solution really fast in coarser parts of your mesh!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice as well that the only simplices that have information flowing into the active simplices are the ones marked in green. This structure is consistent throughout this meshing approach and allows us to not require storing the whole spacetime mesh at any given moment that we produce, but only requires storing spacetime cells that have facets that will be incident on future active simplices. This means we can propagate through spacetime with a mesh proportional to size as the input spatial mesh we started with and that's it! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you consider this algorithmically, for an initial mesh that is &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; in size, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of spatial simplices, each tent-pitch requires &lt;span class=&quot;math-container&quot;&gt;$O(1)$&lt;/span&gt; simplices and in turn has &lt;span class=&quot;math-container&quot;&gt;$O(d)$&lt;/span&gt; degrees of freedom per patch solve, where &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; is the number of degrees of freedom per spacetime cell. Typical implicit (and even some explicit) marching schemes, though, will have to solve a system of &lt;span class=&quot;math-container&quot;&gt;$O(nd)$&lt;/span&gt; equations &lt;em&gt;per time step&lt;/em&gt;, while this technique will be solving &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; systems of &lt;span class=&quot;math-container&quot;&gt;$O(d)$&lt;/span&gt; equations for an equivalent chunk of spacetime. If we assume a solution can be done in &lt;span class=&quot;math-container&quot;&gt;$O(m^3)$&lt;/span&gt; for a linear system with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; equations, this means that the typical implicit marching scheme is &lt;span class=&quot;math-container&quot;&gt;$O(n^3 d^3)$&lt;/span&gt; while this Spacetime Discontinuous Galerkin procedure is &lt;span class=&quot;math-container&quot;&gt;$O(nd^3)$&lt;/span&gt; &lt;em&gt;for the equivalent chunk of spacetime&lt;/em&gt;. That's an enormous difference, especially for large problems where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is large!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here, there are many more benefits. Riemann problems solutions can be found exactly, even in nonlinear cases, since each facet of some spacetime simplex has information either purely incoming or purely outgoing. For parallel implementations, this approach is embarrassingly parallel. For shared memory parallel case, the process is along the lines of:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Grab a local fragment of the mesh incident to some vertex that's a local minima in the spacetime &lt;em&gt;front&lt;/em&gt;, lock the faces of this fragment, and send it off to some thread. Note that the front is essentially the space-like manifold that represents the &quot;top surface&quot; of the current spacetime extrusion you've built up&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On a given thread, pitch the vertex with its local fragment of simplices and construct the patch. Solve the patch.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stitch the recently found information into the spacetime mesh, deleting any spacetime cells not needed anymore and updating the front&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perform 1 through 3 across all your threads and vertices until the minimum vertex time exceeds the time you want your simulation to stop at&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Other benefits include a variety of adaptive operations that can be done in-place or through the construction of specialized patches, so they still leverage the local problem solving properties and the embarrassingly parallel design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Empirically, my groups serial code has run faster than a variety of state of the art parallel codes. Our parallel code is more new but has had extremely good scaling efficiency as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now of course, there are challenges with this technique. If we just consider serial implementation, some key challenges revolve around just implementing the meshing procedure and various data structures needed. This algorithm is actually quite complicated, especially if you want it to generalize to 3dxTime. Once you have the meshing algorithms and data structures in place, a lot of the finite element solving stuff will largely fall into place in a way that is not so different than usual finite element codes. However, adaptive operations can get quite tricky in spacetime, especially for 3dxTime, and complicated both the meshing and finite element side of things. In fact, it is in open problem (that I am working on) to find a &lt;em&gt;provably&lt;/em&gt; good policy in 3dxTime that allows for &lt;em&gt;both&lt;/em&gt; adaptive refinement and coarsening that &lt;em&gt;ensures&lt;/em&gt; we can make progress in spacetime indefinitely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another challenge is visualization because once you get to 3dxTime, you cannot directly visualize patches. Time-slicing at some desired set of times is always possible, though still another challenge because slicing some spacetime simplex for some constant time doesn't always produce a simplex in space (imagine slicing a 4-simplex such that 2 vertices are above the desired time and the other 3 are below.. what do you get?).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In any event, I hope you've got a better idea of how this works and why it could be advantageous and also a challenge to use this approach. I should add a note that this approach is useful for &lt;em&gt;all&lt;/em&gt; problems that are modeled as hyperbolic systems of equations. My group and some collaborators have some related work to converting other PDEs into hyperbolic PDEs so we can use them in this framework. Let's see where the future goes...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the meantime, let me share with you some work by colleagues related to this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://rezaabedi.com/&quot; rel=&quot;noreferrer&quot;&gt;Research site with various visuals and paper references by Prof Reza Abedi, former student of Bob Haber who still does work using this technique&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://jeffe.cs.illinois.edu/pubs/slowpitch.html&quot; rel=&quot;noreferrer&quot;&gt;Building Spacetime Meshes over arbitrary Spatial Meshes, work done by my advisor Jeff Erickson and some of his colleagues&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://jeffe.cs.illinois.edu/pubs/refine.html&quot; rel=&quot;noreferrer&quot;&gt;Paper on Adaptive Refinement and Coarsening for Spacetime Meshing, work done by Reza Abedi, my advisors, and some other collaborators&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9804" LastEditorUserId="9804" LastEditDate="2019-04-07T18:06:32.530" LastActivityDate="2019-04-07T18:06:32.530" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31391" PostTypeId="1" CreationDate="2019-04-08T00:20:49.920" Score="3" ViewCount="2376" Body="&lt;p&gt;I’d like to solve the below equation for the unknown &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_0^\infty \frac{x^2}{\exp\left(\frac{x}{T}\right)-1}\kappa_x \mathrm{d}x = C,$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is a known constant and &lt;span class=&quot;math-container&quot;&gt;$\kappa_x$&lt;/span&gt; is some function of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, although the actual function is unknown – all I have as an array of values of &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; over different values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&#xA;I do not want to have to fit this it to a function first. I was hoping to use one of the SciPy’s numerical integration functions such as &lt;code&gt;integrate.simps&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m confused on how to solve this; since &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; is inside an exponential with &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; dependence I cannot simply pull it out of the integral. It would be great to be able to pass the unknown constant into my numerical integral and receive an answer in terms of the constant. But as far as I know, in Python, I cannot pass an unknown. How can I accomplish this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Here is an example of what my &lt;span class=&quot;math-container&quot;&gt;$\kappa_x$&lt;/span&gt; array could look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k_arr = [1.1e-3, 8.8e-2, ..., 3.7e3]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(i.e., floating point numbers generally ranging from ~ 10^-3 to 10^3. But the length of my real array is about 3000).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To provide some more explanation, the reason I subscript it as &lt;span class=&quot;math-container&quot;&gt;$\kappa_x$&lt;/span&gt; is because each of the values is associated with some &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; value (in my case, &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is an energy or frequency). Previously I used various energies to arrive at &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; values, which are opacities in units of cm^2 g^-1. The integral in question relates to the power per unit mass emitted by dust in a galaxy, where I omitted constants (except for the temperature &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;). It is equal to some known number, so that I can hopefully solve for the temperature of the dust.&lt;/p&gt;&#xA;" OwnerUserId="30414" LastEditorUserId="414" LastEditDate="2019-04-11T08:05:22.487" LastActivityDate="2019-04-14T19:42:15.607" Title="Numerical integration in Python with unknown constant" Tags="|python|scipy|integration|numerics|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31392" PostTypeId="2" ParentId="31391" CreationDate="2019-04-08T09:37:24.080" Score="6" Body="&lt;p&gt;What you want seems inherently impossible, and that’s not due to restrictions of Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way we can arrive at a situation where we only need to apply a single quadrature is to get analytically get rid of all dependencies of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; in the integral.&#xA;To this end, the best we can do is to apply the substitution &lt;span class=&quot;math-container&quot;&gt;$x=Ty$&lt;/span&gt; to your integral:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;I(T) =&#xA;\int_0^∞ \frac{x^2 κ_x}{\exp\left(\frac{x}{T}\right)-1} \mathrm{d}x =&#xA;\int_0^∞ \frac{T^2 y^2 κ_{Ty}}{\exp(y)-1} T \mathrm{d}y =&#xA;T^3\int_0^∞ \frac{y^2 κ_{Ty}}{\exp(y)-1} \mathrm{d}y.&#xA;$$&lt;/span&gt;&#xA;Now, we got rid of all contributions of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; to the integral except as an argument of &lt;span class=&quot;math-container&quot;&gt;$κ$&lt;/span&gt;.&#xA;We can only improve this further by applying special knowledge about &lt;span class=&quot;math-container&quot;&gt;$κ$&lt;/span&gt;, but since &lt;span class=&quot;math-container&quot;&gt;$κ$&lt;/span&gt; is empirical, you probably do not know anything that could help you here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From another perspective, if you apply a method similar to the midpoint method to the values of &lt;span class=&quot;math-container&quot;&gt;$κ$&lt;/span&gt; that you actually have (&lt;span class=&quot;math-container&quot;&gt;$κ_{x_1},…,κ_{x_n}$&lt;/span&gt;), the result &lt;span class=&quot;math-container&quot;&gt;$I(T)$&lt;/span&gt; would be some weighted sum of these values:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ I(T) ≈ \sum_{i=1}^n a_i(T) κ_{x_i}, $$&lt;/span&gt;&#xA;with the weights for the being:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ a_i(T) = \frac{x_i^2}{\exp\left(\frac{x_i}{T}\right)-1} b_i,$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$b_i$&lt;/span&gt; only depends on the spacing of your &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;.&#xA;Obviously, you cannot just factor out the &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; dependency here.&#xA;For other quadrature methods, the weights may become more complicated, but there is no reason to expect this solves the above problem.&lt;/p&gt;&#xA;" OwnerUserId="7574" LastActivityDate="2019-04-08T09:37:24.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31393" PostTypeId="2" ParentId="31391" CreationDate="2019-04-08T11:51:15.753" Score="2" Body="&lt;p&gt;Lets assume that the given expression has a non-infinite integral you could actually write down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question you should be asking -might- be more like: &quot;To what precision do I need to calculate?&quot;. If you have &lt;span class=&quot;math-container&quot;&gt;$$K_x(x)$$&lt;/span&gt; only as an array of values anyway, you might only ever achieve a certain precision. Is an error of 1e-16 okay for your application? Do you need even more precision?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might approach the problem from that perspective by starting to evaluate the expression at equally spaced points x (e.g. the spacing of the array of K_x you have), and apply the weights like @Wrzlprmft wrote. You should be able to estimate how big your error is if you stop the summation at a certain point. The exponential function will force the profile towards zero quite rapidly, so when you are at a high value of x_i of your integration points, you probably already reach the smallest value storable in double precision. Depending on the problem you want to solve that might already be enough. Could you elaborate what the underlying problem is?&lt;/p&gt;&#xA;" OwnerUserId="28636" LastEditorUserId="28636" LastEditDate="2019-04-08T11:58:02.073" LastActivityDate="2019-04-08T11:58:02.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31394" PostTypeId="1" CreationDate="2019-04-08T13:35:52.897" Score="1" ViewCount="503" Body="&lt;p&gt;I have a &lt;span class=&quot;math-container&quot;&gt;$200 \times 200$&lt;/span&gt; matrix representing the values taken by a function over an equally spaced grid. I would like to perform derivatives on it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in its gradient (i.e. its derivative in direction &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and its derivative in direction &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;) and in its Laplacian. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work on the Matlab platform, where I use the built-in functions &lt;code&gt;gradient&lt;/code&gt; and &lt;code&gt;del2&lt;/code&gt; which work very well but their accuracy is limited by the fact that they make use of a small number of points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading this Wikipedia page about &lt;a href=&quot;https://en.wikipedia.org/wiki/Finite_difference_coefficient&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finite difference coefficients&lt;/a&gt;, one can understand that there are finite-difference schemes to perform numerical differentiation in a more accurate way. The price to pay to have an increased accuracy is of course to use more complex formulas, which include more points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there is a library where there are functions capable of doing this job which -I repeat- is: computing the first-order and the second-order derivatives of a 2D matrix (representing the values of a not-explicitly-known function) with a user-defined accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be great if someone could suggest me Matlab libraries, but also C/C++ libraries could work, I guess.&lt;/p&gt;&#xA;" OwnerUserId="29095" LastEditorUserId="29095" LastEditDate="2019-04-08T15:02:13.780" LastActivityDate="2019-04-08T15:02:13.780" Title="High-accuracy numerical differentiation" Tags="|matlab|finite-difference|libraries|laplacian|" AnswerCount="0" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="31396" PostTypeId="2" ParentId="3478" CreationDate="2019-04-08T20:59:23.813" Score="0" Body="&lt;h3&gt;Use the Literate Programming Functionality of &lt;a href=&quot;https://orgmode.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;org-mode&lt;/a&gt;.&lt;/h3&gt;&#xA;&lt;p&gt;Most org-mode users tend to focus exclusively on the either the built-in project/time management functionality or the ability to export documents into multiple popular file formats, e.g. &lt;a href=&quot;https://github.com/Literate-DevOps/literate-programming-tutorials/blob/master/how-to/00-convert-latex-template-into-org-mode-template/self-check-example-04.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt;, from easy to maintain &lt;a href=&quot;https://github.com/Literate-DevOps/literate-programming-tutorials/blob/master/how-to/00-convert-latex-template-into-org-mode-template/self-check-example-04.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;text&lt;/a&gt; files.&lt;/p&gt;&#xA;&lt;p&gt;However, the best feature of org-mode is ability to create literate programs in over 30 languages with more languages added every month by the open source community.&lt;/p&gt;&#xA;&lt;p&gt;Below are trivial code examples using Ruby and Python:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; #+NAME: trivial-code-ex1&#xA; #+BEGIN_SRC ruby &#xA;   &amp;quot;hello world!&amp;quot;&#xA; #+END_SRC&#xA;&#xA; #+RESULTS: trivial-code-ex1&#xA; : hello world!&#xA; &#xA; &#xA; #+NAME: func-of-x-and-y&#xA; #+BEGIN_SRC python :var x=1 :var y=2 :session&#xA;   x + y&#xA; #+END_SRC&#xA;&#xA; #+RESULTS: func-of-x-and-y&#xA; : 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Pros&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Support for over &lt;a href=&quot;https://orgmode.org/manual/Languages.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;30 programming languages&lt;/a&gt;, including R, Python, Ruby, Perl, C, C++, Java, Clojure, Javascript, Common Lisp, Shell, SQL, ...&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The ability to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://orgmode.org/manual/Results-of-Evaluation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Capture&lt;/a&gt; &lt;code&gt;SRC&lt;/code&gt; block results as output and/or value.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://orgmode.org/manual/Results-of-Evaluation.html#index-results_002c-header-argument&quot; rel=&quot;nofollow noreferrer&quot;&gt;Format&lt;/a&gt; &lt;code&gt;SRC&lt;/code&gt; block results as code, lists, table, latex, html&lt;/li&gt;&#xA;&lt;li&gt;Use both external &amp;amp; internal data for variables of &lt;code&gt;SRC&lt;/code&gt; blocks.&lt;/li&gt;&#xA;&lt;li&gt;Use the output of named &lt;code&gt;SRC&lt;/code&gt; blocks into &lt;code&gt;SRC&lt;/code&gt; blocks as variables.&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;a href=&quot;https://orgmode.org/manual/Noweb-Reference-Syntax.html#Noweb-Reference-Syntax&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;noweb&lt;/code&gt;&lt;/a&gt; syntax inside &lt;code&gt;SRC&lt;/code&gt; blocks.&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Pro Tip:&lt;/strong&gt; You can use &lt;code&gt;noweb&lt;/code&gt; syntax to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;insert code from a named &lt;code&gt;SRC&lt;/code&gt; block, e.g. &lt;code&gt;func-of-x-and-y&lt;/code&gt;, inside another &lt;code&gt;SRC&lt;/code&gt; block.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#+BEGIN_SRC python :session :noweb yes &#xA;  x=2&#xA;  y=3&#xA;  &amp;quot;f(x,y) is\n\n &amp;lt;&amp;lt;func-of-x-and-y&amp;gt;&amp;gt; \n\nso f({0},{1}) equals\n\n {2}&amp;quot;.format(x,y,&amp;lt;&amp;lt;func-of-x-and-y&amp;gt;&amp;gt;)&#xA;#+END_SRC&#xA;&#xA;#+RESULTS:&#xA;: f(x,y) is&#xA;: &#xA;:  x + y &#xA;: &#xA;: so f(2,3) equals&#xA;: &#xA;:  5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;insert the results of a named &lt;code&gt;SRC&lt;/code&gt; block, e.g. &lt;code&gt;func-of-x-and-y&lt;/code&gt; inside another &lt;code&gt;SRC&lt;/code&gt; block&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#+BEGIN_SRC python :session :noweb yes &#xA;  &amp;quot;f(x,y) is\n\n &amp;lt;&amp;lt;func-of-x-and-y&amp;gt;&amp;gt; \n\nso f(3,4) equals\n\n &amp;lt;&amp;lt;func-of-x-and-y(x=3,y=4)&amp;gt;&amp;gt;&amp;quot;&#xA;#+END_SRC&#xA;&#xA;#+RESULTS:&#xA;: f(x,y) is&#xA;: &#xA;:  x + y &#xA;: &#xA;: so f(3,4) equals&#xA;: &#xA;:  7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Place named &lt;code&gt;SRC&lt;/code&gt; blocks anywhere in an org-mode file for readability and use &lt;code&gt;:tangle&lt;/code&gt; header or export code into external source files.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Open source project - both free as in beer and free as in freedom.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Text file format works great with version control software like git.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ooodles of other features which I won't go into because this answer is getting long.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Cons&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Need to have gnu emacs installed and configured to use org-mode.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Most of you stopped reading this answer after you read gnu emacs. For the brave souls that are left, you can use your favorite text editor and just call emacs to process your org-mode files from the command-line.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Need to install and configure all of the programming software you need.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Need to install and configure LaTeX utilities if you want to make PDFs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;org-mode is not as well know as &lt;code&gt;ipython notebooks&lt;/code&gt; or &lt;code&gt;Sweave&lt;/code&gt; so you probably won't see as many job postings even though Literate Programming functionality was added in 2008.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Learning org-mode markup syntax&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Potentially learning how to use gnu emacs or spacemacs if want to get most out of the other cool tools that work with org-mode.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Full disclosure:&lt;/strong&gt; I am the primary maintainer of the org-mode &lt;a href=&quot;https://github.com/melioratus/org-mode&quot; rel=&quot;nofollow noreferrer&quot;&gt;package&lt;/a&gt; for the Atom editor.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;The code in this answer was validated using:&lt;/strong&gt;&lt;br /&gt;&#xA;emacs version: GNU Emacs 25.2.1&lt;br /&gt;&#xA;org-mode version: 9.1.2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30420" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2019-04-08T20:59:23.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31397" PostTypeId="2" ParentId="31391" CreationDate="2019-04-09T04:09:33.003" Score="3" Body="&lt;p&gt;It seems to me that this question is about how you handle &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt;. I know you don't want to fit a function to it, but I don't think you can get around sampling it at positions other than the positions in the array. You may also need some kind of asymptotic approximation as &lt;span class=&quot;math-container&quot;&gt;$x \rightarrow \infty$&lt;/span&gt; so you can extrapolate in that range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, I have an idea about how to handle the dependency on &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; that may be easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know anything about SciPy, so you will need to do some algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key observation that you need is: If &lt;span class=&quot;math-container&quot;&gt;$T &amp;gt; 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is a positive integer then:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_{0}^{\infty} \frac{x^k}{e^{x/T} -1}\, dx = k!\,T^{k+1} \zeta(k+1)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\zeta(z)$&lt;/span&gt; is the Riemann zeta function. (Exercise: Prove this.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values of the zeta function at positive integers &lt;a href=&quot;https://en.wikipedia.org/wiki/Particular_values_of_the_Riemann_zeta_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;are well-known&lt;/a&gt;. This gives you a way to calculate:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\int_{0}^{\infty} p(x) \frac{x^2}{e^{x/T} -1}\, dx$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for any polynomial &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;. So if &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; were a polynomial, you could calculate this more or less directly to obtain a polynomial in &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, and then solve for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; is conceptually defined in the range &lt;span class=&quot;math-container&quot;&gt;$[0,\infty)$&lt;/span&gt;, chances are there isn't a good polynomial approximation for it that behaves sensibly over the whole range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my suggestion is a variant of Wrzlprmft's, but instead of using the midpoint method, use &lt;a href=&quot;http://mathworld.wolfram.com/GaussianQuadrature.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian quadrature&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you find the Gauss weights and nodes (which will be functions of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;) for the weight function &lt;span class=&quot;math-container&quot;&gt;$W_T(x) = \frac{x^2}{e^{x/T} - 1}$&lt;/span&gt; over the domain &lt;span class=&quot;math-container&quot;&gt;$[0,\infty)$&lt;/span&gt;, this will give you an approximation to the integral expressed as a finite set of samples of &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;em&gt;may&lt;/em&gt; give you a well-behaved equation that you can solve for &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3686" LastEditorUserId="3686" LastEditDate="2019-04-10T02:38:55.800" LastActivityDate="2019-04-10T02:38:55.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31398" PostTypeId="2" ParentId="31373" CreationDate="2019-04-09T13:34:44.113" Score="2" Body="&lt;p&gt;Welcome to SE SciComp. First of all, I would suggest using Jupyter so that you have access to IPython and its nice timing magics (see &lt;a href=&quot;https://ipython.readthedocs.io/en/stable/interactive/magics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;%time and %timeit magic function&lt;/a&gt;). These magics take into account that you need to run the code a couple of times to get reliable measurements. If you need even more in-depth comparison, you also need to take into account memory allocation and caching for which you can use the &lt;a href=&quot;https://ipython.readthedocs.io/en/stable/interactive/magics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;%prun magic&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, it's a bit unfair to compare timings between a vectorized function (in your case a NumPy call, which uses C as a backend) and a routine where you also perform I/O (i.e. printing your progress bar on screen). I can understand the comparison between the vanilla non-vectorized routine and the NumPy routine, but you shouldn't penalize the vanilla routine by having print outs to the screen. You're comparing grapes to apples... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Third, try to avoid the if-then-else clause in the routines. Write separate routines for each case (vectorized vs non-vectorized) that you want to time/compare. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fourth, in your non-vectorized algorithm you append a value to a list in each iteration. Since you use NumPy, why not pre-allocate a vector of size &lt;code&gt;max_iter&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See code below (note that it has to be run inside an IPython environment, i.e. a Jupyter notebook or Qtconsole or plain IPython console). Is this want you want to achieve?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;def analytical_sine(max_iter):&#xA;    analytical_hist = np.zeros(max_iter)&#xA;    for i in range(max_iter):&#xA;        uni_dist = np.random.uniform(0, 1)&#xA;        analytical_hist[i] = np.arccos(1 - 2*uni_dist)&#xA;    return analytical_hist&#xA;&#xA;def analytical_sine_vec(max_iter):&#xA;    return np.arccos(1 - 2*np.random.uniform(0, 1, max_iter))&#xA;&#xA;print(&quot;Non-vectorized&quot;)&#xA;for i in range(10):&#xA;    print(f&quot;Max_iter = {100*(i+1)}&quot;)&#xA;    %timeit res = analytical_sine(100*(i+1))&#xA;print(&quot;Vectorized&quot;)&#xA;for i in range(10):&#xA;    print(f&quot;Max_iter = {100*(i+1)}&quot;)&#xA;    %timeit res = analytical_sine_vec(100*(i+1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2019-04-10T07:15:43.003" LastActivityDate="2019-04-10T07:15:43.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31400" PostTypeId="1" CreationDate="2019-04-10T05:11:46.807" Score="1" ViewCount="267" Body="&lt;p&gt;I have a cost function with 2 parameters. The variables are dependent on each other. So, if I just take a partial derivative with respect to one variable the slope is in terms of the other variable and vice-versa. So, how can I compute the optimal values for these variables that minimize my cost function?&lt;/p&gt;&#xA;" OwnerUserId="30431" LastActivityDate="2020-12-30T10:04:05.573" Title="Optimize multivariable function with interdependent variables" Tags="|convex-optimization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31401" PostTypeId="2" ParentId="31400" CreationDate="2019-04-10T07:25:42.033" Score="1" Body="&lt;p&gt;What you have appears to be a non-linear function which you are trying to optimize. Whilst not as trivial as linear optimization it is not particularly difficult either. Since numerically you aren't working with symbolic variables but always with numbers (arrays of numbers called vectors) which are computed at each step of the optimization process anew, to adapt to the parameters being considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest you open a book on numerical analysis at the chapter treating non-linear optimization. One method to try would be gradient descent, but there are better variations.&lt;/p&gt;&#xA;" OwnerUserId="28993" LastActivityDate="2019-04-10T07:25:42.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31402" PostTypeId="2" ParentId="3478" CreationDate="2019-04-10T08:22:08.617" Score="3" Body="&lt;p&gt;The LaTeX package &lt;a href=&quot;https://github.com/gpoore/minted&quot; rel=&quot;nofollow noreferrer&quot;&gt;minted&lt;/a&gt; provides very extensive syntax highlighting (based on Pygments) and allows cross-referencing in both directions. You can escape to LaTeX from within the code part (the minted part) and you can refer in your main text to lines of code. On top of that, it provides a listings environment so that you can generate a &quot;list of listings&quot; (like a list of tables) and allows referencing entire listings. See LaTeX MWE and its output with LuaLaTeX below (don't judge the code :-) ). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option would be to use &lt;a href=&quot;https://github.com/gpoore/pythontex&quot; rel=&quot;nofollow noreferrer&quot;&gt;PythonTeX&lt;/a&gt; from the same author/maintainer which allows running the calculations while compiling the LaTeX source, hence paper and code results are always generated together and hence are always coherent. See the &lt;a href=&quot;https://github.com/gpoore/pythontex/blob/master/pythontex_gallery/pythontex_gallery.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PythonTeX gallery&lt;/a&gt; here. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;\documentclass[a4paper,notitlepage,11pt]{article}&#xA;&#xA;\usepackage{amsmath}&#xA;\usepackage{cases}&#xA;\usepackage{minted}&#xA;&#xA;\begin{document}&#xA;&#xA;The mathematical definition of the Fibonacci&#xA;series is given in~Equations~(\ref{eq:fibdef:init1}--\ref{eq:fibdef:rule})&#xA;It can be implemented using either a recursive or iterative algorithm&#xA;in Python.&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{numcases}{f(n)=}&#xA;  \label{eq:fibdef}&#xA;    0               &amp;amp; n = 0 \label{eq:fibdef:init1}\\&#xA;    1               &amp;amp; n = 1 \label{eq:fibdef:init2}\\&#xA;    f(n-1) + f(n-2) &amp;amp; \text{otherwise} \label{eq:fibdef:rule}&#xA;\end{numcases}&lt;/span&gt;&#xA;&#xA;The algorithms below are an implementation of both variants.&#xA;Listing~\ref{alg:fib_recursive} shows the recursive variant (see&#xA;line~\ref{alg:fibo_rec:line_rec} in listing~\ref{alg:fib_recursive}) while&#xA;listing~\ref{alg:fib_iterative} shows the iterative variant. Both can be&#xA;optimized, of course.&#xA;&#xA;\begin{listing}[ht]&#xA;  \begin{minted}[linenos, escapeinside=||]{python}&#xA;def fibo_rec(N):&#xA;    if N == 0:&#xA;        result = 1 |[Comment: See case (\ref{eq:fibdef:init1})]|&#xA;    elif N == 1:&#xA;        result = 1 |[Comment: See case (\ref{eq:fibdef:init2})]|&#xA;    else:&#xA;        result = fibo_rec(N-1) + fibo_rec(N-2) |\label{alg:fibo_rec:line_rec}[Comment: See case (\ref{eq:fibdef:rule})]|&#xA;&#xA;    return result&#xA;  \end{minted}&#xA;\caption{Fibonacci recursive}&#xA;\label{alg:fib_recursive}&#xA;\end{listing}&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{listing}[ht]&#xA;  \begin{minted}[linenos, escapeinside=||]{python}&#xA;def fibo_iter(N):&#xA;    if N == 0:&#xA;        fib_N = 1&#xA;    elif N == 1:&#xA;        fib_N = 1&#xA;    else:&#xA;        fib_Nmin2 = 1&#xA;        fib_Nmin1 = 1&#xA;        for i in range(2,N+1):&#xA;            fib_N = fib_Nmin2 + fib_Nmin1&#xA;            fib_Nmin2 = fib_Nmin1&#xA;            fib_Nmin1 = fib_N&#xA;    return fib_N&#xA;  \end{minted}&#xA;\caption{Fibonacci iterative}&#xA;\label{alg:fib_iterative}&#xA;\end{listing}&lt;/span&gt;&#xA;&#xA;\end{document}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X9omO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X9omO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2019-04-10T08:22:08.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31403" PostTypeId="2" ParentId="20533" CreationDate="2019-04-10T09:41:32.140" Score="5" Body="&lt;p&gt;Plotting the error of &lt;span class=&quot;math-container&quot;&gt;$\ddot q=-q$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$q(0)=1,\dot q(0)=0$&lt;/span&gt; over the full interval, scaled by the power of the step size given by the expected order, gives the plots&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/i2lB9.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i2lB9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As expected, the graphs for increasing number of sub-intervals increasingly approach a limit curve which is the leading error coefficient. In all but one plot this convergence is visibly rapid, there is almost no divergence. This means that even for relatively large step sizes the leading error term dominates all the other terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the 3rd order Ruth method, the leading coefficient of the &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; component is appears to be zero, the visible limit curve is close or equal to the horizontal axis. The visible graphs clearly show the dominance of the 4th order error term. Scaling for an 4th order error gives a plot similar to the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one can see, in all 3 cases the error coefficient of the leading order in the &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; component is zero at &lt;span class=&quot;math-container&quot;&gt;$t=2\pi$&lt;/span&gt; after a full period. In combining the errors of both components, the behavior of the &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; component thus dominates, falsely giving the impression of a 4th order method in the loglog plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A maximal coefficient in the &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; component can be found around &lt;span class=&quot;math-container&quot;&gt;$3\pi/2$&lt;/span&gt;. The loglog plot there should reflect the correct global error orders.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;That the vanishing of the 3rd degree error term in Ruth3p is an artefact of the simplicity of the linear equation shows the non-linear example &lt;span class=&quot;math-container&quot;&gt;$\ddot q=-\sin(q)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$q(0)=1.3,~\dot q(0)=0$&lt;/span&gt; with the corresponding plots&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/72Wkh.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/72Wkh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6839" LastEditorUserId="6839" LastEditDate="2019-04-10T11:15:51.373" LastActivityDate="2019-04-10T11:15:51.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31405" PostTypeId="2" ParentId="31378" CreationDate="2019-04-10T15:34:43.030" Score="2" Body="&lt;p&gt;First of all, you need to rewrite your non-linear differential equation in a set of first order differential equations since the &lt;code&gt;solve_ivp&lt;/code&gt; routine solves problems of the form &lt;span class=&quot;math-container&quot;&gt;$u'(x) = f(x,u), \ u(0) = u_{0}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$u(x)$&lt;/span&gt; can be a vector. So define &lt;span class=&quot;math-container&quot;&gt;$u_{1}(x) = y(x)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u_{2}(x) = y'(x)$&lt;/span&gt; and you can write your original equation as &lt;span class=&quot;math-container&quot;&gt;\begin{array}{lll} u_{1}'(x) &amp;amp;=&amp;amp; u_{2}(x) \\ u_{2}'(x) &amp;amp;=&amp;amp; 2(U(x)-a)u_{1}(x)+4b(u_{1}(x))^{3} - \frac{2}{x}u_{2}(x)\end{array}&lt;/span&gt; which is in the form required. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to set your initial conditions &lt;span class=&quot;math-container&quot;&gt;$u_1(0) = y_{0}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u_{2}(0) = y'(0)$&lt;/span&gt;; the latter which you do not specify in your original problem statement. But looking at the second equation, it would need to be zero since otherwise the last term gets a division by zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, you would need a routine that returns &lt;span class=&quot;math-container&quot;&gt;$U(x)$&lt;/span&gt; as a function of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;. You say that you  have no analytical form but only numerical data. So you need to write a routine that interpolates the available numerical data (or an another type of approximation you see to be fit). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you could launch a series of calculations with different values of your parameter &lt;code&gt;a&lt;/code&gt; to see when the solution tends to zero for &lt;span class=&quot;math-container&quot;&gt;$x\to\infty$&lt;/span&gt;. And wrap these in a secant solver (or any other solver) to find values for &lt;code&gt;a&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2019-04-10T15:34:43.030" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31406" PostTypeId="1" CreationDate="2019-04-10T17:30:39.260" Score="2" ViewCount="398" Body="&lt;p&gt;I have a bunch of data points in 3D that lie along a few planes. What would be the best approaches to estimate the normals of these planes? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: There are roughly equal number of points lying along each plane. All the planes contain the origin (0,0,0). &lt;/p&gt;&#xA;" OwnerUserId="19117" LastEditorUserId="19117" LastEditDate="2019-04-11T19:01:26.117" LastActivityDate="2021-05-02T18:07:40.723" Title="Clustering with points lying along different 3D planes" Tags="|algorithms|data-analysis|clustering|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31407" PostTypeId="2" ParentId="27614" CreationDate="2019-04-10T19:44:52.790" Score="2" Body="&lt;p&gt;You can think of each vector as a point in your linear space. As such, we can use a simple quadtree/octree-like algorithm to map your points into boxes, with &quot;nearby&quot; vectors assigned to the same or an adjacent box. With &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; total vectors the vector-to-box map costs &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O} (n\log n)$&lt;/span&gt;, and once this is done you can choose &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; boxes and select the vector closest to the centroid of the box to obtain your &quot;diverse&quot; set of vectors.&lt;/p&gt;&#xA;" OwnerUserId="27171" LastActivityDate="2019-04-10T19:44:52.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31408" PostTypeId="2" ParentId="31391" CreationDate="2019-04-11T08:51:23.710" Score="1" Body="&lt;p&gt;This could be done purely numerically, if you take the integral as part of your non-linear equation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, I created a interpolating function for your &lt;span class=&quot;math-container&quot;&gt;$\kappa_x$&lt;/span&gt; function. It's just a dummy linear interpolation between data points allowing for end-point constant extrapolation (see &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp1d.html#scipy.interpolate.interp1d&quot; rel=&quot;nofollow noreferrer&quot;&gt;interp1d&lt;/a&gt;). I leave it up to you to modify this according to your problem and your taste (more fancy interpolation scheme). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I define the left-hand-side of the non-linear equation &lt;span class=&quot;math-container&quot;&gt;$f(T) = 0$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$$f(T) = \int_{0}^{\infty} \frac{x^{2}}{\exp(\frac{x}{T})-1}\kappa_{x}\, dx -C.$$&lt;/span&gt; So I need to define a function that takes only takes &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt; as an argument and encapsulate the integral in this function. In Python, this could be something like the code below. Note that I integrate from the machine epsilon on the left boundary to avoid the singularity at &lt;span class=&quot;math-container&quot;&gt;$x = 0$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make sure you read up on the docs for &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.quad.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;quad&lt;/a&gt; and &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fsolve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;fsolve&lt;/a&gt; so that you understand their options and limitations. Depending on your &lt;span class=&quot;math-container&quot;&gt;$\kappa_{x}$&lt;/span&gt; function, your integrand could be peaked. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.integrate import quad&#xA;from scipy.optimize import fsolve&#xA;from scipy.interpolate import interp1d&#xA;&#xA;kappa_x = np.linspace(0, 10, 11)&#xA;kappa_k = np.array([1, 2, 3, 4, 5, 5, 4, 3, 2, 1, 1])&#xA;&#xA;kappa_fun = interp1d(kappa_x, kappa_k, bounds_error = False, fill_value = 1.0)&#xA;&#xA;def RHS(T):&#xA;    C = 1000&#xA;    def integrand(x):&#xA;        return np.power(x, 2) / (np.exp(x / T) - 1.0) * kappa_fun(x)&#xA;    res, error = quad(integrand, np.finfo(float).eps, np.inf)&#xA;    return res - C&#xA;&#xA;T_sol = fsolve(RHS, 100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2019-04-14T19:42:15.607" LastActivityDate="2019-04-14T19:42:15.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31409" PostTypeId="1" AcceptedAnswerId="31410" CreationDate="2019-04-11T10:51:05.140" Score="4" ViewCount="62" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; be an &lt;span class=&quot;math-container&quot;&gt;$3\times N$&lt;/span&gt; matrix (where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is large) with nonnegative real entries. I'd like an algorithm for determining when a vector &lt;span class=&quot;math-container&quot;&gt;$v\in\Bbb R^3$&lt;/span&gt; can be written as &lt;span class=&quot;math-container&quot;&gt;$Aw$&lt;/span&gt; for some vector &lt;span class=&quot;math-container&quot;&gt;$w\in\Bbb R^N$&lt;/span&gt; with each entry of &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; in the range &lt;span class=&quot;math-container&quot;&gt;$[0,1]$&lt;/span&gt;. In other words I'd like to find the image of the unit hypercube in &lt;span class=&quot;math-container&quot;&gt;$\Bbb R^N$&lt;/span&gt; under the map represented by &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. Does there exist an algorithm which is fast even when &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is large?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Everything I've thought of so far is exponential, basically looping over the &lt;span class=&quot;math-container&quot;&gt;$2^N$&lt;/span&gt; vertices of the hypercube. The problem is similar to that of determining the convex hull of a given set of points, except that the entries in &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; needn't sum to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. Obviously taking the convex hull of all &lt;span class=&quot;math-container&quot;&gt;$2^N$&lt;/span&gt; points would work, but it's exponential again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My motivation is that I want to determine the range of colours that can be seen under a given light source, based on the spectral power distribution of the light source and the responsivity curves of the three cone cells in the eye.&lt;/p&gt;&#xA;" OwnerUserId="30442" LastActivityDate="2019-09-08T13:02:10.510" Title="Determine image of hypercube under linear map" Tags="|linear-algebra|matrix|convex-optimization|linear-programming|convex-hull|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31410" PostTypeId="2" ParentId="31409" CreationDate="2019-04-11T12:20:08.597" Score="4" Body="&lt;p&gt;I eventually found an answer. The image of a hypercube in &lt;span class=&quot;math-container&quot;&gt;$\Bbb R^3$&lt;/span&gt; under a linear map is called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Zonohedron&quot; rel=&quot;nofollow noreferrer&quot;&gt;zonohedron&lt;/a&gt;. They can be calculated efficiently, for example using the algorithm in &lt;em&gt;&lt;a href=&quot;https://www.cs.cmu.edu/~ph/zono.ps.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;An Efficient Algorithm for Generating Zonohedra&lt;/a&gt;&lt;/em&gt; (PostScript file) by Paul Heckbert.&lt;/p&gt;&#xA;" OwnerUserId="30442" LastActivityDate="2019-04-11T12:20:08.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31411" PostTypeId="2" ParentId="31383" CreationDate="2019-04-11T13:09:31.207" Score="1" Body="&lt;p&gt;I am not an expert in the field but using DCT's (or DST's) is the standard way to impose Dirichlet (or Neumann) boundary conditions using FFT's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is nicely explained in the documentation of the FFTW package&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.fftw.org/fftw3_doc/Real-even_002fodd-DFTs-_0028cosine_002fsine-transforms_0029.html#Real-even_002fodd-DFTs-_0028cosine_002fsine-transforms_0029&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.fftw.org/fftw3_doc/Real-even_002fodd-DFTs-_0028cosine_002fsine-transforms_0029.html#Real-even_002fodd-DFTs-_0028cosine_002fsine-transforms_0029&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="19030" LastActivityDate="2019-04-11T13:09:31.207" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31412" PostTypeId="2" ParentId="31406" CreationDate="2019-04-11T13:34:43.633" Score="0" Body="&lt;p&gt;I would try using a Hough Transform, easy to find on Wikipedia. There's a thorough discussion there about using the HT to classify lines from noisy images (a 2-parameter problem). Your task of classifying planes from noisy point clouds is similar, but it's a 3-parameter problem .. two spherical angles can describe the direction of the plane normal, then one more scalar to represent the distance from the origin. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach is probably a little old-school, a computer vision expert might have better ideas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: If the plane always contains the origin you only have the angles as (two) free parameters. This is fortunate, because the HT scales terribly with dimension anyway.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="2107" LastEditDate="2019-04-12T03:14:16.773" LastActivityDate="2019-04-12T03:14:16.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31414" PostTypeId="2" ParentId="31406" CreationDate="2019-04-11T22:15:15.413" Score="1" Body="&lt;p&gt;Observe that any data-point &lt;span class=&quot;math-container&quot;&gt;$$x_i = \begin{bmatrix} x_{i1}\\x_{i2}\\x_{i3} \end{bmatrix}$$&lt;/span&gt; can be interpreted as the three-vector pointing from the origin to the point itself (position vector). &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For any two data points &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt;, form the cross-product vector &#xA;&lt;span class=&quot;math-container&quot;&gt;$${u}_{ij} = x_i \times x_j$$&lt;/span&gt;&#xA;Observe that the unique plane that passes through the origin and the two points &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x_j$&lt;/span&gt; is also the unique plane that passes through the origin and is orthogonal to &lt;span class=&quot;math-container&quot;&gt;$u_{ij}$&lt;/span&gt;. If the latter turns out to be zero, then get rid of one of the points.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each vector &lt;span class=&quot;math-container&quot;&gt;$u_{ij}$&lt;/span&gt;, create a pair of points  &lt;span class=&quot;math-container&quot;&gt;$\{- p_{ij}, \, p_{ij}\}$&lt;/span&gt; where&#xA;&lt;span class=&quot;math-container&quot;&gt;$$p_{ij} = \frac{u_{ij}}{\|u_{ij}\|}$$&lt;/span&gt;   The points &lt;span class=&quot;math-container&quot;&gt;$p_{ij}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$-p_{ij}$&lt;/span&gt; lie on the unit sphere &lt;span class=&quot;math-container&quot;&gt;$\mathbb{S}^2 = \{x \in \mathbb{R}^3 \, : \, \|x\| = 1 \}$&lt;/span&gt; and are antipodal. You obtain a set of points &lt;span class=&quot;math-container&quot;&gt;$\{p_{ij}\}$&lt;/span&gt; on the unit sphere, which are symmetric with respect to the antipodal symmetry. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This point it the difficult one. You need to choose a clustering algorithm for the points &lt;span class=&quot;math-container&quot;&gt;$\{p_{ij}\}$&lt;/span&gt; on the sphere, so that you can detect high accumulation ofpoints. To that end, you could use a convex hull algorithm for &lt;span class=&quot;math-container&quot;&gt;$\{p_{ij}\}$&lt;/span&gt;, which will give you the Delaunay decomposition of &lt;span class=&quot;math-container&quot;&gt;$\{p_{ij}\}$&lt;/span&gt; on the unit sphere. It's dual is the Voronoi (nearest neughbour) decomposition of &lt;span class=&quot;math-container&quot;&gt;$\{p_{ij}\}$&lt;/span&gt;. Then you can use this information in your clustering methodology.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As a result of the (non-trivial) step 3, you end up with a bunch of point-clusters &#xA;&lt;span class=&quot;math-container&quot;&gt;$N_1, \, N_2, \, ..., \, N_{2k}$&lt;/span&gt; (you should expect them to have the antipodal symmetry). Take only half of the clusters, say &lt;span class=&quot;math-container&quot;&gt;$N_1, \, N_2, \, ..., \, N_{2k}$&lt;/span&gt; so that no two clusters are antipodally symmetric, and ignore their antipodal images. Take the average point (unit) vector for each cluster &lt;span class=&quot;math-container&quot;&gt;$N_s$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$n_s = \frac{1}{|N_s|} \, \sum_{p_{ij} \in N_s} \, p_{ij}$$&lt;/span&gt; &#xA;The normal unit vectors &lt;span class=&quot;math-container&quot;&gt;$n_s$&lt;/span&gt; should be the normal vectors of the planes that you seek. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You may and up with some redundant planes, so you may want to get rid of some of them. For each plane, represented by a unit normal vector &lt;span class=&quot;math-container&quot;&gt;$n_s$&lt;/span&gt;, calculate the distances between the points and the planes, i.e. for each &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; calculate &lt;span class=&quot;math-container&quot;&gt;$h_{is} = |(x_i \circ n_s)|$&lt;/span&gt; and, say using some reasonable treshold, select only the points which are close enough to the plane of &lt;span class=&quot;math-container&quot;&gt;$n_s$&lt;/span&gt; (so those with small h_{is}), and project them onto the plane: &lt;span class=&quot;math-container&quot;&gt;$x_{is} = x_i - (x_i\circ n_s)\, n_s$&lt;/span&gt;. If the points &lt;span class=&quot;math-container&quot;&gt;$x_{is}$&lt;/span&gt; form a fairly well spread-out, diffuse, pattern on the plane &lt;span class=&quot;math-container&quot;&gt;$n_s$&lt;/span&gt;, you keep the plane. If however, the points &lt;span class=&quot;math-container&quot;&gt;$x_{is}$&lt;/span&gt; form something that looks very much like a smeared (cloudy), set of lines passing through the point of origin, which is on the plane, then you get rid of the plane.     &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You could also work in the projective plane instead of the unit sphere, it is just that the notion of distance might be more cumbersome, but I am not sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the other answers, the recommendation is something called &quot;Hough transform&quot;. Its philosophy however is not that much different from what I sketched above. I recommend the use of projective duality (like what I am using in the methodology outlined above) from projective geometry point of view. In two dimensions, it utilizes the construction that the family of all lines passing through a given point is equivalent to a (dual) line. Consequently, given two points &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt; in the plane, the family of all lines passing through the point &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; forms a dual line &lt;span class=&quot;math-container&quot;&gt;$l_1$&lt;/span&gt;, the family of all lines passing through the point &lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt; forms a dual line &lt;span class=&quot;math-container&quot;&gt;$l_2$&lt;/span&gt;, so the intersection of  the two (dual) lines &lt;span class=&quot;math-container&quot;&gt;$l_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$l_2$&lt;/span&gt; is a (dual) point, which represents the common line between two two original points &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_2$&lt;/span&gt;. This has the same philosophy as this &quot;Hough transform&quot; but is linear in nature, while the &quot;Hough transform&quot; uses curves, which I think is more cumbersome. Even with the Hough transform, you would need to find a way to detect accumulations/clusters of points.        &lt;/p&gt;&#xA;" OwnerUserId="20964" LastEditorUserId="20964" LastEditDate="2019-04-13T14:33:55.100" LastActivityDate="2019-04-13T14:33:55.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31415" PostTypeId="2" ParentId="31406" CreationDate="2019-04-12T10:46:02.280" Score="0" Body="&lt;p&gt;Like rchilton1980, I would suggest the Hough transform. In your case, &lt;em&gt;d&lt;/em&gt; does not to be quantized because it is known to be zero. When quantizing the two angles, beware that the naive quantization (&lt;span class=&quot;math-container&quot;&gt;$\phi + n\cdot\Delta\phi$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\theta +n\cdot\Delta\theta$&lt;/span&gt;) will result in an uneven angle distirbution. I would recommend to use the regularized quantiziation by Jeltsch (the title refers to line detection, but the quantization method applies to any 3D angle quantization):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Jeltsch et al.: &lt;a href=&quot;https://lionel.kr.hsnr.de/~dalitz/data/publications/hough3d-visapp16.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Hough Parameter Space Regularisation for Line Detection in 3D.&quot;&lt;/a&gt;&#xA;  VISAPP 2016, pp. 345-352 (2016) &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For improving the result, an orthogonal least squares fit (which is mathematically equivalent to take the two main components of a PCA) through the points of each plane is a good idea. The plane normal is then simply the eigenvector of teh covariance matrix corresponding to the &lt;em&gt;smallest&lt;/em&gt; eigenvalue:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Späth: &quot;Orthogonal least squares fitting with linear manifolds.&quot;&#xA;  Numerische Mathematik 48, pp. 441–445 (1986)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-04-12T10:46:02.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31416" PostTypeId="2" ParentId="31151" CreationDate="2019-04-12T11:08:35.643" Score="2" Body="&lt;p&gt;This is the classic contour tracking problem, which is usually sovled with an algorithm by Pavlidis described in&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Pavlidis: Algorithms for Grapics and Image Processing. pp. 129-165, Springer, 1982&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A ready to use python function is available as &lt;a href=&quot;https://gamera.informatik.hsnr.de/docs/gamera-docs/analysis.html#contour-pavlidis&quot; rel=&quot;nofollow noreferrer&quot;&gt;contour_pavlidis&lt;/a&gt; in the Gamera framework. The C++ code of this implementation can be &lt;a href=&quot;https://github.com/hsnr-gamera/gamera/blob/master/include/plugins/contour.hpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;found on github&lt;/a&gt;, which is subject to the GNU GPL license.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastActivityDate="2019-04-12T11:08:35.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31417" PostTypeId="2" ParentId="23570" CreationDate="2019-04-12T14:10:00.147" Score="1" Body="&lt;p&gt;I guess that you are not actually interested in the variance, but in a confidence interval for your observable &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. It should be noted that computing the confidence interval from the variance (i.e. &lt;span class=&quot;math-container&quot;&gt;$\hat{\theta}\pm 2\sigma(\theta)$&lt;/span&gt;) is only guaranteed to work when you estimate &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; with a Maximum-Likelihood estimator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with your &quot;block&quot; approach is, that it uses the data in an inefficient way: you only use &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; data points in each estimate and only have &lt;span class=&quot;math-container&quot;&gt;$(n/M)&amp;lt;&amp;lt;n$&lt;/span&gt; sampled variance estimators. More efficient estimates for the variance are the Jackknife or the Bootstrap, which are both resampling methods leading to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; sampled varaince estimators. The Jackknife is very simple to implement and would be an improvement over your &quot;block&quot; method. References:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Efron, Gong: &quot;A leisurely look at thebootstrap,  the  jackknife,&#xA;and  crossvalidation.&quot; The American Statistician 37, pp. 36–48 (1983)&lt;/li&gt;&#xA;&lt;li&gt;Dalitz: &lt;a href=&quot;https://lionel.kr.hsnr.de/~dalitz/data/publications/fb03-tb-2017-01-en.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Construction of confidence intervals.&quot;&lt;/a&gt;&#xA;Technical Report No. 2017-01, pp. 15-28, Hochschule Niederrhein, Fachbereich&#xA;Elektrotechnik und Informatik (2017)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="30365" LastEditDate="2019-04-12T14:17:10.657" LastActivityDate="2019-04-12T14:17:10.657" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31418" PostTypeId="1" CreationDate="2019-04-12T15:56:27.823" Score="3" ViewCount="83" Body="&lt;p&gt;The theta-method is a popular solution for solving time-transient PDEs (or ODEs), which consists of solving the general equation for each time step:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{u^{n+1} - u^{n}}{\Delta t} + (\theta f(u^{n+1}, t^{n+1}) + (1-\theta)f(u^{n}, t^{n})) = 0&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some values of &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; are often use and their pros and cons are well documented, notably:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta = 0$&lt;/span&gt; is purely explicit, and as such has a low computational cost&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta = 0.5$&lt;/span&gt; (AKA the Crank-Nicolson method) has a precision in &lt;span class=&quot;math-container&quot;&gt;$O(\Delta t^2)$&lt;/span&gt;, vs &lt;span class=&quot;math-container&quot;&gt;$O(\Delta t)$&lt;/span&gt; for other values&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\theta = 1$&lt;/span&gt; is unconditionally stable for oscillations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have also sometimes encountered the value &lt;span class=&quot;math-container&quot;&gt;$\theta = \frac{2}{3}$&lt;/span&gt;, referred as the Galerkin method, for example in [1] (p.1814), without any mention of its advantages over other values for &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;. The fact that the Galerkin method also refers to the much more popular method of transforming a PDE problem through a weak formulation doesn't help getting information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, on a specific problem I'm working on, I find that the best results are obtained with a value of &lt;span class=&quot;math-container&quot;&gt;$\theta = 0.9$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know of any resources or have more info about the use non-typical values for &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; in a theta-scheme? What are the possible advantages and caveats of such values?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[1] Dalhuijsen, A. J. and Segal, A. (1986), Comparison of finite element techniques for solidification problems. Int. J. Numer. Meth. Engng., 23: 1807-1829. doi:10.1002/nme.1620231003&lt;/p&gt;&#xA;" OwnerUserId="28047" LastActivityDate="2019-04-12T15:56:27.823" Title="Use of non-typical values of $\theta$ in theta-methods" Tags="|pde|finite-difference|crank-nicolson|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31420" PostTypeId="1" CreationDate="2019-04-12T18:03:01.733" Score="4" ViewCount="897" Body="&lt;p&gt;Solve the following minimization problem in &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X} \in \mathbb{R}^{m \times n}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{array}{ll} \text{minimize} &amp;amp; \frac 12 \| \mathbf{X}\mathbf{X}^T -\mathbf{A} \|^2_\mathcal{F}\\ \text{subject to} &amp;amp; \mathbf{X}^T \mathbf{X} = \mathbf{I}_n\end{array}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{I}_n$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$n \times n$&lt;/span&gt; identity matrix, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A} = \mathbf{B}\mathbf{B}^T$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{B} \in \mathbb{R}^{m\times n}$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was not sure how to write the Lagrangian and differentiate. Most probably, the final solution should be an eigenvalue problem but I am not sure how to arrive at that. Here is my attempt -&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{L}(\mathbf{X},\mathbf{Z})=\frac 12 \| \mathbf{X}\mathbf{X}^T -\mathbf{A} \|^2_\mathcal{F} + &amp;lt;\mathbf{Z},\mathbf{X}^T \mathbf{X}-\mathbf{I}&amp;gt;_\mathcal{F}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Z}$&lt;/span&gt; is the Lagrangian multiplier. Now differentiation with respect to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{Z}$&lt;/span&gt; yields &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}^T\mathbf{X} = \mathbf{I}$&lt;/span&gt;. Now to take differentiation with respect to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt; we expand the Lagrangian as follows -&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\mathcal{L}=\frac{1}{2}\text{Tr}((\mathbf{X}\mathbf{X}^T-\mathbf{A})^T(\mathbf{X}\mathbf{X}^T-\mathbf{A}))+ \text{Tr}(\mathbf{Z}^T(\mathbf{X}^T \mathbf{X}-\mathbf{I}))$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\text{or} \quad \mathcal{L}=\frac{1}{2}\text{Tr}(\mathbf{X}\mathbf{X}^T\mathbf{X}\mathbf{X}^T-\mathbf{X}\mathbf{X}^T\mathbf{A}-\mathbf{A}^T\mathbf{X}\mathbf{X}^T+\mathbf{Z}^T\mathbf{X}^T \mathbf{X})+ \mathcal{L}_0$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}_0$&lt;/span&gt; are terms not containing &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt;. Then I differentiate &lt;span class=&quot;math-container&quot;&gt;$\mathcal{L}$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}$&lt;/span&gt; in order to obtain the second set of equations. So, I was wondering if there are alternative ways to solve the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit : Using properties of trace operation and the constraint, I converted the problem into maximizing &lt;span class=&quot;math-container&quot;&gt;$ \text{Tr}(\mathbf{X}^T\mathbf{A}\mathbf{X})$&lt;/span&gt; with the same constraints.&lt;/p&gt;&#xA;" OwnerUserId="30451" LastEditorUserId="6910" LastEditDate="2019-04-14T21:23:42.760" LastActivityDate="2021-12-12T10:02:01.900" Title="How do I solve the matrix equality constrained optimization problem using Lagrangian multipliers?" Tags="|optimization|matrix|constrained-optimization|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="31421" PostTypeId="1" CreationDate="2019-04-13T00:43:58.343" Score="1" ViewCount="106" Body="&lt;p&gt;I &lt;a href=&quot;https://github.com/taktoa/chemsim/blob/master/src/lbm.rs&quot; rel=&quot;nofollow noreferrer&quot;&gt;wrote&lt;/a&gt; a Lattice Boltzmann Solver in Rust a little while back using both the BGK approximation and the TRT (two relaxation time) method on the D2Q9 lattice. In both cases I run into major stability problems even in simple examples. Here is a video of the output from the simulation of a left-to-right flow with a cylindrical obstruction (using the bounce-back method): &lt;a href=&quot;https://gfycat.com/idlebowedindochinesetiger&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;. This video shows momentum density using lightness as magnitude and color as direction. I can't remember if it was generated with BGK or TRT but I experienced similar failures with both collision operators. I'm using convolution with a &lt;code&gt;[[1, 1, 1], [1, 0, 1], [1, 1, 1]]&lt;/code&gt; stencil to implement the streaming step. I tried following the steps described in &lt;em&gt;The Lattice Boltzmann Method: Principles and Practice&lt;/em&gt; for getting stability in BGK and TRT simulations, but haven't had any luck. Am I doing something obviously wrong?&lt;/p&gt;&#xA;" OwnerUserId="30455" LastActivityDate="2019-04-13T00:43:58.343" Title="Instability in Lattice Boltzmann Solver" Tags="|fluid-dynamics|lattice-boltzmann-methods|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31422" PostTypeId="1" CreationDate="2019-04-13T14:48:19.250" Score="0" ViewCount="104" Body="&lt;p&gt;From the inviscid Burgers' equation: &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial u}{\partial t}+u\frac{\partial u}{\partial x} = 0$&lt;/span&gt;, I get the discretization &lt;span class=&quot;math-container&quot;&gt;$\frac{u_i^{n+1}-u_i^n}{\Delta t}+\frac{(u_i^{n+1})^2-(u_{i-1}^n)^2}{\Delta x}=0$&lt;/span&gt;. Having the starting conditions &lt;span class=&quot;math-container&quot;&gt;$u_0(x)=sin(\pi x)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x\in[0,1]$&lt;/span&gt;, I wish to plot &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; after &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; timesteps. However, I cannot seem to get it to work and would appreciate any help. Python code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import scipy as py&#xA;from scipy.sparse.linalg import gmres&#xA;&#xA;#x&#xA;x0 = 0 #x min&#xA;xe = 1 #x max&#xA;N = 100 #Nodes&#xA;dx = (xe-x0)/N &#xA;x = py.linspace(x0,xe,N)&#xA;&#xA;#t&#xA;t0 = 0 #time&#xA;te = 1 #max time&#xA;dt = 0.0001 #timestep&#xA;M = int((te-t0)/dt) #timesteps&#xA;&#xA;#u&#xA;u0 = py.sin(py.pi*x)&#xA;&#xA;ts = 50&#xA;&#xA;def EulerMethod(ustart):&#xA;    U = np.zeros((N,M))&#xA;    U[:,0] = ustart&#xA;    t=t0&#xA;    for i in range(N-1): #one step Euler forward&#xA;        U[0,1] = 0&#xA;        U[i+1,1] = U[i+1,0] - (dt/dx)*(U[i+1,0]**2-U[i,0]**2)&#xA;    for n in range(ts):&#xA;         U[:,n+2] = NewtonMethod(U[:,n+1],U[:,n]) #Eulers implicit method with Newtons Method        &#xA;         t=t+dt&#xA;    return U&#xA;&#xA;def NewtonMethod(u,c):&#xA;    #c = u_n&#xA;    for i in range(3):&#xA;        F = np.zeros((N,1))&#xA;        J = np.zeros((N,N))&#xA;        J[N-1,N-1] = 1/dt + u[N-1]/dx&#xA;        F[0] = (u[0] - c[0])/dt + (u[0]**2)/2*dx&#xA;        for j in range(N-1):&#xA;            J[j,j] = 1/dt + u[j]/dx&#xA;            J[j+1,j] = 1/dt + u[j]/dx&#xA;            F[j+1] = (u[j+1] - c[j+1])/dt + (u[j+1]**2 - u[j]**2)/2*dx&#xA;        du = gmres(J,-F)[0]&#xA;        u = u + du&#xA;    return u&#xA;&#xA;UM = EulerMethod(u0)&#xA;plt.plot(x,UM[:,ts])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29985" LastEditorUserId="29985" LastEditDate="2019-04-14T17:02:42.593" LastActivityDate="2019-04-14T17:02:42.593" Title="Need help applying Implicit Eulers Method together with Newtons Method on Burgers' Equation" Tags="|python|numerics|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="31424" PostTypeId="1" CreationDate="2019-04-14T01:42:21.703" Score="2" ViewCount="213" Body="&lt;p&gt;In many CFD text books, usually there is a dedicated chapter for advection term discretization. &#xA;Why discretization of such term in advection-dominated problems and near the discontinuities is challenging? I need some mathematical reasoning &#xA;And how this is linked to the problem of boundedness and numerical diffusion?    &lt;/p&gt;&#xA;" OwnerUserId="30461" LastActivityDate="2019-04-14T15:46:27.253" Title="Why the numerical solution of advection-dominant problem is challenging" Tags="|stability|numerics|advection|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31425" PostTypeId="2" ParentId="31420" CreationDate="2019-04-14T15:10:09.443" Score="4" Body="&lt;p&gt;I believe that there is a good alternative that benefits from the geometry of the parameter space and completely eliminates the need for constrained optimization. If you explicitly wanted to make use of Lagrangians, I will definitely not be answering the question, but I thought it might be worthwhile to consider the perspective I will describe.&#xA;In particular, the approach I will be presenting uses &lt;a href=&quot;http://www.cis.upenn.edu/%7Ecis610/Ring-Wirth-optim-Riemann.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Riemannian manifold optimization tools&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note that the parameter of interest &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}\in \mathbb{R}^{n \times k}$&lt;/span&gt; is simply an element of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Stiefel_manifold&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stiefel Manifold&lt;/a&gt; &lt;span class=&quot;math-container&quot;&gt;$\mathcal{M}\equiv V_k(\mathbb{R}^n)$&lt;/span&gt; (the set of all orthonormal &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-frames in &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}^n$&lt;/span&gt;):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;\mathbf{X} &amp;amp;\in V_k(\mathbb{R}^n) \subset \mathbb{R}^{n\times k}\\&#xA;V_k(\mathbb{R}^n) = \{\mathbf{X}&amp;amp;\in\mathbb{R}^{n\times k} : \mathbf{X}^\top\mathbf{X}=\mathbf{I}\}.&#xA;\end{align}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;With this definition, we can benefit from the Riemannian structure of &lt;span class=&quot;math-container&quot;&gt;$V_k(\mathbb{R}^n)$&lt;/span&gt; and optimize the energy with any &lt;strong&gt;unconstrained&lt;/strong&gt; optimization algorithm, such as the &lt;a href=&quot;https://arxiv.org/pdf/1011.0010.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Riemannian-gradient descent&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA; &amp;amp;\text{ While } \mathbf{X}_{k} \text{ does not sufficiently minimize } f \\&#xA; &amp;amp;\text{$\quad$-  Pick a gradient related descent direction }\boldsymbol{\eta}_k\in T_{\mathbf{X}_k}\mathcal{M}\\&#xA; &amp;amp;\text{$\quad$-  Choose a retraction } R_{\mathbf{X}_k}:T_{\mathbf{X}_k}\mathcal{M}\rightarrow \mathcal{M}.\\&#xA; &amp;amp;\text{$\quad$-  Choose a step length } \tau_k\in \mathbb{R}.\\&#xA; &amp;amp;\text{$\quad$-  Set } \mathbf{X}_{k+1}\gets R_{\mathbf{X}_k}(\tau_k\boldsymbol{\eta}_k).\\&#xA; &amp;amp;\text{$\quad$- } k\gets k+1&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; denotes the function we optimize:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f = \frac{1}{2}\|\mathbf{X}\mathbf{X}^\top - \mathbf{A} \|_\mathcal{F}^2 = \frac{1}{2}\text{tr}\big((\mathbf{X}\mathbf{X}^\top-\mathbf{A})(\mathbf{X}\mathbf{X}^\top-\mathbf{A})^\top\big)&#xA;$$&lt;/span&gt;&#xA;whose gradient reads:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\nabla_{\mathbf{X}}f = (\mathbf{X} \mathbf{X}^\top -\mathbf{A})\mathbf{X}+(\mathbf{X}\mathbf{X}^\top +(-\mathbf{A})^\top )\mathbf{X}.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We generally pick the gradient related direction &lt;span class=&quot;math-container&quot;&gt;$\boldsymbol{\eta}$&lt;/span&gt; as the projection of the Euclidean gradient onto the tangent space of the manifold (or for a broad class of manifolds including the Stiefel manifold, we can instead use the logarithmic-map):&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\boldsymbol{\eta}\triangleq \text{grad }_{\mathbf{X}} f = \Pi_{\mathbf{X}} \Big( -\nabla_{\mathbf{X}} f \Big)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For now let us assume that &lt;span class=&quot;math-container&quot;&gt;$\tau_k$&lt;/span&gt; is fixed, &lt;em&gt;e.g.&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\tau_k=0.1$&lt;/span&gt;.&#xA;The &lt;em&gt;retraction operator&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$R(\cdot)$&lt;/span&gt; of the Stiefel manifold is analytically defined or in other words the &lt;em&gt;true&lt;/em&gt; exponential map is available. More details on that are given in &lt;a href=&quot;https://math.stackexchange.com/questions/1371410/geodesic-of-stiefel-manifold/2841306#2841306&quot;&gt;this math-se post&lt;/a&gt; and of course in the &lt;a href=&quot;https://arxiv.org/abs/physics/9806030&quot; rel=&quot;nofollow noreferrer&quot;&gt;seminal paper of Edelman et. al&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Edelman, Alan, Tomás A. Arias, and Steven T. Smith. &amp;quot;The geometry of&#xA;algorithms with orthogonality constraints.&amp;quot; SIAM journal on Matrix&#xA;Analysis and Applications 20.2 (1998): 303-353.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;All the Riemannian operators for Stiefel manifold are included in the toolboxes such as &lt;a href=&quot;https://www.manopt.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manopt&lt;/a&gt; or &lt;a href=&quot;https://github.com/whuang08/ROPTLIB&quot; rel=&quot;nofollow noreferrer&quot;&gt;ROPTLIB&lt;/a&gt;. Finally, we can update the obtain the new value &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}_{k+1}$&lt;/span&gt; by:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\mathbf{X}_{k+1} = R_{\mathbf{X}_k}(\tau_k \boldsymbol{\eta}_k)&#xA;\end{align}&lt;/span&gt;&#xA;Note that &lt;span class=&quot;math-container&quot;&gt;$\mathbf{X}_{k+1}$&lt;/span&gt; will be an orthonormal &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-frame obeying the aforementioned constrained naturally.&#xA;Riemennian gradient descent is the simplest choice of Riemannian optimization and there are many others such as &lt;a href=&quot;https://arxiv.org/abs/1904.05814&quot; rel=&quot;nofollow noreferrer&quot;&gt;Riemannian-LBFGS&lt;/a&gt; or &lt;a href=&quot;https://sites.uclouvain.be/absil/Publi/RTR/RTR_full25.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Riemannian-Trust Region&lt;/a&gt;. Many of those choices vary in how they compute the step size &lt;span class=&quot;math-container&quot;&gt;$\tau_k$&lt;/span&gt;, usually by a form of line-search.&lt;/p&gt;&#xA;&lt;p&gt;More recently, &lt;a href=&quot;https://arxiv.org/pdf/1809.00452.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hu et al.&lt;/a&gt; considered a very similar problem to the one of this questions where the minimization with orthonormality constraints (again, Stiefel manifold) is made efficient. This method too, uses the Riemannian structure of the problem.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2021-12-12T10:02:01.900" LastActivityDate="2021-12-12T10:02:01.900" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31426" PostTypeId="2" ParentId="31424" CreationDate="2019-04-14T15:33:27.837" Score="4" Body="&lt;p&gt;The difficulty is relative to something, in this case it is relative to diffusion dominated problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Diffusion dominated aren't &quot;easy&quot; either, they have their own set of problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll start with some favorable qualities of diffusion problems, and then mention why they are not present for advection problems:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Discretizations are often symmetric and definite. If they aren't symmetric then they still are usually definite. - this enables use of better linear solvers in some cases.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Smoothness of solutions is often guaranteed under very relaxed assumptions on input data (source term, boundary conditions). This isn't &lt;em&gt;always&lt;/em&gt; true, but it often is.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Scale separation: it's often possible to separate small-scale behavior from large-scale behavior. This allows preconditioners/solvers like multigrid where different scales of physics are targeted by different levels of the preconditioner.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now let's look at advection problems:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Discretizations rarely result in symmetric and definite matrix. Can potentially be indefinite.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Much fewer smoothness guarantees. This is problem dependent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(almost) impossible to numerically separate scales. Stability requires you always resolve the smallest scale, which is often expressed as a relationship between grid spacing and advection speed. This means classic multigrid strategies won't work because the coarse grid will be aliased. In some cases this isn't strictly true because the &quot;fine grid&quot; may be very over-resolved compared to advection speed because of things like discontinuous coefficients, and in those cases multigrid may work, but only if the coarse grid respects the stability conditions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But in other respects, advection problems can actually be &lt;em&gt;easier&lt;/em&gt; than diffusion problems. It really depends on what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;time-dependent Advection problems are usually easier to get time-accurate solutions for than diffusion problems, because they often only involve first derivatives and this results in favorable CFL condition (again, depending only on advection speed). You start to lose this advantage when your discretization has variable order of accuracy, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use implicit time-stepping then the matrix conditioning is often better than a similarly discretized diffusion problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I should point out that I haven't touched on nonlinear problems here. That creates its own set of unique challenges which I'm not qualified to comment on.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2019-04-14T15:46:27.253" LastActivityDate="2019-04-14T15:46:27.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31427" PostTypeId="1" CreationDate="2019-04-14T22:52:19.553" Score="1" ViewCount="165" Body="&lt;p&gt;In particular, if Euler's method is implemented on a computer, what's the minimum step size that can be used before rounding errors cause the Euler approximations to become completely unreliable?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I presume it's when step size reaches the machine epsilon? E.g. if machine epsilon is e-16, then once step size is roughly e-16, the Euler approximations are unreliable.&lt;/p&gt;&#xA;" OwnerUserId="30466" LastActivityDate="2019-04-15T18:21:38.790" Title="What's the minimum step size that can be used in Euler's method before it becomes unreliable?" Tags="|ode|integration|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31430" PostTypeId="2" ParentId="30113" CreationDate="2019-04-15T13:09:19.847" Score="2" Body="&lt;p&gt;This is a typical use case for a &lt;em&gt;paired t-test&lt;/em&gt;. The idea is to consider only the runtime &lt;em&gt;difference&lt;/em&gt; &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; for each problem and test for the null hypothesis &lt;span class=&quot;math-container&quot;&gt;$E(\Delta t)=0$&lt;/span&gt;. For a step-by-step explanation, see e.g. (the article refers to segmentation evaluation, but on an abstract level the problem is identiclal to yours):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Mao, Kanungo: &quot;Empirical performance evaluation methodology and its application to page segmentation algorithms&quot;. IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (3), pp. 242-256 (2001)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="24110" LastEditDate="2019-09-13T20:41:07.563" LastActivityDate="2019-09-13T20:41:07.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31432" PostTypeId="1" CreationDate="2019-04-15T13:38:34.583" Score="3" ViewCount="4706" Body="&lt;p&gt;I'm running a simulation, and some linear solvers are returning a message of ill-conditioned matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, I'm looking for a fast, easy to implement, method to detect if a matrix is ill-conditioned, before using the linear solver. And in case it's ill-conditioned, what's good way to make it well-conditioned?&lt;/p&gt;&#xA;" OwnerUserId="24970" LastActivityDate="2019-05-06T13:17:53.940" Title="A fast way to check if a Matrix is ill-conditioned, and turning it into well-conditioned" Tags="|linear-algebra|linear-solver|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="31433" PostTypeId="2" ParentId="23403" CreationDate="2019-04-15T13:40:47.567" Score="6" Body="&lt;p&gt;With the substitution &lt;span class=&quot;math-container&quot;&gt;$y=x+t$&lt;/span&gt;, the &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;-dependency can be factored out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$I(t)=\int_{-\infty}^\infty e^{-(y-t)}F(y)\;dy = e^t \int_{-\infty}^\infty  e^{-y}F(y)\;dy = C\cdot e^t$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This still leaves you with the problem of computing &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, but it is merely a constant number and the &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;-dependency is entirely described by the factor &lt;span class=&quot;math-container&quot;&gt;$e^t$&lt;/span&gt;. The two-sided Laplace transfrom of this function and its region of convergence (if it is non-empty) should be found in a table of Laplace transforms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;On a second thought, I doubt that the two-sided Laplace transform of &lt;span class=&quot;math-container&quot;&gt;$I(t)$&lt;/span&gt; exists even for purely imaginary &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;, because &lt;span class=&quot;math-container&quot;&gt;$I(t)\sim e^t$&lt;/span&gt; is not a tempered distribution.&lt;/p&gt;&#xA;" OwnerUserId="30365" LastEditorUserId="30365" LastEditDate="2019-04-15T15:21:55.947" LastActivityDate="2019-04-15T15:21:55.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31434" PostTypeId="1" CreationDate="2019-04-15T13:52:34.147" Score="1" ViewCount="193" Body="&lt;p&gt;I would like to make a visualization with ParaView for a paper. In order to have a high quality look, I would like the image to be, say, 300 dpi (or vector graphics). Moreover, since I know the column width of the paper, I'd like to somehow specify that width in ParaView so I can adjust the size of the visualization, colorbars, etc., to make it fit within the column. I'd also like to specify the text size and font to be the same as in LaTeX. In other words, I do not want to rescale the image afterwards, because that makes the text too small or too large. In other programs I usually figure out how to do this, but so far no luck in ParaView. Even if I set the font to, say, 10 pt, it turns out way too small when I include the figure in LaTeX with &lt;code&gt;scale=1&lt;/code&gt;. Any help appreciated.&lt;/p&gt;&#xA;" OwnerUserId="26959" LastActivityDate="2019-04-15T13:52:34.147" Title="Properly sizing ParaView plots for LaTeX" Tags="|visualization|paraview|" AnswerCount="0" CommentCount="4" ClosedDate="2019-05-05T10:07:41.133" ContentLicense="CC BY-SA 4.0" />
  <row Id="31436" PostTypeId="1" AcceptedAnswerId="31454" CreationDate="2019-04-15T15:51:32.700" Score="4" ViewCount="1128" Body="&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to convert the general second order linear PDE problem &#xA;  &lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\begin{cases} &#xA;a(x,y)\frac{\partial^2 u}{\partial x^2}+b(x,y) \frac{\partial^2 u}{\partial y^2} +c(x,y)\frac{\partial^2 u}{\partial x \partial y}\\+d(x,y)\frac{\partial u}{\partial x}+e(x,y)\frac{\partial u}{\partial y}+f(x,y)u=g(x,y) &amp;amp; \text{in } R \text{ PDE} \\&#xA;u=u^* &amp;amp; \text{on } S_1 \text{ Dirchlet boundary condition} \\&#xA;\dfrac{\partial u}{\partial n}=q^* &amp;amp; \text{on } S_2 \text{ Neumann boundary condition} \\&#xA;\dfrac{\partial u}{\partial n}=r^*_1-r^*_2 u &amp;amp; \text{on } S_3 \text{ Robin boundary condition} \\&#xA;\end{cases} &#xA;\end{align}&lt;/span&gt;&#xA;  into a weak form suitable for the finite element method. That is into the weak bilinear form &lt;span class=&quot;math-container&quot;&gt;$B(u,v)=L(v)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is bilinear, symmetric and positive definite functional and &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is a linear functional.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Work thus far&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know to how convert the following&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\begin{cases} &#xA;\dfrac{\partial^2 u}{\partial x^2}+\dfrac{\partial^2 u}{\partial y^2}+u=g(x,y) &amp;amp; \text{in } R \text{ PDE} \\&#xA;u=u^* &amp;amp; \text{on } S_1 \text{ Dirchlet boundary condition} \\&#xA;\dfrac{\partial u}{\partial n}=q^* &amp;amp; \text{on } S_2 \text{ Neumann boundary condition} \\&#xA;\dfrac{\partial u}{\partial n}=r^*_1-r^*_2 u &amp;amp; \text{on } S_3 \text{ Robin boundary condition} \\&#xA;\end{cases} &#xA;\end{align}&lt;/span&gt;&#xA;into the weak bilinear form &lt;span class=&quot;math-container&quot;&gt;$B(u,v)=L(v)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is bilinear, symmetric and positive definite and &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is linear. The steps are as follows (note that &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt; is our test function)&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\int \int_{R} \left(\frac{\partial^2 u}{\partial x^2}+\frac{\partial^2 u}{\partial y^2}+u \right) v  \ dA &amp;amp;= \int \int_{R} g(x,y)  v  \ dA &#xA;\end{align}&lt;/span&gt;&#xA;Using the identity&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\int \int_{R} v \nabla^2 u\ dA &amp;amp;= \int_{S} v \frac{\partial u}{\partial n}\ ds-\int\int_{R} \nabla u \cdot \nabla v\ dA&#xA;\end{align}&lt;/span&gt;&#xA;We get&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\int \int_R -\nabla u \cdot \nabla v +uv  \ dA  &amp;amp;= \int \int_R g  v  \ dA - \int_{S} v \frac{\partial u}{\partial n}\ ds \\&#xA;\int \int_R -\nabla u \cdot \nabla v  +uv  \ dA  &amp;amp;= \int \int_R g  v  \ dA - \int_{S_1} v \frac{\partial u}{\partial n}\ ds- \int_{S_2} v \frac{\partial u}{\partial n}\ ds - \int_{S_3} v \frac{\partial u}{\partial n}\ ds \\&#xA;\int \int_R -\nabla u \cdot \nabla v  +uv  \ dA  &amp;amp;= \int \int_R g  v  \ dA - \int_{S_2} v q^* \ ds - \int_{S_3} v (r^*_1-r^*_2 u) \ ds \\&#xA;\int \int_R -\nabla u \cdot \nabla v  +uv  \ dA  &amp;amp;= \int \int_R g  v  \ dA - \int_{S_2} v q^* \ ds - \int_{S_3} v r^*_1\ ds  +\int_{S_3} r^*_2 uv \ ds \\&#xA;\int \int_R -\nabla u \cdot \nabla v  +uv  \ dA +\int_{S_3} r^*_2 uv \ ds  &amp;amp;= \int \int_R g  v  \ dA - \int_{S_2} v q^* \ ds - \int_{S_3} v r^*_1\ ds  \\&#xA;B(u,v)&amp;amp;=L(v)&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Where I am having trouble&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not know what to do with the terms &#xA;&lt;span class=&quot;math-container&quot;&gt;$$c(x,y)\frac{\partial^2 u}{\partial x \partial y}+d(x,y)\frac{\partial u}{\partial x}+e(x,y)\frac{\partial u}{\partial y}$$&lt;/span&gt;&#xA;as using the divergence theorem/integration by parts used in the &lt;em&gt;work thus far section&lt;/em&gt; leaves terms that are not symmetric and therefore does not not satisfy the requirements for &lt;span class=&quot;math-container&quot;&gt;$B(u,v)$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other problem are the terms &#xA;&lt;span class=&quot;math-container&quot;&gt;$$a(x,y)\frac{\partial^2 u}{\partial x^2}+b(x,y) \frac{\partial^2 u}{\partial y^2}$$&lt;/span&gt; the identity that I used in the &lt;em&gt;work thus far section&lt;/em&gt; does not work (I am probably wrong on this part). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could really use some guidance on both of these problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This question is part of a much larger problem in which I have to use the finite element method. Once the problem is in a weak form in which the finite element/galerkin method can be applied I know what to do. From what I know the symmetry of &lt;span class=&quot;math-container&quot;&gt;$B(u,v)$&lt;/span&gt; is essential. If there is some other weak form that works with the finite element (that is suitable for a numerical solution), that would be an acceptable answer to my problem.&lt;/li&gt;&#xA;&lt;li&gt;I have been following &quot;Finite Elements: A Gentle Introduction&quot; I could not find anything in the book that answered the problem. If you have any references that covers my problem that would be great (so far I have found nothing).&lt;/li&gt;&#xA;&lt;li&gt;If you have any questions feel free to ask.&lt;/li&gt;&#xA;&lt;li&gt;I originally posted this question on &lt;a href=&quot;https://math.stackexchange.com/q/3186878/374907&quot;&gt;math stack exchange&lt;/a&gt;. I reposted the question here as it relevant and could bring more interest to my problem. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Notation&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the vector normal to the boundary surface.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$u(x,y)$&lt;/span&gt; is the solution to the given PDE or ODE. &lt;span class=&quot;math-container&quot;&gt;$v(x,y)$&lt;/span&gt; is a test function.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\int \int_{R} * \ dA$&lt;/span&gt; is an integral over region &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\int_{S} * ds$&lt;/span&gt; is a surface integral over &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$u^*, q^*, r^*_1, r^*_2, r^*_3$&lt;/span&gt; are either constants or functions used to define the boundary conditions. &lt;/li&gt;&#xA;&lt;li&gt;The surface (S) boundary conditions can be divided into Dirchlet, Neumann, and Robin boundary conditions. That is &lt;span class=&quot;math-container&quot;&gt;$S=S_1\cup S_2 \cup S_3$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="30468" LastEditorUserId="549" LastEditDate="2019-04-17T21:05:49.670" LastActivityDate="2022-01-17T14:57:30.097" Title="Is a symmetric bilinear form necessary to ensure a weak formulation has a solution?" Tags="|finite-element|pde|galerkin|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="31437" PostTypeId="2" ParentId="31427" CreationDate="2019-04-15T18:21:38.790" Score="3" Body="&lt;p&gt;You can do this computationally for both one-sides (first order similar to Euler) and center (second order) differences easily using the code below.  Basically the error due to floating point takes over much sooner than machine epsilon would suggest even for double precision.  The case is even worse for single precision of course (you can change the below code by changing the &lt;code&gt;delta_x&lt;/code&gt; line to &lt;code&gt;delta_x = numpy.logspace(-17, 1.0, 100, dtype=numpy.float16)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;import matplotlib.pyplot as plt&#xA;&#xA;delta_x = numpy.logspace(-17, 1.0, 100)&#xA;x = 1.0&#xA;f_hat_1 = (numpy.exp(x + delta_x) - numpy.exp(x)) / (delta_x)&#xA;f_hat_2 = (numpy.exp(x + delta_x) - numpy.exp(x - delta_x)) / (2.0 * delta_x)&#xA;&#xA;fig = plt.figure()&#xA;axes = fig.add_subplot(1, 1, 1)&#xA;axes.loglog(delta_x, numpy.abs(f_hat_1 - numpy.exp(1)), 'o-', label=&quot;One-Sided&quot;)&#xA;axes.loglog(delta_x, numpy.abs(f_hat_2 - numpy.exp(1)), 's-', label=&quot;Centered&quot;)&#xA;axes.legend(loc=3)&#xA;axes.set_xlabel(&quot;&lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt;&quot;)&#xA;axes.set_ylabel(&quot;Absolute Error&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kP5jy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kP5jy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="567" LastActivityDate="2019-04-15T18:21:38.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31438" PostTypeId="1" CreationDate="2019-04-15T22:36:34.337" Score="0" ViewCount="49" Body="&lt;p&gt;Can you tell me these equations come from where in MATLAB fmincg.m?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A = 6*(f2-f3)/z3+3*(d2+d3);  % cubic fit&#xA;B = 3*(f3-f2)-z3*(d3+2*d2);&#xA;z2 = (sqrt(B*B-A*d2*z3*z3)-B)/A;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is either cubic extrapolation or interpolation.&#xA;I have looked for these equations in many websites and books of numerical optimization but but I do not find these equations even in the book &quot;Numerical Optimization by Peter Glynn, Stephen M. Robinson p:57&quot;. It is not same in the fmincg.m&lt;/p&gt;&#xA;" OwnerUserId="30483" LastEditorUserId="9667" LastEditDate="2019-04-16T13:38:55.597" LastActivityDate="2019-04-16T13:38:55.597" Title="extrapolation/interpolation in fmincg.m" Tags="|interpolation|extrapolation|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="31440" PostTypeId="1" CreationDate="2019-04-16T08:58:09.200" Score="0" ViewCount="144" Body="&lt;p&gt;Our &lt;a href=&quot;https://clbin.com/C7Jcz&quot; rel=&quot;nofollow noreferrer&quot;&gt;data set&lt;/a&gt; has &lt;span class=&quot;math-container&quot;&gt;$10^4$&lt;/span&gt; data points, but has a long baseline and many gaps. As the histogram shows, the horizontal-axis is time and most of the time, there are no data. The vertical-axis is data counts. The total time is, in fact, short (green lines), but gaps make the time baseline very long.&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BIWcI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BIWcI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we bin the data, there would be &lt;span class=&quot;math-container&quot;&gt;$10^8$&lt;/span&gt; data points&lt;code&gt;[t,value]&lt;/code&gt;, but only about &lt;span class=&quot;math-container&quot;&gt;$1\%$&lt;/span&gt; are non-zero values. After binning, most of those values are zero because of those gaps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to improve detection efficiency(a faster method)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A multi-threading way is possible(especially for Lomb-Scargle)?&lt;/p&gt;&#xA;" OwnerUserId="30488" LastEditorUserId="30488" LastEditDate="2019-04-16T14:30:40.417" LastActivityDate="2023-12-22T20:06:00.870" Title="How to improve the efficiency of periodicity detection for long time based lined and gapped datasets" Tags="|python|data-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31441" PostTypeId="2" ParentId="25323" CreationDate="2019-04-16T14:31:05.983" Score="0" Body="&lt;p&gt;What I would do (have done) if I had to solve a nonlinear time stepping problem..&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do time discretization in the strong form of PDE (use Newmark or whatever you want to)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find the weak form of the time-discretized strong form of the PDE to find &lt;span class=&quot;math-container&quot;&gt;$u^{n+1}$&lt;/span&gt;. You know &lt;span class=&quot;math-container&quot;&gt;$u^n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$u^{n-1}$&lt;/span&gt; from initial conditions.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Linearize the nonlinear system for &lt;span class=&quot;math-container&quot;&gt;$u^{n+1}$&lt;/span&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each Time step to find &lt;span class=&quot;math-container&quot;&gt;$u^{n+1}$&lt;/span&gt; -  Take a newton raphson guess (preferably &lt;span class=&quot;math-container&quot;&gt;$u^n$&lt;/span&gt;, so your guess is very close to your solution and don't diverge) and impose the Dirichlet boundary condition on it. Since you are only solving for free degrees of freedom, you will not touch the solution at the dirichlet nodes. If you have Neumann condition, it will occur naturally in the weak form.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerDisplayName="user30489" LastActivityDate="2019-04-16T14:31:05.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31442" PostTypeId="1" CreationDate="2019-04-16T16:20:22.900" Score="0" ViewCount="496" Body="&lt;p&gt;In our assignment, we are required to find the energies of the ground state and the first two excited states of the Schrödinger equation in a harmonic potential:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$V = \frac{50 x^2}{(10^{-11})^2}\, .$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the Python code for solving Schrödinger equation using secant Runge Kutta method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;m = 9.1094e-31&#xA;hbar = 1.0546e-34&#xA;e = 1.6022e-19&#xA;L = 5.2918e-11&#xA;N = 1000&#xA;h = L/N&#xA;&#xA;def V1(x):&#xA;    return 0.0&#xA;    #return (50*x**2)/(10**-11)**2&#xA;&#xA;def V2(x):&#xA;    return 0.0&#xA;    #return (50*x**4)/(10**-11)**4&#xA;&#xA;def f(r, x, E, V):&#xA;    psi = r[0]&#xA;    phi = r[1]&#xA;    fpsi = phi&#xA;    fphi = (2*m/hbar**2)*(V(x)-E)*psi&#xA;    return np.array([fpsi,fphi], float)&#xA;&#xA;def solve(E, V):&#xA;    psi = 0.0&#xA;    phi = 1.0&#xA;    r = np.array([psi, phi], float)&#xA;&#xA;    for x in np.arange(0,L,h):&#xA;        k1 = h*f(r, x, E, V)&#xA;        k2 = h*f(r+0.5*k1, x+0.5*h, E, V)&#xA;        k3 = h*f(r+0.5*k2, x+0.5*h, E, V)&#xA;        k4 = h*f(r+k3, x+h, E, V)&#xA;        r += (k1 + 2*k2 + 2*k3 + k4)/6&#xA;    return r[0]&#xA;&#xA;print(&quot;Start program&quot;)&#xA;&#xA;E1 = 0.0&#xA;E2 = e&#xA;psi2 = solve(E1, V1)&#xA;&#xA;target = e/1000&#xA;while abs(E1-E2)&amp;gt;target:&#xA;    psi1, psi2 = psi2, solve(E2, V1)&#xA;    E1, E2 = E2, E2-psi2*(E2-E1)/(psi2-psi1)&#xA;&#xA;print(&quot;For V(x) = V0x^2/a^2:&quot;)&#xA;print(&quot;E=&quot;, E2/e, &quot;eV&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, there are two problems that arise that causes me not being able to find the solution to this assignment. And I hope that someone can help me with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, when the function V1(x) returns 0.0, the program accurately outputs the energy of ground state for V1(x) = 0, which is E= 134.28637169369105 eV. However, when we change to V1(x) = V = (50x^2)/(10^-11)^2, which is what our assignment requires us to find, the program creates the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C:/Users/wormw/Desktop/PHYS 3142 - Computational Methods in Physics/Assignment 5/Assignment 5.py:29: RuntimeWarning: overflow encountered in double_scalars&#xA;  return np.array([fpsi,fphi], float)&#xA;C:/Users/wormw/Desktop/PHYS 3142 - Computational Methods in Physics/Assignment 5/Assignment 5.py:54: RuntimeWarning: invalid value encountered in double_scalars&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the output of E is E = nan eV. So, I have no idea why this would happen and how to fix it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, the assignment also requires us to find the first two excited state, but I have no idea how to modify the code to obtain the eigenvalue for getting the excited states&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thirdly, the assignment also states that Try using x = -10a to +10a, with the wave function psi = 0 at both boundaries. Should I change the value of L at the start?&lt;/p&gt;&#xA;" OwnerUserId="30296" LastEditorUserId="9667" LastEditDate="2019-04-16T19:36:19.320" LastActivityDate="2019-04-16T19:36:19.320" Title="Finding second excited state of Schrödinger equation with secant Runge Kutta method" Tags="|python|eigenvalues|runge-kutta|quantum-mechanics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31443" PostTypeId="1" CreationDate="2019-04-16T16:28:59.130" Score="1" ViewCount="87" Body="&lt;p&gt;I have an integer linear programming problem of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\DeclareMathOperator{\tr}{tr} \min \tr WX$$&lt;/span&gt; subject to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;\sum_j X_{ij} &amp;lt; c_i &amp;amp;&amp;amp; \forall i \\&#xA;\sum_i X_{ij} = 1 &amp;amp;&amp;amp; \forall j \\&#xA;X_{ij} \in \{0,1\} &amp;amp;&amp;amp; \forall i,j&#xA;\end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure this is probably a common problem that's been well studied, but I can't find the name for it.  Does anyone know?&lt;/p&gt;&#xA;" OwnerUserId="1997" LastActivityDate="2019-04-18T23:40:51.853" Title="Is there a name for this integer linear optimization problem?" Tags="|optimization|linear-programming|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31444" PostTypeId="2" ParentId="31443" CreationDate="2019-04-16T19:34:52.447" Score="3" Body="&lt;p&gt;This is a minimum cost network flow problem. Construct the network as follows.  You haven't specified the ranges for &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, but I'll assume that we &lt;span class=&quot;math-container&quot;&gt;$i=1, 2, \ldots, m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j=1, 2, \ldots, n$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A source node &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; sourcing &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; units of flow.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; nodes &lt;span class=&quot;math-container&quot;&gt;$u_{i}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$i=1, 2, \ldots, m$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Arcs from &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$u_{i}$&lt;/span&gt; with capacity &lt;span class=&quot;math-container&quot;&gt;$c_{i}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$i=1, 2, \ldots m$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; nodes &lt;span class=&quot;math-container&quot;&gt;$v_{j}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$j=1, 2, \ldots, n$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Arcs &lt;span class=&quot;math-container&quot;&gt;$(u_{i},v_{j})$&lt;/span&gt;, for all &lt;span class=&quot;math-container&quot;&gt;$i,j$&lt;/span&gt;, each with capacity 1.&lt;/li&gt;&#xA;&lt;li&gt;A node &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; that is a sink for &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; units of flow.&lt;/li&gt;&#xA;&lt;li&gt;Arcs from &lt;span class=&quot;math-container&quot;&gt;$v_{j}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; with capcity one for &lt;span class=&quot;math-container&quot;&gt;$j=1, 2, \ldots, n$&lt;/span&gt;.  &lt;/li&gt;&#xA;&lt;li&gt;Weights &lt;span class=&quot;math-container&quot;&gt;$W_{j,i}$&lt;/span&gt; on the arcs from &lt;span class=&quot;math-container&quot;&gt;$u_{i}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$v_{j}$&lt;/span&gt;.  Weights of 0 on the remaining arcs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any feasible flow on this graph corresponds to a solution of the LP relaxation of the above problem.  Since the constraint matrix of the LP is totally-unimodular, any optimal BFS has integer values for all of the variables &lt;span class=&quot;math-container&quot;&gt;$X_{i,j}$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can solve this problem using the simplex method or by using a specialized algorithm for the minimum cost network flow problem.&lt;/p&gt;&#xA;" OwnerUserId="2150" LastEditorUserId="2150" LastEditDate="2019-04-18T23:40:51.853" LastActivityDate="2019-04-18T23:40:51.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31447" PostTypeId="1" AcceptedAnswerId="32620" CreationDate="2019-04-17T12:28:27.493" Score="0" ViewCount="299" Body="&lt;p&gt;I have a &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;-dimensional grid, known at run, not compile-time, that has &lt;span class=&quot;math-container&quot;&gt;$50$&lt;/span&gt; points in each direction and hence &lt;span class=&quot;math-container&quot;&gt;$50^3$&lt;/span&gt; combinations that I would like to first build and then call a function with each point as its single argument, to store the output in a data structure that has a pointer to the argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good way to implement this in C++ by using perhaps a library that is optimised to handle such tasks? Ideally with good bindings to Eigen but I can hack that together myself if need be. More formally:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is now to populate the vector of grid points with each lambda ranging from &lt;span class=&quot;math-container&quot;&gt;$0.95$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; in steps of &lt;span class=&quot;math-container&quot;&gt;$0.001$&lt;/span&gt; so that I have a model of the Cartesian coordinate system &lt;span class=&quot;math-container&quot;&gt;$[0.95,1] \times [0.95,1] \times[0.95,1] \in \mathbb{R}^{3}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first attempt at a solution would be this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class grid_point{&#xA;grid_point(int q){&#xA;&#xA;lambda = VectorXf::Zeros(q);&#xA;}&#xA;&#xA;private:&#xA;VectorXd lambda;&#xA;float likelihood;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then instantiate &lt;code&gt;vector&amp;lt;grid_point&amp;gt; my_grid(num_steps);&lt;/code&gt; where &lt;code&gt;num_steps = pow(50,3)&lt;/code&gt;. I suppose this question &lt;a href=&quot;https://stackoverflow.com/questions/12991758/creating-all-possible-k-combinations-of-n-items-in-c&quot;&gt;here&lt;/a&gt; is similar but it implements a bunch of nested &lt;code&gt;for&lt;/code&gt; loops. I am wondering if there are packages that implement this natively possibly as part of the STL or in some custom package that people on this site are using. Performing grid search must be a well-trodden path in many disciplines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I may be able to use the library &lt;code&gt;&amp;lt;algorithm&amp;gt;&lt;/code&gt; but to me this feels like I am reinventing the wheel as I am sure there must be more efficient ways to&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;generate all tuples that span my model of &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}&#xA;^{3}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;and then call a function with a single parameter on it to write its output into &lt;code&gt;grid_point&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Many thanks, All!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seems like this question is not getting much interest. But here is an update on what can be done:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current sense is to build it from scratch as above and somehow try to find use of this methodology here &lt;a href=&quot;http://www.cplusplus.com/reference/algorithm/next_permutation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29045" LastEditorUserId="29045" LastEditDate="2019-04-19T10:41:34.413" LastActivityDate="2019-05-10T04:00:52.557" Title="Good C, C++ library for efficient grid search / tuples, ideally with bindings to Eigen" Tags="|finite-difference|c++|grid|eigen|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31449" PostTypeId="2" ParentId="31440" CreationDate="2019-04-17T14:00:00.750" Score="0" Body="&lt;p&gt;One way to approach it, could  be to do the analysis in two steps. First, you do a sweep along your dataset, and collect all nonzero values, and their times into a (much shorter!) data structure. You collect a list of tuples [t,value] basically. &#xA;Every sweep after that will be extremely fast as you can safely assume every datapoint not in your list to be zero. Clicking on your link I have a hard time understanding what I see. These are not simple csv-style datapoints. They are in the form of: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;139459196.2742752731&lt;/p&gt;&#xA;&#xA;&lt;p&gt;139462208.5806673169&lt;/p&gt;&#xA;&#xA;&lt;p&gt;139462689.1677284241&lt;/p&gt;&#xA;&#xA;&lt;p&gt;139467485.6161292493&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these just your times and the second number is omitted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do you want to go about the periodicity detection? The obvious way here is to use a fourier transform which, after transform, will show you at which frequencies you have periodicity. Have you researched fast-fourier-transforms for python: &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/tutorial/fftpack.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;(scipy.fftpack)&lt;/a&gt; ? I would be surprised if there would be no parallel implementation for a fourier transform in python. If that is still not fast enough&#xA;there is the FFTW-library, boldly calling itself &quot;Fastest Fourier Transform in The West&quot;, but I have to warn you that it is a pain in the ass to use. [EDIT:] There is a pyhton wrapper for FFTW &lt;a href=&quot;https://hgomersall.github.io/pyFFTW/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyFFTW&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You seem to have an impressive ammount of data. You might get away with doing some averaging, depending on your precision needs. If you take the first, say 10, datapoints, and take the maximum amplitude, store in a shorter array, and repeat for the following. You may shorten your dataset significantly (with loss in precision on your time axis of course).&lt;/p&gt;&#xA;" OwnerUserId="28636" LastActivityDate="2019-04-17T14:00:00.750" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31450" PostTypeId="1" AcceptedAnswerId="31457" CreationDate="2019-04-17T14:10:42.007" Score="1" ViewCount="442" Body="&lt;p&gt;I am currently doing a computational physics homework which asked us to use leapfrog to  give the relations between timevelocities and time-distance of these two objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full question is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider two parallel wire with distance d. A negative charged object(m1) is moving along the wire with initial velocity v0. One positive charged object(m2) is located on another wire with zero initial velocity and located at the original point(x = 0). The initial distance between two objects(x), velocity(v0), charges and(e1,e2) masses(m1,m2) are arbitrary. Relativity effect can be ignored.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write down the equations of motion for these two objects for this problem and convert it from a second-order equation to two-rst order ones. Write down a program using the time-reversal symmetric methods such as leapfrog or Verlet method to give the relations between timevelocities and time-distance of these two objects.(please use initial conditions: m1 = 1; m2 = 1; e1 = −1; e2 = 1; d = 1; x = 100; v0 = −5) &lt;/li&gt;&#xA;&lt;li&gt;Keep the original distance, masses and charges unchanged(m1 = 1; m2 = 1; e1 = −1; e2 = 1; d = 1; x = 100), please plot the nal velocity of m2 with dierent v0 (v0 ∈ (−10,0)). (here, you may need to enlarge the total time to nd the stable nal velocity of m2, roughly should be δv &amp;lt; 10−4). In the distance gure, we can always nd that m2 rst move along positive direction and then move along negative direction. Please nd the maxinum distance(D2max) of m2 in the positive direction at dierent v0 and plot this relation between v0 and D2max&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IAbbH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IAbbH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already found the two 2nd order differential equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dZBet.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dZBet.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DcKd2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DcKd2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And by substituting the initial value as required, we obtained the following result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JkyOz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JkyOz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I have already convert the two equations into first order:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bownc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bownc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, I would like to ask whether or not my steps for now are correct or not, since I am really skeptical on whether I can just directly convert it to first order one since the upsidedown L in the denominator still contains the variable x_1 and x_2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, since there are variable x_1 and x_2 in the upsidedown L, how can I change the equation to the form that I can use in the modified function of rk2a() in&#xA;&lt;a href=&quot;http://www.math-cs.gordon.edu/courses/mat342/python/diffeq.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.math-cs.gordon.edu/courses/mat342/python/diffeq.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have modified the code in rk2a() function to the leapfrog form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def leapfrog(f, x0, t):&#xA;n = len( t )&#xA;x = np.array( [ x0 ] * n )&#xA;for i in range( n - 1 ):&#xA;    h = t[i+1] - t[i]&#xA;    k1 = h * f( x[i], t[i] ) / 2.0&#xA;    if (i%2 == 0):&#xA;        x[i+1] = x[i] + h * f( x[i] + k1, t[i] + h / 2.0 )&#xA;    else:&#xA;        k2 = h * f( x[i], t[i] )&#xA;        x[i+1] = k1 + h * f( x[i] + k2, t[i] + h / 2.0 )&#xA;&#xA;return x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thirdly, I would like to ask whether my modification is correct or not(Hopefully it should be correct i guess, but I don't have any equation to test it with since above, I have trouble changing it to the form required by the function)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the modification is based on the equatuion by my lecture notes:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IXX2n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IXX2n.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30296" LastActivityDate="2019-04-18T20:14:33.407" Title="How to obtain and form a 1st order differential equation for leapfrog integration from second order one in this example of coulomb drag" Tags="|python|computational-physics|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31452" PostTypeId="1" CreationDate="2019-04-17T16:57:25.113" Score="1" ViewCount="95" Body="&lt;p&gt;I'm going to be doing some weak scaling of an &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;-body integrator on AWS. In the past when I've done weak scaling for this integrator I've fixed the number of particles per core (&lt;span class=&quot;math-container&quot;&gt;$N/n = {\rm const}$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I do this I get a weak scaling that is proportional to &lt;span class=&quot;math-container&quot;&gt;$n^\alpha$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of cores, which is tied to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\alpha \geq 1$&lt;/span&gt;. This is because the number of calculations involved in the &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;-body simulations is &lt;span class=&quot;math-container&quot;&gt;$N^2$&lt;/span&gt;. Thus, when I double the number of cores, I also double the number of particles, which actually quadruples the work. Therefore, I get an a scaling that is at least linear in &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding of weak scaling is that you're supposed to fix the problem size per core. In this case, is it more correct for me to fix the work per core than the number of particles per core?&lt;/p&gt;&#xA;" OwnerUserId="30495" LastActivityDate="2019-04-17T22:00:21.740" Title="Weak scaling for N-body simulations" Tags="|hpc|scaling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31454" PostTypeId="2" ParentId="31436" CreationDate="2019-04-17T20:56:55.223" Score="7" Body="&lt;p&gt;To understand the functional analysis (existence &amp;amp; uniqueness) part of the finite element method, it's helpful to see an analogy in linear algebra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the world of linear algebra, when you assemble any system of linear equations, you want to make sure that a solution exists.  Theory suggests that you have to make sure that the (square) matrix system is invertible.  One way to ensure that is to ensure that all the eigenvalues are non-zero. It is sufficient (but not absolutely necessary) to ensure that all eigenvalues are positive.  Symmetric positive definiteness in a linear system of equations (for real systems of equations) ensures that eigenvalues are all positive and thus a solution exists.  It's not the only way to guarantee that a matrix system is invertible, but if a linear system is symmetric positive definite, you get to take advantage of a lot of easy-to-verify analysis tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Think of functional analysis in a similar way.  Weak forms produce linear systems of equations.  You want to make sure that the resulting matrix system is invertible.  If you have a symmetric bilinear form, you get to take advantage of a lot of readily available theorems (e.g. Riesz Representation, Cea’s Lemma) which enable you to prove whether the resulting system of equations is invertible (like having all-positive eigenvalues).  But symmetry is by no means a &lt;strong&gt;&lt;em&gt;necessary&lt;/em&gt;&lt;/strong&gt; condition for the system to be invertible.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are more generalized theorems for bilinear forms, but the one most people are familiar with is Lax-Milgram theorem.  It basically says that in order for a solution to exist to the weak form (i.e. invertability of the resulting matrix system), you need to certain conditions on &lt;span class=&quot;math-container&quot;&gt;$a(u,v)=L(v)$&lt;/span&gt; to be true for the function space &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Boundedness of &lt;span class=&quot;math-container&quot;&gt;$L(.)$&lt;/span&gt; on the space &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; :  In other words, &lt;span class=&quot;math-container&quot;&gt;$||L(v)||\le M||v||_V$&lt;/span&gt;  &lt;/li&gt;&#xA;&lt;li&gt;Coercivity (also known as V-ellipticity) of &lt;span class=&quot;math-container&quot;&gt;$a(.,.)$&lt;/span&gt; on the space &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;: In other words, &lt;span class=&quot;math-container&quot;&gt;$a(v,v)\le C_1||v||^2_V$&lt;/span&gt; &lt;/li&gt;&#xA;&lt;li&gt;Continuity of &lt;span class=&quot;math-container&quot;&gt;$a(.,.)$&lt;/span&gt; on the space &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt;: In other words, &lt;span class=&quot;math-container&quot;&gt;$a(u,v)\le C_2||u||_V||v||_V$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Together, these three conditions guarantee that your weak form (i.e. the system of equations setup by the weak form) is invertible and thus a solution exists.  Symmetry is not necessary to use Lax Milgram Theorem.  Symmetry is necessary to use Cea's lemma though.  The equation &lt;span class=&quot;math-container&quot;&gt;$-\nabla\cdot[k\nabla u]+\nabla\cdot(cu)+au=f$&lt;/span&gt; is non-symmetric and Lax Milgram theorem applies to it as well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if your weak form does not have these conditions, it doesn't guarantee that a solution doesn't exist.  But it's harder to prove that a solution exists by other means.  There are other choices of finite element spaces that require more sophisticated theorems (e.g. mixed finite elements, petrov-galerkin methods, discontinuous galerkin methods, etc..).  I can't speak on those theorems with authority, but there are others here on scicomp who can address questions on those spaces/methods.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2019-04-17T23:16:30.723" LastActivityDate="2019-04-17T23:16:30.723" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31455" PostTypeId="2" ParentId="31452" CreationDate="2019-04-17T22:00:21.740" Score="2" Body="&lt;p&gt;One can define weak scaling in many different ways:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Amount of work per processor stays the same&lt;/li&gt;&#xA;&lt;li&gt;Amount of communication per processor stays the same&lt;/li&gt;&#xA;&lt;li&gt;Amount of memory usage per processor stays the same.&lt;/li&gt;&#xA;&lt;li&gt;Number of objects per processor stays the same.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In many cases, all of these will amount to the same, notably if the amount of work, memory, and communication is proportional to the number of objects. But in some cases it is not. In those cases, choose one of these definitions and be clear in your description what you are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(As a complete aside, for &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;-body simulations, you should really be using something like the multipole expansion to avoid the &lt;span class=&quot;math-container&quot;&gt;$N^2$&lt;/span&gt; bottleneck...)&lt;/p&gt;&#xA;" OwnerUserId="393" LastActivityDate="2019-04-17T22:00:21.740" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31456" PostTypeId="1" AcceptedAnswerId="32529" CreationDate="2019-04-18T00:32:58.807" Score="2" ViewCount="1545" Body="&lt;p&gt;I am trying to solve numerically (obviously) &lt;a href=&quot;https://en.wikipedia.org/wiki/Burgers%27_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;inviscid Burgers' equation&lt;/a&gt; with the finite difference method. The equation is the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \displaystyle \partial_t u + u \, \partial_x u = 0 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which also reads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \displaystyle \partial_t u + \frac{1}{2} \partial_x u^2 = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And since the discrete solution is positive, the upwind scheme reads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \displaystyle  u_j^{n+1} = u_j^n + \frac{\Delta t}{\Delta x} \left[ \left(u_{i-1}^n\right)^2 - \left(u_{i}^n\right)^2 \right] $$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that remains is to define the computational domain and put it all in a loop and that's it, right? To definite this domain, I know we can use the famous CFL condition. I never heard about this condition at school and I just know how to use it for a simple convection case like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \partial_t u + c \, \partial_x u = 0, \qquad \text{CFL} \Rightarrow \frac{c\Delta t}{\Delta x} \leq 1.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't find any documents describing the general method to follow to apply this condition in my case or in any other case. I just found &lt;a href=&quot;https://hal.archives-ouvertes.fr/hal-00576248/document&quot; rel=&quot;nofollow noreferrer&quot;&gt;this work&lt;/a&gt; about the same problem but I didn't understand the explanation of the condition use which is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eVKBo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eVKBo.png&quot; alt=&quot;CFL use&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, I need help to understand how to use this condition. I have managed to implement my algorithm and get a solution but I would like to understand everything that is happening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerDisplayName="user30503" LastActivityDate="2019-04-29T04:13:40.273" Title="CFL equation for non-linear equation" Tags="|finite-difference|cfl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31457" PostTypeId="2" ParentId="31450" CreationDate="2019-04-18T04:08:00.533" Score="0" Body="&lt;p&gt;You have already found the system of two differential equations (I am assuming that by &lt;span class=&quot;math-container&quot;&gt;$\Gamma_{12}$&lt;/span&gt; you mean &lt;span class=&quot;math-container&quot;&gt;$\Gamma_{12} = \sqrt{(x_1-x_2)^2 + d^2\,}$&lt;/span&gt; ) which is&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;m_1 \frac{d^2 x_1}{dt^2} = \frac{e_1e_2}{4\pi \varepsilon_0} \, \frac{x_1-x_2}{\big(\,(x_2-x_1)^2 + d^2\,\big)^{\frac{3}{2}}}\\&#xA;&amp;amp;m_2 \frac{d^2 x_2}{dt^2} = \frac{e_1e_2}{4\pi \varepsilon_0} \, \frac{x_2-x_1}{\big(\,(x_1-x_2)^2 + d^2\,\big)^{\frac{3}{2}}}&#xA;\end{align}&lt;/span&gt;&#xA;According to the assumption of the problem &lt;span class=&quot;math-container&quot;&gt;$m_1=m_2=1, \,\, e_1=-1,\,\,e_2=1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$d=1$&lt;/span&gt;. Consequently&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\frac{d^2 x_1}{dt^2} = k_0 \, \frac{x_2-x_1}{\big(\,(x_1-x_2)^2 + 1\,\big)^{\frac{3}{2}}}\\&#xA;&amp;amp;\frac{d^2 x_2}{dt^2} = k_0  \, \frac{x_1-x_2}{\big(\,(x_1-x_2)^2 + 1\,\big)^{\frac{3}{2}}}&#xA;\end{align}&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$k_0 = \frac{1}{4\pi \varepsilon_0}$&lt;/span&gt; (seems like you re missing this factor). Furthermore, this system reduces from two equations to one, due to the translation invariance (i.e. there is a one parameter group of translation symmetries along the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; axis) which provides a conserved quantity, by Noether's theorem. But we can carry out the reduction by hand, without too much fancy noise and big words, as follows: subtract the second equation from the first one and obtain &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d^2 x_1}{dt^2} - \frac{d^2 x_2}{dt^2} = - k_0 \, \frac{x_1-x_2}{\big(\,(x_1-x_2)^2 + 1\,\big)^{\frac{3}{2}}} - k_0  \, \frac{x_1-x_2}{\big(\,(x_1-x_2)^2 + 1\,\big)^{\frac{3}{2}}}$$&lt;/span&gt; which turns into&#xA; &lt;span class=&quot;math-container&quot;&gt;$$\frac{d^2}{dt^2}(x_1 - x_2)  =  \frac{- 2k_0 \,(x_1-x_2)}{\big(\,(x_1-x_2)^2 + 1\,\big)^{\frac{3}{2}}}$$&lt;/span&gt; Next, set &lt;span class=&quot;math-container&quot;&gt;$x = x_1-x_2$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d^2 x}{dt^2}  =  - \, \frac{2k_0 \, x}{\big(\,1 + x^2\,\big)^{\frac{3}{2}}}$$&lt;/span&gt; The latter equation is the differential equation that describes the time-evolution of  the relative position of the two particles. &#xA;I believe this is the equation that is mentioned in part 1 of your questions (notice is says &quot;and convert &lt;strong&gt;it&lt;/strong&gt; from &lt;strong&gt;a second-order equation&lt;/strong&gt; to &lt;strong&gt;two-rst order ones&lt;/strong&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, if you sum the two equations, you obtain the relation&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d^2 x_1}{dt^2} + \frac{d^2 x_2}{dt^2} = - k_0 \, \frac{x_1-x_2}{\big(\,(x_1-x_2)^2 + 1\,\big)^{\frac{3}{2}}} + k_0  \, \frac{x_1-x_2}{\big(\,(x_1-x_2)^2 + 1\,\big)^{\frac{3}{2}}} = 0$$&lt;/span&gt; i.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d^2}{dt^2}(x_1 + x_2)  = 0$$&lt;/span&gt; and so by integrating two times consequtively &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d x_1}{dt}(t) + \frac{dx_2}{dt}(t) = v_0 + 0 = -5$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$x_1(t) + x_2(t) = x_1(0) + x_2(0) - 5\,t = 100 + 0  - 5\,t = 100 - 5\,t$$&lt;/span&gt;&#xA;Combining the latter equation with the equation &lt;span class=&quot;math-container&quot;&gt;$x_1(t) - x_2(t) = x(t)$&lt;/span&gt; allows you to express everything in terms of &lt;span class=&quot;math-container&quot;&gt;$x = x(t)$&lt;/span&gt; &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;x_1(t) = \frac{1}{2}\left(100 - 5\, t + x(t)\right)\\&#xA;&amp;amp;x_2(t) = \frac{1}{2}\left(100 - 5\, t - x(t)\right)\\ &#xA;&amp;amp;\\&#xA;&amp;amp;v_1(t) = \frac{dx_1}{dt}(t) =  \frac{1}{2}\left(- 5 + v(t)\right)\\&#xA;&amp;amp;v_2(t) = \frac{dx_2}{dt}(t) = \frac{1}{2}\left(- 5 - v(t)\right)\\ &#xA;\end{align}&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$v(t) = \frac{dx}{dt}(t)$&lt;/span&gt;.&#xA;On a side note, the differential equation for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; has a conserved energy integral, so it is in fact completely integrable (solvable in possibly implicit functions and integrals).&#xA;Anyway, the second order differential equation for &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; turns into a system of two first order equations&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\frac{dx}{dt} = v\\&#xA;&amp;amp;\frac{dv}{dt} = - \,\frac{ 2k_0 \, x\, }{\big(\,1 + x^2\,\big)^{\frac{3}{2}}}&#xA;\end{align}&lt;/span&gt; The way I know leapfrog, one looks at the system as a sum of two exactly solvable summands, i.e. &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\frac{d}{dt}\begin{bmatrix}x\\v\end{bmatrix} = \begin{bmatrix}v\\0\end{bmatrix} +  \begin{bmatrix} 0\\ - \, \frac{ 2k_0 \, x\, }{\big(\,1 + x^2\,\big)^{\frac{3}{2}}}\end{bmatrix}&#xA;\end{align}&lt;/span&gt;&#xA;and based on this decomposition, split it into two exactly solvable systems, i.e.&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\frac{dx}{dt} = v\\&#xA;&amp;amp;\frac{dv}{dt} = 0&#xA;\end{align}&lt;/span&gt;&#xA;and&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;\frac{dx}{dt} = 0\\&#xA;&amp;amp;\frac{dv}{dt} = - \,\frac{ 2k_0 \, x\, }{\big(\,1 + x^2\,\big)^{\frac{3}{2}}}&#xA;\end{align}&lt;/span&gt;&#xA;The solutions to the first system look like &#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;x(t) = x_0 + v_0\,t\\&#xA;&amp;amp;v(t) = v_0 &#xA;\end{align}&lt;/span&gt;&#xA;and the solutions of the second system are&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;x(t) = x_0\\&#xA;&amp;amp;v(t) = v_0 - t\,\frac{ 2k_0 \, x_0\, }{\big(\,1 + x_0^2\,\big)^{\frac{3}{2}}}&#xA;\end{align}&lt;/span&gt; Leapfrog is basically the carefully chosen composition of these two sets of solutions: within one time step &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt;, starting from the current &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_i$&lt;/span&gt;, follow the first solution for time &lt;span class=&quot;math-container&quot;&gt;$h/2$&lt;/span&gt;, then the second solution for time &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; and finally again the first solution for time &lt;span class=&quot;math-container&quot;&gt;$h/2$&lt;/span&gt;, finally arriving at &lt;span class=&quot;math-container&quot;&gt;$x_{i+1}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_{i+1}$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;&amp;amp;x_{i+1/3} = x_i + \frac{h}{2}\,v_i \\&#xA;&amp;amp;v_{i+1/3} = v_{i}                     \\&#xA;&amp;amp;\\&#xA;&amp;amp;\\&#xA;&amp;amp;x_{i+2/3} = x_{i+1/3}\\ &#xA;&amp;amp;v_{i+2/3} = v_{i+1/3} - h\,\frac{ 2k_0 \, x_{i+1/3}\, }{\big(\,1 + x_{i+1/3}^2\,\big)^{\frac{3}{2}}} \\&#xA;&amp;amp;\\&#xA;&amp;amp;\\&#xA;&amp;amp;x_{i+1} = x_{i+2/3} + \frac{h}{2}\,v_{i+2/3}\\&#xA;&amp;amp;v_{i+1} = v_{i+2/3}&#xA;\end{align}&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in Matlab code (I do not master python that much) you can write&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X1 = zeros(1,n); %initialization of a one row array of length n&#xA;V1 = zeros(1,n); %initialization of a one row array of length n&#xA;X2 = zeros(1,n); %initialization of a one row array of length n&#xA;V2 = zeros(1,n); %initialization of a one row array of length n&#xA;x = 100; &#xA;v=-5;&#xA;t=0;&#xA;X1(i) = 100;&#xA;X2(i) = 0 ;&#xA;V1(i) = -5;&#xA;V2(i) = 0;&#xA;for i = 2:n&#xA;   t = t + h; &#xA;   x = x + v*h/2;&#xA;   v = v - 2*k0*h*x/(1+x^2)^(3/2);&#xA;   x = x + v*h/2;&#xA;   X1(i) = (100 - 5*t + x)/2;&#xA;   X2(i) = (100 - 5*t - x)/2;&#xA;   V1(i) = (- 5 - v)/2;&#xA;   V2(i) = (- 5 + v)/2;&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20964" LastEditorUserId="20964" LastEditDate="2019-04-18T20:14:33.407" LastActivityDate="2019-04-18T20:14:33.407" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31458" PostTypeId="1" CreationDate="2019-04-18T04:08:06.797" Score="3" ViewCount="119" Body="&lt;p&gt;My task is to solve &lt;span class=&quot;math-container&quot;&gt;$$\eta_k\frac{d^2C_k}{dz}(z)=-e_k, k = 1,2,3$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C_k\ge0$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C_1(0)=0, C_2(0)=A, C_3(0)=0$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$C_1(L)=B, \frac{dC_2}{dz}(L)=0, \frac{dC_3}{dz}(L)=0$$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$$e_1 = -\beta_1-\beta_3$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$e_2=-\beta_1-\beta_2$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$e_3=\beta_2-\beta_3$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;satisfying the constraints &lt;span class=&quot;math-container&quot;&gt;$$-e_k\le \frac{\eta_kC_k}{K_k+C_k}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\eta_k, K_k$&lt;/span&gt; are some constants, where &lt;span class=&quot;math-container&quot;&gt;$\beta_1,\beta_2,\beta_3$&lt;/span&gt; are such that &lt;span class=&quot;math-container&quot;&gt;$$\max 19\beta_1+0.5\beta_2+16\beta_3$$&lt;/span&gt; is obtained.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I try to solve it by iterations, treating the ODE and the optimization as two separated steps. For every iteration,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First solve the ode to get some &lt;span class=&quot;math-container&quot;&gt;$C_k$&lt;/span&gt;;&lt;/li&gt;&#xA;&lt;li&gt;then I fix &lt;span class=&quot;math-container&quot;&gt;$C_k$&lt;/span&gt; and solve the linear programming to get &lt;span class=&quot;math-container&quot;&gt;$e_k$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I repeat the above two steps to iteratively get next &lt;span class=&quot;math-container&quot;&gt;$C_k$&lt;/span&gt;, &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$e_k$&lt;/span&gt;&amp;quot;, and so on. &lt;strong&gt;My main problem&lt;/strong&gt; is I do not know how to properly guarantee &lt;span class=&quot;math-container&quot;&gt;$C_k\ge0$&lt;/span&gt; when solving the ODE. &lt;strong&gt;Any reference to papers or software packages that enforce such constraint&lt;/strong&gt; is very much appreciated!&lt;/p&gt;&#xA;&lt;p&gt;Without being able to guarantee that constraint, the above iterative process does not seem to converge and give weird oscillations.&lt;/p&gt;&#xA;" OwnerUserId="24316" LastEditorUserId="-1" LastEditDate="2020-06-17T09:48:19.083" LastActivityDate="2019-04-19T17:42:35.430" Title="Solve ODE with non-negative and maximization constraints" Tags="|optimization|ode|constrained-optimization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31459" PostTypeId="1" CreationDate="2019-04-18T06:28:22.343" Score="1" ViewCount="157" Body="&lt;p&gt;I'm trying to re-solve the governing equations in hydraulic fracturing modeling as instructed step by step in a paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After (A-9), the author stated that by substituting A-6, A-8 and A-9 into equation A-4 we obtain a linear tridiagonal system which is easily solved for &lt;span class=&quot;math-container&quot;&gt;$∆W_i^m$&lt;/span&gt; at m =1, 2,… Could anyone please help me at this step?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The governing equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{∂q}{∂x}+\frac{2hC}{\sqrt{t-τ(x)}}+\frac{∂A}{∂t}=0, 0&amp;lt;x&amp;lt;L (A-1)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q(0,t)=q_i (A-2)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{∆t_m}{2}\frac{∂}{∂x}({q^{m+1}+q^m})+[{\frac{4hC}{\sqrt{t-τ(x)}}+A}]_{t_m}^{t_{m+1}} =0          (A-3)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$∆t_m=t_{m+1}-t_m,   q^m=q(x,t_m)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Integration A-3 with respect to x from &lt;span class=&quot;math-container&quot;&gt;$x_{i-1/2}$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$x_{i+1/2}$&lt;/span&gt; (with the use of &lt;span class=&quot;math-container&quot;&gt;$A=π/4 Wh$&lt;/span&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{∆t_m}{2∆x}(q_{i+1/2}^{m+1}-q_{i-1/2}^{m+1}+q_{i+1/2}^m-q_{i-1/2}^m )+4hC(\sqrt{t_{m+1}-τ_i}-\sqrt{t_m-τ_i})+\frac{πh}{4}(W_i^{m+1}-W_i^m)=0     (A-4)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$∆x=x_{i+1/2}-x_{i-1/2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_i^m=W(x_i,t_m)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_i=\frac{1}{2}(x_{i+1/2}+x_{i-1/2})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take &lt;span class=&quot;math-container&quot;&gt;$x_{1/2}=0$&lt;/span&gt; then by A-2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_{1/2}^m=q_i (A-5)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_i^0=0(A-6)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q=\frac{-πG}{256(1-ϑ)μ∆x}\frac{∂}{∂x}W^4$&lt;/span&gt;, by finite difference analog gives:   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_{i+1/2}=\frac{-πG}{256(1-ϑ)μ∆x}[{(W_{i+1}^m)}^4-{(W_i^m )}^4 ]   (A-7)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$q_{i+1/2}^m=\frac{-πG}{256(1-ϑ)μ∆x}[{(W_{i+1}^m)}^4-{(W_i^m )}^4 ]$&lt;/span&gt;   (m was missing??)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_i^{m+1}=W_i^m+∆W_i^m        (A-8)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;${(W_i^{m+1} )}^4={(W_i^m)}^4+4{(W_i^m )}^3{∆W}_i^m         (A-9)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I get after substituting A-7 into A-4:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{πG∆t_m}{256(1-ϑ)μ}[{(W_{i+1}^m)}^4-2{(W_i^m)}^4+{(W_{i-1}^m)}^4+2{(W_{i+1}^m)}^3{∆W}_{i+1}^m-4{(W_i^m)}^3{∆W}_i^m+2{(W_{i-1}^m)}^3{∆W}_{i-1}^m]+4hC\sqrt{t_{m+1}-τ_i}-\sqrt{t_m-τ_i}+\frac{πh}{4}{∆W}_i^m=0 (A-10)$&lt;/span&gt;    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to get the linear tridiagonal system from the above equation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much&lt;/p&gt;&#xA;" OwnerUserId="30193" LastActivityDate="2019-08-13T10:02:36.227" Title="How to obtain linear tridiagonal system from PDE" Tags="|pde|finite-difference|linear-system|modeling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31460" PostTypeId="2" ParentId="31459" CreationDate="2019-04-18T11:30:23.567" Score="4" Body="&lt;p&gt;Welcome to the site. You are actually virtually finished, but may not have realised it yet. A tridiagonal linear system is another name for a matrix problem which only has non zero entries on the leading diagonal and the one above and below it, so lets try writing your problem like that. We want a form &lt;span class=&quot;math-container&quot;&gt;$$ \mathbf{A} (\mathbf{\Delta W}^m) = \mathbf{b},$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt; is a matrix with values we know, &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\Delta W}^m$&lt;/span&gt; is the (unknown) vector of updates we want to solve for and &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}$&lt;/span&gt; is a known vector of values on the right hand side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now, reading off the terms in &lt;span class=&quot;math-container&quot;&gt;$\Delta W$&lt;/span&gt; from your equation (A-10) we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A_{ij}=\begin{cases}&#xA;2(W^m_{i-1})^3 &amp;amp;j=i-1\\&#xA;-4(W^m_i)^3 &amp;amp; j=i\\&#xA;2(W^m_{i+1})^3 &amp;amp; j=i+1\\&#xA;0&amp;amp; \mbox{otherwise}\end{cases}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;meanwhile &lt;span class=&quot;math-container&quot;&gt;$\mathbf{b}$&lt;/span&gt; collects all the other terms which don't have a  &lt;span class=&quot;math-container&quot;&gt;$\Delta W$&lt;/span&gt; in. So, this is indeed a linear tridiagonal system. Since we known &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}^0$&lt;/span&gt;, we can invert &lt;span class=&quot;math-container&quot;&gt;$\mathbf{A}$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$\mathbf{\Delta W}^0$&lt;/span&gt;, which gives us &lt;span class=&quot;math-container&quot;&gt;$\mathbf{W}^1$&lt;/span&gt; and so on.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2019-04-18T11:30:23.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31461" PostTypeId="1" AcceptedAnswerId="31470" CreationDate="2019-04-18T11:37:06.150" Score="1" ViewCount="165" Body="&lt;p&gt;I have a diffusion problem with an internal circular dirichlet constraint and a side condition which shall enforce a certain global volume integral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\nabla(D \nabla u(x)) = 0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;outer boundary constraint:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u(x) = 0 ~~\forall~x~\in \partial\Omega$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;inner boundary constraint:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$u(x) = 1 ~~\forall~x~\in \Omega_{int}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\int_{\Omega} u ~dx = V_t$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have successfully discretized the bilinear form into A, and the volume constraint is basically a row/column vector with the cell sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also succeeded in solving the resulting saddle point problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\begin{pmatrix}&#xA;A &amp;amp; B^T \\&#xA;B &amp;amp; 0&#xA;\end{pmatrix} \begin{pmatrix}&#xA;x_1  \\&#xA;x_2 &#xA;\end{pmatrix}  = \begin{pmatrix}&#xA;0  \\&#xA;V_t &#xA;\end{pmatrix} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;via a classical uzawa iteration:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_{1}^{k+1} =x_{1}^{k} - A^{-1}(A x_{1}^{k} + B x_{2}^{k})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_{2}^{k+1} = x_{2}^{k} + \tau (B^{T} x_{1}^{k+1}-V_t)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result looks just like i want it too, and all side conditions are met:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4fn0r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4fn0r.png&quot; alt=&quot;result_classical_uzawa&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read that the classical uzawa iteration is far from being the most efficient, and that there is a lot of approaches to do it better. I have a hard time understanding how the preconditioned Uzawa methods work, and how to actually come up with the preconditioner Matrix without doing costly inversions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the easiest way to properly precondition my uzawa method so that the convergence becomes faster (and hopefully more stable)&lt;/p&gt;&#xA;" OwnerUserId="28636" LastActivityDate="2019-04-19T10:27:54.640" Title="Simplest way to precondition Uzawa iteration" Tags="|convergence|preconditioning|saddlepoint|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31462" PostTypeId="2" ParentId="31456" CreationDate="2019-04-18T15:03:55.507" Score="0" Body="&lt;p&gt;The CFL condition is not always the same. It changes depending on the equation and the discretization. For example, here you have no fixed c, and have a variable speed of sound and a nonlinear PDE. What you should try to do is von neumann analysis to see if you can derive the CFL condition for some basic linear cases like advection or heat diffusion. Then you can try it on a nonlinear case, like this one (there's some special treatment you can use for nonlinear cases iirc, but I don't think you can get a real CFL for burger's equation). And then read over the work that you posted a picture of as it's rather in depth and you need the background to really understand it.&lt;/p&gt;&#xA;" OwnerUserId="9446" LastActivityDate="2019-04-18T15:03:55.507" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="31463" PostTypeId="1" AcceptedAnswerId="31467" CreationDate="2019-04-18T17:06:30.963" Score="1" ViewCount="3928" Body="&lt;p&gt;I am looking for the most efficient way to plot a mesh using Matplotlib given the following information, coordinates of each node, what nodes belong to each element, and the value each node has. Below I have some example data and image showing what the mesh looks like  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nodeinfo=[[0.000,0.000],[1.000,0.000],[2.000,0.500],[0.000,1.000],&#xA;[1.000,1.000],[1.750,1.300],[1.000,1.700]]&#xA;elementInfo=[[1,2,5],[5,4,1],[2,3,6],[6,5,2],[4,5,7],[5,6,7]]&#xA;nodevalues=[1,2,1,2,7,4,5]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zegWQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zegWQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;nodeinfo is the coordinates of each nodes(e.g. node 7 has coordinates (1,1.7)), elementInfo gives what nodes each element is composed of (e.g. element 3 has nodes 2,3,6), nodevalues gives the value of each node(e.g. node 5 has value 7). &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using this info how can I plot meshes with matplotlib with a colour gradient showing the different values of the nodes(if possible it would be great if there was a colour gradient between nodes as each element is linear).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;&#xA;If you want to use it have created a bit of code that organizes the information into node objects.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class node:&#xA;    # Initializer / Instance Attributes&#xA;    def __init__(self, number, xCord, yCord):&#xA;        self.number=number&#xA;        self.value=1&#xA;        self.isOnBoundary=False&#xA;        self.xCord=xCord&#xA;        self.yCord=yCord&#xA;        self.boundaryType=None&#xA;        self.element=[]&#xA;&#xA;    #makes all class variables callable&#xA;    def __call__(self):&#xA;        return self&#xA;&#xA;    def checkIfOnBoundary(self,boundarylist):&#xA;        # Checks if the node is on the boundary when it is invoked&#xA;        # If the node is not on the boundary then it is set to false&#xA;&#xA;        if self.number in boundarylist:&#xA;            self.isOnBoundary=True&#xA;            self.boundaryType=boundarylist[self.number][0]&#xA;            if self.boundaryType == &quot;Dirchlet&quot;:&#xA;                self.value=boundarylist[self.number][1]&#xA;        else:&#xA;            self.isOnBoundary=False&#xA;&#xA;    def setElement(self,elementInfo):&#xA;        #given a list in the form [element1,element2,...,elementn]&#xA;        #where element1 is a list that contains all the nodes that are on that element&#xA;        for element in elementInfo:&#xA;            if self.number in element:&#xA;                self.element.append(elementInfo.index(element)+1)&#xA;&#xA;&#xA;    def setValue(self,value):&#xA;        # changes the value of the node&#xA;        self.value=value&#xA;&#xA;    def description(self):&#xA;        return &quot;Node Number: {}, Node Value: {}, Element Node Belongs to: {}, Is Node On the Boundary: {}&quot;.format(self.number, self.value, self.element, self.isOnBoundary)&#xA;&#xA;nodeinfo=[[0.000,0.000],[1.000,0.000],[2.000,0.500],[0.000,1.000],&#xA;[1.000,1.000],[1.750,1.300],[1.000,1.700]]&#xA;elementInfo=[[1,2,5],[5,4,1],[2,3,6],[6,5,2],[4,5,7],[5,6,7]]&#xA;nodevalues=[1,2,1,2,7,4,5]&#xA;&#xA;#create list of node objects which we will call on often&#xA;nodes=[]&#xA;for i in range(len(nodeinfo)):&#xA;    print(i)&#xA;    nodes.append(node(i+1,nodeinfo[i][0],nodeinfo[i][1]))&#xA;    nodes[i].setElement(elementInfo)&#xA;&#xA;#print information related to each object&#xA;for phi in nodes:&#xA;    print(vars(phi))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="30468" LastEditorUserId="9667" LastEditDate="2020-04-10T18:43:46.260" LastActivityDate="2020-04-10T18:46:21.120" Title="Efficiently plot a finite element mesh solution with Matplotlib" Tags="|finite-element|python|visualization|plotting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31464" PostTypeId="2" ParentId="2360" CreationDate="2019-04-18T18:04:40.500" Score="0" Body="&lt;p&gt;One approach is via the equivalent equation, that is, the differential equation to which your discrete method gives the closest aproximation. This is never the differential equation that you intended to solve. Then you look at the asymptotic solution of the equivalent equation, for a step function as initial data. Look at Bouche, D., Bonnaud, G. and Ramos, D., 2003. Comparison of numerical schemes for solving the advection equation. Applied mathematics letters, 16(2), pp.147-154.&lt;/p&gt;&#xA;" OwnerUserId="23873" LastActivityDate="2019-04-18T18:04:40.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31467" PostTypeId="2" ParentId="31463" CreationDate="2019-04-19T02:46:00.303" Score="3" Body="&lt;p&gt;You might be interested in trying other visualization tools such as &lt;a href=&quot;https://www.paraview.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ParaView&lt;/a&gt;, &lt;a href=&quot;https://docs.enthought.com/mayavi/mayavi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mayavi&lt;/a&gt; or &lt;a href=&quot;https://docs.pyvista.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyVista&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, since the question is about Matplotlib, I would suggest that you use &lt;a href=&quot;https://matplotlib.org/api/_as_gen/matplotlib.axes.Axes.tricontour.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tricontour&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://matplotlib.org/api/_as_gen/matplotlib.pyplot.tricontourf.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tricontourf&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;https://matplotlib.org/api/_as_gen/matplotlib.pyplot.tripcolor.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tripcolor&lt;/code&gt;&lt;/a&gt;. They already accept the data in your format. Keep in mind that the enumeration of nodes starts from 0 in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following snippet show your data visualized.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;nodes= np.array([&#xA;        [0.000, 0.000],&#xA;        [1.000, 0.000],&#xA;        [2.000, 0.500],&#xA;        [0.000, 1.000],&#xA;        [1.000, 1.000],&#xA;        [1.750, 1.300],&#xA;        [1.000, 1.700]])&#xA;eles = np.array([&#xA;        [1, 2, 5],&#xA;        [5, 4, 1],&#xA;        [2, 3, 6],&#xA;        [6, 5, 2],&#xA;        [4, 5, 7],&#xA;        [5, 6, 7]])&#xA;node_vals = [1, 2, 1, 2, 7, 4, 5]&#xA;&#xA;x, y = nodes.T&#xA;plt.tricontourf(x, y, eles - 1, node_vals, 12)&#xA;plt.colorbar()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SNMpu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SNMpu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9667" LastEditorUserId="9667" LastEditDate="2020-04-10T18:46:21.120" LastActivityDate="2020-04-10T18:46:21.120" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31468" PostTypeId="1" CreationDate="2019-04-19T07:49:35.497" Score="3" ViewCount="135" Body="&lt;p&gt;I have to solve a set of nonlinear optimization problems in the subspace defined as the orthogonal space to a given vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely, &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\arg\min f(\vec x)  \qquad \text{with} \qquad \vec x \cdot \vec n =0  &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking of applying the nonlinear conjugate gradient method projecting the direction of descent but I am wondering in which step I should do this projection operation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One idea is that the projection should take place only in the update of the position like&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec d_k = - \vec \nabla f(\vec x_{k-1}) + \beta \vec d_{k-1} \\&#xA;\alpha_k = \arg \min f(\vec x_{k-1} + \alpha \vec d_k) \\&#xA;\vec x_{k} = \vec x_{k-1} + (\alpha_k \vec d_k) - [(\alpha_k \vec d_k)\cdot \vec n ]\vec n&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other idea is to apply projection directly to the gradient itself such that even the past directions are subjected to the constraint of living in the orthogonal subspace to &lt;span class=&quot;math-container&quot;&gt;$\vec n$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\vec g_k =  \vec \nabla f(\vec x_{k-1}) - [\vec \nabla f(\vec x_{k-1})\cdot \vec n]\vec n \\&#xA;\vec d_k = - \vec g_k + \beta \vec d_{k-1} \\&#xA;\alpha_k = \arg \min f(\vec x_{k-1} + \alpha \vec d_k) \\&#xA;\vec x_{k} = \vec x_{k-1} + (\alpha_k \vec d_k)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best way to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="30502" LastEditorUserId="30502" LastEditDate="2019-04-21T19:09:53.600" LastActivityDate="2019-04-22T04:21:03.203" Title="Nonlinear conjugate gradient with orthogonality constraint" Tags="|nonlinear-programming|conjugate-gradient|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31470" PostTypeId="2" ParentId="31461" CreationDate="2019-04-19T10:27:54.640" Score="1" Body="&lt;p&gt;This should be a &quot;comment&quot;, but I don't have the credentials.&#xA;I interpret the question as being about understanding the workings of the Uzawa methods, their stability and optimization, rather than how to solve the diffusion equation.  The following articles are easily readable and address this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ho et al. &lt;a href=&quot;https://arxiv.org/abs/1510.04246&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1510.04246&lt;/a&gt; (&quot;Accelerating the Uzawa Algorithm&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pascanu _et al. &lt;a href=&quot;https://arxiv.org/abs/1405.4604&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1405.4604&lt;/a&gt; (&quot;On the saddle point problem for non-convex optimization&quot;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of these present a clear discussion of the algorithm, giving nice explanations of why it works and when it might be desirable to use use it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The article of&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dauphin et al. &lt;a href=&quot;https://arxiv.org/abs/1406.2572&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1406.2572&lt;/a&gt; (&quot;Identifying and attacking the saddle point problem in high-dimensional non-convex optimization&quot;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;compares the method with Newton-style and gradient descent methods. &#xA;The last two references are aimed mainly towards the neural net learning process. &lt;/p&gt;&#xA;" OwnerUserId="21129" LastActivityDate="2019-04-19T10:27:54.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31472" PostTypeId="1" CreationDate="2019-04-19T16:32:45.417" Score="0" ViewCount="45" Body="&lt;p&gt;I'm currently trying to solve a problem using numerical methods. The set-up is rather long, so I apologize in advance... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: My solutions change depending on how big my steps are and I don't know why.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Set-up&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first part of this problem can be described by a Lagrangian &lt;span class=&quot;math-container&quot;&gt;$L(s,\theta)$&lt;/span&gt;, which is given by&#xA;&lt;span class=&quot;math-container&quot;&gt;$$L = \frac{1}{24}(L^2\dot\theta^2+4b^2\dot\theta^2+24s\dot\theta^2+12\dot s^2+ 12 b\dot s \dot\theta+24gs\sin\theta -12bg\cos\theta).$$&lt;/span&gt;&#xA;This Lagragian gives, by means of the Euler-Lagrange equations,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align*} &#xA; \dot\theta^2 +g \sin\theta &amp;amp;=\ddot s + \frac{1}{2}b\ddot\theta\\ &#xA;\left(\frac{1}{12}L^2+\frac{1}{3}b^2\right)\ddot\theta+\frac{1}{2}b\ddot s &amp;amp;= \frac{1}{2}bg\sin\theta-2\dot s\dot\theta-gs\cos\theta, &#xA;\end{align*}&#xA;$$&lt;/span&gt;&#xA;where you can assume &lt;span class=&quot;math-container&quot;&gt;$L=0.1,\,b=0.02$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$g=9.81$&lt;/span&gt;.&#xA;I also have the initial conditions given by:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$s(0)=0.1,\quad \dot s(0)= 0.3,\quad u(0)= 0,\quad \dot u(0)= 0.$$&lt;/span&gt;&#xA;The generalized coordinates &lt;span class=&quot;math-container&quot;&gt;$s,\theta$&lt;/span&gt; and the real trajectory are connected over&#xA;&lt;span class=&quot;math-container&quot;&gt;$$x = \begin{pmatrix}s\cos\theta + \frac{b}{2}\sin\theta\\ -s\sin\theta +\frac{b}{2}\cos\theta\end{pmatrix}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first part of the problem is to solve this set of differential equation up to the time &lt;span class=&quot;math-container&quot;&gt;$t^*$&lt;/span&gt; when the condition &lt;span class=&quot;math-container&quot;&gt;$\ddot x(t^*) = 0$&lt;/span&gt; is fulfilled, or in terms of &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt; until we have&#xA;&lt;span class=&quot;math-container&quot;&gt;$$-2 \sin\theta(t^*) \dot s(t^*)\dot\theta(t^*) - \cos\theta(t^*) s(t^*)\dot\theta^2(t^*) - \frac{1}{2}b\sin\theta(t^*) \dot\theta^2(t^*) + \cos\theta(t^*) \ddot s(t^*) + \frac{1}{2} b\ddot\theta \cos\theta(t^*) -s(t^*) \ddot\theta(t^*) \sin\theta(t^*) \overset{!}{=} 0.$$&lt;/span&gt;&#xA;After this time &lt;span class=&quot;math-container&quot;&gt;$t^*$&lt;/span&gt; the new trajectory is given by&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\vec{x}= \begin{pmatrix}x\\ - \left[x\tan(|\alpha|)+\frac{gx^2}{2v_0\cos^2(|\alpha|)}\right]\end{pmatrix} \quad\text{and}\quad \theta(t)= \dot \theta(t^*) \cdot t + \theta(t^*)$$&lt;/span&gt; &#xA;where &lt;span class=&quot;math-container&quot;&gt;$\alpha = \theta(t^*)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_0 = \dot s(t^*)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My attempt to solve this goes as follows: First defined&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y = \begin{bmatrix}s\\\dot s\\ \theta\\ \dot\theta\end{bmatrix} \Longrightarrow \dot y = \begin{bmatrix}\dot s\\\ddot s\\\dot \theta\\ \ddot\theta\end{bmatrix}$$&lt;/span&gt;&#xA;I solved the Euler-Lagrange eq. for &lt;span class=&quot;math-container&quot;&gt;$\ddot \theta$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\ddot s$&lt;/span&gt; and inserted them into &lt;span class=&quot;math-container&quot;&gt;$\dot y$&lt;/span&gt; which gives &lt;code&gt;rhs&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def rhs(y, t, g=9.81, L=0.01):&#xA;    return np.array([y[1], &#xA;                    (1/(b**2 + L**2 + 24.0**y[0]**2))*&#xA;                    (6.0*b*g*np.cos(y[2])*y[0] - 7.0 *b**2 *g*np.sin(y[2]) - g* L*2.0 *np.sin(y[2]) - 24.0 *g *y[0]**2*np.sin(y[2]) &#xA;                        - 24.0*b* y[0]* y[1]* y[3] - 8.0* b**2* y[0] * y[3]**2 - 2.0* L**2* y[0]* y[3]**2 - 48.0*y[0]**3* y[3]**2), &#xA;                    y[3], &#xA;                    -12.0*(-g *np.cos(y[2]) * y[0] + b* g* np.sin(y[2]) + 4* y[0] * y[1]* y[3] + b *y[0] *y[3]**2)/(b**2 + L**2 + 24 *y[0]**2)&#xA;                    ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then used &lt;code&gt;scipy&lt;/code&gt;'s &lt;code&gt;odeint&lt;/code&gt;algorithm to solve the problem in the follwoing way&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T = np.linspace(0,10,10000)&#xA;b = 0.02&#xA;lstart = 0.02&#xA;ystart = np.array([np.sqrt(lstart**2+ (b/2.0)**2), v, 0.0, 0.0])&#xA;y, data = odeint(rhs, ystart, T, full_output = True, args=(g, L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This worked out and plotting the solutions gives results that I expected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next thing to do was to figure out &lt;span class=&quot;math-container&quot;&gt;$t^*$&lt;/span&gt;. For this I interpolated the solutions and inserted them into the defining condition to find &lt;span class=&quot;math-container&quot;&gt;$t^*$&lt;/span&gt;. The following function is supposed to return the index of &lt;code&gt;T&lt;/code&gt; at which this happens&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def find_tstar_index(T, y):&#xA;    # Find point where bread leaves table&#xA;    s_spl = UnivariateSpline(T, y[:, 0], s=0, k=4)&#xA;    u_spl = UnivariateSpline(T, y[:, 2], s=0, k=4)&#xA;&#xA;&#xA;&#xA;    s_spl_d = s_spl.derivative(n=1)&#xA;    s_spl_2d = s_spl.derivative(n=2)&#xA;&#xA;    u_spl_d = u_spl.derivative(n=1)&#xA;    u_spl_2d = u_spl.derivative(n=2)&#xA;&#xA;    searching_for_zeros = -2.0*np.sin(u_spl(T))*s_spl_d(T)*u_spl_d(T) \&#xA;                        - np.cos(u_spl(T))* s_spl(T)* u_spl_d(T)**2 \&#xA;                        -1/2.0* b* np.sin(u_spl(T))*u_spl_d(T)**2  \&#xA;                        + np.cos(u_spl(T))*s_spl_2d(T)\&#xA;                        + 1/2.0* b* np.cos(u_spl(T))* u_spl_2d(T) \&#xA;                        - s_spl(T)*np.sin(u_spl(T))*u_spl_2d(T)&#xA;&#xA;    tstar_index = np.where(searching_for_zeros &amp;lt; 1e-25)[0][0]&#xA;    return tstar_index&#xA;&#xA;tstar_index = find_tstar_index(T, y)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The next step would be to set up &lt;span class=&quot;math-container&quot;&gt;$\vec{x}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\theta (t)$&lt;/span&gt; using &lt;span class=&quot;math-container&quot;&gt;$t^*$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual problem starts here: I now definie the follwoing function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;angle = lambda t: y[tstar_index, 3] * t + y[tstar_index, 2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The thing is now that depending on the &lt;code&gt;T&lt;/code&gt; defined in the beginning of my code the function &lt;code&gt;angle&lt;/code&gt; changes significantly. For example &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;T_A = np.linspace(0,1,10000)&#xA;T_A = np.linspace(0,10,10000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;give completely different results for the intervall [0,1] when plugged into the respective &lt;code&gt;angle&lt;/code&gt; function... There has to be an error in my logic but after hours of trying to figure out what the problem is I'm kind of lost...&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Complete Code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll post down here my complete code so that you can try it out yourself, but up to line 142 everything seems to work fine (&lt;code&gt;T_fall&lt;/code&gt; is the same array regardless of what &lt;code&gt;T&lt;/code&gt; is...)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import math&#xA;import matplotlib.pyplot as plt&#xA;from scipy.integrate import odeint&#xA;from scipy.interpolate import UnivariateSpline&#xA;&#xA;g = 9.81&#xA;L = 0.1&#xA;#T = np.linspace(0,10,10000)&#xA;b = 0.02&#xA;lstart = 0.02&#xA;table_hight = 1.3&#xA;&#xA;vs = [0.7]&#xA;&#xA;def find_tstar_index(T, y):&#xA;    # Find point where bread leaves table&#xA;    s_spl = UnivariateSpline(T, y[:, 0], s=0, k=4)&#xA;    u_spl = UnivariateSpline(T, y[:, 2], s=0, k=4)&#xA;&#xA;&#xA;&#xA;    s_spl_d = s_spl.derivative(n=1)&#xA;    s_spl_2d = s_spl.derivative(n=2)&#xA;&#xA;    u_spl_d = u_spl.derivative(n=1)&#xA;    u_spl_2d = u_spl.derivative(n=2)&#xA;&#xA;    searching_for_zeros = -2.0*np.sin(u_spl(T))*s_spl_d(T)*u_spl_d(T) \&#xA;                        - np.cos(u_spl(T))* s_spl(T)* u_spl_d(T)**2 \&#xA;                        -1/2.0* b* np.sin(u_spl(T))*u_spl_d(T)**2  \&#xA;                        + np.cos(u_spl(T))*s_spl_2d(T)\&#xA;                        + 1/2.0* b* np.cos(u_spl(T))* u_spl_2d(T) \&#xA;                        - s_spl(T)*np.sin(u_spl(T))*u_spl_2d(T)&#xA;&#xA;    tstar_index = np.where(searching_for_zeros &amp;lt; 1e-25)[0][0]&#xA;    return tstar_index&#xA;&#xA;def wurfparabel (x, v, beta):&#xA;    b = np.abs(beta)&#xA;    return - (x * np.tan(b) + (g * x**2)/(2* v**2* np.cos(b)**2))&#xA;&#xA;def draw_bread(x_p, u, col):&#xA;&#xA;    x_b = x_p + np.array([-b/2.0 * np.sin(u), -b/2.0 * np.cos(u)])&#xA;    x_left_down = x_b + np.array([-L/2.0 * np.cos(u), L/2.0 * np.sin(u)])&#xA;    x_left_up = x_left_down + np.array([b * np.sin(u), b * np.cos(u)])&#xA;    x_right_up = x_left_up + np.array([L * np.cos(u), -L * np.sin(u)])&#xA;    x_right_down = x_right_up + np.array([- b * np.sin(u), - b * np.cos(u)])&#xA;&#xA;    plt.plot([x_b[0], x_left_down[0], x_left_up[0], x_right_up[0], x_right_down[0], x_b[0]], &#xA;        [x_b[1], x_left_down[1], x_left_up[1], x_right_up[1], x_right_down[1], x_b[1]], col)&#xA;&#xA;&#xA;    pass&#xA;&#xA;def find_nearest(array, value):&#xA;    array = np.asarray(array)&#xA;    idx = (np.abs(np.abs(array) - value)).argmin()&#xA;    return idx&#xA;&#xA;def x_p(y):&#xA;    return np.array([y[:, 0] * np.cos(y[:, 2]) + b/2.0 * np.sin(y[:, 2]), &#xA;                    - y[:, 0] * np.sin(y[:, 2] + b/2.0 * np.cos(y[:, 2]))&#xA;                    ])&#xA;&#xA;plt.figure()&#xA;def test(T):&#xA;&#xA;    def bread_falling_trajectory(v):&#xA;        &quot;&quot;&quot;&#xA;        Function that calculates the trajectory of a piece of toast falling from a table with the initial velocity v.&#xA;        &quot;&quot;&quot;&#xA;&#xA;        # Define starting values. The  first value is chosen in such a way that the Center of mass is of the same hight as the zero point.&#xA;        ystart = np.array([np.sqrt(lstart**2+ (b/2.0)**2), v, 0.0, 0.0])&#xA;&#xA;        x_sol_start = np.array([ystart[0] * np.cos(ystart[2]) + b/2.0 * np.sin(ystart[2]), &#xA;                        - ystart[0] * np.sin(ystart[2] + b/2.0 * np.cos(ystart[2]))&#xA;                        ])&#xA;&#xA;        # Define the right hand side of the differential equations.&#xA;        def rhs(y, t, g, L):&#xA;            return np.array([y[1], &#xA;                            (1/(b**2 + L**2 + 24.0**y[0]**2))*&#xA;                            (6.0*b*g*np.cos(y[2])*y[0] - 7.0 *b**2 *g*np.sin(y[2]) - g* L*2.0 *np.sin(y[2]) - 24.0 *g *y[0]**2*np.sin(y[2]) &#xA;                                - 24.0*b* y[0]* y[1]* y[3] - 8.0* b**2* y[0] * y[3]**2 - 2.0* L**2* y[0]* y[3]**2 - 48.0*y[0]**3* y[3]**2), &#xA;                            y[3], &#xA;                            -12.0*(-g *np.cos(y[2]) * y[0] + b* g* np.sin(y[2]) + 4* y[0] * y[1]* y[3] + b *y[0] *y[3]**2)/(b**2 + L**2 + 24 *y[0]**2)&#xA;                            ])&#xA;&#xA;        # Solve the ODE&#xA;        y, data = odeint(rhs, ystart, T, full_output = True, args=(g, L))&#xA;&#xA;        # Get the trajectory&#xA;        x_sol = x_p(y)&#xA;&#xA;        #Find the index of x_sol at which the bread leaves the table.&#xA;        tstar_index = find_tstar_index(T, y)&#xA;&#xA;        # Lösung ab Freier Fall&#xA;        # EQM is given by \ddot theta = 0, solving this and inserting the boundry conditions leads to winkel&#xA;        winkel = lambda zeit: y[tstar_index, 3] * zeit + y[tstar_index, 2]&#xA;        print(y[tstar_index, 3], y[tstar_index, 2] )&#xA;&#xA;        # for the spatial part we essentially have a free falling rod, so we solve the eom and construct trajectory&#xA;        v0 = 1/2.0 * np.sqrt(4*y[tstar_index,1]**2 +4*b*y[tstar_index,1]* y[tstar_index,3] + (b**2 + 4*y[tstar_index,0]**2)*y[tstar_index,3]**2)&#xA;        beta = y[tstar_index, 2]&#xA;        trajectory_ = lambda x: wurfparabel(x, v0, beta)&#xA;        y_0 = np.abs(x_p(y)[1][tstar_index]-trajectory_(x_p(y)[0][tstar_index]))&#xA;&#xA;        trajectory = lambda x: trajectory_(x)  + y_0&#xA;&#xA;&#xA;        # Creat a plot of the results &#xA;        x_free_fall = np.linspace(x_sol[0][tstar_index], 0.45, N)&#xA;&#xA;        sol_dict = {'x_sol_start': x_sol_start, 'table_traje': x_sol,&#xA;                    'x_free_fall': x_free_fall, 'free_fall_traj': trajectory(x_free_fall), 'y': y}&#xA;&#xA;        return sol_dict, tstar_index, winkel&#xA;&#xA;&#xA;&#xA;    for k, v in enumerate(vs):&#xA;        dicct, index, winkel = bread_falling_trajectory(v)&#xA;        # Plotting all the results&#xA;        #plt.subplot(3,1,k+1)&#xA;        plt.plot(dicct['x_sol_start'][0],dicct['x_sol_start'][1], &quot;b*&quot;)&#xA;        # Plot the points where the bread leaves the table.&#xA;        plt.plot(dicct['table_traje'][0][index], dicct['table_traje'][1][index], &quot;bo&quot;)&#xA;        # Plot the trajectroy up until this point&#xA;        plt.plot(dicct['table_traje'][0][:index+1], dicct['table_traje'][1][:index+1], label=r&quot;&lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;={}&quot;.format(v))&#xA;        # Draw the bread &#xA;        draw_bread(dicct['table_traje'][:,0:index:100], np.pi/6.5 + dicct['y'][0:index:100,2], &quot;g&quot;)&#xA;&#xA;        x_p_free_fall = np.array([dicct['x_free_fall'],dicct['free_fall_traj']])&#xA;&#xA;        spec_index = find_nearest(x_p_free_fall[1], table_hight)&#xA;&#xA;        delta_T = T[-1]/N&#xA;        T_fall = np.linspace(0, spec_index * delta_T, spec_index)&#xA;&#xA;        angle_free_fall = np.pi/6.5 + winkel(T_fall)&#xA;&#xA;        draw_bread(x_p_free_fall[:,0:spec_index:100000], angle_free_fall[0:spec_index:100000], &quot;r&quot;)&#xA;&#xA;&#xA;        # Plot the trajectroy after the bread has left the table&#xA;        plt.plot(dicct['x_free_fall'], dicct['free_fall_traj'])&#xA;&#xA;&#xA;        plt.ylim(-1.5, 0.1)&#xA;        plt.xlim(-0.1, 1.5)&#xA;        plt.plot(0.0, 0.0, &quot;b*&quot;)&#xA;        plt.xlabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; [m]&quot;)&#xA;        plt.ylabel(r&quot;&lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; [m]&quot;)&#xA;        plt.legend()&#xA;&#xA;&#xA;    return T_fall, winkel(T_fall)&#xA;N=1000000&#xA;ta, winekla = test(np.linspace(0,1,N))&#xA;tb, wineklb = test(np.linspace(0,10,N))&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the huge &lt;code&gt;test&lt;/code&gt; function is just to play around with the different &lt;code&gt;T&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; To illustrate the problem, two plots with different time samplings (N=1000,10000,...) and different time lengths (the first picture for T_end = 7s and the second one for T_end=10s).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gBMym.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gBMym.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/m1Qml.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m1Qml.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30521" LastEditorUserId="30521" LastEditDate="2019-04-20T16:54:50.653" LastActivityDate="2019-04-20T16:54:50.653" Title="Time sampling changes solution" Tags="|ode|computational-physics|scipy|numerics|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="31473" PostTypeId="1" CreationDate="2019-04-19T20:01:20.093" Score="5" ViewCount="352" Body="&lt;p&gt;Given a singular matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, what is the fastest method to find a single non-zero solution to &lt;span class=&quot;math-container&quot;&gt;$Ax=0$&lt;/span&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that we are &lt;strong&gt;not&lt;/strong&gt; looking for the whole kernel, we just want any non-zero vector in it. I know we can use SVD to find eigenvectors, but that seems much too complicated.&lt;/p&gt;&#xA;" OwnerUserId="30525" LastEditorUserId="20417" LastEditDate="2019-04-20T16:02:18.917" LastActivityDate="2019-05-23T22:56:58.387" Title="Computing any element of the null space of a singular matrix" Tags="|linear-algebra|matrix|" AnswerCount="2" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31478" PostTypeId="1" CreationDate="2019-04-21T13:28:41.900" Score="5" ViewCount="92" Body="&lt;p&gt;Consider the continuity equation &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\partial_t \rho(x,t) + \operatorname{div} (b(x,t) \rho(x,t)) = 0, \qquad t \in [0,T], \quad x \in \mathbb R^N, &#xA;$$&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$b \in L^1((0,T), W^{1,p}(\mathbb R^N))$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Have any numerical methods been used to approximate the (unique renormalized) distributional solution of this problem? Can you point out any references on this topic?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What if we replace &lt;span class=&quot;math-container&quot;&gt;$W^{1,p}$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$BV$&lt;/span&gt;?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Clearly, if &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is smooth, the problem is much more classical. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I've asked a related question on &lt;a href=&quot;https://mathoverflow.net/questions/328576/numerics-for-continuity-equation-with-sobolev-vector-field&quot;&gt;MathOverflow&lt;/a&gt; and a specific implementation problem (with a specific &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; of bounded variation) on &lt;a href=&quot;https://mathematica.stackexchange.com/questions/195547/numerical-methods-to-solve-a-continuity-equation&quot;&gt;Mathematica StackExchange&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="30536" LastActivityDate="2019-04-27T13:35:49.653" Title="Numerical methods for the continuity equation with Sobolev vector field" Tags="|finite-element|pde|finite-difference|hyperbolic-pde|numerics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31480" PostTypeId="1" CreationDate="2019-04-21T16:16:11.047" Score="3" ViewCount="2033" Body="&lt;p&gt;Consider the explicit midpoint method, i.e&#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_{n+1}-y_{n-1} = 2hf(y_n).$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm asked to apply this method to the linear test equation, &lt;span class=&quot;math-container&quot;&gt;$f(y_n) = \lambda y_n,$&lt;/span&gt; in order to find the method's stability region. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have &#xA;&lt;span class=&quot;math-container&quot;&gt;$$y_{n+1}-y_{n-1} = 2h\lambda y_n.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the problems I've done before I wouldn't have the &lt;span class=&quot;math-container&quot;&gt;$y_{n-1}$&lt;/span&gt; term so I could just easily write &lt;span class=&quot;math-container&quot;&gt;$y_{n+1} = R(z)y_n.$&lt;/span&gt; How would I solve this? &lt;/p&gt;&#xA;" OwnerUserId="29218" LastActivityDate="2019-06-02T13:00:12.077" Title="Stability region of explicit midpoint method" Tags="|stability|numerics|explicit-methods|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31482" PostTypeId="2" ParentId="31468" CreationDate="2019-04-22T04:21:03.203" Score="1" Body="&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$x = (I-nn^T)y$&lt;/span&gt; &amp;nbsp; then for any vector &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; the constraint is automatically satisfied.&#xA;&lt;br&gt;Let &lt;span class=&quot;math-container&quot;&gt;$f_x$&lt;/span&gt; be the gradient of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; with respect to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, then the gradient wrt to &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$f_y=(I-nn^T)f_x$&lt;/span&gt;&#xA;&lt;br&gt;&lt;br&gt;Now solve the &lt;strong&gt;&lt;em&gt;unconstrained&lt;/em&gt;&lt;/strong&gt; minimization problem for &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; via conjugate gradients, and calculate the corresponding &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; afterwards. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\eqalign{&#xA;y^* &amp;amp;= \arg\min f(y) \cr&#xA;x^* &amp;amp;= (I-nn^T)y^* \cr&#xA;}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="27795" LastActivityDate="2019-04-22T04:21:03.203" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="31483" PostTypeId="1" CreationDate="2019-04-22T18:24:35.750" Score="1" ViewCount="52" Body="&lt;p&gt;I try to solve numerically the following PDE for &lt;span class=&quot;math-container&quot;&gt;$E(r, z)$&lt;/span&gt; with a cylindrical symmetrie (i. e. &lt;span class=&quot;math-container&quot;&gt;$E(r, z) = E(-r, z)$&lt;/span&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial E}{\partial z} = \frac{i}{2k} \Delta E + \mathcal{N}(E)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\Delta$&lt;/span&gt; is the Laplace operator in transversal direction and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; a real number. I want to use the Crank-Nicolson scheme to treat the Laplace operator and the Adams-Bashforth scheme to treat the nonlinearity (&lt;span class=&quot;math-container&quot;&gt;$\mathcal N(E)$&lt;/span&gt;). &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is defined by &lt;span class=&quot;math-container&quot;&gt;$r_j = 0 + j\Delta r$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$j = 0 \dots N$&lt;/span&gt;. Since the Laplace in cylinder coordinates is given by: &lt;span class=&quot;math-container&quot;&gt;$\Delta E = \frac{\partial^2E}{\partial r^2} +  \frac{1}{r}\frac{\partial E}{\partial r} $&lt;/span&gt; one gets the following representation of the Laplace operator:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\Delta E_j^n = E^n_{j-1} -2 E^n_j + E^n_{j+1} + \frac{1}{2j}(E^n_{j+1} - E^n_{j-1})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The given PDE equation is therefore given by the following, where the nonlinearity is treated by the Adams-Bashforth scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E_j^{n+1}  - E_j^n = i\delta (\Delta_j E^{n+1}_j + \Delta_j E^{n}_j) + (3/2 \mathcal N^n_j - 1/2 N^{n-1}_{j})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\delta = \frac{\Delta z}{4 k \Delta r^2}$&lt;/span&gt;. From this &lt;span class=&quot;math-container&quot;&gt;$E^{n+1}_j $&lt;/span&gt; can be expressed the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$E^{n+1}_j = L_-^{-1} [L_+E^n_j + 3/2\mathcal N^n_j - 1/2\mathcal N^{n-1}_j]$&lt;/span&gt; with the following matrixes &lt;span class=&quot;math-container&quot;&gt;$L_-$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$L_+$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ L_{\pm} = \left( \begin{array}{rrrr}&#xA;1\mp2i\delta  &amp;amp; \pm i\delta v_0   \\&#xA;\pm i\delta u_1 &amp;amp; 1\mp2i\delta &amp;amp; \pm i\delta v_1\\&#xA; &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp; \\&#xA; &amp;amp;  &amp;amp;  &amp;amp;  &amp;amp; \\&#xA; &amp;amp;  &amp;amp;  &amp;amp; \pm i\delta u_N &amp;amp; 1\mp2i\delta\\&#xA;\end{array}\right) &#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$u_j = 1 - 1/(2j)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$v_j = 1 + 1/(2j)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my problem the following boundary conditions are given: &lt;span class=&quot;math-container&quot;&gt;$ E(r = r_{max}, z) = 0 $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\frac{\partial E}{\partial r} |_{r=0}$&lt;/span&gt;. The first one for &lt;span class=&quot;math-container&quot;&gt;$r=r_{max}$&lt;/span&gt; I can easily incorporate by changing the last row of &lt;span class=&quot;math-container&quot;&gt;$ L_{\pm} $&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other one (&lt;span class=&quot;math-container&quot;&gt;$\frac{\partial E}{\partial r} |_{r=0}$&lt;/span&gt;) gives me trouble. I somehow have to change the entries of the first row of &lt;span class=&quot;math-container&quot;&gt;$L_{\pm}$&lt;/span&gt;, but I don't know how. I got some working boundary conditions for the case the nonlinearity is 0, but they break as soon as I add a nonlinearity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am greatful for any help.&lt;/p&gt;&#xA;" OwnerUserId="30548" LastActivityDate="2019-04-22T18:24:35.750" Title="Boundary conditions for a Non-linear Schrödinger equation using an extended crank nicolson scheme" Tags="|pde|finite-difference|boundary-conditions|crank-nicolson|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31484" PostTypeId="1" AcceptedAnswerId="31490" CreationDate="2019-04-22T18:27:34.587" Score="8" ViewCount="1585" Body="&lt;p&gt;I have coordinates of a set of points that form a closed loop that lies in a 3D surface. I know the equation of the surface and I can calculate it's surface normal at any point. I found that for a unit speed curve &lt;span class=&quot;math-container&quot;&gt;$\alpha(t)$&lt;/span&gt; I can calculate the geodesic curvature as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\kappa_g(t) = \alpha''(t)\cdot (n(t)\times\alpha'(t))&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$n(t)$&lt;/span&gt; is the surface normal. I am confused about how to calculate the unit-speed derivatives &lt;span class=&quot;math-container&quot;&gt;$\alpha'(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\alpha''(t)$&lt;/span&gt;. I think I should use some finite difference scheme like&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\alpha'(t) \approx \frac{\alpha(t + h) - \alpha(t - h)}{2h}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my algorithm for calculating the geodesic curvature at any point &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; of this curve should be as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Identify a reference point on the curve to measure arc-length from.&lt;/li&gt;&#xA;&lt;li&gt;For the point &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; calculate the arc-length by adding up the Euclidean distances between successive points from the reference point to the point P. This arc length gives me &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Find the position vectors of points at &lt;span class=&quot;math-container&quot;&gt;$t + h$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t - h$&lt;/span&gt; by interpolation and then use the above finite difference equation to calculate &lt;span class=&quot;math-container&quot;&gt;$\alpha'(t)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is my algorithm correct? I am not clear about how to get arc-length parameterization from the points on the curve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any standard algorithms (or even Python libraries) for such a calculation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 1:&lt;/strong&gt;&#xA;I implemented @Futurologist 's wonderful answer for the test case of a circle on a cone as shown below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O8a9m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O8a9m.png&quot; alt=&quot;Cone with circle&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the Frenet-Serret frame along the circle as shown below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G4FUR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G4FUR.png&quot; alt=&quot;TNB Frame&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The total curvature at each point on the circle is &lt;span class=&quot;math-container&quot;&gt;$k_i = 1.0$&lt;/span&gt; which is correct because I designed the circle to have a radius of &lt;span class=&quot;math-container&quot;&gt;$1.0$&lt;/span&gt;. I get the geodesic curvature at all points as &lt;span class=&quot;math-container&quot;&gt;$-1/\sqrt{2}$&lt;/span&gt;. So everything looks good. Can someone confirm if the sign of the geodesic curvature or the &lt;span class=&quot;math-container&quot;&gt;$N_i$&lt;/span&gt; vectors in the &lt;strong&gt;TNB&lt;/strong&gt; frames are correct? I know that I can flip it by changing the order in the cross-product for &lt;span class=&quot;math-container&quot;&gt;$b_i$&lt;/span&gt; but I am asking if there are is standard convention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 2:&lt;/strong&gt;&#xA;I had to use &lt;span class=&quot;math-container&quot;&gt;$b_i = (\alpha_{i} - \alpha_{i-1})\times(\alpha_{i+1} - \alpha_i)$&lt;/span&gt; to get the correct sign for &lt;span class=&quot;math-container&quot;&gt;$N_i$&lt;/span&gt; and a right-handed &lt;strong&gt;TNB&lt;/strong&gt; frame. Also at some of the points the tangent &lt;span class=&quot;math-container&quot;&gt;$T_i$&lt;/span&gt; was pointing in the opposite direction of the increasing arc-length parameter due to angle between &lt;span class=&quot;math-container&quot;&gt;$b_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; increasing above &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;. In those cases, I had to flip the &lt;span class=&quot;math-container&quot;&gt;$T_i$&lt;/span&gt;. Now I get the geodesic curvature at all points as &lt;span class=&quot;math-container&quot;&gt;$1/\sqrt{2}$&lt;/span&gt; and the vectors &lt;span class=&quot;math-container&quot;&gt;$N_i$&lt;/span&gt; point inwards as desired.&lt;/p&gt;&#xA;" OwnerUserId="30549" LastEditorUserId="30549" LastEditDate="2019-04-27T18:50:58.977" LastActivityDate="2019-04-27T18:50:58.977" Title="How to calculate the geodesic curvature of a discrete 3D curve?" Tags="|computational-geometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="31486" PostTypeId="1" CreationDate="2019-04-23T06:03:10.517" Score="2" ViewCount="1666" Body="&lt;p&gt;I'm currently analysing some spatial point patterns that come from some fluid dynamics simulations and I'm having some difficulty computing the structure factor, &lt;span class=&quot;math-container&quot;&gt;$S(\pmb{k})$&lt;/span&gt;, from &lt;em&gt;both&lt;/em&gt; the positions of the points and the radial distribution function, &lt;span class=&quot;math-container&quot;&gt;$g(\pmb{r})$&lt;/span&gt;. I have been following the &lt;a href=&quot;https://en.wikipedia.org/wiki/Structure_factor&quot; rel=&quot;nofollow noreferrer&quot;&gt;structure factor Wikipedia page&lt;/a&gt; and also &lt;a href=&quot;https://arxiv.org/pdf/1606.03610v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper on arxiv&lt;/a&gt; but I seem to be getting results that don't make sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is my class object to compute the radial distribution function for said spatial point patterns (I've omitted the help data and assertions from the code for clarity). The class takes in a numpy array or list of numpy arrays, finds the locations of the nonzero values, computes the distances between each nonzero value and then computes the radial distribution. Note that here, we consider an array cell a point if it has a nonzero value&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import bisect&#xA;&#xA;class pair_correlation_function():&#xA;&#xA;    def __init__(self,data,annulus_width,boundary=None):&#xA;&#xA;        self.data = [data] if type(data) is np.ndarray else data&#xA;        self.dr = annulus_width&#xA;        self.boundary = boundary&#xA;&#xA;    def get_positions(self):&#xA;&#xA;        positions = list()&#xA;&#xA;        for item in self.data:&#xA;&#xA;            rows,columns = np.nonzero(item)&#xA;            positions.append(np.array(list(zip(rows,columns))))&#xA;&#xA;        return positions&#xA;&#xA;    def RDF(self):&#xA;&#xA;        positions = self.get_positions()&#xA;&#xA;        Lx,Ly = np.shape(self.data[0])&#xA;        area = Lx*Ly&#xA;        radii = list(x for x in range(int(Lx/(2*self.dr))))&#xA;&#xA;        all_radial_distributions = list()&#xA;&#xA;        for item in positions:&#xA;&#xA;            if self.boundary == 'periodic':&#xA;&#xA;                item_new = np.vstack(np.array([np.abs(item[k]-item[k+1:]) for k in range(len(item)-1)]))&#xA;                item_new[item_new &amp;gt; Lx/2] = Lx-item_new[item_new &amp;gt; Lx/2]&#xA;                norms = [np.sqrt((position**2).sum(axis=0)) for position in item_new]&#xA;                norms = sorted(norms)&#xA;&#xA;            else:&#xA;&#xA;                norms = [np.sqrt(((item[k]-item[k+1:])**2).sum(axis=1)) for k in range(len(item)-1)]&#xA;                norms = sorted(np.hstack(norms))&#xA;&#xA;            number_particles = len(item)&#xA;            item_rdf = list()&#xA;&#xA;            for r in radii:&#xA;&#xA;                i = bisect.bisect_left(norms,r*self.dr)&#xA;                j = bisect.bisect_left(norms,(r+1)*self.dr)&#xA;                if i != len(norms) and j != len(norms): particle_count = len(norms[i:j])&#xA;&#xA;                normalisation = (2*r+self.dr)*np.pi*self.dr*number_particles**2&#xA;                bin_value     = 2*area*particle_count/normalisation&#xA;&#xA;                item_rdf.append(bin_value)&#xA;&#xA;            all_radial_distributions.append(item_rdf)&#xA;&#xA;        radii = np.array(radii)*self.dr&#xA;&#xA;    return radii,all_radial_distributions&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Below is a sample array (a triangular lattice) with which to play around (we are assuming periodic boundary conditions here because my actual data comes from a simulation with periodic boundaries)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tri_period = np.array([[0,0,0,0],[0,1,0,1],[0,0,0,0],[1,0,1,0]])&#xA;triangular_lattice = np.tile(tri_period,(16,16))&#xA;dr = .025&#xA;&#xA;radii, RDF = pair_correlation_function(triangular_lattice,dr,'periodic').RDF()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and plotting gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hq00E.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hq00E.png&quot; alt=&quot;Radial_Distribution_Function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, we see a series of Bragg peaks as expected (though I'm not sure why the peaks segment into smooth, monotone decreasing bands? My actual data doesn't do this though and looks correct.) Now, if I try to compute the structure factor given the positions (see arxiv paper above, equation &lt;span class=&quot;math-container&quot;&gt;$(2)$&lt;/span&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def sf_from_positions(positions,box_dimension):&#xA;&#xA;    sf = list()&#xA;    modes = list(x for x in range(1,int(box_dimension)))&#xA;&#xA;    dk = 2*np.pi/box_dimension&#xA;&#xA;    for h in modes:&#xA;&#xA;        k_vec = np.array([1,1])*dk*h&#xA;        summation = 0&#xA;&#xA;        for position in positions:&#xA;&#xA;            summation += np.exp(-1j*k_vec.dot(position))&#xA;&#xA;        sf.append(abs(summation)**2/len(positions))&#xA;&#xA;    return sf, modes&#xA;&#xA;rows,columns = np.nonzero(triangular_lattice)&#xA;positions = list(zip(rows,columns))&#xA;dimension = np.shape(triangular_lattice)[0]&#xA;factor, wave_modes = sf_from_positions(positions,dimension)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and plotting gives&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YbJy8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YbJy8.png&quot; alt=&quot;Structure_factor_from_positions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which looks off, because the Fourier transform of Bragg peaks should also be Bragg peaks (see &lt;a href=&quot;https://arxiv.org/pdf/1801.06924.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this arxiv paper&lt;/a&gt;, page 12). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my first question is, what is going wrong in my computation of the structure factor? Can anyone see what I have done wrong? And secondly, I was wondering how I would integrate the radial distribution function to get the structure factor, through the formula &lt;span class=&quot;math-container&quot;&gt;$$S(\pmb{k}) = 1 + 4 \pi \rho \int_{\mathbb{R^{n}}} [g(\pmb{r})-1] e^{-i \pmb{k} \cdot \pmb{r}} d \pmb{r} $$&lt;/span&gt; I tried to use the cumtrapz module in scipy but it gave me a structure factor that was negative (impossible) and that oscillated with an increasing wave packet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry if the question is overly involved. Thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="16852" LastEditorUserId="16852" LastEditDate="2019-04-25T15:16:45.273" LastActivityDate="2019-04-25T15:18:27.380" Title="Computing the structure factor from positions and radial distribution function" Tags="|python|data-analysis|" AnswerCount="0" CommentCount="1" ClosedDate="2019-06-19T19:43:45.140" ContentLicense="CC BY-SA 4.0" />
  <row Id="31487" PostTypeId="1" CreationDate="2019-04-23T12:37:27.180" Score="0" ViewCount="184" Body="&lt;p&gt;This is a follow up to my previous question posted &lt;a href=&quot;https://scicomp.stackexchange.com/questions/31388/formulation-of-the-least-squares-parameter-estimation-problem&quot;&gt;here&lt;/a&gt;.&#xA;I am solving an optimization problem using &lt;code&gt;fmincon&lt;/code&gt; in MATLAB. There are no equality constraints in my model.  I want to pass ODEs as constraint to fmincon using, non-linear constraint, nlcon argument.&#xA;The function model  contains odes &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dz = model(t, z, c)  &#xA;              dz(1) = ..&#xA;              dz(2) = ..&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd like to ask how the ode intergrator has to be called in the place of nlcon in fmincon(@objective,p0,A,b,Aeq,beq,lb,ub,nlcon).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%p0 = Initial Value of Parameters&#xA;A = [];&#xA;b = [];&#xA;Aeq = [];&#xA;beq = [];&#xA;nlcon  = ode45(@(t,z)model(t,z,x), tSpan, z0); % Is this correct?&#xA;p = fmincon(@objective,p0,A,b,Aeq,beq,lb,ub,nlcon); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since the ode function is already called in nlcon , is it required to call the function model again inside the objective function?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function cost = objective(c,expZ,tSpan,z0)&#xA;sol = ode45(@(t,z)model(t,z,c), tSpan, z0); % Is this step required?&#xA;ModelZ = sol(end,:)%  the solutions obtained from model at end point of tSpan&#xA;cost = ModelZ-expZ; %expZ contains steady state values of z1,z2.. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="29087" LastActivityDate="2019-04-23T12:37:27.180" Title="Parameter estimation using fmincon" Tags="|optimization|least-squares|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31489" PostTypeId="2" ParentId="31473" CreationDate="2019-04-23T14:14:39.593" Score="0" Body="&lt;p&gt;You said fastest way, I assume you mean calculation operations, as opposed to code run time - not sure if there is an optimized routine for the algorithm I describe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use row reduction (Gaussian Elimination) with pivoting.  See Strang's Linear Algebra textbook (Section 3.2 in 3rd ed) or &lt;a href=&quot;https://en.wikipedia.org/wiki/Gaussian_elimination&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gaussian Elimination&lt;/a&gt;.  It will fail because your matrix is singular.  Keep doing it and you will be able to solve for x by backward substitution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2 2 3&#xA;2 2 1&#xA;1 1 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After subtracting 1.0 times first row from 2nd row, and 0.5 from 3rd row:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 1 1.5&#xA;0 0 -2&#xA;0 0 3.5    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can't get a 1 in 2nd row, 2nd col, even with row exchanges, so you are done.  A solution by back substitution is x=[-1 1 0].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could solve this in MATLAB with null(A) or A\b, but you are right, this is way more calculation than needed.  &lt;/p&gt;&#xA;" OwnerUserId="27724" LastEditorUserId="27724" LastEditDate="2019-04-23T21:00:54.977" LastActivityDate="2019-04-23T21:00:54.977" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="31490" PostTypeId="2" ParentId="31484" CreationDate="2019-04-23T14:51:48.077" Score="8" Body="&lt;p&gt;Why don't you try something geometric rather than numerical. I propose the following approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the points from the loop form the sequence &lt;span class=&quot;math-container&quot;&gt;$\alpha_i \,\, : \,\, i = 1, 2, 3 ... I$&lt;/span&gt; and as you said, all of them lie on a given smooth surface. Furthermore, you know the unit normal &lt;span class=&quot;math-container&quot;&gt;${n}$&lt;/span&gt; everywhere on the surface. Then you know the unit normal to the surface at every point &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; from the loop. Denote that normal by &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each &lt;span class=&quot;math-container&quot;&gt;$i = 2,3,... I-1$&lt;/span&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; First calculate the vector &lt;span class=&quot;math-container&quot;&gt;$$b_i = (\alpha_{i+1} - \alpha_i)\times(\alpha_i - \alpha_{i-1})$$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; Then calculate the unit tangent to the loop at the point &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt;  as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$T_i = \frac{n_i \times b_i}{{\|}n_i \times b_i\|}$$&lt;/span&gt; The idea is that the three points &lt;span class=&quot;math-container&quot;&gt;$\alpha_{i+1}, \, \alpha_i, \, \alpha_{i-1}$&lt;/span&gt; form an approximation of the osculating plane of the loop at &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; and the tangent &lt;span class=&quot;math-container&quot;&gt;$T_i$&lt;/span&gt; must lie simultaneously in the osculating plane and in the plane tangent to the surface at &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt;. The vectors &lt;span class=&quot;math-container&quot;&gt;$\alpha_{i+1} - \alpha_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\alpha_{i} - \alpha_{i-1}$&lt;/span&gt; span the approximate osculating plane so their cross product &lt;span class=&quot;math-container&quot;&gt;$b_i$&lt;/span&gt; is normal to the osculating plane. Furthermore, &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; is normal to the tangent plane to the surface at &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt;. Therefore, the tangent must be normal to both vectors &lt;span class=&quot;math-container&quot;&gt;$b_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt; in order to be aligned with the intersection line of the two planes. Hence the proposed double cross product between &lt;span class=&quot;math-container&quot;&gt;$b_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n_i$&lt;/span&gt;. The denominator normalizes the vector to make it of unit length. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3.&lt;/strong&gt; Next calculate the unit binormal vector to the loop at &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; as follows&#xA;&lt;span class=&quot;math-container&quot;&gt;$$B_i = \frac{b_i}{\|b_i\|}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4.&lt;/strong&gt; The unit normal then can be computed as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$N_i = B_i \times T_i$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;5.&lt;/strong&gt; To calculate the (total) curvature of the loop, compute the scalar&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\kappa_i = \frac{1}{2}\left( \frac{\|T_{i+1} - T_i\|}{\|\alpha_{i+1} - \alpha_i\|} + \frac{\|T_{i} - T_{i-1}\|}{\|\alpha_{i} - \alpha_{i-1}\|}\right)$$&lt;/span&gt;&lt;br&gt;&#xA;The idea is that we approximate &lt;span class=&quot;math-container&quot;&gt;$\Big\|\frac{dT}{ds}\Big\|$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the arc-length parametrization of the loop &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;, by calculating the magnitute of the change of the unit tangent from &lt;span class=&quot;math-container&quot;&gt;$T_i$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$T_{i+1}$&lt;/span&gt; and then dividing by the distance traveled &lt;span class=&quot;math-container&quot;&gt;$\|\alpha_{i+1} - \alpha_i\|$&lt;/span&gt; from point &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; to point &lt;span class=&quot;math-container&quot;&gt;$\alpha_{i+1}$&lt;/span&gt;. The norm  &lt;span class=&quot;math-container&quot;&gt;$\|\alpha_{i+1} - \alpha_i\|$&lt;/span&gt; serves as an approximation of arc-length. We do the same for the loop points with indices &lt;span class=&quot;math-container&quot;&gt;$i-1$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and finally we average the result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;6.&lt;/strong&gt; Now, you can interpret the vector &lt;span class=&quot;math-container&quot;&gt;$\kappa_i N_i$&lt;/span&gt; as an approximation of the derivative &lt;span class=&quot;math-container&quot;&gt;$\frac{dT}{ds} = \frac{d^2\alpha}{ds^2}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the arc-length parametrization of the loop &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt;    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;7.&lt;/strong&gt; The geodesic curvature of the loop at point &lt;span class=&quot;math-container&quot;&gt;$\alpha_i$&lt;/span&gt; can be calculated as&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\kappa_g(i) = \kappa_i \Big(N_i \cdot (n_i \times T_i)\Big)$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="20964" LastEditorUserId="20964" LastEditDate="2019-04-24T03:04:14.010" LastActivityDate="2019-04-24T03:04:14.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="31491" PostTypeId="2" ParentId="31388" CreationDate="2019-04-23T23:47:05.197" Score="2" Body="&lt;p&gt;You can think of the ODE as a constraint between the parameters &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; and the observed variable &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;.&#xA;To enforce constraints in optimization problems, you can introduce a Lagrange multiplier, which we'll call &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;.&#xA;In that case, the Lagrangian for your problem is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$L(y, \lambda, V) = \sum_i|y(t_i) - y_i^\text{experiment}|^2 + \int_0^T\lambda\cdot (\dot y - f(y, V))dt.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll then seek an extremum of the Lagrangian.&#xA;A good understanding of variational calculus is going to be really essential from here on out.&#xA;If that subject is new to you, &lt;a href=&quot;https://books.google.com/books?isbn=0486630692&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weinstock's book&lt;/a&gt; is a great introduction.&#xA;The next thing you'll want to know about is the adjoint method.&#xA;Unfortunately a lot of books and references that describe the adjoint method are very confusing, but it's been discussed on this forum a handful of times, for example &lt;a href=&quot;https://scicomp.stackexchange.com/questions/14259/understanding-the-cost-of-adjoint-method-for-pde-constrained-optimization&quot;&gt;here&lt;/a&gt;.&#xA;Another term you might want to google is &quot;system identification&quot;, but the wikipedia article for it isn't very informative so you may have to do some more digging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a really nice Python library for numerical fitting from symbolic definitions of the models, called &lt;a href=&quot;https://github.com/tBuLi/symfit&quot; rel=&quot;nofollow noreferrer&quot;&gt;symfit&lt;/a&gt;.&#xA;They have an example of ODE models &lt;a href=&quot;https://symfit.readthedocs.io/en/stable/fitting_types.html#ode-fitting&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2019-04-23T23:47:05.197" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32491" PostTypeId="1" AcceptedAnswerId="32493" CreationDate="2019-04-24T10:24:13.507" Score="1" ViewCount="1106" Body="&lt;p&gt;I am trying to solve the Poisson problem with Dirichlet boundary condition in 1D:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\begin{array}{rcl}&#xA;- \mu \Delta u &amp;amp; = &amp;amp; f~in~[0,1], \\&#xA;u(0) &amp;amp; = &amp;amp; 0, \\&#xA;u(1) &amp;amp; = &amp;amp; 0,&#xA;\end{array}&#xA;\end{equation}&lt;/span&gt;&#xA;using the FFT method.&#xA;However, I get strange results for the rate of convergence of the method, compared to the finite difference method: the FFT method only converge with order 1 while the finite difference method is of order 2 in space.&#xA;Is it ok ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is my code in python.&#xA;Thank you for your help.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.fftpack import dst&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import time&#xA;&#xA;def Poisson_1D(mu,f,N):&#xA;    x = np.linspace(0,1,N)&#xA;    hh = (x[1]-x[0])**2&#xA;    NN = x.size&#xA;    u = np.zeros(NN)&#xA;    d = 2*np.ones(NN-2)&#xA;    dd = -1*np.ones(NN-3)&#xA;    A = np.diag(d) + np.diag(dd,1) + np.diag(dd,-1)&#xA;    ff = f(x[1:-1])*hh/mu&#xA;    u[1:-1] = np.linalg.solve(A,ff)&#xA;    return x,u&#xA;&#xA;def Poisson_fft_1D(mu,f,N,L=1):&#xA;    x = np.linspace(0,L,N)&#xA;    ff = f(x)&#xA;    F = dst(ff,type=1)&#xA;    k = np.array([(np.pi*(i+1)/L)**2 for i in range(0,N)])&#xA;    U = F/(mu*k)&#xA;    u = dst(U,type=1)/(2*(N+1))&#xA;    return x, u&#xA;&#xA;def f3(x): return 16*mu*np.pi**2*np.sin(4*np.pi*x)&#xA;def u3(x): return np.sin(4*np.pi*x)&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;&#xA;&#xA;    error_fd = []&#xA;    error_fft = []&#xA;    nlist = np.arange(3,13)&#xA;    for n in nlist:&#xA;        N = 2**n&#xA;        mu = 1&#xA;        # FINITE DIFFERENCE&#xA;        t1 = time.time()&#xA;        x,u = Poisson_1D(mu,f3,N)&#xA;        t2 = time.time() - t1&#xA;        ue = u3(x)&#xA;        # FFT&#xA;        t3 = time.time()&#xA;        x_fft,u_fft = Poisson_fft_1D(mu,f3,N)&#xA;        t4 = time.time() - t3&#xA;        ue_fft = u3(x_fft)&#xA;        # ERRORS&#xA;        e1 = np.sqrt(np.sum((ue_fft-u_fft)**2))&#xA;        e2 = np.sqrt(np.sum((ue-u)**2))&#xA;        error_fft.append( e1 )&#xA;        error_fd.append( e2  )&#xA;        print(&quot;N : {2:10d}   ,   error fft : {0:5.3e}   ,   time fft : {3:5.3e}   ,   error df : {1:5.3e}   ,   time df : {4:5.3e}&quot;.format(e1,e2,N,t4,t2))&#xA;&#xA;plt.plot(np.log2(2**nlist),np.log2(error_fft),&quot;+-&quot;,label='FFT')&#xA;plt.plot(np.log2(2**nlist),np.log2(error_fd),&quot;x-&quot;,label='Finite difference')&#xA;plt.plot(np.log2(2**nlist),-np.log2(2**nlist),&quot;k--&quot;,label=r&quot;&lt;span class=&quot;math-container&quot;&gt;$y=-x$&lt;/span&gt;&quot;)&#xA;plt.plot(np.log2(2**nlist),-2*np.log2(2**nlist),&quot;b--&quot;,label=r&quot;&lt;span class=&quot;math-container&quot;&gt;$y=-2x$&lt;/span&gt;&quot;)&#xA;plt.legend()&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31573" LastEditorUserId="31573" LastEditDate="2019-04-24T14:43:42.607" LastActivityDate="2019-04-24T15:33:36.183" Title="FFT solver for the Poisson problem with Dirichlet boundary conditions" Tags="|python|fourier-analysis|poisson|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="32493" PostTypeId="2" ParentId="32491" CreationDate="2019-04-24T15:33:36.183" Score="0" Body="&lt;p&gt;No, it certainly is not correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would have identified the problem easily if you actually printed the solution and compared it with the exact one. The way how dst works means that the boundary points are &lt;em&gt;not&lt;/em&gt; included. Therefore while your exact solutions contrains the boundary 0 values, the numerical solution using FFT does not.&lt;/p&gt;&#xA;" OwnerUserId="1244" LastActivityDate="2019-04-24T15:33:36.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32495" PostTypeId="1" CreationDate="2019-04-24T19:35:46.697" Score="0" ViewCount="240" Body="&lt;p&gt;I was testing the &lt;a href=&quot;https://docs.scipy.org/doc/numpy/reference/routines.fft.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;.fft package&lt;/a&gt; of numpy 1.16.1 in Python 3.7.2. In particular I was trying to verify that the transform resembles the analytical one for: &lt;span class=&quot;math-container&quot;&gt;$$f(x) = \mathrm{exp}\left[-\left(\frac{x-5}{2}\right)^{2}\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get from &lt;a href=&quot;https://www.wolframalpha.com/input/?i=fourier%20transform%20of%20e%5E%7B-(x-2)%5E%7B2%7D%2F4%7D&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wolfram Alpha&lt;/a&gt; that &lt;span class=&quot;math-container&quot;&gt;$\hat{f} = \mathcal{F}[f]$&lt;/span&gt; looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xfLJU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xfLJU.png&quot; alt=&quot;FT by Wolfram&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I tried to replicate this plot with numpy and matplotlib, with the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;x = np.arange(0, 10, 1/1000)&#xA;y = np.exp(-((x-5)**2)/4)&#xA;&#xA;y_hat = np.fft.fftshift(np.fft.fft(y))&#xA;re_y_hat = np.real(y_hat)&#xA;im_y_hat = np.imag(y_hat)&#xA;&#xA;fig, ax = plt.subplots()&#xA;ax.plot(x, re_y_hat, &quot;b-&quot;, x, im_y_hat, &quot;r-&quot;)&#xA;plt.show()&#xA;plt.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the image I obtain differs greatly from the one Wolfram gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xh3LK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xh3LK.png&quot; alt=&quot;DFT in Python&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the last image the zero frequecy was shifted to the center by using &lt;code&gt;np.fft.fftshift()&lt;/code&gt; so the spike corresponds to frequency zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already figured out that the problem is that nowhere in &lt;code&gt;np.fft.fft()&lt;/code&gt; is the &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; being specified, so what numpy is interpreting is that I have data that varies very slowly, almost constant&lt;span class=&quot;math-container&quot;&gt;$^{1}$&lt;/span&gt;, and thus the transform is close to that of a constant function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked at the numpy documentation and other SE posts to see how this can be fixed but found nothing. &lt;strong&gt;Does anyone know how to fix this?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$^{1}$&lt;/span&gt; We can calculate the average slope of the function numpy &lt;em&gt;sees&lt;/em&gt; by &lt;span class=&quot;math-container&quot;&gt;$\frac{\mathrm{max}\{f\}-\mathrm{min}\{f\}}{x_{f_{\mathrm{max}}}-x_{f_{\mathrm{min}}}} = \frac{f(5)-f(0)}{n\Delta x} \approx \frac{1}{n\Delta x}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of nodes separating the maximum from the minumum. In this case, since numpy takes &lt;span class=&quot;math-container&quot;&gt;$\Delta x = 1$&lt;/span&gt; by default, the slope is about 1/5000=0.0002&lt;/p&gt;&#xA;" OwnerDisplayName="user30058" LastActivityDate="2023-07-18T10:05:55.743" Title="Specifying mesh spacing for DFT in numpy" Tags="|python|numpy|fourier-analysis|fourier-transform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32496" PostTypeId="1" CreationDate="2019-04-25T01:00:39.007" Score="3" ViewCount="115" Body="&lt;p&gt;The eigenvalue problem &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{d^2u}{dx^2}+2i k\frac{du}{dx}-[k^2-6\sin(x)^2]u(x)=-\mu u(x)$$&lt;/span&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;gives the first five eigenvalues with &lt;span class=&quot;math-container&quot;&gt;$k=0$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$k=1$&lt;/span&gt; which are &lt;span class=&quot;math-container&quot;&gt;$2.06$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$2.26$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$5.16$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$6.81$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$7.74$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And those eigenvalues I have calculated, but the corresponding eigenvector that I got did not agree with Fig.2 in the paper &lt;a href=&quot;https://doi.org/10.1103/PhysRevE.75.056602&quot; rel=&quot;nofollow noreferrer&quot;&gt;Z. Shi and J. Yang, &quot;Solitary waves bifurcated from Bloch-band edges in two-dimensional periodic media,&quot; &lt;em&gt;Phys. Rev. E&lt;/em&gt;, vol. 75, no. 5, pp. 056602, May 2007.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dR8Nz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dR8Nz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper the author tells us the vector at the edge is all real, but I calculate it to be complex. Why is there an inconsistency with the paper?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/55718422/calculate-the-bloch-wave&quot;&gt;This question on StackOverflow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PRE75.056602&#xA;&#xA;Clear[&quot;Global`*&quot;]&#xA;Lx = \[Pi];&#xA;k0x = 2 \[Pi]/Lx;&#xA;V = 6 Sin[x]^2;&#xA;kx = 0;&#xA;{eig0, funs} = NDEigensystem[{-D[u[x],{x,2}]-2I kx D[u[x],x]+(kx^2+V)u[x],&#xA;                              u[0]== u[\[Pi]]}, u[x], {x,0,\[Pi]}, 3,&#xA;                              Method-&amp;gt;{&quot;VectorNormalization&quot;-&amp;gt;True,&#xA;                              &quot;PDEDiscretization&quot;-&amp;gt;{&quot;FiniteElement&quot;,&#xA;                              {&quot;MeshOptions&quot;-&amp;gt;{&quot;MaxCellMeasure&quot;-&amp;gt;0.01}}}}]&#xA;&#xA;{2.06318, 6.81429, 7.74678},&#xA;&#xA;&#xA;kx=1;&#xA;{eig2,fun2} = NDEigensystem[{-D[u[x],{x,2}]-2I kx D[u[x],x]+(kx^2+V)u[x],&#xA;                            u[0]== u[\[Pi]]}, u[x], {x,0,\[Pi]}, 3,&#xA;                            Method-&amp;gt;{&quot;PDEDiscretization&quot;-&amp;gt;{&quot;FiniteElement&quot;,&#xA;                                     {&quot;MeshOptions&quot;-&amp;gt;{&quot;MaxCellMeasure&quot;-&amp;gt;0.01}}}}]&#xA;&#xA;{2.26673, 5.16594, 12.0926}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29453" LastEditorUserId="9667" LastEditDate="2019-04-27T19:03:29.570" LastActivityDate="2019-04-27T19:03:29.570" Title="Calculate the Bloch wave" Tags="|ode|eigensystem|mathematica|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32498" PostTypeId="2" ParentId="23222" CreationDate="2019-04-25T03:23:06.250" Score="2" Body="&lt;p&gt;You can use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bentley%E2%80%93Ottmann_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bentley–Ottmann algorithm&lt;/a&gt; for this. Given a set of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; line segments with &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; intersections, the algorithm can identify all intersections in &lt;span class=&quot;math-container&quot;&gt;$O((n+k)\log n)$&lt;/span&gt; time and &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; space. In cases where &lt;span class=&quot;math-container&quot;&gt;$k=o(\frac{n^2}{\log n})$&lt;/span&gt; (that is, cases in which &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; has an appropriate upper bound) this offers time savings versus a naive &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt; algorithm that simply compares all segments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, this problem can be approached using any of a number of &lt;a href=&quot;https://en.wikipedia.org/wiki/Sweep_line_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;sweep line algorithms&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trick, then, is to increment a segment's value in a scoring hash table each time it is involved in an intersection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A couple of hours of fiddling with CGAL didn't reveal an obvious way to do this and other implementations produced incorrect answers due to floating-point issues at the lines' end points. Nonetheless, this is the most computationally-efficient way to approach the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've copied my stab at building a CGAL implementation below with the appropriate spot to edit the code noted:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;//Compile with: g++ -g 23222-line-with-most-intersections.cpp -lCGAL -lgmp -lmpfr&#xA;&#xA;#include &amp;lt;CGAL/Exact_predicates_exact_constructions_kernel.h&amp;gt;&#xA;#include &amp;lt;CGAL/Arr_segment_traits_2.h&amp;gt;&#xA;#include &amp;lt;CGAL/Surface_sweep_2.h&amp;gt;&#xA;#include &amp;lt;CGAL/Surface_sweep_2_algorithms.h&amp;gt;&#xA;//#include &amp;lt;CGAL/Sweep_line_2.h&amp;gt;&#xA;#include &amp;lt;CGAL/Surface_sweep_2/Default_visitor.h&amp;gt;&#xA;#include &amp;lt;CGAL/Surface_sweep_2/Surface_sweep_2_utils.h&amp;gt;&#xA;#include &amp;lt;list&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;&#xA;typedef CGAL::Exact_predicates_exact_constructions_kernel       Kernel;&#xA;typedef Kernel::Point_2                                         Point_2;&#xA;typedef CGAL::Arr_segment_traits_2&amp;lt;Kernel&amp;gt;                      Traits_2;&#xA;typedef Traits_2::Curve_2                                       Segment_2;&#xA;&#xA;&#xA;namespace CGAL {&#xA;namespace Surface_sweep_2 {&#xA;&#xA;template &amp;lt;typename GeometryTraits_2, typename OutputIterator,&#xA;          typename Allocator_ = CGAL_ALLOCATOR(int)&amp;gt;&#xA;class IntersectionCounter :&#xA;  public Default_visitor&amp;lt;IntersectionCounter&amp;lt;GeometryTraits_2,&#xA;                                                      OutputIterator,&#xA;                                                     Allocator_&amp;gt;,&#xA;                         GeometryTraits_2, Allocator_&amp;gt;&#xA;{&#xA;public:&#xA;  typedef GeometryTraits_2                              Geometry_traits_2;&#xA;  typedef OutputIterator                                Output_iterator;&#xA;  typedef Allocator_                                    Allocator;&#xA;&#xA;private:&#xA;  typedef Geometry_traits_2                             Gt2;&#xA;  typedef IntersectionCounter&amp;lt;Gt2, Output_iterator, Allocator&amp;gt;&#xA;                                                        Self;&#xA;  typedef Default_visitor&amp;lt;Self, Gt2, Allocator&amp;gt;         Base;&#xA;&#xA;public:&#xA;  typedef typename Base::Event                          Event;&#xA;  typedef typename Base::Subcurve                       Subcurve;&#xA;&#xA;  typedef typename Subcurve::Status_line_iterator       Status_line_iterator;&#xA;&#xA;  typedef typename Gt2::X_monotone_curve_2              X_monotone_curve_2;&#xA;  typedef typename Gt2::Point_2                         Point_2;&#xA;&#xA;  typedef typename Base::Surface_sweep_2                Surface_sweep_2;&#xA;&#xA;protected:&#xA;  Output_iterator m_out;                 // The output points.&#xA;&#xA;public:&#xA;  IntersectionCounter(Output_iterator out) :&#xA;    m_out(out)&#xA;  {}&#xA;&#xA;  template &amp;lt;typename CurveIterator&amp;gt;&#xA;  void sweep(CurveIterator begin, CurveIterator end)&#xA;  {&#xA;    std::vector&amp;lt;X_monotone_curve_2&amp;gt; curves_vec;&#xA;    std::vector&amp;lt;Point_2&amp;gt; points_vec;&#xA;&#xA;    curves_vec.reserve(std::distance(begin,end));&#xA;    make_x_monotone(begin, end,&#xA;                    std::back_inserter(curves_vec),&#xA;                    std::back_inserter(points_vec),&#xA;                    this-&amp;gt;traits());&#xA;&#xA;    //Original curves get converted into x-monotone curves here, but, since they&#xA;    //are segments, their ordering and data appears to be unaltered&#xA;    std::cout&amp;lt;&amp;lt;&quot;x-monotone curves\n&quot;;&#xA;    for(auto &amp;amp;x: curves_vec)&#xA;      std::cout&amp;lt;&amp;lt;x&amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt;(&amp;amp;x)&amp;lt;&amp;lt;std::endl;&#xA;    std::cout&amp;lt;&amp;lt;&quot;x-monotone points\n&quot;;&#xA;    for(auto &amp;amp;x: points_vec)&#xA;      std::cout&amp;lt;&amp;lt;x&amp;lt;&amp;lt;std::endl;&#xA;&#xA;    //Perform the sweep&#xA;    Surface_sweep_2* sl = this-&amp;gt;surface_sweep();&#xA;    sl-&amp;gt;sweep(curves_vec.begin(), curves_vec.end(),&#xA;              points_vec.begin(), points_vec.end());&#xA;  }&#xA;&#xA;  bool after_handle_event(Event* event,&#xA;                          Status_line_iterator /* iter */,&#xA;                          bool /* flag */)&#xA;  {&#xA;    //TODO: Magic should happen here&#xA;    if ((&#xA;         event-&amp;gt;is_intersection() ||&#xA;         event-&amp;gt;is_weak_intersection()) &amp;amp;&amp;amp; event-&amp;gt;is_closed())&#xA;    {&#xA;      *m_out = event-&amp;gt;point();&#xA;      ++m_out;&#xA;    }&#xA;    return true;&#xA;  }&#xA;&#xA;  Output_iterator output_iterator() { return m_out; }&#xA;};&#xA;&#xA;} // namespace Surface_sweep_2&#xA;&#xA;namespace Ss2 = Surface_sweep_2;&#xA;&#xA;&#xA;&#xA;template &amp;lt;typename CurveInputIterator, typename OutputIterator, typename Traits&amp;gt;&#xA;OutputIterator CountIntersections(&#xA;  CurveInputIterator curves_begin,&#xA;  CurveInputIterator curves_end,&#xA;  OutputIterator points,&#xA;  Traits &amp;amp;tr&#xA;){&#xA;  // Define the surface-sweep types:&#xA;  typedef Ss2::IntersectionCounter&amp;lt;Traits, OutputIterator&amp;gt; Visitor;&#xA;  typedef Ss2::Surface_sweep_2&amp;lt;Visitor&amp;gt;                    Surface_sweep;&#xA;&#xA;  // Perform the sweep and obtain the intersection points.&#xA;  Visitor visitor(points);&#xA;  Surface_sweep surface_sweep(&amp;amp;tr, &amp;amp;visitor);&#xA;  visitor.sweep(curves_begin, curves_end);&#xA;&#xA;  return visitor.output_iterator();&#xA;}&#xA;&#xA;&#xA;&#xA;template &amp;lt;typename CurveInputIterator, typename OutputIterator&amp;gt;&#xA;OutputIterator CountIntersections(&#xA;  CurveInputIterator curves_begin,&#xA;  CurveInputIterator curves_end,&#xA;  OutputIterator points&#xA;){&#xA;  typedef typename std::iterator_traits&amp;lt;CurveInputIterator&amp;gt;::value_type  Curve;&#xA;  typename Default_arr_traits&amp;lt;Curve&amp;gt;::Traits   traits;&#xA;  return CountIntersections(curves_begin, curves_end, points, traits);&#xA;}&#xA;&#xA;&#xA;} // namespace CGAL&#xA;&#xA;&#xA;&#xA;int main(){&#xA;  //Points as extracted from https://scicomp.stackexchange.com/q/23222/17088&#xA;  const std::vector&amp;lt;Point_2&amp;gt; pts = {&#xA;    Point_2( 57,931),&#xA;    Point_2(447,699),&#xA;    Point_2(899,748),&#xA;    Point_2(863,137),&#xA;    Point_2(530, 67),&#xA;    Point_2(142,282)&#xA;  };&#xA;&#xA;  //Points are fully connected&#xA;  std::vector&amp;lt;Segment_2&amp;gt; segments;&#xA;  for(int i=0;  i&amp;lt;pts.size();i++)&#xA;  for(int j=i+1;j&amp;lt;pts.size();j++){&#xA;    segments.emplace_back(pts[i],pts[j]);&#xA;    std::cout&amp;lt;&amp;lt;pts[i]&amp;lt;&amp;lt;&quot;\n&quot;&amp;lt;&amp;lt;pts[j]&amp;lt;&amp;lt;&quot;\n\n&quot;;&#xA;  }&#xA;&#xA;  // Compute all intersection points.&#xA;  std::list&amp;lt;Point_2&amp;gt; ipts;&#xA;  CGAL::CountIntersections(segments.begin(), segments.end(), std::back_inserter(ipts));&#xA;&#xA;  for(const auto &amp;amp;x: segments)&#xA;    std::cout&amp;lt;&amp;lt;(&amp;amp;x)&amp;lt;&amp;lt;std::endl;&#xA;&#xA;  // Print the result.&#xA;  std::cout &amp;lt;&amp;lt; &quot;Found &quot; &amp;lt;&amp;lt; ipts.size() &amp;lt;&amp;lt; &quot; intersection points: &quot; &amp;lt;&amp;lt; std::endl;&#xA;  std::copy(ipts.begin(), ipts.end(),&#xA;            std::ostream_iterator&amp;lt;Point_2&amp;gt;(std::cout, &quot;\n&quot;));&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2019-04-25T03:23:06.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32499" PostTypeId="1" CreationDate="2019-04-25T07:56:14.700" Score="0" ViewCount="326" Body="&lt;p&gt;If each edge of a graph &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is unweighted or has equal weights, then the shortest path between two nodes in that graph is the path that contains the fewest number of edges. Such a path can be obtained by BFS. Here, i consider that each weight of the edge is the minimum of the end vertices and the weight of the path is the sum of the edges weights divided by the number of edges on the path. I am unable to use BFS for such algoritm. Please suggest me a suitable known algorithm to solve such problem. I would also prefer to know if i can use suitable programming codes in C++ to solve such problem? The following problem is solved manually, but i wish to solve it by using some known algorithm or, programming codes. I am new to C programming as well as to this site, so if my question off-topic, then please refer me an appropriate site. Thank you,  &lt;a href=&quot;https://i.stack.imgur.com/WBZN6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WBZN6.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note 1&lt;/strong&gt; The above problem is solved just manually. Instead, i want to use suitable known algorithm or programming codes in C++ to solve this problem. I have just installed Dev-C++ to try.&#xA;&lt;strong&gt;Note 2&lt;/strong&gt; Here, the graph &lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; is complete (except the edge &lt;span class=&quot;math-container&quot;&gt;$(0, 6)$&lt;/span&gt; being removed) having &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; as the source vertex and &lt;span class=&quot;math-container&quot;&gt;$6$&lt;/span&gt; as the target vertex.&lt;/p&gt;&#xA;" OwnerUserId="31584" LastEditorUserId="31586" LastEditDate="2019-04-26T14:58:56.827" LastActivityDate="2019-05-27T19:02:10.933" Title="Finding a shortest path in a graph" Tags="|algorithms|c|graph-theory|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="32500" PostTypeId="1" CreationDate="2019-04-25T08:08:29.677" Score="2" ViewCount="225" Body="&lt;p&gt;Trying to implement an adaptive step size Runge-Kutta Cash-Karp but failing with this error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;home/anaconda/lib/python3.6/site-packages/ipykernel_launcher.py:15: RuntimeWarning: divide by zero encountered in double_scalars from ipykernel import kernelapp as app&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The ODE i am trying to solve (and using in the example below, transformed from higher order to system of 1st order ODEs) is the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$u''(x)+u(x)+u(x)^3 =0 ~~{\rm with}~~ u(0)=u'(0)=1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the implementation I am using: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def rkck(f, x, y, h, tol):&#xA;    #xn = x + h&#xA;    err = 2 * tol&#xA;    while (err &amp;gt; tol):&#xA;        xn = x + h&#xA;        k1 = h*f(x,y)&#xA;        k2 = h*f(x+(1/5)*h,y+((1/5)*k1)) &#xA;        k3 = h*f(x+(3/10)*h,y+((3/40)*k1)+((9/40)*k2))&#xA;        k4 = h*f(x+(3/5)*h,y+((3/10)*k1)-((9/10)*k2)+((6/5)*k3))&#xA;        k5 = h*f(x+(1/1)*h,y-((11/54)*k1)+((5/2)*k2)-((70/27)*k3)+((35/27)*k4))&#xA;        k6 = h*f(x+(7/8)*h,y+((1631/55296)*k1)+((175/512)*k2)+((575/13824)*k3)+((44275/110592)*k4)+((253/4096)*k5))&#xA;        yn4 = y + ((37/378)*k1)+((250/621)*k3)+((125/594)*k4)+((512/1771)*k6)&#xA;        yn5 = y + ((2825/27648)*k1)+((18575/48384)*k3)+((13525/55296)*k4)+((277/14336)*k5)+((1/4)*k6)&#xA;        err = yn4[-1]-yn5[-1]&#xA;        h = 0.8 * h * (tol/err)**(1/float(5))&#xA;        yn = yn4&#xA;    return xn, yn&#xA;&#xA;def integrate_sStepControl(f, t0, y0, tend, h, tol):&#xA;    T = [t0]&#xA;    Y = [y0]&#xA;    t = t0&#xA;    y = y0 &#xA;    while (t &amp;lt; tend):&#xA;        h = min(h, tend-t)&#xA;        t, y = rkck(f, t, y, h, tol)&#xA;        T.append(t)&#xA;        Y.append(y)&#xA;    return np.array(T), np.array(Y)&#xA;&#xA;def f_1(t,y): &#xA;    return np.array([ y[1], -y[0]-(y[0])**3 ])&#xA;&#xA;Y0_f1 = np.array([1.0,1.0])&#xA;&#xA;&#xA;# Execution&#xA;h = 0.05&#xA;tv, yv = integrate_sStepControl(f=f_1, t0=0.0, y0=Y0_f1, tend=100.0, h=h, tol=1.0E-05)&#xA;print(&quot;[ %20.15f, %20.15f]&quot;%(yv[-1,0], yv[-1,1]) )&#xA;plt.plot(tv, yv)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Getting the error above, but it gets plotted. Don't know what I am doing wrong here :-/ &#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZEL70.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZEL70.png&quot; alt=&quot;plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31585" LastActivityDate="2019-04-25T08:08:29.677" Title="How to implement adaptive step size Runge-Kutta Cash-Karp?" Tags="|ode|runge-kutta|numerical-modelling|numerics|differential-equations|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="32501" PostTypeId="1" CreationDate="2019-04-25T08:34:35.593" Score="1" ViewCount="147" Body="&lt;p&gt;I am working with the following differential equation: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The equation is &lt;span class=&quot;math-container&quot;&gt;$$x=\beta +\sqrt{2D} \xi(t)$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\xi(t)$&lt;/span&gt; is a white noise term, with a reflecting wall boundary conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After solving the equation numerically with Euler-Maruyama method, I used the Wiener-Khinchin theorem  (Using FFT) to find a numerical solution for the correlation function. &#xA;My problem is, get every time a new starting point for the correlation function even for exactly the same parameters, no matter how I change the parameters, and I am sure of my code because I have tried it on Orstein-Uhlenbeck and the results were right. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the problem of the varying starting points, or what did I misunderstand about correlation functions that is leading to this  ?? (Please see the code below, and the plots of the correlation function for the same given parameters)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HG6Yq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HG6Yq.png&quot; alt=&quot;Correlation function of the SDE&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Ur5Vc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ur5Vc.png&quot; alt=&quot;Another correlation function for the same SDE with exactly the same parameteres&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Aj1nM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Aj1nM.png&quot; alt=&quot;another example of the same simulation of the correlation function&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code for simulating the SDE with Euler method and simulating the correlation function &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; N = 8192&#xA;dt = 0.3&#xA;T = N*dt&#xA;time = np.arange(N)*dt&#xA;gamma = 0.1&#xA;D=1&#xA;v_factor = math.sqrt(2*D*dt)&#xA;v_data = []&#xA;&#xA;v=1&#xA;&#xA;###Eurler method to solve the SDE&#xA;for t in range(50000):  ##This part just to make sure we get the stationary process&#xA;        F = random.gauss(0,1)&#xA;        v = v - gamma*dt + v_factor*F&#xA;        if v&amp;lt;0:&#xA;            v=-v&#xA;for t in range(N):&#xA;        F = random.gauss(0,1)&#xA;        v = v - gamma*dt + v_factor*F&#xA;        if v&amp;lt;0:&#xA;            v=-v&#xA;        v_data.append(v)&#xA;# Compute numerically the autocorrelation via a Fourier transform&#xA;&#xA;fft_cor = scipy.signal.fftconvolve(v_data, v_data[::-1])[N-1:]&#xA;fft_cor /= (N - np.arange(N))&#xA;plt.plot(time, fft_cor, 'k-', lw=2,label=&quot;Fourier-Transform&quot;)&#xA;&#xA;&#xA;# Compute the autocorrelation via the Wiener-Khinchin theorem&#xA;&#xA;psd = np.fft.fft(v_data)*np.conj(np.fft.fft(v_data))/N&#xA;C = np.fft.ifft(psd).real&#xA;&#xA;plt.plot(time, C,label='Wiener-Khinchin theorem')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="26769" LastActivityDate="2019-04-25T08:34:35.593" Title="Why does the correlation function of this stochastic differential equation starts at different points?" Tags="|python|computational-physics|numerics|statistics|differential-equations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32504" PostTypeId="1" CreationDate="2019-04-25T15:21:46.447" Score="0" ViewCount="102" Body="&lt;p&gt;I am working on conductive polymer modeling and supposed to do one-dimensional diffusion model in the thickness of the polymer, however, due to the small value of thickness in micro, when I use the forward approach it leads to instability, no matter how I decrease the number of nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to use Backward Euler to counter this issue, however, due to the nature of our system, we can only know the initial boundary condition and not the final value, how I can solve this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question: Is there any alternative techniques I can use in numerical analysis while the final boundary condition is unknown?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The mathematical modeling:&lt;/strong&gt;&#xA;For modelling the system, I am using RC transmission line, the mathematical description here: &lt;a href=&quot;http://www.swarthmore.edu/NatSci/echeeve1/Ref/trans/Infinite.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.swarthmore.edu/NatSci/echeeve1/Ref/trans/Infinite.html&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/oQwCX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oQwCX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30458" LastEditorUserId="30458" LastEditDate="2019-04-28T21:15:26.120" LastActivityDate="2019-04-28T21:15:26.120" Title="The final Boundary Condition is Unknown, Is Backward Euler is still valid to be implemented?" Tags="|numerical-modelling|numerics|stability|" AnswerCount="0" CommentCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="32507" PostTypeId="1" CreationDate="2019-04-26T03:36:06.940" Score="2" ViewCount="30" Body="&lt;p&gt;I am currently trying to segment cells from digital pathology images.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method I use is an algorithm based on color distance. This works for most of the cases, however, when dealing with the images which cells are overstaining, then the segmentation results is pretty bad. Is there any golden-standard method in treating these poorly stained with highly adhered-cell images?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HPbHE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HPbHE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31597" LastActivityDate="2019-04-26T03:36:06.940" Title="Cell-segmentation from overstaining images Ask" Tags="|matlab|image-processing|" AnswerCount="0" CommentCount="5" ClosedDate="2019-06-05T21:33:26.170" ContentLicense="CC BY-SA 4.0" />
  <row Id="32509" PostTypeId="2" ParentId="11249" CreationDate="2019-04-26T12:33:45.040" Score="4" Body="&lt;p&gt;Assuming you are trying to differentiate a numerical implementation of a continuous function, there are a large number of methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Automatic differentiation. The most accurate and general method. Painful to code, requiring operator overloading and argument dependent lookup. Puts a burden on the user to understand these concepts. Also struggles with removable singularities, such as differentiating sinc at &lt;span class=&quot;math-container&quot;&gt;$x= 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) A Chebyshev transform. Project your function onto a span of Chebyshev polynomials and differentiate the three term recurrence. Super fast, very accurate. But requires that you have a compactly supported domain of interest; outside the selected domain &lt;span class=&quot;math-container&quot;&gt;$[a,b]$&lt;/span&gt;, the three term recurrence is unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Finite differencing. Underrated in 1D; see Nick Higham's &lt;a href=&quot;https://youtu.be/Q9OLOqEhc64&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tips and Tricks in Numerical Computing&lt;/a&gt;. The idea is that if you balance the truncation error and the roundoff error then you don't need to select a stepsize; it can be chosen automatically. In Boost, this idea is used to recover (by default) 6/7th of the correct digits for the type. (Higham only shows the idea for the simpler case of 1/2 the correct digits, but the idea is easily extended.) The coefficients are from Fornberg's equispaced table, but the stepsize is chosen under the assumption that the function can be evaluated to 1ULP accuracy. The disadvantage is that it requires 2 function evaluations to recover half the digits of the type, 4 to recover 3/4th the digits, so on. In 1D, not a bad deal. In higher dimensions, it's catastrophic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) The complex step derivative. Use &lt;span class=&quot;math-container&quot;&gt;$f'(x) \approx \Im(f(x+ih))$&lt;/span&gt;. Take &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; to be the unit roundoff and this will recover almost every bit correct. However, it's kinda cheating, because it's generally harder to implement a function in the complex plane than to hand code its real derivative. Still a cool idea and useful in certain circumstances.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastActivityDate="2019-04-26T12:33:45.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32511" PostTypeId="1" AcceptedAnswerId="32516" CreationDate="2019-04-26T21:50:49.573" Score="2" ViewCount="112" Body="&lt;p&gt;Let us assume that we have a function, &lt;span class=&quot;math-container&quot;&gt;$f(A)=\text{vec}(A^{-1})^\intercal B$&lt;/span&gt;, dependent on &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt;. However, due to some machine-precision limitations, the programming language I'm using cannot invert &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; even if it has &lt;span class=&quot;math-container&quot;&gt;$Det(A)&amp;gt;0$&lt;/span&gt;. So, I'm thinking of using a pseudo-inverse, instead. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't know if there's a 'distance' relationship between the true inverse and the pseudo-inverse. I just know that when there's an inverse, both are equal. But it would be nice, that when the original matrix A and the 'perturbed/truncated' version of A are close, that the pseudo-inverse be close to the inverse of the original... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm also looking at pseudo-inverses for which there's a relatively easy and efficient way to compute.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: &lt;span class=&quot;math-container&quot;&gt;$\dim(A)=DT\times DT$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\dim(B)=D^2T^2\times D^2$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following Federico Poloni's comment, I think I could actually recompute the function above as &lt;span class=&quot;math-container&quot;&gt;$$\left[\cdots Tr\left(\text{LinearSolve}\left(A^\intercal, C_i \right) \right) \cdots \right]_{1\times i=1,...,D^2}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$C_i=\text{Reshape}_{DT\times DT}\left(\text{Column}(B,i)\right)$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="24970" LastEditorUserId="24970" LastEditDate="2019-04-27T20:40:52.567" LastActivityDate="2019-04-27T20:40:52.567" Title="Which pseudo-inverse to compute when Inverse is not possible? (No linear solve)" Tags="|linear-algebra|inverse|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="32512" PostTypeId="1" CreationDate="2019-04-26T23:24:10.200" Score="-1" ViewCount="99" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MPo9H.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MPo9H.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;This is probably a long shot with such a short time, but I've been trying to get theoretical data for a project I'm working on. The project involves using a very simplified version of magnetic particle imaging to analyze fluids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I've written some code with some help, and I'm pretty sure all the physics is correct but when I run it with time steps (deltaT) of 5E-5 the code calculates velocities of the order of thousands when it should be in millimeters per second. But when the code has time steps of 5E-6 it runs correctly. The problem is the time it takes to complete these simulations at this time step is huge and in my actual experiments, some of these took over 20 minutes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I'm wondering if it's possible to either significantly decrease the run time at this time step or be able to increase the time step of the simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import matplotlib.animation as animation&#xA;&#xA;&#xA;#Arrays for storing time&#xA;#t1 = [0]&#xA;#t2 = [0]&#xA;#t3 = [0]&#xA;#t4 = [0]&#xA;#t5 = [0]&#xA;t6 = [0]&#xA;&#xA;#Time Increment&#xA;deltaT = 5E-7&#xA;#Number of Particles&#xA;N = 100&#xA;&#xA;#Viscosity&#xA;mu_w = 0.0010049&#xA;mu = 3*mu_w&#xA;&#xA;#Geometry&#xA;radius = 0.0095&#xA;d = 0.023&#xA;L = 0.02&#xA;&#xA;#Constants&#xA;mu_0 = 1.25663706E-6&#xA;&#xA;#Particle Properties&#xA;p_rad = (4.9E-6)/2&#xA;m = 1.663223552E-13&#xA;moment = 60*m&#xA;&#xA;x, y = np.mgrid[-1.15:1.15:151j, -1:1:151j]*1E-2&#xA;&#xA;#Method for Magnetic Field&#xA;def B(x,y):&#xA;    H11 = 4/((L+2*y)*np.sqrt((4*(d-2*x)**2/(L+2*y)**2+4)))&#xA;&#xA;    H12 = 4/((L+2*y)*np.sqrt((4*(d+2*x)**2/(L+2*y)**2+4)))&#xA;&#xA;    H13 = 4/((L-2*y)*np.sqrt((4*(d+2*x)**2/(L-2*y)**2+4)))&#xA;&#xA;    H14 = 4/((L-2*y)*np.sqrt((4*(d-2*x)**2/(L-2*y)**2+4)))&#xA;&#xA;    H21 = 1/((d-2*x)**2*(np.sqrt(4*(d-2*x)**2/(L-2*y)**2+4)))&#xA;&#xA;    H22 = 1/((d-2*x)**2*(np.sqrt(4*(d-2*x)**2/(L+2*y)**2+4)))&#xA;&#xA;    H23 = 1/((d+2*x)**2*(np.sqrt(4*(d+2*x)**2/(L-2*y)**2+4)))&#xA;&#xA;    H24 = 1/((d+2*x)**2*(np.sqrt(4*(d+2*x)**2/(L+2*y)**2+4)))&#xA;&#xA;    H1 = (H11 - H12 - H13 + H14)&#xA;    H2 = (-4*(d-2*x))*(H21 - H22) + (-4*(d+2*x))*(H23 - H24)&#xA;&#xA;    B1 = (H1*mu_0)*2666&#xA;    B2 = (H2*mu_0)*1700&#xA;    B = np.sqrt(B1**2+B2**2)&#xA;    return np.dstack((B1,B2,B))&#xA;&#xA;&#xA;B1 = B(x,y)[:,:,0]&#xA;B2 = B(x,y)[:,:,1]&#xA;Bt = B(x,y)[:,:,2]&#xA;&#xA;#Contour Plot&#xA;levels = np.logspace(-2.5,1.23,10,base=10)&#xA;cp = plt.contourf(x, y, Bt, levels=levels, cmap=plt.cm.plasma,extend='max')&#xA;cb = plt.colorbar(cp)&#xA;plt.clim(0,2)&#xA;&#xA;#X and Y increments&#xA;dxy = [0.023/151,0.02/151]&#xA;&#xA;#Gradients of B, Bx &amp;amp; By&#xA;dB = np.gradient(Bt,dxy[0],dxy[1])&#xA;dBy = np.gradient(B1,dxy[1])[0]&#xA;dBx = np.gradient(B2,dxy[0])[1]&#xA;&#xA;#Vector Field Plot&#xA;x, y = np.mgrid[-1.15:1.15:20j, -1:1:20j]*1E-2&#xA;Bq = B(x,y)[:,:,2]&#xA;dxy = [0.023/20, 0.02/20]&#xA;dBq = np.gradient(Bq,dxy[0],dxy[1])&#xA;&#xA;quiv = plt.quiver(x, y, dBq[0], dBq[1])&#xA;plt.show()&#xA;&#xA;#X Gradient Plot&#xA;plt.figure()&#xA;x = np.linspace(-0.0115,0.0115,151)&#xA;plt.plot(x,B2[75,:])&#xA;plt.title('Magnetic Field vs. Position in x-direction')&#xA;plt.xlabel('Position (m)')&#xA;plt.ylabel('Magnetic Field (T)')&#xA;plt.show()&#xA;&#xA;#Y Gradient Plot&#xA;plt.figure()&#xA;y = np.linspace(-0.01,0.01,151)&#xA;plt.plot(y,-B1[:,75])&#xA;plt.title('Magnetic Field vs. Position in y-direction')&#xA;plt.xlabel('Position (m)')&#xA;plt.ylabel('Magnetic Field (T)')&#xA;plt.show()&#xA;&#xA;#-----------------------------------------&#xA;&#xA;yslice = 0.00375&#xA;&#xA;xp = np.random.uniform(-radius,radius,N)&#xA;yp = np.random.uniform(-radius,radius,N)&#xA;&#xA;v_x = np.zeros(N)&#xA;v_y = np.zeros(N)&#xA;&#xA;#Arrays to store counts&#xA;#count1 = [0]&#xA;#count2 = [0]&#xA;#count3 = [0]&#xA;#count4 = [0]&#xA;#count5 = [0]&#xA;count6 = [0]&#xA;&#xA;for i in range(0,N):&#xA;    while xp[i]**2 + yp[i]**2 &amp;gt; radius**2:&#xA;        xp[i] = np.random.uniform(-radius,radius,1)&#xA;        yp[i] = np.random.uniform(-radius,radius,1)&#xA;&#xA;&#xA;for i in range(0,N):&#xA;    if -yslice &amp;lt; yp[i] &amp;lt; yslice:&#xA;        count6[0] = count6[0] + 1&#xA;&#xA;fig1 = plt.figure()&#xA;plt.plot(xp,yp,&quot;.&quot;)&#xA;&#xA;Fmag_x = np.zeros(N)&#xA;Fmag_y = np.zeros(N)&#xA;Fdrag_x = np.zeros(N)&#xA;Fdrag_y = np.zeros(N)&#xA;deltaV_x = np.zeros(N)&#xA;deltaV_y = np.zeros(N)&#xA;&#xA;timestep = 1&#xA;&#xA;#Method to animate simulation&#xA;def update(j,plt,fig1):&#xA;    global xp, yp, v_x, v_y, t, timestep, count&#xA;    plt.cla()&#xA;    if min(xp**2 + yp**2) &amp;lt; radius**2:&#xA;        count6.append(0)&#xA;        t6.append(t6[timestep-1] + deltaT)&#xA;        print(t6[timestep])&#xA;        #print('fmag: ' + str(Fmag_x[0]))&#xA;        #print('fdrag: ' + str(Fdrag_x[0]))&#xA;        print('v(x): ' + str(v_x[0]))&#xA;        for i in range(0,N):&#xA;            if xp[i]**2 + yp[i]**2 &amp;lt; radius**2:&#xA;&#xA;                #Interpolation Method&#xA;                Ix = int((xp[i]-(-0.0115))/(0.0115-(-0.0115))*150)&#xA;                #print(Ix)&#xA;                Iy = int((yp[i]-(-0.01))/(0.01-(-0.01))*150)&#xA;                #print(Iy)&#xA;                dBx1 = dBx[Ix,Iy]&#xA;                dBx2 = dBx[Ix,Iy+1]&#xA;                dBx3 = dBx[Ix+1,Iy]&#xA;                dBx4 = dBx[Ix+1,Iy+1]&#xA;                dBy1 = dBy[Ix,Iy]&#xA;                dBy2 = dBy[Ix,Iy+1]&#xA;                dBy3 = dBy[Ix+1,Iy]&#xA;                dBy4 = dBy[Ix+1,Iy+1]&#xA;                T = (xp[i]-x[Ix])/(x[Ix+1]-x[Ix])&#xA;                #print(T)&#xA;                U = (yp[i]-y[Iy])/(y[Iy+1]-y[Iy])&#xA;                #print(U)&#xA;                #Interpolated Gradients&#xA;                dBxx = (1-T)*(1-U)*dBx1+(1-T)*U*dBx2+T*(1-U)*dBx3+T*U*dBx4&#xA;                #print('dBx ' +str(dBxx))&#xA;                dByy = (1-T)*(1-U)*dBy1+(1-T)*U*dBy2+T*(1-U)*dBy3+T*U*dBy4&#xA;                #print('dBy ' +str(dByy))&#xA;                #Magnetic Force in X and Y&#xA;                Fmag_x[i] = moment*dBxx*(B(xp[i],yp[i])[:,:,0]/np.abs(B(xp[i],yp[i])[:,:,2]))&#xA;                Fmag_y[i] = moment*dByy*(B(xp[i],yp[i])[:,:,1]/np.abs(B(xp[i],yp[i])[:,:,2]))&#xA;&#xA;                #Drag Force in X and Y&#xA;                Fdrag_x[i] = 6*np.pi*mu*p_rad*v_x[i]&#xA;                Fdrag_y[i] = 6*np.pi*mu*p_rad*v_y[i]&#xA;&#xA;                #Increase in velocity&#xA;                deltaV_x[i] = (Fmag_x[i] - Fdrag_x[i])*deltaT/m&#xA;                deltaV_y[i] = (Fmag_y[i] - Fdrag_y[i])*deltaT/m&#xA;&#xA;                #Velocity&#xA;                v_x[i] = v_x[i] + deltaV_x[i]&#xA;                v_y[i] = v_y[i] + deltaV_y[i]&#xA;                #v_x = moment*30/(6*np.pi*mu*p_rad)*xp[i]/np.abs(xp[i])&#xA;                #v_y = moment*40/(6*np.pi*mu*p_rad)*yp[i]/np.abs(yp[i])&#xA;&#xA;                #New X and Y coordinates of particles&#xA;                xp[i] = xp[i] + (v_x[i]*deltaT)&#xA;                yp[i] = yp[i] + (v_y[i]*deltaT)&#xA;&#xA;            #Counter&#xA;            if -yslice &amp;lt; yp[i] &amp;lt; yslice:&#xA;                count6[timestep] = count6[timestep] + 1&#xA;        timestep = timestep + 1&#xA;    plt.plot(xp,yp,&quot;.&quot;)&#xA;    plt.xlim(-radius,radius)&#xA;    plt.ylim(-radius,radius)&#xA;&#xA;ani = animation.FuncAnimation(fig1, update, frames=1, fargs=(plt, fig1), interval=0.1)&#xA;&#xA;#Distribution of particles&#xA;#theta = np.arctan2(yp,xp)&#xA;#plt.hist(theta,50)&#xA;#plt.title('Distribution of Particles Generated')&#xA;#plt.xlabel('Theta (Rads)')&#xA;#plt.ylabel('Number of Particles')&#xA;#plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31605" LastEditorUserId="31605" LastEditDate="2019-04-28T19:33:11.500" LastActivityDate="2019-05-28T20:01:20.170" Title="Simulating magnetic particles in a field free point generated by two opposing magnets" Tags="|python|fluid-dynamics|computational-physics|electromagnetism|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="32513" PostTypeId="1" CreationDate="2019-04-27T03:18:31.787" Score="6" ViewCount="142" Body="&lt;p&gt;Is there either a closed-form expression or fast/elegant algorithm for computing the positive root of the polynomial &#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(x)=x^q + \beta x - \beta,$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\beta&amp;gt;0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$q\geq2$&lt;/span&gt;?  How about the &lt;span class=&quot;math-container&quot;&gt;$q\in\mathbb R$&lt;/span&gt; case with &lt;span class=&quot;math-container&quot;&gt;$q\geq1$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note there is exactly one positive root of the function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;, since &lt;span class=&quot;math-container&quot;&gt;$f(0)=-\beta&amp;lt;0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\lim_{x\to\infty} f(x)=\infty$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; is a convex function on &lt;span class=&quot;math-container&quot;&gt;$\mathbb R_+$&lt;/span&gt; given our bound on &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bracketing/bisection will give an estimate in linear time, and by the argument in &lt;a href=&quot;https://math.stackexchange.com/questions/615923/global-convergence-for-newtons-method-in-one-dimension&quot;&gt;this response&lt;/a&gt; I guess Newton's method will have have global quadratic convergence specifically for this function so long as &lt;span class=&quot;math-container&quot;&gt;$q\geq2$&lt;/span&gt;.  Just wanted to make sure I'm not missing a more slick approach!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll be embarrassed if there's an obvious closed-form expression that I missed :-)&lt;/p&gt;&#xA;" OwnerUserId="5129" LastActivityDate="2019-04-30T06:51:41.920" Title="Positive root of $x^q + bx - b$" Tags="|newton-method|polynomials|rootfinding|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32514" PostTypeId="1" CreationDate="2019-04-27T06:06:59.570" Score="1" ViewCount="48" Body="&lt;p&gt;I have a function which requires the return type to be a container. The problem is that I need to integrate the contents of the container as efficiently as possible and was hoping to use adaptive Gauss-Kronrod integration or something equally efficient (or better).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping to use the GNU Scientific Library quadrature routine qags, but it returns the result with type double.&#xA;As things stand I think I'm probably going to have to re-write sections of the GSL quadrature routines to convert the return type to std vector, but that would be a rather lengthy and possibly error-prone detour. I was hoping somebody might recommend a better solution!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is an example of the kind of function I'm trying to integrate over, presently employing the basic trapezoidal rule but indicating where I would instead prefer to implement the GSL routine gaq. whilst it is much simpler than my actual problem, it demonstrates that each element that is put in the vector is calculated from the previous result, hence the requirement of the container.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;gsl/gsl_integration.h&amp;gt;&#xA;using namespace std;&#xA;&#xA;vector&amp;lt;double&amp;gt; f(double E, int N) {&#xA;    vector&amp;lt;double&amp;gt; result;&#xA;    result.reserve(N);&#xA;    double x = E;&#xA;    for (int it=0; it &amp;lt; N; ++it){&#xA;        result.push_back(x);&#xA;        x *= x;&#xA;    }&#xA;    return result;&#xA;}&#xA;&#xA;vector&amp;lt;double&amp;gt; f_qag(double E, void * params) {&#xA;    int N = *(int *) params;&#xA;    vector&amp;lt;double&amp;gt; result;&#xA;    result.reserve(N);&#xA;    double x = E;&#xA;    for (int it=0; it &amp;lt; N; ++it){&#xA;        result.push_back(x);&#xA;        x *= x;&#xA;    }&#xA;    return result;&#xA;}&#xA;&#xA;&#xA;int main(){&#xA;    vector&amp;lt;double&amp;gt; result;&#xA;    vector&amp;lt;double&amp;gt; integrate;&#xA;    int N = 20;&#xA;    result.reserve(N);&#xA;    integrate.reserve(N);&#xA;    for (int i = 0; i &amp;lt; N; i++)&#xA;        result[i] = 0.;&#xA;    double end = 1.0;&#xA;    double start = -1.0;&#xA;&#xA;    // I would like to use qag here&#xA;    /* gsl_integration_workspace * w = gsl_integration_workspace_alloc (1000); */&#xA;    /* vector&amp;lt;double&amp;gt; error; */&#xA;    /* gsl_function F; */&#xA;    /* F.function = &amp;amp;f_qag; */&#xA;    /* F.params = &amp;amp;N; */&#xA;    /* gsl_integration_qag (&amp;amp;F, start, end, 0, 1e-7, 1000, 1, w, &amp;amp;result, &amp;amp;error); */&#xA;&#xA;    // instead of resorting to the trapezoidal rule here&#xA;    double E;&#xA;    const int n = 1000;&#xA;    double factor = (end - start)/(n*1.);&#xA;    for (int k=0; k&amp;lt;n+1; k++) {&#xA;        E = start + k*factor;&#xA;        integrate = f(E, N);&#xA;        for (int i = 0; i &amp;lt; N; i++){&#xA;            if ((k==0)||(k==n))&#xA;                result[i] += 0.5*factor*integrate[i];&#xA;            else &#xA;                result[i] += factor*integrate[i];&#xA;        }&#xA;    }   &#xA;&#xA;    for (int i = 0; i &amp;lt; N; i++)&#xA;        cout&amp;lt;&amp;lt;i&amp;lt;&amp;lt;&quot; &quot;&amp;lt;&amp;lt;result[i]&amp;lt;&amp;lt;endl;&#xA;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I intend to check for convergence against the pseudo-result conv;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;double conv = 0.;&#xA;for (int i = 0; i &amp;lt; N; i++)&#xA;    conv += abs(integrate[i]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="22395" LastActivityDate="2019-04-27T06:06:59.570" Title="How to integrate the contents of a vector using an adaptive quadrature routine" Tags="|c++|quadrature|vector|gsl|" AnswerCount="0" CommentCount="0" ClosedDate="2019-04-29T15:20:56.717" ContentLicense="CC BY-SA 4.0" />
  <row Id="32515" PostTypeId="2" ParentId="32513" CreationDate="2019-04-27T08:22:40.530" Score="10" Body="&lt;p&gt;According to Wolfram Alpha, &lt;span class=&quot;math-container&quot;&gt;$x^5+3(x-1)=0$&lt;/span&gt; has no closed-form solution, so you can forget about a nice closed-form expression. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see nothing wrong with Newton's method; it should be quick and accurate, and with some analysis like the one you are sketching I think you can identify a safe starting point and prove global convergence. It might even be faster than a complicated closed-form solution for &lt;span class=&quot;math-container&quot;&gt;$q=4$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2019-04-27T08:22:40.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32516" PostTypeId="2" ParentId="32511" CreationDate="2019-04-27T08:32:28.920" Score="6" Body="&lt;p&gt;Pseudoinverses typically will be computed via some truncation procedure to determine the rank, so they are &lt;em&gt;not&lt;/em&gt; close to the original inverse. Example:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$A = Q&#xA;\begin{bmatrix}&#xA;1\\ &amp;amp; 10^{-12} \\ &amp;amp; &amp;amp; 10^{-18}&#xA;\end{bmatrix}Q^{*}&#xA;$$&lt;/span&gt;&#xA;has exact (pseudo)inverse&#xA;&lt;span class=&quot;math-container&quot;&gt;$$A^+ = A^{-1} = Q&#xA;\begin{bmatrix}&#xA;1\\ &amp;amp; 10^{12} \\ &amp;amp; &amp;amp; 10^{18}&#xA;\end{bmatrix}Q^{*},&#xA;$$&lt;/span&gt;&#xA;but any numerical procedure will have to take a decision on the rank and will likely truncate one or two of those diagonal entries to zero, returning for instance&#xA;&lt;span class=&quot;math-container&quot;&gt;$$B = Q&#xA;\begin{bmatrix}&#xA;1\\ &amp;amp; 10^{12} \\ &amp;amp; &amp;amp; 0&#xA;\end{bmatrix}Q^{*},&#xA;$$&lt;/span&gt;&#xA;or &#xA;&lt;span class=&quot;math-container&quot;&gt;$$B = Q&#xA;\begin{bmatrix}&#xA;1\\ &amp;amp; 0 \\ &amp;amp; &amp;amp; 0&#xA;\end{bmatrix}Q^{*},&#xA;$$&lt;/span&gt;&#xA;(depending on truncation thresholds), which are not close at all to &lt;span class=&quot;math-container&quot;&gt;$A^+$&lt;/span&gt;. That is an inherent limitation and I don't think you can overcome it. It's by design: typically, when one writes &lt;code&gt;pinv(A)&lt;/code&gt; one wants &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, not &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt;. If you really want &lt;span class=&quot;math-container&quot;&gt;$A^{-1}$&lt;/span&gt;, then just use &lt;code&gt;inv(A)&lt;/code&gt; and forget about the warnings. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it's probably time to go back to the blackboard, think about what you really need in these edge cases, and see if there is a more stable alternative expression for it. If the exact solution of your problem is &lt;em&gt;that&lt;/em&gt; ill-conditioned, then probably it's not even what you want at the end of the day.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2019-04-27T08:32:28.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32517" PostTypeId="1" CreationDate="2019-04-27T12:41:00.110" Score="2" ViewCount="120" Body="&lt;p&gt;I want to use the Crank-Nicolson scheme to solve the equation &#xA;&lt;span class=&quot;math-container&quot;&gt;$$u_t = iu_{xx}+2iu$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the analysis: Suppose we make a grid, with &lt;span class=&quot;math-container&quot;&gt;$k = dt$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$h = dx$&lt;/span&gt;, the usual notation, and also &lt;span class=&quot;math-container&quot;&gt;$u_j^n = u(x_j,t_n)$&lt;/span&gt; . The round-off error equation for the crank-nicolson scheme for this equation is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\displaystyle\frac{\epsilon_{j}^{n+1}-\epsilon_j^n}{k} = \frac{i}{2h^2}\left(\epsilon_{j-1}^n+\epsilon_{j+1}^n-2\epsilon_j^n+\epsilon_{j-1}^{n+1}+\epsilon_{j+1}^{n+1}-2\epsilon_{j}^{n+1}\right) +2i\epsilon_j^n$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After rearranging it a bit, we can arrive at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\displaystyle -\frac{ki}{2h^2}\left(\epsilon_{j-1}^{n+1}+\epsilon_{j+1}^{n+1}-2\epsilon_j^{n+1}\right)+\epsilon_{j}^{n+1} = \frac{ki}{2h^2}\left(\epsilon_{j-1}^{n}+\epsilon_{j+1}^{n}-2\epsilon_j^n\right)+(1+2ki)\epsilon_j^n$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By writing the Fourier series of epsilon, &lt;span class=&quot;math-container&quot;&gt;$\epsilon_j^n  = r^ne^{i\omega jh}$&lt;/span&gt; and dividing by &lt;span class=&quot;math-container&quot;&gt;$\epsilon_j^n$&lt;/span&gt; we have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\displaystyle -\frac{ki}{2h^2}\big(2r\cos(\omega h)-2r\big)+r=\frac{ki}{2h^2}\big(2\cos(\omega h)-2\big)+1+2ki$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the identity &lt;span class=&quot;math-container&quot;&gt;$1-\cos(\theta) = 2\sin^2\big(\frac{\theta}{2}\big)$&lt;/span&gt; we finally have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\displaystyle \left(1+\frac{2ki}{h^2}\sin^2\bigg(\frac{\omega h}{2}\bigg)\right)r = 1 + 2ki -\frac{2ki}{h^2}\sin^2\bigg(\frac{\omega h}{2}\bigg)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$r = \frac{1 + 2ki -\frac{2ki}{h^2}\sin^2\big(\frac{\omega h}{2}\big)}{1+\frac{2ki}{h^2}\sin^2\big(\frac{\omega h}{2}\big)}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why is this a problem?&lt;/strong&gt; - Consider the Fourier mode &lt;span class=&quot;math-container&quot;&gt;$\omega \approx \frac{2\pi}{h}$&lt;/span&gt;. For that mode, the &lt;span class=&quot;math-container&quot;&gt;$\sin$&lt;/span&gt; is nearly zero. For that fourier mode, we would have roughly speaking &lt;span class=&quot;math-container&quot;&gt;$r = \frac{1+2ki}{1}$&lt;/span&gt;, and that's a problem because &lt;span class=&quot;math-container&quot;&gt;$|r|^2 = \frac{1+4k^2}{1}= 1 + 4k^2 &amp;gt; 1$&lt;/span&gt;, so I expect the error to explode, and the scheme to be unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But it isn't.&lt;/strong&gt; Infact, the code I wrote that implements this scheme works. Why? I expected it to be unstable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;another thing to note is that the problem occurs also at &lt;span class=&quot;math-container&quot;&gt;$\omega = 0$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="30498" LastEditorUserId="20688" LastEditDate="2019-04-27T20:55:23.780" LastActivityDate="2019-05-05T04:55:33.310" Title="Stability of Crank-Nicolson for $u_t = iu_{xx}+2iu$" Tags="|pde|finite-difference|stability|numerics|crank-nicolson|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32518" PostTypeId="2" ParentId="32499" CreationDate="2019-04-27T18:07:13.853" Score="3" Body="&lt;p&gt;Based on the provided description as well as the figure, you have an undirected graph with a single source and a sink at hand. The most famous option is to implement &lt;a href=&quot;https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dijkstra's algorithm&lt;/a&gt;. There are other options that are faster, but as long as your dealing with small instances and have no CPU time limit, I think you're good to go. In case you're interested, Wikipedia has a list of some other option, available from &lt;a href=&quot;https://en.wikipedia.org/wiki/Shortest_path_problem#Single-source_shortest_paths&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the implementation, there are lots of available code on the web (some example are available from &lt;a href=&quot;https://www.quora.com/What-is-the-most-simple-efficient-C++-code-for-Dijkstras-shortest-path-algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.geeksforgeeks.org/c-program-for-dijkstras-shortest-path-algorithm-greedy-algo-7/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but you might find other implementations as well). In addition, if you're interested in learning how to code these kinds of algorithms, a good start point is to study the &lt;a href=&quot;https://en.wikipedia.org/wiki/Pseudocode&quot; rel=&quot;nofollow noreferrer&quot;&gt;pseudo-code&lt;/a&gt; of your chosen algorithm (for the pseudo-code of Dijkstra's algorithm, see &lt;a href=&quot;https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm#Pseudocode&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ps. In case you need to find the shortest paths between all pairs of nodes, you might try using &lt;a href=&quot;https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Floyd–Warshall algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2019-04-27T18:07:13.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32519" PostTypeId="2" ParentId="32517" CreationDate="2019-04-27T21:07:34.670" Score="1" Body="&lt;p&gt;The scheme is indeed unstable. It explodes - but very very slowly. By printing the maximum eigenvalue of the operator i confirmed the instability. It's greater than 1. Then why does it work? because it's &lt;span class=&quot;math-container&quot;&gt;$1.000053263$&lt;/span&gt; and my t_final is small.&lt;/p&gt;&#xA;" OwnerUserId="30498" LastActivityDate="2019-04-27T21:07:34.670" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32523" PostTypeId="1" AcceptedAnswerId="32531" CreationDate="2019-04-28T14:16:42.563" Score="2" ViewCount="7607" Body="&lt;p&gt;I have a set of data obtained for the I-V characteristics of an LED. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.005  -0.004&#xA;0.053  -0.003&#xA;0.101  -0.003&#xA;0.153  -0.002&#xA;0.201  -0.002&#xA;0.252  -0.002&#xA;0.303  -0.004&#xA;0.354  -0.004&#xA;0.403  -0.004&#xA;0.454  -0.003&#xA;0.503  -0.003&#xA;0.554  -0.003&#xA;0.603  -0.003&#xA;0.654  -0.003&#xA;0.702  -0.003&#xA;0.753  -0.002&#xA;0.802  -0.002&#xA;0.853  -0.002&#xA;0.902  -0.002&#xA;0.953  -0.001&#xA;1.002  -0.002&#xA;1.050  -0.001&#xA;1.104  -0.004&#xA;1.153  -0.004&#xA;1.204  -0.004&#xA;1.253  -0.004&#xA;1.304  -0.004&#xA;1.353  -0.003&#xA;1.404  -0.003&#xA;1.452  -0.003&#xA;1.503  -0.003&#xA;1.552  -0.002&#xA;1.603  -0.003&#xA;1.652  -0.002&#xA;1.703  -0.002&#xA;1.752  -0.002&#xA;1.803  -0.002&#xA;1.851  -0.001&#xA;1.903  -0.002&#xA;1.954  -0.004&#xA;2.005  -0.004&#xA;2.054  -0.004&#xA;2.102  -0.003&#xA;2.153  -0.003&#xA;2.202  -0.003&#xA;2.253  -0.003&#xA;2.301  -0.002&#xA;2.352  -0.001&#xA;2.398  0.001&#xA;2.442  0.008&#xA;2.475  0.024&#xA;2.501  0.050&#xA;2.519  0.081&#xA;2.533  0.118&#xA;2.544  0.156&#xA;2.552  0.199&#xA;2.560  0.240&#xA;2.567  0.284&#xA;2.573  0.327&#xA;2.578  0.373&#xA;2.583  0.417&#xA;2.587  0.462&#xA;2.591  0.509&#xA;2.595  0.554&#xA;2.599  0.601&#xA;2.602  0.647&#xA;2.606  0.694&#xA;2.609  0.740&#xA;2.611  0.789&#xA;2.615  0.835&#xA;2.617  0.883&#xA;2.620  0.929&#xA;2.622  0.978&#xA;2.625  1.025&#xA;2.627  1.073&#xA;2.630  1.120&#xA;2.632  1.169&#xA;2.635  1.215&#xA;2.637  1.264&#xA;2.639  1.311&#xA;2.641  1.358&#xA;2.643  1.407&#xA;2.645  1.454&#xA;2.648  1.503&#xA;2.649  1.550&#xA;2.652  1.599&#xA;2.653  1.646&#xA;2.655  1.696&#xA;2.657  1.743&#xA;2.658  1.792&#xA;2.660  1.839&#xA;2.662  1.889&#xA;2.664  1.936&#xA;2.666  1.985&#xA;2.667  2.033&#xA;2.668  2.082&#xA;2.670  2.129&#xA;2.672  2.179&#xA;2.674  2.226&#xA;2.675  2.276&#xA;2.676  2.324&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I fit only a range of values out of this data to a linear function? I am able to achieve this by saving the required range of data as a separate file &lt;code&gt;linear-07.txt&lt;/code&gt; and then fitting this set of values to the function f(x). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set term svg&#xA;set output 'blue.svg'&#xA;set term svg&#xA;set output 'blue.svg'&#xA;&#xA;f(x) = a*x + b&#xA;fit f(x) 'linear-07.txt' via a,b&#xA;&#xA;plot[][-1:3] '07-B.txt' pt 7, f(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However this seems to be exhausting. Is there an alternate method to this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, How do I obtain the x-value (of the fit function) corresponding to f(x)=0?&lt;/p&gt;&#xA;" OwnerUserId="29014" LastActivityDate="2019-05-29T12:02:25.273" Title="Gnuplot: How can I fit a range of points (out of the entire data) to a function?" Tags="|interpolation|regression|curve-fitting|gnuplot|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32524" PostTypeId="1" CreationDate="2019-04-28T17:20:48.243" Score="1" ViewCount="1807" Body="&lt;p&gt;I have a set of data &lt;code&gt;data.txt&lt;/code&gt; which can be fit to a Gaussian function, &lt;code&gt;f(x)&lt;/code&gt;. I want to determine the coordinates of the point of maxima (peak) from both the data given and also from the fit function &lt;code&gt;f(x)&lt;/code&gt;. How can I achieve this in gnuplot?&lt;/p&gt;&#xA;" OwnerUserId="29014" LastEditorUserId="29014" LastEditDate="2019-05-01T20:13:58.420" LastActivityDate="2020-01-26T22:01:30.180" Title="Gnuplot: How can I determine the maxima of a fit function in gnuplot?" Tags="|plotting|curve-fitting|gnuplot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32525" PostTypeId="1" CreationDate="2019-04-28T19:32:09.257" Score="2" ViewCount="70" Body="&lt;p&gt;I am solving the task that is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input:&lt;/strong&gt; a polygon. Can be any kind of polygon without self intersections. Can be a non-convex and with holes inside.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Goal:&lt;/strong&gt; to cover it with 2 (at least) or more parallelograms that together are equal to or contain the whole polygon. The following criteria should be met:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There is no point of the polygon that lies beyond parallelograms.&lt;/li&gt;&#xA;&lt;li&gt;The number of the parallelograms should be the least possible. I.e. we want to find largest parallelograms that cover the polygon.&lt;/li&gt;&#xA;&lt;li&gt;These parallelograms must not intersect within the polygon.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Is this task solved and if it is solved - how? I am looking for a direction where to start with and what related algorithms/theory to learn.&lt;/p&gt;&#xA;" OwnerUserId="31616" LastEditorUserId="31616" LastEditDate="2019-04-30T07:24:16.590" LastActivityDate="2019-04-30T07:24:16.590" Title="Cover a polygon with least amount of parallelograms" Tags="|algorithms|computational-geometry|" AnswerCount="0" CommentCount="5" ClosedDate="2019-05-21T20:15:36.380" ContentLicense="CC BY-SA 4.0" />
  <row Id="32526" PostTypeId="2" ParentId="32512" CreationDate="2019-04-28T19:58:29.487" Score="0" Body="&lt;p&gt;The big derivatives may be a problem when integrating numerically. It seems to me that you're using &lt;a href=&quot;https://en.wikipedia.org/wiki/Euler_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euler's method&lt;/a&gt; with a fixed time step. You may consider other methods such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Predictor%E2%80%93corrector_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;predictor–corrector methods&lt;/a&gt; that may help you to get smaller integration errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, in your model, you get big derivatives whenever the particles are close, since the forces due to magnetic interactions decrease with the distance between particles.&#xA;Hence, although you do not have physical collisions, whenever particles are close enough you have a strong interaction between them. And larger forces mean that the change in velocity is higher within each time step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my advice is that you compute the total acceleration that each particle would feel (you already have to do that in your code) and to treat separately the subset of particles that have an acceleration above a given threshold. For that subset of particles you can subdivide the time step and make more intermediate computations regarding the movement of those particles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also consider to do the same thing for the subset of particles with velocities above another threshold.&lt;/p&gt;&#xA;" OwnerUserId="30463" LastActivityDate="2019-04-28T19:58:29.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32528" PostTypeId="1" CreationDate="2019-04-29T01:45:14.577" Score="2" ViewCount="97" Body="&lt;p&gt;In Matlab, an ILU with threshold and pivoting (ILUTP) can be passed by default as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;setup.type = 'ilutp';&#xA;[L, U] = ilu(A, setup);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Looking for an equivalent in scipy, ex:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sparse.linalg.spilu(A, options = ??)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Tried to look up how matlab does the pivot and threshold but didn't seem to find much information. &lt;/p&gt;&#xA;" OwnerUserId="31617" LastActivityDate="2019-04-29T01:45:14.577" Title="ILUTP in sparse.linalg.spilu?" Tags="|matlab|scipy|matrix-factorization|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32529" PostTypeId="2" ParentId="31456" CreationDate="2019-04-29T04:13:40.273" Score="2" Body="&lt;p&gt;For the simplest case of linear advection, applying von Neumann stability analysis gives the necessary restriction on the time step size, known as the CFL condition:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{c\Delta t}{\Delta x}\le 1 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a nonlinear equation such as Burgers' equation, it is not possible to derive an expression for the necessary restriction on the time step size due to the nonlinearity. In practice for nonlinear problems, the same expression that was derived for the linear advection equation is used, where &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is now replaced by &lt;span class=&quot;math-container&quot;&gt;$S^n_{max}$&lt;/span&gt;, the maximum wavespeed throughout the domain at time &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. For Burgers' equation, an obvious choice for &lt;span class=&quot;math-container&quot;&gt;$S^n_{max}$&lt;/span&gt; is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S^n_{max}=\max_i|u_i^n|$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more thorough choice is to consider the wavespeeds of any shocks or expansion waves that are present:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$S^n_{max}=\max_i\left(\left|\frac{1}{2}(u_i^n+u_{i+1}^n)\right|,\max(u_i^n,u_{i+1}^n)\right)$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The timestep is the calculated as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\Delta t=CFL\Delta x/S^n_{max}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$CFL\le1$&lt;/span&gt; is a user-specified coefficient. In practice, a value of &lt;span class=&quot;math-container&quot;&gt;$CFL=0.9$&lt;/span&gt; is usually appropriate to account for the uncertainty in the estimate (Toro 1997).&lt;/p&gt;&#xA;" OwnerUserId="30358" LastActivityDate="2019-04-29T04:13:40.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32530" PostTypeId="2" ParentId="32495" CreationDate="2019-04-29T06:41:00.813" Score="1" Body="&lt;p&gt;I'd love to say that I understand exactly all of the scale factors and shifts that I did below, but I mostly played around with factors until things matched :) I would wait until I've thought it all through, but I wanted to get this solution to you, so forgive me for the incomplete answer.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;fs=1e2&#xA;t0=-100&#xA;t1=100.0&#xA;x = np.arange(t0,t1, 1./fs)&#xA;y = np.exp(-((x-5)**2)/4)&#xA;&#xA;y_hat = np.fft.fftshift(np.fft.fft(np.fft.fftshift(y))) / x.shape[0] *(t1-t0)/np.sqrt(2.0*np.pi)&#xA;freq=np.arange(-fs/2,fs/2,1.0/(t1-t0))&#xA;omega=2*np.pi*freq&#xA;y_hat_exact=np.sqrt(2.0)*np.exp(-omega**2-5j*omega)&#xA;&#xA;plt.ion()&#xA;&#xA;ff=plt.figure(1)&#xA;ff.clf()&#xA;ax=ff.gca()&#xA;ax.plot(x,y,'b-')&#xA;ax.set_xlabel('x')&#xA;ax.set_ylabel('y(x)')&#xA;&#xA;ff=plt.figure(2)&#xA;ff.clf()&#xA;ax=ff.gca()&#xA;ax.plot(omega, y_hat.real, &quot;b.-&quot;,label='re fft')&#xA;ax.plot(omega, y_hat.imag, &quot;r.-&quot;,label='im fft')&#xA;ax.plot(omega,y_hat_exact.real,'k--',label='re exact')&#xA;ax.plot(omega,y_hat_exact.imag,'k:',label='im exact')&#xA;ax.set_xlabel('Radial frequency &lt;span class=&quot;math-container&quot;&gt;$\omega$&lt;/span&gt; (rad/s)')&#xA;ax.set_ylabel('F[y]')&#xA;ax.set_xlim([-1.0,1.0])&#xA;&#xA;ax.legend()&#xA;&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Some comments:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can't use &lt;code&gt;x&lt;/code&gt; as the independent variable for plotting the FFT, you need frequency (or radial frequency). The code shows how to calculate that.&lt;/li&gt;&#xA;&lt;li&gt;You have to divide the result by the number of samples. This is a convention of an FFT.&lt;/li&gt;&#xA;&lt;li&gt;Dividing by &lt;span class=&quot;math-container&quot;&gt;$\sqrt{2\pi}$&lt;/span&gt; is to match the convention Fourier transform convention used by Wolfram Alpha&lt;/li&gt;&#xA;&lt;li&gt;The scaling by the total time &lt;code&gt;t1-t0&lt;/code&gt; was figured out experimentally. If I think about it long enough, I'm sure it has to do with the fact that the analytical Fourier transform is an integral over this time period. &lt;/li&gt;&#xA;&lt;li&gt;The input to the FFT needs to start from x=0. If there's non-zero components in negative x, then they need to be wrapped around to the end, hence the &lt;code&gt;fftshift&lt;/code&gt; of &lt;code&gt;y&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Your Wolfram Alpha link and corresponding image are for &lt;span class=&quot;math-container&quot;&gt;$f(x) = e^{-(x-2)^2/4}$&lt;/span&gt; while your code is for &lt;span class=&quot;math-container&quot;&gt;$f(x) = e^{-(x-5)^2/4}$&lt;/span&gt;. My example uses the latter. &lt;/li&gt;&#xA;&lt;li&gt;Wolfram Alpha uses a &lt;span class=&quot;math-container&quot;&gt;$+i\omega t$&lt;/span&gt; convention while this Python &lt;code&gt;fft&lt;/code&gt; function uses a &lt;span class=&quot;math-container&quot;&gt;$-i\omega t$&lt;/span&gt; convention, which is why the imaginary component in my plot is flipped compared to Wolfram.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zm99b.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zm99b.jpg&quot; alt=&quot;fft output with proper scaling&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="23262" LastActivityDate="2019-04-29T06:41:00.813" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32531" PostTypeId="2" ParentId="32523" CreationDate="2019-04-29T11:14:37.250" Score="3" Body="&lt;p&gt;Thanks to &lt;a href=&quot;https://scicomp.stackexchange.com/users/31611/thor&quot;&gt;Thor&lt;/a&gt;, I figured out the solutions to my questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As he pointed out, the range of the function can be specified in the &lt;code&gt;fit&lt;/code&gt; command. In the above data set, the linear region starts from &lt;code&gt;x=2.63&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set term svg&#xA;set output 'blue.svg'&#xA;set term svg&#xA;set output 'blue.svg'&#xA;&#xA;f(x) = a*x + b&#xA;fit[2.63:] f(x) '07-B.txt' via a,b&#xA;&#xA;plot[][-1:3] '07-B.txt' pt 7, f(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, to obtain the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; corresponding to &lt;span class=&quot;math-container&quot;&gt;$f(x)=0$&lt;/span&gt;, since &lt;span class=&quot;math-container&quot;&gt;$f(x) = ax + b$&lt;/span&gt;, the required &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is given by &#xA;&lt;span class=&quot;math-container&quot;&gt;$$x = -\dfrac{b}{a}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, adding the line &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print -b/a&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives the required value of &lt;code&gt;x&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29014" LastActivityDate="2019-04-29T11:14:37.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32532" PostTypeId="1" CreationDate="2019-04-29T12:53:16.073" Score="3" ViewCount="231" Body="&lt;p&gt;My question is: how come that certain formulations of the wave equation can be time integrated more efficiently then others?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Le me expand a bit on that. Consider the wave equation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{d^2 p(t,x)}{dt^2} -c^2 \frac{d p(t,x)}{dx^2} = 0 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions &lt;span class=&quot;math-container&quot;&gt;$ p(t,0) = \sin(\omega t) $&lt;/span&gt; and initial conditions &lt;span class=&quot;math-container&quot;&gt;$ p(0,x) = 0 $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ p'(0,x) = 0$&lt;/span&gt;. One thing that is a bit of a concern is that these ics and bcs are not consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation can be written as a system of first order equations. In fact, there are multiple versions the of systems of first order equations the original equation can be re-written in. Here is a system of first order equations that is equivalent to the above equation, let's call it &lt;strong&gt;A&lt;/strong&gt;. We introduce an auxiliary variable &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{d p(t,x)}{dt} - v(t,x) = 0 $$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{d v(t,x)}{dt} -c^2 \frac{d p(t,x)}{dx^2} = 0 $$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions &lt;span class=&quot;math-container&quot;&gt;$ p(t,0) = \sin(\omega t) $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ v(t,0) = \omega \cos(\omega t) $&lt;/span&gt; and initial conditions &lt;span class=&quot;math-container&quot;&gt;$ p(0,x) = 0 $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ v(0,x) = 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, is a second version the original equation can be re-written as, let's call that &lt;strong&gt;B&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{d p(t,x)}{dt} + c^2 \frac{d v(t,x)}{dx} = 0$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ \frac{d v(t,x)}{dt} + \frac{d p(t,x)}{dx} = 0$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with boundary conditions &lt;span class=&quot;math-container&quot;&gt;$ p(t,0) = \sin(\omega t) $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ v(t,0) = 1/c \sin(\omega t) $&lt;/span&gt; and initial conditions &lt;span class=&quot;math-container&quot;&gt;$ p(0,x) = 0 $&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$ v(0,x) = 0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am observing that &lt;strong&gt;B&lt;/strong&gt; can be time integrated much more efficiently than &lt;strong&gt;A&lt;/strong&gt;. Let me show that in Mathematica code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We generate a single FEM mesh that is used in all cases. We set up some constants:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$HistoryLength = 0;&#xA;c = 343;&#xA;omega = 5025;&#xA;tend = 0.002;&#xA;Needs[&quot;NDSolve`FEM`&quot;]&#xA;mesh = ToElementMesh[Line[{{0}, {1}}], &quot;MaxCellMeasure&quot; -&amp;gt; 0.025]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This generates a 1D mesh with about 40 second order line elements and it used in all examples below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Set up the original equations, bcs, and ics:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SecondOrderModel = D[p[t, x], {t, 2}] - c^2*D[p[t, x], {x, 2}] == 0;&#xA;SecondOrderBC = p[t, 0] == Sin[omega t];&#xA;SecondOrderIC = {p[0, x] == 0, Derivative[1, 0][p][0, x] == 0};&#xA;pde1 = {SecondOrderModel, SecondOrderBC, SecondOrderIC};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Set up &lt;strong&gt;A&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NDSolveFirstOrderModel = {&#xA;   D[p[t, x], {t, 1}] - v[t, x] == 0, &#xA;   D[v[t, x], {t, 1}] - c^2*D[p[t, x], {x, 2}] == 0};&#xA;NDSolveFirstOrderIC = {p[0, x] == 0, v[0, x] == 0};&#xA;NDSolveFirstOrderBC = {p[t, 0] == Sin[omega t], &#xA;   v[t, 0] == D[Sin[omega t], t]};&#xA;pde2 = {NDSolveFirstOrderModel, NDSolveFirstOrderIC, &#xA;   NDSolveFirstOrderBC};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Set up &lt;strong&gt;B&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FirstOrderModel = {&#xA;   D[p[t, x], {t, 1}] + c^2*D[v[t, x], {x, 1}] == 0, &#xA;   D[v[t, x], {t, 1}] + D[p[t, x], {x, 1}] == 0};&#xA;FirstOrderIC = {p[0, x] == 0, v[0, x] == 0};&#xA;FirstOrderBC = {p[t, 0] == Sin[omega t], &#xA;   v[t, 0] == 1/c Sin[omega t]};&#xA;pde3 = {FirstOrderModel, FirstOrderIC, FirstOrderBC};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In order to do a fair timing comparison, we solve all equations once in order to warm up the jit compiler, load stuff etc.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NDSolveValue[pde1, p, {t, 0, tend}, Element[{x}, mesh]];&#xA;NDSolveValue[pde2, p, {t, 0, tend}, Element[{x}, mesh]];&#xA;NDSolveValue[pde3, p, {t, 0, tend}, Element[{x}, mesh]];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Solving the original second order equation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AbsoluteTiming[&#xA; sol1 = NDSolveValue[pde1, p, {t, 0, tend}, Element[{x}, mesh]];]&#xA;&#xA;{1.4, Null}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Solving &lt;strong&gt;A&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AbsoluteTiming[&#xA; sol2 = NDSolveValue[pde2, p, {t, 0, tend}, Element[{x}, mesh]];]&#xA;{1.4, Null}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In fact &lt;strong&gt;A&lt;/strong&gt; is what Mathematica automatically generates from the original equation. So no surprise here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving &lt;strong&gt;B&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AbsoluteTiming[&#xA; sol3 = NDSolveValue[pde3, p, {t, 0, tend}, Element[{x}, mesh]]]&#xA;{0.271091, Null}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So this is much faster to time integrate and solutions are all comparable. The reason that time integration is faster is that the automatic time step selection uses much lager steps. My question is why can &lt;strong&gt;B&lt;/strong&gt; integrated with larger time steps than &lt;strong&gt;A&lt;/strong&gt;. I am looking for references where I could read up on that.&lt;/p&gt;&#xA;" OwnerUserId="11893" LastEditorUserId="9667" LastEditDate="2019-04-30T18:37:34.937" LastActivityDate="2019-04-30T18:37:34.937" Title="Time integration of wave equation" Tags="|reference-request|time-integration|adaptive-timestepping|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32533" PostTypeId="2" ParentId="32524" CreationDate="2019-04-29T14:47:27.287" Score="2" Body="&lt;p&gt;According with &lt;a href=&quot;http://gnuplot-surprising.blogspot.com/2012/05/how-to-pick-out-maximum-and-minimum.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; the way to find max e min with gnuplot is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With Gnuplot 4.6 the both the x and y coordinate of maximum and minimum points can be find out easily. The method is using new command &quot;stats&quot;. This command is used for statistic. When it is run, some statistical results will be gotten. If your data file contains two column of data, (STATS_pos_max_y, STATS_max_y) will be the coordinate of the maximum point and (STATS_pos_min_y, STATS_min_y) will be the coordinate of the minimum point.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you want to find maxima from the fit function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; there are different way:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;computational brutal&lt;/strong&gt; make an array, with a suitable range, evaluate it with your &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and find the max in the result array. With the same index you find also the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; coordinate. To find the max and minimun in the arrays you can use the &lt;em&gt;stats&lt;/em&gt; command.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;math&lt;/strong&gt; essentially keep the analytic form of your function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; and study max e min with derivative. Here depends how you build the function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;. For example with gnuplot &lt;a href=&quot;http://gnuplot.sourceforge.net/docs_4.2/node82.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;fit command&lt;/a&gt; use a nonlinear least-squares (NLLS) Marquardt-Levenberg algorithm and with the keyword &lt;em&gt;via&lt;/em&gt; you can indicate the parameter are to be adjusted. If you define &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; as a Gaussian you will obtain the various parameters and you can study it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="22025" LastActivityDate="2019-04-29T14:47:27.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32534" PostTypeId="1" AcceptedAnswerId="32548" CreationDate="2019-04-29T15:30:48.273" Score="2" ViewCount="345" Body="&lt;p&gt;I wanted to multiply two simple (big and sparse) matrix with numpy. And I saw that the calculation fails when matrices are too big. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If i take &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; a random vector (size &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;). With pandas, I transformed this vector with &lt;code&gt;get_dummies&lt;/code&gt; and I obtain a matrix &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;. One can verify that &lt;span class=&quot;math-container&quot;&gt;$M^\top \cdot M$&lt;/span&gt; is a diagonal matrix and the trace is equal to &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;. When I do it on numpy, it doesn't work for &lt;span class=&quot;math-container&quot;&gt;$n&amp;gt;2000$&lt;/span&gt;. The code is below : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;np.random.seed(11111)&#xA;&#xA;&#xA;n_points = 5000&#xA;X = pd.Series(np.random.uniform(low=1,high=10,size=(n_points)))&#xA;X_bis = pd.get_dummies(X.astype('int').astype('str')).values&#xA;print(np.sum(np.sum(np.matmul(X_bis.T,X_bis))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried with &lt;code&gt;scipy.sparse&lt;/code&gt; but it didn't work either. Does somebody have an idea ?&lt;/p&gt;&#xA;" OwnerUserId="31619" LastEditorUserId="9667" LastEditDate="2019-04-29T16:50:14.047" LastActivityDate="2019-04-30T18:21:36.657" Title="Problem of multiplication of big (sparse) matrix with numpy (python)" Tags="|matrix|sparse-matrix|numpy|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32535" PostTypeId="2" ParentId="32513" CreationDate="2019-04-29T16:06:29.980" Score="3" Body="&lt;p&gt;You have demonstrated that the polynomial has only a single positive root (I assume here that you are only interested in positive real roots). Using Cauchy's upper bound for polynomial roots &lt;span class=&quot;math-container&quot;&gt;$$1+ \max\left\{\left|\frac{a_{n-1}}{a_{n}}\right|, \left|\frac{a_{n-2}}{a_{n}}\right|, \ldots, \left|\frac{a_{0}}{a_{n}}\right|\right\}$$&lt;/span&gt; you can obtain an upper bound for the roots of your polynomial equal to &lt;span class=&quot;math-container&quot;&gt;$U = 1+\beta$&lt;/span&gt;. Hence you have your root bracketed &lt;span class=&quot;math-container&quot;&gt;$x_0 \in [0, 1+\beta]$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I would use a guaranteed solver like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Brent%27s_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dekker-Brent method&lt;/a&gt;. This method combines bisection (slow but sure) with inverse quadratic interpolation (fast but could predict outside the bracketed interval). It has superlinear convergence (&lt;span class=&quot;math-container&quot;&gt;$p\approx 1.6$&lt;/span&gt; if I'm not mistaken) so it might be slower than Newton's method but it has the guarantee that it will converge. Newton's method might &quot;jump&quot; to another root. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2019-04-30T06:51:41.920" LastActivityDate="2019-04-30T06:51:41.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32536" PostTypeId="2" ParentId="32532" CreationDate="2019-04-29T16:46:12.337" Score="5" Body="&lt;p&gt;It's not really meaningful to talk about integrating the equation in form A or B, since one way to integrate A is to first transform to B and then discretize.  You can only really compare the actual discretizations, and I don't know what Mathematica is using. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, we can still make an educated guess. Since A is first order in time but second order in space, Mathematica probably discretizes it in a way that requires &lt;span class=&quot;math-container&quot;&gt;$\Delta t \propto \Delta x^2$&lt;/span&gt;. Meanwhile B is first order in both, so Mathematica uses a discretization that requires only &lt;span class=&quot;math-container&quot;&gt;$\Delta t \propto \Delta x$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, B is a much more useful way to write the wave equation than A, both for purposes of analysis and computation. Mathematica is not really a great tool for solving PDEs. &lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2019-04-29T16:46:12.337" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="32539" PostTypeId="1" CreationDate="2019-04-30T07:39:02.980" Score="3" ViewCount="659" Body="&lt;p&gt;I am trying to implement Poisson image blending as in the paper &lt;a href=&quot;https://www.cs.virginia.edu/~connelly/class/2014/comp_photo/proj2/poisson.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Poisson Image Editing&lt;/a&gt;. This is the task of filling in a masked region of an image by minimizing&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\min_f\int_\Omega \left | \nabla f - \mathbf v\right |^2$$&lt;/span&gt; with &lt;span class=&quot;math-container&quot;&gt;$$f|_{\partial\Omega} = f^*_{\partial\Omega}$$&lt;/span&gt;&#xA;which is equivalent to the Poisson problem&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\Delta f = \mathbf \nabla\cdot \mathbf v$$&lt;/span&gt;&#xA;I use the same discretization of the problem as presented there, which is equivalent to using the 4-neighbour discrete Laplacian kernel &#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{bmatrix}&#xA;0 &amp;amp; 1 &amp;amp; 0\\ &#xA;1 &amp;amp; -4 &amp;amp; 1\\ &#xA;0 &amp;amp; 1 &amp;amp; 0&#xA;\end{bmatrix}$$&lt;/span&gt;&#xA;and solving the linear system &lt;span class=&quot;math-container&quot;&gt;$$Lx = b$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the vector of unknown image values, &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the sparse matrix that applies the above kernel to &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, approximating the Laplacian, and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is the sum of boundary terms and gradient guides. Specifically, this is the system of equations&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sum_{q\in N_p\cap\Omega}f_q-\left |N_p\right | f_p=-\sum_{q\in N_p\cap\partial\Omega}f^*_q-\sum_{q\in N_p}v_{pq}$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$N_p$&lt;/span&gt; is the set of 4 neighbors of point &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; (this is the same as the equation 7 in the paper, albeit with opposite signs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My implementation in Python (scipy) is below. I use a sparse factorization of L to solve the system for each color channel in the image.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;import scipy.sparse as sp&#xA;import scipy.sparse.linalg as linalg&#xA;import argparse&#xA;&#xA;def poisson_problem(image, mask, guide=None, threshold = 0.5):&#xA;    indices = np.full(mask.shape, -1) #map of coordinates inside the domain to unique indices&#xA;    invdices = [] #list of coordinates inside the domain&#xA;    ind = 0&#xA;    for p,m in np.ndenumerate(mask):&#xA;        if m &amp;gt; threshold:&#xA;            indices[p] = ind&#xA;            ind += 1&#xA;            invdices.append(p)&#xA;    N = len(invdices)&#xA;    b = np.zeros([N, image.shape[2]]) #for RHS of equation&#xA;&#xA;    #build sparse matrix&#xA;    data = []&#xA;    I = []&#xA;    J = []&#xA;    for i,p in enumerate(invdices):&#xA;        data.append(-4)&#xA;        I.append(i)&#xA;        J.append(i)&#xA;        for dim in (0, 1):&#xA;            for dir in (-1,1):&#xA;                q = [*p]&#xA;                q[dim] += dir&#xA;                if q[dim] &amp;lt; 0 or q[dim] &amp;gt;= mask.shape[dim]:&#xA;                    continue&#xA;                j = indices[(*q,)]&#xA;                if j &amp;gt; -1:&#xA;                    #contribution from inside the domain&#xA;                    data.append(1)&#xA;                    I.append(i)&#xA;                    J.append(j)&#xA;                else:&#xA;                    b[i,:] = -image[(*q,)] #boundary term (outside domain)&#xA;                if guide is not None:&#xA;                    b[i,:] -= image[p] - image[(*q,)] #vector guide term&#xA;    L = sp.csc_matrix((data, (I,J)), shape=(N,N))&#xA;    return L, b, invdices&#xA;&#xA;&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    parser = argparse.ArgumentParser()&#xA;    parser.add_argument('--circle', action='store_true')&#xA;    parser.add_argument('--guide', action='store_true')&#xA;    args = parser.parse_args()&#xA;    img = np.zeros([256, 256, 3])&#xA;    img[:128,:,0] = 1&#xA;    img[129:,:,1] = 1&#xA;&#xA;    if args.circle:&#xA;        inds = np.indices(img.shape, dtype=np.float)[0:2,:,:,0] - np.array([[[img.shape[0]//2]], [[img.shape[1]//2]]])&#xA;        mask = ((inds[0]*inds[0]+inds[1]*inds[1]) &amp;lt; min(*img.shape[0:2])**2/16).astype(np.float)&#xA;    else:&#xA;        mask = np.zeros(img.shape[0:2])&#xA;        mask[img.shape[0]//4:-img.shape[0]//4, img.shape[1]//4:-img.shape[1]//4] = 1&#xA;&#xA;    #solve poisson problem for each color channel (only BCs change)&#xA;    L, b, I = poisson_problem(img, mask, img if args.guide else None)&#xA;    factor = linalg.factorized(L)&#xA;    xs = np.stack([factor(b[:,i]) for i in range(b.shape[1])], 1)&#xA;&#xA;    #composite and display results&#xA;    img2 = np.zeros(img.shape)&#xA;    for i, p in enumerate(I):&#xA;        img2[p] = xs[i]&#xA;    mask = np.expand_dims(mask,2)&#xA;    img3 = img * (1-mask) + img2 * mask&#xA;    plt.imshow(img3)&#xA;    plt.show()&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make sure it works, I'm pasting the original image into the masked region, hoping to get the original image back. When the mask is square and the gradient guides are provided (&lt;code&gt;--guide&lt;/code&gt;), it seems to work without noticeable errors:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0UmDu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0UmDu.png&quot; alt=&quot;red-green striped image with square region masked out and filled using Poisson image blending&quot;&gt;&lt;/a&gt;&#xA;But when the mask is a circle shape (run with the &lt;code&gt;--circle&lt;/code&gt; and &lt;code&gt;--guide&lt;/code&gt; arguments), this is the result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/KA0lJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KA0lJ.png&quot; alt=&quot;red-green striped image with circular region masked out and filled using Poisson image blending&quot;&gt;&lt;/a&gt;&#xA;This does not even appear to be preserving the boundary condition except where the mask boundary is tangent to the axes. This tendency towards 0 seems to occur even when solving the simple Laplace equation, with no guide:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ideHt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ideHt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that this is some kind of error resulting from the discretization, but as far as I can tell, this is the same method as in the paper which claims to work on arbitrary domain shapes. What could be causing this?&lt;/p&gt;&#xA;" OwnerUserId="31624" LastEditorUserId="31624" LastEditDate="2019-04-30T18:11:28.080" LastActivityDate="2021-09-17T07:00:10.890" Title="Poisson image blending artifacts" Tags="|pde|python|scipy|poisson|image-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32540" PostTypeId="1" AcceptedAnswerId="32541" CreationDate="2019-04-30T08:12:52.740" Score="6" ViewCount="657" Body="&lt;p&gt;I have started investigating in mostly Runge Kutta and Runge Kutta Nyström methods and there one of the only differences between the methods of the same type is their Butcher tableu. For the most common ones (RK4, RKF45, DOPRI54) they can be found on Wikipedia but otherwise I find it hard to find them. I could always go back the original paper but it can be a bit complicated for a newbie like me to read and understand. For example the Tsistouras 5 integrator. It seems unnecessary if people have to do all that work everytime to get those. Is there any website that has a larger collection of butcher tableus? &lt;/p&gt;&#xA;" OwnerUserId="30266" LastEditorUserId="414" LastEditDate="2019-04-30T08:57:29.690" LastActivityDate="2020-07-19T18:08:03.600" Title="Is there a database/website with Butcher tableaus?" Tags="|ode|runge-kutta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32541" PostTypeId="2" ParentId="32540" CreationDate="2019-04-30T08:46:22.967" Score="8" Body="&lt;p&gt;The three best sources of Butcher tableaus are, according to me, the reference books&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.wiley.com/en-ai/Numerical+Methods+for+Ordinary+Differential+Equations%2C+3rd+Edition-p-9781119121503&quot; rel=&quot;nofollow noreferrer&quot;&gt;Numerical Methods for Ordinary Differential Equations, 3rd ed., J.C. Butcher&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.springer.com/gp/book/9783540566700&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solving Ordinary Differential Equations I, 2nd ed., Hairer, Norsett and Wanner&lt;/a&gt; and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.springer.com/gp/book/9783540604525&quot; rel=&quot;nofollow noreferrer&quot;&gt;Solving Ordinary Differential Equations II, 2nd ed., Hairer and Wanner&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You will have to browse through the books but all three of them contain a huge number of tableaus with indications of the &amp;quot;method name&amp;quot; (like Dormand-Prince). Unfortunately, the Butcher book has a very limited index so you'll really need to browse the book.&lt;/p&gt;&#xA;&lt;p&gt;There is also a &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;comprehensive list on Wikipedia&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The Julia language extension &lt;a href=&quot;http://diffeq.sciml.ai/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DifferentialEquations.jl&lt;/a&gt; also has a large set of tableaus implemented. You can find the implementation of the tableaus &lt;a href=&quot;https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/tree/master/src/tableaus&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The Python package &lt;a href=&quot;https://github.com/ketch/nodepy&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nodepy&lt;/a&gt; has a number of tableaus, and will also generate tableaus for many different families of methods, including some that aren't traditionally viewed as Runge-Kutta methods but can be written that way (e.g., extrapolation methods and deferred correction methods).&lt;/p&gt;&#xA;&lt;p&gt;Tableaus for many SSP methods can be found at http://sspsite.org; these are usually written in Shu-Osher form rather than Butcher form, since that's also how they are usually implemented.&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="18981" LastEditDate="2020-07-19T18:08:03.600" LastActivityDate="2020-07-19T18:08:03.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32542" PostTypeId="1" CreationDate="2019-04-30T09:06:51.967" Score="0" ViewCount="123" Body="&lt;p&gt;I find myself in the position of having to solve&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$-\Delta u = f$&lt;/span&gt; on a &lt;em&gt;subset&lt;/em&gt; of Cartesian grid points that don't necessarily form a cuboid domain subject to a homogenious Dirichlet boundary condition (&lt;span class=&quot;math-container&quot;&gt;$u = 0$&lt;/span&gt;). The Laplacian filter I require is the standard 7-point stencil for 3D.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for a cuboid domain, I know this can be efficiently solved using a type-I discrete sine transform (DST) since the DST diagonalizes the equation system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder, whether it's possible to leverage the fast DST for domains that are &lt;em&gt;not&lt;/em&gt; cuboid. It's still a Cartesian grid, but only a subset which is not necessarily convex and might include holes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there fast algorithms that can exploit the Cartesian grid structure? Or do I have to use a more general solver (like an algebraic multigrid preconditioned conjugate gradient method or something like this)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe there is an efficient method to reduce my problem to another with an emcompassing cuboid domain so I still get to use the DST? As far as I can tell, I could introduce source terms in &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; on my actual boundary so that the DST-based approach will reconstruct a boundary of zero. But the problem with this is that determining those source terms seems to require solving a big and &quot;ugly&quot; linear equation system (dense matrix which is not trivially diagonalizable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be your recommended solver for this problem? The number of unknowns are in the tens of millions.&lt;/p&gt;&#xA;" OwnerUserId="7374" LastEditorUserId="7374" LastEditDate="2019-04-30T11:21:17.127" LastActivityDate="2019-05-02T17:32:48.513" Title="Fast Poisson solver (with Dirichlet BC zero) on a *truncated* Cartesian 3D grid" Tags="|numerics|poisson|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32543" PostTypeId="1" AcceptedAnswerId="32545" CreationDate="2019-04-30T12:57:01.380" Score="1" ViewCount="550" Body="&lt;p&gt;Is it necessary to reorder nodes (using Reverse Cuthill-Mckee algorithm, for example) if I am already using a CSR or CSC storing technique? Because since CSR/CSC stores only non-zero elements I guess reorder wouldn't be much advantageous.&lt;/p&gt;&#xA;" OwnerDisplayName="user30551" LastEditorDisplayName="user30551" LastEditDate="2019-08-19T20:45:17.677" LastActivityDate="2019-08-19T20:45:17.677" Title="(FEM) Nodes reordering for sparse matrix storing techniques" Tags="|finite-element|sparse-matrix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32544" PostTypeId="1" CreationDate="2019-04-30T13:01:14.673" Score="4" ViewCount="272" Body="&lt;p&gt;For a given integer &lt;span class=&quot;math-container&quot;&gt;$n\in\mathbb{N}^*$&lt;/span&gt;, I want to find a pair &lt;span class=&quot;math-container&quot;&gt;$(x,y)\in{\mathbb{N}^*}^2$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$x*y=n$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$|y-x|$&lt;/span&gt; is as small as possible.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A naive algorithm I found is :  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for k = floor(sqrt(n)) to 1 step -1&#xA;  if n/k is integer then&#xA;    return (k, n/k)&#xA;  end if&#xA;end for&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Although I am almost certain there is a more efficient way to do this, I cannot find one.&lt;/p&gt;&#xA;" OwnerUserId="31628" LastActivityDate="2023-05-28T21:35:53.217" Title="How to find a pair of divisors as close as possible to each other?" Tags="|algorithms|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="32545" PostTypeId="2" ParentId="32543" CreationDate="2019-04-30T13:12:02.897" Score="6" Body="&lt;p&gt;You should use a reordering. Although it's true that storing a sparse matrix requires the same amount of memory whether or not you reorder it using RCM, reordering it should lead to faster calculations (eg matrix-vector products) due to different/better utilization of cache. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something to keep in mind is that the &quot;best&quot; reordering depends upon what you intend to do with the matrix. Bandwidth reduction reorderings (like RCM) help with matvec's, but if you're reordering a matrix for parallel distribution you should look into methods that minimize  edge-cut/communication-volume (like METIS, others), and if you're looking into sparse direct methods you should consider unstructured nested-dissection (METIS/SCOTCH) or fill-minimization (minimum degree or appproximate MD). &lt;/p&gt;&#xA;" OwnerUserId="2107" LastActivityDate="2019-04-30T13:12:02.897" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32546" PostTypeId="2" ParentId="32544" CreationDate="2019-04-30T14:16:23.777" Score="1" Body="&lt;p&gt;Note that although there could be more efficient algorithms, what you found is not a bad algorithm. I will analyze this a little bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us write down the problem formally as a constrained discrete optimization problem:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;(x^\star, y^\star) = \arg\min_{(x,y) \in \mathbb{Z_{+}}} \Big(E \triangleq \big(y-x\big)^2 + \lambda(x*y - n)^2 \Big)&#xA;$$&lt;/span&gt;&#xA;Note that for positive integers the &lt;span class=&quot;math-container&quot;&gt;$L_2$&lt;/span&gt; norm will attain the same minimum as &lt;span class=&quot;math-container&quot;&gt;$L_1$&lt;/span&gt;. So I'll stick to it. &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt; is a Lagrange multiplier and the right-most term is the non-linear constraint. Even if we can compute the partial derivatives and use the &lt;a href=&quot;https://en.wikipedia.org/wiki/Lagrange_multiplier&quot; rel=&quot;nofollow noreferrer&quot;&gt;method of Lagrange multipliers&lt;/a&gt;, the problem is not easy to solve due to the parameters restricted to a discrete space. Hence we will &lt;strong&gt;relax&lt;/strong&gt; the integer constraint to real numbers and solve the problem for &lt;span class=&quot;math-container&quot;&gt;$\mathbb{R}$&lt;/span&gt;. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Relaxation_(approximation)&quot; rel=&quot;nofollow noreferrer&quot;&gt;relaxed optimum&lt;/a&gt; would be attained as follows:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;x^\star = \arg\min_{x \in \mathbb{R}} \Big(\tilde{E} \triangleq \big(x-\frac{n}{x}\big)^2 \Big)&#xA;$$&lt;/span&gt;&#xA;We achieved this by substitution the constraint : &lt;span class=&quot;math-container&quot;&gt;$y \gets \frac{n}{x}$&lt;/span&gt;. Note that this is also suggested by the Largange optimization:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial E}{\partial \lambda} = xy - n^2 = 0 \implies x=\frac{n}{y} \text{ or } y=\frac{n}{x}.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now differentiating w.r.t. &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align}&#xA;\frac{d \tilde{E}}{dx} = 2\big(x-\frac{n}{x}\big) &amp;amp;= 0\\&#xA;x^2 &amp;amp;= n \\&#xA;x = \sqrt{n}&#xA;\end{align}&#xA;$$&lt;/span&gt;&#xA;gives the optimal solution for the relaxed problem. For negative numbers &lt;span class=&quot;math-container&quot;&gt;$x = -\sqrt{n}$&lt;/span&gt; is also the optimum. This we could already detect from the sign of the input. Having solved the relaxed problem, a typical optimizer would operate by projecting the solution into the feasible set and/or applying a projected gradient descent. The initial feasible solution is &lt;span class=&quot;math-container&quot;&gt;$x_0 = \lfloor \sqrt{n} \rceil$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$\lfloor \cdot \rceil$&lt;/span&gt; is the rounding operation or in fancy words the projection onto the integers. From here, it is possible to walk on the Euclidean discrete manifold of 1-dimensional integers by your method: taking steps of &lt;span class=&quot;math-container&quot;&gt;$+1$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$-1$&lt;/span&gt; depending on the direction. That would amount to a gradient descent by the step size &lt;span class=&quot;math-container&quot;&gt;$h=1$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And these steps are what you are following implicitly when arriving at the algorithm you proposed. Unfortunately, the worst case complexity of this algorithm is attained when the number is prime and we have only 2 divisors: &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, the number itself. Therefore, the worst case complexity reaches to &lt;span class=&quot;math-container&quot;&gt;$O(\sqrt{n})$&lt;/span&gt;. As &lt;span class=&quot;math-container&quot;&gt;$n\rightarrow \infty$&lt;/span&gt;, this would resemble a linear search (it is worse than &lt;span class=&quot;math-container&quot;&gt;$\log$&lt;/span&gt;). Now, the set of integers is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Total_order&quot; rel=&quot;nofollow noreferrer&quot;&gt;totally ordered set&lt;/a&gt;. Using this property, instead of using a linear search, one could think of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Binary_search_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;binary search&lt;/a&gt; variant. But as clarified by @WolfgangBangerth, such an algorithm would not exist bounding the complexity in: &lt;span class=&quot;math-container&quot;&gt;$O(\log(\sqrt{n}))&amp;lt;C\leq O(\sqrt{n})$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though this might not be the exact answer you were looking for, I thought it would give a perspective to develop better solutions. For instance, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Integer_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;integer programming&lt;/a&gt; community might have a better solution.&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="6910" LastEditDate="2019-04-30T17:32:56.543" LastActivityDate="2019-04-30T17:32:56.543" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="32547" PostTypeId="1" CreationDate="2019-04-30T17:25:59.683" Score="0" ViewCount="48" Body="&lt;p&gt;I am supposed to do black-box modeling, I need the input and output data for training, however, the authors didn't agree to share the raw data with me, I can only use their publication where these attached images are given below. Of course, I do understand the curve, however, I don't know how to create raw data from figures, is it could be achieved, if so what could be the easiest approach to have raw data from figures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oC1nH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oC1nH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="30458" LastActivityDate="2019-04-30T17:25:59.683" Title="Extracting raw data from a graph" Tags="|visualization|data-sets|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32548" PostTypeId="2" ParentId="32534" CreationDate="2019-04-30T18:21:36.657" Score="1" Body="&lt;p&gt;You are using integers that are not big enough to represent your data. See the following snippet that solves your issue.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;&#xA;np.random.seed(11111)&#xA;n_points = 50000&#xA;X = pd.Series(np.random.randint(low=1, high=10, size=(n_points)))&#xA;X_bis = pd.get_dummies(X, dtype=np.uint64).values&#xA;print(np.sum(X_bis.T @ X_bis))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2019-04-30T18:21:36.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32549" PostTypeId="2" ParentId="32542" CreationDate="2019-04-30T23:34:05.217" Score="2" Body="&lt;p&gt;The short answer is, no; you can't use the DST&#xA;approach for a case with general geometry or boundary&#xA;conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way to understand this is to consider WHY the&#xA;DST approach works for the &quot;rectangular&quot; case. For&#xA;this case, we happen to know that the complete set&#xA;of eigenvectors of the differential equation are&#xA;products of sin functions. We can then conveniently&#xA;express the solution as a sum of the eigenvectors.&#xA;The DST is a fast way to compute the eigenvectors&#xA;in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a general geometry, the eigenvectors are not simple sin functions&#xA;and we have no cheap way to compute them; that makes&#xA;an eigenvector basis unattractive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you say your grid is Cartesian, it seems like algebraic multigrid&#xA;would be an attractive option and is theoretically faster than using FFT.&#xA;As shown on page 2 of these &lt;a href=&quot;https://people.eecs.berkeley.edu/~demmel/ma221_Spr16/Lectures/Multigrid_Ma221_2016_2pp.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;notes&lt;/a&gt; by Demmel, FFT has performance&#xA;proportional to &lt;span class=&quot;math-container&quot;&gt;$N log N$&lt;/span&gt; while multigrid is order &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the&#xA;number of equations.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastEditorUserId="3441" LastEditDate="2019-05-02T17:32:48.513" LastActivityDate="2019-05-02T17:32:48.513" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="32550" PostTypeId="1" CreationDate="2019-04-30T23:57:03.033" Score="5" ViewCount="149" Body="&lt;p&gt;I have a set of &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;-dimensional vectors, and would like to choose &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; of them to become the columns of an &lt;span class=&quot;math-container&quot;&gt;$n\times m$&lt;/span&gt; matrix. I would like to choose the subset that maximizes &lt;span class=&quot;math-container&quot;&gt;$|A^T A|$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$A^T$&lt;/span&gt; is the transpose of &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$||$&lt;/span&gt; is notation for the determinant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this be done faster than trying out each of &lt;span class=&quot;math-container&quot;&gt;$n \choose m$&lt;/span&gt; possible combinations?&lt;/p&gt;&#xA;" OwnerUserId="31635" LastActivityDate="2019-05-01T21:52:58.543" Title="Choose a subset of $m$ columns that maximize $|A^T A|$?" Tags="|optimization|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32551" PostTypeId="2" ParentId="32539" CreationDate="2019-05-01T03:00:30.693" Score="1" Body="&lt;p&gt;It seems I forgot to accumulate the boundary terms for each relevant entry in the right hand side vector &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. So&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;b[i,:] = -image[(*q,)] #boundary term (outside domain)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;should have been&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;b[i,:] -= image[(*q,)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That fixed it.&lt;/p&gt;&#xA;" OwnerUserId="31624" LastActivityDate="2019-05-01T03:00:30.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32552" PostTypeId="1" CreationDate="2019-05-01T06:45:46.373" Score="0" ViewCount="138" Body="&lt;p&gt;This is part of the code in matlab for a random-walk simulation. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To test the code, I'm using steps=[30]; there will be more values, but I want to run it for 1 trial to decrease code processing. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;log_steps = log(1:steps);&lt;/code&gt; &amp;lt;--- corresponds to the log (steps vector) for the x axis of the plot&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;log_AVG = log(d_AVG);&lt;/code&gt; &amp;lt;---- corresponds to the log (average steps sizes) for the y axis of the plot &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The intended approach&lt;/strong&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/XI9yj.png&quot; alt=&quot;d~sqrt(N) to  find model&amp;#39;s p value&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;to prove that &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; which represents the probability of any step (forward || backward) is 0.5&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PROBLEM: the program's p value estimation is 10x larger than it should be. It gives a value between 4 to 5 for p, when p should be about 0.5.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;where is the logic wrong? Relevant code below.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    figure(i+10);&#xA;    hold on;&#xA;    loglog(log_steps, log_AVG,'-s');&#xA;    %loglog(1:steps(i), d_AVG, '-s');&#xA;&#xA;    N=log_steps;&#xA;    c= log_AVG;&#xA;&#xA;&#xA;&#xA;    p = polyfit(N, c,0);&#xA;    f = (c.* (N.^p));&#xA;    hold on;&#xA;    loglog(N, f);&#xA;    hold off;&#xA;    end;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/IZ1QH.png&quot; alt=&quot;&amp;quot;hyperbolic curve&amp;quot; &amp;lt;--- log-log plot of 30 step random walk without polyfit attempt&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/4YRFa.png&quot; alt=&quot;&amp;quot;straight horizontal line&amp;quot; &amp;lt;-- log-log plot of ONLY randomwalk points&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="31639" LastEditorUserId="30463" LastEditDate="2019-05-02T17:01:42.120" LastActivityDate="2019-09-29T18:02:30.337" Title="Monte Carlo - Random Walk Simulation - polyfit the log log data points?" Tags="|matlab|monte-carlo|plotting|random-sampling|information-theory|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32553" PostTypeId="2" ParentId="32550" CreationDate="2019-05-01T09:59:35.500" Score="5" Body="&lt;p&gt;A &lt;em&gt;related&lt;/em&gt; task to this is to find a subset of column vectors that are maximally linearly independent. Linear independence isn't exactly the same thing as asking for a large determinant, but if we can use one as a proxy for the other then this heuristic may help you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rank revealing QR factorization (RRQR) is a good way to achieve this related task. One of the outputs of rank-revealing QR is a permutation of the input vectors that results in an approximate maximal linear independence. In Python for example I can do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import scipy.linalg as la&#xA;&#xA;n=10  # Size of column vectors&#xA;nvectors=500  #Number of vectors to generate&#xA;np.random.seed(3244) #Seed RNG to make result reproducible&#xA;A=np.random.rand(n,nvectors) &#xA;&#xA;(Q,R,P)=la.qr(A,pivoting=True) #pivoting=True switches to RRQR algorithm&#xA;&#xA;k=5 #Choose k vectors out of total&#xA;B=A[:,P[0:k]] #B is vectors from the RRQR ordering&#xA;C=A[:,0:k] #C is vectors in original ordering&#xA;print(&quot;Permuted: {}, Un-Permuted: {}&quot;.format(np.linalg.det(B.T@B),np.linalg.det(C.T@C)))&#xA;&#xA;#output: Permuted: 60.60606973650909, Un-Permuted: 1.005883253383066&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: On second thought this may be much closer to what you explicitly asked for than I realized, and not just a heuristic. You can use the RRQR factorization itself to find the determinant of this submatrix (product of diagonal entries of its upper triangular factor), and since RRQR permutes to make the diagonal entries nonincreasing this suggests we actually get close to the largest possible determinant by using the permutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This may not strictly be the case though because the permutation generated by RRQR is approximate, it isn't necessarily the best possible. In other words: there could be other RRQR factorizations which do a little better.&lt;/p&gt;&#xA;" OwnerUserId="547" LastEditorUserId="547" LastEditDate="2019-05-01T11:09:34.600" LastActivityDate="2019-05-01T11:09:34.600" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32557" PostTypeId="1" AcceptedAnswerId="32586" CreationDate="2019-05-01T19:40:25.683" Score="1" ViewCount="201" Body="&lt;p&gt;I'd like to use the syms package to do some algebra for me, but the baseline assumption seems to be that variables are scalars. I would like to denote some variables as matrices. This will change the symbolic output/calculations. For example: inverse is not division, operations are not all commutative, and things can be transposed. Below I've illustrated a little bit of what I'd like to be able to do.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms x&#xA;b=(x' * x)^(-1)&#xA;b=(x' * x)^(-1) * x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want this code to treat 'x' as a matrix, such that I should get &lt;span class=&quot;math-container&quot;&gt;$(X'X)^{-1}X$&lt;/span&gt;, and there would be no simplification. However, the output is &lt;code&gt;1/x^2&lt;/code&gt; for the first line and &lt;code&gt;1/x&lt;/code&gt; for the second line. Is there a way to do this in octave?&lt;/p&gt;&#xA;" OwnerUserId="31643" LastEditorUserId="31643" LastEditDate="2019-05-04T17:46:13.693" LastActivityDate="2019-05-06T06:15:04.140" Title="denoting a variable as a matrix using octave syms package" Tags="|matrix|octave|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="32558" PostTypeId="2" ParentId="32557" CreationDate="2019-05-01T21:40:17.040" Score="1" Body="&lt;p&gt;You can define the elements of the matrix and then do the rest of the computations as intended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;syms a b c d&#xA;x = [a b; c d]&#xA;y = (x' .* x)^(-1)&#xA;z = y * x&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&#xA;I've found a &lt;a href=&quot;https://stackoverflow.com/a/21484695&quot;&gt;related post&lt;/a&gt;, which I used in the first part of this edit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is to create a matrix of a determined size:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = sym('a' ,[2 3])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The transpose &lt;code&gt;a'&lt;/code&gt; works as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n = sym('n')&#xA;m = sym('m')&#xA;a = sym('a' ,[n m])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it created the matrix &lt;code&gt;a&lt;/code&gt;.&#xA;However, when I tried &lt;code&gt;a'&lt;/code&gt; I got an error message.&lt;/p&gt;&#xA;" OwnerUserId="30463" LastEditorUserId="30463" LastEditDate="2019-05-04T22:52:49.003" LastActivityDate="2019-05-04T22:52:49.003" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="32559" PostTypeId="2" ParentId="32550" CreationDate="2019-05-01T21:52:58.543" Score="4" Body="&lt;p&gt;It's NP-hard, but there are some heuristic algorithms. See &lt;a href=&quot;https://arxiv.org/abs/1502.07838&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1502.07838&lt;/a&gt; which treats this problem.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastActivityDate="2019-05-01T21:52:58.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32560" PostTypeId="1" AcceptedAnswerId="32561" CreationDate="2019-05-01T22:13:47.450" Score="5" ViewCount="1583" Body="&lt;p&gt;Why not just Lagrange polynomials basis functions &lt;/p&gt;&#xA;" OwnerUserId="31644" LastActivityDate="2023-11-30T17:39:50.883" Title="Why do we use hermite interpolation for finite element method in beams?" Tags="|finite-element|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32561" PostTypeId="2" ParentId="32560" CreationDate="2019-05-01T23:45:58.280" Score="13" Body="&lt;p&gt;The weak form of the euler-bernoulli beam equations has second order weak derivatives.  This means that the finite element space requires continuity in the 1st derivatives across each element boundary.  Lagrange elements have continuity in the 0th derivative only (the values match at the boundary, but the 1st derivatives don’t).  Hermite interpolants can be generalized to ensure continuity to any prescribed derivative order.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a theorem which states that for an nth order weak derivative in the weak form, you need (n-1)st order continuity in the interpolants between each element. &lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="393" LastEditDate="2019-05-02T03:14:21.587" LastActivityDate="2019-05-02T03:14:21.587" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="32562" PostTypeId="2" ParentId="32552" CreationDate="2019-05-02T00:52:56.277" Score="1" Body="&lt;p&gt;If I understood the first image correctly, you should do a linear fit with &lt;span class=&quot;math-container&quot;&gt;$\ln(N)$&lt;/span&gt; being the independent variable and, presumably, &lt;span class=&quot;math-container&quot;&gt;$\ln(d)$&lt;/span&gt; the dependent variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, it seems to me that in your code you use &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; as &lt;span class=&quot;math-container&quot;&gt;$\log(d_{AVG})$&lt;/span&gt;, which is different from the first image, where &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; appears to be a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm not giving a full answer because it seems to me that it's a homework question.)&lt;/p&gt;&#xA;" OwnerUserId="30463" LastActivityDate="2019-05-02T00:52:56.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32563" PostTypeId="1" AcceptedAnswerId="32567" CreationDate="2019-05-02T07:43:15.823" Score="2" ViewCount="1992" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Dormand&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Dormand&lt;/a&gt;–Prince_method&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to implement the Dormand-Prince 4(5) version to solve Initial Value problems. Using regular notation I have &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; matrix and the &lt;span class=&quot;math-container&quot;&gt;$c,b,\hat{b}$&lt;/span&gt; vectors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also have the error tolerence of &lt;span class=&quot;math-container&quot;&gt;$ETOL$&lt;/span&gt; absolute error tolerence &lt;span class=&quot;math-container&quot;&gt;$ATOL$&lt;/span&gt; and relative error tolerance &lt;span class=&quot;math-container&quot;&gt;$RTOL$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At each time step &#xA;I compute &lt;span class=&quot;math-container&quot;&gt;$x_n$&lt;/span&gt; by use of &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat{x}_n$&lt;/span&gt; by us of &lt;span class=&quot;math-container&quot;&gt;$\hat{b}$&lt;/span&gt; and estimate the error by as the difference &lt;span class=&quot;math-container&quot;&gt;$e_n=x_n-\hat{x}_n$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way I understand this method, when the error is approximated then I have to do the following to things: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Decide whether to accept the step just computed or not&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;update the stepsize&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Which exact decision rules/algorithm should I use to do the those steps?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the book I am using (Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations by Petzold and Ascher) does not explain this part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Side Note:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I assume there is a way to update the stepsize regarding what kind of RUnge Kutta method you use. &lt;/p&gt;&#xA;" OwnerUserId="30032" LastActivityDate="2019-05-03T06:55:02.300" Title="Dormand–Prince 5(4): How to update the stepsize and make accept/reject decision?" Tags="|ode|runge-kutta|adaptive-timestepping|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32565" PostTypeId="1" CreationDate="2019-05-02T17:55:49.060" Score="3" ViewCount="60" Body="&lt;p&gt;I have a object which is static, the camera is static and light source is moving. How can the depth mask be calculated ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concept is to use - calculate height from shadow length&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yOYDR.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yOYDR.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets imagine a have more images likes this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And light source is moving, which is &lt;em&gt;sun&lt;/em&gt;, and from time it's position, distance, orientation can be calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xTtzi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xTtzi.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yellow lines are sun rays.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those circles are brightest pixel previously(direction of raw) of a very dark pixel. considering them highest i'n my knowledge . And those green lines as shadow. How can I calculated the depth mask of this terrain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm ignoring the fact that, object/terrain can be colorful - no reflection/absorption.  &lt;/p&gt;&#xA;" OwnerUserId="31654" LastEditorUserId="31654" LastEditDate="2019-05-02T20:17:37.267" LastActivityDate="2019-05-02T20:17:37.267" Title="Calculating depth mask from different lighting" Tags="|computational-geometry|geometry|computer-vision|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32567" PostTypeId="2" ParentId="32563" CreationDate="2019-05-03T06:19:27.957" Score="5" Body="&lt;p&gt;The basic idea is that&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You use the estimated error given to you (cheaply) by the embedded methods;&lt;/li&gt;&#xA;&lt;li&gt;You use a metric to define acceptance using a user-defined relative and absolute tolerance;&lt;/li&gt;&#xA;&lt;li&gt;Based on the order properties of the code and this metric, a new step size is computed;&lt;/li&gt;&#xA;&lt;li&gt;You avoid large differences in step sizes from one step to the next by limiting the maximum increase and maximum decrease. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://www.springer.com/gp/book/9783540566700&quot; rel=&quot;noreferrer&quot;&gt;&quot;Solving Ordinary Differential Equations, part I&quot; by Hairer, Norsett and Wanner&lt;/a&gt; : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For two approximations to the solution &lt;span class=&quot;math-container&quot;&gt;$y_{1i}$&lt;/span&gt; (using the lower order) and &lt;span class=&quot;math-container&quot;&gt;$\hat{y}_{1i}$&lt;/span&gt; (using the higher order), we want the error to satisfy (componentwise) &lt;span class=&quot;math-container&quot;&gt;$$|y_{1i} - \hat{y}_{1i}| \leq sc_i|, \quad sc_i = Atol_i + \max\left(|y_{0i}|,|y_{1i}|\right)\cdot Rtol_i.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a measure of the error, one takes &lt;span class=&quot;math-container&quot;&gt;$$err = \sqrt{\frac{1}{n}\sum_{i=1}^n\left(\frac{y_{1i}-\hat{y}_{1i}}{sc_i}\right)^{2}}$$&lt;/span&gt; and this value is compared to one. From the expected error behaviour &lt;span class=&quot;math-container&quot;&gt;$err \propto C\cdot h^{q+1}$&lt;/span&gt; the optimal step size is obtained as &lt;span class=&quot;math-container&quot;&gt;$$h_{\text{opt}} = h\cdot\left(\frac{1}{err}\right)^{\frac{1}{q+1}}$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; is the previous step size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically, one would add a kind of safety factor so that the probability of having selected a good step size is increased. Typically this safety factor &lt;span class=&quot;math-container&quot;&gt;$fac$&lt;/span&gt; is taken &lt;span class=&quot;math-container&quot;&gt;$fac = 0.8$&lt;/span&gt; or some power of the order like &lt;span class=&quot;math-container&quot;&gt;$fac = \left(0.25\right)^{\frac{1}{q+1}}$&lt;/span&gt; so that &lt;span class=&quot;math-container&quot;&gt;$$h_{\text{opt}} = fac\cdot h\cdot\left(\frac{1}{err}\right)^{\frac{1}{q+1}}$$&lt;/span&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a second measure of safety, one wants to prohibit the stepsize to vary too much from one step to the next. Typically, one limits the new stepsize to be less than a factor five of the previous stepsize. And in some codes, a stepsize increase is forbidden if the previous step was rejected (so if in a previous step, &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; was decreased, it can only increase again if a step with that step size is accepted). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The initial step size for the algorithm can be set by the user or it could be estimated using a procedure based on the order of the method. Hindmarsch and Shampine have developed methods to do so. &lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2019-05-03T06:55:02.300" LastActivityDate="2019-05-03T06:55:02.300" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32569" PostTypeId="1" AcceptedAnswerId="32573" CreationDate="2019-05-03T08:18:22.267" Score="0" ViewCount="61" Body="&lt;p&gt;We have &lt;span class=&quot;math-container&quot;&gt;$X=[\mathbf{x}_1,\mathbf{x}_2...,\mathbf{x}_n]\in\mathbb{R}^{d\times n}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$H=[\mathbf{h}_1,\mathbf{h}_2...,\mathbf{h}_n] \in\mathbb{R}^{d\times n}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$d&amp;lt;n$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; has rank &lt;span class=&quot;math-container&quot;&gt;$r\leq d$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; has rank &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume we have &lt;span class=&quot;math-container&quot;&gt;$\|H\|_F\leq C$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\|\mathbf{x}_i\|_2\leq R$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\|\cdot\|_F$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\|\cdot\|_2$&lt;/span&gt; are the Frobenius norm and vectro 2-norm respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do we know about the upper bound of Frobenius inner product &lt;span class=&quot;math-container&quot;&gt;$\left&amp;lt;H,X\right&amp;gt;_F$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hint:&#xA;I know by Cauchy-Schwarz inequality &lt;span class=&quot;math-container&quot;&gt;$\left&amp;lt;H,X\right&amp;gt;_F &amp;lt; \sqrt nCR$&lt;/span&gt;. The equality will not hold because we can not guarantee &lt;span class=&quot;math-container&quot;&gt;$\mathbf{x}_i = k\mathbf{h}_i, \forall i$&lt;/span&gt;. This is because &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is rank deficient. I think there is a tighter upper bound for &lt;span class=&quot;math-container&quot;&gt;$&amp;lt;H,X&amp;gt;_F$&lt;/span&gt; in terms of singular values of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; and rank &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also posted the question &lt;a href=&quot;https://mathoverflow.net/questions/330603/an-upper-bound-of-matrix-inner-product-lefth-x-right-f-with-constrainted-ra&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="27919" LastEditorUserId="27919" LastEditDate="2019-05-03T10:07:36.573" LastActivityDate="2019-05-03T22:24:41.913" Title="An Upper Bound of $\left&lt;H,X\right&gt;_F$ with Constrainted Rank" Tags="|linear-algebra|matrix|eigenvalues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32570" PostTypeId="2" ParentId="31480" CreationDate="2019-05-03T12:32:42.460" Score="3" Body="&lt;p&gt;You have written down the midpoint rule as a two-step method, a member of the family of multi-step methods. For these methods, one can show that a multi-step method &lt;span class=&quot;math-container&quot;&gt;$$\alpha_{k} y_{n+k} + \alpha_{k-1} y_{n+k-1} + \ldots + \alpha_{0} y_{n} = h\left( \beta_{k} f_{n+k} + \ldots + \beta_{0} f_{n}\right)$$&lt;/span&gt; is called stable if the polynomial &lt;span class=&quot;math-container&quot;&gt;$$\rho(z) = \alpha_{k} z^{k} + \alpha_{k-1} z^{k-1} + \ldots + \alpha_{0}$$&lt;/span&gt; has only roots lying on or within the unit circle and that roots lying on the unit circle are simple roots. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it is up to you to demonstrate for which stepsize &lt;span class=&quot;math-container&quot;&gt;$h$&lt;/span&gt; the roots of the polynomial &lt;span class=&quot;math-container&quot;&gt;$\rho(z) = z^{2} - 2h z - z$&lt;/span&gt; satisfy the above criterion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other approach would be as you hint through the test equation. And then, you should indeed re-write the method as a single-step method:&lt;span class=&quot;math-container&quot;&gt;$$y_{n+1} = y_{n} + h\lambda y_{n+\frac{1}{2}} = y_{n} + h\lambda \left(y_{n}  + \frac{1}{2}h \lambda y_{n}\right) = y_n \left(1+h\lambda + \frac{1}{2}\left(h\lambda\right)^{2}\right)$$&lt;/span&gt; and this gives you your &lt;span class=&quot;math-container&quot;&gt;$R(z)$&lt;/span&gt; function as &lt;span class=&quot;math-container&quot;&gt;$$R(z) = 1+h\lambda + \frac{1}{2}\left(h\lambda\right)^{2}$$&lt;/span&gt; so your stability criterion in &lt;span class=&quot;math-container&quot;&gt;$(x,y)$&lt;/span&gt; form says: &lt;span class=&quot;math-container&quot;&gt;$$|1+x+\frac{x^2-y^2}{2}| \leq 1 \ \text{and}\ |y+xy| \leq 1$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solving this gives you a graph like below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/06rD8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/06rD8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="414" LastEditorUserId="414" LastEditDate="2019-05-03T12:50:25.260" LastActivityDate="2019-05-03T12:50:25.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32572" PostTypeId="1" AcceptedAnswerId="32574" CreationDate="2019-05-03T21:56:17.170" Score="0" ViewCount="129" Body="&lt;p&gt;After reading the matrix stability chapter (10) of Hirsch [1], I decided to dive in the reference list of the chapter. One of the papers [2], which is cited as reference shows an very interesting approach to stability analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper shows an stability evaluation of a real world flow solution of a NACA0012 airfoil with artificial dissipation and all the bells and witless which are expected to solve this type of flowfield.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, the matrix stability method, as explained in Hirsch and, apparently used in Ref. [2], is defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an discretized PDE, you end up with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac{d \bf U }{d t} = \bf S \bf U + \bf Q$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you compute the eigenvalues of the Jacobian &lt;span class=&quot;math-container&quot;&gt;$\bf S$&lt;/span&gt;, the maximum real part of the eigenvalues say to you if you spatial discretization scheme is stable or not. A positive real eigenvalue amplifies in time and ends up messing up with your stability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to study the stability of some very simple cases, however, I do not want to use the defined banded matrix definitions of the matrices &lt;span class=&quot;math-container&quot;&gt;$\bf S$&lt;/span&gt;. I want to be able to derive it numerically so, when the cases become less trivial, I will still be able to evaluate the stability of my case. The problem I am facing right now is how to derive the matrix &lt;span class=&quot;math-container&quot;&gt;$\bf S$&lt;/span&gt;, since it is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ \bf S = \frac{\partial\, \bf RHS}{\partial \, \bf U} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$\bf RHS$&lt;/span&gt; represent my spatial discretization scheme, and &lt;span class=&quot;math-container&quot;&gt;$\bf U$&lt;/span&gt; is my solution vector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a 1-D case, &lt;span class=&quot;math-container&quot;&gt;$\bf RHS$&lt;/span&gt; is a vector with the same size as &lt;span class=&quot;math-container&quot;&gt;$\bf U$&lt;/span&gt;. I tried to use simple finite differences, however, they do not seen to be robust enough. Or, at least, trying to compute the derivatives with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(1,n_points):&#xA;    drhs_du = (rhs[i+1] - rhs[i-1]) / 2.0*(u[i+1] - u[i-1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives me a whole lot of by-zero-divisions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw some references using Frechet derivatives to compute the &lt;span class=&quot;math-container&quot;&gt;$\bf S$&lt;/span&gt; Jacobian, but the authors who used it, which I know in person, also reported that this type of numerical derivative is quite troublesome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question, finally, is related to robust suggestions to compute this Jacobian numerically. Which algorithms would you use or, even, which derivative definition would you guys use ? Are there reliable high performance libraries with the algorithms implemented ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] Hirsch.C &quot;Numerical Computation of Internal and External Flows Vol.01&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] Eriksson.L.E. Rizzi &quot;A Computer Aided Analysis of the Convergence to Steady State of Discrete Approximations to the Euler Equations&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit # 01&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can the proper implementation of the method described in the second paragraph of the comment made below be translated as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def frhs(unn,unm1,dx):&#xA;    return (unn - unm1)/dx&#xA;&#xA;eps = 0.0001&#xA;&#xA;drhs_du = np.zeros((nx,nx))&#xA;&#xA;for i in range(1,nx-1):&#xA;    for j in range(1,nx-1):&#xA;        drhs_du[i,j] = (frhs(un[i] + eps,un[i-1] + eps,dx) - frhs(un[j],un[j-1],dx))/eps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;" OwnerUserId="21588" LastEditorUserId="21588" LastEditDate="2019-05-04T09:24:41.690" LastActivityDate="2019-05-04T09:24:41.690" Title="Numerical Stability of a Generalized Spatial Discretization Scheme" Tags="|finite-difference|eigenvalues|stability|jacobian|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32573" PostTypeId="2" ParentId="32569" CreationDate="2019-05-03T22:08:28.433" Score="1" Body="&lt;p&gt;You can try the von Neuman trace inequality&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\langle H,X\rangle\equiv\mathrm{trace}(H^{T}X)\le\sum_{i=1}^{d}\sigma_{i}(H)\sigma_{i}(X)\equiv\langle\sigma(H),\sigma(X)\rangle&#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\sigma_{i}(H)$&lt;/span&gt; is the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;-th largest singular value of &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;&#xA;and &lt;span class=&quot;math-container&quot;&gt;$\sigma(H)$&lt;/span&gt; is the length-&lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; vector of singular values in decreasing&#xA;order. From here, you can apply the Hölder inequality&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\langle\sigma(H),\sigma(X)\rangle\le\|\sigma(H)\|_{1}\|\sigma(X)\|_{\infty}\equiv\sigma_{\max}(X)\;\sum_{i=1}^{d}\sigma_{i}(H).&#xA;$$&lt;/span&gt;&#xA;If &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; is rank-deficient, then we expect the Nuclear norm &lt;span class=&quot;math-container&quot;&gt;$\sum_{i=1}^{d}\sigma_{i}(H)$&lt;/span&gt;&#xA;to be small, and the bound to be relatively tight. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the von Neuman inequality is one of the identities on the wiki page for &lt;a href=&quot;https://en.wikipedia.org/wiki/Trace_inequality&quot; rel=&quot;nofollow noreferrer&quot;&gt;trace inequalities&lt;/a&gt;. Also note that the nuclear norm is closely related to low-rank matrices. One can show that the nuclear norm is the convex envelope for the rank function over the Frobenius ball. See &lt;a href=&quot;https://doi.org/10.1137/070697835&quot; rel=&quot;nofollow noreferrer&quot;&gt;Recht, Fazel, Parrilo (2010)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11716" LastEditorUserId="11716" LastEditDate="2019-05-03T22:24:41.913" LastActivityDate="2019-05-03T22:24:41.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32574" PostTypeId="2" ParentId="32572" CreationDate="2019-05-03T23:01:10.010" Score="1" Body="&lt;p&gt;So it looks to me like there is a misunderstanding. your residual, or right hand side, presumably depends on multiple cells, so when building your jacobian, you should ask yourself how each entry in your residual depends on each cell. Now this should give you an nxn sparse matrix. If you in fact linearize this by hand, you should be able to store it in a vector that scales linearly with the number of cells. Given that you're 1d, I'm assuming it's size 3n, but that depends on your discretization. This is rather simple to do but rather tedious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another method is to calculate numerical derivatives, which you mentioned, and seem to be the source of your misunderstanding. It is not evaluated as you suggest, which is looking at different cells and taking a difference of the residuals and taking a difference of the cell values. Rather it is done through a cell by cell approach, where you perturb each cell's value individually and see how this affects the residual, and you take the difference between the residual from the perturbed and unperturbed states and divide by the perturbation. When doing this, you will discover that the stencil is pretty limited, and you should get a great deal of zeros as you fill out your matrix, and you will be able to store this sparse matrix in a vector as mentioned above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last possibility, is that you in fact do not need the jacobian matrix, you need the jacobian vector product. You can instead perturb the entire field by an infinitesimal complex perturbation in the direction of the vector you are multiplying the jacobian by, evaluate the residual of this perturbed state, and you will have obtained the frechet derivative without round-off error. &lt;/p&gt;&#xA;" OwnerUserId="9446" LastActivityDate="2019-05-03T23:01:10.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32576" PostTypeId="2" ParentId="11276" CreationDate="2019-05-04T18:39:42.410" Score="2" Body="&lt;p&gt;The library &lt;strong&gt;Boost.Numeric.uBlas&lt;/strong&gt; recently added a tensor extension which is shipped with Boost version 1.70. Please have a look at &lt;a href=&quot;https://github.com/boostorg/ublas&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/boostorg/ublas&lt;/a&gt;. It provides standard matrix and tensor operations with runtime-variable order (number of dimensions), dimensions for the first- and last-order storage formats (column- and row-major). You can also easily use the Einstein summation convention to express aribtrary tensor multiplications. Boost.Numeric.uBlas is header only and easy to integrate into existing projects.&lt;/p&gt;&#xA;" OwnerUserId="31664" LastActivityDate="2019-05-04T18:39:42.410" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32577" PostTypeId="1" CreationDate="2019-05-04T19:46:15.703" Score="2" ViewCount="2339" Body="&lt;p&gt;I'm trying to solve a Schrodinger equation of the form &lt;span class=&quot;math-container&quot;&gt;$i\frac{\partial}{\partial t}\psi=-\frac{\partial^2}{\partial x^2}\psi + (V(x)+\alpha|\psi|^2)\psi$&lt;/span&gt; using the split-step Fourier method implemented via MatLab code. In order to make sure it works, I'm testing my code with &lt;span class=&quot;math-container&quot;&gt;$V(x)=x^2,\alpha=0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\psi=Ce^{-x^2/2}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; is the normalization coefficient (it's set to 1 in this code because it shouldn't matter for testing purposes). My MatLab code is as follows:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;N = 100000;                         % Number of Fourier mode&#xA;dt = .001;                          % Time step&#xA;tfinal = 5;                         % Final time&#xA;M = round(tfinal/dt);               % Total number of time steps&#xA;L = 5;                              % Total space length&#xA;h = L/N;                            % Space step&#xA;n =( -N/2:N/2-1);                   % Indices&#xA;x = 2*n*h;                          % Grid points&#xA;u_i = exp(-((x-0).^2)/2);           % Intial pulse&#xA;u = u_i;                            % Make a duplicate of the initial pulse in order to compare at the end.&#xA;k = 2*pi/N * n;                     % wavenumber values.&#xA;epsilon = 0;                        % nonlinear coefficient&#xA;optical_potential=.5*(x-0).^2;&#xA;&#xA;figure(1);                          % Plotting the probability function of the initial wavefunction (not normalised)&#xA;plot(x,abs(u_i).^2);&#xA;hold on;&#xA;plot(x,optical_potential);&#xA;hold off;&#xA;&#xA;&#xA;for m = 1:1:M/2 % Start time loop (M/2 since each loop is 2 time steps)&#xA;    c = (fft(u));                   % Half time-step linear propagation&#xA;    c = exp(-dt/2*1i*k.^2).*c;&#xA;    u = ifft(c);                    % Full time-step nonlinear propagation&#xA;    u = exp(dt*1i*(optical_potential + epsilon *(abs(u).^2))).*u; &#xA;    c = (fft(u));                   % Half time-step linear propagation&#xA;    c = exp(-dt/2*1i*k.^2).*c;&#xA;    u = ifft(c); &#xA;end&#xA;&#xA;u_out=u; &#xA;figure(2);                          % Plotting the probability function of the final wavefunction (not normalised)&#xA;plot(x,abs(u_out).^2);&#xA;hold on;&#xA;plot(x,optical_potential);&#xA;hold off;&#xA;&#xA;error_percent=sum(abs(((abs(u_i) - abs(u_out)))))/sum(abs((abs(u_i))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The final and initial plots should be the same since &lt;span class=&quot;math-container&quot;&gt;$\psi$&lt;/span&gt; is an eigenstate of the potential (harmonic oscillator).&lt;/p&gt;&#xA;&lt;p&gt;My question is:&#xA;What is the correct value for &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;?&#xA;I'm fairly certain what I currently have as my &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; value is incorrect, despite it giving the correct answer within an error on the magnitude of 1e-13. I've tried a few, and a few give the correct answer within reasonable error, so I'm not sure which is the actual correct value.&lt;/p&gt;&#xA;" OwnerUserId="31666" LastEditorUserId="22974" LastEditDate="2023-07-15T17:07:35.090" LastActivityDate="2023-07-15T17:07:35.090" Title="Split-step Fourier method applied on Schrodinger equation" Tags="|fourier-transform|operator-splitting|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="32578" PostTypeId="1" AcceptedAnswerId="32582" CreationDate="2019-05-05T02:48:57.597" Score="4" ViewCount="1388" Body="&lt;p&gt;If I have a real positive definite matrix &lt;span class=&quot;math-container&quot;&gt;$A\in\mathbb{R}^{n\times n}$&lt;/span&gt;, and denote its eigenvalues as &lt;span class=&quot;math-container&quot;&gt;$\lambda_1\leq \lambda_2 \leq ... \leq \lambda_n $&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Define the function as &lt;span class=&quot;math-container&quot;&gt;$f(A)=\sum_{i=1}^{k} \lambda_i$&lt;/span&gt; for a constant &lt;span class=&quot;math-container&quot;&gt;$k&amp;lt;n$&lt;/span&gt;. What do we know about the convexity of &lt;span class=&quot;math-container&quot;&gt;$f(A)$&lt;/span&gt;? Is it convex or concave? &lt;/p&gt;&#xA;" OwnerUserId="27919" LastEditorUserId="27919" LastEditDate="2019-05-05T02:58:10.750" LastActivityDate="2019-05-05T13:38:25.090" Title="Convexity of Sum of $k$-smallest Eigenvalue" Tags="|linear-algebra|optimization|matrix|eigenvalues|convex-optimization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32579" PostTypeId="2" ParentId="32578" CreationDate="2019-05-05T03:26:31.520" Score="0" Body="&lt;p&gt;cvxpy treats that function as being concave (&lt;a href=&quot;https://www.cvxpy.org/tutorial/functions/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="17088" LastActivityDate="2019-05-05T03:26:31.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32580" PostTypeId="2" ParentId="32517" CreationDate="2019-05-05T04:55:33.310" Score="1" Body="&lt;p&gt;Von Neumann (Fourier modes) stability analysis gives you only a sufficient condition for stability if you compare the amplifying coefficient &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; with 1. If you have amplifying coefficient bounded by &lt;span class=&quot;math-container&quot;&gt;$1 + C\tau$&lt;/span&gt;, then after making &lt;span class=&quot;math-container&quot;&gt;$\frac{T}{\tau}$&lt;/span&gt; time steps your error will be bounded by &lt;span class=&quot;math-container&quot;&gt;$(1 + C \tau)^{\frac{T}{\tau}}$&lt;/span&gt; which has a bounded limit when &lt;span class=&quot;math-container&quot;&gt;$\tau \rightarrow 0$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="21958" LastActivityDate="2019-05-05T04:55:33.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32582" PostTypeId="2" ParentId="32578" CreationDate="2019-05-05T13:38:25.090" Score="9" Body="&lt;p&gt;Given &lt;span class=&quot;math-container&quot;&gt;$A \in {\bf S}^n$&lt;/span&gt; (a positive definite matrix) with eigenvalues &lt;span class=&quot;math-container&quot;&gt;$\lambda_1 \leq \lambda_2 \leq \ldots \leq \lambda_n $&lt;/span&gt;, then:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\displaystyle f_k(A)=\sum_{i=1}^{k} \lambda_i$&lt;/span&gt; is &lt;strong&gt;concave&lt;/strong&gt;. Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f_k(A) = \inf \left\{ {\bf tr}(V^T A V) | V \in {\bf R}^{n \times k}, V^T V = I \right\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This follows from the Poincare separation theorem (see e.g. &lt;a href=&quot;http://www.cambridge.org/9780521548236&quot; rel=&quot;noreferrer&quot;&gt;Horn and Johnson's &lt;em&gt;Matrix analysis&lt;/em&gt;, 2nd ed.&lt;/a&gt;, corollaries 4.3.37 and 4.3.39). &lt;span class=&quot;math-container&quot;&gt;$f_k$&lt;/span&gt; is the &lt;em&gt;pointwise infimum&lt;/em&gt; of a family of &lt;em&gt;linear&lt;/em&gt; functions &lt;span class=&quot;math-container&quot;&gt;${\bf tr}(V^T A V)$&lt;/span&gt;, hence it is concave (&lt;a href=&quot;https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf&quot; rel=&quot;noreferrer&quot;&gt;Boyd and Vandenberghe&lt;/a&gt;, section 3.2.3).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\displaystyle g_k(A)=\sum_{i=n-k+1}^{n} \lambda_i$&lt;/span&gt; is &lt;strong&gt;convex&lt;/strong&gt;. Again, we can show that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$g_k(A)=\sum_{i=n-k+1}^{n} \lambda_i(A) = \sup \left\{ {\bf tr}(V^T A V) | V \in {\bf R}^{n \times k}, V^T V = I \right\}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g_k$&lt;/span&gt; is the &lt;em&gt;pointwise supremum&lt;/em&gt; of a family of &lt;em&gt;linear&lt;/em&gt; functions &lt;span class=&quot;math-container&quot;&gt;${\bf tr}(V^T A V)$&lt;/span&gt;, hence it is convex (&lt;a href=&quot;https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf&quot; rel=&quot;noreferrer&quot;&gt;Boyd and Vandenberghe&lt;/a&gt;, section 3.2.3).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17294" LastActivityDate="2019-05-05T13:38:25.090" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32584" PostTypeId="2" ParentId="31432" CreationDate="2019-05-05T17:38:30.760" Score="2" Body="&lt;p&gt;To detect if a matrix is ill-conditioned you can check the &lt;a href=&quot;https://en.wikipedia.org/wiki/Condition_number#Matrices&quot; rel=&quot;nofollow noreferrer&quot;&gt;condition number&lt;/a&gt; defined, for the matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;&#xA;as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;k(A) = ||A|| \, ||A^{-1}||&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For norm 2 this is equal to the ratio of singular values:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;k(A) = \frac{\sigma_{max}(A)}{\sigma_{min}(A)} &#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Numerically there are also other methods to estimate &lt;span class=&quot;math-container&quot;&gt;$k(A)$&lt;/span&gt;. For more details see chapter 15 of [1], and [2] where you can find source code for different methods (Hager, from LINPACK, sampling) in  different languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To threat an ill-conditioned system there are two principal ways:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;preconditioning:&lt;/strong&gt; using this technique you obtain a system mathematically equivalent to the start situation, but with a better condition number. The methods depend on the structure of the matrix that you have, but you can see for example [3] for iterative methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;regularization:&lt;/strong&gt; here you obtain an approximation of the starter system, these methods work also for ill-posed problem. Example of techniques in this family are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://infolab.stanford.edu/pub/cstr/reports/na/m/86/36/NA-M-86-36.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;truncated svd&lt;/a&gt; essentially you discard the smallest singular values&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Tikhonov_regularization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tikhonov Regularization&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For more details and references see for example [4].&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;em&gt;Higham, Nicholas J.&lt;/em&gt;, &lt;a href=&quot;http://dx.doi.org/10.1137/1.9780898718027&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Accuracy and stability of numerical algorithms.&lt;/strong&gt;&lt;/a&gt;, Philadelphia, PA: SIAM. xxx, 680 p. (2002). &lt;a href=&quot;https://zbmath.org/?q=an:1011.65010&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL1011.65010&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] web page &lt;a href=&quot;https://people.sc.fsu.edu/~jburkardt/m_src/condition/condition.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matrix Condition Number Estimation &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[3] &lt;em&gt;Saad, Yousef&lt;/em&gt;, Iterative methods for sparse linear systems., Philadelphia, PA: SIAM Society for Industrial and Applied Mathematics. xviii, 528 p. (2003). &lt;a href=&quot;https://zbmath.org/?q=an:1031.65046&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZBL1031.65046&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[4] &lt;a href=&quot;https://www.math.unipd.it/~gazzola/gazzola_silvia_tesi.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;REGULARIZATION TECHNIQUESBASED ON KRYLOV SUBSPACE METHODSFOR ILL-POSED LINEAR SYSTEMS&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="22025" LastEditorUserId="414" LastEditDate="2019-05-06T13:17:53.940" LastActivityDate="2019-05-06T13:17:53.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32585" PostTypeId="2" ParentId="29377" CreationDate="2019-05-05T18:08:17.853" Score="0" Body="&lt;p&gt;I am not convinced that your code demonstrates anything fundamental about the various quadrature rules and how well they do against noise and fine structure, and believe it likely that if you chose various different fines structure you would find something different. Here's the theorem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;No quadrature method can give low absolute or relative error against a function with unbounded total variation. In a floating point system with unit roundoff &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt;, we have the estimate&lt;/em&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left| \int_{a}^{b} f \, \mathrm{d}x - \hat{Q}[\hat{f}] \right| \le \left| \int_{a}^{b} f \, \mathrm{d}x - Q[f] \right| + \mu\left[ 4\int_{a}^{b} |f| \, \mathrm{d}x +  \int_{a}^{b} |xf'| \, \mathrm{d}x \right] &#xA;$$&lt;/span&gt;&#xA;where &lt;span class=&quot;math-container&quot;&gt;$\hat{Q}$&lt;/span&gt; is the quadrature sum acting on the numerical implementation &lt;span class=&quot;math-container&quot;&gt;$\hat{f}$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Proof: Let the quadrature nodes be &lt;span class=&quot;math-container&quot;&gt;$\{x_i\}_{i=0}^{n-1}$&lt;/span&gt; and the (non-negative) quadrature weights be &lt;span class=&quot;math-container&quot;&gt;$\{w_i\}_{i=0}^{n-1}$&lt;/span&gt; and denote their floating point approximations by &lt;span class=&quot;math-container&quot;&gt;$\hat{w}_{i}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\hat{x}_i$&lt;/span&gt;. Assume that &lt;span class=&quot;math-container&quot;&gt;$\hat{f}$&lt;/span&gt; satisfies &lt;span class=&quot;math-container&quot;&gt;$\hat{f}(x) = f(x)(1+2\delta)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$|\delta| \le \mu$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; is the unit roundoff. Then&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;\hat{Q}[\hat{f}] &#xA;&amp;amp;= \sum_{i=0}^{n-1} \hat{w}_i \otimes \hat{f}(\hat{x}_i) \\&#xA;&amp;amp;= \sum_{i=0}^{n-1} w_i (1+\delta^w_i)f(x_i + \delta_i^x x_i)(1+2\delta_i^{f})(1+\delta_i^{*}) \\&#xA;&amp;amp;\approx \sum_{i=0}^{n-1} w_i \left[f(x_i)+ \delta_i^x x_i f'(x_i) \right] (1+\delta^w_i + 2\delta_i^{f} + \delta_i^*) \\&#xA;&amp;amp;\approx \sum_{i=0}^{n-1} w_i f(x_i) + \sum_{i=0}^{n-1}\delta_i^x w_i x_i f'(x_i)  + w_i f(x_i) (\delta^w_i + 2\delta_i^{f} + \delta_i^*) \\&#xA;\end{align*}&lt;/span&gt;&#xA;so that&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{align*}&#xA;|\hat{Q}[\hat{f}] - Q[f]| &#xA;&amp;amp;\le \mu \sum_{i=0}^{n-1}w_i(|x_i f'(x_i)|  + 4|f(x_i)|) \\&#xA;&amp;amp;\approx 4\mu \int |f| \, \mathrm{d}x + \mu \int |xf'| \, \mathrm{d}x&#xA;\end{align*}&lt;/span&gt;&#xA;This assumes that the sum is computed without error; multiply by &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; to drop that assumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mutatis mutandis you can also show that the result holds in fixed point arithmetic.&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2019-05-06T14:24:18.070" LastActivityDate="2019-05-06T14:24:18.070" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="32586" PostTypeId="2" ParentId="32557" CreationDate="2019-05-05T18:36:01.890" Score="3" Body="&lt;p&gt;I don't think Octave's syms package supports this kind of matrix algebra operations, but you can do that in &lt;a href=&quot;https://mathematica.stackexchange.com/questions/3242/can-mathematica-do-symbolic-linear-algebra&quot;&gt;Mathematica&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: and also in &lt;a href=&quot;https://docs.sympy.org/latest/modules/matrices/expressions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sympy&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4405" LastEditorUserId="4405" LastEditDate="2019-05-06T06:15:04.140" LastActivityDate="2019-05-06T06:15:04.140" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32587" PostTypeId="1" CreationDate="2019-05-05T22:22:00.653" Score="3" ViewCount="321" Body="&lt;p&gt;I have a function &lt;span class=&quot;math-container&quot;&gt;$F(\vec x)$&lt;/span&gt; of many variables (let's say in the order of hundreds of thousands). I need to compute the determinant of the Hessian matrix at the point &lt;span class=&quot;math-container&quot;&gt;$x_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to compute some approximation of it without explicitly computing the determinant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to compute this value because it appears in the formula I want to use to compute a rate in a transition. There is no way to avoid to compute this quantity without renounce to use this theory of transitions and developing another one by myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This function is the discretized version of a functional, this is the reason for the great number of variables I am dealing with. &lt;/p&gt;&#xA;" OwnerUserId="30502" LastEditorUserId="20688" LastEditDate="2019-05-06T10:50:19.737" LastActivityDate="2019-05-06T12:36:59.297" Title="How to compute the determinant of Hessian of a multivariable function?" Tags="|linear-algebra|optimization|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="32588" PostTypeId="2" ParentId="19685" CreationDate="2019-05-06T04:11:07.480" Score="0" Body="&lt;p&gt;The constant pi is only a constant in Euclidean space. The value of pi is different in other geometries. The parameter pi changes the probability mass under the Gaussian. The parameter pi is used to normalize the probabilities. I'm just starting to study this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I concluded some time ago that the space changes from hyperbolic to Euclidean to spherical as the number of sigmas go up. I was happy to run across a discussion of circles in each space and pi as a function of Lp spaces via the parameter p. &lt;/p&gt;&#xA;" OwnerUserId="31675" LastActivityDate="2019-05-06T04:11:07.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32590" PostTypeId="1" CreationDate="2019-05-06T11:22:57.487" Score="1" ViewCount="54" Body="&lt;p&gt;I need you to help me understand the Singular Spectrum Analysis algorithm. I already read a lot of articles about the subject but they never answered my questions like what is the mathematical reason for embedding the time series into a trajectory matrix and why applying the SVD gives us access to such trend and periodic and noise functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, a lot of people compare SSA to PCA for time series but one could easily explain PCA by saying that we want to find the most relevant direction for explaining the variance of the dataset and thus we are aiming to maximize the variance of the projection of the data on this particular direction we are looking for leading to this well-known optimization and eigenvector and diagonalization problem. But I am absolutely unable to find such an explanation for the SSA algorithm even by trying to explain it myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if someone could help me with that I would really appreciate, in fact, it is really important for me to understand deeply how the things work in order to, for instance, understand what window length to choose or how the eigenvalues are related to the importance of the principal components.&lt;/p&gt;&#xA;" OwnerUserId="31676" LastEditorUserId="9667" LastEditDate="2019-05-06T15:21:29.470" LastActivityDate="2019-05-06T15:21:29.470" Title="Singular Spectrum Analysis Explanation" Tags="|numerics|machine-learning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32591" PostTypeId="2" ParentId="32587" CreationDate="2019-05-06T12:36:59.297" Score="3" Body="&lt;p&gt;I would place a wild guess here as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The derivations in the omitted reference used determinants for some theoretical derivations. While determinants are a very useful tool to prove certain things, they are a very lousy tool from the computational science perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the &lt;em&gt;theory of transitions&lt;/em&gt; was developed (which I am not familiar with) to enable numerical computations (possibly) at a large scale. This theory would provide a method to compute the required quantities without the determinant of the Hessian; therefore, leading to a potentially accurate and scalable scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be able to use determinants to verify your implementation at a very small scale (say 3-4 variables, and even then you might end up having trouble due to floating-point arithmetic), but going for large-scale with determinants is a no-go.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-06T12:36:59.297" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32593" PostTypeId="1" CreationDate="2019-05-06T17:33:06.963" Score="0" ViewCount="214" Body="&lt;p&gt;I am running into a strange problem when solving the 2D compressible Euler equations on a inclined wedge. To elaborate, my top boundary condition seems to emitting some type of instability.  I have tried using Neumann BCs for the u,v velocities, but the issue still persists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using a 2nd-order finite volume method on a curvilinear mesh (structured grid).  My flux scheme is the Roe scheme with the Harten entropy fix.  I have attached a movie that illustrates the problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was curious if someone has any thoughts.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/64azs.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/64azs.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/sev3o7olhqqx7h7/animation.mp4?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/sev3o7olhqqx7h7/animation.mp4?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="21337" LastActivityDate="2019-05-06T17:33:06.963" Title="Double mach reflection at a inclined wedge" Tags="|finite-difference|fluid-dynamics|computational-physics|finite-volume|navier-stokes|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32594" PostTypeId="6" CreationDate="2019-05-06T20:09:39.347" Score="0" Body="&lt;p&gt;I would like to nominate myself as a candidate for moderator of scicomp.  I’ve been a moderator here for several years and would love the honor to continue serving the community.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="549" LastEditDate="2019-05-06T20:09:39.347" LastActivityDate="2019-05-06T20:09:39.347" CommentCount="1" CommunityOwnedDate="2019-05-06T20:09:39.347" ContentLicense="CC BY-SA 4.0" />
  <row Id="32595" PostTypeId="6" CreationDate="2019-05-06T22:51:53.033" Score="0" Body="&lt;p&gt;I've been an active member of Computational Science SE for the last 3 years. What started as a simple &quot;&lt;em&gt;I know how to answer this question&lt;/em&gt;&quot;, turned into visiting CompSci SE multiple times a day for editing, reviewing, reading, and answering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why I might be a good moderator?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I enjoy when things are organized. I don't shy away from editing posts (including typing up long equations), as well as proper tagging/retagging&lt;/li&gt;&#xA;&lt;li&gt;I try to help in making the questions/answers better so that they serve the OP and the community&lt;/li&gt;&#xA;&lt;li&gt;I try to limit my direct intervention into the content whenever possible&lt;/li&gt;&#xA;&lt;li&gt;I am an active member regularly answering questions and participating in the discussions (unfortunately, most of the time in the comments rather than in the chat)&lt;/li&gt;&#xA;&lt;li&gt;I visit the website very often as I am always striving for new information, and reading\answering some people's questions is a good way to improve&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I would have to improve in&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Meta participation&lt;/li&gt;&#xA;&lt;li&gt;Strategies to mitigate conflicts&lt;/li&gt;&#xA;&lt;li&gt;Commenting on my actions/intentions wherever applicable&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'd be happy to serve as a moderator for the community I'm already spending a large portion of my life in.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2019-05-06T22:51:53.033" LastActivityDate="2019-05-06T22:51:53.033" CommentCount="1" CommunityOwnedDate="2019-05-06T22:51:53.033" ContentLicense="CC BY-SA 4.0" />
  <row Id="32596" PostTypeId="1" AcceptedAnswerId="32600" CreationDate="2019-05-07T09:21:04.473" Score="0" ViewCount="424" Body="&lt;p&gt;I have this non-linear partial differential equation.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\frac{\partial C}{\partial t}=\left(\frac{\partial C}{\partial x}\right)^2+C\frac{\partial^2 C}{\partial x^2}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use the finite difference method to solve it either with the implicit method or the Crank-Nicolson method, witch I have done with linear PDE's, but how is this done with non-linear PDE's&lt;/p&gt;&#xA;" OwnerUserId="31672" LastActivityDate="2019-05-07T17:16:27.043" Title="Numerically solving a non-linear PDE" Tags="|finite-difference|nonlinear-equations|parabolic-pde|diffusion|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32597" PostTypeId="2" ParentId="30806" CreationDate="2019-05-07T10:39:04.937" Score="1" Body="&lt;p&gt;OP's comment on OP: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;yes,numbers are intergers&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For arbitrarily large integer calculations, MAPLE is an option worth trying. it has very efficient implementations for a number of such calculations.&lt;/p&gt;&#xA;" OwnerUserId="16756" LastActivityDate="2019-05-07T10:39:04.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32598" PostTypeId="1" CreationDate="2019-05-07T12:13:27.357" Score="5" ViewCount="790" Body="&lt;p&gt;Hoping I'm not misunderstanding the concept here, but it is my understanding that Voronoi Diagrams and Delaunay Tesselations are 'dual' to one another, owing to the fact that each' solution makes computational of the other's solution trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm looking to understand is, why? I mean, it obviously happens, but is there some kind of mathematical proof or principle that justifies that convenient coincidence? Or is it just that, a coincidence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose, the voronoi quality of 'each vertex is equidistant from the 3 closest points' means that any matching triangulation is a delaunay triangulation (there is no point closer than the closest points), but I am hoping for something more... in-depth? Is there such a thing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify: the question is not about what them being dual means, like the suggested duplicate. The question is why they behave as duals. &lt;/p&gt;&#xA;" OwnerUserId="31681" LastEditorUserId="31681" LastEditDate="2019-05-07T15:36:28.797" LastActivityDate="2020-04-15T19:09:21.013" Title="How can one prove the duality of Voronoi and Delaunay?" Tags="|computational-geometry|delaunay-triangulation|voronoi-diagrams|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="32600" PostTypeId="2" ParentId="32596" CreationDate="2019-05-07T17:16:27.043" Score="5" Body="&lt;p&gt;First off, the PDE can be rewritten instead as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial t} = \frac{\partial}{\partial x}C\frac{\partial C}{\partial x}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or, by applying the product rule in reverse again, as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial t} = \frac{1}{2}\frac{\partial^2}{\partial x^2}C^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This equation is often referred to as the &lt;em&gt;porous medium equation&lt;/em&gt; or the &lt;em&gt;slow diffusion&lt;/em&gt; equation and it's got some fascinating properties.&#xA;The regular old linear heat equation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial C}{\partial t} = \frac{\partial}{\partial x}D\frac{\partial C}{\partial x}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with conductivity &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; has the property that disturbances propagate with infinite speed.&#xA;The slow diffusion equation, on the other hand, has a well-defined front that propagates with finite speed, which is very unusual.&#xA;This one of the simplest examples of a &lt;em&gt;free boundary problem&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways to discretize nonlinear partial differential equations.&#xA;I'll describe a very blunt and simple approach that should hopefully get you started; it's based on using the first form of the slow diffusion equation that I wrote down.&#xA;The idea is to pretend like it's the linear diffusion equation, but to then take the diffusion coefficient to be the value of &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; from the previous timestep.&#xA;Let&#xA;&lt;span class=&quot;math-container&quot;&gt;$$C_k^n = C(k\cdot\delta x, n\cdot\delta t)$$&lt;/span&gt;&#xA;be the values of the solution at time step &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; and and spatial point &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and let &lt;span class=&quot;math-container&quot;&gt;$D_{k+\frac{1}{2}}$&lt;/span&gt; be the value of the diffusion coefficient in the interval &lt;span class=&quot;math-container&quot;&gt;$[k\cdot\delta x, (k + 1)\delta x]$&lt;/span&gt;.&#xA;An implicit Euler discretization of the linear diffusion equation would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{C^{n + 1}_k - C^n_k}{\delta t} = \frac{D_{k + \frac{1}{2}}\frac{C^{n + 1}_{k + 1} - C^{n + 1}_k}{\delta x} - D_{k - \frac{1}{2}}\frac{C^{n + 1}_k - C^{n + 1}_{k - 1}}{\delta x}}{\delta x}.$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then take the diffusion coefficient in each interval as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$D_{k + \frac{1}{2}} = \frac{C_{k + 1}^n + C_k^n}{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;using the concentration from the previous timestep to approximate the nonlinearity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a more accurate numerical solver, you might want to look into implementing &lt;a href=&quot;https://en.wikipedia.org/wiki/Newton%27s_method_in_optimization#Higher_dimensions&quot; rel=&quot;noreferrer&quot;&gt;Newton's method&lt;/a&gt;.&#xA;That will be more involved though and it's better to try simple approaches first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An important thing to account for is that parabolic PDE have a &lt;em&gt;maximum principle&lt;/em&gt; -- a solution that is initially positive will stay positive.&#xA;The implicit Euler discretization has its own discrete maximum principle, whereas the Crank-Nicholson method does not.&#xA;Despite the fact that the Crank-Nicholson method is formally of higher order in &lt;span class=&quot;math-container&quot;&gt;$\delta t$&lt;/span&gt;, it could actually be much worse in practice for this reason.&lt;/p&gt;&#xA;" OwnerUserId="3481" LastActivityDate="2019-05-07T17:16:27.043" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32602" PostTypeId="1" AcceptedAnswerId="32603" CreationDate="2019-05-08T01:11:09.847" Score="1" ViewCount="81" Body="&lt;p&gt;In Higham's &lt;em&gt;Accuracy and Stability of Numerical Algorithms&lt;/em&gt;, Chapter 15, algorithm 15.3 and 15.4: The topic is ostensibly condition number estimation, but these algorithms show how to compute &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; such that &lt;span class=&quot;math-container&quot;&gt;$\gamma &amp;lt; \left\|A\right\|_{1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I have a matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, I already know how to compute &lt;span class=&quot;math-container&quot;&gt;$\left\|A \right\|_1$&lt;/span&gt;, just compute&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\left\| A \right\|_1 = \max_{j} \sum_{i} |a_{ij}|&#xA;$$&lt;/span&gt;&#xA;That's a quick &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt; flops. So the hard part is computation of &lt;span class=&quot;math-container&quot;&gt;$\left\|A^{-1}\right\|_1$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, so maybe I should read it as &lt;span class=&quot;math-container&quot;&gt;$A \mapsto A^{-1}$&lt;/span&gt;. Then algorithm 15.3 tells me to compute &lt;span class=&quot;math-container&quot;&gt;$y = A^{-1}x$&lt;/span&gt;, or in other words solve &lt;span class=&quot;math-container&quot;&gt;$Ay = x$&lt;/span&gt;. This isn't cheaper than solving the linear system. Is it assumed that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is already decomposed into triangular factors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="4392" LastEditorUserId="4392" LastEditDate="2019-05-08T01:17:34.157" LastActivityDate="2019-05-08T01:17:34.157" Title="Missing something fundamental about condition number estimation" Tags="|condition-number|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32603" PostTypeId="2" ParentId="32602" CreationDate="2019-05-08T01:14:49.217" Score="5" Body="&lt;p&gt;This algorithm is most useful for two situations, which are related to each other in practice:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You don't know the matrix entries explicitly, but instead can only compute matrix-vector products with the matrix (often called &quot;matrix-free&quot;)&lt;/li&gt;&#xA;&lt;li&gt;You want the 1-norm for the &lt;em&gt;inverse&lt;/em&gt; of a matrix. The inverse of a sparse matrix is typically dense, and since sparse matrices tend to be gigantic this turns the &lt;span class=&quot;math-container&quot;&gt;$O(n^2)$&lt;/span&gt; algorithm you just described into an intractable computation. Generally this means you precompute a factorization and use the corresponding solve as the matrix-free evaluation of the inverse operator. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="547" LastActivityDate="2019-05-08T01:14:49.217" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32606" PostTypeId="1" AcceptedAnswerId="32616" CreationDate="2019-05-09T01:04:00.127" Score="2" ViewCount="231" Body="&lt;p&gt;I have a matrix &lt;span class=&quot;math-container&quot;&gt;$A\in\{0,1\}^{d\times n}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$rank(A)=d,d&amp;lt;n$&lt;/span&gt;, and another matrix &lt;span class=&quot;math-container&quot;&gt;$X\in \mathbb{R}^{d\times n}$&lt;/span&gt;, but I do not know the rank of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. What can we say about the rank of their Hadamard product &lt;span class=&quot;math-container&quot;&gt;$rank(A\odot X)$&lt;/span&gt;？&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And what if &lt;span class=&quot;math-container&quot;&gt;$rank(A)=k, k&amp;lt;d$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="27919" LastActivityDate="2019-05-09T18:08:59.200" Title="Rank of Hadamard Product with Masked Matrix" Tags="|linear-algebra|matrix|rank|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32607" PostTypeId="1" AcceptedAnswerId="32612" CreationDate="2019-05-09T07:39:12.603" Score="4" ViewCount="241" Body="&lt;p&gt;When gmsh uses higher-order tetrahedral elements, there is an underlying Lagrange basis used to specify the map from reference space to the element. I'm trying to load a gmsh mesh of 3rd degree tetrahedral elements, but I can't seem to find any precise documentation on the location of the nodes that define the Lagrange basis used to define the map from reference space to world space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The gmsh documentation&lt;/a&gt; indicates the ordering of the nodes, but it is not particularly precise (for instance the tetrahedral examples therein do not even show how the face nodes would be ordered). Is there some way I can answer the question, for any tetrahedral element of order &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, the &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;th node corresponds to coordinate &lt;span class=&quot;math-container&quot;&gt;$(x,y,z)$&lt;/span&gt; on the reference element? Is there some undocumented gmsh library function that I can call?&lt;/p&gt;&#xA;" OwnerUserId="24862" LastEditorUserId="20688" LastEditDate="2019-05-09T17:48:54.310" LastActivityDate="2019-05-09T17:48:54.310" Title="Definition of Lagrange nodes in Gmsh" Tags="|mesh-generation|basis-set|gmsh|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32609" PostTypeId="1" CreationDate="2019-05-09T12:04:30.203" Score="0" ViewCount="1913" Body="&lt;p&gt;Is the theory of 3d finite element method just an assembly of 2d finite element analysis by putting planes on top of each other, or, a much more comple and different theory applies for 3d, with respect to 2d?&#xA;For example I am reading a dam design manual from 1970s, where computers were low power and FEM was gaining new acceptance. It says that 3d analysis can be done by making 2d and then stacking planes on top of each other. But now, after 40 years, is it still the same logic for 3d, or now 3d has its own theory itself, by utilizing much better computer power?&lt;/p&gt;&#xA;" OwnerUserId="31695" LastActivityDate="2019-05-09T17:16:45.190" Title="3d vs 2d finite element method" Tags="|finite-element|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32610" PostTypeId="2" ParentId="32609" CreationDate="2019-05-09T12:19:48.080" Score="1" Body="&lt;p&gt;The theory of 2D and 3D finite elements is quite similar. In a 2D setting you will chose a Triangulation of your domain (tetrahedral/hexahedral grid), then introduce proper base functions defined on those elements. In 2D FEM, you typically couple your cells degrees of freedom with those neighbouring it. So if you have a rectangular 2D grid with one dof per cell, then that might couple with four other cells producing a line in your system matrix with five entries. In 3D your coupling increases, as your cell has more direct neighbours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you say &quot;Stacking 2D planes onto each other&quot;, you have to be careful to respect the coupling which will be in the vertical direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be instructional to start with 1D FEM. Do that discretization &quot;by hand&quot; and write down the resulting matrix. There are a couple of examples out there which have five elements or so and it can be done in approx. 30 min.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From there you may move on to higher dimensions. The differences and the similarities will become clear.&lt;/p&gt;&#xA;" OwnerUserId="28636" LastEditorUserId="393" LastEditDate="2019-05-09T16:33:55.970" LastActivityDate="2019-05-09T16:33:55.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32611" PostTypeId="1" CreationDate="2019-05-09T14:28:12.663" Score="3" ViewCount="907" Body="&lt;p&gt;I'm using BFGS to optimize a smooth but non-convex function &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; that is computed by a simulation. The simulation also gives me a semi-analytical gradient &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;, which is verified by the numerical gradient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; is non-convex and I am using a penalty function to enforce bound constraints, I do not expect to find a global minimum, but rather a local one where the gradient also happens to vanish. However, when BFGS &quot;converges&quot; (in terms of very small &lt;span class=&quot;math-container&quot;&gt;$\Delta x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Delta f$&lt;/span&gt;), &lt;span class=&quot;math-container&quot;&gt;$\Vert g \Vert_\infty$&lt;/span&gt; remains quite large. I can think of a few reasons for this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The penalty function is large close to the bounds -- this is not the case here though, as the penalty function is sufficiently scaled down at this point&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The downward slope of the gradient is intersecting with one of the boundaries -- BFGS can't move away from the boundary since that will increase the function, but also cannot approach it further, causing it to get stuck. This might be the case since the step size is virtually zero at &quot;convergence&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest any other reasons for this behavior, or a way to verify if (2) is indeed the cause?&lt;/p&gt;&#xA;" OwnerUserId="31697" LastActivityDate="2019-05-10T16:08:06.380" Title="Why would BFGS converge to a local minima of a non-convex function but maintain a large gradient?" Tags="|optimization|constrained-optimization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32612" PostTypeId="2" ParentId="32607" CreationDate="2019-05-09T15:01:40.600" Score="4" Body="&lt;p&gt;I think you've got slightly the wrong end of the stick from the documentation. As with a lot of other software in the area, GMSH started out with low order, hard coded numberings. These are the ones with the ASCII art representations, which only give first and second order numberings for tetrahedra (hence there &lt;em&gt;aren't&lt;/em&gt; any face nodes in the 4 node or 10 node &quot;low order&quot; versions). These aren't examples, they're the original canonical numberings from earlier versions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As things have progressed further, the developers have added support for a more arbitrary scheme for higher order elements, with a numbering scheme as &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#High_002dorder-elements&quot; rel=&quot;nofollow noreferrer&quot;&gt;described here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;9.2.2 High-order elements&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The node ordering of a higher order (possibly curved) element is compatible with the numbering of low order element (it is a generalization). We number nodes in the following order:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;the element principal or corner vertices; &lt;/li&gt;&#xA;  &lt;li&gt;the internal nodes for each edge;&lt;/li&gt;&#xA;  &lt;li&gt;the internal nodes for each face;&lt;/li&gt;&#xA;  &lt;li&gt;the volume internal nodes.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;The numbering for internal nodes is recursive, i.e. the numbering follows that of the nodes of an embedded edge/face/volume of lower order. The higher order nodes are assumed to be equispaced. Edges and faces are numbered following the lowest order template that generates a single high-order on this edge/face. Furthermore, an edge is oriented from the node with the lowest to the highest index. The orientation of a face is such that the computed normal points outward; the starting point is the node with the lowest index.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Although this text is rather dense, I believe it to end up as fairly unambiguous, provided you remember that everything is using a dictionary ordering based on the assigned principal vertex numbers.&lt;/p&gt;&#xA;" OwnerUserId="19263" LastActivityDate="2019-05-09T15:01:40.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32613" PostTypeId="2" ParentId="32611" CreationDate="2019-05-09T16:32:29.650" Score="7" Body="&lt;p&gt;For a bounds-constrained problem to minimize &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt; subject to &lt;span class=&quot;math-container&quot;&gt;$h(x)\ge 0$&lt;/span&gt;, there is no reason for &lt;span class=&quot;math-container&quot;&gt;$\nabla f(x^\ast)$&lt;/span&gt; to be small at the optimum &lt;span class=&quot;math-container&quot;&gt;$x^\ast$&lt;/span&gt;. All the theory guarantees is that &lt;span class=&quot;math-container&quot;&gt;$\nabla f(x^\ast) + \lambda^\ast \nabla h(x^\ast)=0$&lt;/span&gt; if the optimum is at a place where the bound is active. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, you are imposing the bound via a penalty approach, which I assume to mean that you are minimizing &lt;span class=&quot;math-container&quot;&gt;$F_\varepsilon(x)=f(x)-\frac{1}{\varepsilon}\ln(h(x))$&lt;/span&gt;. In that case, at the optimum, it is again not reasonable to expect that &lt;span class=&quot;math-container&quot;&gt;$\nabla f(x^\ast)$&lt;/span&gt; is small. All you know is that &lt;span class=&quot;math-container&quot;&gt;$\nabla F_\varepsilon(x^\ast)$&lt;/span&gt; is small -- something you can easily check.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2019-05-10T16:08:06.380" LastActivityDate="2019-05-10T16:08:06.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32615" PostTypeId="2" ParentId="32609" CreationDate="2019-05-09T17:16:45.190" Score="2" Body="&lt;p&gt;In solid mechanics, the essential difference between 3D and 2D FEM is that 3D FEM approximates (&quot;discretizes&quot;) &#xA;the 3D elasticity equations and 2D FEM is based on the elasticity equations simplified to two dimensions. So understanding the applicability of 2D or 3D FEM really is about understanding the assumptions of the two elasticity theories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never seen structural FEM explained in terms of &quot;putting planes on top of each&#xA;other.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest you take a look at this site: &lt;a href=&quot;http://solidmechanics.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;solidmechanics.org&lt;/a&gt;. It begins with a thorough presentation of the elasticity equations, includes an introduction to the finite element method, and includes some sample codes for 2D and 3D FEM.&lt;/p&gt;&#xA;" OwnerUserId="3441" LastActivityDate="2019-05-09T17:16:45.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32616" PostTypeId="2" ParentId="32606" CreationDate="2019-05-09T18:08:59.200" Score="2" Body="&lt;p&gt;For Hadamard product, you have (&lt;a href=&quot;https://math.stackexchange.com/q/812556/332683&quot;&gt;source&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\text{rank}(A\odot X) \leq \text{rank}(A) \text{rank}(X)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is as tight as you can go even with knowing &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(X)$&lt;/span&gt;.&#xA;Without knowing anything about &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, there is certainly nothing you can say. (See the examples that nicoguaro and Wolfgang Bangerth brought in the comments section).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, imagine you know &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(X)=r$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(X)\geq1$&lt;/span&gt; (I think the &lt;em&gt;without loss of generality&lt;/em&gt; clause is applicable here).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(A)=d$&lt;/span&gt;, then the inequality above is totally useless, as it would not bound anything: &lt;span class=&quot;math-container&quot;&gt;$d\cdot r \geq d$&lt;/span&gt;, which is the same conclusion you would get from just looking at &lt;span class=&quot;math-container&quot;&gt;$A\odot X \in \mathbb{F}^{d\times n}, d&amp;lt;n$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;If &lt;span class=&quot;math-container&quot;&gt;$\text{rank}(A)=r&amp;lt;d$&lt;/span&gt;, depending on what you know about &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; can give you something, but still not too exciting.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hadamard products are tricky, and a lot of things one expects from them (based on prior knowledge of the regular matrix-matrix product) do not hold.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-09T18:08:59.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32617" PostTypeId="1" AcceptedAnswerId="32619" CreationDate="2019-05-09T18:38:26.610" Score="0" ViewCount="194" Body="&lt;p&gt;I am using PETSc and libmesh to solve a simple linear elastic problem with quite complicated geometry, using linear tetrahedral elements. I am always using the KSP CG as the solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that for certain meshes, the block-Jacobi (&lt;code&gt;PCBJACOBI&lt;/code&gt;) preconditioner would fail to converge (&lt;code&gt;DIVERGED_INDEFINITE_PC&lt;/code&gt;) but Jacobi (&lt;code&gt;PCJACOBI&lt;/code&gt;) works fine. As a beginner in this field, this sounds very strange to me. Could anyone please help provide a hint?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I read on the PETSc website that both Jacobi and the block-Jacobi support parallel processing. Would that mean that it'd be safe for me just to stay with Jacobi, maybe with some loss in performance?&lt;/p&gt;&#xA;" OwnerUserId="5108" LastEditorUserId="20688" LastEditDate="2019-05-09T20:45:43.127" LastActivityDate="2019-05-10T03:45:15.827" Title="PCJACOBI works but the default PCBJACOBI failed in PETSc" Tags="|iterative-method|petsc|preconditioning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32618" PostTypeId="1" AcceptedAnswerId="32622" CreationDate="2019-05-09T19:29:24.247" Score="0" ViewCount="1342" Body="&lt;p&gt;I am trying to animate a plot of two distinct points (blue and green points) moving about the complex unit circle using Python's Matplotlib library. The problem I am having is that the animation does not remove and update the previous data points but rather sequentially smears it on the unit sphere as in the accompanying image. Hence the animation is just a smudging of the various data points as shown in the image. What I am trying to achieve is two distinct points moving about the unit circle as a function of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is the part of my code where I call 'animation.FuncAnimation' using data in arrays which I call 'A' and 'B'. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;##Python Code for Executing Animation##&#xA;import matplotlib.animation as animation&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;from pylab import *&#xA;&#xA;#Example Data&#xA;A = array([0., 0.03435915,  0.06328989, 0.0880305, 0.14199928, 0.2044361, 0.26287941,  0.32484623])&#xA;B = array([ 1.75, 1.71564086, 1.69358362, 1.68499179, 1.68255084, 1.67808712, 1.66169597,  1.64407287]) &#xA;&#xA;# Total time.&#xA;T = 1.0&#xA;# Number of steps.&#xA;NS = 100&#xA;# Time step size&#xA;dt = T/NS&#xA;t = np.linspace(0.0, NS*dt, NS+1)&#xA;&#xA;# So here are a few utility functions for multiplying scalars and vectors.&#xA;# a scalar times a vector returns a vector&#xA;def scale_vector(scale, vector):&#xA;  result = [0]*len(vector)&#xA;  for i in range(len(result)):&#xA;    result[i] = scale * vector[i]&#xA;  return result&#xA;&#xA;# dot product of two vectors = sum(x[0]*y[0] + ... + x[n-1]*y[n-1])&#xA;def vector_dot(vector1, vector2):&#xA;  result = 0&#xA;  for i in range(len(vector1)):&#xA;    result += vector1[i] * vector2[i]&#xA;  return result&#xA;&#xA;# return real part of a vector&#xA;def real_vector(vector):&#xA;  return map(lambda x: x.real, vector)&#xA;&#xA;# return imaginary part of a vector&#xA;def imag_vector(vector):&#xA;  return map(lambda x: x.imag, vector)&#xA;&#xA;## Creating complex unit circle&#xA;r = []&#xA;im = []&#xA;def main():&#xA;  # Generate numbers around the complex unit circle.&#xA;  N = 128&#xA;  theta = scale_vector(2*pi/N, range(N))&#xA;  exp_theta = map(lambda x: exp(1j * x), theta)&#xA;&#xA;  real_part = real_vector(exp_theta)&#xA;  imag_part = imag_vector(exp_theta)&#xA;&#xA;  r.append(real_part)&#xA;  im.append(imag_part)&#xA;&#xA;  # And wait until the user is done with it.&#xA;  done = raw_input(&quot;done? &quot;)&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;  main()&#xA;&#xA;#Form two arrays which have the real and imaginary components of the unit circle&#xA;r2 = r[0][:]&#xA;im2 = im[0][:]&#xA;&#xA;##Code for Animation##&#xA;Aan = np.zeros([len(A),2], float)&#xA;for i in range(2):&#xA;  for j in range(len(A)):&#xA;    if i == 0:&#xA;      Aan[j][i] = math.cos(A[j])&#xA;    elif i == 1:&#xA;      Aan[j][i] = math.sin(A[j])&#xA;&#xA;Ban = np.zeros([len(B),2], float)&#xA;for i in range(2):&#xA;  for j in range(len(B)):&#xA;    if i == 0:&#xA;      Ban[j][i] = math.cos(B[j])&#xA;    elif i == 1:&#xA;      Ban[j][i] = math.sin(B[j])&#xA;&#xA;##Plots and animation&#xA;fig = figure()&#xA;plt.title('Phase Space')&#xA;plt.xlabel('Re')&#xA;plt.ylabel('Im')&#xA;&#xA;#Plots complex unit circle&#xA;plot1 = plt.plot(r2,im2, color = 'g',alpha = 0.4)&#xA;&#xA;#Animation functions&#xA;def animate(i):&#xA;     plot(Aan[i, 0], Aan[i, 1], color='blue', marker= 'o') &#xA;     plot(Ban[i, 0], Ban[i, 1], color='orange', marker= 'o') &#xA;&#xA;ani = animation.FuncAnimation(fig, animate, interval=101)&#xA;&#xA;show()&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m1aRq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m1aRq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Can anyone advise on how this problem could be solved?&lt;/p&gt;&#xA;" OwnerUserId="26653" LastEditorUserId="414" LastEditDate="2019-05-10T08:51:58.420" LastActivityDate="2019-05-10T08:51:58.420" Title="Animation using matplotlib" Tags="|python|matplotlib|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32619" PostTypeId="2" ParentId="32617" CreationDate="2019-05-09T20:47:30.180" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCJACOBI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pcjacobi&lt;/code&gt;&lt;/a&gt; is a Jacobi preconditioner (also known is point-Jacobi preconditioner). For this preconditioner, you just take the diagonal elements of the matrix and &lt;a href=&quot;https://en.wikipedia.org/wiki/Preconditioner#Jacobi_(or_diagonal)_preconditioner&quot; rel=&quot;nofollow noreferrer&quot;&gt;invert them&lt;/a&gt; for the future use in the iterative solver.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCBJACOBI.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pcbjacobi&lt;/code&gt;&lt;/a&gt; would be a block-Jacobi preconditioner. For block-Jacobi, you don't take individual diagonal entries, but use small matrix blocks that are on the matrix diagonal. Now, you need to &quot;invert them&quot; whatever that means. A properly configured (block size) block-Jacobi preconditioner usually gives better convergence than point-Jacobi. However, preconditioning and iterative solvers are definitely connected to &lt;em&gt;dark arts&lt;/em&gt;, thus you always can find some counter-examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, in your case, probably the matrix does not have any zero diagonal entries; thus point-Jacobi works fine. And the way you configure block-Jacobi results in preconditioner featuring non-invertible (or poorly invertible) blocks, leading to the preconditioning problems. I suggest you take a look at the &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex7.c.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PETSc example on block-Jacobi&lt;/a&gt; to identify your particular issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding parallel processing: I would play more with your particular problem set, understand how different iterative solvers, preconditioners, and their parameters influence the convergence before evaluating the effect from parallelization. And ask it as a separate question later.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2019-05-10T03:45:15.827" LastActivityDate="2019-05-10T03:45:15.827" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32620" PostTypeId="2" ParentId="31447" CreationDate="2019-05-10T04:00:52.557" Score="1" Body="&lt;p&gt;I would second the opinion expressed in the context. For that small problem and limited usage, you don't need a library. Generation of a structured grid and retrieving points can be coded up in at most several screens of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would point out for you &lt;a href=&quot;http://viennagrid.sourceforge.net/73.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ViennaGrid&lt;/a&gt; library, which can be used and actually provides STL iterators. In addition, &lt;a href=&quot;http://viennagrid.sourceforge.net/viennagrid-manual-current.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation for ViennaGrid&lt;/a&gt; also contains some discussion on the alternatives (both for structured and unstructured meshes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, even a small ViennaGrid library would be an overkill for this task, in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-10T04:00:52.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32622" PostTypeId="2" ParentId="32618" CreationDate="2019-05-10T07:46:16.310" Score="1" Body="&lt;p&gt;There are two similar questions: &lt;a href=&quot;https://stackoverflow.com/questions/35521545/animation-moving-point-error&quot;&gt;Animation moving point error&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/9401658/how-to-animate-a-scatter-plot&quot;&gt;How to animate a scatter plot&lt;/a&gt;  on &lt;a href=&quot;https://stackoverflow.com&quot;&gt;StackOverflow&lt;/a&gt;. Both were answered with some example code for you to try. &lt;/p&gt;&#xA;" OwnerUserId="414" LastActivityDate="2019-05-10T07:46:16.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32623" PostTypeId="2" ParentId="8998" CreationDate="2019-05-10T18:51:44.583" Score="3" Body="&#xA;&#xA;&lt;p&gt;Here are my arguments for why functional programming &lt;em&gt;can&lt;/em&gt;, and &lt;em&gt;should&lt;/em&gt; be utilized for computational science. The benefits are vast, and the cons are quickly going away. In my mind there's only one con:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Con&lt;/strong&gt;: lack of language support in C / C++ / Fortran&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least in C++, this con is vanishing - as C++14/17 has added powerful facilities to support functional programming. You might need to write some library / support code yourself, but the language will be your friend. As an example, here is a (warning: plug) library that does immutable multi-dimensional arrays in C++: &lt;a href=&quot;https://github.com/jzrake/ndarray-v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jzrake/ndarray-v2&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, here is a link to a &lt;a href=&quot;https://www.manning.com/books/functional-programming-in-c-plus-plus&quot; rel=&quot;nofollow noreferrer&quot;&gt;good book&lt;/a&gt; on functional programming in C++, although it's not focused on science applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my summary of what I believe are the pro's:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Pros&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Correctness&lt;/li&gt;&#xA;&lt;li&gt;Understandability&lt;/li&gt;&#xA;&lt;li&gt;Performance&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In terms of &lt;strong&gt;correctness&lt;/strong&gt;, functional programs are manifestly &lt;em&gt;well-posed&lt;/em&gt;: they force you to properly define the minimal state of your physics variables, and the function that advances that state forward in time:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c++ prettyprint-override&quot;&gt;&lt;code&gt;int main()&#xA;{&#xA;    auto state = initial_condition();&#xA;&#xA;    while (should_continue(state))&#xA;    {&#xA;        state = advance(state);&#xA;        side_effects(state);&#xA;    }&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Solving a partial differential equation (or ODE) is perfect for functional programming; you're just applying a pure function (&lt;code&gt;advance&lt;/code&gt;) to the current solution to generate the next one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience, physics simulation software is by and large, burdened by poor &lt;em&gt;state management&lt;/em&gt;. Usually, each stage of the algorithm operates on some piece of a shared (effectively global) state. This makes it difficult, or even impossible, to ensure the correct order of operations, leaving the software vulnerable to bugs that can manifest as seg-faults, or worse, error terms that do not crash your code but silently compromise the integrity of its science output. Attempting to manage shared state in a physics simulation also inhibits multi-threading - which is a problem for the future, as supercomputers are moving toward higher core counts, and scaling with MPI often tops out at ~100k tasks. In contrast, functional programming makes shared-memory parallelism trivial, because of immutability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Performance&lt;/strong&gt; is also improved in functional programming due to the lazy evaluation of algorithms (in C++, this means generating many types at compile time - often one for each application of a function). But it reduces the overhead of memory accesses and allocations, as well as eliminating virtual dispatch - allowing the compiler to optimize an entire algorithm by seeing at once all the function objects comprising it. In practice, you'll experiment with different arrangements of the evaluation points (where the algorithm result is cached to a memory buffer) to optimize use of CPU vs. memory allocations. This is rather easy due to the high locality (see the example below) of the algorithm stages compared to what you'll typically see in a module or class-based code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Functional programs are easier to &lt;strong&gt;understand&lt;/strong&gt; insofar as they trivialize the physics state. That is not to say their syntax is readily understandable by all of your colleagues! Authors should be careful to use well-named functions, and researchers in general should get accustomed to seeing algorithms expressed functionally rather than procedurally. I'll admit that the absence of control structures can be off-putting to some, but I don't think that should stop us from going into the future able to do better quality science on computers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is a sample &lt;code&gt;advance&lt;/code&gt; function, adapted from a finite-volume code using the &lt;code&gt;ndarray-v2&lt;/code&gt; package. Note the &lt;code&gt;to_shared&lt;/code&gt; operators - these are the evaluation points I was alluding to earlier.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c++ prettyprint-override&quot;&gt;&lt;code&gt;auto advance(const solution_state_t&amp;amp; state)&#xA;{&#xA;    auto dt = determine_time_step_size(state);&#xA;    auto du = state.u&#xA;    | divide(state.vertices | volume_from_vertices)&#xA;    | nd::map(recover_primitive)&#xA;    | extrapolate_boundary_on_axis(0)&#xA;    | nd::to_shared()&#xA;    | compute_intercell_flux(0)&#xA;    | nd::to_shared()&#xA;    | nd::difference_on_axis(0)&#xA;    | nd::multiply(-dt * mara::make_area(1.0));&#xA;&#xA;    return solution_state_t {&#xA;        state.time + dt,&#xA;        state.iteration + 1,&#xA;        state.vertices,&#xA;        state.u + du | nd::to_shared() };&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31709" LastEditorUserId="31709" LastEditDate="2019-05-11T02:48:42.537" LastActivityDate="2019-05-11T02:48:42.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32624" PostTypeId="1" CreationDate="2019-05-10T19:12:22.693" Score="0" ViewCount="1005" Body="&lt;p&gt;I am trying to solve a non-convex optimization problem using &lt;code&gt;fmincon()&lt;/code&gt;.&#xA;At each iteration, I am iteratively looking for the optimum value and when the termination criterion is satisfied, I keep this value as an initial value for &lt;code&gt;fmincon()&lt;/code&gt; and run the code for the second iteration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is the code is running fine for the first to third iteration and then I get the following error  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;Error using sqpInterface Finite difference derivatives at initial&#xA;  point contain Inf or NaN values. Fmincon cannot continue.&lt;/code&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;Error in fmincon (line 823)&#xA;      [X,FVAL,EXITFLAG,OUTPUT,LAMBDA,GRAD,HESSIAN] = sqpInterface(funfcn,X,full(A),full(B),full(Aeq),full(Beq), .&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How can I get rid of this error?&lt;/p&gt;&#xA;" OwnerUserId="31712" LastEditorUserId="20688" LastEditDate="2019-05-10T19:44:42.300" LastActivityDate="2019-05-10T19:44:42.300" Title="Solving a non-convex optimization problem using fmincon" Tags="|matlab|constrained-optimization|iterative-method|nonconvex|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="32625" PostTypeId="1" AcceptedAnswerId="32626" CreationDate="2019-05-10T20:42:16.757" Score="2" ViewCount="202" Body="&lt;p&gt;I would like to solve the following equations simultaneously and numerically for all &lt;span class=&quot;math-container&quot;&gt;$X, Y, Z, W$&lt;/span&gt; where &lt;code&gt;i = 1:Nw, j = 1:Nl, k = 1:K&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W_\text{net1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W_\text{net2}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$m_\text{net1}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$m_\text{net2}$&lt;/span&gt; are given.&lt;/p&gt;&#xA;&lt;p&gt;How can I apply &lt;code&gt;solve()&lt;/code&gt; or any other function to numerically find the values of &lt;span class=&quot;math-container&quot;&gt;$X, Y, Z, W$&lt;/span&gt;?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yMQeX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yMQeX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the code for implementing the above equation in matlab.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-matlab prettyprint-override&quot;&gt;&lt;code&gt;Nw = 2; Nl=3; K=2;&#xA;ivec = 1 : Nw;&#xA;jvec = 1 : Nl;&#xA;W_net1 = randi([-123, 123], Nw, K);&#xA;W_net2 = randi([-123, 123], Nl, K); &#xA;m_net1 = randi([-123, 123], Nw, K); &#xA;m_net2 = randi([-123, 123], Nl, K); &#xA;&#xA;for k = 1 : K&#xA;    for i = ivec&#xA;        X(i, k) = 2*(1 - 2*W(i,k))/((1 - 2*W(i,k))*(1 + W_net1(i,k)) + W(i,k)*W_net1(i,k)*(1 - (2*W(i,k))^m_net1(i,k)));&#xA;        ii = setdiff(ivec, i);&#xA;        tW1 = prod( 1 - X(ii, k) );&#xA;        tW2 = prod( 1 - Y(jvec, k) );&#xA;        W(i,k) = 1 - tW1 * tW2;&#xA;    end&#xA;    for j = jvec&#xA;        Y(j, k) = 2*(1 - 2*Z(j,k))/((1 - 2*Z(j,k))*(1 + W_net2(j,k)) + Z(j,k)*W_net2(j,k)*(1 - (2*Z(j,k))^m_net2(j,k)));&#xA;        i = ivec;&#xA;        tZ1 = prod(1 - X(i, k));&#xA;        jj = setdiff(jvec, j);&#xA;        tZ2 = prod(1 - Y(jj, k));&#xA;        tZ3 = tZ2 * tZ1;&#xA;        Z(j,k) = 1 - tZ3;&#xA;        S(j,k) = Y(j, k) * tZ3;&#xA;    end&#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="31712" LastEditorUserId="20688" LastEditDate="2021-12-24T18:29:56.887" LastActivityDate="2021-12-24T18:29:56.887" Title="Solving $n$ coupled equations numerically in Matlab" Tags="|matlab|nonlinear-equations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32626" PostTypeId="2" ParentId="32625" CreationDate="2019-05-11T00:04:13.583" Score="3" Body="&lt;p&gt;One way of approaching the problem you posted is to use &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/fsolve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;fsolve&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can start by creating a file with a function that returns the difference &lt;span class=&quot;math-container&quot;&gt;$dX$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dY$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$dW$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dZ$&lt;/span&gt; between an estimate of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Z$&lt;/span&gt; and the respective expressions for these variables, &lt;span class=&quot;math-container&quot;&gt;$f_X$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f_Y$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$f_W$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f_Z$&lt;/span&gt; that you presented in your post:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;dX(i,k) &amp;amp;= f_X(X,Y,W,Z) - X(i,k) \\&#xA;dY(j,k) &amp;amp;= f_Y(X,Y,W,Z) - Y(j,k) \\&#xA;dW(i,k) &amp;amp;= f_W(X,Y,W,Z) - W(i,k) \\&#xA;dZ(j,k) &amp;amp;= f_Z(X,Y,W,Z) - Z(j,k)&#xA;\end{align}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in order to call this function it's easier if you map the variables into a matrix &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; of size &lt;span class=&quot;math-container&quot;&gt;$Nw + Nl + Nw + Nl$&lt;/span&gt; by &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt;. This way, you can use the matrix &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; as the &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/matrix-arguments.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;function argument&lt;/a&gt;. The &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/passing-extra-parameters.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;parameters can be passed to the function&lt;/a&gt; by using function handles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After creating the file with the function, your main program should define an initial guess (can it be all zeros, for instance?) and call &lt;a href=&quot;https://www.mathworks.com/help/optim/ug/fsolve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;fsolve&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to define the function, you'll need to create a file called &lt;code&gt;fun_dif.m&lt;/code&gt; with the following content:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function dU = fun_dif(U, Nw, Nl, K, W_net1, W_net2, m_net1, m_net2)&#xA;&#xA;ivec = 1 : Nw;&#xA;jvec = 1 : Nl;&#xA;&#xA;X(ivec, 1:K) = U(ivec, 1:K);&#xA;Y(jvec, 1:K) = U(Nw + jvec, 1:K);&#xA;W(ivec, 1:K) = U(Nl + Nw + ivec, 1:K);&#xA;Z(jvec, 1:K) = U(Nw + Nl + Nw + jvec, 1:K);&#xA;dU = zeros(Nw + Nl + Nw + Nl, K); % initialize the difference vector&#xA;&#xA;for k = 1 : K&#xA;    for i = ivec&#xA;        dU(i, k) = 2*(1 - 2*W(i,k))/((1 - 2*W(i,k))*(1 + W_net1(i,k)) + W(i,k)*W_net1(i,k)*(1 - (2*W(i,k))^m_net1(i,k))) - X(i, k);&#xA;        ii = setdiff(ivec, i);&#xA;        tW1 = prod( 1 - X(ii, k) );&#xA;        tW2 = prod( 1 - Y(jvec, k) );&#xA;        dU(Nl + Nw + i, k) = 1 - tW1 * tW2 - W(i,k);&#xA;    end&#xA;    for j = jvec&#xA;        dU(Nw + j, k) = 2*(1 - 2*Z(j,k))/((1 - 2*Z(j,k))*(1 + W_net2(j,k)) + Z(j,k)*W_net2(j,k)*(1 - (2*Z(j,k))^m_net2(j,k))) -  Y(j, k);&#xA;        i = ivec;&#xA;        tZ1 = prod(1 - X(i, k));&#xA;        jj = setdiff(jvec, j);&#xA;        tZ2 = prod(1 - Y(jj, k));&#xA;        tZ3 = tZ2 * tZ1;&#xA;        dU(Nw + Nl + Nw + j, k) = 1 - tZ3 -  Z(j,k);&#xA;        S(j,k) = Y(j, k) * tZ3;&#xA;    end&#xA;end &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that it's mostly your code with the computation of the difference &lt;span class=&quot;math-container&quot;&gt;$dU$&lt;/span&gt; between the estimate from the formulas and the current value &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, in your main program you can have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Nw = 2;&#xA;Nl=3;&#xA;K=2;&#xA;W_net1 = randi([-123, 123], Nw, K);&#xA;W_net2 = randi([-123, 123], Nl, K); &#xA;m_net1 = randi([-123, 123], Nw, K); &#xA;m_net2 = randi([-123, 123], Nl, K); &#xA;&#xA;f = @(U)fun_dif(U, Nw, Nl, K, W_net1, W_net2, m_net1, m_net2); % create a function handle&#xA;U0 = 0.5 + zeros(Nw + Nl + Nw + Nl, K); % initial estimate is 0.5 for all matrices&#xA;UU = fsolve(f,U0); % UU will have the final solution&#xA;X(ivec, 1:K) = UU(ivec, 1:K);&#xA;Y(jvec, 1:K) = UU(Nw + jvec, 1:K);&#xA;W(ivec, 1:K) = UU(Nl + Nw + ivec, 1:K);&#xA;Z(jvec, 1:K) = UU(Nw + Nl + Nw + jvec, 1:K);&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be worthy to note that&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ Z(j,k) = 1 - \frac{S(j,k)}{Y(j,k)}$$&lt;/span&gt;&#xA;and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ W(i,k) = 1 -\frac{S(j,k) (1 - Y(j,k))}{Y(j,k) (1 - X(i,k))} = 1 - \bigg(1 - Z(j,k) \bigg) \frac{1 - Y(j,k)}{1 - X(i,k)}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="30463" LastEditorUserId="30463" LastEditDate="2019-05-18T00:58:45.810" LastActivityDate="2019-05-18T00:58:45.810" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="32627" PostTypeId="1" AcceptedAnswerId="32643" CreationDate="2019-05-11T13:22:46.060" Score="1" ViewCount="573" Body="&lt;p&gt;I am trying to find the solution of linear advection equation of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\frac {\partial c}{\partial t}+u\frac {\partial c}{\partial x}=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c(x,0)=0$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c(0,t)=\{c_0 \ \text{for}\ t \leq t_1 \text{and}\ 0 \ \text{for}\ t&amp;gt;t_1 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody help me?&lt;/p&gt;&#xA;" OwnerUserId="31718" LastActivityDate="2019-05-14T09:13:40.837" Title="How to calculate the analytical solution of linear advection equation with Dirichlet's boundary conditions?" Tags="|advection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32630" PostTypeId="1" CreationDate="2019-05-11T12:22:19.397" Score="3" ViewCount="226" Body="&lt;p&gt;Let's assume, that we have damped harmonic oscillation of a body in the form of a cone, immersed in a liquid. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/qjp65.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qjp65.png&quot; alt=&quot;the cone, immersed in a liquid&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Equilibrium condition of the body is:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$m\overrightarrow{a} =  \overrightarrow{F_\text{res}} + m\overrightarrow{g} + \overrightarrow{F_{a}}$$&lt;/span&gt;&#xA;We can describe oscillations with the following differential equation.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{ d^{2}x }{dt^{2}} =   -g +  \frac{ \pi  tg^{2} \alpha  ( h_{0}-x)^{2}}{m}  \left( \frac{\rho_\text{liquid}g}{3}  (h_{0}-x) -  \frac{f}{\sin \alpha}  \frac{dx}{dt}\right)$$&lt;/span&gt;&#xA;Here is a chart which shows the dependence of displacement and speed from time (i.e. damped oscillation)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bNcDn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bNcDn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;My goal is to change the density of the fluid (&lt;span class=&quot;math-container&quot;&gt;$\rho_\text{liquid}$&lt;/span&gt;) and the resistance coefficient(&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;), find their values, starting from which the oscillations stop (i.e. the amplitude reach some small threshold value) after a specified time (e.g. &lt;span class=&quot;math-container&quot;&gt;$t=6$&lt;/span&gt;). At the current moment I solved this task in a very ineficient way with pyhton code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def dec_range(start, end, step):&#xA;         while start &amp;lt;= end:&#xA;         yield start&#xA;         start += step&#xA;&#xA;    for f_v in dec_range(15, 100.0, 0.05):&#xA;        vt, vx, vv = oiler(f = f_v)&#xA;&#xA;        index = 0&#xA;        for i, time in enumerate(vt):&#xA;            if abs(vt[index] - t) &amp;gt; abs(vt[i] - t):&#xA;                index = i&#xA;&#xA;        max_before=max(map(abs, vx[index-15:index]))&#xA;        max_after=max(map(abs, vx[index:index+15]))&#xA;&#xA;        vx_in_point = vx[index]&#xA;        vv_in_point = vv[index]&#xA;&#xA;        if (abs(max_before - vx_in_point) &amp;gt; epsilon) and (abs(max_after - vx_in_point) &amp;lt; epsilon) and (abs(vv_in_point) &amp;lt; epsilon_for_v) :&#xA;            with open(FILE_NAME, 'w') as f:&#xA;                for t, x, v in list(zip(vt, vx, vv)):&#xA;                    f.write(&quot;%0.10f %0.20f %0.20f\n&quot; % (t, x, v))&#xA;            print('koefficient = ', f_v)&#xA;            break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I use the iterative algorithm and change resistance coefficient (&lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt;) with a small step and use the Euler method to solve DE on every iteration. &#xA;What I am trying to achieve is to find an efficient algorithm and find both: &lt;span class=&quot;math-container&quot;&gt;$\rho_\text{liquid}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$f$&lt;/span&gt; &lt;/p&gt;&#xA;" OwnerUserId="31722" OwnerDisplayName="Andrii Ivaniuk" LastEditorUserId="20688" LastEditDate="2019-05-12T01:21:35.657" LastActivityDate="2019-07-17T00:02:35.577" Title="Damped Harmonic Oscillation. Efficient algorithm to find the parameters resulting in threshold oscillation amplitude" Tags="|finite-difference|differential-equations|oscillations|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="32631" PostTypeId="2" ParentId="28678" CreationDate="2019-05-12T07:35:17.007" Score="1" Body="&lt;p&gt;Some improvement to the conservative approach that is proposed in the question. I am not sure if you already implicitly took this into account while describing direct filtering of rows using Hölder's inequality.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Sorry for the change of notations (&lt;span class=&quot;math-container&quot;&gt;$U\mapsto A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V\mapsto B$&lt;/span&gt;) for the low-rank representation, but it makes sense to use &lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V$&lt;/span&gt; for the other things in the following derivations. They would make use of the low-rank property and the low-rank representation of the original matrix, in particular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a matrix &lt;span class=&quot;math-container&quot;&gt;$X=AB^T$&lt;/span&gt; (represented by a low-rank factorization), one can compute its SVD (economic form) in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O\left((n+m)r^2+r^3\right)$&lt;/span&gt; operations. Here, &lt;span class=&quot;math-container&quot;&gt;$X\in\mathbb F^{m\times n}$&lt;/span&gt;, of rank &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compute QR decompositions of &lt;span class=&quot;math-container&quot;&gt;$A=Q_AR_A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B=Q_BR_B$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(mr^2)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(nr^2)$&lt;/span&gt; operations, respectively.&lt;/li&gt;&#xA;&lt;li&gt;Compute matrix &lt;span class=&quot;math-container&quot;&gt;$P=R_AR_B^T$&lt;/span&gt; and its SVD &lt;span class=&quot;math-container&quot;&gt;$P=U_P\Sigma V_P^T$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(r^3)$&lt;/span&gt; operations since &lt;span class=&quot;math-container&quot;&gt;$P,R_A,R_B\in \mathbb F^{r\times r}$&lt;/span&gt;. Notice, that the diagonal matrix of singular values &lt;span class=&quot;math-container&quot;&gt;$\Sigma$&lt;/span&gt; does not only correspond to &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;, but to the original matrix &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;, as well.&lt;/li&gt;&#xA;&lt;li&gt;Compute matrices &lt;span class=&quot;math-container&quot;&gt;$U=Q_AU_P$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V=Q_BV_P$&lt;/span&gt; in &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(mr^2)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\mathcal O(nr^2)$&lt;/span&gt; operations, respectively.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, we got an SVD decomposition of the original matrix &lt;span class=&quot;math-container&quot;&gt;$X=U\Sigma V^T$&lt;/span&gt; since&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;X=AB^T=Q_AR_A(Q_BR_B)^T=\underbrace{Q_AR_A}_A\underbrace{R_B^TQ_B^T}_{B^T}=\underbrace{Q_AU_P}_{U}\Sigma \underbrace{V_PQ_B^T}_{V^T}=U\Sigma V^T&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, we can use Hölder's inequality on&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; (original proposal in the question)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$U\Sigma$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$V^T$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$U$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\Sigma V^T$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$U\sqrt{\Sigma}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sqrt{\Sigma}V^T$&lt;/span&gt; (which is the most balanced)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This will allow to improve the estimates and filter our more things, especially if &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are imbalanced. Technically, &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; different checks can be made without ruining the required complexity; however, I don't think that would make much impact in practice.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-12T07:35:17.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32632" PostTypeId="2" ParentId="31231" CreationDate="2019-05-12T08:47:09.033" Score="1" Body="&lt;p&gt;PETSc even &lt;a href=&quot;https://www.mcs.anl.gov/petsc/documentation/codemanagement.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommends TotalView&lt;/a&gt; as a parallel debugger, so you should be fine using it. However, you must ensure you are using a version of the library with debug symbols in it:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Most code development for PETSc codes should be done on one processor; hence, using a standard debugger such as &lt;code&gt;dbx&lt;/code&gt;, &lt;code&gt;gdb&lt;/code&gt;, &lt;code&gt;xdbx&lt;/code&gt;, etc. is fine. For debugging parallel runs we suggest Totalview if it is available on your machine. Otherwise, you can run each process in a separate debugger; this is not the same as using a parallel debugger, but in most cases it is not so bad. The PETSc run-time options &lt;code&gt;-start_in_debugger [-display xdisplay:0]&lt;/code&gt; will open separate windows and debuggers for each process. You should debug using the debugging versions of the libraries (run &lt;code&gt;./configure&lt;/code&gt; with the additional option &lt;code&gt;--with-debugging&lt;/code&gt; (the default)).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Even though TotalView is a very nice parallel debugger, more common serial debuggers might serve you better. Especially since the debugging is commonly done on relatively small examples. So, I would use &lt;code&gt;gdb&lt;/code&gt; for most of my debugging, unless I am interested in the debugging of parallel features (say some MPI communications I coded up myself).&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-12T08:47:09.033" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32633" PostTypeId="2" ParentId="11460" CreationDate="2019-05-12T09:11:41.570" Score="1" Body="&lt;p&gt;In addition to the suggested in the comments &lt;a href=&quot;https://www.vpython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VPython&lt;/a&gt;, I would look into &lt;a href=&quot;http://gmsh.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMSH&lt;/a&gt;, which now offers &lt;a href=&quot;http://gmsh.info/doc/texinfo/gmsh.html#Gmsh-API&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python API&lt;/a&gt;. It will allow you to create the model relatively easily (offering some primitives to build from), provide cross-platform visualization, and even subsequent meshing (if that is of any interest to you).&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-12T09:11:41.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32634" PostTypeId="2" ParentId="11284" CreationDate="2019-05-12T09:33:01.500" Score="1" Body="&lt;p&gt;PETSc does not immediately create your sparse matrix representation. So, when you call &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetValues.html#MatSetValues&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatSetValues()&lt;/code&gt;&lt;/a&gt;, you, usually, place the values in the intermediate cache which will occasionally get &quot;dumped&quot; into the real sparse matrix storage. So, the impact of the order in which you insert matrix elements is limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that the cache is used, allows multiple optimizations during the actual creation of the matrix and more flexibility in the assembly process (which you take advantage of). That's why you would call &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatAssemblyBegin.html#MatAssemblyBegin&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatAssemblyBegin()&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatAssemblyEnd.html#MatAssemblyEnd&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatAssemblyEnd()&lt;/code&gt;&lt;/a&gt; before you are able to use your matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, it is natural to create a matrix the way you describe in PETSc. When in doubt, see the &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSeqAIJSetPreallocation.html#MatSeqAIJSetPreallocation&quot; rel=&quot;nofollow noreferrer&quot;&gt;relevant PETSc examples&lt;/a&gt; which usually show the best practices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also have an option to create three-array CSR yourself and use &lt;a href=&quot;https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateSeqAIJWithArrays.html#MatCreateSeqAIJWithArrays&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;MatCreateSeqAIJWithArrays()&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-12T09:33:01.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32635" PostTypeId="1" AcceptedAnswerId="32636" CreationDate="2019-05-12T11:09:44.513" Score="1" ViewCount="841" Body="&lt;p&gt;Please consider the assignment I have uploaded on the picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y9IFU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y9IFU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused about the functions &lt;span class=&quot;math-container&quot;&gt;$g_L(t)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g_R(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\eta(x)$&lt;/span&gt;. What are they and how do I find them... My question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is it possbile to find out how &lt;span class=&quot;math-container&quot;&gt;$g_L(t)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g_R(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\eta(x)$&lt;/span&gt; look like purely based on this formulation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to solve this problem I need to know what the functions look like but unfortunately I don't how to determine the functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;strong&gt;EDIT:&lt;/strong&gt; If those are not given; How can I find &lt;em&gt;suitable&lt;/em&gt; function so I can make an numeric implementation?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the book &lt;strong&gt;Finite Difference Methods for Ordinary and Partial Differential equations&lt;/strong&gt; by &lt;em&gt;LeVeque&lt;/em&gt;. In chapter 9 it is presented how to solve the Heat equation but all the methods are based on knowing &lt;span class=&quot;math-container&quot;&gt;$g_L(t)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$g_R(t)$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\eta(x)$&lt;/span&gt; beforehand, which means I have overlooked some simple theory about Dirichlet boundary conditions?.    &lt;/p&gt;&#xA;" OwnerUserId="30032" LastEditorUserId="30032" LastEditDate="2019-05-12T12:28:55.887" LastActivityDate="2019-05-12T12:49:49.170" Title="Dirichlet boundary conditions in the 1D Heat Equation" Tags="|finite-difference|boundary-conditions|parabolic-pde|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32636" PostTypeId="2" ParentId="32635" CreationDate="2019-05-12T11:35:31.607" Score="3" Body="&lt;p&gt;No, you did not overlook anything. You actually have to know specific&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$g_L(t)$&lt;/span&gt; - boundary condition at the left boundary (&lt;span class=&quot;math-container&quot;&gt;$x=-1$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$g_R(t)$&lt;/span&gt; - boundary condition at the right boundary (&lt;span class=&quot;math-container&quot;&gt;$x=1$&lt;/span&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\eta(x)$&lt;/span&gt; - initial condition at &lt;span class=&quot;math-container&quot;&gt;$t=0$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, depending on what those functions are, you will get different solutions. The assignment is just given in general terms. When you would be asked to do a numerical implementation, you will be either given those boundary conditions, or you will come up with your own selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One simple choice would be to keep outer boundaries at a constant temperature:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;g_L(t)=g_R(t) = T&#xA;$$&lt;/span&gt;&#xA;and make the initial temperature at all the interior of the problem &lt;span class=&quot;math-container&quot;&gt;$T_0\neq T$&lt;/span&gt;, except the boundaries (to not contradict the boundary conditions above):&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\eta(x) =&#xA;\begin{cases}&#xA;T_0,&amp;amp;x\in(-1,1),\\&#xA;T, &amp;amp;x\in \{-1,1\}&#xA;\end{cases}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you start with two &quot;walls&quot; kept at a constant temperature of &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;, and with time, your domain should reach the equilibrium state from &lt;span class=&quot;math-container&quot;&gt;$T_0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="20688" LastEditorUserId="20688" LastEditDate="2019-05-12T12:49:49.170" LastActivityDate="2019-05-12T12:49:49.170" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32637" PostTypeId="1" AcceptedAnswerId="33154" CreationDate="2019-05-12T16:39:42.643" Score="2" ViewCount="277" Body="&lt;p&gt;I'm creating too much throw away code for interfacing with the scipy optimize package in a user friendly way.  (See code below for example of interruptible optimization that keeps last optimization value after a KeyBoard interrupt)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def monitoring_callback(x):&#xA;    global callback_dict&#xA;    callback_dict['cached_results'] = x&#xA;    callback_dict['counter']  = callback_dict['counter']  +1&#xA;&#xA;try:&#xA;    fit_results= optimize.minimize(get_mean_squared_error,&#xA;                                   my_fit_params,method=&quot;CG&quot;, &#xA;                                   callback=monitoring_callback)&#xA;    my_fit_params = fit_results&#xA;&#xA;except KeyboardInterrupt:&#xA;    my_fit_params = callback_dict['cached_results']&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there an existing package that does this sort of thing?&#xA;(Also, if it implemented graphics like here, &#xA;&lt;a href=&quot;http://louistiao.me/notes/visualizing-and-animating-optimization-algorithms-with-matplotlib/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://louistiao.me/notes/visualizing-and-animating-optimization-algorithms-with-matplotlib/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that would be really useful.)&lt;/p&gt;&#xA;" OwnerUserId="28443" LastActivityDate="2020-12-01T14:15:07.557" Title="User friendly scipy optimize wrapper package?" Tags="|optimization|scipy|visualization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32638" PostTypeId="2" ParentId="20931" CreationDate="2019-05-12T22:20:33.893" Score="0" Body="&lt;p&gt;The problem turns into an instance of Supermodular minimization if the constants &lt;span class=&quot;math-container&quot;&gt;$a_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a_{ij}$&lt;/span&gt; are positive. It still remains NP-hard but there are guarantees for the greedy approach based solution.&lt;/p&gt;&#xA;" OwnerUserId="21919" LastActivityDate="2019-05-12T22:20:33.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32640" PostTypeId="1" AcceptedAnswerId="32648" CreationDate="2019-05-14T04:03:55.643" Score="3" ViewCount="140" Body="&lt;p&gt;I have proposed an algorithm for a scheduling problem in a submitting paper. In the revision, the reviewer asked us to compare with another algorithm from the literature. Our algorithm is in MATLAB, and the comparing one is in C++, and the code is publicly available. We did not re-implement the C++ code, to avoid any decrease in the efficiency of their algorithm, and to save time as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the reviewer is responding: ``It is probable that there is a significant difference in performance between MATLAB and C++. The authors should make it clear if and how the results were normalized to ensure a fair comparison.''&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is this: Is there any (scientific) ratio or similar comparison between the efficiency of MATLAB and C++?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we opted to use the available code, we thought it is completely OK since MATLAB is known to be slower. So using the comparing algorithm in a faster environment is OK. I should add that our algorithm is now performing much better than the comparing one.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;PS: I asked the same question in Computer Science but some guy referred me to this community.&lt;/p&gt;&#xA;" OwnerUserId="31732" LastEditorUserId="31732" LastEditDate="2020-05-19T04:58:18.393" LastActivityDate="2020-05-19T04:58:18.393" Title="How to justify using available code (in different language) for comparing algorithms" Tags="|matlab|algorithms|c++|efficiency|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="32641" PostTypeId="1" AcceptedAnswerId="32651" CreationDate="2019-05-14T05:46:05.373" Score="2" ViewCount="110" Body="&lt;p&gt;&lt;strong&gt;Background：&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an equation which looks like as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W \times P = R$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\left[\begin{array} &amp;amp;{1}&amp;amp;{0}&amp;amp;{0}&amp;amp;-\frac{w_{1}}{w_{o1}} &amp;amp;\dots &amp;amp;{0} &amp;amp;-\frac{w_{1}}{w_{0} }	\\{0}&amp;amp;{-\frac{w_{2}}{w_{o2}}}&amp;amp;{1}&amp;amp;{0}&amp;amp;\dots &amp;amp;{0} &amp;amp;-\frac{w_{2}}{w_{o2}} 	\\ &amp;amp;&amp;amp;&amp;amp;&amp;amp; {\vdots} 	\\  {1} &amp;amp; {c-1} &amp;amp; {0} &amp;amp; {0} &amp;amp;{\cdots} &amp;amp; {0} &amp;amp; {0} 	\\ {0} &amp;amp; {0} &amp;amp; {1} &amp;amp; {c-1} &amp;amp; {\cdots} &amp;amp; {0} &amp;amp;{0} 	\\ &amp;amp;&amp;amp;&amp;amp;&amp;amp;{\vdots}\end{array}\right] 	\left[ \begin{array}{c}{p_{11}} \\ {p_{1o}} \\ {p_{22}} \\ {p_{2o}} \\ {\vdots} \\ {p_{c c}} \\ {p_{co}}\end{array}\right]	=	\left[ \begin{array}{l}{0} \\ {0} \\ {\vdots} \\ {0} \\ {1} \\ {\vdots} \\ {1} \end{array}\right]$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; denotes the number of classes and here, &lt;span class=&quot;math-container&quot;&gt;$c = 5$&lt;/span&gt;.&lt;br&gt;&#xA;The elements of &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; denote the probability of belonging to each class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &lt;span class=&quot;math-container&quot;&gt;$p_{11}$&lt;/span&gt; denotes the probability the &lt;code&gt;1st&lt;/code&gt; sample belongs to the first class. &#xA;&lt;span class=&quot;math-container&quot;&gt;$p_{1o}$&lt;/span&gt; denotes the probability the &lt;code&gt;1st&lt;/code&gt; sample belongs to the classes except for the first class.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;W = array([[  1.        ,   0.        ,   0.        ,  -1.76690464,&#xA;          0.        ,  -1.76690464,   0.        ,  -1.76690464,&#xA;          0.        ,  -1.76690464],&#xA;       [  0.        , -38.43501272,   1.        ,   0.        ,&#xA;          0.        , -38.43501272,   0.        , -38.43501272,&#xA;          0.        , -38.43501272],&#xA;       [  0.        , -41.64051053,   0.        , -41.64051053,&#xA;          1.        ,   0.        ,   0.        , -41.64051053,&#xA;          0.        , -41.64051053],&#xA;       [  0.        ,  -1.06855322,   0.        ,  -1.06855322,&#xA;          0.        ,  -1.06855322,   1.        ,   0.        ,&#xA;          0.        ,  -1.06855322],&#xA;       [  0.        ,  -2.86308364,   0.        ,  -2.86308364,&#xA;          0.        ,  -2.86308364,   0.        ,  -2.86308364,&#xA;          1.        ,   0.        ],&#xA;       [  1.        ,   4.        ,   0.        ,   0.        ,&#xA;          0.        ,   0.        ,   0.        ,   0.        ,&#xA;          0.        ,   0.        ],&#xA;       [  0.        ,   0.        ,   1.        ,   4.        ,&#xA;          0.        ,   0.        ,   0.        ,   0.        ,&#xA;          0.        ,   0.        ],&#xA;       [  0.        ,   0.        ,   0.        ,   0.        ,&#xA;          1.        ,   4.        ,   0.        ,   0.        ,&#xA;          0.        ,   0.        ],&#xA;       [  0.        ,   0.        ,   0.        ,   0.        ,&#xA;          0.        ,   0.        ,   1.        ,   4.        ,&#xA;          0.        ,   0.        ],&#xA;       [  0.        ,   0.        ,   0.        ,   0.        ,&#xA;          0.        ,   0.        ,   0.        ,   0.        ,&#xA;          1.        ,   4.        ]])&#xA;&#xA;R = array([0., 0., 0., 0., 0., 1., 1., 1., 1., 1.])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I try to solve this equation by using the following method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;P = np.linalg.solve(W, R)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P = array([ 1.29096548, -0.07274137, -1.82110745,  0.70527686, -1.80496726,&#xA;        0.70124182,  0.59473423,  0.10131644,  4.10879343, -0.77719836])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the result &lt;code&gt;P&lt;/code&gt; is not I expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you see, here the elements of &lt;code&gt;P&lt;/code&gt; denote the probability belonging to each class and they should vary from &lt;code&gt;0&lt;/code&gt; to &lt;code&gt;1&lt;/code&gt;. I don't know how to add these constraints to my equation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find that the condition number of &lt;code&gt;W&lt;/code&gt; is &lt;code&gt;537&lt;/code&gt;. Does it cause the problem?&lt;/p&gt;&#xA;" OwnerUserId="28947" LastEditorUserId="20688" LastEditDate="2019-05-15T02:15:51.073" LastActivityDate="2019-05-20T16:06:26.177" Title="How to add extra constraints to a linear system for probabilities？" Tags="|linear-algebra|python|constrained-optimization|probability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32642" PostTypeId="2" ParentId="24463" CreationDate="2019-05-14T08:38:00.637" Score="1" Body="&lt;p&gt;The cylindrical coordinate form of PDE equations are not available as default equations with the PDE Toolbox, but should be able to be entered as a custom PDE equation form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, both the Navier-Stokes equations and heat transfer (heat conduction with convection/advection) is also available for MATLAB as pre-defined (axisymmetric) PDE equations with the &lt;a href=&quot;https://www.featool.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;FEATool Multiphysics toolbox&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user29948" LastActivityDate="2019-05-14T08:38:00.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32643" PostTypeId="2" ParentId="32627" CreationDate="2019-05-14T09:13:40.837" Score="1" Body="&lt;p&gt;The transport equation simply moves your solution to the right with speed &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; (assuming &lt;span class=&quot;math-container&quot;&gt;$u &amp;gt; 0$&lt;/span&gt; and that &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is independent of &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Until time &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt;, you will move a constant solution equal to &lt;span class=&quot;math-container&quot;&gt;$c_0$&lt;/span&gt; into the domain at a speed &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;. The &quot;jump&quot; from &lt;span class=&quot;math-container&quot;&gt;$c_0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; (your initial value), originally located at &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt;, will thus travel rightward with speed &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; and is thus located at &lt;span class=&quot;math-container&quot;&gt;$x = u*t$&lt;/span&gt; (because it starts at &lt;span class=&quot;math-container&quot;&gt;$x=0$&lt;/span&gt;). So until time &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt;, your solution will be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;  c(x,t) = c_0&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for &lt;span class=&quot;math-container&quot;&gt;$0 \leq x \leq u t$&lt;/span&gt; and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;c(x,t) = 0 &#xA;\end{equation}&lt;/span&gt;&#xA;for &lt;span class=&quot;math-container&quot;&gt;$x &amp;gt; u t$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At time &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt;, your boundary condition changes and you now start to move a second jump from &lt;span class=&quot;math-container&quot;&gt;$c_0$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; into the domain, again with speed &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt;. Therefore, you now have a &quot;sawtooth&quot; shaped solution with the right edge at &lt;span class=&quot;math-container&quot;&gt;$x = u*t$&lt;/span&gt; and the left edge at &lt;span class=&quot;math-container&quot;&gt;$x = u(t - t_1)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, for &lt;span class=&quot;math-container&quot;&gt;$t &amp;gt; t_1$&lt;/span&gt;, your solution is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;  c(x,t) = c_0&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if &lt;span class=&quot;math-container&quot;&gt;$u(t - t_1) \leq x \leq u t$&lt;/span&gt; and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA; c(x,t) = 0 &#xA;\end{equation}&lt;/span&gt;&#xA;for all other values of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16921" LastActivityDate="2019-05-14T09:13:40.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32644" PostTypeId="1" CreationDate="2019-05-14T09:48:51.663" Score="1" ViewCount="81" Body="&lt;p&gt;I have proposed an optimisation algorithm which I &lt;strong&gt;claim has improved&lt;/strong&gt; upon the previous algorithm in a number of ways. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of these claims is that my proposed solution requires no &lt;strong&gt;explicit SVD and psuedoinverse calculation&lt;/strong&gt; which I claim is good, since in my case perturbations are important, and &lt;strong&gt;I believe these routines are traditionally unstable to perturbations&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is given the implementations in modern linear algebra computational libraries (such as LAPACK v3.0), how strong is this claim?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To summarise:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My method per iteration:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Only one solution of a generalized eigenvalue problem&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Old method per iteration:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One solution of a generalized eigenvalue problem, AND, &lt;/li&gt;&#xA;&lt;li&gt;One explicit SVD calculation for a Jacobian, &lt;span class=&quot;math-container&quot;&gt;$J = USV^{\intercal}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Two explicit calculations of a pseudoinverse based on the Jacobian, &lt;span class=&quot;math-container&quot;&gt;$J$&lt;/span&gt;. That is, (1) &lt;span class=&quot;math-container&quot;&gt;$J^{\dagger}$&lt;/span&gt; and (2)  &lt;span class=&quot;math-container&quot;&gt;$V^{\dagger}$&lt;/span&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Given that my algorithm will have input perturbations, and considering the stability of modern linear algebra routines, &lt;strong&gt;is my claim a strong selling point?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="31736" LastActivityDate="2019-05-14T09:48:51.663" Title="Stability of SVD, Eigendecompositions, and pseudoinverse procedures in modern LAPACK routines" Tags="|linear-algebra|linear-solver|stability|lapack|svd|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="32646" PostTypeId="1" CreationDate="2019-05-14T14:14:44.003" Score="1" ViewCount="522" Body="&lt;p&gt;I'm trying to find the capacitance matrix for a small array of metal boxes in air using Comsol 5.4. The geometry presented here is a simple geometry that recreates the problem I'm experiencing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Simulation Settings: What Works&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now I set my geometry length unit to &quot;um&quot;. To create my block array I first created a 40 x 40 x 20 unit block and then made a 3 x 3 array of these with 50 spacing in x and y. Meshing this with the default (Free Triangular) mesh gives me the following. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W68vR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W68vR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my simulation I am using the &quot;Electrostatics, Boundary Elements (esbe)&quot; physics. I have my simulation domain selection set to &quot;All voids&quot;.  I have my materials set to &quot;Air&quot; for &quot;All Voids&quot; and did not define a material for the blocks. I left all the default esbe boundary conditions (Charge Conversion, Zero Charge 1, Zero Charge 2, Initial Values). But then overrode nearly all of these by making all 6 boundaries of each block a voltage terminal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This simulation works. I'm able to run my stationary source sweep and extract the Maxwell capacitance matrix and it gives me numbers that look reasonable. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is Broken&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I switched the geometry length unit to &quot;nm&quot;. The resulting mesh looks identical to this one, but with &quot;nm&quot; as the unit instead of &quot;um&quot;. Everything else remains unchanged. I tried running this simulation and got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term1.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term2.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term3.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term4.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term5.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term6.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term7.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term8.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;There are 1 void equations (empty rows in matrix) for the variable comp1.esbe.term9.V0_ode.&#xA;at coordinates:  (0,0,0), ...&#xA;and similarly for the degrees of freedom (empty columns in matrix).&#xA;Returned solution is not converged.&#xA;Not all parameter steps returned.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have been able to manually extract the capacitance matrix from this geometry on the nm scale by manually setting terminal voltages and running a stationary simulation for each terminal. This is tedious, time consuming, and shouldn't be necessary. It is my understanding that a semi-manual terminal sweep should be possible by following the instructions &lt;a href=&quot;https://www.comsol.com/blogs/how-to-calculate-a-capacitance-matrix-in-comsol-multiphysics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but I have been unable to try this as &quot;PortName&quot; never shows up in my Parametric Sweep as a valid parameter (yes, I enabled &quot;Activate Manual Terminal Sweep&quot;). I'm using esbe instead of the es physics used in the tutorial. Maybe there is something else I have to do?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;What can I do to make the stationary source sweep simulation work properly at the nm scale? Or if that is impossible, the parametric terminal sweep.&lt;/p&gt;&#xA;" OwnerUserId="31740" LastEditorUserId="31740" LastEditDate="2019-05-14T14:25:16.980" LastActivityDate="2019-05-14T14:25:16.980" Title="Changing geometry scale breaks simulation" Tags="|comsol|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32647" PostTypeId="1" AcceptedAnswerId="32650" CreationDate="2019-05-14T14:52:37.733" Score="2" ViewCount="5680" Body="&lt;p&gt;I'm a beginner user of FEniCS and still struggling with some of the basics. Specifically,&lt;br&gt;&#xA;I have some issues doing the tutorials in the Langtangen-Logg book &lt;em&gt;Solving PDEs in Python - The FEniCS Tutorial Volume I&lt;/em&gt;. I think some of the problems are due to the different versions of the book and my installation (2019.1.0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the issues is the use of the &lt;code&gt;MeshFunction&lt;/code&gt; or &lt;code&gt;CellFunction&lt;/code&gt;, &lt;code&gt;FacetFunction&lt;/code&gt;, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example (4.3.2 on page 90) says:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;materials = CellFunction(’size_t’, mesh)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which doesn't work for me. After some research I found out that &lt;code&gt;CellFunction&lt;/code&gt; is deprecated since 2017.2.0:&lt;br&gt;&#xA;&lt;em&gt;Deprecate VertexFunction, EdgeFunction, FaceFunction, FacetFunction, CellFunction; use MeshFunction instead&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got the example to work with some trial and error (at least it looks like it's working):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;materials = MeshFunction('size_t', mesh, 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I do not really understand how and why. Especially after trying the next example that uses &lt;code&gt;FacetFunction&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://fenicsproject.org/docs/dolfin/2017.2.0/python/programmers-reference/cpp/mesh/MeshFunction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2017.2.0 doc&lt;/a&gt; doesn't say much about how to use &lt;code&gt;MeshFunction&lt;/code&gt;. The descripton for the &lt;code&gt;dim (unsigned int)&lt;/code&gt; argument says:&lt;br&gt;&#xA;&lt;em&gt;The topological dimension of the MeshFunction. Optional.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The description for &lt;code&gt;CellFunction&lt;/code&gt; is just:&lt;br&gt;&#xA;&lt;em&gt;Create MeshFunction of topological codimension 0 on given mesh.&lt;/em&gt;&lt;br&gt;&#xA;For &lt;code&gt;FacetFunction&lt;/code&gt;:&lt;br&gt;&#xA;&lt;em&gt;Create MeshFunction of topological codimension 1 on given mesh.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn't work with &lt;code&gt;materials = MeshFunction('size_t', mesh, 0)&lt;/code&gt;. I figured that the &lt;em&gt;codimension 0&lt;/em&gt; might mean to use &lt;code&gt;0&lt;/code&gt; for the &lt;code&gt;dim&lt;/code&gt; argument in the case of &quot;CellFunction&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, my questions are:  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is the &quot;codimension&quot; referring to the &lt;code&gt;dim&lt;/code&gt; argument?  &lt;/li&gt;&#xA;&lt;li&gt;What does &quot;topological dimension&quot; and &quot;topological codimension&quot; mean in this context?  &lt;/li&gt;&#xA;&lt;li&gt;If I should use &lt;code&gt;MeshFunction&lt;/code&gt; instead of all those other functions mentioned above, how does using MeshFunction for cells differ from using it for facets, etc.   &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;PS: I am aware of the question &lt;a href=&quot;https://scicomp.stackexchange.com/questions/7330/fenics-meshfunction-usage/7332&quot;&gt;Fenics: Meshfunction usage&lt;/a&gt;, but it doesn't really answer my question&lt;/p&gt;&#xA;" OwnerUserId="29423" LastEditorUserId="29423" LastEditDate="2019-05-14T15:51:26.710" LastActivityDate="2019-05-14T16:51:21.070" Title="How to use MeshFunction in FEniCS (dolfin)?" Tags="|pde|python|fenics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="32648" PostTypeId="2" ParentId="32640" CreationDate="2019-05-14T15:11:42.527" Score="7" Body="&lt;p&gt;So, you are comparing a &lt;em&gt;generally slower&lt;/em&gt; Matlab implementation of algorithm &lt;strong&gt;A&lt;/strong&gt; to a &lt;em&gt;generally faster&lt;/em&gt; C++ implementation of algorithm &lt;strong&gt;B&lt;/strong&gt;, and still getting the advantage for &lt;strong&gt;A&lt;/strong&gt;. I would say, congratulations, you certainly have a stronger point now, since the &quot;competing&quot; algorithm is given an advantage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth no note though, that implementation in C++ is not automatically guaranteed to be fast, since it can use bad design, wrong memory access patterns, and even ruin the theoretical complexity of the algorithm in the worst (no so rare, unfortunately) cases. Simultaneously, a Matlab implementation does not have to be slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, we have at least two questions that discuss a general case:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/2870/20688&quot;&gt;How to report timing?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://scicomp.stackexchange.com/q/10469/20688&quot;&gt;How to perform numerical experiments on a desktop?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Anyway, you have four options, in my mind:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do not change any code. Do not do any new numerical experiments. Explain the testing environment in detail and discuss the advantages of your algorithm.&lt;/li&gt;&#xA;&lt;li&gt;Do not change any code. Perform a sample &quot;renormalization experiment&quot; on the algorithm &lt;strong&gt;C&lt;/strong&gt;. Take an algorithm that is relevant to your field and you can find a C++ and Matlab implementations of that you expect to be of good quality. Compare the timings and deduce a very shaky, subjective, but relevant to your application area comparison metric. Use that metric only to inform the users about a possible scaling factor range for your Matlab code to even further strengthen your thoughts on the proposed algorithm advantage. Do not renormalize your results.&lt;/li&gt;&#xA;&lt;li&gt;Reimplement your algorithm &lt;strong&gt;A&lt;/strong&gt; in C++, then you will have a similar environment, and also will be comparing against some publicly-available reference code.&lt;/li&gt;&#xA;&lt;li&gt;Reimplement their algorithm &lt;strong&gt;B&lt;/strong&gt; in Matlab, then you will have a similar environment; however, you will be comparing two closed-source implementations (solvable) that are done by you.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the ideal world of infinite available time, I would opt into option 3, and tune up my C++ implementation. Option 4 is slightly worse but has a hidden advantage of comparing codes written by the same developer in a familiar language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your situation, I might be totally fine with option 1. Timing comparisons of algorithm implementation are not the best source of data, and usually, give only a rough idea about the algorithm strength. Therefore, the reviewer is probably asking for a detailed description of the test environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things to keep in mind:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Compare the theoretical asymptotic complexities of algorithm &lt;strong&gt;A&lt;/strong&gt; and algorithm &lt;strong&gt;B&lt;/strong&gt;. If they are the same, then you are competing in the constants and lower-order terms. In this case, the timing results are slightly more important. However, a more detailed theoretical analysis (for judging constants, lower-order terms, and relevant problem sizes) would go a much longer way.&lt;/li&gt;&#xA;&lt;li&gt;A clear description of the implementation details, test environments, and reproducibility of your experiments will help both to get the paper published and the readers to get the maximum benefits out of it.&lt;/li&gt;&#xA;&lt;li&gt;Consider making the source code of your algorithm &lt;strong&gt;A&lt;/strong&gt; open-source and refer to it in the paper.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-14T15:11:42.527" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32649" PostTypeId="2" ParentId="32647" CreationDate="2019-05-14T15:30:49.970" Score="1" Body="&lt;p&gt;If you're using the latest version of FEniCS, the meshing tools are deprecated as you've noticed. These tools are now under &lt;code&gt;mshr&lt;/code&gt; module.&#xA;You can install it via terminal: &lt;code&gt;conda install -c conda-forge mshr&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, if you'd like to convert meshes, the &lt;code&gt;dolfin-convert&lt;/code&gt; became deprecated as well. The functions are now bundled in &lt;code&gt;meshio&lt;/code&gt; module. Similarly: &lt;code&gt;conda install -c kayarre meshio&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usage in both cases should be the similar as described in older dolfin/FEniCS documentation.&lt;/p&gt;&#xA;" OwnerUserId="28318" LastActivityDate="2019-05-14T15:30:49.970" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="32650" PostTypeId="2" ParentId="32647" CreationDate="2019-05-14T16:51:21.070" Score="4" Body="&lt;p&gt;The different specialized &lt;code&gt;VertexFunction&lt;/code&gt;, &lt;code&gt;EdgeFunction&lt;/code&gt;, &lt;code&gt;FaceFunction&lt;/code&gt;, &lt;code&gt;FacetFunction&lt;/code&gt;, &lt;code&gt;CellFunction&lt;/code&gt; have indeed been deprecated and (at least) as of version 2018.2 removed. Instead, they have been merged into a general purpose &lt;code&gt;MeshFunction&lt;/code&gt; which takes the kind of mesh object they're defined on as a (mandatory) parameter called the &quot;topological dimension&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand what to put there, it may be helpful to stress the differences between &lt;em&gt;geometric&lt;/em&gt; objects (points, lines, polygons, polyhedra) and &lt;em&gt;topological&lt;/em&gt; objects (vertices, edges, facets, cells) -- while the former have the same dimension (basically by definition) no matter what the definition of the ambient space is (the &quot;n&quot; in &quot;nD&quot;), the latter do not, since they depend on the role they play in forming the mesh. These are instead distinguished by they &lt;em&gt;codimension&lt;/em&gt;, which is &quot;n minus geometric dimension&quot; (how far they are from &quot;full space&quot;):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;cells&lt;/strong&gt; have (by definition) &lt;strong&gt;codimension 0&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;facets&lt;/strong&gt; have (by definition) &lt;strong&gt;codimension 1&lt;/strong&gt; (boundaries of cells)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;edges&lt;/strong&gt; have (by definition) &lt;strong&gt;codimension n-1&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;vertices&lt;/strong&gt; have (by definition) &lt;strong&gt;codimension n&lt;/strong&gt; (boundaries of edges)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example, in 1D a line (geometric dimension 1) is a cell (codimension 0=1-1), while in 2D it's a facet (codimension 1=2-1)&#xA;(These can overlap: In 2D, edges and facets are the same; in 1D, facets and vertices are (and edges make no sense).) &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Back to FEniCS: As the above tried to illustrate, mesh objects are described by their &lt;em&gt;codimension&lt;/em&gt;; nevertheless, &lt;code&gt;MeshFunction&lt;/code&gt; takes the actual (geometric) dimension, which by simple arithmetic is &quot;n minus codimension&quot;. To write this in a dimension-independent way, you can make use of the function `mesh.topology().dim() So for example, to mark&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;cells&lt;/strong&gt;, use &lt;code&gt;MeshFunction('size_t',mesh,mesh.topology().dim())&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;facets&lt;/strong&gt;, use &lt;code&gt;MeshFunction('size_t',mesh,mesh.topology().dim()-1)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;edges&lt;/strong&gt;, use &lt;code&gt;MeshFunction('size_t',mesh,1)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;vertices&lt;/strong&gt;, use &lt;code&gt;MeshFunction('size_t',mesh,0)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;DISCLAIMER:&lt;/strong&gt; This is valid for versions &lt;strong&gt;2018.1&lt;/strong&gt; to &lt;strong&gt;2019.1&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1804" LastActivityDate="2019-05-14T16:51:21.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32651" PostTypeId="2" ParentId="32641" CreationDate="2019-05-14T18:46:16.553" Score="4" Body="&lt;p&gt;You could use a &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.lsq_linear.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;least-square solver with bounds&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.optimize import lsq_linear&#xA;&#xA;W = np.array([&#xA;        [1., 0., 0., -1.76690464, 0., -1.76690464, 0., -1.76690464, 0.,&#xA;         -1.76690464],&#xA;        [0., -38.43501272, 1., 0., 0., -38.43501272, 0., -38.43501272, 0.,&#xA;         -38.43501272], &#xA;        [0., -41.64051053, 0., -41.64051053, 1., 0., 0., -41.64051053, 0.,&#xA;         -41.64051053],&#xA;        [0., -1.06855322, 0., -1.06855322, 0., -1.06855322, 1., 0., 0.,&#xA;         -1.06855322],&#xA;        [0., -2.86308364, 0., -2.86308364, 0., -2.86308364, 0., -2.86308364,&#xA;         1., 0.],&#xA;        [1., 4., 0., 0., 0., 0., 0., 0., 0., 0.],&#xA;        [0., 0., 1., 4., 0., 0., 0., 0., 0., 0.],&#xA;        [0., 0., 0., 0., 1., 4., 0., 0., 0., 0.],&#xA;        [0., 0., 0., 0., 0., 0., 1., 4., 0., 0.],&#xA;        [0., 0., 0., 0., 0., 0., 0., 0., 1., 4.]])&#xA;R = np.array([0., 0., 0., 0., 0., 1., 1., 1., 1., 1.])&#xA;sol = lsq_linear(W, R, bounds=(0, 1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with solution&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;np.array([0.5448, 0., 0.9638, 0.0241, 0.9485, 0.0266, 0.5271, 0., 0.5726, 0.])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9667" LastActivityDate="2019-05-14T18:46:16.553" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="32653" PostTypeId="2" ParentId="7375" CreationDate="2019-05-14T21:17:22.620" Score="1" Body="&lt;p&gt;I would assume that the interfaces had changed and developed within the last 5 years. As of right now, the result can be achieved using the standard functionality of &lt;code&gt;scipy&lt;/code&gt; (as requested in the comments).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.mmread.html#scipy.io.mmread&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.io.mmread&lt;/a&gt; to load the matrix from MatrixMarket file into sparse or dense scipy matrix. I would assume that your matrix is originally sparse, therefore you are good to go.&lt;/li&gt;&#xA;&lt;li&gt;Now, &lt;a href=&quot;https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.sparse.linalg.splu.html#scipy.sparse.linalg.splu&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.sparse.linalg.splu&lt;/a&gt; can be used (or iLU if you opt for incomplete factorization).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Conversely, you can use&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.mmwrite.html#scipy.io.mmwrite&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.io.mmwrite&lt;/a&gt; to write out your matrices into the MatrixMarket files.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20688" LastActivityDate="2019-05-14T21:17:22.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32654" PostTypeId="1" AcceptedAnswerId="32658" CreationDate="2019-05-15T00:16:39.357" Score="3" ViewCount="128" Body="&lt;p&gt;Based on the Navier-Stokes equations and a few parameterizations, the horizontal steady-state wind &lt;span class=&quot;math-container&quot;&gt;$u(z)$&lt;/span&gt; within a forest of height H satisfies:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;a\left(\frac{du}{dz}\right)^2 + b\frac{du}{dz} \frac{d^2u}{dz^2} + cu +d\frac{du}{dz} + eu^2 + f= 0,\quad\text{for}\quad 0&amp;lt;z&amp;lt;H.&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coefficients &lt;span class=&quot;math-container&quot;&gt;$a,b,c,d,e,f$&lt;/span&gt; vary with the altitude &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and are given initially (we can differentiable and integrate them as many times as needed). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At ground level: &lt;span class=&quot;math-container&quot;&gt;$u(z = 0) = 0$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\frac{du}{dz}(z=0) = 0$&lt;/span&gt; (maybe it is not necessary to specify BC on the derivative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At canopy top: &lt;span class=&quot;math-container&quot;&gt;$u(z = H) = U_H$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to solve this equation for &lt;span class=&quot;math-container&quot;&gt;$u(z)$&lt;/span&gt; using a finite difference scheme, it would be great if someone could help me:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are finite differences even a good approach for this kind of problem?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I rewrite the equation using the classical expressions  &lt;span class=&quot;math-container&quot;&gt;$\frac{du}{dz} = \frac{u_{n+1}-u_{n-1}}{2h}, \frac{d^2u}{dz^2} =$&lt;/span&gt; etc... I obtain square terms like &lt;span class=&quot;math-container&quot;&gt;$u_{i+1}u_{i-1}$&lt;/span&gt; and I do not how what to do from there.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I do not know how to use the Newton method or the Picard method correctly, is there a better way to rewrite the equation? Using variables like &lt;span class=&quot;math-container&quot;&gt;$v = \frac{du}{dz}$&lt;/span&gt; for example?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;At that point, I am not even sure if I am missing something obvious or if this is a really hard problem, any help would be greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="31745" LastEditorUserId="31745" LastEditDate="2019-05-15T05:21:29.037" LastActivityDate="2019-05-15T16:57:44.197" Title="Finite difference for a highly nonlinear equation - The wind within the forest" Tags="|finite-difference|nonlinear-equations|differential-equations|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32656" PostTypeId="1" CreationDate="2019-05-15T11:03:37.837" Score="1" ViewCount="215" Body="&lt;p&gt;EDIT: I am trying to specify my Question. Also I am not going to clearify which spaces I use, because I am only interested in the basic idea. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking at a standard elliptic second order PDE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{cases} Lu &amp;amp; = f &amp;amp; \text{in}  &amp;amp;\Omega \\ \quad u &amp;amp; = g &amp;amp; \text{on} &amp;amp; \Gamma \subset \partial\Omega \end{cases}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\Omega \subset \mathbb{R}^n$&lt;/span&gt; is an open bounded Lipschitz domain. My goal is to find the normal derivative &lt;span class=&quot;math-container&quot;&gt;$\partial_n u$&lt;/span&gt; on the boundary &lt;span class=&quot;math-container&quot;&gt;$\Gamma$&lt;/span&gt; using the Dirichlet-to-Neumann (DtN) Operator in the Framework of FEM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know the DtN operator is as map&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\Lambda_\Gamma:g \to\partial_nu \quad with \quad u=0 \quad on\quad \partial\Omega\setminus\Gamma&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; is the solution of the boundary value problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;span class=&quot;math-container&quot;&gt;$Lu = - \Delta u$&lt;/span&gt; the weak form of the pde is given as: Find &lt;span class=&quot;math-container&quot;&gt;$u$&lt;/span&gt; such that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}&#xA;\int_\Omega \nabla u\nabla v \,\,d\Omega = \int_\Omega f v \,\, d\Omega \qquad\forall v&#xA;\end{equation}&lt;/span&gt;&#xA;with &lt;span class=&quot;math-container&quot;&gt;$u = g$&lt;/span&gt; on  &lt;span class=&quot;math-container&quot;&gt;$\Gamma$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how do I use the DtN operator to calculate the normal derivative in this case? &lt;/p&gt;&#xA;" OwnerDisplayName="user29088" LastEditorDisplayName="user29088" LastEditDate="2019-05-16T15:51:30.227" LastActivityDate="2019-05-17T13:47:52.777" Title="Dirichlet to Neumann Operator" Tags="|numerics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32657" PostTypeId="1" CreationDate="2019-05-15T14:24:55.820" Score="2" ViewCount="95" Body="&lt;p&gt;&lt;em&gt;This question &lt;a href=&quot;https://quant.stackexchange.com/q/45466&quot;&gt;is originally posted in Quant.StackExchange&lt;/a&gt; but has been unanswered for some time so I ask in here.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make this simple let us consider the Geometric Brownian Motions (GBM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;1. How can I show that the Euler-Maruyama Method is convergent using GBM?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;2. How can I determine the order of convergence?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background and the Problems I am facing:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Numerical Analysis theory &lt;strong&gt;An SDE solver has order &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; if the expected value of the error is of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;th order in the time step size&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now letting &lt;span class=&quot;math-container&quot;&gt;$S_t$&lt;/span&gt; be a GBM with &lt;span class=&quot;math-container&quot;&gt;$S_0=1,$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\mu=0.1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma=0.15$&lt;/span&gt; then &lt;span class=&quot;math-container&quot;&gt;$E[S_{10}] =e$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run the solver &lt;span class=&quot;math-container&quot;&gt;$10,000$&lt;/span&gt; times for different sizes &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt;, I would expect that the difference between my sample mean and true mean, &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;, will decrease as &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt; gets smaller. However, when I run these simulations this is what I get:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MWb2S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MWb2S.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;strong&gt;This does not indicate that the error converges to zero as &lt;span class=&quot;math-container&quot;&gt;$dt$&lt;/span&gt; goes to zero?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is that ..... Is this because of the randomness? I have actually tried for &lt;span class=&quot;math-container&quot;&gt;$1 \,000$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$10\,000$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$20\,000$&lt;/span&gt; simulations. In general, &lt;span class=&quot;math-container&quot;&gt;$10\,000$&lt;/span&gt; simulations should be enough to remove much of variance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please suggest how I can determine/show the order of convergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone is interested in the code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;&#xA;T = 10&#xA;mu = 0.1&#xA;sigma = 0.15&#xA;S0 = 1&#xA;ns = 10000&#xA;&#xA;&#xA;solution = S0*np.exp( mu*T ) &#xA;&#xA;&#xA;dt_ = np.array([0.1,0.05,0.01,0.005])&#xA;err = np.zeros( len(dt_ )); &#xA;for j in range ( len(dt_ )):&#xA;    dt = dt_[j]&#xA;    Sn = np.zeros( (ns) ) &#xA;    for i in range(ns):&#xA;        N = int(round(T/dt))&#xA;        t = np.linspace(0, T, N)&#xA;        ex= np.linspace(0, T, N)&#xA;        W = np.random.standard_normal(size = N) &#xA;        W = np.cumsum(W)*np.sqrt(dt) ### standard brownian motion ###&#xA;        X = (mu-0.5*sigma**2)*t + sigma*W &#xA;        S = S0*np.exp(X) ### geometric brownian motion ###&#xA;        Sn[i]= S[-1]&#xA;&#xA;    mn          = np.mean(Sn)&#xA;    print(mn)&#xA;    err[j]      = abs( mn - solution)&#xA;&#xA;plt.clf();&#xA;plt.loglog(dt_,err, color =&quot;black&quot;, label = &quot;Error (abs)&quot;);plt.xlabel(&quot;dt&quot;,fontsize = 20); plt.ylabel(&quot;Error (abs)&quot;,fontsize = 20)&#xA;plt.loglog(dt_,err, 'o', color =&quot;black&quot; , label = &quot;x&quot;)&#xA;plt.title(&quot;loglog-plot&quot;,fontsize = 30);&#xA;plt.loglog(dt_,0.05*dt_**0.5, linestyle = &quot;:&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code is mostly just copy-pasted from StackOverflow&lt;/p&gt;&#xA;" OwnerUserId="30032" LastEditorUserId="20688" LastEditDate="2019-05-16T00:28:24.930" LastActivityDate="2019-05-16T00:28:24.930" Title="How to determine the order of convergence of the Euler-Maruyama method?" Tags="|finite-difference|convergence|numerics|stochastic|stochastic-ode|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32658" PostTypeId="2" ParentId="32654" CreationDate="2019-05-15T16:06:57.437" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;I can't speak to the use of the FD approach here, but I don't see any reason it wouldn't work. &lt;/li&gt;&#xA;&lt;li&gt;Its fine to have those square terms and ugliness, it just indicates that your governing equations are nonlinear.&lt;/li&gt;&#xA;&lt;li&gt;One way to approach this to say you have a residual, R, that is expressed like so:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$R(u) = a\left(\frac{du}{dz}\right)^2 + b\frac{du}{dz}\frac{d^2u}{dz^2} + cu + d\frac{du}{dz} + eu^2+f$$&lt;/span&gt; And at steady state, R = 0. You could add a pseudo time derivative, &lt;span class=&quot;math-container&quot;&gt;$\frac{du}{dt}$&lt;/span&gt;, and discretize this term in pseudo time to march to the steady state answer. i.e., &lt;span class=&quot;math-container&quot;&gt;$$u^{n+1} = u^n + \Delta t R(u)$$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\Delta t$&lt;/span&gt; is constrained by the CFL condition. This corresponds to forward euler time stepping.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Should you want to apply the newton method, we begin with BDF1 and with &lt;span class=&quot;math-container&quot;&gt;$u_n$&lt;/span&gt;, for which &lt;span class=&quot;math-container&quot;&gt;$R(u_n)$&lt;/span&gt; does not equal zero, and a &lt;span class=&quot;math-container&quot;&gt;$R(u_{n+1})$&lt;/span&gt; which we want to be equal to zero. If we take the taylor series expansion of &lt;span class=&quot;math-container&quot;&gt;$R(u_{n+1})$&lt;/span&gt; about &lt;span class=&quot;math-container&quot;&gt;$u_n$&lt;/span&gt; we get:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$R(u_{n+1}) = R(u_n) + \frac{\partial R}{\partial u_n}\Delta u = 0$$&lt;/span&gt;&#xA;Where the derivative of the residual is the jacobian matrix. We can then bring the left term to the RHS and get:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\partial R}{\partial u_n}\Delta u = -R(u_n)$$&lt;/span&gt;&#xA;Now, by solving this linear system, we will not actually solve the nonlinear governing equations in one step, because a linear taylor series expansion is not sufficient to solve the nonlinear equation that quickly, but we can repeat this iteration many times to solve the steady s